[ARM] Add __bf16 as new Bfloat16 C Type

[stuij]

Summary

This patch upstreams support for a new storage only bfloat16 C type.
This type is used to implement primitive support for bfloat16 data, in
line with the Bfloat16 extension of the Armv8.6-a architecture, as
detailed here:

https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/posts/arm-architecture-developments-armv8-6-a

The bfloat type, and its properties are specified in the Arm Architecture
Reference Manual:

https://developer.arm.com/docs/ddi0487/latest/arm-architecture-reference-manual-armv8-for-armv8-a-architecture-profile

In detail this patch:

• introduces an opaque, storage-only C-type __bf16, which introduces a new bfloat IR type.

This is part of a patch series, starting with command-line and Bfloat16
assembly support. The subsequent patches will upstream intrinsics
support for BFloat16, followed by Matrix Multiplication and the
remaining Virtualization features of the armv8.6-a architecture.

The following people contributed to this patch:

• Luke Cheeseman
• Momchil Velikov
• Alexandros Lamprineas
• Luke Geeson
• Simon Tatham
• Ties Stuij

[craig.topper]

clang/lib/Basic/Targets/AArch64.cpp

74

Doesn't Bfloat16 have a different number of mantissa and exponent bits than IEEEhalf?

clang/test/CodeGen/arm-mangle-16bit-float.cpp

5

How can bfloat16 be passed as half? Don't they have a different format?

[rjmccall]

I don't understand why you wouldn't add a new IR type for this; doing so should be totally mechanical.

[rjmccall]

Note that we have an IR type for the PPC double-double format, which isn't even hardware-supported. This is literally just an IEEE floating-point format with non-standard parameters, right?

[LukeGeeson]

In D76077#1919897, @rjmccall wrote:

Note that we have an IR type for the PPC double-double format, which isn't even hardware-supported. This is literally just an IEEE floating-point format with non-standard parameters, right?

Indeed yeah, the details are laid out in a blog post here: https://community.arm.com/developer/ip-products/processors/b/ml-ip-blog/posts/bfloat16-processing-for-neural-networks-on-armv8_2d00_a

[LukeGeeson]

In D76077#1919861, @rjmccall wrote:

I don't understand why you wouldn't add a new IR type for this; doing so should be totally mechanical.

We had a few reasons for doing it this way.

By adding a new IR type we would need to consider calling conventions, and IR optimizations for what is essentially an opaque storage-only type.

Bfloat has no soft ABI, and all the supported operations can be handled through intrinsics (in a later patch, removed here due to bloat). If we were to add a new IR type, then we would need to handle many operations which would extend beyond what the type is supported for. For instance in GCC we had to add a new mode in RTL to handle inline memcopy.

[LukeGeeson]

clang/test/CodeGen/arm-mangle-16bit-float.cpp

5

see the above comment about soft-abis

[rjmccall]

In D76077#1921490, @LukeGeeson wrote:

In D76077#1919861, @rjmccall wrote:

I don't understand why you wouldn't add a new IR type for this; doing so should be totally mechanical.

We had a few reasons for doing it this way.

By adding a new IR type we would need to consider calling conventions, and IR optimizations for what is essentially an opaque storage-only type.

IR optimizations should just fall out; the code in APFloat should work for arbitrary FP semantics.

Calling something a "storage-only" type does not get you out of worrying about calling conventions at the ABI level. You can forbid passing your type as an argument or result directly, but structs containing your type as a field can still be passed around, and the behavior for that needs to be defined.

Bfloat has no soft ABI, and all the supported operations can be handled through intrinsics (in a later patch, removed here due to bloat). If we were to add a new IR type, then we would need to handle many operations which would extend beyond what the type is supported for. For instance in GCC we had to add a new mode in RTL to handle inline memcopy.

I don't know why having a soft ABI makes a difference. If the type only works on certain platforms, then it can't be used on other platforms.

[LukeGeeson]

Added a file from a downstream cleanup of the branch.

[jfb]

In D76077#1921973, @rjmccall wrote:

In D76077#1921490, @LukeGeeson wrote:

In D76077#1919861, @rjmccall wrote:

I don't understand why you wouldn't add a new IR type for this; doing so should be totally mechanical.

We had a few reasons for doing it this way.

By adding a new IR type we would need to consider calling conventions, and IR optimizations for what is essentially an opaque storage-only type.

IR optimizations should just fall out; the code in APFloat should work for arbitrary FP semantics.

Calling something a "storage-only" type does not get you out of worrying about calling conventions at the ABI level. You can forbid passing your type as an argument or result directly, but structs containing your type as a field can still be passed around, and the behavior for that needs to be defined.

Bfloat has no soft ABI, and all the supported operations can be handled through intrinsics (in a later patch, removed here due to bloat). If we were to add a new IR type, then we would need to handle many operations which would extend beyond what the type is supported for. For instance in GCC we had to add a new mode in RTL to handle inline memcopy.

I don't know why having a soft ABI makes a difference. If the type only works on certain platforms, then it can't be used on other platforms.

Having an IR type sounds like the right thing to do here.

[stuij]

Thanks for the feedback. We'll work on implementing a bfloat IR type.

[SjoerdMeijer]

That sounds good and that seems to address the feedback given here, which I agree with. Just wanted to briefly add that while it already looks like most interested people are on this ticket, perhaps it good to also share the plan and design on the cfe dev list for visibility.

[stuij]

commandeering as I've got some changes to push

[stuij]

This comment was removed by stuij.

[stuij]

changes: depend on bfloat IR type, and resolve some conflicts that crept in

[stuij]

I've amended this patch to conform to the general Bfloat -> BFloat convention
changes requested in the dependent BFloat IR patch (D78190).

I also folded in two other downstream patches, which allow for the ?: operator
on bf16 values and prohibit conversions between integer and __bf16.

[asmith]

really happy for native bfloat support!

clang/include/clang/AST/ASTContext.h

967

Maybe the comment is unnecessary. I can imagine other targets besides ARM NEON using BFloat

clang/include/clang/AST/Type.h

2011

don't think this comment adds anything

clang/lib/AST/ASTContext.cpp

5993

extra spaces

clang/lib/AST/ItaniumMangle.cpp

3188

Add a space before break and on the line above.

[stuij]

@asmith: Thanks for reviewing :) Fixed your suggestions in the coming revision.

clang/include/clang/AST/ASTContext.h

967

fair

clang/include/clang/AST/Type.h

2011

fair

[stuij]

changes:

• addressed asmith's comments
• Bfloat doesn't depend on half cmdline options and correct code is generation for AArch32 hard and softfp CC

[LukeGeeson]

I authored some of the code for this patch, please update the commit message with my name on the list :)

[majnemer]

clang/include/clang-c/Index.h

3258

Should this be:

CXType_LastBuiltin = CXType_BFloat16,

clang/lib/AST/ItaniumMangle.cpp

3188

Why is this x1?

[majnemer]

clang/lib/Sema/SemaOverload.cpp

1873–1874

This probably needs an explanation.

[SjoerdMeijer]

clang/docs/LanguageExtensions.rst

518

Not my field of expertise, and sorry if I've missed this somewhere, but was wondering if this (i.e. all language modes) means we need C++ name mangling support for bfloat? In the AArch64 backend I saw this:

getBFloat16Mangling() const override { return "u6__bf16"; }

But that's something else I guess, and I was guessing a 2 letter mangling needs to be added here?

https://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangling-builtin

[stuij]

@LukeGeeson: you're already mentioned :) See the commits tab on this review. But Simon Tatham needs a mention as well. I shall add him.

clang/docs/LanguageExtensions.rst

518

Yes, we need name-mangling support, and yes that method you mentioned does that. There's one for AArch32 as well. And yes we have documented it in the 'C++ ABI for the latest Arm Architecture' docs:

aarch32: https://developer.arm.com/docs/ihi0041/latest
aarch64: https://developer.arm.com/docs/ihi0059/latest

clang/include/clang-c/Index.h

3258

thanks

clang/lib/AST/ItaniumMangle.cpp

3188

Yes, that does look odd. The original author of this code has left the company, but I'll ask around.

clang/lib/Sema/SemaOverload.cpp

1873–1874

done

[stuij]

addressing review comments and adding Simon Tatham to contributers

[stuij]

redo: addressing review comments and adding Simon Tatham to contributers

[SjoerdMeijer]

clang/docs/LanguageExtensions.rst

518

But does that mean that for other targets this is not (yet) supported?

[stuij]

clang/docs/LanguageExtensions.rst

518

Currently bfloat is a vendor-extended type, hence the 'u' prefix. So to me it would make sense that backends should implement their own naming, as there's no guarantee that the naming scheme chosen for Arm is compatible with all backends.

[SjoerdMeijer]

clang/docs/LanguageExtensions.rst

518

Ok, true, fair enough

[SjoerdMeijer]

clang/include/clang/Basic/DiagnosticSemaKinds.td

8118

Nit: was wondering if err_cast_to_bfloat16 would be more consistent

clang/lib/AST/ASTContext.cpp

6008

nit: perhaps this error message is not entirely accurate for bfloat16?

clang/lib/Basic/Targets/AArch64.cpp

74

Nit: we use bfloat16 everywhere, would llvm::APFloat::BFloat16() be better for consistency?

clang/lib/CodeGen/CodeGenModule.cpp

115

nit: perhaps getBFloat16Ty()

clang/lib/CodeGen/CodeGenTypeCache.h

39

And so here too, BFloat16Ty?

clang/lib/CodeGen/TargetInfo.cpp

5982

Nit: to distinguish the unfortunate float ABI names, I was wondering whether IsFloatABISoftFP is clearer, or something along those lines, just to make clear it is not the "soft" but the "softfp" variant

[SjoerdMeijer]

There are Sema and CodeGen tests, but was wondering if there would be some value in having an AST test too? There are some other types that do have AST tests.

clang/lib/AST/ASTContext.cpp

2052

Is a break missing here?

clang/lib/CodeGen/CodeGenTypes.cpp

307

Is this covered in tests?

clang/lib/CodeGen/TargetInfo.cpp

6590

nit: camelcase fase for haslegalhalf

clang/test/CodeGen/arm-mangle-16bit-float.cpp

3

Do you need to pass +fullfp16?

5

Probably I am bit confused too now... are the three possible types that we are expecing not bfloat, i32, or i16?

[LukeGeeson]

clang/test/CodeGen/arm-mangle-16bit-float.cpp

5

Hi Sjoerd, Good spot here I forgot to add a default case somewhere, which means AArch32 didn't observe -mfloat-abi softfp by default - and hence used bfloat where i32 was expected.

I have a local patch for this and will send to Ties to merge into this one :)

We're not sure what you mean by i16 however?

[stuij]

no explicit float-abi cmdline arg should default to softfp

[stuij]

addressed review comments and some related general changes:

• renamed IsSoftFloatABI -> IsFloatABISoftFP
• split bfloat tests out of arm-mangle-16bit-float.cpp
• bfloat shouldn't be checking for NativeHalfType
• err_cast_(from|to)_bfloat -> err_cast_(from|to)_bfloat16
• reworded comments and other minor fixes

[stuij]

clang/include/clang/Basic/DiagnosticSemaKinds.td

8118

agree

clang/lib/AST/ASTContext.cpp

2052

tnx

6008

tnx

clang/lib/AST/ItaniumMangle.cpp

3188

I'm not sure it's 100% wrong, but it's not consistent with GCC. Changing to bfloat16_t.

clang/lib/Basic/Targets/AArch64.cpp

74

In the IR world bfloat is consistently called bfloat, without the 16. I think this might turn into bikeshedding to justify if either one or both sides of the divide should or should not include the '16'. I can think of arguments to support the various options.

As I think there's no clear winner, I'd like to take the route of less effort and keep things as is.

clang/lib/CodeGen/CodeGenModule.cpp

115

see previous C - IR divide comment

clang/lib/CodeGen/CodeGenTypeCache.h

39

see previous C - IR divide comment

clang/lib/CodeGen/CodeGenTypes.cpp

307

Thanks. That code doesn't make sense to me. We are not expecting to emulate bfloat operations, and we shouldn't conflate bfloat with half. I'll remove the UseNativeHalf clause, and the one call-site of this fn.

clang/lib/CodeGen/TargetInfo.cpp

5982

Yes, good idea. Done.

clang/test/CodeGen/arm-mangle-16bit-float.cpp

3

I'm splitting out bf16 and fp16 tests. I don't think we should be testing with the union of cmdline options to accommodate testing multiple types in one file.

[SjoerdMeijer]

As I wrote in one of the inlined comments, I am relatively unhappy that we have both bfloat and bfloat16 as names. But that looks like a complain for another patch, and not really this one.

clang/lib/Basic/Targets/AArch64.cpp

74

ah, I couldn't remember on which patch I commented on the bfloat16 vs bfloat naming.
Since everything is called bfloat16, from the architecture extension to the source language type, I find this a bit of a missed opportunity to get the names consistent. And I wouldn't say that naming of types is bikeshedding. But maybe that's just me...

clang/lib/Basic/Targets/ARM.cpp

80

Is this tested? Can it be tested?

clang/lib/CodeGen/CGBuiltin.cpp

4483

nit: the linter says some spaces here.

5514

nit: looks like the hinter right to be unhappy about the formatting

clang/lib/CodeGen/TargetInfo.cpp

6270

nit: ir -> IR

6471

nit: perhaps add TODO, or just remove this.

clang/lib/Sema/SemaOverload.cpp

1874

Nit: conversions -> Conversions, permitted -> permitted.

Same on line no. 1895.

clang/test/CodeGen/arm-bf16-params-returns.c

5

what happens with -mfloat-abi=soft. Does that deserve a test?

clang/test/CodeGen/arm-mangle-16bit-float.cpp

5

Ah, as I said, got confused. Now I see we use i16 for targets that don't support bfloat16. So we probably need a test for that.

clang/test/CodeGen/arm-mangle-bf16.cpp

3

nit: if the name mangling is Independent of the float ABI, then you might as well one runline and the abi option.

[SjoerdMeijer]

clang/test/CodeGen/arm-mangle-bf16.cpp

3

-> ... then you might as well have only one runline and remove the abi option.

[stuij]

clang/lib/Basic/Targets/AArch64.cpp

74

In general naming isn't bikeshedding. And I appreciate discussions on naming. I'm just afraid it might turn into bikeshedding. I do agree that such a remark isn't a very constructive to a conversation. Sorry.

clang/test/CodeGen/arm-bf16-params-returns.c

5

Yes, this one is interesting. I think we shouldn't support bfloat at all in combination with -mfloat-abi=soft. We don't support software emulation of bfloat instructions and all operations on bfloat are simd instructions.

It turns out cc1 will accept -mfloat-abi=soft with neon intrinsics, which will happily churn out neon instructions. This doesn't sound very soft. The driver will ignore -mfloat-abi=soft in certain combinations of cmdline instructions, but I haven't delved deep enough to know what's what.

GCC doesn't allow soft+neon combination. Unfortunately it will actually crash for just a bfloat type by itself, which is quite useless without intrinsics. The Arm GCC folks will raise a ticket on this with as proposed solution to not allow this combination.

As this issue seems bigger than just bfloat, and potentially there's driver code involved as well I thought it'd make sense to handle this in a separate patch.

clang/test/CodeGen/arm-mangle-16bit-float.cpp

5

As I wrote above, I think we currently should not allow it as long as there are no targets that support bfloat and have use for this combination.

clang/test/CodeGen/arm-mangle-bf16.cpp

3

yes, I thought it's better to be defensive here, in case the implementation changes.

[SjoerdMeijer]

clang/test/CodeGen/arm-bf16-params-returns.c

5

I think we first need agreement what -mfloat-abi=soft with bf16 means and how it should behave, document this, and have some tests. Possibly document how we diverge from this.

I think I tend to disagree with this:

I think we shouldn't support bfloat at all in combination with -mfloat-abi=soft.

why would it not be supported in some way (promotions to another type, or even library calls), like the other float-types?

It turns out cc1 will accept -mfloat-abi=soft with neon intrinsics, which will happily churn out neon instructions.

I would say the fact that there are other problems, shouldn't distract us too much from trying to get this right; I think at this point that is not yet a justification.

As this issue seems bigger than just bfloat, and potentially there's driver code involved as well I thought it'd make sense to handle this in a separate patch.

I think at this point I disagree with this, mainly because of my first point: the behaviour should be specified. I would also say that not doing this, could be a bit of bad precedent for adding a new C type.

[labrinea]

clang/lib/CodeGen/CGBuiltin.cpp

4493

This is not what we should be checking for here. Imagine a command line with +bf16 and +mfloat-abi=softfp, that should generate i16, not bfloat. We therefore need a target hook to pass this information. I suggest allowBFloatArgsAndRet() in ABIInfo, returning false by default, overloaded for Arm and AArch64 to return !IsFloatABISoftFP && hasBFloat16Type() and hasBFloat16Type() respectively.

[stuij]

clang/lib/Basic/Targets/ARM.cpp

80

Yes, I think it's a bit tricky to test this all by itself. Right now this is used when return a TypeInfo, which is used in all kinds of different contexts. I think alignment is better tested where it is eventually expressed.

[SjoerdMeijer]

Can you summarise where we are? I.e.,

• float-abi=soft doesn't work. But what is the problem? Are we not simply passing i16s, is that not what we are supposed to do?

Can you also update the description of this patch, I got totally confused by:

• "introduces an opaque, storage-only C-type __bf16, which does not introduce a new LLVM IR type, but maps it to either i16 or half type."

clang/test/CodeGen/arm-mangle-16bit-float.cpp

1

This is testing name mangling of __fp16, so is not related to this patch, please commit this separately if this is not tested yet. But I do suspect Dh is tested already, but I could be wrong.

[stuij]

In D76077#2066323, @SjoerdMeijer wrote:

Can you summarise where we are? I.e.,

• float-abi=soft doesn't work. But what is the problem? Are we not simply passing i16s, is that not what we are supposed to do?

At the moment when going through the GCC compatibility driver (standard interface), we get __bf16 is not supported on this target.
When using -cc1, we can pass -mfloat-abi=soft, we can compile source-code that contains __bf16, and we can also compile neon intrinsics in general. It looks like we just ignore soft was passed and instead we use the hard float-abi. For intrinsics we should error.

In general we pass i32's btw, not i16's. Also for __fp16.

Can you also update the description of this patch, I got totally confused by:

• "introduces an opaque, storage-only C-type __bf16, which does not introduce a new LLVM IR type, but maps it to either i16 or half type."

willdo

[SjoerdMeijer]

At the moment when going through the GCC compatibility driver (standard interface), we get __bf16 is not supported on this target.

If this is the current behaviour, and consistent with GCC, that sounds reasonable. Probably best to be explicit about this and document this in LangRef. With a note that this is subject to change?
Oh, and I guess that means we can add a test this for this? We can check this error message?

[stuij]

In D76077#2066520, @SjoerdMeijer wrote:

At the moment when going through the GCC compatibility driver (standard interface), we get __bf16 is not supported on this target.

If this is the current behaviour, and consistent with GCC, that sounds reasonable.

Unfortunately, this is not consistent with GCC. GCC will happily compile.

[stuij]

Addressed review comments. Notably what to do in combination with -mfloat-abi=soft.

[SjoerdMeijer]

Thanks, LGTM

[labrinea]

clang/lib/CodeGen/CGBuiltin.cpp

6985

shouldn't this be getTargetHooks().getABIInfo().allowBFloatArgsAndRet() ?

[stuij]

clang/lib/CodeGen/CGBuiltin.cpp

6985

thanks, done

[stuij]

Resolved merge conflicts with head. Fixed minor oversight.

clang/docs/LanguageExtensions.rst

.. code-block:: c++
   m4x4_t f(m4x4_t a, m4x4_t b, m4x4_t c) {
     return a + b * c;
   }
 
 
 Half-Precision Floating Point
 =============================
 
-Clang supports two half-precision (16-bit) floating point types: ``__fp16`` and
-``_Float16``.  These types are supported in all language modes.
+Clang supports three half-precision (16-bit) floating point types: ``__fp16``,
+``_Float16`` and ``__bf16``.  These types are supported in all language modes.
 
 ``__fp16`` is supported on every target, as it is purely a storage format; see below.
 ``_Float16`` is currently only supported on the following targets, with further
 targets pending ABI standardization:
 
 * 32-bit ARM
 * 64-bit ARM (AArch64)
 * SPIR
 
 ``_Float16`` will be supported on more targets as they define ABIs for it.
 
+``__bf16`` is purely a storage format; it is currently only supported on the following targets:
+* 32-bit ARM
+* 64-bit ARM (AArch64)
+
 ``__fp16`` is a storage and interchange format only.  This means that values of
 ``__fp16`` are immediately promoted to (at least) ``float`` when used in arithmetic
 operations, so that e.g. the result of adding two ``__fp16`` values has type ``float``.
 The behavior of ``__fp16`` is specified by the ARM C Language Extensions (`ACLE <http://infocenter.arm.com/help/topic/com.arm.doc.ihi0053d/IHI0053D_acle_2_1.pdf>`_).
 Clang uses the ``binary16`` format from IEEE 754-2008 for ``__fp16``, not the ARM
 alternative format.
 
 ``_Float16`` is an extended floating-point type.  This means that, just like arithmetic on

clang/include/clang-c/Index.h

 /*===-- clang-c/Index.h - Indexing Public C Interface -------------*- C -*-===*\
 |*                                                                            *|
 |* Part of the LLVM Project, under the Apache License v2.0 with LLVM          *|
 |* Exceptions.                                                                *|
 |* See https://llvm.org/LICENSE.txt for license information.                  *|
 |* SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception                    *|
 |*                                                                            *|
 |*===----------------------------------------------------------------------===*|
enum CXTypeKind {
   CXType_Half = 31,
   CXType_Float16 = 32,
   CXType_ShortAccum = 33,
   CXType_Accum = 34,
   CXType_LongAccum = 35,
   CXType_UShortAccum = 36,
   CXType_UAccum = 37,
   CXType_ULongAccum = 38,
+  CXType_BFloat16 = 39,
   CXType_FirstBuiltin = CXType_Void,
-  CXType_LastBuiltin = CXType_ULongAccum,
+  CXType_LastBuiltin = CXType_BFloat16,
 
   CXType_Complex = 100,
   CXType_Pointer = 101,
   CXType_BlockPointer = 102,
   CXType_LValueReference = 103,
   CXType_RValueReference = 104,
   CXType_Record = 105,
   CXType_Enum = 106,

clang/include/clang/AST/ASTContext.h

 //===- ASTContext.h - Context to hold long-lived AST nodes ------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
public:
   CanQualType UnsignedShortFractTy, UnsignedFractTy, UnsignedLongFractTy;
   CanQualType SatShortAccumTy, SatAccumTy, SatLongAccumTy;
   CanQualType SatUnsignedShortAccumTy, SatUnsignedAccumTy,
       SatUnsignedLongAccumTy;
   CanQualType SatShortFractTy, SatFractTy, SatLongFractTy;
   CanQualType SatUnsignedShortFractTy, SatUnsignedFractTy,
       SatUnsignedLongFractTy;
   CanQualType HalfTy; // [OpenCL 6.1.1.1], ARM NEON
+  CanQualType BFloat16Ty;
   CanQualType Float16Ty; // C11 extension ISO/IEC TS 18661-3
   CanQualType FloatComplexTy, DoubleComplexTy, LongDoubleComplexTy;
   CanQualType Float128ComplexTy;
   CanQualType VoidPtrTy, NullPtrTy;
   CanQualType DependentTy, OverloadTy, BoundMemberTy, UnknownAnyTy;
   CanQualType BuiltinFnTy;
   CanQualType PseudoObjectTy, ARCUnbridgedCastTy;
   CanQualType ObjCBuiltinIdTy, ObjCBuiltinClassTy, ObjCBuiltinSelTy;

clang/include/clang/AST/BuiltinTypes.def

 FLOATING_TYPE(Double, DoubleTy)
 
 // 'long double'
 FLOATING_TYPE(LongDouble, LongDoubleTy)
 
 // '_Float16'
 FLOATING_TYPE(Float16, HalfTy)
 
+// '__bf16'
+FLOATING_TYPE(BFloat16, BFloat16Ty)
+
 // '__float128'
 FLOATING_TYPE(Float128, Float128Ty)
 
 //===- Language-specific types --------------------------------------------===//
 
 // This is the type of C++0x 'nullptr'.
 BUILTIN_TYPE(NullPtr, NullPtrTy)
 

clang/include/clang/AST/Type.h

 //===- Type.h - C Language Family Type Representation -----------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
public:
   bool isRealFloatingType() const; // C99 6.2.5p10 (float, double, long double)
   /// isComplexType() does *not* include complex integers (a GCC extension).
   /// isComplexIntegerType() can be used to test for complex integers.
   bool isComplexType() const;      // C99 6.2.5p11 (complex)
   bool isAnyComplexType() const;   // C99 6.2.5p11 (complex) + Complex Int.
   bool isFloatingType() const;     // C99 6.2.5p11 (real floating + complex)
   bool isHalfType() const;         // OpenCL 6.1.1.1, NEON (IEEE 754-2008 half)
   bool isFloat16Type() const;      // C11 extension ISO/IEC TS 18661
+  bool isBFloat16Type() const;
   bool isFloat128Type() const;
   bool isRealType() const;         // C99 6.2.5p17 (real floating + integer)
   bool isArithmeticType() const;   // C99 6.2.5p18 (integer + floating)
   bool isVoidType() const;         // C99 6.2.5p19
   bool isScalarType() const;       // C99 6.2.5p21 (arithmetic + pointers)
   bool isAggregateType() const;
   bool isFundamentalType() const;
   bool isCompoundType() const;
inline bool Type::isHalfType() const {
   // FIXME: Should we allow complex __fp16? Probably not.
   return isSpecificBuiltinType(BuiltinType::Half);
 }
 
 inline bool Type::isFloat16Type() const {
   return isSpecificBuiltinType(BuiltinType::Float16);
 }
 
+inline bool Type::isBFloat16Type() const {
+  return isSpecificBuiltinType(BuiltinType::BFloat16);
+}
+
 inline bool Type::isFloat128Type() const {
   return isSpecificBuiltinType(BuiltinType::Float128);
 }
 
 inline bool Type::isNullPtrType() const {
   return isSpecificBuiltinType(BuiltinType::NullPtr);
 }
 

clang/include/clang/Basic/DiagnosticSemaKinds.td

def warn_cast_calling_conv : Warning<
   InGroup<DiagGroup<"cast-calling-convention">>;
 def note_change_calling_conv_fixit : Note<
   "consider defining %0 with the '%1' calling convention">;
 def warn_bad_function_cast : Warning<
   "cast from function call of type %0 to non-matching type %1">,
   InGroup<BadFunctionCast>, DefaultIgnore;
 def err_cast_pointer_to_non_pointer_int : Error<
   "pointer cannot be cast to type %0">;
+def err_cast_to_bfloat16 : Error<"cannot type-cast to __bf16">;
+def err_cast_from_bfloat16 : Error<"cannot type-cast from __bf16">;
 def err_typecheck_expect_scalar_operand : Error<
   "operand of type %0 where arithmetic or pointer type is required">;
 def err_typecheck_cond_incompatible_operands : Error<
   "incompatible operand types%diff{ ($ and $)|}0,1">;
 def err_cast_selector_expr : Error<
   "cannot type cast @selector expression">;
 def ext_typecheck_cond_incompatible_pointers : ExtWarn<
   "pointer type mismatch%diff{ ($ and $)|}0,1">,

clang/include/clang/Basic/Specifiers.h

 //===--- Specifiers.h - Declaration and Type Specifiers ---------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
enum TypeSpecifiersPipe {
     TSP_pipe
   };
 
   /// Specifies the kind of type.
   enum TypeSpecifierType {
     TST_unspecified,
     TST_void,
     TST_char,
     TST_wchar,        // C++ wchar_t
     TST_char8,        // C++20 char8_t (proposed)
     TST_char16,       // C++11 char16_t
     TST_char32,       // C++11 char32_t
     TST_int,
     TST_int128,
     TST_extint,       // Extended Int types.
     TST_half,         // OpenCL half, ARM NEON __fp16
     TST_Float16,      // C11 extension ISO/IEC TS 18661-3
     TST_Accum,        // ISO/IEC JTC1 SC22 WG14 N1169 Extension
     TST_Fract,
+    TST_BFloat16,
     TST_float,
     TST_double,
     TST_float128,
     TST_bool,         // _Bool
     TST_decimal32,    // _Decimal32
     TST_decimal64,    // _Decimal64
     TST_decimal128,   // _Decimal128
     TST_enum,
     TST_union,
     TST_struct,
     TST_class,        // C++ class type
     TST_interface,    // C++ (Microsoft-specific) __interface type
     TST_typename,     // Typedef, C++ class-name or enum name, etc.
     TST_typeofType,
     TST_typeofExpr,
     TST_decltype,         // C++11 decltype
     TST_underlyingType,   // __underlying_type for C++11
     TST_auto,             // C++11 auto
     TST_decltype_auto,    // C++1y decltype(auto)
     TST_auto_type,        // __auto_type extension
     TST_unknown_anytype,  // __unknown_anytype extension
     TST_atomic,           // C11 _Atomic
 #define GENERIC_IMAGE_TYPE(ImgType, Id) TST_##ImgType##_t, // OpenCL image types
 #include "clang/Basic/OpenCLImageTypes.def"

clang/include/clang/Basic/TargetBuiltins.h

 //===--- TargetBuiltins.h - Target specific builtin IDs ---------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
enum EltType {
       Int32,
       Int64,
       Poly8,
       Poly16,
       Poly64,
       Poly128,
       Float16,
       Float32,
-      Float64
+      Float64,
+      BFloat16
     };
 
     NeonTypeFlags(unsigned F) : Flags(F) {}
     NeonTypeFlags(EltType ET, bool IsUnsigned, bool IsQuad) : Flags(ET) {
       if (IsUnsigned)
         Flags |= UnsignedFlag;
       if (IsQuad)
         Flags |= QuadFlag;

clang/include/clang/Basic/TargetInfo.h

 //===--- TargetInfo.h - Expose information about the target -----*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///

 /// Fields controlling how types are laid out in memory; these may need to
 /// be copied for targets like AMDGPU that base their ABIs on an auxiliary
 /// CPU target.
 struct TransferrableTargetInfo {
   unsigned char PointerWidth, PointerAlign;
   unsigned char BoolWidth, BoolAlign;
   unsigned char IntWidth, IntAlign;
   unsigned char HalfWidth, HalfAlign;
+  unsigned char BFloat16Width, BFloat16Align;
   unsigned char FloatWidth, FloatAlign;
   unsigned char DoubleWidth, DoubleAlign;
   unsigned char LongDoubleWidth, LongDoubleAlign, Float128Align;
   unsigned char LargeArrayMinWidth, LargeArrayAlign;
   unsigned char LongWidth, LongAlign;
   unsigned char LongLongWidth, LongLongAlign;
 
   // Fixed point bit widths
struct TransferrableTargetInfo {
   unsigned char SuitableAlign;
   unsigned char DefaultAlignForAttributeAligned;
   unsigned char MinGlobalAlign;
 
   unsigned short NewAlign;
   unsigned short MaxVectorAlign;
   unsigned short MaxTLSAlign;
 
-  const llvm::fltSemantics *HalfFormat, *FloatFormat, *DoubleFormat,
-    *LongDoubleFormat, *Float128Format;
+  const llvm::fltSemantics *HalfFormat, *BFloat16Format, *FloatFormat,
+    *DoubleFormat, *LongDoubleFormat, *Float128Format;
 
   ///===---- Target Data Type Query Methods -------------------------------===//
   enum IntType {
     NoInt = 0,
     SignedChar,
     UnsignedChar,
     SignedShort,
     UnsignedShort,
protected:
   bool BigEndian;
   bool TLSSupported;
   bool VLASupported;
   bool NoAsmVariants;  // True if {|} are normal characters.
   bool HasLegalHalfType; // True if the backend supports operations on the half
                          // LLVM IR type.
   bool HasFloat128;
   bool HasFloat16;
+  bool HasBFloat16;
 
   unsigned char MaxAtomicPromoteWidth, MaxAtomicInlineWidth;
   unsigned short SimdDefaultAlign;
   std::unique_ptr<llvm::DataLayout> DataLayout;
   const char *MCountName;
   unsigned char RegParmMax, SSERegParmMax;
   TargetCXXABI TheCXXABI;
   const LangASMap *AddrSpaceMap;
public:
   virtual bool hasLegalHalfType() const { return HasLegalHalfType; }
 
   /// Determine whether the __float128 type is supported on this target.
   virtual bool hasFloat128Type() const { return HasFloat128; }
 
   /// Determine whether the _Float16 type is supported on this target.
   virtual bool hasFloat16Type() const { return HasFloat16; }
 
+  /// Determine whether the _BFloat16 type is supported on this target.
+  virtual bool hasBFloat16Type() const { return HasBFloat16; }
+
   /// Return the alignment that is suitable for storing any
   /// object with a fundamental alignment requirement.
   unsigned getSuitableAlign() const { return SuitableAlign; }
 
   /// Return the default alignment for __attribute__((aligned)) on
   /// this target, to be used if no alignment value is specified.
   unsigned getDefaultAlignForAttributeAligned() const {
     return DefaultAlignForAttributeAligned;
public:
   unsigned getHalfAlign() const { return HalfAlign; }
   const llvm::fltSemantics &getHalfFormat() const { return *HalfFormat; }
 
   /// getFloatWidth/Align/Format - Return the size/align/format of 'float'.
   unsigned getFloatWidth() const { return FloatWidth; }
   unsigned getFloatAlign() const { return FloatAlign; }
   const llvm::fltSemantics &getFloatFormat() const { return *FloatFormat; }
 
+  /// getBFloat16Width/Align/Format - Return the size/align/format of '__bf16'.
+  unsigned getBFloat16Width() const { return BFloat16Width; }
+  unsigned getBFloat16Align() const { return BFloat16Align; }
+  const llvm::fltSemantics &getBFloat16Format() const { return *BFloat16Format; }
+
   /// getDoubleWidth/Align/Format - Return the size/align/format of 'double'.
   unsigned getDoubleWidth() const { return DoubleWidth; }
   unsigned getDoubleAlign() const { return DoubleAlign; }
   const llvm::fltSemantics &getDoubleFormat() const { return *DoubleFormat; }
 
   /// getLongDoubleWidth/Align/Format - Return the size/align/format of 'long
   /// double'.
   unsigned getLongDoubleWidth() const { return LongDoubleWidth; }
public:
   }
 
   /// Return the mangled code of long double.
   virtual const char *getLongDoubleMangling() const { return "e"; }
 
   /// Return the mangled code of __float128.
   virtual const char *getFloat128Mangling() const { return "g"; }
 
+  /// Return the mangled code of bfloat.
+  virtual const char *getBFloat16Mangling() const {
+    llvm_unreachable("bfloat not implemented on this target");
+  }
+
   /// Return the value for the C99 FLT_EVAL_METHOD macro.
   virtual unsigned getFloatEvalMethod() const { return 0; }
 
   // getLargeArrayMinWidth/Align - Return the minimum array size that is
   // 'large' and its alignment.
   unsigned getLargeArrayMinWidth() const { return LargeArrayMinWidth; }
   unsigned getLargeArrayAlign() const { return LargeArrayAlign; }
 

clang/include/clang/Basic/TokenKinds.def

 
 // Altivec Extension.
 KEYWORD(__vector                    , KEYALTIVEC|KEYZVECTOR)
 KEYWORD(__pixel                     , KEYALTIVEC)
 KEYWORD(__bool                      , KEYALTIVEC|KEYZVECTOR)
 
 // ARM NEON extensions.
 ALIAS("__fp16", half                , KEYALL)
+KEYWORD(__bf16                      , KEYALL)
 
 // OpenCL Extension.
 KEYWORD(half                        , HALFSUPPORT)
 
 // Objective-C ARC keywords.
 KEYWORD(__bridge                     , KEYOBJC)
 KEYWORD(__bridge_transfer            , KEYOBJC)
 KEYWORD(__bridge_retained            , KEYOBJC)

clang/include/clang/Sema/DeclSpec.h

 //===--- DeclSpec.h - Parsed declaration specifiers -------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
public:
   static const TST TST_wchar = clang::TST_wchar;
   static const TST TST_char8 = clang::TST_char8;
   static const TST TST_char16 = clang::TST_char16;
   static const TST TST_char32 = clang::TST_char32;
   static const TST TST_int = clang::TST_int;
   static const TST TST_int128 = clang::TST_int128;
   static const TST TST_extint = clang::TST_extint;
   static const TST TST_half = clang::TST_half;
+  static const TST TST_BFloat16 = clang::TST_BFloat16;
   static const TST TST_float = clang::TST_float;
   static const TST TST_double = clang::TST_double;
   static const TST TST_float16 = clang::TST_Float16;
   static const TST TST_accum = clang::TST_Accum;
   static const TST TST_fract = clang::TST_Fract;
   static const TST TST_float128 = clang::TST_float128;
   static const TST TST_bool = clang::TST_bool;
   static const TST TST_decimal32 = clang::TST_decimal32;

clang/include/clang/Serialization/ASTBitCodes.h

 //===- ASTBitCodes.h - Enum values for the PCH bitcode format ---*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
namespace serialization {
     ///
     /// These type IDs correspond to predefined types in the AST
     /// context, such as built-in types (int) and special place-holder
     /// types (the \<overload> and \<dependent> type markers). Such
     /// types are never actually serialized, since they will be built
     /// by the AST context when it is created.
     enum PredefinedTypeIDs {
       /// The NULL type.
       PREDEF_TYPE_NULL_ID       = 0,
 
       /// The void type.
       PREDEF_TYPE_VOID_ID       = 1,
 
       /// The 'bool' or '_Bool' type.
       PREDEF_TYPE_BOOL_ID       = 2,
 
       /// The 'char' type, when it is unsigned.
       PREDEF_TYPE_CHAR_U_ID     = 3,
 
       /// The 'unsigned char' type.
       PREDEF_TYPE_UCHAR_ID      = 4,
 
       /// The 'unsigned short' type.
       PREDEF_TYPE_USHORT_ID     = 5,
 
       /// The 'unsigned int' type.
       PREDEF_TYPE_UINT_ID       = 6,
 
       /// The 'unsigned long' type.
       PREDEF_TYPE_ULONG_ID      = 7,
 
       /// The 'unsigned long long' type.
       PREDEF_TYPE_ULONGLONG_ID  = 8,
 
       /// The 'char' type, when it is signed.
       PREDEF_TYPE_CHAR_S_ID     = 9,
 
       /// The 'signed char' type.
       PREDEF_TYPE_SCHAR_ID      = 10,
 
       /// The C++ 'wchar_t' type.
       PREDEF_TYPE_WCHAR_ID      = 11,
 
       /// The (signed) 'short' type.
       PREDEF_TYPE_SHORT_ID      = 12,
 
       /// The (signed) 'int' type.
       PREDEF_TYPE_INT_ID        = 13,
 
       /// The (signed) 'long' type.
       PREDEF_TYPE_LONG_ID       = 14,
 
       /// The (signed) 'long long' type.
       PREDEF_TYPE_LONGLONG_ID   = 15,
 
       /// The 'float' type.
       PREDEF_TYPE_FLOAT_ID      = 16,
 
       /// The 'double' type.
       PREDEF_TYPE_DOUBLE_ID     = 17,
 
       /// The 'long double' type.
       PREDEF_TYPE_LONGDOUBLE_ID = 18,
 
       /// The placeholder type for overloaded function sets.
       PREDEF_TYPE_OVERLOAD_ID   = 19,
 
       /// The placeholder type for dependent types.
       PREDEF_TYPE_DEPENDENT_ID  = 20,
 
       /// The '__uint128_t' type.
       PREDEF_TYPE_UINT128_ID    = 21,
 
       /// The '__int128_t' type.
       PREDEF_TYPE_INT128_ID     = 22,
 
       /// The type of 'nullptr'.
       PREDEF_TYPE_NULLPTR_ID    = 23,
 
       /// The C++ 'char16_t' type.
       PREDEF_TYPE_CHAR16_ID     = 24,
 
       /// The C++ 'char32_t' type.
       PREDEF_TYPE_CHAR32_ID     = 25,
 
       /// The ObjC 'id' type.
       PREDEF_TYPE_OBJC_ID       = 26,
 
       /// The ObjC 'Class' type.
       PREDEF_TYPE_OBJC_CLASS    = 27,
 
       /// The ObjC 'SEL' type.
       PREDEF_TYPE_OBJC_SEL      = 28,
 
       /// The 'unknown any' placeholder type.
       PREDEF_TYPE_UNKNOWN_ANY   = 29,
 
       /// The placeholder type for bound member functions.
       PREDEF_TYPE_BOUND_MEMBER  = 30,
 
       /// The "auto" deduction type.
       PREDEF_TYPE_AUTO_DEDUCT   = 31,
 
       /// The "auto &&" deduction type.
       PREDEF_TYPE_AUTO_RREF_DEDUCT = 32,
 
       /// The OpenCL 'half' / ARM NEON __fp16 type.
       PREDEF_TYPE_HALF_ID       = 33,
 
       /// ARC's unbridged-cast placeholder type.
       PREDEF_TYPE_ARC_UNBRIDGED_CAST = 34,
 
       /// The pseudo-object placeholder type.
       PREDEF_TYPE_PSEUDO_OBJECT = 35,
 
       /// The placeholder type for builtin functions.
       PREDEF_TYPE_BUILTIN_FN = 36,
 
       /// OpenCL event type.
       PREDEF_TYPE_EVENT_ID      = 37,
 
       /// OpenCL clk event type.
       PREDEF_TYPE_CLK_EVENT_ID  = 38,
 
       /// OpenCL sampler type.
       PREDEF_TYPE_SAMPLER_ID    = 39,
 
       /// OpenCL queue type.
       PREDEF_TYPE_QUEUE_ID      = 40,
 
       /// OpenCL reserve_id type.
       PREDEF_TYPE_RESERVE_ID_ID = 41,
 
       /// The placeholder type for OpenMP array section.
       PREDEF_TYPE_OMP_ARRAY_SECTION = 42,
 
       /// The '__float128' type
       PREDEF_TYPE_FLOAT128_ID = 43,
 
       /// The '_Float16' type
       PREDEF_TYPE_FLOAT16_ID = 44,
 
       /// The C++ 'char8_t' type.
       PREDEF_TYPE_CHAR8_ID = 45,
 
       /// \brief The 'short _Accum' type
       PREDEF_TYPE_SHORT_ACCUM_ID    = 46,
 
       /// \brief The '_Accum' type
       PREDEF_TYPE_ACCUM_ID      = 47,
 
       /// \brief The 'long _Accum' type
       PREDEF_TYPE_LONG_ACCUM_ID = 48,
 
       /// \brief The 'unsigned short _Accum' type
       PREDEF_TYPE_USHORT_ACCUM_ID   = 49,
 
       /// \brief The 'unsigned _Accum' type
       PREDEF_TYPE_UACCUM_ID     = 50,
 
       /// \brief The 'unsigned long _Accum' type
       PREDEF_TYPE_ULONG_ACCUM_ID    = 51,
 
       /// \brief The 'short _Fract' type
       PREDEF_TYPE_SHORT_FRACT_ID = 52,
 
       /// \brief The '_Fract' type
       PREDEF_TYPE_FRACT_ID = 53,
 
       /// \brief The 'long _Fract' type
enum PredefinedTypeIDs {
       PREDEF_TYPE_SAT_ULONG_FRACT_ID = 69,
 
       /// The placeholder type for OpenMP array shaping operation.
       PREDEF_TYPE_OMP_ARRAY_SHAPING = 70,
 
       /// The placeholder type for OpenMP iterator expression.
       PREDEF_TYPE_OMP_ITERATOR = 71,
 
+      /// \brief The '__bf16' type
+      PREDEF_TYPE_BFLOAT16_ID = 72,
+
       /// OpenCL image types with auto numeration
 #define IMAGE_TYPE(ImgType, Id, SingletonId, Access, Suffix) \
       PREDEF_TYPE_##Id##_ID,
 #include "clang/Basic/OpenCLImageTypes.def"
       /// \brief OpenCL extension types with auto numeration
 #define EXT_OPAQUE_TYPE(ExtType, Id, Ext) \
       PREDEF_TYPE_##Id##_ID,
 #include "clang/Basic/OpenCLExtensionTypes.def"
       // \brief SVE types with auto numeration

clang/lib/AST/ASTContext.cpp

 //===- ASTContext.cpp - Context to hold long-lived AST nodes --------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //

 #include <memory>
 #include <string>
 #include <tuple>
 #include <utility>
 
 using namespace clang;
 
 enum FloatingRank {
-  Float16Rank, HalfRank, FloatRank, DoubleRank, LongDoubleRank, Float128Rank
+  BFloat16Rank, Float16Rank, HalfRank, FloatRank, DoubleRank, LongDoubleRank, Float128Rank
 };
 
 /// \returns location that is relevant when searching for Doc comments related
 /// to \p D.
 static SourceLocation getDeclLocForCommentSearch(const Decl *D,
                                                  SourceManager &SourceMgr) {
   assert(D);
 
#include "clang/Basic/AArch64SVEACLETypes.def"
   }
 
   // nullptr type (C++0x 2.14.7)
   InitBuiltinType(NullPtrTy,           BuiltinType::NullPtr);
 
   // half type (OpenCL 6.1.1.1) / ARM NEON __fp16
   InitBuiltinType(HalfTy, BuiltinType::Half);
 
+  InitBuiltinType(BFloat16Ty, BuiltinType::BFloat16);
+
   // Builtin type used to help define __builtin_va_list.
   VaListTagDecl = nullptr;
 
   // MSVC predeclares struct _GUID, and we need it to create MSGuidDecls.
   if (LangOpts.MicrosoftExt || LangOpts.Borland) {
     MSGuidTagDecl = buildImplicitRecord("_GUID");
     TUDecl->addDecl(MSGuidTagDecl);
   }

 //===----------------------------------------------------------------------===//
 
 /// getFloatTypeSemantics - Return the APFloat 'semantics' for the specified
 /// scalar floating point type.
 const llvm::fltSemantics &ASTContext::getFloatTypeSemantics(QualType T) const {
   switch (T->castAs<BuiltinType>()->getKind()) {
   default:
     llvm_unreachable("Not a floating point type!");
+  case BuiltinType::BFloat16:
+    return Target->getBFloat16Format();
   case BuiltinType::Float16:
   case BuiltinType::Half:
     return Target->getHalfFormat();
   case BuiltinType::Float:      return Target->getFloatFormat();
   case BuiltinType::Double:     return Target->getDoubleFormat();
   case BuiltinType::LongDouble:
     if (getLangOpts().OpenMP && getLangOpts().OpenMPIsDevice)
       return AuxTarget->getLongDoubleFormat();
case BuiltinType::SatUFract:
       break;
     case BuiltinType::LongFract:
     case BuiltinType::ULongFract:
     case BuiltinType::SatLongFract:
     case BuiltinType::SatULongFract:
       Width = Target->getLongFractWidth();
       Align = Target->getLongFractAlign();
       break;
+    case BuiltinType::BFloat16:
+      Width = Target->getBFloat16Width();
+      Align = Target->getBFloat16Align();
+      break;
     case BuiltinType::Float16:
     case BuiltinType::Half:
       if (Target->hasFloat16Type() || !getLangOpts().OpenMP ||
           !getLangOpts().OpenMPIsDevice) {
         Width = Target->getHalfWidth();
         Align = Target->getHalfAlign();
       } else {
         assert(getLangOpts().OpenMP && getLangOpts().OpenMPIsDevice &&
static FloatingRank getFloatingRank(QualType T) {
   switch (T->castAs<BuiltinType>()->getKind()) {
   default: llvm_unreachable("getFloatingRank(): not a floating type");
   case BuiltinType::Float16:    return Float16Rank;
   case BuiltinType::Half:       return HalfRank;
   case BuiltinType::Float:      return FloatRank;
   case BuiltinType::Double:     return DoubleRank;
   case BuiltinType::LongDouble: return LongDoubleRank;
   case BuiltinType::Float128:   return Float128Rank;
+  case BuiltinType::BFloat16:   return BFloat16Rank;
   }
 }
 
 /// getFloatingTypeOfSizeWithinDomain - Returns a real floating
 /// point or a complex type (based on typeDomain/typeSize).
 /// 'typeDomain' is a real floating point or complex type.
 /// 'typeSize' is a real floating point or complex type.
 QualType ASTContext::getFloatingTypeOfSizeWithinDomain(QualType Size,
                                                        QualType Domain) const {
   FloatingRank EltRank = getFloatingRank(Size);
   if (Domain->isComplexType()) {
     switch (EltRank) {
+    case BFloat16Rank: llvm_unreachable("Complex bfloat16 is not supported");
     case Float16Rank:
     case HalfRank: llvm_unreachable("Complex half is not supported");
     case FloatRank:      return FloatComplexTy;
     case DoubleRank:     return DoubleComplexTy;
     case LongDoubleRank: return LongDoubleComplexTy;
     case Float128Rank:   return Float128ComplexTy;
     }
   }
 
   assert(Domain->isRealFloatingType() && "Unknown domain!");
   switch (EltRank) {
   case Float16Rank:    return HalfTy;
+  case BFloat16Rank:   return BFloat16Ty;
   case HalfRank:       return HalfTy;
   case FloatRank:      return FloatTy;
   case DoubleRank:     return DoubleTy;
   case LongDoubleRank: return LongDoubleTy;
   case Float128Rank:   return Float128Ty;
   }
   llvm_unreachable("getFloatingRank(): illegal value for rank");
 }
case BuiltinType::Long:
       return C->getTargetInfo().getLongWidth() == 32 ? 'l' : 'q';
     case BuiltinType::LongLong:   return 'q';
     case BuiltinType::Int128:     return 't';
     case BuiltinType::Float:      return 'f';
     case BuiltinType::Double:     return 'd';
     case BuiltinType::LongDouble: return 'D';
     case BuiltinType::NullPtr:    return '*'; // like char*
 
+    case BuiltinType::BFloat16:
     case BuiltinType::Float16:
     case BuiltinType::Float128:
     case BuiltinType::Half:
     case BuiltinType::ShortAccum:
     case BuiltinType::Accum:
     case BuiltinType::LongAccum:
     case BuiltinType::UShortAccum:
     case BuiltinType::UAccum:
while (!Done) {
     }
   }
 
   QualType Type;
 
   // Read the base type.
   switch (*Str++) {
   default: llvm_unreachable("Unknown builtin type letter!");
+  case 'y':
+    assert(HowLong == 0 && !Signed && !Unsigned &&
+           "Bad modifiers used with 'y'!");
+    Type = Context.BFloat16Ty;
+    break;
   case 'v':
     assert(HowLong == 0 && !Signed && !Unsigned &&
            "Bad modifiers used with 'v'!");
     Type = Context.VoidTy;
     break;
   case 'h':
     assert(HowLong == 0 && !Signed && !Unsigned &&
            "Bad modifiers used with 'h'!");

clang/lib/AST/ItaniumMangle.cpp

 //===--- ItaniumMangle.cpp - Itanium C++ Name Mangling ----------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
void CXXNameMangler::mangleType(const BuiltinType *T) {
   case BuiltinType::Float128: {
     const TargetInfo *TI = getASTContext().getLangOpts().OpenMP &&
                                    getASTContext().getLangOpts().OpenMPIsDevice
                                ? getASTContext().getAuxTargetInfo()
                                : &getASTContext().getTargetInfo();
     Out << TI->getFloat128Mangling();
     break;
   }
+  case BuiltinType::BFloat16: {
+    const TargetInfo *TI = &getASTContext().getTargetInfo();
+    Out << TI->getBFloat16Mangling();
+    break;
+  }
   case BuiltinType::NullPtr:
     Out << "Dn";
     break;
 
 #define BUILTIN_TYPE(Id, SingletonId)
 #define PLACEHOLDER_TYPE(Id, SingletonId) \
   case BuiltinType::Id:
 #include "clang/AST/BuiltinTypes.def"
if (T->getVectorKind() == VectorType::NeonPolyVector) {
     case BuiltinType::Short:     EltName = "int16_t"; break;
     case BuiltinType::UShort:    EltName = "uint16_t"; break;
     case BuiltinType::Int:       EltName = "int32_t"; break;
     case BuiltinType::UInt:      EltName = "uint32_t"; break;
     case BuiltinType::LongLong:  EltName = "int64_t"; break;
     case BuiltinType::ULongLong: EltName = "uint64_t"; break;
     case BuiltinType::Double:    EltName = "float64_t"; break;
     case BuiltinType::Float:     EltName = "float32_t"; break;
-    case BuiltinType::Half:      EltName = "float16_t";break;
+    case BuiltinType::Half:      EltName = "float16_t"; break;
+    case BuiltinType::BFloat16:  EltName = "bfloat16_t"; break;
     default:
       llvm_unreachable("unexpected Neon vector element type");
     }
   }
   const char *BaseName = nullptr;
   unsigned BitSize = (T->getNumElements() *
                       getASTContext().getTypeSize(EltType));
   if (BitSize == 64)
static StringRef mangleAArch64VectorBase(const BuiltinType *EltType) {
   case BuiltinType::ULongLong:
     return "Uint64";
   case BuiltinType::Half:
     return "Float16";
   case BuiltinType::Float:
     return "Float32";
   case BuiltinType::Double:
     return "Float64";
+  case BuiltinType::BFloat16:
+    return "BFloat16";
   default:
     llvm_unreachable("Unexpected vector element base type");
   }
 }
 
 // AArch64's ABI for Neon vector types specifies that they should be mangled as
 // the equivalent internal name. The vector type must be one of the special
 // types predefined by ARM.

clang/lib/AST/MicrosoftMangle.cpp

 //===--- MicrosoftMangle.cpp - Microsoft Visual C++ Name Mangling ---------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
#include "clang/Basic/AArch64SVEACLETypes.def"
   case BuiltinType::SatUAccum:
   case BuiltinType::SatULongAccum:
   case BuiltinType::SatShortFract:
   case BuiltinType::SatFract:
   case BuiltinType::SatLongFract:
   case BuiltinType::SatUShortFract:
   case BuiltinType::SatUFract:
   case BuiltinType::SatULongFract:
+  case BuiltinType::BFloat16:
   case BuiltinType::Float128: {
     DiagnosticsEngine &Diags = Context.getDiags();
     unsigned DiagID = Diags.getCustomDiagID(
         DiagnosticsEngine::Error, "cannot mangle this built-in %0 type yet");
     Diags.Report(Range.getBegin(), DiagID)
         << T->getName(Context.getASTContext().getPrintingPolicy()) << Range;
     break;
   }

clang/lib/AST/NSAPI.cpp

 //===--- NSAPI.cpp - NSFoundation APIs ------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
#include "clang/Basic/AArch64SVEACLETypes.def"
   case BuiltinType::UnknownAny:
   case BuiltinType::ARCUnbridgedCast:
   case BuiltinType::Half:
   case BuiltinType::PseudoObject:
   case BuiltinType::BuiltinFn:
   case BuiltinType::OMPArraySection:
   case BuiltinType::OMPArrayShaping:
   case BuiltinType::OMPIterator:
+  case BuiltinType::BFloat16:
     break;
   }
 
   return None;
 }
 
 /// Returns true if \param T is a typedef of "BOOL" in objective-c.
 bool NSAPI::isObjCBOOLType(QualType T) const {

clang/lib/AST/PrintfFormatString.cpp

 //== PrintfFormatString.cpp - Analysis of printf format strings --*- C++ -*-==//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
bool PrintfSpecifier::fixType(QualType QT, const LangOptions &LangOpt,
   case BuiltinType::WChar_U:
   case BuiltinType::WChar_S:
   case BuiltinType::Char8: // FIXME: Treat like 'char'?
   case BuiltinType::Char16:
   case BuiltinType::Char32:
   case BuiltinType::UInt128:
   case BuiltinType::Int128:
   case BuiltinType::Half:
+  case BuiltinType::BFloat16:
   case BuiltinType::Float16:
   case BuiltinType::Float128:
   case BuiltinType::ShortAccum:
   case BuiltinType::Accum:
   case BuiltinType::LongAccum:
   case BuiltinType::UShortAccum:
   case BuiltinType::UAccum:
   case BuiltinType::ULongAccum:

clang/lib/AST/Type.cpp

 //===- Type.cpp - Type representation and manipulation --------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
bool Type::isRealType() const {
   if (const auto *ET = dyn_cast<EnumType>(CanonicalType))
       return ET->getDecl()->isComplete() && !ET->getDecl()->isScoped();
   return isExtIntType();
 }
 
 bool Type::isArithmeticType() const {
   if (const auto *BT = dyn_cast<BuiltinType>(CanonicalType))
     return BT->getKind() >= BuiltinType::Bool &&
-           BT->getKind() <= BuiltinType::Float128;
+           BT->getKind() <= BuiltinType::Float128 &&
+           BT->getKind() != BuiltinType::BFloat16;
   if (const auto *ET = dyn_cast<EnumType>(CanonicalType))
     // GCC allows forward declaration of enum types (forbid by C99 6.7.2.3p2).
     // If a body isn't seen by the time we get here, return false.
     //
     // C++0x: Enumerations are not arithmetic types. For now, just return
     // false for scoped enumerations since that will disable any
     // unwanted implicit conversions.
     return !ET->getDecl()->isScoped() && ET->getDecl()->isComplete();
StringRef BuiltinType::getName(const PrintingPolicy &Policy) const {
   case ULong:
     return "unsigned long";
   case ULongLong:
     return "unsigned long long";
   case UInt128:
     return "unsigned __int128";
   case Half:
     return Policy.Half ? "half" : "__fp16";
+  case BFloat16:
+    return "__bf16";
   case Float:
     return "float";
   case Double:
     return "double";
   case LongDouble:
     return "long double";
   case ShortAccum:
     return "short _Accum";

clang/lib/AST/TypeLoc.cpp

 //===- TypeLoc.cpp - Type Source Info Wrapper -----------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
TypeSpecifierType BuiltinTypeLoc::getWrittenTypeSpec() const {
   case BuiltinType::SatUAccum:
   case BuiltinType::SatULongAccum:
   case BuiltinType::SatShortFract:
   case BuiltinType::SatFract:
   case BuiltinType::SatLongFract:
   case BuiltinType::SatUShortFract:
   case BuiltinType::SatUFract:
   case BuiltinType::SatULongFract:
+  case BuiltinType::BFloat16:
     llvm_unreachable("Builtin type needs extra local data!");
     // Fall through, if the impossible happens.
 
   case BuiltinType::NullPtr:
   case BuiltinType::Overload:
   case BuiltinType::Dependent:
   case BuiltinType::BoundMember:
   case BuiltinType::UnknownAny:

clang/lib/Basic/TargetInfo.cpp

 //===--- TargetInfo.cpp - Information about Target machine ----------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
TargetInfo::TargetInfo(const llvm::Triple &T) : TargetOpts(), Triple(T) {
   // SPARC.  These should be overridden by concrete targets as needed.
   BigEndian = !T.isLittleEndian();
   TLSSupported = true;
   VLASupported = true;
   NoAsmVariants = false;
   HasLegalHalfType = false;
   HasFloat128 = false;
   HasFloat16 = false;
+  HasBFloat16 = false;
   PointerWidth = PointerAlign = 32;
   BoolWidth = BoolAlign = 8;
   IntWidth = IntAlign = 32;
   LongWidth = LongAlign = 32;
   LongLongWidth = LongLongAlign = 64;
 
   // Fixed point default bit widths
   ShortAccumWidth = ShortAccumAlign = 16;

clang/lib/Basic/Targets/AArch64.h

 //===--- AArch64.h - Declare AArch64 target feature support -----*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
public:
 
   StringRef getConstraintRegister(StringRef Constraint,
                                   StringRef Expression) const override {
     return Expression;
   }
 
   int getEHDataRegisterNumber(unsigned RegNo) const override;
 
+  const char *getBFloat16Mangling() const override { return "u6__bf16"; };
   bool hasInt128Type() const override;
 
   bool hasExtIntType() const override { return true; }
 };
 
 class LLVM_LIBRARY_VISIBILITY AArch64leTargetInfo : public AArch64TargetInfo {
 public:
   AArch64leTargetInfo(const llvm::Triple &Triple, const TargetOptions &Opts);

clang/lib/Basic/Targets/AArch64.cpp

 //===--- AArch64.cpp - Implement AArch64 target feature support -----------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
AArch64TargetInfo::AArch64TargetInfo(const llvm::Triple &Triple,
 
   MaxVectorAlign = 128;
   MaxAtomicInlineWidth = 128;
   MaxAtomicPromoteWidth = 128;
 
   LongDoubleWidth = LongDoubleAlign = SuitableAlign = 128;
   LongDoubleFormat = &llvm::APFloat::IEEEquad();
 
+  BFloat16Width = BFloat16Align = 16;
+  BFloat16Format = &llvm::APFloat::BFloat();
+
   // Make __builtin_ms_va_list available.
   HasBuiltinMSVaList = true;
 
   // Make the SVE types available.  Note that this deliberately doesn't
   // depend on SveMode, since in principle it should be possible to turn
   // SVE on and off within a translation unit.  It should also be possible
   // to compile the global declaration:
   //
bool AArch64TargetInfo::handleTargetFeatures(std::vector<std::string> &Features,
   HasCrypto = false;
   HasUnaligned = true;
   HasFullFP16 = false;
   HasDotProd = false;
   HasFP16FML = false;
   HasMTE = false;
   HasTME = false;
   HasMatMul = false;
+  HasBFloat16 = false;
   ArchKind = llvm::AArch64::ArchKind::ARMV8A;
 
   for (const auto &Feature : Features) {
     if (Feature == "+neon")
       FPU |= NeonMode;
     if (Feature == "+sve")
       FPU |= SveMode;
     if (Feature == "+crc")
for (const auto &Feature : Features) {
     if (Feature == "+fp16fml")
       HasFP16FML = true;
     if (Feature == "+mte")
       HasMTE = true;
     if (Feature == "+tme")
       HasTME = true;
     if (Feature == "+i8mm")
       HasMatMul = true;
+    if (Feature == "+bf16")
+      HasBFloat16 = true;
   }
 
   setDataLayout();
 
   return true;
 }
 
 TargetInfo::CallingConvCheckResult

clang/lib/Basic/Targets/ARM.h

 //===--- ARM.h - Declare ARM target feature support -------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
public:
 
   CallingConvCheckResult checkCallingConvention(CallingConv CC) const override;
 
   int getEHDataRegisterNumber(unsigned RegNo) const override;
 
   bool hasSjLjLowering() const override;
 
   bool hasExtIntType() const override { return true; }
+  
+  const char *getBFloat16Mangling() const override { return "u6__bf16"; };
 };
 
 class LLVM_LIBRARY_VISIBILITY ARMleTargetInfo : public ARMTargetInfo {
 public:
   ARMleTargetInfo(const llvm::Triple &Triple, const TargetOptions &Opts);
   void getTargetDefines(const LangOptions &Opts,
                         MacroBuilder &Builder) const override;
 };

clang/lib/Basic/Targets/ARM.cpp

 //===--- ARM.cpp - Implement ARM target feature support -------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //

 
 using namespace clang;
 using namespace clang::targets;
 
 void ARMTargetInfo::setABIAAPCS() {
   IsAAPCS = true;
 
   DoubleAlign = LongLongAlign = LongDoubleAlign = SuitableAlign = 64;
+  BFloat16Width = BFloat16Align = 16;
+  BFloat16Format = &llvm::APFloat::BFloat();
+
   const llvm::Triple &T = getTriple();
 
   bool IsNetBSD = T.isOSNetBSD();
   bool IsOpenBSD = T.isOSOpenBSD();
   if (!T.isOSWindows() && !IsNetBSD && !IsOpenBSD)
     WCharType = UnsignedInt;
 
   UseBitFieldTypeAlignment = true;
void ARMTargetInfo::setABIAPCS(bool IsAAPCS16) {
   const llvm::Triple &T = getTriple();
 
   IsAAPCS = false;
 
   if (IsAAPCS16)
     DoubleAlign = LongLongAlign = LongDoubleAlign = SuitableAlign = 64;
   else
     DoubleAlign = LongLongAlign = LongDoubleAlign = SuitableAlign = 32;
+  BFloat16Width = BFloat16Align = 16;
+  BFloat16Format = &llvm::APFloat::BFloat();
 
   WCharType = SignedInt;
 
   // Do not respect the alignment of bit-field types when laying out
   // structures. This corresponds to PCC_BITFIELD_TYPE_MATTERS in gcc.
   UseBitFieldTypeAlignment = false;
 
   /// gcc forces the alignment to 4 bytes, regardless of the type of the
bool ARMTargetInfo::handleTargetFeatures(std::vector<std::string> &Features,
   Unaligned = 1;
   SoftFloat = false;
   // Note that SoftFloatABI is initialized in our constructor.
   HWDiv = 0;
   DotProd = 0;
   HasMatMul = 0;
   HasFloat16 = true;
   ARMCDECoprocMask = 0;
+  HasBFloat16 = false;
 
   // This does not diagnose illegal cases like having both
   // "+vfpv2" and "+vfpv3" or having "+neon" and "-fp64".
   for (const auto &Feature : Features) {
     if (Feature == "+soft-float") {
       SoftFloat = true;
     } else if (Feature == "+vfp2sp" || Feature == "+vfp2") {
       FPU |= VFP2FPU;
if (Feature == "+soft-float") {
       MVE |= MVE_INT | MVE_FP;
       HW_FP |= HW_FP_SP | HW_FP_HP;
     } else if (Feature == "+i8mm") {
       HasMatMul = 1;
     } else if (Feature.size() == strlen("+cdecp0") && Feature >= "+cdecp0" &&
                Feature <= "+cdecp7") {
       unsigned Coproc = Feature.back() - '0';
       ARMCDECoprocMask |= (1U << Coproc);
+    } else if (Feature == "+bf16") {
+      HasBFloat16 = true;
     }
   }
 
   switch (ArchVersion) {
   case 6:
     if (ArchProfile == llvm::ARM::ProfileKind::M)
       LDREX = 0;
     else if (ArchKind == llvm::ARM::ArchKind::ARMV6K)

clang/lib/CodeGen/CGBuiltin.cpp

 //===---- CGBuiltin.cpp - Emit LLVM Code for builtins ---------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
Value *CodeGenFunction::EmitTargetBuiltinExpr(unsigned BuiltinID,
   }
 
   return EmitTargetArchBuiltinExpr(this, BuiltinID, E, ReturnValue,
                                    getTarget().getTriple().getArch());
 }
 
 static llvm::VectorType *GetNeonType(CodeGenFunction *CGF,
                                      NeonTypeFlags TypeFlags,
                                      bool HasLegalHalfType=true,
-                                     bool V1Ty=false) {
+                                     bool V1Ty=false,
+                                     bool HasBFloat16Type=true) {
   int IsQuad = TypeFlags.isQuad();
   switch (TypeFlags.getEltType()) {
   case NeonTypeFlags::Int8:
   case NeonTypeFlags::Poly8:
     return llvm::VectorType::get(CGF->Int8Ty, V1Ty ? 1 : (8 << IsQuad));
   case NeonTypeFlags::Int16:
   case NeonTypeFlags::Poly16:
     return llvm::VectorType::get(CGF->Int16Ty, V1Ty ? 1 : (4 << IsQuad));
+  case NeonTypeFlags::BFloat16:
+    if (HasBFloat16Type)
+      return llvm::VectorType::get(CGF->BFloatTy, V1Ty ? 1 : (4 << IsQuad));
+    else
+      return llvm::VectorType::get(CGF->Int16Ty, V1Ty ? 1 : (4 << IsQuad));
   case NeonTypeFlags::Float16:
     if (HasLegalHalfType)
       return llvm::VectorType::get(CGF->HalfTy, V1Ty ? 1 : (4 << IsQuad));
     else
       return llvm::VectorType::get(CGF->Int16Ty, V1Ty ? 1 : (4 << IsQuad));
   case NeonTypeFlags::Int32:
     return llvm::VectorType::get(CGF->Int32Ty, V1Ty ? 1 : (2 << IsQuad));
   case NeonTypeFlags::Int64:
Value *CodeGenFunction::EmitCommonNeonBuiltinExpr(
   if (!Arg->isIntegerConstantExpr(NeonTypeConst, getContext()))
     return nullptr;
 
   // Determine the type of this overloaded NEON intrinsic.
   NeonTypeFlags Type(NeonTypeConst.getZExtValue());
   bool Usgn = Type.isUnsigned();
   bool Quad = Type.isQuad();
   const bool HasLegalHalfType = getTarget().hasLegalHalfType();
+  const bool HasBFloat16Type = getTarget().hasBFloat16Type();
 
-  llvm::VectorType *VTy = GetNeonType(this, Type, HasLegalHalfType);
+  llvm::VectorType *VTy = GetNeonType(this, Type, HasLegalHalfType, false, HasBFloat16Type);
   llvm::Type *Ty = VTy;
   if (!Ty)
     return nullptr;
 
   auto getAlignmentValue32 = [&](Address addr) -> Value* {
     return Builder.getInt32(addr.getAlignment().getQuantity());
   };
 
if (BuiltinID == ARM::BI__builtin_arm_vcvtr_f ||
     return Builder.CreateCall(F, Ops, "vcvtr");
   }
 
   // Determine the type of this overloaded NEON intrinsic.
   NeonTypeFlags Type(Result.getZExtValue());
   bool usgn = Type.isUnsigned();
   bool rightShift = false;
 
   llvm::VectorType *VTy = GetNeonType(this, Type,
-                                      getTarget().hasLegalHalfType());
+                                      getTarget().hasLegalHalfType(),
+                                      false,
+                                      getTarget().hasBFloat16Type());
   llvm::Type *Ty = VTy;
   if (!Ty)
     return nullptr;
 
   // Many NEON builtins have identical semantics and uses in ARM and
   // AArch64. Emit these in a single function.
   auto IntrinsicMap = makeArrayRef(ARMSIMDIntrinsicMap);
   const ARMVectorIntrinsicInfo *Builtin = findARMVectorIntrinsicInMap(

clang/lib/CodeGen/CGDebugInfo.cpp

 //===--- CGDebugInfo.cpp - Emit Debug Information for a Module ------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
case BuiltinType::LongLong:
     break;
   case BuiltinType::Bool:
     Encoding = llvm::dwarf::DW_ATE_boolean;
     break;
   case BuiltinType::Half:
   case BuiltinType::Float:
   case BuiltinType::LongDouble:
   case BuiltinType::Float16:
+  case BuiltinType::BFloat16:
   case BuiltinType::Float128:
   case BuiltinType::Double:
     // FIXME: For targets where long double and __float128 have the same size,
     // they are currently indistinguishable in the debugger without some
     // special treatment. However, there is currently no consensus on encoding
     // and this should be updated once a DWARF encoding exists for distinct
     // floating point types of the same size.
     Encoding = llvm::dwarf::DW_ATE_float;

clang/lib/CodeGen/CodeGenModule.cpp

 //===--- CodeGenModule.cpp - Emit LLVM Code from ASTs for a Module --------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
CodeGenModule::CodeGenModule(ASTContext &C, const HeaderSearchOptions &HSO,
   // Initialize the type cache.
   llvm::LLVMContext &LLVMContext = M.getContext();
   VoidTy = llvm::Type::getVoidTy(LLVMContext);
   Int8Ty = llvm::Type::getInt8Ty(LLVMContext);
   Int16Ty = llvm::Type::getInt16Ty(LLVMContext);
   Int32Ty = llvm::Type::getInt32Ty(LLVMContext);
   Int64Ty = llvm::Type::getInt64Ty(LLVMContext);
   HalfTy = llvm::Type::getHalfTy(LLVMContext);
+  BFloatTy = llvm::Type::getBFloatTy(LLVMContext);
   FloatTy = llvm::Type::getFloatTy(LLVMContext);
   DoubleTy = llvm::Type::getDoubleTy(LLVMContext);
   PointerWidthInBits = C.getTargetInfo().getPointerWidth(0);
   PointerAlignInBytes =
     C.toCharUnitsFromBits(C.getTargetInfo().getPointerAlign(0)).getQuantity();
   SizeSizeInBytes =
     C.toCharUnitsFromBits(C.getTargetInfo().getMaxPointerWidth()).getQuantity();
   IntAlignInBytes =

clang/lib/CodeGen/CodeGenTypeCache.h

 //===--- CodeGenTypeCache.h - Commonly used LLVM types and info -*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //

 /// during IR emission.  It's initialized once in CodeGenModule's
 /// constructor and then copied around into new CodeGenFunctions.
 struct CodeGenTypeCache {
   /// void
   llvm::Type *VoidTy;
 
   /// i8, i16, i32, and i64
   llvm::IntegerType *Int8Ty, *Int16Ty, *Int32Ty, *Int64Ty;
-  /// float, double
-  llvm::Type *HalfTy, *FloatTy, *DoubleTy;
+  /// half, bfloat, float, double
+  llvm::Type *HalfTy, *BFloatTy, *FloatTy, *DoubleTy;
 
   /// int
   llvm::IntegerType *IntTy;
 
   /// intptr_t, size_t, and ptrdiff_t, which we assume are the same size.
   union {
     llvm::IntegerType *IntPtrTy;
     llvm::IntegerType *SizeTy;

clang/lib/CodeGen/CodeGenTypes.cpp

 //===--- CodeGenTypes.cpp - Type translation for LLVM CodeGen -------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
static llvm::Type *getTypeForFormat(llvm::LLVMContext &VMContext,
                                     const llvm::fltSemantics &format,
                                     bool UseNativeHalf = false) {
   if (&format == &llvm::APFloat::IEEEhalf()) {
     if (UseNativeHalf)
       return llvm::Type::getHalfTy(VMContext);
     else
       return llvm::Type::getInt16Ty(VMContext);
   }
+  if (&format == &llvm::APFloat::BFloat())
+    return llvm::Type::getBFloatTy(VMContext);
   if (&format == &llvm::APFloat::IEEEsingle())
     return llvm::Type::getFloatTy(VMContext);
   if (&format == &llvm::APFloat::IEEEdouble())
     return llvm::Type::getDoubleTy(VMContext);
   if (&format == &llvm::APFloat::IEEEquad())
     return llvm::Type::getFP128Ty(VMContext);
   if (&format == &llvm::APFloat::PPCDoubleDouble())
     return llvm::Type::getPPC_FP128Ty(VMContext);
   if (&format == &llvm::APFloat::x87DoubleExtended())
     return llvm::Type::getX86_FP80Ty(VMContext);
   llvm_unreachable("Unknown float format!");
case Type::Builtin: {
 
     case BuiltinType::Half:
       // Half FP can either be storage-only (lowered to i16) or native.
       ResultType = getTypeForFormat(
           getLLVMContext(), Context.getFloatTypeSemantics(T),
           Context.getLangOpts().NativeHalfType ||
               !Context.getTargetInfo().useFP16ConversionIntrinsics());
       break;
+    case BuiltinType::BFloat16:
     case BuiltinType::Float:
     case BuiltinType::Double:
     case BuiltinType::LongDouble:
     case BuiltinType::Float128:
       ResultType = getTypeForFormat(getLLVMContext(),
                                     Context.getFloatTypeSemantics(T),
                                     /* UseNativeHalf = */ false);
       break;

clang/lib/CodeGen/ItaniumCXXABI.cpp

 //===------- ItaniumCXXABI.cpp - Emit LLVM Code from ASTs for a Module ----===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
#include "clang/Basic/AArch64SVEACLETypes.def"
     case BuiltinType::SatUAccum:
     case BuiltinType::SatULongAccum:
     case BuiltinType::SatShortFract:
     case BuiltinType::SatFract:
     case BuiltinType::SatLongFract:
     case BuiltinType::SatUShortFract:
     case BuiltinType::SatUFract:
     case BuiltinType::SatULongFract:
+    case BuiltinType::BFloat16:
       return false;
 
     case BuiltinType::Dependent:
 #define BUILTIN_TYPE(Id, SingletonId)
 #define PLACEHOLDER_TYPE(Id, SingletonId) \
     case BuiltinType::Id:
 #include "clang/AST/BuiltinTypes.def"
       llvm_unreachable("asking for RRTI for a placeholder type!");

clang/lib/CodeGen/TargetInfo.cpp

 //===---- TargetInfo.cpp - Encapsulate target details -----------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
enum ABIKind {
     APCS = 0,
     AAPCS = 1,
     AAPCS_VFP = 2,
     AAPCS16_VFP = 3,
   };
 
 private:
   ABIKind Kind;
+  bool IsFloatABISoftFP;
 
 public:
   ARMABIInfo(CodeGenTypes &CGT, ABIKind _Kind)
       : SwiftABIInfo(CGT), Kind(_Kind) {
     setCCs();
+    IsFloatABISoftFP = CGT.getCodeGenOpts().FloatABI == "softfp" ||
+        CGT.getCodeGenOpts().FloatABI == ""; // default
   }
 
   bool isEABI() const {
     switch (getTarget().getTriple().getEnvironment()) {
     case llvm::Triple::Android:
     case llvm::Triple::EABI:
     case llvm::Triple::EABIHF:
     case llvm::Triple::GNUEABI:
bool IsAAPCS_VFP =
       !isVariadic && isEffectivelyAAPCS_VFP(functionCallConv, /* AAPCS16 */ false);
 
   Ty = useFirstFieldIfTransparentUnion(Ty);
 
   // Handle illegal vector types here.
   if (isIllegalVectorType(Ty))
     return coerceIllegalVector(Ty);
 
-  // _Float16 and __fp16 get passed as if it were an int or float, but with
-  // the top 16 bits unspecified. This is not done for OpenCL as it handles the
-  // half type natively, and does not need to interwork with AAPCS code.
+  // _Float16 and __fp16 get passed as if it were an int or float, but
+  // with the top 16 bits unspecified. This is not done for OpenCL as it handles
+  // the half type natively, and does not need to interwork with AAPCS code.
   if ((Ty->isFloat16Type() || Ty->isHalfType()) &&
       !getContext().getLangOpts().NativeHalfArgsAndReturns) {
     llvm::Type *ResType = IsAAPCS_VFP ?
       llvm::Type::getFloatTy(getVMContext()) :
       llvm::Type::getInt32Ty(getVMContext());
     return ABIArgInfo::getDirect(ResType);
   }
 
+  // __bf16  get passed using the bfloat ir type, or using i32 but
+  // with the top 16 bits unspecified.
+  if (Ty->isBFloat16Type() && IsFloatABISoftFP) {
+    llvm::Type *ResType = llvm::Type::getInt32Ty(getVMContext());
+    return ABIArgInfo::getDirect(ResType);
+  }
+
   if (!isAggregateTypeForABI(Ty)) {
     // Treat an enum type as its underlying type.
     if (const EnumType *EnumTy = Ty->getAs<EnumType>()) {
       Ty = EnumTy->getDecl()->getIntegerType();
     }
 
     if (const auto *EIT = Ty->getAs<ExtIntType>())
       if (EIT->getNumBits() > 64)
ABIArgInfo ARMABIInfo::classifyReturnType(QualType RetTy, bool isVariadic,
 
   if (RetTy->isVoidType())
     return ABIArgInfo::getIgnore();
 
   if (const VectorType *VT = RetTy->getAs<VectorType>()) {
     // Large vector types should be returned via memory.
     if (getContext().getTypeSize(RetTy) > 128)
       return getNaturalAlignIndirect(RetTy);
-    // FP16 vectors should be converted to integer vectors
-    if (!getTarget().hasLegalHalfType() &&
+    // FP16/BF16 vectors should be converted to integer vectors
+    // This check is similar  to isIllegalVectorType - refactor?
+    if ((!getTarget().hasLegalHalfType() &&
         (VT->getElementType()->isFloat16Type() ||
-         VT->getElementType()->isHalfType()))
+         VT->getElementType()->isHalfType())) ||
+        (IsFloatABISoftFP &&
+         VT->getElementType()->isBFloat16Type()))
       return coerceIllegalVector(RetTy);
   }
 
   // _Float16 and __fp16 get returned as if it were an int or float, but with
   // the top 16 bits unspecified. This is not done for OpenCL as it handles the
   // half type natively, and does not need to interwork with AAPCS code.
   if ((RetTy->isFloat16Type() || RetTy->isHalfType()) &&
       !getContext().getLangOpts().NativeHalfArgsAndReturns) {
     llvm::Type *ResType = IsAAPCS_VFP ?
       llvm::Type::getFloatTy(getVMContext()) :
       llvm::Type::getInt32Ty(getVMContext());
     return ABIArgInfo::getDirect(ResType);
   }
 
+  // if we're using the softfp float abi, __bf16 get returned as if it were an
+  // int but with the top 16 bits unspecified.
+  if (RetTy->isBFloat16Type()) {
+    llvm::Type *ResType = IsAAPCS_VFP ?
+      llvm::Type::getBFloatTy(getVMContext()) :
+      llvm::Type::getInt32Ty(getVMContext());
+    return ABIArgInfo::getDirect(ResType);
+  }
+
   if (!isAggregateTypeForABI(RetTy)) {
     // Treat an enum type as its underlying type.
     if (const EnumType *EnumTy = RetTy->getAs<EnumType>())
       RetTy = EnumTy->getDecl()->getIntegerType();
 
     if (const auto *EIT = RetTy->getAs<ExtIntType>())
       if (EIT->getNumBits() > 64)
         return getNaturalAlignIndirect(RetTy, /*ByVal=*/false);
ABIArgInfo ARMABIInfo::classifyReturnType(QualType RetTy, bool isVariadic,
   }
 
   return getNaturalAlignIndirect(RetTy);
 }
 
 /// isIllegalVector - check whether Ty is an illegal vector type.
 bool ARMABIInfo::isIllegalVectorType(QualType Ty) const {
   if (const VectorType *VT = Ty->getAs<VectorType> ()) {
-    // On targets that don't support FP16, FP16 is expanded into float, and we
-    // don't want the ABI to depend on whether or not FP16 is supported in
-    // hardware. Thus return false to coerce FP16 vectors into integer vectors.
-    if (!getTarget().hasLegalHalfType() &&
+    // On targets that don't support half, fp16 or bfloat, they are expanded
+    // into float, and we don't want the ABI to depend on whether or not they
+    // are supported in hardware. Thus return false to coerce vectors of these
+    // types into integer vectors.
+    // We do not depend on hasLegalHalfType for bfloat as it is a
+    // separate IR type.
+    if ((!getTarget().hasLegalHalfType() &&
         (VT->getElementType()->isFloat16Type() ||
-         VT->getElementType()->isHalfType()))
+         VT->getElementType()->isHalfType())) ||
+        (IsFloatABISoftFP &&
+         VT->getElementType()->isBFloat16Type()))
       return true;
     if (isAndroid()) {
       // Android shipped using Clang 3.1, which supported a slightly different
       // vector ABI. The primary differences were that 3-element vector types
       // were legal, and so were sub 32-bit vectors (i.e. <2 x i8>). This path
       // accepts that legacy behavior for Android only.
       // Check whether VT is legal.
       unsigned NumElements = VT->getNumElements();
if (llvm::any_of(RD->fields(), [this](FieldDecl *FD) {
           return FD && containsAnyFP16Vectors(FD->getType());
         }))
       return true;
 
     return false;
   } else {
     if (const VectorType *VT = Ty->getAs<VectorType>())
       return (VT->getElementType()->isFloat16Type() ||
+              VT->getElementType()->isBFloat16Type() ||
               VT->getElementType()->isHalfType());
     return false;
   }
 }
 
 bool ARMABIInfo::isLegalVectorTypeForSwift(CharUnits vectorSize,
                                            llvm::Type *eltTy,
                                            unsigned numElts) const {

clang/lib/Format/FormatToken.cpp

 //===--- FormatToken.cpp - Format C++ code --------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
bool FormatToken::isSimpleTypeSpecifier() const {
   case tok::kw_signed:
   case tok::kw_unsigned:
   case tok::kw_void:
   case tok::kw_char:
   case tok::kw_int:
   case tok::kw_half:
   case tok::kw_float:
   case tok::kw_double:
+  case tok::kw___bf16:
   case tok::kw__Float16:
   case tok::kw___float128:
   case tok::kw_wchar_t:
   case tok::kw_bool:
   case tok::kw___underlying_type:
   case tok::annot_typename:
   case tok::kw_char8_t:
   case tok::kw_char16_t:

clang/lib/Index/USRGeneration.cpp

 //===- USRGeneration.cpp - Routines for USR generation --------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
#include "clang/Basic/AArch64SVEACLETypes.def"
         case BuiltinType::SatUAccum:
         case BuiltinType::SatULongAccum:
         case BuiltinType::SatShortFract:
         case BuiltinType::SatFract:
         case BuiltinType::SatLongFract:
         case BuiltinType::SatUShortFract:
         case BuiltinType::SatUFract:
         case BuiltinType::SatULongFract:
+        case BuiltinType::BFloat16:
           IgnoreResults = true;
           return;
         case BuiltinType::ObjCId:
           c = 'o'; break;
         case BuiltinType::ObjCClass:
           c = 'O'; break;
         case BuiltinType::ObjCSel:
           c = 'e'; break;

clang/lib/Parse/ParseDecl.cpp

 //===--- ParseDecl.cpp - Declaration Parsing --------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
while (1) {
     case tok::kw___int128:
       isInvalid = DS.SetTypeSpecType(DeclSpec::TST_int128, Loc, PrevSpec,
                                      DiagID, Policy);
       break;
     case tok::kw_half:
       isInvalid = DS.SetTypeSpecType(DeclSpec::TST_half, Loc, PrevSpec,
                                      DiagID, Policy);
       break;
+    case tok::kw___bf16:
+      isInvalid = DS.SetTypeSpecType(DeclSpec::TST_BFloat16, Loc, PrevSpec,
+                                     DiagID, Policy);
+      break;
     case tok::kw_float:
       isInvalid = DS.SetTypeSpecType(DeclSpec::TST_float, Loc, PrevSpec,
                                      DiagID, Policy);
       break;
     case tok::kw_double:
       isInvalid = DS.SetTypeSpecType(DeclSpec::TST_double, Loc, PrevSpec,
                                      DiagID, Policy);
       break;
bool Parser::isKnownToBeTypeSpecifier(const Token &Tok) const {
   case tok::kw_void:
   case tok::kw_char:
   case tok::kw_wchar_t:
   case tok::kw_char8_t:
   case tok::kw_char16_t:
   case tok::kw_char32_t:
   case tok::kw_int:
   case tok::kw__ExtInt:
+  case tok::kw___bf16:
   case tok::kw_half:
   case tok::kw_float:
   case tok::kw_double:
   case tok::kw__Accum:
   case tok::kw__Fract:
   case tok::kw__Float16:
   case tok::kw___float128:
   case tok::kw_bool:
bool Parser::isTypeSpecifierQualifier() {
   case tok::kw_char:
   case tok::kw_wchar_t:
   case tok::kw_char8_t:
   case tok::kw_char16_t:
   case tok::kw_char32_t:
   case tok::kw_int:
   case tok::kw__ExtInt:
   case tok::kw_half:
+  case tok::kw___bf16:
   case tok::kw_float:
   case tok::kw_double:
   case tok::kw__Accum:
   case tok::kw__Fract:
   case tok::kw__Float16:
   case tok::kw___float128:
   case tok::kw_bool:
   case tok::kw__Bool:
bool Parser::isDeclarationSpecifier(bool DisambiguatingWithExpression) {
   case tok::kw_wchar_t:
   case tok::kw_char8_t:
   case tok::kw_char16_t:
   case tok::kw_char32_t:
 
   case tok::kw_int:
   case tok::kw__ExtInt:
   case tok::kw_half:
+  case tok::kw___bf16:
   case tok::kw_float:
   case tok::kw_double:
   case tok::kw__Accum:
   case tok::kw__Fract:
   case tok::kw__Float16:
   case tok::kw___float128:
   case tok::kw_bool:
   case tok::kw__Bool:

clang/lib/Parse/ParseExpr.cpp

 //===--- ParseExpr.cpp - Expression Parsing -------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
#undef RTT_JOIN
   case tok::kw___int64:
   case tok::kw___int128:
   case tok::kw__ExtInt:
   case tok::kw_signed:
   case tok::kw_unsigned:
   case tok::kw_half:
   case tok::kw_float:
   case tok::kw_double:
+  case tok::kw___bf16:
   case tok::kw__Float16:
   case tok::kw___float128:
   case tok::kw_void:
   case tok::kw_typename:
   case tok::kw_typeof:
   case tok::kw___vector:
 #define GENERIC_IMAGE_TYPE(ImgType, Id) case tok::kw_##ImgType##_t:
 #include "clang/Basic/OpenCLImageTypes.def"

clang/lib/Parse/ParseExprCXX.cpp

 //===--- ParseExprCXX.cpp - C++ Expression Parsing ------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
case tok::kw_char:
     DS.SetTypeSpecType(DeclSpec::TST_char, Loc, PrevSpec, DiagID, Policy);
     break;
   case tok::kw_int:
     DS.SetTypeSpecType(DeclSpec::TST_int, Loc, PrevSpec, DiagID, Policy);
     break;
   case tok::kw___int128:
     DS.SetTypeSpecType(DeclSpec::TST_int128, Loc, PrevSpec, DiagID, Policy);
     break;
+  case tok::kw___bf16:
+    DS.SetTypeSpecType(DeclSpec::TST_BFloat16, Loc, PrevSpec, DiagID, Policy);
+    break;
   case tok::kw_half:
     DS.SetTypeSpecType(DeclSpec::TST_half, Loc, PrevSpec, DiagID, Policy);
     break;
   case tok::kw_float:
     DS.SetTypeSpecType(DeclSpec::TST_float, Loc, PrevSpec, DiagID, Policy);
     break;
   case tok::kw_double:
     DS.SetTypeSpecType(DeclSpec::TST_double, Loc, PrevSpec, DiagID, Policy);

clang/lib/Parse/ParseTentative.cpp

 //===--- ParseTentative.cpp - Ambiguity Resolution Parsing ----------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
Parser::isCXXDeclarationSpecifier(Parser::TPResult BracedCastResult,
   case tok::kw_long:
   case tok::kw___int64:
   case tok::kw___int128:
   case tok::kw_signed:
   case tok::kw_unsigned:
   case tok::kw_half:
   case tok::kw_float:
   case tok::kw_double:
+  case tok::kw___bf16:
   case tok::kw__Float16:
   case tok::kw___float128:
   case tok::kw_void:
   case tok::annot_decltype:
 #define GENERIC_IMAGE_TYPE(ImgType, Id) case tok::kw_##ImgType##_t:
 #include "clang/Basic/OpenCLImageTypes.def"
     if (NextToken().is(tok::l_paren))
       return TPResult::Ambiguous;
bool Parser::isCXXDeclarationSpecifierAType() {
   case tok::kw_long:
   case tok::kw___int64:
   case tok::kw___int128:
   case tok::kw_signed:
   case tok::kw_unsigned:
   case tok::kw_half:
   case tok::kw_float:
   case tok::kw_double:
+  case tok::kw___bf16:
   case tok::kw__Float16:
   case tok::kw___float128:
   case tok::kw_void:
   case tok::kw___unknown_anytype:
   case tok::kw___auto_type:
 #define GENERIC_IMAGE_TYPE(ImgType, Id) case tok::kw_##ImgType##_t:
 #include "clang/Basic/OpenCLImageTypes.def"
     return true;

clang/lib/Sema/DeclSpec.cpp

 //===--- DeclSpec.cpp - Declaration Specifier Semantic Analysis -----------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
switch (DS.getTypeSpecType()) {
     case TST_extint:
     case TST_struct:
     case TST_interface:
     case TST_union:
     case TST_unknown_anytype:
     case TST_unspecified:
     case TST_void:
     case TST_wchar:
+    case TST_BFloat16:
 #define GENERIC_IMAGE_TYPE(ImgType, Id) case TST_##ImgType##_t:
 #include "clang/Basic/OpenCLImageTypes.def"
       return false;
 
     case TST_decltype_auto:
       // This must have an initializer, so can't be a function declaration,
       // even if the initializer has function type.
       return false;
const char *DeclSpec::getSpecifierName(DeclSpec::TST T,
   case DeclSpec::TST_typeofExpr:  return "typeof";
   case DeclSpec::TST_auto:        return "auto";
   case DeclSpec::TST_auto_type:   return "__auto_type";
   case DeclSpec::TST_decltype:    return "(decltype)";
   case DeclSpec::TST_decltype_auto: return "decltype(auto)";
   case DeclSpec::TST_underlyingType: return "__underlying_type";
   case DeclSpec::TST_unknown_anytype: return "__unknown_anytype";
   case DeclSpec::TST_atomic: return "_Atomic";
+  case DeclSpec::TST_BFloat16: return "__bf16";
 #define GENERIC_IMAGE_TYPE(ImgType, Id) \
   case DeclSpec::TST_##ImgType##_t: \
     return #ImgType "_t";
 #include "clang/Basic/OpenCLImageTypes.def"
   case DeclSpec::TST_error:       return "(error)";
   }
   llvm_unreachable("Unknown typespec!");
 }

clang/lib/Sema/SemaCast.cpp

 //===--- SemaCast.cpp - Semantic Analysis for Casts -----------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
void CastOperation::CheckCStyleCast() {
   // Note that member pointers were filtered out with C++, above.
 
   if (isa<ObjCSelectorExpr>(SrcExpr.get())) {
     Self.Diag(SrcExpr.get()->getExprLoc(), diag::err_cast_selector_expr);
     SrcExpr = ExprError();
     return;
   }
 
+  // Can't cast to or from bfloat
+  if (DestType->isBFloat16Type() && !SrcType->isBFloat16Type()) {
+    Self.Diag(SrcExpr.get()->getExprLoc(), diag::err_cast_to_bfloat16)
+        << SrcExpr.get()->getSourceRange();
+    SrcExpr = ExprError();
+    return;
+  }
+  if (SrcType->isBFloat16Type() && !DestType->isBFloat16Type()) {
+    Self.Diag(SrcExpr.get()->getExprLoc(), diag::err_cast_from_bfloat16)
+        << SrcExpr.get()->getSourceRange();
+    SrcExpr = ExprError();
+    return;
+  }
+
   // If either type is a pointer, the other type has to be either an
   // integer or a pointer.
   if (!DestType->isArithmeticType()) {
     if (!SrcType->isIntegralType(Self.Context) && SrcType->isArithmeticType()) {
       Self.Diag(SrcExpr.get()->getExprLoc(),
                 diag::err_cast_pointer_from_non_pointer_int)
         << SrcType << SrcExpr.get()->getSourceRange();
       SrcExpr = ExprError();

clang/lib/Sema/SemaChecking.cpp

 //===- SemaChecking.cpp - Extra Semantic Checking -------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
case NeonTypeFlags::Float16:
     assert(!shift && "cannot shift float types!");
     return (4 << IsQuad) - 1;
   case NeonTypeFlags::Float32:
     assert(!shift && "cannot shift float types!");
     return (2 << IsQuad) - 1;
   case NeonTypeFlags::Float64:
     assert(!shift && "cannot shift float types!");
     return (1 << IsQuad) - 1;
+  case NeonTypeFlags::BFloat16:
+    assert(!shift && "cannot shift float types!");
+    return (4 << IsQuad) - 1;
   }
   llvm_unreachable("Invalid NeonTypeFlag!");
 }
 
 /// getNeonEltType - Return the QualType corresponding to the elements of
 /// the vector type specified by the NeonTypeFlags.  This is used to check
 /// the pointer arguments for Neon load/store intrinsics.
 static QualType getNeonEltType(NeonTypeFlags Flags, ASTContext &Context,
static QualType getNeonEltType(NeonTypeFlags Flags, ASTContext &Context,
   case NeonTypeFlags::Poly128:
     break;
   case NeonTypeFlags::Float16:
     return Context.HalfTy;
   case NeonTypeFlags::Float32:
     return Context.FloatTy;
   case NeonTypeFlags::Float64:
     return Context.DoubleTy;
+  case NeonTypeFlags::BFloat16:
+    return Context.BFloat16Ty;
   }
   llvm_unreachable("Invalid NeonTypeFlag!");
 }
 
 bool Sema::CheckSVEBuiltinFunctionCall(unsigned BuiltinID, CallExpr *TheCall) {
   // Range check SVE intrinsics that take immediate values.
   SmallVector<std::tuple<int,int,int>, 3> ImmChecks;
 

clang/lib/Sema/SemaDecl.cpp

 //===--- SemaDecl.cpp - Semantic Analysis for Declarations ----------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
bool Sema::isSimpleTypeSpecifier(tok::TokenKind Kind) const {
   case tok::kw_signed:
   case tok::kw_unsigned:
   case tok::kw_void:
   case tok::kw_char:
   case tok::kw_int:
   case tok::kw_half:
   case tok::kw_float:
   case tok::kw_double:
+  case tok::kw___bf16:
   case tok::kw__Float16:
   case tok::kw___float128:
   case tok::kw_wchar_t:
   case tok::kw_bool:
   case tok::kw___underlying_type:
   case tok::kw___auto_type:
     return true;
 

clang/lib/Sema/SemaExpr.cpp

 //===--- SemaExpr.cpp - Semantic Analysis for Expressions -----------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
QualType Sema::CheckConditionalOperands(ExprResult &Cond, ExprResult &LHS,
   // to find a common type: C99 6.5.15p3,5.
   if (LHSTy->isArithmeticType() && RHSTy->isArithmeticType()) {
     LHS = ImpCastExprToType(LHS.get(), ResTy, PrepareScalarCast(LHS, ResTy));
     RHS = ImpCastExprToType(RHS.get(), ResTy, PrepareScalarCast(RHS, ResTy));
 
     return ResTy;
   }
 
+  // And if they're both bfloat (which isn't arithmetic), that's fine too.
+  if (LHSTy->isBFloat16Type() && RHSTy->isBFloat16Type()) {
+    return LHSTy;
+  }
+
   // If both operands are the same structure or union type, the result is that
   // type.
   if (const RecordType *LHSRT = LHSTy->getAs<RecordType>()) {    // C99 6.5.15p3
     if (const RecordType *RHSRT = RHSTy->getAs<RecordType>())
       if (LHSRT->getDecl() == RHSRT->getDecl())
         // "If both the operands have structure or union type, the result has
         // that type."  This implies that CV qualifiers are dropped.
         return LHSTy.getUnqualifiedType();

clang/lib/Sema/SemaOverload.cpp

 //===--- SemaOverload.cpp - C++ Overloading -------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
FromType = ToType.getUnqualifiedType();
              (ToType->isAnyComplexType() && FromType->isArithmeticType())) {
     // Complex-real conversions (C99 6.3.1.7)
     SCS.Second = ICK_Complex_Real;
     FromType = ToType.getUnqualifiedType();
   } else if (FromType->isRealFloatingType() && ToType->isRealFloatingType()) {
     // FIXME: disable conversions between long double and __float128 if
     // their representation is different until there is back end support
     // We of course allow this conversion if long double is really double.
+
+    // conversions between bfloat and other floats are not permitted
+    if (FromType == S.Context.BFloat16Ty || ToType == S.Context.BFloat16Ty)
+      return false;
     if (&S.Context.getFloatTypeSemantics(FromType) !=
         &S.Context.getFloatTypeSemantics(ToType)) {
       bool Float128AndLongDouble = ((FromType == S.Context.Float128Ty &&
                                     ToType == S.Context.LongDoubleTy) ||
                                    (FromType == S.Context.LongDoubleTy &&
                                     ToType == S.Context.Float128Ty));
       if (Float128AndLongDouble &&
           (&S.Context.getFloatTypeSemantics(S.Context.LongDoubleTy) ==
            &llvm::APFloat::PPCDoubleDouble()))
         return false;
     }
     // Floating point conversions (C++ 4.8).
     SCS.Second = ICK_Floating_Conversion;
     FromType = ToType.getUnqualifiedType();
   } else if ((FromType->isRealFloatingType() &&
               ToType->isIntegralType(S.Context)) ||
              (FromType->isIntegralOrUnscopedEnumerationType() &&
               ToType->isRealFloatingType())) {
+    // conversions between bfloat and int are not permitted
+    if (FromType->isBFloat16Type() || ToType->isBFloat16Type())
+      return false;
+
     // Floating-integral conversions (C++ 4.9).
     SCS.Second = ICK_Floating_Integral;
     FromType = ToType.getUnqualifiedType();
   } else if (S.IsBlockPointerConversion(FromType, ToType, FromType)) {
     SCS.Second = ICK_Block_Pointer_Conversion;
   } else if (AllowObjCWritebackConversion &&
              S.isObjCWritebackConversion(FromType, ToType, FromType)) {
     SCS.Second = ICK_Writeback_Conversion;

clang/lib/Sema/SemaTemplateVariadic.cpp

 //===------- SemaTemplateVariadic.cpp - C++ Variadic Templates ------------===/
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //===----------------------------------------------------------------------===/
 //
 //  This file implements semantic analysis for C++0x variadic templates.
bool Sema::containsUnexpandedParameterPacks(Declarator &D) {
   case TST_enum:
   case TST_union:
   case TST_struct:
   case TST_interface:
   case TST_class:
   case TST_auto:
   case TST_auto_type:
   case TST_decltype_auto:
+  case TST_BFloat16:
 #define GENERIC_IMAGE_TYPE(ImgType, Id) case TST_##ImgType##_t:
 #include "clang/Basic/OpenCLImageTypes.def"
   case TST_unknown_anytype:
   case TST_error:
     break;
   }
 
   for (unsigned I = 0, N = D.getNumTypeObjects(); I != N; ++I) {

clang/lib/Sema/SemaType.cpp

 //===--- SemaType.cpp - Semantic Analysis for Types -----------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
case DeclSpec::TST_float16:
     // ToDo: more precise diagnostics for CUDA.
     if (!S.Context.getTargetInfo().hasFloat16Type() && !S.getLangOpts().CUDA &&
         !(S.getLangOpts().OpenMP && S.getLangOpts().OpenMPIsDevice))
       S.Diag(DS.getTypeSpecTypeLoc(), diag::err_type_unsupported)
         << "_Float16";
     Result = Context.Float16Ty;
     break;
   case DeclSpec::TST_half:    Result = Context.HalfTy; break;
+  case DeclSpec::TST_BFloat16:
+    if (!S.Context.getTargetInfo().hasBFloat16Type())
+      S.Diag(DS.getTypeSpecTypeLoc(), diag::err_type_unsupported)
+        << "__bf16";
+    Result = Context.BFloat16Ty;
+    break;
   case DeclSpec::TST_float:   Result = Context.FloatTy; break;
   case DeclSpec::TST_double:
     if (DS.getTypeSpecWidth() == DeclSpec::TSW_long)
       Result = Context.LongDoubleTy;
     else
       Result = Context.DoubleTy;
     break;
   case DeclSpec::TST_float128:
return BTy->getKind() == BuiltinType::SChar ||
          BTy->getKind() == BuiltinType::UShort ||
          BTy->getKind() == BuiltinType::Int ||
          BTy->getKind() == BuiltinType::UInt ||
          BTy->getKind() == BuiltinType::Long ||
          BTy->getKind() == BuiltinType::ULong ||
          BTy->getKind() == BuiltinType::LongLong ||
          BTy->getKind() == BuiltinType::ULongLong ||
          BTy->getKind() == BuiltinType::Float ||
-         BTy->getKind() == BuiltinType::Half;
+         BTy->getKind() == BuiltinType::Half ||
+         BTy->getKind() == BuiltinType::BFloat16;
 }
 
 /// HandleNeonVectorTypeAttr - The "neon_vector_type" and
 /// "neon_polyvector_type" attributes are used to create vector types that
 /// are mangled according to ARM's ABI.  Otherwise, these types are identical
 /// to those created with the "vector_size" attribute.  Unlike "vector_size"
 /// the argument to these Neon attributes is the number of vector elements,
 /// not the vector size in bytes.  The vector width and element type must

clang/lib/Serialization/ASTCommon.cpp

 //===--- ASTCommon.cpp - Common stuff for ASTReader/ASTWriter----*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
case BuiltinType::OMPArraySection:
     ID = PREDEF_TYPE_OMP_ARRAY_SECTION;
     break;
   case BuiltinType::OMPArrayShaping:
     ID = PREDEF_TYPE_OMP_ARRAY_SHAPING;
     break;
   case BuiltinType::OMPIterator:
     ID = PREDEF_TYPE_OMP_ITERATOR;
     break;
+  case BuiltinType::BFloat16:
+    ID = PREDEF_TYPE_BFLOAT16_ID;
+    break;
   }
 
   return TypeIdx(ID);
 }
 
 unsigned serialization::ComputeHash(Selector Sel) {
   unsigned N = Sel.getNumArgs();
   if (N == 0)

clang/lib/Serialization/ASTReader.cpp

 //===- ASTReader.cpp - AST File Reader ------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
case PREDEF_TYPE_LONG_ID:
       T = Context.LongTy;
       break;
     case PREDEF_TYPE_LONGLONG_ID:
       T = Context.LongLongTy;
       break;
     case PREDEF_TYPE_INT128_ID:
       T = Context.Int128Ty;
       break;
+    case PREDEF_TYPE_BFLOAT16_ID:
+      T = Context.BFloat16Ty;
+      break;
     case PREDEF_TYPE_HALF_ID:
       T = Context.HalfTy;
       break;
     case PREDEF_TYPE_FLOAT_ID:
       T = Context.FloatTy;
       break;
     case PREDEF_TYPE_DOUBLE_ID:
       T = Context.DoubleTy;

clang/test/CodeGen/arm-bf16-params-returns.c

+// RUN: %clang_cc1 -triple armv8.6a-arm-none-eabi -target-abi aapcs -mfloat-abi hard -target-feature +bf16 -target-feature +neon -emit-llvm -O2 -o - %s | opt -S -mem2reg -sroa | FileCheck %s --check-prefix=CHECK32-HARD
+// RUN: %clang_cc1 -triple aarch64-arm-none-eabi -target-abi aapcs -mfloat-abi hard -target-feature +bf16 -target-feature +neon -emit-llvm -O2 -o - %s | opt -S -mem2reg -sroa | FileCheck %s --check-prefix=CHECK64-HARD
+// RUN: %clang_cc1 -triple armv8.6a-arm-none-eabi -target-abi aapcs -mfloat-abi softfp -target-feature +bf16 -target-feature +neon -emit-llvm -O2 -o - %s | opt -S -mem2reg -sroa | FileCheck %s --check-prefix=CHECK32-SOFTFP
+// RUN: %clang_cc1 -triple aarch64-arm-none-eabi -target-abi aapcs -mfloat-abi softfp -target-feature +bf16 -target-feature +neon -emit-llvm -O2 -o - %s | opt -S -mem2reg -sroa | FileCheck %s --check-prefix=CHECK64-SOFTFP
+
+// function return types
+__bf16 test_ret_bf16(__bf16 v) {
+  return v;
+}
+// CHECK32-HARD: define arm_aapcs_vfpcc bfloat @test_ret_bf16(bfloat returned %v) {{.*}} {
+// CHECK32-HARD: ret bfloat %v
+// CHECK64-HARD: define bfloat @test_ret_bf16(bfloat returned %v) {{.*}} {
+// CHECK64-HARD: ret bfloat %v
+// CHECK32-SOFTFP: define i32 @test_ret_bf16(i32 [[V0:.*]]) {{.*}} {
+// CHECK32-SOFTFP: %tmp2.0.insert.ext = and i32 [[V0]], 65535
+// CHECK32-SOFTFP: ret i32 %tmp2.0.insert.ext
+// CHECK64-SOFTFP: define bfloat @test_ret_bf16(bfloat returned %v) {{.*}} {
+// CHECK64-SOFTFP: ret bfloat %v

clang/test/CodeGen/arm-mangle-16bit-float.cpp

+// RUN: %clang_cc1 -triple aarch64-arm-none-eabi -fallow-half-arguments-and-returns -target-feature +fullfp16 -emit-llvm -o - %s | FileCheck %s --check-prefix=CHECK64
+// RUN: %clang_cc1 -triple arm-arm-none-eabi -fallow-half-arguments-and-returns -target-feature +fullfp16 -emit-llvm -o - %s | FileCheck %s --check-prefix=CHECK32
+
+// CHECK64: define {{.*}}void @_Z3barDh(half %b)
+// CHECK32: define {{.*}}void @_Z3barDh(i32 %b.coerce)
+void bar(__fp16 b) {}

clang/test/CodeGen/arm-mangle-bf16.cpp

+// RUN: %clang_cc1 -triple aarch64-arm-none-eabi -target-feature +bf16 -emit-llvm -o - %s | FileCheck %s --check-prefix=CHECK64
+// RUN: %clang_cc1 -triple arm-arm-none-eabi -target-feature +bf16 -mfloat-abi hard -emit-llvm -o - %s | FileCheck %s --check-prefix=CHECK32-HARD
+// RUN: %clang_cc1 -triple arm-arm-none-eabi -target-feature +bf16 -mfloat-abi softfp -emit-llvm -o - %s | FileCheck %s --check-prefix=CHECK32-SOFTFP
+
+// CHECK64: define {{.*}}void @_Z3foou6__bf16(bfloat %b)
+// CHECK32-HARD: define {{.*}}void @_Z3foou6__bf16(bfloat %b)
+// CHECK32-SOFTFP: define {{.*}}void @_Z3foou6__bf16(i32 %b.coerce)
+void foo(__bf16 b) {}

clang/test/Sema/arm-bf16-forbidden-ops.c

+// RUN: %clang_cc1 -fsyntax-only -verify -triple aarch64 -target-feature +bf16 %s
+
+__bf16 test_cast_from_float(float in) {
+  return (__bf16)in; // expected-error {{cannot type-cast to __bf16}}
+}
+
+__bf16 test_cast_from_float_literal(void) {
+  return (__bf16)1.0f; // expected-error {{cannot type-cast to __bf16}}
+}
+
+__bf16 test_cast_from_int(int in) {
+  return (__bf16)in; // expected-error {{cannot type-cast to __bf16}}
+}
+
+__bf16 test_cast_from_int_literal(void) {
+  return (__bf16)1; // expected-error {{cannot type-cast to __bf16}}
+}
+
+__bf16 test_cast_bfloat(__bf16 in) {
+  return (__bf16)in; // this one should work
+}
+
+float test_cast_to_float(__bf16 in) {
+  return (float)in; // expected-error {{cannot type-cast from __bf16}}
+}
+
+int test_cast_to_int(__bf16 in) {
+  return (int)in; // expected-error {{cannot type-cast from __bf16}}
+}
+
+__bf16 test_implicit_from_float(float in) {
+  return in; // expected-error {{returning 'float' from a function with incompatible result type '__bf16'}}
+}
+
+__bf16 test_implicit_from_float_literal(void) {
+  return 1.0f; // expected-error {{returning 'float' from a function with incompatible result type '__bf16'}}
+}
+
+__bf16 test_implicit_from_int(int in) {
+  return in; // expected-error {{returning 'int' from a function with incompatible result type '__bf16'}}
+}
+
+__bf16 test_implicit_from_int_literal(void) {
+  return 1; // expected-error {{returning 'int' from a function with incompatible result type '__bf16'}}
+}
+
+__bf16 test_implicit_bfloat(__bf16 in) {
+  return in; // this one should work
+}
+
+float test_implicit_to_float(__bf16 in) {
+  return in; // expected-error {{returning '__bf16' from a function with incompatible result type 'float'}}
+}
+
+int test_implicit_to_int(__bf16 in) {
+  return in; // expected-error {{returning '__bf16' from a function with incompatible result type 'int'}}
+}
+
+__bf16 test_cond(__bf16 a, __bf16 b, _Bool which) {
+  // Conditional operator _should_ be supported, without nonsense
+  // complaints like 'types __bf16 and __bf16 are not compatible'
+  return which ? a : b;
+}
+
+__bf16 test_cond_float(__bf16 a, __bf16 b, _Bool which) {
+  return which ? a : 1.0f; // expected-error {{incompatible operand types ('__bf16' and 'float')}}
+}
+
+__bf16 test_cond_int(__bf16 a, __bf16 b, _Bool which) {
+  return which ? a : 1; // expected-error {{incompatible operand types ('__bf16' and 'int')}}
+}

clang/test/Sema/arm-bf16-forbidden-ops.cpp

+// RUN: %clang_cc1 -fsyntax-only -verify -triple aarch64 -target-feature +bf16 %s
+
+__bf16 test_static_cast_from_float(float in) {
+  return static_cast<__bf16>(in); // expected-error {{static_cast from 'float' to '__bf16' is not allowed}}
+}
+
+__bf16 test_static_cast_from_float_literal(void) {
+  return static_cast<__bf16>(1.0f); // expected-error {{static_cast from 'float' to '__bf16' is not allowed}}
+}
+
+__bf16 test_static_cast_from_int(int in) {
+  return static_cast<__bf16>(in); // expected-error {{static_cast from 'int' to '__bf16' is not allowed}}
+}
+
+__bf16 test_static_cast_from_int_literal(void) {
+  return static_cast<__bf16>(1); // expected-error {{static_cast from 'int' to '__bf16' is not allowed}}
+}
+
+__bf16 test_static_cast_bfloat(__bf16 in) {
+  return static_cast<__bf16>(in); // this one should work
+}
+
+float test_static_cast_to_float(__bf16 in) {
+  return static_cast<float>(in); // expected-error {{static_cast from '__bf16' to 'float' is not allowed}}
+}
+
+int test_static_cast_to_int(__bf16 in) {
+  return static_cast<int>(in); // expected-error {{static_cast from '__bf16' to 'int' is not allowed}}
+}
+
+__bf16 test_implicit_from_float(float in) {
+  return in; // expected-error {{cannot initialize return object of type '__bf16' with an lvalue of type 'float'}}
+}
+
+__bf16 test_implicit_from_float_literal() {
+  return 1.0f; // expected-error {{cannot initialize return object of type '__bf16' with an rvalue of type 'float'}}
+}
+
+__bf16 test_implicit_from_int(int in) {
+  return in; // expected-error {{cannot initialize return object of type '__bf16' with an lvalue of type 'int'}}
+}
+
+__bf16 test_implicit_from_int_literal() {
+  return 1; // expected-error {{cannot initialize return object of type '__bf16' with an rvalue of type 'int'}}
+}
+
+__bf16 test_implicit_bfloat(__bf16 in) {
+  return in; // this one should work
+}
+
+float test_implicit_to_float(__bf16 in) {
+  return in; // expected-error {{cannot initialize return object of type 'float' with an lvalue of type '__bf16'}}
+}
+
+int test_implicit_to_int(__bf16 in) {
+  return in; // expected-error {{cannot initialize return object of type 'int' with an lvalue of type '__bf16'}}
+}
+
+__bf16 test_cond(__bf16 a, __bf16 b, bool which) {
+  // Conditional operator _should_ be supported, without nonsense
+  // complaints like 'types __bf16 and __bf16 are not compatible'
+  return which ? a : b;
+}
+
+__bf16 test_cond_float(__bf16 a, __bf16 b, bool which) {
+  return which ? a : 1.0f; // expected-error {{incompatible operand types ('__bf16' and 'float')}}
+}
+
+__bf16 test_cond_int(__bf16 a, __bf16 b, bool which) {
+  return which ? a : 1; // expected-error {{incompatible operand types ('__bf16' and 'int')}}
+}

clang/test/Sema/arm-bfloat.cpp

+// RUN: %clang_cc1 -fsyntax-only -verify -std=c++11 \
+// RUN: -triple aarch64-arm-none-eabi -target-cpu cortex-a75 \
+// RUN: -target-feature +bf16 -target-feature +neon %s
+// RUN: %clang_cc1 -fsyntax-only -verify -std=c++11 \
+// RUN: -triple arm-arm-none-eabi -target-cpu cortex-a53 \
+// RUN: -target-feature +bf16 -target-feature +neon %s
+
+void test(bool b) {
+  __bf16 bf16;
+
+  bf16 + bf16; // expected-error {{invalid operands to binary expression ('__bf16' and '__bf16')}}
+  bf16 - bf16; // expected-error {{invalid operands to binary expression ('__bf16' and '__bf16')}}
+  bf16 * bf16; // expected-error {{invalid operands to binary expression ('__bf16' and '__bf16')}}
+  bf16 / bf16; // expected-error {{invalid operands to binary expression ('__bf16' and '__bf16')}}
+
+  __fp16 fp16;
+
+  bf16 + fp16; // expected-error {{invalid operands to binary expression ('__bf16' and '__fp16')}}
+  fp16 + bf16; // expected-error {{invalid operands to binary expression ('__fp16' and '__bf16')}}
+  bf16 - fp16; // expected-error {{invalid operands to binary expression ('__bf16' and '__fp16')}}
+  fp16 - bf16; // expected-error {{invalid operands to binary expression ('__fp16' and '__bf16')}}
+  bf16 * fp16; // expected-error {{invalid operands to binary expression ('__bf16' and '__fp16')}}
+  fp16 * bf16; // expected-error {{invalid operands to binary expression ('__fp16' and '__bf16')}}
+  bf16 / fp16; // expected-error {{invalid operands to binary expression ('__bf16' and '__fp16')}}
+  fp16 / bf16; // expected-error {{invalid operands to binary expression ('__fp16' and '__bf16')}}
+  bf16 = fp16; // expected-error {{assigning to '__bf16' from incompatible type '__fp16'}}
+  fp16 = bf16; // expected-error {{assigning to '__fp16' from incompatible type '__bf16'}}
+  bf16 + (b ? fp16 : bf16); // expected-error {{incompatible operand types ('__fp16' and '__bf16')}}
+}

clang/tools/libclang/CXType.cpp

 //===- CXType.cpp - Implements 'CXTypes' aspect of libclang ---------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===--------------------------------------------------------------------===//
 //
switch (K) {
     TKIND(DependentSizedArray);
     TKIND(Vector);
     TKIND(ExtVector);
     TKIND(MemberPointer);
     TKIND(Auto);
     TKIND(Elaborated);
     TKIND(Pipe);
     TKIND(Attributed);
+    TKIND(BFloat16);
 #define IMAGE_TYPE(ImgType, Id, SingletonId, Access, Suffix) TKIND(Id);
 #include "clang/Basic/OpenCLImageTypes.def"
 #undef IMAGE_TYPE
 #define EXT_OPAQUE_TYPE(ExtTYpe, Id, Ext) TKIND(Id);
 #include "clang/Basic/OpenCLExtensionTypes.def"
     TKIND(OCLSampler);
     TKIND(OCLEvent);
     TKIND(OCLQueue);