[ARM] Add __bf16 as new Bfloat16 C Type

[stuij]

Summary

This patch upstreams support for a new storage only bfloat16 C type.
This type is used to implement primitive support for bfloat16 data, in
line with the Bfloat16 extension of the Armv8.6-a architecture, as
detailed here:

https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/posts/arm-architecture-developments-armv8-6-a

The bfloat type, and its properties are specified in the Arm Architecture
Reference Manual:

https://developer.arm.com/docs/ddi0487/latest/arm-architecture-reference-manual-armv8-for-armv8-a-architecture-profile

In detail this patch:

• introduces an opaque, storage-only C-type __bf16, which introduces a new bfloat IR type.

This is part of a patch series, starting with command-line and Bfloat16
assembly support. The subsequent patches will upstream intrinsics
support for BFloat16, followed by Matrix Multiplication and the
remaining Virtualization features of the armv8.6-a architecture.

The following people contributed to this patch:

• Luke Cheeseman
• Momchil Velikov
• Alexandros Lamprineas
• Luke Geeson
• Simon Tatham
• Ties Stuij

[craig.topper]

clang/lib/Basic/Targets/AArch64.cpp

74

Doesn't Bfloat16 have a different number of mantissa and exponent bits than IEEEhalf?

clang/test/CodeGen/arm-mangle-16bit-float.cpp

4 ↗

(On Diff #249965)

How can bfloat16 be passed as half? Don't they have a different format?

[rjmccall]

I don't understand why you wouldn't add a new IR type for this; doing so should be totally mechanical.

[rjmccall]

Note that we have an IR type for the PPC double-double format, which isn't even hardware-supported. This is literally just an IEEE floating-point format with non-standard parameters, right?

[LukeGeeson]

In D76077#1919897, @rjmccall wrote:

Note that we have an IR type for the PPC double-double format, which isn't even hardware-supported. This is literally just an IEEE floating-point format with non-standard parameters, right?

Indeed yeah, the details are laid out in a blog post here: https://community.arm.com/developer/ip-products/processors/b/ml-ip-blog/posts/bfloat16-processing-for-neural-networks-on-armv8_2d00_a

[LukeGeeson]

In D76077#1919861, @rjmccall wrote:

I don't understand why you wouldn't add a new IR type for this; doing so should be totally mechanical.

We had a few reasons for doing it this way.

By adding a new IR type we would need to consider calling conventions, and IR optimizations for what is essentially an opaque storage-only type.

Bfloat has no soft ABI, and all the supported operations can be handled through intrinsics (in a later patch, removed here due to bloat). If we were to add a new IR type, then we would need to handle many operations which would extend beyond what the type is supported for. For instance in GCC we had to add a new mode in RTL to handle inline memcopy.

[LukeGeeson]

clang/test/CodeGen/arm-mangle-16bit-float.cpp

4 ↗

(On Diff #249965)

see the above comment about soft-abis

[rjmccall]

In D76077#1921490, @LukeGeeson wrote:

In D76077#1919861, @rjmccall wrote:

I don't understand why you wouldn't add a new IR type for this; doing so should be totally mechanical.

We had a few reasons for doing it this way.

By adding a new IR type we would need to consider calling conventions, and IR optimizations for what is essentially an opaque storage-only type.

IR optimizations should just fall out; the code in APFloat should work for arbitrary FP semantics.

Calling something a "storage-only" type does not get you out of worrying about calling conventions at the ABI level. You can forbid passing your type as an argument or result directly, but structs containing your type as a field can still be passed around, and the behavior for that needs to be defined.

Bfloat has no soft ABI, and all the supported operations can be handled through intrinsics (in a later patch, removed here due to bloat). If we were to add a new IR type, then we would need to handle many operations which would extend beyond what the type is supported for. For instance in GCC we had to add a new mode in RTL to handle inline memcopy.

I don't know why having a soft ABI makes a difference. If the type only works on certain platforms, then it can't be used on other platforms.

[LukeGeeson]

Added a file from a downstream cleanup of the branch.

[jfb]

In D76077#1921973, @rjmccall wrote:

In D76077#1921490, @LukeGeeson wrote:

In D76077#1919861, @rjmccall wrote:

I don't understand why you wouldn't add a new IR type for this; doing so should be totally mechanical.

We had a few reasons for doing it this way.

By adding a new IR type we would need to consider calling conventions, and IR optimizations for what is essentially an opaque storage-only type.

IR optimizations should just fall out; the code in APFloat should work for arbitrary FP semantics.

Calling something a "storage-only" type does not get you out of worrying about calling conventions at the ABI level. You can forbid passing your type as an argument or result directly, but structs containing your type as a field can still be passed around, and the behavior for that needs to be defined.

Bfloat has no soft ABI, and all the supported operations can be handled through intrinsics (in a later patch, removed here due to bloat). If we were to add a new IR type, then we would need to handle many operations which would extend beyond what the type is supported for. For instance in GCC we had to add a new mode in RTL to handle inline memcopy.

I don't know why having a soft ABI makes a difference. If the type only works on certain platforms, then it can't be used on other platforms.

Having an IR type sounds like the right thing to do here.

[stuij]

Thanks for the feedback. We'll work on implementing a bfloat IR type.

[SjoerdMeijer]

That sounds good and that seems to address the feedback given here, which I agree with. Just wanted to briefly add that while it already looks like most interested people are on this ticket, perhaps it good to also share the plan and design on the cfe dev list for visibility.

[stuij]

commandeering as I've got some changes to push

[stuij]

This comment was removed by stuij.

[stuij]

changes: depend on bfloat IR type, and resolve some conflicts that crept in

[stuij]

I've amended this patch to conform to the general Bfloat -> BFloat convention
changes requested in the dependent BFloat IR patch (D78190).

I also folded in two other downstream patches, which allow for the ?: operator
on bf16 values and prohibit conversions between integer and __bf16.

[asmith]

really happy for native bfloat support!

clang/include/clang/AST/ASTContext.h

967

Maybe the comment is unnecessary. I can imagine other targets besides ARM NEON using BFloat

clang/include/clang/AST/Type.h

2011

don't think this comment adds anything

clang/lib/AST/ASTContext.cpp

5995

extra spaces

clang/lib/AST/ItaniumMangle.cpp

3188

Add a space before break and on the line above.

[stuij]

@asmith: Thanks for reviewing :) Fixed your suggestions in the coming revision.

clang/include/clang/AST/ASTContext.h

967

fair

clang/include/clang/AST/Type.h

2011

fair

[stuij]

changes:

• addressed asmith's comments
• Bfloat doesn't depend on half cmdline options and correct code is generation for AArch32 hard and softfp CC

[LukeGeeson]

I authored some of the code for this patch, please update the commit message with my name on the list :)

[majnemer]

clang/include/clang-c/Index.h

3258

Should this be:

CXType_LastBuiltin = CXType_BFloat16,

clang/lib/AST/ItaniumMangle.cpp

3188

Why is this x1?

[majnemer]

clang/lib/Sema/SemaOverload.cpp

1873–1874

This probably needs an explanation.

[SjoerdMeijer]

clang/docs/LanguageExtensions.rst

520

Not my field of expertise, and sorry if I've missed this somewhere, but was wondering if this (i.e. all language modes) means we need C++ name mangling support for bfloat? In the AArch64 backend I saw this:

getBFloat16Mangling() const override { return "u6__bf16"; }

But that's something else I guess, and I was guessing a 2 letter mangling needs to be added here?

https://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangling-builtin

[stuij]

@LukeGeeson: you're already mentioned :) See the commits tab on this review. But Simon Tatham needs a mention as well. I shall add him.

clang/docs/LanguageExtensions.rst

520

Yes, we need name-mangling support, and yes that method you mentioned does that. There's one for AArch32 as well. And yes we have documented it in the 'C++ ABI for the latest Arm Architecture' docs:

aarch32: https://developer.arm.com/docs/ihi0041/latest
aarch64: https://developer.arm.com/docs/ihi0059/latest

clang/include/clang-c/Index.h

3258

thanks

clang/lib/AST/ItaniumMangle.cpp

3188

Yes, that does look odd. The original author of this code has left the company, but I'll ask around.

clang/lib/Sema/SemaOverload.cpp

1873–1874

done

[stuij]

addressing review comments and adding Simon Tatham to contributers

[stuij]

redo: addressing review comments and adding Simon Tatham to contributers

[SjoerdMeijer]

clang/docs/LanguageExtensions.rst

520

But does that mean that for other targets this is not (yet) supported?

[stuij]

clang/docs/LanguageExtensions.rst

520

Currently bfloat is a vendor-extended type, hence the 'u' prefix. So to me it would make sense that backends should implement their own naming, as there's no guarantee that the naming scheme chosen for Arm is compatible with all backends.

[SjoerdMeijer]

clang/docs/LanguageExtensions.rst

520

Ok, true, fair enough

[SjoerdMeijer]

clang/include/clang/Basic/DiagnosticSemaKinds.td

8124

Nit: was wondering if err_cast_to_bfloat16 would be more consistent

clang/lib/AST/ASTContext.cpp

6010

nit: perhaps this error message is not entirely accurate for bfloat16?

clang/lib/Basic/Targets/AArch64.cpp

74

Nit: we use bfloat16 everywhere, would llvm::APFloat::BFloat16() be better for consistency?

clang/lib/CodeGen/CodeGenModule.cpp

115

nit: perhaps getBFloat16Ty()

clang/lib/CodeGen/CodeGenTypeCache.h

39

And so here too, BFloat16Ty?

clang/lib/CodeGen/TargetInfo.cpp

5987

Nit: to distinguish the unfortunate float ABI names, I was wondering whether IsFloatABISoftFP is clearer, or something along those lines, just to make clear it is not the "soft" but the "softfp" variant

[SjoerdMeijer]

There are Sema and CodeGen tests, but was wondering if there would be some value in having an AST test too? There are some other types that do have AST tests.

clang/lib/AST/ASTContext.cpp

2054

Is a break missing here?

clang/lib/CodeGen/CodeGenTypes.cpp

307

Is this covered in tests?

clang/lib/CodeGen/TargetInfo.cpp

6599

nit: camelcase fase for haslegalhalf

clang/test/CodeGen/arm-mangle-16bit-float.cpp

4 ↗

(On Diff #249965)

Probably I am bit confused too now... are the three possible types that we are expecing not bfloat, i32, or i16?

2 ↗

(On Diff #264562)

Do you need to pass +fullfp16?

[LukeGeeson]

clang/test/CodeGen/arm-mangle-16bit-float.cpp

4 ↗

(On Diff #249965)

Hi Sjoerd, Good spot here I forgot to add a default case somewhere, which means AArch32 didn't observe -mfloat-abi softfp by default - and hence used bfloat where i32 was expected.

I have a local patch for this and will send to Ties to merge into this one :)

We're not sure what you mean by i16 however?

[stuij]

no explicit float-abi cmdline arg should default to softfp

[stuij]

addressed review comments and some related general changes:

• renamed IsSoftFloatABI -> IsFloatABISoftFP
• split bfloat tests out of arm-mangle-16bit-float.cpp
• bfloat shouldn't be checking for NativeHalfType
• err_cast_(from|to)_bfloat -> err_cast_(from|to)_bfloat16
• reworded comments and other minor fixes

[stuij]

clang/include/clang/Basic/DiagnosticSemaKinds.td

8124

agree

clang/lib/AST/ASTContext.cpp

2054

tnx

6010

tnx

clang/lib/AST/ItaniumMangle.cpp

3188

I'm not sure it's 100% wrong, but it's not consistent with GCC. Changing to bfloat16_t.

clang/lib/Basic/Targets/AArch64.cpp

74

In the IR world bfloat is consistently called bfloat, without the 16. I think this might turn into bikeshedding to justify if either one or both sides of the divide should or should not include the '16'. I can think of arguments to support the various options.

As I think there's no clear winner, I'd like to take the route of less effort and keep things as is.

clang/lib/CodeGen/CodeGenModule.cpp

115

see previous C - IR divide comment

clang/lib/CodeGen/CodeGenTypeCache.h

39

see previous C - IR divide comment

clang/lib/CodeGen/CodeGenTypes.cpp

307

Thanks. That code doesn't make sense to me. We are not expecting to emulate bfloat operations, and we shouldn't conflate bfloat with half. I'll remove the UseNativeHalf clause, and the one call-site of this fn.

clang/lib/CodeGen/TargetInfo.cpp

5987

Yes, good idea. Done.

clang/test/CodeGen/arm-mangle-16bit-float.cpp

2 ↗

(On Diff #264562)

I'm splitting out bf16 and fp16 tests. I don't think we should be testing with the union of cmdline options to accommodate testing multiple types in one file.

[SjoerdMeijer]

As I wrote in one of the inlined comments, I am relatively unhappy that we have both bfloat and bfloat16 as names. But that looks like a complain for another patch, and not really this one.

clang/lib/Basic/Targets/AArch64.cpp

74

ah, I couldn't remember on which patch I commented on the bfloat16 vs bfloat naming.
Since everything is called bfloat16, from the architecture extension to the source language type, I find this a bit of a missed opportunity to get the names consistent. And I wouldn't say that naming of types is bikeshedding. But maybe that's just me...

clang/lib/Basic/Targets/ARM.cpp

80

Is this tested? Can it be tested?

clang/lib/CodeGen/CGBuiltin.cpp

4484

nit: the linter says some spaces here.

5517–5518

nit: looks like the hinter right to be unhappy about the formatting

clang/lib/CodeGen/TargetInfo.cpp

6279

nit: ir -> IR

6480

nit: perhaps add TODO, or just remove this.

clang/lib/Sema/SemaOverload.cpp

1874

Nit: conversions -> Conversions, permitted -> permitted.

Same on line no. 1895.

clang/test/CodeGen/arm-bf16-params-returns.c

6

what happens with -mfloat-abi=soft. Does that deserve a test?

clang/test/CodeGen/arm-mangle-16bit-float.cpp

4 ↗

(On Diff #249965)

Ah, as I said, got confused. Now I see we use i16 for targets that don't support bfloat16. So we probably need a test for that.

clang/test/CodeGen/arm-mangle-bf16.cpp

4

nit: if the name mangling is Independent of the float ABI, then you might as well one runline and the abi option.

[SjoerdMeijer]

clang/test/CodeGen/arm-mangle-bf16.cpp

4

-> ... then you might as well have only one runline and remove the abi option.

[stuij]

clang/lib/Basic/Targets/AArch64.cpp

74

In general naming isn't bikeshedding. And I appreciate discussions on naming. I'm just afraid it might turn into bikeshedding. I do agree that such a remark isn't a very constructive to a conversation. Sorry.

clang/test/CodeGen/arm-bf16-params-returns.c

6

Yes, this one is interesting. I think we shouldn't support bfloat at all in combination with -mfloat-abi=soft. We don't support software emulation of bfloat instructions and all operations on bfloat are simd instructions.

It turns out cc1 will accept -mfloat-abi=soft with neon intrinsics, which will happily churn out neon instructions. This doesn't sound very soft. The driver will ignore -mfloat-abi=soft in certain combinations of cmdline instructions, but I haven't delved deep enough to know what's what.

GCC doesn't allow soft+neon combination. Unfortunately it will actually crash for just a bfloat type by itself, which is quite useless without intrinsics. The Arm GCC folks will raise a ticket on this with as proposed solution to not allow this combination.

As this issue seems bigger than just bfloat, and potentially there's driver code involved as well I thought it'd make sense to handle this in a separate patch.

clang/test/CodeGen/arm-mangle-16bit-float.cpp

4 ↗

(On Diff #249965)

As I wrote above, I think we currently should not allow it as long as there are no targets that support bfloat and have use for this combination.

clang/test/CodeGen/arm-mangle-bf16.cpp

4

yes, I thought it's better to be defensive here, in case the implementation changes.

[SjoerdMeijer]

clang/test/CodeGen/arm-bf16-params-returns.c

6

I think we first need agreement what -mfloat-abi=soft with bf16 means and how it should behave, document this, and have some tests. Possibly document how we diverge from this.

I think I tend to disagree with this:

I think we shouldn't support bfloat at all in combination with -mfloat-abi=soft.

why would it not be supported in some way (promotions to another type, or even library calls), like the other float-types?

It turns out cc1 will accept -mfloat-abi=soft with neon intrinsics, which will happily churn out neon instructions.

I would say the fact that there are other problems, shouldn't distract us too much from trying to get this right; I think at this point that is not yet a justification.

As this issue seems bigger than just bfloat, and potentially there's driver code involved as well I thought it'd make sense to handle this in a separate patch.

I think at this point I disagree with this, mainly because of my first point: the behaviour should be specified. I would also say that not doing this, could be a bit of bad precedent for adding a new C type.

[labrinea]

clang/lib/CodeGen/CGBuiltin.cpp

4495

This is not what we should be checking for here. Imagine a command line with +bf16 and +mfloat-abi=softfp, that should generate i16, not bfloat. We therefore need a target hook to pass this information. I suggest allowBFloatArgsAndRet() in ABIInfo, returning false by default, overloaded for Arm and AArch64 to return !IsFloatABISoftFP && hasBFloat16Type() and hasBFloat16Type() respectively.

[stuij]

clang/lib/Basic/Targets/ARM.cpp

80

Yes, I think it's a bit tricky to test this all by itself. Right now this is used when return a TypeInfo, which is used in all kinds of different contexts. I think alignment is better tested where it is eventually expressed.

[SjoerdMeijer]

Can you summarise where we are? I.e.,

• float-abi=soft doesn't work. But what is the problem? Are we not simply passing i16s, is that not what we are supposed to do?

Can you also update the description of this patch, I got totally confused by:

• "introduces an opaque, storage-only C-type __bf16, which does not introduce a new LLVM IR type, but maps it to either i16 or half type."

clang/test/CodeGen/arm-mangle-16bit-float.cpp

1 ↗

(On Diff #265985)

This is testing name mangling of __fp16, so is not related to this patch, please commit this separately if this is not tested yet. But I do suspect Dh is tested already, but I could be wrong.

[stuij]

In D76077#2066323, @SjoerdMeijer wrote:

Can you summarise where we are? I.e.,

• float-abi=soft doesn't work. But what is the problem? Are we not simply passing i16s, is that not what we are supposed to do?

At the moment when going through the GCC compatibility driver (standard interface), we get __bf16 is not supported on this target.
When using -cc1, we can pass -mfloat-abi=soft, we can compile source-code that contains __bf16, and we can also compile neon intrinsics in general. It looks like we just ignore soft was passed and instead we use the hard float-abi. For intrinsics we should error.

In general we pass i32's btw, not i16's. Also for __fp16.

Can you also update the description of this patch, I got totally confused by:

• "introduces an opaque, storage-only C-type __bf16, which does not introduce a new LLVM IR type, but maps it to either i16 or half type."

willdo

[SjoerdMeijer]

At the moment when going through the GCC compatibility driver (standard interface), we get __bf16 is not supported on this target.

If this is the current behaviour, and consistent with GCC, that sounds reasonable. Probably best to be explicit about this and document this in LangRef. With a note that this is subject to change?
Oh, and I guess that means we can add a test this for this? We can check this error message?

[stuij]

In D76077#2066520, @SjoerdMeijer wrote:

At the moment when going through the GCC compatibility driver (standard interface), we get __bf16 is not supported on this target.

If this is the current behaviour, and consistent with GCC, that sounds reasonable.

Unfortunately, this is not consistent with GCC. GCC will happily compile.

[stuij]

Addressed review comments. Notably what to do in combination with -mfloat-abi=soft.

[SjoerdMeijer]

Thanks, LGTM

[labrinea]

clang/lib/CodeGen/CGBuiltin.cpp

6989

shouldn't this be getTargetHooks().getABIInfo().allowBFloatArgsAndRet() ?

[stuij]

clang/lib/CodeGen/CGBuiltin.cpp

6989

thanks, done

[stuij]

Resolved merge conflicts with head. Fixed minor oversight.

clang/docs/LanguageExtensions.rst

.. code-block:: c++
   m4x4_t f(m4x4_t a, m4x4_t b, m4x4_t c) {
     return a + b * c;
   }
 
 
 Half-Precision Floating Point
 =============================
 
-Clang supports two half-precision (16-bit) floating point types: ``__fp16`` and
-``_Float16``.  These types are supported in all language modes.
+Clang supports three half-precision (16-bit) floating point types: ``__fp16``,
+``_Float16`` and ``__bf16``.  These types are supported in all language modes.
 
 ``__fp16`` is supported on every target, as it is purely a storage format; see below.
 ``_Float16`` is currently only supported on the following targets, with further
 targets pending ABI standardization:
 
 * 32-bit ARM
 * 64-bit ARM (AArch64)
 * SPIR
 
 ``_Float16`` will be supported on more targets as they define ABIs for it.
 
+``__bf16`` is purely a storage format; it is currently only supported on the following targets:
+* 32-bit ARM
+* 64-bit ARM (AArch64)
+
+The ``__bf16`` type is only available when supported in hardware.
+
 ``__fp16`` is a storage and interchange format only.  This means that values of
 ``__fp16`` are immediately promoted to (at least) ``float`` when used in arithmetic
 operations, so that e.g. the result of adding two ``__fp16`` values has type ``float``.
 The behavior of ``__fp16`` is specified by the ARM C Language Extensions (`ACLE <http://infocenter.arm.com/help/topic/com.arm.doc.ihi0053d/IHI0053D_acle_2_1.pdf>`_).
 Clang uses the ``binary16`` format from IEEE 754-2008 for ``__fp16``, not the ARM
 alternative format.
 
 ``_Float16`` is an extended floating-point type.  This means that, just like arithmetic on

clang/include/clang-c/Index.h

enum CXTypeKind {
   CXType_Half = 31,
   CXType_Float16 = 32,
   CXType_ShortAccum = 33,
   CXType_Accum = 34,
   CXType_LongAccum = 35,
   CXType_UShortAccum = 36,
   CXType_UAccum = 37,
   CXType_ULongAccum = 38,
+  CXType_BFloat16 = 39,
   CXType_FirstBuiltin = CXType_Void,
-  CXType_LastBuiltin = CXType_ULongAccum,
+  CXType_LastBuiltin = CXType_BFloat16,
 
   CXType_Complex = 100,
   CXType_Pointer = 101,
   CXType_BlockPointer = 102,
   CXType_LValueReference = 103,
   CXType_RValueReference = 104,
   CXType_Record = 105,
   CXType_Enum = 106,

clang/include/clang/AST/ASTContext.h

public:
   CanQualType UnsignedShortFractTy, UnsignedFractTy, UnsignedLongFractTy;
   CanQualType SatShortAccumTy, SatAccumTy, SatLongAccumTy;
   CanQualType SatUnsignedShortAccumTy, SatUnsignedAccumTy,
       SatUnsignedLongAccumTy;
   CanQualType SatShortFractTy, SatFractTy, SatLongFractTy;
   CanQualType SatUnsignedShortFractTy, SatUnsignedFractTy,
       SatUnsignedLongFractTy;
   CanQualType HalfTy; // [OpenCL 6.1.1.1], ARM NEON
+  CanQualType BFloat16Ty;
   CanQualType Float16Ty; // C11 extension ISO/IEC TS 18661-3
   CanQualType FloatComplexTy, DoubleComplexTy, LongDoubleComplexTy;
   CanQualType Float128ComplexTy;
   CanQualType VoidPtrTy, NullPtrTy;
   CanQualType DependentTy, OverloadTy, BoundMemberTy, UnknownAnyTy;
   CanQualType BuiltinFnTy;
   CanQualType PseudoObjectTy, ARCUnbridgedCastTy;
   CanQualType ObjCBuiltinIdTy, ObjCBuiltinClassTy, ObjCBuiltinSelTy;

clang/include/clang/AST/BuiltinTypes.def

 FLOATING_TYPE(Double, DoubleTy)
 
 // 'long double'
 FLOATING_TYPE(LongDouble, LongDoubleTy)
 
 // '_Float16'
 FLOATING_TYPE(Float16, HalfTy)
 
+// '__bf16'
+FLOATING_TYPE(BFloat16, BFloat16Ty)
+
 // '__float128'
 FLOATING_TYPE(Float128, Float128Ty)
 
 //===- Language-specific types --------------------------------------------===//
 
 // This is the type of C++0x 'nullptr'.
 BUILTIN_TYPE(NullPtr, NullPtrTy)
 

clang/include/clang/AST/Type.h

public:
   bool isRealFloatingType() const; // C99 6.2.5p10 (float, double, long double)
   /// isComplexType() does *not* include complex integers (a GCC extension).
   /// isComplexIntegerType() can be used to test for complex integers.
   bool isComplexType() const;      // C99 6.2.5p11 (complex)
   bool isAnyComplexType() const;   // C99 6.2.5p11 (complex) + Complex Int.
   bool isFloatingType() const;     // C99 6.2.5p11 (real floating + complex)
   bool isHalfType() const;         // OpenCL 6.1.1.1, NEON (IEEE 754-2008 half)
   bool isFloat16Type() const;      // C11 extension ISO/IEC TS 18661
+  bool isBFloat16Type() const;
   bool isFloat128Type() const;
   bool isRealType() const;         // C99 6.2.5p17 (real floating + integer)
   bool isArithmeticType() const;   // C99 6.2.5p18 (integer + floating)
   bool isVoidType() const;         // C99 6.2.5p19
   bool isScalarType() const;       // C99 6.2.5p21 (arithmetic + pointers)
   bool isAggregateType() const;
   bool isFundamentalType() const;
   bool isCompoundType() const;
inline bool Type::isHalfType() const {
   // FIXME: Should we allow complex __fp16? Probably not.
   return isSpecificBuiltinType(BuiltinType::Half);
 }
 
 inline bool Type::isFloat16Type() const {
   return isSpecificBuiltinType(BuiltinType::Float16);
 }
 
+inline bool Type::isBFloat16Type() const {
+  return isSpecificBuiltinType(BuiltinType::BFloat16);
+}
+
 inline bool Type::isFloat128Type() const {
   return isSpecificBuiltinType(BuiltinType::Float128);
 }
 
 inline bool Type::isNullPtrType() const {
   return isSpecificBuiltinType(BuiltinType::NullPtr);
 }
 

clang/include/clang/Basic/DiagnosticSemaKinds.td

def warn_cast_calling_conv : Warning<
   InGroup<DiagGroup<"cast-calling-convention">>;
 def note_change_calling_conv_fixit : Note<
   "consider defining %0 with the '%1' calling convention">;
 def warn_bad_function_cast : Warning<
   "cast from function call of type %0 to non-matching type %1">,
   InGroup<BadFunctionCast>, DefaultIgnore;
 def err_cast_pointer_to_non_pointer_int : Error<
   "pointer cannot be cast to type %0">;
+def err_cast_to_bfloat16 : Error<"cannot type-cast to __bf16">;
+def err_cast_from_bfloat16 : Error<"cannot type-cast from __bf16">;
 def err_typecheck_expect_scalar_operand : Error<
   "operand of type %0 where arithmetic or pointer type is required">;
 def err_typecheck_cond_incompatible_operands : Error<
   "incompatible operand types%diff{ ($ and $)|}0,1">;
 def err_cast_selector_expr : Error<
   "cannot type cast @selector expression">;
 def ext_typecheck_cond_incompatible_pointers : ExtWarn<
   "pointer type mismatch%diff{ ($ and $)|}0,1">,

clang/include/clang/Basic/Specifiers.h

enum TypeSpecifierType {
     TST_char32,       // C++11 char32_t
     TST_int,
     TST_int128,
     TST_extint,       // Extended Int types.
     TST_half,         // OpenCL half, ARM NEON __fp16
     TST_Float16,      // C11 extension ISO/IEC TS 18661-3
     TST_Accum,        // ISO/IEC JTC1 SC22 WG14 N1169 Extension
     TST_Fract,
+    TST_BFloat16,
     TST_float,
     TST_double,
     TST_float128,
     TST_bool,         // _Bool
     TST_decimal32,    // _Decimal32
     TST_decimal64,    // _Decimal64
     TST_decimal128,   // _Decimal128
     TST_enum,

clang/include/clang/Basic/TargetBuiltins.h

enum EltType {
       Int32,
       Int64,
       Poly8,
       Poly16,
       Poly64,
       Poly128,
       Float16,
       Float32,
-      Float64
+      Float64,
+      BFloat16
     };
 
     NeonTypeFlags(unsigned F) : Flags(F) {}
     NeonTypeFlags(EltType ET, bool IsUnsigned, bool IsQuad) : Flags(ET) {
       if (IsUnsigned)
         Flags |= UnsignedFlag;
       if (IsQuad)
         Flags |= QuadFlag;

clang/include/clang/Basic/TargetInfo.h

 /// Fields controlling how types are laid out in memory; these may need to
 /// be copied for targets like AMDGPU that base their ABIs on an auxiliary
 /// CPU target.
 struct TransferrableTargetInfo {
   unsigned char PointerWidth, PointerAlign;
   unsigned char BoolWidth, BoolAlign;
   unsigned char IntWidth, IntAlign;
   unsigned char HalfWidth, HalfAlign;
+  unsigned char BFloat16Width, BFloat16Align;
   unsigned char FloatWidth, FloatAlign;
   unsigned char DoubleWidth, DoubleAlign;
   unsigned char LongDoubleWidth, LongDoubleAlign, Float128Align;
   unsigned char LargeArrayMinWidth, LargeArrayAlign;
   unsigned char LongWidth, LongAlign;
   unsigned char LongLongWidth, LongLongAlign;
 
   // Fixed point bit widths
struct TransferrableTargetInfo {
   unsigned char SuitableAlign;
   unsigned char DefaultAlignForAttributeAligned;
   unsigned char MinGlobalAlign;
 
   unsigned short NewAlign;
   unsigned short MaxVectorAlign;
   unsigned short MaxTLSAlign;
 
-  const llvm::fltSemantics *HalfFormat, *FloatFormat, *DoubleFormat,
-    *LongDoubleFormat, *Float128Format;
+  const llvm::fltSemantics *HalfFormat, *BFloat16Format, *FloatFormat,
+    *DoubleFormat, *LongDoubleFormat, *Float128Format;
 
   ///===---- Target Data Type Query Methods -------------------------------===//
   enum IntType {
     NoInt = 0,
     SignedChar,
     UnsignedChar,
     SignedShort,
     UnsignedShort,
protected:
   bool BigEndian;
   bool TLSSupported;
   bool VLASupported;
   bool NoAsmVariants;  // True if {|} are normal characters.
   bool HasLegalHalfType; // True if the backend supports operations on the half
                          // LLVM IR type.
   bool HasFloat128;
   bool HasFloat16;
+  bool HasBFloat16;
 
   unsigned char MaxAtomicPromoteWidth, MaxAtomicInlineWidth;
   unsigned short SimdDefaultAlign;
   std::unique_ptr<llvm::DataLayout> DataLayout;
   const char *MCountName;
   unsigned char RegParmMax, SSERegParmMax;
   TargetCXXABI TheCXXABI;
   const LangASMap *AddrSpaceMap;
public:
   virtual bool hasLegalHalfType() const { return HasLegalHalfType; }
 
   /// Determine whether the __float128 type is supported on this target.
   virtual bool hasFloat128Type() const { return HasFloat128; }
 
   /// Determine whether the _Float16 type is supported on this target.
   virtual bool hasFloat16Type() const { return HasFloat16; }
 
+  /// Determine whether the _BFloat16 type is supported on this target.
+  virtual bool hasBFloat16Type() const { return HasBFloat16; }
+
   /// Return the alignment that is suitable for storing any
   /// object with a fundamental alignment requirement.
   unsigned getSuitableAlign() const { return SuitableAlign; }
 
   /// Return the default alignment for __attribute__((aligned)) on
   /// this target, to be used if no alignment value is specified.
   unsigned getDefaultAlignForAttributeAligned() const {
     return DefaultAlignForAttributeAligned;
public:
   unsigned getHalfAlign() const { return HalfAlign; }
   const llvm::fltSemantics &getHalfFormat() const { return *HalfFormat; }
 
   /// getFloatWidth/Align/Format - Return the size/align/format of 'float'.
   unsigned getFloatWidth() const { return FloatWidth; }
   unsigned getFloatAlign() const { return FloatAlign; }
   const llvm::fltSemantics &getFloatFormat() const { return *FloatFormat; }
 
+  /// getBFloat16Width/Align/Format - Return the size/align/format of '__bf16'.
+  unsigned getBFloat16Width() const { return BFloat16Width; }
+  unsigned getBFloat16Align() const { return BFloat16Align; }
+  const llvm::fltSemantics &getBFloat16Format() const { return *BFloat16Format; }
+
   /// getDoubleWidth/Align/Format - Return the size/align/format of 'double'.
   unsigned getDoubleWidth() const { return DoubleWidth; }
   unsigned getDoubleAlign() const { return DoubleAlign; }
   const llvm::fltSemantics &getDoubleFormat() const { return *DoubleFormat; }
 
   /// getLongDoubleWidth/Align/Format - Return the size/align/format of 'long
   /// double'.
   unsigned getLongDoubleWidth() const { return LongDoubleWidth; }
public:
   }
 
   /// Return the mangled code of long double.
   virtual const char *getLongDoubleMangling() const { return "e"; }
 
   /// Return the mangled code of __float128.
   virtual const char *getFloat128Mangling() const { return "g"; }
 
+  /// Return the mangled code of bfloat.
+  virtual const char *getBFloat16Mangling() const {
+    llvm_unreachable("bfloat not implemented on this target");
+  }
+
   /// Return the value for the C99 FLT_EVAL_METHOD macro.
   virtual unsigned getFloatEvalMethod() const { return 0; }
 
   // getLargeArrayMinWidth/Align - Return the minimum array size that is
   // 'large' and its alignment.
   unsigned getLargeArrayMinWidth() const { return LargeArrayMinWidth; }
   unsigned getLargeArrayAlign() const { return LargeArrayAlign; }
 

clang/include/clang/Basic/TokenKinds.def

 
 // Altivec Extension.
 KEYWORD(__vector                    , KEYALTIVEC|KEYZVECTOR)
 KEYWORD(__pixel                     , KEYALTIVEC)
 KEYWORD(__bool                      , KEYALTIVEC|KEYZVECTOR)
 
 // ARM NEON extensions.
 ALIAS("__fp16", half                , KEYALL)
+KEYWORD(__bf16                      , KEYALL)
 
 // OpenCL Extension.
 KEYWORD(half                        , HALFSUPPORT)
 
 // Objective-C ARC keywords.
 KEYWORD(__bridge                     , KEYOBJC)
 KEYWORD(__bridge_transfer            , KEYOBJC)
 KEYWORD(__bridge_retained            , KEYOBJC)

clang/include/clang/Sema/DeclSpec.h

public:
   static const TST TST_wchar = clang::TST_wchar;
   static const TST TST_char8 = clang::TST_char8;
   static const TST TST_char16 = clang::TST_char16;
   static const TST TST_char32 = clang::TST_char32;
   static const TST TST_int = clang::TST_int;
   static const TST TST_int128 = clang::TST_int128;
   static const TST TST_extint = clang::TST_extint;
   static const TST TST_half = clang::TST_half;
+  static const TST TST_BFloat16 = clang::TST_BFloat16;
   static const TST TST_float = clang::TST_float;
   static const TST TST_double = clang::TST_double;
   static const TST TST_float16 = clang::TST_Float16;
   static const TST TST_accum = clang::TST_Accum;
   static const TST TST_fract = clang::TST_Fract;
   static const TST TST_float128 = clang::TST_float128;
   static const TST TST_bool = clang::TST_bool;
   static const TST TST_decimal32 = clang::TST_decimal32;

clang/include/clang/Serialization/ASTBitCodes.h

enum PredefinedTypeIDs {
       PREDEF_TYPE_OMP_ARRAY_SHAPING = 70,
 
       /// The placeholder type for OpenMP iterator expression.
       PREDEF_TYPE_OMP_ITERATOR = 71,
 
       /// A placeholder type for incomplete matrix index operations.
       PREDEF_TYPE_INCOMPLETE_MATRIX_IDX = 72,
 
+      /// \brief The '__bf16' type
+      PREDEF_TYPE_BFLOAT16_ID = 73,
+
       /// OpenCL image types with auto numeration
 #define IMAGE_TYPE(ImgType, Id, SingletonId, Access, Suffix) \
       PREDEF_TYPE_##Id##_ID,
 #include "clang/Basic/OpenCLImageTypes.def"
       /// \brief OpenCL extension types with auto numeration
 #define EXT_OPAQUE_TYPE(ExtType, Id, Ext) \
       PREDEF_TYPE_##Id##_ID,
 #include "clang/Basic/OpenCLExtensionTypes.def"
       // \brief SVE types with auto numeration

clang/lib/AST/ASTContext.cpp

 #include <memory>
 #include <string>
 #include <tuple>
 #include <utility>
 
 using namespace clang;
 
 enum FloatingRank {
-  Float16Rank, HalfRank, FloatRank, DoubleRank, LongDoubleRank, Float128Rank
+  BFloat16Rank, Float16Rank, HalfRank, FloatRank, DoubleRank, LongDoubleRank, Float128Rank
 };
 
 /// \returns location that is relevant when searching for Doc comments related
 /// to \p D.
 static SourceLocation getDeclLocForCommentSearch(const Decl *D,
                                                  SourceManager &SourceMgr) {
   assert(D);
 
#include "clang/Basic/AArch64SVEACLETypes.def"
   }
 
   // nullptr type (C++0x 2.14.7)
   InitBuiltinType(NullPtrTy,           BuiltinType::NullPtr);
 
   // half type (OpenCL 6.1.1.1) / ARM NEON __fp16
   InitBuiltinType(HalfTy, BuiltinType::Half);
 
+  InitBuiltinType(BFloat16Ty, BuiltinType::BFloat16);
+
   // Builtin type used to help define __builtin_va_list.
   VaListTagDecl = nullptr;
 
   // MSVC predeclares struct _GUID, and we need it to create MSGuidDecls.
   if (LangOpts.MicrosoftExt || LangOpts.Borland) {
     MSGuidTagDecl = buildImplicitRecord("_GUID");
     TUDecl->addDecl(MSGuidTagDecl);
   }

 //===----------------------------------------------------------------------===//
 
 /// getFloatTypeSemantics - Return the APFloat 'semantics' for the specified
 /// scalar floating point type.
 const llvm::fltSemantics &ASTContext::getFloatTypeSemantics(QualType T) const {
   switch (T->castAs<BuiltinType>()->getKind()) {
   default:
     llvm_unreachable("Not a floating point type!");
+  case BuiltinType::BFloat16:
+    return Target->getBFloat16Format();
   case BuiltinType::Float16:
   case BuiltinType::Half:
     return Target->getHalfFormat();
   case BuiltinType::Float:      return Target->getFloatFormat();
   case BuiltinType::Double:     return Target->getDoubleFormat();
   case BuiltinType::LongDouble:
     if (getLangOpts().OpenMP && getLangOpts().OpenMPIsDevice)
       return AuxTarget->getLongDoubleFormat();
case BuiltinType::SatUFract:
       break;
     case BuiltinType::LongFract:
     case BuiltinType::ULongFract:
     case BuiltinType::SatLongFract:
     case BuiltinType::SatULongFract:
       Width = Target->getLongFractWidth();
       Align = Target->getLongFractAlign();
       break;
+    case BuiltinType::BFloat16:
+      Width = Target->getBFloat16Width();
+      Align = Target->getBFloat16Align();
+      break;
     case BuiltinType::Float16:
     case BuiltinType::Half:
       if (Target->hasFloat16Type() || !getLangOpts().OpenMP ||
           !getLangOpts().OpenMPIsDevice) {
         Width = Target->getHalfWidth();
         Align = Target->getHalfAlign();
       } else {
         assert(getLangOpts().OpenMP && getLangOpts().OpenMPIsDevice &&
static FloatingRank getFloatingRank(QualType T) {
   switch (T->castAs<BuiltinType>()->getKind()) {
   default: llvm_unreachable("getFloatingRank(): not a floating type");
   case BuiltinType::Float16:    return Float16Rank;
   case BuiltinType::Half:       return HalfRank;
   case BuiltinType::Float:      return FloatRank;
   case BuiltinType::Double:     return DoubleRank;
   case BuiltinType::LongDouble: return LongDoubleRank;
   case BuiltinType::Float128:   return Float128Rank;
+  case BuiltinType::BFloat16:   return BFloat16Rank;
   }
 }
 
 /// getFloatingTypeOfSizeWithinDomain - Returns a real floating
 /// point or a complex type (based on typeDomain/typeSize).
 /// 'typeDomain' is a real floating point or complex type.
 /// 'typeSize' is a real floating point or complex type.
 QualType ASTContext::getFloatingTypeOfSizeWithinDomain(QualType Size,
                                                        QualType Domain) const {
   FloatingRank EltRank = getFloatingRank(Size);
   if (Domain->isComplexType()) {
     switch (EltRank) {
+    case BFloat16Rank: llvm_unreachable("Complex bfloat16 is not supported");
     case Float16Rank:
     case HalfRank: llvm_unreachable("Complex half is not supported");
     case FloatRank:      return FloatComplexTy;
     case DoubleRank:     return DoubleComplexTy;
     case LongDoubleRank: return LongDoubleComplexTy;
     case Float128Rank:   return Float128ComplexTy;
     }
   }
 
   assert(Domain->isRealFloatingType() && "Unknown domain!");
   switch (EltRank) {
   case Float16Rank:    return HalfTy;
+  case BFloat16Rank:   return BFloat16Ty;
   case HalfRank:       return HalfTy;
   case FloatRank:      return FloatTy;
   case DoubleRank:     return DoubleTy;
   case LongDoubleRank: return LongDoubleTy;
   case Float128Rank:   return Float128Ty;
   }
   llvm_unreachable("getFloatingRank(): illegal value for rank");
 }
case BuiltinType::Long:
       return C->getTargetInfo().getLongWidth() == 32 ? 'l' : 'q';
     case BuiltinType::LongLong:   return 'q';
     case BuiltinType::Int128:     return 't';
     case BuiltinType::Float:      return 'f';
     case BuiltinType::Double:     return 'd';
     case BuiltinType::LongDouble: return 'D';
     case BuiltinType::NullPtr:    return '*'; // like char*
 
+    case BuiltinType::BFloat16:
     case BuiltinType::Float16:
     case BuiltinType::Float128:
     case BuiltinType::Half:
     case BuiltinType::ShortAccum:
     case BuiltinType::Accum:
     case BuiltinType::LongAccum:
     case BuiltinType::UShortAccum:
     case BuiltinType::UAccum:
while (!Done) {
     }
   }
 
   QualType Type;
 
   // Read the base type.
   switch (*Str++) {
   default: llvm_unreachable("Unknown builtin type letter!");
+  case 'y':
+    assert(HowLong == 0 && !Signed && !Unsigned &&
+           "Bad modifiers used with 'y'!");
+    Type = Context.BFloat16Ty;
+    break;
   case 'v':
     assert(HowLong == 0 && !Signed && !Unsigned &&
            "Bad modifiers used with 'v'!");
     Type = Context.VoidTy;
     break;
   case 'h':
     assert(HowLong == 0 && !Signed && !Unsigned &&
            "Bad modifiers used with 'h'!");

clang/lib/AST/ItaniumMangle.cpp

void CXXNameMangler::mangleType(const BuiltinType *T) {
   case BuiltinType::Float128: {
     const TargetInfo *TI = getASTContext().getLangOpts().OpenMP &&
                                    getASTContext().getLangOpts().OpenMPIsDevice
                                ? getASTContext().getAuxTargetInfo()
                                : &getASTContext().getTargetInfo();
     Out << TI->getFloat128Mangling();
     break;
   }
+  case BuiltinType::BFloat16: {
+    const TargetInfo *TI = &getASTContext().getTargetInfo();
+    Out << TI->getBFloat16Mangling();
+    break;
+  }
   case BuiltinType::NullPtr:
     Out << "Dn";
     break;
 
 #define BUILTIN_TYPE(Id, SingletonId)
 #define PLACEHOLDER_TYPE(Id, SingletonId) \
   case BuiltinType::Id:
 #include "clang/AST/BuiltinTypes.def"
if (T->getVectorKind() == VectorType::NeonPolyVector) {
     case BuiltinType::Short:     EltName = "int16_t"; break;
     case BuiltinType::UShort:    EltName = "uint16_t"; break;
     case BuiltinType::Int:       EltName = "int32_t"; break;
     case BuiltinType::UInt:      EltName = "uint32_t"; break;
     case BuiltinType::LongLong:  EltName = "int64_t"; break;
     case BuiltinType::ULongLong: EltName = "uint64_t"; break;
     case BuiltinType::Double:    EltName = "float64_t"; break;
     case BuiltinType::Float:     EltName = "float32_t"; break;
-    case BuiltinType::Half:      EltName = "float16_t";break;
+    case BuiltinType::Half:      EltName = "float16_t"; break;
+    case BuiltinType::BFloat16:  EltName = "bfloat16_t"; break;
     default:
       llvm_unreachable("unexpected Neon vector element type");
     }
   }
   const char *BaseName = nullptr;
   unsigned BitSize = (T->getNumElements() *
                       getASTContext().getTypeSize(EltType));
   if (BitSize == 64)
static StringRef mangleAArch64VectorBase(const BuiltinType *EltType) {
   case BuiltinType::ULongLong:
     return "Uint64";
   case BuiltinType::Half:
     return "Float16";
   case BuiltinType::Float:
     return "Float32";
   case BuiltinType::Double:
     return "Float64";
+  case BuiltinType::BFloat16:
+    return "BFloat16";
   default:
     llvm_unreachable("Unexpected vector element base type");
   }
 }
 
 // AArch64's ABI for Neon vector types specifies that they should be mangled as
 // the equivalent internal name. The vector type must be one of the special
 // types predefined by ARM.

clang/lib/AST/MicrosoftMangle.cpp

#include "clang/Basic/AArch64SVEACLETypes.def"
   case BuiltinType::SatUAccum:
   case BuiltinType::SatULongAccum:
   case BuiltinType::SatShortFract:
   case BuiltinType::SatFract:
   case BuiltinType::SatLongFract:
   case BuiltinType::SatUShortFract:
   case BuiltinType::SatUFract:
   case BuiltinType::SatULongFract:
+  case BuiltinType::BFloat16:
   case BuiltinType::Float128: {
     DiagnosticsEngine &Diags = Context.getDiags();
     unsigned DiagID = Diags.getCustomDiagID(
         DiagnosticsEngine::Error, "cannot mangle this built-in %0 type yet");
     Diags.Report(Range.getBegin(), DiagID)
         << T->getName(Context.getASTContext().getPrintingPolicy()) << Range;
     break;
   }

clang/lib/AST/NSAPI.cpp

#include "clang/Basic/AArch64SVEACLETypes.def"
   case BuiltinType::ARCUnbridgedCast:
   case BuiltinType::Half:
   case BuiltinType::PseudoObject:
   case BuiltinType::BuiltinFn:
   case BuiltinType::IncompleteMatrixIdx:
   case BuiltinType::OMPArraySection:
   case BuiltinType::OMPArrayShaping:
   case BuiltinType::OMPIterator:
+  case BuiltinType::BFloat16:
     break;
   }
 
   return None;
 }
 
 /// Returns true if \param T is a typedef of "BOOL" in objective-c.
 bool NSAPI::isObjCBOOLType(QualType T) const {

clang/lib/AST/PrintfFormatString.cpp

bool PrintfSpecifier::fixType(QualType QT, const LangOptions &LangOpt,
   case BuiltinType::WChar_U:
   case BuiltinType::WChar_S:
   case BuiltinType::Char8: // FIXME: Treat like 'char'?
   case BuiltinType::Char16:
   case BuiltinType::Char32:
   case BuiltinType::UInt128:
   case BuiltinType::Int128:
   case BuiltinType::Half:
+  case BuiltinType::BFloat16:
   case BuiltinType::Float16:
   case BuiltinType::Float128:
   case BuiltinType::ShortAccum:
   case BuiltinType::Accum:
   case BuiltinType::LongAccum:
   case BuiltinType::UShortAccum:
   case BuiltinType::UAccum:
   case BuiltinType::ULongAccum:

clang/lib/AST/Type.cpp

bool Type::isRealType() const {
   if (const auto *ET = dyn_cast<EnumType>(CanonicalType))
       return ET->getDecl()->isComplete() && !ET->getDecl()->isScoped();
   return isExtIntType();
 }
 
 bool Type::isArithmeticType() const {
   if (const auto *BT = dyn_cast<BuiltinType>(CanonicalType))
     return BT->getKind() >= BuiltinType::Bool &&
-           BT->getKind() <= BuiltinType::Float128;
+           BT->getKind() <= BuiltinType::Float128 &&
+           BT->getKind() != BuiltinType::BFloat16;
   if (const auto *ET = dyn_cast<EnumType>(CanonicalType))
     // GCC allows forward declaration of enum types (forbid by C99 6.7.2.3p2).
     // If a body isn't seen by the time we get here, return false.
     //
     // C++0x: Enumerations are not arithmetic types. For now, just return
     // false for scoped enumerations since that will disable any
     // unwanted implicit conversions.
     return !ET->getDecl()->isScoped() && ET->getDecl()->isComplete();
StringRef BuiltinType::getName(const PrintingPolicy &Policy) const {
   case ULong:
     return "unsigned long";
   case ULongLong:
     return "unsigned long long";
   case UInt128:
     return "unsigned __int128";
   case Half:
     return Policy.Half ? "half" : "__fp16";
+  case BFloat16:
+    return "__bf16";
   case Float:
     return "float";
   case Double:
     return "double";
   case LongDouble:
     return "long double";
   case ShortAccum:
     return "short _Accum";

clang/lib/AST/TypeLoc.cpp

TypeSpecifierType BuiltinTypeLoc::getWrittenTypeSpec() const {
   case BuiltinType::SatUAccum:
   case BuiltinType::SatULongAccum:
   case BuiltinType::SatShortFract:
   case BuiltinType::SatFract:
   case BuiltinType::SatLongFract:
   case BuiltinType::SatUShortFract:
   case BuiltinType::SatUFract:
   case BuiltinType::SatULongFract:
+  case BuiltinType::BFloat16:
     llvm_unreachable("Builtin type needs extra local data!");
     // Fall through, if the impossible happens.
 
   case BuiltinType::NullPtr:
   case BuiltinType::Overload:
   case BuiltinType::Dependent:
   case BuiltinType::BoundMember:
   case BuiltinType::UnknownAny:

clang/lib/Basic/TargetInfo.cpp

TargetInfo::TargetInfo(const llvm::Triple &T) : TargetOpts(), Triple(T) {
   // SPARC.  These should be overridden by concrete targets as needed.
   BigEndian = !T.isLittleEndian();
   TLSSupported = true;
   VLASupported = true;
   NoAsmVariants = false;
   HasLegalHalfType = false;
   HasFloat128 = false;
   HasFloat16 = false;
+  HasBFloat16 = false;
   PointerWidth = PointerAlign = 32;
   BoolWidth = BoolAlign = 8;
   IntWidth = IntAlign = 32;
   LongWidth = LongAlign = 32;
   LongLongWidth = LongLongAlign = 64;
 
   // Fixed point default bit widths
   ShortAccumWidth = ShortAccumAlign = 16;

clang/lib/Basic/Targets/AArch64.h

public:
 
   StringRef getConstraintRegister(StringRef Constraint,
                                   StringRef Expression) const override {
     return Expression;
   }
 
   int getEHDataRegisterNumber(unsigned RegNo) const override;
 
+  const char *getBFloat16Mangling() const override { return "u6__bf16"; };
   bool hasInt128Type() const override;
 
   bool hasExtIntType() const override { return true; }
 };
 
 class LLVM_LIBRARY_VISIBILITY AArch64leTargetInfo : public AArch64TargetInfo {
 public:
   AArch64leTargetInfo(const llvm::Triple &Triple, const TargetOptions &Opts);

clang/lib/Basic/Targets/AArch64.cpp

AArch64TargetInfo::AArch64TargetInfo(const llvm::Triple &Triple,
 
   MaxVectorAlign = 128;
   MaxAtomicInlineWidth = 128;
   MaxAtomicPromoteWidth = 128;
 
   LongDoubleWidth = LongDoubleAlign = SuitableAlign = 128;
   LongDoubleFormat = &llvm::APFloat::IEEEquad();
 
+  BFloat16Width = BFloat16Align = 16;
+  BFloat16Format = &llvm::APFloat::BFloat();
+
   // Make __builtin_ms_va_list available.
   HasBuiltinMSVaList = true;
 
   // Make the SVE types available.  Note that this deliberately doesn't
   // depend on SveMode, since in principle it should be possible to turn
   // SVE on and off within a translation unit.  It should also be possible
   // to compile the global declaration:
   //
bool AArch64TargetInfo::handleTargetFeatures(std::vector<std::string> &Features,
   HasCrypto = false;
   HasUnaligned = true;
   HasFullFP16 = false;
   HasDotProd = false;
   HasFP16FML = false;
   HasMTE = false;
   HasTME = false;
   HasMatMul = false;
+  HasBFloat16 = false;
   ArchKind = llvm::AArch64::ArchKind::ARMV8A;
 
   for (const auto &Feature : Features) {
     if (Feature == "+neon")
       FPU |= NeonMode;
     if (Feature == "+sve")
       FPU |= SveMode;
     if (Feature == "+crc")
for (const auto &Feature : Features) {
     if (Feature == "+fp16fml")
       HasFP16FML = true;
     if (Feature == "+mte")
       HasMTE = true;
     if (Feature == "+tme")
       HasTME = true;
     if (Feature == "+i8mm")
       HasMatMul = true;
+    if (Feature == "+bf16")
+      HasBFloat16 = true;
   }
 
   setDataLayout();
 
   return true;
 }
 
 TargetInfo::CallingConvCheckResult

clang/lib/Basic/Targets/ARM.h

bool isValidFeatureName(StringRef Feature) const override {
     return Feature != "soft-float-abi";
   }
 
   bool handleTargetFeatures(std::vector<std::string> &Features,
                             DiagnosticsEngine &Diags) override;
 
   bool hasFeature(StringRef Feature) const override;
 
+  bool hasBFloat16Type() const override;
+
   bool isValidCPUName(StringRef Name) const override;
   void fillValidCPUList(SmallVectorImpl<StringRef> &Values) const override;
 
   bool setCPU(const std::string &Name) override;
 
   bool setFPMath(StringRef Name) override;
 
   bool useFP16ConversionIntrinsics() const override {
public:
 
   CallingConvCheckResult checkCallingConvention(CallingConv CC) const override;
 
   int getEHDataRegisterNumber(unsigned RegNo) const override;
 
   bool hasSjLjLowering() const override;
 
   bool hasExtIntType() const override { return true; }
+  
+  const char *getBFloat16Mangling() const override { return "u6__bf16"; };
 };
 
 class LLVM_LIBRARY_VISIBILITY ARMleTargetInfo : public ARMTargetInfo {
 public:
   ARMleTargetInfo(const llvm::Triple &Triple, const TargetOptions &Opts);
   void getTargetDefines(const LangOptions &Opts,
                         MacroBuilder &Builder) const override;
 };

clang/lib/Basic/Targets/ARM.cpp

 
 using namespace clang;
 using namespace clang::targets;
 
 void ARMTargetInfo::setABIAAPCS() {
   IsAAPCS = true;
 
   DoubleAlign = LongLongAlign = LongDoubleAlign = SuitableAlign = 64;
+  BFloat16Width = BFloat16Align = 16;
+  BFloat16Format = &llvm::APFloat::BFloat();
+
   const llvm::Triple &T = getTriple();
 
   bool IsNetBSD = T.isOSNetBSD();
   bool IsOpenBSD = T.isOSOpenBSD();
   if (!T.isOSWindows() && !IsNetBSD && !IsOpenBSD)
     WCharType = UnsignedInt;
 
   UseBitFieldTypeAlignment = true;
void ARMTargetInfo::setABIAPCS(bool IsAAPCS16) {
   const llvm::Triple &T = getTriple();
 
   IsAAPCS = false;
 
   if (IsAAPCS16)
     DoubleAlign = LongLongAlign = LongDoubleAlign = SuitableAlign = 64;
   else
     DoubleAlign = LongLongAlign = LongDoubleAlign = SuitableAlign = 32;
+  BFloat16Width = BFloat16Align = 16;
+  BFloat16Format = &llvm::APFloat::BFloat();
 
   WCharType = SignedInt;
 
   // Do not respect the alignment of bit-field types when laying out
   // structures. This corresponds to PCC_BITFIELD_TYPE_MATTERS in gcc.
   UseBitFieldTypeAlignment = false;
 
   /// gcc forces the alignment to 4 bytes, regardless of the type of the
bool ARMTargetInfo::handleTargetFeatures(std::vector<std::string> &Features,
   Unaligned = 1;
   SoftFloat = false;
   // Note that SoftFloatABI is initialized in our constructor.
   HWDiv = 0;
   DotProd = 0;
   HasMatMul = 0;
   HasFloat16 = true;
   ARMCDECoprocMask = 0;
+  HasBFloat16 = false;
 
   // This does not diagnose illegal cases like having both
   // "+vfpv2" and "+vfpv3" or having "+neon" and "-fp64".
   for (const auto &Feature : Features) {
     if (Feature == "+soft-float") {
       SoftFloat = true;
     } else if (Feature == "+vfp2sp" || Feature == "+vfp2") {
       FPU |= VFP2FPU;
if (Feature == "+soft-float") {
       MVE |= MVE_INT | MVE_FP;
       HW_FP |= HW_FP_SP | HW_FP_HP;
     } else if (Feature == "+i8mm") {
       HasMatMul = 1;
     } else if (Feature.size() == strlen("+cdecp0") && Feature >= "+cdecp0" &&
                Feature <= "+cdecp7") {
       unsigned Coproc = Feature.back() - '0';
       ARMCDECoprocMask |= (1U << Coproc);
+    } else if (Feature == "+bf16") {
+      HasBFloat16 = true;
     }
   }
 
   switch (ArchVersion) {
   case 6:
     if (ArchProfile == llvm::ARM::ProfileKind::M)
       LDREX = 0;
     else if (ArchKind == llvm::ARM::ArchKind::ARMV6K)
return llvm::StringSwitch<bool>(Feature)
       .Case("neon", (FPU & NeonFPU) && !SoftFloat)
       .Case("vfp", FPU && !SoftFloat)
       .Case("hwdiv", HWDiv & HWDivThumb)
       .Case("hwdiv-arm", HWDiv & HWDivARM)
       .Case("mve", hasMVE())
       .Default(false);
 }
 
+bool ARMTargetInfo::hasBFloat16Type() const {
+  return HasBFloat16 && !SoftFloat;
+}
+
 bool ARMTargetInfo::isValidCPUName(StringRef Name) const {
   return Name == "generic" ||
          llvm::ARM::parseCPUArch(Name) != llvm::ARM::ArchKind::INVALID;
 }
 
 void ARMTargetInfo::fillValidCPUList(SmallVectorImpl<StringRef> &Values) const {
   llvm::ARM::fillValidCPUArchList(Values);
 }

clang/lib/CodeGen/ABIInfo.h

namespace swiftcall {
   public:
     ABIInfo(CodeGen::CodeGenTypes &cgt)
         : CGT(cgt), RuntimeCC(llvm::CallingConv::C) {}
 
     virtual ~ABIInfo();
 
     virtual bool supportsSwift() const { return false; }
 
+    virtual bool allowBFloatArgsAndRet() const { return false; }
+
     CodeGen::CGCXXABI &getCXXABI() const;
     ASTContext &getContext() const;
     llvm::LLVMContext &getVMContext() const;
     const llvm::DataLayout &getDataLayout() const;
     const TargetInfo &getTarget() const;
     const CodeGenOptions &getCodeGenOpts() const;
 
     /// Return the calling convention to use for system runtime

clang/lib/CodeGen/CGBuiltin.cpp

Value *CodeGenFunction::EmitTargetBuiltinExpr(unsigned BuiltinID,
   }
 
   return EmitTargetArchBuiltinExpr(this, BuiltinID, E, ReturnValue,
                                    getTarget().getTriple().getArch());
 }
 
 static llvm::VectorType *GetNeonType(CodeGenFunction *CGF,
                                      NeonTypeFlags TypeFlags,
-                                     bool HasLegalHalfType=true,
-                                     bool V1Ty=false) {
+                                     bool HasLegalHalfType = true,
+                                     bool V1Ty = false,
+                                     bool AllowBFloatArgsAndRet = true) {
   int IsQuad = TypeFlags.isQuad();
   switch (TypeFlags.getEltType()) {
   case NeonTypeFlags::Int8:
   case NeonTypeFlags::Poly8:
     return llvm::FixedVectorType::get(CGF->Int8Ty, V1Ty ? 1 : (8 << IsQuad));
   case NeonTypeFlags::Int16:
   case NeonTypeFlags::Poly16:
     return llvm::FixedVectorType::get(CGF->Int16Ty, V1Ty ? 1 : (4 << IsQuad));
+  case NeonTypeFlags::BFloat16:
+    if (AllowBFloatArgsAndRet)
+      return llvm::FixedVectorType::get(CGF->BFloatTy, V1Ty ? 1 : (4 << IsQuad));
+    else
+      return llvm::FixedVectorType::get(CGF->Int16Ty, V1Ty ? 1 : (4 << IsQuad));
   case NeonTypeFlags::Float16:
     if (HasLegalHalfType)
       return llvm::FixedVectorType::get(CGF->HalfTy, V1Ty ? 1 : (4 << IsQuad));
     else
       return llvm::FixedVectorType::get(CGF->Int16Ty, V1Ty ? 1 : (4 << IsQuad));
   case NeonTypeFlags::Int32:
     return llvm::FixedVectorType::get(CGF->Int32Ty, V1Ty ? 1 : (2 << IsQuad));
   case NeonTypeFlags::Int64:
Value *CodeGenFunction::EmitCommonNeonBuiltinExpr(
   if (!Arg->isIntegerConstantExpr(NeonTypeConst, getContext()))
     return nullptr;
 
   // Determine the type of this overloaded NEON intrinsic.
   NeonTypeFlags Type(NeonTypeConst.getZExtValue());
   bool Usgn = Type.isUnsigned();
   bool Quad = Type.isQuad();
   const bool HasLegalHalfType = getTarget().hasLegalHalfType();
+  const bool AllowBFloatArgsAndRet =
+      getTargetHooks().getABIInfo().allowBFloatArgsAndRet();
 
-  llvm::VectorType *VTy = GetNeonType(this, Type, HasLegalHalfType);
+  llvm::VectorType *VTy = GetNeonType(this, Type, HasLegalHalfType, false,
+                                      AllowBFloatArgsAndRet);
   llvm::Type *Ty = VTy;
   if (!Ty)
     return nullptr;
 
   auto getAlignmentValue32 = [&](Address addr) -> Value* {
     return Builder.getInt32(addr.getAlignment().getQuantity());
   };
 
Value *CodeGenFunction::EmitARMBuiltinExpr(unsigned BuiltinID,
   }
 
   // Determine the type of this overloaded NEON intrinsic.
   NeonTypeFlags Type(Result.getZExtValue());
   bool usgn = Type.isUnsigned();
   bool rightShift = false;
 
   llvm::VectorType *VTy = GetNeonType(this, Type,
-                                      getTarget().hasLegalHalfType());
+                                      getTarget().hasLegalHalfType(),
+                                      false,
+                                      getTarget().hasBFloat16Type());
   llvm::Type *Ty = VTy;
   if (!Ty)
     return nullptr;
 
   // Many NEON builtins have identical semantics and uses in ARM and
   // AArch64. Emit these in a single function.
   auto IntrinsicMap = makeArrayRef(ARMSIMDIntrinsicMap);
   const ARMVectorIntrinsicInfo *Builtin = findARMVectorIntrinsicInMap(

clang/lib/CodeGen/CGDebugInfo.cpp

case BuiltinType::LongLong:
     break;
   case BuiltinType::Bool:
     Encoding = llvm::dwarf::DW_ATE_boolean;
     break;
   case BuiltinType::Half:
   case BuiltinType::Float:
   case BuiltinType::LongDouble:
   case BuiltinType::Float16:
+  case BuiltinType::BFloat16:
   case BuiltinType::Float128:
   case BuiltinType::Double:
     // FIXME: For targets where long double and __float128 have the same size,
     // they are currently indistinguishable in the debugger without some
     // special treatment. However, there is currently no consensus on encoding
     // and this should be updated once a DWARF encoding exists for distinct
     // floating point types of the same size.
     Encoding = llvm::dwarf::DW_ATE_float;

clang/lib/CodeGen/CodeGenModule.cpp

CodeGenModule::CodeGenModule(ASTContext &C, const HeaderSearchOptions &HSO,
   // Initialize the type cache.
   llvm::LLVMContext &LLVMContext = M.getContext();
   VoidTy = llvm::Type::getVoidTy(LLVMContext);
   Int8Ty = llvm::Type::getInt8Ty(LLVMContext);
   Int16Ty = llvm::Type::getInt16Ty(LLVMContext);
   Int32Ty = llvm::Type::getInt32Ty(LLVMContext);
   Int64Ty = llvm::Type::getInt64Ty(LLVMContext);
   HalfTy = llvm::Type::getHalfTy(LLVMContext);
+  BFloatTy = llvm::Type::getBFloatTy(LLVMContext);
   FloatTy = llvm::Type::getFloatTy(LLVMContext);
   DoubleTy = llvm::Type::getDoubleTy(LLVMContext);
   PointerWidthInBits = C.getTargetInfo().getPointerWidth(0);
   PointerAlignInBytes =
     C.toCharUnitsFromBits(C.getTargetInfo().getPointerAlign(0)).getQuantity();
   SizeSizeInBytes =
     C.toCharUnitsFromBits(C.getTargetInfo().getMaxPointerWidth()).getQuantity();
   IntAlignInBytes =

clang/lib/CodeGen/CodeGenTypeCache.h

 /// during IR emission.  It's initialized once in CodeGenModule's
 /// constructor and then copied around into new CodeGenFunctions.
 struct CodeGenTypeCache {
   /// void
   llvm::Type *VoidTy;
 
   /// i8, i16, i32, and i64
   llvm::IntegerType *Int8Ty, *Int16Ty, *Int32Ty, *Int64Ty;
-  /// float, double
-  llvm::Type *HalfTy, *FloatTy, *DoubleTy;
+  /// half, bfloat, float, double
+  llvm::Type *HalfTy, *BFloatTy, *FloatTy, *DoubleTy;
 
   /// int
   llvm::IntegerType *IntTy;
 
   /// intptr_t, size_t, and ptrdiff_t, which we assume are the same size.
   union {
     llvm::IntegerType *IntPtrTy;
     llvm::IntegerType *SizeTy;

clang/lib/CodeGen/CodeGenTypes.cpp

static llvm::Type *getTypeForFormat(llvm::LLVMContext &VMContext,
                                     const llvm::fltSemantics &format,
                                     bool UseNativeHalf = false) {
   if (&format == &llvm::APFloat::IEEEhalf()) {
     if (UseNativeHalf)
       return llvm::Type::getHalfTy(VMContext);
     else
       return llvm::Type::getInt16Ty(VMContext);
   }
+  if (&format == &llvm::APFloat::BFloat())
+    return llvm::Type::getBFloatTy(VMContext);
   if (&format == &llvm::APFloat::IEEEsingle())
     return llvm::Type::getFloatTy(VMContext);
   if (&format == &llvm::APFloat::IEEEdouble())
     return llvm::Type::getDoubleTy(VMContext);
   if (&format == &llvm::APFloat::IEEEquad())
     return llvm::Type::getFP128Ty(VMContext);
   if (&format == &llvm::APFloat::PPCDoubleDouble())
     return llvm::Type::getPPC_FP128Ty(VMContext);
   if (&format == &llvm::APFloat::x87DoubleExtended())
     return llvm::Type::getX86_FP80Ty(VMContext);
   llvm_unreachable("Unknown float format!");
case Type::Builtin: {
 
     case BuiltinType::Half:
       // Half FP can either be storage-only (lowered to i16) or native.
       ResultType = getTypeForFormat(
           getLLVMContext(), Context.getFloatTypeSemantics(T),
           Context.getLangOpts().NativeHalfType ||
               !Context.getTargetInfo().useFP16ConversionIntrinsics());
       break;
+    case BuiltinType::BFloat16:
     case BuiltinType::Float:
     case BuiltinType::Double:
     case BuiltinType::LongDouble:
     case BuiltinType::Float128:
       ResultType = getTypeForFormat(getLLVMContext(),
                                     Context.getFloatTypeSemantics(T),
                                     /* UseNativeHalf = */ false);
       break;

clang/lib/CodeGen/ItaniumCXXABI.cpp

#include "clang/Basic/AArch64SVEACLETypes.def"
     case BuiltinType::SatUAccum:
     case BuiltinType::SatULongAccum:
     case BuiltinType::SatShortFract:
     case BuiltinType::SatFract:
     case BuiltinType::SatLongFract:
     case BuiltinType::SatUShortFract:
     case BuiltinType::SatUFract:
     case BuiltinType::SatULongFract:
+    case BuiltinType::BFloat16:
       return false;
 
     case BuiltinType::Dependent:
 #define BUILTIN_TYPE(Id, SingletonId)
 #define PLACEHOLDER_TYPE(Id, SingletonId) \
     case BuiltinType::Id:
 #include "clang/AST/BuiltinTypes.def"
       llvm_unreachable("asking for RRTI for a placeholder type!");

clang/lib/CodeGen/TargetInfo.cpp

bool shouldPassIndirectlyForSwift(ArrayRef<llvm::Type*> scalars,
     return occupiesMoreThan(CGT, scalars, /*total*/ 4);
   }
   bool isSwiftErrorInRegister() const override {
     return true;
   }
 
   bool isLegalVectorTypeForSwift(CharUnits totalSize, llvm::Type *eltTy,
                                  unsigned elts) const override;
+
+  bool allowBFloatArgsAndRet() const override {
+    return getTarget().hasBFloat16Type();
+  }
 };
 
 class AArch64TargetCodeGenInfo : public TargetCodeGenInfo {
 public:
   AArch64TargetCodeGenInfo(CodeGenTypes &CGT, AArch64ABIInfo::ABIKind Kind)
       : TargetCodeGenInfo(std::make_unique<AArch64ABIInfo>(CGT, Kind)) {}
 
   StringRef getARCRetainAutoreleasedReturnValueMarker() const override {
enum ABIKind {
     APCS = 0,
     AAPCS = 1,
     AAPCS_VFP = 2,
     AAPCS16_VFP = 3,
   };
 
 private:
   ABIKind Kind;
+  bool IsFloatABISoftFP;
 
 public:
   ARMABIInfo(CodeGenTypes &CGT, ABIKind _Kind)
       : SwiftABIInfo(CGT), Kind(_Kind) {
     setCCs();
+    IsFloatABISoftFP = CGT.getCodeGenOpts().FloatABI == "softfp" ||
+        CGT.getCodeGenOpts().FloatABI == ""; // default
   }
 
   bool isEABI() const {
     switch (getTarget().getTriple().getEnvironment()) {
     case llvm::Triple::Android:
     case llvm::Triple::EABI:
     case llvm::Triple::EABIHF:
     case llvm::Triple::GNUEABI:
case llvm::Triple::MuslEABIHF:
       return true;
     default:
       return false;
     }
   }
 
   ABIKind getABIKind() const { return Kind; }
 
+  bool allowBFloatArgsAndRet() const override {
+    return !IsFloatABISoftFP && getTarget().hasBFloat16Type();
+  }
+
 private:
   ABIArgInfo classifyReturnType(QualType RetTy, bool isVariadic,
                                 unsigned functionCallConv) const;
   ABIArgInfo classifyArgumentType(QualType RetTy, bool isVariadic,
                                   unsigned functionCallConv) const;
   ABIArgInfo classifyHomogeneousAggregate(QualType Ty, const Type *Base,
                                           uint64_t Members) const;
   ABIArgInfo coerceIllegalVector(QualType Ty) const;
bool IsAAPCS_VFP =
       !isVariadic && isEffectivelyAAPCS_VFP(functionCallConv, /* AAPCS16 */ false);
 
   Ty = useFirstFieldIfTransparentUnion(Ty);
 
   // Handle illegal vector types here.
   if (isIllegalVectorType(Ty))
     return coerceIllegalVector(Ty);
 
-  // _Float16 and __fp16 get passed as if it were an int or float, but with
-  // the top 16 bits unspecified. This is not done for OpenCL as it handles the
-  // half type natively, and does not need to interwork with AAPCS code.
+  // _Float16 and __fp16 get passed as if it were an int or float, but
+  // with the top 16 bits unspecified. This is not done for OpenCL as it handles
+  // the half type natively, and does not need to interwork with AAPCS code.
   if ((Ty->isFloat16Type() || Ty->isHalfType()) &&
       !getContext().getLangOpts().NativeHalfArgsAndReturns) {
     llvm::Type *ResType = IsAAPCS_VFP ?
       llvm::Type::getFloatTy(getVMContext()) :
       llvm::Type::getInt32Ty(getVMContext());
     return ABIArgInfo::getDirect(ResType);
   }
 
+  // __bf16 gets passed using the bfloat IR type, or using i32 but
+  // with the top 16 bits unspecified.
+  if (Ty->isBFloat16Type() && IsFloatABISoftFP) {
+    llvm::Type *ResType = llvm::Type::getInt32Ty(getVMContext());
+    return ABIArgInfo::getDirect(ResType);
+  }
+
   if (!isAggregateTypeForABI(Ty)) {
     // Treat an enum type as its underlying type.
     if (const EnumType *EnumTy = Ty->getAs<EnumType>()) {
       Ty = EnumTy->getDecl()->getIntegerType();
     }
 
     if (const auto *EIT = Ty->getAs<ExtIntType>())
       if (EIT->getNumBits() > 64)
ABIArgInfo ARMABIInfo::classifyReturnType(QualType RetTy, bool isVariadic,
 
   if (RetTy->isVoidType())
     return ABIArgInfo::getIgnore();
 
   if (const VectorType *VT = RetTy->getAs<VectorType>()) {
     // Large vector types should be returned via memory.
     if (getContext().getTypeSize(RetTy) > 128)
       return getNaturalAlignIndirect(RetTy);
-    // FP16 vectors should be converted to integer vectors
-    if (!getTarget().hasLegalHalfType() &&
+    // TODO: FP16/BF16 vectors should be converted to integer vectors
+    // This check is similar  to isIllegalVectorType - refactor?
+    if ((!getTarget().hasLegalHalfType() &&
         (VT->getElementType()->isFloat16Type() ||
-         VT->getElementType()->isHalfType()))
+         VT->getElementType()->isHalfType())) ||
+        (IsFloatABISoftFP &&
+         VT->getElementType()->isBFloat16Type()))
       return coerceIllegalVector(RetTy);
   }
 
   // _Float16 and __fp16 get returned as if it were an int or float, but with
   // the top 16 bits unspecified. This is not done for OpenCL as it handles the
   // half type natively, and does not need to interwork with AAPCS code.
   if ((RetTy->isFloat16Type() || RetTy->isHalfType()) &&
       !getContext().getLangOpts().NativeHalfArgsAndReturns) {
     llvm::Type *ResType = IsAAPCS_VFP ?
       llvm::Type::getFloatTy(getVMContext()) :
       llvm::Type::getInt32Ty(getVMContext());
     return ABIArgInfo::getDirect(ResType);
   }
 
+  // if we're using the softfp float abi, __bf16 get returned as if it were an
+  // int but with the top 16 bits unspecified.
+  if (RetTy->isBFloat16Type()) {
+    llvm::Type *ResType = IsAAPCS_VFP ?
+      llvm::Type::getBFloatTy(getVMContext()) :
+      llvm::Type::getInt32Ty(getVMContext());
+    return ABIArgInfo::getDirect(ResType);
+  }
+
   if (!isAggregateTypeForABI(RetTy)) {
     // Treat an enum type as its underlying type.
     if (const EnumType *EnumTy = RetTy->getAs<EnumType>())
       RetTy = EnumTy->getDecl()->getIntegerType();
 
     if (const auto *EIT = RetTy->getAs<ExtIntType>())
       if (EIT->getNumBits() > 64)
         return getNaturalAlignIndirect(RetTy, /*ByVal=*/false);
ABIArgInfo ARMABIInfo::classifyReturnType(QualType RetTy, bool isVariadic,
   }
 
   return getNaturalAlignIndirect(RetTy);
 }
 
 /// isIllegalVector - check whether Ty is an illegal vector type.
 bool ARMABIInfo::isIllegalVectorType(QualType Ty) const {
   if (const VectorType *VT = Ty->getAs<VectorType> ()) {
-    // On targets that don't support FP16, FP16 is expanded into float, and we
-    // don't want the ABI to depend on whether or not FP16 is supported in
-    // hardware. Thus return false to coerce FP16 vectors into integer vectors.
-    if (!getTarget().hasLegalHalfType() &&
+    // On targets that don't support half, fp16 or bfloat, they are expanded
+    // into float, and we don't want the ABI to depend on whether or not they
+    // are supported in hardware. Thus return false to coerce vectors of these
+    // types into integer vectors.
+    // We do not depend on hasLegalHalfType for bfloat as it is a
+    // separate IR type.
+    if ((!getTarget().hasLegalHalfType() &&
         (VT->getElementType()->isFloat16Type() ||
-         VT->getElementType()->isHalfType()))
+         VT->getElementType()->isHalfType())) ||
+        (IsFloatABISoftFP &&
+         VT->getElementType()->isBFloat16Type()))
       return true;
     if (isAndroid()) {
       // Android shipped using Clang 3.1, which supported a slightly different
       // vector ABI. The primary differences were that 3-element vector types
       // were legal, and so were sub 32-bit vectors (i.e. <2 x i8>). This path
       // accepts that legacy behavior for Android only.
       // Check whether VT is legal.
       unsigned NumElements = VT->getNumElements();
if (llvm::any_of(RD->fields(), [this](FieldDecl *FD) {
           return FD && containsAnyFP16Vectors(FD->getType());
         }))
       return true;
 
     return false;
   } else {
     if (const VectorType *VT = Ty->getAs<VectorType>())
       return (VT->getElementType()->isFloat16Type() ||
+              VT->getElementType()->isBFloat16Type() ||
               VT->getElementType()->isHalfType());
     return false;
   }
 }
 
 bool ARMABIInfo::isLegalVectorTypeForSwift(CharUnits vectorSize,
                                            llvm::Type *eltTy,
                                            unsigned numElts) const {

clang/lib/Format/FormatToken.cpp

bool FormatToken::isSimpleTypeSpecifier() const {
   case tok::kw_signed:
   case tok::kw_unsigned:
   case tok::kw_void:
   case tok::kw_char:
   case tok::kw_int:
   case tok::kw_half:
   case tok::kw_float:
   case tok::kw_double:
+  case tok::kw___bf16:
   case tok::kw__Float16:
   case tok::kw___float128:
   case tok::kw_wchar_t:
   case tok::kw_bool:
   case tok::kw___underlying_type:
   case tok::annot_typename:
   case tok::kw_char8_t:
   case tok::kw_char16_t:

clang/lib/Index/USRGeneration.cpp

#include "clang/Basic/AArch64SVEACLETypes.def"
         case BuiltinType::SatUAccum:
         case BuiltinType::SatULongAccum:
         case BuiltinType::SatShortFract:
         case BuiltinType::SatFract:
         case BuiltinType::SatLongFract:
         case BuiltinType::SatUShortFract:
         case BuiltinType::SatUFract:
         case BuiltinType::SatULongFract:
+        case BuiltinType::BFloat16:
           IgnoreResults = true;
           return;
         case BuiltinType::ObjCId:
           c = 'o'; break;
         case BuiltinType::ObjCClass:
           c = 'O'; break;
         case BuiltinType::ObjCSel:
           c = 'e'; break;

clang/lib/Parse/ParseDecl.cpp

while (1) {
     case tok::kw___int128:
       isInvalid = DS.SetTypeSpecType(DeclSpec::TST_int128, Loc, PrevSpec,
                                      DiagID, Policy);
       break;
     case tok::kw_half:
       isInvalid = DS.SetTypeSpecType(DeclSpec::TST_half, Loc, PrevSpec,
                                      DiagID, Policy);
       break;
+    case tok::kw___bf16:
+      isInvalid = DS.SetTypeSpecType(DeclSpec::TST_BFloat16, Loc, PrevSpec,
+                                     DiagID, Policy);
+      break;
     case tok::kw_float:
       isInvalid = DS.SetTypeSpecType(DeclSpec::TST_float, Loc, PrevSpec,
                                      DiagID, Policy);
       break;
     case tok::kw_double:
       isInvalid = DS.SetTypeSpecType(DeclSpec::TST_double, Loc, PrevSpec,
                                      DiagID, Policy);
       break;
bool Parser::isKnownToBeTypeSpecifier(const Token &Tok) const {
   case tok::kw_void:
   case tok::kw_char:
   case tok::kw_wchar_t:
   case tok::kw_char8_t:
   case tok::kw_char16_t:
   case tok::kw_char32_t:
   case tok::kw_int:
   case tok::kw__ExtInt:
+  case tok::kw___bf16:
   case tok::kw_half:
   case tok::kw_float:
   case tok::kw_double:
   case tok::kw__Accum:
   case tok::kw__Fract:
   case tok::kw__Float16:
   case tok::kw___float128:
   case tok::kw_bool:
bool Parser::isTypeSpecifierQualifier() {
   case tok::kw_char:
   case tok::kw_wchar_t:
   case tok::kw_char8_t:
   case tok::kw_char16_t:
   case tok::kw_char32_t:
   case tok::kw_int:
   case tok::kw__ExtInt:
   case tok::kw_half:
+  case tok::kw___bf16:
   case tok::kw_float:
   case tok::kw_double:
   case tok::kw__Accum:
   case tok::kw__Fract:
   case tok::kw__Float16:
   case tok::kw___float128:
   case tok::kw_bool:
   case tok::kw__Bool:
bool Parser::isDeclarationSpecifier(bool DisambiguatingWithExpression) {
   case tok::kw_wchar_t:
   case tok::kw_char8_t:
   case tok::kw_char16_t:
   case tok::kw_char32_t:
 
   case tok::kw_int:
   case tok::kw__ExtInt:
   case tok::kw_half:
+  case tok::kw___bf16:
   case tok::kw_float:
   case tok::kw_double:
   case tok::kw__Accum:
   case tok::kw__Fract:
   case tok::kw__Float16:
   case tok::kw___float128:
   case tok::kw_bool:
   case tok::kw__Bool:

clang/lib/Parse/ParseExpr.cpp

#undef RTT_JOIN
   case tok::kw___int64:
   case tok::kw___int128:
   case tok::kw__ExtInt:
   case tok::kw_signed:
   case tok::kw_unsigned:
   case tok::kw_half:
   case tok::kw_float:
   case tok::kw_double:
+  case tok::kw___bf16:
   case tok::kw__Float16:
   case tok::kw___float128:
   case tok::kw_void:
   case tok::kw_typename:
   case tok::kw_typeof:
   case tok::kw___vector:
 #define GENERIC_IMAGE_TYPE(ImgType, Id) case tok::kw_##ImgType##_t:
 #include "clang/Basic/OpenCLImageTypes.def"

clang/lib/Parse/ParseExprCXX.cpp

case tok::kw_char:
     DS.SetTypeSpecType(DeclSpec::TST_char, Loc, PrevSpec, DiagID, Policy);
     break;
   case tok::kw_int:
     DS.SetTypeSpecType(DeclSpec::TST_int, Loc, PrevSpec, DiagID, Policy);
     break;
   case tok::kw___int128:
     DS.SetTypeSpecType(DeclSpec::TST_int128, Loc, PrevSpec, DiagID, Policy);
     break;
+  case tok::kw___bf16:
+    DS.SetTypeSpecType(DeclSpec::TST_BFloat16, Loc, PrevSpec, DiagID, Policy);
+    break;
   case tok::kw_half:
     DS.SetTypeSpecType(DeclSpec::TST_half, Loc, PrevSpec, DiagID, Policy);
     break;
   case tok::kw_float:
     DS.SetTypeSpecType(DeclSpec::TST_float, Loc, PrevSpec, DiagID, Policy);
     break;
   case tok::kw_double:
     DS.SetTypeSpecType(DeclSpec::TST_double, Loc, PrevSpec, DiagID, Policy);

clang/lib/Parse/ParseTentative.cpp

Parser::isCXXDeclarationSpecifier(Parser::TPResult BracedCastResult,
   case tok::kw_long:
   case tok::kw___int64:
   case tok::kw___int128:
   case tok::kw_signed:
   case tok::kw_unsigned:
   case tok::kw_half:
   case tok::kw_float:
   case tok::kw_double:
+  case tok::kw___bf16:
   case tok::kw__Float16:
   case tok::kw___float128:
   case tok::kw_void:
   case tok::annot_decltype:
 #define GENERIC_IMAGE_TYPE(ImgType, Id) case tok::kw_##ImgType##_t:
 #include "clang/Basic/OpenCLImageTypes.def"
     if (NextToken().is(tok::l_paren))
       return TPResult::Ambiguous;
bool Parser::isCXXDeclarationSpecifierAType() {
   case tok::kw_long:
   case tok::kw___int64:
   case tok::kw___int128:
   case tok::kw_signed:
   case tok::kw_unsigned:
   case tok::kw_half:
   case tok::kw_float:
   case tok::kw_double:
+  case tok::kw___bf16:
   case tok::kw__Float16:
   case tok::kw___float128:
   case tok::kw_void:
   case tok::kw___unknown_anytype:
   case tok::kw___auto_type:
 #define GENERIC_IMAGE_TYPE(ImgType, Id) case tok::kw_##ImgType##_t:
 #include "clang/Basic/OpenCLImageTypes.def"
     return true;

clang/lib/Sema/DeclSpec.cpp

switch (DS.getTypeSpecType()) {
     case TST_extint:
     case TST_struct:
     case TST_interface:
     case TST_union:
     case TST_unknown_anytype:
     case TST_unspecified:
     case TST_void:
     case TST_wchar:
+    case TST_BFloat16:
 #define GENERIC_IMAGE_TYPE(ImgType, Id) case TST_##ImgType##_t:
 #include "clang/Basic/OpenCLImageTypes.def"
       return false;
 
     case TST_decltype_auto:
       // This must have an initializer, so can't be a function declaration,
       // even if the initializer has function type.
       return false;
const char *DeclSpec::getSpecifierName(DeclSpec::TST T,
   case DeclSpec::TST_typeofExpr:  return "typeof";
   case DeclSpec::TST_auto:        return "auto";
   case DeclSpec::TST_auto_type:   return "__auto_type";
   case DeclSpec::TST_decltype:    return "(decltype)";
   case DeclSpec::TST_decltype_auto: return "decltype(auto)";
   case DeclSpec::TST_underlyingType: return "__underlying_type";
   case DeclSpec::TST_unknown_anytype: return "__unknown_anytype";
   case DeclSpec::TST_atomic: return "_Atomic";
+  case DeclSpec::TST_BFloat16: return "__bf16";
 #define GENERIC_IMAGE_TYPE(ImgType, Id) \
   case DeclSpec::TST_##ImgType##_t: \
     return #ImgType "_t";
 #include "clang/Basic/OpenCLImageTypes.def"
   case DeclSpec::TST_error:       return "(error)";
   }
   llvm_unreachable("Unknown typespec!");
 }

clang/lib/Sema/SemaCast.cpp

void CastOperation::CheckCStyleCast() {
   // Note that member pointers were filtered out with C++, above.
 
   if (isa<ObjCSelectorExpr>(SrcExpr.get())) {
     Self.Diag(SrcExpr.get()->getExprLoc(), diag::err_cast_selector_expr);
     SrcExpr = ExprError();
     return;
   }
 
+  // Can't cast to or from bfloat
+  if (DestType->isBFloat16Type() && !SrcType->isBFloat16Type()) {
+    Self.Diag(SrcExpr.get()->getExprLoc(), diag::err_cast_to_bfloat16)
+        << SrcExpr.get()->getSourceRange();
+    SrcExpr = ExprError();
+    return;
+  }
+  if (SrcType->isBFloat16Type() && !DestType->isBFloat16Type()) {
+    Self.Diag(SrcExpr.get()->getExprLoc(), diag::err_cast_from_bfloat16)
+        << SrcExpr.get()->getSourceRange();
+    SrcExpr = ExprError();
+    return;
+  }
+
   // If either type is a pointer, the other type has to be either an
   // integer or a pointer.
   if (!DestType->isArithmeticType()) {
     if (!SrcType->isIntegralType(Self.Context) && SrcType->isArithmeticType()) {
       Self.Diag(SrcExpr.get()->getExprLoc(),
                 diag::err_cast_pointer_from_non_pointer_int)
         << SrcType << SrcExpr.get()->getSourceRange();
       SrcExpr = ExprError();

clang/lib/Sema/SemaChecking.cpp

case NeonTypeFlags::Float16:
     assert(!shift && "cannot shift float types!");
     return (4 << IsQuad) - 1;
   case NeonTypeFlags::Float32:
     assert(!shift && "cannot shift float types!");
     return (2 << IsQuad) - 1;
   case NeonTypeFlags::Float64:
     assert(!shift && "cannot shift float types!");
     return (1 << IsQuad) - 1;
+  case NeonTypeFlags::BFloat16:
+    assert(!shift && "cannot shift float types!");
+    return (4 << IsQuad) - 1;
   }
   llvm_unreachable("Invalid NeonTypeFlag!");
 }
 
 /// getNeonEltType - Return the QualType corresponding to the elements of
 /// the vector type specified by the NeonTypeFlags.  This is used to check
 /// the pointer arguments for Neon load/store intrinsics.
 static QualType getNeonEltType(NeonTypeFlags Flags, ASTContext &Context,
static QualType getNeonEltType(NeonTypeFlags Flags, ASTContext &Context,
   case NeonTypeFlags::Poly128:
     break;
   case NeonTypeFlags::Float16:
     return Context.HalfTy;
   case NeonTypeFlags::Float32:
     return Context.FloatTy;
   case NeonTypeFlags::Float64:
     return Context.DoubleTy;
+  case NeonTypeFlags::BFloat16:
+    return Context.BFloat16Ty;
   }
   llvm_unreachable("Invalid NeonTypeFlag!");
 }
 
 bool Sema::CheckSVEBuiltinFunctionCall(unsigned BuiltinID, CallExpr *TheCall) {
   // Range check SVE intrinsics that take immediate values.
   SmallVector<std::tuple<int,int,int>, 3> ImmChecks;
 

clang/lib/Sema/SemaDecl.cpp

bool Sema::isSimpleTypeSpecifier(tok::TokenKind Kind) const {
   case tok::kw_signed:
   case tok::kw_unsigned:
   case tok::kw_void:
   case tok::kw_char:
   case tok::kw_int:
   case tok::kw_half:
   case tok::kw_float:
   case tok::kw_double:
+  case tok::kw___bf16:
   case tok::kw__Float16:
   case tok::kw___float128:
   case tok::kw_wchar_t:
   case tok::kw_bool:
   case tok::kw___underlying_type:
   case tok::kw___auto_type:
     return true;
 

clang/lib/Sema/SemaExpr.cpp

QualType Sema::CheckConditionalOperands(ExprResult &Cond, ExprResult &LHS,
   // to find a common type: C99 6.5.15p3,5.
   if (LHSTy->isArithmeticType() && RHSTy->isArithmeticType()) {
     LHS = ImpCastExprToType(LHS.get(), ResTy, PrepareScalarCast(LHS, ResTy));
     RHS = ImpCastExprToType(RHS.get(), ResTy, PrepareScalarCast(RHS, ResTy));
 
     return ResTy;
   }
 
+  // And if they're both bfloat (which isn't arithmetic), that's fine too.
+  if (LHSTy->isBFloat16Type() && RHSTy->isBFloat16Type()) {
+    return LHSTy;
+  }
+
   // If both operands are the same structure or union type, the result is that
   // type.
   if (const RecordType *LHSRT = LHSTy->getAs<RecordType>()) {    // C99 6.5.15p3
     if (const RecordType *RHSRT = RHSTy->getAs<RecordType>())
       if (LHSRT->getDecl() == RHSRT->getDecl())
         // "If both the operands have structure or union type, the result has
         // that type."  This implies that CV qualifiers are dropped.
         return LHSTy.getUnqualifiedType();

clang/lib/Sema/SemaOverload.cpp

FromType = ToType.getUnqualifiedType();
              (ToType->isAnyComplexType() && FromType->isArithmeticType())) {
     // Complex-real conversions (C99 6.3.1.7)
     SCS.Second = ICK_Complex_Real;
     FromType = ToType.getUnqualifiedType();
   } else if (FromType->isRealFloatingType() && ToType->isRealFloatingType()) {
     // FIXME: disable conversions between long double and __float128 if
     // their representation is different until there is back end support
     // We of course allow this conversion if long double is really double.
+
+    // Conversions between bfloat and other floats are not permitted.
+    if (FromType == S.Context.BFloat16Ty || ToType == S.Context.BFloat16Ty)
+      return false;
     if (&S.Context.getFloatTypeSemantics(FromType) !=
         &S.Context.getFloatTypeSemantics(ToType)) {
       bool Float128AndLongDouble = ((FromType == S.Context.Float128Ty &&
                                     ToType == S.Context.LongDoubleTy) ||
                                    (FromType == S.Context.LongDoubleTy &&
                                     ToType == S.Context.Float128Ty));
       if (Float128AndLongDouble &&
           (&S.Context.getFloatTypeSemantics(S.Context.LongDoubleTy) ==
            &llvm::APFloat::PPCDoubleDouble()))
         return false;
     }
     // Floating point conversions (C++ 4.8).
     SCS.Second = ICK_Floating_Conversion;
     FromType = ToType.getUnqualifiedType();
   } else if ((FromType->isRealFloatingType() &&
               ToType->isIntegralType(S.Context)) ||
              (FromType->isIntegralOrUnscopedEnumerationType() &&
               ToType->isRealFloatingType())) {
+    // Conversions between bfloat and int are not permitted.
+    if (FromType->isBFloat16Type() || ToType->isBFloat16Type())
+      return false;
+
     // Floating-integral conversions (C++ 4.9).
     SCS.Second = ICK_Floating_Integral;
     FromType = ToType.getUnqualifiedType();
   } else if (S.IsBlockPointerConversion(FromType, ToType, FromType)) {
     SCS.Second = ICK_Block_Pointer_Conversion;
   } else if (AllowObjCWritebackConversion &&
              S.isObjCWritebackConversion(FromType, ToType, FromType)) {
     SCS.Second = ICK_Writeback_Conversion;

clang/lib/Sema/SemaTemplateVariadic.cpp

bool Sema::containsUnexpandedParameterPacks(Declarator &D) {
   case TST_enum:
   case TST_union:
   case TST_struct:
   case TST_interface:
   case TST_class:
   case TST_auto:
   case TST_auto_type:
   case TST_decltype_auto:
+  case TST_BFloat16:
 #define GENERIC_IMAGE_TYPE(ImgType, Id) case TST_##ImgType##_t:
 #include "clang/Basic/OpenCLImageTypes.def"
   case TST_unknown_anytype:
   case TST_error:
     break;
   }
 
   for (unsigned I = 0, N = D.getNumTypeObjects(); I != N; ++I) {

clang/lib/Sema/SemaType.cpp

case DeclSpec::TST_float16:
     // ToDo: more precise diagnostics for CUDA.
     if (!S.Context.getTargetInfo().hasFloat16Type() && !S.getLangOpts().CUDA &&
         !(S.getLangOpts().OpenMP && S.getLangOpts().OpenMPIsDevice))
       S.Diag(DS.getTypeSpecTypeLoc(), diag::err_type_unsupported)
         << "_Float16";
     Result = Context.Float16Ty;
     break;
   case DeclSpec::TST_half:    Result = Context.HalfTy; break;
+  case DeclSpec::TST_BFloat16:
+    if (!S.Context.getTargetInfo().hasBFloat16Type())
+      S.Diag(DS.getTypeSpecTypeLoc(), diag::err_type_unsupported)
+        << "__bf16";
+    Result = Context.BFloat16Ty;
+    break;
   case DeclSpec::TST_float:   Result = Context.FloatTy; break;
   case DeclSpec::TST_double:
     if (DS.getTypeSpecWidth() == DeclSpec::TSW_long)
       Result = Context.LongDoubleTy;
     else
       Result = Context.DoubleTy;
     break;
   case DeclSpec::TST_float128:
return BTy->getKind() == BuiltinType::SChar ||
          BTy->getKind() == BuiltinType::UShort ||
          BTy->getKind() == BuiltinType::Int ||
          BTy->getKind() == BuiltinType::UInt ||
          BTy->getKind() == BuiltinType::Long ||
          BTy->getKind() == BuiltinType::ULong ||
          BTy->getKind() == BuiltinType::LongLong ||
          BTy->getKind() == BuiltinType::ULongLong ||
          BTy->getKind() == BuiltinType::Float ||
-         BTy->getKind() == BuiltinType::Half;
+         BTy->getKind() == BuiltinType::Half ||
+         BTy->getKind() == BuiltinType::BFloat16;
 }
 
 /// HandleNeonVectorTypeAttr - The "neon_vector_type" and
 /// "neon_polyvector_type" attributes are used to create vector types that
 /// are mangled according to ARM's ABI.  Otherwise, these types are identical
 /// to those created with the "vector_size" attribute.  Unlike "vector_size"
 /// the argument to these Neon attributes is the number of vector elements,
 /// not the vector size in bytes.  The vector width and element type must

clang/lib/Serialization/ASTCommon.cpp

case BuiltinType::OMPArraySection:
     ID = PREDEF_TYPE_OMP_ARRAY_SECTION;
     break;
   case BuiltinType::OMPArrayShaping:
     ID = PREDEF_TYPE_OMP_ARRAY_SHAPING;
     break;
   case BuiltinType::OMPIterator:
     ID = PREDEF_TYPE_OMP_ITERATOR;
     break;
+  case BuiltinType::BFloat16:
+    ID = PREDEF_TYPE_BFLOAT16_ID;
+    break;
   }
 
   return TypeIdx(ID);
 }
 
 unsigned serialization::ComputeHash(Selector Sel) {
   unsigned N = Sel.getNumArgs();
   if (N == 0)

clang/lib/Serialization/ASTReader.cpp

case PREDEF_TYPE_LONG_ID:
       T = Context.LongTy;
       break;
     case PREDEF_TYPE_LONGLONG_ID:
       T = Context.LongLongTy;
       break;
     case PREDEF_TYPE_INT128_ID:
       T = Context.Int128Ty;
       break;
+    case PREDEF_TYPE_BFLOAT16_ID:
+      T = Context.BFloat16Ty;
+      break;
     case PREDEF_TYPE_HALF_ID:
       T = Context.HalfTy;
       break;
     case PREDEF_TYPE_FLOAT_ID:
       T = Context.FloatTy;
       break;
     case PREDEF_TYPE_DOUBLE_ID:
       T = Context.DoubleTy;

clang/test/CodeGen/arm-bf16-params-returns.c

+// RUN: %clang_cc1 -triple armv8.6a-arm-none-eabi -target-abi aapcs -mfloat-abi hard -target-feature +bf16 -target-feature +neon -emit-llvm -O2 -o - %s | opt -S -mem2reg -sroa | FileCheck %s --check-prefix=CHECK32-HARD
+// RUN: %clang_cc1 -triple aarch64-arm-none-eabi -target-abi aapcs -mfloat-abi hard -target-feature +bf16 -target-feature +neon -emit-llvm -O2 -o - %s | opt -S -mem2reg -sroa | FileCheck %s --check-prefix=CHECK64-HARD
+// RUN: %clang_cc1 -triple armv8.6a-arm-none-eabi -target-abi aapcs -mfloat-abi softfp -target-feature +bf16 -target-feature +neon -emit-llvm -O2 -o - %s | opt -S -mem2reg -sroa | FileCheck %s --check-prefix=CHECK32-SOFTFP
+// RUN: %clang_cc1 -triple aarch64-arm-none-eabi -target-abi aapcs -mfloat-abi softfp -target-feature +bf16 -target-feature +neon -emit-llvm -O2 -o - %s | opt -S -mem2reg -sroa | FileCheck %s --check-prefix=CHECK64-SOFTFP
+
+// function return types
+__bf16 test_ret_bf16(__bf16 v) {
+  return v;
+}
+// CHECK32-HARD: define arm_aapcs_vfpcc bfloat @test_ret_bf16(bfloat returned %v) {{.*}} {
+// CHECK32-HARD: ret bfloat %v
+// CHECK64-HARD: define bfloat @test_ret_bf16(bfloat returned %v) {{.*}} {
+// CHECK64-HARD: ret bfloat %v
+// CHECK32-SOFTFP: define i32 @test_ret_bf16(i32 [[V0:.*]]) {{.*}} {
+// CHECK32-SOFTFP: %tmp2.0.insert.ext = and i32 [[V0]], 65535
+// CHECK32-SOFTFP: ret i32 %tmp2.0.insert.ext
+// CHECK64-SOFTFP: define bfloat @test_ret_bf16(bfloat returned %v) {{.*}} {
+// CHECK64-SOFTFP: ret bfloat %v

clang/test/CodeGen/arm-bf16-softfloat.c

+// RUN: not %clang -target arm-arm-eabi -march=armv8-a+bf16 -mfloat-abi=soft -c %s 2>&1 | FileCheck %s
+
+// CHECK: error: __bf16 is not supported on this target
+extern __bf16 var;

clang/test/CodeGen/arm-mangle-bf16.cpp

+// RUN: %clang_cc1 -triple aarch64-arm-none-eabi -target-feature +bf16 -emit-llvm -o - %s | FileCheck %s --check-prefix=CHECK64
+// RUN: %clang_cc1 -triple arm-arm-none-eabi -target-feature +bf16 -mfloat-abi hard -emit-llvm -o - %s | FileCheck %s --check-prefix=CHECK32-HARD
+// RUN: %clang_cc1 -triple arm-arm-none-eabi -target-feature +bf16 -mfloat-abi softfp -emit-llvm -o - %s | FileCheck %s --check-prefix=CHECK32-SOFTFP
+
+// CHECK64: define {{.*}}void @_Z3foou6__bf16(bfloat %b)
+// CHECK32-HARD: define {{.*}}void @_Z3foou6__bf16(bfloat %b)
+// CHECK32-SOFTFP: define {{.*}}void @_Z3foou6__bf16(i32 %b.coerce)
+void foo(__bf16 b) {}

clang/test/Sema/arm-bf16-forbidden-ops.c

+// RUN: %clang_cc1 -fsyntax-only -verify -triple aarch64 -target-feature +bf16 %s
+
+__bf16 test_cast_from_float(float in) {
+  return (__bf16)in; // expected-error {{cannot type-cast to __bf16}}
+}
+
+__bf16 test_cast_from_float_literal(void) {
+  return (__bf16)1.0f; // expected-error {{cannot type-cast to __bf16}}
+}
+
+__bf16 test_cast_from_int(int in) {
+  return (__bf16)in; // expected-error {{cannot type-cast to __bf16}}
+}
+
+__bf16 test_cast_from_int_literal(void) {
+  return (__bf16)1; // expected-error {{cannot type-cast to __bf16}}
+}
+
+__bf16 test_cast_bfloat(__bf16 in) {
+  return (__bf16)in; // this one should work
+}
+
+float test_cast_to_float(__bf16 in) {
+  return (float)in; // expected-error {{cannot type-cast from __bf16}}
+}
+
+int test_cast_to_int(__bf16 in) {
+  return (int)in; // expected-error {{cannot type-cast from __bf16}}
+}
+
+__bf16 test_implicit_from_float(float in) {
+  return in; // expected-error {{returning 'float' from a function with incompatible result type '__bf16'}}
+}
+
+__bf16 test_implicit_from_float_literal(void) {
+  return 1.0f; // expected-error {{returning 'float' from a function with incompatible result type '__bf16'}}
+}
+
+__bf16 test_implicit_from_int(int in) {
+  return in; // expected-error {{returning 'int' from a function with incompatible result type '__bf16'}}
+}
+
+__bf16 test_implicit_from_int_literal(void) {
+  return 1; // expected-error {{returning 'int' from a function with incompatible result type '__bf16'}}
+}
+
+__bf16 test_implicit_bfloat(__bf16 in) {
+  return in; // this one should work
+}
+
+float test_implicit_to_float(__bf16 in) {
+  return in; // expected-error {{returning '__bf16' from a function with incompatible result type 'float'}}
+}
+
+int test_implicit_to_int(__bf16 in) {
+  return in; // expected-error {{returning '__bf16' from a function with incompatible result type 'int'}}
+}
+
+__bf16 test_cond(__bf16 a, __bf16 b, _Bool which) {
+  // Conditional operator _should_ be supported, without nonsense
+  // complaints like 'types __bf16 and __bf16 are not compatible'
+  return which ? a : b;
+}
+
+__bf16 test_cond_float(__bf16 a, __bf16 b, _Bool which) {
+  return which ? a : 1.0f; // expected-error {{incompatible operand types ('__bf16' and 'float')}}
+}
+
+__bf16 test_cond_int(__bf16 a, __bf16 b, _Bool which) {
+  return which ? a : 1; // expected-error {{incompatible operand types ('__bf16' and 'int')}}
+}

clang/test/Sema/arm-bf16-forbidden-ops.cpp

+// RUN: %clang_cc1 -fsyntax-only -verify -triple aarch64 -target-feature +bf16 %s
+
+__bf16 test_static_cast_from_float(float in) {
+  return static_cast<__bf16>(in); // expected-error {{static_cast from 'float' to '__bf16' is not allowed}}
+}
+
+__bf16 test_static_cast_from_float_literal(void) {
+  return static_cast<__bf16>(1.0f); // expected-error {{static_cast from 'float' to '__bf16' is not allowed}}
+}
+
+__bf16 test_static_cast_from_int(int in) {
+  return static_cast<__bf16>(in); // expected-error {{static_cast from 'int' to '__bf16' is not allowed}}
+}
+
+__bf16 test_static_cast_from_int_literal(void) {
+  return static_cast<__bf16>(1); // expected-error {{static_cast from 'int' to '__bf16' is not allowed}}
+}
+
+__bf16 test_static_cast_bfloat(__bf16 in) {
+  return static_cast<__bf16>(in); // this one should work
+}
+
+float test_static_cast_to_float(__bf16 in) {
+  return static_cast<float>(in); // expected-error {{static_cast from '__bf16' to 'float' is not allowed}}
+}
+
+int test_static_cast_to_int(__bf16 in) {
+  return static_cast<int>(in); // expected-error {{static_cast from '__bf16' to 'int' is not allowed}}
+}
+
+__bf16 test_implicit_from_float(float in) {
+  return in; // expected-error {{cannot initialize return object of type '__bf16' with an lvalue of type 'float'}}
+}
+
+__bf16 test_implicit_from_float_literal() {
+  return 1.0f; // expected-error {{cannot initialize return object of type '__bf16' with an rvalue of type 'float'}}
+}
+
+__bf16 test_implicit_from_int(int in) {
+  return in; // expected-error {{cannot initialize return object of type '__bf16' with an lvalue of type 'int'}}
+}
+
+__bf16 test_implicit_from_int_literal() {
+  return 1; // expected-error {{cannot initialize return object of type '__bf16' with an rvalue of type 'int'}}
+}
+
+__bf16 test_implicit_bfloat(__bf16 in) {
+  return in; // this one should work
+}
+
+float test_implicit_to_float(__bf16 in) {
+  return in; // expected-error {{cannot initialize return object of type 'float' with an lvalue of type '__bf16'}}
+}
+
+int test_implicit_to_int(__bf16 in) {
+  return in; // expected-error {{cannot initialize return object of type 'int' with an lvalue of type '__bf16'}}
+}
+
+__bf16 test_cond(__bf16 a, __bf16 b, bool which) {
+  // Conditional operator _should_ be supported, without nonsense
+  // complaints like 'types __bf16 and __bf16 are not compatible'
+  return which ? a : b;
+}
+
+__bf16 test_cond_float(__bf16 a, __bf16 b, bool which) {
+  return which ? a : 1.0f; // expected-error {{incompatible operand types ('__bf16' and 'float')}}
+}
+
+__bf16 test_cond_int(__bf16 a, __bf16 b, bool which) {
+  return which ? a : 1; // expected-error {{incompatible operand types ('__bf16' and 'int')}}
+}

clang/test/Sema/arm-bfloat.cpp

+// RUN: %clang_cc1 -fsyntax-only -verify -std=c++11 \
+// RUN: -triple aarch64-arm-none-eabi -target-cpu cortex-a75 \
+// RUN: -target-feature +bf16 -target-feature +neon %s
+// RUN: %clang_cc1 -fsyntax-only -verify -std=c++11 \
+// RUN: -triple arm-arm-none-eabi -target-cpu cortex-a53 \
+// RUN: -target-feature +bf16 -target-feature +neon %s
+
+void test(bool b) {
+  __bf16 bf16;
+
+  bf16 + bf16; // expected-error {{invalid operands to binary expression ('__bf16' and '__bf16')}}
+  bf16 - bf16; // expected-error {{invalid operands to binary expression ('__bf16' and '__bf16')}}
+  bf16 * bf16; // expected-error {{invalid operands to binary expression ('__bf16' and '__bf16')}}
+  bf16 / bf16; // expected-error {{invalid operands to binary expression ('__bf16' and '__bf16')}}
+
+  __fp16 fp16;
+
+  bf16 + fp16; // expected-error {{invalid operands to binary expression ('__bf16' and '__fp16')}}
+  fp16 + bf16; // expected-error {{invalid operands to binary expression ('__fp16' and '__bf16')}}
+  bf16 - fp16; // expected-error {{invalid operands to binary expression ('__bf16' and '__fp16')}}
+  fp16 - bf16; // expected-error {{invalid operands to binary expression ('__fp16' and '__bf16')}}
+  bf16 * fp16; // expected-error {{invalid operands to binary expression ('__bf16' and '__fp16')}}
+  fp16 * bf16; // expected-error {{invalid operands to binary expression ('__fp16' and '__bf16')}}
+  bf16 / fp16; // expected-error {{invalid operands to binary expression ('__bf16' and '__fp16')}}
+  fp16 / bf16; // expected-error {{invalid operands to binary expression ('__fp16' and '__bf16')}}
+  bf16 = fp16; // expected-error {{assigning to '__bf16' from incompatible type '__fp16'}}
+  fp16 = bf16; // expected-error {{assigning to '__fp16' from incompatible type '__bf16'}}
+  bf16 + (b ? fp16 : bf16); // expected-error {{incompatible operand types ('__fp16' and '__bf16')}}
+}

clang/tools/libclang/CXType.cpp

switch (K) {
     TKIND(DependentSizedArray);
     TKIND(Vector);
     TKIND(ExtVector);
     TKIND(MemberPointer);
     TKIND(Auto);
     TKIND(Elaborated);
     TKIND(Pipe);
     TKIND(Attributed);
+    TKIND(BFloat16);
 #define IMAGE_TYPE(ImgType, Id, SingletonId, Access, Suffix) TKIND(Id);
 #include "clang/Basic/OpenCLImageTypes.def"
 #undef IMAGE_TYPE
 #define EXT_OPAQUE_TYPE(ExtTYpe, Id, Ext) TKIND(Id);
 #include "clang/Basic/OpenCLExtensionTypes.def"
     TKIND(OCLSampler);
     TKIND(OCLEvent);
     TKIND(OCLQueue);