[Sema][AArch64] Support arm_sve_vector_bits attribute

[c-rhodes]

Summary

This patch implements the semantics for the 'arm_sve_vector_bits' type
attribute, defined by the Arm C Language Extensions (ACLE) for SVE [1].
The purpose of this attribute is to define vector-length-specific (VLS)
versions of existing vector-length-agnostic (VLA) types.

The semantics were already implemented by D83551, although the
implementation approach has since changed to represent VLSTs as
VectorType in the AST and fixed-length vectors in the IR everywhere
except in function args/returns. This is described in the prototype
patch D85128 demonstrating the new approach.

The semantic changes added in D83551 are changed since the
AttributedType is replaced by VectorType in the AST. Minimal changes
were necessary in the previous patch as the canonical type for both VLA
and VLS was the same (i.e. sizeless), except in constructs such as
globals and structs where sizeless types are unsupported. This patch
reverts the changes that permitted VLS types that were represented as
sizeless types in such circumstances, and adds support for implicit
casting between VLA <-> VLS types as described in section 3.7.3.2 of the
ACLE.

Since the SVE builtin types for bool and uint8 are both represented as
BuiltinType::UChar in VLSTs, two new vector kinds are implemented to
distinguish predicate and data vectors.

[1] https://developer.arm.com/documentation/100987/latest

[c-rhodes]

Added missing implicit conversions for C++. I considered handling this with the
existing implicit vector conversion although one side of the conversion will be
an SVE builtin, so instead I've added a new conversion specifically for SVE. I
suspect the existing one could support this but I wasn't sure if that was a good
idea (?). In C++ implicit conversions between VLA/VLS have a rank just below
derived-to-base conversion.

[rsandifo-arm]

clang/include/clang/AST/Type.h

1897

It feels to me like the information is more general than that, since the type of the element is meaningful independently of the new attribute. How about just getSveEltType instead?

clang/lib/AST/ASTContext.cpp

1941

The alignment of the SVE data vectors should be 128 too (see the alignof stuff in the ACLE doc). FWIW, there were two reasons for defining it like that:

• the fixed-length types map to the same ABI machine type as the variable-length types
• the length isn't required to be a power of 2: an implementation that supports fixed-length 384-bit vectors could define __ARM_FEATURE_SVE_BITS to 384.

clang/lib/AST/TextNodeDumper.cpp

1413

Is it worth distinguishing these, similarly to altivec?

clang/lib/AST/Type.cpp

2338

This is just a note, not sure what can be done about it, but:

The element types are defined in terms of the stdint.h types. One snag is that ILP32 newlib defines int32_t to be UnsignedLongTy instead of UnsignedIntTy, and defines uint64_t to UnsignedLongLongTy.

In GCC we got away with this because GCC already has hard-coded knowledge about the target C library's choices. I don't think that information is available here though.

Like I say, there's nothing that necessarily needs to be “fixed”/dealt-with now, just thought it was worth mentioning.

clang/lib/Sema/SemaExpr.cpp

8906

I guess this is personal preference, but it seems more natural to use [&] and not pass the context. Maybe different names from LHSType and RHSType would be better for the nested function, since it's called with both orders.

8914

Is the isVector just being defensive? I'd have expected it to be redundant, since we shouldn't create SveFixedLengthPredicateVectors with invalid element types.

8917

It looks like this could still trigger for SveBools. Maybe it would be better to have:

if (BT->getKind() == BuiltinType::SveBool) {
  …
} else {
  …
}

instead. Perhaps it'd also be worth having an assert to show that we've already checked that any builtin type is an SVE type.

9912

Seems like it might be useful to have a helper function for testing this pair of vector kinds.

[c-rhodes]

Address @rsandifo-arm comments.

[c-rhodes]

clang/lib/AST/ASTContext.cpp

1941

The alignment of the SVE data vectors should be 128 too (see the alignof stuff in the ACLE doc).

Ah, good point. This would've bitten us when when we come to supporting non-power-of-2 vector lengths.

clang/lib/AST/Type.cpp

2338

Thanks for the heads up! I'm not familiar with ILP32 so that's good to know. So if -mabi=ilp32 is passed then I guess we should handle that here. I'll make a note of this and address it in a later patch.

clang/lib/Sema/SemaExpr.cpp

8906

I guess this is personal preference, but it seems more natural to use [&] and not pass the context.

Agreed, that's nicer.

Maybe different names from LHSType and RHSType would be better for the nested function, since it's called with both orders.

I've renamed them to first/second.

8914

Is the isVector just being defensive? I'd have expected it to be redundant, since we shouldn't create SveFixedLengthPredicateVectors with invalid element types.

Yeah although as you say it's not really necessary, fixed.

8917

I've simplified this somewhat although I'm not sure if it's what you meant. If it's a predicate vector kind then it's compatible if the builtin is an SVE bool, otherwise it compares the types. Worth noting I've moved this to ASTContext also since it's used for the C++ conversions.

[rsandifo-arm]

LGTM from a spec point of view, but I don't think I should be the one to approve.

[c-rhodes]

ping

[aaron.ballman]

LGTM aside from some minor nits.

clang/include/clang/AST/Type.h

3228

Add a bit of vertical whitespace for consistency with surrounding enumerators?

clang/include/clang/Basic/DiagnosticSemaKinds.td

2929

vector -> vectors
(alternatively, a fixed-length and a sizeless vector)

clang/lib/AST/ASTContext.cpp

1945

else if?

8509

This only needs to capture this and not reference capture everything, right? I'd prefer that (esp given that the lambda parameters shadow the names of the function parameters).

clang/lib/Sema/SemaExpr.cpp

9911

No need for the parameters to be non-const references (you can use value types here, they're cheap).

clang/lib/Sema/SemaOverload.cpp

1656

This can be hoisted into the preceding if statement.

clang/lib/Sema/SemaType.cpp

7867

Full stop at the end of the comment.

[c-rhodes]

Address comments

[c-rhodes]

@rsandifo-arm @aaron.ballman thanks for reviewing!

clang/include/clang/AST/ASTContext.h

public:
 
   /// Return true if the given vector types are of the same unqualified
   /// type or if they are equivalent to the same GCC vector type.
   ///
   /// \note This ignores whether they are target-specific (AltiVec or Neon)
   /// types.
   bool areCompatibleVectorTypes(QualType FirstVec, QualType SecondVec);
 
+  /// Return true if the given types are an SVE builtin and a VectorType that
+  /// is a fixed-length representation of the SVE builtin for a specific
+  /// vector-length.
+  bool areCompatibleSveTypes(QualType FirstType, QualType SecondType);
+
   /// Return true if the type has been explicitly qualified with ObjC ownership.
   /// A type may be implicitly qualified with ownership under ObjC ARC, and in
   /// some cases the compiler treats these differently.
   bool hasDirectOwnershipQualifier(QualType Ty) const;
 
   /// Return true if this is an \c NSObject object with its \c NSObject
   /// attribute set.
   static bool isObjCNSObjectType(QualType Ty) {
if (Ty->isIncompleteType() || Ty->isDependentType())
       return None;
     return getTypeSizeInChars(Ty);
   }
 
   Optional<CharUnits> getTypeSizeInCharsIfKnown(const Type *Ty) const {
     return getTypeSizeInCharsIfKnown(QualType(Ty, 0));
   }
 
-  /// Returns the bitwidth of \p T, an SVE type attributed with
-  /// 'arm_sve_vector_bits'. Should only be called if T->isVLST().
-  unsigned getBitwidthForAttributedSveType(const Type *T) const;
-
   /// Return the ABI-specified alignment of a (complete) type \p T, in
   /// bits.
   unsigned getTypeAlign(QualType T) const { return getTypeInfo(T).Align; }
   unsigned getTypeAlign(const Type *T) const { return getTypeInfo(T).Align; }
 
   /// Return the ABI-specified natural alignment of a (complete) type \p T,
   /// before alignment adjustments, in bits.
   ///

clang/include/clang/AST/Type.h

public:
   /// every type is one or the other.  Standard types are all sized;
   /// sizeless types are purely an extension.
   ///
   /// Sizeless types contain data with no specified size, alignment,
   /// or layout.
   bool isSizelessType() const;
   bool isSizelessBuiltinType() const;
 
-  /// Determines if this is a vector-length-specific type (VLST), i.e. a
-  /// sizeless type with the 'arm_sve_vector_bits' attribute applied.
-  bool isVLST() const;
   /// Determines if this is a sizeless type supported by the
   /// 'arm_sve_vector_bits' type attribute, which can be applied to a single
   /// SVE vector or predicate, excluding tuple types such as svint32x4_t.
   bool isVLSTBuiltinType() const;
 
+  /// Returns the representative type for the element of an SVE builtin type.
+  /// This is used to represent fixed-length SVE vectors created with the
+  /// 'arm_sve_vector_bits' type attribute as VectorType.
+  QualType getSveEltType(const ASTContext &Ctx) const;
+
   /// Types are partitioned into 3 broad categories (C99 6.2.5p1):
   /// object types, function types, and incomplete types.
 
   /// Return true if this is an incomplete type.
   /// A type that can describe objects, but which lacks information needed to
   /// determine its size (e.g. void, or a fwd declared struct). Clients of this
   /// routine will need to determine if the size is actually required.
   ///
enum VectorKind {
 
     /// is AltiVec 'vector bool ...'
     AltiVecBool,
 
     /// is ARM Neon vector
     NeonVector,
 
     /// is ARM Neon polynomial vector
-    NeonPolyVector
+    NeonPolyVector,
+
+    /// is AArch64 SVE fixed-length data vector
+    SveFixedLengthDataVector,
+
+    /// is AArch64 SVE fixed-length predicate vector
+    SveFixedLengthPredicateVector
   };
 
 protected:
   friend class ASTContext; // ASTContext creates these.
 
   /// The element type of the vector.
   QualType ElementType;
 

clang/include/clang/Basic/Attr.td

 }
 
 def ArmSveVectorBits : TypeAttr {
   let Spellings = [GNU<"arm_sve_vector_bits">];
   let Subjects = SubjectList<[TypedefName], ErrorDiag>;
   let Args = [UnsignedArgument<"NumBits">];
   let Documentation = [ArmSveVectorBitsDocs];
   let PragmaAttributeSupport = 0;
+  // Represented as VectorType instead.
+  let ASTNode = 0;
 }
 
 def ArmMveStrictPolymorphism : TypeAttr, TargetSpecificAttr<TargetARM> {
   let Spellings = [Clang<"__clang_arm_mve_strict_polymorphism">];
   let Documentation = [ArmMveStrictPolymorphismDocs];
 }
 
 def NoUniqueAddress : InheritableAttr, TargetSpecificAttr<TargetItaniumCXXABI> {

clang/include/clang/Basic/DiagnosticSemaKinds.td

 def err_callback_implicit_this_not_available : Error<
   "'callback' argument at position %0 references unavailable implicit 'this'">;
 def err_init_method_bad_return_type : Error<
   "init methods must return an object pointer type, not %0">;
 def err_attribute_invalid_size : Error<
   "vector size not an integral multiple of component size">;
 def err_attribute_zero_size : Error<"zero %0 size">;
 def err_attribute_size_too_large : Error<"%0 size too large">;
+def err_typecheck_vector_not_convertable_sizeless : Error<
+  "cannot convert between a fixed-length and a sizeless vector (%0 and %1)">;
 def err_typecheck_vector_not_convertable_implict_truncation : Error<
    "cannot convert between %select{scalar|vector}0 type %1 and vector type"
    " %2 as implicit conversion would cause truncation">;
 def err_typecheck_vector_not_convertable : Error<
   "cannot convert between vector values of different size (%0 and %1)">;
 def err_typecheck_vector_not_convertable_non_scalar : Error<
   "cannot convert between vector and non-scalar values (%0 and %1)">;
 def err_typecheck_vector_lengths_not_equal : Error<

clang/include/clang/Sema/Overload.h

enum ImplicitConversionKind {
     ICK_Compatible_Conversion,
 
     /// Derived-to-base (C++ [over.best.ics])
     ICK_Derived_To_Base,
 
     /// Vector conversions
     ICK_Vector_Conversion,
 
+    /// Arm SVE Vector conversions
+    ICK_SVE_Vector_Conversion,
+
     /// A vector splat from an arithmetic type
     ICK_Vector_Splat,
 
     /// Complex-real conversions (C99 6.3.1.7)
     ICK_Complex_Real,
 
     /// Block Pointer conversions
     ICK_Block_Pointer_Conversion,

clang/include/clang/Sema/Sema.h

bool RequireCompleteType(SourceLocation Loc, QualType T, unsigned DiagID,
     BoundTypeDiagnoser<Ts...> Diagnoser(DiagID, Args...);
     return RequireCompleteType(Loc, T, Diagnoser);
   }
 
   template <typename... Ts>
   bool RequireCompleteSizedType(SourceLocation Loc, QualType T, unsigned DiagID,
                                 const Ts &... Args) {
     SizelessTypeDiagnoser<Ts...> Diagnoser(DiagID, Args...);
-    CompleteTypeKind Kind = CompleteTypeKind::Normal;
-    if (T->isVLST())
-      Kind = CompleteTypeKind::AcceptSizeless;
-    return RequireCompleteType(Loc, T, Kind, Diagnoser);
+    return RequireCompleteType(Loc, T, CompleteTypeKind::Normal, Diagnoser);
   }
 
   void completeExprArrayBound(Expr *E);
   bool RequireCompleteExprType(Expr *E, CompleteTypeKind Kind,
                                TypeDiagnoser &Diagnoser);
   bool RequireCompleteExprType(Expr *E, unsigned DiagID);
 
   template <typename... Ts>
   bool RequireCompleteExprType(Expr *E, unsigned DiagID, const Ts &...Args) {
     BoundTypeDiagnoser<Ts...> Diagnoser(DiagID, Args...);
     return RequireCompleteExprType(E, CompleteTypeKind::Default, Diagnoser);
   }
 
   template <typename... Ts>
   bool RequireCompleteSizedExprType(Expr *E, unsigned DiagID,
                                     const Ts &... Args) {
     SizelessTypeDiagnoser<Ts...> Diagnoser(DiagID, Args...);
-    CompleteTypeKind Kind = CompleteTypeKind::Normal;
-    if (E->getType()->isVLST())
-      Kind = CompleteTypeKind::AcceptSizeless;
-    return RequireCompleteExprType(E, Kind, Diagnoser);
+    return RequireCompleteExprType(E, CompleteTypeKind::Normal, Diagnoser);
   }
 
   bool RequireLiteralType(SourceLocation Loc, QualType T,
                           TypeDiagnoser &Diagnoser);
   bool RequireLiteralType(SourceLocation Loc, QualType T, unsigned DiagID);
 
   template <typename... Ts>
   bool RequireLiteralType(SourceLocation Loc, QualType T, unsigned DiagID,

clang/lib/AST/ASTContext.cpp

if (I != MemoizedTypeInfo.end())
     return I->second;
 
   // This call can invalidate MemoizedTypeInfo[T], so we need a second lookup.
   TypeInfo TI = getTypeInfoImpl(T);
   MemoizedTypeInfo[T] = TI;
   return TI;
 }
 
-static unsigned getSveVectorWidth(const Type *T) {
-  // Get the vector size from the 'arm_sve_vector_bits' attribute via the
-  // AttributedTypeLoc associated with the typedef decl.
-  if (const auto *TT = T->getAs<TypedefType>()) {
-    const TypedefNameDecl *Typedef = TT->getDecl();
-    TypeSourceInfo *TInfo = Typedef->getTypeSourceInfo();
-    TypeLoc TL = TInfo->getTypeLoc();
-    if (AttributedTypeLoc ATL = TL.getAs<AttributedTypeLoc>())
-      if (const auto *Attr = ATL.getAttrAs<ArmSveVectorBitsAttr>())
-        return Attr->getNumBits();
-  }
-
-  llvm_unreachable("bad 'arm_sve_vector_bits' attribute!");
-}
-
-static unsigned getSvePredWidth(const ASTContext &Context, const Type *T) {
-  return getSveVectorWidth(T) / Context.getCharWidth();
-}
-
-unsigned ASTContext::getBitwidthForAttributedSveType(const Type *T) const {
-  assert(T->isVLST() &&
-         "getBitwidthForAttributedSveType called for non-attributed type!");
-
-  switch (T->castAs<BuiltinType>()->getKind()) {
-  default:
-    llvm_unreachable("unknown builtin type!");
-  case BuiltinType::SveInt8:
-  case BuiltinType::SveInt16:
-  case BuiltinType::SveInt32:
-  case BuiltinType::SveInt64:
-  case BuiltinType::SveUint8:
-  case BuiltinType::SveUint16:
-  case BuiltinType::SveUint32:
-  case BuiltinType::SveUint64:
-  case BuiltinType::SveFloat16:
-  case BuiltinType::SveFloat32:
-  case BuiltinType::SveFloat64:
-  case BuiltinType::SveBFloat16:
-    return getSveVectorWidth(T);
-  case BuiltinType::SveBool:
-    return getSvePredWidth(*this, T);
-  }
-}
-
 /// getTypeInfoImpl - Return the size of the specified type, in bits.  This
 /// method does not work on incomplete types.
 ///
 /// FIXME: Pointers into different addr spaces could have different sizes and
 /// alignment requirements: getPointerInfo should take an AddrSpace, this
 /// should take a QualType, &c.
 TypeInfo ASTContext::getTypeInfoImpl(const Type *T) const {
   uint64_t Width = 0;
case Type::Vector: {
     if (Align & (Align-1)) {
       Align = llvm::NextPowerOf2(Align);
       Width = llvm::alignTo(Width, Align);
     }
     // Adjust the alignment based on the target max.
     uint64_t TargetVectorAlign = Target->getMaxVectorAlign();
     if (TargetVectorAlign && TargetVectorAlign < Align)
       Align = TargetVectorAlign;
+    if (VT->getVectorKind() == VectorType::SveFixedLengthDataVector)
+      // Adjust the alignment for fixed-length SVE vectors. This is important
+      // for non-power-of-2 vector lengths.
+      Align = 128;
+    else if (VT->getVectorKind() == VectorType::SveFixedLengthPredicateVector)
+      // Adjust the alignment for fixed-length SVE predicates.
+      Align = 16;
     break;
   }
 
   case Type::ConstantMatrix: {
     const auto *MT = cast<ConstantMatrixType>(T);
     TypeInfo ElementInfo = getTypeInfo(MT->getElementType());
     // The internal layout of a matrix value is implementation defined.
     // Initially be ABI compatible with arrays with respect to alignment and
case Type::Typedef: {
     // attribute(aligned) can only round up) but matches its implementation.
     if (unsigned AttrAlign = Typedef->getMaxAlignment()) {
       Align = AttrAlign;
       AlignIsRequired = true;
     } else {
       Align = Info.Align;
       AlignIsRequired = Info.AlignIsRequired;
     }
-    if (T->isVLST())
-      Width = getBitwidthForAttributedSveType(T);
-    else
-      Width = Info.Width;
     break;
   }
 
   case Type::Elaborated:
     return getTypeInfo(cast<ElaboratedType>(T)->getNamedType().getTypePtr());
 
   case Type::Attributed:
     return getTypeInfo(
if (First->getNumElements() == Second->getNumElements() &&
       First->getVectorKind() != VectorType::AltiVecBool &&
       Second->getVectorKind() != VectorType::AltiVecPixel &&
       Second->getVectorKind() != VectorType::AltiVecBool)
     return true;
 
   return false;
 }
 
+bool ASTContext::areCompatibleSveTypes(QualType FirstType,
+                                       QualType SecondType) {
+  assert(((FirstType->isSizelessBuiltinType() && SecondType->isVectorType()) ||
+          (FirstType->isVectorType() && SecondType->isSizelessBuiltinType())) &&
+         "Expected SVE builtin type and vector type!");
+
+  auto IsValidCast = [this](QualType FirstType, QualType SecondType) {
+    if (const auto *BT = FirstType->getAs<BuiltinType>()) {
+      if (const auto *VT = SecondType->getAs<VectorType>()) {
+        // Predicates have the same representation as uint8 so we also have to
+        // check the kind to make these types incompatible.
+        if (VT->getVectorKind() == VectorType::SveFixedLengthPredicateVector)
+          return BT->getKind() == BuiltinType::SveBool;
+        else if (VT->getVectorKind() == VectorType::SveFixedLengthDataVector)
+          return VT->getElementType().getCanonicalType() ==
+                 FirstType->getSveEltType(*this);
+      }
+    }
+    return false;
+  };
+
+  return IsValidCast(FirstType, SecondType) ||
+         IsValidCast(SecondType, FirstType);
+}
+
 bool ASTContext::hasDirectOwnershipQualifier(QualType Ty) const {
   while (true) {
     // __strong id
     if (const AttributedType *Attr = dyn_cast<AttributedType>(Ty)) {
       if (Attr->getAttrKind() == attr::ObjCOwnership)
         return true;
 
       Ty = Attr->getModifiedType();

clang/lib/AST/JSONNodeDumper.cpp

case VectorType::AltiVecBool:
     JOS.attribute("vectorKind", "altivec bool");
     break;
   case VectorType::NeonVector:
     JOS.attribute("vectorKind", "neon");
     break;
   case VectorType::NeonPolyVector:
     JOS.attribute("vectorKind", "neon poly");
     break;
+  case VectorType::SveFixedLengthDataVector:
+    JOS.attribute("vectorKind", "fixed-length sve data vector");
+    break;
+  case VectorType::SveFixedLengthPredicateVector:
+    JOS.attribute("vectorKind", "fixed-length sve predicate vector");
+    break;
   }
 }
 
 void JSONNodeDumper::VisitUnresolvedUsingType(const UnresolvedUsingType *UUT) {
   JOS.attribute("decl", createBareDeclRef(UUT->getDecl()));
 }
 
 void JSONNodeDumper::VisitUnaryTransformType(const UnaryTransformType *UTT) {

clang/lib/AST/TextNodeDumper.cpp

case VectorType::AltiVecBool:
     OS << " altivec bool";
     break;
   case VectorType::NeonVector:
     OS << " neon";
     break;
   case VectorType::NeonPolyVector:
     OS << " neon poly";
     break;
+  case VectorType::SveFixedLengthDataVector:
+    OS << " fixed-length sve data vector";
+    break;
+  case VectorType::SveFixedLengthPredicateVector:
+    OS << " fixed-length sve predicate vector";
+    break;
   }
   OS << " " << T->getNumElements();
 }
 
 void TextNodeDumper::VisitFunctionType(const FunctionType *T) {
   auto EI = T->getExtInfo();
   if (EI.getNoReturn())
     OS << " noreturn";

clang/lib/AST/Type.cpp

case BuiltinType::SveBool:
       return true;
     default:
       return false;
     }
   }
   return false;
 }
 
-bool Type::isVLST() const {
-  if (!isVLSTBuiltinType())
-    return false;
+QualType Type::getSveEltType(const ASTContext &Ctx) const {
+  assert(isVLSTBuiltinType() && "unsupported type!");
 
-  return hasAttr(attr::ArmSveVectorBits);
+  const BuiltinType *BTy = getAs<BuiltinType>();
+  switch (BTy->getKind()) {
+  default:
+    llvm_unreachable("Unknown builtin SVE type!");
+  case BuiltinType::SveInt8:
+    return Ctx.SignedCharTy;
+  case BuiltinType::SveUint8:
+  case BuiltinType::SveBool:
+    // Represent predicates as i8 rather than i1 to avoid any layout issues.
+    // The type is bitcasted to a scalable predicate type when casting between
+    // scalable and fixed-length vectors.
+    return Ctx.UnsignedCharTy;
+  case BuiltinType::SveInt16:
+    return Ctx.ShortTy;
+  case BuiltinType::SveUint16:
+    return Ctx.UnsignedShortTy;
+  case BuiltinType::SveInt32:
+    return Ctx.IntTy;
+  case BuiltinType::SveUint32:
+    return Ctx.UnsignedIntTy;
+  case BuiltinType::SveInt64:
+    return Ctx.LongTy;
+  case BuiltinType::SveUint64:
+    return Ctx.UnsignedLongTy;
+  case BuiltinType::SveFloat16:
+    return Ctx.Float16Ty;
+  case BuiltinType::SveBFloat16:
+    return Ctx.BFloat16Ty;
+  case BuiltinType::SveFloat32:
+    return Ctx.FloatTy;
+  case BuiltinType::SveFloat64:
+    return Ctx.DoubleTy;
+  }
 }
 
 bool QualType::isPODType(const ASTContext &Context) const {
   // C++11 has a more relaxed definition of POD.
   if (Context.getLangOpts().CPlusPlus11)
     return isCXX11PODType(Context);
 
   return isCXX98PODType(Context);

clang/lib/AST/TypePrinter.cpp

case VectorType::GenericVector: {
     OS << "__attribute__((__vector_size__("
        << T->getNumElements()
        << " * sizeof(";
     print(T->getElementType(), OS, StringRef());
     OS << ")))) ";
     printBefore(T->getElementType(), OS);
     break;
   }
+  case VectorType::SveFixedLengthDataVector:
+  case VectorType::SveFixedLengthPredicateVector:
+    // FIXME: We prefer to print the size directly here, but have no way
+    // to get the size of the type.
+    OS << "__attribute__((__arm_sve_vector_bits__(";
+
+    if (T->getVectorKind() == VectorType::SveFixedLengthPredicateVector)
+      // Predicates take a bit per byte of the vector size, multiply by 8 to
+      // get the number of bits passed to the attribute.
+      OS << T->getNumElements() * 8;
+    else
+      OS << T->getNumElements();
+
+    OS << " * sizeof(";
+    print(T->getElementType(), OS, StringRef());
+    // Multiply by 8 for the number of bits.
+    OS << ") * 8))) ";
+    printBefore(T->getElementType(), OS);
   }
 }
 
 void TypePrinter::printVectorAfter(const VectorType *T, raw_ostream &OS) {
   printAfter(T->getElementType(), OS);
 }
 
 void TypePrinter::printDependentVectorBefore(
case VectorType::GenericVector: {
     if (T->getSizeExpr())
       T->getSizeExpr()->printPretty(OS, nullptr, Policy);
     OS << " * sizeof(";
     print(T->getElementType(), OS, StringRef());
     OS << ")))) ";
     printBefore(T->getElementType(), OS);
     break;
   }
+  case VectorType::SveFixedLengthDataVector:
+  case VectorType::SveFixedLengthPredicateVector:
+    // FIXME: We prefer to print the size directly here, but have no way
+    // to get the size of the type.
+    OS << "__attribute__((__arm_sve_vector_bits__(";
+    if (T->getSizeExpr()) {
+      T->getSizeExpr()->printPretty(OS, nullptr, Policy);
+      if (T->getVectorKind() == VectorType::SveFixedLengthPredicateVector)
+        // Predicates take a bit per byte of the vector size, multiply by 8 to
+        // get the number of bits passed to the attribute.
+        OS << " * 8";
+      OS << " * sizeof(";
+      print(T->getElementType(), OS, StringRef());
+      // Multiply by 8 for the number of bits.
+      OS << ") * 8";
+    }
+    OS << "))) ";
+    printBefore(T->getElementType(), OS);
   }
 }
 
 void TypePrinter::printDependentVectorAfter(
     const DependentVectorType *T, raw_ostream &OS) {
   printAfter(T->getElementType(), OS);
 }
 
case attr::NoDeref:
     OS << "noderef";
     break;
   case attr::AcquireHandle:
     OS << "acquire_handle";
     break;
   case attr::ArmMveStrictPolymorphism:
     OS << "__clang_arm_mve_strict_polymorphism";
     break;
-  case attr::ArmSveVectorBits:
-    OS << "arm_sve_vector_bits";
-    break;
   }
   OS << "))";
 }
 
 void TypePrinter::printObjCInterfaceBefore(const ObjCInterfaceType *T,
                                            raw_ostream &OS) {
   OS << T->getDecl()->getName();
   spaceBeforePlaceHolder(OS);

clang/lib/Sema/SemaDecl.cpp

void Sema::CheckVariableDeclarationType(VarDecl *NewVD) {
   }
 
   if (!NewVD->hasLocalStorage() && NewVD->hasAttr<BlocksAttr>()) {
     Diag(NewVD->getLocation(), diag::err_block_on_nonlocal);
     NewVD->setInvalidDecl();
     return;
   }
 
-  if (!NewVD->hasLocalStorage() && T->isSizelessType() && !T->isVLST()) {
+  if (!NewVD->hasLocalStorage() && T->isSizelessType()) {
     Diag(NewVD->getLocation(), diag::err_sizeless_nonlocal) << T;
     NewVD->setInvalidDecl();
     return;
   }
 
   if (isVM && NewVD->hasAttr<BlocksAttr>()) {
     Diag(NewVD->getLocation(), diag::err_block_on_vm);
     NewVD->setInvalidDecl();

clang/lib/Sema/SemaExpr.cpp

if (const VectorType *VT = QT->getAs<VectorType>())
     return VT->getElementType().getCanonicalType() == ElementType;
   return false;
 }
 
 /// CheckAssignmentConstraints (C99 6.5.16) - This routine currently
 /// has code to accommodate several GCC extensions when type checking
 /// pointers. Here are some objectionable examples that GCC considers warnings:
 ///
 ///  int a, *pint;
 ///  short *pshort;
 ///  struct foo *pfoo;
 ///
 ///  pint = pshort; // warning: assignment from incompatible pointer type
 ///  a = pint; // warning: assignment makes integer from pointer without a cast
 ///  pint = a; // warning: assignment makes pointer from integer without a cast
 ///  pint = pfoo; // warning: assignment from incompatible pointer type
 ///
 /// As a result, the code for dealing with pointers is more complex than the
 /// C99 spec dictates.
 ///
 /// Sets 'Kind' for any result kind except Incompatible.
 Sema::AssignConvertType
 Sema::CheckAssignmentConstraints(QualType LHSType, ExprResult &RHS,
                                  CastKind &Kind, bool ConvertRHS) {
   QualType RHSType = RHS.get()->getType();
   QualType OrigLHSType = LHSType;
 
   // Get canonical types.  We're not formatting these types, just comparing
if (LHSType->isScalarType()) {
           isLaxVectorConversion(RHSType, LHSType)) {
         ExprResult *VecExpr = &RHS;
         *VecExpr = ImpCastExprToType(VecExpr->get(), LHSType, CK_BitCast);
         Kind = CK_BitCast;
         return Compatible;
       }
     }
 
+    // Allow assignments between fixed-length and sizeless SVE vectors.
+    if (((LHSType->isSizelessBuiltinType() && RHSType->isVectorType()) ||
+         (LHSType->isVectorType() && RHSType->isSizelessBuiltinType())) &&
+        Context.areCompatibleSveTypes(LHSType, RHSType)) {
+      Kind = CK_BitCast;
+      return Compatible;
+    }
+
     return Incompatible;
   }
 
   // Diagnose attempts to convert between __float128 and long double where
   // such conversions currently can't be handled.
   if (unsupportedTypeConversion(*this, LHSType, RHSType))
     return Incompatible;
 
// <1 x T> -> T. The result is also a vector type.
       ExprResult *RHSExpr = &RHS;
       *RHSExpr = ImpCastExprToType(RHSExpr->get(), LHSType, CK_BitCast);
       return VecType;
     }
   }
 
   // Okay, the expression is invalid.
 
+  // Returns true if the operands are SVE VLA and VLS types.
+  auto IsSveConversion = [](QualType FirstType, QualType SecondType) {
+    const VectorType *VecType = SecondType->getAs<VectorType>();
+    return FirstType->isSizelessBuiltinType() && VecType &&
+           (VecType->getVectorKind() == VectorType::SveFixedLengthDataVector ||
+            VecType->getVectorKind() ==
+                VectorType::SveFixedLengthPredicateVector);
+  };
+
+  // If there's a sizeless and fixed-length operand, diagnose that.
+  if (IsSveConversion(LHSType, RHSType) || IsSveConversion(RHSType, LHSType)) {
+    Diag(Loc, diag::err_typecheck_vector_not_convertable_sizeless)
+        << LHSType << RHSType;
+    return QualType();
+  }
+
   // If there's a non-vector, non-real operand, diagnose that.
   if ((!RHSVecType && !RHSType->isRealType()) ||
       (!LHSVecType && !LHSType->isRealType())) {
     Diag(Loc, diag::err_typecheck_vector_not_convertable_non_scalar)
       << LHSType << RHSType
       << LHS.get()->getSourceRange() << RHS.get()->getSourceRange();
     return QualType();
   }

clang/lib/Sema/SemaExprCXX.cpp

case ICK_Derived_To_Base: {
     break;
   }
 
   case ICK_Vector_Conversion:
     From = ImpCastExprToType(From, ToType, CK_BitCast,
                              VK_RValue, /*BasePath=*/nullptr, CCK).get();
     break;
 
+  case ICK_SVE_Vector_Conversion:
+    From = ImpCastExprToType(From, ToType, CK_BitCast, VK_RValue,
+                             /*BasePath=*/nullptr, CCK)
+               .get();
+    break;
+
   case ICK_Vector_Splat: {
     // Vector splat from any arithmetic type to a vector.
     Expr *Elem = prepareVectorSplat(ToType, From).get();
     From = ImpCastExprToType(Elem, ToType, CK_VectorSplat, VK_RValue,
                              /*BasePath=*/nullptr, CCK).get();
     break;
   }
 

clang/lib/Sema/SemaOverload.cpp

static const ImplicitConversionRank
     ICR_Conversion,
     ICR_Conversion,
     ICR_Conversion,
     ICR_Conversion,
     ICR_Conversion,
     ICR_Conversion,
     ICR_Conversion,
     ICR_Conversion,
+    ICR_Conversion,
     ICR_OCL_Scalar_Widening,
     ICR_Complex_Real_Conversion,
     ICR_Conversion,
     ICR_Conversion,
     ICR_Writeback_Conversion,
     ICR_Exact_Match, // NOTE(gbiv): This may not be completely right --
                      // it was omitted by the patch that added
                      // ICK_Zero_Event_Conversion
static const char* const Name[(int)ICK_Num_Conversion_Kinds] = {
     "Complex conversion",
     "Floating-integral conversion",
     "Pointer conversion",
     "Pointer-to-member conversion",
     "Boolean conversion",
     "Compatible-types conversion",
     "Derived-to-base conversion",
     "Vector conversion",
+    "SVE Vector conversion",
     "Vector splat",
     "Complex-real conversion",
     "Block Pointer conversion",
     "Transparent Union Conversion",
     "Writeback conversion",
     "OpenCL Zero Event Conversion",
     "C specific type conversion",
     "Incompatible pointer conversion"
if (ToType->isExtVectorType()) {
 
     // Vector splat from any arithmetic type to a vector.
     if (FromType->isArithmeticType()) {
       ICK = ICK_Vector_Splat;
       return true;
     }
   }
 
+  if ((ToType->isSizelessBuiltinType() || FromType->isSizelessBuiltinType()) &&
+      S.Context.areCompatibleSveTypes(FromType, ToType)) {
+    ICK = ICK_SVE_Vector_Conversion;
+    return true;
+  }
+
   // We can perform the conversion between vector types in the following cases:
   // 1)vector types are equivalent AltiVec and GCC vector types
   // 2)lax vector conversions are permitted and the vector types are of the
   //   same size
   // 3)the destination type does not have the ARM MVE strict-polymorphism
   //   attribute, which inhibits lax vector conversion for overload resolution
   //   only
   if (ToType->isVectorType() && FromType->isVectorType()) {
bool SCS2IsCompatibleVectorConversion = S.Context.areCompatibleVectorTypes(
         SCS2.getFromType(), SCS2.getToType(2));
 
     if (SCS1IsCompatibleVectorConversion != SCS2IsCompatibleVectorConversion)
       return SCS1IsCompatibleVectorConversion
                  ? ImplicitConversionSequence::Better
                  : ImplicitConversionSequence::Worse;
   }
 
+  if (SCS1.Second == ICK_SVE_Vector_Conversion &&
+      SCS2.Second == ICK_SVE_Vector_Conversion) {
+    bool SCS1IsCompatibleSVEVectorConversion =
+        S.Context.areCompatibleSveTypes(SCS1.getFromType(), SCS1.getToType(2));
+    bool SCS2IsCompatibleSVEVectorConversion =
+        S.Context.areCompatibleSveTypes(SCS2.getFromType(), SCS2.getToType(2));
+
+    if (SCS1IsCompatibleSVEVectorConversion !=
+        SCS2IsCompatibleSVEVectorConversion)
+      return SCS1IsCompatibleSVEVectorConversion
+                 ? ImplicitConversionSequence::Better
+                 : ImplicitConversionSequence::Worse;
+  }
+
   return ImplicitConversionSequence::Indistinguishable;
 }
 
 /// CompareQualificationConversions - Compares two standard conversion
 /// sequences to determine whether they can be ranked based on their
 /// qualification conversions (C++ 13.3.3.2p3 bullet 3).
 static ImplicitConversionSequence::CompareKind
 CompareQualificationConversions(Sema &S,
static bool CheckConvertedConstantConversions(Sema &S,
   case ICK_Floating_Promotion:
   case ICK_Complex_Promotion:
   case ICK_Floating_Conversion:
   case ICK_Complex_Conversion:
   case ICK_Floating_Integral:
   case ICK_Compatible_Conversion:
   case ICK_Derived_To_Base:
   case ICK_Vector_Conversion:
+  case ICK_SVE_Vector_Conversion:
   case ICK_Vector_Splat:
   case ICK_Complex_Real:
   case ICK_Block_Pointer_Conversion:
   case ICK_TransparentUnionConversion:
   case ICK_Writeback_Conversion:
   case ICK_Zero_Event_Conversion:
   case ICK_C_Only_Conversion:
   case ICK_Incompatible_Pointer_Conversion:

clang/lib/Sema/SemaType.cpp

QualType Sema::BuildArrayType(QualType T, ArrayType::ArraySizeModifier ASM,
   } else {
     // C99 6.7.5.2p1: If the element type is an incomplete or function type,
     // reject it (e.g. void ary[7], struct foo ary[7], void ary[7]())
     if (RequireCompleteSizedType(Loc, T,
                                  diag::err_array_incomplete_or_sizeless_type))
       return QualType();
   }
 
-  if (T->isSizelessType() && !T->isVLST()) {
+  if (T->isSizelessType()) {
     Diag(Loc, diag::err_array_incomplete_or_sizeless_type) << 1 << T;
     return QualType();
   }
 
   if (T->isFunctionType()) {
     Diag(Loc, diag::err_illegal_decl_array_of_functions)
       << getPrintableNameForEntity(Entity) << T;
     return QualType();
static void HandleNeonVectorTypeAttr(QualType &CurType, const ParsedAttr &Attr,
   }
 
   CurType = S.Context.getVectorType(CurType, numElts, VecKind);
 }
 
 /// HandleArmSveVectorBitsTypeAttr - The "arm_sve_vector_bits" attribute is
 /// used to create fixed-length versions of sizeless SVE types defined by
 /// the ACLE, such as svint32_t and svbool_t.
-static void HandleArmSveVectorBitsTypeAttr(TypeProcessingState &State,
-                                           QualType &CurType,
-                                           ParsedAttr &Attr) {
-  Sema &S = State.getSema();
-  ASTContext &Ctx = S.Context;
-
+static void HandleArmSveVectorBitsTypeAttr(QualType &CurType, ParsedAttr &Attr,
+                                           Sema &S) {
   // Target must have SVE.
-  if (!Ctx.getTargetInfo().hasFeature("sve")) {
+  if (!S.Context.getTargetInfo().hasFeature("sve")) {
     S.Diag(Attr.getLoc(), diag::err_attribute_unsupported) << Attr;
     Attr.setInvalid();
     return;
   }
 
   // Attribute is unsupported if '-msve-vector-bits=<bits>' isn't specified.
   if (!S.getLangOpts().ArmSveVectorBits) {
     S.Diag(Attr.getLoc(), diag::err_attribute_arm_feature_sve_bits_unsupported)
static void HandleArmSveVectorBitsTypeAttr(QualType &CurType, ParsedAttr &Attr,
   // Attribute can only be attached to a single SVE vector or predicate type.
   if (!CurType->isVLSTBuiltinType()) {
     S.Diag(Attr.getLoc(), diag::err_attribute_invalid_sve_type)
         << Attr << CurType;
     Attr.setInvalid();
     return;
   }
 
-  auto *A = ::new (Ctx) ArmSveVectorBitsAttr(Ctx, Attr, VecSize);
-  CurType = State.getAttributedType(A, CurType, CurType);
+  const auto *BT = CurType->castAs<BuiltinType>();
+
+  QualType EltType = CurType->getSveEltType(S.Context);
+  unsigned TypeSize = S.Context.getTypeSize(EltType);
+  VectorType::VectorKind VecKind = VectorType::SveFixedLengthDataVector;
+  if (BT->getKind() == BuiltinType::SveBool) {
+    // Predicates are represented as i8.
+    VecSize /= S.Context.getCharWidth() * S.Context.getCharWidth();
+    VecKind = VectorType::SveFixedLengthPredicateVector;
+  } else
+    VecSize /= TypeSize;
+  CurType = S.Context.getVectorType(EltType, VecSize, VecKind);
 }
 
 static void HandleArmMveStrictPolymorphismAttr(TypeProcessingState &State,
                                                QualType &CurType,
                                                ParsedAttr &Attr) {
   const VectorType *VT = dyn_cast<VectorType>(CurType);
   if (!VT || VT->getVectorKind() != VectorType::NeonVector) {
     State.getSema().Diag(Attr.getLoc(),
case ParsedAttr::AT_NeonVectorType:
       attr.setUsedAsTypeAttr();
       break;
     case ParsedAttr::AT_NeonPolyVectorType:
       HandleNeonVectorTypeAttr(type, attr, state.getSema(),
                                VectorType::NeonPolyVector);
       attr.setUsedAsTypeAttr();
       break;
     case ParsedAttr::AT_ArmSveVectorBits:
-      HandleArmSveVectorBitsTypeAttr(state, type, attr);
+      HandleArmSveVectorBitsTypeAttr(type, attr, state.getSema());
       attr.setUsedAsTypeAttr();
       break;
     case ParsedAttr::AT_ArmMveStrictPolymorphism: {
       HandleArmMveStrictPolymorphismAttr(state, type, attr);
       attr.setUsedAsTypeAttr();
       break;
     }
     case ParsedAttr::AT_OpenCLAccess:

clang/test/Sema/attr-arm-sve-vector-bits.c

 int alignof_int8_var = __alignof__(*extern_int8_ptr);
 int alignof_int8_var_ptr = __alignof__(extern_int8_ptr);
 
 void f(int c) {
   fixed_int8_t fs8;
   svint8_t ss8;
 
   void *sel __attribute__((unused));
-  sel = c ? ss8 : fs8; // expected-error {{incompatible operand types ('svint8_t' (aka '__SVInt8_t') and 'fixed_int8_t' (aka '__SVInt8_t'))}}
-  sel = c ? fs8 : ss8; // expected-error {{incompatible operand types ('fixed_int8_t' (aka '__SVInt8_t') and 'svint8_t' (aka '__SVInt8_t'))}}
+  sel = c ? ss8 : fs8; // expected-error {{cannot convert between a fixed-length and a sizeless vector}}
+  sel = c ? fs8 : ss8; // expected-error {{cannot convert between a fixed-length and a sizeless vector}}
+
+  sel = fs8 + ss8; // expected-error {{cannot convert between a fixed-length and a sizeless vector}}
+  sel = ss8 + fs8; // expected-error {{cannot convert between a fixed-length and a sizeless vector}}
 }
 
 // --------------------------------------------------------------------------//
 // Sizeof
 
 #define VECTOR_SIZE ((N / 8))
 #define PRED_SIZE ((N / 64))
 

 TEST_CAST(uint64)
 TEST_CAST(float16)
 TEST_CAST(float32)
 TEST_CAST(float64)
 TEST_CAST(bfloat16)
 TEST_CAST(bool)
 
 // Test the implicit conversion only applies to valid types
-fixed_int8_t to_fixed_int8_t__from_svuint8_t(svuint8_t x) { return x; } // expected-error {{returning 'svuint8_t' (aka '__SVUint8_t') from a function with incompatible result type 'fixed_int8_t' (aka '__SVInt8_t')}}
-fixed_bool_t to_fixed_bool_t__from_svint32_t(svint32_t x) { return x; } // expected-error {{returning 'svint32_t' (aka '__SVInt32_t') from a function with incompatible result type 'fixed_bool_t' (aka '__SVBool_t')}}
+fixed_int8_t to_fixed_int8_t__from_svuint8_t(svuint8_t x) { return x; } // expected-error-re {{returning 'svuint8_t' (aka '__SVUint8_t') from a function with incompatible result type 'fixed_int8_t' (vector of {{[0-9]+}} 'signed char' values)}}
+fixed_bool_t to_fixed_bool_t__from_svint32_t(svint32_t x) { return x; } // expected-error-re {{returning 'svint32_t' (aka '__SVInt32_t') from a function with incompatible result type 'fixed_bool_t' (vector of {{[0-9]+}} 'unsigned char' values)}}
+
+// Test conversion between predicate and uint8 is invalid, both have the same
+// memory representation.
+fixed_bool_t to_fixed_bool_t__from_svuint8_t(svuint8_t x) { return x; } // expected-error-re {{returning 'svuint8_t' (aka '__SVUint8_t') from a function with incompatible result type 'fixed_bool_t' (vector of {{[0-9]+}} 'unsigned char' values)}}
 
 // Test the implicit conversion only applies to fixed-length types
 typedef signed int vSInt32 __attribute__((__vector_size__(16)));
-svint32_t to_svint32_t_from_gnut(vSInt32 x) { return x; } // expected-error {{returning 'vSInt32' (vector of 4 'int' values) from a function with incompatible result type 'svint32_t' (aka '__SVInt32_t')}}
+svint32_t to_svint32_t_from_gnut(vSInt32 x) { return x; } // expected-error-re {{returning 'vSInt32' (vector of {{[0-9]+}} 'int' values) from a function with incompatible result type 'svint32_t' (aka '__SVInt32_t')}}
 
-vSInt32 to_gnut_from_svint32_t(svint32_t x) { return x; } // expected-error {{returning 'svint32_t' (aka '__SVInt32_t') from a function with incompatible result type 'vSInt32' (vector of 4 'int' values)}}
+vSInt32 to_gnut_from_svint32_t(svint32_t x) { return x; } // expected-error-re {{returning 'svint32_t' (aka '__SVInt32_t') from a function with incompatible result type 'vSInt32' (vector of {{[0-9]+}} 'int' values)}}
 
 // --------------------------------------------------------------------------//
 // Test the scalable and fixed-length types can be used interchangeably
 
 svint32_t __attribute__((overloadable)) svfunc(svint32_t op1, svint32_t op2);
 svfloat64_t __attribute__((overloadable)) svfunc(svfloat64_t op1, svfloat64_t op2);
 svbool_t __attribute__((overloadable)) svfunc(svbool_t op1, svbool_t op2);
 

clang/test/SemaCXX/attr-arm-sve-vector-bits.cpp

+// RUN: %clang_cc1 -triple aarch64-none-linux-gnu -target-feature +sve -target-feature +bf16 -fsyntax-only -verify -std=c++11 -msve-vector-bits=512 -fallow-half-arguments-and-returns %s
+// expected-no-diagnostics
+
+#define N __ARM_FEATURE_SVE_BITS_EXPERIMENTAL
+
+typedef __SVInt8_t svint8_t;
+typedef svint8_t fixed_int8_t __attribute__((arm_sve_vector_bits(N)));
+
+template<typename T> struct S { T var; };
+
+S<fixed_int8_t> s;
+
+svint8_t to_svint8_t(fixed_int8_t x) { return x; }
+fixed_int8_t from_svint8_t(svint8_t x) { return x; }