[GlobalISel] Support vector-of-pointers in LLT

[kristof.beyls]

Summary

This fixes PR32471.

As comment 10 on that bug report highlights
(https://bugs.llvm.org//show_bug.cgi?id=32471#c10), there are quite a
few different defendable design tradeoffs that could be made, including
not representing pointers at all in LLT.

I decided to go for representing vector-of-pointer as a concept in LLT,
while keeping the size of the LLT type 64 bits (this is an increase from
48 bits before). My rationale for keeping pointers explicit is that on
some targets probably it's very handy to have the distinction between
pointer and non-pointer (e.g. 68K has a different register bank for
pointers IIRC). If we keep a scalar pointer, it probably is easiest to
also have a vector-of-pointers to keep LLT relatively conceptually clean
and orthogonal, while we don't have a very strong reason to break that
orthogonality. Once we gain more experience on the use of LLT, we can
of course reconsider this direction.

Rejecting vector-of-pointer types in the IRTranslator is also an option
to avoid the crash reported in PR32471, but that is only a very
short-term solution; also needs quite a bit of code tweaks in places,
and is probably fragile. Therefore I didn't consider this the best
option.

[ab]

You're right, this is probably the best path forward. A few nits inline

include/llvm/Support/LowLevelTypeImpl.h

45–46

Maybe:

return LLT{/*isPointer=*/false, /*isVector=*/false, /*NumElements=*/0, SizeInBits, /*AddressSpace=*/0};

Makes it a bit easier to read IMO

204

Maybe make this 16 bits to match the vector-of-pointer case? I'm a bit worried about it being different.

220

val -> Val (here and elsewhere)

233

Maybe add some static consts for the masks? You could then merge those with the comment explaining the layout.

252

Perhaps:

&& "Invalid type"

lib/Target/AArch64/AArch64RegisterBankInfo.cpp

489

Huh, how does an instruction survive this far with an invalid type?

test/CodeGen/AArch64/GlobalISel/arm64-fallback.ll

159

Immediately stale comment? ;)

[kristof.beyls]

update patch taking into account Ahmed's review feedback.

[kristof.beyls]

Thanks very much for the review Ahmed! I think I took all feedback into account in the new version of the patch. Please have another look.

include/llvm/Support/LowLevelTypeImpl.h

233

I experimented with a number of different ways to improve this.
The main issue here is that there isn't a way that is both portable and simple to have bitfield unions with full control on bitfield layout in C++ (at least not that I know off).
Apart from resorting to try and create a "union bitfield" ADT, I think either the original code, or the new code here are the most reasonable ways to encode this. I think these *Field descriptors in the new patch is slightly better than the original code I wrote with hard-coded masks etc., but only just.

lib/Target/AArch64/AArch64RegisterBankInfo.cpp

489

I see the following test failing here if this doesn't get changed: CodeGen/AArch64/GlobalISel/regbankselect-dbg-value.mir.
Before this patch, LLT::getSizeInBits() didn't assert on an invalid type, so the invalid type from that test case always seems to have ended up here.
IIRC, it was the following IR line that triggered this, more specifically the operand "debug-use _".

DBG_VALUE debug-use %0(s32), debug-use _, !7, !9, debug-location !10

I didn't investigate further.

[ab]

include/llvm/Support/LowLevelTypeImpl.h

233

The [2] are a bit weird, but it does make the bit magic more readable, thanks!

260

Not a big deal, but maybe use getMask() in the asserts too? (and make the message more generic, like 'AddressSpace too large')

lib/Target/AArch64/AArch64RegisterBankInfo.cpp

489

Ah, then I suspect this should ignore %noreg:

if (!MO.getReg())
  continue;

before trying to get the vreg type. Does that help?

540

This would probably also need to check that the vreg isn't 0.

[kristof.beyls]

Further tweaks based on Ahmed's feedback.

[kristof.beyls]

include/llvm/Support/LowLevelTypeImpl.h

233

Yeah, I agree the [2] makes it a bit weird.
I introduced a "typedef int BitFieldInfo[2];", so the [2] doesn't appear in lots of places to make the code read a bit more fluently.
I also renamed the BitFieldInfo variables to end with name "FieldInfo" rather than "Field", as they're not fields, but info about fields.

260

I intended to do that in the previous iteration of the patch, but forgot...
I decided to move this assert to the maskAndShift method, so that I didn't unnecessarily have to copy paste the check.

lib/Target/AArch64/AArch64RegisterBankInfo.cpp

489

Indeed, that was what was needed. Thanks for the pointer!

[ab]

LGTM, thanks for taking care of this!

include/llvm/Support/LowLevelTypeImpl.h

280

Maybe move these up with the other public members?

[kristof.beyls]

Move public methods to all be in the same location, as suggested by Ahmed, now that the implementation details are hidden more in private methods.

[kristof.beyls]

Committed in r300664

include/llvm/Support/LowLevelTypeImpl.h

 namespace llvm {
 
 class DataLayout;
 class Type;
 class raw_ostream;
 
 class LLT {
 public:
-  enum TypeKind : uint16_t {
-    Invalid,
-    Scalar,
-    Pointer,
-    Vector,
-  };
-
   /// Get a low-level scalar or aggregate "bag of bits".
   static LLT scalar(unsigned SizeInBits) {
     assert(SizeInBits > 0 && "invalid scalar size");
-    return LLT{Scalar, 1, SizeInBits};
+    return LLT{/*isPointer=*/false, /*isVector=*/false, /*NumElements=*/0,
+               SizeInBits, /*AddressSpace=*/0};
   }
 
   /// Get a low-level pointer in the given address space (defaulting to 0).
   static LLT pointer(uint16_t AddressSpace, unsigned SizeInBits) {
-    return LLT{Pointer, AddressSpace, SizeInBits};
+    assert(SizeInBits > 0 && "invalid pointer size");
+    return LLT{/*isPointer=*/true, /*isVector=*/false, /*NumElements=*/0,
+               SizeInBits, AddressSpace};
   }
 
   /// Get a low-level vector of some number of elements and element width.
   /// \p NumElements must be at least 2.
   static LLT vector(uint16_t NumElements, unsigned ScalarSizeInBits) {
     assert(NumElements > 1 && "invalid number of vector elements");
-    return LLT{Vector, NumElements, ScalarSizeInBits};
+    assert(ScalarSizeInBits > 0 && "invalid vector element size");
+    return LLT{/*isPointer=*/false, /*isVector=*/true, NumElements,
+               ScalarSizeInBits, /*AddressSpace=*/0};
   }
 
   /// Get a low-level vector of some number of elements and element type.
   static LLT vector(uint16_t NumElements, LLT ScalarTy) {
     assert(NumElements > 1 && "invalid number of vector elements");
-    assert(ScalarTy.isScalar() && "invalid vector element type");
-    return LLT{Vector, NumElements, ScalarTy.getSizeInBits()};
+    assert(!ScalarTy.isVector() && "invalid vector element type");
+    return LLT{ScalarTy.isPointer(), /*isVector=*/true, NumElements,
+               ScalarTy.getSizeInBits(),
+               ScalarTy.isPointer() ? ScalarTy.getAddressSpace() : 0};
   }
 
-  explicit LLT(TypeKind Kind, uint16_t NumElements, unsigned SizeInBits)
-    : SizeInBits(SizeInBits), ElementsOrAddrSpace(NumElements), Kind(Kind) {
-    assert((Kind != Vector || ElementsOrAddrSpace > 1) &&
-           "invalid number of vector elements");
+  explicit LLT(bool isPointer, bool isVector, uint16_t NumElements,
+               unsigned SizeInBits, unsigned AddressSpace) {
+    init(isPointer, isVector, NumElements, SizeInBits, AddressSpace);
   }
-
-  explicit LLT() : SizeInBits(0), ElementsOrAddrSpace(0), Kind(Invalid) {}
+  explicit LLT() : IsPointer(false), IsVector(false), RawData(0) {}
 
   explicit LLT(MVT VT);
 
-  bool isValid() const { return Kind != Invalid; }
+  bool isValid() const { return RawData != 0; }
 
-  bool isScalar() const { return Kind == Scalar; }
+  bool isScalar() const { return isValid() && !IsPointer && !IsVector; }
 
-  bool isPointer() const { return Kind == Pointer; }
+  bool isPointer() const { return isValid() && IsPointer && !IsVector; }
 
-  bool isVector() const { return Kind == Vector; }
+  bool isVector() const { return isValid() && IsVector; }
 
   /// Returns the number of elements in a vector LLT. Must only be called on
   /// vector types.
   uint16_t getNumElements() const {
-    assert(isVector() && "cannot get number of elements on scalar/aggregate");
-    return ElementsOrAddrSpace;
+    assert(IsVector && "cannot get number of elements on scalar/aggregate");
+    if (!IsPointer)
+      return getFieldValue(VectorElementsFieldInfo);
+    else
+      return getFieldValue(PointerVectorElementsFieldInfo);
   }
 
   /// Returns the total size of the type. Must only be called on sized types.
   unsigned getSizeInBits() const {
     if (isPointer() || isScalar())
-      return SizeInBits;
-    return SizeInBits * ElementsOrAddrSpace;
+      return getScalarSizeInBits();
+    return getScalarSizeInBits() * getNumElements();
   }
 
   unsigned getScalarSizeInBits() const {
-    return SizeInBits;
+    assert(RawData != 0 && "Invalid Type");
+    if (!IsVector) {
+      if (!IsPointer)
+        return getFieldValue(ScalarSizeFieldInfo);
+      else
+        return getFieldValue(PointerSizeFieldInfo);
+    } else {
+      if (!IsPointer)
+        return getFieldValue(VectorSizeFieldInfo);
+      else
+        return getFieldValue(PointerVectorSizeFieldInfo);
+    }
   }
 
   unsigned getAddressSpace() const {
-    assert(isPointer() && "cannot get address space of non-pointer type");
-    return ElementsOrAddrSpace;
+    assert(RawData != 0 && "Invalid Type");
+    assert(IsPointer && "cannot get address space of non-pointer type");
+    if (!IsVector)
+      return getFieldValue(PointerAddressSpaceFieldInfo);
+    else
+      return getFieldValue(PointerVectorAddressSpaceFieldInfo);
   }
 
   /// Returns the vector's element type. Only valid for vector types.
   LLT getElementType() const {
     assert(isVector() && "cannot get element type of scalar/aggregate");
-    return scalar(SizeInBits);
+    if (IsPointer)
+      return pointer(getAddressSpace(), getScalarSizeInBits());
+    else
+      return scalar(getScalarSizeInBits());
   }
 
   /// Get a low-level type with half the size of the original, by halving the
   /// size of the scalar type involved. For example `s32` will become `s16`,
   /// `<2 x s32>` will become `<2 x s16>`.
   LLT halfScalarSize() const {
-    assert(!isPointer() && getScalarSizeInBits() > 1 &&
+    assert(!IsPointer && getScalarSizeInBits() > 1 &&
            getScalarSizeInBits() % 2 == 0 && "cannot half size of this type");
-    return LLT{Kind, ElementsOrAddrSpace, SizeInBits / 2};
+    return LLT{/*isPointer=*/false, IsVector ? true : false,
+               IsVector ? getNumElements() : (uint16_t)0,
+               getScalarSizeInBits() / 2, /*AddressSpace=*/0};
   }
 
   /// Get a low-level type with twice the size of the original, by doubling the
   /// size of the scalar type involved. For example `s32` will become `s64`,
   /// `<2 x s32>` will become `<2 x s64>`.
   LLT doubleScalarSize() const {
-    assert(!isPointer() && "cannot change size of this type");
-    return LLT{Kind, ElementsOrAddrSpace, SizeInBits * 2};
+    assert(!IsPointer && "cannot change size of this type");
+    return LLT{/*isPointer=*/false, IsVector ? true : false,
+               IsVector ? getNumElements() : (uint16_t)0,
+               getScalarSizeInBits() * 2, /*AddressSpace=*/0};
   }
 
   /// Get a low-level type with half the size of the original, by halving the
   /// number of vector elements of the scalar type involved. The source must be
   /// a vector type with an even number of elements. For example `<4 x s32>`
   /// will become `<2 x s32>`, `<2 x s32>` will become `s32`.
   LLT halfElements() const {
-    assert(isVector() && ElementsOrAddrSpace % 2 == 0 &&
-           "cannot half odd vector");
-    if (ElementsOrAddrSpace == 2)
-      return scalar(SizeInBits);
-
-    return LLT{Vector, static_cast<uint16_t>(ElementsOrAddrSpace / 2),
-               SizeInBits};
+    assert(isVector() && getNumElements() % 2 == 0 && "cannot half odd vector");
+    if (getNumElements() == 2)
+      return scalar(getScalarSizeInBits());
+
+    return LLT{/*isPointer=*/false, /*isVector=*/true,
+               (uint16_t)(getNumElements() / 2), getScalarSizeInBits(),
+               /*AddressSpace=*/0};
   }
 
   /// Get a low-level type with twice the size of the original, by doubling the
   /// number of vector elements of the scalar type involved. The source must be
   /// a vector type. For example `<2 x s32>` will become `<4 x s32>`. Doubling
   /// the number of elements in sN produces <2 x sN>.
   LLT doubleElements() const {
-    assert(!isPointer() && "cannot double elements in pointer");
-    return LLT{Vector, static_cast<uint16_t>(ElementsOrAddrSpace * 2),
-               SizeInBits};
+    return LLT{IsPointer ? true : false, /*isVector=*/true,
+               (uint16_t)(getNumElements() * 2), getScalarSizeInBits(),
+               IsPointer ? getAddressSpace() : 0};
   }
 
   void print(raw_ostream &OS) const;
 
   bool operator==(const LLT &RHS) const {
-    return Kind == RHS.Kind && SizeInBits == RHS.SizeInBits &&
-           ElementsOrAddrSpace == RHS.ElementsOrAddrSpace;
+    return IsPointer == RHS.IsPointer && IsVector == RHS.IsVector &&
+           RHS.RawData == RawData;
   }
 
   bool operator!=(const LLT &RHS) const { return !(*this == RHS); }
 
   friend struct DenseMapInfo<LLT>;
+
 private:
-  unsigned SizeInBits;
-  uint16_t ElementsOrAddrSpace;
-  TypeKind Kind;
+  /// LLT is packed into 64 bits as follows:
+  /// isPointer : 1
+  /// isVector  : 1
+  /// with 62 bits remaining for Kind-specific data, packed in bitfields
+  /// as described below. As there isn't a simple portable way to pack bits
+  /// into bitfields, here the different fields in the packed structure is
+  /// described in static const *Field variables. Each of these variables
+  /// is a 2-element array, with the first element describing the bitfield size
+  /// and the second element describing the bitfield offset.
+  typedef int BitFieldInfo[2];
+  ///
+  /// This is how the bitfields are packed per Kind:
+  /// * Invalid:
+  ///   gets encoded as RawData == 0, as that is an invalid encoding, since for
+  ///   valid encodings, SizeInBits/SizeOfElement must be larger than 0.
+  /// * Non-pointer scalar (isPointer == 0 && isVector == 0):
+  ///   SizeInBits: 32;
+  static const constexpr BitFieldInfo ScalarSizeFieldInfo{32, 0};
+  /// * Pointer (isPointer == 1 && isVector == 0):
+  ///   SizeInBits: 16;
+  ///   AddressSpace: 23;
+  static const constexpr BitFieldInfo PointerSizeFieldInfo{16, 0};
+  static const constexpr BitFieldInfo PointerAddressSpaceFieldInfo{
+      23, PointerSizeFieldInfo[0] + PointerSizeFieldInfo[1]};
+  /// * Vector-of-non-pointer (isPointer == 0 && isVector == 1):
+  ///   NumElements: 16;
+  ///   SizeOfElement: 32;
+  static const constexpr BitFieldInfo VectorElementsFieldInfo{16, 0};
+  static const constexpr BitFieldInfo VectorSizeFieldInfo{
+      32, VectorElementsFieldInfo[0] + VectorElementsFieldInfo[1]};
+  /// * Vector-of-pointer (isPointer == 1 && isVector == 1):
+  ///   NumElements: 16;
+  ///   SizeOfElement: 16;
+  ///   AddressSpace: 23;
+  static const constexpr BitFieldInfo PointerVectorElementsFieldInfo{16, 0};
+  static const constexpr BitFieldInfo PointerVectorSizeFieldInfo{
+      16,
+      PointerVectorElementsFieldInfo[1] + PointerVectorElementsFieldInfo[0]};
+  static const constexpr BitFieldInfo PointerVectorAddressSpaceFieldInfo{
+      23, PointerVectorSizeFieldInfo[1] + PointerVectorSizeFieldInfo[0]};
+
+  uint64_t IsPointer : 1;
+  uint64_t IsVector : 1;
+  uint64_t RawData : 62;
+
+  static uint64_t getMask(const BitFieldInfo FieldInfo) {
+    const int FieldSizeInBits = FieldInfo[0];
+    return (((uint64_t)1) << FieldSizeInBits) - 1;
+  }
+  static uint64_t maskAndShift(uint64_t Val, uint64_t Mask, uint8_t Shift) {
+    assert(Val <= Mask && "Value too large for field");
+    return (Val & Mask) << Shift;
+  }
+  static uint64_t maskAndShift(uint64_t Val, const BitFieldInfo FieldInfo) {
+    return maskAndShift(Val, getMask(FieldInfo), FieldInfo[1]);
+  }
+  uint64_t getFieldValue(const BitFieldInfo FieldInfo) const {
+    return getMask(FieldInfo) & (RawData >> FieldInfo[1]);
+  }
+
+  void init(bool IsPointer, bool IsVector, uint16_t NumElements,
+            unsigned SizeInBits, unsigned AddressSpace) {
+    this->IsPointer = IsPointer;
+    this->IsVector = IsVector;
+    if (!IsVector) {
+      if (!IsPointer)
+        RawData = maskAndShift(SizeInBits, ScalarSizeFieldInfo);
+      else
+        RawData = maskAndShift(SizeInBits, PointerSizeFieldInfo) |
+                  maskAndShift(AddressSpace, PointerAddressSpaceFieldInfo);
+    } else {
+      assert(NumElements > 1 && "invalid number of vector elements");
+      if (!IsPointer)
+        RawData = maskAndShift(NumElements, VectorElementsFieldInfo) |
+                  maskAndShift(SizeInBits, VectorSizeFieldInfo);
+      else
+        RawData =
+            maskAndShift(NumElements, PointerVectorElementsFieldInfo) |
+            maskAndShift(SizeInBits, PointerVectorSizeFieldInfo) |
+            maskAndShift(AddressSpace, PointerVectorAddressSpaceFieldInfo);
+    }
+  }
 };
 
 inline raw_ostream& operator<<(raw_ostream &OS, const LLT &Ty) {
   Ty.print(OS);
   return OS;
 }
 
 template<> struct DenseMapInfo<LLT> {
   static inline LLT getEmptyKey() {
-    return LLT{LLT::Invalid, 0, -1u};
+    LLT Invalid;
+    Invalid.IsPointer = true;
+    return Invalid;
   }
   static inline LLT getTombstoneKey() {
-    return LLT{LLT::Invalid, 0, -2u};
+    LLT Invalid;
+    Invalid.IsVector = true;
+    return Invalid;
   }
   static inline unsigned getHashValue(const LLT &Ty) {
-    uint64_t Val = ((uint64_t)Ty.SizeInBits << 32) |
-                   ((uint64_t)Ty.ElementsOrAddrSpace << 16) | (uint64_t)Ty.Kind;
+    uint64_t Val = ((uint64_t)Ty.RawData) << 2 | ((uint64_t)Ty.IsPointer) << 1 |
+                   ((uint64_t)Ty.IsVector);
     return DenseMapInfo<uint64_t>::getHashValue(Val);
   }
   static bool isEqual(const LLT &LHS, const LLT &RHS) {
     return LHS == RHS;
   }
 };
 
 }
 
 #endif // LLVM_SUPPORT_LOWLEVELTYPEIMPL_H

lib/CodeGen/GlobalISel/MachineIRBuilder.cpp

MachineInstrBuilder MachineIRBuilder::buildInsertVectorElement(unsigned Res,
                                                                unsigned Elt,
                                                                unsigned Idx) {
 #ifndef NDEBUG
   LLT ResTy = MRI->getType(Res);
   LLT ValTy = MRI->getType(Val);
   LLT EltTy = MRI->getType(Elt);
   LLT IdxTy = MRI->getType(Idx);
   assert(ResTy.isVector() && ValTy.isVector() && "invalid operand type");
-  assert(EltTy.isScalar() && IdxTy.isScalar() && "invalid operand type");
+  assert(IdxTy.isScalar() && "invalid operand type");
   assert(ResTy.getNumElements() == ValTy.getNumElements() && "type mismatch");
   assert(ResTy.getElementType() == EltTy && "type mismatch");
 #endif
 
   return buildInstr(TargetOpcode::G_INSERT_VECTOR_ELT)
       .addDef(Res)
       .addUse(Val)
       .addUse(Elt)
       .addUse(Idx);
 }
 
 MachineInstrBuilder MachineIRBuilder::buildExtractVectorElement(unsigned Res,
                                                                 unsigned Val,
                                                                 unsigned Idx) {
 #ifndef NDEBUG
   LLT ResTy = MRI->getType(Res);
   LLT ValTy = MRI->getType(Val);
   LLT IdxTy = MRI->getType(Idx);
   assert(ValTy.isVector() && "invalid operand type");
-  assert(ResTy.isScalar() && IdxTy.isScalar() && "invalid operand type");
+  assert((ResTy.isScalar() || ResTy.isPointer()) && "invalid operand type");
+  assert(IdxTy.isScalar() && "invalid operand type");
   assert(ValTy.getElementType() == ResTy && "type mismatch");
 #endif
 
   return buildInstr(TargetOpcode::G_EXTRACT_VECTOR_ELT)
       .addDef(Res)
       .addUse(Val)
       .addUse(Idx);
 }

lib/CodeGen/LowLevelType.cpp

 #include "llvm/IR/DataLayout.h"
 #include "llvm/IR/DerivedTypes.h"
 #include "llvm/Support/raw_ostream.h"
 using namespace llvm;
 
 LLT llvm::getLLTForType(Type &Ty, const DataLayout &DL) {
   if (auto VTy = dyn_cast<VectorType>(&Ty)) {
     auto NumElements = VTy->getNumElements();
-    auto ScalarSizeInBits = VTy->getElementType()->getPrimitiveSizeInBits();
+    LLT ScalarTy = getLLTForType(*VTy->getElementType(), DL);
     if (NumElements == 1)
-      return LLT::scalar(ScalarSizeInBits);
-    return LLT::vector(NumElements, ScalarSizeInBits);
+      return ScalarTy;
+    return LLT::vector(NumElements, ScalarTy);
   } else if (auto PTy = dyn_cast<PointerType>(&Ty)) {
     return LLT::pointer(PTy->getAddressSpace(), DL.getTypeSizeInBits(&Ty));
   } else if (Ty.isSized()) {
     // Aggregates are no different from real scalars as far as GlobalISel is
     // concerned.
     auto SizeInBits = DL.getTypeSizeInBits(&Ty);
     assert(SizeInBits != 0 && "invalid zero-sized type");
     return LLT::scalar(SizeInBits);
   }
   return LLT();
 }

lib/Support/LowLevelType.cpp

 //===----------------------------------------------------------------------===//
 
 #include "llvm/Support/LowLevelTypeImpl.h"
 #include "llvm/Support/raw_ostream.h"
 using namespace llvm;
 
 LLT::LLT(MVT VT) {
   if (VT.isVector()) {
-    SizeInBits = VT.getVectorElementType().getSizeInBits();
-    ElementsOrAddrSpace = VT.getVectorNumElements();
-    Kind = ElementsOrAddrSpace == 1 ? Scalar : Vector;
+    init(/*isPointer=*/false, VT.getVectorNumElements() > 1,
+         VT.getVectorNumElements(), VT.getVectorElementType().getSizeInBits(),
+         /*AddressSpace=*/0);
   } else if (VT.isValid()) {
     // Aggregates are no different from real scalars as far as GlobalISel is
     // concerned.
-    Kind = Scalar;
-    SizeInBits = VT.getSizeInBits();
-    ElementsOrAddrSpace = 1;
-    assert(SizeInBits != 0 && "invalid zero-sized type");
+    assert(VT.getSizeInBits() != 0 && "invalid zero-sized type");
+    init(/*isPointer=*/false, /*isVector=*/false, /*NumElements=*/0,
+         VT.getSizeInBits(), /*AddressSpace=*/0);
   } else {
-    Kind = Invalid;
-    SizeInBits = ElementsOrAddrSpace = 0;
+    IsPointer = false;
+    IsVector = false;
+    RawData = 0;
   }
 }
 
 void LLT::print(raw_ostream &OS) const {
   if (isVector())
-    OS << "<" << ElementsOrAddrSpace << " x s" << SizeInBits << ">";
+    OS << "<" << getNumElements() << " x " << getElementType() << ">";
   else if (isPointer())
     OS << "p" << getAddressSpace();
   else if (isValid()) {
     assert(isScalar() && "unexpected type");
     OS << "s" << getScalarSizeInBits();
   } else
     llvm_unreachable("trying to print an invalid type");
 }
+
+const LLT::BitFieldInfo LLT::ScalarSizeFieldInfo;
+const LLT::BitFieldInfo LLT::PointerSizeFieldInfo;
+const LLT::BitFieldInfo LLT::PointerAddressSpaceFieldInfo;
+const LLT::BitFieldInfo LLT::VectorElementsFieldInfo;
+const LLT::BitFieldInfo LLT::VectorSizeFieldInfo;
+const LLT::BitFieldInfo LLT::PointerVectorElementsFieldInfo;
+const LLT::BitFieldInfo LLT::PointerVectorSizeFieldInfo;
+const LLT::BitFieldInfo LLT::PointerVectorAddressSpaceFieldInfo;

lib/Target/AArch64/AArch64RegisterBankInfo.cpp

AArch64RegisterBankInfo::getInstrMapping(const MachineInstr &MI) const {
 
   unsigned NumOperands = MI.getNumOperands();
 
   // Track the size and bank of each register.  We don't do partial mappings.
   SmallVector<unsigned, 4> OpSize(NumOperands);
   SmallVector<PartialMappingIdx, 4> OpRegBankIdx(NumOperands);
   for (unsigned Idx = 0; Idx < NumOperands; ++Idx) {
     auto &MO = MI.getOperand(Idx);
-    if (!MO.isReg())
+    if (!MO.isReg() || !MO.getReg())
       continue;
 
     LLT Ty = MRI.getType(MO.getReg());
     OpSize[Idx] = Ty.getSizeInBits();
 
     // As a top-level guess, vectors go in FPRs, scalars and pointers in GPRs.
     // For floating-point instructions, scalars go in FPRs.
     if (Ty.isVector() || isPreISelGenericFloatingPointOpcode(Opc))
       OpRegBankIdx[Idx] = PMI_FirstFPR;
     else
       OpRegBankIdx[Idx] = PMI_FirstGPR;
   }
case TargetOpcode::G_LOAD:
     break;
   }
 
   // Finally construct the computed mapping.
   RegisterBankInfo::InstructionMapping Mapping =
       InstructionMapping{DefaultMappingID, Cost, nullptr, NumOperands};
   SmallVector<const ValueMapping *, 8> OpdsMapping(NumOperands);
   for (unsigned Idx = 0; Idx < NumOperands; ++Idx) {
-    if (MI.getOperand(Idx).isReg()) {
+    if (MI.getOperand(Idx).isReg() && MI.getOperand(Idx).getReg()) {
       auto Mapping = getValueMapping(OpRegBankIdx[Idx], OpSize[Idx]);
       if (!Mapping->isValid())
         return InstructionMapping();
 
       OpdsMapping[Idx] = Mapping;
     }
   }
 
   Mapping.setOperandsMapping(getOperandsMapping(OpdsMapping));
   return Mapping;
 }

test/CodeGen/AArch64/GlobalISel/arm64-fallback.ll

 
 ; Check that we fallback on invoke translation failures.
 ; FALLBACK-WITH-REPORT-ERR: remark: <unknown>:0:0: unable to legalize instruction: %vreg0<def>(s128) = G_FCONSTANT quad 2
 ; FALLBACK-WITH-REPORT-ERR: warning: Instruction selection used fallback path for test_quad_dump
 ; FALLBACK-WITH-REPORT-OUT-LABEL: test_quad_dump:
 define fp128 @test_quad_dump() {
   ret fp128 0xL00000000000000004000000000000000
 }
+
+; FALLBACK-WITH-REPORT-ERR: remark: <unknown>:0:0: unable to legalize instruction: %vreg0<def>(p0) = G_EXTRACT_VECTOR_ELT %vreg1, %vreg2; (in function: vector_of_pointers_extractelement)
+; FALLBACK-WITH-REPORT-ERR: warning: Instruction selection used fallback path for vector_of_pointers_extractelement
+; FALLBACK-WITH-REPORT-OUT-LABEL: vector_of_pointers_extractelement:
+define void @vector_of_pointers_extractelement() {
+  %dummy = extractelement <2 x i16*> undef, i32 0
+  ret void
+}
+
+; FALLBACK-WITH-REPORT-ERR: remark: <unknown>:0:0: unable to legalize instruction: %vreg0<def>(<2 x p0>) = G_INSERT_VECTOR_ELT %vreg1, %vreg2, %vreg3; (in function: vector_of_pointers_insertelement
+; FALLBACK-WITH-REPORT-ERR: warning: Instruction selection used fallback path for vector_of_pointers_insertelement
+; FALLBACK-WITH-REPORT-OUT-LABEL: vector_of_pointers_insertelement:
+define void @vector_of_pointers_insertelement() {
+  %dummy = insertelement <2 x i16*> undef, i16* null, i32 0
+  ret void
+}

unittests/CodeGen/LowLevelTypeTest.cpp

 }
 
 TEST(LowLevelTypeTest, Pointer) {
   LLVMContext C;
   DataLayout DL("");
 
   for (unsigned AS : {0U, 1U, 127U, 0xffffU}) {
     const LLT Ty = LLT::pointer(AS, DL.getPointerSizeInBits(AS));
+    const LLT VTy = LLT::vector(4, Ty);
 
     // Test kind.
     ASSERT_TRUE(Ty.isValid());
     ASSERT_TRUE(Ty.isPointer());
 
     ASSERT_FALSE(Ty.isScalar());
     ASSERT_FALSE(Ty.isVector());
 
+    ASSERT_TRUE(VTy.isValid());
+    ASSERT_TRUE(VTy.isVector());
+    ASSERT_TRUE(VTy.getElementType().isPointer());
+
     // Test addressspace.
     EXPECT_EQ(AS, Ty.getAddressSpace());
+    EXPECT_EQ(AS, VTy.getElementType().getAddressSpace());
 
     // Test equality operators.
     EXPECT_TRUE(Ty == Ty);
     EXPECT_FALSE(Ty != Ty);
+    EXPECT_TRUE(VTy == VTy);
+    EXPECT_FALSE(VTy != VTy);
 
     // Test Type->LLT conversion.
     Type *IRTy = PointerType::get(IntegerType::get(C, 8), AS);
     EXPECT_EQ(Ty, getLLTForType(*IRTy, DL));
+    Type *IRVTy =
+        VectorType::get(PointerType::get(IntegerType::get(C, 8), AS), 4);
+    EXPECT_EQ(VTy, getLLTForType(*IRVTy, DL));
   }
 }
 
 TEST(LowLevelTypeTest, Invalid) {
   const LLT Ty;
 
   ASSERT_FALSE(Ty.isValid());
   ASSERT_FALSE(Ty.isScalar());
   ASSERT_FALSE(Ty.isPointer());
   ASSERT_FALSE(Ty.isVector());
 }
 
 }