Implementation of new polymorphic type llvm_anyref_ty

[pmatos]

Summary

This polymorphic type accepts any WebAssembly reference type, at the
moment externref or funcref.

[pmatos]

@tlively This is early days of the intrinsics implementation, so it's not ready for review.
However, I wanted to share to ensure that the progress of the implementation is public and to avoid going into a design dead end.

At the moment, I am fighting tablegen, which doesn't like the table.grow pattern due to:

Type set is empty for each HW mode:
possible type contradiction in the pattern below (use -print-records with llvm-tblgen to see all expanded records).

I am still investigating, but the overall implementation will allow us to use table.grow and table.size at the IR level through the intrinsics. A few notes:

• As far as I understand intrinsics cannot be polymorphic, therefore table.grow and ref.null intrinsics come in two versions: one for externref and one for funcref.
• An intrinsic for ref.null is required so that we can create a constant to pass to table.grow to initialize the new slots on a table but also for other general uses.
• In order to specify the types (params and return) in the intrinsics definition, I needed to add a bunch of definitions for llvm_externref_ty and llvm_funcref_ty and plug them the correct types in other enumerations.

Hoping I can make peace with TableGen and sort the current problem. I feel like this should be relatively straightforward once I pass the TableGen hurdle.

P.S.: Pay no attention to the test llvm/test/CodeGen/WebAssembly/table-grow.ll whose contents are not there yet, and was committed by mistake.

[sbc100]

In D111227#3045279, @pmatos wrote:

@tlively This is early days of the intrinsics implementation, so it's not ready for review.
However, I wanted to share to ensure that the progress of the implementation is public and to avoid going into a design dead end.

At the moment, I am fighting tablegen, which doesn't like the table.grow pattern due to:

Type set is empty for each HW mode:
possible type contradiction in the pattern below (use -print-records with llvm-tblgen to see all expanded records).

I am still investigating, but the overall implementation will allow us to use table.grow and table.size at the IR level through the intrinsics. A few notes:

• As far as I understand intrinsics cannot be polymorphic, therefore table.grow and ref.null intrinsics come in two versions: one for externref and one for funcref.
• An intrinsic for ref.null is required so that we can create a constant to pass to table.grow to initialize the new slots on a table but also for other general uses.
• In order to specify the types (params and return) in the intrinsics definition, I needed to add a bunch of definitions for llvm_externref_ty and llvm_funcref_ty and plug them the correct types in other enumerations.

Hoping I can make peace with TableGen and sort the current problem. I feel like this should be relatively straightforward once I pass the TableGen hurdle.

P.S.: Pay no attention to the test llvm/test/CodeGen/WebAssembly/table-grow.ll whose contents are not there yet, and was committed by mistake.

Could we bypass/delay a lot of this complexity by specifying the initial version of intrinsic for table.grow as "grow_table_with_null" .. then we don't need ref.null intrinsic or these new types? (at least not yet).

[pmatos]

In D111227#3045475, @sbc100 wrote:

Could we bypass/delay a lot of this complexity by specifying the initial version of intrinsic for table.grow as "grow_table_with_null" .. then we don't need ref.null intrinsic or these new types? (at least not yet).

If the motivation is to delay the complexity, then just having grow_table_with_null wouldn't suffice because we would still need the new types to define the table.size and table.grow param types. I also worry that doing something half-baked now will come to haunt us later if we then do it "properly" and have to keep old intrinsics for the sake of backwards compatibility.

[sbc100]

In D111227#3046124, @pmatos wrote:

In D111227#3045475, @sbc100 wrote:

Could we bypass/delay a lot of this complexity by specifying the initial version of intrinsic for table.grow as "grow_table_with_null" .. then we don't need ref.null intrinsic or these new types? (at least not yet).

If the motivation is to delay the complexity, then just having grow_table_with_null wouldn't suffice because we would still need the new types to define the table.size and table.grow param types. I also worry that doing something half-baked now will come to haunt us later if we then do it "properly" and have to keep old intrinsics for the sake of backwards compatibility.

Isn't "got with null" likely to be by far the most likely thing to be used? I'm tempted to call that __table_grow and go with __table_grow_with_default_elem for the less-used one.

[tlively]

In D111227#3045279, @pmatos wrote:

• As far as I understand intrinsics cannot be polymorphic, therefore table.grow and ref.null intrinsics come in two versions: one for externref and one for funcref.

Intrinsics actually can be polymorphic. For an example, see the definition of int_wasm_sub_sat_signed, which is polymorphic over all vector types (although not all vector types are actually supported in the backend). I don't know how much work it would be to support that kind of polymorphism for reference types, though.

• An intrinsic for ref.null is required so that we can create a constant to pass to table.grow to initialize the new slots on a table but also for other general uses.

Would it be more trouble than it's worth to try to get a constant zero to mean ref.null?

[pmatos]

In D111227#3046551, @tlively wrote:

In D111227#3045279, @pmatos wrote:

• As far as I understand intrinsics cannot be polymorphic, therefore table.grow and ref.null intrinsics come in two versions: one for externref and one for funcref.

Intrinsics actually can be polymorphic. For an example, see the definition of int_wasm_sub_sat_signed, which is polymorphic over all vector types (although not all vector types are actually supported in the backend). I don't know how much work it would be to support that kind of polymorphism for reference types, though.

You mean the polymorphism achieve through llvm_anyvector_ty? So I guess the way to go about it would be to try to add a wasm_anyreftype_ty... I will take a look.

[tlively]

In D111227#3066115, @pmatos wrote:

In D111227#3046551, @tlively wrote:

In D111227#3045279, @pmatos wrote:

• As far as I understand intrinsics cannot be polymorphic, therefore table.grow and ref.null intrinsics come in two versions: one for externref and one for funcref.

Intrinsics actually can be polymorphic. For an example, see the definition of int_wasm_sub_sat_signed, which is polymorphic over all vector types (although not all vector types are actually supported in the backend). I don't know how much work it would be to support that kind of polymorphism for reference types, though.

You mean the polymorphism achieve through llvm_anyvector_ty? So I guess the way to go about it would be to try to add a wasm_anyreftype_ty... I will take a look.

Yes, exactly. There's also the even more permissive llvm_any_ty. It looks like this would involve updating llvm/include/llvm/CodeGen/ValueTypes.td and llvm/include/llvm/Support/MachineValueType.h to add a new rAny value type that would be used by tablegen in llvm/utils/TableGen/{CodeGenDAGPatterns,CodeGenTarget,IntrinsicEmitter}.cpp.

[pmatos]

Follow the breadcrumbs to define llvm_extenref_ty and llvm_funcref_ty

[pmatos]

In D111227#3069277, @tlively wrote:

In D111227#3066115, @pmatos wrote:

In D111227#3046551, @tlively wrote:

In D111227#3045279, @pmatos wrote:

• As far as I understand intrinsics cannot be polymorphic, therefore table.grow and ref.null intrinsics come in two versions: one for externref and one for funcref.

Intrinsics actually can be polymorphic. For an example, see the definition of int_wasm_sub_sat_signed, which is polymorphic over all vector types (although not all vector types are actually supported in the backend). I don't know how much work it would be to support that kind of polymorphism for reference types, though.

You mean the polymorphism achieve through llvm_anyvector_ty? So I guess the way to go about it would be to try to add a wasm_anyreftype_ty... I will take a look.

Yes, exactly. There's also the even more permissive llvm_any_ty. It looks like this would involve updating llvm/include/llvm/CodeGen/ValueTypes.td and llvm/include/llvm/Support/MachineValueType.h to add a new rAny value type that would be used by tablegen in llvm/utils/TableGen/{CodeGenDAGPatterns,CodeGenTarget,IntrinsicEmitter}.cpp.

That makes sense. Thanks for the pointers.
I am sort of going the rabbit hole of trying to understand how far I need to go to have llvm_externref_ty and llvm_funcref_ty in an intrinsic. This seems to require changes in a wide range of files - assuming it would be similar (if not worse) with a polymorphic type. Unfortunately, it is still now compiling as there are further changes required.

[pmatos]

In D111227#3069965, @pmatos wrote:

That makes sense. Thanks for the pointers.
I am sort of going the rabbit hole of trying to understand how far I need to go to have llvm_externref_ty and llvm_funcref_ty in an intrinsic. This seems to require changes in a wide range of files - assuming it would be similar (if not worse) with a polymorphic type. Unfortunately, it is still now compiling as there are further changes required.

This feels like going into the rabbit hole of adding a new type as mentioned in https://www.llvm.org/docs/ExtendingLLVM.html#adding-a-new-type
Unsure this is necessary, but it feels like it might be if we need to specify a reference type for the intrinsic. On the other hand it's not great as I am starting to touch things related to bc encoding for the new ref types.

[pmatos]

In D111227#3070024, @pmatos wrote:

In D111227#3069965, @pmatos wrote:

That makes sense. Thanks for the pointers.
I am sort of going the rabbit hole of trying to understand how far I need to go to have llvm_externref_ty and llvm_funcref_ty in an intrinsic. This seems to require changes in a wide range of files - assuming it would be similar (if not worse) with a polymorphic type. Unfortunately, it is still now compiling as there are further changes required.

This feels like going into the rabbit hole of adding a new type as mentioned in https://www.llvm.org/docs/ExtendingLLVM.html#adding-a-new-type
Unsure this is necessary, but it feels like it might be if we need to specify a reference type for the intrinsic. On the other hand it's not great as I am starting to touch things related to bc encoding for the new ref types.

I keep thinking that we should be able to model these as pointers to something instead of new types, however the llvm_ptr_ty doesn't seem to allow specifying the pointee type. I will continue an investigation in this direction.

[pmatos]

Implements the intrinsic generating ref.null.

The test ref-null.ll passes but the implementation raised questions on how funcref is represented.
Because (apparently) IITDescriptor didn't include the ability to represent functions, I added a
straightforward way in the current patch but it's definitely a draft version that provides food
for thought.

Also raises questions on how to represent funcrefs that represent different FunctionTypes, i.e.
different parameter count/type and different return type.

[tlively]

llvm/include/llvm/IR/Intrinsics.h

144

Both externref and funcref are modeled as pointers, so I'm surprised we have to add a new type here. It looks like there is already a field in the following union for Pointer_AddressSpace, so hopefully that would be sufficient?

llvm/lib/CodeGen/ValueTypes.cpp

207 ↗

(On Diff #380716)

Maybe we should go along with the transition to opaque pointers and do PointerType::get(Context, 20)? If so, we should probably do the same for externref.

[pmatos]

llvm/lib/CodeGen/ValueTypes.cpp

207 ↗

(On Diff #380716)

Tbh, I thought I was using that here when doing PointerType::get(0, 20), but you're right. I need to do PointerType::get(Context, 20). When I do so, Iearn that I need to enable opaque pointers mode through Context.enableOpaquePointers().

Curiously, nobody calls that function at the moment, but I have now done so to understand the consequences for us and what kind of pandora's box I am opening.

[tlively]

llvm/lib/CodeGen/ValueTypes.cpp

207 ↗

(On Diff #380716)

Oh hmm, that doesn't sound like something we want to do. Context.enableOpaquePointers() sounds like some sort of global configuration that we don't want to be responsible for here.

I think the PointerType::get(0, 20) is interpreted as passing nullptr as the Type* parameter.

[pmatos]

The current patch pushes the implementation forward.

It currently implements intrinsics for table.size and ref.null as well as all the dependencies of this, the largest of which is a new polymorphic type llvm_anyref_ty.

There are a few niggles that need to be fixed and discussed, as well as the final implementation of table.grow intrinsic before a final patch is presented for review.

[pmatos]

.gitignore

63 ↗

(On Diff #385437)

Ignore this for now... local change committed by mistake.

llvm/include/llvm/IR/Intrinsics.h

162

This single change caused me to lose countless hours, until I understood that in IntrinsicsEmitter.cpp, the order of the cases statement for rAny depends on the order of this. I added some comments in IntrinsicsEmitter.cpp to ensure others don't lose the same amount of time. However, this (the comments) could be a separate patch.

llvm/include/llvm/IR/Intrinsics.td

238

Maybe this could be a separate patch - adding the new polymorphic type, what do you think?

llvm/include/llvm/IR/Type.h

80

I would appreciate some suggestions here. The predicate to recognize reference types, which is required by the polymorphic type llvm_anyref_ty is currently defined in WebAssemblyUtils. However, we need it to be in Type.h. What can we do here? It feels like pulling all this target dependent stuff into here is just not ideal. Minimally, we could include the WebAssemblyUtils header here and create a wrapper for these predicates in Type. But that still involves including a target dependent header here. Any suggestions?

llvm/lib/CodeGen/ValueTypes.cpp

207 ↗

(On Diff #380716)

I took the opaque pointer stuff a bit too far and got stuck in a few places where LLVM still looks for the pointee type and crashed, which should be fixed but felt like I was diving into a pandora's box.

After discussing with @wingo he suggested using pointers to i8, which works...
Although I feel the cleaner solution is opaque pointers, I am unsure we should be the first users of opaque pointers in LLVM at the moment. Therefore funcrefs are now pointers to i8 which are cast to the respective type when used.

[pmatos]

Implement table.grow intrinsic

[tlively]

llvm/include/llvm/CodeGen/ValueTypes.td

240

Hmm, I wonder what "current pointer size" means here. The contrast between this and iPTRAny makes me think that there should be a way to control the address space of iPTR arguments. Maybe it's the address space attached to the call instruction for the intrinsic? If we can find a target-independent way to pass pointers to particular address spaces to intrinsics, that seems like it would be a better solution that putting specific knowledge of WebAssembly address spaces into IR/Type.h. This also might be worth asking about on llvm-dev.

llvm/lib/CodeGen/ValueTypes.cpp

207 ↗

(On Diff #380716)

Using pointers to i8 seems fine to me. Another option would be using pointers to pointers to opaque structs like we do (IIRC) with externrefs. Or maybe externrefs should also be pointers to i8. It would nice to be consistent between them if funcrefs can't just be function pointers.

[pmatos]

llvm/include/llvm/CodeGen/ValueTypes.td

240

Good point. I will take a look at this and if I can't figure it out in a reasonable amount of time, I will ask llvm-dev.

llvm/lib/CodeGen/ValueTypes.cpp

207 ↗

(On Diff #380716)

That's what we used to do but it doesn't work because the intrinsic validator checks that the type defined for funcref matches the type of the argument / return value of the intrinsic, so you cannot define funcref to be a pointer to opaque and then define the intrinsic as getting a pointer to function.

[pmatos]

llvm/include/llvm/CodeGen/ValueTypes.td

240

hummm, I think this might not be that useful, a quick glance at the code shows:

// An int value the size of the pointer of the current
// target to any address space. This must only be used internal to
// tblgen. Other than for overloading, we treat iPTRAny the same as iPTR.
iPTRAny        = 250,

and

// An int value the size of the pointer of the current
// target.  This should only be used internal to tblgen!
iPTR           = 254,

I think that this is independent of address space. iPTRAny is barely used in code. I think for most purposes it's identical to iPTR. I will discuss this problem in llvm-dev.

[pmatos]

llvm/include/llvm/CodeGen/ValueTypes.td

240

I brought this issue up here: https://lists.llvm.org/pipermail/llvm-dev/2021-November/153640.html

[pmatos]

Update the revision to contain only the changes relevant to the implementation of the new polymorphic type llvm_anyref_ty.

[tlively]

llvm/include/llvm/IR/Intrinsics.td

171–175

Instead of having to build Wasm-specific knowledge into so many new parts of LLVM, can we instead use LLVMQualPointerType? I guess this wouldn't allow us to define polymorphic intrinsics after all, though. As long as we have just externref and funcref, that might be fine, but do we know whether we will be adding more reference types in the future? The more we will need to add, the more amenable to these changes I would be. One option is to not land this change now, but keep it in our back pocket for if/when we need to add more reference types in the future.

llvm/include/llvm/Support/MachineValueType.h

314

This should also have the This is only for tblgen's consumption! warning.

[pmatos]

@tlively Thanks for the comments, I agree that it might be too much hassle to define a polymorphic type to just range over the current two available ref types, so I am dropping this revision. I have a revised patch with intrinsics implementation without the need for a polymorphic type that I will push to D113420.

llvm/include/llvm/CodeGen/ValueTypes.h

public:
 
     /// Return true if this is a 2048-bit vector type.
     bool is2048BitVector() const {
       return isSimple() ? V.is2048BitVector() : isExtended2048BitVector();
     }
 
     /// Return true if this is an overloaded type for TableGen.
     bool isOverloaded() const {
-      return (V==MVT::iAny || V==MVT::fAny || V==MVT::vAny || V==MVT::iPTRAny);
+      return (V == MVT::rAny || V == MVT::iAny || V == MVT::fAny ||
+              V == MVT::vAny || V == MVT::iPTRAny);
     }
 
     /// Return true if the bit size is a multiple of 8.
     bool isByteSized() const {
       return !isZeroSized() && getSizeInBits().isKnownMultipleOf(8);
     }
 
     /// Return true if the size is a power-of-two number of bytes.

llvm/include/llvm/CodeGen/ValueTypes.td

 def FlagVT    : ValueType<0,    172>;  // Pre-RA sched glue
 def isVoid    : ValueType<0,    173>;  // Produces no value
 def untyped   : ValueType<8,    174>;  // Produces an untyped value
 def funcref   : ValueType<0,    175>;  // WebAssembly's funcref type
 def externref : ValueType<0,    176>;  // WebAssembly's externref type
 def x86amx    : ValueType<8192, 177>;  // X86 AMX value
 def i64x8     : ValueType<512,  178>;  // 8 Consecutive GPRs (AArch64)
 
-
-def token      : ValueType<0, 248>;  // TokenTy
-def MetadataVT : ValueType<0, 249>;  // Metadata
+def token      : ValueType<0, 247>;  // TokenTy
+def MetadataVT : ValueType<0, 248>;  // Metadata
 
 // Pseudo valuetype mapped to the current pointer size to any address space.
 // Should only be used in TableGen.
-def iPTRAny    : ValueType<0, 250>;
+def iPTRAny    : ValueType<0, 249>;
 
 // Pseudo valuetype to represent "vector of any size"
-def vAny       : ValueType<0, 251>;
+def vAny       : ValueType<0, 250>;
 
 // Pseudo valuetype to represent "float of any format"
-def fAny       : ValueType<0, 252>;
+def fAny       : ValueType<0, 251>;
 
 // Pseudo valuetype to represent "integer of any bit width"
-def iAny       : ValueType<0, 253>;
+def iAny       : ValueType<0, 252>;
+
+// Pseudo valuetype to represent "any reference type"
+def rAny       : ValueType<0, 253>;
 
 // Pseudo valuetype mapped to the current pointer size.
 def iPTR       : ValueType<0, 254>;
 
 // Pseudo valuetype to represent "any type of any size".
 def Any        : ValueType<0, 255>;
 
 /// This class is for targets that want to use pointer types in patterns
 /// with the GlobalISelEmitter.  Targets must define their own pointer
 /// derived from this class.  The scalar argument should be an
 /// integer type with the same bit size as the pointer.
 /// e.g. def p0 : PtrValueType <i64, 0>;
 
 class PtrValueType <ValueType scalar, int addrspace> :
     ValueType<scalar.Size, scalar.Value> {
   int AddrSpace = addrspace;
 }

llvm/include/llvm/IR/Intrinsics.h

#undef GET_INTRINSIC_ENUM_VALUES
 
   /// Return the attributes for an intrinsic.
   AttributeList getAttributes(LLVMContext &C, ID id);
 
   /// Create or insert an LLVM Function declaration for an intrinsic, and return
   /// it.
   ///
   /// The Tys parameter is for intrinsics with overloaded types (e.g., those
-  /// using iAny, fAny, vAny, or iPTRAny).  For a declaration of an overloaded
-  /// intrinsic, Tys must provide exactly one type for each overloaded type in
-  /// the intrinsic.
+  /// using rAny, iAny, fAny, vAny, or iPTRAny).  For a declaration of an
+  /// overloaded intrinsic, Tys must provide exactly one type for each
+  /// overloaded type in the intrinsic.
   Function *getDeclaration(Module *M, ID id, ArrayRef<Type*> Tys = None);
 
   /// Looks up Name in NameTable via binary search. NameTable must be sorted
   /// and all entries must start with "llvm.".  If NameTable contains an exact
   /// match for Name or a prefix of Name followed by a dot, its index in
   /// NameTable is returned. Otherwise, -1 is returned.
   int lookupLLVMIntrinsicByName(ArrayRef<const char *> NameTable,
                                 StringRef Name);
enum IITDescriptorKind {
       PtrToArgument,
       PtrToElt,
       VecOfAnyPtrsToElt,
       VecElementArgument,
       Subdivide2Argument,
       Subdivide4Argument,
       VecOfBitcastsToInt,
       AMX,
       PPCQuad,
     } Kind;
 
     union {
       unsigned Integer_Width;
       unsigned Float_Width;
       unsigned Pointer_AddressSpace;
       unsigned Struct_NumElements;
       unsigned Argument_Info;
       ElementCount Vector_Width;
     };
 
     enum ArgKind {
       AK_Any,
       AK_AnyInteger,
       AK_AnyFloat,
       AK_AnyVector,
       AK_AnyPointer,
+      AK_AnyRef,
       AK_MatchType = 7
     };
 
     unsigned getArgumentNumber() const {
       assert(Kind == Argument || Kind == ExtendArgument ||
              Kind == TruncArgument || Kind == HalfVecArgument ||
              Kind == SameVecWidthArgument || Kind == PtrToArgument ||
              Kind == PtrToElt || Kind == VecElementArgument ||

llvm/include/llvm/IR/Intrinsics.td

 // Types used by intrinsics.
 //===----------------------------------------------------------------------===//
 
 class LLVMType<ValueType vt> {
   ValueType VT = vt;
   int isAny = false;
 }
 
 class LLVMQualPointerType<LLVMType elty, int addrspace>
   : LLVMType<iPTR>{
   LLVMType ElTy = elty;
   int AddrSpace = addrspace;
 }
 
 class LLVMPointerType<LLVMType elty>
   : LLVMQualPointerType<elty, 0>;
 
 class LLVMAnyPointerType<LLVMType elty>
   : LLVMType<iPTRAny>{
   LLVMType ElTy = elty;
 

 class LLVMVectorOfBitcastsToInt<int num> : LLVMMatchType<num>;
 
 def llvm_void_ty       : LLVMType<isVoid>;
 let isAny = true in {
   def llvm_any_ty        : LLVMType<Any>;
   def llvm_anyint_ty     : LLVMType<iAny>;
   def llvm_anyfloat_ty   : LLVMType<fAny>;
   def llvm_anyvector_ty  : LLVMType<vAny>;
+  def llvm_anyref_ty     : LLVMType<rAny>;
 }
 def llvm_i1_ty         : LLVMType<i1>;
 def llvm_i8_ty         : LLVMType<i8>;
 def llvm_i16_ty        : LLVMType<i16>;
 def llvm_i32_ty        : LLVMType<i32>;
 def llvm_i64_ty        : LLVMType<i64>;
 def llvm_half_ty       : LLVMType<f16>;
 def llvm_bfloat_ty     : LLVMType<bf16>;

 def llvm_v1f64_ty      : LLVMType<v1f64>;    //  1 x double
 def llvm_v2f64_ty      : LLVMType<v2f64>;    //  2 x double
 def llvm_v4f64_ty      : LLVMType<v4f64>;    //  4 x double
 def llvm_v8f64_ty      : LLVMType<v8f64>;    //  8 x double
 def llvm_v16f64_ty     : LLVMType<v16f64>;   // 16 x double
 
 def llvm_vararg_ty     : LLVMType<isVoid>;   // this means vararg here
 
+def llvm_externref_ty  : LLVMType<externref>;
+def llvm_funcref_ty    : LLVMType<funcref>;
+
 //===----------------------------------------------------------------------===//
 // Intrinsic Definitions.
 //===----------------------------------------------------------------------===//
 
 // Intrinsic class - This is used to define one LLVM intrinsic.  The name of the
 // intrinsic definition should start with "int_", then match the LLVM intrinsic
 // name with the "llvm." prefix removed, and all "."s turned into "_"s.  For
 // example, llvm.bswap.i16 -> int_bswap_i16.

llvm/include/llvm/IR/Type.h

enum TypeID {
     FunctionTyID,      ///< Functions
     PointerTyID,       ///< Pointers
     StructTyID,        ///< Structures
     ArrayTyID,         ///< Arrays
     FixedVectorTyID,   ///< Fixed width SIMD vector type
     ScalableVectorTyID ///< Scalable SIMD vector type
   };
 
+  enum WasmAddressSpace : unsigned {
+    // Default address space, for pointers to linear memory (stack, heap, data).
+    WASM_ADDRESS_SPACE_DEFAULT = 0,
+    // A non-integral address space for pointers to named objects outside of
+    // linear memory: WebAssembly globals or WebAssembly locals.  Loads and
+    // stores
+    // to these pointers are lowered to global.get / global.set or local.get /
+    // local.set, as appropriate.
+    WASM_ADDRESS_SPACE_VAR = 1,
+    // A non-integral address space for externref values
+    WASM_ADDRESS_SPACE_EXTERNREF = 10,
+    // A non-integral address space for funcref values
+    WASM_ADDRESS_SPACE_FUNCREF = 20,
+  };
+
 private:
   /// This refers to the LLVMContext in which this type was uniqued.
   LLVMContext &Context;
 
   TypeID   ID : 8;            // The current base type of this type.
   unsigned SubclassData : 24; // Space for subclasses to store data.
                               // Note that this should be synchronized with
                               // MAX_INT_BITS value in IntegerType class.
public:
   /// If this is a vector type, return the element type, otherwise return
   /// 'this'.
   inline Type *getScalarType() const {
     if (isVectorTy())
       return getContainedType(0);
     return const_cast<Type *>(this);
   }
 
+  bool isWasmFuncrefType() const {
+    return isa<PointerType>(this) &&
+           getPointerAddressSpace() ==
+               WasmAddressSpace::WASM_ADDRESS_SPACE_FUNCREF;
+  }
+
+  bool isWasmExternrefType() const {
+    return isa<PointerType>(this) &&
+           getPointerAddressSpace() ==
+               WasmAddressSpace::WASM_ADDRESS_SPACE_EXTERNREF;
+  }
+
+  bool isWasmRefType() const {
+    return isWasmFuncrefType() || isWasmExternrefType();
+  }
+
   //===--------------------------------------------------------------------===//
   // Type Iteration support.
   //
   using subtype_iterator = Type * const *;
 
   subtype_iterator subtype_begin() const { return ContainedTys; }
   subtype_iterator subtype_end() const { return &ContainedTys[NumContainedTys];}
   ArrayRef<Type*> subtypes() const {

llvm/include/llvm/Support/MachineValueType.h

enum SimpleValueType : uint8_t {
       isVoid         = 173,    // This has no value
 
       Untyped        = 174,    // This value takes a register, but has
                                // unspecified type.  The register class
                                // will be determined by the opcode.
 
       funcref        = 175,    // WebAssembly's funcref type
       externref      = 176,    // WebAssembly's externref type
+
+      FIRST_REFTYPE_VALUETYPE = funcref,
+      LAST_REFTYPE_VALUETYPE = externref,
+      
       x86amx         = 177,    // This is an X86 AMX value
       i64x8          = 178,    // 8 Consecutive GPRs (AArch64)
 
       FIRST_VALUETYPE =  1,    // This is always the beginning of the list.
       LAST_VALUETYPE = i64x8,  // This always remains at the end of the list.
       VALUETYPE_SIZE = LAST_VALUETYPE + 1,
 
       // This is the current maximum for LAST_VALUETYPE.
       // MVT::MAX_ALLOWED_VALUETYPE is used for asserts and to size bit vectors
       // This value must be a multiple of 32.
       MAX_ALLOWED_VALUETYPE = 192,
       
       // A value of type llvm::TokenTy
-      token          = 248,
+      token          = 247,
 
       // This is MDNode or MDString.
-      Metadata       = 249,
+      Metadata       = 248,
 
       // An int value the size of the pointer of the current
       // target to any address space. This must only be used internal to
       // tblgen. Other than for overloading, we treat iPTRAny the same as iPTR.
-      iPTRAny        = 250,
+      iPTRAny        = 249,
 
       // A vector with any length and element size. This is used
       // for intrinsics that have overloadings based on vector types.
       // This is only for tblgen's consumption!
-      vAny           = 251,
+      vAny           = 250,
 
       // Any floating-point or vector floating-point value. This is used
       // for intrinsics that have overloadings based on floating-point types.
       // This is only for tblgen's consumption!
-      fAny           = 252,
+      fAny           = 251,
 
       // An integer or vector integer value of any bit width. This is
       // used for intrinsics that have overloadings based on integer bit widths.
       // This is only for tblgen's consumption!
-      iAny           = 253,
+      iAny           = 252,
+
+      // Any Wasm reference type
+      rAny           = 253,
 
       // An int value the size of the pointer of the current
       // target.  This should only be used internal to tblgen!
       iPTR           = 254,
 
       // Any type. This is used for intrinsics that have overloadings.
       // This is only for tblgen's consumption!
       Any            = 255
bool is2048BitVector() const {
               SimpleTy == MVT::v128f16 || SimpleTy == MVT::v64f32  ||
               SimpleTy == MVT::v32f64  || SimpleTy == MVT::v128bf16);
     }
 
     /// Return true if this is an overloaded type for TableGen.
     bool isOverloaded() const {
       return (SimpleTy == MVT::Any || SimpleTy == MVT::iAny ||
               SimpleTy == MVT::fAny || SimpleTy == MVT::vAny ||
-              SimpleTy == MVT::iPTRAny);
+              SimpleTy == MVT::iPTRAny || SimpleTy == MVT::rAny);
     }
 
     /// Return a vector with the same number of elements as this vector, but
     /// with the element type converted to an integer type with the same
     /// bitwidth.
     MVT changeVectorElementTypeToInteger() const {
       MVT EltTy = getVectorElementType();
       MVT IntTy = MVT::getIntegerVT(EltTy.getSizeInBits());
TypeSize getSizeInBits() const {
         llvm_unreachable("Value type is non-standard value, Other.");
       case iPTR:
         llvm_unreachable("Value type size is target-dependent. Ask TLI.");
       case iPTRAny:
       case iAny:
       case fAny:
       case vAny:
       case Any:
+      case rAny:
         llvm_unreachable("Value type is overloaded.");
       case token:
         llvm_unreachable("Token type is a sentinel that cannot be used "
                          "in codegen and has no size");
       case Metadata:
         llvm_unreachable("Value type is metadata.");
       case i1:
       case v1i1: return TypeSize::Fixed(1);

   public:
     /// SimpleValueType Iteration
     /// @{
     static auto all_valuetypes() {
       return enum_seq_inclusive(MVT::FIRST_VALUETYPE, MVT::LAST_VALUETYPE,
                                 force_iteration_on_noniterable_enum);
     }
 
+    static auto ref_valuetypes() {
+      return enum_seq_inclusive(MVT::FIRST_REFTYPE_VALUETYPE,
+                                MVT::LAST_REFTYPE_VALUETYPE,
+                                force_iteration_on_noniterable_enum);
+    }
+
     static auto integer_valuetypes() {
       return enum_seq_inclusive(MVT::FIRST_INTEGER_VALUETYPE,
                                 MVT::LAST_INTEGER_VALUETYPE,
                                 force_iteration_on_noniterable_enum);
     }
 
     static auto fp_valuetypes() {
       return enum_seq_inclusive(MVT::FIRST_FP_VALUETYPE, MVT::LAST_FP_VALUETYPE,

llvm/lib/IR/Function.cpp

case IITDescriptor::Argument:
       ArgTys.push_back(Ty);
 
       switch (D.getArgumentKind()) {
         case IITDescriptor::AK_Any:        return false; // Success
         case IITDescriptor::AK_AnyInteger: return !Ty->isIntOrIntVectorTy();
         case IITDescriptor::AK_AnyFloat:   return !Ty->isFPOrFPVectorTy();
         case IITDescriptor::AK_AnyVector:  return !isa<VectorType>(Ty);
         case IITDescriptor::AK_AnyPointer: return !isa<PointerType>(Ty);
+        case IITDescriptor::AK_AnyRef:
+          return !Ty->isWasmRefType();
         default:                           break;
       }
       llvm_unreachable("all argument kinds not covered");
 
     case IITDescriptor::ExtendArgument: {
       // If this is a forward reference, defer the check for later.
       if (D.getArgumentNumber() >= ArgTys.size())
         return IsDeferredCheck || DeferCheck(Ty);

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.h

 /// utility functions.
 ///
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_LIB_TARGET_WEBASSEMBLY_UTILS_WEBASSEMBLYUTILITIES_H
 #define LLVM_LIB_TARGET_WEBASSEMBLY_UTILS_WEBASSEMBLYUTILITIES_H
 
 #include "llvm/IR/DerivedTypes.h"
+#include "llvm/IR/Type.h"
 
 namespace llvm {
 
 class MachineBasicBlock;
 class MachineInstr;
 class MachineOperand;
 class MCContext;
 class MCSymbolWasm;
 class StringRef;
 class WebAssemblyFunctionInfo;
 class WebAssemblySubtarget;
 
 namespace WebAssembly {
 
-enum WasmAddressSpace : unsigned {
-  // Default address space, for pointers to linear memory (stack, heap, data).
-  WASM_ADDRESS_SPACE_DEFAULT = 0,
-  // A non-integral address space for pointers to named objects outside of
-  // linear memory: WebAssembly globals or WebAssembly locals.  Loads and stores
-  // to these pointers are lowered to global.get / global.set or local.get /
-  // local.set, as appropriate.
-  WASM_ADDRESS_SPACE_VAR = 1,
-  // A non-integral address space for externref values
-  WASM_ADDRESS_SPACE_EXTERNREF = 10,
-  // A non-integral address space for funcref values
-  WASM_ADDRESS_SPACE_FUNCREF = 20,
-};
-
 inline bool isDefaultAddressSpace(unsigned AS) {
-  return AS == WASM_ADDRESS_SPACE_DEFAULT;
+  return AS == Type::WASM_ADDRESS_SPACE_DEFAULT;
 }
 inline bool isWasmVarAddressSpace(unsigned AS) {
-  return AS == WASM_ADDRESS_SPACE_VAR;
+  return AS == Type::WASM_ADDRESS_SPACE_VAR;
 }
 inline bool isValidAddressSpace(unsigned AS) {
   return isDefaultAddressSpace(AS) || isWasmVarAddressSpace(AS);
 }
-inline bool isFuncrefType(const Type *Ty) {
-  return isa<PointerType>(Ty) &&
-         Ty->getPointerAddressSpace() ==
-             WasmAddressSpace::WASM_ADDRESS_SPACE_FUNCREF;
-}
-inline bool isExternrefType(const Type *Ty) {
-  return isa<PointerType>(Ty) &&
-         Ty->getPointerAddressSpace() ==
-             WasmAddressSpace::WASM_ADDRESS_SPACE_EXTERNREF;
-}
-inline bool isRefType(const Type *Ty) {
-  return isFuncrefType(Ty) || isExternrefType(Ty);
-}
 
 bool isChild(const MachineInstr &MI, const WebAssemblyFunctionInfo &MFI);
 bool mayThrow(const MachineInstr &MI);
 
 // Exception-related function names
 extern const char *const ClangCallTerminateFn;
 extern const char *const CxaBeginCatchFn;
 extern const char *const CxaRethrowFn;

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

 #include "llvm/CodeGen/MachineRegisterInfo.h"
 #include "llvm/CodeGen/SelectionDAG.h"
 #include "llvm/CodeGen/SelectionDAGNodes.h"
 #include "llvm/IR/DiagnosticInfo.h"
 #include "llvm/IR/DiagnosticPrinter.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/Intrinsics.h"
 #include "llvm/IR/IntrinsicsWebAssembly.h"
+#include "llvm/IR/Type.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/KnownBits.h"
 #include "llvm/Support/MathExtras.h"
 #include "llvm/Support/raw_ostream.h"
 #include "llvm/Target/TargetOptions.h"
 using namespace llvm;
 
WebAssemblyTargetLowering::WebAssemblyTargetLowering(
   // Always convert switches to br_tables unless there is only one case, which
   // is equivalent to a simple branch. This reduces code size for wasm, and we
   // defer possible jump table optimizations to the VM.
   setMinimumJumpTableEntries(2);
 }
 
 MVT WebAssemblyTargetLowering::getPointerTy(const DataLayout &DL,
                                             uint32_t AS) const {
-  if (AS == WebAssembly::WasmAddressSpace::WASM_ADDRESS_SPACE_EXTERNREF)
+  if (AS == Type::WasmAddressSpace::WASM_ADDRESS_SPACE_EXTERNREF)
     return MVT::externref;
-  if (AS == WebAssembly::WasmAddressSpace::WASM_ADDRESS_SPACE_FUNCREF)
+  if (AS == Type::WasmAddressSpace::WASM_ADDRESS_SPACE_FUNCREF)
     return MVT::funcref;
   return TargetLowering::getPointerTy(DL, AS);
 }
 
 MVT WebAssemblyTargetLowering::getPointerMemTy(const DataLayout &DL,
                                                uint32_t AS) const {
-  if (AS == WebAssembly::WasmAddressSpace::WASM_ADDRESS_SPACE_EXTERNREF)
+  if (AS == Type::WasmAddressSpace::WASM_ADDRESS_SPACE_EXTERNREF)
     return MVT::externref;
-  if (AS == WebAssembly::WasmAddressSpace::WASM_ADDRESS_SPACE_FUNCREF)
+  if (AS == Type::WasmAddressSpace::WASM_ADDRESS_SPACE_FUNCREF)
     return MVT::funcref;
   return TargetLowering::getPointerMemTy(DL, AS);
 }
 
 TargetLowering::AtomicExpansionKind
 WebAssemblyTargetLowering::shouldExpandAtomicRMWInIR(AtomicRMWInst *AI) const {
   // We have wasm instructions for these
   switch (AI->getOperation()) {
if (In.Flags.isInConsecutiveRegsLast())
            "WebAssembly hasn't implemented cons regs last return values");
     // Ignore In.getNonZeroOrigAlign() because all our arguments are passed in
     // registers.
     InTys.push_back(In.VT);
   }
 
   // Lastly, if this is a call to a funcref we need to add an instruction
   // table.set to the chain and transform the call.
-  if (CLI.CB &&
-      WebAssembly::isFuncrefType(CLI.CB->getCalledOperand()->getType())) {
+  if (CLI.CB && CLI.CB->getCalledOperand()->getType()->isWasmFuncrefType()) {
     // In the absence of function references proposal where a funcref call is
     // lowered to call_ref, using reference types we generate a table.set to set
     // the funcref to a special table used solely for this purpose, followed by
     // a call_indirect. Here we just generate the table set, and return the
     // SDValue of the table.set so that LowerCall can finalize the lowering by
     // generating the call_indirect.
     SDValue Chain = Ops[0];
 
     MCSymbolWasm *Table = WebAssembly::getOrCreateFuncrefCallTableSymbol(
         MF.getContext(), Subtarget);
     SDValue Sym = DAG.getMCSymbol(Table, PtrVT);
     SDValue TableSlot = DAG.getConstant(0, DL, MVT::i32);
     SDValue TableSetOps[] = {Chain, Sym, TableSlot, Callee};
     SDValue TableSet = DAG.getMemIntrinsicNode(
         WebAssemblyISD::TABLE_SET, DL, DAG.getVTList(MVT::Other), TableSetOps,
         MVT::funcref,
         // Machine Mem Operand args
-        MachinePointerInfo(
-            WebAssembly::WasmAddressSpace::WASM_ADDRESS_SPACE_FUNCREF),
+        MachinePointerInfo(Type::WasmAddressSpace::WASM_ADDRESS_SPACE_FUNCREF),
         CLI.CB->getCalledOperand()->getPointerAlignment(DAG.getDataLayout()),
         MachineMemOperand::MOStore);
 
     Ops[0] = TableSet; // The new chain is the TableSet itself
   }
 
   if (CLI.IsTailCall) {
     // ret_calls do not return values to the current frame

 }
 
 static bool IsWebAssemblyTable(SDValue Op) {
   const GlobalAddressSDNode *GA = dyn_cast<GlobalAddressSDNode>(Op);
   if (GA && WebAssembly::isWasmVarAddressSpace(GA->getAddressSpace())) {
     const GlobalValue *Value = GA->getGlobal();
     const Type *Ty = Value->getValueType();
 
-    if (Ty->isArrayTy() && WebAssembly::isRefType(Ty->getArrayElementType()))
+    if (Ty->isArrayTy() && Ty->getArrayElementType()->isWasmRefType())
       return true;
   }
   return false;
 }
 
 // This function will accept as Op any access to a table, so Op can
 // be the actual table or an offset into the table.
 static bool IsWebAssemblyTableWithOffset(SDValue Op) {

llvm/lib/Target/WebAssembly/WebAssemblyLowerRefTypesIntPtrConv.cpp

bool WebAssemblyLowerRefTypesIntPtrConv::runOnFunction(Function &F) {
   // We replace the instruction by a trap instruction
   // and its uses by null in the case of inttoptr and 0 in the
   // case of ptrtoint.
   std::set<Instruction *> worklist;
 
   for (inst_iterator I = inst_begin(F), E = inst_end(F); I != E; ++I) {
     PtrToIntInst *PTI = dyn_cast<PtrToIntInst>(&*I);
     IntToPtrInst *ITP = dyn_cast<IntToPtrInst>(&*I);
-    if (!(PTI && WebAssembly::isRefType(PTI->getPointerOperand()->getType())) &&
-        !(ITP && WebAssembly::isRefType(ITP->getDestTy())))
+    if (!(PTI && PTI->getPointerOperand()->getType()->isWasmRefType()) &&
+        !(ITP && ITP->getDestTy()->isWasmRefType()))
       continue;
 
     UndefValue *U = UndefValue::get(I->getType());
     I->replaceAllUsesWith(U);
 
     Function *TrapIntrin =
         Intrinsic::getDeclaration(F.getParent(), Intrinsic::debugtrap);
     CallInst::Create(TrapIntrin, {}, "", &*I);

llvm/lib/Target/WebAssembly/WebAssemblyMCInstLower.cpp

 #include "Utils/WebAssemblyTypeUtilities.h"
 #include "Utils/WebAssemblyUtilities.h"
 #include "WebAssemblyAsmPrinter.h"
 #include "WebAssemblyISelLowering.h"
 #include "WebAssemblyMachineFunctionInfo.h"
 #include "llvm/CodeGen/AsmPrinter.h"
 #include "llvm/CodeGen/MachineFunction.h"
 #include "llvm/IR/Constants.h"
+#include "llvm/IR/Type.h"
 #include "llvm/MC/MCAsmInfo.h"
 #include "llvm/MC/MCContext.h"
 #include "llvm/MC/MCExpr.h"
 #include "llvm/MC/MCInst.h"
 #include "llvm/MC/MCSymbolWasm.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/raw_ostream.h"
 
if (WebAssembly::isWasmVarAddressSpace(Global->getAddressSpace()) &&
       SmallVector<MVT, 1> VTs;
       computeLegalValueVTs(CurrentFunc, TM, GlobalVT, VTs);
 
       // Tables are represented as Arrays in LLVM IR therefore
       // they reach this point as aggregate Array types with an element type
       // that is a reference type.
       wasm::ValType Type;
       if (GlobalVT->isArrayTy() &&
-          WebAssembly::isRefType(GlobalVT->getArrayElementType())) {
+          GlobalVT->getArrayElementType()->isWasmRefType()) {
         MVT VT;
         switch (GlobalVT->getArrayElementType()->getPointerAddressSpace()) {
-        case WebAssembly::WasmAddressSpace::WASM_ADDRESS_SPACE_FUNCREF:
+        case Type::WasmAddressSpace::WASM_ADDRESS_SPACE_FUNCREF:
           VT = MVT::funcref;
           break;
-        case WebAssembly::WasmAddressSpace::WASM_ADDRESS_SPACE_EXTERNREF:
+        case Type::WasmAddressSpace::WASM_ADDRESS_SPACE_EXTERNREF:
           VT = MVT::externref;
           break;
         default:
           report_fatal_error("unhandled address space type");
         }
         Type = WebAssembly::toValType(VT);
       } else if (VTs.size() == 1) {
         Type = WebAssembly::toValType(VTs[0]);

llvm/utils/TableGen/CodeGenDAGPatterns.cpp

switch (Ov.SimpleTy) {
           if (Legal.count(T))
             Out.insert(T);
         return;
       case MVT::vAny:
         for (MVT T : MVT::vector_valuetypes())
           if (Legal.count(T))
             Out.insert(T);
         return;
+      case MVT::rAny:
+        for (MVT T : MVT::ref_valuetypes())
+          if (Legal.count(T))
+            Out.insert(T);
+        return;
       case MVT::Any:
         for (MVT T : MVT::all_valuetypes())
           if (Legal.count(T))
             Out.insert(T);
         return;
       default:
         break;
     }

llvm/utils/TableGen/CodeGenTarget.cpp

StringRef llvm::getEnumName(MVT::SimpleValueType T) {
   case MVT::i16:      return "MVT::i16";
   case MVT::i32:      return "MVT::i32";
   case MVT::i64:      return "MVT::i64";
   case MVT::i128:     return "MVT::i128";
   case MVT::Any:      return "MVT::Any";
   case MVT::iAny:     return "MVT::iAny";
   case MVT::fAny:     return "MVT::fAny";
   case MVT::vAny:     return "MVT::vAny";
+  case MVT::rAny:
+    return "MVT::rAny";
   case MVT::f16:      return "MVT::f16";
   case MVT::bf16:     return "MVT::bf16";
   case MVT::f32:      return "MVT::f32";
   case MVT::f64:      return "MVT::f64";
   case MVT::f80:      return "MVT::f80";
   case MVT::f128:     return "MVT::f128";
   case MVT::ppcf128:  return "MVT::ppcf128";
   case MVT::x86mmx:   return "MVT::x86mmx";

llvm/utils/TableGen/IntrinsicEmitter.cpp

else if (R->isSubClassOf("LLVMVectorOfBitcastsToInt"))
       Sig.push_back(IIT_VEC_OF_BITCASTS_TO_INT);
     else
       Sig.push_back(IIT_ARG);
     return Sig.push_back((Number << 3) | 7 /*IITDescriptor::AK_MatchType*/);
   }
 
   MVT::SimpleValueType VT = getValueType(R->getValueAsDef("VT"));
 
+  // Note: The order in which these cases show up need
+  // to match the inverse order of the respective ArgKind in
+  // Intrinsics.h. The value of Tmp when it reaches MVT::Any needs
+  // to be the value of the respective ArgKind enum.
   unsigned Tmp = 0;
   switch (VT) {
   default: break;
+  case MVT::rAny:
+    ++Tmp;
+    LLVM_FALLTHROUGH;
   case MVT::iPTRAny: ++Tmp; LLVM_FALLTHROUGH;
   case MVT::vAny: ++Tmp;    LLVM_FALLTHROUGH;
   case MVT::fAny: ++Tmp;    LLVM_FALLTHROUGH;
   case MVT::iAny: ++Tmp;    LLVM_FALLTHROUGH;
   case MVT::Any: {
     // If this is an "any" valuetype, then the type is the type of the next
     // type in the list specified to getIntrinsic().
     Sig.push_back(IIT_ARG);
static void UpdateArgCodes(Record *R, std::vector<unsigned char> &ArgCodes,
   if (R->isSubClassOf("LLVMMatchType")) {
     if (R->isSubClassOf("LLVMVectorOfAnyPointersToElt")) {
       ArgCodes.push_back(3 /*vAny*/);
       ++NumInserted;
     }
     return;
   }
 
+  // Note: The order in which these cases show up need
+  // to match the inverse order of the respective ArgKind in
+  // Intrinsics.h. The value of Tmp when it reaches MVT::Any needs
+  // to be the value of the respective ArgKind enum.
   unsigned Tmp = 0;
   switch (getValueType(R->getValueAsDef("VT"))) {
   default: break;
   case MVT::iPTR:
     UpdateArgCodes(R->getValueAsDef("ElTy"), ArgCodes, NumInserted, Mapping);
     break;
+  case MVT::rAny:
+    ++Tmp;
+    LLVM_FALLTHROUGH;
   case MVT::iPTRAny:
     ++Tmp;
     LLVM_FALLTHROUGH;
   case MVT::vAny:
     ++Tmp;
     LLVM_FALLTHROUGH;
   case MVT::fAny:
     ++Tmp;