[WebAssembly] Implement table instruction intrinsics and ref.null

[pmatos]

Summary

This change implements intrinsics for ref.null, table.grow,
table.fill, table.size and table.copy.

[pmatos]

After D111227 was refactored to just contain the polymorphic type implementation, I refactored this revision to contain all the implementation of the table intrinsics and ref.null

[tlively]

llvm/lib/CodeGen/ValueTypes.cpp

203–206

I think we should probably simplify things by making externref be an i8 pointer as well. That's simpler than the other arbitrary type (pointer (to pointer?) to opaque struct) that we are using now, and ultimately the type doesn't really matter. Maybe these comments should say "arbitrary pointer" rather than "opaque pointer" (which I don't think is quite correct, since it's the struct that's opaque rather than the pointer) or "void pointer" (which is not really a concept that exists in LLVM).

llvm/lib/IR/Function.cpp

927

The changes in this file look like they should be in the previous revision rather than this one. Is that right?

llvm/lib/Target/WebAssembly/WebAssemblyInstrRef.td

14–19

Since the rc, vt, and ht all correspond 1:1 with each other, you can package them together into records that let you just have a single parameter here. For an example of this pattern, see class Vec and all its instantiations in WebAssemblyInstrSIMD.td.

llvm/test/CodeGen/WebAssembly/funcref-call.ll

7–9

I'm not sure it should be necessary to modify these tests to use i8 as the pointee type. Ultimately the pointee type shouldn't really matter at all (except at the point when a funcref is called) and we should be relying entirely on the address space to figure out what Wasm type it should be. In fact, it would be useful to test that multiple pointee types work correctly. That will give us confidence that nothing will break when opaque pointers are enabled.

llvm/test/CodeGen/WebAssembly/table-copy.ll

21

Can you skip the previous getelementptr and cast the original table pointer directly to i8 addrspace(1)*, at least when the offset is 0? It would also be good to test with various nontrivial offsets and when copying from a table to itself. Same for the other intrinsics.

llvm/utils/TableGen/IntrinsicEmitter.cpp

197

These changes also looks like they should be in the previous revision rather than this one.

[pmatos]

After closing D111227, I moved the implementation of all intrinsics to this revision without the need for the polymorphic llvm_anyref_ty type.

There's a single niggle here that I haven't been able to fix and would like further comments on.

Test for reference-types.s is failing due to the parsing of ref.null $heaptype, be it func or extern.
The reason for this is failing is that I have removed the need to specify the input argument heaptype for the ref.null instruction. This allows me to define an intrinsic that doesn't receive an argument. If I specify a heaptype as an input argument, then TableGen really wants the argument to show up in the instruction pattern, but I would really like it to be ignored. Adding extern or func as an argument to the intrinsic just seems overcomplicated. On the other hand, I have been stuck with this for a while and would appreciate some fresh ideas.

[pmatos]

llvm/lib/Target/WebAssembly/WebAssemblyInstrRef.td

18

The problem I alluded to in https://reviews.llvm.org/D113420#3151549 happens here. Because I don't mention heaptype as inputs, the parser fails. But if I do add as inputs heaptype, and not in the instruction pattern (where the intrinsic is), then tablegen does not accept it. It makes, however, no sense to have heaptype as an argument since it can be suffix to the intrinsic name.

[pmatos]

Post discussion with @tlively I have changed the textual representation of ref.null to include the heaptype as suffix.
As a consequence the current patch includes all the intrinsics implementations but also the changes to the tests that now use ref.null_extern and ref.null_func, and the removal of the references to HeapType which is now not required anymore.

[pmatos]

To make the changes easier to understand, I am splitting this into two. One for ref.null + HeapType changes and another for the table intrinsics.

[pmatos]

Remove changes related to ref.null.

ref.null intrinsic implementation and heaptype removal work is currently in D114979 and this change builds upon that.

[pmatos]

llvm/lib/CodeGen/ValueTypes.cpp

203–206

I agree that the comments should be changed, but don't you think that externref should still be a pointer to an opaque struct? This seems to match better with what externref truly represents: a host value that you cannot introspect - therefore opaque. I am not sure it matters much as you say, but semantically it seems to be closer to what externref is.

llvm/test/CodeGen/WebAssembly/table-copy.ll

21

Good point! I will take a look at these.

[pmatos]

llvm/test/CodeGen/WebAssembly/funcref-call.ll

7–9

The reason I changed this for all tests is for consistency with intrinsics tests. Intrinsics tests need to define the funcref as i8 pointers to addspace(20), because the intrinsics validator post parse checks that the type defined in LLVM IR matches the type of the intrinsic. Even though it might not be necessary to change it in this specific testcase, I find that for consistency we should always define the funcref type as a type i8 addspace(20).

[pmatos]

Fix a couple of comments - add a new test to table.copy for copies within the same table.

llvm/test/CodeGen/WebAssembly/table-copy.ll

30

@tlively is this the kind of test you were referring to?

[tlively]

LGTM with the extra suggested test, if possible.

llvm/test/CodeGen/WebAssembly/funcref-call.ll

7–9

I agree that it makes sense to be consistent in the majority of the tests, but perhaps we could have an additional test that only checks that the pointee type doesn't matter? That will help us be confident that we will be ready for the removal of pointee types once the wider project makes that switch.

llvm/test/CodeGen/WebAssembly/table-copy.ll

30

Looks good! I was probably thinking of having different offsets in the getelementptr, but not supporting that for now and taking the offsets as arguments to the copy intrinsic seems much simpler.

[pmatos]

llvm/test/CodeGen/WebAssembly/table-copy.ll

30

Humm, I see what you mean but isn't it always the case that since we are dereferencing a global, the GEP will always have zero offsets? I cannot imagine a case, where we use non-zero offsets to dereference a global representing a table, like we do for example in the llvm/test/CodeGen/WebAssembly/funcref-tableget.ll test.

[ljmf00]

In D113420#3151549, @pmatos wrote:

After closing D111420, I moved the implementation of all intrinsics to this revision without the need for the polymorphic llvm_anyref_ty type.

@pmatos You likely referenced the wrong diff. I would suggest amending this to avoid confusion in the future :)

[pmatos]

In D113420#3239319, @ljmf00 wrote:

In D113420#3151549, @pmatos wrote:

After closing D111420, I moved the implementation of all intrinsics to this revision without the need for the polymorphic llvm_anyref_ty type.

@pmatos You likely referenced the wrong diff. I would suggest amending this to avoid confusion in the future :)

Done - thanks for letting me know.

llvm/include/llvm/IR/Intrinsics.td

 def llvm_v1f64_ty      : LLVMType<v1f64>;    //  1 x double
 def llvm_v2f64_ty      : LLVMType<v2f64>;    //  2 x double
 def llvm_v4f64_ty      : LLVMType<v4f64>;    //  4 x double
 def llvm_v8f64_ty      : LLVMType<v8f64>;    //  8 x double
 def llvm_v16f64_ty     : LLVMType<v16f64>;   // 16 x double
 
 def llvm_vararg_ty     : LLVMType<isVoid>;   // this means vararg here
 
+def llvm_externref_ty  : LLVMType<externref>;
+def llvm_funcref_ty    : LLVMType<funcref>;
+
 //===----------------------------------------------------------------------===//
 // Intrinsic Definitions.
 //===----------------------------------------------------------------------===//
 
 // Intrinsic class - This is used to define one LLVM intrinsic.  The name of the
 // intrinsic definition should start with "int_", then match the LLVM intrinsic
 // name with the "llvm." prefix removed, and all "."s turned into "_"s.  For
 // example, llvm.bswap.i16 -> int_bswap_i16.

llvm/include/llvm/IR/IntrinsicsWebAssembly.td

 //===- IntrinsicsWebAssembly.td - Defines wasm intrinsics --*- tablegen -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// This file defines all of the WebAssembly-specific intrinsics.
 ///
 //===----------------------------------------------------------------------===//
 
+// Type definition for a table in an intrinsic
+def llvm_table_ty : LLVMQualPointerType<llvm_i8_ty, 1>;
+
 let TargetPrefix = "wasm" in {  // All intrinsics start with "llvm.wasm.".
 
 // Query the current memory size, and increase the current memory size.
 // Note that memory.size is not IntrNoMem because it must be sequenced with
 // respect to memory.grow calls.
 def int_wasm_memory_size : Intrinsic<[llvm_anyint_ty],
                                      [llvm_i32_ty],
                                      [IntrReadMem]>;
 def int_wasm_memory_grow : Intrinsic<[llvm_anyint_ty],
                                      [llvm_i32_ty, LLVMMatchType<0>],
                                      []>;
 
 //===----------------------------------------------------------------------===//
+// Table intrinsics
+//===----------------------------------------------------------------------===//
+def int_wasm_ref_null_extern : Intrinsic<[llvm_externref_ty], [], [IntrNoMem]>;
+def int_wasm_ref_null_func : Intrinsic<[llvm_funcref_ty], [], [IntrNoMem]>;
+
+//===----------------------------------------------------------------------===//
+// Table intrinsics
+//===----------------------------------------------------------------------===//
+// Query the current table size, and increase the current table size.
+def int_wasm_table_size : Intrinsic<[llvm_i32_ty],
+                                    [llvm_table_ty],
+                                    [IntrReadMem]>;
+def int_wasm_table_copy : Intrinsic<[],
+                                    [llvm_table_ty, llvm_table_ty, llvm_i32_ty, llvm_i32_ty, llvm_i32_ty],
+                                    []>;
+def int_wasm_table_grow_externref : Intrinsic<[llvm_i32_ty],
+                                              [llvm_table_ty, llvm_externref_ty, llvm_i32_ty],
+                                              []>;
+def int_wasm_table_grow_funcref : Intrinsic<[llvm_i32_ty],
+                                            [llvm_table_ty, llvm_funcref_ty, llvm_i32_ty],
+                                            []>;
+def int_wasm_table_fill_externref : Intrinsic<[],
+                                              [llvm_table_ty, llvm_i32_ty, llvm_externref_ty, llvm_i32_ty],
+                                              []>;
+def int_wasm_table_fill_funcref : Intrinsic<[],
+                                            [llvm_table_ty, llvm_i32_ty, llvm_funcref_ty, llvm_i32_ty],
+                                            []>;
+
+//===----------------------------------------------------------------------===//
 // Trapping float-to-int conversions
 //===----------------------------------------------------------------------===//
 
 def int_wasm_trunc_signed : Intrinsic<[llvm_anyint_ty],
                                       [llvm_anyfloat_ty],
                                       [IntrNoMem]>;
 def int_wasm_trunc_unsigned : Intrinsic<[llvm_anyint_ty],
                                         [llvm_anyfloat_ty],

llvm/lib/CodeGen/ValueTypes.cpp

Type *EVT::getTypeForEVT(LLVMContext &Context) const {
   case MVT::f32:     return Type::getFloatTy(Context);
   case MVT::f64:     return Type::getDoubleTy(Context);
   case MVT::f80:     return Type::getX86_FP80Ty(Context);
   case MVT::f128:    return Type::getFP128Ty(Context);
   case MVT::ppcf128: return Type::getPPC_FP128Ty(Context);
   case MVT::x86mmx:  return Type::getX86_MMXTy(Context);
   case MVT::x86amx:  return Type::getX86_AMXTy(Context);
   case MVT::i64x8:   return IntegerType::get(Context, 512);
   case MVT::externref:
-    return PointerType::get(StructType::create(Context), 10);
+    return PointerType::get(StructType::create(Context),
+                            10); // opaque pointer to addrspace(10)
   case MVT::funcref:
-    return PointerType::get(StructType::create(Context), 20);
+    return PointerType::get(Type::getInt8Ty(Context),
+                            20); // void pointer to addrspace(20)
   case MVT::v1i1:
     return FixedVectorType::get(Type::getInt1Ty(Context), 1);
   case MVT::v2i1:
     return FixedVectorType::get(Type::getInt1Ty(Context), 2);
   case MVT::v4i1:
     return FixedVectorType::get(Type::getInt1Ty(Context), 4);
   case MVT::v8i1:
     return FixedVectorType::get(Type::getInt1Ty(Context), 8);

llvm/lib/IR/Function.cpp

 std::string Intrinsic::getNameNoUnnamedTypes(ID Id, ArrayRef<Type *> Tys) {
   return getIntrinsicNameImpl(Id, Tys, nullptr, nullptr, false);
 }
 
 /// IIT_Info - These are enumerators that describe the entries returned by the
 /// getIntrinsicInfoTableEntries function.
 ///
 /// NOTE: This must be kept in synch with the copy in TblGen/IntrinsicEmitter!
 enum IIT_Info {
   // Common values should be encoded with 0-15.
   IIT_Done = 0,
-  IIT_I1   = 1,
-  IIT_I8   = 2,
-  IIT_I16  = 3,
-  IIT_I32  = 4,
-  IIT_I64  = 5,
-  IIT_F16  = 6,
-  IIT_F32  = 7,
-  IIT_F64  = 8,
-  IIT_V2   = 9,
-  IIT_V4   = 10,
-  IIT_V8   = 11,
-  IIT_V16  = 12,
-  IIT_V32  = 13,
-  IIT_PTR  = 14,
-  IIT_ARG  = 15,
+  IIT_I1 = 1,
+  IIT_I8 = 2,
+  IIT_I16 = 3,
+  IIT_I32 = 4,
+  IIT_I64 = 5,
+  IIT_F16 = 6,
+  IIT_F32 = 7,
+  IIT_F64 = 8,
+  IIT_V2 = 9,
+  IIT_V4 = 10,
+  IIT_V8 = 11,
+  IIT_V16 = 12,
+  IIT_V32 = 13,
+  IIT_PTR = 14,
+  IIT_ARG = 15,
 
   // Values from 16+ are only encodable with the inefficient encoding.
-  IIT_V64  = 16,
-  IIT_MMX  = 17,
+  IIT_V64 = 16,
+  IIT_MMX = 17,
   IIT_TOKEN = 18,
   IIT_METADATA = 19,
   IIT_EMPTYSTRUCT = 20,
   IIT_STRUCT2 = 21,
   IIT_STRUCT3 = 22,
   IIT_STRUCT4 = 23,
   IIT_STRUCT5 = 24,
   IIT_EXTEND_ARG = 25,
   IIT_TRUNC_ARG = 26,
   IIT_ANYPTR = 27,
-  IIT_V1   = 28,
+  IIT_V1 = 28,
   IIT_VARARG = 29,
   IIT_HALF_VEC_ARG = 30,
   IIT_SAME_VEC_WIDTH_ARG = 31,
   IIT_PTR_TO_ARG = 32,
   IIT_PTR_TO_ELT = 33,
   IIT_VEC_OF_ANYPTRS_TO_ELT = 34,
   IIT_I128 = 35,
   IIT_V512 = 36,
   IIT_V1024 = 37,
   IIT_STRUCT6 = 38,
   IIT_STRUCT7 = 39,
   IIT_STRUCT8 = 40,
   IIT_F128 = 41,
   IIT_VEC_ELEMENT = 42,
   IIT_SCALABLE_VEC = 43,
   IIT_SUBDIVIDE2_ARG = 44,
   IIT_SUBDIVIDE4_ARG = 45,
   IIT_VEC_OF_BITCASTS_TO_INT = 46,
   IIT_V128 = 47,
   IIT_BF16 = 48,
   IIT_STRUCT9 = 49,
   IIT_V256 = 50,
-  IIT_AMX  = 51,
-  IIT_PPCF128 = 52
+  IIT_AMX = 51,
+  IIT_PPCF128 = 52,
+  IIT_EXTERNREF = 53,
+  IIT_FUNCREF = 54
 };
 
 static void DecodeIITType(unsigned &NextElt, ArrayRef<unsigned char> Infos,
                       IIT_Info LastInfo,
                       SmallVectorImpl<Intrinsic::IITDescriptor> &OutputTable) {
   using namespace Intrinsic;
 
   bool IsScalableVector = (LastInfo == IIT_SCALABLE_VEC);
static void DecodeIITType(unsigned &NextElt, ArrayRef<unsigned char> Infos,
   case IIT_V512:
     OutputTable.push_back(IITDescriptor::getVector(512, IsScalableVector));
     DecodeIITType(NextElt, Infos, Info, OutputTable);
     return;
   case IIT_V1024:
     OutputTable.push_back(IITDescriptor::getVector(1024, IsScalableVector));
     DecodeIITType(NextElt, Infos, Info, OutputTable);
     return;
+  case IIT_EXTERNREF:
+    OutputTable.push_back(IITDescriptor::get(IITDescriptor::Pointer, 10));
+    OutputTable.push_back(IITDescriptor::get(IITDescriptor::Struct, 0));
+    return;
+  case IIT_FUNCREF:
+    OutputTable.push_back(IITDescriptor::get(IITDescriptor::Pointer, 20));
+    OutputTable.push_back(IITDescriptor::get(IITDescriptor::Integer, 8));
+    return;
   case IIT_PTR:
     OutputTable.push_back(IITDescriptor::get(IITDescriptor::Pointer, 0));
     DecodeIITType(NextElt, Infos, Info, OutputTable);
     return;
   case IIT_ANYPTR: {  // [ANYPTR addrspace, subtype]
     OutputTable.push_back(IITDescriptor::get(IITDescriptor::Pointer,
                                              Infos[NextElt++]));
     DecodeIITType(NextElt, Infos, Info, OutputTable);

llvm/lib/Target/WebAssembly/AsmParser/WebAssemblyAsmParser.cpp

bool ParseInstruction(ParseInstructionInfo & /*Info*/, StringRef Name,
     Operands.push_back(std::make_unique<WebAssemblyOperand>(
         WebAssemblyOperand::Token, NameLoc, SMLoc::getFromPointer(Name.end()),
         WebAssemblyOperand::TokOp{Name}));
 
     // If this instruction is part of a control flow structure, ensure
     // proper nesting.
     bool ExpectBlockType = false;
     bool ExpectFuncType = false;
-    bool ExpectHeapType = false;
     std::unique_ptr<WebAssemblyOperand> FunctionTable;
     if (Name == "block") {
       push(Block);
       ExpectBlockType = true;
     } else if (Name == "loop") {
       push(Loop);
       ExpectBlockType = true;
     } else if (Name == "try") {
if (Name == "block") {
         return true;
     } else if (Name == "call_indirect" || Name == "return_call_indirect") {
       // These instructions have differing operand orders in the text format vs
       // the binary formats.  The MC instructions follow the binary format, so
       // here we stash away the operand and append it later.
       if (parseFunctionTableOperand(&FunctionTable))
         return true;
       ExpectFuncType = true;
-    } else if (Name == "ref.null") {
-      ExpectHeapType = true;
     }
 
     if (ExpectFuncType || (ExpectBlockType && Lexer.is(AsmToken::LParen))) {
       // This has a special TYPEINDEX operand which in text we
       // represent as a signature, such that we can re-build this signature,
       // attach it to an anonymous symbol, which is what WasmObjectWriter
       // expects to be able to recreate the actual unique-ified type indices.
       auto Loc = Parser.getTok();
while (Lexer.isNot(AsmToken::EndOfStatement)) {
         auto &Id = Lexer.getTok();
         if (ExpectBlockType) {
           // Assume this identifier is a block_type.
           auto BT = WebAssembly::parseBlockType(Id.getString());
           if (BT == WebAssembly::BlockType::Invalid)
             return error("Unknown block type: ", Id);
           addBlockTypeOperand(Operands, NameLoc, BT);
           Parser.Lex();
-        } else if (ExpectHeapType) {
-          auto HeapType = WebAssembly::parseHeapType(Id.getString());
-          if (HeapType == WebAssembly::HeapType::Invalid) {
-            return error("Expected a heap type: ", Id);
-          }
-          Operands.push_back(std::make_unique<WebAssemblyOperand>(
-              WebAssemblyOperand::Integer, Id.getLoc(), Id.getEndLoc(),
-              WebAssemblyOperand::IntOp{static_cast<int64_t>(HeapType)}));
-          Parser.Lex();
         } else {
           // Assume this identifier is a label.
           const MCExpr *Val;
           SMLoc End;
           if (Parser.parseExpression(Val, End))
             return error("Cannot parse symbol: ", Lexer.getTok());
           Operands.push_back(std::make_unique<WebAssemblyOperand>(
               WebAssemblyOperand::Symbol, Id.getLoc(), Id.getEndLoc(),

llvm/lib/Target/WebAssembly/Disassembler/WebAssemblyDisassembler.cpp

case WebAssembly::OPERAND_SIGNATURE: {
         auto *WasmSym = cast<MCSymbolWasm>(Sym);
         WasmSym->setType(wasm::WASM_SYMBOL_TYPE_FUNCTION);
         const MCExpr *Expr = MCSymbolRefExpr::create(
             WasmSym, MCSymbolRefExpr::VK_WASM_TYPEINDEX, getContext());
         MI.addOperand(MCOperand::createExpr(Expr));
       }
       break;
     }
-    // heap_type operands, for e.g. ref.null:
-    case WebAssembly::OPERAND_HEAPTYPE: {
-      int64_t Val;
-      uint64_t PrevSize = Size;
-      if (!nextLEB(Val, Bytes, Size, true))
-        return MCDisassembler::Fail;
-      if (Val < 0 && Size == PrevSize + 1) {
-        // The HeapType encoding is like BlockType, in that encodings that
-        // decode as negative values indicate ValTypes.  In practice we expect
-        // either wasm::ValType::EXTERNREF or wasm::ValType::FUNCREF here.
-        //
-        // The positive SLEB values are reserved for future expansion and are
-        // expected to be type indices in the typed function references
-        // proposal, and should disassemble as MCSymbolRefExpr as in BlockType
-        // above.
-        MI.addOperand(MCOperand::createImm(Val & 0x7f));
-      } else {
-        MI.addOperand(
-            MCOperand::createImm(int64_t(WebAssembly::HeapType::Invalid)));
-      }
-      break;
-    }
     // FP operands.
     case WebAssembly::OPERAND_F32IMM: {
       if (!parseImmediate<float>(MI, Size, Bytes))
         return MCDisassembler::Fail;
       break;
     }
     case WebAssembly::OPERAND_F64IMM: {
       if (!parseImmediate<double>(MI, Size, Bytes))

llvm/lib/Target/WebAssembly/MCTargetDesc/WebAssemblyInstPrinter.h

public:
   // Used by tblegen code.
   void printOperand(const MCInst *MI, unsigned OpNo, raw_ostream &O,
                     bool IsVariadicDef = false);
   void printBrList(const MCInst *MI, unsigned OpNo, raw_ostream &O);
   void printWebAssemblyP2AlignOperand(const MCInst *MI, unsigned OpNo,
                                       raw_ostream &O);
   void printWebAssemblySignatureOperand(const MCInst *MI, unsigned OpNo,
                                         raw_ostream &O);
-  void printWebAssemblyHeapTypeOperand(const MCInst *MI, unsigned OpNo,
-                                       raw_ostream &O);
 
   // Autogenerated by tblgen.
   std::pair<const char *, uint64_t> getMnemonic(const MCInst *MI) override;
   void printInstruction(const MCInst *MI, uint64_t Address, raw_ostream &O);
   static const char *getRegisterName(unsigned RegNo);
 };
 
 } // end namespace llvm
 
 #endif

llvm/lib/Target/WebAssembly/MCTargetDesc/WebAssemblyInstPrinter.cpp

if (Op.isImm()) {
     if (Sym->getSignature()) {
       O << WebAssembly::signatureToString(Sym->getSignature());
     } else {
       // Disassembler does not currently produce a signature
       O << "unknown_type";
     }
   }
 }
-
-void WebAssemblyInstPrinter::printWebAssemblyHeapTypeOperand(const MCInst *MI,
-                                                             unsigned OpNo,
-                                                             raw_ostream &O) {
-  const MCOperand &Op = MI->getOperand(OpNo);
-  if (Op.isImm()) {
-    switch (Op.getImm()) {
-    case long(wasm::ValType::EXTERNREF):
-      O << "extern";
-      break;
-    case long(wasm::ValType::FUNCREF):
-      O << "func";
-      break;
-    default:
-      O << "unsupported_heap_type_value";
-      break;
-    }
-  } else {
-    // Typed function references and other subtypes of funcref and externref
-    // currently unimplemented.
-    O << "unsupported_heap_type_operand";
-  }
-}

llvm/lib/Target/WebAssembly/MCTargetDesc/WebAssemblyMCCodeEmitter.cpp

void WebAssemblyMCCodeEmitter::encodeInstruction(
   // each table entry, and the default operand.
   if (MI.getOpcode() == WebAssembly::BR_TABLE_I32_S ||
       MI.getOpcode() == WebAssembly::BR_TABLE_I64_S)
     encodeULEB128(MI.getNumOperands() - 1, OS);
   if (MI.getOpcode() == WebAssembly::BR_TABLE_I32 ||
       MI.getOpcode() == WebAssembly::BR_TABLE_I64)
     encodeULEB128(MI.getNumOperands() - 2, OS);
 
+  // For REF_NULL, which has no HeapType explicit argument we just explicitly
+  // emit the encoding for the operand manually
+  if (MI.getOpcode() == WebAssembly::REF_NULL_FUNCREF ||
+      MI.getOpcode() == WebAssembly::REF_NULL_FUNCREF_S)
+    OS << uint8_t(0x70);
+  if (MI.getOpcode() == WebAssembly::REF_NULL_EXTERNREF ||
+      MI.getOpcode() == WebAssembly::REF_NULL_EXTERNREF_S)
+    OS << uint8_t(0x6f);
+
   const MCInstrDesc &Desc = MCII.get(MI.getOpcode());
   for (unsigned I = 0, E = MI.getNumOperands(); I < E; ++I) {
     const MCOperand &MO = MI.getOperand(I);
     if (MO.isReg()) {
       /* nothing to encode */
 
     } else if (MO.isImm()) {
       if (I < Desc.getNumOperands()) {
         const MCOperandInfo &Info = Desc.OpInfo[I];
         LLVM_DEBUG(dbgs() << "Encoding immediate: type="
                           << int(Info.OperandType) << "\n");
         switch (Info.OperandType) {
         case WebAssembly::OPERAND_I32IMM:
           encodeSLEB128(int32_t(MO.getImm()), OS);
           break;
         case WebAssembly::OPERAND_OFFSET32:
           encodeULEB128(uint32_t(MO.getImm()), OS);
           break;
         case WebAssembly::OPERAND_I64IMM:
           encodeSLEB128(int64_t(MO.getImm()), OS);
           break;
         case WebAssembly::OPERAND_SIGNATURE:
-        case WebAssembly::OPERAND_HEAPTYPE:
-          OS << uint8_t(MO.getImm());
-          break;
         case WebAssembly::OPERAND_VEC_I8IMM:
           support::endian::write<uint8_t>(OS, MO.getImm(), support::little);
           break;
         case WebAssembly::OPERAND_VEC_I16IMM:
           support::endian::write<uint16_t>(OS, MO.getImm(), support::little);
           break;
         case WebAssembly::OPERAND_VEC_I32IMM:
           support::endian::write<uint32_t>(OS, MO.getImm(), support::little);

llvm/lib/Target/WebAssembly/MCTargetDesc/WebAssemblyMCTargetDesc.h

enum OperandType {
   /// type signature immediate for call_indirect.
   OPERAND_TYPEINDEX,
   /// Tag index.
   OPERAND_TAG,
   /// A list of branch targets for br_list.
   OPERAND_BRLIST,
   /// 32-bit unsigned table number.
   OPERAND_TABLE,
-  /// heap type immediate for ref.null.
-  OPERAND_HEAPTYPE,
 };
 } // end namespace WebAssembly
 
 namespace WebAssemblyII {
 
 /// Target Operand Flag enum.
 enum TOF {
   MO_NO_FLAG = 0,

llvm/lib/Target/WebAssembly/Utils/WebAssemblyTypeUtilities.h

enum class BlockType : unsigned {
   // Multivalue blocks (and other non-void blocks) are only emitted when the
   // blocks will never be exited and are at the ends of functions (see
   // WebAssemblyCFGStackify::fixEndsAtEndOfFunction). They also are never made
   // to pop values off the stack, so the exact multivalue signature can always
   // be inferred from the return type of the parent function in MCInstLower.
   Multivalue = 0xffff,
 };
 
-/// Used as immediate MachineOperands for heap types, e.g. for ref.null.
-enum class HeapType : unsigned {
-  Invalid = 0x00,
-  Externref = unsigned(wasm::ValType::EXTERNREF),
-  Funcref = unsigned(wasm::ValType::FUNCREF),
-};
-
 // Convert StringRef to ValType / HealType / BlockType
 
 Optional<wasm::ValType> parseType(StringRef Type);
-HeapType parseHeapType(StringRef Type);
 BlockType parseBlockType(StringRef Type);
 MVT parseMVT(StringRef Type);
 
 // Convert ValType or a list/signature of ValTypes to a string.
 
 // Convert an unsinged integer, which can be among wasm::ValType enum, to its
 // type name string. If the input is not within wasm::ValType, returns
 // "invalid_type".

llvm/lib/Target/WebAssembly/Utils/WebAssemblyTypeUtilities.cpp

if (Type == "v128" || Type == "i8x16" || Type == "i16x8" || Type == "i32x4" ||
     return wasm::ValType::V128;
   if (Type == "funcref")
     return wasm::ValType::FUNCREF;
   if (Type == "externref")
     return wasm::ValType::EXTERNREF;
   return Optional<wasm::ValType>();
 }
 
-WebAssembly::HeapType WebAssembly::parseHeapType(StringRef Type) {
-  return StringSwitch<WebAssembly::HeapType>(Type)
-      .Case("extern", WebAssembly::HeapType::Externref)
-      .Case("func", WebAssembly::HeapType::Funcref)
-      .Default(WebAssembly::HeapType::Invalid);
-}
-
 WebAssembly::BlockType WebAssembly::parseBlockType(StringRef Type) {
   // Multivalue block types are handled separately in parseSignature
   return StringSwitch<WebAssembly::BlockType>(Type)
       .Case("i32", WebAssembly::BlockType::I32)
       .Case("i64", WebAssembly::BlockType::I64)
       .Case("f32", WebAssembly::BlockType::F32)
       .Case("f64", WebAssembly::BlockType::F64)
       .Case("v128", WebAssembly::BlockType::V128)

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

Register RegZero =
         MF.getRegInfo().createVirtualRegister(&WebAssembly::I32RegClass);
     MachineInstr *Const0 =
         BuildMI(MF, DL, TII.get(WebAssembly::CONST_I32), RegZero).addImm(0);
     BB->insertAfter(MIB.getInstr()->getIterator(), Const0);
 
     Register RegFuncref =
         MF.getRegInfo().createVirtualRegister(&WebAssembly::FUNCREFRegClass);
     MachineInstr *RefNull =
-        BuildMI(MF, DL, TII.get(WebAssembly::REF_NULL_FUNCREF), RegFuncref)
-            .addImm(static_cast<int32_t>(WebAssembly::HeapType::Funcref));
+        BuildMI(MF, DL, TII.get(WebAssembly::REF_NULL_FUNCREF), RegFuncref);
     BB->insertAfter(Const0->getIterator(), RefNull);
 
     MachineInstr *TableSet =
         BuildMI(MF, DL, TII.get(WebAssembly::TABLE_SET_FUNCREF))
             .addSym(Table)
             .addReg(RegZero)
             .addReg(RegFuncref);
     BB->insertAfter(RefNull->getIterator(), TableSet);

llvm/lib/Target/WebAssembly/WebAssemblyInstrInfo.td

 
 } // OperandType = "OPERAND_P2ALIGN"
 
 let OperandType = "OPERAND_SIGNATURE" in
 def Signature : Operand<i32> {
   let PrintMethod = "printWebAssemblySignatureOperand";
 }
 
-let OperandType = "OPERAND_HEAPTYPE" in
-def HeapType : Operand<i32> {
-  let PrintMethod = "printWebAssemblyHeapTypeOperand";
-}
-
 let OperandType = "OPERAND_TYPEINDEX" in
 def TypeIndex : Operand<i32>;
 
 } // OperandNamespace = "WebAssembly"
 
 // TODO: Find more places to use this.
 def bool_node : PatLeaf<(i32 I32:$cond), [{
   return CurDAG->computeKnownBits(SDValue(N, 0)).countMinLeadingZeros() == 31;

llvm/lib/Target/WebAssembly/WebAssemblyInstrRef.td

 // WebAssemblyInstrRef.td - WebAssembly reference type codegen --*- tablegen -*-
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// WebAssembly reference type operand codegen constructs.
 ///
 //===----------------------------------------------------------------------===//
 
-multiclass REF_I<WebAssemblyRegClass rc, ValueType vt> {
-  defm REF_NULL_#rc : I<(outs rc:$res), (ins HeapType:$heaptype),
-                        (outs), (ins HeapType:$heaptype),
-                        [],
-                        "ref.null\t$res, $heaptype",
-                        "ref.null\t$heaptype",
+multiclass REF_I<WebAssemblyRegClass rc, ValueType vt, string ht> {
+  defm REF_NULL_#rc : I<(outs rc:$dst), (ins),
+                        (outs), (ins),
+                        [(set rc:$dst, (!cast<Intrinsic>("int_wasm_ref_null_" # ht)))],
+                        "ref.null_" # ht # "$dst",
+                        "ref.null_" # ht,
                         0xd0>,
                       Requires<[HasReferenceTypes]>;
   defm SELECT_#rc: I<(outs rc:$dst), (ins rc:$lhs, rc:$rhs, I32:$cond),
                      (outs), (ins),
                      [(set rc:$dst,
                        (select I32:$cond, rc:$lhs, rc:$rhs))],
                      vt#".select\t$dst, $lhs, $rhs, $cond",
                      vt#".select", 0x1b>,
                    Requires<[HasReferenceTypes]>;
 }
 
-defm "" : REF_I<FUNCREF, funcref>;
-defm "" : REF_I<EXTERNREF, externref>;
+defm "" : REF_I<FUNCREF, funcref, "func">;
+defm "" : REF_I<EXTERNREF, externref, "extern">;
 
 foreach rc = [FUNCREF, EXTERNREF] in {
 def : Pat<(select (i32 (setne I32:$cond, 0)), rc:$lhs, rc:$rhs),
           (!cast<Instruction>("SELECT_"#rc) rc:$lhs, rc:$rhs, I32:$cond)>;
 def : Pat<(select (i32 (seteq I32:$cond, 0)), rc:$lhs, rc:$rhs),
           (!cast<Instruction>("SELECT_"#rc) rc:$rhs, rc:$lhs, I32:$cond)>;
 }

llvm/lib/Target/WebAssembly/WebAssemblyInstrTable.td

 def WebAssemblyTableSet : SDNode<"WebAssemblyISD::TABLE_SET", WebAssemblyTableSet_t,
                                  [SDNPHasChain, SDNPMayStore, SDNPMemOperand]>;
 
 def WebAssemblyTableGet_t : SDTypeProfile<1, 2, [SDTCisPtrTy<1>]>;
 def WebAssemblyTableGet : SDNode<"WebAssemblyISD::TABLE_GET", WebAssemblyTableGet_t,
                                  [SDNPHasChain, SDNPMayLoad, SDNPMemOperand]>;
 
 
-multiclass TABLE<WebAssemblyRegClass rc> {
+multiclass TABLE<WebAssemblyRegClass rc, string suffix> {
   let mayLoad = 1 in
   defm TABLE_GET_#rc : I<(outs rc:$res), (ins table32_op:$table, I32:$i),
                          (outs), (ins table32_op:$table),
                          [],
                          "table.get\t$res, $table, $i",
                          "table.get\t$table",
                          0x25>;
 
   let mayStore = 1 in
   defm TABLE_SET_#rc : I<(outs), (ins table32_op:$table, I32:$i, rc:$val),
                          (outs), (ins table32_op:$table),
                          [],
                          "table.set\t$table, $i, $val",
                          "table.set\t$table",
                          0x26>;
 
   defm TABLE_GROW_#rc : I<(outs I32:$sz), (ins table32_op:$table, rc:$val, I32:$n),
                           (outs), (ins table32_op:$table),
-                          [],
+                          [(set I32:$sz, (!cast<Intrinsic>("int_wasm_table_grow_" # suffix) (WebAssemblyWrapper tglobaladdr:$table), rc:$val, I32:$n))],
                           "table.grow\t$sz, $table, $val, $n",
                           "table.grow\t$table",
                           0xfc0f>;
 
   defm TABLE_FILL_#rc : I<(outs), (ins table32_op:$table, I32:$i, rc:$val, I32:$n),
                           (outs), (ins table32_op:$table),
-                          [],
+                          [(!cast<Intrinsic>("int_wasm_table_fill_" # suffix) (WebAssemblyWrapper tglobaladdr:$table), I32:$i, rc:$val, I32:$n)],
                           "table.fill\t$table, $i, $val, $n",
                           "table.fill\t$table",
                           0xfc11>;
 
   foreach vt = rc.RegTypes in {
     def : Pat<(vt (WebAssemblyTableGet (WebAssemblyWrapper tglobaladdr:$table), i32:$idx)),
               (!cast<NI>("TABLE_GET_" # rc) tglobaladdr:$table, i32:$idx)>;
     def : Pat<(WebAssemblyTableSet
                (WebAssemblyWrapper tglobaladdr:$table),
                i32:$idx,
                vt:$src),
               (!cast<NI>("TABLE_SET_" # rc) tglobaladdr:$table, i32:$idx, vt:$src)>;
   }
 }
 
-defm "" : TABLE<FUNCREF>, Requires<[HasReferenceTypes]>;
-defm "" : TABLE<EXTERNREF>, Requires<[HasReferenceTypes]>;
+defm "" : TABLE<FUNCREF, "funcref">, Requires<[HasReferenceTypes]>;
+defm "" : TABLE<EXTERNREF, "externref">, Requires<[HasReferenceTypes]>;
 
 def : Pat<(WebAssemblyTableSet mcsym:$table, i32:$idx, funcref:$r),
           (TABLE_SET_FUNCREF mcsym:$table, i32:$idx, funcref:$r)>,
           Requires<[HasReferenceTypes]>;
 
 defm TABLE_SIZE : I<(outs I32:$sz), (ins table32_op:$table),
                     (outs), (ins table32_op:$table),
-                    [],
+                    [(set I32:$sz, (int_wasm_table_size (WebAssemblyWrapper tglobaladdr:$table)))],
                     "table.size\t$sz, $table",
                     "table.size\t$table",
                     0xfc10>,
                     Requires<[HasReferenceTypes]>;
 
 
 defm TABLE_COPY : I<(outs), (ins table32_op:$table1, table32_op:$table2, I32:$d, I32:$s, I32:$n),
                     (outs), (ins table32_op:$table1, table32_op:$table2),
-                    [],
+                    [(int_wasm_table_copy (WebAssemblyWrapper tglobaladdr:$table1),
+                                          (WebAssemblyWrapper tglobaladdr:$table2),
+                                          I32:$d, I32:$s, I32:$n)],
                     "table.copy\t$table1, $table2, $d, $s, $n",
                     "table.copy\t$table1, $table2",
                     0xfc0e>,
                     Requires<[HasReferenceTypes]>;

llvm/lib/Target/WebAssembly/WebAssemblyMCInstLower.cpp

case MachineOperand::MO_Immediate: {
           assert(BT != WebAssembly::BlockType::Invalid);
           if (BT == WebAssembly::BlockType::Multivalue) {
             SmallVector<wasm::ValType, 1> Returns;
             getFunctionReturns(MI, Returns);
             MCOp = lowerTypeIndexOperand(std::move(Returns),
                                          SmallVector<wasm::ValType, 4>());
             break;
           }
-        } else if (Info.OperandType == WebAssembly::OPERAND_HEAPTYPE) {
-          assert(static_cast<WebAssembly::HeapType>(MO.getImm()) !=
-                 WebAssembly::HeapType::Invalid);
-          // With typed function references, this will need a case for type
-          // index operands.  Otherwise, fall through.
         }
       }
       MCOp = MCOperand::createImm(MO.getImm());
       break;
     }
     case MachineOperand::MO_FPImmediate: {
       const ConstantFP *Imm = MO.getFPImm();
       const uint64_t BitPattern =

llvm/test/CodeGen/WebAssembly/funcref-call.ll

 ; RUN: llc < %s --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types | FileCheck %s
 
-%func = type void ()
-%funcref = type %func addrspace(20)* ;; addrspace 20 is nonintegral
+%funcptr = type void () addrspace(20)*
+%funcref = type i8 addrspace(20)* ;; addrspace 20 is nonintegral
 
 define void @call_funcref(%funcref %ref) {
-  call addrspace(20) void %ref() 
+  %f = bitcast %funcref %ref to %funcptr
+  call addrspace(20) void %f() 
   ret void
 }
 
 ; CHECK: .tabletype __funcref_call_table, funcref, 1
 
 ; CHECK-LABEL: call_funcref:
 ; CHECK-NEXT: functype       call_funcref (funcref) -> ()
 ; CHECK-NEXT: i32.const 0
 ; CHECK-NEXT: local.get 0
 ; CHECK-NEXT: table.set __funcref_call_table
 ; CHECK-NEXT: i32.const 0
 ; CHECK-NEXT: call_indirect __funcref_call_table, () -> ()
 ; CHECK-NEXT: i32.const 0
-; CHECK-NEXT: ref.null func
+; CHECK-NEXT: ref.null_func
 ; CHECK-NEXT: table.set __funcref_call_table
 ; CHECK-NEXT: end_function
 

llvm/test/CodeGen/WebAssembly/funcref-globalget.ll

 ; RUN: llc < %s --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types | FileCheck %s
 
-%func = type opaque
-%funcref = type %func addrspace(20)* ;; addrspace 20 is nonintegral
+%funcref = type i8 addrspace(20)* ;; addrspace 20 is nonintegral
 
 @funcref_global = local_unnamed_addr addrspace(1) global %funcref undef
 
 define %funcref @return_funcref_global() {
   ;; this generates a global.get of @funcref_global
   %ref = load %funcref, %funcref addrspace(1)* @funcref_global
   ret %funcref %ref
 }
 
 ; CHECK-LABEL: return_funcref_global:
 ; CHECK-NEXT: .functype       return_funcref_global () -> (funcref)
 ; CHECK-NEXT: global.get funcref_global
 ; CHECK-NEXT: end_function
 
 ; CHECK: .globl funcref_global

llvm/test/CodeGen/WebAssembly/funcref-globalset.ll

 ; RUN: llc < %s --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types | FileCheck %s
 
-%func = type opaque
-%funcref = type %func addrspace(20)* ;; addrspace 20 is nonintegral
+%funcref = type i8 addrspace(20)* ;; addrspace 20 is nonintegral
 
 @funcref_global = local_unnamed_addr addrspace(1) global %funcref undef
 
 define void @set_funcref_global(%funcref %g) {
   ;; this generates a global.set of @funcref_global
   store %funcref %g, %funcref addrspace(1)* @funcref_global
   ret void
 }
 
 ; CHECK-LABEL: set_funcref_global:
 ; CHECK-NEXT: functype       set_funcref_global (funcref) -> ()
 ; CHECK-NEXT: local.get      0
 ; CHECK-NEXT: global.set     funcref_global
 ; CHECK-NEXT: end_function
 
 ; CHECK: .globl funcref_global

llvm/test/CodeGen/WebAssembly/funcref-table_call.ll

 ; RUN: llc < %s --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types | FileCheck %s
 
-%func = type void ()
-%funcref = type %func addrspace(20)* ;; addrspace 20 is nonintegral
+%funcptr = type void () addrspace(20)*
+%funcref = type i8 addrspace(20)* ;; addrspace 20 is nonintegral
 
 @funcref_table = local_unnamed_addr addrspace(1) global [0 x %funcref] undef
 
 define void @call_funcref_from_table(i32 %i) {
   %p = getelementptr [0 x %funcref], [0 x %funcref] addrspace (1)* @funcref_table, i32 0, i32 %i
   %ref = load %funcref, %funcref addrspace(1)* %p
-  call addrspace(20) void %ref()
+  %fn = bitcast %funcref %ref to %funcptr
+  call addrspace(20) void %fn()
   ret void
 }
 
 ; CHECK: .tabletype __funcref_call_table, funcref, 1
 
 ; CHECK-LABEL: call_funcref_from_table:
 ; CHECK-NEXT: .functype       call_funcref_from_table (i32) -> ()
 ; CHECK-NEXT: i32.const       0
 ; CHECK-NEXT: local.get       0
 ; CHECK-NEXT: table.get       funcref_table
 ; CHECK-NEXT: table.set       __funcref_call_table
 ; CHECK-NEXT: i32.const       0
 ; CHECK-NEXT: call_indirect    __funcref_call_table, () -> ()
 ; CHECK-NEXT: i32.const       0
-; CHECK-NEXT: ref.null        func
+; CHECK-NEXT: ref.null_func
 ; CHECK-NEXT: table.set       __funcref_call_table
 ; CHECK-NEXT: end_function
 
 
 ; CHECK:     .globl funcref_table
 ; CHECK-NEXT .globaltype     funcref_table, funcref

llvm/test/CodeGen/WebAssembly/funcref-tableget.ll

 ; RUN: llc < %s --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types | FileCheck %s
 
-%func = type void ()
-%funcref = type %func addrspace(20)* ;; addrspace 20 is nonintegral
+%funcref = type i8 addrspace(20)* ;; addrspace 20 is nonintegral
 
 @funcref_table = local_unnamed_addr addrspace(1) global [0 x %funcref] undef
 
 define %funcref @get_funcref_from_table(i32 %i) {
 ; CHECK-LABEL: get_funcref_from_table:
 ; CHECK-NEXT: .functype       get_funcref_from_table (i32) -> (funcref)
 ; CHECK-NEXT: local.get 0
 ; CHECK-NEXT: table.get funcref_table

llvm/test/CodeGen/WebAssembly/funcref-tableset.ll

 ; RUN: llc --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types < %s | FileCheck %s
 
-%func = type void ()
-%funcref = type %func addrspace(20)* ;; addrspace 20 is nonintegral
+%funcref = type i8 addrspace(20)* ;; addrspace 20 is nonintegral
 
 @funcref_table = local_unnamed_addr addrspace(1) global [0 x %funcref] undef
 
 define void @set_funcref_table(%funcref %g, i32 %i) {
 ; CHECK-LABEL: set_funcref_table:
 ; CHECK-NEXT: .functype       set_funcref_table (funcref, i32) -> ()
 ; CHECK-NEXT: local.get      1
 ; CHECK-NEXT: local.get      0

llvm/test/CodeGen/WebAssembly/ref-null.ll

+; RUN: llc --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types < %s | FileCheck %s
+
+%extern = type opaque
+%externref = type %extern addrspace(10)* ;; addrspace 10 is nonintegral
+%funcref = type i8 addrspace(20)*      ;; addrspace 20 is nonintegral
+
+declare %externref @llvm.wasm.ref.null.extern() nounwind
+declare %funcref @llvm.wasm.ref.null.func() nounwind
+
+define %externref @get_null_extern() {
+; CHECK-LABEL: get_null_extern:
+; CHECK-NEXT:  .functype       get_null_extern () -> (externref)
+; CHECK-NEXT:  ref.null_extern
+; CHECK-NEXT:  end_function
+  %null = call %externref @llvm.wasm.ref.null.extern()
+  ret %externref %null
+}
+
+define %funcref @get_null_func() {
+; CHECK-LABEL: get_null_func:
+; CHECK-NEXT:  .functype       get_null_func () -> (funcref)
+; CHECK-NEXT:  ref.null_func
+; CHECK-NEXT:  end_function
+  %null = call %funcref @llvm.wasm.ref.null.func()
+  ret %funcref %null
+}

llvm/test/CodeGen/WebAssembly/table-copy.ll

+; RUN: llc --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types < %s | FileCheck %s
+
+%extern = type opaque
+%externref = type %extern addrspace(10)* ;; addrspace 10 is nonintegral
+
+@externref_table1 = local_unnamed_addr addrspace(1) global [0 x %externref] undef
+@externref_table2 = local_unnamed_addr addrspace(1) global [0 x %externref] undef
+
+declare void @llvm.wasm.table.copy(i8 addrspace(1)*, i8 addrspace(1)*, i32, i32, i32) nounwind readonly
+
+define void @table_copy(i32 %dst, i32 %src, i32 %len) {
+; CHECK-LABEL: table_copy:
+; CHECK-NEXT:  .functype  table_copy (i32, i32, i32) -> ()
+; CHECK-NEXT:  local.get    0
+; CHECK-NEXT:  local.get    1
+; CHECK-NEXT:  local.get    2
+; CHECK-NEXT:  table.copy externref_table1, externref_table2
+; CHECK-NEXT:  end_function
+  %tableptr1 = getelementptr [0 x %externref], [0 x %externref] addrspace(1)* @externref_table1, i32 0, i32 0
+  %tb1 = bitcast %externref addrspace(1)* %tableptr1 to i8 addrspace(1)*
+  %tableptr2 = getelementptr [0 x %externref], [0 x %externref] addrspace(1)* @externref_table2, i32 0, i32 0
+  %tb2 = bitcast %externref addrspace(1)* %tableptr2 to i8 addrspace(1)*
+  call void @llvm.wasm.table.copy(i8 addrspace(1)* %tb1, i8 addrspace(1)* %tb2, i32 %dst, i32 %src, i32 %len)
+  ret void
+}

llvm/test/CodeGen/WebAssembly/table-fill.ll

+; RUN: llc --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types < %s | FileCheck %s
+
+%extern = type opaque
+%externref = type %extern addrspace(10)* ;; addrspace 10 is nonintegral
+
+@externref_table = local_unnamed_addr addrspace(1) global [0 x %externref] undef
+
+declare void @llvm.wasm.table.fill.externref(i8 addrspace(1)*, i32, %externref, i32) nounwind readonly
+
+define void @table_fill(i32 %start, i32 %len, %externref %val) {
+; CHECK-LABEL: table_fill:
+; CHECK-NEXT:  .functype  table_fill (i32, i32, externref) -> ()
+; CHECK-NEXT:  local.get    0
+; CHECK-NEXT:  local.get    2
+; CHECK-NEXT:  local.get    1
+; CHECK-NEXT:  table.fill externref_table
+; CHECK-NEXT:  end_function
+  %tableptr = getelementptr [0 x %externref], [0 x %externref] addrspace(1)* @externref_table, i32 0, i32 0
+  %tb = bitcast %externref addrspace(1)* %tableptr to i8 addrspace(1)*
+  call void @llvm.wasm.table.fill.externref(i8 addrspace(1)* %tb, i32 %start, %externref %val, i32 %len)
+  ret void
+}

llvm/test/CodeGen/WebAssembly/table-grow.ll

+; RUN: llc --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types < %s | FileCheck %s
+
+%extern = type opaque
+%externref = type %extern addrspace(10)* ;; addrspace 10 is nonintegral
+
+@externref_table = local_unnamed_addr addrspace(1) global [0 x %externref] undef
+
+declare i32 @llvm.wasm.table.grow.externref(i8 addrspace(1)*, %externref, i32) nounwind readonly
+declare %externref @llvm.wasm.ref.null.extern() nounwind readonly
+
+define i32 @table_grow(i32 %sz) {
+; CHECK-LABEL: table_grow:
+; CHECK-NEXT:  .functype  table_grow (i32) -> (i32)
+; CHECK-NEXT:  ref.null_extern
+; CHECK-NEXT:  local.get  0
+; CHECK-NEXT:  table.grow externref_table
+; CHECK-NEXT:  end_function
+  %null = call %externref @llvm.wasm.ref.null.extern()
+  %tableptr = getelementptr [0 x %externref], [0 x %externref] addrspace(1)* @externref_table, i32 0, i32 0
+  %tb = bitcast %externref addrspace(1)* %tableptr to i8 addrspace(1)*
+  %newsz = call i32 @llvm.wasm.table.grow.externref(i8 addrspace(1)* %tb, %externref %null, i32 %sz)
+  ret i32 %newsz
+}

llvm/test/CodeGen/WebAssembly/table-size.ll

+; RUN: llc --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types < %s | FileCheck %s
+
+%extern = type opaque
+%externref = type %extern addrspace(10)* ;; addrspace 10 is nonintegral
+
+@externref_table = local_unnamed_addr addrspace(1) global [0 x %externref] undef
+
+declare i32 @llvm.wasm.table.size(i8 addrspace(1)*) nounwind readonly
+
+define i32 @table_size() {
+; CHECK-LABEL: table_size:
+; CHECK-NEXT:  .functype       table_size () -> (i32)
+; CHECK-NEXT:  table.size      externref_table
+; CHECK-NEXT:  end_function
+  %tableptr = getelementptr [0 x %externref], [0 x %externref] addrspace(1)* @externref_table, i32 0, i32 0
+  %tb = bitcast %externref addrspace(1)* %tableptr to i8 addrspace(1)*
+  %sz = call i32 @llvm.wasm.table.size(i8 addrspace(1)* %tb)
+  ret i32 %sz
+}

llvm/test/MC/WebAssembly/reference-types.s

 # RUN: llvm-mc -show-encoding -triple=wasm32-unknown-unknown -mattr=+reference-types < %s | FileCheck %s
 # RUN: llvm-mc -show-encoding -triple=wasm64-unknown-unknown -mattr=+reference-types < %s | FileCheck %s
 
 # CHECK-LABEL: ref_null_test:
-# CHECK: ref.null func   # encoding: [0xd0,0x70]
-# CHECK: ref.null extern # encoding: [0xd0,0x6f]
+# CHECK: ref.null_func   # encoding: [0xd0,0x70]
+# CHECK: ref.null_extern # encoding: [0xd0,0x6f]
 ref_null_test:
   .functype ref_null_test () -> ()
-  ref.null func
+  ref.null_func
   drop
-  ref.null extern
+  ref.null_extern
   drop
   end_function
 
 # CHECK-LABEL: ref_sig_test_funcref:
 # CHECK-NEXT: .functype ref_sig_test_funcref (funcref) -> (funcref)
 ref_sig_test_funcref:
   .functype ref_sig_test_funcref (funcref) -> (funcref)
   local.get 0
   end_function
 
 # CHECK-LABEL: ref_sig_test_externref:
 # CHECK-NEXT: .functype ref_sig_test_externref (externref) -> (externref)
 ref_sig_test_externref:
   .functype ref_sig_test_externref (externref) -> (externref)
   local.get 0
   end_function
 
 # CHECK-LABEL: ref_select_test:
 # CHECK: funcref.select   # encoding: [0x1b]
 # CHECK: externref.select # encoding: [0x1b]
 ref_select_test:
   .functype ref_select_test () -> ()
-  ref.null func
-  ref.null func
+  ref.null_func
+  ref.null_func
   i32.const 0
   funcref.select
   drop
-  ref.null extern
-  ref.null extern
+  ref.null_extern
+  ref.null_extern
   i32.const 0
   externref.select
   drop
   end_function
 
 # CHECK-LABEL: ref_block_test:
 # CHECK: block funcref
 # CHECK: block externref
 ref_block_test:
   .functype ref_block_test () -> (externref, funcref)
   block funcref
   block externref
-  ref.null extern
+  ref.null_extern
   end_block
-  ref.null func
+  ref.null_func
   end_block
   end_function

llvm/utils/TableGen/IntrinsicEmitter.cpp

void IntrinsicEmitter::EmitIntrinsicToOverloadTable(
   OS << "\n};\n\n";
   // OTable contains a true bit at the position if the intrinsic is overloaded.
   OS << "return (OTable[id/8] & (1 << (id%8))) != 0;\n";
   OS << "#endif\n\n";
 }
 
 
 // NOTE: This must be kept in synch with the copy in lib/IR/Function.cpp!
 enum IIT_Info {
   // Common values should be encoded with 0-15.
   IIT_Done = 0,
-  IIT_I1   = 1,
-  IIT_I8   = 2,
-  IIT_I16  = 3,
-  IIT_I32  = 4,
-  IIT_I64  = 5,
-  IIT_F16  = 6,
-  IIT_F32  = 7,
-  IIT_F64  = 8,
-  IIT_V2   = 9,
-  IIT_V4   = 10,
-  IIT_V8   = 11,
-  IIT_V16  = 12,
-  IIT_V32  = 13,
-  IIT_PTR  = 14,
-  IIT_ARG  = 15,
+  IIT_I1 = 1,
+  IIT_I8 = 2,
+  IIT_I16 = 3,
+  IIT_I32 = 4,
+  IIT_I64 = 5,
+  IIT_F16 = 6,
+  IIT_F32 = 7,
+  IIT_F64 = 8,
+  IIT_V2 = 9,
+  IIT_V4 = 10,
+  IIT_V8 = 11,
+  IIT_V16 = 12,
+  IIT_V32 = 13,
+  IIT_PTR = 14,
+  IIT_ARG = 15,
 
   // Values from 16+ are only encodable with the inefficient encoding.
-  IIT_V64  = 16,
-  IIT_MMX  = 17,
+  IIT_V64 = 16,
+  IIT_MMX = 17,
   IIT_TOKEN = 18,
   IIT_METADATA = 19,
   IIT_EMPTYSTRUCT = 20,
   IIT_STRUCT2 = 21,
   IIT_STRUCT3 = 22,
   IIT_STRUCT4 = 23,
   IIT_STRUCT5 = 24,
   IIT_EXTEND_ARG = 25,
   IIT_TRUNC_ARG = 26,
   IIT_ANYPTR = 27,
-  IIT_V1   = 28,
+  IIT_V1 = 28,
   IIT_VARARG = 29,
   IIT_HALF_VEC_ARG = 30,
   IIT_SAME_VEC_WIDTH_ARG = 31,
   IIT_PTR_TO_ARG = 32,
   IIT_PTR_TO_ELT = 33,
   IIT_VEC_OF_ANYPTRS_TO_ELT = 34,
   IIT_I128 = 35,
   IIT_V512 = 36,
   IIT_V1024 = 37,
   IIT_STRUCT6 = 38,
   IIT_STRUCT7 = 39,
   IIT_STRUCT8 = 40,
   IIT_F128 = 41,
   IIT_VEC_ELEMENT = 42,
   IIT_SCALABLE_VEC = 43,
   IIT_SUBDIVIDE2_ARG = 44,
   IIT_SUBDIVIDE4_ARG = 45,
   IIT_VEC_OF_BITCASTS_TO_INT = 46,
   IIT_V128 = 47,
   IIT_BF16 = 48,
   IIT_STRUCT9 = 49,
   IIT_V256 = 50,
-  IIT_AMX  = 51,
-  IIT_PPCF128 = 52
+  IIT_AMX = 51,
+  IIT_PPCF128 = 52,
+  IIT_EXTERNREF = 53,
+  IIT_FUNCREF = 54
 };
 
 static void EncodeFixedValueType(MVT::SimpleValueType VT,
                                  std::vector<unsigned char> &Sig) {
   if (MVT(VT).isInteger()) {
     unsigned BitWidth = MVT(VT).getFixedSizeInBits();
     switch (BitWidth) {
     default: PrintFatalError("unhandled integer type width in intrinsic!");
static void EncodeFixedValueType(MVT::SimpleValueType VT,
   case MVT::token: return Sig.push_back(IIT_TOKEN);
   case MVT::Metadata: return Sig.push_back(IIT_METADATA);
   case MVT::x86mmx: return Sig.push_back(IIT_MMX);
   case MVT::x86amx: return Sig.push_back(IIT_AMX);
   // MVT::OtherVT is used to mean the empty struct type here.
   case MVT::Other: return Sig.push_back(IIT_EMPTYSTRUCT);
   // MVT::isVoid is used to represent varargs here.
   case MVT::isVoid: return Sig.push_back(IIT_VARARG);
+  case MVT::externref:
+    return Sig.push_back(IIT_EXTERNREF);
+  case MVT::funcref:
+    return Sig.push_back(IIT_FUNCREF);
   }
 }
 
 #if defined(_MSC_VER) && !defined(__clang__)
 #pragma optimize("",off) // MSVC 2015 optimizer can't deal with this function.
 #endif
 
 static void EncodeFixedType(Record *R, std::vector<unsigned char> &ArgCodes,