[WebAssembly] Implementation of global.get/set for reftypes in LLVM IR

[pmatos]

Summary

Reland of 31859f896.

This change implements new DAG notes GLOBAL_GET/GLOBAL_SET, and
lowering methods for load and stores of reference types from IR
globals. Once the lowering creates the new nodes, tablegen pattern
matches those and converts them to Wasm global.get/set.

[pmatos]

This patch would have fixed the problems with AArch64 caused by D95425. However, since that was landed and reverted, this landed: https://github.com/llvm/llvm-project/commit/ac81cb7e6dde9b0890ee1780eae94ab96743569b

This breaks the test llvm/test/CodeGen/WebAssembly/externref-ptrtoint.ll which was expected to fail. I still think that a ptrtoint on an externref value should fail, and yet since ac81cb7e allows ptrtoint on non-integral pointer types, the verifier is letting this test to go through and it crashes LLVM later on during some DAG node analysis.

@tlively Do you think it would be ok to re-add the code removed in ac81cb7e but only error if the pointer is to an opaque non-integral type?

[pmatos]

Adding @reames to subscribers since he's the author of https://github.com/llvm/llvm-project/commit/ac81cb7e6dde9b0890ee1780eae94ab96743569b and might have something to add to the discussion.

[tlively]

In D104797#2836475, @pmatos wrote:

@tlively Do you think it would be ok to re-add the code removed in ac81cb7e but only error if the pointer is to an opaque non-integral type?

Unfortunately that wouldn't be future-proof given the ongoing project to remove type information from pointers. I think the best we can do right now is call report_fatal_error from the backend whenever a reference type is the operand or result of ptrtoint or inttoptr.

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.h

69

It might be worth a comment explaining why the memtype is also externref and funcref. Is this just for lack of a more meaningful type to return?

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

124–125

hasReferenceTypes should also be taking the CPU into account, not just the feature string. Normally this would be done via getSubtargetImpl, but I guess we probably can't call that this early in the construction of the WebAssemblyTargetMachine. Would anything break if we just unconditionally added the reference types address spaces to the data layout string?

[pmatos]

In D104797#2837417, @tlively wrote:

In D104797#2836475, @pmatos wrote:

@tlively Do you think it would be ok to re-add the code removed in ac81cb7e but only error if the pointer is to an opaque non-integral type?

Unfortunately that wouldn't be future-proof given the ongoing project to remove type information from pointers. I think the best we can do right now is call report_fatal_error from the backend whenever a reference type is the operand or result of ptrtoint or inttoptr.

Which 'ongoing project' is this and how can I obtain more information about it? Who's leading it?
Before the crashing, the only place we seem to touch the Wasm backend is in WebAssemblyDAGToDAGISel::runOnMachineFunction. Then this calls SelectionDAGISel::runOnMachineFunction, which visits each node in the Function and crashes when trying to call getPtrExtOrTrunc because it cannot create a TRUNCATE node for a zero sized type. So, if you prefer to have a report_fatal_error in the WebAssembly backend, we will need to visit the nodes in the function in WebAssemblyDAGToDAGISel::runOnMachineFunction before calling the generic runOnMachineFunction. I will look at how to proceed in this direction, but I am also interested in following this project on removing type information from pointers in LLVM, so any refs would be great. Thanks.

[tlively]

The opaque pointers project is documented here: https://llvm.org/docs/OpaquePointers.html. It's been making very slow progress for the past few years but has recently been picking up steam under the direction of @aeubanks. Search llvm-dev for "opaque pointers" and you will see a few recent threads about it.

[pmatos]

In D104797#2838653, @tlively wrote:

The opaque pointers project is documented here: https://llvm.org/docs/OpaquePointers.html. It's been making very slow progress for the past few years but has recently been picking up steam under the direction of @aeubanks. Search llvm-dev for "opaque pointers" and you will see a few recent threads about it.

Thanks for this.

[pmatos]

Checks for inttoptr and ptrtoint insts in wasm backend and bails out if necessary.

Also addresses @tlively's comments to the original patch.

[pmatos]

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

124–125

Regarding this change, I don't quite understand why referencetypes should take the CPU into account. Are there CPU variants for the wasm backend? I haven't touched the conditional because that would mean touching the several tests that don't enable reference types and use the old data layout string. However, I would think that if that's the path we want to follow here, we could do it and change all wasm tests to use the layout string with reference types.

[sbc100]

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1138

I was expecting to see a TABLE_SET back to a null after the call here to avoid the GC root being leaked in table slot 0 (as above). I must be missing something?

[tlively]

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

124–125

Yes, there are CPU variants defined here: https://github.com/llvm/llvm-project/blob/7ac0442fe59dbe0f9127e79e8786a7dd6345c537/llvm/lib/Target/WebAssembly/WebAssembly.td#L89-L100. Note that the CPU may enable reference types even if the feature string does not. If it doesn't break anything, then unconditionally updating the layout string sounds like the best option.

[pmatos]

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1138

@sbc100, that's what's happening in line 583.

[pmatos]

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

124–125

Interesting - had not come accross it. Bleeding edge does not seem to include reference-types. What's the reason for this?

[tlively]

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

124–125

We don't have a well-defined process for adding features to bleeding-edge, but I think typically they're added once they're mostly stable and usable in the tools.

[pmatos]

Unconditionally set Wasm layout string to include non-integral AS 10 and 20.

[pmatos]

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

124–125

@tlively I have now unconditionally updated the layout string. No failures. How does this look like now?

[pmatos]

Remove FIXME comment on data layout strings

[pmatos]

Make the linter happy

[pmatos]

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.h

69

As far as I understand that's the type in memory anyway, so it sounds like the right type to return in getPointerMemTy. I would think that if this question was for getPointerTy then the answer would be yes.

[tlively]

Thanks!

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

124–125

Looks good, thanks!

[lebedev.ri]

Reverted again

********************
FAIL: LLVM :: CodeGen/WebAssembly/funcref-call.ll (44466 of 44468)
******************** TEST 'LLVM :: CodeGen/WebAssembly/funcref-call.ll' FAILED ********************
Script:
--
: 'RUN: at line 1';   /builddirs/llvm-project/build-Clang12/bin/llc < /repositories/llvm-project/llvm/test/CodeGen/WebAssembly/funcref-call.ll --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types | /builddirs/llvm-project/build-Clang12/bin/FileCheck /repositories/llvm-project/llvm/test/CodeGen/WebAssembly/funcref-call.ll
--
Exit Code: 2

Command Output (stderr):
--
llc: /repositories/llvm-project/llvm/include/llvm/Support/LowLevelTypeImpl.h:44: static llvm::LLT llvm::LLT::scalar(unsigned int): Assertion `SizeInBits > 0 && "invalid scalar size"' failed.

Are you building llvm without assertions?

[pmatos]

In D104797#2854939, @lebedev.ri wrote:

Reverted again

********************
FAIL: LLVM :: CodeGen/WebAssembly/funcref-call.ll (44466 of 44468)
******************** TEST 'LLVM :: CodeGen/WebAssembly/funcref-call.ll' FAILED ********************
Script:
--
: 'RUN: at line 1';   /builddirs/llvm-project/build-Clang12/bin/llc < /repositories/llvm-project/llvm/test/CodeGen/WebAssembly/funcref-call.ll --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types | /builddirs/llvm-project/build-Clang12/bin/FileCheck /repositories/llvm-project/llvm/test/CodeGen/WebAssembly/funcref-call.ll
--
Exit Code: 2

Command Output (stderr):
--
llc: /repositories/llvm-project/llvm/include/llvm/Support/LowLevelTypeImpl.h:44: static llvm::LLT llvm::LLT::scalar(unsigned int): Assertion `SizeInBits > 0 && "invalid scalar size"' failed.

Are you building llvm without assertions?

I am sorry - thanks for reverting.
Last time I tested with check-all, nothing was failing. Then I updated to main to commit but because check-all takes a while and I didn't want to miss the boat to push, I pushed the commit.
However, the right procedure would have been probably to maybe run a quick llvm-lit on the WebAssembly tests. I didn't and between the main commit my patch was based on and current main HEAD a commit broke my patch again, specifically in sha 990278d026 (June 29).

I am now analysing this patch to reland, so I can fix mine once again and reland.

[pmatos]

After @arsenm 's commit https://github.com/llvm/llvm-project/commit/990278d026d680942c859be70836ad34a9a716f7, MachineOperands store LLT for size instead of uint64_t and the constructor called from SelectionDAG::getLoad, calls MachineMemOperand which does:

: MachineMemOperand(ptrinfo, f,
                    s == ~UINT64_C(0) ? LLT() : LLT::scalar(8 * s), a,
                    AAInfo, Ranges, SSID, Ordering, FailureOrdering) {}

Because s is zero for reference types, LLT::scalar(0) is called, which asserts because scalar LLTs cannot have zero size. Why do we have the check s == ~UINT64_C(0), instead of ss == UINT64_C(0) given opaque types and therefore reference types are zero sized?

[pmatos]

In D104797#2855322, @pmatos wrote:

After @arsenm 's commit https://github.com/llvm/llvm-project/commit/990278d026d680942c859be70836ad34a9a716f7, MachineOperands store LLT for size instead of uint64_t and the constructor called from SelectionDAG::getLoad, calls MachineMemOperand which does:

: MachineMemOperand(ptrinfo, f,
                    s == ~UINT64_C(0) ? LLT() : LLT::scalar(8 * s), a,
                    AAInfo, Ranges, SSID, Ordering, FailureOrdering) {}

Because s is zero for reference types, LLT::scalar(0) is called, which asserts because scalar LLTs cannot have zero size. Why do we have the check s == ~UINT64_C(0), instead of ss == UINT64_C(0) given opaque types and therefore reference types are zero sized?

OK, sort of replying to myself ~UINT64_T(0) is the size for invalid types. I think at this point we need LLT support for opaque types. Maybe we couldn't have opaque types stored in memory but we now can with reference types in WebAssembly.

[pmatos]

D105423 implements the required support in LLT of opaque types for this patch to work again.

[pmatos]

Remove some extraneous code

[pmatos]

In D104797#2864102, @pmatos wrote:

Remove some extraneous code

Argh, sorry. This was meant for another revision and I added code that's not meant for this revision.

[pmatos]

Revert code incorrectly submitted as part of diff

[pmatos]

@tlively once D105423 lands, is it enough to test and reland it under this revision or shall i open a new one?

[pmatos]

Remove hunks touching LLTs accidentally added before

[tlively]

In D104797#2879855, @pmatos wrote:

@tlively once D105423 lands, is it enough to test and reland it under this revision or shall i open a new one?

You can just reland it under this revision.

[pmatos]

Rebase on top of main and add non-integral address spaces to emscripten OS

clang/lib/Basic/Targets/WebAssembly.h

 };
 
 class LLVM_LIBRARY_VISIBILITY WebAssembly32TargetInfo
     : public WebAssemblyTargetInfo {
 public:
   explicit WebAssembly32TargetInfo(const llvm::Triple &T,
                                    const TargetOptions &Opts)
       : WebAssemblyTargetInfo(T, Opts) {
-    resetDataLayout("e-m:e-p:32:32-i64:64-n32:64-S128-ni:1");
+    resetDataLayout("e-m:e-p:32:32-i64:64-n32:64-S128-ni:1:10:20");
   }
 
 protected:
   void getTargetDefines(const LangOptions &Opts,
                         MacroBuilder &Builder) const override;
 };
 
 class LLVM_LIBRARY_VISIBILITY WebAssembly64TargetInfo
     : public WebAssemblyTargetInfo {
 public:
   explicit WebAssembly64TargetInfo(const llvm::Triple &T,
                                    const TargetOptions &Opts)
       : WebAssemblyTargetInfo(T, Opts) {
     LongAlign = LongWidth = 64;
     PointerAlign = PointerWidth = 64;
     SizeType = UnsignedLong;
     PtrDiffType = SignedLong;
     IntPtrType = SignedLong;
-    resetDataLayout("e-m:e-p:64:64-i64:64-n32:64-S128-ni:1");
+    resetDataLayout("e-m:e-p:64:64-i64:64-n32:64-S128-ni:1:10:20");
   }
 
 protected:
   void getTargetDefines(const LangOptions &Opts,
                         MacroBuilder &Builder) const override;
 };
 } // namespace targets
 } // namespace clang
 #endif // LLVM_CLANG_LIB_BASIC_TARGETS_WEBASSEMBLY_H

clang/test/CodeGen/target-data.c

 // ARM-NACL: target datalayout = "e-m:e-p:32:32-Fi8-i64:64-v128:64:128-a:0:32-n32-S128"
 
 // RUN: %clang_cc1 -triple mipsel-nacl -o - -emit-llvm %s | \
 // RUN: FileCheck %s -check-prefix=MIPS-NACL
 // MIPS-NACL: target datalayout = "e-m:m-p:32:32-i8:8:32-i16:16:32-i64:64-n32-S64"
 
 // RUN: %clang_cc1 -triple wasm32-unknown-unknown -o - -emit-llvm %s | \
 // RUN: FileCheck %s -check-prefix=WEBASSEMBLY32
-// WEBASSEMBLY32: target datalayout = "e-m:e-p:32:32-i64:64-n32:64-S128-ni:1"
+// WEBASSEMBLY32: target datalayout = "e-m:e-p:32:32-i64:64-n32:64-S128-ni:1:10:20"
 
 // RUN: %clang_cc1 -triple wasm64-unknown-unknown -o - -emit-llvm %s | \
 // RUN: FileCheck %s -check-prefix=WEBASSEMBLY64
-// WEBASSEMBLY64: target datalayout = "e-m:e-p:64:64-i64:64-n32:64-S128-ni:1"
+// WEBASSEMBLY64: target datalayout = "e-m:e-p:64:64-i64:64-n32:64-S128-ni:1:10:20"
 
 // RUN: %clang_cc1 -triple lanai-unknown-unknown -o - -emit-llvm %s | \
 // RUN: FileCheck %s -check-prefix=LANAI
 // LANAI: target datalayout = "E-m:e-p:32:32-i64:64-a:0:32-n32-S64"
 
 // RUN: %clang_cc1 -triple powerpc-unknown -o - -emit-llvm %s | \
 // RUN: FileCheck %s -check-prefix=PPC
 // PPC: target datalayout = "E-m:e-p:32:32-i64:64-n32"

llvm/include/llvm/CodeGen/TargetLowering.h

public:
   /// FIXME: The default needs to be removed once all the code is updated.
   virtual MVT getPointerTy(const DataLayout &DL, uint32_t AS = 0) const {
     return MVT::getIntegerVT(DL.getPointerSizeInBits(AS));
   }
 
   /// Return the in-memory pointer type for the given address space, defaults to
   /// the pointer type from the data layout.  FIXME: The default needs to be
   /// removed once all the code is updated.
-  MVT getPointerMemTy(const DataLayout &DL, uint32_t AS = 0) const {
+  virtual MVT getPointerMemTy(const DataLayout &DL, uint32_t AS = 0) const {
     return MVT::getIntegerVT(DL.getPointerSizeInBits(AS));
   }
 
   /// Return the type for frame index, which is determined by
   /// the alloca address space specified through the data layout.
   MVT getFrameIndexTy(const DataLayout &DL) const {
     return getPointerTy(DL, DL.getAllocaAddrSpace());
   }

llvm/include/llvm/CodeGen/ValueTypes.h

EVT changeTypeToInteger() {
       if (isVector())
         return changeVectorElementTypeToInteger();
 
       if (isSimple())
         return getSimpleVT().changeTypeToInteger();
       return changeExtendedTypeToInteger();
     }
 
+    /// Test if the given EVT has zero size, this will fail if called on a
+    /// scalable type
+    bool isZeroSized() const {
+      return !isScalableVector() && getSizeInBits() == 0;
+    }
+
     /// Test if the given EVT is simple (as opposed to being extended).
     bool isSimple() const {
       return V.SimpleTy != MVT::INVALID_SIMPLE_VALUE_TYPE;
     }
 
     /// Test if the given EVT is extended (as opposed to being simple).
     bool isExtended() const {
       return !isSimple();
public:
     }
 
     /// Return true if this is an overloaded type for TableGen.
     bool isOverloaded() const {
       return (V==MVT::iAny || V==MVT::fAny || V==MVT::vAny || V==MVT::iPTRAny);
     }
 
     /// Return true if the bit size is a multiple of 8.
-    bool isByteSized() const { return getSizeInBits().isKnownMultipleOf(8); }
+    bool isByteSized() const {
+      return !isZeroSized() && getSizeInBits().isKnownMultipleOf(8);
+    }
 
     /// Return true if the size is a power-of-two number of bytes.
     bool isRound() const {
       if (isScalableVector())
         return false;
       unsigned BitSize = getSizeInBits();
       return BitSize >= 8 && !(BitSize & (BitSize - 1));
     }

llvm/lib/CodeGen/CodeGenPrepare.cpp

bool CodeGenPrepare::optimizeLoadExt(LoadInst *Load) {
   SmallVector<Instruction *, 8> WorkList;
   SmallPtrSet<Instruction *, 16> Visited;
   SmallVector<Instruction *, 8> AndsToMaybeRemove;
   for (auto *U : Load->users())
     WorkList.push_back(cast<Instruction>(U));
 
   EVT LoadResultVT = TLI->getValueType(*DL, Load->getType());
   unsigned BitWidth = LoadResultVT.getSizeInBits();
+  // If the BitWidth is 0, do not try to optimize the type
+  if (BitWidth == 0)
+    return false;
+
   APInt DemandBits(BitWidth, 0);
   APInt WidestAndBits(BitWidth, 0);
 
   while (!WorkList.empty()) {
     Instruction *I = WorkList.back();
     WorkList.pop_back();
 
     // Break use-def graph loops.

llvm/lib/CodeGen/MachineOperand.cpp

if (const Value *Val = getValue()) {
     }
   } else if (getOpaqueValue() == nullptr && getOffset() != 0) {
     OS << ((isLoad() && isStore()) ? " on "
            : isLoad()              ? " from "
                                    : " into ")
        << "unknown-address";
   }
   MachineOperand::printOperandOffset(OS, getOffset());
-  if (getAlign() != getSize())
+  if (getSize() > 0 && getAlign() != getSize())
     OS << ", align " << getAlign().value();
   if (getAlign() != getBaseAlign())
     OS << ", basealign " << getBaseAlign().value();
   auto AAInfo = getAAInfo();
   if (AAInfo.TBAA) {
     OS << ", !tbaa ";
     AAInfo.TBAA->printAsOperand(OS, MST);
   }

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

bool DAGCombiner::parallelizeChainedStores(StoreSDNode *St) {
   // We must have a base and an offset.
   if (!BasePtr.getBase().getNode())
     return false;
 
   // Do not handle stores to undef base pointers.
   if (BasePtr.getBase().isUndef())
     return false;
 
+  // Do not handle stores to opaque types
+  if (St->getMemoryVT().isZeroSized())
+    return false;
+
   // BaseIndexOffset assumes that offsets are fixed-size, which
   // is not valid for scalable vectors where the offsets are
   // scaled by `vscale`, so bail out early.
   if (St->getMemoryVT().isScalableVector())
     return false;
 
   // Add ST's interval.
   Intervals.insert(0, (St->getMemoryVT().getSizeInBits() + 7) / 8, Unit);

llvm/lib/CodeGen/TargetLoweringBase.cpp

bool TargetLoweringBase::allowsMemoryAccessForAlignment(
     LLVMContext &Context, const DataLayout &DL, EVT VT, unsigned AddrSpace,
     Align Alignment, MachineMemOperand::Flags Flags, bool *Fast) const {
   // Check if the specified alignment is sufficient based on the data layout.
   // TODO: While using the data layout works in practice, a better solution
   // would be to implement this check directly (make this a virtual function).
   // For example, the ABI alignment may change based on software platform while
   // this function should only be affected by hardware implementation.
   Type *Ty = VT.getTypeForEVT(Context);
-  if (Alignment >= DL.getABITypeAlign(Ty)) {
+  if (VT.isZeroSized() || Alignment >= DL.getABITypeAlign(Ty)) {
     // Assume that an access that meets the ABI-specified alignment is fast.
     if (Fast != nullptr)
       *Fast = true;
     return true;
   }
 
   // This is a misaligned access.
   return allowsMisalignedMemoryAccesses(VT, AddrSpace, Alignment, Flags, Fast);

llvm/lib/CodeGen/ValueTypes.cpp

Type *EVT::getTypeForEVT(LLVMContext &Context) const {
   case MVT::bf16:     return Type::getBFloatTy(Context);
   case MVT::f32:     return Type::getFloatTy(Context);
   case MVT::f64:     return Type::getDoubleTy(Context);
   case MVT::f80:     return Type::getX86_FP80Ty(Context);
   case MVT::f128:    return Type::getFP128Ty(Context);
   case MVT::ppcf128: return Type::getPPC_FP128Ty(Context);
   case MVT::x86mmx:  return Type::getX86_MMXTy(Context);
   case MVT::x86amx:  return Type::getX86_AMXTy(Context);
+  case MVT::externref:
+    return PointerType::get(StructType::create(Context), 10);
+  case MVT::funcref:
+    return PointerType::get(StructType::create(Context), 20);
   case MVT::v1i1:
     return FixedVectorType::get(Type::getInt1Ty(Context), 1);
   case MVT::v2i1:
     return FixedVectorType::get(Type::getInt1Ty(Context), 2);
   case MVT::v4i1:
     return FixedVectorType::get(Type::getInt1Ty(Context), 4);
   case MVT::v8i1:
     return FixedVectorType::get(Type::getInt1Ty(Context), 8);

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.h

 const MachineOperand &getCalleeOp(const MachineInstr &MI);
 
 /// Returns the __indirect_function_table, for use in call_indirect and in
 /// function bitcasts.
 MCSymbolWasm *
 getOrCreateFunctionTableSymbol(MCContext &Ctx,
                                const WebAssemblySubtarget *Subtarget);
 
+/// Returns the __funcref_call_table, for use in funcref calls when lowered to
+/// table.set + call_indirect.
+MCSymbolWasm *
+getOrCreateFuncrefCallTableSymbol(MCContext &Ctx,
+                                  const WebAssemblySubtarget *Subtarget);
+
 /// Find a catch instruction from an EH pad. Returns null if no catch
 /// instruction found or the catch is in an invalid location.
 MachineInstr *findCatch(MachineBasicBlock *EHPad);
 
 } // end namespace WebAssembly
 
 } // end namespace llvm
 
 #endif

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.cpp

if (Sym) {
     Sym->setUndefined();
   }
   // MVP object files can't have symtab entries for tables.
   if (!(Subtarget && Subtarget->hasReferenceTypes()))
     Sym->setOmitFromLinkingSection();
   return Sym;
 }
 
+MCSymbolWasm *WebAssembly::getOrCreateFuncrefCallTableSymbol(
+    MCContext &Ctx, const WebAssemblySubtarget *Subtarget) {
+  StringRef Name = "__funcref_call_table";
+  MCSymbolWasm *Sym = cast_or_null<MCSymbolWasm>(Ctx.lookupSymbol(Name));
+  if (Sym) {
+    if (!Sym->isFunctionTable())
+      Ctx.reportError(SMLoc(), "symbol is not a wasm funcref table");
+  } else {
+    Sym = cast<MCSymbolWasm>(Ctx.getOrCreateSymbol(Name));
+
+    // Setting Weak ensure only one table is left after linking when multiple
+    // modules define the table.
+    Sym->setWeak(true);
+
+    wasm::WasmLimits Limits = {0, 1, 1};
+    wasm::WasmTableType TableType = {wasm::WASM_TYPE_FUNCREF, Limits};
+    Sym->setType(wasm::WASM_SYMBOL_TYPE_TABLE);
+    Sym->setTableType(TableType);
+  }
+  // MVP object files can't have symtab entries for tables.
+  if (!(Subtarget && Subtarget->hasReferenceTypes()))
+    Sym->setOmitFromLinkingSection();
+  return Sym;
+}
+
 // Find a catch instruction from an EH pad.
 MachineInstr *WebAssembly::findCatch(MachineBasicBlock *EHPad) {
   assert(EHPad->isEHPad());
   auto Pos = EHPad->begin();
   // Skip any label or debug instructions. Also skip 'end' marker instructions
   // that may exist after marker placement in CFGStackify.
   while (Pos != EHPad->end() &&
          (Pos->isLabel() || Pos->isDebugInstr() || isMarker(Pos->getOpcode())))
     Pos++;
   if (Pos != EHPad->end() && WebAssembly::isCatch(Pos->getOpcode()))
     return &*Pos;
   return nullptr;
 }

llvm/lib/Target/WebAssembly/WebAssemblyFastISel.cpp

MVT::SimpleValueType getLegalType(MVT::SimpleValueType VT) {
     case MVT::i1:
     case MVT::i8:
     case MVT::i16:
       return MVT::i32;
     case MVT::i32:
     case MVT::i64:
     case MVT::f32:
     case MVT::f64:
+      return VT;
     case MVT::funcref:
     case MVT::externref:
+      if (Subtarget->hasReferenceTypes())
         return VT;
+      break;
     case MVT::f16:
       return MVT::f32;
     case MVT::v16i8:
     case MVT::v8i16:
     case MVT::v4i32:
     case MVT::v4f32:
     case MVT::v2i64:
     case MVT::v2f64:

llvm/lib/Target/WebAssembly/WebAssemblyISD.def

 HANDLE_NODETYPE(CATCH)
 HANDLE_NODETYPE(MEMORY_COPY)
 HANDLE_NODETYPE(MEMORY_FILL)
 
 // Memory intrinsics
 HANDLE_MEM_NODETYPE(LOAD_SPLAT)
 HANDLE_MEM_NODETYPE(GLOBAL_GET)
 HANDLE_MEM_NODETYPE(GLOBAL_SET)
+HANDLE_MEM_NODETYPE(TABLE_SET)

llvm/lib/Target/WebAssembly/WebAssemblyISelDAGToDAG.cpp

 //- WebAssemblyISelDAGToDAG.cpp - A dag to dag inst selector for WebAssembly -//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// This file defines an instruction selector for the WebAssembly target.
 ///
 //===----------------------------------------------------------------------===//
 
 #include "MCTargetDesc/WebAssemblyMCTargetDesc.h"
 #include "WebAssembly.h"
+#include "WebAssemblyISelLowering.h"
 #include "WebAssemblyTargetMachine.h"
 #include "llvm/CodeGen/MachineFrameInfo.h"
 #include "llvm/CodeGen/SelectionDAGISel.h"
 #include "llvm/IR/DiagnosticInfo.h"
 #include "llvm/IR/Function.h" // To access function attributes.
 #include "llvm/IR/IntrinsicsWebAssembly.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/KnownBits.h"
WebAssemblyDAGToDAGISel(WebAssemblyTargetMachine &TM,
                           CodeGenOpt::Level OptLevel)
       : SelectionDAGISel(TM, OptLevel), Subtarget(nullptr) {
   }
 
   StringRef getPassName() const override {
     return "WebAssembly Instruction Selection";
   }
 
+  void checkForInvalidNodes(const Function &F) {
+    // This function will check for uses of ptrtoint on reference types and
+    // report a fatal error if these are found.
+    for (const BasicBlock &BB : F) {
+      for (const Instruction &I : BB) {
+        if (const PtrToIntInst *PTI = dyn_cast<const PtrToIntInst>(&I)) {
+          const Value *V = PTI->getPointerOperand();
+          if (WebAssemblyTargetLowering::isFuncrefType(V->getType()) ||
+              WebAssemblyTargetLowering::isExternrefType(V->getType()))
+            report_fatal_error("ptrtoint not allowed on reference types");
+        } else if (const IntToPtrInst *ITP = dyn_cast<const IntToPtrInst>(&I)) {
+          if (WebAssemblyTargetLowering::isFuncrefType(ITP->getDestTy()) ||
+              WebAssemblyTargetLowering::isExternrefType(ITP->getDestTy()))
+            report_fatal_error("inttoptr not allowed on reference types");
+        }
+      }
+    }
+  }
+
   bool runOnMachineFunction(MachineFunction &MF) override {
     LLVM_DEBUG(dbgs() << "********** ISelDAGToDAG **********\n"
                          "********** Function: "
                       << MF.getName() << '\n');
 
+    checkForInvalidNodes(MF.getFunction());
+
     Subtarget = &MF.getSubtarget<WebAssemblySubtarget>();
 
     return SelectionDAGISel::runOnMachineFunction(MF);
   }
 
   void PreprocessISelDAG() override;
 
   void Select(SDNode *Node) override;
 
   bool SelectInlineAsmMemoryOperand(const SDValue &Op, unsigned ConstraintID,
                                     std::vector<SDValue> &OutOps) override;
+  bool SelectExternRefAddr(const SDValue &Addr, const SDValue &Base);
 
 // Include the pieces autogenerated from the target description.
 #include "WebAssemblyGenDAGISel.inc"
 
 private:
   // add select functions here...
 };
 } // end anonymous namespace

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.h

 
 class WebAssemblySubtarget;
 
 class WebAssemblyTargetLowering final : public TargetLowering {
 public:
   WebAssemblyTargetLowering(const TargetMachine &TM,
                             const WebAssemblySubtarget &STI);
 
+  enum WasmAddressSpace : unsigned {
+    // WebAssembly uses the following address spaces:
+    // AS 0 : is the default address space for values in linear memory
+    DEFAULT = 0,
+    // AS 1 : is a non-integral address space for global variables
+    GLOBAL = 1,
+    // AS 10 : is a non-integral address space for externref values
+    EXTERNREF = 10,
+    // AS 20 : is a non-integral address space for funcref values
+    FUNCREF = 20,
+  };
+
+  MVT getPointerTy(const DataLayout &DL, uint32_t AS = 0) const override {
+    if (AS == WasmAddressSpace::EXTERNREF)
+      return MVT::externref;
+    if (AS == WasmAddressSpace::FUNCREF)
+      return MVT::funcref;
+    return TargetLowering::getPointerTy(DL, AS);
+  }
+  MVT getPointerMemTy(const DataLayout &DL, uint32_t AS = 0) const override {
+    if (AS == WasmAddressSpace::EXTERNREF)
+      return MVT::externref;
+    if (AS == WasmAddressSpace::FUNCREF)
+      return MVT::funcref;
+    return TargetLowering::getPointerMemTy(DL, AS);
+  }
+
+  static bool isFuncrefType(const Type *Ty);
+  static bool isExternrefType(const Type *Ty);
+
 private:
   /// Keep a pointer to the WebAssemblySubtarget around so that we can make the
   /// right decision when generating code for different targets.
   const WebAssemblySubtarget *Subtarget;
 
   AtomicExpansionKind shouldExpandAtomicRMWInIR(AtomicRMWInst *) const override;
   FastISel *createFastISel(FunctionLoweringInfo &FuncInfo,
                            const TargetLibraryInfo *LibInfo) const override;

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

 //=- WebAssemblyISelLowering.cpp - WebAssembly DAG Lowering Implementation -==//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// This file implements the WebAssemblyTargetLowering class.
 ///
 //===----------------------------------------------------------------------===//
 
 #include "WebAssemblyISelLowering.h"
 #include "MCTargetDesc/WebAssemblyMCTargetDesc.h"
+#include "Utils/WebAssemblyTypeUtilities.h"
 #include "Utils/WebAssemblyUtilities.h"
 #include "WebAssemblyMachineFunctionInfo.h"
 #include "WebAssemblySubtarget.h"
 #include "WebAssemblyTargetMachine.h"
 #include "llvm/CodeGen/CallingConvLower.h"
 #include "llvm/CodeGen/MachineInstrBuilder.h"
 #include "llvm/CodeGen/MachineJumpTableInfo.h"
 #include "llvm/CodeGen/MachineModuleInfo.h"
WebAssemblyTargetLowering::WebAssemblyTargetLowering(
   if (Subtarget->hasSIMD128()) {
     addRegisterClass(MVT::v16i8, &WebAssembly::V128RegClass);
     addRegisterClass(MVT::v8i16, &WebAssembly::V128RegClass);
     addRegisterClass(MVT::v4i32, &WebAssembly::V128RegClass);
     addRegisterClass(MVT::v4f32, &WebAssembly::V128RegClass);
     addRegisterClass(MVT::v2i64, &WebAssembly::V128RegClass);
     addRegisterClass(MVT::v2f64, &WebAssembly::V128RegClass);
   }
+  if (Subtarget->hasReferenceTypes()) {
+    addRegisterClass(MVT::externref, &WebAssembly::EXTERNREFRegClass);
+    addRegisterClass(MVT::funcref, &WebAssembly::FUNCREFRegClass);
+  }
   // Compute derived properties from the register classes.
   computeRegisterProperties(Subtarget->getRegisterInfo());
 
   // Transform loads and stores to pointers in address space 1 to loads and
   // stores to WebAssembly global variables, outside linear memory.
   for (auto T : {MVT::i32, MVT::i64, MVT::f32, MVT::f64}) {
     setOperationAction(ISD::LOAD, T, Custom);
     setOperationAction(ISD::STORE, T, Custom);
   }
   if (Subtarget->hasSIMD128()) {
     for (auto T : {MVT::v16i8, MVT::v8i16, MVT::v4i32, MVT::v4f32, MVT::v2i64,
                    MVT::v2f64}) {
       setOperationAction(ISD::LOAD, T, Custom);
       setOperationAction(ISD::STORE, T, Custom);
     }
   }
+  if (Subtarget->hasReferenceTypes()) {
+    for (auto T : {MVT::externref, MVT::funcref}) {
+      setOperationAction(ISD::LOAD, T, Custom);
+      setOperationAction(ISD::STORE, T, Custom);
+    }
+  }
 
   setOperationAction(ISD::GlobalAddress, MVTPtr, Custom);
   setOperationAction(ISD::GlobalTLSAddress, MVTPtr, Custom);
   setOperationAction(ISD::ExternalSymbol, MVTPtr, Custom);
   setOperationAction(ISD::JumpTable, MVTPtr, Custom);
   setOperationAction(ISD::BlockAddress, MVTPtr, Custom);
   setOperationAction(ISD::BRIND, MVT::Other, Custom);
 
LowerCallResults(MachineInstr &CallResults, DebugLoc DL, MachineBasicBlock *BB,
   MachineInstr &CallParams = *CallResults.getPrevNode();
   assert(CallParams.getOpcode() == WebAssembly::CALL_PARAMS);
   assert(CallResults.getOpcode() == WebAssembly::CALL_RESULTS ||
          CallResults.getOpcode() == WebAssembly::RET_CALL_RESULTS);
 
   bool IsIndirect = CallParams.getOperand(0).isReg();
   bool IsRetCall = CallResults.getOpcode() == WebAssembly::RET_CALL_RESULTS;
 
+  bool IsFuncrefCall = false;
+  if (IsIndirect) {
+    Register Reg = CallParams.getOperand(0).getReg();
+    const MachineFunction *MF = BB->getParent();
+    const MachineRegisterInfo &MRI = MF->getRegInfo();
+    const TargetRegisterClass *TRC = MRI.getRegClass(Reg);
+    IsFuncrefCall = (TRC == &WebAssembly::FUNCREFRegClass);
+    assert(!IsFuncrefCall || Subtarget->hasReferenceTypes());
+  }
+
   unsigned CallOp;
   if (IsIndirect && IsRetCall) {
     CallOp = WebAssembly::RET_CALL_INDIRECT;
   } else if (IsIndirect) {
     CallOp = WebAssembly::CALL_INDIRECT;
   } else if (IsRetCall) {
     CallOp = WebAssembly::RET_CALL;
   } else {
LowerCallResults(MachineInstr &CallResults, DebugLoc DL, MachineBasicBlock *BB,
 
   for (auto Def : CallResults.defs())
     MIB.add(Def);
 
   if (IsIndirect) {
     // Placeholder for the type index.
     MIB.addImm(0);
     // The table into which this call_indirect indexes.
-    MCSymbolWasm *Table =
-        WebAssembly::getOrCreateFunctionTableSymbol(MF.getContext(), Subtarget);
+    MCSymbolWasm *Table = IsFuncrefCall
+                              ? WebAssembly::getOrCreateFuncrefCallTableSymbol(
+                                    MF.getContext(), Subtarget)
+                              : WebAssembly::getOrCreateFunctionTableSymbol(
+                                    MF.getContext(), Subtarget);
     if (Subtarget->hasReferenceTypes()) {
       MIB.addSym(Table);
     } else {
       // For the MVP there is at most one table whose number is 0, but we can't
       // write a table symbol or issue relocations.  Instead we just ensure the
       // table is live and write a zero.
       Table->setNoStrip();
       MIB.addImm(0);
     }
   }
 
   for (auto Use : CallParams.uses())
     MIB.add(Use);
 
   BB->insert(CallResults.getIterator(), MIB);
   CallParams.eraseFromParent();
   CallResults.eraseFromParent();
 
+  // If this is a funcref call, to avoid hidden GC roots, we need to clear the
+  // table slot with ref.null upon call_indirect return.
+  //
+  // This generates the following code, which comes right after a call_indirect
+  // of a funcref:
+  //
+  //    i32.const 0
+  //    ref.null func
+  //    table.set __funcref_call_table
+  if (IsIndirect && IsFuncrefCall) {
+    MCSymbolWasm *Table = WebAssembly::getOrCreateFuncrefCallTableSymbol(
+        MF.getContext(), Subtarget);
+    Register RegZero =
+        MF.getRegInfo().createVirtualRegister(&WebAssembly::I32RegClass);
+    MachineInstr *Const0 =
+        BuildMI(MF, DL, TII.get(WebAssembly::CONST_I32), RegZero).addImm(0);
+    BB->insertAfter(MIB.getInstr()->getIterator(), Const0);
+
+    Register RegFuncref =
+        MF.getRegInfo().createVirtualRegister(&WebAssembly::FUNCREFRegClass);
+    MachineInstr *RefNull =
+        BuildMI(MF, DL, TII.get(WebAssembly::REF_NULL_FUNCREF), RegFuncref)
+            .addImm(static_cast<int32_t>(WebAssembly::HeapType::Funcref));
+    BB->insertAfter(Const0->getIterator(), RefNull);
+
+    MachineInstr *TableSet =
+        BuildMI(MF, DL, TII.get(WebAssembly::TABLE_SET_FUNCREF))
+            .addSym(Table)
+            .addReg(RegZero)
+            .addReg(RegFuncref);
+    BB->insertAfter(RefNull->getIterator(), TableSet);
+  }
+
   return BB;
 }
 
 MachineBasicBlock *WebAssemblyTargetLowering::EmitInstrWithCustomInserter(
     MachineInstr &MI, MachineBasicBlock *BB) const {
   const TargetInstrInfo &TII = *Subtarget->getInstrInfo();
   DebugLoc DL = MI.getDebugLoc();
 
for (const auto &In : Ins) {
     if (In.Flags.isInConsecutiveRegsLast())
       fail(DL, DAG,
            "WebAssembly hasn't implemented cons regs last return values");
     // Ignore In.getNonZeroOrigAlign() because all our arguments are passed in
     // registers.
     InTys.push_back(In.VT);
   }
 
+  // Lastly, if this is a call to a funcref we need to add an instruction
+  // table.set to the chain and transform the call.
+  if (CLI.CB && isFuncrefType(CLI.CB->getCalledOperand()->getType())) {
+    // In the absence of function references proposal where a funcref call is
+    // lowered to call_ref, using reference types we generate a table.set to set
+    // the funcref to a special table used solely for this purpose, followed by
+    // a call_indirect. Here we just generate the table set, and return the
+    // SDValue of the table.set so that LowerCall can finalize the lowering by
+    // generating the call_indirect.
+    SDValue Chain = Ops[0];
+
+    MCSymbolWasm *Table = WebAssembly::getOrCreateFuncrefCallTableSymbol(
+        MF.getContext(), Subtarget);
+    SDValue Sym = DAG.getMCSymbol(Table, PtrVT);
+    SDValue TableSlot = DAG.getConstant(0, DL, MVT::i32);
+    SDValue TableSetOps[] = {Chain, Sym, TableSlot, Callee};
+    SDValue TableSet = DAG.getMemIntrinsicNode(
+        WebAssemblyISD::TABLE_SET, DL, DAG.getVTList(MVT::Other), TableSetOps,
+        MVT::funcref,
+        // Machine Mem Operand args
+        MachinePointerInfo(WasmAddressSpace::FUNCREF),
+        CLI.CB->getCalledOperand()->getPointerAlignment(DAG.getDataLayout()),
+        MachineMemOperand::MOStore);
+
+    Ops[0] = TableSet; // The new chain is the TableSet itself
+  }
+
   if (CLI.IsTailCall) {
     // ret_calls do not return values to the current frame
     SDVTList NodeTys = DAG.getVTList(MVT::Other, MVT::Glue);
     return DAG.getNode(WebAssemblyISD::RET_CALL, DL, NodeTys, Ops);
   }
 
   InTys.push_back(MVT::Other);
   SDVTList InTyList = DAG.getVTList(InTys);
static Optional<unsigned> IsWebAssemblyLocal(SDValue Op, SelectionDAG &DAG) {
   const FrameIndexSDNode *FI = dyn_cast<FrameIndexSDNode>(Op);
   if (!FI)
     return None;
 
   auto &MF = DAG.getMachineFunction();
   return WebAssemblyFrameLowering::getLocalForStackObject(MF, FI->getIndex());
 }
 
+bool WebAssemblyTargetLowering::isFuncrefType(const Type *Ty) {
+  return isa<PointerType>(Ty) &&
+         Ty->getPointerAddressSpace() == WasmAddressSpace::FUNCREF;
+}
+
+bool WebAssemblyTargetLowering::isExternrefType(const Type *Ty) {
+  return isa<PointerType>(Ty) &&
+         Ty->getPointerAddressSpace() == WasmAddressSpace::EXTERNREF;
+}
+
 SDValue WebAssemblyTargetLowering::LowerStore(SDValue Op,
                                               SelectionDAG &DAG) const {
   SDLoc DL(Op);
   StoreSDNode *SN = cast<StoreSDNode>(Op.getNode());
   const SDValue &Value = SN->getValue();
   const SDValue &Base = SN->getBasePtr();
   const SDValue &Offset = SN->getOffset();
 

llvm/lib/Target/WebAssembly/WebAssemblyInstrTable.td

 // WebAssemblyInstrTable.td - WebAssembly Table codegen support -*- tablegen -*-
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// WebAssembly Table operand code-gen constructs.
 /// Instructions that handle tables
 //===----------------------------------------------------------------------===//
 
-
 multiclass TABLE<WebAssemblyRegClass rt> {
-  defm TABLE_GET_#rt : I<(outs rt:$res), (ins table32_op:$table),
+  let mayLoad = 1 in
+  defm TABLE_GET_#rt : I<(outs rt:$res), (ins table32_op:$table, I32:$i),
                          (outs), (ins table32_op:$table),
                          [],
-                         "table.get\t$res, $table",
+                         "table.get\t$res, $table, $i",
                          "table.get\t$table",
                          0x25>;
 
+  let mayStore = 1 in
   defm TABLE_SET_#rt : I<(outs), (ins table32_op:$table, I32:$i, rt:$val),
                          (outs), (ins table32_op:$table),
                          [],
                          "table.set\t$table, $i, $val",
                          "table.set\t$table",
                          0x26>;
 
   defm TABLE_GROW_#rt : I<(outs I32:$sz), (ins table32_op:$table, rt:$val, I32:$n),
defm TABLE_FILL_#rt : I<(outs), (ins table32_op:$table, I32:$i, rt:$val, I32:$n),
                           "table.fill\t$table",
                           0xfc11>;
 
 }
 
 defm "" : TABLE<FUNCREF>, Requires<[HasReferenceTypes]>;
 defm "" : TABLE<EXTERNREF>, Requires<[HasReferenceTypes]>;
 
+def wasm_table_set_t : SDTypeProfile<0, 3, []>;
+def wasm_table_set : SDNode<"WebAssemblyISD::TABLE_SET", wasm_table_set_t,
+                            [SDNPHasChain, SDNPMayStore, SDNPMemOperand]>;
+
+def : Pat<(wasm_table_set i32:$table, i32:$idx, funcref:$r),
+          (TABLE_SET_FUNCREF i32:$table, i32:$idx, funcref:$r)>,
+          Requires<[HasReferenceTypes]>;
+def : Pat<(wasm_table_set i32:$table, i32:$idx, externref:$r),
+          (TABLE_SET_EXTERNREF i32:$table, i32:$idx, externref:$r)>,
+          Requires<[HasReferenceTypes]>;
+
 defm TABLE_SIZE : I<(outs I32:$sz), (ins table32_op:$table),
                     (outs), (ins table32_op:$table),
                     [],
                     "table.size\t$sz, $table",
                     "table.size\t$table",
                     0xfc10>,
                     Requires<[HasReferenceTypes]>;
 
 
 defm TABLE_COPY : I<(outs), (ins table32_op:$table1, table32_op:$table2, I32:$d, I32:$s, I32:$n),
                     (outs), (ins table32_op:$table1, table32_op:$table2),
                     [],
                     "table.copy\t$table1, $table2, $d, $s, $n",
                     "table.copy\t$table1, $table2",
                     0xfc0e>,
                     Requires<[HasReferenceTypes]>;

llvm/lib/Target/WebAssembly/WebAssemblyMCInstLower.cpp

static wasm::ValType getType(const TargetRegisterClass *RC) {
   if (RC == &WebAssembly::I64RegClass)
     return wasm::ValType::I64;
   if (RC == &WebAssembly::F32RegClass)
     return wasm::ValType::F32;
   if (RC == &WebAssembly::F64RegClass)
     return wasm::ValType::F64;
   if (RC == &WebAssembly::V128RegClass)
     return wasm::ValType::V128;
+  if (RC == &WebAssembly::EXTERNREFRegClass)
+    return wasm::ValType::EXTERNREF;
+  if (RC == &WebAssembly::FUNCREFRegClass)
+    return wasm::ValType::FUNCREF;
   llvm_unreachable("Unexpected register class");
 }
 
 static void getFunctionReturns(const MachineInstr *MI,
                                SmallVectorImpl<wasm::ValType> &Returns) {
   const Function &F = MI->getMF()->getFunction();
   const TargetMachine &TM = MI->getMF()->getTarget();
   Type *RetTy = F.getReturnType();

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

 /// Create an WebAssembly architecture model.
 ///
 WebAssemblyTargetMachine::WebAssemblyTargetMachine(
     const Target &T, const Triple &TT, StringRef CPU, StringRef FS,
     const TargetOptions &Options, Optional<Reloc::Model> RM,
     Optional<CodeModel::Model> CM, CodeGenOpt::Level OL, bool JIT)
     : LLVMTargetMachine(T,
                         TT.isArch64Bit()
-                            ? "e-m:e-p:64:64-i64:64-n32:64-S128-ni:1"
-                            : "e-m:e-p:32:32-i64:64-n32:64-S128-ni:1",
+                            ? "e-m:e-p:64:64-i64:64-n32:64-S128-ni:1:10:20"
+                            : "e-m:e-p:32:32-i64:64-n32:64-S128-ni:1:10:20",
                         TT, CPU, FS, Options, getEffectiveRelocModel(RM, TT),
                         getEffectiveCodeModel(CM, CodeModel::Large), OL),
       TLOF(new WebAssemblyTargetObjectFile()) {
   // WebAssembly type-checks instructions, but a noreturn function with a return
   // type that doesn't match the context will cause a check failure. So we lower
   // LLVM 'unreachable' to ISD::TRAP and then lower that to WebAssembly's
   // 'unreachable' instructions which is meant for that case.
   this->Options.TrapUnreachable = true;

llvm/test/CodeGen/WebAssembly/externref-globalget.ll

+; RUN: llc < %s --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types | FileCheck %s
+
+%extern = type opaque
+%externref = type %extern addrspace(10)* ;; addrspace 10 is nonintegral
+
+@externref_global = local_unnamed_addr addrspace(1) global %externref undef
+
+define %externref @return_externref_global() {
+  ;; this generates a global.get of @externref_global
+  %ref = load %externref, %externref addrspace(1)* @externref_global
+  ret %externref %ref
+}
+
+; CHECK-LABEL: return_externref_global:
+; CHECK-NEXT: functype       return_externref_global () -> (externref)
+; CHECK-NEXT: global.get externref_global
+; CHECK-NEXT: end_function
+
+; CHECK: .globl externref_global

llvm/test/CodeGen/WebAssembly/externref-globalset.ll

+; RUN: llc --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types < %s | FileCheck %s
+
+%extern = type opaque
+%externref = type %extern addrspace(10)* ;; addrspace 10 is nonintegral
+
+@externref_global = local_unnamed_addr addrspace(1) global %externref undef
+
+define void @set_externref_global(%externref %g) {
+  ;; this generates a global.set of @externref.global
+  store %externref %g, %externref addrspace(1)* @externref_global
+  ret void
+}
+
+; CHECK-LABEL: set_externref_global:
+; CHECK-NEXT: functype       set_externref_global (externref) -> ()
+; CHECK-NEXT: local.get      0
+; CHECK-NEXT: global.set     externref_global
+; CHECK-NEXT: end_function
+
+; CHECK: .globl externref_global

llvm/test/CodeGen/WebAssembly/externref-inttoptr.ll

+; RUN: not --crash llc --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types < %s 2>&1 | FileCheck %s --check-prefix=CHECK-ERROR
+
+%extern = type opaque
+%externref = type %extern addrspace(10)*
+
+define %externref @int_to_externref(i32 %i) {
+  %ref = inttoptr i32 %i to %externref
+  ret %externref %ref
+}
+
+; CHECK-ERROR: LLVM ERROR: inttoptr not allowed on reference types

llvm/test/CodeGen/WebAssembly/externref-ptrtoint.ll

+; RUN: not --crash llc --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types < %s 2>&1 | FileCheck %s --check-prefix=CHECK-ERROR
+
+%extern = type opaque
+%externref = type %extern addrspace(10)*
+
+define i32 @externref_to_int(%externref %ref) {
+  %i = ptrtoint %externref %ref to i32
+  ret i32 %i
+}
+
+; CHECK-ERROR: LLVM ERROR: ptrtoint not allowed on reference types

llvm/test/CodeGen/WebAssembly/externref-undef.ll

+; RUN: llc < %s --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types | FileCheck %s
+
+%extern = type opaque
+%externref = type %extern addrspace(10)* ;; addrspace 10 is nonintegral
+
+@externref_global = local_unnamed_addr addrspace(1) global %externref undef
+
+define %extern @return_extern_undef() {
+  ; Returning a ref.null or an uninitialized externref would make
+  ; more sense if the return type would be %externref. However, in
+  ; this case this is an %extern value, which really is an opaque
+  ; type and should never really happen.
+  ret %extern undef
+}
+
+; CHECK-LABEL: return_extern_undef:
+; CHECK-NEXT: functype       return_extern_undef () -> ()
+; CHECK-NEXT: end_function
+
+; CHECK: .globl externref_global
+

llvm/test/CodeGen/WebAssembly/externref-unsized-load.ll

+; RUN: not llc --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types < %s 2>&1 | FileCheck %s --check-prefix=CHECK-ERROR
+
+%extern = type opaque
+%externref = type %extern addrspace(10)*
+
+define void @load_extern(%externref %ref) {
+  %e = load %extern, %externref %ref
+  ret void
+}
+
+; CHECK-ERROR: error: loading unsized types is not allowed

llvm/test/CodeGen/WebAssembly/externref-unsized-store.ll

+; RUN: not llc --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types < %s 2>&1 | FileCheck %s --check-prefix=CHECK-ERROR
+
+%extern = type opaque
+%externref = type %extern addrspace(10)*
+
+define void @store_extern(%externref %ref) {
+  store %extern undef, %externref %ref
+  ret void
+}
+
+; CHECK-ERROR: error: storing unsized types is not allowed

llvm/test/CodeGen/WebAssembly/funcref-call.ll

+; RUN: llc < %s --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types | FileCheck %s
+
+%func = type void ()
+%funcref = type %func addrspace(20)* ;; addrspace 20 is nonintegral
+
+define void @call_funcref(%funcref %ref) {
+  call addrspace(20) void %ref() 
+  ret void
+}
+
+; CHECK-LABEL: call_funcref:
+; CHECK-NEXT: functype       call_funcref (funcref) -> ()
+; CHECK-NEXT: i32.const 0
+; CHECK-NEXT: local.get 0
+; CHECK-NEXT: table.set __funcref_call_table
+; CHECK-NEXT: local.get 0
+; CHECK-NEXT: call_indirect __funcref_call_table, () -> ()
+; CHECK-NEXT: i32.const 0
+; CHECK-NEXT: ref.null func
+; CHECK-NEXT: table.set __funcref_call_table
+; CHECK-NEXT: end_function
+
+; CHECK: .tabletype __funcref_call_table, funcref

llvm/test/CodeGen/WebAssembly/funcref-globalget.ll

+; RUN: llc < %s --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types | FileCheck %s
+
+%func = type opaque
+%funcref = type %func addrspace(20)* ;; addrspace 20 is nonintegral
+
+@funcref_global = local_unnamed_addr addrspace(1) global %funcref undef
+
+define %funcref @return_funcref_global() {
+  ;; this generates a global.get of @funcref_global
+  %ref = load %funcref, %funcref addrspace(1)* @funcref_global
+  ret %funcref %ref
+}
+
+; CHECK-LABEL: return_funcref_global:
+; CHECK-NEXT: .functype       return_funcref_global () -> (funcref)
+; CHECK-NEXT: global.get funcref_global
+; CHECK-NEXT: end_function
+
+; CHECK: .globl funcref_global

llvm/test/CodeGen/WebAssembly/funcref-globalset.ll

+; RUN: llc < %s --mtriple=wasm32-unknown-unknown -asm-verbose=false -mattr=+reference-types | FileCheck %s
+
+%func = type opaque
+%funcref = type %func addrspace(20)* ;; addrspace 20 is nonintegral
+
+@funcref_global = local_unnamed_addr addrspace(1) global %funcref undef
+
+define void @set_funcref_global(%funcref %g) {
+  ;; this generates a global.set of @funcref_global
+  store %funcref %g, %funcref addrspace(1)* @funcref_global
+  ret void
+}
+
+; CHECK-LABEL: set_funcref_global:
+; CHECK-NEXT: functype       set_funcref_global (funcref) -> ()
+; CHECK-NEXT: local.get      0
+; CHECK-NEXT: global.set     funcref_global
+; CHECK-NEXT: end_function
+
+; CHECK: .globl funcref_global