[WebAssembly] Support for WebAssembly globals in LLVM IR

[wingo]

Summary

This patch adds support for WebAssembly globals in LLVM IR, representing
them as pointers to global values, in a non-default, non-integral
address space. Instruction selection legalizes loads and stores to
these pointers to new WebAssemblyISD nodes GLOBAL_GET and GLOBAL_SET.
Once the lowering creates the new nodes, tablegen pattern matches those
and converts them to Wasm global.get/set of the appropriate type.

Based on work by Paulo Matos in https://reviews.llvm.org/D95425.

[wingo]

Context here is that I realized that in D95425, that we could use a single address space for globals of any type, and that it might make sense to do so for "ordinary" globals like i32 etc. Then we can express adding support for funcref / externref on top of this. WDYT @pmatos ?

[wingo]

Fix clang datalayout expectations

[tlively]

Nice! I think that splitting this out of the change that adds reference types is a good idea. Regarding the FIXME in the test, is it the case that the globals are also not emitted in the binary format, or is it just the assembly output that is broken?

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1318

I'm not sure it's necessary to create a MERGE_NODES node here, since the the GlobalGet already contains the chain. For a similar example, see how LOAD_SPLAT nodes are created and returned.

llvm/lib/Target/WebAssembly/WebAssemblyInstrInfo.td

268

In other places we have used the abbreviation rc as the name for WebAssemblyRegClass parameters, but I don't know if we do that everywhere.

333

Ahhhh, I did not know you could do this! Very cool.

[pmatos]

This looks good to me - I will rebase D95425 on this.

[pmatos]

A quick note - apparently phab felt that since I accepted the revision, it's now ready to land. Probably obvious but I accepted expecting @tlively comments to be addressed first. :)

[wingo]

Rebase on main

[wingo]

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1318

I think it is

1318

Aaaah thanks for this tip! Took some finagling to find the right formula but I got it now.

llvm/lib/Target/WebAssembly/WebAssemblyInstrInfo.td

268

Good tip; apparently it's consistent except in WebAssemblyInstrRef.td, where I had initiailly looked. Will go ahead and fix that one too.

[wingo]

Address feedback

[wingo]

Pull in fixes from D101140

[sbc100]

Does this mean that the magic __stack_pointer global can be referenced at the IR level? I wonder if there are some hacks around handling of __stack_pointer that could be removed after this lands?

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.h

40

What does "object" mean here? Are we just talking about reference types? Or also wasm globals that hold integers (like __stack_pointer). If its just ref types that live in this address space should this be called WASM_ADDRESS_SPACE_ANYREF? If its the latter should this be called WASM_ADDRESS_SPACE_WASM_GLOBAL?

llvm/lib/Target/WebAssembly/WebAssemblyISD.def

48

Is this just for ref types or also for global that hold integers too (like __stack_pointer)

llvm/test/CodeGen/WebAssembly/global-get.ll

14

I don't suppose there is any way to give addrspace(1) a symbolic name is there?

[sbc100]

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1373

Is it worth checking this is within WasmAddressSpace range?

[tlively]

LGTM with Sam's comments resolved!

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.h

40

I was also wondering about the best name here because OBJECT is somewhat vague. I think the idea is that this address space can be used for arbitrary Wasm globals of any type, but it could also be used later for things like additional tables and memories. It's unclear whether those would need separate address spaces for some reason, but if they don't, re-using this address space 1 would be best.

Maybe WASM_ADDRESS_SPACE_STATIC would be a better name because it will be used for things that are given static indices in the final module?

llvm/lib/Target/WebAssembly/WebAssemblyISD.def

48

Yep, it's for arbitrary types (see the tests), so this comment should be updated.

llvm/test/CodeGen/WebAssembly/global-get.ll

14

Not that I know of, unfortunately :(

[sbc100]

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.h

40

I was also wondering about the best name here because OBJECT is somewhat vague. I think the idea is that this address space can be used for arbitrary Wasm globals of any type, but it could also be used later for things like additional tables and memories. It's unclear whether those would need separate address spaces for some reason, but if they don't, re-using this address space 1 would be best.

Maybe WASM_ADDRESS_SPACE_STATIC would be a better name because it will be used for things that are given static indices in the final module?

But that is also true for static data symbols that point memory addresess.

[tlively]

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.h

40

Hmm, maybe WASM_ADDRESS_SPACE_MODULE_ELEMENT?

[sbc100]

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.h

40

The address space is the space of wasm globals right? So I feel like it should probably contain the word global... shouldn't it?

[tlively]

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.h

40

No, see my previous comment. It's more general than just globals. Tables and Memories may also reside in this address space in the future. That being said, I guess we could just name it GLOBAL for now and change the name once we get table and memory support.

Although now that I think about it, https://reviews.llvm.org/D101140 also uses this address space for locals, so it's even more general than I was thinking.

[sbc100]

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.h

40

No, see my previous comment. It's more general than just globals. Tables and Memories may also reside in this address space in the future. That being said, I guess we could just name it GLOBAL for now and change the name once we get table and memory support.

Although now that I think about it, https://reviews.llvm.org/D101140 also uses this address space for locals, so it's even more general than I was thinking.

But are those locals backed by global.get and global.set?

OK well I don't feel so strongly about it then. I had assumed that address space corresponded to wasm globals...

[tlively]

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.h

40

No, those locals turn into normal locals backed by local.get and local.set. In that case, the address space is used in the type of alloca instructions.

I guess maybe OBJECT is an appropriately general label for such a general address space. Looking forward to seeing if @wingo has any better ideas!

[sbc100]

How about WASM_ADDRESS_SPACE_NON_INTEGRAL or WASM_ADDRESS_SPACE_OPAQUE ?

[wingo]

Thanks for the review!

Regarding the address space name -- the intention with "object" was to rhyme a bit with "object capabilities", somehow, but I see that it is confusing.

How about WASM_ADDRESS_SPACE_MANAGED ? I.e. this address space is for "managed" variables. Doesn't mean that the variables are under GC control (though they may be), but rather that the wasm implementation manages their storage itself, instead of having them be in linear memory. Will post a renamed patch, lmk what you think.

[wingo]

llvm/lib/Target/WebAssembly/WebAssemblyISD.def

48

FWIW I think you would be able to define __stack_pointer from IR via

@__stack_pointer = external addrspace(1) global i32

and then to use it:

%current_sp = load i32, i32 addrspace(1)* @__stack_pointer

[wingo]

Rename "object" address space to "managed"; address feedback.

[wingo]

yarr, shiver me timbers, fix me typos

[tlively]

This LGTM! MANAGED sounds like a good address space name to me. @sbc100, do you have any final comments?

[sbc100]

managed seems reasonable yes.

[tlively]

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1274

Actually, should we enforce that these LLVM IR globals be thread_local, since the resulting Wasm globals will be thread_local? I don't know if that will affect any optimizations, but it seems like a more accurate modeling. If we do that, CoalesceLocalsAndStripAtomics::stripThreadLocals in WebAssemblyTargetMachine.cpp will have to be updated to not strip the thread locals corresponding to Wasm globals.

1274

I'd be fine handling this in a follow-up if you want to get this landed.

[sunfish]

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.h

39

Sorry to throw more paint at the bikeshed here, but as someone who's only following along at a high-level here, I found it confusing whether this is talking about the wasm globals themselves, or the objects referred to by reference values in the wasm globals. I think the feature here is talking about the wasm globals themselves, but "managed" initially made me think it might be talking about the objects they reference, which in a browser context especially are "managed" in every sense of the word.

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1274

Note that this will be true of Worker-based threads, but not of the expected future "wasm native threads". I wonder if this means that clang/llvm will (eventually) need to be aware of this difference.

[tlively]

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.h

39

Fair point. What does everyone think about INDEXED, because it is used to represent objects given static indexes (in the WebAssembly sense) in the final binary.

[sbc100]

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1274

Don't you think is very likely that even in "wasm native threads" we would want to opt into sharing like we do with wasm memories? That path would seem better for compatibility with existing worker-based threading.

Obviously once we do have shared wasm globals we will need to modify llvm's assumptions, but for now I think it is a reasonable assumption/assertion that wasm globals are TLS.

[sunfish]

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1274

Yeah, I agree we'll likely want some way to opt into sharing. So the difference here is, worker-based threads won't have the option of sharing, so maybe requiring thread_local makes sense. But native threads could opt into sharing, so they could be thread_local or not.

[sbc100]

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1274

Yup. And who know .. maybe by the time shared globals and native threads are added, Workers might also add shared globals?

[sunfish]

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.h

39

Do I understand correctly that global variables and local variables are being assigned addresses within the same conceptual address space here?

How about WASM_VARIABLES or WASM_VARS? The wasm spec terms for these are global variables and local variables.

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

1274

It's possible :-). In any case, I don't think it's anything we need to do anything about right now.

[tlively]

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.h

39

Sure, that works for me. We may want to rename it in the future if we end up using the same address space for tables and memories, but we can cross that bridge when we get there.

[sunfish]

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.h

39

Tables and memories have their own index spaces in wasm, so it seems like they'd want their own address spaces here, perhaps 2 and 3, respectively? I also agree that we can figure this out later.

[wingo]

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.h

39

WASM_VARS is fine with me, with one caveat -- in https://reviews.llvm.org/D101140 where we add support for locals, we add a new "core" TargetStackID enum which describes static alloca that is actually a wasm local. Really this enum describes objects allocated on a parallel stack, outside of linear memory. It was nice to use the same name to describe the address space and the stack id, but I don't think variable or wasm-var are good core enum names. It feels like there is a core concept here for systems that have two stacks -- one of named objects and one of linear-memory bytes. I thought object or managed could be a good name for the former, but evidently not :)

I will let this one sit overnight; if there are no other good ideas before tomorrow I will do WASM_ADDRESS_SPACE_WASM_VARS and either find a new name for TargetStackID or attempt to use a target-specific definition.

[wingo]

Rename "managed" address space to "wasm vars"

clang/lib/Basic/Targets/WebAssembly.h

 };
 
 class LLVM_LIBRARY_VISIBILITY WebAssembly32TargetInfo
     : public WebAssemblyTargetInfo {
 public:
   explicit WebAssembly32TargetInfo(const llvm::Triple &T,
                                    const TargetOptions &Opts)
       : WebAssemblyTargetInfo(T, Opts) {
-    resetDataLayout("e-m:e-p:32:32-i64:64-n32:64-S128");
+    resetDataLayout("e-m:e-p:32:32-i64:64-n32:64-S128-ni:1");
   }
 
 protected:
   void getTargetDefines(const LangOptions &Opts,
                         MacroBuilder &Builder) const override;
 };
 
 class LLVM_LIBRARY_VISIBILITY WebAssembly64TargetInfo
     : public WebAssemblyTargetInfo {
 public:
   explicit WebAssembly64TargetInfo(const llvm::Triple &T,
                                    const TargetOptions &Opts)
       : WebAssemblyTargetInfo(T, Opts) {
     LongAlign = LongWidth = 64;
     PointerAlign = PointerWidth = 64;
     SizeType = UnsignedLong;
     PtrDiffType = SignedLong;
     IntPtrType = SignedLong;
-    resetDataLayout("e-m:e-p:64:64-i64:64-n32:64-S128");
+    resetDataLayout("e-m:e-p:64:64-i64:64-n32:64-S128-ni:1");
   }
 
 protected:
   void getTargetDefines(const LangOptions &Opts,
                         MacroBuilder &Builder) const override;
 };
 } // namespace targets
 } // namespace clang
 #endif // LLVM_CLANG_LIB_BASIC_TARGETS_WEBASSEMBLY_H

clang/test/CodeGen/target-data.c

 // ARM-NACL: target datalayout = "e-m:e-p:32:32-Fi8-i64:64-v128:64:128-a:0:32-n32-S128"
 
 // RUN: %clang_cc1 -triple mipsel-nacl -o - -emit-llvm %s | \
 // RUN: FileCheck %s -check-prefix=MIPS-NACL
 // MIPS-NACL: target datalayout = "e-m:m-p:32:32-i8:8:32-i16:16:32-i64:64-n32-S64"
 
 // RUN: %clang_cc1 -triple wasm32-unknown-unknown -o - -emit-llvm %s | \
 // RUN: FileCheck %s -check-prefix=WEBASSEMBLY32
-// WEBASSEMBLY32: target datalayout = "e-m:e-p:32:32-i64:64-n32:64-S128"
+// WEBASSEMBLY32: target datalayout = "e-m:e-p:32:32-i64:64-n32:64-S128-ni:1"
 
 // RUN: %clang_cc1 -triple wasm64-unknown-unknown -o - -emit-llvm %s | \
 // RUN: FileCheck %s -check-prefix=WEBASSEMBLY64
-// WEBASSEMBLY64: target datalayout = "e-m:e-p:64:64-i64:64-n32:64-S128"
+// WEBASSEMBLY64: target datalayout = "e-m:e-p:64:64-i64:64-n32:64-S128-ni:1"
 
 // RUN: %clang_cc1 -triple lanai-unknown-unknown -o - -emit-llvm %s | \
 // RUN: FileCheck %s -check-prefix=LANAI
 // LANAI: target datalayout = "E-m:e-p:32:32-i64:64-a:0:32-n32-S64"
 
 // RUN: %clang_cc1 -triple powerpc-unknown -o - -emit-llvm %s | \
 // RUN: FileCheck %s -check-prefix=PPC
 // PPC: target datalayout = "E-m:e-p:32:32-i64:64-n32"

llvm/lib/Target/WebAssembly/Utils/WebAssemblyUtilities.h

 class MCContext;
 class MCSymbolWasm;
 class StringRef;
 class WebAssemblyFunctionInfo;
 class WebAssemblySubtarget;
 
 namespace WebAssembly {
 
+enum WasmAddressSpace : unsigned {
+  // Default address space, for pointers to linear memory (stack, heap, data).
+  WASM_ADDRESS_SPACE_DEFAULT = 0,
+  // A non-integral address space for pointers to named objects outside of
+  // linear memory: WebAssembly globals or WebAssembly locals.  Loads and stores
+  // to these pointers are lowered to global.get / global.set or local.get /
+  // local.set, as appropriate.
+  WASM_ADDRESS_SPACE_WASM_VAR = 1
+};
+
+inline bool isDefaultAddressSpace(unsigned AS) {
+  return AS == WASM_ADDRESS_SPACE_DEFAULT;
+}
+inline bool isWasmVarAddressSpace(unsigned AS) {
+  return AS == WASM_ADDRESS_SPACE_WASM_VAR;
+}
+inline bool isValidAddressSpace(unsigned AS) {
+  return isDefaultAddressSpace(AS) || isWasmVarAddressSpace(AS);
+}
+
 bool isChild(const MachineInstr &MI, const WebAssemblyFunctionInfo &MFI);
 bool mayThrow(const MachineInstr &MI);
 
 // Exception-related function names
 extern const char *const ClangCallTerminateFn;
 extern const char *const CxaBeginCatchFn;
 extern const char *const CxaRethrowFn;
 extern const char *const StdTerminateFn;

llvm/lib/Target/WebAssembly/WebAssemblyFastISel.cpp

bool WebAssemblyFastISel::selectBitCast(const Instruction *I) {
   updateValueMap(I, Reg);
   return true;
 }
 
 bool WebAssemblyFastISel::selectLoad(const Instruction *I) {
   const auto *Load = cast<LoadInst>(I);
   if (Load->isAtomic())
     return false;
+  if (!WebAssembly::isDefaultAddressSpace(Load->getPointerAddressSpace()))
+    return false;
   if (!Subtarget->hasSIMD128() && Load->getType()->isVectorTy())
     return false;
 
   Address Addr;
   if (!computeAddress(Load->getPointerOperand(), Addr))
     return false;
 
   // TODO: Fold a following sign-/zero-extend into the load instruction.
bool WebAssemblyFastISel::selectLoad(const Instruction *I) {
   updateValueMap(Load, ResultReg);
   return true;
 }
 
 bool WebAssemblyFastISel::selectStore(const Instruction *I) {
   const auto *Store = cast<StoreInst>(I);
   if (Store->isAtomic())
     return false;
+  if (!WebAssembly::isDefaultAddressSpace(Store->getPointerAddressSpace()))
+    return false;
   if (!Subtarget->hasSIMD128() &&
       Store->getValueOperand()->getType()->isVectorTy())
     return false;
 
   Address Addr;
   if (!computeAddress(Store->getPointerOperand(), Addr))
     return false;
 

llvm/lib/Target/WebAssembly/WebAssemblyISD.def

 // NOTE: NO INCLUDE GUARD DESIRED!
 
 HANDLE_NODETYPE(CALL)
 HANDLE_NODETYPE(RET_CALL)
 HANDLE_NODETYPE(RETURN)
 HANDLE_NODETYPE(ARGUMENT)
 // A wrapper node for TargetExternalSymbol, TargetGlobalAddress, and MCSymbol
 HANDLE_NODETYPE(Wrapper)
-// A special wapper used in PIC code for __memory_base/__table_base relcative
+// A special wapper used in PIC code for __memory_base/__table_base relative
 // access.
 HANDLE_NODETYPE(WrapperPIC)
 HANDLE_NODETYPE(BR_IF)
 HANDLE_NODETYPE(BR_TABLE)
 HANDLE_NODETYPE(SHUFFLE)
 HANDLE_NODETYPE(SWIZZLE)
 HANDLE_NODETYPE(VEC_SHL)
 HANDLE_NODETYPE(VEC_SHR_S)
 HANDLE_NODETYPE(VEC_SHR_U)
 HANDLE_NODETYPE(EXTEND_LOW_S)
 HANDLE_NODETYPE(EXTEND_LOW_U)
 HANDLE_NODETYPE(EXTEND_HIGH_S)
 HANDLE_NODETYPE(EXTEND_HIGH_U)
 HANDLE_NODETYPE(CONVERT_LOW_S)
 HANDLE_NODETYPE(CONVERT_LOW_U)
 HANDLE_NODETYPE(TRUNC_SAT_ZERO_S)
 HANDLE_NODETYPE(TRUNC_SAT_ZERO_U)
 HANDLE_NODETYPE(THROW)
 HANDLE_NODETYPE(CATCH)
 HANDLE_NODETYPE(MEMORY_COPY)
 HANDLE_NODETYPE(MEMORY_FILL)
 
 // Memory intrinsics
 HANDLE_MEM_NODETYPE(LOAD_SPLAT)
+HANDLE_MEM_NODETYPE(GLOBAL_GET)
+HANDLE_MEM_NODETYPE(GLOBAL_SET)

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.h

private:
   SDValue LowerIntrinsic(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerSIGN_EXTEND_INREG(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerBUILD_VECTOR(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerSETCC(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerAccessVectorElement(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerShift(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerFP_TO_INT_SAT(SDValue Op, SelectionDAG &DAG) const;
+  SDValue LowerLoad(SDValue Op, SelectionDAG &DAG) const;
+  SDValue LowerStore(SDValue Op, SelectionDAG &DAG) const;
 
   // Custom DAG combine hooks
   SDValue
   PerformDAGCombine(SDNode *N,
                     TargetLowering::DAGCombinerInfo &DCI) const override;
 };
 
 namespace WebAssembly {
 FastISel *createFastISel(FunctionLoweringInfo &funcInfo,
                          const TargetLibraryInfo *libInfo);
 } // end namespace WebAssembly
 
 } // end namespace llvm
 
 #endif

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

 //===----------------------------------------------------------------------===//
 
 #include "WebAssemblyISelLowering.h"
 #include "MCTargetDesc/WebAssemblyMCTargetDesc.h"
 #include "Utils/WebAssemblyUtilities.h"
 #include "WebAssemblyMachineFunctionInfo.h"
 #include "WebAssemblySubtarget.h"
 #include "WebAssemblyTargetMachine.h"
-#include "llvm/CodeGen/Analysis.h"
 #include "llvm/CodeGen/CallingConvLower.h"
 #include "llvm/CodeGen/MachineInstrBuilder.h"
 #include "llvm/CodeGen/MachineJumpTableInfo.h"
 #include "llvm/CodeGen/MachineModuleInfo.h"
 #include "llvm/CodeGen/MachineRegisterInfo.h"
 #include "llvm/CodeGen/SelectionDAG.h"
+#include "llvm/CodeGen/SelectionDAGNodes.h"
 #include "llvm/CodeGen/WasmEHFuncInfo.h"
 #include "llvm/IR/DiagnosticInfo.h"
 #include "llvm/IR/DiagnosticPrinter.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/Intrinsics.h"
 #include "llvm/IR/IntrinsicsWebAssembly.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/ErrorHandling.h"
if (Subtarget->hasSIMD128()) {
     addRegisterClass(MVT::v4i32, &WebAssembly::V128RegClass);
     addRegisterClass(MVT::v4f32, &WebAssembly::V128RegClass);
     addRegisterClass(MVT::v2i64, &WebAssembly::V128RegClass);
     addRegisterClass(MVT::v2f64, &WebAssembly::V128RegClass);
   }
   // Compute derived properties from the register classes.
   computeRegisterProperties(Subtarget->getRegisterInfo());
 
+  // Transform loads and stores to pointers in address space 1 to loads and
+  // stores to WebAssembly global variables, outside linear memory.
+  for (auto T : {MVT::i32, MVT::i64, MVT::f32, MVT::f64}) {
+    setOperationAction(ISD::LOAD, T, Custom);
+    setOperationAction(ISD::STORE, T, Custom);
+  }
+  if (Subtarget->hasSIMD128()) {
+    for (auto T : {MVT::v16i8, MVT::v8i16, MVT::v4i32, MVT::v4f32, MVT::v2i64,
+                   MVT::v2f64}) {
+      setOperationAction(ISD::LOAD, T, Custom);
+      setOperationAction(ISD::STORE, T, Custom);
+    }
+  }
+
   setOperationAction(ISD::GlobalAddress, MVTPtr, Custom);
   setOperationAction(ISD::GlobalTLSAddress, MVTPtr, Custom);
   setOperationAction(ISD::ExternalSymbol, MVTPtr, Custom);
   setOperationAction(ISD::JumpTable, MVTPtr, Custom);
   setOperationAction(ISD::BlockAddress, MVTPtr, Custom);
   setOperationAction(ISD::BRIND, MVT::Other, Custom);
 
   // Take the default expansion for va_arg, va_copy, and va_end. There is no
case ISD::SETCC:
     return LowerSETCC(Op, DAG);
   case ISD::SHL:
   case ISD::SRA:
   case ISD::SRL:
     return LowerShift(Op, DAG);
   case ISD::FP_TO_SINT_SAT:
   case ISD::FP_TO_UINT_SAT:
     return LowerFP_TO_INT_SAT(Op, DAG);
+  case ISD::LOAD:
+    return LowerLoad(Op, DAG);
+  case ISD::STORE:
+    return LowerStore(Op, DAG);
   }
 }
 
+static bool IsWebAssemblyGlobal(SDValue Op) {
+  if (const GlobalAddressSDNode *GA = dyn_cast<GlobalAddressSDNode>(Op))
+    return WebAssembly::isWasmVarAddressSpace(GA->getAddressSpace());
+
+  return false;
+}
+
+SDValue WebAssemblyTargetLowering::LowerStore(SDValue Op,
+                                              SelectionDAG &DAG) const {
+  SDLoc DL(Op);
+  StoreSDNode *SN = cast<StoreSDNode>(Op.getNode());
+  const SDValue &Value = SN->getValue();
+  const SDValue &Base = SN->getBasePtr();
+  const SDValue &Offset = SN->getOffset();
+
+  if (IsWebAssemblyGlobal(Base)) {
+    if (!Offset->isUndef())
+      report_fatal_error("unexpected offset when storing to webassembly global",
+                         false);
+
+    SDVTList Tys = DAG.getVTList(MVT::Other);
+    SDValue Ops[] = {SN->getChain(), Value, Base};
+    return DAG.getMemIntrinsicNode(WebAssemblyISD::GLOBAL_SET, DL, Tys, Ops,
+                                   SN->getMemoryVT(), SN->getMemOperand());
+  }
+
+  return Op;
+}
+
+SDValue WebAssemblyTargetLowering::LowerLoad(SDValue Op,
+                                             SelectionDAG &DAG) const {
+  SDLoc DL(Op);
+  LoadSDNode *LN = cast<LoadSDNode>(Op.getNode());
+  const SDValue &Base = LN->getBasePtr();
+  const SDValue &Offset = LN->getOffset();
+
+  if (IsWebAssemblyGlobal(Base)) {
+    if (!Offset->isUndef())
+      report_fatal_error(
+          "unexpected offset when loading from webassembly global", false);
+
+    SDVTList Tys = DAG.getVTList(LN->getValueType(0), MVT::Other);
+    SDValue Ops[] = {LN->getChain(), Base};
+    return DAG.getMemIntrinsicNode(WebAssemblyISD::GLOBAL_GET, DL, Tys, Ops,
+                                   LN->getMemoryVT(), LN->getMemOperand());
+  }
+
+  return Op;
+}
+
 SDValue WebAssemblyTargetLowering::LowerCopyToReg(SDValue Op,
                                                   SelectionDAG &DAG) const {
   SDValue Src = Op.getOperand(2);
   if (isa<FrameIndexSDNode>(Src.getNode())) {
     // CopyToReg nodes don't support FrameIndex operands. Other targets select
     // the FI to some LEA-like instruction, but since we don't have that, we
     // need to insert some kind of instruction that can take an FI operand and
     // produces a value usable by CopyToReg (i.e. in a vreg). So insert a dummy

 
 SDValue WebAssemblyTargetLowering::LowerGlobalAddress(SDValue Op,
                                                       SelectionDAG &DAG) const {
   SDLoc DL(Op);
   const auto *GA = cast<GlobalAddressSDNode>(Op);
   EVT VT = Op.getValueType();
   assert(GA->getTargetFlags() == 0 &&
          "Unexpected target flags on generic GlobalAddressSDNode");
-  if (GA->getAddressSpace() != 0)
-    fail(DL, DAG, "WebAssembly only expects the 0 address space");
+  if (!WebAssembly::isValidAddressSpace(GA->getAddressSpace()))
+    fail(DL, DAG, "Invalid address space for WebAssembly target");
 
   unsigned OperandFlags = 0;
   if (isPositionIndependent()) {
     const GlobalValue *GV = GA->getGlobal();
     if (getTargetMachine().shouldAssumeDSOLocal(*GV->getParent(), GV)) {
       MachineFunction &MF = DAG.getMachineFunction();
       MVT PtrVT = getPointerTy(MF.getDataLayout());
       const char *BaseName;

llvm/lib/Target/WebAssembly/WebAssemblyInstrInfo.td

 def SDT_WebAssemblyArgument   : SDTypeProfile<1, 1, [SDTCisVT<1, i32>]>;
 def SDT_WebAssemblyReturn     : SDTypeProfile<0, -1, []>;
 def SDT_WebAssemblyWrapper    : SDTypeProfile<1, 1, [SDTCisSameAs<0, 1>,
                                                      SDTCisPtrTy<0>]>;
 def SDT_WebAssemblyWrapperPIC : SDTypeProfile<1, 1, [SDTCisSameAs<0, 1>,
                                                      SDTCisPtrTy<0>]>;
 def SDT_WebAssemblyThrow      : SDTypeProfile<0, -1, []>;
 def SDT_WebAssemblyCatch      : SDTypeProfile<1, 1, [SDTCisPtrTy<0>]>;
+def SDT_WebAssemblyGlobalGet  : SDTypeProfile<1, 1, [SDTCisPtrTy<1>]>;
+def SDT_WebAssemblyGlobalSet  : SDTypeProfile<0, 2, [SDTCisPtrTy<1>]>;
 
 //===----------------------------------------------------------------------===//
 // WebAssembly-specific DAG Nodes.
 //===----------------------------------------------------------------------===//
 
 def WebAssemblycallseq_start :
     SDNode<"ISD::CALLSEQ_START", SDT_WebAssemblyCallSeqStart,
            [SDNPHasChain, SDNPOutGlue]>;

 def WebAssemblywrapper  : SDNode<"WebAssemblyISD::Wrapper",
                                  SDT_WebAssemblyWrapper>;
 def WebAssemblywrapperPIC  : SDNode<"WebAssemblyISD::WrapperPIC",
                                      SDT_WebAssemblyWrapperPIC>;
 def WebAssemblythrow : SDNode<"WebAssemblyISD::THROW", SDT_WebAssemblyThrow,
                               [SDNPHasChain, SDNPVariadic]>;
 def WebAssemblycatch : SDNode<"WebAssemblyISD::CATCH", SDT_WebAssemblyCatch,
                               [SDNPHasChain, SDNPSideEffect]>;
+def WebAssemblyglobal_get :
+    SDNode<"WebAssemblyISD::GLOBAL_GET", SDT_WebAssemblyGlobalGet,
+           [SDNPHasChain, SDNPMayLoad, SDNPMemOperand]>;
+def WebAssemblyglobal_set :
+    SDNode<"WebAssemblyISD::GLOBAL_SET", SDT_WebAssemblyGlobalSet,
+           [SDNPHasChain, SDNPMayStore, SDNPMemOperand]>;
 
 //===----------------------------------------------------------------------===//
 // WebAssembly-specific Operands.
 //===----------------------------------------------------------------------===//
 
 // Default Operand has AsmOperandClass "Imm" which is for integers (and
 // symbols), so specialize one for floats:
 class FPImmAsmOperand<ValueType ty> : AsmOperandClass {

 //===----------------------------------------------------------------------===//
 
 include "WebAssemblyInstrFormats.td"
 
 //===----------------------------------------------------------------------===//
 // Additional instructions.
 //===----------------------------------------------------------------------===//
 
-multiclass ARGUMENT<WebAssemblyRegClass reg, ValueType vt> {
+multiclass ARGUMENT<WebAssemblyRegClass rc, ValueType vt> {
   let hasSideEffects = 1, isCodeGenOnly = 1, Defs = []<Register>,
       Uses = [ARGUMENTS] in
   defm ARGUMENT_#vt :
-    I<(outs reg:$res), (ins i32imm:$argno), (outs), (ins i32imm:$argno),
-      [(set (vt reg:$res), (WebAssemblyargument timm:$argno))]>;
+    I<(outs rc:$res), (ins i32imm:$argno), (outs), (ins i32imm:$argno),
+      [(set (vt rc:$res), (WebAssemblyargument timm:$argno))]>;
 }
 defm "": ARGUMENT<I32, i32>;
 defm "": ARGUMENT<I64, i64>;
 defm "": ARGUMENT<F32, f32>;
 defm "": ARGUMENT<F64, f64>;
 defm "": ARGUMENT<FUNCREF, funcref>;
 defm "": ARGUMENT<EXTERNREF, externref>;
 
 // local.get and local.set are not generated by instruction selection; they
 // are implied by virtual register uses and defs.
-multiclass LOCAL<WebAssemblyRegClass vt, Operand global_op> {
+multiclass LOCAL<WebAssemblyRegClass rc, Operand global_op> {
   let hasSideEffects = 0 in {
   // COPY is not an actual instruction in wasm, but since we allow local.get and
   // local.set to be implicit during most of codegen, we can have a COPY which
   // is actually a no-op because all the work is done in the implied local.get
   // and local.set. COPYs are eliminated (and replaced with
   // local.get/local.set) in the ExplicitLocals pass.
   let isAsCheapAsAMove = 1, isCodeGenOnly = 1 in
-  defm COPY_#vt : I<(outs vt:$res), (ins vt:$src), (outs), (ins), [],
+  defm COPY_#rc : I<(outs rc:$res), (ins rc:$src), (outs), (ins), [],
                     "local.copy\t$res, $src", "local.copy">;
 
   // TEE is similar to COPY, but writes two copies of its result. Typically
   // this would be used to stackify one result and write the other result to a
   // local.
   let isAsCheapAsAMove = 1, isCodeGenOnly = 1 in
-  defm TEE_#vt : I<(outs vt:$res, vt:$also), (ins vt:$src), (outs), (ins), [],
+  defm TEE_#rc : I<(outs rc:$res, rc:$also), (ins rc:$src), (outs), (ins), [],
                    "local.tee\t$res, $also, $src", "local.tee">;
 
   // This is the actual local.get instruction in wasm. These are made explicit
   // by the ExplicitLocals pass. It has mayLoad because it reads from a wasm
   // local, which is a side effect not otherwise modeled in LLVM.
   let mayLoad = 1, isAsCheapAsAMove = 1 in
-  defm LOCAL_GET_#vt : I<(outs vt:$res), (ins local_op:$local),
+  defm LOCAL_GET_#rc : I<(outs rc:$res), (ins local_op:$local),
                          (outs), (ins local_op:$local), [],
                          "local.get\t$res, $local", "local.get\t$local", 0x20>;
 
   // This is the actual local.set instruction in wasm. These are made explicit
   // by the ExplicitLocals pass. It has mayStore because it writes to a wasm
   // local, which is a side effect not otherwise modeled in LLVM.
   let mayStore = 1, isAsCheapAsAMove = 1 in
-  defm LOCAL_SET_#vt : I<(outs), (ins local_op:$local, vt:$src),
+  defm LOCAL_SET_#rc : I<(outs), (ins local_op:$local, rc:$src),
                          (outs), (ins local_op:$local), [],
                          "local.set\t$local, $src", "local.set\t$local", 0x21>;
 
   // This is the actual local.tee instruction in wasm. TEEs are turned into
   // LOCAL_TEEs by the ExplicitLocals pass. It has mayStore for the same reason
   // as LOCAL_SET.
   let mayStore = 1, isAsCheapAsAMove = 1 in
-  defm LOCAL_TEE_#vt : I<(outs vt:$res), (ins local_op:$local, vt:$src),
+  defm LOCAL_TEE_#rc : I<(outs rc:$res), (ins local_op:$local, rc:$src),
                          (outs), (ins local_op:$local), [],
                          "local.tee\t$res, $local, $src", "local.tee\t$local",
                          0x22>;
 
   // Unused values must be dropped in some contexts.
-  defm DROP_#vt : I<(outs), (ins vt:$src), (outs), (ins), [],
+  defm DROP_#rc : I<(outs), (ins rc:$src), (outs), (ins), [],
                     "drop\t$src", "drop", 0x1a>;
 
   let mayLoad = 1 in
-  defm GLOBAL_GET_#vt : I<(outs vt:$res), (ins global_op:$local),
-                          (outs), (ins global_op:$local), [],
-                          "global.get\t$res, $local", "global.get\t$local",
+  defm GLOBAL_GET_#rc : I<(outs rc:$res), (ins global_op:$addr),
+                          (outs), (ins global_op:$addr), [],
+                           "global.get\t$res, $addr", "global.get\t$addr",
                           0x23>;
 
   let mayStore = 1 in
-  defm GLOBAL_SET_#vt : I<(outs), (ins global_op:$local, vt:$src),
-                          (outs), (ins global_op:$local), [],
-                          "global.set\t$local, $src", "global.set\t$local",
+  defm GLOBAL_SET_#rc : I<(outs), (ins global_op:$addr, rc:$src),
+                          (outs), (ins global_op:$addr), [],
+                          "global.set\t$addr, $src", "global.set\t$addr",
                           0x24>;
 
   } // hasSideEffects = 0
+  foreach vt = rc.RegTypes in {
+    def : Pat<(vt (WebAssemblyglobal_get
+                   (WebAssemblywrapper tglobaladdr:$addr))),
+              (!cast<NI>("GLOBAL_GET_" # rc) tglobaladdr:$addr)>;
+    def : Pat<(WebAssemblyglobal_set
+               vt:$src, (WebAssemblywrapper tglobaladdr:$addr)),
+              (!cast<NI>("GLOBAL_SET_" # rc) tglobaladdr:$addr, vt:$src)>;
+  }
 }
 defm "" : LOCAL<I32, global_op32>;
 defm "" : LOCAL<I64, global_op64>;  // 64-bit only needed for pointers.
 defm "" : LOCAL<F32, global_op32>;
 defm "" : LOCAL<F64, global_op32>;
 defm "" : LOCAL<V128, global_op32>, Requires<[HasSIMD128]>;
 defm "" : LOCAL<FUNCREF, global_op32>, Requires<[HasReferenceTypes]>;
 defm "" : LOCAL<EXTERNREF, global_op32>, Requires<[HasReferenceTypes]>;

llvm/lib/Target/WebAssembly/WebAssemblyInstrRef.td

 // WebAssemblyInstrRef.td - WebAssembly reference type codegen --*- tablegen -*-
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// WebAssembly reference type operand codegen constructs.
 ///
 //===----------------------------------------------------------------------===//
 
-multiclass REF_I<WebAssemblyRegClass reg, ValueType vt> {
-  defm REF_NULL_#reg : I<(outs reg:$res), (ins HeapType:$heaptype),
-                         (outs), (ins HeapType:$heaptype),
-                         [],
-                         "ref.null\t$res, $heaptype",
-                         "ref.null\t$heaptype",
-                         0xd0>,
-                       Requires<[HasReferenceTypes]>;
-  defm SELECT_#reg: I<(outs reg:$dst), (ins reg:$lhs, reg:$rhs, I32:$cond),
-                      (outs), (ins),
-                      [(set reg:$dst,
-                        (select I32:$cond, reg:$lhs, reg:$rhs))],
-                      vt#".select\t$dst, $lhs, $rhs, $cond",
-                      vt#".select", 0x1b>,
-                    Requires<[HasReferenceTypes]>;
+multiclass REF_I<WebAssemblyRegClass rc, ValueType vt> {
+  defm REF_NULL_#rc : I<(outs rc:$res), (ins HeapType:$heaptype),
+  defm SELECT_#rc: I<(outs rc:$dst), (ins rc:$lhs, rc:$rhs, I32:$cond),
+                     [(set rc:$dst,
+                       (select I32:$cond, rc:$lhs, rc:$rhs))],
 }
 
 defm "" : REF_I<FUNCREF, funcref>;
 defm "" : REF_I<EXTERNREF, externref>;
 
-foreach reg = [FUNCREF, EXTERNREF] in {
-def : Pat<(select (i32 (setne I32:$cond, 0)), reg:$lhs, reg:$rhs),
-          (!cast<Instruction>("SELECT_"#reg) reg:$lhs, reg:$rhs, I32:$cond)>;
-def : Pat<(select (i32 (seteq I32:$cond, 0)), reg:$lhs, reg:$rhs),
-          (!cast<Instruction>("SELECT_"#reg) reg:$rhs, reg:$lhs, I32:$cond)>;
+foreach rc = [FUNCREF, EXTERNREF] in {
+def : Pat<(select (i32 (setne I32:$cond, 0)), rc:$lhs, rc:$rhs),
+          (!cast<Instruction>("SELECT_"#rc) rc:$lhs, rc:$rhs, I32:$cond)>;
+def : Pat<(select (i32 (seteq I32:$cond, 0)), rc:$lhs, rc:$rhs),
+          (!cast<Instruction>("SELECT_"#rc) rc:$rhs, rc:$lhs, I32:$cond)>;
 }

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

 
 /// Create an WebAssembly architecture model.
 ///
 WebAssemblyTargetMachine::WebAssemblyTargetMachine(
     const Target &T, const Triple &TT, StringRef CPU, StringRef FS,
     const TargetOptions &Options, Optional<Reloc::Model> RM,
     Optional<CodeModel::Model> CM, CodeGenOpt::Level OL, bool JIT)
     : LLVMTargetMachine(T,
-                        TT.isArch64Bit() ? "e-m:e-p:64:64-i64:64-n32:64-S128"
-                                         : "e-m:e-p:32:32-i64:64-n32:64-S128",
+                        TT.isArch64Bit()
+                            ? "e-m:e-p:64:64-i64:64-n32:64-S128-ni:1"
+                            : "e-m:e-p:32:32-i64:64-n32:64-S128-ni:1",
                         TT, CPU, FS, Options, getEffectiveRelocModel(RM, TT),
                         getEffectiveCodeModel(CM, CodeModel::Large), OL),
       TLOF(new WebAssemblyTargetObjectFile()) {
   // WebAssembly type-checks instructions, but a noreturn function with a return
   // type that doesn't match the context will cause a check failure. So we lower
   // LLVM 'unreachable' to ISD::TRAP and then lower that to WebAssembly's
   // 'unreachable' instructions which is meant for that case.
   this->Options.TrapUnreachable = true;

llvm/test/CodeGen/WebAssembly/global-get.ll

+; RUN: llc < %s --mtriple=wasm32-unknown-unknown -asm-verbose=false | FileCheck %s
+
+@i32_global = local_unnamed_addr addrspace(1) global i32 undef
+@i64_global = local_unnamed_addr addrspace(1) global i64 undef
+@f32_global = local_unnamed_addr addrspace(1) global float undef
+@f64_global = local_unnamed_addr addrspace(1) global double undef
+
+define i32 @return_i32_global() {
+; CHECK-LABEL: return_i32_global:
+; CHECK-NEXT: functype       return_i32_global () -> (i32)
+; CHECK-NEXT: global.get i32_global
+; CHECK-NEXT: end_function
+  %v = load i32, i32 addrspace(1)* @i32_global
+  ret i32 %v
+}
+
+define i64 @return_i64_global() {
+; CHECK-LABEL: return_i64_global:
+; CHECK-NEXT: functype       return_i64_global () -> (i64)
+; CHECK-NEXT: global.get i64_global
+; CHECK-NEXT: end_function
+  %v = load i64, i64 addrspace(1)* @i64_global
+  ret i64 %v
+}
+
+define float @return_f32_global() {
+; CHECK-LABEL: return_f32_global:
+; CHECK-NEXT: functype       return_f32_global () -> (f32)
+; CHECK-NEXT: global.get f32_global
+; CHECK-NEXT: end_function
+  %v = load float, float addrspace(1)* @f32_global
+  ret float %v
+}
+
+define double @return_f64_global() {
+; CHECK-LABEL: return_f64_global:
+; CHECK-NEXT: functype       return_f64_global () -> (f64)
+; CHECK-NEXT: global.get f64_global
+; CHECK-NEXT: end_function
+  %v = load double, double addrspace(1)* @f64_global
+  ret double %v
+}
+
+
+;; LLVM doesn't yet declare proper WebAssembly globals for these values,
+;; instead placing them in linear memory.  To fix in a followup.
+; FIXME-CHECK: .globl i32_global
+; FIXME-CHECK: .globaltype i32_global, i32
+; FIXME-CHECK: .globl i64_global
+; FIXME-CHECK: .globaltype i64_global, i64
+; FIXME-CHECK: .globl f32_global
+; FIXME-CHECK: .globaltype f32_global, f32
+; FIXME-CHECK: .globl f64_global
+; FIXME-CHECK: .globaltype f64_global, f64

llvm/test/CodeGen/WebAssembly/global-set.ll

+; RUN: llc --mtriple=wasm32-unknown-unknown -asm-verbose=false < %s | FileCheck %s
+
+@i32_global = local_unnamed_addr addrspace(1) global i32 undef
+@i64_global = local_unnamed_addr addrspace(1) global i64 undef
+@f32_global = local_unnamed_addr addrspace(1) global float undef
+@f64_global = local_unnamed_addr addrspace(1) global double undef
+
+define void @set_i32_global(i32 %v) {
+; CHECK-LABEL: set_i32_global:
+; CHECK-NEXT: functype       set_i32_global (i32) -> ()
+; CHECK-NEXT: local.get 0
+; CHECK-NEXT: global.set i32_global
+; CHECK-NEXT: end_function
+  store i32 %v, i32 addrspace(1)* @i32_global
+  ret void
+}
+
+define void @set_i64_global(i64 %v) {
+; CHECK-LABEL: set_i64_global:
+; CHECK-NEXT: functype       set_i64_global (i64) -> ()
+; CHECK-NEXT: local.get 0
+; CHECK-NEXT: global.set i64_global
+; CHECK-NEXT: end_function
+  store i64 %v, i64 addrspace(1)* @i64_global
+  ret void
+}
+
+define void @set_f32_global(float %v) {
+; CHECK-LABEL: set_f32_global:
+; CHECK-NEXT: functype       set_f32_global (f32) -> ()
+; CHECK-NEXT: local.get 0
+; CHECK-NEXT: global.set f32_global
+; CHECK-NEXT: end_function
+  store float %v, float addrspace(1)* @f32_global
+  ret void
+}
+
+define void @set_f64_global(double %v) {
+; CHECK-LABEL: set_f64_global:
+; CHECK-NEXT: functype       set_f64_global (f64) -> ()
+; CHECK-NEXT: local.get 0
+; CHECK-NEXT: global.set f64_global
+; CHECK-NEXT: end_function
+  store double %v, double addrspace(1)* @f64_global
+  ret void
+}
+
+;; LLVM doesn't yet declare proper WebAssembly globals for these values,
+;; instead placing them in linear memory.  To fix in a followup.
+; FIXME-CHECK: .globl i32_global
+; FIXME-CHECK: .globaltype i32_global, i32
+; FIXME-CHECK: .globl i64_global
+; FIXME-CHECK: .globaltype i64_global, i64
+; FIXME-CHECK: .globl f32_global
+; FIXME-CHECK: .globaltype f32_global, f32
+; FIXME-CHECK: .globl f64_global
+; FIXME-CHECK: .globaltype f64_global, f64