[WebAssmebly] Add support for defined wasm globals in MC and lld

[sbc100]

Summary

This change add supprot for defined wasm globas in the .s format
and in the MC layer.

Currently there is no support custom initialization and all wasm
globals are initialized to zero.

Fixes: PR45742

[aardappel]

lld/test/wasm/globals.s

6

I presume there's a reason or precedent why this is not .global ?

14

You were saying earlier there's a lot of boilerplate to writing a function in asm.. this seems pretty reasonable?

32

Are these labels required and if so, why?

llvm/lib/MC/WasmObjectWriter.cpp

797

So why is this section entirely new? Where did __stackpointer etc go previously?

llvm/test/MC/WebAssembly/globals.s

5

I guess having almost the same test in LLD and MC is unavoidable?

[sbc100]

lld/test/wasm/globals.s

6

I don't really know. It seems that both ".global" and ".globl" are accepted in the input but when .s format is written it always seems to write the shorter form, so I assumed it was preferred.

Indeed, looking at the code it looks like GlobalDirective in MCAsmInfo is always set to: "\t.globl\t", even though to parser accepts the long form.

14

You are right, its not too bad. I wonder if we could make end_function optional though? Also, IIRC that location of the .functype directive is tied (needlessly?) to this position at the start I think.

32

Yes, these actually define the symbols.

If you just say .globaltype bar_global, f32 you are saying there is some global with that name and type, but you are not defining it. Same with functions. Creating the label like this actually creates the thing.

In the next phase we should probably require some kind of initialization too. But that is not needed for this change.

llvm/lib/MC/WasmObjectWriter.cpp

797

Because __stack_pointer is always undefined right up until the linker synthesizes it.

llvm/test/MC/WebAssembly/globals.s

5

Yes, I think inputs will look similar for simple test cases.

[aardappel]

lld/test/wasm/globals.s

6

might be nice to standardize on .global then? up to you.

14

end_function is an actual instruction, not just a marker, so that's probably not a good idea.

[sbc100]

lld/test/wasm/globals.s

6

You mean .globl ? . it seem that that would be the logical choice since that is what MC always outputs.

14

Right.. but in that case maybe we should just call it end since that is the name of the instruction.

Also, this IIUC is not present at all in the way format.. so there is a precedent for not including this.

[aardappel]

lld/test/wasm/globals.s

14

We define end_block, end_loop etc. as distinct instructions in tablegen defs and thuout LLVM, they just map to the same opcode. The asm output is just a reflection of that.

[vitalybuka]

http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/40961/steps/check-llvm%20msan/logs/stdio

FAIL: LLVM :: ObjectYAML/wasm/global_section.yaml (28409 of 37227)
******************** TEST 'LLVM :: ObjectYAML/wasm/global_section.yaml' FAILED ********************
Script:
--
: 'RUN: at line 1';   /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/yaml2obj /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/ObjectYAML/wasm/global_section.yaml | /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/obj2yaml | /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/ObjectYAML/wasm/global_section.yaml
--
Exit Code: 2

Command Output (stderr):
--
YAML:13:28: error: invalid number
          Value:           -5
                           ^~
yaml2obj: error: failed to parse YAML input: Invalid argument
Error reading file: <stdin>: The file was not recognized as a valid object file
FileCheck error: '<stdin>' is empty.
FileCheck command line:  /b/sanitizer-x86_64-linux-fast/build/llvm_build_msan/bin/FileCheck /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/ObjectYAML/wasm/global_section.yaml

--

********************
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.. 
********************
Failing Tests (1):
  LLVM :: ObjectYAML/wasm/global_section.yaml


Testing Time: 192.29s

[vitalybuka]

This comment was removed by vitalybuka.

[sbc100]

Sorry for the breakage.. I have a simple fix in flight.

[sbc100]

Fixed in 65e64f6d652

lld/test/wasm/globals.s

+# RUN: llvm-mc -filetype=obj -triple=wasm32-unknown-unknown -o %t.o %s
+# RUN: wasm-ld %t.o -o %t.wasm
+# RUN: obj2yaml %t.wasm | FileCheck %s
+
+.globl _start
+.globl read_global
+.globl write_global
+
+.globaltype foo_global, i32
+.globaltype bar_global, f32
+
+read_global:
+  .functype read_global () -> (i32)
+  global.get foo_global
+  end_function
+
+write_global:
+  .functype write_global (i32) -> ()
+  local.get 0
+  global.set foo_global
+  f32.const 1.0
+  global.set bar_global
+  end_function
+
+_start:
+  .functype _start () -> ()
+  i32.const 1
+  call write_global
+  end_function
+
+foo_global:
+bar_global:
+
+# CHECK:       - Type:            GLOBAL
+# CHECK-NEXT:    Globals:
+# CHECK-NEXT:      - Index:           0
+# CHECK-NEXT:        Type:            I32
+# CHECK-NEXT:        Mutable:         true
+# CHECK-NEXT:        InitExpr:
+# CHECK-NEXT:          Opcode:          I32_CONST
+# CHECK-NEXT:          Value:           66560
+# CHECK-NEXT:      - Index:           1
+# CHECK-NEXT:        Type:            I32
+# CHECK-NEXT:        Mutable:         true
+# CHECK-NEXT:        InitExpr:
+# CHECK-NEXT:          Opcode:          I32_CONST
+# CHECK-NEXT:          Value:           0
+# CHECK-NEXT:      - Index:           2
+# CHECK-NEXT:        Type:            F32
+# CHECK-NEXT:        Mutable:         true
+# CHECK-NEXT:        InitExpr:
+# CHECK-NEXT:          Opcode:          F32_CONST
+# CHECK-NEXT:          Value:           0

lld/wasm/WriterUtils.cpp

void writeU8(raw_ostream &os, uint8_t byte, const Twine &msg) {
   os << byte;
 }
 
 void writeU32(raw_ostream &os, uint32_t number, const Twine &msg) {
   debugWrite(os.tell(), msg + "[0x" + utohexstr(number) + "]");
   support::endian::write(os, number, support::little);
 }
 
+void writeU64(raw_ostream &os, uint64_t number, const Twine &msg) {
+  debugWrite(os.tell(), msg + "[0x" + utohexstr(number) + "]");
+  support::endian::write(os, number, support::little);
+}
+
 void writeValueType(raw_ostream &os, ValType type, const Twine &msg) {
   writeU8(os, static_cast<uint8_t>(type),
           msg + "[type: " + toString(type) + "]");
 }
 
 void writeSig(raw_ostream &os, const WasmSignature &sig) {
   writeU8(os, WASM_TYPE_FUNC, "signature type");
   writeUleb128(os, sig.Params.size(), "param Count");
void writeInitExpr(raw_ostream &os, const WasmInitExpr &initExpr) {
   writeU8(os, initExpr.Opcode, "opcode");
   switch (initExpr.Opcode) {
   case WASM_OPCODE_I32_CONST:
     writeSleb128(os, initExpr.Value.Int32, "literal (i32)");
     break;
   case WASM_OPCODE_I64_CONST:
     writeSleb128(os, initExpr.Value.Int64, "literal (i64)");
     break;
+  case WASM_OPCODE_F32_CONST:
+    writeU32(os, initExpr.Value.Float32, "literal (f32)");
+    break;
+  case WASM_OPCODE_F64_CONST:
+    writeU64(os, initExpr.Value.Float64, "literal (f64)");
+    break;
   case WASM_OPCODE_GLOBAL_GET:
     writeUleb128(os, initExpr.Value.Global, "literal (global index)");
     break;
   default:
     fatal("unknown opcode in init expr: " + Twine(initExpr.Opcode));
   }
   writeU8(os, WASM_OPCODE_END, "opcode:end");
 }

llvm/include/llvm/BinaryFormat/Wasm.h

 struct WasmTable {
   uint8_t ElemType;
   WasmLimits Limits;
 };
 
 struct WasmInitExpr {
   uint8_t Opcode;
   union {
-    int32_t Int32;
-    int64_t Int64;
-    int32_t Float32;
-    int64_t Float64;
+    uint32_t Int32;
+    uint64_t Int64;
+    uint32_t Float32;
+    uint64_t Float64;
     uint32_t Global;
   } Value;
 };
 
 struct WasmGlobalType {
   uint8_t Type;
   bool Mutable;
 };

llvm/lib/MC/WasmObjectWriter.cpp

static void writePatchableSLEB(raw_pwrite_stream &Stream, int32_t X,
                                uint64_t Offset) {
   uint8_t Buffer[5];
   unsigned SizeLen = encodeSLEB128(X, Buffer, 5);
   assert(SizeLen == 5);
   Stream.pwrite((char *)Buffer, SizeLen, Offset);
 }
 
 // Write X as a plain integer value at offset Offset in Stream.
-static void writeI32(raw_pwrite_stream &Stream, uint32_t X, uint64_t Offset) {
+static void patchI32(raw_pwrite_stream &Stream, uint32_t X, uint64_t Offset) {
   uint8_t Buffer[4];
   support::endian::write32le(Buffer, X);
   Stream.pwrite((char *)Buffer, sizeof(Buffer), Offset);
 }
 
 class WasmObjectWriter : public MCObjectWriter {
   support::endian::Writer W;
 
private:
 
   uint64_t writeObject(MCAssembler &Asm, const MCAsmLayout &Layout) override;
 
   void writeString(const StringRef Str) {
     encodeULEB128(Str.size(), W.OS);
     W.OS << Str;
   }
 
+  void writeI32(int32_t val) {
+    char Buffer[4];
+    support::endian::write32le(Buffer, val);
+    W.OS.write(Buffer, sizeof(Buffer));
+  }
+
+  void writeI64(int64_t val) {
+    char Buffer[8];
+    support::endian::write64le(Buffer, val);
+    W.OS.write(Buffer, sizeof(Buffer));
+  }
+
   void writeValueType(wasm::ValType Ty) { W.OS << static_cast<char>(Ty); }
 
   void writeTypeSection(ArrayRef<WasmSignature> Signatures);
   void writeImportSection(ArrayRef<wasm::WasmImport> Imports, uint32_t DataSize,
                           uint32_t NumElements);
   void writeFunctionSection(ArrayRef<WasmFunction> Functions);
   void writeExportSection(ArrayRef<wasm::WasmExport> Exports);
   void writeElemSection(ArrayRef<uint32_t> TableElems);
   void writeDataCountSection();
   void writeCodeSection(const MCAssembler &Asm, const MCAsmLayout &Layout,
                         ArrayRef<WasmFunction> Functions);
   void writeDataSection();
   void writeEventSection(ArrayRef<wasm::WasmEventType> Events);
+  void writeGlobalSection(ArrayRef<wasm::WasmGlobal> Globals);
   void writeRelocSection(uint32_t SectionIndex, StringRef Name,
                          std::vector<WasmRelocationEntry> &Relocations);
   void writeLinkingMetaDataSection(
       ArrayRef<wasm::WasmSymbolInfo> SymbolInfos,
       ArrayRef<std::pair<uint16_t, uint32_t>> InitFuncs,
       const std::map<StringRef, std::vector<WasmComdatEntry>> &Comdats);
   void writeCustomSection(WasmCustomSection &CustomSection,
                           const MCAssembler &Asm, const MCAsmLayout &Layout);
for (const WasmRelocationEntry &RelEntry : Relocations) {
     case wasm::R_WASM_EVENT_INDEX_LEB:
       writePatchableLEB(Stream, Value, Offset);
       break;
     case wasm::R_WASM_TABLE_INDEX_I32:
     case wasm::R_WASM_MEMORY_ADDR_I32:
     case wasm::R_WASM_FUNCTION_OFFSET_I32:
     case wasm::R_WASM_SECTION_OFFSET_I32:
     case wasm::R_WASM_GLOBAL_INDEX_I32:
-      writeI32(Stream, Value, Offset);
+      patchI32(Stream, Value, Offset);
       break;
     case wasm::R_WASM_TABLE_INDEX_SLEB:
     case wasm::R_WASM_TABLE_INDEX_REL_SLEB:
     case wasm::R_WASM_MEMORY_ADDR_SLEB:
     case wasm::R_WASM_MEMORY_ADDR_REL_SLEB:
       writePatchableSLEB(Stream, Value, Offset);
       break;
     default:
void WasmObjectWriter::writeEventSection(ArrayRef<wasm::WasmEventType> Events) {
   for (const wasm::WasmEventType &Event : Events) {
     encodeULEB128(Event.Attribute, W.OS);
     encodeULEB128(Event.SigIndex, W.OS);
   }
 
   endSection(Section);
 }
 
+void WasmObjectWriter::writeGlobalSection(ArrayRef<wasm::WasmGlobal> Globals) {
+  if (Globals.empty())
+    return;
+
+  SectionBookkeeping Section;
+  startSection(Section, wasm::WASM_SEC_GLOBAL);
+
+  encodeULEB128(Globals.size(), W.OS);
+  for (const wasm::WasmGlobal &Global : Globals) {
+    encodeULEB128(Global.Type.Type, W.OS);
+    W.OS << char(Global.Type.Mutable);
+    W.OS << char(Global.InitExpr.Opcode);
+    switch (Global.Type.Type) {
+    case wasm::WASM_TYPE_I32:
+      encodeSLEB128(0, W.OS);
+      break;
+    case wasm::WASM_TYPE_I64:
+      encodeSLEB128(0, W.OS);
+      break;
+    case wasm::WASM_TYPE_F32:
+      writeI32(0);
+      break;
+    case wasm::WASM_TYPE_F64:
+      writeI64(0);
+      break;
+    default:
+      llvm_unreachable("unexpected type");
+    }
+    W.OS << char(wasm::WASM_OPCODE_END);
+  }
+
+  endSection(Section);
+}
+
 void WasmObjectWriter::writeExportSection(ArrayRef<wasm::WasmExport> Exports) {
   if (Exports.empty())
     return;
 
   SectionBookkeeping Section;
   startSection(Section, wasm::WASM_SEC_EXPORT);
 
   encodeULEB128(Exports.size(), W.OS);
uint64_t WasmObjectWriter::writeObject(MCAssembler &Asm,
   LLVM_DEBUG(dbgs() << "WasmObjectWriter::writeObject\n");
 
   // Collect information from the available symbols.
   SmallVector<WasmFunction, 4> Functions;
   SmallVector<uint32_t, 4> TableElems;
   SmallVector<wasm::WasmImport, 4> Imports;
   SmallVector<wasm::WasmExport, 4> Exports;
   SmallVector<wasm::WasmEventType, 1> Events;
+  SmallVector<wasm::WasmGlobal, 1> Globals;
   SmallVector<wasm::WasmSymbolInfo, 4> SymbolInfos;
   SmallVector<std::pair<uint16_t, uint32_t>, 2> InitFuncs;
   std::map<StringRef, std::vector<WasmComdatEntry>> Comdats;
   uint32_t DataSize = 0;
 
   // For now, always emit the memory import, since loads and stores are not
   // valid without it. In the future, we could perhaps be more clever and omit
   // it if there are no loads or stores.
if (WS.isFunction()) {
           DataSection.getSegmentIndex(),
           static_cast<uint32_t>(Layout.getSymbolOffset(WS)),
           static_cast<uint32_t>(Size)};
       DataLocations[&WS] = Ref;
       LLVM_DEBUG(dbgs() << "  -> segment index: " << Ref.Segment << "\n");
 
     } else if (WS.isGlobal()) {
       // A "true" Wasm global (currently just __stack_pointer)
-      if (WS.isDefined())
-        report_fatal_error("don't yet support defined globals");
-
+      if (WS.isDefined()) {
+        assert(WasmIndices.count(&WS) == 0);
+        wasm::WasmGlobal Global;
+        Global.Type = WS.getGlobalType();
+        Global.Index = NumGlobalImports + Globals.size();
+        switch (Global.Type.Type) {
+        case wasm::WASM_TYPE_I32:
+          Global.InitExpr.Opcode = wasm::WASM_OPCODE_I32_CONST;
+          break;
+        case wasm::WASM_TYPE_I64:
+          Global.InitExpr.Opcode = wasm::WASM_OPCODE_I64_CONST;
+          break;
+        case wasm::WASM_TYPE_F32:
+          Global.InitExpr.Opcode = wasm::WASM_OPCODE_F32_CONST;
+          break;
+        case wasm::WASM_TYPE_F64:
+          Global.InitExpr.Opcode = wasm::WASM_OPCODE_F64_CONST;
+          break;
+        default:
+          llvm_unreachable("unexpected type");
+        }
+        WasmIndices[&WS] = Global.Index;
+        Globals.push_back(Global);
+      } else {
         // An import; the index was assigned above
         LLVM_DEBUG(dbgs() << "  -> global index: "
                           << WasmIndices.find(&WS)->second << "\n");
-
+      }
     } else if (WS.isEvent()) {
       // C++ exception symbol (__cpp_exception)
       unsigned Index;
       if (WS.isDefined()) {
+        assert(WasmIndices.count(&WS) == 0);
         Index = NumEventImports + Events.size();
         wasm::WasmEventType Event;
         Event.SigIndex = getEventType(WS);
         Event.Attribute = wasm::WASM_EVENT_ATTRIBUTE_EXCEPTION;
-        assert(WasmIndices.count(&WS) == 0);
         WasmIndices[&WS] = Index;
         Events.push_back(Event);
       } else {
         // An import; the index was assigned above.
         assert(WasmIndices.count(&WS) > 0);
       }
       LLVM_DEBUG(dbgs() << "  -> event index: " << WasmIndices.find(&WS)->second
                         << "\n");
uint64_t WasmObjectWriter::writeObject(MCAssembler &Asm,
   writeHeader(Asm);
 
   writeTypeSection(Signatures);
   writeImportSection(Imports, DataSize, TableElems.size());
   writeFunctionSection(Functions);
   // Skip the "table" section; we import the table instead.
   // Skip the "memory" section; we import the memory instead.
   writeEventSection(Events);
+  writeGlobalSection(Globals);
   writeExportSection(Exports);
   writeElemSection(TableElems);
   writeDataCountSection();
   writeCodeSection(Asm, Layout, Functions);
   writeDataSection();
   for (auto &CustomSection : CustomSections)
     writeCustomSection(CustomSection, Asm, Layout);
   writeLinkingMetaDataSection(SymbolInfos, InitFuncs, Comdats);

llvm/test/MC/WebAssembly/globals.s

+# RUN: llvm-mc -triple=wasm32-unknown-unknown < %s | FileCheck %s
+# RUN: llvm-mc -triple=wasm32-unknown-unknown -filetype=obj < %s | obj2yaml | FileCheck -check-prefix=BIN %s
+
+# Tests creating an accessing actual wasm globals
+
+.globl read_global
+.globl write_global
+.globaltype foo_global, i32
+.globaltype global2, i64
+.globaltype global3, f32
+.globaltype global4, f64
+
+read_global:
+  .functype read_global () -> (i32)
+  global.get foo_global
+  end_function
+
+write_global:
+  .functype write_global (i32) -> ()
+  local.get 0
+  global.set foo_global
+  global.set global2
+  global.set global3
+  global.set global4
+  end_function
+
+foo_global:
+global2:
+global3:
+global4:
+
+# CHECK: .globl  read_global
+# CNEXT: .globl  write_global
+# CHECK: .globaltype foo_global, i32
+# CHECK: foo_global:
+
+#      BIN: - Type:            GLOBAL
+# BIN-NEXT:   Globals:
+# BIN-NEXT:     - Index:           0
+# BIN-NEXT:       Type:            I32
+# BIN-NEXT:       Mutable:         true
+# BIN-NEXT:       InitExpr:
+# BIN-NEXT:         Opcode:          I32_CONST
+# BIN-NEXT:         Value:           0
+
+#      BIN:  - Type:            CUSTOM
+# BIN-NEXT:    Name:            linking
+# BIN-NEXT:    Version:         2
+# BIN-NEXT:    SymbolTable:
+# BIN-NEXT:      - Index:           0
+# BIN-NEXT:        Kind:            FUNCTION
+# BIN-NEXT:        Name:            read_global
+# BIN-NEXT:        Flags:           [  ]
+# BIN-NEXT:        Function:        0
+# BIN-NEXT:      - Index:           1
+# BIN-NEXT:        Kind:            FUNCTION
+# BIN-NEXT:        Name:            write_global
+# BIN-NEXT:        Flags:           [  ]
+# BIN-NEXT:        Function:        1
+# BIN-NEXT:      - Index:           2
+# BIN-NEXT:        Kind:            GLOBAL
+# BIN-NEXT:        Name:            foo_global
+# BIN-NEXT:        Flags:           [ BINDING_LOCAL ]
+# BIN-NEXT:        Global:          0
+# BIN-NEXT:      - Index:           3
+# BIN-NEXT:        Kind:            GLOBAL
+# BIN-NEXT:        Name:            global2
+# BIN-NEXT:        Flags:           [ BINDING_LOCAL ]
+# BIN-NEXT:        Global:          1
+# BIN-NEXT:      - Index:           4
+# BIN-NEXT:        Kind:            GLOBAL
+# BIN-NEXT:        Name:            global3
+# BIN-NEXT:        Flags:           [ BINDING_LOCAL ]
+# BIN-NEXT:        Global:          2
+# BIN-NEXT:      - Index:           5
+# BIN-NEXT:        Kind:            GLOBAL
+# BIN-NEXT:        Name:            global4
+# BIN-NEXT:        Flags:           [ BINDING_LOCAL ]
+# BIN-NEXT:        Global:          3