[WebAssembly] Implement @llvm.global_ctors and @llvm.global_dtors

[sunfish]

Summary

[UPDATE] This patch

• lowers @llvm.global_dtors by adding @llvm.global_ctors functions which register the destructors with __cxa_atexit.
• impements @llvm.global_ctors with wasm start functions and linker metadata

See here for more background.

[sunfish]

Updated patch:

• optional __dso_handle support
• optional comdat support
• tests
• better naming

[sunfish]

This update adds @llvm.global_ctors support to the wasm object writer, as layed out here, including init function priorities as sketched here.

[sbc100]

How would you feel about landing a simplified version of this change that doesn't use the start section (i.e. always put start functions in the metadata)? Seems a little strange to special case the single start function. Also, I like the idea of letting the linker decide if/when to switch to using the start section (for now anyway).

[sbc100]

I'd like to land this soon if possible (sans the start section if possible) so I start working on the lld side.

[sunfish]

Omitting the start section part for now is ok with me. It should be sufficient to just remove the Priority == UINT16_MAX special cases.

I'm also adding comments for a few minor issues in the patch that I thought of. The COMDAT thing would be nice to do now so that we don't have to coordinate adding it in the future.

I won't be able to land anything this week, so I'm ok if you want to land this without the start section parts right now.

lib/MC/WasmObjectWriter.cpp

966 ↗

(On Diff #125936)

We should reserve room here for an optional COMDAT symbol too. For now, it's probably sufficient to just stub it out with a encodeULEB128(0, getStream()).

test/CodeGen/WebAssembly/lower-global-dtors.ll

17

This should be named orig_dtor65535.

[ncw]

lib/MC/WasmObjectWriter.cpp

1378 ↗

(On Diff #125936)

I think these changes are good, my stuff for .init_array support is obsolete really and I'll withdraw that change request.

It is a bit surprising (to me) that the .o files have a start function. If I'm reading this correctly, is it arbitrarily putting one of the constructors as the start function, and the rest as "extra start functions" in the linking data? That's oddly non-uniform, surely at the very least all of them should go in the linking data, rather than storing the first one somewhere else.

[sbc100]

lib/MC/WasmObjectWriter.cpp

1378 ↗

(On Diff #125936)

Yes, that was my feedback too. Especially since we are probably not going to be using the start section at all for the time being. So I'm proposing the land a version of this without the use of the start section (at least initially).

I was thinking of trying to land this today by forking this change, removing the start section and add a obj<->yaml test for these.

[ncw]

I've implemented the Musl side of this change. It was pretty simple really, I've just added the following to crt1.o, which is pulled in by the default link-line:

extern void __init_libc(char **envp, char *pn);

__attribute__((constructor(1)))
void __libc_ctor()
{
        __init_libc((char**)&__wasm_init_data.envp0, (char*)__wasm_init_data.argv0);
}

It sounds like Sam is working on the LLD-side of this change, to actually use the init-data section to synthesise an entrypoint. I look forward to testing with it!

lib/MC/WasmObjectWriter.cpp

1350 ↗

(On Diff #125936)

Using UINT16_MAX as the default priority for constructors has always been a bit imperfect.

I don't suppose it could be changed? (Here and previously in TargetLoweringObjectFileWasm::getStaticCtorSection, where the default priority has already been assigned.) It might just be something we have to live with, but it's a longstanding gripe against GCC/ELF that you can't designate code to run after the default priority.

The default should be UINT16_MAX/2, nicely in the middle of the allowed range. I don't really have an urgent use-case currently for this change, but I know that people do find it frustrating sometimes. I was wondering whether we could avoid repeating GCC's mistake for a new platform like Wasm

[sbc100]

I landed a modified version of this change in rL320774. I assume its OK to close this now.

lib/Target/WebAssembly/CMakeLists.txt

add_llvm_target(WebAssemblyCodeGen
   WebAssemblyFixIrreducibleControlFlow.cpp
   WebAssemblyFixFunctionBitcasts.cpp
   WebAssemblyFrameLowering.cpp
   WebAssemblyISelDAGToDAG.cpp
   WebAssemblyISelLowering.cpp
   WebAssemblyInstrInfo.cpp
   WebAssemblyLowerBrUnless.cpp
   WebAssemblyLowerEmscriptenEHSjLj.cpp
+  WebAssemblyLowerGlobalDtors.cpp
   WebAssemblyMachineFunctionInfo.cpp
   WebAssemblyMCInstLower.cpp
   WebAssemblyOptimizeLiveIntervals.cpp
   WebAssemblyOptimizeReturned.cpp
   WebAssemblyPeephole.cpp
   WebAssemblyPrepareForLiveIntervals.cpp
   WebAssemblyRegisterInfo.cpp
   WebAssemblyRegColoring.cpp

lib/Target/WebAssembly/WebAssembly.h

 
 class WebAssemblyTargetMachine;
 class ModulePass;
 class FunctionPass;
 
 // LLVM IR passes.
 ModulePass *createWebAssemblyLowerEmscriptenEHSjLj(bool DoEH, bool DoSjLj);
 void initializeWebAssemblyLowerEmscriptenEHSjLjPass(PassRegistry &);
+ModulePass *createWebAssemblyLowerGlobalDtors();
 ModulePass *createWebAssemblyFixFunctionBitcasts();
 FunctionPass *createWebAssemblyOptimizeReturned();
 
 // ISel and immediate followup passes.
 FunctionPass *createWebAssemblyISelDag(WebAssemblyTargetMachine &TM,
                                        CodeGenOpt::Level OptLevel);
 FunctionPass *createWebAssemblyArgumentMove();
 FunctionPass *createWebAssemblySetP2AlignOperands();

lib/Target/WebAssembly/WebAssemblyLowerGlobalDtors.cpp

+//===-- WebAssemblyLowerGlobalDtors.cpp - Lower @llvm.global_dtors --------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// \brief Lower @llvm.global_dtors.
+///
+/// WebAssembly doesn't have a builtin way to invoke static destructors.
+/// Implement @llvm.global_dtors by creating wrapper functions that are
+/// registered in @llvm.global_ctors and which contain a call to
+/// `__cxa_atexit` to register their destructor functions.
+///
+//===----------------------------------------------------------------------===//
+
+#include "WebAssembly.h"
+#include "llvm/IR/Constants.h"
+#include "llvm/IR/Instructions.h"
+#include "llvm/IR/Intrinsics.h"
+#include "llvm/IR/Module.h"
+#include "llvm/Transforms/Utils/ModuleUtils.h"
+#include "llvm/Pass.h"
+#include "llvm/ADT/MapVector.h"
+#include "llvm/Support/Debug.h"
+#include "llvm/Support/raw_ostream.h"
+using namespace llvm;
+
+#define DEBUG_TYPE "wasm-lower-global-dtors"
+
+namespace {
+class LowerGlobalDtors final : public ModulePass {
+  StringRef getPassName() const override {
+    return "WebAssembly Lower @llvm.global_dtors";
+  }
+
+  void getAnalysisUsage(AnalysisUsage &AU) const override {
+    AU.setPreservesCFG();
+    ModulePass::getAnalysisUsage(AU);
+  }
+
+  bool runOnModule(Module &M) override;
+
+public:
+  static char ID;
+  LowerGlobalDtors() : ModulePass(ID) {}
+};
+} // End anonymous namespace
+
+char LowerGlobalDtors::ID = 0;
+ModulePass *llvm::createWebAssemblyLowerGlobalDtors() {
+  return new LowerGlobalDtors();
+}
+
+bool LowerGlobalDtors::runOnModule(Module &M) {
+  GlobalVariable *GV = M.getGlobalVariable("llvm.global_dtors");
+  if (!GV)
+    return false;
+
+  const ConstantArray *InitList = dyn_cast<ConstantArray>(GV->getInitializer());
+  if (!InitList)
+    return false;
+
+  // Sanity-check @llvm.global_dtor's type.
+  StructType *ETy = dyn_cast<StructType>(InitList->getType()->getElementType());
+  if (!ETy || ETy->getNumElements() != 3 ||
+      !ETy->getTypeAtIndex(0U)->isIntegerTy() ||
+      !ETy->getTypeAtIndex(1U)->isPointerTy() ||
+      !ETy->getTypeAtIndex(2U)->isPointerTy())
+    return false; // Not (int, ptr, ptr).
+
+  // Collect the contents of @llvm.global_dtors, collated by priority and
+  // associated symbol.
+  std::map<uint16_t, MapVector<Constant *, std::vector<Constant *> > > DtorFuncs;
+  for (Value *O : InitList->operands()) {
+    ConstantStruct *CS = dyn_cast<ConstantStruct>(O);
+    if (!CS) continue; // Malformed.
+
+    ConstantInt *Priority = dyn_cast<ConstantInt>(CS->getOperand(0));
+    if (!Priority) continue; // Malformed.
+    uint16_t PriorityValue = Priority->getLimitedValue(UINT16_MAX);
+
+    Constant *DtorFunc = CS->getOperand(1);
+    if (DtorFunc->isNullValue())
+      break;  // Found a null terminator, skip the rest.
+
+    Constant *Associated = CS->getOperand(2);
+    Associated = cast<Constant>(Associated->stripPointerCastsNoFollowAliases());
+
+    DtorFuncs[PriorityValue][Associated].push_back(DtorFunc);
+  }
+  if (DtorFuncs.empty())
+    return false;
+
+  // extern "C" int __cxa_atexit(void (*f)(void *), void *p, void *d);
+  LLVMContext &C = M.getContext();
+  PointerType *VoidStar = Type::getInt8PtrTy(C);
+  Type *AtExitFuncArgs[] = { VoidStar };
+  FunctionType *AtExitFuncTy = FunctionType::get(
+          Type::getVoidTy(C),
+          AtExitFuncArgs,
+          /*isVarArg=*/false);
+
+  Type *AtExitArgs[] = {
+    PointerType::get(AtExitFuncTy, 0),
+    VoidStar,
+    VoidStar
+  };
+  FunctionType *AtExitTy = FunctionType::get(
+          Type::getInt32Ty(C),
+          AtExitArgs,
+          /*isVarArg=*/false);
+  Constant *AtExit = M.getOrInsertFunction("__cxa_atexit", AtExitTy);
+
+  // Declare __dso_local.
+  Constant *DsoHandle = M.getNamedValue("__dso_handle");
+  if (!DsoHandle) {
+    Type *DsoHandleTy = Type::getInt8Ty(C);
+    GlobalVariable *Handle =
+        new GlobalVariable(M, DsoHandleTy, /*isConstant=*/true,
+                           GlobalVariable::ExternalWeakLinkage,
+                           nullptr, "__dso_handle");
+    Handle->setVisibility(GlobalVariable::HiddenVisibility);
+    DsoHandle = Handle;
+  }
+
+  // For each unique priority level and associated symbol, generate a function
+  // to call all the destructors at that level, and a function to register the
+  // first function with __cxa_atexit.
+  for (auto &PriorityAndMore : DtorFuncs) {
+    uint16_t Priority = PriorityAndMore.first;
+    for (auto &AssociatedAndMore : PriorityAndMore.second) {
+      Constant *Associated = AssociatedAndMore.first;
+
+      Function *CallDtors = Function::Create(
+              AtExitFuncTy, Function::PrivateLinkage,
+              "call_dtors." + Twine(Priority) +
+              (!Associated->isNullValue() ?
+                 (Twine(".") + Associated->getName()) : Twine()),
+              &M);
+      BasicBlock *BB = BasicBlock::Create(C, "body", CallDtors);
+
+      for (auto Dtor : AssociatedAndMore.second)
+        CallInst::Create(Dtor, "", BB);
+      ReturnInst::Create(C, BB);
+
+      FunctionType *VoidVoid = FunctionType::get(Type::getVoidTy(C),
+                                                 /*isVarArg=*/false);
+      Function *RegisterCallDtors = Function::Create(
+              VoidVoid, Function::PrivateLinkage,
+              "register_call_dtors." + Twine(Priority) +
+              (!Associated->isNullValue() ?
+                 (Twine(".") + Associated->getName()) : Twine()),
+              &M);
+      BasicBlock *EntryBB = BasicBlock::Create(C, "body", RegisterCallDtors);
+      BasicBlock *FailBB = BasicBlock::Create(C, "fail", RegisterCallDtors);
+      BasicBlock *RetBB = BasicBlock::Create(C, "return", RegisterCallDtors);
+
+      Value *Null = ConstantPointerNull::get(VoidStar);
+      Value *Args[] = { CallDtors, Null, DsoHandle };
+      Value *Res = CallInst::Create(AtExit, Args, "call", EntryBB);
+      Value *Cmp = new ICmpInst(*EntryBB, ICmpInst::ICMP_NE, Res,
+                                Constant::getNullValue(Res->getType()));
+      BranchInst::Create(FailBB, RetBB, Cmp, EntryBB);
+
+      // If `__cxa_atexit` hits out-of-memory, trap, so that we don't misbehave.
+      // This should be very rare, because if the process is running out of memory
+      // before main has even started, something is wrong.
+      CallInst::Create(Intrinsic::getDeclaration(&M, Intrinsic::trap),
+                       "", FailBB);
+      new UnreachableInst(C, FailBB);
+
+      ReturnInst::Create(C, RetBB);
+
+      // Now register the registration function with @llvm.global_ctors.
+      appendToGlobalCtors(M, RegisterCallDtors, Priority, Associated);
+    }
+  }
+
+  // Now that we've lowered everything, remove @llvm.global_dtors.
+  GV->eraseFromParent();
+
+  return true;
+}

lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

void WebAssemblyPassConfig::addIRPasses() {
   if (TM->Options.ThreadModel == ThreadModel::Single)
     // In "single" mode, atomics get lowered to non-atomics.
     addPass(createLowerAtomicPass());
   else
     // Expand some atomic operations. WebAssemblyTargetLowering has hooks which
     // control specifically what gets lowered.
     addPass(createAtomicExpandPass());
 
+  // Lower .llvm.global_dtors into .llvm_global_ctors with __cxa_atexit calls.
+  addPass(createWebAssemblyLowerGlobalDtors());
+
   // Fix function bitcasts, as WebAssembly requires caller and callee signatures
   // to match.
   addPass(createWebAssemblyFixFunctionBitcasts());
 
   // Optimize "returned" function attributes.
   if (getOptLevel() != CodeGenOpt::None)
     addPass(createWebAssemblyOptimizeReturned());
 

test/CodeGen/WebAssembly/lower-global-dtors.ll

+; RUN: llc < %s -asm-verbose=false | FileCheck --check-prefix=CHECK --check-prefix=FINI --check-prefix=NULL %s
+
+target datalayout = "e-m:e-p:32:32-i64:64-n32:64-S128"
+target triple = "wasm32-unknown-unknown-wasm"
+
+; Test that @llvm.global_dtors is properly lowered into @llvm.global_ctors,
+; grouping dtor calls by priority and associated symbol.
+
+declare void @orig_ctor()
+declare void @orig_dtor0()
+declare void @orig_dtor1a()
+declare void @orig_dtor1b()
+declare void @orig_dtor1c0()
+declare void @orig_dtor1c1a()
+declare void @orig_dtor1c1b()
+declare void @orig_dtor65536()
+declare void @after_the_null()
+
+@associated1c0 = external global i8
+@associated1c1 = external global i8
+
[e-mail]_ctors = appending global
+[1 x { i32, void ()*, i8* }]
+[
+  { i32, void ()*, i8* } { i32 200, void ()* @orig_ctor, i8* null }
+]
+
[e-mail]_dtors = appending global
+[9 x { i32, void ()*, i8* }]
+[
+  { i32, void ()*, i8* } { i32 0, void ()* @orig_dtor0, i8* null },
+  { i32, void ()*, i8* } { i32 1, void ()* @orig_dtor1a, i8* null },
+  { i32, void ()*, i8* } { i32 1, void ()* @orig_dtor1b, i8* null },
+  { i32, void ()*, i8* } { i32 1, void ()* @orig_dtor1c0, i8* @associated1c0 },
+  { i32, void ()*, i8* } { i32 1, void ()* @orig_dtor1c1a, i8* @associated1c1 },
+  { i32, void ()*, i8* } { i32 1, void ()* @orig_dtor1c1b, i8* @associated1c1 },
+  { i32, void ()*, i8* } { i32 65535, void ()* @orig_dtor65536, i8* null },
+  { i32, void ()*, i8* } { i32 65535, void ()* null, i8* null },
+  { i32, void ()*, i8* } { i32 65535, void ()* @after_the_null, i8* null }
+]
+
+; CHECK-LABEL: .Lcall_dtors.0:
+; CHECK-NEXT: .param          i32{{$}}
+; CHECK-NEXT: call            orig_dtor0@FUNCTION{{$}}
+
+; CHECK-LABEL: .Lregister_call_dtors.0:
+; CHECK-NEXT: block
+; CHECK-NEXT: i32.const       $push2=, .Lcall_dtors.0@FUNCTION{{$}}
+; CHECK-NEXT: i32.const       $push1=, 0
+; CHECK-NEXT: i32.const       $push0=, __dso_handle
+; CHECK-NEXT: i32.call        $push3=, __cxa_atexit@FUNCTION, $pop2, $pop1, $pop0{{$}}
+; CHECK-NEXT: br_if           0, $pop3
+; CHECK-NEXT: return
+;      CHECK: end_block
+; CHECK-NEXT: unreachable
+
+; CHECK-LABEL: .Lcall_dtors.1:
+; CHECK-NEXT: .param          i32{{$}}
+; CHECK-NEXT: call            orig_dtor1a@FUNCTION{{$}}
+; CHECK-NEXT: call            orig_dtor1b@FUNCTION{{$}}
+
+; CHECK-LABEL: .Lregister_call_dtors.1:
+; CHECK-NEXT: block
+; CHECK-NEXT: i32.const       $push2=, .Lcall_dtors.1@FUNCTION{{$}}
+; CHECK-NEXT: i32.const       $push1=, 0
+; CHECK-NEXT: i32.const       $push0=, __dso_handle
+; CHECK-NEXT: i32.call        $push3=, __cxa_atexit@FUNCTION, $pop2, $pop1, $pop0{{$}}
+; CHECK-NEXT: br_if           0, $pop3
+; CHECK-NEXT: return
+;      CHECK: end_block
+; CHECK-NEXT: unreachable
+
+; CHECK-LABEL: .Lcall_dtors.1.associated1c0:
+; CHECK-NEXT: .param          i32{{$}}
+; CHECK-NEXT: call            orig_dtor1c0@FUNCTION{{$}}
+
+; CHECK-LABEL: .Lregister_call_dtors.1.associated1c0:
+; CHECK-NEXT: block
+; CHECK-NEXT: i32.const       $push2=, .Lcall_dtors.1.associated1c0@FUNCTION{{$}}
+; CHECK-NEXT: i32.const       $push1=, 0
+; CHECK-NEXT: i32.const       $push0=, __dso_handle
+; CHECK-NEXT: i32.call        $push3=, __cxa_atexit@FUNCTION, $pop2, $pop1, $pop0{{$}}
+; CHECK-NEXT: br_if           0, $pop3
+; CHECK-NEXT: return
+;      CHECK: end_block
+; CHECK-NEXT: unreachable
+
+; CHECK-LABEL: .Lcall_dtors.1.associated1c1:
+; CHECK-NEXT: .param          i32{{$}}
+; CHECK-NEXT: call            orig_dtor1c1a@FUNCTION{{$}}
+; CHECK-NEXT: call            orig_dtor1c1b@FUNCTION{{$}}
+
+; CHECK-LABEL: .Lregister_call_dtors.1.associated1c1:
+; CHECK-NEXT: block
+; CHECK-NEXT: i32.const       $push2=, .Lcall_dtors.1.associated1c1@FUNCTION{{$}}
+; CHECK-NEXT: i32.const       $push1=, 0
+; CHECK-NEXT: i32.const       $push0=, __dso_handle
+; CHECK-NEXT: i32.call        $push3=, __cxa_atexit@FUNCTION, $pop2, $pop1, $pop0{{$}}
+; CHECK-NEXT: br_if           0, $pop3
+; CHECK-NEXT: return
+;      CHECK: end_block
+; CHECK-NEXT: unreachable
+
+; CHECK-LABEL: .Lcall_dtors.65535:
+; CHECK-NEXT: .param          i32{{$}}
+; CHECK-NEXT: call            orig_dtor65536@FUNCTION{{$}}
+
+; CHECK-LABEL: .Lregister_call_dtors.65535:
+; CHECK-NEXT: block
+; CHECK-NEXT: i32.const       $push2=, .Lcall_dtors.65535@FUNCTION{{$}}
+; CHECK-NEXT: i32.const       $push1=, 0
+; CHECK-NEXT: i32.const       $push0=, __dso_handle
+; CHECK-NEXT: i32.call        $push3=, __cxa_atexit@FUNCTION, $pop2, $pop1, $pop0{{$}}
+; CHECK-NEXT: br_if           0, $pop3
+; CHECK-NEXT: return
+;      CHECK: end_block
+; CHECK-NEXT: unreachable
+
+; CHECK-LABEL: .section .init_array,"",@
+;      CHECK: .int32  .Lregister_call_dtors.0@FUNCTION{{$}}
+; CHECK-NEXT: .int32  .Lregister_call_dtors.1@FUNCTION{{$}}
+; CHECK-NEXT: .int32  orig_ctor@FUNCTION{{$}}
+; CHECK-NEXT: .int32  .Lregister_call_dtors.65535@FUNCTION{{$}}
+
+; CHECK-LABEL: .weak __dso_handle
+
+; CHECK-LABEL: .functype __cxa_atexit, i32, i32, i32, i32{{$}}
+
+; We shouldn't make use of a .fini_array section.
+
+; FINI-NOT: fini_array
+
+; This function is listed after the null terminator, so it should
+; be excluded.
+
+; NULL-NOT: after_the_null