[lld][WebAssembly] Add support for `--export-dynamic-symbol`

[sbc100]

Summary

Unlike the existing --export option this will not causes errors
or warnings if the specified symbol is not defined.

This is an existing flag from GNU ld and lld ELF backend.

See: https://github.com/emscripten-core/emscripten/issues/13736

[MaskRay]

lld/wasm/Options.td

59

The ELF specific comment needs rework.

[sbc100]

• rebase

[dschuff]

LGTM modulo MaskRay's comment about the help text. Also I think @sunfish had a question earlier today that we didn't get to talk about.

[sbc100]

Feedback

[MaskRay]

lld/wasm/Options.td

58

Drop period.

[sbc100]

• feedback

[sbc100]

@sunfish any objections to landing this?

[sunfish]

I'm a little confused by the name "export-dynamic-symbol". I understand it's an ELF flag, though it seems to have a slightly different meaning here. What would you think about adding a warning when using this flag is -experimental-pic isn't passed, similar to what we do with -shared?

[sbc100]

In D101271#2723352, @sunfish wrote:

I'm a little confused by the name "export-dynamic-symbol". I understand it's an ELF flag, though it seems to have a slightly different meaning here. What would you think about adding a warning when using this flag is -experimental-pic isn't passed, similar to what we do with -shared?

This flag, although it has dynamic its is name, does not imply PIC ABI stuff or imply -shared or -pie so I don't think it makes sense to link it to -experimental-pic. In works just like the existing --export flag but allows the symbol to be undefined. I plan to use this in emscripten when we are doing static linking.

[sbc100]

The existing --export flag, one could argue, already promotes symbols from the static world to the dyanmic (runtime) world, its just that it also implies --require-defined=foo. In fact one could potentially argue that --export should be deprecated in favor of --require-defined= + --export-dynamic-symbol= (although I'm not doing that).

My original version of this patch used the name --export-if-defined=.. if you feel strongly we could go back to that name instead, but I like sharing flags where possible rather than inventing new ones, and the meaning of this flag seems to match pretty well here.

[sunfish]

Looking at the ld documentation for export-dynamic-symbol:

When creating a dynamically linked executable, symbols matching glob will be added to the dynamic symbol table.

We're applying this to statically linked executables too.

When creating a shared library, references to symbols matching glob will not be bound to the definitions within the shared library.

We don't have the semantics of allowing symbols to be preempted like ELF does.

I'm looking ahead to wasm-ld supporting new dynamic linking ABIs, which may be significantly different from ELF. So my preference here would be, if we have functionality which exactly matches ELF, it makes sense to reuse the ELF-specific names, otherwise we should use names which make sense for wasm on their own terms. Does that make sense?

[sbc100]

In D101271#2723568, @sunfish wrote:

Looking at the ld documentation for export-dynamic-symbol:

When creating a dynamically linked executable, symbols matching glob will be added to the dynamic symbol table.

We're applying this to statically linked executables too.

When creating a shared library, references to symbols matching glob will not be bound to the definitions within the shared library.

We don't have the semantics of allowing symbols to be preempted like ELF does.

I'm looking ahead to wasm-ld supporting new dynamic linking ABIs, which may be significantly different from ELF. So my preference here would be, if we have functionality which exactly matches ELF, it makes sense to reuse the ELF-specific names, otherwise we should use names which make sense for wasm on their own terms. Does that make sense?

So you think it would be preferable to go with --export-if-defined then, or something wasm specific?

[sunfish]

In D101271#2724145, @sbc100 wrote:

So you think it would be preferable to go with --export-if-defined then, or something wasm specific?

Yes, --export-if-defined would be my preference here. "export" has a clear meaning in wasm terms, and we already have --export, so the meaning of --export-if-defined is clear.

[sbc100]

Closing in favor of original change: https://reviews.llvm.org/D99887

lld/docs/WebAssembly.rst

 be exported.
 
 When building an executable, only the entry point (``_start``) and symbols with
 the ``WASM_SYMBOL_EXPORTED`` flag are exported by default.  In LLVM the
 ``WASM_SYMBOL_EXPORTED`` flag is set by the ``wasm-export-name`` attribute which
 in turn can be set using ``__attribute__((export_name))`` clang attribute.
 
 In addition, symbols can be exported via the linker command line using
-``--export``.
+``--export``, or ``--export-dynamic-symbol``.  The former will generate an
+error if the symbol is not defined whereas the later will not.
 
 Finally, just like with native ELF linker the ``--export-dynamic`` flag can be
 used to export symbols in the executable which are marked as
 ``visibility=default``.
 
 Imports
 ~~~~~~~
 

lld/test/wasm/export-dynamic-symbol.s

+# RUN: llvm-mc -filetype=obj -triple=wasm32-unknown-unknown -o %t.o %s
+# RUN: wasm-ld --export-dynamic-symbol=foo -o %t1.wasm %t.o
+# RUN: obj2yaml %t1.wasm | FileCheck %s
+
+# RUN: wasm-ld --export-dynamic-symbol=bar -o %t2.wasm %t.o
+# RUN: obj2yaml %t2.wasm | FileCheck %s --check-prefixes=MISSING
+
+.globl foo
+foo:
+  .functype foo () -> ()
+  end_function
+
+.globl _start
+_start:
+  .functype _start () -> ()
+  end_function
+
+#      CHECK:   - Type:            EXPORT
+# CHECK-NEXT:     Exports:
+# CHECK-NEXT:       - Name:            memory
+# CHECK-NEXT:         Kind:            MEMORY
+# CHECK-NEXT:         Index:           0
+# CHECK-NEXT:       - Name:            foo
+# CHECK-NEXT:         Kind:            FUNCTION
+# CHECK-NEXT:         Index:           0
+# CHECK-NEXT:       - Name:            _start
+# CHECK-NEXT:         Kind:            FUNCTION
+# CHECK-NEXT:         Index:           1
+
+#      MISSING:   - Type:            EXPORT
+# MISSING-NEXT:     Exports:
+# MISSING-NEXT:       - Name:            memory
+# MISSING-NEXT:         Kind:            MEMORY
+# MISSING-NEXT:         Index:           0
+# MISSING-NEXT:       - Name:            _start
+# MISSING-NEXT:         Kind:            FUNCTION
+# MISSING-NEXT:         Index:           0

lld/wasm/Config.h

struct Configuration {
 
   llvm::StringRef entry;
   llvm::StringRef mapFile;
   llvm::StringRef outputFile;
   llvm::StringRef thinLTOCacheDir;
 
   llvm::StringSet<> allowUndefinedSymbols;
   llvm::StringSet<> exportedSymbols;
+  std::vector<llvm::StringRef> requiredExports;
   std::vector<llvm::StringRef> searchPaths;
   llvm::CachePruningPolicy thinLTOCachePolicy;
   llvm::Optional<std::vector<std::string>> features;
 
   // The following config options do not directly correspond to any
   // particualr command line options.
 
   // True if we are creating position-independent code.

lld/wasm/Driver.cpp

if (auto e = tryCreateFile(config->mapFile))
     error("cannot open map file " + config->mapFile + ": " + e.message());
   if (errorCount())
     return;
 
   // Handle --trace-symbol.
   for (auto *arg : args.filtered(OPT_trace_symbol))
     symtab->trace(arg->getValue());
 
-  for (auto *arg : args.filtered(OPT_export))
+  for (auto *arg : args.filtered(OPT_export_dynamic_symbol))
     config->exportedSymbols.insert(arg->getValue());
 
+  for (auto *arg : args.filtered(OPT_export)) {
+    config->exportedSymbols.insert(arg->getValue());
+    config->requiredExports.push_back(arg->getValue());
+  }
+
   createSyntheticSymbols();
 
   // Add all files to the symbol table. This will add almost all
   // symbols that we need to the symbol table.
   for (InputFile *f : files)
     symtab->addFile(f);
   if (errorCount())
     return;
 
   // Handle the `--undefined <sym>` options.
   for (auto *arg : args.filtered(OPT_undefined))
     handleUndefined(arg->getValue());
 
   // Handle the `--export <sym>` options
   // This works like --undefined but also exports the symbol if its found
-  for (auto *arg : args.filtered(OPT_export))
-    handleUndefined(arg->getValue());
+  for (auto &iter : config->exportedSymbols)
+    handleUndefined(iter.first());
 
   Symbol *entrySym = nullptr;
   if (!config->relocatable && !config->entry.empty()) {
     entrySym = handleUndefined(config->entry);
     if (entrySym && entrySym->isDefined())
       entrySym->forceExport = true;
     else
       error("entry symbol not defined (pass --no-entry to suppress): " +
void LinkerDriver::linkerMain(ArrayRef<const char *> argsArr) {
   symtab->handleSymbolVariants();
   if (errorCount())
     return;
 
   // Apply symbol renames for -wrap.
   if (!wrapped.empty())
     wrapSymbols(wrapped);
 
-  for (auto *arg : args.filtered(OPT_export)) {
-    Symbol *sym = symtab->find(arg->getValue());
+  for (auto &iter : config->exportedSymbols) {
+    Symbol *sym = symtab->find(iter.first());
     if (sym && sym->isDefined())
       sym->forceExport = true;
   }
 
   if (!config->relocatable && !config->isPic) {
     // Add synthetic dummies for weak undefined functions.  Must happen
     // after LTO otherwise functions may not yet have signatures.
     symtab->handleWeakUndefines();

lld/wasm/Options.td

 include "llvm/Option/OptParser.td"
 
 // For options whose names are multiple letters, either one dash or
 // two can precede the option name except those that start with 'o'.
 class F<string name>: Flag<["--", "-"], name>;
 class J<string name>: Joined<["--", "-"], name>;
 class S<string name>: Separate<["--", "-"], name>;
 
+multiclass EEq<string name, string help> {
+  def NAME: Separate<["--"], name>;
+  def NAME # _eq: Joined<["--"], name # "=">, Alias<!cast<Separate>(NAME)>,
+    HelpText<help>;
+}
+
 multiclass Eq<string name, string help> {
   def NAME: Separate<["--", "-"], name>;
   def NAME # _eq: Joined<["--", "-"], name # "=">, Alias<!cast<Separate>(NAME)>,
     HelpText<help>;
 }
 
 multiclass B<string name, string help1, string help2> {
   def NAME: Flag<["--", "-"], name>, HelpText<help1>;

 
 def error_unresolved_symbols: F<"error-unresolved-symbols">,
   HelpText<"Report unresolved symbols as errors">;
 
 defm export_dynamic: B<"export-dynamic",
     "Put symbols in the dynamic symbol table",
     "Do not put symbols in the dynamic symbol table (default)">;
 
+defm export_dynamic_symbol : EEq<"export-dynamic-symbol",
+    "Export a given symbol if it is defined in the output.">;
+
 def entry: S<"entry">, MetaVarName<"<entry>">,
   HelpText<"Name of entry point symbol">;
 
 def error_limit: J<"error-limit=">,
   HelpText<"Maximum number of errors to emit before stopping (0 = no limit)">;
 
 def fatal_warnings: F<"fatal-warnings">,
   HelpText<"Treat warnings as errors">;

lld/wasm/Writer.cpp

void Writer::run() {
   if (!config->relocatable) {
     // Create linker synthesized __start_SECNAME/__stop_SECNAME symbols
     // This has to be done after memory layout is performed.
     for (const OutputSegment *seg : segments) {
       addStartStopSymbols(seg);
     }
   }
 
+  for (auto &pair : config->exportedSymbols) {
+    Symbol *sym = symtab->find(pair.first());
+    if (sym && sym->isDefined())
+      sym->forceExport = true;
+  }
+
   // Delay reporting error about explict exports until after addStartStopSymbols
   // which can create optional symbols.
-  for (auto &entry : config->exportedSymbols) {
-    StringRef name = entry.first();
+  for (auto &name : config->requiredExports) {
     Symbol *sym = symtab->find(name);
-    if (sym && sym->isDefined())
-      sym->forceExport = true;
-    else if (config->unresolvedSymbols == UnresolvedPolicy::ReportError)
+    if (!sym || !sym->isDefined()) {
+      if (config->unresolvedSymbols == UnresolvedPolicy::ReportError)
         error(Twine("symbol exported via --export not found: ") + name);
-    else if (config->unresolvedSymbols == UnresolvedPolicy::Warn)
+      if (config->unresolvedSymbols == UnresolvedPolicy::Warn)
         warn(Twine("symbol exported via --export not found: ") + name);
     }
+  }
 
   if (config->isPic) {
     log("-- combineOutputSegments");
     combineOutputSegments();
   }
 
   log("-- createSyntheticSectionsPostLayout");
   createSyntheticSectionsPostLayout();