[LLD][ELF] Add --lto-export-symbol-list

[samitolvanen]

Summary

Normally when compiling bitcode in a relocatable link, LLD sets
the VisibleToRegularObj flag in the symbol resolution for all
globals, which prevents LTO from dropping globals that may be
needed later. However, in situations where we know which globals
we want to export, LTO could better optimize the code if we
marked only the exported symbols visible. This is the case in the
Linux kernel, for example, where all the bitcode in the kernel
is compiled during a relocatable link, and we have a list of
exported symbols.

Add an --lto-export-symbol-list flag to LLD, which accepts a list
of exported globals in the version script format, and uses the
list to set the bindings for non-exported globals to STB_LOCAL,
and sets the VisibleToRegularObj flag in symbol resolution only
for the exported symbols.

[samitolvanen]

@pcc, @MaskRay, based on earlier discussions (e.g. https://github.com/ClangBuiltLinux/linux/issues/1737), does something like this look like a reasonable approach?

[pcc]

What do you think about making this apply not only to LTO but to other -r links as well? This would be useful for use cases like the internal symbolizer used by the sanitizers, which currently (ab)uses LTO for a similar effect. I was imagining that the user interface would be that we would make --version-script compatible with -r links, with the restriction being that it would not be possible to specify a symbol version (as this patch does).

[samitolvanen]

In D142163#4067630, @pcc wrote:

What do you think about making this apply not only to LTO but to other -r links as well?

Sure, I'm fine with that. Looks like we would just have to apply computeBinding to relocatable links as well when export symbols are specified. @MaskRay, what do you think?

[MaskRay]

In D142163#4077996, @samitolvanen wrote:

In D142163#4067630, @pcc wrote:

What do you think about making this apply not only to LTO but to other -r links as well?

Sure, I'm fine with that. Looks like we would just have to apply computeBinding to relocatable links as well when export symbols are specified. @MaskRay, what do you think?

(On a trip; don't have much time checking emails or looking into patch details)
The feature applying to -r makes sense.

lld/ELF/Symbols.h

175

wrong case

[samitolvanen]

Don't apply to isUsedInRegularObj symbols.

[samitolvanen]

So, I looked into applying this to all object files with by adding similar code to Writer<ELFT>::finalizeSections(). This is not quite what we want in the kernel though. The primary goal is to better optimize bitcode during a relocatable link, and while we can conveniently generate a list of functions exported to kernel modules, we don't necessarily have a complete list of symbols (typically defined in stand-alone assembly) that are later needed in linker scripts or whose bindings objtool etc. assume to remain untouched, for example. In order to avoid additional hacks in the kernel, I would prefer to keep this flag specific to LTO. I do see the value in having a similar option for all object files, but perhaps that can be a separate option?

[MaskRay]

lld/ELF/LTO.cpp

262

objSym.isExecutable() this condition is strange and does not fit into the spirit that we ignore symbol types for various symbol list features.
Can the user specify all data symbols?

lld/ELF/Symbols.cpp

272

This makes the Writer.cpp caller slow. Suggest: add a new function. I need to think more why we don't make the feature available to non-LTO use cases.

lld/test/ELF/driver.test

74

Move this to the end of lto-export-symbol-list.ll.

lld/test/ELF/lto/lto-export-symbol-list-unexpected-end.s

1

Move this to the end of lto-export-symbol-list.ll. Use RUN: rm -rf %t && split-file %s %t && cd %t.

lld/test/ELF/lto/lto-export-symbol-list.ll

5

llvm-readobj -s is not recommended for new tests (too verbose). Use llvm-readelf -s (see my changes rewriting some tests to use it)

[samitolvanen]

lld/ELF/LTO.cpp

262

I agree, this is bit awkward. I'll have to see if there's a convenient way for us to specify data symbols in the kernel. The problem is that some of these are referenced from linker scripts that are used later when linking vmlinux.

lld/ELF/Config.h

struct Config {
   llvm::SmallVector<llvm::StringRef, 0> auxiliaryList;
   llvm::SmallVector<llvm::StringRef, 0> filterList;
   llvm::SmallVector<llvm::StringRef, 0> passPlugins;
   llvm::SmallVector<llvm::StringRef, 0> searchPaths;
   llvm::SmallVector<llvm::StringRef, 0> symbolOrderingFile;
   llvm::SmallVector<llvm::StringRef, 0> thinLTOModulesToCompile;
   llvm::SmallVector<llvm::StringRef, 0> undefined;
   llvm::SmallVector<SymbolVersion, 0> dynamicList;
+  llvm::SmallVector<SymbolVersion, 0> ltoExportSymbolList;
   llvm::SmallVector<uint8_t, 0> buildIdVector;
   llvm::SmallVector<llvm::StringRef, 0> mllvmOpts;
   llvm::MapVector<std::pair<const InputSectionBase *, const InputSectionBase *>,
                   uint64_t>
       callGraphProfile;
   bool allowMultipleDefinition;
   bool androidPackDynRelocs = false;
   bool armHasBlx = false;
struct Config {
   bool gnuUnique;
   bool hasDynSymTab;
   bool ignoreDataAddressEquality;
   bool ignoreFunctionAddressEquality;
   bool ltoCSProfileGenerate;
   bool ltoPGOWarnMismatch;
   bool ltoDebugPassManager;
   bool ltoEmitAsm;
+  bool ltoExportSymbols;
   bool ltoUniqueBasicBlockSectionNames;
   bool ltoWholeProgramVisibility;
   bool mergeArmExidx;
   bool mipsN32Abi = false;
   bool mmapOutputFile;
   bool nmagic;
   bool noDynamicLinker = false;
   bool noinhibitExec;

lld/ELF/Driver.cpp

if (config->relocatable) {
     if (config->icf != ICFLevel::None)
       error("-r and --icf may not be used together");
     if (config->pie)
       error("-r and -pie may not be used together");
     if (config->exportDynamic)
       error("-r and --export-dynamic may not be used together");
   }
 
+  if (config->ltoExportSymbols && !config->relocatable)
+    error("--lto-export-symbol-list may not be used without -r");
+
   if (config->executeOnly) {
     if (config->emachine != EM_AARCH64)
       error("--execute-only is only supported on AArch64 targets");
 
     if (config->singleRoRx && !script->hasSectionsCommand)
       error("--execute-only and --no-rosegment cannot be used together");
   }
 
static void readConfigs(opt::InputArgList &args) {
 
   for (auto *arg : args.filtered(OPT_version_script))
     if (std::optional<std::string> path = searchScript(arg->getValue())) {
       if (std::optional<MemoryBufferRef> buffer = readFile(*path))
         readVersionScript(*buffer);
     } else {
       error(Twine("cannot find version script ") + arg->getValue());
     }
+
+  config->ltoExportSymbols = args.hasArg(OPT_lto_export_symbol_list);
+  for (auto *arg : args.filtered(OPT_lto_export_symbol_list))
+    if (std::optional<MemoryBufferRef> buffer = readFile(arg->getValue()))
+      readLTOExportSymbolList(*buffer);
 }
 
 // Some Config members do not directly correspond to any particular
 // command line options, but computed based on other Config values.
 // This function initialize such members. See Config.h for the details
 // of these values.
 static void setConfigs(opt::InputArgList &args) {
   ELFKind k = config->ekind;
void LinkerDriver::link(opt::InputArgList &args) {
   // For a relocatable output, version scripts don't make sense, and
   // parsing a symbol version string (e.g. dropping "@ver1" from a symbol
   // name "foo@ver1") rather do harm, so we don't call this if -r is given.
   if (!config->relocatable) {
     llvm::TimeTraceScope timeScope("Process symbol versions");
     symtab.scanVersionScript();
   }
 
+  if (config->relocatable) {
+    llvm::TimeTraceScope timeScope("Process LTO export symbols");
+    symtab.scanLTOExportList();
+  }
+
   // Skip the normal linked output if some LTO options are specified.
   //
   // For --thinlto-index-only, index file creation is performed in
   // compileBitcodeFiles, so we are done afterwards. --plugin-opt=emit-llvm and
   // --plugin-opt=emit-asm create output files in bitcode or assembly code,
   // respectively. When only certain thinLTO modules are specified for
   // compilation, the intermediate object file are the expected output.
   const bool skipLinkedOutput = config->thinLTOIndexOnly || config->emitLLVM ||

lld/ELF/DriverUtils.cpp

for (auto *arg : args) {
     case OPT_lto_sample_profile:
       os << arg->getSpelling() << quote(rewritePath(arg->getValue())) << "\n";
       break;
     case OPT_call_graph_ordering_file:
     case OPT_dynamic_list:
     case OPT_export_dynamic_symbol_list:
     case OPT_just_symbols:
     case OPT_library_path:
+    case OPT_lto_export_symbol_list:
     case OPT_retain_symbols_file:
     case OPT_rpath:
     case OPT_script:
     case OPT_symbol_ordering_file:
     case OPT_sysroot:
     case OPT_version_script:
       os << arg->getSpelling() << " " << quote(rewritePath(arg->getValue()))
          << "\n";

lld/ELF/LTO.cpp

for (size_t i = 0, e = syms.size(); i != e; ++i) {
     // Ideally we shouldn't check for SF_Undefined but currently IRObjectFile
     // reports two symbols for module ASM defined. Without this check, lld
     // flags an undefined in IR with a definition in ASM as prevailing.
     // Once IRObjectFile is fixed to report only one symbol this hack can
     // be removed.
     r.Prevailing = !objSym.isUndefined() && sym->file == &f;
 
     // We ask LTO to preserve following global symbols:
-    // 1) All symbols when doing relocatable link, so that them can be used
-    //    for doing final link.
+    // 1) All symbols when doing relocatable link, so that they can be used
+    //    for doing final link, unless
+    //    a) the symbol is in --lto-export-symbol-list, and
+    //    b) the symbol is executable.
     // 2) Symbols that are used in regular objects.
     // 3) C named sections if we have corresponding __start_/__stop_ symbol.
     // 4) Symbols that are defined in bitcode files and used for dynamic
     //    linking.
     // 5) Symbols that will be referenced after linker wrapping is performed.
-    r.VisibleToRegularObj = config->relocatable || sym->isUsedInRegularObj ||
-                            sym->referencedAfterWrap ||
-                            (r.Prevailing && sym->includeInDynsym()) ||
-                            usedStartStop.count(objSym.getSectionName());
+    bool useExports = config->ltoExportSymbols && !sym->isUsedInRegularObj &&
+                      objSym.isExecutable();
+    uint8_t binding = sym->computeBinding(useExports);
+    r.VisibleToRegularObj =
+        (config->relocatable && !useExports) || sym->isUsedInRegularObj ||
+        (r.Prevailing &&
+         (useExports ? binding != STB_LOCAL : sym->includeInDynsym())) ||
+
     // Identify symbols exported dynamically, and that therefore could be
     // referenced by a shared library not visible to the linker.
     r.ExportDynamic =
-        sym->computeBinding() != STB_LOCAL &&
+        binding != STB_LOCAL &&
         (config->exportDynamic || sym->exportDynamic || sym->inDynamicList);
     const auto *dr = dyn_cast<Defined>(sym);
     r.FinalDefinitionInLinkageUnit =
         (isExec || sym->visibility() != STV_DEFAULT) && dr &&
         // Skip absolute symbols from ELF objects, otherwise PC-rel relocations
         // will be generated by for them, triggering linker errors.
         // Symbol section is always null for bitcode symbols, hence the check
         // for isElf(). Skip linker script defined symbols as well: they have

lld/ELF/Options.td

 
 // LTO-related options.
 def lto_aa_pipeline: JJ<"lto-aa-pipeline=">,
   HelpText<"AA pipeline to run during LTO. Used in conjunction with -lto-newpm-passes">;
 def lto_debug_pass_manager: FF<"lto-debug-pass-manager">,
   HelpText<"Debug new pass manager">;
 def lto_emit_asm: FF<"lto-emit-asm">,
   HelpText<"Emit assembly code">;
+defm lto_export_symbol_list : EEq<"lto-export-symbol-list",
+   "Specify a list of symbols that are marked visible to regular objects in a relocatable LTO link">,
+    MetaVarName<"file">;
 def lto_newpm_passes: JJ<"lto-newpm-passes=">,
   HelpText<"Passes to run during LTO">;
 def lto_O: JJ<"lto-O">, MetaVarName<"<opt-level>">,
   HelpText<"Optimization level for LTO">;
 def lto_partitions: JJ<"lto-partitions=">,
   HelpText<"Number of LTO codegen partitions">;
 def lto_cs_profile_generate: FF<"lto-cs-profile-generate">,
   HelpText<"Perform context sensitive PGO instrumentation">;

lld/ELF/ScriptParser.h

 // lld::elf::config and lld::elf::script.
 void readLinkerScript(MemoryBufferRef mb);
 
 // Parses a version script.
 void readVersionScript(MemoryBufferRef mb);
 
 void readDynamicList(MemoryBufferRef mb);
 
+void readLTOExportSymbolList(MemoryBufferRef mb);
+
 // Parses the defsym expression.
 void readDefsym(StringRef name, MemoryBufferRef mb);
 
 bool hasWildcard(StringRef s);
 
 } // namespace lld::elf
 
 #endif

lld/ELF/ScriptParser.cpp

for (; !path.empty(); path = sys::path::parent_path(path)) {
       isUnderSysroot = true;
       return;
     }
   }
 
   void readLinkerScript();
   void readVersionScript();
   void readDynamicList();
+  void readLTOExportSymbolList();
   void readDefsym(StringRef name);
 
 private:
   void addFile(StringRef path);
 
   void readAsNeeded();
   void readEntry();
   void readExtern();
if (!locals.empty()) {
     setError("\"local:\" scope not supported in --dynamic-list");
     return;
   }
 
   for (SymbolVersion v : globals)
     config->dynamicList.push_back(v);
 }
 
+void ScriptParser::readLTOExportSymbolList() {
+  expect("{");
+  SmallVector<SymbolVersion, 0> locals;
+  SmallVector<SymbolVersion, 0> globals;
+  std::tie(locals, globals) = readSymbols();
+  expect(";");
+
+  if (!atEOF()) {
+    setError("EOF expected, but got " + next());
+    return;
+  }
+  if (!locals.empty()) {
+    setError("\"local:\" scope not supported in --lto-export-symbol-list");
+    return;
+  }
+
+  for (SymbolVersion v : globals)
+    config->ltoExportSymbolList.push_back(v);
+}
+
 void ScriptParser::readVersionScript() {
   readVersionScriptCommand();
   if (!atEOF())
     setError("EOF expected, but got " + next());
 }
 
 void ScriptParser::readVersionScriptCommand() {
   if (consume("{")) {
void elf::readVersionScript(MemoryBufferRef mb) {
   ScriptParser(mb).readVersionScript();
 }
 
 void elf::readDynamicList(MemoryBufferRef mb) {
   llvm::TimeTraceScope timeScope("Read dynamic list", mb.getBufferIdentifier());
   ScriptParser(mb).readDynamicList();
 }
 
+void elf::readLTOExportSymbolList(MemoryBufferRef mb) {
+  llvm::TimeTraceScope timeScope("Read LTO export symbol list",
+                                 mb.getBufferIdentifier());
+  ScriptParser(mb).readLTOExportSymbolList();
+}
+
 void elf::readDefsym(StringRef name, MemoryBufferRef mb) {
   llvm::TimeTraceScope timeScope("Read defsym input", name);
   ScriptParser(mb).readDefsym(name);
 }

lld/ELF/SymbolTable.h

template <typename T> Symbol *addSymbol(const T &newSym) {
     Symbol *sym = insert(newSym.getName());
     sym->resolve(newSym);
     return sym;
   }
   Symbol *addAndCheckDuplicate(const Defined &newSym);
 
   void scanVersionScript();
 
+  void scanLTOExportList();
+
   Symbol *find(StringRef name);
 
   void handleDynamicList();
 
   // Set of .so files to not link the same shared object file more than once.
   llvm::DenseMap<llvm::CachedHashStringRef, SharedFile *> soNames;
 
   // Comdat groups define "link once" sections. If two comdat groups have the

lld/ELF/SymbolTable.cpp

if (sym->hasVersionSuffix)
       sym->parseSymbolVersion();
 
   // isPreemptible is false at this point. To correctly compute the binding of a
   // Defined (which is used by includeInDynsym()), we need to know if it is
   // VER_NDX_LOCAL or not. Compute symbol versions before handling
   // --dynamic-list.
   handleDynamicList();
 }
+
+void SymbolTable::scanLTOExportList() {
+  SmallVector<Symbol *, 0> syms;
+  for (SymbolVersion &ver : config->ltoExportSymbolList) {
+    if (ver.hasWildcard)
+      syms = findAllByVersion(ver, /*includeNonDefault=*/true);
+    else
+      syms = findByVersion(ver);
+
+    for (Symbol *sym : syms)
+      sym->ltoExportSymbol = true;
+  }
+}

lld/ELF/Symbols.h

public:
   //   canBeOmittedFromSymbolTable().
   uint8_t exportDynamic : 1;
 
   // True if the symbol is in the --dynamic-list file. A Defined symbol with
   // protected or default visibility with this property is required to be
   // exported into .dynsym.
   uint8_t inDynamicList : 1;
 
+  // True if the symbol is in the --lto-export-symbol-list file.
+  uint8_t ltoExportSymbol : 1;
+
   // Used to track if there has been at least one undefined reference to the
   // symbol. For Undefined and SharedSymbol, the binding may change to STB_WEAK
   // if the first undefined reference from a non-shared object is weak.
   uint8_t referenced : 1;
 
   // Used to track if this symbol will be referenced after wrapping is performed
   // (i.e. this will be true for foo if __real_foo is referenced, and will be
   // true for __wrap_foo if foo is referenced).
   uint8_t referencedAfterWrap : 1;
 
   // True if this symbol is specified by --trace-symbol option.
   uint8_t traced : 1;
 
   // True if the name contains '@'.
   uint8_t hasVersionSuffix : 1;
 
   // Symbol visibility. This is the computed minimum visibility of all
   // observed non-DSO symbols.
   uint8_t visibility() const { return stOther & 3; }
   void setVisibility(uint8_t visibility) {
     stOther = (stOther & ~3) | visibility;
   }
 
   bool includeInDynsym() const;
-  uint8_t computeBinding() const;
+  uint8_t computeBinding(bool useExports = false) const;
   bool isGlobal() const { return binding == llvm::ELF::STB_GLOBAL; }
   bool isWeak() const { return binding == llvm::ELF::STB_WEAK; }
 
   bool isUndefined() const { return symbolKind == UndefinedKind; }
   bool isCommon() const { return symbolKind == CommonKind; }
   bool isDefined() const { return symbolKind == DefinedKind; }
   bool isShared() const { return symbolKind == SharedKind; }
   bool isPlaceholder() const { return symbolKind == PlaceholderKind; }

lld/ELF/Symbols.cpp

 
 void Symbol::extract() const {
   if (file->lazy) {
     file->lazy = false;
     parseFile(file);
   }
 }
 
-uint8_t Symbol::computeBinding() const {
+uint8_t Symbol::computeBinding(bool useExports /*=false*/) const {
+  if (useExports && config->ltoExportSymbols && binding == STB_GLOBAL &&
+      !ltoExportSymbol)
+    return STB_LOCAL;
   auto v = visibility();
   if ((v != STV_DEFAULT && v != STV_PROTECTED) || versionId == VER_NDX_LOCAL)
     return STB_LOCAL;
   if (binding == STB_GNU_UNIQUE && !config->gnuUnique)
     return STB_GLOBAL;
   return binding;
 }
 

lld/test/ELF/driver.test

 
 # RUN: not ld.lld %t -z max-page-size 2>&1 | FileCheck -check-prefix=ERR11 %s
 # ERR11: unknown -z value: max-page-size
 
 ## Attempt to use -r and --export-dynamic together
 # RUN: not ld.lld -r -export-dynamic %t -o /dev/null 2>&1 | FileCheck -check-prefix=ERR12 %s
 # ERR12: -r and --export-dynamic may not be used together
 
+## Attempt to use --lto-export-symbol-list without -r
+# RUN: echo "{ };" > %t.list
+# RUN: not ld.lld --lto-export-symbol-list %t.list %t -o /dev/null 2>&1 | FileCheck -check-prefix=ERR13 %s
+# ERR13: --lto-export-symbol-list may not be used without -r
+
 .globl _start
 _start:
   nop

lld/test/ELF/lto/lto-export-symbol-list-locals.s

+# REQUIRES: x86
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-pc-linux %s -o %t.o
+# RUN: echo "{ local: *; };" > %t.list
+# RUN: not ld.lld --lto-export-symbol-list %t.list -r %t.o -o /dev/null 2>&1 | FileCheck %s
+
+# CHECK: error: {{.*}}:1: "local:" scope not supported in --lto-export-symbol-list

lld/test/ELF/lto/lto-export-symbol-list-unexpected-end.s

+# REQUIRES: x86
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-pc-linux %s -o %t.o
+# RUN: echo "{ }; foo;" > %t.list
+# RUN: not ld.lld --lto-export-symbol-list %t.list -r %t.o -o /dev/null 2>&1 | FileCheck %s
+
+# CHECK: error: {{.*}}:1: EOF expected, but got foo

lld/test/ELF/lto/lto-export-symbol-list.ll

+; REQUIRES: x86
+; RUN: llvm-as %s -o %t1.o
+
+; RUN: ld.lld %t1.o -r -o %t2.o
+; RUN: llvm-readobj --symbols %t2.o | FileCheck %s --check-prefixes=CHECK,NOLIST
+
+; RUN: echo "{ f2; };" > %t.list
+; RUN: ld.lld %t1.o -r --lto-export-symbol-list %t.list -o %t
+; RUN: llvm-readobj --symbols %t | FileCheck %s --check-prefixes=CHECK,LIST
+
+;; f3 is referenced, but not in --lto-export-symbol-list.
+; LIST:           Name: f3
+; LIST-NEXT:      Value:
+; LIST-NEXT:      Size:
+; LIST-NEXT:      Binding: Local
+; LIST-NEXT:      Type: Function
+; LIST-NEXT:      Other:
+; LIST-NEXT:      Section: .text
+; LIST-NEXT:   }
+
+;; f1 is not used and not in --export-symbol-list.
+; LIST-NOT:       Name: f1
+; NOLIST:         Name: f1
+; NOLIST-NEXT:    Value:
+; NOLIST-NEXT:    Size:
+; NOLIST-NEXT:    Binding: Global
+; NOLIST-NEXT:    Type: Function
+; NOLIST-NEXT:    Other:
+; NOLIST-NEXT:    Section: .text
+; NOLIST-NEXT: }
+
+;; f2 must always be exported.
+; CHECK:          Name: f2
+; CHECK-NEXT:     Value:
+; CHECK-NEXT:     Size:
+; CHECK-NEXT:     Binding: Global
+; CHECK-NEXT:     Type: Function
+; CHECK-NEXT:     Other:
+; CHECK-NEXT:     Section: .text
+; CHECK-NEXT:  }
+
+;; f3 without --lto-export-symbol-list must be exported.
+; NOLIST:         Name: f3
+; NOLIST-NEXT:    Value:
+; NOLIST-NEXT:    Size:
+; NOLIST-NEXT:    Binding: Global
+; NOLIST-NEXT:    Type: Function
+; NOLIST-NEXT:    Other:
+; NOLIST-NEXT:    Section: .text
+; NOLIST-NEXT: }
+
+;; p is not in an executable section, so it must not be affected.
+; CHECK:          Name: p
+; CHECK-NEXT:     Value:
+; CHECK-NEXT:     Size:
+; CHECK-NEXT:     Binding: Global
+; CHECK-NEXT:     Type: Object
+; CHECK-NEXT:     Other:
+; CHECK-NEXT:     Section: .data
+; CHECK-NEXT:  }
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define void @f1() {
+  ret void
+}
+
+define void @f2() {
+  ret void
+}
+
+define void @f3() {
+  ret void
+}
+
+@p = global ptr @f3, align 8