[ELF] Support --exclude-inputs

[melver]

Summary

Linking of binaries in large projects often results in processing of
thousands of linker inputs. In such projects, controlling all transitive
dependencies' inputs and libraries to be added to the final linker
command is (unfortunately) often too complex.

In theory the build system should provide a convenient way to simply
exclude a given library, but the reality is that few build systems do.

The simplest possible design, that will work across all build systems,
is therefore to pass an option to the linker to exclude inputs matching
a given pattern. The possible use cases are:

1. Removing inputs added by dependencies (e.g. via pkg-config) where modification of the build scripts (or build systems such as Bazel) is infeasible.
2. Replacing inputs added by dependencies where modification of the build scripts is infeasible.
3. Replacing default inputs with custom implementations.

Introduce --exclude-inputs, which accomplishes just that.

[MaskRay]

I feel that this debugging feature will be useful. Can you please create a post on https://discourse.llvm.org/c/subprojects/lld/7 so that more people can comment? It's possible that some linkers have similar features and we can learn from their experience.

lld/ELF/Driver.cpp

233

no brace for cascading single-line while/for/if

Actually, it may be useful to add a log comment in --verbose mode.

lld/test/ELF/exclude-inputs.s

10

If %t (an absolute path) contains foo, this check may go wrong.

Add # RUN: rm -rf %t && mkdir %t && cd %t in the first line can avoid this issue.

[melver]

• fix test if 'foo' or 'bar' in absolute path
• formatting of for-if
• add log()

[melver]

In D148726#4280936, @MaskRay wrote:

I feel that this debugging feature will be useful. Can you please create a post on https://discourse.llvm.org/c/subprojects/lld/7 so that more people can comment? It's possible that some linkers have similar features and we can learn from their experience.

Thanks, posted https://discourse.llvm.org/t/rfc-support-exclude-inputs/70070

[melver]

Possible alternative: https://reviews.llvm.org/D148795

lld/ELF/Config.h

llvm::SmallVector<std::pair<llvm::GlobPattern, uint64_t>, 0>
       deadRelocInNonAlloc;
   bool demangle = true;
   bool dependentLibraries;
   bool disableVerify;
   bool ehFrameHdr;
   bool emitLLVM;
   bool emitRelocs;
   bool enableNewDtags;
+  llvm::SmallVector<llvm::GlobPattern, 0> excludeInputs;
   bool executeOnly;
   bool exportDynamic;
   bool fixCortexA53Errata843419;
   bool fixCortexA8;
   bool formatBinary = false;
   bool fortranCommon;
   bool gcSections;
   bool gdbIndex;

lld/ELF/Driver.cpp

 static bool isBitcode(MemoryBufferRef mb) {
   return identify_magic(mb.getBuffer()) == llvm::file_magic::bitcode;
 }
 
 // Opens a file and create a file object. Path has to be resolved already.
 void LinkerDriver::addFile(StringRef path, bool withLOption) {
   using namespace sys::fs;
 
+  for (const llvm::GlobPattern &pat : config->excludeInputs)
+    if (pat.match(path)) {
+      log("excluding input " + Twine(path));
+      return;
+    }
+
   std::optional<MemoryBufferRef> buffer = readFile(path);
   if (!buffer)
     return;
   MemoryBufferRef mbref = *buffer;
 
   if (config->formatBinary) {
     files.push_back(make<BinaryFile>(mbref));
     return;
static void readConfigs(opt::InputArgList &args) {
 
   for (auto *arg : args.filtered(OPT_version_script))
     if (std::optional<std::string> path = searchScript(arg->getValue())) {
       if (std::optional<MemoryBufferRef> buffer = readFile(*path))
         readVersionScript(*buffer);
     } else {
       error(Twine("cannot find version script ") + arg->getValue());
     }
+
+  for (opt::Arg *arg : args.filtered(OPT_exclude_inputs)) {
+    StringRef pattern(arg->getValue());
+    if (Expected<GlobPattern> pat = GlobPattern::create(pattern))
+      config->excludeInputs.push_back(std::move(*pat));
+    else
+      error(arg->getSpelling() + ": " + toString(pat.takeError()));
+  }
 }
 
 // Some Config members do not directly correspond to any particular
 // command line options, but computed based on other Config values.
 // This function initialize such members. See Config.h for the details
 // of these values.
 static void setConfigs(opt::InputArgList &args) {
   ELFKind k = config->ekind;

lld/ELF/Options.td

defm error_limit:
   EEq<"error-limit", "Maximum number of errors to emit before stopping (0 = no limit)">;
 
 def error_unresolved_symbols: F<"error-unresolved-symbols">,
   HelpText<"Report unresolved symbols as errors">;
 
 defm error_handling_script: EEq<"error-handling-script",
     "Specify an error handling script">;
 
+defm exclude_inputs: EEq<"exclude-inputs", "Do not link inputs matching given pattern">,
+  MetaVarName<"<pattern>">;
+
 defm exclude_libs: Eq<"exclude-libs", "Exclude static libraries from automatic export">;
 
 defm execute_only: BB<"execute-only",
     "Mark executable sections unreadable",
     "Mark executable sections readable (default)">;
 
 defm export_dynamic: B<"export-dynamic",
     "Put symbols in the dynamic symbol table",

lld/docs/ld.lld.1

 may be
 .Cm missing-lib
 followed by the name of the missing library.
 .Cm undefined-symbol
 followed by the name of the undefined symbol.
 .It Fl -execute-only
 Mark executable sections unreadable.
 This option is currently only supported on AArch64.
+.It Fl -exclude-inputs Ns = Ns Ar pattern
+Do not link inputs matching given pattern.
 .It Fl -exclude-libs Ns = Ns Ar value
 Exclude static libraries from automatic export.
 .It Fl -export-dynamic , Fl E
 Put symbols in the dynamic symbol table.
 .It Fl -export-dynamic-symbol Ns = Ns Ar glob
 (executable) Put matched non-local defined symbols to the dynamic symbol table.
 (shared object) References to matched non-local STV_DEFAULT symbols shouldn't be bound to definitions within the shared object even if they would otherwise be due to
 .Cm -Bsymbolic

lld/test/ELF/exclude-inputs.s

+// REQUIRES: x86
+
+// RUN: rm -rf %t && mkdir %t && cd %t
+// RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o foo.o
+// RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux --defsym BAR=1 %s -o bar.o
+// RUN: llvm-ar rcs bar.a bar.o
+
+// RUN: ld.lld --exclude-inputs='*foo*' --whole-archive foo.o bar.a -o exe
+// RUN: llvm-readelf -s exe | FileCheck --check-prefix=OBJECT %s
+// OBJECT: bar
+// OBJECT-NOT: foo
+
+// RUN: ld.lld --exclude-inputs='*bar*' --whole-archive foo.o bar.a -o exe
+// RUN: llvm-readelf -s exe | FileCheck --check-prefix=ARCHIVE %s
+// ARCHIVE: foo
+// ARCHIVE-NOT: bar
+
+.ifdef BAR
+.globl bar
+bar:
+  nop
+.else
+.globl foo
+foo:
+  nop
+.endif