[ELF] Handle -u before input files

[MaskRay]

Summary

If both a.a and b.so define foo

ld.bfd -u foo a.a b.so  # foo is defined
ld.bfd a.a b.so -u foo  # foo is defined
ld.bfd -u foo b.so a.a  # foo is undefined (provided at runtime by b.so)
ld.bfd b.so a.a -u foo  # foo is undefined (provided at runtime by b.so)

In all cases we make foo undefined in the output. I tend to think the
GNU ld behavior makes more sense.

• In their model, they have to treat -u as a fake object file with an undefined symbol before all input files, otherwise the first archive would not be fetched.
• Following their behavior allows us to drop a --warn-backrefs special case.

[MaskRay]

Don't move a isUsedInRegularObj block

[psmith]

Looks sensible to me. A couple of comment suggestions only.

lld/ELF/Driver.cpp

1473

We now no longer use this function for -u, will be worth updating comment.

1885–1888

IIUC this is to prevent LTO from removing definitions. May be worth saying

// Prevent LTO from removing any definition referenced by an undefined.

[MaskRay]

Address comments

[MaskRay]

lld/ELF/Driver.cpp

1473

All handleUndefined call sites may use addUndefined instead. I'll experiment. Two --wrap uses of addUndefined should probably be addUnusedUndefined instead (to fix https://bugs.llvm.org/show_bug.cgi?id=46169 )

1885–1888

Thanks for the suggestion!

The logic was introduced in D38348 .

We seem to consider a bitcode file a variant of an archive (its symbol table does not translate to .symtab in the output, but -u can force it).

[psmith]

Thanks for the update. I don't have any more comments. Looks good to me.

[troyj]

In all cases we make foo undefined in the output.

Why? We're having trouble with this downstream for the case of:

-u foo a.o b.a

where foo is weak in a.o and provided by b.a. The result is that foo is undefined in the output, whereas the previous behavior was for foo to be defined.

[MaskRay]

In D81052#2315733, @troyj wrote:

In all cases we make foo undefined in the output.

Why?

Undefined weak is difficult. You can check out D63974 D64136 D65584 D66992 is you want to learn more.

We're having trouble with this downstream for the case of:

-u foo a.o b.a

where foo is weak in a.o and provided by b.a. The result is that foo is undefined in the output, whereas the previous behavior was for foo to be defined.

Sent D88945

lld/ELF/Driver.cpp

static void excludeLibs(opt::InputArgList &args) {
 
   for (InputFile *file : objectFiles)
     visit(file);
 
   for (BitcodeFile *file : bitcodeFiles)
     visit(file);
 }
 
-// Force Sym to be entered in the output. Used for -u or equivalent.
+// Force Sym to be entered in the output.
 static void handleUndefined(Symbol *sym) {
   // Since a symbol may not be used inside the program, LTO may
   // eliminate it. Mark the symbol as "used" to prevent it.
   sym->isUsedInRegularObj = true;
 
-  // GNU linkers allow -u foo -ldef -lref. We should not treat it as a backward
-  // reference.
-  backwardReferences.erase(sym);
-
   if (sym->isLazy())
     sym->fetch();
 }
 
 // As an extension to GNU linkers, lld supports a variant of `-u`
 // which accepts wildcard patterns. All symbols that match a given
 // pattern are handled as if they were given by `-u`.
 static void handleUndefinedGlob(StringRef arg) {
static void readSymbolPartitionSection(InputSectionBase *s) {
   sym->partition = newPart.getNumber();
 }
 
 static Symbol *addUndefined(StringRef name) {
   return symtab->addSymbol(
       Undefined{nullptr, name, STB_GLOBAL, STV_DEFAULT, 0});
 }
 
+static Symbol *addUnusedUndefined(StringRef name) {
+  Undefined sym{nullptr, name, STB_GLOBAL, STV_DEFAULT, 0};
+  sym.isUsedInRegularObj = false;
+  return symtab->addSymbol(sym);
+}
+
 // This function is where all the optimizations of link-time
 // optimization takes place. When LTO is in use, some input files are
 // not in native object file format but in the LLVM bitcode format.
 // This function compiles bitcode files into a few big native files
 // using LLVM functions and replaces bitcode symbols with the results.
 // Because all bitcode files that the program consists of are passed to
 // the compiler at once, it can do a whole-program optimization.
 template <class ELFT> void LinkerDriver::compileBitcodeFiles() {
config->warnMissingEntry =
       (!config->entry.empty() || (!config->shared && !config->relocatable));
   if (config->entry.empty() && !config->relocatable)
     config->entry = (config->emachine == EM_MIPS) ? "__start" : "_start";
 
   // Handle --trace-symbol.
   for (auto *arg : args.filtered(OPT_trace_symbol))
     symtab->insert(arg->getValue())->traced = true;
 
+  // Handle -u/--undefined before input files. If both a.a and b.so define foo,
+  // -u foo a.a b.so will fetch a.a.
+  for (StringRef name : config->undefined)
+    addUnusedUndefined(name);
+
   // Add all files to the symbol table. This will add almost all
   // symbols that we need to the symbol table. This process might
   // add files to the link, via autolinking, these files are always
   // appended to the Files vector.
   {
     llvm::TimeTraceScope timeScope("Parse input files");
     for (size_t i = 0; i < files.size(); ++i)
       parseFile(files[i]);
   }
 
   // Now that we have every file, we can decide if we will need a
   // dynamic symbol table.
   // We need one if we were asked to export dynamic symbols or if we are
   // producing a shared library.
   // We also need one if any shared libraries are used and for pie executables
   // (probably because the dynamic linker needs it).
   config->hasDynSymTab =
       !sharedFiles.empty() || config->isPic || config->exportDynamic;
 
   // Some symbols (such as __ehdr_start) are defined lazily only when there
   // are undefined symbols for them, so we add these to trigger that logic.
   for (StringRef name : script->referencedSymbols)
     addUndefined(name);
 
-  // Handle the `--undefined <sym>` options.
-  for (StringRef arg : config->undefined)
-    if (Symbol *sym = symtab->find(arg))
-      handleUndefined(sym);
+  // Prevent LTO from removing any definition referenced by -u.
+  for (StringRef name : config->undefined)
+    if (Defined *sym = dyn_cast_or_null<Defined>(symtab->find(name)))
+      sym->isUsedInRegularObj = true;
 
   // If an entry symbol is in a static archive, pull out that file now.
   if (Symbol *sym = symtab->find(config->entry))
     handleUndefined(sym);
 
   // Handle the `--undefined-glob <pattern>` options.
   for (StringRef pat : args::getStrings(args, OPT_undefined_glob))
     handleUndefinedGlob(pat);

lld/test/ELF/undefined-opt.s

 # RUN: ld.lld -shared -o %t4 %t.o %tar.a -u unknown
 # RUN: llvm-readobj --dyn-symbols %t4 | \
 # RUN:     FileCheck --check-prefix=UNK-UNDEFINED-SO %s
 # UNK-UNDEFINED-SO: DynamicSymbols [
 # UNK-UNDEFINED-SO-NOT:     Name: unknown
 # UNK-UNDEFINED-SO: ]
 
 # Added undefined symbols should appear in the dynamic table if necessary.
-# RUN: ld.lld -shared -o %t5 %t.o -u export
-# RUN: llvm-readobj --dyn-symbols %t5 | \
+# RUN: ld.lld -shared -soname=t -o %t.so %t.o -u export
+# RUN: llvm-readobj --dyn-symbols %t.so | \
 # RUN:     FileCheck --check-prefix=EXPORT-SO %s
 # EXPORT-SO: DynamicSymbols [
 # EXPORT-SO:   Name: export
 # EXPORT-SO: ]
 
+## Test that we handle -u before input files: if we handle -u after
+## %t.so, foo would be undefined in the output.
+# RUN: rm -f %t.a && llvm-ar rc %t.a %t.o
+# RUN: ld.lld -u export %t.a %t.so -o %t5
+# RUN: llvm-readobj --dyn-symbols %t5 | FileCheck --check-prefix=EXPORT-SO %s
+
 .globl _start
 _start:
 
 .globl export
 export: