Add --undefined-glob which is an --undefined with wildcard pattern match.

[ruiu]

Summary

This patch adds new command line option --undefined-glob to lld.
That option is a variant of --undefined but accepts wildcard
patterns so that all symbols that match with a given pattern are
handled as if they were given by -u.

-u foo is to force resolve symbol foo if foo is not a defined symbol
and there's a static archive that contains a definition of symbol foo.

Now, you can specify a wildcard pattern as an argument for --undefined-glob.
So, if you want to include all JNI symbols (which start with "Java_"), you
can do that by passing --undefined-glob "Java_*" to the linker, for example.

In this patch, I use the same glob pattern matcher as the version script
processor is using, so it does not only support * but also ? and [...].

Discussion thread: http://lists.llvm.org/pipermail/llvm-dev/2019-June/132961.html

[MaskRay]

lld/ELF/Driver.cpp

1327 ↗

(On Diff #204439)

clang-format.

here not -> here is not

1343 ↗

(On Diff #204439)

Name.find_first_of("?*[")

[ruiu]

• address review comments

[peter.smith]

Looks good to me as well.

lld/ELF/Driver.cpp

1335 ↗

(On Diff #204445)

Might be worth extracting this block, and the equivalent one in handleUndefined into a separate function. That way the action to take to once a symbol has been selected by a pattern is always the same in both functions, including the comment about LTO. I've not got a strong opinion though as I don't expect much change.

[ruiu]

• factor out common code and unify two functions into one

[grimar]

LGTM. Minor suggestion is inlined.

lld/test/ELF/undefined-wildcard.s

17 ↗

(On Diff #204450)

hint: you should be able to use ----implicit-check-not=foo --implicit-check-not=bar
This way can probably help to make checks below a bit simpler.

[jyknight]

Actually linker scripts do support metacharacter escaping (although, it seems lld does not support this correctly, yet).

E.g., this should match only a function named literally "?"

VER {
  global:
   "?";
};

While this matches any one-letter function:

VER {
  global:
   ?;
};

So perhaps this functionality should be given its own command-line argument, e.g. --undefined-glob, so as to not cause any such confusion.

[ruiu]

In D63244#1541879, @jyknight wrote:

Actually linker scripts do support metacharacter escaping (although, it seems lld does not support this correctly, yet).

E.g., this should match only a function named literally "?"

VER {
  global:
   "?";
};

While this matches any one-letter function:

VER {
  global:
   ?;
};

So perhaps this functionality should be given its own command-line argument, e.g. --undefined-glob, so as to not cause any such confusion.

I think I like the idea to add this as a new option, instead of changing the meaning of the existing option in a subtle way. I'll give it a new name as you suggested.

[ruiu]

• rename the option --undefined-glob.

[ruiu]

• update comment

[MaskRay]

The code change looks good. Nit about the help text.

lld/docs/ld.lld.1

509 ↗

(On Diff #204700)

I was puzzled when I first noticed this option and its help text: Force to be an undefined symbol during linking.

If the symbol does not exist, there is no undefined symbol in the link result.
If the symbol is defined, this option does not make the symbol undefined.

Now that you are adding a similar option, maybe revise the documentation a bit to address the confusion?

[ruiu]

• updated the man page

lld/trunk/ELF/Driver.cpp

 #include "lld/Common/TargetOptionsCommandFlags.h"
 #include "lld/Common/Threads.h"
 #include "lld/Common/Version.h"
 #include "llvm/ADT/SetVector.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringSwitch.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Compression.h"
+#include "llvm/Support/GlobPattern.h"
 #include "llvm/Support/LEB128.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/TarWriter.h"
 #include "llvm/Support/TargetSelect.h"
 #include "llvm/Support/raw_ostream.h"
 #include <cstdlib>
 #include <utility>
 
static void excludeLibs(opt::InputArgList &Args) {
   for (InputFile *File : ObjectFiles)
     Visit(File);
 
   for (BitcodeFile *File : BitcodeFiles)
     Visit(File);
 }
 
 // Force Sym to be entered in the output. Used for -u or equivalent.
-static void handleUndefined(StringRef Name) {
-  Symbol *Sym = Symtab->find(Name);
-  if (!Sym)
-    return;
-
-  // Since symbol S may not be used inside the program, LTO may
+static void handleUndefined(Symbol *Sym) {
+  // Since a symbol may not be used inside the program, LTO may
   // eliminate it. Mark the symbol as "used" to prevent it.
   Sym->IsUsedInRegularObj = true;
 
   if (Sym->isLazy())
     Sym->fetch();
 }
 
+// As an extention to GNU linkers, lld supports a variant of `-u`
+// which accepts wildcard patterns. All symbols that match a given
+// pattern are handled as if they were given by `-u`.
+static void handleUndefinedGlob(StringRef Arg) {
+  Expected<GlobPattern> Pat = GlobPattern::create(Arg);
+  if (!Pat) {
+    error("--undefined-glob: " + toString(Pat.takeError()));
+    return;
+  }
+
+  std::vector<Symbol *> Syms;
+  Symtab->forEachSymbol([&](Symbol *Sym) {
+    // Calling Sym->fetch() from here is not safe because it may
+    // add new symbols to the symbol table, invalidating the
+    // current iterator. So we just keep a note.
+    if (Pat->match(Sym->getName()))
+      Syms.push_back(Sym);
+  });
+
+  for (Symbol *Sym : Syms)
+    handleUndefined(Sym);
+}
+
 static void handleLibcall(StringRef Name) {
   Symbol *Sym = Symtab->find(Name);
   if (!Sym || !Sym->isLazy())
     return;
 
   MemoryBufferRef MB;
   if (auto *LO = dyn_cast<LazyObject>(Sym))
     MB = LO->File->MB;
Config->HasDynSymTab =
       !SharedFiles.empty() || Config->Pic || Config->ExportDynamic;
 
   // Some symbols (such as __ehdr_start) are defined lazily only when there
   // are undefined symbols for them, so we add these to trigger that logic.
   for (StringRef Name : Script->ReferencedSymbols)
     addUndefined(Name);
 
   // Handle the `--undefined <sym>` options.
-  for (StringRef S : Config->Undefined)
-    handleUndefined(S);
+  for (StringRef Arg : Config->Undefined)
+    if (Symbol *Sym = Symtab->find(Arg))
+      handleUndefined(Sym);
 
   // If an entry symbol is in a static archive, pull out that file now.
-  handleUndefined(Config->Entry);
+  if (Symbol *Sym = Symtab->find(Config->Entry))
+    handleUndefined(Sym);
+
+  // Handle the `--undefined-glob <pattern>` options.
+  for (StringRef Pat : args::getStrings(Args, OPT_undefined_glob))
+    handleUndefinedGlob(Pat);
 
   // If any of our inputs are bitcode files, the LTO code generator may create
   // references to certain library functions that might not be explicit in the
   // bitcode file's symbol table. If any of those library functions are defined
   // in a bitcode file in an archive member, we need to arrange to use LTO to
   // compile those archive members by adding them to the link beforehand.
   //
   // However, adding all libcall symbols to the link can have undesired

lld/trunk/ELF/Options.td

 
 def trace: F<"trace">, HelpText<"Print the names of the input files">;
 
 defm trace_symbol: Eq<"trace-symbol", "Trace references to symbols">;
 
 defm undefined: Eq<"undefined", "Force undefined symbol during linking">,
   MetaVarName<"<symbol>">;
 
+defm undefined_glob: Eq<"undefined-glob", "Force undefined symbol during linking">,
+  MetaVarName<"<pattern>">;
+
 defm unresolved_symbols:
   Eq<"unresolved-symbols", "Determine how to handle unresolved symbols">;
 
 defm undefined_version: B<"undefined-version",
   "Allow unused version in version script (default)",
   "Report version scripts that refer undefined symbols">;
 
 defm rsp_quoting: Eq<"rsp-quoting", "Quoting style for response files">,

lld/trunk/docs/ld.lld.1

 Run the linker multi-threaded.
 This option is enabled by default.
 .It Fl -trace
 Print the names of the input files.
 .It Fl -trace-symbol Ns = Ns Ar symbol , Fl y Ar symbol
 Trace references to
 .Ar symbol .
 .It Fl -undefined Ns = Ns Ar symbol , Fl u Ar symbol
-Force
+If
 .Ar symbol
-to be an undefined symbol during linking.
+is not defined after symbol resolution, and there's a static library
+that contains an object file defining the symbol, load the member
+to include the object file in the output file.
+.It Fl -undefined-glob Ns = Ns Ar pattern
+Synonym for
+.Fl -undefined ,
+except that it takes a glob pattern. In a glob pattern,
+.Cm *
+matches zero or more characters,
+.Cm ?
+matches any single character, and
+.Cm [...]
+matches the characters within brackets. All symbols that match
+a given pattern are handled as if they were given as arguments of
+.Fl -undefined .
 .It Fl -unresolved-symbols Ns = Ns Ar value
 Determine how to handle unresolved symbols.
 .It Fl -use-android-relr-tags
 Use SHT_ANDROID_RELR / DT_ANDROID_RELR* tags instead of SHT_RELR / DT_RELR*.
 .It Fl v
 Display the version number and proceed with linking if object files are
 specified.
 .It Fl V , Fl -version

lld/trunk/test/ELF/undefined-glob.s

+# REQUIRES: x86
+
+# RUN: echo '.globl foo1; foo1:' | \
+# RUN:   llvm-mc -filetype=obj -triple=x86_64-unknown-linux - -o %t1.o
+# RUN: echo '.globl foo2; foo2:' | \
+# RUN:   llvm-mc -filetype=obj -triple=x86_64-unknown-linux - -o %t2.o
+# RUN: echo '.globl foo32; foo32:' | \
+# RUN:   llvm-mc -filetype=obj -triple=x86_64-unknown-linux - -o %t3.o
+# RUN: echo '.globl bar; bar:' | \
+# RUN:   llvm-mc -filetype=obj -triple=x86_64-unknown-linux - -o %t4.o
+# RUN: rm -f %t.a
+# RUN: llvm-ar rcs %t.a %t1.o %t2.o %t3.o %t4.o
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t.o
+
+# RUN: ld.lld -o %t.exe %t.o %t.a
+# RUN: llvm-readobj --symbols %t.exe | FileCheck --check-prefix=NO-OPT %s
+
+# NO-OPT-NOT: foo
+# NO-OPT-NOT: bar
+
+# RUN: ld.lld -o %t.exe %t.o %t.a --undefined-glob foo1
+# RUN: llvm-readobj --symbols %t.exe | FileCheck --check-prefix=FOO1 %s
+
+# FOO1: foo1
+# FOO1-NOT: foo2
+
+# RUN: ld.lld -o %t.exe %t.o %t.a --undefined-glob 'foo*'
+# RUN: llvm-readobj --symbols %t.exe | FileCheck --check-prefix=FOO-STAR %s
+
+# FOO-STAR: foo1
+# FOO-STAR: foo2
+# FOO-STAR: foo32
+# FOO-STAR-NOT: bar
+
+# RUN: ld.lld -o %t.exe %t.o %t.a --undefined-glob 'foo?'
+# RUN: llvm-readobj --symbols %t.exe | FileCheck --check-prefix=FOO-Q %s
+
+# FOO-Q: foo1
+# FOO-Q: foo2
+# FOO-Q-NOT: foo32
+# FOO-Q-NOT: bar
+
+# RUN: ld.lld -o %t.exe %t.o %t.a --undefined-glob 'foo[13]*'
+# RUN: llvm-readobj --symbols %t.exe | FileCheck --check-prefix=FOO13 %s
+
+# FOO13: foo1
+# FOO13-NOT: foo2
+# FOO13: foo32
+# FOO13-NOT: bar
+
+# RUN: not ld.lld -o %t.exe %t.o %t.a --undefined-glob '[' 2>&1 | \
+# RUN:   FileCheck -check-prefix=BAD-PATTERN %s
+
+# BAD-PATTERN: --undefined-glob: invalid glob pattern: [
+
+.globl _start
+_start: