Add --undefined-glob which is an --undefined with wildcard pattern match.

[ruiu]

Summary

This patch adds new command line option --undefined-glob to lld.
That option is a variant of --undefined but accepts wildcard
patterns so that all symbols that match with a given pattern are
handled as if they were given by -u.

-u foo is to force resolve symbol foo if foo is not a defined symbol
and there's a static archive that contains a definition of symbol foo.

Now, you can specify a wildcard pattern as an argument for --undefined-glob.
So, if you want to include all JNI symbols (which start with "Java_"), you
can do that by passing --undefined-glob "Java_*" to the linker, for example.

In this patch, I use the same glob pattern matcher as the version script
processor is using, so it does not only support * but also ? and [...].

Discussion thread: http://lists.llvm.org/pipermail/llvm-dev/2019-June/132961.html

[MaskRay]

lld/ELF/Driver.cpp

1327

clang-format.

here not -> here is not

1343

Name.find_first_of("?*[")

[ruiu]

• address review comments

[peter.smith]

Looks good to me as well.

lld/ELF/Driver.cpp

1335

Might be worth extracting this block, and the equivalent one in handleUndefined into a separate function. That way the action to take to once a symbol has been selected by a pattern is always the same in both functions, including the comment about LTO. I've not got a strong opinion though as I don't expect much change.

[ruiu]

• factor out common code and unify two functions into one

[grimar]

LGTM. Minor suggestion is inlined.

lld/test/ELF/undefined-wildcard.s

18

hint: you should be able to use ----implicit-check-not=foo --implicit-check-not=bar
This way can probably help to make checks below a bit simpler.

[jyknight]

Actually linker scripts do support metacharacter escaping (although, it seems lld does not support this correctly, yet).

E.g., this should match only a function named literally "?"

VER {
  global:
   "?";
};

While this matches any one-letter function:

VER {
  global:
   ?;
};

So perhaps this functionality should be given its own command-line argument, e.g. --undefined-glob, so as to not cause any such confusion.

[ruiu]

In D63244#1541879, @jyknight wrote:

Actually linker scripts do support metacharacter escaping (although, it seems lld does not support this correctly, yet).

E.g., this should match only a function named literally "?"

VER {
  global:
   "?";
};

While this matches any one-letter function:

VER {
  global:
   ?;
};

So perhaps this functionality should be given its own command-line argument, e.g. --undefined-glob, so as to not cause any such confusion.

I think I like the idea to add this as a new option, instead of changing the meaning of the existing option in a subtle way. I'll give it a new name as you suggested.

[ruiu]

• rename the option --undefined-glob.

[ruiu]

• update comment

[MaskRay]

The code change looks good. Nit about the help text.

lld/docs/ld.lld.1

509–510

I was puzzled when I first noticed this option and its help text: Force to be an undefined symbol during linking.

If the symbol does not exist, there is no undefined symbol in the link result.
If the symbol is defined, this option does not make the symbol undefined.

Now that you are adding a similar option, maybe revise the documentation a bit to address the confusion?

[ruiu]

• updated the man page

lld/ELF/Driver.cpp

 #include "lld/Common/TargetOptionsCommandFlags.h"
 #include "lld/Common/Threads.h"
 #include "lld/Common/Version.h"
 #include "llvm/ADT/SetVector.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/StringSwitch.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Compression.h"
+#include "llvm/Support/GlobPattern.h"
 #include "llvm/Support/LEB128.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/TarWriter.h"
 #include "llvm/Support/TargetSelect.h"
 #include "llvm/Support/raw_ostream.h"
 #include <cstdlib>
 #include <utility>
 
static void excludeLibs(opt::InputArgList &Args) {
 
   for (InputFile *File : ObjectFiles)
     Visit(File);
 
   for (BitcodeFile *File : BitcodeFiles)
     Visit(File);
 }
 
+// A helper function for -u to do wildcard pattern match.
+static void handleUndefinedWildcard(const GlobPattern &Pat) {
+  std::vector<Symbol *> Syms;
+
+  Symtab->forEachSymbol([&](Symbol *Sym) {
+    // Calling Sym->fetch() from here is not safe as it may add new
+    // symbols to the symbol table, invalidating the current iterator.
+    // So we just keep a note.
+    if (Pat.match(Sym->getName()))
+      Syms.push_back(Sym);
+  });
+
+  for (Symbol *Sym : Syms) {
+    Sym->IsUsedInRegularObj = true;
+    if (Sym->isLazy())
+      Sym->fetch();
+  }
+}
+
 // Force Sym to be entered in the output. Used for -u or equivalent.
 static void handleUndefined(StringRef Name) {
+  if (Name.find_first_of("*?[")) {
+    Expected<GlobPattern> Pat = GlobPattern::create(Name);
+    if (Pat) {
+      handleUndefinedWildcard(*Pat);
+      return;
+    }
+  }
+
   Symbol *Sym = Symtab->find(Name);
   if (!Sym)
     return;
 
   // Since symbol S may not be used inside the program, LTO may
   // eliminate it. Mark the symbol as "used" to prevent it.
   Sym->IsUsedInRegularObj = true;
 

lld/docs/ld.lld.1

 .It Fl -trace
 Print the names of the input files.
 .It Fl -trace-symbol Ns = Ns Ar symbol , Fl y Ar symbol
 Trace references to
 .Ar symbol .
 .It Fl -undefined Ns = Ns Ar symbol , Fl u Ar symbol
 Force
 .Ar symbol
-to be an undefined symbol during linking.
+to be an undefined symbol during linking. You can use wildcard patterns in
+.Ar symbol .
 .It Fl -unresolved-symbols Ns = Ns Ar value
 Determine how to handle unresolved symbols.
 .It Fl -use-android-relr-tags
 Use SHT_ANDROID_RELR / DT_ANDROID_RELR* tags instead of SHT_RELR / DT_RELR*.
 .It Fl v
 Display the version number and proceed with linking if object files are
 specified.
 .It Fl V , Fl -version

lld/test/ELF/undefined-wildcard.s

+# REQUIRES: x86
+
+# RUN: echo '.globl foo1; foo1:' | \
+# RUN:   llvm-mc -filetype=obj -triple=x86_64-unknown-linux - -o %t1.o
+# RUN: echo '.globl foo2; foo2:' | \
+# RUN:   llvm-mc -filetype=obj -triple=x86_64-unknown-linux - -o %t2.o
+# RUN: echo '.globl foo32; foo32:' | \
+# RUN:   llvm-mc -filetype=obj -triple=x86_64-unknown-linux - -o %t3.o
+# RUN: echo '.globl bar; bar:' | \
+# RUN:   llvm-mc -filetype=obj -triple=x86_64-unknown-linux - -o %t4.o
+# RUN: rm -f %t.a
+# RUN: llvm-ar rcs %t.a %t1.o %t2.o %t3.o %t4.o
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t.o
+
+# RUN: ld.lld -o %t.exe %t.o %t.a
+# RUN: llvm-readobj --symbols %t.exe | FileCheck --check-prefix=NO-OPT %s
+
+# NO-OPT-NOT: foo
+# NO-OPT-NOT: bar
+
+# RUN: ld.lld -o %t.exe %t.o %t.a -u foo1
+# RUN: llvm-readobj --symbols %t.exe | FileCheck --check-prefix=FOO1 %s
+
+# FOO1: foo1
+# FOO1-NOT: foo2
+
+# RUN: ld.lld -o %t.exe %t.o %t.a -u 'foo*'
+# RUN: llvm-readobj --symbols %t.exe | FileCheck --check-prefix=FOO-STAR %s
+
+# FOO-STAR: foo1
+# FOO-STAR: foo2
+# FOO-STAR: foo32
+# FOO-STAR-NOT: bar
+
+# RUN: ld.lld -o %t.exe %t.o %t.a -u 'foo?'
+# RUN: llvm-readobj --symbols %t.exe | FileCheck --check-prefix=FOO-Q %s
+
+# FOO-Q: foo1
+# FOO-Q: foo2
+# FOO-Q-NOT: foo32
+# FOO-Q-NOT: bar
+
+# RUN: ld.lld -o %t.exe %t.o %t.a -u 'foo[13]*'
+# RUN: llvm-readobj --symbols %t.exe | FileCheck --check-prefix=FOO13 %s
+
+# FOO13: foo1
+# FOO13-NOT: foo2
+# FOO13: foo32
+# FOO13-NOT: bar
+
+.globl _start
+_start: