[ELF] For relative paths in INPUT() and GROUP(), search the directory of the current linker script before searching other paths

[MaskRay]

Summary

For a relative path in INPUT() or GROUP(), this patch changes the search order by adding the directory of the current linker script.
The new search order (consistent with GNU ld >= 2.35 regarding the new test test/ELF/input-relative.s):

1. the directory of the current linker script (GNU ld from Binutils 2.35 onwards; https://sourceware.org/bugzilla/show_bug.cgi?id=25806)
2. the current working directory
3. library paths (-L)

This behavior makes it convenient to replace a .so or .a with a linker script with additional input. For example, glibc

% cat /usr/lib/x86_64-linux-gnu/libm.a
/* GNU ld script
*/
OUTPUT_FORMAT(elf64-x86-64)
GROUP ( /usr/lib/x86_64-linux-gnu/libm-2.29.a /usr/lib/x86_64-linux-gnu/libmvec.a )

could be simplified as GROUP(libm-2.29.a libmvec.a).

Another example is to make libc++.a a linker script:

INPUT(libc++.a.1 libc++abi.a)

[MaskRay]

😷 -> work from 🏠

[psmith]

The feature seems reasonable to me. It is a potentially breaking change for someone that had two libraries called libfoo.a.1 one in CWD and one adjacent to libfoo.a (where libfoo.a is a linker script with INPUT(libfooa.a.1)). As I understand it we'd prefer the libfoo.a.1 from the same directory as libfoo over the version in CWD. I think that in this case it is very unlikely so a release note entry and documentation (LinkerScript.rst?) would be sufficient.

Will be worth making sure that if BFD follow a slightly different route then we synchronize with them rather than gold as I think BFD is more widely used in practice.

Given it is Easter this week and people may be on holiday, will be worth waiting a bit for further comments, if we don't get any more I'll be happy to approve next week.

lld/ELF/ScriptParser.cpp

291

Now that the rules are getting a bit more complicated. It would be good to have a comment like Gold that explains them.

[MaskRay]

Add comments.

Will be worth making sure that if BFD follow a slightly different route then we synchronize with them rather than gold as I think BFD is more widely used in practice.

Thanks for the advice. I'll try to make the linkers align on this feature. I filed a feature request (https://sourceware.org/bugzilla/show_bug.cgi?id=25806) to get binutils maintainers' attention... Just created a GNU ld patch by myself..

[MaskRay]

Improve tests

[MaskRay]

In D77779#1991477, @MaskRay wrote:

Add comments.

Will be worth making sure that if BFD follow a slightly different route then we synchronize with them rather than gold as I think BFD is more widely used in practice.

Thanks for the advice. I'll try to make the linkers align on this feature. I filed a feature request (https://sourceware.org/bugzilla/show_bug.cgi?id=25806) to get binutils maintainers' attention... Just created a GNU ld patch by myself..

WoW, On GNU ld side, Nick applied my patch (https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=161719466ac9ea5f186514312f6bce842181804f). My mouthful description was lost, though...

[psmith]

LGTM. Thanks for the updates and congratulations on getting the binutils patch accepted.

lld/ELF/ScriptLexer.h

public:
   bool consumeLabel(StringRef tok);
   std::string getCurrentLocation();
 
   std::vector<MemoryBufferRef> mbs;
   std::vector<StringRef> tokens;
   bool inExpr = false;
   size_t pos = 0;
 
+protected:
+  MemoryBufferRef getCurrentMB();
+
 private:
   void maybeSplitExpr();
   StringRef getLine();
   size_t getLineNumber();
   size_t getColumnNumber();
-
-  MemoryBufferRef getCurrentMB();
 };
 
 } // namespace elf
 } // namespace lld
 
 #endif

lld/ELF/ScriptParser.cpp

void ScriptParser::addFile(StringRef s) {
   if (isUnderSysroot && s.startswith("/")) {
     SmallString<128> pathData;
     StringRef path = (config->sysroot + s).toStringRef(pathData);
     if (sys::fs::exists(path)) {
       driver->addFile(saver.save(path), /*withLOption=*/false);
       return;
     }
   }
 
   if (s.startswith("/")) {
+    // Case 1: s is an absolute path. Just open it.
     driver->addFile(s, /*withLOption=*/false);
   } else if (s.startswith("=")) {
+    // Case 2: relative to the sysroot.
     if (config->sysroot.empty())
       driver->addFile(s.substr(1), /*withLOption=*/false);
     else
       driver->addFile(saver.save(config->sysroot + "/" + s.substr(1)),
                       /*withLOption=*/false);
   } else if (s.startswith("-l")) {
+    // Case 3: search in the list of library paths.
     driver->addLibrary(s.substr(2));
-  } else if (sys::fs::exists(s)) {
+  } else {
+    // Case 4: s is a relative path. Search in the directory of the script file.
+    std::string filename = std::string(getCurrentMB().getBufferIdentifier());
+    StringRef directory = sys::path::parent_path(filename);
+    if (!directory.empty()) {
+      SmallString<0> path(directory);
+      sys::path::append(path, s);
+      if (sys::fs::exists(path)) {
+        driver->addFile(path, /*withLOption=*/false);
+        return;
+      }
+    }
+    // Then search in the current working directory.
+    if (sys::fs::exists(s)) {
       driver->addFile(s, /*withLOption=*/false);
     } else {
+      // Finally, search in the list of library paths.
       if (Optional<std::string> path = findFromSearchPaths(s))
         driver->addFile(saver.save(*path), /*withLOption=*/true);
       else
         setError("unable to find " + s);
     }
   }
+}
 
 void ScriptParser::readAsNeeded() {
   expect("(");
   bool orig = config->asNeeded;
   config->asNeeded = true;
   while (!errorCount() && !consume(")"))
     addFile(unquote(next()));
   config->asNeeded = orig;

lld/test/ELF/linkerscript/input-relative.s

+# REQUIRES: x86
+## For a relative pathname in INPUT() or GROUP(), the parent directory of
+## the current linker script has priority over current working directory and -L.
+
+# RUN: rm -rf %t.dir
+# RUN: mkdir %t.dir
+# RUN: cd %t.dir
+
+# RUN: mkdir dir
+# RUN: llvm-mc -filetype=obj -triple=x86_64 %s -o a.o
+# RUN: echo '.globl b, cwd; b: cwd:' | llvm-mc -filetype=obj -triple=x86_64 - -o b.o
+# RUN: echo '.globl b, dir; b: dir:' | llvm-mc -filetype=obj -triple=x86_64 - -o dir/b.o
+# RUN: llvm-ar rc libb.a b.o
+# RUN: llvm-ar rc dir/libb.a dir/b.o
+
+## A relative pathname is relative to the parent directory of the current linker script.
+## The directory has priority over current working directory and -L.
+# RUN: echo 'INPUT(libb.a)' > dir/relative.lds
+# RUN: ld.lld -L. a.o dir/relative.lds -o relative
+# RUN: llvm-nm relative | FileCheck --check-prefix=DIR %s
+## GROUP() uses the same search order.
+# RUN: echo 'GROUP(libb.a)' > dir/relative1.lds
+# RUN: ld.lld -L. a.o dir/relative1.lds -o relative1
+# RUN: llvm-nm relative1 | FileCheck --check-prefix=DIR %s
+
+# DIR: T dir
+
+## -l does not use the special rule.
+# RUN: echo 'INPUT(-lb)' > dir/cwd.lds
+# RUN: ld.lld -L. a.o dir/cwd.lds -o cwd
+# RUN: llvm-nm cwd | FileCheck --check-prefix=CWD %s
+# RUN: echo 'GROUP(-lb)' > dir/cwd1.lds
+# RUN: ld.lld -L. a.o dir/cwd1.lds -o cwd1
+# RUN: llvm-nm cwd1 | FileCheck --check-prefix=CWD %s
+
+# CWD: T cwd
+
+## If the parent directory of the current linker script does not contain the file,
+## fall back to the current working directory.
+# RUN: cp libb.a libc.a
+# RUN: echo 'INPUT(libc.a)' > dir/fallback.lds
+# RUN: ld.lld a.o dir/fallback.lds -o /dev/null
+
+.globl _start
+_start:
+  call b