[ELF] Fix the file look up algorithm used in the linker script GROUP command.

[atanasyan]

Summary

In general the linker scripts' GROUP command works like a pair of command line options --start-group/--end-group. But there is a difference in the files look up algorithm.

The --start-group/--end-group commands use a trivial approach:
a) If the path has '-l' prefix, add 'lib' prefix and '.a'/'.so' suffix and search the path through library search directories.
b) Otherwise, use the path 'as-is'.

The GROUP command implements more compicated approach:
a) If the path has '-l' prefix, add 'lib' prefix and '.a'/'.so' suffix and search the path through library search directories.
b) If the path does not have '-l' prefix, and sysroot is configured, and the path starts with the / character, and the script being processed is located inside the sysroot, search the path under the sysroot. Otherwise, try to open the path in the current directory. If it is not found, search through library search directories.

https://www.sourceware.org/binutils/docs-2.24/ld/File-Commands.html

[shankarke]

include/lld/ReaderWriter/ELFLinkingContext.h

176–180

There is also a third thing, to support namespaces. You could refer to -l:mylib.a, the linker would search by not adding a prefix 'lib'.

lib/Driver/GnuLdInputGraph.cpp

76–86

can we just use realpath and compare if the file is a prefix of another ?

95

we should probably reject sysroot being empty during parsing command line options, that way we can get rid of empty() comparison.

[atanasyan]

include/lld/ReaderWriter/ELFLinkingContext.h

176–180

I did not plan to support namespaces in this fix. But now I think it is a good idea to cover all cases by this patch. Will implement this soon.

lib/Driver/GnuLdInputGraph.cpp

76–86

I use the llvm::sys::fs::equivalent() to make this code a bit more platform independent. As far as I understand on Windows it is possible to get sysroot = 'C:\work_folder\SysRoot' and path = 'c:\work_folder\sysroot\file_name'. So on Windows we have to use case insensitive string comparison and case sensitive string comparison on Unix.

95

I check that the sysroot is not empty here to recognize a valid case when a user does not provide --sysroot option at all.

[ruiu]

LGTM

[atanasyan]

Thanks for review.

Closed by commit rL207769 (authored by @atanasyan)

include/lld/ReaderWriter/ELFLinkingContext.h

176–180

Namespace support makes this patch too big. For example, now we cannot correctly parse namespace in a linker script at all. So I will implement this in a separate patch.

include/lld/Driver/GnuLdInputGraph.h

 public:
   /// \brief The attributes class provides a way for a input file to look into
   /// all the positional attributes that were specified in the command line.
   /// There are few positional operators and the number of arguments to the
   /// ELFFileNode class keeps growing. This achieves code to be clean as well.
   class Attributes {
   public:
     Attributes()
-        : _isWholeArchive(false), _asNeeded(false), _isDashlPrefix(false) {}
+        : _isWholeArchive(false), _asNeeded(false), _isDashlPrefix(false),
+          _isSysRooted(false) {}
     void setWholeArchive(bool isWholeArchive) {
       _isWholeArchive = isWholeArchive;
     }
     void setAsNeeded(bool asNeeded) { _asNeeded = asNeeded; }
     void setDashlPrefix(bool isDashlPrefix) { _isDashlPrefix = isDashlPrefix; }
+    void setSysRooted(bool isSysRooted) { _isSysRooted = isSysRooted; }
 
   public:
     bool _isWholeArchive;
     bool _asNeeded;
     bool _isDashlPrefix;
+    bool _isSysRooted;
   };
 
   ELFFileNode(ELFLinkingContext &ctx, StringRef path, Attributes &attributes)
       : FileNode(path), _elfLinkingContext(ctx), _attributes(attributes) {}
 
   ErrorOr<StringRef> getPath(const LinkingContext &ctx) const override;
 
   /// \brief create an error string for printing purposes

include/lld/ReaderWriter/ELFLinkingContext.h

public:
   virtual void setOutputMagic(OutputMagic magic) { _outputMagic = magic; }
 
   /// \brief Disallow dynamic libraries during linking
   virtual void setNoAllowDynamicLibraries() { _noAllowDynamicLibraries = true; }
 
   /// Searches directories for a match on the input File
   ErrorOr<StringRef> searchLibrary(StringRef libName) const;
 
+  /// \brief Searches directories for a match on the input file.
+  /// If \p fileName is an absolute path and \p isSysRooted is true, check
+  /// the file under sysroot directory. If \p fileName is a relative path
+  /// and is not in the current directory, search the file through library
+  /// search directories.
+  ErrorOr<StringRef> searchFile(StringRef fileName, bool isSysRooted) const;
+
   /// Get the entry symbol name
   StringRef entrySymbolName() const override;
 
   /// add to the list of initializer functions
   void addInitFunction(StringRef name) { _initFunctions.push_back(name); }
 
   /// add to the list of finalizer functions
   void addFiniFunction(StringRef name) { _finiFunctions.push_back(name); }
public:
   range<const StringRef *> finiFunctions() const { return _finiFunctions; }
 
   void setSharedObjectName(StringRef soname) {
     _soname = soname;
   }
 
   StringRef sharedObjectName() const { return _soname; }
 
+  StringRef getSysroot() const { return _sysrootPath; }
+
   /// \brief Set path to the system root
   void setSysroot(StringRef path) {
     _sysrootPath = path;
   }
 
   void addRpath(StringRef path) {
    _rpathList.push_back(path);
   }

lib/Driver/GnuLdDriver.cpp

static bool parseDefsymAsAlias(StringRef opt, StringRef &sym,
   if (equalPos == 0 || equalPos == StringRef::npos)
     return false;
   sym = opt.substr(0, equalPos);
   target = opt.substr(equalPos + 1);
   return !target.empty();
 }
 
 llvm::ErrorOr<StringRef> ELFFileNode::getPath(const LinkingContext &) const {
-  if (!_attributes._isDashlPrefix)
-    return _path;
+  if (_attributes._isDashlPrefix)
     return _elfLinkingContext.searchLibrary(_path);
+  return _elfLinkingContext.searchFile(_path, _attributes._isSysRooted);
 }
 
 std::string ELFFileNode::errStr(error_code errc) {
   if (errc == llvm::errc::no_such_file_or_directory) {
     if (_attributes._isDashlPrefix)
       return (Twine("Unable to find library -l") + _path).str();
     return (Twine("Unable to find file ") + _path).str();
   }

lib/Driver/GnuLdInputGraph.cpp

 //===- lib/Driver/GnuLdInputGraph.cpp -------------------------------------===//
 //
 //                             The LLVM Linker
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "lld/Driver/GnuLdInputGraph.h"
 #include "lld/ReaderWriter/LinkerScript.h"
 
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/Path.h"
+
 using namespace lld;
 
 /// \brief Parse the input file to lld::File.
 error_code ELFFileNode::parse(const LinkingContext &ctx,
                               raw_ostream &diagnostics) {
   ErrorOr<StringRef> filePath = getPath(ctx);
   if (error_code ec = filePath.getError())
     return ec;
error_code GNULdScript::parse(const LinkingContext &ctx,
   _linkerScript = _parser->parse();
 
   if (!_linkerScript)
     return LinkerScriptReaderError::parse_error;
 
   return error_code::success();
 }
 
+static bool isPathUnderSysroot(StringRef sysroot, StringRef path) {
+  // TODO: Handle the case when 'sysroot' and/or 'path' are symlinks.
+  if (sysroot.empty() || sysroot.size() >= path.size())
+    return false;
+  if (llvm::sys::path::is_separator(sysroot.back()))
+    sysroot = sysroot.substr(0, sysroot.size() - 1);
+  if (!llvm::sys::path::is_separator(path[sysroot.size()]))
+    return false;
+
+  return llvm::sys::fs::equivalent(sysroot, path.substr(0, sysroot.size()));
+}
+
 /// \brief Handle GnuLD script for ELF.
 error_code ELFGNULdScript::parse(const LinkingContext &ctx,
                                  raw_ostream &diagnostics) {
   ELFFileNode::Attributes attributes;
   if (error_code ec = GNULdScript::parse(ctx, diagnostics))
     return ec;
+  StringRef sysRoot = _elfLinkingContext.getSysroot();
+  if (!sysRoot.empty() && isPathUnderSysroot(sysRoot, *getPath(ctx)))
+    attributes.setSysRooted(true);
   for (const script::Command *c : _linkerScript->_commands) {
     auto *group = dyn_cast<script::Group>(c);
     if (!group)
       continue;
     std::unique_ptr<Group> groupStart(new Group());
     for (const script::Path &path : group->getPaths()) {
       // TODO : Propagate Set WholeArchive/dashlPrefix
       attributes.setAsNeeded(path._asNeeded);

lib/ReaderWriter/ELF/ELFLinkingContext.cpp

if (llvm::sys::fs::exists(path.str()))
       return StringRef(*new (_allocator) std::string(path.str()));
   }
   if (!llvm::sys::fs::exists(libName))
     return llvm::make_error_code(llvm::errc::no_such_file_or_directory);
 
   return libName;
 }
 
+ErrorOr<StringRef> ELFLinkingContext::searchFile(StringRef fileName,
+                                                 bool isSysRooted) const {
+  SmallString<128> path;
+  if (llvm::sys::path::is_absolute(fileName) && isSysRooted) {
+    path.assign(_sysrootPath);
+    path.append(fileName);
+    if (llvm::sys::fs::exists(path.str()))
+      return StringRef(*new (_allocator) std::string(path.str()));
+  } else if (llvm::sys::fs::exists(fileName))
+    return fileName;
+
+  if (llvm::sys::path::is_absolute(fileName))
+    return llvm::make_error_code(llvm::errc::no_such_file_or_directory);
+
+  for (StringRef dir : _inputSearchPaths) {
+    buildSearchPath(path, dir, _sysrootPath);
+    llvm::sys::path::append(path, fileName);
+    if (llvm::sys::fs::exists(path.str()))
+      return StringRef(*new (_allocator) std::string(path.str()));
+  }
+  return llvm::make_error_code(llvm::errc::no_such_file_or_directory);
+}
+
 void ELFLinkingContext::createInternalFiles(
     std::vector<std::unique_ptr<File>> &files) const {
   std::unique_ptr<SimpleFile> file(
       new SimpleFile("<internal file for --defsym>"));
   for (auto &i : getAbsoluteSymbols()) {
     StringRef sym = i.first;
     uint64_t val = i.second;
     file->addAtom(*(new (_allocator) CommandLineAbsoluteAtom(*file, sym, val)));

test/elf/Inputs/group-cmd-search-1.ls

+GROUP ( shared.so-x86-64 )

test/elf/Inputs/group-cmd-search-2.ls

+GROUP ( /shared.so-x86-64 )

test/elf/group-cmd-search.test

+/*
+  In general the linker scripts's GROUP command works like a pair
+  of command line options --start-group/--end-group. But there is
+  a difference in the files look up algorithm.
+
+  The --start-group/--end-group commands use a trivial approach:
+  a) If the path has '-l' prefix, add 'lib' prefix and '.a'/'.so'
+     suffix and search the path through library search directories.
+  b) Otherwise, use the path 'as-is'.
+
+  The GROUP command implements more compicated approach:
+  a) If the path has '-l' prefix, add 'lib' prefix and '.a'/'.so'
+     suffix and search the path through library search directories.
+  b) If the path does not have '-l' prefix, and sysroot is configured,
+     and the path starts with the / character, and the script being
+     processed is located inside the sysroot, search the path under
+     the sysroot. Otherwise, try to open the path in the current
+     directory. If it is not found, search through library search
+     directories.
+*/
+
+/*
+  This link should finish successfully. The --start-group/--end-group
+  contains an existing absolute path to the file.
+
+RUN: lld -flavor gnu -target x86_64 -shared \
+RUN:     -L%p/Inputs %p/Inputs/use-shared.x86-64 \
+RUN:     --start-group %p/Inputs/shared.so-x86-64 --end-group -o %t1
+*/
+
+/*
+  This link should fail with unknown input file format error.
+  There is no shared.so-x86-64 file in the current directory.
+
+RUN: not \
+RUN: lld -flavor gnu -target x86_64 -shared \
+RUN:     -L%p/Inputs %p/Inputs/use-shared.x86-64 \
+RUN:     --start-group shared.so-x86-64 --end-group -o %t2
+*/
+
+/*
+  This link should fail with unknown input file format error.
+  The absolute path /shared.so-x86-64 does not exist and the linker
+  should not attempt to search it under the sysroot directory.
+
+RUN: not \
+RUN: lld -flavor gnu -target x86_64 -shared --sysroot=%p/Inputs \
+RUN:     -L%p/Inputs %p/Inputs/use-shared.x86-64 \
+RUN:     --start-group /shared.so-x86-64 --end-group -o %t3
+*/
+
+/*
+  This link should finish successfully. The group-cmd-search-1.ls
+  script contains "GROUP ( shared.so-x86-64 )" command and the linker
+  has to search shared.so-x86-64 through the library search paths.
+
+RUN: lld -flavor gnu -target x86_64 -shared \
+RUN:     -L%p/Inputs %p/Inputs/use-shared.x86-64 \
+RUN:     %p/Inputs/group-cmd-search-1.ls -o %t4
+*/
+
+/*
+  This link should fail with unknown input file format error.
+  The group-cmd-search-2.ls script contains GROUP command with
+  a non-existing absolute path but there is no --sysroot argument.
+
+RUN: not \
+RUN: lld -flavor gnu -target x86_64 -shared \
+RUN:     -L%p/Inputs %p/Inputs/use-shared.x86-64 \
+RUN:     %p/Inputs/group-cmd-search-2.ls -o %t5
+*/
+
+/*
+  This link should finish successfully. The group-cmd-search-2.ls
+  script contains GROUP command with an absolute path and the sysroot
+  directory is provided. The linker has to search the absolute path
+  under the sysroot directory.
+
+RUN: lld -flavor gnu -target x86_64 -shared --sysroot=%p/Inputs \
+RUN:     -L%p/Inputs %p/Inputs/use-shared.x86-64 \
+RUN:     %p/Inputs/group-cmd-search-2.ls -o %t6
+*/
+
+/*
+  This link should fail with unknown input file format error.
+  The linker script from this file contains GROUP with an absolute
+  path which can be found under provided sysroot directory.
+  But the linker script itself is not under the sysroot.
+
+RUN: not \
+RUN: lld -flavor gnu -target x86_64 -shared --sysroot=%p/Inputs \
+RUN:     -L%p/Inputs %p/Inputs/use-shared.x86-64 \
+RUN:     %s -o %t7
+*/
+GROUP ( /shared.so-x86-64 )