[ELF] Create and export symbols provided by a linker script if they referenced by DSOs.

[ikudrin]

Summary

It should be possible to resolve undefined symbols in dynamic libraries
using symbols defined in a linker script.

[grimar]

ELF/InputFiles.h

347

It seems like instead of adding global variable you can make helper that
would scan over SharedFiles and invoke SharedFile::getUndefinedSymbols
to build the same set and then pass it to shouldDefineSym ?

[ikudrin]

ELF/InputFiles.h

347

We call shouldDefineSym not only in declareSymbols but also in addSymbol, which might be called several times. As a result, the set itself would be created several times.

Moreover, SharedFiles requires a template argument, so all involved methods would become templates too.

I thought about keeping PROVIDE symbols which are not created during declareSymbols call. SymbolTable::scanShlibUndefined would then ask LinkerScript to create referenced symbols. However, it looks like blurring of responsibility to me.

[grimar]

ELF/InputFiles.h

347

Probably getUndefinedSymbols can be moved to InputFile with corresponding member.
And then it can assert that it is called only for SharedKind, like we do in other methods there.
We already moved some functionality to InputFile to avoid dealing with templates before.

I do not think building of such set is very expensive and so it should be ok to do that twice.
In processSectionCommands and in declareSymbols.

It is also seems that after landing D43008 it should be possible to remove call from 'addSymbol`.
Since declareSymbols sets Cmd->Sym, I think addSymbol can probably do

if (!Cmd->Sym)
  return;

instead of

if (!shouldDefineSym(Cmd))
  return;

I did not try it though. But at least such approach avoids introducing one more global variable
and looks more like a correct direction IMO.

[ikudrin]

• Move Undefs and getUndefinedSymbols() into InputFile
• Change type of Undefs to DenseSet<StringRef> so that it can be used to search for the symbol without creating another filter container.

[grimar]

This LGTM, thanks (minor nit below).
Please wait for approval from other reviewers.

ELF/LinkerScript.cpp

134

You should be able to use count I believe:

for (InputFile *F : SharedFiles)
  if (F->getUndefinedSymbols().count(Cmd->Name))
    return true;

[ikudrin]

• Use DenseSet<>::count(), as George suggested.

ELF/Driver.cpp

template <class ELFT> void LinkerDriver::link(opt::InputArgList &Args) {
   // to complete the symbol table. After this, no new names except a
   // few linker-synthesized ones will be added to the symbol table.
   Symtab->fetchIfLazy<ELFT>(Config->Entry);
 
   // Return if there were name resolution errors.
   if (errorCount())
     return;
 
+  // We want to declare linker script's symbols early,
+  // so that we can version them.
+  // They also might be exported if referenced by DSOs.
+  Script->declareSymbols();
+
   // Handle undefined symbols in DSOs.
   if (!Config->Shared)
     Symtab->scanShlibUndefined<ELFT>();
 
   // Handle the -exclude-libs option.
   if (Args.hasArg(OPT_exclude_libs))
     excludeLibs<ELFT>(Args);
 
   // Create ElfHeader early. We need a dummy section in
   // addReservedSymbols to mark the created symbols as not absolute.
   Out::ElfHeader = make<OutputSection>("", 0, SHF_ALLOC);
   Out::ElfHeader->Size = sizeof(typename ELFT::Ehdr);
 
   // We need to create some reserved symbols such as _end. Create them.
   if (!Config->Relocatable)
     addReservedSymbols();
 
-  // We want to declare linker script's symbols early,
-  // so that we can version them.
-  Script->declareSymbols();
-
   // Apply version scripts.
   //
   // For a relocatable output, version scripts don't make sense, and
   // parsing a symbol version string (e.g. dropping "@ver1" from a symbol
   // name "foo@ver1") rather do harm, so we don't call this if -r is given.
   if (!Config->Relocatable)
     Symtab->scanVersionScript();
 

ELF/InputFiles.h

public:
   // Returns object file symbols. It is a runtime error to call this
   // function on files of other types.
   ArrayRef<Symbol *> getSymbols() {
     assert(FileKind == BinaryKind || FileKind == ObjKind ||
            FileKind == BitcodeKind);
     return Symbols;
   }
 
+  // Returns undefined symbols of a shared library.
+  // It is a runtime error to call this function on files of other types.
+  const llvm::DenseSet<StringRef> &getUndefinedSymbols() {
+    assert(FileKind == SharedKind);
+    return Undefs;
+  }
+
   // Filename of .a which contained this file. If this file was
   // not in an archive file, it is the empty string. We use this
   // string for creating error messages.
   std::string ArchiveName;
 
   // If this is an architecture-specific file, the following members
   // have ELF type (i.e. ELF{32,64}{LE,BE}) and target machine type.
   ELFKind EKind = ELFNoneKind;
   uint16_t EMachine = llvm::ELF::EM_NONE;
   uint8_t OSABI = 0;
 
   // Cache for toString(). Only toString() should use this member.
   mutable std::string ToStringCache;
 
   std::string getSrcMsg(const Symbol &Sym, InputSectionBase &Sec,
                         uint64_t Offset);
 
 protected:
   InputFile(Kind K, MemoryBufferRef M);
   std::vector<InputSectionBase *> Sections;
   std::vector<Symbol *> Symbols;
+  llvm::DenseSet<StringRef> Undefs;
 
 private:
   const Kind FileKind;
 };
 
 template <typename ELFT> class ELFFileBase : public InputFile {
 public:
   typedef typename ELFT::Shdr Elf_Shdr;
template <class ELFT> class SharedFile : public ELFFileBase<ELFT> {
   typedef ELFFileBase<ELFT> Base;
   typedef typename ELFT::Dyn Elf_Dyn;
   typedef typename ELFT::Shdr Elf_Shdr;
   typedef typename ELFT::Sym Elf_Sym;
   typedef typename ELFT::SymRange Elf_Sym_Range;
   typedef typename ELFT::Verdef Elf_Verdef;
   typedef typename ELFT::Versym Elf_Versym;
 
-  std::vector<StringRef> Undefs;
   const Elf_Shdr *VersymSec = nullptr;
   const Elf_Shdr *VerdefSec = nullptr;
 
 public:
   std::vector<const Elf_Verdef *> Verdefs;
   std::string SoName;
 
-  llvm::ArrayRef<StringRef> getUndefinedSymbols() { return Undefs; }
-
   static bool classof(const InputFile *F) {
     return F->kind() == Base::SharedKind;
   }
 
   SharedFile(MemoryBufferRef M, StringRef DefaultSoName);
 
   void parseSoName();
   void parseRest();

 InputFile *createObjectFile(MemoryBufferRef MB, StringRef ArchiveName = "",
                             uint64_t OffsetInArchive = 0);
 InputFile *createSharedFile(MemoryBufferRef MB, StringRef DefaultSoName);
 
 extern std::vector<BinaryFile *> BinaryFiles;
 extern std::vector<BitcodeFile *> BitcodeFiles;
 extern std::vector<InputFile *> ObjectFiles;
 extern std::vector<InputFile *> SharedFiles;
 
 } // namespace elf
 } // namespace lld
 
 #endif

ELF/InputFiles.cpp

if (Versym) {
       VersymIndex = Versym->vs_index;
       ++Versym;
     }
     bool Hidden = VersymIndex & VERSYM_HIDDEN;
     VersymIndex = VersymIndex & ~VERSYM_HIDDEN;
 
     StringRef Name = CHECK(Sym.getName(this->StringTable), this);
     if (Sym.isUndefined()) {
-      Undefs.push_back(Name);
+      this->Undefs.insert(Name);
       continue;
     }
 
     if (Sym.getBinding() == STB_LOCAL) {
       warn("found local symbol '" + Name +
            "' in global part of symbol table in file " + toString(this));
       continue;
     }

ELF/LinkerScript.cpp

if (Cmd->Name == ".")
     return false;
 
   if (!Cmd->Provide)
     return true;
 
   // If a symbol was in PROVIDE(), we need to define it only
   // when it is a referenced undefined symbol.
   Symbol *B = Symtab->find(Cmd->Name);
-  if (!B || B->isDefined())
-    return false;
+  if (B && !B->isDefined())
+    return true;
+  // It might also be referenced by a DSO.
+  for (InputFile *F : SharedFiles)
+    if (F->getUndefinedSymbols().count(Cmd->Name))
       return true;
+  return false;
 }
 
 // This function is called from processSectionCommands,
 // while we are fixing the output section layout.
 void LinkerScript::addSymbol(SymbolAssignment *Cmd) {
   if (!shouldDefineSym(Cmd))
     return;
 

ELF/SymbolTable.cpp

 // the user program (not the other way, which is usual). Shared libraries
 // may have undefined symbols, expecting that the user program provides
 // the definitions for them. An example is BSD's __progname symbol.
 // We need to put such symbols to the main program's .dynsym so that
 // shared libraries can find them.
 // Except this, we ignore undefined symbols in DSOs.
 template <class ELFT> void SymbolTable::scanShlibUndefined() {
   for (InputFile *F : SharedFiles) {
-    for (StringRef U : cast<SharedFile<ELFT>>(F)->getUndefinedSymbols()) {
+    for (StringRef U : F->getUndefinedSymbols()) {
       Symbol *Sym = find(U);
       if (!Sym)
         continue;
       if (auto *L = dyn_cast<Lazy>(Sym))
         if (InputFile *File = L->fetch())
           addFile<ELFT>(File);
 
       if (Sym->isDefined())

test/ELF/linkerscript/Inputs/provide-shared2.s

+.global foo
+.data
+.dc.a foo

test/ELF/linkerscript/provide-shared2.s

+# REQUIRES: x86
+# RUN: llvm-mc -filetype=obj -triple=x86_64-pc-linux %s -o %t.o
+# RUN: llvm-mc -filetype=obj -triple=x86_64-pc-linux %p/Inputs/provide-shared2.s -o %t2.o
+# RUN: ld.lld %t2.o -o %t2.so -shared
+# RUN: echo "SECTIONS { . = . + SIZEOF_HEADERS; PROVIDE(foo = 42); }" > %t.script
+# RUN: ld.lld -o %t --script %t.script %t.o %t2.so
+# RUN: llvm-readelf --dyn-symbols %t | FileCheck  %s
+
+# CHECK: 1 1: 000000000000002a 0 NOTYPE GLOBAL DEFAULT ABS foo@
+
+.global _start
+_start:
+        nop