Implement --just-symbols.

[ruiu]

Summary

Implement --just-symbols.

[mcgrathr]

This doesn't implement the -R switch name, which is the traditional interface from time immemorial. I can use --just-symbols, but you really should handle -R too.

Just copying symbol values will suffice for my case, but it's not really the right semantics. The rest of the ElfNN_Sym fields matter too.

This crashes on my case. I don't know what a trivial test case that would make it crash might look like, but I'll send you my actual case separately.

lld/ELF/Driver.cpp

1065 ↗

(On Diff #120497)

--just-symbols (double dash), better known in traditional Unix as -R.

1068 ↗

(On Diff #120497)

s/programs/program/

lld/ELF/InputFiles.cpp

1053 ↗

(On Diff #120497)

--just-symbols (double dash)

1055 ↗

(On Diff #120497)

The canonical semantics are to copy all the other fields as well, not just st_value.
Only st_shndx is replaced with SHN_ABS. The others are preserved (size, visibility, type, etc.).

lld/ELF/InputFiles.h

62 ↗

(On Diff #120497)

--just-symbols

lld/ELF/Options.td

150 ↗

(On Diff #120497)

--just-symbols is the GNU long option for this, but -R is the original switch.
Since -R has been overloaded for --rpath, the GNU implementations examine the argument file for -R and treat it as --rpath if it's a directory and as --just-symbols if not.

[mcgrathr]

lld/ELF/SymbolTable.cpp

145 ↗

(On Diff #120497)

Stray debugging output.

[ruiu]

lld/ELF/InputFiles.cpp

1055 ↗

(On Diff #120497)

Yeah that's true, but I wonder if real programs depend on it.

[ruiu]

• address review comments

[PkmX]

If the only input file is specified as the argument of --just-symbol, lld will error out (ld.bfd accepts this):

$ riscv64-unknown-linux-gnu-ld.lld -r --just-symbols foo.o -o bar.o
riscv64-unknown-linux-gnu-ld.lld: error: no input files
riscv64-unknown-linux-gnu-ld.lld: error: target emulation unknown: -m or at least one .o file required

I'd suggest adding a separate kind of InputFile whose symbols only gets added to the symbol table as absolute (but don't add its sections). I have a patch that implements this if you'd like to review it.

[grimar]

In D39348#950733, @PkmX wrote:

If the only input file is specified as the argument of --just-symbol, lld will error out (ld.bfd accepts this):

$ riscv64-unknown-linux-gnu-ld.lld -r --just-symbols foo.o -o bar.o
riscv64-unknown-linux-gnu-ld.lld: error: no input files
riscv64-unknown-linux-gnu-ld.lld: error: target emulation unknown: -m or at least one .o file required

I'd suggest adding a separate kind of InputFile whose symbols only gets added to the symbol table as absolute (but don't add its sections). I have a patch that implements this if you'd like to review it.

What is your use case to specify only single --just-symbol file ?

[PkmX]

In D39348#950848, @grimar wrote:

What is your use case to specify only single --just-symbol file ?

It can be used to produce an object file containing the offset of symbols in vdso:
https://github.com/riscv/riscv-linux/blob/ae64f9bd1d3621b5e60d7363bc20afb46aede215/arch/riscv/kernel/vdso/Makefile#L41

[ruiu]

PkmX,

If you already have a patch, please send it to me and llvm-commits.

[ruiu]

• Copy symbol type, size, binding, etc. when creating an absolute symbol.

[mcgrathr]

Works for me. D41277 works just as well. I don't know how you plan to reconcile the two.

[mcgrathr]

ping. Will this or D41277 land?

[ruiu]

• rebased

[ruiu]

I tried to create file objects for --just-symbols so that the linker command that just specifies --just-symbols without any real object file works fine.

https://reviews.llvm.org/D42025

But I really like this one over that one. The other one isn't necessarily bad, but it looks like it is a bit too complicated for a use case that I doubt is realistic. So my preference is to submit this one.

ELF/Driver.cpp

template <class ELFT> void LinkerDriver::link(opt::InputArgList &Args) {
   // are undefined symbols for them, so we add these to trigger that logic.
   for (StringRef Sym : Script->ReferencedSymbols)
     Symtab->addUndefined<ELFT>(Sym);
 
   // Handle the `--undefined <sym>` options.
   for (StringRef S : Config->Undefined)
     Symtab->fetchIfLazy<ELFT>(S);
 
+  // Handle the --just-symbols option. This may add absolute symbols
+  // to the symbol table.
+  for (auto *Arg : Args.filtered(OPT_just_symbols))
+    if (Optional<MemoryBufferRef> MB = readFile(Arg->getValue()))
+      readJustSymbolsFile<ELFT>(*MB);
+
   // If an entry symbol is in a static archive, pull out that file now
   // to complete the symbol table. After this, no new names except a
   // few linker-synthesized ones will be added to the symbol table.
   Symtab->fetchIfLazy<ELFT>(Config->Entry);
 
   // Return if there were name resolution errors.
   if (errorCount())
     return;

ELF/InputFiles.h

public:
   static bool classof(const InputFile *F) { return F->kind() == BinaryKind; }
   void parse();
 };
 
 InputFile *createObjectFile(MemoryBufferRef MB, StringRef ArchiveName = "",
                             uint64_t OffsetInArchive = 0);
 InputFile *createSharedFile(MemoryBufferRef MB, StringRef DefaultSoName);
 
+// For --just-symbols
+template <class ELFT> void readJustSymbolsFile(MemoryBufferRef MB);
+
 extern std::vector<BinaryFile *> BinaryFiles;
 extern std::vector<BitcodeFile *> BitcodeFiles;
 extern std::vector<InputFile *> ObjectFiles;
 extern std::vector<InputFile *> SharedFiles;
 
 } // namespace elf
 } // namespace lld
 
 #endif

ELF/InputFiles.cpp

case ELF64LEKind:
     return getElfSymbols<ELF64LE>();
   case ELF64BEKind:
     return getElfSymbols<ELF64BE>();
   default:
     llvm_unreachable("getELFKind");
   }
 }
 
+// This is for --just-symbols.
+//
+// This option allows you to link your output against other existing
+// program, so that if you load both your program and the other program
+// into memory, your output can use program's symbols.
+//
+// What we are doing here is to read defined symbols from a given ELF
+// file and add them as absolute symbols.
+template <class ELFT> void elf::readJustSymbolsFile(MemoryBufferRef MB) {
+  typedef typename ELFT::Shdr Elf_Shdr;
+  typedef typename ELFT::Sym Elf_Sym;
+  typedef typename ELFT::SymRange Elf_Sym_Range;
+
+  StringRef ObjName = MB.getBufferIdentifier();
+  ELFFile<ELFT> Obj = check(ELFFile<ELFT>::create(MB.getBuffer()));
+  ArrayRef<Elf_Shdr> Sections = CHECK(Obj.sections(), ObjName);
+
+  for (const Elf_Shdr &Sec : Sections) {
+    if (Sec.sh_type != SHT_SYMTAB)
+      continue;
+
+    Elf_Sym_Range Syms = CHECK(Obj.symbols(&Sec), ObjName);
+    uint32_t FirstNonLocal = Sec.sh_info;
+    StringRef StringTable =
+        CHECK(Obj.getStringTableForSymtab(Sec, Sections), ObjName);
+
+    std::vector<std::pair<StringRef, uint64_t>> Ret;
+    for (const Elf_Sym &Sym : Syms.slice(FirstNonLocal))
+      if (Sym.st_shndx != SHN_UNDEF)
+        Symtab->addRegular(CHECK(Sym.getName(StringTable), ObjName),
+                           Sym.st_other, Sym.getType(), Sym.st_value,
+                           Sym.st_size, Sym.getBinding(), nullptr, nullptr);
+    return;
+  }
+}
+
 template void ArchiveFile::parse<ELF32LE>();
 template void ArchiveFile::parse<ELF32BE>();
 template void ArchiveFile::parse<ELF64LE>();
 template void ArchiveFile::parse<ELF64BE>();
 
 template void BitcodeFile::parse<ELF32LE>(DenseSet<CachedHashStringRef> &);
 template void BitcodeFile::parse<ELF32BE>(DenseSet<CachedHashStringRef> &);
 template void BitcodeFile::parse<ELF64LE>(DenseSet<CachedHashStringRef> &);

 template class elf::ObjFile<ELF32BE>;
 template class elf::ObjFile<ELF64LE>;
 template class elf::ObjFile<ELF64BE>;
 
 template class elf::SharedFile<ELF32LE>;
 template class elf::SharedFile<ELF32BE>;
 template class elf::SharedFile<ELF64LE>;
 template class elf::SharedFile<ELF64BE>;
+
+template void elf::readJustSymbolsFile<ELF32LE>(MemoryBufferRef);
+template void elf::readJustSymbolsFile<ELF32BE>(MemoryBufferRef);
+template void elf::readJustSymbolsFile<ELF64LE>(MemoryBufferRef);
+template void elf::readJustSymbolsFile<ELF64BE>(MemoryBufferRef);

ELF/Options.td

 def ignore_data_address_equality: F<"ignore-data-address-equality">,
   HelpText<"lld can break the address equality of data">;
 
 defm image_base : Eq<"image-base">, HelpText<"Set the base address">;
 
 defm init: Eq<"init">, HelpText<"Specify an initializer function">,
   MetaVarName<"<symbol>">;
 
+defm just_symbols: Eq<"just-symbols">, HelpText<"Just link symbols">;
+
 defm library: Eq<"library">, HelpText<"Root name of library to use">,
   MetaVarName<"<libName>">;
 
 def lto_O: J<"lto-O">, MetaVarName<"<opt-level>">,
   HelpText<"Optimization level for LTO">;
 
 def m: JoinedOrSeparate<["-"], "m">, HelpText<"Set target emulation">;
 

test/ELF/Inputs/just-symbols.s

+.globl foo, bar
+foo:
+  ret
+
+.section .data
+.type bar, @object
+.size bar, 40
+bar:
+  .zero 40

test/ELF/just-symbols.s

+# REQUIRES: x86
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %S/Inputs/just-symbols.s -o %t1
+# RUN: ld.lld %t1 -o %t1.exe -Ttext=0x10000
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t2
+# RUN: ld.lld %t2 -just-symbols=%t1.exe -o %t2.exe
+# RUN: llvm-readelf -symbols %t2.exe | FileCheck %s
+
+# CHECK: 0000000000011000    40 OBJECT  GLOBAL DEFAULT  ABS bar
+# CHECK: 0000000000010000     0 NOTYPE  GLOBAL DEFAULT  ABS foo
+
+.globl _start
+_start:
+  call foo
+  call bar
+  ret