[ELF] Infer OSABI from input files for -m options without explicit OSABI

[arichardson]

Summary

Currently only -m *_fbsd and -m msp430elf explicitly specifiy the OSABI
field, all others set the value to zero.
FreeBSD expects the OSABI field to be set for all executables, but both
GCC and Clang pass -melf64lriscv to the linker. This causes all RISC-V
executables to have a SysV OSABI which confuses tools such as ldd.

This patch changes the behaviour to infer the OSABI from the first input
.o file for -m options that don't encode a specific OSABI.

[jrtc27]

@bsdjhb has been intending to fix ldd to recognise the ELF note rather than requiring the OSABI be FreeBSD. The only reason to set the OSABI to FreeBSD is so that the various FOO_LOOS-FOO_HIOS can be interpreted correctly.

[kevans]

In D95370#2520210, @jrtc27 wrote:

@bsdjhb has been intending to fix ldd to recognise the ELF note rather than requiring the OSABI be FreeBSD. The only reason to set the OSABI to FreeBSD is so that the various FOO_LOOS-FOO_HIOS can be interpreted correctly.

It'd be nice to get something in sooner rather than later to fix the immediate issue, though... we've got at least one person that's actively working to identify and fix ports issues that have cropped up with other tier-2 platforms that also apply to riscv64, but sqlite3 being unpackageable is a pretty big downer that blocks over a thousand ports from being examined.

[MaskRay]

Can you give links why this is needed to change in the linker?

[jrtc27]

In D95370#2520240, @MaskRay wrote:

Can you give links why this is needed to change in the linker?

For older FreeBSD ports, the emulation has always been foo_fbsd. For AArch64, it was for a long time just using the unsuffixed version. At some point a few years ago, GNU toolchains started to use the _fbsd-suffixed emulation, but Clang continues to pass the unsuffixed one. For RISC-V, neither the GNU toolchain nor Clang include the suffix.

It's debatable whether this should be done in the linker. But the .o's coming out of Clang have ELFOSABI_FREEBSD and the linker loses that information if you pass an explicit non-suffixed emulation (if you don't pass anything it defaults to the first file's).

It's all a bit strange, and differs from Linux where you almost always see ELFOSABI_NONE rather than ELFOSABI_LINUX. But I would like to see the significance of EI_OSABI be decreased such that it's only about the ELF extensions used and not about the OS it expects to run in, as we have much more expressive ELF notes for the latter.

[arichardson]

In D95370#2520240, @MaskRay wrote:

Can you give links why this is needed to change in the linker?

For ld.lld we could obviously also drop the -m flag from the clang driver or add _fbsd. However, I believe that will break ld.bfd since it doesn't support the _fbsd variants.

$ ./ld.bfd -V
GNU ld (GNU Binutils) 2.33.1
  Supported emulations:
   elf64lriscv
   elf32lriscv

[MaskRay]

GNU as and GNU ld by default emit ELFOSABI_NONE, but will promote ELFOSABI_NONE to ELFOSABI_GNU if certain GNU ABI ELF features are used.
The single value nature makes it difficult to encode more than one ABI flavors. In practice many OSABI platforms adopt some GNU extensions.
It is possible to use extensions from more than one osabi platforms and we treat ELFOSABI_GNU an overridable osabi value (can be overridden by ELFOSABI_FREEBSD).

In https://sourceware.org/bugzilla/show_bug.cgi?id=12913#c9

It may have been an unfortunate choice to use EI_OSABI for the meaning that ELFOSABI_GNU has. But it's already been done and it's too late now.

I think that is a good summary from a retrospective viewpoint. It would be nice if GNU extensions could be taken as need, used as mixins. A single osabi value does not encode this intention well.

inferMachineType infers ekind/e_machine/EI_OSABI. ekind/e_machine are good to infer but EI_OSABI is not.
This patch makes the logic smarter: skipping ELFOSABI_NONE, but I don't see it fundamentally improves things.

E.g. if the first few object files have osabi values of ELFOSABI_NONE, ELFOSABI_GNU, ELFOSABI_FREEBSD, ELFOSABI_FREEBSD, ...
Do we pick ELFOSABI_GNU as the output? It is not better than ELFOSABI_NONE as ELFOSABI_FREEBSD should be the correct one.

The problem, according to @kevans, is no fbsd specific -m is specified clang/lib/Driver/ToolChains/FreeBSD.cpp, and ultimately the problem is that GNU ld does not understand a FreeBSD flavor RISC-V emulation...
So seems that binutils should be fixed. If you can detect -fuse-ld=, you can pass the emulation...

Filed https://sourceware.org/bugzilla/show_bug.cgi?id=27296 and sent a patch.

[MaskRay]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252490

[arichardson]

No longer necessary

lld/ELF/Config.h

struct VersionDefinition {
   std::vector<SymbolVersion> patterns;
 };
 
 // This struct contains the global configuration for the linker.
 // Most fields are direct mapping from the command line options
 // and such fields have the same name as the corresponding options.
 // Most fields are initialized by the driver.
 struct Configuration {
-  uint8_t osabi = 0;
+  llvm::Optional<uint8_t> osabi;
   uint32_t andFeatures = 0;
   llvm::CachePruningPolicy thinLTOCachePolicy;
   llvm::SetVector<llvm::CachedHashString> dependencyFiles; // for --dependency-file
   llvm::StringMap<uint64_t> sectionStartMap;
   llvm::StringRef bfdname;
   llvm::StringRef chroot;
   llvm::StringRef dependencyFile;
   llvm::StringRef dwoDir;

lld/ELF/Driver.cpp

bool elf::link(ArrayRef<const char *> args, bool canExitEarly,
 
   bool ret = errorCount() == 0;
   if (!canExitEarly)
     errorHandler().reset();
   return ret;
 }
 
 // Parses a linker -m option.
-static std::tuple<ELFKind, uint16_t, uint8_t> parseEmulation(StringRef emul) {
-  uint8_t osabi = 0;
+static std::tuple<ELFKind, uint16_t, Optional<uint8_t>>
+parseEmulation(StringRef emul) {
+  Optional<uint8_t> osabi;
   StringRef s = emul;
   if (s.endswith("_fbsd")) {
     s = s.drop_back(5);
     osabi = ELFOSABI_FREEBSD;
   }
 
   std::pair<ELFKind, uint16_t> ret =
       StringSwitch<std::pair<ELFKind, uint16_t>>(s)
void LinkerDriver::createFiles(opt::InputArgList &args) {
   }
 
   if (files.empty() && errorCount() == 0)
     error("no input files");
 }
 
 // If -m <machine_type> was not given, infer it from object files.
 void LinkerDriver::inferMachineType() {
-  if (config->ekind != ELFNoneKind)
+  if (config->ekind != ELFNoneKind && config->osabi)
     return;
 
   for (InputFile *f : files) {
     if (f->ekind == ELFNoneKind)
       continue;
+    if (config->ekind == ELFNoneKind) {
       config->ekind = f->ekind;
       config->emachine = f->emachine;
-    config->osabi = f->osabi;
       config->mipsN32Abi = config->emachine == EM_MIPS && isMipsN32Abi(f);
+    }
+    if (!config->osabi)
+      config->osabi = f->osabi;
     return;
   }
   error("target emulation unknown: -m or at least one .o file required");
 }
 
 // Parse -z max-page-size=<value>. The default value is defined by
 // each target.
 static uint64_t getMaxPageSize(opt::InputArgList &args) {

lld/ELF/SyntheticSections.cpp

template <typename ELFT> void elf::writeEhdr(uint8_t *buf, Partition &part) {
   // in header are zero-cleared, instead of having trap instructions.
   memset(buf, 0, sizeof(typename ELFT::Ehdr));
   memcpy(buf, "\177ELF", 4);
 
   auto *eHdr = reinterpret_cast<typename ELFT::Ehdr *>(buf);
   eHdr->e_ident[EI_CLASS] = config->is64 ? ELFCLASS64 : ELFCLASS32;
   eHdr->e_ident[EI_DATA] = config->isLE ? ELFDATA2LSB : ELFDATA2MSB;
   eHdr->e_ident[EI_VERSION] = EV_CURRENT;
-  eHdr->e_ident[EI_OSABI] = config->osabi;
+  eHdr->e_ident[EI_OSABI] = *config->osabi;
   eHdr->e_ident[EI_ABIVERSION] = getAbiVersion();
   eHdr->e_machine = config->emachine;
   eHdr->e_version = EV_CURRENT;
   eHdr->e_flags = config->eflags;
   eHdr->e_ehsize = sizeof(typename ELFT::Ehdr);
   eHdr->e_phnum = part.phdrs.size();
   eHdr->e_shentsize = sizeof(typename ELFT::Shdr);
 

lld/test/ELF/basic-freebsd.s

 # REQUIRES: x86
 # Verify that OSABI is set to the correct value.
 
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-freebsd %s -o %t
+# RUN: llvm-readobj --file-headers %t | FileCheck %s --check-prefixes CHECK,CHECK-OBJ
 # RUN: ld.lld %t -o %t2
-# RUN: llvm-readobj --file-headers %t2 | FileCheck %s
+# RUN: llvm-readobj --file-headers %t2 | FileCheck %s --check-prefixes CHECK,CHECK-EXE
+# An explicit _fbsd emulation should set the FreeBSD OSABI
+# RUN: ld.lld -m elf_x86_64_fbsd %t -o %t2
+# RUN: llvm-readobj --file-headers %t2 | FileCheck %s --check-prefixes CHECK,CHECK-EXE
+# If the emulation does not encode a specific OSABI, we should infer
+# it from the first input .o file rather than using SysV.
+# RUN: ld.lld -m elf_x86_64 %t -o %t2
+# RUN: llvm-readobj --file-headers %t2 | FileCheck %s --check-prefixes CHECK,CHECK-EXE
 
 .globl _start
 _start:
   mov $1, %rax
   mov $42, %rdi
   syscall
 
 # CHECK: ElfHeader {
 # CHECK-NEXT:   Ident {
 # CHECK-NEXT:     Magic: (7F 45 4C 46)
 # CHECK-NEXT:     Class: 64-bit (0x2)
 # CHECK-NEXT:     DataEncoding: LittleEndian (0x1)
 # CHECK-NEXT:     FileVersion: 1
 # CHECK-NEXT:     OS/ABI: FreeBSD (0x9)
 # CHECK-NEXT:     ABIVersion: 0
 # CHECK-NEXT:     Unused: (00 00 00 00 00 00 00)
 # CHECK-NEXT:   }
-# CHECK-NEXT:   Type: Executable (0x2)
-# CHECK-NEXT:   Machine: EM_X86_64 (0x3E)
+# CHECK-OBJ-NEXT: Type: Relocatable (0x1)
+# CHECK-EXE-NEXT: Type: Executable (0x2)
+# CHECK-NEXT:     Machine: EM_X86_64 (0x3E)