Do not merge input sections if --relocatable.

[ruiu]

Summary

This was discussed in https://reviews.llvm.org/D24728 and I think
it's worth to bring up as a separate and concrete patch so that
we can discuss about it.

This is a patch to disable section merging if --relocatable option
is given. Currently, if --relocatable is given, LLD produces a
relinkable object file, but multiple .text input sections are merged
together to produce a single .text output section (and so is true
for other regular output sections such as .data, .rodata, etc.)

Such behavior is the same as what GNU linkers do, which is an
obvious advantage, but it also has a few drawbacks.

First, it is not always easy to merge sections for --relocatable.
That is discussed in the review thread for D24728. For D24728,
we wanted to merge exception table for ARM, and merging .ARM.exidx
is tricky because we have to create corresponding sh_link sections
for a newly created output section. If we don't merge input sections,
but instead pass them through, we can eliminate that complexity
completely.

Second, --gc-sections and --icf doesn't work well for merged inputs.
Once we merge multiple .text secitons, the output is very likely to
be eliminated by --gc-sections on the final link.

Overall, I think not merging input sections is a better choice for us.
It is easier to impolement than merging sections and will yield
better final outputs. Therefore, I want to change the current behavior
with this change.

[peter.smith]

I've done a bit of digging on the uses for ld -r relocatable. It seems to be that the primary and perhaps only use case for the existing behaviour of ld -r are relocatable kernel modules (on linux files with the .ko extension). Single file modules can be loaded directly by a simple linker/loader in the kernel, more complex ones are combined with ld -r (https://www.kernel.org/doc/Documentation/kbuild/makefiles.txt).

My limited understanding of kernel modules is that they are written in C with no use of the standard library allowed. There is no support for C++ and certainly not C++ exceptions.

While I couldn't say for sure I would expect that a kernel module will rely on being "pre-processed" by ld -r first, if only to collate together the module id and some other custom sections.

As to what this tells us; I think the ability to combine sections has a reasonable use case. I don't think that trying to make all possible features work seamlessly with it is strictly necessary. Some possible options:

• Do not support kernel modules, this would be a problem for freebsd if their kernel modules work this way. I'm sure linux kernel modules can/will use GNU ld.
• ld -r works as it is but does not guarantee to support complicated cases like ARM C++ exceptions (error message given?)
• ld -r works as it is when provided with a linker script, but by default it won't combine sections.
• Make ld -r work, at least for those who are using its output as an input to another system linker.

I suspect that if freebsd's module system uses relocatable objects like Linux I think the existing behaviour of ld -r will be important enough to preserve in some form. Given that kernel modules don't support C++ I think it is acceptable, if not ideal, to not support ARM C++ exceptions with ld -r combining sections.

Some comments on the patch:

• Last time I tried ld -r wasn't working at all with linker scripts but it will be worth checking that the ldscript interacts as expected. At present it looks like the individual InputSections are not combined, but they are renamed to whatever OutputSection they matched in the linker script.
• The SHF_LINK_ORDER will still need a bit of added complexity, the sh_link to the InputSection will need to be updated to whatever the section index of the OutputSection is.

ELF/OutputSections.cpp

 }
 
 template <class ELFT>
 std::pair<OutputSectionBase<ELFT> *, bool>
 OutputSectionFactory<ELFT>::create(const SectionKey<ELFT::Is64Bits> &Key,
                                    InputSectionBase<ELFT> *C) {
   uintX_t Flags = getOutFlags(C);
   OutputSectionBase<ELFT> *&Sec = Map[Key];
-  if (Sec) {
+
+  // If --relocatable, we do not merge regular input sections
+  // and instead let the final link merge them. This behavior
+  // is different from GNU linkers, but we chose this behavior
+  // because 1) --gc-sections and --icf work better with fine-
+  // grained inputs and 2) reading some sections such as .ARM.exidx
+  // and reproduce it for later use is not easy.
+  if (!Config->Relocatable && Sec) {
     Sec->updateFlags(Flags);
     return {Sec, false};
   }
 
   uint32_t Type = C->getSectionHdr()->sh_type;
   switch (C->kind()) {
   case InputSectionBase<ELFT>::Regular:
     Sec = new OutputSection<ELFT>(Key.Name, Type, Flags);

ELF/Writer.cpp

private:
 
   uintX_t FileSize;
   uintX_t SectionHeaderOff;
 };
 } // anonymous namespace
 
 template <class ELFT>
 StringRef elf::getOutputSectionName(InputSectionBase<ELFT> *S) {
+  // If --relocatable, we don't merge sections.
+  if (Config->Relocatable)
+    return S->Name;
+
   StringRef Name = S->Name;
   for (StringRef V :
        {".text.", ".rodata.", ".data.rel.ro.", ".data.", ".bss.",
         ".init_array.", ".fini_array.", ".ctors.", ".dtors.", ".tbss.",
         ".gcc_except_table.", ".tdata.", ".ARM.exidx."}) {
     StringRef Prefix = V.drop_back();
     if (Name.startswith(V) || Name == Prefix)
       return Prefix;

test/ELF/relocatable-ehframe.s

 # CHECK-NEXT: Info: 0
 # CHECK-NEXT: AddressAlignment: 1
 # CHECK-NEXT: EntrySize: 0
 # CHECK-NEXT: SectionData (
 # CHECK-NEXT:   0000: 00005F73 74617274 00                 |.._start.|
 # CHECK-NEXT: )
 
 # CHECK:      Relocations [
-# CHECK-NEXT:   Section {{.*}} .rela.eh_frame {
+# CHECK-NEXT:   Section (11) .rela.eh_frame {
 # CHECK-NEXT:     0x20 R_X86_64_PC32 foo 0x0
 # CHECK-NEXT:     0x34 R_X86_64_PC32 bar 0x0
 # CHECK-NEXT:     0x48 R_X86_64_PC32 dah 0x0
-# CHECK-NEXT:     0x78 R_X86_64_PC32 foo1 0x0
-# CHECK-NEXT:     0x8C R_X86_64_PC32 bar1 0x0
-# CHECK-NEXT:     0xA0 R_X86_64_PC32 dah1 0x0
+# CHECK-NEXT:   }
+# CHECK-NEXT:   Section (12) .rela.eh_frame {
+# CHECK-NEXT:     0x20 R_X86_64_PC32 foo1 0x0
+# CHECK-NEXT:     0x34 R_X86_64_PC32 bar1 0x0
+# CHECK-NEXT:     0x48 R_X86_64_PC32 dah1 0x0
 # CHECK-NEXT:   }
 # CHECK-NEXT: ]
 
 .section foo,"ax",@progbits
 .cfi_startproc
  nop
 .cfi_endproc
 

test/ELF/relocatable-reloc.s

 // RUN: llvm-mc -filetype=obj %s -o %t.o -triple=x86_64-pc-linux
 // RUN: ld.lld %t.o %t.o -r -o %t2.o
 // RUN: llvm-readobj -r %t2.o | FileCheck %s
 
 .weak foo
 foo:
 .quad foo
 
 // CHECK:      Relocations [
-// CHECK-NEXT:   Section ({{.*}}) .rela.text {
+// CHECK-NEXT:   Section (3) .rela.text {
+// CHECK-NEXT:     0x0 R_X86_64_64 foo 0x0
+// CHECK-NEXT:   }
+// CHECK-NEXT:   Section (4) .rela.text {
 // CHECK-NEXT:     0x0 R_X86_64_64 foo 0x0
-// CHECK-NEXT:     0x8 R_X86_64_64 foo 0x0
 // CHECK-NEXT:   }
 // CHECK-NEXT: ]

test/ELF/relocatable.s

 # REQUIRES: x86
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t1.o
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %p/Inputs/relocatable.s -o %t2.o
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %p/Inputs/relocatable2.s -o %t3.o
 # RUN: ld.lld -r %t1.o %t2.o %t3.o -o %t
 # RUN: llvm-readobj -file-headers -sections -program-headers -symbols -r %t | FileCheck %s
-# RUN: llvm-objdump -s -d %t | FileCheck -check-prefix=CHECKTEXT %s
+# RUN: llvm-objdump -s -d %t | FileCheck -check-prefix=TEXT %s
 
 ## Test --relocatable alias
 # RUN: ld.lld --relocatable %t1.o %t2.o %t3.o -o %t
 # RUN: llvm-readobj -file-headers -sections -program-headers -symbols -r %t | FileCheck %s
-# RUN: llvm-objdump -s -d %t | FileCheck -check-prefix=CHECKTEXT %s
+# RUN: llvm-objdump -s -d %t | FileCheck -check-prefix=TEXT %s
 
 ## Verify that we can use our relocation output as input to produce executable
 # RUN: ld.lld -e main %t -o %texec
-# RUN: llvm-readobj -file-headers %texec | FileCheck -check-prefix=CHECKEXE %s
+# RUN: llvm-readobj -file-headers %texec | FileCheck -check-prefix=EXE %s
 
 # CHECK:       ElfHeader {
 # CHECK-NEXT:  Ident {
 # CHECK-NEXT:    Magic: (7F 45 4C 46)
 # CHECK-NEXT:    Class: 64-bit
 # CHECK-NEXT:    DataEncoding: LittleEndian
 # CHECK-NEXT:    FileVersion: 1
 # CHECK-NEXT:    OS/ABI: SystemV
 # CHECK-NEXT:    ABIVersion: 0
 # CHECK-NEXT:    Unused: (00 00 00 00 00 00 00)
 # CHECK-NEXT:  }
 # CHECK-NEXT:  Type: Relocatable
 # CHECK-NEXT:  Machine: EM_X86_64
 # CHECK-NEXT:  Version: 1
 # CHECK-NEXT:  Entry: 0x0
 # CHECK-NEXT:  ProgramHeaderOffset: 0x0
 # CHECK-NEXT:  SectionHeaderOffset: 0x2C0
 # CHECK-NEXT:  Flags [
 # CHECK-NEXT:  ]
 # CHECK-NEXT:  HeaderSize: 64
 # CHECK-NEXT:  ProgramHeaderEntrySize: 0
 # CHECK-NEXT:  ProgramHeaderCount: 0
 # CHECK-NEXT:  SectionHeaderEntrySize: 64
-# CHECK-NEXT:  SectionHeaderCount: 7
-# CHECK-NEXT:  StringTableSectionIndex: 5
+# CHECK-NEXT:  SectionHeaderCount: 13
+# CHECK-NEXT:  StringTableSectionIndex: 11
 # CHECK-NEXT:  }
 
-# CHECK:       Relocations [
-# CHECK-NEXT:  Section (3) .rela.text {
-# CHECK-NEXT:    0x3 R_X86_64_32S x 0x0
-# CHECK-NEXT:    0xE R_X86_64_32S y 0x0
-# CHECK-NEXT:    0x23 R_X86_64_32S xx 0x0
-# CHECK-NEXT:    0x2E R_X86_64_32S yy 0x0
-# CHECK-NEXT:    0x43 R_X86_64_32S xxx 0x0
-# CHECK-NEXT:    0x4E R_X86_64_32S yyy 0x0
-# CHECK-NEXT:  }
+# CHECK:      Relocations [
+# CHECK-NEXT:   Section (7) .rela.text {
+# CHECK-NEXT:     0x3 R_X86_64_32S x 0x0
+# CHECK-NEXT:     0xE R_X86_64_32S y 0x0
+# CHECK-NEXT:   }
+# CHECK-NEXT:   Section (8) .rela.text {
+# CHECK-NEXT:     0x3 R_X86_64_32S xx 0x0
+# CHECK-NEXT:     0xE R_X86_64_32S yy 0x0
+# CHECK-NEXT:   }
+# CHECK-NEXT:   Section (9) .rela.text {
+# CHECK-NEXT:     0x3 R_X86_64_32S xxx 0x0
+# CHECK-NEXT:     0xE R_X86_64_32S yyy 0x0
+# CHECK-NEXT:   }
+# CHECK-NEXT: ]
 
-# CHECKTEXT:      Disassembly of section .text:
-# CHECKTEXT-NEXT: main:
-# CHECKTEXT-NEXT: 0: c7 04 25 00 00 00 00 05 00 00 00 movl $5, 0
-# CHECKTEXT-NEXT: b: c7 04 25 00 00 00 00 07 00 00 00 movl $7, 0
-# CHECKTEXT:      foo:
-# CHECKTEXT-NEXT: 20: c7 04 25 00 00 00 00 01 00 00 00 movl $1, 0
-# CHECKTEXT-NEXT: 2b: c7 04 25 00 00 00 00 02 00 00 00 movl $2, 0
-# CHECKTEXT:      bar:
-# CHECKTEXT-NEXT: 40: c7 04 25 00 00 00 00 08 00 00 00 movl $8, 0
-# CHECKTEXT-NEXT: 4b: c7 04 25 00 00 00 00 09 00 00 00 movl $9, 0
-
-# CHECKEXE:       Format: ELF64-x86-64
-# CHECKEXE-NEXT:  Arch: x86_64
-# CHECKEXE-NEXT:  AddressSize: 64bit
-# CHECKEXE-NEXT:  LoadName:
-# CHECKEXE-NEXT:  ElfHeader {
-# CHECKEXE-NEXT:    Ident {
-# CHECKEXE-NEXT:      Magic: (7F 45 4C 46)
-# CHECKEXE-NEXT:      Class: 64-bit
-# CHECKEXE-NEXT:      DataEncoding: LittleEndian
-# CHECKEXE-NEXT:      FileVersion: 1
-# CHECKEXE-NEXT:      OS/ABI: SystemV (0x0)
-# CHECKEXE-NEXT:      ABIVersion: 0
-# CHECKEXE-NEXT:      Unused: (00 00 00 00 00 00 00)
-# CHECKEXE-NEXT:    }
-# CHECKEXE-NEXT:    Type: Executable
-# CHECKEXE-NEXT:    Machine: EM_X86_64
-# CHECKEXE-NEXT:    Version: 1
-# CHECKEXE-NEXT:    Entry: 0x11000
-# CHECKEXE-NEXT:    ProgramHeaderOffset: 0x40
-# CHECKEXE-NEXT:    SectionHeaderOffset: 0x11E8
-# CHECKEXE-NEXT:    Flags [
-# CHECKEXE-NEXT:    ]
-# CHECKEXE-NEXT:    HeaderSize: 64
-# CHECKEXE-NEXT:    ProgramHeaderEntrySize: 56
-# CHECKEXE-NEXT:    ProgramHeaderCount: 5
-# CHECKEXE-NEXT:    SectionHeaderEntrySize: 64
-# CHECKEXE-NEXT:    SectionHeaderCount: 6
-# CHECKEXE-NEXT:    StringTableSectionIndex: 4
-# CHECKEXE-NEXT:  }
+# TEXT:      Disassembly of section .text:
+# TEXT-NEXT: main:
+# TEXT-NEXT: 0: c7 04 25 00 00 00 00 05 00 00 00 movl $5, 0
+# TEXT-NEXT: b: c7 04 25 00 00 00 00 07 00 00 00 movl $7, 0
+# TEXT-NEXT: Disassembly of section .text:
+# TEXT-NEXT: foo:
+# TEXT-NEXT: 0: c7 04 25 00 00 00 00 01 00 00 00 movl $1, 0
+# TEXT-NEXT: b: c7 04 25 00 00 00 00 02 00 00 00 movl $2, 0
+# TEXT-NEXT: Disassembly of section .text:
+# TEXT-NEXT: bar:
+# TEXT-NEXT: 0: c7 04 25 00 00 00 00 08 00 00 00 movl $8, 0
+# TEXT-NEXT: b: c7 04 25 00 00 00 00 09 00 00 00 movl $9, 0
+
+# EXE:       Format: ELF64-x86-64
+# EXE-NEXT:  Arch: x86_64
+# EXE-NEXT:  AddressSize: 64bit
+# EXE-NEXT:  LoadName:
+# EXE-NEXT:  ElfHeader {
+# EXE-NEXT:    Ident {
+# EXE-NEXT:      Magic: (7F 45 4C 46)
+# EXE-NEXT:      Class: 64-bit
+# EXE-NEXT:      DataEncoding: LittleEndian
+# EXE-NEXT:      FileVersion: 1
+# EXE-NEXT:      OS/ABI: SystemV (0x0)
+# EXE-NEXT:      ABIVersion: 0
+# EXE-NEXT:      Unused: (00 00 00 00 00 00 00)
+# EXE-NEXT:    }
+# EXE-NEXT:    Type: Executable
+# EXE-NEXT:    Machine: EM_X86_64
+# EXE-NEXT:    Version: 1
+# EXE-NEXT:    Entry: 0x11000
+# EXE-NEXT:    ProgramHeaderOffset: 0x40
+# EXE-NEXT:    SectionHeaderOffset: 0x11E8
+# EXE-NEXT:    Flags [
+# EXE-NEXT:    ]
+# EXE-NEXT:    HeaderSize: 64
+# EXE-NEXT:    ProgramHeaderEntrySize: 56
+# EXE-NEXT:    ProgramHeaderCount: 5
+# EXE-NEXT:    SectionHeaderEntrySize: 64
+# EXE-NEXT:    SectionHeaderCount: 6
+# EXE-NEXT:    StringTableSectionIndex: 4
+# EXE-NEXT:  }
 
 .text
 .type x,@object
 .bss
 .globl x
 .align 4
 x:
 .long 0