[LLD][LinkerScript] Allow non-alloc sections to be assigned to segments.

[andrewng]

Summary

This patch makes changes to allow sections without the SHF_ALLOC bit to be
assigned to segments in a linker script.

The assignment of output sections to segments is performed in
LinkerScript::createPhdrs. Previously, this function would bail as soon as it
encountered an output section which did not have the SHF_ALLOC bit set, thus
preventing any output section without SHF_ALLOC from being assigned to a
segment.

This restriction has now been removed from LinkerScript::createPhdrs and instead
a check for SHF_ALLOC has been added to LinkerScript::adjustSectionsAfterSorting
to not propagate program headers to sections without SHF_ALLOC which matches the
behaviour of bfd linker scripts.

[ruiu]

Hi Andrew,

What are you actually trying to address with this patch? I'd like to know more about the background as to why you want this. Is there any reason you can't fix your program rather than the linker?

[andrewng]

In D34204#784448, @ruiu wrote:

Hi Andrew,

What are you actually trying to address with this patch? I'd like to know more about the background as to why you want this. Is there any reason you can't fix your program rather than the linker?

Hi Rui,

Basically, we have custom program headers in our binaries that are used by some of our tools. We need to be able to populate these segments with the sections containing the appropriate data. These sections are not allocated as they are not required to be loaded at runtime. The reason we need these program headers is that the section headers may be stripped from the binaries but our tools still need access to the data.

Note that GNU ld supports this behaviour too.

Cheers,
Andrew

[ruiu]

Can you change your program?

I guess that the GNU linker is not designed to support this, but it's just that it happens to work with their linker. We are not aiming 100% compatibility with GNU as it is impossible (and doesn't make sense) to copy all the corner cases like this. So even if it works with the GNU linker, we sometimes want to stop and think whether it semantically makes sense or not.

[andrewng]

In D34204#786240, @ruiu wrote:

Can you change your program?

There are a number tools that require this information, some are not in our control.

I guess that the GNU linker is not designed to support this, but it's just that it happens to work with their linker. We are not aiming 100% compatibility with GNU as it is impossible (and doesn't make sense) to copy all the corner cases like this. So even if it works with the GNU linker, we sometimes want to stop and think whether it semantically makes sense or not.

What we're trying to achieve is to place a non allocatable section into a custom PHDR, which to us makes perfect sense and isn't a corner case. However, we do accept that enabling this could cause interesting corner cases if mixing alloc and non-alloc sections in a PHDR.

Cheers,
Andrew

[andrewng]

Hi,

Is there any more feedback on this patch or is it good to land?

Cheers,
Andrew

[ruiu]

This is probably OK but I'd like to hear from Rafael as he wrote the code you are updating.

ELF/LinkerScript.cpp

758 ↗

(On Diff #102539)

Describe not what this code does but why you want it.

761–762 ↗

(On Diff #102539)

Please format according to the LLVM coding style.

[andrewng]

Updated for review comments.

[andrewng]

Hi Rafael,

Is this patch now good to land?

Cheers,
Andrew

[andrewng]

In D34204#790836, @andrewng wrote:

Hi Rafael,

Is this patch now good to land?

Cheers,
Andrew

Ping.

lld/trunk/ELF/LinkerScript.cpp

void LinkerScript::adjustSectionsAfterSorting() {
 
   // Walk the commands and propagate the program headers to commands that don't
   // explicitly specify them.
   for (BaseCommand *Base : Opt.Commands) {
     auto *Cmd = dyn_cast<OutputSectionCommand>(Base);
     if (!Cmd)
       continue;
 
-    if (Cmd->Phdrs.empty())
+    if (Cmd->Phdrs.empty()) {
+      OutputSection *Sec = Cmd->Sec;
+      // To match the bfd linker script behaviour, only propagate program
+      // headers to sections that are allocated.
+      if (Sec && (Sec->Flags & SHF_ALLOC))
         Cmd->Phdrs = DefPhdrs;
-    else
+    } else {
       DefPhdrs = Cmd->Phdrs;
     }
+  }
 
   removeEmptyCommands();
 }
 
 void LinkerScript::createOrphanCommands() {
   for (OutputSection *Sec : OutputSections) {
     if (Sec->SectionIndex != INT_MAX)
       continue;
if (Cmd.LMAExpr) {
       Phdr.p_paddr = Cmd.LMAExpr().getValue();
       Phdr.HasLMA = true;
     }
   }
 
   // Add output sections to program headers.
   for (OutputSectionCommand *Cmd : OutputSectionCommands) {
     OutputSection *Sec = Cmd->Sec;
-    if (!(Sec->Flags & SHF_ALLOC))
-      break;
 
     // Assign headers specified by linker script
     for (size_t Id : getPhdrIndices(Sec)) {
       Ret[Id].add(Sec);
       if (Opt.PhdrsCommands[Id].Flags == UINT_MAX)
         Ret[Id].p_flags |= Sec->getPhdrFlags();
     }
   }

lld/trunk/test/ELF/linkerscript/non-alloc-segment.s

+# REQUIRES: x86
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t.o
+
+################################################################################
+## Test that non-alloc section .foo can be assigned to a segment. Check that
+## the values of the offset and file size of this segment's PHDR are correct.
+##
+## This functionality allows non-alloc metadata, which is not required at
+## run-time, to be added to a custom segment in a file. This metadata may be
+## read/edited by tools/loader using the values of the offset and file size from
+## the custom segment's PHDR. This is particularly important if section headers
+## have been stripped.
+# RUN: echo "PHDRS {text PT_LOAD; foo 0x12345678;} \
+# RUN:       SECTIONS { \
+# RUN:           .text : {*(.text .text*)} :text \
+# RUN:           .foo : {*(.foo)} :foo \
+# RUN:       }" > %t.script
+# RUN: ld.lld -o %t --script %t.script %t.o
+# RUN: llvm-readobj -elf-output-style=GNU -s -l %t | FileCheck %s
+# RUN: llvm-readobj -l %t | FileCheck --check-prefix=PHDR %s
+
+# CHECK: Program Headers:
+# CHECK-NEXT:  Type
+# CHECK-NEXT:  LOAD
+# CHECK-NEXT:  <unknown>: 0x12345678
+
+# CHECK:      Section to Segment mapping:
+# CHECK-NEXT:  Segment Sections...
+# CHECK-NEXT:   00     .text
+# CHECK-NEXT:   01     .foo
+
+# PHDR: Type:  (0x12345678)
+# PHDR-NEXT: Offset: 0x1004
+# PHDR-NEXT: VirtualAddress
+# PHDR-NEXT: PhysicalAddress
+# PHDR-NEXT: FileSize: 4
+
+.global _start
+_start:
+ nop
+
+.section .foo
+ .align 4
+ .long 0