[ELF] Propagate LMA offset to sections with neither AT() nor AT>

[MaskRay]

Summary

Fixes https://bugs.llvm.org/show_bug.cgi?id=45313
Also fixes linkerscript/{at4.s,overlay.test} LMA address issues exposed by
011b785505b1f6d315a93fd0a0409576ad8d1805.
Related: D74297

This patch improves emulation of GNU ld's heuristics on the difference
between the LMA and the VMA:
https://sourceware.org/binutils/docs/ld/Output-Section-LMA.html#Output-Section-LMA

New test linkerscript/lma-offset.s (based on at4.s) demonstrates some behaviors.

[MaskRay]

Improve docs/ELF/linker_script.rst

[grimar]

This looks fine to me. I'd suggest to wait for Peter's opinion regarding this.

[psmith]

I think there may be one case from ld that we aren't handling, and I've made a few suggestions for the documentation, otherwise looks like a good step in the right direction.

Apologies in advance for being slow to respond to reviews this week.

lld/ELF/LinkerScript.cpp

866–867

At a glance, it looks like we might not handle this particular case, is this intentional?

• If the section has a specific VMA address, then this is used as the LMA address as well.

I think that this is where there is an addrExpr but no lmaExpr we should set ctx->lmaOffset = 0;

lld/docs/ELF/linker_script.rst

64

The lack of ``AT>lma_region``
  means the default region is used. Note, GNU ld propagates the previous LMA
  memory region when ``address`` is not specified.

My understanding of the sentence is that when there is no AT > lma_region LLD does not behave like GNU ld. LLD always uses the default region, whereas GNU ld uses the previous OutputSections lma_region when address is not specified. If I'm right then is this an intentional change or a current limitation? I think we should be explicit, whichever way it is. For example:

If the OutputSection has no ``AT>lma_region`` then LLD will always use the default memory region. Note that this is in contrast with GNU ld which will propagate the lma_region of the previous OutputSection when ``address`` is not specified. For a script that is compatible with
GNU ld and LLD you must set ``AT>lma_region`` explicitly and not rely on propagation.

67

In the absence of a PHDRS command, if the LMA region is different from the previous one, this section will start a new PT_LOAD segment.

Suggest an alternative:

If the linker script does not have a PHDRS command then if the ``lma_region`` for an OutputSection is different to the ``lma_region`` for the previous OutputSection a new loadable segment will be generated.

70

Suggest a new line or paragraph before If neither `AT(lma) nor AT>lma_region` is specified:

[MaskRay]

Incorporate documentation suggestions

lld/ELF/LinkerScript.cpp

866–867

See below. This and whereas GNU ld uses the previous OutputSections lma_region when address is not specified need more thoughts.

lld/docs/ELF/linker_script.rst

64

whereas GNU ld uses the previous OutputSections lma_region when address is not specified.

Yes. See the summary of D74297. The patch was made when I did not have a good understanding of LMA (I still don't but things start to become clearer). We can revisit the decision and fix this discrepancy as well if not difficult to implement.

[psmith]

LGTM thanks for the update. I think it would be good to fix the discrepancies between LLD and BFD in follow up patches if it is reasonable to do so.

[MaskRay]

Rename lma-difference.s to lma-offset.s

[MaskRay]

In D76995#1954314, @psmith wrote:

LGTM thanks for the update. I think it would be good to fix the discrepancies between LLD and BFD in follow up patches if it is reasonable to do so.

Looks like our current behavior is close to GNU ld enough. Actually, GNU ld's behavior is different from the documentation

% cat a.s
SECTIONS {
  . = 0x1000;
  .a : { *(.a) }
  .b : AT(0x2005) { *(.b) }
/* "If the section has a specific VMA address, then this is used as the LMA address as well." is not obeyed for .c */
  .c 0x3006 : { *(.c) }
}

Program Headers:
  Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
  LOAD           0x001000 0x0000000000001000 0x0000000000001000 0x000002 0x000002 R E 0x1000
  LOAD           0x001002 0x0000000000001002 0x0000000000002005 0x000001 0x000001 R   0x1000
  LOAD           0x001006 0x0000000000003006 0x0000000000004009 0x000002 0x000002 R   0x1000
  LOAD           0x001008 0x0000000000003008 0x0000000000004010 0x000001 0x000001 RW  0x1000

 Section to Segment mapping:
  Segment Sections...
   00     .text .a 
   01     .b 
   02     .c .d 
   03     .data

Asked on https://sourceware.org/pipermail/binutils/2020-April/110503.html

lld/ELF/LinkerScript.cpp

 //===- LinkerScript.cpp ---------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //

 }
 
 // This function assigns offsets to input sections and an output section
 // for a single sections command (e.g. ".text { *(.text); }").
 void LinkerScript::assignOffsets(OutputSection *sec) {
   if (!(sec->flags & SHF_ALLOC))
     dot = 0;
 
+  bool prevLMARegionIsDefault = ctx->lmaRegion == nullptr;
   ctx->memRegion = sec->memRegion;
   ctx->lmaRegion = sec->lmaRegion;
   if (ctx->memRegion)
     dot = ctx->memRegion->curPos;
 
   if ((sec->flags & SHF_ALLOC) && sec->addrExpr)
     setDot(sec->addrExpr, sec->location, false);
 
   // If the address of the section has been moved forward by an explicit
   // expression so that it now starts past the current curPos of the enclosing
   // region, we need to expand the current region to account for the space
   // between the previous section, if any, and the start of this section.
   if (ctx->memRegion && ctx->memRegion->curPos < dot)
     expandMemoryRegion(ctx->memRegion, dot - ctx->memRegion->curPos,
                        ctx->memRegion->name, sec->name);
 
   switchTo(sec);
 
-  ctx->lmaOffset = 0;
-
+  // ctx->lmaOffset is LMA minus VMA. If LMA is explicitly specified via AT() or
+  // AT>, recompute ctx->lmaOffset; otherwise, if both previous/current LMA
+  // region is the default, reuse previous lmaOffset; otherwise, reset lmaOffset
+  // to 0. This emulates heuristics described in
+  // https://sourceware.org/binutils/docs/ld/Output-Section-LMA.html
   if (sec->lmaExpr)
     ctx->lmaOffset = sec->lmaExpr().getValue() - dot;
-  if (MemoryRegion *mr = sec->lmaRegion)
+  else if (MemoryRegion *mr = sec->lmaRegion)
     ctx->lmaOffset = alignTo(mr->curPos, sec->alignment) - dot;
+  else if (!prevLMARegionIsDefault)
+    ctx->lmaOffset = 0;
 
-  // If neither AT nor AT> is specified for an allocatable section, the linker
-  // will set the LMA such that the difference between VMA and LMA for the
-  // section is the same as the preceding output section in the same region
-  // https://sourceware.org/binutils/docs-2.20/ld/Output-Section-LMA.html
-  // This, however, should only be done by the first "non-header" section
-  // in the segment.
+  // Propagate ctx->lmaOffset to the first "non-header" section.
   if (PhdrEntry *l = ctx->outSec->ptLoad)
     if (sec == findFirstSection(l))
       l->lmaOffset = ctx->lmaOffset;
 
   // We can call this method multiple times during the creation of
   // thunks and want to start over calculation each time.
   sec->size = 0;
 

lld/docs/ELF/linker_script.rst

 ------------------------
 
 sh_addralign of an *OutputSection* *S* is the maximum of
 ``ALIGN(section_align)`` and the maximum alignment of the input sections in
 *S*.
 
 When an *OutputSection* *S* has both ``address`` and ``ALIGN(section_align)``,
 GNU ld will set sh_addralign to ``ALIGN(section_align)``.
+
+Output section LMA
+------------------
+
+A load address (LMA) can be specified by ``AT(lma)`` or ``AT>lma_region``.
+
+- ``AT(lma)`` specifies the exact load address. If the linker script does not
+  have a PHDRS command, then a new loadable segment will be generated.
+- ``AT>lma_region`` specifies the LMA region. The lack of ``AT>lma_region``
+  means the default region is used. Note, GNU ld propagates the previous LMA
+  memory region when ``address`` is not specified. The LMA is set to the
+  current location of the memory region aligned to the section alignment.
+  If the linker script does not have a PHDRS command, then if
+  ``lma_region`` is different from the ``lma_region`` for
+  the previous OutputSection a new loadable segment will be generated.
+
+The two keywords cannot be specified at the same time.
+
+If neither ``AT(lma)`` nor ``AT>lma_region`` is specified:
+
+- If the previous section is also in the default LMA region, the difference
+  between the LMA and the VMA is computed to be the same as the previous
+  difference.
+- Otherwise, the LMA is set to the VMA.

lld/test/ELF/linkerscript/at4.s

-# REQUIRES: x86
-# RUN: llvm-mc -filetype=obj -triple=x86_64 %s -o %t.o
-# RUN: echo "SECTIONS { \
-# RUN:  . = 0x1000; \
-# RUN:  .aaa : { *(.aaa) } \
-# RUN:  .bbb : AT(0x2008) { *(.bbb) } \
-# RUN:  .ccc : { *(.ccc) } \
-# RUN: }" > %t.script
-# RUN: ld.lld %t.o --script %t.script -o %t
-# RUN: llvm-readelf -l %t | FileCheck %s
-
-# CHECK:      Type  Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
-# CHECK-NEXT: LOAD  0x001000 0x0000000000001000 0x0000000000001000 0x000008 0x000008 R   0x1000
-# CHECK-NEXT: LOAD  0x001008 0x0000000000001008 0x0000000000002008 0x000010 0x000010 R   0x1000
-# CHECK-NEXT: LOAD  0x001018 0x0000000000001018 0x0000000000001018 0x000001 0x000001 R E 0x1000
-
-.global _start
-_start:
- nop
-
-.section .aaa, "a"
-.quad 0
-
-.section .bbb, "a"
-.quad 0
-
-.section .ccc, "a"
-.quad 0

lld/test/ELF/linkerscript/lma-offset.s

+# REQUIRES: x86
+# RUN: echo '.globl _start; _start: ret; \
+# RUN:   .section .a,"a"; .byte 0; \
+# RUN:   .section .b,"a"; .byte 0; \
+# RUN:   .section .c,"a"; .byte 0; \
+# RUN:   .section .d,"a"; .byte 0; \
+# RUN:   .data; .byte 0' | \
+# RUN:   llvm-mc -filetype=obj -triple=x86_64 - -o %t.o
+# RUN: ld.lld -T %s %t.o -o %t
+# RUN: llvm-readelf -l %t | FileCheck %s
+
+# CHECK:      Type  Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
+# CHECK-NEXT: LOAD  0x001000 0x0000000000001000 0x0000000000001000 0x000001 0x000001 R   0x1000
+
+## .b has AT(). It starts a PT_LOAD segment which also includes .c
+# CHECK-NEXT: LOAD  0x001001 0x0000000000001001 0x0000000000002005 0x000002 0x000002 R   0x1000
+
+## .d has AT(). It starts a PT_LOAD segment, even if the difference between
+## LMA and VMA (0x2007-0x1003) is the same as the previous one.
+# CHECK-NEXT: LOAD  0x001003 0x0000000000001003 0x0000000000002007 0x000001 0x000001 R   0x1000
+
+## The orphan section .text starts a PT_LOAD segment. The difference between
+## LMA and VMA (0x2008-0x1004) remains the same
+# CHECK-NEXT: LOAD  0x001004 0x0000000000001004 0x0000000000002008 0x000001 0x000001 R E 0x1000
+
+## .data starts a PT_LOAD segment. The difference remains the same.
+# CHECK-NEXT: LOAD  0x001005 0x0000000000001005 0x0000000000002009 0x000001 0x000001 RW  0x1000
+
+SECTIONS {
+  . = 0x1000;
+  .a : { *(.a) }
+  .b : AT(0x2005) { *(.b) }
+  .c : { *(.c) }
+  .d : AT(0x2007) { *(.d) }
+  ## Orphan section .text will be inserted here.
+  .data : { *(.data) }
+}

lld/test/ELF/linkerscript/loadaddr.s

 # RUN: llvm-objdump -t %t2 | FileCheck %s
 # RUN: echo "SECTIONS { v = LOADADDR(.zzz); }" > %t.script
 # RUN: not ld.lld %t --script %t.script -o /dev/null 2>&1 | FileCheck --check-prefix=ERROR %s
 
 # CHECK:      0000000000002000 g       *ABS*  0000000000000000 aaa_lma
 # CHECK-NEXT: 0000000000002008 g       *ABS*  0000000000000000 bbb_lma
 # CHECK-NEXT: 0000000000003000 g       *ABS*  0000000000000000 ccc_lma
 # CHECK-NEXT: 0000000000004000 g       *ABS*  0000000000000000 ddd_lma
-# CHECK-NEXT: 0000000000001020 g       *ABS*  0000000000000000 txt_lma
+# CHECK-NEXT: 0000000000004008 g       *ABS*  0000000000000000 txt_lma
 # ERROR: {{.*}}.script:1: undefined section .zzz
 
 .global _start
 _start:
  nop
 
 .section .aaa, "a"
 .quad 0

lld/test/ELF/linkerscript/map-file2.test

 # CHECK-NEXT:       1010             3000        8     1         {{.*}}{{/|\\}}map-file2.test.tmp.o:(.ccc)
 # CHECK-NEXT:       1018             3008      100     1 . += 0x100
 # CHECK-NEXT:       1118             3108      109     1 .ddd
 # CHECK-NEXT:       1118             3108        1     1         BYTE ( 0x11 )
 # CHECK-NEXT:       1119             3109      100     1         . += 0x100
 # CHECK-NEXT:       1219             3209        8     1         {{.*}}{{/|\\}}map-file2.test.tmp.o:(.ddd)
 # CHECK-NEXT:       1228             3218       34     8 .eh_frame
 # CHECK-NEXT:       1228             3218       30     1         {{.*}}{{/|\\}}map-file2.test.tmp.o:(.eh_frame+0x0)
-# CHECK-NEXT:       125c             125c        1     4 .text
-# CHECK-NEXT:       125c             125c        1     4         {{.*}}{{/|\\}}map-file2.test.tmp.o:(.text)
-# CHECK-NEXT:       125c             125c        0     1                 f(int)
-# CHECK-NEXT:       125c             125c        0     1                 _start
+# CHECK-NEXT:       125c             324c        1     4 .text
+# CHECK-NEXT:       125c             324c        1     4         {{.*}}{{/|\\}}map-file2.test.tmp.o:(.text)
+# CHECK-NEXT:       125c             324c        0     1                 f(int)
+# CHECK-NEXT:       125c             324c        0     1                 _start
 # CHECK-NEXT:          0                0        8     1 .comment
 # CHECK-NEXT:          0                0        8     1         <internal>:(.comment)
 # CHECK-NEXT:          0                0       48     8 .symtab
 # CHECK-NEXT:          0                0       48     8         <internal>:(.symtab)
 # CHECK-NEXT:          0                0       48     1 .shstrtab
 # CHECK-NEXT:          0                0       48     1         <internal>:(.shstrtab)
 # CHECK-NEXT:          0                0        e     1 .strtab
 # CHECK-NEXT:          0                0        e     1         <internal>:(.strtab)

lld/test/ELF/linkerscript/overlay.test

 # CHECK: .out.big   PROGBITS 0000000000001000 001000 000008
 # CHECK: .out.small PROGBITS 0000000000001000 002000 000004
 # CHECK: .text      PROGBITS 0000000000001008 002008 000001
 
 # CHECK:      Program Headers:
 # CHECK:      Type Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
 # CHECK-NEXT: LOAD 0x001000 0x0000000000001000 0x0000000000004000 0x000008 0x000008 R   0x1000
 # CHECK-NEXT: LOAD 0x002000 0x0000000000001000 0x0000000000004008 0x000004 0x000004 R   0x1000
-# CHECK-NEXT: LOAD 0x002008 0x0000000000001008 0x0000000000001008 0x000001 0x000001 R E 0x1000
+# CHECK-NEXT: LOAD 0x002008 0x0000000000001008 0x0000000000004010 0x000001 0x000001 R E 0x1000