This is an archive of the discontinued LLVM Phabricator instance.

[ELF] - Implement linker script OVERLAYs.
ClosedPublic

Authored by grimar on Mar 22 2018, 7:27 AM.

Details

Summary

This is PR36768.

Linker script OVERLAYs are described in 4.6.9. Overlay Description of the spec:
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/4/html/Using_ld_the_GNU_Linker/sections.html

They are used to allow output sections which have different LMAs but the same VAs
and used for embedded programming.

Currently, LLD restricts overlapping of sections and that seems to be the most desired
behaviour for defaults. My thoughts about possible approaches for PR36768 are on the bug page,
this patch implements OVERLAY keyword and allows VAs overlapping for sections that within the overlay.

Diff Detail

Event Timeline

grimar created this revision.Mar 22 2018, 7:27 AM
grimar updated this revision to Diff 139439.Mar 22 2018, 7:35 AM
  • Minor simplification.

I agree that adding this is better than relaxing our section overlap checks for every linker script.

Please also add a test showing that we reject explicit addresses and memory regions for the sections inside the overlay.

ELF/LinkerScript.h
315 ↗(On Diff #139439)

This could be a bit in OutputSection, no?

ELF/ScriptParser.cpp
451

Are we missing the alignment?

grimar updated this revision to Diff 140077.Mar 28 2018, 8:17 AM
grimar marked an inline comment as done.
  • Addressed review comments.
  • Added load_start_*/load_stop* symbols as specification requests.
ELF/LinkerScript.h
315 ↗(On Diff #139439)

Yes. Ok.

ELF/ScriptParser.cpp
451

GNU ld does not seem to respect the alignment here.

For example for code and script:

.section .aaa, "a"; 
.byte 0; 

.section .bbb, "a"; 
.align 4; 
.byte 0;
SECTIONS {
 . = 0x1000;
  OVERLAY 0x1000 : AT ( 0x4000 ) {
    .aaa { *(.aaa) } 
    .bbb { *(.bbb) } 
  } 
}

Output is:

Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .aaa              PROGBITS         0000000000001000  00001000
       0000000000000001  0000000000000000   A       0     0     1
  [ 2] .bbb              PROGBITS         0000000000001000  00002000
       0000000000000001  0000000000000000   A       0     0     4

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  LOAD           0x0000000000001000 0x0000000000001000 0x0000000000004000
                 0x0000000000000001 0x0000000000000001  R      1000
  LOAD           0x0000000000002000 0x0000000000001000 0x0000000000004001
                 0x0000000000000001 0x0000000000000001  R      1000

 Section to Segment mapping:
  Segment Sections...
   00     .aaa 
   01     .bbb

And I might be missing something, but it seems correct to me. I see no reason to
waste space with alignments for OVERLAYs, because overlay manager is responsible to
copy sections in and out of the runtime memory address and therefore I think alignment
is just unuseful here.
(4.6.9 Overlay Description, https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/4/html/Using_ld_the_GNU_Linker/sections.html)

ruiu added a comment.Mar 28 2018, 1:47 PM

I'm not opposing this patch, but I'm not entirely convinced that we need this feature in the first place. If you need it, please say so.

ELF/ScriptParser.cpp
432

This function is too long.

grimar updated this revision to Diff 140214.Mar 29 2018, 5:51 AM
  • Addressed review comment.

I'm not opposing this patch, but I'm not entirely convinced that we need this feature in the first place. If you need it, please say so.

I am implementing it because we have the PR (and because the alternative would be to relax the linker script checks, which is a bit scary way).

I'm not opposing this patch, but I'm not entirely convinced that we need this feature in the first place. If you need it, please say so.

We would like something like it.

With bfd this is just syntactic sugar, but only so because it is very liberal about allowing sections to overlap. We could do that, but only allowing overlaps when there is an explicit OVERLAY is better IMHO.

espindola added inline comments.Apr 2 2018, 12:44 PM
ELF/ScriptParser.cpp
433

Can't we create these symbols in the existing addStartStopSymbols function?

I might be a good idea to leave the symbols for a followup patch.

grimar added inline comments.Apr 3 2018, 6:37 AM
ELF/ScriptParser.cpp
433

I do not think we can use addStartStopSymbols because it uses just created Defined symbols
which values are based on VAs, but here symbols have LMA based values.
Also first what this function does is rejects the sections with !isValidCIdentifier names. This is
different from OVERLAY symbol crafting rules which just drop non-C symbols from the name.

Leaving the symbols for followup sounds OK (if we will choose this way and not the relaxing option instead).

grimar updated this revision to Diff 140922.Apr 4 2018, 3:27 AM
  • Rebased.
  • Removed start/stop symbols creation part from this patch.
grimar updated this revision to Diff 140923.Apr 4 2018, 3:33 AM
  • Minor simplification.
ruiu added inline comments.Apr 4 2018, 2:37 PM
ELF/ScriptParser.cpp
734

We shouldn't add a new parameter to this existing function for the minor feature. Please just remove it.

grimar updated this revision to Diff 141120.Apr 5 2018, 3:26 AM
  • Addressed review comment.
ELF/ScriptParser.cpp
734

Do you mean we should just accept the syntax which is documented as not
supported for overlayed sections, like AT/ALIGN/SUBALIGN and memory
regions?

I am not fond of that idea honestly, but if so, there would be one more difference in
the syntax that does not allow me simply to drop the InOverlay:

Overlay declaration looks like:

 OVERLAY 0x1000 : AT ( 0x4000 ) {
    .abc { *(.foo) } 
...

Regular section declaration is:

abc : { *(foo) }

Notice that : follows the abc for regular section declaration syntax. And I have to skip it somehow.
I do not think we can use consume(":") in place of expect(":") as it would relax all of our scripts.

What we can probably do instead then is to introduce readOverlaySectionDescription with the minimal needed
syntax support. I did it.

grimar updated this revision to Diff 141133.Apr 5 2018, 5:33 AM
  • Rebased + simplified a bit.
ruiu added inline comments.Apr 5 2018, 1:29 PM
ELF/ScriptParser.cpp
432

This needs a pointer to the page describing the OVERLAY command.

444–445

This is more conventional.

OutputSection *Prev = nullptr;
for (...) {
  ...
  Prev = OS;
}
462

Do you have to compute MaxSize inside a lambda?

ELF/Writer.cpp
2069–2070

This seems a overdesign. Implementation simplicity is important. Just skip if "InOverlay" is true. Remove OverlapKind enum.

grimar updated this revision to Diff 141303.Apr 6 2018, 3:21 AM
grimar marked 3 inline comments as done.
  • Addressed review comments.
ELF/ScriptParser.cpp
462

At the moment of parsing the linker script declaration, we do not know the output section sizes yet.
The lambda is needed to delay the computation. I see no other good way to do that.

ELF/Writer.cpp
2069–2070

I think that was one of the reasons to implement the OVERLAY instead of using the --no-check-sections.
It was mentioned in comments on bug page (https://bugs.llvm.org/show_bug.cgi?id=36768) that
"One problem with that is that it disables all checking"

Rafael?

espindola added inline comments.Apr 6 2018, 9:42 AM
ELF/ScriptParser.cpp
734

A possible option is to parse as usual and then error if one of the fields is set, no?

ELF/Writer.cpp
2069–2070

I agree with George on this one. We should not disable more checks than necessary.

But you only need a IsVirtualAddresses boolean instead of the enum, no?

grimar updated this revision to Diff 141596.Apr 9 2018, 1:48 AM
  • Addressed review comments.
ELF/ScriptParser.cpp
734

Unfortunately for the particular place, I am referring to it would not work,
and would need something like InOverlay flag anyways:

We can't parse .abc { *(.foo) } and .abc : { *(.foo) } in the same way I believe.
: is always present for the non-overlay case and always absent for overlays.
So we need to skip it somehow when parsing the overlay or to relax the
syntax parser at this place.

ELF/Writer.cpp
2069–2070

Done.

espindola accepted this revision.Apr 9 2018, 12:42 PM

I think this is the best way to offer this feature.

Do wait for Rui to have a look too.

This revision is now accepted and ready to land.Apr 9 2018, 12:42 PM
grimar updated this revision to Diff 152848.Jun 26 2018, 12:39 AM

Ping.

  • Rebased.
ruiu added inline comments.Jun 26 2018, 1:52 AM
ELF/ScriptParser.cpp
451

You always call this function with next(), so it is better to call next() inside that function rather than at the call site.

454

Can't you use the same LMAExpr again?

ELF/Writer.cpp
2105

Since this is a file-scope local function that is written close enough to this call site, you don't need /* IsVirtualAddress */ comment.

grimar updated this revision to Diff 152863.Jun 26 2018, 3:05 AM
grimar marked 2 inline comments as done.
  • Addressed comments.
ELF/ScriptParser.cpp
451

Fixed.

454

Sorry, I did not understand what was your suggestion about using LMAExpr here?

ELF/Writer.cpp
2105

Fixed.

ruiu added inline comments.Jun 26 2018, 9:56 PM
ELF/ScriptParser.cpp
454

Maybe it's not just doable, but what I was wondering was whether you could repalce this line with OS->LMAExpr = LMAExpr or not.

grimar added inline comments.Jun 27 2018, 12:44 AM
ELF/ScriptParser.cpp
454

Like that?

OutputSection *OS = readOverlaySectionDescription();
OS->AddrExpr = AddrExpr;
OS->LMAExpr = LMAExpr;
V.push_back(OS);

Then no, I can't, because it would cause sections to overlap, for example overlay.test would fail with:

error : section .out.big load address range overlaps with .out.small
>
>  >>> .out.big range is [0x4000, 0x4007]
>
>  >>> .out.small range is [0x4000, 0x4003]
>

It's happening because we need to assign consecutive addresses, but with the code above we would use the same start LMA
for each output section, what is wrong.

ruiu accepted this revision.Jun 27 2018, 12:49 AM

LGTM

ELF/ScriptParser.cpp
465

Maybe MoveDot should suffice as the variable has a very narrow scope.

466

LIkewise, Max would suffice.

ELF/Writer.cpp
2052

IsVirtualAddr

This revision was automatically updated to reflect the committed changes.
This revision was automatically updated to reflect the committed changes.
grimar marked 2 inline comments as done.