This is an archive of the discontinued LLVM Phabricator instance.

[ELF] Make overlapping output sections an error
ClosedPublic

Authored by arichardson on Dec 9 2017, 1:11 AM.

Details

Summary

While trying to make a linker script behave the same way with lld as it did
with bfd, I discovered that lld currently doesn't diagnose overlapping
output sections. I was getting very strange runtime failures which I
tracked down to overlapping sections in the resulting binary. When linking
with ld.bfd overlapping output sections are an error unless
--noinhibit-exec is passed and I believe lld should behave the same way
here to avoid surprising crashes at runtime.

Diff Detail

Repository
rL LLVM

Event Timeline

arichardson created this revision.Dec 9 2017, 1:11 AM

Speed up lld invocation in test/ELF/many-alloc-sections.s lld invocation from 290s to <1s by not checking empty sections

grimar edited edge metadata.Dec 11 2017, 4:12 AM

Speed up lld invocation in test/ELF/many-alloc-sections.s lld invocation from 290s to <1s by not checking empty sections

I think we already had patches for that purpose before (I can be missing details, though I mean general idea was to avoid overlapping things when linker script is used,
probably it were PT_LOADs, but anyways).
And I think last time we stopped on a point that we are not going to stop users from breaking output if they want to do that, because with linker script it is easy to break everything.

That position can be revisited probably, if you have good arguments for that :)

Speed up lld invocation in test/ELF/many-alloc-sections.s lld invocation from 290s to <1s by not checking empty sections

I think we already had patches for that purpose before (I can be missing details, though I mean general idea was to avoid overlapping things when linker script is used,
probably it were PT_LOADs, but anyways).
And I think last time we stopped on a point that we are not going to stop users from breaking output if they want to do that, because with linker script it is easy to break everything.

That position can be revisited probably, if you have good arguments for that :)

My argument is that I wasted hours of debugging time due to a typo in the linker script. If lld had warned me about the overlapping output sections that would not have happened.
If a user really wants to create broken output with a linker script we have --noinhibit-exec for that use case.

ruiu edited edge metadata.Dec 11 2017, 9:15 PM

The amount of new code seems a bit overwhelming. I wonder if there's a better and simpler way of doing it. In what condition do sections overlap?

We do have a check if "." doesn't rewind. Doesn't it work as a prevention for section overlapping?

In D41046#952044, @ruiu wrote:

We do have a check if "." doesn't rewind. Doesn't it work as a prevention for section overlapping?

We removed this check for linking linux kernel. We check that Dot does not rewind inside a section declaration, but do not have a check for outside case.
(https://github.com/llvm-mirror/lld/blob/master/ELF/LinkerScript.cpp#L105)

ruiu added a comment.Dec 11 2017, 9:55 PM

If that's the case, creating overlapping sections is not an issue, but actually we have to be able to create such sections, no?

grimar added a comment.EditedDec 11 2017, 10:14 PM
In D41046#952097, @ruiu wrote:

If that's the case, creating overlapping sections is not an issue, but actually we have to be able to create such sections, no?

Not exactly. Linux kernel script does not have overlapping sections.

It changes the Dot though (from something like 0xFFFFXXXX to 0 as far I remember) in the way the simple check that Dot after
change is greater than Dot before change simply did not work, so we had to remove it.

In D41046#952044, @ruiu wrote:

The amount of new code seems a bit overwhelming. I wonder if there's a better and simpler way of doing it. In what condition do sections overlap?

We do have a check if "." doesn't rewind. Doesn't it work as a prevention for section overlapping?

The overlapping that I got happened because we explicitly assign (virtual) address 0x8000 to a section and 0x8200 to another section.
Also I don't think overlapping load addresses can be fixed by checking for . not rewinding.

I can definitely reduce the amount of new code by inlining struct AddressRange but I felt that adding it made the code easier to understand.
Most of the new code is comments. I'm not sure I can make it much simpler while checking file offsets as well as load and virtual addresess.
I believe having this check in LLD is not only useful to catch errors in linker scripts but also to ensure that the code assigning addresses and offsets doesn't break accidentally.

  • Rebased
  • slightly simpified patch
  • fixed an error in a test that was found by this patch
  • Rebased
  • Made slightly more efficient by using std::partition instead of comparing in every loop iteration
ruiu added inline comments.Jan 30 2018, 11:00 AM
ELF/Writer.cpp
1880–1882 ↗(On Diff #131968)

Remove llvm::.

1885 ↗(On Diff #131968)

It's not a trivial function, so it needs a function comment.

1889–1890 ↗(On Diff #131968)

Please do not use auto in lld unless the actual type is obvious in a very narrow context (e.g. RHS).

ruiu added inline comments.Jan 30 2018, 11:22 AM
ELF/Writer.cpp
1889–1890 ↗(On Diff #131968)

These variable names are a bit too lengthy and don't match other code in lld. Also this patch uses higher order functions than other code, so it feels different. I'd make this consistent with other code.

E.g. I'd write this part as

std::vector<OutputSection *> Sections;
for (OutputSection *Sec : AllSections)
  if (!ShouldSkip(Sec))
    Sections.push_back(Sec);
1893 ↗(On Diff #131968)

Insert a blank line before a multi-line comment.

1904 ↗(On Diff #131968)

Please don't use auto

1908 ↗(On Diff #131968)

Isn't this condition enough?

if (GetStart(Other) < GetStart(Sec) + Sec->Size)
1929 ↗(On Diff #131968)

This is not a performance critical path, so we shouldn't optimize it by using reserve.

1932–1933 ↗(On Diff #131968)

I'd write this with a plain for-loop for consistency with other code.

for (OutputSection *Sec : OutputSections)
  if (Sec->Size > 0)
    Sections.push_back(Sec);
1964 ↗(On Diff #131968)

Insert a blank line.

arichardson marked 10 inline comments as done.Jan 30 2018, 12:10 PM
arichardson added inline comments.
ELF/Writer.cpp
1889–1890 ↗(On Diff #131968)

Looking at this code again, the previous version without std::partition was much easier to understand. I've also updated to use for loops instead of copy_if

1908 ↗(On Diff #131968)

It is now that the sections are sorted. Originally I didn't sort by address so it was required.

arichardson marked 2 inline comments as done.

Addressed comments

ruiu added inline comments.Jan 30 2018, 1:18 PM
ELF/Writer.cpp
1878 ↗(On Diff #132027)

Addr and Len

1936–1940 ↗(On Diff #132027)

I don't think we need this as well. In general we shouldn't optimize for speed unless it is necessary.

arichardson marked an inline comment as done.

Updated for review comments

ELF/Writer.cpp
1936–1940 ↗(On Diff #132027)

Since I simplified the check to GetStart(Other) < Start + Sec->Size it is actually required. I have folded the Size > 0 check into the if (ShouldSkip(Sec))and added a comment instead.

ruiu accepted this revision.Jan 30 2018, 1:55 PM

LGTM with these changes. Thanks!

ELF/Writer.cpp
1915–1916 ↗(On Diff #132044)

Since it's obvious, you may use auto * here if you want.

1921 ↗(On Diff #132044)

nit: else break seems a bit tricky. I'd prefer this style because of less indentation.

if (Start + Sec->Size <= GetStart(Other))
  break;
errorOrWarn(...);
This revision is now accepted and ready to land.Jan 30 2018, 1:55 PM
This revision was automatically updated to reflect the committed changes.
arichardson marked an inline comment as done.
jhenderson added a subscriber: jhenderson.EditedFeb 2 2018, 2:52 AM

Following this change, there's now always a warning or error with overlapping sections, but this seems undesirable in the occasional case where a user actually wants this behaviour, so I've suggested that we also add --no-check-sections from ld.bfd in https://bugs.llvm.org/show_bug.cgi?id=36205.