This is an archive of the discontinued LLVM Phabricator instance.

[ELF] Change default output section type to SHT_PROGBITS
ClosedPublic

Authored by andrewng on Apr 2 2019, 8:20 AM.

Details

Summary

This fixes an issue where a symbol only section at the start of a
PT_LOAD segment, causes incorrect alignment of the file offset for the
start of the segment. This results in the output of an invalid ELF.

This change breaks an existing test related to orphan sections. See test
orphan-phdrs.s for details.

Diff Detail

Repository
rLLD LLVM Linker

Event Timeline

andrewng created this revision.Apr 2 2019, 8:20 AM

The default was SHT_PROGBITS in the past, and was changed to SHT_NOBITS to "reduce file size". However, it's not clear that it actually reduces output file size, but only reduces the value of file size in the output. In any case, it does cause the alignment problem described and shown in the symbol-only-align.test test.

I intend to fix the orphan behaviour in a separate change, which is why it has been marked as XFAIL for now.

grimar added a comment.Apr 3 2019, 2:22 AM

I intend to fix the orphan behavior in a separate change, which is why it has been marked as XFAIL for now.

I am not sure we have the practice to disable the test cases. What do you think about uploading the fix for orphan behavior
in a separate change rebased on this patch too, so that these 2 patches can be reviewed together?
(in that case, having XFAIL here is fine I think).

test/ELF/linkerscript/symbol-only-align.test
6

In LLD we avoid using multiple echo calls. Please do it with a single call.

11

Could you add a brief description of what this test does and what it checks here please?

andrewng added a subscriber: edd.Apr 3 2019, 2:27 AM
andrewng updated this revision to Diff 193460.Apr 3 2019, 3:07 AM

Updated to address review comments.

andrewng marked 3 inline comments as done.Apr 3 2019, 3:09 AM
andrewng added inline comments.
test/ELF/linkerscript/symbol-only-align.test
6

Had to add ';' for a single echo to work.

andrewng marked an inline comment as done.Apr 3 2019, 3:13 AM

I am not sure we have the practice to disable the test cases. What do you think about uploading the fix for orphan behavior
in a separate change rebased on this patch too, so that these 2 patches can be reviewed together?
(in that case, having XFAIL here is fine I think).

I'm still not 100% sure of the best approach for the other patch for the orphan behaviour. However, this fix is more important.

This seems like the wrong fix to me. Shouldn't we be fixing the alignment bug with respect to segments not changing things like this?

If I recall correctly I made a change to make NOBITS the default because some sections nthat could have been NOBITS we're winding up as PROGBITS. I don't remember the exact details. I'm fairly sure this change regresses that point. You've changedna bunch of test from expecting NOBITS to expecting PROGBITS but that seems unjustified.

This seems like the wrong fix to me. Shouldn't we be fixing the alignment bug with respect to segments not changing things like this?

If I recall correctly I made a change to make NOBITS the default because some sections nthat could have been NOBITS we're winding up as PROGBITS. I don't remember the exact details. I'm fairly sure this change regresses that point. You've changedna bunch of test from expecting NOBITS to expecting PROGBITS but that seems unjustified.

When I first looked into this issue, I did in fact look at the "layout" side for a potential fix. But as I progressed, it was not clear that it might be the most appropriate (or simplest) fix. So then I investigated why this situation was arising at all and it lead me back to this change of the default output section from SHT_PROGBITS to SHT_NOBITS. I tried to find more details for why this change was made, but couldn't find anything concrete other than the reduction in "file size" which does not appear to benefit the actual file output size in most cases.

If the default output section must be SHT_NOBITS, then an alternative solution will be required, but is this really a requirement?

Ping.

Is this patch acceptable or do we need to start looking for an alternative (and most probably more complex solution) to the issue described above?

ruiu added a comment.Apr 17 2019, 11:05 PM

IIUC, you are fixing an issue that a segment whose size is 0 doesn't have a file offset that is congruent modulo page size. Is this correct?

The file offset for an empty segment is not actually significant; we can just set a dummy value. Have you considered that?

test/ELF/linkerscript/orphan-phdrs.s
1

We shouldn't keep a test that doesn't run at all. Please just remove.

test/ELF/linkerscript/symbol-only-align.test
3–6

nit: You can just write them in a single line.

andrewng marked 2 inline comments as done.Apr 18 2019, 12:10 AM

IIUC, you are fixing an issue that a segment whose size is 0 doesn't have a file offset that is congruent modulo page size. Is this correct?

The file offset for an empty segment is not actually significant; we can just set a dummy value. Have you considered that?

No, the actual issue is that a non-empty PT_LOAD segment that starts with an empty symbol only section does not have a file offset that is congruent modulo the page size. As a result, the output is not a valid ELF.

test/ELF/linkerscript/orphan-phdrs.s
1

Just to be clear, you would like me to remove this valid test in this patch and then add it back again in the follow up patch D60273? Or should I submit D60273 first, even though it is the change in this patch that necessitates the change in D60273?

test/ELF/linkerscript/symbol-only-align.test
3–6

I deliberately put them on separate lines for readability. Would you prefer a single line?

ruiu accepted this revision.Apr 18 2019, 12:35 AM

LGTM

No, the actual issue is that a non-empty PT_LOAD segment that starts with an empty symbol only section does not have a file offset that is congruent modulo the page size. As a result, the output is not a valid ELF.

Got it, thanks.

test/ELF/linkerscript/symbol-only-align.test
3–6

Yeah, I'd prefer not to contain too many space characters in a test file. The result of the concatenation is a long single line, as an escaped newline is removed before given to a command. If something goes wrong, that resulted in something like this, which isn't easy to read:

SECTIONS { foo                                                bar                                                baz                                                foo                                                bar                                                baz                        }
.                                                                                                                                                                                                                                            ^ invalid parameter

For this particular case, the concatenated line wouldn't be that long, but in general, I'd like to not use too many escaped newlines.

This revision is now accepted and ready to land.Apr 18 2019, 12:35 AM
This revision was automatically updated to reflect the committed changes.
Herald added a project: Restricted Project. · View Herald TranscriptApr 23 2019, 5:36 AM
phosek added a subscriber: phosek.Apr 29 2019, 12:27 PM

This change broke our kernel build, I filed a bug PR41653 which has more details, but it seems like that with this change, linker produces wrong layout. Can we please revert this change until we figure out how to address this issue?

This change broke our kernel build, I filed a bug PR41653 which has more details, but it seems like that with this change, linker produces wrong layout. Can we please revert this change until we figure out how to address this issue?

Do either or both of D61186 and D61197 help with your issue, as these are related to orphan behaviour as a result of this change to the default output section type.

D61197 seems to have addressed the issue, D61186 made no difference.

D61197 seems to have addressed the issue, D61186 made no difference.

Hopefully, I'll get an LGTM for D61197 soon and will be able to land it. We're waiting on this fix too :). The default output section type of SHT_NOBITS was actually "hiding" a number of issues...

D61197 seems to have addressed the issue, D61186 made no difference.

Hopefully, I'll get an LGTM for D61197 soon and will be able to land it. We're waiting on this fix too :). The default output section type of SHT_NOBITS was actually "hiding" a number of issues...

There's still no response on D61197, can we revert this change until the other one is approved and then reland it? I'd argue that in the current state it's actually worse than before.