If I read correctly, this is to assign /DISCARD to the memory region rodata and expect .rodata to inherit the memory region?

Sorry I didn't follow the discussion in D61186 and didn't notice D61251. It seems @grimar and @peter.smith were fine with D61251 (this patch is essentially D61251). @andrewng if you are ok with letting this somewhat contrived example fail, we can probably still mark this as resolved (wontfix).

George's comment reminds me why we didn't take this approach in the first place, as it doesn't work correctly for the following test case:

In D61599#1497031, @mvhooren wrote:

Applied Clang Format to the changed files. Renamed bufferSize -> BufSize to match naming convention..

Some of the changes in "lib/Support/Unix/Memory.inc" don't appear to have been clang formatted.

In D61599#1493926, @mvhooren wrote:

In D61599#1493889, @lhames wrote:

I don't think this is the right solution. MemoryBlock's purpose is to track the allocated memory. What the client does with that memory (including subdividing it) is their business. The client already knows the number of bytes they're requesting, so the fix for fragmentation in SectionMemoryManager should be limited to fixes in SectionMemoryManager.

Yes I agree it would be better if the Size member of MemoryBlock just held the allocated size. This would mean that r357058 and r351916 (and perhaps others) should be reverted.

Submitted bug https://bugs.llvm.org/show_bug.cgi?id=41673 regarding the issue that the /DISCARD/ section is in the output.

Can I land this patch?

In D60131#1483142, @phosek wrote:

D61197 seems to have addressed the issue, D61186 made no difference.

In D60131#1482865, @phosek wrote:

This change broke our kernel build, I filed a bug PR41653 which has more details, but it seems like that with this change, linker produces wrong layout. Can we please revert this change until we figure out how to address this issue?

In D61186#1482564, @peter.smith wrote:

Just to clarify my understanding, are you referring to (https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/4/html/Using_ld_the_GNU_Linker/sections.html#OUTPUT-SECTION-PHDR)

If a section is assigned to one or more segments, then all subsequent allocated sections will be assigned to those segments as well, unless they use an explicitly :phdr modifier. You can use :NONE to tell the linker to not put the section in any segment at all.

Something like:

/DISCARD/ : { *(.comment) } :my_phdr
 .name : { *(.name) }
...

Where we'd want .name to to be assigned to my_phdr and any following OutputSections? I can see that this is how ld.bfd and ld.gold behave, although as you say this is somewhat of a contrived example.

This is how this function used to behave (see r316307), when findOrphanPos ignored non-live sections.

In D61186#1482350, @grimar wrote:

I am still not sure we should not just remove the /DISCARD/ like D61251 did (your explanation makes sense to me, though).

Simplified test as suggested in review.

In D61186#1482074, @grimar wrote:

I think it is 2 lines of code change actually :) I believe the much more correct way to fix it is this: https://reviews.llvm.org/D61251
With this approach there is no /DISCARD/ in the output.
(note, I did not change the original test case there).

In D61186#1480437, @grimar wrote:

From what I see in the output for this case LLD creates an /DISCARD/ output section.
This does not look correct to me, because /DISCARD/ is kind of key word, but not a section name I believe.
Also, bfd does not do that.

So I think issue first of all is that we are trying to create output section with this name, it should never be in the output.

In D60131#1471202, @ruiu wrote:

IIUC, you are fixing an issue that a segment whose size is 0 doesn't have a file offset that is congruent modulo page size. Is this correct?

The file offset for an empty segment is not actually significant; we can just set a dummy value. Have you considered that?

Is this patch acceptable or do we need to start looking for an alternative (and most probably more complex solution) to the issue described above?

This would be the follow up patch to D60131 as a fix for the orphan PHDR test failure.

The follow up patch to fix the orphan PHDR test is D60273.

In D60131#1453128, @jakehehrlich wrote:

This seems like the wrong fix to me. Shouldn't we be fixing the alignment bug with respect to segments not changing things like this?

If I recall correctly I made a change to make NOBITS the default because some sections nthat could have been NOBITS we're winding up as PROGBITS. I don't remember the exact details. I'm fairly sure this change regresses that point. You've changedna bunch of test from expecting NOBITS to expecting PROGBITS but that seems unjustified.

I am not sure we have the practice to disable the test cases. What do you think about uploading the fix for orphan behavior
in a separate change rebased on this patch too, so that these 2 patches can be reviewed together?
(in that case, having XFAIL here is fine I think).

Updated to address review comments.

The default was SHT_PROGBITS in the past, and was changed to SHT_NOBITS to "reduce file size". However, it's not clear that it actually reduces output file size, but only reduces the value of file size in the output. In any case, it does cause the alignment problem described and shown in the symbol-only-align.test test.

In D59824#1443159, @tlively wrote:

I think there are more tests that should use -o - in wasm/target-feature-none.yaml, but otherwise the wasm changes lgtm.

I've added D59824 to restore testing and usage of output to "-".

The stdout.s test you added in fact failed on Windows because this change was missing, and @thakis disabled it in rL351949. So, I think we can just re-enable stdout.s to test this.

Ping...

Updated the patch based on review suggestion.

As suggested, I have switched back to what was my original change to fix this issue.

In D57960#1390773, @ruiu wrote:

This class should do a minimum number of attempt to remove a file, instead of trying to remove a file at various points to increase the chance to successfully removing the file. Why don't you close the file before removing the file?

Update to the eh-frame-negative-pcrel-sdata8.s test.

Updated testing as per review comments.

In D49557#1168549, @ruiu wrote:

Thanks! How did yoy find this?

Abandoned in favour of D44923.

Updated to address review comments.

In D44898#1048789, @ruiu wrote:

2.8%. Somehow clang-fsds got 2.39% faster.

That speedup is interesting. I don't have a good theory to explain it though.

Even if this patch is somewhat performance-neutral, we still should avoid extra memory allocation if we can. Let's try ignoring synthetic sections in ICF. That's I think better.

In D44898#1048688, @ruiu wrote:

How much did Scylla get slower?

Maybe the best option is just disabling ICF of synthetic sections?

That sounds like a better way of fixing the issue.

Updated to fix issue with clang compilation.

Removed assertions and tidied up white space a little.

Updated test to address review comments.

In D44898#1048263, @espindola wrote:

Seems reasonable. I will just do a quick check on the performance impact.

I think the PLT writing should handle the padding itself. It certainly feels more correct. It looks though D44775 is heading in that direction too.

In D44682#1043455, @ruiu wrote:

Instead of doing this, we should fill executable segments with trap instructions even if linker script is in use, shouldn't we?

Looking at the EhFrameSection writing and finalize code, it would seem that
some kind of alignment is being applied. This aligning is also the reason
why for x86_64 the mapping of CIE/FDE's to the output is so fragmented,
because all input we have seen is 4-byte aligned, but LLD uses 8-byte
alignment for the output.

Apart from the typo and the value for the alignment, LGTM.

Abandoning this revision in favour of D44168.

There are also some slight differences in the output between this implementation and the one in D44168. This patch does not output the "+0x0" for a zero offset and outputs the input section's alignment for the "Align" column. Which style of output is preferred?

Updated to address review comments.

I agree that this change is simpler, but it also doesn't produce the same output as D42960. Part of the reason for the "complexity" in my change in D42960 was to try to prevent the map file output from getting too big/verbose.

Updated to address comments.

Rebased and updated with minor change to make the output offset hex rather than decimal. Modified existing map file test to demonstrate the proposed .eh_frame output.

In D40366#933070, @zturner wrote:

I absolutely do not want to disable support for ConcRT under any circumstances unless we find that it is a completely broken library, which I highly doubt. Even then, I would want to ask Microsoft to fix it.

What is the exact problem you're running into?

In D40119#927529, @grimar wrote:

I think this made few bots be unhappy:
http://lab.llvm.org:8011/builders/clang-with-thin-lto-ubuntu/builds/6726
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/builds/3069

Workaround no longer required. This issue is also fixed by commit r315932.

Committed in rL315292: [LLD] Fix findOrphanPos to consistently ignore "dead" OutputSection's.

Abandoned in favour of change https://reviews.llvm.org/D38571.

In D38539#889005, @ruiu wrote:

Can you try https://reviews.llvm.org/D38571 to see if it will fix your problem? Thanks.

What is the problem? std::thread::detach is a part of C++11, and it is hard to believe that we shouldn't use it at all.

In D38539#888333, @ruiu wrote:

From my understanding of FileOutputBuffer::commit in its current form, it does a call to sys::fs::rename for a regular file. So there is no direct association to the "unlink" performance issue described in the comment to unlinkAsync. Although, it could be that the implementation of rename on Linux could suffer from similar performance problems (i.e. it effectively still performs an unlink). Previously, FileOutputBuffer::create did remove (unlink) the target file. I believe that you recently removed this behaviour and it was this unlink that the unlinkAsync was trying to improve.

If you try it yourself, you'd realize that rename(2)'ing onto an existing file is as slow as calling unlink(2) on that file. My previous change to FileOutputBuffer doesn't fix the correctness issue in the usual use case, but it also improved the speed of lld, because lld no longer tries to unlink a file in the main thread. Before making that change, it tried to do that in the Driver by tryCreateFile.

Hi Rui,

Updated based on suggestion. I still think that keeping the common "AlignSize" is better and safer.

A similar fix was required for the string table, see https://reviews.llvm.org/D36267.