[LLD] Add support for --unique option

[davidb]

Summary

Places orphan sections into a unique output section. This prevents the merging of orphan sections of the same name.
Matches behaviour of GNU ld --unique. --unique=pattern is not implemented.

Motivated user case shown in the test has 2 local symbols as they would appear if C++ source has been compiled with -ffunction-sections. The merging of these sections in the case of a partial link (-r) may limit the effectiveness of -gc-sections of a subsequent link.

[davidb]

Fix lint checks

[davidb]

rebase

[MaskRay]

Do you have a plan to implement --unique=pattern (overrides output section descriptions)? It is different from --unique (only applies to orphan sections).

The merging of these sections in the case of a partial link (-r) inhibits deadstripping.

Dead stripping is a Mach-O term. Non --unique does not inhibit --gc-sections (without -r).
Without --unique, some orphan sections may be unnecessarily kept (as a whole their "liveness" can be larger than the composing parts).

Note that -r --gc-sections is not implemented in lld.

lld/ELF/LinkerScript.cpp

691 ↗

(On Diff #247933)

Use else if

lld/ELF/Options.td

378 ↗

(On Diff #247933)

No period.

Add an item to lld/docs/ld.lld.1 (manpage)

lld/test/ELF/Inputs/unique-orphans.s

1 ↗

(On Diff #247933)

Delete the file.

lld/test/ELF/unique-orphans.s

2

Delete -unknown-linux

3

Keep one REQUIRES

6

The instructions are verbose and unnecessary.

.section .foo,"a",unique,1
.byte 1

.section .foo,"a",unique,2
.byte 2

.section .foo,"a",unique,3
.byte 3

Check both ld.lld %t.o -o %t and ld.lld -r %t.o -o %t.ro

7

Test an output section description suppresses --unique.

echo 'SECTIONS { .foo : { *(.foo) }}' > %t.script`

See test/ELF/linkerscript/input-archive.s for a recent example.

25

Check sections with llvm-readelf -S, instead of objdump (may not even be installed)

29

Not necessary. There are plenty of tests for the non --unique case.

[davidb]

In D75536#1903727, @MaskRay wrote:

Do you have a plan to implement --unique=pattern (overrides output section descriptions)? It is different from --unique (only applies to orphan sections).

I don't have a plan to implement --unique=pattern, as the default is good enough for my needs. However I would consider getting back and doing it when I next have some free cycles.

The merging of these sections in the case of a partial link (-r) inhibits deadstripping.

Dead stripping is a Mach-O term. Non --unique does not inhibit --gc-sections (without -r).
Without --unique, some orphan sections may be unnecessarily kept (as a whole their "liveness" can be larger than the composing parts).

Note that -r --gc-sections is not implemented in lld.

Sorry, force of habit. PlayStation toolchains also use the term dead-stripping for ELF binaries. I did indeed mean gc-sections.

The current workflow involves multiple link stages, involving a partial link and then an executable link. It is the second link that performs -gc-sections. We would like to keep all functions in their own individual sections after the partial link.

I could update the test to demonstrate this if it is at all useful.

[MaskRay]

In D75536#1903914, @davidb wrote:

In D75536#1903727, @MaskRay wrote:

Do you have a plan to implement --unique=pattern (overrides output section descriptions)? It is different from --unique (only applies to orphan sections).

I don't have a plan to implement --unique=pattern, as the default is good enough for my needs. However I would consider getting back and doing it when I next have some free cycles.

The merging of these sections in the case of a partial link (-r) inhibits deadstripping.

Dead stripping is a Mach-O term. Non --unique does not inhibit --gc-sections (without -r).
Without --unique, some orphan sections may be unnecessarily kept (as a whole their "liveness" can be larger than the composing parts).

Note that -r --gc-sections is not implemented in lld.

Sorry, force of habit. PlayStation toolchains also use the term dead-stripping for ELF binaries. I did indeed mean gc-sections.

The current workflow involves multiple link stages, involving a partial link and then an executable link. It is the second link that performs -gc-sections. We would like to keep all functions in their own individual sections after the partial link.

I could update the test to demonstrate this if it is at all useful.

A test is not needed. Just make the test minimal (for example, as I suggested).

Please update SUMMARY by mentioning the --gc-sections scenario and delete the mention of deadstripping.

[davidb]

• review comments

[davidb]

In D75536#1904145, @MaskRay wrote:

In D75536#1903914, @davidb wrote:

In D75536#1903727, @MaskRay wrote:

Do you have a plan to implement --unique=pattern (overrides output section descriptions)? It is different from --unique (only applies to orphan sections).

I don't have a plan to implement --unique=pattern, as the default is good enough for my needs. However I would consider getting back and doing it when I next have some free cycles.

The merging of these sections in the case of a partial link (-r) inhibits deadstripping.

Dead stripping is a Mach-O term. Non --unique does not inhibit --gc-sections (without -r).
Without --unique, some orphan sections may be unnecessarily kept (as a whole their "liveness" can be larger than the composing parts).

Note that -r --gc-sections is not implemented in lld.

Sorry, force of habit. PlayStation toolchains also use the term dead-stripping for ELF binaries. I did indeed mean gc-sections.

The current workflow involves multiple link stages, involving a partial link and then an executable link. It is the second link that performs -gc-sections. We would like to keep all functions in their own individual sections after the partial link.

I could update the test to demonstrate this if it is at all useful.

A test is not needed. Just make the test minimal (for example, as I suggested).

Please update SUMMARY by mentioning the --gc-sections scenario and delete the mention of deadstripping.

Thank you for the review comments, especially wrt the tests. I have updated the summary, just need to add a comment to the manpage, which I'll try to get round to in the morning.

[jhenderson]

lld/ELF/Options.td

377 ↗

(On Diff #248045)

Nit: too many blank lines here.

lld/test/ELF/unique-orphans.s

13

instance -> instances (here and below)

30

Nit: no newline at EOF.

[grimar]

lld/test/ELF/unique-orphans.s

13

Consider using ## for comments. It is a modern common way.

17

Seems you can have a single chack block:

# CHECK-3:   .foo
# CHECK-NOT: .foo

And remove --check-prefix from everywhere?

19

.text.foo -> .foo? (and above)

[MaskRay]

lld/test/ELF/unique-orphans.s

25

Add a comment, along the lines of

## Test that --unique does not affect sections specified in output section descriptions.

25

Not done. Use llvm-readelf -S

[davidb]

lld/test/ELF/unique-orphans.s

25

I added the llvm-prefix so it should pick the right objdump. There are 2x more uses of llvm-objdump than readelf in the lld test suite, 4x more in LLVM as a whole. What is the motivation for readelf?

[davidb]

• review comments

[davidb]

• fix formatting from clang-format

[davidb]

• review comments

[davidb]

• replace objdump with readelf

[davidb]

• rebase

[MaskRay]

lld/ELF/Config.h

197 ↗

(On Diff #248456)

Delete = false

lld/ELF/Driver.cpp

970 ↗

(On Diff #248456)

lexicographical order

lld/test/ELF/unique-orphans.s

14

The comment ## Test with -r is redundant and should be deleted.

[davidb]

• review comments

[davidb]

lld/ELF/Config.h

197 ↗

(On Diff #248456)

Why do you not want me to be explicit here? Many other options above are

lld/ELF/Driver.cpp

970 ↗

(On Diff #248456)

was result of rebase

[davidb]

lld/test/ELF/unique-orphans.s

14

shame.

[MaskRay]

Places orphan sections into a unique output section. This prevents the merging of orphan sections of the same name. Matches behaviour of gnu LD option --unique[=SECTION], with the exception I haven't added the wildcard SECTION support.

gnu LD -> GNU ld

Matches behaviour of GNU ld --unique. --unique=pattern is not implemented.

lld/test/ELF/unique-orphans.s

24

Nit: --check-prefix=SCRIPT

UNIQUE_ is not necessary in the context.

[grimar]

lld/ELF/Config.h

197 ↗

(On Diff #248456)

There is a difference. Look at sysvHash for example whitch is explicitly set to false.
sysvHash is not always set to something in the code. Hence we have to initialize it.

You always assing unique: config->unique = args.hasArg(OPT_unique);. So no need to initialize it explicitly.

[davidb]

• review comments

[davidb]

• rebase

[davidb]

• formatting

[MaskRay]

LGTM. Please wait for @grimar's opinions.

[grimar]

LGTM

lld/test/ELF/unique-orphans.s

 # REQUIRES: x86
 # RUN: llvm-mc -filetype=obj -triple=x86_64 %s -o %t.o
 
 .section .foo,"a",@progbits,unique,1
 .byte 1
 
 .section .foo,"a",@progbits,unique,2
 .byte 2
 
 .section .foo,"a",@progbits,unique,3
 .byte 3
 
 ## We should have 3 instances of orphan section foo.
 ## Test with -r
 # RUN: ld.lld %t.o -r -o %t.ro --unique
-# RUN: llvm-objdump -h %t.ro | FileCheck  %s
+# RUN: llvm-readelf -S %t.ro | FileCheck  %s
 ## Test for execuable link
 # RUN: ld.lld %t.o -o %t.elf --unique 
-# RUN: llvm-objdump -h %t.elf | FileCheck %s
+# RUN: llvm-readelf -S %t.elf | FileCheck %s
 
 # CHECK-COUNT-3: .foo
 # CHECK-NOT: .foo
 
 ## Test that --unique does not affect sections specified in output section descriptions.
 # RUN: echo 'SECTIONS { .foo : { *(.foo) }}' > %t.script
 # RUN: ld.lld %t.o -o %t2.elf -T %t.script --unique 
-# RUN: llvm-objdump -h %t2.elf | FileCheck --check-prefix UNIQUE_SCRIPT %s
+# RUN: llvm-readelf -S %t2.elf | FileCheck --check-prefix UNIQUE_SCRIPT %s
 # UNIQUE_SCRIPT: .foo
 # UNIQUE_SCRIPT-NOT: .foo