[ELF] - Allow LLD to produce file symbols.

[grimar]

Summary

This is for PR36716 and
this enables emitting file local symbols.

Output size affect is minor:
lld binary size changes from 52,883,408 to 52,949,400
clang binary size changes from 83,136,456 to 83,219,600

[• espindola]

test/ELF/file-sym.s

0

You can probably just delete this file now. We will add a more complete one with the fix to PR36716.

[• espindola]

BTW, how come lld is larger than clang in your measurements?

[grimar]

In D45261#1057011, @espindola wrote:

BTW, how come lld is larger than clang in your measurements?

It was a mistake in the description (wrong numbers order), I fixed it.

[grimar]

In D45261#1057011, @espindola wrote:

BTW, how come lld is larger than clang in your measurements?

Ah, sorry. I need to change 'lld' and 'clang' too. My mistake. Sure lld < clang.

[grimar]

• Remove the file-sym.s test case.

[• espindola]

LGTM, now we just have to sort them :-)

[ruiu]

By the way, when you mention a bug in a commit message, can you use a link (e.g. https://bugs.llvm.org/show_bug.cgi?id=36716) instead of PRxxxxx for me? Currently it takes like 10 seconds to just open a bug described in a commit message.

[smeenai]

@ruiu just in case you didn't know, the URL https://llvm.org/PR36716 will do a redirect to the right place.

[ruiu]

Thank you for the tip!

George, please just append https://llvm.org/ when you mention a bug. :)

[ruiu]

Peter,

What do you think about this change?

[pcc]

In D45261#1057057, @espindola wrote:

LGTM, now we just have to sort them :-)

Don't we need to do that at the same time as emitting them? The generic ABI says:

STT_FILE
Conventionally, the symbol's name gives the name of the source file associated with the object file. A file symbol has STB_LOCAL binding, its section index is SHN_ABS, and it precedes the other STB_LOCAL symbols for the file, if it is present.

[• espindola]

STT_FILE
Conventionally, the symbol's name gives the name of the source file associated with the object file. A file symbol has STB_LOCAL binding, its section index is SHN_ABS, and it precedes the other STB_LOCAL symbols for the file, if it is present.

Yes, looks like we have to fix PR36716 in one go. Do you have a preference over the two orders listed on the bug?

[pcc]

This one seems a little better in my opinion, for the reason you mentioned.

file1, local1, hidden1, file2, local2, hidden2

It also seems to be the only one that conforms to the ABI because hidden symbols will be STB_LOCAL in the output file.

[grimar]

In D45261#1057743, @espindola wrote:

STT_FILE
Conventionally, the symbol's name gives the name of the source file associated with the object file. A file symbol has STB_LOCAL binding, its section index is SHN_ABS, and it precedes the other STB_LOCAL symbols for the file, if it is present.

Yes, looks like we have to fix PR36716 in one go.

precedes the other STB_LOCAL symbols for the file

I think this is always true with this patch. Even with asm below, first local symbol in the object is a file symbol.
LLD naturally places all local symbols in order in copyLocalSymbols.

.global bar1
.hidden bar1
bar1:

.file "file1"
foo1:

I'll post the patch fixing PR36716 in a few minutes, but I think we are safe to land this first to simplify the review of the second one.

[• espindola]

True, this patch seems safe.

Please check in, but add a trivial test from PR36716 so that we see what it is producing.

ELF/Writer.cpp

if (errorCount())
     return;
 
   if (auto E = Buffer->commit())
     error("failed to write to the output file: " + toString(std::move(E)));
 }
 
 static bool shouldKeepInSymtab(SectionBase *Sec, StringRef SymName,
                                const Symbol &B) {
-  if (B.isFile() || B.isSection())
+  if (B.isSection())
     return false;
 
   // If sym references a section in a discarded group, don't keep it.
   if (Sec == &InputSection::Discarded)
     return false;
 
   if (Config->Discard == DiscardPolicy::None)
     return true;

test/ELF/file-sym.s

-# Check that we do not keep STT_FILE symbols in the symbol table
-
-# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t.o
-# RUN: ld.lld %t.o -shared -o %t.so
-# RUN: llvm-readobj -symbols %t.so | FileCheck %s
-
-# REQUIRES: x86
-
-# CHECK-NOT: Name: xxx
-
-.file "xxx"
-.file ""

test/ELF/local-symbols-order.s

+# REQUIRES: x86
+
+# RUN: echo '.file "file2"; foo2:; .global bar2; .hidden bar2; bar2:' > %t2.s
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %t2.s -o %t2.o
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t1.o
+
+# RUN: ld.lld -o %t %t1.o %t2.o
+# RUN: llvm-readobj -symbols -elf-output-style=GNU %t | FileCheck %s
+
+## Show the order of the local symbols produced currently by LLD.
+
+# CHECK:      Num:    Value          Size Type    Bind   Vis      Ndx Name
+# CHECK-NEXT:   0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
+# CHECK-NEXT:   1: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS file1
+# CHECK-NEXT:   2: 0000000000201000     0 NOTYPE  LOCAL  DEFAULT    1 foo1
+# CHECK-NEXT:   3: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS file2
+# CHECK-NEXT:   4: 0000000000201000     0 NOTYPE  LOCAL  DEFAULT    1 foo2
+# CHECK-NEXT:   5: 0000000000201000     0 NOTYPE  LOCAL  HIDDEN     1 bar1
+# CHECK-NEXT:   6: 0000000000201000     0 NOTYPE  LOCAL  HIDDEN     1 bar2
+
+foo1:
+
+.global bar1
+.hidden bar1
+bar1:
+
+.file "file1"

test/ELF/lto/data-ordering-lto.s

 # REQUIRES: x86
 # RUN: llvm-mc -filetype=obj -triple=x86_64-scei-ps4 %s -o %t.o
 # RUN: llvm-as %p/Inputs/data-ordering-lto.ll -o %t.bc
 
 # Set up the symbol file
 # RUN: echo "tin  " > %t_order_lto.txt
 # RUN: echo "dipsy " >> %t_order_lto.txt
 # RUN: echo "pat " >> %t_order_lto.txt
 
 # RUN: ld.lld --symbol-ordering-file %t_order_lto.txt %t.o %t.bc -o %t2.out
 # RUN: llvm-readobj -elf-output-style=GNU -t %t2.out| FileCheck %s
 
 # Check that the order is tin -> dipsy -> pat.
 
-# CHECK: Symbol table '.symtab' contains 5 entries:
+# CHECK: Symbol table '.symtab' contains 6 entries:
 # CHECK-NEXT:    Num:    Value          Size Type    Bind   Vis      Ndx Name
 # CHECK-NEXT:      0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
-# CHECK-NEXT:      1: 0000000000201000     0 NOTYPE  GLOBAL DEFAULT    1 _start
-# CHECK-NEXT:      2: 0000000000202004     4 OBJECT  GLOBAL DEFAULT    2 dipsy
-# CHECK-NEXT:      3: 0000000000202008     4 OBJECT  GLOBAL DEFAULT    2 pat
-# CHECK-NEXT:      4: 0000000000202000     4 OBJECT  GLOBAL DEFAULT    2 tin
+# CHECK-NEXT:      1: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS {{.*}}.o
+# CHECK-NEXT:      2: 0000000000201000     0 NOTYPE  GLOBAL DEFAULT    1 _start
+# CHECK-NEXT:      3: 0000000000202004     4 OBJECT  GLOBAL DEFAULT    2 dipsy
+# CHECK-NEXT:      4: 0000000000202008     4 OBJECT  GLOBAL DEFAULT    2 pat
+# CHECK-NEXT:      5: 0000000000202000     4 OBJECT  GLOBAL DEFAULT    2 tin
 
 .globl _start
 _start:
   movl $pat, %ecx
   movl $dipsy, %ebx
   movl $tin, %eax

test/ELF/lto/parallel-internalize.ll

 ; CHECK-NEXT:     Value: 0x0
 ; CHECK-NEXT:     Size: 0
 ; CHECK-NEXT:     Binding: Local (0x0)
 ; CHECK-NEXT:     Type: None (0x0)
 ; CHECK-NEXT:     Other: 0
 ; CHECK-NEXT:     Section: Undefined (0x0)
 ; CHECK-NEXT:   }
 ; CHECK-NEXT:   Symbol {
+; CHECK-NEXT:     Name: {{.*}}.o
+; CHECK-NEXT:     Value: 0x0
+; CHECK-NEXT:     Size: 0
+; CHECK-NEXT:     Binding: Local
+; CHECK-NEXT:     Type: File
+; CHECK-NEXT:     Other: 0
+; CHECK-NEXT:     Section: Absolute
+; CHECK-NEXT:   }
+; CHECK-NEXT:   Symbol {
+; CHECK-NEXT:     Name: {{.*}}.o
+; CHECK-NEXT:     Value: 0x0
+; CHECK-NEXT:     Size: 0
+; CHECK-NEXT:     Binding: Local
+; CHECK-NEXT:     Type: File
+; CHECK-NEXT:     Other: 0
+; CHECK-NEXT:     Section: Absolute
+; CHECK-NEXT:   }
+; CHECK-NEXT:   Symbol {
 ; CHECK-NEXT:     Name: bar
 ; CHECK-NEXT:     Value: 0x201010
 ; CHECK-NEXT:     Size: 8
 ; CHECK-NEXT:     Binding: Local (0x0)
 ; CHECK-NEXT:     Type: Function (0x2)
 ; CHECK-NEXT:     Other [ (0x2)
 ; CHECK-NEXT:       STV_HIDDEN (0x2)
 ; CHECK-NEXT:     ]

test/ELF/lto/relocatable.ll

 ; REQUIRES: x86
 ; RUN: llvm-as %s -o %t1.o
 ; RUN: ld.lld %t1.o -r -o %t
 ; RUN: llvm-readobj -symbols %t | FileCheck %s
 
 ; CHECK:       Symbols [
 ; CHECK-NEXT:   Symbol {
 ; CHECK-NEXT:     Name:
 ; CHECK-NEXT:     Value: 0x0
 ; CHECK-NEXT:     Size: 0
 ; CHECK-NEXT:     Binding: Local
 ; CHECK-NEXT:     Type: None
 ; CHECK-NEXT:     Other: 0
 ; CHECK-NEXT:     Section: Undefined
 ; CHECK-NEXT:   }
 ; CHECK-NEXT:   Symbol {
+; CHECK-NEXT:     Name: {{.*}}.o
+; CHECK-NEXT:     Value: 0x0
+; CHECK-NEXT:     Size: 0
+; CHECK-NEXT:     Binding: Local
+; CHECK-NEXT:     Type: File
+; CHECK-NEXT:     Other: 0
+; CHECK-NEXT:     Section: Absolute
+; CHECK-NEXT:   }
+; CHECK-NEXT:   Symbol {
 ; CHECK-NEXT:     Name:
 ; CHECK-NEXT:     Value: 0x0
 ; CHECK-NEXT:     Size: 0
 ; CHECK-NEXT:     Binding: Local
 ; CHECK-NEXT:     Type: Section
 ; CHECK-NEXT:     Other: 0
 ; CHECK-NEXT:     Section: .text
 ; CHECK-NEXT:   }

test/ELF/lto/symbol-ordering-lto.s

 # REQUIRES: x86
 # RUN: llvm-mc -filetype=obj -triple=x86_64-scei-ps4 %s -o %t.o
 # RUN: llvm-as %p/Inputs/symbol-ordering-lto.ll -o %t.bc
 
 # Set up the symbol file
 # RUN: echo "tin  " > %t_order_lto.txt
 # RUN: echo "_start " >> %t_order_lto.txt
 # RUN: echo "pat " >> %t_order_lto.txt
 
 # RUN: ld.lld --symbol-ordering-file %t_order_lto.txt %t.o %t.bc -o %t2.out
 # RUN: llvm-readobj -elf-output-style=GNU -t %t2.out| FileCheck %s
 
 # Check that the order is tin -> _start -> pat.
 
-# CHECK: Symbol table '.symtab' contains 4 entries:
+# CHECK: Symbol table '.symtab' contains 5 entries:
 # CHECK-NEXT:   Num:    Value          Size Type    Bind   Vis      Ndx Name
 # CHECK-NEXT:     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
-# CHECK-NEXT:     1: 0000000000201008     0 NOTYPE  GLOBAL DEFAULT    1 _start
-# CHECK-NEXT:     2: 0000000000201020     6 FUNC    GLOBAL DEFAULT    1 pat
-# CHECK-NEXT:     3: 0000000000201000     6 FUNC    GLOBAL DEFAULT    1 tin
+# CHECK-NEXT:     1: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS {{.*}}.o
+# CHECK-NEXT:     2: 0000000000201008     0 NOTYPE  GLOBAL DEFAULT    1 _start
+# CHECK-NEXT:     3: 0000000000201020     6 FUNC    GLOBAL DEFAULT    1 pat
+# CHECK-NEXT:     4: 0000000000201000     6 FUNC    GLOBAL DEFAULT    1 tin
 
 .globl _start
 _start:
   call pat
   call tin