[lld][ELF][test] Add additional test coverage for LTO

[jhenderson]

Summary

These are all inspired by existing test coverage we have in an internal testsuite.

[jhenderson]

Add missing new line at EOF.

[grimar]

It feels like common-mixed1.ll and common-mixed2.ll are very similar and can be merged?

lld/test/ELF/lto/bitcode-wrapper.ll

52

I'd comment fields here. They are:
[Magic32, Version32, Offset32, Size32, CPUType32]

lld/test/ELF/lto/common-mixed2.ll

7 ↗

(On Diff #299304)

Perhaps, use split-file or just echo rather than adding a new tiny input?

[MaskRay]

lld/test/ELF/lto/Inputs/common-mixed2.s

1 ↗

(On Diff #299304)

This can be inlined into common-mixed2.ll

[jhenderson]

I'll see if I can merge common-mixed1.ll and common-mixed2.ll and inline the input files. That should allow me to avoid renaming common4.ll too.

lld/test/ELF/lto/resolution.ll

22–23

This test is failing for reasons I've yet to discover. I thought it passed previously, but it's possible it didn't. Currently, it fails because both versions of b are present in the contents of .data, which clearly isn't correct (only one b appears in the symbol table).

[jhenderson]

Reworked resolution.ll and common.ll, fixing the former, and inlining the input files. Also undid the test renaming and made the wrap_bitcode.py script python3 compatible.

[grimar]

You need to update the patch description which still mentions "common-mixed1.ll".
The rest looks fine to me.

lld/test/ELF/lto/bitcode-wrapper.ll

28

aside: we probably want to lowercase this message.

[jhenderson]

In D89775#2346865, @grimar wrote:

You need to update the patch description which still mentions "common-mixed1.ll".
The rest looks fine to me.

Done, thanks.

[grimar]

LGTM. But please wait for @MaskRay.

[jhenderson]

lld/test/ELF/lto/bitcode-wrapper.ll

28

I took a quick look, and this is just one of several error messages within the BitcodeReader and BitcodeAnalyzer files, and most of them use upper-case, so that would probably want to be part of a much wider change. Think we'd need to discuss the preferred style of the library with whoever owns/maintains it before doing that.

[grimar]

lld/test/ELF/lto/bitcode-wrapper.ll

28

Think we'd need to discuss the preferred style of the library with whoever owns/maintains it before doing that.

perhaps we could just lower case all error messages on the lld side.

[jhenderson]

lld/test/ELF/lto/bitcode-wrapper.ll

28

I thought about that, but that would risk us occasionally converting strings to lower-case that shouldn't be (e.g. file paths, names, accronyms like ELF or DWARF etc).

[grimar]

lld/test/ELF/lto/bitcode-wrapper.ll

28

I meant to lower case just the first character. Perhaps it is a low risk here. Though probably
a more safe way is to convert particular error messages. I.e. if it comes from BitcodeReader/BitcodeAnalyzer we could do it in case if we can't just switch to lower case right there.

[MaskRay]

LGTM.

lld/test/ELF/lto/bitcode-wrapper.ll

28

I'd prefer not transforming the error messages. If people search for "Invalid bitcode wrapper header", they can find occurrences in llvm/ and lld/test/ELF. If they happen to change the messages (e.g. to add more information; many BitcodeReader.cpp messages are duplicated in multiple places and are generally not so informational), they can update the lld one as well. Suredly they can use grep -i or rg -i, or not search for the first character, but that is extra hurdle.

We lose a bit of inconsistency, but the loss seems very small to me.

lld/test/ELF/lto/Inputs/common.s

-        .comm   a,8,4

lld/test/ELF/lto/Inputs/resolution.s

-        .data
-        .global a
-a:
-        .long 9

lld/test/ELF/lto/bitcode-wrapper.ll

+; REQUIRES: x86
+
+;; The LLVM bitcode format allows for an optional wrapper header. This test
+;; shows that LLD can handle bitcode wrapped in this way, and also that an
+;; invalid offset in the wrapper header is handled cleanly.
+
+; RUN: rm -rf %t
+; RUN: split-file %s %t
+; RUN: llvm-as %t/ir.ll -o %t.bc
+
+;; Basic case:
+; RUN: %python %t/wrap_bitcode.py %t.bc %t.o 0 0x14
+; RUN: ld.lld %t.o -o %t.elf
+; RUN: llvm-readelf -s %t.elf | FileCheck %s
+
+;; Padding between wrapper header and body:
+; RUN: %python %t/wrap_bitcode.py %t.bc %t.o 0x10 0x24
+; RUN: ld.lld %t.o -o %t.elf
+; RUN: llvm-readelf -s %t.elf | FileCheck %s
+
+; CHECK: _start
+
+;; Invalid offset past end of file:
+; RUN: %python %t/wrap_bitcode.py %t.bc %t2.o 0x10 0xffffffff
+; RUN: not ld.lld %t2.o -o %t2.elf 2>&1 | FileCheck %s --check-prefix=ERR1 -DFILE=%t2.o
+
+; ERR1: error: [[FILE]]: Invalid bitcode wrapper header
+
+;; Invalid offset within file:
+; RUN: %python %t/wrap_bitcode.py %t.bc %t3.o 0x10 0x14
+; RUN: not ld.lld %t3.o -o %t3.elf 2>&1 | FileCheck %s --check-prefix=ERR2 -DFILE=%t3.o
+
+; ERR2: error: [[FILE]]: file doesn't start with bitcode header
+
+;--- ir.ll
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+@_start = global i32 0
+
+;--- wrap_bitcode.py
+## Arguments are: input file, output file, padding size, offset value.
+import struct
+import sys
+
+with open(sys.argv[1], 'rb') as input:
+    bitcode = input.read()
+
+padding = int(sys.argv[3], 16) * b'\0'
+offset = int(sys.argv[4], 16)
+header = struct.pack('<IIIII', 0x0B17C0DE, 0, offset, len(bitcode), 0)
+with open(sys.argv[2], 'wb') as output:
+    output.write(header)
+    output.write(padding)
+    output.write(bitcode)

lld/test/ELF/lto/common.ll

 ; REQUIRES: x86
-; RUN: llvm-as %s -o %t1.o
-; RUN: llvm-mc -triple=x86_64-pc-linux %p/Inputs/common.s -o %t2.o -filetype=obj
-; RUN: ld.lld %t1.o %t2.o -o %t.so -shared
-; RUN: llvm-readobj -S --symbols %t.so | FileCheck %s
+
+;; Show that common symbols defined in both native objects and bitcode files are
+;; properly resolved.
+
+; RUN: rm -rf %t.dir
+; RUN: split-file %s %t.dir
+; RUN: cd %t.dir
+
+;; Case 1: bitcode file has smaller symbol but larger alignment.
+; RUN: llvm-as smaller-sym.ll -o smaller-sym.bc
+; RUN: llc -filetype=obj larger-sym.ll -o larger-sym.o
+; RUN: ld.lld smaller-sym.bc larger-sym.o -o 1.so -shared
+; RUN: llvm-readobj -S --symbols 1.so | FileCheck %s -DALIGN=8
+
+;; Case 2: bitcode file has larger symbol but smaller alignment.
+; RUN: llvm-as larger-sym.ll -o larger-sym.bc
+; RUN: llc -filetype=obj smaller-sym.ll -o smaller-sym.o
+; RUN: ld.lld smaller-sym.o larger-sym.bc -o 2.so -shared
+;; FIXME: This alignment should be 8, but LLD is ignoring the alignment of a
+;; symbol in a native object file when the larger symbol is in a bitcode file.
+;; See https://bugs.llvm.org/show_bug.cgi?id=47819.
+; RUN: llvm-readobj -S --symbols 2.so | FileCheck %s -DALIGN=4
 
 ; CHECK:      Name: .bss
 ; CHECK-NEXT: Type: SHT_NOBITS
 ; CHECK-NEXT: Flags [
 ; CHECK-NEXT:   SHF_ALLOC
 ; CHECK-NEXT:   SHF_WRITE
 ; CHECK-NEXT: ]
 ; CHECK-NEXT: Address:
 ; CHECK-NEXT: Offset:
-; CHECK-NEXT: Size: 8
+; CHECK-NEXT: Size: 2
 ; CHECK-NEXT: Link: 0
 ; CHECK-NEXT: Info: 0
-; CHECK-NEXT: AddressAlignment: 8
+; CHECK-NEXT: AddressAlignment: [[ALIGN]]
 
 ; CHECK:      Name: a
 ; CHECK-NEXT: Value:
-; CHECK-NEXT: Size: 8
+; CHECK-NEXT: Size: 2
 ; CHECK-NEXT: Binding: Global
 ; CHECK-NEXT: Type: Object
 ; CHECK-NEXT: Other: 0
 ; CHECK-NEXT: Section: .bss
 
+;--- smaller-sym.ll
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+@a = common global i8 0, align 8
+
+;--- larger-sym.ll
 target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
-@a = common global i32 0, align 8
+@a = common global i16 0, align 4

lld/test/ELF/lto/non-ascii-filenames.ll

+; REQUIRES: x86
+
+;; Show that both regular LTO and ThinLTO work correctly when an input file uses
+;; a non-ascii filename.
+
+;; Regular LTO.
+; RUN: llvm-as %s -o %t£.o
+; RUN: ld.lld %t£.o -o %t
+; RUN: llvm-readelf -s %t | FileCheck %s
+
+;; Thin LTO.
+; RUN: opt -module-summary %s -o %t-thin£.o
+; RUN: ld.lld %t-thin£.o -o %t-thin
+; RUN: llvm-readelf -s %t-thin | FileCheck %s
+
+; CHECK: _start
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+@_start = global i32 0

lld/test/ELF/lto/resolution.ll

 ; REQUIRES: x86
-; RUN: llvm-as %s -o %t1.o
-; RUN: llvm-mc -triple=x86_64-pc-linux %p/Inputs/resolution.s -o %t2.o -filetype=obj
+;; Show that resolution of weak + global symbols works correctly when one is 
+;; defined in native object and the other in a bitcode file.
+;; The global symbol in both cases should be kept. LTO should throw away the
+;; data for the discarded weak symbol defined in bitcode. The data for the
+;; weak symbol in a native object will be kept, but will be unlabelled.
+
+; RUN: rm -rf %t.dir
+; RUN: split-file %s %t.dir
+; RUN: llvm-as %t.dir/1.ll -o %t1.o
+; RUN: llvm-mc -triple=x86_64-pc-linux %t.dir/2.s -o %t2.o -filetype=obj
 ; RUN: ld.lld %t1.o %t2.o -o %t.so -shared
-; RUN: llvm-readobj -S --section-data %t.so | FileCheck %s
+; RUN: llvm-readobj --symbols -S --section-data %t.so | FileCheck %s
 
 ; CHECK:      Name: .data
 ; CHECK-NEXT: Type: SHT_PROGBITS
 ; CHECK-NEXT: Flags [
 ; CHECK-NEXT:   SHF_ALLOC
 ; CHECK-NEXT:   SHF_WRITE
 ; CHECK-NEXT: ]
-; CHECK-NEXT: Address:
+; CHECK-NEXT: Address: 0x[[#%x,ADDR:]]
 ; CHECK-NEXT: Offset:
-; CHECK-NEXT: Size: 4
-; CHECK-NEXT: Link: 0
-; CHECK-NEXT: Info: 0
-; CHECK-NEXT: AddressAlignment: 1
-; CHECK-NEXT: EntrySize: 0
+; CHECK-NEXT: Size: 12
+; CHECK-NEXT: Link: 
+; CHECK-NEXT: Info:
+; CHECK-NEXT: AddressAlignment:
+; CHECK-NEXT: EntrySize:
 ; CHECK-NEXT: SectionData (
-; CHECK-NEXT:   0000: 09000000 |{{.*}}|
+; CHECK-NEXT:   0000: 09000000 05000000 04000000 |{{.*}}|
 ; CHECK-NEXT: )
 
+; CHECK:      Name: a{{ }}
+; CHECK-NEXT: Value: 0x[[#%x,ADDR]]
+
+; CHECK:      Name: b{{ }}
+; CHECK-NEXT: Value: 0x[[#%x,ADDR+8]]
+
+;--- 1.ll
 target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 @a = weak global i32 8
+@b = global i32 4
+
+;--- 2.s
+.data
+.global a
+a:
+.long 9
+
+.weak b
+b:
+.long 5