[lld][ELF][test] Add additional test coverage for LTO

[jhenderson]

Summary

These are all inspired by existing test coverage we have in an internal testsuite.

[jhenderson]

Add missing new line at EOF.

[grimar]

It feels like common-mixed1.ll and common-mixed2.ll are very similar and can be merged?

lld/test/ELF/lto/bitcode-wrapper.ll

51

I'd comment fields here. They are:
[Magic32, Version32, Offset32, Size32, CPUType32]

lld/test/ELF/lto/common-mixed2.ll

7

Perhaps, use split-file or just echo rather than adding a new tiny input?

[MaskRay]

lld/test/ELF/lto/Inputs/common-mixed2.s

1

This can be inlined into common-mixed2.ll

[jhenderson]

I'll see if I can merge common-mixed1.ll and common-mixed2.ll and inline the input files. That should allow me to avoid renaming common4.ll too.

lld/test/ELF/lto/resolution.ll

15

This test is failing for reasons I've yet to discover. I thought it passed previously, but it's possible it didn't. Currently, it fails because both versions of b are present in the contents of .data, which clearly isn't correct (only one b appears in the symbol table).

[jhenderson]

Reworked resolution.ll and common.ll, fixing the former, and inlining the input files. Also undid the test renaming and made the wrap_bitcode.py script python3 compatible.

[grimar]

You need to update the patch description which still mentions "common-mixed1.ll".
The rest looks fine to me.

lld/test/ELF/lto/bitcode-wrapper.ll

28

aside: we probably want to lowercase this message.

[jhenderson]

In D89775#2346865, @grimar wrote:

You need to update the patch description which still mentions "common-mixed1.ll".
The rest looks fine to me.

Done, thanks.

[grimar]

LGTM. But please wait for @MaskRay.

[jhenderson]

lld/test/ELF/lto/bitcode-wrapper.ll

28

I took a quick look, and this is just one of several error messages within the BitcodeReader and BitcodeAnalyzer files, and most of them use upper-case, so that would probably want to be part of a much wider change. Think we'd need to discuss the preferred style of the library with whoever owns/maintains it before doing that.

[grimar]

lld/test/ELF/lto/bitcode-wrapper.ll

28

Think we'd need to discuss the preferred style of the library with whoever owns/maintains it before doing that.

perhaps we could just lower case all error messages on the lld side.

[jhenderson]

lld/test/ELF/lto/bitcode-wrapper.ll

28

I thought about that, but that would risk us occasionally converting strings to lower-case that shouldn't be (e.g. file paths, names, accronyms like ELF or DWARF etc).

[grimar]

lld/test/ELF/lto/bitcode-wrapper.ll

28

I meant to lower case just the first character. Perhaps it is a low risk here. Though probably
a more safe way is to convert particular error messages. I.e. if it comes from BitcodeReader/BitcodeAnalyzer we could do it in case if we can't just switch to lower case right there.

[MaskRay]

LGTM.

lld/test/ELF/lto/bitcode-wrapper.ll

28

I'd prefer not transforming the error messages. If people search for "Invalid bitcode wrapper header", they can find occurrences in llvm/ and lld/test/ELF. If they happen to change the messages (e.g. to add more information; many BitcodeReader.cpp messages are duplicated in multiple places and are generally not so informational), they can update the lld one as well. Suredly they can use grep -i or rg -i, or not search for the first character, but that is extra hurdle.

We lose a bit of inconsistency, but the loss seems very small to me.

lld/test/ELF/lto/Inputs/common-mixed2.s

+        .comm   a,1,8

lld/test/ELF/lto/Inputs/resolution.s

         .data
         .global a
 a:
         .long 9
+
+        .weak b
+b:
+        .long 5

lld/test/ELF/lto/bitcode-wrapper.ll

+; REQUIRES: x86
+
+;; The LLVM bitcode format allows for an optional wrapper header. This test
+;; shows that LLD can handle bitcode wrapped in this way, and also that an
+;; invalid offset in the wrapper header is handled cleanly.
+
+; RUN: rm -rf %t
+; RUN: split-file %s %t
+; RUN: llvm-as %t/ir.ll -o %t.bc
+
+;; Basic case:
+; RUN: %python %t/wrap_bitcode.py %t.bc %t.o 0 0x14
+; RUN: ld.lld %t.o -o %t.elf
+; RUN: llvm-readelf -s %t.elf | FileCheck %s
+
+;; Padding between wrapper header and body:
+; RUN: %python %t/wrap_bitcode.py %t.bc %t.o 0x10 0x24
+; RUN: ld.lld %t.o -o %t.elf
+; RUN: llvm-readelf -s %t.elf | FileCheck %s
+
+; CHECK: _start
+
+;; Invalid offset past end of file:
+; RUN: %python %t/wrap_bitcode.py %t.bc %t2.o 0x10 0xffffffff
+; RUN: not ld.lld %t2.o -o %t2.elf 2>&1 | FileCheck %s --check-prefix=ERR1 -DFILE=%t2.o
+
+; ERR1: error: [[FILE]]: Invalid bitcode wrapper header
+
+;; Invalid offset within file:
+; RUN: %python %t/wrap_bitcode.py %t.bc %t3.o 0x10 0x14
+; RUN: not ld.lld %t3.o -o %t3.elf 2>&1 | FileCheck %s --check-prefix=ERR2 -DFILE=%t3.o
+
+; ERR2: error: [[FILE]]: file doesn't start with bitcode header
+
+;--- ir.ll
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+@_start = global i32 0
+
+;--- wrap_bitcode.py
+## Arguments are: input file, output file, padding size, offset value.
+import struct
+import sys
+
+with open(sys.argv[1], 'rb') as input:
+    bitcode = input.read()
+
+padding = int(sys.argv[3], 16) * '\0'
+offset = int(sys.argv[4], 16)
+header = struct.pack('<IIIII', 0x0B17C0DE, 0, offset, len(bitcode), 0)
+with open(sys.argv[2], 'wb') as output:
+    output.write(header)
+    output.write(padding)
+    output.write(bitcode)

lld/test/ELF/lto/common-mixed1.ll

+; REQUIRES: x86
+
+;; Show that common symbols defined in both native objects and bitcode files are
+;; properly resolved. In this case, the native object symbol is larger, but has
+;; a smaller alignment.
+; RUN: llvm-as %s -o %t1.o
+; RUN: llvm-mc -triple=x86_64-pc-linux %p/Inputs/common-mixed1.s -o %t2.o -filetype=obj
+; RUN: ld.lld %t1.o %t2.o -o %t.so -shared
+; RUN: llvm-readobj -S --symbols %t.so | FileCheck %s
+
+; CHECK:      Name: .bss
+; CHECK-NEXT: Type: SHT_NOBITS
+; CHECK-NEXT: Flags [
+; CHECK-NEXT:   SHF_ALLOC
+; CHECK-NEXT:   SHF_WRITE
+; CHECK-NEXT: ]
+; CHECK-NEXT: Address:
+; CHECK-NEXT: Offset:
+; CHECK-NEXT: Size: 8
+; CHECK-NEXT: Link: 0
+; CHECK-NEXT: Info: 0
+; CHECK-NEXT: AddressAlignment: 8
+
+; CHECK:      Name: a
+; CHECK-NEXT: Value:
+; CHECK-NEXT: Size: 8
+; CHECK-NEXT: Binding: Global
+; CHECK-NEXT: Type: Object
+; CHECK-NEXT: Other: 0
+; CHECK-NEXT: Section: .bss
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+@a = common global i32 0, align 8

lld/test/ELF/lto/common-mixed2.ll

+; REQUIRES: x86
+
+;; Show that common symbols defined in both native objects and bitcode files are
+;; properly resolved. In this case, the bitcode symbol is larger, but has a
+;; smaller alignment.
+; RUN: llvm-as %s -o %t1.o
+; RUN: llvm-mc -triple=x86_64-pc-linux %p/Inputs/common-mixed2.s -o %t2.o -filetype=obj
+; RUN: ld.lld %t1.o %t2.o -o %t.so -shared --save-temps
+; RUN: llvm-readobj -S --symbols %t.so | FileCheck %s
+
+; CHECK:      Name: .bss
+; CHECK-NEXT: Type: SHT_NOBITS
+; CHECK-NEXT: Flags [
+; CHECK-NEXT:   SHF_ALLOC
+; CHECK-NEXT:   SHF_WRITE
+; CHECK-NEXT: ]
+; CHECK-NEXT: Address:
+; CHECK-NEXT: Offset:
+; CHECK-NEXT: Size: 2
+; CHECK-NEXT: Link: 0
+; CHECK-NEXT: Info: 0
+;; FIXME: This alignment should be 8, but LLD is ignoring the alignment of a
+;; symbol in a native object file when the larger symbol is in a bitcode file.
+;; See https://bugs.llvm.org/show_bug.cgi?id=47819.
+; CHECK-NEXT: AddressAlignment: 4
+
+; CHECK:      Name: a
+; CHECK-NEXT: Value:
+; CHECK-NEXT: Size: 2
+; CHECK-NEXT: Binding: Global
+; CHECK-NEXT: Type: Object
+; CHECK-NEXT: Other: 0
+; CHECK-NEXT: Section: .bss
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+@a = common global i16 0, align 4

lld/test/ELF/lto/common.ll

 ; REQUIRES: x86
-; RUN: llvm-as %s -o %t1.o
-; RUN: llvm-mc -triple=x86_64-pc-linux %p/Inputs/common.s -o %t2.o -filetype=obj
-; RUN: ld.lld %t1.o %t2.o -o %t.so -shared
-; RUN: llvm-readobj -S --symbols %t.so | FileCheck %s
-
-; CHECK:      Name: .bss
-; CHECK-NEXT: Type: SHT_NOBITS
-; CHECK-NEXT: Flags [
-; CHECK-NEXT:   SHF_ALLOC
-; CHECK-NEXT:   SHF_WRITE
-; CHECK-NEXT: ]
-; CHECK-NEXT: Address:
-; CHECK-NEXT: Offset:
-; CHECK-NEXT: Size: 8
-; CHECK-NEXT: Link: 0
-; CHECK-NEXT: Info: 0
-; CHECK-NEXT: AddressAlignment: 8
-
-; CHECK:      Name: a
-; CHECK-NEXT: Value:
-; CHECK-NEXT: Size: 8
-; CHECK-NEXT: Binding: Global
-; CHECK-NEXT: Type: Object
-; CHECK-NEXT: Other: 0
-; CHECK-NEXT: Section: .bss
+
+;; Make sure that common symbols are properly internalized.
+;; In this file, @a does not interpose any symbol in a DSO,
+;; so LTO should be able to internalize it.
+
+; RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux /dev/null -o %t.so.o
+; RUN: ld.lld -shared -o %t.so %t.so.o
+
+; RUN: llvm-as %s -o %t.o
+; RUN: ld.lld -o %t.exe -save-temps %t.o %t.so
+; RUN: llvm-dis < %t.exe.0.2.internalize.bc | FileCheck %s
+
+; RUN: ld.lld -pie -o %t.exe -save-temps %t.o
+; RUN: llvm-dis < %t.exe.0.2.internalize.bc | FileCheck %s
 
 target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
-@a = common global i32 0, align 8
+@a = common dso_local local_unnamed_addr global i32 0, align 4
+; CHECK-DAG: @a = internal global i32 0, align 4

lld/test/ELF/lto/common4.ll

-; REQUIRES: x86
-
-;; Make sure that common symbols are properly internalized.
-;; In this file, @a does not interpose any symbol in a DSO,
-;; so LTO should be able to internalize it.
-
-; RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux /dev/null -o %t.so.o
-; RUN: ld.lld -shared -o %t.so %t.so.o
-
-; RUN: llvm-as %s -o %t.o
-; RUN: ld.lld -o %t.exe -save-temps %t.o %t.so
-; RUN: llvm-dis < %t.exe.0.2.internalize.bc | FileCheck %s
-
-; RUN: ld.lld -pie -o %t.exe -save-temps %t.o
-; RUN: llvm-dis < %t.exe.0.2.internalize.bc | FileCheck %s
-
-target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
-target triple = "x86_64-unknown-linux-gnu"
-
-@a = common dso_local local_unnamed_addr global i32 0, align 4
-; CHECK-DAG: @a = internal global i32 0, align 4

lld/test/ELF/lto/non-ascii-filenames.ll

+; REQUIRES: x86
+
+;; Show that both regular LTO and ThinLTO work correctly when an input file uses
+;; a non-ascii filename.
+
+;; Regular LTO.
+; RUN: llvm-as %s -o %t£.o
+; RUN: ld.lld %t£.o -o %t
+; RUN: llvm-readelf -s %t | FileCheck %s
+
+;; Thin LTO.
+; RUN: opt -module-summary %s -o %t-thin£.o
+; RUN: ld.lld %t-thin£.o -o %t-thin
+; RUN: llvm-readelf -s %t-thin | FileCheck %s
+
+; CHECK: _start
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+@_start = global i32 0

lld/test/ELF/lto/resolution.ll

 ; REQUIRES: x86
 ; RUN: llvm-as %s -o %t1.o
 ; RUN: llvm-mc -triple=x86_64-pc-linux %p/Inputs/resolution.s -o %t2.o -filetype=obj
 ; RUN: ld.lld %t1.o %t2.o -o %t.so -shared
 ; RUN: llvm-readobj -S --section-data %t.so | FileCheck %s
 
 ; CHECK:      Name: .data
 ; CHECK-NEXT: Type: SHT_PROGBITS
 ; CHECK-NEXT: Flags [
 ; CHECK-NEXT:   SHF_ALLOC
 ; CHECK-NEXT:   SHF_WRITE
 ; CHECK-NEXT: ]
 ; CHECK-NEXT: Address:
 ; CHECK-NEXT: Offset:
-; CHECK-NEXT: Size: 4
+; CHECK-NEXT: Size: 8
 ; CHECK-NEXT: Link: 0
 ; CHECK-NEXT: Info: 0
 ; CHECK-NEXT: AddressAlignment: 1
 ; CHECK-NEXT: EntrySize: 0
 ; CHECK-NEXT: SectionData (
-; CHECK-NEXT:   0000: 09000000 |{{.*}}|
+; CHECK-NEXT:   0000: 09000000 04000000 |{{.*}}|
 ; CHECK-NEXT: )
 
 target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 @a = weak global i32 8
+@b = global i32 4