[HIP] Fix lld failure when devie object is empty

[yaxunl]

Summary

When -fgpu-rdc is used for linking relocatable objects, clang driver launches
clang-offload-bundler to extract a device relocatable object from each input
relocatable object file and passes the extracted files to lld. The input relocatable
object file could either come from HIP program or C++ program. The relocatable
object file from C++ program does not contain device relocatable objects, therefore
clang-offload-bundler extracts an empty file and passes it to lld, which causes lld to fail.

This patch adds "elf64_amdgpu" to lld so that lld always know the target
no matter input file is empty or not.

[yaxunl]

fix typo

[yaxunl]

need to specify osabi for elf64_amdgpu

[MaskRay]

lld/ELF/Driver.cpp

179

This needs an emulation-amdgpu.s test.

[MaskRay]

Some host relocatable objects may not contain device relocatable objects, where an empty file is passed to lld, which causes lld to fail.

How is an empty file (size=0) passed to lld? If a dummy relocatable object file is parsed to lld, lld can infer the machine type from e_machine in the ELF header.
How does it work in other cases? Because a non-empty relocatable object file is used?

I do not object to a new emulation which does not exist in GNU ld, but the description needs to be clarified.

[yaxunl]

add test emulation-amdgpu.s

[yaxunl]

In D138221#3939384, @MaskRay wrote:

Some host relocatable objects may not contain device relocatable objects, where an empty file is passed to lld, which causes lld to fail.

How is an empty file (size=0) passed to lld? If a dummy relocatable object file is parsed to lld, lld can infer the machine type from e_machine in the ELF header.
How does it work in other cases? Because a non-empty relocatable object file is used?

I do not object to a new emulation which does not exist in GNU ld, but the description needs to be clarified.

Updated description. The empty file is generated by clang-offload-bundler, which does not know how to create a dummy relocatable object file for a device target.

lld/ELF/Driver.cpp

179

added

[MaskRay]

LGTM.

[MaskRay]

In D138221#3941173, @yaxunl wrote:

In D138221#3939384, @MaskRay wrote:

Some host relocatable objects may not contain device relocatable objects, where an empty file is passed to lld, which causes lld to fail.

How is an empty file (size=0) passed to lld? If a dummy relocatable object file is parsed to lld, lld can infer the machine type from e_machine in the ELF header.
How does it work in other cases? Because a non-empty relocatable object file is used?

I do not object to a new emulation which does not exist in GNU ld, but the description needs to be clarified.

Updated description. The empty file is generated by clang-offload-bundler, which does not know how to create a dummy relocatable object file for a device target.

A size=0 file is handled as a linker script, not a relocatable object file. The empty linker script has no-op.

[yaxunl]

In D138221#3942095, @MaskRay wrote:

In D138221#3941173, @yaxunl wrote:

In D138221#3939384, @MaskRay wrote:

Some host relocatable objects may not contain device relocatable objects, where an empty file is passed to lld, which causes lld to fail.

How is an empty file (size=0) passed to lld? If a dummy relocatable object file is parsed to lld, lld can infer the machine type from e_machine in the ELF header.
How does it work in other cases? Because a non-empty relocatable object file is used?

I do not object to a new emulation which does not exist in GNU ld, but the description needs to be clarified.

Updated description. The empty file is generated by clang-offload-bundler, which does not know how to create a dummy relocatable object file for a device target.

A size=0 file is handled as a linker script, not a relocatable object file. The empty linker script has no-op.

I see. We got into a situation where all input files were empty and lld emitted the error:

lld: error: target emulation unknown: -m or at least one .o file required

adding -m fixes that.

clang/lib/Driver/ToolChains/HIPAMD.cpp

 }
 
 void AMDGCN::Linker::constructLldCommand(Compilation &C, const JobAction &JA,
                                          const InputInfoList &Inputs,
                                          const InputInfo &Output,
                                          const llvm::opt::ArgList &Args) const {
   // Construct lld command.
   // The output from ld.lld is an HSA code object file.
-  ArgStringList LldArgs{"-flavor", "gnu", "--no-undefined", "-shared",
+  ArgStringList LldArgs{"-flavor",
+                        "gnu",
+                        "-m",
+                        "elf64_amdgpu",
+                        "--no-undefined",
+                        "-shared",
                         "-plugin-opt=-amdgpu-internalize-symbols"};
 
   auto &TC = getToolChain();
   auto &D = TC.getDriver();
   assert(!Inputs.empty() && "Must have at least one input.");
   bool IsThinLTO = D.getLTOMode(/*IsOffload=*/true) == LTOK_Thin;
   addLTOOptions(TC, Args, LldArgs, Output, Inputs[0], IsThinLTO);
 

clang/test/Driver/hip-toolchain-device-only.hip

 // REQUIRES: amdgpu-registered-target
 
 // RUN: %clang -### --target=x86_64-linux-gnu \
 // RUN:   --offload-arch=gfx803 --offload-arch=gfx900 \
 // RUN:   --cuda-device-only -nogpuinc -nogpulib -c \
 // RUN:   %s 2>&1 | FileCheck %s
 
 // CHECK-NOT: error:
 
 // CHECK: [[CLANG:".*clang.*"]] "-cc1"{{.*}} "-triple" "amdgcn-amd-amdhsa"
 // CHECK-SAME: "-fcuda-is-device"
 // CHECK-SAME: "-target-cpu" "gfx803"
 // CHECK-SAME: {{.*}} "-o" [[OBJ_DEV_A_803:".*o"]] "-x" "hip"
 
-// CHECK: [[LLD: ".*lld.*"]] "-flavor" "gnu" "--no-undefined" "-shared"
+// CHECK: [[LLD: ".*lld.*"]] "-flavor" "gnu" "-m" "elf64_amdgpu" "--no-undefined" "-shared"
 // CHECK-SAME: "-o" "[[IMG_DEV_A_803:.*out]]" [[OBJ_DEV_A_803]]
 
 // CHECK: [[CLANG:".*clang.*"]] "-cc1"{{.*}} "-triple" "amdgcn-amd-amdhsa"
 // CHECK-SAME: "-emit-obj"
 // CHECK-SAME: "-fcuda-is-device"
 // CHECK-SAME: "-target-cpu" "gfx900"
 // CHECK-SAME: {{.*}} "-o" [[OBJ_DEV_A_900:".*o"]] "-x" "hip"
 
-// CHECK: [[LLD]] "-flavor" "gnu" "--no-undefined" "-shared"
+// CHECK: [[LLD]] "-flavor" "gnu" "-m" "elf64_amdgpu" "--no-undefined" "-shared"
 // CHECK-SAME: "-o" "[[IMG_DEV_A_900:.*out]]" [[OBJ_DEV_A_900]]
 
 // CHECK: [[BUNDLER:".*clang-offload-bundler"]] "-type=o"
 // CHECK-SAME: "-targets={{.*}},hip{{.*}}-amdgcn-amd-amdhsa--gfx803,hip{{.*}}-amdgcn-amd-amdhsa--gfx900"
 // CHECK-SAME: "-input={{.*}}" "-input=[[IMG_DEV_A_803]]" "-input=[[IMG_DEV_A_900]]" "-output=[[BUNDLE_A:.*hipfb]]"

clang/test/Driver/hip-toolchain-no-rdc.hip

 // CHECK-SAME: "-target-cpu" "gfx803"
 // CHECK-SAME: {{.*}} "-o" [[OBJ_DEV_A_803:".*o"]] "-x" "hip"
 // CHECK-SAME: {{.*}} [[A_SRC:".*a.cu"]]
 
 // CHECK-NOT: {{".*llvm-link"}}
 // CHECK-NOT: {{".*opt"}}
 // CHECK-NOT: {{".*llc"}}
 
-// CHECK: [[LLD: ".*lld.*"]] "-flavor" "gnu" "--no-undefined" "-shared"
+// CHECK: [[LLD: ".*lld.*"]] "-flavor" "gnu" "-m" "elf64_amdgpu" "--no-undefined" "-shared"
 // CHECK-SAME: "-o" "[[IMG_DEV_A_803:.*out]]" [[OBJ_DEV_A_803]]
 
 //
 // Compile device code in a.cu to code object for gfx900.
 //
 
 // CHECK: [[CLANG:".*clang.*"]] "-cc1" "-triple" "amdgcn-amd-amdhsa"
 // CHECK-SAME: "-aux-triple" "x86_64-unknown-linux-gnu"
 // CHECK-SAME: "-emit-obj"
 // CHECK-SAME: {{.*}} "-main-file-name" "a.cu"
 // CHECK-SAME: "-fcuda-is-device" "-mllvm" "-amdgpu-internalize-symbols"
 // CHECK-SAME: "-fcuda-allow-variadic-functions" "-fvisibility=hidden"
 // CHECK-SAME: "-fapply-global-visibility-to-externs"
 // CHECK-SAME: "{{.*}}lib1.bc" "{{.*}}lib2.bc"
 // CHECK-SAME: "-target-cpu" "gfx900"
 // CHECK-SAME: {{.*}} "-o" [[OBJ_DEV_A_900:".*o"]] "-x" "hip"
 // CHECK-SAME: {{.*}} [[A_SRC]]
 
 // CHECK-NOT: {{".*llvm-link"}}
 // CHECK-NOT: {{".*opt"}}
 // CHECK-NOT: {{".*llc"}}
 
-// CHECK: [[LLD]] "-flavor" "gnu" "--no-undefined" "-shared"
+// CHECK: [[LLD]] "-flavor" "gnu" "-m" "elf64_amdgpu" "--no-undefined" "-shared"
 // CHECK-SAME: "-o" "[[IMG_DEV_A_900:.*out]]" [[OBJ_DEV_A_900]]
 
 //
 // Bundle and embed device code in host object for a.cu.
 //
 
 // CHECK: [[BUNDLER:".*clang-offload-bundler"]] "-type=o"
 // CHECK-SAME: "-bundle-align=4096"

 // CHECK-SAME: "-target-cpu" "gfx803"
 // CHECK-SAME: {{.*}} "-o" [[OBJ_DEV_B_803:".*o"]] "-x" "hip"
 // CHECK-SAME: {{.*}} [[B_SRC:".*b.hip"]]
 
 // CHECK-NOT: {{".*llvm-link"}}
 // CHECK-NOT: {{".*opt"}}
 // CHECK-NOT: {{".*llc"}}
 
-// CHECK: [[LLD]] "-flavor" "gnu" "--no-undefined" "-shared"
+// CHECK: [[LLD]] "-flavor" "gnu" "-m" "elf64_amdgpu" "--no-undefined" "-shared"
 // CHECK-SAME: "-o" "[[IMG_DEV_B_803:.*out]]" [[OBJ_DEV_B_803]]
 
 //
 // Compile device code in b.hip to code object for gfx900.
 //
 
 // CHECK: [[CLANG:".*clang.*"]] "-cc1" "-triple" "amdgcn-amd-amdhsa"
 // CHECK-SAME: "-aux-triple" "x86_64-unknown-linux-gnu"
 // CHECK-SAME: "-emit-obj"
 // CHECK-SAME: {{.*}} "-main-file-name" "b.hip"
 // CHECK-SAME: "-fcuda-is-device" "-mllvm" "-amdgpu-internalize-symbols"
 // CHECK-SAME: "-fcuda-allow-variadic-functions" "-fvisibility=hidden"
 // CHECK-SAME: "-fapply-global-visibility-to-externs"
 // CHECK-SAME: "{{.*}}lib1.bc" "{{.*}}lib2.bc"
 // CHECK-SAME: "-target-cpu" "gfx900"
 // CHECK-SAME: {{.*}} "-o" [[OBJ_DEV_B_900:".*o"]] "-x" "hip"
 // CHECK-SAME: {{.*}} [[B_SRC]]
 
 // CHECK-NOT: {{".*llvm-link"}}
 // CHECK-NOT: {{".*opt"}}
 // CHECK-NOT: {{".*llc"}}
 
-// CHECK: [[LLD]] "-flavor" "gnu" "--no-undefined" "-shared"
+// CHECK: [[LLD]] "-flavor" "gnu" "-m" "elf64_amdgpu" "--no-undefined" "-shared"
 // CHECK-SAME: "-o" "[[IMG_DEV_B_900:.*out]]" [[OBJ_DEV_B_900]]
 
 //
 // Bundle and embed device code in host object for b.hip.
 //
 
 // CHECK: [[BUNDLER:".*clang-offload-bundler"]] "-type=o"
 // CHECK-SAME: "-bundle-align=4096"

lld/ELF/Driver.cpp

std::pair<ELFKind, uint16_t> ret =
           .Case("elf64lriscv", {ELF64LEKind, EM_RISCV})
           .Case("elf64ppc", {ELF64BEKind, EM_PPC64})
           .Case("elf64lppc", {ELF64LEKind, EM_PPC64})
           .Cases("elf_amd64", "elf_x86_64", {ELF64LEKind, EM_X86_64})
           .Case("elf_i386", {ELF32LEKind, EM_386})
           .Case("elf_iamcu", {ELF32LEKind, EM_IAMCU})
           .Case("elf64_sparc", {ELF64BEKind, EM_SPARCV9})
           .Case("msp430elf", {ELF32LEKind, EM_MSP430})
+          .Case("elf64_amdgpu", {ELF64LEKind, EM_AMDGPU})
           .Default({ELFNoneKind, EM_NONE});
 
   if (ret.first == ELFNoneKind)
     error("unknown emulation: " + emul);
   if (ret.second == EM_MSP430)
     osabi = ELFOSABI_STANDALONE;
+  else if (ret.second == EM_AMDGPU)
+    osabi = ELFOSABI_AMDGPU_HSA;
   return std::make_tuple(ret.first, ret.second, osabi);
 }
 
 // Returns slices of MB by parsing MB as an archive file.
 // Each slice consists of a member file in the archive.
 std::vector<std::pair<MemoryBufferRef, uint64_t>> static getArchiveMembers(
     MemoryBufferRef mb) {
   std::unique_ptr<Archive> file =

lld/test/ELF/emulation-amdgpu.s

+# REQUIRES: amdgpu
+
+# RUN: llvm-mc -filetype=obj -triple=amdgcn-amd-amdhsa %s -o %t.o
+# RUN: ld.lld %t.o -o %t
+# RUN: llvm-readobj --file-headers %t | FileCheck %s
+# RUN: ld.lld -m elf64_amdgpu %t.o -o %t
+# RUN: llvm-readobj --file-headers %t | FileCheck %s
+
+# CHECK:      ElfHeader {
+# CHECK-NEXT:   Ident {
+# CHECK-NEXT:     Magic: (7F 45 4C 46)
+# CHECK-NEXT:     Class: 64-bit (0x2)
+# CHECK-NEXT:     DataEncoding: LittleEndian (0x1)
+# CHECK-NEXT:     FileVersion: 1
+# CHECK-NEXT:     OS/ABI: AMDGPU_HSA (0x40)
+# CHECK-NEXT:     ABIVersion: 2
+# CHECK-NEXT:     Unused: (00 00 00 00 00 00 00)
+# CHECK-NEXT:   }
+# CHECK-NEXT:   Type: Executable (0x2)
+# CHECK-NEXT:   Machine: EM_AMDGPU (0xE0)
+# CHECK-NEXT:   Version: 1
+# CHECK-NEXT:   Entry:
+# CHECK-NEXT:   ProgramHeaderOffset: 0x40
+# CHECK-NEXT:   SectionHeaderOffset:
+# CHECK-NEXT:   Flags [ (0x0)
+# CHECK-NEXT:   ]
+# CHECK-NEXT:   HeaderSize: 64
+# CHECK-NEXT:   ProgramHeaderEntrySize: 56
+# CHECK-NEXT:   ProgramHeaderCount:
+# CHECK-NEXT:   SectionHeaderEntrySize: 64
+# CHECK-NEXT:   SectionHeaderCount:
+# CHECK-NEXT:   StringTableSectionIndex:
+# CHECK-NEXT: }
+
+.globl _start
+_start: