[OpenMP] Use new offloading binary when embedding offloading images

[jhuber6]

Summary

The previous patch introduced the offloading binary format so we can
store some metada along with the binary image. This patch introduces
using this inside the linker wrapper and Clang instead of the previous
method that embedded the metadata in the section name.

[jhuber6]

Fixing unrelated formatting changes.

[jhuber6]

Cleanup.

[jhuber6]

Fix test after move to opaque pointers.

[jdoerfert]

clang/test/Frontend/embed-object.ll

6

Can we salvage the test intention and embed two objects?

clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

314

This should be a proper conditional and error check.

319

Why is the cast at the call site not in the callee? Are these casts even necessary at all?

455

Isn't this 99% the same as above, from offset = 0 to the end?

[jhuber6]

Addressing comments.

[jdoerfert]

LG

[mgorny]

I'm sorry for noticing it this late but this change seems to have broken the test on 32-bit x86:

FAIL: Clang :: Frontend/embed-object.ll (8280 of 15613)
******************** TEST 'Clang :: Frontend/embed-object.ll' FAILED ********************
Script:
--
: 'RUN: at line 1';   /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/x/y/clang-abi_x86_32.x86/bin/clang -cc1 -internal-isystem /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/x/y/clang-abi_x86_32.x86/bin/../../../../lib/clang/15.0.0/include -nostdsysteminc -triple x86_64-unknown-linux-gnu -emit-llvm     -fembed-offload-object=/var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/Inputs/empty.h     -fembed-offload-object=/var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/Inputs/empty.h -x ir /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/embed-object.ll -o -     | /usr/lib/llvm/15/bin/FileCheck /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/embed-object.ll -check-prefix=CHECK
--
Exit Code: 1

Command Output (stderr):
--
+ : 'RUN: at line 1'
+ /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/x/y/clang-abi_x86_32.x86/bin/clang -cc1 -internal-isystem /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/x/y/clang-abi_x86_32.x86/bin/../../../../lib/clang/15.0.0/include -nostdsysteminc -triple x86_64-unknown-linux-gnu -emit-llvm -fembed-offload-object=/var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/Inputs/empty.h -fembed-offload-object=/var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/Inputs/empty.h -x ir /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/embed-object.ll -o -
+ /usr/lib/llvm/15/bin/FileCheck /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/embed-object.ll -check-prefix=CHECK
warning: overriding the module target triple with x86_64-unknown-linux-gnu [-Woverride-module]
1 warning generated.
/var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/embed-object.ll:6:10: error: CHECK: expected string not found in input
; CHECK: @[[OBJECT_1:.+]] = private constant [0 x i8] zeroinitializer, section ".llvm.offloading", align 8, !exclude
         ^
<stdin>:1:1: note: scanning from here
; ModuleID = '/var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/embed-object.ll'
^
<stdin>:8:20: note: possible intended match here
@llvm.embedded.object.1 = private constant [0 x i8] zeroinitializer, section ".llvm.offloading", align 4, !exclude !0
                   ^

Input file: <stdin>
Check file: /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/embed-object.ll

-dump-input=help explains the following input dump.

Input was:
<<<<<<
           1: ; ModuleID = '/var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/embed-object.ll' 
check:6'0     X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
           2: source_filename = "/var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/embed-object.ll" 
check:6'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           3: target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128" 
check:6'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           4: target triple = "x86_64-unknown-linux-gnu" 
check:6'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           5:  
check:6'0     ~
           6: @x = private constant i8 1 
check:6'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~
           7: @llvm.embedded.object = private constant [0 x i8] zeroinitializer, section ".llvm.offloading", align 4, !exclude !0 
check:6'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           8: @llvm.embedded.object.1 = private constant [0 x i8] zeroinitializer, section ".llvm.offloading", align 4, !exclude !0 
check:6'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
check:6'1                        ?                                                                                                   possible intended match
           9: @llvm.compiler.used = appending global [3 x ptr] [ptr @x, ptr @llvm.embedded.object, ptr @llvm.embedded.object.1], section "llvm.metadata" 
check:6'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          10:  
check:6'0     ~
          11: define i32 @foo() { 
check:6'0     ~~~~~~~~~~~~~~~~~~~~
          12:  ret i32 0 
check:6'0     ~~~~~~~~~~~
          13: } 
check:6'0     ~~
           .
           .
           .
>>>>>>

--

********************
FAIL: Clang :: Frontend/embed-object.c (8281 of 15613)
******************** TEST 'Clang :: Frontend/embed-object.c' FAILED ********************
Script:
--
: 'RUN: at line 1';   /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/x/y/clang-abi_x86_32.x86/bin/clang -cc1 -internal-isystem /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/x/y/clang-abi_x86_32.x86/bin/../../../../lib/clang/15.0.0/include -nostdsysteminc -x c -triple x86_64-unknown-linux-gnu -emit-llvm -fembed-offload-object=/var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/Inputs/empty.h -o - /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/embed-object.c | /usr/lib/llvm/15/bin/FileCheck /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/embed-object.c
--
Exit Code: 1

Command Output (stderr):
--
+ : 'RUN: at line 1'
+ /usr/lib/llvm/15/bin/FileCheck /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/embed-object.c
+ /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/x/y/clang-abi_x86_32.x86/bin/clang -cc1 -internal-isystem /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/x/y/clang-abi_x86_32.x86/bin/../../../../lib/clang/15.0.0/include -nostdsysteminc -x c -triple x86_64-unknown-linux-gnu -emit-llvm -fembed-offload-object=/var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/Inputs/empty.h -o - /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/embed-object.c
/var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/embed-object.c:3:11: error: CHECK: expected string not found in input
// CHECK: @[[OBJECT:.+]] = private constant [0 x i8] zeroinitializer, section ".llvm.offloading", align 8, !exclude
          ^
<stdin>:1:1: note: scanning from here
; ModuleID = '/var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/embed-object.c'
^
<stdin>:6:13: note: possible intended match here
@llvm.embedded.object = private constant [0 x i8] zeroinitializer, section ".llvm.offloading", align 4, !exclude !0
            ^

Input file: <stdin>
Check file: /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/embed-object.c

-dump-input=help explains the following input dump.

Input was:
<<<<<<
           1: ; ModuleID = '/var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/embed-object.c' 
check:3'0     X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
           2: source_filename = "/var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang/test/Frontend/embed-object.c" 
check:3'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           3: target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128" 
check:3'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           4: target triple = "x86_64-unknown-linux-gnu" 
check:3'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           5:  
check:3'0     ~
           6: @llvm.embedded.object = private constant [0 x i8] zeroinitializer, section ".llvm.offloading", align 4, !exclude !0 
check:3'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
check:3'1                 ?                                                                                                        possible intended match
           7: @llvm.compiler.used = appending global [1 x ptr] [ptr @llvm.embedded.object], section "llvm.metadata" 
check:3'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           8:  
check:3'0     ~
           9: ; Function Attrs: noinline nounwind optnone 
check:3'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          10: define dso_local void @foo() #0 { 
check:3'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          11: entry: 
check:3'0     ~~~~~~~
           .
           .
           .
>>>>>>

--

********************

[jhuber6]

In D122683#3668412, @mgorny wrote:

I'm sorry for noticing it this late but this change seems to have broken the test on 32-bit x86:

Seems to be a difference in the alignment, I'm not sure why this changes though because the test should specify x86_64.

[jhuber6]

In D122683#3668423, @jhuber6 wrote:

In D122683#3668412, @mgorny wrote:

I'm sorry for noticing it this late but this change seems to have broken the test on 32-bit x86:

Seems to be a difference in the alignment, I'm not sure why this changes though because the test should specify x86_64.

Actually I think the alignment comes from alignof on a struct which doesn't care about the target, just the machine compiling it. Even still that struct has int64_t members so I figured it would be 8-byte aligned. I should probably just hard-code it to 8.

clang/include/clang/Basic/CodeGenOptions.h

public:
 
   /// Prefix to use for -save-temps output.
   std::string SaveTempsFilePrefix;
 
   /// Name of file passed with -fcuda-include-gpubinary option to forward to
   /// CUDA runtime back-end for incorporating them into host-side object file.
   std::string CudaGpuBinaryFileName;
 
-  /// List of filenames and section name pairs passed in using the
-  /// -fembed-offload-object option to embed device-side offloading objects into
-  /// the host as a named section. Input passed in as '<filename>,<section>'
+  /// List of filenames and metadata passed in using the -fembed-offload-object
+  /// option to embed device-side offloading objects into the host as a named
+  /// section. Input passed in as 'filename,kind,triple,arch'.
+  ///
+  /// NOTE: This will need to be expanded whenever we want to pass in more
+  ///       metadata, at some point this should be its own clang tool.
   std::vector<std::string> OffloadObjects;
 
   /// The name of the file to which the backend should save YAML optimization
   /// records.
   std::string OptRecordFile;
 
   /// The regex that filters the passes that should be saved to the optimization
   /// records.

clang/lib/CodeGen/BackendUtil.cpp

 #include "llvm/IR/Module.h"
 #include "llvm/IR/ModuleSummaryIndex.h"
 #include "llvm/IR/PassManager.h"
 #include "llvm/IR/Verifier.h"
 #include "llvm/LTO/LTOBackend.h"
 #include "llvm/MC/MCAsmInfo.h"
 #include "llvm/MC/SubtargetFeature.h"
 #include "llvm/MC/TargetRegistry.h"
+#include "llvm/Object/OffloadBinary.h"
 #include "llvm/Passes/PassBuilder.h"
 #include "llvm/Passes/PassPlugin.h"
 #include "llvm/Passes/StandardInstrumentations.h"
 #include "llvm/Support/BuryPointer.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/PrettyStackTrace.h"
 #include "llvm/Support/TimeProfiler.h"

 }
 
 void clang::EmbedObject(llvm::Module *M, const CodeGenOptions &CGOpts,
                         DiagnosticsEngine &Diags) {
   if (CGOpts.OffloadObjects.empty())
     return;
 
   for (StringRef OffloadObject : CGOpts.OffloadObjects) {
-    if (OffloadObject.count(',') != 1)
-      Diags.Report(Diags.getCustomDiagID(
-          DiagnosticsEngine::Error, "Invalid string pair for embedding '%0'"))
-          << OffloadObject;
-    auto FilenameAndSection = OffloadObject.split(',');
+        SmallVector<StringRef, 4> ObjectFields;
+    OffloadObject.split(ObjectFields, ',');
+
+    if (ObjectFields.size() != 4) {
+      auto DiagID = Diags.getCustomDiagID(
+          DiagnosticsEngine::Error, "Expected at least four arguments '%0'");
+      Diags.Report(DiagID) << OffloadObject;
+      return;
+    }
+
     llvm::ErrorOr<std::unique_ptr<llvm::MemoryBuffer>> ObjectOrErr =
-        llvm::MemoryBuffer::getFileOrSTDIN(FilenameAndSection.first);
+        llvm::MemoryBuffer::getFileOrSTDIN(ObjectFields[0]);
     if (std::error_code EC = ObjectOrErr.getError()) {
       auto DiagID = Diags.getCustomDiagID(DiagnosticsEngine::Error,
                                           "could not open '%0' for embedding");
-      Diags.Report(DiagID) << FilenameAndSection.first;
+      Diags.Report(DiagID) << ObjectFields[0];
       return;
     }
 
-    SmallString<128> SectionName(
-        {".llvm.offloading.", FilenameAndSection.second});
-    llvm::embedBufferInModule(*M, **ObjectOrErr, SectionName);
+    OffloadBinary::OffloadingImage Image{};
+    Image.TheImageKind = getImageKind(ObjectFields[0].rsplit(".").second);
+    Image.TheOffloadKind = getOffloadKind(ObjectFields[1]);
+    Image.StringData = {{"triple", ObjectFields[2]}, {"arch", ObjectFields[3]}};
+    Image.Image = **ObjectOrErr;
+
+    std::unique_ptr<MemoryBuffer> OffloadBuffer = OffloadBinary::write(Image);
+    llvm::embedBufferInModule(*M, *OffloadBuffer, ".llvm.offloading",
+                              Align(OffloadBinary::getAlignment()));
   }
 }

clang/lib/Driver/ToolChains/Clang.cpp

if (IsOpenMPHost && !OpenMPHostInputs.empty()) {
     auto InputFile = OpenMPHostInputs.begin();
     auto OpenMPTCs = C.getOffloadToolChains<Action::OFK_OpenMP>();
     for (auto TI = OpenMPTCs.first, TE = OpenMPTCs.second; TI != TE;
          ++TI, ++InputFile) {
       const ToolChain *TC = TI->second;
       const ArgList &TCArgs = C.getArgsForToolChain(TC, "", Action::OFK_OpenMP);
       StringRef File =
           C.getArgs().MakeArgString(TC->getInputFilename(*InputFile));
-      StringRef InputName = Clang::getBaseInputStem(Args, Inputs);
 
-      CmdArgs.push_back(Args.MakeArgString(
-          "-fembed-offload-object=" + File + "," +
-          Action::GetOffloadKindName(Action::OFK_OpenMP) + "." +
-          TC->getTripleString() + "." +
-          TCArgs.getLastArgValue(options::OPT_march_EQ) + "." + InputName));
+      CmdArgs.push_back(
+          Args.MakeArgString("-fembed-offload-object=" + File + "," +
+                             Action::GetOffloadKindName(Action::OFK_OpenMP) +
+                             "," + TC->getTripleString() + "," +
+                             TCArgs.getLastArgValue(options::OPT_march_EQ)));
     }
   }
 
   if (Triple.isAMDGPU()) {
     handleAMDGPUCodeObjectVersionOptions(D, Args, CmdArgs);
 
     if (Args.hasFlag(options::OPT_munsafe_fp_atomics,
                      options::OPT_mno_unsafe_fp_atomics, /*Default=*/false))

clang/test/Driver/openmp-offload-gpu.c

 // NEW_DRIVER: "nvptx64-nvidia-cuda" - "clang", inputs: ["[[DEVICE_INPUT:.+]]", "[[HOST_BC]]"], output: "[[DEVICE_ASM:.+]]"
 // NEW_DRIVER: "nvptx64-nvidia-cuda" - "NVPTX::Assembler", inputs: ["[[DEVICE_ASM]]"], output: "[[DEVICE_OBJ:.+]]" 
 
 // RUN:   %clang -### -fopenmp=libomp -fopenmp-targets=nvptx64-nvidia-cuda -Xopenmp-target=nvptx64-nvida-cuda -march=sm_70 \
 // RUN:          --libomptarget-nvptx-bc-path=%S/Inputs/libomptarget/libomptarget-new-nvptx-test.bc \
 // RUN:          -fopenmp-new-driver -no-canonical-prefixes -nogpulib %s -o openmp-offload-gpu 2>&1 \
 // RUN:   | FileCheck -check-prefix=NEW_DRIVER_EMBEDDING %s
 
-// NEW_DRIVER_EMBEDDING: -fembed-offload-object=[[CUBIN:.*\.cubin]],openmp.nvptx64-nvidia-cuda.sm_70
+// NEW_DRIVER_EMBEDDING: -fembed-offload-object=[[CUBIN:.*\.cubin]],openmp,nvptx64-nvidia-cuda,sm_70

clang/test/Frontend/embed-object.c

-// RUN: %clang_cc1 -x c -triple x86_64-unknown-linux-gnu -emit-llvm -fembed-offload-object=%S/Inputs/empty.h,section
+// RUN: %clang_cc1 -x c -triple x86_64-unknown-linux-gnu -emit-llvm -fembed-offload-object=%S/Inputs/empty.h,,, -o - | FileCheck %s
 
-// CHECK: @[[OBJECT:.+]] = private constant [0 x i8] zeroinitializer, section ".llvm.offloading.section"
-// CHECK: @llvm.compiler.used = appending global [3 x i8*] [i8* getelementptr inbounds ([0 x i8], [0 x i8]* @[[OBJECT1]]], section "llvm.metadata"
+// CHECK: @[[OBJECT:.+]] = private constant [120 x i8] c"\10\FF\10\AD\01{{.*}}\00\00", section ".llvm.offloading", align 8
+// CHECK: @llvm.compiler.used = appending global [1 x i8*] [i8* getelementptr inbounds ([120 x i8], [120 x i8]* @[[OBJECT]], i32 0, i32 0)], section "llvm.metadata"
 
 void foo(void) {}

clang/test/Frontend/embed-object.ll

 ; RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -emit-llvm \
-; RUN:    -fembed-offload-object=%S/Inputs/empty.h,section1 \
-; RUN:    -fembed-offload-object=%S/Inputs/empty.h,section2 -x ir %s -o - \
+; RUN:    -fembed-offload-object=%S/Inputs/empty.h,,, -x ir %s -o - \
 ; RUN:    | FileCheck %s -check-prefix=CHECK
 
-; CHECK: @[[OBJECT1:.+]] = hidden constant [0 x i8] zeroinitializer, section ".llvm.offloading.section1"
-; CHECK: @[[OBJECT2:.+]] = hidden constant [0 x i8] zeroinitializer, section ".llvm.offloading.section2"
-; CHECK: @llvm.compiler.used = appending global [3 x i8*] [i8* @x, i8* getelementptr inbounds ([0 x i8], [0 x i8]* @[[OBJECT1]], i32 0, i32 0), i8* getelementptr inbounds ([0 x i8], [0 x i8]* @[[OBJECT2]], i32 0, i32 0)], section "llvm.metadata"
+; CHECK: @[[OBJECT:.+]] = private constant [120 x i8] c"\10\FF\10\AD{{.*}}\00", section ".llvm.offloading", align 8
+; CHECK: @llvm.compiler.used = appending global [2 x i8*] [i8* @x, i8* getelementptr inbounds ([120 x i8], [120 x i8]* @[[OBJECT]], i32 0, i32 0)], section "llvm.metadata"
 
 @x = private constant i8 1
 @llvm.compiler.used = appending global [1 x i8*] [i8* @x], section "llvm.metadata"
 
 define i32 @foo() {
   ret i32 0
 }

clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

 // linked device images for offloading. It scans the linker's input for embedded
 // device offloading data stored in sections `.llvm.offloading.<triple>.<arch>`
 // and extracts it as a temporary file. The extracted device files will then be
 // passed to a device linking job to create a final device image.
 //
 //===---------------------------------------------------------------------===//
 
 #include "OffloadWrapper.h"
+
 #include "clang/Basic/Version.h"
 #include "llvm/BinaryFormat/Magic.h"
 #include "llvm/Bitcode/BitcodeWriter.h"
 #include "llvm/CodeGen/CommandFlags.h"
 #include "llvm/IR/Constants.h"
 #include "llvm/IR/DiagnosticPrinter.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IRReader/IRReader.h"
 #include "llvm/LTO/LTO.h"
 #include "llvm/MC/TargetRegistry.h"
 #include "llvm/Object/Archive.h"
 #include "llvm/Object/ArchiveWriter.h"
 #include "llvm/Object/Binary.h"
 #include "llvm/Object/ObjectFile.h"
+#include "llvm/Object/OffloadBinary.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Errc.h"
 #include "llvm/Support/FileOutputBuffer.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Host.h"
 #include "llvm/Support/InitLLVM.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/Program.h"
 #include "llvm/Support/Signals.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/StringSaver.h"
 #include "llvm/Support/TargetSelect.h"
 #include "llvm/Support/WithColor.h"
 #include "llvm/Support/raw_ostream.h"
 #include "llvm/Target/TargetMachine.h"
 
 using namespace llvm;
+using namespace clang;
 using namespace llvm::object;
 
 static cl::opt<bool> Help("h", cl::desc("Alias for -help"), cl::Hidden);
 
 enum DebugKind {
   NoDebugInfo,
   DirectivesOnly,
   FullDebugInfo,

 
 /// Temporary files created by the linker wrapper.
 static SmallVector<std::string, 16> TempFiles;
 
 /// Codegen flags for LTO backend.
 static codegen::RegisterCodeGenFlags CodeGenFlags;
 
 /// Magic section string that marks the existence of offloading data. The
-/// section string will be formatted as `.llvm.offloading.<triple>.<arch>`.
-#define OFFLOAD_SECTION_MAGIC_STR ".llvm.offloading."
+/// section will contain one or more offloading binaries stored contiguously.
+#define OFFLOAD_SECTION_MAGIC_STR ".llvm.offloading"
 
+// TODO: Replace all uses of DeviceFile with OffloadFile.
 /// Information for a device offloading file extracted from the host.
 struct DeviceFile {
   DeviceFile(StringRef Kind, StringRef TheTriple, StringRef Arch,
              StringRef Filename)
       : Kind(Kind), TheTriple(TheTriple), Arch(Arch), Filename(Filename) {}
 
   std::string Kind;
   std::string TheTriple;
void printCommands(ArrayRef<StringRef> CmdArgs) {
   if (CmdArgs.empty())
     return;
 
   llvm::errs() << " \"" << CmdArgs.front() << "\" ";
   for (auto IC = std::next(CmdArgs.begin()), IE = CmdArgs.end(); IC != IE; ++IC)
     llvm::errs() << *IC << (std::next(IC) != IE ? " " : "\n");
 }
 
-static StringRef getDeviceFileExtension(StringRef DeviceTriple,
-                                        bool IsBitcode = false) {
-  Triple TheTriple(DeviceTriple);
-  if (TheTriple.isAMDGPU() || IsBitcode)
-    return "bc";
-  if (TheTriple.isNVPTX())
-    return "cubin";
-  return "o";
-}
-
 std::string getMainExecutable(const char *Name) {
   void *Ptr = (void *)(intptr_t)&getMainExecutable;
   auto COWPath = sys::fs::getMainExecutable(Name, Ptr);
   return sys::path::parent_path(COWPath).str();
 }
 
 /// Extract the device file from the string '<kind>-<triple>-<arch>=<library>'.
 DeviceFile getBitcodeLibrary(StringRef LibraryStr) {

 
 Expected<Optional<std::string>>
 extractFromBinary(const ObjectFile &Obj,
                   SmallVectorImpl<DeviceFile> &DeviceFiles) {
   StringRef Extension = sys::path::extension(Obj.getFileName()).drop_front();
   StringRef Prefix = sys::path::stem(Obj.getFileName());
   SmallVector<StringRef, 4> ToBeStripped;
 
-  // Extract data from sections of the form `.llvm.offloading.<triple>.<arch>`.
+  // Extract offloading binaries from sections with the name `.llvm.offloading`.
   for (const SectionRef &Sec : Obj.sections()) {
     Expected<StringRef> Name = Sec.getName();
-    if (!Name || !Name->startswith(OFFLOAD_SECTION_MAGIC_STR))
+    if (!Name || !Name->equals(OFFLOAD_SECTION_MAGIC_STR))
       continue;
 
-    SmallVector<StringRef, 4> SectionFields;
-    Name->split(SectionFields, '.');
-    StringRef Kind = SectionFields[3];
-    StringRef DeviceTriple = SectionFields[4];
-    StringRef Arch = SectionFields[5];
+    Expected<StringRef> Contents = Sec.getContents();
+    if (!Contents)
+      return Contents.takeError();
+
+    uint64_t Offset = 0;
+    // There could be multiple offloading binaries stored at this section.
+    while (Offset < Contents->size()) {
+      std::unique_ptr<MemoryBuffer> Buffer =
+          MemoryBuffer::getMemBuffer(Contents->drop_front(Offset), *Name,
+                                     /*RequiresNullTerminator*/ false);
+      auto BinaryOrErr = OffloadBinary::create(*Buffer);
+      if (!BinaryOrErr)
+        return BinaryOrErr.takeError();
+      OffloadBinary &Binary = **BinaryOrErr;
+
+      assert(Binary.getVersion() == 1 && "Incompatible device image");
+
+      StringRef Kind =
+          getOffloadKindName(static_cast<OffloadKind>(Binary.getOffloadKind()));
+      StringRef Suffix =
+          getImageKindName(static_cast<ImageKind>(Binary.getImageKind()));
 
-    if (Expected<StringRef> Contents = Sec.getContents()) {
       SmallString<128> TempFile;
-      StringRef DeviceExtension = getDeviceFileExtension(
-          DeviceTriple, identify_magic(*Contents) == file_magic::bitcode);
-      if (Error Err = createOutputFile(Prefix + "-" + Kind + "-" +
-                                           DeviceTriple + "-" + Arch,
-                                       DeviceExtension, TempFile))
+      if (Error Err =
+              createOutputFile(Prefix + "-" + Kind + "-" + Binary.getTriple() +
+                                   "-" + Binary.getArch(),
+                               Suffix, TempFile))
         return std::move(Err);
 
       Expected<std::unique_ptr<FileOutputBuffer>> OutputOrErr =
-          FileOutputBuffer::create(TempFile, Sec.getSize());
+          FileOutputBuffer::create(TempFile, Binary.getImage().size());
       if (!OutputOrErr)
         return OutputOrErr.takeError();
       std::unique_ptr<FileOutputBuffer> Output = std::move(*OutputOrErr);
-      std::copy(Contents->begin(), Contents->end(), Output->getBufferStart());
+      std::copy(Binary.getImage().bytes_begin(), Binary.getImage().bytes_end(),
+                Output->getBufferStart());
       if (Error E = Output->commit())
         return std::move(E);
 
-      DeviceFiles.emplace_back(Kind, DeviceTriple, Arch, TempFile);
-      ToBeStripped.push_back(*Name);
+      DeviceFiles.emplace_back(Kind, Binary.getTriple(), Binary.getArch(),
+                               TempFile);
+
+      Offset += Binary.getSize();
     }
+
+    ToBeStripped.push_back(*Name);
   }
 
   if (ToBeStripped.empty() || !StripSections)
     return None;
 
   // If the object file to strip doesn't exist we need to write it so we can
   // pass it to llvm-strip.
   SmallString<128> StripFile = Obj.getFileName();
return createStringError(inconvertibleErrorCode(),
                              "Failed to create module");
 
   StringRef Extension = sys::path::extension(M->getName()).drop_front();
   StringRef Prefix =
       sys::path::stem(M->getName()).take_until([](char C) { return C == '-'; });
 
   SmallVector<GlobalVariable *, 4> ToBeDeleted;
 
-  // Extract data from the global string containing a section of the form
-  // `.llvm.offloading.<triple>.<arch>`.
+  // Extract offloading data from globals with the `.llvm.offloading` section
+  // name.
   for (GlobalVariable &GV : M->globals()) {
-    if (!GV.hasSection() ||
-        !GV.getSection().startswith(OFFLOAD_SECTION_MAGIC_STR))
+    if (!GV.hasSection() || !GV.getSection().equals(OFFLOAD_SECTION_MAGIC_STR))
       continue;
 
     auto *CDS = dyn_cast<ConstantDataSequential>(GV.getInitializer());
     if (!CDS)
       continue;
 
-    SmallVector<StringRef, 4> SectionFields;
-    GV.getSection().split(SectionFields, '.');
-    StringRef Kind = SectionFields[3];
-    StringRef DeviceTriple = SectionFields[4];
-    StringRef Arch = SectionFields[5];
-
     StringRef Contents = CDS->getAsString();
+
+    uint64_t Offset = 0;
+    // There could be multiple offloading binaries stored at this section.
+    while (Offset < Contents.size()) {
+      std::unique_ptr<MemoryBuffer> Buffer =
+          MemoryBuffer::getMemBuffer(Contents.drop_front(Offset), GV.getName(),
+                                     /*RequiresNullTerminator*/ false);
+      auto BinaryOrErr = OffloadBinary::create(*Buffer);
+      if (!BinaryOrErr)
+        return BinaryOrErr.takeError();
+      OffloadBinary &Binary = **BinaryOrErr;
+
+      assert(Binary.getVersion() == 1 && "Incompatible device image");
+
+      StringRef Kind =
+          getOffloadKindName(static_cast<OffloadKind>(Binary.getOffloadKind()));
+      StringRef Suffix =
+          getImageKindName(static_cast<ImageKind>(Binary.getImageKind()));
+
       SmallString<128> TempFile;
-    StringRef DeviceExtension = getDeviceFileExtension(
-        DeviceTriple, identify_magic(Contents) == file_magic::bitcode);
-    if (Error Err = createOutputFile(Prefix + "-" + Kind + "-" + DeviceTriple +
-                                         "-" + Arch,
-                                     DeviceExtension, TempFile))
+      if (Error Err =
+              createOutputFile(Prefix + "-" + Kind + "-" + Binary.getTriple() +
+                                   "-" + Binary.getArch(),
+                               Suffix, TempFile))
         return std::move(Err);
 
       Expected<std::unique_ptr<FileOutputBuffer>> OutputOrErr =
-        FileOutputBuffer::create(TempFile, Contents.size());
+          FileOutputBuffer::create(TempFile, Binary.getImage().size());
       if (!OutputOrErr)
         return OutputOrErr.takeError();
       std::unique_ptr<FileOutputBuffer> Output = std::move(*OutputOrErr);
-    std::copy(Contents.begin(), Contents.end(), Output->getBufferStart());
+      std::copy(Binary.getImage().bytes_begin(), Binary.getImage().bytes_end(),
+                Output->getBufferStart());
       if (Error E = Output->commit())
         return std::move(E);
 
-    DeviceFiles.emplace_back(Kind, DeviceTriple, Arch, TempFile);
+      DeviceFiles.emplace_back(Kind, Binary.getTriple(), Binary.getArch(),
+                               TempFile);
+
+      Offset += Binary.getSize();
+    }
     ToBeDeleted.push_back(&GV);
   }
 
   if (ToBeDeleted.empty() || !StripSections)
     return None;
 
   // We need to materialize the lazy module before we make any changes.
   if (Error Err = M->materializeAll())

llvm/include/llvm/Object/OffloadBinary.h

public:
   /// Attempt to parse the offloading binary stored in \p Data.
   static Expected<std::unique_ptr<OffloadBinary>> create(MemoryBufferRef);
 
   /// Serialize the contents of \p File to a binary buffer to be read later.
   static std::unique_ptr<MemoryBuffer> write(const OffloadingImage &);
 
   static uint64_t getAlignment() { return alignof(Header); }
 
+  uint32_t getVersion() const { return TheHeader->Version; }
   uint16_t getImageKind() const { return TheEntry->ImageKind; }
   uint16_t getOffloadKind() const { return TheEntry->OffloadKind; }
   uint32_t getFlags() const { return TheEntry->Flags; }
   uint64_t getSize() const { return TheHeader->Size; }
 
   StringRef getTriple() const { return getString("triple"); }
   StringRef getArch() const { return getString("arch"); }
   StringRef getImage() const {

llvm/include/llvm/Transforms/Utils/ModuleUtils.h

 //===-- ModuleUtils.h - Functions to manipulate Modules ---------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This family of functions perform manipulations on Modules.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_TRANSFORMS_UTILS_MODULEUTILS_H
 #define LLVM_TRANSFORMS_UTILS_MODULEUTILS_H
 
 #include "llvm/ADT/StringRef.h"
+#include "llvm/Support/Alignment.h"
 #include "llvm/Support/MemoryBufferRef.h"
 #include <utility> // for std::pair
 
 namespace llvm {
 template <typename T> class SmallVectorImpl;
 
 template <typename T> class ArrayRef;
 class Module;

 ///
 /// If the module has no strong external symbols (such a module may still have a
 /// semantic effect if it performs global initialization), we cannot produce a
 /// unique identifier for this module, so we return the empty string.
 std::string getUniqueModuleId(Module *M);
 
 /// Embed the memory buffer \p Buf into the module \p M as a global using the
 /// specified section name.
-void embedBufferInModule(Module &M, MemoryBufferRef Buf, StringRef SectionName);
+void embedBufferInModule(Module &M, MemoryBufferRef Buf, StringRef SectionName,
+                         Align Alignment = Align(1));
 
 class CallInst;
 namespace VFABI {
 /// Overwrite the Vector Function ABI variants attribute with the names provide
 /// in \p VariantMappings.
 void setVectorVariantNames(CallInst *CI, ArrayRef<std::string> VariantMappings);
 } // End VFABI namespace
 } // End llvm namespace
 
 #endif // LLVM_TRANSFORMS_UTILS_MODULEUTILS_H

llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp

static SectionKind getELFKindForNamedSection(StringRef Name, SectionKind K) {
   //
   //   .section   .eh_frame,"a",@progbits
 
   if (Name == getInstrProfSectionName(IPSK_covmap, Triple::ELF,
                                       /*AddSegmentInfo=*/false) ||
       Name == getInstrProfSectionName(IPSK_covfun, Triple::ELF,
                                       /*AddSegmentInfo=*/false) ||
       Name == ".llvmbc" || Name == ".llvmcmd" ||
-      Name.startswith(".llvm.offloading."))
+      Name.startswith(".llvm.offloading"))
     return SectionKind::getMetadata();
 
   if (Name.empty() || Name[0] != '.') return K;
 
   // Default implementation based on some magic section names.
   if (Name == ".bss" ||
       Name.startswith(".bss.") ||
       Name.startswith(".gnu.linkonce.b.") ||

llvm/lib/Transforms/Utils/ModuleUtils.cpp

assert(M->getNamedValue(VI.getValue().VectorName) &&
            "vector function declaration is missing.");
   }
 #endif
   CI->addFnAttr(
       Attribute::get(M->getContext(), MappingsAttrName, Buffer.str()));
 }
 
 void llvm::embedBufferInModule(Module &M, MemoryBufferRef Buf,
-                               StringRef SectionName) {
-  // Embed the buffer into the module.
+                               StringRef SectionName, Align Alignment) {
+  // Embed the memory buffer into the module.
   Constant *ModuleConstant = ConstantDataArray::get(
       M.getContext(), makeArrayRef(Buf.getBufferStart(), Buf.getBufferSize()));
   GlobalVariable *GV = new GlobalVariable(
-      M, ModuleConstant->getType(), true, GlobalValue::ExternalLinkage,
-      ModuleConstant, SectionName.drop_front());
+      M, ModuleConstant->getType(), true, GlobalValue::PrivateLinkage,
+      ModuleConstant, "llvm.embedded.object");
   GV->setSection(SectionName);
-  GV->setVisibility(GlobalValue::HiddenVisibility);
+  GV->setAlignment(Alignment);
 
   appendToCompilerUsed(M, GV);
 }