[Clang][OpenMP Offload] Add new tool for wrapping offload device binaries

[sdmitriev]

Summary

This patch removes the remaining part of the OpenMP offload linker scripts which was used for inserting device binaries into the output linked binary. Device binaries are now inserted into the host binary with a help of the wrapper bit-code file which contains device binaries as data. Wrapper bit-code file is dynamically created by the clang driver with a help of new tool clang-offload-wrapper which takes device binaries as input and produces bit-code file with required contents. Wrapper bit-code is then compiled to an object and resulting object is appended to the host linking by the clang driver.

This is the second part of the patch for eliminating OpenMP linker script (please see https://reviews.llvm.org/D64943).

[sdmitriev]

Ping.

[ABataev]

clang/lib/Driver/Driver.cpp

2283–2287

Could you fix these comments is a separate patch?

[JonChesterfield]

This comment was removed by JonChesterfield.

[sdmitriev]

clang/lib/Driver/Driver.cpp

2283–2287

Sure. I will prepare a separate patch for the comment change.

[JonChesterfield]

I think this patch is a behaviour change. Currently, the target binary is embedded in the host binary at link time. With this change, the contents of the binary are embedded in bitcode which is subsequently fed into the link. If indeed so, that seems strictly better - code in the host that cares about the size of the bitcode now has it available at opt time, instead of at link time. The target specific nastiness objcopy would introduce is neatly sidestepped.

This change takes N binaries (that I think need to be for different triples, or the loop doesn't work) and puts them in separate section-annotated bitcode arrays. Equivalent behaviour would result from calling the tool once per binary and passing the N results onward, e.g. to llvm-link.

The functionality of 'take a binary and embed it in bitcode as a const array' is likely to be useful outside of openmp. I've done similar things in the past in non-portable fashion. Aside from the section and symbol names, I don't think there's anything specific to openmp in the tool.

How would you feel about simplifying the tool to work on one file at a time, with an interface that takes the host target (could default to whatever is running the tool) and a string for section name, which generates some bitcode containing that file as a const array plus start/end symbols derived from the section name? The change would involve deleting the multiple file handling and renaming OffloadTargets to SectionName or similar.

clang-offload-wrapper than becomes binary-to-bitcode-embedder (or better, names are hard), with the intent that projects outside of the openmp target offload compiler could use it.

edit: Or keep the multiple file handling if you prefer, preferably raising an error if there are duplicates in the requested section names

clang/tools/clang-offload-wrapper/ClangOffloadWrapper.cpp

85

I don't think this works for multiple binaries with the same target triple. They'll all be put in the same section and there will be duplicate symbols for start/end.

[sdmitriev]

Addressed some comments and rebased patch.

[sdmitriev]

In D68166#1692071, @JonChesterfield wrote:

I think this patch is a behaviour change. Currently, the target binary is embedded in the host binary at link time. With this change, the contents of the binary are embedded in bitcode which is subsequently fed into the link. If indeed so, that seems strictly better - code in the host that cares about the size of the bitcode now has it available at opt time, instead of at link time. The target specific nastiness objcopy would introduce is neatly sidestepped.

This change takes N binaries (that I think need to be for different triples, or the loop doesn't work) and puts them in separate section-annotated bitcode arrays. Equivalent behaviour would result from calling the tool once per binary and passing the N results onward, e.g. to llvm-link.

The functionality of 'take a binary and embed it in bitcode as a const array' is likely to be useful outside of openmp. I've done similar things in the past in non-portable fashion. Aside from the section and symbol names, I don't think there's anything specific to openmp in the tool.

How would you feel about simplifying the tool to work on one file at a time, with an interface that takes the host target (could default to whatever is running the tool) and a string for section name, which generates some bitcode containing that file as a const array plus start/end symbols derived from the section name? The change would involve deleting the multiple file handling and renaming OffloadTargets to SectionName or similar.

clang-offload-wrapper than becomes binary-to-bitcode-embedder (or better, names are hard), with the intent that projects outside of the openmp target offload compiler could use it.

edit: Or keep the multiple file handling if you prefer, preferably raising an error if there are duplicates in the requested section names

The tool indeed does not have anything specific to OpenMP at this step, but that will change in the 3rd part of the D64943 where I am planning to move offload registration code generation from clang to the wrapper tool. So it will have OpenMP specifics in future, though I do not see any problems with enabling it for other offloading models. We can always change driver to pass an additional information that represent 'offload kind' to the wrapper tool (can for example be done in a similar way how it is passed to the bundler tool), and wrapper will customize output bit-code depending on the offloading model if there would be a need for that.

Regarding the multiple vs single file handling. Wrapping each device binary independently would still be possible with multi-file wrapping support, but it will just increase startup time without adding any benefits in return (once we move offload registration code to the wrapper). So, I think that for OpenMP it does not make sense to do it (I cannot say anything about the other offloading models though).

Anyway, I suggest so start with something that eliminates OpenMP linker script. We can always customize/improve tools in future once there would be a need for that.

clang/tools/clang-offload-wrapper/ClangOffloadWrapper.cpp

85

Adding the same target triple to the list of OpenMP targets more than once is not supported, so such use case isn't viable:

bash-4.2$ clang -fopenmp -fopenmp-targets=x86_64-pc-linux-gnu,x86_64-pc-linux-gnu test.c
clang-10: warning: The OpenMP offloading target 'x86_64-pc-linux-gnu' is similar to target 'x86_64-pc-linux-gnu' already specified - will be ignored. [-Wopenmp-target]
bash-4.2$

But in any case I am going to remove the code which passes offload target triples to the wrapper tool in the last part of D64943 because they will not be needed for creating wrapper bit-code. As you know start/end symbols are referenced from the offload registration code only, so, moving offload registration code to the wrapper bit-code eliminates the need to create global start/end symbols with predefined names derived from the triple.

[JonChesterfield]

The tool indeed does not have anything specific to OpenMP at this step, but that will change...

That makes sense to me, thanks.

I think we're going to have some trouble adapting this to our build as there's already a standalone tool that runs at link time. Overall dropping the linker script is probably worth the integration headache.

clang/tools/clang-offload-wrapper/ClangOffloadWrapper.cpp

85

That's true. It seems a shame that we can embed at most one device binary per architecture into the host, but that's an existing limitation.

[sdmitriev]

Rebased patch and changed clang-offload-wrapper CMakeLists.txt to use add_clang_tool() rather than add_clang_executable() with a custom install rule.

[JonChesterfield]

The direction is good and I believe all the feedback from D64943 has already been incorporated. LGTM, thanks.

[thakis]

Out of interest (or ignorance :) ), why is this a separate binary instead of just part of the normal clang driver? C, C++, Objective-C, and assembly all can do with a single driver, yet the offload stuff now has both clang-offload-wrapper and clang-offload-bundler. Why isn't just clang enough?

[sdmitriev]

In D68166#1702487, @thakis wrote:

Out of interest (or ignorance :) ), why is this a separate binary instead of just part of the normal clang driver? C, C++, Objective-C, and assembly all can do with a single driver, yet the offload stuff now has both clang-offload-wrapper and clang-offload-bundler. Why isn't just clang enough?

Well, theoretically both bunder and wrapper functionality can be implemented directly in the clang driver, and so technically these tools can be eliminated. But it is just a matter of splitting functionality into well-defined logical pieces:))

[keryell]

By looking at this, did we forgot about adding some documentation along what we have for https://clang.llvm.org/docs/ClangOffloadBundler.html ?

clang/include/clang/Driver/Action.h

enum ActionClass {
     AssembleJobClass,
     LinkJobClass,
     LipoJobClass,
     DsymutilJobClass,
     VerifyDebugInfoJobClass,
     VerifyPCHJobClass,
     OffloadBundlingJobClass,
     OffloadUnbundlingJobClass,
+    OffloadWrapperJobClass,
 
     JobClassFirst = PreprocessJobClass,
-    JobClassLast = OffloadUnbundlingJobClass
+    JobClassLast = OffloadWrapperJobClass
   };
 
   // The offloading kind determines if this action is binded to a particular
   // programming model. Each entry reserves one bit. We also have a special kind
   // to designate the host offloading tool chain.
   enum OffloadKind {
     OFK_None = 0x00,
 
ArrayRef<DependentActionInfo> getDependentActionsInfo() const {
     return DependentActionInfoArray;
   }
 
   static bool classof(const Action *A) {
     return A->getKind() == OffloadUnbundlingJobClass;
   }
 };
 
+class OffloadWrapperJobAction : public JobAction {
+  void anchor() override;
+
+public:
+  OffloadWrapperJobAction(ActionList &Inputs, types::ID Type);
+
+  static bool classof(const Action *A) {
+    return A->getKind() == OffloadWrapperJobClass;
+  }
+};
+
 } // namespace driver
 } // namespace clang
 
 #endif // LLVM_CLANG_DRIVER_ACTION_H

clang/include/clang/Driver/Options.td

 def fopenmp_version_EQ : Joined<["-"], "fopenmp-version=">, Group<f_Group>, Flags<[CC1Option, NoArgumentUnused]>;
 def fopenmp_EQ : Joined<["-"], "fopenmp=">, Group<f_Group>;
 def fopenmp_use_tls : Flag<["-"], "fopenmp-use-tls">, Group<f_Group>,
   Flags<[NoArgumentUnused, HelpHidden]>;
 def fnoopenmp_use_tls : Flag<["-"], "fnoopenmp-use-tls">, Group<f_Group>,
   Flags<[CC1Option, NoArgumentUnused, HelpHidden]>;
 def fopenmp_targets_EQ : CommaJoined<["-"], "fopenmp-targets=">, Flags<[DriverOption, CC1Option]>,
   HelpText<"Specify comma-separated list of triples OpenMP offloading targets to be supported">;
-def fopenmp_dump_offload_linker_script : Flag<["-"], "fopenmp-dump-offload-linker-script">,
-  Group<f_Group>, Flags<[NoArgumentUnused, HelpHidden]>;
 def fopenmp_relocatable_target : Flag<["-"], "fopenmp-relocatable-target">,
   Group<f_Group>, Flags<[CC1Option, NoArgumentUnused, HelpHidden]>;
 def fnoopenmp_relocatable_target : Flag<["-"], "fnoopenmp-relocatable-target">,
   Group<f_Group>, Flags<[CC1Option, NoArgumentUnused, HelpHidden]>;
 def fopenmp_simd : Flag<["-"], "fopenmp-simd">, Group<f_Group>, Flags<[CC1Option, NoArgumentUnused]>,
   HelpText<"Emit OpenMP code only for SIMD-based constructs.">;
 def fno_openmp_simd : Flag<["-"], "fno-openmp-simd">, Group<f_Group>, Flags<[CC1Option, NoArgumentUnused]>;
 def fopenmp_cuda_mode : Flag<["-"], "fopenmp-cuda-mode">, Group<f_Group>,

clang/include/clang/Driver/ToolChain.h

private:
 
   /// The list of toolchain specific path prefixes to search for programs.
   path_list ProgramPaths;
 
   mutable std::unique_ptr<Tool> Clang;
   mutable std::unique_ptr<Tool> Assemble;
   mutable std::unique_ptr<Tool> Link;
   mutable std::unique_ptr<Tool> OffloadBundler;
+  mutable std::unique_ptr<Tool> OffloadWrapper;
 
   Tool *getClang() const;
   Tool *getAssemble() const;
   Tool *getLink() const;
   Tool *getClangAs() const;
   Tool *getOffloadBundler() const;
+  Tool *getOffloadWrapper() const;
 
   mutable std::unique_ptr<SanitizerArgs> SanitizerArguments;
   mutable std::unique_ptr<XRayArgs> XRayArguments;
 
   /// The effective clang triple for the current Job.
   mutable llvm::Triple EffectiveTriple;
 
   /// Set the toolchain's effective clang triple.

clang/lib/Driver/Action.cpp

const char *Action::getClassName(ActionClass AC) {
   case LipoJobClass: return "lipo";
   case DsymutilJobClass: return "dsymutil";
   case VerifyDebugInfoJobClass: return "verify-debug-info";
   case VerifyPCHJobClass: return "verify-pch";
   case OffloadBundlingJobClass:
     return "clang-offload-bundler";
   case OffloadUnbundlingJobClass:
     return "clang-offload-unbundler";
+  case OffloadWrapperJobClass:
+    return "clang-offload-wrapper";
   }
 
   llvm_unreachable("invalid class");
 }
 
 void Action::propagateDeviceOffloadInfo(OffloadKind OKind, const char *OArch) {
   // Offload action set its own kinds on their dependences.
   if (Kind == OffloadClass)

 
 OffloadBundlingJobAction::OffloadBundlingJobAction(ActionList &Inputs)
     : JobAction(OffloadBundlingJobClass, Inputs, Inputs.back()->getType()) {}
 
 void OffloadUnbundlingJobAction::anchor() {}
 
 OffloadUnbundlingJobAction::OffloadUnbundlingJobAction(Action *Input)
     : JobAction(OffloadUnbundlingJobClass, Input, Input->getType()) {}
+
+void OffloadWrapperJobAction::anchor() {}
+
+OffloadWrapperJobAction::OffloadWrapperJobAction(ActionList &Inputs,
+                                                 types::ID Type)
+  : JobAction(OffloadWrapperJobClass, Inputs, Type) {}

clang/lib/Driver/Driver.cpp

getDeviceDependences(OffloadAction::DeviceDependences &DA,
       return ABRT_Inactive;
     }
 
     /// Update the state to include the provided host action \a HostAction as a
     /// dependency of the current device action. By default it is inactive.
     virtual ActionBuilderReturnCode addDeviceDepences(Action *HostAction) {
       return ABRT_Inactive;
     }
 
     /// Append top level actions generated by the builder.
     virtual void appendTopLevelActions(ActionList &AL) {}
 
     /// Append linker actions generated by the builder.
+    virtual void appendLinkActions(ActionList &AL) {}
+
+    /// Append linker actions generated by the builder.
     virtual void appendLinkDependences(OffloadAction::DeviceDependences &DA) {}
 
     /// Initialize the builder. Return true if any initialization errors are
     /// found.
     virtual bool initialize() { return false; }
 
     /// Return true if the builder can use bundling/unbundling.
     virtual bool canUseBundlerUnbundler() const { return false; }
void appendTopLevelActions(ActionList &AL) override {
         Dep.add(*A, **TI, /*BoundArch=*/nullptr, Action::OFK_OpenMP);
         AL.push_back(C.MakeAction<OffloadAction>(Dep, A->getType()));
         ++TI;
       }
       // We no longer need the action stored in this builder.
       OpenMPDeviceActions.clear();
     }
 
-    void appendLinkDependences(OffloadAction::DeviceDependences &DA) override {
+    void appendLinkActions(ActionList &AL) override {
       assert(ToolChains.size() == DeviceLinkerInputs.size() &&
              "Toolchains and linker inputs sizes do not match.");
 
       // Append a new link action for each device.
       auto TC = ToolChains.begin();
       for (auto &LI : DeviceLinkerInputs) {
         auto *DeviceLinkAction =
             C.MakeAction<LinkJobAction>(LI, types::TY_Image);
-        DA.add(*DeviceLinkAction, **TC, /*BoundArch=*/nullptr,
-               Action::OFK_OpenMP);
+        OffloadAction::DeviceDependences DeviceLinkDeps;
+        DeviceLinkDeps.add(*DeviceLinkAction, **TC, /*BoundArch=*/nullptr,
+            Action::OFK_OpenMP);
+        AL.push_back(C.MakeAction<OffloadAction>(DeviceLinkDeps,
+            DeviceLinkAction->getType()));
         ++TC;
       }
+      DeviceLinkerInputs.clear();
     }
 
+    void appendLinkDependences(OffloadAction::DeviceDependences &DA) override {}
+
     bool initialize() override {
       // Get the OpenMP toolchains. If we don't get any, the action builder will
       // know there is nothing to do related to OpenMP offloading.
       auto OpenMPTCRange = C.getOffloadToolChains<Action::OFK_OpenMP>();
       for (auto TI = OpenMPTCRange.first, TE = OpenMPTCRange.second; TI != TE;
            ++TI)
         ToolChains.push_back(TI->second);
 
bool appendTopLevelActions(ActionList &AL, Action *HostAction,
     // Propagate to the current host action (if any) the offload information
     // associated with the current input.
     if (HostAction)
       HostAction->propagateHostOffloadInfo(InputArgToOffloadKindMap[InputArg],
                                            /*BoundArch=*/nullptr);
     return false;
   }
 
+  Action* makeHostLinkAction() {
+    // Build a list of device linking actions.
+    ActionList DeviceAL;
+    for (DeviceActionBuilder *SB : SpecializedBuilders) {
+      if (!SB->isValid())
+        continue;
+      SB->appendLinkActions(DeviceAL);
+    }
+
+    if (DeviceAL.empty())
+      return nullptr;
+
+    // Create wrapper bitcode from the result of device link actions and compile
+    // it to an object which will be added to the host link command.
+    auto *BC = C.MakeAction<OffloadWrapperJobAction>(DeviceAL, types::TY_LLVM_BC);
+    auto *ASM = C.MakeAction<BackendJobAction>(BC, types::TY_PP_Asm);
+    return C.MakeAction<AssembleJobAction>(ASM, types::TY_Object);
+  }
+
   /// Processes the host linker action. This currently consists of replacing it
   /// with an offload action if there are device link objects and propagate to
   /// the host action all the offload kinds used in the current compilation. The
   /// resulting action is returned.
   Action *processHostLinkAction(Action *HostAction) {
     // Add all the dependences from the device linking actions.
     OffloadAction::DeviceDependences DDeps;
     for (auto *SB : SpecializedBuilders) {
if (Current)
       Actions.push_back(Current);
 
     // Add any top level actions generated for offloading.
     OffloadBuilder.appendTopLevelActions(Actions, Current, InputArg);
   }
 
   // Add a link action if necessary.
   if (!LinkerInputs.empty()) {
+    if (Action *Wrapper = OffloadBuilder.makeHostLinkAction())
+      LinkerInputs.push_back(Wrapper);
     Action *LA = C.MakeAction<LinkJobAction>(LinkerInputs, types::TY_Image);
     LA = OffloadBuilder.processHostLinkAction(LA);
     Actions.push_back(LA);
   }
 
   // If --print-supported-cpus, -mcpu=? or -mtune=? is specified, build a custom
   // Compile phase that prints out supported cpu models and quits.
   if (Arg *A = Args.getLastArg(options::OPT_print_supported_cpus)) {
const Tool *combineAssembleBackend(ArrayRef<JobActionInfo> ActionInfo,
                                      ActionList &CollapsedOffloadAction) {
     if (ActionInfo.size() < 2 || !canCollapseAssembleAction())
       return nullptr;
     auto *AJ = dyn_cast<AssembleJobAction>(ActionInfo[0].JA);
     auto *BJ = dyn_cast<BackendJobAction>(ActionInfo[1].JA);
     if (!AJ || !BJ)
       return nullptr;
 
-    // Retrieve the compile job, backend action must always be preceded by one.
-    ActionList CompileJobOffloadActions;
-    auto *CJ = getPrevDependentAction(BJ->getInputs(), CompileJobOffloadActions,
-                                      /*CanBeCollapsed=*/false);
-    if (!AJ || !BJ || !CJ)
-      return nullptr;
-
-    assert(isa<CompileJobAction>(CJ) &&
-           "Expecting compile job preceding backend job.");
-
-    // Get compiler tool.
-    const Tool *T = TC.SelectTool(*CJ);
+    // Get backend tool.
+    const Tool *T = TC.SelectTool(*BJ);
     if (!T)
       return nullptr;
 
     if (!T->hasIntegratedAssembler())
       return nullptr;
 
     Inputs = BJ->getInputs();
     AppendCollapsedOffloadAction(CollapsedOffloadAction, ActionInfo,
if (auto *UA = dyn_cast<OffloadUnbundlingJobAction>(JA)) {
     Result = InputInfo(A, BaseInput);
   else {
     // We only have to generate a prefix for the host if this is not a top-level
     // action.
     std::string OffloadingPrefix = Action::GetOffloadingFileNamePrefix(
         A->getOffloadingDeviceKind(), TC->getTriple().normalize(),
         /*CreatePrefixForHost=*/!!A->getOffloadingHostActiveKinds() &&
             !AtTopLevel);
+    if (isa<OffloadWrapperJobAction>(JA)) {
+      OffloadingPrefix += "-wrapper";
+      if (Arg *FinalOutput = C.getArgs().getLastArg(options::OPT_o))
+        BaseInput = FinalOutput->getValue();
+      else
+        BaseInput = getDefaultImageName();
+    }
     Result = InputInfo(A, GetNamedOutputPath(C, *JA, BaseInput, BoundArch,
                                              AtTopLevel, MultipleArchs,
                                              OffloadingPrefix),
                        BaseInput);
   }
 
   if (CCCPrintBindings && !CCGenDiagnostics) {
     llvm::errs() << "# \"" << T->getToolChain().getTripleString() << '"'

clang/lib/Driver/ToolChain.cpp

 }
 
 Tool *ToolChain::getOffloadBundler() const {
   if (!OffloadBundler)
     OffloadBundler.reset(new tools::OffloadBundler(*this));
   return OffloadBundler.get();
 }
 
+Tool *ToolChain::getOffloadWrapper() const {
+  if (!OffloadWrapper)
+    OffloadWrapper.reset(new tools::OffloadWrapper(*this));
+  return OffloadWrapper.get();
+}
+
 Tool *ToolChain::getTool(Action::ActionClass AC) const {
   switch (AC) {
   case Action::AssembleJobClass:
     return getAssemble();
 
   case Action::LinkJobClass:
     return getLink();
 
Tool *ToolChain::getTool(Action::ActionClass AC) const {
   case Action::MigrateJobClass:
   case Action::VerifyPCHJobClass:
   case Action::BackendJobClass:
     return getClang();
 
   case Action::OffloadBundlingJobClass:
   case Action::OffloadUnbundlingJobClass:
     return getOffloadBundler();
+
+  case Action::OffloadWrapperJobClass:
+    return getOffloadWrapper();
   }
 
   llvm_unreachable("Invalid tool kind.");
 }
 
 static StringRef getArchNameForCompilerRTLib(const ToolChain &TC,
                                              const ArgList &Args) {
   const llvm::Triple &Triple = TC.getTriple();

clang/lib/Driver/ToolChains/Clang.h

void ConstructJob(Compilation &C, const JobAction &JA,
                     const llvm::opt::ArgList &TCArgs,
                     const char *LinkingOutput) const override;
   void ConstructJobMultipleOutputs(Compilation &C, const JobAction &JA,
                                    const InputInfoList &Outputs,
                                    const InputInfoList &Inputs,
                                    const llvm::opt::ArgList &TCArgs,
                                    const char *LinkingOutput) const override;
 };
+
+/// Offload wrapper tool.
+class LLVM_LIBRARY_VISIBILITY OffloadWrapper final : public Tool {
+public:
+  OffloadWrapper(const ToolChain &TC)
+      : Tool("offload wrapper", "clang-offload-wrapper", TC) {}
+
+  bool hasIntegratedCPP() const override { return false; }
+  void ConstructJob(Compilation &C, const JobAction &JA,
+                    const InputInfo &Output, const InputInfoList &Inputs,
+                    const llvm::opt::ArgList &TCArgs,
+                    const char *LinkingOutput) const override;
+};
+
 } // end namespace tools
 
 } // end namespace driver
 } // end namespace clang
 
 #endif // LLVM_CLANG_LIB_DRIVER_TOOLCHAINS_CLANG_H

clang/lib/Driver/ToolChains/Clang.cpp

void OffloadBundler::ConstructJobMultipleOutputs(
   CmdArgs.push_back("-unbundle");
 
   // All the inputs are encoded as commands.
   C.addCommand(std::make_unique<Command>(
       JA, *this,
       TCArgs.MakeArgString(getToolChain().GetProgramPath(getShortName())),
       CmdArgs, None));
 }
+
+void OffloadWrapper::ConstructJob(Compilation &C, const JobAction &JA,
+                                  const InputInfo &Output,
+                                  const InputInfoList &Inputs,
+                                  const ArgList &Args,
+                                  const char *LinkingOutput) const {
+  ArgStringList CmdArgs;
+
+  const llvm::Triple &Triple = getToolChain().getEffectiveTriple();
+
+  // Add the "effective" target triple.
+  CmdArgs.push_back("-target");
+  CmdArgs.push_back(Args.MakeArgString(Triple.getTriple()));
+
+  assert(JA.getInputs().size() == Inputs.size() &&
+         "Not have inputs for all dependence actions??");
+
+  // Add offload targets. It is a comma-separated list of offload target
+  // triples.
+  SmallString<128> Targets;
+  Targets += "-offload-targets=";
+  for (unsigned I = 0; I < Inputs.size(); ++I) {
+    if (I)
+      Targets += ',';
+
+    // Get input's Offload Kind and ToolChain.
+    const auto *OA = cast<OffloadAction>(JA.getInputs()[I]);
+    assert(OA->hasSingleDeviceDependence(/*DoNotConsiderHostActions=*/true) &&
+           "Expected one device dependence!");
+    const ToolChain *DeviceTC = nullptr;
+    OA->doOnEachDependence([&DeviceTC](Action *, const ToolChain *TC,
+                                       const char *) { DeviceTC = TC; });
+
+    // And add it to the offload targets.
+    Targets += DeviceTC->getTriple().normalize();
+  }
+  CmdArgs.push_back(Args.MakeArgString(Targets));
+
+  // Add the output file name.
+  assert(Output.isFilename() && "Invalid output.");
+  CmdArgs.push_back("-o");
+  CmdArgs.push_back(Output.getFilename());
+
+  // Add inputs.
+  for (const InputInfo &I : Inputs) {
+    assert(I.isFilename() && "Invalid input.");
+    CmdArgs.push_back(I.getFilename());
+  }
+
+  C.addCommand(std::make_unique<Command>(
+    JA, *this,
+    Args.MakeArgString(getToolChain().GetProgramPath(getShortName())),
+    CmdArgs, Inputs));
+}

clang/lib/Driver/ToolChains/CommonArgs.h

 
 void linkXRayRuntimeDeps(const ToolChain &TC,
                          llvm::opt::ArgStringList &CmdArgs);
 
 void AddRunTimeLibs(const ToolChain &TC, const Driver &D,
                     llvm::opt::ArgStringList &CmdArgs,
                     const llvm::opt::ArgList &Args);
 
-void AddOpenMPLinkerScript(const ToolChain &TC, Compilation &C,
-                           const InputInfo &Output,
-                           const InputInfoList &Inputs,
-                           const llvm::opt::ArgList &Args,
-                           llvm::opt::ArgStringList &CmdArgs,
-                           const JobAction &JA);
-
 void AddHIPLinkerScript(const ToolChain &TC, Compilation &C,
                         const InputInfo &Output, const InputInfoList &Inputs,
                         const llvm::opt::ArgList &Args,
                         llvm::opt::ArgStringList &CmdArgs, const JobAction &JA,
                         const Tool &T);
 
 const char *SplitDebugName(const llvm::opt::ArgList &Args,
                            const InputInfo &Input, const InputInfo &Output);

clang/lib/Driver/ToolChains/CommonArgs.cpp

if (TC.getTriple().isKnownWindowsMSVCEnvironment()) {
             << Args.getLastArg(options::OPT_rtlib_EQ)->getValue() << "MSVC";
       }
     } else
       AddLibgcc(TC, D, CmdArgs, Args);
     break;
   }
 }
 
-/// Add OpenMP linker script arguments at the end of the argument list so that
-/// the fat binary is built by embedding each of the device images into the
-/// host. The linker script also defines a few symbols required by the code
-/// generation so that the images can be easily retrieved at runtime by the
-/// offloading library. This should be used only in tool chains that support
-/// linker scripts.
-void tools::AddOpenMPLinkerScript(const ToolChain &TC, Compilation &C,
-                                  const InputInfo &Output,
-                                  const InputInfoList &Inputs,
-                                  const ArgList &Args, ArgStringList &CmdArgs,
-                                  const JobAction &JA) {
-
-  // If this is not an OpenMP host toolchain, we don't need to do anything.
-  if (!JA.isHostOffloading(Action::OFK_OpenMP))
-    return;
-
-  // Create temporary linker script. Keep it if save-temps is enabled.
-  const char *LKS;
-  SmallString<256> Name = llvm::sys::path::filename(Output.getFilename());
-  if (C.getDriver().isSaveTempsEnabled()) {
-    llvm::sys::path::replace_extension(Name, "lk");
-    LKS = C.getArgs().MakeArgString(Name.c_str());
-  } else {
-    llvm::sys::path::replace_extension(Name, "");
-    Name = C.getDriver().GetTemporaryPath(Name, "lk");
-    LKS = C.addTempFile(C.getArgs().MakeArgString(Name.c_str()));
-  }
-
-  // Add linker script option to the command.
-  CmdArgs.push_back("-T");
-  CmdArgs.push_back(LKS);
-
-  // Create a buffer to write the contents of the linker script.
-  std::string LksBuffer;
-  llvm::raw_string_ostream LksStream(LksBuffer);
-
-  // Get the OpenMP offload tool chains so that we can extract the triple
-  // associated with each device input.
-  auto OpenMPToolChains = C.getOffloadToolChains<Action::OFK_OpenMP>();
-  assert(OpenMPToolChains.first != OpenMPToolChains.second &&
-         "No OpenMP toolchains??");
-
-  // Track the input file name and device triple in order to build the script,
-  // inserting binaries in the designated sections.
-  SmallVector<std::pair<std::string, const char *>, 8> InputBinaryInfo;
-
-  // Add commands to embed target binaries. We ensure that each section and
-  // image is 16-byte aligned. This is not mandatory, but increases the
-  // likelihood of data to be aligned with a cache block in several main host
-  // machines.
-  LksStream << "/*\n";
-  LksStream << "       OpenMP Offload Linker Script\n";
-  LksStream << " *** Automatically generated by Clang ***\n";
-  LksStream << "*/\n";
-  LksStream << "TARGET(binary)\n";
-  auto DTC = OpenMPToolChains.first;
-  for (auto &II : Inputs) {
-    const Action *A = II.getAction();
-    // Is this a device linking action?
-    if (A && isa<LinkJobAction>(A) &&
-        A->isDeviceOffloading(Action::OFK_OpenMP)) {
-      assert(DTC != OpenMPToolChains.second &&
-             "More device inputs than device toolchains??");
-      InputBinaryInfo.push_back(std::make_pair(
-          DTC->second->getTriple().normalize(), II.getFilename()));
-      ++DTC;
-      LksStream << "INPUT(" << II.getFilename() << ")\n";
-    }
-  }
-
-  assert(DTC == OpenMPToolChains.second &&
-         "Less device inputs than device toolchains??");
-
-  LksStream << "SECTIONS\n";
-  LksStream << "{\n";
-
-  // Put each target binary into a separate section.
-  for (const auto &BI : InputBinaryInfo) {
-    LksStream << "  .omp_offloading." << BI.first << " :\n";
-    LksStream << "  ALIGN(0x10)\n";
-    LksStream << "  {\n";
-    LksStream << "    PROVIDE_HIDDEN(.omp_offloading.img_start." << BI.first
-              << " = .);\n";
-    LksStream << "    " << BI.second << "\n";
-    LksStream << "    PROVIDE_HIDDEN(.omp_offloading.img_end." << BI.first
-              << " = .);\n";
-    LksStream << "  }\n";
-  }
-
-  LksStream << "}\n";
-  LksStream << "INSERT BEFORE .data\n";
-  LksStream.flush();
-
-  // Dump the contents of the linker script if the user requested that. We
-  // support this option to enable testing of behavior with -###.
-  if (C.getArgs().hasArg(options::OPT_fopenmp_dump_offload_linker_script))
-    llvm::errs() << LksBuffer;
-
-  // If this is a dry run, do not create the linker script file.
-  if (C.getArgs().hasArg(options::OPT__HASH_HASH_HASH))
-    return;
-
-  // Open script file and write the contents.
-  std::error_code EC;
-  llvm::raw_fd_ostream Lksf(LKS, EC, llvm::sys::fs::OF_None);
-
-  if (EC) {
-    C.getDriver().Diag(clang::diag::err_unable_to_make_temp) << EC.message();
-    return;
-  }
-
-  Lksf << LksBuffer;
-}
-
 /// Add HIP linker script arguments at the end of the argument list so that
 /// the fat binary is built by embedding the device images into the host. The
 /// linker script also defines a symbol required by the code generation so that
 /// the image can be retrieved at runtime. This should be used only in tool
 /// chains that support linker scripts.
 void tools::AddHIPLinkerScript(const ToolChain &TC, Compilation &C,
                                const InputInfo &Output,
                                const InputInfoList &Inputs, const ArgList &Args,

clang/lib/Driver/ToolChains/Cuda.cpp

if (!II.isFilename())
       continue;
 
     const char *CubinF = C.addTempFile(
         C.getArgs().MakeArgString(getToolChain().getInputFilename(II)));
 
     CmdArgs.push_back(CubinF);
   }
 
-  AddOpenMPLinkerScript(getToolChain(), C, Output, Inputs, Args, CmdArgs, JA);
-
   const char *Exec =
       Args.MakeArgString(getToolChain().GetProgramPath("nvlink"));
   C.addCommand(std::make_unique<Command>(JA, *this, Exec, CmdArgs, Inputs));
 }
 
 /// CUDA toolchain.  Our assembler is ptxas, and our "linker" is fatbinary,
 /// which isn't properly a linker but nonetheless performs the step of stitching
 /// together object files from the assembler into a single blob.

clang/lib/Driver/ToolChains/Gnu.cpp

else if (HasCRTBeginEndFiles) {
           crtbegin = isAndroid ? "crtbegin_so.o" : "crtbeginS.o";
         else if (IsPIE || IsStaticPIE)
           crtbegin = isAndroid ? "crtbegin_dynamic.o" : "crtbeginS.o";
         else
           crtbegin = isAndroid ? "crtbegin_dynamic.o" : "crtbegin.o";
         P = ToolChain.GetFilePath(crtbegin);
       }
       CmdArgs.push_back(Args.MakeArgString(P));
-    }
+    }
 
     // Add crtfastmath.o if available and fast math is enabled.
     ToolChain.AddFastMathRuntimeIfAvailable(Args, CmdArgs);
   }
 
   Args.AddAllArgs(CmdArgs, options::OPT_L);
   Args.AddAllArgs(CmdArgs, options::OPT_u);
 
if (!Args.hasArg(options::OPT_nostartfiles) && !IsIAMCU) {
         }
         CmdArgs.push_back(Args.MakeArgString(P));
       }
       if (!isAndroid)
         CmdArgs.push_back(Args.MakeArgString(ToolChain.GetFilePath("crtn.o")));
     }
   }
 
-  // Add OpenMP offloading linker script args if required.
-  AddOpenMPLinkerScript(getToolChain(), C, Output, Inputs, Args, CmdArgs, JA);
-
   // Add HIP offloading linker script args if required.
   AddHIPLinkerScript(getToolChain(), C, Output, Inputs, Args, CmdArgs, JA,
                      *this);
 
   const char *Exec = Args.MakeArgString(ToolChain.GetLinkerPath());
   C.addCommand(std::make_unique<Command>(JA, *this, Exec, CmdArgs, Inputs));
 }
 

clang/test/Driver/clang-offload-wrapper.c

+// REQUIRES: x86-registered-target
+
+//
+// Check help message.
+//
+// RUN: clang-offload-wrapper --help | FileCheck %s --check-prefix CHECK-HELP
+// CHECK-HELP: {{.*}}OVERVIEW: A tool to create a wrapper bitcode for offload target binaries. Takes offload
+// CHECK-HELP: {{.*}}target binaries as input and produces bitcode file containing target binaries packaged
+// CHECK-HELP: {{.*}}as data.
+// CHECK-HELP: {{.*}}USAGE: clang-offload-wrapper [options] <input files>
+// CHECK-HELP: {{.*}}  -o=<filename>               - Output filename
+// CHECK-HELP: {{.*}}  --offload-targets=<triples> - Comma-separated list of device target triples
+// CHECK-HELP: {{.*}}  --target=<triple>           - Target triple for the output module
+
+//
+// Generate a file to wrap.
+//
+// RUN: echo 'Content of device file' > %t.tgt
+
+//
+// Check bitcode produced by the wrapper tool.
+//
+// RUN: clang-offload-wrapper -target=x86_64-pc-linux-gnu -offload-targets=powerpc64le-ibm-linux-gnu -o %t.wrapper.bc %t.tgt
+// RUN: llvm-dis %t.wrapper.bc -o - | FileCheck %s --check-prefix CHECK-IR
+
+// CHECK-IR: target triple = "x86_64-pc-linux-gnu"
+
+// CHECK-IR: @.omp_offloading.img_start.powerpc64le-ibm-linux-gnu = hidden unnamed_addr constant [{{[0-9]+}} x i8] c"Content of device file{{.+}}", section ".omp_offloading.powerpc64le-ibm-linux-gnu"
+// CHECK-IR: @.omp_offloading.img_end.powerpc64le-ibm-linux-gnu = hidden unnamed_addr constant [0 x i8] zeroinitializer, section ".omp_offloading.powerpc64le-ibm-linux-gnu"
+

clang/test/Driver/openmp-offload-gpu.c

 // RUN:   %clang -### -no-canonical-prefixes -target powerpc64le-unknown-linux-gnu -fopenmp=libomp \
 // RUN:          -fopenmp-targets=nvptx64-nvidia-cuda -save-temps %s 2>&1 \
 // RUN:   | FileCheck -check-prefix=CHK-CUBIN-NVLINK %s
 /// Check cubin file generation and usage by nvlink when toolchain has BindArchAction
 // RUN:   %clang -### -no-canonical-prefixes -target x86_64-apple-darwin17.0.0 -fopenmp=libomp \
 // RUN:          -fopenmp-targets=nvptx64-nvidia-cuda %s 2>&1 \
 // RUN:   | FileCheck -check-prefix=CHK-CUBIN-NVLINK %s
 
-// CHK-CUBIN-NVLINK: clang{{.*}}" "-o" "[[PTX:.*\.s]]"
+// CHK-CUBIN-NVLINK: clang{{.*}}" {{.*}}"-fopenmp-is-device" {{.*}}"-o" "[[PTX:.*\.s]]"
 // CHK-CUBIN-NVLINK-NEXT: ptxas{{.*}}" "--output-file" "[[CUBIN:.*\.cubin]]" {{.*}}"[[PTX]]"
 // CHK-CUBIN-NVLINK-NEXT: nvlink{{.*}}" {{.*}}"[[CUBIN]]"
 
 /// ###########################################################################
 
 /// Check unbundlink of assembly file, cubin file generation and usage by nvlink
 // RUN:   touch %t.s
 // RUN:   %clang -### -target powerpc64le-unknown-linux-gnu -fopenmp=libomp -fopenmp-targets=nvptx64-nvidia-cuda \

clang/test/Driver/openmp-offload.c

 /// action.
 // RUN:   %clang -ccc-print-phases -fopenmp=libomp -target powerpc64le-ibm-linux-gnu -fopenmp-targets=x86_64-pc-linux-gnu %s 2>&1 \
 // RUN:   | FileCheck -check-prefix=CHK-PHASES %s
 // CHK-PHASES: 0: input, "[[INPUT:.+\.c]]", c, (host-openmp)
 // CHK-PHASES: 1: preprocessor, {0}, cpp-output, (host-openmp)
 // CHK-PHASES: 2: compiler, {1}, ir, (host-openmp)
 // CHK-PHASES: 3: backend, {2}, assembler, (host-openmp)
 // CHK-PHASES: 4: assembler, {3}, object, (host-openmp)
-// CHK-PHASES: 5: linker, {4}, image, (host-openmp)
-// CHK-PHASES: 6: input, "[[INPUT]]", c, (device-openmp)
-// CHK-PHASES: 7: preprocessor, {6}, cpp-output, (device-openmp)
-// CHK-PHASES: 8: compiler, {7}, ir, (device-openmp)
-// CHK-PHASES: 9: offload, "host-openmp (powerpc64le-ibm-linux-gnu)" {2}, "device-openmp (x86_64-pc-linux-gnu)" {8}, ir
-// CHK-PHASES: 10: backend, {9}, assembler, (device-openmp)
-// CHK-PHASES: 11: assembler, {10}, object, (device-openmp)
-// CHK-PHASES: 12: linker, {11}, image, (device-openmp)
-// CHK-PHASES: 13: offload, "host-openmp (powerpc64le-ibm-linux-gnu)" {5}, "device-openmp (x86_64-pc-linux-gnu)" {12}, image
+// CHK-PHASES: 5: input, "[[INPUT]]", c, (device-openmp)
+// CHK-PHASES: 6: preprocessor, {5}, cpp-output, (device-openmp)
+// CHK-PHASES: 7: compiler, {6}, ir, (device-openmp)
+// CHK-PHASES: 8: offload, "host-openmp (powerpc64le-ibm-linux-gnu)" {2}, "device-openmp (x86_64-pc-linux-gnu)" {7}, ir
+// CHK-PHASES: 9: backend, {8}, assembler, (device-openmp)
+// CHK-PHASES: 10: assembler, {9}, object, (device-openmp)
+// CHK-PHASES: 11: linker, {10}, image, (device-openmp)
+// CHK-PHASES: 12: offload, "device-openmp (x86_64-pc-linux-gnu)" {11}, image
+// CHK-PHASES: 13: clang-offload-wrapper, {12}, ir, (host-openmp)
+// CHK-PHASES: 14: backend, {13}, assembler, (host-openmp)
+// CHK-PHASES: 15: assembler, {14}, object, (host-openmp)
+// CHK-PHASES: 16: linker, {4, 15}, image, (host-openmp)
 
 /// ###########################################################################
 
 /// Check the phases when using multiple targets. Here we also add a library to
 /// make sure it is treated as input by the device.
 // RUN:   %clang -ccc-print-phases -lsomelib -fopenmp=libomp -target powerpc64-ibm-linux-gnu -fopenmp-targets=x86_64-pc-linux-gnu,powerpc64-ibm-linux-gnu %s 2>&1 \
 // RUN:   | FileCheck -check-prefix=CHK-PHASES-LIB %s
 // CHK-PHASES-LIB: 0: input, "somelib", object, (host-openmp)
 // CHK-PHASES-LIB: 1: input, "[[INPUT:.+\.c]]", c, (host-openmp)
 // CHK-PHASES-LIB: 2: preprocessor, {1}, cpp-output, (host-openmp)
 // CHK-PHASES-LIB: 3: compiler, {2}, ir, (host-openmp)
 // CHK-PHASES-LIB: 4: backend, {3}, assembler, (host-openmp)
 // CHK-PHASES-LIB: 5: assembler, {4}, object, (host-openmp)
-// CHK-PHASES-LIB: 6: linker, {0, 5}, image, (host-openmp)
-// CHK-PHASES-LIB: 7: input, "somelib", object, (device-openmp)
-// CHK-PHASES-LIB: 8: input, "[[INPUT]]", c, (device-openmp)
-// CHK-PHASES-LIB: 9: preprocessor, {8}, cpp-output, (device-openmp)
-// CHK-PHASES-LIB: 10: compiler, {9}, ir, (device-openmp)
-// CHK-PHASES-LIB: 11: offload, "host-openmp (powerpc64-ibm-linux-gnu)" {3}, "device-openmp (x86_64-pc-linux-gnu)" {10}, ir
-// CHK-PHASES-LIB: 12: backend, {11}, assembler, (device-openmp)
-// CHK-PHASES-LIB: 13: assembler, {12}, object, (device-openmp)
-// CHK-PHASES-LIB: 14: linker, {7, 13}, image, (device-openmp)
+// CHK-PHASES-LIB: 6: input, "somelib", object, (device-openmp)
+// CHK-PHASES-LIB: 7: input, "[[INPUT]]", c, (device-openmp)
+// CHK-PHASES-LIB: 8: preprocessor, {7}, cpp-output, (device-openmp)
+// CHK-PHASES-LIB: 9: compiler, {8}, ir, (device-openmp)
+// CHK-PHASES-LIB: 10: offload, "host-openmp (powerpc64-ibm-linux-gnu)" {3}, "device-openmp (x86_64-pc-linux-gnu)" {9}, ir
+// CHK-PHASES-LIB: 11: backend, {10}, assembler, (device-openmp)
+// CHK-PHASES-LIB: 12: assembler, {11}, object, (device-openmp)
+// CHK-PHASES-LIB: 13: linker, {6, 12}, image, (device-openmp)
+// CHK-PHASES-LIB: 14: offload, "device-openmp (x86_64-pc-linux-gnu)" {13}, image
 // CHK-PHASES-LIB: 15: input, "somelib", object, (device-openmp)
 // CHK-PHASES-LIB: 16: input, "[[INPUT]]", c, (device-openmp)
 // CHK-PHASES-LIB: 17: preprocessor, {16}, cpp-output, (device-openmp)
 // CHK-PHASES-LIB: 18: compiler, {17}, ir, (device-openmp)
 // CHK-PHASES-LIB: 19: offload, "host-openmp (powerpc64-ibm-linux-gnu)" {3}, "device-openmp (powerpc64-ibm-linux-gnu)" {18}, ir
 // CHK-PHASES-LIB: 20: backend, {19}, assembler, (device-openmp)
 // CHK-PHASES-LIB: 21: assembler, {20}, object, (device-openmp)
 // CHK-PHASES-LIB: 22: linker, {15, 21}, image, (device-openmp)
-// CHK-PHASES-LIB: 23: offload, "host-openmp (powerpc64-ibm-linux-gnu)" {6}, "device-openmp (x86_64-pc-linux-gnu)" {14}, "device-openmp (powerpc64-ibm-linux-gnu)" {22}, image
-
+// CHK-PHASES-LIB: 23: offload, "device-openmp (powerpc64-ibm-linux-gnu)" {22}, image
+// CHK-PHASES-LIB: 24: clang-offload-wrapper, {14, 23}, ir, (host-openmp)
+// CHK-PHASES-LIB: 25: backend, {24}, assembler, (host-openmp)
+// CHK-PHASES-LIB: 26: assembler, {25}, object, (host-openmp)
+// CHK-PHASES-LIB: 27: linker, {0, 5, 26}, image, (host-openmp)
 
 /// ###########################################################################
 
 /// Check the phases when using multiple targets and multiple source files
 // RUN:   echo " " > %t.c
 // RUN:   %clang -ccc-print-phases -lsomelib -fopenmp=libomp -target powerpc64-ibm-linux-gnu -fopenmp-targets=x86_64-pc-linux-gnu,powerpc64-ibm-linux-gnu %s %t.c 2>&1 \
 // RUN:   | FileCheck -check-prefix=CHK-PHASES-FILES %s
 // CHK-PHASES-FILES: 0: input, "somelib", object, (host-openmp)
 // CHK-PHASES-FILES: 1: input, "[[INPUT1:.+\.c]]", c, (host-openmp)
 // CHK-PHASES-FILES: 2: preprocessor, {1}, cpp-output, (host-openmp)
 // CHK-PHASES-FILES: 3: compiler, {2}, ir, (host-openmp)
 // CHK-PHASES-FILES: 4: backend, {3}, assembler, (host-openmp)
 // CHK-PHASES-FILES: 5: assembler, {4}, object, (host-openmp)
 // CHK-PHASES-FILES: 6: input, "[[INPUT2:.+\.c]]", c, (host-openmp)
 // CHK-PHASES-FILES: 7: preprocessor, {6}, cpp-output, (host-openmp)
 // CHK-PHASES-FILES: 8: compiler, {7}, ir, (host-openmp)
 // CHK-PHASES-FILES: 9: backend, {8}, assembler, (host-openmp)
 // CHK-PHASES-FILES: 10: assembler, {9}, object, (host-openmp)
-// CHK-PHASES-FILES: 11: linker, {0, 5, 10}, image, (host-openmp)
-// CHK-PHASES-FILES: 12: input, "somelib", object, (device-openmp)
-// CHK-PHASES-FILES: 13: input, "[[INPUT1]]", c, (device-openmp)
-// CHK-PHASES-FILES: 14: preprocessor, {13}, cpp-output, (device-openmp)
-// CHK-PHASES-FILES: 15: compiler, {14}, ir, (device-openmp)
-// CHK-PHASES-FILES: 16: offload, "host-openmp (powerpc64-ibm-linux-gnu)" {3}, "device-openmp (x86_64-pc-linux-gnu)" {15}, ir
-// CHK-PHASES-FILES: 17: backend, {16}, assembler, (device-openmp)
-// CHK-PHASES-FILES: 18: assembler, {17}, object, (device-openmp)
-// CHK-PHASES-FILES: 19: input, "[[INPUT2]]", c, (device-openmp)
-// CHK-PHASES-FILES: 20: preprocessor, {19}, cpp-output, (device-openmp)
-// CHK-PHASES-FILES: 21: compiler, {20}, ir, (device-openmp)
-// CHK-PHASES-FILES: 22: offload, "host-openmp (powerpc64-ibm-linux-gnu)" {8}, "device-openmp (x86_64-pc-linux-gnu)" {21}, ir
-// CHK-PHASES-FILES: 23: backend, {22}, assembler, (device-openmp)
-// CHK-PHASES-FILES: 24: assembler, {23}, object, (device-openmp)
-// CHK-PHASES-FILES: 25: linker, {12, 18, 24}, image, (device-openmp)
+// CHK-PHASES-FILES: 11: input, "somelib", object, (device-openmp)
+// CHK-PHASES-FILES: 12: input, "[[INPUT1]]", c, (device-openmp)
+// CHK-PHASES-FILES: 13: preprocessor, {12}, cpp-output, (device-openmp)
+// CHK-PHASES-FILES: 14: compiler, {13}, ir, (device-openmp)
+// CHK-PHASES-FILES: 15: offload, "host-openmp (powerpc64-ibm-linux-gnu)" {3}, "device-openmp (x86_64-pc-linux-gnu)" {14}, ir
+// CHK-PHASES-FILES: 16: backend, {15}, assembler, (device-openmp)
+// CHK-PHASES-FILES: 17: assembler, {16}, object, (device-openmp)
+// CHK-PHASES-FILES: 18: input, "[[INPUT2]]", c, (device-openmp)
+// CHK-PHASES-FILES: 19: preprocessor, {18}, cpp-output, (device-openmp)
+// CHK-PHASES-FILES: 20: compiler, {19}, ir, (device-openmp)
+// CHK-PHASES-FILES: 21: offload, "host-openmp (powerpc64-ibm-linux-gnu)" {8}, "device-openmp (x86_64-pc-linux-gnu)" {20}, ir
+// CHK-PHASES-FILES: 22: backend, {21}, assembler, (device-openmp)
+// CHK-PHASES-FILES: 23: assembler, {22}, object, (device-openmp)
+// CHK-PHASES-FILES: 24: linker, {11, 17, 23}, image, (device-openmp)
+// CHK-PHASES-FILES: 25: offload, "device-openmp (x86_64-pc-linux-gnu)" {24}, image
 // CHK-PHASES-FILES: 26: input, "somelib", object, (device-openmp)
 // CHK-PHASES-FILES: 27: input, "[[INPUT1]]", c, (device-openmp)
 // CHK-PHASES-FILES: 28: preprocessor, {27}, cpp-output, (device-openmp)
 // CHK-PHASES-FILES: 29: compiler, {28}, ir, (device-openmp)
 // CHK-PHASES-FILES: 30: offload, "host-openmp (powerpc64-ibm-linux-gnu)" {3}, "device-openmp (powerpc64-ibm-linux-gnu)" {29}, ir
 // CHK-PHASES-FILES: 31: backend, {30}, assembler, (device-openmp)
 // CHK-PHASES-FILES: 32: assembler, {31}, object, (device-openmp)
 // CHK-PHASES-FILES: 33: input, "[[INPUT2]]", c, (device-openmp)
 // CHK-PHASES-FILES: 34: preprocessor, {33}, cpp-output, (device-openmp)
 // CHK-PHASES-FILES: 35: compiler, {34}, ir, (device-openmp)
 // CHK-PHASES-FILES: 36: offload, "host-openmp (powerpc64-ibm-linux-gnu)" {8}, "device-openmp (powerpc64-ibm-linux-gnu)" {35}, ir
 // CHK-PHASES-FILES: 37: backend, {36}, assembler, (device-openmp)
 // CHK-PHASES-FILES: 38: assembler, {37}, object, (device-openmp)
 // CHK-PHASES-FILES: 39: linker, {26, 32, 38}, image, (device-openmp)
-// CHK-PHASES-FILES: 40: offload, "host-openmp (powerpc64-ibm-linux-gnu)" {11}, "device-openmp (x86_64-pc-linux-gnu)" {25}, "device-openmp (powerpc64-ibm-linux-gnu)" {39}, image
+// CHK-PHASES-FILES: 40: offload, "device-openmp (powerpc64-ibm-linux-gnu)" {39}, image
+// CHK-PHASES-FILES: 41: clang-offload-wrapper, {25, 40}, ir, (host-openmp)
+// CHK-PHASES-FILES: 42: backend, {41}, assembler, (host-openmp)
+// CHK-PHASES-FILES: 43: assembler, {42}, object, (host-openmp)
+// CHK-PHASES-FILES: 44: linker, {0, 5, 10, 43}, image, (host-openmp)
 
 /// ###########################################################################
 
 /// Check the phases graph when using a single GPU target, and check the OpenMP
 /// and CUDA phases are articulated correctly.
 // RUN:   %clang -ccc-print-phases -fopenmp=libomp -target powerpc64le-ibm-linux-gnu -fopenmp-targets=nvptx64-nvidia-cuda -x cuda %s 2>&1 \
 // RUN:   | FileCheck -check-prefix=CHK-PHASES-WITH-CUDA %s
 // CHK-PHASES-WITH-CUDA: 0: input, "[[INPUT:.+\.c]]", cuda, (host-cuda-openmp)
 // CHK-PHASES-WITH-CUDA: 1: preprocessor, {0}, cuda-cpp-output, (host-cuda-openmp)
 // CHK-PHASES-WITH-CUDA: 2: compiler, {1}, ir, (host-cuda-openmp)
 // CHK-PHASES-WITH-CUDA: 3: input, "[[INPUT]]", cuda, (device-cuda, sm_20)
 // CHK-PHASES-WITH-CUDA: 4: preprocessor, {3}, cuda-cpp-output, (device-cuda, sm_20)
 // CHK-PHASES-WITH-CUDA: 5: compiler, {4}, ir, (device-cuda, sm_20)
 // CHK-PHASES-WITH-CUDA: 6: backend, {5}, assembler, (device-cuda, sm_20)
 // CHK-PHASES-WITH-CUDA: 7: assembler, {6}, object, (device-cuda, sm_20)
 // CHK-PHASES-WITH-CUDA: 8: offload, "device-cuda (nvptx64-nvidia-cuda:sm_20)" {7}, object
 // CHK-PHASES-WITH-CUDA: 9: offload, "device-cuda (nvptx64-nvidia-cuda:sm_20)" {6}, assembler
 // CHK-PHASES-WITH-CUDA: 10: linker, {8, 9}, cuda-fatbin, (device-cuda)
 // CHK-PHASES-WITH-CUDA: 11: offload, "host-cuda-openmp (powerpc64le-ibm-linux-gnu)" {2}, "device-cuda (nvptx64-nvidia-cuda)" {10}, ir
 // CHK-PHASES-WITH-CUDA: 12: backend, {11}, assembler, (host-cuda-openmp)
 // CHK-PHASES-WITH-CUDA: 13: assembler, {12}, object, (host-cuda-openmp)
-// CHK-PHASES-WITH-CUDA: 14: linker, {13}, image, (host-cuda-openmp)
-// CHK-PHASES-WITH-CUDA: 15: input, "[[INPUT]]", cuda, (device-openmp)
-// CHK-PHASES-WITH-CUDA: 16: preprocessor, {15}, cuda-cpp-output, (device-openmp)
-// CHK-PHASES-WITH-CUDA: 17: compiler, {16}, ir, (device-openmp)
-// CHK-PHASES-WITH-CUDA: 18: offload, "host-cuda-openmp (powerpc64le-ibm-linux-gnu)" {2}, "device-openmp (nvptx64-nvidia-cuda)" {17}, ir
-// CHK-PHASES-WITH-CUDA: 19: backend, {18}, assembler, (device-openmp)
-// CHK-PHASES-WITH-CUDA: 20: assembler, {19}, object, (device-openmp)
-// CHK-PHASES-WITH-CUDA: 21: linker, {20}, image, (device-openmp)
-// CHK-PHASES-WITH-CUDA: 22: offload, "host-cuda-openmp (powerpc64le-ibm-linux-gnu)" {14}, "device-openmp (nvptx64-nvidia-cuda)" {21}, image
+// CHK-PHASES-WITH-CUDA: 14: input, "[[INPUT]]", cuda, (device-openmp)
+// CHK-PHASES-WITH-CUDA: 15: preprocessor, {14}, cuda-cpp-output, (device-openmp)
+// CHK-PHASES-WITH-CUDA: 16: compiler, {15}, ir, (device-openmp)
+// CHK-PHASES-WITH-CUDA: 17: offload, "host-cuda-openmp (powerpc64le-ibm-linux-gnu)" {2}, "device-openmp (nvptx64-nvidia-cuda)" {16}, ir
+// CHK-PHASES-WITH-CUDA: 18: backend, {17}, assembler, (device-openmp)
+// CHK-PHASES-WITH-CUDA: 19: assembler, {18}, object, (device-openmp)
+// CHK-PHASES-WITH-CUDA: 20: linker, {19}, image, (device-openmp)
+// CHK-PHASES-WITH-CUDA: 21: offload, "device-openmp (nvptx64-nvidia-cuda)" {20}, image
+// CHK-PHASES-WITH-CUDA: 22: clang-offload-wrapper, {21}, ir, (host-cuda-openmp)
+// CHK-PHASES-WITH-CUDA: 23: backend, {22}, assembler, (host-cuda-openmp)
+// CHK-PHASES-WITH-CUDA: 24: assembler, {23}, object, (host-cuda-openmp)
+// CHK-PHASES-WITH-CUDA: 25: linker, {13, 24}, image, (host-cuda-openmp)
 
 /// ###########################################################################
 
 /// Check of the commands passed to each tool when using valid OpenMP targets.
 /// Here we also check that offloading does not break the use of integrated
 /// assembler. It does however preclude the merge of the host compile and
 /// backend phases. There are also two offloading specific options:
 /// -fopenmp-is-device: will tell the frontend that it will generate code for a
 /// target.
 /// -fopenmp-host-ir-file-path: specifies the host IR file that can be loaded by
 /// the target code generation to gather information about which declaration
 /// really need to be emitted.
-/// We use -fopenmp-dump-offload-linker-script to dump the linker script and
-/// check its contents.
 ///
-// RUN:   %clang -### -fopenmp=libomp -o %t.out -target powerpc64le-linux -fopenmp-targets=powerpc64le-ibm-linux-gnu,x86_64-pc-linux-gnu %s -fopenmp-dump-offload-linker-script -no-canonical-prefixes 2>&1 \
-// RUN:   | FileCheck -check-prefix=CHK-COMMANDS -check-prefix=CHK-LKS -check-prefix=CHK-LKS-REG %s
-// RUN:   %clang -### -fopenmp=libomp -o %t.out -target powerpc64le-linux -fopenmp-targets=powerpc64le-ibm-linux-gnu,x86_64-pc-linux-gnu %s -save-temps -fopenmp-dump-offload-linker-script -no-canonical-prefixes 2>&1 \
-// RUN:   | FileCheck -check-prefix=CHK-COMMANDS-ST -check-prefix=CHK-LKS -check-prefix=CHK-LKS-ST %s
-
-// Make sure we are not dumping the script unless the user requested it.
 // RUN:   %clang -### -fopenmp=libomp -o %t.out -target powerpc64le-linux -fopenmp-targets=powerpc64le-ibm-linux-gnu,x86_64-pc-linux-gnu %s -no-canonical-prefixes 2>&1 \
-// RUN:   | FileCheck -check-prefix=CHK-LKS-NODUMP %s
+// RUN:   | FileCheck -check-prefix=CHK-COMMANDS %s
 // RUN:   %clang -### -fopenmp=libomp -o %t.out -target powerpc64le-linux -fopenmp-targets=powerpc64le-ibm-linux-gnu,x86_64-pc-linux-gnu %s -save-temps -no-canonical-prefixes 2>&1 \
-// RUN:   | FileCheck -check-prefix=CHK-LKS-NODUMP %s
-
-//
-// Check the linker script contains what we expect.
-//
-// CHK-LKS: /*
-// CHK-LKS:                   OpenMP Offload Linker Script
-// CHK-LKS:             *** Automatically generated by Clang ***
-// CHK-LKS-NODUMP-NOT:  OpenMP Offload Linker Script.
-// CHK-LKS: */
-// CHK-LKS: TARGET(binary)
-// CHK-LKS-REG: INPUT([[T1BIN:.+\.out]])
-// CHK-LKS-REG: INPUT([[T2BIN:.+\.out]])
-// CHK-LKS-ST: INPUT([[T1BIN:.+\.out-openmp-powerpc64le-ibm-linux-gnu]])
-// CHK-LKS-ST: INPUT([[T2BIN:.+\.out-openmp-x86_64-pc-linux-gnu]])
-// CHK-LKS: SECTIONS
-// CHK-LKS: {
-// CHK-LKS:   .omp_offloading.powerpc64le-ibm-linux-gnu :
-// CHK-LKS:   ALIGN(0x10)
-// CHK-LKS:   {
-// CHK-LKS:     PROVIDE_HIDDEN(.omp_offloading.img_start.powerpc64le-ibm-linux-gnu = .);
-// CHK-LKS:     [[T1BIN]]
-// CHK-LKS:     PROVIDE_HIDDEN(.omp_offloading.img_end.powerpc64le-ibm-linux-gnu = .);
-// CHK-LKS:   }
-// CHK-LKS:   .omp_offloading.x86_64-pc-linux-gnu :
-// CHK-LKS:   ALIGN(0x10)
-// CHK-LKS:   {
-// CHK-LKS:     PROVIDE_HIDDEN(.omp_offloading.img_start.x86_64-pc-linux-gnu = .);
-// CHK-LKS:     [[T2BIN]]
-// CHK-LKS:     PROVIDE_HIDDEN(.omp_offloading.img_end.x86_64-pc-linux-gnu = .);
-// CHK-LKS:   }
-// CHK-LKS: }
-// CHK-LKS: INSERT BEFORE .data
+// RUN:   | FileCheck -check-prefix=CHK-COMMANDS-ST %s
 
 //
-// Generate host BC file.
+// Generate host BC file and host object.
 //
 // CHK-COMMANDS: clang{{.*}}" "-cc1" "-triple" "powerpc64le-unknown-linux" {{.*}}"-emit-llvm-bc"
 // CHK-COMMANDS-SAME: "-fopenmp-targets=powerpc64le-ibm-linux-gnu,x86_64-pc-linux-gnu"
 // CHK-COMMANDS-SAME: "-o" "
 // CHK-COMMANDS-SAME: [[HOSTBC:[^\\/]+\.bc]]" "-x" "c" "
 // CHK-COMMANDS-SAME: [[INPUT:[^\\/]+\.c]]"
+// CHK-COMMANDS: clang{{.*}}" "-cc1" "-triple" "powerpc64le-unknown-linux" {{.*}}"-emit-obj" {{.*}}"-fopenmp" {{.*}}"-o" "
+// CHK-COMMANDS-SAME: [[HOSTOBJ:[^\\/]+\.o]]" "-x" "ir" "{{.*}}[[HOSTBC]]"
 // CHK-COMMANDS-ST: clang{{.*}}" "-cc1" "-triple" "powerpc64le-unknown-linux" {{.*}}"-E" {{.*}}"-fopenmp" {{.*}}"-o" "
 // CHK-COMMANDS-ST-SAME: [[HOSTPP:[^\\/]+\.i]]" "-x" "c" "
 // CHK-COMMANDS-ST-SAME: [[INPUT:[^\\/]+\.c]]"
 // CHK-COMMANDS-ST: clang{{.*}}" "-cc1" "-triple" "powerpc64le-unknown-linux" {{.*}}"-emit-llvm-bc" {{.*}}"-fopenmp" {{.*}}"-fopenmp-targets=powerpc64le-ibm-linux-gnu,x86_64-pc-linux-gnu" {{.*}}"-o" "
 // CHK-COMMANDS-ST-SAME: [[HOSTBC:[^\\/]+\.bc]]" "-x" "cpp-output" "{{.*}}[[HOSTPP]]"
+// CHK-COMMANDS-ST: clang{{.*}}" "-cc1" "-triple" "powerpc64le-unknown-linux" {{.*}}"-S" {{.*}}"-fopenmp" {{.*}}"-o" "
+// CHK-COMMANDS-ST-SAME: [[HOSTASM:[^\\/]+\.s]]" "-x" "ir" "{{.*}}[[HOSTBC]]"
+// CHK-COMMANDS-ST: clang{{.*}}" "-cc1as" "-triple" "powerpc64le-unknown-linux" "-filetype" "obj" {{.*}}"-o" "
+// CHK-COMMANDS-ST-SAME: [[HOSTOBJ:[^\\/]+\.o]]" "{{.*}}[[HOSTASM]]"
 
 //
 // Compile for the powerpc device.
 //
 // CHK-COMMANDS: clang{{.*}}" "-cc1" "-triple" "powerpc64le-ibm-linux-gnu" "-aux-triple" "powerpc64le-unknown-linux" {{.*}}"-emit-obj" {{.*}}"-pic-level" "2" {{.*}}"-fopenmp"
 // CHK-COMMANDS-SAME: "-fopenmp-is-device" "-fopenmp-host-ir-file-path" "{{.*}}[[HOSTBC]]" {{.*}}"-o" "
 // CHK-COMMANDS-SAME: [[T1OBJ:[^\\/]+\.o]]" "-x" "c" "{{.*}}[[INPUT]]"
 // CHK-COMMANDS: ld{{(\.exe)?}}" {{.*}}"-o" "

 // CHK-COMMANDS-ST: clang{{.*}}" "-cc1" "-triple" "x86_64-pc-linux-gnu" "-aux-triple" "powerpc64le-unknown-linux" {{.*}}"-S" {{.*}}"-fopenmp" {{.*}}"-o" "
 // CHK-COMMANDS-ST-SAME: [[T2ASM:[^\\/]+\.s]]" "-x" "ir" "{{.*}}[[T2BC]]"
 // CHK-COMMANDS-ST: clang{{.*}}" "-cc1as" "-triple" "x86_64-pc-linux-gnu" "-filetype" "obj" {{.*}}"-o" "
 // CHK-COMMANDS-ST-SAME: [[T2OBJ:[^\\/]+\.o]]" "{{.*}}[[T2ASM]]"
 // CHK-COMMANDS-ST: ld{{(\.exe)?}}" {{.*}}"-shared" {{.*}}"-o" "
 // CHK-COMMANDS-ST-SAME: [[T2BIN:[^\\/]+\.out-openmp-x86_64-pc-linux-gnu]]" {{.*}}"{{.*}}[[T2OBJ]]"
 
 //
-// Generate host object from the BC file and link using the linker script.
+// Create wrapper BC file and wrapper object.
 //
+// CHK-COMMANDS: clang-offload-wrapper{{(\.exe)?}}" "-target" "powerpc64le-unknown-linux" {{.*}}"-o" "
+// CHK-COMMANDS-SAME: [[WRAPPERBC:[^\\/]+\.bc]]" "{{.*}}[[T1BIN]]" "{{.*}}[[T2BIN]]"
 // CHK-COMMANDS: clang{{.*}}" "-cc1" "-triple" "powerpc64le-unknown-linux" {{.*}}"-emit-obj" {{.*}}"-fopenmp" {{.*}}"-o" "
-// CHK-COMMANDS-SAME: [[HOSTOBJ:[^\\/]+\.o]]" "-x" "ir" "{{.*}}[[HOSTBC]]"
-// CHK-COMMANDS: ld{{(\.exe)?}}" {{.*}}"-o" "
-// CHK-COMMANDS-SAME: [[HOSTBIN:[^\\/]+\.out]]"  {{.*}}"-lomptarget" {{.*}}"-T" "
-// CHK-COMMANDS-SAME: [[HOSTLK:[^\\/]+\.lk]]"
+// CHK-COMMANDS-SAME: [[WRAPPEROBJ:[^\\/]+\.o]]" "-x" "ir" "{{.*}}[[WRAPPERBC]]"
+// CHK-COMMANDS-ST: clang-offload-wrapper{{(\.exe)?}}" "-target" "powerpc64le-unknown-linux" {{.*}}"-o" "
+// CHK-COMMANDS-ST-SAME: [[WRAPPERBC:[^\\/]+\.bc]]" "{{.*}}[[T1BIN]]" "{{.*}}[[T2BIN]]"
 // CHK-COMMANDS-ST: clang{{.*}}" "-cc1" "-triple" "powerpc64le-unknown-linux" {{.*}}"-S" {{.*}}"-fopenmp" {{.*}}"-o" "
-// CHK-COMMANDS-ST-SAME: [[HOSTASM:[^\\/]+\.s]]" "-x" "ir" "{{.*}}[[HOSTBC]]"
+// CHK-COMMANDS-ST-SAME: [[WRAPPERASM:[^\\/]+\.s]]" "-x" "ir" "{{.*}}[[WRAPPERBC]]"
 // CHK-COMMANDS-ST: clang{{.*}}" "-cc1as" "-triple" "powerpc64le-unknown-linux" "-filetype" "obj" {{.*}}"-o" "
-// CHK-COMMANDS-ST-SAME: [[HOSTOBJ:[^\\/]+\.o]]" "{{.*}}[[HOSTASM]]"
-// CHK-COMMANDS-ST: ld{{(\.exe)?}}" {{.*}}"-o" "
-// CHK-COMMANDS-ST-SAME: [[HOSTBIN:[^\\/]+\.out]]"  {{.*}}"-lomptarget" {{.*}}"-T" "
-// CHK-COMMANDS-ST-SAME: [[HOSTLK:[^\\/]+\.lk]]"
+// CHK-COMMANDS-ST-SAME: [[WRAPPEROBJ:[^\\/]+\.o]]" "{{.*}}[[WRAPPERASM]]"
 
+//
+// Link host binary.
+//
+// CHK-COMMANDS: ld{{(\.exe)?}}" {{.*}}"-o" "
+// CHK-COMMANDS-SAME: [[HOSTBIN:[^\\/]+\.out]]" {{.*}}"{{.*}}[[HOSTOBJ]]" "{{.*}}[[WRAPPEROBJ]]" {{.*}}"-lomptarget"
+// CHK-COMMANDS-ST: ld{{(\.exe)?}}" {{.*}}"-o" "
+// CHK-COMMANDS-ST-SAME: [[HOSTBIN:[^\\/]+\.out]]" {{.*}}"{{.*}}[[HOSTOBJ]]" "{{.*}}[[WRAPPEROBJ]]" {{.*}}"-lomptarget"
 
 /// ###########################################################################
 
 /// Check separate compilation with offloading - bundling actions
 // RUN:   %clang -### -ccc-print-phases -fopenmp=libomp -c -o %t.o %S/Input/in.so -lsomelib -target powerpc64le-linux -fopenmp-targets=powerpc64le-ibm-linux-gnu,x86_64-pc-linux-gnu %s -no-canonical-prefixes 2>&1 \
 // RUN:   | FileCheck -check-prefix=CHK-BUACTIONS %s
 
 // CHK-BUACTIONS: 0: input, "[[INPUT:.+\.c]]", c, (host-openmp)

 // RUN:   | FileCheck -check-prefix=CHK-UBACTIONS %s
 
 // CHK-UBACTIONS: 0: input, "somelib", object, (host-openmp)
 // CHK-UBACTIONS: 1: input, "[[INPUT:.+\.i]]", cpp-output, (host-openmp)
 // CHK-UBACTIONS: 2: clang-offload-unbundler, {1}, cpp-output, (host-openmp)
 // CHK-UBACTIONS: 3: compiler, {2}, ir, (host-openmp)
 // CHK-UBACTIONS: 4: backend, {3}, assembler, (host-openmp)
 // CHK-UBACTIONS: 5: assembler, {4}, object, (host-openmp)
-// CHK-UBACTIONS: 6: linker, {0, 5}, image, (host-openmp)
-// CHK-UBACTIONS: 7: input, "somelib", object, (device-openmp)
-// CHK-UBACTIONS: 8: compiler, {2}, ir, (device-openmp)
-// CHK-UBACTIONS: 9: offload, "host-openmp (powerpc64le-unknown-linux)" {3}, "device-openmp (powerpc64le-ibm-linux-gnu)" {8}, ir
-// CHK-UBACTIONS: 10: backend, {9}, assembler, (device-openmp)
-// CHK-UBACTIONS: 11: assembler, {10}, object, (device-openmp)
-// CHK-UBACTIONS: 12: linker, {7, 11}, image, (device-openmp)
+// CHK-UBACTIONS: 6: input, "somelib", object, (device-openmp)
+// CHK-UBACTIONS: 7: compiler, {2}, ir, (device-openmp)
+// CHK-UBACTIONS: 8: offload, "host-openmp (powerpc64le-unknown-linux)" {3}, "device-openmp (powerpc64le-ibm-linux-gnu)" {7}, ir
+// CHK-UBACTIONS: 9: backend, {8}, assembler, (device-openmp)
+// CHK-UBACTIONS: 10: assembler, {9}, object, (device-openmp)
+// CHK-UBACTIONS: 11: linker, {6, 10}, image, (device-openmp)
+// CHK-UBACTIONS: 12: offload, "device-openmp (powerpc64le-ibm-linux-gnu)" {11}, image
 // CHK-UBACTIONS: 13: input, "somelib", object, (device-openmp)
 // CHK-UBACTIONS: 14: compiler, {2}, ir, (device-openmp)
 // CHK-UBACTIONS: 15: offload, "host-openmp (powerpc64le-unknown-linux)" {3}, "device-openmp (x86_64-pc-linux-gnu)" {14}, ir
 // CHK-UBACTIONS: 16: backend, {15}, assembler, (device-openmp)
 // CHK-UBACTIONS: 17: assembler, {16}, object, (device-openmp)
 // CHK-UBACTIONS: 18: linker, {13, 17}, image, (device-openmp)
-// CHK-UBACTIONS: 19: offload, "host-openmp (powerpc64le-unknown-linux)" {6}, "device-openmp (powerpc64le-ibm-linux-gnu)" {12}, "device-openmp (x86_64-pc-linux-gnu)" {18}, image
+// CHK-UBACTIONS: 19: offload, "device-openmp (x86_64-pc-linux-gnu)" {18}, image
+// CHK-UBACTIONS: 20: clang-offload-wrapper, {12, 19}, ir, (host-openmp)
+// CHK-UBACTIONS: 21: backend, {20}, assembler, (host-openmp)
+// CHK-UBACTIONS: 22: assembler, {21}, object, (host-openmp)
+// CHK-UBACTIONS: 23: linker, {0, 5, 22}, image, (host-openmp)
 
 /// ###########################################################################
 
 /// Check separate compilation with offloading - unbundling/bundling actions
 // RUN:   touch %t.i
 // RUN:   %clang -### -ccc-print-phases -fopenmp=libomp -c -o %t.o -lsomelib -target powerpc64le-linux -fopenmp-targets=powerpc64le-ibm-linux-gnu,x86_64-pc-linux-gnu %t.i -no-canonical-prefixes 2>&1 \
 // RUN:   | FileCheck -check-prefix=CHK-UBUACTIONS %s
 

 // Unbundle and create host BC.
 // CHK-UBJOBS: clang-offload-bundler{{.*}}" "-type=i" "-targets=host-powerpc64le-unknown-linux,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu" "-inputs=
 // CHK-UBJOBS-SAME: [[INPUT:[^\\/]+\.i]]" "-outputs=
 // CHK-UBJOBS-SAME: [[HOSTPP:[^\\/]+\.i]],
 // CHK-UBJOBS-SAME: [[T1PP:[^\\/]+\.i]],
 // CHK-UBJOBS-SAME: [[T2PP:[^\\/]+\.i]]" "-unbundle"
 // CHK-UBJOBS: clang{{.*}}" "-cc1" "-triple" "powerpc64le-unknown-linux" {{.*}}"-emit-llvm-bc"  {{.*}}"-fopenmp" {{.*}}"-fopenmp-targets=powerpc64le-ibm-linux-gnu,x86_64-pc-linux-gnu" {{.*}}"-o" "
 // CHK-UBJOBS-SAME: [[HOSTBC:[^\\/]+\.bc]]" "-x" "cpp-output" "{{.*}}[[HOSTPP]]"
+// CHK-UBJOBS: clang{{.*}}" "-cc1" "-triple" "powerpc64le-unknown-linux" {{.*}}"-emit-obj" {{.*}}"-fopenmp" {{.*}}"-o" "
+// CHK-UBJOBS-SAME: [[HOSTOBJ:[^\\/]+\.o]]" "-x" "ir" "{{.*}}[[HOSTBC]]"
 // CHK-UBJOBS-ST: clang-offload-bundler{{.*}}" "-type=i" "-targets=host-powerpc64le-unknown-linux,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu" "-inputs=
 // CHK-UBJOBS-ST-SAME: [[INPUT:[^\\/]+\.i]]" "-outputs=
 // CHK-UBJOBS-ST-SAME: [[HOSTPP:[^\\/,]+\.i]],
 // CHK-UBJOBS-ST-SAME: [[T1PP:[^\\/,]+\.i]],
 // CHK-UBJOBS-ST-SAME: [[T2PP:[^\\/,]+\.i]]" "-unbundle"
 // CHK-UBJOBS-ST: clang{{.*}}" "-cc1" "-triple" "powerpc64le-unknown-linux" {{.*}}"-emit-llvm-bc"  {{.*}}"-fopenmp" {{.*}}"-fopenmp-targets=powerpc64le-ibm-linux-gnu,x86_64-pc-linux-gnu" {{.*}}"-o" "
 // CHK-UBJOBS-ST-SAME: [[HOSTBC:[^\\/]+\.bc]]" "-x" "cpp-output" "{{.*}}[[HOSTPP]]"
+// CHK-UBJOBS-ST: clang{{.*}}" "-cc1" "-triple" "powerpc64le-unknown-linux" {{.*}}"-S" {{.*}}"-fopenmp" {{.*}}"-o" "
+// CHK-UBJOBS-ST-SAME: [[HOSTASM:[^\\/]+\.s]]" "-x" "ir" "{{.*}}[[HOSTBC]]"
+// CHK-UBJOBS-ST: clang{{.*}}" "-cc1as" "-triple" "powerpc64le-unknown-linux" "-filetype" "obj" {{.*}}"-o" "
+// CHK-UBJOBS-ST-SAME: [[HOSTOBJ:[^\\/]+\.o]]" "{{.*}}[[HOSTASM]]"
 
 // Create target 1 object.
 // CHK-UBJOBS: clang{{.*}}" "-cc1" "-triple" "powerpc64le-ibm-linux-gnu" "-aux-triple" "powerpc64le-unknown-linux" {{.*}}"-emit-obj" {{.*}}"-fopenmp" {{.*}}"-fopenmp-is-device" "-fopenmp-host-ir-file-path" "{{.*}}[[HOSTBC]]" {{.*}}"-o" "
 // CHK-UBJOBS-SAME: [[T1OBJ:[^\\/]+\.o]]" "-x" "cpp-output" "{{.*}}[[T1PP]]"
 // CHK-UBJOBS: ld{{(\.exe)?}}" {{.*}}"-o" "
 // CHK-UBJOBS-SAME: [[T1BIN:[^\\/]+\.out]]" {{.*}}"{{.*}}[[T1OBJ]]"
 // CHK-UBJOBS-ST: clang{{.*}}" "-cc1" "-triple" "powerpc64le-ibm-linux-gnu" "-aux-triple" "powerpc64le-unknown-linux" {{.*}}"-emit-llvm-bc" {{.*}}"-fopenmp" {{.*}}"-fopenmp-is-device" "-fopenmp-host-ir-file-path" "{{.*}}[[HOSTBC]]" {{.*}}"-o" "
 // CHK-UBJOBS-ST-SAME: [[T1BC:[^\\/]+\.bc]]" "-x" "cpp-output" "{{.*}}[[T1PP]]"

 // CHK-UBJOBS-ST-SAME: [[T2BC:[^\\/]+\.bc]]" "-x" "cpp-output" "{{.*}}[[T2PP]]"
 // CHK-UBJOBS-ST: clang{{.*}}" "-cc1" "-triple" "x86_64-pc-linux-gnu" "-aux-triple" "powerpc64le-unknown-linux" {{.*}}"-S" {{.*}}"-fopenmp" {{.*}}"-o" "
 // CHK-UBJOBS-ST-SAME: [[T2ASM:[^\\/]+\.s]]" "-x" "ir" "{{.*}}[[T2BC]]"
 // CHK-UBJOBS-ST: clang{{.*}}" "-cc1as" "-triple" "x86_64-pc-linux-gnu" "-filetype" "obj" {{.*}}"-o" "
 // CHK-UBJOBS-ST-SAME: [[T2OBJ:[^\\/]+\.o]]" "{{.*}}[[T2ASM]]"
 // CHK-UBJOBS-ST: ld{{(\.exe)?}}" {{.*}}"-o" "
 // CHK-UBJOBS-ST-SAME: [[T2BIN:[^\\/]+\.out-openmp-x86_64-pc-linux-gnu]]" {{.*}}"{{.*}}[[T2OBJ]]"
 
-// Create binary.
+// Create wrapper BC file and wrapper object.
+// CHK-UBJOBS: clang-offload-wrapper{{(\.exe)?}}" "-target" "powerpc64le-unknown-linux" {{.*}}"-o" "
+// CHK-UBJOBS-SAME: [[WRAPPERBC:[^\\/]+\.bc]]" "{{.*}}[[T1BIN]]" "{{.*}}[[T2BIN]]"
 // CHK-UBJOBS: clang{{.*}}" "-cc1" "-triple" "powerpc64le-unknown-linux" {{.*}}"-emit-obj" {{.*}}"-fopenmp" {{.*}}"-o" "
-// CHK-UBJOBS-SAME: [[HOSTOBJ:[^\\/]+\.o]]" "-x" "ir" "{{.*}}[[HOSTBC]]"
-// CHK-UBJOBS: ld{{(\.exe)?}}" {{.*}}"-o" "
-// CHK-UBJOBS-SAME: [[HOSTBIN:[^\\/]+\.out]]" {{.*}}"{{.*}}[[HOSTOBJ]]" {{.*}}"-T" "
-// CHK-UBJOBS-SAME: [[LKS:[^\\/]+\.lk]]"
+// CHK-UBJOBS-SAME: [[WRAPPEROBJ:[^\\/]+\.o]]" "-x" "ir" "{{.*}}[[WRAPPERBC]]"
+// CHK-UBJOBS-ST: clang-offload-wrapper{{(\.exe)?}}" "-target" "powerpc64le-unknown-linux" {{.*}}"-o" "
+// CHK-UBJOBS-ST-SAME: [[WRAPPERBC:[^\\/]+\.bc]]" "{{.*}}[[T1BIN]]" "{{.*}}[[T2BIN]]"
 // CHK-UBJOBS-ST: clang{{.*}}" "-cc1" "-triple" "powerpc64le-unknown-linux" {{.*}}"-S" {{.*}}"-fopenmp" {{.*}}"-o" "
-// CHK-UBJOBS-ST-SAME: [[HOSTASM:[^\\/]+\.s]]" "-x" "ir" "{{.*}}[[HOSTBC]]"
+// CHK-UBJOBS-ST-SAME: [[WRAPPERASM:[^\\/]+\.s]]" "-x" "ir" "{{.*}}[[WRAPPERBC]]"
 // CHK-UBJOBS-ST: clang{{.*}}" "-cc1as" "-triple" "powerpc64le-unknown-linux" "-filetype" "obj" {{.*}}"-o" "
-// CHK-UBJOBS-ST-SAME: [[HOSTOBJ:[^\\/]+\.o]]" "{{.*}}[[HOSTASM]]"
+// CHK-UBJOBS-ST-SAME: [[WRAPPEROBJ:[^\\/]+\.o]]" "{{.*}}[[WRAPPERASM]]"
+
+// Create binary.
+// CHK-UBJOBS: ld{{(\.exe)?}}" {{.*}}"-o" "
+// CHK-UBJOBS-SAME: [[HOSTBIN:[^\\/]+\.out]]" {{.*}}"{{.*}}[[HOSTOBJ]]" "{{.*}}[[WRAPPEROBJ]]"
 // CHK-UBJOBS-ST: ld{{(\.exe)?}}" {{.*}}"-o" "
-// CHK-UBJOBS-ST-SAME: [[HOSTBIN:[^\\/]+\.out]]" {{.*}}"{{.*}}[[HOSTOBJ]]" {{.*}}"-T" "
-// CHK-UBJOBS-ST-SAME: [[LKS:[^\\/]+\.lk]]"
+// CHK-UBJOBS-ST-SAME: [[HOSTBIN:[^\\/]+\.out]]" {{.*}}"{{.*}}[[HOSTOBJ]]" "{{.*}}[[WRAPPEROBJ]]"
 
 // Unbundle object file.
 // CHK-UBJOBS2: clang-offload-bundler{{.*}}" "-type=o" "-targets=host-powerpc64le-unknown-linux,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu" "-inputs=
 // CHK-UBJOBS2-SAME: [[INPUT:[^\\/]+\.o]]" "-outputs=
 // CHK-UBJOBS2-SAME: [[HOSTOBJ:[^\\/]+\.o]],
 // CHK-UBJOBS2-SAME: [[T1OBJ:[^\\/]+\.o]],
 // CHK-UBJOBS2-SAME: [[T2OBJ:[^\\/]+\.o]]" "-unbundle"
 // CHK-UBJOBS2: ld{{(\.exe)?}}" {{.*}}"-o" "
 // CHK-UBJOBS2-SAME: [[T1BIN:[^\\/]+\.out]]" {{.*}}"{{.*}}[[T1OBJ]]"
 // CHK-UBJOBS2: ld{{(\.exe)?}}" {{.*}}"-o" "
 // CHK-UBJOBS2-SAME: [[T2BIN:[^\\/]+\.out]]" {{.*}}"{{.*}}[[T2OBJ]]"
+// CHK-UBJOBS2: clang-offload-wrapper{{(\.exe)?}}" "-target" "powerpc64le-unknown-linux" {{.*}}"-o" "
+// CHK-UBJOBS2-SAME: [[WRAPPERBC:[^\\/]+\.bc]]" "{{.*}}[[T1BIN]]" "{{.*}}[[T2BIN]]"
+// CHK-UBJOBS2: clang{{.*}}" "-cc1" "-triple" "powerpc64le-unknown-linux" "-emit-obj" {{.*}}"-fopenmp" {{.*}}"-o" "
+// CHK-UBJOBS2-SAME: [[WRAPPEROBJ:[^\\/]+\.o]]" "-x" "ir" "{{.*}}[[WRAPPERBC]]"
 // CHK-UBJOBS2: ld{{(\.exe)?}}" {{.*}}"-o" "
-// CHK-UBJOBS2-SAME: [[HOSTBIN:[^\\/]+\.out]]" {{.*}}"{{.*}}[[HOSTOBJ]]" {{.*}}"-T" "
-// CHK-UBJOBS2-SAME: [[LKS:[^\\/]+\.lk]]"
+// CHK-UBJOBS2-SAME: [[HOSTBIN:[^\\/]+\.out]]" {{.*}}"{{.*}}[[HOSTOBJ]]" "{{.*}}[[WRAPPEROBJ]]"
 // CHK-UBJOBS2-ST-NOT: clang-offload-bundler{{.*}}in.so
 // CHK-UBJOBS2-ST: clang-offload-bundler{{.*}}" "-type=o" "-targets=host-powerpc64le-unknown-linux,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu" "-inputs=
 // CHK-UBJOBS2-ST-SAME: [[INPUT:[^\\/]+\.o]]" "-outputs=
 // CHK-UBJOBS2-ST-SAME: [[HOSTOBJ:[^\\/,]+\.o]],
 // CHK-UBJOBS2-ST-SAME: [[T1OBJ:[^\\/,]+\.o]],
 // CHK-UBJOBS2-ST-SAME: [[T2OBJ:[^\\/,]+\.o]]" "-unbundle"
 // CHK-UBJOBS2-ST-NOT: clang-offload-bundler{{.*}}in.so
 // CHK-UBJOBS2-ST: ld{{(\.exe)?}}" {{.*}}"-o" "
 // CHK-UBJOBS2-ST-SAME: [[T1BIN:[^\\/]+\.out-openmp-powerpc64le-ibm-linux-gnu]]" {{.*}}"{{.*}}[[T1OBJ]]"
 // CHK-UBJOBS2-ST: ld{{(\.exe)?}}" {{.*}}"-o" "
 // CHK-UBJOBS2-ST-SAME: [[T2BIN:[^\\/]+\.out-openmp-x86_64-pc-linux-gnu]]" {{.*}}"{{.*}}[[T2OBJ]]"
+// CHK-UBJOBS2-ST: clang-offload-wrapper{{(\.exe)?}}" "-target" "powerpc64le-unknown-linux" {{.*}}"-o" "
+// CHK-UBJOBS2-ST-SAME: [[WRAPPERBC:[^\\/]+\.bc]]" "{{.*}}[[T1BIN]]" "{{.*}}[[T2BIN]]"
+// CHK-UBJOBS2-ST: clang{{.*}}" "-cc1" "-triple" "powerpc64le-unknown-linux" "-S" {{.*}}"-fopenmp" {{.*}}"-o" "
+// CHK-UBJOBS2-ST-SAME: [[WRAPPERASM:[^\\/]+\.s]]" "-x" "ir" "{{.*}}[[WRAPPERBC]]"
+// CHK-UBJOBS2-ST: clang{{.*}}" "-cc1as" "-triple" "powerpc64le-unknown-linux" "-filetype" "obj" {{.*}}"-o" "
+// CHK-UBJOBS2-ST-SAME: [[WRAPPEROBJ:[^\\/]+\.o]]" "{{.*}}[[WRAPPERASM]]"
 // CHK-UBJOBS2-ST: ld{{(\.exe)?}}" {{.*}}"-o" "
-// CHK-UBJOBS2-ST-SAME: [[HOSTBIN:[^\\/]+\.out]]" {{.*}}"{{.*}}[[HOSTOBJ]]" {{.*}}"-T" "
-// CHK-UBJOBS2-ST-SAME: [[LKS:[^\\/]+\.lk]]"
+// CHK-UBJOBS2-ST-SAME: [[HOSTBIN:[^\\/]+\.out]]" {{.*}}"{{.*}}[[HOSTOBJ]]" "{{.*}}[[WRAPPEROBJ]]"
 
 /// ###########################################################################
 
 /// Check separate compilation with offloading - unbundling/bundling jobs
 /// construct
 // RUN:   touch %t.i
 // RUN:   %clang -### -fopenmp=libomp -c %t.o -lsomelib -target powerpc64le-linux -fopenmp-targets=powerpc64le-ibm-linux-gnu,x86_64-pc-linux-gnu %t.i -no-canonical-prefixes 2>&1 \
 // RUN:   | FileCheck -check-prefix=CHK-UBUJOBS %s

clang/tools/CMakeLists.txt

 create_subdirectory_options(CLANG TOOL)
 
 add_clang_subdirectory(diagtool)
 add_clang_subdirectory(driver)
 add_clang_subdirectory(clang-diff)
 add_clang_subdirectory(clang-format)
 add_clang_subdirectory(clang-format-vs)
 add_clang_subdirectory(clang-fuzzer)
 add_clang_subdirectory(clang-import-test)
 add_clang_subdirectory(clang-offload-bundler)
+add_clang_subdirectory(clang-offload-wrapper)
 add_clang_subdirectory(clang-scan-deps)
 
 add_clang_subdirectory(c-index-test)
 
 add_clang_subdirectory(clang-rename)
 add_clang_subdirectory(clang-refactor)
 if(UNIX)
   add_clang_subdirectory(clang-shlib)

clang/tools/clang-offload-wrapper/CMakeLists.txt

+set(LLVM_LINK_COMPONENTS BitWriter Core Support TransformUtils)
+
+if(NOT CLANG_BUILT_STANDALONE)
+  set(tablegen_deps intrinsics_gen)
+endif()
+
+add_clang_executable(clang-offload-wrapper
+  ClangOffloadWrapper.cpp
+
+  DEPENDS
+  ${tablegen_deps}
+  )
+
+set(CLANG_OFFLOAD_WRAPPER_LIB_DEPS
+  clangBasic
+  )
+
+add_dependencies(clang clang-offload-wrapper)
+
+target_link_libraries(clang-offload-wrapper
+  PRIVATE
+  ${CLANG_OFFLOAD_WRAPPER_LIB_DEPS}
+  )
+
+install(TARGETS clang-offload-wrapper RUNTIME DESTINATION bin)

clang/tools/clang-offload-wrapper/ClangOffloadWrapper.cpp

+//===-- clang-offload-wrapper/ClangOffloadWrapper.cpp -----------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// Implementation of the offload wrapper tool. It takes offload target binaries
+/// as input and creates wrapper bitcode file containing target binaries
+/// packaged as data.
+///
+//===----------------------------------------------------------------------===//
+
+#include "clang/Basic/Version.h"
+#include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/Triple.h"
+#include "llvm/Bitcode/BitcodeWriter.h"
+#include "llvm/IR/Constants.h"
+#include "llvm/IR/GlobalVariable.h"
+#include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/Module.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/Errc.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/ErrorOr.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Signals.h"
+#include "llvm/Support/StringSaver.h"
+#include "llvm/Support/ToolOutputFile.h"
+#include "llvm/Support/WithColor.h"
+#include "llvm/Support/raw_ostream.h"
+#include <cassert>
+#include <cstdint>
+
+using namespace llvm;
+
+static cl::opt<bool> Help("h", cl::desc("Alias for -help"), cl::Hidden);
+
+// Mark all our options with this category, everything else (except for -version
+// and -help) will be hidden.
+static cl::OptionCategory
+    ClangOffloadWrapperCategory("clang-offload-wrapper options");
+
+static cl::opt<std::string> Output("o", cl::Required,
+                                   cl::desc("Output filename"),
+                                   cl::value_desc("filename"),
+                                   cl::cat(ClangOffloadWrapperCategory));
+
+static cl::list<std::string> Inputs(cl::Positional, cl::OneOrMore,
+                                    cl::desc("<input files>"),
+                                    cl::cat(ClangOffloadWrapperCategory));
+
+static cl::opt<std::string>
+    Target("target", cl::Required,
+           cl::desc("Target triple for the output module"),
+           cl::value_desc("triple"), cl::cat(ClangOffloadWrapperCategory));
+
+static cl::list<std::string>
+    OffloadTargets("offload-targets", cl::CommaSeparated, cl::OneOrMore,
+                   cl::desc("Comma-separated list of device target triples"),
+                   cl::value_desc("triples"),
+                   cl::cat(ClangOffloadWrapperCategory));
+
+namespace {
+
+class BinaryWrapper {
+public:
+  // Binary descriptor. The first field is the a reference to the binary bits,
+  // and the second is the target triple the binary was built for.
+  using BinaryDesc = std::pair<ArrayRef<char>, StringRef>;
+
+private:
+  LLVMContext C;
+  Module M;
+
+  // Saver for generated strings.
+  BumpPtrAllocator Alloc;
+  UniqueStringSaver SS;
+
+private:
+  void createImages(ArrayRef<BinaryDesc> Binaries) {
+    for (const BinaryDesc &Bin : Binaries) {
+      StringRef SectionName = SS.save(".omp_offloading." + Bin.second);
+
+      auto *DataC = ConstantDataArray::get(C, Bin.first);
+      auto *ImageB =
+          new GlobalVariable(M, DataC->getType(), /*isConstant=*/true,
+                             GlobalVariable::ExternalLinkage, DataC,
+                             ".omp_offloading.img_start." + Bin.second);
+      ImageB->setSection(SectionName);
+      ImageB->setUnnamedAddr(GlobalValue::UnnamedAddr::Global);
+      ImageB->setVisibility(llvm::GlobalValue::HiddenVisibility);
+
+      auto *EmptyC =
+          ConstantAggregateZero::get(ArrayType::get(Type::getInt8Ty(C), 0u));
+      auto *ImageE =
+          new GlobalVariable(M, EmptyC->getType(), /*isConstant=*/true,
+                             GlobalVariable::ExternalLinkage, EmptyC,
+                             ".omp_offloading.img_end." + Bin.second);
+      ImageE->setSection(SectionName);
+      ImageE->setUnnamedAddr(GlobalValue::UnnamedAddr::Global);
+      ImageE->setVisibility(GlobalValue::HiddenVisibility);
+    }
+  }
+
+public:
+  BinaryWrapper(StringRef Target) : M("offload.wrapper.object", C), SS(Alloc) {
+    M.setTargetTriple(Target);
+  }
+
+  const Module &wrapBinaries(ArrayRef<BinaryDesc> Binaries) {
+    createImages(Binaries);
+    return M;
+  }
+};
+
+} // anonymous namespace
+
+int main(int argc, const char **argv) {
+  sys::PrintStackTraceOnErrorSignal(argv[0]);
+
+  cl::HideUnrelatedOptions(ClangOffloadWrapperCategory);
+  cl::SetVersionPrinter([](raw_ostream &OS) {
+    OS << clang::getClangToolFullVersion("clang-offload-wrapper") << '\n';
+  });
+  cl::ParseCommandLineOptions(
+      argc, argv,
+      "A tool to create a wrapper bitcode for offload target binaries. Takes "
+      "offload\ntarget binaries as input and produces bitcode file containing "
+      "target binaries packaged\nas data.\n");
+
+  if (Help) {
+    cl::PrintHelpMessage();
+    return 0;
+  }
+
+  auto reportError = [argv](Error E) {
+    logAllUnhandledErrors(std::move(E), WithColor::error(errs(), argv[0]));
+  };
+
+  if (Triple(Target).getArch() == Triple::UnknownArch) {
+    reportError(createStringError(
+        errc::invalid_argument, "'" + Target + "': unsupported target triple"));
+    return 1;
+  }
+
+  if (Inputs.size() != OffloadTargets.size()) {
+    reportError(createStringError(
+        errc::invalid_argument,
+        "number of input files and offload targets should match"));
+    return 1;
+  }
+
+  // Read device binaries.
+  SmallVector<std::unique_ptr<MemoryBuffer>, 4u> Buffers;
+  SmallVector<BinaryWrapper::BinaryDesc, 4u> Images;
+  Buffers.reserve(Inputs.size());
+  Images.reserve(Inputs.size());
+  for (unsigned I = 0; I < Inputs.size(); ++I) {
+    const std::string &File = Inputs[I];
+    ErrorOr<std::unique_ptr<MemoryBuffer>> BufOrErr =
+        MemoryBuffer::getFileOrSTDIN(File);
+    if (!BufOrErr) {
+      reportError(createFileError(File, BufOrErr.getError()));
+      return 1;
+    }
+    const std::unique_ptr<MemoryBuffer> &Buf =
+        Buffers.emplace_back(std::move(*BufOrErr));
+    Images.emplace_back(
+        makeArrayRef(Buf->getBufferStart(), Buf->getBufferSize()),
+        OffloadTargets[I]);
+  }
+
+  // Create the output file to write the resulting bitcode to.
+  std::error_code EC;
+  ToolOutputFile Out(Output, EC, sys::fs::OF_None);
+  if (EC) {
+    reportError(createFileError(Output, EC));
+    return 1;
+  }
+
+  // Create a wrapper for device binaries and write its bitcode to the file.
+  WriteBitcodeToFile(BinaryWrapper(Target).wrapBinaries(
+                         makeArrayRef(Images.data(), Images.size())),
+                     Out.os());
+  if (Out.os().has_error()) {
+    reportError(createFileError(Output, Out.os().error()));
+    return 1;
+  }
+
+  // Success.
+  Out.keep();
+  return 0;
+}