[HIPSPV][1/4] Refactor HIP tool chain

[linjamaki]

Summary

This patch refactors the HIP tool chain for new HIP tool chain, HIPSPV
tool chain, which is added in the follow up patch part 2.

• Rename HIPToolChain to HIPAMDToolChain and Renames HIP.* files to HIPAMD.*.

• Introduce HIPUtility.* file where common HIP utilities, shared among HIP tool chain implementations, are placed in.

• Move constructHIPFatbinCommand() and constructGenerateObjFileFromHIPFatBinary() to HIPUtility. HIPSPV tool chain is going to use them.

• Tweak bundle target ID in constructHIPFatbinCommand(): extra dashes are dropped if the Target ID is empty and 'hip' offload kind is made default for non-AMD targets.

[linjamaki]

Style fixes.

[linjamaki]

Rebase.

[linjamaki]

Gentle ping.

[linjamaki]

Rebase.

[linjamaki]

Ping?

[yaxunl]

LGTM. Thanks.

[tra]

clang/lib/Driver/ToolChains/HIPUtility.cpp

120–134

Using MC for just wrapping a blob into an object file strikes me as something MC is not intended for.

@echristo -- is it OK to use MC as a tool in the standard compilation pipeline? I vaguely recall we had a conversaion about using objcopy for similar purposes during early days of CUDA and the conclusion was that we generally don't want that. I might be wrong, too, it's been too long ago.

[echristo]

clang/lib/Driver/ToolChains/HIPUtility.cpp

120–134

Using llvm-mc the binary? No. That's definitely not something that should be done. And as far as objcopy we spent quite a bit of time pulling it out for split dwarf and, if possible, it shouldn't be used there either.

[yaxunl]

clang/lib/Driver/ToolChains/HIPUtility.cpp

120–134

HIP toolchain has used llvm-mc to embed fat binary in a host object for a long time.

This is for supporting -fgpu-rdc since separated compilation/link will result in host object and device bit code. After device bit code are linked and fat binary are generated, fat binary needs to be embedded in some host object and linked with other host objects.

In the beginning, we used linker script. That works until we need to support windows. Then we adopted this suggestion (https://reviews.llvm.org/D46472#inline-412796) and use llvm-mc to generate a host object containing the fat binary, which works for both Linux and Windows.

I am wondering what is wrong with this approach and what is a better approach.

[linjamaki]

Thanks for the review.

[linjamaki]

Rebase.

[linjamaki]

Rebase.

[linjamaki]

Assuming this patch is ready to land. @yaxunl, Could you please commit this patch to the LLVM for us. Thanks.

[yaxunl]

In D110549#3185409, @linjamaki wrote:

Assuming this patch is ready to land. @yaxunl, Could you please commit this patch to the LLVM for us. Thanks.

I will test it with our internal CI, then commit it.

[linjamaki]

Thanks, @yaxunl.

clang/lib/Driver/CMakeLists.txt

add_clang_library(clangDriver
   ToolChains/Cuda.cpp
   ToolChains/Darwin.cpp
   ToolChains/DragonFly.cpp
   ToolChains/Flang.cpp
   ToolChains/FreeBSD.cpp
   ToolChains/Fuchsia.cpp
   ToolChains/Gnu.cpp
   ToolChains/Haiku.cpp
-  ToolChains/HIP.cpp
+  ToolChains/HIPUtility.cpp
+  ToolChains/HIPAMD.cpp
   ToolChains/Hexagon.cpp
   ToolChains/Hurd.cpp
   ToolChains/Linux.cpp
   ToolChains/MipsLinux.cpp
   ToolChains/MinGW.cpp
   ToolChains/Minix.cpp
   ToolChains/MSP430.cpp
   ToolChains/MSVC.cpp

clang/lib/Driver/Driver.cpp

 #include "ToolChains/Contiki.h"
 #include "ToolChains/CrossWindows.h"
 #include "ToolChains/Cuda.h"
 #include "ToolChains/Darwin.h"
 #include "ToolChains/DragonFly.h"
 #include "ToolChains/FreeBSD.h"
 #include "ToolChains/Fuchsia.h"
 #include "ToolChains/Gnu.h"
-#include "ToolChains/HIP.h"
+#include "ToolChains/HIPAMD.h"
 #include "ToolChains/Haiku.h"
 #include "ToolChains/Hexagon.h"
 #include "ToolChains/Hurd.h"
 #include "ToolChains/Lanai.h"
 #include "ToolChains/Linux.h"
 #include "ToolChains/MSP430.h"
 #include "ToolChains/MSVC.h"
 #include "ToolChains/MinGW.h"
if (IsCuda) {
     const ToolChain *HostTC = C.getSingleOffloadToolChain<Action::OFK_Host>();
     const llvm::Triple &HostTriple = HostTC->getTriple();
     auto OFK = Action::OFK_HIP;
     llvm::Triple HIPTriple = getHIPOffloadTargetTriple();
     // Use the HIP and host triples as the key into the ToolChains map,
     // because the device toolchain we create depends on both.
     auto &HIPTC = ToolChains[HIPTriple.str() + "/" + HostTriple.str()];
     if (!HIPTC) {
-      HIPTC = std::make_unique<toolchains::HIPToolChain>(
+      HIPTC = std::make_unique<toolchains::HIPAMDToolChain>(
           *this, HIPTriple, *HostTC, C.getInputArgs());
     }
     C.addOffloadDeviceToolChain(HIPTC.get(), OFK);
   }
 
   //
   // OpenMP
   //

clang/lib/Driver/ToolChains/Clang.cpp

if (const auto *OA = dyn_cast<OffloadAction>(CurDep)) {
         CurKind = A->getOffloadingDeviceKind();
         CurTC = TC;
       });
     }
     Triples += Action::GetOffloadKindName(CurKind);
     Triples += '-';
     Triples += CurTC->getTriple().normalize();
     if ((CurKind == Action::OFK_HIP || CurKind == Action::OFK_Cuda) &&
-        CurDep->getOffloadingArch()) {
+        !StringRef(CurDep->getOffloadingArch()).empty()) {
       Triples += '-';
       Triples += CurDep->getOffloadingArch();
     }
 
     // TODO: Replace parsing of -march flag. Can be done by storing GPUArch
     //       with each toolchain.
     StringRef GPUArchName;
     if (CurKind == Action::OFK_OpenMP) {

clang/lib/Driver/ToolChains/CommonArgs.cpp

 #include "Arch/AArch64.h"
 #include "Arch/ARM.h"
 #include "Arch/M68k.h"
 #include "Arch/Mips.h"
 #include "Arch/PPC.h"
 #include "Arch/SystemZ.h"
 #include "Arch/VE.h"
 #include "Arch/X86.h"
-#include "HIP.h"
+#include "HIPAMD.h"
 #include "Hexagon.h"
 #include "clang/Basic/CharInfo.h"
 #include "clang/Basic/LangOptions.h"
 #include "clang/Basic/ObjCRuntime.h"
 #include "clang/Basic/Version.h"
 #include "clang/Config/config.h"
 #include "clang/Driver/Action.h"
 #include "clang/Driver/Compilation.h"

clang/lib/Driver/ToolChains/HIPAMD.h

-//===--- HIP.h - HIP ToolChain Implementations ------------------*- C++ -*-===//
+//===--- HIPAMD.h - HIP ToolChain Implementations ---------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
-#ifndef LLVM_CLANG_LIB_DRIVER_TOOLCHAINS_HIP_H
-#define LLVM_CLANG_LIB_DRIVER_TOOLCHAINS_HIP_H
+#ifndef LLVM_CLANG_LIB_DRIVER_TOOLCHAINS_HIPAMD_H
+#define LLVM_CLANG_LIB_DRIVER_TOOLCHAINS_HIPAMD_H
 
-#include "clang/Driver/ToolChain.h"
-#include "clang/Driver/Tool.h"
 #include "AMDGPU.h"
+#include "clang/Driver/Tool.h"
+#include "clang/Driver/ToolChain.h"
 
 namespace clang {
 namespace driver {
 
 namespace tools {
 
 namespace AMDGCN {
-  // Construct command for creating HIP fatbin.
-  void constructHIPFatbinCommand(Compilation &C, const JobAction &JA,
-                  StringRef OutputFileName, const InputInfoList &Inputs,
-                  const llvm::opt::ArgList &TCArgs, const Tool& T);
-
 // Runs llvm-link/opt/llc/lld, which links multiple LLVM bitcode, together with
 // device library, then compiles it to ISA in a shared object.
 class LLVM_LIBRARY_VISIBILITY Linker : public Tool {
 public:
   Linker(const ToolChain &TC) : Tool("AMDGCN::Linker", "amdgcn-link", TC) {}
 
   bool hasIntegratedCPP() const override { return false; }
 
   void ConstructJob(Compilation &C, const JobAction &JA,
                     const InputInfo &Output, const InputInfoList &Inputs,
                     const llvm::opt::ArgList &TCArgs,
                     const char *LinkingOutput) const override;
 
 private:
-
   void constructLldCommand(Compilation &C, const JobAction &JA,
                            const InputInfoList &Inputs, const InputInfo &Output,
                            const llvm::opt::ArgList &Args) const;
-
-  // Construct command for creating Object from HIP fatbin.
-  void constructGenerateObjFileFromHIPFatBinary(Compilation &C,
-                                                const InputInfo &Output,
-                                                const InputInfoList &Inputs,
-                                                const llvm::opt::ArgList &Args,
-                                                const JobAction &JA) const;
 };
 
 } // end namespace AMDGCN
 } // end namespace tools
 
 namespace toolchains {
 
-class LLVM_LIBRARY_VISIBILITY HIPToolChain final : public ROCMToolChain {
+class LLVM_LIBRARY_VISIBILITY HIPAMDToolChain final : public ROCMToolChain {
 public:
-  HIPToolChain(const Driver &D, const llvm::Triple &Triple,
+  HIPAMDToolChain(const Driver &D, const llvm::Triple &Triple,
                   const ToolChain &HostTC, const llvm::opt::ArgList &Args);
 
   const llvm::Triple *getAuxTriple() const override {
     return &HostTC.getTriple();
   }
 
   llvm::opt::DerivedArgList *
   TranslateArgs(const llvm::opt::DerivedArgList &Args, StringRef BoundArch,
                 Action::OffloadKind DeviceOffloadKind) const override;
-  void addClangTargetOptions(const llvm::opt::ArgList &DriverArgs,
-                             llvm::opt::ArgStringList &CC1Args,
-                             Action::OffloadKind DeviceOffloadKind) const override;
+  void
+  addClangTargetOptions(const llvm::opt::ArgList &DriverArgs,
   void addClangWarningOptions(llvm::opt::ArgStringList &CC1Args) const override;
   CXXStdlibType GetCXXStdlibType(const llvm::opt::ArgList &Args) const override;
   void
   AddClangSystemIncludeArgs(const llvm::opt::ArgList &DriverArgs,
                             llvm::opt::ArgStringList &CC1Args) const override;
   void AddClangCXXStdlibIncludeArgs(
       const llvm::opt::ArgList &Args,
       llvm::opt::ArgStringList &CC1Args) const override;

 protected:
   Tool *buildLinker() const override;
 };
 
 } // end namespace toolchains
 } // end namespace driver
 } // end namespace clang
 
-#endif // LLVM_CLANG_LIB_DRIVER_TOOLCHAINS_HIP_H
+#endif // LLVM_CLANG_LIB_DRIVER_TOOLCHAINS_HIPAMD_H

clang/lib/Driver/ToolChains/HIPAMD.cpp

-//===--- HIP.cpp - HIP Tool and ToolChain Implementations -------*- C++ -*-===//
+//===--- HIPAMD.cpp - HIP Tool and ToolChain Implementations ----*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
-#include "HIP.h"
+#include "HIPAMD.h"
 #include "AMDGPU.h"
 #include "CommonArgs.h"
+#include "HIPUtility.h"
 #include "clang/Basic/Cuda.h"
 #include "clang/Basic/TargetID.h"
 #include "clang/Driver/Compilation.h"
 #include "clang/Driver/Driver.h"
 #include "clang/Driver/DriverDiagnostic.h"
 #include "clang/Driver/InputInfo.h"
 #include "clang/Driver/Options.h"
 #include "clang/Driver/SanitizerArgs.h"
Diags.Report(
         clang::diag::warn_drv_unsupported_option_for_offload_arch_req_feature)
         << A->getAsString(DriverArgs) << TargetID << "xnack+";
     return true;
   }
   return false;
 }
 
 void AMDGCN::Linker::constructLldCommand(Compilation &C, const JobAction &JA,
-                                          const InputInfoList &Inputs,
-                                          const InputInfo &Output,
-                                          const llvm::opt::ArgList &Args) const {
   // Construct lld command.
   // The output from ld.lld is an HSA code object file.
   ArgStringList LldArgs{"-flavor", "gnu", "--no-undefined", "-shared",
                         "-plugin-opt=-amdgpu-internalize-symbols"};
 
   auto &TC = getToolChain();
   auto &D = TC.getDriver();
   assert(!Inputs.empty() && "Must have at least one input.");
void AMDGCN::Linker::constructLldCommand(Compilation &C, const JobAction &JA,
   for (auto Input : Inputs)
     LldArgs.push_back(Input.getFilename());
 
   const char *Lld = Args.MakeArgString(getToolChain().GetProgramPath("lld"));
   C.addCommand(std::make_unique<Command>(JA, *this, ResponseFileSupport::None(),
                                          Lld, LldArgs, Inputs, Output));
 }
 
-// Construct a clang-offload-bundler command to bundle code objects for
-// different GPU's into a HIP fat binary.
-void AMDGCN::constructHIPFatbinCommand(Compilation &C, const JobAction &JA,
-                  StringRef OutputFileName, const InputInfoList &Inputs,
-                  const llvm::opt::ArgList &Args, const Tool& T) {
-  // Construct clang-offload-bundler command to bundle object files for
-  // for different GPU archs.
-  ArgStringList BundlerArgs;
-  BundlerArgs.push_back(Args.MakeArgString("-type=o"));
-  BundlerArgs.push_back(
-      Args.MakeArgString("-bundle-align=" + Twine(HIPCodeObjectAlign)));
-
-  // ToDo: Remove the dummy host binary entry which is required by
-  // clang-offload-bundler.
-  std::string BundlerTargetArg = "-targets=host-x86_64-unknown-linux";
-  std::string BundlerInputArg = "-inputs=" NULL_FILE;
-
-  // For code object version 2 and 3, the offload kind in bundle ID is 'hip'
-  // for backward compatibility. For code object version 4 and greater, the
-  // offload kind in bundle ID is 'hipv4'.
-  std::string OffloadKind = "hip";
-  if (getAMDGPUCodeObjectVersion(C.getDriver(), Args) >= 4)
-    OffloadKind = OffloadKind + "v4";
-  for (const auto &II : Inputs) {
-    const auto* A = II.getAction();
-    BundlerTargetArg = BundlerTargetArg + "," + OffloadKind +
-                       "-amdgcn-amd-amdhsa--" +
-                       StringRef(A->getOffloadingArch()).str();
-    BundlerInputArg = BundlerInputArg + "," + II.getFilename();
-  }
-  BundlerArgs.push_back(Args.MakeArgString(BundlerTargetArg));
-  BundlerArgs.push_back(Args.MakeArgString(BundlerInputArg));
-
-  std::string Output = std::string(OutputFileName);
-  auto BundlerOutputArg =
-      Args.MakeArgString(std::string("-outputs=").append(Output));
-  BundlerArgs.push_back(BundlerOutputArg);
-
-  const char *Bundler = Args.MakeArgString(
-      T.getToolChain().GetProgramPath("clang-offload-bundler"));
-  C.addCommand(std::make_unique<Command>(
-      JA, T, ResponseFileSupport::None(), Bundler, BundlerArgs, Inputs,
-      InputInfo(&JA, Args.MakeArgString(Output))));
-}
-
-/// Add Generated HIP Object File which has device images embedded into the
-/// host to the argument list for linking. Using MC directives, embed the
-/// device code and also define symbols required by the code generation so that
-/// the image can be retrieved at runtime.
-void AMDGCN::Linker::constructGenerateObjFileFromHIPFatBinary(
-    Compilation &C, const InputInfo &Output,
-    const InputInfoList &Inputs, const ArgList &Args,
-    const JobAction &JA) const {
-  const ToolChain &TC = getToolChain();
-  std::string Name = std::string(llvm::sys::path::stem(Output.getFilename()));
-
-  // Create Temp Object File Generator,
-  // Offload Bundled file and Bundled Object file.
-  // Keep them if save-temps is enabled.
-  const char *McinFile;
-  const char *BundleFile;
-  if (C.getDriver().isSaveTempsEnabled()) {
-    McinFile = C.getArgs().MakeArgString(Name + ".mcin");
-    BundleFile = C.getArgs().MakeArgString(Name + ".hipfb");
-  } else {
-    auto TmpNameMcin = C.getDriver().GetTemporaryPath(Name, "mcin");
-    McinFile = C.addTempFile(C.getArgs().MakeArgString(TmpNameMcin));
-    auto TmpNameFb = C.getDriver().GetTemporaryPath(Name, "hipfb");
-    BundleFile = C.addTempFile(C.getArgs().MakeArgString(TmpNameFb));
-  }
-  constructHIPFatbinCommand(C, JA, BundleFile, Inputs, Args, *this);
-
-  // Create a buffer to write the contents of the temp obj generator.
-  std::string ObjBuffer;
-  llvm::raw_string_ostream ObjStream(ObjBuffer);
-
-  auto HostTriple =
-      C.getSingleOffloadToolChain<Action::OFK_Host>()->getTriple();
-
-  // Add MC directives to embed target binaries. We ensure that each
-  // section and image is 16-byte aligned. This is not mandatory, but
-  // increases the likelihood of data to be aligned with a cache block
-  // in several main host machines.
-  ObjStream << "#       HIP Object Generator\n";
-  ObjStream << "# *** Automatically generated by Clang ***\n";
-  if (HostTriple.isWindowsMSVCEnvironment()) {
-    ObjStream << "  .section .hip_fatbin, \"dw\"\n";
-  } else {
-    ObjStream << "  .protected __hip_fatbin\n";
-    ObjStream << "  .type __hip_fatbin,@object\n";
-    ObjStream << "  .section .hip_fatbin,\"a\",@progbits\n";
-  }
-  ObjStream << "  .globl __hip_fatbin\n";
-  ObjStream << "  .p2align " << llvm::Log2(llvm::Align(HIPCodeObjectAlign))
-            << "\n";
-  ObjStream << "__hip_fatbin:\n";
-  ObjStream << "  .incbin ";
-  llvm::sys::printArg(ObjStream, BundleFile, /*Quote=*/true);
-  ObjStream << "\n";
-  ObjStream.flush();
-
-  // Dump the contents of the temp object file gen if the user requested that.
-  // We support this option to enable testing of behavior with -###.
-  if (C.getArgs().hasArg(options::OPT_fhip_dump_offload_linker_script))
-    llvm::errs() << ObjBuffer;
-
-  // Open script file and write the contents.
-  std::error_code EC;
-  llvm::raw_fd_ostream Objf(McinFile, EC, llvm::sys::fs::OF_None);
-
-  if (EC) {
-    C.getDriver().Diag(clang::diag::err_unable_to_make_temp) << EC.message();
-    return;
-  }
-
-  Objf << ObjBuffer;
-
-  ArgStringList McArgs{"-triple", Args.MakeArgString(HostTriple.normalize()),
-                       "-o",      Output.getFilename(),
-                       McinFile,  "--filetype=obj"};
-  const char *Mc = Args.MakeArgString(TC.GetProgramPath("llvm-mc"));
-  C.addCommand(std::make_unique<Command>(JA, *this, ResponseFileSupport::None(),
-                                         Mc, McArgs, Inputs, Output));
-}
-
 // For amdgcn the inputs of the linker job are device bitcode and output is
 // object file. It calls llvm-link, opt, llc, then lld steps.
 void AMDGCN::Linker::ConstructJob(Compilation &C, const JobAction &JA,
-                                   const InputInfo &Output,
-                                   const InputInfoList &Inputs,
-                                   const ArgList &Args,
-                                   const char *LinkingOutput) const {
   if (Inputs.size() > 0 &&
       Inputs[0].getType() == types::TY_Image &&
       JA.getType() == types::TY_Object)
-    return constructGenerateObjFileFromHIPFatBinary(C, Output, Inputs, Args, JA);
+    return HIP::constructGenerateObjFileFromHIPFatBinary(C, Output, Inputs,
+                                                         Args, JA, *this);
 
   if (JA.getType() == types::TY_HIP_FATBIN)
-    return constructHIPFatbinCommand(C, JA, Output.getFilename(), Inputs, Args, *this);
+    return HIP::constructHIPFatbinCommand(C, JA, Output.getFilename(), Inputs,
+                                          Args, *this);
 
   return constructLldCommand(C, JA, Inputs, Output, Args);
 }
 
-HIPToolChain::HIPToolChain(const Driver &D, const llvm::Triple &Triple,
+HIPAMDToolChain::HIPAMDToolChain(const Driver &D, const llvm::Triple &Triple,
                                  const ToolChain &HostTC, const ArgList &Args)
     : ROCMToolChain(D, Triple, Args), HostTC(HostTC) {
   // Lookup binaries into the driver directory, this is used to
   // discover the clang-offload-bundler executable.
   getProgramPaths().push_back(getDriver().Dir);
 
   // Diagnose unsupported sanitizer options only once.
   for (auto A : Args.filtered(options::OPT_fsanitize_EQ)) {
     SanitizerMask K = parseSanitizerValue(A->getValue(), /*AllowGroups=*/false);
     if (K != SanitizerKind::Address)
       D.getDiags().Report(clang::diag::warn_drv_unsupported_option_for_target)
           << A->getAsString(Args) << getTriple().str();
   }
 }
 
-void HIPToolChain::addClangTargetOptions(
-    const llvm::opt::ArgList &DriverArgs,
-    llvm::opt::ArgStringList &CC1Args,
+void HIPAMDToolChain::addClangTargetOptions(
+    const llvm::opt::ArgList &DriverArgs, llvm::opt::ArgStringList &CC1Args,
     Action::OffloadKind DeviceOffloadingKind) const {
   HostTC.addClangTargetOptions(DriverArgs, CC1Args, DeviceOffloadingKind);
 
   assert(DeviceOffloadingKind == Action::OFK_HIP &&
          "Only HIP offloading kinds are supported for GPUs.");
 
   CC1Args.push_back("-fcuda-is-device");
 
void HIPAMDToolChain::addClangTargetOptions(
   llvm::for_each(getHIPDeviceLibs(DriverArgs), [&](auto BCFile) {
     CC1Args.push_back(BCFile.ShouldInternalize ? "-mlink-builtin-bitcode"
                                                : "-mlink-bitcode-file");
     CC1Args.push_back(DriverArgs.MakeArgString(BCFile.Path));
   });
 }
 
 llvm::opt::DerivedArgList *
-HIPToolChain::TranslateArgs(const llvm::opt::DerivedArgList &Args,
+HIPAMDToolChain::TranslateArgs(const llvm::opt::DerivedArgList &Args,
                                StringRef BoundArch,
                                Action::OffloadKind DeviceOffloadKind) const {
   DerivedArgList *DAL =
       HostTC.TranslateArgs(Args, BoundArch, DeviceOffloadKind);
   if (!DAL)
     DAL = new DerivedArgList(Args.getBaseArgs());
 
   const OptTable &Opts = getDriver().getOpts();
 
   for (Arg *A : Args) {
     if (!shouldSkipArgument(A) &&
         !shouldSkipSanitizeOption(*this, Args, BoundArch, A))
       DAL->append(A);
   }
 
   if (!BoundArch.empty()) {
     DAL->eraseArg(options::OPT_mcpu_EQ);
     DAL->AddJoinedArg(nullptr, Opts.getOption(options::OPT_mcpu_EQ), BoundArch);
     checkTargetID(*DAL);
   }
 
   return DAL;
 }
 
-Tool *HIPToolChain::buildLinker() const {
+Tool *HIPAMDToolChain::buildLinker() const {
   assert(getTriple().getArch() == llvm::Triple::amdgcn);
   return new tools::AMDGCN::Linker(*this);
 }
 
-void HIPToolChain::addClangWarningOptions(ArgStringList &CC1Args) const {
+void HIPAMDToolChain::addClangWarningOptions(ArgStringList &CC1Args) const {
   HostTC.addClangWarningOptions(CC1Args);
 }
 
 ToolChain::CXXStdlibType
-HIPToolChain::GetCXXStdlibType(const ArgList &Args) const {
+HIPAMDToolChain::GetCXXStdlibType(const ArgList &Args) const {
   return HostTC.GetCXXStdlibType(Args);
 }
 
-void HIPToolChain::AddClangSystemIncludeArgs(const ArgList &DriverArgs,
+void HIPAMDToolChain::AddClangSystemIncludeArgs(const ArgList &DriverArgs,
                                                 ArgStringList &CC1Args) const {
   HostTC.AddClangSystemIncludeArgs(DriverArgs, CC1Args);
 }
 
-void HIPToolChain::AddClangCXXStdlibIncludeArgs(const ArgList &Args,
-                                                 ArgStringList &CC1Args) const {
+void HIPAMDToolChain::AddClangCXXStdlibIncludeArgs(
+    const ArgList &Args, ArgStringList &CC1Args) const {
   HostTC.AddClangCXXStdlibIncludeArgs(Args, CC1Args);
 }
 
-void HIPToolChain::AddIAMCUIncludeArgs(const ArgList &Args,
+void HIPAMDToolChain::AddIAMCUIncludeArgs(const ArgList &Args,
                                           ArgStringList &CC1Args) const {
   HostTC.AddIAMCUIncludeArgs(Args, CC1Args);
 }
 
-void HIPToolChain::AddHIPIncludeArgs(const ArgList &DriverArgs,
+void HIPAMDToolChain::AddHIPIncludeArgs(const ArgList &DriverArgs,
                                         ArgStringList &CC1Args) const {
   RocmInstallation.AddHIPIncludeArgs(DriverArgs, CC1Args);
 }
 
-SanitizerMask HIPToolChain::getSupportedSanitizers() const {
-  // The HIPToolChain only supports sanitizers in the sense that it allows
+SanitizerMask HIPAMDToolChain::getSupportedSanitizers() const {
+  // The HIPAMDToolChain only supports sanitizers in the sense that it allows
   // sanitizer arguments on the command line if they are supported by the host
-  // toolchain. The HIPToolChain will actually ignore any command line
+  // toolchain. The HIPAMDToolChain will actually ignore any command line
   // arguments for any of these "supported" sanitizers. That means that no
   // sanitization of device code is actually supported at this time.
   //
   // This behavior is necessary because the host and device toolchains
   // invocations often share the command line, so the device toolchain must
   // tolerate flags meant only for the host toolchain.
   return HostTC.getSupportedSanitizers();
 }
 
-VersionTuple HIPToolChain::computeMSVCVersion(const Driver *D,
+VersionTuple HIPAMDToolChain::computeMSVCVersion(const Driver *D,
                                                  const ArgList &Args) const {
   return HostTC.computeMSVCVersion(D, Args);
 }
 
 llvm::SmallVector<ToolChain::BitCodeLibraryInfo, 12>
-HIPToolChain::getHIPDeviceLibs(const llvm::opt::ArgList &DriverArgs) const {
+HIPAMDToolChain::getHIPDeviceLibs(const llvm::opt::ArgList &DriverArgs) const {
   llvm::SmallVector<BitCodeLibraryInfo, 12> BCLibs;
   if (DriverArgs.hasArg(options::OPT_nogpulib))
     return {};
   ArgStringList LibraryPaths;
 
   // Find in --hip-device-lib-path and HIP_LIBRARY_PATH.
   for (auto Path : RocmInstallation.getRocmDeviceLibPathArg())
     LibraryPaths.push_back(DriverArgs.MakeArgString(Path));
if (llvm::sys::fs::exists(InstLib))
       BCLibs.push_back(InstLib);
     else
       getDriver().Diag(diag::err_drv_no_such_file) << InstLib;
   }
 
   return BCLibs;
 }
 
-void HIPToolChain::checkTargetID(const llvm::opt::ArgList &DriverArgs) const {
+void HIPAMDToolChain::checkTargetID(
+    const llvm::opt::ArgList &DriverArgs) const {
   auto PTID = getParsedTargetID(DriverArgs);
   if (PTID.OptionalTargetID && !PTID.OptionalGPUArch) {
     getDriver().Diag(clang::diag::err_drv_bad_target_id)
         << PTID.OptionalTargetID.getValue();
   }
   return;
 }

clang/lib/Driver/ToolChains/HIPUtility.h

+//===--- HIPUtility.h - Common HIP Tool Chain Utilities ---------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_LIB_DRIVER_TOOLCHAINS_HIPUTILITY_H
+#define LLVM_CLANG_LIB_DRIVER_TOOLCHAINS_HIPUTILITY_H
+
+#include "clang/Driver/Tool.h"
+
+namespace clang {
+namespace driver {
+namespace tools {
+namespace HIP {
+
+// Construct command for creating HIP fatbin.
+void constructHIPFatbinCommand(Compilation &C, const JobAction &JA,
+                               StringRef OutputFileName,
+                               const InputInfoList &Inputs,
+                               const llvm::opt::ArgList &TCArgs, const Tool &T);
+
+// Construct command for creating Object from HIP fatbin.
+void constructGenerateObjFileFromHIPFatBinary(
+    Compilation &C, const InputInfo &Output, const InputInfoList &Inputs,
+    const llvm::opt::ArgList &Args, const JobAction &JA, const Tool &T);
+
+} // namespace HIP
+} // namespace tools
+} // namespace driver
+} // namespace clang
+
+#endif // LLVM_CLANG_LIB_DRIVER_TOOLCHAINS_HIPUTILITY_H

clang/lib/Driver/ToolChains/HIPUtility.cpp

+//===--- HIPUtility.cpp - Common HIP Tool Chain Utilities -------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "HIPUtility.h"
+#include "CommonArgs.h"
+#include "clang/Driver/Compilation.h"
+#include "llvm/ADT/Triple.h"
+#include "llvm/Support/Path.h"
+
+using namespace clang::driver;
+using namespace clang::driver::tools;
+using namespace llvm::opt;
+
+#if defined(_WIN32) || defined(_WIN64)
+#define NULL_FILE "nul"
+#else
+#define NULL_FILE "/dev/null"
+#endif
+
+namespace {
+const unsigned HIPCodeObjectAlign = 4096;
+} // namespace
+
+// Constructs a triple string for clang offload bundler.
+static std::string normalizeForBundler(const llvm::Triple &T,
+                                       bool HasTargetID) {
+  return HasTargetID ? (T.getArchName() + "-" + T.getVendorName() + "-" +
+                        T.getOSName() + "-" + T.getEnvironmentName())
+                           .str()
+                     : T.normalize();
+}
+
+// Construct a clang-offload-bundler command to bundle code objects for
+// different devices into a HIP fat binary.
+void HIP::constructHIPFatbinCommand(Compilation &C, const JobAction &JA,
+                                    StringRef OutputFileName,
+                                    const InputInfoList &Inputs,
+                                    const llvm::opt::ArgList &Args,
+                                    const Tool &T) {
+  // Construct clang-offload-bundler command to bundle object files for
+  // for different GPU archs.
+  ArgStringList BundlerArgs;
+  BundlerArgs.push_back(Args.MakeArgString("-type=o"));
+  BundlerArgs.push_back(
+      Args.MakeArgString("-bundle-align=" + Twine(HIPCodeObjectAlign)));
+
+  // ToDo: Remove the dummy host binary entry which is required by
+  // clang-offload-bundler.
+  std::string BundlerTargetArg = "-targets=host-x86_64-unknown-linux";
+  std::string BundlerInputArg = "-inputs=" NULL_FILE;
+
+  // AMDGCN:
+  // For code object version 2 and 3, the offload kind in bundle ID is 'hip'
+  // for backward compatibility. For code object version 4 and greater, the
+  // offload kind in bundle ID is 'hipv4'.
+  std::string OffloadKind = "hip";
+  auto &TT = T.getToolChain().getTriple();
+  if (TT.isAMDGCN() && getAMDGPUCodeObjectVersion(C.getDriver(), Args) >= 4)
+    OffloadKind = OffloadKind + "v4";
+  for (const auto &II : Inputs) {
+    const auto *A = II.getAction();
+    auto ArchStr = StringRef(A->getOffloadingArch());
+    BundlerTargetArg +=
+        "," + OffloadKind + "-" + normalizeForBundler(TT, !ArchStr.empty());
+    if (!ArchStr.empty())
+      BundlerTargetArg += "-" + ArchStr.str();
+    BundlerInputArg = BundlerInputArg + "," + II.getFilename();
+  }
+  BundlerArgs.push_back(Args.MakeArgString(BundlerTargetArg));
+  BundlerArgs.push_back(Args.MakeArgString(BundlerInputArg));
+
+  std::string Output = std::string(OutputFileName);
+  auto *BundlerOutputArg =
+      Args.MakeArgString(std::string("-outputs=").append(Output));
+  BundlerArgs.push_back(BundlerOutputArg);
+
+  const char *Bundler = Args.MakeArgString(
+      T.getToolChain().GetProgramPath("clang-offload-bundler"));
+  C.addCommand(std::make_unique<Command>(
+      JA, T, ResponseFileSupport::None(), Bundler, BundlerArgs, Inputs,
+      InputInfo(&JA, Args.MakeArgString(Output))));
+}
+
+/// Add Generated HIP Object File which has device images embedded into the
+/// host to the argument list for linking. Using MC directives, embed the
+/// device code and also define symbols required by the code generation so that
+/// the image can be retrieved at runtime.
+void HIP::constructGenerateObjFileFromHIPFatBinary(
+    Compilation &C, const InputInfo &Output, const InputInfoList &Inputs,
+    const ArgList &Args, const JobAction &JA, const Tool &T) {
+  const ToolChain &TC = T.getToolChain();
+  std::string Name = std::string(llvm::sys::path::stem(Output.getFilename()));
+
+  // Create Temp Object File Generator,
+  // Offload Bundled file and Bundled Object file.
+  // Keep them if save-temps is enabled.
+  const char *McinFile;
+  const char *BundleFile;
+  if (C.getDriver().isSaveTempsEnabled()) {
+    McinFile = C.getArgs().MakeArgString(Name + ".mcin");
+    BundleFile = C.getArgs().MakeArgString(Name + ".hipfb");
+  } else {
+    auto TmpNameMcin = C.getDriver().GetTemporaryPath(Name, "mcin");
+    McinFile = C.addTempFile(C.getArgs().MakeArgString(TmpNameMcin));
+    auto TmpNameFb = C.getDriver().GetTemporaryPath(Name, "hipfb");
+    BundleFile = C.addTempFile(C.getArgs().MakeArgString(TmpNameFb));
+  }
+  HIP::constructHIPFatbinCommand(C, JA, BundleFile, Inputs, Args, T);
+
+  // Create a buffer to write the contents of the temp obj generator.
+  std::string ObjBuffer;
+  llvm::raw_string_ostream ObjStream(ObjBuffer);
+
+  auto HostTriple =
+      C.getSingleOffloadToolChain<Action::OFK_Host>()->getTriple();
+
+  // Add MC directives to embed target binaries. We ensure that each
+  // section and image is 16-byte aligned. This is not mandatory, but
+  // increases the likelihood of data to be aligned with a cache block
+  // in several main host machines.
+  ObjStream << "#       HIP Object Generator\n";
+  ObjStream << "# *** Automatically generated by Clang ***\n";
+  if (HostTriple.isWindowsMSVCEnvironment()) {
+    ObjStream << "  .section .hip_fatbin, \"dw\"\n";
+  } else {
+    ObjStream << "  .protected __hip_fatbin\n";
+    ObjStream << "  .type __hip_fatbin,@object\n";
+    ObjStream << "  .section .hip_fatbin,\"a\",@progbits\n";
+  }
+  ObjStream << "  .globl __hip_fatbin\n";
+  ObjStream << "  .p2align " << llvm::Log2(llvm::Align(HIPCodeObjectAlign))
+            << "\n";
+  ObjStream << "__hip_fatbin:\n";
+  ObjStream << "  .incbin ";
+  llvm::sys::printArg(ObjStream, BundleFile, /*Quote=*/true);
+  ObjStream << "\n";
+  ObjStream.flush();
+
+  // Dump the contents of the temp object file gen if the user requested that.
+  // We support this option to enable testing of behavior with -###.
+  if (C.getArgs().hasArg(options::OPT_fhip_dump_offload_linker_script))
+    llvm::errs() << ObjBuffer;
+
+  // Open script file and write the contents.
+  std::error_code EC;
+  llvm::raw_fd_ostream Objf(McinFile, EC, llvm::sys::fs::OF_None);
+
+  if (EC) {
+    C.getDriver().Diag(clang::diag::err_unable_to_make_temp) << EC.message();
+    return;
+  }
+
+  Objf << ObjBuffer;
+
+  ArgStringList McArgs{"-triple", Args.MakeArgString(HostTriple.normalize()),
+                       "-o",      Output.getFilename(),
+                       McinFile,  "--filetype=obj"};
+  const char *Mc = Args.MakeArgString(TC.GetProgramPath("llvm-mc"));
+  C.addCommand(std::make_unique<Command>(JA, T, ResponseFileSupport::None(), Mc,
+                                         McArgs, Inputs, Output));
+}