[SPIR-V] Add a toolchain for SPIR-V in clang

[Anastasia]

Summary

This patch adds a toolchain (TC) for SPIR-V. The TC is not complete but it is functional enough for producing SPIR-V assembly*1 and object code directly via clang from a language of choice, for example:

clang -target spirv64 foo.cl -c -o foo.spv
clang -target spirv64 baz.clcpp -c -o baz.spv
clang -target spirv64 bar.c -S -o bar.spt

The SPIR-V code is generated by the SPIRV-LLVM translator tool named llvm-spirv that is sought in PATH.

Linking of multiple input files is currently not supported but can be added separately. The options are:

• Link using LLVM IR linker and then invoke llvm-spirv on the final linked module.
• Linking using spirv-link of SPIR-V object files generated from llvm-spirv. Creates another dependency on external tools.

Feedback would be greatly appreciated...

Changes in the Driver and base ToolChain and Tool:
Added a mechanism to work with the lack of SPIR-V backend in LLVM for SPIR-V TC. Until SPIR-V backend lands on LLVM, compilation phases/actions should be bound for SPIR-V in the meantime as following:

• compile -> tools::Clang
• backend -> tools::SPIRV::Translator
• assemble -> tools::SPIRV::Translator

However, Driver’s ToolSelector collapses compile-backend-assemble and compile-backend sequences to tools::Clang. To prevent this, added new {use,has}IntegratedBackend properties in ToolChain and Tool to which the ToolSelector reacts on, and which SPIR-V TC overrides.

This is contributed originally by @linjamaki along with the previous changes in the stack (D112404 - [SPIR-V] Add translator tool) to address feedback in (https://reviews.llvm.org/D110618#3062078) and other concerns around the SPIR-V toolchain adoption for languages other than HIP.

[1]: https://github.com/KhronosGroup/SPIRV-LLVM-Translator/wiki/SPIRV-Toolchain-for-Clang

This change relies on the following changes to llvm-spirv:

• Accepting SPIR-V as a valid triple: https://github.com/KhronosGroup/SPIRV-LLVM-Translator/pull/1305
• [Optional but desirable] Generation of standard SPIR-V assembly syntax: https://github.com/KhronosGroup/SPIRV-LLVM-Translator/pull/1294

[Anastasia]

Cool. I like the direction... In the end it probably makes sense to separate clang compilation flow into more separate steps. This makes it more suitable to non-LLVM based backends or even MLIR towards Clang Intermediate Language. This looks like a fairly good starting point! Thanks!

Thus, please consider this as a potentially useful starting point for further work needed to support other frontends, hopefully good enough minimal code to get this side started with and our patch set integrated to the master.

Do I read it correctly that you would have no objections for others to rework this patch if needed? :)

[linjamaki]

Do I read it correctly that you would have no objections for others to rework this patch if needed? :)

Yes, anyone can work on this patch.

[linjamaki]

Rebase on updated llvm-spirv tool (D112404).

[Anastasia]

• Rebased.
• Added support for default compilation into binary without -c or -emit-llvm.
• Added a warning about unsupported linking when multiple files are passed.
• Added documentation.
• Added more test cases.

[Anastasia]

• Exported full diff.
• Added forgotten test cases.
• Fixed typo in docs.

[svenvh]

clang/lib/Driver/Driver.cpp

3728

FIXME? (assuming this is something we want to address eventually)

clang/test/Driver/spirv-toolchain.cl

10 ↗

(On Diff #391064)

Any reason to not just check for llvm-spirv{{.*}}, for consistency with the clang check above?

clang/test/Misc/warning-flags.c

21 ↗

(On Diff #391064)

The comment above says: "The list of warnings below should NEVER grow.", and the current patch violates that. You'll need to add a warning group to the new warning.

[Anastasia]

clang/docs/UsersManual.rst

3559 ↗

(On Diff #391064)

Btw do we want to say anything about the limitations of translating IR when optimizations are enabled?

clang/test/Driver/spirv-toolchain.cl

10 ↗

(On Diff #391064)

Good question, apparently some tools get some target prefixes like if you look at Driver::generatePrefixedToolNames:
https://clang.llvm.org/doxygen/Driver_8cpp_source.html#l05169

But perhaps it doesn't happen for llvm-spirv and we can safely omit the prefix?

[svenvh]

clang/test/Driver/spirv-toolchain.cl

10 ↗

(On Diff #391064)

Having a target prefix for llvm-spirv seems a bit redundant indeed. Driver::generatePrefixedToolNames seems to add both a prefixed and non-prefixed tool name.

[Anastasia]

Addressed review comments from Sven:

• Improved docs and code comments;
• Simplified tests;
• Added warning in a group.
• Aligned with translator PRs after merge.

Note, that this change now references documentation from PR https://github.com/KhronosGroup/SPIRV-LLVM-Translator/pull/1342

[svenvh]

Mostly some minor comments that you can address at commit time.

It would be good to get approval from another reviewer.

clang/docs/UsersManual.rst

3538 ↗

(On Diff #394353)

clang/test/Driver/spirv-toolchain.cl

10 ↗

(On Diff #391064)

I see you have taken out the checking of the prefix; my suggestion was to write llvm-spirv{{.*}}, to align with the clang check above.

[linjamaki]

If I understand correctly, the default optimization level is -O2 for the OpenCL and OpenCL++ language mode (according to CompilerInvocation.cpp:getOptimizationLevel()). Since higher than -O0 may not work with the translator, should the SPIR-V tool chain override this default and set it to -O0 (unless a user passes some -O option)?

[Anastasia]

In D112410#3197441, @linjamaki wrote:

If I understand correctly, the default optimization level is -O2 for the OpenCL and OpenCL++ language mode (according to CompilerInvocation.cpp:getOptimizationLevel()). Since higher than -O0 may not work with the translator, should the SPIR-V tool chain override this default and set it to -O0 (unless a user passes some -O option)?

We could do that, although I also find it a bit confusing if one target behaves differently from the rest. I believe -O2 is clang's default, not specific to a particular language...

Also I have an impression that translating optimised IR mostly works.

And another aspect to consider is that we are also integrating the SPIR-V backend in parallel which should be supporting the optimizations fully. So I would prefer not to tailor the clang architecture to the use of the translator too much as it is only a temporary solution...

[MaskRay]

I slightly adjusted the test in eafc64ed6325eba962096d4a947d7e45e909bfde :)

-no-canonical-prefixes can usually be omitted.

[Anastasia]

In D112410#3208353, @MaskRay wrote:

I slightly adjusted the test in eafc64ed6325eba962096d4a947d7e45e909bfde :)

-no-canonical-prefixes can usually be omitted.

Thanks!

clang/include/clang/Driver/Tool.h

public:
 
   const char *getName() const { return Name; }
 
   const char *getShortName() const { return ShortName; }
 
   const ToolChain &getToolChain() const { return TheToolChain; }
 
   virtual bool hasIntegratedAssembler() const { return false; }
+  virtual bool hasIntegratedBackend() const { return true; }
   virtual bool canEmitIR() const { return false; }
   virtual bool hasIntegratedCPP() const = 0;
   virtual bool isLinkJob() const { return false; }
   virtual bool isDsymutilJob() const { return false; }
 
   /// Does this tool have "good" standardized diagnostics, or should the
   /// driver add an additional "command failed" diagnostic on failures.
   virtual bool hasGoodDiagnostics() const { return false; }

clang/include/clang/Driver/ToolChain.h

public:
 
   /// IsIntegratedAssemblerDefault - Does this tool chain enable -integrated-as
   /// by default.
   virtual bool IsIntegratedAssemblerDefault() const { return false; }
 
   /// Check if the toolchain should use the integrated assembler.
   virtual bool useIntegratedAs() const;
 
+  /// Check if the toolchain should use the integrated backend.
+  virtual bool useIntegratedBackend() const { return true; }
+
   /// Check if the toolchain should use AsmParser to parse inlineAsm when
   /// integrated assembler is not default.
   virtual bool parseInlineAsmUsingAsmParser() const { return false; }
 
   /// IsMathErrnoDefault - Does this tool chain use -fmath-errno by default.
   virtual bool IsMathErrnoDefault() const { return true; }
 
   /// IsEncodeExtendedBlockSignatureDefault - Does this tool chain enable

clang/lib/Driver/Driver.cpp

 #include "ToolChains/MipsLinux.h"
 #include "ToolChains/Myriad.h"
 #include "ToolChains/NaCl.h"
 #include "ToolChains/NetBSD.h"
 #include "ToolChains/OpenBSD.h"
 #include "ToolChains/PPCLinux.h"
 #include "ToolChains/PS4CPU.h"
 #include "ToolChains/RISCVToolchain.h"
+#include "ToolChains/SPIRV.h"
 #include "ToolChains/Solaris.h"
 #include "ToolChains/TCE.h"
 #include "ToolChains/VEToolchain.h"
 #include "ToolChains/WebAssembly.h"
 #include "ToolChains/XCore.h"
 #include "ToolChains/ZOS.h"
 #include "clang/Basic/TargetID.h"
 #include "clang/Basic/Version.h"
void Driver::BuildActions(Compilation &C, DerivedArgList &Args,
   if (Arg *A = Args.getLastArg(options::OPT__SLASH_o)) {
     if (A->getValue()[0] == '\0') {
       // It has to have a value.
       Diag(clang::diag::err_drv_missing_argument) << A->getSpelling() << 1;
       Args.eraseArg(options::OPT__SLASH_o);
     }
   }
 
   handleArguments(C, Args, Inputs, Actions);
 
   // Builder to be used to build offloading actions.
   OffloadingActionBuilder OffloadBuilder(C, Args, Inputs);
 
   // Construct the actions to perform.
   HeaderModulePrecompileJobAction *HeaderModuleAction = nullptr;
   ActionList LinkerInputs;
   ActionList MergerInputs;
combineAssembleBackendCompile(ArrayRef<JobActionInfo> ActionInfo,
     if (!AJ || !BJ || !CJ)
       return nullptr;
 
     // Get compiler tool.
     const Tool *T = TC.SelectTool(*CJ);
     if (!T)
       return nullptr;
 
+    // Can't collapse if we don't have codegen support unless we are
+    // emitting LLVM IR.
+    bool OutputIsLLVM = types::isLLVMIR(ActionInfo[0].JA->getType());
+    if (!T->hasIntegratedBackend() && !(OutputIsLLVM && T->canEmitIR()))
+      return nullptr;
+
     // When using -fembed-bitcode, it is required to have the same tool (clang)
     // for both CompilerJA and BackendJA. Otherwise, combine two stages.
     if (EmbedBitcode) {
       const Tool *BT = TC.SelectTool(*BJ);
       if (BT == T)
         return nullptr;
     }
 
const Tool *combineBackendCompile(ArrayRef<JobActionInfo> ActionInfo,
     if (!InputIsBitcode && !canCollapsePreprocessorAction())
       return nullptr;
 
     // Get compiler tool.
     const Tool *T = TC.SelectTool(*CJ);
     if (!T)
       return nullptr;
 
+    // Can't collapse if we don't have codegen support unless we are
+    // emitting LLVM IR.
+    bool OutputIsLLVM = types::isLLVMIR(ActionInfo[0].JA->getType());
+    if (!T->hasIntegratedBackend() && !(OutputIsLLVM && T->canEmitIR()))
+      return nullptr;
+
     if (T->canEmitIR() && ((SaveTemps && !InputIsBitcode) || EmbedBitcode))
       return nullptr;
 
     Inputs = CJ->getInputs();
     AppendCollapsedOffloadAction(CollapsedOffloadAction, ActionInfo,
                                  /*NumElements=*/2);
     return T;
   }
default:
           TC =
               std::make_unique<toolchains::RISCVToolChain>(*this, Target, Args);
         else
           TC = std::make_unique<toolchains::BareMetal>(*this, Target, Args);
         break;
       case llvm::Triple::ve:
         TC = std::make_unique<toolchains::VEToolChain>(*this, Target, Args);
         break;
+      case llvm::Triple::spirv32:
+      case llvm::Triple::spirv64:
+        TC = std::make_unique<toolchains::SPIRVToolChain>(*this, Target, Args);
+        break;
       default:
         if (Target.getVendor() == llvm::Triple::Myriad)
           TC = std::make_unique<toolchains::MyriadToolChain>(*this, Target,
                                                               Args);
         else if (toolchains::BareMetal::handlesTarget(Target))
           TC = std::make_unique<toolchains::BareMetal>(*this, Target, Args);
         else if (Target.isOSBinFormatELF())
           TC = std::make_unique<toolchains::Generic_ELF>(*this, Target, Args);

clang/lib/Driver/ToolChain.cpp

 }
 
 bool ToolChain::IsUnwindTablesDefault(const ArgList &Args) const {
   return false;
 }
 
 Tool *ToolChain::getClang() const {
   if (!Clang)
-    Clang.reset(new tools::Clang(*this));
+    Clang.reset(new tools::Clang(*this, useIntegratedBackend()));
   return Clang.get();
 }
 
 Tool *ToolChain::getFlang() const {
   if (!Flang)
     Flang.reset(new tools::Flang(*this));
   return Flang.get();
 }

clang/lib/Driver/ToolChains/Clang.h

 namespace clang {
 class ObjCRuntime;
 namespace driver {
 
 namespace tools {
 
 /// Clang compiler tool.
 class LLVM_LIBRARY_VISIBILITY Clang : public Tool {
+  bool HasBackend;
+
 public:
   static const char *getBaseInputName(const llvm::opt::ArgList &Args,
                                       const InputInfo &Input);
   static const char *getBaseInputStem(const llvm::opt::ArgList &Args,
                                       const InputInfoList &Inputs);
   static const char *getDependencyFileName(const llvm::opt::ArgList &Args,
                                            const InputInfoList &Inputs);
 
void DumpCompilationDatabase(Compilation &C, StringRef Filename,
                                const InputInfo &Output, const InputInfo &Input,
                                const llvm::opt::ArgList &Args) const;
 
   void DumpCompilationDatabaseFragmentToDir(
       StringRef Dir, Compilation &C, StringRef Target, const InputInfo &Output,
       const InputInfo &Input, const llvm::opt::ArgList &Args) const;
 
 public:
-  Clang(const ToolChain &TC);
+  Clang(const ToolChain &TC, bool HasBackend = true);
   ~Clang() override;
 
   bool hasGoodDiagnostics() const override { return true; }
   bool hasIntegratedAssembler() const override { return true; }
+  bool hasIntegratedBackend() const override { return HasBackend; }
   bool hasIntegratedCPP() const override { return true; }
   bool canEmitIR() const override { return true; }
 
   void ConstructJob(Compilation &C, const JobAction &JA,
                     const InputInfo &Output, const InputInfoList &Inputs,
                     const llvm::opt::ArgList &TCArgs,
                     const char *LinkingOutput) const override;
 };

clang/lib/Driver/ToolChains/Clang.cpp

   // care to warn the user about.
   Args.ClaimAllArgs(options::OPT_clang_ignored_f_Group);
   Args.ClaimAllArgs(options::OPT_clang_ignored_m_Group);
 
   // Disable warnings for clang -E -emit-llvm foo.c
   Args.ClaimAllArgs(options::OPT_emit_llvm);
 }
 
-Clang::Clang(const ToolChain &TC)
+Clang::Clang(const ToolChain &TC, bool HasBackendVal)
     // CAUTION! The first constructor argument ("clang") is not arbitrary,
     // as it is for other tools. Some operations on a Tool actually test
     // whether that tool is Clang based on the Tool's Name as a string.
-    : Tool("clang", "clang frontend", TC) {}
+    : Tool("clang", "clang frontend", TC), HasBackend(HasBackendVal) {}
 
 Clang::~Clang() {}
 
 /// Add options related to the Objective-C runtime/ABI.
 ///
 /// Returns true if the runtime is non-fragile.
 ObjCRuntime Clang::AddObjCRuntimeArgs(const ArgList &args,
                                       const InputInfoList &inputs,

clang/lib/Driver/ToolChains/SPIRV.h

public:
   void ConstructJob(Compilation &C, const JobAction &JA,
                     const InputInfo &Output, const InputInfoList &Inputs,
                     const llvm::opt::ArgList &TCArgs,
                     const char *LinkingOutput) const override;
 };
 
 } // namespace SPIRV
 } // namespace tools
+
+namespace toolchains {
+
+class LLVM_LIBRARY_VISIBILITY SPIRVToolChain final : public ToolChain {
+  mutable std::unique_ptr<Tool> Translator;
+
+public:
+  SPIRVToolChain(const Driver &D, const llvm::Triple &Triple,
+                 const llvm::opt::ArgList &Args)
+      : ToolChain(D, Triple, Args) {}
+
+  bool useIntegratedAs() const override { return true; }
+  bool useIntegratedBackend() const override { return false; }
+
+  bool IsMathErrnoDefault() const override { return false; }
+  bool isCrossCompiling() const override { return true; }
+  bool isPICDefault() const override { return false; }
+  bool isPIEDefault() const override { return false; }
+  bool isPICDefaultForced() const override { return false; }
+  bool SupportsProfiling() const override { return false; }
+
+  clang::driver::Tool *SelectTool(const JobAction &JA) const override;
+
+protected:
+  clang::driver::Tool *getTool(Action::ActionClass AC) const override;
+
+private:
+  clang::driver::Tool *getTranslator() const;
+};
+
+} // namespace toolchains
 } // namespace driver
 } // namespace clang
 #endif

clang/lib/Driver/ToolChains/SPIRV.cpp

 //===--- SPIRV.cpp - SPIR-V Tool Implementations ----------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 #include "SPIRV.h"
 #include "CommonArgs.h"
 #include "clang/Driver/Compilation.h"
 #include "clang/Driver/Driver.h"
 #include "clang/Driver/InputInfo.h"
 #include "clang/Driver/Options.h"
 
+using namespace clang::driver::toolchains;
 using namespace clang::driver::tools;
 using namespace llvm::opt;
 
 void SPIRV::constructTranslateCommand(Compilation &C, const Tool &T,
                                       const JobAction &JA,
                                       const InputInfo &Output,
                                       const InputInfo &Input,
                                       const llvm::opt::ArgStringList &Args) {
void SPIRV::Translator::ConstructJob(Compilation &C, const JobAction &JA,
                                      const InputInfoList &Inputs,
                                      const ArgList &Args,
                                      const char *LinkingOutput) const {
   claimNoWarnArgs(Args);
   if (Inputs.size() != 1)
     llvm_unreachable("Invalid number of input files.");
   constructTranslateCommand(C, *this, JA, Output, Inputs[0], {});
 }
+
+clang::driver::Tool *SPIRVToolChain::getTranslator() const {
+  if (!Translator)
+    Translator = std::make_unique<SPIRV::Translator>(*this);
+  return Translator.get();
+}
+
+clang::driver::Tool *SPIRVToolChain::SelectTool(const JobAction &JA) const {
+  Action::ActionClass AC = JA.getKind();
+  return SPIRVToolChain::getTool(AC);
+}
+
+clang::driver::Tool *SPIRVToolChain::getTool(Action::ActionClass AC) const {
+  switch (AC) {
+  default:
+    break;
+  case Action::BackendJobClass:
+  case Action::AssembleJobClass:
+    return SPIRVToolChain::getTranslator();
+  }
+  return ToolChain::getTool(AC);
+}

clang/test/Driver/spirv-toolchain.c

+// Check object emission.
+// RUN: %clang -### -no-canonical-prefixes -target spirv64 -x cl -c %s 2>&1 | FileCheck --check-prefix=SPV64 %s
+// RUN: %clang -### -no-canonical-prefixes -target spirv64 -x ir -c %s 2>&1 | FileCheck --check-prefix=SPV64 %s
+// RUN: %clang -### -no-canonical-prefixes -target spirv64 -x clcpp -c %s 2>&1 | FileCheck --check-prefix=SPV64 %s
+// RUN: %clang -### -no-canonical-prefixes -target spirv64 -x c -c %s 2>&1 | FileCheck --check-prefix=SPV64 %s
+
+// SPV64: clang{{.*}} "-cc1" "-triple" "spirv64"
+// SPV64-SAME: "-o" [[BC:".*bc"]]
+// SPV64: {{".*llvm-spirv.*"}} [[BC]] "-o" {{".*o"}}
+
+// RUN: %clang -### -no-canonical-prefixes -target spirv32 -x cl -c %s 2>&1 | FileCheck --check-prefix=SPV32 %s
+// RUN: %clang -### -no-canonical-prefixes -target spirv32 -x ir -c %s 2>&1 | FileCheck --check-prefix=SPV32 %s
+// RUN: %clang -### -no-canonical-prefixes -target spirv32 -x clcpp -c %s 2>&1 | FileCheck --check-prefix=SPV32 %s
+// RUN: %clang -### -no-canonical-prefixes -target spirv32 -x c -c %s 2>&1 | FileCheck --check-prefix=SPV32 %s
+
+// SPV32: clang{{.*}} "-cc1" "-triple" "spirv32"
+// SPV32-SAME: "-o" [[BC:".*bc"]]
+// SPV32: {{".*llvm-spirv.*"}} [[BC]] "-o" {{".*o"}}
+
+//-----------------------------------------------------------------------------
+// Check Assembly emission.
+// RUN: %clang -### -no-canonical-prefixes -target spirv64 -x cl -S %s 2>&1 | FileCheck --check-prefix=SPT64 %s
+// RUN: %clang -### -no-canonical-prefixes -target spirv64 -x ir -S %s 2>&1 | FileCheck --check-prefix=SPT64 %s
+// RUN: %clang -### -no-canonical-prefixes -target spirv64 -x clcpp -c %s 2>&1 | FileCheck --check-prefix=SPV64 %s
+// RUN: %clang -### -no-canonical-prefixes -target spirv64 -x c -S %s 2>&1 | FileCheck --check-prefix=SPT64 %s
+
+// SPT64: clang{{.*}} "-cc1" "-triple" "spirv64"
+// SPT64-SAME: "-o" [[BC:".*bc"]]
+// SPT64: {{".*llvm-spirv.*"}} [[BC]] "-spirv-text" "-o" {{".*s"}}
+
+// RUN: %clang -### -no-canonical-prefixes -target spirv32 -x cl -S %s 2>&1 | FileCheck --check-prefix=SPT32 %s
+// RUN: %clang -### -no-canonical-prefixes -target spirv32 -x ir -S %s 2>&1 | FileCheck --check-prefix=SPT32 %s
+// RUN: %clang -### -no-canonical-prefixes -target spirv32 -x clcpp -c %s 2>&1 | FileCheck --check-prefix=SPV32 %s
+// RUN: %clang -### -no-canonical-prefixes -target spirv32 -x c -S %s 2>&1 | FileCheck --check-prefix=SPT32 %s
+
+// SPT32: clang{{.*}} "-cc1" "-triple" "spirv32"
+// SPT32-SAME: "-o" [[BC:".*bc"]]
+// SPT32: {{".*llvm-spirv.*"}} [[BC]] "-spirv-text" "-o" {{".*s"}}
+
+//-----------------------------------------------------------------------------
+// Check assembly input -> object output
+// RUN: %clang -### -no-canonical-prefixes -target spirv64 -x assembler -c %s 2>&1 | FileCheck --check-prefix=ASM %s
+// RUN: %clang -### -no-canonical-prefixes -target spirv32 -x assembler -c %s 2>&1 | FileCheck --check-prefix=ASM %s
+// ASM: {{".*llvm-spirv.*"}} {{".*"}} "-to-binary" "-o" {{".*o"}}
+
+//-----------------------------------------------------------------------------
+// Check --save-temps.
+// RUN: %clang -### -no-canonical-prefixes -target spirv64 -x cl -c %s --save-temps 2>&1 | FileCheck --check-prefix=TMP %s
+
+// TMP: clang{{.*}} "-cc1" "-triple" "spirv64"
+// TMP-SAME: "-E"
+// TMP-SAME: "-o" [[I:".*i"]]
+// TMP: clang{{.*}} "-cc1" "-triple" "spirv64"
+// TMP-SAME: "-o" [[BC:".*bc"]]
+// TMP-SAME: [[I]]
+// TMP: {{".*llvm-spirv.*"}} [[BC]] "-spirv-text" "-o" [[S:".*s"]]
+// TMP: {{".*llvm-spirv.*"}} [[S]] "-to-binary" "-o" {{".*o"}}