[DirectX backend] Add pass to lower llvm intrinsic into dxil op function.

[python3kgae]

Summary

A new pass DXILOpLowering was added.
It will scan all llvm intrinsics, create dxil op function if it can map to dxil op function.
Then translate call instructions on the intrinsic into call on dxil op function.
dxil op function will add i32 argument to the begining of args for dxil opcode.
So cannot use setCalledFunction to update the call instruction on intrinsic.

This commit only support sin to start the work.

[nhaehnle]

Overall approach looks good to me, I just have a whole bunch of comments on the details.

llvm/lib/Target/DirectX/DXILOpLowering.cpp

33–34

Non-trivial globals require an initializer that is executed when the libLLVM.so (or DLL) is loaded, which is a negative impact on a huge number of users of LLVM, including those not using the DirectX target.

I'd suggest making this a static function-local variable.

163–166

Please do TableGen this. When you do, please consider replacing the inline const char* by an offset into a large string constant. This has two benefits:

1. It allows you to pack the struct more tightly, using a 32-bit or even 16-bit index instead of a 64-bit pointer.
2. It avoids the need for constant data relocations when loading libLLVM.so.

I'm okay with keeping the FIXME comment for now.

174–179

I believe you can use llvm::lower_bound(OpCodeProps, ...) here.

180–182

Just *Pos? Or rename Pos to Prop?

217–219

Nit: You don't need braces around single-line if bodies, and personally I think this is more readable without.

234–236

You may want to already make this code ready for use with the NewPassManager.

llvm/test/CodeGen/DirectX/sin.ll

2

Could this be an opt test?

[python3kgae]

Update based on comments.

[kuhar]

llvm/lib/Target/DirectX/DXILOpLowering.cpp

32

nit: prefer StringLiteral

34

nit: prefer enum class

84

Use cast if you know that the cast must succeed
also nit: don't prefix pointer variables with p

95

Can this be some other value than these cases, e.g., 128 or 63 bits? Should we have a default case with llvm_unreachable?

122–123

nit: you can do os.str() which will flush for you.

144

Shouldn't we check that this array element can be accessed?

150

I think you can do Twine(x) + y, without creating a second twine. If not, consider changing char * to StringLiteral

158–161

Could we *not* introduce 2/3-letter-long acronyms like this? I think this hurts readability unless such types are very well known.

174

nit: use llvm::lower_bound

176

I'm confused by this division. Can you pull this out into a local variable with a descriptive name and/or add a comment?

188

nit: can you compare to zero instead of implicitly converting to bool and negating?

193

nit: llvm prefers !ptr instead of ptr == nullptr. Also, please capitalize the assertion message.

198

Why exactly 4? If we don't know the expected size, prefer the automatic one with SmallVector<Type *>

211–212

nit: Could we make_early_inc_range instead?

214

nit: !CI

216

Why do we need a second builder?

217

same here

230

I think SmallDenseMap would work here

232–233

also here

[kuhar]

llvm/lib/Target/DirectX/DXILOpLowering.cpp

151

nit: no need for else after return: https://llvm.org/docs/CodingStandards.html#don-t-use-else-after-a-return

[python3kgae]

Update to fix commented issue.

llvm/lib/Target/DirectX/DXILOpLowering.cpp

34

enum class will require cast on case like OverloadKind::FLOAT | OverloadKind::HALF.
I prefer not to have
(uint16_t)OverloadKind::FLOAT | (uint16_t)OverloadKind::HALF.

Is it OK to keep it without enum class?

[kuhar]

llvm/lib/Target/DirectX/DXILOpLowering.cpp

34

There are 3 options:

1. Keep the code as-is
2. Add enum class and cast manually
3. Add enum class and overload the operators you need and do the casting inside.

I think both 1 and 3 are fine. 1 is less code and this enum is not in a header, so I'd be leaning towards keeping this as-is.

[python3kgae]

Rebase to avoid merge conflict.

[kuhar]

LGTM but please get another one before submitting

llvm/lib/Target/DirectX/DXILOpLowering.cpp

122

nit: no else after return: https://www.llvm.org/docs/CodingStandards.html#don-t-use-else-after-a-return

[python3kgae]

Use static instead of namespace.

[beanz]

Two small nitpicks, otherwise LGTM.

llvm/lib/Target/DirectX/DXILOpLowering.cpp

64

Rather than a default case here, fill in all possible enum values (I think you are just missing void, user-defined, and object). Covering all cases will result in a diagnostic if new cases are added and not handled.

193

nit: exists

[python3kgae]

Update for comments.

llvm/lib/Target/DirectX/CMakeLists.txt

 add_llvm_component_group(DirectX)
 
 set(LLVM_TARGET_DEFINITIONS DirectX.td)
 
 tablegen(LLVM DirectXGenSubtargetInfo.inc -gen-subtarget)
 
 add_public_tablegen_target(DirectXCommonTableGen)
 
 add_llvm_target(DirectXCodeGen
   DirectXSubtarget.cpp
   DirectXTargetMachine.cpp
+  DXILOpLowering.cpp
   DXILPointerType.cpp
   DXILPrepare.cpp
   PointerTypeAnalysis.cpp
 
   LINK_COMPONENTS
   Core
   Support
   DirectXInfo

llvm/lib/Target/DirectX/DXILConstants.h

+//===- DXILConstants.h - Essential DXIL constants -------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file This file contains essential DXIL constants.
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIB_TARGET_DIRECTX_DXILCONSTANTS_H
+#define LLVM_LIB_TARGET_DIRECTX_DXILCONSTANTS_H
+
+namespace llvm {
+namespace DXIL {
+// Enumeration for operations specified by DXIL
+enum class OpCode : unsigned {
+  Sin = 13, // returns sine(theta) for theta in radians.
+};
+// Groups for DXIL operations with equivalent function templates
+enum class OpCodeClass : unsigned {
+  Unary,
+};
+
+} // namespace DXIL
+} // namespace llvm
+
+#endif

llvm/lib/Target/DirectX/DXILOpLowering.cpp

+//===- DXILOpLower.cpp - Lowering LLVM intrinsic to DIXLOp function -------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file This file contains passes and utilities to lower llvm intrinsic call
+/// to DXILOp function call.
+//===----------------------------------------------------------------------===//
+
+#include "DXILConstants.h"
+#include "DirectX.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/CodeGen/Passes.h"
+#include "llvm/IR/IRBuilder.h"
+#include "llvm/IR/Instruction.h"
+#include "llvm/IR/Intrinsics.h"
+#include "llvm/IR/Module.h"
+#include "llvm/IR/PassManager.h"
+#include "llvm/Pass.h"
+#include "llvm/Support/ErrorHandling.h"
+
+#define DEBUG_TYPE "dxil-op-lower"
+
+using namespace llvm;
+using namespace llvm::DXIL;
+
+constexpr StringLiteral DXILOpNamePrefix = "dx.op.";
+
+enum OverloadKind : uint16_t {
+  VOID = 1,
+  HALF = 1 << 1,
+  FLOAT = 1 << 2,
+  DOUBLE = 1 << 3,
+  I1 = 1 << 4,
+  I8 = 1 << 5,
+  I16 = 1 << 6,
+  I32 = 1 << 7,
+  I64 = 1 << 8,
+  UserDefineType = 1 << 9,
+  ObjectType = 1 << 10,
+};
+
+static const char *getOverloadTypeName(OverloadKind Kind) {
+  switch (Kind) {
+  case OverloadKind::HALF:
+    return "f16";
+  case OverloadKind::FLOAT:
+    return "f32";
+  case OverloadKind::DOUBLE:
+    return "f64";
+  case OverloadKind::I1:
+    return "i1";
+  case OverloadKind::I8:
+    return "i8";
+  case OverloadKind::I16:
+    return "i16";
+  case OverloadKind::I32:
+    return "i32";
+  case OverloadKind::I64:
+    return "i64";
+  default:
+    llvm_unreachable("invalid overload type for name");
+    break;
+  }
+}
+
+static OverloadKind getOverloadKind(Type *Ty) {
+  Type::TypeID T = Ty->getTypeID();
+  switch (T) {
+  case Type::VoidTyID:
+    return OverloadKind::VOID;
+  case Type::HalfTyID:
+    return OverloadKind::HALF;
+  case Type::FloatTyID:
+    return OverloadKind::FLOAT;
+  case Type::DoubleTyID:
+    return OverloadKind::DOUBLE;
+  case Type::IntegerTyID: {
+    IntegerType *ITy = cast<IntegerType>(Ty);
+    unsigned Bits = ITy->getBitWidth();
+    switch (Bits) {
+    case 1:
+      return OverloadKind::I1;
+    case 8:
+      return OverloadKind::I8;
+    case 16:
+      return OverloadKind::I16;
+    case 32:
+      return OverloadKind::I32;
+    case 64:
+      return OverloadKind::I64;
+    default:
+      llvm_unreachable("invalid overload type");
+      return OverloadKind::VOID;
+    }
+  }
+  case Type::PointerTyID:
+    return OverloadKind::UserDefineType;
+  case Type::StructTyID:
+    return OverloadKind::ObjectType;
+  default:
+    llvm_unreachable("invalid overload type");
+    return OverloadKind::VOID;
+  }
+}
+
+static std::string getTypeName(OverloadKind Kind, Type *Ty) {
+  if (Kind < OverloadKind::UserDefineType) {
+    return getOverloadTypeName(Kind);
+  } else if (Kind == OverloadKind::UserDefineType) {
+    StructType *ST = cast<StructType>(Ty);
+    return ST->getStructName().str();
+  } else if (Kind == OverloadKind::ObjectType) {
+    StructType *ST = cast<StructType>(Ty);
+    return ST->getStructName().str();
+  } else {
+    std::string Str;
+    raw_string_ostream OS(Str);
+    Ty->print(OS);
+    return OS.str();
+  }
+}
+
+// Static properties.
+struct OpCodeProperty {
+  DXIL::OpCode OpCode;
+  // FIXME: change OpCodeName into index to a large string constant when move to
+  // tableGen.
+  const char *OpCodeName;
+  DXIL::OpCodeClass OpCodeClass;
+  uint16_t OverloadTys;
+  llvm::Attribute::AttrKind FuncAttr;
+};
+
+static const char *getOpCodeClassName(const OpCodeProperty &Prop) {
+  // FIXME: generate this table with tableGen.
+  static const char *OpCodeClassNames[] = {
+      "unary",
+  };
+  unsigned Index = static_cast<unsigned>(Prop.OpCodeClass);
+  assert(Index < (sizeof(OpCodeClassNames) / sizeof(OpCodeClassNames[0])) &&
+         "Out of bound OpCodeClass");
+  return OpCodeClassNames[Index];
+}
+
+static std::string constructOverloadName(OverloadKind Kind, Type *Ty,
+                                         const OpCodeProperty &Prop) {
+  if (Kind == OverloadKind::VOID) {
+    return (Twine(DXILOpNamePrefix) + getOpCodeClassName(Prop)).str();
+  }
+  return (Twine(DXILOpNamePrefix) + getOpCodeClassName(Prop) + "." +
+          getTypeName(Kind, Ty))
+      .str();
+}
+
+static const OpCodeProperty *getOpCodeProperty(DXIL::OpCode DXILOp) {
+  // FIXME: generate this table with tableGen.
+  static const OpCodeProperty OpCodeProps[] = {
+      {DXIL::OpCode::Sin, "Sin", OpCodeClass::Unary,
+       OverloadKind::FLOAT | OverloadKind::HALF, Attribute::AttrKind::ReadNone},
+  };
+  // FIXME: change search to indexing with
+  // DXILOp once all DXIL op is added.
+  OpCodeProperty TmpProp;
+  TmpProp.OpCode = DXILOp;
+  const OpCodeProperty *Prop =
+      llvm::lower_bound(OpCodeProps, TmpProp,
+                        [](const OpCodeProperty &A, const OpCodeProperty &B) {
+                          return A.OpCode < B.OpCode;
+                        });
+  return Prop;
+}
+
+static FunctionCallee createDXILOpFunction(DXIL::OpCode DXILOp, Function &F,
+                                           Module &M) {
+  const OpCodeProperty *Prop = getOpCodeProperty(DXILOp);
+
+  // Get return type as overload type for DXILOp.
+  // Only simple mapping case here, so return type is good enough.
+  Type *OverloadTy = F.getReturnType();
+
+  OverloadKind Kind = getOverloadKind(OverloadTy);
+  // FIXME: find the issue and report error in clang instead of check it in
+  // backend.
+  if ((Prop->OverloadTys & (uint16_t)Kind) == 0) {
+    llvm_unreachable("invalid overload");
+  }
+
+  std::string FnName = constructOverloadName(Kind, OverloadTy, *Prop);
+  assert(!M.getFunction(FnName) && "Function already exist");
+
+  auto &Ctx = M.getContext();
+  Type *OpCodeTy = Type::getInt32Ty(Ctx);
+
+  SmallVector<Type *> ArgTypes;
+  // DXIL has i32 opcode as first arg.
+  ArgTypes.emplace_back(OpCodeTy);
+  FunctionType *FT = F.getFunctionType();
+  ArgTypes.append(FT->param_begin(), FT->param_end());
+  FunctionType *DXILOpFT = FunctionType::get(OverloadTy, ArgTypes, false);
+  return M.getOrInsertFunction(FnName, DXILOpFT);
+}
+
+static void lowerIntrinsic(DXIL::OpCode DXILOp, Function &F, Module &M) {
+  auto DXILOpFn = createDXILOpFunction(DXILOp, F, M);
+  IRBuilder<> B(M.getContext());
+  Value *DXILOpArg = B.getInt32(static_cast<unsigned>(DXILOp));
+  for (User *U : make_early_inc_range(F.users())) {
+    CallInst *CI = dyn_cast<CallInst>(U);
+    if (!CI)
+      continue;
+
+    SmallVector<Value *> Args;
+    Args.emplace_back(DXILOpArg);
+    Args.append(CI->arg_begin(), CI->arg_end());
+    B.SetInsertPoint(CI);
+    CallInst *DXILCI = B.CreateCall(DXILOpFn, Args);
+    CI->replaceAllUsesWith(DXILCI);
+    CI->eraseFromParent();
+  }
+  if (F.user_empty())
+    F.eraseFromParent();
+}
+
+static bool lowerIntrinsics(Module &M) {
+  bool Updated = false;
+  static SmallDenseMap<Intrinsic::ID, DXIL::OpCode> LowerMap = {
+      {Intrinsic::sin, DXIL::OpCode::Sin}};
+  for (Function &F : make_early_inc_range(M.functions())) {
+    if (!F.isDeclaration())
+      continue;
+    Intrinsic::ID ID = F.getIntrinsicID();
+    auto LowerIt = LowerMap.find(ID);
+    if (LowerIt == LowerMap.end())
+      continue;
+    lowerIntrinsic(LowerIt->second, F, M);
+    Updated = true;
+  }
+  return Updated;
+}
+
+namespace {
+/// A pass that transforms external global definitions into declarations.
+class DXILOpLowering : public PassInfoMixin<DXILOpLowering> {
+public:
+  PreservedAnalyses run(Module &M, ModuleAnalysisManager &) {
+    if (lowerIntrinsics(M))
+      return PreservedAnalyses::none();
+    return PreservedAnalyses::all();
+  }
+};
+} // namespace
+
+namespace {
+class DXILOpLoweringLegacy : public ModulePass {
+public:
+  bool runOnModule(Module &M) override { return lowerIntrinsics(M); }
+  StringRef getPassName() const override { return "DXIL Op Lowering"; }
+  DXILOpLoweringLegacy() : ModulePass(ID) {}
+
+  static char ID; // Pass identification.
+};
+char DXILOpLoweringLegacy::ID = 0;
+
+} // end anonymous namespace
+
+INITIALIZE_PASS_BEGIN(DXILOpLoweringLegacy, DEBUG_TYPE, "DXIL Op Lowering",
+                      false, false)
+INITIALIZE_PASS_END(DXILOpLoweringLegacy, DEBUG_TYPE, "DXIL Op Lowering", false,
+                    false)
+
+ModulePass *llvm::createDXILOpLoweringLegacyPass() {
+  return new DXILOpLoweringLegacy();
+}

llvm/lib/Target/DirectX/DirectX.h

 /// Initializer for dxil writer pass
 void initializeWriteDXILPassPass(PassRegistry &);
 
 /// Initializer for DXIL-prepare
 void initializeDXILPrepareModulePass(PassRegistry &);
 
 /// Pass to convert modules into DXIL-compatable modules
 ModulePass *createDXILPrepareModulePass();
+
+/// Initializer for DXILOpLowering
+void initializeDXILOpLoweringLegacyPass(PassRegistry &);
+
+/// Pass to lowering LLVM intrinsic call to DXIL op function call.
+ModulePass *createDXILOpLoweringLegacyPass();
+
 } // namespace llvm
 
 #endif // LLVM_LIB_TARGET_DIRECTX_DIRECTX_H

llvm/lib/Target/DirectX/DirectXTargetMachine.cpp

 #include "llvm/Target/TargetLoweringObjectFile.h"
 
 using namespace llvm;
 
 extern "C" LLVM_EXTERNAL_VISIBILITY void LLVMInitializeDirectXTarget() {
   RegisterTargetMachine<DirectXTargetMachine> X(getTheDirectXTarget());
   auto *PR = PassRegistry::getPassRegistry();
   initializeDXILPrepareModulePass(*PR);
+  initializeDXILOpLoweringLegacyPass(*PR);
 }
 
 class DXILTargetObjectFile : public TargetLoweringObjectFile {
 public:
   DXILTargetObjectFile() = default;
 
   MCSection *getExplicitSectionGlobal(const GlobalObject *GO, SectionKind Kind,
                                       const TargetMachine &TM) const override {
: LLVMTargetMachine(T,
       Subtarget(std::make_unique<DirectXSubtarget>(TT, CPU, FS, *this)) {}
 
 DirectXTargetMachine::~DirectXTargetMachine() {}
 
 bool DirectXTargetMachine::addPassesToEmitFile(
     PassManagerBase &PM, raw_pwrite_stream &Out, raw_pwrite_stream *DwoOut,
     CodeGenFileType FileType, bool DisableVerify,
     MachineModuleInfoWrapperPass *MMIWP) {
+  PM.add(createDXILOpLoweringLegacyPass());
   PM.add(createDXILPrepareModulePass());
   switch (FileType) {
   case CGFT_AssemblyFile:
     PM.add(createPrintModulePass(Out, "", true));
     break;
   case CGFT_ObjectFile:
     // TODO: Use MC Object streamer to write DXContainer
     PM.add(createDXILWriterPass(Out));

llvm/test/CodeGen/DirectX/sin.ll

+; RUN: opt -S -dxil-op-lower < %s | FileCheck %s
+
+; Make sure dxil operation function calls for sin are generated for float and half.
+; CHECK:call float @dx.op.unary.f32(i32 13, float %{{.*}})
+; CHECK:call half @dx.op.unary.f16(i32 13, half %{{.*}})
+
+target datalayout = "e-m:e-p:32:32-i1:32-i8:8-i16:16-i32:32-i64:64-f16:16-f32:32-f64:64-n8:16:32:64"
+target triple = "dxil-pc-shadermodel6.7-library"
+
+; Function Attrs: noinline nounwind optnone
+define noundef float @_Z3foof(float noundef %a) #0 {
+entry:
+  %a.addr = alloca float, align 4
+  store float %a, ptr %a.addr, align 4
+  %0 = load float, ptr %a.addr, align 4
+  %1 = call float @llvm.sin.f32(float %0)
+  ret float %1
+}
+
+; Function Attrs: nocallback nofree nosync nounwind readnone speculatable willreturn
+declare float @llvm.sin.f32(float) #1
+
+; Function Attrs: noinline nounwind optnone
+define noundef half @_Z3barDh(half noundef %a) #0 {
+entry:
+  %a.addr = alloca half, align 2
+  store half %a, ptr %a.addr, align 2
+  %0 = load half, ptr %a.addr, align 2
+  %1 = call half @llvm.sin.f16(half %0)
+  ret half %1
+}
+
+; Function Attrs: nocallback nofree nosync nounwind readnone speculatable willreturn
+declare half @llvm.sin.f16(half) #1
+
+attributes #0 = { noinline nounwind optnone "frame-pointer"="none" "min-legal-vector-width"="0" "no-trapping-math"="true" "stack-protector-buffer-size"="8" }
+attributes #1 = { nocallback nofree nosync nounwind readnone speculatable willreturn }
+
+!llvm.module.flags = !{!0}
+!llvm.ident = !{!1}
+
+!0 = !{i32 1, !"wchar_size", i32 4}
+!1 = !{!"clang version 15.0.0 (https://github.com/llvm/llvm-project.git 73417c517644db5c419c85c0b3cb6750172fcab5)"}

llvm/tools/opt/opt.cpp

std::vector<StringRef> PassNameExactToIgnore = {
       "amdgpu-always-inline"};
   if (llvm::is_contained(PassNameExactToIgnore, Pass))
     return false;
 
   std::vector<StringRef> PassNamePrefix = {
       "x86-",    "xcore-", "wasm-",  "systemz-", "ppc-",    "nvvm-",
       "nvptx-",  "mips-",  "lanai-", "hexagon-", "bpf-",    "avr-",
       "thumb2-", "arm-",   "si-",    "gcn-",     "amdgpu-", "aarch64-",
-      "amdgcn-", "polly-", "riscv-"};
+      "amdgcn-", "polly-", "riscv-", "dxil-"};
   std::vector<StringRef> PassNameContain = {"ehprepare"};
   std::vector<StringRef> PassNameExact = {
       "safe-stack",           "cost-model",
       "codegenprepare",       "interleaved-load-combine",
       "unreachableblockelim", "verify-safepoint-ir",
       "atomic-expand",        "expandvp",
       "hardware-loops",       "type-promotion",
       "mve-tail-predication", "interleaved-access",