[WIP] Supporting C++ based kernel languages on AMDGPU Target

[yaxunl]

Summary

In C++ based languages targeting GPUGPU, e.g. HCC, OpenMP or Cuda, it is desirable to keep the AST free of explicit address space qualifier for simplified parsing and type checking. We can assume that all the pointers are generic pointers which can pointing to any real address spaces in the target IR. This results in a consistent and clean AST and therefore IR since every pointer is a generic pointer.

In Clang codegen, when we need to emit global variables or allocas in real address space, we cast the resulting pointer to a generic pointer and put them in the address map. Later on, when we need to refer to the address, we always get a generic pointer. In this way, we can keep the translation of AST consistent.

Currently Clang assumes that the address space for a generic pointer (a pointer which can pointing to any real memory region) is 0, which is correct for x86 and nvptx target, but is not correct for some other targets, e.g., amdgcn target, for which the generic address space is 4.

This patch introduce a virtual function TargetInfo::getDefaultTargetAddressSpace which allows each target to specify the default target address space based on language options. It also introduces ASTContext::getTargetGlobalAddressSpace and ASTContext::getTargetConstantAddressSpace as an abstraction for a unified approach to support different languages on address space aware targets, e.g. amdgcn and nvptx.

It also fixes various places in codegen for incorrect assumptions about default address space.

With this change, Clang will be able to generate correct IR for C++ based kernel languages e.g. HCC, OpenMP and Cuda for a greater range of targets.

It will not change the emitted IR for targets using 0 as generic address space. For these targets, since alloca returns generic pointer, they usually require some LLVM passes to cast alloca to real address space. This patch can help those targets generate correct alloca in Clang codegen.

This patch was co-authored by Yaxun (Sam) Liu and Wen-Heng (Jack) Chung.

[rjmccall]

What address space should local variables be in?

[yaxunl]

In D27627#618597, @rjmccall wrote:

What address space should local variables be in?

should always be 0 since that's llvm convention.

[rjmccall]

Ok, I'm not understanding this, then.

I declare a local variable:

int x;

According to you, x should be in the generic address space, i.e. the private address space. And at the LLVM IR level, this will be implemented with an AllocaInst, which always creates memory in the private address space, i.e. LLVM address space 0.

But Sema believes that the l-value 'x' has type int, without an address space, and therefore that '&x' has type int*, again without an explicit address space. And you want to say that that type actually points into a different address space, and to make sure that that pointer is in LLVM address space 4.

Something about this does not work.

[rjmccall]

I suspect that what you actually want to do in order to implement HCC is change SemaType so that 'int*' is implicitly interpreted as 'int __something *'. But I don't know how not having explicit address spaces actually works in the HCC language design, given that it's based on C++.

[rjmccall]

Because, notably, if you do that, then an attempt to pass &x as an int* will fail, which means this isn't really C++ anymore... and yet that appears to be exactly what you want.

[yaxunl]

Because, notably, if you do that, then an attempt to pass &x as an int* will fail, which means this isn't really C++ anymore... and yet that appears to be exactly what you want.

How about when generating alloca instruction, I insert addrspacecast to the default address space, then everything is in default address space.

[rjmccall]

In D27627#619928, @yaxunl wrote:

Because, notably, if you do that, then an attempt to pass &x as an int* will fail, which means this isn't really C++ anymore... and yet that appears to be exactly what you want.

How about when generating alloca instruction, I insert addrspacecast to the default address space, then everything is in default address space.

My question is really about your language design. You have multiple address spaces in the implementation, but you're (apparently?) pretending that they're all part of the same address space in the source language. How is that expected to work? Does the default address space embed all other address spaces?

[yaxunl]

In D27627#621473, @rjmccall wrote:

In D27627#619928, @yaxunl wrote:

Because, notably, if you do that, then an attempt to pass &x as an int* will fail, which means this isn't really C++ anymore... and yet that appears to be exactly what you want.

How about when generating alloca instruction, I insert addrspacecast to the default address space, then everything is in default address space.

My question is really about your language design. You have multiple address spaces in the implementation, but you're (apparently?) pretending that they're all part of the same address space in the source language. How is that expected to work? Does the default address space embed all other address spaces?

Yes the default address space embeds all other address spaces.

[yaxunl]

Cast alloca to default address space.

[yaxunl]

In D27627#621533, @yaxunl wrote:

In D27627#621473, @rjmccall wrote:

In D27627#619928, @yaxunl wrote:

Because, notably, if you do that, then an attempt to pass &x as an int* will fail, which means this isn't really C++ anymore... and yet that appears to be exactly what you want.

How about when generating alloca instruction, I insert addrspacecast to the default address space, then everything is in default address space.

My question is really about your language design. You have multiple address spaces in the implementation, but you're (apparently?) pretending that they're all part of the same address space in the source language. How is that expected to work? Does the default address space embed all other address spaces?

Yes the default address space embeds all other address spaces.

Since the default address space (generic address space 4) can represent all other address spaces, we want to use it uniformly. For any pointer which is not in the default address space, e.g. alloca, we want to cast it to default address space and then use it. Therefore, in the address space agnostic languages, we can assume everything is in the default address space. Our backend is able to handle load/store in the default address space.

[yaxunl]

Synch up to trunk.
Change bitcast to pointer cast in varous places to accommodation addr space difference.

[yaxunl]

Fix invalid bitcast for global variables.
Add run line for amdgcn target for a Cuda lit test.

[yaxunl]

Fixed invalid bitcasts due to vtable.
Added run line for amdgcn target to more Cuda lit tests.

[kzhuravl]

Once you rebase, can you also update AMDGPUTargetInfo::getDWARFAddressSpace to use AddrSpaceKind?

[yaxunl]

We consider to switch to use 0 as generic address space and abandon this since 0 as generic address space is more agreeable to both LLVM and Clang's convention.

[rjmccall]

Okay. That does seem more sensible than trying to change everything around the other way.

include/clang/AST/ASTContext.h

   /// \param typeSize a real floating point or complex type.
   QualType getFloatingTypeOfSizeWithinDomain(QualType typeSize,
                                              QualType typeDomain) const;
 
   unsigned getTargetAddressSpace(QualType T) const {
     return getTargetAddressSpace(T.getQualifiers());
   }
 
-  unsigned getTargetAddressSpace(Qualifiers Q) const {
-    return getTargetAddressSpace(Q.getAddressSpace());
-  }
+  unsigned getTargetAddressSpace(Qualifiers Q) const;
 
   unsigned getTargetAddressSpace(unsigned AS) const {
     if (AS < LangAS::Offset || AS >= LangAS::Offset + LangAS::Count)
       return AS;
     else
       return (*AddrSpaceMap)[AS - LangAS::Offset];
   }
 

include/clang/Basic/TargetInfo.h

public:
   virtual const char *getStaticInitSectionSpecifier() const {
     return nullptr;
   }
 
   const LangAS::Map &getAddressSpaceMap() const {
     return *AddrSpaceMap;
   }
 
+  virtual unsigned getDefaultTargetAddressSpace(LangOptions &Opt) const {
+    return 0;
+  }
+
   /// \brief Retrieve the name of the platform as it is used in the
   /// availability attribute.
   StringRef getPlatformName() const { return PlatformName; }
 
   /// \brief Retrieve the minimum desired version of the platform, to
   /// which the program should be compiled.
   VersionTuple getPlatformMinVersion() const { return PlatformMinVersion; }
 

lib/AST/ASTContext.cpp

   if (QT->getUnqualifiedDesugaredType()->isNullPtrType())
     AS = 0;
   else
     AS = QT->getPointeeType().getAddressSpace();
 
   return getTargetInfo().getNullPointerValue(AS);
 }
 
+unsigned ASTContext::getTargetAddressSpace(Qualifiers Q) const {
+  return Q.hasAddressSpace()
+             ? getTargetAddressSpace(Q.getAddressSpace())
+             : getTargetInfo().getDefaultTargetAddressSpace(LangOpts);
+}
+
 // Explicitly instantiate this in case a Redeclarable<T> is used from a TU that
 // doesn't include ASTContext.h
 template
 clang::LazyGenerationalUpdatePtr<
     const Decl *, Decl *, &ExternalASTSource::CompleteRedeclChain>::ValueType
 clang::LazyGenerationalUpdatePtr<
     const Decl *, Decl *, &ExternalASTSource::CompleteRedeclChain>::makeValue(
         const clang::ASTContext &Ctx, Decl *Value);

lib/Basic/Targets.cpp

     PointerWidth = PointerAlign = 64;
     SizeType = TargetInfo::UnsignedLong;
     PtrDiffType = TargetInfo::SignedLong;
     IntPtrType = TargetInfo::SignedLong;
     resetDataLayout("e-i64:64-v16:16-v32:32-n16:32:64");
   }
 };
 
-static const unsigned AMDGPUAddrSpaceMap[] = {
-  1,    // opencl_global
-  3,    // opencl_local
-  2,    // opencl_constant
-  4,    // opencl_generic
-  1,    // cuda_device
-  2,    // cuda_constant
-  3     // cuda_shared
-};
-
 // If you edit the description strings, make sure you update
 // getPointerWidthV().
 
 static const char *const DataLayoutStringR600 =
   "e-p:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128"
   "-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64";
 
 static const char *const DataLayoutStringSI =
   "e-p:32:32-p1:64:64-p2:64:64-p3:32:32-p4:64:64-p5:32:32"
   "-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128"
   "-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64";
 
 class AMDGPUTargetInfo final : public TargetInfo {
+  static const unsigned AddrSpaceMap_[7];
   static const Builtin::Info BuiltinInfo[];
   static const char * const GCCRegNames[];
 
+  enum AddrSpaceKind {
+    AS_Private = 0,
+    AS_Global = 1,
+    AS_Constant = 2,
+    AS_Local = 3,
+    AS_Generic = 4
+  };
+
   /// \brief The GPU profiles supported by the AMDGPU target.
   enum GPUKind {
     GK_NONE,
     GK_R600,
     GK_R600_DOUBLE_OPS,
     GK_R700,
     GK_R700_DOUBLE_OPS,
     GK_EVERGREEN,

       hasFP64 = true;
       hasFMAF = true;
       hasLDEXPF = true;
     }
 
     resetDataLayout(getTriple().getArch() == llvm::Triple::amdgcn ?
                     DataLayoutStringSI : DataLayoutStringR600);
 
-    AddrSpaceMap = &AMDGPUAddrSpaceMap;
+    AddrSpaceMap = &AddrSpaceMap_;
     UseAddrSpaceMapMangling = true;
   }
 
   uint64_t getPointerWidthV(unsigned AddrSpace) const override {
     if (GPU <= GK_CAYMAN)
       return 32;
 
     switch(AddrSpace) {

       Opts.support("cl_khr_mipmap_image");
       Opts.support("cl_khr_subgroups");
       Opts.support("cl_khr_3d_image_writes");
       Opts.support("cl_amd_media_ops");
       Opts.support("cl_amd_media_ops2");
     }
   }
 
+  unsigned getDefaultTargetAddressSpace(LangOptions &Opts) const override {
+    // OpenCL sets address space explicitly in AST. The default case (type
+    // qualifier containing no address space) represents private address space.
+    if (Opts.OpenCL)
+      return AS_Private;
+    return AS_Generic;
+  }
+
   LangAS::ID getOpenCLImageAddrSpace() const override {
     return LangAS::opencl_constant;
   }
 
   CallingConvCheckResult checkCallingConvention(CallingConv CC) const override {
     switch (CC) {
       default:
         return CCCR_Warning;
       case CC_C:
       case CC_OpenCLKernel:
         return CCCR_OK;
     }
   }
 
   // In amdgcn target the null pointer in global, constant, and generic
   // address space has value 0 but in private and local address space has
   // value ~0.
   uint64_t getNullPointerValue(unsigned AS) const override {
     return AS != LangAS::opencl_local && AS != 0 ? 0 : ~0;
   }
 };
 
+const unsigned AMDGPUTargetInfo::AddrSpaceMap_[] = {
+    AS_Global,   // opencl_global
+    AS_Local,    // opencl_local
+    AS_Constant, // opencl_constant
+    AS_Generic,  // opencl_generic
+    AS_Global,   // cuda_device
+    AS_Constant, // cuda_constant
+    AS_Local     // cuda_shared
+};
+
 const Builtin::Info AMDGPUTargetInfo::BuiltinInfo[] = {
 #define BUILTIN(ID, TYPE, ATTRS)                \
   { #ID, TYPE, ATTRS, nullptr, ALL_LANGUAGES, nullptr },
 #define TARGET_BUILTIN(ID, TYPE, ATTRS, FEATURE)                               \
   { #ID, TYPE, ATTRS, nullptr, ALL_LANGUAGES, FEATURE },
 #include "clang/Basic/BuiltinsAMDGPU.def"
 };
 const char * const AMDGPUTargetInfo::GCCRegNames[] = {

test/CodeGen/default-addr-space.cpp

+// RUN: %clang_cc1 -O0 -std=c++11 -emit-llvm -o - -triple amdgcn-- %s | FileCheck %s
+
+// CHECK-LABEL: @_Z1fPi(i32 addrspace(4)* %a)
+// CHECK:  %[[a_addr:.*]] = alloca i32 addrspace(4)*
+// CHECK:  %b = alloca i32
+// CHECK:  store i32 addrspace(4)* %a, i32 addrspace(4)** %[[a_addr]]
+// CHECK:  store i32 1, i32* %b
+// CHECK:  %[[r0:.*]] = load i32, i32* %b
+// CHECK:  %[[r1:.*]] = load i32 addrspace(4)*, i32 addrspace(4)** %[[a_addr]]
+// CHECK:  store i32 %[[r0]], i32 addrspace(4)* %[[r1]]
+void f(int* a) {
+  int b = 1;
+  *a = b;
+}