[HLSL] CodeGen hlsl cbuffer/tbuffer.

[python3kgae]

Summary

cbuffer A {

float a;
float b;

}

will be translated to a global variable.

Something like

struct CB_Ty {

float a;
float b;

};

CB_Ty A;

And all use of a and b will be replaced with A.a and A.b.

Only support none-legacy cbuffer layout now.
CodeGen for Resource binding will be in separate patch.
In the separate patch, resource binding will map the resource information to the global variable.

[Anastasia]

clang/lib/CodeGen/CGHLSLRuntime.cpp

61

I don't think I understand the intent of this function along with CGHLSLRuntime::addConstant that populates this collection.

62

It might be better to avoid using hard-coded constants. Are you adding new entires in clang's AddressSpace enum to represent different logical memory segment of the language?

158

Is this case covered by the test?

clang/lib/CodeGen/CGHLSLRuntime.h

42

what does this field represent?

[python3kgae]

clang/lib/CodeGen/CGHLSLRuntime.cpp

158

Nice catch.
I'll add a test for this.

clang/lib/CodeGen/CGHLSLRuntime.h

42

The field is for marking the CBuffer as a cbuffer or tbuffer.
The name is confusing.
How about adding an enum BufferKind { CBuffer, TBuffer };?

[python3kgae]

clang/lib/CodeGen/CGHLSLRuntime.cpp

61

It is translating

 cbuffer A {
   float a;
   float b;
}
float foo() {
  return a + b;
}

into

struct cb.A.Ty {
  float a;
  float b;
};

cbuffer A {
  cb.A.Ty CBA;
}
float foo() {
  return CBA.a + CBA.b;
}

Both a and b are in the global scope and will get a GlobalVariable in clang codeGen.
By doing the translation, we can ensure each buffer map to one GlobalVariable and save cbuffer layout in the value type of that GlobalVariable.

62

Thanks for pointing it out. I'll add the new address space to LangAS.

[python3kgae]

Take care alignment when layout CBuffer.
Add hlsl_cbuffer/tbuffer to clang::LangAS.
Change CGHLSLRuntime::CBuffer to CGHLSLRuntime::Buffer to match HLSLBufferDecl.

[python3kgae]

Add more comments.

[python3kgae]

clang/lib/CodeGen/CGHLSLRuntime.cpp

158

Cannot add a test before other HLSL resource type like Texture2D is supported :(

clang/lib/CodeGen/CGHLSLRuntime.h

42

Change struct CBuffer into struct Buffer to match HLSLBufferDecl.

[beanz]

clang/lib/CodeGen/CGHLSLRuntime.cpp

158

Texture2D will be a CXXRecordDecl, not a HLSLBufferDecl, we only need buffer decls for the legacy buffers. So this should be testable by nesting a cbuffer inside a cbuffer right?

[python3kgae]

clang/lib/CodeGen/CGHLSLRuntime.cpp

158

I see.
I was thinking about test the error path.
I'll add a test for nested cbuffer.

[beanz]

clang/lib/Basic/Targets/SPIR.h

46

I don’t think you need this comment. Most of those adds space values aren’t applicable to these targets, that’s why they are all 0.

clang/lib/CodeGen/CGHLSLRuntime.cpp

185

Why look both of these values up when you’re only going to use one?

189

nit: Don’t use auto in places where the type isn’t obvious from the expression.

204

nit: Rather than using int max as a sentinel here, why not use an llvm::Optional?

[python3kgae]

Add test for nested cbuffer.

[python3kgae]

Remove useless comment.
Use llvm::Optional.
Use real type instead of auto.

[Anastasia]

clang/lib/CodeGen/CGHLSLRuntime.cpp

61

Ok, I see, so if we are to translate it to C it would be something similar to:

struct A {
   float a;
   float b;
} cbuffer_A __attribute__((address_space(256)));

float foo() {
  return cbuffer_A.a + cbuffer_A.b;
}

Maybe you can add some comments to explain the intent of this code at a higher level... not sure if the generation can reuse or be made a bit close to the regular C structs + address spaces...

119

btw is this covered in the test?

152

Ok I think you don't have this in the tests too and the one below?

183

I think it's better to use getTargetAddressSpace from ASTContext.

clang/test/CodeGenHLSL/nest_cbuf.hlsl

8

is this generated as nested structs?

[python3kgae]

Add more test and comment.
Also use getTargetAddressSpace.

[python3kgae]

clang/lib/CodeGen/CGHLSLRuntime.cpp

61

Added comments.
Will check how union is supported in clang codeGen. The behavior feels similar, hope could share some code.

clang/test/CodeGenHLSL/nest_cbuf.hlsl

8

No. This will generate as two separate structs.

[beanz]

clang/lib/CodeGen/CGHLSLRuntime.cpp

95

Why are you manually inserting padding?

IR level accesses don't require explicit layout, and we don't do this in DXC either.

[beanz]

Now that I'm seeing the code in D131370, I don't know that this is the right way to do things.

I think using address spaces like this is odd. Address spaces aren't really intended for use differentiating high level access types, but rather memory regions and properties of that memory. I feel like the use of address spaces in this change and in D131370 just makes it more complicated and gets in the way.

In DXIL, since we don't really have raw memory load and stores, none of these address spaces mean anything. It is probably just cleaner to not use address spaces in the CodeGen here either.

[python3kgae]

In D130131#3720552, @beanz wrote:

Now that I'm seeing the code in D131370, I don't know that this is the right way to do things.

I think using address spaces like this is odd. Address spaces aren't really intended for use differentiating high level access types, but rather memory regions and properties of that memory. I feel like the use of address spaces in this change and in D131370 just makes it more complicated and gets in the way.

In DXIL, since we don't really have raw memory load and stores, none of these address spaces mean anything. It is probably just cleaner to not use address spaces in the CodeGen here either.

If not use address space, how do llvm passes know a cbuffer ptr is different from a tbuffer ptr or ptr to a static global variable? Or llvm passes don't need to know the difference?

clang/lib/CodeGen/CGHLSLRuntime.cpp

95

Does DL take care of the alignment

[python3kgae]

Remove address space and manual align.

[python3kgae]

In D130131#3720552, @beanz wrote:

Now that I'm seeing the code in D131370, I don't know that this is the right way to do things.

I think using address spaces like this is odd. Address spaces aren't really intended for use differentiating high level access types, but rather memory regions and properties of that memory. I feel like the use of address spaces in this change and in D131370 just makes it more complicated and gets in the way.

In DXIL, since we don't really have raw memory load and stores, none of these address spaces mean anything. It is probably just cleaner to not use address spaces in the CodeGen here either.

Removed address space.
Also removed nested HLSLBuffer and namespace inside HLSLBuffer.

[python3kgae]

Remove nested HLSLBuffer and namespace inside HLSLBuffer.

[aaron.ballman]

clang/lib/CodeGen/CGHLSLRuntime.cpp

74–76

Please spell out these types.

94–95

Please spell out these types.

101

Same here

143

145–148

178

Please spell out the type.

clang/lib/CodeGen/CGHLSLRuntime.h

40

58

[python3kgae]

Code cleanup to match comments

[beanz]

This is looking great. Thank you for all your work iterating on this.

Please update the description to reflect the changes in implementation.

clang/lib/CodeGen/CGHLSLRuntime.cpp

113

Please remove the empty function. We can add it when it does something. Also make sure we have an issue for the FIXME.

133

Please file an issue for this or link to the issue if it exists.

[python3kgae]

Create issue for FIXME.

[aaron.ballman]

No additional comments from me, but I don't feel comfortable signing off on the codegen bits; adding some codegen code owners.

[beanz]

+@asl for codegen owner perspective.

This LGTM too. The changes here are well isolated to HLSL, so they should have no adverse impact on other language support.

@efriedma, @asl & @rjmccall any feedback?

[python3kgae]

gentle ping.

[efriedma]

clang/lib/CodeGen/CGDecl.cpp

131

I'm a little confused by this. If it's possible to declare an HLSLBuffer inside a function, why don't you need to handle it? If it isn't possible to declare an HLSLBuffer this way, can you just move this to use the llvm_unreachable()?

clang/lib/CodeGen/CGHLSLRuntime.cpp

105

Messing with globals like this feels a little weird, but I guess it's fine if nothing actually tries to use the erased globals after this code runs. I'm a little concerned that someone might accidentally rearrange the relevant code in the future (CodeGenModule has a bunch of maps which aren't cleared before this code runs).

[python3kgae]

Go llvm_unreachable is hit HLSLBuffer inside function.

[python3kgae]

clang/lib/CodeGen/CGDecl.cpp

131

Nice catch.
Fixed.

clang/lib/CodeGen/CGHLSLRuntime.cpp

105

These globals should not be in any map except Buf.Constants which is used here.
If another map has these globals, we should remove them from the map.
Cannot see any other map has these globals now.

[efriedma]

clang/lib/CodeGen/CGHLSLRuntime.cpp

105

Oh, I see. Probably not an issue in that case.

clang/include/clang/Basic/AddressSpaces.h

enum class LangAS : unsigned {
   sycl_local,
   sycl_private,
 
   // Pointer size and extension address spaces.
   ptr32_sptr,
   ptr32_uptr,
   ptr64,
 
+  // HLSL specific address spaces.
+  hlsl_cbuffer,
+  hlsl_tbuffer,
+
   // This denotes the count of language-specific address spaces and also
   // the offset added to the target-specific address spaces, which are usually
   // specified by address space attributes __attribute__(address_space(n))).
   FirstTargetAddressSpace
 };
 
 /// The type of a lookup table which maps from language-specific address spaces
 /// to target-specific ones.

clang/lib/AST/ASTContext.cpp

static const unsigned FakeAddrSpaceMap[] = {
         9,  // cuda_shared
         1,  // sycl_global
         5,  // sycl_global_device
         6,  // sycl_global_host
         3,  // sycl_local
         0,  // sycl_private
         10, // ptr32_sptr
         11, // ptr32_uptr
-        12  // ptr64
+        12, // ptr64
+        13, // hlsl_cbuffer
+        14, // hlsl_tbuffer
     };
     return &FakeAddrSpaceMap;
   } else {
     return &T.getAddressSpaceMap();
   }
 }
 
 static bool isAddrSpaceMapManglingEnabled(const TargetInfo &TI,

clang/lib/Basic/Targets/AMDGPU.cpp

const LangASMap AMDGPUTargetInfo::AMDGPUDefIsGenMap = {
     Local,    // cuda_shared
     Global,   // sycl_global
     Global,   // sycl_global_device
     Global,   // sycl_global_host
     Local,    // sycl_local
     Private,  // sycl_private
     Generic,  // ptr32_sptr
     Generic,  // ptr32_uptr
-    Generic   // ptr64
+    Generic,  // ptr64
+    0,        // hlsl_cbuffer
+    0,        // hlsl_tbuffer
 };
 
 const LangASMap AMDGPUTargetInfo::AMDGPUDefIsPrivMap = {
     Private,  // Default
     Global,   // opencl_global
     Local,    // opencl_local
     Constant, // opencl_constant
     Private,  // opencl_private
     Generic,  // opencl_generic
     Global,   // opencl_global_device
     Global,   // opencl_global_host
     Global,   // cuda_device
     Constant, // cuda_constant
     Local,    // cuda_shared
     // SYCL address space values for this map are dummy
     Generic,  // sycl_global
     Generic,  // sycl_global_device
     Generic,  // sycl_global_host
     Generic,  // sycl_local
     Generic,  // sycl_private
     Generic,  // ptr32_sptr
     Generic,  // ptr32_uptr
-    Generic   // ptr64
+    Generic,  // ptr64
+    0,        // hlsl_cbuffer
+    0,        // hlsl_tbuffer
 
 };
 } // namespace targets
 } // namespace clang
 
 const Builtin::Info AMDGPUTargetInfo::BuiltinInfo[] = {
 #define BUILTIN(ID, TYPE, ATTRS)                                               \
   {#ID, TYPE, ATTRS, nullptr, ALL_LANGUAGES, nullptr},

clang/lib/Basic/Targets/DirectX.h

static const unsigned DirectXAddrSpaceMap[] = {
     // SYCL address space values for this map are dummy
     0, // sycl_global
     0, // sycl_global_device
     0, // sycl_global_host
     0, // sycl_local
     0, // sycl_private
     0, // ptr32_sptr
     0, // ptr32_uptr
-    0  // ptr64
+    0, // ptr64
+    4, // hlsl_cbuffer
+    5, // hlsl_tbuffer
 };
 
 class LLVM_LIBRARY_VISIBILITY DirectXTargetInfo : public TargetInfo {
 public:
   DirectXTargetInfo(const llvm::Triple &Triple, const TargetOptions &)
       : TargetInfo(Triple) {
     TLSSupported = false;
     VLASupported = false;

clang/lib/Basic/Targets/NVPTX.h

static const unsigned NVPTXAddrSpaceMap[] = {
     3, // cuda_shared
     1, // sycl_global
     1, // sycl_global_device
     1, // sycl_global_host
     3, // sycl_local
     0, // sycl_private
     0, // ptr32_sptr
     0, // ptr32_uptr
-    0  // ptr64
+    0, // ptr64
+    0, // hlsl_cbuffer
+    0, // hlsl_tbuffer
 };
 
 /// The DWARF address class. Taken from
 /// https://docs.nvidia.com/cuda/archive/10.0/ptx-writers-guide-to-interoperability/index.html#cuda-specific-dwarf
 static const int NVPTXDWARFAddrSpaceMap[] = {
     -1, // Default, opencl_private or opencl_generic - not defined
     5,  // opencl_global
     -1,

clang/lib/Basic/Targets/SPIR.h

static const unsigned SPIRDefIsPrivMap[] = {
     // SYCL address space values for this map are dummy
     0, // sycl_global
     0, // sycl_global_device
     0, // sycl_global_host
     0, // sycl_local
     0, // sycl_private
     0, // ptr32_sptr
     0, // ptr32_uptr
-    0  // ptr64
+    0, // ptr64
+    0, // hlsl_cbuffer
+    0, // hlsl_tbuffer
 };
 
 // Used by both the SPIR and SPIR-V targets.
 static const unsigned SPIRDefIsGenMap[] = {
     4, // Default
     // OpenCL address space values for this map are dummy and they can't be used
     0, // opencl_global
     0, // opencl_local
static const unsigned SPIRDefIsGenMap[] = {
     3, // cuda_shared
     1, // sycl_global
     5, // sycl_global_device
     6, // sycl_global_host
     3, // sycl_local
     0, // sycl_private
     0, // ptr32_sptr
     0, // ptr32_uptr
-    0  // ptr64
+    0, // ptr64
+    0, // hlsl_cbuffer
+    0, // hlsl_tbuffer
 };
 
 // Base class for SPIR and SPIR-V target info.
 class LLVM_LIBRARY_VISIBILITY BaseSPIRTargetInfo : public TargetInfo {
 protected:
   BaseSPIRTargetInfo(const llvm::Triple &Triple, const TargetOptions &)
       : TargetInfo(Triple) {
     assert((Triple.isSPIR() || Triple.isSPIRV()) &&

clang/lib/Basic/Targets/TCE.h

static const unsigned TCEOpenCLAddrSpaceMap[] = {
     0, // sycl_global
     0, // sycl_global_device
     0, // sycl_global_host
     0, // sycl_local
     0, // sycl_private
     0, // ptr32_sptr
     0, // ptr32_uptr
     0, // ptr64
+    0, // hlsl_cbuffer
+    0, // hlsl_tbuffer
 };
 
 class LLVM_LIBRARY_VISIBILITY TCETargetInfo : public TargetInfo {
 public:
   TCETargetInfo(const llvm::Triple &Triple, const TargetOptions &)
       : TargetInfo(Triple) {
     TLSSupported = false;
     IntWidth = 32;

clang/lib/Basic/Targets/X86.h

static const unsigned X86AddrSpaceMap[] = {
     0,   // cuda_shared
     0,   // sycl_global
     0,   // sycl_global_device
     0,   // sycl_global_host
     0,   // sycl_local
     0,   // sycl_private
     270, // ptr32_sptr
     271, // ptr32_uptr
-    272  // ptr64
+    272, // ptr64
+    0,   // hlsl_cbuffer
+    0,   // hlsl_tbuffer
 };
 
 // X86 target abstract base class; x86-32 and x86-64 are very close, so
 // most of the implementation can be shared.
 class LLVM_LIBRARY_VISIBILITY X86TargetInfo : public TargetInfo {
 
   enum X86SSEEnum {
     NoSSE,

clang/lib/CodeGen/CGDecl.cpp

void CodeGenFunction::EmitDecl(const Decl &D) {
   case Decl::OMPThreadPrivate:
   case Decl::OMPAllocate:
   case Decl::OMPCapturedExpr:
   case Decl::OMPRequires:
   case Decl::Empty:
   case Decl::Concept:
   case Decl::LifetimeExtendedTemporary:
   case Decl::RequiresExprBody:
+  case Decl::HLSLBuffer:
     // None of these decls require codegen support.
     return;
 
   case Decl::NamespaceAlias:
     if (CGDebugInfo *DI = getDebugInfo())
         DI->EmitNamespaceAlias(cast<NamespaceAliasDecl>(D));
     return;
   case Decl::Using:          // using X; [C++]

clang/lib/CodeGen/CGHLSLRuntime.h

 // subclasses of this implement code generation for specific HLSL
 // runtime libraries.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_LIB_CODEGEN_CGHLSLRUNTIME_H
 #define LLVM_CLANG_LIB_CODEGEN_CGHLSLRUNTIME_H
 
+#include "llvm/ADT/Optional.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringRef.h"
+#include <vector>
+
+namespace llvm {
+class GlobalVariable;
+class StructType;
+} // namespace llvm
+
 namespace clang {
+class HLSLBufferDecl;
+class VarDecl;
+class DeclContext;
 
 namespace CodeGen {
 
 class CodeGenModule;
 
 class CGHLSLRuntime {
+public:
+  struct Buffer {
+    Buffer(HLSLBufferDecl *D);
+    llvm::StringRef Name;
+    // IsCBuffer - Whether the buffer is a cbuffer (and not a tbuffer).
+    bool IsCBuffer;
+    llvm::Optional<unsigned> Reg;
+    unsigned Space;
+    // Global variable and offset for each constant.
+    std::vector<std::pair<llvm::GlobalVariable *, unsigned>> Constants;
+    llvm::StructType *LayoutStruct = nullptr;
+  };
+
 protected:
   CodeGenModule &CGM;
 
 public:
   CGHLSLRuntime(CodeGenModule &CGM) : CGM(CGM) {}
   virtual ~CGHLSLRuntime() {}
-
+  void addBuffer(HLSLBufferDecl *D);
   void finishCodeGen();
+
+private:
+  void addConstant(VarDecl *D, Buffer &CB);
+  void addBufferDecls(DeclContext *DC, Buffer &CB);
+  llvm::SmallVector<Buffer> Buffers;
 };
 
 } // namespace CodeGen
 } // namespace clang
 
 #endif

clang/lib/CodeGen/CGHLSLRuntime.cpp

 //===----- CGHLSLRuntime.cpp - Interface to HLSL Runtimes -----------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This provides an abstract class for HLSL code generation.  Concrete
 // subclasses of this implement code generation for specific HLSL
 // runtime libraries.
 //
 //===----------------------------------------------------------------------===//
 
 #include "CGHLSLRuntime.h"
+#include "CGDebugInfo.h"
 #include "CodeGenModule.h"
+
 #include "clang/Basic/TargetOptions.h"
 #include "llvm/IR/Metadata.h"
 #include "llvm/IR/Module.h"
 
 using namespace clang;
 using namespace CodeGen;
 using namespace llvm;
 
 namespace {
+
 void addDxilValVersion(StringRef ValVersionStr, llvm::Module &M) {
   // The validation of ValVersionStr is done at HLSLToolChain::TranslateArgs.
   // Assume ValVersionStr is legal here.
   VersionTuple Version;
   if (Version.tryParse(ValVersionStr) || Version.getBuild() ||
       Version.getSubminor() || !Version.getMinor()) {
     return;
   }
 
   uint64_t Major = Version.getMajor();
   uint64_t Minor = *Version.getMinor();
 
   auto &Ctx = M.getContext();
   IRBuilder<> B(M.getContext());
   MDNode *Val = MDNode::get(Ctx, {ConstantAsMetadata::get(B.getInt32(Major)),
                                   ConstantAsMetadata::get(B.getInt32(Minor))});
   StringRef DxilValKey = "dx.valver";
   M.addModuleFlag(llvm::Module::ModFlagBehavior::AppendUnique, DxilValKey, Val);
 }
+
+void layoutBuffer(CGHLSLRuntime::Buffer &Buf, const DataLayout &DL) {
+  if (Buf.Constants.empty())
+    return;
+
+  // FIXME: support legacy cbuffer layout.
+  auto &Ctx = Buf.Constants[0].first->getContext();
+  std::vector<llvm::Type *> EltTys;
+  uint64_t Offset = 0;
+  for (auto &Const : Buf.Constants) {
+    auto *GV = Const.first;
+    Const.second = EltTys.size();
+    auto *Ty = GV->getValueType();
+    auto Align = DL.getPrefTypeAlign(Ty);
+    if (uint64_t PaddingSize = (Offset % Align.value())) {
+      // Not aligned.
+      // Adding padding.
+      auto *PaddingTy = IntegerType::get(Ctx, 8 * PaddingSize);
+      EltTys.emplace_back(PaddingTy);
+      Offset += PaddingSize;
+    }
+    // Save Index for Ty into Const.second.
+    Const.second = EltTys.size();
+    EltTys.emplace_back(Ty);
+
+    Offset += DL.getTypeAllocSize(Ty);
+  }
+  Buf.LayoutStruct = llvm::StructType::get(EltTys[0]->getContext(), EltTys);
+}
+
+GlobalVariable *replaceBuffer(CGHLSLRuntime::Buffer &Buf,
+                              const unsigned AddressSpace) {
+  // Create global variable for CB.
+  GlobalVariable *CBGV =
+      new GlobalVariable(Buf.LayoutStruct, /*isConstant*/ true,
+                         GlobalValue::LinkageTypes::ExternalLinkage, nullptr,
+                         Buf.Name + (Buf.IsCBuffer ? ".cb." : ".tb."),
+                         GlobalValue::NotThreadLocal, AddressSpace);
+
+  IRBuilder<> B(CBGV->getContext());
+  Value *ZeroIdx = B.getInt32(0);
+  // Replace Const use with CB use.
+  for (auto &Const : Buf.Constants) {
+    auto *EltTy = Buf.LayoutStruct->getElementType(Const.second);
+    auto *GV = Const.first;
+    unsigned Offset = Const.second;
+
+    Value *GEP =
+        B.CreateGEP(Buf.LayoutStruct, CBGV, {ZeroIdx, B.getInt32(Offset)});
+
+    auto *GVTy = GV->getValueType();
+    assert(EltTy == GVTy && "constant type mismatch");
+
+    // Cast address space.
+    GEP = B.CreateAddrSpaceCast(GEP, GVTy->getPointerTo());
+    // Replace.
+    GV->replaceAllUsesWith(GEP);
+    // Erase GV.
+    GV->removeDeadConstantUsers();
+    GV->eraseFromParent();
+  }
+  return CBGV;
+}
+
+void addResourceBinding(GlobalVariable *GV, CGHLSLRuntime::Buffer &CB) {
+  // FIXME: add resource binding to GV.
+}
+
 } // namespace
 
+void CGHLSLRuntime::addConstant(VarDecl *D, Buffer &CB) {
+  if (D->getStorageClass() == SC_Static) {
+    // For static inside cbuffer, take as global static.
+    // Don't add to cbuffer.
+    CGM.EmitGlobal(D);
+    return;
+  }
+
+  auto *GV = cast<GlobalVariable>(CGM.GetAddrOfGlobalVar(D));
+  // Add debug info for constVal.
+  if (CGDebugInfo *DI = CGM.getModuleDebugInfo())
+    if (CGM.getCodeGenOpts().getDebugInfo() >=
+        codegenoptions::DebugInfoKind::LimitedDebugInfo)
+      DI->EmitGlobalVariable(cast<GlobalVariable>(GV), D);
+
+  // FIXME: support packoffset.
+  uint32_t Offset = 0;
+  bool HasUserOffset = false;
+
+  unsigned LowerBound = HasUserOffset ? Offset : UINT_MAX;
+  CB.Constants.emplace_back(std::make_pair(GV, LowerBound));
+}
+
+void CGHLSLRuntime::addBufferDecls(DeclContext *DC, Buffer &CB) {
+  for (Decl *it : DC->decls()) {
+    if (VarDecl *ConstDecl = dyn_cast<VarDecl>(it)) {
+      addConstant(ConstDecl, CB);
+    } else if (isa<EmptyDecl>(*it)) {
+      // Nothing to do for this declaration.
+    } else if (isa<CXXRecordDecl>(it)) {
+      // Nothing to do for this declaration.
+    } else if (isa<FunctionDecl>(it)) {
+      // A function within an cbuffer is effectively a top-level function,
+      // as it only refers to globally scoped declarations.
+      CGM.EmitTopLevelDecl(it);
+    } else if (NamespaceDecl *ND = dyn_cast<NamespaceDecl>(it)) {
+      addBufferDecls(ND, CB);
+    } else {
+      HLSLBufferDecl *Inner = dyn_cast<HLSLBufferDecl>(it);
+      if (!Inner) {
+        // FIXME: add test after more Resource Type like Texture2D is supported.
+        DiagnosticsEngine &Diags = CGM.getDiags();
+        unsigned DiagID = Diags.getCustomDiagID(DiagnosticsEngine::Error,
+                                                "invalid decl inside cbuffer");
+        Diags.Report(it->getLocation(), DiagID);
+        return;
+      }
+      addBuffer(Inner);
+    }
+  }
+}
+
+void CGHLSLRuntime::addBuffer(HLSLBufferDecl *D) {
+  Buffers.emplace_back(Buffer(D));
+  addBufferDecls(D, Buffers.back());
+}
+
 void CGHLSLRuntime::finishCodeGen() {
   auto &TargetOpts = CGM.getTarget().getTargetOpts();
 
   llvm::Module &M = CGM.getModule();
   addDxilValVersion(TargetOpts.DxilValidatorVersion, M);
+  auto &DL = M.getDataLayout();
+  auto &ASMap = CGM.getTarget().getAddressSpaceMap();
+  const unsigned CBufferAddressSpace =
+      ASMap[(unsigned)clang::LangAS::hlsl_cbuffer];
+  const unsigned TBufferAddressSpace =
+      ASMap[(unsigned)clang::LangAS::hlsl_tbuffer];
+
+  for (auto &Buf : Buffers) {
+    layoutBuffer(Buf, DL);
+    const unsigned AddressSapce =
+        Buf.IsCBuffer ? CBufferAddressSpace : TBufferAddressSpace;
+    GlobalVariable *GV = replaceBuffer(Buf, AddressSapce);
+    M.getGlobalList().push_back(GV);
+    addResourceBinding(GV, Buf);
+  }
+}
+
+CGHLSLRuntime::Buffer::Buffer(HLSLBufferDecl *D) {
+  Name = D->getName();
+  IsCBuffer = D->isCBuffer();
+  if (auto *Binding = D->getAttr<HLSLResourceBindingAttr>()) {
+    Reg = Binding->getID();
+    Space = Binding->getSpace();
+  } else {
+    Space = 0;
+  }
 }

clang/lib/CodeGen/CodeGenModule.cpp

case Decl::Record:
     break;
 
   case Decl::Enum:
     if (CGDebugInfo *DI = getModuleDebugInfo())
       if (cast<EnumDecl>(D)->getDefinition())
         DI->EmitAndRetainType(getContext().getEnumType(cast<EnumDecl>(D)));
     break;
 
+  case Decl::HLSLBuffer:
+    getHLSLRuntime().addBuffer(cast<HLSLBufferDecl>(D));
+    break;
+
   default:
     // Make sure we handled everything we should, every other kind is a
     // non-top-level decl.  FIXME: Would be nice to have an isTopLevelDeclKind
     // function. Need to recode Decl::Kind to do that easily.
     assert(isa<TypeDecl>(D) && "Unsupported decl kind");
     break;
   }
 }

clang/test/CodeGenHLSL/cbuf.hlsl

+// RUN: %clang_dxc -Tlib_6_7 -fcgl  -Fo - %s | FileCheck %s
+
+// CHECK: @[[CB:.+]] = external addrspace(4) constant { float, i32, double }
+cbuffer A : register(b0, space1) {
+  float a;
+  double b;
+}
+
+// CHECK: @[[TB:.+]] = external addrspace(5) constant { float, i32, double }
+tbuffer A : register(t2, space1) {
+  float c;
+  double d;
+}
+
+float foo() {
+// CHECK: load float, ptr addrspacecast (ptr addrspace(4) @[[CB]] to ptr), align 4
+// CHECK: load double, ptr addrspacecast (ptr addrspace(4) getelementptr inbounds ({ float, i32, double }, ptr addrspace(4) @[[CB]], i32 0, i32 2) to ptr), align 8
+// CHECK: load float, ptr addrspacecast (ptr addrspace(5) @[[TB]] to ptr), align 4
+// CHECK: load double, ptr addrspacecast (ptr addrspace(5) getelementptr inbounds ({ float, i32, double }, ptr addrspace(5) @[[TB]], i32 0, i32 2) to ptr), align 8
+  return a + b + c*d;
+}

clang/test/CodeGenHLSL/nest_cbuf.hlsl

+// RUN: %clang_dxc -Tlib_6_7 -fcgl  -Fo - %s | FileCheck %s
+
+// CHECK: @[[CB:.+]] = external addrspace(4) constant { float, i32, double }
+cbuffer A {
+  float a;
+  double b;
+  // CHECK: @[[TB:.+]] = external addrspace(5) constant { float, i32, double }
+  tbuffer A : register(t2, space1) {
+    float c;
+    double d;
+  }
+}
+
+float foo() {
+// CHECK: load float, ptr addrspacecast (ptr addrspace(4) @[[CB]] to ptr), align 4
+// CHECK: load double, ptr addrspacecast (ptr addrspace(4) getelementptr inbounds ({ float, i32, double }, ptr addrspace(4) @[[CB]], i32 0, i32 2) to ptr), align 8
+// CHECK: load float, ptr addrspacecast (ptr addrspace(5) @[[TB]] to ptr), align 4
+// CHECK: load double, ptr addrspacecast (ptr addrspace(5) getelementptr inbounds ({ float, i32, double }, ptr addrspace(5) @[[TB]], i32 0, i32 2) to ptr), align 8
+  return a + b + c*d;
+}