AMDGPU: Emit runtime metadata as a note element in .note section

[yaxunl]

Summary

Currently runtime metadata is emitted as an ELF section with name .AMDGPU.runtime_metadata.

However there is a standard way to convey vendor specific information about how to run an ELF binary, which is called vendor-specific note element (http://www.netbsd.org/docs/kernel/elf-notes.html).

This patch lets AMDGPU backend emits runtime metadata as a note element in .note section.

[yaxunl]

Revised by Laurent's suggestions.

Fixed size of name and desc. They should not include padding 0's.

Extracted enums of note type as a separate header file so that it can be shared between backend and runtime.

[• tstellarAMD]

lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp

47–84 ↗

(On Diff #75200)

These functions belong in AMDGPUTargetStreamer classes.

lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.cpp

14 ↗

(On Diff #75200)

Are the changes in this file necessary?

[kzhuravl]

lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp

1130–1131 ↗

(On Diff #75200)

Once you move it to AMDGPUTargetStreamer classes as suggested by Tom, use AMDGPUTargetELFStreamer::NoteName instead of defining the name and size (see rL284760 for more info)

lib/Target/AMDGPU/AMDGPUNoteType.h

20–32 ↗

(On Diff #75200)

Note types are defined in AMDGPUTargetELFStreamer. Why redefine them?

lib/Target/AMDGPU/AMDGPURuntimeMetadata.h

41 ↗

(On Diff #75200)

I do not think we need to include this here.

52 ↗

(On Diff #75200)

This seems like a property of an elf, and not RuntimeMD. I would suggest making it static const char * in AMDGPUTargetELFStreamer (probably along with flags, etc.)

[yaxunl]

Revised by Tom and Konstantin's comments.

[yaxunl]

lib/Target/AMDGPU/AMDGPUNoteType.h

20–32 ↗

(On Diff #75200)

Moved to AMDGPUPTNote.h, which is shared between compiler and runtime.

The runtime needs to know the enum value for note type for runtime. To avoid maintaining two copies of enum definitions, I moved these enum definitions to the shared header file.

lib/Target/AMDGPU/AMDGPURuntimeMetadata.h

52 ↗

(On Diff #75200)

This needs to be defined in the shared header file between runtime and compiler.

As you said, it is not runtime metadata specific, so I created AMDPTNote.h and moved it there.

lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.cpp

14 ↗

(On Diff #75200)

We need to share the enums for PT_NOTE sections with runtime.

[kzhuravl]

The runtime needs to know the enum value for note type for runtime. To avoid maintaining two copies of enum definitions, I moved these enum definitions to the shared header file.

+ @tony-tye .

This is the right direction, but I think we should discuss this in more details. There are a few files that need to be shared between compiler and runtime (some elf stuff, amd_kernel_code_t, amd_queue_t? amd_signal_t?). Creating a file specifically for note stuff seems like an overkill to me IMHO.

I cleaned up files that are shared between compiler and runtime files a while ago, they are here:

• https://github.com/RadeonOpenCompute/ROCR-Runtime/blob/master/src/inc/amd_hsa_common.h
• https://github.com/RadeonOpenCompute/ROCR-Runtime/blob/master/src/inc/amd_hsa_elf.h
• https://github.com/RadeonOpenCompute/ROCR-Runtime/blob/master/src/inc/amd_hsa_kernel_code.h
• https://github.com/RadeonOpenCompute/ROCR-Runtime/blob/master/src/inc/amd_hsa_queue.h
• https://github.com/RadeonOpenCompute/ROCR-Runtime/blob/master/src/inc/amd_hsa_signal.h

They still need more clean up since we switched to code object v2+. But maybe we can take those as a base.

lib/Target/AMDGPU/AMDGPUPTNote.h

26 ↗

(On Diff #75581)

AMD\0

lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.cpp

111 ↗

(On Diff #75581)

.note -> PT_NOTE::NoteName, same in the below one.

496 ↗

(On Diff #75581)

I think ELF streaming belongs to AMDGPUTargetELFStreamer?

lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.h

46 ↗

(On Diff #75581)

Doxygenify?

[yaxunl]

This is the right direction, but I think we should discuss this in more details. There are a few files that need to be shared between compiler and runtime (some elf stuff, amd_kernel_code_t, amd_queue_t? amd_signal_t?). Creating a file specifically for note stuff seems like an overkill to me IMHO.

I cleaned up files that are shared between compiler and runtime files a while ago, they are here:

• https://github.com/RadeonOpenCompute/ROCR-Runtime/blob/master/src/inc/amd_hsa_common.h
• https://github.com/RadeonOpenCompute/ROCR-Runtime/blob/master/src/inc/amd_hsa_elf.h
• https://github.com/RadeonOpenCompute/ROCR-Runtime/blob/master/src/inc/amd_hsa_kernel_code.h
• https://github.com/RadeonOpenCompute/ROCR-Runtime/blob/master/src/inc/amd_hsa_queue.h
• https://github.com/RadeonOpenCompute/ROCR-Runtime/blob/master/src/inc/amd_hsa_signal.h

They still need more clean up since we switched to code object v2+. But maybe we can take those as a base.

I agree. Since it will take time and efforts to clean up and incorporate these header files, I would suggest to leave it to another patch.

lib/Target/AMDGPU/AMDGPUPTNote.h

26 ↗

(On Diff #75581)

This is unnecessary since the literal string "AMD" has an implied trailing 0 and sizeof(NoteName) will include that, i.e., sizeof(NoteName)==4.

lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.cpp

496 ↗

(On Diff #75581)

Normally I should define virtual functions for streaming the runtime metadata in textual and binary form separately in AMGGPUTargetASMStreamer and AMDGPUTargetELFStreamer. However we will replace this runtime metadata format with YAML based runtime metadata format soon, so I feel not worth the efforts to implement the textual format for the current runtime metadata format. After we move to YAML based runtime metadata format, I will come back to a better solution.

[yaxunl]

Revised by Konstantin's comments.

[yaxunl]

Ping.

[kzhuravl]

Minor formatting + question

lib/Target/AMDGPU/AMDGPUPTNote.h

29–38 ↗

(On Diff #76040)

This might have been formatted incorrectly? Seems like '};' at column 2, but 'enum NoteType' at column 0?

lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.cpp

496 ↗

(On Diff #75581)

Shouldn't we define pure virtual functions in base class, only implement them in AMDGPUTargetELFStreamer subclass, and possibly define same functions in AMGGPUTargetASMStreamer subclass but leave the body empty?

363 ↗

(On Diff #76040)

Formatting

[yaxunl]

lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.cpp

496 ↗

(On Diff #75581)

If we do this, there will be no runtime metadata in the llc output. Then how do we test this?

[kzhuravl]

lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.cpp

496 ↗

(On Diff #75581)

-filetype=obj will cause llc to produce elf, which will have runtime metadata. Since it is a temporary solution as you indicated before, I am fine with what is currently done. Please, also, run it by Matt or Tom.

[yaxunl]

lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.cpp

496 ↗

(On Diff #75581)

This requests rewriting the test in a new way. Currently there is not enough time to make this change. Can we leave this to future?

[yaxunl]

Fix format issue.

[kzhuravl]

LGTM

llvm/trunk/lib/Target/AMDGPU/AMDGPUAsmPrinter.h

public:
 
   bool isBlockOnlyReachableByFallthrough(
     const MachineBasicBlock *MBB) const override;
 
   bool PrintAsmOperand(const MachineInstr *MI, unsigned OpNo,
                        unsigned AsmVariant, const char *ExtraCode,
                        raw_ostream &O) override;
 
-  void emitStartOfRuntimeMetadata(const Module &M);
-
-  void emitRuntimeMetadata(const Function &F);
-
 protected:
   std::vector<std::string> DisasmLines, HexLines;
   size_t DisasmLineMaxLen;
 };
 
 } // End anonymous llvm
 
 #endif

llvm/trunk/lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp

 #include "llvm/IR/DiagnosticInfo.h"
 #include "llvm/MC/MCContext.h"
 #include "llvm/MC/MCSectionELF.h"
 #include "llvm/MC/MCStreamer.h"
 #include "llvm/Support/ELF.h"
 #include "llvm/Support/MathExtras.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Target/TargetLoweringObjectFile.h"
-#include "AMDGPURuntimeMetadata.h"
 
-using namespace ::AMDGPU;
 using namespace llvm;
 
 // TODO: This should get the default rounding mode from the kernel. We just set
 // the default here, but this could change if the OpenCL rounding mode pragmas
 // are used.
 //
 // The denormal mode here should match what is reported by the OpenCL runtime
 // for the CL_FP_DENORM bit from CL_DEVICE_{HALF|SINGLE|DOUBLE}_FP_CONFIG, but
void AMDGPUAsmPrinter::EmitStartOfAsmFile(Module &M) {
   AMDGPUTargetStreamer *TS =
       static_cast<AMDGPUTargetStreamer *>(OutStreamer->getTargetStreamer());
 
   TS->EmitDirectiveHSACodeObjectVersion(2, 1);
 
   AMDGPU::IsaVersion ISA = AMDGPU::getIsaVersion(STI->getFeatureBits());
   TS->EmitDirectiveHSACodeObjectISA(ISA.Major, ISA.Minor, ISA.Stepping,
                                     "AMD", "AMDGPU");
-  emitStartOfRuntimeMetadata(M);
+
+  // Emit runtime metadata.
+  TS->emitRuntimeMetadataAsNoteElement(M);
 }
 
 bool AMDGPUAsmPrinter::isBlockOnlyReachableByFallthrough(
   const MachineBasicBlock *MBB) const {
   if (!AsmPrinter::isBlockOnlyReachableByFallthrough(MBB))
     return false;
 
   if (MBB->empty())
for (size_t i = 0; i < DisasmLines.size(); ++i) {
       std::string Comment(DisasmLineMaxLen - DisasmLines[i].size(), ' ');
       Comment += " ; " + HexLines[i] + "\n";
 
       OutStreamer->EmitBytes(StringRef(DisasmLines[i]));
       OutStreamer->EmitBytes(StringRef(Comment));
     }
   }
 
-  emitRuntimeMetadata(*MF.getFunction());
-
   return false;
 }
 
 void AMDGPUAsmPrinter::EmitProgramInfoR600(const MachineFunction &MF) {
   unsigned MaxGPR = 0;
   bool killPixel = false;
   const R600Subtarget &STM = MF.getSubtarget<R600Subtarget>();
   const R600RegisterInfo *RI = STM.getRegisterInfo();
if (ExtraCode && ExtraCode[0]) {
     }
   }
 
   AMDGPUInstPrinter::printRegOperand(MI->getOperand(OpNo).getReg(), O,
                    *TM.getSubtargetImpl(*MF->getFunction())->getRegisterInfo());
   return false;
 }
 
-// Emit a key and an integer value for runtime metadata.
-static void emitRuntimeMDIntValue(MCStreamer &Streamer,
-                                  RuntimeMD::Key K, uint64_t V,
-                                  unsigned Size) {
-  Streamer.EmitIntValue(K, 1);
-  Streamer.EmitIntValue(V, Size);
-}
-
-// Emit a key and a string value for runtime metadata.
-static void emitRuntimeMDStringValue(MCStreamer &Streamer,
-                                     RuntimeMD::Key K, StringRef S) {
-  Streamer.EmitIntValue(K, 1);
-  Streamer.EmitIntValue(S.size(), 4);
-  Streamer.EmitBytes(S);
-}
-
-// Emit a key and three integer values for runtime metadata.
-// The three integer values are obtained from MDNode \p Node;
-static void emitRuntimeMDThreeIntValues(MCStreamer &Streamer,
-                                        RuntimeMD::Key K, MDNode *Node,
-                                        unsigned Size) {
-  assert(Node->getNumOperands() == 3);
-
-  Streamer.EmitIntValue(K, 1);
-  for (const MDOperand &Op : Node->operands()) {
-    const ConstantInt *CI = mdconst::extract<ConstantInt>(Op);
-    Streamer.EmitIntValue(CI->getZExtValue(), Size);
-  }
-}
-
-void AMDGPUAsmPrinter::emitStartOfRuntimeMetadata(const Module &M) {
-  OutStreamer->SwitchSection(getObjFileLowering().getContext()
-    .getELFSection(RuntimeMD::SectionName, ELF::SHT_PROGBITS, 0));
-
-  emitRuntimeMDIntValue(*OutStreamer, RuntimeMD::KeyMDVersion,
-                        RuntimeMD::MDVersion << 8 | RuntimeMD::MDRevision, 2);
-  if (auto MD = M.getNamedMetadata("opencl.ocl.version")) {
-    if (MD->getNumOperands() != 0) {
-      auto Node = MD->getOperand(0);
-      if (Node->getNumOperands() > 1) {
-        emitRuntimeMDIntValue(*OutStreamer, RuntimeMD::KeyLanguage,
-                              RuntimeMD::OpenCL_C, 1);
-        uint16_t Major = mdconst::extract<ConstantInt>(Node->getOperand(0))
-                         ->getZExtValue();
-        uint16_t Minor = mdconst::extract<ConstantInt>(Node->getOperand(1))
-                         ->getZExtValue();
-        emitRuntimeMDIntValue(*OutStreamer, RuntimeMD::KeyLanguageVersion,
-                              Major * 100 + Minor * 10, 2);
-      }
-    }
-  }
-
-  if (auto MD = M.getNamedMetadata("llvm.printf.fmts")) {
-    for (unsigned I = 0; I < MD->getNumOperands(); ++I) {
-      auto Node = MD->getOperand(I);
-      if (Node->getNumOperands() > 0)
-        emitRuntimeMDStringValue(*OutStreamer, RuntimeMD::KeyPrintfInfo,
-            cast<MDString>(Node->getOperand(0))->getString());
-    }
-  }
-}
-
-static std::string getOCLTypeName(Type *Ty, bool Signed) {
-  switch (Ty->getTypeID()) {
-  case Type::HalfTyID:
-    return "half";
-  case Type::FloatTyID:
-    return "float";
-  case Type::DoubleTyID:
-    return "double";
-  case Type::IntegerTyID: {
-    if (!Signed)
-      return (Twine('u') + getOCLTypeName(Ty, true)).str();
-    unsigned BW = Ty->getIntegerBitWidth();
-    switch (BW) {
-    case 8:
-      return "char";
-    case 16:
-      return "short";
-    case 32:
-      return "int";
-    case 64:
-      return "long";
-    default:
-      return (Twine('i') + Twine(BW)).str();
-    }
-  }
-  case Type::VectorTyID: {
-    VectorType *VecTy = cast<VectorType>(Ty);
-    Type *EleTy = VecTy->getElementType();
-    unsigned Size = VecTy->getVectorNumElements();
-    return (Twine(getOCLTypeName(EleTy, Signed)) + Twine(Size)).str();
-  }
-  default:
-    return "unknown";
-  }
-}
-
-static RuntimeMD::KernelArg::ValueType getRuntimeMDValueType(
-  Type *Ty, StringRef TypeName) {
-  switch (Ty->getTypeID()) {
-  case Type::HalfTyID:
-    return RuntimeMD::KernelArg::F16;
-  case Type::FloatTyID:
-    return RuntimeMD::KernelArg::F32;
-  case Type::DoubleTyID:
-    return RuntimeMD::KernelArg::F64;
-  case Type::IntegerTyID: {
-    bool Signed = !TypeName.startswith("u");
-    switch (Ty->getIntegerBitWidth()) {
-    case 8:
-      return Signed ? RuntimeMD::KernelArg::I8 : RuntimeMD::KernelArg::U8;
-    case 16:
-      return Signed ? RuntimeMD::KernelArg::I16 : RuntimeMD::KernelArg::U16;
-    case 32:
-      return Signed ? RuntimeMD::KernelArg::I32 : RuntimeMD::KernelArg::U32;
-    case 64:
-      return Signed ? RuntimeMD::KernelArg::I64 : RuntimeMD::KernelArg::U64;
-    default:
-      // Runtime does not recognize other integer types. Report as struct type.
-      return RuntimeMD::KernelArg::Struct;
-    }
-  }
-  case Type::VectorTyID:
-    return getRuntimeMDValueType(Ty->getVectorElementType(), TypeName);
-  case Type::PointerTyID:
-    return getRuntimeMDValueType(Ty->getPointerElementType(), TypeName);
-  default:
-    return RuntimeMD::KernelArg::Struct;
-  }
-}
-
-static RuntimeMD::KernelArg::AddressSpaceQualifer getRuntimeAddrSpace(
-    AMDGPUAS::AddressSpaces A) {
-  switch (A) {
-  case AMDGPUAS::GLOBAL_ADDRESS:
-    return RuntimeMD::KernelArg::Global;
-  case AMDGPUAS::CONSTANT_ADDRESS:
-    return RuntimeMD::KernelArg::Constant;
-  case AMDGPUAS::LOCAL_ADDRESS:
-    return RuntimeMD::KernelArg::Local;
-  case AMDGPUAS::FLAT_ADDRESS:
-    return RuntimeMD::KernelArg::Generic;
-  case AMDGPUAS::REGION_ADDRESS:
-    return RuntimeMD::KernelArg::Region;
-  default:
-    return RuntimeMD::KernelArg::Private;
-  }
-}
-
-static void emitRuntimeMetadataForKernelArg(const DataLayout &DL,
-    MCStreamer &OutStreamer, Type *T,
-    RuntimeMD::KernelArg::Kind Kind,
-    StringRef BaseTypeName = "", StringRef TypeName = "",
-    StringRef ArgName = "", StringRef TypeQual = "", StringRef AccQual = "") {
-  // Emit KeyArgBegin.
-  OutStreamer.EmitIntValue(RuntimeMD::KeyArgBegin, 1);
-
-  // Emit KeyArgSize and KeyArgAlign.
-  emitRuntimeMDIntValue(OutStreamer, RuntimeMD::KeyArgSize,
-                        DL.getTypeAllocSize(T), 4);
-  emitRuntimeMDIntValue(OutStreamer, RuntimeMD::KeyArgAlign,
-                        DL.getABITypeAlignment(T), 4);
-  if (auto PT = dyn_cast<PointerType>(T)) {
-    auto ET = PT->getElementType();
-    if (PT->getAddressSpace() == AMDGPUAS::LOCAL_ADDRESS && ET->isSized())
-      emitRuntimeMDIntValue(OutStreamer, RuntimeMD::KeyArgPointeeAlign,
-                        DL.getABITypeAlignment(ET), 4);
-  }
-
-  // Emit KeyArgTypeName.
-  if (!TypeName.empty())
-    emitRuntimeMDStringValue(OutStreamer, RuntimeMD::KeyArgTypeName, TypeName);
-
-  // Emit KeyArgName.
-  if (!ArgName.empty())
-    emitRuntimeMDStringValue(OutStreamer, RuntimeMD::KeyArgName, ArgName);
-
-  // Emit KeyArgIsVolatile, KeyArgIsRestrict, KeyArgIsConst and KeyArgIsPipe.
-  SmallVector<StringRef, 1> SplitQ;
-  TypeQual.split(SplitQ, " ", -1, false /* Drop empty entry */);
-
-  for (StringRef KeyName : SplitQ) {
-    auto Key = StringSwitch<RuntimeMD::Key>(KeyName)
-      .Case("volatile", RuntimeMD::KeyArgIsVolatile)
-      .Case("restrict", RuntimeMD::KeyArgIsRestrict)
-      .Case("const",    RuntimeMD::KeyArgIsConst)
-      .Case("pipe",     RuntimeMD::KeyArgIsPipe)
-      .Default(RuntimeMD::KeyNull);
-    OutStreamer.EmitIntValue(Key, 1);
-  }
-
-  // Emit KeyArgKind.
-  emitRuntimeMDIntValue(OutStreamer, RuntimeMD::KeyArgKind, Kind, 1);
-
-  // Emit KeyArgValueType.
-  emitRuntimeMDIntValue(OutStreamer, RuntimeMD::KeyArgValueType,
-                        getRuntimeMDValueType(T, BaseTypeName), 2);
-
-  // Emit KeyArgAccQual.
-  if (!AccQual.empty()) {
-    auto AQ = StringSwitch<RuntimeMD::KernelArg::AccessQualifer>(AccQual)
-      .Case("read_only",  RuntimeMD::KernelArg::ReadOnly)
-      .Case("write_only", RuntimeMD::KernelArg::WriteOnly)
-      .Case("read_write", RuntimeMD::KernelArg::ReadWrite)
-      .Default(RuntimeMD::KernelArg::None);
-    emitRuntimeMDIntValue(OutStreamer, RuntimeMD::KeyArgAccQual, AQ, 1);
-  }
-
-  // Emit KeyArgAddrQual.
-  if (auto *PT = dyn_cast<PointerType>(T))
-    emitRuntimeMDIntValue(OutStreamer, RuntimeMD::KeyArgAddrQual,
-        getRuntimeAddrSpace(static_cast<AMDGPUAS::AddressSpaces>(
-            PT->getAddressSpace())), 1);
-
-  // Emit KeyArgEnd
-  OutStreamer.EmitIntValue(RuntimeMD::KeyArgEnd, 1);
-}
-
-void AMDGPUAsmPrinter::emitRuntimeMetadata(const Function &F) {
-  if (!F.getMetadata("kernel_arg_type"))
-    return;
-
-  MCContext &Context = getObjFileLowering().getContext();
-  OutStreamer->SwitchSection(
-      Context.getELFSection(RuntimeMD::SectionName, ELF::SHT_PROGBITS, 0));
-  OutStreamer->EmitIntValue(RuntimeMD::KeyKernelBegin, 1);
-  emitRuntimeMDStringValue(*OutStreamer, RuntimeMD::KeyKernelName, F.getName());
-
-  const DataLayout &DL = F.getParent()->getDataLayout();
-  for (auto &Arg : F.args()) {
-    unsigned I = Arg.getArgNo();
-    Type *T = Arg.getType();
-    auto TypeName = dyn_cast<MDString>(F.getMetadata(
-        "kernel_arg_type")->getOperand(I))->getString();
-    auto BaseTypeName = cast<MDString>(F.getMetadata(
-        "kernel_arg_base_type")->getOperand(I))->getString();
-    StringRef ArgName;
-    if (auto ArgNameMD = F.getMetadata("kernel_arg_name"))
-      ArgName = cast<MDString>(ArgNameMD->getOperand(I))->getString();
-    auto TypeQual = cast<MDString>(F.getMetadata(
-        "kernel_arg_type_qual")->getOperand(I))->getString();
-    auto AccQual = cast<MDString>(F.getMetadata(
-        "kernel_arg_access_qual")->getOperand(I))->getString();
-    RuntimeMD::KernelArg::Kind Kind;
-    if (TypeQual.find("pipe") != StringRef::npos)
-      Kind = RuntimeMD::KernelArg::Pipe;
-    else Kind = StringSwitch<RuntimeMD::KernelArg::Kind>(BaseTypeName)
-      .Case("sampler_t", RuntimeMD::KernelArg::Sampler)
-      .Case("queue_t",   RuntimeMD::KernelArg::Queue)
-      .Cases("image1d_t", "image1d_array_t", "image1d_buffer_t",
-             "image2d_t" , "image2d_array_t",  RuntimeMD::KernelArg::Image)
-      .Cases("image2d_depth_t", "image2d_array_depth_t",
-             "image2d_msaa_t", "image2d_array_msaa_t",
-             "image2d_msaa_depth_t",  RuntimeMD::KernelArg::Image)
-      .Cases("image2d_array_msaa_depth_t", "image3d_t",
-             RuntimeMD::KernelArg::Image)
-      .Default(isa<PointerType>(T) ?
-                   (T->getPointerAddressSpace() == AMDGPUAS::LOCAL_ADDRESS ?
-                   RuntimeMD::KernelArg::DynamicSharedPointer :
-                   RuntimeMD::KernelArg::GlobalBuffer) :
-                   RuntimeMD::KernelArg::ByValue);
-    emitRuntimeMetadataForKernelArg(DL, *OutStreamer, T,
-        Kind, BaseTypeName, TypeName, ArgName, TypeQual, AccQual);
-  }
-
-  // Emit hidden kernel arguments for OpenCL kernels.
-  if (F.getParent()->getNamedMetadata("opencl.ocl.version")) {
-    auto Int64T = Type::getInt64Ty(F.getContext());
-    emitRuntimeMetadataForKernelArg(DL, *OutStreamer, Int64T,
-                                    RuntimeMD::KernelArg::HiddenGlobalOffsetX);
-    emitRuntimeMetadataForKernelArg(DL, *OutStreamer, Int64T,
-                                    RuntimeMD::KernelArg::HiddenGlobalOffsetY);
-    emitRuntimeMetadataForKernelArg(DL, *OutStreamer, Int64T,
-                                    RuntimeMD::KernelArg::HiddenGlobalOffsetZ);
-    if (F.getParent()->getNamedMetadata("llvm.printf.fmts")) {
-      auto Int8PtrT = Type::getInt8PtrTy(F.getContext(),
-          RuntimeMD::KernelArg::Global);
-      emitRuntimeMetadataForKernelArg(DL, *OutStreamer, Int8PtrT,
-                                      RuntimeMD::KernelArg::HiddenPrintfBuffer);
-    }
-  }
-
-  // Emit KeyReqdWorkGroupSize, KeyWorkGroupSizeHint, and KeyVecTypeHint.
-  if (auto RWGS = F.getMetadata("reqd_work_group_size")) {
-    emitRuntimeMDThreeIntValues(*OutStreamer, RuntimeMD::KeyReqdWorkGroupSize,
-                                RWGS, 4);
-  }
-
-  if (auto WGSH = F.getMetadata("work_group_size_hint")) {
-    emitRuntimeMDThreeIntValues(*OutStreamer, RuntimeMD::KeyWorkGroupSizeHint,
-                                WGSH, 4);
-  }
-
-  if (auto VTH = F.getMetadata("vec_type_hint")) {
-    auto TypeName = getOCLTypeName(cast<ValueAsMetadata>(
-      VTH->getOperand(0))->getType(), mdconst::extract<ConstantInt>(
-      VTH->getOperand(1))->getZExtValue());
-    emitRuntimeMDStringValue(*OutStreamer, RuntimeMD::KeyVecTypeHint, TypeName);
-  }
-
-  // Emit KeyKernelEnd
-  OutStreamer->EmitIntValue(RuntimeMD::KeyKernelEnd, 1);
-}

llvm/trunk/lib/Target/AMDGPU/AMDGPUPTNote.h

+//===-- AMDGPUNoteType.h - AMDGPU ELF PT_NOTE section info-------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+/// \file
+///
+/// Enums and constants for AMDGPU PT_NOTE sections.
+///
+//
+//===----------------------------------------------------------------------===//
+//
+#ifndef LLVM_LIB_TARGET_AMDGPU_AMDGPUPTNOTE_H
+#define LLVM_LIB_TARGET_AMDGPU_AMDGPUPTNOTE_H
+
+namespace AMDGPU {
+
+namespace PT_NOTE {
+
+const char SectionName[] = ".note";
+
+const char NoteName[] = "AMD";
+
+enum NoteType{
+    NT_AMDGPU_HSA_CODE_OBJECT_VERSION = 1,
+    NT_AMDGPU_HSA_HSAIL = 2,
+    NT_AMDGPU_HSA_ISA = 3,
+    NT_AMDGPU_HSA_PRODUCER = 4,
+    NT_AMDGPU_HSA_PRODUCER_OPTIONS = 5,
+    NT_AMDGPU_HSA_EXTENSION = 6,
+    NT_AMDGPU_HSA_RUNTIME_METADATA = 7,
+    NT_AMDGPU_HSA_HLDEBUG_DEBUG = 101,
+    NT_AMDGPU_HSA_HLDEBUG_TARGET = 102
+};
+}
+}
+
+#endif // LLVM_LIB_TARGET_AMDGPU_AMDGPUNOTETYPE_H

llvm/trunk/lib/Target/AMDGPU/AMDGPURuntimeMetadata.h

 //===-- AMDGPURuntimeMetadata.h - AMDGPU Runtime Metadata -------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 /// \file
 ///
 /// Enums and structure types used by runtime metadata.
 ///
 /// Runtime requests certain information (metadata) about kernels to be able
 /// to execute the kernels and answer the queries about the kernels.
-/// The metadata is represented as a byte stream in an ELF section of a
-/// binary (code object). The byte stream consists of key-value pairs.
-/// Each key is an 8 bit unsigned integer. Each value can be an integer,
-/// a string, or a stream of key-value pairs. There are 3 levels of key-value
-/// pair streams. At the beginning of the ELF section is the top level
+/// The metadata is represented as a note element in the .note ELF section of a
+/// binary (code object). The desc field of the note element consists of
+/// key-value pairs. Each key is an 8 bit unsigned integer. Each value can be
+/// an integer, a string, or a stream of key-value pairs. There are 3 levels of
+/// key-value pair streams. At the beginning of the ELF section is the top level
 /// key-value pair stream. A kernel-level key-value pair stream starts after
 /// encountering KeyKernelBegin and ends immediately before encountering
 /// KeyKernelEnd. A kernel-argument-level key-value pair stream starts
 /// after encountering KeyArgBegin and ends immediately before encountering
 /// KeyArgEnd. A kernel-level key-value pair stream can only appear in a top
 /// level key-value pair stream. A kernel-argument-level key-value pair stream
 /// can only appear in a kernel-level key-value pair stream.
 ///

 namespace AMDGPU {
 
 namespace RuntimeMD {
 
   // Version and revision of runtime metadata
   const unsigned char MDVersion   = 1;
   const unsigned char MDRevision  = 0;
 
-  // ELF section name containing runtime metadata
-  const char SectionName[] = ".AMDGPU.runtime_metadata";
-
   // Enumeration values of keys in runtime metadata.
   enum Key {
     KeyNull                     = 0, // Place holder. Ignored when encountered
     KeyMDVersion                = 1, // Runtime metadata version
     KeyLanguage                 = 2, // Language
     KeyLanguageVersion          = 3, // Language version
     KeyKernelBegin              = 4, // Beginning of kernel-level stream
     KeyKernelEnd                = 5, // End of kernel-level stream

llvm/trunk/lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.h

 //===-- AMDGPUTargetStreamer.h - AMDGPU Target Streamer --------*- C++ -*--===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_LIB_TARGET_AMDGPU_MCTARGETDESC_AMDGPUTARGETSTREAMER_H
 #define LLVM_LIB_TARGET_AMDGPU_MCTARGETDESC_AMDGPUTARGETSTREAMER_H
 
 #include "AMDKernelCodeT.h"
 #include "llvm/MC/MCStreamer.h"
 
 namespace llvm {
+#include "AMDGPURuntimeMetadata.h"
 
+class DataLayout;
+class Function;
 class MCELFStreamer;
 class MCSymbol;
+class MDNode;
+class Module;
+class Type;
 
 class AMDGPUTargetStreamer : public MCTargetStreamer {
 public:
   AMDGPUTargetStreamer(MCStreamer &S);
   virtual void EmitDirectiveHSACodeObjectVersion(uint32_t Major,
                                                  uint32_t Minor) = 0;
 
   virtual void EmitDirectiveHSACodeObjectISA(uint32_t Major, uint32_t Minor,
                                              uint32_t Stepping,
                                              StringRef VendorName,
                                              StringRef ArchName) = 0;
 
   virtual void EmitAMDKernelCodeT(const amd_kernel_code_t &Header) = 0;
 
   virtual void EmitAMDGPUSymbolType(StringRef SymbolName, unsigned Type) = 0;
 
   virtual void EmitAMDGPUHsaModuleScopeGlobal(StringRef GlobalName) = 0;
 
   virtual void EmitAMDGPUHsaProgramScopeGlobal(StringRef GlobalName) = 0;
+
+  /// Emit runtime metadata as a note element.
+  void emitRuntimeMetadataAsNoteElement(Module &M);
+
+private:
+  void emitRuntimeMetadata(Module &M);
+  void emitStartOfRuntimeMetadata(const Module &M);
+
+  /// Emit runtime metadata for a kernel function.
+  void emitRuntimeMetadata(const Function &F);
+
+  // Emit runtime metadata for a kernel argument.
+  void emitRuntimeMetadataForKernelArg(const DataLayout &DL,
+      Type *T, AMDGPU::RuntimeMD::KernelArg::Kind Kind,
+      StringRef BaseTypeName = "", StringRef TypeName = "",
+      StringRef ArgName = "", StringRef TypeQual = "",
+      StringRef AccQual = "");
+
+  /// Emit a key and an integer value for runtime metadata.
+  void emitRuntimeMDIntValue(AMDGPU::RuntimeMD::Key K,
+      uint64_t V, unsigned Size);
+
+  /// Emit a key and a string value for runtime metadata.
+  void emitRuntimeMDStringValue(AMDGPU::RuntimeMD::Key K,
+      StringRef S);
+
+  /// Emit a key and three integer values for runtime metadata.
+  /// The three integer values are obtained from MDNode \p Node;
+  void emitRuntimeMDThreeIntValues(AMDGPU::RuntimeMD::Key K, MDNode *Node,
+                                   unsigned Size);
 };
 
 class AMDGPUTargetAsmStreamer : public AMDGPUTargetStreamer {
   formatted_raw_ostream &OS;
 public:
   AMDGPUTargetAsmStreamer(MCStreamer &S, formatted_raw_ostream &OS);
   void EmitDirectiveHSACodeObjectVersion(uint32_t Major,
                                          uint32_t Minor) override;
 
   void EmitDirectiveHSACodeObjectISA(uint32_t Major, uint32_t Minor,
                                      uint32_t Stepping, StringRef VendorName,
                                      StringRef ArchName) override;
 
   void EmitAMDKernelCodeT(const amd_kernel_code_t &Header) override;
 
   void EmitAMDGPUSymbolType(StringRef SymbolName, unsigned Type) override;
 
   void EmitAMDGPUHsaModuleScopeGlobal(StringRef GlobalName) override;
 
   void EmitAMDGPUHsaProgramScopeGlobal(StringRef GlobalName) override;
 };
 
 class AMDGPUTargetELFStreamer : public AMDGPUTargetStreamer {
-
-  enum NoteType {
-    NT_AMDGPU_HSA_CODE_OBJECT_VERSION = 1,
-    NT_AMDGPU_HSA_HSAIL = 2,
-    NT_AMDGPU_HSA_ISA = 3,
-    NT_AMDGPU_HSA_PRODUCER = 4,
-    NT_AMDGPU_HSA_PRODUCER_OPTIONS = 5,
-    NT_AMDGPU_HSA_EXTENSION = 6,
-    NT_AMDGPU_HSA_HLDEBUG_DEBUG = 101,
-    NT_AMDGPU_HSA_HLDEBUG_TARGET = 102
-  };
-
-  static const char *NoteName;
   MCStreamer &Streamer;
 
 public:
   AMDGPUTargetELFStreamer(MCStreamer &S);
 
   MCELFStreamer &getStreamer();
 
   void EmitDirectiveHSACodeObjectVersion(uint32_t Major,

llvm/trunk/lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.cpp

 //===-- AMDGPUTargetStreamer.cpp - Mips Target Streamer Methods -----------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This file provides AMDGPU specific target streamer methods.
 //
 //===----------------------------------------------------------------------===//
 
+#include "AMDGPU.h"
 #include "AMDGPUTargetStreamer.h"
 #include "SIDefines.h"
 #include "Utils/AMDGPUBaseInfo.h"
 #include "Utils/AMDKernelCodeTUtils.h"
 #include "llvm/ADT/Twine.h"
+#include "llvm/IR/Constants.h"
+#include "llvm/IR/Function.h"
+#include "llvm/IR/Metadata.h"
+#include "llvm/IR/Module.h"
 #include "llvm/MC/MCContext.h"
 #include "llvm/MC/MCELFStreamer.h"
 #include "llvm/MC/MCObjectFileInfo.h"
 #include "llvm/MC/MCSectionELF.h"
 #include "llvm/Support/ELF.h"
 #include "llvm/Support/FormattedStream.h"
 
+namespace llvm {
+#include "AMDGPUPTNote.h"
+}
+
 using namespace llvm;
+using namespace llvm::AMDGPU;
 
 AMDGPUTargetStreamer::AMDGPUTargetStreamer(MCStreamer &S)
-    : MCTargetStreamer(S) { }
+    : MCTargetStreamer(S) {}
 
 //===----------------------------------------------------------------------===//
 // AMDGPUTargetAsmStreamer
 //===----------------------------------------------------------------------===//
 
 AMDGPUTargetAsmStreamer::AMDGPUTargetAsmStreamer(MCStreamer &S,
                                                  formatted_raw_ostream &OS)
     : AMDGPUTargetStreamer(S), OS(OS) { }
void AMDGPUTargetAsmStreamer::EmitAMDGPUHsaProgramScopeGlobal(
     StringRef GlobalName) {
   OS << "\t.amdgpu_hsa_program_global " << GlobalName << '\n';
 }
 
 //===----------------------------------------------------------------------===//
 // AMDGPUTargetELFStreamer
 //===----------------------------------------------------------------------===//
 
-const char *AMDGPUTargetELFStreamer::NoteName = "AMD";
-
 AMDGPUTargetELFStreamer::AMDGPUTargetELFStreamer(MCStreamer &S)
-    : AMDGPUTargetStreamer(S), Streamer(S) { }
+    : AMDGPUTargetStreamer(S), Streamer(S) {}
 
 MCELFStreamer &AMDGPUTargetELFStreamer::getStreamer() {
   return static_cast<MCELFStreamer &>(Streamer);
 }
 
 void
 AMDGPUTargetELFStreamer::EmitDirectiveHSACodeObjectVersion(uint32_t Major,
                                                            uint32_t Minor) {
   MCStreamer &OS = getStreamer();
   MCSectionELF *Note =
-      OS.getContext().getELFSection(".note", ELF::SHT_NOTE, ELF::SHF_ALLOC);
+      OS.getContext().getELFSection(PT_NOTE::NoteName, ELF::SHT_NOTE,
+                                    ELF::SHF_ALLOC);
 
+  auto NameSZ = sizeof(PT_NOTE::NoteName);
   OS.PushSection();
   OS.SwitchSection(Note);
-  OS.EmitIntValue(strlen(NoteName) + 1, 4);                // namesz
-  OS.EmitIntValue(8, 4);                                   // descz
-  OS.EmitIntValue(NT_AMDGPU_HSA_CODE_OBJECT_VERSION, 4);   // type
-  OS.EmitBytes(StringRef(NoteName, strlen(NoteName) + 1)); // name
+  OS.EmitIntValue(NameSZ, 4);                                     // namesz
+  OS.EmitIntValue(8, 4);                                          // descz
+  OS.EmitIntValue(PT_NOTE::NT_AMDGPU_HSA_CODE_OBJECT_VERSION, 4); // type
+  OS.EmitBytes(StringRef(PT_NOTE::NoteName, NameSZ));             // name
   OS.EmitValueToAlignment(4);
-  OS.EmitIntValue(Major, 4);                               // desc
+  OS.EmitIntValue(Major, 4);                                      // desc
   OS.EmitIntValue(Minor, 4);
   OS.EmitValueToAlignment(4);
   OS.PopSection();
 }
 
 void
 AMDGPUTargetELFStreamer::EmitDirectiveHSACodeObjectISA(uint32_t Major,
                                                        uint32_t Minor,
                                                        uint32_t Stepping,
                                                        StringRef VendorName,
                                                        StringRef ArchName) {
   MCStreamer &OS = getStreamer();
   MCSectionELF *Note =
-      OS.getContext().getELFSection(".note", ELF::SHT_NOTE, ELF::SHF_ALLOC);
+      OS.getContext().getELFSection(PT_NOTE::NoteName, ELF::SHT_NOTE,
+                                    ELF::SHF_ALLOC);
 
   uint16_t VendorNameSize = VendorName.size() + 1;
   uint16_t ArchNameSize = ArchName.size() + 1;
   unsigned DescSZ = sizeof(VendorNameSize) + sizeof(ArchNameSize) +
                     sizeof(Major) + sizeof(Minor) + sizeof(Stepping) +
                     VendorNameSize + ArchNameSize;
 
   OS.PushSection();
   OS.SwitchSection(Note);
-  OS.EmitIntValue(strlen(NoteName) + 1, 4);                // namesz
+  auto NameSZ = sizeof(PT_NOTE::NoteName);
+  OS.EmitIntValue(NameSZ, 4);                              // namesz
   OS.EmitIntValue(DescSZ, 4);                              // descsz
-  OS.EmitIntValue(NT_AMDGPU_HSA_ISA, 4);                   // type
-  OS.EmitBytes(StringRef(NoteName, strlen(NoteName) + 1)); // name
+  OS.EmitIntValue(PT_NOTE::NT_AMDGPU_HSA_ISA, 4);          // type
+  OS.EmitBytes(StringRef(PT_NOTE::NoteName, NameSZ));      // name
   OS.EmitValueToAlignment(4);
   OS.EmitIntValue(VendorNameSize, 2);                      // desc
   OS.EmitIntValue(ArchNameSize, 2);
   OS.EmitIntValue(Major, 4);
   OS.EmitIntValue(Minor, 4);
   OS.EmitIntValue(Stepping, 4);
   OS.EmitBytes(VendorName);
   OS.EmitIntValue(0, 1); // NULL terminate VendorName

 void AMDGPUTargetELFStreamer::EmitAMDGPUHsaProgramScopeGlobal(
     StringRef GlobalName) {
 
   MCSymbolELF *Symbol = cast<MCSymbolELF>(
       getStreamer().getContext().getOrCreateSymbol(GlobalName));
   Symbol->setType(ELF::STT_OBJECT);
   Symbol->setBinding(ELF::STB_GLOBAL);
 }
+
+void AMDGPUTargetStreamer::emitRuntimeMDIntValue(RuntimeMD::Key K, uint64_t V,
+                                                 unsigned Size) {
+  auto &S = getStreamer();
+  S.EmitIntValue(K, 1);
+  S.EmitIntValue(V, Size);
+}
+
+void AMDGPUTargetStreamer::emitRuntimeMDStringValue(RuntimeMD::Key K,
+                                                    StringRef R) {
+  auto &S = getStreamer();
+  S.EmitIntValue(K, 1);
+  S.EmitIntValue(R.size(), 4);
+  S.EmitBytes(R);
+}
+
+void AMDGPUTargetStreamer::emitRuntimeMDThreeIntValues(RuntimeMD::Key K,
+                                                       MDNode *Node,
+                                                       unsigned Size) {
+  assert(Node->getNumOperands() == 3);
+
+  auto &S = getStreamer();
+  S.EmitIntValue(K, 1);
+  for (const MDOperand &Op : Node->operands()) {
+    const ConstantInt *CI = mdconst::extract<ConstantInt>(Op);
+    S.EmitIntValue(CI->getZExtValue(), Size);
+  }
+}
+
+void AMDGPUTargetStreamer::emitStartOfRuntimeMetadata(const Module &M) {
+  emitRuntimeMDIntValue(RuntimeMD::KeyMDVersion,
+                        RuntimeMD::MDVersion << 8 | RuntimeMD::MDRevision, 2);
+  if (auto MD = M.getNamedMetadata("opencl.ocl.version")) {
+    if (MD->getNumOperands() != 0) {
+      auto Node = MD->getOperand(0);
+      if (Node->getNumOperands() > 1) {
+        emitRuntimeMDIntValue(RuntimeMD::KeyLanguage,
+                              RuntimeMD::OpenCL_C, 1);
+        uint16_t Major = mdconst::extract<ConstantInt>(Node->getOperand(0))
+                         ->getZExtValue();
+        uint16_t Minor = mdconst::extract<ConstantInt>(Node->getOperand(1))
+                         ->getZExtValue();
+        emitRuntimeMDIntValue(RuntimeMD::KeyLanguageVersion,
+                              Major * 100 + Minor * 10, 2);
+      }
+    }
+  }
+
+  if (auto MD = M.getNamedMetadata("llvm.printf.fmts")) {
+    for (unsigned I = 0; I < MD->getNumOperands(); ++I) {
+      auto Node = MD->getOperand(I);
+      if (Node->getNumOperands() > 0)
+        emitRuntimeMDStringValue(RuntimeMD::KeyPrintfInfo,
+            cast<MDString>(Node->getOperand(0))->getString());
+    }
+  }
+}
+
+static std::string getOCLTypeName(Type *Ty, bool Signed) {
+  switch (Ty->getTypeID()) {
+  case Type::HalfTyID:
+    return "half";
+  case Type::FloatTyID:
+    return "float";
+  case Type::DoubleTyID:
+    return "double";
+  case Type::IntegerTyID: {
+    if (!Signed)
+      return (Twine('u') + getOCLTypeName(Ty, true)).str();
+    unsigned BW = Ty->getIntegerBitWidth();
+    switch (BW) {
+    case 8:
+      return "char";
+    case 16:
+      return "short";
+    case 32:
+      return "int";
+    case 64:
+      return "long";
+    default:
+      return (Twine('i') + Twine(BW)).str();
+    }
+  }
+  case Type::VectorTyID: {
+    VectorType *VecTy = cast<VectorType>(Ty);
+    Type *EleTy = VecTy->getElementType();
+    unsigned Size = VecTy->getVectorNumElements();
+    return (Twine(getOCLTypeName(EleTy, Signed)) + Twine(Size)).str();
+  }
+  default:
+    return "unknown";
+  }
+}
+
+static RuntimeMD::KernelArg::ValueType getRuntimeMDValueType(
+  Type *Ty, StringRef TypeName) {
+  switch (Ty->getTypeID()) {
+  case Type::HalfTyID:
+    return RuntimeMD::KernelArg::F16;
+  case Type::FloatTyID:
+    return RuntimeMD::KernelArg::F32;
+  case Type::DoubleTyID:
+    return RuntimeMD::KernelArg::F64;
+  case Type::IntegerTyID: {
+    bool Signed = !TypeName.startswith("u");
+    switch (Ty->getIntegerBitWidth()) {
+    case 8:
+      return Signed ? RuntimeMD::KernelArg::I8 : RuntimeMD::KernelArg::U8;
+    case 16:
+      return Signed ? RuntimeMD::KernelArg::I16 : RuntimeMD::KernelArg::U16;
+    case 32:
+      return Signed ? RuntimeMD::KernelArg::I32 : RuntimeMD::KernelArg::U32;
+    case 64:
+      return Signed ? RuntimeMD::KernelArg::I64 : RuntimeMD::KernelArg::U64;
+    default:
+      // Runtime does not recognize other integer types. Report as struct type.
+      return RuntimeMD::KernelArg::Struct;
+    }
+  }
+  case Type::VectorTyID:
+    return getRuntimeMDValueType(Ty->getVectorElementType(), TypeName);
+  case Type::PointerTyID:
+    return getRuntimeMDValueType(Ty->getPointerElementType(), TypeName);
+  default:
+    return RuntimeMD::KernelArg::Struct;
+  }
+}
+
+static RuntimeMD::KernelArg::AddressSpaceQualifer getRuntimeAddrSpace(
+    AMDGPUAS::AddressSpaces A) {
+  switch (A) {
+  case AMDGPUAS::GLOBAL_ADDRESS:
+    return RuntimeMD::KernelArg::Global;
+  case AMDGPUAS::CONSTANT_ADDRESS:
+    return RuntimeMD::KernelArg::Constant;
+  case AMDGPUAS::LOCAL_ADDRESS:
+    return RuntimeMD::KernelArg::Local;
+  case AMDGPUAS::FLAT_ADDRESS:
+    return RuntimeMD::KernelArg::Generic;
+  case AMDGPUAS::REGION_ADDRESS:
+    return RuntimeMD::KernelArg::Region;
+  default:
+    return RuntimeMD::KernelArg::Private;
+  }
+}
+
+void AMDGPUTargetStreamer::emitRuntimeMetadataForKernelArg(const DataLayout &DL,
+    Type *T, RuntimeMD::KernelArg::Kind Kind,
+    StringRef BaseTypeName, StringRef TypeName,
+    StringRef ArgName, StringRef TypeQual, StringRef AccQual) {
+  auto &S = getStreamer();
+
+  // Emit KeyArgBegin.
+  S.EmitIntValue(RuntimeMD::KeyArgBegin, 1);
+
+  // Emit KeyArgSize and KeyArgAlign.
+  emitRuntimeMDIntValue(RuntimeMD::KeyArgSize,
+                        DL.getTypeAllocSize(T), 4);
+  emitRuntimeMDIntValue(RuntimeMD::KeyArgAlign,
+                        DL.getABITypeAlignment(T), 4);
+  if (auto PT = dyn_cast<PointerType>(T)) {
+    auto ET = PT->getElementType();
+    if (PT->getAddressSpace() == AMDGPUAS::LOCAL_ADDRESS && ET->isSized())
+      emitRuntimeMDIntValue(RuntimeMD::KeyArgPointeeAlign,
+                            DL.getABITypeAlignment(ET), 4);
+  }
+
+  // Emit KeyArgTypeName.
+  if (!TypeName.empty())
+    emitRuntimeMDStringValue(RuntimeMD::KeyArgTypeName, TypeName);
+
+  // Emit KeyArgName.
+  if (!ArgName.empty())
+    emitRuntimeMDStringValue(RuntimeMD::KeyArgName, ArgName);
+
+  // Emit KeyArgIsVolatile, KeyArgIsRestrict, KeyArgIsConst and KeyArgIsPipe.
+  SmallVector<StringRef, 1> SplitQ;
+  TypeQual.split(SplitQ, " ", -1, false /* Drop empty entry */);
+
+  for (StringRef KeyName : SplitQ) {
+    auto Key = StringSwitch<RuntimeMD::Key>(KeyName)
+      .Case("volatile", RuntimeMD::KeyArgIsVolatile)
+      .Case("restrict", RuntimeMD::KeyArgIsRestrict)
+      .Case("const",    RuntimeMD::KeyArgIsConst)
+      .Case("pipe",     RuntimeMD::KeyArgIsPipe)
+      .Default(RuntimeMD::KeyNull);
+    S.EmitIntValue(Key, 1);
+  }
+
+  // Emit KeyArgKind.
+  emitRuntimeMDIntValue(RuntimeMD::KeyArgKind, Kind, 1);
+
+  // Emit KeyArgValueType.
+  emitRuntimeMDIntValue(RuntimeMD::KeyArgValueType,
+                        getRuntimeMDValueType(T, BaseTypeName), 2);
+
+  // Emit KeyArgAccQual.
+  if (!AccQual.empty()) {
+    auto AQ = StringSwitch<RuntimeMD::KernelArg::AccessQualifer>(AccQual)
+      .Case("read_only",  RuntimeMD::KernelArg::ReadOnly)
+      .Case("write_only", RuntimeMD::KernelArg::WriteOnly)
+      .Case("read_write", RuntimeMD::KernelArg::ReadWrite)
+      .Default(RuntimeMD::KernelArg::None);
+    emitRuntimeMDIntValue(RuntimeMD::KeyArgAccQual, AQ, 1);
+  }
+
+  // Emit KeyArgAddrQual.
+  if (auto *PT = dyn_cast<PointerType>(T))
+    emitRuntimeMDIntValue(RuntimeMD::KeyArgAddrQual,
+        getRuntimeAddrSpace(static_cast<AMDGPUAS::AddressSpaces>(
+            PT->getAddressSpace())), 1);
+
+  // Emit KeyArgEnd
+  S.EmitIntValue(RuntimeMD::KeyArgEnd, 1);
+}
+
+void AMDGPUTargetStreamer::emitRuntimeMetadata(const Function &F) {
+  if (!F.getMetadata("kernel_arg_type"))
+    return;
+  auto &S = getStreamer();
+  S.EmitIntValue(RuntimeMD::KeyKernelBegin, 1);
+  emitRuntimeMDStringValue(RuntimeMD::KeyKernelName, F.getName());
+
+  const DataLayout &DL = F.getParent()->getDataLayout();
+  for (auto &Arg : F.args()) {
+    unsigned I = Arg.getArgNo();
+    Type *T = Arg.getType();
+    auto TypeName = dyn_cast<MDString>(F.getMetadata(
+        "kernel_arg_type")->getOperand(I))->getString();
+    auto BaseTypeName = cast<MDString>(F.getMetadata(
+        "kernel_arg_base_type")->getOperand(I))->getString();
+    StringRef ArgName;
+    if (auto ArgNameMD = F.getMetadata("kernel_arg_name"))
+      ArgName = cast<MDString>(ArgNameMD->getOperand(I))->getString();
+    auto TypeQual = cast<MDString>(F.getMetadata(
+        "kernel_arg_type_qual")->getOperand(I))->getString();
+    auto AccQual = cast<MDString>(F.getMetadata(
+        "kernel_arg_access_qual")->getOperand(I))->getString();
+    RuntimeMD::KernelArg::Kind Kind;
+    if (TypeQual.find("pipe") != StringRef::npos)
+      Kind = RuntimeMD::KernelArg::Pipe;
+    else Kind = StringSwitch<RuntimeMD::KernelArg::Kind>(BaseTypeName)
+      .Case("sampler_t", RuntimeMD::KernelArg::Sampler)
+      .Case("queue_t",   RuntimeMD::KernelArg::Queue)
+      .Cases("image1d_t", "image1d_array_t", "image1d_buffer_t",
+             "image2d_t" , "image2d_array_t",  RuntimeMD::KernelArg::Image)
+      .Cases("image2d_depth_t", "image2d_array_depth_t",
+             "image2d_msaa_t", "image2d_array_msaa_t",
+             "image2d_msaa_depth_t",  RuntimeMD::KernelArg::Image)
+      .Cases("image2d_array_msaa_depth_t", "image3d_t",
+             RuntimeMD::KernelArg::Image)
+      .Default(isa<PointerType>(T) ?
+                   (T->getPointerAddressSpace() == AMDGPUAS::LOCAL_ADDRESS ?
+                   RuntimeMD::KernelArg::DynamicSharedPointer :
+                   RuntimeMD::KernelArg::GlobalBuffer) :
+                   RuntimeMD::KernelArg::ByValue);
+    emitRuntimeMetadataForKernelArg(DL, T,
+        Kind, BaseTypeName, TypeName, ArgName, TypeQual, AccQual);
+  }
+
+  // Emit hidden kernel arguments for OpenCL kernels.
+  if (F.getParent()->getNamedMetadata("opencl.ocl.version")) {
+    auto Int64T = Type::getInt64Ty(F.getContext());
+    emitRuntimeMetadataForKernelArg(DL, Int64T,
+                                    RuntimeMD::KernelArg::HiddenGlobalOffsetX);
+    emitRuntimeMetadataForKernelArg(DL, Int64T,
+                                    RuntimeMD::KernelArg::HiddenGlobalOffsetY);
+    emitRuntimeMetadataForKernelArg(DL, Int64T,
+                                    RuntimeMD::KernelArg::HiddenGlobalOffsetZ);
+    if (F.getParent()->getNamedMetadata("llvm.printf.fmts")) {
+      auto Int8PtrT = Type::getInt8PtrTy(F.getContext(),
+          RuntimeMD::KernelArg::Global);
+      emitRuntimeMetadataForKernelArg(DL, Int8PtrT,
+                                      RuntimeMD::KernelArg::HiddenPrintfBuffer);
+    }
+  }
+
+  // Emit KeyReqdWorkGroupSize, KeyWorkGroupSizeHint, and KeyVecTypeHint.
+  if (auto RWGS = F.getMetadata("reqd_work_group_size")) {
+    emitRuntimeMDThreeIntValues(RuntimeMD::KeyReqdWorkGroupSize,
+                                RWGS, 4);
+  }
+
+  if (auto WGSH = F.getMetadata("work_group_size_hint")) {
+    emitRuntimeMDThreeIntValues(RuntimeMD::KeyWorkGroupSizeHint,
+                                WGSH, 4);
+  }
+
+  if (auto VTH = F.getMetadata("vec_type_hint")) {
+    auto TypeName = getOCLTypeName(cast<ValueAsMetadata>(
+      VTH->getOperand(0))->getType(), mdconst::extract<ConstantInt>(
+      VTH->getOperand(1))->getZExtValue());
+    emitRuntimeMDStringValue(RuntimeMD::KeyVecTypeHint, TypeName);
+  }
+
+  // Emit KeyKernelEnd
+  S.EmitIntValue(RuntimeMD::KeyKernelEnd, 1);
+}
+
+void AMDGPUTargetStreamer::emitRuntimeMetadataAsNoteElement(Module &M) {
+  auto &S = getStreamer();
+  auto &Context = S.getContext();
+
+  auto NameSZ = sizeof(PT_NOTE::NoteName); // Size of note name including trailing null.
+
+  S.PushSection();
+  S.SwitchSection(Context.getELFSection(
+      PT_NOTE::SectionName, ELF::SHT_NOTE, ELF::SHF_ALLOC));
+
+  // Create two labels to mark the beginning and end of the desc field
+  // and a MCExpr to calculate the size of the desc field.
+  auto *DescBegin = Context.createTempSymbol();
+  auto *DescEnd = Context.createTempSymbol();
+  auto *DescSZ = MCBinaryExpr::createSub(
+      MCSymbolRefExpr::create(DescEnd, Context),
+      MCSymbolRefExpr::create(DescBegin, Context), Context);
+
+  // Emit the note element for runtime metadata.
+  // Name and desc should be padded to 4 byte boundary but size of name and
+  // desc should not include padding 0's.
+  S.EmitIntValue(NameSZ, 4);                                  // namesz
+  S.EmitValue(DescSZ, 4);                                     // descz
+  S.EmitIntValue(PT_NOTE::NT_AMDGPU_HSA_RUNTIME_METADATA, 4); // type
+  S.EmitBytes(StringRef(PT_NOTE::NoteName, NameSZ));          // name
+  S.EmitValueToAlignment(4);                                  // padding 0
+  S.EmitLabel(DescBegin);
+  emitRuntimeMetadata(M);                                     // desc
+  S.EmitLabel(DescEnd);
+  S.EmitValueToAlignment(4);                                  // padding 0
+  S.PopSection();
+}
+
+void AMDGPUTargetStreamer::emitRuntimeMetadata(Module &M) {
+  emitStartOfRuntimeMetadata(M);
+  for (auto &F : M.functions())
+    emitRuntimeMetadata(F);
+}
+

llvm/trunk/test/CodeGen/AMDGPU/invalid-opencl-version-metadata1.ll

 ; RUN: llc -mtriple=amdgcn--amdhsa < %s | FileCheck %s
 ; check llc does not crash for invalid opencl version metadata
 
-; CHECK: .section        .AMDGPU.runtime_metadata
-; CHECK-NEXT: .byte 1
-; CHECK-NEXT: .short  256
+; CHECK: .section        .note,#alloc
+; CHECK-NEXT: .long   4
+; CHECK-NEXT: .long   {{.+}}
+; CHECK-NEXT: .long   7
+; CHECK-NEXT: .asciz  "AMD"
 
 !opencl.ocl.version = !{}

llvm/trunk/test/CodeGen/AMDGPU/invalid-opencl-version-metadata2.ll

 ; RUN: llc -mtriple=amdgcn--amdhsa < %s | FileCheck %s
 ; check llc does not crash for invalid opencl version metadata
 
-; CHECK: .section        .AMDGPU.runtime_metadata
-; CHECK-NEXT: .byte 1
-; CHECK-NEXT: .short  256
+; CHECK: .section        .note,#alloc
+; CHECK-NEXT: .long   4
+; CHECK-NEXT: .long   {{.+}}
+; CHECK-NEXT: .long   7
+; CHECK-NEXT: .asciz  "AMD"
 
 !opencl.ocl.version = !{!0}
 !0 = !{}

llvm/trunk/test/CodeGen/AMDGPU/invalid-opencl-version-metadata3.ll

 ; RUN: llc -mtriple=amdgcn--amdhsa < %s | FileCheck %s
 ; check llc does not crash for invalid opencl version metadata
 
-; CHECK: .section        .AMDGPU.runtime_metadata
-; CHECK-NEXT: .byte 1
-; CHECK-NEXT: .short  256
+; CHECK: .section        .note,#alloc
+; CHECK-NEXT: .long   4
+; CHECK-NEXT: .long   {{.+}}
+; CHECK-NEXT: .long   7
+; CHECK-NEXT: .asciz  "AMD"
 
 !opencl.ocl.version = !{!0}
 !0 = !{i32 1}

llvm/trunk/test/CodeGen/AMDGPU/runtime-metadata.ll

 ; RUN: llc -mtriple=amdgcn--amdhsa < %s | FileCheck %s
 
 %struct.A = type { i8, float }
 %opencl.image1d_t = type opaque
 %opencl.image2d_t = type opaque
 %opencl.image3d_t = type opaque
 %opencl.queue_t = type opaque
 %opencl.pipe_t = type opaque
 %struct.B = type { i32 addrspace(1)*}
 %opencl.clk_event_t = type opaque
 
-; CHECK: .section        .AMDGPU.runtime_metadata
+; CHECK: .section        .note,#alloc
+; CHECK-NEXT: .long   4
+; CHECK-NEXT: .long   [[Ltmp1:.+]]-[[Ltmp0:.+]]
+; CHECK-NEXT: .long   7
+; CHECK-NEXT: .asciz  "AMD"
+; CHECK-NEXT: .p2align        2
+; CHECK-NEXT: [[Ltmp0]]:
 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .short  256
 ; CHECK-NEXT: .byte 2
 ; CHECK-NEXT: .byte 0
 ; CHECK-NEXT: .byte 3
 ; CHECK-NEXT: .short  200
 ; CHECK-NEXT: .byte 30
 ; CHECK-NEXT: .long 10
 ; CHECK-NEXT: .ascii  "1:1:4:%d\\n"
 ; CHECK-NEXT: .byte 30
 ; CHECK-NEXT: .long 10
 ; CHECK-NEXT: .ascii  "2:1:8:%g\\n"
 
-; CHECK-LABEL:{{^}}test_char:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 9
 ; CHECK-NEXT: .ascii  "test_char"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 1
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_char(i8 %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !9 !kernel_arg_base_type !9 !kernel_arg_type_qual !4 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_ushort2:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 12
 ; CHECK-NEXT: .ascii  "test_ushort2"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 4
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_ushort2(<2 x i16> %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !10 !kernel_arg_base_type !10 !kernel_arg_type_qual !4 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_int3:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 9
 ; CHECK-NEXT: .ascii  "test_int3"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 16
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_int3(<3 x i32> %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !11 !kernel_arg_base_type !11 !kernel_arg_type_qual !4 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_ulong4:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 11
 ; CHECK-NEXT: .ascii  "test_ulong4"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 32
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_ulong4(<4 x i64> %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !12 !kernel_arg_base_type !12 !kernel_arg_type_qual !4 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_half8:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 10
 ; CHECK-NEXT: .ascii  "test_half8"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 16
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_half8(<8 x half> %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !13 !kernel_arg_base_type !13 !kernel_arg_type_qual !4 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_float16:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 12
 ; CHECK-NEXT: .ascii  "test_float16"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 64
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_float16(<16 x float> %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !14 !kernel_arg_base_type !14 !kernel_arg_type_qual !4 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_double16:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 13
 ; CHECK-NEXT: .ascii  "test_double16"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 128
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_double16(<16 x double> %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !15 !kernel_arg_base_type !15 !kernel_arg_type_qual !4 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_pointer:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 12
 ; CHECK-NEXT: .ascii  "test_pointer"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 8
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_pointer(i32 addrspace(1)* %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !16 !kernel_arg_base_type !16 !kernel_arg_type_qual !4 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_image:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 10
 ; CHECK-NEXT: .ascii  "test_image"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 8
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_image(%opencl.image2d_t addrspace(1)* %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !17 !kernel_arg_base_type !17 !kernel_arg_type_qual !4 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_sampler:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 12
 ; CHECK-NEXT: .ascii  "test_sampler"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 4
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_sampler(i32 %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !18 !kernel_arg_base_type !18 !kernel_arg_type_qual !4 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_queue:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 10
 ; CHECK-NEXT: .ascii  "test_queue"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 8
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_queue(%opencl.queue_t addrspace(1)* %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !19 !kernel_arg_base_type !19 !kernel_arg_type_qual !4 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_struct:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 11
 ; CHECK-NEXT: .ascii  "test_struct"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 4
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_struct(%struct.A* byval %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !20 !kernel_arg_base_type !20 !kernel_arg_type_qual !4 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_i128:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 9
 ; CHECK-NEXT: .ascii  "test_i128"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 16
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_i128(i128 %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !21 !kernel_arg_base_type !21 !kernel_arg_type_qual !4 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_multi_arg:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 14
 ; CHECK-NEXT: .ascii  "test_multi_arg"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 4
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_multi_arg(i32 %a, <2 x i16> %b, <3 x i8> %c) !kernel_arg_addr_space !22 !kernel_arg_access_qual !23 !kernel_arg_type !24 !kernel_arg_base_type !24 !kernel_arg_type_qual !25 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_addr_space:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 15
 ; CHECK-NEXT: .ascii  "test_addr_space"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 8
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_addr_space(i32 addrspace(1)* %g, i32 addrspace(2)* %c, i32 addrspace(3)* %l) !kernel_arg_addr_space !50 !kernel_arg_access_qual !23 !kernel_arg_type !51 !kernel_arg_base_type !51 !kernel_arg_type_qual !25 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_type_qual:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 14
 ; CHECK-NEXT: .ascii  "test_type_qual"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 8
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_type_qual(i32 addrspace(1)* %a, i32 addrspace(1)* %b, %opencl.pipe_t addrspace(1)* %c) !kernel_arg_addr_space !22 !kernel_arg_access_qual !23 !kernel_arg_type !51 !kernel_arg_base_type !51 !kernel_arg_type_qual !70 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_access_qual:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 16
 ; CHECK-NEXT: .ascii  "test_access_qual"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 8
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_access_qual(%opencl.image1d_t addrspace(1)* %ro, %opencl.image2d_t addrspace(1)* %wo, %opencl.image3d_t addrspace(1)* %rw) !kernel_arg_addr_space !60 !kernel_arg_access_qual !61 !kernel_arg_type !62 !kernel_arg_base_type !62 !kernel_arg_type_qual !25 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_vec_type_hint_half:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 23
 ; CHECK-NEXT: .ascii  "test_vec_type_hint_half"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 4
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .long 4
 ; CHECK-NEXT: .ascii  "half"
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_vec_type_hint_half(i32 %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !3 !kernel_arg_base_type !3 !kernel_arg_type_qual !4 !vec_type_hint !26 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_vec_type_hint_float:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 24
 ; CHECK-NEXT: .ascii  "test_vec_type_hint_float"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 4
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .long 5
 ; CHECK-NEXT: .ascii  "float"
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_vec_type_hint_float(i32 %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !3 !kernel_arg_base_type !3 !kernel_arg_type_qual !4 !vec_type_hint !27 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_vec_type_hint_double:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 25
 ; CHECK-NEXT: .ascii  "test_vec_type_hint_double"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 4
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .long 6
 ; CHECK-NEXT: .ascii  "double"
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_vec_type_hint_double(i32 %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !3 !kernel_arg_base_type !3 !kernel_arg_type_qual !4 !vec_type_hint !28 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_vec_type_hint_char:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 23
 ; CHECK-NEXT: .ascii  "test_vec_type_hint_char"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 4
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .long 4
 ; CHECK-NEXT: .ascii  "char"
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_vec_type_hint_char(i32 %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !3 !kernel_arg_base_type !3 !kernel_arg_type_qual !4 !vec_type_hint !29 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_vec_type_hint_short:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 24
 ; CHECK-NEXT: .ascii  "test_vec_type_hint_short"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 4
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .long 5
 ; CHECK-NEXT: .ascii  "short"
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_vec_type_hint_short(i32 %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !3 !kernel_arg_base_type !3 !kernel_arg_type_qual !4 !vec_type_hint !30 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_vec_type_hint_long:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 23
 ; CHECK-NEXT: .ascii  "test_vec_type_hint_long"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 4
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .long 4
 ; CHECK-NEXT: .ascii  "long"
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_vec_type_hint_long(i32 %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !3 !kernel_arg_base_type !3 !kernel_arg_type_qual !4 !vec_type_hint !31 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_vec_type_hint_unknown:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 26
 ; CHECK-NEXT: .ascii  "test_vec_type_hint_unknown"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 4
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .long 7
 ; CHECK-NEXT: .ascii  "unknown"
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_vec_type_hint_unknown(i32 %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !3 !kernel_arg_base_type !3 !kernel_arg_type_qual !4 !vec_type_hint !32 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_reqd_wgs_vec_type_hint:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 27
 ; CHECK-NEXT: .ascii  "test_reqd_wgs_vec_type_hint"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 4
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .long 3
 ; CHECK-NEXT: .ascii  "int"
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_reqd_wgs_vec_type_hint(i32 %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !3 !kernel_arg_base_type !3 !kernel_arg_type_qual !4 !vec_type_hint !5 !reqd_work_group_size !6 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_wgs_hint_vec_type_hint:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 27
 ; CHECK-NEXT: .ascii  "test_wgs_hint_vec_type_hint"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 4
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .long 5
 ; CHECK-NEXT: .ascii  "uint4"
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_wgs_hint_vec_type_hint(i32 %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !3 !kernel_arg_base_type !3 !kernel_arg_type_qual !4 !vec_type_hint !7 !work_group_size_hint !8 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_arg_ptr_to_ptr:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 19
 ; CHECK-NEXT: .ascii  "test_arg_ptr_to_ptr"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 8
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_arg_ptr_to_ptr(i32 * addrspace(1)* %a) !kernel_arg_addr_space !81 !kernel_arg_access_qual !2 !kernel_arg_type !80 !kernel_arg_base_type !80 !kernel_arg_type_qual !4 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_arg_struct_contains_ptr:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 28
 ; CHECK-NEXT: .ascii  "test_arg_struct_contains_ptr"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 4
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_arg_struct_contains_ptr(%struct.B * byval %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !82 !kernel_arg_base_type !82 !kernel_arg_type_qual !4 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_arg_vector_of_ptr:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 22
 ; CHECK-NEXT: .ascii  "test_arg_vector_of_ptr"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 16
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_arg_vector_of_ptr(<2 x i32 addrspace(1)*> %a) !kernel_arg_addr_space !1 !kernel_arg_access_qual !2 !kernel_arg_type !83 !kernel_arg_base_type !83 !kernel_arg_type_qual !4 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_arg_unknown_builtin_type:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 29
 ; CHECK-NEXT: .ascii  "test_arg_unknown_builtin_type"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 8
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 
 define amdgpu_kernel void @test_arg_unknown_builtin_type(%opencl.clk_event_t addrspace(1)* %a) !kernel_arg_addr_space !81 !kernel_arg_access_qual !2 !kernel_arg_type !84 !kernel_arg_base_type !84 !kernel_arg_type_qual !4 {
   ret void
 }
 
-; CHECK-LABEL:{{^}}test_pointee_align:
-; CHECK: .section        .AMDGPU.runtime_metadata
 ; CHECK-NEXT: .byte 4
 ; CHECK-NEXT: .byte 6
 ; CHECK-NEXT: .long 18
 ; CHECK-NEXT: .ascii  "test_pointee_align"
 ; CHECK-NEXT: .byte 7
 ; CHECK-NEXT: .byte 9
 ; CHECK-NEXT: .long 8
 ; CHECK-NEXT: .byte 10

 ; CHECK-NEXT: .byte 15
 ; CHECK-NEXT: .byte 1
 ; CHECK-NEXT: .byte 8
 ; CHECK-NEXT: .byte 5
 
 define amdgpu_kernel void @test_pointee_align(i64 addrspace(1)* %a, i8 addrspace(3)* %b, <2 x i8> addrspace(3)* %c, <3 x i8> addrspace(3)* %d, <4 x i8> addrspace(3)* %e, <8 x i8> addrspace(3)* %f, <16 x i8> addrspace(3)* %g) !kernel_arg_addr_space !91 !kernel_arg_access_qual !92 !kernel_arg_type !93 !kernel_arg_base_type !93 !kernel_arg_type_qual !94 {
   ret void
 }
+; CHECK-NEXT: [[Ltmp1]]:
+; CHECK-NEXT: .p2align        2
 
 !llvm.printf.fmts = !{!100, !101}
 
 !1 = !{i32 0}
 !2 = !{!"none"}
 !3 = !{!"int"}
 !4 = !{!""}
 !5 = !{i32 undef, i32 1}