AMDGPU: Add support for global variable sections

[ruiu]

Before reviewing these patches, I'd like to know your plan about LLD and
AMDGPU suppport. You are adding code to the old LLD, but it's pretty likely
that we'll start handling the new LLD as default for ELF. Can you please
explain your plan? Thanks.

[• tstellarAMD]

In D14626#288460, @ruiu wrote:

Before reviewing these patches, I'd like to know your plan about LLD and
AMDGPU suppport. You are adding code to the old LLD, but it's pretty likely
that we'll start handling the new LLD as default for ELF. Can you please
explain your plan? Thanks.

HI Rui,

My current plan is to implement the AMD 'code object' format that is understood by the HSA runtime. The code object contains the program ISA and also other information that the GPU needs to execute the code.

This format is relatively simple, it's basically a static archive with program headers and a few other things. Besides the patches I've submitted, I have a few other local patches that I need to clean up and send for review. On my local branch, I have implemented a majority of the features required, and I don't anticipate a lot of major changes after that.
I would really like to get all these patches upstream, just so I have a complete implementation that people can use.

[• rafael]

But the question in why in the old lld? The idea is to delete it once
ELF2 catches up, which should be really soon now.

[• tstellarAMD]

In D14626#289031, @rafael wrote:

But the question in why in the old lld? The idea is to delete it once
ELF2 catches up, which should be really soon now.

I'm just using old lld because of inertia. I was aware of ELF2, but I didn't really know what the deprecation plan was or when it would be ready, so I just kept working on the existing AMDGPU target that used old ELF.

[ruiu]

I can probably say that the new LLD is now as ready as the old one for ELF. Especially if you are adding new features, you want to work on the new one because that's where active development is going.

We don't have a deadline or something right now, but it's pretty likely that we'll ask you to port your features to the new one in future if you add that to the old one.

[• tstellarAMD]

I've sent a patch adding AMDGPU support to new ELF.

lib/ReaderWriter/ELF/AMDGPU/AMDGPUELFFile.cpp

ELFDefinedAtom<ELF64LE> *AMDGPUELFFile::createDefinedAtom(
     unsigned int referenceStart, unsigned int referenceEnd,
     std::vector<ELFReference<ELF64LE> *> &referenceList) {
 
   // The st_value field in the symbol table for symbols in the .hsatext
   // section must be section relative, so we need to store the original
   // symbol offset, which is section relative, so we can write it to the
   // symbol table in AMDDGPUSymbolTable::finalize().
   bool HasSectionRelativeOffset = false;
-  if (sectionName == ".hsatext" || sectionName == ".hsarodata_readonly_agent")
+  if (sectionName == ".hsatext" || sectionName == ".hsarodata_readonly_agent" ||
+      sectionName == ".hsadata_global_program" ||
+      sectionName == ".hsadata_global_agent")
     HasSectionRelativeOffset = true;
 
   return new (this->_readerStorage) AMDGPUELFDefinedAtom(
       *this, symName, sectionName, sym, sectionHdr, contentData, referenceStart,
       referenceEnd, referenceList, HasSectionRelativeOffset);
 }
 
 } // elf
 } // lld

lib/ReaderWriter/ELF/AMDGPU/AMDGPUSectionChunks.h

 namespace lld {
 namespace elf {
 
 class HSARoDataReadOnlySection : public AtomSection<ELF64LE> {
 public:
   HSARoDataReadOnlySection(const ELFLinkingContext &ctx);
 };
 
+class HSADataGlobalProgramSection : public AtomSection <ELF64LE> {
+public:
+  HSADataGlobalProgramSection(const ELFLinkingContext &ctx);
+};
+
+class HSADataGlobalAgentSection : public AtomSection <ELF64LE> {
+public:
+  HSADataGlobalAgentSection(const ELFLinkingContext &ctx);
+};
+
 } // elf
 } // lld
 
 #endif

lib/ReaderWriter/ELF/AMDGPU/AMDGPUSectionChunks.cpp

HSARoDataReadOnlySection::HSARoDataReadOnlySection(const ELFLinkingContext &ctx)
   _flags = llvm::ELF::SHF_ALLOC | llvm::ELF::SHF_AMDGPU_HSA_READONLY |
            llvm::ELF::SHF_AMDGPU_HSA_AGENT;
 
   // FIXME: What is the correct alignment?
   _alignment = 4;
 
 }
 
+HSADataGlobalProgramSection::HSADataGlobalProgramSection(const ELFLinkingContext &ctx)
+  : AtomSection(ctx, ".hsadata_global_program", DefinedAtom::typeData, 0, 0) {
+
+  _type = llvm::ELF::SHT_PROGBITS;
+  _flags = llvm::ELF::SHF_ALLOC | llvm::ELF::SHF_WRITE |
+           llvm::ELF::SHF_AMDGPU_HSA_GLOBAL;
+
+  // FIXME: What is the correct alignment?
+  _alignment = 4;
+
+}
+
+HSADataGlobalAgentSection::HSADataGlobalAgentSection(const ELFLinkingContext &ctx)
+  : AtomSection(ctx, ".hsadata_global_agent", DefinedAtom::typeData, 0, 0) {
+
+  _type = llvm::ELF::SHT_PROGBITS;
+  _flags = llvm::ELF::SHF_ALLOC | llvm::ELF::SHF_WRITE |
+           llvm::ELF::SHF_AMDGPU_HSA_GLOBAL |
+           llvm::ELF::SHF_AMDGPU_HSA_AGENT;
+
+  // FIXME: What is the correct alignment?
+  _alignment = 4;
+
+}
+
 } // elf
 } // lld

lib/ReaderWriter/ELF/AMDGPU/AMDGPUTargetHandler.h

createSection(StringRef name, int32_t contentType,
                 DefinedAtom::ContentPermissions contentPermissions,
                 TargetLayout::SectionOrder sectionOrder) override {
     if (name == ".hsatext")
       return new (_allocator) HSATextSection(_ctx);
 
     if (name == ".hsarodata_readonly_agent")
       return new (_allocator) HSARoDataReadOnlySection(_ctx);
 
+    if (name == ".hsadata_global_program")
+      return new (_allocator) HSADataGlobalProgramSection(_ctx);
+
+    if (name == ".hsadata_global_agent")
+      return new (_allocator) HSADataGlobalAgentSection(_ctx);
+
     if (name == ".note")
       contentType = DefinedAtom::typeRONote;
 
     return TargetLayout::createSection(name, contentType, contentPermissions,
                                        sectionOrder);
   }
 };
 

lib/ReaderWriter/ELF/AMDGPU/AMDGPUTargetHandler.cpp

for (Section<ELF64LE> *section : osi->sections()) {
       if (InputSectionName == ".hsatext") {
         Segment<ELF64LE> *segment = new (_allocator) Segment<ELF64LE>(
             _ctx, "PT_AMDGPU_HSA_LOAD_CODE_AGENT", PT_AMDGPU_HSA_LOAD_CODE_AGENT);
         _segments.push_back(segment);
         assert(segment);
         segment->append(section);
       }
 
+      if (InputSectionName == ".hsadata_global_agent") {
+        Segment<ELF64LE> *segment = new (_allocator) Segment<ELF64LE>(_ctx,
+            "PT_AMDGPU_HSA_LOAD_GLOBAL_AGENT", PT_AMDGPU_HSA_LOAD_GLOBAL_AGENT);
+        _segments.push_back(segment);
+        segment->append(section);
+      }
 
       if (InputSectionName == ".hsarodata_readonly_agent") {
         Segment<ELF64LE> *segment = new (_allocator) Segment<ELF64LE>(_ctx,
             "PT_AMDGPU_HSA_LOAD_READONLY_AGENT", PT_AMDGPU_HSA_LOAD_READONLY_AGENT);
         _segments.push_back(segment);
         segment->append(section);
       }
+
+      if (InputSectionName == ".hsadata_global_program") {
+        Segment<ELF64LE> *segment = new (_allocator) Segment<ELF64LE>(_ctx,
+            "PT_AMDGPU_HSA_LOAD_GLOBAL_PROGRAM", PT_AMDGPU_HSA_LOAD_GLOBAL_PROGRAM);
+        _segments.push_back(segment);
+        segment->append(section);
+      }
     }
   }
 }
 
 } // namespace elf
 } // namespace lld

test/elf/AMDGPU/hsa-sections.test

 # RUN: yaml2obj -format=elf %s > %t.obj
 # RUN: lld -flavor gnu -target amdgcn--hsa %t.obj -o %t.exe --noinhibit-exec
 # RUN: llvm-readobj -h -program-headers -s -symbols %t.exe | FileCheck %s
 
 # CHECK: Section {
+# CHECK: Name: .hsadata_global_agent
+# CHECK: Type: SHT_PROGBITS (0x1)
+# CHECK: Flags [ (0x900003
+# CHECK: SHF_ALLOC (0x2)
+# CHECK: SHF_AMDGPU_HSA_AGENT (0x800000)
+# CHECK: SHF_AMDGPU_HSA_GLOBAL (0x100000)
+# CHECK: SHF_WRITE (0x1)
+# CHECK: ]
+# CHECK: }
+
+# CHECK: Section {
+# CHECK: Name: .hsadata_global_program
+# CHECK: Type: SHT_PROGBITS (0x1)
+# CHECK: Flags [ (0x100003
+# CHECK: SHF_ALLOC (0x2)
+# CHECK: SHF_AMDGPU_HSA_GLOBAL (0x100000)
+# CHECK: SHF_WRITE (0x1)
+# CHECK: ]
+# CHECK: }
+
+# CHECK: Section {
 # CHECK: Name: .hsarodata_readonly_agent
 # CHECK: Type: SHT_PROGBITS
 # CHECK: Flags [ (0xA00002
 # CHECK: SHF_ALLOC (0x2)
 # CHECK: SHF_AMDGPU_HSA_AGENT (0x800000) 
 # CHECK: SHF_AMDGPU_HSA_READONLY (0x200000)
 # CHECK: ]
 # CHECK: }
 
+
+# CHECK: ProgramHeader {
+# CHECK: Type: PT_AMDGPU_HSA_LOAD_GLOBAL_PROGRAM (0x60000000)
+# CHECK: }
+
+# CHECK: ProgramHeader {
+# CHECK: Type: PT_AMDGPU_HSA_LOAD_GLOBAL_AGENT (0x60000001)
+# CHECK: }
+
 # CHECK: ProgramHeader {
 # CHECK: Type: PT_AMDGPU_HSA_LOAD_READONLY_AGENT (0x60000002)
 # CHECK: }
 
 ---
 FileHeader:
   Class:           ELFCLASS64
   Data:            ELFDATA2LSB
   OSABI:           ELFOSABI_GNU
   Type:            ET_REL
   Machine:         EM_AMDGPU
 Sections:
+  - Name: .hsadata_global_agent
+    Type:  SHT_PROGBITS
+    Flags: [ SHF_ALLOC, SHF_WRITE, SHF_AMDGPU_HSA_GLOBAL, SHF_AMDGPU_HSA_AGENT ]
+
+  - Name: .hsadata_global_program
+    Type: SHT_PROGBITS
+    Flags: [ SHF_ALLOC, SHF_WRITE, SHF_AMDGPU_HSA_GLOBAL ]
+
   - Name: .hsarodata_readonly_agent
     Type: SHT_PROGBITS
     Flags: [ SHF_ALLOC, SHF_AMDGPU_HSA_AGENT, SHF_AMDGPU_HSA_READONLY ]
 ...