[ObjectYAML] Add offloading binary implementations for obj2yaml and yaml2obj

[jhuber6]

Summary

This patchs adds the necessary code for inspecting or creating offloading
binaries using the standing obj2yaml and yaml2obj features in LLVM.

Depends on D127774

[jhenderson]

Have you got any reference material that I can read for the format of offloading binaries? I can't really review that this implementation does the right thing without knowing the format!

llvm/lib/ObjectYAML/OffloadYAML.cpp

27

Here and with all the other fields, you need test cases that show that, if they're optional, the fields can be omitted (and what their default values are if so), and if they're required, an error is emitted if the field is missing.

52

Here and below, these error messages should start with a lower-case letter, I suspect (see the Coding Standards for error message format).

That being said, in ELF yaml2obj, we allow specification of values that may not be permitted by the spec but could theoretically appear in the field, if something wrote the wrong value. This allows for testing of error paths more effectively. I can't give further pointers though without knowing the format and what makes sense to allow to handcraft.

llvm/test/ObjectYAML/Offload/multiple_members.yaml

2

I don't think you need --strict-whitespace as you're not really interested in the whitespace in this test.

I'm not sure there's much benefit in having a single and a multiple entry test case. I'd just have the multiple test case, and drop the single one.

13

Let's use a different Content for the two members. It avoids any risk of a content field being cached from one member to the other.

llvm/tools/obj2yaml/offload2yaml.cpp

2

This file isn't in "utils", so the header tag here is wrong.

[jhuber6]

In D127776#3584397, @jhenderson wrote:

Have you got any reference material that I can read for the format of offloading binaries? I can't really review that this implementation does the right thing without knowing the format!

Sure, it's documented here https://clang.llvm.org/docs/ClangOffloadPackager.html. More or less, it's just a few integers followed by a list of offsets into an Elf-like string table that forms a string map.

llvm/lib/ObjectYAML/OffloadYAML.cpp

27

It's not really important that these be optional since there's not that many of them. I may just make them all required to avoid the hassle.

52

Honestly this error message isn't strictly necessary. In the linker wrapper where we actually use these binaries we handle unknown cases, so I may just remove these validation steps.

llvm/tools/obj2yaml/offload2yaml.cpp

2

Copied the header from archive2yaml.cpp which should probably be fixed as well.

[jhuber6]

Addressing comments.

[jhenderson]

llvm/include/llvm/ObjectYAML/OffloadYAML.h

32–33

I'd rather expect these two to be enums that correspond to the IMG and OFK values in the spec, with the option of specifying an arbitrary uint16_t value so that you can provide arbitrary values that don't have a defined mapping (this is a common pattern in ELF yaml2obj, for example).

39

The spec discsusses a wider range of fields than just the members themselves. In particular, I'd expect to see an optional Version field, so that you can test using an arbitrary version number. Having optional size and offset fields would also allow you to override the defaults of these values, allowing you to test error paths better (e.g. to show that if the offset is past the end of the file, an error message is reported). The one field you probably don't need is the Magic field, in my opinion.

llvm/test/ObjectYAML/Offload/default.yaml

13

Nit: looks like you've got multiple blank lines at EOF. The file should end in exactly one '\n', no more, no less.

[jhuber6]

llvm/include/llvm/ObjectYAML/OffloadYAML.h

39

Those field are internal to the binary format, and testing them is done by even being able to extract the members correctly in obj2yaml. I'd prefer not to add in ways to change it as it would massively clutter up the API for creating them. I don't see the utility in changing any of the offsets because there's no way to know if they are valid without crashing or printing out garbage, so we just assume every binary with a valid magic number is valid. The version only exists to change the ABI in the future if we ever need to, it's a constant as far as I'm concerned and if it's not 1 we just hard error immediately. I could add it in if you really, really, really think it's worth testing these paths. But I would rather not.

[jhuber6]

Moving to enumeration values.

[jhenderson]

llvm/include/llvm/ObjectYAML/OffloadYAML.h

39

The purpose of having yaml2obj support for file formats is primarily to be able to create test cases for both common use-cases and edge cases, particularly malformed input cases. For example, in the ELF YAML format, we allow overriding the sh_offset and other section header fields with values that wouldn't make sense. This in turn allows us to write tests for things like "llvm-readobj emits an error (instead of crashing) if the section header offset points past the end of the file)". So, the utility for changing the offsets is to be able to craft test cases that show that tools don't crash and instead give a sensible error if being given invalid values (clearly we can't usually tell the difference between a valid offset and another offset that points within the file, but we can at least make sure the offset is not past the end of the file). Similarly, how do you test that the tool correctly emits an error if a version > 1 is detected without having the ability to change the version field? Certainly in tools like llvm-readobj and llvm-objdump, we expect test cases for every code path (including error paths). This is important because we have to be able to rely on these low-level tools because they are so widely used in testing other tools.

obj2yaml support is not all that important typically, but just sort of falls out of the wash typically because of how it interacts with yaml2obj.

[jhuber6]

Add tweaking of specific binary features and add a test to make sure we handle it explicitly.

[jhenderson]

llvm/test/ObjectYAML/Offload/malformed.yaml

1 ↗

(On Diff #439417)

Thanks for the update, mostly looks good, but can I request this be split into a sequence of different test cases within the same file, where you only set one of the fields in each case? In other words, one with Version 2, another with just Size set etc.

[jhuber6]

Splitting tests

[jhenderson]

LGTM.

llvm/test/ObjectYAML/Offload/malformed-offset.yaml

16

I assume the body of this error message comes from code outside of this patch?

llvm/include/llvm/Object/OffloadBinary.h

public:
   string_iterator_range strings() const {
     return string_iterator_range(StringData.begin(), StringData.end());
   }
 
   StringRef getString(StringRef Key) const { return StringData.lookup(Key); }
 
   static bool classof(const Binary *V) { return V->isOffloadFile(); }
 
-private:
   struct Header {
     uint8_t Magic[4] = {0x10, 0xFF, 0x10, 0xAD}; // 0x10FF10AD magic bytes.
     uint32_t Version = OffloadBinary::Version;   // Version identifier.
     uint64_t Size;        // Size in bytes of this entire binary.
     uint64_t EntryOffset; // Offset of the metadata entry in bytes.
     uint64_t EntrySize;   // Size of the metadata entry in bytes.
   };
 
   struct Entry {
     ImageKind TheImageKind;     // The kind of the image stored.
     OffloadKind TheOffloadKind; // The producer of this image.
     uint32_t Flags;             // Additional flags associated with the image.
     uint64_t StringOffset;      // Offset in bytes to the string map.
     uint64_t NumStrings;        // Number of entries in the string map.
     uint64_t ImageOffset;       // Offset in bytes of the actual binary image.
     uint64_t ImageSize;         // Size in bytes of the binary image.
   };
 
   struct StringEntry {
     uint64_t KeyOffset;
     uint64_t ValueOffset;
   };
 
+private:
   OffloadBinary(MemoryBufferRef Source, const Header *TheHeader,
                 const Entry *TheEntry)
       : Binary(Binary::ID_Offload, Source), Buffer(Source.getBufferStart()),
         TheHeader(TheHeader), TheEntry(TheEntry) {
     const StringEntry *StringMapBegin =
         reinterpret_cast<const StringEntry *>(&Buffer[TheEntry->StringOffset]);
     for (uint64_t I = 0, E = TheEntry->NumStrings; I != E; ++I) {
       StringRef Key = &Buffer[StringMapBegin[I].KeyOffset];

llvm/include/llvm/ObjectYAML/ObjectYAML.h

 #define LLVM_OBJECTYAML_OBJECTYAML_H
 
 #include "llvm/ObjectYAML/ArchiveYAML.h"
 #include "llvm/ObjectYAML/COFFYAML.h"
 #include "llvm/ObjectYAML/DXContainerYAML.h"
 #include "llvm/ObjectYAML/ELFYAML.h"
 #include "llvm/ObjectYAML/MachOYAML.h"
 #include "llvm/ObjectYAML/MinidumpYAML.h"
+#include "llvm/ObjectYAML/OffloadYAML.h"
 #include "llvm/ObjectYAML/WasmYAML.h"
 #include "llvm/ObjectYAML/XCOFFYAML.h"
 #include "llvm/Support/YAMLTraits.h"
 #include <memory>
 
 namespace llvm {
 namespace yaml {
 
 class IO;
 
 struct YamlObjectFile {
   std::unique_ptr<ArchYAML::Archive> Arch;
   std::unique_ptr<ELFYAML::Object> Elf;
   std::unique_ptr<COFFYAML::Object> Coff;
   std::unique_ptr<MachOYAML::Object> MachO;
   std::unique_ptr<MachOYAML::UniversalBinary> FatMachO;
   std::unique_ptr<MinidumpYAML::Object> Minidump;
+  std::unique_ptr<OffloadYAML::Binary> Offload;
   std::unique_ptr<WasmYAML::Object> Wasm;
   std::unique_ptr<XCOFFYAML::Object> Xcoff;
   std::unique_ptr<DXContainerYAML::Object> DXContainer;
 };
 
 template <> struct MappingTraits<YamlObjectFile> {
   static void mapping(IO &IO, YamlObjectFile &ObjectFile);
 };
 
 } // end namespace yaml
 } // end namespace llvm
 
 #endif // LLVM_OBJECTYAML_OBJECTYAML_H

llvm/include/llvm/ObjectYAML/OffloadYAML.h

+//===- OffloadYAML.h - Offload Binary YAMLIO implementation -----*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// This file declares classes for handling the YAML representation of
+/// offloading binaries.
+///
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_OBJECTYAML_OFFLOADYAML_H
+#define LLVM_OBJECTYAML_OFFLOADYAML_H
+
+#include "llvm/ADT/MapVector.h"
+#include "llvm/Object/OffloadBinary.h"
+#include "llvm/ObjectYAML/YAML.h"
+#include "llvm/Support/YAMLTraits.h"
+
+namespace llvm {
+namespace OffloadYAML {
+
+struct Binary {
+  struct StringEntry {
+    StringRef Key;
+    StringRef Value;
+  };
+
+  struct Member {
+    Optional<object::ImageKind> ImageKind;
+    Optional<object::OffloadKind> OffloadKind;
+    Optional<uint32_t> Flags;
+    Optional<std::vector<StringEntry>> StringEntries;
+    Optional<yaml::BinaryRef> Content;
+  };
+
+  Optional<uint32_t> Version;
+  Optional<uint64_t> Size;
+  Optional<uint64_t> EntryOffset;
+  Optional<uint64_t> EntrySize;
+  std::vector<Member> Members;
+};
+
+} // end namespace OffloadYAML
+} // end namespace llvm
+
+LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::OffloadYAML::Binary::Member)
+LLVM_YAML_IS_SEQUENCE_VECTOR(llvm::OffloadYAML::Binary::StringEntry)
+
+namespace llvm {
+namespace yaml {
+
+template <> struct ScalarEnumerationTraits<object::ImageKind> {
+  static void enumeration(IO &IO, object::ImageKind &Value);
+};
+
+template <> struct ScalarEnumerationTraits<object::OffloadKind> {
+  static void enumeration(IO &IO, object::OffloadKind &Value);
+};
+
+template <> struct MappingTraits<OffloadYAML::Binary> {
+  static void mapping(IO &IO, OffloadYAML::Binary &O);
+};
+
+template <> struct MappingTraits<OffloadYAML::Binary::StringEntry> {
+  static void mapping(IO &IO, OffloadYAML::Binary::StringEntry &M);
+};
+
+template <> struct MappingTraits<OffloadYAML::Binary::Member> {
+  static void mapping(IO &IO, OffloadYAML::Binary::Member &M);
+};
+
+} // end namespace yaml
+} // end namespace llvm
+
+#endif // LLVM_OBJECTYAML_ARCHIVEYAML_H

llvm/include/llvm/ObjectYAML/yaml2obj.h

 namespace ELFYAML {
 struct Object;
 }
 
 namespace MinidumpYAML {
 struct Object;
 }
 
+namespace OffloadYAML {
+struct Binary;
+}
+
 namespace WasmYAML {
 struct Object;
 }
 
 namespace XCOFFYAML {
 struct Object;
 }
 

 
 bool yaml2archive(ArchYAML::Archive &Doc, raw_ostream &Out, ErrorHandler EH);
 bool yaml2coff(COFFYAML::Object &Doc, raw_ostream &Out, ErrorHandler EH);
 bool yaml2elf(ELFYAML::Object &Doc, raw_ostream &Out, ErrorHandler EH,
               uint64_t MaxSize);
 bool yaml2macho(YamlObjectFile &Doc, raw_ostream &Out, ErrorHandler EH);
 bool yaml2minidump(MinidumpYAML::Object &Doc, raw_ostream &Out,
                    ErrorHandler EH);
+bool yaml2offload(OffloadYAML::Binary &Doc, raw_ostream &Out, ErrorHandler EH);
 bool yaml2wasm(WasmYAML::Object &Doc, raw_ostream &Out, ErrorHandler EH);
 bool yaml2xcoff(XCOFFYAML::Object &Doc, raw_ostream &Out, ErrorHandler EH);
 bool yaml2dxcontainer(DXContainerYAML::Object &Doc, raw_ostream &Out,
                       ErrorHandler EH);
 
 bool convertYAML(Input &YIn, raw_ostream &Out, ErrorHandler ErrHandler,
                  unsigned DocNum = 1, uint64_t MaxSize = UINT64_MAX);
 

llvm/lib/ObjectYAML/CMakeLists.txt

add_llvm_component_library(LLVMObjectYAML
   DXContainerYAML.cpp
   ELFEmitter.cpp
   ELFYAML.cpp
   MachOEmitter.cpp
   MachOYAML.cpp
   ObjectYAML.cpp
   MinidumpEmitter.cpp
   MinidumpYAML.cpp
+  OffloadEmitter.cpp
+  OffloadYAML.cpp
   WasmEmitter.cpp
   WasmYAML.cpp
   XCOFFEmitter.cpp
   XCOFFYAML.cpp
   YAML.cpp
   yaml2obj.cpp
 
   ADDITIONAL_HEADER_DIRS

llvm/lib/ObjectYAML/ObjectYAML.cpp

if (IO.mapTag("!Arch")) {
       MappingTraits<MachOYAML::Object>::mapping(IO, *ObjectFile.MachO);
     } else if (IO.mapTag("!fat-mach-o")) {
       ObjectFile.FatMachO.reset(new MachOYAML::UniversalBinary());
       MappingTraits<MachOYAML::UniversalBinary>::mapping(IO,
                                                          *ObjectFile.FatMachO);
     } else if (IO.mapTag("!minidump")) {
       ObjectFile.Minidump.reset(new MinidumpYAML::Object());
       MappingTraits<MinidumpYAML::Object>::mapping(IO, *ObjectFile.Minidump);
+    } else if (IO.mapTag("!Offload")) {
+      ObjectFile.Offload.reset(new OffloadYAML::Binary());
+      MappingTraits<OffloadYAML::Binary>::mapping(IO, *ObjectFile.Offload);
     } else if (IO.mapTag("!WASM")) {
       ObjectFile.Wasm.reset(new WasmYAML::Object());
       MappingTraits<WasmYAML::Object>::mapping(IO, *ObjectFile.Wasm);
     } else if (IO.mapTag("!XCOFF")) {
       ObjectFile.Xcoff.reset(new XCOFFYAML::Object());
       MappingTraits<XCOFFYAML::Object>::mapping(IO, *ObjectFile.Xcoff);
     } else if (IO.mapTag("!dxcontainer")) {
       ObjectFile.DXContainer.reset(new DXContainerYAML::Object());

llvm/lib/ObjectYAML/OffloadEmitter.cpp

+//===- OffloadEmitter.cpp -------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Object/OffloadBinary.h"
+#include "llvm/ObjectYAML/OffloadYAML.h"
+#include "llvm/ObjectYAML/yaml2obj.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/raw_ostream.h"
+
+using namespace llvm;
+using namespace OffloadYAML;
+
+namespace llvm {
+namespace yaml {
+
+bool yaml2offload(Binary &Doc, raw_ostream &Out, ErrorHandler EH) {
+  for (const auto &Member : Doc.Members) {
+    object::OffloadBinary::OffloadingImage Image{};
+    if (Member.ImageKind)
+      Image.TheImageKind = *Member.ImageKind;
+    if (Member.OffloadKind)
+      Image.TheOffloadKind = *Member.OffloadKind;
+    if (Member.Flags)
+      Image.Flags = *Member.Flags;
+
+    StringMap<StringRef> &StringData = Image.StringData;
+    if (Member.StringEntries) {
+      for (const auto &Entry : *Member.StringEntries) {
+        StringData[Entry.Key] = Entry.Value;
+      }
+    }
+
+    SmallVector<char, 1024> Data;
+    raw_svector_ostream OS(Data);
+    if (Member.Content)
+      Member.Content->writeAsBinary(OS);
+    Image.Image = MemoryBuffer::getMemBufferCopy(OS.str());
+
+    std::unique_ptr<MemoryBuffer> Binary = object::OffloadBinary::write(Image);
+
+    // Copy the data to a new buffer so we can modify the bytes directly.
+    SmallVector<char> NewBuffer;
+    std::copy(Binary->getBufferStart(), Binary->getBufferEnd(),
+              std::back_inserter(NewBuffer));
+    auto *TheHeader =
+        reinterpret_cast<object::OffloadBinary::Header *>(&NewBuffer[0]);
+    if (Doc.Version)
+      TheHeader->Version = *Doc.Version;
+    if (Doc.Size)
+      TheHeader->Size = *Doc.Size;
+    if (Doc.EntryOffset)
+      TheHeader->EntryOffset = *Doc.EntryOffset;
+    if (Doc.EntrySize)
+      TheHeader->EntrySize = *Doc.EntrySize;
+
+    Out.write(NewBuffer.begin(), NewBuffer.size());
+  }
+
+  return true;
+}
+
+} // namespace yaml
+} // namespace llvm

llvm/lib/ObjectYAML/OffloadYAML.cpp

+//===- OffloadYAML.cpp - Offload Binary YAMLIO implementation -------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file defines classes for handling the YAML representation of offload
+// binaries.
+//
+//===----------------------------------------------------------------------===//
+
+#include <llvm/ObjectYAML/OffloadYAML.h>
+
+namespace llvm {
+
+namespace yaml {
+
+void ScalarEnumerationTraits<object::ImageKind>::enumeration(
+    IO &IO, object::ImageKind &Value) {
+#define ECase(X) IO.enumCase(Value, #X, object::X)
+  ECase(IMG_None);
+  ECase(IMG_Object);
+  ECase(IMG_Bitcode);
+  ECase(IMG_Cubin);
+  ECase(IMG_Fatbinary);
+  ECase(IMG_PTX);
+  ECase(IMG_LAST);
+#undef ECase
+  IO.enumFallback<Hex16>(Value);
+}
+
+void ScalarEnumerationTraits<object::OffloadKind>::enumeration(
+    IO &IO, object::OffloadKind &Value) {
+#define ECase(X) IO.enumCase(Value, #X, object::X)
+  ECase(OFK_None);
+  ECase(OFK_OpenMP);
+  ECase(OFK_Cuda);
+  ECase(OFK_HIP);
+  ECase(OFK_LAST);
+#undef ECase
+  IO.enumFallback<Hex16>(Value);
+}
+
+void MappingTraits<OffloadYAML::Binary>::mapping(IO &IO,
+                                                 OffloadYAML::Binary &O) {
+  assert(!IO.getContext() && "The IO context is initialized already");
+  IO.setContext(&O);
+  IO.mapTag("!Offload", true);
+  IO.mapOptional("Version", O.Version);
+  IO.mapOptional("Size", O.Size);
+  IO.mapOptional("EntryOffset", O.EntryOffset);
+  IO.mapOptional("EntrySize", O.EntrySize);
+  IO.mapRequired("Members", O.Members);
+  IO.setContext(nullptr);
+}
+
+void MappingTraits<OffloadYAML::Binary::StringEntry>::mapping(
+    IO &IO, OffloadYAML::Binary::StringEntry &SE) {
+  assert(IO.getContext() && "The IO context is not initialized");
+  IO.mapRequired("Key", SE.Key);
+  IO.mapRequired("Value", SE.Value);
+}
+
+void MappingTraits<OffloadYAML::Binary::Member>::mapping(
+    IO &IO, OffloadYAML::Binary::Member &M) {
+  assert(IO.getContext() && "The IO context is not initialized");
+  IO.mapOptional("ImageKind", M.ImageKind);
+  IO.mapOptional("OffloadKind", M.OffloadKind);
+  IO.mapOptional("Flags", M.Flags);
+  IO.mapOptional("String", M.StringEntries);
+  IO.mapOptional("Content", M.Content);
+}
+
+} // namespace yaml
+
+} // namespace llvm

llvm/lib/ObjectYAML/yaml2obj.cpp

do {
     if (Doc.Elf)
       return yaml2elf(*Doc.Elf, Out, ErrHandler, MaxSize);
     if (Doc.Coff)
       return yaml2coff(*Doc.Coff, Out, ErrHandler);
     if (Doc.MachO || Doc.FatMachO)
       return yaml2macho(Doc, Out, ErrHandler);
     if (Doc.Minidump)
       return yaml2minidump(*Doc.Minidump, Out, ErrHandler);
+    if (Doc.Offload)
+      return yaml2offload(*Doc.Offload, Out, ErrHandler);
     if (Doc.Wasm)
       return yaml2wasm(*Doc.Wasm, Out, ErrHandler);
     if (Doc.Xcoff)
       return yaml2xcoff(*Doc.Xcoff, Out, ErrHandler);
     if (Doc.DXContainer)
       return yaml2dxcontainer(*Doc.DXContainer, Out, ErrHandler);
 
     ErrHandler("unknown document type");

llvm/test/ObjectYAML/Offload/binary.yaml

+# RUN: yaml2obj %s | obj2yaml | FileCheck %s
+!Offload
+Members:
+  - ImageKind:        IMG_Cubin
+    OffloadKind:      OFK_OpenMP
+    Flags:            0
+    String:
+    - Key:              "triple"
+      Value:            "nvptx64-nvidia-cuda"
+    - Key:              "arch"
+      Value:            "sm_70"
+    Content:          "deadbeef"  
+
+     # CHECK: --- !Offload
+# CHECK-NEXT: Members:
+# CHECK-NEXT:   - ImageKind:       IMG_Cubin
+# CHECK-NEXT:     OffloadKind:     OFK_OpenMP
+# CHECK-NEXT:     Flags:           0
+# CHECK-NEXT:     String:
+# CHECK-NEXT:       - Key:             triple
+# CHECK-NEXT:         Value:           nvptx64-nvidia-cuda
+# CHECK-NEXT:       - Key:             arch
+# CHECK-NEXT:         Value:           sm_70
+# CHECK-NEXT:     Content:         DEADBEEF
+# CHECK-NEXT: ...

llvm/test/ObjectYAML/Offload/default.yaml

+# RUN: yaml2obj %s | obj2yaml | FileCheck %s
+!Offload
+Members:
+  - 
+
+# CHECK: --- !Offload
+# CHECK-NEXT: Members:
+# CHECK-NEXT:   - ImageKind:       IMG_None
+# CHECK-NEXT:     OffloadKind:     OFK_None
+# CHECK-NEXT:     Flags:           0
+# CHECK-NEXT: ...

llvm/test/ObjectYAML/Offload/enumeration_kinds.yaml

+# RUN: yaml2obj %s | obj2yaml | FileCheck %s
+!Offload
+Members:
+  - ImageKind:        IMG_None
+    OffloadKind:      OFK_None
+  - ImageKind:        IMG_Object
+    OffloadKind:      OFK_OpenMP
+  - ImageKind:        IMG_Bitcode
+    OffloadKind:      OFK_Cuda
+  - ImageKind:        IMG_Cubin
+    OffloadKind:      OFK_HIP
+  - ImageKind:        IMG_Fatbinary
+  - ImageKind:        IMG_PTX
+  - ImageKind:        999
+    OffloadKind:      999
+
+#      CHECK:       --- !Offload
+# CHECK-NEXT: Members:
+# CHECK-NEXT:   - ImageKind:       IMG_None
+# CHECK-NEXT:     OffloadKind:     OFK_None
+# CHECK-NEXT:     Flags:           0
+# CHECK-NEXT:   - ImageKind:       IMG_Object
+# CHECK-NEXT:     OffloadKind:     OFK_OpenMP
+# CHECK-NEXT:     Flags:           0
+# CHECK-NEXT:   - ImageKind:       IMG_Bitcode
+# CHECK-NEXT:     OffloadKind:     OFK_Cuda
+# CHECK-NEXT:     Flags:           0
+# CHECK-NEXT:   - ImageKind:       IMG_Cubin
+# CHECK-NEXT:     OffloadKind:     OFK_HIP
+# CHECK-NEXT:     Flags:           0
+# CHECK-NEXT:   - ImageKind:       IMG_Fatbinary
+# CHECK-NEXT:     OffloadKind:     OFK_None
+# CHECK-NEXT:     Flags:           0
+# CHECK-NEXT:   - ImageKind:       IMG_PTX
+# CHECK-NEXT:     OffloadKind:     OFK_None
+# CHECK-NEXT:     Flags:           0
+# CHECK-NEXT:   - ImageKind:       0x3E7
+# CHECK-NEXT:     OffloadKind:     0x3E7
+# CHECK-NEXT:     Flags:           0
+# CHECK-NEXT: ...

llvm/test/ObjectYAML/Offload/malformed-entry-size.yaml

+# RUN: yaml2obj %s | not obj2yaml 2>&1 | FileCheck %s
+!Offload
+EntrySize: 999999999
+Members:
+  - ImageKind:        IMG_Cubin
+    OffloadKind:      OFK_OpenMP
+    Flags:            0
+    String:
+    - Key:              "triple"
+      Value:            "nvptx64-nvidia-cuda"
+    - Key:              "arch"
+      Value:            "sm_70"
+    Content:          "deadbeef"  
+
+# CHECK: Error reading file: <stdin>: The end of the file was unexpectedly encountered

llvm/test/ObjectYAML/Offload/malformed-offset.yaml

+# RUN: yaml2obj %s | not obj2yaml 2>&1 | FileCheck %s
+!Offload
+EntryOffset: 999999999
+Members:
+  - ImageKind:        IMG_Cubin
+    OffloadKind:      OFK_OpenMP
+    Flags:            0
+    String:
+    - Key:              "triple"
+      Value:            "nvptx64-nvidia-cuda"
+    - Key:              "arch"
+      Value:            "sm_70"
+    Content:          "deadbeef"  
+
+# CHECK: Error reading file: <stdin>: The end of the file was unexpectedly encountered

llvm/test/ObjectYAML/Offload/malformed-size.yaml

+# RUN: yaml2obj %s | not obj2yaml 2>&1 | FileCheck %s
+!Offload
+Size: 999999999
+Members:
+  - ImageKind:        IMG_Cubin
+    OffloadKind:      OFK_OpenMP
+    Flags:            0
+    String:
+    - Key:              "triple"
+      Value:            "nvptx64-nvidia-cuda"
+    - Key:              "arch"
+      Value:            "sm_70"
+    Content:          "deadbeef"  
+
+# CHECK: Error reading file: <stdin>: The end of the file was unexpectedly encountered

llvm/test/ObjectYAML/Offload/malformed-version.yaml

+# RUN: yaml2obj %s | not obj2yaml 2>&1 | FileCheck %s
+!Offload
+Version: 2
+Members:
+  - ImageKind:        IMG_Cubin
+    OffloadKind:      OFK_OpenMP
+    Flags:            0
+    String:
+    - Key:              "triple"
+      Value:            "nvptx64-nvidia-cuda"
+    - Key:              "arch"
+      Value:            "sm_70"
+    Content:          "deadbeef"  
+
+# CHECK: Error reading file: <stdin>: Invalid data was encountered while parsing the file

llvm/test/ObjectYAML/Offload/multiple_members.yaml

+# RUN: yaml2obj %s | obj2yaml | FileCheck %s
+!Offload
+Members:
+  - ImageKind:        IMG_Cubin
+    OffloadKind:      OFK_OpenMP
+    Flags:            0
+    String:
+    - Key:              "triple"
+      Value:            "nvptx64-nvidia-cuda"
+    - Key:              "arch"
+      Value:            "sm_70"
+    Content:          "deadbeef"  
+  - ImageKind:        IMG_Bitcode
+    OffloadKind:      OFK_OpenMP
+    Flags:            0
+    String:
+    - Key:              "triple"
+      Value:            "amdgcn-amd-amdhsa"
+    - Key:              "arch"
+      Value:            "gfx908"
+    Content:          "cafefeed"  
+
+# CHECK: --- !Offload
+# CHECK-NEXT: Members:
+# CHECK-NEXT:   - ImageKind:       IMG_Cubin
+# CHECK-NEXT:     OffloadKind:     OFK_OpenMP
+# CHECK-NEXT:     Flags:           0
+# CHECK-NEXT:     String:
+# CHECK-NEXT:       - Key:             triple
+# CHECK-NEXT:         Value:           nvptx64-nvidia-cuda
+# CHECK-NEXT:       - Key:             arch
+# CHECK-NEXT:         Value:           sm_70
+# CHECK-NEXT:     Content:         DEADBEEF
+# CHECK-NEXT:   - ImageKind:       IMG_Bitcode
+# CHECK-NEXT:     OffloadKind:     OFK_OpenMP
+# CHECK-NEXT:     Flags:           0
+# CHECK-NEXT:     String:
+# CHECK-NEXT:       - Key:             triple
+# CHECK-NEXT:         Value:           amdgcn-amd-amdhsa
+# CHECK-NEXT:       - Key:             arch
+# CHECK-NEXT:         Value:           gfx908
+# CHECK-NEXT:     Content:         CAFEFEED
+# CHECK-NEXT: ...

llvm/tools/obj2yaml/CMakeLists.txt

add_llvm_utility(obj2yaml
   archive2yaml.cpp
   obj2yaml.cpp
   coff2yaml.cpp
   dwarf2yaml.cpp
   dxcontainer2yaml.cpp
   elf2yaml.cpp
   macho2yaml.cpp
   minidump2yaml.cpp
+  offload2yaml.cpp
   xcoff2yaml.cpp
   wasm2yaml.cpp
   )

llvm/tools/obj2yaml/obj2yaml.h

 
 #ifndef LLVM_TOOLS_OBJ2YAML_OBJ2YAML_H
 #define LLVM_TOOLS_OBJ2YAML_OBJ2YAML_H
 
 #include "llvm/Object/COFF.h"
 #include "llvm/Object/Minidump.h"
 #include "llvm/Object/Wasm.h"
 #include "llvm/Object/XCOFFObjectFile.h"
-#include "llvm/Support/raw_ostream.h"
 #include "llvm/Support/MemoryBufferRef.h"
+#include "llvm/Support/raw_ostream.h"
 #include <system_error>
 
 enum RawSegments : unsigned { none = 0, data = 1, linkedit = 1 << 1 };
 std::error_code coff2yaml(llvm::raw_ostream &Out,
                           const llvm::object::COFFObjectFile &Obj);
 llvm::Error elf2yaml(llvm::raw_ostream &Out,
-                         const llvm::object::ObjectFile &Obj);
 llvm::Error macho2yaml(llvm::raw_ostream &Out, const llvm::object::Binary &Obj,
                        unsigned RawSegments);
 llvm::Error minidump2yaml(llvm::raw_ostream &Out,
                           const llvm::object::MinidumpFile &Obj);
 llvm::Error xcoff2yaml(llvm::raw_ostream &Out,
                        const llvm::object::XCOFFObjectFile &Obj);
 std::error_code wasm2yaml(llvm::raw_ostream &Out,
                           const llvm::object::WasmObjectFile &Obj);
 llvm::Error archive2yaml(llvm::raw_ostream &Out, llvm::MemoryBufferRef Source);
+llvm::Error offload2yaml(llvm::raw_ostream &Out, llvm::MemoryBufferRef Source);
 llvm::Error dxcontainer2yaml(llvm::raw_ostream &Out,
                              llvm::MemoryBufferRef Source);
 
 // Forward decls for dwarf2yaml
 namespace llvm {
 class DWARFContext;
 namespace DWARFYAML {
 struct Data;
 }
-}
+} // namespace llvm
 
 void dumpDebugAbbrev(llvm::DWARFContext &DCtx, llvm::DWARFYAML::Data &Y);
 llvm::Error dumpDebugAddr(llvm::DWARFContext &DCtx, llvm::DWARFYAML::Data &Y);
 llvm::Error dumpDebugARanges(llvm::DWARFContext &DCtx,
                              llvm::DWARFYAML::Data &Y);
 void dumpDebugPubSections(llvm::DWARFContext &DCtx, llvm::DWARFYAML::Data &Y);
 void dumpDebugInfo(llvm::DWARFContext &DCtx, llvm::DWARFYAML::Data &Y);
 void dumpDebugLines(llvm::DWARFContext &DCtx, llvm::DWARFYAML::Data &Y);
 llvm::Error dumpDebugRanges(llvm::DWARFContext &DCtx, llvm::DWARFYAML::Data &Y);
 llvm::Error dumpDebugStrings(llvm::DWARFContext &DCtx,
                              llvm::DWARFYAML::Data &Y);
 
 #endif

llvm/tools/obj2yaml/obj2yaml.cpp

if (std::error_code EC = FileOrErr.getError())
     return errorCodeToError(EC);
   std::unique_ptr<MemoryBuffer> &Buffer = FileOrErr.get();
   MemoryBufferRef MemBuf = Buffer->getMemBufferRef();
   switch (identify_magic(MemBuf.getBuffer())) {
   case file_magic::archive:
     return archive2yaml(outs(), MemBuf);
   case file_magic::dxcontainer_object:
     return dxcontainer2yaml(outs(), MemBuf);
+  case file_magic::offload_binary:
+    return offload2yaml(outs(), MemBuf);
   default:
     break;
   }
 
   Expected<std::unique_ptr<Binary>> BinOrErr =
       createBinary(MemBuf, /*Context=*/nullptr);
   if (!BinOrErr)
     return BinOrErr.takeError();

llvm/tools/obj2yaml/offload2yaml.cpp

+//===------ offload2yaml.cpp - obj2yaml conversion tool ---*- C++ -------*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "obj2yaml.h"
+#include "llvm/BinaryFormat/Magic.h"
+#include "llvm/Object/OffloadBinary.h"
+#include "llvm/ObjectYAML/OffloadYAML.h"
+#include "llvm/Support/StringSaver.h"
+
+using namespace llvm;
+
+namespace {
+
+void populateYAML(OffloadYAML::Binary &YAMLBinary, object::OffloadBinary &OB,
+                  UniqueStringSaver Saver) {
+  YAMLBinary.Members.emplace_back();
+  auto &Member = YAMLBinary.Members.back();
+  Member.ImageKind = OB.getImageKind();
+  Member.OffloadKind = OB.getOffloadKind();
+  Member.Flags = OB.getFlags();
+  if (!OB.strings().empty()) {
+    Member.StringEntries = std::vector<OffloadYAML::Binary::StringEntry>();
+    for (const auto &Entry : OB.strings())
+      Member.StringEntries->emplace_back(OffloadYAML::Binary::StringEntry(
+          {Saver.save(Entry.getKey()), Saver.save(Entry.getValue())}));
+  }
+
+  if (!OB.getImage().empty())
+    Member.Content = arrayRefFromStringRef(OB.getImage());
+}
+
+Expected<OffloadYAML::Binary *> dump(MemoryBufferRef Source,
+                                     UniqueStringSaver Saver) {
+  Expected<std::unique_ptr<object::OffloadBinary>> OB =
+      object::OffloadBinary::create(Source);
+  if (!OB)
+    return OB.takeError();
+
+  std::unique_ptr<OffloadYAML::Binary> YAMLBinary =
+      std::make_unique<OffloadYAML::Binary>();
+
+  YAMLBinary->Members = std::vector<OffloadYAML::Binary::Member>();
+
+  uint64_t Offset = 0;
+  while (Offset < (*OB)->getMemoryBufferRef().getBufferSize()) {
+    MemoryBufferRef Buffer = MemoryBufferRef(
+        (*OB)->getData().drop_front(Offset), (*OB)->getFileName());
+    auto BinaryOrErr = object::OffloadBinary::create(Buffer);
+    if (!BinaryOrErr)
+      return BinaryOrErr.takeError();
+
+    object::OffloadBinary &Binary = **BinaryOrErr;
+
+    populateYAML(*YAMLBinary, Binary, Saver);
+
+    Offset += Binary.getSize();
+  }
+
+  return YAMLBinary.release();
+}
+
+} // namespace
+
+Error offload2yaml(raw_ostream &Out, MemoryBufferRef Source) {
+  BumpPtrAllocator Alloc;
+  UniqueStringSaver Saver(Alloc);
+
+  Expected<OffloadYAML::Binary *> YAMLOrErr = dump(Source, Saver);
+  if (!YAMLOrErr)
+    return YAMLOrErr.takeError();
+
+  std::unique_ptr<OffloadYAML::Binary> YAML(YAMLOrErr.get());
+  yaml::Output Yout(Out);
+  Yout << *YAML;
+
+  return Error::success();
+}