Universal MachO: support LLVM IR objects

[aguinet]

Summary

This adds support for LLVM IR "objects" within universal MachO, as this can happen when using LTO when compiling iOS/tvOS applications (or any universal MachO object compiled with clang).

This also improves llvm-lipo in order to support these LLVM IR objects. llvm-lipo is now able to extract/thin, create and dump information for universal MachO with LLVM IR bitcode objects.

One strategy to test this properly would require implementing LLVM IR description within the ObjectYAML library. Before doing so, I would like to here about other potential strategies!

[alexander-shaposhnikov]

@aguinet,
thanks for the patch!

1. This code has been recently refactored (see https://github.com/llvm/llvm-project/commit/df69492cdfa82ff6453c887cd45b3a5903b79afb ) so you'd need to rebase your diff.
2. Bitcode files can also come from archives (static libraries) (e.g. an archive can contain bitcode files exclusively) - so when we create a Slice from an archive we also need to handle that.
3. We need tests.

Somewhat related to this area - If I'm not mistaken there is a small issue in SymbolicFile::createSymbolicFile - when LLVMContext is null the function appears to return a misleading error - I was planning to fix that soon.

[aguinet]

Hello @alexshap!

Thanks for the comments. Answers below:

In D85740#2210765, @alexshap wrote:

1. This code has been recently refactored (see https://github.com/llvm/llvm-project/commit/df69492cdfa82ff6453c887cd45b3a5903b79afb ) so you'd need to rebase your diff.

I pushed a new patch rebased on master, sorry about this conflict (did this patch before going to holidays, should have rebased before submitting when coming back :)).

1. Bitcode files can also come from archives (static libraries) (e.g. an archive can contain bitcode files exclusively) - so when we create a Slice from an archive we also need to handle that.

That's a good catch. I will take a closer look and adapt the patch accordingly.

1. We need tests.

About that part, like I said in the description, the way I see it is to enhance the ObjectYAML library so that we can generate LLVM bitcode file format (as described in https://llvm.org/docs/BitCodeFormat.html), so that yaml2obj can generate appropriate LLVM IR test objects. That would be used for instance to test the -create flag. Do you see another way to do this?

Somewhat related to this area - If I'm not mistaken there is a small issue in SymbolicFile::createSymbolicFile - when LLVMContext is null the function appears to return a misleading error - I was planning to fix that soon.

Indeed. If LLVMCOntext is null, it says that it is in an invalid format. Not ideal what should be the error, as this is an error which is forwarded to the end user (and I'm not sure the user would understand "you need to pass an LLVMContext object to SymbolicFile::createSymbolicFile to parse IR objects" :))

[smeenai]

About that part, like I said in the description, the way I see it is to enhance the ObjectYAML library so that we can generate LLVM bitcode file format (as described in https://llvm.org/docs/BitCodeFormat.html), so that yaml2obj can generate appropriate LLVM IR test objects. That would be used for instance to test the -create flag. Do you see another way to do this?

Can't you just use llvm-as to produce bitcode files and feed those to llvm-lipo?

[aguinet]

In D85740#2214477, @smeenai wrote:

About that part, like I said in the description, the way I see it is to enhance the ObjectYAML library so that we can generate LLVM bitcode file format (as described in https://llvm.org/docs/BitCodeFormat.html), so that yaml2obj can generate appropriate LLVM IR test objects. That would be used for instance to test the -create flag. Do you see another way to do this?

Can't you just use llvm-as to produce bitcode files and feed those to llvm-lipo?

Indeed, I can, that's easier... I will try and do that properly next week. Thanks for the hint!

[alexander-shaposhnikov]

llvm/tools/llvm-lipo/llvm-lipo.cpp

33

nit: these includes need to be moved up (if i remember correctly we usually try to sort includes alphabetically)

[aguinet]

Hello @alexshap,

Here is a new patch that includes:

• support for archives that includes LLVM IR objects
• testing using llvm-ar
• fix the order of includes in llvm-lipo (as per your comment)

It also includes a few refactoring of code.

I don't know how much we need to be consistent on the original lipo regarding the error message, but I tried to follow the existing convention.

[aguinet]

Fix fatal error message of MachOUniversalBinary::ObjectForArch::getAsIRObject

[aguinet]

Clang-tidy + format remarks

[aguinet]

Cleanup/refactoring

[alexander-shaposhnikov]

llvm/lib/Object/MachOUniversalWriter.cpp

150

not particularly important, but I'd probably convert "if" + "continue" into

if ( ... ) { ... } else if ( ... ) { ... } else { ... }

(it seems to be more readable)

203–213

perhaps, given the check above this assert is not super useful
if you decide to keep it then we need to add a string message

assert (.... && "....")

212

nit:

1. auto -> explicit type (since it's not clear from the right-hand side),
2. for a single-line "if" braces are not necessary.

(and the same applies to lines 216 - 219)

llvm/tools/llvm-lipo/llvm-lipo.cpp

128

what would you say to the following renaming:

irSlice -> createSliceFromIR
archiveSlice -> createSliceFromArchive

128

I'm wondering if we really need to pass the name of the file here - can't it be extracted from the first argument (IRO->getFileName()) ?

(if yes - then probably it can be removed from archiveSlice as well)

P.S. https://llvm.org/doxygen/MachOUniversal_8cpp_source.html#l00078

399

nit: explicit type

566

nit: explicit type

580

nit: explicit type

[alexander-shaposhnikov]

llvm/lib/Object/MachOUniversalWriter.cpp

134

nit: explicit type

[aguinet]

Thanks @alexshap for your review and comments! Answers are inline.

I will try and push a new diff with the suggested modifications during the day.

llvm/lib/Object/MachOUniversalWriter.cpp

150

Okay will fix with new diff!

203–213

Indeed. I put it during development to make sure I didn't make any mistake, but I will remove it.

212

Ack will fix with new diff

llvm/tools/llvm-lipo/llvm-lipo.cpp

128

Probably. I will test and let you know :)

No problem for me for the renaming you propose, will put it in the new diff.

[aguinet]

@alexshap there we go, it should fix all your remarks. Please let my know if there are still some issues.

[aguinet]

Just a small variable naming cleanup

[alexander-shaposhnikov]

LGTM

[smeenai]

I think llvm-libtool-darwin would also need some adjustments to handle bitcode objects, archives, and universal binaries. That's definitely out of scope for this diff, but I'm just flagging it as another tool with missing bitcode support.

llvm/include/llvm/Object/MachOUniversalWriter.h

19

Nit: how come you're including the header here instead of forward declaring IRObjectFile?

[alexander-shaposhnikov]

llvm/lib/Object/MachOUniversalWriter.cpp

92

nit: a blank line between 90 and 91 would not hurt

[aguinet]

Fresh air!

[aguinet]

Thanks for the new reviews! Fixed in the incoming diff

llvm/include/llvm/Object/MachOUniversalWriter.h

19

Nice catch! Fixed in the new diff

[aguinet]

@alexshap note that I don't have RW access to the llvm repository, so I can't merge this myself

[aguinet]

This patch fixes one bug that was present: running llvm-lipo -archs on an LLVM IR object would crash, because we didn't check for that case in printBinaryArchs. This new patch fixes it, with an associated test.

[aguinet]

Thanks for the merge and reviews @alexshap (and @smeenai) !

llvm/include/llvm/Object/Binary.h

 template <typename T> T* OwningBinary<T>::getBinary() {
   return Bin.get();
 }
 
 template <typename T> const T* OwningBinary<T>::getBinary() const {
   return Bin.get();
 }
 
-Expected<OwningBinary<Binary>> createBinary(StringRef Path);
+Expected<OwningBinary<Binary>> createBinary(StringRef Path,
+                                            LLVMContext *Context = nullptr);
 
 } // end namespace object
 
 } // end namespace llvm
 
 #endif // LLVM_OBJECT_BINARY_H

llvm/include/llvm/Object/MachOUniversal.h

 #include "llvm/ADT/iterator_range.h"
 #include "llvm/BinaryFormat/MachO.h"
 #include "llvm/Object/Archive.h"
 #include "llvm/Object/Binary.h"
 #include "llvm/Object/MachO.h"
 
 namespace llvm {
 class StringRef;
+class Module;
+class LLVMContext;
 
 namespace object {
+class IRObjectFile;
 
 class MachOUniversalBinary : public Binary {
   virtual void anchor();
 
   uint32_t Magic;
   uint32_t NumberOfObjects;
 public:
   static constexpr uint32_t MaxSectionAlignment = 15; /* 2**15 or 0x8000 */
public:
     std::string getArchFlagName() const {
       const char *McpuDefault, *ArchFlag;
       MachOObjectFile::getArchTriple(getCPUType(), getCPUSubType(),
                                      &McpuDefault, &ArchFlag);
       return ArchFlag ? ArchFlag : std::string();
     }
 
     Expected<std::unique_ptr<MachOObjectFile>> getAsObjectFile() const;
+    Expected<std::unique_ptr<IRObjectFile>>
+    getAsIRObject(LLVMContext &Ctx) const;
 
     Expected<std::unique_ptr<Archive>> getAsArchive() const;
   };
 
   class object_iterator {
     ObjectForArch Obj;
   public:
     object_iterator(const ObjectForArch &Obj) : Obj(Obj) {}
public:
   }
 
   Expected<ObjectForArch>
   getObjectForArch(StringRef ArchName) const;
 
   Expected<std::unique_ptr<MachOObjectFile>>
   getMachOObjectForArch(StringRef ArchName) const;
 
+  Expected<std::unique_ptr<IRObjectFile>>
+  getIRObjectForArch(StringRef ArchName, LLVMContext &Ctx) const;
+
   Expected<std::unique_ptr<Archive>>
   getArchiveForArch(StringRef ArchName) const;
 };
 
 }
 }
 
 #endif

llvm/include/llvm/Object/MachOUniversalWriter.h

 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_OBJECT_MACHOUNIVERSALWRITER_H
 #define LLVM_OBJECT_MACHOUNIVERSALWRITER_H
 
 #include "llvm/Object/Archive.h"
 #include "llvm/Object/Binary.h"
 #include "llvm/Object/MachO.h"
 
 namespace llvm {
+class LLVMContext;
+
 namespace object {
+class IRObjectFile;
 
 class Slice {
   const Binary *B;
   uint32_t CPUType;
   uint32_t CPUSubType;
   std::string ArchName;
 
   // P2Alignment field stores slice alignment values from universal
   // binaries. This is also needed to order the slices so the total
   // file size can be calculated before creating the output buffer.
   uint32_t P2Alignment;
 
+  Slice(const IRObjectFile *IRO, uint32_t CPUType, uint32_t CPUSubType,
+        std::string ArchName, uint32_t Align);
+
 public:
   explicit Slice(const MachOObjectFile &O);
 
   Slice(const MachOObjectFile &O, uint32_t Align);
 
-  static Expected<Slice> create(const Archive *A);
+  static Expected<Slice> create(const Archive *A,
+                                LLVMContext *LLVMCtx = nullptr);
+
+  static Expected<Slice> create(const IRObjectFile *IRO, uint32_t Align);
 
   void setP2Alignment(uint32_t Align) { P2Alignment = Align; }
 
   const Binary *getBinary() const { return B; }
 
   uint32_t getCPUType() const { return CPUType; }
 
   uint32_t getCPUSubType() const { return CPUSubType; }

llvm/lib/Object/Binary.cpp

Expected<std::unique_ptr<Binary>> object::createBinary(MemoryBufferRef Buffer,
   case file_magic::minidump:
     return MinidumpFile::create(Buffer);
   case file_magic::tapi_file:
     return TapiUniversal::create(Buffer);
   }
   llvm_unreachable("Unexpected Binary File Type");
 }
 
-Expected<OwningBinary<Binary>> object::createBinary(StringRef Path) {
+Expected<OwningBinary<Binary>> object::createBinary(StringRef Path,
+                                                    LLVMContext *Context) {
   ErrorOr<std::unique_ptr<MemoryBuffer>> FileOrErr =
       MemoryBuffer::getFileOrSTDIN(Path, /*FileSize=*/-1,
                                    /*RequiresNullTerminator=*/false);
   if (std::error_code EC = FileOrErr.getError())
     return errorCodeToError(EC);
   std::unique_ptr<MemoryBuffer> &Buffer = FileOrErr.get();
 
   Expected<std::unique_ptr<Binary>> BinOrErr =
-      createBinary(Buffer->getMemBufferRef());
+      createBinary(Buffer->getMemBufferRef(), Context);
   if (!BinOrErr)
     return BinOrErr.takeError();
   std::unique_ptr<Binary> &Bin = BinOrErr.get();
 
   return OwningBinary<Binary>(std::move(Bin), std::move(Buffer));
 }

llvm/lib/Object/MachOUniversal.cpp

 //===- MachOUniversal.cpp - Mach-O universal binary -------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file defines the MachOUniversalBinary class.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Object/MachOUniversal.h"
 #include "llvm/Object/Archive.h"
+#include "llvm/Object/IRObjectFile.h"
 #include "llvm/Object/MachO.h"
 #include "llvm/Object/ObjectFile.h"
 #include "llvm/Support/Casting.h"
 #include "llvm/Support/Host.h"
 #include "llvm/Support/MemoryBuffer.h"
 
 using namespace llvm;
 using namespace object;
if (Parent->getMagic() == MachO::FAT_MAGIC) {
     ObjectData = ParentData.substr(Header64.offset, Header64.size);
     cputype = Header64.cputype;
   }
   StringRef ObjectName = Parent->getFileName();
   MemoryBufferRef ObjBuffer(ObjectData, ObjectName);
   return ObjectFile::createMachOObjectFile(ObjBuffer, cputype, Index);
 }
 
+Expected<std::unique_ptr<IRObjectFile>>
+MachOUniversalBinary::ObjectForArch::getAsIRObject(LLVMContext &Ctx) const {
+  if (!Parent)
+    report_fatal_error("MachOUniversalBinary::ObjectForArch::getAsIRObject() "
+                       "called when Parent is a nullptr");
+
+  StringRef ParentData = Parent->getData();
+  StringRef ObjectData;
+  if (Parent->getMagic() == MachO::FAT_MAGIC) {
+    ObjectData = ParentData.substr(Header.offset, Header.size);
+  } else { // Parent->getMagic() == MachO::FAT_MAGIC_64
+    ObjectData = ParentData.substr(Header64.offset, Header64.size);
+  }
+  StringRef ObjectName = Parent->getFileName();
+  MemoryBufferRef ObjBuffer(ObjectData, ObjectName);
+
+  return IRObjectFile::create(ObjBuffer, Ctx);
+}
+
 Expected<std::unique_ptr<Archive>>
 MachOUniversalBinary::ObjectForArch::getAsArchive() const {
   if (!Parent)
     report_fatal_error("MachOUniversalBinary::ObjectForArch::getAsArchive() "
                        "called when Parent is a nullptr");
 
   StringRef ParentData = Parent->getData();
   StringRef ObjectData;

 Expected<std::unique_ptr<MachOObjectFile>>
 MachOUniversalBinary::getMachOObjectForArch(StringRef ArchName) const {
   Expected<ObjectForArch> O = getObjectForArch(ArchName);
   if (!O)
     return O.takeError();
   return O->getAsObjectFile();
 }
 
+Expected<std::unique_ptr<IRObjectFile>>
+MachOUniversalBinary::getIRObjectForArch(StringRef ArchName,
+                                         LLVMContext &Ctx) const {
+  Expected<ObjectForArch> O = getObjectForArch(ArchName);
+  if (!O)
+    return O.takeError();
+  return O->getAsIRObject(Ctx);
+}
+
 Expected<std::unique_ptr<Archive>>
 MachOUniversalBinary::getArchiveForArch(StringRef ArchName) const {
   Expected<ObjectForArch> O = getObjectForArch(ArchName);
   if (!O)
     return O.takeError();
   return O->getAsArchive();
 }

llvm/lib/Object/MachOUniversalWriter.cpp

 //===- MachOUniversalWriter.cpp - MachO universal binary writer---*- C++-*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // Defines the Slice class and writeUniversalBinary function for writing a MachO
 // universal binary file.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Object/MachOUniversalWriter.h"
+#include "llvm/ADT/Triple.h"
 #include "llvm/Object/Archive.h"
 #include "llvm/Object/Binary.h"
 #include "llvm/Object/Error.h"
+#include "llvm/Object/IRObjectFile.h"
 #include "llvm/Object/MachO.h"
 #include "llvm/Object/MachOUniversal.h"
 #include "llvm/Support/FileOutputBuffer.h"
 
 using namespace llvm;
 using namespace object;
 
 // For compatibility with cctools lipo, a file's alignment is calculated as the

 }
 
 Slice::Slice(const MachOObjectFile &O, uint32_t Align)
     : B(&O), CPUType(O.getHeader().cputype),
       CPUSubType(O.getHeader().cpusubtype),
       ArchName(std::string(O.getArchTriple().getArchName())),
       P2Alignment(Align) {}
 
+Slice::Slice(const IRObjectFile *IRO, uint32_t CPUType, uint32_t CPUSubType,
+             std::string ArchName, uint32_t Align)
+    : B(IRO), CPUType(CPUType), CPUSubType(CPUSubType),
+      ArchName(std::move(ArchName)), P2Alignment(Align) {}
+
 Slice::Slice(const MachOObjectFile &O) : Slice(O, calculateAlignment(O)) {}
 
-Expected<Slice> Slice::create(const Archive *A) {
+using MachoCPUTy = std::pair<unsigned, unsigned>;
+
+static Expected<MachoCPUTy> getMachoCPUFromTriple(Triple TT) {
+  auto CPU = std::make_pair(MachO::getCPUType(TT), MachO::getCPUSubType(TT));
+  if (!CPU.first) {
+    return CPU.first.takeError();
+  }
+  if (!CPU.second) {
+    return CPU.second.takeError();
+  }
+  return std::make_pair(*CPU.first, *CPU.second);
+}
+
+static Expected<MachoCPUTy> getMachoCPUFromTriple(StringRef TT) {
+  return getMachoCPUFromTriple(Triple{TT});
+}
+
+Expected<Slice> Slice::create(const Archive *A, LLVMContext *LLVMCtx) {
   Error Err = Error::success();
-  std::unique_ptr<MachOObjectFile> FO = nullptr;
+  std::unique_ptr<MachOObjectFile> MFO = nullptr;
+  std::unique_ptr<IRObjectFile> IRFO = nullptr;
   for (const Archive::Child &Child : A->children(Err)) {
-    Expected<std::unique_ptr<Binary>> ChildOrErr = Child.getAsBinary();
+    Expected<std::unique_ptr<Binary>> ChildOrErr = Child.getAsBinary(LLVMCtx);
     if (!ChildOrErr)
       return createFileError(A->getFileName(), ChildOrErr.takeError());
     Binary *Bin = ChildOrErr.get().get();
     if (Bin->isMachOUniversalBinary())
       return createStringError(std::errc::invalid_argument,
                                ("archive member " + Bin->getFileName() +
                                 " is a fat file (not allowed in an archive)")
                                    .str()
                                    .c_str());
-    if (!Bin->isMachO())
+    if (Bin->isMachO()) {
+      MachOObjectFile *O = cast<MachOObjectFile>(Bin);
+      if (IRFO) {
         return createStringError(
             std::errc::invalid_argument,
-          ("archive member " + Bin->getFileName() +
-           " is not a MachO file (not allowed in an archive)")
-              .str()
-              .c_str());
-    MachOObjectFile *O = cast<MachOObjectFile>(Bin);
-    if (FO && std::tie(FO->getHeader().cputype, FO->getHeader().cpusubtype) !=
-                  std::tie(O->getHeader().cputype, O->getHeader().cpusubtype)) {
+            "archive member %s is a MachO, while previous archive member "
+            "%s was an IR LLVM object",
+            O->getFileName().str().c_str(), IRFO->getFileName().str().c_str());
+      }
+      if (MFO &&
+          std::tie(MFO->getHeader().cputype, MFO->getHeader().cpusubtype) !=
         return createStringError(
             std::errc::invalid_argument,
             ("archive member " + O->getFileName() + " cputype (" +
              Twine(O->getHeader().cputype) + ") and cpusubtype(" +
              Twine(O->getHeader().cpusubtype) +
              ") does not match previous archive members cputype (" +
-           Twine(FO->getHeader().cputype) + ") and cpusubtype(" +
-           Twine(FO->getHeader().cpusubtype) + ") (all members must match) " +
-           FO->getFileName())
+             Twine(MFO->getHeader().cputype) + ") and cpusubtype(" +
+             Twine(MFO->getHeader().cpusubtype) +
+             ") (all members must match) " + MFO->getFileName())
+                .str()
+                .c_str());
+      }
+      if (!MFO) {
+        ChildOrErr.get().release();
+        MFO.reset(O);
+      }
+    } else if (Bin->isIR()) {
+      IRObjectFile *O = cast<IRObjectFile>(Bin);
+      if (MFO) {
+        return createStringError(std::errc::invalid_argument,
+                                 "archive member '%s' is an LLVM IR object, "
+                                 "while previous archive member "
+                                 "'%s' was a MachO",
+                                 O->getFileName().str().c_str(),
+                                 MFO->getFileName().str().c_str());
+      }
+      if (IRFO) {
+        Expected<MachoCPUTy> CPUO = getMachoCPUFromTriple(O->getTargetTriple());
+        Expected<MachoCPUTy> CPUFO =
+            getMachoCPUFromTriple(IRFO->getTargetTriple());
+        if (!CPUO)
+          return CPUO.takeError();
+        if (!CPUFO)
+          return CPUFO.takeError();
+        if (*CPUO != *CPUFO) {
+          return createStringError(
+              std::errc::invalid_argument,
+              ("archive member " + O->getFileName() + " cputype (" +
+               Twine(CPUO->first) + ") and cpusubtype(" + Twine(CPUO->second) +
+               ") does not match previous archive members cputype (" +
+               Twine(CPUFO->first) + ") and cpusubtype(" +
+               Twine(CPUFO->second) + ") (all members must match) " +
+               IRFO->getFileName())
                   .str()
                   .c_str());
         }
-    if (!FO) {
+      } else {
         ChildOrErr.get().release();
-      FO.reset(O);
+        IRFO.reset(O);
       }
+    } else
+      return createStringError(std::errc::invalid_argument,
+                               ("archive member " + Bin->getFileName() +
+                                " is neither a MachO file or an LLVM IR file "
+                                "(not allowed in an archive)")
+                                   .str()
+                                   .c_str());
   }
   if (Err)
     return createFileError(A->getFileName(), std::move(Err));
-  if (!FO)
+  if (!MFO && !IRFO)
     return createStringError(
         std::errc::invalid_argument,
         ("empty archive with no architecture specification: " +
          A->getFileName() + " (can't determine architecture for it)")
             .str()
             .c_str());
 
-  Slice ArchiveSlice = Slice(*(FO.get()), FO->is64Bit() ? 3 : 2);
+  if (MFO) {
+    Slice ArchiveSlice(*(MFO.get()), MFO->is64Bit() ? 3 : 2);
     ArchiveSlice.B = A;
     return ArchiveSlice;
   }
 
+  // For IR objects
+  Expected<Slice> ArchiveSliceOrErr = Slice::create(IRFO.get(), 0);
+  if (!ArchiveSliceOrErr)
+    return createFileError(A->getFileName(), ArchiveSliceOrErr.takeError());
+  auto &ArchiveSlice = ArchiveSliceOrErr.get();
+  ArchiveSlice.B = A;
+  return Slice{std::move(ArchiveSlice)};
+}
+
+Expected<Slice> Slice::create(const IRObjectFile *IRO, uint32_t Align) {
+  Expected<MachoCPUTy> CPUOrErr = getMachoCPUFromTriple(IRO->getTargetTriple());
+  if (!CPUOrErr)
+    return CPUOrErr.takeError();
+  unsigned CPUType, CPUSubType;
+  std::tie(CPUType, CPUSubType) = CPUOrErr.get();
+  // We don't directly use the architecture name of the target triple T, as,
+  // for instance, thumb is treated as ARM by the MachOUniversal object.
+  std::string ArchName(
+      MachOObjectFile::getArchTriple(CPUType, CPUSubType).getArchName());
+  return Slice{IRO, CPUType, CPUSubType, std::move(ArchName), Align};
+}
+
 static Expected<SmallVector<MachO::fat_arch, 2>>
 buildFatArchList(ArrayRef<Slice> Slices) {
   SmallVector<MachO::fat_arch, 2> FatArchList;
   uint64_t Offset =
       sizeof(MachO::fat_header) + Slices.size() * sizeof(MachO::fat_arch);
 
   for (const auto &S : Slices) {
     Offset = alignTo(Offset, 1ull << S.getP2Alignment());

llvm/test/tools/llvm-lipo/Inputs/arm64-ios.ll

+target triple = "arm64-apple-ios8.0.0"

llvm/test/tools/llvm-lipo/Inputs/armv7-ios.ll

+target triple = "thumbv7-apple-ios8.0.0"

llvm/test/tools/llvm-lipo/Inputs/x64-osx.ll

+target triple = "x86_64-apple-macosx10.15.0"

llvm/test/tools/llvm-lipo/create-arch-ir.test

+# RUN: llvm-as %p/Inputs/armv7-ios.ll -o %t-armv7.o
+# RUN: llvm-as %p/Inputs/arm64-ios.ll -o %t-arm64.o
+
+# RUN: llvm-lipo %t-armv7.o %t-arm64.o -create -output %t-universal.o
+# RUN: llvm-lipo -arch armv7 %t-armv7.o -arch arm64 %t-arm64.o -create -output %t-universal-1.o
+# RUN: cmp %t-universal.o %t-universal-1.o
+#
+# RUN: not llvm-lipo -arch armv7 %t-arm64.o -create -output /dev/null 2>&1 | FileCheck --check-prefix=ARCH_NOT_MATCHED %s
+# ARCH_NOT_MATCHED: error: specified architecture: armv7 for file: {{.*}} does not match the file's architecture (arm64)

llvm/test/tools/llvm-lipo/create-archive-input.test

 # RUN: yaml2obj %p/Inputs/i386-slice.yaml -o %t-i386.o
 # RUN: yaml2obj %p/Inputs/x86_64-slice.yaml -o %t-x86_64.o
 # RUN: yaml2obj %p/Inputs/i386-x86_64-universal.yaml -o %t-universal.o
+# RUN: llvm-as %p/Inputs/armv7-ios.ll -o %t-ir-armv7.o
+# RUN: llvm-as %p/Inputs/x64-osx.ll -o %t-ir-x86_64.o
 
 # RUN: llvm-ar cr %t.empty.a
 # RUN: not llvm-lipo %t.empty.a -create -output /dev/null 2>&1 | FileCheck --check-prefix=EMPTY-ARCHIVE %s
 
 # RUN: llvm-ar cr %t.different_architectures.a %t-i386.o %t-x86_64.o
 # RUN: not llvm-lipo %t.different_architectures.a -create -output /dev/null 2>&1 | FileCheck --check-prefix=ARCHIVE-WITH-DIFFERENT-ARCHS %s
 
 # RUN: llvm-ar cr %t.contains_fat_binary.a %t-universal.o 
 # RUN: not llvm-lipo %t.contains_fat_binary.a -create -output /dev/null 2>&1 | FileCheck --check-prefix=ARCHIVE-WITH-FAT-BINARY %s
 
 # RUN: llvm-ar cr %t-i386-lib.a %t-i386.o
 # RUN: llvm-lipo %t-i386-lib.a %t-x86_64.o -create -output %t-i386-x86_64-universal.o
 # RUN: llvm-lipo %t-i386-x86_64-universal.o -info | FileCheck --check-prefix=INFO-i386-x86_64 %s
 # RUN: llvm-lipo %t-i386-x86_64-universal.o -thin i386 -output %t-extracted-i386-lib.a
 # RUN: cmp %t-extracted-i386-lib.a %t-i386-lib.a
 # RUN: llvm-lipo %t-i386-x86_64-universal.o -thin x86_64 -output %t-extracted-x86_64.o
 # RUN: cmp %t-extracted-x86_64.o %t-x86_64.o
-#
+
+# RUN: llvm-ar cr %t-ir-armv7-lib.a %t-ir-armv7.o
+# RUN: llvm-lipo %t-ir-armv7-lib.a %t-ir-x86_64.o -create -output %t-ir-armv7-x86_64-universal.o
+# RUN: llvm-lipo %t-ir-armv7-x86_64-universal.o -thin armv7 -output %t-ir-extracted-armv7-lib.a
+# RUN: cmp %t-ir-extracted-armv7-lib.a %t-ir-armv7-lib.a
+# RUN: llvm-lipo %t-ir-armv7-x86_64-universal.o -thin x86_64 -output %t-ir-extracted-x86_64.o
+# RUN: cmp %t-ir-extracted-x86_64.o %t-ir-x86_64.o
+
+# RUN: llvm-ar cr %t.different_types0.a %t-i386.o %t-ir-x86_64.o
+# RUN: not llvm-lipo -create %t.different_types0.a -output /dev/null 2>&1 | FileCheck --check-prefix=ARCHIVE-WITH-MACHO-AND-IR %s
+# RUN: llvm-ar cr %t.different_types1.a %t-ir-x86_64.o %t-i386.o 
+# RUN: not llvm-lipo -create %t.different_types1.a -output /dev/null 2>&1 | FileCheck --check-prefix=ARCHIVE-WITH-IR-AND-MACHO %s
+# RUN: llvm-ar cr %t.different_architectures_ir.a %t-ir-x86_64.o %t-ir-armv7.o
+# RUN: not llvm-lipo -create %t.different_architectures_ir.a -output /dev/null 2>&1 | FileCheck --check-prefix=ARCHIVE-WITH-DIFFERENT-ARCHS %s
+
 # EMPTY-ARCHIVE: empty archive
 # ARCHIVE-WITH-DIFFERENT-ARCHS: all members must match
+# ARCHIVE-WITH-MACHO-AND-IR: is an LLVM IR object, while previous archive member {{.*}} was a MachO
+# ARCHIVE-WITH-IR-AND-MACHO: is a MachO, while previous archive member {{.*}} was an IR LLVM object
 # ARCHIVE-WITH-FAT-BINARY: fat file (not allowed in an archive)
 #
 # INFO-i386-x86_64: i386 x86_64

llvm/test/tools/llvm-lipo/info-universal-binary-ir.test

+# RUN: llvm-as %p/Inputs/armv7-ios.ll -o %t-armv7.o
+# RUN: llvm-as %p/Inputs/arm64-ios.ll -o %t-arm64.o
+# RUN: llvm-as %p/Inputs/x64-osx.ll -o %t-x64.o
+
+# RUN: llvm-lipo %t-arm64.o %t-x64.o -create -output %t-universal0.o
+# RUN: llvm-lipo -info %t-universal0.o | FileCheck %s
+#
+# CHECK: Architectures in the fat file: {{.*}} are: x86_64 arm64
+
+# Test that, despites the "thumbv7" triple present in armv7-ios.ll,
+# architecture is still reported as armv7
+# RUN: llvm-lipo %t-armv7.o -create -output %t-universal1.o
+# RUN: llvm-lipo -info %t-universal1.o | FileCheck --check-prefix=ARMV7 %s
+#
+# ARMV7: armv7
+# ARMV7-NOT: thumbv7

llvm/test/tools/llvm-lipo/thin-universal-binary-ir.test

+# RUN: llvm-as %p/Inputs/armv7-ios.ll -o %t-armv7.o
+# RUN: llvm-as %p/Inputs/arm64-ios.ll -o %t-arm64.o
+
+# RUN: llvm-lipo %t-armv7.o %t-arm64.o -create -output %t-universal.o
+# RUN: llvm-lipo %t-universal.o -thin arm64 -output %t-arm64.out
+# RUN: opt -S %t-arm64.out | FileCheck --check-prefix=ARM64 %s
+# RUN: llvm-lipo %t-universal.o -thin armv7 -output %t-armv7.out
+# RUN: opt -S %t-armv7.out | FileCheck --check-prefix=ARMV7 %s
+
+# ARM64: arm64-apple-ios8.0.0
+# ARMV7: thumbv7-apple-ios8.0.0

llvm/tools/llvm-lipo/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   ${LLVM_TARGETS_TO_BUILD}
   Object
   Option
   Support
   TextAPI
+  Core
+  BinaryFormat
 )
 
 set(LLVM_TARGET_DEFINITIONS LipoOpts.td)
 tablegen(LLVM LipoOpts.inc -gen-opt-parser-defs)
 add_public_tablegen_target(LipoOptsTableGen)
 
 add_llvm_tool(llvm-lipo
   llvm-lipo.cpp
   DEPENDS
   LipoOptsTableGen
 )
 
 if(LLVM_INSTALL_CCTOOLS_SYMLINKS)
   add_llvm_tool_symlink(lipo llvm-lipo)
 endif()

llvm/tools/llvm-lipo/LLVMBuild.txt

 ;
 ;   http://llvm.org/docs/LLVMBuild.html
 ;
 ;===------------------------------------------------------------------------===;
 [component_0]
 type = Tool
 name = llvm-lipo
 parent = Tools
-required_libraries = Object Option Support
+required_libraries = Object Option Support BinaryFormat

llvm/tools/llvm-lipo/llvm-lipo.cpp

 //===-- llvm-lipo.cpp - a tool for manipulating universal binaries --------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // A utility for creating / splitting / inspecting universal binaries.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/Triple.h"
+#include "llvm/BinaryFormat/MachO.h"
+#include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/Module.h"
 #include "llvm/Object/Binary.h"
+#include "llvm/Object/IRObjectFile.h"
 #include "llvm/Object/MachO.h"
 #include "llvm/Object/MachOUniversal.h"
 #include "llvm/Object/MachOUniversalWriter.h"
 #include "llvm/Object/ObjectFile.h"
 #include "llvm/Option/Arg.h"
 #include "llvm/Option/ArgList.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/FileOutputBuffer.h"
 #include "llvm/Support/InitLLVM.h"
 #include "llvm/Support/WithColor.h"
 #include "llvm/TextAPI/MachO/Architecture.h"
 
 using namespace llvm;
 using namespace llvm::object;
 
 static const StringRef ToolName = "llvm-lipo";
+static LLVMContext LLVMCtx;
 
 LLVM_ATTRIBUTE_NORETURN static void reportError(Twine Message) {
   WithColor::error(errs(), ToolName) << Message << "\n";
   errs().flush();
   exit(EXIT_FAILURE);
 }
 
 LLVM_ATTRIBUTE_NORETURN static void reportError(Error E) {
struct Config {
   SmallVector<std::string, 1> VerifyArchList;
   SmallVector<InputFile, 1> ReplacementFiles;
   StringMap<const uint32_t> SegmentAlignments;
   std::string ArchType;
   std::string OutputFile;
   LipoAction ActionToPerform;
 };
 
-static Slice archiveSlice(const Archive *A, StringRef File) {
-  Expected<Slice> ArchiveOrSlice = Slice::create(A);
+static Slice createSliceFromArchive(const Archive *A) {
+  Expected<Slice> ArchiveOrSlice = Slice::create(A, &LLVMCtx);
   if (!ArchiveOrSlice)
-    reportError(File, ArchiveOrSlice.takeError());
+    reportError(A->getFileName(), ArchiveOrSlice.takeError());
   return *ArchiveOrSlice;
 }
 
+static Slice createSliceFromIR(const IRObjectFile *IRO, unsigned Align) {
+  Expected<Slice> IROrErr = Slice::create(IRO, Align);
+  if (!IROrErr)
+    reportError(IRO->getFileName(), IROrErr.takeError());
+  return *IROrErr;
+}
+
 } // end namespace
 
 static void validateArchitectureName(StringRef ArchitectureName) {
   if (!MachOObjectFile::isValidArch(ArchitectureName)) {
     std::string Buf;
     raw_string_ostream OS(Buf);
     OS << "Invalid architecture: " << ArchitectureName
        << "\nValid architecture names are:";
default:
     reportError("llvm-lipo action unspecified");
   }
 }
 
 static SmallVector<OwningBinary<Binary>, 1>
 readInputBinaries(ArrayRef<InputFile> InputFiles) {
   SmallVector<OwningBinary<Binary>, 1> InputBinaries;
   for (const InputFile &IF : InputFiles) {
-    Expected<OwningBinary<Binary>> BinaryOrErr = createBinary(IF.FileName);
+    Expected<OwningBinary<Binary>> BinaryOrErr =
+        createBinary(IF.FileName, &LLVMCtx);
     if (!BinaryOrErr)
       reportError(IF.FileName, BinaryOrErr.takeError());
     const Binary *B = BinaryOrErr->getBinary();
-    if (!B->isArchive() && !B->isMachO() && !B->isMachOUniversalBinary())
+    if (!B->isArchive() && !B->isMachO() && !B->isMachOUniversalBinary() &&
+        !B->isIR())
       reportError("File " + IF.FileName + " has unsupported binary format");
-    if (IF.ArchType && (B->isMachO() || B->isArchive())) {
-      const auto S = B->isMachO() ? Slice(*cast<MachOObjectFile>(B))
-                                  : archiveSlice(cast<Archive>(B), IF.FileName);
+    if (IF.ArchType && (B->isMachO() || B->isArchive() || B->isIR())) {
+      const auto S = B->isMachO()
+                         ? Slice(*cast<MachOObjectFile>(B))
+                         : B->isArchive()
+                               ? createSliceFromArchive(cast<Archive>(B))
+                               : createSliceFromIR(cast<IRObjectFile>(B), 0);
       const auto SpecifiedCPUType = MachO::getCPUTypeFromArchitecture(
                                         MachO::getArchitectureFromName(
                                             Triple(*IF.ArchType).getArchName()))
                                         .first;
       // For compatibility with cctools' lipo the comparison is relaxed just to
       // checking cputypes.
       if (S.getCPUType() != SpecifiedCPUType)
         reportError("specified architecture: " + *IF.ArchType +
static void verifyArch(ArrayRef<OwningBinary<Binary>> InputBinaries,
   }
   exit(EXIT_SUCCESS);
 }
 
 static void printBinaryArchs(const Binary *Binary, raw_ostream &OS) {
   // Prints trailing space for compatibility with cctools lipo.
   if (auto UO = dyn_cast<MachOUniversalBinary>(Binary)) {
     for (const auto &O : UO->objects()) {
+      // Order here is important, because both MachOObjectFile and
+      // IRObjectFile can be created with a binary that has embedded bitcode.
       Expected<std::unique_ptr<MachOObjectFile>> MachOObjOrError =
           O.getAsObjectFile();
       if (MachOObjOrError) {
         OS << Slice(*(MachOObjOrError->get())).getArchString() << " ";
         continue;
       }
+      Expected<std::unique_ptr<IRObjectFile>> IROrError =
+          O.getAsIRObject(LLVMCtx);
+      if (IROrError) {
+        consumeError(MachOObjOrError.takeError());
+        Expected<Slice> SliceOrErr =
+            Slice::create(IROrError->get(), O.getAlign());
+        if (!SliceOrErr) {
+          reportError(Binary->getFileName(), SliceOrErr.takeError());
+          continue;
+        }
+        OS << SliceOrErr.get().getArchString() << " ";
+        continue;
+      }
       Expected<std::unique_ptr<Archive>> ArchiveOrError = O.getAsArchive();
       if (ArchiveOrError) {
         consumeError(MachOObjOrError.takeError());
-        OS << archiveSlice(ArchiveOrError->get(), Binary->getFileName())
-                  .getArchString()
+        consumeError(IROrError.takeError());
+        OS << createSliceFromArchive(ArchiveOrError->get()).getArchString()
            << " ";
         continue;
       }
       consumeError(ArchiveOrError.takeError());
       reportError(Binary->getFileName(), MachOObjOrError.takeError());
+      reportError(Binary->getFileName(), IROrError.takeError());
     }
     OS << "\n";
     return;
   }
   OS << Slice(*cast<MachOObjectFile>(Binary)).getArchString() << " \n";
 }
 
 LLVM_ATTRIBUTE_NORETURN
reportError("input file " +
                 InputBinaries.front().getBinary()->getFileName() +
                 " must be a fat file when the -thin option is specified");
     exit(EXIT_FAILURE);
   }
 
   auto *UO = cast<MachOUniversalBinary>(InputBinaries.front().getBinary());
   Expected<std::unique_ptr<MachOObjectFile>> Obj =
       UO->getMachOObjectForArch(ArchType);
+  Expected<std::unique_ptr<IRObjectFile>> IRObj =
+      UO->getIRObjectForArch(ArchType, LLVMCtx);
   Expected<std::unique_ptr<Archive>> Ar = UO->getArchiveForArch(ArchType);
-  if (!Obj && !Ar)
+  if (!Obj && !IRObj && !Ar)
     reportError("fat input file " + UO->getFileName() +
                 " does not contain the specified architecture " + ArchType +
                 " to thin it to");
-  Binary *B = Obj ? static_cast<Binary *>(Obj->get())
-                  : static_cast<Binary *>(Ar->get());
+  Binary *B;
+  // Order here is important, because both Obj and IRObj will be valid with a
+  // binary that has embedded bitcode.
+  if (Obj)
+    B = Obj->get();
+  else if (IRObj)
+    B = IRObj->get();
+  else
+    B = Ar->get();
+
   Expected<std::unique_ptr<FileOutputBuffer>> OutFileOrError =
       FileOutputBuffer::create(OutputFileName,
                                B->getMemoryBufferRef().getBufferSize(),
                                sys::fs::can_execute(UO->getFileName())
                                    ? FileOutputBuffer::F_executable
                                    : 0);
   if (!OutFileOrError)
     reportError(OutputFileName, OutFileOrError.takeError());
for (StringRef Arch : Alignments.keys())
     if (!HasArch(Arch))
       reportError("-segalign " + Arch +
                   " <value> specified but resulting fat file does not contain "
                   "that architecture ");
 }
 
 // Updates vector ExtractedObjects with the MachOObjectFiles extracted from
 // Universal Binary files to transfer ownership.
-static SmallVector<Slice, 2> buildSlices(
-    ArrayRef<OwningBinary<Binary>> InputBinaries,
+static SmallVector<Slice, 2>
+buildSlices(ArrayRef<OwningBinary<Binary>> InputBinaries,
             const StringMap<const uint32_t> &Alignments,
-    SmallVectorImpl<std::unique_ptr<MachOObjectFile>> &ExtractedObjects) {
+            SmallVectorImpl<std::unique_ptr<SymbolicFile>> &ExtractedObjects) {
   SmallVector<Slice, 2> Slices;
   for (auto &IB : InputBinaries) {
     const Binary *InputBinary = IB.getBinary();
     if (auto UO = dyn_cast<MachOUniversalBinary>(InputBinary)) {
       for (const auto &O : UO->objects()) {
+        // Order here is important, because both MachOObjectFile and
+        // IRObjectFile can be created with a binary that has embedded bitcode.
         Expected<std::unique_ptr<MachOObjectFile>> BinaryOrError =
             O.getAsObjectFile();
-        if (!BinaryOrError)
-          reportError(InputBinary->getFileName(), BinaryOrError.takeError());
+        if (BinaryOrError) {
+          Slices.emplace_back(*(BinaryOrError.get()), O.getAlign());
           ExtractedObjects.push_back(std::move(BinaryOrError.get()));
-        Slices.emplace_back(*(ExtractedObjects.back().get()), O.getAlign());
+          continue;
+        }
+        Expected<std::unique_ptr<IRObjectFile>> IROrError =
+            O.getAsIRObject(LLVMCtx);
+        if (IROrError) {
+          consumeError(BinaryOrError.takeError());
+          Slice S = createSliceFromIR(
+              static_cast<IRObjectFile *>(IROrError.get().get()), O.getAlign());
+          ExtractedObjects.emplace_back(std::move(IROrError.get()));
+          Slices.emplace_back(std::move(S));
+          continue;
+        }
+        reportError(InputBinary->getFileName(), BinaryOrError.takeError());
       }
     } else if (auto O = dyn_cast<MachOObjectFile>(InputBinary)) {
       Slices.emplace_back(*O);
     } else if (auto A = dyn_cast<Archive>(InputBinary)) {
-      Slices.push_back(archiveSlice(A, InputBinary->getFileName()));
+      Slices.push_back(createSliceFromArchive(A));
+    } else if (const auto *IRO = dyn_cast<IRObjectFile>(InputBinary)) {
+      // Original Apple's lipo set the alignment to 0
+      Expected<Slice> SliceOrErr = Slice::create(IRO, 0);
+      if (!SliceOrErr) {
+        reportError(InputBinary->getFileName(), SliceOrErr.takeError());
+        continue;
+      }
+      Slices.emplace_back(std::move(SliceOrErr.get()));
     } else {
       llvm_unreachable("Unexpected binary format");
     }
   }
   updateAlignments(Slices, Alignments);
   return Slices;
 }
 
 LLVM_ATTRIBUTE_NORETURN
 static void createUniversalBinary(ArrayRef<OwningBinary<Binary>> InputBinaries,
                                   const StringMap<const uint32_t> &Alignments,
                                   StringRef OutputFileName) {
   assert(InputBinaries.size() >= 1 && "Incorrect number of input binaries");
   assert(!OutputFileName.empty() && "Create expects a single output file");
 
-  SmallVector<std::unique_ptr<MachOObjectFile>, 1> ExtractedObjects;
+  SmallVector<std::unique_ptr<SymbolicFile>, 1> ExtractedObjects;
   SmallVector<Slice, 1> Slices =
       buildSlices(InputBinaries, Alignments, ExtractedObjects);
   checkArchDuplicates(Slices);
   checkUnusedAlignments(Slices, Alignments);
 
   llvm::stable_sort(Slices);
   if (Error E = writeUniversalBinary(Slices, OutputFileName))
     reportError(std::move(E));
static void extractSlice(ArrayRef<OwningBinary<Binary>> InputBinaries,
   assert(!OutputFileName.empty() && "Thin expects a single output file");
 
   if (InputBinaries.front().getBinary()->isMachO()) {
     reportError("input file " +
                 InputBinaries.front().getBinary()->getFileName() +
                 " must be a fat file when the -extract option is specified");
   }
 
-  SmallVector<std::unique_ptr<MachOObjectFile>, 2> ExtractedObjects;
+  SmallVector<std::unique_ptr<SymbolicFile>, 2> ExtractedObjects;
   SmallVector<Slice, 2> Slices =
       buildSlices(InputBinaries, Alignments, ExtractedObjects);
   erase_if(Slices, [ArchType](const Slice &S) {
     return ArchType != S.getArchString();
   });
 
   if (Slices.empty())
     reportError(
reportError("input file " +
                 InputBinaries.front().getBinary()->getFileName() +
                 " must be a fat file when the -replace option is specified");
 
   SmallVector<OwningBinary<Binary>, 1> ReplacementBinaries =
       readInputBinaries(ReplacementFiles);
 
   StringMap<Slice> ReplacementSlices =
       buildReplacementSlices(ReplacementBinaries, Alignments);
-  SmallVector<std::unique_ptr<MachOObjectFile>, 2> ExtractedObjects;
+  SmallVector<std::unique_ptr<SymbolicFile>, 2> ExtractedObjects;
   SmallVector<Slice, 2> Slices =
       buildSlices(InputBinaries, Alignments, ExtractedObjects);
 
   for (auto &Slice : Slices) {
     auto It = ReplacementSlices.find(Slice.getArchString());
     if (It != ReplacementSlices.end()) {
       Slice = It->second;
       ReplacementSlices.erase(It); // only keep remaining replacing arch_types