LTO: introduce object file-based on-disk module format.

[pcc]

Summary

This format is simply a regular object file with the bitcode stored in a
section named ".llvmbc", plus any number of other (non-allocated) sections.

One immediate use case for this is to accommodate compilation processes
which expect the object file to contain metadata in non-allocated sections,
such as the ".go_export" section used by some Go compilers [1], although I
imagine that in the future we could consider compiling parts of the module
(such as large non-inlinable functions) directly into the object file to
improve LTO efficiency.

[1] http://golang.org/doc/install/gccgo#Imports

[• rafael]

+ /// Returns 'true' if the bitcode bc is for the specified target triple.
+ static bool isTargetMatch(StringRef bc, const char *triplePrefix);

Using a StringRef is a nice independent improvement in here
(isTargetMatch), please commit it first. Nit: take the opportunity to
rename the argument names to match the llvm style.

LTOModule *LTOModule::createFromBuffer(const void *mem, size_t length,

                                     TargetOptions options,
                                     std::string &errMsg, StringRef path) {
std::unique_ptr<MemoryBuffer> buffer(makeBuffer(mem, length, path));
if (!buffer)
  return nullptr;

• return makeLTOModule(buffer.release(), options, errMsg);

+ return makeLTOModule(StringRef((const char *)mem, length), options, errMsg);

The buffer is not need in here anymore, is it?

• return makeLTOModule(buffer.release(), options, errMsg);

+ return makeLTOModule(buffer->getBuffer(), options, errMsg);

The MemoryBuffer is now destroyed immediately instead of being
transferred into the produced LTOModule. This would work with the
current setup, except you also do

• m->materializeAllPermanently();

Which means the IRReader will be pointed to unmapped memory, no?

BTW, the materializeAllPermanently is a very unfortunate but necessary
thing for the C api. The reason it is there is the way ld64 implements
an optimization:

• We want to avoid putting linkonce_odr symbols in the symbol table

unless their address is significant.

• The way ld64 implements this is being told for every symbol if it

can be dropped from the symbol table.

• To tell ld64 about that we need to look at every use of a symbol.
• To look at every use of a symbol we have to materialize all function

bodies since they may contain an use.

Two higher level questions about this approach:

• It is effectively a way of doing fat object files. GCC does

something similar. One small but recurring annoyance I had when
working with GCC LTO is that it can end up falling back silently to
non-LTO since tho outer container is ELF, with IR in a section. With
LLVM right now we get an error, since without our help the linker is
clueless as to what LLVM IR is. Can't you invert this? Put the ELF in
the IR (or even represent .go_export directly in some metadata) and
extent the gold plugin API to handle this? Another advantage is that
it should be easier to support fat COFF and MachO objects too.

• If we do need to go with IR in ELF instead of ELF in IR, we also

need to update lib/Object to handle it. We still want to get an
IRObjectFile (by default at least) when given one of those files so
that llvm-nm/llvm-ar do the right thing.

[pcc]

Rebase, add LLVMObject support

[• rafael]

It feels like these new format support should be documented somewhere. BitCodeFormat.rst maybe?

Also, an entry in the release notes would be nice.

include/llvm/Object/IRObjectFile.h

54 ↗

(On Diff #13563)

Some comments saying what these function do would be nice.

I wonder if findBitcodeIn... might be a more appropriate name. Not sure.

lib/LTO/LTOModule.cpp

58 ↗

(On Diff #13563)

I don't think we use auto for cases like this.

60 ↗

(On Diff #13563)

Is the empty check needed? It seems better to say that an empty section would be a broken bitcode. That is, this should return true and we would print an error afterwards.

64 ↗

(On Diff #13563)

It seems tempting to just get the buffer and forward to IRObjectFile::getBitcodeFromMemBuffer. Would avoid the duplicated switch over the type.

lib/Object/IRObjectFile.cpp

268 ↗

(On Diff #13563)

Maybe this should take an ObjectFile&?

271 ↗

(On Diff #13563)

You don't need the {

276 ↗

(On Diff #13563)

You don't need the {

lib/Object/SymbolicFile.cpp

65 ↗

(On Diff #13563)

Explicit type, not auto I think.

66 ↗

(On Diff #13563)

The empty check looks odd.

tools/gold/gold-plugin.cpp

551 ↗

(On Diff #13563)

You have to handle the case where we don't have a get_view (bfd using the plugin).

On trunk we create a BufferRef in both branches of the if, so you can probably just move this after the end of the else.

[pcc]

Address comments.

[pcc]

Added docs.

include/llvm/Object/IRObjectFile.h

54 ↗

(On Diff #13563)

Some comments saying what these function do would be nice.

Added.

I wonder if findBitcodeIn... might be a more appropriate name. Not sure.

Ok, I like that name better. Done.

lib/LTO/LTOModule.cpp

58 ↗

(On Diff #13563)

Changed.

60 ↗

(On Diff #13563)

Previously I returned an empty memory buffer from getBitcodeFromMemBuffer if the file was of the wrong type or there was no .llvmbc section present. That was a bad decision in retrospect; that function now returns error codes in those cases, which lets me drop the empty check here and elsewhere.

64 ↗

(On Diff #13563)

Done. Now that I think about it, the previous way wasn't saving us much.

lib/Object/IRObjectFile.cpp

268 ↗

(On Diff #13563)

It can even take a const ObjectFile &. Done.

271 ↗

(On Diff #13563)

Done.

276 ↗

(On Diff #13563)

Done.

lib/Object/SymbolicFile.cpp

65 ↗

(On Diff #13563)

Done.

66 ↗

(On Diff #13563)

Removed.

tools/gold/gold-plugin.cpp

551 ↗

(On Diff #13563)

Are you thinking of another function (claim_file_hook?) It looks like the code in this function (getModuleForFile) uses get_view unconditionally. I don't know if this is correct or not, but it seems to have worked up to now.

[• rafael]

LGTM with a nit.

test/tools/gold/bcsection.ll

5 ↗

(On Diff #13798)

Since the intention is to check that %T/bcsection.o is not just a copy of %T/bcsection.bco, how about extending this to:

llvm-nm -no-llvm-bc %T/bcsection.bco | FileCheck %s --check-prefix=NO-MAIN
lvm-nm %T/bcsection.bco | FileCheck %s
lvm-nm %T/bcsection.o| FileCheck %s

?

[pcc]

test/tools/gold/bcsection.ll

5 ↗

(On Diff #13798)

The first test seems useful (as a negative test for -no-llvm-bc); added.

The LTO test suite already has variations of the second test.

I don't see what the third one would accomplish. More specifically, I don't see how it could fail if "llvm-nm -no-llvm-bc %T/bcsection.o | FileCheck %s" passes. Maybe if the object file had an .llvmbc section whose bitcode does not define 'main'? But this seems unlikely.

[pcc]

Closed by commit rL218078 (authored by @pcc).

[echristo]

Hi Peter,

Sorry to perform some thread necromancy here, but I didn't see this go around the first time. What are you trying to accomplish with this? I'm not sure I see how this is necessary to push metadata around, but maybe I'm missing something. Can you explain a bit?

Thanks!

-eric

[pcc]

Hi Eric,

Have you seen my original proposal? (http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-July/074540.html) In the second paragraph in particular it explains how llgo uses this feature.

Peter

[echristo]

Hi Peter,

If the imports are for .o files and the bitcode is for before compilation
is complete? If it's post compilation then we'll have a .o file and can put
the metadata in as normal? Otherwise if we're using lazy compilation you'll
need to teach something how to read llvm-ir anyhow and so might as well not
bother storing it in an elf file?

I could easily be missing something of the compilation pipeline here.

-eric

llvm/trunk/docs/BitCodeFormat.rst

 
 The bitstream format is an abstract encoding of structured data, very similar to
 XML in some ways.  Like XML, bitstream files contain tags, and nested
 structures, and you can parse the file without having to understand the tags.
 Unlike XML, the bitstream format is a binary encoding, and unlike XML it
 provides a mechanism for the file to self-describe "abbreviations", which are
 effectively size optimizations for the content.
 
-LLVM IR files may be optionally embedded into a `wrapper`_ structure that makes
-it easy to embed extra data along with LLVM IR files.
+LLVM IR files may be optionally embedded into a `wrapper`_ structure, or in a
+`native object file`_. Both of these mechanisms make it easy to embed extra
+data along with LLVM IR files.
 
 This document first describes the LLVM bitstream format, describes the wrapper
 format, then describes the record structure used by LLVM IR files.
 
 .. _bitstream container format:
 
 Bitstream Format
 ================

 
 Each of the fields are 32-bit fields stored in little endian form (as with the
 rest of the bitcode file fields).  The Magic number is always ``0x0B17C0DE`` and
 the version is currently always ``0``.  The Offset field is the offset in bytes
 to the start of the bitcode stream in the file, and the Size field is the size
 in bytes of the stream. CPUType is a target-specific value that can be used to
 encode the CPU of the target.
 
+.. _native object file:
+
+Native Object File Wrapper Format
+=================================
+
+Bitcode files for LLVM IR may also be wrapped in a native object file
+(i.e. ELF, COFF, Mach-O).  The bitcode must be stored in a section of the
+object file named ``.llvmbc``.  This wrapper format is useful for accommodating
+LTO in compilation pipelines where intermediate objects must be native object
+files which contain metadata in other sections.
+
+Not all tools support this format.
+
 .. _encoding of LLVM IR:
 
 LLVM IR Encoding
 ================
 
 LLVM IR is encoded into a bitstream by defining blocks and records.  It uses
 blocks for things like constant pools, functions, symbol tables, etc.  It uses
 records for things like instructions, global variable descriptors, type

llvm/trunk/docs/ReleaseNotes.rst

.. NOTE
    For small 1-3 sentence descriptions, just add an entry at the end of
    this list. If your description won't fit comfortably in one bullet
    point (e.g. maybe you would like to give an example of the
    functionality, or simply have a lot to talk about), see the `NOTE` below
    for adding a new subsection.
 
 * Support for AuroraUX has been removed.
 
+* Added support for a `native object file-based bitcode wrapper format
+  <BitCodeFormat.html#native-object-file>`_.
+
 * ... next change ...
 
 .. NOTE
    If you would like to document a larger change, then you can add a
    subsection about it right here. You can copy the following boilerplate
    and un-indent it (the indentation causes it to be inside this comment).
 
    Special New Feature

llvm/trunk/include/llvm/Object/Error.h

 
 const std::error_category &object_category();
 
 enum class object_error {
   success = 0,
   arch_not_found,
   invalid_file_type,
   parse_failed,
-  unexpected_eof
+  unexpected_eof,
+  bitcode_section_not_found,
 };
 
 inline std::error_code make_error_code(object_error e) {
   return std::error_code(static_cast<int>(e), object_category());
 }
 
 } // end namespace object.
 
 } // end namespace llvm.
 
 namespace std {
 template <>
 struct is_error_code_enum<llvm::object::object_error> : std::true_type {};
 }
 
 #endif

llvm/trunk/include/llvm/Object/IRObjectFile.h

 #include "llvm/Object/SymbolicFile.h"
 
 namespace llvm {
 class Mangler;
 class Module;
 class GlobalValue;
 
 namespace object {
+class ObjectFile;
+
 class IRObjectFile : public SymbolicFile {
   std::unique_ptr<Module> M;
   std::unique_ptr<Mangler> Mang;
   std::vector<std::pair<std::string, uint32_t>> AsmSymbols;
 
 public:
   IRObjectFile(MemoryBufferRef Object, std::unique_ptr<Module> M);
   ~IRObjectFile();
public:
   Module &getModule() {
     return *M;
   }
 
   static inline bool classof(const Binary *v) {
     return v->isIR();
   }
 
+  /// \brief Finds and returns bitcode embedded in the given object file, or an
+  /// error code if not found.
+  static ErrorOr<MemoryBufferRef> findBitcodeInObject(const ObjectFile &Obj);
+
+  /// \brief Finds and returns bitcode in the given memory buffer (which may
+  /// be either a bitcode file or a native object file with embedded bitcode),
+  /// or an error code if not found.
+  static ErrorOr<MemoryBufferRef>
+  findBitcodeInMemBuffer(MemoryBufferRef Object);
+
   static ErrorOr<std::unique_ptr<IRObjectFile>>
   createIRObjectFile(MemoryBufferRef Object, LLVMContext &Context);
 };
 }
 }
 
 #endif

llvm/trunk/lib/LTO/LTOModule.cpp

 #include "llvm/MC/MCInst.h"
 #include "llvm/MC/MCInstrInfo.h"
 #include "llvm/MC/MCParser/MCAsmParser.h"
 #include "llvm/MC/MCSection.h"
 #include "llvm/MC/MCSubtargetInfo.h"
 #include "llvm/MC/MCSymbol.h"
 #include "llvm/MC/MCTargetAsmParser.h"
 #include "llvm/MC/SubtargetFeature.h"
+#include "llvm/Object/IRObjectFile.h"
+#include "llvm/Object/ObjectFile.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Host.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Support/TargetSelect.h"
 #include "llvm/Target/TargetLowering.h"
 #include "llvm/Target/TargetLoweringObjectFile.h"
 #include "llvm/Target/TargetRegisterInfo.h"
 #include "llvm/Target/TargetSubtargetInfo.h"
 #include "llvm/Transforms/Utils/GlobalStatus.h"
 #include <system_error>
 using namespace llvm;
+using namespace llvm::object;
 
 LTOModule::LTOModule(std::unique_ptr<object::IRObjectFile> Obj,
                      llvm::TargetMachine *TM)
     : IRFile(std::move(Obj)), _target(TM) {}
 
 /// isBitcodeFile - Returns 'true' if the file (or memory contents) is LLVM
 /// bitcode.
-bool LTOModule::isBitcodeFile(const void *mem, size_t length) {
-  return sys::fs::identify_magic(StringRef((const char *)mem, length)) ==
-         sys::fs::file_magic::bitcode;
+bool LTOModule::isBitcodeFile(const void *Mem, size_t Length) {
+  ErrorOr<MemoryBufferRef> BCData = IRObjectFile::findBitcodeInMemBuffer(
+      MemoryBufferRef(StringRef((const char *)Mem, Length), "<mem>"));
+  return bool(BCData);
 }
 
-bool LTOModule::isBitcodeFile(const char *path) {
-  sys::fs::file_magic type;
-  if (sys::fs::identify_magic(path, type))
+bool LTOModule::isBitcodeFile(const char *Path) {
+  ErrorOr<std::unique_ptr<MemoryBuffer>> BufferOrErr =
+      MemoryBuffer::getFile(Path);
+  if (!BufferOrErr)
     return false;
-  return type == sys::fs::file_magic::bitcode;
+
+  ErrorOr<MemoryBufferRef> BCData = IRObjectFile::findBitcodeInMemBuffer(
+      BufferOrErr.get()->getMemBufferRef());
+  return bool(BCData);
 }
 
-bool LTOModule::isBitcodeForTarget(MemoryBuffer *buffer,
-                                   StringRef triplePrefix) {
-  std::string Triple =
-      getBitcodeTargetTriple(buffer->getMemBufferRef(), getGlobalContext());
-  return StringRef(Triple).startswith(triplePrefix);
+bool LTOModule::isBitcodeForTarget(MemoryBuffer *Buffer,
+                                   StringRef TriplePrefix) {
+  ErrorOr<MemoryBufferRef> BCOrErr =
+      IRObjectFile::findBitcodeInMemBuffer(Buffer->getMemBufferRef());
+  if (!BCOrErr)
+    return false;
+  std::string Triple = getBitcodeTargetTriple(*BCOrErr, getGlobalContext());
+  return StringRef(Triple).startswith(TriplePrefix);
 }
 
 LTOModule *LTOModule::createFromFile(const char *path, TargetOptions options,
                                      std::string &errMsg) {
   ErrorOr<std::unique_ptr<MemoryBuffer>> BufferOrErr =
       MemoryBuffer::getFile(path);
   if (std::error_code EC = BufferOrErr.getError()) {
     errMsg = EC.message();
LTOModule *LTOModule::createFromBuffer(const void *mem, size_t length,
   StringRef Data((const char *)mem, length);
   MemoryBufferRef Buffer(Data, path);
   return makeLTOModule(Buffer, options, errMsg);
 }
 
 LTOModule *LTOModule::makeLTOModule(MemoryBufferRef Buffer,
                                     TargetOptions options,
                                     std::string &errMsg) {
-  ErrorOr<Module *> MOrErr = parseBitcodeFile(Buffer, getGlobalContext());
+  ErrorOr<MemoryBufferRef> MBOrErr =
+      IRObjectFile::findBitcodeInMemBuffer(Buffer);
+  if (std::error_code EC = MBOrErr.getError()) {
+    errMsg = EC.message();
+    return nullptr;
+  }
+  ErrorOr<Module *> MOrErr = parseBitcodeFile(*MBOrErr, getGlobalContext());
   if (std::error_code EC = MOrErr.getError()) {
     errMsg = EC.message();
     return nullptr;
   }
   std::unique_ptr<Module> M(MOrErr.get());
 
   std::string TripleStr = M->getTargetTriple();
   if (TripleStr.empty())

llvm/trunk/lib/Object/Error.cpp

std::string _object_error_category::message(int EV) const {
   case object_error::arch_not_found:
     return "No object file for requested architecture";
   case object_error::invalid_file_type:
     return "The file was not recognized as a valid object file";
   case object_error::parse_failed:
     return "Invalid data was encountered while parsing the file";
   case object_error::unexpected_eof:
     return "The end of the file was unexpectedly encountered";
+  case object_error::bitcode_section_not_found:
+    return "Bitcode section not found in object file";
   }
   llvm_unreachable("An enumerator of object_error does not have a message "
                    "defined.");
 }
 
 const std::error_category &object::object_category() {
   static _object_error_category o;
   return o;
 }

llvm/trunk/lib/Object/IRObjectFile.cpp

 #include "llvm/IR/Module.h"
 #include "llvm/MC/MCRegisterInfo.h"
 #include "llvm/MC/MCAsmInfo.h"
 #include "llvm/MC/MCContext.h"
 #include "llvm/MC/MCInstrInfo.h"
 #include "llvm/MC/MCObjectFileInfo.h"
 #include "llvm/MC/MCTargetAsmParser.h"
 #include "llvm/MC/MCParser/MCAsmParser.h"
+#include "llvm/Object/ObjectFile.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Support/raw_ostream.h"
 using namespace llvm;
 using namespace object;
 
 IRObjectFile::IRObjectFile(MemoryBufferRef Object, std::unique_ptr<Module> Mod)

 basic_symbol_iterator IRObjectFile::symbol_end_impl() const {
   DataRefImpl Ret;
   uint64_t NumAsm = AsmSymbols.size();
   NumAsm <<= 2;
   Ret.p = 3 | NumAsm;
   return basic_symbol_iterator(BasicSymbolRef(Ret, this));
 }
 
+ErrorOr<MemoryBufferRef> IRObjectFile::findBitcodeInObject(const ObjectFile &Obj) {
+  for (const SectionRef &Sec : Obj.sections()) {
+    StringRef SecName;
+    if (std::error_code EC = Sec.getName(SecName))
+      return EC;
+    if (SecName == ".llvmbc") {
+      StringRef SecContents;
+      if (std::error_code EC = Sec.getContents(SecContents))
+        return EC;
+      return MemoryBufferRef(SecContents, Obj.getFileName());
+    }
+  }
+
+  return object_error::bitcode_section_not_found;
+}
+
+ErrorOr<MemoryBufferRef> IRObjectFile::findBitcodeInMemBuffer(MemoryBufferRef Object) {
+  sys::fs::file_magic Type = sys::fs::identify_magic(Object.getBuffer());
+  switch (Type) {
+  case sys::fs::file_magic::bitcode:
+    return Object;
+  case sys::fs::file_magic::elf_relocatable:
+  case sys::fs::file_magic::macho_object:
+  case sys::fs::file_magic::coff_object: {
+    ErrorOr<std::unique_ptr<ObjectFile>> ObjFile =
+        ObjectFile::createObjectFile(Object, Type);
+    if (!ObjFile)
+      return ObjFile.getError();
+    return findBitcodeInObject(*ObjFile->get());
+  }
+  default:
+    return object_error::invalid_file_type;
+  }
+}
+
 ErrorOr<std::unique_ptr<IRObjectFile>>
 llvm::object::IRObjectFile::createIRObjectFile(MemoryBufferRef Object,
                                                LLVMContext &Context) {
+  ErrorOr<MemoryBufferRef> BCOrErr = findBitcodeInMemBuffer(Object);
+  if (!BCOrErr)
+    return BCOrErr.getError();
 
-  std::unique_ptr<MemoryBuffer> Buff(MemoryBuffer::getMemBuffer(Object, false));
+  std::unique_ptr<MemoryBuffer> Buff(
+      MemoryBuffer::getMemBuffer(BCOrErr.get(), false));
 
   ErrorOr<Module *> MOrErr = getLazyBitcodeModule(std::move(Buff), Context);
   if (std::error_code EC = MOrErr.getError())
     return EC;
 
   std::unique_ptr<Module> M(MOrErr.get());
   return llvm::make_unique<IRObjectFile>(Object, std::move(M));
 }

llvm/trunk/lib/Object/SymbolicFile.cpp

case sys::fs::file_magic::bitcode:
     if (Context)
       return IRObjectFile::createIRObjectFile(Object, *Context);
   // Fallthrough
   case sys::fs::file_magic::unknown:
   case sys::fs::file_magic::archive:
   case sys::fs::file_magic::macho_universal_binary:
   case sys::fs::file_magic::windows_resource:
     return object_error::invalid_file_type;
-  case sys::fs::file_magic::elf_relocatable:
   case sys::fs::file_magic::elf_executable:
   case sys::fs::file_magic::elf_shared_object:
   case sys::fs::file_magic::elf_core:
-  case sys::fs::file_magic::macho_object:
   case sys::fs::file_magic::macho_executable:
   case sys::fs::file_magic::macho_fixed_virtual_memory_shared_lib:
   case sys::fs::file_magic::macho_core:
   case sys::fs::file_magic::macho_preload_executable:
   case sys::fs::file_magic::macho_dynamically_linked_shared_lib:
   case sys::fs::file_magic::macho_dynamic_linker:
   case sys::fs::file_magic::macho_bundle:
   case sys::fs::file_magic::macho_dynamically_linked_shared_lib_stub:
   case sys::fs::file_magic::macho_dsym_companion:
-  case sys::fs::file_magic::coff_object:
   case sys::fs::file_magic::coff_import_library:
   case sys::fs::file_magic::pecoff_executable:
     return ObjectFile::createObjectFile(Object, Type);
+  case sys::fs::file_magic::elf_relocatable:
+  case sys::fs::file_magic::macho_object:
+  case sys::fs::file_magic::coff_object: {
+    ErrorOr<std::unique_ptr<ObjectFile>> Obj =
+        ObjectFile::createObjectFile(Object, Type);
+    if (!Obj || !Context)
+      return std::move(Obj);
+
+    ErrorOr<MemoryBufferRef> BCData =
+        IRObjectFile::findBitcodeInObject(*Obj->get());
+    if (!BCData)
+      return std::move(Obj);
+
+    return IRObjectFile::createIRObjectFile(
+        MemoryBufferRef(BCData->getBuffer(), Object.getBufferIdentifier()),
+        *Context);
+  }
   }
   llvm_unreachable("Unexpected Binary File Type");
 }

llvm/trunk/test/LTO/Inputs/bcsection.macho.s

+.section .llvmbc,.llvmbc
+.incbin "bcsection.bc"

llvm/trunk/test/LTO/Inputs/bcsection.s

+.section .llvmbc
+.incbin "bcsection.bc"

llvm/trunk/test/LTO/bcsection.ll

+; RUN: llvm-as -o %T/bcsection.bc %s
+
+; RUN: llvm-mc -I=%T -filetype=obj -triple=x86_64-pc-win32 -o %T/bcsection.coff.bco %p/Inputs/bcsection.s
+; RUN: llvm-nm %T/bcsection.coff.bco | FileCheck %s
+; RUN: llvm-lto -exported-symbol=main -o %T/bcsection.coff.o %T/bcsection.coff.bco
+; RUN: llvm-nm %T/bcsection.coff.o | FileCheck %s
+
+; RUN: llvm-mc -I=%T -filetype=obj -triple=x86_64-unknown-linux-gnu -o %T/bcsection.elf.bco %p/Inputs/bcsection.s
+; RUN: llvm-nm %T/bcsection.elf.bco | FileCheck %s
+; RUN: llvm-lto -exported-symbol=main -o %T/bcsection.elf.o %T/bcsection.elf.bco
+; RUN: llvm-nm %T/bcsection.elf.o | FileCheck %s
+
+; RUN: llvm-mc -I=%T -filetype=obj -triple=x86_64-apple-darwin11 -o %T/bcsection.macho.bco %p/Inputs/bcsection.macho.s
+; RUN: llvm-nm %T/bcsection.macho.bco | FileCheck %s
+; RUN: llvm-lto -exported-symbol=main -o %T/bcsection.macho.o %T/bcsection.macho.bco
+; RUN: llvm-nm %T/bcsection.macho.o | FileCheck %s
+
+; CHECK: main
+define i32 @main() {
+  ret i32 0
+}

llvm/trunk/test/tools/gold/Inputs/bcsection.s

+.section .llvmbc
+.incbin "bcsection.bc"

llvm/trunk/test/tools/gold/bcsection.ll

+; RUN: llvm-as -o %T/bcsection.bc %s
+
+; RUN: llvm-mc -I=%T -filetype=obj -o %T/bcsection.bco %p/Inputs/bcsection.s
+; RUN: llvm-nm -no-llvm-bc %T/bcsection.bco | count 0
+; RUN: ld -r -o %T/bcsection.o -plugin %llvmshlibdir/LLVMgold.so %T/bcsection.bco
+; RUN: llvm-nm -no-llvm-bc %T/bcsection.o | FileCheck %s
+
+; CHECK: main
+define i32 @main() {
+  ret i32 0
+}

llvm/trunk/tools/gold/gold-plugin.cpp

if (get_view) {
     }
     Buffer = std::move(BufferOrErr.get());
     BufferRef = Buffer->getMemBufferRef();
   }
 
   ErrorOr<std::unique_ptr<object::IRObjectFile>> ObjOrErr =
       object::IRObjectFile::createIRObjectFile(BufferRef, Context);
   std::error_code EC = ObjOrErr.getError();
-  if (EC == BitcodeError::InvalidBitcodeSignature)
+  if (EC == BitcodeError::InvalidBitcodeSignature ||
+      EC == object::object_error::invalid_file_type ||
+      EC == object::object_error::bitcode_section_not_found)
     return LDPS_OK;
 
   *claimed = 1;
 
   if (EC) {
     message(LDPL_ERROR, "LLVM gold plugin has failed to create LTO module: %s",
             EC.message().c_str());
     return LDPS_ERR;
getModuleForFile(LLVMContext &Context, claimed_file &F, raw_fd_ostream *ApiFile,
 
   if (get_symbols(F.handle, F.syms.size(), &F.syms[0]) != LDPS_OK)
     message(LDPL_FATAL, "Failed to get symbol information");
 
   const void *View;
   if (get_view(F.handle, &View) != LDPS_OK)
     message(LDPL_FATAL, "Failed to get a view of file");
 
-  std::unique_ptr<MemoryBuffer> Buffer = MemoryBuffer::getMemBuffer(
-      StringRef((char *)View, File.filesize), "", false);
+  llvm::ErrorOr<MemoryBufferRef> MBOrErr =
+      object::IRObjectFile::findBitcodeInMemBuffer(
+          MemoryBufferRef(StringRef((const char *)View, File.filesize), ""));
+  if (std::error_code EC = MBOrErr.getError())
+    message(LDPL_FATAL, "Could not read bitcode from file : %s",
+            EC.message().c_str());
+
+  std::unique_ptr<MemoryBuffer> Buffer =
+      MemoryBuffer::getMemBuffer(MBOrErr->getBuffer(), "", false);
 
   if (release_input_file(F.handle) != LDPS_OK)
     message(LDPL_FATAL, "Failed to release file information");
 
   ErrorOr<Module *> MOrErr = getLazyBitcodeModule(std::move(Buffer), Context);
 
   if (std::error_code EC = MOrErr.getError())
     message(LDPL_FATAL, "Could not read bitcode from file : %s",

llvm/trunk/tools/llvm-nm/llvm-nm.cpp

 // line.
 cl::list<std::string> SegSect("s", cl::Positional, cl::ZeroOrMore,
                               cl::desc("Dump only symbols from this segment "
                                        "and section name, Mach-O only"));
 
 cl::opt<bool> FormatMachOasHex("x", cl::desc("Print symbol entry in hex, "
                                              "Mach-O only"));
 
+cl::opt<bool> NoLLVMBitcode("no-llvm-bc",
+                            cl::desc("Disable LLVM bitcode reader"));
+
 bool PrintAddress = true;
 
 bool MultipleFiles = false;
 
 bool HadError = false;
 
 std::string ToolName;
 }

 
 static void dumpSymbolNamesFromFile(std::string &Filename) {
   ErrorOr<std::unique_ptr<MemoryBuffer>> BufferOrErr =
       MemoryBuffer::getFileOrSTDIN(Filename);
   if (error(BufferOrErr.getError(), Filename))
     return;
 
   LLVMContext &Context = getGlobalContext();
-  ErrorOr<std::unique_ptr<Binary>> BinaryOrErr =
-      createBinary(BufferOrErr.get()->getMemBufferRef(), &Context);
+  ErrorOr<std::unique_ptr<Binary>> BinaryOrErr = createBinary(
+      BufferOrErr.get()->getMemBufferRef(), NoLLVMBitcode ? nullptr : &Context);
   if (error(BinaryOrErr.getError(), Filename))
     return;
   Binary &Bin = *BinaryOrErr.get();
 
   if (Archive *A = dyn_cast<Archive>(&Bin)) {
     if (ArchiveMap) {
       Archive::symbol_iterator I = A->symbol_begin();
       Archive::symbol_iterator E = A->symbol_end();