[LLVM-C] Begin to Expose A More General Binary Interface

[CodaFi]

Summary

Provides a new type, LLVMBinaryRef, and a binding to llvm::object::createBinary for more general interoperation with binary files than LLVMObjectFileRef. It also provides the proper non-consuming API for input buffers and populates an out parameter for error handling if necessary - two things the previous API did not do.

In a follow-up, I'll define section and symbol iterators and begin to build upon the existing test infrastructure.

This patch is a first step towards deprecating that API and replacing it with something more robust.

[whitequark]

Can you provide a bit more context for this change? It looks good, I just want to understand the plan here in more detail.

[CodaFi]

The ObjectFile APIs seem to have rotted a bit. There's a bunch of pitfalls when working with the thing like the fact that it goes out of its way to move the input buffer despite the create APIs taking non-owning references to the buffer in C++, accessors that no longer do anything useful (LLVMGetRelocationValueString) and inconsistent error handling routines (some functions return NULL, LLVMGetSymbolName and LLVMGetSymbolName reports a fatal error). On top of that, it's not actually a general interface to ObjectFile - it can't handle Mach-O Universal binaries as inputs for one.

I want to re-approach these APIs from the perspective of llvm::object::Binary first, then specialize them for ObjectFiles, Archives, etc. The idea being to eventually grow a Binary.h once we can provide a more comprehensive API.

[whitequark]

SGTM, just one question about ownership.

llvm/include/llvm-c/Object.h

74

Who owns the returned memory buffer?

[CodaFi]

llvm/include/llvm-c/Object.h

74

Good point, it's created from a MemoryBuffer Ref, so whoever owns the memory buffer that created this binary file owns the returned memory buffer. Deallocating the buffer returned from this function will also deallocate that buffer and cause all kinds of fun nosebats to take flight.

I'll update the docs.

[CodaFi]

llvm/include/llvm-c/Object.h

74

Ugh, no, that's wrong. It's a non-owning reference to the actually block of memory but it's still a pointer value that needs to be freed by the caller.

[CodaFi]

Rename LLVMBinaryGetMemoryBuffer to LLVMBinaryCopyMemoryBuffer to better convey ownership semantics. Update the docs.

[whitequark]

LGTM

llvm/include/llvm-c/Object.h

  */
 
 // Opaque type wrappers
 typedef struct LLVMOpaqueObjectFile *LLVMObjectFileRef;
 typedef struct LLVMOpaqueSectionIterator *LLVMSectionIteratorRef;
 typedef struct LLVMOpaqueSymbolIterator *LLVMSymbolIteratorRef;
 typedef struct LLVMOpaqueRelocationIterator *LLVMRelocationIteratorRef;
 
+/**
+ * Create a binary file from the given memory buffer.
+ *
+ * The exact type of the binary file will be inferred automatically, and the
+ * appropriate implementation selected.  The context may be NULL except if
+ * the resulting file is an LLVM IR file.
+ *
+ * The memory buffer is not consumed by this function.  It is the responsibilty
+ * of the caller to free it with \c LLVMDisposeMemoryBuffer.
+ *
+ * If NULL is returned, the \p ErrorMessage parameter is populated with the
+ * error's description.  It is then the caller's responsibility to free this
+ * message by calling \c LLVMDisposeMessage.
+ *
+ * @see llvm::object::createBinary
+ */
+LLVMBinaryRef LLVMCreateBinary(LLVMMemoryBufferRef MemBuf,
+                               LLVMContextRef Context,
+                               char **ErrorMessage);
+
+/**
+ * Dispose of a binary file.
+ *
+ * The binary file does not own its backing buffer.  It is the responsibilty
+ * of the caller to free it with \c LLVMDisposeMemoryBuffer.
+ */
+void LLVMDisposeBinary(LLVMBinaryRef BR);
+
+/**
+ * Retrieves a copy of the memory buffer associated with this object file.
+ *
+ * The returned buffer is merely a shallow copy and does not own the actual
+ * backing buffer of the binary. Nevertheless, it is the responsibility of the
+ * caller to free it with \c LLVMDisposeMemoryBuffer.
+ *
+ * @see llvm::object::getMemoryBufferRef
+ */
+LLVMMemoryBufferRef LLVMBinaryCopyMemoryBuffer(LLVMBinaryRef BR);
+
 // ObjectFile creation
 LLVMObjectFileRef LLVMCreateObjectFile(LLVMMemoryBufferRef MemBuf);
 void LLVMDisposeObjectFile(LLVMObjectFileRef ObjectFile);
 
 // ObjectFile Section iterators
 LLVMSectionIteratorRef LLVMGetSections(LLVMObjectFileRef ObjectFile);
 void LLVMDisposeSectionIterator(LLVMSectionIteratorRef SI);
 LLVMBool LLVMIsSectionIteratorAtEnd(LLVMObjectFileRef ObjectFile,

llvm/include/llvm-c/Types.h

 typedef struct LLVMOpaqueModuleFlagEntry LLVMModuleFlagEntry;
 
 /**
  * @see llvm::JITEventListener
  */
 typedef struct LLVMOpaqueJITEventListener *LLVMJITEventListenerRef;
 
 /**
+ * @see llvm::object::Binary
+ */
+typedef struct LLVMOpaqueBinary *LLVMBinaryRef;
+
+/**
  * @}
  */
 
 #ifdef __cplusplus
 }
 #endif
 
 #endif

llvm/include/llvm/Object/Binary.h

 //===- Binary.h - A generic binary file -------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file declares the Binary class.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_OBJECT_BINARY_H
 #define LLVM_OBJECT_BINARY_H
 
+#include "llvm-c/Types.h"
 #include "llvm/ADT/Triple.h"
 #include "llvm/Object/Error.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include <algorithm>
 #include <memory>
 #include <utility>
 
if (Addr + Size < Addr || Addr + Size < Size ||
         Addr + Size > uintptr_t(M.getBufferEnd()) ||
         Addr < uintptr_t(M.getBufferStart())) {
       return object_error::unexpected_eof;
     }
     return std::error_code();
   }
 };
 
+// Create wrappers for C Binding types (see CBindingWrapping.h).
+DEFINE_ISA_CONVERSION_FUNCTIONS(Binary, LLVMBinaryRef)
+
 /// Create a Binary from Source, autodetecting the file type.
 ///
 /// @param Source The data to create the Binary from.
 Expected<std::unique_ptr<Binary>> createBinary(MemoryBufferRef Source,
                                                LLVMContext *Context = nullptr);
 
 template <typename T> class OwningBinary {
   std::unique_ptr<T> Bin;

llvm/lib/Object/Object.cpp

 //===- Object.cpp - C bindings to the object file library--------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file defines the C bindings to the file-format-independent object
 // library.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm-c/Object.h"
 #include "llvm/ADT/SmallVector.h"
+#include "llvm/IR/LLVMContext.h"
 #include "llvm/Object/ObjectFile.h"
 
 using namespace llvm;
 using namespace object;
 
 inline OwningBinary<ObjectFile> *unwrap(LLVMObjectFileRef OF) {
   return reinterpret_cast<OwningBinary<ObjectFile> *>(OF);
 }

 }
 
 inline LLVMRelocationIteratorRef
 wrap(const relocation_iterator *SI) {
   return reinterpret_cast<LLVMRelocationIteratorRef>
     (const_cast<relocation_iterator*>(SI));
 }
 
+/*--.. Operations on binary files ..........................................--*/
+
+LLVMBinaryRef LLVMCreateBinary(LLVMMemoryBufferRef MemBuf,
+                               LLVMContextRef Context,
+                               char **ErrorMessage) {
+  auto maybeContext = Context ? unwrap(Context) : nullptr;
+  Expected<std::unique_ptr<Binary>> ObjOrErr(
+      createBinary(unwrap(MemBuf)->getMemBufferRef(), maybeContext));
+  if (!ObjOrErr) {
+    *ErrorMessage = strdup(toString(ObjOrErr.takeError()).c_str());
+    return nullptr;
+  }
+
+  return wrap(ObjOrErr.get().release());
+}
+
+LLVMMemoryBufferRef LLVMBinaryCopyMemoryBuffer(LLVMBinaryRef BR) {
+  auto Buf = unwrap(BR)->getMemoryBufferRef();
+  return wrap(llvm::MemoryBuffer::getMemBuffer(
+                Buf.getBuffer(), Buf.getBufferIdentifier(),
+                /*RequiresNullTerminator*/false).release());
+}
+
+void LLVMDisposeBinary(LLVMBinaryRef BR) {
+  delete unwrap(BR);
+}
+
 // ObjectFile creation
 LLVMObjectFileRef LLVMCreateObjectFile(LLVMMemoryBufferRef MemBuf) {
   std::unique_ptr<MemoryBuffer> Buf(unwrap(MemBuf));
   Expected<std::unique_ptr<ObjectFile>> ObjOrErr(
       ObjectFile::createObjectFile(Buf->getMemBufferRef()));
   std::unique_ptr<ObjectFile> Obj;
   if (!ObjOrErr) {
     // TODO: Actually report errors helpfully.