Index: docs/Readers.rst =================================================================== --- docs/Readers.rst +++ docs/Readers.rst @@ -52,10 +52,9 @@ -------------- The lld project already has a skeleton of source code for Readers for -``ELF``, ``PECOFF``, ``MachO``, and lld's native Atom graph format -(both binary ``Native`` and ``YAML`` representations). If your file format -is a variant of one of those, you should modify the existing Reader to -support your variant. This is done by customizing the Options +``ELF``, ``PECOFF``, ``MachO``, and lld's native ``YAML`` graph format. +If your file format is a variant of one of those, you should modify the +existing Reader to support your variant. This is done by customizing the Options class for the Reader and making appropriate changes to the ``.cpp`` file to interpret those options and act accordingly. Index: docs/design.rst =================================================================== --- docs/design.rst +++ docs/design.rst @@ -300,60 +300,13 @@ lld::File representations ------------------------- -Just as LLVM has three representations of its IR model, lld has three +Just as LLVM has three representations of its IR model, lld has two representations of its File/Atom/Reference model: * In memory, abstract C++ classes (lld::Atom, lld::Reference, and lld::File). * textual (in YAML) - * binary format ("native") - -Binary File Format -~~~~~~~~~~~~~~~~~~ - -In theory, lld::File objects could be written to disk in an existing Object File -format standard (e.g. ELF). Instead we choose to define a new binary file -format. There are two main reasons for this: fidelity and performance. In order -for lld to work as a linker on all platforms, its internal model must be rich -enough to model all CPU and OS linking features. But if we choose an existing -Object File format as the lld binary format, that means an on going need to -retrofit each platform specific feature needed from alternate platforms into the -existing Object File format. Having our own "native" binary format side steps -that issue. We still need to be able to binary encode all the features, but -once the in-memory model can represent the feature, it is straight forward to -binary encode it. - -The reason to use a binary file format at all, instead of a textual file format, -is speed. You want the binary format to be as fast as possible to read into the -in-memory model. Given that we control the in-memory model and the binary -format, the obvious way to make reading super fast it to make the file format be -basically just an array of atoms. The reader just mmaps in the file and looks -at the header to see how many atoms there are and instantiate that many atom -objects with the atom attribute information coming from that array. The trick -is designing this in a way that can be extended as the Atom mode evolves and new -attributes are added. - -The native object file format starts with a header that lists how many "chunks" -are in the file. A chunk is an array of "ivar data". The native file reader -instantiates an array of Atom objects (with one large malloc call). Each atom -contains just a pointer to its vtable and a pointer to its ivar data. All -methods on lld::Atom are virtual, so all the method implementations return -values based on the ivar data to which it has a pointer. If a new linking -features is added which requires a change to the lld::Atom model, a new native -reader class (e.g. version 2) is defined which knows how to read the new feature -information from the new ivar data. The old reader class (e.g. version 1) is -updated to do its best to model (the lack of the new feature) given the old ivar -data in existing native object files. - -With this model for the native file format, files can be read and turned -into the in-memory graph of lld::Atoms with just a few memory allocations. -And the format can easily adapt over time to new features. - -The binary file format follows the ReaderWriter patterns used in lld. The lld -library comes with the classes: ReaderNative and WriterNative. So, switching -between file formats is as easy as switching which Reader subclass is used. - Textual representations in YAML ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Index: docs/index.rst =================================================================== --- docs/index.rst +++ docs/index.rst @@ -26,8 +26,6 @@ * Extensive unit tests * Internal linker model can be dumped/read to textual format - * Internal linker model can be dumped/read to a new native format - * Native format designed to be fast to read and write * Additional linking features can be plugged in as "passes" * OS specific and CPU specific code factored out Index: include/lld/Core/Error.h =================================================================== --- include/lld/Core/Error.h +++ include/lld/Core/Error.h @@ -19,22 +19,6 @@ namespace lld { -const std::error_category &native_reader_category(); - -enum class NativeReaderError { - success = 0, - unknown_file_format, - file_too_short, - file_malformed, - unknown_chunk_type, - memory_error, - conflicting_target_machine, -}; - -inline std::error_code make_error_code(NativeReaderError e) { - return std::error_code(static_cast(e), native_reader_category()); -} - const std::error_category &YamlReaderCategory(); enum class YamlReaderError { @@ -72,8 +56,6 @@ } // end namespace lld namespace std { -template <> -struct is_error_code_enum : std::true_type {}; template <> struct is_error_code_enum : std::true_type {}; template <> struct is_error_code_enum : std::true_type {}; Index: include/lld/Core/LinkingContext.h =================================================================== --- include/lld/Core/LinkingContext.h +++ include/lld/Core/LinkingContext.h @@ -41,7 +41,6 @@ enum class OutputFileType : uint8_t { Default, // The default output type for this target YAML, // The output type is set to YAML - Native // The output file format is Native (Atoms) }; virtual ~LinkingContext(); @@ -273,13 +272,11 @@ /// Set the various output file types that the linker would /// create bool setOutputFileType(StringRef outputFileType) { - if (outputFileType.equals_lower("yaml")) + if (outputFileType.equals_lower("yaml")) { _outputFileType = OutputFileType::YAML; - else if (outputFileType.equals_lower("native")) - _outputFileType = OutputFileType::YAML; - else - return false; - return true; + return true; + } + return false; } /// Returns the output file type that that the linker needs to create. Index: include/lld/Core/Reader.h =================================================================== --- include/lld/Core/Reader.h +++ include/lld/Core/Reader.h @@ -36,7 +36,7 @@ /// \brief An abstract class for reading object files, library files, and /// executable files. /// -/// Each file format (e.g. ELF, mach-o, PECOFF, native, etc) have a concrete +/// Each file format (e.g. ELF, mach-o, PECOFF, etc) have a concrete /// subclass of Reader. class Reader { public: @@ -115,7 +115,6 @@ // as parameters to the addSupport*() method. void addSupportArchives(bool logLoading); void addSupportYamlFiles(); - void addSupportNativeObjects(); void addSupportCOFFObjects(PECOFFLinkingContext &); void addSupportCOFFImportLibraries(PECOFFLinkingContext &); void addSupportMachOObjects(MachOLinkingContext &); Index: include/lld/Core/TODO.txt =================================================================== --- include/lld/Core/TODO.txt +++ include/lld/Core/TODO.txt @@ -1,7 +1,7 @@ include/lld/Core ~~~~~~~~~~~~~~~~ -* The native/yaml reader/writer interfaces should be changed to return +* The yaml reader/writer interfaces should be changed to return an explanatory string if there is an error. The existing error_code abstraction only works for returning low level OS errors. It does not work for describing formatting issues. Index: include/lld/Core/Writer.h =================================================================== --- include/lld/Core/Writer.h +++ include/lld/Core/Writer.h @@ -23,7 +23,7 @@ /// \brief The Writer is an abstract class for writing object files, shared /// library files, and executable files. Each file format (e.g. ELF, mach-o, -/// PECOFF, native, etc) have a concrete subclass of Writer. +/// PECOFF, etc) have a concrete subclass of Writer. class Writer { public: virtual ~Writer(); @@ -44,7 +44,6 @@ std::unique_ptr createWriterELF(const ELFLinkingContext &); std::unique_ptr createWriterMachO(const MachOLinkingContext &); std::unique_ptr createWriterPECOFF(const PECOFFLinkingContext &); -std::unique_ptr createWriterNative(); std::unique_ptr createWriterYAML(const LinkingContext &); } // end namespace lld Index: lib/Core/Error.cpp =================================================================== --- lib/Core/Error.cpp +++ lib/Core/Error.cpp @@ -16,39 +16,6 @@ using namespace lld; -class _NativeReaderErrorCategory : public std::error_category { -public: - const char* name() const LLVM_NOEXCEPT override { - return "lld.native.reader"; - } - - std::string message(int ev) const override { - switch (static_cast(ev)) { - case NativeReaderError::success: - return "Success"; - case NativeReaderError::unknown_file_format: - return "Unknown file format"; - case NativeReaderError::file_too_short: - return "file truncated"; - case NativeReaderError::file_malformed: - return "file malformed"; - case NativeReaderError::memory_error: - return "out of memory"; - case NativeReaderError::unknown_chunk_type: - return "unknown chunk type"; - case NativeReaderError::conflicting_target_machine: - return "conflicting target machine"; - } - llvm_unreachable("An enumerator of NativeReaderError does not have a " - "message defined."); - } -}; - -const std::error_category &lld::native_reader_category() { - static _NativeReaderErrorCategory o; - return o; -} - class _YamlReaderErrorCategory : public std::error_category { public: const char* name() const LLVM_NOEXCEPT override { Index: lib/Core/TODO.txt =================================================================== --- lib/Core/TODO.txt +++ /dev/null @@ -1,18 +0,0 @@ -lib/Core -~~~~~~~~ - -* Add endianness support to the native reader and writer. - -* The NativeReader has lots of similar code for converting arrays of ivar - data in mapped memory into arrays of objects. The commonality can be - factored out, maybe templatized. - -* The NativeFileFormat.h is old school C structs and constants. We scope - things better by defining constants used with a struct inside the struct - declaration. - -* The native reader and writer currently just blast in memory enumeration - values (e.g. DefinedAtom::Scope) into a byte in the disk format. To support - future changes to the enumerations, there should be a translation layer - to map disk values to in-memory values. - Index: lib/Driver/CoreDriver.cpp =================================================================== --- lib/Driver/CoreDriver.cpp +++ lib/Driver/CoreDriver.cpp @@ -77,7 +77,6 @@ CoreLinkingContext ctx; // Register possible input file parsers. - ctx.registry().addSupportNativeObjects(); ctx.registry().addSupportYamlFiles(); ctx.registry().addKindTable(Reference::KindNamespace::testing, Reference::KindArch::all, coreKindStrings); Index: lib/Driver/DarwinLdDriver.cpp =================================================================== --- lib/Driver/DarwinLdDriver.cpp +++ lib/Driver/DarwinLdDriver.cpp @@ -544,7 +544,6 @@ if (!ctx.doNothing()) { ctx.registry().addSupportMachOObjects(ctx); ctx.registry().addSupportArchives(ctx.logInputFiles()); - ctx.registry().addSupportNativeObjects(); ctx.registry().addSupportYamlFiles(); } Index: lib/Driver/GnuLdDriver.cpp =================================================================== --- lib/Driver/GnuLdDriver.cpp +++ lib/Driver/GnuLdDriver.cpp @@ -632,7 +632,6 @@ ctx->registry().addSupportELFObjects(*ctx); ctx->registry().addSupportArchives(ctx->logInputFiles()); ctx->registry().addSupportYamlFiles(); - ctx->registry().addSupportNativeObjects(); if (ctx->allowLinkWithDynamicLibraries()) ctx->registry().addSupportELFDynamicSharedObjects(*ctx); @@ -739,9 +738,6 @@ case LinkingContext::OutputFileType::YAML: ctx->setOutputPath("-"); break; - case LinkingContext::OutputFileType::Native: - ctx->setOutputPath("a.native"); - break; default: ctx->setOutputPath("a.out"); break; Index: lib/Driver/GnuLdOptions.td =================================================================== --- lib/Driver/GnuLdOptions.td +++ lib/Driver/GnuLdOptions.td @@ -313,7 +313,7 @@ def grp_extns : OptionGroup<"opts">, HelpText<"Extensions">; def output_filetype: Separate<["--"], "output-filetype">, - HelpText<"Specify what type of output file that lld creates, YAML/Native">, + HelpText<"Specify yaml to create an output in YAML format">, Group; def alias_output_filetype: Joined<["--"], "output-filetype=">, Alias; Index: lib/Driver/WinLinkDriver.cpp =================================================================== --- lib/Driver/WinLinkDriver.cpp +++ lib/Driver/WinLinkDriver.cpp @@ -862,7 +862,6 @@ ctx.registry().addSupportCOFFObjects(ctx); ctx.registry().addSupportCOFFImportLibraries(ctx); ctx.registry().addSupportArchives(ctx.logInputFiles()); - ctx.registry().addSupportNativeObjects(); ctx.registry().addSupportYamlFiles(); std::vector newargv = processLinkEnv(ctx, argc, argv); Index: lib/ReaderWriter/CMakeLists.txt =================================================================== --- lib/ReaderWriter/CMakeLists.txt +++ lib/ReaderWriter/CMakeLists.txt @@ -1,6 +1,5 @@ add_subdirectory(ELF) add_subdirectory(MachO) -add_subdirectory(Native) add_subdirectory(PECOFF) add_subdirectory(YAML) Index: lib/ReaderWriter/ELF/ELFLinkingContext.cpp =================================================================== --- lib/ReaderWriter/ELF/ELFLinkingContext.cpp +++ lib/ReaderWriter/ELF/ELFLinkingContext.cpp @@ -75,9 +75,6 @@ case LinkingContext::OutputFileType::YAML: _writer = createWriterYAML(*this); break; - case LinkingContext::OutputFileType::Native: - llvm_unreachable("Unimplemented"); - break; default: _writer = createWriterELF(*this); break; Index: lib/ReaderWriter/Native/CMakeLists.txt =================================================================== --- lib/ReaderWriter/Native/CMakeLists.txt +++ /dev/null @@ -1,7 +0,0 @@ -add_llvm_library(lldNative - ReaderNative.cpp - WriterNative.cpp - LINK_LIBS - lldCore - LLVMSupport - ) Index: lib/ReaderWriter/Native/NativeFileFormat.h =================================================================== --- lib/ReaderWriter/Native/NativeFileFormat.h +++ /dev/null @@ -1,258 +0,0 @@ -//===- lib/ReaderWriter/Native/NativeFileFormat.h -------------------------===// -// -// The LLVM Linker -// -// This file is distributed under the University of Illinois Open Source -// License. See LICENSE.TXT for details. -// -//===----------------------------------------------------------------------===// - -#ifndef LLD_READER_WRITER_NATIVE_NATIVE_FILE_FORMAT_H -#define LLD_READER_WRITER_NATIVE_NATIVE_FILE_FORMAT_H - -#include "llvm/Support/DataTypes.h" -#include - -namespace lld { - -// -// Overview: -// -// The number one design goal of this file format is enable the linker to -// read object files into in-memory Atom objects extremely quickly. -// The second design goal is to enable future modifications to the -// Atom attribute model. -// -// The llvm native object file format is not like traditional object file -// formats (e.g. ELF, COFF, mach-o). There is no symbol table and no -// sections. Instead the file is essentially an array of archived Atoms. -// It is *not* serialized Atoms which would require deserialization into -// in memory objects. Instead it is an array of read-only info about each -// Atom. The NativeReader bulk creates in-memory Atoms which just have -// an ivar which points to the read-only info for that Atom. No additional -// processing is done to construct the in-memory Atoms. All Atom attribute -// getter methods are virtual calls which dig up the info they need from the -// ivar data. -// -// To support the gradual evolution of Atom attributes, the Atom read-only -// data is versioned. The NativeReader chooses which in-memory Atom class -// to use based on the version. What this means is that if new attributes -// are added (or changed) in the Atom model, a new native atom class and -// read-only atom info struct needs to be defined. Then, all the existing -// native reader atom classes need to be modified to do their best effort -// to map their old style read-only data to the new Atom model. At some point -// some classes to support old versions may be dropped. -// -// -// Details: -// -// The native object file format consists of a header that specifies the -// endianness of the file and the architecture along with a list of "chunks" -// in the file. A Chunk is simply a tagged range of the file. There is -// one chunk for the array of atom infos. There is another chunk for the -// string pool, and another for the content pool. -// -// It turns out there most atoms have very similar sets of attributes, only -// the name and content attribute vary. To exploit this fact to reduce the file -// size, the atom read-only info contains just the name and content info plus -// a reference to which attribute set it uses. The attribute sets are stored -// in another chunk. -// - - -// -// An entry in the NativeFileHeader that describes one chunk of the file. -// -struct NativeChunk { - uint32_t signature; - uint32_t fileOffset; - uint32_t fileSize; - uint32_t elementCount; -}; - - -// -// The header in a native object file -// -struct NativeFileHeader { - uint8_t magic[16]; - uint32_t endian; - uint32_t architecture; - uint32_t fileSize; - uint32_t chunkCount; - // NativeChunk chunks[] -}; - -// -// Possible values for NativeChunk.signature field -// -enum NativeChunkSignatures { - NCS_DefinedAtomsV1 = 1, - NCS_AttributesArrayV1 = 2, - NCS_AbsoluteAttributesV1 = 12, - NCS_UndefinedAtomsV1 = 3, - NCS_SharedLibraryAtomsV1 = 4, - NCS_AbsoluteAtomsV1 = 5, - NCS_Strings = 6, - NCS_ReferencesArrayV1 = 7, - NCS_ReferencesArrayV2 = 8, - NCS_TargetsTable = 9, - NCS_AddendsTable = 10, - NCS_Content = 11, -}; - -// -// The 16-bytes at the start of a native object file -// -#define NATIVE_FILE_HEADER_MAGIC "llvm nat obj v1 " - -// -// Possible values for the NativeFileHeader.endian field -// -enum { - NFH_BigEndian = 0x42696745, - NFH_LittleEndian = 0x4574696c -}; - - -// -// Possible values for the NativeFileHeader.architecture field -// -enum { - NFA_x86 = 1, - NFA_x86_64 = 2, - NFA_armv6 = 3, - NFA_armv7 = 4, -}; - - -// -// The NCS_DefinedAtomsV1 chunk contains an array of these structs -// -struct NativeDefinedAtomIvarsV1 { - uint32_t nameOffset; - uint32_t attributesOffset; - uint32_t referencesStartIndex; - uint32_t referencesCount; - uint32_t contentOffset; - uint32_t contentSize; - uint64_t sectionSize; -}; - - -// -// The NCS_AttributesArrayV1 chunk contains an array of these structs -// -struct NativeAtomAttributesV1 { - uint32_t sectionNameOffset; - uint16_t align; - uint16_t alignModulus; - uint8_t scope; - uint8_t interposable; - uint8_t merge; - uint8_t contentType; - uint8_t sectionChoice; - uint8_t deadStrip; - uint8_t dynamicExport; - uint8_t permissions; - uint8_t alias; - uint8_t codeModel; -}; - - - -// -// The NCS_UndefinedAtomsV1 chunk contains an array of these structs -// -struct NativeUndefinedAtomIvarsV1 { - uint32_t nameOffset; - uint32_t flags; - uint32_t fallbackNameOffset; -}; - - -// -// The NCS_SharedLibraryAtomsV1 chunk contains an array of these structs -// -struct NativeSharedLibraryAtomIvarsV1 { - uint64_t size; - uint32_t nameOffset; - uint32_t loadNameOffset; - uint32_t type; - uint32_t flags; -}; - - - -// -// The NCS_AbsoluteAtomsV1 chunk contains an array of these structs -// -struct NativeAbsoluteAtomIvarsV1 { - uint32_t nameOffset; - uint32_t attributesOffset; - uint32_t reserved; - uint64_t value; -}; - - - -// -// The NCS_ReferencesArrayV1 chunk contains an array of these structs -// -struct NativeReferenceIvarsV1 { - enum { - noTarget = UINT16_MAX - }; - uint32_t offsetInAtom; - uint16_t kindValue; - uint8_t kindNamespace; - uint8_t kindArch; - uint16_t targetIndex; - uint16_t addendIndex; -}; - - -// -// The NCS_ReferencesArrayV2 chunk contains an array of these structs -// -struct NativeReferenceIvarsV2 { - enum : unsigned { - noTarget = UINT32_MAX - }; - uint64_t offsetInAtom; - int64_t addend; - uint16_t kindValue; - uint8_t kindNamespace; - uint8_t kindArch; - uint32_t targetIndex; - uint32_t tag; -}; - - -// -// The NCS_TargetsTable chunk contains an array of uint32_t entries. -// The C++ class Reference has a target() method that returns a -// pointer to another Atom. We can't have pointers in object files, -// so instead NativeReferenceIvarsV1 contains an index to the target. -// The index is into this NCS_TargetsTable of uint32_t entries. -// The values in this table are the index of the (target) atom in this file. -// For DefinedAtoms the value is from 0 to NCS_DefinedAtomsV1.elementCount. -// For UndefinedAtoms the value is from NCS_DefinedAtomsV1.elementCount to -// NCS_DefinedAtomsV1.elementCount+NCS_UndefinedAtomsV1.elementCount. -// - - -// -// The NCS_AddendsTable chunk contains an array of int64_t entries. -// If we allocated space for addends directly in NativeReferenceIvarsV1 -// it would double the size of that struct. But since addends are rare, -// we instead just keep a pool of addends and have NativeReferenceIvarsV1 -// (if it needs an addend) just store the index (into the pool) of the -// addend it needs. -// - - - -} // namespace lld - -#endif // LLD_READER_WRITER_NATIVE_NATIVE_FILE_FORMAT_H Index: lib/ReaderWriter/Native/ReaderNative.cpp =================================================================== --- lib/ReaderWriter/Native/ReaderNative.cpp +++ /dev/null @@ -1,881 +0,0 @@ -//===- lib/ReaderWriter/Native/ReaderNative.cpp ---------------------------===// -// -// The LLVM Linker -// -// This file is distributed under the University of Illinois Open Source -// License. See LICENSE.TXT for details. -// -//===----------------------------------------------------------------------===// - -#include "NativeFileFormat.h" -#include "lld/Core/Atom.h" -#include "lld/Core/Error.h" -#include "lld/Core/File.h" -#include "lld/Core/Reader.h" -#include "lld/Core/Simple.h" -#include "llvm/ADT/ArrayRef.h" -#include "llvm/ADT/StringRef.h" -#include "llvm/Support/Debug.h" -#include "llvm/Support/ErrorHandling.h" -#include "llvm/Support/Format.h" -#include "llvm/Support/MemoryBuffer.h" -#include "llvm/Support/raw_ostream.h" -#include -#include - -namespace lld { -namespace native { - -// forward reference -class File; - -// -// An object of this class is instantied for each NativeDefinedAtomIvarsV1 -// struct in the NCS_DefinedAtomsV1 chunk. -// -class NativeDefinedAtomV1 : public DefinedAtom { -public: - NativeDefinedAtomV1(const File& f, - const NativeDefinedAtomIvarsV1* ivarData) - : _file(&f), _ivarData(ivarData) { } - - const lld::File& file() const override; - - uint64_t ordinal() const override; - - StringRef name() const override; - - uint64_t size() const override { return _ivarData->contentSize; } - - uint64_t sectionSize() const override { return _ivarData->sectionSize; } - - DefinedAtom::Scope scope() const override { - return (DefinedAtom::Scope)(attributes().scope); - } - - DefinedAtom::Interposable interposable() const override { - return (DefinedAtom::Interposable)(attributes().interposable); - } - - DefinedAtom::Merge merge() const override { - return (DefinedAtom::Merge)(attributes().merge); - } - - DefinedAtom::ContentType contentType() const override { - const NativeAtomAttributesV1& attr = attributes(); - return (DefinedAtom::ContentType)(attr.contentType); - } - - DefinedAtom::Alignment alignment() const override { - return DefinedAtom::Alignment(attributes().align, - attributes().alignModulus); - } - - DefinedAtom::SectionChoice sectionChoice() const override { - return (DefinedAtom::SectionChoice)(attributes().sectionChoice); - } - - StringRef customSectionName() const override; - - DefinedAtom::DeadStripKind deadStrip() const override { - return (DefinedAtom::DeadStripKind)(attributes().deadStrip); - } - - DynamicExport dynamicExport() const override { - return (DynamicExport)attributes().dynamicExport; - } - - DefinedAtom::CodeModel codeModel() const override { - return DefinedAtom::CodeModel(attributes().codeModel); - } - - DefinedAtom::ContentPermissions permissions() const override { - return (DefinedAtom::ContentPermissions)(attributes().permissions); - } - - ArrayRef rawContent() const override; - - reference_iterator begin() const override; - - reference_iterator end() const override; - - const Reference* derefIterator(const void*) const override; - - void incrementIterator(const void*& it) const override; - -private: - const NativeAtomAttributesV1& attributes() const; - - const File *_file; - const NativeDefinedAtomIvarsV1 *_ivarData; -}; - - - -// -// An object of this class is instantied for each NativeUndefinedAtomIvarsV1 -// struct in the NCS_UndefinedAtomsV1 chunk. -// -class NativeUndefinedAtomV1 : public UndefinedAtom { -public: - NativeUndefinedAtomV1(const File& f, - const NativeUndefinedAtomIvarsV1* ivarData) - : _file(&f), _ivarData(ivarData) { } - - const lld::File& file() const override; - StringRef name() const override; - - CanBeNull canBeNull() const override { - return (CanBeNull)(_ivarData->flags & 0x3); - } - - const UndefinedAtom *fallback() const override; - -private: - const File *_file; - const NativeUndefinedAtomIvarsV1 *_ivarData; - mutable std::unique_ptr _fallback; -}; - - -// -// An object of this class is instantied for each NativeUndefinedAtomIvarsV1 -// struct in the NCS_SharedLibraryAtomsV1 chunk. -// -class NativeSharedLibraryAtomV1 : public SharedLibraryAtom { -public: - NativeSharedLibraryAtomV1(const File& f, - const NativeSharedLibraryAtomIvarsV1* ivarData) - : _file(&f), _ivarData(ivarData) { } - - const lld::File& file() const override; - StringRef name() const override; - StringRef loadName() const override; - - bool canBeNullAtRuntime() const override { - return (_ivarData->flags & 0x1); - } - - Type type() const override { - return (Type)_ivarData->type; - } - - uint64_t size() const override { - return _ivarData->size; - } - -private: - const File *_file; - const NativeSharedLibraryAtomIvarsV1 *_ivarData; -}; - - -// -// An object of this class is instantied for each NativeAbsoluteAtomIvarsV1 -// struct in the NCS_AbsoluteAtomsV1 chunk. -// -class NativeAbsoluteAtomV1 : public AbsoluteAtom { -public: - NativeAbsoluteAtomV1(const File& f, - const NativeAbsoluteAtomIvarsV1* ivarData) - : _file(&f), _ivarData(ivarData) { } - - const lld::File& file() const override; - StringRef name() const override; - Scope scope() const override { - const NativeAtomAttributesV1& attr = absAttributes(); - return (Scope)(attr.scope); - } - uint64_t value() const override { - return _ivarData->value; - } - -private: - const NativeAtomAttributesV1& absAttributes() const; - const File *_file; - const NativeAbsoluteAtomIvarsV1 *_ivarData; -}; - - -// -// An object of this class is instantied for each NativeReferenceIvarsV1 -// struct in the NCS_ReferencesArrayV1 chunk. -// -class NativeReferenceV1 : public Reference { -public: - NativeReferenceV1(const File &f, const NativeReferenceIvarsV1 *ivarData) - : Reference((KindNamespace)ivarData->kindNamespace, - (KindArch)ivarData->kindArch, ivarData->kindValue), - _file(&f), _ivarData(ivarData) {} - - uint64_t offsetInAtom() const override { - return _ivarData->offsetInAtom; - } - - const Atom* target() const override; - Addend addend() const override; - void setTarget(const Atom* newAtom) override; - void setAddend(Addend a) override; - -private: - const File *_file; - const NativeReferenceIvarsV1 *_ivarData; -}; - - -// -// An object of this class is instantied for each NativeReferenceIvarsV1 -// struct in the NCS_ReferencesArrayV1 chunk. -// -class NativeReferenceV2 : public Reference { -public: - NativeReferenceV2(const File &f, const NativeReferenceIvarsV2 *ivarData) - : Reference((KindNamespace)ivarData->kindNamespace, - (KindArch)ivarData->kindArch, ivarData->kindValue), - _file(&f), _ivarData(ivarData) {} - - uint64_t offsetInAtom() const override { - return _ivarData->offsetInAtom; - } - - const Atom* target() const override; - Addend addend() const override; - void setTarget(const Atom* newAtom) override; - void setAddend(Addend a) override; - uint32_t tag() const override; - -private: - const File *_file; - const NativeReferenceIvarsV2 *_ivarData; -}; - - -// -// lld::File object for native llvm object file -// -class File : public lld::File { -public: - File(std::unique_ptr mb) - : lld::File(mb->getBufferIdentifier(), kindObject), - _mb(std::move(mb)), // Reader now takes ownership of buffer - _header(nullptr), _targetsTable(nullptr), _targetsTableCount(0), - _strings(nullptr), _stringsMaxOffset(0), _addends(nullptr), - _addendsMaxIndex(0), _contentStart(nullptr), _contentEnd(nullptr) { - _header = - reinterpret_cast(_mb->getBufferStart()); - } - - /// Parses a File object from a native object file. - std::error_code doParse() override { - const uint8_t *const base = - reinterpret_cast(_mb->getBufferStart()); - StringRef path(_mb->getBufferIdentifier()); - const NativeFileHeader *const header = - reinterpret_cast(base); - const NativeChunk *const chunks = - reinterpret_cast(base + sizeof(NativeFileHeader)); - // make sure magic matches - if (memcmp(header->magic, NATIVE_FILE_HEADER_MAGIC, - sizeof(header->magic)) != 0) - return make_error_code(NativeReaderError::unknown_file_format); - - // make sure mapped file contains all needed data - const size_t fileSize = _mb->getBufferSize(); - if (header->fileSize > fileSize) - return make_error_code(NativeReaderError::file_too_short); - - DEBUG_WITH_TYPE("ReaderNative", - llvm::dbgs() << " Native File Header:" << " fileSize=" - << header->fileSize << " chunkCount=" - << header->chunkCount << "\n"); - - // process each chunk - for (uint32_t i = 0; i < header->chunkCount; ++i) { - std::error_code ec; - const NativeChunk* chunk = &chunks[i]; - // sanity check chunk is within file - if ( chunk->fileOffset > fileSize ) - return make_error_code(NativeReaderError::file_malformed); - if ( (chunk->fileOffset + chunk->fileSize) > fileSize) - return make_error_code(NativeReaderError::file_malformed); - // process chunk, based on signature - switch ( chunk->signature ) { - case NCS_DefinedAtomsV1: - ec = processDefinedAtomsV1(base, chunk); - break; - case NCS_AttributesArrayV1: - ec = processAttributesV1(base, chunk); - break; - case NCS_UndefinedAtomsV1: - ec = processUndefinedAtomsV1(base, chunk); - break; - case NCS_SharedLibraryAtomsV1: - ec = processSharedLibraryAtomsV1(base, chunk); - break; - case NCS_AbsoluteAtomsV1: - ec = processAbsoluteAtomsV1(base, chunk); - break; - case NCS_AbsoluteAttributesV1: - ec = processAbsoluteAttributesV1(base, chunk); - break; - case NCS_ReferencesArrayV1: - ec = processReferencesV1(base, chunk); - break; - case NCS_ReferencesArrayV2: - ec = processReferencesV2(base, chunk); - break; - case NCS_TargetsTable: - ec = processTargetsTable(base, chunk); - break; - case NCS_AddendsTable: - ec = processAddendsTable(base, chunk); - break; - case NCS_Content: - ec = processContent(base, chunk); - break; - case NCS_Strings: - ec = processStrings(base, chunk); - break; - default: - return make_error_code(NativeReaderError::unknown_chunk_type); - } - if ( ec ) { - return ec; - } - } - // TO DO: validate enough chunks were used - - DEBUG_WITH_TYPE("ReaderNative", { - llvm::dbgs() << " ReaderNative DefinedAtoms:\n"; - for (const DefinedAtom *a : defined()) { - llvm::dbgs() << llvm::format(" 0x%09lX", a) - << ", name=" << a->name() - << ", size=" << a->size() << "\n"; - for (const Reference *r : *a) { - llvm::dbgs() << " offset=" - << llvm::format("0x%03X", r->offsetInAtom()) - << ", kind=" << r->kindValue() - << ", target=" << r->target() << "\n"; - } - } - }); - return make_error_code(NativeReaderError::success); - } - - virtual ~File() { - // _mb is automatically deleted because of std::unique_ptr<> - - // All other ivar pointers are pointers into the MemoryBuffer, except - // the _definedAtoms array which was allocated to contain an array - // of Atom objects. The atoms have empty destructors, so it is ok - // to just delete the memory. - delete _referencesV1.arrayStart; - delete _referencesV2.arrayStart; - delete [] _targetsTable; - } - - const AtomVector &defined() const override { - return _definedAtoms; - } - const AtomVector &undefined() const override { - return _undefinedAtoms; - } - const AtomVector &sharedLibrary() const override { - return _sharedLibraryAtoms; - } - const AtomVector &absolute() const override { - return _absoluteAtoms; - } - -private: - friend NativeDefinedAtomV1; - friend NativeUndefinedAtomV1; - friend NativeSharedLibraryAtomV1; - friend NativeAbsoluteAtomV1; - friend NativeReferenceV1; - friend NativeReferenceV2; - template class AtomArray; - - // instantiate array of BASeT from IvarsT data in file - template - std::error_code processAtoms(AtomVector &result, const uint8_t *base, - const NativeChunk *chunk) { - std::vector vec(chunk->elementCount); - const size_t ivarElementSize = chunk->fileSize / chunk->elementCount; - if (ivarElementSize != sizeof(IvarsT)) - return make_error_code(NativeReaderError::file_malformed); - auto *ivar = reinterpret_cast(base + chunk->fileOffset); - for (size_t i = 0; i < chunk->elementCount; ++i) - vec[i] = new (_alloc) AtomT(*this, ivar++); - result = std::move(vec); - return make_error_code(NativeReaderError::success); - } - - // instantiate array of DefinedAtoms from v1 ivar data in file - std::error_code processDefinedAtomsV1(const uint8_t *base, - const NativeChunk *chunk) { - return processAtoms(this->_definedAtoms, base, - chunk); - } - - // set up pointers to attributes array - std::error_code processAttributesV1(const uint8_t *base, - const NativeChunk *chunk) { - this->_attributes = base + chunk->fileOffset; - this->_attributesMaxOffset = chunk->fileSize; - DEBUG_WITH_TYPE("ReaderNative", llvm::dbgs() - << " chunk AttributesV1: " - << " count=" << chunk->elementCount - << " chunkSize=" << chunk->fileSize - << "\n"); - return make_error_code(NativeReaderError::success); - } - - // set up pointers to attributes array - std::error_code processAbsoluteAttributesV1(const uint8_t *base, - const NativeChunk *chunk) { - this->_absAttributes = base + chunk->fileOffset; - this->_absAbsoluteMaxOffset = chunk->fileSize; - DEBUG_WITH_TYPE("ReaderNative", llvm::dbgs() - << " chunk AbsoluteAttributesV1: " - << " count=" << chunk->elementCount - << " chunkSize=" << chunk->fileSize - << "\n"); - return make_error_code(NativeReaderError::success); - } - - // instantiate array of UndefinedAtoms from v1 ivar data in file - std::error_code processUndefinedAtomsV1(const uint8_t *base, - const NativeChunk *chunk) { - return processAtoms(this->_undefinedAtoms, base, - chunk); - } - - - // instantiate array of ShareLibraryAtoms from v1 ivar data in file - std::error_code processSharedLibraryAtomsV1(const uint8_t *base, - const NativeChunk *chunk) { - return processAtoms( - this->_sharedLibraryAtoms, base, chunk); - } - - - // instantiate array of AbsoluteAtoms from v1 ivar data in file - std::error_code processAbsoluteAtomsV1(const uint8_t *base, - const NativeChunk *chunk) { - return processAtoms(this->_absoluteAtoms, base, - chunk); - } - - template - std::error_code - processReferences(const uint8_t *base, const NativeChunk *chunk, - uint8_t *&refsStart, uint8_t *&refsEnd) const { - if (chunk->elementCount == 0) - return make_error_code(NativeReaderError::success); - size_t refsArraySize = chunk->elementCount * sizeof(T); - refsStart = reinterpret_cast( - operator new(refsArraySize, std::nothrow)); - if (refsStart == nullptr) - return make_error_code(NativeReaderError::memory_error); - const size_t ivarElementSize = chunk->fileSize / chunk->elementCount; - if (ivarElementSize != sizeof(U)) - return make_error_code(NativeReaderError::file_malformed); - refsEnd = refsStart + refsArraySize; - const U* ivarData = reinterpret_cast(base + chunk->fileOffset); - for (uint8_t *s = refsStart; s != refsEnd; s += sizeof(T), ++ivarData) { - T *atomAllocSpace = reinterpret_cast(s); - new (atomAllocSpace) T(*this, ivarData); - } - return make_error_code(NativeReaderError::success); - } - - // instantiate array of References from v1 ivar data in file - std::error_code processReferencesV1(const uint8_t *base, - const NativeChunk *chunk) { - uint8_t *refsStart, *refsEnd; - if (std::error_code ec = - processReferences( - base, chunk, refsStart, refsEnd)) - return ec; - this->_referencesV1.arrayStart = refsStart; - this->_referencesV1.arrayEnd = refsEnd; - this->_referencesV1.elementSize = sizeof(NativeReferenceV1); - this->_referencesV1.elementCount = chunk->elementCount; - DEBUG_WITH_TYPE("ReaderNative", { - llvm::dbgs() << " chunk ReferencesV1: " - << " count=" << chunk->elementCount - << " chunkSize=" << chunk->fileSize << "\n"; - }); - return make_error_code(NativeReaderError::success); - } - - // instantiate array of References from v2 ivar data in file - std::error_code processReferencesV2(const uint8_t *base, - const NativeChunk *chunk) { - uint8_t *refsStart, *refsEnd; - if (std::error_code ec = - processReferences( - base, chunk, refsStart, refsEnd)) - return ec; - this->_referencesV2.arrayStart = refsStart; - this->_referencesV2.arrayEnd = refsEnd; - this->_referencesV2.elementSize = sizeof(NativeReferenceV2); - this->_referencesV2.elementCount = chunk->elementCount; - DEBUG_WITH_TYPE("ReaderNative", { - llvm::dbgs() << " chunk ReferencesV2: " - << " count=" << chunk->elementCount - << " chunkSize=" << chunk->fileSize << "\n"; - }); - return make_error_code(NativeReaderError::success); - } - - // set up pointers to target table - std::error_code processTargetsTable(const uint8_t *base, - const NativeChunk *chunk) { - const uint32_t* targetIndexes = reinterpret_cast - (base + chunk->fileOffset); - this->_targetsTableCount = chunk->elementCount; - this->_targetsTable = new const Atom*[chunk->elementCount]; - for (uint32_t i=0; i < chunk->elementCount; ++i) { - const uint32_t index = targetIndexes[i]; - if (index < _definedAtoms.size()) { - this->_targetsTable[i] = _definedAtoms[index]; - continue; - } - const uint32_t undefIndex = index - _definedAtoms.size(); - if (undefIndex < _undefinedAtoms.size()) { - this->_targetsTable[i] = _undefinedAtoms[index]; - continue; - } - const uint32_t slIndex = undefIndex - _undefinedAtoms.size(); - if (slIndex < _sharedLibraryAtoms.size()) { - this->_targetsTable[i] = _sharedLibraryAtoms[slIndex]; - continue; - } - const uint32_t abIndex = slIndex - _sharedLibraryAtoms.size(); - if (abIndex < _absoluteAtoms.size()) { - this->_targetsTable[i] = _absoluteAtoms[abIndex]; - continue; - } - return make_error_code(NativeReaderError::file_malformed); - } - DEBUG_WITH_TYPE("ReaderNative", llvm::dbgs() - << " chunk Targets Table: " - << " count=" << chunk->elementCount - << " chunkSize=" << chunk->fileSize - << "\n"); - return make_error_code(NativeReaderError::success); - } - - - // set up pointers to addend pool in file - std::error_code processAddendsTable(const uint8_t *base, - const NativeChunk *chunk) { - this->_addends = reinterpret_cast - (base + chunk->fileOffset); - this->_addendsMaxIndex = chunk->elementCount; - DEBUG_WITH_TYPE("ReaderNative", llvm::dbgs() - << " chunk Addends: " - << " count=" << chunk->elementCount - << " chunkSize=" << chunk->fileSize - << "\n"); - return make_error_code(NativeReaderError::success); - } - - // set up pointers to string pool in file - std::error_code processStrings(const uint8_t *base, - const NativeChunk *chunk) { - this->_strings = reinterpret_cast(base + chunk->fileOffset); - this->_stringsMaxOffset = chunk->fileSize; - DEBUG_WITH_TYPE("ReaderNative", llvm::dbgs() - << " chunk Strings: " - << " chunkSize=" << chunk->fileSize - << "\n"); - return make_error_code(NativeReaderError::success); - } - - // set up pointers to content area in file - std::error_code processContent(const uint8_t *base, - const NativeChunk *chunk) { - this->_contentStart = base + chunk->fileOffset; - this->_contentEnd = base + chunk->fileOffset + chunk->fileSize; - DEBUG_WITH_TYPE("ReaderNative", llvm::dbgs() - << " chunk content: " - << " chunkSize=" << chunk->fileSize - << "\n"); - return make_error_code(NativeReaderError::success); - } - - StringRef string(uint32_t offset) const { - assert(offset < _stringsMaxOffset); - return StringRef(&_strings[offset]); - } - - Reference::Addend addend(uint32_t index) const { - if ( index == 0 ) - return 0; // addend index zero is used to mean "no addend" - assert(index <= _addendsMaxIndex); - return _addends[index-1]; // one-based indexing - } - - const NativeAtomAttributesV1& attribute(uint32_t off) const { - assert(off < _attributesMaxOffset); - return *reinterpret_cast(_attributes + off); - } - - const NativeAtomAttributesV1& absAttribute(uint32_t off) const { - assert(off < _absAbsoluteMaxOffset); - return *reinterpret_cast(_absAttributes + off); - } - - const uint8_t* content(uint32_t offset, uint32_t size) const { - const uint8_t* result = _contentStart + offset; - assert((result+size) <= _contentEnd); - return result; - } - - const Reference* referenceByIndex(uintptr_t index) const { - if (index < _referencesV1.elementCount) { - return reinterpret_cast( - _referencesV1.arrayStart + index * _referencesV1.elementSize); - } - assert(index < _referencesV2.elementCount); - return reinterpret_cast( - _referencesV2.arrayStart + index * _referencesV2.elementSize); - } - - const Atom* targetV1(uint16_t index) const { - if ( index == NativeReferenceIvarsV1::noTarget ) - return nullptr; - assert(index < _targetsTableCount); - return _targetsTable[index]; - } - - void setTargetV1(uint16_t index, const Atom* newAtom) const { - assert(index != NativeReferenceIvarsV1::noTarget); - assert(index > _targetsTableCount); - _targetsTable[index] = newAtom; - } - - const Atom* targetV2(uint32_t index) const { - if (index == NativeReferenceIvarsV2::noTarget) - return nullptr; - assert(index < _targetsTableCount); - return _targetsTable[index]; - } - - void setTargetV2(uint32_t index, const Atom* newAtom) const { - assert(index != NativeReferenceIvarsV2::noTarget); - assert(index > _targetsTableCount); - _targetsTable[index] = newAtom; - } - - struct IvarArray { - IvarArray() : - arrayStart(nullptr), - arrayEnd(nullptr), - elementSize(0), - elementCount(0) { } - - const uint8_t* arrayStart; - const uint8_t* arrayEnd; - uint32_t elementSize; - uint32_t elementCount; - }; - - std::unique_ptr _mb; - const NativeFileHeader* _header; - AtomVector _definedAtoms; - AtomVector _undefinedAtoms; - AtomVector _sharedLibraryAtoms; - AtomVector _absoluteAtoms; - const uint8_t* _absAttributes; - uint32_t _absAbsoluteMaxOffset; - const uint8_t* _attributes; - uint32_t _attributesMaxOffset; - IvarArray _referencesV1; - IvarArray _referencesV2; - const Atom** _targetsTable; - uint32_t _targetsTableCount; - const char* _strings; - uint32_t _stringsMaxOffset; - const Reference::Addend* _addends; - uint32_t _addendsMaxIndex; - const uint8_t *_contentStart; - const uint8_t *_contentEnd; - llvm::BumpPtrAllocator _alloc; -}; - -inline const lld::File &NativeDefinedAtomV1::file() const { - return *_file; -} - -inline uint64_t NativeDefinedAtomV1::ordinal() const { - const uint8_t* p = reinterpret_cast(_ivarData); - auto *start = reinterpret_cast( - _file->_definedAtoms[0]); - const uint8_t *startp = reinterpret_cast(start->_ivarData); - return p - startp; -} - -inline StringRef NativeDefinedAtomV1::name() const { - return _file->string(_ivarData->nameOffset); -} - -inline const NativeAtomAttributesV1& NativeDefinedAtomV1::attributes() const { - return _file->attribute(_ivarData->attributesOffset); -} - -inline ArrayRef NativeDefinedAtomV1::rawContent() const { - if (!occupiesDiskSpace()) - return ArrayRef(); - const uint8_t* p = _file->content(_ivarData->contentOffset, - _ivarData->contentSize); - return ArrayRef(p, _ivarData->contentSize); -} - -inline StringRef NativeDefinedAtomV1::customSectionName() const { - uint32_t offset = attributes().sectionNameOffset; - return _file->string(offset); -} - -DefinedAtom::reference_iterator NativeDefinedAtomV1::begin() const { - uintptr_t index = _ivarData->referencesStartIndex; - const void* it = reinterpret_cast(index); - return reference_iterator(*this, it); -} - -DefinedAtom::reference_iterator NativeDefinedAtomV1::end() const { - uintptr_t index = _ivarData->referencesStartIndex+_ivarData->referencesCount; - const void* it = reinterpret_cast(index); - return reference_iterator(*this, it); -} - -const Reference* NativeDefinedAtomV1::derefIterator(const void* it) const { - uintptr_t index = reinterpret_cast(it); - return _file->referenceByIndex(index); -} - -void NativeDefinedAtomV1::incrementIterator(const void*& it) const { - uintptr_t index = reinterpret_cast(it); - ++index; - it = reinterpret_cast(index); -} - -inline const lld::File& NativeUndefinedAtomV1::file() const { - return *_file; -} - -inline StringRef NativeUndefinedAtomV1::name() const { - return _file->string(_ivarData->nameOffset); -} - -inline const UndefinedAtom *NativeUndefinedAtomV1::fallback() const { - if (!_ivarData->fallbackNameOffset) - return nullptr; - if (!_fallback) - _fallback.reset(new SimpleUndefinedAtom( - *_file, _file->string(_ivarData->fallbackNameOffset))); - return _fallback.get(); -} - -inline const lld::File& NativeSharedLibraryAtomV1::file() const { - return *_file; -} - -inline StringRef NativeSharedLibraryAtomV1::name() const { - return _file->string(_ivarData->nameOffset); -} - -inline StringRef NativeSharedLibraryAtomV1::loadName() const { - return _file->string(_ivarData->loadNameOffset); -} - - - -inline const lld::File& NativeAbsoluteAtomV1::file() const { - return *_file; -} - -inline StringRef NativeAbsoluteAtomV1::name() const { - return _file->string(_ivarData->nameOffset); -} - -inline const NativeAtomAttributesV1& NativeAbsoluteAtomV1::absAttributes() const { - return _file->absAttribute(_ivarData->attributesOffset); -} - -inline const Atom* NativeReferenceV1::target() const { - return _file->targetV1(_ivarData->targetIndex); -} - -inline Reference::Addend NativeReferenceV1::addend() const { - return _file->addend(_ivarData->addendIndex); -} - -inline void NativeReferenceV1::setTarget(const Atom* newAtom) { - return _file->setTargetV1(_ivarData->targetIndex, newAtom); -} - -inline void NativeReferenceV1::setAddend(Addend a) { - // Do nothing if addend value is not being changed. - if (addend() == a) - return; - llvm_unreachable("setAddend() not supported"); -} - -inline const Atom* NativeReferenceV2::target() const { - return _file->targetV2(_ivarData->targetIndex); -} - -inline Reference::Addend NativeReferenceV2::addend() const { - return _ivarData->addend; -} - -inline void NativeReferenceV2::setTarget(const Atom* newAtom) { - return _file->setTargetV2(_ivarData->targetIndex, newAtom); -} - -inline void NativeReferenceV2::setAddend(Addend a) { - // Do nothing if addend value is not being changed. - if (addend() == a) - return; - llvm_unreachable("setAddend() not supported"); -} - -uint32_t NativeReferenceV2::tag() const { return _ivarData->tag; } - -} // end namespace native - -namespace { - -class NativeReader : public Reader { -public: - bool canParse(file_magic magic, const MemoryBuffer &mb) const override { - const NativeFileHeader *const header = - reinterpret_cast(mb.getBufferStart()); - return (memcmp(header->magic, NATIVE_FILE_HEADER_MAGIC, - sizeof(header->magic)) == 0); - } - - virtual std::error_code - loadFile(std::unique_ptr mb, const class Registry &, - std::vector> &result) const override { - auto *file = new lld::native::File(std::move(mb)); - result.push_back(std::unique_ptr(file)); - return std::error_code(); - } -}; - -} - -void Registry::addSupportNativeObjects() { - add(std::unique_ptr(new NativeReader())); -} - -} // end namespace lld Index: lib/ReaderWriter/Native/WriterNative.cpp =================================================================== --- lib/ReaderWriter/Native/WriterNative.cpp +++ /dev/null @@ -1,566 +0,0 @@ -//===- lib/ReaderWriter/Native/WriterNative.cpp ---------------------------===// -// -// The LLVM Linker -// -// This file is distributed under the University of Illinois Open Source -// License. See LICENSE.TXT for details. -// -//===----------------------------------------------------------------------===// - -#include "NativeFileFormat.h" -#include "lld/Core/File.h" -#include "lld/Core/LinkingContext.h" -#include "lld/Core/Writer.h" -#include "llvm/ADT/ArrayRef.h" -#include "llvm/ADT/DenseMap.h" -#include "llvm/ADT/StringRef.h" -#include "llvm/Support/raw_ostream.h" -#include -#include -#include -#include - -namespace lld { -namespace native { - -/// -/// Class for writing native object files. -/// -class Writer : public lld::Writer { -public: - std::error_code writeFile(const lld::File &file, StringRef outPath) override { - // reserve first byte for unnamed atoms - _stringPool.push_back('\0'); - // visit all atoms - for ( const DefinedAtom *defAtom : file.defined() ) { - this->addIVarsForDefinedAtom(*defAtom); - // We are trying to process all atoms, but the defined() iterator does not - // return group children. So, when a group parent is found, we need to - // handle each child atom. - if (defAtom->isGroupParent()) { - for (const Reference *r : *defAtom) { - if (r->kindNamespace() != lld::Reference::KindNamespace::all) - continue; - if (r->kindValue() == lld::Reference::kindGroupChild) { - const DefinedAtom *target = dyn_cast(r->target()); - assert(target && "Internal Error: kindGroupChild references need " - "to be associated with Defined Atoms only"); - this->addIVarsForDefinedAtom(*target); - } - } - } - } - for ( const UndefinedAtom *undefAtom : file.undefined() ) { - this->addIVarsForUndefinedAtom(*undefAtom); - } - for ( const SharedLibraryAtom *shlibAtom : file.sharedLibrary() ) { - this->addIVarsForSharedLibraryAtom(*shlibAtom); - } - for ( const AbsoluteAtom *absAtom : file.absolute() ) { - this->addIVarsForAbsoluteAtom(*absAtom); - } - - maybeConvertReferencesToV1(); - - // construct file header based on atom information accumulated - this->makeHeader(); - - std::error_code ec; - llvm::raw_fd_ostream out(outPath, ec, llvm::sys::fs::F_None); - if (ec) - return ec; - - this->write(out); - - return std::error_code(); - } - - virtual ~Writer() { - } - -private: - - // write the lld::File in native format to the specified stream - void write(raw_ostream &out) { - assert(out.tell() == 0); - out.write((char*)_headerBuffer, _headerBufferSize); - - writeChunk(out, _definedAtomIvars, NCS_DefinedAtomsV1); - writeChunk(out, _attributes, NCS_AttributesArrayV1); - writeChunk(out, _undefinedAtomIvars, NCS_UndefinedAtomsV1); - writeChunk(out, _sharedLibraryAtomIvars, NCS_SharedLibraryAtomsV1); - writeChunk(out, _absoluteAtomIvars, NCS_AbsoluteAtomsV1); - writeChunk(out, _absAttributes, NCS_AbsoluteAttributesV1); - writeChunk(out, _stringPool, NCS_Strings); - writeChunk(out, _referencesV1, NCS_ReferencesArrayV1); - writeChunk(out, _referencesV2, NCS_ReferencesArrayV2); - - if (!_targetsTableIndex.empty()) { - assert(out.tell() == findChunk(NCS_TargetsTable).fileOffset); - writeTargetTable(out); - } - - if (!_addendsTableIndex.empty()) { - assert(out.tell() == findChunk(NCS_AddendsTable).fileOffset); - writeAddendTable(out); - } - - writeChunk(out, _contentPool, NCS_Content); - } - - template - void writeChunk(raw_ostream &out, std::vector &vector, uint32_t signature) { - if (vector.empty()) - return; - assert(out.tell() == findChunk(signature).fileOffset); - out.write((char*)&vector[0], vector.size() * sizeof(T)); - } - - void addIVarsForDefinedAtom(const DefinedAtom& atom) { - _definedAtomIndex[&atom] = _definedAtomIvars.size(); - NativeDefinedAtomIvarsV1 ivar; - unsigned refsCount; - ivar.nameOffset = getNameOffset(atom); - ivar.attributesOffset = getAttributeOffset(atom); - ivar.referencesStartIndex = getReferencesIndex(atom, refsCount); - ivar.referencesCount = refsCount; - ivar.contentOffset = getContentOffset(atom); - ivar.contentSize = atom.size(); - ivar.sectionSize = atom.sectionSize(); - _definedAtomIvars.push_back(ivar); - } - - void addIVarsForUndefinedAtom(const UndefinedAtom& atom) { - _undefinedAtomIndex[&atom] = _undefinedAtomIvars.size(); - NativeUndefinedAtomIvarsV1 ivar; - ivar.nameOffset = getNameOffset(atom); - ivar.flags = (atom.canBeNull() & 0x03); - ivar.fallbackNameOffset = 0; - if (atom.fallback()) - ivar.fallbackNameOffset = getNameOffset(*atom.fallback()); - _undefinedAtomIvars.push_back(ivar); - } - - void addIVarsForSharedLibraryAtom(const SharedLibraryAtom& atom) { - _sharedLibraryAtomIndex[&atom] = _sharedLibraryAtomIvars.size(); - NativeSharedLibraryAtomIvarsV1 ivar; - ivar.size = atom.size(); - ivar.nameOffset = getNameOffset(atom); - ivar.loadNameOffset = getSharedLibraryNameOffset(atom.loadName()); - ivar.type = (uint32_t)atom.type(); - ivar.flags = atom.canBeNullAtRuntime(); - _sharedLibraryAtomIvars.push_back(ivar); - } - - void addIVarsForAbsoluteAtom(const AbsoluteAtom& atom) { - _absoluteAtomIndex[&atom] = _absoluteAtomIvars.size(); - NativeAbsoluteAtomIvarsV1 ivar; - ivar.nameOffset = getNameOffset(atom); - ivar.attributesOffset = getAttributeOffset(atom); - ivar.reserved = 0; - ivar.value = atom.value(); - _absoluteAtomIvars.push_back(ivar); - } - - void convertReferencesToV1() { - for (const NativeReferenceIvarsV2 &v2 : _referencesV2) { - NativeReferenceIvarsV1 v1; - v1.offsetInAtom = v2.offsetInAtom; - v1.kindNamespace = v2.kindNamespace; - v1.kindArch = v2.kindArch; - v1.kindValue = v2.kindValue; - v1.targetIndex = (v2.targetIndex == NativeReferenceIvarsV2::noTarget) ? - (uint16_t)NativeReferenceIvarsV1::noTarget : v2.targetIndex; - v1.addendIndex = this->getAddendIndex(v2.addend); - _referencesV1.push_back(v1); - } - _referencesV2.clear(); - } - - bool canConvertReferenceToV1(const NativeReferenceIvarsV2 &ref) { - bool validOffset = (ref.offsetInAtom == NativeReferenceIvarsV2::noTarget) || - ref.offsetInAtom < NativeReferenceIvarsV1::noTarget; - return validOffset && ref.targetIndex < UINT16_MAX; - } - - // Convert vector of NativeReferenceIvarsV2 to NativeReferenceIvarsV1 if - // possible. - void maybeConvertReferencesToV1() { - std::set addends; - for (const NativeReferenceIvarsV2 &ref : _referencesV2) { - if (!canConvertReferenceToV1(ref)) - return; - addends.insert(ref.addend); - if (addends.size() >= UINT16_MAX) - return; - } - convertReferencesToV1(); - } - - // fill out native file header and chunk directory - void makeHeader() { - const bool hasDefines = !_definedAtomIvars.empty(); - const bool hasUndefines = !_undefinedAtomIvars.empty(); - const bool hasSharedLibraries = !_sharedLibraryAtomIvars.empty(); - const bool hasAbsolutes = !_absoluteAtomIvars.empty(); - const bool hasReferencesV1 = !_referencesV1.empty(); - const bool hasReferencesV2 = !_referencesV2.empty(); - const bool hasTargetsTable = !_targetsTableIndex.empty(); - const bool hasAddendTable = !_addendsTableIndex.empty(); - const bool hasContent = !_contentPool.empty(); - - int chunkCount = 1; // always have string pool chunk - if ( hasDefines ) chunkCount += 2; - if ( hasUndefines ) ++chunkCount; - if ( hasSharedLibraries ) ++chunkCount; - if ( hasAbsolutes ) chunkCount += 2; - if ( hasReferencesV1 ) ++chunkCount; - if ( hasReferencesV2 ) ++chunkCount; - if ( hasTargetsTable ) ++chunkCount; - if ( hasAddendTable ) ++chunkCount; - if ( hasContent ) ++chunkCount; - - _headerBufferSize = sizeof(NativeFileHeader) - + chunkCount*sizeof(NativeChunk); - _headerBuffer = reinterpret_cast - (operator new(_headerBufferSize, std::nothrow)); - NativeChunk *chunks = - reinterpret_cast(reinterpret_cast(_headerBuffer) - + sizeof(NativeFileHeader)); - memcpy(_headerBuffer->magic, NATIVE_FILE_HEADER_MAGIC, - sizeof(_headerBuffer->magic)); - _headerBuffer->endian = NFH_LittleEndian; - _headerBuffer->architecture = 0; - _headerBuffer->fileSize = 0; - _headerBuffer->chunkCount = chunkCount; - - // create chunk for defined atom ivar array - int nextIndex = 0; - uint32_t nextFileOffset = _headerBufferSize; - if (hasDefines) { - fillChunkHeader(chunks[nextIndex++], nextFileOffset, _definedAtomIvars, - NCS_DefinedAtomsV1); - - // create chunk for attributes - fillChunkHeader(chunks[nextIndex++], nextFileOffset, _attributes, - NCS_AttributesArrayV1); - } - - // create chunk for undefined atom array - if (hasUndefines) - fillChunkHeader(chunks[nextIndex++], nextFileOffset, _undefinedAtomIvars, - NCS_UndefinedAtomsV1); - - // create chunk for shared library atom array - if (hasSharedLibraries) - fillChunkHeader(chunks[nextIndex++], nextFileOffset, - _sharedLibraryAtomIvars, NCS_SharedLibraryAtomsV1); - - // create chunk for shared library atom array - if (hasAbsolutes) { - fillChunkHeader(chunks[nextIndex++], nextFileOffset, _absoluteAtomIvars, - NCS_AbsoluteAtomsV1); - - // create chunk for attributes - fillChunkHeader(chunks[nextIndex++], nextFileOffset, _absAttributes, - NCS_AbsoluteAttributesV1); - } - - // create chunk for symbol strings - // pad end of string pool to 4-bytes - while ((_stringPool.size() % 4) != 0) - _stringPool.push_back('\0'); - fillChunkHeader(chunks[nextIndex++], nextFileOffset, _stringPool, - NCS_Strings); - - // create chunk for referencesV2 - if (hasReferencesV1) - fillChunkHeader(chunks[nextIndex++], nextFileOffset, _referencesV1, - NCS_ReferencesArrayV1); - - // create chunk for referencesV2 - if (hasReferencesV2) - fillChunkHeader(chunks[nextIndex++], nextFileOffset, _referencesV2, - NCS_ReferencesArrayV2); - - // create chunk for target table - if (hasTargetsTable) { - NativeChunk& cht = chunks[nextIndex++]; - cht.signature = NCS_TargetsTable; - cht.fileOffset = nextFileOffset; - cht.fileSize = _targetsTableIndex.size() * sizeof(uint32_t); - cht.elementCount = _targetsTableIndex.size(); - nextFileOffset = cht.fileOffset + cht.fileSize; - } - - // create chunk for addend table - if (hasAddendTable) { - NativeChunk& chad = chunks[nextIndex++]; - chad.signature = NCS_AddendsTable; - chad.fileOffset = nextFileOffset; - chad.fileSize = _addendsTableIndex.size() * sizeof(Reference::Addend); - chad.elementCount = _addendsTableIndex.size(); - nextFileOffset = chad.fileOffset + chad.fileSize; - } - - // create chunk for content - if (hasContent) - fillChunkHeader(chunks[nextIndex++], nextFileOffset, _contentPool, - NCS_Content); - - _headerBuffer->fileSize = nextFileOffset; - } - - template - void fillChunkHeader(NativeChunk &chunk, uint32_t &nextFileOffset, - const std::vector &data, uint32_t signature) { - chunk.signature = signature; - chunk.fileOffset = nextFileOffset; - chunk.fileSize = data.size() * sizeof(T); - chunk.elementCount = data.size(); - nextFileOffset = chunk.fileOffset + chunk.fileSize; - } - - // scan header to find particular chunk - NativeChunk& findChunk(uint32_t signature) { - const uint32_t chunkCount = _headerBuffer->chunkCount; - NativeChunk* chunks = - reinterpret_cast(reinterpret_cast(_headerBuffer) - + sizeof(NativeFileHeader)); - for (uint32_t i=0; i < chunkCount; ++i) { - if ( chunks[i].signature == signature ) - return chunks[i]; - } - llvm_unreachable("findChunk() signature not found"); - } - - // append atom name to string pool and return offset - uint32_t getNameOffset(const Atom& atom) { - return this->getNameOffset(atom.name()); - } - - // check if name is already in pool or append and return offset - uint32_t getSharedLibraryNameOffset(StringRef name) { - assert(!name.empty()); - // look to see if this library name was used by another atom - for (auto &it : _sharedLibraryNames) - if (name.equals(it.first)) - return it.second; - // first use of this library name - uint32_t result = this->getNameOffset(name); - _sharedLibraryNames.push_back(std::make_pair(name, result)); - return result; - } - - // append atom name to string pool and return offset - uint32_t getNameOffset(StringRef name) { - if ( name.empty() ) - return 0; - uint32_t result = _stringPool.size(); - _stringPool.insert(_stringPool.end(), name.begin(), name.end()); - _stringPool.push_back(0); - return result; - } - - // append atom cotent to content pool and return offset - uint32_t getContentOffset(const DefinedAtom& atom) { - if (!atom.occupiesDiskSpace()) - return 0; - uint32_t result = _contentPool.size(); - ArrayRef cont = atom.rawContent(); - _contentPool.insert(_contentPool.end(), cont.begin(), cont.end()); - return result; - } - - // reuse existing attributes entry or create a new one and return offet - uint32_t getAttributeOffset(const DefinedAtom& atom) { - NativeAtomAttributesV1 attrs = computeAttributesV1(atom); - return getOrPushAttribute(_attributes, attrs); - } - - uint32_t getAttributeOffset(const AbsoluteAtom& atom) { - NativeAtomAttributesV1 attrs = computeAbsoluteAttributes(atom); - return getOrPushAttribute(_absAttributes, attrs); - } - - uint32_t getOrPushAttribute(std::vector &dest, - const NativeAtomAttributesV1 &attrs) { - for (size_t i = 0, e = dest.size(); i < e; ++i) { - if (!memcmp(&dest[i], &attrs, sizeof(attrs))) { - // found that this set of attributes already used, so re-use - return i * sizeof(attrs); - } - } - // append new attribute set to end - uint32_t result = dest.size() * sizeof(attrs); - dest.push_back(attrs); - return result; - } - - uint32_t sectionNameOffset(const DefinedAtom& atom) { - // if section based on content, then no custom section name available - if (atom.sectionChoice() == DefinedAtom::sectionBasedOnContent) - return 0; - StringRef name = atom.customSectionName(); - assert(!name.empty()); - // look to see if this section name was used by another atom - for (auto &it : _sectionNames) - if (name.equals(it.first)) - return it.second; - // first use of this section name - uint32_t result = this->getNameOffset(name); - _sectionNames.push_back(std::make_pair(name, result)); - return result; - } - - NativeAtomAttributesV1 computeAttributesV1(const DefinedAtom& atom) { - NativeAtomAttributesV1 attrs; - attrs.sectionNameOffset = sectionNameOffset(atom); - attrs.align = atom.alignment().value; - attrs.alignModulus = atom.alignment().modulus; - attrs.scope = atom.scope(); - attrs.interposable = atom.interposable(); - attrs.merge = atom.merge(); - attrs.contentType = atom.contentType(); - attrs.sectionChoice = atom.sectionChoice(); - attrs.deadStrip = atom.deadStrip(); - attrs.dynamicExport = atom.dynamicExport(); - attrs.codeModel = atom.codeModel(); - attrs.permissions = atom.permissions(); - return attrs; - } - - NativeAtomAttributesV1 computeAbsoluteAttributes(const AbsoluteAtom& atom) { - NativeAtomAttributesV1 attrs; - attrs.scope = atom.scope(); - return attrs; - } - - // add references for this atom in a contiguous block in NCS_ReferencesArrayV2 - uint32_t getReferencesIndex(const DefinedAtom& atom, unsigned& refsCount) { - size_t startRefSize = _referencesV2.size(); - uint32_t result = startRefSize; - for (const Reference *ref : atom) { - NativeReferenceIvarsV2 nref; - nref.offsetInAtom = ref->offsetInAtom(); - nref.kindNamespace = (uint8_t)ref->kindNamespace(); - nref.kindArch = (uint8_t)ref->kindArch(); - nref.kindValue = ref->kindValue(); - nref.targetIndex = this->getTargetIndex(ref->target()); - nref.addend = ref->addend(); - nref.tag = ref->tag(); - _referencesV2.push_back(nref); - } - refsCount = _referencesV2.size() - startRefSize; - return (refsCount == 0) ? 0 : result; - } - - uint32_t getTargetIndex(const Atom* target) { - if ( target == nullptr ) - return NativeReferenceIvarsV2::noTarget; - TargetToIndex::const_iterator pos = _targetsTableIndex.find(target); - if ( pos != _targetsTableIndex.end() ) { - return pos->second; - } - uint32_t result = _targetsTableIndex.size(); - _targetsTableIndex[target] = result; - return result; - } - - void writeTargetTable(raw_ostream &out) { - // Build table of target indexes - uint32_t maxTargetIndex = _targetsTableIndex.size(); - assert(maxTargetIndex > 0); - std::vector targetIndexes(maxTargetIndex); - for (auto &it : _targetsTableIndex) { - const Atom* atom = it.first; - uint32_t targetIndex = it.second; - assert(targetIndex < maxTargetIndex); - - TargetToIndex::iterator pos = _definedAtomIndex.find(atom); - if (pos != _definedAtomIndex.end()) { - targetIndexes[targetIndex] = pos->second; - continue; - } - uint32_t base = _definedAtomIvars.size(); - - pos = _undefinedAtomIndex.find(atom); - if (pos != _undefinedAtomIndex.end()) { - targetIndexes[targetIndex] = pos->second + base; - continue; - } - base += _undefinedAtomIndex.size(); - - pos = _sharedLibraryAtomIndex.find(atom); - if (pos != _sharedLibraryAtomIndex.end()) { - targetIndexes[targetIndex] = pos->second + base; - continue; - } - base += _sharedLibraryAtomIndex.size(); - - pos = _absoluteAtomIndex.find(atom); - assert(pos != _absoluteAtomIndex.end()); - targetIndexes[targetIndex] = pos->second + base; - } - // write table - out.write((char*)&targetIndexes[0], maxTargetIndex * sizeof(uint32_t)); - } - - uint32_t getAddendIndex(Reference::Addend addend) { - if ( addend == 0 ) - return 0; // addend index zero is used to mean "no addend" - AddendToIndex::const_iterator pos = _addendsTableIndex.find(addend); - if ( pos != _addendsTableIndex.end() ) { - return pos->second; - } - uint32_t result = _addendsTableIndex.size() + 1; // one-based index - _addendsTableIndex[addend] = result; - return result; - } - - void writeAddendTable(raw_ostream &out) { - // Build table of addends - uint32_t maxAddendIndex = _addendsTableIndex.size(); - std::vector addends(maxAddendIndex); - for (auto &it : _addendsTableIndex) { - Reference::Addend addend = it.first; - uint32_t index = it.second; - assert(index <= maxAddendIndex); - addends[index-1] = addend; - } - // write table - out.write((char*)&addends[0], maxAddendIndex*sizeof(Reference::Addend)); - } - - typedef std::vector> NameToOffsetVector; - - typedef llvm::DenseMap TargetToIndex; - typedef llvm::DenseMap AddendToIndex; - - NativeFileHeader* _headerBuffer; - size_t _headerBufferSize; - std::vector _stringPool; - std::vector _contentPool; - std::vector _definedAtomIvars; - std::vector _attributes; - std::vector _absAttributes; - std::vector _undefinedAtomIvars; - std::vector _sharedLibraryAtomIvars; - std::vector _absoluteAtomIvars; - std::vector _referencesV1; - std::vector _referencesV2; - TargetToIndex _targetsTableIndex; - TargetToIndex _definedAtomIndex; - TargetToIndex _undefinedAtomIndex; - TargetToIndex _sharedLibraryAtomIndex; - TargetToIndex _absoluteAtomIndex; - AddendToIndex _addendsTableIndex; - NameToOffsetVector _sectionNames; - NameToOffsetVector _sharedLibraryNames; -}; -} // end namespace native - -std::unique_ptr createWriterNative() { - return std::unique_ptr(new native::Writer()); -} -} // end namespace lld Index: lib/ReaderWriter/PECOFF/ReaderCOFF.cpp =================================================================== --- lib/ReaderWriter/PECOFF/ReaderCOFF.cpp +++ lib/ReaderWriter/PECOFF/ReaderCOFF.cpp @@ -312,11 +312,10 @@ if (getMachineType() != llvm::COFF::IMAGE_FILE_MACHINE_UNKNOWN && getMachineType() != _ctx.getMachineType()) { - llvm::errs() << "module machine type '" - << getMachineName(getMachineType()) - << "' conflicts with target machine type '" - << getMachineName(_ctx.getMachineType()) << "'\n"; - return NativeReaderError::conflicting_target_machine; + return make_dynamic_error_code(Twine("module machine type '") + + getMachineName(getMachineType()) + + "' conflicts with target machine type '" + + getMachineName(_ctx.getMachineType()) + "'"); } if (std::error_code ec = getReferenceArch(_referenceArch)) Index: lib/ReaderWriter/PECOFF/ReaderImportHeader.cpp =================================================================== --- lib/ReaderWriter/PECOFF/ReaderImportHeader.cpp +++ lib/ReaderWriter/PECOFF/ReaderImportHeader.cpp @@ -257,7 +257,7 @@ // Check if the total size is valid. if (std::size_t(end - buf) != sizeof(COFF::ImportHeader) + dataSize) - return make_error_code(NativeReaderError::unknown_file_format); + return make_dynamic_error_code(StringRef("Broken import library")); uint16_t hint = read16le(buf + offsetof(COFF::ImportHeader, OrdinalHint)); StringRef symbolName(buf + sizeof(COFF::ImportHeader));