Implement tool to convert bitcode to text.

[kschimpf]

Summary

Provide a new tool for LLVM bitcode files that makes it easier to test
and fuzz. That is, a simple textual form that captures the contents of
binary bitcode files.

The textual form removes all references to abbreviations in the binary
bitcode file. This is done to simplify its form.

A textual bitcode record is a sequence of (unsigned) integers,
separated by commas, and terminated with a semicolin. Each bitcode
record appears on a separate line.

The tool llvm-bcanalyzer also provides a textual version of bitcode
files. However, it doesn't provide anyway to convert the output back
to a binary bitcode file. It is also much more complex.

Besides providing a command line tool, it set up for use with lit
(LLVM Integrated Tester). Invalid bitcode tests, that don't
corresponding to abbreviations and bitstream issues, can be written as
a text file. This text file can run tests and check the results using
FileCheck.

The use of textual bitcode files (unlike the current invalid bitcode
tests) can use a single file to describe the invalid test. It also has
the advantage of being easier to port if the format of bitcode records
change. In such cases, you only need to edit the problematic textual
bitcode records that have changed since the test was added. Current
tests are in binary form, and very difficult to figure out how to
change when the bitcode format changes.

For fuzzing, the text form is easy to model using tokens (the digits,
separator ',', and terminator ';\n'). This allows (local) mutations to
easily be formed by existing fuzzers (such as afl-fuzz) without
modification. The textual-to-binary conversion methods can be used as
a post-processor of the fuzzer, or as a preprocessor to the llvm tool
being tested.

The motivation for this tool has been the success PNaCl
(www.gonacl.com) has had in testing its tools, as well as the ease of
generating invalid test cases.

[jvoung]

Made a quick scan over (not a full review).

Would be nice if folks who are more involved with fuzzing could chime in about whether or not they agree if this tool will be useful to them =)

include/llvm/Bitcode/BitcodeConvert.h

2

Short description next to the file name?

2

BitcodeConvert might be a bit too generic of a name. What does it convert to-from?

I'm sure more experienced reviewers would have a better suggestion. TextualBitcodeConvert ?

14

semicolin -> semicolon (here and below)

17

For "easy to fuzz", I don't know if you mentioned the issue with binary form. The binary form isn't byte aligned, etc...

59

fo -> for

60

Do you have any experience for supporting ErrorRecovery mode?

lib/Bitcode/Reader/BitcodeRecordReader.cpp

1 ↗

(On Diff #29360)

Wonder if these various fils should be TextualBitcode...

18 ↗

(On Diff #29360)

semicolin -> semicolon in this file too

62 ↗

(On Diff #29360)

Don't need to reference "pnacl" here...

133 ↗

(On Diff #29360)

Clarify that if not found, it will not advance the cursor.

142 ↗

(On Diff #29360)

How about skipWhitespaceLines() as an alternate name? This doesn't skip whitespace in the middle of a line.

262 ↗

(On Diff #29360)

read -> Read

271 ↗

(On Diff #29360)

read -> Read

288 ↗

(On Diff #29360)

Could this be shared with the "other" bitcode reader? (binary to IR reader, which has two instances of this already?).

The refactoring can be a separate patch too...

369 ↗

(On Diff #29360)

Doesn't this already check isBitcodeWrapper ?

lib/Bitcode/Writer/BitcodeRecordWriter.cpp

1 ↗

(On Diff #29360)

BitcodeRecordWriter.cpp (capital W)

Wonder if this file should also be prefixed with Textual...

12 ↗

(On Diff #29360)

Capital T for Textual seems a bit inconsistent (binary doesn't get a capital B) in the docs. So, perhaps just lower case it.

243 ↗

(On Diff #29360)

If you are refactoring the reader to share code, in a separate patch, you could share this writeHeader also.

tools/Makefile

31–32

let's keep the P in PARALLEL

tools/llvm-bcconv/Makefile

12

BitReader BitWriter

tools/llvm-bcconv/llvm-bcconv.cpp

2

Update the header (it's not llvm-bcanalyzer)

24

semicolon

31

Is the motivation for making it optional, so that a fuzzer doesn't go off and start filling up the file with comments which don't affect anything?

I guess I would have hoped that a fuzzer would be smart enough to detect that such changes aren't useful (no increase in coverage), but maybe it's still helpful to make it optional...

[kcc]

Will this work with libFuzzer out of the box?
Do you need something form libFuzzer to make it work (better|at all)?

[kschimpf]

I believe so, but haven't tried it yet. I will be on vacation next week,
but will look into this further when I get back on the 20th.

Karl

Karl Schimpf

[kcc]

In D11072#202331, @kschimpf wrote:

I believe so, but haven't tried it yet. I will be on vacation next week,
but will look into this further when I get back on the 20th.

I don't want to block you on this, but my first question is whether we need this change to fuzz using libFuzzer at all.
If I were doing a mutator for llvm bit code, I'd probably do something different.
We even have an interface for custom mutators:
https://github.com/llvm-mirror/llvm/blob/master/lib/Fuzzer/FuzzerInterface.h#L51

Another story is if we also have to support afl-fuzz, w/o adding features to afl-fuzz.

[kschimpf]

I guess I'm confused. I thought you were talking about a custom mutator.
One of the main goals is to allow it to use this API. The fuzzer would fuzz
the textual version. The custom mutator would then modify the fuzzed
bitcode into bitcode records using function readTextualBitcode, and then
reformat it to the binary form by calling writeBinaryBitcode.

This would allow the LLVM bitcode reader to work out of the box, without
having to change the code.

The tool llvm-bcconv just makes it convenient to build unit test cases for
LLVM, and to convert (assembled) llvm fuzzing test cases into the textual
form.

Karl Schimpf

[• rafael]

If we are going to have this (not convinced we should), it should be
another library and not modify lib/Bitcode.

[kschimpf]

Apply fixes from review, and add "simplified" form of bitcode.

[kschimpf]

I have added the new form "simplified" bitcode to this, based on conversations with Kostya and libFuzzer. I also moved the code into a separate library as suggested by Rafael.

include/llvm/Bitcode/BitcodeConvert.h

2

I don't like TextualBitcodeConvert because it converts back and forth between the text, binary, and (new) simplified forms.

The simplified form is based on discussions with Kostya and the way that libFuzzer works. It is intended to improve the ability of that fuzzer to mutate simplified forms.

17

Changed the discussion to try and make this more clear.

60

The main purpose for supporting error recovery is to be able to convert fuzzed (i.e. mutated) bitcode files into something that is (bitstream) readable. It is not clear if they are needed for the other forms (i.e. textual and the new variant "simplified"). However, since all three forms use the same driver, it is easier to assume that we can recover when writing all forms.

lib/Bitcode/Reader/BitcodeRecordReader.cpp

1 ↗

(On Diff #29360)

Decided to separate out each of the reader/writers into separate files, and moved then to a separate subdirectory (and library).

288 ↗

(On Diff #29360)

I agree that this should be factored out, however, I will do it in a separate patch.

369 ↗

(On Diff #29360)

Yes. Forgot to remove TODO after fix.

lib/Bitcode/Writer/BitcodeRecordWriter.cpp

1 ↗

(On Diff #29360)

Refactored the code so that each bitcode writer is in a separate source file.

243 ↗

(On Diff #29360)

Agreed. Will create separate patch for this.

tools/llvm-bcconv/llvm-bcconv.cpp

31

Yes. I'm not sure that it will guarantee that the fuzzer will notice this, so I was being safe.

[• rafael]

Please move the library outside of Bitcode. This is just another
textual representation. This patch should not change anything in
lib/Bitcode.

[kschimpf]

I'm a bit confused on naming conventions. lib/Bitcode represents multiple libraries:

lib/Bitcode/Reader
lib/Bitcode/Writer

I assume that they are contained in lib/Bitcode because they represent Bitcode, bitcode records, and the corresponding bitstreams.

Similarly, the bitcode converter reads/writes bitcode records without converting to IR, or starting with IR. Rather, it only uses the (same) bitcode records that the bitcode reader and writer use. For this reason, I created a separate library lib/Bitcode/Convert.

Are you suggesting that I should (instead) create a separate directory lib/BitcodeConvert, even though lib/Bitcode defines bitcode records as well?

[kschimpf]

Rafael wrote:

I am simply not convinced of the need for this and want it to be as
out of the way as possible.

Given that this is just used by tools/llvm-bcconv, probably the best

i> s to just move *all* the code to tools/llvm-bcconv and not have a

library at all.

I did think of putting it all in tools/llvm-bcconv. However, I would rather have a library. Part of this work is to build fuzzers for LLVM bitcode files. I already have code that fuzzes LLVM bitcode files using lib/Bitcode/Convert and lib/Fuzzer. That is my motivation for having a library.

[kschimpf]

Rafael wrote:

Why can't the fuzzer write a file?

lib/Fuzzer runs the fuzzing in the same process as the test. It doesn't create an intermediate file. Even if it did, the conversion must still take place.

The reason for this is that the what the LLVM bitstream is modeled, it is not conducive to fuzzing. That is, values are variable-rate bit encoded, based on the value. This implies that mutating a couple of bits will (almost always) make the rest of the input bitstream unreadable. The point of the fuzzed mutations is to change small portions of the input, and leave the rest alone. This is not possible with the binary form of bitcode.

Hence (as mentioned in the comments for include/llvm/Bitcode/BitcodeConvert.h), this CL defines two alternative forms: simplified and textual. Both are essentially the same, except that the textual form is human readable, by using textual digits rather than binary bytes to define numbers. These alternative forms are specifically designed to all small mutations to only effect the bitcode record (and possibly the immediately surrounding records), when the contents of a bitcode record is mutated. This makes the mutations of bitcode tractable.

[• rafael]

If it doesn't create files, why do you need a text representation? If
you don't need the text representation, than it is llvm-bcconv that
should be deleted and this library merged into a tools/llvm-bc-fuzzer.

The summary is that this is a lot of code with unproven value. So far
filcab has found more bugs with just afl.

[kcc]

afl is great, but in my experience more fuzzers lead to more bugs.
Besides, libFuzzer being in-process makes it much faster.
(I am biased, of course)
Maybe if Karl can add the actual libFuzzer glue to this patch and demonstrate a couple of bugs found this way
it will be easier to make it through.

[kschimpf]

Files in llvm/test/Convert show how this code could be used.

Files in llvm/test/Convert are as follows:

File MyMain.cpp that shows how the bitcode coversion library of
http://reviews.llvm.org/D11072 can be used. The single input fed to
the generated fuzzer is a trivial program in foo.ll.

Within less than a minute I was able to generate the crashes in this
CL (files with prefix "crash"). Files with extension '.bc' are LLVM
bitcode files. Files with extension '.sbc' are the corresponding
bitcode files in simplified (binary) form. Files with extension '.err'
are the corresponding crashes caused by the '.bc' file.

[kschimpf]

Note: The code in test/Convert will be removed before committing. It was included to show how the bitcode conversion library/tool can be used.

[• rafael]

I am all for fuzzing stuff, but I don't think I will have the time to really review this much code right now.

Kostya is the fuzzer expert and Duncan the bitcode one. If you guys are OK with this, go for it.

I just have two passing requests:

• Please git-clang-format the patch.
• Please don't add unnecessary features. No additional text format if it is not need. No additional library if only one program will use it. Etc.

[kcc]

• Please don't add unnecessary features. No additional text format if it is not need. No additional library if only one program will use it. Etc.

Karl,
Within the Rafael's constraints, to which I tend to agree, you may want to back up to my initial proposal:
Implement a fuzzer with a custom mutator:
https://github.com/llvm-mirror/llvm/blob/master/lib/Fuzzer/FuzzerInterface.h
https://github.com/llvm-mirror/llvm/blob/master/lib/Fuzzer/test/UserSuppliedFuzzerTest.cpp

It can all be done simple and in a single file:

1. Read and parse the bit code
2. translate it into some simple binary form in-memory. The simpler the format is the better.
3. let the fuzzer mutate it
4. translate the mutated binary form back to bitcode. If the translation was successful -- run optiimzations. This step is the most important: how many of the mutated units will lead to a valid bitcode?
5. profit

[kschimpf]

Kostya,

I agree that we should keep it simple. The text format isn't necessary, but
it does make writing unit tests much easier to maintain. Since the records
are simple lists of (textual) numbers, it is easy to update a record in
this form, should the format change.

When the code is in binary form, this is much harder to do.

However, for a first step, I agree that we should not worry about the
textual form. That can be left to be introduced later, when more concrete
examples of its utility can be shown.

I will also look at your suggested list of flow for mutators, to see how it
effects things.

I also agree that we want to keep things as simple as possible. We might be
able to put it all in a custom mutator library. I will look into that.

Finally, if possible, I will split this patch into smaller patches. I
initially presented as a single patch mainly to make it clear how the
pieces fit together.

Karl

Karl Schimpf

[kcc]

When the code is in binary form, this is much harder to do.

Just to clarify:
My suggestion is that files on disk should be in one of the two standard formats: textual bit code format or binary bitcode format.
The "fuzzer-friendly" format should exist only in memory and never be written on disk.

[kschimpf]

Met with Chandler Carruth, JF Bastien , Kostya Serebryany, and myself (Karl Schimpf) to discuss how to proceed with helping the LLVM community using fuzzing. Briefly discussed patch D11072. Then discussed the overall goals of the LLVM community with respect to fuzzing.

We discussed that there really should be 3 levels of fuzzing:

1. Parsing .ll files to generate IR.
2. Optimization phases of the compiler
3. Using bitcode to represent other structured objects.

It was also noted that once these levels are more stable, one could move into more structural fuzzing (details omitted here).

As such, patch D11072 doesn’t really fit this strategy. Hence, this patch will be abandoned.

However, I will continue to work towards fuzzing LLVM to improve the code quality. My initial goals will be to apply afl-fuzz to a corpus of .ll files to find errors in parsing .ll files (i.e. run llvm-as).

I will also work with Kostya on building a fuzzer using libFuzzer to also test parsing .ll files to generate IR. To improve code coverage, the goal is to call functions that remove the bitcode writing step of llvm-as.

Once llvm-as is relatively stable, focus will then move forward to the optimization phases of the compiler.

Any help or comments from the general LLVM community to help with fuzzing is greatly appreciated.

include/llvm/Bitcode/BitcodeConvert.h

+//===- BitcodeConvert.h - Convert format of bitcode files -------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// Models three different forms of (assembled) bitcode files: binary, textual,
+// and simplified.
+//
+// The binary form of an assembled bitcode file is the form generated when you
+// assemble bitcode using llvm-as. The textual form is a human readable form of
+// the contents of the assembled bitcode (and not the preassembled text of the
+// corresponding LLVM instructions).
+//
+// The textual and simplified forms of bitcode files is modeled as a list of
+// bitcode records. Each bitcode record is a list of (unsigned) 64-bit integer
+// values. In the textual form, each line contains a bitcode record, and the
+// bitcode record is terminated with a semicolon.  Bitcode record values are
+// separated by commas.
+//
+// The simplified form is like the textual form, except that the bitcode records
+// are encoded using binary values rather than (just) printable characters.
+//
+// It should be noted that the major problem with the binary form of bitcode
+// records is that each value inserted into the bitcode file has a variable
+// length (bit) width that depends on the value stored. The encoding of the
+// length of the value is part of the written value.  Hence, most mutatations of
+// the saved value is likely to cause the bitstream reader to get lost.
+//
+// Also, some bitcode records (such as enter block) are context sensitive in
+// that it contains a position index to the end of the corresponding bitcode
+// block (in the bitcode file).
+//
+// As a result, almost any bit-twiddling will generate a bitcode file that is
+// not readable by the binary bitstream reader.  This is the major reason that
+// the binary bitcode form is not conducive to mutations (i.e. fuzzing).
+//
+// The goal of textual bitcode records is to provide a simple API to the
+// contents of bitcode files that is human readable, easy to mutate, and easy to
+// write test cases. To do this, the textual form removes the bitcode header.
+// It also removes all notions of abbreviations, and thier corresponding
+// bit-encoded forms. Rather, just the raw data within the bitcode records is
+// written.
+//
+// To model block enter/exits as a bitcode record, special symbolic constants
+// have been defined.  The enter block gets a single parameter, the block ID,
+// and the block exits gets no parameters.
+//
+// If comments are allowed, the comment begins with a semicolin. Any text after
+// a semicolin, to the end of the line is considered a comment. Empty lines are
+// also allowed.
+//
+// The simplified form of bitcode records is like the textual form, except that
+// the bitcode records are written out using a single, simple binary encoding.
+// Like the textual form, the bitcode header is removed, as well as
+// abbreviations.
+//
+// This makes the simplified form conducive to fuzzing tools that do not
+// implement tokenized mutations.
+//
+// The simplified form converts the sequence of bitcode records to a sequence of
+// bytes. Each value is written out using the minimal number of bytes to capture
+// the corresponding 'sign-rotated' value, followed by a special ACCEPT_VALUE
+// byte. Each record is followed by a special ACCEPT_RECORD byte. If the end of
+// the bitcode file doesn't contain a (necessary) ACCEPT_VALUE or ACCEPT_RECORD,
+// they are automatically inserted.
+//
+// A sign-rotated value is generated by shifting left the absolute value of the
+// integer, and moving the sign to the least significant bit.
+//
+// There are three special byte values used in the simplified form,
+// corresponding to an action to take. They are:
+//
+//   ACCEPT_VALUE: Accept the value defined by the bytes between this action,
+//      and the previous action.
+//
+//   ACCEPT_RECORD: Accept the record defined by the sequence of values between
+//      this action, and the previous ACCEPT_RECORD action.
+//
+//   ESCAPE: Treat the next byte as input rather than an action.
+//
+// Note: The main advantage of the textual and simplified forms is that bit
+// twiddling mutations can be applied to this simplified form without garbling
+// the rest of the input (this is the major problem with the binary form of
+// bitcode files). Rather, just the surrounding values/records are effected by a
+// mutation.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_BITCODE_BITCODECONVERT_H
+#define LLVM_BITCODE_BITCODECONVERT_H
+
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/Support/MemoryBuffer.h"
+
+#include <vector>
+
+namespace llvm {
+
+namespace bcconv {
+
+// Codes to represent block enter/exit in bitcode records.
+enum SpecialBlockCodes { BLK_CODE_ENTER = 65535, BLK_CODE_EXIT = 65534 };
+
+typedef SmallVectorImpl<char> OutputBuffer;
+typedef std::vector<uint64_t> BitcodeRecord;
+typedef std::vector<std::unique_ptr<BitcodeRecord>> BitcodeRecordList;
+
+/// Holds (programmable) actions for the simplified form of the bitcode
+/// reader/writer.
+class SimplifiedBitcodeActions {
+  SimplifiedBitcodeActions(const SimplifiedBitcodeActions &) = delete;
+  void operator=(const SimplifiedBitcodeActions &) = delete;
+
+public:
+  SimplifiedBitcodeActions();
+  uint8_t AcceptValue = AcceptValueDefault;
+  uint8_t AcceptRecord = AcceptRecordDefault;
+  uint8_t AcceptEscape = AcceptEscapeDefault;
+
+  static const uint8_t AcceptValueDefault;
+  static const uint8_t AcceptRecordDefault;
+  static const uint8_t AcceptEscapeDefault;
+
+  static const SimplifiedBitcodeActions Default;
+};
+
+/// Reads binary bitcode from Input, and updates Records with its contents.
+std::error_code readBinaryBitcode(StringRef Input, BitcodeRecordList &Records);
+
+/// Reads textual bitcode from Input, and updates Records with its contents.  If
+/// AllowComments is true, the rules are relaxed to allow empty lines and
+/// comment lines beginning with a semicolon.
+std::error_code readTextualBitcode(StringRef Input, BitcodeRecordList &Records,
+                                   bool AllowComments = false);
+
+/// Reads simplified bitcode from input, and updates Records with its contents.
+std::error_code readSimplifiedBitcode(StringRef Input,
+                                      BitcodeRecordList &Records,
+                                      const SimplifiedBitcodeActions &Actions);
+
+/// Writes the binary representation for Records into output.
+bool writeBinaryBitcode(OutputBuffer &Output, const BitcodeRecordList &Records,
+                        std::string &ErrorMessages, bool ErrorRecover = false);
+
+/// Writes the textual representation for Records into output.
+bool writeTextualBitcode(OutputBuffer &Output, const BitcodeRecordList &Records,
+                         std::string &ErrorMessages, bool ErrorRecover = false);
+
+/// Writes the simplified representation for records into output.
+bool writeSimplifiedBitcode(OutputBuffer &Output,
+                            const BitcodeRecordList &Records,
+                            const SimplifiedBitcodeActions &Actions,
+                            std::string &ErrorMessages,
+                            bool ErrorRecover = false);
+
+} // end of namespace llvm::bcconv
+} // end of namespace llvm
+
+#endif //  LLVM_BITCODE_BITCODECONVERT_H

lib/Bitcode/CMakeLists.txt

+add_subdirectory(Convert)
 add_subdirectory(Reader)
 add_subdirectory(Writer)

lib/Bitcode/Convert/BinaryBitcodeReader.cpp

+//===- BinaryBitcodeReader.cpp - Reads binary bitcode ---------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// Implements the binary bitcode reader that builds the corresponding list of
+// bitcode records.
+//
+// ===---------------------------------------------------------------------===//
+
+#include "llvm/Bitcode/BitcodeConvert.h"
+
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/Bitcode/BitstreamReader.h"
+#include "llvm/Bitcode/ReaderWriter.h"
+
+#include "llvm/Support/raw_ostream.h"
+
+using namespace llvm;
+using namespace llvm::bcconv;
+
+namespace {
+
+// Defines reader error codes.
+enum ReaderErrorType {
+  MalformedBlock = 1,
+  MalformedIRFile,
+  RecordAtTopLevel,
+  InvalidBitcodeSignature
+};
+
+// Defines the corresponding error messages.
+class ReaderErrorCategoryType : public std::error_category {
+  ReaderErrorCategoryType(const ReaderErrorCategoryType &) = delete;
+  void operator=(const ReaderErrorCategoryType &) = delete;
+
+public:
+  static const ReaderErrorCategoryType &get() { return Sentinel; }
+
+private:
+  static const ReaderErrorCategoryType Sentinel;
+  ReaderErrorCategoryType() {}
+  const char *name() const LLVM_NOEXCEPT override {
+    return "Binary Bitcode Convert Reader";
+  }
+  std::string message(int IndexError) const override {
+    switch (static_cast<ReaderErrorType>(IndexError)) {
+    case MalformedBlock:
+      return "Malformed block found in bitcode";
+    case MalformedIRFile:
+      return "Malformed IR file";
+    case RecordAtTopLevel:
+      return "Record not allowed outside block";
+    case InvalidBitcodeSignature:
+      return "Bitcode has invalid bitcode signature";
+    }
+    llvm_unreachable("Unknown error type!");
+  }
+  ~ReaderErrorCategoryType() override = default;
+};
+
+const ReaderErrorCategoryType ReaderErrorCategoryType::Sentinel;
+
+std::error_code error(ReaderErrorType Error) {
+  return std::error_code(Error, ReaderErrorCategoryType::get());
+}
+
+// Parses binary bitcode records.
+class BinaryRecordParser {
+  BinaryRecordParser(const BinaryRecordParser &) = delete;
+  void operator=(const BinaryRecordParser &) = delete;
+
+public:
+  BinaryRecordParser(StringRef &Input, BitcodeRecordList &Records)
+      : Input(Input), Records(Records) {}
+
+  /// Read in a module block.
+  std::error_code readModule();
+
+private:
+  StringRef &Input;
+  BitcodeRecordList &Records;
+  std::unique_ptr<BitstreamReader> Reader;
+  std::unique_ptr<BitstreamCursor> Cursor;
+
+  // Read block with ID.
+  std::error_code readBlock(unsigned ID);
+};
+
+std::error_code BinaryRecordParser::readModule() {
+  const unsigned char *BufBegin = (const unsigned char *)Input.begin();
+  const unsigned char *BufEnd = (const unsigned char *)Input.end();
+  if (isBitcodeWrapper(BufBegin, BufEnd)) {
+    constexpr bool VerifyBufferSize = true;
+    if (SkipBitcodeWrapperHeader(BufBegin, BufEnd, VerifyBufferSize))
+      return error(MalformedIRFile);
+  }
+
+  Reader = make_unique<BitstreamReader>(BufBegin, BufEnd);
+  Cursor = make_unique<BitstreamCursor>(*Reader);
+
+  // Sniff for the signature.
+  if (Cursor->Read(8) != 'B' || Cursor->Read(8) != 'C' ||
+      Cursor->Read(4) != 0x0 || Cursor->Read(4) != 0xC ||
+      Cursor->Read(4) != 0xE || Cursor->Read(4) != 0xD)
+    return error(InvalidBitcodeSignature);
+
+  while (1) {
+    if (Cursor->AtEndOfStream())
+      return error(MalformedIRFile);
+
+    BitstreamEntry Entry =
+        Cursor->advance(BitstreamCursor::AF_DontAutoprocessAbbrevs);
+
+    if (Entry.Kind != BitstreamEntry::SubBlock)
+      return error(MalformedIRFile);
+    return readBlock(Entry.ID);
+  }
+}
+
+std::error_code BinaryRecordParser::readBlock(unsigned BlockID) {
+  std::unique_ptr<std::vector<uint64_t>> RecordValues =
+      make_unique<std::vector<uint64_t>>();
+  RecordValues->push_back(BLK_CODE_ENTER);
+  RecordValues->push_back(BlockID);
+  Records.push_back(std::move(RecordValues));
+  Cursor->EnterSubBlock(BlockID);
+
+  SmallVector<uint64_t, 32> Values;
+  while (1) {
+    BitstreamEntry Entry = Cursor->advance();
+    switch (Entry.Kind) {
+    case BitstreamEntry::Error:
+      return error(MalformedBlock);
+    case BitstreamEntry::EndBlock: {
+      std::unique_ptr<std::vector<uint64_t>> RecordValues =
+          make_unique<std::vector<uint64_t>>();
+      RecordValues->push_back(BLK_CODE_EXIT);
+      Records.push_back(std::move(RecordValues));
+      return std::error_code();
+    }
+    case BitstreamEntry::SubBlock: {
+      if (Entry.ID == bitc::BLOCKINFO_BLOCK_ID) {
+        if (Cursor->ReadBlockInfoBlock())
+          return error(MalformedBlock);
+        continue;
+      }
+      if (std::error_code EC = readBlock(Entry.ID))
+        return EC;
+      continue;
+    }
+    case BitstreamEntry::Record: {
+      unsigned Code = Cursor->readRecord(Entry.ID, Values);
+      std::unique_ptr<std::vector<uint64_t>> RecordValues =
+          make_unique<std::vector<uint64_t>>();
+      RecordValues->push_back(Code);
+      for (const auto Val : Values)
+        RecordValues->push_back(Val);
+      Records.push_back(std::move(RecordValues));
+      Values.clear();
+      continue;
+    }
+    }
+  }
+  llvm_unreachable("Should not reach here!");
+}
+
+} // end of anonymous namespace
+
+namespace llvm {
+namespace bcconv {
+
+std::error_code readBinaryBitcode(StringRef Input, BitcodeRecordList &Records) {
+  BinaryRecordParser Parser(Input, Records);
+  return Parser.readModule();
+}
+
+} // end of namespace llvm::bcconv
+} // end of namespace llvm

lib/Bitcode/Convert/BinaryBitcodeWriter.cpp

+//===- BinaryBitcodeWriter.cpp - Writes binary bitcode from records -------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// Implements the binary bitcode writer when given bitcode records.
+//
+//===----------------------------------------------------------------------===//
+
+#include "BitcodeWriter.h"
+
+#include "llvm/Bitcode/BitstreamWriter.h"
+#include <climits>
+
+using namespace llvm;
+using namespace llvm::bcconv;
+
+namespace {
+
+// Writer of binary bitcode records to a byte buffer.
+class BinaryRecordWriter : public BitcodeWriter {
+  BinaryRecordWriter(const BinaryRecordWriter &) = delete;
+  void operator=(const BinaryRecordWriter &) = delete;
+
+public:
+  BinaryRecordWriter(OutputBuffer &Buffer, std::string &ErrorMessages,
+                     bool ErrorRecover)
+      : BitcodeWriter(ErrorMessages, ErrorRecover), Writer(Buffer) {
+    writeHeader();
+  }
+
+  ~BinaryRecordWriter() final = default;
+
+private:
+  BitstreamWriter Writer;
+
+  void writeHeader() {
+    Writer.Emit((unsigned)'B', 8);
+    Writer.Emit((unsigned)'C', 8);
+    Writer.Emit(0x0, 4);
+    Writer.Emit(0xC, 4);
+    Writer.Emit(0xE, 4);
+    Writer.Emit(0xD, 4);
+  }
+
+  void writeEnterBlock(uint64_t ID) final {
+    BitcodeWriter::writeEnterBlock(ID);
+    Writer.EnterSubblock(ID, 2);
+  }
+
+  void writeExitBlock() final {
+    BitcodeWriter::writeExitBlock();
+    Writer.ExitBlock();
+  }
+
+  void writeRecord(const BitcodeRecord &Record) final {
+    unsigned Code = Record[0];
+    SmallVector<uint64_t, 6> Values;
+    for (size_t i = 1; i < Record.size(); ++i)
+      Values.push_back(Record[i]);
+    Writer.EmitRecord(Code, Values);
+  }
+
+  bool finish() final;
+};
+
+bool BinaryRecordWriter::finish() {
+  bool Success = true;
+  // First try to finish gracefully.
+  if (!BitcodeWriter::finish())
+    Success = false;
+
+  // Force out exit blocks, if needed, to avoid the bitstream writer from assert
+  // failing.
+  while (!atOutermostScope())
+    writeExitBlock();
+
+  // Make sure file length divisible by 4.
+  uint64_t TrailingBits = Writer.GetCurrentBitNo() % 4 * CHAR_BIT;
+  if (TrailingBits) {
+    recoverableError() << "Generated bitstream not word aligned\n";
+    Writer.Emit(0, TrailingBits);
+    if (!ErrorRecover)
+      return false;
+  }
+  return Success;
+}
+
+} // end of anonymous namespace
+
+namespace llvm {
+namespace bcconv {
+
+bool writeBinaryBitcode(OutputBuffer &Buffer, const BitcodeRecordList &Records,
+                        std::string &ErrorMessages, bool ErrorRecover) {
+  BinaryRecordWriter RecordWriter(Buffer, ErrorMessages, ErrorRecover);
+  return RecordWriter.writeBitcodeRecords(Records);
+}
+
+} // end of namespace llvm::bcconv
+} // end of namespace llvm

lib/Bitcode/Convert/BitcodeWriter.h

+//===- BitcodeWriter.h - Driver for bitcode writers -------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// Defines the driver class for writing bitcode records into a character buffer.
+// Used to write to binary, textual, and simplfied forms.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Bitcode/BitcodeConvert.h"
+#include "llvm/Support/raw_ostream.h"
+
+namespace llvm {
+
+class raw_ostream;
+
+namespace bcconv {
+
+// Abstract bitcode record writer.
+class BitcodeWriter {
+  BitcodeWriter(const BitcodeWriter &) = delete;
+  void operator=(const BitcodeWriter &) = delete;
+
+public:
+  virtual ~BitcodeWriter();
+
+  // Writes out Records.
+  bool writeBitcodeRecords(const BitcodeRecordList &Records);
+
+protected:
+  // The block ID associated with records not in any block.
+  static const unsigned UnknownBlockID = std::numeric_limits<uint32_t>::max();
+  // Holds error messages
+  std::string ErrorMessages;
+  // Stream used to write error messages into ErrorMessages;
+  raw_string_ostream ErrStream;
+  // True if error recovery should be applied.
+  bool ErrorRecover;
+  // Nest level of blocks within bitcode being written.
+  size_t BlockDepth = 0;
+  // The number of errors found.
+  size_t NumErrors = 0;
+  // The number of error repairs applied.
+  size_t NumRepairs = 0;
+
+  explicit BitcodeWriter(std::string &ErrorMessages, bool ErrorRecover)
+      : ErrorMessages(ErrorMessages), ErrStream(ErrorMessages),
+        ErrorRecover(ErrorRecover) {}
+
+  // Records that an error has occurred, and returns stream to print the
+  // corresponding error message to.
+  raw_ostream &error();
+
+  // Records that a recoverable error has occurred, and returns the stream to
+  // print the corresponding error message to.
+  raw_ostream &recoverableError();
+
+  bool isSuccessful() const {
+    return NumErrors == 0 || (ErrorRecover && NumErrors == NumRepairs);
+  }
+
+  bool atOutermostScope() { return BlockDepth == 0; }
+
+  // Emit the contents of Record.
+  bool emitRecord(const BitcodeRecord &Record);
+
+  // Write out the enter record for block ID.
+  virtual void writeEnterBlock(uint64_t ID) {
+    (void)ID;
+    ++BlockDepth;
+  }
+
+  // Write out the contents of the Record in the current block.
+  virtual void writeRecord(const BitcodeRecord &Record) = 0;
+
+  // Write out the exit record for the current block.
+  virtual void writeExitBlock() { --BlockDepth; }
+
+  // Apply any necessary cleanups after writing bitcode.
+  virtual bool finish();
+};
+
+} // end of namespace llvm::bcconv
+} // end of namespace llvm

lib/Bitcode/Convert/BitcodeWriter.cpp

+//===- BitcodeWriter.cpp - Driver for bitcode writers ---------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// Implements the driver class for writing bitcode records into a character
+// buffer.  Used to write to binary, textual, and simplfied forms.
+//
+//===----------------------------------------------------------------------===//
+
+#include "BitcodeWriter.h"
+
+#include "llvm/Bitcode/BitCodes.h"
+#include "llvm/Support/raw_ostream.h"
+
+namespace llvm {
+namespace bcconv {
+
+BitcodeWriter::~BitcodeWriter() {
+  // Force error messages into field ErrorMessages.
+  ErrStream.flush();
+}
+
+raw_ostream &BitcodeWriter::error() {
+  ++NumErrors;
+  return ErrStream;
+}
+
+raw_ostream &BitcodeWriter::recoverableError() {
+  if (ErrorRecover)
+    ++NumRepairs;
+  return error();
+}
+
+bool BitcodeWriter::finish() {
+  if (atOutermostScope())
+    return true;
+
+  recoverableError() << "Missing " << BlockDepth << " close blocks.\n";
+  if (!ErrorRecover)
+    return false;
+
+  while (!atOutermostScope())
+    writeExitBlock();
+  return true;
+}
+
+bool BitcodeWriter::writeBitcodeRecords(const BitcodeRecordList &Records) {
+  for (const std::unique_ptr<BitcodeRecord> &Rec : Records) {
+    if (!emitRecord(*Rec))
+      return false;
+  }
+  if (!finish())
+    return false;
+  return isSuccessful();
+}
+
+bool BitcodeWriter::emitRecord(const BitcodeRecord &Record) {
+  size_t NumValues = Record.size();
+  if (NumValues == 0) {
+    recoverableError() << "Empty records not allowed in bitcode\n";
+    return ErrorRecover;
+  }
+  switch (Record[0]) {
+  case BLK_CODE_ENTER: {
+    uint64_t WriteBlockID = UnknownBlockID;
+    if (NumValues != 2) {
+      recoverableError() << "Block enter record not size 2\n";
+      if (!ErrorRecover)
+        return false;
+    }
+    if (NumValues > 1)
+      WriteBlockID = Record[1];
+    if (WriteBlockID == bitc::BLOCKINFO_BLOCK_ID) {
+      recoverableError() << "BlockInfo block not allowed in bitcode records\n";
+      if (!ErrorRecover)
+        return false;
+      WriteBlockID = UnknownBlockID;
+    }
+    writeEnterBlock(WriteBlockID);
+    return true;
+  }
+  case BLK_CODE_EXIT: {
+    if (atOutermostScope()) {
+      recoverableError() << "Extraneous exit block\n";
+      return ErrorRecover;
+    }
+    writeExitBlock();
+    return true;
+  }
+  default: {
+    if (atOutermostScope()) {
+      recoverableError() << "Record not allowed outside block\n";
+      if (!ErrorRecover)
+        return false;
+      writeEnterBlock(UnknownBlockID);
+    }
+    writeRecord(Record);
+    return true;
+  }
+  }
+}
+
+} // end of namespace llvm::bcconv
+} // end of namespace llvm

lib/Bitcode/Convert/CMakeLists.txt

+add_llvm_library(LLVMBitcodeConv
+  BinaryBitcodeReader.cpp
+  BinaryBitcodeWriter.cpp
+  BitcodeWriter.cpp
+  SimplifiedBitcodeReader.cpp
+  SimplifiedBitcodeWriter.cpp
+  TextualBitcodeReader.cpp
+  TextualBitcodeWriter.cpp
+
+  DEPENDS
+  intrinsics_gen
+  LLVMBitReader 
+  )

lib/Bitcode/Convert/LLVMBuild.txt

-;===- ./lib/Bitcode/LLVMBuild.txt ------------------------------*- Conf -*--===;
+;===- ./lib/Bitcode/Convert/LLVMBuild.txt ----------------------*- Conf -*--===;
 ;
 ;                     The LLVM Compiler Infrastructure
 ;
 ; This file is distributed under the University of Illinois Open Source
 ; License. See LICENSE.TXT for details.
 ;
 ;===------------------------------------------------------------------------===;
 ;
 ; This is an LLVMBuild description file for the components in this subdirectory.
 ;
 ; For more information on the LLVMBuild system, please see:
 ;
 ;   http://llvm.org/docs/LLVMBuild.html
 ;
 ;===------------------------------------------------------------------------===;
 
-[common]
-subdirectories = Reader Writer
-
 [component_0]
-type = Group
-name = Bitcode
-parent = Libraries
+type = Library
+name = BitcodeConv
+parent = Bitcode
+required_libraries = Core Support

lib/Bitcode/Convert/Makefile

-##===- lib/Bitcode/Makefile --------------------------------*- Makefile -*-===##
+##===- lib/Bitcode/Convert/Makefile ------------------------*- Makefile -*-===##
 #
 #                     The LLVM Compiler Infrastructure
 #
 # This file is distributed under the University of Illinois Open Source
 # License. See LICENSE.TXT for details.
 #
 ##===----------------------------------------------------------------------===##
 
-LEVEL = ../..
-PARALLEL_DIRS = Reader Writer
+LEVEL = ../../..
+LIBRARYNAME = LLVMBitcodeConv
+BUILD_ARCHIVE = 1
 
 include $(LEVEL)/Makefile.common
-

lib/Bitcode/Convert/SimplifiedBitcodeReader.cpp

+//===- TextualBitcodeReader.cpp -------------------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// Implements a reader that takes the the textual form of bitcode records, and
+// generates the corresponding sequence of bitcode records.
+//
+// ===---------------------------------------------------------------------===//
+
+#include "llvm/Bitcode/BitcodeConvert.h"
+
+#include "llvm/ADT/STLExtras.h"
+
+using namespace llvm;
+using namespace llvm::bcconv;
+
+namespace {
+
+enum ReaderErrorType { NoRecordCode = 1, MalformedInput };
+
+// Defines reader error codes.
+enum SimplifiedActionKind {
+  AcceptAsIsKind,
+  AcceptValueKind,
+  AcceptRecordKind,
+  AcceptEscapeKind
+};
+
+// Defines the corresponding error messages.
+class ReaderErrorCategoryType : public std::error_category {
+  ReaderErrorCategoryType(const ReaderErrorCategoryType &) = delete;
+  void operator=(const ReaderErrorCategoryType &) = delete;
+
+public:
+  static const ReaderErrorCategoryType &get() { return Sentinel; }
+
+private:
+  static const ReaderErrorCategoryType Sentinel;
+  ReaderErrorCategoryType() {}
+  const char *name() const LLVM_NOEXCEPT override {
+    return "Simplified Bitcode Convert Reader";
+  }
+  std::string message(int IndexError) const override {
+    switch (static_cast<ReaderErrorType>(IndexError)) {
+    case NoRecordCode:
+      return "No record code for bitcode record";
+    case MalformedInput:
+      return "Malformed input. Doesn't end with accept record byte.";
+    }
+    llvm_unreachable("Unknown error type!");
+  }
+  ~ReaderErrorCategoryType() override = default;
+};
+
+const ReaderErrorCategoryType ReaderErrorCategoryType::Sentinel;
+
+static const size_t NumByteValues =
+    std::numeric_limits<uint8_t>::max() + (size_t)1;
+
+class SimplifiedBitcodeReader {
+public:
+  SimplifiedBitcodeReader(StringRef &Input, BitcodeRecordList &Records,
+                          const SimplifiedBitcodeActions &Actions)
+      : Input(Input), Records(Records), Actions(Actions) {
+    init();
+  }
+
+  std::error_code read() {
+    std::vector<uint64_t> Values;
+    int64_t Number = 0;
+    bool FoundNumber = false;
+    while (Cursor < Input.size()) {
+      uint8_t Byte = Input[Cursor++];
+      switch (ByteAction[Byte]) {
+      case AcceptEscapeKind:
+        if (Cursor == Input.size())
+          return std::error_code(MalformedInput,
+                                 ReaderErrorCategoryType::get());
+        Byte = Input[Cursor++];
+      // Intentionally drop to the next case.
+      case AcceptAsIsKind:
+        Number = (Number << 8) + Byte;
+        FoundNumber = true;
+        break;
+      case AcceptValueKind:
+        Values.push_back(signRotatedValue(Number));
+        Number = 0;
+        FoundNumber = false;
+        break;
+      case AcceptRecordKind:
+        if (Values.empty())
+          return std::error_code(NoRecordCode, ReaderErrorCategoryType::get());
+        Records.push_back(make_unique<BitcodeRecord>(Values));
+        Values.clear();
+        Number = 0;
+        FoundNumber = false;
+        break;
+      }
+    }
+    // Add value/record terminators if applicable.
+    if (FoundNumber)
+      Values.push_back(signRotatedValue(Number));
+
+    if (!Values.empty())
+      Records.push_back(make_unique<BitcodeRecord>(Values));
+    return std::error_code();
+  }
+
+private:
+  // The input to parse.
+  StringRef &Input;
+  // The list of bitcode records to generate.
+  BitcodeRecordList &Records;
+  const SimplifiedBitcodeActions &Actions;
+  // The current location within the input.
+  size_t Cursor = 0;
+  SimplifiedActionKind ByteAction[NumByteValues];
+
+  void init() {
+    for (size_t i = 0; i < NumByteValues; ++i)
+      ByteAction[i] = AcceptAsIsKind;
+    ByteAction[Actions.AcceptValue] = AcceptValueKind;
+    ByteAction[Actions.AcceptRecord] = AcceptRecordKind;
+    ByteAction[Actions.AcceptEscape] = AcceptEscapeKind;
+  }
+
+  // signed value stored with the sign bit in the LSB for dense
+  // encoding.
+  uint64_t signRotatedValue(uint64_t V) {
+    if ((V & 1) == 0)
+      return V >> 1;
+    if (V != 1)
+      return -(V >> 1);
+    // There is no such thing as -0 with integers.  "-0" really means MININT.
+    return 1ULL << 63;
+  }
+};
+
+} // end of anonymous namespace
+
+namespace llvm {
+namespace bcconv {
+
+const uint8_t SimplifiedBitcodeActions::AcceptValueDefault = 0xAA;
+const uint8_t SimplifiedBitcodeActions::AcceptRecordDefault = 0xBA;
+const uint8_t SimplifiedBitcodeActions::AcceptEscapeDefault = 0xBB;
+
+const SimplifiedBitcodeActions SimplifiedBitcodeActions::Default;
+
+SimplifiedBitcodeActions::SimplifiedBitcodeActions() {}
+
+std::error_code readSimplifiedBitcode(StringRef Input,
+                                      BitcodeRecordList &Records,
+                                      const SimplifiedBitcodeActions &Actions) {
+  SimplifiedBitcodeReader Reader(Input, Records, Actions);
+  return Reader.read();
+}
+
+} // end of namespace llvm::bcconv
+} // end of namespace llvm

lib/Bitcode/Convert/SimplifiedBitcodeWriter.cpp

+//===- SimplifiedBitcodeWriter.cpp - Writes simp bitcode from records -----===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// Implements the simplified bitcode writer when given bitcode records.
+//
+//===----------------------------------------------------------------------===//
+
+#include "BitcodeWriter.h"
+#include <climits>
+
+using namespace llvm;
+using namespace llvm::bcconv;
+
+namespace {
+
+class SimplifiedBitcodeWriter : public BitcodeWriter {
+  SimplifiedBitcodeWriter(const SimplifiedBitcodeWriter &) = delete;
+  void operator=(const SimplifiedBitcodeWriter &) = delete;
+
+public:
+  SimplifiedBitcodeWriter(OutputBuffer &Buffer,
+                          const SimplifiedBitcodeActions &Actions,
+                          std::string &ErrorMessages, bool ErrorRecover)
+      : BitcodeWriter(ErrorMessages, ErrorRecover), Buffer(Buffer),
+        Actions(Actions) {
+    init();
+  }
+
+  ~SimplifiedBitcodeWriter() final = default;
+
+private:
+  // Buffer to write textual bitcode records into.
+  OutputBuffer &Buffer;
+  const SimplifiedBitcodeActions &Actions;
+  // Defines what byte values need to be prefixed with an escape action.
+  static constexpr size_t NumByteValues = 1 << CHAR_BIT;
+  bool NeedsEscape[NumByteValues];
+
+  void init() {
+    assert(std::numeric_limits<uint8_t>::max() < NumByteValues);
+    for (size_t i = 0; i < NumByteValues; ++i)
+      NeedsEscape[i] = false;
+    NeedsEscape[Actions.AcceptValue] = true;
+    NeedsEscape[Actions.AcceptRecord] = true;
+    NeedsEscape[Actions.AcceptEscape] = true;
+  }
+
+  void writeValue(uint64_t Value);
+
+  void writeTerminator() { Buffer.push_back(Actions.AcceptRecord); }
+
+  void writeEnterBlock(uint64_t ID) final {
+    BitcodeWriter::writeEnterBlock(ID);
+    writeValue(BLK_CODE_ENTER);
+    writeValue(ID);
+    writeTerminator();
+  }
+
+  void writeExitBlock() final {
+    BitcodeWriter::writeExitBlock();
+    writeValue(BLK_CODE_EXIT);
+    writeTerminator();
+  }
+
+  void writeRecord(const BitcodeRecord &Record) final {
+    for (const auto Value : Record)
+      writeValue(Value);
+    writeTerminator();
+  }
+
+  // signed value stored with the sign bit moved to the LSB for dense
+  // encoding.
+  uint64_t rotateSign(uint64_t U) {
+    int64_t I = U;
+    return I < 0 ? ~(U << 1) : U << 1;
+  }
+};
+
+static const int LeftmostShift = 7 * CHAR_BIT;
+static const uint64_t LeftmostByteMask = ((uint64_t)0xFF) << LeftmostShift;
+
+void SimplifiedBitcodeWriter::writeValue(uint64_t Value) {
+  Value = rotateSign(Value);
+  bool WroteByte = false;
+  int Shift = LeftmostShift;
+  uint64_t Mask = LeftmostByteMask;
+  while (Mask) {
+    uint8_t Byte = (Value & Mask) >> Shift;
+    if (Byte || WroteByte) {
+      if (NeedsEscape[Byte])
+        Buffer.push_back(Actions.AcceptEscape);
+      Buffer.push_back(Byte);
+      WroteByte = true;
+    }
+    Mask >>= CHAR_BIT;
+    Shift -= CHAR_BIT;
+  }
+  Buffer.push_back(Actions.AcceptValue);
+}
+
+} // end of anonymous namespace
+
+namespace llvm {
+namespace bcconv {
+
+bool writeSimplifiedBitcode(OutputBuffer &Output,
+                            const BitcodeRecordList &Records,
+                            const SimplifiedBitcodeActions &Actions,
+                            std::string &ErrorMessages, bool ErrorRecover) {
+  SimplifiedBitcodeWriter Writer(Output, Actions, ErrorMessages, ErrorRecover);
+  return Writer.writeBitcodeRecords(Records);
+}
+
+} // end of namespace llvm::bcconv
+} // end of namespace llvm

lib/Bitcode/Convert/TextualBitcodeReader.cpp

+//===- TextualBitcodeReader.cpp - Reads textual bitcode -------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// Implements the textual bitcode reader that builds the corresponding list of
+// bitcode records.
+//
+// ===---------------------------------------------------------------------===//
+
+#include "llvm/Bitcode/BitcodeConvert.h"
+
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/Bitcode/BitstreamReader.h"
+#include "llvm/Bitcode/ReaderWriter.h"
+
+#include "llvm/Support/raw_ostream.h"
+
+using namespace llvm;
+using namespace llvm::bcconv;
+
+namespace {
+
+// Defines reader error codes.
+enum ReaderErrorType {
+  NoCodeForRecord = 1,
+  NoValueAfterSeparator,
+  NoSeparatorOrTerminator,
+  NoNewlineAfterTerminator
+};
+
+// Defines the corresponding error messages.
+class ReaderErrorCategoryType : public std::error_category {
+  ReaderErrorCategoryType(const ReaderErrorCategoryType &) = delete;
+  void operator=(const ReaderErrorCategoryType &) = delete;
+
+public:
+  static const ReaderErrorCategoryType &get() { return Sentinel; }
+
+private:
+  static const ReaderErrorCategoryType Sentinel;
+  ReaderErrorCategoryType() {}
+  const char *name() const LLVM_NOEXCEPT override {
+    return "Textual Bitcode Convert Reader";
+  }
+  std::string message(int IndexError) const override {
+    switch (static_cast<ReaderErrorType>(IndexError)) {
+    case NoCodeForRecord:
+      return "Bitcode record doesn't begin with a record code";
+    case NoValueAfterSeparator:
+      return "Value expected after separator, but not found";
+    case NoSeparatorOrTerminator:
+      return "Separator/terminator expected after value";
+    case NoNewlineAfterTerminator:
+      return "Newline expecded after terminating semicolon";
+    }
+    llvm_unreachable("Unknown error type!");
+  }
+  ~ReaderErrorCategoryType() override = default;
+};
+
+const ReaderErrorCategoryType ReaderErrorCategoryType::Sentinel;
+
+std::error_code error(ReaderErrorType Error) {
+  return std::error_code(Error, ReaderErrorCategoryType::get());
+}
+
+/// Parses text bitcode records, putting them in Records.
+class TextRecordParser {
+  TextRecordParser(const TextRecordParser &) = delete;
+  void operator=(const TextRecordParser &) = delete;
+
+public:
+  /// Creates a parser to parse records from the Input, and put them into
+  /// Records. If AllowComments is true, relax the input rules to allow comments
+  /// as well.
+  TextRecordParser(StringRef &Input, BitcodeRecordList &Records,
+                   bool AllowComments)
+      : Input(Input), Records(Records), AllowComments(AllowComments) {}
+
+  /// Reads in the list of bitcode records in the input buffer.
+  std::error_code read();
+
+private:
+  // The input to parse.
+  StringRef &Input;
+  // The list of bitcode records to generate.
+  BitcodeRecordList &Records;
+  // Allow comments in bitcode file.
+  bool AllowComments;
+  // The current location within the input.
+  size_t Cursor = 0;
+  // The separator character.
+  static const char *Separator;
+  // The terminator character.
+  static const char *Terminator;
+  // The newline character that must follow a terminator.
+  static const char *Newline;
+  // Valid digits that can be used to define numbers.
+  static const char *Digits;
+
+  // Returns true if we have reached the end of the input.
+  bool atEof() const { return Cursor == Input.size(); }
+
+  // Tries to read a character in the given set of Characters. Returns the
+  // character found, or 0 if not found. If a character is not found, it will
+  // not advance the cursor.
+  char readChar(const char *Characters);
+
+  // Tries to read a (integral) number. If successful, Value is set to
+  // the parsed number and returns true. Otherwise false is returned.
+  // Does not check for number overflow.
+  bool readNumber(uint64_t &Value);
+
+  // Skip empty lines, and lines beginning with a semicolin.
+  bool skipWhitespaceLine();
+
+  // Skip rest of line.
+  void skipComment() {
+    while (!atEof()) {
+      char Ch = Input[Cursor++];
+      if (Ch == *Newline)
+        return;
+    }
+  }
+
+  // Reads a record from the input.
+  std::error_code readRecord();
+};
+
+const char *TextRecordParser::Newline = "\n";
+const char *TextRecordParser::Separator = ",";
+const char *TextRecordParser::Terminator = ";";
+const char *TextRecordParser::Digits = "0123456789";
+
+std::error_code TextRecordParser::read() {
+  while (!atEof()) {
+    if (AllowComments)
+      while (skipWhitespaceLine())
+        ;
+    if (std::error_code EC = readRecord())
+      return EC;
+  }
+  return std::error_code();
+}
+
+bool TextRecordParser::skipWhitespaceLine() {
+  if (readChar(Newline))
+    return true;
+  if (readChar(Terminator)) {
+    skipComment();
+    return true;
+  }
+  return false;
+}
+
+char TextRecordParser::readChar(const char *Characters) {
+  if (atEof())
+    return 0;
+  char Ch = Input[Cursor];
+  if (std::strchr(Characters, Ch) == 0)
+    return 0;
+  ++Cursor;
+  return Ch;
+}
+
+bool TextRecordParser::readNumber(uint64_t &Value) {
+  Value = 0;
+  bool NumberFound = false;
+  while (1) {
+    char Ch = readChar(Digits);
+    if (!Ch)
+      return NumberFound;
+    Value = (Value * 10) + (Ch - '0');
+    NumberFound = true;
+  }
+}
+
+std::error_code TextRecordParser::readRecord() {
+  // States of parser used to parse bitcode records.
+  enum ParseState {
+    // Begin parsing a record.
+    StartParse,
+    // Before a value in the record.
+    BeforeValue,
+    // Immediately after a value in the record.
+    AfterValue
+  } State = StartParse;
+  std::vector<uint64_t> Values;
+  uint64_t Number = 0;
+  while (1) {
+    switch (State) {
+    case StartParse:
+      if (!readNumber(Number)) {
+        if (atEof())
+          return std::error_code();
+        return error(NoCodeForRecord);
+      }
+      Values.push_back(Number);
+      State = AfterValue;
+      continue;
+    case BeforeValue:
+      if (!readNumber(Number))
+        return error(NoValueAfterSeparator);
+      Values.push_back(Number);
+      State = AfterValue;
+      continue;
+    case AfterValue:
+      if (readChar(Separator)) {
+        State = BeforeValue;
+        continue;
+      }
+      if (readChar(Terminator)) {
+        if (AllowComments)
+          skipComment();
+        else if (!readChar(Newline))
+          return error(NoNewlineAfterTerminator);
+        Records.push_back(make_unique<BitcodeRecord>(Values));
+        return std::error_code();
+      }
+      return error(NoSeparatorOrTerminator);
+    }
+  }
+}
+
+} // end of anonymous namespace
+
+namespace llvm {
+namespace bcconv {
+
+std::error_code readTextualBitcode(StringRef Input, BitcodeRecordList &Records,
+                                   bool AllowComments) {
+  TextRecordParser Reader(Input, Records, AllowComments);
+  return Reader.read();
+}
+
+} // end of namespace llvm::bcconv
+} // end of namespace llvm

lib/Bitcode/Convert/TextualBitcodeWriter.cpp

+//===- TextualBitcodeWriter.cpp - Writes textual bitcode from records -----===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// Implements the textual bitcode writer when given bitcode records.
+//
+//===----------------------------------------------------------------------===//
+
+#include "BitcodeWriter.h"
+
+#include "llvm/Support/raw_ostream.h"
+
+using namespace llvm;
+using namespace llvm::bcconv;
+
+namespace {
+
+// Writer of textual bitcode records to a byte buffer.
+class TextRecordWriter : public BitcodeWriter {
+  TextRecordWriter(const TextRecordWriter &) = delete;
+  void operator=(const TextRecordWriter &) = delete;
+
+public:
+  TextRecordWriter(OutputBuffer &Buffer, std::string &ErrorMessages,
+                   bool ErrorRecover)
+      : BitcodeWriter(ErrorMessages, ErrorRecover), Buffer(Buffer),
+        ValueStream(ValueBuffer) {}
+
+  ~TextRecordWriter() final = default;
+
+private:
+  // Buffer to write textual bitcode records into.
+  OutputBuffer &Buffer;
+  // String stream to convert integers to strings.
+  std::string ValueBuffer;
+  raw_string_ostream ValueStream;
+
+  void writeValue(uint64_t Value);
+
+  void writeSeparator() { Buffer.push_back(','); }
+
+  void writeTerminator() {
+    Buffer.push_back(';');
+    Buffer.push_back('\n');
+  }
+
+  void writeEnterBlock(uint64_t ID) final {
+    BitcodeWriter::writeEnterBlock(ID);
+    writeValue(BLK_CODE_ENTER);
+    writeSeparator();
+    writeValue(ID);
+    writeTerminator();
+  }
+
+  void writeExitBlock() final {
+    BitcodeWriter::writeExitBlock();
+    writeValue(BLK_CODE_EXIT);
+    writeTerminator();
+  }
+
+  void writeRecord(const BitcodeRecord &Record) final {
+    bool IsFirst = true;
+    for (const auto Value : Record) {
+      if (IsFirst)
+        IsFirst = false;
+      else
+        writeSeparator();
+      writeValue(Value);
+    }
+    writeTerminator();
+  }
+};
+
+void TextRecordWriter::writeValue(uint64_t Value) {
+  ValueStream << Value;
+  ValueStream.flush();
+  for (auto ch : ValueBuffer)
+    Buffer.push_back(ch);
+  ValueBuffer.clear();
+}
+
+} // end of anonymous namespace
+
+namespace llvm {
+namespace bcconv {
+
+bool writeTextualBitcode(OutputBuffer &Buffer, const BitcodeRecordList &Records,
+                         std::string &ErrorMessages, bool ErrorRecover) {
+  TextRecordWriter RecordWriter(Buffer, ErrorMessages, ErrorRecover);
+  return RecordWriter.writeBitcodeRecords(Records);
+}
+
+} // end of namespace llvm::bcconv
+} // end of namespace llvm

lib/Bitcode/LLVMBuild.txt

 ;
 ; For more information on the LLVMBuild system, please see:
 ;
 ;   http://llvm.org/docs/LLVMBuild.html
 ;
 ;===------------------------------------------------------------------------===;
 
 [common]
-subdirectories = Reader Writer
+subdirectories = Convert Reader Writer
 
 [component_0]
 type = Group
 name = Bitcode
 parent = Libraries

lib/Bitcode/Makefile

 ##===- lib/Bitcode/Makefile --------------------------------*- Makefile -*-===##
 #
 #                     The LLVM Compiler Infrastructure
 #
 # This file is distributed under the University of Illinois Open Source
 # License. See LICENSE.TXT for details.
 #
 ##===----------------------------------------------------------------------===##
 
 LEVEL = ../..
-PARALLEL_DIRS = Reader Writer
+PARALLEL_DIRS = Convert Reader Writer
 
 include $(LEVEL)/Makefile.common
 

test/Bitcode/text-bitcode.ll

+; Tests that we can convert LLVM bitcode to textual form, and back to binary.
+
+; RUN: llvm-as %s -o - | llvm-bcconv -from-binary -to-text \
+; RUN:                 | FileCheck --check-prefix=TBC %s
+
+; RUN: llvm-as %s -o - | llvm-bcconv -from-binary -to-text \
+; RUN:                 | llvm-bcconv -from-text -to-binary \
+; RUN:                 | llvm-dis | FileCheck %s
+
+; RUN: llvm-as %s -o - | llvm-bcconv -from-binary -to-simplified \
+; RUN:                 | llvm-bcconv -from-simplified -to-binary \
+; RUN:                 | llvm-dis | FileCheck %s
+
+define void @f() {
+  ret void
+}
+
+; CHECK:      define void @f() {
+; CHECK-NEXT:   ret void
+; CHECK-NEXT: }
+
+; TBC:      65535,8;
+; TBC-NEXT: 1,1;
+; TBC-NEXT: 65535,17;
+; TBC-NEXT: 1,4;
+; TBC-NEXT: 2;
+; TBC-NEXT: 21,0,0;
+; TBC-NEXT: 8,1,0;
+; TBC-NEXT: 16;          
+; TBC-NEXT: 65534;
+; TBC-NEXT: 8,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0;
+; TBC-NEXT: 65535,15;
+;           Intentionally skip contents of metadata block.
+; TBC:      65534;
+; TBC-NEXT: 65535,14;
+; TBC-NEXT: 1,0,102;
+; TBC-NEXT: 65534;
+; TBC-NEXT: 65535,12;
+; TBC-NEXT: 1,1;
+; TBC-NEXT: 10;
+; TBC-NEXT: 65534;
+; TBC-NEXT: 65534;

test/Convert/MyMain.cpp

+#include "llvm/ADT/StringRef.h"
+#include "llvm/Bitcode/BitcodeConvert.h"
+#include "llvm/Bitcode/ReaderWriter.h"
+#include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/Module.h"
+#include "llvm/Support/ErrorHandling.h"
+#include "llvm/Support/raw_ostream.h"
+
+extern "C" {
+#include <setjmp.h>
+}
+
+using namespace llvm;
+using namespace llvm::bcconv;
+
+static jmp_buf JmpBuf;
+
+namespace {
+
+void MyBitcodeDiagnosticHandler(const DiagnosticInfo&Info) {
+  // Do nothing, so that parser returns error code.
+}
+
+void MyFatalErrorHandler(void *user_data, const std::string& reason,
+                         bool gen_crash_diag) {
+  // Don't bother printing reason, just return to the test function,
+  // since a fatal error represents a successful parse (i.e. it correctly
+  // terminated with an error message to the user).
+  longjmp(JmpBuf, 1);
+}
+
+static bool InstalledHandler = false;
+
+} // end of anonymous namespace
+
+static size_t TestCount = 0;
+static size_t TextReadable = 0;
+static size_t BinaryWritable = 0;
+static size_t Interesting = 0;
+
+extern "C" void LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
+  if (TestCount % 1000000 == 0)
+    errs() << "Ran " << TestCount << " tests, "
+           << TextReadable << " text readable, "
+           << BinaryWritable << " binary writable, "
+           << Interesting << " interesting!\n";
+
+  ++TestCount;
+
+  // Allocate space for locals before setjmp so that memory can be collected
+  // if parse exits prematurely.
+  StringRef Input((const char *)Data, Size);
+  BitcodeRecordList Records;
+  SimplifiedBitcodeActions Actions;
+  SmallVector<char, 1024> Binary;
+  std::string ErrorMessages;
+
+  if (setjmp(JmpBuf))
+    // If reached, we have returned with non-zero status, so exit.
+    return;
+
+  if (!InstalledHandler) {
+    llvm::install_fatal_error_handler(::MyFatalErrorHandler, nullptr);
+    InstalledHandler = true;
+  }
+
+  // convert Data to bitcode records.
+  if (readSimplifiedBitcode(Input, Records, Actions))
+    return;
+
+  ++TextReadable;
+
+  // Convert bitcode records to binary bitcode representation.
+  if (!writeBinaryBitcode(Binary, Records, ErrorMessages, true))
+    return;
+
+  ++BinaryWritable;
+
+  // read generated binary bitcode to generate module.
+  StringRef RealInput(Binary.data(), Binary.size());
+  ErrorOr<std::unique_ptr<Module>> Result =
+      getLazyBitcodeModule(MemoryBuffer::getMemBuffer(RealInput, "", false),
+                           getGlobalContext(),
+                           (DiagnosticHandlerFunction)::MyBitcodeDiagnosticHandler);
+
+  if (!Result.getError())
+    ++Interesting;
+}

test/Convert/crash1.err

+$ llvm-dis -disable-output crash1.bc
+terminate called after throwing an instance of 'std::length_error'
+  what():  vector::_M_default_append
+#0 0x5eeeac llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x5eeeac)
+#1 0x5ef1c1 PrintStackTraceSignalHandler(void*) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x5ef1c1)
+#2 0x5ed93d llvm::sys::RunSignalHandlers() (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x5ed93d)
+#3 0x5ede14 SignalHandler(int) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x5ede14)
+#4 0x7f97dc373340 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x10340)
+#5 0x7f97db7b4cc9 gsignal /build/buildd/eglibc-2.19/signal/../nptl/sysdeps/unix/sysv/linux/raise.c:56:0
+#6 0x7f97db7b80d8 abort /build/buildd/eglibc-2.19/stdlib/abort.c:91:0
+#7 0x7f97dc0bf535 __gnu_cxx::__verbose_terminate_handler() (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0x60535)
+#8 0x7f97dc0bd6d6 (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0x5e6d6)
+#9 0x7f97dc0bd703 (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0x5e703)
+#10 0x7f97dc0bd922 (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0x5e922)
+#11 0x7f97dc10f387 std::__throw_length_error(char const*) (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0xb0387)
+#12 0x441322 std::vector<llvm::Type*, std::allocator<llvm::Type*> >::_M_check_len(unsigned long, char const*) const (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x441322)
+#13 0x439466 std::vector<llvm::Type*, std::allocator<llvm::Type*> >::_M_default_append(unsigned long) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x439466)
+#14 0x430745 std::vector<llvm::Type*, std::allocator<llvm::Type*> >::resize(unsigned long) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x430745)
+#15 0x41474a (anonymous namespace)::BitcodeReader::parseTypeTableBody() (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x41474a)
+#16 0x414445 (anonymous namespace)::BitcodeReader::parseTypeTable() (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x414445)
+#17 0x420138 (anonymous namespace)::BitcodeReader::parseModule(bool, bool) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x420138)
+#18 0x4225af (anonymous namespace)::BitcodeReader::parseBitcodeInto(std::unique_ptr<llvm::DataStreamer, std::default_delete<llvm::DataStreamer> >, llvm::Module*, bool) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x4225af)
+#19 0x42c3dc getBitcodeModuleImpl(std::unique_ptr<llvm::DataStreamer, std::default_delete<llvm::DataStreamer> >, llvm::StringRef, (anonymous namespace)::BitcodeReader*, llvm::LLVMContext&, bool, bool) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x42c3dc)
+#20 0x42c667 llvm::getStreamedBitcodeModule(llvm::StringRef, std::unique_ptr<llvm::DataStreamer, std::default_delete<llvm::DataStreamer> >, llvm::LLVMContext&, std::function<void (llvm::DiagnosticInfo const&)>) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x42c667)
+#21 0x406132 main (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x406132)
+#22 0x7f97db79fec5 __libc_start_main /build/buildd/eglibc-2.19/csu/libc-start.c:321:0
+#23 0x4045f9 _start (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x4045f9)
+Stack dump:
+0.  Program arguments: llvm-dis -disable-output crash1.bc 
+Aborted
+

test/Convert/crash4.err

+$ llvm-dis -disable-output crash4.bc
+llvm-dis: /workspace/llvm-dev/llvm/lib/IR/Type.cpp:337: llvm::FunctionType::FunctionType(llvm::Type*, llvm::ArrayRef<llvm::Type*>, bool): Assertion `isValidReturnType(Result) && "invalid return type for function"' failed.
+#0 0x5eeeac llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x5eeeac)
+#1 0x5ef1c1 PrintStackTraceSignalHandler(void*) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x5ef1c1)
+#2 0x5ed93d llvm::sys::RunSignalHandlers() (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x5ed93d)
+#3 0x5ede14 SignalHandler(int) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x5ede14)
+#4 0x7f3114582340 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x10340)
+#5 0x7f31139c3cc9 gsignal /build/buildd/eglibc-2.19/signal/../nptl/sysdeps/unix/sysv/linux/raise.c:56:0
+#6 0x7f31139c70d8 abort /build/buildd/eglibc-2.19/stdlib/abort.c:91:0
+#7 0x7f31139bcb86 __assert_fail_base /build/buildd/eglibc-2.19/assert/assert.c:92:0
+#8 0x7f31139bcc32 (/lib/x86_64-linux-gnu/libc.so.6+0x2fc32)
+#9 0x58b222 llvm::FunctionType::FunctionType(llvm::Type*, llvm::ArrayRef<llvm::Type*>, bool) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x58b222)
+#10 0x58b421 llvm::FunctionType::get(llvm::Type*, llvm::ArrayRef<llvm::Type*>, bool) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x58b421)
+#11 0x414f5f (anonymous namespace)::BitcodeReader::parseTypeTableBody() (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x414f5f)
+#12 0x414445 (anonymous namespace)::BitcodeReader::parseTypeTable() (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x414445)
+#13 0x420138 (anonymous namespace)::BitcodeReader::parseModule(bool, bool) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x420138)
+#14 0x4225af (anonymous namespace)::BitcodeReader::parseBitcodeInto(std::unique_ptr<llvm::DataStreamer, std::default_delete<llvm::DataStreamer> >, llvm::Module*, bool) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x4225af)
+#15 0x42c3dc getBitcodeModuleImpl(std::unique_ptr<llvm::DataStreamer, std::default_delete<llvm::DataStreamer> >, llvm::StringRef, (anonymous namespace)::BitcodeReader*, llvm::LLVMContext&, bool, bool) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x42c3dc)
+#16 0x42c667 llvm::getStreamedBitcodeModule(llvm::StringRef, std::unique_ptr<llvm::DataStreamer, std::default_delete<llvm::DataStreamer> >, llvm::LLVMContext&, std::function<void (llvm::DiagnosticInfo const&)>) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x42c667)
+#17 0x406132 main (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x406132)
+#18 0x7f31139aeec5 __libc_start_main /build/buildd/eglibc-2.19/csu/libc-start.c:321:0
+#19 0x4045f9 _start (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x4045f9)
+Stack dump:
+0.  Program arguments: llvm-dis -disable-output crash4.bc 

test/Convert/crash5.err

+$ llvm-dis -disable-output crash5.bc
+llvm-dis: /workspace/llvm-dev/llvm/lib/Bitcode/Reader/BitcodeReader.cpp:1018: void decodeLLVMAttributesForBitcode(llvm::AttrBuilder&, uint64_t): Assertion `(!Alignment || isPowerOf2_32(Alignment)) && "Alignment must be a power of two."' failed.
+#0 0x5eeeac llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x5eeeac)
+#1 0x5ef1c1 PrintStackTraceSignalHandler(void*) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x5ef1c1)
+#2 0x5ed93d llvm::sys::RunSignalHandlers() (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x5ed93d)
+#3 0x5ede14 SignalHandler(int) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x5ede14)
+#4 0x7f272d7f0340 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x10340)
+#5 0x7f272cc31cc9 gsignal /build/buildd/eglibc-2.19/signal/../nptl/sysdeps/unix/sysv/linux/raise.c:56:0
+#6 0x7f272cc350d8 abort /build/buildd/eglibc-2.19/stdlib/abort.c:91:0
+#7 0x7f272cc2ab86 __assert_fail_base /build/buildd/eglibc-2.19/assert/assert.c:92:0
+#8 0x7f272cc2ac32 (/lib/x86_64-linux-gnu/libc.so.6+0x2fc32)
+#9 0x413258 decodeLLVMAttributesForBitcode(llvm::AttrBuilder&, unsigned long) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x413258)
+#10 0x41358f (anonymous namespace)::BitcodeReader::parseAttributeBlock() (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x41358f)
+#11 0x4200a0 (anonymous namespace)::BitcodeReader::parseModule(bool, bool) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x4200a0)
+#12 0x4225af (anonymous namespace)::BitcodeReader::parseBitcodeInto(std::unique_ptr<llvm::DataStreamer, std::default_delete<llvm::DataStreamer> >, llvm::Module*, bool) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x4225af)
+#13 0x42c3dc getBitcodeModuleImpl(std::unique_ptr<llvm::DataStreamer, std::default_delete<llvm::DataStreamer> >, llvm::StringRef, (anonymous namespace)::BitcodeReader*, llvm::LLVMContext&, bool, bool) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x42c3dc)
+#14 0x42c667 llvm::getStreamedBitcodeModule(llvm::StringRef, std::unique_ptr<llvm::DataStreamer, std::default_delete<llvm::DataStreamer> >, llvm::LLVMContext&, std::function<void (llvm::DiagnosticInfo const&)>) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x42c667)
+#15 0x406132 main (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x406132)
+#16 0x7f272cc1cec5 __libc_start_main /build/buildd/eglibc-2.19/csu/libc-start.c:321:0
+#17 0x4045f9 _start (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x4045f9)
+Stack dump:
+0.  Program arguments: llvm-dis -disable-output crash5.bc 
+Aborted
+

test/Convert/crash6.err

+$ llvm-dis -disable-output crash6.bc
+llvm-dis: /workspace/llvm-dev/llvm/include/llvm/IR/Type.h:106: void llvm::Type::setSubclassData(unsigned int): Assertion `getSubclassData() == val && "Subclass data too large for field"' failed.
+#0 0x5eeeac llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x5eeeac)
+#1 0x5ef1c1 PrintStackTraceSignalHandler(void*) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x5ef1c1)
+#2 0x5ed93d llvm::sys::RunSignalHandlers() (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x5ed93d)
+#3 0x5ede14 SignalHandler(int) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x5ede14)
+#4 0x7f42b671f340 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x10340)
+#5 0x7f42b5b60cc9 gsignal /build/buildd/eglibc-2.19/signal/../nptl/sysdeps/unix/sysv/linux/raise.c:56:0
+#6 0x7f42b5b640d8 abort /build/buildd/eglibc-2.19/stdlib/abort.c:91:0
+#7 0x7f42b5b59b86 __assert_fail_base /build/buildd/eglibc-2.19/assert/assert.c:92:0
+#8 0x7f42b5b59c32 (/lib/x86_64-linux-gnu/libc.so.6+0x2fc32)
+#9 0x55be26 llvm::Type::setSubclassData(unsigned int) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x55be26)
+#10 0x58c88b llvm::PointerType::PointerType(llvm::Type*, unsigned int) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x58c88b)
+#11 0x58c833 llvm::PointerType::get(llvm::Type*, unsigned int) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x58c833)
+#12 0x414ac3 (anonymous namespace)::BitcodeReader::parseTypeTableBody() (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x414ac3)
+#13 0x414445 (anonymous namespace)::BitcodeReader::parseTypeTable() (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x414445)
+#14 0x420138 (anonymous namespace)::BitcodeReader::parseModule(bool, bool) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x420138)
+#15 0x4225af (anonymous namespace)::BitcodeReader::parseBitcodeInto(std::unique_ptr<llvm::DataStreamer, std::default_delete<llvm::DataStreamer> >, llvm::Module*, bool) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x4225af)
+#16 0x42c3dc getBitcodeModuleImpl(std::unique_ptr<llvm::DataStreamer, std::default_delete<llvm::DataStreamer> >, llvm::StringRef, (anonymous namespace)::BitcodeReader*, llvm::LLVMContext&, bool, bool) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x42c3dc)
+#17 0x42c667 llvm::getStreamedBitcodeModule(llvm::StringRef, std::unique_ptr<llvm::DataStreamer, std::default_delete<llvm::DataStreamer> >, llvm::LLVMContext&, std::function<void (llvm::DiagnosticInfo const&)>) (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x42c667)
+#18 0x406132 main (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x406132)
+#19 0x7f42b5b4bec5 __libc_start_main /build/buildd/eglibc-2.19/csu/libc-start.c:321:0
+#20 0x4045f9 _start (/workspace/llvm-dev/build-bcconv/bin/llvm-dis+0x4045f9)
+Stack dump:
+0.  Program arguments: llvm-dis -disable-output crash6.bc 
+Aborted

test/Convert/foo.ll

+; ModuleID = 'send/foo.bc'
+
+define i32 @fib(i32 %p0) {
+  %v0 = icmp ult i32 %p0, 1
+  br i1 %v0, label %true, label %false
+
+true:                                             ; preds = %0
+  ret i32 1
+
+false:                                            ; preds = %0
+  %v2 = sub i32 %p0, 1
+  %v3 = call i32 @fib(i32 %v2)
+  %v4 = add i32 %v3, %p0
+  ret i32 %v4
+}

test/lit.cfg

 NOJUNK = r"(?<!\.|-|\^|/)"
 
 for pattern in [r"\bbugpoint\b(?!-)",
                 NOJUNK + r"\bllc\b",
                 r"\blli\b",
                 r"\bllvm-ar\b",
                 r"\bllvm-as\b",
                 r"\bllvm-bcanalyzer\b",
+                r"\bllvm-bcconv\b",
                 r"\bllvm-config\b",
                 r"\bllvm-cov\b",
                 r"\bllvm-cxxdump\b",
                 r"\bllvm-diff\b",
                 r"\bllvm-dis\b",
                 r"\bllvm-dsymutil\b",
                 r"\bllvm-dwarfdump\b",
                 r"\bllvm-extract\b",

tools/LLVMBuild.txt

 subdirectories =
  bugpoint
  dsymutil
  llc
  lli
  llvm-ar
  llvm-as
  llvm-bcanalyzer
+ llvm-bcconv
  llvm-cov
  llvm-diff
  llvm-dis
  llvm-dwarfdump
  llvm-extract
  llvm-jitlistener
  llvm-link
  llvm-lto

tools/Makefile

 # the wider LLVM infrastructure (including Clang).
 OPTIONAL_DIRS := lldb
 
 # NOTE: The tools are organized into five groups of four consisting of one
 # large and three small executables. This is done to minimize memory load
 # in parallel builds.  Please retain this ordering.
 DIRS := llvm-config
 PARALLEL_DIRS := opt llvm-as llvm-dis llc llvm-ar llvm-nm llvm-link \
-                 lli llvm-extract llvm-mc bugpoint llvm-bcanalyzer llvm-diff \
-                 macho-dump llvm-objdump llvm-readobj llvm-rtdyld \
+                 lli llvm-extract llvm-mc bugpoint llvm-bcanalyzer llvm-bcconv \
+                 llvm-diff macho-dump llvm-objdump llvm-readobj llvm-rtdyld \
                  llvm-dwarfdump llvm-cov llvm-size llvm-stress llvm-mcmarkup \
                  llvm-profdata llvm-symbolizer obj2yaml yaml2obj llvm-c-test \
                  llvm-cxxdump verify-uselistorder dsymutil llvm-pdbdump
 
 # If Intel JIT Events support is configured, build an extra tool to test it.
 ifeq ($(USE_INTEL_JITEVENTS), 1)
   PARALLEL_DIRS += llvm-jitlistener
 endif

tools/llvm-bcconv/CMakeLists.txt

+set(LLVM_LINK_COMPONENTS
+  BitcodeConv
+  BitReader
+  BitWriter
+  Support
+  )
+
+add_llvm_tool(llvm-bcconv
+  llvm-bcconv.cpp
+  )

tools/llvm-bcconv/LLVMBuild.txt

-;===- ./lib/Bitcode/LLVMBuild.txt ------------------------------*- Conf -*--===;
+;===- ./tools/llvm-bcconv/LLVMBuild.txt ------------------------*- Conf -*--===;
 ;
 ;                     The LLVM Compiler Infrastructure
 ;
 ; This file is distributed under the University of Illinois Open Source
 ; License. See LICENSE.TXT for details.
 ;
 ;===------------------------------------------------------------------------===;
 ;
 ; This is an LLVMBuild description file for the components in this subdirectory.
 ;
 ; For more information on the LLVMBuild system, please see:
 ;
 ;   http://llvm.org/docs/LLVMBuild.html
 ;
 ;===------------------------------------------------------------------------===;
 
-[common]
-subdirectories = Reader Writer
-
 [component_0]
-type = Group
-name = Bitcode
-parent = Libraries
+type = Tool
+name = llvm-bcconv
+parent = Tools
+required_libraries = BitcodeConv BitReader BitWriter Support

tools/llvm-bcconv/Makefile

-##===- lib/Bitcode/Makefile --------------------------------*- Makefile -*-===##
-#
+##===- tools/llvm-bcconv/Makefile -------------------------*- Makefile -*-====##
+# 
 #                     The LLVM Compiler Infrastructure
 #
 # This file is distributed under the University of Illinois Open Source
 # License. See LICENSE.TXT for details.
-#
+# 
 ##===----------------------------------------------------------------------===##
 
-LEVEL = ../..
-PARALLEL_DIRS = Reader Writer
+LEVEL := ../..
+TOOLNAME := llvm-bcconv
+LINK_COMPONENTS := BitcodeConv BitReader BitWriter Support
 
-include $(LEVEL)/Makefile.common
+# This tool has no plugins, optimize startup time.
+TOOL_NO_EXPORTS := 1
 
+include $(LEVEL)/Makefile.common

tools/llvm-bcconv/llvm-bcconv.cpp

+//===-- llvm-bcconv.cpp - Bitcode format converter ------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This tool converts bitcode files from one format, to another. Formats
+// are binary, textual, and simplified. See file "llvm/Bitcode/BitcodeConvert.h"
+// for definitions of these bitcode file formats.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/Bitcode/BitcodeConvert.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/ManagedStatic.h"
+#include "llvm/Support/PrettyStackTrace.h"
+#include "llvm/Support/Signals.h"
+#include "llvm/Support/ToolOutputFile.h"
+#include "llvm/Support/raw_ostream.h"
+
+using namespace llvm;
+using namespace llvm::bcconv;
+
+namespace {
+
+static cl::opt<std::string>
+    InputFilename(cl::Positional, cl::desc("<input bitcode>"), cl::init("-"));
+
+static cl::opt<std::string> OutputFilename(
+    "o", cl::desc("Write output to specified filename (rather than stdout)"),
+    cl::init("-"));
+
+// Form to read/write.
+enum BitcodeForm {
+  Binary,
+  Textual,
+  Simplified
+};
+
+cl::opt<BitcodeForm> InputForm(
+    cl::Required, cl::desc("Input bitcode form:"),
+    cl::values(clEnumValN(Binary, "from-binary",
+                          "Read a binary bitcode file"),
+               clEnumValN(Textual, "from-text",
+                          "Read a textual bitcode file (not .ll file)"),
+               clEnumValN(Simplified, "from-simplified",
+                          "Read a simplified bitcode file"),
+               clEnumValEnd));
+
+cl::opt<BitcodeForm> OutputForm(
+    cl::Required, cl::desc("Output bitcode form:"),
+    cl::values(clEnumValN(Binary, "to-binary",
+                          "Write a binary bitcode file"),
+               clEnumValN(Textual, "to-text",
+                          "Write a textual bitcode file (not .ll file)"),
+               clEnumValN(Simplified, "to-simplified",
+                          "Write a simplified bitcode file"),
+               clEnumValEnd));
+
+static cl::opt<bool>
+    AllowTextComments("allow-comments",
+                      cl::desc("Allow comments in textual bitcode"),
+                      cl::init(false));
+
+static cl::opt<bool> ErrorRecover("error-recover",
+                                  cl::desc("Try to recover on errors"),
+                                  cl::init(false));
+
+static cl::opt<uint8_t> AcceptValueFlag(
+    "accept-value",
+    cl::desc("Value defining value acceptance in simplified form"),
+    cl::init(SimplifiedBitcodeActions::AcceptValueDefault));
+
+static cl::opt<uint8_t> AcceptRecordFlag(
+    "accept-record",
+    cl::desc("Value defining record acceptance for simplified form"),
+    cl::init(SimplifiedBitcodeActions::AcceptRecordDefault));
+
+static cl::opt<uint8_t> AcceptEscapeFlag(
+    "escape-byte",
+    cl::desc("Value defining byte escape for simplified form"),
+    cl::init(SimplifiedBitcodeActions::AcceptEscapeDefault));
+
+static bool writeOutput(SmallVectorImpl<char> &Output,
+                        sys::fs::OpenFlags OpenFlags) {
+  std::error_code EC;
+  auto FDOut = make_unique<tool_output_file>(OutputFilename, EC, OpenFlags);
+  if (EC) {
+    errs() << EC.message() << '\n';
+    return false;
+  }
+  for (auto Val : Output)
+    FDOut->os() << Val;
+  FDOut->keep();
+  return true;
+}
+
+inline static int convertToExitStatus(bool Value) { return !Value; }
+
+void initializeSimplifiedBitcodeActions(SimplifiedBitcodeActions &Actions) {
+  Actions.AcceptValue = AcceptValueFlag;
+  Actions.AcceptRecord = AcceptRecordFlag;
+  Actions.AcceptEscape = AcceptEscapeFlag;
+}
+
+} // end of anonymous namespace
+
+int main(int argc, char *argv[]) {
+  // Print a stack trace if we signal out.
+  sys::PrintStackTraceOnErrorSignal();
+  PrettyStackTraceProgram X(argc, argv);
+  llvm_shutdown_obj Y; // Call llvm_shutdown() on exit.
+  cl::ParseCommandLineOptions(argc, argv,
+                              "llvm-bcconv: Bitcode format converter\n");
+
+  ErrorOr<std::unique_ptr<MemoryBuffer>> ErrOrInput =
+      MemoryBuffer::getFileOrSTDIN(InputFilename);
+  if (std::error_code EC = ErrOrInput.getError()) {
+    errs() << "Error reading " << InputFilename << ": " << EC.message() << "\n";
+    return 1;
+  }
+
+  std::unique_ptr<MemoryBuffer> Input(ErrOrInput.get().release());
+  StringRef InputText(Input->getBuffer());
+  BitcodeRecordList Records;
+  std::error_code EC;
+  switch (InputForm) {
+  case Binary:
+    EC = readBinaryBitcode(InputText, Records);
+    break;
+  case Textual:
+    EC = readTextualBitcode(InputText, Records, AllowTextComments);
+    break;
+  case Simplified: {
+    SimplifiedBitcodeActions Actions;
+    initializeSimplifiedBitcodeActions(Actions);
+    EC = readSimplifiedBitcode(InputText, Records, Actions);
+    break;
+  }
+  }
+
+  if (EC) {
+    errs() << "Error: " << EC.message() << "\n";
+    return convertToExitStatus(false);
+  }
+
+  SmallVector<char, 1024> Output;
+  std::string ErrorMessages;
+  bool WriteSuccessful = false;
+  switch (OutputForm) {
+  case Binary:
+    WriteSuccessful = writeBinaryBitcode(Output, Records, ErrorMessages,
+                                         ErrorRecover);
+    break;
+  case Textual:
+    WriteSuccessful = writeTextualBitcode(Output, Records, ErrorMessages,
+                                          ErrorRecover);
+    break;
+  case Simplified: {
+    SimplifiedBitcodeActions Actions;
+    initializeSimplifiedBitcodeActions(Actions);
+    WriteSuccessful = writeSimplifiedBitcode(Output, Records, Actions,
+                                             ErrorMessages, ErrorRecover);
+    break;
+  }
+  }
+
+  if (!ErrorMessages.empty())
+    errs() << ErrorMessages;
+
+  if (!WriteSuccessful) {
+    return convertToExitStatus(false);
+  }
+
+  return convertToExitStatus(writeOutput(
+      Output, OutputForm == Textual ? sys::fs::F_Text : sys::fs::F_None));
+}