[BinaryFormat] Add MsgPackTypes

[scott.linder]

Summary

Add data structure to represent MessagePack "documents" and convert to/from both MessagePack and YAML encodings.

This patch is part of a series required for AMDGPU to support it's new MessagePack metadata.

[scott.linder]

Rebase and ping

[dblaikie]

include/llvm/BinaryFormat/MsgPackTypes.h

32–35 ↗

(On Diff #158010)

What's the need for this?

I suspect it's probably best to have the APIs always return unique_ptrs - which are implicitly convertible to shared_ptrs if the users of the API need that.

50 ↗

(On Diff #158010)

Looks like this probably needs a virtual dtor? Unless there's something to prevent polymorphic ownership (ie: to prevent someone taking, say, a unique_ptr<ArrayNode> to a unique_ptr<Node>) - such as having a protected dtor.

unittests/BinaryFormat/MsgPackTypesTest.cpp

133 ↗

(On Diff #158010)

Looks like this could be a direct value rather than dynamically allocated:

MapNode<> M;

134–137 ↗

(On Diff #158010)

Drop extra parens around these RHS, so it's like this:

... = makePtr<...>(...);

[scott.linder]

include/llvm/BinaryFormat/MsgPackTypes.h

32–35 ↗

(On Diff #158010)

The reason for this is that the methods to stream in MessagePack/YAML have to know which type to construct the document tree with. A single unique_ptr is implicitly convertible, but with a tree of them the conversions are not useful if you are trying to share parts of the tree. Unfortunately this does complicate the YAML trait specialization a bit.

50 ↗

(On Diff #158010)

Thank you for the catch, fixed.

[scott.linder]

Add virtual destructor to base class and clean up tests.

[dblaikie]

include/llvm/BinaryFormat/MsgPackTypes.h

32–35 ↗

(On Diff #158010)

Is there a significant win to using unique_ptr rather than using shared_ptr always, then?

& what's the sharing situation? Perhaps we could discuss that & see if there's a reasonable single ownership model that'd work.

[scott.linder]

I've updated the patch to just support shared_ptr; I think the more generic option is right for now.

[dblaikie]

In D48175#1216609, @scott.linder wrote:

I've updated the patch to just support shared_ptr; I think the more generic option is right for now.

I'd still be curious to understand the usage you're picturing that creates the requirement for shared ownership, if you could paint a bit of a picture around that, that'd be super handy!

[scott.linder]

The use I have in mind is actually an external component which depends on LLVM. We currently implement reading MessagePack metadata there, but when we land the changes to generate it in LLVM it would be nice to avoid duplicating the code there.

The rationale for a shared_ptr tree is that our API hands out opaque handles to sub-trees in the metadata which have their own lifetime independent of the root. With shared_ptr this just naturally falls out, but with unique_ptr it requires some book-keeping.

I do not see a scenario where the shared_ptr will be needed in LLVM, so if you prefer I am happy to update the patch to use unique_ptr.

[dblaikie]

In D48175#1219832, @scott.linder wrote:

The use I have in mind is actually an external component which depends on LLVM. We currently implement reading MessagePack metadata there, but when we land the changes to generate it in LLVM it would be nice to avoid duplicating the code there.

The rationale for a shared_ptr tree is that our API hands out opaque handles to sub-trees in the metadata which have their own lifetime independent of the root. With shared_ptr this just naturally falls out, but with unique_ptr it requires some book-keeping.

I do not see a scenario where the shared_ptr will be needed in LLVM, so if you prefer I am happy to update the patch to use unique_ptr.

Difficult call, but seems OK to me since you'll mostly be the only folks using the data structure anyway.

Could you include a representative unit test of this ownership situation? (can be fairly simple, but demonstrate that the root may be built, a subtree taken, then the tree destroyed and the subtree still used).

[scott.linder]

Added test for shared ownership. I've run it locally with ASan and UBSan enabled.

And thank you, this will simplify our use-case considerably :)

[dblaikie]

Looks good - thanks

llvm/trunk/include/llvm/BinaryFormat/MsgPackTypes.h

+//===- MsgPackTypes.h - MsgPack Types ---------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+/// \file
+/// This is a data structure for representing MessagePack "documents", with
+/// methods to go to and from MessagePack. The types also specialize YAMLIO
+/// traits in order to go to and from YAML.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/ADT/Optional.h"
+#include "llvm/BinaryFormat/MsgPackReader.h"
+#include "llvm/BinaryFormat/MsgPackWriter.h"
+#include "llvm/Support/Casting.h"
+#include "llvm/Support/YAMLTraits.h"
+#include <vector>
+
+#ifndef LLVM_BINARYFORMAT_MSGPACKTYPES_H
+#define LLVM_BINARYFORMAT_MSGPACKTYPES_H
+
+namespace llvm {
+namespace msgpack {
+
+class Node;
+
+/// Short-hand for a Node pointer.
+using NodePtr = std::shared_ptr<Node>;
+
+/// Short-hand for an Optional Node pointer.
+using OptNodePtr = Optional<NodePtr>;
+
+/// Abstract base-class which can be any MessagePack type.
+class Node {
+public:
+  enum NodeKind {
+    NK_Scalar,
+    NK_Array,
+    NK_Map,
+  };
+
+private:
+  virtual void anchor() = 0;
+  const NodeKind Kind;
+
+  static Expected<OptNodePtr> readArray(Reader &MPReader, size_t Length);
+  static Expected<OptNodePtr> readMap(Reader &MPReader, size_t Length);
+
+public:
+  NodeKind getKind() const { return Kind; }
+
+  /// Construct a Node. Used by derived classes to track kind information.
+  Node(NodeKind Kind) : Kind(Kind) {}
+
+  virtual ~Node() = default;
+
+  /// Read from a MessagePack reader \p MPReader, returning an error if one is
+  /// encountered, or None if \p MPReader is at the end of stream, or some Node
+  /// pointer if some type is read.
+  static Expected<OptNodePtr> read(Reader &MPReader);
+
+  /// Write to a MessagePack writer \p MPWriter.
+  virtual void write(Writer &MPWriter) = 0;
+};
+
+/// A MessagePack scalar.
+class ScalarNode : public Node {
+public:
+  enum ScalarKind {
+    SK_Int,
+    SK_UInt,
+    SK_Nil,
+    SK_Boolean,
+    SK_Float,
+    SK_String,
+    SK_Binary,
+  };
+
+private:
+  void anchor() override;
+
+  void destroy();
+
+  ScalarKind SKind;
+
+  union {
+    int64_t IntValue;
+    uint64_t UIntValue;
+    bool BoolValue;
+    double FloatValue;
+    std::string StringValue;
+  };
+
+public:
+  /// Construct an Int ScalarNode.
+  ScalarNode(int64_t IntValue);
+  /// Construct an Int ScalarNode.
+  ScalarNode(int32_t IntValue);
+  /// Construct an UInt ScalarNode.
+  ScalarNode(uint64_t UIntValue);
+  /// Construct an UInt ScalarNode.
+  ScalarNode(uint32_t UIntValue);
+  /// Construct a Nil ScalarNode.
+  ScalarNode();
+  /// Construct a Boolean ScalarNode.
+  ScalarNode(bool BoolValue);
+  /// Construct a Float ScalarNode.
+  ScalarNode(double FloatValue);
+  /// Construct a String ScalarNode.
+  ScalarNode(StringRef StringValue);
+  /// Construct a String ScalarNode.
+  ScalarNode(const char *StringValue);
+  /// Construct a String ScalarNode.
+  ScalarNode(std::string &&StringValue);
+  /// Construct a Binary ScalarNode.
+  ScalarNode(MemoryBufferRef BinaryValue);
+
+  ~ScalarNode();
+
+  ScalarNode &operator=(const ScalarNode &RHS) = delete;
+  /// A ScalarNode can only be move assigned.
+  ScalarNode &operator=(ScalarNode &&RHS);
+
+  /// Change the kind of this ScalarNode, zero initializing it to the new type.
+  void setScalarKind(ScalarKind SKind) {
+    switch (SKind) {
+    case SK_Int:
+      *this = int64_t(0);
+      break;
+    case SK_UInt:
+      *this = uint64_t(0);
+      break;
+    case SK_Boolean:
+      *this = false;
+      break;
+    case SK_Float:
+      *this = 0.0;
+      break;
+    case SK_String:
+      *this = StringRef();
+      break;
+    case SK_Binary:
+      *this = MemoryBufferRef("", "");
+      break;
+    case SK_Nil:
+      *this = ScalarNode();
+      break;
+    }
+  }
+
+  /// Get the current kind of ScalarNode.
+  ScalarKind getScalarKind() { return SKind; }
+
+  /// Get the value of an Int scalar.
+  ///
+  /// \warning Assumes getScalarKind() == SK_Int
+  int64_t getInt() {
+    assert(SKind == SK_Int);
+    return IntValue;
+  }
+
+  /// Get the value of a UInt scalar.
+  ///
+  /// \warning Assumes getScalarKind() == SK_UInt
+  uint64_t getUInt() {
+    assert(SKind == SK_UInt);
+    return UIntValue;
+  }
+
+  /// Get the value of an Boolean scalar.
+  ///
+  /// \warning Assumes getScalarKind() == SK_Boolean
+  bool getBool() {
+    assert(SKind == SK_Boolean);
+    return BoolValue;
+  }
+
+  /// Get the value of an Float scalar.
+  ///
+  /// \warning Assumes getScalarKind() == SK_Float
+  double getFloat() {
+    assert(SKind == SK_Float);
+    return FloatValue;
+  }
+
+  /// Get the value of a String scalar.
+  ///
+  /// \warning Assumes getScalarKind() == SK_String
+  StringRef getString() {
+    assert(SKind == SK_String);
+    return StringValue;
+  }
+
+  /// Get the value of a Binary scalar.
+  ///
+  /// \warning Assumes getScalarKind() == SK_Binary
+  StringRef getBinary() {
+    assert(SKind == SK_Binary);
+    return StringValue;
+  }
+
+  static bool classof(const Node *N) { return N->getKind() == NK_Scalar; }
+
+  void write(Writer &MPWriter) override;
+
+  /// Parse a YAML scalar of the current ScalarKind from \p ScalarStr.
+  ///
+  /// \returns An empty string on success, otherwise an error message.
+  StringRef inputYAML(StringRef ScalarStr);
+
+  /// Output a YAML scalar of the current ScalarKind into \p OS.
+  void outputYAML(raw_ostream &OS) const;
+
+  /// Determine which YAML quoting type the current value would need when
+  /// output.
+  yaml::QuotingType mustQuoteYAML(StringRef ScalarStr) const;
+
+  /// Get the YAML tag for the current ScalarKind.
+  StringRef getYAMLTag() const;
+
+  /// Flag which affects how the type handles YAML tags when reading and
+  /// writing.
+  ///
+  /// When false, tags are used when reading and writing. When reading, the tag
+  /// is used to decide the ScalarKind before parsing. When writing, the tag is
+  /// output along with the value.
+  ///
+  /// When true, tags are ignored when reading and writing. When reading, the
+  /// ScalarKind is always assumed to be String. When writing, the tag is not
+  /// output.
+  bool IgnoreTag = false;
+
+  static const char *IntTag;
+  static const char *NilTag;
+  static const char *BooleanTag;
+  static const char *FloatTag;
+  static const char *StringTag;
+  static const char *BinaryTag;
+};
+
+class ArrayNode : public Node, public std::vector<NodePtr> {
+  void anchor() override;
+
+public:
+  ArrayNode() : Node(NK_Array) {}
+  static bool classof(const Node *N) { return N->getKind() == NK_Array; }
+
+  void write(Writer &MPWriter) override {
+    MPWriter.writeArraySize(this->size());
+    for (auto &N : *this)
+      N->write(MPWriter);
+  }
+};
+
+class MapNode : public Node, public StringMap<NodePtr> {
+  void anchor() override;
+
+public:
+  MapNode() : Node(NK_Map) {}
+  static bool classof(const Node *N) { return N->getKind() == NK_Map; }
+
+  void write(Writer &MPWriter) override {
+    MPWriter.writeMapSize(this->size());
+    for (auto &N : *this) {
+      MPWriter.write(N.first());
+      N.second->write(MPWriter);
+    }
+  }
+};
+
+} // end namespace msgpack
+
+namespace yaml {
+
+template <> struct PolymorphicTraits<msgpack::NodePtr> {
+  static NodeKind getKind(const msgpack::NodePtr &N) {
+    if (isa<msgpack::ScalarNode>(*N))
+      return NodeKind::Scalar;
+    if (isa<msgpack::MapNode>(*N))
+      return NodeKind::Map;
+    if (isa<msgpack::ArrayNode>(*N))
+      return NodeKind::Sequence;
+    llvm_unreachable("NodeKind not supported");
+  }
+  static msgpack::ScalarNode &getAsScalar(msgpack::NodePtr &N) {
+    if (!N || !isa<msgpack::ScalarNode>(*N))
+      N.reset(new msgpack::ScalarNode());
+    return *cast<msgpack::ScalarNode>(N.get());
+  }
+  static msgpack::MapNode &getAsMap(msgpack::NodePtr &N) {
+    if (!N || !isa<msgpack::MapNode>(*N))
+      N.reset(new msgpack::MapNode());
+    return *cast<msgpack::MapNode>(N.get());
+  }
+  static msgpack::ArrayNode &getAsSequence(msgpack::NodePtr &N) {
+    if (!N || !isa<msgpack::ArrayNode>(*N))
+      N.reset(new msgpack::ArrayNode());
+    return *cast<msgpack::ArrayNode>(N.get());
+  }
+};
+
+template <> struct TaggedScalarTraits<msgpack::ScalarNode> {
+  static void output(const msgpack::ScalarNode &S, void *Ctxt,
+                     raw_ostream &ScalarOS, raw_ostream &TagOS) {
+    if (!S.IgnoreTag)
+      TagOS << S.getYAMLTag();
+    S.outputYAML(ScalarOS);
+  }
+
+  static StringRef input(StringRef ScalarStr, StringRef Tag, void *Ctxt,
+                         msgpack::ScalarNode &S) {
+    if (Tag == msgpack::ScalarNode::IntTag) {
+      S.setScalarKind(msgpack::ScalarNode::SK_UInt);
+      if (S.inputYAML(ScalarStr) == StringRef())
+        return StringRef();
+      S.setScalarKind(msgpack::ScalarNode::SK_Int);
+      return S.inputYAML(ScalarStr);
+    }
+
+    if (S.IgnoreTag || Tag == msgpack::ScalarNode::StringTag ||
+        Tag == "tag:yaml.org,2002:str")
+      S.setScalarKind(msgpack::ScalarNode::SK_String);
+    else if (Tag == msgpack::ScalarNode::NilTag)
+      S.setScalarKind(msgpack::ScalarNode::SK_Nil);
+    else if (Tag == msgpack::ScalarNode::BooleanTag)
+      S.setScalarKind(msgpack::ScalarNode::SK_Boolean);
+    else if (Tag == msgpack::ScalarNode::FloatTag)
+      S.setScalarKind(msgpack::ScalarNode::SK_Float);
+    else if (Tag == msgpack::ScalarNode::StringTag)
+      S.setScalarKind(msgpack::ScalarNode::SK_String);
+    else if (Tag == msgpack::ScalarNode::BinaryTag)
+      S.setScalarKind(msgpack::ScalarNode::SK_Binary);
+    else
+      return "Unsupported messagepack tag";
+
+    return S.inputYAML(ScalarStr);
+  }
+
+  static QuotingType mustQuote(const msgpack::ScalarNode &S, StringRef Str) {
+    return S.mustQuoteYAML(Str);
+  }
+};
+
+template <> struct CustomMappingTraits<msgpack::MapNode> {
+  static void inputOne(IO &IO, StringRef Key, msgpack::MapNode &M) {
+    IO.mapRequired(Key.str().c_str(), M[Key]);
+  }
+  static void output(IO &IO, msgpack::MapNode &M) {
+    for (auto &N : M)
+      IO.mapRequired(N.getKey().str().c_str(), N.getValue());
+  }
+};
+
+template <> struct SequenceTraits<msgpack::ArrayNode> {
+  static size_t size(IO &IO, msgpack::ArrayNode &A) { return A.size(); }
+  static msgpack::NodePtr &element(IO &IO, msgpack::ArrayNode &A,
+                                   size_t Index) {
+    if (Index >= A.size())
+      A.resize(Index + 1);
+    return A[Index];
+  }
+};
+
+} // end namespace yaml
+} // end namespace llvm
+
+#endif //  LLVM_BINARYFORMAT_MSGPACKTYPES_H

llvm/trunk/lib/BinaryFormat/CMakeLists.txt

 add_llvm_library(LLVMBinaryFormat
   Dwarf.cpp
   Magic.cpp
   MsgPackReader.cpp
+  MsgPackTypes.cpp
   MsgPackWriter.cpp
   Wasm.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${LLVM_MAIN_INCLUDE_DIR}/llvm/BinaryFormat
   )
   

llvm/trunk/lib/BinaryFormat/MsgPackTypes.cpp

+//===- MsgPackTypes.cpp - MsgPack Types -------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+/// \file
+/// Implementation of types representing MessagePack "documents".
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/BinaryFormat/MsgPackTypes.h"
+#include "llvm/Support/Error.h"
+
+using namespace llvm;
+using namespace msgpack;
+
+namespace llvm {
+namespace msgpack {
+void ScalarNode::anchor() {}
+void ArrayNode::anchor() {}
+void MapNode::anchor() {}
+}
+}
+
+Expected<OptNodePtr> Node::readArray(Reader &MPReader, size_t Length) {
+  auto A = std::make_shared<ArrayNode>();
+  for (size_t I = 0; I < Length; ++I) {
+    auto OptNodeOrErr = Node::read(MPReader);
+    if (auto Err = OptNodeOrErr.takeError())
+      return std::move(Err);
+    if (!*OptNodeOrErr)
+      return make_error<StringError>(
+          "Insufficient array elements",
+          std::make_error_code(std::errc::invalid_argument));
+    A->push_back(std::move(**OptNodeOrErr));
+  }
+  return OptNodePtr(std::move(A));
+}
+
+Expected<OptNodePtr> Node::readMap(Reader &MPReader, size_t Length) {
+  auto M = std::make_shared<MapNode>();
+  for (size_t I = 0; I < Length; ++I) {
+    auto OptKeyOrErr = Node::read(MPReader);
+    if (auto Err = OptKeyOrErr.takeError())
+      return std::move(Err);
+    if (!*OptKeyOrErr)
+      return make_error<StringError>(
+          "Insufficient map elements",
+          std::make_error_code(std::errc::invalid_argument));
+    auto OptValOrErr = Node::read(MPReader);
+    if (auto Err = OptValOrErr.takeError())
+      return std::move(Err);
+    if (!*OptValOrErr)
+      return make_error<StringError>(
+          "Insufficient map elements",
+          std::make_error_code(std::errc::invalid_argument));
+    auto *Key = dyn_cast<ScalarNode>((*OptKeyOrErr)->get());
+    if (!Key)
+      return make_error<StringError>(
+          "Only string map keys are supported",
+          std::make_error_code(std::errc::invalid_argument));
+    if (Key->getScalarKind() != ScalarNode::SK_String)
+      return make_error<StringError>(
+          "Only string map keys are supported",
+          std::make_error_code(std::errc::invalid_argument));
+    M->try_emplace(Key->getString(), std::move(**OptValOrErr));
+  }
+  return OptNodePtr(std::move(M));
+}
+
+Expected<OptNodePtr> Node::read(Reader &MPReader) {
+  Object Obj;
+
+  auto ContinueOrErr = MPReader.read(Obj);
+  if (auto Err = ContinueOrErr.takeError())
+    return std::move(Err);
+  if (!*ContinueOrErr)
+    return None;
+
+  switch (Obj.Kind) {
+  case Type::Int:
+    return OptNodePtr(std::make_shared<ScalarNode>(Obj.Int));
+  case Type::UInt:
+    return OptNodePtr(std::make_shared<ScalarNode>(Obj.UInt));
+  case Type::Nil:
+    return OptNodePtr(std::make_shared<ScalarNode>());
+  case Type::Boolean:
+    return OptNodePtr(std::make_shared<ScalarNode>(Obj.Bool));
+  case Type::Float:
+    return OptNodePtr(std::make_shared<ScalarNode>(Obj.Float));
+  case Type::String:
+    return OptNodePtr(std::make_shared<ScalarNode>(Obj.Raw));
+  case Type::Binary:
+    return OptNodePtr(std::make_shared<ScalarNode>(Obj.Raw));
+  case Type::Array:
+    return Node::readArray(MPReader, Obj.Length);
+  case Type::Map:
+    return Node::readMap(MPReader, Obj.Length);
+  case Type::Extension:
+    return make_error<StringError>(
+        "Extension types are not supported",
+        std::make_error_code(std::errc::invalid_argument));
+  }
+  llvm_unreachable("msgpack::Type not handled");
+}
+
+void ScalarNode::destroy() {
+  switch (SKind) {
+  case SK_String:
+  case SK_Binary:
+    StringValue.~basic_string();
+    break;
+  default:
+    // POD types do not require destruction
+    break;
+  }
+}
+
+ScalarNode::ScalarNode(int64_t IntValue)
+    : Node(NK_Scalar), SKind(SK_Int), IntValue(IntValue) {}
+
+ScalarNode::ScalarNode(int32_t IntValue)
+    : ScalarNode(static_cast<int64_t>(IntValue)) {}
+
+ScalarNode::ScalarNode(uint64_t UIntValue)
+    : Node(NK_Scalar), SKind(SK_UInt), UIntValue(UIntValue) {}
+
+ScalarNode::ScalarNode(uint32_t IntValue)
+    : ScalarNode(static_cast<uint64_t>(IntValue)) {}
+
+ScalarNode::ScalarNode() : Node(NK_Scalar), SKind(SK_Nil) {}
+
+ScalarNode::ScalarNode(bool BoolValue)
+    : Node(NK_Scalar), SKind(SK_Boolean), BoolValue(BoolValue) {}
+
+ScalarNode::ScalarNode(double FloatValue)
+    : Node(NK_Scalar), SKind(SK_Float), BoolValue(FloatValue) {}
+
+ScalarNode::ScalarNode(StringRef StringValue)
+    : Node(NK_Scalar), SKind(SK_String) {
+  new (&this->StringValue) std::string(StringValue);
+}
+
+ScalarNode::ScalarNode(const char *StringValue)
+    : ScalarNode(StringRef(StringValue)) {}
+
+ScalarNode::ScalarNode(std::string &&StringValue)
+    : Node(NK_Scalar), SKind(SK_String) {
+  new (&this->StringValue) std::string(StringValue);
+}
+
+ScalarNode::ScalarNode(MemoryBufferRef BinaryValue)
+    : Node(NK_Scalar), SKind(SK_Binary) {
+  new (&StringValue) std::string(BinaryValue.getBuffer());
+}
+
+ScalarNode::~ScalarNode() { destroy(); }
+
+ScalarNode &ScalarNode::operator=(ScalarNode &&RHS) {
+  destroy();
+  switch (SKind = RHS.SKind) {
+  case SK_Int:
+    IntValue = RHS.IntValue;
+    break;
+  case SK_UInt:
+    UIntValue = RHS.UIntValue;
+    break;
+  case SK_Boolean:
+    BoolValue = RHS.BoolValue;
+    break;
+  case SK_Float:
+    FloatValue = RHS.FloatValue;
+    break;
+  case SK_String:
+  case SK_Binary:
+    new (&StringValue) std::string(std::move(RHS.StringValue));
+    break;
+  case SK_Nil:
+    // pass
+    break;
+  }
+  return *this;
+}
+
+StringRef ScalarNode::inputYAML(StringRef ScalarStr) {
+  switch (SKind) {
+  case SK_Int:
+    return yaml::ScalarTraits<int64_t>::input(ScalarStr, nullptr, IntValue);
+  case SK_UInt:
+    return yaml::ScalarTraits<uint64_t>::input(ScalarStr, nullptr, UIntValue);
+  case SK_Nil:
+    return StringRef();
+  case SK_Boolean:
+    return yaml::ScalarTraits<bool>::input(ScalarStr, nullptr, BoolValue);
+  case SK_Float:
+    return yaml::ScalarTraits<double>::input(ScalarStr, nullptr, FloatValue);
+  case SK_Binary:
+  case SK_String:
+    return yaml::ScalarTraits<std::string>::input(ScalarStr, nullptr,
+                                                  StringValue);
+  }
+  llvm_unreachable("unrecognized ScalarKind");
+}
+
+void ScalarNode::outputYAML(raw_ostream &OS) const {
+  switch (SKind) {
+  case SK_Int:
+    yaml::ScalarTraits<int64_t>::output(IntValue, nullptr, OS);
+    break;
+  case SK_UInt:
+    yaml::ScalarTraits<uint64_t>::output(UIntValue, nullptr, OS);
+    break;
+  case SK_Nil:
+    yaml::ScalarTraits<StringRef>::output("", nullptr, OS);
+    break;
+  case SK_Boolean:
+    yaml::ScalarTraits<bool>::output(BoolValue, nullptr, OS);
+    break;
+  case SK_Float:
+    yaml::ScalarTraits<double>::output(FloatValue, nullptr, OS);
+    break;
+  case SK_Binary:
+  case SK_String:
+    yaml::ScalarTraits<std::string>::output(StringValue, nullptr, OS);
+    break;
+  }
+}
+
+yaml::QuotingType ScalarNode::mustQuoteYAML(StringRef ScalarStr) const {
+  switch (SKind) {
+  case SK_Int:
+    return yaml::ScalarTraits<int64_t>::mustQuote(ScalarStr);
+  case SK_UInt:
+    return yaml::ScalarTraits<uint64_t>::mustQuote(ScalarStr);
+  case SK_Nil:
+    return yaml::ScalarTraits<StringRef>::mustQuote(ScalarStr);
+  case SK_Boolean:
+    return yaml::ScalarTraits<bool>::mustQuote(ScalarStr);
+  case SK_Float:
+    return yaml::ScalarTraits<double>::mustQuote(ScalarStr);
+  case SK_Binary:
+  case SK_String:
+    return yaml::ScalarTraits<std::string>::mustQuote(ScalarStr);
+  }
+  llvm_unreachable("unrecognized ScalarKind");
+}
+
+const char *ScalarNode::IntTag = "!int";
+const char *ScalarNode::NilTag = "!nil";
+const char *ScalarNode::BooleanTag = "!bool";
+const char *ScalarNode::FloatTag = "!float";
+const char *ScalarNode::StringTag = "!str";
+const char *ScalarNode::BinaryTag = "!bin";
+
+StringRef ScalarNode::getYAMLTag() const {
+  switch (SKind) {
+  case SK_Int:
+    return IntTag;
+  case SK_UInt:
+    return IntTag;
+  case SK_Nil:
+    return NilTag;
+  case SK_Boolean:
+    return BooleanTag;
+  case SK_Float:
+    return FloatTag;
+  case SK_String:
+    return StringTag;
+  case SK_Binary:
+    return BinaryTag;
+  }
+  llvm_unreachable("unrecognized ScalarKind");
+}
+
+void ScalarNode::write(Writer &MPWriter) {
+  switch (SKind) {
+  case SK_Int:
+    MPWriter.write(IntValue);
+    break;
+  case SK_UInt:
+    MPWriter.write(UIntValue);
+    break;
+  case SK_Nil:
+    MPWriter.writeNil();
+    break;
+  case SK_Boolean:
+    MPWriter.write(BoolValue);
+    break;
+  case SK_Float:
+    MPWriter.write(FloatValue);
+    break;
+  case SK_String:
+    MPWriter.write(StringValue);
+    break;
+  case SK_Binary:
+    MPWriter.write(MemoryBufferRef(StringValue, ""));
+    break;
+  }
+}

llvm/trunk/unittests/BinaryFormat/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   BinaryFormat
   )
 
 add_llvm_unittest(BinaryFormatTests
   DwarfTest.cpp
   MachOTest.cpp
   MsgPackReaderTest.cpp
+  MsgPackTypesTest.cpp
   MsgPackWriterTest.cpp
   TestFileMagic.cpp
   )
 

llvm/trunk/unittests/BinaryFormat/MsgPackTypesTest.cpp

+//===- MsgPackTypesTest.cpp -------------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/BinaryFormat/MsgPackTypes.h"
+#include "gtest/gtest.h"
+
+using namespace llvm;
+using namespace msgpack;
+
+TEST(MsgPackTypes, TestReadInt) {
+  Reader MPReader(StringRef("\xd0\x00", 2));
+  auto OptNodeOrErr = Node::read(MPReader);
+  ASSERT_TRUE(static_cast<bool>(OptNodeOrErr));
+  ASSERT_TRUE(*OptNodeOrErr);
+  auto *S = dyn_cast<ScalarNode>((*OptNodeOrErr)->get());
+  ASSERT_TRUE(S);
+  ASSERT_EQ(S->getScalarKind(), ScalarNode::SK_Int);
+  ASSERT_EQ(S->getInt(), 0);
+}
+
+TEST(MsgPackTypes, TestReadArray) {
+  Reader MPReader(StringRef("\x92\xd0\x01\xc0"));
+  auto OptNodeOrErr = Node::read(MPReader);
+  ASSERT_TRUE(static_cast<bool>(OptNodeOrErr));
+  ASSERT_TRUE(*OptNodeOrErr);
+  auto *A = dyn_cast<ArrayNode>((*OptNodeOrErr)->get());
+  ASSERT_TRUE(A);
+  ASSERT_EQ(A->size(), 2u);
+  auto *SI = dyn_cast<ScalarNode>((*A)[0].get());
+  ASSERT_TRUE(SI);
+  ASSERT_EQ(SI->getScalarKind(), ScalarNode::SK_Int);
+  ASSERT_EQ(SI->getInt(), 1);
+  auto *SN = dyn_cast<ScalarNode>((*A)[1].get());
+  ASSERT_TRUE(SN);
+  ASSERT_EQ(SN->getScalarKind(), ScalarNode::SK_Nil);
+}
+
+TEST(MsgPackTypes, TestReadMap) {
+  Reader MPReader(StringRef("\x82\xa3"
+                            "foo"
+                            "\xd0\x01\xa3"
+                            "bar"
+                            "\xd0\x02"));
+  auto OptNodeOrErr = Node::read(MPReader);
+  ASSERT_TRUE(static_cast<bool>(OptNodeOrErr));
+  ASSERT_TRUE(*OptNodeOrErr);
+  auto *A = dyn_cast<MapNode>((*OptNodeOrErr)->get());
+  ASSERT_TRUE(A);
+  ASSERT_EQ(A->size(), 2u);
+  auto *FooS = dyn_cast<ScalarNode>((*A)["foo"].get());
+  ASSERT_TRUE(FooS);
+  ASSERT_EQ(FooS->getScalarKind(), ScalarNode::SK_Int);
+  ASSERT_EQ(FooS->getInt(), 1);
+  auto *BarS = dyn_cast<ScalarNode>((*A)["bar"].get());
+  ASSERT_TRUE(BarS);
+  ASSERT_EQ(BarS->getScalarKind(), ScalarNode::SK_Int);
+  ASSERT_EQ(BarS->getInt(), 2);
+}
+
+TEST(MsgPackTypes, TestWriteInt) {
+  std::string Buffer;
+  raw_string_ostream OStream(Buffer);
+  Writer MPWriter(OStream);
+  ScalarNode I(int64_t(1));
+  I.write(MPWriter);
+  ASSERT_EQ(OStream.str(), "\x01");
+}
+
+TEST(MsgPackTypes, TestWriteArray) {
+  std::string Buffer;
+  raw_string_ostream OStream(Buffer);
+  Writer MPWriter(OStream);
+  ArrayNode A;
+  A.push_back(std::make_shared<ScalarNode>(int64_t(1)));
+  A.push_back(std::make_shared<ScalarNode>());
+  A.write(MPWriter);
+  ASSERT_EQ(OStream.str(), "\x92\x01\xc0");
+}
+
+TEST(MsgPackTypes, TestWriteMap) {
+  std::string Buffer;
+  raw_string_ostream OStream(Buffer);
+  Writer MPWriter(OStream);
+  MapNode M;
+  M["foo"] = std::make_shared<ScalarNode>(int64_t(1));
+  M["bar"] = std::make_shared<ScalarNode>(int64_t(2));
+  M.write(MPWriter);
+  ASSERT_EQ(OStream.str(), "\x82\xa3"
+                           "foo"
+                           "\x01\xa3"
+                           "bar"
+                           "\x02");
+}
+
+TEST(MsgPackTypes, TestOutputYAMLArray) {
+  std::string Buffer;
+  raw_string_ostream OStream(Buffer);
+  yaml::Output yout(OStream);
+  ArrayNode A;
+  A.push_back(std::make_shared<ScalarNode>(int64_t(1)));
+  A.push_back(std::make_shared<ScalarNode>(int64_t(2)));
+  yout << A;
+  ASSERT_EQ(OStream.str(), "---\n- !int 1\n- !int 2\n...\n");
+}
+
+TEST(MsgPackTypes, TestInputYAMLArray) {
+  NodePtr RootNode;
+  yaml::Input yin("---\n- !int 1\n- !str 2\n...\n");
+  yin >> RootNode;
+  auto *A = dyn_cast<ArrayNode>(RootNode.get());
+  ASSERT_TRUE(A);
+  ASSERT_EQ(A->size(), 2u);
+  auto *SI = dyn_cast<ScalarNode>((*A)[0].get());
+  ASSERT_TRUE(SI);
+  ASSERT_EQ(SI->getScalarKind(), ScalarNode::SK_UInt);
+  ASSERT_EQ(SI->getUInt(), 1u);
+  auto *SS = dyn_cast<ScalarNode>((*A)[1].get());
+  ASSERT_TRUE(SS);
+  ASSERT_EQ(SS->getScalarKind(), ScalarNode::SK_String);
+  ASSERT_EQ(SS->getString(), "2");
+}
+
+TEST(MsgPackTypes, TestOutputYAMLMap) {
+  std::string Buffer;
+  raw_string_ostream OStream(Buffer);
+  yaml::Output yout(OStream);
+  MapNode M;
+  M["foo"] = std::make_shared<ScalarNode>(int64_t(1));
+  M["bar"] = std::make_shared<ScalarNode>(uint64_t(2));
+  auto N = std::make_shared<MapNode>();
+  (*N)["baz"] = std::make_shared<ScalarNode>(true);
+  M["qux"] = std::move(N);
+  yout << M;
+  ASSERT_EQ(OStream.str(), "---\nfoo:             !int 1\nbar:             "
+                           "!int 2\nqux:             \n  baz:             "
+                           "!bool true\n...\n");
+}
+
+TEST(MsgPackTypes, TestInputYAMLMap) {
+  NodePtr RootNode;
+  yaml::Input yin("---\nfoo: !int 1\nbaz: !str 2\n...\n");
+  yin >> RootNode;
+  auto *M = dyn_cast<MapNode>(RootNode.get());
+  ASSERT_TRUE(M);
+  ASSERT_EQ(M->size(), 2u);
+  auto *SI = dyn_cast<ScalarNode>((*M)["foo"].get());
+  ASSERT_TRUE(SI);
+  ASSERT_EQ(SI->getScalarKind(), ScalarNode::SK_UInt);
+  ASSERT_EQ(SI->getUInt(), 1u);
+  auto *SS = dyn_cast<ScalarNode>((*M)["baz"].get());
+  ASSERT_TRUE(SS);
+  ASSERT_EQ(SS->getScalarKind(), ScalarNode::SK_String);
+  ASSERT_EQ(SS->getString(), "2");
+}
+
+// Test that the document is parsed into a tree of shared_ptr where each node
+// can have multiple owners.
+TEST(MsgPackTypes, TestInputShared) {
+  yaml::Input yin("---\nfoo:\n  bar: !int 1\n...\n");
+  NodePtr InnerMap;
+  NodePtr IntNode;
+  {
+    {
+      {
+        NodePtr RootNode;
+        yin >> RootNode;
+        auto *M = dyn_cast<MapNode>(RootNode.get());
+        ASSERT_TRUE(M);
+        ASSERT_EQ(M->size(), 1u);
+        InnerMap = (*M)["foo"];
+      }
+      auto *N = dyn_cast<MapNode>(InnerMap.get());
+      ASSERT_TRUE(N);
+      ASSERT_EQ(N->size(), 1u);
+      IntNode = (*N)["bar"];
+    }
+    auto *S = dyn_cast<ScalarNode>(IntNode.get());
+    ASSERT_TRUE(S);
+    ASSERT_EQ(S->getScalarKind(), ScalarNode::SK_UInt);
+    ASSERT_EQ(S->getUInt(), 1u);
+  }
+}