Adds a json::Expr type to represent intermediate JSON expressions.

[sammccall]

Summary

WIP: needs the rest of the lit tests updated.

This form can be created with a nice clang-format-friendly literal syntax,
and gets escaping right. It knows how to call unparse() on our Protocol types.
All the places where we pass around JSON internally now use this type.

Object properties are sorted (stored as std::map) and so serialization is
canonicalized, with optional prettyprinting (triggered by a -pretty flag).
This makes the lit tests much nicer to read and somewhat nicer to debug.

Compared to the current approach, it has some efficiencies like avoiding copies
of string literals used as object keys, but is probably slower overall.
I think the code/test quality benefits are worth it.

This patch doesn't attempt to do anything about JSON *parsing*.
It takes direction from the proposal in this doc[1], but is limited in scope
and visibility, for now.
I am of half a mind just to use Expr as the target of a parser, and maybe do a
little string deduplication, but not bother with clever memory allocation.
That would be simple, and fast enough for clangd...
[1] https://docs.google.com/document/d/1OEF9IauWwNuSigZzvvbjc1cVS1uGHRyGTXaoy3DjqM4/edit

+cc d0k so he can tell me not to use std::map.

[sammccall]

Twine->StringRef for unowned strings, and remove Twine support.

(I had the owneship model all wrong, oops!)

[sammccall]

Add comment about the horrible mutable/const&& hack.

[ioeric]

The json:: code looks great. Would it make sense to add unit tests for it?

I'm not very familiar with the clangd code, but the clangd changes also look fine to me... someone else could take another look.

clangd/JSONExpr.cpp

200

nit: /*Radix=*/10

clangd/JSONExpr.h

40

s/arr/ary/ ?

same below.

[sammccall]

Add unittests for Expr, and remove the default constructor so that {} is a list.

[bkramer]

I think we can go ahead with this. Using std::map is also fine if your elements are heavy and you need sorting :)

[ioeric]

lgtm

[rwols]

clangd/JSONExpr.h

34

Why can't unparse be a free function?

117

Maybe use std::addressof?

[sammccall]

Update all lit tests. Unfortunately the completion ones can't use prettyprinting because I can't find a way to get CHECK-DAG-like behavior for multiline blocks.

[sammccall]

clangd/JSONExpr.h

34

This seems like a good idea (though not without downsides - I like that T::unparse doesn't allow accidental conversions, for example).

However, the current scheme (all the existing code in Protocol) is T::unparse, and I'd rather not make this patch even bigger, so I've added a FIXME comment. This is a mechanically easy followup change if we want it.

117

You mean like this?

new (template std::addressof(as<T>)()) T(std::forward<U>(V)...);

This certainly seems harder to read, and IIUC it guards against T having an overloaded operator&, which it never does (bool, double, StringRef, std::string, Object, Array). So I'm not sure we need the future-proofing here.

[sammccall]

Merge with upstream changes and clang-format

[arphaman]

Please CC cfe-commits when submitting clangd patches.

[sammccall]

In D39435#916907, @arphaman wrote:

Please CC cfe-commits when submitting clangd patches.

Sigh, I'm really sorry - I do try not to forget this.
Is there a way we can have this happen automatically? It looks like an admin could add a Herald rule, I guess there's a reason this isn't set up.

Please feel free to leave comments, will address them.

[arphaman]

In D39435#916928, @sammccall wrote:

In D39435#916907, @arphaman wrote:

Please CC cfe-commits when submitting clangd patches.

Sigh, I'm really sorry - I do try not to forget this.
Is there a way we can have this happen automatically? It looks like an admin could add a Herald rule, I guess there's a reason this isn't set up.

I don't think so. I think there have been some discussions about this, but I'm not sure if they led to something yet.

Please feel free to leave comments, will address them.

[malaperle]

Hi Sam! After this patch, I see odd behavior when using Clangd in both VS Code and Eclipse. It looks like basically nothing happens after the initialize response:
<-- {"jsonrpc":"2.0","id":0,"method":"initialize","params":{.....}}
--> {"id":0,"jsonrpc":"2.0","result":{...}}

Have you seen this behavior? I haven't had the time to debug yet. If it is a real bug and not just on my machine, it's a bit alarming that the tests didn't catch it!

[malaperle]

clang-tools-extra/trunk/clangd/ClangdLSPServer.cpp

39 ↗

(On Diff #121736)

I think "capabilities" was removed by mistake?

clangd/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   Support
   )
 
 add_clang_library(clangDaemon
   ClangdLSPServer.cpp
   ClangdServer.cpp
   ClangdUnit.cpp
   ClangdUnitStore.cpp
   DraftStore.cpp
   GlobalCompilationDatabase.cpp
   JSONRPCDispatcher.cpp
+  JSONExpr.cpp
   Logger.cpp
   Protocol.cpp
   ProtocolHandlers.cpp
   Trace.cpp
 
   LINK_LIBS
   clangAST
   clangBasic

clangd/ClangdLSPServer.cpp

 #include "ClangdLSPServer.h"
 #include "JSONRPCDispatcher.h"
 
 using namespace clang::clangd;
 using namespace clang;
 
 namespace {
 
-std::string
+json::ary
 replacementsToEdits(StringRef Code,
                     const std::vector<tooling::Replacement> &Replacements) {
   // Turn the replacements into the format specified by the Language Server
   // Protocol. Fuse them into one big JSON array.
-  std::string Edits;
+  json::ary Edits;
   for (auto &R : Replacements) {
     Range ReplacementRange = {
         offsetToPosition(Code, R.getOffset()),
         offsetToPosition(Code, R.getOffset() + R.getLength())};
     TextEdit TE = {ReplacementRange, R.getReplacementText()};
-    Edits += TextEdit::unparse(TE);
-    Edits += ',';
+    Edits.push_back(TE);
   }
-  if (!Edits.empty())
-    Edits.pop_back();
-
   return Edits;
 }
 
 } // namespace
 
 void ClangdLSPServer::onInitialize(Ctx C, InitializeParams &Params) {
-  C.reply(
-      R"({"capabilities":{
-          "textDocumentSync": 1,
-          "documentFormattingProvider": true,
-          "documentRangeFormattingProvider": true,
-          "documentOnTypeFormattingProvider": {"firstTriggerCharacter":"}","moreTriggerCharacter":[]},
-          "codeActionProvider": true,
-          "completionProvider": {"resolveProvider": false, "triggerCharacters": [".",">",":"]},
-          "signatureHelpProvider": {"triggerCharacters": ["(",","]},
-          "definitionProvider": true
-        }})");
+  C.reply(json::obj{
+      {"textDocumentSync", 1},
+      {"documentFormattingProvider", true},
+      {"documentRangeFormattingProvider", true},
+      {"documentOnTypeFormattingProvider",
+       json::obj{
+           {"firstTriggerCharacter", "}"},
+           {"moreTriggerCharacter", {}},
+       }},
+      {"codeActionProvider", true},
+      {"completionProvider",
+       json::obj{
+           {"resolveProvider", false},
+           {"triggerCharacters", {".", ">", ":"}},
+       }},
+      {"signatureHelpProvider",
+       json::obj{
+           {"triggerCharacters", {"(", ","}},
+       }},
+      {"definitionProvider", true},
+  });
   if (Params.rootUri && !Params.rootUri->file.empty())
     Server.setRootPath(Params.rootUri->file);
   else if (Params.rootPath && !Params.rootPath->empty())
     Server.setRootPath(*Params.rootPath);
 }
 
 void ClangdLSPServer::onShutdown(Ctx C, ShutdownParams &Params) {
   // Do essentially nothing, just say we're ready to exit.
   ShutdownRequestReceived = true;
-  C.reply("null");
+  C.reply(nullptr);
 }
 
 void ClangdLSPServer::onExit(Ctx C, ExitParams &Params) { IsDone = true; }
 
 void ClangdLSPServer::onDocumentDidOpen(Ctx C,
                                         DidOpenTextDocumentParams &Params) {
   if (Params.metadata && !Params.metadata->extraFlags.empty())
     CDB.setExtraFlagsForFile(Params.textDocument.uri.file,
void ClangdLSPServer::onDocumentDidClose(Ctx C,
                                          DidCloseTextDocumentParams &Params) {
   Server.removeDocument(Params.textDocument.uri.file);
 }
 
 void ClangdLSPServer::onDocumentOnTypeFormatting(
     Ctx C, DocumentOnTypeFormattingParams &Params) {
   auto File = Params.textDocument.uri.file;
   std::string Code = Server.getDocument(File);
-  std::string Edits =
-      replacementsToEdits(Code, Server.formatOnType(File, Params.position));
-  C.reply("[" + Edits + "]");
+  C.reply(
+      replacementsToEdits(Code, Server.formatOnType(File, Params.position)));
 }
 
 void ClangdLSPServer::onDocumentRangeFormatting(
     Ctx C, DocumentRangeFormattingParams &Params) {
   auto File = Params.textDocument.uri.file;
   std::string Code = Server.getDocument(File);
-  std::string Edits =
-      replacementsToEdits(Code, Server.formatRange(File, Params.range));
-  C.reply("[" + Edits + "]");
+  C.reply(replacementsToEdits(Code, Server.formatRange(File, Params.range)));
 }
 
 void ClangdLSPServer::onDocumentFormatting(Ctx C,
                                            DocumentFormattingParams &Params) {
   auto File = Params.textDocument.uri.file;
   std::string Code = Server.getDocument(File);
-  std::string Edits = replacementsToEdits(Code, Server.formatFile(File));
-  C.reply("[" + Edits + "]");
+  C.reply(replacementsToEdits(Code, Server.formatFile(File)));
 }
 
 void ClangdLSPServer::onCodeAction(Ctx C, CodeActionParams &Params) {
   // We provide a code action for each diagnostic at the requested location
   // which has FixIts available.
   std::string Code = Server.getDocument(Params.textDocument.uri.file);
-  std::string Commands;
+  json::ary Commands;
   for (Diagnostic &D : Params.context.diagnostics) {
     std::vector<clang::tooling::Replacement> Fixes =
         getFixIts(Params.textDocument.uri.file, D);
-    std::string Edits = replacementsToEdits(Code, Fixes);
-
+    auto Edits = replacementsToEdits(Code, Fixes);
     if (!Edits.empty())
-      Commands +=
-          R"({"title":"Apply FixIt ')" + llvm::yaml::escape(D.message) +
-          R"('", "command": "clangd.applyFix", "arguments": [")" +
-          llvm::yaml::escape(Params.textDocument.uri.uri) +
-          R"(", [)" + Edits +
-          R"(]]},)";
-  }
-  if (!Commands.empty())
-    Commands.pop_back();
-  C.reply("[" + Commands + "]");
+      Commands.push_back(json::obj{
+          {"title", llvm::formatv("Apply FixIt {0}", D.message)},
+          {"command", "clangd.applyFix"},
+          {"arguments", {Params.textDocument.uri.uri, std::move(Edits)}},
+      });
+  }
+  C.reply(std::move(Commands));
 }
 
 void ClangdLSPServer::onCompletion(Ctx C, TextDocumentPositionParams &Params) {
   auto Items = Server
                    .codeComplete(Params.textDocument.uri.file,
                                  Position{Params.position.line,
                                           Params.position.character})
                    .get() // FIXME(ibiryukov): This could be made async if we
                           // had an API that would allow to attach callbacks to
                           // futures returned by ClangdServer.
                    .Value;
-
-  std::string Completions;
-  for (const auto &Item : Items) {
-    Completions += CompletionItem::unparse(Item);
-    Completions += ",";
-  }
-  if (!Completions.empty())
-    Completions.pop_back();
-  C.reply("[" + Completions + "]");
+  C.reply(json::ary(Items));
 }
 
 void ClangdLSPServer::onSignatureHelp(Ctx C,
                                       TextDocumentPositionParams &Params) {
   auto SignatureHelp = Server.signatureHelp(
       Params.textDocument.uri.file,
       Position{Params.position.line, Params.position.character});
   if (!SignatureHelp)
     return C.replyError(-32602, llvm::toString(SignatureHelp.takeError()));
-  C.reply(SignatureHelp::unparse(SignatureHelp->Value));
+  C.reply(SignatureHelp->Value);
 }
 
 void ClangdLSPServer::onGoToDefinition(Ctx C,
                                        TextDocumentPositionParams &Params) {
   auto Items = Server.findDefinitions(
       Params.textDocument.uri.file,
       Position{Params.position.line, Params.position.character});
   if (!Items)
     return C.replyError(-32602, llvm::toString(Items.takeError()));
-
-  std::string Locations;
-  for (const auto &Item : Items->Value) {
-    Locations += Location::unparse(Item);
-    Locations += ",";
-  }
-  if (!Locations.empty())
-    Locations.pop_back();
-  C.reply("[" + Locations + "]");
+  C.reply(json::ary(Items->Value));
 }
 
 void ClangdLSPServer::onSwitchSourceHeader(Ctx C,
                                            TextDocumentIdentifier &Params) {
   llvm::Optional<Path> Result = Server.switchSourceHeader(Params.uri.file);
   std::string ResultUri;
-  C.reply(Result ? URI::unparse(URI::fromFile(*Result)) : R"("")");
+  C.reply(Result ? URI::fromFile(*Result).uri : "");
 }
 
 ClangdLSPServer::ClangdLSPServer(JSONOutput &Out, unsigned AsyncThreadsCount,
                                  bool SnippetCompletions,
                                  llvm::Optional<StringRef> ResourceDir,
                                  llvm::Optional<Path> CompileCommandsDir)
     : Out(Out), CDB(/*Logger=*/Out, std::move(CompileCommandsDir)),
       Server(CDB, /*DiagConsumer=*/*this, FSProvider, AsyncThreadsCount,
ClangdLSPServer::getFixIts(StringRef File, const clangd::Diagnostic &D) {
   if (FixItsIter == DiagToFixItsMap.end())
     return {};
 
   return FixItsIter->second;
 }
 
 void ClangdLSPServer::onDiagnosticsReady(
     PathRef File, Tagged<std::vector<DiagWithFixIts>> Diagnostics) {
-  std::string DiagnosticsJSON;
+  json::ary DiagnosticsJSON;
 
   DiagnosticToReplacementMap LocalFixIts; // Temporary storage
   for (auto &DiagWithFixes : Diagnostics.Value) {
     auto Diag = DiagWithFixes.Diag;
-    DiagnosticsJSON +=
-        R"({"range":)" + Range::unparse(Diag.range) +
-        R"(,"severity":)" + std::to_string(Diag.severity) +
-        R"(,"message":")" + llvm::yaml::escape(Diag.message) +
-        R"("},)";
-
+    DiagnosticsJSON.push_back(json::obj{
+        {"range", Diag.range},
+        {"severity", Diag.severity},
+        {"message", Diag.message},
+    });
     // We convert to Replacements to become independent of the SourceManager.
     auto &FixItsForDiagnostic = LocalFixIts[Diag];
     std::copy(DiagWithFixes.FixIts.begin(), DiagWithFixes.FixIts.end(),
               std::back_inserter(FixItsForDiagnostic));
   }
 
   // Cache FixIts
   {
     // FIXME(ibiryukov): should be deleted when documents are removed
     std::lock_guard<std::mutex> Lock(FixItsMutex);
     FixItsMap[File] = LocalFixIts;
   }
 
   // Publish diagnostics.
-  if (!DiagnosticsJSON.empty())
-    DiagnosticsJSON.pop_back(); // Drop trailing comma.
-  Out.writeMessage(
-      R"({"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"uri":")" +
-      URI::fromFile(File).uri + R"(","diagnostics":[)" + DiagnosticsJSON +
-      R"(]}})");
+  Out.writeMessage(json::obj{
+      {"jsonrpc", "2.0"},
+      {"method", "textDocument/publishDiagnostics"},
+      {"params",
+       json::obj{
+           {"uri", URI::fromFile(File)},
+           {"diagnostics", std::move(DiagnosticsJSON)},
+       }},
+  });
 }

clangd/JSONExpr.h

+//===--- JSONExpr.h - composable JSON expressions ---------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===---------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_JSON_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_JSON_H
+
+#include <map>
+
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/FormatVariadic.h"
+#include "llvm/Support/raw_ostream.h"
+
+namespace clang {
+namespace clangd {
+namespace json {
+
+// An Expr is an opaque temporary JSON structure used to compose documents.
+// They can be copied, but should generally be moved.
+//
+// You can implicitly construct literals from:
+//   - strings: std::string, SmallString, formatv, StringRef, char*
+//              (char*, and StringRef are references, not copies!)
+//   - numbers
+//   - booleans
+//   - null: nullptr
+//   - arrays: {"foo", 42.0, false}
+//   - serializable things: any T with a T::unparse(const T&) -> Expr
+//
+// They can also be constructed from object/array helpers:
+//   - json::obj is a type like map<StringExpr, Expr>
+//   - json::ary is a type like vector<Expr>
+// These can be list-initialized, or used to build up collections in a loop.
+// json::ary(Collection) converts all items in a collection to Exprs.
+//
+// Exprs can be serialized to JSON:
+//   1) raw_ostream << Expr                    // Basic formatting.
+//   2) raw_ostream << formatv("{0}", Expr)    // Basic formatting.
+//   3) raw_ostream << formatv("{0:2}", Expr)  // Pretty-print with indent 2.
+class Expr {
+public:
+  class Object;
+  class ObjectKey;
+  class Array;
+
+  // It would be nice to have Expr() be null. But that would make {} null too...
+  Expr(const Expr &M) { copyFrom(M); }
+  Expr(Expr &&M) { moveFrom(std::move(M)); }
+  // "cheating" move-constructor for moving from initializer_list.
+  Expr(const Expr &&M) { moveFrom(std::move(M)); }
+  Expr(std::initializer_list<Expr> Elements) : Expr(Array(Elements)) {}
+  Expr(Array &&Elements) : Type(T_Array) { create<Array>(std::move(Elements)); }
+  Expr(Object &&Properties) : Type(T_Object) {
+    create<Object>(std::move(Properties));
+  }
+  // Strings: types with value semantics.
+  Expr(std::string &&V) : Type(T_String) { create<std::string>(std::move(V)); }
+  Expr(const std::string &V) : Type(T_String) { create<std::string>(V); }
+  Expr(const llvm::SmallVectorImpl<char> &V) : Type(T_String) {
+    create<std::string>(V.begin(), V.end());
+  }
+  Expr(const llvm::formatv_object_base &V) : Expr(V.str()){};
+  // Strings: types with reference semantics.
+  Expr(llvm::StringRef V) : Type(T_StringRef) { create<llvm::StringRef>(V); }
+  Expr(const char *V) : Type(T_StringRef) { create<llvm::StringRef>(V); }
+  Expr(std::nullptr_t) : Type(T_Null) {}
+  // Prevent implicit conversions to boolean.
+  template <typename T, typename = typename std::enable_if<
+                            std::is_same<T, bool>::value>::type>
+  Expr(T B) : Type(T_Boolean) {
+    create<bool>(B);
+  }
+  // Numbers: arithmetic types that are not boolean.
+  template <
+      typename T,
+      typename = typename std::enable_if<std::is_arithmetic<T>::value>::type,
+      typename = typename std::enable_if<std::integral_constant<
+          bool, !std::is_same<T, bool>::value>::value>::type>
+  Expr(T D) : Type(T_Number) {
+    create<double>(D);
+  }
+  // Types with a static T::unparse function returning an Expr.
+  template <typename T,
+            typename = typename std::enable_if<std::is_same<
+                Expr, decltype(T::unparse(*(const T *)nullptr))>::value>>
+  Expr(const T &V) : Expr(T::unparse(V)) {}
+
+  Expr &operator=(const Expr &M) {
+    destroy();
+    copyFrom(M);
+    return *this;
+  }
+  Expr &operator=(Expr &&M) {
+    destroy();
+    moveFrom(std::move(M));
+    return *this;
+  }
+  ~Expr() { destroy(); }
+
+  friend llvm::raw_ostream &operator<<(llvm::raw_ostream &, const Expr &);
+
+private:
+  void destroy();
+  void copyFrom(const Expr &M);
+  // We allow moving from *const* Exprs, by marking all members as mutable!
+  // This hack is needed to support initializer-list syntax efficiently.
+  // (std::initializer_list<T> is a container of const T).
+  void moveFrom(const Expr &&M);
+
+  template <typename T, typename... U> void create(U &&... V) {
+    new (&as<T>()) T(std::forward<U>(V)...);
+  }
+  template <typename T> T &as() const {
+    return *reinterpret_cast<T *>(Union.buffer);
+  }
+
+  template <typename Indenter>
+  void print(llvm::raw_ostream &, const Indenter &) const;
+  friend struct llvm::format_provider<clang::clangd::json::Expr>;
+
+  enum ExprType : char {
+    T_Null,
+    T_Boolean,
+    T_Number,
+    T_StringRef,
+    T_String,
+    T_Object,
+    T_Array,
+  };
+  mutable ExprType Type;
+
+public:
+  // ObjectKey is a used to capture keys in Expr::Objects. It's like Expr but:
+  //   - only strings are allowed
+  //   - it's copyable (for std::map)
+  //   - we're slightly more eager to copy, to allow efficient key compares
+  //   - it's optimized for the string literal case (Owned == nullptr)
+  class ObjectKey {
+  public:
+    ObjectKey(const char *S) : Data(S) {}
+    ObjectKey(llvm::StringRef S) : Data(S) {}
+    ObjectKey(std::string &&V)
+        : Owned(new std::string(std::move(V))), Data(*Owned) {}
+    ObjectKey(const std::string &V) : Owned(new std::string(V)), Data(*Owned) {}
+    ObjectKey(const llvm::SmallVectorImpl<char> &V)
+        : ObjectKey(std::string(V.begin(), V.end())) {}
+    ObjectKey(const llvm::formatv_object_base &V) : ObjectKey(V.str()) {}
+
+    ObjectKey(const ObjectKey &C) { *this = C; }
+    ObjectKey(ObjectKey &&C) = default;
+    ObjectKey &operator=(const ObjectKey &C) {
+      if (C.Owned) {
+        Owned.reset(new std::string(*C.Owned));
+        Data = *Owned;
+      } else {
+        Data = C.Data;
+      }
+      return *this;
+    }
+    ObjectKey &operator=(ObjectKey &&) = default;
+
+    operator llvm::StringRef() const { return Data; }
+
+    friend bool operator<(const ObjectKey &L, const ObjectKey &R) {
+      return L.Data < R.Data;
+    }
+
+    // "cheating" move-constructor for moving from initializer_list.
+    ObjectKey(const ObjectKey &&V) {
+      Owned = std::move(V.Owned);
+      Data = V.Data;
+    }
+
+  private:
+    mutable std::unique_ptr<std::string> Owned; // mutable for cheating.
+    llvm::StringRef Data;
+  };
+
+  class Object : public std::map<ObjectKey, Expr> {
+  public:
+    explicit Object() {}
+    // Use a custom struct for list-init, because pair forces extra copies.
+    struct KV;
+    explicit Object(std::initializer_list<KV> Properties);
+
+    // Allow [] as if Expr was default-constructible as null.
+    Expr &operator[](const ObjectKey &K) {
+      return emplace(K, Expr(nullptr)).first->second;
+    }
+    Expr &operator[](ObjectKey &&K) {
+      return emplace(std::move(K), Expr(nullptr)).first->second;
+    }
+  };
+
+  class Array : public std::vector<Expr> {
+  public:
+    explicit Array() {}
+    explicit Array(std::initializer_list<Expr> Elements) {
+      reserve(Elements.size());
+      for (const Expr &V : Elements)
+        emplace_back(std::move(V));
+    };
+    template <typename Collection> explicit Array(const Collection &C) {
+      for (const auto &V : C)
+        emplace_back(V);
+    }
+  };
+
+private:
+  mutable llvm::AlignedCharArrayUnion<bool, double, llvm::StringRef,
+                                      std::string, Array, Object>
+      Union;
+};
+
+struct Expr::Object::KV {
+  ObjectKey K;
+  Expr V;
+};
+
+inline Expr::Object::Object(std::initializer_list<KV> Properties) {
+  for (const auto &P : Properties)
+    emplace(std::move(P.K), std::move(P.V));
+}
+
+// Give Expr::{Object,Array} more convenient names for literal use.
+using obj = Expr::Object;
+using ary = Expr::Array;
+
+} // namespace json
+} // namespace clangd
+} // namespace clang
+
+namespace llvm {
+template <> struct format_provider<clang::clangd::json::Expr> {
+  static void format(const clang::clangd::json::Expr &, raw_ostream &,
+                     StringRef);
+};
+} // namespace llvm
+
+#endif

clangd/JSONExpr.cpp

+#include "JSONExpr.h"
+
+#include "llvm/Support/Format.h"
+
+namespace clang {
+namespace clangd {
+namespace json {
+using namespace llvm;
+
+void Expr::copyFrom(const Expr &M) {
+  Type = M.Type;
+  switch (Type) {
+  case T_Null:
+  case T_Boolean:
+  case T_Number:
+    memcpy(Union.buffer, M.Union.buffer, sizeof(Union.buffer));
+    break;
+  case T_StringRef:
+    create<StringRef>(M.as<StringRef>());
+    break;
+  case T_String:
+    create<std::string>(M.as<std::string>());
+    break;
+  case T_Object:
+    create<Object>(M.as<Object>());
+    break;
+  case T_Array:
+    create<Array>(M.as<Array>());
+    break;
+  }
+}
+
+void Expr::moveFrom(const Expr &&M) {
+  Type = M.Type;
+  switch (Type) {
+  case T_Null:
+  case T_Boolean:
+  case T_Number:
+    memcpy(Union.buffer, M.Union.buffer, sizeof(Union.buffer));
+    break;
+  case T_StringRef:
+    create<StringRef>(M.as<StringRef>());
+    break;
+  case T_String:
+    create<std::string>(std::move(M.as<std::string>()));
+    M.Type = T_Null;
+    break;
+  case T_Object:
+    create<Object>(std::move(M.as<Object>()));
+    M.Type = T_Null;
+    break;
+  case T_Array:
+    create<Array>(std::move(M.as<Array>()));
+    M.Type = T_Null;
+    break;
+  }
+}
+
+void Expr::destroy() {
+  switch (Type) {
+  case T_Null:
+  case T_Boolean:
+  case T_Number:
+    break;
+  case T_StringRef:
+    as<StringRef>().~StringRef();
+    break;
+  case T_String:
+    as<std::string>().~basic_string();
+    break;
+  case T_Object:
+    as<Object>().~Object();
+    break;
+  case T_Array:
+    as<Array>().~Array();
+    break;
+  }
+}
+
+} // namespace json
+} // namespace clangd
+} // namespace clang
+
+namespace {
+void quote(llvm::raw_ostream &OS, llvm::StringRef S) {
+  OS << '\"';
+  for (unsigned char C : S) {
+    if (C == 0x22 || C == 0x5C)
+      OS << '\\';
+    if (C >= 0x20) {
+      OS << C;
+      continue;
+    }
+    OS << '\\';
+    switch (C) {
+    // A few characters are common enough to make short escapes worthwhile.
+    case '\t':
+      OS << 't';
+      break;
+    case '\n':
+      OS << 'n';
+      break;
+    case '\r':
+      OS << 'r';
+      break;
+    default:
+      OS << 'u';
+      llvm::write_hex(OS, C, llvm::HexPrintStyle::Lower, 4);
+      break;
+    }
+  }
+  OS << '\"';
+}
+
+enum IndenterAction {
+  Indent,
+  Outdent,
+  Newline,
+  Space,
+};
+} // namespace
+
+// Prints JSON. The indenter can be used to control formatting.
+template <typename Indenter>
+void clang::clangd::json::Expr::print(raw_ostream &OS,
+                                      const Indenter &I) const {
+  switch (Type) {
+  case T_Null:
+    OS << "null";
+    break;
+  case T_Boolean:
+    OS << (as<bool>() ? "true" : "false");
+    break;
+  case T_Number:
+    OS << format("%g", as<double>());
+    break;
+  case T_StringRef:
+    quote(OS, as<StringRef>());
+    break;
+  case T_String:
+    quote(OS, as<std::string>());
+    break;
+  case T_Object: {
+    bool Comma = false;
+    OS << '{';
+    I(Indent);
+    for (const auto &P : as<Expr::Object>()) {
+      if (Comma)
+        OS << ',';
+      Comma = true;
+      I(Newline);
+      quote(OS, P.first);
+      OS << ':';
+      I(Space);
+      P.second.print(OS, I);
+    }
+    I(Outdent);
+    if (Comma)
+      I(Newline);
+    OS << '}';
+    break;
+  }
+  case T_Array: {
+    bool Comma = false;
+    OS << '[';
+    I(Indent);
+    for (const auto &E : as<Expr::Array>()) {
+      if (Comma)
+        OS << ',';
+      Comma = true;
+      I(Newline);
+      E.print(OS, I);
+    }
+    I(Outdent);
+    if (Comma)
+      I(Newline);
+    OS << ']';
+    break;
+  }
+  }
+}
+
+llvm::raw_ostream &clang::clangd::json::operator<<(raw_ostream &OS,
+                                                   const Expr &E) {
+  E.print(OS, [](IndenterAction A) { /*ignore*/ });
+  return OS;
+}
+
+void llvm::format_provider<clang::clangd::json::Expr>::format(
+    const clang::clangd::json::Expr &E, raw_ostream &OS, StringRef Options) {
+  if (Options.empty()) {
+    OS << E;
+    return;
+  }
+  unsigned IndentAmount = 0;
+  if (Options.getAsInteger(/*Radix=*/10, IndentAmount))
+    assert(false && "json::Expr format options should be an integer");
+  unsigned IndentLevel = 0;
+  E.print(OS, [&](IndenterAction A) {
+    switch (A) {
+    case Newline:
+      OS << '\n';
+      OS.indent(IndentLevel);
+      break;
+    case Space:
+      OS << ' ';
+      break;
+    case Indent:
+      IndentLevel += IndentAmount;
+      break;
+    case Outdent:
+      IndentLevel -= IndentAmount;
+      break;
+    };
+  });
+}

clangd/JSONRPCDispatcher.h

 //===--- JSONRPCDispatcher.h - Main JSON parser entry point -----*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_JSONRPCDISPATCHER_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_JSONRPCDISPATCHER_H
 
+#include "JSONExpr.h"
 #include "Logger.h"
 #include "clang/Basic/LLVM.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/Support/YAMLParser.h"
 #include <iosfwd>
 #include <mutex>
 
 namespace clang {
 namespace clangd {
 
 /// Encapsulates output and logs streams and provides thread-safe access to
 /// them.
 class JSONOutput : public Logger {
 public:
   JSONOutput(llvm::raw_ostream &Outs, llvm::raw_ostream &Logs,
-             llvm::raw_ostream *InputMirror = nullptr)
-      : Outs(Outs), Logs(Logs), InputMirror(InputMirror) {}
+             llvm::raw_ostream *InputMirror = nullptr, bool Pretty = false)
+      : Outs(Outs), Logs(Logs), InputMirror(InputMirror), Pretty(Pretty) {}
 
   /// Emit a JSONRPC message.
-  void writeMessage(const Twine &Message);
+  void writeMessage(const json::Expr &Result);
 
   /// Write to the logging stream.
   /// No newline is implicitly added. (TODO: we should fix this!)
   void log(const Twine &Message) override;
 
   /// Mirror \p Message into InputMirror stream. Does nothing if InputMirror is
   /// null.
   /// Unlike other methods of JSONOutput, mirrorInput is not thread-safe.
   void mirrorInput(const Twine &Message);
 
 private:
   llvm::raw_ostream &Outs;
   llvm::raw_ostream &Logs;
   llvm::raw_ostream *InputMirror;
+  bool Pretty;
 
   std::mutex StreamMutex;
 };
 
 /// Context object passed to handlers to allow replies.
 class RequestContext {
 public:
-  RequestContext(JSONOutput &Out, StringRef ID) : Out(Out), ID(ID) {}
+  RequestContext(JSONOutput &Out, llvm::Optional<json::Expr> ID)
+      : Out(Out), ID(std::move(ID)) {}
 
-  /// Sends a successful reply. Result should be well-formed JSON.
-  void reply(const Twine &Result);
+  /// Sends a successful reply.
+  void reply(json::Expr &&Result);
   /// Sends an error response to the client, and logs it.
   void replyError(int code, const llvm::StringRef &Message);
 
 private:
   JSONOutput &Out;
-  llvm::SmallString<64> ID; // Valid JSON, or empty for a notification.
+  llvm::Optional<json::Expr> ID;
 };
 
 /// Main JSONRPC entry point. This parses the JSONRPC "header" and calls the
 /// registered Handler for the method received.
 class JSONRPCDispatcher {
 public:
   // A handler responds to requests for a particular method name.
   using Handler =

clangd/JSONRPCDispatcher.cpp

 //===--- JSONRPCDispatcher.cpp - Main JSON parser entry point -------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "JSONRPCDispatcher.h"
+#include "JSONExpr.h"
 #include "ProtocolHandlers.h"
 #include "Trace.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/YAMLParser.h"
 #include <istream>
 
 using namespace clang;
 using namespace clangd;
 
-void JSONOutput::writeMessage(const Twine &Message) {
-  llvm::SmallString<128> Storage;
-  StringRef M = Message.toStringRef(Storage);
+void JSONOutput::writeMessage(const json::Expr &Message) {
+  std::string S;
+  llvm::raw_string_ostream OS(S);
+  if (Pretty)
+    OS << llvm::formatv("{0:2}", Message);
+  else
+    OS << Message;
+  OS.flush();
 
   std::lock_guard<std::mutex> Guard(StreamMutex);
   // Log without headers.
-  Logs << "--> " << M << '\n';
+  Logs << "--> " << S << '\n';
   Logs.flush();
 
   // Emit message with header.
-  Outs << "Content-Length: " << M.size() << "\r\n\r\n" << M;
+  Outs << "Content-Length: " << S.size() << "\r\n\r\n" << S;
   Outs.flush();
 }
 
 void JSONOutput::log(const Twine &Message) {
   trace::log(Message);
   std::lock_guard<std::mutex> Guard(StreamMutex);
   Logs << Message;
   Logs.flush();
 }
 
 void JSONOutput::mirrorInput(const Twine &Message) {
   if (!InputMirror)
     return;
 
   *InputMirror << Message;
   InputMirror->flush();
 }
 
-void RequestContext::reply(const llvm::Twine &Result) {
-  if (ID.empty()) {
+void RequestContext::reply(json::Expr &&Result) {
+  if (!ID) {
     Out.log("Attempted to reply to a notification!\n");
     return;
   }
-  Out.writeMessage(llvm::Twine(R"({"jsonrpc":"2.0","id":)") + ID +
-                   R"(,"result":)" + Result + "}");
+  Out.writeMessage(json::obj{
+      {"jsonrpc", "2.0"},
+      {"id", *ID},
+      {"result", std::move(Result)},
+  });
 }
 
 void RequestContext::replyError(int code, const llvm::StringRef &Message) {
   Out.log("Error " + llvm::Twine(code) + ": " + Message + "\n");
-  if (!ID.empty()) {
-    Out.writeMessage(llvm::Twine(R"({"jsonrpc":"2.0","id":)") + ID +
-                     R"(,"error":{"code":)" + llvm::Twine(code) +
-                     R"(,"message":")" + llvm::yaml::escape(Message) +
-                     R"("}})");
+  if (ID) {
+    Out.writeMessage(json::obj{
+        {"jsonrpc", "2.0"},
+        {"id", *ID},
+        {"result", Message},
+    });
   }
 }
 
 void JSONRPCDispatcher::registerHandler(StringRef Method, Handler H) {
   assert(!Handlers.count(Method) && "Handler already registered!");
   Handlers[Method] = std::move(H);
 }
 
 static void
 callHandler(const llvm::StringMap<JSONRPCDispatcher::Handler> &Handlers,
-            llvm::yaml::ScalarNode *Method, llvm::yaml::ScalarNode *Id,
+            llvm::yaml::ScalarNode *Method, llvm::Optional<json::Expr> ID,
             llvm::yaml::MappingNode *Params,
             const JSONRPCDispatcher::Handler &UnknownHandler, JSONOutput &Out) {
   llvm::SmallString<64> MethodStorage;
   llvm::StringRef MethodStr = Method->getValue(MethodStorage);
   auto I = Handlers.find(MethodStr);
   auto &Handler = I != Handlers.end() ? I->second : UnknownHandler;
   trace::Span Tracer(MethodStr);
-  Handler(RequestContext(Out, Id ? Id->getRawValue() : ""), Params);
+  Handler(RequestContext(Out, std::move(ID)), Params);
 }
 
 bool JSONRPCDispatcher::call(StringRef Content, JSONOutput &Out) const {
   llvm::SourceMgr SM;
   llvm::yaml::Stream YAMLStream(Content, SM);
 
   auto Doc = YAMLStream.begin();
   if (Doc == YAMLStream.end())
     return false;
 
   auto *Object = dyn_cast_or_null<llvm::yaml::MappingNode>(Doc->getRoot());
   if (!Object)
     return false;
 
   llvm::yaml::ScalarNode *Version = nullptr;
   llvm::yaml::ScalarNode *Method = nullptr;
   llvm::yaml::MappingNode *Params = nullptr;
-  llvm::yaml::ScalarNode *Id = nullptr;
+  llvm::Optional<json::Expr> ID;
   for (auto &NextKeyValue : *Object) {
     auto *KeyString =
         dyn_cast_or_null<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
     if (!KeyString)
       return false;
 
     llvm::SmallString<10> KeyStorage;
     StringRef KeyValue = KeyString->getValue(KeyStorage);
     llvm::yaml::Node *Value = NextKeyValue.getValue();
     if (!Value)
       return false;
 
     if (KeyValue == "jsonrpc") {
       // This should be "2.0". Always.
       Version = dyn_cast<llvm::yaml::ScalarNode>(Value);
       if (!Version || Version->getRawValue() != "\"2.0\"")
         return false;
     } else if (KeyValue == "method") {
       Method = dyn_cast<llvm::yaml::ScalarNode>(Value);
     } else if (KeyValue == "id") {
-      Id = dyn_cast<llvm::yaml::ScalarNode>(Value);
+      // ID may be either a string or a number.
+      if (auto *IdNode = dyn_cast<llvm::yaml::ScalarNode>(Value)) {
+        llvm::SmallString<32> S;
+        llvm::StringRef V = IdNode->getValue(S);
+        if (IdNode->getRawValue().startswith("\"")) {
+          ID.emplace(V.str());
+        } else {
+          double D;
+          if (!V.getAsDouble(D))
+            ID.emplace(D);
+        }
+      }
     } else if (KeyValue == "params") {
       if (!Method)
         return false;
       // We have to interleave the call of the function here, otherwise the
       // YAMLParser will die because it can't go backwards. This is unfortunate
       // because it will break clients that put the id after params. A possible
       // fix would be to split the parsing and execution phases.
       Params = dyn_cast<llvm::yaml::MappingNode>(Value);
-      callHandler(Handlers, Method, Id, Params, UnknownHandler, Out);
+      callHandler(Handlers, Method, std::move(ID), Params, UnknownHandler, Out);
       return true;
     } else {
       return false;
     }
   }
 
   // In case there was a request with no params, call the handler on the
   // leftovers.
   if (!Method)
     return false;
-  callHandler(Handlers, Method, Id, nullptr, UnknownHandler, Out);
+  callHandler(Handlers, Method, std::move(ID), nullptr, UnknownHandler, Out);
 
   return true;
 }
 
 void clangd::runLanguageServerLoop(std::istream &In, JSONOutput &Out,
                                    JSONRPCDispatcher &Dispatcher,
                                    bool &IsDone) {
   while (In.good()) {

clangd/Protocol.h

 // Each struct has a parse and unparse function, that converts back and forth
 // between the struct and a JSON representation.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_PROTOCOL_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_PROTOCOL_H
 
+#include "JSONExpr.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/Support/YAMLParser.h"
 #include <string>
 #include <vector>
 
 namespace clang {
 namespace clangd {
 
 class Logger;
 
 struct URI {
   std::string uri;
   std::string file;
 
   static URI fromUri(llvm::StringRef uri);
   static URI fromFile(llvm::StringRef file);
 
   static URI parse(llvm::yaml::ScalarNode *Param);
-  static std::string unparse(const URI &U);
+  static json::Expr unparse(const URI &U);
 
   friend bool operator==(const URI &LHS, const URI &RHS) {
     return LHS.uri == RHS.uri;
   }
 
   friend bool operator!=(const URI &LHS, const URI &RHS) {
     return !(LHS == RHS);
   }
struct Position {
   }
   friend bool operator<(const Position &LHS, const Position &RHS) {
     return std::tie(LHS.line, LHS.character) <
            std::tie(RHS.line, RHS.character);
   }
 
   static llvm::Optional<Position> parse(llvm::yaml::MappingNode *Params,
                                         clangd::Logger &Logger);
-  static std::string unparse(const Position &P);
+  static json::Expr unparse(const Position &P);
 };
 
 struct Range {
   /// The range's start position.
   Position start;
 
   /// The range's end position.
   Position end;
 
   friend bool operator==(const Range &LHS, const Range &RHS) {
     return std::tie(LHS.start, LHS.end) == std::tie(RHS.start, RHS.end);
   }
   friend bool operator<(const Range &LHS, const Range &RHS) {
     return std::tie(LHS.start, LHS.end) < std::tie(RHS.start, RHS.end);
   }
 
   static llvm::Optional<Range> parse(llvm::yaml::MappingNode *Params,
                                      clangd::Logger &Logger);
-  static std::string unparse(const Range &P);
+  static json::Expr unparse(const Range &P);
 };
 
 struct Location {
   /// The text document's URI.
   URI uri;
   Range range;
 
   friend bool operator==(const Location &LHS, const Location &RHS) {
     return LHS.uri == RHS.uri && LHS.range == RHS.range;
   }
 
   friend bool operator!=(const Location &LHS, const Location &RHS) {
     return !(LHS == RHS);
   }
 
   friend bool operator<(const Location &LHS, const Location &RHS) {
     return std::tie(LHS.uri, LHS.range) < std::tie(RHS.uri, RHS.range);
   }
 
-  static std::string unparse(const Location &P);
+  static json::Expr unparse(const Location &P);
 };
 
 struct Metadata {
   std::vector<std::string> extraFlags;
 
   static llvm::Optional<Metadata> parse(llvm::yaml::MappingNode *Params,
                                         clangd::Logger &Logger);
 };
 
 struct TextEdit {
   /// The range of the text document to be manipulated. To insert
   /// text into a document create a range where start === end.
   Range range;
 
   /// The string to be inserted. For delete operations use an
   /// empty string.
   std::string newText;
 
   static llvm::Optional<TextEdit> parse(llvm::yaml::MappingNode *Params,
                                         clangd::Logger &Logger);
-  static std::string unparse(const TextEdit &P);
+  static json::Expr unparse(const TextEdit &P);
 };
 
 struct TextDocumentItem {
   /// The text document's URI.
   URI uri;
 
   /// The text document's language identifier.
   std::string languageId;
struct FormattingOptions {
   /// Size of a tab in spaces.
   int tabSize;
 
   /// Prefer spaces over tabs.
   bool insertSpaces;
 
   static llvm::Optional<FormattingOptions>
   parse(llvm::yaml::MappingNode *Params, clangd::Logger &Logger);
-  static std::string unparse(const FormattingOptions &P);
+  static json::Expr unparse(const FormattingOptions &P);
 };
 
 struct DocumentRangeFormattingParams {
   /// The document to format.
   TextDocumentIdentifier textDocument;
 
   /// The range to format
   Range range;
struct CompletionItem {
   // TODO(krasimir): The following optional fields defined by the language
   // server protocol are unsupported:
   //
   // command?: Command - An optional command that is executed *after* inserting
   //                     this completion.
   //
   // data?: any - A data entry field that is preserved on a completion item
   //              between a completion and a completion resolve request.
-  static std::string unparse(const CompletionItem &P);
+  static json::Expr unparse(const CompletionItem &P);
 };
 
 /// A single parameter of a particular signature.
 struct ParameterInformation {
 
   /// The label of this parameter. Mandatory.
   std::string label;
 
   /// The documentation of this parameter. Optional.
   std::string documentation;
 
-  static std::string unparse(const ParameterInformation &);
+  static json::Expr unparse(const ParameterInformation &);
 };
 
 /// Represents the signature of something callable.
 struct SignatureInformation {
 
   /// The label of this signature. Mandatory.
   std::string label;
 
   /// The documentation of this signature. Optional.
   std::string documentation;
 
   /// The parameters of this signature.
   std::vector<ParameterInformation> parameters;
 
-  static std::string unparse(const SignatureInformation &);
+  static json::Expr unparse(const SignatureInformation &);
 };
 
 /// Represents the signature of a callable.
 struct SignatureHelp {
 
   /// The resulting signatures.
   std::vector<SignatureInformation> signatures;
 
   /// The active signature.
   int activeSignature = 0;
 
   /// The active parameter of the active signature.
   int activeParameter = 0;
 
-  static std::string unparse(const SignatureHelp &);
+  static json::Expr unparse(const SignatureHelp &);
 };
 
 } // namespace clangd
 } // namespace clang
 
 #endif

clangd/Protocol.cpp

URI URI::fromFile(llvm::StringRef file) {
   return Result;
 }
 
 URI URI::parse(llvm::yaml::ScalarNode *Param) {
   llvm::SmallString<10> Storage;
   return URI::fromUri(Param->getValue(Storage));
 }
 
-std::string URI::unparse(const URI &U) { return "\"" + U.uri + "\""; }
+json::Expr URI::unparse(const URI &U) { return U.uri; }
 
 llvm::Optional<TextDocumentIdentifier>
 TextDocumentIdentifier::parse(llvm::yaml::MappingNode *Params,
                               clangd::Logger &Logger) {
   TextDocumentIdentifier Result;
   for (auto &NextKeyValue : *Params) {
     auto *KeyString = dyn_cast<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
     if (!KeyString)
if (KeyValue == "line") {
       Result.character = Val;
     } else {
       logIgnoredField(KeyValue, Logger);
     }
   }
   return Result;
 }
 
-std::string Position::unparse(const Position &P) {
-  std::string Result;
-  llvm::raw_string_ostream(Result)
-      << llvm::format(R"({"line": %d, "character": %d})", P.line, P.character);
-  return Result;
+json::Expr Position::unparse(const Position &P) {
+  return json::obj{
+      {"line", P.line},
+      {"character", P.character},
+  };
 }
 
 llvm::Optional<Range> Range::parse(llvm::yaml::MappingNode *Params,
                                    clangd::Logger &Logger) {
   Range Result;
   for (auto &NextKeyValue : *Params) {
     auto *KeyString = dyn_cast<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
     if (!KeyString)
if (KeyValue == "start") {
       Result.end = std::move(*Parsed);
     } else {
       logIgnoredField(KeyValue, Logger);
     }
   }
   return Result;
 }
 
-std::string Range::unparse(const Range &P) {
-  std::string Result;
-  llvm::raw_string_ostream(Result) << llvm::format(
-      R"({"start": %s, "end": %s})", Position::unparse(P.start).c_str(),
-      Position::unparse(P.end).c_str());
-  return Result;
+json::Expr Range::unparse(const Range &P) {
+  return json::obj{
+      {"start", P.start},
+      {"end", P.end},
+  };
 }
 
-std::string Location::unparse(const Location &P) {
-  std::string Result;
-  llvm::raw_string_ostream(Result) << llvm::format(
-      R"({"uri": %s, "range": %s})", URI::unparse(P.uri).c_str(),
-      Range::unparse(P.range).c_str());
-  return Result;
+json::Expr Location::unparse(const Location &P) {
+  return json::obj{
+      {"uri", P.uri},
+      {"range", P.range},
+  };
 }
 
 llvm::Optional<TextDocumentItem>
 TextDocumentItem::parse(llvm::yaml::MappingNode *Params,
                         clangd::Logger &Logger) {
   TextDocumentItem Result;
   for (auto &NextKeyValue : *Params) {
     auto *KeyString = dyn_cast<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
if (KeyValue == "range") {
       Result.newText = Node->getValue(Storage);
     } else {
       logIgnoredField(KeyValue, Logger);
     }
   }
   return Result;
 }
 
-std::string TextEdit::unparse(const TextEdit &P) {
-  std::string Result;
-  llvm::raw_string_ostream(Result) << llvm::format(
-      R"({"range": %s, "newText": "%s"})", Range::unparse(P.range).c_str(),
-      llvm::yaml::escape(P.newText).c_str());
-  return Result;
+json::Expr TextEdit::unparse(const TextEdit &P) {
+  return json::obj{
+      {"range", P.range},
+      {"newText", P.newText},
+  };
 }
 
 namespace {
 TraceLevel getTraceLevel(llvm::StringRef TraceLevelStr,
                          clangd::Logger &Logger) {
   if (TraceLevelStr == "off")
     return TraceLevel::Off;
   else if (TraceLevelStr == "messages")
if (KeyValue == "tabSize") {
       Result.insertSpaces = Val;
     } else {
       logIgnoredField(KeyValue, Logger);
     }
   }
   return Result;
 }
 
-std::string FormattingOptions::unparse(const FormattingOptions &P) {
-  std::string Result;
-  llvm::raw_string_ostream(Result) << llvm::format(
-      R"({"tabSize": %d, "insertSpaces": %d})", P.tabSize, P.insertSpaces);
-  return Result;
+json::Expr FormattingOptions::unparse(const FormattingOptions &P) {
+  return json::obj{
+      {"tabSize", P.tabSize},
+      {"insertSpaces", P.insertSpaces},
+  };
 }
 
 llvm::Optional<DocumentRangeFormattingParams>
 DocumentRangeFormattingParams::parse(llvm::yaml::MappingNode *Params,
                                      clangd::Logger &Logger) {
   DocumentRangeFormattingParams Result;
   for (auto &NextKeyValue : *Params) {
     auto *KeyString = dyn_cast<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
if (KeyValue == "textDocument") {
       Result.position = std::move(*Parsed);
     } else {
       logIgnoredField(KeyValue, Logger);
     }
   }
   return Result;
 }
 
-std::string CompletionItem::unparse(const CompletionItem &CI) {
-  std::string Result = "{";
-  llvm::raw_string_ostream Os(Result);
+json::Expr CompletionItem::unparse(const CompletionItem &CI) {
   assert(!CI.label.empty() && "completion item label is required");
-  Os << R"("label":")" << llvm::yaml::escape(CI.label) << R"(",)";
+  json::obj Result{{"label", CI.label}};
   if (CI.kind != CompletionItemKind::Missing)
-    Os << R"("kind":)" << static_cast<int>(CI.kind) << R"(,)";
+    Result["kind"] = static_cast<int>(CI.kind);
   if (!CI.detail.empty())
-    Os << R"("detail":")" << llvm::yaml::escape(CI.detail) << R"(",)";
+    Result["detail"] = CI.detail;
   if (!CI.documentation.empty())
-    Os << R"("documentation":")" << llvm::yaml::escape(CI.documentation)
-       << R"(",)";
+    Result["documentation"] = CI.documentation;
   if (!CI.sortText.empty())
-    Os << R"("sortText":")" << llvm::yaml::escape(CI.sortText) << R"(",)";
+    Result["sortText"] = CI.sortText;
   if (!CI.filterText.empty())
-    Os << R"("filterText":")" << llvm::yaml::escape(CI.filterText) << R"(",)";
+    Result["filterText"] = CI.filterText;
   if (!CI.insertText.empty())
-    Os << R"("insertText":")" << llvm::yaml::escape(CI.insertText) << R"(",)";
-  if (CI.insertTextFormat != InsertTextFormat::Missing) {
-    Os << R"("insertTextFormat":)" << static_cast<int>(CI.insertTextFormat)
-       << R"(,)";
-  }
+    Result["insertText"] = CI.insertText;
+  if (CI.insertTextFormat != InsertTextFormat::Missing)
+    Result["insertTextFormat"] = static_cast<int>(CI.insertTextFormat);
   if (CI.textEdit)
-    Os << R"("textEdit":)" << TextEdit::unparse(*CI.textEdit) << ',';
-  if (!CI.additionalTextEdits.empty()) {
-    Os << R"("additionalTextEdits":[)";
-    for (const auto &Edit : CI.additionalTextEdits)
-      Os << TextEdit::unparse(Edit) << ",";
-    Os.flush();
-    // The list additionalTextEdits is guaranteed nonempty at this point.
-    // Replace the trailing comma with right brace.
-    Result.back() = ']';
-  }
-  Os.flush();
-  // Label is required, so Result is guaranteed to have a trailing comma.
-  Result.back() = '}';
-  return Result;
+    Result["textEdit"] = *CI.textEdit;
+  if (!CI.additionalTextEdits.empty())
+    Result["additionalTextEdits"] = json::ary(CI.additionalTextEdits);
+  return std::move(Result);
 }
 
-std::string ParameterInformation::unparse(const ParameterInformation &PI) {
-  std::string Result = "{";
-  llvm::raw_string_ostream Os(Result);
+json::Expr ParameterInformation::unparse(const ParameterInformation &PI) {
   assert(!PI.label.empty() && "parameter information label is required");
-  Os << R"("label":")" << llvm::yaml::escape(PI.label) << '\"';
+  json::obj Result{{"label", PI.label}};
   if (!PI.documentation.empty())
-    Os << R"(,"documentation":")" << llvm::yaml::escape(PI.documentation)
-       << '\"';
-  Os << '}';
-  Os.flush();
-  return Result;
+    Result["documentation"] = PI.documentation;
+  return std::move(Result);
 }
 
-std::string SignatureInformation::unparse(const SignatureInformation &SI) {
-  std::string Result = "{";
-  llvm::raw_string_ostream Os(Result);
+json::Expr SignatureInformation::unparse(const SignatureInformation &SI) {
   assert(!SI.label.empty() && "signature information label is required");
-  Os << R"("label":")" << llvm::yaml::escape(SI.label) << '\"';
+  json::obj Result{
+      {"label", SI.label},
+      {"parameters", json::ary(SI.parameters)},
+  };
   if (!SI.documentation.empty())
-    Os << R"(,"documentation":")" << llvm::yaml::escape(SI.documentation)
-       << '\"';
-  Os << R"(,"parameters":[)";
-  for (const auto &Parameter : SI.parameters) {
-    Os << ParameterInformation::unparse(Parameter) << ',';
-  }
-  Os.flush();
-  if (SI.parameters.empty())
-    Result.push_back(']');
-  else
-    Result.back() = ']'; // Replace the last `,` with an `]`.
-  Result.push_back('}');
-  return Result;
+    Result["documentation"] = SI.documentation;
+  return std::move(Result);
 }
 
-std::string SignatureHelp::unparse(const SignatureHelp &SH) {
-  std::string Result = "{";
-  llvm::raw_string_ostream Os(Result);
+json::Expr SignatureHelp::unparse(const SignatureHelp &SH) {
   assert(SH.activeSignature >= 0 &&
          "Unexpected negative value for number of active signatures.");
   assert(SH.activeParameter >= 0 &&
          "Unexpected negative value for active parameter index");
-  Os << R"("activeSignature":)" << SH.activeSignature
-     << R"(,"activeParameter":)" << SH.activeParameter << R"(,"signatures":[)";
-  for (const auto &Signature : SH.signatures) {
-    Os << SignatureInformation::unparse(Signature) << ',';
-  }
-  Os.flush();
-  if (SH.signatures.empty())
-    Result.push_back(']');
-  else
-    Result.back() = ']'; // Replace the last `,` with an `]`.
-  Result.push_back('}');
-  return Result;
+  return json::obj{
+      {"activeSignature", SH.activeSignature},
+      {"activeParameter", SH.activeParameter},
+      {"signatures", json::ary(SH.signatures)},
+  };
 }

clangd/tool/ClangdMain.cpp

WorkerThreadsCount("j",
                        llvm::cl::init(getDefaultAsyncThreadsCount()));
 
 static llvm::cl::opt<bool> EnableSnippets(
     "enable-snippets",
     llvm::cl::desc(
         "Present snippet completions instead of plaintext completions"),
     llvm::cl::init(false));
 
+static llvm::cl::opt<bool>
+    PrettyPrint("pretty", llvm::cl::desc("Pretty-print JSON output"),
+                llvm::cl::init(false));
+
 static llvm::cl::opt<bool> RunSynchronously(
     "run-synchronously",
     llvm::cl::desc("Parse on main thread. If set, -j is ignored"),
     llvm::cl::init(false), llvm::cl::Hidden);
 
 static llvm::cl::opt<Path>
     ResourceDir("resource-dir",
                 llvm::cl::desc("Directory for system clang headers"),
if (!TraceFile.empty()) {
     } else {
       TraceSession = trace::Session::create(*TraceStream);
     }
   }
 
   llvm::raw_ostream &Outs = llvm::outs();
   llvm::raw_ostream &Logs = llvm::errs();
   JSONOutput Out(Outs, Logs,
-                 InputMirrorStream ? InputMirrorStream.getPointer() : nullptr);
+                 InputMirrorStream ? InputMirrorStream.getPointer() : nullptr,
+                 PrettyPrint);
 
   // If --compile-commands-dir arg was invoked, check value and override default
   // path.
   llvm::Optional<Path> CompileCommandsDirPath;
 
   if (CompileCommandsDir.empty()) {
     CompileCommandsDirPath = llvm::None;
   } else if (!llvm::sys::path::is_absolute(CompileCommandsDir) ||

test/clangd/initialize-params.test

-# RUN: clangd -run-synchronously < %s | FileCheck %s
+# RUN: clangd -pretty -run-synchronously < %s | FileCheck -strict-whitespace %s
 # It is absolutely vital that this file has CRLF line endings.
 #
 # Test initialize request parameters with rootUri
 Content-Length: 143
 
 {"jsonrpc":"2.0","id":0,"method":"initialize","params":{"processId":123,"rootUri":"file:///path/to/workspace","capabilities":{},"trace":"off"}}
-# CHECK: Content-Length: 535
-# CHECK: {"jsonrpc":"2.0","id":0,"result":{"capabilities":{
-# CHECK:   "textDocumentSync": 1,
-# CHECK:   "documentFormattingProvider": true,
-# CHECK:   "documentRangeFormattingProvider": true,
-# CHECK:   "documentOnTypeFormattingProvider": {"firstTriggerCharacter":"}","moreTriggerCharacter":[]},
-# CHECK:   "codeActionProvider": true,
-# CHECK:   "completionProvider": {"resolveProvider": false, "triggerCharacters": [".",">",":"]},
-# CHECK:   "signatureHelpProvider": {"triggerCharacters": ["(",","]},
-# CHECK:   "definitionProvider": true
-# CHECK: }}}
-#
+#      CHECK:  "id": 0,
+# CHECK-NEXT:  "jsonrpc": "2.0",
+# CHECK-NEXT:  "result": {
+# CHECK-NEXT:    "codeActionProvider": true,
+# CHECK-NEXT:    "completionProvider": {
+# CHECK-NEXT:      "resolveProvider": false,
+# CHECK-NEXT:      "triggerCharacters": [
+# CHECK-NEXT:        ".",
+# CHECK-NEXT:        ">",
+# CHECK-NEXT:        ":"
+# CHECK-NEXT:      ]
+# CHECK-NEXT:    },
+# CHECK-NEXT:    "definitionProvider": true,
+# CHECK-NEXT:    "documentFormattingProvider": true,
+# CHECK-NEXT:    "documentOnTypeFormattingProvider": {
+# CHECK-NEXT:      "firstTriggerCharacter": "}",
+# CHECK-NEXT:      "moreTriggerCharacter": null
+# CHECK-NEXT:    },
+# CHECK-NEXT:    "documentRangeFormattingProvider": true,
+# CHECK-NEXT:    "signatureHelpProvider": {
+# CHECK-NEXT:      "triggerCharacters": [
+# CHECK-NEXT:        "(",
+# CHECK-NEXT:        ","
+# CHECK-NEXT:      ]
+# CHECK-NEXT:    },
+# CHECK-NEXT:    "textDocumentSync": 1
+# CHECK-NEXT:  }
 Content-Length: 44
 
 {"jsonrpc":"2.0","id":3,"method":"shutdown"}
-# CHECK: {"jsonrpc":"2.0","id":3,"result":null}
+#      CHECK:  "id": 3,
+# CHECK-NEXT:  "jsonrpc": "2.0",
+# CHECK-NEXT:  "result": null
 Content-Length: 33
 
 {"jsonrpc":"2.0":"method":"exit"}

unittests/clangd/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   support
   )
 
 get_filename_component(CLANGD_SOURCE_DIR
   ${CMAKE_CURRENT_SOURCE_DIR}/../../clangd REALPATH)
 include_directories(
   ${CLANGD_SOURCE_DIR}
   )
 
 add_extra_unittest(ClangdTests
   ClangdTests.cpp
+  JSONExprTests.cpp
   TraceTests.cpp
   )
 
 target_link_libraries(ClangdTests
   clangBasic
   clangDaemon
   clangFormat
   clangFrontend
   clangSema
   clangTooling
   clangToolingCore
   LLVMSupport
   )

unittests/clangd/JSONExprTests.cpp

+//===-- JSONExprTests.cpp - JSON expression unit tests ----------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "JSONExpr.h"
+
+#include "gmock/gmock.h"
+#include "gtest/gtest.h"
+
+namespace clang {
+namespace clangd {
+namespace json {
+namespace {
+
+std::string s(const Expr& E) { return llvm::formatv("{0}", E).str(); }
+std::string sp(const Expr& E) { return llvm::formatv("{0:2}", E).str(); }
+
+TEST(JSONExprTests, Types) {
+  EXPECT_EQ("true", s(true));
+  EXPECT_EQ("null", s(nullptr));
+  EXPECT_EQ("2.5", s(2.5));
+  EXPECT_EQ(R"("foo")", s("foo"));
+  EXPECT_EQ("[1,2,3]", s({1,2,3}));
+  EXPECT_EQ(R"({"x":10,"y":20})", s(obj{{"x", 10}, {"y", 20}}));
+}
+
+TEST(JSONExprTests, Constructors) {
+  // Lots of edge cases around empty and singleton init lists.
+  EXPECT_EQ("[[[3]]]", s({{{3}}}));
+  EXPECT_EQ("[[[]]]", s({{{}}}));
+  EXPECT_EQ("[[{}]]", s({{obj{}}}));
+  EXPECT_EQ(R"({"A":{"B":{}}})", s(obj{{"A",obj{{"B",obj{}}}}}));
+  EXPECT_EQ(R"({"A":{"B":{"X":"Y"}}})",
+            s(obj{{"A", obj{{"B", obj{{"X", "Y"}}}}}}));
+}
+
+TEST(JSONExprTests, StringOwnership) {
+  char X[] = "Hello";
+  Expr Alias = static_cast<const char*>(X);
+  X[1] = 'a';
+  EXPECT_EQ(R"("Hallo")", s(Alias));
+
+  std::string Y = "Hello";
+  Expr Copy = Y;
+  Y[1] = 'a';
+  EXPECT_EQ(R"("Hello")", s(Copy));
+}
+
+TEST(JSONExprTests, CanonicalOutput) {
+  // Objects are sorted (but arrays aren't)!
+  EXPECT_EQ(R"({"a":1,"b":2,"c":3})", s(obj{{"a", 1}, {"c", 3}, {"b", 2}}));
+  EXPECT_EQ(R"(["a","c","b"])", s({"a", "c", "b"}));
+  EXPECT_EQ("3", s(3.0));
+}
+
+TEST(JSONExprTests, Escaping) {
+  std::string test = {
+      0,                    // Strings may contain nulls.
+      '\b',   '\f',         // Have mnemonics, but we escape numerically.
+      '\r',   '\n',   '\t', // Escaped with mnemonics.
+      'S',    '\"',   '\\', // Printable ASCII characters.
+      '\x7f',               // Delete is not escaped.
+      '\xce', '\x94',       // Non-ASCII UTF-8 is not escaped.
+  };
+  EXPECT_EQ(R"("\u0000\u0008\u000c\r\n\tS\"\\)"
+            u8"\x7fΔ\"",
+            s(test));
+
+  EXPECT_EQ(R"({"object keys are\nescaped":true})",
+            s(obj{{"object keys are\nescaped", true}}));
+}
+
+TEST(JSONExprTests, PrettyPrinting) {
+  EXPECT_EQ(
+      R"({
+  "empty_array": [],
+  "empty_object": {},
+  "full_array": [
+    1,
+    null
+  ],
+  "full_object": {
+    "nested_array": [
+      {
+        "property": "value"
+      }
+    ]
+  }
+})",
+      sp(obj{
+          {"empty_object", obj{}},
+          {"empty_array", {}},
+          {"full_array", {1, nullptr}},
+          {"full_object",
+           obj{
+               {"nested_array",
+                {obj{
+                    {"property", "value"},
+                }}},
+           }},
+      }));
+}
+
+} // namespace
+} // namespace json
+} // namespace clangd
+} // namespace clang