Add a prototype for clangd v0.1

[bkramer]

Summary

clangd is a language server protocol implementation based on clang. It's
supposed to provide editor integration while not suffering from the
confined ABI of libclang.

This implementation is limited to the bare minimum functionality of
doing (whole-document) formatting and rangeFormatting. The JSON parsing
is based on LLVM's YAMLParser but yet most of the code of clangd is
currently dealing with JSON serialization and deserialization.

This was only tested with VS Code so far, mileage with other LSP clients
may vary.

[krasimir]

looks good!

[djasper]

Just a few nits. I think this looks like a great starting point!

clangd/ClangDMain.cpp

33 ↗

(On Diff #86800)

Maybe add "Implement" :)

52 ↗

(On Diff #86800)

Nit: string also has "empty()", right? Maybe only convert to StringRef after this check.

62 ↗

(On Diff #86800)

Should you somehow actually read them and assert that they are "\r\n"?

74 ↗

(On Diff #86800)

So you currently always print this to std::err.. Should you guard this in DEBUG or something?

clangd/DocumentStore.h

25 ↗

(On Diff #86800)

I'd call this addDocument for symmetry with removeDocument. The fact that you phrase the comment that way, also make me think that this will be the more intuitive name.

clangd/JSONRPCDispatcher.cpp

89 ↗

(On Diff #86800)

I'd pull out this case and move it to the front to show that this is a guaranteed early exit and the function can never fall through to the following code. But I am not sure.

clangd/ProtocolHandlers.h

27 ↗

(On Diff #86800)

In the long run, we probably don't want to specify all of these in the same file, but for now this seems fine.

test/clangd/formatting.txt

9 ↗

(On Diff #86800)

How did you come up with this indentation?

[Adi]

I had some spare time ... I hope you don't mind the comments. Hopefully you will find something useful :)

clangd/ClangDMain.cpp

65–70 ↗

(On Diff #86800)

Avoid unnecessary JSON.resize(Len) & potential reallocation during JSON.push_back('\0') by allocating enough space in advance: std::vector<char> JSON(Len + 1);

73–77 ↗

(On Diff #86800)

Since llvm::StringRef does not own the data, I think it would make more sense to do the logging after Dispatcher.call().

80 ↗

(On Diff #86800)

You are building llvm::StringRef without the trailing null byte here, yet comment above states that YAML parser requires a null byte.

80 ↗

(On Diff #86800)

Perhaps make a log entry if Dispatcher.call() failed.

clangd/JSONRPCDispatcher.cpp

32–40 ↗

(On Diff #86800)

I would extract this implementation to the separate handler class (e.g. OperationNotSupportedHandler) and make this an interface class. It would make code and intent more explicit.

82–83 ↗

(On Diff #86800)

dyn_cast might fail and leave you with Version set to nullptr. Using static_cast is better approach if you are sure the cast will be successful.

112–120 ↗

(On Diff #86800)

Perhaps refactor this code and the one above into the separate method to gain some more readability.

clangd/JSONRPCDispatcher.h

25–32 ↗

(On Diff #86800)

To avoid virtual dispatch one may rewrite this class in terms of CRTP. E.g.

template <typename T>
class Handler {
public:
    Handler(llvm::raw_ostream &Outs, llvm::raw_ostream &Logs)
      : Outs(Outs), Logs(Logs) {}
    virtual ~Handler() = default;

    void handleMethod(const llvm::yaml::MappingMode *Params, StringRef ID) { 
        static_cast<T*>(this)->handleMethod(Params, ID); 
    }
    void handleNotification(const llvm::yaml::MappingMode *Params) { 
        static_cast<T*>(this)->handleNotification(Params); 
    }

protected:
    llvm::raw_ostream &Outs;
    llvm::raw_ostream &Logs;

    void writeMessage(const Twine &Message);    
};

And then use it as:

struct MyConcreteHandler : public Handler<MyConcreteHandler> {
    MyConcreteHandler(llvm::raw_ostream &Outs, llvm::raw_ostream &Logs /* other params if necessary */) 
      : Handler(Outs, Logs) /* init other members */
    {}
 
    void handleMethod(const llvm::yaml::MappingMode *Params, StringRef ID) { 
        // impl
    }
    void handleNotification(const llvm::yaml::MappingMode *Params) { 
        // impl 
    }
};

29 ↗

(On Diff #86800)

const ptr/ref to Params?

32 ↗

(On Diff #86800)

const ptr/ref to Params?

clangd/Protocol.cpp

11 ↗

(On Diff #86800)

To some degree it could be refactored. I would go for CRTP again here. It would simplify the code a little bit. E.g.

template <typename T>
struct Params {
    static llvm::Optional<T> parse(const llvm::yaml::MappingNode& node) {
        T Result;
        for (auto& NextKeyValue : node) {
            auto *KeyString = dyn_cast<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
            if (!KeyString)
                return llvm::None;
            
            llvm::SmallString<10> KeyStorage;
            StringRef KeyValue = KeyString->getValue(KeyStorage);
            auto *Value = dyn_cast_or_null<llvm::yaml::ScalarNode>(NextKeyValue.getValue());
            if (!Value)
                return llvm::None;

            Result = static_cast<T*>(this)->setParam(KeyValue, *Value); // or make Result an argument if worried about RVO
            if (!Result) // we can make this more neat by placing it inside a for loop condition (ranged-for doesn't allow this)
                return llvm::none;
        }
        return Result;
    }
};

struct TextDocumentIdentifier : public Params<TextDocumentIdentifier> {
    /// The text document's URI.
    std::string uri;

    llvm::Optional<TextDocumentIdentifier> setParam(StringRef KeyValue, const llvm::yaml::ScalarNode& Value) {
        TextDocumentIdentifier Result;
        llvm::SmallString<10> Storage;
        if (KeyValue == "uri") {
            Result.uri = Value.getValue(Storage);
        } else if (KeyValue == "version") {
            // FIXME: parse version, but only for VersionedTextDocumentIdentifiers.
        } else {
            return llvm::None;
        }
        return Result;
    }
};

struct Position : public Params<Position> {
    /// Line position in a document (zero-based).
    int line;

    /// Character offset on a line in a document (zero-based).
    int character;

    llvm::Optional<Position> setParam(StringRef KeyValue, const llvm::yaml::ScalarNode& Value) {
        Position Result;
        llvm::SmallString<10> Storage;
        if (KeyValue == "line") {
            long long Val;
            if (llvm::getAsSignedInteger(Value.getValue(Storage), 0, Val))
                return llvm::None;
            Result.line = Val;
        } else if (KeyValue == "character") {
            long long Val;
            if (llvm::getAsSignedInteger(Value.getValue(Storage), 0, Val))
                return llvm::None;
            Result.character = Val;
        } else {
            return llvm::None;
        }
        return Result;
    }
};

I am not really familiar with all of the dependencies here but I may suggest that one might also try to generalize this approach even further by encapsulating the handling of KeyValue into the common implementation. E.g.

template <typename T>
T fromKeyValueToParam(StringRef KeyValue, const llvm::yaml::ScalarNode& Value) {
    T Result;
    llvm::SmallString<10> Storage;

    if (KeyValue == "line")
        long long Val;
        if (llvm::getAsSignedInteger(Value.getValue(Storage), 0, Val))
            return llvm::None;
        Result.line = Val;
    } else if (KeyValue == "character") {
        long long Val;
        if (llvm::getAsSignedInteger(Value.getValue(Storage), 0, Val))
            return llvm::None;
        Result.character = Val;
    }
    else if (KeyValue == "textDocument") {
        auto Parsed = TextDocumentIdentifier::parse(Value);
        if (!Parsed)
            return llvm::None;
        Result.textDocument = std::move(*Parsed);
    }
    else if (KeyValue == "options") {
        auto Parsed = FormattingOptions::parse(Value);
        if (!Parsed)
            return llvm::None;
        Result.options = std::move(*Parsed);else if (KeyValue == "range")
    }
    else if (KeyValue == "range") {
        auto Parsed = Range::parse(Value);
        if (!Parsed)
            return llvm::None;
        Result.range = std::move(*Parsed);
    }
    else if ( ... ) {
        ...
    }
    
    ...

    return Result;
}

Then you would be able to handle everything in Params<T>::parse() without any other specialized code required to be written in concrete parameters. Code could look something like this:

template <typename T>
struct Params {
    static llvm::Optional<T> parse(const llvm::yaml::MappingNode& Params) {
        T Result;
        for (auto& NextKeyValue : Params) {
            auto *KeyString = dyn_cast<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
            if (!KeyString)
                return llvm::None;
            
            llvm::SmallString<10> KeyStorage;
            StringRef KeyValue = KeyString->getValue(KeyStorage);
            auto *Value = dyn_cast_or_null<llvm::yaml::ScalarNode>(NextKeyValue.getValue());
            if (!Value)
                return llvm::None;

            Result = fromKeyValueToParam(KeyValue, *Value);
            if (!Result) // we can make this more neat by placing it inside a for loop condition (ranged-for doesn't allow us that)
                return llvm::none;
        }
        return Result;
    }
};

struct Position : public Params<Position> {
    /// Line position in a document (zero-based).
    int line;

    /// Character offset on a line in a document (zero-based).
    int character;
};

struct TextDocumentIdentifier : public Params<TextDocumentIdentifier> {
    /// The text document's URI.
    std::string uri;
};

[klimek]

LG. Couple of questions.

clangd/ClangDMain.cpp

65 ↗

(On Diff #86800)

Shouldn't that be unsigned char for raw bytes?

clangd/JSONRPCDispatcher.h

29 ↗

(On Diff #86800)

Here and below, document what the default implementations do.

29–32 ↗

(On Diff #86800)

Can we make those Params pointers-to-const?

[klimek]

clangd/JSONRPCDispatcher.h

25–32 ↗

(On Diff #86800)

To avoid virtual dispatch one may rewrite this class in terms of CRTP. E.g.

Why would virtual dispatch be a problem here? It seems strictly simpler this way.

[Adi]

clangd/JSONRPCDispatcher.h

25–32 ↗

(On Diff #86800)

I've mentioned it only as an alternative approach. I was not implying that virtual dispatch is a problem. It's a matter of taste in this case really.

clangd/Protocol.cpp

11 ↗

(On Diff #86800)

fromKeyValueToParam() will of course not compile but you've got the idea. I think it can be achieved in this way somehow.

[arphaman]

This might be a bad question, but is there any particular reason why you didn't use the YAML Traits API for parsing instead of the raw YAML Stream API? In my experience the traits based API is quite convenient for structured data like various options and parameters in this protocol.

[bkramer]

In D29451#667606, @arphaman wrote:

This might be a bad question, but is there any particular reason why you didn't use the YAML Traits API for parsing instead of the raw YAML Stream API? In my experience the traits based API is quite convenient for structured data like various options and parameters in this protocol.

It's an excellent question. I actually tried to make YAMLTraits work for this, but it gets annoying due to the dynamic parts of JSONRPC. You have to first parse the method string and then dispatch based on that string. This is not something that the YAMLTraits interface supports. I think it could be extended to allow mixing code written against YAMLParser with YAMLTraits code and then we can clean up the manual code here. Before that happens I want the YAMLParser implementation working, because there are some complicated union types in the Language Server Protocol that might require additional tweaks to the YAML infrastructure.

clangd/ClangDMain.cpp

52 ↗

(On Diff #86800)

But string doesn't have trim(). An "empty" line still has a \n, which I want to discard.

65 ↗

(On Diff #86800)

I agree, but that would add pointer punning on every usage, as iostreams and YAML parser want normal chars.

73–77 ↗

(On Diff #86800)

But that will confuse the logging. You'd get the query after the answer.

74 ↗

(On Diff #86800)

My goal is to wire this up to the editor in some way and make it possible to discard the log with a command line flag (send it to nulls()). I think using DEBUG only makes that more complicated.

That said, the logging parts of clangd probably need a redesign at some point ;)

80 ↗

(On Diff #86800)

This is intentional. YAML parser treats the pointer inside of the StringRef as a null terminated string, reading past the end of the StringRef.

clangd/DocumentStore.h

25 ↗

(On Diff #86800)

I picked set because it can overwrite contents and that's how it used. Your comment makes sense though so I changed it back to add.

clangd/JSONRPCDispatcher.cpp

32–40 ↗

(On Diff #86800)

The goal of the default methods is to make it easy to implement only one of the two methods. If I want to do that with an OperationNotSupportedHandler I'd have to inherit from that, which is more awkward.

82–83 ↗

(On Diff #86800)

It's supposed to not crash and return an error. This is user input. Added the missing nullptr check.

89 ↗

(On Diff #86800)

Not sure if I understand. It's vital that this part is called when we just read params, otherwise the YAMLParser will crash.

clangd/JSONRPCDispatcher.h

29 ↗

(On Diff #86800)

YAML nodes are supposed to be immutable and don't provide const members.

29–32 ↗

(On Diff #86800)

No. YAML nodes are supposed to be immutable and don't provide const members.

clangd/Protocol.cpp

11 ↗

(On Diff #86800)

I don't think this really helps. The most annoying repetitive part is the walk over the fields, which is still there.

In the long term I want to replace this with a YAMLTraits-like thing, but that will take a while.

test/clangd/formatting.txt

9 ↗

(On Diff #86800)

It's hardcoded as a raw string literal in the source. It looks better there ;)

[bkramer]

Address review comments. Make test actually run (missing cmake file)

[krasimir]

test/clangd/formatting.txt

9 ↗

(On Diff #86800)

Maybe you could consider clang-format-ing the json output too? :D

clang-tools-extra/trunk/CMakeLists.txt

 add_subdirectory(clang-apply-replacements)
 add_subdirectory(clang-rename)
 add_subdirectory(clang-reorder-fields)
 add_subdirectory(modularize)
 if(CLANG_ENABLE_STATIC_ANALYZER)
 add_subdirectory(clang-tidy)
 add_subdirectory(clang-tidy-vs)
 endif()
 
 add_subdirectory(change-namespace)
 add_subdirectory(clang-query)
 add_subdirectory(clang-move)
+add_subdirectory(clangd)
 add_subdirectory(include-fixer)
 add_subdirectory(pp-trace)
 add_subdirectory(tool-template)
 
 # Add the common testsuite after all the tools.
 # TODO: Support tests with more granularity when features are off?
 if(CLANG_ENABLE_STATIC_ANALYZER AND CLANG_INCLUDE_TESTS)
 add_subdirectory(test)

clang-tools-extra/trunk/clangd/CMakeLists.txt

+add_clang_executable(clangd
+  ClangDMain.cpp
+  JSONRPCDispatcher.cpp
+  Protocol.cpp
+  ProtocolHandlers.cpp
+  )
+
+target_link_libraries(clangd
+  clangBasic
+  clangFormat
+  LLVMSupport
+  )

clang-tools-extra/trunk/clangd/ClangDMain.cpp

+//===--- ClangDMain.cpp - clangd server loop ------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "DocumentStore.h"
+#include "JSONRPCDispatcher.h"
+#include "ProtocolHandlers.h"
+#include "llvm/Support/FileSystem.h"
+#include <iostream>
+#include <string>
+using namespace clang::clangd;
+
+int main(int argc, char *argv[]) {
+  llvm::raw_ostream &Outs = llvm::outs();
+  llvm::raw_ostream &Logs = llvm::errs();
+
+  // Set up a document store and intialize all the method handlers for JSONRPC
+  // dispatching.
+  DocumentStore Store;
+  JSONRPCDispatcher Dispatcher(llvm::make_unique<Handler>(Outs, Logs));
+  Dispatcher.registerHandler("initialize",
+                             llvm::make_unique<InitializeHandler>(Outs, Logs));
+  Dispatcher.registerHandler("shutdown",
+                             llvm::make_unique<ShutdownHandler>(Outs, Logs));
+  Dispatcher.registerHandler(
+      "textDocument/didOpen",
+      llvm::make_unique<TextDocumentDidOpenHandler>(Outs, Logs, Store));
+  // FIXME: Implement textDocument/didClose.
+  Dispatcher.registerHandler(
+      "textDocument/didChange",
+      llvm::make_unique<TextDocumentDidChangeHandler>(Outs, Logs, Store));
+  Dispatcher.registerHandler(
+      "textDocument/rangeFormatting",
+      llvm::make_unique<TextDocumentRangeFormattingHandler>(Outs, Logs, Store));
+  Dispatcher.registerHandler(
+      "textDocument/formatting",
+      llvm::make_unique<TextDocumentFormattingHandler>(Outs, Logs, Store));
+
+  while (std::cin.good()) {
+    // A Language Server Protocol message starts with a HTTP header, delimited
+    // by \r\n.
+    std::string Line;
+    std::getline(std::cin, Line);
+
+    // Skip empty lines.
+    llvm::StringRef LineRef(Line);
+    if (LineRef.trim().empty())
+      continue;
+
+    unsigned long long Len = 0;
+    // FIXME: Content-Type is a specified header, but does nothing.
+    // Content-Length is a mandatory header. It specifies the length of the
+    // following JSON.
+    if (LineRef.consume_front("Content-Length: "))
+      llvm::getAsUnsignedInteger(LineRef.trim(), 0, Len);
+
+    // Check if the next line only contains \r\n. If not this is another header,
+    // which we ignore.
+    char NewlineBuf[2];
+    std::cin.read(NewlineBuf, 2);
+    if (std::memcmp(NewlineBuf, "\r\n", 2) != 0)
+      continue;
+
+    // Now read the JSON. Insert a trailing null byte as required by the YAML
+    // parser.
+    std::vector<char> JSON(Len + 1);
+    std::cin.read(JSON.data(), Len);
+
+    if (Len > 0) {
+      // Log the message.
+      Logs << "<-- ";
+      Logs.write(JSON.data(), JSON.size());
+      Logs << '\n';
+      Logs.flush();
+
+      // Finally, execute the action for this JSON message.
+      if (!Dispatcher.call(llvm::StringRef(JSON.data(), JSON.size() - 1)))
+        Logs << "JSON dispatch failed!\n";
+    }
+  }
+}

clang-tools-extra/trunk/clangd/DocumentStore.h

+//===--- DocumentStore.h - File contents container --------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_DOCUMENTSTORE_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_DOCUMENTSTORE_H
+
+#include "clang/Basic/LLVM.h"
+#include "llvm/ADT/StringMap.h"
+#include <string>
+
+namespace clang {
+namespace clangd {
+
+/// A container for files opened in a workspace, addressed by URI. The contents
+/// are owned by the DocumentStore.
+class DocumentStore {
+public:
+  /// Add a document to the store. Overwrites existing contents.
+  void addDocument(StringRef Uri, StringRef Text) { Docs[Uri] = Text; }
+  /// Delete a document from the store.
+  void removeDocument(StringRef Uri) { Docs.erase(Uri); }
+  /// Retrieve a document from the store. Empty string if it's unknown.
+  StringRef getDocument(StringRef Uri) const { return Docs.lookup(Uri); }
+
+private:
+  llvm::StringMap<std::string> Docs;
+};
+
+} // namespace clangd
+} // namespace clang
+
+#endif

clang-tools-extra/trunk/clangd/JSONRPCDispatcher.h

+//===--- JSONRPCDispatcher.h - Main JSON parser entry point -----*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_JSONRPCDISPATCHER_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_JSONRPCDISPATCHER_H
+
+#include "clang/Basic/LLVM.h"
+#include "llvm/ADT/StringMap.h"
+#include "llvm/Support/YAMLParser.h"
+
+namespace clang {
+namespace clangd {
+
+/// Callback for messages sent to the server, called by the JSONRPCDispatcher.
+class Handler {
+public:
+  Handler(llvm::raw_ostream &Outs, llvm::raw_ostream &Logs)
+      : Outs(Outs), Logs(Logs) {}
+  virtual ~Handler() = default;
+
+  /// Called when the server receives a method call. This is supposed to return
+  /// a result on Outs. The default implementation returns an "unknown method"
+  /// error to the client and logs a warning.
+  virtual void handleMethod(llvm::yaml::MappingNode *Params, StringRef ID);
+  /// Called when the server receives a notification. No result should be
+  /// written to Outs. The default implemetation logs a warning.
+  virtual void handleNotification(llvm::yaml::MappingNode *Params);
+
+protected:
+  llvm::raw_ostream &Outs;
+  llvm::raw_ostream &Logs;
+
+  /// Helper to write a JSONRPC result to Outs.
+  void writeMessage(const Twine &Message);
+};
+
+/// Main JSONRPC entry point. This parses the JSONRPC "header" and calls the
+/// registered Handler for the method received.
+class JSONRPCDispatcher {
+public:
+  /// Create a new JSONRPCDispatcher. UnknownHandler is called when an unknown
+  /// method is received.
+  JSONRPCDispatcher(std::unique_ptr<Handler> UnknownHandler)
+      : UnknownHandler(std::move(UnknownHandler)) {}
+
+  /// Registers a Handler for the specified Method.
+  void registerHandler(StringRef Method, std::unique_ptr<Handler> H);
+
+  /// Parses a JSONRPC message and calls the Handler for it.
+  bool call(StringRef Content) const;
+
+private:
+  llvm::StringMap<std::unique_ptr<Handler>> Handlers;
+  std::unique_ptr<Handler> UnknownHandler;
+};
+
+} // namespace clangd
+} // namespace clang
+
+#endif

clang-tools-extra/trunk/clangd/JSONRPCDispatcher.cpp

+//===--- JSONRPCDispatcher.cpp - Main JSON parser entry point -------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "JSONRPCDispatcher.h"
+#include "ProtocolHandlers.h"
+#include "llvm/ADT/SmallString.h"
+#include "llvm/Support/SourceMgr.h"
+#include "llvm/Support/YAMLParser.h"
+using namespace clang;
+using namespace clangd;
+
+void Handler::writeMessage(const Twine &Message) {
+  llvm::SmallString<128> Storage;
+  StringRef M = Message.toStringRef(Storage);
+
+  // Log without headers.
+  Logs << "--> " << M << '\n';
+  Logs.flush();
+
+  // Emit message with header.
+  Outs << "Content-Length: " << M.size() << "\r\n\r\n" << M;
+  Outs.flush();
+}
+
+void Handler::handleMethod(llvm::yaml::MappingNode *Params, StringRef ID) {
+  Logs << "Method ignored.\n";
+  // Return that this method is unsupported.
+  writeMessage(
+      R"({"jsonrpc":"2.0","id":)" + ID +
+      R"(,"error":{"code":-32601}})");
+}
+
+void Handler::handleNotification(llvm::yaml::MappingNode *Params) {
+  Logs << "Notification ignored.\n";
+}
+
+void JSONRPCDispatcher::registerHandler(StringRef Method,
+                                        std::unique_ptr<Handler> H) {
+  assert(!Handlers.count(Method) && "Handler already registered!");
+  Handlers[Method] = std::move(H);
+}
+
+static void
+callHandler(const llvm::StringMap<std::unique_ptr<Handler>> &Handlers,
+            llvm::yaml::ScalarNode *Method, llvm::yaml::ScalarNode *Id,
+            llvm::yaml::MappingNode *Params, Handler *UnknownHandler) {
+  llvm::SmallString<10> MethodStorage;
+  auto I = Handlers.find(Method->getValue(MethodStorage));
+  auto *Handler = I != Handlers.end() ? I->second.get() : UnknownHandler;
+  if (Id)
+    Handler->handleMethod(Params, Id->getRawValue());
+  else
+    Handler->handleNotification(Params);
+}
+
+bool JSONRPCDispatcher::call(StringRef Content) const {
+  llvm::SourceMgr SM;
+  llvm::yaml::Stream YAMLStream(Content, SM);
+
+  auto Doc = YAMLStream.begin();
+  if (Doc == YAMLStream.end())
+    return false;
+
+  auto *Root = Doc->getRoot();
+  if (!Root)
+    return false;
+
+  auto *Object = dyn_cast<llvm::yaml::MappingNode>(Root);
+  if (!Object)
+    return false;
+
+  llvm::yaml::ScalarNode *Version = nullptr;
+  llvm::yaml::ScalarNode *Method = nullptr;
+  llvm::yaml::MappingNode *Params = nullptr;
+  llvm::yaml::ScalarNode *Id = nullptr;
+  for (auto &NextKeyValue : *Object) {
+    auto *KeyString = dyn_cast<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
+    if (!KeyString)
+      return false;
+
+    llvm::SmallString<10> KeyStorage;
+    StringRef KeyValue = KeyString->getValue(KeyStorage);
+    llvm::yaml::Node *Value = NextKeyValue.getValue();
+    if (!Value)
+      return false;
+
+    if (KeyValue == "jsonrpc") {
+      // This should be "2.0". Always.
+      Version = dyn_cast<llvm::yaml::ScalarNode>(Value);
+      if (!Version || Version->getRawValue() != "\"2.0\"")
+        return false;
+    } else if (KeyValue == "method") {
+      Method = dyn_cast<llvm::yaml::ScalarNode>(Value);
+    } else if (KeyValue == "id") {
+      Id = dyn_cast<llvm::yaml::ScalarNode>(Value);
+    } else if (KeyValue == "params") {
+      if (!Method)
+        return false;
+      // We have to interleave the call of the function here, otherwise the
+      // YAMLParser will die because it can't go backwards. This is unfortunate
+      // because it will break clients that put the id after params. A possible
+      // fix would be to split the parsing and execution phases.
+      Params = dyn_cast<llvm::yaml::MappingNode>(Value);
+      callHandler(Handlers, Method, Id, Params, UnknownHandler.get());
+      return true;
+    } else {
+      return false;
+    }
+  }
+
+  // In case there was a request with no params, call the handler on the
+  // leftovers.
+  if (!Method)
+    return false;
+  callHandler(Handlers, Method, Id, nullptr, UnknownHandler.get());
+
+  return true;
+}

clang-tools-extra/trunk/clangd/Protocol.h

+//===--- Protocol.h - Language Server Protocol Implementation ---*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file contains structs based on the LSP specification at
+// https://github.com/Microsoft/language-server-protocol/blob/master/protocol.md
+//
+// This is not meant to be a complete implementation, new interfaces are added
+// when they're needed.
+//
+// Each struct has a parse and unparse function, that converts back and forth
+// between the struct and a JSON representation.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_PROTOCOL_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_PROTOCOL_H
+
+#include "llvm/ADT/Optional.h"
+#include "llvm/Support/YAMLParser.h"
+#include <string>
+
+namespace clang {
+namespace clangd {
+
+struct TextDocumentIdentifier {
+  /// The text document's URI.
+  std::string uri;
+
+  static llvm::Optional<TextDocumentIdentifier>
+  parse(llvm::yaml::MappingNode *Params);
+};
+
+struct Position {
+  /// Line position in a document (zero-based).
+  int line;
+
+  /// Character offset on a line in a document (zero-based).
+  int character;
+
+  static llvm::Optional<Position> parse(llvm::yaml::MappingNode *Params);
+  static std::string unparse(const Position &P);
+};
+
+struct Range {
+  /// The range's start position.
+  Position start;
+
+  /// The range's end position.
+  Position end;
+
+  static llvm::Optional<Range> parse(llvm::yaml::MappingNode *Params);
+  static std::string unparse(const Range &P);
+};
+
+struct TextEdit {
+  /// The range of the text document to be manipulated. To insert
+  /// text into a document create a range where start === end.
+  Range range;
+
+  /// The string to be inserted. For delete operations use an
+  /// empty string.
+  std::string newText;
+
+  static llvm::Optional<TextEdit> parse(llvm::yaml::MappingNode *Params);
+  static std::string unparse(const TextEdit &P);
+};
+
+struct TextDocumentItem {
+  /// The text document's URI.
+  std::string uri;
+
+  /// The text document's language identifier.
+  std::string languageId;
+
+  /// The version number of this document (it will strictly increase after each
+  int version;
+
+  /// The content of the opened text document.
+  std::string text;
+
+  static llvm::Optional<TextDocumentItem>
+  parse(llvm::yaml::MappingNode *Params);
+};
+
+struct DidOpenTextDocumentParams {
+  /// The document that was opened.
+  TextDocumentItem textDocument;
+
+  static llvm::Optional<DidOpenTextDocumentParams>
+  parse(llvm::yaml::MappingNode *Params);
+};
+
+struct TextDocumentContentChangeEvent {
+  /// The new text of the document.
+  std::string text;
+
+  static llvm::Optional<TextDocumentContentChangeEvent>
+  parse(llvm::yaml::MappingNode *Params);
+};
+
+struct DidChangeTextDocumentParams {
+  /// The document that did change. The version number points
+  /// to the version after all provided content changes have
+  /// been applied.
+  TextDocumentIdentifier textDocument;
+
+  /// The actual content changes.
+  std::vector<TextDocumentContentChangeEvent> contentChanges;
+
+  static llvm::Optional<DidChangeTextDocumentParams>
+  parse(llvm::yaml::MappingNode *Params);
+};
+
+struct FormattingOptions {
+  /// Size of a tab in spaces.
+  int tabSize;
+
+  /// Prefer spaces over tabs.
+  bool insertSpaces;
+
+  static llvm::Optional<FormattingOptions>
+  parse(llvm::yaml::MappingNode *Params);
+  static std::string unparse(const FormattingOptions &P);
+};
+
+struct DocumentRangeFormattingParams {
+  /// The document to format.
+  TextDocumentIdentifier textDocument;
+
+  /// The range to format
+  Range range;
+
+  /// The format options
+  FormattingOptions options;
+
+  static llvm::Optional<DocumentRangeFormattingParams>
+  parse(llvm::yaml::MappingNode *Params);
+};
+
+struct DocumentFormattingParams {
+  /// The document to format.
+  TextDocumentIdentifier textDocument;
+
+  /// The format options
+  FormattingOptions options;
+
+  static llvm::Optional<DocumentFormattingParams>
+  parse(llvm::yaml::MappingNode *Params);
+};
+
+} // namespace clangd
+} // namespace clang
+
+#endif

clang-tools-extra/trunk/clangd/Protocol.cpp

+//===--- Protocol.cpp - Language Server Protocol Implementation -----------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file contains the serialization code for the LSP structs.
+// FIXME: This is extremely repetetive and ugly. Is there a better way?
+//
+//===----------------------------------------------------------------------===//
+
+#include "Protocol.h"
+#include "clang/Basic/LLVM.h"
+#include "llvm/ADT/SmallString.h"
+#include "llvm/Support/Format.h"
+#include "llvm/Support/raw_ostream.h"
+using namespace clang::clangd;
+
+llvm::Optional<TextDocumentIdentifier>
+TextDocumentIdentifier::parse(llvm::yaml::MappingNode *Params) {
+  TextDocumentIdentifier Result;
+  for (auto &NextKeyValue : *Params) {
+    auto *KeyString = dyn_cast<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
+    if (!KeyString)
+      return llvm::None;
+
+    llvm::SmallString<10> KeyStorage;
+    StringRef KeyValue = KeyString->getValue(KeyStorage);
+    auto *Value =
+        dyn_cast_or_null<llvm::yaml::ScalarNode>(NextKeyValue.getValue());
+    if (!Value)
+      return llvm::None;
+
+    llvm::SmallString<10> Storage;
+    if (KeyValue == "uri") {
+      Result.uri = Value->getValue(Storage);
+    } else if (KeyValue == "version") {
+      // FIXME: parse version, but only for VersionedTextDocumentIdentifiers.
+    } else {
+      return llvm::None;
+    }
+  }
+  return Result;
+}
+
+llvm::Optional<Position> Position::parse(llvm::yaml::MappingNode *Params) {
+  Position Result;
+  for (auto &NextKeyValue : *Params) {
+    auto *KeyString = dyn_cast<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
+    if (!KeyString)
+      return llvm::None;
+
+    llvm::SmallString<10> KeyStorage;
+    StringRef KeyValue = KeyString->getValue(KeyStorage);
+    auto *Value =
+        dyn_cast_or_null<llvm::yaml::ScalarNode>(NextKeyValue.getValue());
+    if (!Value)
+      return llvm::None;
+
+    llvm::SmallString<10> Storage;
+    if (KeyValue == "line") {
+      long long Val;
+      if (llvm::getAsSignedInteger(Value->getValue(Storage), 0, Val))
+        return llvm::None;
+      Result.line = Val;
+    } else if (KeyValue == "character") {
+      long long Val;
+      if (llvm::getAsSignedInteger(Value->getValue(Storage), 0, Val))
+        return llvm::None;
+      Result.character = Val;
+    } else {
+      return llvm::None;
+    }
+  }
+  return Result;
+}
+
+std::string Position::unparse(const Position &P) {
+  std::string Result;
+  llvm::raw_string_ostream(Result)
+      << llvm::format(R"({"line": %d, "character": %d})", P.line, P.character);
+  return Result;
+}
+
+llvm::Optional<Range> Range::parse(llvm::yaml::MappingNode *Params) {
+  Range Result;
+  for (auto &NextKeyValue : *Params) {
+    auto *KeyString = dyn_cast<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
+    if (!KeyString)
+      return llvm::None;
+
+    llvm::SmallString<10> KeyStorage;
+    StringRef KeyValue = KeyString->getValue(KeyStorage);
+    auto *Value =
+        dyn_cast_or_null<llvm::yaml::MappingNode>(NextKeyValue.getValue());
+    if (!Value)
+      return llvm::None;
+
+    llvm::SmallString<10> Storage;
+    if (KeyValue == "start") {
+      auto Parsed = Position::parse(Value);
+      if (!Parsed)
+        return llvm::None;
+      Result.start = std::move(*Parsed);
+    } else if (KeyValue == "end") {
+      auto Parsed = Position::parse(Value);
+      if (!Parsed)
+        return llvm::None;
+      Result.end = std::move(*Parsed);
+    } else {
+      return llvm::None;
+    }
+  }
+  return Result;
+}
+
+std::string Range::unparse(const Range &P) {
+  std::string Result;
+  llvm::raw_string_ostream(Result) << llvm::format(
+      R"({"start": %s, "end": %s})", Position::unparse(P.start).c_str(),
+      Position::unparse(P.end).c_str());
+  return Result;
+}
+
+llvm::Optional<TextDocumentItem>
+TextDocumentItem::parse(llvm::yaml::MappingNode *Params) {
+  TextDocumentItem Result;
+  for (auto &NextKeyValue : *Params) {
+    auto *KeyString = dyn_cast<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
+    if (!KeyString)
+      return llvm::None;
+
+    llvm::SmallString<10> KeyStorage;
+    StringRef KeyValue = KeyString->getValue(KeyStorage);
+    auto *Value =
+        dyn_cast_or_null<llvm::yaml::ScalarNode>(NextKeyValue.getValue());
+    if (!Value)
+      return llvm::None;
+
+    llvm::SmallString<10> Storage;
+    if (KeyValue == "uri") {
+      Result.uri = Value->getValue(Storage);
+    } else if (KeyValue == "languageId") {
+      Result.languageId = Value->getValue(Storage);
+    } else if (KeyValue == "version") {
+      long long Val;
+      if (llvm::getAsSignedInteger(Value->getValue(Storage), 0, Val))
+        return llvm::None;
+      Result.version = Val;
+    } else if (KeyValue == "text") {
+      Result.text = Value->getValue(Storage);
+    } else {
+      return llvm::None;
+    }
+  }
+  return Result;
+}
+
+llvm::Optional<TextEdit> TextEdit::parse(llvm::yaml::MappingNode *Params) {
+  TextEdit Result;
+  for (auto &NextKeyValue : *Params) {
+    auto *KeyString = dyn_cast<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
+    if (!KeyString)
+      return llvm::None;
+
+    llvm::SmallString<10> KeyStorage;
+    StringRef KeyValue = KeyString->getValue(KeyStorage);
+    auto *Value = NextKeyValue.getValue();
+
+    llvm::SmallString<10> Storage;
+    if (KeyValue == "range") {
+      auto *Map = dyn_cast<llvm::yaml::MappingNode>(Value);
+      if (!Map)
+        return llvm::None;
+      auto Parsed = Range::parse(Map);
+      if (!Parsed)
+        return llvm::None;
+      Result.range = std::move(*Parsed);
+    } else if (KeyValue == "newText") {
+      auto *Node = dyn_cast<llvm::yaml::ScalarNode>(Value);
+      if (!Node)
+        return llvm::None;
+      Result.newText = Node->getValue(Storage);
+    } else {
+      return llvm::None;
+    }
+  }
+  return Result;
+}
+
+std::string TextEdit::unparse(const TextEdit &P) {
+  std::string Result;
+  llvm::raw_string_ostream(Result) << llvm::format(
+      R"({"range": %s, "newText": "%s"})", Range::unparse(P.range).c_str(),
+      llvm::yaml::escape(P.newText).c_str());
+  return Result;
+}
+
+llvm::Optional<DidOpenTextDocumentParams>
+DidOpenTextDocumentParams::parse(llvm::yaml::MappingNode *Params) {
+  DidOpenTextDocumentParams Result;
+  for (auto &NextKeyValue : *Params) {
+    auto *KeyString = dyn_cast<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
+    if (!KeyString)
+      return llvm::None;
+
+    llvm::SmallString<10> KeyStorage;
+    StringRef KeyValue = KeyString->getValue(KeyStorage);
+    auto *Value =
+        dyn_cast_or_null<llvm::yaml::MappingNode>(NextKeyValue.getValue());
+    if (!Value)
+      return llvm::None;
+
+    llvm::SmallString<10> Storage;
+    if (KeyValue == "textDocument") {
+      auto Parsed = TextDocumentItem::parse(Value);
+      if (!Parsed)
+        return llvm::None;
+      Result.textDocument = std::move(*Parsed);
+    } else {
+      return llvm::None;
+    }
+  }
+  return Result;
+}
+
+llvm::Optional<DidChangeTextDocumentParams>
+DidChangeTextDocumentParams::parse(llvm::yaml::MappingNode *Params) {
+  DidChangeTextDocumentParams Result;
+  for (auto &NextKeyValue : *Params) {
+    auto *KeyString = dyn_cast<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
+    if (!KeyString)
+      return llvm::None;
+
+    llvm::SmallString<10> KeyStorage;
+    StringRef KeyValue = KeyString->getValue(KeyStorage);
+    auto *Value = NextKeyValue.getValue();
+
+    llvm::SmallString<10> Storage;
+    if (KeyValue == "textDocument") {
+      auto *Map = dyn_cast<llvm::yaml::MappingNode>(Value);
+      if (!Map)
+        return llvm::None;
+      auto Parsed = TextDocumentIdentifier::parse(Map);
+      if (!Parsed)
+        return llvm::None;
+      Result.textDocument = std::move(*Parsed);
+    } else if (KeyValue == "contentChanges") {
+      auto *Seq = dyn_cast<llvm::yaml::SequenceNode>(Value);
+      if (!Seq)
+        return llvm::None;
+      for (auto &Item : *Seq) {
+        auto *I = dyn_cast<llvm::yaml::MappingNode>(&Item);
+        if (!I)
+          return llvm::None;
+        auto Parsed = TextDocumentContentChangeEvent::parse(I);
+        if (!Parsed)
+          return llvm::None;
+        Result.contentChanges.push_back(std::move(*Parsed));
+      }
+    } else {
+      return llvm::None;
+    }
+  }
+  return Result;
+}
+
+llvm::Optional<TextDocumentContentChangeEvent>
+TextDocumentContentChangeEvent::parse(llvm::yaml::MappingNode *Params) {
+  TextDocumentContentChangeEvent Result;
+  for (auto &NextKeyValue : *Params) {
+    auto *KeyString = dyn_cast<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
+    if (!KeyString)
+      return llvm::None;
+
+    llvm::SmallString<10> KeyStorage;
+    StringRef KeyValue = KeyString->getValue(KeyStorage);
+    auto *Value =
+        dyn_cast_or_null<llvm::yaml::ScalarNode>(NextKeyValue.getValue());
+    if (!Value)
+      return llvm::None;
+
+    llvm::SmallString<10> Storage;
+    if (KeyValue == "text") {
+      Result.text = Value->getValue(Storage);
+    } else {
+      return llvm::None;
+    }
+  }
+  return Result;
+}
+
+llvm::Optional<FormattingOptions>
+FormattingOptions::parse(llvm::yaml::MappingNode *Params) {
+  FormattingOptions Result;
+  for (auto &NextKeyValue : *Params) {
+    auto *KeyString = dyn_cast<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
+    if (!KeyString)
+      return llvm::None;
+
+    llvm::SmallString<10> KeyStorage;
+    StringRef KeyValue = KeyString->getValue(KeyStorage);
+    auto *Value =
+        dyn_cast_or_null<llvm::yaml::ScalarNode>(NextKeyValue.getValue());
+    if (!Value)
+      return llvm::None;
+
+    llvm::SmallString<10> Storage;
+    if (KeyValue == "tabSize") {
+      long long Val;
+      if (llvm::getAsSignedInteger(Value->getValue(Storage), 0, Val))
+        return llvm::None;
+      Result.tabSize = Val;
+    } else if (KeyValue == "insertSpaces") {
+      long long Val;
+      StringRef Str = Value->getValue(Storage);
+      if (llvm::getAsSignedInteger(Str, 0, Val)) {
+        if (Str == "true")
+          Val = 1;
+        else if (Str == "false")
+          Val = 0;
+        else
+          return llvm::None;
+      }
+      Result.insertSpaces = Val;
+    } else {
+      return llvm::None;
+    }
+  }
+  return Result;
+}
+
+std::string FormattingOptions::unparse(const FormattingOptions &P) {
+  std::string Result;
+  llvm::raw_string_ostream(Result) << llvm::format(
+      R"({"tabSize": %d, "insertSpaces": %d})", P.tabSize, P.insertSpaces);
+  return Result;
+}
+
+llvm::Optional<DocumentRangeFormattingParams>
+DocumentRangeFormattingParams::parse(llvm::yaml::MappingNode *Params) {
+  DocumentRangeFormattingParams Result;
+  for (auto &NextKeyValue : *Params) {
+    auto *KeyString = dyn_cast<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
+    if (!KeyString)
+      return llvm::None;
+
+    llvm::SmallString<10> KeyStorage;
+    StringRef KeyValue = KeyString->getValue(KeyStorage);
+    auto *Value =
+        dyn_cast_or_null<llvm::yaml::MappingNode>(NextKeyValue.getValue());
+    if (!Value)
+      return llvm::None;
+
+    llvm::SmallString<10> Storage;
+    if (KeyValue == "textDocument") {
+      auto Parsed = TextDocumentIdentifier::parse(Value);
+      if (!Parsed)
+        return llvm::None;
+      Result.textDocument = std::move(*Parsed);
+    } else if (KeyValue == "range") {
+      auto Parsed = Range::parse(Value);
+      if (!Parsed)
+        return llvm::None;
+      Result.range = std::move(*Parsed);
+    } else if (KeyValue == "options") {
+      auto Parsed = FormattingOptions::parse(Value);
+      if (!Parsed)
+        return llvm::None;
+      Result.options = std::move(*Parsed);
+    } else {
+      return llvm::None;
+    }
+  }
+  return Result;
+}
+
+llvm::Optional<DocumentFormattingParams>
+DocumentFormattingParams::parse(llvm::yaml::MappingNode *Params) {
+  DocumentFormattingParams Result;
+  for (auto &NextKeyValue : *Params) {
+    auto *KeyString = dyn_cast<llvm::yaml::ScalarNode>(NextKeyValue.getKey());
+    if (!KeyString)
+      return llvm::None;
+
+    llvm::SmallString<10> KeyStorage;
+    StringRef KeyValue = KeyString->getValue(KeyStorage);
+    auto *Value =
+        dyn_cast_or_null<llvm::yaml::MappingNode>(NextKeyValue.getValue());
+    if (!Value)
+      return llvm::None;
+
+    llvm::SmallString<10> Storage;
+    if (KeyValue == "textDocument") {
+      auto Parsed = TextDocumentIdentifier::parse(Value);
+      if (!Parsed)
+        return llvm::None;
+      Result.textDocument = std::move(*Parsed);
+    } else if (KeyValue == "options") {
+      auto Parsed = FormattingOptions::parse(Value);
+      if (!Parsed)
+        return llvm::None;
+      Result.options = std::move(*Parsed);
+    } else {
+      return llvm::None;
+    }
+  }
+  return Result;
+}

clang-tools-extra/trunk/clangd/ProtocolHandlers.h

+//===--- ProtocolHandlers.h - LSP callbacks ---------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file contains the actions performed when the server gets a specific
+// request.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_PROTOCOLHANDLERS_H
+#define LLVM_CLANG_TOOLS_EXTRA_CLANGD_PROTOCOLHANDLERS_H
+
+#include "JSONRPCDispatcher.h"
+#include "Protocol.h"
+#include "llvm/ADT/Twine.h"
+#include "llvm/Support/raw_ostream.h"
+
+namespace clang {
+namespace clangd {
+class DocumentStore;
+
+struct InitializeHandler : Handler {
+  InitializeHandler(llvm::raw_ostream &Outs, llvm::raw_ostream &Logs)
+      : Handler(Outs, Logs) {}
+
+  void handleMethod(llvm::yaml::MappingNode *Params, StringRef ID) override {
+    writeMessage(
+        R"({"jsonrpc":"2.0","id":)" + ID +
+        R"(,"result":{"capabilities":{
+          "textDocumentSync": 1,
+          "documentFormattingProvider": true,
+          "documentRangeFormattingProvider": true
+        }}})");
+  }
+};
+
+struct ShutdownHandler : Handler {
+  ShutdownHandler(llvm::raw_ostream &Outs, llvm::raw_ostream &Logs)
+      : Handler(Outs, Logs) {}
+
+  void handleMethod(llvm::yaml::MappingNode *Params, StringRef ID) override {
+    // FIXME: Calling exit is rude, can we communicate to main somehow?
+    exit(0);
+  }
+};
+
+struct TextDocumentDidOpenHandler : Handler {
+  TextDocumentDidOpenHandler(llvm::raw_ostream &Outs, llvm::raw_ostream &Logs,
+                             DocumentStore &Store)
+      : Handler(Outs, Logs), Store(Store) {}
+
+  void handleNotification(llvm::yaml::MappingNode *Params) override;
+
+private:
+  DocumentStore &Store;
+};
+
+struct TextDocumentDidChangeHandler : Handler {
+  TextDocumentDidChangeHandler(llvm::raw_ostream &Outs, llvm::raw_ostream &Logs,
+                               DocumentStore &Store)
+      : Handler(Outs, Logs), Store(Store) {}
+
+  void handleNotification(llvm::yaml::MappingNode *Params) override;
+
+private:
+  DocumentStore &Store;
+};
+
+struct TextDocumentRangeFormattingHandler : Handler {
+  TextDocumentRangeFormattingHandler(llvm::raw_ostream &Outs,
+                                     llvm::raw_ostream &Logs,
+                                     DocumentStore &Store)
+      : Handler(Outs, Logs), Store(Store) {}
+
+  void handleMethod(llvm::yaml::MappingNode *Params, StringRef ID) override;
+
+private:
+  DocumentStore &Store;
+};
+
+struct TextDocumentFormattingHandler : Handler {
+  TextDocumentFormattingHandler(llvm::raw_ostream &Outs,
+                                llvm::raw_ostream &Logs, DocumentStore &Store)
+      : Handler(Outs, Logs), Store(Store) {}
+
+  void handleMethod(llvm::yaml::MappingNode *Params, StringRef ID) override;
+
+private:
+  DocumentStore &Store;
+};
+
+} // namespace clangd
+} // namespace clang
+
+#endif

clang-tools-extra/trunk/clangd/ProtocolHandlers.cpp

+//===--- ProtocolHandlers.cpp - LSP callbacks -----------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "ProtocolHandlers.h"
+#include "DocumentStore.h"
+#include "clang/Format/Format.h"
+using namespace clang;
+using namespace clangd;
+
+void TextDocumentDidOpenHandler::handleNotification(
+    llvm::yaml::MappingNode *Params) {
+  auto DOTDP = DidOpenTextDocumentParams::parse(Params);
+  if (!DOTDP) {
+    Logs << "Failed to decode DidOpenTextDocumentParams!\n";
+    return;
+  }
+  Store.addDocument(DOTDP->textDocument.uri, DOTDP->textDocument.text);
+}
+
+void TextDocumentDidChangeHandler::handleNotification(
+    llvm::yaml::MappingNode *Params) {
+  auto DCTDP = DidChangeTextDocumentParams::parse(Params);
+  if (!DCTDP || DCTDP->contentChanges.size() != 1) {
+    Logs << "Failed to decode DidChangeTextDocumentParams!\n";
+    return;
+  }
+  // We only support full syncing right now.
+  Store.addDocument(DCTDP->textDocument.uri, DCTDP->contentChanges[0].text);
+}
+
+/// Turn a [line, column] pair into an offset in Code.
+static size_t positionToOffset(StringRef Code, Position P) {
+  size_t Offset = 0;
+  for (int I = 0; I != P.line; ++I) {
+    // FIXME: \r\n
+    // FIXME: UTF-8
+    size_t F = Code.find('\n', Offset);
+    if (F == StringRef::npos)
+      return 0; // FIXME: Is this reasonable?
+    Offset = F + 1;
+  }
+  return (Offset == 0 ? 0 : (Offset - 1)) + P.character;
+}
+
+/// Turn an offset in Code into a [line, column] pair.
+static Position offsetToPosition(StringRef Code, size_t Offset) {
+  StringRef JustBefore = Code.substr(0, Offset);
+  // FIXME: \r\n
+  // FIXME: UTF-8
+  int Lines = JustBefore.count('\n');
+  int Cols = JustBefore.size() - JustBefore.rfind('\n') - 1;
+  return {Lines, Cols};
+}
+
+static std::string formatCode(StringRef Code, StringRef Filename,
+                              ArrayRef<tooling::Range> Ranges, StringRef ID) {
+  // Call clang-format.
+  // FIXME: Don't ignore style.
+  format::FormatStyle Style = format::getLLVMStyle();
+  tooling::Replacements Replacements =
+      format::reformat(Style, Code, Ranges, Filename);
+
+  // Now turn the replacements into the format specified by the Language Server
+  // Protocol. Fuse them into one big JSON array.
+  std::string Edits;
+  for (auto &R : Replacements) {
+    Range ReplacementRange = {
+        offsetToPosition(Code, R.getOffset()),
+        offsetToPosition(Code, R.getOffset() + R.getLength())};
+    TextEdit TE = {ReplacementRange, R.getReplacementText()};
+    Edits += TextEdit::unparse(TE);
+    Edits += ',';
+  }
+  if (!Edits.empty())
+    Edits.pop_back();
+
+  return R"({"jsonrpc":"2.0","id":)" + ID.str() +
+         R"(,"result":[)" + Edits + R"(]})";
+}
+
+void TextDocumentRangeFormattingHandler::handleMethod(
+    llvm::yaml::MappingNode *Params, StringRef ID) {
+  auto DRFP = DocumentRangeFormattingParams::parse(Params);
+  if (!DRFP) {
+    Logs << "Failed to decode DocumentRangeFormattingParams!\n";
+    return;
+  }
+
+  StringRef Code = Store.getDocument(DRFP->textDocument.uri);
+
+  size_t Begin = positionToOffset(Code, DRFP->range.start);
+  size_t Len = positionToOffset(Code, DRFP->range.end) - Begin;
+
+  writeMessage(formatCode(Code, DRFP->textDocument.uri,
+                          {clang::tooling::Range(Begin, Len)}, ID));
+}
+
+void TextDocumentFormattingHandler::handleMethod(
+    llvm::yaml::MappingNode *Params, StringRef ID) {
+  auto DFP = DocumentFormattingParams::parse(Params);
+  if (!DFP) {
+    Logs << "Failed to decode DocumentFormattingParams!\n";
+    return;
+  }
+
+  // Format everything.
+  StringRef Code = Store.getDocument(DFP->textDocument.uri);
+  writeMessage(formatCode(Code, DFP->textDocument.uri,
+                          {clang::tooling::Range(0, Code.size())}, ID));
+}

clang-tools-extra/trunk/test/CMakeLists.txt

set(CLANG_TOOLS_TEST_DEPS
   clang-headers
 
   # For the clang-tidy libclang integration test.
   c-index-test
 
   # Individual tools we test.
   clang-apply-replacements
   clang-change-namespace
+  clangd
   clang-include-fixer
   clang-move
   clang-query
   clang-rename
   clang-reorder-fields
   clang-tidy
   find-all-symbols
   modularize

clang-tools-extra/trunk/test/clangd/formatting.test

+# RUN: sed -e '/^#/d' %s | clangd | FileCheck %s
+# It is absolutely vital that this file has CRLF line endings.
+#
+Content-Length: 125
+
+{"jsonrpc":"2.0","id":0,"method":"initialize","params":{"processId":123,"rootPath":"clangd","capabilities":{},"trace":"off"}}
+# CHECK: Content-Length: 191
+# CHECK: {"jsonrpc":"2.0","id":0,"result":{"capabilities":{
+# CHECK:   "textDocumentSync": 1,
+# CHECK:   "documentFormattingProvider": true,
+# CHECK:   "documentRangeFormattingProvider": true
+# CHECK: }}}
+#
+Content-Length: 193
+
+{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"file:///foo.c","languageId":"c","version":1,"text":"int foo ( int x ) {\n    x = x+1;\n    return x;\n    }"}}}
+#
+#
+Content-Length: 233
+
+{"jsonrpc":"2.0","id":1,"method":"textDocument/rangeFormatting","params":{"textDocument":{"uri":"file:///foo.c"},"range":{"start":{"line":1,"character":4},"end":{"line":1,"character":12}},"options":{"tabSize":4,"insertSpaces":true}}}
+# CHECK: {"jsonrpc":"2.0","id":1,"result":[{"range": {"start": {"line": 0, "character": 19}, "end": {"line": 1, "character": 4}}, "newText": "\n  "},{"range": {"start": {"line": 1, "character": 9}, "end": {"line": 1, "character": 9}}, "newText": " "},{"range": {"start": {"line": 1, "character": 10}, "end": {"line": 1, "character": 10}}, "newText": " "},{"range": {"start": {"line": 1, "character": 12}, "end": {"line": 2, "character": 4}}, "newText": "\n  "}]}
+#
+#
+Content-Length: 197
+
+{"jsonrpc":"2.0","method":"textDocument/didChange","params":{"textDocument":{"uri":"file:///foo.c","version":5},"contentChanges":[{"text":"int foo ( int x ) {\n  x = x + 1;\n  return x;\n    }"}]}}
+#
+#
+Content-Length: 233
+
+{"jsonrpc":"2.0","id":2,"method":"textDocument/rangeFormatting","params":{"textDocument":{"uri":"file:///foo.c"},"range":{"start":{"line":1,"character":2},"end":{"line":1,"character":12}},"options":{"tabSize":4,"insertSpaces":true}}}
+# CHECK: {"jsonrpc":"2.0","id":2,"result":[]}
+#
+Content-Length: 153
+
+{"jsonrpc":"2.0","id":3,"method":"textDocument/formatting","params":{"textDocument":{"uri":"file:///foo.c"},"options":{"tabSize":4,"insertSpaces":true}}}
+# CHECK: {"jsonrpc":"2.0","id":3,"result":[{"range": {"start": {"line": 0, "character": 7}, "end": {"line": 0, "character": 8}}, "newText": ""},{"range": {"start": {"line": 0, "character": 9}, "end": {"line": 0, "character": 10}}, "newText": ""},{"range": {"start": {"line": 0, "character": 15}, "end": {"line": 0, "character": 16}}, "newText": ""},{"range": {"start": {"line": 2, "character": 11}, "end": {"line": 3, "character": 4}}, "newText": "\n"}]}
+#
+#
+Content-Length: 190
+
+{"jsonrpc":"2.0","method":"textDocument/didChange","params":{"textDocument":{"uri":"file:///foo.c","version":9},"contentChanges":[{"text":"int foo(int x) {\n  x = x + 1;\n  return x;\n}"}]}}
+#
+#
+Content-Length: 153
+
+{"jsonrpc":"2.0","id":4,"method":"textDocument/formatting","params":{"textDocument":{"uri":"file:///foo.c"},"options":{"tabSize":4,"insertSpaces":true}}}
+# CHECK: {"jsonrpc":"2.0","id":4,"result":[]}
+#
+Content-Length: 44
+
+{"jsonrpc":"2.0","id":5,"method":"shutdown"}