Remove possibility to change compile database path at runtime

[simark]

Summary

This patch removes the possibility to change the compilation database
path at runtime using the didChangeConfiguration request. Instead, it
is suggested to use the setting on the initialize request, and clangd
whenever the user wants to use a different build configuration.

[sammccall]

Thanks for cleaning this up!

clangd/ClangdLSPServer.cpp

435–436

This isn't needed, the compilation database can only be set during initialization.

clangd/ClangdLSPServer.h

90 ↗

(On Diff #169493)

Prefer a different name for this function - an overload set should have similar semantics and these are quite different (pseudo-constructor vs mutation allowed at any time).

clangd/Protocol.h

422

Can we just move this to InitializeParams as a clangd extension?
Doing tricks with inheritance here makes the protocol harder to understand.

[simark]

clangd/ClangdLSPServer.h

90 ↗

(On Diff #169493)

Ok, I'll think about a better name.

clangd/Protocol.h

422

Can we just move this to InitializeParams as a clangd extension?

Do you mean move it in the JSON, so it looks like this on the wire?

{
  "method": "initialize",
  "params": {
    "compilationDatabasePath": "<path>",
    ...
  }
}

instead of

{
  "method": "initialize",
  "params": {
    "initializationOptions": {
          "compilationDatabasePath": "<path>"
    },
    ...
  }
}

?

I think initializationOptions is a good place for this to be, I wouldn't change that.. If you don't like the inheritance, we can just get rid of it in our code and have two separate versions of the deserializing code. We designed it so didChangeConfiguration and the initialization options would share the same structure, but it doesn't have to stay that way.

[sammccall]

I think it'd be a good idea to separate out the on-initialization vs dynamically-changing parameters more - I think they should probably be disjoint in fact.

But we can discuss/implement that separately from this patch - the change itself is really nice.

clangd/ClangdLSPServer.cpp

435–436

It's still here... maybe forgot to upload a new diff?
(Just to be clear: I meant reparseOpenFiles doesn't need to be called, as there are none.)

clangd/ClangdLSPServer.h

90 ↗

(On Diff #169493)

(In fact, this could also live directly in onInitialize, I think?)

clangd/Protocol.h

422

Do you mean move it in the JSON, so it looks like this on the wire?

Well, since you asked... :-) I'm not going to push hard for it (this patch is certainly a win already), but I do think that would be much clearer.

The current protocol has InitializeParams and ClangdInitializationOptions, and it's not clear semantically what the distinction between them is.

With hindsight, I think something like this would be easier to follow:

// Clangd options that may be set at startup.
struct InitializeParams {
  // Clangd extension: override the path used to load the CDB.
  Optional<string> compilationDatabasePath;
  // Provides initial configuration as if by workspace/updateConfiguration.
  Optional<ClangdConfigurationParamsChange> initialConfiguration;
}
// Clangd options that may be set dynamically at runtime.
struct ClangdConfigurationParamsChange { ... }

though even here, the benefit from being able to inline the initial configuration into the initalize message is unclear to me. The implementation has to support dynamic updates in any case, so why not make use of that?

We designed it so didChangeConfiguration and the initialization options would share the same structure

This makes sense, but if they're diverging, I'm not sure that keeping them *mostly* the same brings more benefits than confusion.

---

That said, if you prefer to keep the JSON as it is, that's fine. (If we grow more extensions, we may want to reorganize in future though?)
My main concern is the use of inheritance here, and how it provides a default (configuration-change options can be provided at startup) that doesn't seem obviously correct and is hard to opt out of.

[simark]

clangd/ClangdLSPServer.cpp

435–436

Oh I have only done the change locally, I have not uploaded a new diff yet.

clangd/ClangdLSPServer.h

90 ↗

(On Diff #169493)

Indeed, since it's things that can only be set during initialization.

clangd/Protocol.h

422

The current protocol has InitializeParams and ClangdInitializationOptions, and it's not clear semantically what the distinction between them is.

InitializeParams is the type for the parameters of the initialize request. In it, there is this field:

	/**
	 * User provided initialization options.
	 */
	initializationOptions?: any;

which is made to pass such language-server-specific options. Since it's of the any type, we gave it a name, ClangdInitializationOptions.

With hindsight, I think something like this would be easier to follow:

[snippet]

I don't really understand why putting fields as extra fields in InitializeParams would be any easier than putting them in the object that exists specifically for that purpose.

though even here, the benefit from being able to inline the initial configuration into the initalize message is unclear to me. The implementation has to support dynamic updates in any case, so why not make use of that?

I'd say, to avoid having the server start some work (like indexing some files using a certain set of flags) that will be invalidated when it receives some config changes moments later.

This makes sense, but if they're diverging, I'm not sure that keeping them *mostly* the same brings more benefits than confusion.

I think you are exaggerating the confusion. It is pretty straightforward: everything you can change during execution, you can also specify at initialize time.

[simark]

Address comments

[sammccall]

Still LG!

clangd/Protocol.h

422

which is made to pass such language-server-specific options. Since it's of the any type, we gave it a name, ClangdInitializationOptions.

Fair enough. I personally think initializationOptions in LSP is not useful as specified for various reasons, and would rather avoid it (it's optional), but following the spirit of the spec makes sense too.

I'd say, to avoid having the server start some work (like indexing some files using a certain set of flags) that will be invalidated when it receives some config changes moments later.

Doing this to enable optimizations definitely makes sense.

It does seem premature when we don't have any such optimization: it puts more burden on clients by giving them multiple equivalent ways to do things, and hinting that they're not quite equivalent.

clangd/ClangdLSPServer.cpp

for (size_t I = CompletionItemKindMin;
        I <= static_cast<size_t>(CompletionItemKind::Reference); ++I)
     Defaults.set(I);
   return Defaults;
 }
 
 } // namespace
 
 void ClangdLSPServer::onInitialize(InitializeParams &Params) {
-  if (Params.initializationOptions)
-    applyConfiguration(*Params.initializationOptions);
+  if (Params.initializationOptions) {
+    const ClangdInitializationOptions &Opts = *Params.initializationOptions;
+
+    // Explicit compilation database path.
+    if (Opts.compilationDatabasePath.hasValue()) {
+      CDB.setCompileCommandsDir(Opts.compilationDatabasePath.getValue());
+    }
+
+    applyConfiguration(Opts.ParamsChange);
+  }
 
   if (Params.rootUri && *Params.rootUri)
     Server->setRootPath(Params.rootUri->file());
   else if (Params.rootPath && !Params.rootPath->empty())
     Server->setRootPath(*Params.rootPath);
 
   CCOpts.EnableSnippets =
       Params.capabilities.textDocument.completion.completionItem.snippetSupport;
Server->findHover(Params.textDocument.uri.file(), Params.position,
                                    llvm::toString(H.takeError()));
                         return;
                       }
 
                       reply(*H);
                     });
 }
 
 void ClangdLSPServer::applyConfiguration(
-    const ClangdConfigurationParamsChange &Settings) {
-  // Compilation database change.
-  if (Settings.compilationDatabasePath.hasValue()) {
-    CDB.setCompileCommandsDir(Settings.compilationDatabasePath.getValue());
-
-    reparseOpenedFiles();
-  }
-
-  // Update to the compilation database.
-  if (Settings.compilationDatabaseChanges) {
-    const auto &CompileCommandUpdates = *Settings.compilationDatabaseChanges;
+    const ClangdConfigurationParamsChange &Params) {
+  // Per-file update to the compilation database.
+  if (Params.compilationDatabaseChanges) {
+    const auto &CompileCommandUpdates = *Params.compilationDatabaseChanges;
     bool ShouldReparseOpenFiles = false;
     for (auto &Entry : CompileCommandUpdates) {
       /// The opened files need to be reparsed only when some existing
       /// entries are changed.
       PathRef File = Entry.first;
       if (!CDB.setCompilationCommandForFile(
               File, tooling::CompileCommand(
                         std::move(Entry.second.workingDirectory), File,

clangd/Protocol.h

struct ClangdCompileCommand {
   std::vector<std::string> compilationCommand;
 };
 bool fromJSON(const llvm::json::Value &, ClangdCompileCommand &);
 
 /// Clangd extension to set clangd-specific "initializationOptions" in the
 /// "initialize" request and for the "workspace/didChangeConfiguration"
 /// notification since the data received is described as 'any' type in LSP.
 struct ClangdConfigurationParamsChange {
-  llvm::Optional<std::string> compilationDatabasePath;
-
   // The changes that happened to the compilation database.
   // The key of the map is a file name.
   llvm::Optional<std::map<std::string, ClangdCompileCommand>>
       compilationDatabaseChanges;
 };
 bool fromJSON(const llvm::json::Value &, ClangdConfigurationParamsChange &);
 
-struct ClangdInitializationOptions : public ClangdConfigurationParamsChange {};
+struct ClangdInitializationOptions {
+  // What we can change throught the didChangeConfiguration request, we can
+  // also set through the initialize request (initializationOptions field).
+  ClangdConfigurationParamsChange ParamsChange;
+
+  llvm::Optional<std::string> compilationDatabasePath;
+};
+bool fromJSON(const llvm::json::Value &, ClangdInitializationOptions &);
 
 struct InitializeParams {
   /// The process Id of the parent process that started
   /// the server. Is null if the process has not been started by another
   /// process. If the parent process is not alive then the server should exit
   /// (see exit notification) its process.
   llvm::Optional<int> processId;
 

clangd/Protocol.cpp

bool fromJSON(const llvm::json::Value &Params,
   json::ObjectMapper O(Params);
   return O && O.map("workingDirectory", CDbUpdate.workingDirectory) &&
          O.map("compilationCommand", CDbUpdate.compilationCommand);
 }
 
 bool fromJSON(const json::Value &Params,
               ClangdConfigurationParamsChange &CCPC) {
   json::ObjectMapper O(Params);
-  return O && O.map("compilationDatabasePath", CCPC.compilationDatabasePath) &&
+  return O &&
          O.map("compilationDatabaseChanges", CCPC.compilationDatabaseChanges);
 }
 
+bool fromJSON(const json::Value &Params, ClangdInitializationOptions &Opts) {
+  if (!fromJSON(Params, Opts.ParamsChange)) {
+    return false;
+  }
+
+  json::ObjectMapper O(Params);
+  return O && O.map("compilationDatabasePath", Opts.compilationDatabasePath);
+}
+
 bool fromJSON(const json::Value &Params, ReferenceParams &R) {
   TextDocumentPositionParams &Base = R;
   return fromJSON(Params, Base);
 }
 
 } // namespace clangd
 } // namespace clang

test/clangd/compile-commands-path-in-initialize.test

 # Test that we can set choose a configuration/build directly in the initialize
 # request.
 
 # RUN: rm -rf %t.dir/* && mkdir -p %t.dir
 # RUN: mkdir %t.dir/build-1
-# RUN: mkdir %t.dir/build-2
 # RUN: echo '[{"directory": "%/t.dir", "command": "c++ the-file.cpp", "file": "the-file.cpp"}]' > %t.dir/compile_commands.json
 # RUN: echo '[{"directory": "%/t.dir/build-1", "command": "c++ -DMACRO=1 the-file.cpp", "file": "../the-file.cpp"}]' > %t.dir/build-1/compile_commands.json
-# RUN: echo '[{"directory": "%/t.dir/build-2", "command": "c++ -DMACRO=2 the-file.cpp", "file": "../the-file.cpp"}]' > %t.dir/build-2/compile_commands.json
 
 # RUN: sed -e "s|INPUT_DIR|%/t.dir|g" %s > %t.test.1
 
 # On Windows, we need the URI in didOpen to look like "uri":"file:///C:/..."
 # (with the extra slash in the front), so we add it here.
 # RUN: sed -e "s|file://\([A-Z]\):/|file:///\1:/|g" %t.test.1 > %t.test
 
 # RUN: clangd -lit-test < %t.test | FileCheck -strict-whitespace %t.test
 
 {"jsonrpc":"2.0","id":0,"method":"initialize","params":{"initializationOptions":{"compilationDatabasePath":"INPUT_DIR/build-1"}}}
 ---
-{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"file://INPUT_DIR/the-file.cpp","languageId":"cpp","version":1,"text":"#if !defined(MACRO)\n#pragma message (\"MACRO is not defined\")\n#elif MACRO == 1\n#pragma message (\"MACRO is one\")\n#elif MACRO == 2\n#pragma message (\"MACRO is two\")\n#else\n#pragma message (\"woops\")\n#endif\nint main() {}\n"}}}
+{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"file://INPUT_DIR/the-file.cpp","languageId":"cpp","version":1,"text":"#if !defined(MACRO)\n#pragma message (\"MACRO is not defined\")\n#elif MACRO == 1\n#pragma message (\"MACRO is one\")\n#else\n#pragma message (\"woops\")\n#endif\nint main() {}\n"}}}
 # CHECK:   "method": "textDocument/publishDiagnostics",
 # CHECK-NEXT:   "params": {
 # CHECK-NEXT:     "diagnostics": [
 # CHECK-NEXT:       {
 # CHECK-NEXT:         "message": "MACRO is one",
 ---
-{"jsonrpc":"2.0","id":0,"method":"workspace/didChangeConfiguration","params":{"settings":{"compilationDatabasePath":"INPUT_DIR/build-2"}}}
-# CHECK:   "method": "textDocument/publishDiagnostics",
-# CHECK-NEXT:   "params": {
-# CHECK-NEXT:     "diagnostics": [
-# CHECK-NEXT:       {
-# CHECK-NEXT:         "message": "MACRO is two",
----
 {"jsonrpc":"2.0","id":10000,"method":"shutdown"}

test/clangd/compile-commands-path.test

-# Test that we can switch between configuration/build using the
-# workspace/didChangeConfiguration notification.
-
-# RUN: rm -rf %t.dir/* && mkdir -p %t.dir
-# RUN: mkdir %t.dir/build-1
-# RUN: mkdir %t.dir/build-2
-# RUN: echo '[{"directory": "%/t.dir", "command": "c++ the-file.cpp", "file": "the-file.cpp"}]' > %t.dir/compile_commands.json
-# RUN: echo '[{"directory": "%/t.dir/build-1", "command": "c++ -DMACRO=1 the-file.cpp", "file": "../the-file.cpp"}]' > %t.dir/build-1/compile_commands.json
-# RUN: echo '[{"directory": "%/t.dir/build-2", "command": "c++ -DMACRO=2 the-file.cpp", "file": "../the-file.cpp"}]' > %t.dir/build-2/compile_commands.json
-
-# RUN: sed -e "s|INPUT_DIR|%/t.dir|g" %s > %t.test.1
-
-# On Windows, we need the URI in didOpen to look like "uri":"file:///C:/..."
-# (with the extra slash in the front), so we add it here.
-# RUN: sed -e "s|file://\([A-Z]\):/|file:///\1:/|g" %t.test.1 > %t.test
-
-# RUN: clangd -lit-test < %t.test | FileCheck -strict-whitespace %t.test
-
-{"jsonrpc":"2.0","id":0,"method":"initialize","params":{}}
----
-{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"file://INPUT_DIR/the-file.cpp","languageId":"cpp","version":1,"text":"#if !defined(MACRO)\n#pragma message (\"MACRO is not defined\")\n#elif MACRO == 1\n#pragma message (\"MACRO is one\")\n#elif MACRO == 2\n#pragma message (\"MACRO is two\")\n#else\n#pragma message (\"woops\")\n#endif\nint main() {}\n"}}}
-# CHECK:   "method": "textDocument/publishDiagnostics",
-# CHECK-NEXT:   "params": {
-# CHECK-NEXT:     "diagnostics": [
-# CHECK-NEXT:       {
-# CHECK-NEXT:         "message": "MACRO is not defined",
----
-{"jsonrpc":"2.0","id":0,"method":"workspace/didChangeConfiguration","params":{"settings":{"compilationDatabasePath":"INPUT_DIR/build-1"}}}
-# CHECK:   "method": "textDocument/publishDiagnostics",
-# CHECK-NEXT:   "params": {
-# CHECK-NEXT:     "diagnostics": [
-# CHECK-NEXT:       {
-# CHECK-NEXT:         "message": "MACRO is one",
----
-{"jsonrpc":"2.0","id":0,"method":"workspace/didChangeConfiguration","params":{"settings":{"compilationDatabasePath":"INPUT_DIR/build-2"}}}
-# CHECK:   "method": "textDocument/publishDiagnostics",
-# CHECK-NEXT:   "params": {
-# CHECK-NEXT:     "diagnostics": [
-# CHECK-NEXT:       {
-# CHECK-NEXT:         "message": "MACRO is two",
----
-{"jsonrpc":"2.0","id":10000,"method":"shutdown"}