[clangd] Generate ConfigFragment/YAML/docs from one tablegen source

[sammccall]

Summary

The idea is to reduce the number of places that need to be updated as we add
more config settings, encourage regular patterns, and make cross-cutting changes
easier (e.g. reformat docs).

[nridge]

The patch is failing to build for me locally, with clang-tools-extra/clangd/ConfigFragment.h:100:10: fatal error: 'ConfigFragment.gen.h' file not found. I did re-run cmake and the error persists. Is there another step I'm missing?

[sammccall]

In D115425#3190418, @nridge wrote:

The patch is failing to build for me locally, with clang-tools-extra/clangd/ConfigFragment.h:100:10: fatal error: 'ConfigFragment.gen.h' file not found. I did re-run cmake and the error persists. Is there another step I'm missing?

You're not, sorry. It should be #include "ConfigFragment.inc". (I renamed the generated file and it kept working for me locally because cmake doesn't clean up the old one).

I had a discussion with @kadircet about this, we're not sure whether it factors out enough to be better/simpler overall. (Hand-written ConfigFragment + ConfigYAML + docs, vs table + generator + stubs of ConfigFragment + ConfigYAML + docs).
Any thoughts?

[nridge]

In D115425#3190603, @sammccall wrote:

I had a discussion with @kadircet about this, we're not sure whether it factors out enough to be better/simpler overall. (Hand-written ConfigFragment + ConfigYAML + docs, vs table + generator + stubs of ConfigFragment + ConfigYAML + docs).
Any thoughts?

I feel like clangd's lack of support for non-self-contained files (https://github.com/clangd/clangd/issues/45) is relevant here.

With this patch, opening up the .inc files gives you a bunch of errors, and refs in those files do not appear to be found. So, while the patch makes it easier to add new config options, reading / understanding the relevant code becomes harder. Since code is read more often than it's written, that may not be a win.

Additionally, there's perhaps a matter of principle where clangd should try to avoid using code patterns (here, non-self-contained files) in its implementation code that it does not support operating on as a tool.

These make me lean towards not doing this.

[sammccall]

That bothers me too.
I'm sure it's possible to factor the generated files so that they're self-contained, I didn't work hard on that yet. So I don't think it's central to the question.

there's perhaps a matter of principle where clangd should try to avoid using code patterns (here, non-self-contained files) in its implementation code that it does not support

Agree. However the counterargument is putting such patterns in front of our face all the time might encourage us to add some level of support!

[nridge]

In D115425#3190698, @sammccall wrote:

there's perhaps a matter of principle where clangd should try to avoid using code patterns (here, non-self-contained files) in its implementation code that it does not support

Agree. However the counterargument is putting such patterns in front of our face all the time might encourage us to add some level of support!

Apologies if this is veering off topic (we can continue in a Github Discussion if you'd like), but I got the impression that clangd's lack of support for non-self-contained files was at least partly an opinionated statement that code patterns involving such files are not an important part of modern C++ development.

If that impression is mistaken, and it's just a matter of no one having gotten around to implementing such support, perhaps we could clarify that in https://github.com/clangd/clangd/issues/45 (and maybe put out a "help wanted" / "contributions welcome" announcement somewhere)?

[sammccall]

In D115425#3190740, @nridge wrote:

In D115425#3190698, @sammccall wrote:

there's perhaps a matter of principle where clangd should try to avoid using code patterns (here, non-self-contained files) in its implementation code that it does not support

Agree. However the counterargument is putting such patterns in front of our face all the time might encourage us to add some level of support!

Apologies if this is veering off topic (we can continue in a Github Discussion if you'd like), but I got the impression that clangd's lack of support for non-self-contained files was at least partly an opinionated statement that code patterns involving such files are not an important part of modern C++ development.

I think "at least partly" is fair. But this is more a statement of priorities and effort - I think if the feature were free or cheap, everyone would be happy to have it.

(FWIW I do find LLVM's reliance on preprocessor tricks to be not great. Between wanting to fit in with existing infrastructure, and not being able to easily add dependencies, it doesn't seem worth fighting. But this is just another way to say "sure, there are circumstances where it's reasonable to use such patterns")

If that impression is mistaken, and it's just a matter of no one having gotten around to implementing such support

It's definitely more than implementation, the design needs to solve some hard constraints:

• There needs to be a clear scope and idea of how it'll solve the problems. For instance:
  • does it support files with parse context (e.g. snippets of class bodies like ConfigFragment.inc), or just PP context? If parse context, how is this persisted or recreated? (PCH can't do this today). How does this interact with preambles?
  • how is the file-that-provides-context identified? Is the existing index actually good/available enough for this?
• it needs to not add _too much_ complexity, or at least isolate the complexity to a few places. "Priorities and effort" also means this can't become a maintenance timesink.
• it needs to be useful/powerful enough to justify the complexity that adds. (This ~certainly means it needs to be safe to turn on by default eventually)

I'm not certain it's possible to satisfy all these, but it seems like it might be possible.

perhaps we could clarify that in https://github.com/clangd/clangd/issues/45 (and maybe put out a "help wanted" / "contributions welcome" announcement somewhere)?

Agreed. I'll post a version of this here.
I'm a little worried that people will expect that if someone makes an honest attempt and gets anywhere then it'll be merged. But I should be less paranoid.
I'm also aware that I & others haven't been a model of working on designs collaboratively in public, we're pretty reliant on getting in a room. So it may be unfair to expect others to do better.

[njames93]

For the website, we use tags to specify the clangd version that the option was first supported in.
I'd suggest that we also add a Version field to the Field class.
This could also cause problem down the line if we ever wanted to remove a config option.

[sammccall]

In D115425#3222782, @njames93 wrote:

For the website, we use tags to specify the clangd version that the option was first supported in.
I'd suggest that we also add a Version field to the Field class.
This could also cause problem down the line if we ever wanted to remove a config option.

I definitely agree.
FWIW, I didn't add all features or send this for review because I wasn't sure about the idea.

Talked to @kadircet offline a bit, and I hope he doesn't mind me trying to summarize...

• avoiding a couple of duplicated parts is definitely good
• there's still config.h and the mapping thereto, which I don't think can be tablegenerated
• tablegen syntax is really bad and also hard to browse with the code (I agree)
• I don't see a better representation within the tablegen language (which is dumb, because this is a simple tree, but tablegen isn't good at trees)
• We couldn't think of a representation that would be *nice* to edit/maintain that's easy to have in-tree (apart from the current C++ structs, but parsing C++ at build time is terrible)

Actually we didn't discuss the option of using YAML itself as the format.
The downsides I can think of (vs current C++):

• no type system/checking/assistance for our weird DSL (tablegen a bit)
• have to use strings as comments rather than using comments as comments (though at least YAML has nice strings for this purpose)
• YAML is a pretty quirky language
• the inherent level-confusion of using YAML to describe a YAML schema

Maybe I'll mock this up, but I have some more pressing things to do so if anyone wants to shoot the idea first that'd be nice :-)

clang-tools-extra/clangd/CMakeLists.txt

set(LLVM_LINK_COMPONENTS
   AllTargetsInfos
   FrontendOpenMP
   Option
   )
 
 include(${CMAKE_CURRENT_SOURCE_DIR}/quality/CompletionModel.cmake)
 gen_decision_forest(${CMAKE_CURRENT_SOURCE_DIR}/quality/model CompletionModel clang::clangd::Example)
 
+clang_tablegen(ConfigFragment.inc -gen-clangd-config-header SOURCE Config.td TARGET clangd_config_fragment_gen)
+clang_tablegen(ConfigYAML.inc -gen-clangd-config-yaml SOURCE Config.td TARGET clangd_config_yaml_gen)
+clang_tablegen(ConfigDocs.md -gen-clangd-config-docs SOURCE Config.td TARGET clangd_config_docs_gen)
+
 if(MSVC AND NOT CLANG_CL)
  set_source_files_properties(CompileCommands.cpp PROPERTIES COMPILE_FLAGS -wd4130) # disables C4130: logical operation on address of string constant
 endif()
 
 include_directories(BEFORE "${CMAKE_CURRENT_BINARY_DIR}/../clang-tidy")
 
 add_clang_library(clangDaemon
   AST.cpp
add_clang_library(clangDaemon
   index/dex/PostingList.cpp
   index/dex/Trigram.cpp
 
   refactor/Rename.cpp
   refactor/Tweak.cpp
 
   DEPENDS
   omp_gen
+  clangd_config_fragment_gen
+  clangd_config_yaml_gen
   )
 
 # Include generated CompletionModel headers.
 target_include_directories(clangDaemon PUBLIC
   $<BUILD_INTERFACE:${CMAKE_CURRENT_BINARY_DIR}>
 )
 
 clang_target_link_libraries(clangDaemon

clang-tools-extra/clangd/Config.td

+class Field;
+
+// Possible types of fields.
+class Type {
+  string spelling = ?;
+  bit nullable = 0; // True if the type has an "empty" value.
+  string field_spelling = !if(nullable, spelling,
+                              "llvm::Optional<" # spelling # ">");
+}
+// Primitives.
+def String : Type { let spelling = "Located<std::string>"; }
+def Boolean : Type { let spelling = "Located<bool>"; }
+// Lists (JSON array or YAML sequence)
+class List<Type T> : Type {
+  Type element = T;
+  let spelling = "std::vector<" # T.spelling # ">";
+  let nullable = 1;
+}
+// Key-value pairs, keys are arbitrary strings (JSON object or YAML mapping)
+class KeyValues<Type T> : Type {
+  Type element = T;
+  let spelling = "std::vector<std::pair<Located<std::string>, " # T.spelling # ">>";
+  let nullable = 1;
+}
+// Structure with enumerated fields (JSON object or YAML mapping node)
+class Struct<string Name, list<Field> Fields, bit Nullable> : Type {
+  string struct_name = Name # "Block";
+  // Retain location info only if presence is significant.
+  let spelling = !if(nullable, struct_name, "Located<" # struct_name # ">");
+  string name = Name;
+  list<Field> fields = Fields;
+  let nullable = Nullable;
+}
+
+// Fields appear within structs.
+class Field<string Name, Type T, string Doc = ""> {
+  string doc = Doc;
+  bit parsed = 1;
+  bit documented = 1;
+  string name = Name;
+  Type type = T;
+}
+// An unparsed field will not have marshalling code.
+// It will be documented only if the docstring is nonempty.
+class Unparsed<Field F> : Field<F.name, F.type, F.doc> {
+  let parsed = 0;
+  let documented = !ne(doc, "");
+}
+// A block is a field Foo whose type is a structure named FooBlock.
+class Block<string Name, string Doc, list<Field> Fields>
+    : Field<Name, Struct<Name, Fields, /*Nullable=*/1>, Doc> {
+}
+// The presence/absence of an optional block is significant.
+class OptionalBlock<string Name, string Doc, list<Field> Fields>
+    : Field<Name, Struct<Name, Fields, /*Nullable=*/0>, Doc> {
+}
+
+def Fragment : Struct<"Fragment", [
+  Block<"If", [{
+    Conditions in the If block restrict when a Fragment applies.
+
+    ```yaml
+    If:                               # Apply this config conditionally
+      PathMatch: .*\.h                # to all headers...
+      PathExclude: include/llvm-c/.*  # except those under include/llvm-c/
+    ```
+
+    Each separate condition must match (combined with AND).
+    When one condition has multiple values, any may match (combined with OR).
+    e.g. `PathMatch: [foo/.*, bar/.*]` matches files in either directory.
+
+    Conditions based on a file's path use the following form:
+
+    - if the fragment came from a project directory, the path is relative
+    - if the fragment is global (e.g. user config), the path is absolute
+    - paths always use forward-slashes (UNIX-style)
+
+    If no file is being processed, these conditions will not match.
+  }], [
+    Field<"PathMatch", List<String>,
+      [{The file being processed must fully match a regular expression.}]>,
+    Field<"PathExclude", List<String>,
+      [{The file being processed must *not* fully match a regular expression.}]>,
+    Unparsed<Field<"HasUnrecognizedCondition", Boolean>>,
+  ]>,
+
+  Block<"CompileFlags", [{
+    Affects how a file is parsed.
+
+    ```yaml
+    CompileFlags:                 # Tweak the parse settings
+      Add: [-xc++, -Wall]         # treat all files as C++, enable more warnings
+      Remove: -W*                 # strip all other warning-related flags
+    ```
+
+    clangd emulates how clang would interpret a file.
+    By default, it behaves roughly like `clang $FILENAME`, but real projects
+    usually require setting the include path (with the `-I` flag), defining
+    preprocessor symbols, configuring warnings etc.
+
+    Often, a compilation database specifies these compile commands. clangd
+    searches for compile_commands.json in parents of the source file.
+
+    This section modifies how the compile command is constructed.
+  }], [
+    Field<"Add", List<String>,
+      [{List of flags to append to the compile command.}]>,
+    Field<"Remove", List<String>, [{
+      List of flags to remove from the compile command.
+
+      - If the value is a recognized clang flag (like `-I`) then it will be
+        removed along with any arguments. Synonyms like `--include-dir=` will
+        also be removed.
+      - Otherwise, if the value ends in `*` (like `-DFOO=*`) then any argument
+        with the prefix will be removed.
+      - Otherwise any argument exactly matching the value is removed.
+
+      In all cases, `-Xclang` is also removed where needed.
+
+      Example:
+
+      - Command: `clang++ --include-directory=/usr/include -DFOO=42 foo.cc`
+      - Configuration: `Remove: [-I, -DFOO=*]`
+      - Result: `clang++ foo.cc`
+
+      Flags added by the same CompileFlags entry will not be removed.
+    }]>,
+    Field<"CompilationDatabase", String, [{
+      Directory to search for compilation database (compile_comands.json etc).
+
+      Valid values are:
+
+      - A single path to a directory (absolute, or relative to the fragment)
+      - Ancestors: search all parent directories (the default)
+      - None: do not use a compilation database, just default flags.
+    }]>,
+  ]>,
+
+  Block<"Index", [{
+    Controls how clangd understands code outside the current file.
+
+    ```yaml
+    Index:
+      Background: Skip     # Disable slow background indexing of these files.
+    ```
+
+    clangd's indexes provide information about symbols that isn't available
+    to clang's parser, such as incoming references.
+  }], [
+    Field<"Background", String, [{
+      Whether files are built in the background to produce a project index.
+      This is checked for translation units only, not headers they include.
+      Legal values are `Build` (the default) or `Skip`.
+    }]>,
+    Unparsed<OptionalBlock<"External", [{
+      Used to define an external index source, such as an index file or server.
+
+      ```yaml
+      Index:
+        External:
+          File: /abs/path/to/an/index.idx
+          # OR
+          Server: my.index.server.com:50051
+          MountPoint: /files/under/this/project/
+      ```
+
+      Exactly one of `File` or `Server` needs to be specified.
+
+      Declaring an `External` index disables background-indexing implicitly for
+      files under the `MountPoint`. Users can turn it back on, by explicitly
+      mentioning `Background: Build` in a later fragment.
+    }], [
+      Field<"File", String,
+        [{Path to an monolithic index file produced by `clangd-indexer`}]>,
+      Field<"Server", String,
+        [{Address of a remote index server}]>,
+      Field<"MountPoint", String,
+        [{Specifies the source root for the index, needed for relative path
+        conversions. Defaults to the location of this config fragment.
+        In a project config, this can be a relative path}]>,
+      Unparsed<Field<"IsNone", Boolean>>,
+    ]>>,
+  ]>,
+
+  Block<"Diagnostics", [{
+    Controls behavior of diagnostics (errors and warnings).
+  }], [
+    Field<"Suppress", List<String>, [{
+      Diagnostic codes that should be suppressed.
+
+      Valid values are:
+
+      - `*`, to disable all diagnostics
+      - diagnostic codes exposed by clangd (e.g `unknown_type`, `-Wunused-result`)
+      - clang internal diagnostic codes (e.g. `err_unknown_type`)
+      - warning categories (e.g. `unused-result`)
+      - clang-tidy check names (e.g. `bugprone-narrowing-conversions`)
+
+      This is a simple filter. Diagnostics can be controlled in other ways
+      (e.g. by disabling a clang-tidy check, or the `-Wunused` compile flag).
+      This often has other advantages, such as skipping some analysis.
+    }]>,
+    Field<"UnusedIncludes", String>,
+    Block<"ClangTidy", [{
+      Controls how clang-tidy will run over the code base.
+
+      The settings are merged with any settings found in .clang-tidy
+      configiration files with these ones taking precedence.
+    }], [
+      Field<"Add", List<String>,
+      [{List of checks to enable, can be globs.}]>,
+      Field<"Remove", List<String>, [{
+        List of checks to disable, can be globs.
+
+        This takes precedence over Add, this supports enabling all checks from a module apart from some specific checks.
+
+        Example to use all modernize module checks apart from use trailing return type:
+
+        ```yaml
+        Diagnostics:
+          ClangTidy:
+            Add: modernize*
+            Remove: modernize-use-trailing-return-type
+        ```
+      }]>,
+      Field<"CheckOptions", KeyValues<String>, [{
+        Key-value pairs of options for clang-tidy checks.
+        Available options for all checks can be found [here](https://clang.llvm.org/extra/clang-tidy/checks/list.html).
+
+        Note the format here is slightly different to `.clang-tidy` configuration
+        files as we don't specify `key: <key>, value: <value>`. Instead just use
+        `<key>: <value>`
+
+        ```yaml
+        Diagnostics:
+          ClangTidy:
+            CheckOptions:
+              readability-identifier-naming.VariableCase: CamelCase
+        ```
+      }]>,
+    ]>,
+  ]>,
+
+  Block<"Style", [{
+    Describes the style of the codebase, beyond formatting.
+  }], [
+    Field<"FullyQualifiedNamespaces", List<String>, [{
+      Namespaces that should always be fully qualified, meaning no "using"
+      declarations, always spell out the whole name (with or without leading::).
+      All nested namespaces are affected as well.
+      Affects availability of the AddUsing tweak.
+    }]>,
+  ]>,
+
+  Block<"Completion", [{
+    Describes code completion preferences.
+  }], [
+    Field<"AllScopes", Boolean, [{
+      Whether code completion should include suggestions from scopes that are
+      not visible. The required scope prefix will be inserted.
+    }]>,
+  ]>,
+
+  Block<"Hover", [{
+    Describes hover preferences.
+  }], [
+    Field<"ShowAKA", Boolean,
+      [{Whether a.k.a. types are shown for type aliases}]>,
+  ]>,
+], /*Nullable=*/1> {
+  let struct_name = name; // Fragment, not FragmentBlock
+}

clang-tools-extra/clangd/ConfigCompile.cpp

if (External.Server) {
       elog("Clangd isn't compiled with remote index support, ignoring Server: "
            "{0}",
            *External.Server);
       External.Server.reset();
     }
 #endif
     // Make sure exactly one of the Sources is set.
     unsigned SourceCount = External.File.hasValue() +
-                           External.Server.hasValue() + *External.IsNone;
+                           External.Server.hasValue() +
+                           External.IsNone.hasValue();
     if (SourceCount != 1) {
       diag(Error, "Exactly one of File, Server or None must be set.",
            BlockRange);
       return;
     }
     Config::ExternalIndexSpec Spec;
     if (External.Server) {
       Spec.Kind = Config::ExternalIndexSpec::Server;
       Spec.Location = std::move(**External.Server);
     } else if (External.File) {
       Spec.Kind = Config::ExternalIndexSpec::File;
       auto AbsPath = makeAbsolute(std::move(*External.File), "File",
                                   llvm::sys::path::Style::native);
       if (!AbsPath)
         return;
       Spec.Location = std::move(*AbsPath);
     } else {
-      assert(*External.IsNone);
+      assert(**External.IsNone);
       Spec.Kind = Config::ExternalIndexSpec::None;
     }
     if (Spec.Kind != Config::ExternalIndexSpec::None) {
       // Make sure MountPoint is an absolute path with forward slashes.
       if (!External.MountPoint)
         External.MountPoint.emplace(FragmentDirectory);
       if ((**External.MountPoint).empty()) {
         diag(Error, "A mountpoint is required.", BlockRange);

clang-tools-extra/clangd/ConfigFragment.h

 // configuration as obtained from a source like a file.
 //
 // This is distinct from how the config is interpreted (CompiledFragment),
 // combined (Provider) and exposed to the rest of clangd (Config).
 //
 //===----------------------------------------------------------------------===//
 //
 // To add a new configuration option, you must:
-//  - add its syntactic form to Fragment
-//  - update ConfigYAML.cpp to parse it
+//  - add its syntactic form to Config.td
+//    (This is used to generate Fragment, ConfigYAML, and documentation)
 //  - add its semantic form to Config (in Config.h)
 //  - update ConfigCompile.cpp to map Fragment -> Config
 //  - make use of the option inside clangd
-//  - document the new option (config.md in the llvm/clangd-www repository)
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_TOOLS_EXTRA_CLANGD_CONFIGFRAGMENT_H
 #define LLVM_CLANG_TOOLS_EXTRA_CLANGD_CONFIGFRAGMENT_H
 
 #include "ConfigProvider.h"
 #include "llvm/ADT/Optional.h"
-#include "llvm/ADT/STLExtras.h"
-#include "llvm/Support/Error.h"
 #include "llvm/Support/SMLoc.h"
 #include "llvm/Support/SourceMgr.h"
 #include <string>
 #include <vector>
 
 namespace clang {
 namespace clangd {
 namespace config {
struct SourceInfo {
     /// paths mentioned in the fragment are resolved against this.
     std::string Directory;
     /// Whether this fragment is allowed to make critical security/privacy
     /// decisions.
     bool Trusted = false;
   };
   SourceInfo Source;
 
-  /// Conditions in the If block restrict when a Fragment applies.
-  ///
-  /// Each separate condition must match (combined with AND).
-  /// When one condition has multiple values, any may match (combined with OR).
-  /// e.g. `PathMatch: [foo/.*, bar/.*]` matches files in either directory.
-  ///
-  /// Conditions based on a file's path use the following form:
-  /// - if the fragment came from a project directory, the path is relative
-  /// - if the fragment is global (e.g. user config), the path is absolute
-  /// - paths always use forward-slashes (UNIX-style)
-  /// If no file is being processed, these conditions will not match.
-  struct IfBlock {
-    /// The file being processed must fully match a regular expression.
-    std::vector<Located<std::string>> PathMatch;
-    /// The file being processed must *not* fully match a regular expression.
-    std::vector<Located<std::string>> PathExclude;
-
-    /// An unrecognized key was found while parsing the condition.
-    /// The condition will evaluate to false.
-    bool HasUnrecognizedCondition = false;
-  };
-  IfBlock If;
-
-  /// Conditions in the CompileFlags block affect how a file is parsed.
-  ///
-  /// clangd emulates how clang would interpret a file.
-  /// By default, it behaves roughly like `clang $FILENAME`, but real projects
-  /// usually require setting the include path (with the `-I` flag), defining
-  /// preprocessor symbols, configuring warnings etc.
-  /// Often, a compilation database specifies these compile commands. clangd
-  /// searches for compile_commands.json in parents of the source file.
-  ///
-  /// This section modifies how the compile command is constructed.
-  struct CompileFlagsBlock {
-    /// List of flags to append to the compile command.
-    std::vector<Located<std::string>> Add;
-    /// List of flags to remove from the compile command.
-    ///
-    /// - If the value is a recognized clang flag (like "-I") then it will be
-    ///   removed along with any arguments. Synonyms like --include-dir= will
-    ///   also be removed.
-    /// - Otherwise, if the value ends in * (like "-DFOO=*") then any argument
-    ///   with the prefix will be removed.
-    /// - Otherwise any argument exactly matching the value is removed.
-    ///
-    /// In all cases, -Xclang is also removed where needed.
-    ///
-    /// Example:
-    ///   Command: clang++ --include-directory=/usr/include -DFOO=42 foo.cc
-    ///   Remove: [-I, -DFOO=*]
-    ///   Result: clang++ foo.cc
-    ///
-    /// Flags added by the same CompileFlags entry will not be removed.
-    std::vector<Located<std::string>> Remove;
-
-    /// Directory to search for compilation database (compile_comands.json etc).
-    /// Valid values are:
-    /// - A single path to a directory (absolute, or relative to the fragment)
-    /// - Ancestors: search all parent directories (the default)
-    /// - None: do not use a compilation database, just default flags.
-    llvm::Optional<Located<std::string>> CompilationDatabase;
-  };
-  CompileFlagsBlock CompileFlags;
-
-  /// Controls how clangd understands code outside the current file.
-  /// clangd's indexes provide information about symbols that isn't available
-  /// to clang's parser, such as incoming references.
-  struct IndexBlock {
-    /// Whether files are built in the background to produce a project index.
-    /// This is checked for translation units only, not headers they include.
-    /// Legal values are "Build" or "Skip".
-    llvm::Optional<Located<std::string>> Background;
-    /// An external index uses data source outside of clangd itself. This is
-    /// usually prepared using clangd-indexer.
-    /// Exactly one source (File/Server) should be configured.
-    struct ExternalBlock {
-      /// Whether the block is explicitly set to `None`. Can be used to clear
-      /// any external index specified before.
-      Located<bool> IsNone = false;
-      /// Path to an index file generated by clangd-indexer. Relative paths may
-      /// be used, if config fragment is associated with a directory.
-      llvm::Optional<Located<std::string>> File;
-      /// Address and port number for a clangd-index-server. e.g.
-      /// `123.1.1.1:13337`.
-      llvm::Optional<Located<std::string>> Server;
-      /// Source root governed by this index. Default is the directory
-      /// associated with the config fragment. Absolute in case of user config
-      /// and relative otherwise. Should always use forward-slashes.
-      llvm::Optional<Located<std::string>> MountPoint;
-    };
-    llvm::Optional<Located<ExternalBlock>> External;
-  };
-  IndexBlock Index;
-
-  /// Controls behavior of diagnostics (errors and warnings).
-  struct DiagnosticsBlock {
-    /// Diagnostic codes that should be suppressed.
-    ///
-    /// Valid values are:
-    /// - *, to disable all diagnostics
-    /// - diagnostic codes exposed by clangd (e.g unknown_type, -Wunused-result)
-    /// - clang internal diagnostic codes (e.g. err_unknown_type)
-    /// - warning categories (e.g. unused-result)
-    /// - clang-tidy check names (e.g. bugprone-narrowing-conversions)
-    ///
-    /// This is a simple filter. Diagnostics can be controlled in other ways
-    /// (e.g. by disabling a clang-tidy check, or the -Wunused compile flag).
-    /// This often has other advantages, such as skipping some analysis.
-    std::vector<Located<std::string>> Suppress;
-
-    /// Controls how clangd will correct "unnecessary #include directives.
-    /// clangd can warn if a header is `#include`d but not used, and suggest
-    /// removing it.
-    //
-    /// Strict means a header is unused if it does not *directly* provide any
-    /// symbol used in the file. Removing it may still break compilation if it
-    /// transitively includes headers that are used. This should be fixed by
-    /// including those headers directly.
-    ///
-    /// Valid values are:
-    /// - Strict
-    /// - None
-    llvm::Optional<Located<std::string>> UnusedIncludes;
-
-    /// Controls how clang-tidy will run over the code base.
-    ///
-    /// The settings are merged with any settings found in .clang-tidy
-    /// configiration files with these ones taking precedence.
-    struct ClangTidyBlock {
-      std::vector<Located<std::string>> Add;
-      /// List of checks to disable.
-      /// Takes precedence over Add. To enable all llvm checks except include
-      /// order:
-      ///   Add: llvm-*
-      ///   Remove: llvm-include-onder
-      std::vector<Located<std::string>> Remove;
-
-      /// A Key-Value pair list of options to pass to clang-tidy checks
-      /// These take precedence over options specified in clang-tidy
-      /// configuration files. Example:
-      ///   CheckOptions:
-      ///     readability-braces-around-statements.ShortStatementLines: 2
-      std::vector<std::pair<Located<std::string>, Located<std::string>>>
-          CheckOptions;
-    };
-    ClangTidyBlock ClangTidy;
-  };
-  DiagnosticsBlock Diagnostics;
-
-  // Describes the style of the codebase, beyond formatting.
-  struct StyleBlock {
-    // Namespaces that should always be fully qualified, meaning no "using"
-    // declarations, always spell out the whole name (with or without leading
-    // ::). All nested namespaces are affected as well.
-    // Affects availability of the AddUsing tweak.
-    std::vector<Located<std::string>> FullyQualifiedNamespaces;
-  };
-  StyleBlock Style;
-
-  /// Describes code completion preferences.
-  struct CompletionBlock {
-    /// Whether code completion should include suggestions from scopes that are
-    /// not visible. The required scope prefix will be inserted.
-    llvm::Optional<Located<bool>> AllScopes;
-  };
-  CompletionBlock Completion;
-
-  /// Describes hover preferences.
-  struct HoverBlock {
-    /// Whether hover show a.k.a type.
-    llvm::Optional<Located<bool>> ShowAKA;
-  };
-  HoverBlock Hover;
+#include "ConfigFragment.gen.h"
 };
 
 } // namespace config
 } // namespace clangd
 } // namespace clang
 
 #endif

clang-tools-extra/clangd/ConfigYAML.cpp

 //===--- ConfigYAML.cpp - Loading configuration fragments from YAML files -===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 #include "ConfigFragment.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/ADT/SmallSet.h"
+#include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/YAMLParser.h"
 #include <string>
 #include <system_error>
 
 namespace clang {
class Parser {
   bool HadError = false;
 
 public:
   Parser(llvm::SourceMgr &SM) : SM(SM) {}
 
   // Tries to parse N into F, returning false if it failed and we couldn't
   // meaningfully recover (YAML syntax error, or hard semantic error).
   bool parse(Fragment &F, Node &N) {
-    DictParser Dict("Config", this);
-    Dict.handle("If", [&](Node &N) { parse(F.If, N); });
-    Dict.handle("CompileFlags", [&](Node &N) { parse(F.CompileFlags, N); });
-    Dict.handle("Index", [&](Node &N) { parse(F.Index, N); });
-    Dict.handle("Style", [&](Node &N) { parse(F.Style, N); });
-    Dict.handle("Diagnostics", [&](Node &N) { parse(F.Diagnostics, N); });
-    Dict.handle("Completion", [&](Node &N) { parse(F.Completion, N); });
-    Dict.handle("Hover", [&](Node &N) { parse(F.Hover, N); });
-    Dict.parse(N);
+    parse(F, N, "Config");
     return !(N.failed() || HadError);
   }
 
 private:
-  void parse(Fragment::IfBlock &F, Node &N) {
-    DictParser Dict("If", this);
-    Dict.unrecognized([&](Located<std::string>, Node &) {
-      F.HasUnrecognizedCondition = true;
-      return true; // Emit a warning for the unrecognized key.
-    });
-    Dict.handle("PathMatch", [&](Node &N) {
-      if (auto Values = scalarValues(N))
-        F.PathMatch = std::move(*Values);
-    });
-    Dict.handle("PathExclude", [&](Node &N) {
-      if (auto Values = scalarValues(N))
-        F.PathExclude = std::move(*Values);
-    });
-    Dict.parse(N);
+  class DictParser;
+#include "ConfigYAML.inc"
+
+  // Parse Optional<string>
+  void parse(llvm::Optional<std::string> &F, Node &N, llvm::StringRef Name) {
+    if (auto *S = llvm::dyn_cast<ScalarNode>(&N)) {
+      llvm::SmallString<256> Buf;
+      F = S->getValue(Buf).str();
+      return;
+    }
+    if (auto *BS = llvm::dyn_cast<BlockScalarNode>(&N)) {
+      F = BS->getValue().str();
+      return;
+    }
+    warning(Name + " should be scalar", N);
   }
 
-  void parse(Fragment::CompileFlagsBlock &F, Node &N) {
-    DictParser Dict("CompileFlags", this);
-    Dict.handle("Add", [&](Node &N) {
-      if (auto Values = scalarValues(N))
-        F.Add = std::move(*Values);
-    });
-    Dict.handle("Remove", [&](Node &N) {
-      if (auto Values = scalarValues(N))
-        F.Remove = std::move(*Values);
-    });
-    Dict.handle("CompilationDatabase", [&](Node &N) {
-      F.CompilationDatabase = scalarValue(N, "CompilationDatabase");
-    });
-    Dict.parse(N);
+  // Parse Optional<bool>
+  void parse(llvm::Optional<bool> &F, Node &N, llvm::StringRef Name) {
+    llvm::Optional<std::string> Scalar;
+    parse(Scalar, N, Name);
+    if (Scalar) {
+      if (auto Bool = llvm::yaml::parseBool(*Scalar)) {
+        F = *Bool;
+      } else {
+        warning(Name + " should be a boolean", N);
+      }
+    }
   }
 
-  void parse(Fragment::StyleBlock &F, Node &N) {
-    DictParser Dict("Style", this);
-    Dict.handle("FullyQualifiedNamespaces", [&](Node &N) {
-      if (auto Values = scalarValues(N))
-        F.FullyQualifiedNamespaces = std::move(*Values);
-    });
-    Dict.parse(N);
+  // Parse Located<T> if we can parse T.
+  template <typename T>
+  void parse(llvm::Optional<Located<T>> &F, Node &N, llvm::StringRef Name) {
+    llvm::Optional<T> Value;
+    parse(Value, N, Name);
+    if (Value)
+      F.emplace(std::move(*Value), N.getSourceRange());
   }
 
-  void parse(Fragment::DiagnosticsBlock &F, Node &N) {
-    DictParser Dict("Diagnostics", this);
-    Dict.handle("Suppress", [&](Node &N) {
-      if (auto Values = scalarValues(N))
-        F.Suppress = std::move(*Values);
-    });
-    Dict.handle("UnusedIncludes", [&](Node &N) {
-      F.UnusedIncludes = scalarValue(N, "UnusedIncludes");
-    });
-    Dict.handle("ClangTidy", [&](Node &N) { parse(F.ClangTidy, N); });
-    Dict.parse(N);
+  // Parse Optional<T> if we can parse T.
+  template <typename T>
+  void parse(llvm::Optional<T> &F, Node &N, llvm::StringRef Name) {
+    F.emplace();
+    parse(*F, N, Name);
   }
 
-  void parse(Fragment::DiagnosticsBlock::ClangTidyBlock &F, Node &N) {
-    DictParser Dict("ClangTidy", this);
-    Dict.handle("Add", [&](Node &N) {
-      if (auto Values = scalarValues(N))
-        F.Add = std::move(*Values);
-    });
-    Dict.handle("Remove", [&](Node &N) {
-      if (auto Values = scalarValues(N))
-        F.Remove = std::move(*Values);
-    });
-    Dict.handle("CheckOptions", [&](Node &N) {
-      DictParser CheckOptDict("CheckOptions", this);
-      CheckOptDict.unrecognized([&](Located<std::string> &&Key, Node &Val) {
-        if (auto Value = scalarValue(Val, *Key))
-          F.CheckOptions.emplace_back(std::move(Key), std::move(*Value));
-        return false; // Don't emit a warning
-      });
-      CheckOptDict.parse(N);
-    });
-    Dict.parse(N);
+  // Parse vector<T> if we can parse Optional<T>
+  template <typename T>
+  void parse(std::vector<T> &F, Node &N, llvm::StringRef Name) {
+    llvm::Optional<T> Element;
+    if (auto *S = llvm::dyn_cast<SequenceNode>(&N)) {
+      // We *must* consume all items, even on error, or the parser will assert.
+      std::string EltName = (Name + " element").str();
+      for (auto &Child : *S) {
+        Element.reset();
+        parse(Element, Child, EltName);
+        if (Element)
+          F.push_back(std::move(*Element));
+      }
+      return;
+    }
+
+    parse(Element, N, Name);
+    if (Element)
+      F.push_back(std::move(*Element));
+  }
+
+  // Parse vector<pair<Located<string>, T> if we can parse Optional<T>
+  template <typename T>
+  void parse(std::vector<std::pair<Located<std::string>, T>> &F, Node &N,
+             llvm::StringRef Name) {
+    DictParser CheckOptDict(Name, this);
+      llvm::Optional<T> Value;
+      parse(Value, Val, *Key);
+      if (Value)
+        F.emplace_back(std::move(Key), std::move(*Value));
   }
 
-  void parse(Fragment::IndexBlock &F, Node &N) {
-    DictParser Dict("Index", this);
-    Dict.handle("Background",
-                [&](Node &N) { F.Background = scalarValue(N, "Background"); });
-    Dict.handle("External", [&](Node &N) {
-      Fragment::IndexBlock::ExternalBlock External;
-      // External block can either be a mapping or a scalar value. Dispatch
-      // accordingly.
-      if (N.getType() == Node::NK_Mapping) {
-        parse(External, N);
-      } else if (N.getType() == Node::NK_Scalar ||
-                 N.getType() == Node::NK_BlockScalar) {
-        parse(External, scalarValue(N, "External").getValue());
-      } else {
-        error("External must be either a scalar or a mapping.", N);
-        return;
-      }
-      F.External.emplace(std::move(External));
-      F.External->Range = N.getSourceRange();
+  // By default, don't customize block parsing.
+  template <typename T> void parseCustom(T &, DictParser &) {}
+
+  // For the If block, record whether we see unrecognized conditions.
+  void parseCustom(Fragment::IfBlock &F, DictParser &Dict) {
+    Dict.unrecognized([&](Located<std::string>, Node &) {
+      F.HasUnrecognizedCondition = true;
+      return true; // Emit a warning for the unrecognized key.
     });
-    Dict.parse(N);
   }
 
-  void parse(Fragment::IndexBlock::ExternalBlock &F,
-             Located<std::string> ExternalVal) {
-    if (!llvm::StringRef(*ExternalVal).equals_insensitive("none")) {
-      error("Only scalar value supported for External is 'None'",
-            ExternalVal.Range);
+  // For the Index block, External can be a magic scalar "None".
+  void parseCustom(Fragment::IndexBlock &F, DictParser &Dict) {
+    Dict.handle("External", [&](Node &N) {
+      if (auto Scalar = peekScalar(N))
+        if (llvm::StringRef(*Scalar).equals_insensitive("none")) {
+          Fragment::IndexBlock::ExternalBlock External;
+          External.IsNone = {true, N.getSourceRange()};
+          F.External = {External, N.getSourceRange()};
           return;
         }
-    F.IsNone = true;
-    F.IsNone.Range = ExternalVal.Range;
-  }
 
-  void parse(Fragment::IndexBlock::ExternalBlock &F, Node &N) {
-    DictParser Dict("External", this);
-    Dict.handle("File", [&](Node &N) { F.File = scalarValue(N, "File"); });
-    Dict.handle("Server",
-                [&](Node &N) { F.Server = scalarValue(N, "Server"); });
-    Dict.handle("MountPoint",
-                [&](Node &N) { F.MountPoint = scalarValue(N, "MountPoint"); });
-    Dict.parse(N);
-  }
-
-  void parse(Fragment::CompletionBlock &F, Node &N) {
-    DictParser Dict("Completion", this);
-    Dict.handle("AllScopes", [&](Node &N) {
-      if (auto Value = scalarValue(N, "AllScopes")) {
-        if (auto AllScopes = llvm::yaml::parseBool(**Value))
-          F.AllScopes = *AllScopes;
-        else
-          warning("AllScopes should be a boolean", N);
-      }
+      parse(F.External, N, "External");
     });
-    Dict.parse(N);
   }
 
-  void parse(Fragment::HoverBlock &F, Node &N) {
-    DictParser Dict("Hover", this);
-    Dict.handle("ShowAKA", [&](Node &N) {
-      if (auto Value = scalarValue(N, "ShowAKA")) {
-        if (auto ShowAKA = llvm::yaml::parseBool(**Value))
-          F.ShowAKA = *ShowAKA;
-        else
-          warning("ShowAKA should be a boolean", N);
-      }
-    });
-    Dict.parse(N);
+  llvm::Optional<std::string> peekScalar(Node &N) {
+    llvm::Optional<std::string> Result;
+    if (N.getType() == Node::NK_Scalar || N.getType() == Node::NK_BlockScalar)
+      parse(Result, N, "");
+    return Result;
   }
 
   // Helper for parsing mapping nodes (dictionaries).
   // We don't use YamlIO as we want to control over unknown keys.
   class DictParser {
     llvm::StringRef Description;
     std::vector<std::pair<llvm::StringRef, std::function<void(Node &)>>> Keys;
     std::function<bool(Located<std::string>, Node &)> UnknownHandler;
void parse(Node &N) const {
       }
       llvm::SmallSet<std::string, 8> Seen;
       llvm::SmallVector<Located<std::string>, 0> UnknownKeys;
       // We *must* consume all items, even on error, or the parser will assert.
       for (auto &KV : llvm::cast<MappingNode>(N)) {
         auto *K = KV.getKey();
         if (!K) // YAMLParser emitted an error.
           continue;
-        auto Key = Outer->scalarValue(*K, "Dictionary key");
+        llvm::Optional<Located<std::string>> Key;
+        Outer->parse(Key, *K, "Dictionary key");
         if (!Key)
           continue;
         if (!Seen.insert(**Key).second) {
           Outer->warning("Duplicate key " + **Key + " is ignored", *K);
           if (auto *Value = KV.getValue())
             Value->skip();
           continue;
         }
void warnUnknownKeys(llvm::ArrayRef<Located<std::string>> UnknownKeys,
                          UnknownKey.Range);
         else
           Outer->warning("Unknown " + Description + " key '" + *UnknownKey +
                              "'",
                          UnknownKey.Range);
     }
   };
 
-  // Try to parse a single scalar value from the node, warn on failure.
-  llvm::Optional<Located<std::string>> scalarValue(Node &N,
-                                                   llvm::StringRef Desc) {
-    llvm::SmallString<256> Buf;
-    if (auto *S = llvm::dyn_cast<ScalarNode>(&N))
-      return Located<std::string>(S->getValue(Buf).str(), N.getSourceRange());
-    if (auto *BS = llvm::dyn_cast<BlockScalarNode>(&N))
-      return Located<std::string>(BS->getValue().str(), N.getSourceRange());
-    warning(Desc + " should be scalar", N);
-    return llvm::None;
-  }
-
-  // Try to parse a list of single scalar values, or just a single value.
-  llvm::Optional<std::vector<Located<std::string>>> scalarValues(Node &N) {
-    std::vector<Located<std::string>> Result;
-    if (auto *S = llvm::dyn_cast<ScalarNode>(&N)) {
-      llvm::SmallString<256> Buf;
-      Result.emplace_back(S->getValue(Buf).str(), N.getSourceRange());
-    } else if (auto *S = llvm::dyn_cast<BlockScalarNode>(&N)) {
-      Result.emplace_back(S->getValue().str(), N.getSourceRange());
-    } else if (auto *S = llvm::dyn_cast<SequenceNode>(&N)) {
-      // We *must* consume all items, even on error, or the parser will assert.
-      for (auto &Child : *S) {
-        if (auto Value = scalarValue(Child, "List item"))
-          Result.push_back(std::move(*Value));
-      }
-    } else {
-      warning("Expected scalar or list of scalars", N);
-      return llvm::None;
-    }
-    return Result;
-  }
-
   // Report a "hard" error, reflecting a config file that can never be valid.
   void error(const llvm::Twine &Msg, llvm::SMRange Range) {
     HadError = true;
     SM.PrintMessage(Range.Start, llvm::SourceMgr::DK_Error, Msg, Range);
   }
   void error(const llvm::Twine &Msg, const Node &N) {
     return error(Msg, N.getSourceRange());
   }

clang-tools-extra/clangd/unittests/ConfigYAMLTests.cpp

Index:
   auto Results =
       Fragment::parseYAML(YAML.code(), "config.yaml", Diags.callback());
   ASSERT_THAT(Diags.Diagnostics, IsEmpty());
   ASSERT_EQ(Results.size(), 1u);
   ASSERT_TRUE(Results[0].Index.External);
   EXPECT_FALSE(Results[0].Index.External.getValue()->File.hasValue());
   EXPECT_FALSE(Results[0].Index.External.getValue()->MountPoint.hasValue());
   EXPECT_FALSE(Results[0].Index.External.getValue()->Server.hasValue());
-  EXPECT_THAT(*Results[0].Index.External.getValue()->IsNone, testing::Eq(true));
+  EXPECT_THAT(**Results[0].Index.External.getValue()->IsNone,
+              testing::Eq(true));
 }
 
 TEST(ParseYAML, ExternalBlock) {
   CapturedDiags Diags;
   Annotations YAML(R"yaml(
 Index:
   External:
     File: "foo"
     Server: ^"bar"
     MountPoint: "baz"
   )yaml");
   auto Results =
       Fragment::parseYAML(YAML.code(), "config.yaml", Diags.callback());
   ASSERT_EQ(Results.size(), 1u);
   ASSERT_TRUE(Results[0].Index.External);
   EXPECT_THAT(*Results[0].Index.External.getValue()->File, Val("foo"));
   EXPECT_THAT(*Results[0].Index.External.getValue()->MountPoint, Val("baz"));
   ASSERT_THAT(Diags.Diagnostics, IsEmpty());
   EXPECT_THAT(*Results[0].Index.External.getValue()->Server, Val("bar"));
 }
 
-TEST(ParseYAML, AllScopes) {
-  CapturedDiags Diags;
-  Annotations YAML(R"yaml(
-Completion:
-  AllScopes: True
-  )yaml");
-  auto Results =
-      Fragment::parseYAML(YAML.code(), "config.yaml", Diags.callback());
-  ASSERT_THAT(Diags.Diagnostics, IsEmpty());
-  ASSERT_EQ(Results.size(), 1u);
-  EXPECT_THAT(Results[0].Completion.AllScopes, llvm::ValueIs(Val(true)));
-}
-
-TEST(ParseYAML, AllScopesWarn) {
-  CapturedDiags Diags;
-  Annotations YAML(R"yaml(
-Completion:
-  AllScopes: $diagrange[[Truex]]
-  )yaml");
-  auto Results =
-      Fragment::parseYAML(YAML.code(), "config.yaml", Diags.callback());
-  EXPECT_THAT(Diags.Diagnostics,
-              ElementsAre(AllOf(DiagMessage("AllScopes should be a boolean"),
-                                DiagKind(llvm::SourceMgr::DK_Warning),
-                                DiagPos(YAML.range("diagrange").start),
-                                DiagRange(YAML.range("diagrange")))));
-  ASSERT_EQ(Results.size(), 1u);
-  EXPECT_THAT(Results[0].Completion.AllScopes, testing::Eq(llvm::None));
-}
-
-TEST(ParseYAML, ShowAKA) {
-  CapturedDiags Diags;
-  Annotations YAML(R"yaml(
-Hover:
-  ShowAKA: True
-  )yaml");
-  auto Results =
-      Fragment::parseYAML(YAML.code(), "config.yaml", Diags.callback());
-  ASSERT_THAT(Diags.Diagnostics, IsEmpty());
-  ASSERT_EQ(Results.size(), 1u);
-  EXPECT_THAT(Results[0].Hover.ShowAKA, llvm::ValueIs(Val(true)));
-}
 } // namespace
 } // namespace config
 } // namespace clangd
 } // namespace clang

clang/utils/TableGen/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS Support)
 
 add_tablegen(clang-tblgen CLANG
   ASTTableGen.cpp
   ClangASTNodesEmitter.cpp
   ClangASTPropertiesEmitter.cpp
   ClangAttrEmitter.cpp
   ClangCommentCommandInfoEmitter.cpp
   ClangCommentHTMLNamedCharacterReferenceEmitter.cpp
   ClangCommentHTMLTagsEmitter.cpp
   ClangDataCollectorsEmitter.cpp
   ClangDiagnosticsEmitter.cpp
+  ClangdConfigEmitter.cpp
   ClangOpcodesEmitter.cpp
   ClangOpenCLBuiltinEmitter.cpp
   ClangOptionDocEmitter.cpp
   ClangSACheckersEmitter.cpp
   ClangSyntaxEmitter.cpp
   ClangTypeNodesEmitter.cpp
   MveEmitter.cpp
   NeonEmitter.cpp
   RISCVVEmitter.cpp
   SveEmitter.cpp
   TableGen.cpp
   )
 set_target_properties(clang-tblgen PROPERTIES FOLDER "Clang tablegenning")

clang/utils/TableGen/ClangdConfigEmitter.cpp

+//=- ClangDiagnosticsEmitter.cpp - Generate clangd tables ----------*- C++ -*-//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// These tablegen backends emit clangd config tables.
+//
+//===----------------------------------------------------------------------===//
+
+#include "TableGenBackends.h"
+#include "llvm/ADT/ScopeExit.h"
+#include "llvm/ADT/StringExtras.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/TableGen/Record.h"
+#include "llvm/TableGen/TableGenBackend.h"
+
+namespace {
+class Emitter {
+  auto indent() {
+    Indent += 2;
+    return llvm::make_scope_exit([this] { Indent -= 2; });
+  }
+  llvm::raw_ostream &line() {
+    OS << "\n";
+    OS.indent(Indent);
+    return OS;
+  }
+  auto sep() {
+    return [Sep(""), this]() mutable {
+      OS << Sep;
+      Sep = "\n";
+    };
+  }
+
+  static std::string quote(llvm::StringRef S) {
+    std::string Buf;
+    llvm::raw_string_ostream OS(Buf);
+    OS << '"';
+    llvm::printEscapedString(S, OS);
+    OS << '"';
+    OS.flush();
+    return Buf;
+  }
+
+  // Returns lines from a docstring:
+  //  - with leading and trailing whitespace-only lines removed
+  //  - with indentation stripped based on the first (non-empty) line
+  static llvm::SmallVector<llvm::StringRef> docLines(llvm::StringRef Doc) {
+    llvm::Optional<unsigned> Indent; // from first nonempty line
+    llvm::SmallVector<llvm::StringRef> Result;
+    for (const llvm::StringRef Line : llvm::split(Doc, "\n")) {
+      if (!Indent.hasValue()) {
+        llvm::StringRef Trimmed = Line.ltrim();
+        if (Trimmed.empty())
+          continue;
+        Indent = Line.size() - Trimmed.size();
+        Result.push_back(Trimmed);
+      } else {
+        assert(Line.take_front(*Indent).ltrim().empty() &&
+               "Later lines less indented than first!");
+        Result.push_back(Line.substr(*Indent));
+      }
+    }
+    while (!Result.empty() && Result.back().ltrim().empty())
+      Result.pop_back();
+    return Result;
+  }
+
+public:
+  Emitter(llvm::raw_ostream &OS, llvm::RecordKeeper &Records)
+      : OS(OS), Records(Records) {
+    llvm::SmallVector<llvm::StringRef> Qual;
+    fillBlockQualNames(*Fragment, Qual);
+  }
+
+  void emitFragment() { emitStructBody(*Fragment); }
+
+  void emitYAMLParser() {
+    auto _ = indent();
+    auto Sep = sep();
+    for (const llvm::Record *BT : Records.getAllDerivedDefinitions("Struct")) {
+      Sep();
+      emitParseBlock(*BT);
+    }
+  }
+
+  void emitDocs() { emitStructDocs(*Fragment, /*Depth=*/2); }
+
+private:
+  void emitStructBody(const llvm::Record &Type) {
+    assert(Type.hasDirectSuperClass(Struct));
+    auto _ = indent();
+    auto Sep = sep();
+    for (const auto *Field : Type.getValueAsListOfDefs("fields")) {
+      Sep();
+      emitField(*Field);
+    }
+  }
+
+  void emitField(const llvm::Record &Field) {
+    llvm::StringRef Doc = Field.getValueAsString("doc");
+    if (!Doc.empty()) {
+      for (llvm::StringRef Line : docLines(Doc))
+        line() << "// " << Line;
+    }
+
+    const llvm::Record &Type = *Field.getValueAsDef("type");
+    const llvm::Record &LeafType = leafType(Type);
+    if (LeafType.hasDirectSuperClass(Struct)) {
+      line() << "struct " << Type.getValueAsString("struct_name") << " {";
+      emitStructBody(LeafType);
+      line() << "};";
+    }
+
+    line() << Type.getValueAsString("field_spelling") << " "
+           << Field.getValueAsString("name") << ";";
+  }
+
+  void emitParseBlock(const llvm::Record &Type) {
+    line() << "void parse(" << llvm::join(BlockQualNames.lookup(&Type), "::")
+           << "& F, Node& N, llvm::StringRef Name) {";
+    {
+      auto _ = indent();
+      line() << "DictParser Dict(Name, this);";
+      line() << "parseCustom(F, Dict);";
+      for (const llvm::Record *Field : Type.getValueAsListOfDefs("fields")) {
+        if (!Field->getValueAsBit("parsed"))
+          continue;
+        llvm::StringRef FieldName = Field->getValueAsString("name");
+        line() << "Dict.handle(" << quote(FieldName) << ", [&](Node &N) { "
+               << "parse(F." << FieldName << ", N, " << quote(FieldName) << ");"
+               << " });";
+      }
+      line() << "Dict.parse(N);";
+    }
+    line() << "}";
+  }
+
+  void emitStructDocs(const llvm::Record &Type, unsigned Depth) {
+    for (const auto *Field : Type.getValueAsListOfDefs("fields")) {
+      if (!Field->getValueAsBit("documented"))
+        return;
+      emitFieldDocs(*Field, Depth);
+    }
+  }
+
+  void emitFieldDocs(const llvm::Record &Field, unsigned Depth) {
+    line() << std::string(Depth, '#') << " " << Field.getValueAsString("name");
+    line();
+
+    llvm::StringRef Doc = Field.getValueAsString("doc");
+    if (!Doc.empty()) {
+      for (llvm::StringRef Line : docLines(Doc))
+        line() << Line;
+      line();
+    }
+
+    const llvm::Record &Type = *Field.getValueAsDef("type");
+    const llvm::Record &LeafType = leafType(Type);
+    if (LeafType.hasDirectSuperClass(Struct))
+      emitStructDocs(LeafType, Depth + 1);
+  }
+
+  const llvm::Record &leafType(const llvm::Record &Type) {
+    if (Type.hasDirectSuperClass(List) || Type.hasDirectSuperClass(KeyValues))
+      return leafType(*Type.getValueAsDef("element"));
+    return Type;
+  }
+
+  void fillBlockQualNames(const llvm::Record &Type,
+                          llvm::SmallVector<llvm::StringRef> &Qual) {
+    Qual.push_back(Type.getValueAsString("struct_name"));
+    BlockQualNames.try_emplace(&Type, Qual);
+    for (const auto *Field : Type.getValueAsListOfDefs("fields")) {
+      const llvm::Record &LeafType = leafType(*Field->getValueAsDef("type"));
+      if (LeafType.hasDirectSuperClass(Struct))
+        fillBlockQualNames(LeafType, Qual);
+    }
+    Qual.pop_back();
+  }
+
+  llvm::DenseMap<const llvm::Record *, llvm::SmallVector<llvm::StringRef>>
+      BlockQualNames;
+  llvm::raw_ostream &OS;
+  llvm::RecordKeeper &Records;
+  const llvm::Record *Fragment = Records.getDef("Fragment"),
+                     *Struct = Records.getClass("Struct"),
+                     *List = Records.getClass("List"),
+                     *KeyValues = Records.getClass("KeyValues");
+  unsigned Indent = 0;
+};
+} // namespace
+
+void clang::EmitClangdConfigHeader(llvm::RecordKeeper &Records,
+                                   llvm::raw_ostream &OS) {
+  llvm::emitSourceFileHeader("Clangd config fragment definition", OS);
+
+  Emitter(OS, Records).emitFragment();
+}
+
+void clang::EmitClangdConfigYAML(llvm::RecordKeeper &Records,
+                                 llvm::raw_ostream &OS) {
+  llvm::emitSourceFileHeader("Clangd config fragment YAML parser", OS);
+
+  Emitter(OS, Records).emitYAMLParser();
+}
+
+void clang::EmitClangdConfigDocs(llvm::RecordKeeper &Records,
+                                 llvm::raw_ostream &OS) {
+  Emitter(OS, Records).emitDocs();
+}

clang/utils/TableGen/TableGen.cpp

enum ActionType {
   GenArmCdeBuiltinAliases,
   GenRISCVVectorHeader,
   GenRISCVVectorBuiltins,
   GenRISCVVectorBuiltinCG,
   GenAttrDocs,
   GenDiagDocs,
   GenOptDocs,
   GenDataCollectors,
-  GenTestPragmaAttributeSupportedAttributes
+  GenTestPragmaAttributeSupportedAttributes,
+  GenClangdConfigHeader,
+  GenClangdConfigYAML,
+  GenClangdConfigDocs,
 };
 
 namespace {
 cl::opt<ActionType> Action(
     cl::desc("Action to perform:"),
     cl::values(
         clEnumValN(PrintRecords, "print-records",
                    "Print all records to stdout (default)"),
cl::values(
         clEnumValN(GenDiagDocs, "gen-diag-docs",
                    "Generate diagnostic documentation"),
         clEnumValN(GenOptDocs, "gen-opt-docs", "Generate option documentation"),
         clEnumValN(GenDataCollectors, "gen-clang-data-collectors",
                    "Generate data collectors for AST nodes"),
         clEnumValN(GenTestPragmaAttributeSupportedAttributes,
                    "gen-clang-test-pragma-attribute-supported-attributes",
                    "Generate a list of attributes supported by #pragma clang "
-                   "attribute for testing purposes")));
+                   "attribute for testing purposes"),
+        clEnumValN(GenClangdConfigHeader, "gen-clangd-config-header",
+                   "Generate clangd Config.h struct"),
+        clEnumValN(GenClangdConfigYAML, "gen-clangd-config-yaml",
+                   "Generate clangd config YAML parser"),
+        clEnumValN(GenClangdConfigDocs, "gen-clangd-config-docs",
+                   "Generate clangd config documentation")));
 
 cl::opt<std::string>
 ClangComponent("clang-component",
                cl::desc("Only use warnings from specified component"),
                cl::value_desc("component"), cl::Hidden);
 
 bool ClangTableGenMain(raw_ostream &OS, RecordKeeper &Records) {
   switch (Action) {
case GenOptDocs:
     EmitClangOptDocs(Records, OS);
     break;
   case GenDataCollectors:
     EmitClangDataCollectors(Records, OS);
     break;
   case GenTestPragmaAttributeSupportedAttributes:
     EmitTestPragmaAttributeSupportedAttributes(Records, OS);
     break;
+  case GenClangdConfigHeader:
+    EmitClangdConfigHeader(Records, OS);
+    break;
+  case GenClangdConfigYAML:
+    EmitClangdConfigYAML(Records, OS);
+    break;
+  case GenClangdConfigDocs:
+    EmitClangdConfigDocs(Records, OS);
+    break;
   }
 
   return false;
 }
 }
 
 int main(int argc, char **argv) {
   sys::PrintStackTraceOnErrorSignal(argv[0]);

clang/utils/TableGen/TableGenBackends.h

void EmitClangOpenCLBuiltinTests(llvm::RecordKeeper &Records,
                                  llvm::raw_ostream &OS);
 
 void EmitClangDataCollectors(llvm::RecordKeeper &Records,
                              llvm::raw_ostream &OS);
 
 void EmitTestPragmaAttributeSupportedAttributes(llvm::RecordKeeper &Records,
                                                 llvm::raw_ostream &OS);
 
+void EmitClangdConfigHeader(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
+void EmitClangdConfigYAML(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
+void EmitClangdConfigDocs(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
+
 } // end namespace clang
 
 #endif