Add User docs for ASTImporter

[martong]

Summary

This document includes the description of the ASTImporter from the user/client
perspective.
A subsequent patch will describe the development internals.

[aprantl]

Thanks, the extra documentation is highly appreciated!

clang/docs/LibASTImporter.rst

19 ↗

(On Diff #212798)

Stylistic note: usually LLVM documentation is written avoiding "we". So for example:

In some cases it is preferable to work with more than one ``ASTContext``, for example when parsing multiple files inside one Clang-based tool.

33 ↗

(On Diff #212798)

Importing ... copies that node ...

[shafik]

Thank you for writing this up! I just have a few minor comments.

clang/docs/LibASTImporter.rst

110 ↗

(On Diff #212798)

Maybe helpful to link to the Matching the Clang AST and AST Matcher Reference

283 ↗

(On Diff #212798)

Note ODR is a C++ concept C does not have ODR

[gamesh411]

Lovely documentation with practical use-cases!
I left a few inline remarks.
Also wouldn't it be nice to have a section which introduces the -ast-merge command-line option. It would be helpful to see an example where a PCH is dumped and merged into another TU. You could mention, that this can be used to debug ASTImporter functionality.
Cheers!

clang/docs/LibASTImporter.rst

26 ↗

(On Diff #212798)

I'd prefer:
... translation unit (TU). This way, the analysis can ...

32 ↗

(On Diff #212798)

I'd put a comma here:
By importing one AST node, we copy

34 ↗

(On Diff #212798)

Leave the just, or rephrase:
Isn't it enough to insert ...

42 ↗

(On Diff #212798)

I'd put a comma here:
... existing definitions, and ...

74 ↗

(On Diff #212798)

I'd put a comma here:
... identifier name, and their ...

154 ↗

(On Diff #212798)

Leave the 'And'

190 ↗

(On Diff #212798)

I'd put a comma here:
... without definition, and we have ...

272 ↗

(On Diff #212798)

Leave the 'And'

283 ↗

(On Diff #212798)

I'd put a comma here:
If these definitions differ, then we ...

347 ↗

(On Diff #212798)

I'd use simple present here:
The AST will not contain the ... -> The AST does not contain the ...

388 ↗

(On Diff #212798)

Not a native, but this stood out:
In this case, we can see that an error is associated (`getImportDeclErrorIfAny`) with/to the specialization also, not just with/to the field.

402 ↗

(On Diff #212798)

I'd switch these:
set also -> also set

410 ↗

(On Diff #212798)

I'd phrase it like this:
We may recognize an error during the import of a dependent node. However, by that time, we have already created the dependant.

476 ↗

(On Diff #212798)

I'd put a comma here:
If we take a look at the AST, then we can ...

[martong]

Thanks for the review!

clang/docs/LibASTImporter.rst

19 ↗

(On Diff #212798)

In one hand, I accept that maybe the word "we" is overused in this document.
I'll try to rephrase those sentences where it feels awkward.

On the other hand, I feel if all occurrences of "we" were replaced then the document would become filled with passive voice. That would make it too formal and would provide harder reading in my opinion.
Also, other AST related documentation uses "we" quite frequently. E.g. https://clang.llvm.org/docs/LibASTMatchersTutorial.html has 42 matches of "we".

l have rephrased those sentences which you explicitly mentioned, please add comments to those sentences where you still feel the use of "we" inappropriate.

32 ↗

(On Diff #212798)

I changed to Adrian's suggestion.

110 ↗

(On Diff #212798)

Ok, I've added them to the beginning of the doc.

283 ↗

(On Diff #212798)

Changed to
"If these definitions differ, then we have a name conflict, in C++ it is known as ODR (one definition rule) violation."

410 ↗

(On Diff #212798)

I changed to use past perfect as well to emphasize that the node had been created before the error was recognized.
"We may recognize an error during the import of a dependent node. However, by that time, we had already created the dependant."

[martong]

• Address comments

[martong]

• Add description for -ast-merge

[a_sidorin]

That's incredible. Thank you!

cfe/trunk/docs/LibASTImporter.rst

215

We can avoid const_cast if we change the example function signature to const Node *.

258

CMakeLists.txt

557

let's (small letter).

[martong]

In D65573#1625881, @a_sidorin wrote:

That's incredible. Thank you!

Thanks Alexei for the review! I commited a fix for the typos.

cfe/trunk/docs/LibASTImporter.rst

215

Indeed... Still, I did not change that to keep consistency with other parts of the examples.

cfe/trunk/docs/LibASTImporter.rst

+===============================
+ASTImporter: Merging Clang ASTs
+===============================
+
+The ``ASTImporter`` class is part of Clang's core library, the AST library.
+It imports nodes of an ``ASTContext`` into another ``ASTContext``.
+
+In this document, we assume basic knowledge about the Clang AST.  See the :doc:`Introduction
+to the Clang AST <IntroductionToTheClangAST>` if you want to learn more
+about how the AST is structured.
+Knowledge about :doc:`matching the Clang AST <LibASTMatchers>` and the `reference for the matchers <https://clang.llvm.org/docs/LibASTMatchersReference.html>`_ are also useful.
+
+.. contents::
+   :local:
+
+Introduction
+------------
+
+``ASTContext`` holds long-lived AST nodes (such as types and decls) that can be referred to throughout the semantic analysis of a file.
+In some cases it is preferable to work with more than one ``ASTContext``.
+For example, we'd like to parse multiple different files inside the same Clang tool.
+It may be convenient if we could view the set of the resulting ASTs as if they were one AST resulting from the parsing of each file together.
+``ASTImporter`` provides the way to copy types or declarations from one ``ASTContext`` to another.
+We refer to the context from which we import as the **"from" context** or *source context*; and the context into which we import as the **"to" context** or *destination context*.
+
+Existing clients of the ``ASTImporter`` library are Cross Translation Unit (CTU) static analysis and the LLDB expression parser.
+CTU static analysis imports a definition of a function if its definition is found in another translation unit (TU).
+This way the analysis can breach out from the single TU limitation.
+LLDB's ``expr`` command parses a user-defined expression, creates an ``ASTContext`` for that and then imports the missing definitions from the AST what we got from the debug information (DWARF, etc).
+
+Algorithm of the import
+-----------------------
+
+Importing one AST node copies that node into the destination ``ASTContext``.
+Why do we have to copy the node?
+Isn't enough to insert the pointer to that node into the destination context?
+One reason is that the "from" context may outlive the "to" context.
+Also, the Clang AST consider nodes (or certain properties of nodes) equivalent if they have the same address!
+
+The import algorithm has to ensure that the structurally equivalent nodes in the different translation units are not getting duplicated in the merged AST.
+E.g. if we include the definition of the vector template (``#include <vector>``) in two translation units, then their merged AST should have only one node which represents the template.
+Also, we have to discover *one definition rule* (ODR) violations.
+For instance, if there is a class definition with the same name in both translation units, but one of the definition contains a different number of fields.
+So, we look up existing definitions, and then we check the structural equivalency on those nodes.
+The following pseudo-code demonstrates the basics of the import mechanism:
+
+.. code-block:: cpp
+
+  // Pseudo-code(!) of import:
+  ErrorOrDecl Import(Decl *FromD) {
+    Decl *ToDecl = nullptr;
+    FoundDeclsList = Look up all Decls in the "to" Ctx with the same name of FromD;
+    for (auto FoundDecl : FoundDeclsList) {
+      if (StructurallyEquivalentDecls(FoundDecl, FromD)) {
+        ToDecl = FoundDecl;
+        Mark FromD as imported;
+        break;
+      } else {
+        Report ODR violation;
+        return error;
+      }
+    }
+    if (FoundDeclsList is empty) {
+      Import dependent declarations and types of ToDecl;
+      ToDecl = create a new AST node in "to" Ctx;
+      Mark FromD as imported;
+    }
+    return ToDecl;
+  }
+
+Two AST nodes are *structurally equivalent* if they are
+
+- builtin types and refer to the same type, e.g. ``int`` and ``int`` are structurally equivalent,
+- function types and all their parameters have structurally equivalent types,
+- record types and all their fields in order of their definition have the same identifier names and structurally equivalent types,
+- variable or function declarations and they have the same identifier name and their types are structurally equivalent.
+
+We could extend the definition of structural equivalency to templates similarly.
+
+If A and B are AST nodes and *A depends on B*, then we say that A is a **dependant** of B and B is a **dependency** of A.
+The words "dependant" and "dependency" are nouns in British English.
+Unfortunately, in American English, the adjective "dependent" is used for both meanings.
+In this document, with the "dependent" adjective we always address the dependencies, the B node in the example.
+
+API
+---
+
+Let's create a tool which uses the ASTImporter class!
+First, we build two ASTs from virtual files; the content of the virtual files are synthesized from string literals:
+
+.. code-block:: cpp
+
+  std::unique_ptr<ASTUnit> ToUnit = buildASTFromCode(
+      "", "to.cc"); // empty file
+  std::unique_ptr<ASTUnit> FromUnit = buildASTFromCode(
+      R"(
+      class MyClass {
+        int m1;
+        int m2;
+      };
+      )",
+      "from.cc");
+
+The first AST corresponds to the destination ("to") context - which is empty - and the second for the source ("from") context.
+Next, we define a matcher to match ``MyClass`` in the "from" context:
+
+.. code-block:: cpp
+
+  auto Matcher = cxxRecordDecl(hasName("C"));
+  auto *From = getFirstDecl<CXXRecordDecl>(Matcher, FromUnit);
+
+Now we create the Importer and do the import:
+
+.. code-block:: cpp
+
+  ASTImporter Importer(ToUnit->getASTContext(), ToUnit->getFileManager(),
+                       FromUnit->getASTContext(), FromUnit->getFileManager(),
+                       /*MinimalImport=*/true);
+  llvm::Expected<Decl *> ImportedOrErr = Importer.Import(From);
+
+The ``Import`` call returns with ``llvm::Expected``, so, we must check for any error.
+Please refer to the `error handling <http://llvm.org/docs/ProgrammersManual.html#recoverable-errors>`_ documentation for details.
+
+.. code-block:: cpp
+
+  if (!ImportedOrErr) {
+    llvm::Error Err = ImportedOrErr.takeError();
+    llvm::errs() << "ERROR: " << Err << "\n";
+    consumeError(std::move(Err));
+    return 1;
+  }
+
+If there's no error then we can get the underlying value.
+In this example we will print the AST of the "to" context.
+
+.. code-block:: cpp
+
+  Decl *Imported = *ImportedOrErr;
+  Imported->getTranslationUnitDecl()->dump();
+
+Since we set **minimal import** in the constructor of the importer, the AST will not contain the declaration of the members (once we run the test tool).
+
+.. code-block:: bash
+
+  TranslationUnitDecl 0x68b9a8 <<invalid sloc>> <invalid sloc>
+  `-CXXRecordDecl 0x6c7e30 <line:2:7, col:13> col:13 class MyClass definition
+    `-DefinitionData pass_in_registers standard_layout trivially_copyable trivial literal
+      |-DefaultConstructor exists trivial needs_implicit
+      |-CopyConstructor simple trivial has_const_param needs_implicit implicit_has_const_param
+      |-MoveConstructor exists simple trivial needs_implicit
+      |-CopyAssignment trivial has_const_param needs_implicit implicit_has_const_param
+      |-MoveAssignment exists simple trivial needs_implicit
+      `-Destructor simple irrelevant trivial needs_implicit
+
+We'd like to get the members too, so, we use ``ImportDefinition`` to copy the whole definition of ``MyClass`` into the "to" context.
+Then we dump the AST again.
+
+.. code-block:: cpp
+
+  if (llvm::Error Err = Importer.ImportDefinition(From)) {
+    llvm::errs() << "ERROR: " << Err << "\n";
+    consumeError(std::move(Err));
+    return 1;
+  }
+  llvm::errs() << "Imported definition.\n";
+  Imported->getTranslationUnitDecl()->dump();
+
+This time the AST is going to contain the members too.
+
+.. code-block:: bash
+
+  TranslationUnitDecl 0x68b9a8 <<invalid sloc>> <invalid sloc>
+  `-CXXRecordDecl 0x6c7e30 <line:2:7, col:13> col:13 class MyClass definition
+    |-DefinitionData pass_in_registers standard_layout trivially_copyable trivial literal
+    | |-DefaultConstructor exists trivial needs_implicit
+    | |-CopyConstructor simple trivial has_const_param needs_implicit implicit_has_const_param
+    | |-MoveConstructor exists simple trivial needs_implicit
+    | |-CopyAssignment trivial has_const_param needs_implicit implicit_has_const_param
+    | |-MoveAssignment exists simple trivial needs_implicit
+    | `-Destructor simple irrelevant trivial needs_implicit
+    |-CXXRecordDecl 0x6c7f48 <col:7, col:13> col:13 implicit class MyClass
+    |-FieldDecl 0x6c7ff0 <line:3:9, col:13> col:13 m1 'int'
+    `-FieldDecl 0x6c8058 <line:4:9, col:13> col:13 m2 'int'
+
+We can spare the call for ``ImportDefinition`` if we set up the importer to do a "normal" (not minimal) import.
+
+.. code-block:: cpp
+
+  ASTImporter Importer( ....  /*MinimalImport=*/false);
+
+With **normal import**, all dependent declarations are imported normally.
+However, with minimal import, the dependent Decls are imported without definition, and we have to import their definition for each if we later need that.
+
+Putting this all together here is how the source of the tool looks like:
+
+.. code-block:: cpp
+
+  #include "clang/AST/ASTImporter.h"
+  #include "clang/ASTMatchers/ASTMatchFinder.h"
+  #include "clang/ASTMatchers/ASTMatchers.h"
+  #include "clang/Tooling/Tooling.h"
+
+  using namespace clang;
+  using namespace tooling;
+  using namespace ast_matchers;
+
+  template <typename Node, typename Matcher>
+  Node *getFirstDecl(Matcher M, const std::unique_ptr<ASTUnit> &Unit) {
+    auto MB = M.bind("bindStr"); // Bind the to-be-matched node to a string key.
+    auto MatchRes = match(MB, Unit->getASTContext());
+    // We should have at least one match.
+    assert(MatchRes.size() >= 1);
+    // Get the first matched and bound node.
+    Node *Result =
+        const_cast<Node *>(MatchRes[0].template getNodeAs<Node>("bindStr"));
+    assert(Result);
+    return Result;
+  }
+
+  int main() {
+    std::unique_ptr<ASTUnit> ToUnit = buildASTFromCode(
+        "", "to.cc");
+    std::unique_ptr<ASTUnit> FromUnit = buildASTFromCode(
+        R"(
+        class MyClass {
+          int m1;
+          int m2;
+        };
+        )",
+        "from.cc");
+    auto Matcher = cxxRecordDecl(hasName("MyClass"));
+    auto *From = getFirstDecl<CXXRecordDecl>(Matcher, FromUnit);
+
+    ASTImporter Importer(ToUnit->getASTContext(), ToUnit->getFileManager(),
+                         FromUnit->getASTContext(), FromUnit->getFileManager(),
+                         /*MinimalImport=*/true);
+    llvm::Expected<Decl *> ImportedOrErr = Importer.Import(From);
+    if (!ImportedOrErr) {
+      llvm::Error Err = ImportedOrErr.takeError();
+      llvm::errs() << "ERROR: " << Err << "\n";
+      consumeError(std::move(Err));
+      return 1;
+    }
+    Decl *Imported = *ImportedOrErr;
+    Imported->getTranslationUnitDecl()->dump();
+
+    if (llvm::Error Err = Importer.ImportDefinition(From)) {
+      llvm::errs() << "ERROR: " << Err << "\n";
+      consumeError(std::move(Err));
+      return 1;
+    }
+    llvm::errs() << "Imported definition.\n";
+    Imported->getTranslationUnitDecl()->dump();
+
+    return 0;
+  };
+
+We may extend the ``CmakeLists.txt`` under let's say ``clang/tools`` with the build and link instructions:
+
+.. code-block:: bash
+
+  add_clang_executable(astimporter-demo ASTImporterDemo.cpp)
+  clang_target_link_libraries(astimporter-demo
+    PRIVATE
+    LLVMSupport
+    clangAST
+    clangASTMatchers
+    clangBasic
+    clangFrontend
+    clangSerialization
+    clangTooling
+    )
+
+Then we can build and execute the new tool.
+
+.. code-block:: bash
+
+  $ ninja astimporter-demo && ./bin/astimporter-demo
+
+Errors during the import process
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Normally, either the source or the destination context contains the definition of a declaration.
+However, there may be cases when both of the contexts have a definition for a given symbol.
+If these definitions differ, then we have a name conflict, in C++ it is known as ODR (one definition rule) violation.
+Let's modify the previous tool we had written and try to import a ``ClassTemplateSpecializationDecl`` with a conflicting definition:
+
+.. code-block:: cpp
+
+  int main() {
+    std::unique_ptr<ASTUnit> ToUnit = buildASTFromCode(
+        R"(
+        // primary template
+        template <typename T>
+        struct X {};
+        // explicit specialization
+        template<>
+        struct X<int> { int i; };
+        )",
+        "to.cc");
+    ToUnit->enableSourceFileDiagnostics();
+    std::unique_ptr<ASTUnit> FromUnit = buildASTFromCode(
+        R"(
+        // primary template
+        template <typename T>
+        struct X {};
+        // explicit specialization
+        template<>
+        struct X<int> { int i2; };
+        // field mismatch:  ^^
+        )",
+        "from.cc");
+    FromUnit->enableSourceFileDiagnostics();
+    auto Matcher = classTemplateSpecializationDecl(hasName("X"));
+    auto *From = getFirstDecl<ClassTemplateSpecializationDecl>(Matcher, FromUnit);
+    auto *To = getFirstDecl<ClassTemplateSpecializationDecl>(Matcher, ToUnit);
+
+    ASTImporter Importer(ToUnit->getASTContext(), ToUnit->getFileManager(),
+                         FromUnit->getASTContext(), FromUnit->getFileManager(),
+                         /*MinimalImport=*/false);
+    llvm::Expected<Decl *> ImportedOrErr = Importer.Import(From);
+    if (!ImportedOrErr) {
+      llvm::Error Err = ImportedOrErr.takeError();
+      llvm::errs() << "ERROR: " << Err << "\n";
+      consumeError(std::move(Err));
+      To->getTranslationUnitDecl()->dump();
+      return 1;
+    }
+    return 0;
+  };
+
+When we run the tool we have the following warning:
+
+.. code-block:: bash
+
+  to.cc:7:14: warning: type 'X<int>' has incompatible definitions in different translation units [-Wodr]
+        struct X<int> { int i; };
+               ^
+  to.cc:7:27: note: field has name 'i' here
+        struct X<int> { int i; };
+                            ^
+  from.cc:7:27: note: field has name 'i2' here
+        struct X<int> { int i2; };
+                          ^
+
+Note, because of these diagnostics we had to call ``enableSourceFileDiagnostics`` on the ``ASTUnit`` objects.
+
+Since we could not import the specified declaration (``From``), we get an error in the return value.
+The AST does not contain the conflicting definition, so we are left with the original AST.
+
+.. code-block:: bash
+
+  ERROR: NameConflict
+  TranslationUnitDecl 0xe54a48 <<invalid sloc>> <invalid sloc>
+  |-ClassTemplateDecl 0xe91020 <to.cc:3:7, line:4:17> col:14 X
+  | |-TemplateTypeParmDecl 0xe90ed0 <line:3:17, col:26> col:26 typename depth 0 index 0 T
+  | |-CXXRecordDecl 0xe90f90 <line:4:7, col:17> col:14 struct X definition
+  | | |-DefinitionData empty aggregate standard_layout trivially_copyable pod trivial literal has_constexpr_non_copy_move_ctor can_const_default_init
+  | | | |-DefaultConstructor exists trivial constexpr needs_implicit defaulted_is_constexpr
+  | | | |-CopyConstructor simple trivial has_const_param needs_implicit implicit_has_const_param
+  | | | |-MoveConstructor exists simple trivial needs_implicit
+  | | | |-CopyAssignment trivial has_const_param needs_implicit implicit_has_const_param
+  | | | |-MoveAssignment exists simple trivial needs_implicit
+  | | | `-Destructor simple irrelevant trivial needs_implicit
+  | | `-CXXRecordDecl 0xe91270 <col:7, col:14> col:14 implicit struct X
+  | `-ClassTemplateSpecialization 0xe91340 'X'
+  `-ClassTemplateSpecializationDecl 0xe91340 <line:6:7, line:7:30> col:14 struct X definition
+    |-DefinitionData pass_in_registers aggregate standard_layout trivially_copyable pod trivial literal
+    | |-DefaultConstructor exists trivial needs_implicit
+    | |-CopyConstructor simple trivial has_const_param needs_implicit implicit_has_const_param
+    | |-MoveConstructor exists simple trivial needs_implicit
+    | |-CopyAssignment trivial has_const_param needs_implicit implicit_has_const_param
+    | |-MoveAssignment exists simple trivial needs_implicit
+    | `-Destructor simple irrelevant trivial needs_implicit
+    |-TemplateArgument type 'int'
+    |-CXXRecordDecl 0xe91558 <col:7, col:14> col:14 implicit struct X
+    `-FieldDecl 0xe91600 <col:23, col:27> col:27 i 'int'
+
+Error propagation
+"""""""""""""""""
+
+If there is a dependent node we have to import before we could import a given node then the import error associated to the dependency propagates to the dependant node.
+Let's modify the previous example and import a ``FieldDecl`` instead of the ``ClassTemplateSpecializationDecl``.
+
+.. code-block:: cpp
+
+  auto Matcher = fieldDecl(hasName("i2"));
+  auto *From = getFirstDecl<FieldDecl>(Matcher, FromUnit);
+
+In this case we can see that an error is associated (``getImportDeclErrorIfAny``) to the specialization also, not just to the field:
+
+.. code-block:: cpp
+
+  llvm::Expected<Decl *> ImportedOrErr = Importer.Import(From);
+  if (!ImportedOrErr) {
+    llvm::Error Err = ImportedOrErr.takeError();
+    consumeError(std::move(Err));
+
+    // check that the ClassTemplateSpecializationDecl is also marked as
+    // erroneous.
+    auto *FromSpec = getFirstDecl<ClassTemplateSpecializationDecl>(
+        classTemplateSpecializationDecl(hasName("X")), FromUnit);
+    assert(Importer.getImportDeclErrorIfAny(FromSpec));
+    // Btw, the error is also set for the FieldDecl.
+    assert(Importer.getImportDeclErrorIfAny(From));
+    return 1;
+  }
+
+Polluted AST
+""""""""""""
+
+We may recognize an error during the import of a dependent node. However, by that time, we had already created the dependant.
+In these cases we do not remove the existing erroneous node from the "to" context, rather we associate an error to that node.
+Let's extend the previous example with another class ``Y``.
+This class has a forward definition in the "to" context, but its definition is in the "from" context.
+We'd like to import the definition, but it contains a member whose type conflicts with the type in the "to" context:
+
+.. code-block:: cpp
+
+  std::unique_ptr<ASTUnit> ToUnit = buildASTFromCode(
+      R"(
+      // primary template
+      template <typename T>
+      struct X {};
+      // explicit specialization
+      template<>
+      struct X<int> { int i; };
+
+      class Y;
+      )",
+      "to.cc");
+  ToUnit->enableSourceFileDiagnostics();
+  std::unique_ptr<ASTUnit> FromUnit = buildASTFromCode(
+      R"(
+      // primary template
+      template <typename T>
+      struct X {};
+      // explicit specialization
+      template<>
+      struct X<int> { int i2; };
+      // field mismatch:  ^^
+
+      class Y { void f() { X<int> xi; } };
+      )",
+      "from.cc");
+  FromUnit->enableSourceFileDiagnostics();
+  auto Matcher = cxxRecordDecl(hasName("Y"));
+  auto *From = getFirstDecl<CXXRecordDecl>(Matcher, FromUnit);
+  auto *To = getFirstDecl<CXXRecordDecl>(Matcher, ToUnit);
+
+This time we create a shared_ptr for ``ASTImporterSharedState`` which owns the associated errors for the "to" context.
+Note, there may be several different ASTImporter objects which import into the same "to" context but from different "from" contexts; they should share the same ``ASTImporterSharedState``.
+(Also note, we have to include the corresponding ``ASTImporterSharedState.h`` header file.)
+
+.. code-block:: cpp
+
+  auto ImporterState = std::make_shared<ASTImporterSharedState>();
+  ASTImporter Importer(ToUnit->getASTContext(), ToUnit->getFileManager(),
+                       FromUnit->getASTContext(), FromUnit->getFileManager(),
+                       /*MinimalImport=*/false, ImporterState);
+  llvm::Expected<Decl *> ImportedOrErr = Importer.Import(From);
+  if (!ImportedOrErr) {
+    llvm::Error Err = ImportedOrErr.takeError();
+    consumeError(std::move(Err));
+
+    // ... but the node had been created.
+    auto *ToYDef = getFirstDecl<CXXRecordDecl>(
+        cxxRecordDecl(hasName("Y"), isDefinition()), ToUnit);
+    ToYDef->dump();
+    // An error is set for "ToYDef" in the shared state.
+    Optional<ImportError> OptErr =
+        ImporterState->getImportDeclErrorIfAny(ToYDef);
+    assert(OptErr);
+
+    return 1;
+  }
+
+If we take a look at the AST, then we can see that the Decl with the definition is created, but the field is missing.
+
+.. code-block:: bash
+
+  |-CXXRecordDecl 0xf66678 <line:9:7, col:13> col:13 class Y
+  `-CXXRecordDecl 0xf66730 prev 0xf66678 <:10:7, col:13> col:13 class Y definition
+    |-DefinitionData pass_in_registers empty aggregate standard_layout trivially_copyable pod trivial literal has_constexpr_non_copy_move_ctor can_const_default_init
+    | |-DefaultConstructor exists trivial constexpr needs_implicit defaulted_is_constexpr
+    | |-CopyConstructor simple trivial has_const_param needs_implicit implicit_has_const_param
+    | |-MoveConstructor exists simple trivial needs_implicit
+    | |-CopyAssignment trivial has_const_param needs_implicit implicit_has_const_param
+    | |-MoveAssignment exists simple trivial needs_implicit
+    | `-Destructor simple irrelevant trivial needs_implicit
+    `-CXXRecordDecl 0xf66828 <col:7, col:13> col:13 implicit class Y
+
+We do not remove the erroneous nodes because by the time when we recognize the error it is too late to remove the node, there may be additional references to that already in the AST.
+This is aligned with the overall `design principle of the Clang AST <InternalsManual.html#immutability>`_: Clang AST nodes (types, declarations, statements, expressions, and so on) are generally designed to be **immutable once created**.
+Thus, clients of the ASTImporter library should always check if there is any associated error for the node which they inspect in the destination context.
+We recommend skipping the processing of those nodes which have an error associated with them.
+
+Using the ``-ast-merge`` Clang front-end action
+-----------------------------------------------
+
+The ``-ast-merge <pch-file>`` command-line switch can be used to merge from the given serialized AST file.
+This file represents the source context.
+When this switch is present then each top-level AST node of the source context is being merged into the destination context.
+If the merge was successful then ``ASTConsumer::HandleTopLevelDecl`` is called for the Decl.
+This results that we can execute the original front-end action on the extended AST.
+
+Example for C
+^^^^^^^^^^^^^
+
+Let's consider the following three files:
+
+.. code-block:: c
+
+  // bar.h
+  #ifndef BAR_H
+  #define BAR_H
+  int bar();
+  #endif /* BAR_H */
+
+  // bar.c
+  #include "bar.h"
+  int bar() {
+    return 41;
+  }
+
+  // main.c
+  #include "bar.h"
+  int main() {
+      return bar();
+  }
+
+Let's generate the AST files for the two source files:
+
+.. code-block:: bash
+
+  $ clang -cc1 -emit-pch -o bar.ast bar.c
+  $ clang -cc1 -emit-pch -o main.ast main.c
+
+Then, let's check how the merged AST would look like if we consider only the ``bar()`` function:
+
+.. code-block:: bash
+
+  $ clang -cc1 -ast-merge bar.ast -ast-merge main.ast /dev/null -ast-dump
+  TranslationUnitDecl 0x12b0738 <<invalid sloc>> <invalid sloc>
+  |-FunctionDecl 0x12b1470 </path/bar.h:4:1, col:9> col:5 used bar 'int ()'
+  |-FunctionDecl 0x12b1538 prev 0x12b1470 </path/bar.c:3:1, line:5:1> line:3:5 used bar 'int ()'
+  | `-CompoundStmt 0x12b1608 <col:11, line:5:1>
+  |   `-ReturnStmt 0x12b15f8 <line:4:3, col:10>
+  |     `-IntegerLiteral 0x12b15d8 <col:10> 'int' 41
+  |-FunctionDecl 0x12b1648 prev 0x12b1538 </path/bar.h:4:1, col:9> col:5 used bar 'int ()'
+
+We can inspect that the prototype of the function and the definition of it is merged into the same redeclaration chain.
+What's more there is a third prototype declaration merged to the chain.
+The functions are merged in a way that prototypes are added to the redecl chain if they refer to the same type, but we can have only one definition.
+The first two declarations are from ``bar.ast``, the third is from ``main.ast``.
+
+Now, Let's create an object file from the merged AST:
+
+.. code-block:: bash
+
+  $ clang -cc1 -ast-merge bar.ast -ast-merge main.ast /dev/null -emit-obj -o main.o
+
+Next, we may call the linker and execute the created binary file.
+
+.. code-block:: bash
+
+  $ clang -o a.out main.o
+  $ ./a.out
+  $ echo $?
+  41
+  $
+
+Example for C++
+^^^^^^^^^^^^^^^
+
+In the case of C++, the generation of the AST files and the way how we invoke the front-end is a bit different.
+Assuming we have these three files:
+
+.. code-block:: cpp
+
+  // foo.h
+  #ifndef FOO_H
+  #define FOO_H
+  struct foo {
+      virtual int fun();
+  };
+  #endif /* FOO_H */
+
+  // foo.cpp
+  #include "foo.h"
+  int foo::fun() {
+    return 42;
+  }
+
+  // main.cpp
+  #include "foo.h"
+  int main() {
+      return foo().fun();
+  }
+
+We shall generate the AST files, merge them, create the executable and then run it:
+
+.. code-block:: bash
+
+  $ clang++ -x c++-header -o foo.ast foo.cpp
+  $ clang++ -x c++-header -o main.ast main.cpp
+  $ clang++ -cc1 -x c++ -ast-merge foo.ast -ast-merge main.ast /dev/null -ast-dump
+  $ clang++ -cc1 -x c++ -ast-merge foo.ast -ast-merge main.ast /dev/null -emit-obj -o main.o
+  $ clang++ -o a.out main.o
+  $ ./a.out
+  $ echo $?
+  42
+  $

cfe/trunk/docs/index.rst

.. toctree::
    ExternalClangExamples
    IntroductionToTheClangAST
    LibTooling
    LibFormat
    ClangPlugins
    RAVFrontendAction
    LibASTMatchersTutorial
    LibASTMatchers
+   LibASTImporter
    HowToSetupToolingForLLVM
    JSONCompilationDatabase
    RefactoringEngine
 
 Using Clang Tools
 =================
 
 .. toctree::