[docs] Add "C++20 Modules"

[ChuanqiXu]

Summary

We get some C++20 module things done in clang15.x. But we lack a user documentation for it. The implementation of c++20 modules share a big part of codes with clang modules. But they have very different semantics and user interfaces, so I think it is necessary to add a document for C++20 modules. Previously, there were also some people ask the document for C++20 Modules and I couldn't offer that time.

I wish we could land this before September and I can backport this to 15.x. This implies that even we solve any bug during the reviewing process. We shouldn't edit the document in time. We should land it and edit it.

Tested with grammarly and make docs-clang-html.

[ChuanqiXu]

clang/docs/CPlusPlus20Modules.rst

378–394

I'm not sure if this could be rendered properly, if there is any suggestion?

586–588

@iains maybe we need to double check on this. I also filed an issue: https://github.com/llvm/llvm-project/issues/56899. My reading for [cpp.import]p5 is the point of undefinition is also a macro definition, so we should emit it. But I maybe wrong.

If I am wrong, I wish we could get an example to show differences (If any)

[ChuanqiXu]

clang/docs/CPlusPlus20Modules.rst

469–470

I am not sure if we should remain this. I feel both (remain or not remain) is OK.

[ChuanqiXu]

Remove the discussion about the difference about macro in clang module and header units since we need more discussion about it. And the discussion is not necessary about the docs.

[Mordante]

Thanks a lot for working on this!

I wonder whether it would be better to have some of the more technical information in a separate file and let this file mainly have an introduction to modules in general and how to get started using them in Clang.

I haven't tested the steps yet, but I intend to see whether the examples work for me.

I haven't validated whether the module explanation is correct; I'm sure there are other reviewers who have a deeper knowledge of modules.

clang/docs/CPlusPlus20Modules.rst

12

Clang is capitalized in its documentation.

19

Is Modules_ intended to be a link?

21

23

IMO we don't need to justify why we add a new page. WDYT?

36

How about mentioning the state of modules in Clang? In your GitHub repository you mention some limitations. (A link to a status page could be an alternative.)

57

80

Did you render the document? I would have expected double backticks.

118

Maybe make this hello world example a bit simpler by not using partitions and have a separate example with partitions. This example gives the impression a simple "hello modular world" is a lot harder to achieve than a normal "hello world".

134

Does import <iostream>; work? If not is that a Clang or libc++ limitation?

165

Maybe explain what all steps do.

179

184

I assume -std=c++2b works too. Maybe rephrase the wording like this.

185

I would remove this sentence, it's not too common to backport language features.

322

540

Maybe provide a link to this module map. (FYI the libc++ module map is generated by CMake using a .in file.)

[ChuanqiXu]

• Address comments.
• Tested with make docs-clang-html.
• Add Known Problems section.

clang/docs/CPlusPlus20Modules.rst

19

Yes, it is linked to the clang modules page: https://clang.llvm.org/docs/Modules.html. This mimics the style of https://clang.llvm.org/docs/Modules.html#where-to-learn-more-about-modules

23

I guess it is not bad to add it. Since it is really confusing about clang modules and C++20 Modules at first for people who are not so familiar to these topics.

80

Yeah, I had rendered it in https://overbits.herokuapp.com/rsteditor/. But it is not good. I found http://rst.aaroniles.net which looks better.

134

Yes, import <iostream> works here. But I don't want to introduce too many information here. And I mentioned it indirectly later in Include translation section.

165

I made a simple explanation in the # part and I intended to explain them in more detail in the following sections. I feel it may be confusing for readers who reads at the first time?

184

I sent a backport patch to 15.x: https://github.com/llvm/llvm-project/issues/56803. So it -std=c++2b should be fine in 15.x

378–394

Now I've tested it with make docs-clang-html, and it looks not bad.

[ChuanqiXu]

In D131062#3697193, @Mordante wrote:

Thanks a lot for working on this!

I wonder whether it would be better to have some of the more technical information in a separate file and let this file mainly have an introduction to modules in general and how to get started using them in Clang.

What do you mean about the more technical information? Do you refer to the section How module speed up the compilation?

I added the How module speed up the compilation section since I know people are curious about why and how modules could speedup the compilation. And this section tries tell them that they may probably get what they want in O0 but not so much in O2 (and still some improvements after all) and also the reasons.

let this file mainly have an introduction to modules in general

If I don't misread, do you say that we need make this one like a modules tutorial? If yes, I think it is not what I thought. I treat this one like a toolchain documentation instead of a language feature documentation. Everything in the doc is about the building.
I put the language background since I want to make the doc self-contained. And everything else is related to the building. I understand it is good/needed to introduce about modules since it really brings many new concepts. Personally, I feel like it would be done by somebody else besides of clang docs. The compiler and languages are deeply connected but they are still different to me.

I haven't tested the steps yet, but I intend to see whether the examples work for me.

I haven't validated whether the module explanation is correct; I'm sure there are other reviewers who have a deeper knowledge of modules.

Thanks. I've tested all the examples. It is always good to have more testers. It should be good at trunk. If you want to test with 15.x, you may need to wait for the merge of https://github.com/llvm/llvm-project-release-prs/pull/40 and https://github.com/llvm/llvm-project-release-prs/pull/40

clang/docs/CPlusPlus20Modules.rst

36

Yeah, I've added the Known Problems section.

[h-vetinari]

Aside from a couple typos, I was wondering if it would be welcome to do a pass w.r.t stylistic improvements (e.g. "Modules have a lot of meanings." --> "The term 'modules' has a lot of meanings"). I don't want to be too nit-picky - the sentences are all understandable, but sometimes slightly unusual to a native speaker in their formulation.

[ChuanqiXu]

In D131062#3699341, @h-vetinari wrote:

Aside from a couple typos, I was wondering if it would be welcome to do a pass w.r.t stylistic improvements (e.g. "Modules have a lot of meanings." --> "The term 'modules' has a lot of meanings"). I don't want to be too nit-picky - the sentences are all understandable, but sometimes slightly unusual to a native speaker in their formulation.

It would be greatly welcome for such comments!

[h-vetinari]

It would be greatly welcome for such comments!

OK, here goes. Sorry for the large volume of comments. In addition to typos and stylistic improvements, I've had a few questions where the content wasn't clear to me (but note I'm not experienced with modules at all, so this may be obvious to others). I've also added a few line breaks where Phab was awkwardly overflowing. The position of the linebreaks is obviously irrelevant, but I thought it might help further review.

clang/docs/CPlusPlus20Modules.rst

11

13–14

15–16

Not 100% what the intention of the last sentence is - I presume it sets the goal for this document?

20–21

Due to the C++20 modules having very different semantics, it might be more friendly for users who care about C++20 modules only to create a new page.

Isn't "C++20 modules" what this page intends to do? Which new page are we talking about then?

24–26

31–35

49–51

64–66

The dot `.` in the name has no special meaning.

Not sure if this is intended to say that the dot in the regex is not needed, or that it has no semantic significance.

Also, when wanting to match a literal "." in a regex, I'd consider it beneficial for clarity to escape it ("\."), even though it does what's intended in the context of [].

78–80

94–98

This seems quite important for the terminology of the rest of the document, so I'd structure it in a way that stand out visually, e.g. the suggestion above.

147

Missing space & inconsistent spelling compared to above.

I'd personally write it as (including the quotes): "hello world" example

189

213–214

219

224–226

As a reader, I have no idea how to parse

must end with `.cppm (or .ccm, .cxxm`, etc)

On the one hand there's a clear restriction ("must"), on the other there's an open-ended list, whose contents are not clear to me.

228–230

Similarly here; can we describe the origin or the restrictions should/must - i.e. do they come from the standard or from the Clang implementation? Is there any enforcement, or do things just error otherwise?

235–236

238

Is this equivalent to "-fprebuilt-module-interface". Are there relevant differences, and if so, should the reader of this document know about them?

238

Moving this up from further down, but I might be getting things wrong.

240–241

247

I didn't understand what would be more convenient at first, but I'm guessing that one takes a directory and the other takes a file? I made a suggestion further up how to reflect this.

252–253

261–262

296

304

309

343

364

366–367

371–372

374–375

400–403

426–428

"It is not acceptable" -> "It would be very unfortunate"

Since it sounds like that's what will happen when switching on optimization?

449–450

458–459

464–466

473–478

498

506

511–513

523–528

564

574–575

641–644

646–647

649–651

[ChuanqiXu]

In D131062#3704017, @h-vetinari wrote:

It would be greatly welcome for such comments!

OK, here goes. Sorry for the large volume of comments. In addition to typos and stylistic improvements, I've had a few questions where the content wasn't clear to me (but note I'm not experienced with modules at all, so this may be obvious to others). I've also added a few line breaks where Phab was awkwardly overflowing. The position of the linebreaks is obviously irrelevant, but I thought it might help further review.

Many thanks for the though review! It is very helpful.

I've had a few questions where the content wasn't clear to me (but note I'm not experienced with modules at all, so this may be obvious to others)

This is welcome. In fact, it is a failure if the document requires the reader to have experienced with modules.

clang/docs/CPlusPlus20Modules.rst

15–16

The intention was to describe the motivation of the document, but we have stated it again in the next paragraph. So the suggestion looks fine.

20–21

The new page here refers to the document. The intention here is that the users may be confused about clang modules and c++20 modules and there is already a document about clang modules. So I am wondering it might be helpful to explain why we need a new document for C++20 Modules instead of clang modules.

64–66

Not sure if this is intended to say that the dot in the regex is not needed, or that it has no semantic significance.

I mean that it has no semantic significance. For example, the user from python may feel like the dot . may be related to hierarchy. But it is not the case for C++.

224–226

What I want to say is for etc is usual c++ source file suffix + m. Currently, the list is .cppm, .ccm, .cxxm and .c++m. The reason why I put etc is that I am afraid of the list may goes longer in the future. But it looks better to state things clear now. I'll follow the suggestion.

228–230

The comes from the clang implementation. If the user don't follow the restrictions, then the clang may fail to build the module. For example, in the "hello world" example, if the name of module file is M.module instead of M.pcm, the the clang would fail to find the corresponding M module.

238

Yes, it is true that -fprebuilt-module-interface takes a directory and -fmodule-file takes a specific file. I'll follow this.

247

I'm guessing that one takes a directory and the other takes a file?

Yes.

I didn't understand what would be more convenient at first

I guess you have understood it. But let me explain it again to avoid any misunderstanding. If there are n dependent module files in the same directory, we may need to n -fmodule-file options, or a -fprebuilt-module-interface option. So I said -fprebuilt-module-interface is more convenient.

426–428

Since it sounds like that's what will happen when switching on optimization?

No, it wouldn't happen. I want to say the users of modules won't get worth performance; but they can't get so much speedup either.

What I want to say in the first sentence is that if the compilation process with optimization is the same with the one with debug, the users will get worth performance, which is unacceptable.

I'm not sure if the current wording eliminates the misunderstanding.

646–647

If we decide to Clang's modulemap

If we decide to reuse Clang's modulemap? It looks like it lacks a verb.

[ChuanqiXu]

Address comments.

[h-vetinari]

It gets very confusing that phab now attaches the old review comments in the wrong place.

clang/docs/CPlusPlus20Modules.rst

23

OK that's fine. Then this should be changed from

Due to the C++20 modules having very different semantics, it might be more friendly for users who care about C++20 modules only to create a new page.

to something like

Due to the C++20 modules having very different semantics, this page deals with them separately.

49–51

Ugh, this should have been "These are".

53

I had corrected myself on second pass, but got confused with phab and ended up not submitting it....

65–68

234–236

The comes from the clang implementation. If the user don't follow the restrictions, then the clang may fail to build the module. For example, in the "hello world" example, if the name of module file is M.module instead of M.pcm, the the clang would fail to find the corresponding M module.

Can we add a sentence to this effect?

317–318

I realized afterwards that it's the inverse - debugging code might depend on the _absence_ of NDEBUG which switches of debug stuff.

472

[ChuanqiXu]

clang/docs/CPlusPlus20Modules.rst

317–318

From my experience, NDEBUG is used more frequently in practice. We generally write:

C++
#ifndef NDEBUG
...
#endif

I guess this might not be a big deal.

[ChuanqiXu]

Address comments.

[ChuanqiXu]

In D131062#3705767, @h-vetinari wrote:

It gets very confusing that phab now attaches the old review comments in the wrong place.

Yeah, I am also worrying if the comments may block other reviewers to review. Do you know any method to ignore these comments in phab? If not, I guess I need to create a new page (but this is what we want?)

[h-vetinari]

In D131062#3705864, @ChuanqiXu wrote:

Yeah, I am also worrying if the comments may block other reviewers to review. Do you know any method to ignore these comments in phab? If not, I guess I need to create a new page (but this is what we want?)

I don't know phab well at all, but I understand that old review comments remain is a feature and not a bug. But purely for pragmatic reasons already, it might be better to open a new revision.

[h-vetinari]

If you do open a new revision, please also consider breaking the lines at a length that phabricator doesn't overflow (seems to be 122 characters), and change all occurrences of "codes" to "code".

clang/docs/CPlusPlus20Modules.rst

677–679

[ChuanqiXu]

clang/docs/CPlusPlus20Modules.rst

677–679

Since it says in the future, if it is better to use may be or will be than are ?

[h-vetinari]

clang/docs/CPlusPlus20Modules.rst

677–679

Since it says in the future, if it is better to use may be or will be than are ?

The "we think" already contains built-in subjectiveness, so the "may be" is redundant in terms of uncertainty. It's not a big deal, but in general: either "they may be" or "we think they are".

It would also be possible to say something like:

So the final answer for why we don't reuse the interface of Clang modules for header units is that
there are some differences between header units and Clang modules and that ignoring those
differences now would likely become a problem in the future.

[iains]

@h-vetinari you are right, this has become difficult to review - I will try and do some more later - just the one comment for now.

clang/docs/CPlusPlus20Modules.rst

677–679

Is it not simpler than this?

Since clang header modules have different semantics from c++20 header units, if we were to force c++20 semantics on clang header modules, that would break existing code (and therefore to support existing code, we would expect clang header modules to be retained indefinitely).

[ChuanqiXu]

clang/docs/CPlusPlus20Modules.rst

677–679

I think we need to express the concern about future changes. We may be able to handle the differences properly now.

[ChuanqiXu]

Let's review it in https://reviews.llvm.org/D131388 later.

[h-vetinari]

clang/docs/CPlusPlus20Modules.rst

65–68

This is marked as done, but is still unchanged in D131388

clang/docs/CPlusPlus20Modules.rst

+=============
+C++20 Modules
+=============
+
+.. contents::
+   :local:
+
+Introduction
+============
+
+Modules have a lot of meanings. For the users of Clang compiler, modules may
+refer to ``Objective-C Modules``, ``Clang C++ Modules`` (or ``Clang Header Modules``,
+etc) and C++20 modules. The implementation of all kinds of the modules in Clang 
+share a big part of codes. But from the perspective of users, their semantics and
+command line interfaces are very different. So it should be helpful for the users
+to introduce how to use C++20 modules.
+
+There is already a detailed document about `Clang modules <Modules.html>`_, it
+should be helpful to read `Clang modules <Modules.html>`_ if you want to know
+more about the general idea of modules. Due to the C++20 modules having very
+different semantics, it might be more friendly for users who care about C++20
+modules only to create a new page.
+
+Although the term ``modules`` has a unique meaning in C++20 Language Specification,
+when people talk about C++20 modules, they may refer to another C++20 feature:
+header units. So this document would try to cover header units too.
+
+C++20 Modules
+=============
+
+This document was intended to be pure manual. But it should be helpful to
+introduce some language background here for readers who are not familiar with
+the new language feature. This document is not intended to be a language
+tutorial. The document would only introduce concepts about the the
+structure and building of the project.
+
+Background and terminology
+--------------------------
+
+Modules
+~~~~~~~
+
+In this document, the term ``Modules``/``modules`` refers to C++20 modules
+feature if it is not decorated by ``Clang``.
+
+Clang Modules
+~~~~~~~~~~~~~
+
+In this document, the term ``Clang Modules``/``Clang modules`` refer to Clang
+c++ modules extension. It is also known as ``Clang header modules``,
+``Clang module map modules`` or ``Clang c++ modules``.
+
+Module and module unit
+~~~~~~~~~~~~~~~~~~~~~~
+
+A module consists of one or more module units. A module unit is a special
+translation unit. Every module unit should have a module declaration. The syntax
+of the module declaration is:
+
+.. code-block:: c++
+
+  [export] module module_name[:partition_name];
+
+Things in ``[]`` means optional. The syntax of ``module_name`` and ``partition_name``
+in regex form should be ``[a-zA-Z_][a-zA-Z_0-9.]*``. The dot ``.`` in the name has
+no special meaning.
+
+In this document, module units are classified into:
+
+* Primary module interface unit.
+
+* Module implementation unit.
+
+* Module partition interface unit.
+
+* Module partition implementation unit.
+
+A primary module interface unit is a module unit whose module declaration is
+``export module module_name;``. The ``module_name`` here denotes the name of the
+module. A module should have one and only primary module interface unit.
+
+A module implementation unit is a module unit whose module declaration is
+``module module_name;``. A module could have multiple module implementation
+units with the same declaration.
+
+A module partition interface unit is a module unit whose module declaration is
+``export module module_name:partition_name;``. The ``partition_name`` should be
+unique to the module.
+
+A module partition implementation unit is a module unit whose module declaration
+is ``module module_name:partition_name;``. The ``partition_name`` should be
+unique to the module.
+
+In this document, we call ``primary module interface unit`` and
+``module partition interface unit`` as ``module interface unit``. We call ``module
+interface unit`` and ``module partition implementation unit`` as
+``importable module unit``. We call ``module partition interface unit`` and
+``module partition implementation unit`` as ``module partition unit``.
+
+Module file
+~~~~~~~~~~~
+
+A module file stands for the precompiled result of an importable module unit.
+
+Global module fragment
+~~~~~~~~~~~~~~~~~~~~~~
+
+In a module unit, the section from ``module;`` to the module declaration is called the global module fragment.
+
+
+How to build projects using modules
+-----------------------------------
+
+Quick Start
+~~~~~~~~~~~
+
+Let's see a hello world example to use modules.
+
+.. code-block:: c++
+
+  // Hello.cppm
+  module;
+  #include <iostream>
+  export module Hello;
+  export void hello() {
+    std::cout << "Hello World!\n";
+  }
+
+  // use.cpp
+  import Hello;
+  int main() {
+    hello();
+    return 0;
+  }
+
+Then we type:
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 Hello.cppm --precompile -o Hello.pcm
+  $ clang++ -std=c++20 use.cpp -fprebuilt-module-path=. Hello.pcm -o Hello.out
+  $ ./Hello.out
+  Hello World!
+
+In this example, we make and use a simple module ``Hello`` which contains only a primary module interface unit ``Hello.cppm``.
+
+Then let's see a little bit more complex HelloWorld example which uses the 4 kinds of module units.
+
+.. code-block:: c++
+
+  // M.cppm
+  export module M;
+  export import :interface_part;
+  import :impl_part;
+  export void Hello();
+
+  // interface_part.cppm
+  export module M:interface_part;
+  export void World();
+
+  // impl_part.cppm
+  module;
+  #include <iostream>
+  #include <string>
+  module M:impl_part;
+  import :interface_part;
+
+  std::string W = "World.";
+  void World() {
+    std::cout << W << std::endl;
+  }
+
+  // Impl.cpp
+  module;
+  #include <iostream>
+  module M;
+  void Hello() {
+    std::cout << "Hello ";
+  }
+
+  // User.cpp
+  import M;
+  int main() {
+    Hello();
+    World();
+    return 0;
+  }
+
+Then we could compile the example by the following command:
+
+.. code-block:: console
+
+  # Precompiling the module
+  $ clang++ -std=c++20 interface_part.cppm --precompile -o M-interface_part.pcm
+  $ clang++ -std=c++20 impl_part.cppm --precompile -fprebuilt-module-path=. -o M-impl_part.pcm
+  $ clang++ -std=c++20 M.cppm --precompile -fprebuilt-module-path=. -o M.pcm
+  $ clang++ -std=c++20 Impl.cpp -fmodule-file=M.pcm -c -o Impl.o
+
+  # Compiling the user
+  $ clang++ -std=c++20 User.cpp -fprebuilt-module-path=. -c -o User.o
+
+  # Compiling the module and linking it together
+  $ clang++ -std=c++20 M-interface_part.pcm -c -o M-interface_part.o
+  $ clang++ -std=c++20 M-impl_part.pcm -c -o M-impl_part.o
+  $ clang++ -std=c++20 M.pcm -c -o M.o
+  $ clang++ User.o M-interface_part.o  M-impl_part.o M.o Impl.o -o a.out
+
+We explain the options in the following sections.
+
+How to enable C++20 Modules
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Currently, C++20 Modules is enabled automatically if the language standard is ``-std=c++20`` or newer.
+The ``-fmodules-ts`` option is deprecated and is planned to be removed.
+
+How to produce a module file
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+We could generate a module file for an importable module unit by ``--precompile`` option.
+
+File name requirement
+~~~~~~~~~~~~~~~~~~~~~
+
+The file name of ``importable module unit`` must end with ``.cppm``
+(or ``.ccm``, ``.cxxm``, etc). The file name of ``module implementation unit``
+should end with ``.cpp`` (or ``.cc``, ``.cxx``, etc).
+
+The file name of module files should end with ``.pcm``.
+The file name of the module file of a ``primary module interface unit`` should be ``module_name.pcm``.
+The file name of module files of ``module partition unit`` should be ``module_name-partition_name.pcm``.
+
+How to specify the dependent module files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+We could use ``-fprebuilt-module-interface`` to tell the compiler the path to search the dependent module files.
+``-fprebuilt-module-interface`` could occur multiple times just like ``-I``.
+
+Another way to specify the dependent module files is to use ``-fmodule-file``.
+
+When we compile ``module implementation unit``, we must pass the module file of the corresponding ``primary module interface unit`` by ``-fmodule-file``.
+The ``-fmodule-file`` option could occur multiple times. For example, the command line to compile ``M.cppm`` in the above example could be rewritten into:
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 M.cppm --precompile -fmodule-file=M-interface_part.pcm -fmodule-file=M-impl_part.pcm -o M.pcm
+
+``-fprebuilt-module-interface`` is more convenient and ``-fmodule-file`` is faster since it would save the time for file lookup.
+
+Remember to linking module files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+It is easy to forget to link module files at first since we may envision module interfaces like headers. It is not true.
+Module units are translation units. We need to compile them and link them like the example shows.
+
+If we want to create libraries for the module files, we can't wrap these module files directly.
+We must compile these module files(``*.pcm``) into object files(``*.o``) and wrap these object files.
+
+Consistency Requirement
+~~~~~~~~~~~~~~~~~~~~~~~
+
+If we envision modules as a cache to speed up compilation, then it is important to keep the cache consistency as other cache techniques.
+So **currently** Clang would do very strict check for consistency.
+
+Options consistency
+^^^^^^^^^^^^^^^^^^^
+
+The language option of module units and their non-module-unit users should be consistent. The following example is not allowed:
+
+.. code-block:: c++
+
+  // M.cppm
+  export module M;
+  
+  // Use.cpp
+  import M;
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 M.cppm --precompile -o M.pcm
+  $ clang++ -std=c++2b Use.cpp -fprebuilt-module-path=.
+
+The compiler would reject the example due to the inconsistent language options.
+Not all options are language options.
+For example, the following example is allowed:
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 M.cppm --precompile -o M.pcm
+  # Inconsistent optimization level.
+  $ clang++ -std=c++20 -O3 Use.cpp -fprebuilt-module-path=.
+  # Inconsistent debugging level.
+  $ clang++ -std=c++20 -g Use.cpp -fprebuilt-module-path=. 
+
+Although the two examples have inconsistent optimization and debugging level, both of them are accepted.
+
+Note that **currently** the compiler don't think it is a problem about inconsistent macro definition. For example:
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 M.cppm --precompile -o M.pcm
+  # Inconsistent optimization level.
+  $ clang++ -std=c++20 -O3 -DNDEBUG Use.cpp -fprebuilt-module-path=.
+
+Currently Clang would accept the above example. But it may produce surprising result if the debugging code dependents on each other.
+
+Source content consistency
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When the compiler reads a module file, the compiler would check the consistency of the corresponding source files. For example:
+
+.. code-block:: c++
+
+  // M.cppm
+  export module M;
+  export template <class T>
+  T foo(T t) {
+    return t;
+  }
+
+  // Use.cpp
+  import M;
+  void bar() {
+    foo(5);
+  }
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 M.cppm --precompile -o M.pcm
+  $ rm -f M.cppm
+  $ clang++ -std=c++20 Use.cpp -fmodule-file=M.pcm
+
+The compiler would reject the example since the compiler failed to find the source file to check the consistency.
+So the following example would be rejected too.
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 M.cppm --precompile -o M.pcm
+  $ echo "int i=0;" >> M.cppm
+  $ clang++ -std=c++20 Use.cpp -fmodule-file=M.pcm
+
+The compiler would reject it too since the compiler detected the file was changed.
+
+But it is OK to move the module file as long as the source files remained:
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 M.cppm --precompile -o M.pcm
+  $ mkdir -p tmp
+  $ mv M.pcm tmp/M.pcm
+  $ clang++ -std=c++20 Use.cpp -fmodule-file=tmp/M.pcm
+
+The above example would be accepted.
+
+If the user doesn't want to follow the consistency requirement due to some reasons (e.g., distributing module files),
+the user could try to use ``-XClang -fmodules-embed-all-files`` when producing module files. For example:
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 M.cppm --precompile -XClang -fmodules-embed-all-files -o M.pcm
+  $ rm -f M.cppm
+  $ clang++ -std=c++20 Use.cpp -fmodule-file=M.pcm
+
+Now the compiler would accept the above example.
+Important note: XClang options are intended to be used by compiler internally and its semantics are not guaranteed to preserve in future versions.
+
+How module speed up the compilation
+-----------------------------------
+
+A classic theory for the reason why modules speed up the compilation is:
+if there are ``n`` headers and ``m`` source files and each header is included by each source file, then the complexity of the compilation is ``O(nm)``;
+But if there are ``n`` module interfaces and ``m`` source files, the complexity of the compilation is ``O(n+m)``. So the modules would get a big win
+when scaling. In a simpler word, we could get rid of many redundant compilations by using modules.
+
+Roughly, the theory is correct. But the problem is that it is too rough. Let's see what would happen actually. And it depends on the optimization level
+actually. 
+
+First is ``O0``. The compilation process is described in the following graph.
+
+.. code-block:: none
+
+  ├-------------frontend----------┼-------------middle end----------------┼----backend----┤
+  │                               │                                       │               │
+  └---parsing----sema----codegen--┴----- transformations ---- codegen ----┴---- codegen --┘
+
+  ┌---------------------------------------------------------------------------------------┐
+  |                                                                                       │
+  |                                     source file                                       │
+  |                                                                                       │
+  └---------------------------------------------------------------------------------------┘
+
+      ┌--------┐
+      │        │
+      │imported│
+      │        │
+      │ codes  │
+      │        │
+      └--------┘
+
+Here we can see that the source file (could be a non-module unit or a module unit) would get processed by the whole pipeline.
+But the imported codes would only get involved in semantical analysis, which is mainly about name lookup, overload resolution and template instantiation.
+All of these processes are fast to the whole compilation process.
+The imported codes could save the time for the fronend code generation and the whole middle end and the backend.
+So we could get big win for the compilation time in O0.
+
+But with optimizations, things are different:
+
+.. code-block:: none
+
+  ├-------- frontend ---------┼--------------- middle end --------------------┼------ backend ----┤
+  │                           │                                               │                   │
+  └--- parsing ---- sema -----┴--- optimizations --- IPO ---- optimizations---┴--- optimizations -┘
+                                                                                                            
+  ┌-----------------------------------------------------------------------------------------------┐
+  │                                                                                               │
+  │                                         source file                                           │
+  │                                                                                               │
+  └-----------------------------------------------------------------------------------------------┘
+      ┌---------------------------------------┐
+      │                                       │
+      │                                       │
+      │            imported codes             │
+      │                                       │
+      │                                       │
+      └---------------------------------------┘
+
+It is not acceptable if we get performance loss after we use modules. The main concern is that when we compile a source file, the compiler need to see the function body
+of imported module units so that the compiler could perform IPO (InterProcedural Optimization, primarily inlining in practice) to optimize functions in current source file
+by the information provided by the imported module units.
+In other words, the imported codes would be processed again and again in importee units by optimizations (including IPO itself).
+The optimizations before IPO and the IPO itself are the most time-consuming part in whole compilation process.
+So from this perspective, we might not be able to get the improvements described in the theory.
+But we could still save the time for optimizations after IPO and the whole backend.
+
+ABI Impacts
+-----------
+
+The declarations in module unit which are not in global module fragment would get new linkage names.
+
+For example,
+
+.. code-block:: c++
+
+  export module M;
+  namespace NS {
+    export int foo();
+  }
+
+The linkage name of ``NS::foo()`` would be ``_ZN2NSW1M3fooEv``.
+This couldn't be demangled by low versions of debugger or demangler.
+User could use ``llvm-cxxfilt`` since 15.x to demangle this:
+
+.. code-block:: console
+
+  $ llvm-cxxfilt _ZN2NSW1M3fooEv
+
+The result would be ``NS::foo@M()``, which reads as ``NS::foo()`` in module ``M``.
+
+The ABI implies that we can't declare something in a module unit and define it in a non-module unit (or vice-versa).
+Since it would meet linking errors.
+
+Known Problems
+--------------
+
+Here lists some problems of modules. You could visit https://github.com/llvm/llvm-project/labels/clang%3Amodules for more issues
+or file a new issue if you don't find an existing one. If you're going to create a new issue for C++20 modules, it is better to add
+tags ``clang:modules`` and start the title with ``[C++20] [Modules]``.
+
+For higher level support for proposals, you could visit https://clang.llvm.org/cxx_status.html.
+
+Support for clang-scan-deps
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The support for clang-scan-deps may be the most urgent problem for modules now.
+Without the support for clang-scan-deps, the build systems are hard to get involved.
+So that the users could only play modules with makefiles or even write a parser by hand.
+It blocks more uses for modules, which will block more defect reports or requirements.
+
+We could track the issue at: https://github.com/llvm/llvm-project/issues/51792.
+
+Ambiguous deduction guide
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Currently, when we call deduction guides in global module fragment,
+we may get incorrect diagnosing message like: `ambiguous deduction`.
+
+So if we're using deduction guide from global module fragment, we probably need to write:
+
+.. code-block:: c++
+
+  std::lock_guard<std::mutex> lk(mutex);
+
+instead of
+
+.. code-block:: c++
+
+  std::lock_guard lk(mutex);
+
+We could tract the issue at: https://github.com/llvm/llvm-project/issues/56916
+
+Ignored PreferredName Attribute
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Due to a tricky problem, when Clang writes module files, Clang will ignore the ``preferred_name`` attribute, if any.
+This implies that the ``preferred_name`` wouldn't show in debugger or dumping.
+
+We could tract the issue at: https://github.com/llvm/llvm-project/issues/56490
+
+Don't emit macros about module declaration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This is covered by P1857R3. We mentione it again here since users may abuse it before we implement it.
+
+The users may want to write codes which could be compiled by modules or non-modules. A direct idea would be use macros like:
+
+.. code-block:: c++
+
+  MODULE
+  IMPORT header_name
+  EXPORT_MODULE MODULE_NAME;
+  IMPORT header_name
+  EXPORT ...
+
+So this file could be triggered like a module unit or a non-module unit by the definition of the macros.
+However, the usage is forbidden by P1857R3 but we haven't implemented P1857R3.
+So it is possible to write illegal modules codes now.
+A simple suggestion would be "Don't play macro tricks with module declarations".
+
+We could tract the issue at: https://github.com/llvm/llvm-project/issues/56917
+
+Header Units
+============
+
+How to build projects using header unit
+---------------------------------------
+
+Quick Start
+~~~~~~~~~~~
+
+For the following example,
+
+.. code-block:: c++
+
+  import <iostream>;
+  int main() {
+    std::cout << "Hello World.\n";
+  }
+
+we could compile it as
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 -xc++-system-header --precompile iostream -o iostream.pcm
+  $ clang++ -std=c++20 -fmodule-file=iostream.pcm main.cpp
+
+How to produce module files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Similar to modules, we could use ``--precompile`` to produce the module file.
+But we need to specify that the input file is a header by ``-xc++-system-header`` or ``-xc++-user-header``.
+
+How to specify the dependent module files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+We could use ``-fmodule-file`` to specify the module files. ``-fmodule-file`` could occur multiple times too.
+
+But what's different is that we can't use ``-fprebuilt-module-path`` to search the module file for header units.
+
+(This may be an implementation defect. Although we could argue that header units have no name according to the spec,
+it is natural that we couldn't search it. But this is not user friendly enough.)
+
+Don't compile the module file
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Another difference with modules is that we can't compile the module file.
+It makes sense due to the semantics of header unit are just like headers.
+
+Include translation
+~~~~~~~~~~~~~~~~~~~
+
+The C++ spec allows the vendors to convert ``#include header-name`` to ``import header-name;`` when possible.
+Currently, Clang would do this translation for the ``#include`` in the global module fragment.
+
+For example, the following two examples are the same:
+
+.. code-block:: c++
+
+  module;
+  import <iostream>;
+  export module M;
+  export void Hello() {
+    std::cout << "Hello.\n";
+  }
+
+with the following one:
+
+.. code-block:: c++
+
+  module;
+  #include <iostream>
+  export module M;
+  export void Hello() {
+      std::cout << "Hello.\n";
+  }
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 -xc++-system-header --precompile iostream -o iostream.pcm
+  $ clang++ -std=c++20 -fmodule-file=iostream.pcm --precompile M.cppm -o M.cpp
+
+In the latter example, the Clang could find the module file for the ``<iostream>``
+so it would try to replace the ``#include <iostream>`` to ``import <iostream>;`` automatically.
+
+
+Relationships between Clang modules
+-----------------------------------
+
+Header units have pretty similar semantics with Clang modules.
+The semantics of both of them are like headers.
+
+In fact, we could even "mimic" the sytle of header units by Clang modules:
+
+.. code-block:: c++
+
+  module "iostream" {
+    export *
+    header "/path/to/libstdcxx/iostream"
+  }
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 -fimplicit-modules -fmodule-map-file=.modulemap main.cpp
+
+It would be simpler if we are using libcxx:
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 main.cpp -fimplicit-modules -fimplicit-module-maps 
+
+Since there is already one `module map <https://github.com/llvm/llvm-project/blob/main/libcxx/include/module.modulemap.in>`_ in the source of libcxx.
+
+Then here will be a direct question: why don't we implement header units by Clang header modules.
+
+Since Clang modules have more semantics like hierarchy, wrapping multiple headrs together as a big module. All of this are not part of C++20 Header units.
+We're afraid that users may abuse these features and think they are using C++20 things.
+
+Another reason is that there are proposals to introduce module mappers to the C++ standard (for example, https://wg21.link/p1184r2).
+Then if we decide to resued Clang's modulemap, we may get in problem once we need to introduce antoher module mapper.
+
+So the final answer for why don't we reuse the interface of Clang modules for header units is that
+we've see some differences between header units and Clang modules and we guess the differences may be bigger
+so that we can't accept it.

clang/docs/index.rst

.. toctree::
    SanitizerCoverage
    SanitizerStats
    SanitizerSpecialCaseList
    ControlFlowIntegrity
    LTOVisibility
    SafeStack
    ShadowCallStack
    SourceBasedCodeCoverage
+   CPlusPlus20Modules
    Modules
    MSVCCompatibility
    MisExpect
    OpenCLSupport
    OpenMPSupport
    SYCLSupport
    HLSLSupport
    ThinLTO