[docs] Add "Standard C++ Modules"

[ChuanqiXu]

Summary

Successor of D131062. We create the new page due to the old one has too many review comments to review. So we decide to create the new page to review more clearly.

---

We get some standard C++ module things done in clang15.x. But we lack a user documentation for it. The implementation of standard C++ modules share a big part of codes with clang modules. But they have very different semantics and user interfaces, so I think it is necessary to add a document for Standard C++ modules. Previously, there were also some people ask the document for standard C++ Modules and I couldn't offer that time.

I wish we could land this before September and I can backport this to 15.x. This implies that even we solve any bug during the reviewing process. We shouldn't edit the document in time. We should land it and edit it.

Tested with grammarly and make docs-clang-html.

[h-vetinari]

Thanks! Repeating a point that might have been overlooked from D131062 (this time not as comments in the diff to avoid the "pollution" that caused the move to this PR):

If you do open a new revision, please also consider breaking the lines at a length that phabricator doesn't overflow (seems to be 122 characters), and change all occurrences of "codes" to "code".

[iains]

general comment.

Do we encourage contractions (don't, can't) etc. in documentation?
I would suggest that to assist in any translation process it is better to write "do not" or "can not" instead (but that's just an opinion, not a matter of correctness).

clang/docs/CPlusPlus20Modules.rst

20–22 ↗

(On Diff #450747)

I do not think we need to justify the existence of the page, just identify its purpose - maybe like:

"Since C++20 modules have different semantics (and work flows) from `Clang modules this page describes the background and use of Clang with C++20 modules."

25–26 ↗

(On Diff #450747)

this also includes C++20 header units, which are also covered in this document.

57 ↗

(On Diff #450747)

must

219 ↗

(On Diff #450747)

procedural point: do we ever actually remove an existing option - or does it just become a "NOP"?

259 ↗

(On Diff #450747)

link

458 ↗

(On Diff #450747)

The declarations in a module unit which are not in global module fragment have new linkage names.

592 ↗

(On Diff #450747)

With the existing implementation `-fprebuilt-module-path cannot be used for header units (since they are nominally anonymous). For header units use -fmodule-file` to include the relevant PCM file for each header unit.

594–595 ↗

(On Diff #450747)

Notes ( we have an internal implementation name, which can be used by planned tooling and/or mapping changes)

• so maybe:

this is expect to be solved in future editions of the compiler either by the tooling finding and specifying the -fmodule-file or by the use of a module-mapper that understands how to map the header name to their PCMs.

[iains]

clang/docs/CPlusPlus20Modules.rst

673–674 ↗

(On Diff #450747)

as an aside :

there is an open question in the implementation of p1184r2 as to whether one form of input that the module mapper could consume would be module map files (but, obviously, producing C++20 compliant output).

I wonder if this section is adding information that is useful to the user ?
(perhaps it is more documentation of implementation decisions?)

As noted before semantics of clang module headers != semantics of C++20 header units seems a sufficient reason for keeping them separate?

I am not sure what could change in the future to alter this, since existing code will have the current semantics, even if some change is later made to the semantics of clang header modules ?

[ChuanqiXu]

In D131388#3706072, @h-vetinari wrote:

Thanks! Repeating a point that might have been overlooked from D131062 (this time not as comments in the diff to avoid the "pollution" that caused the move to this PR):

If you do open a new revision, please also consider breaking the lines at a length that phabricator doesn't overflow (seems to be 122 characters), and change all occurrences of "codes" to "code".

Yeah, I think I've addressed it.

In D131388#3706080, @iains wrote:

general comment.

Do we encourage contractions (don't, can't) etc. in documentation?
I would suggest that to assist in any translation process it is better to write "do not" or "can not" instead (but that's just an opinion, not a matter of correctness).

Yes. I've used many must/should in the document. Maybe there is someplace missing.

clang/docs/CPlusPlus20Modules.rst

25–26 ↗

(On Diff #450747)

I took half of the suggestion. Since I still want to clarify that the two terms modules and header units are different.

219 ↗

(On Diff #450747)

According to the history, we did actually remove some option: https://github.com/llvm/llvm-project/commit/29f852a1516bcd3928dad74835965f238de34409

673–674 ↗

(On Diff #450747)

I wonder if this section is adding information that is useful to the user ?

(perhaps it is more documentation of implementation decisions?)

I think this section is helpful to eliminate the confusion of the users. Since the users may be confused (or even complain) why don't we reuse the existing tools.

As noted before semantics of clang module headers != semantics of C++20 header units seems a sufficient reason for keeping them separate?

I am not sure what could change in the future to alter this, since existing code will have the current semantics, even if some change is later made to the semantics of clang header modules ?

To be honest, I think the differences mentioned above could be handled by compiler internally. (maybe a lot of if (isHeaderUnit)....). Or even the folks of clang modules could get involved in and say "OK, we could accept the cost." But it is a strong argument that it is possible to introduce new module mappers. So the current decision is much more comprehensible to me.

[ChuanqiXu]

Address comments.

[ruoso]

clang/docs/CPlusPlus20Modules.rst

70–76 ↗

(On Diff #451030)

The terminology here is a bit different than what we've been building the consensus on. Please take a look at sourcefiles.tex (or section [source.types] in the rendered version

[ruoso]

clang/docs/CPlusPlus20Modules.rst

225 ↗

(On Diff #451030)

Likewise, here the term "Built Module Interface file", with the acronym "BMI" is what we're generally using when talking about the generated file.

238 ↗

(On Diff #451030)

Is that actually true? Or does it require explicit arguments to explain how the compiler needs to translate the file?

243–244 ↗

(On Diff #451030)

Is that the case for C++20 named modules as well? I thought this was just for clang modules. How does the lookup work?

250–253 ↗

(On Diff #451030)

I wonder if it's easier to explain that a module implementation unit implicitly imports the primary module interface unit, and therefore it needs the BMI for that interface to be provided, just like it's the case for every other import statement.

[ChuanqiXu]

clang/docs/CPlusPlus20Modules.rst

70–76 ↗

(On Diff #451030)

If there is a consensus already, we should follow.

(BTW, I thought we'll discuss module things in SG2 but it looks like we're discussing them in SG15... my bad)

225 ↗

(On Diff #451030)

Yeah, this is what I was confused in the chat. In my mind, "BMI" describes a compatible interface format like ABI (like Itanium ABI). In another words, a BMI could be compiled by compiler which follows the BMI standard (like clang and gcc both accepts Itanium ABI). But currrently, a module file couldn't be compiled by clang in different versions.

So from my point of view, the term module file is more appropriate than BMI now.

243–244 ↗

(On Diff #451030)

In this document, modules are referring to C++20 Named Modules by default. The option is borrowed from clang modules. The look up rule here is:

(1) When we import module M. The compiler would look up M.pcm in the directories specified by `-fprebuilt-module-interface`.
(2) When we import partition module unit M:P. The compiler would look up M-P.pcm in the directories specified by `-fprebuilt-module-interface`.

I thought this is clear enough. Do you have suggestion to improve? I am a little worried to be wordy.

250–253 ↗

(On Diff #451030)

OK, I was afraid to cite too many standard paragraphs to scare readers. But this point looks necessary

[dblaikie]

clang/docs/CPlusPlus20Modules.rst

31 ↗

(On Diff #451030)

122 ↗

(On Diff #451030)

262 ↗

(On Diff #451030)

312–321 ↗

(On Diff #451030)

this sort of aside might be best left for a separate part of the document - an FAQ/side-notes (a footnote, perhaps?), etc to keep the rest of the document more focussed?

347 ↗

(On Diff #451030)

Could probably skip the -f?

395–396 ↗

(On Diff #451030)

I'm not sure I'm able to follow the example and how it justifies the rough theory as inadequate to explain the motivation for modules - could you clarify more directly (in comments, and then we can discuss how to word it) what the motivation for this section is/what you're trying to convey?

610 ↗

(On Diff #451030)

Might need some more words here - I guess this means "there is no .o for a .pcm from a header unit" basically?

[ruoso]

clang/docs/CPlusPlus20Modules.rst

70–76 ↗

(On Diff #451030)

Yes, please consider joining us for the sg15 meetings, we've been working through quite a few things related to C++ modules.

225 ↗

(On Diff #451030)

BMI is definitely not as compatible as the ABI. I think there's some confusion here of what the term BMI is. In this case the pcm file is the BMI.

243–244 ↗

(On Diff #451030)

Yes, thanks for the clarification. I think it would be worth including a description of how the lookup works as a part of the doc.

[ChuanqiXu]

Address comments.

[ChuanqiXu]

clang/docs/CPlusPlus20Modules.rst

31 ↗

(On Diff #451030)

The was here is intended. I want to say I thought to not add the all other information and just write it like a manual, which might look like:

-unwindlib=<value>      Unwind library to use
-U <macro>              Undefine macro <macro>
--verify-debug-info     Verify the binary representation of debug output
-verify-pch             Load and verify that a pre-compiled header file is not stale
--version               Print version information
-v                      Show commands to run and use verbose output
-Wa,<arg>               Pass the comma separated arguments in <arg> to the assembler
-Wdeprecated            Enable warnings for deprecated constructs and define __DEPRECATED
-Wl,<arg>               Pass the comma separated arguments in <arg> to the linker
-working-directory <value>
                        Resolve file paths relative to the specified directory
-Wp,<arg>               Pass the comma separated arguments in <arg> to the preprocessor
-W<warning>             Enable the specified warning
-w                      Suppress all warnings
-Xanalyzer <arg>        Pass <arg> to the static analyzer
-Xarch_device <arg>     Pass <arg> to the CUDA/HIP device compilation
-Xarch_host <arg>       Pass <arg> to the CUDA/HIP host compilation
-Xassembler <arg>       Pass <arg> to the assembler
-Xclang=<arg>           Alias for -Xclang
-Xclang <arg>           Pass <arg> to clang -cc1
-Xcuda-fatbinary <arg>  Pass <arg> to fatbinary invocation
-Xcuda-ptxas <arg>      Pass <arg> to the ptxas assembler
-Xlinker <arg>          Pass <arg> to the linker
-Xoffload-linker<triple> <arg>
                        Pass <arg> to the offload linkers or the ones idenfied by -<triple>
-Xopenmp-target=<triple> <arg>
                        Pass <arg> to the target offloading toolchain identified by <triple>.
-Xopenmp-target <arg>   Pass <arg> to the target offloading toolchain.
-Xpreprocessor <arg>    Pass <arg> to the preprocessor
-x <language>           Treat subsequent input files as having type <language>
-z <arg>                Pass -z <arg> to the linker

70–76 ↗

(On Diff #451030)

(Yeah, I've subscribed the sg15 mailing list.)

I've followed the terminology in the link except importable unit. In that link, importable unit may contain header units. But in this documentation, we want to split modules and header units. So I take importable module unit here. I guess it might not be a big deal.

225 ↗

(On Diff #451030)

Oh, it confuses me. In my mind, BMI is corresponding to ABI (although the format of BMI is not standardized) and the .pcm file is corresponding to .o files naturally (to me). So generally, we don't say a .o file is the ABI. And similar, it is weird to me to say the .pcm files is the BMI.

I've added a sentence in the terminology section of module file to explain the term 'BMI'. I tried to copy it from the module-ecosystem-technical-report but I failed to find one. So I just put the simple explanation.

312–321 ↗

(On Diff #451030)

I guess it is better to remain them here. Since the sentences tries to say the macro definitions are not language options, which is consistent to the above paragraphs to me.

347 ↗

(On Diff #451030)

In my system, when I run rm M.cppm, it asks rm: remove regular file ‘M.cppm’? and I need to enter y then it would remove the file actually. So it looks better to add -f to me.

395–396 ↗

(On Diff #451030)

Let me answer the motivation first. The motivation comes from my personal experience. I feel like when most people heard modules, they would ask "how much speedup could we get"? And there are some other questions like "why does modules speedup the compilation?". So I guess the readers of the document may have similar questions and I try to answer it here.

The complexity theory is correct but it may be too abstract to our users. Since the complexity theory is about the scaling. But for certain users, the scales of their codes are temporarily fixed. So when they try to use modules but find the speedup doesn't meet their expectation in O2. They may feel frustrated. And it doesn't work if I say, "hey, you'll get much better speedup if the your codes get 10x longer." I guess they won't buy in. So what I try to do here is to manage the user's expectation to avoid any misunderstanding.

Following off is about the explanation. For example, there are 1 module interface and 10 users. There is a function F in the module interface and the function is used by every users. And let's say we need a T time to compile the function F and each users without the function F.
In O0, the function F will get compiled completely once and get involved in the Sema part 10 times. Due to the Sema part is relatively fast and let's say the Sema part would take 0.1T. Given we compile them serially, we need 12T to compile the project.

But if we are with optimizations, each function F will get involved in optimizations and IPO in every users. And these optimizations are most time-consuming. Let's say these optimizations will consume 0.8T. And the time required will be 19T. It is easy to say the we need 20T to compile the project if we're using headers. So we could find the speedup with optimization is much slower.

BTW, if we write the required time with variables, it will be nT + mT + T*m*additional_compilation_part. The additional_compilation_part here corresponds to the time percentage of Sema or Optimizations. And since T and additional_compilation_part are both constant. So if we write them in O() form, it would be O(n+m).
So the theory is still correct.

610 ↗

(On Diff #451030)

I guess this means "there is no .o for a .pcm from a header unit" basically?

Yes.

[dblaikie]

(I'll +1 to @ruoso's comments about using BMI - I think it might be best not to introduce a different term ("Module file") as it might confuse people as it's fairly close to "module source file", etc. BMI makes it clear/unambiguous that it's the binary file, not the source file)

clang/docs/CPlusPlus20Modules.rst

665 ↗

(On Diff #451361)

could this use a strikethrough, perhaps? (not sure if you can strikethrough inside a code block)

347 ↗

(On Diff #451030)

Perhaps your rm is aliased to rm -i? ( https://superuser.com/questions/345923/remove-file-without-asking ) - rm generally/without flags wouldn't prompt for removal of a writable file & encouraging people to get in the habit of using -f (especially when they probably don't need to - even if you do on your system) seems a bit problematic. I think not including the input/output of rm in case some users have it setup to require some interaction is OK - a plain rm is probably enough for users to understand what to do on their system.

395–396 ↗

(On Diff #451030)

I think the message is getting a bit lost in the text (both in the proposed text, and the comment here).

"At -O0 implementations of non-inline functions defined in a module will not impact module users, but at higher optimization levels the definitions of such functions are provided to user compilations for the purposes of optimization (but definitions of these functions are still not included in the use's object file) - this means build speed at higher optimization levels may be lower than expected given -O0 experience, but does provide by more optimization opportunities"

[ChuanqiXu]

Address comments:

• Change module file to BMI.
• Replace rm -f with rm.
• Add a conclusion paragraph to the How module speed up compilation section.

[ChuanqiXu]

clang/docs/CPlusPlus20Modules.rst

347 ↗

(On Diff #451030)

Oh, got it.

395–396 ↗

(On Diff #451030)

Yes, it is hard to talk clearly and briefly. In your suggested wording, you mentioned non-inline function, it is accurate but bring new information to this document. I'm worrying if the reader could understand it if the reader don't know c++ so much.

I put the suggested wording as the conclusion paragraph for the section and hope it could make the reader focus on the intention of the section.

[ChuanqiXu]

clang/docs/CPlusPlus20Modules.rst

665 ↗

(On Diff #451361)

It looks like we can't do it in code blocks.. maybe we could only depend the comments above.

[dblaikie]

clang/docs/CPlusPlus20Modules.rst

395–396 ↗

(On Diff #451030)

Maybe "non-inline" could be replaced by "module implementation details" (but "function bodies" sounds OK too)

I think the issue for me is that the current description seems to go into more detail about compiler implementation details than might be helpful for a document at this level. I was/am hoping maybe a one paragraph summary might be simpler/more approachable/sufficiently accurate for the audience.

[ChuanqiXu]

Remove non-inline terms.

[ChuanqiXu]

clang/docs/CPlusPlus20Modules.rst

395–396 ↗

(On Diff #451030)

Yeah, it is hard to control the balance between readability vs accuracy. From my personal experience, the 3-stage compilation model is relatively easy to be understood. I've explained the 3-stage compilation model for some our friends who are not majored in CS and all of them could understand it. But I know some programmers still think inline specifier is a optimization hint to the compiler..

After all, it is hard to tell if this is helpful for most readers. But I think the answer is yes from my personal experience.

[ruoso]

clang/docs/CPlusPlus20Modules.rst

309 ↗

(On Diff #452095)

I think this is a bit confusing. The BMI is not linked...

Maybe something like: "Remember that modules still have an object counterpart to the BMI".

Specifically, it may be important to make a distinction between when the compiler is invoked to produce the bmi and when it's invoked to produce the object, and that you use the bmi when translating code that imports it, and the object when linking.

[dblaikie]

Actually, when it comes to diagrams - maybe what'd be good is a diagram of classic compilation and a diagram of modules and header modules

src1.cpp -+> clang++ src1.cpp --> src1.o ---, 
hdr1.h  --'                                 +-> clang++ src1.o src2.o ->  executable
hdr2.h  --,                                 |
src2.cpp -+> clang++ src2.cpp --> src2.o ---'

              src1.cpp ----------------------------------------+> clang++ src1.cpp -------> src1.o -, 
(header unit) hdr1.h    -> clang++ hdr1.h ...    -> hdr1.pcm --'                                    +-> clang++ src1.o mod1.o src2.o ->  executable
              mod1.cppm -> clang++ mod1.cppm ... -> mod1.pcm --,--> clang++ mod1.pcm ... -> mod1.o -+
              src2.cpp ----------------------------------------+> clang++ src2.cpp -------> src2.o -'

(no doubt could look a lot better - but would pretty quickly summarize what's going where - maybe the command lines are too long to include nicely in such a diagram and so the diagram could be a more simplified block diagram and the full command lines shown separately)

clang/docs/CPlusPlus20Modules.rst

395–396 ↗

(On Diff #451030)

I still think it's a lot of text and diagrams that, to me, don't convey enough detail/clearly, to spend on what I'd consider a fairly side issue to the main discussion.

[ChuanqiXu]

Address comments.

[ChuanqiXu]

Address comments.

[ChuanqiXu]

In D131388#3719794, @dblaikie wrote:

Actually, when it comes to diagrams - maybe what'd be good is a diagram of classic compilation and a diagram of modules and header modules

src1.cpp -+> clang++ src1.cpp --> src1.o ---, 
hdr1.h  --'                                 +-> clang++ src1.o src2.o ->  executable
hdr2.h  --,                                 |
src2.cpp -+> clang++ src2.cpp --> src2.o ---'

              src1.cpp ----------------------------------------+> clang++ src1.cpp -------> src1.o -, 
(header unit) hdr1.h    -> clang++ hdr1.h ...    -> hdr1.pcm --'                                    +-> clang++ src1.o mod1.o src2.o ->  executable
              mod1.cppm -> clang++ mod1.cppm ... -> mod1.pcm --,--> clang++ mod1.pcm ... -> mod1.o -+
              src2.cpp ----------------------------------------+> clang++ src2.cpp -------> src2.o -'

(no doubt could look a lot better - but would pretty quickly summarize what's going where - maybe the command lines are too long to include nicely in such a diagram and so the diagram could be a more simplified block diagram and the full command lines shown separately)

Thanks for the diagrams! It is helpful. I added a sentence: (But we can't do this for the BMI from header units. See the later section for the definition of header units) since I'm not sure if all readers are familiar with header units.

clang/docs/CPlusPlus20Modules.rst

309 ↗

(On Diff #452095)

Specifically, it may be important to make a distinction between when the compiler is invoked to produce the bmi and when it's invoked to produce the object, and that you use the bmi when translating code that imports it, and the object when linking.

I don't get the meaning here. The doc said the user need to compile *.cppm to *.pcm before imported it and the doc also said the user need to compile *.pcm to *.o to link. So I guess the doc has stated it clear?

395–396 ↗

(On Diff #451030)

Got it. I still want to remain the parts since I'm sure that there will be readers who are confusing about it. But your words make sense too. I address your comments by moving the part into the end of the document in Possible Question section. Do you think this makes sense?

[ruoso]

clang/docs/CPlusPlus20Modules.rst

338–339 ↗

(On Diff #452612)

[ChuanqiXu]

Address comments.

[ChuanqiXu]

Add description to use -fmodule-header to generate BMI for header units.

[ChuanqiXu]

If no objection (not comments) come in, I want to land this in next Friday (8.25) since I want to backport this to 15.x and 15.x is going to release in September.

[JohelEGP]

clang/docs/CPlusPlus20Modules.rst

290 ↗

(On Diff #453207)

457 ↗

(On Diff #453207)

[ChuanqiXu]

Address comments.

[ChuanqiXu]

clang/docs/CPlusPlus20Modules.rst

457 ↗

(On Diff #453207)

Thanks for reviewing. BTW, the request to change button is fair to use. But it scares me a little with typo comments..

[ChuanqiXu]

Replace C++20 Modules with Standard C++ Modules since @ruoso pointed out that Modules is not specific to certain versions of C++ (for example, Modules may get some big changes in C++23, C++26, ...).

[h-vetinari]

In D131388#3748062, @ChuanqiXu wrote:

Replace C++20 Modules with Standard C++ Modules since @ruoso pointed out that Modules is not specific to certain versions of C++ (for example, Modules may get some big changes in C++23, C++26, ...).

The filename (and hence future doc URL) still contains the "20".

[ruoso]

I just noticed that the document was implying that Header Units were separate from the Standard C++ Modules, but they are an integral part of the language in the specification of modules. The contrast that we do is between "Named Modules" and "Header Units". I did some edits to help convey that.

clang/docs/CPlusPlus20Modules.rst

24–26 ↗

(On Diff #455462)

28–29 ↗

(On Diff #455462)

285–287 ↗

(On Diff #455462)

Which one takes precedence?

585–586 ↗

(On Diff #455462)

728–729 ↗

(On Diff #455462)

[ChuanqiXu]

In D131388#3748769, @ruoso wrote:

I just noticed that the document was implying that Header Units were separate from the Standard C++ Modules, but they are an integral part of the language in the specification of modules. The contrast that we do is between "Named Modules" and "Header Units". I did some edits to help convey that.

Yeah, I know header units are part of modules. Technically, the "header units" are special synthesized units . And the entities in header units part of the global module. And global module have no name. So here is the term Named Modules. I didn't mention this since I feel this bring some language details to the users but I feel like this is not helpful for users. For me, it is simpler and workable to think "header units" are special headers and treat "named modules" and "modules" interchangeably. But given you are doing a lot of modules documentation work, I'll follow your suggestion to make the terms consistency.

(BTW, it is a little bit weird to me since "header units" should be "units" literally but we call it modules)

[ChuanqiXu]

Address comments:

• Change the file name to StandardCPlusPlusModules
• Combine header units with modules more tightly.

[ChuanqiXu]

In D131388#3748769, @ruoso wrote:

I just noticed that the document was implying that Header Units were separate from the Standard C++ Modules, but they are an integral part of the language in the specification of modules. The contrast that we do is between "Named Modules" and "Header Units". I did some edits to help convey that.

Done. Although I wanted to land this today but I feel it will be better to be approved by you. So I'll wait longer.

[ruoso]

Thanks for taking my last minute suggestions

[bbrown105]

Nice writeup! I appreciate the work that went into this.

clang/docs/StandardCPlusPlusModules.rst

27

Just matching the capitalization of the other sections and subsections.

87

This is a small tweak but it should clarify that using the same partition_name with entirely different module_names is fine.

89

91

243–244

Is there a tracking issue to revise this choice? I seem to recall that we settled on recommending a different file extension, *.ixx, for at least primary module interface units.

We didn't really discuss extensions for other units, but maybe we should if different toolchains are going to have different expectations. Of course, build systems can work around this sort of thing. But if the expectation of clang is that, practically speaking, you'll probably be using a build system, maybe the docs should clarify as much. We could document that parts of this document are intended for people configuring or implementing build systems.

FYI @Bigcheese

744

[aaronmondal]

@ChuanqiXu Thanks a lot for this!

We are currently implementing this module logic in the Bazel ruleset rules_ll.
This page helps clarify the differences between clang modules and C++ modules a lot. If I knew about this earlier we'd probably have saved a lot of time 😅

After reading this doc, a few things came to my mind. I can add a patch for this as a follow-up diff when I better understand how things work.

• The path logic that references the source files in BMIs is very important. When precompiling a module in one directory and then moving a header included in the global module fragment from the source file, things will break.
• As a result of this, some unintuitive things can happen when using sandboxed compilation. E.g. precompile BMI in sandbox, move BMI out of that sandbox, then start another sandbox that loads the previously built BMI. If the BMI used headers in a global module fragment, and those headers were in the original sandboxe, the header in the new sandbox will be in a different location (because the new sandbox is at a new location) and things will break.
• Clang modulemaps are cool, but for build systems that have to declare every output, implicit modulemaps seem like a bad idea. Clang's caching system, even with custom cache paths seems like a bad idea as well. It is probably helpful to mention caveats of implicit modules and how to disable them (or link to the Clang Module docs for this).
• The doc describes how to build e.g. an iostream module using the C++ semantics, but doesn't mention that AFAIK there is no "native" C++ Module build for libcxx yet. One would currently actually use Clang modules with implicit modulemaps. It may be helpful to mention this to avoid confusion.
• Issue https://github.com/llvm/llvm-project/issues/56770 and seems-to-be-a-consequence-issue https://github.com/llvm/llvm-project/issues/57293 may be relevant apart from the already mentioned issue about clang-scan-deps.

[ChuanqiXu]

Address comments.

[ChuanqiXu]

Add contents for "When precompiling a module in one directory and then moving a header included in the global module fragment from the source file, things will break."

[ChuanqiXu]

In D131388#3753522, @aaronmondal wrote:

@ChuanqiXu Thanks a lot for this!

We are currently implementing this module logic in the Bazel ruleset rules_ll.
This page helps clarify the differences between clang modules and C++ modules a lot. If I knew about this earlier we'd probably have saved a lot of time 😅

Yeah, the reason why I want to land this faster is that I've seen some people confused about this too.

(I've watched rules_ll for releases. So it will be better to *release* it if you guys get something done)

After reading this doc, a few things came to my mind. I can add a patch for this as a follow-up diff when I better understand how things work.

• The path logic that references the source files in BMIs is very important. When precompiling a module in one directory and then moving a header included in the global module fragment from the source file, things will break.

Yes and I mentioned it implicitly that we can workaround it by -Xclang -fmodules-embed-all-files options. And I add a paragraph at the end of Source content consistency section to make it explicitly. Would you like to take a look?

• Clang modulemaps are cool, but for build systems that have to declare every output, implicit modulemaps seem like a bad idea. Clang's caching system, even with custom cache paths seems like a bad idea as well. It is probably helpful to mention caveats of implicit modules and how to disable them (or link to the Clang Module docs for this).
• The doc describes how to build e.g. an iostream module using the C++ semantics, but doesn't mention that AFAIK there is no "native" C++ Module build for libcxx yet. One would currently actually use Clang modules with implicit modulemaps. It may be helpful to mention this to avoid confusion.

In fact, initially I want this document to be separate from clang module map modules. Although they share a lot in the implementation, they are totally different to the users. (Although they share some options too). To be clear, I'm not saying we'll abandon/disallow the mixed use of clang module map modules with standard c++ modules. The status quo is that every thing is still unclear. I'm not sure if they will work together correctly or what the cost we'll pay for to make them work together. So when I wrote this document, I intentionally skipped the part of how to use clang module map modules.

And I think it will be great to add a section "interaction with clang module map modules" once we have the use experience of mixed form.

• Issue https://github.com/llvm/llvm-project/issues/56770 and seems-to-be-a-consequence-issue https://github.com/llvm/llvm-project/issues/57293 may be relevant apart from the already mentioned issue about clang-scan-deps.

I'm not sure. And for this document, I feel it is fine to not include all the issues. Otherwise it may be a little bit scary to people who reads it first.

---

@aaronmondal @bbrown105 Are you happy to land this document in the current status? So that I can backport this to 15.x in time. (clang 15.x is going to be released in the next week). Then the readers could read the document for 15.x at https://releases.llvm.org/15.0.0/tools/clang/docs/StandardCPlusPlusModules.html. Then we can keep maintaining it further. This should be helpful for version controlling.

clang/docs/StandardCPlusPlusModules.rst

243–244

Is there a tracking issue to revise this choice? I seem to recall that we settled on recommending a different file extension, *.ixx, for at least primary module interface units.

I remember there is not an existing one. And I just filed https://github.com/llvm/llvm-project/issues/57416. I think you can post the consensus of SG15 in that issue and we can follow it.

I edit the wording slightly (must -> should) and add a known problem section for it. I think we can edit the document further after we get more consensus.

But if the expectation of clang is that, practically speaking, you'll probably be using a build system, maybe the docs should clarify as much.

No, I am not using/developing a build system for modules (Currently I write command line options manually by Makefile). And the suffix expectation are required by the current clang15.x. And we can change it in further versions.

[aaronmondal]

Thanks for addressing my comment. I think I overlooked the part about -fmodules-embed-all-files 😅

I'm fine with this revision 😊

[aaronmondal]

I just noticed that there probably needs to be one change. The doc sometimes describes -fprebuilt-module-interface, which is not a valid option. I think the occurrences were meant to be -fpbrebuilt-module-path.

[ChuanqiXu]

Address comments.

[ChuanqiXu]

In D131388#3758898, @aaronmondal wrote:

I just noticed that there probably needs to be one change. The doc sometimes describes -fprebuilt-module-interface, which is not a valid option. I think the occurrences were meant to be -fpbrebuilt-module-path.

Oh, thanks for noticing. It is weird that I made such a mistake...

[ChuanqiXu]

@bbrown105 I am going to land this today. it looks like you're not objecting it and we don't have the time to wait for your formal approval...

[ChuanqiXu]

I'm going to land this. Thanks for everyone who reviewed this!

clang/docs/StandardCPlusPlusModules.rst

+====================
+Standard C++ Modules
+====================
+
+.. contents::
+   :local:
+
+Introduction
+============
+
+The term ``modules`` has a lot of meanings. For the users of Clang, modules may
+refer to ``Objective-C Modules``, ``Clang C++ Modules`` (or ``Clang Header Modules``,
+etc.) or ``Standard C++ Modules``. The implementation of all these kinds of modules in Clang
+has a lot of shared code, but from the perspective of users, their semantics and
+command line interfaces are very different. This document focuses on
+an introduction of how to use standard C++ modules in Clang.
+
+There is already a detailed document about `Clang modules <Modules.html>`_, it
+should be helpful to read `Clang modules <Modules.html>`_ if you want to know
+more about the general idea of modules. Since standard C++ modules have different semantics
+(and work flows) from `Clang modules`, this page describes the background and use of
+Clang with standard C++ modules.
+
+Modules exist in two forms in the C++ Language Specification. They can refer to
+either "Named Modules" or to "Header Units". This document covers both forms.
+
+standard C++ Named modules
+==========================
+
+This document was intended to be a manual first and foremost, however, we consider it helpful to
+introduce some language background here for readers who are not familiar with
+the new language feature. This document is not intended to be a language
+tutorial; it will only introduce necessary concepts about the
+structure and building of the project.
+
+Background and terminology
+--------------------------
+
+Modules
+~~~~~~~
+
+In this document, the term ``Modules``/``modules`` refers to standard C++ modules
+feature if it is not decorated by ``Clang``.
+
+Clang Modules
+~~~~~~~~~~~~~
+
+In this document, the term ``Clang Modules``/``Clang modules`` refer to Clang
+c++ modules extension. These are also known as ``Clang header modules``,
+``Clang module map modules`` or ``Clang c++ modules``.
+
+Module and module unit
+~~~~~~~~~~~~~~~~~~~~~~
+
+A module consists of one or more module units. A module unit is a special
+translation unit. Every module unit must have a module declaration. The syntax
+of the module declaration is:
+
+.. code-block:: c++
+
+  [export] module module_name[:partition_name];
+
+Terms enclosed in ``[]`` are optional. The syntax of ``module_name`` and ``partition_name``
+in regex form corresponds to ``[a-zA-Z_][a-zA-Z_0-9\.]*``. In particular, a literal dot ``.``
+in the name has no semantic meaning (e.g. implying a hierarchy).
+
+In this document, module units are classified into:
+
+* Primary module interface unit.
+
+* Module implementation unit.
+
+* Module interface partition unit.
+
+* Internal module partition unit.
+
+A primary module interface unit is a module unit whose module declaration is
+``export module module_name;``. The ``module_name`` here denotes the name of the
+module. A module should have one and only one primary module interface unit.
+
+A module implementation unit is a module unit whose module declaration is
+``module module_name;``. A module could have multiple module implementation
+units with the same declaration.
+
+A module interface partition unit is a module unit whose module declaration is
+``export module module_name:partition_name;``. The ``partition_name`` should be
+unique to the module.
+
+A internal module partition unit is a module unit whose module declaration
+is ``module module_name:partition_name;``. The ``partition_name`` should be
+unique to the module.
+
+In this document, we use the following umbrella terms:
+
+* A ``module interface unit`` refers to either a ``primary module interface unit``
+  or a ``module interface partition unit``.
+
+* An ``importable module unit`` refers to either a ``module interface unit``
+  or a ``internal module partition unit``.
+
+* A ``module partition unit`` refers to either a ``module interface partition unit``
+  or a ``internal module partition unit``.
+
+Built Module Interface file
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+A ``Built Module Interface file`` stands for the precompiled result of an importable module unit.
+It is also called the acronym ``BMI`` genrally.
+
+Global module fragment
+~~~~~~~~~~~~~~~~~~~~~~
+
+In a module unit, the section from ``module;`` to the module declaration is called the global module fragment.
+
+
+How to build projects using modules
+-----------------------------------
+
+Quick Start
+~~~~~~~~~~~
+
+Let's see a "hello world" example that uses modules.
+
+.. code-block:: c++
+
+  // Hello.cppm
+  module;
+  #include <iostream>
+  export module Hello;
+  export void hello() {
+    std::cout << "Hello World!\n";
+  }
+
+  // use.cpp
+  import Hello;
+  int main() {
+    hello();
+    return 0;
+  }
+
+Then we type:
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 Hello.cppm --precompile -o Hello.pcm
+  $ clang++ -std=c++20 use.cpp -fprebuilt-module-path=. Hello.pcm -o Hello.out
+  $ ./Hello.out
+  Hello World!
+
+In this example, we make and use a simple module ``Hello`` which contains only a
+primary module interface unit ``Hello.cppm``.
+
+Then let's see a little bit more complex "hello world" example which uses the 4 kinds of module units.
+
+.. code-block:: c++
+
+  // M.cppm
+  export module M;
+  export import :interface_part;
+  import :impl_part;
+  export void Hello();
+
+  // interface_part.cppm
+  export module M:interface_part;
+  export void World();
+
+  // impl_part.cppm
+  module;
+  #include <iostream>
+  #include <string>
+  module M:impl_part;
+  import :interface_part;
+
+  std::string W = "World.";
+  void World() {
+    std::cout << W << std::endl;
+  }
+
+  // Impl.cpp
+  module;
+  #include <iostream>
+  module M;
+  void Hello() {
+    std::cout << "Hello ";
+  }
+
+  // User.cpp
+  import M;
+  int main() {
+    Hello();
+    World();
+    return 0;
+  }
+
+Then we are able to compile the example by the following command:
+
+.. code-block:: console
+
+  # Precompiling the module
+  $ clang++ -std=c++20 interface_part.cppm --precompile -o M-interface_part.pcm
+  $ clang++ -std=c++20 impl_part.cppm --precompile -fprebuilt-module-path=. -o M-impl_part.pcm
+  $ clang++ -std=c++20 M.cppm --precompile -fprebuilt-module-path=. -o M.pcm
+  $ clang++ -std=c++20 Impl.cpp -fmodule-file=M.pcm -c -o Impl.o
+
+  # Compiling the user
+  $ clang++ -std=c++20 User.cpp -fprebuilt-module-path=. -c -o User.o
+
+  # Compiling the module and linking it together
+  $ clang++ -std=c++20 M-interface_part.pcm -c -o M-interface_part.o
+  $ clang++ -std=c++20 M-impl_part.pcm -c -o M-impl_part.o
+  $ clang++ -std=c++20 M.pcm -c -o M.o
+  $ clang++ User.o M-interface_part.o  M-impl_part.o M.o Impl.o -o a.out
+
+We explain the options in the following sections.
+
+How to enable standard C++ modules
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Currently, standard C++ modules are enabled automatically
+if the language standard is ``-std=c++20`` or newer.
+The ``-fmodules-ts`` option is deprecated and is planned to be removed.
+
+How to produce a BMI
+~~~~~~~~~~~~~~~~~~~~
+
+It is possible to generate a BMI for an importable module unit by specifying the ``--precompile`` option.
+
+File name requirement
+~~~~~~~~~~~~~~~~~~~~~
+
+The file name of an ``importable module unit`` must end with ``.cppm``
+(or ``.ccm``, ``.cxxm``, ``.c++m``). The file name of a ``module implementation unit``
+should end with ``.cpp`` (or ``.cc``, ``.cxx``, ``.c++``).
+
+The file name of BMIs should end with ``.pcm``.
+The file name of the BMI of a ``primary module interface unit`` should be ``module_name.pcm``.
+The file name of BMIs of ``module partition unit`` should be ``module_name-partition_name.pcm``.
+
+If the file names use different extensions, Clang may fail to build the module.
+For example, if the filename of an ``importable module unit`` ends with ``.cpp`` instead of ``.cppm``,
+then we can't generate a BMI for the ``importable module unit`` by ``--precompile`` option
+since ``--precompile`` option now would only run preprocessor, which is equal to `-E` now.
+If we still want the filename of an ``importable module unit`` ends with ``.cpp`` instead of ``.cppm``,
+we could put ``-x c++-module`` in front of the file. For example,
+
+.. code-block:: c++
+
+  // Hello.cpp
+  module;
+  #include <iostream>
+  export module Hello;
+  export void hello() {
+    std::cout << "Hello World!\n";
+  }
+
+  // use.cpp
+  import Hello;
+  int main() {
+    hello();
+    return 0;
+  }
+
+Now the filename of the ``module interface`` ends with ``.cpp`` instead of ``.cppm``,
+we can't compile them by the original command lines. But we are still able to do it by:
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 -x c++-module Hello.cpp --precompile -o Hello.pcm
+  $ clang++ -std=c++20 use.cpp -fprebuilt-module-path=. Hello.pcm -o Hello.out
+  $ ./Hello.out
+  Hello World!
+
+How to specify the dependent BMIs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The option ``-fprebuilt-module-interface`` tells the compiler the path where to search for dependent BMIs.
+It may be used multiple times just like ``-I`` for specifying paths for header files. The look up rule here is:
+
+* (1) When we import module M. The compiler would look up M.pcm in the directories specified
+  by ``-fprebuilt-module-interface``.
+* (2) When we import partition module unit M:P. The compiler would look up M-P.pcm in the
+  directories specified by ``-fprebuilt-module-interface``.
+
+Another way to specify the dependent BMIs is to use ``-fmodule-file``. The main difference
+is that ``-fprebuilt-module-interface`` takes a directory, whereas ``-fmodule-file`` requires a
+specific file. In case both the ``-fprebuilt-module-interface`` and ``-fmodule-file`` exist, the 
+``-fmodule-file`` option takes higher precedence. In another word, if the compiler finds the wanted
+BMI specified by ``-fmodule-file``, the compiler wouldn't look up again in the directories specified
+by ``-fprebuilt-module-interface``.
+
+When we compile a ``module implementation unit``, we must pass the BMI of the corresponding
+``primary module interface unit`` by ``-fmodule-file``
+since the language specification says a module implementation unit implicitly imports
+the primary module interface unit.
+
+  [module.unit]p8
+
+  A module-declaration that contains neither an export-keyword nor a module-partition implicitly
+  imports the primary module interface unit of the module as if by a module-import-declaration.
+
+Again, the option ``-fmodule-file`` may occur multiple times.
+For example, the command line to compile ``M.cppm`` in
+the above example could be rewritten into:
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 M.cppm --precompile -fmodule-file=M-interface_part.pcm -fmodule-file=M-impl_part.pcm -o M.pcm
+
+``-fprebuilt-module-interface`` is more convenient and ``-fmodule-file`` is faster since
+it saves time for file lookup.
+
+Remember that module units still have an object counterpart to the BMI
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+It is easy to forget to compile BMIs at first since we may envision module interfaces like headers.
+However, this is not true.
+Module units are translation units. We need to compile them to object files
+and link the object files like the example shows.
+
+For example, the traditional compilation processes for headers are like:
+
+.. code-block:: text
+
+  src1.cpp -+> clang++ src1.cpp --> src1.o ---, 
+  hdr1.h  --'                                 +-> clang++ src1.o src2.o ->  executable
+  hdr2.h  --,                                 |
+  src2.cpp -+> clang++ src2.cpp --> src2.o ---'
+
+And the compilation process for module units are like:
+
+.. code-block:: text
+
+                src1.cpp ----------------------------------------+> clang++ src1.cpp -------> src1.o -, 
+  (header unit) hdr1.h    -> clang++ hdr1.h ...    -> hdr1.pcm --'                                    +-> clang++ src1.o mod1.o src2.o ->  executable
+                mod1.cppm -> clang++ mod1.cppm ... -> mod1.pcm --,--> clang++ mod1.pcm ... -> mod1.o -+
+                src2.cpp ----------------------------------------+> clang++ src2.cpp -------> src2.o -'
+
+As the diagrams show, we need to compile the BMI from module units to object files and link the object files.
+(But we can't do this for the BMI from header units. See the later section for the definition of header units)
+
+If we want to create a module library, we can't just ship the BMIs in an archive.
+We must compile these BMIs(``*.pcm``) into object files(``*.o``) and add those object files to the archive instead.
+
+Consistency Requirement
+~~~~~~~~~~~~~~~~~~~~~~~
+
+If we envision modules as a cache to speed up compilation, then - as with other caching techniques -
+it is important to keep cache consistency.
+So **currently** Clang will do very strict check for consistency.
+
+Options consistency
+^^^^^^^^^^^^^^^^^^^
+
+The language option of module units and their non-module-unit users should be consistent.
+The following example is not allowed:
+
+.. code-block:: c++
+
+  // M.cppm
+  export module M;
+  
+  // Use.cpp
+  import M;
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 M.cppm --precompile -o M.pcm
+  $ clang++ -std=c++2b Use.cpp -fprebuilt-module-path=.
+
+The compiler would reject the example due to the inconsistent language options.
+Not all options are language options.
+For example, the following example is allowed:
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 M.cppm --precompile -o M.pcm
+  # Inconsistent optimization level.
+  $ clang++ -std=c++20 -O3 Use.cpp -fprebuilt-module-path=.
+  # Inconsistent debugging level.
+  $ clang++ -std=c++20 -g Use.cpp -fprebuilt-module-path=. 
+
+Although the two examples have inconsistent optimization and debugging level, both of them are accepted.
+
+Note that **currently** the compiler doesn't consider inconsistent macro definition a problem. For example:
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 M.cppm --precompile -o M.pcm
+  # Inconsistent optimization level.
+  $ clang++ -std=c++20 -O3 -DNDEBUG Use.cpp -fprebuilt-module-path=.
+
+Currently Clang would accept the above example. But it may produce surprising results if the
+debugging code depends on consistent use of ``NDEBUG`` also in other translation units.
+
+Source content consistency
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When the compiler reads a BMI, the compiler will check the consistency of the corresponding
+source files. For example:
+
+.. code-block:: c++
+
+  // M.cppm
+  export module M;
+  export template <class T>
+  T foo(T t) {
+    return t;
+  }
+
+  // Use.cpp
+  import M;
+  void bar() {
+    foo(5);
+  }
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 M.cppm --precompile -o M.pcm
+  $ rm M.cppm
+  $ clang++ -std=c++20 Use.cpp -fmodule-file=M.pcm
+
+The compiler would reject the example since the compiler failed to find the source file to check the consistency.
+So the following example would be rejected too.
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 M.cppm --precompile -o M.pcm
+  $ echo "int i=0;" >> M.cppm
+  $ clang++ -std=c++20 Use.cpp -fmodule-file=M.pcm
+
+The compiler would reject it too since the compiler detected the file was changed.
+
+But it is OK to move the BMI as long as the source files remain:
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 M.cppm --precompile -o M.pcm
+  $ mkdir -p tmp
+  $ mv M.pcm tmp/M.pcm
+  $ clang++ -std=c++20 Use.cpp -fmodule-file=tmp/M.pcm
+
+The above example would be accepted.
+
+If the user doesn't want to follow the consistency requirement due to some reasons (e.g., distributing BMI),
+the user could try to use ``-XClang -fmodules-embed-all-files`` when producing BMI. For example:
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 M.cppm --precompile -XClang -fmodules-embed-all-files -o M.pcm
+  $ rm M.cppm
+  $ clang++ -std=c++20 Use.cpp -fmodule-file=M.pcm
+
+Now the compiler would accept the above example.
+Important note: XClang options are intended to be used by compiler internally and its semantics
+are not guaranteed to be preserved in future versions.
+
+ABI Impacts
+-----------
+
+The declarations in a module unit which are not in the global module fragment have new linkage names.
+
+For example,
+
+.. code-block:: c++
+
+  export module M;
+  namespace NS {
+    export int foo();
+  }
+
+The linkage name of ``NS::foo()`` would be ``_ZN2NSW1M3fooEv``.
+This couldn't be demangled by previous versions of the debugger or demangler.
+As of LLVM 15.x, users can utilize ``llvm-cxxfilt`` to demangle this:
+
+.. code-block:: console
+
+  $ llvm-cxxfilt _ZN2NSW1M3fooEv
+
+The result would be ``NS::foo@M()``, which reads as ``NS::foo()`` in module ``M``.
+
+The ABI implies that we can't declare something in a module unit and define it in a non-module unit (or vice-versa),
+as this would result in linking errors.
+
+Known Problems
+--------------
+
+The following describes issues in the current implementation of modules.
+Please see https://github.com/llvm/llvm-project/labels/clang%3Amodules for more issues
+or file a new issue if you don't find an existing one.
+If you're going to create a new issue for standard C++ modules,
+please start the title with ``[C++20] [Modules]`` (or ``[C++2b] [Modules]``, etc)
+and add the label ``clang:modules`` (if you have permissions for that).
+
+For higher level support for proposals, you could visit https://clang.llvm.org/cxx_status.html.
+
+Support for clang-scan-deps
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The support for clang-scan-deps may be the most urgent problem for modules now.
+Without the support for clang-scan-deps, it's hard to involve build systems.
+This means that users could only play with modules through makefiles or by writing a parser by hand.
+It blocks more uses for modules, which will block more defect reports or requirements.
+
+This is tracked in: https://github.com/llvm/llvm-project/issues/51792.
+
+Ambiguous deduction guide
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Currently, when we call deduction guides in global module fragment,
+we may get incorrect diagnosing message like: `ambiguous deduction`.
+
+So if we're using deduction guide from global module fragment, we probably need to write:
+
+.. code-block:: c++
+
+  std::lock_guard<std::mutex> lk(mutex);
+
+instead of
+
+.. code-block:: c++
+
+  std::lock_guard lk(mutex);
+
+This is tracked in: https://github.com/llvm/llvm-project/issues/56916
+
+Ignored PreferredName Attribute
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Due to a tricky problem, when Clang writes BMIs, Clang will ignore the ``preferred_name`` attribute, if any.
+This implies that the ``preferred_name`` wouldn't show in debugger or dumping.
+
+This is tracked in: https://github.com/llvm/llvm-project/issues/56490
+
+Don't emit macros about module declaration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This is covered by P1857R3. We mention it again here since users may abuse it before we implement it.
+
+Someone may want to write code which could be compiled both by modules or non-modules.
+A direct idea would be use macros like:
+
+.. code-block:: c++
+
+  MODULE
+  IMPORT header_name
+  EXPORT_MODULE MODULE_NAME;
+  IMPORT header_name
+  EXPORT ...
+
+So this file could be triggered like a module unit or a non-module unit depending on the definition
+of some macros.
+However, this kind of usage is forbidden by P1857R3 but we haven't implemented P1857R3 yet.
+This means that is possible to write illegal modules code now, and obviously this will stop working
+once P1857R3 is implemented.
+A simple suggestion would be "Don't play macro tricks with module declarations".
+
+This is tracked in: https://github.com/llvm/llvm-project/issues/56917
+
+Header Units
+============
+
+How to build projects using header unit
+---------------------------------------
+
+Quick Start
+~~~~~~~~~~~
+
+For the following example,
+
+.. code-block:: c++
+
+  import <iostream>;
+  int main() {
+    std::cout << "Hello World.\n";
+  }
+
+we could compile it as
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 -xc++-system-header --precompile iostream -o iostream.pcm
+  $ clang++ -std=c++20 -fmodule-file=iostream.pcm main.cpp
+
+How to produce BMIs
+~~~~~~~~~~~~~~~~~~~
+
+Similar to named modules, we could use ``--precompile`` to produce the BMI.
+But we need to specify that the input file is a header by ``-xc++-system-header`` or ``-xc++-user-header``.
+
+Also we could use `-fmodule-header={user,system}` option to produce the BMI for header units
+which has suffix like `.h` or `.hh`.
+The value of `-fmodule-header` means the user search path or the system search path.
+The default value for `-fmodule-header` is `user`.
+For example,
+
+.. code-block:: c++
+
+  // foo.h
+  #include <iostream>
+  void Hello() {
+    std::cout << "Hello World.\n";
+  }
+
+  // use.cpp
+  import "foo.h";
+  int main() {
+    Hello();
+  }
+
+We could compile it as:
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 -fmodule-header foo.h -o foo.pcm
+  $ clang++ -std=c++20 -fmodule-file=foo.pcm use.cpp
+
+For headers which don't have a suffix, we need to pass ``-xc++-header``
+(or ``-xc++-system-header`` or ``-xc++-user-header``) to mark it as a header.
+For example,
+
+.. code-block:: c++
+
+  // use.cpp
+  import "foo.h";
+  int main() {
+    Hello();
+  }
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 -fmodule-header=system -xc++-header iostream -o iostream.pcm
+  $ clang++ -std=c++20 -fmodule-file=iostream.pcm use.cpp
+
+How to specify the dependent BMIs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+We could use ``-fmodule-file`` to specify the BMIs, and this option may occur multiple times as well.
+
+With the existing implementation ``-fprebuilt-module-path`` cannot be used for header units
+(since they are nominally anonymous).
+For header units, use  ``-fmodule-file`` to include the relevant PCM file for each header unit.
+
+This is expect to be solved in future editions of the compiler either by the tooling finding and specifying
+the -fmodule-file or by the use of a module-mapper that understands how to map the header name to their PCMs.
+
+Don't compile the BMI
+~~~~~~~~~~~~~~~~~~~~~
+
+Another difference with modules is that we can't compile the BMI from a header unit.
+For example:
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 -xc++-system-header --precompile iostream -o iostream.pcm
+  # This is not allowed!
+  $ clang++ iostream.pcm -c -o iostream.o
+
+It makes sense due to the semantics of header units, which are just like headers.
+
+Include translation
+~~~~~~~~~~~~~~~~~~~
+
+The C++ spec allows the vendors to convert ``#include header-name`` to ``import header-name;`` when possible.
+Currently, Clang would do this translation for the ``#include`` in the global module fragment.
+
+For example, the following two examples are the same:
+
+.. code-block:: c++
+
+  module;
+  import <iostream>;
+  export module M;
+  export void Hello() {
+    std::cout << "Hello.\n";
+  }
+
+with the following one:
+
+.. code-block:: c++
+
+  module;
+  #include <iostream>
+  export module M;
+  export void Hello() {
+      std::cout << "Hello.\n";
+  }
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 -xc++-system-header --precompile iostream -o iostream.pcm
+  $ clang++ -std=c++20 -fmodule-file=iostream.pcm --precompile M.cppm -o M.cpp
+
+In the latter example, the Clang could find the BMI for the ``<iostream>``
+so it would try to replace the ``#include <iostream>`` to ``import <iostream>;`` automatically.
+
+
+Relationships between Clang modules
+-----------------------------------
+
+Header units have pretty similar semantics with Clang modules.
+The semantics of both of them are like headers.
+
+In fact, we could even "mimic" the sytle of header units by Clang modules:
+
+.. code-block:: c++
+
+  module "iostream" {
+    export *
+    header "/path/to/libstdcxx/iostream"
+  }
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 -fimplicit-modules -fmodule-map-file=.modulemap main.cpp
+
+It would be simpler if we are using libcxx:
+
+.. code-block:: console
+
+  $ clang++ -std=c++20 main.cpp -fimplicit-modules -fimplicit-module-maps 
+
+Since there is already one
+`module map <https://github.com/llvm/llvm-project/blob/main/libcxx/include/module.modulemap.in>`_
+in the source of libcxx.
+
+Then immediately leads to the question: why don't we implement header units through Clang header modules?
+
+The main reason for this is that Clang modules have more semantics like hierarchy or
+wrapping multiple headers together as a big module.
+However, these things are not part of Standard C++ Header units,
+and we want to avoid the impression that these additional semantics get interpreted as Standard C++ behavior.
+
+Another reason is that there are proposals to introduce module mappers to the C++ standard
+(for example, https://wg21.link/p1184r2).
+If we decide to reuse Clang's modulemap, we may get in trouble once we need to introduce another module mapper.
+
+So the final answer for why we don't reuse the interface of Clang modules for header units is that
+there are some differences between header units and Clang modules and that ignoring those
+differences now would likely become a problem in the future.
+
+Possible Questions
+==================
+
+How module speed up compilation
+-------------------------------
+
+A classic theory for the reason why modules speed up the compilation is:
+if there are ``n`` headers and ``m`` source files and each header is included by each source file,
+then the complexity of the compilation is ``O(n*m)``;
+But if there are ``n`` module interfaces and ``m`` source files, the complexity of the compilation is
+``O(n+m)``. So, using modules would be a big win when scaling.
+In a simpler word, we could get rid of many redundant compilations by using modules.
+
+Roughly, this theory is correct. But the problem is that it is too rough.
+The behavior depends on the optimization level, as we will illustrate below.
+
+First is ``O0``. The compilation process is described in the following graph.
+
+.. code-block:: none
+
+  ├-------------frontend----------┼-------------middle end----------------┼----backend----┤
+  │                               │                                       │               │
+  └---parsing----sema----codegen--┴----- transformations ---- codegen ----┴---- codegen --┘
+
+  ┌---------------------------------------------------------------------------------------┐
+  |                                                                                       │
+  |                                     source file                                       │
+  |                                                                                       │
+  └---------------------------------------------------------------------------------------┘
+
+              ┌--------┐
+              │        │
+              │imported│
+              │        │
+              │  code  │
+              │        │
+              └--------┘
+
+Here we can see that the source file (could be a non-module unit or a module unit) would get processed by the
+whole pipeline.
+But the imported code would only get involved in semantic analysis, which is mainly about name lookup,
+overload resolution and template instantiation.
+All of these processes are fast relative to the whole compilation process.
+More importantly, the imported code only needs to be processed once in frontend code generation,
+as well as the whole middle end and backend.
+So we could get a big win for the compilation time in O0.
+
+But with optimizations, things are different:
+
+(we omit ``code generation`` part for each end due to the limited space) 
+
+.. code-block:: none
+
+  ├-------- frontend ---------┼--------------- middle end --------------------┼------ backend ----┤
+  │                           │                                               │                   │
+  └--- parsing ---- sema -----┴--- optimizations --- IPO ---- optimizations---┴--- optimizations -┘
+                                                                                                            
+  ┌-----------------------------------------------------------------------------------------------┐
+  │                                                                                               │
+  │                                         source file                                           │
+  │                                                                                               │
+  └-----------------------------------------------------------------------------------------------┘
+                ┌---------------------------------------┐
+                │                                       │
+                │                                       │
+                │            imported code              │
+                │                                       │
+                │                                       │
+                └---------------------------------------┘
+
+It would be very unfortunate if we end up with worse performance after using modules.
+The main concern is that when we compile a source file, the compiler needs to see the function body
+of imported module units so that it can perform IPO (InterProcedural Optimization, primarily inlining
+in practice) to optimize functions in current source file with the help of the information provided by
+the imported module units.
+In other words, the imported code would be processed again and again in importee units
+by optimizations (including IPO itself).
+The optimizations before IPO and the IPO itself are the most time-consuming part in whole compilation process.
+So from this perspective, we might not be able to get the improvements described in the theory.
+But we could still save the time for optimizations after IPO and the whole backend.
+
+Overall, at ``O0`` the implementations of functions defined in a module will not impact module users,
+but at higher optimization levels the definitions of such functions are provided to user compilations for the
+purposes of optimization (but definitions of these functions are still not included in the use's object file)-
+this means the build speedup at higher optimization levels may be lower than expected given ``O0`` experience, 
+but does provide by more optimization opportunities.
+

clang/docs/index.rst

.. toctree::
    SanitizerCoverage
    SanitizerStats
    SanitizerSpecialCaseList
    ControlFlowIntegrity
    LTOVisibility
    SafeStack
    ShadowCallStack
    SourceBasedCodeCoverage
+   StandardCPlusPlusModules
    Modules
    MSVCCompatibility
    MisExpect
    OpenCLSupport
    OpenMPSupport
    SYCLSupport
    HLSL/HLSLDocs
    ThinLTO