[C++20][Modules][Driver][HU 4/N] Add fdirectives-only mode for preprocessing output.

[iains]

Summary

When the -fdirectives-only option is used together with -E, the preprocessor
output reflects evaluation of if/then/else directives.

Thus it preserves macros that are still live after such processing.
This output can be consumed by a second compilation to produce a header unit.

We automatically invoke this (with -E) when we know that the job produces a
header unit so that the preprocessed output reflects the macros that will be
defined when the binary HU is emitted.

[ChuanqiXu]

Oh, I found all the series of patches looks confusing to me. Maybe due to that I lack a use experience with header unit. I really couldn't judge if it is necessary or useful. Could you elaborate more end use cases? For example, I could understand the sentence:

Thus it preserves macros that are still live after such processing.
This output can be consumed by a second compilation to produce a header unit.

But where could it be useful? I think this is worth mentioning.

---

Note: After the 2 series of patches, I think it is necessary to edit the Release Note and the command line documents.

clang/test/Driver/cxx20-fdirectives-only.cpp

11

Maybe it's worth to add a test:

%clang++ -### -std=c++20 -fmodule-header=user foo.hh  2>&1

And check not for -fdirectives-only.

[iains]

In D121591#3381348, @ChuanqiXu wrote:

Oh, I found all the series of patches looks confusing to me. Maybe due to that I lack a use experience with header unit. I really couldn't judge if it is necessary or useful. Could you elaborate more end use cases? For example, I could understand the sentence:

Thus it preserves macros that are still live after such processing.
This output can be consumed by a second compilation to produce a header unit.

But where could it be useful? I think this is worth mentioning.

OK - I could but the uses of pre-processed output are not really novel to header units?

as I noted elsewhere:

1. usually it is an aid to debugging / problem reporting (since the pre-processed output flattens included headers)
2. in the case of a header unit, actually there could be a purpose in scanning the pre-processed output to determine the dependencies in force after directives have been processed.

---

Note: After the 2 series of patches, I think it is necessary to edit the Release Note and the command line documents.

Ah good catch, I forgot that _ will do an additional patch,

[iains]

rebased.

[iains]

rebased, adjusted testcases to avoid using clang++.

clang/lib/Driver/Driver.cpp

case phases::Preprocess: {
     types::ID OutputTy;
     // -M and -MM specify the dependency file name by altering the output type,
     // -if -MD and -MMD are not specified.
     if (Args.hasArg(options::OPT_M, options::OPT_MM) &&
         !Args.hasArg(options::OPT_MD, options::OPT_MMD)) {
       OutputTy = types::TY_Dependencies;
     } else {
       OutputTy = Input->getType();
+      // For these cases, the preprocessor is only translating forms, the Output
+      // still needs preprocessing.
       if (!Args.hasFlag(options::OPT_frewrite_includes,
                         options::OPT_fno_rewrite_includes, false) &&
           !Args.hasFlag(options::OPT_frewrite_imports,
                         options::OPT_fno_rewrite_imports, false) &&
+          !Args.hasFlag(options::OPT_fdirectives_only,
+                        options::OPT_fno_directives_only, false) &&
           !CCGenDiagnostics)
         OutputTy = types::getPreprocessedType(OutputTy);
       assert(OutputTy != types::TY_INVALID &&
              "Cannot preprocess this input type!");
     }
     return C.MakeAction<PreprocessJobAction>(Input, OutputTy);
   }
   case phases::Precompile: {

clang/lib/Driver/ToolChains/Clang.cpp

void Clang::ConstructJob(Compilation &C, const JobAction &JA,
   } else if (isa<PreprocessJobAction>(JA)) {
     if (Output.getType() == types::TY_Dependencies)
       CmdArgs.push_back("-Eonly");
     else {
       CmdArgs.push_back("-E");
       if (Args.hasArg(options::OPT_rewrite_objc) &&
           !Args.hasArg(options::OPT_g_Group))
         CmdArgs.push_back("-P");
+      else if (JA.getType() == types::TY_PP_CXXHeaderUnit)
+        CmdArgs.push_back("-fdirectives-only");
     }
   } else if (isa<AssembleJobAction>(JA)) {
     CmdArgs.push_back("-emit-obj");
 
     CollectArgsForIntegratedAssembler(C, Args, CmdArgs, D);
 
     // Also ignore explicit -force_cpusubtype_ALL option.
     (void)Args.hasArg(options::OPT_force__cpusubtype__ALL);

   if (willEmitRemarks(Args) && checkRemarksOptions(D, Args, Triple))
     renderRemarksOptions(Args, CmdArgs, Triple, Input, Output, JA);
 
   bool RewriteImports = Args.hasFlag(options::OPT_frewrite_imports,
                                      options::OPT_fno_rewrite_imports, false);
   if (RewriteImports)
     CmdArgs.push_back("-frewrite-imports");
 
+  if (Args.hasFlag(options::OPT_fdirectives_only,
+                   options::OPT_fno_directives_only, false))
+    CmdArgs.push_back("-fdirectives-only");
+
   // Enable rewrite includes if the user's asked for it or if we're generating
   // diagnostics.
   // TODO: Once -module-dependency-dir works with -frewrite-includes it'd be
   // nice to enable this when doing a crashdump for modules as well.
   if (Args.hasFlag(options::OPT_frewrite_includes,
                    options::OPT_fno_rewrite_includes, false) ||
       (C.isForDiagnostics() && !HaveModules))
     CmdArgs.push_back("-frewrite-includes");

clang/test/Driver/cxx20-fdirectives-only.cpp

+// Test -fdirectives-only cases.
+
+// We can manullay specify fdirectives-only, for any pre-processor job.
+// RUN: %clang -### -std=c++20 -E -fdirectives-only foo.hh  2>&1 | \
+// RUN:   FileCheck -check-prefix=CHECK-NON-HU %s
+
+// Check that we automatically append -fdirectives-only for header-unit
+// preprocessor jobs.
+// RUN: %clang -### -std=c++20 -E -fmodule-header=user foo.hh  2>&1 | \
+// RUN:   FileCheck -check-prefix=CHECK-HU %s
+
+// CHECK-NON-HU: "-E"
+// CHECK-NON-HU-SAME: "-fdirectives-only"
+// CHECK-NON-HU-SAME: "-o" "-"
+// CHECK-NON-HU-SAME: "-x" "c++-header" "foo.hh"
+
+// CHECK-HU: "-E"
+// CHECK-HU-SAME: "-fdirectives-only"
+// CHECK-HU-SAME: "-o" "-"
+// CHECK-HU-SAME: "-x" "c++-user-header" "foo.hh"