Add a -gmodules option to the clang driver.

[aprantl]

Summary

This patch adds a -gmodules option to the driver and a -dwarf-ext-refs to cc1 to enable the use of external type references in the debug info (a.k.a. module debugging).

The driver expands -gmodules to "-g -fmodule-format=obj -dwarf-ext-refs" and passes that to cc1.
Most options that start with -g (e.g., -gdwarf-2) also turn on -g, and module requires object-container-wrapped modules, "-dwarf-ext-refs" been the actual low-level option for turning on external type references.

Rationale for the choice of names (and this is really all there is to review in this patch):
"-gmodules": is meant to pair nicely with "-fmodules"
"-dwarf-ext-refs": Fits into the naming scheme of similar options like "-dwarf-column-info" and "-dwarf-debug-flags". Spelling out the option "-dwarf-external-type-references" seemed to be overkill.

All this does at the moment is set a flag codegenopts. Having this flag in place is a prerequisite for emitting debug info into modules: The debug info for a module needs to use external type references for types defined in (other) modules or we would violate the minimal deserialization requirements (cf. test/PCH/check-deserializations.cpp).

[aprantl]

Ping.

[rsmith]

include/clang/Frontend/CodeGenOptions.def

164–165

Does this flag end up in the module hash? It presumably should do so.

lib/Driver/Tools.cpp

3708

What happens if we get a module cache hash collision between module files with different formats? This didn't really matter when the choice was essentially made per-platform, but if it's affected by a user-facing -g flag, we should make sure this works reliably. (You can probably test this by using -disable-module-hash and then using the same module with two different module formats specified.)

[aprantl]

Maybe this is missing the point, but a hash collision between module files with different formats should be highly unlikely because we add the module format into the hash.

llvm::hash_code Hash =
  llvm::hash_combine(DirName.lower(), FileName.lower(),
                     HSOpts->ModuleFormat);

Or do you mean a collision in spite of this?

• adrian

[aprantl]

include/clang/Frontend/CodeGenOptions.def

164–165

It does not get hashed:

• If the module has no debug info, then this flag is meaningless, because it only affects the debug info output.
• If the module has debug info, the module format is "obj" (hence a different hash) and this flag must be turned on. I will make sure this is enforced via an assertion.

[I don't see a point in having "obj" modules without debug info, If we want these, we'll need to add it to the hash.]

[aprantl]

Alright, let's hash it, then.

[aprantl]

This landed a long time ago as r246192.

docs/CommandGuide/clang.rst

Specify which optimization level to use:
     :option:`-O4` and higher
 
       Currently equivalent to :option:`-O3`
 
 .. option:: -g
 
   Generate debug information.  Note that Clang debug information works best at -O0.
 
+.. option:: -gmodules
+
+  Generate debug information that contains external references to
+  types defined in clang modules or precompiled headers instead of
+  emitting redundant debug type information into every object file.
+  This option implies `-fmodule-format=obj`.
+  
 .. option:: -fstandalone-debug -fno-standalone-debug
 
   Clang supports a number of optimizations to reduce the size of debug
   information in the binary. They work based on the assumption that the
   debug type information can be spread out over multiple compilation units.
   For instance, Clang will not emit type definitions for types that are not
   needed by a module and could be replaced with a forward declaration.
   Further, Clang will only emit type info for a dynamic C++ class in the

include/clang/Driver/CC1Options.td

 def dwarf_column_info : Flag<["-"], "dwarf-column-info">,
   HelpText<"Turn on column location information.">;
 def split_dwarf : Flag<["-"], "split-dwarf">,
   HelpText<"Split out the dwarf .dwo sections">;
 def gnu_pubnames : Flag<["-"], "gnu-pubnames">,
   HelpText<"Emit newer GNU style pubnames">;
 def arange_sections : Flag<["-"], "arange_sections">,
   HelpText<"Emit DWARF .debug_arange sections">;
+def dwarf_ext_refs : Flag<["-"], "dwarf-ext-refs">,
+  HelpText<"Generate debug info with external references to clang modules"
+           " or precompiled headers">;
 def fforbid_guard_variables : Flag<["-"], "fforbid-guard-variables">,
   HelpText<"Emit an error if a C++ static local initializer would need a guard variable">;
 def no_implicit_float : Flag<["-"], "no-implicit-float">,
   HelpText<"Don't generate implicit floating point instructions">;
 def fdump_vtable_layouts : Flag<["-"], "fdump-vtable-layouts">,
   HelpText<"Dump the layouts of all vtables that will be emitted in a translation unit">;
 def fmerge_functions : Flag<["-"], "fmerge-functions">,
   HelpText<"Permit merging of identical functions when optimizing.">;

include/clang/Driver/Options.td

def gno_record_gcc_switches : Flag<["-"], "gno-record-gcc-switches">,
   Group<g_flags_Group>;
 def gstrict_dwarf : Flag<["-"], "gstrict-dwarf">, Group<g_flags_Group>;
 def gno_strict_dwarf : Flag<["-"], "gno-strict-dwarf">, Group<g_flags_Group>;
 def gcolumn_info : Flag<["-"], "gcolumn-info">, Group<g_flags_Group>;
 def gno_column_info : Flag<["-"], "gno-column-info">, Group<g_flags_Group>;
 def gsplit_dwarf : Flag<["-"], "gsplit-dwarf">, Group<g_flags_Group>;
 def ggnu_pubnames : Flag<["-"], "ggnu-pubnames">, Group<g_flags_Group>;
 def gdwarf_aranges : Flag<["-"], "gdwarf-aranges">, Group<g_flags_Group>;
+def gmodules : Flag <["-"], "gmodules">, Group<f_Group>,
+  HelpText<"Generate debug info with external references to clang modules"
+           " or precompiled headers">;
 def headerpad__max__install__names : Joined<["-"], "headerpad_max_install_names">;
 def help : Flag<["-", "--"], "help">, Flags<[CC1Option,CC1AsOption]>,
   HelpText<"Display available options">;
 def index_header_map : Flag<["-"], "index-header-map">, Flags<[CC1Option]>,
   HelpText<"Make the next included directory (-I or -F) an indexer header map">;
 def idirafter : JoinedOrSeparate<["-"], "idirafter">, Group<clang_i_Group>, Flags<[CC1Option]>,
   HelpText<"Add directory to AFTER include search path">;
 def iframework : JoinedOrSeparate<["-"], "iframework">, Group<clang_i_Group>, Flags<[CC1Option]>,

include/clang/Frontend/CodeGenOptions.def

CODEGENOPT(UseInitArray      , 1, 0) ///< Control whether to use .init_array or
                                      ///< .ctors.
 VALUE_CODEGENOPT(StackAlignment    , 32, 0) ///< Overrides default stack 
                                             ///< alignment, if not 0.
 VALUE_CODEGENOPT(StackProbeSize    , 32, 4096) ///< Overrides default stack
                                                ///< probe size, even if 0.
 CODEGENOPT(DebugColumnInfo, 1, 0) ///< Whether or not to use column information
                                   ///< in debug info.
 
+CODEGENOPT(DebugTypeExtRefs, 1, 0) ///< Whether or not debug info should contain
+                                   ///< external references to a PCH or module.
+
 CODEGENOPT(EmitLLVMUseLists, 1, 0) ///< Control whether to serialize use-lists.
 
 /// The user specified number of registers to be used for integral arguments,
 /// or 0 if unspecified.
 VALUE_CODEGENOPT(NumRegisterParameters, 32, 0)
 
 /// The lower bound for a buffer to be considered for stack protection.
 VALUE_CODEGENOPT(SSPBufferSize, 32, 0)

lib/CodeGen/CGDebugInfo.h

 /// This class gathers all debug information during compilation and is
 /// responsible for emitting to llvm globals or pass directly to the
 /// backend.
 class CGDebugInfo {
   friend class ApplyDebugLocation;
   friend class SaveAndRestoreLocation;
   CodeGenModule &CGM;
   const CodeGenOptions::DebugInfoKind DebugKind;
+  bool DebugTypeExtRefs;
   llvm::DIBuilder DBuilder;
   llvm::DICompileUnit *TheCU = nullptr;
   SourceLocation CurLoc;
   llvm::DIType *VTablePtrType = nullptr;
   llvm::DIType *ClassTy = nullptr;
   llvm::DICompositeType *ObjTy = nullptr;
   llvm::DIType *SelTy = nullptr;
   llvm::DIType *OCLImage1dDITy = nullptr;

lib/CodeGen/CGDebugInfo.cpp

 #include "llvm/Support/Dwarf.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Path.h"
 using namespace clang;
 using namespace clang::CodeGen;
 
 CGDebugInfo::CGDebugInfo(CodeGenModule &CGM)
     : CGM(CGM), DebugKind(CGM.getCodeGenOpts().getDebugInfo()),
+      DebugTypeExtRefs(CGM.getCodeGenOpts().DebugTypeExtRefs),
       DBuilder(CGM.getModule()) {
   CreateCompileUnit();
 }
 
 CGDebugInfo::~CGDebugInfo() {
   assert(LexicalBlockStack.empty() &&
          "Region stack mismatch, stack not empty!");
 }

lib/CodeGen/ObjectFilePCHContainerOperations.cpp

PCHContainerGenerator(DiagnosticsEngine &diags,
                         raw_pwrite_stream *OS,
                         std::shared_ptr<PCHBuffer> Buffer)
       : Diags(diags), Ctx(nullptr), HeaderSearchOpts(HSO), PreprocessorOpts(PPO),
         TargetOpts(TO), LangOpts(LO), OS(OS), Buffer(Buffer) {
     // The debug info output isn't affected by CodeModel and
     // ThreadModel, but the backend expects them to be nonempty.
     CodeGenOpts.CodeModel = "default";
     CodeGenOpts.ThreadModel = "single";
+    CodeGenOpts.DebugTypeExtRefs = true;
     CodeGenOpts.setDebugInfo(CodeGenOptions::FullDebugInfo);
     CodeGenOpts.SplitDwarfFile = OutputFileName;
   }
 
   virtual ~PCHContainerGenerator() {}
 
   void Initialize(ASTContext &Context) override {
     if (Ctx) {

lib/Driver/Tools.cpp

#endif
 
   // We ignore flags -gstrict-dwarf and -grecord-gcc-switches for now.
   Args.ClaimAllArgs(options::OPT_g_flags_Group);
   if (Args.hasFlag(options::OPT_gcolumn_info, options::OPT_gno_column_info,
                    /*Default*/ true))
     CmdArgs.push_back("-dwarf-column-info");
 
   // FIXME: Move backend command line options to the module.
+  if (Args.hasArg(options::OPT_gmodules)) {
+    CmdArgs.push_back("-g");
+    CmdArgs.push_back("-dwarf-ext-refs");
+    CmdArgs.push_back("-fmodule-format=obj");
+  }
+
   // -gsplit-dwarf should turn on -g and enable the backend dwarf
   // splitting and extraction.
   // FIXME: Currently only works on Linux.
   if (getToolChain().getTriple().isOSLinux() && SplitDwarfArg) {
     CmdArgs.push_back("-g");
     CmdArgs.push_back("-backend-option");
     CmdArgs.push_back("-split-dwarf=Enable");
   }

lib/Frontend/CompilerInvocation.cpp

static bool ParseCodeGenArgs(CodeGenOptions &Opts, ArgList &Args, InputKind IK,
   }
   Opts.SplitDwarfFile = Args.getLastArgValue(OPT_split_dwarf_file);
   if (Args.hasArg(OPT_gdwarf_2))
     Opts.DwarfVersion = 2;
   else if (Args.hasArg(OPT_gdwarf_3))
     Opts.DwarfVersion = 3;
   else if (Args.hasArg(OPT_gdwarf_4))
     Opts.DwarfVersion = 4;
+  Opts.DebugTypeExtRefs = Args.hasArg(OPT_dwarf_ext_refs);
 
   if (const Arg *A =
           Args.getLastArg(OPT_emit_llvm_uselists, OPT_no_emit_llvm_uselists))
     Opts.EmitLLVMUseLists = A->getOption().getID() == OPT_emit_llvm_uselists;
 
   Opts.DisableLLVMOpts = Args.hasArg(OPT_disable_llvm_optzns);
   Opts.DisableLLVMPasses = Args.hasArg(OPT_disable_llvm_passes);
   Opts.DisableRedZone = Args.hasArg(OPT_disable_red_zone);

test/Driver/debug-options.c

 // RUN: %clang -### -fdebug-types-section -fno-debug-types-section %s 2>&1 \
 // RUN:        | FileCheck -check-prefix=NOFDTS %s
 //
 // RUN: %clang -### -g -gno-column-info %s 2>&1 \
 // RUN:        | FileCheck -check-prefix=NOCI %s
 //
 // RUN: %clang -### -g %s 2>&1 | FileCheck -check-prefix=CI %s
 //
+// RUN: %clang -### -gmodules %s 2>&1 \
+// RUN:        | FileCheck -check-prefix=GEXTREFS %s
+//
 // G: "-cc1"
 // G: "-g"
 //
 // G_DARWIN: "-cc1"
 // G_DARWIN: "-gdwarf-2"
 //
 // G_D2: "-cc1"
 // G_D2: "-gdwarf-2"

 //
 // FDTS: "-backend-option" "-generate-type-units"
 //
 // NOFDTS-NOT: "-backend-option" "-generate-type-units"
 //
 // CI: "-dwarf-column-info"
 //
 // NOCI-NOT: "-dwarf-column-info"
+//
+// GEXTREFS: "-g" "-dwarf-ext-refs" "-fmodule-format=obj"