diff --git a/clang/docs/Modules.rst b/clang/docs/Modules.rst --- a/clang/docs/Modules.rst +++ b/clang/docs/Modules.rst @@ -225,6 +225,100 @@ ``-fprebuilt-module-path=`` Specify the path to the prebuilt modules. If specified, we will look for modules in this directory for a given top-level module name. We don't need a module map for loading prebuilt modules in this directory and the compiler will not try to rebuild these modules. This can be specified multiple times. +``-fprebuilt-implicit-modules`` + Enable prebuilt implicit modules. If a prebuilt module is not found in the + prebuilt modules paths (specified via ``-fprebuilt-module-path``), we will + look for a matching implicit modules in the prebuilt modules paths. + +Using Prebuilt Modules +---------------------- + +Below are a few examples illustrating uses of prebuilt modules via the different options. + +First, let's set up files for our examples. + +.. code-block:: c + + /* A.h */ + // empty + +.. code-block:: c + + /* B.h */ + #include "A.h" + +.. code-block:: c + + /* use.c */ + #include "B.h" + +.. code-block:: c + + /* module.modulemap */ + module A { + header "A.h" + } + module B { + header "B.h" + } + +In examples below, the compilation of ``use.c`` can be done without ``-cc1``, but the commands used to prebuild the modules would need to be updated to take into account the default options passed to ``clang -cc1``. (See ``clang use.c -v``) + +First let us use an explicit mapping from modules to files. + +.. code-block:: sh + + rm -rf prebuilt ; mkdir prebuilt + clang -cc1 -x c module.modulemap -fmodules -emit-module -fmodule-name=A -o prebuilt/A.pcm + clang -cc1 -x c module.modulemap -fmodules -emit-module -fmodule-name=B -o prebuilt/B.pcm -fmodule-file=A=prebuilt/A.pcm + clang -cc1 -x c use.c -fmodules -o use.o -fmodule-file=A=prebuilt/A.pcm -fmodule-file=B=prebuilt/B.pcm + +Instead of of specifying the mappings manually, it can be convenient to use the ``-fprebuilt-module-path`` option. + +.. code-block:: sh + + rm -rf prebuilt ; mkdir prebuilt + clang -cc1 -x c module.modulemap -fmodules -emit-module -fmodule-name=A -o prebuilt/A.pcm + clang -cc1 -x c module.modulemap -fmodules -emit-module -fmodule-name=B -o prebuilt/B.pcm -fmodule-file=A=prebuilt/A.pcm + clang -cc1 -x c use.c -fmodules -o use.o -fprebuilt-module-path=prebuilt + +A trick to prebuilt required modules in one command is to generate implicit modules using the ``-fdisable-module-hash`` option. + +.. code-block:: sh + + rm -rf prebuilt ; mkdir prebuilt + clang -cc1 -x c module.modulemap -fmodules -emit-module -fmodule-name=B -fmodules-cache-path=prebuilt -fdisable-module-hash + ls prebuilt/*.pcm + # prebuilt/A.pcm prebuilt/B.pcm + clang -cc1 -x c use.c -fmodules -fprebuilt-module-path=prebuilt -o /use.o + +Maintaining multiple versions of prebuilt modules requires using a different mapping or pointing to a different prebuilt modules cache path. For example: + +.. code-block:: sh + + rm -rf prebuilt_1 ; rm -rf prebuilt_2 ; mkdir prebuilt_1 ; mkdir prebuilt_2 + clang -cc1 -x c module.modulemap -fmodules -emit-module -fmodule-name=B -fmodules-cache-path=prebuilt_1 -D_CONFIG_1 + clang -cc1 -x c module.modulemap -fmodules -emit-module -fmodule-name=B -fmodules-cache-path=prebuilt_2 -D_CONFIG_2 + clang -cc1 -x c use.c -o use.o -fmodules -fprebuilt-module-path=prebuilt_1 -DCONFIG_1 + clang -cc1 -x c use.c -o use.o -fmodules -fprebuilt-module-path=prebuilt_2 -DCONFIG_2 + +Instead, one can build implicit modules in a given cache path (using +``-fmodules-cache-path``, and reuse them as prebuilt implicit modules by +passing ``-fprebuilt-module-path`` and ``-fprebuilt-implicit-modules``. + +.. code-block:: sh + + rm -rf prebuilt; mkdir prebuilt + clang -cc1 -x c module.modulemap -fmodules -emit-module -fmodule-name=B -fmodules-cache-path=prebuilt -D_CONFIG_1 + clang -cc1 -x c module.modulemap -fmodules -emit-module -fmodule-name=B -fmodules-cache-path=prebuilt -D_CONFIG_2 + find prebuilt -name "*.pcm" + # prebuilt/W7OPMD3I127H/A-285BFOGQN82QX.pcm + # prebuilt/W7OPMD3I127H/B-285BFOGQN82QX.pcm + # prebuilt/Y4JCOK22SBQC/A-285BFOGQN82QX.pcm + # prebuilt/Y4JCOK22SBQC/B-285BFOGQN82QX.pcm + clang -cc1 -x c use.c -o /use.o -fmodules -fmodules-cache-path=cache -fprebuilt-module-path=prebuilt -fprebuilt-implicit-modules -D_CONFIG_1 + clang -cc1 -x c use.c -o /use.o -fmodules -fmodules-cache-path=cache -fprebuilt-module-path=prebuilt -fprebuilt-implicit-modules -D_CONFIG_2 + Module Semantics ================ diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td --- a/clang/include/clang/Driver/Options.td +++ b/clang/include/clang/Driver/Options.td @@ -1341,6 +1341,9 @@ def fprebuilt_module_path : Joined<["-"], "fprebuilt-module-path=">, Group, Flags<[DriverOption, CC1Option]>, MetaVarName<"">, HelpText<"Specify the prebuilt module path">; +def fprebuilt_implicit_modules : Flag <["-"], "fprebuilt-implicit-modules">, Group, + Flags<[DriverOption, CC1Option]>, + HelpText<"Look up implicit modules in the prebuilt module path">; def fmodules_prune_interval : Joined<["-"], "fmodules-prune-interval=">, Group, Flags<[CC1Option]>, MetaVarName<"">, HelpText<"Specify the interval (in seconds) between attempts to prune the module cache">; diff --git a/clang/include/clang/Lex/HeaderSearch.h b/clang/include/clang/Lex/HeaderSearch.h --- a/clang/include/clang/Lex/HeaderSearch.h +++ b/clang/include/clang/Lex/HeaderSearch.h @@ -505,6 +505,15 @@ std::string getPrebuiltModuleFileName(StringRef ModuleName, bool FileMapOnly = false); + /// Retrieve the name of the prebuilt module file that should be used + /// to load the given module. + /// + /// \param Module The module whose module file name will be returned. + /// + /// \returns The name of the module file that corresponds to this module, + /// or an empty string if this module does not correspond to any module file. + std::string getPrebuiltImplicitModuleFileName(Module *Module); + /// Retrieve the name of the (to-be-)cached module file that should /// be used to load a module with the given name. /// @@ -601,6 +610,22 @@ Module *lookupModule(StringRef ModuleName, StringRef SearchName, bool AllowExtraModuleMapSearch = false); + /// Retrieve the name of the (to-be-)cached module file that should + /// be used to load a module with the given name. + /// + /// \param ModuleName The module whose module file name will be returned. + /// + /// \param ModuleMapPath A path that when combined with \c ModuleName + /// uniquely identifies this module. See Module::ModuleMap. + /// + /// \param CachePath A path to the module cache. + /// + /// \returns The name of the module file that corresponds to this module, + /// or an empty string if this module does not correspond to any module file. + std::string getCachedModuleFileNameImpl(StringRef ModuleName, + StringRef ModuleMapPath, + StringRef CachePath); + /// Retrieve a module with the given name, which may be part of the /// given framework. /// diff --git a/clang/include/clang/Lex/HeaderSearchOptions.h b/clang/include/clang/Lex/HeaderSearchOptions.h --- a/clang/include/clang/Lex/HeaderSearchOptions.h +++ b/clang/include/clang/Lex/HeaderSearchOptions.h @@ -141,6 +141,10 @@ /// file. unsigned ModuleMapFileHomeIsCwd : 1; + /// Also search for prebuilt implicit modules in the prebuilt module cache + /// path. + unsigned EnablePrebuiltImplicitModules : 1; + /// The interval (in seconds) between pruning operations. /// /// This operation is expensive, because it requires Clang to walk through @@ -205,8 +209,9 @@ HeaderSearchOptions(StringRef _Sysroot = "/") : Sysroot(_Sysroot), ModuleFormat("raw"), DisableModuleHash(false), ImplicitModuleMaps(false), ModuleMapFileHomeIsCwd(false), - UseBuiltinIncludes(true), UseStandardSystemIncludes(true), - UseStandardCXXIncludes(true), UseLibcxx(false), Verbose(false), + EnablePrebuiltImplicitModules(false), UseBuiltinIncludes(true), + UseStandardSystemIncludes(true), UseStandardCXXIncludes(true), + UseLibcxx(false), Verbose(false), ModulesValidateOncePerBuildSession(false), ModulesValidateSystemHeaders(false), UseDebugInfo(false), ModulesValidateDiagnosticOptions(true), ModulesHashContent(false) {} diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp --- a/clang/lib/Driver/ToolChains/Clang.cpp +++ b/clang/lib/Driver/ToolChains/Clang.cpp @@ -2777,6 +2777,8 @@ std::string("-fprebuilt-module-path=") + A->getValue())); A->claim(); } + if (Args.hasFlag(options::OPT_fprebuilt_implicit_modules, false)) + CmdArgs.push_back("-fprebuilt-implicit-modules"); } // -fmodule-name specifies the module that is currently being built (or diff --git a/clang/lib/Frontend/CompilerInstance.cpp b/clang/lib/Frontend/CompilerInstance.cpp --- a/clang/lib/Frontend/CompilerInstance.cpp +++ b/clang/lib/Frontend/CompilerInstance.cpp @@ -1673,6 +1673,9 @@ !HSOpts.PrebuiltModulePaths.empty())) { ModuleFileName = PP->getHeaderSearchInfo().getPrebuiltModuleFileName(ModuleName); + if (HSOpts.EnablePrebuiltImplicitModules && ModuleFileName.empty()) + ModuleFileName = + PP->getHeaderSearchInfo().getPrebuiltImplicitModuleFileName(Module); if (!ModuleFileName.empty()) Source = PrebuiltModulePath; } diff --git a/clang/lib/Frontend/CompilerInvocation.cpp b/clang/lib/Frontend/CompilerInvocation.cpp --- a/clang/lib/Frontend/CompilerInvocation.cpp +++ b/clang/lib/Frontend/CompilerInvocation.cpp @@ -2069,6 +2069,8 @@ !Args.hasArg(OPT_fmodules_disable_diagnostic_validation); Opts.ImplicitModuleMaps = Args.hasArg(OPT_fimplicit_module_maps); Opts.ModuleMapFileHomeIsCwd = Args.hasArg(OPT_fmodule_map_file_home_is_cwd); + Opts.EnablePrebuiltImplicitModules = + Args.hasArg(OPT_fprebuilt_implicit_modules); Opts.ModuleCachePruneInterval = getLastArgIntValue(Args, OPT_fmodules_prune_interval, 7 * 24 * 60 * 60); Opts.ModuleCachePruneAfter = diff --git a/clang/lib/Lex/HeaderSearch.cpp b/clang/lib/Lex/HeaderSearch.cpp --- a/clang/lib/Lex/HeaderSearch.cpp +++ b/clang/lib/Lex/HeaderSearch.cpp @@ -164,14 +164,41 @@ return {}; } +std::string HeaderSearch::getPrebuiltImplicitModuleFileName(Module *Module) { + const FileEntry *ModuleMap = + getModuleMap().getModuleMapFileForUniquing(Module); + StringRef ModuleName = Module->Name; + StringRef ModuleMapPath = ModuleMap->getName(); + StringRef ModuleCacheHash = HSOpts->DisableModuleHash + ? "" + : llvm::sys::path::stem(getModuleCachePath()); + for (const std::string &Dir : HSOpts->PrebuiltModulePaths) { + SmallString<256> CachePath(Dir); + llvm::sys::fs::make_absolute(CachePath); + llvm::sys::path::append(CachePath, ModuleCacheHash); + std::string FileName = + getCachedModuleFileNameImpl(ModuleName, ModuleMapPath, CachePath); + if (!FileName.empty() && getFileMgr().getFile(FileName)) + return FileName; + } + return {}; +} + std::string HeaderSearch::getCachedModuleFileName(StringRef ModuleName, StringRef ModuleMapPath) { + return getCachedModuleFileNameImpl(ModuleName, ModuleMapPath, + getModuleCachePath()); +} + +std::string HeaderSearch::getCachedModuleFileNameImpl(StringRef ModuleName, + StringRef ModuleMapPath, + StringRef CachePath) { // If we don't have a module cache path or aren't supposed to use one, we // can't do anything. - if (getModuleCachePath().empty()) + if (CachePath.empty()) return {}; - SmallString<256> Result(getModuleCachePath()); + SmallString<256> Result(CachePath); llvm::sys::fs::make_absolute(Result); if (HSOpts->DisableModuleHash) { diff --git a/clang/lib/Serialization/ASTReader.cpp b/clang/lib/Serialization/ASTReader.cpp --- a/clang/lib/Serialization/ASTReader.cpp +++ b/clang/lib/Serialization/ASTReader.cpp @@ -5726,6 +5726,7 @@ HSOpts.DisableModuleHash = Record[Idx++]; HSOpts.ImplicitModuleMaps = Record[Idx++]; HSOpts.ModuleMapFileHomeIsCwd = Record[Idx++]; + HSOpts.EnablePrebuiltImplicitModules = Record[Idx++]; HSOpts.UseBuiltinIncludes = Record[Idx++]; HSOpts.UseStandardSystemIncludes = Record[Idx++]; HSOpts.UseStandardCXXIncludes = Record[Idx++]; diff --git a/clang/lib/Serialization/ASTWriter.cpp b/clang/lib/Serialization/ASTWriter.cpp --- a/clang/lib/Serialization/ASTWriter.cpp +++ b/clang/lib/Serialization/ASTWriter.cpp @@ -1675,6 +1675,7 @@ Record.push_back(HSOpts.DisableModuleHash); Record.push_back(HSOpts.ImplicitModuleMaps); Record.push_back(HSOpts.ModuleMapFileHomeIsCwd); + Record.push_back(HSOpts.EnablePrebuiltImplicitModules); Record.push_back(HSOpts.UseBuiltinIncludes); Record.push_back(HSOpts.UseStandardSystemIncludes); Record.push_back(HSOpts.UseStandardCXXIncludes); diff --git a/clang/test/Modules/Inputs/prebuilt-implicit-module/a.h b/clang/test/Modules/Inputs/prebuilt-implicit-module/a.h new file mode 100644 --- /dev/null +++ b/clang/test/Modules/Inputs/prebuilt-implicit-module/a.h @@ -0,0 +1 @@ +const int a = 1; diff --git a/clang/test/Modules/Inputs/prebuilt-implicit-module/module.modulemap b/clang/test/Modules/Inputs/prebuilt-implicit-module/module.modulemap new file mode 100644 --- /dev/null +++ b/clang/test/Modules/Inputs/prebuilt-implicit-module/module.modulemap @@ -0,0 +1 @@ +module module_a { header "a.h" } diff --git a/clang/test/Modules/prebuilt-implicit-modules.m b/clang/test/Modules/prebuilt-implicit-modules.m new file mode 100644 --- /dev/null +++ b/clang/test/Modules/prebuilt-implicit-modules.m @@ -0,0 +1,27 @@ +// RUN: rm -rf %t +// RUN: %clang_cc1 -x objective-c -fmodules %S/Inputs/prebuilt-implicit-module/module.modulemap -emit-module -fmodule-name=module_a -fmodules-cache-path=%t +// RUN: find %t -name "module_a*.pcm" | grep module_a +// +// Check we use a prebuilt module when available, and do not build an implicit module. +// RUN: rm -rf %t1 +// RUN: mkdir -p %t1 +// RUN: %clang_cc1 -x objective-c %s -I%S/Inputs/prebuilt-implicit-module -fmodules -fmodule-map-file=%S/Inputs/prebuilt-implicit-module/module.modulemap -fprebuilt-implicit-modules -fprebuilt-module-path=%t -fmodules-cache-path=%t1 +// RUN: find %t1 -name "module_a*.pcm" | not grep module_e +// +// Check that we correctly fall back to implicit modules if the prebuilt implicit module is not found. +// RUN: %clang_cc1 -x objective-c %s -I%S/Inputs/prebuilt-implicit-module -fmodules -fmodule-map-file=%S/Inputs/prebuilt-implicit-module/module.modulemap -fprebuilt-implicit-modules -fprebuilt-module-path=%t -fmodules-cache-path=%t1 -fno-signed-char +// RUN: find %t1 -name "module_a*.pcm" | grep module_a + +// Check that non-implicit prebuilt modules are always preferred to prebuilt implicit modules. +// RUN: rm -rf %t2 +// RUN: mkdir -p %t2 +// RUN: %clang_cc1 -x objective-c -fmodules %S/Inputs/prebuilt-implicit-module/module.modulemap -emit-module -fmodule-name=module_a -fmodules-cache-path=%t +// RUN: %clang_cc1 -x objective-c -fmodules %S/Inputs/prebuilt-implicit-module/module.modulemap -emit-module -fmodule-name=module_a -o %t/module_a.pcm -fno-signed-char +// RUN: not %clang_cc1 -x objective-c %s -I%S/Inputs/prebuilt-implicit-module -fmodules -fmodule-map-file=%S/Inputs/prebuilt-implicit-module/module.modulemap -fprebuilt-implicit-modules -fprebuilt-module-path=%t -fmodules-cache-path=%t2 +// RUN: find %t2 -name "module_a*.pcm" | not grep module_a + +// expected-no-diagnostics +@import module_a; +int test() { + return a; +}