diff --git a/clang/docs/Modules.rst b/clang/docs/Modules.rst --- a/clang/docs/Modules.rst +++ b/clang/docs/Modules.rst @@ -225,6 +225,11 @@ ``-fprebuilt-module-path=`` Specify the path to the prebuilt modules. If specified, we will look for modules in this directory for a given top-level module name. We don't need a module map for loading prebuilt modules in this directory and the compiler will not try to rebuild these modules. This can be specified multiple times. +``-fprebuilt-implicit-modules`` + Enable prebuilt implicit modules. If a prebuilt module is not found in the + prebuilt modules paths (specified via ``-fprebuilt-module-path``), we will + look for a matching implicit module in the prebuilt modules paths. + -cc1 Options ~~~~~~~~~~~~ @@ -235,6 +240,123 @@ being built if the command line arguments are not homogeneous across your build. +Using Prebuilt Modules +---------------------- + +Below are a few examples illustrating uses of prebuilt modules via the different options. + +First, let's set up files for our examples. + +.. code-block:: c + + /* A.h */ + #ifdef ENABLE_A + void a() {} + #endif + +.. code-block:: c + + /* B.h */ + #include "A.h" + +.. code-block:: c + + /* use.c */ + #include "B.h" + void use() { + #ifdef ENABLE_A + a(); + #endif + } + +.. code-block:: c + + /* module.modulemap */ + module A { + header "A.h" + } + module B { + header "B.h" + export * + } + +In the examples below, the compilation of ``use.c`` can be done without ``-cc1``, but the commands used to prebuild the modules would need to be updated to take into account the default options passed to ``clang -cc1``. (See ``clang use.c -v``) +Note also that, since we use ``-cc1``, we specify the ``-fmodule-map-file=`` or ``-fimplicit-module-maps`` options explicitly. When using the clang driver, ``-fimplicit-module-maps`` is implied by ``-fmodules``. + +First let us use an explicit mapping from modules to files. + +.. code-block:: sh + + rm -rf prebuilt ; mkdir prebuilt + clang -cc1 -emit-module -o prebuilt/A.pcm -fmodules module.modulemap -fmodule-name=A + clang -cc1 -emit-module -o prebuilt/B.pcm -fmodules module.modulemap -fmodule-name=B -fmodule-file=A=prebuilt/A.pcm + clang -cc1 -emit-obj use.c -fmodules -fmodule-map-file=module.modulemap -fmodule-file=A=prebuilt/A.pcm -fmodule-file=B=prebuilt/B.pcm + +Instead of of specifying the mappings manually, it can be convenient to use the ``-fprebuilt-module-path`` option. Let's also use ``-fimplicit-module-maps`` instead of manually pointing to our module map. + +.. code-block:: sh + + rm -rf prebuilt; mkdir prebuilt + clang -cc1 -emit-module -o prebuilt/A.pcm -fmodules module.modulemap -fmodule-name=A + clang -cc1 -emit-module -o prebuilt/B.pcm -fmodules module.modulemap -fmodule-name=B -fprebuilt-module-path=prebuilt + clang -cc1 -emit-obj use.c -fmodules -fimplicit-module-maps -fprebuilt-module-path=prebuilt + +A trick to prebuild all modules required for our source file in one command is to generate implicit modules while using the ``-fdisable-module-hash`` option. + +.. code-block:: sh + + rm -rf prebuilt ; mkdir prebuilt + clang -cc1 -emit-obj use.c -fmodules -fimplicit-module-maps -fmodules-cache-path=prebuilt -fdisable-module-hash + ls prebuilt/*.pcm + # prebuilt/A.pcm prebuilt/B.pcm + +Note that with explicit or prebuilt modules, we are responsible for, and should be particularly careful about the compatibility of our modules. +Using mismatching compilation options and modules may lead to issues. + +.. code-block:: sh + + clang -cc1 -emit-obj use.c -fmodules -fimplicit-module-maps -fprebuilt-module-path=prebuilt -DENABLE_A + # use.c:4:10: warning: implicit declaration of function 'a' is invalid in C99 [-Wimplicit-function-declaration] + # return a(x); + # ^ + # 1 warning generated. + +So we need to maintain multiple versions of prebuilt modules. We can do so using a manual module mapping, or pointing to a different prebuilt module cache path. For example: + +.. code-block:: sh + + rm -rf prebuilt ; mkdir prebuilt ; rm -rf prebuilt_a ; mkdir prebuilt_a + clang -cc1 -emit-obj use.c -fmodules -fimplicit-module-maps -fmodules-cache-path=prebuilt -fdisable-module-hash + clang -cc1 -emit-obj use.c -fmodules -fimplicit-module-maps -fmodules-cache-path=prebuilt_a -fdisable-module-hash -DENABLE_A + clang -cc1 -emit-obj use.c -fmodules -fimplicit-module-maps -fprebuilt-module-path=prebuilt + clang -cc1 -emit-obj use.c -fmodules -fimplicit-module-maps -fprebuilt-module-path=prebuilt_a -DENABLE_A + + +Instead of managing the different module versions manually, we can build implicit modules in a given cache path (using ``-fmodules-cache-path``), and reuse them as prebuilt implicit modules by passing ``-fprebuilt-module-path`` and ``-fprebuilt-implicit-modules``. + +.. code-block:: sh + + rm -rf prebuilt; mkdir prebuilt + clang -cc1 -emit-obj -o use.o use.c -fmodules -fimplicit-module-maps -fmodules-cache-path=prebuilt + clang -cc1 -emit-obj -o use.o use.c -fmodules -fimplicit-module-maps -fmodules-cache-path=prebuilt -DENABLE_A + find prebuilt -name "*.pcm" + # prebuilt/1AYBIGPM8R2GA/A-3L1K4LUA6O31.pcm + # prebuilt/1AYBIGPM8R2GA/B-3L1K4LUA6O31.pcm + # prebuilt/VH0YZMF1OIRK/A-3L1K4LUA6O31.pcm + # prebuilt/VH0YZMF1OIRK/B-3L1K4LUA6O31.pcm + clang -cc1 -emit-obj -o use.o use.c -fmodules -fimplicit-module-maps -fprebuilt-module-path=prebuilt -fprebuilt-implicit-modules + clang -cc1 -emit-obj -o use.o use.c -fmodules -fimplicit-module-maps -fprebuilt-module-path=prebuilt -fprebuilt-implicit-modules -DENABLE_A + +Finally we want to allow implicit modules for configurations that were not prebuilt. When using the clang driver a module cache path is implicitly selected. Using ``-cc1``, we simply add use the ``-fmodules-cache-path`` option. + +.. code-block:: sh + + clang -cc1 -emit-obj -o use.o use.c -fmodules -fimplicit-module-maps -fprebuilt-module-path=prebuilt -fprebuilt-implicit-modules -fmodules-cache-path=cache + clang -cc1 -emit-obj -o use.o use.c -fmodules -fimplicit-module-maps -fprebuilt-module-path=prebuilt -fprebuilt-implicit-modules -fmodules-cache-path=cache -DENABLE_A + clang -cc1 -emit-obj -o use.o use.c -fmodules -fimplicit-module-maps -fprebuilt-module-path=prebuilt -fprebuilt-implicit-modules -fmodules-cache-path=cache -DENABLE_A -DOTHER_OPTIONS + +This way, a single directory containing multiple variants of modules can be prepared and reused. The options configuring the module cache are independent of other options. + Module Semantics ================ diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td --- a/clang/include/clang/Driver/Options.td +++ b/clang/include/clang/Driver/Options.td @@ -1439,6 +1439,9 @@ def fprebuilt_module_path : Joined<["-"], "fprebuilt-module-path=">, Group, Flags<[NoXarchOption, CC1Option]>, MetaVarName<"">, HelpText<"Specify the prebuilt module path">; +def fprebuilt_implicit_modules : Flag <["-"], "fprebuilt-implicit-modules">, Group, + Flags<[NoXarchOption, CC1Option]>, + HelpText<"Look up implicit modules in the prebuilt module path">; def fmodules_prune_interval : Joined<["-"], "fmodules-prune-interval=">, Group, Flags<[CC1Option]>, MetaVarName<"">, HelpText<"Specify the interval (in seconds) between attempts to prune the module cache">; diff --git a/clang/include/clang/Frontend/CompilerInstance.h b/clang/include/clang/Frontend/CompilerInstance.h --- a/clang/include/clang/Frontend/CompilerInstance.h +++ b/clang/include/clang/Frontend/CompilerInstance.h @@ -649,7 +649,10 @@ /// and replace any existing one with it. void createPreprocessor(TranslationUnitKind TUKind); - std::string getSpecificModuleCachePath(); + std::string getSpecificModuleCachePath(StringRef ModuleHash); + std::string getSpecificModuleCachePath() { + return getSpecificModuleCachePath(getInvocation().getModuleHash()); + } /// Create the AST context. void createASTContext(); diff --git a/clang/include/clang/Lex/HeaderSearch.h b/clang/include/clang/Lex/HeaderSearch.h --- a/clang/include/clang/Lex/HeaderSearch.h +++ b/clang/include/clang/Lex/HeaderSearch.h @@ -183,6 +183,9 @@ /// a system header. std::vector> SystemHeaderPrefixes; + /// The hash used for module cache paths. + std::string ModuleHash; + /// The path to the module cache. std::string ModuleCachePath; @@ -319,11 +322,17 @@ return {}; } + /// Set the hash to use for module cache paths. + void setModuleHash(StringRef Hash) { ModuleHash = std::string(Hash); } + /// Set the path to the module cache. void setModuleCachePath(StringRef CachePath) { ModuleCachePath = std::string(CachePath); } + /// Retrieve the module hash. + StringRef getModuleHash() const { return ModuleHash; } + /// Retrieve the path to the module cache. StringRef getModuleCachePath() const { return ModuleCachePath; } @@ -512,6 +521,15 @@ std::string getPrebuiltModuleFileName(StringRef ModuleName, bool FileMapOnly = false); + /// Retrieve the name of the prebuilt module file that should be used + /// to load the given module. + /// + /// \param Module The module whose module file name will be returned. + /// + /// \returns The name of the module file that corresponds to this module, + /// or an empty string if this module does not correspond to any module file. + std::string getPrebuiltImplicitModuleFileName(Module *Module); + /// Retrieve the name of the (to-be-)cached module file that should /// be used to load a module with the given name. /// @@ -614,6 +632,22 @@ Module *lookupModule(StringRef ModuleName, StringRef SearchName, bool AllowExtraModuleMapSearch = false); + /// Retrieve the name of the (to-be-)cached module file that should + /// be used to load a module with the given name. + /// + /// \param ModuleName The module whose module file name will be returned. + /// + /// \param ModuleMapPath A path that when combined with \c ModuleName + /// uniquely identifies this module. See Module::ModuleMap. + /// + /// \param CachePath A path to the module cache. + /// + /// \returns The name of the module file that corresponds to this module, + /// or an empty string if this module does not correspond to any module file. + std::string getCachedModuleFileNameImpl(StringRef ModuleName, + StringRef ModuleMapPath, + StringRef CachePath); + /// Retrieve a module with the given name, which may be part of the /// given framework. /// diff --git a/clang/include/clang/Lex/HeaderSearchOptions.h b/clang/include/clang/Lex/HeaderSearchOptions.h --- a/clang/include/clang/Lex/HeaderSearchOptions.h +++ b/clang/include/clang/Lex/HeaderSearchOptions.h @@ -142,6 +142,10 @@ /// file. unsigned ModuleMapFileHomeIsCwd : 1; + /// Also search for prebuilt implicit modules in the prebuilt module cache + /// path. + unsigned EnablePrebuiltImplicitModules : 1; + /// The interval (in seconds) between pruning operations. /// /// This operation is expensive, because it requires Clang to walk through @@ -217,8 +221,9 @@ HeaderSearchOptions(StringRef _Sysroot = "/") : Sysroot(_Sysroot), ModuleFormat("raw"), DisableModuleHash(false), ImplicitModuleMaps(false), ModuleMapFileHomeIsCwd(false), - UseBuiltinIncludes(true), UseStandardSystemIncludes(true), - UseStandardCXXIncludes(true), UseLibcxx(false), Verbose(false), + EnablePrebuiltImplicitModules(false), UseBuiltinIncludes(true), + UseStandardSystemIncludes(true), UseStandardCXXIncludes(true), + UseLibcxx(false), Verbose(false), ModulesValidateOncePerBuildSession(false), ModulesValidateSystemHeaders(false), ValidateASTInputFilesContent(false), UseDebugInfo(false), diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp --- a/clang/lib/Driver/ToolChains/Clang.cpp +++ b/clang/lib/Driver/ToolChains/Clang.cpp @@ -3379,6 +3379,8 @@ std::string("-fprebuilt-module-path=") + A->getValue())); A->claim(); } + if (Args.hasFlag(options::OPT_fprebuilt_implicit_modules, false)) + CmdArgs.push_back("-fprebuilt-implicit-modules"); if (Args.hasFlag(options::OPT_fmodules_validate_input_files_content, options::OPT_fno_modules_validate_input_files_content, false)) diff --git a/clang/lib/Frontend/CompilerInstance.cpp b/clang/lib/Frontend/CompilerInstance.cpp --- a/clang/lib/Frontend/CompilerInstance.cpp +++ b/clang/lib/Frontend/CompilerInstance.cpp @@ -428,8 +428,12 @@ PP->setPreprocessedOutput(getPreprocessorOutputOpts().ShowCPP); - if (PP->getLangOpts().Modules && PP->getLangOpts().ImplicitModules) - PP->getHeaderSearchInfo().setModuleCachePath(getSpecificModuleCachePath()); + if (PP->getLangOpts().Modules && PP->getLangOpts().ImplicitModules) { + std::string ModuleHash = getInvocation().getModuleHash(); + PP->getHeaderSearchInfo().setModuleHash(ModuleHash); + PP->getHeaderSearchInfo().setModuleCachePath( + getSpecificModuleCachePath(ModuleHash)); + } // Handle generating dependencies, if requested. const DependencyOutputOptions &DepOpts = getDependencyOutputOpts(); @@ -477,13 +481,11 @@ } } -std::string CompilerInstance::getSpecificModuleCachePath() { - // Set up the module path, including the hash for the - // module-creation options. +std::string CompilerInstance::getSpecificModuleCachePath(StringRef ModuleHash) { + // Set up the module path, including the hash for the module-creation options. SmallString<256> SpecificModuleCache(getHeaderSearchOpts().ModuleCachePath); if (!SpecificModuleCache.empty() && !getHeaderSearchOpts().DisableModuleHash) - llvm::sys::path::append(SpecificModuleCache, - getInvocation().getModuleHash()); + llvm::sys::path::append(SpecificModuleCache, ModuleHash); return std::string(SpecificModuleCache.str()); } @@ -1673,6 +1675,8 @@ if (!HSOpts.PrebuiltModuleFiles.empty() || !HSOpts.PrebuiltModulePaths.empty()) { ModuleFilename = HS.getPrebuiltModuleFileName(ModuleName); + if (HSOpts.EnablePrebuiltImplicitModules && ModuleFilename.empty()) + ModuleFilename = HS.getPrebuiltImplicitModuleFileName(M); if (!ModuleFilename.empty()) return MS_PrebuiltModulePath; } diff --git a/clang/lib/Frontend/CompilerInvocation.cpp b/clang/lib/Frontend/CompilerInvocation.cpp --- a/clang/lib/Frontend/CompilerInvocation.cpp +++ b/clang/lib/Frontend/CompilerInvocation.cpp @@ -2191,6 +2191,8 @@ !Args.hasArg(OPT_fmodules_disable_diagnostic_validation); Opts.ImplicitModuleMaps = Args.hasArg(OPT_fimplicit_module_maps); Opts.ModuleMapFileHomeIsCwd = Args.hasArg(OPT_fmodule_map_file_home_is_cwd); + Opts.EnablePrebuiltImplicitModules = + Args.hasArg(OPT_fprebuilt_implicit_modules); Opts.ModuleCachePruneInterval = getLastArgIntValue(Args, OPT_fmodules_prune_interval, 7 * 24 * 60 * 60); Opts.ModuleCachePruneAfter = diff --git a/clang/lib/Lex/HeaderSearch.cpp b/clang/lib/Lex/HeaderSearch.cpp --- a/clang/lib/Lex/HeaderSearch.cpp +++ b/clang/lib/Lex/HeaderSearch.cpp @@ -164,14 +164,39 @@ return {}; } +std::string HeaderSearch::getPrebuiltImplicitModuleFileName(Module *Module) { + const FileEntry *ModuleMap = + getModuleMap().getModuleMapFileForUniquing(Module); + StringRef ModuleName = Module->Name; + StringRef ModuleMapPath = ModuleMap->getName(); + StringRef ModuleCacheHash = HSOpts->DisableModuleHash ? "" : getModuleHash(); + for (const std::string &Dir : HSOpts->PrebuiltModulePaths) { + SmallString<256> CachePath(Dir); + llvm::sys::fs::make_absolute(CachePath); + llvm::sys::path::append(CachePath, ModuleCacheHash); + std::string FileName = + getCachedModuleFileNameImpl(ModuleName, ModuleMapPath, CachePath); + if (!FileName.empty() && getFileMgr().getFile(FileName)) + return FileName; + } + return {}; +} + std::string HeaderSearch::getCachedModuleFileName(StringRef ModuleName, StringRef ModuleMapPath) { + return getCachedModuleFileNameImpl(ModuleName, ModuleMapPath, + getModuleCachePath()); +} + +std::string HeaderSearch::getCachedModuleFileNameImpl(StringRef ModuleName, + StringRef ModuleMapPath, + StringRef CachePath) { // If we don't have a module cache path or aren't supposed to use one, we // can't do anything. - if (getModuleCachePath().empty()) + if (CachePath.empty()) return {}; - SmallString<256> Result(getModuleCachePath()); + SmallString<256> Result(CachePath); llvm::sys::fs::make_absolute(Result); if (HSOpts->DisableModuleHash) { diff --git a/clang/lib/Serialization/ASTReader.cpp b/clang/lib/Serialization/ASTReader.cpp --- a/clang/lib/Serialization/ASTReader.cpp +++ b/clang/lib/Serialization/ASTReader.cpp @@ -5844,6 +5844,7 @@ HSOpts.DisableModuleHash = Record[Idx++]; HSOpts.ImplicitModuleMaps = Record[Idx++]; HSOpts.ModuleMapFileHomeIsCwd = Record[Idx++]; + HSOpts.EnablePrebuiltImplicitModules = Record[Idx++]; HSOpts.UseBuiltinIncludes = Record[Idx++]; HSOpts.UseStandardSystemIncludes = Record[Idx++]; HSOpts.UseStandardCXXIncludes = Record[Idx++]; diff --git a/clang/lib/Serialization/ASTWriter.cpp b/clang/lib/Serialization/ASTWriter.cpp --- a/clang/lib/Serialization/ASTWriter.cpp +++ b/clang/lib/Serialization/ASTWriter.cpp @@ -1320,6 +1320,7 @@ Record.push_back(HSOpts.DisableModuleHash); Record.push_back(HSOpts.ImplicitModuleMaps); Record.push_back(HSOpts.ModuleMapFileHomeIsCwd); + Record.push_back(HSOpts.EnablePrebuiltImplicitModules); Record.push_back(HSOpts.UseBuiltinIncludes); Record.push_back(HSOpts.UseStandardSystemIncludes); Record.push_back(HSOpts.UseStandardCXXIncludes); diff --git a/clang/test/Modules/Inputs/prebuilt-implicit-module/a.h b/clang/test/Modules/Inputs/prebuilt-implicit-module/a.h new file mode 100644 --- /dev/null +++ b/clang/test/Modules/Inputs/prebuilt-implicit-module/a.h @@ -0,0 +1 @@ +const int a = 1; diff --git a/clang/test/Modules/Inputs/prebuilt-implicit-module/module.modulemap b/clang/test/Modules/Inputs/prebuilt-implicit-module/module.modulemap new file mode 100644 --- /dev/null +++ b/clang/test/Modules/Inputs/prebuilt-implicit-module/module.modulemap @@ -0,0 +1 @@ +module module_a { header "a.h" } diff --git a/clang/test/Modules/prebuilt-implicit-modules.m b/clang/test/Modules/prebuilt-implicit-modules.m new file mode 100644 --- /dev/null +++ b/clang/test/Modules/prebuilt-implicit-modules.m @@ -0,0 +1,35 @@ +// RUN: rm -rf %t +// RUN: %clang_cc1 -x objective-c -fmodules %S/Inputs/prebuilt-implicit-module/module.modulemap -emit-module -fmodule-name=module_a -fmodules-cache-path=%t +// RUN: find %t -name "module_a*.pcm" | grep module_a + +// Check we use a prebuilt module when available, and do not build an implicit module. +// RUN: rm -rf %t1 +// RUN: mkdir -p %t1 +// RUN: %clang_cc1 -x objective-c %s -I%S/Inputs/prebuilt-implicit-module -fmodules -fmodule-map-file=%S/Inputs/prebuilt-implicit-module/module.modulemap -fprebuilt-implicit-modules -fprebuilt-module-path=%t -fmodules-cache-path=%t1 +// RUN: find %t1 -name "module_a*.pcm" | not grep module_a + +// Check a module cache path is not required when all modules resolve to +// prebuilt implicit modules. +// RUN: rm -rf %t1 +// RUN: mkdir -p %t1 +// RUN: %clang_cc1 -x objective-c %s -I%S/Inputs/prebuilt-implicit-module -fmodules -fmodule-map-file=%S/Inputs/prebuilt-implicit-module/module.modulemap -fprebuilt-implicit-modules -fprebuilt-module-path=%t + +// Check that we correctly fall back to implicit modules if the prebuilt implicit module is not found. +// RUN: rm -rf %t1 +// RUN: mkdir -p %t1 +// RUN: %clang_cc1 -x objective-c %s -I%S/Inputs/prebuilt-implicit-module -fmodules -fmodule-map-file=%S/Inputs/prebuilt-implicit-module/module.modulemap -fprebuilt-implicit-modules -fprebuilt-module-path=%t -fmodules-cache-path=%t1 -fno-signed-char +// RUN: find %t1 -name "module_a*.pcm" | grep module_a + +// Check that non-implicit prebuilt modules are always preferred to prebuilt implicit modules. +// RUN: rm -rf %t2 +// RUN: mkdir -p %t2 +// RUN: %clang_cc1 -x objective-c -fmodules %S/Inputs/prebuilt-implicit-module/module.modulemap -emit-module -fmodule-name=module_a -fmodules-cache-path=%t +// RUN: %clang_cc1 -x objective-c -fmodules %S/Inputs/prebuilt-implicit-module/module.modulemap -emit-module -fmodule-name=module_a -o %t/module_a.pcm -fno-signed-char +// RUN: not %clang_cc1 -x objective-c %s -I%S/Inputs/prebuilt-implicit-module -fmodules -fmodule-map-file=%S/Inputs/prebuilt-implicit-module/module.modulemap -fprebuilt-implicit-modules -fprebuilt-module-path=%t -fmodules-cache-path=%t2 +// RUN: find %t2 -name "module_a*.pcm" | not grep module_a + +// expected-no-diagnostics +@import module_a; +int test() { + return a; +}