Modules: add command line option to support loading prebuilt modules on demand, without parsing any module map

[manmanren]

Summary

We add -fmodules-use-prebuilt-modules to support using prebuilt modules. In this mode, there is no need to load any module map and the programmer can simply use "@import" syntax to load the module directly from module cache without parsing any module map. We don't support rebuilding of the modules and ignore configuration mismatches.

[benlangmuir]

How about -fmodules-use-prebuilt-module-cache for the flag name? Saying "prebuilt-modules" is confusing to me, since -fmodule-file can also be used to load a prebuilt module, but doesn't use a cache.

lib/Driver/Tools.cpp

5416

if (IsPrebuilt) {

lib/Frontend/CompilerInstance.cpp

1491

Why do we assume this?

[manmanren]

lib/Driver/Tools.cpp

5416

Will do.

lib/Frontend/CompilerInstance.cpp

1491

Thanks for reviewing so quickly!

I was not sure about what value we should set IsSystem. Since we are mostly prebuilding system modules, I assumed "true". Do you have any suggestion here?

[benlangmuir]

Sorry for the delay, I apparently forgot to hit submit. Replied inline.

lib/Frontend/CompilerInstance.cpp

1491

I guess treating it as system is the more forgiving option, so I'm okay with keeping it. It's probably worth adding a comment that this is not always the correct choice, since it can affect diagnostics, etc.

[manmanren]

Add comments for setting IsSystem to true.

[benlangmuir]

Still missing the change for if (IsPrebuilt) {, but otherwise LGTM.

[manmanren]

In D23125#504942, @benlangmuir wrote:

How about -fmodules-use-prebuilt-module-cache for the flag name? Saying "prebuilt-modules" is confusing to me, since -fmodule-file can also be used to load a prebuilt module, but doesn't use a cache.

Oops, forgot to address your other comments.
Yes, I will change the flag name as you suggested and use if (IsPrebuilt).

Cheers,
Manman

[rsmith]

This doesn't seem like quite the right user interface for this feature. The module cache is volatile, and it's not really reasonable for people to assume they know what is and isn't within it. Instead, it seems like what we want to provide is zero or more paths to directories containing prebuilt .pcm files whose file names are the same as their top-level module names, completely separate from the module cache.

Specifically, rather than spelling this as -fmodules-use-prebuilt-modules -fmodules-cache-path=X, I'd suggest spelling it as -fprebuilt-module-path=X or similar (which can be repeated to provide multiple search paths). That would then combine orthogonally with -fno-implicit-modules and -fno-implicit-module-maps, where implicit modules would still use the module cache. Does that make sense for your use case?

You're also missing updates to the modules documentation.

[rsmith]

lib/Frontend/CompilerInstance.cpp

1483

This looks wrong: you're asking ReadAST to not diagnose a configuration mismatch, and you also don't diagnose it yourself.

1499–1507

You requested that ReadAST produces diagnostics for the OutOfDate and Missing cases when using prebuilt modules, so this diagnostic is redundant.

lib/Serialization/ASTReader.cpp

495–498

This seems unnecessarily complicated. Perhaps you could make CompilerInstance::loadModule treat all prebuilt modules as MK_ExplicitModule, or add a third ModuleKind for them?

[manmanren]

In D23125#510632, @rsmith wrote:

This doesn't seem like quite the right user interface for this feature. The module cache is volatile, and it's not really reasonable for people to assume they know what is and isn't within it. Instead, it seems like what we want to provide is zero or more paths to directories containing prebuilt .pcm files whose file names are the same as their top-level module names, completely separate from the module cache.

Specifically, rather than spelling this as -fmodules-use-prebuilt-modules -fmodules-cache-path=X, I'd suggest spelling it as -fprebuilt-module-path=X or similar (which can be repeated to provide multiple search paths). That would then combine orthogonally with -fno-implicit-modules and -fno-implicit-module-maps, where implicit modules would still use the module cache. Does that make sense for your use case?

What we want is to achieve performance by prebuilding the modules and using @import to load the corresponding pcm without parsing the module maps. In ASTReader, we want to treat these modules in the same way as the explicit modules.

-fprebuilt-module-path sounds reasonable. Let me see how the implementation goes ...

You're also missing updates to the modules documentation.

Yes, I will update the document.

Cheers,
Manman

lib/Frontend/CompilerInstance.cpp

1499–1507

Yes, I can remove this diagnostic.

lib/Serialization/ASTReader.cpp

495–498

I tried adding a ModuleKind at some point, but somehow it didn't work out. I will take another look at that.

[manmanren]

Addressing Richard's comments.

[rsmith]

include/clang/Lex/HeaderSearchOptions.h

96–97

It would seem preferable to allow multiple of these, to support mixing (for instance) a path to prebuilt system modules and a path to some per-project modules. I'd be fine handling that as a separate patch / discussion if you prefer.

lib/Frontend/CompilerInstance.cpp

1435–1439

It looks like the logic here is:

• if we found the module in a module map, then look for it in the module cache
• otherwise, if we have a prebuilt module path, then look for it there
• otherwise, error

I think that may be surprising; if I explicitly load a module map, perhaps via -fmodule-map-file=, and supply a prebuilt form of that module in the prebuilt modules path, I'd expect my prebuilt form to still be used.

Perhaps instead we could first search the prebuilt module path for an existing .pcm file, and if that fails and we have a Module then look for it in the module cache?

1485–1493

The module file will have provided a Module instance, if it is in fact a module file for the requested module. Instead of inventing a Module here, can you check that one exists and produce an error if not?

lib/Serialization/ASTReader.cpp

2279–2280

You're allowing configuration mismatches that we don't consider to be "compatible configuration mismatches" here. That seems really scary -- I believe this allows using a C++ module from C and similar unreasonable things, that will cause clang to crash. Did you really mean to do that?

[manmanren]

Thanks,
Manman

include/clang/Lex/HeaderSearchOptions.h

96–97

It sounds reasonable to provide prebuilt system modules and prebuilt project modules. I assume via the same command-line option -fprebuilt-module-path.

lib/Frontend/CompilerInstance.cpp

1435–1439

I was wondering about the priority of loading from prebuilt module path vs. loading from module cache. Your suggestion makes sense.

1485–1493

This is nice to know. I didn't realize we construct a Module when calling ReadAST. Can you be more specific on how to check if a Module exists?

lib/Serialization/ASTReader.cpp

2279–2280

I originally had this in my testing patch to be really relaxing on prebuilt modules. I will remove this :]

[bruno]

lib/Frontend/CompilerInstance.cpp

1435–1439

+1 to Richard's suggestion here.

I wonder about the workflow on generating such modules. Suppose I want to build some modules to re-use later with -fprebuilt-module-path=. RIght now, we get a module cache with hash in the names unless we disable hash computation via cc1 option -fdisable-module-hash. Should we assume that the user will have to copy out the cache dir and rename the modules OR that it will use -fdisable-module-hash instead? Should this option also try to read out modules with a hash in the name? Should we promote -fdisable-module-hash to a driver option as well?

@Manman, do you have any workflow in mind? @Richard, do you have any feedback in this respect?

In the crash reproducer we often have to rebuild the modules when running the script in a different machine because of the paths used to hash the modules, it would be nice (depending on the resulting design) if the crash reproducer could use -fprebuilt-module-path in order to try to re-use the pre-built modules resulting from the crash.

[manmanren]

lib/Frontend/CompilerInstance.cpp

1435–1439

If the modules are built implicitly with module hash, to load the module files as prebuilt modules, the user needs to copy out the cache dir and rename the modules. The user can choose to emit the modules explicitly or disable module hash when building the modules.

1485–1493

I assumed ReadAST only constructed Module instances for submodules, because it is inside ReadSubmoduleBlock. It turns out it also constructed Module instance for the top level module. I will update the patch.

[manmanren]

Addressing Richard's comments.

[manmanren]

Richard,

Are you okay with the patch now?

Thanks,
Manman

[rsmith]

Just a couple of things, but this basically LGTM.

docs/Modules.rst

217 ↗

(On Diff #67727)

Please document that the option can be given multiple times.

lib/Frontend/CompilerInstance.cpp

1486

This should have a real diagnostic rather than an assertion. This would happen if the pcm file in the prebuilt module path had the wrong filename, for instance.

It'd also be good to check that Module->ASTFile actually refers to the file that we found in the prebuilt module path, and not (say) to one of its transitive dependencies that ReadAST also loaded.

[manmanren]

Thanks,

Manman

lib/Frontend/CompilerInstance.cpp

1486

Updated the patch to use error diagnostics instead of assertion.

<<<
It'd also be good to check that Module->ASTFile actually refers to the file that we found in the prebuilt module path, and not (say) to one of its transitive dependencies that ReadAST also loaded.

Did we already check this when creating the module in ASTReader?

if (!ParentModule) {
  if (const FileEntry *CurFile = CurrentModule->getASTFile()) {
    if (CurFile != F.File) {
      ...
    }
  }

  CurrentModule->setASTFile(F.File);
}

[rsmith]

lib/Frontend/CompilerInstance.cpp

1486

The AST reader has no idea which module we're trying to load a module file for, so it can't check this. The check you quoted is ensuring that we don't have two different module files providing the same module.

A testcase for this would be:

module A is in prebuilt-module-cache/x.pcm
module B is in prebuilt-module-cache/A.pcm and depends on x.pcm

When trying to load module A, ReadAST will load A.pcm, and we'll find that we now have a definition for module A, but we still didn't get the module from the expected file.

[manmanren]

Cheers,

Manman

lib/Frontend/CompilerInstance.cpp

1486

Thanks for the explanation, it makes sense.

Updated the patch accordingly.

[rsmith]

lib/Frontend/CompilerInstance.cpp

1487

It'd be safer to check that FileMgr.getFile(ModuleFileName) == Module->getASTFile() in case the filename gets canonicalized by the file manager in some way.

[manmanren]

lib/Frontend/CompilerInstance.cpp

1487

Yes you are right. I will update the patch and commit the change.

Thanks for all the useful information!

include/clang/Basic/DiagnosticDriverKinds.td

 def err_analyzer_config_no_value : Error<
   "analyzer-config option '%0' has a key but no value">;
 def err_analyzer_config_multiple_values : Error<
   "analyzer-config option '%0' should contain only one '='">;
 
 def err_drv_modules_validate_once_requires_timestamp : Error<
   "option '-fmodules-validate-once-per-build-session' requires "
   "'-fbuild-session-timestamp=<seconds since Epoch>' or '-fbuild-session-file=<file>'">;
+def err_drv_modules_use_prebuilt_modules_conflicts_implicit : Error<
+  "option '-fmodules-use-prebuilt-modules' conflicts with '-fimplicit-module-maps'">;
 
 def err_test_module_file_extension_format : Error<
   "-ftest-module-file-extension argument '%0' is not of the required form "
   "'blockname:major:minor:hashed:user info'">;
 
 def warn_drv_invoking_fallback : Warning<"falling back to %0">,
   InGroup<Fallback>;
 

include/clang/Driver/Options.td

def fbuild_session_file : Joined<["-"], "fbuild-session-file=">,
   HelpText<"Use the last modification time of <file> as the build session timestamp">;
 def fmodules_validate_once_per_build_session : Flag<["-"], "fmodules-validate-once-per-build-session">,
   Group<i_Group>, Flags<[CC1Option]>,
   HelpText<"Don't verify input files for the modules if the module has been "
            "successfully validated or loaded during this build session">;
 def fmodules_disable_diagnostic_validation : Flag<["-"], "fmodules-disable-diagnostic-validation">,
   Group<i_Group>, Flags<[CC1Option]>,
   HelpText<"Disable validation of the diagnostic options when loading the module">;
+def fmodules_use_prebuilt_modules : Flag<["-"], "fmodules-use-prebuilt-modules">,
+  Group<i_Group>, Flags<[CC1Option]>,
+  HelpText<"Use prebuilt modules without loading any module map">;
 def fmodules_validate_system_headers : Flag<["-"], "fmodules-validate-system-headers">,
   Group<i_Group>, Flags<[CC1Option]>,
   HelpText<"Validate the system headers that a module depends on when loading the module">;
 def fmodules : Flag <["-"], "fmodules">, Group<f_Group>,
   Flags<[DriverOption, CC1Option]>,
   HelpText<"Enable the 'modules' language feature">;
 def fimplicit_module_maps : Flag <["-"], "fimplicit-module-maps">, Group<f_Group>,
   Flags<[DriverOption, CC1Option]>,

include/clang/Lex/HeaderSearchOptions.h

public:
   std::string ResourceDir;
 
   /// \brief The directory used for the module cache.
   std::string ModuleCachePath;
 
   /// \brief The directory used for a user build.
   std::string ModuleUserBuildPath;
 
   /// The module/pch container format.
   std::string ModuleFormat;
 
   /// \brief Whether we should disable the use of the hash string within the
   /// module cache.
   ///
   /// Note: Only used for testing!
   unsigned DisableModuleHash : 1;
 
   /// \brief Implicit module maps.  This option is enabld by default when
public:
   /// \brief Whether to validate system input files when a module is loaded.
   unsigned ModulesValidateSystemHeaders : 1;
 
   /// Whether the module includes debug information (-gmodules).
   unsigned UseDebugInfo : 1;
 
   unsigned ModulesValidateDiagnosticOptions : 1;
 
+  unsigned ModulesUsePrebuiltModules : 1;
+
   HeaderSearchOptions(StringRef _Sysroot = "/")
       : Sysroot(_Sysroot), ModuleFormat("raw"), DisableModuleHash(0),
         ImplicitModuleMaps(0), ModuleMapFileHomeIsCwd(0),
         ModuleCachePruneInterval(7 * 24 * 60 * 60),
         ModuleCachePruneAfter(31 * 24 * 60 * 60), BuildSessionTimestamp(0),
         UseBuiltinIncludes(true), UseStandardSystemIncludes(true),
         UseStandardCXXIncludes(true), UseLibcxx(false), Verbose(false),
         ModulesValidateOncePerBuildSession(false),
         ModulesValidateSystemHeaders(false),
-        UseDebugInfo(false), ModulesValidateDiagnosticOptions(true) {}
+        UseDebugInfo(false), ModulesValidateDiagnosticOptions(true),
+        ModulesUsePrebuiltModules(false) {}
 
   /// AddPath - Add the \p Path path to the specified \p Group list.
   void AddPath(StringRef Path, frontend::IncludeDirGroup Group,
                bool IsFramework, bool IgnoreSysRoot) {
     UserEntries.emplace_back(Path, Group, IsFramework, IgnoreSysRoot);
   }
 
   /// AddSystemHeaderPrefix - Override whether \#include directives naming a

lib/Driver/Tools.cpp

if (Args.hasFlag(options::OPT_fmodules, options::OPT_fno_modules, false)) {
     if (AllowedInCXX || !types::isCXX(InputType)) {
       CmdArgs.push_back("-fmodules");
       HaveModules = true;
     }
   }
 
   // -fmodule-maps enables implicit reading of module map files. By default,
   // this is enabled if we are using precompiled modules.
+  bool IsPrebuilt = Args.getLastArg(options::OPT_fmodules_use_prebuilt_modules);
   if (Args.hasFlag(options::OPT_fimplicit_module_maps,
-                   options::OPT_fno_implicit_module_maps, HaveModules)) {
+                   options::OPT_fno_implicit_module_maps,
+                   HaveModules && !IsPrebuilt)) {
+    // When using prebuilt modules, the default is no implicit module maps;
+    // if user specifies implicit module maps, emit error.
+    if (IsPrebuilt)
+      D.Diag(diag::err_drv_modules_use_prebuilt_modules_conflicts_implicit);
+    else
       CmdArgs.push_back("-fimplicit-module-maps");
   }
 
   // -fmodules-decluse checks that modules used are declared so (off by
   // default).
   if (Args.hasFlag(options::OPT_fmodules_decluse,
                    options::OPT_fno_modules_decluse, false)) {
     CmdArgs.push_back("-fmodules-decluse");
   }
if (Args.getLastArg(options::OPT_fmodules_validate_once_per_build_session)) {
 
     Args.AddLastArg(CmdArgs,
                     options::OPT_fmodules_validate_once_per_build_session);
   }
 
   Args.AddLastArg(CmdArgs, options::OPT_fmodules_validate_system_headers);
   Args.AddLastArg(CmdArgs, options::OPT_fmodules_disable_diagnostic_validation);
 
+  if (Args.getLastArg(options::OPT_fmodules_use_prebuilt_modules)) {
+    // When using prebuilt modules, we disable module hash.
+    CmdArgs.push_back("-fdisable-module-hash");
+    Args.AddLastArg(CmdArgs, options::OPT_fmodules_use_prebuilt_modules);
+  }
+
   // -faccess-control is default.
   if (Args.hasFlag(options::OPT_fno_access_control,
                    options::OPT_faccess_control, false))
     CmdArgs.push_back("-fno-access-control");
 
   // -felide-constructors is the default.
   if (Args.hasFlag(options::OPT_fno_elide_constructors,
                    options::OPT_felide_constructors, false))

lib/Frontend/CompilerInstance.cpp

if (Known != KnownModules.end()) {
     // Retrieve the cached top-level module.
     Module = Known->second;    
   } else if (ModuleName == getLangOpts().CurrentModule) {
     // This is the module we're building. 
     Module = PP->getHeaderSearchInfo().lookupModule(ModuleName);
     Known = KnownModules.insert(std::make_pair(Path[0].first, Module)).first;
   } else {
     // Search for a module with the given name.
     Module = PP->getHeaderSearchInfo().lookupModule(ModuleName);
-    if (!Module) {
+    HeaderSearchOptions &HSOpts =
+        PP->getHeaderSearchInfo().getHeaderSearchOpts();
+    if (!Module && !HSOpts.ModulesUsePrebuiltModules) {
       getDiagnostics().Report(ModuleNameLoc, diag::err_module_not_found)
       << ModuleName
       << SourceRange(ImportLoc, ModuleNameLoc);
       ModuleBuildFailed = true;
       return ModuleLoadResult();
     }
+    std::string ModuleFileName;
+    if (HSOpts.ModulesUsePrebuiltModules)
+      // Try to load the pcm file without a module map.
+      ModuleFileName =
+          PP->getHeaderSearchInfo().getModuleFileName(ModuleName, "");
+    else
+      ModuleFileName = PP->getHeaderSearchInfo().getModuleFileName(Module);
 
-    std::string ModuleFileName =
-        PP->getHeaderSearchInfo().getModuleFileName(Module);
     if (ModuleFileName.empty()) {
-      if (Module->HasIncompatibleModuleFile) {
+      if (!HSOpts.ModulesUsePrebuiltModules &&
+          Module->HasIncompatibleModuleFile) {
         // We tried and failed to load a module file for this module. Fall
         // back to textual inclusion for its headers.
         return ModuleLoadResult(nullptr, /*missingExpected*/true);
       }
 
       getDiagnostics().Report(ModuleNameLoc, diag::err_module_build_disabled)
           << ModuleName;
       ModuleBuildFailed = true;
       return ModuleLoadResult();
     }
 
     // If we don't already have an ASTReader, create one now.
     if (!ModuleManager)
       createModuleManager();
 
     llvm::Timer Timer;
     if (FrontendTimerGroup)
       Timer.init("Loading " + ModuleFileName, *FrontendTimerGroup);
     llvm::TimeRegion TimeLoading(FrontendTimerGroup ? &Timer : nullptr);
 
-    // Try to load the module file.
+    // Try to load the module file. If we are using prebuilt modules, we
+    // don't have the module map files and don't know how to rebuild modules.
     unsigned ARRFlags = ASTReader::ARR_OutOfDate | ASTReader::ARR_Missing;
     switch (ModuleManager->ReadAST(ModuleFileName,
                                    serialization::MK_ImplicitModule,
-                                   ImportLoc, ARRFlags)) {
-    case ASTReader::Success:
+                                   ImportLoc,
+                                   HSOpts.ModulesUsePrebuiltModules ?
+                                   ASTReader::ARR_ConfigurationMismatch :
+                                   ARRFlags)) {
+    case ASTReader::Success: {
+      if (HSOpts.ModulesUsePrebuiltModules && !Module) {
+        // Create a Module if we are using the prebuilt modules.
+        Module = PP->getHeaderSearchInfo().getModuleMap().findOrCreateModule(
+            ModuleName, nullptr, false/*IsFramework*/,
+            false/*IsExplicit*/).first;
+        Module->IsSystem = true;
+        Module->IsFromModuleFile = true;
+      }
       break;
+    }
 
     case ASTReader::OutOfDate:
     case ASTReader::Missing: {
+      if (HSOpts.ModulesUsePrebuiltModules) {
+        // We can't rebuild the module without a module map. Throw diagnostics
+        // here if using prebuilt modules.
+        getDiagnostics().Report(ModuleNameLoc, diag::err_module_not_found)
+        << ModuleName
+        << SourceRange(ImportLoc, ModuleNameLoc);
+        ModuleBuildFailed = true;
+        return ModuleLoadResult();
+      }
+
       // The module file is missing or out-of-date. Build it.
       assert(Module && "missing module file");
       // Check whether there is a cycle in the module graph.
       ModuleBuildStack ModPath = getSourceManager().getModuleBuildStack();
       ModuleBuildStack::iterator Pos = ModPath.begin(), PosEnd = ModPath.end();
       for (; Pos != PosEnd; ++Pos) {
         if (Pos->first == ModuleName)
           break;

lib/Frontend/CompilerInvocation.cpp

static void ParseHeaderSearchArgs(HeaderSearchOptions &Opts, ArgList &Args) {
   if (const Arg *A = Args.getLastArg(OPT_stdlib_EQ))
     Opts.UseLibcxx = (strcmp(A->getValue(), "libc++") == 0);
   Opts.ResourceDir = Args.getLastArgValue(OPT_resource_dir);
   Opts.ModuleCachePath = Args.getLastArgValue(OPT_fmodules_cache_path);
   Opts.ModuleUserBuildPath = Args.getLastArgValue(OPT_fmodules_user_build_path);
   Opts.DisableModuleHash = Args.hasArg(OPT_fdisable_module_hash);
   Opts.ModulesValidateDiagnosticOptions =
       !Args.hasArg(OPT_fmodules_disable_diagnostic_validation);
+  Opts.ModulesUsePrebuiltModules =
+      Args.hasArg(OPT_fmodules_use_prebuilt_modules);
   Opts.ImplicitModuleMaps = Args.hasArg(OPT_fimplicit_module_maps);
   Opts.ModuleMapFileHomeIsCwd = Args.hasArg(OPT_fmodule_map_file_home_is_cwd);
   Opts.ModuleCachePruneInterval =
       getLastArgIntValue(Args, OPT_fmodules_prune_interval, 7 * 24 * 60 * 60);
   Opts.ModuleCachePruneAfter =
       getLastArgIntValue(Args, OPT_fmodules_prune_after, 31 * 24 * 60 * 60);
   Opts.ModulesValidateOncePerBuildSession =
       Args.hasArg(OPT_fmodules_validate_once_per_build_session);

lib/Serialization/ASTReader.cpp

for (unsigned I = 0, N = PPOpts.Macros.size(); I != N; ++I) {
     }
 
     if (MacroNames && !Macros.count(MacroName))
       MacroNames->push_back(MacroName);
     Macros[MacroName] = std::make_pair(MacroBody, false);
   }
 }
 
+static bool isPrebuiltModule(Preprocessor &PP, ModuleKind MK) {
+  return (MK == MK_ExplicitModule || MK == MK_ImplicitModule) &&
+      PP.getHeaderSearchInfo().getHeaderSearchOpts().ModulesUsePrebuiltModules;
+}
+
 /// \brief Check the preprocessor options deserialized from the control block
 /// against the preprocessor options in an existing preprocessor.
 ///
 /// \param Diags If non-null, produce diagnostics for any mismatches incurred.
 static bool checkPreprocessorOptions(const PreprocessorOptions &PPOpts,
                                      const PreprocessorOptions &ExistingPPOpts,
                                      DiagnosticsEngine *Diags,
                                      FileManager &FileMgr,
while (1) {
     case llvm::BitstreamEntry::EndBlock: {
       // Validate input files.
       const HeaderSearchOptions &HSOpts =
           PP.getHeaderSearchInfo().getHeaderSearchOpts();
 
       // All user input files reside at the index range [0, NumUserInputs), and
       // system input files reside at [NumUserInputs, NumInputs). For explicitly
       // loaded module files, ignore missing inputs.
-      if (!DisableValidation && F.Kind != MK_ExplicitModule) {
+      if (!DisableValidation && F.Kind != MK_ExplicitModule &&
+          !isPrebuiltModule(PP, F.Kind)) {
         bool Complain = (ClientLoadCapabilities & ARR_OutOfDate) == 0;
 
         // If we are reading a module, we will create a verification timestamp,
         // so we verify all input files.  Otherwise, verify only user input
         // files.
 
         unsigned N = NumUserInputs;
         if (ValidateSystemInputs ||
case llvm::BitstreamEntry::EndBlock: {
 
       if (Listener && Listener->needsInputFileVisitation()) {
         unsigned N = Listener->needsSystemInputFileVisitation() ? NumInputs
                                                                 : NumUserInputs;
         for (unsigned I = 0; I < N; ++I) {
           bool IsSystem = I >= NumUserInputs;
           InputFileInfo FI = readInputFileInfo(F, I+1);
           Listener->visitInputFile(FI.Filename, IsSystem, FI.Overridden,
-                                   F.Kind == MK_ExplicitModule);
+                                   F.Kind == MK_ExplicitModule ||
+                                   isPrebuiltModule(PP, F.Kind));
         }
       }
 
       return Result;
     }
 
     case llvm::BitstreamEntry::SubBlock:
       switch (Entry.ID) {
case llvm::BitstreamEntry::SubBlock:
         // is required, because we checked them when we built it.
         if (Listener && !ImportedBy) {
           // Should we allow the configuration of the module file to differ from
           // the configuration of the current translation unit in a compatible
           // way?
           //
           // FIXME: Allow this for files explicitly specified with -include-pch.
           bool AllowCompatibleConfigurationMismatch =
-              F.Kind == MK_ExplicitModule;
+              F.Kind == MK_ExplicitModule || isPrebuiltModule(PP, F.Kind);
           const HeaderSearchOptions &HSOpts =
               PP.getHeaderSearchInfo().getHeaderSearchOpts();
 
           Result = ReadOptionsBlock(Stream, ClientLoadCapabilities,
                                     AllowCompatibleConfigurationMismatch,
                                     *Listener, SuggestedPredefines,
                                     HSOpts.ModulesValidateDiagnosticOptions);
           if (Result == Failure) {
             Error("malformed block record in AST file");
             return Result;
           }
 
           if (DisableValidation ||
-              (AllowConfigurationMismatch && Result == ConfigurationMismatch))
+              ((AllowConfigurationMismatch || isPrebuiltModule(PP, F.Kind)) &&
+               Result == ConfigurationMismatch))
             Result = Success;
 
           // If we can't load the module, exit early since we likely
           // will rebuild the module anyway. The stream may be in the
           // middle of a block.
           if (Result != Success)
             return Result;
         } else if (Stream.SkipBlock()) {
case MODULE_DIRECTORY: {
       assert(!F.ModuleName.empty() &&
              "MODULE_DIRECTORY found before MODULE_NAME");
       // If we've already loaded a module map file covering this module, we may
       // have a better path for it (relative to the current build).
       Module *M = PP.getHeaderSearchInfo().lookupModule(F.ModuleName);
       if (M && M->Directory) {
         // If we're implicitly loading a module, the base directory can't
         // change between the build and use.
-        if (F.Kind != MK_ExplicitModule) {
+        if (F.Kind != MK_ExplicitModule &&
+            !isPrebuiltModule(PP, F.Kind)) {
           const DirectoryEntry *BuildDir =
               PP.getFileManager().getDirectory(Blob);
           if (!BuildDir || BuildDir != M->Directory) {
             if ((ClientLoadCapabilities & ARR_OutOfDate) == 0)
               Diag(diag::err_imported_module_relocated)
                   << F.ModuleName << Blob << M->Directory->getName();
             return OutOfDate;
           }

 
 ASTReader::ASTReadResult
 ASTReader::ReadModuleMapFileBlock(RecordData &Record, ModuleFile &F,
                                   const ModuleFile *ImportedBy,
                                   unsigned ClientLoadCapabilities) {
   unsigned Idx = 0;
   F.ModuleMapPath = ReadPath(F, Record, Idx);
 
-  if (F.Kind == MK_ExplicitModule) {
+  if (F.Kind == MK_ExplicitModule || isPrebuiltModule(PP, F.Kind)) {
     // For an explicitly-loaded module, we don't care whether the original
     // module map file exists or matches.
     return Success;
   }
 
   // Try to resolve ModuleName in the current header search context and
   // verify that it is found in the same module map file as we saved. If the
   // top-level AST file is a main file, skip this check because there is no

test/Driver/modules.m

 // MODULES_VALIDATE_SYSTEM_HEADERS: -fmodules-validate-system-headers
 
 // RUN: %clang -### %s 2>&1 | FileCheck -check-prefix=MODULES_DISABLE_DIAGNOSTIC_VALIDATION_DEFAULT %s
 // MODULES_DISABLE_DIAGNOSTIC_VALIDATION_DEFAULT-NOT: -fmodules-disable-diagnostic-validation
 
 // RUN: %clang -fmodules-disable-diagnostic-validation -### %s 2>&1 | FileCheck -check-prefix=MODULES_DISABLE_DIAGNOSTIC_VALIDATION %s
 // MODULES_DISABLE_DIAGNOSTIC_VALIDATION: -fmodules-disable-diagnostic-validation
 
+// RUN: %clang -fmodules -### %s 2>&1 | FileCheck -check-prefix=MODULES_USE_PREBUILT_MODULES_DEFAULT %s
+// MODULES_USE_PREBUILT_MODULES_DEFAULT-NOT: -fmodules-use-prebuilt-modules
+
+// RUN: %clang -fmodules -fmodules-use-prebuilt-modules -### %s 2>&1 | FileCheck -check-prefix=MODULES_USE_PREBUILT_MODULES %s
+// MODULES_USE_PREBUILT_MODULES: "-fdisable-module-hash"
+// MODULES_USE_PREBUILT_MODULES: "-fmodules-use-prebuilt-modules"
+
+// RUN: %clang -fmodules -fmodules-use-prebuilt-modules -fimplicit-module-maps -### %s 2>&1 | FileCheck -check-prefix=MODULES_USE_PREBUILT_MODULES_ERR %s
+// MODULES_USE_PREBUILT_MODULES_ERR: option '-fmodules-use-prebuilt-modules' conflicts with '-fimplicit-module-maps'
+
 // RUN: %clang -fmodules -fmodule-map-file=foo.map -fmodule-map-file=bar.map -### %s 2>&1 | FileCheck -check-prefix=CHECK-MODULE-MAP-FILES %s
 // CHECK-MODULE-MAP-FILES: "-fmodules"
 // CHECK-MODULE-MAP-FILES: "-fmodule-map-file=foo.map"
 // CHECK-MODULE-MAP-FILES: "-fmodule-map-file=bar.map"
 
 // RUN: %clang -fmodules -fmodule-file=foo.pcm -fmodule-file=bar.pcm -### %s 2>&1 | FileCheck -check-prefix=CHECK-MODULE-FILES %s
 // CHECK-MODULE-FILES: "-fmodules"
 // CHECK-MODULE-FILES: "-fmodule-file=foo.pcm"
 // CHECK-MODULE-FILES: "-fmodule-file=bar.pcm"
 
 // RUN: %clang -fno-modules -fmodule-file=foo.pcm -fmodule-file=bar.pcm -### %s 2>&1 | FileCheck -check-prefix=CHECK-NO-MODULE-FILES %s
 // CHECK-NO-MODULE-FILES-NOT: "-fmodules"
 // CHECK-NO-MODULE-FILES-NOT: "-fmodule-file=foo.pcm"
 // CHECK-NO-MODULE-FILES-NOT: "-fmodule-file=bar.pcm"

test/Modules/Inputs/prebuilt-module/a.h

+const int a = 1;

test/Modules/Inputs/prebuilt-module/module.modulemap

+module prebuilt { header "a.h" }

test/Modules/prebuilt-module.m

+// RUN: rm -rf %t
+//
+// RUN: %clang_cc1 -fmodules -x objective-c -I %S/Inputs/prebuilt-module -triple %itanium_abi_triple -emit-module %S/Inputs/prebuilt-module/module.modulemap -fmodule-name=prebuilt -o %t/prebuilt.pcm
+// RUN: %clang_cc1 -fmodules -fmodules-use-prebuilt-modules -fdisable-module-hash -fmodules-cache-path=%t/ %s -verify
+
+// expected-no-diagnostics
+@import prebuilt;
+int test() {
+  return a;
+}