[clang][deps] Split translation units into individual -cc1 or other commands

[benlangmuir]

Summary

Instead of trying to "fix" the original driver invocation by appending arguments to it, split it into multiple commands, and for each -cc1 command use a CompilerInvocation to give precise control over the invocation.

This change should make it easier to (in the future) canonicalize the command-line (e.g. to improve hits in something like ccache), apply optimizations, or start supporting multi-arch builds, which would require different modules for each arch.

In the long run it may make sense to treat the TU commands as a dependency graph, each with their own dependencies on modules or earlier TU commands, but for now they are simply a list that is executed in order, and the dependencies are simply duplicated. Since we currently only support single-arch builds, there is no parallelism available in the execution.

[benlangmuir]

clang/test/ClangScanDeps/diagnostics.c

31

I didn't want to radically change all the tests, so in most cases I just loosened the matching so it would match inside the "commands". The modules-full.c and multiple-commands.c cover the whole structure in detail.

[tschuett]

clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h

43

Why is this not an enum class?

79

Why are all members public?

[jansvoboda11]

clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h

33

Have you considered using the Job/Command classes the driver uses? What are the downsides of doing that?

clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h

45

Is clang:: necessary here?

112

Maybe we could add this API directly to CompilerInvocation. I can see lots of clients not caring about the (potentially more efficient) CompilerInvocation::generateCC1CommandLine() with StringAllocator and inventing their own implementation of this.

clang/lib/Tooling/DependencyScanning/DependencyScanningTool.cpp

154

Shouldn't this be a responsibility of the dependency scanner?

196

Just a heads-up: after I land D132066 today, you might need to update this to respect the eager/lazy loading mode.

clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp

476

Wondering if we could move this to CompilerInvocation, right besides resetNonModularOptions().

clang/test/ClangScanDeps/diagnostics.c

31

That sounds good to me.

[benlangmuir]

clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h

33

The driver::Command/driver::JobList is not standalone from the driver - it depends on an external const Action &, const Tool &, and a few const char *, so if we used it we would also need to preserve the Compilation and Driver they came from somewhere, or change the API so the commands were only available inside a callback with limited scope that they are immediately copied out of. I'm also not sure what the consequences of mutating Commands are (there is a replaceArguments, but I don't know if it could break invariants).

clang/lib/Tooling/DependencyScanning/DependencyScanningTool.cpp

154

Good question! I was mirroring how it works for modules where we do this in the ModuleDepCollector and store the whole invocation in the ModuleDeps. But I guess that's "inside" the dep scanner, and this is "outside". Happy to shuffle things around.

What is your opinion of takeFullDependencies() adding to FrontendOpts.ModuleFiles, ? That is also arguably the scanner's responsibility.

Another thing we could do here is remove BuildInvocation from FullDependencies and ModuleDeps and only expose the serialized -cc1 arguments. It would simplify the new code a bit since there'd only be one kind of "command". On the other hand, I could also see it being potentially useful to have the whole invocation available if you were writing a C++ tool that used the scanner API rather than just clang-scan-deps itself.

[benlangmuir]

clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h

112

https://reviews.llvm.org/D132419

clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp

476

https://reviews.llvm.org/D132419

[jansvoboda11]

clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h

33

Makes sense, let's keep this as-is then.

clang/lib/Tooling/DependencyScanning/DependencyScanningTool.cpp

154

Good question! I was mirroring how it works for modules where we do this in the ModuleDepCollector and store the whole invocation in the ModuleDeps. But I guess that's "inside" the dep scanner, and this is "outside". Happy to shuffle things around.

As discussed offline, I'd prefer keeping CompilerInvocation internal to the dependency scanner and exposing simple types (std::vector<std::string> for the command line) unless clients need more flexibility.

What is your opinion of takeFullDependencies() adding to FrontendOpts.ModuleFiles, ? That is also arguably the scanner's responsibility.

I'd prefer adding to FrontendOpts.ModuleFiles earlier, before the consumer.

Another thing we could do here is remove BuildInvocation from FullDependencies and ModuleDeps and only expose the serialized -cc1 arguments. It would simplify the new code a bit since there'd only be one kind of "command". On the other hand, I could also see it being potentially useful to have the whole invocation available if you were writing a C++ tool that used the scanner API rather than just clang-scan-deps itself.

+1

[benlangmuir]

• Remove CompilerInvocation from Command and ModuleDeps. Only the arg strings are exposed now.
• Simplify Command, which is now just a simple struct.
• Move the logic for mutating the CompilerInvocation for the TU into the scanner.
• Minor refactoring to reduce nesting in DependencyScanningWorker.
• Rebase

[benlangmuir]

clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h

43

Removed in latest change.

79

Latest change simplifies this to a struct.

clang/lib/Tooling/DependencyScanning/DependencyScanningTool.cpp

154

Done in latest changes. The logic for mutating the compiler invocation is now contained in the ModuleDepCollector, alongside the existing logic for doing the same kind of changes to module build commands.

[jansvoboda11]

I left a couple of smaller initial comments. I'd like to see this split into multiple patches. I can see some formatting changes, removal of CompilerInvocation from ModuleDeps, isolated changes to Tooling, etc. That'd make it much easier to review.

clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h

196

Nit: handle*() functions are associated with DependencyConsumer in my brain. Could we find a distinct name to avoid confusion between all the different types we're using here?

212

I assume you're not using llvm::DenseMap<ModuleID, ModuleDeps *> because it's a hassle to implement for custom keys and performance is not a concern, correct? Any other aspects?

214

Would this allow us to remove ModuleDepCollectorPP::DirectPrebuiltModularDeps?

clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp

343

As said in a previous comment, we might avoid the copy by storing this in MDC in the first place.

452

Nit: this function might get easier to think about if we did this in one step with the context hash calculation:

MDC.associateWithContextHash(MD, CI);
MDC.addOutputPaths(MD, CI);
MD.BuildArguments = CI.getCC1CommandLine();

[benlangmuir]

In D132405#3747232, @jansvoboda11 wrote:

I'd like to see this split into multiple patches. I can see some formatting changes, removal of CompilerInvocation from ModuleDeps, isolated changes to Tooling, etc. That'd make it much easier to review.

ToolInvocation change: https://reviews.llvm.org/D132615
Remove CompilerInvocation from ModuleDeps: https://reviews.llvm.org/D132616
Factoring out the addModule*Files functions, sink DirectPrebuiltModularDeps, etc: https://reviews.llvm.org/D132617

[benlangmuir]

clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h

212

No good reason. I switched it to DenseMap and included the change in https://reviews.llvm.org/D132617

214

I sunk it to MDC in https://reviews.llvm.org/D132617. It needed to change to MapVector since we are also uniquing them.

clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp

343

We don't save any copies, since we need to actually store the PrebuiltModuleDep in MDC in order to apply it to a secondary CompilerInvocation after the preprocessor etc. is gone. But I still like sinking it down.

452

Good idea, included in https://reviews.llvm.org/D132617

[benlangmuir]

• Rebased, picking up the changes that were previously split out.
• Broke up applying changes to CI from passing to consumer. The MDC is no longer responsible for passing invocations to the consumer, and it is done at the Worker level.

[benlangmuir]

clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h

196

I split up applying changes to the CI from passing to the consumer. Hopefully it's a little simpler now.

clang/test/ClangScanDeps/modules-implicit-dot-private.m

70

Calling out this change so it doesn't disappear into the other test changes. The order changed because in the driver version of this we were appending -fmodule-file= options in the order of dependencies, but in the new code we insert into PrebuiltModuleFiles which is a std::map, so the entries are alphabetically sorted.

[jansvoboda11]

clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h

27

Not needed anymore, I assume.

43

Would it make sense to accept the clang::tooling::dependencies::Command struct here?

clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp

165

This makes sure we only run scan once per driver invocation? Can you expand on this a bit? Maybe even put the reasoning into a comment in the code.

455

I'm not particularly fond of the fact that Consumer.handleBuildCommand() is called in this lambda directly in the non-clang case, but several objects deep in the normal case (ToolInvocation -> DependencyScanningAction). I think a clearer way to do this would be to somehow extract the CompilerInvocation (or Command) result from ToolInvocation and report it in this lambda too.

clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp

177

I think this could be useful for other tools too in the future. Do you think it would make sense to put this in a more prominent header, so that other people can find it and reuse it more easily?

clang/test/ClangScanDeps/deprecated-driver-api.c

2

Please summarize what this test does.

clang/test/ClangScanDeps/multiple-commands.c

2

Please summarize what this test does.

[benlangmuir]

• handleBuildCommand now takes a Command
• Removed now-unused forward decl
• Expanded comment about scan-once behaviour
• Moved all calls to handleBuildCommand to a single place
• Added comments to new tests

[benlangmuir]

clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp

165

In theory you want to scan once for each independent chain of -cc1 commands, but since we don't yet support multi-arch builds in the scanner that just means scan once per driver invocation.

I'll add a comment.

455

Yeah, this area has gone through a lot of churn as I try to balance the desire to keep it clear where the consumer should be called vs. trying to keep the MDC contained to the action.

I took another crack at it in the latest diff.

clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp

177

I would prefer not to expose this without more understanding of what other use cases there are. It seems like there are many ways to interpret "needsModules" -- most of the time you probably want something more like LangOptions::Modules.

[jansvoboda11]

LGTM. Up to you if you act on the last comment. Thanks for seeing this through!

clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp

295

I'm not a fan of storing MDC as a member, but I don't see any clearly better alternatives. One option would be to pass CompilerInvocation out-parameter to ModuleDepCollector's constructor and relying on the collector to apply discovered dependencies, but that's not super obvious.

[benlangmuir]

clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp

295

We are applying changes to multiple CompilerInvocations (the scanned one, but also any later ones that are downstream of it), so it's less obvious to me how to do that here. We would need to pass in all of the invocations, but currently only one of them is in-memory at a time. I'm open to other ideas here, but I haven't come up with any great ideas. Maybe there's a refactoring of MDC that splits the scanning state from the state necessary to apply changes to the CI so we would only expose the minimum information needed, but I expect that would be an invasive change so prefer not to attempt it in this patch.

clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h

 
 /// A callback to lookup module outputs for "-fmodule-file=", "-o" etc.
 using LookupModuleOutputCallback =
     llvm::function_ref<std::string(const ModuleID &, ModuleOutputKind)>;
 
 /// The full dependencies and module graph for a specific input.
 struct FullDependencies {
   /// The identifier of the C++20 module this translation unit exports.
   ///
   /// If the translation unit is not a module then \c ID.ModuleName is empty.
   ModuleID ID;
 
   /// A collection of absolute paths to files that this translation unit
   /// directly depends on, not including transitive dependencies.
   std::vector<std::string> FileDeps;
 
   /// A collection of prebuilt modules this translation unit directly depends
   /// on, not including transitive dependencies.
   std::vector<PrebuiltModuleDep> PrebuiltModuleDeps;
 
   /// A list of modules this translation unit directly depends on, not including
   /// transitive dependencies.
   ///
   /// This may include modules with a different context hash when it can be
   /// determined that the differences are benign for this compilation.
   std::vector<ModuleID> ClangModuleDeps;
 
-  /// The command line of the TU (excluding the compiler executable).
-  std::vector<std::string> CommandLine;
+  /// The sequence of commands required to build the translation unit. Commands
+  /// should be executed in order.
+  ///
+  /// FIXME: If we add support for multi-arch builds in clang-scan-deps, we
+  /// should make the dependencies between commands explicit to enable parallel
+  /// builds of each architecture.
+  std::vector<Command> Commands;
+
+  /// Deprecated driver command-line. This will be removed in a future version.
+  std::vector<std::string> DriverCommandLine;
 };
 
 struct FullDependenciesResult {
   FullDependencies FullDeps;
   std::vector<ModuleDeps> DiscoveredModules;
 };
 
 /// The high-level implementation of the dependency discovery tool that runs on
 /// an individual worker thread.
 class DependencyScanningTool {
 public:
   /// Construct a dependency scanning tool.
   DependencyScanningTool(DependencyScanningService &Service,
                          llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem> FS =
                              llvm::vfs::createPhysicalFileSystem());
 
   /// Print out the dependency information into a string using the dependency
   /// file format that is specified in the options (-MD is the default) and
   /// return it. If \p ModuleName isn't empty, this function returns the
   /// dependency information of module \p ModuleName.
   ///
   /// \returns A \c StringError with the diagnostic output if clang errors
   /// occurred, dependency file contents otherwise.
   llvm::Expected<std::string>
   getDependencyFile(const std::vector<std::string> &CommandLine, StringRef CWD,
                     llvm::Optional<StringRef> ModuleName = None);
public:
   /// \returns a \c StringError with the diagnostic output if clang errors
   /// occurred, \c FullDependencies otherwise.
   llvm::Expected<FullDependenciesResult>
   getFullDependencies(const std::vector<std::string> &CommandLine,
                       StringRef CWD, const llvm::StringSet<> &AlreadySeen,
                       LookupModuleOutputCallback LookupModuleOutput,
                       llvm::Optional<StringRef> ModuleName = None);
 
+  llvm::Expected<FullDependenciesResult> getFullDependenciesLegacyDriverCommand(
+      const std::vector<std::string> &CommandLine, StringRef CWD,
+      const llvm::StringSet<> &AlreadySeen,
+      LookupModuleOutputCallback LookupModuleOutput,
+      llvm::Optional<StringRef> ModuleName = None);
+
 private:
   DependencyScanningWorker Worker;
 };
 
 class FullDependencyConsumer : public DependencyConsumer {
 public:
   FullDependencyConsumer(const llvm::StringSet<> &AlreadySeen,
                          LookupModuleOutputCallback LookupModuleOutput,
                          bool EagerLoadModules)
       : AlreadySeen(AlreadySeen), LookupModuleOutput(LookupModuleOutput),
         EagerLoadModules(EagerLoadModules) {}
 
+  void handleBuildCommand(Command Cmd) override {
+    Commands.push_back(std::move(Cmd));
+  }
+
   void handleDependencyOutputOpts(const DependencyOutputOptions &) override {}
 
   void handleFileDependency(StringRef File) override {
     Dependencies.push_back(std::string(File));
   }
 
   void handlePrebuiltModuleDependency(PrebuiltModuleDep PMD) override {
     PrebuiltModuleDeps.emplace_back(std::move(PMD));
   }
 
   void handleModuleDependency(ModuleDeps MD) override {
     ClangModuleDeps[MD.ID.ContextHash + MD.ID.ModuleName] = std::move(MD);
   }
 
   void handleContextHash(std::string Hash) override {
     ContextHash = std::move(Hash);
   }
 
   std::string lookupModuleOutput(const ModuleID &ID,
                                  ModuleOutputKind Kind) override {
     return LookupModuleOutput(ID, Kind);
   }
 
-  FullDependenciesResult getFullDependencies(
+  FullDependenciesResult getFullDependenciesLegacyDriverCommand(
       const std::vector<std::string> &OriginalCommandLine) const;
 
+  FullDependenciesResult takeFullDependencies();
+
 private:
   std::vector<std::string> Dependencies;
   std::vector<PrebuiltModuleDep> PrebuiltModuleDeps;
   llvm::MapVector<std::string, ModuleDeps, llvm::StringMap<unsigned>>
       ClangModuleDeps;
+  std::vector<Command> Commands;
   std::string ContextHash;
   std::vector<std::string> OutputPaths;
   const llvm::StringSet<> &AlreadySeen;
   LookupModuleOutputCallback LookupModuleOutput;
   bool EagerLoadModules;
 };
 
 } // end namespace dependencies
 } // end namespace tooling
 } // end namespace clang
 
 #endif // LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_DEPENDENCYSCANNINGTOOL_H

clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h

 #include "llvm/Support/FileSystem.h"
 #include <string>
 
 namespace clang {
 
 class DependencyOutputOptions;
 
 namespace tooling {
 namespace dependencies {
 
 class DependencyScanningWorkerFilesystem;
 
+/// A command-line tool invocation that is part of building a TU.
+///
+/// \see FullDependencies::Commands.
+struct Command {
+  std::string Executable;
+  std::vector<std::string> Arguments;
+};
+
 class DependencyConsumer {
 public:
   virtual ~DependencyConsumer() {}
 
+  virtual void handleBuildCommand(Command Cmd) = 0;
+
   virtual void
   handleDependencyOutputOpts(const DependencyOutputOptions &Opts) = 0;
 
   virtual void handleFileDependency(StringRef Filename) = 0;
 
   virtual void handlePrebuiltModuleDependency(PrebuiltModuleDep PMD) = 0;
 
   virtual void handleModuleDependency(ModuleDeps MD) = 0;
 
private:
   /// Whether to set up command-lines to load PCM files eagerly.
   bool EagerLoadModules;
 };
 
 } // end namespace dependencies
 } // end namespace tooling
 } // end namespace clang
 
 #endif // LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_DEPENDENCYSCANNINGWORKER_H

clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h

 };
 
 /// Collects modular and non-modular dependencies of the main file by attaching
 /// \c ModuleDepCollectorPP to the preprocessor.
 class ModuleDepCollector final : public DependencyCollector {
 public:
   ModuleDepCollector(std::unique_ptr<DependencyOutputOptions> Opts,
                      CompilerInstance &ScanInstance, DependencyConsumer &C,
-                     CompilerInvocation &&OriginalCI, bool OptimizeArgs,
+                     CompilerInvocation OriginalCI, bool OptimizeArgs,
                      bool EagerLoadModules);
 
   void attachToPreprocessor(Preprocessor &PP) override;
   void attachToASTReader(ASTReader &R) override;
 
+  /// Apply any changes implied by the discovered dependencies to the given
+  /// invocation, (e.g. disable implicit modules, add explicit module paths).
+  void applyDiscoveredDependencies(CompilerInvocation &CI);
+
 private:
   friend ModuleDepCollectorPP;
 
   /// The compiler instance for scanning the current translation unit.
   CompilerInstance &ScanInstance;
   /// The consumer of collected dependency information.
   DependencyConsumer &Consumer;
   /// Path to the main source file.
   std::string MainFile;
   /// Hash identifying the compilation conditions of the current TU.
   std::string ContextHash;
   /// Non-modular file dependencies. This includes the main source file and
   /// textually included header files.
   std::vector<std::string> FileDeps;
   /// Direct and transitive modular dependencies of the main source file.
   llvm::MapVector<const Module *, std::unique_ptr<ModuleDeps>> ModularDeps;
   /// Secondary mapping for \c ModularDeps allowing lookup by ModuleID without
   /// a preprocessor. Storage owned by \c ModularDeps.
   llvm::DenseMap<ModuleID, ModuleDeps *> ModuleDepsByID;
   /// Direct modular dependencies that have already been built.
   llvm::MapVector<const Module *, PrebuiltModuleDep> DirectPrebuiltModularDeps;
   /// Options that control the dependency output generation.
   std::unique_ptr<DependencyOutputOptions> Opts;
   /// The original Clang invocation passed to dependency scanner.
   CompilerInvocation OriginalInvocation;
   /// Whether to optimize the modules' command-line arguments.
   bool OptimizeArgs;
   /// Whether to set up command-lines to load PCM files eagerly.
   bool EagerLoadModules;

clang/lib/Tooling/DependencyScanning/DependencyScanningTool.cpp

DependencyScanningTool::DependencyScanningTool(
     : Worker(Service, std::move(FS)) {}
 
 llvm::Expected<std::string> DependencyScanningTool::getDependencyFile(
     const std::vector<std::string> &CommandLine, StringRef CWD,
     llvm::Optional<StringRef> ModuleName) {
   /// Prints out all of the gathered dependencies into a string.
   class MakeDependencyPrinterConsumer : public DependencyConsumer {
   public:
+    void handleBuildCommand(Command) override {}
+
     void
     handleDependencyOutputOpts(const DependencyOutputOptions &Opts) override {
       this->Opts = std::make_unique<DependencyOutputOptions>(Opts);
     }
 
     void handleFileDependency(StringRef File) override {
       Dependencies.push_back(std::string(File));
     }
DependencyScanningTool::getFullDependencies(
     LookupModuleOutputCallback LookupModuleOutput,
     llvm::Optional<StringRef> ModuleName) {
   FullDependencyConsumer Consumer(AlreadySeen, LookupModuleOutput,
                                   Worker.shouldEagerLoadModules());
   llvm::Error Result =
       Worker.computeDependencies(CWD, CommandLine, Consumer, ModuleName);
   if (Result)
     return std::move(Result);
-  return Consumer.getFullDependencies(CommandLine);
+  return Consumer.takeFullDependencies();
+}
+
+llvm::Expected<FullDependenciesResult>
+DependencyScanningTool::getFullDependenciesLegacyDriverCommand(
+    const std::vector<std::string> &CommandLine, StringRef CWD,
+    const llvm::StringSet<> &AlreadySeen,
+    LookupModuleOutputCallback LookupModuleOutput,
+    llvm::Optional<StringRef> ModuleName) {
+  FullDependencyConsumer Consumer(AlreadySeen, LookupModuleOutput,
+                                  Worker.shouldEagerLoadModules());
+  llvm::Error Result =
+      Worker.computeDependencies(CWD, CommandLine, Consumer, ModuleName);
+  if (Result)
+    return std::move(Result);
+  return Consumer.getFullDependenciesLegacyDriverCommand(CommandLine);
+}
+
+FullDependenciesResult FullDependencyConsumer::takeFullDependencies() {
+  FullDependenciesResult FDR;
+  FullDependencies &FD = FDR.FullDeps;
+
+  FD.ID.ContextHash = std::move(ContextHash);
+  FD.FileDeps = std::move(Dependencies);
+  FD.PrebuiltModuleDeps = std::move(PrebuiltModuleDeps);
+  FD.Commands = std::move(Commands);
+
+  for (auto &&M : ClangModuleDeps) {
+    auto &MD = M.second;
+    if (MD.ImportedByMainFile)
+      FD.ClangModuleDeps.push_back(MD.ID);
+    // TODO: Avoid handleModuleDependency even being called for modules
+    //   we've already seen.
+    if (AlreadySeen.count(M.first))
+      continue;
+    FDR.DiscoveredModules.push_back(std::move(MD));
+  }
+
+  return FDR;
 }
 
-FullDependenciesResult FullDependencyConsumer::getFullDependencies(
+FullDependenciesResult
+FullDependencyConsumer::getFullDependenciesLegacyDriverCommand(
     const std::vector<std::string> &OriginalCommandLine) const {
   FullDependencies FD;
 
-  FD.CommandLine = makeTUCommandLineWithoutPaths(
+  FD.DriverCommandLine = makeTUCommandLineWithoutPaths(
       ArrayRef<std::string>(OriginalCommandLine).slice(1));
 
   FD.ID.ContextHash = std::move(ContextHash);
 
   FD.FileDeps.assign(Dependencies.begin(), Dependencies.end());
 
   for (const PrebuiltModuleDep &PMD : PrebuiltModuleDeps)
-    FD.CommandLine.push_back("-fmodule-file=" + PMD.PCMFile);
+    FD.DriverCommandLine.push_back("-fmodule-file=" + PMD.PCMFile);
 
   for (auto &&M : ClangModuleDeps) {
     auto &MD = M.second;
     if (MD.ImportedByMainFile) {
       FD.ClangModuleDeps.push_back(MD.ID);
       auto PCMPath = LookupModuleOutput(MD.ID, ModuleOutputKind::ModuleFile);
       if (EagerLoadModules) {
-        FD.CommandLine.push_back("-fmodule-file=" + PCMPath);
+        FD.DriverCommandLine.push_back("-fmodule-file=" + PCMPath);
       } else {
-        FD.CommandLine.push_back("-fmodule-map-file=" + MD.ClangModuleMapFile);
-        FD.CommandLine.push_back("-fmodule-file=" + MD.ID.ModuleName + "=" +
-                                 PCMPath);
+        FD.DriverCommandLine.push_back("-fmodule-map-file=" +
+                                       MD.ClangModuleMapFile);
+        FD.DriverCommandLine.push_back("-fmodule-file=" + MD.ID.ModuleName +
+                                       "=" + PCMPath);
       }
     }
   }
 
   FD.PrebuiltModuleDeps = std::move(PrebuiltModuleDeps);
 
   FullDependenciesResult FDR;
 
   for (auto &&M : ClangModuleDeps) {
     // TODO: Avoid handleModuleDependency even being called for modules
     //   we've already seen.
     if (AlreadySeen.count(M.first))
       continue;
     FDR.DiscoveredModules.push_back(std::move(M.second));
   }
 
   FDR.FullDeps = std::move(FD);
   return FDR;
 }

clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp

 //===- DependencyScanningWorker.cpp - clang-scan-deps worker --------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "clang/Tooling/DependencyScanning/DependencyScanningWorker.h"
+#include "clang/Basic/DiagnosticFrontend.h"
 #include "clang/CodeGen/ObjectFilePCHContainerOperations.h"
+#include "clang/Driver/Compilation.h"
+#include "clang/Driver/Driver.h"
+#include "clang/Driver/Job.h"
+#include "clang/Driver/Tool.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/CompilerInvocation.h"
 #include "clang/Frontend/FrontendActions.h"
 #include "clang/Frontend/TextDiagnosticPrinter.h"
 #include "clang/Frontend/Utils.h"
 #include "clang/Lex/PreprocessorOptions.h"
 #include "clang/Tooling/DependencyScanning/DependencyScanningService.h"
 #include "clang/Tooling/DependencyScanning/ModuleDepCollector.h"
 #include "clang/Tooling/Tooling.h"
+#include "llvm/Support/Host.h"
 
 using namespace clang;
 using namespace tooling;
 using namespace dependencies;
 
 namespace {
 
 /// Forwards the gatherered dependencies to the consumer.
bool runInvocation(std::shared_ptr<CompilerInvocation> Invocation,
                      FileManager *FileMgr,
                      std::shared_ptr<PCHContainerOperations> PCHContainerOps,
                      DiagnosticConsumer *DiagConsumer) override {
     // Make a deep copy of the original Clang invocation.
     CompilerInvocation OriginalInvocation(*Invocation);
     // Restore the value of DisableFree, which may be modified by Tooling.
     OriginalInvocation.getFrontendOpts().DisableFree = DisableFree;
 
+    if (Scanned) {
+      // Scanning runs once for the first -cc1 invocation in a chain of driver
+      // jobs. For any dependent jobs, reuse the scanning result and just
+      // update the LastCC1Arguments to correspond to the new invocation.
+      // FIXME: to support multi-arch builds, each arch requires a separate scan
+      setLastCC1Arguments(std::move(OriginalInvocation));
+      return true;
+    }
+
+    Scanned = true;
+
     // Create a compiler instance to handle the actual work.
     CompilerInstance ScanInstance(std::move(PCHContainerOps));
     ScanInstance.setInvocation(std::move(Invocation));
 
     // Create the compiler's actual diagnostics engine.
     sanitizeDiagOpts(ScanInstance.getDiagnosticOpts());
     ScanInstance.createDiagnostics(DiagConsumer, /*ShouldOwnClient=*/false);
     if (!ScanInstance.hasDiagnostics())
bool runInvocation(std::shared_ptr<CompilerInvocation> Invocation,
 
     switch (Format) {
     case ScanningOutputFormat::Make:
       ScanInstance.addDependencyCollector(
           std::make_shared<DependencyConsumerForwarder>(
               std::move(Opts), WorkingDirectory, Consumer));
       break;
     case ScanningOutputFormat::Full:
-      ScanInstance.addDependencyCollector(std::make_shared<ModuleDepCollector>(
-          std::move(Opts), ScanInstance, Consumer,
-          std::move(OriginalInvocation), OptimizeArgs, EagerLoadModules));
+      MDC = std::make_shared<ModuleDepCollector>(
+          std::move(Opts), ScanInstance, Consumer, OriginalInvocation,
+          OptimizeArgs, EagerLoadModules);
+      ScanInstance.addDependencyCollector(MDC);
       break;
     }
 
     // Consider different header search and diagnostic options to create
     // different modules. This avoids the unsound aliasing of module PCMs.
     //
     // TODO: Implement diagnostic bucketing to reduce the impact of strict
     // context hashing.
     ScanInstance.getHeaderSearchOpts().ModulesStrictContextHash = true;
 
     std::unique_ptr<FrontendAction> Action;
 
     if (ModuleName)
       Action = std::make_unique<GetDependenciesByModuleNameAction>(*ModuleName);
     else
       Action = std::make_unique<ReadPCHAndPreprocessAction>();
 
     const bool Result = ScanInstance.ExecuteAction(*Action);
     if (!DepFS)
       FileMgr->clearStatCache();
+
+    if (Result)
+      setLastCC1Arguments(std::move(OriginalInvocation));
+
+    return Result;
+  }
+
+  bool hasScanned() const { return Scanned; }
+
+  /// Take the cc1 arguments corresponding to the most recent invocation used
+  /// with this action. Any modifications implied by the discovered dependencies
+  /// will have already been applied.
+  std::vector<std::string> takeLastCC1Arguments() {
+    std::vector<std::string> Result;
+    std::swap(Result, LastCC1Arguments); // Reset LastCC1Arguments to empty.
     return Result;
   }
 
 private:
+  void setLastCC1Arguments(CompilerInvocation &&CI) {
+    if (MDC)
+      MDC->applyDiscoveredDependencies(CI);
+    LastCC1Arguments = CI.getCC1CommandLine();
+  }
+
+private:
   StringRef WorkingDirectory;
   DependencyConsumer &Consumer;
   llvm::IntrusiveRefCntPtr<DependencyScanningWorkerFilesystem> DepFS;
   ScanningOutputFormat Format;
   bool OptimizeArgs;
   bool EagerLoadModules;
   bool DisableFree;
   llvm::Optional<StringRef> ModuleName;
+  std::shared_ptr<ModuleDepCollector> MDC;
+  std::vector<std::string> LastCC1Arguments;
+  bool Scanned = false;
 };
 
 } // end anonymous namespace
 
 DependencyScanningWorker::DependencyScanningWorker(
     DependencyScanningService &Service,
     llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem> FS)
     : Format(Service.getFormat()), OptimizeArgs(Service.canOptimizeArgs()),
runWithDiags(DiagnosticOptions *DiagOpts,
   TextDiagnosticPrinter DiagPrinter(DiagnosticsOS, DiagOpts);
 
   if (BodyShouldSucceed(DiagPrinter, *DiagOpts))
     return llvm::Error::success();
   return llvm::make_error<llvm::StringError>(DiagnosticsOS.str(),
                                              llvm::inconvertibleErrorCode());
 }
 
+static bool forEachDriverJob(
+    ArrayRef<std::string> Args, DiagnosticsEngine &Diags, FileManager &FM,
+    llvm::function_ref<bool(const driver::Command &Cmd)> Callback) {
+  std::unique_ptr<driver::Driver> Driver = std::make_unique<driver::Driver>(
+      Args[0], llvm::sys::getDefaultTargetTriple(), Diags,
+      "clang LLVM compiler", &FM.getVirtualFileSystem());
+  Driver->setTitle("clang_based_tool");
+
+  std::vector<const char *> Argv;
+  for (const std::string &Arg : Args)
+    Argv.push_back(Arg.c_str());
+
+  // The "input file not found" diagnostics from the driver are useful.
+  // The driver is only aware of the VFS working directory, but some clients
+  // change this at the FileManager level instead.
+  // In this case the checks have false positives, so skip them.
+  if (!FM.getFileSystemOpts().WorkingDir.empty())
+    Driver->setCheckInputsExist(false);
+  const std::unique_ptr<driver::Compilation> Compilation(
+      Driver->BuildCompilation(llvm::makeArrayRef(Argv)));
+  if (!Compilation)
+    return false;
+
+  for (const driver::Command &Job : Compilation->getJobs()) {
+    if (!Callback(Job))
+      return false;
+  }
+  return true;
+}
+
 llvm::Error DependencyScanningWorker::computeDependencies(
     StringRef WorkingDirectory, const std::vector<std::string> &CommandLine,
     DependencyConsumer &Consumer, llvm::Optional<StringRef> ModuleName) {
   // Reset what might have been modified in the previous worker invocation.
   RealFS->setCurrentWorkingDirectory(WorkingDirectory);
   if (Files)
     Files->setVirtualFileSystem(RealFS);
 
llvm::Error DependencyScanningWorker::computeDependencies(
 
   const std::vector<std::string> &FinalCommandLine =
       ModifiedCommandLine ? *ModifiedCommandLine : CommandLine;
 
   std::vector<const char *> FinalCCommandLine(CommandLine.size(), nullptr);
   llvm::transform(CommandLine, FinalCCommandLine.begin(),
                   [](const std::string &Str) { return Str.c_str(); });
 
-  return runWithDiags(CreateAndPopulateDiagOpts(FinalCCommandLine).release(),
-                      [&](DiagnosticConsumer &DC, DiagnosticOptions &DiagOpts) {
-                        // DisableFree is modified by Tooling for running
-                        // in-process; preserve the original value, which is
-                        // always true for a driver invocation.
-                        bool DisableFree = true;
-                        DependencyScanningAction Action(
-                            WorkingDirectory, Consumer, DepFS, Format,
-                            OptimizeArgs, EagerLoadModules, DisableFree,
-                            ModuleName);
-                        // Create an invocation that uses the underlying file
-                        // system to ensure that any file system requests that
-                        // are made by the driver do not go through the
-                        // dependency scanning filesystem.
-                        ToolInvocation Invocation(FinalCommandLine, &Action,
-                                                  CurrentFiles.get(),
-                                                  PCHContainerOps);
-                        Invocation.setDiagnosticConsumer(&DC);
-                        Invocation.setDiagnosticOptions(&DiagOpts);
-                        return Invocation.run();
-                      });
+  return runWithDiags(
+      CreateAndPopulateDiagOpts(FinalCCommandLine).release(),
+        IntrusiveRefCntPtr<DiagnosticsEngine> Diags =
+            CompilerInstance::createDiagnostics(&DiagOpts, &DC, false);
+        // Although `Diagnostics` are used only for command-line parsing, the
+        // custom `DiagConsumer` might expect a `SourceManager` to be present.
+        SourceManager SrcMgr(*Diags, *CurrentFiles);
+        Diags->setSourceManager(&SrcMgr);
+        DependencyScanningAction Action(WorkingDirectory, Consumer, DepFS,
+                                        Format, OptimizeArgs, EagerLoadModules,
+                                        DisableFree, ModuleName);
+        bool Success = forEachDriverJob(
+            FinalCommandLine, *Diags, *CurrentFiles,
+            [&](const driver::Command &Cmd) {
+              if (StringRef(Cmd.getCreator().getName()) != "clang") {
+                // Non-clang command. Just pass through to the dependency
+                // consumer.
+                Consumer.handleBuildCommand(
+                    {Cmd.getExecutable(),
+                     {Cmd.getArguments().begin(), Cmd.getArguments().end()}});
+                return true;
+              }
+
+              std::vector<std::string> Argv;
+              Argv.push_back(Cmd.getExecutable());
+              Argv.insert(Argv.end(), Cmd.getArguments().begin(),
+                          Cmd.getArguments().end());
+
+              ToolInvocation Invocation(std::move(Argv), &Action,
+                                        &*CurrentFiles, PCHContainerOps);
+              Invocation.setDiagnosticConsumer(Diags->getClient());
+              Invocation.setDiagnosticOptions(&Diags->getDiagnosticOptions());
+              if (!Invocation.run())
+                return false;
+
+              std::vector<std::string> Args = Action.takeLastCC1Arguments();
+              Consumer.handleBuildCommand(
+                  {Cmd.getExecutable(), std::move(Args)});
+              return true;
+            });
+
+        if (Success && !Action.hasScanned()) {
+          Diags->Report(diag::err_fe_expected_compiler_job)
+              << llvm::join(FinalCommandLine, " ");
+        }
+        return Success && Action.hasScanned();
 }

clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp

for (const ModuleID &MID : ClangModuleDeps) {
     if (EagerLoadModules)
       CI.getFrontendOpts().ModuleFiles.push_back(std::move(PCMPath));
     else
       CI.getHeaderSearchOpts().PrebuiltModuleFiles.insert(
           {MID.ModuleName, std::move(PCMPath)});
   }
 }
 
+static bool needsModules(FrontendInputFile FIF) {
+  switch (FIF.getKind().getLanguage()) {
+  case Language::Unknown:
+  case Language::Asm:
+  case Language::LLVM_IR:
+    return false;
+  default:
+    return true;
+  }
+}
+
+void ModuleDepCollector::applyDiscoveredDependencies(CompilerInvocation &CI) {
+  CI.clearImplicitModuleBuildOptions();
+
+  if (llvm::any_of(CI.getFrontendOpts().Inputs, needsModules)) {
+    SmallVector<ModuleID> DirectDeps;
+    for (const auto &KV : ModularDeps)
+      if (KV.second->ImportedByMainFile)
+        DirectDeps.push_back(KV.second->ID);
+
+    addModuleMapFiles(CI, DirectDeps);
+    addModuleFiles(CI, DirectDeps);
+
+    for (const auto &KV : DirectPrebuiltModularDeps)
+      CI.getFrontendOpts().ModuleFiles.push_back(KV.second.PCMFile);
+  }
+}
+
 static std::string getModuleContextHash(const ModuleDeps &MD,
                                         const CompilerInvocation &CI,
                                         bool EagerLoadModules) {
   llvm::HashBuilder<llvm::TruncatedBLAKE3<16>,
                     llvm::support::endianness::native>
       HashBuilder;
   SmallString<32> Scratch;
 
void ModuleDepCollectorPP::EndOfMainFile() {
 
   for (auto &&I : MDC.ModularDeps)
     MDC.Consumer.handleModuleDependency(*I.second);
 
   for (auto &&I : MDC.FileDeps)
     MDC.Consumer.handleFileDependency(I);
 
   for (auto &&I : MDC.DirectPrebuiltModularDeps)
     MDC.Consumer.handlePrebuiltModuleDependency(I.second);
 }
 
 ModuleID ModuleDepCollectorPP::handleTopLevelModule(const Module *M) {
   assert(M == M->getTopLevelModule() && "Expected top level module!");
 
   // If this module has been handled already, just return its ID.
   auto ModI = MDC.ModularDeps.insert({M, nullptr});
   if (!ModI.second)
ModuleID ModuleDepCollectorPP::handleTopLevelModule(const Module *M) {
 
   MDC.associateWithContextHash(CI, MD);
 
   // Finish the compiler invocation. Requires dependencies and the context hash.
   MDC.addOutputPaths(CI, MD);
 
   MD.BuildArguments = CI.getCC1CommandLine();
 
   return MD.ID;
 }
 
 static void forEachSubmoduleSorted(const Module *M,
                                    llvm::function_ref<void(const Module *)> F) {
   // Submodule order depends on order of header includes for inferred submodules
   // we don't care about the exact order, so sort so that it's consistent across
   // TUs to improve sharing.
   SmallVector<const Module *> Submodules(M->submodule_begin(),
if (Affecting != M->getTopLevelModule() &&
         MD.ClangModuleDeps.push_back(ImportID);
     }
   }
 }
 
 ModuleDepCollector::ModuleDepCollector(
     std::unique_ptr<DependencyOutputOptions> Opts,
     CompilerInstance &ScanInstance, DependencyConsumer &C,
-    CompilerInvocation &&OriginalCI, bool OptimizeArgs, bool EagerLoadModules)
+    CompilerInvocation OriginalCI, bool OptimizeArgs, bool EagerLoadModules)
     : ScanInstance(ScanInstance), Consumer(C), Opts(std::move(Opts)),
       OriginalInvocation(std::move(OriginalCI)), OptimizeArgs(OptimizeArgs),
       EagerLoadModules(EagerLoadModules) {}
 
 void ModuleDepCollector::attachToPreprocessor(Preprocessor &PP) {
   PP.addPPCallbacks(std::make_unique<ModuleDepCollectorPP>(*this));
 }
 

clang/test/ClangScanDeps/deprecated-driver-api.c

+// Test the deprecated version of the API that returns a driver command instead
+// of multiple -cc1 commands.
+
+// RUN: rm -rf %t
+// RUN: split-file %s %t
+// RUN: sed -e "s|DIR|%/t|g" %t/cdb.json.in > %t/cdb.json
+
+// RUN: clang-scan-deps -compilation-database=%t/cdb.json -format experimental-full \
+// RUN:   -deprecated-driver-command | sed 's:\\\\\?:/:g' | FileCheck %s
+
+// CHECK: "command-line": [
+// CHECK:   "-c"
+// CHECK:   "{{.*}}tu.c"
+// CHECK:   "-save-temps"
+// CHECK:   "-fno-implicit-modules"
+// CHECK:   "-fno-implicit-module-maps"
+// CHECK: ]
+// CHECK: "file-deps": [
+// CHECK:   "{{.*}}tu.c",
+// CHECK:   "{{.*}}header.h"
+// CHECK: ]
+
+//--- cdb.json.in
+[{
+  "directory": "DIR",
+  "command": "clang -c DIR/tu.c -save-temps",
+  "file": "DIR/tu.c"
+}]
+
+//--- header.h
+void bar(void);
+
+//--- tu.c
+#include "header.h"
+
+void foo(void) {
+  bar();
+}

clang/test/ClangScanDeps/diagnostics.c

 // CHECK-NEXT:         "[[PREFIX]]/mod.h"
 // CHECK-NEXT:         "[[PREFIX]]/module.modulemap"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "name": "mod"
 // CHECK-NEXT:     }
 // CHECK-NEXT:   ],
 // CHECK-NEXT:   "translation-units": [
 // CHECK-NEXT:     {
-// CHECK-NEXT:       "clang-context-hash": "[[HASH_TU:.*]],
+// CHECK:            "clang-context-hash": "[[HASH_TU:.*]],
 // CHECK-NEXT:       "clang-module-deps": [
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "[[HASH_MOD]]",
 // CHECK-NEXT:           "module-name": "mod"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "command-line": [
-// CHECK:              "-fno-implicit-modules"
-// CHECK:              "-fno-implicit-module-maps"
+// CHECK-NOT:          "-fimplicit-modules"
+// CHECK-NOT:          "-fimplicit-module-maps"
 // CHECK:            ],
-// CHECK-NEXT:       "file-deps": [
+// CHECK:            "file-deps": [
 // CHECK-NEXT:         "[[PREFIX]]/tu.c"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "input-file": "[[PREFIX]]/tu.c"
 // CHECK-NEXT:     }
-// CHECK-NEXT:   ]
-// CHECK-NEXT: }

clang/test/ClangScanDeps/header-search-pruning-transitive.c

 // CHECK-NEXT:         "[[PREFIX]]/./end/end.h",
 // CHECK-NEXT:         "[[PREFIX]]/./module.modulemap"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "name": "Y"
 // CHECK-NEXT:     }
 // CHECK-NEXT:   ],
 // CHECK-NEXT:   "translation-units": [
 // CHECK-NEXT:     {
-// CHECK-NEXT:       "clang-context-hash": "{{.*}}",
+// CHECK:            "clang-context-hash": "{{.*}}",
 // CHECK-NEXT:       "clang-module-deps": [
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "[[HASH_X]]",
 // CHECK-NEXT:           "module-name": "X"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "command-line": [
 // CHECK:            ],
-// CHECK-NEXT:       "file-deps": [
+// CHECK:            "file-deps": [
 // CHECK-NEXT:         "[[PREFIX]]/test.c"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "input-file": "[[PREFIX]]/test.c"
 // CHECK-NEXT:     }
-// CHECK-NEXT:   ]
-// CHECK-NEXT: }
-// CHECK-NEXT: {
+
+// CHECK:      {
 // CHECK-NEXT:   "modules": [
 // CHECK-NEXT:     {
 // CHECK-NEXT:       "clang-module-deps": [
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "[[HASH_Y_WITHOUT_A:.*]]",
 // CHECK-NEXT:           "module-name": "Y"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ],

 // CHECK-NEXT:         "[[PREFIX]]/./end/end.h",
 // CHECK-NEXT:         "[[PREFIX]]/./module.modulemap"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "name": "Y"
 // CHECK-NEXT:     }
 // CHECK-NEXT:   ],
 // CHECK-NEXT:   "translation-units": [
 // CHECK-NEXT:     {
-// CHECK-NEXT:       "clang-context-hash": "{{.*}}",
+// CHECK:            "clang-context-hash": "{{.*}}",
 // CHECK-NEXT:       "clang-module-deps": [
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "{{.*}}",
 // CHECK-NEXT:           "module-name": "X"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "command-line": [
 // CHECK:            ],
-// CHECK-NEXT:       "file-deps": [
+// CHECK:            "file-deps": [
 // CHECK-NEXT:         "[[PREFIX]]/test.c"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "input-file": "[[PREFIX]]/test.c"
 // CHECK-NEXT:     }
-// CHECK-NEXT:   ]
-// CHECK-NEXT: }

clang/test/ClangScanDeps/modules-context-hash-ignore-macros.c

 // CHECK:            "clang-module-deps": [
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "[[HASH_NO_FOO]]"
 // CHECK-NEXT:            "module-name": "Mod"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ]
 // CHECK-NEXT:       "command-line": [
 // CHECK-NOT:          "-DFOO"
+// CHECK-NOT:          "FOO"
 // CHECK:            ]
 // CHECK:            "input-file": "{{.*}}tu1.c"
 // CHECK-NEXT:     }
-// CHECK-NEXT:     {
+// CHECK:          {
 // CHECK:            "clang-module-deps": [
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "[[HASH_FOO]]"
 // CHECK-NEXT:            "module-name": "Mod"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ]
 // CHECK-NEXT:       "command-line": [
-// CHECK:              "-DFOO"
+// CHECK:              "-D"
+// CHECK-NEXT:         "FOO"
 // CHECK:            ]
 // CHECK:            "input-file": "{{.*}}tu2.c"
 // CHECK-NEXT:     }
-// CHECK-NEXT:     {
+// CHECK:          {
 // CHECK:            "clang-module-deps": [
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "[[HASH_NO_FOO]]"
 // CHECK-NEXT:            "module-name": "Mod"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ]
 // CHECK-NEXT:       "command-line": [
-// CHECK:              "-DFOO"
 // CHECK:              "-fmodules-ignore-macro=FOO"
+// CHECK:              "-D"
+// CHECK-NEXT:         "FOO"
 // CHECK:            ]
 // CHECK:            "input-file": "{{.*}}tu3.c"
 
 //--- cdb.json.template
 [
   {
     "directory": "DIR",
     "command": "clang -fsyntax-only DIR/tu1.c -fmodules -fimplicit-module-maps -fmodules-cache-path=DIR/cache",

clang/test/ClangScanDeps/modules-context-hash-outputs.c

 // CHECK-NEXT:     {
 // CHECK:            "clang-module-deps": [
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "[[HASH1]]"
 // CHECK-NEXT:            "module-name": "Mod"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ]
 // CHECK-NEXT:       "command-line": [
-// CHECK:              "-MF"
+// CHECK:              "-dependency-file"
 // CHECK:            ]
 // CHECK:            "input-file": "{{.*}}tu1.c"
 // CHECK-NEXT:     }
-// CHECK-NEXT:     {
+// CHECK:          {
 // CHECK:            "clang-module-deps": [
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "[[HASH2]]"
 // CHECK-NEXT:            "module-name": "Mod"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ]
 // CHECK-NEXT:       "command-line": [
 // CHECK-NOT:          "-MF"
+// CHECK-NOT:          "-dependency-file"
 // CHECK:            ]
 // CHECK:            "input-file": "{{.*}}tu2.c"
 
 //--- cdb.json.template
 [
   {
     "directory": "DIR",
     "command": "clang -MD -MF DIR/tu1.d -fsyntax-only DIR/tu1.c -fmodules -fimplicit-module-maps -fmodules-cache-path=DIR/cache",

clang/test/ClangScanDeps/modules-context-hash-warnings.c

 // CHECK-NEXT:            "module-name": "Mod"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ]
 // CHECK-NEXT:       "command-line": [
 // CHECK:              "-Wall"
 // CHECK:            ]
 // CHECK:            "input-file": "{{.*}}tu1.c"
 // CHECK-NEXT:     }
-// CHECK-NEXT:     {
+// CHECK:          {
 // CHECK:            "clang-module-deps": [
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "[[HASH2]]"
 // CHECK-NEXT:            "module-name": "Mod"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ]
 // CHECK-NEXT:       "command-line": [
 // CHECK-NOT:          "-Wall"

clang/test/ClangScanDeps/modules-context-hash.c

 // CHECK-NEXT:         "[[PREFIX]]/mod.h",
 // CHECK-NEXT:         "[[PREFIX]]/module.modulemap"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "name": "mod"
 // CHECK-NEXT:     }
 // CHECK-NEXT:   ],
 // CHECK-NEXT:   "translation-units": [
 // CHECK-NEXT:     {
-// CHECK-NEXT:       "clang-context-hash": "{{.*}}",
+// CHECK:            "clang-context-hash": "{{.*}}",
 // CHECK-NEXT:       "clang-module-deps": [
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "[[HASH_MOD_A]]",
 // CHECK-NEXT:           "module-name": "mod"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "command-line": [
 // CHECK:            ],
-// CHECK-NEXT:       "file-deps": [
+// CHECK:            "file-deps": [
 // CHECK-NEXT:         "[[PREFIX]]/tu.c"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "input-file": "[[PREFIX]]/tu.c"
 // CHECK-NEXT:     }
-// CHECK-NEXT:   ]
-// CHECK-NEXT: }
-// CHECK-NEXT: {
-// CHECK-NEXT:   "modules": [
+
+// CHECK:       "modules": [
 // CHECK-NEXT:     {
 // CHECK-NEXT:       "clang-module-deps": [],
 // CHECK-NEXT:       "clang-modulemap-file": "[[PREFIX]]/module.modulemap",
 // CHECK-NEXT:       "command-line": [
 // CHECK-NEXT:         "-cc1"
 // CHECK:              "-emit-module"
 // CHECK:              "-I"
 // CHECK:              "[[PREFIX]]/b"
 // CHECK:              "-fmodule-name=mod"
 // CHECK:            ],
 // CHECK-NOT:        "context-hash": "[[HASH_MOD_A]]",
 // CHECK:            "file-deps": [
 // CHECK-NEXT:         "[[PREFIX]]/b/dep.h",
 // CHECK-NEXT:         "[[PREFIX]]/mod.h",
 // CHECK-NEXT:         "[[PREFIX]]/module.modulemap"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "name": "mod"
 // CHECK-NEXT:     }
 // CHECK-NEXT:   ],
 // CHECK-NEXT:   "translation-units": [
 // CHECK-NEXT:     {
-// CHECK-NEXT:       "clang-context-hash": "{{.*}}",
+// CHECK:            "clang-context-hash": "{{.*}}",
 // CHECK-NEXT:       "clang-module-deps": [
 // CHECK-NEXT:         {
 // CHECK-NOT:            "context-hash": "[[HASH_MOD_A]]",
 // CHECK:                "module-name": "mod"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "command-line": [
 // CHECK:            ],
-// CHECK-NEXT:       "file-deps": [
+// CHECK:            "file-deps": [
 // CHECK-NEXT:         "[[PREFIX]]/tu.c"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "input-file": "[[PREFIX]]/tu.c"
 // CHECK-NEXT:     }
-// CHECK-NEXT:   ]
-// CHECK-NEXT: }

clang/test/ClangScanDeps/modules-dep-args.c

 // CHECK-NEXT:         "[[PREFIX]]/module.modulemap",
 // CHECK-NEXT:         "[[PREFIX]]/transitive.h"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "name": "Transitive"
 // CHECK-NEXT:     }
 // CHECK-NEXT:   ],
 // CHECK-NEXT:   "translation-units": [
 // CHECK-NEXT:     {
-// CHECK-NEXT:       "clang-context-hash": "{{.*}}",
+// CHECK:            "clang-context-hash": "{{.*}}",
 // CHECK-NEXT:       "clang-module-deps": [
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "{{.*}}",
 // CHECK-NEXT:           "module-name": "Direct"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "command-line": [
 // CHECK_CACHE:        "-fmodule-file={{.*}}/cache/{{.*}}/Direct-{{.*}}.pcm"
 // CHECK_BUILD:        "-fmodule-file={{.*}}/build/{{.*}}/Direct-{{.*}}.pcm"
 // CHECK_LAZY:         "-fmodule-map-file=[[PREFIX]]/module.modulemap"
 // CHECK_LAZY:         "-fmodule-file=Direct=[[PREFIX]]/{{.*}}/Direct-{{.*}}.pcm"
 // CHECK_EAGER-NOT:    "-fmodule-map-file={{.*}}"
 // CHECK_EAGER:        "-fmodule-file=[[PREFIX]]/{{.*}}/Direct-{{.*}}.pcm"
 // CHECK:            ],
-// CHECK-NEXT:       "file-deps": [
+// CHECK:            "file-deps": [
 // CHECK-NEXT:         "[[PREFIX]]/tu.c"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "input-file": "[[PREFIX]]/tu.c"
 // CHECK-NEXT:     }
-// CHECK-NEXT:   ]
-// CHECK-NEXT: }
 

clang/test/ClangScanDeps/modules-fmodule-name-no-module-built.m

 // CHECK-NEXT:         "[[PREFIX]]/Inputs/header2.h",
 // CHECK-NEXT:         "[[PREFIX]]/Inputs/module.modulemap"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "name": "header2"
 // CHECK-NEXT:     }
 // CHECK-NEXT:   ],
 // CHECK-NEXT:   "translation-units": [
 // CHECK-NEXT:     {
-// CHECK-NEXT:       "clang-context-hash": "[[HASH_TU:[A-Z0-9]+]]",
+// CHECK:            "clang-context-hash": "[[HASH_TU:[A-Z0-9]+]]",
 // CHECK-NEXT:       "clang-module-deps": [
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "[[HASH_H2]]",
 // CHECK-NEXT:           "module-name": "header2"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "command-line": [
 // CHECK:            ],
-// CHECK-NEXT:       "file-deps": [
+// CHECK:            "file-deps": [
 // CHECK-NEXT:         "[[PREFIX]]/modules-fmodule-name-no-module-built.m"
 // CHECK-NEXT:         "[[PREFIX]]/Inputs/header3.h"
 // CHECK-NEXT:         "[[PREFIX]]/Inputs/header.h"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "input-file": "[[PREFIX]]/modules-fmodule-name-no-module-built.m"
 // CHECK-NEXT:     }
-// CHECK-NEXT:   ]
-// CHECK-NEXT: }

clang/test/ClangScanDeps/modules-full.cpp

 // CHECK-NEXT:         "[[PREFIX]]/Inputs/header2.h",
 // CHECK-NEXT:         "[[PREFIX]]/Inputs/module.modulemap"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "name": "header2"
 // CHECK-NEXT:     }
 // CHECK-NEXT:   ],
 // CHECK-NEXT:   "translation-units": [
 // CHECK-NEXT:     {
-// CHECK-NEXT:       "clang-context-hash": "[[HASH_TU:[A-Z0-9]+]]",
-// CHECK-NEXT:       "clang-module-deps": [
-// CHECK-NEXT:         {
-// CHECK-NEXT:           "context-hash": "[[HASH_H1]]",
-// CHECK-NEXT:           "module-name": "header1"
-// CHECK-NEXT:         }
-// CHECK-NEXT:       ],
-// CHECK-NEXT:       "command-line": [
-// CHECK:              "-fno-implicit-modules"
-// CHECK:              "-fno-implicit-module-maps"
-// CHECK:              "-fmodule-file={{.*}}[[PREFIX]]/module-cache{{(_clangcl)?}}/[[HASH_H1]]/header1-{{[A-Z0-9]+}}.pcm"
-// CHECK:            ],
-// CHECK-NEXT:       "file-deps": [
-// CHECK-NEXT:         "[[PREFIX]]/modules_cdb_input.cpp"
-// CHECK-NEXT:       ],
-// CHECK-NEXT:       "input-file": "[[PREFIX]]/modules_cdb_input.cpp"
+// CHECK-NEXT:       "commands": [
+// CHECK-NEXT:         {
+// CHECK-NEXT:           "clang-context-hash": "[[HASH_TU:[A-Z0-9]+]]",
+// CHECK-NEXT:           "clang-module-deps": [
+// CHECK-NEXT:             {
+// CHECK-NEXT:               "context-hash": "[[HASH_H1]]",
+// CHECK-NEXT:               "module-name": "header1"
+// CHECK-NEXT:             }
+// CHECK-NEXT:           ],
+// CHECK-NEXT:           "command-line": [
+// CHECK-NOT:              "-fimplicit-modules"
+// CHECK-NOT:              "-fimplicit-module-maps"
+// CHECK:                  "-fmodule-file={{.*}}[[PREFIX]]/module-cache{{(_clangcl)?}}/[[HASH_H1]]/header1-{{[A-Z0-9]+}}.pcm"
+// CHECK:                ],
+// CHECK-NEXT:           "executable": "{{.*}}clang{{.*}}"
+// CHECK-NEXT:           "file-deps": [
+// CHECK-NEXT:             "[[PREFIX]]/modules_cdb_input.cpp"
+// CHECK-NEXT:           ],
+// CHECK-NEXT:           "input-file": "[[PREFIX]]/modules_cdb_input.cpp"
+// CHECK-NEXT:         }
+// CHECK-NEXT:       ]
 // CHECK-NEXT:     },
 // CHECK-NEXT:     {
-// CHECK-NEXT:       "clang-context-hash": "[[HASH_TU:[A-Z0-9]+]]",
-// CHECK-NEXT:       "clang-module-deps": [
-// CHECK-NEXT:         {
-// CHECK-NEXT:           "context-hash": "[[HASH_H1]]",
-// CHECK-NEXT:           "module-name": "header1"
-// CHECK-NEXT:         }
-// CHECK-NEXT:       ],
-// CHECK-NEXT:       "command-line": [
-// CHECK:              "-fno-implicit-modules"
-// CHECK:              "-fno-implicit-module-maps"
-// CHECK:              "-fmodule-file={{.*}}[[PREFIX]]/module-cache{{(_clangcl)?}}/[[HASH_H1]]/header1-{{[A-Z0-9]+}}.pcm"
-// CHECK:            ],
-// CHECK-NEXT:       "file-deps": [
-// CHECK-NEXT:         "[[PREFIX]]/modules_cdb_input.cpp"
-// CHECK-NEXT:       ],
-// CHECK-NEXT:       "input-file": "[[PREFIX]]/modules_cdb_input.cpp"
+// CHECK-NEXT:       "commands": [
+// CHECK-NEXT:         {
+// CHECK-NEXT:           "clang-context-hash": "[[HASH_TU:[A-Z0-9]+]]",
+// CHECK-NEXT:           "clang-module-deps": [
+// CHECK-NEXT:             {
+// CHECK-NEXT:               "context-hash": "[[HASH_H1]]",
+// CHECK-NEXT:               "module-name": "header1"
+// CHECK-NEXT:             }
+// CHECK-NEXT:           ],
+// CHECK-NEXT:           "command-line": [
+// CHECK-NOT:              "-fimplicit-modules"
+// CHECK-NOT:              "-fimplicit-module-maps"
+// CHECK:                  "-fmodule-file={{.*}}[[PREFIX]]/module-cache{{(_clangcl)?}}/[[HASH_H1]]/header1-{{[A-Z0-9]+}}.pcm"
+// CHECK:                ],
+// CHECK-NEXT:           "executable": "{{.*}}clang{{.*}}"
+// CHECK-NEXT:           "file-deps": [
+// CHECK-NEXT:             "[[PREFIX]]/modules_cdb_input.cpp"
+// CHECK-NEXT:           ],
+// CHECK-NEXT:           "input-file": "[[PREFIX]]/modules_cdb_input.cpp"
+// CHECK-NEXT:         }
+// CHECK-NEXT:       ]
 // CHECK-NEXT:     },
 // CHECK-NEXT:     {
-// CHECK-NEXT:       "clang-context-hash": "[[HASH_TU:[A-Z0-9]+]]",
-// CHECK-NEXT:       "clang-module-deps": [
-// CHECK-NEXT:         {
-// CHECK-NEXT:           "context-hash": "[[HASH_H1]]",
-// CHECK-NEXT:           "module-name": "header1"
-// CHECK-NEXT:         }
-// CHECK-NEXT:       ],
-// CHECK-NEXT:       "command-line": [
-// CHECK:              "-fno-implicit-modules"
-// CHECK:              "-fno-implicit-module-maps"
-// CHECK:              "-fmodule-file={{.*}}[[PREFIX]]/module-cache{{(_clangcl)?}}/[[HASH_H1]]/header1-{{[A-Z0-9]+}}.pcm"
-// CHECK:            ],
-// CHECK-NEXT:       "file-deps": [
-// CHECK-NEXT:         "[[PREFIX]]/modules_cdb_input.cpp"
-// CHECK-NEXT:       ],
-// CHECK-NEXT:       "input-file": "[[PREFIX]]/modules_cdb_input.cpp"
+// CHECK-NEXT:       "commands": [
+// CHECK-NEXT:         {
+// CHECK-NEXT:           "clang-context-hash": "[[HASH_TU:[A-Z0-9]+]]",
+// CHECK-NEXT:           "clang-module-deps": [
+// CHECK-NEXT:             {
+// CHECK-NEXT:               "context-hash": "[[HASH_H1]]",
+// CHECK-NEXT:               "module-name": "header1"
+// CHECK-NEXT:             }
+// CHECK-NEXT:           ],
+// CHECK-NEXT:           "command-line": [
+// CHECK-NOT:              "-fimplicit-modules"
+// CHECK-NOT:              "-fimplicit-module-maps"
+// CHECK:                  "-fmodule-file={{.*}}[[PREFIX]]/module-cache{{(_clangcl)?}}/[[HASH_H1]]/header1-{{[A-Z0-9]+}}.pcm"
+// CHECK:                ],
+// CHECK-NEXT:           "executable": "{{.*}}clang{{.*}}"
+// CHECK-NEXT:           "file-deps": [
+// CHECK-NEXT:             "[[PREFIX]]/modules_cdb_input.cpp"
+// CHECK-NEXT:           ],
+// CHECK-NEXT:           "input-file": "[[PREFIX]]/modules_cdb_input.cpp"
+// CHECK-NEXT:         }
+// CHECK-NEXT:       ]
 // CHECK-NEXT:     },
 // CHECK-NEXT:     {
-// CHECK-NEXT:       "clang-context-hash": "[[HASH_TU_DINCLUDE:[A-Z0-9]+]]",
-// CHECK-NEXT:       "clang-module-deps": [
-// CHECK-NEXT:         {
-// CHECK-NEXT:           "context-hash": "[[HASH_H1_DINCLUDE]]",
-// CHECK-NEXT:           "module-name": "header1"
-// CHECK-NEXT:         }
-// CHECK-NEXT:       ],
-// CHECK-NEXT:       "command-line": [
-// CHECK:              "-fno-implicit-modules"
-// CHECK:              "-fno-implicit-module-maps"
-// CHECK:              "-fmodule-file={{.*}}[[PREFIX]]/module-cache{{(_clangcl)?}}/[[HASH_H1_DINCLUDE]]/header1-{{[A-Z0-9]+}}.pcm"
-// CHECK:            ],
-// CHECK-NEXT:       "file-deps": [
-// CHECK-NEXT:         "[[PREFIX]]/modules_cdb_input2.cpp"
-// CHECK-NEXT:       ],
-// CHECK-NEXT:       "input-file": "[[PREFIX]]/modules_cdb_input2.cpp"
-// CHECK-NEXT:     }
-// CHECK-NEXT:   ]
-// CHECK-NEXT: }
+// CHECK-NEXT:       "commands": [
+// CHECK-NEXT:         {
+// CHECK-NEXT:           "clang-context-hash": "[[HASH_TU_DINCLUDE:[A-Z0-9]+]]",
+// CHECK-NEXT:           "clang-module-deps": [
+// CHECK-NEXT:             {
+// CHECK-NEXT:               "context-hash": "[[HASH_H1_DINCLUDE]]",
+// CHECK-NEXT:               "module-name": "header1"
+// CHECK-NEXT:             }
+// CHECK-NEXT:           ],
+// CHECK-NEXT:           "command-line": [
+// CHECK-NOT:              "-fimplicit-modules"
+// CHECK-NOT:              "-fimplicit-module-maps"
+// CHECK:                  "-fmodule-file={{.*}}[[PREFIX]]/module-cache{{(_clangcl)?}}/[[HASH_H1_DINCLUDE]]/header1-{{[A-Z0-9]+}}.pcm"
+// CHECK:                ],
+// CHECK-NEXT:           "executable": "{{.*}}clang{{.*}}"
+// CHECK-NEXT:           "file-deps": [
+// CHECK-NEXT:             "[[PREFIX]]/modules_cdb_input2.cpp"
+// CHECK-NEXT:           ],
+// CHECK-NEXT:           "input-file": "[[PREFIX]]/modules_cdb_input2.cpp"
+// CHECK-NEXT:         }
+// CHECK-NEXT:       ]
+// CHECK-NEXT:     }
+// CHECK-NEXT:   ]
+// CHECK-NEXT: }

clang/test/ClangScanDeps/modules-implicit-dot-private.m

 @import FW.Private;
 
 // RUN: sed -e "s|DIR|%/t|g" %t/cdb.json.template > %t/cdb.json
 // RUN: clang-scan-deps -compilation-database %t/cdb.json -format experimental-full > %t/result.json
 // RUN: cat %t/result.json | sed 's:\\\\\?:/:g' | FileCheck %s -DPREFIX=%/t
 // CHECK:      {
 // CHECK-NEXT:   "modules": [
 // CHECK-NEXT:     {
-// CHECK-NEXT:       "clang-module-deps": [],
+// CHECK:            "clang-module-deps": [],
 // CHECK-NEXT:       "clang-modulemap-file": "[[PREFIX]]/frameworks/FW.framework/Modules/module.modulemap",
 // CHECK-NEXT:       "command-line": [
 // CHECK:            ],
 // CHECK-NEXT:       "context-hash": "{{.*}}",
-// CHECK-NEXT:       "file-deps": [
+// CHECK:            "file-deps": [
 // CHECK-NEXT:         "[[PREFIX]]/frameworks/FW.framework/Headers/FW.h",
 // CHECK-NEXT:         "[[PREFIX]]/frameworks/FW.framework/Modules/module.modulemap",
 // CHECK-NEXT:         "[[PREFIX]]/frameworks/FW.framework/Modules/module.private.modulemap"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "name": "FW"
-// CHECK-NEXT:     },
-// CHECK-NEXT:     {
-// CHECK-NEXT:       "clang-module-deps": [],
+// CHECK:          },
+// CHECK:          {
+// CHECK:            "clang-module-deps": [],
 // CHECK-NEXT:       "clang-modulemap-file": "[[PREFIX]]/frameworks/FW.framework/Modules/module.private.modulemap",
 // CHECK-NEXT:       "command-line": [
 // CHECK:            ],
 // CHECK-NEXT:       "context-hash": "{{.*}}",
-// CHECK-NEXT:       "file-deps": [
+// CHECK:            "file-deps": [
 // CHECK-NEXT:         "[[PREFIX]]/frameworks/FW.framework/Modules/module.modulemap",
 // CHECK-NEXT:         "[[PREFIX]]/frameworks/FW.framework/Modules/module.private.modulemap",
 // CHECK-NEXT:         "[[PREFIX]]/frameworks/FW.framework/PrivateHeaders/FW_Private.h"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "name": "FW_Private"
-// CHECK-NEXT:     }
-// CHECK-NEXT:   ],
+// CHECK:           }
+// CHECK:        ],
 // CHECK-NEXT:   "translation-units": [
 // CHECK-NEXT:     {
-// CHECK-NEXT:       "clang-context-hash": "{{.*}}",
+// CHECK:            "clang-context-hash": "{{.*}}",
 // CHECK-NEXT:       "clang-module-deps": [
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "{{.*}}",
 // CHECK-NEXT:           "module-name": "FW"
 // CHECK-NEXT:         },
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "{{.*}}",
 // CHECK-NEXT:           "module-name": "FW_Private"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "command-line": [
-// CHECK:              "-fmodule-file={{.*}}/FW_Private-{{.*}}.pcm"
 // CHECK:              "-fmodule-file={{.*}}/FW-{{.*}}.pcm"
+// CHECK:              "-fmodule-file={{.*}}/FW_Private-{{.*}}.pcm"
 // CHECK:            ],
-// CHECK-NEXT:       "file-deps": [
+// CHECK:            "file-deps": [
 // CHECK-NEXT:         "[[PREFIX]]/tu.m"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "input-file": "[[PREFIX]]/tu.m"
 // CHECK-NEXT:     }
-// CHECK-NEXT:   ]
-// CHECK-NEXT: }
 
 // RUN: %deps-to-rsp %t/result.json --module-name=FW > %t/FW.cc1.rsp
 // RUN: %deps-to-rsp %t/result.json --module-name=FW_Private > %t/FW_Private.cc1.rsp
 // RUN: %deps-to-rsp %t/result.json --tu-index=0 > %t/tu.rsp
 // RUN: %clang @%t/FW.cc1.rsp
 // RUN: %clang @%t/FW_Private.cc1.rsp
 // RUN: %clang @%t/tu.rsp

clang/test/ClangScanDeps/modules-incomplete-umbrella.c

 // CHECK_TU-NEXT:         "[[PREFIX]]/frameworks/FW.framework/PrivateHeaders/FW_Private.h",
 // CHECK_TU-NEXT:         "[[PREFIX]]/frameworks/FW.framework/PrivateHeaders/One.h"
 // CHECK_TU-NEXT:       ],
 // CHECK_TU-NEXT:       "name": "FW_Private"
 // CHECK_TU-NEXT:     }
 // CHECK_TU-NEXT:   ],
 // CHECK_TU-NEXT:   "translation-units": [
 // CHECK_TU-NEXT:     {
-// CHECK_TU-NEXT:       "clang-context-hash": "{{.*}}",
+// CHECK_TU:            "clang-context-hash": "{{.*}}",
 // CHECK_TU-NEXT:       "clang-module-deps": [
 // CHECK_TU-NEXT:         {
 // CHECK_TU-NEXT:           "context-hash": "{{.*}}",
 // CHECK_TU-NEXT:           "module-name": "FW"
 // CHECK_TU-NEXT:         },
 // CHECK_TU-NEXT:         {
 // CHECK_TU-NEXT:           "context-hash": "{{.*}}",
 // CHECK_TU-NEXT:           "module-name": "FW_Private"
 // CHECK_TU-NEXT:         }
 // CHECK_TU-NEXT:       ],
 // CHECK_TU-NEXT:       "command-line": [
 // CHECK_TU:              "-fmodule-file={{.*}}/FW-{{.*}}.pcm"
 // CHECK_TU:              "-fmodule-file={{.*}}/FW_Private-{{.*}}.pcm"
 // CHECK_TU:            ],
-// CHECK_TU-NEXT:       "file-deps": [
+// CHECK_TU:            "file-deps": [
 // CHECK_TU-NEXT:         "[[PREFIX]]/from_tu.m",
 // CHECK_TU-NEXT:         "[[PREFIX]]/frameworks/FW.framework/PrivateHeaders/Two.h"
 // CHECK_TU-NEXT:       ],
 // CHECK_TU-NEXT:       "input-file": "[[PREFIX]]/from_tu.m"
 // CHECK_TU-NEXT:     }
-// CHECK_TU-NEXT:   ]
-// CHECK_TU-NEXT: }
 
 // RUN: %deps-to-rsp %t/from_tu_result.json --module-name=FW > %t/FW.cc1.rsp
 // RUN: %deps-to-rsp %t/from_tu_result.json --module-name=FW_Private > %t/FW_Private.cc1.rsp
 // RUN: %deps-to-rsp %t/from_tu_result.json --tu-index=0 > %t/tu.rsp
 // RUN: %clang @%t/FW.cc1.rsp
 // RUN: %clang @%t/FW_Private.cc1.rsp
 // RUN: %clang @%t/tu.rsp
 

 // CHECK_MODULE-NEXT:         "[[PREFIX]]/frameworks/FW.framework/PrivateHeaders/Two.h",
 // CHECK_MODULE-NEXT:         "[[PREFIX]]/module.modulemap"
 // CHECK_MODULE-NEXT:       ],
 // CHECK_MODULE-NEXT:       "name": "Mod"
 // CHECK_MODULE-NEXT:     }
 // CHECK_MODULE-NEXT:   ],
 // CHECK_MODULE-NEXT:   "translation-units": [
 // CHECK_MODULE-NEXT:     {
-// CHECK_MODULE-NEXT:       "clang-context-hash": "{{.*}}",
+// CHECK_MODULE:            "clang-context-hash": "{{.*}}",
 // CHECK_MODULE-NEXT:       "clang-module-deps": [
 // CHECK_MODULE-NEXT:         {
 // CHECK_MODULE-NEXT:           "context-hash": "{{.*}}",
 // CHECK_MODULE-NEXT:           "module-name": "Mod"
 // CHECK_MODULE-NEXT:         }
 // CHECK_MODULE-NEXT:       ],
 // CHECK_MODULE-NEXT:       "command-line": [
 // CHECK_MODULE:            ],
-// CHECK_MODULE-NEXT:       "file-deps": [
+// CHECK_MODULE:            "file-deps": [
 // CHECK_MODULE-NEXT:         "[[PREFIX]]/from_module.m"
 // CHECK_MODULE-NEXT:       ],
 // CHECK_MODULE-NEXT:       "input-file": "[[PREFIX]]/from_module.m"
 // CHECK_MODULE-NEXT:     }
-// CHECK_MODULE-NEXT:   ]
-// CHECK_MODULE-NEXT: }
 
 // RUN: %deps-to-rsp %t/from_module_result.json --module-name=FW > %t/FW.cc1.rsp
 // RUN: %deps-to-rsp %t/from_module_result.json --module-name=FW_Private > %t/FW_Private.cc1.rsp
 // RUN: %deps-to-rsp %t/from_module_result.json --module-name=Mod > %t/Mod.cc1.rsp
 // RUN: %deps-to-rsp %t/from_module_result.json --tu-index=0 > %t/tu.rsp
 // RUN: %clang @%t/FW.cc1.rsp
 // RUN: %clang @%t/FW_Private.cc1.rsp
 // RUN: %clang @%t/Mod.cc1.rsp
 // RUN: %clang @%t/tu.rsp

clang/test/ClangScanDeps/modules-inferred.m

 // CHECK-NEXT:         "[[SOURCEDIR]]/Inputs/frameworks/Inferred.framework/Headers/Inferred.h",
 // CHECK-NEXT:         "[[SOURCEDIR]]/Inputs/frameworks/module.modulemap"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "name": "Inferred"
 // CHECK-NEXT:     }
 // CHECK-NEXT:   ],
 // CHECK-NEXT:   "translation-units": [
 // CHECK-NEXT:     {
-// CHECK-NEXT:       "clang-context-hash": "[[HASH_TU:[A-Z0-9]+]]",
+// CHECK:            "clang-context-hash": "[[HASH_TU:[A-Z0-9]+]]",
 // CHECK-NEXT:       "clang-module-deps": [
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "[[HASH_INFERRED]]",
 // CHECK-NEXT:           "module-name": "Inferred"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "command-line": [
 // CHECK:            ],
-// CHECK-NEXT:       "file-deps": [
+// CHECK:            "file-deps": [
 // CHECK-NEXT:         "[[PREFIX]]/modules_cdb_input.cpp"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "input-file": "[[PREFIX]]/modules_cdb_input.cpp"
 // CHECK-NEXT:     }
-// CHECK-NEXT:   ]
-// CHECK-NEXT: }

clang/test/ClangScanDeps/modules-no-undeclared-includes.c

 // CHECK-NEXT:         "[[PREFIX]]/undeclared/module.modulemap",
 // CHECK-NEXT:         "[[PREFIX]]/user.h"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "name": "User"
 // CHECK-NEXT:     }
 // CHECK-NEXT:   ],
 // CHECK-NEXT:   "translation-units": [
 // CHECK-NEXT:     {
-// CHECK-NEXT:       "clang-context-hash": "{{.*}}"
+// CHECK:            "clang-context-hash": "{{.*}}"
 // CHECK-NEXT:       "clang-module-deps": [
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "{{.*}}"
 // CHECK-NEXT:           "module-name": "User"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "command-line": [
 // CHECK:            ],
-// CHECK-NEXT:       "file-deps": [
+// CHECK:            "file-deps": [
 // CHECK-NEXT:         "[[PREFIX]]/test.c"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "input-file": "[[PREFIX]]/test.c"
 // CHECK-NEXT:     }
-// CHECK:        ]
-// CHECK-NEXT: }
 
 // RUN: %deps-to-rsp %t/result.json --module-name=User > %t/User.cc1.rsp
 // RUN: %deps-to-rsp %t/result.json --tu-index=0 > %t/tu.rsp
 //
 // RUN: %clang @%t/User.cc1.rsp
 // RUN: %clang @%t/tu.rsp

clang/test/ClangScanDeps/modules-pch-common-submodule.c

 // CHECK-PCH-NEXT:         "[[PREFIX]]/mod_common_sub.h",
 // CHECK-PCH-NEXT:         "[[PREFIX]]/module.modulemap"
 // CHECK-PCH-NEXT:       ],
 // CHECK-PCH-NEXT:       "name": "ModCommon"
 // CHECK-PCH-NEXT:     }
 // CHECK-PCH-NEXT:   ],
 // CHECK-PCH-NEXT:   "translation-units": [
 // CHECK-PCH-NEXT:     {
-// CHECK-PCH-NEXT:       "clang-context-hash": "[[HASH_PCH:.*]]",
+// CHECK-PCH:            "clang-context-hash": "[[HASH_PCH:.*]]",
 // CHECK-PCH-NEXT:       "clang-module-deps": [
 // CHECK-PCH-NEXT:         {
 // CHECK-PCH-NEXT:           "context-hash": "[[HASH_MOD_COMMON]]",
 // CHECK-PCH-NEXT:           "module-name": "ModCommon"
 // CHECK-PCH-NEXT:         }
 // CHECK-PCH-NEXT:       ],
 // CHECK-PCH-NEXT:       "command-line": [
 // CHECK-PCH:            ],
-// CHECK-PCH-NEXT:       "file-deps": [
+// CHECK-PCH:            "file-deps": [
 // CHECK-PCH-NEXT:         "[[PREFIX]]/pch.h"
 // CHECK-PCH-NEXT:       ],
 // CHECK-PCH-NEXT:       "input-file": "[[PREFIX]]/pch.h"
 // CHECK-PCH-NEXT:     }
-// CHECK-PCH-NEXT:   ]
-// CHECK-PCH-NEXT: }
 
 // Explicitly build the PCH:
 //
 // RUN: %deps-to-rsp %t/result_pch.json --module-name=ModCommon > %t/mod_common.cc1.rsp
 // RUN: %deps-to-rsp %t/result_pch.json --tu-index=0 > %t/pch.rsp
 //
 // RUN: %clang @%t/mod_common.cc1.rsp
 // RUN: %clang @%t/pch.rsp

 // CHECK-TU-NEXT:         "[[PREFIX]]/mod_tu.h",
 // CHECK-TU-NEXT:         "[[PREFIX]]/module.modulemap"
 // CHECK-TU-NEXT:       ],
 // CHECK-TU-NEXT:       "name": "ModTU"
 // CHECK-TU-NEXT:     }
 // CHECK-TU-NEXT:   ],
 // CHECK-TU-NEXT:   "translation-units": [
 // CHECK-TU-NEXT:     {
-// CHECK-TU-NEXT:       "clang-context-hash": "[[HASH_TU:.*]]",
+// CHECK-TU:            "clang-context-hash": "[[HASH_TU:.*]]",
 // CHECK-TU-NEXT:       "clang-module-deps": [
 // CHECK-TU-NEXT:         {
 // CHECK-TU-NEXT:           "context-hash": "[[HASH_MOD_TU]]"
 // CHECK-TU-NEXT:           "module-name": "ModTU"
 // CHECK-TU-NEXT:         }
 // CHECK-TU-NEXT:       ],
 // CHECK-TU-NEXT:       "command-line": [
 // CHECK-TU:            ],
-// CHECK-TU-NEXT:       "file-deps": [
+// CHECK-TU:            "file-deps": [
 // CHECK-TU-NEXT:         "[[PREFIX]]/tu.c",
 // CHECK-TU-NEXT:         "[[PREFIX]]/pch.h.gch"
 // CHECK-TU-NEXT:       ],
 // CHECK-TU-NEXT:       "input-file": "[[PREFIX]]/tu.c"
 // CHECK-TU-NEXT:     }
-// CHECK-TU-NEXT:   ]
-// CHECK-TU-NEXT: }
 
 // Explicitly build the TU:
 //
 // RUN: %deps-to-rsp %t/result_tu.json --module-name=ModTU > %t/mod_tu.cc1.rsp
 // RUN: %deps-to-rsp %t/result_tu.json --tu-index=0 > %t/tu.rsp
 //
 // RUN: %clang @%t/mod_tu.cc1.rsp
 // RUN: %clang @%t/tu.rsp

clang/test/ClangScanDeps/modules-pch-common-via-submodule.c

 // CHECK-PCH-NEXT:         "[[PREFIX]]/mod_common.h",
 // CHECK-PCH-NEXT:         "[[PREFIX]]/module.modulemap"
 // CHECK-PCH-NEXT:       ],
 // CHECK-PCH-NEXT:       "name": "ModCommon"
 // CHECK-PCH-NEXT:     }
 // CHECK-PCH-NEXT:   ],
 // CHECK-PCH-NEXT:   "translation-units": [
 // CHECK-PCH-NEXT:     {
-// CHECK-PCH-NEXT:       "clang-context-hash": "[[HASH_PCH:.*]]",
+// CHECK-PCH:            "clang-context-hash": "[[HASH_PCH:.*]]",
 // CHECK-PCH-NEXT:       "clang-module-deps": [
 // CHECK-PCH-NEXT:         {
 // CHECK-PCH-NEXT:           "context-hash": "[[HASH_MOD_COMMON]]",
 // CHECK-PCH-NEXT:           "module-name": "ModCommon"
 // CHECK-PCH-NEXT:         }
 // CHECK-PCH-NEXT:       ],
 // CHECK-PCH-NEXT:       "command-line": [
 // CHECK-PCH:            ],
-// CHECK-PCH-NEXT:       "file-deps": [
+// CHECK-PCH:            "file-deps": [
 // CHECK-PCH-NEXT:         "[[PREFIX]]/pch.h"
 // CHECK-PCH-NEXT:       ],
 // CHECK-PCH-NEXT:       "input-file": "[[PREFIX]]/pch.h"
 // CHECK-PCH-NEXT:     }
-// CHECK-PCH-NEXT:   ]
-// CHECK-PCH-NEXT: }
 
 // Explicitly build the PCH:
 //
 // RUN: %deps-to-rsp %t/result_pch.json --module-name=ModCommon > %t/mod_common.cc1.rsp
 // RUN: %deps-to-rsp %t/result_pch.json --tu-index=0 > %t/pch.rsp
 //
 // RUN: %clang @%t/mod_common.cc1.rsp
 // RUN: %clang @%t/pch.rsp

 // CHECK-TU-NEXT:         "[[PREFIX]]/mod_tu_sub.h",
 // CHECK-TU-NEXT:         "[[PREFIX]]/module.modulemap"
 // CHECK-TU-NEXT:       ],
 // CHECK-TU-NEXT:       "name": "ModTU"
 // CHECK-TU-NEXT:     }
 // CHECK-TU-NEXT:   ],
 // CHECK-TU-NEXT:   "translation-units": [
 // CHECK-TU-NEXT:     {
-// CHECK-TU-NEXT:       "clang-context-hash": "[[HASH_TU:.*]]",
+// CHECK-TU:            "clang-context-hash": "[[HASH_TU:.*]]",
 // CHECK-TU-NEXT:       "clang-module-deps": [
 // CHECK-TU-NEXT:         {
 // CHECK-TU-NEXT:           "context-hash": "[[HASH_MOD_TU]]"
 // CHECK-TU-NEXT:           "module-name": "ModTU"
 // CHECK-TU-NEXT:         }
 // CHECK-TU-NEXT:       ],
 // CHECK-TU-NEXT:       "command-line": [
 // CHECK-TU:            ],
-// CHECK-TU-NEXT:       "file-deps": [
+// CHECK-TU:            "file-deps": [
 // CHECK-TU-NEXT:         "[[PREFIX]]/tu.c",
 // CHECK-TU-NEXT:         "[[PREFIX]]/pch.h.gch"
 // CHECK-TU-NEXT:       ],
 // CHECK-TU-NEXT:       "input-file": "[[PREFIX]]/tu.c"
 // CHECK-TU-NEXT:     }
-// CHECK-TU-NEXT:   ]
-// CHECK-TU-NEXT: }
 
 // Explicitly build the TU:
 //
 // RUN: %deps-to-rsp %t/result_tu.json --module-name=ModTU > %t/mod_tu.cc1.rsp
 // RUN: %deps-to-rsp %t/result_tu.json --tu-index=0 > %t/tu.rsp
 //
 // RUN: %clang @%t/mod_tu.cc1.rsp
 // RUN: %clang @%t/tu.rsp

clang/test/ClangScanDeps/modules-pch.c

 // CHECK-PCH-NEXT:         "[[PREFIX]]/mod_pch.h",
 // CHECK-PCH-NEXT:         "[[PREFIX]]/module.modulemap"
 // CHECK-PCH-NEXT:       ],
 // CHECK-PCH-NEXT:       "name": "ModPCH"
 // CHECK-PCH-NEXT:     }
 // CHECK-PCH-NEXT:   ],
 // CHECK-PCH-NEXT:   "translation-units": [
 // CHECK-PCH-NEXT:     {
-// CHECK-PCH-NEXT:       "clang-context-hash": "[[HASH_PCH:.*]]",
+// CHECK-PCH:            "clang-context-hash": "[[HASH_PCH:.*]]",
 // CHECK-PCH-NEXT:       "clang-module-deps": [
 // CHECK-PCH-NEXT:         {
 // CHECK-PCH-NEXT:           "context-hash": "[[HASH_MOD_COMMON_1]]",
 // CHECK-PCH-NEXT:           "module-name": "ModCommon1"
 // CHECK-PCH-NEXT:         },
 // CHECK-PCH-NEXT:         {
 // CHECK-PCH-NEXT:           "context-hash": "[[HASH_MOD_PCH]]",
 // CHECK-PCH-NEXT:           "module-name": "ModPCH"
 // CHECK-PCH-NEXT:         }
 // CHECK-PCH-NEXT:       ],
 // CHECK-PCH-NEXT:       "command-line": [
 // CHECK-PCH:            ],
-// CHECK-PCH-NEXT:       "file-deps": [
+// CHECK-PCH:            "file-deps": [
 // CHECK-PCH-NEXT:         "[[PREFIX]]/pch.h"
 // CHECK-PCH-NEXT:       ],
 // CHECK-PCH-NEXT:       "input-file": "[[PREFIX]]/pch.h"
 // CHECK-PCH-NEXT:     }
-// CHECK-PCH-NEXT:   ]
-// CHECK-PCH-NEXT: }
 
 // Explicitly build the PCH:
 //
 // RUN: %deps-to-rsp %t/result_pch.json --module-name=ModCommon1 > %t/mod_common_1.cc1.rsp
 // RUN: %deps-to-rsp %t/result_pch.json --module-name=ModCommon2 > %t/mod_common_2.cc1.rsp
 // RUN: %deps-to-rsp %t/result_pch.json --module-name=ModPCH > %t/mod_pch.cc1.rsp
 // RUN: %deps-to-rsp %t/result_pch.json --tu-index=0 > %t/pch.rsp
 //

 // CHECK-TU-NEXT:         "[[PREFIX]]/mod_tu.h",
 // CHECK-TU-NEXT:         "[[PREFIX]]/module.modulemap"
 // CHECK-TU-NEXT:       ],
 // CHECK-TU-NEXT:       "name": "ModTU"
 // CHECK-TU-NEXT:     }
 // CHECK-TU-NEXT:   ],
 // CHECK-TU-NEXT:   "translation-units": [
 // CHECK-TU-NEXT:     {
-// CHECK-TU-NEXT:       "clang-context-hash": "[[HASH_TU:.*]]",
+// CHECK-TU:            "clang-context-hash": "[[HASH_TU:.*]]",
 // CHECK-TU-NEXT:       "clang-module-deps": [
 // CHECK-TU-NEXT:         {
 // CHECK-TU-NEXT:           "context-hash": "[[HASH_MOD_TU]]",
 // CHECK-TU-NEXT:           "module-name": "ModTU"
 // CHECK-TU-NEXT:         }
 // CHECK-TU-NEXT:       ],
 // CHECK-TU-NEXT:       "command-line": [
 // CHECK-TU:            ],
-// CHECK-TU-NEXT:       "file-deps": [
+// CHECK-TU:            "file-deps": [
 // CHECK-TU-NEXT:         "[[PREFIX]]/tu.c",
 // CHECK-TU-NEXT:         "[[PREFIX]]/pch.h.gch"
 // CHECK-TU-NEXT:       ],
 // CHECK-TU-NEXT:       "input-file": "[[PREFIX]]/tu.c"
 // CHECK-TU-NEXT:     }
-// CHECK-TU-NEXT:   ]
-// CHECK-TU-NEXT: }
 
 // Explicitly build the TU:
 //
 // RUN: %deps-to-rsp %t/result_tu.json --module-name=ModTU > %t/mod_tu.cc1.rsp
 // RUN: %deps-to-rsp %t/result_tu.json --tu-index=0 > %t/tu.rsp
 //
 // RUN: %clang @%t/mod_tu.cc1.rsp
 // RUN: %clang @%t/tu.rsp

 // CHECK-TU-WITH-COMMON-NEXT:         "[[PREFIX]]/mod_tu_with_common.h",
 // CHECK-TU-WITH-COMMON-NEXT:         "[[PREFIX]]/module.modulemap"
 // CHECK-TU-WITH-COMMON-NEXT:       ],
 // CHECK-TU-WITH-COMMON-NEXT:       "name": "ModTUWithCommon"
 // CHECK-TU-WITH-COMMON-NEXT:     }
 // CHECK-TU-WITH-COMMON-NEXT:   ],
 // CHECK-TU-WITH-COMMON-NEXT:   "translation-units": [
 // CHECK-TU-WITH-COMMON-NEXT:     {
-// CHECK-TU-WITH-COMMON-NEXT:       "clang-context-hash": "[[HASH_TU_WITH_COMMON:.*]]",
+// CHECK-TU-WITH-COMMON:            "clang-context-hash": "[[HASH_TU_WITH_COMMON:.*]]",
 // CHECK-TU-WITH-COMMON-NEXT:       "clang-module-deps": [
 // CHECK-TU-WITH-COMMON-NEXT:         {
 // CHECK-TU-WITH-COMMON-NEXT:           "context-hash": "[[HASH_MOD_TU_WITH_COMMON]]",
 // CHECK-TU-WITH-COMMON-NEXT:           "module-name": "ModTUWithCommon"
 // CHECK-TU-WITH-COMMON-NEXT:         }
 // CHECK-TU-WITH-COMMON-NEXT:       ],
 // CHECK-TU-WITH-COMMON-NEXT:       "command-line": [
 // CHECK-TU-WITH-COMMON:              "-fmodule-file=[[PREFIX]]/build/{{.*}}/ModCommon2-{{.*}}.pcm"
 // CHECK-TU-WITH-COMMON:            ],
-// CHECK-TU-WITH-COMMON-NEXT:       "file-deps": [
+// CHECK-TU-WITH-COMMON:            "file-deps": [
 // CHECK-TU-WITH-COMMON-NEXT:         "[[PREFIX]]/tu_with_common.c",
 // CHECK-TU-WITH-COMMON-NEXT:         "[[PREFIX]]/pch.h.gch"
 // CHECK-TU-WITH-COMMON-NEXT:       ],
 // CHECK-TU-WITH-COMMON-NEXT:       "input-file": "[[PREFIX]]/tu_with_common.c"
 // CHECK-TU-WITH-COMMON-NEXT:     }
-// CHECK-TU-WITH-COMMON-NEXT:   ]
-// CHECK-TU-WITH-COMMON-NEXT: }
 
 // Explicitly build the TU that has common modules with the PCH:
 //
 // RUN: %deps-to-rsp %t/result_tu_with_common.json --module-name=ModTUWithCommon > %t/mod_tu_with_common.cc1.rsp
 // RUN: %deps-to-rsp %t/result_tu_with_common.json --tu-index=0 > %t/tu_with_common.rsp
 //
 // RUN: %clang @%t/mod_tu_with_common.cc1.rsp
 // RUN: %clang @%t/tu_with_common.rsp

clang/test/ClangScanDeps/multiple-commands.c

+// Test scanning when the driver requires multiple jobs. E.g. with -save-temps
+// there will be separate -E, -emit-llvm-bc, -S, and -cc1as jobs, which should
+// each result in a "command" in the output.
+
+// RUN: rm -rf %t
+// RUN: split-file %s %t
+// RUN: sed -e "s|DIR|%/t|g" %t/cdb.json.in > %t/cdb.json
+
+// RUN: clang-scan-deps -compilation-database %t/cdb.json -module-files-dir %t/modules \
+// RUN:   -j 1 -format experimental-full -mode preprocess-dependency-directives \
+// RUN:   > %t/deps.json
+
+// RUN: cat %t/deps.json | sed 's:\\\\\?:/:g' | FileCheck %s -DPREFIX=%/t
+
+// Build the -save-temps + -fmodules case
+// RUN: %deps-to-rsp %t/deps.json --module-name=Mod > %t/Mod.rsp
+// RUN: %deps-to-rsp %t/deps.json --tu-index 1 --tu-cmd-index 0 > %t/tu-cpp.rsp
+// RUN: %deps-to-rsp %t/deps.json --tu-index 1 --tu-cmd-index 1 > %t/tu-emit-ir.rsp
+// RUN: %deps-to-rsp %t/deps.json --tu-index 1 --tu-cmd-index 2 > %t/tu-emit-asm.rsp
+// RUN: %deps-to-rsp %t/deps.json --tu-index 1 --tu-cmd-index 3 > %t/tu-cc1as.rsp
+// RUN: %clang @%t/Mod.rsp
+// RUN: %clang @%t/tu-cpp.rsp
+// RUN: ls %t/tu_save_temps_module.i
+// RUN: %clang @%t/tu-emit-ir.rsp
+// RUN: ls %t/tu_save_temps_module.bc
+// RUN: %clang @%t/tu-emit-asm.rsp
+// RUN: ls %t/tu_save_temps_module.s
+// RUN: %clang @%t/tu-cc1as.rsp
+// RUN: ls %t/tu_save_temps_module.o
+
+
+// CHECK:      "modules": [
+// CHECK-NEXT:   {
+// CHECK:          "clang-modulemap-file": "[[PREFIX]]{{.}}module.modulemap"
+// CHECK:          "name": "Mod"
+// CHECK:        }
+// CHECK-NEXT: ]
+// CHECK-NEXT: "translation-units": [
+// CHECK-NEXT:   {
+// CHECK:          "commands": [
+// CHECK-NEXT:       {
+// CHECK-NEXT:         "clang-context-hash":
+// CHECK-NEXT:         "clang-module-deps": []
+// CHECK-NEXT:         "command-line": [
+// CHECK-NEXT:           "-cc1"
+// CHECK:                "-o"
+// CHECK-NEXT:           "{{.*}}tu_no_integrated_cpp{{.*}}.i"
+// CHECK:                "-E"
+// CHECK:              ]
+// CHECK-NEXT:         "executable": "clang_tool"
+// CHECK:              "input-file": "[[PREFIX]]{{.}}tu_no_integrated_cpp.c"
+// CHECK-NEXT:       }
+// CHECK-NEXT:       {
+// CHECK-NEXT:         "clang-context-hash":
+// CHECK-NEXT:         "clang-module-deps": []
+// CHECK-NEXT:         "command-line": [
+// CHECK-NEXT:           "-cc1"
+// CHECK:                "-o"
+// CHECK-NEXT:           "{{.*}}tu_no_integrated_cpp.o"
+// CHECK:                "-emit-obj"
+// CHECK:                "{{.*}}tu_no_integrated_cpp{{.*}}.i"
+// CHECK:              ]
+// CHECK-NEXT:         "executable": "clang_tool"
+// CHECK:              "input-file": "[[PREFIX]]{{.}}tu_no_integrated_cpp.c"
+// CHECK-NEXT:       }
+// CHECK-NEXT:     ]
+// CHECK-NEXT:   }
+// CHECK-NEXT:   {
+// CHECK-NEXT:     "commands": [
+// CHECK-NEXT:       {
+// CHECK:              "clang-module-deps": [
+// CHECK-NEXT:           {
+// CHECK:                  "module-name": "Mod"
+// CHECK-NEXT:           }
+// CHECK-NEXT:         ]
+// CHECK-NEXT:         "command-line": [
+// CHECK-NEXT:           "-cc1"
+// CHECK:                "-o"
+// CHECK-NEXT:           "{{.*}}tu_save_temps_module.i"
+// CHECK:                "-E"
+// CHECK:                "-fmodule-file={{.*}}[[PREFIX]]{{.}}modules{{.*}}Mod-{{.*}}.pcm"
+// CHECK:                "{{.*}}tu_save_temps_module.c"
+// CHECK:              ]
+// CHECK-NEXT:         "executable": "clang_tool"
+// CHECK:              "input-file": "[[PREFIX]]{{.}}tu_save_temps_module.c"
+// CHECK-NEXT:       }
+// CHECK-NEXT:       {
+// CHECK:              "clang-module-deps": [
+// CHECK-NEXT:           {
+// CHECK:                  "module-name": "Mod"
+// CHECK-NEXT:           }
+// CHECK-NEXT:         ]
+// CHECK-NEXT:         "command-line": [
+// CHECK-NEXT:           "-cc1"
+// CHECK:                "-o"
+// CHECK-NEXT:           "{{.*}}tu_save_temps_module.bc"
+// CHECK:                "-emit-llvm-bc"
+// CHECK:                "{{.*}}tu_save_temps_module.i"
+// CHECK:                "-fmodule-file={{.*}}[[PREFIX]]{{.}}modules{{.*}}Mod-{{.*}}.pcm"
+// CHECK:              ]
+// CHECK-NEXT:         "executable": "clang_tool"
+// CHECK:              "input-file": "[[PREFIX]]{{.}}tu_save_temps_module.c"
+// CHECK-NEXT:       }
+// CHECK-NEXT:       {
+// CHECK:              "clang-module-deps": [
+// CHECK-NEXT:           {
+// CHECK:                  "module-name": "Mod"
+// CHECK-NEXT:           }
+// CHECK-NEXT:         ]
+// CHECK-NEXT:         "command-line": [
+// CHECK-NEXT:           "-cc1"
+// CHECK:                "-o"
+// CHECK-NEXT:           "{{.*}}tu_save_temps_module.s"
+// CHECK:                "-S"
+// CHECK:                "{{.*}}tu_save_temps_module.bc"
+// CHECK:              ]
+// CHECK-NEXT:         "executable": "clang_tool"
+// CHECK:              "input-file": "[[PREFIX]]{{.}}tu_save_temps_module.c"
+// CHECK-NEXT:       }
+// CHECK-NEXT:       {
+// CHECK:              "clang-module-deps": [
+// CHECK-NEXT:           {
+// CHECK:                  "module-name": "Mod"
+// CHECK-NEXT:           }
+// CHECK-NEXT:         ]
+// CHECK-NEXT:         "command-line": [
+// CHECK-NEXT:           "-cc1as"
+// CHECK:                "-o"
+// CHECK-NEXT:           "{{.*}}tu_save_temps_module.o"
+// CHECK:                "{{.*}}tu_save_temps_module.s"
+// CHECK:              ]
+// CHECK-NEXT:         "executable": "clang_tool"
+// CHECK:              "input-file": "[[PREFIX]]{{.}}tu_save_temps_module.c"
+// CHECK-NEXT:       }
+// CHECK-NEXT:     ]
+// CHECK-NEXT:   }
+// CHECK-NEXT: ]
+
+
+//--- cdb.json.in
+[
+  {
+    "directory": "DIR"
+    "command": "clang_tool -c DIR/tu_no_integrated_cpp.c -no-integrated-cpp -o DIR/tu_no_integrated_cpp.o"
+    "file": "DIR/tu_no_integrated_cpp.c"
+  },
+  {
+    "directory": "DIR"
+    "command": "clang_tool -c DIR/tu_save_temps_module.c -save-temps=obj -o DIR/tu_save_temps_module.o -fmodules -fimplicit-modules -fimplicit-module-maps"
+    "file": "DIR/tu_save_temps_module.c"
+  }
+]
+
+//--- plain_header.h
+void foo(void);
+
+//--- module_header.h
+void bar(void);
+
+//--- module.modulemap
+module Mod { header "module_header.h" }
+
+//--- tu_no_integrated_cpp.c
+#include "plain_header.h"
+void tu_no_integrated_cpp(void) { foo(); }
+
+//--- tu_save_temps_module.c
+#include "module_header.h"
+void tu_save_temps(void) { bar(); }

clang/test/ClangScanDeps/removed-args.c

 // CHECK-NEXT:         "[[PREFIX]]/mod_tu.h",
 // CHECK-NEXT:         "[[PREFIX]]/module.modulemap"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "name": "ModTU"
 // CHECK-NEXT:     }
 // CHECK-NEXT:   ],
 // CHECK-NEXT:   "translation-units": [
 // CHECK-NEXT:     {
-// CHECK-NEXT:       "clang-context-hash": "[[HASH_TU:.*]]",
+// CHECK:            "clang-context-hash": "[[HASH_TU:.*]]",
 // CHECK-NEXT:       "clang-module-deps": [
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "[[HASH_MOD_HEADER]]",
 // CHECK-NEXT:           "module-name": "ModHeader"
 // CHECK-NEXT:         },
 // CHECK-NEXT:         {
 // CHECK-NEXT:           "context-hash": "[[HASH_MOD_TU]]",
 // CHECK-NEXT:           "module-name": "ModTU"
 // CHECK-NEXT:         }
 // CHECK-NEXT:       ]
 // CHECK-NEXT:       "command-line": [
-// CHECK-NEXT:         "-fsyntax-only",
+// CHECK-NEXT:         "-cc1",
 // CHECK-NOT:          "-fmodules-cache-path=
 // CHECK-NOT:          "-fmodules-validate-once-per-build-session"
+// CHECK-NOT:          "-fbuild-session-timestamp=
 // CHECK-NOT:          "-fbuild-session-file=
 // CHECK-NOT:          "-fmodules-prune-interval=
 // CHECK-NOT:          "-fmodules-prune-after=
 // CHECK:            ],
-// CHECK:          }
-// CHECK-NEXT:   ]
-// CHECK-NEXT: }

clang/tools/clang-scan-deps/ClangScanDeps.cpp

llvm::cl::opt<std::string> ModuleName(
     llvm::cl::desc("the module of which the dependencies are to be computed"),
     llvm::cl::cat(DependencyScannerCategory));
 
 llvm::cl::list<std::string> ModuleDepTargets(
     "dependency-target",
     llvm::cl::desc("The names of dependency targets for the dependency file"),
     llvm::cl::cat(DependencyScannerCategory));
 
+llvm::cl::opt<bool> DeprecatedDriverCommand(
+    "deprecated-driver-command", llvm::cl::Optional,
+    llvm::cl::desc("use a single driver command to build the tu (deprecated)"),
+    llvm::cl::cat(DependencyScannerCategory));
+
 enum ResourceDirRecipeKind {
   RDRK_ModifyCompilerPath,
   RDRK_InvokeCompiler,
 };
 
 static llvm::cl::opt<ResourceDirRecipeKind> ResourceDirRecipe(
     "resource-dir-recipe",
     llvm::cl::desc("How to produce missing '-resource-dir' argument"),
static llvm::json::Array toJSONSorted(std::vector<ModuleID> V) {
   return Ret;
 }
 
 // Thread safe.
 class FullDeps {
 public:
   void mergeDeps(StringRef Input, FullDependenciesResult FDR,
                  size_t InputIndex) {
-    const FullDependencies &FD = FDR.FullDeps;
+    FullDependencies &FD = FDR.FullDeps;
 
     InputDeps ID;
     ID.FileName = std::string(Input);
     ID.ContextHash = std::move(FD.ID.ContextHash);
     ID.FileDeps = std::move(FD.FileDeps);
     ID.ModuleDeps = std::move(FD.ClangModuleDeps);
 
     std::unique_lock<std::mutex> ul(Lock);
     for (const ModuleDeps &MD : FDR.DiscoveredModules) {
       auto I = Modules.find({MD.ID, 0});
       if (I != Modules.end()) {
         I->first.InputIndex = std::min(I->first.InputIndex, InputIndex);
         continue;
       }
       Modules.insert(I, {{MD.ID, InputIndex}, std::move(MD)});
     }
 
-    ID.CommandLine = FD.CommandLine;
+    ID.DriverCommandLine = std::move(FD.DriverCommandLine);
+    ID.Commands = std::move(FD.Commands);
     Inputs.push_back(std::move(ID));
   }
 
   void printFullOutput(raw_ostream &OS) {
     // Sort the modules by name to get a deterministic order.
     std::vector<IndexedModuleID> ModuleIDs;
     for (auto &&M : Modules)
       ModuleIDs.push_back(M.first);
for (auto &&ModID : ModuleIDs) {
           {"clang-modulemap-file", MD.ClangModuleMapFile},
           {"command-line", MD.BuildArguments},
       };
       OutModules.push_back(std::move(O));
     }
 
     Array TUs;
     for (auto &&I : Inputs) {
+      Array Commands;
+      if (I.DriverCommandLine.empty()) {
+        for (const auto &Cmd : I.Commands) {
           Object O{
               {"input-file", I.FileName},
               {"clang-context-hash", I.ContextHash},
               {"file-deps", I.FileDeps},
               {"clang-module-deps", toJSONSorted(I.ModuleDeps)},
-          {"command-line", I.CommandLine},
+              {"executable", Cmd.Executable},
+              {"command-line", Cmd.Arguments},
           };
-      TUs.push_back(std::move(O));
+          Commands.push_back(std::move(O));
+        }
+      } else {
+        Object O{
+            {"input-file", I.FileName},
+            {"clang-context-hash", I.ContextHash},
+            {"file-deps", I.FileDeps},
+            {"clang-module-deps", toJSONSorted(I.ModuleDeps)},
+            {"executable", "clang"},
+            {"command-line", I.DriverCommandLine},
+        };
+        Commands.push_back(std::move(O));
+      }
+      TUs.push_back(Object{
+          {"commands", std::move(Commands)},
+      });
     }
 
     Object Output{
         {"modules", std::move(OutModules)},
         {"translation-units", std::move(TUs)},
     };
 
     OS << llvm::formatv("{0:2}\n", Value(std::move(Output)));
struct IndexedModuleIDHasher {
     }
   };
 
   struct InputDeps {
     std::string FileName;
     std::string ContextHash;
     std::vector<std::string> FileDeps;
     std::vector<ModuleID> ModuleDeps;
-    std::vector<std::string> CommandLine;
+    std::vector<std::string> DriverCommandLine;
+    std::vector<Command> Commands;
   };
 
   std::mutex Lock;
   std::unordered_map<IndexedModuleID, ModuleDeps, IndexedModuleIDHasher>
       Modules;
   std::vector<InputDeps> Inputs;
 };
 
Pool.async([I, &Lock, &Index, &Inputs, &HadErrors, &FD, &WorkerTools,
 
         // Run the tool on it.
         if (Format == ScanningOutputFormat::Make) {
           auto MaybeFile = WorkerTools[I]->getDependencyFile(
               Input->CommandLine, CWD, MaybeModuleName);
           if (handleMakeDependencyToolResult(Filename, MaybeFile, DependencyOS,
                                              Errs))
             HadErrors = true;
+        } else if (DeprecatedDriverCommand) {
+          auto MaybeFullDeps =
+              WorkerTools[I]->getFullDependenciesLegacyDriverCommand(
+                  Input->CommandLine, CWD, AlreadySeenModules, LookupOutput,
+                  MaybeModuleName);
+          if (handleFullDependencyToolResult(Filename, MaybeFullDeps, FD,
+                                             LocalIndex, DependencyOS, Errs))
+            HadErrors = true;
         } else {
           auto MaybeFullDeps = WorkerTools[I]->getFullDependencies(
               Input->CommandLine, CWD, AlreadySeenModules, LookupOutput,
               MaybeModuleName);
           if (handleFullDependencyToolResult(Filename, MaybeFullDeps, FD,
                                              LocalIndex, DependencyOS, Errs))
             HadErrors = true;
         }

clang/utils/module-deps-to-rsp.py

def main():
   parser = argparse.ArgumentParser()
   parser.add_argument("full_deps_file", help="Path to the full dependencies json file",
                       type=str)
   action = parser.add_mutually_exclusive_group(required=True)
   action.add_argument("--module-name", help="The name of the module to get arguments for",
                       type=str)
   action.add_argument("--tu-index", help="The index of the translation unit to get arguments for",
                       type=int)
+  parser.add_argument("--tu-cmd-index",
+                      help="The index of the command within the translation unit (default=0)",
+                      type=int, default=0)
   args = parser.parse_args()
 
   full_deps = parseFullDeps(json.load(open(args.full_deps_file, 'r')))
 
   try:
     cmd = []
 
     if args.module_name:
       cmd = findModule(args.module_name, full_deps)['command-line']
     elif args.tu_index != None:
-      cmd = full_deps.translation_units[args.tu_index]['command-line']
+      tu = full_deps.translation_units[args.tu_index]
+      cmd = tu['commands'][args.tu_cmd_index]['command-line']
 
     print(" ".join(map(quote, cmd)))
   except:
     print("Unexpected error:", sys.exc_info()[0])
     raise
 
 if __name__ == '__main__':
   main()