[clang][deps] Override dependency and serialized diag files for modules

[benlangmuir]

Summary

When building modules, override secondary outputs (dependency file, dependency targets, serialized diagnostic file) in addition to the pcm file path. This avoids inheriting per-TU command-line options that cause non-determinism in the results (non-deterministic command-line for the module build, non-determinism in which TU's .diag and .d files will contain the module outputs). In clang-scan-deps we infer whether to generate dependency or serialized diagnostic files based on an original command-line. In a real build system this should be modeled explicitly.

[benlangmuir]

Updates:

• Made lookup of module outputs fallible. Not currently used by clang-scan-deps, but since the expectation is for a build system to provide these settings account for possibility of errors.
• Attempt to fix windows path issue in test

[jansvoboda11]

This looks pretty nice. My only concern is the ad-hoc command line parsing.

clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h

55

I'm curious whether you have encountered situations where being able to return an error from LookupModuleOutputs is useful.

clang/test/ClangScanDeps/preserved-args.c

0

I think we can delete this test at this point, since we check both interesting cases in generate-modules-path-args.c and removed-args.c.

If I remember correctly, this test was originally checking that the scanner does not change -MT and friends before the scan and doesn't leak that into the resulting command lines.

clang/tools/clang-scan-deps/ClangScanDeps.cpp

297

Is my understanding correct that this lambda of type const ModuleOutputOptions &(ModuleID) gets implicitly converted to llvm::function_ref<Expected<const ModuleOutputOptions &>(const ModuleID &)>?

If so, I think it would be clearer to take the ModuleID by const ref here also and wrap the return type with Expected, to match the... expected function_ref type. WDYT?

398

I think we should be avoiding ad-hoc command line parsing, it can be incorrect in many ways. Could we move this check somewhere where we have a properly constructed CompilerInvocation? I think we could do something like this in ModuleDeps::getCanonicalCommandLine:

if (!CI.getDiagnosticOpts().DiagnosticSerializationFile.empty())
  CI.getDiagnosticOpts().DiagnosticSerializationFile
        = LookupModuleOutput(ID, ModuleOutputKind::DiagnosticSerializationFile);

[benlangmuir]

clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h

55

I ended up backing this change out: it was motivated by a downstream libclang API change that I have now re-evaluated based on your other feedback to use a per-output callback instead of a single callback.

clang/tools/clang-scan-deps/ClangScanDeps.cpp

297

This was unintentional, I just missed these couple of places when I changed the API from ModuleID to const ModuleID &. Will fix, thanks!

398

Yeah, we can do that. I originally avoided this due to it "leaking" whether the TU used these outputs into the module command-lines (since the set of callbacks would differ), but I suspect in practice that doesn't matter since you're unlikely to mix compilations that have and don't have serialized diagnostics. To be 100% sound, it will require adding the existence of the outputs to the module context hash (not the actual path, just whether there was a diag and/or d file at all). I will do the context hash change later if you're okay with it - there's nowhere to feed the extra info into getModuleHash right now, but I was already planning to change the hashing which will make it easier to do. If you think it's critical we could add a parameter to getModuleHash temporarily to handle it.

I liked your idea to make the callback per-output as well.

[benlangmuir]

Updates per review

• Switched to a per-output callback
• Removed preserved-args.c test
• Removed error handling that I no longer have a real use for
• Only request .d and .diag paths if they were enabled in the original TU

[jansvoboda11]

LGTM! Thanks.

clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp

341–342

This is a bit unfortunate but I don't see a better alternative.

Ideally, we would compute the hash with the .d and .dia paths already filled in. That would ensure the command line we end up reporting to the build system really does have the context hash associated with the module. (We'd need to include every field set in getCanonicalCommandLine() too.) But for the path lookup, we already need some kind of (currently partial) context hash.

[benlangmuir]

clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp

341–342

The other things added in getCanonicalCommandLine are currently:

• module map path - I'm planning to include this in my changes to the context hash since it is significant exactly how the path is spelled. This is already part of the implicit module build's notion of identity.
• pcm paths - these are sound as long as we always get the same paths returned in the callback during a build (across separate builds it would be fine to change them as long as you're going to rebuild anything whose path changed, and anything downstream of that).

clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h

struct FullDependencies {
   /// determined that the differences are benign for this compilation.
   std::vector<ModuleID> ClangModuleDeps;
 
   /// The original command line of the TU (excluding the compiler executable).
   std::vector<std::string> OriginalCommandLine;
 
   /// Get the full command line.
   ///
-  /// \param LookupPCMPath This function is called to fill in "-fmodule-file="
-  ///                      arguments and the "-o" argument. It needs to return
-  ///                      a path for where the PCM for the given module is to
-  ///                      be located.
-  std::vector<std::string>
-  getCommandLine(std::function<StringRef(ModuleID)> LookupPCMPath) const;
+  /// \param LookupModuleOutput This function is called to fill in
+  ///                           "-fmodule-file=", "-o" and other output
+  ///                           arguments for dependencies.
+  std::vector<std::string> getCommandLine(
+      llvm::function_ref<std::string(const ModuleID &, ModuleOutputKind)>
+          LookupOutput) const;
 
   /// Get the full command line, excluding -fmodule-file=" arguments.
   std::vector<std::string> getCommandLineWithoutModulePaths() const;
 };
 
 struct FullDependenciesResult {
   FullDependencies FullDeps;
   std::vector<ModuleDeps> DiscoveredModules;

clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h

 };
 
 struct ModuleIDHasher {
   std::size_t operator()(const ModuleID &MID) const {
     return llvm::hash_combine(MID.ModuleName, MID.ContextHash);
   }
 };
 
+/// An output from a module compilation, such as the path of the module file.
+enum class ModuleOutputKind {
+  /// The module file (.pcm). Required.
+  ModuleFile,
+  /// The path of the dependency file (.d), if any.
+  DependencyFile,
+  /// The null-separated list of names to use as the targets in the dependency
+  /// file, if any.
+  DependencyTargets,
+  /// The path of the serialized diagnostic file (.dia), if any.
+  DiagnosticSerializationFile,
+};
+
 struct ModuleDeps {
   /// The identifier of the module.
   ModuleID ID;
 
   /// Whether this is a "system" module.
   bool IsSystem;
 
   /// The path to the modulemap file which defines this module.
struct ModuleDeps {
   /// This may include modules with a different context hash when it can be
   /// determined that the differences are benign for this compilation.
   std::vector<ModuleID> ClangModuleDeps;
 
   // Used to track which modules that were discovered were directly imported by
   // the primary TU.
   bool ImportedByMainFile = false;
 
+  /// Whether the TU had a dependency file. The path in \c BuildInvocation is
+  /// cleared to avoid leaking the specific path from the TU into the module.
+  bool HadDependencyFile = false;
+
+  /// Whether the TU had serialized diagnostics. The path in \c BuildInvocation
+  /// is cleared to avoid leaking the specific path from the TU into the module.
+  bool HadSerializedDiagnostics = false;
+
   /// Compiler invocation that can be used to build this module (without paths).
   CompilerInvocation BuildInvocation;
 
   /// Gets the canonical command line suitable for passing to clang.
   ///
-  /// \param LookupPCMPath This function is called to fill in "-fmodule-file="
-  ///                      arguments and the "-o" argument. It needs to return
-  ///                      a path for where the PCM for the given module is to
-  ///                      be located.
+  /// \param LookupModuleOutput This function is called to fill in
+  ///                           "-fmodule-file=", "-o" and other output
+  ///                           arguments.
   std::vector<std::string> getCanonicalCommandLine(
-      std::function<StringRef(ModuleID)> LookupPCMPath) const;
+      llvm::function_ref<std::string(const ModuleID &, ModuleOutputKind)>
+          LookupModuleOutput) const;
 
   /// Gets the canonical command line suitable for passing to clang, excluding
   /// "-fmodule-file=" and "-o" arguments.
   std::vector<std::string> getCanonicalCommandLineWithoutModulePaths() const;
 };
 
 class ModuleDepCollector;
 

clang/lib/Tooling/DependencyScanning/DependencyScanningTool.cpp

 //===- DependencyScanningTool.cpp - clang-scan-deps service ---------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "clang/Tooling/DependencyScanning/DependencyScanningTool.h"
 #include "clang/Frontend/Utils.h"
 
 using namespace clang;
 using namespace tooling;
 using namespace dependencies;
 
 std::vector<std::string> FullDependencies::getCommandLine(
-    std::function<StringRef(ModuleID)> LookupPCMPath) const {
+    llvm::function_ref<std::string(const ModuleID &, ModuleOutputKind)>
+        LookupModuleOutput) const {
   std::vector<std::string> Ret = getCommandLineWithoutModulePaths();
 
-  for (ModuleID MID : ClangModuleDeps)
-    Ret.push_back(("-fmodule-file=" + LookupPCMPath(MID)).str());
+  for (ModuleID MID : ClangModuleDeps) {
+    auto PCM = LookupModuleOutput(MID, ModuleOutputKind::ModuleFile);
+    Ret.push_back("-fmodule-file=" + PCM);
+  }
 
   return Ret;
 }
 
 std::vector<std::string>
 FullDependencies::getCommandLineWithoutModulePaths() const {
   std::vector<std::string> Args = OriginalCommandLine;
 

clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp

CompilerInvocation ModuleDepCollector::makeInvocationForModuleBuildWithoutPaths(
 
   // Remove options incompatible with explicit module build or are likely to
   // differ between identical modules discovered from different translation
   // units.
   CI.getFrontendOpts().Inputs.clear();
   CI.getFrontendOpts().OutputFile.clear();
   CI.getCodeGenOpts().MainFileName.clear();
   CI.getCodeGenOpts().DwarfDebugFlags.clear();
+  CI.getDiagnosticOpts().DiagnosticSerializationFile.clear();
+  CI.getDependencyOutputOpts().OutputFile.clear();
+  CI.getDependencyOutputOpts().Targets.clear();
 
   CI.getFrontendOpts().ProgramAction = frontend::GenerateModule;
   CI.getLangOpts()->ModuleName = Deps.ID.ModuleName;
   CI.getFrontendOpts().IsSystemModule = Deps.IsSystem;
 
   // Disable implicit modules and canonicalize options that are only used by
   // implicit modules.
   CI.getLangOpts()->ImplicitModules = false;
serializeCompilerInvocation(const CompilerInvocation &CI) {
   // Synthesize full command line from the CompilerInvocation, including "-cc1".
   SmallVector<const char *, 32> Args{"-cc1"};
   CI.generateCC1CommandLine(Args, SA);
 
   // Convert arguments to the return type.
   return std::vector<std::string>{Args.begin(), Args.end()};
 }
 
+static std::vector<std::string> splitString(std::string S, char Separator) {
+  SmallVector<StringRef> Segments;
+  StringRef(S).split(Segments, Separator);
+  std::vector<std::string> Result;
+  Result.reserve(Segments.size());
+  for (StringRef Segment : Segments)
+    Result.push_back(Segment.str());
+  return Result;
+}
+
 std::vector<std::string> ModuleDeps::getCanonicalCommandLine(
-    std::function<StringRef(ModuleID)> LookupPCMPath) const {
+    llvm::function_ref<std::string(const ModuleID &, ModuleOutputKind)>
+        LookupModuleOutput) const {
   CompilerInvocation CI(BuildInvocation);
   FrontendOptions &FrontendOpts = CI.getFrontendOpts();
 
   InputKind ModuleMapInputKind(FrontendOpts.DashX.getLanguage(),
                                InputKind::Format::ModuleMap);
   FrontendOpts.Inputs.emplace_back(ClangModuleMapFile, ModuleMapInputKind);
-  FrontendOpts.OutputFile = std::string(LookupPCMPath(ID));
+  FrontendOpts.OutputFile =
+      LookupModuleOutput(ID, ModuleOutputKind::ModuleFile);
+  if (HadSerializedDiagnostics)
+    CI.getDiagnosticOpts().DiagnosticSerializationFile =
+        LookupModuleOutput(ID, ModuleOutputKind::DiagnosticSerializationFile);
+  if (HadDependencyFile) {
+    CI.getDependencyOutputOpts().OutputFile =
+        LookupModuleOutput(ID, ModuleOutputKind::DependencyFile);
+    CI.getDependencyOutputOpts().Targets = splitString(
+        LookupModuleOutput(ID, ModuleOutputKind::DependencyTargets), '\0');
+  }
 
   for (ModuleID MID : ClangModuleDeps)
-    FrontendOpts.ModuleFiles.emplace_back(LookupPCMPath(MID));
+    FrontendOpts.ModuleFiles.push_back(
+        LookupModuleOutput(MID, ModuleOutputKind::ModuleFile));
 
   return serializeCompilerInvocation(CI);
 }
 
 std::vector<std::string>
 ModuleDeps::getCanonicalCommandLineWithoutModulePaths() const {
   return serializeCompilerInvocation(BuildInvocation);
 }
ModuleID ModuleDepCollectorPP::handleTopLevelModule(const Module *M) {
   addAllSubmodulePrebuiltDeps(M, MD, SeenModules);
 
   MD.BuildInvocation = MDC.makeInvocationForModuleBuildWithoutPaths(
       MD, [&](CompilerInvocation &BuildInvocation) {
         if (MDC.OptimizeArgs)
           optimizeHeaderSearchOpts(BuildInvocation.getHeaderSearchOpts(),
                                    *MDC.ScanInstance.getASTReader(), *MF);
       });
+  MD.HadSerializedDiagnostics = !MDC.OriginalInvocation.getDiagnosticOpts()
+                                     .DiagnosticSerializationFile.empty();
+  MD.HadDependencyFile =
+      !MDC.OriginalInvocation.getDependencyOutputOpts().OutputFile.empty();
+  // FIXME: HadSerializedDiagnostics and HadDependencyFile should be included in
+  // the context hash since it can affect the command-line.
   MD.ID.ContextHash = MD.BuildInvocation.getModuleHash();
 
   llvm::DenseSet<const Module *> AddedModules;
   addAllSubmoduleDeps(M, MD, AddedModules);
 
   return MD.ID;
 }
 

clang/test/ClangScanDeps/Inputs/preserved-args/cdb.json.template

-[
-  {
-    "directory": "DIR",
-    "command": "clang -MD -MT my_target -serialize-diagnostics DIR/tu.dia -fsyntax-only DIR/tu.c -fmodules -fimplicit-module-maps -fmodules-cache-path=DIR/cache -fmodule-file=Foo=DIR/foo.pcm -o DIR/tu.o",
-    "file": "DIR/tu.c"
-  }
-]

clang/test/ClangScanDeps/Inputs/preserved-args/mod.h

-// mod.h

clang/test/ClangScanDeps/Inputs/preserved-args/module.modulemap

-module Mod {
-  header "mod.h"
-}

clang/test/ClangScanDeps/Inputs/preserved-args/tu.c

-#include "mod.h"

clang/test/ClangScanDeps/Inputs/removed-args/cdb.json.template

 [
   {
     "directory": "DIR",
-    "command": "clang -fsyntax-only DIR/tu.c -fmodules -fimplicit-module-maps -fmodules-validate-once-per-build-session -fbuild-session-file=DIR/build-session -fmodules-prune-interval=123 -fmodules-prune-after=123 -fmodules-cache-path=DIR/cache -include DIR/header.h -grecord-command-line -o DIR/tu.o",
+    "command": "clang -fsyntax-only DIR/tu.c -fmodules -fimplicit-module-maps -fmodules-validate-once-per-build-session -fbuild-session-file=DIR/build-session -fmodules-prune-interval=123 -fmodules-prune-after=123 -fmodules-cache-path=DIR/cache -include DIR/header.h -grecord-command-line -o DIR/tu.o -serialize-diagnostics DIR/tu.diag -MT tu -MD -MF DIR/tu.d",
     "file": "DIR/tu.c"
   }
 ]

clang/test/ClangScanDeps/generate-modules-path-args.c

+// RUN: rm -rf %t
+// RUN: split-file %s %t
+// RUN: sed "s|DIR|%/t|g" %t/cdb.json.template > %t/cdb.json
+// RUN: sed "s|DIR|%/t|g" %t/cdb_without.json.template > %t/cdb_without.json
+// RUN: clang-scan-deps -compilation-database %t/cdb.json \
+// RUN:   -format experimental-full -generate-modules-path-args > %t/deps.json
+// RUN: cat %t/deps.json | sed 's:\\\\\?:/:g' | FileCheck -DPREFIX=%/t %s
+// RUN: clang-scan-deps -compilation-database %t/cdb_without.json \
+// RUN:   -format experimental-full -generate-modules-path-args > %t/deps_without.json
+// RUN: cat %t/deps_without.json | sed 's:\\\\\?:/:g' | FileCheck -DPREFIX=%/t -check-prefix=WITHOUT %s
+
+// CHECK:      {
+// CHECK-NEXT:   "modules": [
+// CHECK-NEXT:     {
+// CHECK:            "command-line": [
+// CHECK-NEXT:         "-cc1"
+// CHECK:              "-serialize-diagnostic-file"
+// CHECK-NEXT:         "[[PREFIX]]{{.*}}Mod{{.*}}.diag"
+// CHECK:              "-dependency-file"
+// CHECK-NEXT:         "[[PREFIX]]{{.*}}Mod{{.*}}.d"
+// CHECK:            ],
+
+// WITHOUT:      {
+// WITHOUT-NEXT:   "modules": [
+// WITHOUT-NEXT:     {
+// WITHOUT:            "command-line": [
+// WITHOUT-NEXT:         "-cc1"
+// WITHOUT-NOT:          "-serialize-diagnostic-file"
+// WITHOUT-NOT:          "-dependency-file"
+// WITHOUT:            ],
+
+//--- cdb.json.template
+[{
+  "directory": "DIR",
+  "command": "clang -fsyntax-only DIR/tu.c -fmodules -fimplicit-module-maps -fmodules-cache-path=DIR/cache -serialize-diagnostics DIR/tu.diag -MD -MT tu -MF DIR/tu.d",
+  "file": "DIR/tu.c"
+}]
+
+//--- cdb_without.json.template
+[{
+  "directory": "DIR",
+  "command": "clang -fsyntax-only DIR/tu.c -fmodules -fimplicit-module-maps -fmodules-cache-path=DIR/cache",
+  "file": "DIR/tu.c"
+}]
+
+//--- module.modulemap
+module Mod { header "Mod.h" }
+
+//--- Mod.h
+
+//--- tu.c
+#include "Mod.h"

clang/test/ClangScanDeps/preserved-args.c

-// RUN: rm -rf %t && mkdir %t
-// RUN: cp -r %S/Inputs/preserved-args/* %t
-// RUN: sed -e "s|DIR|%/t|g" %t/cdb.json.template > %t/cdb.json
-
-// RUN: clang-scan-deps -compilation-database %t/cdb.json -format experimental-full > %t/result.json
-// RUN: cat %t/result.json | sed 's:\\\\\?:/:g' | FileCheck %s -DPREFIX=%/t
-
-// CHECK:      {
-// CHECK-NEXT:   "modules": [
-// CHECK-NEXT:     {
-// CHECK:            "command-line": [
-// CHECK-NEXT:         "-cc1"
-// CHECK:              "-serialize-diagnostic-file"
-// CHECK-NEXT:         "[[PREFIX]]/tu.dia"
-// CHECK:              "-fmodule-file=Foo=[[PREFIX]]/foo.pcm"
-// CHECK:              "-MT"
-// CHECK-NEXT:         "my_target"
-// CHECK:              "-dependency-file"
-// CHECK-NEXT:         "[[PREFIX]]/tu.d"
-// CHECK:            ],
-// CHECK:            "name": "Mod"
-// CHECK-NEXT:     }
-// CHECK-NEXT:   ]
-// CHECK:      }

clang/test/ClangScanDeps/removed-args.c

 // CHECK-NOT:          "-dwarf-debug-flags"
 // CHECK-NOT:          "-main-file-name"
 // CHECK-NOT:          "-include"
 // CHECK-NOT:          "-fmodules-cache-path=
 // CHECK-NOT:          "-fmodules-validate-once-per-build-session"
 // CHECK-NOT:          "-fbuild-session-timestamp=
 // CHECK-NOT:          "-fmodules-prune-interval=
 // CHECK-NOT:          "-fmodules-prune-after=
+// CHECK-NOT:          "-dependency-file"
+// CHECK-NOT:          "-MT"
+// CHECK-NOT:          "-serialize-diagnostic-file"
 // CHECK:            ],
 // CHECK-NEXT:       "context-hash": "[[HASH_MOD_HEADER:.*]]",
 // CHECK-NEXT:       "file-deps": [
 // CHECK-NEXT:         "[[PREFIX]]/mod_header.h",
 // CHECK-NEXT:         "[[PREFIX]]/module.modulemap"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "name": "ModHeader"
 // CHECK-NEXT:     },
 // CHECK-NEXT:     {
 // CHECK-NEXT:       "clang-module-deps": [],
 // CHECK-NEXT:       "clang-modulemap-file": "[[PREFIX]]/module.modulemap",
 // CHECK-NEXT:       "command-line": [
 // CHECK-NEXT:         "-cc1"
 // CHECK-NOT:          "-dwarf-debug-flags"
 // CHECK-NOT:          "-main-file-name"
 // CHECK-NOT:          "-include"
 // CHECK-NOT:          "-fmodules-cache-path=
 // CHECK-NOT:          "-fmodules-validate-once-per-build-session"
 // CHECK-NOT:          "-fbuild-session-timestamp=
 // CHECK-NOT:          "-fmodules-prune-interval=
 // CHECK-NOT:          "-fmodules-prune-after=
+// CHECK-NOT:          "-dependency-file"
+// CHECK-NOT:          "-MT"
+// CHECK-NOT:          "-serialize-diagnostic-file"
 // CHECK:            ],
 // CHECK-NEXT:       "context-hash": "[[HASH_MOD_TU:.*]]",
 // CHECK-NEXT:       "file-deps": [
 // CHECK-NEXT:         "[[PREFIX]]/mod_tu.h",
 // CHECK-NEXT:         "[[PREFIX]]/module.modulemap"
 // CHECK-NEXT:       ],
 // CHECK-NEXT:       "name": "ModTU"
 // CHECK-NEXT:     }

clang/tools/clang-scan-deps/ClangScanDeps.cpp

for (const ModuleDeps &MD : FDR.DiscoveredModules) {
       auto I = Modules.find({MD.ID, 0});
       if (I != Modules.end()) {
         I->first.InputIndex = std::min(I->first.InputIndex, InputIndex);
         continue;
       }
       Modules.insert(I, {{MD.ID, InputIndex}, std::move(MD)});
     }
 
-    ID.CommandLine = GenerateModulesPathArgs
-                         ? FD.getCommandLine(
-                               [&](ModuleID MID) { return lookupPCMPath(MID); })
-                         : FD.getCommandLineWithoutModulePaths();
-
+    ID.CommandLine =
+        GenerateModulesPathArgs
+            ? FD.getCommandLine([&](const ModuleID &MID, ModuleOutputKind MOK) {
+                return lookupModuleOutput(MID, MOK);
+              })
     Inputs.push_back(std::move(ID));
   }
 
   void printFullOutput(raw_ostream &OS) {
     // Sort the modules by name to get a deterministic order.
     std::vector<IndexedModuleID> ModuleIDs;
     for (auto &&M : Modules)
       ModuleIDs.push_back(M.first);
     llvm::sort(ModuleIDs,
for (auto &&ModID : ModuleIDs) {
           {"name", MD.ID.ModuleName},
           {"context-hash", MD.ID.ContextHash},
           {"file-deps", toJSONSorted(MD.FileDeps)},
           {"clang-module-deps", toJSONSorted(MD.ClangModuleDeps)},
           {"clang-modulemap-file", MD.ClangModuleMapFile},
           {"command-line",
            GenerateModulesPathArgs
                ? MD.getCanonicalCommandLine(
-                     [&](ModuleID MID) { return lookupPCMPath(MID); })
+                     [&](const ModuleID &MID, ModuleOutputKind MOK) {
+                       return lookupModuleOutput(MID, MOK);
+                     })
                : MD.getCanonicalCommandLineWithoutModulePaths()},
       };
       OutModules.push_back(std::move(O));
     }
 
     Array TUs;
     for (auto &&I : Inputs) {
       Object O{
Object Output{
         {"modules", std::move(OutModules)},
         {"translation-units", std::move(TUs)},
     };
 
     OS << llvm::formatv("{0:2}\n", Value(std::move(Output)));
   }
 
 private:
-  StringRef lookupPCMPath(ModuleID MID) {
+  std::string lookupModuleOutput(const ModuleID &MID, ModuleOutputKind MOK) {
+    // Cache the PCM path, since it will be queried repeatedly for each module.
+    // The other outputs are only queried once during getCanonicalCommandLine.
     auto PCMPath = PCMPaths.insert({MID, ""});
     if (PCMPath.second)
       PCMPath.first->second = constructPCMPath(MID);
+    switch (MOK) {
+    case ModuleOutputKind::ModuleFile:
       return PCMPath.first->second;
+    case ModuleOutputKind::DependencyFile:
+      return PCMPath.first->second + ".d";
+    case ModuleOutputKind::DependencyTargets:
+      return ""; // Will get the default target name.
+    case ModuleOutputKind::DiagnosticSerializationFile:
+      return PCMPath.first->second + ".diag";
+    }
   }
 
   /// Construct a path for the explicitly built PCM.
   std::string constructPCMPath(ModuleID MID) const {
     auto MDIt = Modules.find(IndexedModuleID{MID, 0});
     assert(MDIt != Modules.end());
     const ModuleDeps &MD = MDIt->second;
 
     StringRef Filename = llvm::sys::path::filename(MD.ImplicitModulePCMPath);
     StringRef ModuleCachePath = llvm::sys::path::parent_path(
         llvm::sys::path::parent_path(MD.ImplicitModulePCMPath));
 
     SmallString<256> ExplicitPCMPath(!ModuleFilesDir.empty() ? ModuleFilesDir
                                                              : ModuleCachePath);
     llvm::sys::path::append(ExplicitPCMPath, MD.ID.ContextHash, Filename);
     return std::string(ExplicitPCMPath);
   }
 
   struct IndexedModuleID {
     ModuleID ID;
     mutable size_t InputIndex;
 
     bool operator==(const IndexedModuleID &Other) const {
       return ID.ModuleName == Other.ID.ModuleName &&
              ID.ContextHash == Other.ID.ContextHash;
     }
   };
 
   struct IndexedModuleIDHasher {
     std::size_t operator()(const IndexedModuleID &IMID) const {
       using llvm::hash_combine;
 
       return hash_combine(IMID.ID.ModuleName, IMID.ID.ContextHash);