[clang][modules] Add -cc1 option to backup PCM files

[jansvoboda11]

Summary

During a highly parallel build with -fimplicit-modules, multiple translation units may generate a PCM file for the same module at the same filesystem path in a short sequence.

When the first Clang instance tries to verify signature of the module on import, it discovers the new version of the PCM file (produced by latter Clang instance) that may have a different signature, leading to a mismatch and failed build.

To be able to debug such mismatch, it's invaluable to be able to compare the latter PCM (on disk) with the first one (overwritten).

This patch adds new -cc1 option -fbackup-module that tells Clang to store each PCM to the normal location and also to a path with an unique suffix (the PID of the Clang instance).

This is mostly additional change, but PCHContainerGenerator now doesn't unconditionally destroy the given PCHBuffer anymore and instead just decreases its reference count.

[dexonsmith]

A few high-level comments:

• Something like this *does* seem really useful for debugging problems with implicit module builds using a fuzzy context hash. But I feel like the use case is carved out at the wrong place; I feel like we either want this to be:
  • more restrictive: only supported for implicit+fuzzy modules, so it's not misused; or
  • more general: supported for all output files (CompilerInstance::createOutputFile), to provide a debugging option in other volatile filesystem situations.
• Changing the file extension doesn't seem ideal; it'd be better to embed the PID (or whatever it is) before the extension.
  • I'm not sure I love adding the pid to the filename at all TBH; although I'm not sure what'd be better.
• From a file-handling level, there's definite crossover with temporary files (write to temporary location, then move atomically into place). It might be cleaner to:
  • Write to the temp location, copy the temp to the .pid location, and move the temp to the final location.
  • Write to the unique/uncontended location (as the new temporary location), then copy the file (e.g., using llvm::copy_file) to the final location. I don't know if llvm::copy_file is atomic though...

clang/include/clang/Driver/Options.td

def fmodule_name_EQ : Joined<["-"], "fmodule-name=">, Group<f_Group>,
   Flags<[NoXarchOption,CC1Option]>, MetaVarName<"<name>">,
   HelpText<"Specify the name of the module to build">,
   MarshallingInfoString<LangOpts<"ModuleName">>;
 def fmodule_implementation_of : Separate<["-"], "fmodule-implementation-of">,
   Flags<[CC1Option]>, Alias<fmodule_name_EQ>;
 def fsystem_module : Flag<["-"], "fsystem-module">, Flags<[CC1Option]>,
   HelpText<"Build this module as a system module. Only used with -emit-module">,
   MarshallingInfoFlag<FrontendOpts<"IsSystemModule">>;
+def fbackup_module : Flag<["-"], "fbackup-module">,
+  Group<f_Group>, Flags<[CC1Option, NoDriverOption]>,
+  HelpText<"Backup the generated pre-compiled module/header to an unique location.">,
+  MarshallingInfoFlag<FrontendOpts<"BackupGeneratedModuleFile">>;
 def fmodule_map_file : Joined<["-"], "fmodule-map-file=">,
   Group<f_Group>, Flags<[NoXarchOption,CC1Option]>, MetaVarName<"<file>">,
   HelpText<"Load this module map file">,
   MarshallingInfoStringVector<FrontendOpts<"ModuleMapFiles">>;
 def fmodule_file : Joined<["-"], "fmodule-file=">,
   Group<i_Group>, Flags<[NoXarchOption,CC1Option]>, MetaVarName<"[<name>=]<file>">,
   HelpText<"Specify the mapping of module name to precompiled module file, or load a module file if name is omitted.">;
 def fmodules_ignore_macro : Joined<["-"], "fmodules-ignore-macro=">, Group<f_Group>, Flags<[CC1Option]>,

clang/include/clang/Frontend/FrontendOptions.h

public:
   unsigned SkipFunctionBodies : 1;
 
   /// Whether we can use the global module index if available.
   unsigned UseGlobalModuleIndex : 1;
 
   /// Whether we can generate the global module index if needed.
   unsigned GenerateGlobalModuleIndex : 1;
 
+  /// Whether we are backing up the generated module file to an unique location.
+  unsigned BackupGeneratedModuleFile : 1;
+
   /// Whether we include declaration dumps in AST dumps.
   unsigned ASTDumpDecls : 1;
 
   /// Whether we deserialize all decls when forming AST dumps.
   unsigned ASTDumpAll : 1;
 
   /// Whether we include lookup table dumps in AST dumps.
   unsigned ASTDumpLookups : 1;

 
 public:
   FrontendOptions()
       : DisableFree(false), RelocatablePCH(false), ShowHelp(false),
         ShowStats(false), TimeTrace(false), ShowVersion(false),
         FixWhatYouCan(false), FixOnlyWarnings(false), FixAndRecompile(false),
         FixToTemporaries(false), ARCMTMigrateEmitARCErrors(false),
         SkipFunctionBodies(false), UseGlobalModuleIndex(true),
-        GenerateGlobalModuleIndex(true), ASTDumpDecls(false),
-        ASTDumpLookups(false), BuildingImplicitModule(false),
-        ModulesEmbedAllFiles(false), IncludeTimestamps(true),
-        UseTemporary(true), AllowPCMWithCompilerErrors(false),
-        TimeTraceGranularity(500) {}
+        GenerateGlobalModuleIndex(true), BackupGeneratedModuleFile(false),
+        ASTDumpDecls(false), ASTDumpLookups(false),
+        BuildingImplicitModule(false), ModulesEmbedAllFiles(false),
+        IncludeTimestamps(true), UseTemporary(true),
+        AllowPCMWithCompilerErrors(false), TimeTraceGranularity(500) {}
 
   /// getInputKindForExtension - Return the appropriate input kind for a file
   /// extension. For example, "c" would return Language::C.
   ///
   /// \return The input kind for the extension, or Language::Unknown if the
   /// extension is not recognized.
   static InputKind getInputKindForExtension(StringRef Extension);
 };
 
 } // namespace clang
 
 #endif // LLVM_CLANG_FRONTEND_FRONTENDOPTIONS_H

clang/lib/CodeGen/ObjectFilePCHContainerOperations.cpp

public:
 
   /// Emit a container holding the serialized AST.
   void HandleTranslationUnit(ASTContext &Ctx) override {
     assert(M && VMContext && Builder);
     // Delete these on function exit.
     std::unique_ptr<llvm::LLVMContext> VMContext = std::move(this->VMContext);
     std::unique_ptr<llvm::Module> M = std::move(this->M);
     std::unique_ptr<CodeGen::CodeGenModule> Builder = std::move(this->Builder);
+    // Decrease the reference count on function exit.
+    std::shared_ptr<PCHBuffer> Buffer = std::move(this->Buffer);
 
     if (Diags.hasErrorOccurred())
       return;
 
     M->setTargetTriple(Ctx.getTargetInfo().getTriple().getTriple());
     M->setDataLayout(Ctx.getTargetInfo().getDataLayoutString());
 
     // PCH files don't have a signature field in the control block,
LLVM_DEBUG({
       llvm::dbgs() << Buffer;
     });
 
     // Use the LLVM backend to emit the pch container.
     clang::EmitBackendOutput(Diags, HeaderSearchOpts, CodeGenOpts, TargetOpts,
                              LangOpts,
                              Ctx.getTargetInfo().getDataLayoutString(), M.get(),
                              BackendAction::Backend_EmitObj, std::move(OS));
-
-    // Free the memory for the temporary buffer.
-    llvm::SmallVector<char, 0> Empty;
-    SerializedAST = std::move(Empty);
   }
 };
 
 } // anonymous namespace
 
 std::unique_ptr<ASTConsumer>
 ObjectFilePCHContainerWriter::CreatePCHContainerGenerator(
     CompilerInstance &CI, const std::string &MainFileName,

clang/lib/Frontend/FrontendActions.cpp

 //===--- FrontendActions.cpp ----------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "clang/Frontend/FrontendActions.h"
 #include "clang/AST/ASTConsumer.h"
 #include "clang/Basic/FileManager.h"
-#include "clang/Basic/TargetInfo.h"
 #include "clang/Basic/LangStandard.h"
+#include "clang/Basic/TargetInfo.h"
 #include "clang/Frontend/ASTConsumers.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/FrontendDiagnostic.h"
 #include "clang/Frontend/MultiplexConsumer.h"
 #include "clang/Frontend/Utils.h"
 #include "clang/Lex/DependencyDirectivesSourceMinimizer.h"
 #include "clang/Lex/HeaderSearch.h"
 #include "clang/Lex/Preprocessor.h"
 #include "clang/Lex/PreprocessorOptions.h"
 #include "clang/Sema/TemplateInstCallback.h"
 #include "clang/Serialization/ASTReader.h"
 #include "clang/Serialization/ASTWriter.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
+#include "llvm/Support/Process.h"
 #include "llvm/Support/YAMLTraits.h"
 #include "llvm/Support/raw_ostream.h"
 #include <memory>
 #include <system_error>
 
 using namespace clang;
 
 namespace {
 CodeCompleteConsumer *GetCodeCompletionConsumer(CompilerInstance &CI) {
   return CI.hasCodeCompletionConsumer() ? &CI.getCodeCompletionConsumer()
                                         : nullptr;
 }
 
 void EnsureSemaIsCreated(CompilerInstance &CI, FrontendAction &Action) {
   if (Action.hasCodeCompletionSupport() &&
       !CI.getFrontendOpts().CodeCompletionAt.FileName.empty())
     CI.createCodeCompletionConsumer();
 
   if (!CI.hasSema())
     CI.createSema(Action.getTranslationUnitKind(),
                   GetCodeCompletionConsumer(CI));
 }
+
+static void
+AddBackupPCHWriter(std::vector<std::unique_ptr<ASTConsumer>> &Consumers,
+                   CompilerInstance &CI, StringRef InFile,
+                   const std::string &OutFile,
+                   std::shared_ptr<PCHBuffer> &Buffer) {
+  std::string BackupOutFile =
+      OutFile + "." + std::to_string(llvm::sys::Process::getProcessId());
+
+  std::error_code EC;
+  auto OS = std::make_unique<llvm::raw_fd_ostream>(BackupOutFile, EC);
+  if (EC) {
+    CI.getDiagnostics().Report(diag::err_fe_error_opening)
+        << BackupOutFile << EC.message();
+    return;
+  }
+
+  Consumers.push_back(CI.getPCHContainerWriter().CreatePCHContainerGenerator(
+      CI, std::string(InFile), BackupOutFile, std::move(OS), Buffer));
+}
 } // namespace
 
 //===----------------------------------------------------------------------===//
 // Custom Actions
 //===----------------------------------------------------------------------===//
 
 std::unique_ptr<ASTConsumer>
 InitOnlyAction::CreateASTConsumer(CompilerInstance &CI, StringRef InFile) {
GeneratePCHAction::CreateASTConsumer(CompilerInstance &CI, StringRef InFile) {
   std::vector<std::unique_ptr<ASTConsumer>> Consumers;
   Consumers.push_back(std::make_unique<PCHGenerator>(
       CI.getPreprocessor(), CI.getModuleCache(), OutputFile, Sysroot, Buffer,
       FrontendOpts.ModuleFileExtensions,
       CI.getPreprocessorOpts().AllowPCHWithCompilerErrors,
       FrontendOpts.IncludeTimestamps, +CI.getLangOpts().CacheGeneratedPCH));
   Consumers.push_back(CI.getPCHContainerWriter().CreatePCHContainerGenerator(
       CI, std::string(InFile), OutputFile, std::move(OS), Buffer));
+  if (CI.getInvocation().getFrontendOpts().BackupGeneratedModuleFile)
+    AddBackupPCHWriter(Consumers, CI, InFile, OutputFile, Buffer);
 
   return std::make_unique<MultiplexConsumer>(std::move(Consumers));
 }
 
 bool GeneratePCHAction::ComputeASTConsumerArguments(CompilerInstance &CI,
                                                     std::string &Sysroot) {
   Sysroot = CI.getHeaderSearchOpts().Sysroot;
   if (CI.getFrontendOpts().RelocatablePCH && Sysroot.empty()) {
Consumers.push_back(std::make_unique<PCHGenerator>(
       /*AllowASTWithErrors=*/
       +CI.getFrontendOpts().AllowPCMWithCompilerErrors,
       /*IncludeTimestamps=*/
       +CI.getFrontendOpts().BuildingImplicitModule,
       /*ShouldCacheASTInMemory=*/
       +CI.getFrontendOpts().BuildingImplicitModule));
   Consumers.push_back(CI.getPCHContainerWriter().CreatePCHContainerGenerator(
       CI, std::string(InFile), OutputFile, std::move(OS), Buffer));
+  if (CI.getInvocation().getFrontendOpts().BackupGeneratedModuleFile)
+    AddBackupPCHWriter(Consumers, CI, InFile, OutputFile, Buffer);
   return std::make_unique<MultiplexConsumer>(std::move(Consumers));
 }
 
 bool GenerateModuleAction::shouldEraseOutputFiles() {
   return !getCompilerInstance().getFrontendOpts().AllowPCMWithCompilerErrors &&
          ASTFrontendAction::shouldEraseOutputFiles();
 }
 

clang/lib/Serialization/PCHContainerOperations.cpp

 public:
   RawPCHContainerGenerator(std::unique_ptr<llvm::raw_pwrite_stream> OS,
                            std::shared_ptr<PCHBuffer> Buffer)
       : Buffer(std::move(Buffer)), OS(std::move(OS)) {}
 
   ~RawPCHContainerGenerator() override = default;
 
   void HandleTranslationUnit(ASTContext &Ctx) override {
+    // Decrease the reference count on function exit.
+    std::shared_ptr<PCHBuffer> Buffer = std::move(this->Buffer);
+
     if (Buffer->IsComplete) {
       // Make sure it hits disk now.
       *OS << Buffer->Data;
       OS->flush();
     }
-    // Free the space of the temporary buffer.
-    llvm::SmallVector<char, 0> Empty;
-    Buffer->Data = std::move(Empty);
   }
 };
 
 } // anonymous namespace
 
 std::unique_ptr<ASTConsumer> RawPCHContainerWriter::CreatePCHContainerGenerator(
     CompilerInstance &CI, const std::string &MainFileName,
     const std::string &OutputFileName, std::unique_ptr<llvm::raw_pwrite_stream> OS,

clang/test/Modules/backup-module.h

+int function(void);

clang/test/Modules/backup-module.c

+// RUN: rm -rf %t && mkdir %t
+// RUN: %clang_cc1 %S/backup-module.h -emit-pch -o %t/backup-module.pch
+// RUN: find %t -name 'backup-module.pch.*' | not grep backup-module
+//
+// RUN: %clang_cc1 %S/backup-module.h -emit-pch -o %t/backup-module.pch -fbackup-module
+// RUN: find %t -name 'backup-module.pch.*' | grep backup-module
+// RUN: find %t -name 'backup-module.pch.*' | xargs diff %t/backup-module.pch
+
+// The purpose of this test is to make sure that the module file backup has
+// the same contents as the original.