Reduce the size of output with -print-before/after-all by avoid duplicated dump

[inouehrs]

Summary

The IR transformation log generated with -print-before/after-all is essential information to understand the LLVM behavior.
However, the size of the output often becomes quite large.
This patch reduces the size of the output by avoiding print the detail if the function is not modified by a previous pass. This functionality is controlled by -omit-duplicated-dump option and is not enabled by default.
Since, there is many passes that do not modify functions (including analysis passes), the size of output becomes about half in many cases.

If the function is not modified, output looks like as follow:

*** IR Dump After Global Value Numbering ***
; Function Attrs: norecurse nounwind uwtable
define i32 @main() local_unnamed_addr #2 {
  listing omitted because it is not modified since last dump.
}

# *** IR Dump After StackMap Liveness Analysis ***:
# Machine code for function main: NoPHIs, TracksLiveness, NoVRegs

  listing omitted because it is not modified since last dump.

# End machine code for function main.

[npjdesres]

This looks useful. One suggestion might be to change this diagnostic:

listing omitted because it is not modified since last dump.

To someting more greppable, such as:

Pass "StackMap Liveness Analysis" did not modify the IR

[mehdi_amini]

Could be useful, but that seems a bit intrusive right now (ad-hoc hooks in Module and PassManager)

lib/IR/Module.cpp

96

Why aren't you using a smart pointer?

[inouehrs]

• rewrite using a smart pointer.
• try to make the changes looking less intrusive in PassManagers.

[inouehrs]

@mehdi_amini I want to be less intrusive especially in Module. I like to pack most of the changes in PMDataManager to be less intrusive in Module, but I cannot find a good way to access PMDataManager (i.e. PassManager) from AsmWriter. I appreciate any further suggestions.

@npjdesres Your suggested output looks quite nice. But I do not implement it so far because I need to be add further code to record the last pass without modification. I am investigating a good way to improve the message.

lib/IR/Module.cpp

96

I rewrote the change in Module with a smart pointer.

[inouehrs]

I like to redesign this. Thank you for advises.

include/llvm/IR/LegacyPassManagers.h

public:
   void dumpLastUses(Pass *P, unsigned Offset) const;
   void dumpPassArguments() const;
   void dumpPassInfo(Pass *P, enum PassDebuggingString S1,
                     enum PassDebuggingString S2, StringRef Msg);
   void dumpRequiredSet(const Pass *P) const;
   void dumpPreservedSet(const Pass *P) const;
   void dumpUsedSet(const Pass *P) const;
 
+  // Mark the function to dump in print pass
+  void markChangedFunc(const Function &F) const;
+
+  // Mark all functions in the module to dump in print pass
+  void markChangedFunc(const Module &M) const;
+
   unsigned getNumContainedPasses() const {
     return (unsigned)PassVector.size();
   }
 
   virtual PassManagerType getPassManagerType() const {
     assert ( 0 && "Invalid use of getPassManagerType");
     return PMT_Unknown;
   }

include/llvm/IR/Module.h

private:
   std::string ModuleID;           ///< Human readable identifier for the module
   std::string SourceFileName;     ///< Original source file name for module,
                                   ///< recorded in bitcode.
   std::string TargetTriple;       ///< Platform target triple Module compiled on
                                   ///< Format: (arch)(sub)-(vendor)-(sys0-(abi)
   void *NamedMDSymTab;            ///< NamedMDNode names.
   DataLayout DL;                  ///< DataLayout associated with the module
 
+  /// Functions that are not modified after the previous dump.
+  /// Used to avoid redundant dump with -print-{before,after}-all.
+  std::unique_ptr<SmallPtrSetImpl<const Function *>> FuncsUnchangedAfterDump;
+
   friend class Constant;
 
 /// @}
 /// @name Constructors
 /// @{
 public:
   /// The Module constructor. Note that there is no default constructor. You
   /// must provide a name for the module upon construction.
/// @}
   void setProfileSummary(Metadata *M);
 
   /// \brief Returns profile summary metadata
   Metadata *getProfileSummary();
   /// @}
 
   /// Take ownership of the given memory buffer.
   void setOwnedMemoryBuffer(std::unique_ptr<MemoryBuffer> MB);
+
+  /// Returns set of functions need not to dump
+  SmallPtrSetImpl<const Function *> *getUnchangedFuncs() const {
+    return FuncsUnchangedAfterDump.get();
+  }
 };
 
 /// \brief Given "llvm.used" or "llvm.compiler.used" as a global name, collect
 /// the initializer elements of that global in Set and return the global itself.
 GlobalVariable *collectUsedGlobalVariables(const Module &M,
                                            SmallPtrSetImpl<GlobalValue *> &Set,
                                            bool CompilerUsed);
 

lib/CodeGen/MachineFunction.cpp

for (MachineRegisterInfo::livein_iterator
       if (std::next(I) != E)
         OS << ", ";
     }
     OS << '\n';
   }
 
   ModuleSlotTracker MST(getFunction()->getParent());
   MST.incorporateFunction(*getFunction());
+
+  const Function *F = getFunction();
+  SmallPtrSetImpl<const Function *> *UFs = F->getParent()->getUnchangedFuncs();
+  if (UFs && UFs->count(F) != 0)
+    OS << "\n  listing omitted because it is not modified since last dump.\n";
+  else {
     for (const auto &BB : *this) {
       OS << '\n';
       BB.print(OS, MST, Indexes);
     }
+    if (UFs) UFs->insert(F);
+  }
 
   OS << "\n# End machine code for function " << getName() << ".\n\n";
 }
 
 namespace llvm {
   template<>
   struct DOTGraphTraits<const MachineFunction*> : public DefaultDOTGraphTraits {
 

lib/IR/AsmWriter.cpp

void AssemblyWriter::printFunction(const Function *F) {
   if (F->isDeclaration()) {
     Out << '\n';
   } else {
     SmallVector<std::pair<unsigned, MDNode *>, 4> MDs;
     F->getAllMetadata(MDs);
     printMetadataAttachments(MDs, " ");
 
     Out << " {";
+
+    SmallPtrSetImpl<const Function *> *UFs = F->getParent()->getUnchangedFuncs();
+    if (UFs && UFs->count(F) != 0)
+      Out << "\n  listing omitted because it is not modified since last dump.\n";
+    else {
       // Output all of the function's basic blocks.
       for (const BasicBlock &BB : *F)
         printBasicBlock(&BB);
+      if (UFs) UFs->insert(F);
+    }
 
     // Output the function's use-lists.
     printUseLists(F);
 
     Out << "}\n";
   }
 
   Machine.purgeFunction();

lib/IR/LegacyPassManager.cpp

if (!PInf) {
       dbgs() << " Uninitialized Pass";
       continue;
     }
     dbgs() << ' ' << PInf->getPassName();
   }
   dbgs() << '\n';
 }
 
+void PMDataManager::markChangedFunc(const Function &F) const {
+  SmallPtrSetImpl<const Function *> *UFs = F.getParent()->getUnchangedFuncs();
+  if (UFs) UFs->erase(&F);
+}
+
+void PMDataManager::markChangedFunc(const Module &M) const {
+  SmallPtrSetImpl<const Function *> *UFs = M.getUnchangedFuncs();
+  if (UFs) UFs->clear();
+}
+
 /// Add RequiredPass into list of lower level passes required by pass P.
 /// RequiredPass is run on the fly by Pass Manager when P requests it
 /// through getAnalysis interface.
 /// This should be handled by specific pass manager.
 void PMDataManager::addLowerLevelRequiredPass(Pass *P, Pass *RequiredPass) {
   if (TPM) {
     TPM->dumpArguments();
     TPM->dumpPasses();
for (unsigned Index = 0; Index < getNumContainedPasses(); ++Index) {
         // If the pass crashes, remember this.
         PassManagerPrettyStackEntry X(BP, *I);
         TimeRegion PassTimer(getPassTimer(BP));
 
         LocalChanged |= BP->runOnBasicBlock(*I);
       }
 
       Changed |= LocalChanged;
-      if (LocalChanged)
+      if (LocalChanged) {
         dumpPassInfo(BP, MODIFICATION_MSG, ON_BASICBLOCK_MSG,
                      I->getName());
+        markChangedFunc(F);
+      }
       dumpPreservedSet(BP);
       dumpUsedSet(BP);
 
       verifyPreservedAnalysis(BP);
       removeNotPreservedAnalysis(BP);
       recordAvailableAnalysis(BP);
       removeDeadPasses(BP, I->getName(), ON_BASICBLOCK_MSG);
     }
for (unsigned Index = 0; Index < getNumContainedPasses(); ++Index) {
     {
       PassManagerPrettyStackEntry X(FP, F);
       TimeRegion PassTimer(getPassTimer(FP));
 
       LocalChanged |= FP->runOnFunction(F);
     }
 
     Changed |= LocalChanged;
-    if (LocalChanged)
+    if (LocalChanged) {
       dumpPassInfo(FP, MODIFICATION_MSG, ON_FUNCTION_MSG, F.getName());
+      markChangedFunc(F);
+    }
     dumpPreservedSet(FP);
     dumpUsedSet(FP);
 
     verifyPreservedAnalysis(FP);
     removeNotPreservedAnalysis(FP);
     recordAvailableAnalysis(FP);
     removeDeadPasses(FP, F.getName(), ON_FUNCTION_MSG);
   }
for (unsigned Index = 0; Index < getNumContainedPasses(); ++Index) {
     {
       PassManagerPrettyStackEntry X(MP, M);
       TimeRegion PassTimer(getPassTimer(MP));
 
       LocalChanged |= MP->runOnModule(M);
     }
 
     Changed |= LocalChanged;
-    if (LocalChanged)
+    if (LocalChanged) {
       dumpPassInfo(MP, MODIFICATION_MSG, ON_MODULE_MSG,
                    M.getModuleIdentifier());
+      markChangedFunc(M);
+    }
     dumpPreservedSet(MP);
     dumpUsedSet(MP);
 
     verifyPreservedAnalysis(MP);
     removeNotPreservedAnalysis(MP);
     recordAvailableAnalysis(MP);
     removeDeadPasses(MP, M.getModuleIdentifier(), ON_MODULE_MSG);
   }

lib/IR/Module.cpp

 #include <cassert>
 #include <cstdint>
 #include <memory>
 #include <utility>
 #include <vector>
 
 using namespace llvm;
 
+static cl::opt<bool> OmitDuplicatedDump("omit-duplicated-dump",
+cl::init(false), cl::Hidden);
+
 //===----------------------------------------------------------------------===//
 // Methods to implement the globals and functions lists.
 //
 
 // Explicit instantiations of SymbolTableListTraits since some of the methods
 // are not in the public header file.
 template class llvm::SymbolTableListTraits<Function>;
 template class llvm::SymbolTableListTraits<GlobalVariable>;
 template class llvm::SymbolTableListTraits<GlobalAlias>;
 template class llvm::SymbolTableListTraits<GlobalIFunc>;
 
 //===----------------------------------------------------------------------===//
 // Primitive Module methods.
 //
 
 Module::Module(StringRef MID, LLVMContext &C)
-    : Context(C), Materializer(), ModuleID(MID), SourceFileName(MID), DL("") {
+    : Context(C), Materializer(), ModuleID(MID), SourceFileName(MID), DL(""),
+      FuncsUnchangedAfterDump() {
   ValSymTab = new ValueSymbolTable();
   NamedMDSymTab = new StringMap<NamedMDNode *>();
   Context.addModule(this);
+  if (OmitDuplicatedDump)
+    FuncsUnchangedAfterDump.reset(new SmallPtrSet<const Function*, 16>());
 }
 
 Module::~Module() {
   Context.removeModule(this);
   dropAllReferences();
   GlobalList.clear();
   FunctionList.clear();
   AliasList.clear();
   IFuncList.clear();
   NamedMDList.clear();
   delete ValSymTab;
   delete static_cast<StringMap<NamedMDNode *> *>(NamedMDSymTab);
 }
 
 RandomNumberGenerator *Module::createRNG(const Pass* P) const {
   SmallString<32> Salt(P->getPassName());
 
   // This RNG is guaranteed to produce the same random stream only
   // when the Module ID and thus the input filename is the same. This
   // might be problematic if the input filename extension changes
   // (e.g. from .c to .bc or .ll).
   //