[Debugify][OriginalDIMode] Export the report into JSON file

[djtodoro]

Summary

By using the option in the combination with the llvm/utils/llvm-original-di-preservation.py it becomes very user friendly tool. An example of the HTML page with the issues related to debug info can be found at [0].

[0] https://djolertrk.github.io/di-checker-html-report-example/

[djtodoro]

• Rebase

[djtodoro]

• Rebase on top of D83391

[djtodoro]

• Rebase

[djtodoro]

• Rebasing
• Refactoring

[djtodoro]

-Rebasing

[djtodoro]

• Rebase on top of trunk

[djtodoro]

Ping :)

[StephenTozer]

The fundamentals of the patch look solid; minor point about the HTML templating, and also I think there should be a couple of tests for the llvm-original-di-preservation script - nothing too complicated, just verifying that it produces the correct HTML for a given input, as with the opt-viewer tests.

llvm/utils/llvm-original-di-preservation.py

32

For each of these blocks, instead of using repeated html_header += you could instead use triple quotes and just write out the full HTML verbatim (using formatting to insert variables as needed), as in the other HTML generating python scripts.

[djtodoro]

@StephenTozer Thanks!

In D82546#2596942, @StephenTozer wrote:

and also I think there should be a couple of tests for the llvm-original-di-preservation script - nothing too complicated, just verifying that it produces the correct HTML for a given input, as with the opt-viewer tests.

I'll check this.

llvm/utils/llvm-original-di-preservation.py

32

It makes sense.

[djtodoro]

• adding a test

[djtodoro]

• adjust the test to work on Windows as well

[djtodoro]

ping :)

[aprantl]

llvm/include/llvm/Transforms/Utils/Debugify.h

169

The = ""; is redundant.

llvm/docs/HowToUpdateDebugInfo.rst

 .. code-block:: bash
 
   # Run the pass by checking original Debug Info preservation.
   $ opt -verify-debuginfo-preserve -pass-to-test sample.ll
 
   # Check the preservation of original Debug Info after each pass.
   $ opt -verify-each-debuginfo-preserve -O2 sample.ll
 
+Furthermore, there is a way to export the issues that have been found into
+a JSON file as follows:
+
+.. code-block:: bash
+
+  $ opt -verify-debuginfo-preserve -verify-di-preserve-export=sample.json -pass-to-test sample.ll
+
+and then use the ``llvm/utils/llvm-original-di-preservation.py`` script
+to generate an HTML page with the issues reported in a more human readable form
+as follows:
+
+.. code-block:: bash
+
+  $ llvm-original-di-preservation.py sample.json sample.html
+
 Mutation testing for MIR-level transformations
 ----------------------------------------------
 
 A variant of the ``debugify`` utility described in
 :ref:`Mutation testing for IR-level transformations<IRDebugify>` can be used
 for MIR-level transformations as well: much like the IR-level pass,
 ``mir-debugify`` inserts sequentially increasing line locations to each
 ``MachineInstr`` in a ``Module``. And the MIR-level ``mir-check-debugify`` is

llvm/include/llvm/Transforms/Utils/Debugify.h

 /// \param M The module to collect debug information from.
 /// \param Functions A range of functions to collect debug information from.
 /// \param DIPreservationMap A map used to check collected the DI metadata.
 /// \param Banner A prefix string to add to debug/error messages.
 /// \param NameOfWrappedPass A name of a pass to add to debug/error messages.
 bool checkDebugInfoMetadata(Module &M,
                             iterator_range<Module::iterator> Functions,
                             DebugInfoPerPassMap &DIPreservationMap,
-                            StringRef Banner, StringRef NameOfWrappedPass);
+                            StringRef Banner, StringRef NameOfWrappedPass,
+                            StringRef OrigDIVerifyBugsReportFilePath);
 } // namespace llvm
 
 /// Used to check whether we track synthetic or original debug info.
 enum class DebugifyMode { NoDebugify, SyntheticDebugInfo, OriginalDebugInfo };
 
 llvm::ModulePass *createDebugifyModulePass(
     enum DebugifyMode Mode = DebugifyMode::SyntheticDebugInfo,
     llvm::StringRef NameOfWrappedPass = "",

 
 /// Map pass names to a per-pass DebugifyStatistics instance.
 using DebugifyStatsMap = llvm::MapVector<llvm::StringRef, DebugifyStatistics>;
 
 llvm::ModulePass *createCheckDebugifyModulePass(
     bool Strip = false, llvm::StringRef NameOfWrappedPass = "",
     DebugifyStatsMap *StatsMap = nullptr,
     enum DebugifyMode Mode = DebugifyMode::SyntheticDebugInfo,
-    DebugInfoPerPassMap *DIPreservationMap = nullptr);
+    DebugInfoPerPassMap *DIPreservationMap = nullptr,
+    llvm::StringRef OrigDIVerifyBugsReportFilePath = "");
 
 llvm::FunctionPass *createCheckDebugifyFunctionPass(
     bool Strip = false, llvm::StringRef NameOfWrappedPass = "",
     DebugifyStatsMap *StatsMap = nullptr,
     enum DebugifyMode Mode = DebugifyMode::SyntheticDebugInfo,
-    DebugInfoPerPassMap *DIPreservationMap = nullptr);
+    DebugInfoPerPassMap *DIPreservationMap = nullptr,
+    llvm::StringRef OrigDIVerifyBugsReportFilePath = "");
 
 struct NewPMCheckDebugifyPass
     : public llvm::PassInfoMixin<NewPMCheckDebugifyPass> {
   llvm::PreservedAnalyses run(llvm::Module &M, llvm::ModuleAnalysisManager &AM);
 };
 
 namespace llvm {
 void exportDebugifyStats(StringRef Path, const DebugifyStatsMap &Map);
 
 struct DebugifyEachInstrumentation {
   DebugifyStatsMap StatsMap;
 
   void registerCallbacks(PassInstrumentationCallbacks &PIC);
 };
 
 /// DebugifyCustomPassManager wraps each pass with the debugify passes if
 /// needed.
 /// NOTE: We support legacy custom pass manager only.
 /// TODO: Add New PM support for custom pass manager.
 class DebugifyCustomPassManager : public legacy::PassManager {
+  StringRef OrigDIVerifyBugsReportFilePath = "";
   DebugifyStatsMap *DIStatsMap = nullptr;
   DebugInfoPerPassMap *DIPreservationMap = nullptr;
   enum DebugifyMode Mode = DebugifyMode::NoDebugify;
 
 public:
   using super = legacy::PassManager;
 
   void add(Pass *P) override {
     // Wrap each pass with (-check)-debugify passes if requested, making
     // exceptions for passes which shouldn't see -debugify instrumentation.
-    bool WrapWithDebugify =
-        Mode != DebugifyMode::NoDebugify &&
+    bool WrapWithDebugify = Mode != DebugifyMode::NoDebugify &&
                             !P->getAsImmutablePass() && !isIRPrintingPass(P) &&
                             !isBitcodeWriterPass(P);
     if (!WrapWithDebugify) {
       super::add(P);
       return;
     }
 
     // Either apply -debugify/-check-debugify before/after each pass and collect
     // debug info loss statistics, or collect and check original debug info in
     // the optimizations.
     PassKind Kind = P->getPassKind();
     StringRef Name = P->getPassName();
 
     // TODO: Implement Debugify for LoopPass.
     switch (Kind) {
     case PT_Function:
       super::add(createDebugifyFunctionPass(Mode, Name, DIPreservationMap));
       super::add(P);
       super::add(createCheckDebugifyFunctionPass(
-          isSyntheticDebugInfo(), Name, DIStatsMap, Mode, DIPreservationMap));
+          isSyntheticDebugInfo(), Name, DIStatsMap, Mode, DIPreservationMap,
+          OrigDIVerifyBugsReportFilePath));
       break;
     case PT_Module:
       super::add(createDebugifyModulePass(Mode, Name, DIPreservationMap));
       super::add(P);
       super::add(createCheckDebugifyModulePass(
-          isSyntheticDebugInfo(), Name, DIStatsMap, Mode, DIPreservationMap));
+          isSyntheticDebugInfo(), Name, DIStatsMap, Mode, DIPreservationMap,
+          OrigDIVerifyBugsReportFilePath));
       break;
     default:
       super::add(P);
       break;
     }
   }
 
   // Used within DebugifyMode::SyntheticDebugInfo mode.
   void setDIStatsMap(DebugifyStatsMap &StatMap) { DIStatsMap = &StatMap; }
   // Used within DebugifyMode::OriginalDebugInfo mode.
   void setDIPreservationMap(DebugInfoPerPassMap &PerPassMap) {
     DIPreservationMap = &PerPassMap;
   }
+  void setOrigDIVerifyBugsReportFilePath(StringRef BugsReportFilePath) {
+    OrigDIVerifyBugsReportFilePath = BugsReportFilePath;
+  }
+  StringRef getOrigDIVerifyBugsReportFilePath() const {
+    return OrigDIVerifyBugsReportFilePath;
+  }
+
   void setDebugifyMode(enum DebugifyMode M) { Mode = M; }
 
   bool isSyntheticDebugInfo() const {
     return Mode == DebugifyMode::SyntheticDebugInfo;
   }
   bool isOriginalDebugInfoMode() const {
     return Mode == DebugifyMode::OriginalDebugInfo;
   }
 
   const DebugifyStatsMap &getDebugifyStatsMap() const { return *DIStatsMap; }
   DebugInfoPerPassMap &getDebugInfoPerPassMap() { return *DIPreservationMap; }
 };
 } // namespace llvm
 
 #endif // LLVM_TRANSFORMS_UTILS_DEBUGIFY_H

llvm/lib/Transforms/Utils/Debugify.cpp

 #include "llvm/IR/DebugInfo.h"
 #include "llvm/IR/InstIterator.h"
 #include "llvm/IR/Instructions.h"
 #include "llvm/IR/IntrinsicInst.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/PassInstrumentation.h"
 #include "llvm/Pass.h"
 #include "llvm/Support/CommandLine.h"
+#include "llvm/Support/JSON.h"
 
 #define DEBUG_TYPE "debugify"
 
 using namespace llvm;
 
 namespace {
 
 cl::opt<bool> Quiet("debugify-quiet",
bool llvm::collectDebugInfoMetadata(Module &M,
 
   return true;
 }
 
 // This checks the preservation of original debug info attached to functions.
 static bool checkFunctions(const DebugFnMap &DIFunctionsBefore,
                            const DebugFnMap &DIFunctionsAfter,
                            StringRef NameOfWrappedPass,
-                           StringRef FileNameFromCU) {
+                           StringRef FileNameFromCU, bool ShouldWriteIntoJSON,
+                           llvm::json::Array &Bugs) {
   bool Preserved = true;
   for (const auto &F : DIFunctionsAfter) {
     if (F.second)
       continue;
     auto SPIt = DIFunctionsBefore.find(F.first);
     if (SPIt == DIFunctionsBefore.end()) {
+      if (ShouldWriteIntoJSON)
+        Bugs.push_back(llvm::json::Object({{"metadata", "DISubprogram"},
+                                           {"name", F.first},
+                                           {"action", "not-generate"}}));
+      else
         dbg() << "ERROR: " << NameOfWrappedPass
               << " did not generate DISubprogram for " << F.first << " from "
               << FileNameFromCU << '\n';
       Preserved = false;
     } else {
       auto SP = SPIt->second;
       if (!SP)
         continue;
       // If the function had the SP attached before the pass, consider it as
       // a debug info bug.
+      if (ShouldWriteIntoJSON)
+        Bugs.push_back(llvm::json::Object({{"metadata", "DISubprogram"},
+                                           {"name", F.first},
+                                           {"action", "drop"}}));
+      else
         dbg() << "ERROR: " << NameOfWrappedPass << " dropped DISubprogram of "
               << F.first << " from " << FileNameFromCU << '\n';
       Preserved = false;
     }
   }
 
   return Preserved;
 }
 
 // This checks the preservation of the original debug info attached to
 // instructions.
 static bool checkInstructions(const DebugInstMap &DILocsBefore,
                               const DebugInstMap &DILocsAfter,
                               const WeakInstValueMap &InstToDelete,
                               StringRef NameOfWrappedPass,
-                              StringRef FileNameFromCU) {
+                              StringRef FileNameFromCU,
+                              bool ShouldWriteIntoJSON,
+                              llvm::json::Array &Bugs) {
   bool Preserved = true;
   for (const auto &L : DILocsAfter) {
     if (L.second)
       continue;
     auto Instr = L.first;
 
     // In order to avoid pointer reuse/recycling, skip the values that might
     // have been deleted during a pass.
     auto WeakInstrPtr = InstToDelete.find(Instr);
     if (WeakInstrPtr != InstToDelete.end() && !WeakInstrPtr->second)
       continue;
 
     auto FnName = Instr->getFunction()->getName();
     auto BB = Instr->getParent();
     auto BBName = BB->hasName() ? BB->getName() : "no-name";
+    auto InstName = Instruction::getOpcodeName(Instr->getOpcode());
 
     auto InstrIt = DILocsBefore.find(Instr);
     if (InstrIt == DILocsBefore.end()) {
+      if (ShouldWriteIntoJSON)
+        Bugs.push_back(llvm::json::Object({{"metadata", "DILocation"},
+                                           {"fn-name", FnName.str()},
+                                           {"bb-name", BBName.str()},
+                                           {"instr", InstName},
+                                           {"action", "not-generate"}}));
+      else
         dbg() << "WARNING: " << NameOfWrappedPass
               << " did not generate DILocation for " << *Instr
               << " (BB: " << BBName << ", Fn: " << FnName
               << ", File: " << FileNameFromCU << ")\n";
       Preserved = false;
     } else {
       if (!InstrIt->second)
         continue;
       // If the instr had the !dbg attached before the pass, consider it as
       // a debug info issue.
+      if (ShouldWriteIntoJSON)
+        Bugs.push_back(llvm::json::Object({{"metadata", "DILocation"},
+                                           {"fn-name", FnName.str()},
+                                           {"bb-name", BBName.str()},
+                                           {"instr", InstName},
+                                           {"action", "drop"}}));
+      else
         dbg() << "WARNING: " << NameOfWrappedPass << " dropped DILocation of "
               << *Instr << " (BB: " << BBName << ", Fn: " << FnName
               << ", File: " << FileNameFromCU << ")\n";
       Preserved = false;
     }
   }
 
   return Preserved;
 }
 
+// Write the json data into the specifed file.
+static void writeJSON(StringRef OrigDIVerifyBugsReportFilePath,
+                      StringRef FileNameFromCU, StringRef NameOfWrappedPass,
+                      llvm::json::Array &Bugs) {
+  std::error_code EC;
+  raw_fd_ostream OS_FILE{OrigDIVerifyBugsReportFilePath, EC,
+                         sys::fs::OF_Append | sys::fs::OF_Text};
+  if (EC) {
+    errs() << "Could not open file: " << EC.message() << ", "
+           << OrigDIVerifyBugsReportFilePath << '\n';
+    return;
+  }
+
+  OS_FILE << "{\"file\":\"" << FileNameFromCU << "\", ";
+  OS_FILE << "\"pass\":\"" << NameOfWrappedPass << "\", ";
+
+  llvm::json::Value BugsToPrint{std::move(Bugs)};
+  OS_FILE << "\"bugs\": " << BugsToPrint;
+
+  OS_FILE << "}\n";
+}
+
 bool llvm::checkDebugInfoMetadata(Module &M,
                                   iterator_range<Module::iterator> Functions,
                                   DebugInfoPerPassMap &DIPreservationMap,
-                                  StringRef Banner,
-                                  StringRef NameOfWrappedPass) {
+                                  StringRef Banner, StringRef NameOfWrappedPass,
+                                  StringRef OrigDIVerifyBugsReportFilePath) {
   LLVM_DEBUG(dbgs() << Banner << ": (after) " << NameOfWrappedPass << '\n');
 
   if (!M.getNamedMetadata("llvm.dbg.cu")) {
     dbg() << Banner << ": Skipping module without debug info\n";
     return false;
   }
 
   // Map the debug info holding DIs after a pass.
   DebugInfoPerPassMap DIPreservationAfter;
 
   // Visit each instruction.
   for (Function &F : Functions) {
     if (isFunctionSkipped(F))
       continue;
 
     // TODO: Collect metadata other than DISubprograms.
     // Collect the DISubprogram.
     auto *SP = F.getSubprogram();
-    DIPreservationAfter[NameOfWrappedPass].DIFunctions.insert({F.getName(), SP});
+    DIPreservationAfter[NameOfWrappedPass].DIFunctions.insert(
+        {F.getName(), SP});
     if (SP)
       LLVM_DEBUG(dbgs() << "  Collecting subprogram: " << *SP << '\n');
 
     for (BasicBlock &BB : F) {
       // Collect debug locations (!dbg attachments).
       // TODO: Collect dbg.values.
       for (Instruction &I : BB) {
         // Skip PHIs.
bool llvm::checkDebugInfoMetadata(Module &M,
   auto DIFunctionsBefore = DIPreservationMap[NameOfWrappedPass].DIFunctions;
   auto DIFunctionsAfter = DIPreservationAfter[NameOfWrappedPass].DIFunctions;
 
   auto DILocsBefore = DIPreservationMap[NameOfWrappedPass].DILocations;
   auto DILocsAfter = DIPreservationAfter[NameOfWrappedPass].DILocations;
 
   auto InstToDelete = DIPreservationAfter[NameOfWrappedPass].InstToDelete;
 
-  bool ResultForFunc = checkFunctions(DIFunctionsBefore, DIFunctionsAfter,
-                                      NameOfWrappedPass, FileNameFromCU);
-  bool ResultForInsts =
-      checkInstructions(DILocsBefore, DILocsAfter, InstToDelete,
-                        NameOfWrappedPass, FileNameFromCU);
+  bool ShouldWriteIntoJSON = !OrigDIVerifyBugsReportFilePath.empty();
+  llvm::json::Array Bugs;
+
+  bool ResultForFunc =
+      checkFunctions(DIFunctionsBefore, DIFunctionsAfter, NameOfWrappedPass,
+                     FileNameFromCU, ShouldWriteIntoJSON, Bugs);
+  bool ResultForInsts = checkInstructions(
+      DILocsBefore, DILocsAfter, InstToDelete, NameOfWrappedPass,
+      FileNameFromCU, ShouldWriteIntoJSON, Bugs);
   bool Result = ResultForFunc && ResultForInsts;
 
   StringRef ResultBanner = NameOfWrappedPass != "" ? NameOfWrappedPass : Banner;
+  if (ShouldWriteIntoJSON && !Bugs.empty())
+    writeJSON(OrigDIVerifyBugsReportFilePath, FileNameFromCU, ResultBanner,
+              Bugs);
+
   if (Result)
     dbg() << ResultBanner << ": PASS\n";
   else
     dbg() << ResultBanner << ": FAIL\n";
 
   LLVM_DEBUG(dbgs() << "\n\n");
   return Result;
 }

 
 /// ModulePass for checking debug info inserted by -debugify, used with the
 /// legacy module pass manager.
 struct CheckDebugifyModulePass : public ModulePass {
   bool runOnModule(Module &M) override {
     if (Mode == DebugifyMode::SyntheticDebugInfo)
       return checkDebugifyMetadata(M, M.functions(), NameOfWrappedPass,
                                    "CheckModuleDebugify", Strip, StatsMap);
     return checkDebugInfoMetadata(
         M, M.functions(), *DIPreservationMap,
-        "CheckModuleDebugify (original debuginfo)", NameOfWrappedPass);
+        "CheckModuleDebugify (original debuginfo)", NameOfWrappedPass,
+        OrigDIVerifyBugsReportFilePath);
   }
 
   CheckDebugifyModulePass(
       bool Strip = false, StringRef NameOfWrappedPass = "",
       DebugifyStatsMap *StatsMap = nullptr,
       enum DebugifyMode Mode = DebugifyMode::SyntheticDebugInfo,
-      DebugInfoPerPassMap *DIPreservationMap = nullptr)
+      DebugInfoPerPassMap *DIPreservationMap = nullptr,
+      StringRef OrigDIVerifyBugsReportFilePath = "")
       : ModulePass(ID), NameOfWrappedPass(NameOfWrappedPass),
+        OrigDIVerifyBugsReportFilePath(OrigDIVerifyBugsReportFilePath),
         StatsMap(StatsMap), DIPreservationMap(DIPreservationMap), Mode(Mode),
         Strip(Strip) {}
 
   void getAnalysisUsage(AnalysisUsage &AU) const override {
     AU.setPreservesAll();
   }
 
   static char ID; // Pass identification.
 
 private:
   StringRef NameOfWrappedPass;
+  StringRef OrigDIVerifyBugsReportFilePath;
   DebugifyStatsMap *StatsMap;
   DebugInfoPerPassMap *DIPreservationMap;
   enum DebugifyMode Mode;
   bool Strip;
 };
 
 /// FunctionPass for checking debug info inserted by -debugify-function, used
 /// with the legacy module pass manager.
 struct CheckDebugifyFunctionPass : public FunctionPass {
   bool runOnFunction(Function &F) override {
     Module &M = *F.getParent();
     auto FuncIt = F.getIterator();
     if (Mode == DebugifyMode::SyntheticDebugInfo)
       return checkDebugifyMetadata(M, make_range(FuncIt, std::next(FuncIt)),
                                    NameOfWrappedPass, "CheckFunctionDebugify",
                                    Strip, StatsMap);
     return checkDebugInfoMetadata(
         M, make_range(FuncIt, std::next(FuncIt)), *DIPreservationMap,
-        "CheckFunctionDebugify (original debuginfo)", NameOfWrappedPass);
+        "CheckFunctionDebugify (original debuginfo)", NameOfWrappedPass,
+        OrigDIVerifyBugsReportFilePath);
   }
 
   CheckDebugifyFunctionPass(
       bool Strip = false, StringRef NameOfWrappedPass = "",
       DebugifyStatsMap *StatsMap = nullptr,
       enum DebugifyMode Mode = DebugifyMode::SyntheticDebugInfo,
-      DebugInfoPerPassMap *DIPreservationMap = nullptr)
+      DebugInfoPerPassMap *DIPreservationMap = nullptr,
+      StringRef OrigDIVerifyBugsReportFilePath = "")
       : FunctionPass(ID), NameOfWrappedPass(NameOfWrappedPass),
+        OrigDIVerifyBugsReportFilePath(OrigDIVerifyBugsReportFilePath),
         StatsMap(StatsMap), DIPreservationMap(DIPreservationMap), Mode(Mode),
         Strip(Strip) {}
 
   void getAnalysisUsage(AnalysisUsage &AU) const override {
     AU.setPreservesAll();
   }
 
   static char ID; // Pass identification.
 
 private:
   StringRef NameOfWrappedPass;
+  StringRef OrigDIVerifyBugsReportFilePath;
   DebugifyStatsMap *StatsMap;
   DebugInfoPerPassMap *DIPreservationMap;
   enum DebugifyMode Mode;
   bool Strip;
 };
 
 } // end anonymous namespace
 

 PreservedAnalyses NewPMDebugifyPass::run(Module &M, ModuleAnalysisManager &) {
   applyDebugifyMetadata(M, M.functions(),
                         "ModuleDebugify: ", /*ApplyToMF*/ nullptr);
   return PreservedAnalyses::all();
 }
 
 ModulePass *createCheckDebugifyModulePass(
     bool Strip, StringRef NameOfWrappedPass, DebugifyStatsMap *StatsMap,
-    enum DebugifyMode Mode, DebugInfoPerPassMap *DIPreservationMap) {
+    enum DebugifyMode Mode, DebugInfoPerPassMap *DIPreservationMap,
+    StringRef OrigDIVerifyBugsReportFilePath) {
   if (Mode == DebugifyMode::SyntheticDebugInfo)
     return new CheckDebugifyModulePass(Strip, NameOfWrappedPass, StatsMap);
   assert(Mode == DebugifyMode::OriginalDebugInfo && "Must be original mode");
   return new CheckDebugifyModulePass(false, NameOfWrappedPass, nullptr, Mode,
-                                     DIPreservationMap);
+                                     DIPreservationMap,
+                                     OrigDIVerifyBugsReportFilePath);
 }
 
 FunctionPass *createCheckDebugifyFunctionPass(
     bool Strip, StringRef NameOfWrappedPass, DebugifyStatsMap *StatsMap,
-    enum DebugifyMode Mode, DebugInfoPerPassMap *DIPreservationMap) {
+    enum DebugifyMode Mode, DebugInfoPerPassMap *DIPreservationMap,
+    StringRef OrigDIVerifyBugsReportFilePath) {
   if (Mode == DebugifyMode::SyntheticDebugInfo)
     return new CheckDebugifyFunctionPass(Strip, NameOfWrappedPass, StatsMap);
   assert(Mode == DebugifyMode::OriginalDebugInfo && "Must be original mode");
   return new CheckDebugifyFunctionPass(false, NameOfWrappedPass, nullptr, Mode,
-                                       DIPreservationMap);
+                                       DIPreservationMap,
+                                       OrigDIVerifyBugsReportFilePath);
 }
 
 PreservedAnalyses NewPMCheckDebugifyPass::run(Module &M,
                                               ModuleAnalysisManager &) {
   checkDebugifyMetadata(M, M.functions(), "", "CheckModuleDebugify", false,
                         nullptr);
   return PreservedAnalyses::all();
 }

llvm/tools/opt/opt.cpp

static cl::opt<bool> VerifyDebugInfoPreserve(
     cl::desc("Start the pipeline with collecting and end it with checking of "
              "debug info preservation."));
 
 static cl::opt<bool> VerifyEachDebugInfoPreserve(
     "verify-each-debuginfo-preserve",
     cl::desc("Start each pass with collecting and end it with checking of "
              "debug info preservation."));
 
+static cl::opt<std::string>
+    VerifyDIPreserveExport("verify-di-preserve-export",
+                   cl::desc("Export debug info preservation failures into "
+                            "specified (JSON) file (should be abs path as we use"
+                            " append mode to insert new JSON objects)"),
+                   cl::value_desc("filename"), cl::init(""));
+
 static cl::opt<bool>
 PrintBreakpoints("print-breakpoints-for-testing",
                  cl::desc("Print select breakpoints location for testing"));
 
 static cl::opt<std::string> ClDataLayout("data-layout",
                                          cl::desc("data layout string to use"),
                                          cl::value_desc("layout-string"),
                                          cl::init(""));
#endif
   DebugifyStatsMap DIStatsMap;
   DebugInfoPerPassMap DIPreservationMap;
   if (DebugifyEach) {
     Passes.setDebugifyMode(DebugifyMode::SyntheticDebugInfo);
     Passes.setDIStatsMap(DIStatsMap);
   } else if (VerifyEachDebugInfoPreserve) {
     Passes.setDebugifyMode(DebugifyMode::OriginalDebugInfo);
     Passes.setDIPreservationMap(DIPreservationMap);
+    if (!VerifyDIPreserveExport.empty())
+      Passes.setOrigDIVerifyBugsReportFilePath(VerifyDIPreserveExport);
   }
 
   bool AddOneTimeDebugifyPasses =
       (EnableDebugify && !DebugifyEach) ||
       (VerifyDebugInfoPreserve && !VerifyEachDebugInfoPreserve);
 
   Passes.add(new TargetLibraryInfoWrapperPass(TLII));
 

llvm/utils/llvm-original-di-preservation.py

+#!/usr/bin/env python
+#
+# Debugify summary for the original debug info testing.
+#
+
+from __future__ import print_function
+import argparse
+import os
+import sys
+from json import loads
+from collections import defaultdict
+from collections import OrderedDict
+
+class DILocBug:
+  def __init__(self, action, bb_name, fn_name, instr):
+    self.action = action
+    self.bb_name = bb_name
+    self.fn_name = fn_name
+    self.instr = instr
+
+class DISPBug:
+  def __init__(self, action, fn_name):
+    self.action = action
+    self.fn_name = fn_name
+
+# Report the bugs in form of html.
+def generate_html_report(di_location_bugs, di_subprogram_bugs, \
+                         di_location_bugs_summary, di_sp_bugs_summary, \
+                         html_file):
+  fileout = open(html_file, "w")
+
+  html_header = "<html>\n"
+  html_header += "<head>\n"
+  html_header += "<style>\n"
+  html_header += "table, th, td {\n"
+  html_header += "  border: 1px solid black;\n"
+  html_header += "}\n"
+  html_header += "table.center {\n"
+  html_header += "  margin-left: auto;\n"
+  html_header += "  margin-right: auto;\n"
+  html_header += "}\n"
+  html_header += "</style>\n"
+  html_header += "</head>\n"
+  html_header += "<body>\n"
+
+  # Create the table for Location bugs.
+  table_di_loc = "<table>\n"
+
+  # Create the table's column headers.
+  table_title_di_loc = "Location Bugs found by the LLVM DI Checker"
+  table_di_loc += "<caption><b>"
+  table_di_loc += table_title_di_loc
+  table_di_loc += "</b></caption>\n"
+
+  header_di_loc = ["File", "LLVM Pass Name", "LLVM IR Instruction", \
+                   "Function Name", "Basic Block Name", "Action"]
+  table_di_loc += "  <tr>\n"
+
+  for column in header_di_loc:
+    table_di_loc += "    <th>{0}</th>\n".format(column.strip())
+  table_di_loc += "  </tr>\n"
+
+  at_least_one_bug_found = False
+
+  # Handle loction bugs.
+  for file, per_file_bugs in di_location_bugs.items():
+    for llvm_pass, per_pass_bugs in per_file_bugs.items():
+      # No location bugs for the pass.
+      if len(per_pass_bugs) == 0:
+        continue
+      at_least_one_bug_found = True
+      row = []
+      table_di_loc += "  </tr>\n"
+      # Get the bugs info.
+      for x in per_pass_bugs:
+        row.append("    <tr>\n")
+        row.append(file)
+        row.append(llvm_pass)
+        row.append(x.instr)
+        row.append(x.fn_name)
+        row.append(x.bb_name)
+        row.append(x.action)
+        row.append("    </tr>\n")
+      # Dump the bugs info into the table.
+      for column in row:
+        # The same file-pass pair can have multiple bugs.
+        if (column == "    <tr>\n" or column == "    </tr>\n"):
+          table_di_loc += column
+          continue
+        table_di_loc += "    <td>{0}</td>\n".format(column.strip())
+      table_di_loc += "  <tr>\n"
+
+  if not at_least_one_bug_found:
+    table_di_loc += "  <tr>\n"
+    table_di_loc += "    <td colspan='7'> No bugs found </td>\n"
+    table_di_loc += "  </tr>\n"
+
+  table_di_loc += "</table>\n"
+
+  # Create the summary table for the loc bugs.
+  table_di_loc_sum = "<table>\n"
+
+  # Create the table's column headers.
+  table_title_di_loc_sum = "DI Checker Summary of Location Bugs"
+  table_di_loc_sum += "<caption><b>"
+  table_di_loc_sum += table_title_di_loc_sum
+  table_di_loc_sum += "</b></caption>\n"
+
+  header_di_loc_sum = ["LLVM Pass Name", "Number of bugs"]
+  table_di_loc_sum += "  <tr>\n"
+
+  for column in header_di_loc_sum:
+    table_di_loc_sum += "    <th>{0}</th>\n".format(column.strip())
+  table_di_loc_sum += "  </tr>\n"
+
+  # Print the summary.
+  row = []
+  for llvm_pass, num in sorted(di_location_bugs_summary.items()):
+    row.append("    <tr>\n")
+    row.append(llvm_pass)
+    row.append(str(num))
+    row.append("    </tr>\n")
+  for column in row:
+    if (column == "    <tr>\n" or column == "    </tr>\n"):
+      table_di_loc_sum += column
+      continue
+    table_di_loc_sum += "    <td>{0}</td>\n".format(column.strip())
+  table_di_loc_sum += "  <tr>\n"
+
+  if not at_least_one_bug_found:
+    table_di_loc_sum += "  <tr>\n"
+    table_di_loc_sum += "    <td colspan='2'> No bugs found </td>\n"
+    table_di_loc_sum += "  </tr>\n"
+  table_di_loc_sum += "</table>\n"
+
+  # Create the table for SP bugs.
+  table_di_sp = "<table>\n"
+
+  # Create the table's column headers.
+  table_title_di_sp = "SP Bugs found by the LLVM DI Checker"
+  table_di_sp += "<caption><b>"
+  table_di_sp += table_title_di_sp
+  table_di_sp += "</b></caption>\n"
+
+  header_di_sp = ["File", "LLVM Pass Name", "Function Name", "Action"]
+  table_di_sp += "  <tr>\n"
+
+  for column in header_di_sp:
+    table_di_sp += "    <th>{0}</th>\n".format(column.strip())
+  table_di_sp += "  </tr>\n"
+
+  at_least_one_bug_found = False
+
+  # Handle loction bugs.
+  for file, per_file_bugs in di_subprogram_bugs.items():
+    for llvm_pass, per_pass_bugs in per_file_bugs.items():
+      # No SP bugs for the pass.
+      if len(per_pass_bugs) == 0:
+        continue
+      at_least_one_bug_found = True
+      row = []
+      table_di_sp += "  </tr>\n"
+      # Get the bugs info.
+      for x in per_pass_bugs:
+        row.append("    <tr>\n")
+        row.append(file)
+        row.append(llvm_pass)
+        row.append(x.fn_name)
+        row.append(x.action)
+        row.append("    </tr>\n")
+      # Dump the bugs info into the table.
+      for column in row:
+        # The same file-pass pair can have multiple bugs.
+        if (column == "    <tr>\n" or column == "    </tr>\n"):
+          table_di_sp += column
+          continue
+        table_di_sp += "    <td>{0}</td>\n".format(column.strip())
+      table_di_sp += "  <tr>\n"
+
+  if not at_least_one_bug_found:
+    table_di_sp += "  <tr>\n"
+    table_di_sp += "    <td colspan='4'> No bugs found </td>\n"
+    table_di_sp += "  </tr>\n"
+
+  table_di_sp += "</table>\n"
+
+  # Create the summary table for the sp bugs.
+  table_di_sp_sum = "<table>\n"
+
+  # Create the table's column headers.
+  table_title_di_sp_sum = "DI Checker Summary of SP Bugs"
+  table_di_sp_sum += "<caption><b>"
+  table_di_sp_sum += table_title_di_sp_sum
+  table_di_sp_sum += "</b></caption>\n"
+
+  header_di_sp_sum = ["LLVM Pass Name", "Number of bugs"]
+  table_di_sp_sum += "  <tr>\n"
+
+  for column in header_di_sp_sum:
+    table_di_sp_sum += "    <th>{0}</th>\n".format(column.strip())
+  table_di_sp_sum += "  </tr>\n"
+
+  # Print the summary.
+  row = []
+  for llvm_pass, num in sorted(di_sp_bugs_summary.items()):
+    row.append("    <tr>\n")
+    row.append(llvm_pass)
+    row.append(str(num))
+    row.append("    </tr>\n")
+  for column in row:
+    if (column == "    <tr>\n" or column == "    </tr>\n"):
+      table_di_sp_sum += column
+      continue
+    table_di_sp_sum += "    <td>{0}</td>\n".format(column.strip())
+  table_di_sp_sum += "  <tr>\n"
+
+  if not at_least_one_bug_found:
+    table_di_sp_sum += "  <tr>\n"
+    table_di_sp_sum += "    <td colspan='2'> No bugs found </td>\n"
+    table_di_sp_sum += "  </tr>\n"
+  table_di_sp_sum += "</table>\n"
+
+
+  # Finish the html page.
+  html_footer = "</body>\n"
+  html_footer += "</html>\n"
+  new_line = "<br>\n"
+
+  fileout.writelines(html_header)
+  fileout.writelines(table_di_loc)
+  fileout.writelines(new_line)
+  fileout.writelines(table_di_loc_sum)
+  fileout.writelines(new_line)
+  fileout.writelines(new_line)
+  fileout.writelines(table_di_sp)
+  fileout.writelines(new_line)
+  fileout.writelines(table_di_sp_sum)
+  fileout.writelines(html_footer)
+  fileout.close()
+
+  print("The " + html_file + " generated.")
+
+# Read the JSON file.
+def get_json(file):
+  json_parsed = None
+  di_checker_data = []
+
+  # The file contains json object per line.
+  # An example of the line (formatted json):
+  # {
+  #  "file": "simple.c",
+  #  "pass": "Deduce function attributes in RPO",
+  #  "bugs": [
+  #    [
+  #      {
+  #        "action": "drop",
+  #        "metadata": "DISubprogram",
+  #        "name": "fn2"
+  #      },
+  #      {
+  #        "action": "drop",
+  #        "metadata": "DISubprogram",
+  #        "name": "fn1"
+  #      }
+  #    ]
+  #  ]
+  #}
+  with open(file) as json_objects_file:
+    for json_object_line in json_objects_file:
+      try:
+        json_object = loads(json_object_line)
+      except:
+        print ("error: No valid di-checker data found.")
+        sys.exit(1)
+      di_checker_data.append(json_object)
+
+  return di_checker_data
+
+# Parse the program arguments.
+def parse_program_args(parser):
+  parser.add_argument("file_name", type=str, help="json file to process")
+  parser.add_argument("html_file", type=str, help="html file to output data")
+
+  return parser.parse_args()
+
+def Main():
+  parser = argparse.ArgumentParser()
+  opts = parse_program_args(parser)
+
+  if not opts.html_file.endswith('.html'):
+    print ("error: The output file must be '.html'.")
+    sys.exit(1)
+
+  debug_info_bugs = get_json(opts.file_name)
+
+  # Use the defaultdict in order to make multidim dicts.
+  di_location_bugs = defaultdict(lambda: defaultdict(dict))
+  di_subprogram_bugs = defaultdict(lambda: defaultdict(dict))
+
+  # Use the ordered dict to make a summary.
+  di_location_bugs_summary = OrderedDict()
+  di_sp_bugs_summary = OrderedDict()
+
+  # Map the bugs into the file-pass pairs.
+  for bugs_per_pass in debug_info_bugs:
+    bugs_file = bugs_per_pass["file"]
+    bugs_pass = bugs_per_pass["pass"]
+
+    bugs = bugs_per_pass["bugs"][0]
+
+    di_loc_bugs = []
+    di_sp_bugs = []
+    for bug in bugs:
+      bugs_metadata = bug["metadata"]
+      if bugs_metadata == "DILocation":
+        action = bug["action"]
+        bb_name = bug["bb-name"]
+        fn_name = bug["fn-name"]
+        instr = bug["instr"]
+        di_loc_bugs.append(DILocBug(action, bb_name, fn_name, instr))
+
+        # Fill the summary dict.
+        if bugs_pass in di_location_bugs_summary:
+          di_location_bugs_summary[bugs_pass] += 1
+        else:
+          di_location_bugs_summary[bugs_pass] = 1
+      elif bugs_metadata == "DISubprogram":
+        action = bug["action"]
+        name = bug["name"]
+        di_sp_bugs.append(DISPBug(action, name))
+
+        # Fill the summary dict.
+        if bugs_pass in di_sp_bugs_summary:
+          di_sp_bugs_summary[bugs_pass] += 1
+        else:
+          di_sp_bugs_summary[bugs_pass] = 1
+      else:
+        print ("error: Only DILocation and DISubprogram are supported.")
+        sys.exit(1)
+
+    di_location_bugs[bugs_file][bugs_pass] = di_loc_bugs
+    di_subprogram_bugs[bugs_file][bugs_pass] = di_sp_bugs
+
+  generate_html_report(di_location_bugs, di_subprogram_bugs, \
+                       di_location_bugs_summary, di_sp_bugs_summary, \
+                       opts.html_file)
+
+if __name__ == "__main__":
+  Main()
+  sys.exit(0)