[Analyzer] Add support for displaying cross-file diagnostic paths in HTML output

[vlad.tsyrklevich]

Summary

While looking at checker output with D30289 applied I realized I was missing results. After some digging I found that it's because the Linux kernel makes heavy use of inlined functions included in header files, and the resulting diagnostic path now included cross-file diagnostics which are unsupported. This change adds support for cross-file diagnostic paths. If the diagnostic path is not cross-file, there is no change in the output.

[vlad.tsyrklevich]

Here's an example 3 file report from the Linux source: https://cdn.rawgit.com/vlad902/73bd32181151f2f0b07fa501a0f0b627/raw/0106f09cdb0d045b757de3d71ffffddc58072d33/report2.html

It's not as immediately clear this is a multi-file output. Ideas to improve this could include anchor tags in the header for every file included, or augmenting the line count to something like '$FILE:$LINE' instead. I'm open to suggestions on this front.

[NoQ]

I think this is great. We've been hearing a lot of complaints on the mailing lists recently about that problem.

Did you check that scan-build properly de-duplicates cross-file reports that originate from different clang runs but point to the same header? With your approach i think it should work out of the box, but i'd rather be sure.

It's not as immediately clear this is a multi-file output.

I'm still to have a closer look at the actual code, sorry for the delays.

I'm not having problems with that, i think. If you want to fix that, maybe a list of files at the header, with links to the beginning of each file, would be enough. Or maybe modify arrows between diagnostic pieces to highlight cases when they cross file boundaries: <- 5. Calling foo() -> (into file foo.c).

[zaks.anna]

No multi-file support is a long outstanding limitation of scan-build html output. Great to see the patch!! Thank you for working on it!

It's not as immediately clear this is a multi-file output.

In addition to Artem's suggestions, you might want to insert multiple lines of padding to make the distinction on the border more clear. I think it would help especially when scrolling a large report like in the link for the Linux source.

Also, could you put this behind an option or introduce a new format like -analyzer-output=plist-multi-file but for html? Just in case someone is relying on a single file output format, we'd want to have an option to give them to turn it on.

[vlad.tsyrklevich]

Updated the formatting to make the file split more obvious (padding/line break height) and added simple navigation across files, example here: https://rawgit.com/vlad902/4aaa4e5e7a777b7098337370791352d7/raw/fc88cef667935e1eeae492b15590c18516e645dd/report.html

@NoQ: I had not thought about deduplication. I looked at scan-build and it looks like it currently de-dups based on MD5 matches of the HTML files. The simple case of multi-file reports with the same MD5 (e.g. because of repeated compilations of a single file during a build) being deduplicated still works correctly.

I also looked at what would happen if two separate C files include a common header file that only generates a report with -analyzer-opt-analyze-headers specified (e.g. if the path originates in the header file and doesn’t traverse the main C file at all.) This generated zero reports! My logic assumed the main file would always be included in the path and threw away -analyzer-opt-analyze-headers reports incorrectly. I was able to fix the logic such that these reports are correctly generated and also properly deduplicated by structuring the reports such that the main C file source is not included if it’s not traversed in the path.

@zaks.anna: I’ve added the single file output option but I would like to keep the multi-file option default—I suspect there are very few users parsing the HTML manually when plist is available and I’d like to ensure people don’t miss any results or have to dig into the source to understand they needed something other than the default output setting. What do you think?

[vlad.tsyrklevich]

lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp

211 ↗

(On Diff #90417)

Is there a cleaner way to do these two comparisons?

[zaks.anna]

I’ve added the single file output option but I would like to keep the multi-file option default

This sounds good to me! I agree that this is a very useful addition.

[vlad.tsyrklevich]

Updates to cleanly rebase on a recent clang master. Ran all tests with ASan to ensure I avoided a broken build post-merge as with https://reviews.llvm.org/D30909

[vlad.tsyrklevich]

After reviewing this patch again last night I:

1. Updated some "%clang_cc1 -analyze" calls with "%clang_analyze_cc1" due to 2cfd901321423a96edd8513afc7c7c2bb0d18b2e
2. Isolated HTML generation into a separate function to simplify HTMLDiagnostics::ReportDiag even further

[zaks.anna]

LGTM. Thank you!

(Not sure if @NoQ wants to do a final review.)

Do you have commit access or should we commit on your behalf?

[vlad.tsyrklevich]

Please commit on my behalf.

cfe/trunk/include/clang/StaticAnalyzer/Core/Analyses.def

 
 ANALYSIS_CONSTRAINTS(RangeConstraints, "range", "Use constraint tracking of concrete value ranges", CreateRangeConstraintManager)
 ANALYSIS_CONSTRAINTS(Z3Constraints, "z3", "Use Z3 contraint solver", CreateZ3ConstraintManager)
 
 #ifndef ANALYSIS_DIAGNOSTICS
 #define ANALYSIS_DIAGNOSTICS(NAME, CMDFLAG, DESC, CREATEFN)
 #endif
 
-ANALYSIS_DIAGNOSTICS(HTML,  "html",  "Output analysis results using HTML",   createHTMLDiagnosticConsumer)
+ANALYSIS_DIAGNOSTICS(HTML, "html", "Output analysis results using HTML", createHTMLDiagnosticConsumer)
+ANALYSIS_DIAGNOSTICS(HTML_SINGLE_FILE, "html-single-file", "Output analysis results using HTML (not allowing for multi-file bugs)", createHTMLSingleFileDiagnosticConsumer)
 ANALYSIS_DIAGNOSTICS(PLIST, "plist", "Output analysis results using Plists", createPlistDiagnosticConsumer)
-ANALYSIS_DIAGNOSTICS(PLIST_MULTI_FILE, "plist-multi-file", "Output analysis results using Plists (allowing for mult-file bugs)", createPlistMultiFileDiagnosticConsumer)
+ANALYSIS_DIAGNOSTICS(PLIST_MULTI_FILE, "plist-multi-file", "Output analysis results using Plists (allowing for multi-file bugs)", createPlistMultiFileDiagnosticConsumer)
 ANALYSIS_DIAGNOSTICS(PLIST_HTML, "plist-html", "Output analysis results using HTML wrapped with Plists", createPlistHTMLDiagnosticConsumer)
 ANALYSIS_DIAGNOSTICS(TEXT, "text", "Text output of analysis results", createTextPathDiagnosticConsumer)
 
 #ifndef ANALYSIS_PURGE
 #define ANALYSIS_PURGE(NAME, CMDFLAG, DESC)
 #endif
 
 ANALYSIS_PURGE(PurgeStmt,  "statement", "Purge symbols, bindings, and constraints before every statement")

cfe/trunk/lib/Rewrite/HTMLRewrite.cpp

void html::AddHeaderFooterInternalBuiltinCSS(Rewriter &R, FileID FID,
 
   if (!title.empty())
     os << "<title>" << html::EscapeText(title) << "</title>\n";
 
   os << "<style type=\"text/css\">\n"
       " body { color:#000000; background-color:#ffffff }\n"
       " body { font-family:Helvetica, sans-serif; font-size:10pt }\n"
       " h1 { font-size:14pt }\n"
+      " .FileName { margin-top: 5px; margin-bottom: 5px; display: inline; }\n"
+      " .FileNav { margin-left: 5px; margin-right: 5px; display: inline; }\n"
+      " .FileNav a { text-decoration:none; font-size: larger; }\n"
+      " .divider { margin-top: 30px; margin-bottom: 30px; height: 15px; }\n"
+      " .divider { background-color: gray; }\n"
       " .code { border-collapse:collapse; width:100%; }\n"
       " .code { font-family: \"Monospace\", monospace; font-size:10pt }\n"
       " .code { line-height: 1.2em }\n"
       " .comment { color: green; font-style: oblique }\n"
       " .keyword { color: blue }\n"
       " .string_literal { color: red }\n"
       " .directive { color: darkmagenta }\n"
       // Macro expansions.

cfe/trunk/lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp

 
 namespace {
 
 class HTMLDiagnostics : public PathDiagnosticConsumer {
   std::string Directory;
   bool createdDir, noDir;
   const Preprocessor &PP;
   AnalyzerOptions &AnalyzerOpts;
+  const bool SupportsCrossFileDiagnostics;
 public:
-  HTMLDiagnostics(AnalyzerOptions &AnalyzerOpts, const std::string& prefix, const Preprocessor &pp);
+  HTMLDiagnostics(AnalyzerOptions &AnalyzerOpts,
+                  const std::string& prefix,
+                  const Preprocessor &pp,
+                  bool supportsMultipleFiles);
 
   ~HTMLDiagnostics() override { FlushDiagnostics(nullptr); }
 
   void FlushDiagnosticsImpl(std::vector<const PathDiagnostic *> &Diags,
                             FilesMade *filesMade) override;
 
   StringRef getName() const override {
     return "HTMLDiagnostics";
   }
 
+  bool supportsCrossFileDiagnostics() const override {
+    return SupportsCrossFileDiagnostics;
+  }
+
   unsigned ProcessMacroPiece(raw_ostream &os,
                              const PathDiagnosticMacroPiece& P,
                              unsigned num);
 
   void HandlePiece(Rewriter& R, FileID BugFileID,
                    const PathDiagnosticPiece& P, unsigned num, unsigned max);
 
   void HighlightRange(Rewriter& R, FileID BugFileID, SourceRange Range,
                       const char *HighlightStart = "<span class=\"mrange\">",
                       const char *HighlightEnd = "</span>");
 
   void ReportDiag(const PathDiagnostic& D,
                   FilesMade *filesMade);
+
+  // Generate the full HTML report
+  std::string GenerateHTML(const PathDiagnostic& D, Rewriter &R,
+                           const SourceManager& SMgr, const PathPieces& path,
+                           const char *declName);
+
+  // Add HTML header/footers to file specified by FID
+  void FinalizeHTML(const PathDiagnostic& D, Rewriter &R,
+                    const SourceManager& SMgr, const PathPieces& path,
+                    FileID FID, const FileEntry *Entry, const char *declName);
+
+  // Rewrite the file specified by FID with HTML formatting.
+  void RewriteFile(Rewriter &R, const SourceManager& SMgr,
+                   const PathPieces& path, FileID FID);
 };
 
 } // end anonymous namespace
 
 HTMLDiagnostics::HTMLDiagnostics(AnalyzerOptions &AnalyzerOpts,
                                  const std::string& prefix,
-                                 const Preprocessor &pp)
-    : Directory(prefix), createdDir(false), noDir(false), PP(pp), AnalyzerOpts(AnalyzerOpts) {
-}
+                                 const Preprocessor &pp,
+                                 bool supportsMultipleFiles)
+    : Directory(prefix),
+      createdDir(false),
+      noDir(false),
+      PP(pp),
+      AnalyzerOpts(AnalyzerOpts),
+      SupportsCrossFileDiagnostics(supportsMultipleFiles) {}
 
 void ento::createHTMLDiagnosticConsumer(AnalyzerOptions &AnalyzerOpts,
                                         PathDiagnosticConsumers &C,
                                         const std::string& prefix,
                                         const Preprocessor &PP) {
-  C.push_back(new HTMLDiagnostics(AnalyzerOpts, prefix, PP));
+  C.push_back(new HTMLDiagnostics(AnalyzerOpts, prefix, PP, true));
+}
+
+void ento::createHTMLSingleFileDiagnosticConsumer(AnalyzerOptions &AnalyzerOpts,
+                                                  PathDiagnosticConsumers &C,
+                                                  const std::string& prefix,
+                                                  const Preprocessor &PP) {
+  C.push_back(new HTMLDiagnostics(AnalyzerOpts, prefix, PP, false));
 }
 
 //===----------------------------------------------------------------------===//
 // Report processing.
 //===----------------------------------------------------------------------===//
 
 void HTMLDiagnostics::FlushDiagnosticsImpl(
   std::vector<const PathDiagnostic *> &Diags,
void HTMLDiagnostics::ReportDiag(const PathDiagnostic& D,
   }
 
   if (noDir)
     return;
 
   // First flatten out the entire path to make it easier to use.
   PathPieces path = D.path.flatten(/*ShouldFlattenMacros=*/false);
 
-  // The path as already been prechecked that all parts of the path are
-  // from the same file and that it is non-empty.
-  const SourceManager &SMgr = path.front()->getLocation().getManager();
+  // The path as already been prechecked that the path is non-empty.
   assert(!path.empty());
-  FileID FID =
-    path.front()->getLocation().asLocation().getExpansionLoc().getFileID();
-  assert(FID.isValid());
+  const SourceManager &SMgr = path.front()->getLocation().getManager();
 
   // Create a new rewriter to generate HTML.
   Rewriter R(const_cast<SourceManager&>(SMgr), PP.getLangOpts());
 
+  // The file for the first path element is considered the main report file, it
+  // will usually be equivalent to SMgr.getMainFileID(); however, it might be a
+  // header when -analyzer-opt-analyze-headers is used.
+  FileID ReportFile = path.front()->getLocation().asLocation().getExpansionLoc().getFileID();
+
   // Get the function/method name
   SmallString<128> declName("unknown");
   int offsetDecl = 0;
   if (const Decl *DeclWithIssue = D.getDeclWithIssue()) {
-      if (const NamedDecl *ND = dyn_cast<NamedDecl>(DeclWithIssue)) {
+      if (const NamedDecl *ND = dyn_cast<NamedDecl>(DeclWithIssue))
           declName = ND->getDeclName().getAsString();
-      }
 
       if (const Stmt *Body = DeclWithIssue->getBody()) {
           // Retrieve the relative position of the declaration which will be used
           // for the file name
           FullSourceLoc L(
               SMgr.getExpansionLoc(path.back()->getLocation().asLocation()),
               SMgr);
           FullSourceLoc FunL(SMgr.getExpansionLoc(Body->getLocStart()), SMgr);
           offsetDecl = L.getExpansionLineNumber() - FunL.getExpansionLineNumber();
       }
   }
 
-  // Process the path.
-  // Maintain the counts of extra note pieces separately.
-  unsigned TotalPieces = path.size();
-  unsigned TotalNotePieces =
-      std::count_if(path.begin(), path.end(),
-                    [](const std::shared_ptr<PathDiagnosticPiece> &p) {
-                      return isa<PathDiagnosticNotePiece>(*p);
-                    });
+  std::string report = GenerateHTML(D, R, SMgr, path, declName.c_str());
+  if (report.empty()) {
+    llvm::errs() << "warning: no diagnostics generated for main file.\n";
+    return;
+  }
 
-  unsigned TotalRegularPieces = TotalPieces - TotalNotePieces;
-  unsigned NumRegularPieces = TotalRegularPieces;
-  unsigned NumNotePieces = TotalNotePieces;
+  // Create a path for the target HTML file.
+  int FD;
+  SmallString<128> Model, ResultPath;
+
+  if (!AnalyzerOpts.shouldWriteStableReportFilename()) {
+      llvm::sys::path::append(Model, Directory, "report-%%%%%%.html");
+      if (std::error_code EC =
+          llvm::sys::fs::make_absolute(Model)) {
+          llvm::errs() << "warning: could not make '" << Model
+                       << "' absolute: " << EC.message() << '\n';
+        return;
+      }
+      if (std::error_code EC =
+          llvm::sys::fs::createUniqueFile(Model, FD, ResultPath)) {
+          llvm::errs() << "warning: could not create file in '" << Directory
+                       << "': " << EC.message() << '\n';
+          return;
+      }
 
-  for (auto I = path.rbegin(), E = path.rend(); I != E; ++I) {
-    if (isa<PathDiagnosticNotePiece>(I->get())) {
-      // This adds diagnostic bubbles, but not navigation.
-      // Navigation through note pieces would be added later,
-      // as a separate pass through the piece list.
-      HandlePiece(R, FID, **I, NumNotePieces, TotalNotePieces);
-      --NumNotePieces;
-    } else {
-      HandlePiece(R, FID, **I, NumRegularPieces, TotalRegularPieces);
-      --NumRegularPieces;
+      int i = 1;
+      std::error_code EC;
+      do {
+          // Find a filename which is not already used
+          const FileEntry* Entry = SMgr.getFileEntryForID(ReportFile);
+          std::stringstream filename;
+          Model = "";
+          filename << "report-"
+                   << llvm::sys::path::filename(Entry->getName()).str()
+                   << "-" << declName.c_str()
+                   << "-" << offsetDecl
+                   << "-" << i << ".html";
+          llvm::sys::path::append(Model, Directory,
+                                  filename.str());
+          EC = llvm::sys::fs::openFileForWrite(Model,
+                                               FD,
+                                               llvm::sys::fs::F_RW |
+                                               llvm::sys::fs::F_Excl);
+          if (EC && EC != llvm::errc::file_exists) {
+              llvm::errs() << "warning: could not create file '" << Model
+                           << "': " << EC.message() << '\n';
+              return;
           }
+          i++;
+      } while (EC);
   }
 
-  // Add line numbers, header, footer, etc.
+  llvm::raw_fd_ostream os(FD, true);
 
-  // unsigned FID = R.getSourceMgr().getMainFileID();
-  html::EscapeText(R, FID);
-  html::AddLineNumbers(R, FID);
+  if (filesMade)
+    filesMade->addDiagnostic(D, getName(),
+                             llvm::sys::path::filename(ResultPath));
 
-  // If we have a preprocessor, relex the file and syntax highlight.
-  // We might not have a preprocessor if we come from a deserialized AST file,
-  // for example.
+  // Emit the HTML to disk.
+  os << report;
+}
 
-  html::SyntaxHighlight(R, FID, PP);
-  html::HighlightMacros(R, FID, PP);
+std::string HTMLDiagnostics::GenerateHTML(const PathDiagnostic& D, Rewriter &R,
+    const SourceManager& SMgr, const PathPieces& path, const char *declName) {
+
+  // Rewrite source files as HTML for every new file the path crosses
+  std::vector<FileID> FileIDs;
+  for (auto I : path) {
+    FileID FID = I->getLocation().asLocation().getExpansionLoc().getFileID();
+    if (std::find(FileIDs.begin(), FileIDs.end(), FID) != FileIDs.end())
+      continue;
+
+    FileIDs.push_back(FID);
+    RewriteFile(R, SMgr, path, FID);
+  }
+
+  if (SupportsCrossFileDiagnostics && FileIDs.size() > 1) {
+    // Prefix file names, anchor tags, and nav cursors to every file
+    for (auto I = FileIDs.begin(), E = FileIDs.end(); I != E; I++) {
+      std::string s;
+      llvm::raw_string_ostream os(s);
 
-  // Get the full directory name of the analyzed file.
+      if (I != FileIDs.begin())
+        os << "<hr class=divider>\n";
 
-  const FileEntry* Entry = SMgr.getFileEntryForID(FID);
+      os << "<div id=File" << I->getHashValue() << ">\n";
 
+      // Left nav arrow
+      if (I != FileIDs.begin())
+        os << "<div class=FileNav><a href=\"#File" << (I - 1)->getHashValue()
+           << "\">&#x2190;</a></div>";
+
+      os << "<h4 class=FileName>" << SMgr.getFileEntryForID(*I)->getName()
+         << "</h4>\n";
+
+      // Right nav arrow
+      if (I + 1 != E)
+        os << "<div class=FileNav><a href=\"#File" << (I + 1)->getHashValue()
+           << "\">&#x2192;</a></div>";
+
+      os << "</div>\n";
+
+      R.InsertTextBefore(SMgr.getLocForStartOfFile(*I), os.str());
+    }
+
+    // Append files to the main report file in the order they appear in the path
+    for (auto I : llvm::make_range(FileIDs.begin() + 1, FileIDs.end())) {
+      std::string s;
+      llvm::raw_string_ostream os(s);
+
+      const RewriteBuffer *Buf = R.getRewriteBufferFor(I);
+      for (auto BI : *Buf)
+        os << BI;
+
+      R.InsertTextAfter(SMgr.getLocForEndOfFile(FileIDs[0]), os.str());
+    }
+  }
+
+  const RewriteBuffer *Buf = R.getRewriteBufferFor(FileIDs[0]);
+  if (!Buf)
+    return "";
+
+  // Add CSS, header, and footer.
+  const FileEntry* Entry = SMgr.getFileEntryForID(FileIDs[0]);
+  FinalizeHTML(D, R, SMgr, path, FileIDs[0], Entry, declName);
+
+  std::string file;
+  llvm::raw_string_ostream os(file);
+  for (auto BI : *Buf)
+    os << BI;
+
+  return os.str();
+}
+
+void HTMLDiagnostics::FinalizeHTML(const PathDiagnostic& D, Rewriter &R,
+    const SourceManager& SMgr, const PathPieces& path, FileID FID,
+    const FileEntry *Entry, const char *declName) {
   // This is a cludge; basically we want to append either the full
   // working directory if we have no directory information.  This is
   // a work in progress.
 
   llvm::SmallString<0> DirName;
 
   if (llvm::sys::path::is_relative(Entry->getName())) {
     llvm::sys::fs::current_path(DirName);
// Embed meta-data tags.
 
     // Mark the end of the tags.
     os << "\n<!-- BUGMETAEND -->\n";
 
     // Insert the text.
     R.InsertTextBefore(SMgr.getLocForStartOfFile(FID), os.str());
   }
 
-  // Add CSS, header, and footer.
-
   html::AddHeaderFooterInternalBuiltinCSS(R, FID, Entry->getName());
-
-  // Get the rewrite buffer.
-  const RewriteBuffer *Buf = R.getRewriteBufferFor(FID);
-
-  if (!Buf) {
-    llvm::errs() << "warning: no diagnostics generated for main file.\n";
-    return;
-  }
 
-  // Create a path for the target HTML file.
-  int FD;
-  SmallString<128> Model, ResultPath;
+void HTMLDiagnostics::RewriteFile(Rewriter &R, const SourceManager& SMgr,
+    const PathPieces& path, FileID FID) {
+  // Process the path.
+  // Maintain the counts of extra note pieces separately.
+  unsigned TotalPieces = path.size();
+  unsigned TotalNotePieces =
+      std::count_if(path.begin(), path.end(),
+                    [](const std::shared_ptr<PathDiagnosticPiece> &p) {
+                      return isa<PathDiagnosticNotePiece>(*p);
+                    });
 
-  if (!AnalyzerOpts.shouldWriteStableReportFilename()) {
-      llvm::sys::path::append(Model, Directory, "report-%%%%%%.html");
-      if (std::error_code EC =
-          llvm::sys::fs::make_absolute(Model)) {
-          llvm::errs() << "warning: could not make '" << Model
-                       << "' absolute: " << EC.message() << '\n';
-        return;
-      }
-      if (std::error_code EC =
-          llvm::sys::fs::createUniqueFile(Model, FD, ResultPath)) {
-          llvm::errs() << "warning: could not create file in '" << Directory
-                       << "': " << EC.message() << '\n';
-          return;
-      }
+  unsigned TotalRegularPieces = TotalPieces - TotalNotePieces;
+  unsigned NumRegularPieces = TotalRegularPieces;
+  unsigned NumNotePieces = TotalNotePieces;
 
+  for (auto I = path.rbegin(), E = path.rend(); I != E; ++I) {
+    if (isa<PathDiagnosticNotePiece>(I->get())) {
+      // This adds diagnostic bubbles, but not navigation.
+      // Navigation through note pieces would be added later,
+      // as a separate pass through the piece list.
+      HandlePiece(R, FID, **I, NumNotePieces, TotalNotePieces);
+      --NumNotePieces;
     } else {
-      int i = 1;
-      std::error_code EC;
-      do {
-          // Find a filename which is not already used
-          std::stringstream filename;
-          Model = "";
-          filename << "report-"
-                   << llvm::sys::path::filename(Entry->getName()).str()
-                   << "-" << declName.c_str()
-                   << "-" << offsetDecl
-                   << "-" << i << ".html";
-          llvm::sys::path::append(Model, Directory,
-                                  filename.str());
-          EC = llvm::sys::fs::openFileForWrite(Model,
-                                               FD,
-                                               llvm::sys::fs::F_RW |
-                                               llvm::sys::fs::F_Excl);
-          if (EC && EC != llvm::errc::file_exists) {
-              llvm::errs() << "warning: could not create file '" << Model
-                           << "': " << EC.message() << '\n';
-              return;
-          }
-          i++;
-      } while (EC);
+      HandlePiece(R, FID, **I, NumRegularPieces, TotalRegularPieces);
+      --NumRegularPieces;
   }
 
-  llvm::raw_fd_ostream os(FD, true);
+  // Add line numbers, header, footer, etc.
 
-  if (filesMade)
-    filesMade->addDiagnostic(D, getName(),
-                             llvm::sys::path::filename(ResultPath));
+  html::EscapeText(R, FID);
+  html::AddLineNumbers(R, FID);
 
-  // Emit the HTML to disk.
-  for (RewriteBuffer::iterator I = Buf->begin(), E = Buf->end(); I!=E; ++I)
-      os << *I;
+  // If we have a preprocessor, relex the file and syntax highlight.
+  // We might not have a preprocessor if we come from a deserialized AST file,
+  // for example.
+
+  html::SyntaxHighlight(R, FID, PP);
+  html::HighlightMacros(R, FID, PP);
 }
 
 void HTMLDiagnostics::HandlePiece(Rewriter& R, FileID BugFileID,
                                   const PathDiagnosticPiece& P,
                                   unsigned num, unsigned max) {
 
   // For now, just draw a box above the line in question, and emit the
   // warning.

cfe/trunk/test/Analysis/diagnostics/diag-cross-file-boundaries.h

-static void f() {
-  int *p = 0;
-  *p = 1;       // expected-warning{{Dereference of null pointer}}
-}

cfe/trunk/test/Analysis/diagnostics/diag-cross-file-boundaries.c

-// RUN: %clang_analyze_cc1 -analyzer-checker=core -verify %s
-// RUN: %clang_analyze_cc1 -analyzer-checker=core -analyzer-output=html -o PR12421.html %s 2>&1 | FileCheck %s
-
-// Test for PR12421
-#include "diag-cross-file-boundaries.h"
-
-int main(){
-  f();
-  return 0;
-}
-
-// CHECK: warning: Path diagnostic report is not generated.

cfe/trunk/test/Analysis/html-diag-singlefile.h

+static void f() {
+  int *p = 0;
+  *p = 1;       // expected-warning{{Dereference of null pointer}}
+}

cfe/trunk/test/Analysis/html-diag-singlefile.c

+// RUN: %clang_analyze_cc1 -analyzer-checker=core -verify %s
+// RUN: %clang_analyze_cc1 -analyzer-checker=core -analyzer-output=html-single-file -o D30406.html %s 2>&1 | FileCheck %s
+
+// Check that single file HTML output does not process multi-file diagnostics.
+// (This used to test for PR12421, before the introduction of the html-single-file format)
+
+#include "html-diag-singlefile.h"
+
+int main(){
+  f();
+  return 0;
+}
+
+// CHECK: warning: Path diagnostic report is not generated.

cfe/trunk/test/Analysis/html-diags-analyze-headers.h

+#include "html-diags-multifile.h"
+
+void test_call_macro() {
+  has_bug(0);
+}

cfe/trunk/test/Analysis/html-diags-analyze-headers.c

+// RUN: mkdir -p %t.dir
+// RUN: %clang_analyze_cc1 -analyzer-opt-analyze-headers -analyzer-output=html -analyzer-checker=core -o %t.dir %s
+// RUN: ls %t.dir | grep report
+// RUN: rm -rf %t.dir
+
+// This tests that we emit HTML diagnostics for reports in headers when the
+// analyzer is run with -analyzer-opt-analyze-headers. This was handled
+// incorrectly in the first iteration of D30406.
+
+#include "html-diags-analyze-headers.h"

cfe/trunk/test/Analysis/html-diags-multifile.c

 // RUN: mkdir -p %t.dir
 // RUN: %clang_analyze_cc1 -analyzer-output=html -analyzer-checker=core -o %t.dir %s
-// RUN: ls %t.dir | not grep report
+// RUN: ls %t.dir | grep report
 // RUN: rm -fR %t.dir
 
-// This tests that we do not currently emit HTML diagnostics for reports that
-// cross file boundaries.
+// This tests that we emit HTML diagnostics for reports that cross file boundaries.
 
 #include "html-diags-multifile.h"
 
 #define CALL_HAS_BUG(q) has_bug(q)
 
 void test_call_macro() {
   CALL_HAS_BUG(0);
 }
 

cfe/trunk/test/Analysis/html-diags.c

 // RUN: rm -fR %T/dir
 // RUN: mkdir %T/dir
 // RUN: %clang_analyze_cc1 -analyzer-output=html -analyzer-checker=core -o %T/dir %s
 // RUN: ls %T/dir | grep report
 
+// D30406: Test new html-single-file output
+// RUN: rm -fR %T/dir
+// RUN: mkdir %T/dir
+// RUN: %clang_analyze_cc1 -analyzer-output=html-single-file -analyzer-checker=core -o %T/dir %s
+// RUN: ls %T/dir | grep report
+
 // PR16547: Test relative paths
 // RUN: cd %T/dir
 // RUN: %clang_analyze_cc1 -analyzer-output=html -analyzer-checker=core -o testrelative %s
 // RUN: ls %T/dir/testrelative | grep report
 
 // Currently this test mainly checks that the HTML diagnostics doesn't crash
 // when handling macros will calls with macros.  We should actually validate
 // the output, but that requires being able to match against a specifically

cfe/trunk/test/Coverage/html-diagnostics.c

 // RUN: rm -rf %t
 // RUN: %clang_cc1 -analyze -analyzer-output=html -analyzer-checker=core -o %t %s
 // RUN: find %t -name "*.html" -exec cat "{}" ";" | FileCheck %s
+//
+// RUN: rm -rf %t
+// RUN: %clang_cc1 -analyze -analyzer-output=html-single-file -analyzer-checker=core -o %t %s
+// RUN: find %t -name "*.html" -exec cat "{}" ";" | FileCheck %s
 
 // REQUIRES: staticanalyzer
 
 // CHECK: <h3>Annotated Source Code</h3>
 
+// Make sure it's not generated as a multi-file HTML output
+// CHECK-NOT: <h4 class=FileName>{{.*}}
+
 // Without tweaking expr, the expr would hit to the line below
 // emitted to the output as comment.
 // CHECK: {{[D]ereference of null pointer}}
 
 void f0(int x) {
   int *p = &x;
 
   if (x > 10) {
     if (x == 22)
       p = 0;
   }
 
   *p = 10;
 }
 
 

cfe/trunk/test/Coverage/html-multifile-diagnostics.h

+void f1(int *ptr) {
+  *ptr = 0;
+}

cfe/trunk/test/Coverage/html-multifile-diagnostics.c

+// RUN: rm -rf %t
+// RUN: %clang_cc1 -analyze -analyzer-output=html -analyzer-checker=core -o %t %s
+// RUN: find %t -name "*.html" -exec cat "{}" ";" | FileCheck %s
+
+// REQUIRES: staticanalyzer
+
+// CHECK: <h3>Annotated Source Code</h3>
+
+// Make sure it's generated as multi-file HTML output
+// CHECK: <h4 class=FileName>{{.*}}html-multifile-diagnostics.c</h4>
+// CHECK: <h4 class=FileName>{{.*}}html-multifile-diagnostics.h</h4>
+
+// Without tweaking expr, the expr would hit to the line below
+// emitted to the output as comment.
+// CHECK: {{[D]ereference of null pointer}}
+
+#include "html-multifile-diagnostics.h"
+
+void f0() {
+  f1((int*)0);
+}

cfe/trunk/www/analyzer/open_projects.html

<ul>
     analysis. 
     <i>(Difficulty: Hard)</i></p>
     </li>    
   </ul>
   </li>
 
   <li>Bug Reporting 
   <ul>
-    <li>Add support for displaying cross-file diagnostic paths in HTML output
-    (used by <tt>scan-build</tt>).
-    <p>Currently <tt>scan-build</tt> output does not display reports that span 
-    multiple files. The main problem is that we do not have a good format to
-    display such paths in HTML output. <i>(Difficulty: Medium)</i> </p>
-    </li>
-    
     <li>Refactor path diagnostic generation in <a href="http://clang.llvm.org/doxygen/BugReporter_8cpp_source.html">BugReporter.cpp</a>.
     <p>It would be great to have more code reuse between "Minimal" and 
     "Extensive" PathDiagnostic generation algorithms. One idea is to create an 
     IR for representing path diagnostics, which would be later be used to 
     generate minimal or extensive report output. <i>(Difficulty: Medium)</i></p>
     </li>
   </ul>
   </li>