[analyzer] Show full analyzer invocation for reproducibility in HTML reports

[george.karpenkov]

Summary

Analyzing problems which appear in scan-build results can be very difficult, as after the launch no exact invocation is stored, and it's super-hard to launch the debugger.
With this patch, the exact analyzer invocation appears in the header hidden under the spoiler CSS tag, and can be copied to debug/check reproducibility/etc.

rdar://35980230

[a.sidorin]

Hello George,

Maybe it is better to put this information into header? The most important warning information is shown in the header.

lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp

480

Can we use SmallString instead?

[george.karpenkov]

Maybe it is better to put this information into header? The most important warning information is shown in the header.

The problem is on large projects the invocation is huge, taking easily one third of the screen.
We might even move into another direction, hiding it further with the HTML comment.
The information is not really relevant to the end user, and analyzer developers would be able to grab it from the source.

lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp

480

I guess; previous construction at line 425 uses std::string

[NoQ]

I guess at the top < at the bottom < at the top under a CSS spoiler (collapsed by default).

[george.karpenkov]

Now with spoiler-tag approach

[NoQ]

Yay!

[george.karpenkov]

Committed in git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@323245 91177308-0d34-0410-b5e6-96231b3b80d8

[dcoughlin]

Remember that html reports are not just used by analyzer developers but also by end users. For end users, the clang invocation is an implementation detail of the analyzer and not part of its user model. We shouldn't display this to users.

You should either hide this feature behind a flag (so users won't see it but analyzer developers can turn it on) or put this information into an HTML comment (so users won't see it but analyzer developers can look at the HTML source to determine the invocation).

[george.karpenkov]

@dcoughlin I think having it beside the flag would defeat the whole purpose, as you would like to try to reproduce the run by getting a report from unsuspecting user.
So you think even the spoiler tag is unacceptable? (preview below)

I think that hiding in comments would hurt discoverability, and given the usefulness of the feature I think it justifies having an extra irrelevant line for the user.
(and also for the power user the compiler arguments might even be useful).
It's also nicer to copy-n-paste without having to hunt for the line inside the source view.
But of course if you insist I could hide it in the comment.

include/clang/StaticAnalyzer/Core/AnalyzerOptions.h

public:
   /// \brief A key-value table of use-specified configuration values.
   ConfigTable Config;
   AnalysisStores AnalysisStoreOpt;
   AnalysisConstraints AnalysisConstraintsOpt;
   AnalysisDiagClients AnalysisDiagOpt;
   AnalysisPurgeMode AnalysisPurgeOpt;
   
   std::string AnalyzeSpecificFunction;
-  
+  /// Store full compiler invocation for reproducible instructions in the
+  /// generated report.
+  std::string FullCompilerInvocation;
+  
   /// \brief The maximum number of times the analyzer visits a block.
   unsigned maxBlockVisitOnPath;
   
   
   /// \brief Disable all analyzer checks.
   ///
   /// This flag allows one to disable analyzer checks on the code processed by
   /// the given analysis consumer. Note, the code will get parsed and the

lib/Frontend/CompilerInvocation.cpp

for (unsigned i = 0, e = configVals.size(); i != e; ++i) {
           << configVals[i];
         Success = false;
         break;
       }
       Opts.Config[key] = val;
     }
   }
 
+  llvm::raw_string_ostream os(Opts.FullCompilerInvocation);
+  for (unsigned i=0; i<Args.getNumInputArgStrings(); i++) {
+    if (i != 0)
+      os << " ";
+    os << Args.getArgString(i);
+  }
+  os.flush();
+
   return Success;
 }
 
 static bool ParseMigratorArgs(MigratorOptions &Opts, ArgList &Args) {
   Opts.NoNSAllocReallocError = Args.hasArg(OPT_migrator_no_nsalloc_error);
   Opts.NoFinalizeRemoval = Args.hasArg(OPT_migrator_no_finalize_removal);
   return true;
 }

lib/Rewrite/HTMLRewrite.cpp

 }
 td.rowname {
   text-align: right;
   vertical-align: top;
   font-weight: bold;
   color:#444444;
   padding-right:2ex;
 }
+
+/* Hidden text. */
+input.spoilerhider + label {
+  cursor: pointer;
+  text-decoration: underline;
+  display: block;
+}
+input.spoilerhider {
+ display: none;
+}
+input.spoilerhider ~ .spoiler {
+  overflow: hidden;
+  margin: 10px auto 0;
+  height: 0;
+  opacity: 0;
+}
+input.spoilerhider:checked + label + .spoiler{
+  height: auto;
+  opacity: 1;
+}
 </style>
 </head>
 <body>)<<<";
 
   // Generate header
   R.InsertTextBefore(StartLoc, os.str());
   // Generate footer
 

lib/StaticAnalyzer/Core/HTMLDiagnostics.cpp

// Add the name of the file as an <h1> tag.
     }
 
     os << R"<<<(
 </table>
 <!-- REPORTSUMMARYEXTRA -->
 <h3>Annotated Source Code</h3>
 <p>Press <a href="#" onclick="toggleHelp(); return false;">'?'</a>
    to see keyboard shortcuts</p>
+<input type="checkbox" class="spoilerhider" id="showinvocation" />
+<label for="showinvocation" >Show analyzer invocation</label>
+<div class="spoiler">clang -cc1 )<<<";
+    os << html::EscapeText(AnalyzerOpts.FullCompilerInvocation);
+    os << R"<<<(
+</div>
 <div id='tooltiphint' hidden="true">
   <p>Keyboard shortcuts: </p>
   <ul>
     <li>Use 'j/k' keys for keyboard navigation</li>
     <li>Use 'Shift+S' to show/hide relevant lines</li>
     <li>Use '?' to toggle this window</li>
   </ul>
   <a href="#" onclick="toggleHelp(); return false;">Close</a>
 </div>
 )<<<";
-
     R.InsertTextBefore(SMgr.getLocForStartOfFile(FID), os.str());
   }
 
   // Embed meta-data tags.
   {
     std::string s;
     llvm::raw_string_ostream os(s);
 
// Embed meta-data tags.
     // Mark the end of the tags.
     os << "\n<!-- BUGMETAEND -->\n";
 
     // Insert the text.
     R.InsertTextBefore(SMgr.getLocForStartOfFile(FID), os.str());
   }
 
   html::AddHeaderFooterInternalBuiltinCSS(R, FID, Entry->getName());
 }
 
 std::string HTMLDiagnostics::showHelpJavascript() {
   return R"<<<(
 <script type='text/javascript'>
 
 var toggleHelp = function() {
     var hint = document.querySelector("#tooltiphint");
     var attributeName = "hidden";