bugpoint: Prototype disabling symbolication of bugpoint-executed programs

[dblaikie]

Summary

This shows the basic idea, using command line arguments. Further work
would be done to plumb through the argument to other tools (llc, lli).

This wouldn't handle things like external scripts (bugpoint's
-compile-command feature) which would need to explicitly opt-in by
passing the argument to any LLVM tools they invoke to avoid the
performance penalty of symbolication.

Alternatively (as suggested by chandlerc@) an environment variable would
be used. This would allow the option to pass transparently through user
scripts, pass to compilers if they happened to be LLVM-ish, etc.

I worry a bit about using cl::opt in the crash handling code - LLVM
might crash early, perhaps before the cl::opt is properly initialized?
Or at least before arguments have been parsed?

I shyed away from doing this with an environment variable when I
realized that would require copying the existing environment and
appending the env variable of interest. But it seems there's no existing
LLVM API for accessing the environment (even the Support tests for
process launching have their own ifdefs for getting the environment). It
could be added, but seemed like a higher bar/untested codepath to
actually add environment variables.

Anyway - so looking for folks thoughts/preferences/etc. This change as
is does fix the original issue I came across, with this change
test/BugPoint/metadata.ll goes from 1m34s to 6.5s (& I think it hits all
the other bugpoint tests too - so I guess they don't have loads of
coverage on the different programs that can be executed, etc)

[• dberlin]

I think this is the right thing to do given what we've seen. Nobody has come up with a use case where they current crawl backtraces in their scripts, etc.
If it becomes common, we can always revisit.

[chandlerc]

I'm pretty happy to start simple with a flag and go from there.

lib/Support/Signals.cpp

39–42

Especially because this is initialized to false initially, I think it'll be fine even if we crash in weird places.

tools/bugpoint/ToolRunner.cpp

66–67

I assume this was just an experiment?

85–88

same...

lib/Support/Signals.cpp

 #include "llvm/Support/Format.h"
 #include "llvm/Support/ManagedStatic.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Mutex.h"
 #include "llvm/Support/Program.h"
 #include "llvm/Support/Signals.h"
 #include "llvm/Support/StringSaver.h"
 #include "llvm/Support/raw_ostream.h"
+#include "llvm/Support/Options.h"
 #include <vector>
 
-namespace llvm {
-
 //===----------------------------------------------------------------------===//
 //=== WARNING: Implementation here must contain only TRULY operating system
 //===          independent code.
 //===----------------------------------------------------------------------===//
 
+using namespace llvm;
+
+static cl::opt<bool>
+    DisableSymbolication("disable-symbolication",
+                         cl::desc("Disable symbolizing crash backtraces."),
+                         cl::init(false), cl::Hidden);
+
 static ManagedStatic<std::vector<std::pair<void (*)(void *), void *>>>
     CallBacksToRun;
 void sys::RunSignalHandlers() {
   if (!CallBacksToRun.isConstructed())
     return;
   for (auto &I : *CallBacksToRun)
     I.first(I.second);
   CallBacksToRun->clear();
 }
-}
-
-using namespace llvm;
 
 static bool findModulesAndOffsets(void **StackTrace, int Depth,
                                   const char **Modules, intptr_t *Offsets,
                                   const char *MainExecutableName,
                                   StringSaver &StrPool);
 
 /// Format a pointer value as hexadecimal. Zero pad it out so its always the
 /// same width.
 static FormattedNumber format_ptr(void *PC) {
   // Each byte is two hex digits plus 2 for the 0x prefix.
   unsigned PtrWidth = 2 + 2 * sizeof(void *);
   return format_hex((uint64_t)PC, PtrWidth);
 }
 
 static bool printSymbolizedStackTrace(StringRef Argv0,
                                       void **StackTrace, int Depth,
                                       llvm::raw_ostream &OS)
   LLVM_ATTRIBUTE_USED;
 
 /// Helper that launches llvm-symbolizer and symbolizes a backtrace.
 static bool printSymbolizedStackTrace(StringRef Argv0,
                                       void **StackTrace, int Depth,
                                       llvm::raw_ostream &OS) {
+  if (DisableSymbolication)
+    return false;
+
   // Don't recursively invoke the llvm-symbolizer binary.
   if (Argv0.find("llvm-symbolizer") != std::string::npos)
     return false;
 
   // FIXME: Subtract necessary number from StackTrace entries to turn return addresses
   // into actual instruction addresses.
   // Use llvm-symbolizer tool to symbolize the stack traces. First look for it
   // alongside our binary, then in $PATH.

tools/bugpoint/OptimizerDriver.cpp

bool BugDriver::runPasses(Module *Program,
   if (UseValgrind) {
     Args.push_back("valgrind");
     Args.push_back("--error-exitcode=1");
     Args.push_back("-q");
     Args.push_back(tool.c_str());
   } else
     Args.push_back(tool.c_str());
 
+  Args.push_back("-disable-symbolication");
   Args.push_back("-o");
   Args.push_back(OutputFilename.c_str());
   for (unsigned i = 0, e = OptArgs.size(); i != e; ++i)
     Args.push_back(OptArgs[i].c_str());
   std::vector<std::string> pass_args;
   for (unsigned i = 0, e = PluginLoader::getNumPlugins(); i != e; ++i) {
     pass_args.push_back(std::string("-load"));
     pass_args.push_back(PluginLoader::getPlugin(i));

tools/bugpoint/ToolRunner.h

 #include "llvm/Support/Path.h"
 #include "llvm/Support/SystemUtils.h"
 #include <exception>
 #include <vector>
 
 namespace llvm {
 
 extern cl::opt<bool> SaveTemps;
+extern cl::opt<bool> EnableSymbolizer;
 extern Triple TargetTriple;
 
 class LLC;
 
 //===---------------------------------------------------------------------===//
 // CC abstraction
 //
 class CC {

tools/bugpoint/ToolRunner.cpp

 #include <utility>
 using namespace llvm;
 
 #define DEBUG_TYPE "toolrunner"
 
 namespace llvm {
 cl::opt<bool> SaveTemps("save-temps", cl::init(false),
                         cl::desc("Save temporary files"));
+cl::opt<bool> EnableSymbolizer(
+    "enable-symbolizer", cl::init(false),
+    cl::desc(
+        "Enable symbolized backtraces on crashes of processes under test"));
 }
 
 namespace {
 cl::opt<std::string>
     RemoteClient("remote-client",
                  cl::desc("Remote execution client (rsh/ssh)"));
 
 cl::opt<std::string> RemoteHost("remote-host",

 /// to the sys::Program::ExecuteAndWait interface.
 /// @see sys::Program::ExecuteAndWait
 static int RunProgramWithTimeout(StringRef ProgramPath, const char **Args,
                                  StringRef StdInFile, StringRef StdOutFile,
                                  StringRef StdErrFile, unsigned NumSeconds = 0,
                                  unsigned MemoryLimit = 0,
                                  std::string *ErrMsg = nullptr) {
   const StringRef *Redirects[3] = {&StdInFile, &StdOutFile, &StdErrFile};
-  return sys::ExecuteAndWait(ProgramPath, Args, nullptr, Redirects, NumSeconds,
+  std::vector<const char*> Env = {"LLVM_DISABLE_SYMBOLICATION"};
+  return sys::ExecuteAndWait(ProgramPath, Args, Env.data(), Redirects, NumSeconds,
                              MemoryLimit, ErrMsg);
 }
 
 /// RunProgramRemotelyWithTimeout - This function runs the given program
 /// remotely using the given remote client and the sys::Program::ExecuteAndWait.
 /// Returns the remote program exit code or reports a remote client error if it
 /// fails. Remote client is required to return 255 if it failed or program exit
 /// code otherwise.
 /// @see sys::Program::ExecuteAndWait
 static int RunProgramRemotelyWithTimeout(StringRef RemoteClientPath,
                                          const char **Args, StringRef StdInFile,
                                          StringRef StdOutFile,
                                          StringRef StdErrFile,
                                          unsigned NumSeconds = 0,
                                          unsigned MemoryLimit = 0) {
   const StringRef *Redirects[3] = {&StdInFile, &StdOutFile, &StdErrFile};
 
+  std::vector<const char*> Env = {"LLVM_DISABLE_SYMBOLICATION"};
+
   // Run the program remotely with the remote client
-  int ReturnCode = sys::ExecuteAndWait(RemoteClientPath, Args, nullptr,
+  int ReturnCode = sys::ExecuteAndWait(RemoteClientPath, Args, Env.data(),
                                        Redirects, NumSeconds, MemoryLimit);
 
   // Has the remote client fail?
   if (255 == ReturnCode) {
     std::ostringstream OS;
     OS << "\nError running remote client:\n ";
     for (const char **Arg = Args; *Arg; ++Arg)
       OS << " " << *Arg;