Suggest lldb-dotest to reproduce a failure.

[JDevlieghere]

Summary

Rather than trying to figure out what arguments to pass to dotest.py, suggest using lldb-dotest. Athough this obviously won't work for all invocations (i.e. when you're passing custom arguments to lldb-dotest) it will work in the most common case (which is unfortunately not true today).

[davide]

Can you commit the whitespace fixes separately?

[aprantl]

packages/Python/lldbsuite/test/dosep.py

122

What do you think about adding the full path to lldb-dotest? I usually have at least three concurrent checkouts of lldb at any time.

[JDevlieghere]

In D47062#1104558, @davide wrote:

Can you commit the whitespace fixes separately?

Sure.

packages/Python/lldbsuite/test/dosep.py

122

Me too, but how would we get that information here? On of lldb-dotest's goals is that you can execute it from anywhere, so you'd either have to pass it explicitly to dotest.py (either using a custom argument and cmake variable combo, or by assuming it lives in the same directory as the current lldb and appending -dotest) but that feels like it might not be worth it. Maybe there's another solution you had in mind?

[aprantl]

packages/Python/lldbsuite/test/dosep.py

122

The --executable option that dotest receives should point to an lldb that ought to have a matching lldb-dotest next to it. I don't think that that is particularly ugly.

[JDevlieghere]

• Use absolute path to lldb-dotest when --executable is passed.

[labath]

Looks fine to me, just make the path computation windows-compatible.

packages/Python/lldbsuite/test/dotest.py

386

On windows, this will end up being lldb.exe-dotest

[JDevlieghere]

Make this work for Windows

[JDevlieghere]

I'm going to hold off on this until we have decided what to do for D46005. If we run the test cases as separate lit invocations, then the test format is in a better position to report this information.

[labath]

In D47062#1109399, @JDevlieghere wrote:

I'm going to hold off on this until we have decided what to do for D46005. If we run the test cases as separate lit invocations, then the test format is in a better position to report this information.

It looks like that patch has gotten a bit stuck, and I'm not sure when/if I'll have the time to unstuck it. So, if you still want to go ahead with this approach, then feel free to commit it.

[JDevlieghere]

Abandoning this until we can replace the dotest driver with lit.

packages/Python/lldbsuite/test/dosep.py

with output_lock:
         if not (RESULTS_FORMATTER and RESULTS_FORMATTER.is_using_terminal()):
             print(file=sys.stderr)
             print(output, file=sys.stderr)
             if timeout:
                 timeout_str = " (TIMEOUT)"
             else:
                 timeout_str = ""
             print("[%s FAILED]%s" % (name, timeout_str), file=sys.stderr)
-            print("Command invoked: %s" % ' '.join(command), file=sys.stderr)
+            print("Reproduce with: lldb-dotest -f {}".format(name), file=sys.stderr)
         update_progress(name)
 
 
 def report_test_pass(name, output):
     global output_lock
     with output_lock:
         update_progress(name)
 

packages/Python/lldbsuite/test/dotest.py

def parseOptionsAndInitTestdirs():
 
     if args.framework:
         configuration.lldbFrameworkPath = args.framework
 
     if args.executable:
         # lldb executable is passed explicitly
         lldbtest_config.lldbExec = os.path.realpath(args.executable)
         if not is_exe(lldbtest_config.lldbExec):
             lldbtest_config.lldbExec = which(args.executable)
         if not is_exe(lldbtest_config.lldbExec):
             logging.error(
                     '%s is not a valid executable to test; aborting...',
                     args.executable)
             sys.exit(-1)
 
     if args.server:
         os.environ['LLDB_DEBUGSERVER_PATH'] = args.server
if ("GDB_REMOTE_LOG" in os.environ):
         ci.HandleCommand(
             "log enable -n -f " + os.environ["GDB_REMOTE_LOG"] + " gdb-remote "
             + gdb_remote_log_option,
             res)
         if not res.Succeeded():
             raise Exception(
                 'log enable failed (check GDB_REMOTE_LOG env variable)')
 
-
-def getMyCommandLine():
-    return ' '.join(sys.argv)
-
 # ======================================== #
 #                                          #
 # Execution of the test driver starts here #
 #                                          #
 # ======================================== #
 
 
 def checkDsymForUUIDIsNotOn():
if not configuration.sdir_name:
         configuration.sdir_name = timestamp_started
     os.environ["LLDB_SESSION_DIRNAME"] = os.path.join(
         os.getcwd(), configuration.sdir_name)
 
     sys.stderr.write(
         "\nSession logs for test failures/errors/unexpected successes"
         " will go into directory '%s'\n" %
         configuration.sdir_name)
-    sys.stderr.write("Command invoked: %s\n" % getMyCommandLine())
+    sys.stderr.write("Command invoked: %s\n" % ' '.join(sys.argv))
 
     if not os.path.isdir(configuration.sdir_name):
         try:
             os.mkdir(configuration.sdir_name)
         except OSError as exception:
             if exception.errno != errno.EEXIST:
                 raise
 

packages/Python/lldbsuite/test/lldbtest.py

class Base(unittest2.TestCase):
 
     def getBuildDirBasename(self):
         return self.__class__.__module__ + "." + self.testMethodName
 
     def getBuildDir(self):
         """Return the full path to the current test."""
         return os.path.join(os.environ["LLDB_BUILD"], self.mydir,
                             self.getBuildDirBasename())
-    
-     
     def makeBuildDir(self):
         """Create the test-specific working directory, deleting any previous
         contents."""
         # See also dotest.py which sets up ${LLDB_BUILD}.
         bdir = self.getBuildDir()
         if os.path.isdir(bdir):
             shutil.rmtree(bdir)
         lldbutil.mkdir_p(bdir)
- 
     def getBuildArtifact(self, name="a.out"):
         """Return absolute path to an artifact in the test's build directory."""
         return os.path.join(self.getBuildDir(), name)
- 
     def getSourcePath(self, name):
         """Return absolute path to a file in the test's source directory."""
         return os.path.join(self.getSourceDir(), name)
 
     def setUp(self):
         """Fixture for unittest test case setup.
 
         It works with the test driver to conditionally skip tests and does other
def markUnexpectedSuccess(self, bugnumber):
             if bugnumber is None:
                 print("unexpected success", file=sbuf)
             else:
                 print(
                     "unexpected success (problem id:" + str(bugnumber) + ")",
                     file=sbuf)
 
     def getRerunArgs(self):
-        return " -f %s.%s" % (self.__class__.__name__, self._testMethodName)
+        return "-f %s.%s" % (self.__class__.__name__, self._testMethodName)
 
     def getLogBasenameForCurrentTest(self, prefix=None):
         """
         returns a partial path that can be used as the beginning of the name of multiple
         log files pertaining to this test
 
         <session-dir>/<arch>-<compiler>-<test-file>.<test-class>.<test-method>
         """
def dumpSessionInfo(self):
         # put footer (timestamp/rerun instructions) into session
         testMethod = getattr(self, self._testMethodName)
         if getattr(testMethod, "__benchmarks_test__", False):
             benchmarks = True
         else:
             benchmarks = False
 
         import datetime
-        print(
-            "Session info generated @",
+        print("Session info generated @",
             datetime.datetime.now().ctime(),
             file=self.session)
-        print(
-            "To rerun this test, issue the following command from the 'test' directory:\n",
+        print("To rerun this test, issue the following command:\n",
             file=self.session)
-        print(
-            "./dotest.py %s -v %s %s" %
-            (self.getRunOptions(),
-             ('+b' if benchmarks else '-t'),
-                self.getRerunArgs()),
+        print("lldb-dotest {}".format(self.getRerunArgs()),
             file=self.session)
         self.session.close()
         del self.session
 
         # process the log files
         log_files_for_this_test = glob.glob(self.log_basename + "*")
 
         if prefix != 'Success' or lldbtest_config.log_success:
def expectedArch(self, archs):
             return True
 
         for arch in archs:
             if arch in self.getArchitecture():
                 return True
 
         return False
 
-    def getRunOptions(self):
-        """Command line option for -A and -C to run this test again, called from
-        self.dumpSessionInfo()."""
-        arch = self.getArchitecture()
-        comp = self.getCompiler()
-        option_str = ""
-        if arch:
-            option_str = "-A " + arch
-        if comp:
-            option_str += " -C " + comp
-        return option_str
-
     def getDebugInfo(self):
         method = getattr(self, self.testMethodName)
         return getattr(method, "debug_info", None)
 
     # ==================================================
     # Build methods supported through a plugin interface
     # ==================================================
 
class TestBase(Base):
     # Can be overridden by the LLDB_TIME_WAIT_NEXT_LAUNCH environment variable.
     timeWaitNextLaunch = 1.0
 
     def generateSource(self, source):
         template = source + '.template'
         temp = os.path.join(self.getSourceDir(), template)
         with open(temp, 'r') as f:
             content = f.read()
-            
         public_api_dir = os.path.join(
             os.environ["LLDB_SRC"], "include", "lldb", "API")
 
         # Look under the include/lldb/API directory and add #include statements
         # for all the SB API headers.
         public_headers = os.listdir(public_api_dir)
         # For different platforms, the include statement can vary.
         if self.hasDarwinFramework():