Fix several test failures on Linux/FreeBSD caused by compiler configuration and invalid environment

[ovyalov]

Summary

Fix several test failures on Linux/FreeBSD caused by compiler configuration and invalid environment:

1. TestSharedLib/TestSharedLibStrippedSymbols - set up explicit environment for target.LaunchSimple in case of Linux/FreeBSD.
2. Fix compiler version detection - since CC environment variable may contain flags along with binary (e.g. "clang -Qunused-arguments -fcolor-diagnostics") we need to extract CC binary to allow which(..) to deduce correctly a compiler's full path .

[emaste]

Seems reasonable overall, with one comment.

test/lang/c/shared_lib/TestSharedLib.py

62–63

Will we need this on FreeBSD and Linux later on? I.e., do we need to append this to the environment?

[ovyalov]

Please see my comment.

Thank you!

test/lang/c/shared_lib/TestSharedLib.py

62–63

Presumably we may move Linux/BSD logic from here to lldbtest.py (https://github.com/llvm-mirror/lldb/blob/master/test/lldbtest.py#L1672) - as TODO suggests to expand this function with support for other platforms.
For now, we still need this environment variable to allow dynamic loader to find the shared library .

[clayborg]

Note the duplicated code in both the TestSharedLib.py and TestSharedLibStrippedSymbols.py test files. This work should all be done in the registerSharedLibrariesWithTarget() function. You can pass an array of full shared library paths to the self.registerSharedLibrariesWithTarget(...) function call, or the function knows how to locate the file in the build directory if you give it library base names.

[ovyalov]

Thanks for suggestion - reworked tests to make registerSharedLibrariesWithTarget to support multiple platforms.
Please take another look.

[clayborg]

The remote install file spec only needs to be set for remote targets and you must use the "lldb.remote_platform.GetWorkingDirectory()" as the directory (since it represents the remote install location) instead of "working_dir" which is on the current host.

To fix this you need to change:

1. We must set the remote install location if we want the shared library
2. to get uploaded to the remote target

remote_shlib_path = os.path.join(working_dir, os.path.basename(local_shlib_path))
shlib_module.SetRemoteInstallFileSpec(lldb.SBFileSpec(remote_shlib_path, False))

to be:

if lldb.remote_platform:

1. We must set the remote install location if we want the shared library
2. to get uploaded to the remote target remote_shlib_path = os.path.join(lldb.remote_platform.GetWorkingDirectory(), os.path.basename(local_shlib_path)) shlib_module.SetRemoteInstallFileSpec(lldb.SBFileSpec(remote_shlib_path, False))

[clayborg]

Try this again with correct formatting:

The remote install file spec only needs to be set for remote targets and you must use the "lldb.remote_platform.GetWorkingDirectory()" as the directory (since it represents the remote install location) instead of "working_dir" which is on the current host.

To fix this you need to change:

# We must set the remote install location if we want the shared library
remote_shlib_path = os.path.join(working_dir, os.path.basename(local_shlib_path))
shlib_module.SetRemoteInstallFileSpec(lldb.SBFileSpec(remote_shlib_path, False))

To:

if lldb.remote_platform:
    # We must set the remote install location if we want the shared library
    # to get uploaded to the remote target
    remote_shlib_path = os.path.join(lldb.remote_platform.GetWorkingDirectory(), os.path.basename(local_shlib_path))
    shlib_module.SetRemoteInstallFileSpec(lldb.SBFileSpec(remote_shlib_path, False))

[ovyalov]

Thanks for catching this - fixed.

How I can run tests with remote_platform tuned on? My understanding that lldb.remote_platform is initialized from platform-name/platform-url/platform-working-dir dotest flags, but if you have an example of command line for remote platform - could you share it please?

[clayborg]

Looks good.

[zturner]

test/lldbtest.py

721–724

This breaks windows because a platformContext is not set for Windows. Do you have suggestions on how to fix this for Windows? Perhaps a default implementation that does nothing?

[zturner]

I posted a proposed fix for this, please take a look.

http://reviews.llvm.org/D6484

[clayborg]

Check my comments which you will want to fix with a new fix.

[emaste]

What's the next step with this change?

[ovyalov]

Closed with r223091

test/lang/c/shared_lib/TestSharedLib.py

def test_frame_variable_with_dwarf(self):
         self.frame_var()
 
     def setUp(self):
         # Call super's setUp().
         TestBase.setUp(self)
         # Find the line number to break inside main().
         self.source = 'main.c'
         self.line = line_number(self.source, '// Set breakpoint 0 here.')
-        if sys.platform.startswith("freebsd") or sys.platform.startswith("linux"):
-            if "LD_LIBRARY_PATH" in os.environ:
-                self.runCmd("settings set target.env-vars " + self.dylibPath + "=" + os.environ["LD_LIBRARY_PATH"] + ":" + os.getcwd())
-            else:
-                self.runCmd("settings set target.env-vars " + self.dylibPath + "=" + os.getcwd())
-            self.addTearDownHook(lambda: self.runCmd("settings remove target.env-vars " + self.dylibPath))
         self.shlib_names = ["foo"]
     
     def common_setup(self):
         # Run in synchronous mode
         self.dbg.SetAsync(False)
 
         # Create a target by the debugger.
         target = self.dbg.CreateTarget("a.out")
         self.assertTrue(target, VALID_TARGET)
 
         # Break inside the foo function which takes a bar_ptr argument.
         lldbutil.run_break_set_by_file_and_line (self, self.source, self.line, num_expected_locations=1, loc_exact=True)
 
+        if sys.platform.startswith("freebsd") or sys.platform.startswith("linux"):
+            if self.dylibPath in os.environ:
+                environment = [self.dylibPath + "=" + os.environ[self.dylibPath] + ":" + os.getcwd()]
+            else:
+                environment = [self.dylibPath + "=" + os.getcwd()]
+        else:
             # Register our shared libraries for remote targets so they get automatically uploaded
             environment = self.registerSharedLibrariesWithTarget(target, self.shlib_names)
 
         # Now launch the process, and do not stop at entry point.
         process = target.LaunchSimple (None, environment, self.get_process_working_directory())
         self.assertTrue(process, PROCESS_IS_VALID)
 
         # The stop reason of the thread should be breakpoint.
         self.expect("thread list", STOPPED_DUE_TO_BREAKPOINT,
             substrs = ['stopped',

test/lang/c/shared_lib_stripped_symbols/TestSharedLibStrippedSymbols.py

def test_frame_variable_with_dwarf(self):
         self.frame_var()
 
     def setUp(self):
         # Call super's setUp().
         TestBase.setUp(self)
         # Find the line number to break inside main().
         self.source = 'main.c'
         self.line = line_number(self.source, '// Set breakpoint 0 here.')
-        if sys.platform.startswith("freebsd") or sys.platform.startswith("linux"):
-            if "LD_LIBRARY_PATH" in os.environ:
-                self.runCmd("settings set target.env-vars " + self.dylibPath + "=" + os.environ["LD_LIBRARY_PATH"] + ":" + os.getcwd())
-            else:
-                self.runCmd("settings set target.env-vars " + self.dylibPath + "=" + os.getcwd())
-            self.addTearDownHook(lambda: self.runCmd("settings remove target.env-vars " + self.dylibPath))
         self.shlib_names = ["foo"]
 
     def common_setup(self):
         # Run in synchronous mode
         self.dbg.SetAsync(False)
 
         # Create a target by the debugger.
         target = self.dbg.CreateTarget("a.out")
         self.assertTrue(target, VALID_TARGET)
 
         # Break inside the foo function which takes a bar_ptr argument.
         lldbutil.run_break_set_by_file_and_line (self, self.source, self.line, num_expected_locations=1, loc_exact=True)
 
+        if sys.platform.startswith("freebsd") or sys.platform.startswith("linux"):
+            if self.dylibPath in os.environ:
+                environment = [self.dylibPath + "=" + os.environ[self.dylibPath] + ":" + os.getcwd()]
+            else:
+                environment = [self.dylibPath + "=" + os.getcwd()]
+        else:
             # Register our shared libraries for remote targets so they get automatically uploaded
             environment = self.registerSharedLibrariesWithTarget(target, self.shlib_names)
 
         # Now launch the process, and do not stop at entry point.
         process = target.LaunchSimple (None, environment, self.get_process_working_directory())
         self.assertTrue(process, PROCESS_IS_VALID)
 
         # The stop reason of the thread should be breakpoint.
         self.expect("thread list", STOPPED_DUE_TO_BREAKPOINT,
             substrs = ['stopped',

test/lldbtest.py

def setUpClass(cls):
         # Change current working directory if ${LLDB_TEST} is defined.
         # See also dotest.py which sets up ${LLDB_TEST}.
         if ("LLDB_TEST" in os.environ):
             if traceAlways:
                 print >> sys.stderr, "Change dir to:", os.path.join(os.environ["LLDB_TEST"], cls.mydir)
             os.chdir(os.path.join(os.environ["LLDB_TEST"], cls.mydir))
 
     @classmethod
     def tearDownClass(cls):
         """
         Python unittest framework class teardown fixture.
         Do class-wide cleanup.
         """
 
         if doCleanup and not lldb.skip_build_and_cleanup:
             # First, let's do the platform-specific cleanup.
             module = builder_module()
             if not module.cleanup():
                 raise Exception("Don't know how to do cleanup")
 
def getArchitecture(self):
         module = builder_module()
         return module.getArchitecture()
 
     def getCompiler(self):
         """Returns the compiler in effect the test suite is running with."""
         module = builder_module()
         return module.getCompiler()
 
+    def getCompilerBinary(self):
+        """Returns the compiler binary the test suite is running with."""
+        return self.getCompiler().split()[0]
+
     def getCompilerVersion(self):
         """ Returns a string that represents the compiler version.
             Supports: llvm, clang.
         """
         from lldbutil import which
         version = 'unknown'
 
-        compiler = self.getCompiler()
+        compiler = self.getCompilerBinary()
         version_output = system([[which(compiler), "-v"]])[1]
         for line in version_output.split(os.linesep):
             m = re.search('version ([0-9\.]+)', line)
             if m:
                 version = m.group(1)
         return version
 
     def isIntelCompiler(self):