Fix several test failures on Linux/FreeBSD caused by compiler configuration and invalid environment

[ovyalov]

Summary

Fix several test failures on Linux/FreeBSD caused by compiler configuration and invalid environment:

1. TestSharedLib/TestSharedLibStrippedSymbols - set up explicit environment for target.LaunchSimple in case of Linux/FreeBSD.
2. Fix compiler version detection - since CC environment variable may contain flags along with binary (e.g. "clang -Qunused-arguments -fcolor-diagnostics") we need to extract CC binary to allow which(..) to deduce correctly a compiler's full path .

[emaste]

Seems reasonable overall, with one comment.

test/lang/c/shared_lib/TestSharedLib.py

60–61

Will we need this on FreeBSD and Linux later on? I.e., do we need to append this to the environment?

[ovyalov]

Please see my comment.

Thank you!

test/lang/c/shared_lib/TestSharedLib.py

60–61

Presumably we may move Linux/BSD logic from here to lldbtest.py (https://github.com/llvm-mirror/lldb/blob/master/test/lldbtest.py#L1672) - as TODO suggests to expand this function with support for other platforms.
For now, we still need this environment variable to allow dynamic loader to find the shared library .

[clayborg]

Note the duplicated code in both the TestSharedLib.py and TestSharedLibStrippedSymbols.py test files. This work should all be done in the registerSharedLibrariesWithTarget() function. You can pass an array of full shared library paths to the self.registerSharedLibrariesWithTarget(...) function call, or the function knows how to locate the file in the build directory if you give it library base names.

[ovyalov]

Thanks for suggestion - reworked tests to make registerSharedLibrariesWithTarget to support multiple platforms.
Please take another look.

[clayborg]

The remote install file spec only needs to be set for remote targets and you must use the "lldb.remote_platform.GetWorkingDirectory()" as the directory (since it represents the remote install location) instead of "working_dir" which is on the current host.

To fix this you need to change:

1. We must set the remote install location if we want the shared library
2. to get uploaded to the remote target

remote_shlib_path = os.path.join(working_dir, os.path.basename(local_shlib_path))
shlib_module.SetRemoteInstallFileSpec(lldb.SBFileSpec(remote_shlib_path, False))

to be:

if lldb.remote_platform:

1. We must set the remote install location if we want the shared library
2. to get uploaded to the remote target remote_shlib_path = os.path.join(lldb.remote_platform.GetWorkingDirectory(), os.path.basename(local_shlib_path)) shlib_module.SetRemoteInstallFileSpec(lldb.SBFileSpec(remote_shlib_path, False))

[clayborg]

Try this again with correct formatting:

The remote install file spec only needs to be set for remote targets and you must use the "lldb.remote_platform.GetWorkingDirectory()" as the directory (since it represents the remote install location) instead of "working_dir" which is on the current host.

To fix this you need to change:

# We must set the remote install location if we want the shared library
remote_shlib_path = os.path.join(working_dir, os.path.basename(local_shlib_path))
shlib_module.SetRemoteInstallFileSpec(lldb.SBFileSpec(remote_shlib_path, False))

To:

if lldb.remote_platform:
    # We must set the remote install location if we want the shared library
    # to get uploaded to the remote target
    remote_shlib_path = os.path.join(lldb.remote_platform.GetWorkingDirectory(), os.path.basename(local_shlib_path))
    shlib_module.SetRemoteInstallFileSpec(lldb.SBFileSpec(remote_shlib_path, False))

[ovyalov]

Thanks for catching this - fixed.

How I can run tests with remote_platform tuned on? My understanding that lldb.remote_platform is initialized from platform-name/platform-url/platform-working-dir dotest flags, but if you have an example of command line for remote platform - could you share it please?

[clayborg]

Looks good.

[zturner]

test/lldbtest.py

747–750

This breaks windows because a platformContext is not set for Windows. Do you have suggestions on how to fix this for Windows? Perhaps a default implementation that does nothing?

[zturner]

I posted a proposed fix for this, please take a look.

http://reviews.llvm.org/D6484

[clayborg]

Check my comments which you will want to fix with a new fix.

[emaste]

What's the next step with this change?

[ovyalov]

Closed with r223091

test/lang/c/shared_lib/TestSharedLib.py

 """Test that types defined in shared libraries work correctly."""
 
-import os, time
 import unittest2
 import lldb
 from lldbtest import *
 import lldbutil
 
 class SharedLibTestCase(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
def common_setup(self):
 
         # Create a target by the debugger.
         target = self.dbg.CreateTarget("a.out")
         self.assertTrue(target, VALID_TARGET)
 
         # Break inside the foo function which takes a bar_ptr argument.
         lldbutil.run_break_set_by_file_and_line (self, self.source, self.line, num_expected_locations=1, loc_exact=True)
 
-        if sys.platform.startswith("freebsd") or sys.platform.startswith("linux"):
-            if self.dylibPath in os.environ:
-                environment = [self.dylibPath + "=" + os.environ[self.dylibPath] + ":" + os.getcwd()]
-            else:
-                environment = [self.dylibPath + "=" + os.getcwd()]
-        else:
-            # Register our shared libraries for remote targets so they get automatically uploaded
-            environment = self.registerSharedLibrariesWithTarget(target, self.shlib_names)
 
         # Now launch the process, and do not stop at entry point.
         process = target.LaunchSimple (None, environment, self.get_process_working_directory())
         self.assertTrue(process, PROCESS_IS_VALID)
 
         # The stop reason of the thread should be breakpoint.
         self.expect("thread list", STOPPED_DUE_TO_BREAKPOINT,
             substrs = ['stopped',
                        'stop reason = breakpoint'])
 
         # The breakpoint should have a hit count of 1.
         self.expect("breakpoint list -f", BREAKPOINT_HIT_ONCE,
             substrs = [' resolved, hit count = 1'])

test/lang/c/shared_lib_stripped_symbols/TestSharedLibStrippedSymbols.py

 """Test that types defined in shared libraries with stripped symbols work correctly."""
 
-import os, time
 import unittest2
 import lldb
 from lldbtest import *
 import lldbutil
 
 class SharedLibStrippedTestCase(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
def common_setup(self):
 
         # Create a target by the debugger.
         target = self.dbg.CreateTarget("a.out")
         self.assertTrue(target, VALID_TARGET)
 
         # Break inside the foo function which takes a bar_ptr argument.
         lldbutil.run_break_set_by_file_and_line (self, self.source, self.line, num_expected_locations=1, loc_exact=True)
 
-        if sys.platform.startswith("freebsd") or sys.platform.startswith("linux"):
-            if self.dylibPath in os.environ:
-                environment = [self.dylibPath + "=" + os.environ[self.dylibPath] + ":" + os.getcwd()]
-            else:
-                environment = [self.dylibPath + "=" + os.getcwd()]
-        else:
-            # Register our shared libraries for remote targets so they get automatically uploaded
-            environment = self.registerSharedLibrariesWithTarget(target, self.shlib_names)
 
         # Now launch the process, and do not stop at entry point.
         process = target.LaunchSimple (None, environment, self.get_process_working_directory())
         self.assertTrue(process, PROCESS_IS_VALID)
 
         # The stop reason of the thread should be breakpoint.
         self.expect("thread list", STOPPED_DUE_TO_BREAKPOINT,
             substrs = ['stopped',

test/lldbtest.py

def wrapper(*args, **kwargs):
         self = args[0]
         if "i386" == self.getArchitecture():
             self.skipTest("skipping because i386 is not a supported architecture")
         else:
             func(*args, **kwargs)
     return wrapper
 
 
+class _PlatformContext(object):
+    """Value object class which contains platform-specific options."""
+
+    def __init__(self, shlib_environment_var, shlib_prefix, shlib_extension):
+        self.shlib_environment_var = shlib_environment_var
+        self.shlib_prefix = shlib_prefix
+        self.shlib_extension = shlib_extension
+
+
 class Base(unittest2.TestCase):
     """
     Abstract base for performing lldb (see TestBase) or other generic tests (see
     BenchBase for one example).  lldbtest.Base works with the test driver to
     accomplish things.
     
     """
 
     # The concrete subclass should override this attribute.
     mydir = None
 
     # Keep track of the old current working directory.
     oldcwd = None
-    
     @staticmethod
     def compute_mydir(test_file):
         '''Subclasses should call this function to correctly calculate the required "mydir" attribute as follows: 
             
             mydir = TestBase.compute_mydir(__file__)'''
         test_dir = os.path.dirname(test_file)
         return test_dir[len(os.environ["LLDB_TEST"])+1:]
     
def setUpClass(cls):
 
         # Change current working directory if ${LLDB_TEST} is defined.
         # See also dotest.py which sets up ${LLDB_TEST}.
         if ("LLDB_TEST" in os.environ):
             if traceAlways:
                 print >> sys.stderr, "Change dir to:", os.path.join(os.environ["LLDB_TEST"], cls.mydir)
             os.chdir(os.path.join(os.environ["LLDB_TEST"], cls.mydir))
 
+        # Set platform context.
+        if sys.platform.startswith('darwin'):
+            cls.platformContext = _PlatformContext('DYLD_LIBRARY_PATH', 'lib', 'dylib')
+        elif sys.platform.startswith('linux') or sys.platform.startswith('freebsd'):
+            cls.platformContext = _PlatformContext('LD_LIBRARY_PATH', 'lib', 'so')
+
     @classmethod
     def tearDownClass(cls):
         """
         Python unittest framework class teardown fixture.
         Do class-wide cleanup.
         """
 
         if doCleanup and not lldb.skip_build_and_cleanup:
def setUp(self):
         # See addTearDownHook(self, hook) which allows the client to add a hook
         # function to be run during tearDown() time.
         self.hooks = []
 
         # See HideStdout(self).
         self.sys_stdout_hidden = False
 
         # set environment variable names for finding shared libraries
-        if sys.platform.startswith("darwin"):
-            self.dylibPath = 'DYLD_LIBRARY_PATH'
-        elif sys.platform.startswith("linux") or sys.platform.startswith("freebsd"):
-            self.dylibPath = 'LD_LIBRARY_PATH'
+        self.dylibPath = self.platformContext.shlib_environment_var
 
     def runHooks(self, child=None, child_prompt=None, use_cmd_api=False):
         """Perform the run hooks to bring lldb debugger to the desired state.
 
         By default, expect a pexpect spawned child and child prompt to be
         supplied (use_cmd_api=False).  If use_cmd_api is true, ignore the child
         and child prompt and use self.runCmd() to run the hooks one by one.
 
class TestBase(Base):
     def registerSharedLibrariesWithTarget(self, target, shlibs):
         '''If we are remotely running the test suite, register the shared libraries with the target so they get uploaded, otherwise do nothing
         
         Any modules in the target that have their remote install file specification set will
         get uploaded to the remote host. This function registers the local copies of the
         shared libraries with the target and sets their remote install locations so they will
         be uploaded when the target is run.
         '''
-        environment = None
-        if lldb.remote_platform and shlibs:
-            dyld_environment_var = 'DYLD_FRAMEWORK_PATH' # TODO: localize this for remote systems other than darwin
-            shlib_prefix = "lib"
-            shlib_extension = ".dylib" # TODO: localize this for remote systems other than darwin
-
-            remote_working_dir = lldb.remote_platform.GetWorkingDirectory()
-            # TODO: localize this environment variable for systems other than darwin
-            environment = ['%s=%s' % (dyld_environment_var, remote_working_dir)]
-            # Add any shared libraries to our target if remote so they get
-            # uploaded into the working directory on the remote side
-            for name in shlibs:
-                # The path can be a full path to a shared library, or a make file name like "Foo" for
-                # "libFoo.dylib" or "libFoo.so", or "Foo.so" for "Foo.so" or "libFoo.so", or just a
-                # basename like "libFoo.so". So figure out which one it is and resolve the local copy
-                # of the shared library accordingly
-                if os.path.exists(name):
-                    local_shlib_path = name # name is the full path to the local shared library
-                else:
-                    # Check relative names
-                    local_shlib_path = os.path.join(os.getcwd(), shlib_prefix + name + shlib_extension)
-                    if not os.path.exists(local_shlib_path):
-                        local_shlib_path = os.path.join(os.getcwd(), name + shlib_extension)
-                        if not os.path.exists(local_shlib_path):
-                            local_shlib_path = os.path.join(os.getcwd(), name)
+        if not shlibs:
+            return None
+
+        shlib_environment_var = self.platformContext.shlib_environment_var
+        shlib_prefix = self.platformContext.shlib_prefix
+        shlib_extension = '.' + self.platformContext.shlib_extension
+
+        working_dir = self.get_process_working_directory()
+        environment = ['%s=%s' % (shlib_environment_var, working_dir)]
 
-                    # Make sure we found the local shared library in the above code
-                    self.assertTrue(os.path.exists(local_shlib_path))
-                
-                # Add the shared library to our target
-                shlib_module = target.AddModule(local_shlib_path, None, None, None)
+            if lldb.remote_platform:
                 # We must set the remote install location if we want the shared library
                 # to get uploaded to the remote target
                 remote_shlib_path = os.path.join(lldb.remote_platform.GetWorkingDirectory(), os.path.basename(local_shlib_path))
                 shlib_module.SetRemoteInstallFileSpec(lldb.SBFileSpec(remote_shlib_path, False))
-        environment
-        
+
+        return environment
     # utility methods that tests can use to access the current objects
     def target(self):
         if not self.dbg:
             raise Exception('Invalid debugger instance')
         return self.dbg.GetSelectedTarget()
 
     def process(self):
         if not self.dbg: