Skip AsanTestCase and AsanTestReportDataCase on Darwin

[ki.stfu]

Summary

This patch skips tests which cause the following error:

1: test_with_dsym (TestMemoryHistory.AsanTestCase) ... 
os command: make clean ; make MAKE_DSYM=YES ARCH=x86_64 CC="/Users/IliaK/p/llvm/build_ninja/bin/clang" 
with pid: 9475
stdout: rm -f "a.out"  main.o main.d main.d.tmp  
rm -f -r "a.out.dSYM"
/Users/IliaK/p/llvm/build_ninja/bin/clang  -fsanitize=address -fsanitize-address-field-padding=1 -g -arch x86_64   -I/Users/IliaK/p/llvm/tools/lldb/test/make/../../include   -c -o main.o main.c
/Users/IliaK/p/llvm/build_ninja/bin/clang  main.o  -fsanitize=address -fsanitize-address-field-padding=1 -g -arch x86_64   -I/Users/IliaK/p/llvm/tools/lldb/test/make/../../include   -o "a.out"

stderr: clang: error: unknown argument: '-fsanitize-address-field-padding=1'
clang: error: unsupported argument 'address' to option 'fsanitize='
ld: file not found: /Users/IliaK/p/llvm/build_ninja/bin/../lib/clang/3.7.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib
clang-3.7: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [a.out] Error 1

retcode: 2


ERROR

os command: make clean 
with pid: 9521
stdout: rm -f "a.out"  main.o main.d main.d.tmp  
rm -f -r "a.out.dSYM"

stderr: 
retcode: 0


Restore dir to: /Users/IliaK/p/llvm/tools/lldb

======================================================================
ERROR: test_with_dsym (TestMemoryHistory.AsanTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/IliaK/p/llvm/tools/lldb/test/lldbtest.py", line 612, in wrapper
    func(*args, **kwargs)
  File "/Users/IliaK/p/llvm/tools/lldb/test/lldbtest.py", line 456, in wrapper
    return func(self, *args, **kwargs)
  File "/Users/IliaK/p/llvm/tools/lldb/test/functionalities/asan/TestMemoryHistory.py", line 24, in test_with_dsym
    self.buildDsym (None, compiler)
  File "/Users/IliaK/p/llvm/tools/lldb/test/lldbtest.py", line 1496, in buildDsym
    if not module.buildDsym(self, architecture, compiler, dictionary, clean):
  File "/Users/IliaK/p/llvm/tools/lldb/test/plugins/builder_darwin.py", line 16, in buildDsym
    lldbtest.system(commands, sender=sender)
  File "/Users/IliaK/p/llvm/tools/lldb/test/lldbtest.py", line 370, in system
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command 'make clean ; make MAKE_DSYM=YES ARCH=x86_64 CC="/Users/IliaK/p/llvm/build_ninja/bin/clang" ' returned non-zero exit status 2
Config=x86_64-clang
----------------------------------------------------------------------

Also this patch fixes findBuiltClang() by looking a clang in the build folder.

BTW, another patch was made in October 2014, but it wasn't committed: http://reviews.llvm.org/D6272.

[ki.stfu]

Friendly ping.

[clayborg]

This test works when LLDB builds it:

 % ./dotest.py functionalities/asan
LLDB library dir: /Volumes/work/gclayton/Documents/src/lldb/trunk/build/Debug
lldb-330.99.0
The 'lldb-mi' executable cannot be located.  Some of the tests may not be run as a result.
lldb.pre_flight: None
lldb.post_flight: None

Session logs for test failures/errors/unexpected successes will go into directory '2015-03-03-12_57_29'
Command invoked: ./dotest.py functionalities/asan
compilers=['clang']

Configuration: arch=x86_64 compiler=clang
----------------------------------------------------------------------
Collected 4 tests

.=success F=fail E=error s=skipped x=expected-fail u=unexpected-success

AddressSanitizer debugger support is active. Memory error breakpoint has been installed and you can now use the 'memory history' command.
AddressSanitizer debugger support is active. Memory error breakpoint has been installed and you can now use the 'memory history' command.
.AddressSanitizer debugger support is active. Memory error breakpoint has been installed and you can now use the 'memory history' command.
AddressSanitizer debugger support is active. Memory error breakpoint has been installed and you can now use the 'memory history' command.
.AddressSanitizer debugger support is active. Memory error breakpoint has been installed and you can now use the 'memory history' command.
AddressSanitizer debugger support is active. Memory error breakpoint has been installed and you can now use the 'memory history' command.
AddressSanitizer report breakpoint hit. Use 'thread info -s' to get extended information about the report.
.AddressSanitizer debugger support is active. Memory error breakpoint has been installed and you can now use the 'memory history' command.
AddressSanitizer debugger support is active. Memory error breakpoint has been installed and you can now use the 'memory history' command.
AddressSanitizer report breakpoint hit. Use 'thread info -s' to get extended information about the report.
.
----------------------------------------------------------------------
Ran 4 tests in 23.301s

OK

So please fix the test case so it works. It sounds like the search path for the libclang_rt.asan_osx_dynamic.dylib isn't being added to the link line. It works fine when lldb is built using xcodebuild.

[ki.stfu]

Enable AsanTestCase and AsanTestReportDataCase tests on Darwin

[ki.stfu]

In D7958#133783, @clayborg wrote:

So please fix the test case so it works. It sounds like the search path for the libclang_rt.asan_osx_dynamic.dylib isn't being added to the link line. It works fine when lldb is built using xcodebuild.

You was right. I added compiler-rt project to llvm/projects and now these tests work on my side.

But I think we should add check for it to be able skip these tests when compiler-rt wasn't built. For example, we can do an alternative version of D6272 (or modify it). How you think?

[clayborg]

Please do add a check for compiler-rt at testing startup and enable the tests if it is present.

[ki.stfu]

Add @skipUnlessCompilerRt decorator

[ki.stfu]

I added @skipUnlessCompilerRt decorator. At the moment, if you have compiler-rt support then these tests pass:

$ ./dotest.py -v --executable $BUILDDIR/bin/lldb functionalities/asan/
...
1: test_with_dsym (TestMemoryHistory.AsanTestCase) ... AddressSanitizer debugger support is active. Memory error breakpoint has been installed and you can now use the 'memory history' command.
AddressSanitizer debugger support is active. Memory error breakpoint has been installed and you can now use the 'memory history' command.
ok
2: test_with_dwarf (TestMemoryHistory.AsanTestCase) ... AddressSanitizer debugger support is active. Memory error breakpoint has been installed and you can now use the 'memory history' command.
AddressSanitizer debugger support is active. Memory error breakpoint has been installed and you can now use the 'memory history' command.
ok
3: test_with_dsym (TestReportData.AsanTestReportDataCase) ... AddressSanitizer debugger support is active. Memory error breakpoint has been installed and you can now use the 'memory history' command.
AddressSanitizer debugger support is active. Memory error breakpoint has been installed and you can now use the 'memory history' command.
AddressSanitizer report breakpoint hit. Use 'thread info -s' to get extended information about the report.
ok
4: test_with_dwarf (TestReportData.AsanTestReportDataCase) ... AddressSanitizer debugger support is active. Memory error breakpoint has been installed and you can now use the 'memory history' command.
AddressSanitizer debugger support is active. Memory error breakpoint has been installed and you can now use the 'memory history' command.
AddressSanitizer report breakpoint hit. Use 'thread info -s' to get extended information about the report.
ok

----------------------------------------------------------------------
Ran 4 tests in 18.729s

OK

But if you don't then tests are ignored:

$ ./dotest.py -v --executable $BUILDDIR/bin/lldb functionalities/asan/
...
1: test_with_dsym (TestMemoryHistory.AsanTestCase) ... skipped 'skip if compiler-rt not found'
2: test_with_dwarf (TestMemoryHistory.AsanTestCase) ... skipped 'skip if compiler-rt not found'
3: test_with_dsym (TestReportData.AsanTestReportDataCase) ... skipped 'skip if compiler-rt not found'
4: test_with_dwarf (TestReportData.AsanTestReportDataCase) ... skipped 'skip if compiler-rt not found'

----------------------------------------------------------------------
Ran 4 tests in 0.071s

OK (skipped=4)

[clayborg]

Looks good.

test/functionalities/asan/TestMemoryHistory.py

 class AsanTestCase(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
     # The default compiler ("clang") may not support Address Sanitizer or it
     # may not have the debugging API which was recently added, so we're calling
     # self.useBuiltClang() to use clang from the llvm-build directory instead
 
-    @unittest2.skipUnless(sys.platform.startswith("darwin"), "requires Darwin")
-    @skipIfRemote
     @dsym_test
+    @skipIfRemote
+    @skipUnlessCompilerRt
+    @unittest2.skipUnless(sys.platform.startswith("darwin"), "requires Darwin")
     def test_with_dsym (self):
         compiler = self.findBuiltClang ()
         self.buildDsym (None, compiler)
         self.asan_tests ()
 
+    @dwarf_test
+    @expectedFailureLinux # non-core functionality, need to reenable and fix later (DES 2014.11.07)
     @skipIfFreeBSD # llvm.org/pr21136 runtimes not yet available by default
     @skipIfRemote
-    @expectedFailureLinux # non-core functionality, need to reenable and fix later (DES 2014.11.07)
-    @dwarf_test
+    @skipUnlessCompilerRt
     def test_with_dwarf (self):
         compiler = self.findBuiltClang ()
         self.buildDwarf (None, compiler)
         self.asan_tests ()
 
     def setUp(self):
         # Call super's setUp().
         TestBase.setUp(self)

test/functionalities/asan/TestReportData.py

 class AsanTestReportDataCase(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
     # The default compiler ("clang") may not support Address Sanitizer or it
     # may not have the debugging API which was recently added, so we're calling
     # self.useBuiltClang() to use clang from the llvm-build directory instead
 
-    @unittest2.skipUnless(sys.platform.startswith("darwin"), "requires Darwin")
-    @skipIfRemote
     @dsym_test
+    @skipIfRemote
+    @skipUnlessCompilerRt
+    @unittest2.skipUnless(sys.platform.startswith("darwin"), "requires Darwin")
     def test_with_dsym (self):
         compiler = self.findBuiltClang ()
         self.buildDsym (None, compiler)
         self.asan_tests ()
 
+    @dwarf_test
+    @expectedFailureLinux # non-core functionality, need to reenable and fix later (DES 2014.11.07)
     @skipIfFreeBSD # llvm.org/pr21136 runtimes not yet available by default
     @skipIfRemote
-    @expectedFailureLinux # non-core functionality, need to reenable and fix later (DES 2014.11.07)
-    @dwarf_test
+    @skipUnlessCompilerRt
     def test_with_dwarf (self):
         compiler = self.findBuiltClang ()
         self.buildDwarf (None, compiler)
         self.asan_tests ()
 
     def setUp(self):
         # Call super's setUp().
         TestBase.setUp(self)

test/lldbtest.py

def wrapper(*args, **kwargs):
         from unittest2 import case
         self = args[0]
         if "i386" == self.getArchitecture():
             self.skipTest("skipping because i386 is not a supported architecture")
         else:
             func(*args, **kwargs)
     return wrapper
 
+def skipUnlessCompilerRt(func):
+    """Decorate the item to skip tests if testing remotely."""
+    if isinstance(func, type) and issubclass(func, unittest2.TestCase):
+        raise Exception("@skipUnless can only be used to decorate a test method")
+    @wraps(func)
+    def wrapper(*args, **kwargs):
+        from unittest2 import case
+        import os.path
+        compilerRtPath = os.path.join(os.path.dirname(__file__), "..", "..", "..", "projects", "compiler-rt")
+        if not os.path.exists(compilerRtPath):
+            self = args[0]
+            self.skipTest("skip if compiler-rt not found")
+        else:
+            func(*args, **kwargs)
+    return wrapper
 
 class _PlatformContext(object):
     """Value object class which contains platform-specific options."""
 
     def __init__(self, shlib_environment_var, shlib_prefix, shlib_extension):
         self.shlib_environment_var = shlib_environment_var
         self.shlib_prefix = shlib_prefix
         self.shlib_extension = shlib_extension
def findBuiltClang(self):
           "llvm-build/Release/x86_64/Release/bin/clang",
           "llvm-build/Debug/x86_64/Debug/bin/clang",
         ]
         lldb_root_path = os.path.join(os.path.dirname(__file__), "..")
         for p in paths_to_try:
             path = os.path.join(lldb_root_path, p)
             if os.path.exists(path):
                 return path
-        
+        # Tries to find clang at the same folder as the lldb
+        path = os.path.join(os.path.dirname(self.lldbExec), "clang")
+        if os.path.exists(path):
+            return path
+        
         return os.environ["CC"]
 
     def getBuildFlags(self, use_cpp11=True, use_libcxx=False, use_libstdcxx=False):
         """ Returns a dictionary (which can be provided to build* functions above) which
             contains OS-specific build flags.
         """
         cflags = ""
         ldflags = ""