Make TestHelloWorld.py to pass remotely on Linux.

[ovyalov]

Summary

TestHelloWorld uses stack unwinding to get a thread's backtrace.
In case of remote run the test should use system libraries (like libc.so) from a target in order to lookup symbols correctly:

• Added optional environment variable LLDB_IMAGE_SEARCH_PATH that represents root directory for custom search paths.
• lldbtest.addImageSearchPaths registers custom target image search paths for directories that contain at least one file.

In total, the test uses for unwinding a downloaded version of libc.so instead of local one.

[clayborg]

looks good.

By the way: on MacOSX we have UUIDs in all of our binaries, so our IDE just gives us all of the files that it builds by adding them to our target. They first create a target, then they add all built modules using:

lldb::SBModule module = target.AddModule (const char *path, const char *triple, const char *uuid_cstr, const char *symfile);

Then when we run, we discover, via the dynamic loader, that we need some binary whose UUID is 1234. Since we were given the local copy of the binary already, we can just match it up and not worry about search paths. Any files that we weren't given binaries for, we expect them to be in the sys root of the platform (where all the /usr/lib binaries are cached, etc). So we expect any binaries that were locally built to be supplied to us (which can be done in the test suite for example) and all others to be in the SDK and the platform plug-in will locate them.

[ovyalov]

In D7574#122311, @clayborg wrote:

looks good.

By the way: on MacOSX we have UUIDs in all of our binaries, so our IDE just gives us all of the files that it builds by adding them to our target. They first create a target, then they add all built modules using:

lldb::SBModule module = target.AddModule (const char *path, const char *triple, const char *uuid_cstr, const char *symfile);

Then when we run, we discover, via the dynamic loader, that we need some binary whose UUID is 1234. Since we were given the local copy of the binary already, we can just match it up and not worry about search paths. Any files that we weren't given binaries for, we expect them to be in the sys root of the platform (where all the /usr/lib binaries are cached, etc). So we expect any binaries that were locally built to be supplied to us (which can be done in the test suite for example) and all others to be in the SDK and the platform plug-in will locate them.

Thank you for context. UUID in some way does simplify correct symbols discovery.
Does it mean that every time you re-build a binary it gets a new UUID assigned (if it's a part of compilation or linker stage)?

[jingham]

It is part of the linker stage so each rebuild should produce an unique UUID. Note, the MacOS X linker actually makes the UUID by checksumming the binary - minus some obvious bits the linker knows to contain timestamps - so if you rebuild the same sources with the same compiler & compiler options, you will get the same UUID. This can cause some confusion, for instance changing just a comment and rebuilding will produce the same UUID... That is an implementation detail, however, and usually doesn't cause problems...

Jim

[emaste]

Thank you for context. UUID in some way does simplify correct symbols discovery.
Does it mean that every time you re-build a binary it gets a new UUID assigned (if it's a part of compilation or linker stage)?

Note that GNU toolchains do the same, it's called build-ID. The actual algorithm is implementation-defined, but I think generally includes all content so unlike OS X changing debug info etc. will produce a new build-ID. When I last looked the gold linker just hashes the whole object.

include/lldb/API/SBTarget.h

#endif
     bool
     GetDescription (lldb::SBStream &description, lldb::DescriptionLevel description_level);
 
     lldb::SBValue
     EvaluateExpression (const char *expr, const SBExpressionOptions &options);
 
     lldb::addr_t
     GetStackRedZoneSize();
-    
+    void
+    AddImageSearchPath (const char* path, const char* replacement);
+
 protected:
     friend class SBAddress;
     friend class SBBlock;
     friend class SBDebugger;
     friend class SBExecutionContext;
     friend class SBFunction;
     friend class SBInstruction;
     friend class SBModule;

scripts/Python/interface/SBTarget.i

public:
     FindSymbols (const char *name, lldb::SymbolType type = eSymbolTypeAny);
 
     bool
     GetDescription (lldb::SBStream &description, lldb::DescriptionLevel description_level);
     
     lldb::addr_t
     GetStackRedZoneSize();
 
+    void
+    AddImageSearchPath (const char* path, const char* replacement);
+
     bool
     operator == (const lldb::SBTarget &rhs) const;
 
     bool
     operator != (const lldb::SBTarget &rhs) const;
 
     lldb::SBValue
     EvaluateExpression (const char *expr, const lldb::SBExpressionOptions &options);

source/API/SBTarget.cpp

if (target_sp)
             abi_sp = process_sp->GetABI();
         else
             abi_sp = ABI::FindPlugin(target_sp->GetArchitecture());
         if (abi_sp)
             return abi_sp->GetRedZoneSize();
     }
     return 0;
 }
+
+void
+SBTarget::AddImageSearchPath (const char* path, const char* replacement)
+{
+    TargetSP target_sp(GetSP());
+    if (target_sp)
+    {
+        target_sp->GetImageSearchPathList().Append(ConstString(path),
+                                                   ConstString(replacement),
+                                                   true);
+    }
+}

test/lldbtest.py

class Base(unittest2.TestCase):
     def getstdFlag(self):
         """ Returns the proper stdflag. """
         if "gcc" in self.getCompiler() and "4.6" in self.getCompilerVersion():
           stdflag = "-std=c++0x"
         else:
           stdflag = "-std=c++11"
         return stdflag
 
+    def addImageSearchPaths(self, target):
+        """Adds image search paths to the target."""
+        img_search_path = os.environ.get('LLDB_IMAGE_SEARCH_PATH', None)
+        if img_search_path:
+            self.addImageSearchPathsInternal(target, img_search_path, '')
+
+    def addImageSearchPathsInternal(self, target, base_dir, rel_dir):
+        files_found = False
+        full_dir_path = os.path.join(base_dir, rel_dir)
+        for p in os.listdir(full_dir_path):
+            full_path = os.path.join(full_dir_path, p)
+            if os.path.isfile(full_path):
+                files_found = True
+            else:
+                self.addImageSearchPathsInternal(target, base_dir, os.path.join(rel_dir, p))
+
+        # Add image search path only a directory that contains files.
+        if files_found:
+            from_path = '/' + rel_dir
+            replacement_path = full_dir_path
+
+            if self.TraceOn():
+                print "Add image search path (%s, %s)" % (from_path, replacement_path)
+            target.AddImageSearchPath(from_path, replacement_path)
+
     def buildDriver(self, sources, exe_name):
         """ Platform-specific way to build a program that links with LLDB (via the liblldb.so
             or LLDB.framework).
         """
 
         stdflag = self.getstdFlag()
         stdlibflag = self.getstdlibFlag()
 

test/python_api/hello_world/TestHelloWorld.py

def test_with_dsym_and_attach_to_process_with_id_api(self):
         """Create target, spawn a process, and attach to it with process id.
 
         Use dsym info and attach to process with id API.
         """
         self.buildDsym(dictionary=self.d)
         self.setTearDownCleanup(dictionary=self.d)
         self.hello_world_attach_with_id_api()
 
-    @not_remote_testsuite_ready
     @python_api_test
     @dwarf_test
     @expectedFailurei386 # llvm.org/pr17384: lldb needs to be aware of linux-vdso.so to unwind stacks properly
     def test_with_dwarf_and_attach_to_process_with_id_api(self):
         """Create target, spawn a process, and attach to it with process id.
 
         Use dwarf map (no dsym) and attach to process with id API.
         """
def test_with_dsym_and_attach_to_process_with_name_api(self):
         """Create target, spawn a process, and attach to it with process name.
 
         Use dsym info and attach to process with name API.
         """
         self.buildDsym(dictionary=self.d)
         self.setTearDownCleanup(dictionary=self.d)
         self.hello_world_attach_with_name_api()
 
-    @not_remote_testsuite_ready
     @python_api_test
     @dwarf_test
     @expectedFailurei386 # llvm.org/pr17384: lldb needs to be aware of linux-vdso.so to unwind stacks properly
     def test_with_dwarf_and_attach_to_process_with_name_api(self):
         """Create target, spawn a process, and attach to it with process name.
 
         Use dwarf map (no dsym) and attach to process with name API.
         """
def hello_world_python(self):
 
         # The breakpoint should have a hit count of 1.
         self.assertTrue(breakpoint.GetHitCount() == 1, BREAKPOINT_HIT_ONCE)
 
     def hello_world_attach_with_id_api(self):
         """Create target, spawn a process, and attach to it by id."""
 
         target = self.dbg.CreateTarget(self.exe)
+        self.addImageSearchPaths(target)
 
         # Spawn a new process
         popen = self.spawnSubprocess(self.exe, ["abc", "xyz"])
         self.addTearDownHook(self.cleanupSubprocesses)
 
         # Give the subprocess time to start and wait for user input
         time.sleep(0.25)
 
def hello_world_attach_with_id_api(self):
         self.expect(stacktraces, exe=False,
             substrs = ['main.c:%d' % self.line2,
                        '(int)argc=3'])
 
     def hello_world_attach_with_name_api(self):
         """Create target, spawn a process, and attach to it by name."""
 
         target = self.dbg.CreateTarget(self.exe)
+        self.addImageSearchPaths(target)
 
         # Spawn a new process
         popen = self.spawnSubprocess(self.exe, ["abc", "xyz"])
         self.addTearDownHook(self.cleanupSubprocesses)
 
         # Give the subprocess time to start and wait for user input
         time.sleep(0.25)