[lldb] Add support for shared library load when executable called through ld.

[rdhindsa]

Summary

Add support for shared library load when executable called through ld.

Currently, when an executable is called like: ld-linux --library-path <path> executable, lldb is not able to load the shared libraries needed by the executable provided by library-path. lldb is not able to set Rendezvous breakpoint in this case.

This patch adds the support to enable setting rendezvous breakpoint when called using ld-*.so. It enables it to hit the breakpoint and extract the address of the rendezvous structure, which is how lldb is now able to load the shared libraries.

[clayborg]

Thanks for getting to this. I had a diff going a few weeks ago, but never got things fully working!

lldb/include/lldb/Symbol/ObjectFile.h

465–479 ↗

(On Diff #366373)

No need to add this to the ObjectFile class, just search for the symbol using the object file you already have in DYLDRendezvous.cpp. See comments down below.

We try to keep ObjectFile as an abstraction that sits on top of any object file type (ELF, mach-p, COFF), and this one is very specific to ELF. I would rather keep this out of the ObjectFile API, and we have an easy work around where we just search for the symbol by name in the module anyway.

lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DYLDRendezvous.cpp

83–84

Inline contents of ObjectFileELF::GetRendezvousStructureAddress(...) to here. We already have the object file we need, so we can just extract the symbol from here.

I had started to try and fix this as well a few weeks ago, and I used Module::FindFirstSymbolWithNameAndType(...) to avoid having to get multiple results back:

const Symbol* _r_debug = target->GetExecutableModule()->FindFirstSymbolWithNameAndType(ConstString("_r_debug"));
if (_r_debug) {
  info_addr = _r_debug->GetAddress().GetLoadAddress(target);
  if (info_addr != LLDB_INVALID_ADDRESS) {
    LLDB_LOGF(log,
              "%s resolved by finding symbol '_r_debug' whose value is 0x%" PRIx64,
            __FUNCTION__, info_addr);
    return info_addr;
  }
}

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp

885–899 ↗

(On Diff #366373)

Revert and move contents to where this was called in DYLDRendezvous.cpp. It is very easy to have the DYLDRendezvous just search for the symbol itself.

lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.h

134–136 ↗

(On Diff #366373)

Revert

[rdhindsa]

Reverted code changes from ObjectFileELF as suggested and added directly to DYLDRendezvous.cpp.

Is there a better place to place the test, or is it okay to keep under ObjectFile.

[clayborg]

In D108061#2947526, @rdhindsa wrote:

Reverted code changes from ObjectFileELF as suggested and added directly to DYLDRendezvous.cpp.

Is there a better place to place the test, or is it okay to keep under ObjectFile.

I think a test would be better added to the python test area of LLDB.

The test will look something like:

"""
Test that LLDB can launch a linux executable through the dynamic loader and still hit a breakpoint.
"""

import lldb

from lldbsuite.test.decorators import *
from lldbsuite.test.lldbtest import *
from lldbsuite.test import lldbutil

class TestLinux64LaunchingViaDynamicLoader(TestBase):
    mydir = TestBase.compute_mydir(__file__)

    def test(self):
        """Test we can launch and hit a breakpoint when we run our program through the dynamic loader"""
        self.build()
        exe = "/lib64/ld-linux-x86-64.so.2"
        launch_info = lldb.SBLaunchInfo([exe, self.getBuildArtifact("a.out")])
        launch_info.SetWorkingDirectory(self.get_process_working_directory())
        lldbutil.run_to_source_breakpoint(self, "// break here", lldb.SBFileSpec("main.cpp"), exe=exe, in_cwd=False, launch_info=launch_info)

This should do everything that is needed!

So the steps to create the test are:

• make a new directory in "lldb/test/API/functionalities/dyld-launch-linux"
  • create the main.cpp as noted above
  • add a TestDyldLaunchLinux.py and put the above test case python into it
  • copy a Makefile from another test like "lldb/test/API/commands/expression/anonymous-struct/Makefile" into this directory

To run the test, build "check-lldb" first and kill it once it starts running the test suite. Then you can type this from the llvm build directory:

$ ./bin/llvm-lit -sv ../llvm-project/lldb/test/API/functionalities/dyld-launch-linux

(fix the path to llvm-project if needed in the above command).

Let me know if there are any questions. You can then remove all of the test files from lldb/test/Shell/ObjectFile/ELF

lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp

356–361

I know you copied this from below, but LLVM coding conventions say to do comments like:

https://llvm.org/docs/CodingStandards.html#id13

And might be nice to fix the ones below too.

lldb/test/Shell/ObjectFile/ELF/ld_test.test

15 ↗

(On Diff #366692)

So I worry the "/lib64/ld-linux-x86-64.so.2" path won't be portable across different linux distributions. Anyone know if it is the same on every linux?

[rdhindsa]

Updated the test to check for the respective function name from the shared library in the frames.

[rdhindsa]

Updated the test to check for the respective function name from the shared library in the frames.
I didn't add run_to_source_breakpoint since we can't set a breakpoint on main initially before running the program. Hence it would need to be run first and then we can check later to see if the function name is available. Please let me know if you think there is any concern with this way.

[clayborg]

In D108061#2950858, @rdhindsa wrote:

Updated the test to check for the respective function name from the shared library in the frames.
I didn't add run_to_source_breakpoint since we can't set a breakpoint on main initially before running the program.
Hence it would need to be run first and then we can check later to see if the function name is available. Please let me know if you think there is any concern with this way.

We need to figure out where the main program is loaded somehow and the dynamic loader needs to set the load address for "a.out" so that breakpoints can be resolved and hit. We _do_ need to make sure a breakpoint can be set and hit otherwise this patch will just show us a debug session that has the dynamic loader as the main executable, but we will never see where the program it will run is loaded. That is the tricky part of this patch and where I got stuck. GDB is able to do this, but I didn't look at how they did it. We will want our program to show up in the "image list" output with a valid load address.

Are you able to hit a breakpoint if one is set in the a.out program?

[rdhindsa]

I checked the status with both gdb and lldb using test case added here. It looks like breakpoint on main doesn't get hit even for gdb when called through ld.so. However, breakpoint on shared library functions is supported by gdb. With this patch, we are now able to set breakpoint on functions in shared libraries as supported by gdb.

bin/lldb -- /lib64/ld-linux-x86-64.so.2 --library-path $PWD ./a.out
(lldb) target create "/lib64/ld-linux-x86-64.so.2"
 Current executable set to '/lib64/ld-linux-x86-64.so.2' (x86_64).
(lldb) settings set -- target.run-args  "--library-path" "........" "./a.out"
(lldb) b main
Breakpoint 1: no locations (pending).
WARNING:  Unable to resolve breakpoint to any actual locations.
(lldb) b get_signal_crash
Breakpoint 2: no locations (pending).
WARNING:  Unable to resolve breakpoint to any actual locations.
(lldb) run
Process 526132 launched: '/lib64/ld-linux-x86-64.so.2' (x86_64)
1 location added to breakpoint 2
Process 526132 stopped
* thread #1, name = 'ld-linux-x86-64', stop reason = breakpoint 2.1
    frame #0: 0x00007ffff7fa1110 libsignal_file.so`get_signal_crash()

gdb --args /lib64/ld-linux-x86-64.so.2 --library-path $PWD ./a.out
(gdb) b main
Function "main" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (main) pending.
(gdb) b get_signal_crash
Function "get_signal_crash" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 2 (get_signal_crash) pending.
(gdb) run
Starting program: /lib64/ld-linux-x86-64.so.2 --library-path ...... ./a.out
Breakpoint 2, 0x00007ffff7fa1114 in get_signal_crash()

This patch adds the support for shared library load. Do you think if it is okay if the functionality to be able to breakpoint on the main executable is added as next step?

[rdhindsa]

Looking further as to why executable symbols are not loaded, it looks like when we read the link_map in lldb, AddSOEntries() is called for all the entries in link_map read from ld.so. The first entry's name(which corresponds to executable) is empty, which seems to be set by ld.so here (https://github.com/bminor/glibc/blob/master/elf/rtld.c#L1389). main_map at this location has the entry for respective executable, but its name is removed, hence lldb while going through entries can't read entry about main executable.

I verified that lldb does iterate through the link_map list with this patch, reading that file name as empty. I also verified that if respective line is commented in ld.so , and hence the name of executable is retained, lldb with this patch is able to set breakpoint on main as well. Hence, it would require change to glibc as well. Once that is in, we can add additional test for that feature later on as a follow-up patch. Does that sound okay?

[clayborg]

In D108061#2955707, @rdhindsa wrote:

Looking further as to why executable symbols are not loaded, it looks like when we read the link_map in lldb, AddSOEntries() is called for all the entries in link_map read from ld.so. The first entry's name(which corresponds to executable) is empty, which seems to be set by ld.so here (https://github.com/bminor/glibc/blob/master/elf/rtld.c#L1389). main_map at this location has the entry for respective executable, but its name is removed, hence lldb while going through entries can't read entry about main executable.
I verified that lldb does iterate through the link_map list with this patch, reading that file name as empty. I also verified that if respective line is commented in ld.so , and hence the name of executable is retained, lldb with this patch is able to set breakpoint on main as well. Hence, it would require change to glibc as well. Once that is in, we can add additional test for that feature later on as a follow-up patch. Does that sound okay?

Since the executable name is empty, I wonder what would happen if we looked up the address in the memory region information. This information is populated by the /proc/<pid>/maps file, and each memory region has a name, and that name is usually the path to the object file on disk. As one last effort can you check if the address' memory region info has the path to the executable?

[clayborg]

The code to get a memory region is on the process:

virtual Status Process::GetMemoryRegionInfo(lldb::addr_t load_addr, MemoryRegionInfo &range_info);

Then the memory region info might have a valid name:

ConstString region_name = range_info.GetName();

My guess in that "region_name" might contain the path we need!

[rdhindsa]

Updated to extract executable file name using GetMemoryRegionInfo since ld.so returns empty file name for executable file name.

[clayborg]

I am so happy that the memory region trick worked! Thanks for working on this and trying out the extra fixes, we now have great support for debugging via the dynamic linker!

[rdhindsa]

Thank you for the review and suggestion to check memory region.

[MaskRay]

lldb/test/API/functionalities/dyld-launch-linux/Makefile

2

CXX_SOURCES := main.cpp
DYLIB_NAME := signal_file
DYLIB_CXX_SOURCES := signal_file.cpp

include Makefile.rules

macOS uses .dylib dynamic libraries and the hard-coded .so can cause build failures, even if the test skips execution on macOS.

[MaskRay]

lldb/test/API/functionalities/dyld-launch-linux/TestDyldLaunchLinux.py

17

39

space after comma

46

delete blank lines

lldb/test/API/functionalities/dyld-launch-linux/signal_file.h

2

C++ doesn't typically use (void). Just use ()

[MaskRay]

lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DYLDRendezvous.cpp

294

This sentence may be ambiguous, because all dynamically linked executables are interpreted by ld.so

"When argv[0] is ld.so"

[teemperor]

lldb/test/API/functionalities/dyld-launch-linux/TestDyldLaunchLinux.py

14

Could we just make the test Linux-specific? I doubt we'll ever find a /lib64/ld-linux-x86-64.so.2 on macOS. In theory you could also move the build behind the file-existence check, but then the test will PASS instead of UNSUPPORTED (and the later better fits the situation that this doesn't run because it isn't supported).

Also if you add no_debug_info_test then we only run this test once (which makes the test about 3x faster).

@skipIf(oslist=no_match(['linux']))
@no_debug_info_test
def test(self):

16

Could we make this based on a list of files? There will be people with slightly different paths/file names so I anyway anticipate that we get more candidates to check here:

candidates = [
    "/lib64/ld-linux-x86-64.so.2",
    "/usr/lib/ld-linux-x86-64.so.2"
]
exe = next((c for c in candidates if os.path.exists(c)), None)
if not exe:
    return

[rdhindsa]

This patch caused a test failure when it was submitted two days ago, so I reverted it that day. However, I have not been able to reproduce the failure locally even using the cmake config from bot.

Failure was in api/multithreaded/TestMultithreaded.py

Failing bot link: https://lab.llvm.org/buildbot/#/builders/68/builds/17556/steps/6/logs/stdio

I would appreciate any ideas to be able to repro it locally. The email from buildbot showed Sigsegv:

Traceback (most recent call last):
  File "/home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/packages/Python/lldbsuite/test/decorators.py", line 149, in wrapper
    return func(*args, **kwargs)
  File "/home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/test/API/api/multithreaded/TestMultithreaded.py", line 37, in test_python_stop_hook
    'test_python_stop_hook')
  File "/home/worker/2.0.1/lldb-x86_64-debian/llvm-project/lldb/test/API/api/multithreaded/TestMultithreaded.py", line 116, in build_and_test
    check_call(exe, env=env, stdout=fnull, stderr=fnull)
  File "/usr/lib/python3.7/subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/worker/2.0.1/lldb-x86_64-debian/build/lldb-test-build.noindex/api/multithreaded/TestMultithreaded.test_python_stop_hook/test_python_stop_hook', '/home/worker/2.0.1/lldb-x86_64-debian/build/lldb-test-build.noindex/api/multithreaded/TestMultithreaded.test_python_stop_hook/inferior_program']' died with <Signals.SIGSEGV: 11>.
Config=x86_64-/home/worker/2.0.1/lldb-x86_64-debian/build/bin/clang

[clayborg]

the multi-threaded test often times out for me. I wonder if it is timing out for you as well when you run the test suite locally? Maybe this is covering up the crash because the crash doesn't happen before the test times out for you?

[rdhindsa]

For me, this test passes every time. I have tried multiple times with instructions from failing bot, but haven't been able to reproduce the failure locally yet.

[clayborg]

In D108061#2973624, @rdhindsa wrote:

For me, this test passes every time. I have tried multiple times with instructions from failing bot, but haven't been able to reproduce the failure locally yet.

I will patch this into my linux server and try out the test suite

[clayborg]

In D108061#2973624, @rdhindsa wrote:

For me, this test passes every time. I have tried multiple times with instructions from failing bot, but haven't been able to reproduce the failure locally yet.

A few things you can try:

• build in "Release" or "RelWithDebInfo" and run the test suite. Probably closer to what the buildbots do
• build with asserts enabled if you aren't already
• build with ASAN or UBSAN and run that specific test only and see if anything gets triggered

[clayborg]

I tried with RelWithDebInfo and it failed, but it was because it wasn't building:

devvm1358:~/local/llvm/main/Release $ ./bin/llvm-lit -sv ../llvm-project/lldb/test/API/api/multithreaded/TestMultithreaded.py
llvm-lit: /home/gclayton/local/llvm/main/llvm-project/lldb/test/API/lit.cfg.py:165: warning: Could not set a default per-test timeout. Requires the Python psutil module but it could not be found. Try installing it via pip or via your operating system's package manager.
FAIL: lldb-api :: api/multithreaded/TestMultithreaded.py (1 of 1)
******************** TEST 'lldb-api :: api/multithreaded/TestMultithreaded.py' FAILED ********************
Script:
--
/usr/local/fbcode/platform009/bin/python3.8 /home/gclayton/local/llvm/main/llvm-project/lldb/test/API/dotest.py -u CXXFLAGS -u CFLAGS --env ARCHIVER=/data/users/gclayton/toolchain_dev/build/msdk/bootstrap/c432e0bc77938fdcb571d65e559987f5f537c1ca-69c93d769e854e5ef9a51400675f23e221fdd322/Toolchains/facebook-Asserts-12.xctoolchain/usr/bin/llvm-ar --env OBJCOPY=/home/gclayton/local/toolchain_dev/build/msdk/bootstrap/c432e0bc77938fdcb571d65e559987f5f537c1ca-69c93d769e854e5ef9a51400675f23e221fdd322/Toolchains/facebook-Asserts-12.xctoolchain/usr/bin/objcopy --env LLVM_LIBS_DIR=/home/gclayton/local/llvm/main/Release/./lib --arch x86_64 --build-dir /home/gclayton/local/llvm/main/Release/lldb-test-build.noindex --lldb-module-cache-dir /home/gclayton/local/llvm/main/Release/lldb-test-build.noindex/module-cache-lldb/lldb-api --clang-module-cache-dir /home/gclayton/local/llvm/main/Release/lldb-test-build.noindex/module-cache-clang/lldb-api --executable /home/gclayton/local/llvm/main/Release/./bin/lldb --compiler /home/gclayton/local/llvm/main/Release/./bin/clang --dsymutil /home/gclayton/local/llvm/main/Release/./bin/dsymutil --llvm-tools-dir /home/gclayton/local/llvm/main/Release/./bin --lldb-libs-dir /home/gclayton/local/llvm/main/Release/./lib /data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -p TestMultithreaded.py
--
Exit Code: 1

Command Output (stdout):
--
lldb version 14.0.0 (https://github.com/llvm/llvm-project revision c728bd5bbaab5dad3bf0703f1a85d65cd1237e79)
  clang revision c728bd5bbaab5dad3bf0703f1a85d65cd1237e79
  llvm revision c728bd5bbaab5dad3bf0703f1a85d65cd1237e79
Skipping the following test categories: ['libc++', 'dsym', 'gmodules', 'debugserver', 'objc']

--
Command Output (stderr):
--
FAIL: LLDB (/home/gclayton/local/llvm/main/Release/bin/clang-x86_64) :: test_breakpoint_callback (TestMultithreaded.SBBreakpointCallbackCase)
FAIL: LLDB (/home/gclayton/local/llvm/main/Release/bin/clang-x86_64) :: test_python_stop_hook (TestMultithreaded.SBBreakpointCallbackCase)
FAIL: LLDB (/home/gclayton/local/llvm/main/Release/bin/clang-x86_64) :: test_sb_api_listener_event_description (TestMultithreaded.SBBreakpointCallbackCase)
FAIL: LLDB (/home/gclayton/local/llvm/main/Release/bin/clang-x86_64) :: test_sb_api_listener_event_process_state (TestMultithreaded.SBBreakpointCallbackCase)
UNSUPPORTED: LLDB (/home/gclayton/local/llvm/main/Release/bin/clang-x86_64) :: test_sb_api_listener_resume (TestMultithreaded.SBBreakpointCallbackCase) (skipping due to the following parameter(s): target o/s) 
======================================================================
ERROR: test_breakpoint_callback (TestMultithreaded.SBBreakpointCallbackCase)
   Test the that SBBreakpoint callback is invoked when a breakpoint is hit.
----------------------------------------------------------------------
Error when building test subject.

Build Command:
make VPATH=/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -C /home/gclayton/local/llvm/main/Release/lldb-test-build.noindex/api/multithreaded/TestMultithreaded.test_breakpoint_callback -I /data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -I /data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make -f /data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded/Makefile all  ARCH=x86_64 CC="/home/gclayton/local/llvm/main/Release/bin/clang"   CLANG_MODULE_CACHE_DIR=/home/gclayton/local/llvm/main/Release/lldb-test-build.noindex/module-cache-clang/lldb-api CXX_SOURCES='driver.cpp test_breakpoint_callback.cpp' EXE='test_breakpoint_callback' CFLAGS_EXTRAS='-std=c++11  -I/data/users/gclayton/llvm/main/llvm-project/lldb/include' LD_EXTRAS='-L/home/gclayton/local/llvm/main/Release/./lib -llldb -Wl,-rpath,/home/gclayton/local/llvm/main/Release/./lib'

Build Command Output:
make: Entering directory '/data/users/gclayton/llvm/main/Release/lldb-test-build.noindex/api/multithreaded/TestMultithreaded.test_breakpoint_callback'
/home/gclayton/local/llvm/main/Release/bin/clang  -std=c++11 -g -O0 -fno-builtin -m64  -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/../../../../../include -I/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make -include /data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/test_common.h  -fno-limit-debug-info  -std=c++11  -I/data/users/gclayton/llvm/main/llvm-project/lldb/include   --driver-mode=g++ -MT driver.o -MD -MP -MF driver.d -c -o driver.o driver.cpp
/home/gclayton/local/llvm/main/Release/bin/clang  -std=c++11 -g -O0 -fno-builtin -m64  -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/../../../../../include -I/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make -include /data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/test_common.h  -fno-limit-debug-info  -std=c++11  -I/data/users/gclayton/llvm/main/llvm-project/lldb/include   --driver-mode=g++ -MT test_breakpoint_callback.o -MD -MP -MF test_breakpoint_callback.d -c -o test_breakpoint_callback.o test_breakpoint_callback.cpp
/home/gclayton/local/llvm/main/Release/bin/clang  driver.o test_breakpoint_callback.o -g -O0 -fno-builtin -m64  -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/../../../../../include -I/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make -include /data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/test_common.h  -fno-limit-debug-info  -std=c++11  -I/data/users/gclayton/llvm/main/llvm-project/lldb/include -L/home/gclayton/local/llvm/main/Release/./lib -llldb -Wl,-rpath,/home/gclayton/local/llvm/main/Release/./lib  -pthread --driver-mode=g++ -o "test_breakpoint_callback"
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `std::_Sp_make_shared_tag::_S_eq(std::type_info const&)@GLIBCXX_3.4.26'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `log2@GLIBC_2.29'
//usr/local/fbcode/platform009/lib/librt.so.1: undefined reference to `__clock_getres@GLIBC_PRIVATE'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `exp2@GLIBC_2.29'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `log@GLIBC_2.29'
//usr/local/fbcode/platform009/lib/librt.so.1: undefined reference to `__clock_nanosleep@GLIBC_PRIVATE'
//usr/local/fbcode/platform009/lib/librt.so.1: undefined reference to `__clock_getcpuclockid@GLIBC_PRIVATE'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::basic_stringstream()@GLIBCXX_3.4.26'
//usr/local/fbcode/platform009/lib/librt.so.1: undefined reference to `__clock_settime@GLIBC_PRIVATE'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `exp@GLIBC_2.29'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `pow@GLIBC_2.29'
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile.rules:501: test_breakpoint_callback] Error 1
make: Leaving directory '/data/users/gclayton/llvm/main/Release/lldb-test-build.noindex/api/multithreaded/TestMultithreaded.test_breakpoint_callback'

Test Directory:
/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded
======================================================================
ERROR: test_python_stop_hook (TestMultithreaded.SBBreakpointCallbackCase)
   Test that you can run a python command in a stop-hook when stdin is File based.
----------------------------------------------------------------------
Error when building test subject.

Build Command:
make VPATH=/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -C /home/gclayton/local/llvm/main/Release/lldb-test-build.noindex/api/multithreaded/TestMultithreaded.test_python_stop_hook -I /data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -I /data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make -f /data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded/Makefile all  ARCH=x86_64 CC="/home/gclayton/local/llvm/main/Release/bin/clang"   CLANG_MODULE_CACHE_DIR=/home/gclayton/local/llvm/main/Release/lldb-test-build.noindex/module-cache-clang/lldb-api CXX_SOURCES='driver.cpp test_stop-hook.cpp' EXE='test_python_stop_hook' CFLAGS_EXTRAS='-std=c++11  -I/data/users/gclayton/llvm/main/llvm-project/lldb/include' LD_EXTRAS='-L/home/gclayton/local/llvm/main/Release/./lib -llldb -Wl,-rpath,/home/gclayton/local/llvm/main/Release/./lib'

Build Command Output:
make: Entering directory '/data/users/gclayton/llvm/main/Release/lldb-test-build.noindex/api/multithreaded/TestMultithreaded.test_python_stop_hook'
/home/gclayton/local/llvm/main/Release/bin/clang  -std=c++11 -g -O0 -fno-builtin -m64  -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/../../../../../include -I/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make -include /data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/test_common.h  -fno-limit-debug-info  -std=c++11  -I/data/users/gclayton/llvm/main/llvm-project/lldb/include   --driver-mode=g++ -MT driver.o -MD -MP -MF driver.d -c -o driver.o driver.cpp
/home/gclayton/local/llvm/main/Release/bin/clang  -std=c++11 -g -O0 -fno-builtin -m64  -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/../../../../../include -I/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make -include /data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/test_common.h  -fno-limit-debug-info  -std=c++11  -I/data/users/gclayton/llvm/main/llvm-project/lldb/include   --driver-mode=g++ -MT test_stop-hook.o -MD -MP -MF test_stop-hook.d -c -o test_stop-hook.o test_stop-hook.cpp
/home/gclayton/local/llvm/main/Release/bin/clang  driver.o test_stop-hook.o -g -O0 -fno-builtin -m64  -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/../../../../../include -I/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make -include /data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/test_common.h  -fno-limit-debug-info  -std=c++11  -I/data/users/gclayton/llvm/main/llvm-project/lldb/include -L/home/gclayton/local/llvm/main/Release/./lib -llldb -Wl,-rpath,/home/gclayton/local/llvm/main/Release/./lib  -pthread --driver-mode=g++ -o "test_python_stop_hook"
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `std::_Sp_make_shared_tag::_S_eq(std::type_info const&)@GLIBCXX_3.4.26'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `log2@GLIBC_2.29'
//usr/local/fbcode/platform009/lib/librt.so.1: undefined reference to `__clock_getres@GLIBC_PRIVATE'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `exp2@GLIBC_2.29'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `log@GLIBC_2.29'
//usr/local/fbcode/platform009/lib/librt.so.1: undefined reference to `__clock_nanosleep@GLIBC_PRIVATE'
//usr/local/fbcode/platform009/lib/librt.so.1: undefined reference to `__clock_getcpuclockid@GLIBC_PRIVATE'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::basic_stringstream()@GLIBCXX_3.4.26'
//usr/local/fbcode/platform009/lib/librt.so.1: undefined reference to `__clock_settime@GLIBC_PRIVATE'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `exp@GLIBC_2.29'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `pow@GLIBC_2.29'
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile.rules:501: test_python_stop_hook] Error 1
make: Leaving directory '/data/users/gclayton/llvm/main/Release/lldb-test-build.noindex/api/multithreaded/TestMultithreaded.test_python_stop_hook'

Test Directory:
/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded
======================================================================
ERROR: test_sb_api_listener_event_description (TestMultithreaded.SBBreakpointCallbackCase)
   Test the description of an SBListener breakpoint event is valid.
----------------------------------------------------------------------
Error when building test subject.

Build Command:
make VPATH=/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -C /home/gclayton/local/llvm/main/Release/lldb-test-build.noindex/api/multithreaded/TestMultithreaded.test_sb_api_listener_event_description -I /data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -I /data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make -f /data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded/Makefile all  ARCH=x86_64 CC="/home/gclayton/local/llvm/main/Release/bin/clang"   CLANG_MODULE_CACHE_DIR=/home/gclayton/local/llvm/main/Release/lldb-test-build.noindex/module-cache-clang/lldb-api CXX_SOURCES='driver.cpp listener_test.cpp test_listener_event_description.cpp' EXE='test_listener_event_description' CFLAGS_EXTRAS='-std=c++11  -I/data/users/gclayton/llvm/main/llvm-project/lldb/include' LD_EXTRAS='-L/home/gclayton/local/llvm/main/Release/./lib -llldb -Wl,-rpath,/home/gclayton/local/llvm/main/Release/./lib'

Build Command Output:
make: Entering directory '/data/users/gclayton/llvm/main/Release/lldb-test-build.noindex/api/multithreaded/TestMultithreaded.test_sb_api_listener_event_description'
/home/gclayton/local/llvm/main/Release/bin/clang  -std=c++11 -g -O0 -fno-builtin -m64  -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/../../../../../include -I/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make -include /data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/test_common.h  -fno-limit-debug-info  -std=c++11  -I/data/users/gclayton/llvm/main/llvm-project/lldb/include   --driver-mode=g++ -MT driver.o -MD -MP -MF driver.d -c -o driver.o driver.cpp
/home/gclayton/local/llvm/main/Release/bin/clang  -std=c++11 -g -O0 -fno-builtin -m64  -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/../../../../../include -I/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make -include /data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/test_common.h  -fno-limit-debug-info  -std=c++11  -I/data/users/gclayton/llvm/main/llvm-project/lldb/include   --driver-mode=g++ -MT listener_test.o -MD -MP -MF listener_test.d -c -o listener_test.o listener_test.cpp
/home/gclayton/local/llvm/main/Release/bin/clang  -std=c++11 -g -O0 -fno-builtin -m64  -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/../../../../../include -I/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make -include /data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/test_common.h  -fno-limit-debug-info  -std=c++11  -I/data/users/gclayton/llvm/main/llvm-project/lldb/include   --driver-mode=g++ -MT test_listener_event_description.o -MD -MP -MF test_listener_event_description.d -c -o test_listener_event_description.o test_listener_event_description.cpp
/home/gclayton/local/llvm/main/Release/bin/clang  driver.o listener_test.o test_listener_event_description.o -g -O0 -fno-builtin -m64  -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/../../../../../include -I/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make -include /data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/test_common.h  -fno-limit-debug-info  -std=c++11  -I/data/users/gclayton/llvm/main/llvm-project/lldb/include -L/home/gclayton/local/llvm/main/Release/./lib -llldb -Wl,-rpath,/home/gclayton/local/llvm/main/Release/./lib  -pthread --driver-mode=g++ -o "test_listener_event_description"
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `std::_Sp_make_shared_tag::_S_eq(std::type_info const&)@GLIBCXX_3.4.26'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `log2@GLIBC_2.29'
//usr/local/fbcode/platform009/lib/librt.so.1: undefined reference to `__clock_getres@GLIBC_PRIVATE'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `exp2@GLIBC_2.29'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `log@GLIBC_2.29'
//usr/local/fbcode/platform009/lib/librt.so.1: undefined reference to `__clock_nanosleep@GLIBC_PRIVATE'
//usr/local/fbcode/platform009/lib/librt.so.1: undefined reference to `__clock_getcpuclockid@GLIBC_PRIVATE'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::basic_stringstream()@GLIBCXX_3.4.26'
//usr/local/fbcode/platform009/lib/librt.so.1: undefined reference to `__clock_settime@GLIBC_PRIVATE'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `exp@GLIBC_2.29'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `pow@GLIBC_2.29'
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile.rules:501: test_listener_event_description] Error 1
make: Leaving directory '/data/users/gclayton/llvm/main/Release/lldb-test-build.noindex/api/multithreaded/TestMultithreaded.test_sb_api_listener_event_description'

Test Directory:
/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded
======================================================================
ERROR: test_sb_api_listener_event_process_state (TestMultithreaded.SBBreakpointCallbackCase)
   Test that a registered SBListener receives events when a process
----------------------------------------------------------------------
Error when building test subject.

Build Command:
make VPATH=/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -C /home/gclayton/local/llvm/main/Release/lldb-test-build.noindex/api/multithreaded/TestMultithreaded.test_sb_api_listener_event_process_state -I /data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -I /data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make -f /data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded/Makefile all  ARCH=x86_64 CC="/home/gclayton/local/llvm/main/Release/bin/clang"   CLANG_MODULE_CACHE_DIR=/home/gclayton/local/llvm/main/Release/lldb-test-build.noindex/module-cache-clang/lldb-api CXX_SOURCES='driver.cpp listener_test.cpp test_listener_event_process_state.cpp' EXE='test_listener_event_process_state' CFLAGS_EXTRAS='-std=c++11  -I/data/users/gclayton/llvm/main/llvm-project/lldb/include' LD_EXTRAS='-L/home/gclayton/local/llvm/main/Release/./lib -llldb -Wl,-rpath,/home/gclayton/local/llvm/main/Release/./lib'

Build Command Output:
make: Entering directory '/data/users/gclayton/llvm/main/Release/lldb-test-build.noindex/api/multithreaded/TestMultithreaded.test_sb_api_listener_event_process_state'
/home/gclayton/local/llvm/main/Release/bin/clang  -std=c++11 -g -O0 -fno-builtin -m64  -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/../../../../../include -I/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make -include /data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/test_common.h  -fno-limit-debug-info  -std=c++11  -I/data/users/gclayton/llvm/main/llvm-project/lldb/include   --driver-mode=g++ -MT driver.o -MD -MP -MF driver.d -c -o driver.o driver.cpp
/home/gclayton/local/llvm/main/Release/bin/clang  -std=c++11 -g -O0 -fno-builtin -m64  -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/../../../../../include -I/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make -include /data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/test_common.h  -fno-limit-debug-info  -std=c++11  -I/data/users/gclayton/llvm/main/llvm-project/lldb/include   --driver-mode=g++ -MT listener_test.o -MD -MP -MF listener_test.d -c -o listener_test.o listener_test.cpp
/home/gclayton/local/llvm/main/Release/bin/clang  -std=c++11 -g -O0 -fno-builtin -m64  -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/../../../../../include -I/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make -include /data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/test_common.h  -fno-limit-debug-info  -std=c++11  -I/data/users/gclayton/llvm/main/llvm-project/lldb/include   --driver-mode=g++ -MT test_listener_event_process_state.o -MD -MP -MF test_listener_event_process_state.d -c -o test_listener_event_process_state.o test_listener_event_process_state.cpp
/home/gclayton/local/llvm/main/Release/bin/clang  driver.o listener_test.o test_listener_event_process_state.o -g -O0 -fno-builtin -m64  -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/../../../../../include -I/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded -I/data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make -include /data/users/gclayton/llvm/main/llvm-project/lldb/packages/Python/lldbsuite/test/make/test_common.h  -fno-limit-debug-info  -std=c++11  -I/data/users/gclayton/llvm/main/llvm-project/lldb/include -L/home/gclayton/local/llvm/main/Release/./lib -llldb -Wl,-rpath,/home/gclayton/local/llvm/main/Release/./lib  -pthread --driver-mode=g++ -o "test_listener_event_process_state"
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `std::_Sp_make_shared_tag::_S_eq(std::type_info const&)@GLIBCXX_3.4.26'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `log2@GLIBC_2.29'
//usr/local/fbcode/platform009/lib/librt.so.1: undefined reference to `__clock_getres@GLIBC_PRIVATE'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `exp2@GLIBC_2.29'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `log@GLIBC_2.29'
//usr/local/fbcode/platform009/lib/librt.so.1: undefined reference to `__clock_nanosleep@GLIBC_PRIVATE'
//usr/local/fbcode/platform009/lib/librt.so.1: undefined reference to `__clock_getcpuclockid@GLIBC_PRIVATE'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::basic_stringstream()@GLIBCXX_3.4.26'
//usr/local/fbcode/platform009/lib/librt.so.1: undefined reference to `__clock_settime@GLIBC_PRIVATE'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `exp@GLIBC_2.29'
/home/gclayton/local/llvm/main/Release/./lib/liblldb.so: undefined reference to `pow@GLIBC_2.29'
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile.rules:501: test_listener_event_process_state] Error 1
make: Leaving directory '/data/users/gclayton/llvm/main/Release/lldb-test-build.noindex/api/multithreaded/TestMultithreaded.test_sb_api_listener_event_process_state'

Test Directory:
/data/users/gclayton/llvm/main/llvm-project/lldb/test/API/api/multithreaded
----------------------------------------------------------------------
Ran 5 tests in 14.773s

RESULT: FAILED (0 passes, 0 failures, 4 errors, 1 skipped, 0 expected failures, 0 unexpected successes)

--

********************
********************
Failed Tests (1):
  lldb-api :: api/multithreaded/TestMultithreaded.py


Testing Time: 15.03s
  Failed: 1

1 warning(s) in tests

[rdhindsa]

Thank you for giving it a try! It passes for me for both Release and RelWithDebInfo along with asserts enabled in each case. I can give it a try by recommitting and checking the status, this patch is in Needs Review status presently. Hence, requesting for a review.

[clayborg]

Lets try again with the buildbots. The only way I can think that this would happen is if we have rendezvous entries with no names that happens in other debug sessions and that is causing new files to be loaded that we don't expect. To find out you could disable your test, and then put an assert into the code to see if any normal debug sessions run during the full test suite run trigger the "find the path from the memory region info" trick we used.

[rdhindsa]

Thank you for suggestion to try!
After adding the assertions in memory region info trick, 3 testcases from only one test failed because of assertion : lldb/test/API/functionalities/load_unload/TestLoadUnload.py

test_lldb_process_load_and_unload_commands
test_load_unload
test_step_over_load

File path found for these test cases corresponds to the respective a.out files:
lldb-test-build.noindex/functionalities/load_unload/TestLoadUnload.test_lldb_process_load_and_unload_commands/a.out
lldb-test-build.noindex/functionalities/load_unload/TestLoadUnload.test_load_unload/a.out
lldb-test-build.noindex/functionalities/load_unload/TestLoadUnload.test_step_over_load/a.out

So this test seems to be working as expected with the patch. It passes if I remove the assertions. However, I didn't find any other test failing due to assertions. Even the multithreaded test passes with assertions.

[labath]

lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DYLDRendezvous.cpp

588–596

Setting this here defeats all the if (!SOEntryIsMainExecutable(entry)) checks in this code, for the "normal" case of running the executable directly. I'm guessing that will result in multiple modules being added for the main executable.

I think it would be better to remove this logic from here (it seems a bit too fancy for a function called ReadSOEntryFromMemory), and then teach the higher levels that they ought to treat "" specially (in some circumstances).

lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp

345–350

This check is unnecessary. A statically linked executable can still contain the dynamic linker structures (even though it does not have an elf interpreter), if it contains runtime calls to dlopen. In this case the main executable will contain _dl_debug_state et al. through the inclusion of libdl.a, and omitting this check should make that case work too.

[labath]

lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DYLDRendezvous.cpp

149

At this point, you can just make ResolveRendezvousAddress a (private) member of the DYLDRendezvous class.

314

Since the function already has IfNecessary in the name, you don't need to guard the call once more.

446–448

How about this:

// If were debugging ld.so, then all SOEntries should be treated as libraries, including the "main" one (denoted by an empty string). 
if (m_executable_interpreter)
  return false;
return !entry.file_spec;

543

I don't think this should make a difference, but if(!entry.file_spec) is shorter and consistent with the logic in SOEntryIsMainExecutable.

lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DYLDRendezvous.h

192

I get what you're trying to say, but I'm having issues with these comments, as they're wrong on multiple levels. For one, argv[0] essentially has no relationship to the binary being executed (ld.so even has an argument to supply an arbitrary string as argv[0] to the executable). But more importantly, when the executable is "run" in this way (as an argument to ld.so), as far as the operating system (and lldb) is concerned ld.so _is_ the main executable. You can see this in the /proc/$PID/exe symlink, or in all the target->GetExecutableModule calls in this patch. What this patch actually does is to _disable_ the logic of treating the "main executable" (the one with an empty name in the rendezvous structure) as special, so that it can be treated as one of the libraries.

With that in mind, I think it'd be better to say that this flag indicates whether the main program is the dynamic linker/loader/program interpreter.

lldb/test/API/functionalities/dyld-launch-linux/TestDyldLaunchLinux.py

19–25

It should be possible to retrieve the actual interpreter path by inspecting the compiled executable. Something like SBModule(exe)->FindSection(".interp")->GetSectionData() ought to do it.

[rdhindsa]

Thanks Pavel for investigating and figuring out that the upstream failing test had libxml support disabled. I was able to reproduce the failure locally by adding -DLLDB_ENABLE_LIBXML2=False to cmake.
Updated UpdateFileSpecIfNecessary function to only update filename if the region.GetName().AsCString() is not empty, which checks for both null and empty.

[labath]

Ok, let's give this one more shot.

Thanks.

[clayborg]

Thanks Pavel for figuring out the XML issue. I will need to add that to my box of tricks when buildbots fail! I look forward to seeing this patch go in

[rdhindsa]

@teemperor: I believe I have addressed your comments. If there is any other concern, I can address it post-commit.

Thank you everyone for the review and help with the patch!

[rdhindsa]

With this patch, it looks like the test added in this patch fails for lldb-arm-ubuntu:
https://lab.llvm.org/buildbot/#/builders/17/builds/10498

Everything passes on these buildbots: lldb-aarch64-ubuntu, lldb-x64-windows-ninja, lldb-x86_64-debian.

The test logic checks for breakpoint for main as well as for shared library load. It is able to extract that information using thread.GetFrameAtIndex(0). However, when it gets the signal, it is not able to extract the back trace and hence fails with the following error:

ERROR: test (TestDyldLaunchLinux.TestLinux64LaunchingViaDynamicLoader)
Traceback (most recent call last):
  File "/home/tcwg-buildslave/worker/lldb-arm-ubuntu/llvm-project/lldb/test/API/functionalities/dyld-launch-linux/TestDyldLaunchLinux.py", line 57, in test
    self.assertIn("raise", thread.GetFrameAtIndex(0).GetDisplayFunctionName())
  File "/home/tcwg-buildslave/worker/lldb-arm-ubuntu/llvm-project/lldb/third_party/Python/module/unittest2/unittest2/case.py", line 884, in assertIn
    if member not in container:
TypeError: argument of type 'NoneType' is not iterable
Config=arm-/home/tcwg-buildslave/worker/lldb-arm-ubuntu/build/bin/clang

@DavidSpickett, @zatrazz: If possible, could you please take a look. Is this expected behavior and test should be marked as unsupported for arm, or am I missing something here?

lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DYLDRendezvous.h

struct Rendezvous {
     lldb::addr_t map_addr = 0;
     lldb::addr_t brk = 0;
     uint64_t state = 0;
     lldb::addr_t ldbase = 0;
 
     Rendezvous() = default;
   };
 
+  /// Locates the address of the rendezvous structure.  It updates
+  /// m_executable_interpreter if address is extracted from _r_debug.
+  ///
+  /// \returns address on success and LLDB_INVALID_ADDRESS on failure.
+  lldb::addr_t ResolveRendezvousAddress();
+
 public:
   // Various metadata supplied by the inferior's threading library to describe
   // the per-thread state.
   struct ThreadInfo {
     bool valid;             // whether we read valid metadata
     uint32_t dtv_offset;    // offset of DTV pointer within pthread
     uint32_t dtv_slot_size; // size of one DTV slot
     uint32_t modid_offset;  // offset of module ID within link_map
protected:
   lldb_private::Process *m_process;
 
   // Cached copy of executable file spec
   lldb_private::FileSpec m_exe_file_spec;
 
   /// Location of the r_debug structure in the inferiors address space.
   lldb::addr_t m_rendezvous_addr;
 
+  // True if the main program is the dynamic linker/loader/program interpreter.
+  bool m_executable_interpreter;
+
   /// Current and previous snapshots of the rendezvous structure.
   Rendezvous m_current;
   Rendezvous m_previous;
 
   /// List of currently loaded SO modules
   LoadedModuleInfoList m_loaded_modules;
 
   /// List of SOEntry objects corresponding to the current link map state.
protected:
   bool RemoveSOEntriesFromRemote(const LoadedModuleInfoList &module_list);
 
   bool AddSOEntries();
 
   bool RemoveSOEntries();
 
   void UpdateBaseAddrIfNecessary(SOEntry &entry, std::string const &file_path);
 
+  void UpdateFileSpecIfNecessary(SOEntry &entry);
+
   bool SOEntryIsMainExecutable(const SOEntry &entry);
 
   /// Reads the current list of shared objects according to the link map
   /// supplied by the runtime linker.
   bool TakeSnapshot(SOEntryList &entry_list);
 
   enum PThreadField { eSize, eNElem, eOffset };
 

lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DYLDRendezvous.cpp

 
 #include "llvm/Support/Path.h"
 
 #include "DYLDRendezvous.h"
 
 using namespace lldb;
 using namespace lldb_private;
 
-/// Locates the address of the rendezvous structure.  Returns the address on
-/// success and LLDB_INVALID_ADDRESS on failure.
-static addr_t ResolveRendezvousAddress(Process *process) {
+DYLDRendezvous::DYLDRendezvous(Process *process)
+    : m_process(process), m_rendezvous_addr(LLDB_INVALID_ADDRESS),
+      m_executable_interpreter(false), m_current(), m_previous(),
+      m_loaded_modules(), m_soentries(), m_added_soentries(),
+      m_removed_soentries() {
+  m_thread_info.valid = false;
+  UpdateExecutablePath();
+}
+
+addr_t DYLDRendezvous::ResolveRendezvousAddress() {
   Log *log(GetLogIfAnyCategoriesSet(LIBLLDB_LOG_DYNAMIC_LOADER));
   addr_t info_location;
   addr_t info_addr;
   Status error;
 
-  if (!process) {
+  if (!m_process) {
     LLDB_LOGF(log, "%s null process provided", __FUNCTION__);
     return LLDB_INVALID_ADDRESS;
   }
 
   // Try to get it from our process.  This might be a remote process and might
   // grab it via some remote-specific mechanism.
-  info_location = process->GetImageInfoAddress();
+  info_location = m_process->GetImageInfoAddress();
   LLDB_LOGF(log, "%s info_location = 0x%" PRIx64, __FUNCTION__, info_location);
 
   // If the process fails to return an address, fall back to seeing if the
   // local object file can help us find it.
   if (info_location == LLDB_INVALID_ADDRESS) {
-    Target *target = &process->GetTarget();
+    Target *target = &m_process->GetTarget();
     if (target) {
       ObjectFile *obj_file = target->GetExecutableModule()->GetObjectFile();
       Address addr = obj_file->GetImageInfoAddress(target);
 
       if (addr.IsValid()) {
         info_location = addr.GetLoadAddress(target);
         LLDB_LOGF(log,
                   "%s resolved via direct object file approach to 0x%" PRIx64,
                   __FUNCTION__, info_location);
       } else {
+        const Symbol *_r_debug =
+            target->GetExecutableModule()->FindFirstSymbolWithNameAndType(
+                ConstString("_r_debug"));
+        if (_r_debug) {
+          info_addr = _r_debug->GetAddress().GetLoadAddress(target);
+          if (info_addr != LLDB_INVALID_ADDRESS) {
+            LLDB_LOGF(log,
+                      "%s resolved by finding symbol '_r_debug' whose value is "
+                      "0x%" PRIx64,
+                      __FUNCTION__, info_addr);
+            m_executable_interpreter = true;
+            return info_addr;
+          }
+        }
         LLDB_LOGF(log,
                   "%s FAILED - direct object file approach did not yield a "
                   "valid address",
                   __FUNCTION__);
       }
     }
   }
 
   if (info_location == LLDB_INVALID_ADDRESS) {
     LLDB_LOGF(log, "%s FAILED - invalid info address", __FUNCTION__);
     return LLDB_INVALID_ADDRESS;
   }
 
   LLDB_LOGF(log, "%s reading pointer (%" PRIu32 " bytes) from 0x%" PRIx64,
-            __FUNCTION__, process->GetAddressByteSize(), info_location);
+            __FUNCTION__, m_process->GetAddressByteSize(), info_location);
 
-  info_addr = process->ReadPointerFromMemory(info_location, error);
+  info_addr = m_process->ReadPointerFromMemory(info_location, error);
   if (error.Fail()) {
     LLDB_LOGF(log, "%s FAILED - could not read from the info location: %s",
               __FUNCTION__, error.AsCString());
     return LLDB_INVALID_ADDRESS;
   }
 
   if (info_addr == 0) {
     LLDB_LOGF(log,
               "%s FAILED - the rendezvous address contained at 0x%" PRIx64
               " returned a null value",
               __FUNCTION__, info_location);
     return LLDB_INVALID_ADDRESS;
   }
 
   return info_addr;
 }
 
-DYLDRendezvous::DYLDRendezvous(Process *process)
-    : m_process(process), m_rendezvous_addr(LLDB_INVALID_ADDRESS), m_current(),
-      m_previous(), m_loaded_modules(), m_soentries(), m_added_soentries(),
-      m_removed_soentries() {
-  m_thread_info.valid = false;
-  UpdateExecutablePath();
-}
-
 void DYLDRendezvous::UpdateExecutablePath() {
   if (m_process) {
     Log *log(GetLogIfAnyCategoriesSet(LIBLLDB_LOG_DYNAMIC_LOADER));
     Module *exe_mod = m_process->GetTarget().GetExecutableModulePointer();
     if (exe_mod) {
       m_exe_file_spec = exe_mod->GetPlatformFileSpec();
       LLDB_LOGF(log, "DYLDRendezvous::%s exe module executable path set: '%s'",
                 __FUNCTION__, m_exe_file_spec.GetCString());
bool DYLDRendezvous::Resolve() {
 
   address_size = m_process->GetAddressByteSize();
   padding = address_size - word_size;
   LLDB_LOGF(log,
             "DYLDRendezvous::%s address size: %" PRIu64 ", padding %" PRIu64,
             __FUNCTION__, uint64_t(address_size), uint64_t(padding));
 
   if (m_rendezvous_addr == LLDB_INVALID_ADDRESS)
-    cursor = info_addr = ResolveRendezvousAddress(m_process);
+    cursor = info_addr =
+        ResolveRendezvousAddress();
   else
     cursor = info_addr = m_rendezvous_addr;
   LLDB_LOGF(log, "DYLDRendezvous::%s cursor = 0x%" PRIx64, __FUNCTION__,
             cursor);
 
   if (cursor == LLDB_INVALID_ADDRESS)
     return false;
 
if (!modInfo.get_link_map(link_map_addr) || !modInfo.get_base(base_addr) ||
       !modInfo.get_dynamic(dyn_addr) || !modInfo.get_name(name))
     return false;
 
   entry.link_addr = link_map_addr;
   entry.base_addr = base_addr;
   entry.dyn_addr = dyn_addr;
 
   entry.file_spec.SetFile(name, FileSpec::Style::native);
 
   UpdateBaseAddrIfNecessary(entry, name);
 
   // not needed if we're using ModuleInfos
   entry.next = 0;
   entry.prev = 0;
   entry.path_addr = 0;
 
   return true;
 }
 
 bool DYLDRendezvous::SaveSOEntriesFromRemote(
     const LoadedModuleInfoList &module_list) {
   for (auto const &modInfo : module_list.m_list) {
     SOEntry entry;
     if (!FillSOEntryFromModuleInfo(modInfo, entry))
       return false;
 
     // Only add shared libraries and not the executable.
-    if (!SOEntryIsMainExecutable(entry))
+    if (!SOEntryIsMainExecutable(entry)) {
+      UpdateFileSpecIfNecessary(entry);
       m_soentries.push_back(entry);
     }
+  }
 
   m_loaded_modules = module_list;
   return true;
 }
 
 bool DYLDRendezvous::AddSOEntriesFromRemote(
     const LoadedModuleInfoList &module_list) {
   for (auto const &modInfo : module_list.m_list) {
if (found)
       continue;
 
     SOEntry entry;
     if (!FillSOEntryFromModuleInfo(modInfo, entry))
       return false;
 
     // Only add shared libraries and not the executable.
     if (!SOEntryIsMainExecutable(entry)) {
+      UpdateFileSpecIfNecessary(entry);
       m_soentries.push_back(entry);
       m_added_soentries.push_back(entry);
     }
   }
 
   m_loaded_modules = module_list;
   return true;
 }
bool DYLDRendezvous::AddSOEntries() {
   for (addr_t cursor = m_current.map_addr; cursor != 0; cursor = entry.next) {
     if (!ReadSOEntryFromMemory(cursor, entry))
       return false;
 
     // Only add shared libraries and not the executable.
     if (SOEntryIsMainExecutable(entry))
       continue;
 
+    UpdateFileSpecIfNecessary(entry);
+
     pos = std::find(m_soentries.begin(), m_soentries.end(), entry);
     if (pos == m_soentries.end()) {
       m_soentries.push_back(entry);
       m_added_soentries.push_back(entry);
     }
   }
 
   return true;
bool DYLDRendezvous::SOEntryIsMainExecutable(const SOEntry &entry) {
   auto triple = m_process->GetTarget().GetArchitecture().GetTriple();
   switch (triple.getOS()) {
   case llvm::Triple::FreeBSD:
   case llvm::Triple::NetBSD:
     return entry.file_spec == m_exe_file_spec;
   case llvm::Triple::Linux:
     if (triple.isAndroid())
       return entry.file_spec == m_exe_file_spec;
+    // If we are debugging ld.so, then all SOEntries should be treated as
+    // libraries, including the "main" one (denoted by an empty string).
+    if (!entry.file_spec && m_executable_interpreter)
+      return false;
     return !entry.file_spec;
   default:
     return false;
   }
 }
 
 bool DYLDRendezvous::TakeSnapshot(SOEntryList &entry_list) {
   SOEntry entry;
 
   if (m_current.map_addr == 0)
     return false;
 
   // Clear previous entries since we are about to obtain an up to date list.
   entry_list.clear();
 
   for (addr_t cursor = m_current.map_addr; cursor != 0; cursor = entry.next) {
     if (!ReadSOEntryFromMemory(cursor, entry))
       return false;
 
     // Only add shared libraries and not the executable.
     if (SOEntryIsMainExecutable(entry))
       continue;
 
+    UpdateFileSpecIfNecessary(entry);
+
     entry_list.push_back(entry);
   }
 
   return true;
 }
 
 addr_t DYLDRendezvous::ReadWord(addr_t addr, uint64_t *dst, size_t size) {
   Status error;
if (isLoadBiasIncorrect(m_process->GetTarget(), file_path)) {
     bool is_loaded = false;
     Status error =
         m_process->GetFileLoadAddress(entry.file_spec, is_loaded, load_addr);
     if (error.Success() && is_loaded)
       entry.base_addr = load_addr;
   }
 }
 
+void DYLDRendezvous::UpdateFileSpecIfNecessary(SOEntry &entry) {
+  // Updates filename if empty. It is useful while debugging ld.so,
+  // when the link map returns empty string for the main executable.
+  if (!entry.file_spec) {
+    MemoryRegionInfo region;
+    Status region_status =
+        m_process->GetMemoryRegionInfo(entry.dyn_addr, region);
+    if (!region.GetName().IsEmpty())
+      entry.file_spec.SetFile(region.GetName().AsCString(),
+                              FileSpec::Style::native);
+  }
+}
+
 bool DYLDRendezvous::ReadSOEntryFromMemory(lldb::addr_t addr, SOEntry &entry) {
   entry.clear();
 
   entry.link_addr = addr;
 
   if (!(addr = ReadPointer(addr, &entry.base_addr)))
     return false;
 
bool DYLDRendezvous::ReadSOEntryFromMemory(lldb::addr_t addr, SOEntry &entry) {
 
   if (!(addr = ReadPointer(addr, &entry.next)))
     return false;
 
   if (!(addr = ReadPointer(addr, &entry.prev)))
     return false;
 
   std::string file_path = ReadStringFromMemory(entry.path_addr);
   entry.file_spec.SetFile(file_path, FileSpec::Style::native);
 
   UpdateBaseAddrIfNecessary(entry, file_path);
 
   return true;
 }
 
 bool DYLDRendezvous::FindMetadata(const char *name, PThreadField field,
                                   uint32_t &value) {
   Target &target = m_process->GetTarget();
 
   SymbolContextList list;
   target.GetImages().FindSymbolsWithNameAndType(ConstString(name),
                                                 eSymbolTypeAny, list);
   if (list.IsEmpty())
   return false;
 

lldb/source/Plugins/DynamicLoader/POSIX-DYLD/DynamicLoaderPOSIXDYLD.cpp

if (m_rendezvous.IsValid()) {
     LLDB_LOG(log, "Setting rendezvous break address for pid {0} at {1:x}",
              m_process ? m_process->GetID() : LLDB_INVALID_PROCESS_ID,
              break_addr);
     dyld_break = target.CreateBreakpoint(break_addr, true, false);
   } else {
     LLDB_LOG(log, "Rendezvous structure is not set up yet. "
                   "Trying to locate rendezvous breakpoint in the interpreter "
                   "by symbol name.");
-    ModuleSP interpreter = LoadInterpreterModule();
-    if (!interpreter) {
-      LLDB_LOG(log, "Can't find interpreter, rendezvous breakpoint isn't set.");
-      return false;
-    }
-
-    // Function names from different dynamic loaders that are known to be used
-    // as rendezvous between the loader and debuggers.
+    // Function names from different dynamic loaders that are known to be
+    // used as rendezvous between the loader and debuggers.
     static std::vector<std::string> DebugStateCandidates{
         "_dl_debug_state", "rtld_db_dlactivity", "__dl_rtld_db_dlactivity",
         "r_debug_state",   "_r_debug_state",     "_rtld_debug_state",
     };
 
+    ModuleSP interpreter = LoadInterpreterModule();
+    if (!interpreter) {
+      FileSpecList containingModules;
+      containingModules.Append(
+          m_process->GetTarget().GetExecutableModulePointer()->GetFileSpec());
+
+      dyld_break = target.CreateBreakpoint(
+          &containingModules, /*containingSourceFiles=*/nullptr,
+          DebugStateCandidates, eFunctionNameTypeFull, eLanguageTypeC,
+          /*offset=*/0,
+          /*skip_prologue=*/eLazyBoolNo,
+          /*internal=*/true,
+          /*request_hardware=*/false);
+    } else {
       FileSpecList containingModules;
       containingModules.Append(interpreter->GetFileSpec());
       dyld_break = target.CreateBreakpoint(
-        &containingModules, nullptr /* containingSourceFiles */,
+          &containingModules, /*containingSourceFiles=*/nullptr,
           DebugStateCandidates, eFunctionNameTypeFull, eLanguageTypeC,
-        0,           /* offset */
-        eLazyBoolNo, /* skip_prologue */
-        true,        /* internal */
-        false /* request_hardware */);
+          /*offset=*/0,
+          /*skip_prologue=*/eLazyBoolNo,
+          /*internal=*/true,
+          /*request_hardware=*/false);
+    }
   }
 
   if (dyld_break->GetNumResolvedLocations() != 1) {
     LLDB_LOG(
         log,
         "Rendezvous breakpoint has abnormal number of"
         " resolved locations ({0}) in pid {1}. It's supposed to be exactly 1.",
         dyld_break->GetNumResolvedLocations(),

lldb/test/API/functionalities/dyld-launch-linux/Makefile

+CXX_SOURCES := main.cpp
+DYLIB_NAME := signal_file
+DYLIB_CXX_SOURCES := signal_file.cpp
+include Makefile.rules

lldb/test/API/functionalities/dyld-launch-linux/TestDyldLaunchLinux.py

+"""
+Test that LLDB can launch a linux executable through the dynamic loader and still hit a breakpoint.
+"""
+
+import lldb
+import os
+
+from lldbsuite.test.decorators import *
+from lldbsuite.test.lldbtest import *
+
+class TestLinux64LaunchingViaDynamicLoader(TestBase):
+    mydir = TestBase.compute_mydir(__file__)
+
+    @skipIf(oslist=no_match(['linux']))
+    @no_debug_info_test
+    def test(self):
+        self.build()
+
+        # Extracts path of the interpreter.
+        spec = lldb.SBModuleSpec()
+        spec.SetFileSpec(lldb.SBFileSpec(self.getBuildArtifact("a.out")))
+        interp_section = lldb.SBModule(spec).FindSection(".interp")
+        if not interp_section:
+          return
+        section_data = interp_section.GetSectionData()
+        error = lldb.SBError()
+        exe = section_data.GetString(error,0)
+        if error.Fail():
+          return
+
+        target = self.dbg.CreateTarget(exe)
+        self.assertTrue(target, VALID_TARGET)
+
+        # Set breakpoints both on shared library function as well as on
+        # main. Both of them will be pending breakpoints.
+        breakpoint_main = target.BreakpointCreateBySourceRegex("// Break here", lldb.SBFileSpec("main.cpp"))
+        breakpoint_shared_library = target.BreakpointCreateBySourceRegex("get_signal_crash", lldb.SBFileSpec("signal_file.cpp"))
+        launch_info = lldb.SBLaunchInfo([ "--library-path", self.get_process_working_directory(), self.getBuildArtifact("a.out")])
+        launch_info.SetWorkingDirectory(self.get_process_working_directory())
+        error = lldb.SBError()
+        process = target.Launch(launch_info, error)
+        self.assertTrue(error.Success())
+
+        # Stopped on main here.
+        self.assertEqual(process.GetState(), lldb.eStateStopped)
+        thread = process.GetSelectedThread()
+        self.assertIn("main", thread.GetFrameAtIndex(0).GetDisplayFunctionName())
+        process.Continue()
+
+        # Stopped on get_signal_crash function here.
+        self.assertEqual(process.GetState(), lldb.eStateStopped)
+        self.assertIn("get_signal_crash", thread.GetFrameAtIndex(0).GetDisplayFunctionName())
+        process.Continue()
+
+        # Stopped because of generated signal.
+        self.assertEqual(process.GetState(), lldb.eStateStopped)
+        self.assertIn("raise", thread.GetFrameAtIndex(0).GetDisplayFunctionName())
+        self.assertIn("get_signal_crash", thread.GetFrameAtIndex(1).GetDisplayFunctionName())

lldb/test/API/functionalities/dyld-launch-linux/main.cpp

+#include "signal_file.h"
+
+int main() {
+  // Break here
+  return get_signal_crash();
+}

lldb/test/API/functionalities/dyld-launch-linux/signal_file.h

+int get_signal_crash();

lldb/test/API/functionalities/dyld-launch-linux/signal_file.cpp

+#include "signal_file.h"
+#include <signal.h>
+
+int get_signal_crash(void) {
+  raise(SIGSEGV);
+  return 0;
+}