This is an archive of the discontinued LLVM Phabricator instance.

[lli] Test debug support in RuntimeDyld with built-in functions
ClosedPublic

Authored by sgraenitz on Mar 1 2021, 8:38 AM.

Details

Summary

When lli runs the below IR, it emits in-memory debug objects and registers them with the GDB JIT interface. The tests dump and check the registered information. IR has limited ability to produce complex output in a portable way. Instead the tests rely on built-in functions implemented in lli. They use a new command line flag -generate=function-name to instruct the ORC JIT to expose the built-in function with the given name to the JITed program.

debug-descriptor-elf-minimal.ll calls __dump_jit_debug_descriptor() to reflect the list of debug entries issued for itself after emitting the main module. The output is textual and can be checked straight away.

debug-objects-elf-minimal.ll calls __dump_jit_debug_objects(), which instructs lli to walk through the list of debug entries and append the encountered in-memory objects to the program output. We feed this output into llvm-dwarfdump to parse the DWARF in each file and dump their structures.

We can do the same for JITLink once D97335 has landed.

Diff Detail

Event Timeline

sgraenitz created this revision.Mar 1 2021, 8:38 AM
sgraenitz requested review of this revision.Mar 1 2021, 8:38 AM
Herald added a project: Restricted Project. · View Herald Transcript
Herald added a subscriber: sstefan1. · View Herald Transcript

I can imagine tests like these will be handy for testing advanced JIT features inside LLVM in the future. Adding the -generate flag with a hard-coded set of options was not my preferred solution, but all other attempts failed for portability reasons. I am happy to discuss ideas how to improve it.

llvm/tools/lli/ExecutionUtils.cpp
92

This global static pointer is working for the moment, but of course it's not exactly ideal. I need a better way to expose a member function callback. TPCIndirectionUtils offers writeResolverBlock(), but it only works with TPC. LocalTrampolinePool also uses resolver blocks, but I don't need the landing address resolution.

@lhames Can you think of a solution that also works for RuntimeDyld?

lhames accepted this revision.Mar 1 2021, 5:50 PM

I can imagine tests like these will be handy for testing advanced JIT features inside LLVM in the future. Adding the -generate flag with a hard-coded set of options was not my preferred solution, but all other attempts failed for portability reasons. I am happy to discuss ideas how to improve it.

What other options did you consider?
How about a list of strings, rather than hard-coded options?
Or writing __dump_jit_debug_descriptor and __dump_jit_debug_objects as extern "C" functions and dispensing with the generator entirely (they'd be picked up as process symbols)?

I think you can land this as-is and we can revise the design later if we come up with a better approach.

This revision is now accepted and ready to land.Mar 1 2021, 5:50 PM

I can imagine tests like these will be handy for testing advanced JIT features inside LLVM in the future. Adding the -generate flag with a hard-coded set of options was not my preferred solution, but all other attempts failed for portability reasons. I am happy to discuss ideas how to improve it.

What other options did you consider?

In my first approach in D97337 I put all the code in IR, but I didn't get the dumping via printf to work cross platform. I considered platform-specific versions, but they appear a lot less attractive, because they add much extra work and they would be restricted to test only their native object formats. In a second approach, I moved the dumping code to an external C file and compiled it to the LLVM build-tree with the native toolchain. The idea was to add it to the execution session with the -extra-objects command line option. Unfortunately, it shows the same limitation as the "IR in platform-specific versions" approach. Due to mangling differences it's restricted to its native object format: leading underscore with MachO, MSVC mangling with COFF (and in lookups ORC assumes mangling to be identical across all translation units). As a result, the lookup of printf failed (for ELF on MachO e.g. it was looking for _printf).

How about a list of strings, rather than hard-coded options?
Or writing __dump_jit_debug_descriptor and __dump_jit_debug_objects as extern "C" functions and dispensing with the generator entirely (they'd be picked up as process symbols)?

I considered that, but as they are built-ins, they would still be hard-coded eventually. For the reader it would obfuscate the connection between the command line argument and the implementation of a built-in. The user couldn't lookup valid argument values anymore (via lli --help-hidden). Also, __dump_jit_debug_objects has an extra requirement: it needs to create a tool output stream to append the debug objects. With a static function it required a global static pointer in the same way as I use it now for the generator.

I think you can land this as-is and we can revise the design later if we come up with a better approach.

Yes let's do that. It's all in the lli tool and not in libraries, so there is no risk to break client code.

This revision was landed with ongoing or failed builds.Mar 2 2021, 1:39 AM
This revision was automatically updated to reflect the committed changes.

This test is causing a segfault on 32-bit x86: https://bugs.llvm.org/show_bug.cgi?id=51292

Could you take a look, please?

Yes, I can make some time for it next week. Do we have a bot that builds the relevant configuration? Otherwise, is it straightforward for you to reproduce on Linux with LLVM_BUILD_32_BITS=ON? Or does it take any extra steps/settings?

For the moment D107640 limits the debug object tests to 64-bit processes on x86_64 hosts. This is what the debug plugin and tests were made for.