[docs] Update DebuggingJITedCode page after fix in LLDB

[sgraenitz]

Summary

Generalize the documentation to include both, GDB and LLDB. Add a link to the interface
definition. Make a note on MCJIT's restriction to ELF. Mention the regression and bugfix
in LLDB as well as the jit-loader setting for macOS. Update the command line session to
use LLDB instead of GDB.

[sgraenitz]

Preview for the update: https://weliveindetail.github.io/docs-pub/html/DebuggingJITedCode.html

[sgraenitz]

LLDB regression https://bugs.llvm.org/show_bug.cgi?id=36209 fixed with D61611 and D90769

[anarazel]

It's not really the fault of this change, but the gdb support isn't really restricted to mcjit. registerJITEventListener() works with RTDyld based Orc too.

[sgraenitz]

Thanks for having a look.

That's right and it's on my wishlist. However, the example doesn't actually work with --jit-kind=orc-lazy (yet). On Linux it should use RTDyldObjectLinkingLayer and so it should notify LLDB when new code is loaded, but somehow it doesn't:

(lldb) run --jit-kind=orc-lazy showdebug.ll 5
Process 25021 launched: '/usr/bin/lli' (x86_64)
Process 25021 exited with status = 120 (0x00000078)

So far I haven't analyzed it in detail. My plan was to finish the ongoing work, update the docs, close PR36209 (finally) and schedule follow-up tasks once I find the time.

[sgraenitz]

One issue with the orc-lazy JIT will be symbol renaming for lazy reexports: Upfront we only emit stubs for our function symbols and they trigger compilation of the actual code in a separate logical unit once they are called. The debug object we hand over to LLDB doesn't take that into account. (Maybe we are just inserting the wrong load address? Don't know.) Additionally, I am not sure how debug data is handled in the partitioning logic of the CompileOnDemandLayer.

If we had a orc-greedymode in lli, we could easily demonstrate debugging here (for RTDyld-based platforms) and we could also port quite a number of tests to OrcV2. Might be worth thinking about, but so far we have MCJIT for that and it still works.

Not sure any of this is worth noting in the docs?

[sgraenitz]

Does anyone have objections to this documentation update? Preview is here still:
https://weliveindetail.github.io/docs-pub/html/DebuggingJITedCode.html

[sgraenitz]

I'd like to land this before release 12 branches. If you have any objections, please speak up before the end of the week. Thanks.

[lhames]

Belatedly -- LGTM.

I think an orc-greedy mode for LLI sounds like a great idea. Also, once we have it we should probably make it the default mode instead of MCJIT.

[sgraenitz]

Thanks! Eventually, I closed https://llvm.org/PR36209

llvm/docs/DebuggingJITedCode.rst

-==============================
-Debugging JIT-ed Code With GDB
-==============================
+=====================
+Debugging JIT-ed Code
+=====================
 
 Background
 ==========
 
-Without special runtime support, debugging dynamically generated code with
-GDB (as well as most debuggers) can be quite painful.  Debuggers generally
-read debug information from the object file of the code, but for JITed
-code, there is no such file to look for.
-
-In order to communicate the necessary debug info to GDB, an interface for
-registering JITed code with debuggers has been designed and implemented for
-GDB and LLVM MCJIT.  At a high level, whenever MCJIT generates new machine code,
-it does so in an in-memory object file that contains the debug information in
-DWARF format.  MCJIT then adds this in-memory object file to a global list of
-dynamically generated object files and calls a special function
-(``__jit_debug_register_code``) marked noinline that GDB knows about.  When
-GDB attaches to a process, it puts a breakpoint in this function and loads all
-of the object files in the global list.  When MCJIT calls the registration
-function, GDB catches the breakpoint signal, loads the new object file from
-the inferior's memory, and resumes the execution.  In this way, GDB can get the
-necessary debug information.
+Without special runtime support, debugging dynamically generated code can be
+quite painful.  Debuggers generally read debug information from object files on
+disk, but for JITed code there is no such file to look for.
+
+In order to hand over the necessary debug info, `GDB established an
+interface <https://sourceware.org/gdb/current/onlinedocs/gdb/JIT-Interface.html>`_
+for registering JITed code with debuggers. LLDB implements it in the
+JITLoaderGDB plugin.  On the JIT side, LLVM MCJIT does implement the interface
+for ELF object files.
+
+At a high level, whenever MCJIT generates new machine code, it does so in an
+in-memory object file that contains the debug information in DWARF format.
+MCJIT then adds this in-memory object file to a global list of dynamically
+generated object files and calls a special function
+``__jit_debug_register_code`` that the debugger knows about. When the debugger
+attaches to a process, it puts a breakpoint in this function and associates a
+special handler with it.  Once MCJIT calls the registration function, the
+debugger catches the breakpoint signal, loads the new object file from the
+inferior's memory and resumes execution.  This way it can obtain debug
+information for pure in-memory object files.
+
 
 GDB Version
 ===========
 
 In order to debug code JIT-ed by LLVM, you need GDB 7.0 or newer, which is
 available on most modern distributions of Linux.  The version of GDB that
-Apple ships with Xcode has been frozen at 6.3 for a while.  LLDB may be a
-better option for debugging JIT-ed code on macOS.
+Apple ships with Xcode has been frozen at 6.3 for a while.
+
+
+LLDB Version
+============
+
+Due to a regression in release 6.0, LLDB didn't support JITed code debugging for
+a while.  The bug was fixed in mainline recently, so that debugging JITed ELF
+objects should be possible again from the upcoming release 12.0 on. On macOS the
+feature must be enabled explicitly using the ``plugin.jit-loader.gdb.enable``
+setting.
 
 
 Debugging MCJIT-ed code
 =======================
 
 The emerging MCJIT component of LLVM allows full debugging of JIT-ed code with
 GDB.  This is due to MCJIT's ability to use the MC emitter to provide full
 DWARF debugging information to GDB.
 
-Note that lli has to be passed the ``-jit-kind=mcjit`` flag to JIT the code with
-MCJIT instead of the old JIT.
+Note that lli has to be passed the ``--jit-kind=mcjit`` flag to JIT the code
+with MCJIT instead of the newer ORC JIT.
 
 Example
 -------
 
 Consider the following C code (with line numbers added to make the example
 easier to follow):
 
 ..
.. code-block:: c
    17      char firstletter = argv[1][0];
    18      int result = compute_factorial(firstletter - '0');
    19
    20      // Returned result is clipped at 255...
    21      return result;
    22  }
 
 Here is a sample command line session that shows how to build and run this
-code via ``lli`` inside GDB:
+code via ``lli`` inside LLDB:
 
 .. code-block:: bash
 
-   $ $BINPATH/clang -cc1 -O0 -g -emit-llvm showdebug.c
-   $ gdb --quiet --args $BINPATH/lli -jit-kind=mcjit showdebug.ll 5
-   Reading symbols from $BINPATH/lli...done.
-   (gdb) b showdebug.c:6
-   No source file named showdebug.c.
-   Make breakpoint pending on future shared library load? (y or [n]) y
-   Breakpoint 1 (showdebug.c:6) pending.
-   (gdb) r
-   Starting program: $BINPATH/lli -jit-kind=mcjit showdebug.ll 5
-   [Thread debugging using libthread_db enabled]
-
-   Breakpoint 1, compute_factorial (n=5) at showdebug.c:6
-   6      int f = n;
-   (gdb) p n
-   $1 = 5
-   (gdb) p f
-   $2 = 0
-   (gdb) n
-   7      while (--n > 1)
-   (gdb) p f
-   $3 = 5
-   (gdb) b showdebug.c:9
-   Breakpoint 2 at 0x7ffff7ed404c: file showdebug.c, line 9.
-   (gdb) c
-   Continuing.
-
-   Breakpoint 2, compute_factorial (n=1) at showdebug.c:9
-   9      return f;
-   (gdb) p f
-   $4 = 120
-   (gdb) bt
-   #0  compute_factorial (n=1) at showdebug.c:9
-   #1  0x00007ffff7ed40a9 in main (argc=2, argv=0x16677e0) at showdebug.c:18
-   #2  0x3500000001652748 in ?? ()
-   #3  0x00000000016677e0 in ?? ()
-   #4  0x0000000000000002 in ?? ()
-   #5  0x0000000000d953b3 in llvm::MCJIT::runFunction (this=0x16151f0, F=0x1603020, ArgValues=...) at /home/ebenders_test/llvm_svn_rw/lib/ExecutionEngine/MCJIT/MCJIT.cpp:161
-   #6  0x0000000000dc8872 in llvm::ExecutionEngine::runFunctionAsMain (this=0x16151f0, Fn=0x1603020, argv=..., envp=0x7fffffffe040)
-       at /home/ebenders_test/llvm_svn_rw/lib/ExecutionEngine/ExecutionEngine.cpp:397
-   #7  0x000000000059c583 in main (argc=4, argv=0x7fffffffe018, envp=0x7fffffffe040) at /home/ebenders_test/llvm_svn_rw/tools/lli/lli.cpp:324
-   (gdb) finish
-   Run till exit from #0  compute_factorial (n=1) at showdebug.c:9
-   0x00007ffff7ed40a9 in main (argc=2, argv=0x16677e0) at showdebug.c:18
-   18      int result = compute_factorial(firstletter - '0');
-   Value returned is $5 = 120
-   (gdb) p result
-   $6 = 23406408
-   (gdb) n
-   21      return result;
-   (gdb) p result
-   $7 = 120
-   (gdb) c
-   Continuing.
-
-   Program exited with code 0170.
-   (gdb)
+   > export BINPATH=/workspaces/llvm-project/build/bin
+   > $BINPATH/clang -g -S -emit-llvm --target=x86_64-unknown-unknown-elf showdebug.c
+   > lldb $BINPATH/lli
+   (lldb) target create "/workspaces/llvm-project/build/bin/lli"
+   Current executable set to '/workspaces/llvm-project/build/bin/lli' (x86_64).
+   (lldb) settings set plugin.jit-loader.gdb.enable on
+   (lldb) b compute_factorial
+   Breakpoint 1: no locations (pending).
+   WARNING:  Unable to resolve breakpoint to any actual locations.
+   (lldb) run --jit-kind=mcjit showdebug.ll 5
+   1 location added to breakpoint 1
+   Process 21340 stopped
+   * thread #1, name = 'lli', stop reason = breakpoint 1.1
+      frame #0: 0x00007ffff7fd0007 JIT(0x45c2cb0)`compute_factorial(n=5) at showdebug.c:3:11
+      1    int compute_factorial(int n)
+      2    {
+   -> 3        if (n <= 1)
+      4            return 1;
+      5        int f = n;
+      6        while (--n > 1)
+      7            f *= n;
+   (lldb) p n
+   (int) $0 = 5
+   (lldb) b showdebug.c:9
+   Breakpoint 2: where = JIT(0x45c2cb0)`compute_factorial + 60 at showdebug.c:9:1, address = 0x00007ffff7fd003c
+   (lldb) c
+   Process 21340 resuming
+   Process 21340 stopped
+   * thread #1, name = 'lli', stop reason = breakpoint 2.1
+      frame #0: 0x00007ffff7fd003c JIT(0x45c2cb0)`compute_factorial(n=1) at showdebug.c:9:1
+      6        while (--n > 1)
+      7            f *= n;
+      8        return f;
+   -> 9    }
+      10  
+      11   int main(int argc, char** argv)
+      12   {
+   (lldb) p f
+   (int) $1 = 120
+   (lldb) bt
+   * thread #1, name = 'lli', stop reason = breakpoint 2.1
+   * frame #0: 0x00007ffff7fd003c JIT(0x45c2cb0)`compute_factorial(n=1) at showdebug.c:9:1
+      frame #1: 0x00007ffff7fd0095 JIT(0x45c2cb0)`main(argc=2, argv=0x00000000046122f0) at showdebug.c:16:18
+      frame #2: 0x0000000002a8306e lli`llvm::MCJIT::runFunction(this=0x000000000458ed10, F=0x0000000004589ff8, ArgValues=ArrayRef<llvm::GenericValue> @ 0x00007fffffffc798) at MCJIT.cpp:554:31
+      frame #3: 0x00000000029bdb45 lli`llvm::ExecutionEngine::runFunctionAsMain(this=0x000000000458ed10, Fn=0x0000000004589ff8, argv=size=0, envp=0x00007fffffffe140) at ExecutionEngine.cpp:467:10
+      frame #4: 0x0000000001f2fc2f lli`main(argc=4, argv=0x00007fffffffe118, envp=0x00007fffffffe140) at lli.cpp:643:18
+      frame #5: 0x00007ffff788c09b libc.so.6`__libc_start_main(main=(lli`main at lli.cpp:387), argc=4, argv=0x00007fffffffe118, init=<unavailable>, fini=<unavailable>, rtld_fini=<unavailable>, stack_end=0x00007fffffffe108) at libc-start.c:308:16
+      frame #6: 0x0000000001f2dc7a lli`_start + 42
+   (lldb) finish
+   Process 21340 stopped
+   * thread #1, name = 'lli', stop reason = step out
+   Return value: (int) $2 = 120
+
+      frame #0: 0x00007ffff7fd0095 JIT(0x45c2cb0)`main(argc=2, argv=0x00000000046122f0) at showdebug.c:16:9
+      13       if (argc < 2)
+      14           return -1;
+      15       char firstletter = argv[1][0];
+   -> 16       int result = compute_factorial(firstletter - '0');
+      17  
+      18       // Returned result is clipped at 255...
+      19       return result;
+   (lldb) p result
+   (int) $3 = 73670648
+   (lldb) n
+   Process 21340 stopped
+   * thread #1, name = 'lli', stop reason = step over
+      frame #0: 0x00007ffff7fd0098 JIT(0x45c2cb0)`main(argc=2, argv=0x00000000046122f0) at showdebug.c:19:12
+      16       int result = compute_factorial(firstletter - '0');
+      17  
+      18       // Returned result is clipped at 255...
+   -> 19       return result;
+      20   }
+   (lldb) p result
+   (int) $4 = 120
+   (lldb) expr result=42
+   (int) $5 = 42
+   (lldb) p result
+   (int) $6 = 42
+   (lldb) c
+   Process 21340 resuming
+   Process 21340 exited with status = 42 (0x0000002a)
+   (lldb) exit