This is an archive of the discontinued LLVM Phabricator instance.

[RISCV] Add SystemV ABI
Needs ReviewPublic

Authored by luismarques on May 31 2019, 7:06 AM.

Details

Summary

This implements the SystemV ABI for RISC-V for LLDB.

Diff Detail

Event Timeline

simoncook created this revision.May 31 2019, 7:06 AM
Herald added a project: Restricted Project. · View Herald TranscriptMay 31 2019, 7:06 AM
mhorne added a subscriber: mhorne.Jun 1 2019, 10:55 AM

You are following all of the patterns for all of the architectures. It would be nice for us to cleanup DynamicRegisterInfo.cpp, Platform.cpp, and Thread.cpp eventually so we don't need to modify these files when a few arch is added by abstracting things into lldb_private::Architecture, but that is beyond the scope of this change.

lldb/source/Plugins/ABI/SysV-riscv/ABISysV_riscv.cpp
41

No need to fill in the "eRegisterKindProcessPlugin" field of each register info. That is designed to be the number that any process plug-in (probably ProcessGDBRemote.cpp) fills in, so best to not fill this in. The orders are: EH frame, DWARF, generic, process plug-in and then LLDB. So I would change all of these to:

{0, 0, LLDB_INVALID_REGNUM, LLDB_INVALID_REGNUM, 0},
lldb/source/Plugins/Process/Utility/DynamicRegisterInfo.cpp
624–635

Seems like you filled all of this info in the registers defs already in ABISysV_riscv.cpp? Do we need this here? Some platforms weren't setting these correctly and this code was meant to correct that.

lldb/source/Target/Platform.cpp
1914–1922

This would be great to factor out into lldb_private::Architecture eventually. You are correctly following the patterns that are in LLDB. I will see if I can get this to happen soon so we don't run into these issues again.

lldb/source/Target/Thread.cpp
2060–2084

Seems like the original logic before your mods is not correct. Not sure the default case ever gets used here. The arch of x86_64, x86, arm, aarch64 or thumb should all be caught before we get to the default statement for apple targets. I will add Jason Molenda to the change to get a comment on this.

Jason, can you take a look at "Thread.cpp" and my inlined comment?

xiaobai added subscribers: compnerd, xiaobai.

@compnerd You might be interested in this.

jasonmolenda added inline comments.Jun 3 2019, 4:27 PM
lldb/source/Plugins/Process/Utility/DynamicRegisterInfo.cpp
624–635

There's AugmentRegisterInfoViaABI() over in ProcessGDBRemote which does fill in eh_frame, dwarf, and generic register numbers based on the ABI's register table. It seems like that should be sufficient - although I see a bunch of other architectures doing the exact same thing here in DynamicRegisterInfo::AddRegister.

lldb/source/Target/Thread.cpp
2060–2084

The default case shouldn't ever be taken - it was the unwinder we used (I think you wrote it during bringup) before UnwindLLDB existed. I think we've kept it around in case people need an initial unwinder during bringup of new architectures, but maybe it's not serving any purpose at all any more.

simoncook updated this revision to Diff 205287.Jun 18 2019, 2:49 AM
  • Refactored register tables to match style used in i386/x86_64
  • Add enum for RISC-V DWARF numbers
  • Add F registers (assuming 32-bit, at runtime this seems to be overwritten to 64-bit if D extension is provided)
  • Add default unwind plan for first frame
simoncook retitled this revision from [WIP][RISCV] Initial port of LLDB for RISC-V to [RISCV] Add SystemV ABI.
simoncook edited the summary of this revision. (Show Details)

Rebase, implement all hooks that aren't PrepareTrivialCall/function calling related. If its possible to commit these two separately, I think it would be best to have that as a separate patch whereby preparing arguments for the various RISC-V hard-float ABIs can be done independently of breakpoints/unwinding/etc.

echristo edited reviewers, added: labath; removed: espindola.Nov 4 2019, 2:12 PM
abidh added a subscriber: abidh.Nov 7 2019, 3:53 AM

So, if I understand correctly, this patch doesn't just add an "ABI" plugin (for which we have in the past agreed that we don't know of a better way to test than just running "TestReturnValue" on the given architecture), but it actually adds all the bits and pieces necessary to support risc-v debugging, *sans* an actual debug server. This means that one cannot test this code, even if he happened to have risc-v hardware around. I'm not really sure what's our position on that, which is why it took me so long to write this comment, and why I am reluctant to hit accept (even though the code itself seems pretty good).

I take it you have some proprietary/downstream debug server/hardware probe that you're using this with? And you're not putting that stuff in lldb-server because lldb-server is too heavy for your use case ?

Yeah, I don't think we want to be merging code we can't test even in a non-automated way. Even if this code is completely bug-free, the inability to test it just means we risk having it bit-rot with nobody noticing.

Yeah, I don't think we want to be merging code we can't test even in a non-automated way. Even if this code is completely bug-free, the inability to test it just means we risk having it bit-rot with nobody noticing.

We now have two open-source debug servers that we can use to test this patch: gdbserver and QEMU's gdbstub, both of which now have RISC-V support. I rebased this patch and tested it with both.
My initial tests were with QEMU/gdbstub, and I was quite pleased with the results. I could set breakpoints, continue execution, step line-by-line and instruction-by-instruction and print registers (GPRs and CSRs). The backtrace was only showing the current frame. Still, I would say that LLDB was in a good enough state to be useful. Next, I tested it against gdbserver (compiled from git main) running in a Fedora RISC-V machine. Setting a breakpoint would seemingly succeed (e.g. it reported success, with the correct code address), but the code doesn't actually trap. You can still break with Ctrl-C though, and print registers and so on. So there seem to be some additional issues, but I also had problems when connecting less recent versions of GDB to that debug server, so we might just need to do some adjustments to match the GDB changes.

It seems that, once rebased, this patch is in pretty good shape. lowRISC is keen to see LLDB get RISC-V support, and I'm now ramping up work to help with that. Given the great work that we already have in this patch, I think an important first step would be to land it. None of the issues I encountered were obviously due to the code in this patch (and the things still marked as TODO), so I don't see any major downside to merging it. I will do a more in-depth investigation of the issues I encountered, but that shouldn't be a blocker to this patch. Landing this contribution would make it easier for others to contribute the missing pieces, and fixes for the issues, without having to submit duplicate work.

If it helps, feel free to use this commit [1], it's pretty much just a rebase of this patch.
@simoncook please let me know if you plan to update this patch and get it approved and landed (LGTM, once rebased), or other ways in which it would make sense to coordinate our work.

[1] https://github.com/luismarques/llvm-project/commit/4a0cccb0cfa8cb5c59c8803077a76751498447ec

Thanks for looking at this @luismarques We had planned to put more effort into this patch, but time got in the way for quite a lot, but I'm glad it's working; thanks for the rebase I'll update this to match shortly. And it's good that it looks like it's mostly working. I'm curious about your backtrace showing one frame, is that something without debug information, since the example I was using when writing this did show a backtrace back to main? It would be good to understand why that disn't produce the expected output.

As for next steps, if we're happy with the state then I think this should land (assuming qemu is sufficient given it is public), and then we can flesh out other bits which give a better experience. I'm not sure how to connect this to any automated testing, or where to document any way of checking this manually, the state of that isn't quite clear, so any clarity there helps.

Beyond this I think the next stage is implementing the parts for calling functions within a target, which if you could help with that would be great. I see that as a follow up patch to this, I don't see the two necessarily having to land together, since this part enables a useful debugging experience already.

As for next steps, if we're happy with the state then I think this should land (assuming qemu is sufficient given it is public), and then we can flesh out other bits which give a better experience. I'm not sure how to connect this to any automated testing, or where to document any way of checking this manually, the state of that isn't quite clear, so any clarity there helps.

Yeah, I think that after the rebase this is nearly ready to land. The only additional suggestions I have at this point are:

  • We should probably follow the convention and put the registers in Plugins/Process/Utility/RegisterInfos_riscv.h, like is done for other archs. If that isn't trivial I guess it could be a follow-up patch.
  • Review the list of callee-saved registers. Aren't x25 and x26 missing?
  • Nit: there's a typo in this patch that I missed in my rebased commit and should be corrected: "poinrter".
  • I had renamed the RISCVflags members, if you use my rebased commit please check if you agree with that alternative.

I'm curious about your backtrace showing one frame, is that something without debug information, since the example I was using when writing this did show a backtrace back to main? It would be good to understand why that disn't produce the expected output.

It is with debug information. I had been looking at other issues, but I'm going to look into this and I'll let you know what I find out.

Beyond this I think the next stage is implementing the parts for calling functions within a target, which if you could help with that would be great. I see that as a follow up patch to this, I don't see the two necessarily having to land together, since this part enables a useful debugging experience already.

I agree, we can land this and provide follow-up patches. For my next steps I was looking into fixing the issues I was experiencing, if possible, and then implementing those TODOs. One of the issues I've diagnosed is that we need to add extra logic to handle RV32 vs RV64 based on the ELF header, like is done for MIPS, otherwise it incorrectly assumes RV32 when it should be RV64. I'll provide a follow-up patch.

Thanks for the feedback and the patch @simoncook!

luismarques commandeered this revision.Sep 3 2020, 5:42 AM
luismarques added a reviewer: simoncook.

Commandeering the patch, as discussed in the last RISC-V sync-up call. I'll update it with only minor changes; further development can be done in follow-up patches.

  • Fix list of callee saved registers;
  • Fix typo;
  • Fix trivial clang-format issue.

@labath @jrtc27 @clayborg Now that we have at least 3 open-source debug servers that we can use to test this with (OpenOCD, QEMU gdbstub, gdbserver) perhaps this can be merged? I had very good results using this patch with OpenOCD. This patch doesn't include automated tests, but I'm not sure what tests would be required for this patch, or that it makes sense to require them at this point. I'll be doing more work for LLDB RISC-V support, and I'll provide tests for specific fixes going forward.

@luismarques , what's the recommended gdb-server implementation you recommend for me to try this on a riscv machine?

@luismarques , what's the recommended gdb-server implementation you recommend for me to try this on a riscv machine?

I compiled mine from the mainline binutils-gdb git repository (running on Fedora riscv64). But I suggest that you used OpenOCD instead, as I think that RISC-V gdbserver had some issues. If I recall correctly, gdbserver wouldn't return from the c command even if it hit a breakpoint. GDB was using vCont instead so it didn't run into that problem. Anyway, that's in my TODO list to better reduce the issue and report bugs/send patches, but just to let you know.

@labath @jrtc27 @clayborg Now that we have at least 3 open-source debug servers that we can use to test this with (OpenOCD, QEMU gdbstub, gdbserver) perhaps this can be merged? I had very good results using this patch with OpenOCD. This patch doesn't include automated tests, but I'm not sure what tests would be required for this patch, or that it makes sense to require them at this point. I'll be doing more work for LLDB RISC-V support, and I'll provide tests for specific fixes going forward.

Sounds reasonable to me. Maybe it's still to early for that but have you tried running (part of) the test suite under QEMU yet? It should give you a pretty good idea of the state of things and gives you a bunch of test coverage for free. You'll probably want to consider setting up a bot for that down the road anyway.

Sounds reasonable to me. Maybe it's still to early for that but have you tried running (part of) the test suite under QEMU yet? It should give you a pretty good idea of the state of things and gives you a bunch of test coverage for free. You'll probably want to consider setting up a bot for that down the road anyway.

I haven't tried that yet. When I did try to build LLVM with LLDB support under a QEMU VM (Fedora RISC-V) it failed because CreateHostNativeRegisterContextLinux didn't yet have a RISC-V implementation. I imagine that it might be possible to run part of the tests without solving that issue, but I added that to my TODO list anyway. I'll be able to devote more time to LLDB work around the beginning of October.

labath added a comment.Oct 1 2020, 7:32 AM

ABI plugins are one of the hardest things to test in lldb, particularly without actual hardware. That's why we've let them be added in the past without any accompanying tests. The situation is not ideal though, because we've accumulated various ABI plugins which are hard to change without breaking (or even to know if they work) because they only work with third-party (possibly proprietary) stubs.

I'm not sure what's the state of risc-v hardware these days and how much resources do you have available, but if it's at all possible, I'd definitely recommend adding the lldb-server bits for risc-v and adding a builtbot for testing this configuration.

Even if there's no hardware which could run the lldb test suite in a reasonable amount of time, the availability of an lldb-server implementation would enable anyone to test this via qemu -- that's the strategy we're pursuing for the ARM SVE stuff, and we're about to add instructions on how to run lldb+sve+qemu here: D82064.

lldb/source/Plugins/ABI/RISCV/ABISysV_riscv.cpp
37–276 ↗(On Diff #289696)

It would be good to make this an MCBasedABI (that's a new concept, introduced after the inception of this patch). That should make all of this goo disappear.

lldb/source/Utility/ArchSpec.cpp
454–457

These bits should go in as a part of D86292.

  • Use MCBasedABI
  • Remove ArchSpec core bits, to be moved to D86292

I'm not sure what's the state of risc-v hardware these days and how much resources do you have available, but if it's at all possible, I'd definitely recommend adding the lldb-server bits for risc-v and adding a builtbot for testing this configuration.

That's on my TODO list. Still, if that would take, say, ~2 months, perhaps it would still make sense to merge this and D86292 before then? Please let me know if there's anything else that should be addressed in these patches themselves.

labath added a comment.Nov 2 2020, 1:32 AM

Two months isn't that long, actually.

Let's first see if there's anything else in this patch that we *can* test.

It seems like we should be able to test the remainder of the ObjectFileELF changes if we extended lldb-test to dump the architecture flags too. Can you try something like that?

And the disassembler change, I'd expect that one could be tested by just llvm-mc-ing some risc-v code and then asking lldb to disassemble it (without running anything).

lldb/source/Plugins/ABI/RISCV/ABISysV_riscv.h
9 ↗(On Diff #302182)

We use a different format for header guards these days.

jade added a subscriber: jade.Apr 22 2021, 12:05 AM
lenary removed a subscriber: lenary.Apr 22 2021, 2:09 AM
jade added a comment.Apr 22 2021, 3:16 AM

ABI plugins are one of the hardest things to test in lldb, particularly without actual hardware. That's why we've let them be added in the past without any accompanying tests. The situation is not ideal though, because we've accumulated various ABI plugins which are hard to change without breaking (or even to know if they work) because they only work with third-party (possibly proprietary) stubs.

I'm not sure what's the state of risc-v hardware these days and how much resources do you have available, but if it's at all possible, I'd definitely recommend adding the lldb-server bits for risc-v and adding a builtbot for testing this configuration.

Even if there's no hardware which could run the lldb test suite in a reasonable amount of time, the availability of an lldb-server implementation would enable anyone to test this via qemu -- that's the strategy we're pursuing for the ARM SVE stuff, and we're about to add instructions on how to run lldb+sve+qemu here: D82064.

This is also testable with the gdbserver built into qemu, at least for kernel debugging. It seems to work, although for some reason, step-out is broken for me on a rust target (https://github.com/lf-/mu):

(lldb) fin
error: Could not create return address breakpoint.

I have yet to debug this but I think I will do so when I get more time.

rkruppe removed a subscriber: rkruppe.Apr 22 2021, 8:28 AM
jade added a comment.Apr 27 2021, 1:27 AM

ABI plugins are one of the hardest things to test in lldb, particularly without actual hardware. That's why we've let them be added in the past without any accompanying tests. The situation is not ideal though, because we've accumulated various ABI plugins which are hard to change without breaking (or even to know if they work) because they only work with third-party (possibly proprietary) stubs.

I'm not sure what's the state of risc-v hardware these days and how much resources do you have available, but if it's at all possible, I'd definitely recommend adding the lldb-server bits for risc-v and adding a builtbot for testing this configuration.

Even if there's no hardware which could run the lldb test suite in a reasonable amount of time, the availability of an lldb-server implementation would enable anyone to test this via qemu -- that's the strategy we're pursuing for the ARM SVE stuff, and we're about to add instructions on how to run lldb+sve+qemu here: D82064.

This is also testable with the gdbserver built into qemu, at least for kernel debugging. It seems to work, although for some reason, step-out is broken for me on a rust target (https://github.com/lf-/mu):

(lldb) fin
error: Could not create return address breakpoint.

I have yet to debug this but I think I will do so when I get more time.

I have looked at this. On my bare metal target with the integrated qemu gdbserver, bt, in several instances, is thinking it is at level zero of the stack and thus can't find another frame to put a breakpoint at. I've made a simple bare metal project that hits this issue with the unwinder. You just need a qemu build with emulated architecture support enabled (package qemu-arch-extra in many distros) and a copy of clang/lld/lldb for this.

These files are also available as a gist, which can be grabbed with: git clone https://gist.github.com/e2efac2f780ed820277dbaf608805f4e lldb-riscv-repro

Makefile:

QEMU = qemu-system-riscv64
TARGETFLAGS = --target=riscv64-none-unknown-elf -march=rv64imac
CFLAGS =  $(TARGETFLAGS) -mno-relax -fPIC -g
QEMUOPTS = -machine virt -bios none -kernel kern -m 128M \
		   			-nographic -s -S

LINKFLAGS = $(TARGETFLAGS) -fuse-ld=lld -nostdlib -nodefaultlibs \
			-Wl,-Tshoo.ld
CC = clang

kern: init.o start.o
	$(CC) $(LINKFLAGS) $^ -o $@

qemu: kern
	$(QEMU) $(QEMUOPTS)
%.o: %.s
	$(CC) $(CFLAGS) -c $^

.PHONY: clean qemu
clean:
	rm *.o kern

shoo.ld:

OUTPUT_ARCH("riscv64")
ENTRY(_entry)

SECTIONS {
    /* VIRT_DRAM */
    . = 0x80000000;

    .text : {
        *(.text.first)
        *(.text .text.*)
        . = ALIGN(0x1000);
    }

    .rodata ALIGN(0x1000) : {
        . = ALIGN(16);
        PROVIDE(srodata = .);
        *(.srodata .srodata.*)
        . = ALIGN(16);
        *(.rodata .rodata.*)
        . = ALIGN(0x1000);
        PROVIDE(erodata = .);
    }

    .data ALIGN(0x1000) : {
        PROVIDE(srwdata = .);
        *(.sdata .sdata.*)
        . = ALIGN(16);
        *(.data .data.*)
    }

    .bss ALIGN(0x1000) : {
        . = ALIGN(16);
        *(.sbss .sbss.*)
        . = ALIGN(16);
        *(.bss .bss.*)
    }
}

init.s:

// kernel entry point
// largely nicked from https://github.com/mit-pdos/xv6-riscv/blob/riscv/kernel/entry.S
.data
.globl STACKS
.text
.section .text.first
.globl start
.globl _entry
// _entry(mhartid: usize, dtb: *const u8)
_entry:
    // we arrive here, in machine mode, once qemu jumps to the start of memory

    // set up a stack
    la sp, STACKS
    li t0, 16384     // use 16k stacks

    // we want to get the pointer to the top of the (descending) stack
    // thus we want 16k * (hartid + 1)
    addi t1, a0, 1
    mul t0, t0, t1
    add sp, sp, t0

    call startup
spin:
    j spin // startup() will not return
.text

start.c:

char STACKS[4*4096];
volatile int i1 = 0;
volatile int i2 = 0;
volatile int i3 = 0;

void fn3(void) {
    i3 = 10;
}

void fn2(void) {
    i2 = 3;
    fn3();
}

void fn1(void) {
    fn2();
    fn2();
    i1 = 18;
}

void startup(void) {
    fn1();
    while (1) {}
}

Reproduction:

$ make qemu
# (in a separate window)
$ lldb -a riscv64
(lldb) target create kern
Current executable set to '/tmp/lldb-issue/kern' (riscv64).
(lldb) gdb-remote 1234
Process 1 stopped
* thread #1, stop reason = signal SIGTRAP
    frame #0: 0x0000000000001000
->  0x1000: auipc  t0, 0
    0x1004: addi   a2, t0, 40
(lldb) b fn3
Breakpoint 1: where = kern`fn3 + 2 at start.c:7:8, address = 0x0000000080000022
(lldb) c
Process 1 resuming
Process 1 stopped
* thread #1, stop reason = breakpoint 1.1
    frame #0: 0x0000000080000022 kern`fn3 at start.c:7:8
   4   	volatile int i3 = 0;
   5   	
   6   	void fn3(void) {
-> 7   	    i3 = 10;
   8   	}
   9   	
   10  	void fn2(void) {
kern`fn3:
->  0x80000022 <+2>: sd     ra, 8(sp)
    0x80000024 <+4>: sd     s0, 0(sp)
(lldb) bt
* thread #1, stop reason = breakpoint 1.1
  * frame #0: 0x0000000080000022 kern`fn3 at start.c:7:8
(lldb) fin
error: Could not create return address breakpoint.

Expected result:

(gdb) target remote :1234
Remote debugging using :1234
0x0000000000001000 in ?? ()
=> 0x0000000000001000:	97 02 00 00	auipc	t0,0x0
(gdb) b fn3
Breakpoint 1 at 0x80000022: file start.c, line 7.
(gdb) c
Continuing.

Breakpoint 1, fn3 () at start.c:7
7	    i3 = 10;
(gdb) bt
#0  fn3 () at start.c:7
#1  0x0000000080000058 in fn2 () at start.c:12
#2  0x0000000080000070 in fn1 () at start.c:16
#3  0x000000008000009c in startup () at start.c:22
#4  0x000000008000001c in _entry () at init.s:23
Backtrace stopped: frame did not save the PC

Another thing I've noticed while testing this patch is that the ABI names for the registers are the only ones accessible via reg read: reg read x1 does not work, for instance. lldb has an alternate register name mechanism in RegisterInfo, which is used for arg1, arg2, etc, on Arm, and probably some others. It would be useful to have this for the xN register names in riscv.

These files are also available as a gist, which can be grabbed with: git clone https://gist.github.com/e2efac2f780ed820277dbaf608805f4e lldb-riscv-repro

Thank you for the reproducer. I think I've isolated the issue. I'll see if I can provide a fix.

Rebase and fix the unwinding issue reported by @jade.

Any update on the testing strategy yet? It seems like that this is actively being used and worked on, so I'm generally supportive of landing this with some elementary coverage.

Any update on the testing strategy yet? It seems like that this is actively being used and worked on, so I'm generally supportive of landing this with some elementary coverage.

A while ago I got a skeleton of lldb-server compiling for RISC-V, but it seemed like it would be quite a bit of work to flesh out that implementation. I can only allocate some amount of effort to that right now, so that part might take a while to get finished.

I was going to follow up on @labath's suggestions of further testing just this patch (the ObjectFileELF and the disassembler changes), but some other work got in the way. That bit should be easier to prioritize, especially if we agreed that such kind of testing would be enough to get this landed.

Any update on the testing strategy yet? It seems like that this is actively being used and worked on, so I'm generally supportive of landing this with some elementary coverage.

A while ago I got a skeleton of lldb-server compiling for RISC-V, but it seemed like it would be quite a bit of work to flesh out that implementation. I can only allocate some amount of effort to that right now, so that part might take a while to get finished.

Thanks for the update!

I was going to follow up on @labath's suggestions of further testing just this patch (the ObjectFileELF and the disassembler changes), but some other work got in the way. That bit should be easier to prioritize, especially if we agreed that such kind of testing would be enough to get this landed.

Sounds reasonable to me.

aprantl added inline comments.
lldb/include/lldb/Utility/ArchSpec.h
94

In modern LLDB style we would format this as:

/// RISCV specific flags.
enum class RISCVflags {
    arch_c = 0x00000001, ///< Some comment explaining what this is.
    abi_f = 0x00000010, ///< ...
    abi_d = 0x00000020
  };
lldb/source/Plugins/ABI/RISCV/ABISysV_riscv.cpp
1 ↗(On Diff #341153)

-*- C++ -*- markers only make sense for .h files, where the language is ambiguous.

Address review feedback.

luismarques marked 2 inline comments as done.Apr 29 2021, 4:43 AM
housel added a subscriber: housel.May 27 2021, 12:01 PM
sven added a subscriber: sven.May 31 2021, 5:36 AM

@luismarques
I have tried the patch with llvm12, test case is provide by jade(https://gist.github.com/e2efac2f780ed820277dbaf608805f4e), but it didn't worked for me.
After execute 'gdb-remote 1234', the terminal shows:

Process 1 stopped
* Thread #1, stop reason = signal SIGTRAP
       frame #0: 0xffffffffffffffff
}

It seems that the unwind didn't succeed, can you figure out the problem?
my qemu vesion: 4.2.1
Thanks.

jade added a comment.May 31 2021, 11:08 AM

It seems that the unwind didn't succeed, can you figure out the problem?
my qemu vesion: 4.2.1
Thanks.

Unfortunately, from my last adventures through the lldb codebase in debugging that (see bugs.llvm.org/show_bug.cgi?id=49941 for pertinent places to look at internals), that particular symptom may not provide enough detail to track this down to a root cause necessarily (in the case of missing this patch for instance, it would be that the debugger didn't know what the stack pointer was called). It could be that the patch is not applied, it could be something else entirely. I believe if you call Dump() with the debugger on an instance of one of the gdb protocol classes that has the register information (sorry, away from a computer and can't remember which one it is), you can see what registers it knows about. But overall our error reporting needs to be vastly improved on this case. I want to write a patch for it but I've not had the chance.

I have tried the patch with llvm12, test case is provide by jade(https://gist.github.com/e2efac2f780ed820277dbaf608805f4e), but it didn't worked for me.
It seems that the unwind didn't succeed, can you figure out the problem?

That's surprising. I'll see if I can figure out what the issue might be. Thanks.

That's surprising. I'll see if I can figure out what the issue might be. Thanks.

Confirmed. Something must have broken since the last patch revision. I'll see if I can figure out / fix this soon.

sven added a comment.EditedMay 31 2021, 7:53 PM

That's surprising. I'll see if I can figure out what the issue might be. Thanks.

Confirmed. Something must have broken since the last patch revision. I'll see if I can figure out / fix this soon.

Hi @luismarques, @jade I have fixed the issue by install libxml2-dev, then recompile lldb and it works.
The cause of this issue is that LLDB doesn't send qXfer command for register info which defined in qemu-gdb-stub xml if libxml2 is not installed.
See ProcessGDBRemote.cpp::GetGDBServerRegisterInfo().
Thank you for your help.

// query the target of gdb-remote for extended target information returns
// true on success (got register definitions), false on failure (did not).
bool ProcessGDBRemote::GetGDBServerRegisterInfo(ArchSpec &arch_to_use) {
  // Make sure LLDB has an XML parser it can use first
  if (!XMLDocument::XMLEnabled())
    return false;

But the unwind can not work on my machine, the issue is similar to which @jade reported

(lldb) bt
* thread #1, stop reason = breakpoint 1.1
  * frame #0: 0x0000000080000022 kern`fn3 at start.c:7:8

Can you reproduce this problem? Or please show me how you fix the issue, thanks very much.

Hi, I have my fingers crossed since this request was opened in 2019. It seems like it compiles and usable to certain degree. Can this patch be merged and included in llvm 13 as initial riscv64 support? We can then improve it subsequently if bugs show up. Otherwise it will be one more year of waiting for the consumers. Thank you for your effort!

Hi, I have my fingers crossed since this request was opened in 2019. It seems like it compiles and usable to certain degree. Can this patch be merged and included in llvm 13 as initial riscv64 support? We can then improve it subsequently if bugs show up. Otherwise it will be one more year of waiting for the consumers. Thank you for your effort!

I think the main blocker for merging was testing. If it helps, I now have the RISC-V server in my hands and I should be able to set up a buildbot soon.

I think the main blocker for merging was testing. If it helps, I now have the RISC-V server in my hands and I should be able to set up a buildbot soon.

It certainly will help. Thank you! :)

On July 27, 13.x will be branched out. If we can squeeze it in that would be a huge thing! I can help anyway you want with testing (I don't have the device atm, but can run the builds in Debian and Alpine qmeu-based chroot environments). Otherwise, we can request the maintainers later if they would cherry-pick this patch once merged.

jrtc27 added inline comments.Jul 13 2021, 3:07 PM
lldb/source/Plugins/ABI/RISCV/ABISysV_riscv.cpp
118 ↗(On Diff #341471)
141 ↗(On Diff #341471)
142 ↗(On Diff #341471)

And SP is just.. the same? Surely the default unwind plan is to load SP and RA via FP? You won't get very far with this.

167–168 ↗(On Diff #341471)

What about if I have the D extension but only use LP64F as the ABI? Does it matter that this says f9 is callee-saved but in reality only the low 32 bits are?

lldb/source/Plugins/ABI/RISCV/ABISysV_riscv.h
16 ↗(On Diff #341471)

IsRV64?

70–71 ↗(On Diff #341471)

2 and 3 mod 4 are nevertheless invalid if you don't have the C extension and will take an alignment fault

72–73 ↗(On Diff #341471)

Are addresses reliably zero-extended or can you ever get cases where they're sign-extended from a valid 32-bit address to a 64-bit address?

lldb/source/Plugins/Disassembler/LLVMC/DisassemblerLLVMC.cpp
1152 ↗(On Diff #341471)

This will override whatever the ELF says in its attributes section. This might make sense as a default for if you don't have a binary and are just poking at memory, but it makes things worse when you do, the common case that need to work as best as we can manage.

MaskRay added a subscriber: felixonmars.EditedSep 10 2021, 11:03 AM

Hi Luís, is this still needed after D86292? Or are there missing pieces?
@felixonmars reported that https://archriscv.felixc.at/.status/logs/lldb.log still failed to build on riscv64 Arch Linux.
It's 12.0.0 but I thought that you may have an idea :)

/usr/bin/ld: lib/liblldbPluginProcessLinux.a(NativeThreadLinux.cpp.o): in function `.L181':
NativeThreadLinux.cpp:(.text._ZN12lldb_private13process_linux17NativeThreadLinuxC2ERNS0_18NativeProcessLinuxEm+0x7c): undefined reference to `lldb_private::process_linux::NativeRegisterContextLinux::CreateHostNativeRegisterContextLinux(lldb_private::ArchSpec const&, lldb_private::NativeThreadProtocol&)'
collect2: error: ld returned 1 exit status

Hi Luís, is this still needed after D86292? Or are there missing pieces?
@felixonmars reported that https://archriscv.felixc.at/.status/logs/lldb.log still failed to build on riscv64 Arch Linux.
It's 12.0.0 but I thought that you may have an idea :)

/usr/bin/ld: lib/liblldbPluginProcessLinux.a(NativeThreadLinux.cpp.o): in function `.L181':
NativeThreadLinux.cpp:(.text._ZN12lldb_private13process_linux17NativeThreadLinuxC2ERNS0_18NativeProcessLinuxEm+0x7c): undefined reference to `lldb_private::process_linux::NativeRegisterContextLinux::CreateHostNativeRegisterContextLinux(lldb_private::ArchSpec const&, lldb_private::NativeThreadProtocol&)'
collect2: error: ld returned 1 exit status

That just teaches it that RISC-V exists, it can't really do anything useful with it, you need this patch to teach it about relevant registers, how to disassemble, etc, and even then only for bare-metal, there is no Linux or FreeBSD arch-specific plugin.

trlim added a subscriber: trlim.Nov 28 2021, 10:39 PM

Hi Luis, are you planning on adding plugin architecture support (in lldb/source/Plugins/Architecture) as part of this work?

lldb/source/Plugins/Disassembler/LLVMC/DisassemblerLLVMC.cpp
1152 ↗(On Diff #341471)
if (arch.GetFlags() & ArchSpec::eRISCV_arch_c) {
  features_str += "+c,";
}

and so on (like the case with MIPS below). Maybe we can define a and m as well in ArchSpec?

Hi Luis, are you planning on adding plugin architecture support (in lldb/source/Plugins/Architecture) as part of this work?

I commandeered this patch from Simon Cook, the original author, but I haven't been a good steward. For a while I seemed to always have something higher priority in my queue and eventually I kinda stopped thinking about it. So don't rely on my plans for this patch. Feel free to adopt it and continue this line of work. My apologies to the community.

tzb99 added a subscriber: tzb99.Jul 26 2022, 9:14 AM
tzb99 added a comment.EditedJul 26 2022, 10:07 AM

That's surprising. I'll see if I can figure out what the issue might be. Thanks.

Confirmed. Something must have broken since the last patch revision. I'll see if I can figure out / fix this soon.

Hi @luismarques, @jade I have fixed the issue by install libxml2-dev, then recompile lldb and it works.
The cause of this issue is that LLDB doesn't send qXfer command for register info which defined in qemu-gdb-stub xml if libxml2 is not installed.
See ProcessGDBRemote.cpp::GetGDBServerRegisterInfo().
Thank you for your help.

// query the target of gdb-remote for extended target information returns
// true on success (got register definitions), false on failure (did not).
bool ProcessGDBRemote::GetGDBServerRegisterInfo(ArchSpec &arch_to_use) {
  // Make sure LLDB has an XML parser it can use first
  if (!XMLDocument::XMLEnabled())
    return false;

But the unwind can not work on my machine, the issue is similar to which @jade reported

(lldb) bt
* thread #1, stop reason = breakpoint 1.1
  * frame #0: 0x0000000080000022 kern`fn3 at start.c:7:8

Can you reproduce this problem? Or please show me how you fix the issue, thanks very much.

That's surprising. I'll see if I can figure out what the issue might be. Thanks.

Confirmed. Something must have broken since the last patch revision. I'll see if I can figure out / fix this soon.

Hi @luismarques, @jade I have fixed the issue by install libxml2-dev, then recompile lldb and it works.
The cause of this issue is that LLDB doesn't send qXfer command for register info which defined in qemu-gdb-stub xml if libxml2 is not installed.
See ProcessGDBRemote.cpp::GetGDBServerRegisterInfo().
Thank you for your help.

// query the target of gdb-remote for extended target information returns
// true on success (got register definitions), false on failure (did not).
bool ProcessGDBRemote::GetGDBServerRegisterInfo(ArchSpec &arch_to_use) {
  // Make sure LLDB has an XML parser it can use first
  if (!XMLDocument::XMLEnabled())
    return false;

But the unwind can not work on my machine, the issue is similar to which @jade reported

(lldb) bt
* thread #1, stop reason = breakpoint 1.1
  * frame #0: 0x0000000080000022 kern`fn3 at start.c:7:8

Can you reproduce this problem? Or please show me how you fix the issue, thanks very much.

Hi: I encountered the similar issue with the frame address showing all 1s. I tried to install libxml2-dev and wanted to recompile lldb. How did you recompile lldb? Do you cross compile or compile inside the qemu environment? If you do cross-compile, would you mind show the arguments of cmake? Thank you very much!

PS: I use cmake arguments and followed the instructions on the lldb website to do the LLVM in-tree build. I will encounter errors if I set enable xml2 to be ON:

/usr/include/libxml2/libxml/encoding.h:31:10: fatal error: unicode/ucnv.h: No such file or directory

31 | #include <unicode/ucnv.h>
   |          ^~~~~~~~~~~~~~~~

compilation terminated.

But the unwind can not work on my machine, the issue is similar to which @jade reported

(lldb) bt
* thread #1, stop reason = breakpoint 1.1
  * frame #0: 0x0000000080000022 kern`fn3 at start.c:7:8

Can you reproduce this problem? Or please show me how you fix the issue, thanks very much.

Hi: I encountered the similar issue with the frame address showing all 1s. I tried to install libxml2-dev and wanted to recompile lldb. How did you recompile lldb? Do you cross compile or compile inside the qemu environment? If you do cross-compile, would you mind show the arguments of cmake? Thank you very much!

PS: I use cmake arguments and followed the instructions on the lldb website to do the LLVM in-tree build. I will encounter errors if I set enable xml2 to be ON:

/usr/include/libxml2/libxml/encoding.h:31:10: fatal error: unicode/ucnv.h: No such file or directory

31 | #include <unicode/ucnv.h>
   |          ^~~~~~~~~~~~~~~~

compilation terminated.

lldb/source/Plugins/ABI/RISCV/ABISysV_riscv.cpp
142 ↗(On Diff #341471)

Following up to @jrtc27 's comment here because I saw people saying they were getting only one stack frame in their backtraces. I don't know this ISA/ABI, but this DefaultUnwindPlan probably only works correctly on the first instruction of a function. Normally the DefaultUnwindPlan should be expressed in terms of how to find the caller stack frame from the middle of the function body -- once the prologue has executed, and you're using a frame pointer register to track your stack frame, most likely. For real quality unwinds, we can use DWARF debug_frame or eh_frame, and/or we have an instruction emulation engine that tracks register spills and stack frame setup/teardown and creates an unwind plan at every instruction point using that. But the RISC-V back-end would need to be written for that; a bit of work. For a starting point, unwinding from the middle of the function is the right thing to do by default.

tzb99 added a comment.Aug 3 2022, 5:14 PM

But the unwind can not work on my machine, the issue is similar to which @jade reported

(lldb) bt
* thread #1, stop reason = breakpoint 1.1
  * frame #0: 0x0000000080000022 kern`fn3 at start.c:7:8

Can you reproduce this problem? Or please show me how you fix the issue, thanks very much.

Hi: I encountered the similar issue with the frame address showing all 1s. I tried to install libxml2-dev and wanted to recompile lldb. How did you recompile lldb? Do you cross compile or compile inside the qemu environment? If you do cross-compile, would you mind show the arguments of cmake? Thank you very much!

PS: I use cmake arguments and followed the instructions on the lldb website to do the LLVM in-tree build. I will encounter errors if I set enable xml2 to be ON:

/usr/include/libxml2/libxml/encoding.h:31:10: fatal error: unicode/ucnv.h: No such file or directory

31 | #include <unicode/ucnv.h>
   |          ^~~~~~~~~~~~~~~~

compilation terminated.

But the unwind can not work on my machine, the issue is similar to which @jade reported

(lldb) bt
* thread #1, stop reason = breakpoint 1.1
  * frame #0: 0x0000000080000022 kern`fn3 at start.c:7:8

Can you reproduce this problem? Or please show me how you fix the issue, thanks very much.

Hi: I encountered the similar issue with the frame address showing all 1s. I tried to install libxml2-dev and wanted to recompile lldb. How did you recompile lldb? Do you cross compile or compile inside the qemu environment? If you do cross-compile, would you mind show the arguments of cmake? Thank you very much!

PS: I use cmake arguments and followed the instructions on the lldb website to do the LLVM in-tree build. I will encounter errors if I set enable xml2 to be ON:

/usr/include/libxml2/libxml/encoding.h:31:10: fatal error: unicode/ucnv.h: No such file or directory

31 | #include <unicode/ucnv.h>
   |          ^~~~~~~~~~~~~~~~

compilation terminated.

Thank you! I checked the DefaultUnwindPlan on the ABISysV_riscv and found it is not called at all in the lldb-server process. Could this patch not work on the qemu system mode? I am currently using the qemu system mode. In qemu, it allows me to install GDB and GDB works well. However, the lldb cannot stop on the breakpoint and the lldb-server returns the stack frame with all 1s. By taking further investigation, the GetExpeditedRegisters does not work well, since it cannot identify the sp, fp, ra, pc registers. The fucntions called Handle_qMemoryRegionInfo and Handle_Memory_Read don't get called, either. So my suspicion might fall on either the cross-compilation issue or the insufficiency of the current patch. Could someone show the cross-compilation recipe, or help with my issue? I would really appreciate that.

We should close this as it has been pushed upstream as part of https://reviews.llvm.org/rG847de9c332775d1841fec9fea5cb5c41592a4c8f