This is an archive of the discontinued LLVM Phabricator instance.

[LLDB][RISCV] Add RVM and RVA instruction support for EmulateInstructionRISCV
ClosedPublic

Authored by Emmmer on Sep 11 2022, 8:28 AM.

Details

Summary

Add:

  • RVM and RVA instructions sets.
  • corresponding unittests.

Further work:

  • implement RVC, RVF, RVD, and RVV extension.

Diff Detail

Event Timeline

Emmmer created this revision.Sep 11 2022, 8:28 AM
Herald added a project: Restricted Project. · View Herald TranscriptSep 11 2022, 8:28 AM
Emmmer requested review of this revision.Sep 11 2022, 8:28 AM

General comment on all this, and forgive me if this sounds outlandish but are we going to end up with a complete riscv emulator here? Given that until hardware single step arrives we're going to have to emulate any possible instruction that we might breakpoint. Or am I mistaken here?

If it is the case that we're going to end up with almost every instruction here, I don't necessarily object on a technical level. Without spec updates there is no alternative. However, I do think that it would be worth getting someone who works on riscv daily to review for instruction correctness. I've written emulators before and getting one instruction slightly off can be a nightmare to find after the fact.

Ideally you could run a whole pile of programs single stepped and verify their results but obviously that is easier said than done. Something like csmith can give you programs, but not ones with specific results. Might find some crashes though if you got some memory operands wrong somewhere. Anyway, just some ideas because I am concerned about how sustainable this effort is (and how much time it'll cost you to get right).

lldb/source/Plugins/Instruction/RISCV/EmulateInstructionRISCV.cpp
304

Can you make these a bit more obvious? Like:

// RV32I & RV64I (integer instructions)

Probably obvious if you live in riscv land but with the amount of single letter extensions it's hard to tell sometimes.

Also if this is a section marker, as opposed to marking this one function, make it more obvious like:

<empty line>
// RV32I & RV64I //
<empty line>

I don't see an llvm style suggestion for that but you get the idea. Otherwise the reader thinks it's attached to that one function.

383

Why do we need a special case here? Looks like the same thing with more steps. If it isn't please add a comment explaining what corner case this is catching.

776

In general prefer smaller patches. E.g. Adding all these MUL could be its own patch and you'd have less reason to leave yourself these markers.

TODOs in committed code tend to stay there forever.

793

Is this effected by the target's current FP mode?

831

Don't do this for this change, but I've seen so many of this pattern I'm sure you could reduce the boilerplate a bit.

struct RS1RS2Values {
  RegisterValue rs1;
  RegisterValue rs2;
  bool valid;
};
RS1RS2Values getRS1RS2(EmulateInstructionRISCV &emulator, uint32_t inst) {
  // decode both and read, set valid false on failure of either.
}
...
auto regvals = getRS1RS2(emulator, inst);
if (!regvals.valid)
  return false;
838

This is the dividend, right? You could use dividend directly to make that clear.

973

Please document the return value of this function. Given that 0 is a valid memory address (ok, in 99% of circumstances it'll fault but there is a mapping there of some form).

From what I see, 0 means that the address in reg was not suitable for doing an atomic access. And perhaps the instruction itself makes the assumption that 0 will never be used as an address, but it would be good to document that.

(given that address 0 is aligned to any size you want, if you want to be pedantic about it)

974

Can align be negative, is unsigned fine here?

983

You could oneline this:

return addr % align == 0 ? addr : 0;
1090

For this and the prior 2 please document which fields these masks are looking for and what their expected values are.

For example you might say I'm looking for op=4 rd=5, foo=1, whatever names your spec uses.

1095

For another change, but this should really be llvm::Optional<lldb::addr_t> ReadPC();. Your use here is fine given the existing API though.

1101

I assume +4 is a safe assumption for these "atomic sequences" ? As in, no one's gonna put compressed code in here and if they did, it wouldn't be a valid sequence.

Can you write a comment with some example assembly to show that? E.g.

// An atomic sequence must be of the form
// instr1 a b c
// instr2 d e f
// ....
lldb/source/Plugins/Instruction/RISCV/EmulateInstructionRISCV.h
90

Try using llvm::Optional<T> for this, it will save having to remember to make success false before calling the function.

And if you're not going to do that, at least make success a bool& (even if you need to take its address to pass down to ReadMemoryUnsigned).

91

Stray newline.

lldb/unittests/Instruction/RISCV/TestRISCVEmulator.cpp
65

Do you expect to hit this in working, normal testing? If not there should be an assert.

77

Also assert here if this is an unexpected case.

Do riscv extensions come with any kind of assembly self test suites? If such a thing existed you could single step those as a way to test this sort of thing.

General comment on all this, and forgive me if this sounds outlandish but are we going to end up with a complete riscv emulator here? Given that until hardware single step arrives we're going to have to emulate any possible instruction that we might breakpoint. Or am I mistaken here?

fwiw we had to do this for the earliest iphones, where there were no hardware breakpoints and no instruction single step. We had a tiny engine that could calculate the address of the next instruction on those armv6 chips. It was pretty limited in what it needed to be able to do, but I guess extending the instruction emulation engine to support this use case is reasonable. It really only needs to emulate instructions well enough to track where registers are spilled to stack, saved in other registers, etc, for unwinding.

After one or two revisions, we got hardware breakpoints and were able to set a "stop when pc != current address" which was effectively single step. Then in ARMv8 AArch32/AArch64 gave us genuine instruction step and that's what we use now.

We had a tiny engine that could calculate the address of the next instruction on those armv6 chips. It was pretty limited in what it needed to be able to do, but I guess extending the instruction emulation engine to support this use case is reasonable. It really only needs to emulate instructions well enough to track where registers are spilled to stack, saved in other registers, etc, for unwinding.

That was poorly worded. The emulation engine right now only needs to track side effects relevant to unwind. It doesn't track control flow side effects of instructions, short of branches within a function where we "forward" the unwind state to the target instruction. The instruction emulation engine (today) runs over a function statically, and generates an intermediate representation of the unwind state at every instruction point.

What you'd need for instruction single step is a similar intermediate representation that expressed branch behavior for every branchable instruction. Does this instruction depend on a condition flag? Does it branch to an address in a register? That kind of thing.

Or you can do the approach debugserver took so many years ago, where we are stopped at a specific instruction, and we decode it well enough to know where it will execute next, no intermediate representation of those details.

All of this assuming David is correct, that this is the problem being looked at.

Or you can do the approach debugserver took so many years ago, where we are stopped at a specific instruction, and we decode it well enough to know where it will execute next, no intermediate representation of those details.

And thinking about it further, the whole point of emulating all of the instructions in a function is that we need the full sequence to generate accurate unwind records. If we're only interested in what the next instruction address will be, we need the current register context and know how to decode one instruction. It's a bit of a different problem.

fwiw we had to do this for the earliest iphones, where there were no hardware breakpoints and no instruction single step. We had a tiny engine that could calculate the address of the next instruction on those armv6 chips. It was pretty limited in what it needed to be able to do, but I guess extending the instruction emulation engine to support this use case is reasonable. It really only needs to emulate instructions well enough to track where registers are spilled to stack, saved in other registers, etc, for unwinding.

Yes, riscv does not currently implement hardware breakpoints or instruction single step.

Almost every instruction in riscv is able to read and write to registers (rs1, rs2, and rd), so for those developers who wish to control every register, it is necessary to simulate every instruction completely.

For example, after an instruction that changes a register, the program may use the JALR to jump to the address stored in that register.

Ideally you could run a whole pile of programs single stepped and verify their results but obviously that is easier said than done. Something like csmith can give you programs, but not ones with specific results. Might find some crashes though if you got some memory operands wrong somewhere. Anyway, just some ideas because I am concerned about how sustainable this effort is (and how much time it'll cost you to get right).

For now we have few method to test a program, the best way is using unittest to cover this emulator.

Do riscv extensions come with any kind of assembly self test suites? If such a thing existed you could single step those as a way to test this sort of thing.

Yes, we can try to use this test suite https://github.com/riscv-software-src/riscv-tests

Almost every instruction in riscv is able to read and write to registers (rs1, rs2, and rd), so for those developers who wish to control every register, it is necessary to simulate every instruction completely.

For example, after an instruction that changes a register, the program may use the JALR to jump to the address stored in that register.

So can most instructions write to the PC as one of the registers? I see that if you were on the following instruction, you'd have to emulate it to know the place it jumps to:

add pc, #4

On AArch64 the PC is often not allowed so if we were to emulate there, it cuts down the work needed a lot.

With the jalr isn't it more like:

add r1, #4
jalr r1

So we could (or do) put a breakpoint on the jalr because we know that the add doesn't do a branch itself. Once we hit the jalr we can read registers to get r1 and know where to break next.

Meaning for cases like that you only need a "can this branch" rather than working out the whole thing. Of course if almost any instruction can write to the pc, "can branch" is almost always true.

Yes, we can try to use this test suite https://github.com/riscv-software-src/riscv-tests

Cool! Well, I'm not requiring you to do that but keep it in mind if you end up spending a lot of time bug fixing this code.
(random idea: make lldb's emulation dump its state like the Arm emulation tests do, in a format that matches some existing riscv emulator, diff the two).

On the review side, @asb can you reccomend anyone who would be able to review the riscv specifics? (just because I remember you have done a lot of work on riscv)

(again not blocking this change on that, but I think it would be more sustainable going forward if there was a riscv specific reviewer too)

PC isn't exposed as a GPR on RISC-V, only explicit branch/jump instructions can alter control flow

PC isn't exposed as a GPR on RISC-V, only explicit branch/jump instructions can alter control flow

Thanks. In which case does that mean we only need to emulate instructions that would alter control flow? Perhaps that is technically true but not a good experience in practice.

I will put some time into reading exactly how lldb implements software single step but in the meantime maybe @jasonmolenda will know from experience.

Emmmer updated this revision to Diff 460135.Sep 14 2022, 9:40 AM
Emmmer marked 16 inline comments as done.
Emmmer edited the summary of this revision. (Show Details)

address review comments.

Emmmer added inline comments.Sep 14 2022, 9:42 AM
lldb/source/Plugins/Instruction/RISCV/EmulateInstructionRISCV.cpp
793

No. It does not affect.

Mostly grumbling about optional from me :) Once that is addressed I think that'll be it.

For future patches please keep the number of added instructions small (many patches is fine, just keep each review manageable) and again, if you can find a riscv expert to review the specifics of the instructions that would be a massive help.

lldb/source/Plugins/Instruction/RISCV/EmulateInstructionRISCV.cpp
247–248

My point with the optional is that Optional<T> is equivalent to T and bool. So you're almost there.

I should have given an example, this is what I meant:

template <typename T>
static std::enable_if_t<std::is_integral_v<T>, llvm::Optional<T>>
ReadMem(EmulateInstructionRISCV &emulator, uint64_t addr) {
  // local success for the underlying API
  bool success = false;
  T result = 0;
  // call that function which sets success and value
  if (!success)
    return {}; // aka return false
  return value;
}

Which means you can call it like this:

llvm::Optional<uint64_t> value = ReadMem<uint64_T>(emulator, addr);
if (!value)
  return false;
// otherwise use *value

Which prevents you from doing this by accident:

bool success = false;
some_function(&success)
if (!success)
  return false;
// Oops forgot to reset success, sure hope some_function initialises it for me
some_function(&success);
if (!success)
  return false;

See what I mean? Granted, our functions probably set success whether they fail or not, but this way you encode it into the types and we aren't returning things by ref which requires the reader to go look at how the function uses that ref/pointer.

381

I'm still wondering why we need a special case here, though adding a comment that this also covers the pseudo instruction is
a good idea regardless.

if (imm == 1) {
  if (rs.rs1().trunc<uint64_t>() == 0)
    return WriteRegister(emulator, DecodeRD(inst),
                         RegisterValue(uint64_t(1)));
  return WriteRegister(emulator, DecodeRD(inst), RegisterValue(uint64_t(0)));
}
  • Assume imm is 1
  • If rs1 is 0 then set rd to 1
  • If rs1 is not 0 then set rd to 0
uint64_t result = rs.rs1().trunc<uint64_t>() < uint64_t(imm);
return WriteRegister(emulator, DecodeRD(inst), RegisterValue(result));

Same thing with this code:

  • Assume imm is 1
  • result = rs1 < 1
  • if rs1 is 0 then set rd to 1
  • if rs1 is not 0 then set rd to 0

Am I missing something? Seems like the same logic works for both and we can just use the second implemntation.

1053

Two things:

  1. I mistook RegisterContextUnwind::ReadPC for this one, which is why I thought it was an existing API.
  2. The point was that these two are equivalent from a types point of view:
lldb::addr_t ReadPC(bool *success);
llvm::Optional<lldb::addr_t> ReadPC();

Except that the second is easier to handle (feel free to disagree, I am big optional enthusiast so I am biased here) and is more obvious when you read code that calls that function. In that you don't have to read the function, the type tells you everything you need to know.

Also, if you're gonna have a function that returns by pointer, but never checks that that pointer is null, just make it a ref. ReadPC(bool &success). If some underlying function N levels down insists on taking a pointer, keep it as a ref for as long as possible before converting it.

This is not a comment on your choice here by the way, lldb has a lot, a lot, of this pattern that I want to replace as new code gets added. So I appreciate you sticking with the local style but in this case there's reason to modernise.

1154

These comments are exactly what I wanted, thanks.

1201

sigle -> single and add a full stop on the end of the sentence.

Emmmer updated this revision to Diff 460438.Sep 15 2022, 9:15 AM
Emmmer marked 3 inline comments as done.

address review comments.

For future patches please keep the number of added instructions small (many patches is fine, just keep each review manageable) and again, if you can find a riscv expert to review the specifics of the instructions that would be a massive help.

Sorry for the inconvenience, I will pay attention to the size of the patch next time.

lldb/source/Plugins/Instruction/RISCV/EmulateInstructionRISCV.cpp
381

Sorry, seems I mistook the example given by the spec as a special case because they use "NOTE".

1053

I just tried llvm::Optional, but it didn't reduce the number of null-checks, and we use Optional for that purpose (by chaining them with .map or .flatMap).

At the moment it seems to be missing some handy ADT functions like and_then : Optional<T> -> (T -> Optional<U>) -> Optional<U>, I can add them to llvm::Optional in the next patch together with some modernization.

BTW, The emulation API uses mostly bool &success to return the result, It is sure a huge refactoring.

Emmmer updated this revision to Diff 460652.Sep 15 2022, 11:26 PM

change ReadPC(bool *success) -> ReadPC(bool &success)

DavidSpickett accepted this revision.Sep 16 2022, 2:05 AM

One idea I had to help review future patches, if the authors of the llvm support for the extension are still active you could add them as reviewers. I'm happy to continue reviewing of course, just aware that I'm not doing the most in depth job here.

This change LGTM.

lldb/source/Plugins/Instruction/RISCV/EmulateInstructionRISCV.cpp
1053

The point is not that optional reduces checks for false, but it reduces the number of ways you can screw that up and encodes the intent in the types more accurately. You're not wrong that it isn't as nice as say rust's foo()?.bar() and the like, but it's better than returning a bool by ref and forgetting to reset it each time.

Yes refactoring the emulation API is a big job that will likely never happen, which is why writing new code in a better style is the way to go about it.

But hey, let's not let style get in the way of the point here which is to improve riscv support, and leave things as they are.

I'll get off my soapbox and maybe I'll implement what I'm talking about at some point in the future :)

This revision is now accepted and ready to land.Sep 16 2022, 2:05 AM

Hey @Emmmer the unit test is throwing a UBSAN failure, could you please take a look?

https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-sanitized/lastFailedBuild/testReport/lldb-unit/Instruction___EmulatorTests_3/26/

Stacktrace
Script(shard):
--
GTEST_OUTPUT=json:/Users/buildslave/jenkins/workspace/lldb-cmake-sanitized/lldb-build/tools/lldb/unittests/Instruction/./EmulatorTests-lldb-unit-48390-3-26.json GTEST_SHUFFLE=1 GTEST_TOTAL_SHARDS=26 GTEST_SHARD_INDEX=3 /Users/buildslave/jenkins/workspace/lldb-cmake-sanitized/lldb-build/tools/lldb/unittests/Instruction/./EmulatorTests
--

Note: This is test shard 4 of 26.
Note: Randomizing tests' orders with a seed of 42603 .
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from RISCVEmulatorTester
[ RUN      ] RISCVEmulatorTester.TestAtomicSequence
/Users/buildslave/jenkins/workspace/lldb-cmake-sanitized/llvm-project/lldb/unittests/Instruction/RISCV/TestRISCVEmulator.cpp:254:3: runtime error: store to misaligned address 0x61b000001034 for type 'uint64_t' (aka 'unsigned long long'), which requires 8 byte alignment
0x61b000001034: note: pointer points here
  af 27 04 10 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /Users/buildslave/jenkins/workspace/lldb-cmake-sanitized/llvm-project/lldb/unittests/Instruction/RISCV/TestRISCVEmulator.cpp:254:3 in 

--
exit: -6
--
shard JSON output does not exist: /Users/buildslave/jenkins/workspace/lldb-cmake-sanitized/lldb-build/tools/lldb/unittests/Instruction/./EmulatorTests-lldb-unit-48390-3-26.json

Hey @Emmmer the unit test is throwing a UBSAN failure, could you please take a look?

https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-sanitized/lastFailedBuild/testReport/lldb-unit/Instruction___EmulatorTests_3/26/

Stacktrace
Script(shard):
--
GTEST_OUTPUT=json:/Users/buildslave/jenkins/workspace/lldb-cmake-sanitized/lldb-build/tools/lldb/unittests/Instruction/./EmulatorTests-lldb-unit-48390-3-26.json GTEST_SHUFFLE=1 GTEST_TOTAL_SHARDS=26 GTEST_SHARD_INDEX=3 /Users/buildslave/jenkins/workspace/lldb-cmake-sanitized/lldb-build/tools/lldb/unittests/Instruction/./EmulatorTests
--

Note: This is test shard 4 of 26.
Note: Randomizing tests' orders with a seed of 42603 .
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from RISCVEmulatorTester
[ RUN      ] RISCVEmulatorTester.TestAtomicSequence
/Users/buildslave/jenkins/workspace/lldb-cmake-sanitized/llvm-project/lldb/unittests/Instruction/RISCV/TestRISCVEmulator.cpp:254:3: runtime error: store to misaligned address 0x61b000001034 for type 'uint64_t' (aka 'unsigned long long'), which requires 8 byte alignment
0x61b000001034: note: pointer points here
  af 27 04 10 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /Users/buildslave/jenkins/workspace/lldb-cmake-sanitized/llvm-project/lldb/unittests/Instruction/RISCV/TestRISCVEmulator.cpp:254:3 in 

--
exit: -6
--
shard JSON output does not exist: /Users/buildslave/jenkins/workspace/lldb-cmake-sanitized/lldb-build/tools/lldb/unittests/Instruction/./EmulatorTests-lldb-unit-48390-3-26.json

Thanks for your feedback! it was fixed in rG07d0ef306b74fbcede432ad3480d2f299c051a98

Awesome, bot is green again!