[RuntimeDyld][RISCV] Minimal riscv64 support

[dnpetrov-sc]

Summary

llvm-exegesis uses JIT to compile synthetic code snippets.

Focus of this patch is to provide support necessary for llvm-exegesis to work on the riscv64 platform.
It also fixes the following LLVM-Unit OrcJIT tests on riscv64:

ExecutionEngine/Orc/./OrcJITTests/OrcCAPITestBase.AddObjectBuffer
ExecutionEngine/Orc/./OrcJITTests/OrcCAPITestBase.ExecutionTest
ExecutionEngine/Orc/./OrcJITTests/OrcCAPITestBase.ResourceTrackerDefinitionLifetime
ExecutionEngine/Orc/./OrcJITTests/OrcCAPITestBase.ResourceTrackerTransfer

Some of the OrcJIT tests keep failing on riscv64, just with a different message - unimplemented relocation type X instead of unsupported processor type.

NB I don't have commit access to the LLVM repo, so someone should commit on my behalf.

[lhames]

Thanks very much for raising this issue Dmitrii. We actually have relatively advanced RISCV support JITLink now, but it doesn't look like llvm-exegesis is using it. We should aim to update the llvm-exegesis tool, rather than RuntimeDyld. Do you know who is responsible for that tool these days?

Some of the OrcJIT tests keep failing on riscv64, just with a different message - unimplemented relocation type X instead of unsupported processor type.

Can you share the failure message and context? This may be of interest to @StephenFan.

[lhames]

@courbet Are you the right person to talk to about the llvm-exegesis tool? Can it be moved over to ORC/JITLink?

It doesn't look like it's using any of the JITEventListeners, so I'm hoping we can just switch to LLJIT (maybe with some conditional hacks to support getting the function range in RuntimeDyld vs JITLink). Does that sound reasonable?

[dnpetrov-sc]

In D127842#3586397, @lhames wrote:

Thanks very much for raising this issue Dmitrii. We actually have relatively advanced RISCV support JITLink now, but it doesn't look like llvm-exegesis is using it. We should aim to update the llvm-exegesis tool, rather than RuntimeDyld. Do you know who is responsible for that tool these days?

Some of the OrcJIT tests keep failing on riscv64, just with a different message - unimplemented relocation type X instead of unsupported processor type.

Can you share the failure message and context? This may be of interest to @StephenFan.

Build on riscv64, run ExecutionEngine tests.
'main' fails with fatal error "Unsupported CPU type!" (in RuntimeDyldELF::resolveRelocation, RuntimeDyldELF.cpp:1076) in the following tests:

ExecutionEngine/frem.ll
ExecutionEngine/mov64zext32.ll
ExecutionEngine/test-interp-vec-arithm_float.ll
ExecutionEngine/test-interp-vec-arithm_int.ll
ExecutionEngine/test-interp-vec-logical.ll
ExecutionEngine/test-interp-vec-setcond-fp.ll
ExecutionEngine/test-interp-vec-setcond-int.ll
ExecutionEngine/Orc/./OrcJITTests/OrcCAPITestBase.AddObjectBuffer
ExecutionEngine/Orc/./OrcJITTests/OrcCAPITestBase.ExecutionTest
ExecutionEngine/Orc/./OrcJITTests/OrcCAPITestBase.ResourceTrackerDefinitionLifetime
ExecutionEngine/Orc/./OrcJITTests/OrcCAPITestBase.ResourceTrackerTransfer

Side note: it looks like there's a bug in ELFJITLinker_riscv::applyFixup (ELF_riscv.cpp:375):

case R_RISCV_SUB6: {
  int64_t Value =
      *(reinterpret_cast<const uint8_t *>(FixupAddress.getValue())) &
      0x3f - E.getTarget().getAddress().getValue() - E.getAddend();
  *FixupPtr = (*FixupPtr & 0xc0) | (static_cast<uint8_t>(Value) & 0x3f);
  break;
}

& has lower priority. That should be (<old value> & <mask>) - <offset>, but actually is <old value> & (<mask> - <offset>), which doesn't look right. I don't have a reproducer on hand, though.

[courbet]

In D127842#3587277, @lhames wrote:

@courbet Are you the right person to talk to about the llvm-exegesis tool? Can it be moved over to ORC/JITLink?

I am. What we require is being able to work at the MC rather and IR level, because we want exact control over which instructions are emitted I'm not familiar with ORCJit, but from a quick look at the doc it looks like it should be doable: replacements for earlier LLVM JIT APIs (e.g. MCJIT).

Is MCJIT going away ?

[lhames]

In D127842#3588313, @courbet wrote:

In D127842#3587277, @lhames wrote:

@courbet Are you the right person to talk to about the llvm-exegesis tool? Can it be moved over to ORC/JITLink?

I am. What we require is being able to work at the MC rather and IR level, because we want exact control over which instructions are emitted I'm not familiar with ORCJit, but from a quick look at the doc it looks like it should be doable: replacements for earlier LLVM JIT APIs (e.g. MCJIT).

Yep -- you can definitely load up object files directly.

Is MCJIT going away ?

We don't have plans to remove it any time soon, but it's no longer being actively developed.

[asb]

I'm not very familiar with testing for RuntimeDyld, but shouldn't there be test cases added to llvm/test/ExecutionEngine/RuntimeDyld/RISCV to cover this change? (also - thanks for the contribution!)

llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.h

void resolvePPC64Relocation(const SectionEntry &Section, uint64_t Offset,
                               uint64_t Value, uint32_t Type, int64_t Addend);
 
   void resolveSystemZRelocation(const SectionEntry &Section, uint64_t Offset,
                                 uint64_t Value, uint32_t Type, int64_t Addend);
 
   void resolveBPFRelocation(const SectionEntry &Section, uint64_t Offset,
                             uint64_t Value, uint32_t Type, int64_t Addend);
 
+  void resolveRISCV64Relocation(const SectionEntry &Section, uint64_t Offset,
+                                uint64_t Value, uint32_t Type, int64_t Addend);
+
   unsigned getMaxStubSize() const override {
     if (Arch == Triple::aarch64 || Arch == Triple::aarch64_be)
       return 20; // movz; movk; movk; movk; br
     if (Arch == Triple::arm || Arch == Triple::thumb)
       return 8; // 32-bit instruction and 32-bit address
     else if (IsMipsO32ABI || IsMipsN32ABI)
       return 16;
     else if (IsMipsN64ABI)

llvm/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp

case ELF::R_BPF_64_ABS32: {
     write(isBE, Section.getAddressWithOffset(Offset), static_cast<uint32_t>(Value));
     LLVM_DEBUG(dbgs() << "Writing " << format("%p", Value) << " at "
                       << format("%p\n", Section.getAddressWithOffset(Offset)));
     break;
   }
   }
 }
 
+void RuntimeDyldELF::resolveRISCV64Relocation(const SectionEntry &Section,
+                                              uint64_t Offset, uint64_t Value,
+                                              uint32_t Type, int64_t Addend) {
+  uint8_t *LocalAddress = Section.getAddressWithOffset(Offset);
+  uint32_t *TargetPtr = reinterpret_cast<uint32_t *>(LocalAddress);
+  uint64_t FinalAddress = Section.getLoadAddressWithOffset(Offset);
+
+  LLVM_DEBUG(dbgs() << "resolveRISCV64Relocation, "
+                    << "LocalAddress: 0x" << format("%llx", LocalAddress)
+                    << " FinalAddress: 0x" << format("%llx", FinalAddress)
+                    << " Value: 0x" << format("%llx", Value) << " Addend: 0x"
+                    << format("%llx", Addend) << " Type: 0x"
+                    << format("%x", Type) << "="
+                    << getELFRelocationTypeName(ELF::EM_RISCV, Type) << "\n");
+
+  switch (Type) {
+  default:
+    report_fatal_error(
+        Twine("Relocation type not implemented yet: ")
+            .concat(getELFRelocationTypeName(ELF::EM_RISCV, Type)));
+    break;
+  case ELF::R_RISCV_SET6: {
+    uint64_t Result = Value + Addend;
+    uint32_t RawData = read32le(TargetPtr);
+    write32le(TargetPtr, (RawData & 0xFFFF'FFC0) | (Result & 0x3F));
+    break;
+  }
+  case ELF::R_RISCV_SUB6: {
+    uint64_t Result = Value + Addend;
+    uint32_t RawData = read32le(TargetPtr);
+    uint32_t Word6 = ((RawData & 0x3F) - Result) & 0x3F;
+    write32le(TargetPtr, (RawData & 0xFFFF'FFC0) | Word6);
+    break;
+  }
+  case ELF::R_RISCV_32_PCREL: {
+    uint64_t Result = Value + Addend;
+    write32le(TargetPtr, Result - FinalAddress);
+    break;
+  }
+  case ELF::R_RISCV_ADD32: {
+    uint64_t Result = Value + Addend;
+    uint32_t RawData = read32le(TargetPtr);
+    write32le(TargetPtr, RawData + Result);
+    break;
+  }
+  case ELF::R_RISCV_SUB32: {
+    uint64_t Result = Value + Addend;
+    uint32_t RawData = read32le(TargetPtr);
+    write32le(TargetPtr, RawData - Result);
+    break;
+  }
+  }
+}
+
 // The target location for the relocation is described by RE.SectionID and
 // RE.Offset.  RE.SectionID can be used to find the SectionEntry.  Each
 // SectionEntry has three members describing its location.
 // SectionEntry::Address is the address at which the section has been loaded
 // into memory in the current (host) process.  SectionEntry::LoadAddress is the
 // address that the section will have in the target process.
 // SectionEntry::ObjAddress is the address of the bits for this section in the
 // original emitted object image (also in the current address space).
case Triple::ppc64le:
     break;
   case Triple::systemz:
     resolveSystemZRelocation(Section, Offset, Value, Type, Addend);
     break;
   case Triple::bpfel:
   case Triple::bpfeb:
     resolveBPFRelocation(Section, Offset, Value, Type, Addend);
     break;
+  case Triple::riscv64:
+    resolveRISCV64Relocation(Section, Offset, Value, Type, Addend);
+    break;
   default:
     llvm_unreachable("Unsupported CPU type!");
   }
 }
 
 void *RuntimeDyldELF::computePlaceholderAddress(unsigned SectionID, uint64_t Offset) const {
   return (void *)(Sections[SectionID].getObjAddress() + Offset);
 }