[XRay] Add support for RISCV

[ashwin98]

Summary

Adds XRay support for RISCV. Currently, only RISCV64 has been tested. Changes required to add support for RISCV32 are present, but are commented out.

The modifications (as in the case of RISCVAsmPrinter) were made with the intention of making as few changes/touching as few files as possible to reduce the possibility of breaking any other functionality.

The double precision floating point extension was assumed to be part of the ISA while designing the trampolines (and this has been enforced in the Subtarget file). This implementation currently does not support compressed instructions.

[jrtc27]

1. Please upload patches with full context
2. You should not need to have separate xray_riscv32/64.cpp, a single shared file with a small amount of XLEN-based conditions should suffice and reduce a whole load of code duplication. Possibly also applies to the trampoline assembly but maybe not, there are lots of constants and lw/ld's in there... though would be nice if that were macro'd so they don't get out of sync
3. Why comment out riscv32? At least uncomment everything except the one CMake place that says it exists, having ~10 different places with commented-out code is ugly.

llvm/lib/Target/RISCV/RISCVAsmPrinter.cpp

260

# before immediates is an AArch64-ism, on RISC-V the syntax is just the plain integer

261

This doesn't match what you actually do

272

addExpr seems fine? That's what you use for an MCSRE. Don't see why it needs commentary here.

276

?

284

Indentation is off

[jrtc27]

compiler-rt/cmake/Modules/AllSupportedArchDefs.cmake

77

Does that actually only comment out RISCV32? Phabricator's syntax highlighting thinks not, but it could just be overly simplistic.

compiler-rt/lib/xray/xray_riscv32.cpp

11 ↗

(On Diff #402107)

You sure about that 64-bit?

24 ↗

(On Diff #402107)

Whitespace here is a mess

33 ↗

(On Diff #402107)

Register names have an x prefix not an r prefix

96 ↗

(On Diff #402107)

Why are these comments way off to the right like that? This is borderline unreadable.

101 ↗

(On Diff #402107)

No. %hi(0x800) *is* 1. Having the if/else results in double-counting were this to be treated as actual assembly. The +1 is part of how hi/lo relocation pairs are defined.

127 ↗

(On Diff #402107)

is how you implement hi/lo pairs in a branchless manner, exploiting carry propagation

llvm/lib/Target/RISCV/RISCVAsmPrinter.cpp

255

These magic numbers need explaining

257

This sentence ends abruptly (and why is the paragraph wrapped at such a tiny character count?)

llvm/test/CodeGen/RISCV/xray-attribute-instrumentation.ll

2

-filetype=asm is redundant, as is -o -, and you should probably be passing -verify-machineinstrs like our other tests.

Can this be made to work with update_llc_test_checks.py? My guess is probably not due to the filtering it does, but it would be nice if it can be.

7

Do you really need all of those attributes? I highly doubt noinline is needed, sane for uwtable. I don't think you'll get CFI either without nounwind, but could be wrong, never quite know when it's needed to squash CFI.

67

Not sure what this {{.*}} achieves

69

Given you've already explicitly stated the label names in the CHECK lines for the function body it would seem prudent to verify the right ones are used here

[ashwin98]

Thank you for your feedback! I could combine the riscv32 and 64 cpp files with some xlen conditions if that will work better, but that might take a bit of a hit in terms of readability (do I explain both sleds in the comments preceding the implementation). I have commented riscv32 out because I haven't managed to test it out yet, I've had some difficulty getting llvm set up for riscv 32 - I will leave riscv32 commented out only in the cmake file.

I wasn't too sure about how to work around sign extension in RISCV, which you have picked up on - adding 0x800 wasn't something I thought of. Thinking about it a bit more, it makes sense, we're not adding 4096 like how I was, though it has the same effect; I'll reason it out, I'm sure carry propagation deals with it like you've mentioned. I'll update the code to reflect the same.

I had a related question with respect to the 64 bit sleds though - given that lui is also sign extended, we need a work around for it as well while constructing the 32 bit values, and while combining the 2 32 bit values into a 64 bit value. I have currently been getting rid of the upper 32 bits by performing a left shift followed by a right shift, but I'm sure there is a better solution to it.

[jrtc27]

In D117929#3263462, @ashwin98 wrote:

Thank you for your feedback! I could combine the riscv32 and 64 cpp files with some xlen conditions if that will work better, but that might take a bit of a hit in terms of readability (do I explain both sleds in the comments preceding the implementation).

In one sense yes it will be slightly less readable. In another sense it actually makes it more readable, because seeing the XLEN-based conditions makes it clear what things are word-sized and what things are explicitly 32-bit (in the RV32 code any LW is unclear whether it's loading an int, a size_t or a void *). And yes, you explain both, but most of it is the same so can be combined into a single explanation, e.g. like LLD does for documenting its PLT stubs in lld/ELF/Arch/RISCV.cpp.

I wasn't too sure about how to work around sign extension in RISCV, which you have picked up on - adding 0x800 wasn't something I thought of. Thinking about it a bit more, it makes sense, we're not adding 4096 like how I was, though it has the same effect; I'll reason it out, I'm sure carry propagation deals with it like you've mentioned. I'll update the code to reflect the same.

It's important that it's only added when computing the HI relocation. As an example, %hi(0x81734) (to pick a number at random that's not too boring) would be (0x81734 + 0x800) >> 12 = 0x81f34 >> 12 = 0x81, whereas %hi(0x81934) = (0x81934 + 0x800) >> 12 = 0x82134 >> 12 = 0x82. You can see how the adding 1 << 11 combined with right-shifting by one more results in adding one to the upper 20 bits if and only if bit 11 of the input is 1; if it's 0 there is no carry out so the only bit that's modified is bit 11, which the right shift will shift out.

I had a related question with respect to the 64 bit sleds though - given that lui is also sign extended, we need a work around for it as well while constructing the 32 bit values, and while combining the 2 32 bit values into a 64 bit value. I have currently been getting rid of the upper 32 bits by performing a left shift followed by a right shift, but I'm sure there is a better solution to it.

That's one way of doing it, though requires more than one temporary register. RISCVMatInt's generateInstSeqImpl has an alternate sequence documented for the general case (as well as various optimised special cases) that is the same number of instructions but only needs one register. If you have multiple registers then your sequence probably performs better on an out-of-order core. Synthesising 64-bit addresses is pretty inefficient; you might prefer instead just loading from a constant pool adjacent to the code.

[ashwin98]

Updated the diff, made the following changes:

1. Merged the riscv files into xray_riscv.cpp and removed the if-else code for %hi()
2. Cleaned up the issues related to indenting and comments in RISCVAsmPrinter.cpp
3. Updated the test file to pass -verify-machineinstrs and remove unnecessary attributes as well as {{.*}}s
4. Fixed riscv32 comments - it is now only commented out in cmake/Modules/AllSupportedArchDefs.cmake

I have been testing this patch on qemu using ubuntu for riscv64, the comment that Phabricator detects in the supported architecture definitions cmake file is probably an issue with syntax highlighting. Nevertheless, we could instead comment out riscv32 in clang/lib/Driver/XRayArgs, which would also throw up an error during compilation stating that the target is not supported.

[dberris]

It looks like you'll need to address the lint issues (using clang-format).

I'm not an expert on RISCV assembly so you might need to get someone familiar with the ISA reviewing this too.

For tests, I recommend also adding some for the tooling that consumes the sections of the binary where we expect metadata that XRay expects to be present. You should be able to re-use/extend some of the tests we already have for those to see the end-to-end result (compiling and linking a binary which has Xray instrumentation sleds and associated maps).

[ashwin98]

Made changes to handle lint issues.

[ashwin98]

Fixed another lint issue, they should all be done for now hopefully.

@dberris Sorry, I'm a little confused, do you mean I need to update some of the clang tests for XRay instrumentation to test compilation and linking, or add new ones for the llvm-xray tool under llvm/test/tools?

[jrtc27]

compiler-rt/lib/xray/xray_riscv.cpp

123–124

Why? You shift this whole thing left by 32 later

128–129

You might be able to avoid this by adding 0x80000800 before computing "%higher" and "%highest" (feels rather MIPSy... not official things)? Not sure, would need to think this through more, but it feels like it should be possible...

163

This is definitely wrong; you probably mean (((TracingHook >> 32) + 0x800) >> 12) & 0xfffff?

llvm/lib/Target/RISCV/RISCVAsmPrinter.cpp

105–106

"Keeping changes minimal" seems like the kind of thing that belongs in a commit message, not a code comment

llvm/test/CodeGen/RISCV/xray-attribute-instrumentation.ll

2–3

Triples are overly verbose; riscv32-unknown-elf is normally just written riscv32, and riscv32-unknown-linux-gnu as riscv64-linux-gnu, though I don't see what point having both serves, we normally only use the bare-metal triples unless something has an OS-specific aspect

[ashwin98]

compiler-rt/lib/xray/xray_riscv.cpp

123–124

Right, I'll update this

128–129

Adding 0x80000000 may be enough, the lower 12 bits should be taken care of when we construct the lower 32 bits, if we choose to use two registers. If we wish to use one register to load all values, then 0x80000800 may be needed - I'm not too sure. About the MIPS and AArch terminology in some places - yeah if there's anything that is not official or consistent with RISCV, please let me know, I frequently consulted the files for the other ISAs to figure out XRay's implementation and ended up using inconsistent terminology at some places.

163

True

[ashwin98]

Updated the riscv64 sled function to fix the addition/shift operations and get rid of the superfluous slli and srli instructions. Cut out unnecessary comments in the ASM Printer.

[ashwin98]

llvm/test/CodeGen/RISCV/xray-attribute-instrumentation.ll

2–3

I will cut it down to just the ones with linux. Clang has a check to see if the OS is Linux when passed the flag for XRay instrumentation.

[ashwin98]

Removed extra triples from the test.

[dberris]

In D117929#3276843, @ashwin98 wrote:

Fixed another lint issue, they should all be done for now hopefully.

@dberris Sorry, I'm a little confused, do you mean I need to update some of the clang tests for XRay instrumentation to test compilation and linking, or add new ones for the llvm-xray tool under llvm/test/tools?

Yes to both. :)

There are already some tests that ensure the supported triples build with XRay instrumentation and that we can get the instrumentation map with the tooling on binaries generated with clang+lld. Extending those to support risc64 and ensure we're able to find the instrumentation with the tooling gives us higher confidence that this will work when the patch is accepted.

[ashwin98]

In D117929#3299751, @dberris wrote:

In D117929#3276843, @ashwin98 wrote:

Fixed another lint issue, they should all be done for now hopefully.

@dberris Sorry, I'm a little confused, do you mean I need to update some of the clang tests for XRay instrumentation to test compilation and linking, or add new ones for the llvm-xray tool under llvm/test/tools?

Yes to both. :)

There are already some tests that ensure the supported triples build with XRay instrumentation and that we can get the instrumentation map with the tooling on binaries generated with clang+lld. Extending those to support risc64 and ensure we're able to find the instrumentation with the tooling gives us higher confidence that this will work when the patch is accepted.

Before running the tests, I've been testing some of the subcommands with the generated logs. When xray account or stack was run directly on the log file, the data for the instrumented functions was output (I believe the XRay tool recognizes it as a YAML file and processes it as such). However, when passing the --instr_map flag, issues crop up. I originally ran into an error about the executable not being an ELF binary. I confirmed that clang was generating ELF binaries. There was a test checking the triple architecture in InstrumentationMap.cpp that seemed to be causing this.

On including RISCV64 to the check, I see new issues while reading the instrumentation map. How do I verify that the instrumentation map is being generated correctly (or if there is a problem with it), and where is the code that is responsible for the generation of the instrumentation map (is it in xray_init.cpp)? I'm not sure if this is a RISCV compatibility issue with the xray tool, or if I've missed something that is causing problems during the instrumentation map initialization.

[ashwin98]

In D117929#3314355, @ashwin98 wrote:

In D117929#3299751, @dberris wrote:

In D117929#3276843, @ashwin98 wrote:

Fixed another lint issue, they should all be done for now hopefully.

@dberris Sorry, I'm a little confused, do you mean I need to update some of the clang tests for XRay instrumentation to test compilation and linking, or add new ones for the llvm-xray tool under llvm/test/tools?

Yes to both. :)

There are already some tests that ensure the supported triples build with XRay instrumentation and that we can get the instrumentation map with the tooling on binaries generated with clang+lld. Extending those to support risc64 and ensure we're able to find the instrumentation with the tooling gives us higher confidence that this will work when the patch is accepted.

Before running the tests, I've been testing some of the subcommands with the generated logs. When xray account or stack was run directly on the log file, the data for the instrumented functions was output (I believe the XRay tool recognizes it as a YAML file and processes it as such). However, when passing the --instr_map flag, issues crop up. I originally ran into an error about the executable not being an ELF binary. I confirmed that clang was generating ELF binaries. There was a test checking the triple architecture in InstrumentationMap.cpp that seemed to be causing this.

On including RISCV64 to the check, I see new issues while reading the instrumentation map. How do I verify that the instrumentation map is being generated correctly (or if there is a problem with it), and where is the code that is responsible for the generation of the instrumentation map (is it in xray_init.cpp)? I'm not sure if this is a RISCV compatibility issue with the xray tool, or if I've missed something that is causing problems during the instrumentation map initialization.

I traced the root cause of the issue. It seems to be stemming from the instrumentation map's relocation handling, specifically, at line 129 of InstrumentationMap.cpp, when we try to extract the load address of the symbol. I believe that part of code was written keeping AArch64 in mind, but I'm not too sure about what changes will need to be made to add RISCV64 compatibility, I'm trying to figure it out.

[dberris]

In D117929#3317813, @ashwin98 wrote:

In D117929#3314355, @ashwin98 wrote:

On including RISCV64 to the check, I see new issues while reading the instrumentation map. How do I verify that the instrumentation map is being generated correctly (or if there is a problem with it), and where is the code that is responsible for the generation of the instrumentation map (is it in xray_init.cpp)? I'm not sure if this is a RISCV compatibility issue with the xray tool, or if I've missed something that is causing problems during the instrumentation map initialization.

I traced the root cause of the issue. It seems to be stemming from the instrumentation map's relocation handling, specifically, at line 129 of InstrumentationMap.cpp, when we try to extract the load address of the symbol. I believe that part of code was written keeping AArch64 in mind, but I'm not too sure about what changes will need to be made to add RISCV64 compatibility, I'm trying to figure it out.

If you can turn the relocations you're emitting in the assembly to be relative instead of absolute when building the instrumentation map like in other architectures, then the tooling will be able to resolve them. Maybe that helps?

[MaskRay]

I am still interested in a RISC-V XRay port :)

llvm/lib/Target/RISCV/RISCVAsmPrinter.cpp

280

int

https://llvm.org/docs/CodingStandards.html#prefer-preincrement

llvm/test/CodeGen/RISCV/xray-attribute-instrumentation.ll

9

Use COUNT https://llvm.org/docs/CommandGuide/FileCheck.html#the-check-count-directive

See llvm/test/CodeGen/AArch64/xray-attribute-instrumentation.ll

55

See llvm/test/CodeGen/X86/xray-attribute-instrumentation.ll

use [[TMP:.Ltmp[0-9]+]] to avoid updating .LtmpN when there are somehow other temporary labels.

[MaskRay]

compiler-rt/lib/xray/xray_trampoline_riscv32.S

17

Omit .file

Consider using #include "../sanitizer_common/sanitizer_asm.h" like aarch64 and x86-64.
This is in case we have RISC-V for other object file formats (say Mach-O) in the future.

[ashwin98]

In D117929#4445858, @MaskRay wrote:

I am still interested in a RISC-V XRay port :)

Sorry - lost track of this, things have been hectic for the last few months, but we also seem to have got custom events working with riscv64. Should hopefully be able to get back to this by some time in August.

[ashwin98]

Ok, I've tried to make the changes you've suggested @MaskRay , let me know if I've missed something.

On another note, I lost my previous qemu VM and have rebuilt LLVM with this patch in a new one; I am now getting the following issue when I try running a binary with XRay basic enabled:

SanitizerTool: CHECK failed: sanitizer_allocator_primary32.h:292 "((res)) < ((kNumPossibleRegions))" (0x7fff93a, 0x40000) (tid=24873)

A previous thread (https://reviews.llvm.org/D87580) suggests this might be a qemu issue, but I am using qemu-system-riscv64 (and not qemu user-mode.) However, I figured I'd check anyway.

[MaskRay]

Context not available

See https://llvm.org/docs/Phabricator.html#requesting-a-review-via-the-web-interface

compiler-rt/cmake/Modules/AllSupportedArchDefs.cmake

83

Just make RISCV32 available? Otherwise we wouldn't need riscv32_SOURCES

compiler-rt/lib/xray/xray_riscv.cpp

52

For new functions, consider dropping XRAY_NEVER_INSTRUMENT. The runtime library cannot be instrumented with -fxray-instrument and I an unsure why the original impl adds a lot of XRAY_NEVER_INSTRUMENT.

53

163

(x + 0x800) >> 12 is used many times. We need a helper like lld/ELF/Arch/RISCV.cpp hi20. Ditto for lo12.

287

Unimplemented features should use TODO instead of FIXME.

llvm/lib/Target/RISCV/RISCVAsmPrinter.cpp

322

I have many comments in D140727 (LoongArch port). Many are probably useful here as well.

[ashwin98]

compiler-rt/cmake/Modules/AllSupportedArchDefs.cmake

83

Done

compiler-rt/lib/xray/xray_riscv.cpp

52

Done

53

Done

163

Done. I think I'd frequently looked at the MIPS trampolines while writing this code - I noticed that they were casting the Tracing Hook to int64_t, instead of uint64_t (and int32/uint32 for the 32 bit ISA), which requires the & operation, since >> is an arithmetic shift operation for signed integers. Changing the cast operations to uint64_t should eliminate those & operations without breaking anything else, right?

287

Done

llvm/lib/Target/RISCV/RISCVAsmPrinter.cpp

322

I had a look at this. Two things which stuck out to me (both in xray_riscv.cpp):

• changing inline static to static inline (done)
• your note about the PO_ style hurting readability when instructions are only used once. In this case, we're using some instructions repeatedly, so I'm guessing it makes sense to continue with the enum, but I can get rid of it if that works better.

[ashwin98]

Sorry - I missed adding updates with the last diff - I think I've addressed most of the comments. Just realized that the diff didn't include context, updated that as well.

[MaskRay]

compiler-rt/lib/xray/xray_trampoline_riscv32.S

55

This local symbol doesn't seem useful. We can just use numbers (1f and 1:) or LOCAL_LABEL(...), so that the symbol table will not have the unneeded symbol entries.

86

stray ASM_SIZE?

compiler-rt/lib/xray/xray_trampoline_riscv64.S

23

See the aarch64 implementation to use ASM_SIZE(__xray_FunctionEntry) to set .size

llvm/lib/Target/RISCV/RISCVAsmPrinter.cpp

185

omit braces for single-line single statement body.

llvm/lib/Target/RISCV/RISCVSubtarget.h

164

"does not support compressed instructions" makes this feature infeasible for many systems, I'll say probably almost all systems that may consider XRay.

[ashwin98]

Sorry for the delay in getting back to you. I've updated the NOP sled generation and tested it out, it now seems to work with compressed instructions - the patched sleds were identical to what I'd previously observed when compiling with march=rv64g. When I had originally written this in 2021/early 2022, some of the instructions were getting corrupted when they were being patched in - either this was a bug in my implementation that has been fixed over the course of the review process, or it might've been an issue with qemu or llvm at the time that has been fixed in the years since.

[ashwin98]

compiler-rt/lib/xray/xray_trampoline_riscv32.S

55

Done

86

Fixed - I looked at your AArch64 code and tried to follow the same process here

llvm/lib/Target/RISCV/RISCVAsmPrinter.cpp

185

Done

clang/lib/Driver/XRayArgs.cpp

if (Triple.isMacOSX()) {
     case llvm::Triple::aarch64:
     case llvm::Triple::hexagon:
     case llvm::Triple::ppc64le:
     case llvm::Triple::loongarch64:
     case llvm::Triple::mips:
     case llvm::Triple::mipsel:
     case llvm::Triple::mips64:
     case llvm::Triple::mips64el:
+    case llvm::Triple::riscv32:
+    case llvm::Triple::riscv64:
       break;
     default:
       D.Diag(diag::err_drv_unsupported_opt_for_target)
           << XRayInstrument->getSpelling() << Triple.str();
     }
   } else {
     D.Diag(diag::err_drv_unsupported_opt_for_target)
         << XRayInstrument->getSpelling() << Triple.str();

compiler-rt/cmake/Modules/AllSupportedArchDefs.cmake

set(ALL_TSAN_SUPPORTED_ARCH ${X86_64} ${MIPS64} ${ARM64} ${PPC64} ${S390X}
     ${LOONGARCH64})
 set(ALL_UBSAN_SUPPORTED_ARCH ${X86} ${X86_64} ${ARM32} ${ARM64} ${RISCV64}
     ${MIPS32} ${MIPS64} ${PPC64} ${S390X} ${SPARC} ${SPARCV9} ${HEXAGON}
     ${LOONGARCH64})
 set(ALL_SAFESTACK_SUPPORTED_ARCH ${X86} ${X86_64} ${ARM64} ${MIPS32} ${MIPS64}
     ${HEXAGON} ${LOONGARCH64})
 set(ALL_CFI_SUPPORTED_ARCH ${X86} ${X86_64} ${ARM32} ${ARM64} ${MIPS64}
     ${HEXAGON})
 set(ALL_SCUDO_STANDALONE_SUPPORTED_ARCH ${X86} ${X86_64} ${ARM32} ${ARM64}
     ${MIPS32} ${MIPS64} ${PPC64} ${HEXAGON} ${LOONGARCH64} ${RISCV64})
 if(APPLE)
 set(ALL_XRAY_SUPPORTED_ARCH ${X86_64} ${ARM64})
 else()
 set(ALL_XRAY_SUPPORTED_ARCH ${X86_64} ${ARM32} ${ARM64} ${MIPS32} ${MIPS64}
-    powerpc64le ${HEXAGON} ${LOONGARCH64})
+    powerpc64le ${HEXAGON} ${LOONGARCH64} ${RISCV32} ${RISCV64})
 endif()
 set(ALL_SHADOWCALLSTACK_SUPPORTED_ARCH ${ARM64})
 
 if (UNIX)
   if (OS_NAME MATCHES "Linux")
     set(ALL_ORC_SUPPORTED_ARCH ${X86_64} ${ARM64} ${ARM32} ${PPC64})
   else()
     set(ALL_ORC_SUPPORTED_ARCH ${X86_64} ${ARM64} ${ARM32})
   endif()
 endif()
 
 if (WIN32)
   set(ALL_ORC_SUPPORTED_ARCH ${X86_64})
 endif()

compiler-rt/lib/xray/CMakeLists.txt

set(powerpc64le_SOURCES
   xray_trampoline_powerpc64_asm.S
   )
 
 set(hexagon_SOURCES
   xray_hexagon.cpp
   xray_trampoline_hexagon.S
   )
 
+set(riscv32_SOURCES
+  xray_riscv.cpp
+  xray_trampoline_riscv32.S
+  )
+
+set(riscv64_SOURCES
+  xray_riscv.cpp
+  xray_trampoline_riscv64.S
+  )
+
 set(XRAY_SOURCE_ARCHS
   arm
   armhf
   aarch64
   hexagon
   mips
   mipsel
   mips64
set(XRAY_ALL_SOURCE_FILES
   ${armhf_SOURCES}
   ${hexagon_SOURCES}
   ${loongarch64_SOURCES}
   ${mips_SOURCES}
   ${mipsel_SOURCES}
   ${mips64_SOURCES}
   ${mips64el_SOURCES}
   ${powerpc64le_SOURCES}
+  ${riscv32_SOURCES}
+  ${riscv64_SOURCES}
   ${XRAY_IMPL_HEADERS}
   )
 list(REMOVE_DUPLICATES XRAY_ALL_SOURCE_FILES)
 # Make list that uses absolute paths
 set(XRAY_ALL_SOURCE_FILES_ABS_PATHS "")
 foreach (src_file ${XRAY_ALL_SOURCE_FILES})
   list(APPEND
     XRAY_ALL_SOURCE_FILES_ABS_PATHS

compiler-rt/lib/xray/xray_interface.cpp

 #elif SANITIZER_MIPS32
 static const int16_t cSledLength = 48;
 #elif SANITIZER_MIPS64
 static const int16_t cSledLength = 64;
 #elif defined(__powerpc64__)
 static const int16_t cSledLength = 8;
 #elif defined(__hexagon__)
 static const int16_t cSledLength = 20;
+#elif SANITIZER_RISCV64
+static const int16_t cSledLength = 76;
+#elif defined(__riscv) && (__riscv_xlen == 32)
+static const int16_t cSledLength = 60;
 #else
 #error "Unsupported CPU Architecture"
 #endif /* CPU architecture */
 
 // This is the function to call when we encounter the entry or exit sleds.
 atomic_uintptr_t XRayPatchedFunction{0};
 
 // This is the function to call from the arg1-enabled sleds/trampolines.

compiler-rt/lib/xray/xray_riscv.cpp

+//===-- xray_riscv.cpp ----------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file is a part of XRay, a dynamic runtime instrumentation system.
+//
+// Implementation of riscv-specific routines (32- and 64-bit).
+//
+//===----------------------------------------------------------------------===//
+#include "sanitizer_common/sanitizer_common.h"
+#include "xray_defs.h"
+#include "xray_interface_internal.h"
+#include <atomic>
+
+namespace __xray {
+
+// The machine codes for some instructions used in runtime patching.
+enum PatchOpcodes : uint32_t {
+  PO_ADDI = 0x00000013, // addi rd, rs1, imm
+  PO_ADD = 0x00000033,  // add rd, rs1, rs2
+  PO_SW = 0x00002023,   // sw rt, base(offset)
+  PO_SD = 0x00003023,   // sd rt, base(offset)
+  PO_LUI = 0x00000037,  // lui rd, imm
+  PO_ORI = 0x00006013,  // ori rd, rs1, imm
+  PO_OR = 0x00006033,   // or rd, rs1, rs2
+  PO_SLLI = 0x00001013, // slli rd, rs, shamt
+  PO_SRLI = 0x00005013, // srli rd, rs, shamt
+  PO_JALR = 0x00000067, // jalr rs
+  PO_LW = 0x00002003,   // lw rd, base(offset)
+  PO_LD = 0x00003003,   // ld rd, base(offset)
+  PO_J = 0x0000006f,    // jal #n_bytes
+  PO_NOP = 0x00000013,  // nop - pseduo-instruction, same as addi x0, x0, 0
+};
+
+enum RegNum : uint32_t {
+  RN_R0 = 0x0,
+  RN_RA = 0x1,
+  RN_SP = 0x2,
+  RN_T0 = 0x5,
+  RN_T1 = 0x6,
+  RN_T2 = 0x7,
+  RN_A0 = 0xa,
+};
+
+static inline uint32_t
+encodeRTypeInstruction(uint32_t Opcode, uint32_t Rs1, uint32_t Rs2,
+                       uint32_t Rd) {
+  return Rs2 << 20 | Rs1 << 15 | Rd << 7 | Opcode;
+}
+
+static inline uint32_t
+encodeITypeInstruction(uint32_t Opcode, uint32_t Rs1, uint32_t Rd,
+                       uint32_t Imm) {
+  return Imm << 20 | Rs1 << 15 | Rd << 7 | Opcode;
+}
+
+static inline uint32_t
+encodeSTypeInstruction(uint32_t Opcode, uint32_t Rs1, uint32_t Rs2,
+                       uint32_t Imm) {
+  uint32_t imm_msbs = (Imm & 0xfe0) << 25;
+  uint32_t imm_lsbs = (Imm & 0x01f) << 7;
+  return imm_msbs | Rs2 << 20 | Rs1 << 15 | imm_lsbs | Opcode;
+}
+
+static inline uint32_t
+encodeUTypeInstruction(uint32_t Opcode, uint32_t Rd,
+                       uint32_t Imm) {
+  return Imm << 12 | Rd << 7 | Opcode;
+}
+
+static inline uint32_t
+encodeJTypeInstruction(uint32_t Opcode, uint32_t Rd,
+                       uint32_t Imm) {
+  uint32_t imm_msb = (Imm & 0x80000) << 31;
+  uint32_t imm_lsbs = (Imm & 0x003ff) << 21;
+  uint32_t imm_11 = (Imm & 0x00400) << 20;
+  uint32_t imm_1912 = (Imm & 0x7f800) << 12;
+  return imm_msb | imm_lsbs | imm_11 | imm_1912 | Rd << 7 | Opcode;
+}
+
+#if SANITIZER_RISCV64
+static uint32_t hi20(uint64_t val) { return (val + 0x800) >> 12; }
+static uint32_t lo12(uint64_t val) { return val & 0xfff; }
+#elif defined(__riscv) && (__riscv_xlen == 32)
+static uint32_t hi20(uint32_t val) { return (val + 0x800) >> 12; }
+static uint32_t lo12(uint32_t val) { return val & 0xfff; }
+#endif
+
+static inline bool patchSled(const bool Enable, const uint32_t FuncId,
+                             const XRaySledEntry &Sled,
+                             void (*TracingHook)()) XRAY_NEVER_INSTRUMENT {
+  // When |Enable| == true,
+  // We replace the following compile-time stub (sled):
+  //
+  // xray_sled_n:
+  //  J .tmpN
+  //  14 or 18 NOPs (56 or 72 bytes)
+  //  .tmpN
+  //
+  // With one of the following runtime patches:
+  //
+  // xray_sled_n (32-bit):
+  //    addi sp, sp, -16                                ;create stack frame
+  //    sw ra, 12(sp)                                   ;save return address
+  //    sw t2, 8(sp)                                    ;save register t2
+  //    sw t1, 4(sp)                                    ;save register t1
+  //    sw a0, 0(sp)                                    ;save register a0
+  //    lui t1, %hi(__xray_FunctionEntry/Exit)
+  //    addi t1, t1, %lo(__xray_FunctionEntry/Exit)
+  //    lui a0, %hi(function_id)
+  //    addi a0, a0, %lo(function_id)                   ;pass function id
+  //    jalr t1                                         ;call Tracing hook
+  //    lw a0, 0(sp)                                    ;restore register a0
+  //    lw t1, 4(sp)                                    ;restore register t1
+  //    lw t2, 8(sp)                                    ;restore register t2
+  //    lw ra, 12(sp)                                   ;restore return address
+  //    addi sp, sp, 16                                 ;delete stack frame
+  //
+  // xray_sled_n (64-bit):
+  //    addi sp, sp, -32                                ;create stack frame
+  //    sd ra, 24(sp)                                   ;save return address
+  //    sd t2, 16(sp)                                   ;save register t2
+  //    sd t1, 8(sp)                                    ;save register t1
+  //    sd a0, 0(sp)                                    ;save register a0
+  //    lui t2, %highest(__xray_FunctionEntry/Exit)
+  //    addi t2, t2, %higher(__xray_FunctionEntry/Exit)
+  //    slli t2, t2, 32
+  //    lui t1, t1, %hi(__xray_FunctionEntry/Exit)
+  //    addi t1, t1, %lo(__xray_FunctionEntry/Exit)
+  //    add t1, t2, t1
+  //    lui a0, %hi(function_id)
+  //    addi a0, a0, %lo(function_id)                   ;pass function id
+  //    jalr t1                                         ;call Tracing hook
+  //    ld a0, 0(sp)                                    ;restore register a0
+  //    ld t1, 8(sp)                                    ;restore register t1
+  //    ld t2, 16(sp)                                   ;restore register t2
+  //    ld ra, 24(sp)                                   ;restore return address
+  //    addi sp, sp, 32                                 ;delete stack frame
+  //
+  // Replacement of the first 4-byte instruction should be the last and atomic
+  // operation, so that the user code which reaches the sled concurrently
+  // either jumps over the whole sled, or executes the whole sled when the
+  // latter is ready.
+  //
+  // When |Enable|==false, we set back the first instruction in the sled to be
+  //   J 60 bytes (rv32)
+  //   J 76 bytes (rv64)
+
+  uint32_t *Address = reinterpret_cast<uint32_t *>(Sled.address());
+  if (Enable) {
+    // If the ISA is RISCV 64, the Tracing Hook needs to be typecast to a 64 bit
+    // value
+#if SANITIZER_RISCV64
+    uint32_t LoTracingHookAddr = lo12(reinterpret_cast<uint64_t>(TracingHook));
+    uint32_t HiTracingHookAddr = hi20(reinterpret_cast<uint64_t>(TracingHook));
+    uint32_t HigherTracingHookAddr =
+        lo12((reinterpret_cast<uint64_t>(TracingHook) + 0x80000000) >> 32);
+    uint32_t HighestTracingHookAddr =
+        hi20((reinterpret_cast<uint64_t>(TracingHook) + 0x80000000) >> 32);
+    // We typecast the Tracing Hook to a 32 bit value for RISCV32
+#elif defined(__riscv) && (__riscv_xlen == 32)
+    uint32_t LoTracingHookAddr = lo12(reinterpret_cast<uint32_t>(TracingHook));
+    uint32_t HiTracingHookAddr = hi20((reinterpret_cast<uint32_t>(TracingHook));
+#endif
+    uint32_t LoFunctionID = lo12(FuncId);
+    uint32_t HiFunctionID = hi20(FuncId);
+    // The sled that is patched in for RISCV64 defined below. We need the entire
+    // sleds corresponding to both ISAs to be protected by defines because the
+    // first few instructions are all different, because we store doubles in
+    // case of RV64 and store words for RV32. Subsequently, we have LUI - and in
+    // case of RV64, we need extra instructions from this point on, so we see
+    // differences in addresses to which instructions are stored.
+#if SANITIZER_RISCV64
+    Address[1] = encodeSTypeInstruction(PatchOpcodes::PO_SD, RegNum::RN_SP,
+                                        RegNum::RN_RA, 0x18);
+    Address[2] = encodeSTypeInstruction(PatchOpcodes::PO_SD, RegNum::RN_SP,
+                                        RegNum::RN_T2, 0x10);
+    Address[3] = encodeSTypeInstruction(PatchOpcodes::PO_SD, RegNum::RN_SP,
+                                        RegNum::RN_T1, 0x8);
+    Address[4] = encodeSTypeInstruction(PatchOpcodes::PO_SD, RegNum::RN_SP,
+                                        RegNum::RN_A0, 0x0);
+    Address[5] = encodeUTypeInstruction(PatchOpcodes::PO_LUI, RegNum::RN_T2,
+                                        HighestTracingHookAddr);
+    Address[6] = encodeITypeInstruction(PatchOpcodes::PO_ADDI, RegNum::RN_T2,
+                                        RegNum::RN_T2, HigherTracingHookAddr);
+    Address[7] = encodeITypeInstruction(PatchOpcodes::PO_SLLI, RegNum::RN_T2,
+                                        RegNum::RN_T2, 0x20);
+    Address[8] = encodeUTypeInstruction(PatchOpcodes::PO_LUI, RegNum::RN_T1,
+                                        HiTracingHookAddr);
+    Address[9] = encodeITypeInstruction(PatchOpcodes::PO_ADDI, RegNum::RN_T1,
+                                        RegNum::RN_T1, LoTracingHookAddr);
+    Address[10] = encodeRTypeInstruction(PatchOpcodes::PO_ADD, RegNum::RN_T1,
+                                         RegNum::RN_T2, RegNum::RN_T1);
+    Address[11] = encodeUTypeInstruction(PatchOpcodes::PO_LUI, RegNum::RN_A0,
+                                         HiFunctionID);
+    Address[12] = encodeITypeInstruction(PatchOpcodes::PO_ADDI, RegNum::RN_A0,
+                                         RegNum::RN_A0, LoFunctionID);
+    Address[13] = encodeITypeInstruction(PatchOpcodes::PO_JALR, RegNum::RN_T1,
+                                         RegNum::RN_RA, 0x0);
+    Address[14] = encodeITypeInstruction(PatchOpcodes::PO_LD, RegNum::RN_SP,
+                                         RegNum::RN_A0, 0x0);
+    Address[15] = encodeITypeInstruction(PatchOpcodes::PO_LD, RegNum::RN_SP,
+                                         RegNum::RN_T1, 0x8);
+    Address[16] = encodeITypeInstruction(PatchOpcodes::PO_LD, RegNum::RN_SP,
+                                         RegNum::RN_T2, 0x10);
+    Address[17] = encodeITypeInstruction(PatchOpcodes::PO_LD, RegNum::RN_SP,
+                                         RegNum::RN_RA, 0x18);
+    Address[18] = encodeITypeInstruction(PatchOpcodes::PO_ADDI, RegNum::RN_SP,
+                                         RegNum::RN_SP, 0x20);
+    uint32_t CreateStackSpace = encodeITypeInstruction(
+        PatchOpcodes::PO_ADDI, RegNum::RN_SP, RegNum::RN_SP, 0xffe0);
+#elif defined(__riscv) && (__riscv_xlen == 32)
+    Address[1] = encodeSTypeInstruction(PatchOpcodes::PO_SW, RegNum::RN_SP,
+                                        RegNum::RN_RA, 0x0c);
+    Address[2] = encodeSTypeInstruction(PatchOpcodes::PO_SW, RegNum::RN_SP,
+                                        RegNum::RN_T2, 0x08);
+    Address[3] = encodeSTypeInstruction(PatchOpcodes::PO_SW, RegNum::RN_SP,
+                                        RegNum::RN_T1, 0x4);
+    Address[4] = encodeSTypeInstruction(PatchOpcodes::PO_SW, RegNum::RN_SP,
+                                        RegNum::RN_A0, 0x0);
+    Address[5] = encodeUTypeInstruction(PatchOpcodes::PO_LUI, RegNum::RN_T1,
+                                        HiTracingHookAddr);
+    Address[6] = encodeITypeInstruction(PatchOpcodes::PO_ADDI, RegNum::RN_T1,
+                                        RegNum::RN_T1, LoTracingHookAddr);
+    Address[7] = encodeUTypeInstruction(PatchOpcodes::PO_LUI, RegNum::RN_A0,
+                                        HiFunctionID);
+    Address[8] = encodeITypeInstruction(PatchOpcodes::PO_ADDI, RegNum::RN_A0,
+                                        RegNum::RN_A0, LoFunctionID);
+    Address[9] = encodeITypeInstruction(PatchOpcodes::PO_JALR, RegNum::RN_T1,
+                                        RegNum::RN_RA, 0x0);
+    Address[10] = encodeITypeInstruction(PatchOpcodes::PO_LW, RegNum::RN_SP,
+                                         RegNum::RN_A0, 0x0);
+    Address[11] = encodeITypeInstruction(PatchOpcodes::PO_LW, RegNum::RN_SP,
+                                         RegNum::RN_T1, 0x4);
+    Address[12] = encodeITypeInstruction(PatchOpcodes::PO_LW, RegNum::RN_SP,
+                                         RegNum::RN_T2, 0x08);
+    Address[13] = encodeITypeInstruction(PatchOpcodes::PO_LW, RegNum::RN_SP,
+                                         RegNum::RN_RA, 0x0c);
+    Address[14] = encodeITypeInstruction(PatchOpcodes::PO_ADDI, RegNum::RN_SP,
+                                         RegNum::RN_SP, 0x10);
+    uint32_t CreateStackSpace = encodeITypeInstruction(
+        PatchOpcodes::PO_ADDI, RegNum::RN_SP, RegNum::RN_SP, 0xfff0);
+#endif
+    std::atomic_store_explicit(
+        reinterpret_cast<std::atomic<uint32_t> *>(Address), CreateStackSpace,
+        std::memory_order_release);
+  } else {
+    uint32_t CreateBranch = encodeJTypeInstruction(
+    // Jump distance is different in both ISAs due to difference in size of
+    // sleds
+#if SANITIZER_RISCV64
+        PatchOpcodes::PO_J, RegNum::RN_R0,
+        0x026); // jump encodes an offset in multiples of 2 bytes. 38*2 = 76
+#elif defined(__riscv) && (__riscv_xlen == 32)
+        PatchOpcodes::PO_J, RegNum::RN_R0,
+        0x01e); // jump encodes an offset in multiples of 2 bytes. 30*2 = 60
+#endif
+    std::atomic_store_explicit(
+        reinterpret_cast<std::atomic<uint32_t> *>(Address), CreateBranch,
+        std::memory_order_release);
+  }
+  return true;
+}
+
+bool patchFunctionEntry(const bool Enable, const uint32_t FuncId,
+                        const XRaySledEntry &Sled,
+                        void (*Trampoline)()) XRAY_NEVER_INSTRUMENT {
+  return patchSled(Enable, FuncId, Sled, Trampoline);
+}
+
+bool patchFunctionExit(const bool Enable, const uint32_t FuncId,
+                       const XRaySledEntry &Sled) XRAY_NEVER_INSTRUMENT {
+  return patchSled(Enable, FuncId, Sled, __xray_FunctionExit);
+}
+
+bool patchFunctionTailExit(const bool Enable, const uint32_t FuncId,
+                           const XRaySledEntry &Sled) XRAY_NEVER_INSTRUMENT {
+  // TODO: Implement tail exits for riscv
+  return patchSled(Enable, FuncId, Sled, __xray_FunctionExit);
+  // return patchSled(Enable, FuncId, Sled, __xray_FunctionTailExit);
+}
+
+bool patchCustomEvent(const bool Enable, const uint32_t FuncId,
+                      const XRaySledEntry &Sled) XRAY_NEVER_INSTRUMENT {
+  // TODO: Implement for riscv?
+  return false;
+}
+
+bool patchTypedEvent(const bool Enable, const uint32_t FuncId,
+                     const XRaySledEntry &Sled) XRAY_NEVER_INSTRUMENT {
+  // TODO: Implement for riscv?
+  return false;
+}
+} // namespace __xray
+
+extern "C" void __xray_ArgLoggerEntry() XRAY_NEVER_INSTRUMENT {
+  // TODO: this will have to be implemented in the trampoline assembly file
+}

compiler-rt/lib/xray/xray_trampoline_riscv32.S

+//===-- xray_trampoline_riscv32.s ----------------------------------*- ASM -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file is a part of XRay, a dynamic runtime instrumentation system.
+//
+// This implements the riscv32-specific assembler for the trampolines.
+//
+//===----------------------------------------------------------------------===//
+
+#include "../builtins/assembly.h"
+#include "../sanitizer_common/sanitizer_asm.h"
+
+  .text
+  .p2align 2
+  .global ASM_SYMBOL(__xray_FunctionEntry)
+  ASM_TYPE_FUNCTION(__xray_FunctionEntry)
+ASM_SYMBOL(__xray_FunctionEntry):
+  CFI_STARTPROC
+  // Push argument registers to stack
+  addi  sp, sp, -100
+  .cfi_def_cfa_offset 100
+  sw  ra, 96(sp)
+  .cfi_offset ra, -8
+  sw  a7, 92(sp)
+  sw  a6, 88(sp)
+  sw  a5, 84(sp)
+  sw  a4, 80(sp)
+  sw  a3, 76(sp)
+  sw  a2, 72(sp)
+  sw  a1, 68(sp)
+  sw  a0, 64(sp)
+  // The current implementation only supports double precision floats
+  // In case of RISCV32F, then these (faX) would still be 32 bit
+  // wide registers, so we should be incrementing by 4
+  fsd fa7, 56(sp)
+  fsd fa6, 48(sp)
+  fsd fa5, 40(sp)
+  fsd fa4, 32(sp)
+  fsd fa3, 24(sp)
+  fsd fa2, 16(sp)
+  fsd fa1, 8(sp)
+  fsd fa0, 0(sp)
+
+  // Load the handler function pointer into a2
+  li  a2, ASM_SYMBOL(_ZN6__xray19XRayPatchedFunctionE)
+  lw  a2, 0(a2)
+
+  // Handler address will be null if it is not set
+  beq a2, x0, 1f
+
+  // If we reach here, we are tracing an event
+  // a0 already contains function id
+  // a1 = 0 means we are tracing an entry event
+  mv  a1, x0
+  jalr  a2
+
+1:
+  // Restore argument registers
+  fld fa0, 0(sp)
+  fld fa1, 8(sp)
+  fld fa2, 16(sp)
+  fld fa3, 24(sp)
+  fld fa4, 32(sp)
+  fld fa5, 40(sp)
+  fld fa6, 48(sp)
+  fld fa7, 56(sp)
+  lw  a0, 64(sp)
+  lw  a1, 68(sp)
+  lw  a2, 72(sp)
+  lw  a3, 76(sp)
+  lw  a4, 80(sp)
+  lw  a5, 84(sp)
+  lw  a6, 88(sp)
+  lw  a7, 92(sp)
+  lw  ra, 96(sp)
+  addi  sp, sp, 100
+  jr  ra
+  ASM_SIZE(__xray_FunctionEntry)
+  CFI_ENDPROC
+
+  .text
+  .p2align 2
+  .global ASM_SYMBOL(__xray_FunctionExit)
+  ASM_TYPE_FUNCTION(__xray_FunctionExit)
+ASM_SYMBOL(__xray_FunctionExit):
+  CFI_STARTPROC
+  // Push return registers to stack
+  addi  sp, sp, -28
+  .cfi_def_cfa_offset 28
+  sw  ra, 24(sp)
+  .cfi_offset ra, -8
+  sw  a1, 20(sp)
+  sw  a0, 16(sp)
+  // The current implementation only supports double precision floats
+  // In case of RISCV32F, then these (faX) would still be 32 bit
+  // wide registers, so we should be incrementing by 4
+  fsd fa1, 8(sp)
+  fsd fa0, 0(sp)
+
+  // Load the handler function pointer into a2
+  li  a2, ASM_SYMBOL(_ZN6__xray19XRayPatchedFunctionE)
+  lw  a2, 0(a2)
+
+  // Handler address will be null if it is not set
+  beq a2, x0, 1f
+
+  // If we reach here, we are tracing an event
+  // a0 already contains function id
+  // a1 = 1 means we are tracing an exit event
+  addi  a1, x0, 1
+  jalr  a2
+
+1:
+  // Restore return registers
+  fld fa0, 0(sp)
+  fld fa1, 8(sp)
+  lw  a0, 16(sp)
+  lw  a1, 20(sp)
+  lw  ra, 24(sp)
+  addi  sp, sp, 28
+  jr  ra
+  ASM_SIZE(__xray_FunctionExit)
+  CFI_ENDPROC

compiler-rt/lib/xray/xray_trampoline_riscv64.S

+//===-- xray_trampoline_riscv64.s ----------------------------------*- ASM -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file is a part of XRay, a dynamic runtime instrumentation system.
+//
+// This implements the riscv64-specific assembler for the trampolines.
+//
+//===----------------------------------------------------------------------===//
+
+#include "../builtins/assembly.h"
+#include "../sanitizer_common/sanitizer_asm.h"
+
+  .text
+  .p2align 2
+  .global ASM_SYMBOL(__xray_FunctionEntry)
+  ASM_TYPE_FUNCTION(__xray_FunctionEntry)
+ASM_SYMBOL(__xray_FunctionEntry):
+  CFI_STARTPROC
+  // Push argument registers to stack
+  addi  sp, sp, -136
+  .cfi_def_cfa_offset 136
+  sd  ra, 128(sp)
+  .cfi_offset ra, -8
+  sd  a7, 120(sp)
+  sd  a6, 112(sp)
+  sd  a5, 104(sp)
+  sd  a4, 96(sp)
+  sd  a3, 88(sp)
+  sd  a2, 80(sp)
+  sd  a1, 72(sp)
+  sd  a0, 64(sp)
+  fsd fa7, 56(sp)
+  fsd fa6, 48(sp)
+  fsd fa5, 40(sp)
+  fsd fa4, 32(sp)
+  fsd fa3, 24(sp)
+  fsd fa2, 16(sp)
+  fsd fa1, 8(sp)
+  fsd fa0, 0(sp)
+
+  // Load the handler function pointer into a2
+  la  a2, ASM_SYMBOL(_ZN6__xray19XRayPatchedFunctionE)
+  ld  a2, 0(a2)
+
+  // Handler address will be null if it is not set
+  beq a2, x0, 1f
+
+  // If we reach here, we are tracing an event
+  // a0 already contains function id
+  // a1 = 0 means we are tracing an entry event
+  mv  a1, x0
+  jalr  a2
+
+1:
+  // Restore argument registers
+  fld fa0, 0(sp)
+  fld fa1, 8(sp)
+  fld fa2, 16(sp)
+  fld fa3, 24(sp)
+  fld fa4, 32(sp)
+  fld fa5, 40(sp)
+  fld fa6, 48(sp)
+  fld fa7, 56(sp)
+  ld  a0, 64(sp)
+  ld  a1, 72(sp)
+  ld  a2, 80(sp)
+  ld  a3, 88(sp)
+  ld  a4, 96(sp)
+  ld  a5, 104(sp)
+  ld  a6, 112(sp)
+  ld  a7, 120(sp)
+  ld  ra, 128(sp)
+  addi  sp, sp, 136
+  jr  ra
+  ASM_SIZE(__xray_FunctionEntry)
+  CFI_ENDPROC
+
+  .text
+  .p2align 2
+  .global ASM_SYMBOL(__xray_FunctionExit)
+  ASM_TYPE_FUNCTION(__xray_FunctionExit)
+ASM_SYMBOL(__xray_FunctionExit):
+  CFI_STARTPROC
+  // Push return registers to stack
+  addi  sp, sp, -40
+  .cfi_def_cfa_offset 40
+  sd  ra, 32(sp)
+  .cfi_offset ra, -8
+  sd  a1, 24(sp)
+  sd  a0, 16(sp)
+  fsd fa1, 8(sp)
+  fsd fa0, 0(sp)
+
+  // Load the handler function pointer into a2
+  la  a2, ASM_SYMBOL(_ZN6__xray19XRayPatchedFunctionE)
+  ld  a2, 0(a2)
+
+  // Handler address will be null if it is not set
+  beq a2, x0, 1f
+
+  // If we reach here, we are tracing an event
+  // a0 already contains function id
+  // a1 = 1 means we are tracing an exit event
+  addi  a1, x0, 1
+  jalr  a2
+
+1:
+  // Restore return registers
+  fld fa0, 0(sp)
+  fld fa1, 8(sp)
+  ld  a0, 16(sp)
+  ld  a1, 24(sp)
+  ld  ra, 32(sp)
+  addi  sp, sp, 40
+  jr  ra
+  ASM_SIZE(__xray_FunctionExit)
+  CFI_ENDPROC

compiler-rt/lib/xray/xray_tsc.h

 
 #else // SANITIZER_FUCHSIA
 
 #if defined(__x86_64__)
 #include "xray_x86_64.inc"
 #elif defined(__powerpc64__)
 #include "xray_powerpc64.inc"
 #elif defined(__arm__) || defined(__aarch64__) || defined(__mips__) ||         \
-    defined(__hexagon__) || defined(__loongarch_lp64)
+    defined(__hexagon__) || defined(__loongarch_lp64) || defined(__riscv)
 // Emulated TSC.
 // There is no instruction like RDTSCP in user mode on ARM. ARM's CP15 does
 //   not have a constant frequency like TSC on x86(_64), it may go faster
 //   or slower depending on CPU turbo or power saving mode. Furthermore,
 //   to read from CP15 on ARM a kernel modification or a driver is needed.
 //   We can not require this from users of compiler-rt.
 // So on ARM we use clock_gettime() which gives the result in nanoseconds.
 //   To get the measurements per second, we scale this by the number of

llvm/lib/CodeGen/XRayInstrumentation.cpp

if (!F.hasFnAttribute("xray-skip-exit")) {
     case Triple::ArchType::arm:
     case Triple::ArchType::thumb:
     case Triple::ArchType::aarch64:
     case Triple::ArchType::hexagon:
     case Triple::ArchType::loongarch64:
     case Triple::ArchType::mips:
     case Triple::ArchType::mipsel:
     case Triple::ArchType::mips64:
-    case Triple::ArchType::mips64el: {
+    case Triple::ArchType::mips64el:
+    case Triple::ArchType::riscv32:
+    case Triple::ArchType::riscv64: {
       // For the architectures which don't have a single return instruction
       InstrumentationOptions op;
       op.HandleTailcall = false;
       op.HandleAllReturns = true;
       prependRetWithPatchableExit(MF, TII, op);
       break;
     }
     case Triple::ArchType::ppc64le: {

llvm/lib/Target/RISCV/RISCVAsmPrinter.cpp

 //
 //===----------------------------------------------------------------------===//
 
 #include "MCTargetDesc/RISCVBaseInfo.h"
 #include "MCTargetDesc/RISCVInstPrinter.h"
 #include "MCTargetDesc/RISCVMCExpr.h"
 #include "MCTargetDesc/RISCVTargetStreamer.h"
 #include "RISCV.h"
+#include "RISCVSubtarget.h"
 #include "RISCVMachineFunctionInfo.h"
 #include "RISCVTargetMachine.h"
 #include "TargetInfo/RISCVTargetInfo.h"
 #include "llvm/ADT/APInt.h"
 #include "llvm/ADT/Statistic.h"
 #include "llvm/BinaryFormat/ELF.h"
 #include "llvm/CodeGen/AsmPrinter.h"
 #include "llvm/CodeGen/MachineConstantPool.h"
public:
   std::map<HwasanMemaccessTuple, MCSymbol *> HwasanMemaccessSymbols;
   void LowerHWASAN_CHECK_MEMACCESS(const MachineInstr &MI);
   void LowerKCFI_CHECK(const MachineInstr &MI);
   void EmitHwasanMemaccessSymbols(Module &M);
 
   // Wrapper needed for tblgenned pseudo lowering.
   bool lowerOperand(const MachineOperand &MO, MCOperand &MCOp) const;
 
+  // XRay Support
+  void LowerPATCHABLE_FUNCTION_ENTER(const MachineInstr *MI);
+  void LowerPATCHABLE_FUNCTION_EXIT(const MachineInstr *MI);
+  void LowerPATCHABLE_TAIL_CALL(const MachineInstr *MI);
+
   void emitStartOfAsmFile(Module &M) override;
   void emitEndOfAsmFile(Module &M) override;
 
   void emitFunctionEntryLabel() override;
   void emitDirectiveOptionArch();
   bool isSameAttribute();
 
 private:
   void emitAttributes();
+  // XRay Support
+  void emitSled(const MachineInstr *MI, SledKind Kind);
 
   void emitNTLHint(const MachineInstr *MI);
 
   bool lowerToMCInst(const MachineInstr *MI, MCInst &OutMI);
 };
 }
 
 void RISCVAsmPrinter::EmitToStreamer(MCStreamer &S, const MCInst &Inst) {
   MCInst CInst;
   bool Res = RISCVRVC::compress(CInst, Inst, *STI);
   if (Res)
void RISCVAsmPrinter::emitInstruction(const MachineInstr *MI) {
   case RISCV::KCFI_CHECK:
     LowerKCFI_CHECK(*MI);
     return;
   case RISCV::PseudoRVVInitUndefM1:
   case RISCV::PseudoRVVInitUndefM2:
   case RISCV::PseudoRVVInitUndefM4:
   case RISCV::PseudoRVVInitUndefM8:
     return;
+  case TargetOpcode::PATCHABLE_FUNCTION_ENTER: {
+    // This switch case section is only for handling XRay sleds.
+    //
+    // patchable-function-entry is handled in lowerToMCInst
+    // Therefore, we break out of the switch statement if we encounter it here.
+    const Function &F = MI->getParent()->getParent()->getFunction();
+    if (F.hasFnAttribute("patchable-function-entry"))
+      break;
+
+    LowerPATCHABLE_FUNCTION_ENTER(MI);
+    return;
+  }
+  case TargetOpcode::PATCHABLE_FUNCTION_EXIT: {
+    LowerPATCHABLE_FUNCTION_EXIT(MI);
+    return;
+  }
+  case TargetOpcode::PATCHABLE_TAIL_CALL: {
+    LowerPATCHABLE_TAIL_CALL(MI);
+    return;
+  }
   }
 
   MCInst OutInst;
   if (!lowerToMCInst(MI, OutInst))
     EmitToStreamer(*OutStreamer, OutInst);
 }
 
 bool RISCVAsmPrinter::PrintAsmOperand(const MachineInstr *MI, unsigned OpNo,
bool RISCVAsmPrinter::PrintAsmOperand(const MachineInstr *MI, unsigned OpNo,
   default:
     break;
   }
 
   return true;
 }
 
 bool RISCVAsmPrinter::PrintAsmMemoryOperand(const MachineInstr *MI,
                                             unsigned OpNo,
                                             const char *ExtraCode,
                                             raw_ostream &OS) {
   if (ExtraCode)
     return AsmPrinter::PrintAsmMemoryOperand(MI, OpNo, ExtraCode, OS);
 
   const MachineOperand &AddrReg = MI->getOperand(OpNo);
   assert(MI->getNumOperands() > OpNo + 1 && "Expected additional operand");
   const MachineOperand &DispImm = MI->getOperand(OpNo + 1);
   // All memory operands should have a register and an immediate operand (see
   // RISCVDAGToDAGISel::SelectInlineAsmMemoryOperand).
   if (!AddrReg.isReg())
     return true;
   if (!DispImm.isImm())
     return true;
 
   OS << DispImm.getImm() << "("
      << RISCVInstPrinter::getRegisterName(AddrReg.getReg()) << ")";
   return false;
 }
 
 void RISCVAsmPrinter::emitDirectiveOptionArch() {
   RISCVTargetStreamer &RTS =
       static_cast<RISCVTargetStreamer &>(*OutStreamer->getTargetStreamer());
   SmallVector<RISCVOptionArchArg> NeedEmitStdOptionArgs;
   const MCSubtargetInfo &MCSTI = *TM.getMCSubtargetInfo();
   for (const auto &Feature : RISCVFeatureKV) {
     if (STI->hasFeature(Feature.Value) == MCSTI.hasFeature(Feature.Value))
       continue;
 
     if (!llvm::RISCVISAInfo::isSupportedExtensionFeature(Feature.Key))
       continue;
 
     auto Delta = STI->hasFeature(Feature.Value) ? RISCVOptionArchArgType::Plus
                                                 : RISCVOptionArchArgType::Minus;
     NeedEmitStdOptionArgs.emplace_back(Delta, Feature.Key);
   }
   if (!NeedEmitStdOptionArgs.empty())
bool RISCVAsmPrinter::runOnMachineFunction(MachineFunction &MF) {
   if (!isSameAttribute()) {
     RTS.emitDirectiveOptionPush();
     emitDirectiveOptionArch();
   }
 
   SetupMachineFunction(MF);
   emitFunctionBody();
 
+  // Emit the XRay table
+  emitXRayTable();
+
   if (!isSameAttribute())
     RTS.emitDirectiveOptionPop();
   return false;
 }
 
+void RISCVAsmPrinter::LowerPATCHABLE_FUNCTION_ENTER(const MachineInstr *MI) {
+  emitSled(MI, SledKind::FUNCTION_ENTER);
+}
+
+void RISCVAsmPrinter::LowerPATCHABLE_FUNCTION_EXIT(const MachineInstr *MI) {
+  emitSled(MI, SledKind::FUNCTION_EXIT);
+}
+
+void RISCVAsmPrinter::LowerPATCHABLE_TAIL_CALL(const MachineInstr *MI) {
+  emitSled(MI, SledKind::TAIL_CALL);
+}
+
+void RISCVAsmPrinter::emitSled(const MachineInstr *MI, SledKind Kind) {
+  // The following variable holds the count of the number of NOPs to be patched
+  // in for XRay instrumentation during compilation. RISCV64 needs 36 C.NOPs,
+  // RISCV32 needs 28 C.NOPs.
+  const uint8_t NoopsInSledCount =
+      MI->getParent()->getParent()->getSubtarget<RISCVSubtarget>().is64Bit()
+          ? 36
+          : 28;
+
+  // We want to emit the jump instruction and the nops constituting the sled.
+  // The format is as follows:
+  // .Lxray_sled_N
+  //   ALIGN
+  //   J .tmpN (60 or 76 byte jump, depending on ISA)
+  //   28 or 36 C.NOP instructions
+  // .tmpN
+
+  OutStreamer->emitCodeAlignment(Align(4), &getSubtargetInfo());
+  auto CurSled = OutContext.createTempSymbol("xray_sled_", true);
+  OutStreamer->emitLabel(CurSled);
+  auto Target = OutContext.createTempSymbol();
+
+  const MCExpr *TargetExpr = MCSymbolRefExpr::create(
+      Target, MCSymbolRefExpr::VariantKind::VK_None, OutContext);
+
+  // Emit "J bytes" instruction, which jumps over the nop sled to the actual
+  // start of function.
+  EmitToStreamer(
+      *OutStreamer,
+      MCInstBuilder(RISCV::JAL).addReg(RISCV::X0).addExpr(TargetExpr));
+
+  // Emit NOP instructions
+  for (int8_t I = 0; I < NoopsInSledCount; ++I)
+    EmitToStreamer(*OutStreamer, MCInstBuilder(RISCV::ADDI)
+                                     .addReg(RISCV::X0)
+                                     .addReg(RISCV::X0)
+                                     .addImm(0));
+
+  OutStreamer->emitLabel(Target);
+  recordSled(CurSled, *MI, Kind, 2);
+}
+
 void RISCVAsmPrinter::emitStartOfAsmFile(Module &M) {
   RISCVTargetStreamer &RTS =
       static_cast<RISCVTargetStreamer &>(*OutStreamer->getTargetStreamer());
   if (const MDString *ModuleTargetABI =
           dyn_cast_or_null<MDString>(M.getModuleFlag("target-abi")))
     RTS.setTargetABI(RISCVABI::getTargetABI(ModuleTargetABI->getString()));
   if (TM.getTargetTriple().isOSBinFormatELF())
     emitAttributes();

llvm/lib/Target/RISCV/RISCVSubtarget.h

bool isSoftFPABI() const {
     return TargetABI == RISCVABI::ABI_LP64 ||
            TargetABI == RISCVABI::ABI_ILP32 ||
            TargetABI == RISCVABI::ABI_ILP32E;
   }
   bool isRegisterReservedByUser(Register i) const {
     assert(i < RISCV::NUM_TARGET_REGS && "Register out of range");
     return UserReservedRegister[i];
   }
+  // Add XRay support - assume D and C extensions available
+  bool isXRaySupported() const override {
+    return hasStdExtD() && hasStdExtC();
+  }
 
   bool hasMacroFusion() const { return hasLUIADDIFusion(); }
 
   // Vector codegen related methods.
   bool hasVInstructions() const { return HasStdExtZve32x; }
   bool hasVInstructionsI64() const { return HasStdExtZve64x; }
   bool hasVInstructionsF16() const { return HasStdExtZvfh; }
   bool hasVInstructionsBF16() const { return HasStdExtZvfbfmin; }

llvm/test/CodeGen/RISCV/xray-attribute-instrumentation.ll

+; RUN: llc -mtriple=riscv32-unknown-linux-gnu -mattr=+d -verify-machineinstrs < %s | FileCheck --check-prefix=CHECK %s
+; RUN: llc -mtriple=riscv64-unknown-linux-gnu -mattr=+d -verify-machineinstrs < %s | FileCheck --check-prefix=CHECK --check-prefix=CHECK-RISCV64 %s
+
+define i32 @foo() nounwind "function-instrument"="xray-always" {
+; CHECK:                        .p2align 2
+; CHECK-LABEL:                  .Lxray_sled_0:
+; CHECK-NEXT:                   j .Ltmp0
+; CHECK-COUNT-14:               nop
+; CHECK-RISCV64-COUNT-4:        nop
+; CHECK-LABEL:                  .Ltmp0:
+  ret i32 0
+; CHECK:                        .p2align 2
+; CHECK-LABEL:                  .Lxray_sled_1:
+; CHECK-NEXT:                   j .Ltmp1
+; CHECK-COUNT-14:               nop
+; CHECK-RISCV64-COUNT-4:        nop
+; CHECK-LABEL:                  .Ltmp1:
+; CHECK-NEXT:                   ret
+}
+; CHECK:                        .section xray_instr_map,"ao",@progbits,foo
+; CHECK-LABEL:                  .Lxray_sleds_start0:
+; CHECK:                        .Lxray_sled_0-[[TMP:.Ltmp[0-9]+]]
+; CHECK:                        .Lxray_sled_1-[[TMP:.Ltmp[0-9]+]]
+; CHECK-LABEL:                  .Lxray_sleds_end0: