[compiler-rt] Allow 3 simultaneous interceptors on Linux

[melver]

Summary

Rework Linux (and *BSD) interceptors to allow for up to 3 (2 for *BSD)
simultaneous interceptors. See code comments for details.

The main motivation is to support new sampling sanitizers (in the spirit
of GWP-ASan), that have to intercept few functions. Unfortunately, the
reality is that there are user interceptors that exist in the wild.

To support foreign user interceptors, foreign dynamic analysis
interceptors, and compiler-rt interceptors all at the same time,
including any combination of them, this change enables up to 3
interceptors on Linux (2 on *BSD).

v2:

• Revert to to the simpler "weak wrapper -(alias)-> __interceptor" scheme on architectures that cannot implement a trampoline efficiently due to complexities of resolving a preemptible symbol (PowerPC64 ELFv2 global entry, and i386 PIC).
• Avoid duplicate intercepted functions in gen_dynamic_list.py, due to matching interceptor_X and _interceptor_X.
• Fix s390 __tls_get_offset.

[dvyukov]

compiler-rt/lib/interception/interception.h

173

Can't we still use an alias to detect if our interceptor is installed or not?
Namely: give the actual function with ASM_JUMP body some new name (actual_func); make func a weak alias to actual_func; use comparison with __actual_func to detect override.

[melver]

compiler-rt/lib/interception/interception.h

173

Ok, I've changed it. PTAL.

[melver]

• Remove use of section.
• Implement for .asm files.

[melver]

• Commit message
• Fix all tests

[dvyukov]

Overall looks good.
I think not having a section will increase chances of sticking since this is very close to what we are doing now.

compiler-rt/lib/interception/tests/interception_linux_foreign_test.cpp

36

I think this should call double underscore.

41

It would be useful to test "triple" interception as well and also test the order of calls.
One way to test order of all calls is to intercept printf/write and print something to stdout in each interceptor, then an output test can ensure order of all calls.

compiler-rt/lib/sanitizer_common/sanitizer_internal_defs.h

269 ↗

(On Diff #524474)

I think this should be volatile, the block does not return anything, so compiler is free to remove it.

272 ↗

(On Diff #524474)

If we define JUMP_INSTR just b/jmp/tail, then we can (1) dedup this macro, (2) reuse it in ASM_INTERCEPTOR_TRAMPOLINE, so that callers don't need to pass it.

compiler-rt/lib/sanitizer_common/sanitizer_stacktrace_printer.cpp

37

Do we now pass custom prefix anywhere? Or is it only for tests?

[vitalybuka]

Could this be split into smaller parts? I bet we are going to land/revert some parts. Better to do with smaller ones.

compiler-rt/lib/interception/interception.h

161

it would be better if you introduce new macros as define to existing one and replace users as NFC patch

compiler-rt/lib/sanitizer_common/sanitizer_common_interceptors_vfork_aarch64.inc.S

43 ↗

(On Diff #524474)

you can introduce these macros as NFC to existing implementation

[melver]

• Address comments and various cleanups.

[melver]

compiler-rt/lib/interception/tests/interception_linux_foreign_test.cpp

41

Let's keep using unit tests. I verify order by just doing some non-commutative math.

compiler-rt/lib/sanitizer_common/sanitizer_internal_defs.h

272 ↗

(On Diff #524474)

It's called "ASM_TAIL_CALL" now, because apparently riscv is special.

compiler-rt/lib/sanitizer_common/sanitizer_stacktrace_printer.cpp

37

Not sure where else it uses custom prefix - but I didn't want to change API of this function here.

[dvyukov]

compiler-rt/lib/interception/tests/interception_linux_foreign_test.cpp

47

I think it will more readable/maintainable if we always multiply old value by 10 and add a value unique to interceptor (1/2). Then each digit will be journal of calls.

84

Reset these to 0 between calls, then they won't accumulate.

compiler-rt/test/msan/Linux/b64.cpp

63 ↗

(On Diff #525117)

If we now always strip __interceptor_ prefix, why do they appear in reports?

[melver]

compiler-rt/test/msan/Linux/b64.cpp

63 ↗

(On Diff #525117)

msan seems to have its own function name printing.
Let's do this: https://reviews.llvm.org/D151343

[melver]

• Improve test.
• Move msan changes to D151343

[MaskRay]

compiler-rt/lib/interception/interception.h

81

the interceptor implementation is in ___interceptor_func, which is aliased by a weak function __interceptor_func

___interceptor_func occurs twice?

[MaskRay]

x86-64's -m32 mode libclang_rt.ubsan_standalone.so doesn't link

ld.lld: error: relocation R_386_PC32 cannot be used against symbol '__interceptor_signal'; recompile with -fPIC
>>> defined in projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan_standalone.i386.dir/ubsan_signals_standalone.cpp.o
>>> referenced by sanitizer_signal_interceptors.inc:55 (/home/maskray/llvm/compiler-rt/lib/ubsan/../sanitizer_common/sanitizer_signal_intercep
tors.inc:55)
>>>               projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan_standalone.i386.dir/ubsan_signals_standalone.cpp.o:(__interceptor_trampoline_signal)

__interceptor_signal is a preemptible symbol, which cannot be referenced by a direct access relocation. jmp __interceptor_signal creates an R_386_PC32 relocation (direct access relocation). jmp __interceptor_signal@plt is needed to generate R_386_PLT32.
(See https://maskray.me/blog/2021-01-09-copy-relocations-canonical-plt-entries-and-protected "Non-default visibility ifunc and R_386_PC32" for detail.)

# define ASM_INTERCEPTOR_TRAMPOLINE(name) needs a special case for x86-32. Alternatively, make # define ASM_TAIL_CALL jmp a function-like macro to add the @plt modifier (suffix).

[MaskRay]

compiler-rt/lib/interception/interception.h

191

Unfortunately naked is not a generic attribute. Many targets don't support naked. See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=77882 for GCC AArch64.

[MaskRay]

Rework Linux (and *BSD) interceptors to allow for up to 3 (2) simultaneous interceptors. See code comments for details.

What "3 (2) simultaneous interceptors" mean seem confusing (what are counted as interceptors?)... It'd be good to give an example in the summary.

[dvyukov]

compiler-rt/lib/interception/interception.h

81

It's 2 vs 3 underscores :)

191

Is it possible to emit whole function inside of asm block? It can emit symbols, so should be possible, right?
We could create one fake function with inline asm, which will create another complete function.

[melver]

In D151085#4370760, @MaskRay wrote:

x86-64's -m32 mode libclang_rt.ubsan_standalone.so doesn't link

ld.lld: error: relocation R_386_PC32 cannot be used against symbol '__interceptor_signal'; recompile with -fPIC
>>> defined in projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan_standalone.i386.dir/ubsan_signals_standalone.cpp.o
>>> referenced by sanitizer_signal_interceptors.inc:55 (/home/maskray/llvm/compiler-rt/lib/ubsan/../sanitizer_common/sanitizer_signal_intercep
tors.inc:55)
>>>               projects/compiler-rt/lib/ubsan/CMakeFiles/RTUbsan_standalone.i386.dir/ubsan_signals_standalone.cpp.o:(__interceptor_trampoline_signal)

__interceptor_signal is a preemptible symbol, which cannot be referenced by a direct access relocation. jmp __interceptor_signal creates an R_386_PC32 relocation (direct access relocation). jmp __interceptor_signal@plt is needed to generate R_386_PLT32.
(See https://maskray.me/blog/2021-01-09-copy-relocations-canonical-plt-entries-and-protected "Non-default visibility ifunc and R_386_PC32" for detail.)

# define ASM_INTERCEPTOR_TRAMPOLINE(name) needs a special case for x86-32. Alternatively, make # define ASM_TAIL_CALL jmp a function-like macro to add the @plt modifier (suffix).

Ok, I've changed it. PTAL.

compiler-rt/lib/interception/interception.h

191

Doing all in asm now.

[melver]

• Address comments.

[melver]

In D151085#4370806, @MaskRay wrote:

Rework Linux (and *BSD) interceptors to allow for up to 3 (2) simultaneous interceptors. See code comments for details.

What "3 (2) simultaneous interceptors" mean seem confusing (what are counted as interceptors?)... It'd be good to give an example in the summary.

The wording there was confusing (2 for *BSD).

The commit message says "support foreign user interceptors, foreign dynamic analysis interceptors, and compiler-rt interceptors all at the same time, including any combination of them," - so in this case, there can be up to 3 interceptors that want to know when some function was called. We also have to support the case where one of them does not exist, and provide the protocol to follow to make it all work without special builds for each combination.

[melver]

• Document implementation better - now that it's all in asm it's hard to immediately see what it's doing.

[MaskRay]

@plt will break on most non-x86 targets. This should be fixed first.

compiler-rt/lib/sanitizer_common/sanitizer_asm.h

78

Many targets don't use @plt or at least, don't recommend @PLT.

@PLT probably should be x86 only.

For example, aarch64 doesn't support @plt.

compiler-rt/lib/sanitizer_common/scripts/gen_dynamic_list.py

128

__? or _?_

[MaskRay]

compiler-rt/lib/sanitizer_common/sanitizer_asm.h

55

#error doesn't need double quotes. Un-capitalized messages are more common.

[melver]

• Use @plt only with x86 ELF.
• No " with #error
• Improve regex in dynamic sym script.

[melver]

compiler-rt/lib/sanitizer_common/sanitizer_asm.h

55

Changed.

But somehow gcc docs say otherwise: https://gcc.gnu.org/onlinedocs/cpp/Diagnostics.html

The line must consist of complete tokens. It is wisest to make the argument of these directives be a single string constant; this avoids problems with apostrophes and the like.

Maybe that is no longer an issue?

78

Right - getting this right everywhere is a bit confusing to say the least (because I see mentions of "@plt" in tests on aarch64, etc.).

So now added a macro to add this conditionally only if "ELF && x86".

Relying on your expertise here. Thanks!

PTAL.

[MaskRay]

The current version will break most check-asan tests on ppc64le due to complexity of global entry/local entry in to TOC ABI.

(gdb) bt
#0  0x00000001001be578 in __sanitizer::SuppressionContext::Match(char const*, char const*, __sanitizer::Suppression**) ()
#1  0x00000001001a2710 in __asan::IsInterceptorSuppressed(char const*) ()
#2  0x000000010018f774 in __interceptor_memcpy ()
#3  0x00007ffff7b7b1f4 in __libc_sigaction (sig=<optimized out>, act=0x7ffffffff300, oact=0x0) at ../sysdeps/unix/sysv/linux/sigaction.c:51
#4  0x00007ffff7b674d0 in __pthread_initialize_minimal_internal () at nptl-init.c:290
#5  0x00007ffff7b66488 in _init () at ../sysdeps/powerpc/powerpc64/crti.S:78
#6  0x00007ffff7fc02a0 in call_init (env=0x7ffffffff558, argv=0x7ffffffff548, argc=1, l=0x7ffff7ff29b0) at dl-init.c:58
#7  call_init (env=0x7ffffffff558, argv=0x7ffffffff548, argc=1, l=0x7ffff7ff29b0) at dl-init.c:28
#8  _dl_init (main_map=0x7ffff7ff1170, argc=<optimized out>, argv=0x7ffffffff548, env=0x7ffffffff558) at dl-init.c:86
#9  0x00007ffff7fa158c in _dl_start_user () from /lib64/ld64.so.2
(gdb)

To fix Power ELFv2, we can use

#ifdef __powerpc64__
#  define PPC64_GLOBALENTRY(trampoline)                                        \
    "addis 2, 12, .TOC.-" trampoline "@ha\n"                                   \
    "addi 2, 2, .TOC.-" trampoline "@l\n"                                      \
    ".localentry " trampoline ",.-" trampoline "\n"
#else
#  define PPC64_GLOBALENTRY(trampoline)
#endif

#  define DECLARE_WRAPPER(ret_type, func, ...)                                 \
    extern "C" ret_type func(__VA_ARGS__);                                     \
    extern "C" ret_type TRAMPOLINE(func)(__VA_ARGS__);                         \
    extern "C" ret_type __interceptor_##func(__VA_ARGS__)                      \
        INTERCEPTOR_ATTRIBUTE __attribute__((weak)) ALIAS(WRAP(func));         \
    asm(                                                                       \
      ".text\n"                                                                \
      __ASM_WEAK_WRAPPER(func)                                                 \
      ".set " #func ", " SANITIZER_STRINGIFY(TRAMPOLINE(func)) "\n"            \
      ".globl " SANITIZER_STRINGIFY(TRAMPOLINE(func)) "\n"                     \
      ".type  " SANITIZER_STRINGIFY(TRAMPOLINE(func)) ", @function\n"          \
      SANITIZER_STRINGIFY(TRAMPOLINE(func)) ":\n"                              \
      SANITIZER_STRINGIFY(CFI_STARTPROC) "\n"                                  \
      PPC64_GLOBALENTRY(SANITIZER_STRINGIFY(TRAMPOLINE(func)))                 \
      SANITIZER_STRINGIFY(ASM_TAIL_CALL) " __interceptor_"                     \
        SANITIZER_STRINGIFY(ASM_PREEMPTIBLE_SYM(func)) "\n"                    \
      SANITIZER_STRINGIFY(CFI_ENDPROC) "\n"                                    \
      ".size  " SANITIZER_STRINGIFY(TRAMPOLINE(func)) ", "                     \
           ".-" SANITIZER_STRINGIFY(TRAMPOLINE(func)) "\n"                     \
    );

I have some brief description on
https://maskray.me/blog/2023-02-26-linker-notes-on-power-isa#tail-call

[jrtc27]

RISC-V should have @plt due to the stupid broken GNU toolchain semantics (which may or may not have been fixed to match LLVM these days)

[MaskRay]

In D151085#4382021, @jrtc27 wrote:

RISC-V should have @plt due to the stupid broken GNU toolchain semantics (which may or may not have been fixed to match LLVM these days)

Is that special handling of undefined weak for R_RISCV_CALL in GNU ld?
The trampoline ensures the ASM_PREEMPTIBLE_SYM jump target is always defined, so the semantics do not apply.

---

This patch will break Power ELFv2, so I requested changes. I'll update the previous comment to be clearer.

[jrtc27]

In D151085#4382139, @MaskRay wrote:

In D151085#4382021, @jrtc27 wrote:

RISC-V should have @plt due to the stupid broken GNU toolchain semantics (which may or may not have been fixed to match LLVM these days)

Is that special handling of undefined weak for R_RISCV_CALL in GNU ld?
The trampoline ensures the ASM_PREEMPTIBLE_SYM jump target is always defined, so the semantics do not apply.

No, there are other weird and broken behaviours when you don't use the PLT variant. They probably don't apply here (AFAIR they were to do with mixing PLT and non-PLT relocations), but it's always better to use the PLT variant.

[melver]

• Fix RISC-V preemptible symbol, use @plt (reported by jrtc27)
• Fix PPC64 tail call, needs global entry (reported by MaskRay)

[melver]

In D151085#4382018, @MaskRay wrote:

The current version will break most check-asan tests on ppc64le due to complexity of global entry/local entry in to TOC ABI.
[...]

To fix Power ELFv2, we can use

#ifdef __powerpc64__
#  define PPC64_GLOBALENTRY(trampoline)                                        \
    "addis 2, 12, .TOC.-" trampoline "@ha\n"                                   \
    "addi 2, 2, .TOC.-" trampoline "@l\n"                                      \
    ".localentry " trampoline ",.-" trampoline "\n"
#else
#  define PPC64_GLOBALENTRY(trampoline)
#endif

#  define DECLARE_WRAPPER(ret_type, func, ...)                                 \
    extern "C" ret_type func(__VA_ARGS__);                                     \
    extern "C" ret_type TRAMPOLINE(func)(__VA_ARGS__);                         \
    extern "C" ret_type __interceptor_##func(__VA_ARGS__)                      \
        INTERCEPTOR_ATTRIBUTE __attribute__((weak)) ALIAS(WRAP(func));         \
    asm(                                                                       \
      ".text\n"                                                                \
      __ASM_WEAK_WRAPPER(func)                                                 \
      ".set " #func ", " SANITIZER_STRINGIFY(TRAMPOLINE(func)) "\n"            \
      ".globl " SANITIZER_STRINGIFY(TRAMPOLINE(func)) "\n"                     \
      ".type  " SANITIZER_STRINGIFY(TRAMPOLINE(func)) ", @function\n"          \
      SANITIZER_STRINGIFY(TRAMPOLINE(func)) ":\n"                              \
      SANITIZER_STRINGIFY(CFI_STARTPROC) "\n"                                  \
      PPC64_GLOBALENTRY(SANITIZER_STRINGIFY(TRAMPOLINE(func)))                 \
      SANITIZER_STRINGIFY(ASM_TAIL_CALL) " __interceptor_"                     \
        SANITIZER_STRINGIFY(ASM_PREEMPTIBLE_SYM(func)) "\n"                    \
      SANITIZER_STRINGIFY(CFI_ENDPROC) "\n"                                    \
      ".size  " SANITIZER_STRINGIFY(TRAMPOLINE(func)) ", "                     \
           ".-" SANITIZER_STRINGIFY(TRAMPOLINE(func)) "\n"                     \
    );

I have some brief description on
https://maskray.me/blog/2023-02-26-linker-notes-on-power-isa#tail-call

Thanks - should be fixed (for the inline asm and normal asm version).

[MaskRay]

powerpc64{,le} ELFv2 breakages are mostly fixed, but -shared-libsan will fail. -shared-libsan is used by GCC, so we should fix it, not just UNSUPPORTED the test (compiler-rt/test/asan/TestCases/replaceable_new_delete_shared.cpp).

(gdb) bt
#0  0x00007ffff75818f8 in __plt___interceptor_memcpy () from /home/maskray/llvm/out/debug/lib/clang/17/lib/powerpc64le-unknown-linux-gnu/libclang_rt.asan.so
#1  0x00007ffff6ccb1f4 in __libc_sigaction (sig=<optimized out>, act=0x7ffffffff260, oact=0x0) at ../sysdeps/unix/sysv/linux/sigaction.c:51
#2  0x00007ffff6cb74d0 in __pthread_initialize_minimal_internal () at nptl-init.c:290
#3  0x00007ffff6cb6488 in _init () at ../sysdeps/powerpc/powerpc64/crti.S:78
#4  0x00007ffff7fc02a0 in call_init (env=0x7ffffffff4b8, argv=0x7ffffffff4a8, argc=1, l=0x7ffff7ff44d0) at dl-init.c:58
#5  call_init (env=0x7ffffffff4b8, argv=0x7ffffffff4a8, argc=1, l=0x7ffff7ff44d0) at dl-init.c:28
#6  _dl_init (main_map=0x7ffff7ff1170, argc=<optimized out>, argv=0x7ffffffff4a8, env=0x7ffffffff4b8) at dl-init.c:86
#7  0x00007ffff7fa158c in _dl_start_user () from /lib64/ld64.so.2

For powerpc64{,le} ELFv2, we should use b foo; nop instead of b foo.
Linkers rewrite the nop if it is preemptible.

Therefore, this part

SANITIZER_STRINGIFY(ASM_TAIL_CALL) " __interceptor_"                     \
        SANITIZER_STRINGIFY(ASM_PREEMPTIBLE_SYM(func)) "\n"

needs adjustment for powerpc64.

# objdump -wdr projects/compiler-rt/test/asan/POWERPC64LELinuxConfig/TestCases/Output/use-after-free.cpp.tmp
0000000000000018 <__interceptor_trampoline_memcpy>:
      18:       00 00 4c 3c     addis   r2,r12,0        18: R_PPC64_REL16_HA    .TOC.
      1c:       00 00 42 38     addi    r2,r2,0 1c: R_PPC64_REL16_LO    .TOC.+0x4
      20:       00 00 00 48     b       20 <__interceptor_trampoline_memcpy+0x8>        20: R_PPC64_REL24       __interceptor_memcpy


# objdump -wdr lib/clang/17/lib/powerpc64le-unknown-linux-gnu/libclang_rt.asan.so
00000000001d2cb8 <__interceptor_trampoline_memcpy>:
  1d2cb8:       04 00 41 e8     ld      r2,4(r1)
  1d2cbc:       a0 86 42 38     addi    r2,r2,-31072
  1d2cc0:       30 ec 01 48     b       1f18f0 <__plt___interceptor_memcpy>

[melver]

• Fix ASM_PREEMPTIBLE_SYM for ppc64 (it wants a nop after 'b sym').

[melver]

In D151085#4385185, @MaskRay wrote:

powerpc64{,le} ELFv2 breakages are mostly fixed, but -shared-libsan will fail. -shared-libsan is used by GCC, so we should fix it, not just UNSUPPORTED the test (compiler-rt/test/asan/TestCases/replaceable_new_delete_shared.cpp).

[...]

For powerpc64{,le} ELFv2, we should use b foo; nop instead of b foo.
Linkers rewrite the nop if it is preemptible.

Ouch - thanks, done.

I'm starting to think if there was a nice way to get what we want in plain C++, but it'd require updating all interceptors to take the number of function args e.g. INTERCEPTOR{1,2,3,4,5}.

But that doesn't solve the problem with pure .S implementations, which may not be tested in all these cases, and without the inline asm() version we'd not catch these issues as easily.
I'm torn, but probably keeping it all in asm is, paradoxically, more reliable in this case...

[MaskRay]

compiler-rt/lib/interception/tests/interception_linux_foreign_test.cpp

39

For * and +, the precedence is apparent and we don't usually add parens.

[MaskRay]

If powerpc64{,le} ELFv2 continue to be a problem, perhaps we should give up 3 interceptors for it.

[melver]

• Simplify arith expressions in tests.

[melver]

• Do not implement trampoline for i386 and powerpc64 due to complexities of resolving a preemptible symbol. We may implement them properly at a later point if the need arises (howeer, I suspect nobody needs this on i386 anyway).
• Attempt to fix s390.

[ro]

This patch caused 4 regressions on Solaris/SPARC:

SanitizerCommon-ubsan-sparc-SunOS :: Posix/replace_sigaction.cpp
SanitizerCommon-ubsan-sparc-SunOS :: Posix/signal.cpp
SanitizerCommon-ubsan-sparcv9-SunOS :: Posix/replace_sigaction.cpp
SanitizerCommon-ubsan-sparcv9-SunOS :: Posix/signal.cpp

E.g. Posix/replace_sigaction.cpp FAILs with

UndefinedBehaviorSanitizer:DEADLYSIGNAL
UndefinedBehaviorSanitizer:DEADLYSIGNAL
UndefinedBehaviorSanitizer: nested bug in the same thread, aborting.

gdb shows

Thread 2 received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1 (LWP 1)]
0x00000008 in ?? ()
(gdb) bt
#0  0x00000008 in ?? ()
#1  0xff1ae3a8 in defrag () from /usr/lib/ld.so.1
#2  0x0005aedc in install<void (int)>(void (*)(int), sigaction*) (
    handler=0x5a4f8 <sahandler(int)>, prev=0x0)
    at /vol/llvm/src/llvm-project/local/compiler-rt/test/sanitizer_common/TestCases/Posix/replace_sigaction.cpp:23
#3  0x0005a604 in test<void (int), void (int)>(void (*)(int), void (*)(int)) (
    from=0x5a4f8 <sahandler(int)>, to=0x5a4f8 <sahandler(int)>)
    at /vol/llvm/src/llvm-project/local/compiler-rt/test/sanitizer_common/TestCases/Posix/replace_sigaction.cpp:40
#4  0x0005a560 in testAll<void (int)>(void (*)(int)) (
    to=0x5a4f8 <sahandler(int)>)
    at /vol/llvm/src/llvm-project/local/compiler-rt/test/sanitizer_common/TestCases/Posix/replace_sigaction.cpp:55
#5  0x0005a4c8 in main ()
    at /vol/llvm/src/llvm-project/local/compiler-rt/test/sanitizer_common/TestCases/Posix/replace_sigaction.cpp:62

truss (the Solaris syscall tracer) gives

Incurred fault #6, FLTBOUNDS  %pc = 0x00000008
  siginfo: SIGSEGV SEGV_MAPERR addr=0x00000008
Received signal #11, SIGSEGV [caught]
  siginfo: SIGSEGV SEGV_MAPERR addr=0x00000008

I suspected that this section in compiler-rt/lib/sanitizer_common/sanitizer_asm.h

#if defined(__x86_64__) || defined(__i386__) || defined(__sparc__)
# define ASM_TAIL_CALL jmp

is the culprit, and indeed this doesn't set the jmp insn's delay slot at all. If you disassemble the resulting code, you get

(gdb) x/10i __interceptor_trampoline_signal
   0x594c4 <signal>:	jmp  4
   0x594c8 <sigaction>:	jmp  8
   0x594cc <__interceptor_signal>:	save  %sp, -96, %sp

which explains the SEGV. I managed to avoid this with the following snippet:

--- a/compiler-rt/lib/sanitizer_common/sanitizer_asm.h
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_asm.h
@@ -66,7 +66,7 @@
 # define ASM_SIZE(symbol) .size symbol, .-symbol
 # define ASM_SYMBOL(symbol) symbol
 # define ASM_SYMBOL_INTERCEPTOR(symbol) symbol
-# if defined(__i386__) || defined(__powerpc__)
+# if defined(__i386__) || defined(__powerpc__) || defined(__sparc__)
 // For details, see interception.h
 #  define ASM_WRAPPER_NAME(symbol) __interceptor_##symbol
 #  define ASM_TRAMPOLINE_ALIAS(symbol, name)                                   
\

I'd expect that other targets with delay slots have similar problems.

[melver]

In D151085#4416709, @ro wrote:

is the culprit, and indeed this doesn't set the `jmp` insn's delay slot at all.  If you disassemble the resulting code, you get

(gdb) x/10i __interceptor_trampoline_signal

0x594c4 <signal>:	jmp  4
0x594c8 <sigaction>:	jmp  8
0x594cc <__interceptor_signal>:	save  %sp, -96, %sp

which explains the `SEGV`.  I managed to avoid this with the following snippet:

• a/compiler-rt/lib/sanitizer_common/sanitizer_asm.h

+++ b/compiler-rt/lib/sanitizer_common/sanitizer_asm.h
@@ -66,7 +66,7 @@

1. define ASM_SIZE(symbol) .size symbol, .-symbol
2. define ASM_SYMBOL(symbol) symbol
3. define ASM_SYMBOL_INTERCEPTOR(symbol) symbol
4. if defined(i386) || defined(powerpc)

+# if defined(i386) || defined(powerpc) || defined(sparc)
// For details, see interception.h

1. define ASM_WRAPPER_NAME(symbol) __interceptor_##symbol
2. define ASM_TRAMPOLINE_ALIAS(symbol, name)

\

I'd expect that other targets with delay slots have similar problems.

Do you think you could fix up ASM_TAIL_CALL for sparc, or just revert to the interception without trampoline?
In the latter case, I can just commit the above if you like - let me know what you prefer.

[ro]

In D151085#4416927, @melver wrote:

Do you think you could fix up ASM_TAIL_CALL for sparc, or just revert to the interception without trampoline?
In the latter case, I can just commit the above if you like - let me know what you prefer.

I tried the following:

--- a/compiler-rt/lib/sanitizer_common/sanitizer_asm.h
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_asm.h
@@ -53,6 +53,10 @@
 # define ASM_TAIL_CALL tail
 #endif
 
+#if defined(__sparc__)
+# define ASM_DELAY_INSN nop
+#endif
+
 #if defined(__ELF__) && defined(__x86_64__) || defined(__i386__) || \
     defined(__riscv)
 # define ASM_PREEMPTIBLE_SYM(sym) sym@plt
@@ -88,6 +92,7 @@
          __interceptor_trampoline_##name:                                      \
                  CFI_STARTPROC;                                                \
                  ASM_TAIL_CALL ASM_PREEMPTIBLE_SYM(__interceptor_##name);      \
+                 ASM_DELAY_INSN;                                               \
                  CFI_ENDPROC;                                                  \
          ASM_SIZE(__interceptor_trampoline_##name)
 #  define ASM_INTERCEPTOR_TRAMPOLINE_SUPPORT 1

but that didn't even get the nop into the object. Somehow, I also failed to use -save-temps to check the assembler output. Even if this can be made to work, it would need extra scrutiny by someone who really understands the SPARC ISA.

At least for now, I guess it's best to go for my workaround patch.

[melver]

In D151085#4417052, @ro wrote:

In D151085#4416927, @melver wrote:

Do you think you could fix up ASM_TAIL_CALL for sparc, or just revert to the interception without trampoline?
In the latter case, I can just commit the above if you like - let me know what you prefer.

I tried the following:

--- a/compiler-rt/lib/sanitizer_common/sanitizer_asm.h
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_asm.h
@@ -53,6 +53,10 @@
 # define ASM_TAIL_CALL tail
 #endif
 
+#if defined(__sparc__)
+# define ASM_DELAY_INSN nop
+#endif
+
 #if defined(__ELF__) && defined(__x86_64__) || defined(__i386__) || \
     defined(__riscv)
 # define ASM_PREEMPTIBLE_SYM(sym) sym@plt
@@ -88,6 +92,7 @@
          __interceptor_trampoline_##name:                                      \
                  CFI_STARTPROC;                                                \
                  ASM_TAIL_CALL ASM_PREEMPTIBLE_SYM(__interceptor_##name);      \
+                 ASM_DELAY_INSN;                                               \
                  CFI_ENDPROC;                                                  \
          ASM_SIZE(__interceptor_trampoline_##name)
 #  define ASM_INTERCEPTOR_TRAMPOLINE_SUPPORT 1

but that didn't even get the nop into the object. Somehow, I also failed to use -save-temps to check the assembler output. Even if this can be made to work, it would need extra scrutiny by someone who really understands the SPARC ISA.

There are 2 places that need to be kept in sync per comment "// Keep trampoline implementation in sync with interception/interception.h".

At least for now, I guess it's best to go for my workaround patch.

But I think it's good to just do the simple fix first, and if you feel that you need the support as described in this patch, then we can see if it's possible to get it to work.

https://reviews.llvm.org/rG2c1ec06d45a9

[ro]

In D151085#4417098, @melver wrote:

In D151085#4417052, @ro wrote:

In D151085#4416927, @melver wrote:

Do you think you could fix up ASM_TAIL_CALL for sparc, or just revert to the interception without trampoline?
In the latter case, I can just commit the above if you like - let me know what you prefer.

I tried the following:

--- a/compiler-rt/lib/sanitizer_common/sanitizer_asm.h
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_asm.h
@@ -53,6 +53,10 @@
 # define ASM_TAIL_CALL tail
 #endif
 
+#if defined(__sparc__)
+# define ASM_DELAY_INSN nop
+#endif
+
 #if defined(__ELF__) && defined(__x86_64__) || defined(__i386__) || \
     defined(__riscv)
 # define ASM_PREEMPTIBLE_SYM(sym) sym@plt
@@ -88,6 +92,7 @@
          __interceptor_trampoline_##name:                                      \
                  CFI_STARTPROC;                                                \
                  ASM_TAIL_CALL ASM_PREEMPTIBLE_SYM(__interceptor_##name);      \
+                 ASM_DELAY_INSN;                                               \
                  CFI_ENDPROC;                                                  \
          ASM_SIZE(__interceptor_trampoline_##name)
 #  define ASM_INTERCEPTOR_TRAMPOLINE_SUPPORT 1

but that didn't even get the nop into the object. Somehow, I also failed to use -save-temps to check the assembler output. Even if this can be made to work, it would need extra scrutiny by someone who really understands the SPARC ISA.

There are 2 places that need to be kept in sync per comment "// Keep trampoline implementation in sync with interception/interception.h".

Indeed: that way I got the nop into the object. However, the jmp insn still jumps to an absolute (unmapped) address, causing the SEGV. I don't yet see why.

At least for now, I guess it's best to go for my workaround patch.

But I think it's good to just do the simple fix first, and if you feel that you need the support as described in this patch, then we can see if it's possible to get it to work.

https://reviews.llvm.org/rG2c1ec06d45a9

Excellent, thanks a lot. After all, the trampoline is just an optimization, so correctness should go first.

compiler-rt/lib/interception/interception.h

 //===-- interception.h ------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file is a part of AddressSanitizer, an address sanity checker.
 //
 // Machinery for providing replacements/wrappers for system functions.
 //===----------------------------------------------------------------------===//
 
 #ifndef INTERCEPTION_H
 #define INTERCEPTION_H
 
+#include "sanitizer_common/sanitizer_asm.h"
 #include "sanitizer_common/sanitizer_internal_defs.h"
 
 #if !SANITIZER_LINUX && !SANITIZER_FREEBSD && !SANITIZER_APPLE &&      \
     !SANITIZER_NETBSD && !SANITIZER_WINDOWS && !SANITIZER_FUCHSIA && \
     !SANITIZER_SOLARIS
 #  error "Interception doesn't work on this operating system."
 #endif
 

 //           DECLARE_REAL(int, foo, const char *bar, double baz) in your
 //           source file (to define a pointer to overriden function).
 //        3. Some Mac functions have symbol variants discriminated by
 //           additional suffixes, e.g. _$UNIX2003 (see
 //           https://developer.apple.com/library/mac/#releasenotes/Darwin/SymbolVariantsRelNotes/index.html
 //           for more details). To intercept such functions you need to use the
 //           INTERCEPTOR_WITH_SUFFIX(...) macro.
 
-// How it works:
-// To replace system functions on Linux we just need to declare functions
-// with same names in our library and then obtain the real function pointers
+// How it works on Linux
+// ---------------------
+//
+// To replace system functions on Linux we just need to declare functions with
+// the same names in our library and then obtain the real function pointers
 // using dlsym().
-// There is one complication. A user may also intercept some of the functions
-// we intercept. To resolve this we declare our interceptors with __interceptor_
-// prefix, and then make actual interceptors weak aliases to __interceptor_
-// functions.
-//
-// This is not so on Mac OS, where the two-level namespace makes
-// our replacement functions invisible to other libraries. This may be overcomed
-// using the DYLD_FORCE_FLAT_NAMESPACE, but some errors loading the shared
-// libraries in Chromium were noticed when doing so.
+//
+// There is one complication: a user may also intercept some of the functions we
+// intercept. To allow for up to 3 interceptors (including ours) of a given
+// function "func", the interceptor implementation is in ___interceptor_func,
+// which is aliased by a weak function __interceptor_func, which in turn is
+// aliased (via a trampoline) by weak wrapper function "func".
+//
+// Most user interceptors should define a foreign interceptor as follows:
+//
+//  - provide a non-weak function "func" that performs interception;
+//  - if __interceptor_func exists, call it to perform the real functionality;
+//  - if it does not exist, figure out the real function and call it instead.
+//
+// In rare cases, a foreign interceptor (of another dynamic analysis runtime)
+// may be defined as follows:
+//
+//  - provide a non-weak function __interceptor_func that performs interception;
+//  - if ___interceptor_func exists, call it to perform the real functionality;
+//  - if it does not exist, figure out the real function and call it instead;
+//  - provide a weak function "func" that is an alias to __interceptor_func.
+//
+// With this protocol, sanitizer interceptors, foreign user interceptors, and
+// foreign interceptors of other dynamic analysis runtimes, or any combination
+// thereof, may co-exist simultaneously.
+//
+// How it works on Mac OS
+// ----------------------
+//
+// This is not so on Mac OS, where the two-level namespace makes our replacement
+// functions invisible to other libraries. This may be overcomed using the
+// DYLD_FORCE_FLAT_NAMESPACE, but some errors loading the shared libraries in
+// Chromium were noticed when doing so.
+//
 // Instead we create a dylib containing a __DATA,__interpose section that
 // associates library functions with their wrappers. When this dylib is
-// preloaded before an executable using DYLD_INSERT_LIBRARIES, it routes all
-// the calls to interposed functions done through stubs to the wrapper
-// functions.
+// preloaded before an executable using DYLD_INSERT_LIBRARIES, it routes all the
+// calls to interposed functions done through stubs to the wrapper functions.
+//
 // As it's decided at compile time which functions are to be intercepted on Mac,
 // INTERCEPT_FUNCTION() is effectively a no-op on this system.
 
 #if SANITIZER_APPLE
 #include <sys/cdefs.h>  // For __DARWIN_ALIAS_C().
 
 // Just a pair of pointers.
 struct interpose_substitution {

 # define WRAP(x) __asan_wrap_##x
 # define TRAMPOLINE(x) WRAP(x)
 # define INTERCEPTOR_ATTRIBUTE __declspec(dllexport)
 # define DECLARE_WRAPPER(ret_type, func, ...) \
     extern "C" ret_type func(__VA_ARGS__);
 # define DECLARE_WRAPPER_WINAPI(ret_type, func, ...) \
     extern "C" __declspec(dllimport) ret_type __stdcall func(__VA_ARGS__);
 #elif !SANITIZER_FUCHSIA  // LINUX, FREEBSD, NETBSD, SOLARIS
-# define WRAP(x) __interceptor_ ## x
-# define TRAMPOLINE(x) WRAP(x)
+# define WRAP(x) ___interceptor_ ## x
+# define TRAMPOLINE(x) __interceptor_trampoline_ ## x
 # define INTERCEPTOR_ATTRIBUTE __attribute__((visibility("default")))
 # if SANITIZER_FREEBSD || SANITIZER_NETBSD
 // FreeBSD's dynamic linker (incompliantly) gives non-weak symbols higher
 // priority than weak ones so weak aliases won't work for indirect calls
 // in position-independent (-fPIC / -fPIE) mode.
-# define OVERRIDE_ATTRIBUTE
+#  define __ASM_WEAK_WRAPPER(func)
 # else  // SANITIZER_FREEBSD || SANITIZER_NETBSD
-# define OVERRIDE_ATTRIBUTE __attribute__((weak))
+#  define __ASM_WEAK_WRAPPER(func) ".weak " #func "\n"
 # endif  // SANITIZER_FREEBSD || SANITIZER_NETBSD
+//
+// Note: Weak aliases of weak aliases do not work, therefore we need to set up a
+// trampoline function. The function "func" is a weak alias to the trampoline
+// (so that we may check if "func" was overridden), which calls the weak
+// function __interceptor_func, which in turn aliases the actual interceptor
+// implementation ___interceptor_func:
+//
+//    [wrapper "func": weak] --(alias)--> [TRAMPOLINE(func)]
+//                                                |
+//                     +--------(tail call)-------+
+//                     |
+//                     v
+//      [__interceptor_func: weak] --(alias)--> [WRAP(func)]
+//
+// We use inline assembly to define most of this, because not all compilers
+// support functions with the "naked" attribute with every architecture.
+//
 # define DECLARE_WRAPPER(ret_type, func, ...)                                  \
-    extern "C" ret_type func(__VA_ARGS__) INTERCEPTOR_ATTRIBUTE                \
-      OVERRIDE_ATTRIBUTE ALIAS(WRAP(func));
+    extern "C" ret_type func(__VA_ARGS__);                                     \
+    extern "C" ret_type TRAMPOLINE(func)(__VA_ARGS__);                         \
+    extern "C" ret_type __interceptor_##func(__VA_ARGS__)                      \
+      INTERCEPTOR_ATTRIBUTE __attribute__((weak)) ALIAS(WRAP(func));           \
+    asm(                                                                       \
+      ".text\n"                                                                \
+      __ASM_WEAK_WRAPPER(func)                                                 \
+      ".set " #func ", " SANITIZER_STRINGIFY(TRAMPOLINE(func)) "\n"            \
+      ".globl " SANITIZER_STRINGIFY(TRAMPOLINE(func)) "\n"                     \
+      ".type  " SANITIZER_STRINGIFY(TRAMPOLINE(func)) ", @function\n"          \
+      SANITIZER_STRINGIFY(TRAMPOLINE(func)) ":\n"                              \
+      SANITIZER_STRINGIFY(CFI_STARTPROC) "\n"                                  \
+      SANITIZER_STRINGIFY(ASM_TAIL_CALL) " __interceptor_"                     \
+        SANITIZER_STRINGIFY(ASM_PREEMPTIBLE_SYM(func)) "\n"                    \
+      SANITIZER_STRINGIFY(CFI_ENDPROC) "\n"                                    \
+      ".size  " SANITIZER_STRINGIFY(TRAMPOLINE(func)) ", "                     \
+           ".-" SANITIZER_STRINGIFY(TRAMPOLINE(func)) "\n"                     \
+    );
 #endif
 
 #if SANITIZER_FUCHSIA
 // There is no general interception at all on Fuchsia.
 // Sanitizer runtimes just define functions directly to preempt them,
 // and have bespoke ways to access the underlying libc functions.
 # include <zircon/sanitizer.h>
 # define INTERCEPTOR_ATTRIBUTE __attribute__((visibility("default")))

compiler-rt/lib/interception/tests/CMakeLists.txt

 include(CompilerRTCompile)
 
 filter_available_targets(INTERCEPTION_UNITTEST_SUPPORTED_ARCH x86_64 i386 mips64 mips64el)
 
 set(INTERCEPTION_UNITTESTS
   interception_linux_test.cpp
+  interception_linux_foreign_test.cpp
   interception_test_main.cpp
   interception_win_test.cpp
   )
 
 set(INTERCEPTION_TEST_HEADERS)
 
 set(INTERCEPTION_TEST_CFLAGS_COMMON
   ${COMPILER_RT_UNITTEST_CFLAGS}
   ${COMPILER_RT_GTEST_CFLAGS}
   ${SANITIZER_TEST_CXX_CFLAGS}
   -I${COMPILER_RT_SOURCE_DIR}/include
   -I${COMPILER_RT_SOURCE_DIR}/lib
   -I${COMPILER_RT_SOURCE_DIR}/lib/interception
   -DSANITIZER_COMMON_NO_REDEFINE_BUILTINS
   -fno-rtti
+  -fno-builtin-isdigit
+  -fno-builtin-isalpha
+  -fno-builtin-isalnum
+  -fno-builtin-islower
   -O2
   -Werror=sign-compare)
 
 set(INTERCEPTION_TEST_LINK_FLAGS_COMMON
   ${COMPILER_RT_UNITTEST_LINK_FLAGS}
   ${COMPILER_RT_UNWINDER_LINK_LIBS}
   ${SANITIZER_TEST_CXX_LIBRARIES})
 

compiler-rt/lib/interception/tests/interception_linux_foreign_test.cpp

+//===-- interception_linux_foreign_test.cpp -------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file is a part of ThreadSanitizer/AddressSanitizer runtime.
+//
+// Tests that foreign interceptors work.
+//
+//===----------------------------------------------------------------------===//
+
+// Do not declare functions in ctype.h.
+#define __NO_CTYPE
+
+#include "gtest/gtest.h"
+#include "sanitizer_common/sanitizer_internal_defs.h"
+
+#if SANITIZER_LINUX
+
+extern "C" int isalnum(int d);
+extern "C" int __interceptor_isalpha(int d);
+extern "C" int ___interceptor_isalnum(int d);  // the sanitizer interceptor
+extern "C" int ___interceptor_islower(int d);  // the sanitizer interceptor
+
+namespace __interception {
+extern int isalpha_called;
+extern int isalnum_called;
+extern int islower_called;
+}  // namespace __interception
+using namespace __interception;
+
+// Direct foreign interceptor. This is the "normal" protocol that other
+// interceptors should follow.
+extern "C" int isalpha(int d) {
+  // Use non-commutative arithmetic to verify order of calls.
+  isalpha_called = (isalpha_called * 10) + 1;
+  return __interceptor_isalpha(d);
+}
+
+// Indirect foreign interceptor. This pattern should only be used to co-exist
+// with direct foreign interceptors and sanitizer interceptors.
+extern "C" int __interceptor_isalnum(int d) {
+  isalnum_called = (isalnum_called * 10) + 1;
+  return ___interceptor_isalnum(d);
+}
+
+extern "C" int __interceptor_islower(int d) {
+  islower_called = (islower_called * 10) + 2;
+  return ___interceptor_islower(d);
+}
+
+extern "C" int islower(int d) {
+  islower_called = (islower_called * 10) + 1;
+  return __interceptor_islower(d);
+}
+
+namespace __interception {
+
+TEST(ForeignInterception, ForeignOverrideDirect) {
+  isalpha_called = 0;
+  EXPECT_NE(0, isalpha('a'));
+  EXPECT_EQ(13, isalpha_called);
+  isalpha_called = 0;
+  EXPECT_EQ(0, isalpha('_'));
+  EXPECT_EQ(13, isalpha_called);
+}
+
+TEST(ForeignInterception, ForeignOverrideIndirect) {
+  isalnum_called = 0;
+  EXPECT_NE(0, isalnum('a'));
+  EXPECT_EQ(13, isalnum_called);
+  isalnum_called = 0;
+  EXPECT_EQ(0, isalnum('_'));
+  EXPECT_EQ(13, isalnum_called);
+}
+
+TEST(ForeignInterception, ForeignOverrideThree) {
+  islower_called = 0;
+  EXPECT_NE(0, islower('a'));
+  EXPECT_EQ(123, islower_called);
+  islower_called = 0;
+  EXPECT_EQ(0, islower('_'));
+  EXPECT_EQ(123, islower_called);
+}
+
+}  // namespace __interception
+
+#endif  // SANITIZER_LINUX

compiler-rt/lib/interception/tests/interception_linux_test.cpp

 //===-- interception_linux_test.cpp ---------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file is a part of ThreadSanitizer/AddressSanitizer runtime.
 // Tests for interception_linux.h.
 //
 //===----------------------------------------------------------------------===//
 
-// Do not declare isdigit in ctype.h.
+// Do not declare functions in ctype.h.
 #define __NO_CTYPE
 
 #include "interception/interception.h"
 
+#include <stdlib.h>
+
 #include "gtest/gtest.h"
 
-// Too slow for debug build
-#if !SANITIZER_DEBUG
 #if SANITIZER_LINUX
 
-static int InterceptorFunctionCalled;
+static int isdigit_called;
+namespace __interception {
+int isalpha_called;
+int isalnum_called;
+int islower_called;
+}  // namespace __interception
+using namespace __interception;
 
 DECLARE_REAL(int, isdigit, int);
+DECLARE_REAL(int, isalpha, int);
+DECLARE_REAL(int, isalnum, int);
+DECLARE_REAL(int, islower, int);
+
+INTERCEPTOR(void *, malloc, SIZE_T s) { return calloc(1, s); }
+INTERCEPTOR(void, dummy_doesnt_exist__, ) { __builtin_trap(); }
 
 INTERCEPTOR(int, isdigit, int d) {
-  ++InterceptorFunctionCalled;
+  ++isdigit_called;
   return d >= '0' && d <= '9';
 }
 
+INTERCEPTOR(int, isalpha, int d) {
+  // Use non-commutative arithmetic to verify order of calls.
+  isalpha_called = (isalpha_called * 10) + 3;
+  return (d >= 'a' && d <= 'z') || (d >= 'A' && d <= 'Z');
+}
+
+INTERCEPTOR(int, isalnum, int d) {
+  isalnum_called = (isalnum_called * 10) + 3;
+  return __interceptor_isalpha(d) || __interceptor_isdigit(d);
+}
+
+INTERCEPTOR(int, islower, int d) {
+  islower_called = (islower_called * 10) + 3;
+  return d >= 'a' && d <= 'z';
+}
+
 namespace __interception {
 
 TEST(Interception, InterceptFunction) {
   uptr malloc_address = 0;
-  EXPECT_TRUE(InterceptFunction("malloc", &malloc_address, 0, 0));
+  EXPECT_TRUE(InterceptFunction("malloc", &malloc_address, (uptr)&malloc,
+                                (uptr)&__interceptor_trampoline_malloc));
   EXPECT_NE(0U, malloc_address);
-  EXPECT_FALSE(InterceptFunction("malloc", &malloc_address, 0, 1));
+  EXPECT_FALSE(InterceptFunction("malloc", &malloc_address, (uptr)&calloc,
+                                 (uptr)&__interceptor_trampoline_malloc));
 
   uptr dummy_address = 0;
-  EXPECT_FALSE(InterceptFunction("dummy_doesnt_exist__", &dummy_address, 0, 0));
+  EXPECT_FALSE(InterceptFunction(
+      "dummy_doesnt_exist__", &dummy_address, (uptr)&dummy_doesnt_exist__,
+      (uptr)&__interceptor_trampoline_dummy_doesnt_exist__));
   EXPECT_EQ(0U, dummy_address);
 }
 
 TEST(Interception, Basic) {
   EXPECT_TRUE(INTERCEPT_FUNCTION(isdigit));
 
   // After interception, the counter should be incremented.
-  InterceptorFunctionCalled = 0;
+  isdigit_called = 0;
   EXPECT_NE(0, isdigit('1'));
-  EXPECT_EQ(1, InterceptorFunctionCalled);
+  EXPECT_EQ(1, isdigit_called);
   EXPECT_EQ(0, isdigit('a'));
-  EXPECT_EQ(2, InterceptorFunctionCalled);
+  EXPECT_EQ(2, isdigit_called);
 
   // Calling the REAL function should not affect the counter.
-  InterceptorFunctionCalled = 0;
+  isdigit_called = 0;
   EXPECT_NE(0, REAL(isdigit)('1'));
   EXPECT_EQ(0, REAL(isdigit)('a'));
-  EXPECT_EQ(0, InterceptorFunctionCalled);
+  EXPECT_EQ(0, isdigit_called);
+}
+
+TEST(Interception, ForeignOverrideDirect) {
+  // Actual interceptor is overridden.
+  EXPECT_FALSE(INTERCEPT_FUNCTION(isalpha));
+
+  isalpha_called = 0;
+  EXPECT_NE(0, isalpha('a'));
+  EXPECT_EQ(13, isalpha_called);
+  isalpha_called = 0;
+  EXPECT_EQ(0, isalpha('_'));
+  EXPECT_EQ(13, isalpha_called);
+
+  isalpha_called = 0;
+  EXPECT_NE(0, REAL(isalpha)('a'));
+  EXPECT_EQ(0, REAL(isalpha)('_'));
+  EXPECT_EQ(0, isalpha_called);
+}
+
+TEST(Interception, ForeignOverrideIndirect) {
+  // Actual interceptor is _not_ overridden.
+  EXPECT_TRUE(INTERCEPT_FUNCTION(isalnum));
+
+  isalnum_called = 0;
+  EXPECT_NE(0, isalnum('a'));
+  EXPECT_EQ(13, isalnum_called);
+  isalnum_called = 0;
+  EXPECT_EQ(0, isalnum('_'));
+  EXPECT_EQ(13, isalnum_called);
+
+  isalnum_called = 0;
+  EXPECT_NE(0, REAL(isalnum)('a'));
+  EXPECT_EQ(0, REAL(isalnum)('_'));
+  EXPECT_EQ(0, isalnum_called);
+}
+
+TEST(Interception, ForeignOverrideThree) {
+  // Actual interceptor is overridden.
+  EXPECT_FALSE(INTERCEPT_FUNCTION(islower));
+
+  islower_called = 0;
+  EXPECT_NE(0, islower('a'));
+  EXPECT_EQ(123, islower_called);
+  islower_called = 0;
+  EXPECT_EQ(0, islower('A'));
+  EXPECT_EQ(123, islower_called);
+
+  islower_called = 0;
+  EXPECT_NE(0, REAL(islower)('a'));
+  EXPECT_EQ(0, REAL(islower)('A'));
+  EXPECT_EQ(0, islower_called);
 }
 
 }  // namespace __interception
 
 #endif  // SANITIZER_LINUX
-#endif  // #if !SANITIZER_DEBUG

compiler-rt/lib/sanitizer_common/sanitizer_asm.h

 # define CFI_DEF_CFA_OFFSET(n)
 # define CFI_REL_OFFSET(reg, n)
 # define CFI_OFFSET(reg, n)
 # define CFI_DEF_CFA_REGISTER(reg)
 # define CFI_DEF_CFA(reg, n)
 # define CFI_RESTORE(reg)
 #endif
 
+#if defined(__x86_64__) || defined(__i386__) || defined(__sparc__)
+# define ASM_TAIL_CALL jmp
+#elif defined(__arm__) || defined(__aarch64__) || defined(__mips__) || \
+    defined(__powerpc__) || defined(__loongarch_lp64)
+# define ASM_TAIL_CALL b
+#elif defined(__s390__)
+# define ASM_TAIL_CALL jg
+#elif defined(__riscv)
+# define ASM_TAIL_CALL tail
+#else
+# error ASM_TAIL_CALL: Unimplemented architecture
+#endif
+
+#if defined(__ELF__) && (defined(__x86_64__) || defined(__i386__))
+# define ASM_PREEMPTIBLE_SYM(sym) sym@plt
+#else
+# define ASM_PREEMPTIBLE_SYM(sym) sym
+#endif
+
 #if !defined(__APPLE__)
 # define ASM_HIDDEN(symbol) .hidden symbol
 # define ASM_TYPE_FUNCTION(symbol) .type symbol, %function
 # define ASM_SIZE(symbol) .size symbol, .-symbol
 # define ASM_SYMBOL(symbol) symbol
 # define ASM_SYMBOL_INTERCEPTOR(symbol) symbol
-# define ASM_WRAPPER_NAME(symbol) __interceptor_##symbol
-# define ASM_TRAMPOLINE_ALIAS(symbol, name)                     \
-        .weak symbol;                                           \
-        .set symbol, ASM_WRAPPER_NAME(name)
-# define ASM_INTERCEPTOR_TRAMPOLINE(name)
+# define ASM_WRAPPER_NAME(symbol) ___interceptor_##symbol
+# define ASM_TRAMPOLINE_ALIAS(symbol, name)                                    \
+        .weak symbol;                                                          \
+        .set symbol, __interceptor_trampoline_##name
+# define ASM_INTERCEPTOR_TRAMPOLINE(name)                                      \
+        .weak __interceptor_##name;                                            \
+        .set __interceptor_##name, ASM_WRAPPER_NAME(name);                     \
+        .globl __interceptor_trampoline_##name;                                \
+        ASM_TYPE_FUNCTION(__interceptor_trampoline_##name);                    \
+        __interceptor_trampoline_##name:                                       \
+                CFI_STARTPROC;                                                 \
+                ASM_TAIL_CALL ASM_PREEMPTIBLE_SYM(__interceptor_##name);       \
+                CFI_ENDPROC;                                                   \
+        ASM_SIZE(__interceptor_trampoline_##name)
 #else
 # define ASM_HIDDEN(symbol)
 # define ASM_TYPE_FUNCTION(symbol)
 # define ASM_SIZE(symbol)
 # define ASM_SYMBOL(symbol) _##symbol
 # define ASM_SYMBOL_INTERCEPTOR(symbol) _wrap_##symbol
 # define ASM_WRAPPER_NAME(symbol) __interceptor_##symbol
 #endif

compiler-rt/lib/sanitizer_common/sanitizer_stacktrace_printer.cpp

if (!internal_strncmp(function, prefix, prefix_len))
       return function + prefix_len;
     return nullptr;
   };
   if (SANITIZER_APPLE) {
     if (const char *s = try_strip("wrap_"))
       return s;
   } else if (SANITIZER_WINDOWS) {
     if (const char *s = try_strip("__asan_wrap_"))
       return s;
   } else {
+    if (const char *s = try_strip("___interceptor_"))
+      return s;
     if (const char *s = try_strip("__interceptor_"))
       return s;
   }
   return function;
 }
 
 // sanitizer_symbolizer_markup.cpp implements these differently.
 #if !SANITIZER_SYMBOLIZER_MARKUP

compiler-rt/lib/sanitizer_common/scripts/gen_dynamic_list.py

for library in args.libraries:
         all_functions.extend(get_global_functions(args.nm_executable, library))
     function_set = set(all_functions)
     for func in all_functions:
         # Export new/delete operators.
         if func in new_delete:
             result.append(func)
             continue
         # Export interceptors.
-        match = re.match("__interceptor_(.*)", func)
+        match = re.match("_?__interceptor_(.*)", func)
         if match:
             result.append(func)
             # We have to avoid exporting the interceptors for versioned library
             # functions due to gold internal error.
             orig_name = match.group(1)
             if orig_name in function_set and (
                 args.version_list or orig_name not in versioned_functions
             ):