Ping

More workarounds for AArch64 + more compatibility with non-rewrite mode. Also bail out when --rewrite is passed but no relocs are present'

clang-format

Ping

Use last segment's offset+filesz as an offset for zero-sized stray sections

(once again) use -s flag with fuser

Put stray zero-sized sections in the end of the last segment.
Previously they were assigned old address/offset, which potentially could put them out of address space and cause assertion failures.

Pushed wrong patch last time

• Use fuser -s instead of /dev/null redirection
• Fix wait_file.sh invoked with one extra argument

Add return value checks

Fix LongJMP issues by putting .plt before .text.
Compute text address in advance in non-rewrite case

Fix some test failures, add got/plt handling for RISC-V
I tested RISC-V helloworld on qemu and it did fail after BOLT, but then i tried vanilla BOLT and it couldn't even process the binary, so the changes are probably fine.

Gentle ping

In D147544#4477299, @jobnoorman wrote:

I'm not familiar with LongJmpPass but I did notice the following comment:

We pull this pass inside BOLT because here we can do a better job at stub inserting by manipulating the CFG, something linkers can't do.

So wouldn't you lose something if this was moved to JITLink?

@rafauler sorry for late response, i missed your message. Here is a rebase, and also there is a branch in case the diff fails to apply later. A couple of X86 tests fail because of wrong section indices in symtab, but otherwise it should be ok.

For now i just added fuser requirement to the test.

Okay, this is stupid: it seems debian in CI just lacks fuser command, or it's not in path. I couldn't see the error because of stderr redirection to /dev/null. Here i add more return code checks and we should get an error that fuser is not found. What should we do next?

Changing test to no-pie made no difference. Now i replace wait_file calls with sleeps and if it passes, the script is definitely the issue. Even though i have no idea why.

Change test to no-pie to check if PIE is the issue. If the failure persists, it must be bash script.
Unfortunately i can't reproduce it locally so i have to poke it in random places.

Add -fpie compiler flag
Hope it helps with failures

Quote variables in bash

getBaseAddress -> getTextBaseAddress
Use movabsq to allow big addresses
Add a comment

Currently i cannot reproduce a failure which happens in CI

Hello @Amir, i think it's ok to reland it now, do you agree?

In D154436#4474610, @rafauler wrote:

Thanks for working on this @treapster . Can you add to your diff stack a diff that adds a test that uses this improved mechanism?

Wait for profile in test - depends on D154436

In D147544#4474522, @rafauler wrote:

I think I already asked this but remind me again, why is LongJmp being used for X86? Is this golang support?

Hello, @jobnoorman, out team is currently evaluating possible improvements to LongJmp pass, and we're wondering what are the opportunities for doing relaxation/stub insertion on JITLink side? I see currently JITLink can build PLT with TableManager and relax some calls to it and GOT accesses, do you think it's possible/worth it to to extend this functionality further to support a more general replacement of instructions as we do in LongJMP? It seems it already works like that for RISCV because of D149526, does that mean we can implement similar patches for AArch64/x86 and throw LongJmp away? I guess it would be tricky for X86 because of all the encoding mess, but i'm wondering what your thoughts are.

Add diff context

SEEK_END -> SEEK_SET

Fix values

Since we don't need generic global placement new operator, and defining it in class scope is legal according to cppreference, i decided to move the operator to BumpPtrAllocator scope and define it there.

At first i couldn't reproduce with cmake configured to use clang, but with gcc the runtime library does indeed break. Turns out, the definition for

void *operator new(size_t, void *) noexcept;

Thanks for catching&reverting that, i'll try to reproduce a bit later and report back.

Add assertions to other 2 mmap calls

Turns out, the cause of the issue was getBaseAddress() being in the extern "C" block(it wasn't on the server). Although i still don't get why it had such effect. Also, sorry for all the edits - i just tried to make underscores work in the diff description, but apparently there is no way without backticks.

While this works on our server, i get the following error on my x86_64 laptop:

FAILED: CMakeFiles/bolt_rt_instr_osx.dir/instr.cpp.o 
/usr/bin/clang++  -I/home/denis/randomshit/build/tools/bolt/bolt_rt-bins -O3 -DNDEBUG -std=c++17 -target x86_64-apple-darwin19.6.0 -ffreestanding -fno-exceptions -fno-rtti -fno-stack-protector -mno-sse -fPIC -MD -MT CMakeFiles/bolt_rt_instr_osx.dir/instr.cpp.o -MF CMakeFiles/bolt_rt_instr_osx.dir/instr.cpp.o.d -o CMakeFiles/bolt_rt_instr_osx.dir/instr.cpp.o -c /home/denis/randomshit/llvm-project/bolt/runtime/instr.cpp
In file included from /home/denis/randomshit/llvm-project/bolt/runtime/instr.cpp:44:
/home/denis/randomshit/llvm-project/bolt/runtime/common.h:146:48: error: 32-bit absolute addressing is not supported in 64-bit mode
  __asm__ volatile("leaq __hot_end(%%rip), %0\n\t"
                                               ^
<inline asm>:2:2: note: instantiated into assembly here
        leaq __hot_end, %rcx
        ^

I don't get why it says 32-bit addressing when leaq is 64-bit instruction? Changing it to movq makes no difference, but it definetely has something to do with OSX which i know nothing about.

Fix nits, add mmap check

Add MAP_FAILED

! In D153771#4453963, @rafauler wrote:
If you like, we can commit this diff forcing the test to be no-pie and then work on the fix for PIE on another diff.

Fix test, use no-pie executable.

Add comment and buffer overflow assertion

In D153771#4453993, @rafauler wrote:

In D153771#4452452, @treapster wrote:

Turns out, the [unknown] entries in profile are because addresses in indirect call descriptions are not relocated, which makes them meaningless in PIE because of ASLR. When the test is compiled with no-pie, indirect calls are recorded accurately. So, we should either compute base address and add it to stored entries, or produce dynamic relocations.

Turns out, the [unknown] entries in profile are because addresses in indirect call descriptions are not relocated, which makes them meaningless in PIE because of ASLR. When the test is compiled with no-pie, indirect calls are recorded accurately. So, we should either compute base address and add it to stored entries, or produce dynamic relocations.

In D151920#4450871, @rafauler wrote:

We need a test that checks indirect calls are correctly handled, like the one we have in D153771

Fix nits

For some reason CHECK-ALL-DAG clause does not always work as expected on the second invocation, what may be happening here? I tried using {{}} regex to match the words exactly, but it didn't help

Hello, @rafaelauler, can you please check whether failures you talked about still reproduce with current branch? I understand there's a lot going on with JITLink and RISC-V diffs, but would still appreciate your feedback.

Rebase + handle CALL64m relocations (fixes perlbench failure) + update dynamic relocations for end-of-section symbols.