[ORC] Support ADR_PREL_PG_HI21 & ADD_ABS_LO12_NC for aarch64

[dongAxis1944]

Summary

summary

This patch supports the two relocation types, R_AARCH64_ADR_PREL_PG_HI21 and R_AARCH64_ADD_ABS_LO12_NC.

Test Plan:

check-llvm

[dongAxis1944]

update patch

[dongAxis1944]

llvm/test/ExecutionEngine/JITLink/AArch64/ELF_aarch64_adrp.s

9

Hello, since I do not write raw assembly by hand, this case might be optimized.

1. The addend of this case is 0, I do not know how to set it to the other value;
2. The target and str are on the same page, so "decode_operand(main, 1)" is always zero, should I put the two symbols on different pages?

[dongAxis1944]

fix compile error

[lhames]

Reviewing the patch as-is because I think it's a great addition. If work on the ELF/aarch64 backend is starting in earnest though I would prefer to create a new generic aarch64.h header (along the lines of riscv.h and x86_64.h) and populate it with the existing MachO relocations -- the fixup logic is generally sharable, so there shouldn't be any need to duplicate it, though ELF will probably require some extra relocations beyond the set MachO uses.

If that sounds good to you I'll try to find some time to do that next week.

llvm/lib/ExecutionEngine/JITLink/ELF_aarch64.cpp

46–48

To keep the fixup code idiomatic I would reference these values directly in each of the switch cases that use them, rather than naming them before the start of the switch.

This is particularly true for RawInstr: The fixup content isn't used for every relocation, and not all relocations are fixing up instructions.

84

Should this be RawInstr &= ~Mask?

llvm/test/ExecutionEngine/JITLink/AArch64/ELF_aarch64_adrp.s

9

This doesn't pass on my machine. Is it passing for you locally?

1. I think you could do it by referencing an offset from str? (That's how I'd do it on MachO, but I haven't checked on ELF).

2. You could use a fill directive to insert some filler in a way that would guarantee that str would be on a different page. That's worth testing too (both the "same page" and "different page" cases.

Your testcase doesn't actually need to run, so I'd make main separate and name each instruction to emphasize the test;

# RUN: rm -rf %t && mkdir -p %t
# RUN: llvm-mc -triple=aarch64-unknown-linux-gnu -relax-relocations=false -position-independent -filetype=obj -o %t/elf_aarch64_adrp.o %s
# RUN: llvm-jitlink -noexec -check %s %t/elf_aarch64_adrp.o

        .text

# Add a no-op main as an entry-point.
        .globl    main
        .p2align        2
main:
        ret

# Test R_AARCH64_ADR_PREL_PG_HI21
# jitlink-check: decode_operand(test_page21, 1) = str[32:12] - test_page21[32:12]
        .globl  test_page21
        .p2align        2
        .type test_page21,@function
test_page21:
        adrp    x0, str
        .size   test_page21, .-test_page21

# Test R_AARCH64_ADD_ABS_LO12_NC
# jitlink-check: decode_operand(test_pageoff12, 2) = str[11:0]
        .globl  test_pageoff12
        .p2align        2
        .type test_pageoff12,@function
test_pageoff12:
        add     x0, x0, :lo12:str
        .size   test_pageoff12, .-test_pageoff12

        .globl  str
        .type   str,@object
str:
        .asciz  "hello world"
        .size   str, 12

[dongAxis1944]

In D118346#3281579, @lhames wrote:

Reviewing the patch as-is because I think it's a great addition. If work on the ELF/aarch64 backend is starting in earnest though I would prefer to create a new generic aarch64.h header (along the lines of riscv.h and x86_64.h) and populate it with the existing MachO relocations -- the fixup logic is generally sharable, so there shouldn't be any need to duplicate it, though ELF will probably require some extra relocations beyond the set MachO uses.

If that sounds good to you I'll try to find some time to do that next week.

thanks for reviewing. I actually run the test locally, but I will rerun it again and find out the errors.

[sgraenitz]

Thanks for the patch. I can't dig into the details deeper right now, but if the test issues remain I do so next week.

llvm/lib/ExecutionEngine/JITLink/ELF_aarch64.cpp

107

Unrelated change?

llvm/test/ExecutionEngine/JITLink/AArch64/ELF_aarch64_adrp.s

9

Same here. The current test is failing for me with:

Error evaluating expression 'decode_operand(main, 1) = (((str + 0) & 0xFFFFFFFFFFFFF000) - (main & 0xFFFFFFFFFFFFF000))[32:12]': Couldn't decode instruction at 'main'

I still see the same Couldn't decode instruction error with Lang's proposed test. Not sure what is the issue, but it seems related to the str operand in adrp x0, str. It doesn't show up with an immediate value like #0x1000.

Test Plan:
check-llvm

BTW from your build root you should be able to run the single test this way:

> bin/llvm-lit -vv --filter=ELF_aarch64_adrp test

[dongAxis1944]

In D118346#3282793, @sgraenitz wrote:

Thanks for the patch. I can't dig into the details deeper right now, but if the test issues remain I do so next week.

thanks, it is wired for me. since this test just pass on my mac (T.T). I will recheck it. thanks in advance.

[sunho]

@dongAxis1944 Can you still work on this? I have a fix in my local. Only few bitmask arithematic errors.

[sgraenitz]

@sunho implemented it in D126287. I think this review should be closed.

llvm/include/llvm/ExecutionEngine/JITLink/aarch64.h

 namespace aarch64 {
 
 /// Represets aarch64 fixups
 enum EdgeKind_aarch64 : Edge::Kind {
 
   /// Set a CALL immediate field to bits [27:2] of X = Target - Fixup + Addend
   R_AARCH64_CALL26 = Edge::FirstRelocation,
 
+  /// Set an ADRP immediate value to bits [32:12] of the X
+  ///
+  /// Fixup experssion
+  ///   Fixup <- Page(Target + Addend) - Page(Fixup)
+  ///
+  /// Overflow
+  ///   -2^32 <= X <= 2^32
+  ///
+  /// Attentions:
+  /// According to the arm/aarch64 document:
+  /// Page(expr) is the page address of the expression expr, defined as (expr &
+  /// ~0xFFF). (This applies even if the machine page size supported by the
+  /// platform has a different value.)
+  ///
+  R_AARCH64_ADR_PREL_PG_HI21,
+
+  /// Set an ADD immediate value to bits [11:0] of X. No overflow check. Used
+  /// with relocations ADR_PREL_PG_HI21 and ADR_PREL_PG_HI21_NC
+  /// Fixup expression:
+  ///   Fixup <- Target + Addend
+  ///
+  R_AARCH64_ADD_ABS_LO12_NC,
 };
 
 /// Returns a string name for the given aarch64 edge. For debugging purposes
 /// only
 const char *getEdgeKindName(Edge::Kind K);
 
 } // namespace aarch64
 } // namespace jitlink
 } // namespace llvm
 
 #endif // LLVM_EXECUTIONENGINE_JITLINK_AARCH64_H

llvm/include/llvm/ExecutionEngine/Orc/Shared/ExecutorAddress.h

ExecutorAddr &operator+=(const ExecutorAddrDiff &Delta) {
     return *this;
   }
 
   ExecutorAddr &operator-=(const ExecutorAddrDiff &Delta) {
     Addr -= Delta;
     return *this;
   }
 
+  ExecutorAddr &operator&(const ExecutorAddrDiff Mask) {
+    Addr &= Mask;
+    return *this;
+  }
+
 private:
   uint64_t Addr = 0;
 };
 
 /// Subtracting two addresses yields an offset.
 inline ExecutorAddrDiff operator-(const ExecutorAddr &LHS,
                                   const ExecutorAddr &RHS) {
   return ExecutorAddrDiff(LHS.getValue() - RHS.getValue());

llvm/lib/ExecutionEngine/JITLink/ELF_aarch64.cpp

 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/ExecutionEngine/JITLink/ELF_aarch64.h"
 #include "ELFLinkGraphBuilder.h"
 #include "JITLinkGeneric.h"
 #include "llvm/BinaryFormat/ELF.h"
 #include "llvm/ExecutionEngine/JITLink/aarch64.h"
+#include "llvm/ExecutionEngine/Orc/Shared/ExecutorAddress.h"
 #include "llvm/Object/ELFObjectFile.h"
 #include "llvm/Support/MathExtras.h"
 
 #define DEBUG_TYPE "jitlink"
 
 using namespace llvm;
 using namespace llvm::jitlink;
 

 
 private:
   Error applyFixup(LinkGraph &G, Block &B, const Edge &E) const {
     using namespace aarch64;
     using namespace llvm::support;
 
     char *BlockWorkingMem = B.getAlreadyMutableContent().data();
     char *FixupPtr = BlockWorkingMem + E.getOffset();
+    uint32_t RawInstr = *(little32_t *)FixupPtr;
+    auto Addend = E.getAddend();
+    auto SymbolAddress = E.getTarget().getAddress();
     auto FixupAddress = B.getAddress() + E.getOffset();
     switch (E.getKind()) {
-    case aarch64::R_AARCH64_CALL26: {
+    case R_AARCH64_CALL26: {
       assert((FixupAddress.getValue() & 0x3) == 0 &&
              "Call-inst is not 32-bit aligned");
       int64_t Value = E.getTarget().getAddress() - FixupAddress + E.getAddend();
 
       if (static_cast<uint64_t>(Value) & 0x3)
         return make_error<JITLinkError>("Call target is not 32-bit aligned");
 
       if (!isInt<28>(Value))
         return makeTargetOutOfRangeError(G, B, E);
 
-      uint32_t RawInstr = *(little32_t *)FixupPtr;
       assert((RawInstr & 0x7fffffff) == 0x14000000 &&
              "RawInstr isn't a B or BR immediate instruction");
       uint32_t Imm = (static_cast<uint32_t>(Value) & ((1 << 28) - 1)) >> 2;
       uint32_t FixedInstr = RawInstr | Imm;
       *(little32_t *)FixupPtr = FixedInstr;
       break;
     }
+    case R_AARCH64_ADR_PREL_PG_HI21: {
+      auto PagedSymbolAddr = (SymbolAddress + Addend) & ~0xFFFLL;
+      auto PagedFixupAddr = FixupAddress & ~0xFFFLL;
+      int64_t Value = PagedSymbolAddr - PagedFixupAddr;
+      if (!isInt<33>(Value))
+        return makeTargetOutOfRangeError(G, B, E);
+
+      // Set value to the imm field of adrp instruction
+      // adrp format:
+      // C6.2.9 in ARM Architecture Reference Manual - ARMv8, for ARMv8-A
+      // architecture profile
+      uint32_t Imm32_12 = (static_cast<uint32_t>(Value) >> 11) & 0x3FFFFF;
+      uint32_t ImmLo = (Imm32_12 & 0x3) << 29;
+      uint32_t ImmHi = ((Imm32_12 & 0x1FFFFC) >> 2) << 5;
+      uint32_t Mask = ((0x1FFFFC >> 2) << 5) | (0x3U << 29);
+      RawInstr |= ~Mask;
+      *(little32_t *)FixupPtr = RawInstr | ImmLo | ImmHi;
+      break;
+    }
+    case R_AARCH64_ADD_ABS_LO12_NC: {
+      // set value to the imm field of add* instruction
+      // add format:
+      // C6.2.4 in ARM Architecture Reference Manual - ARMv8, for ARMv8-A
+      // architecture profile
+      int64_t Value = (SymbolAddress + Addend).getValue();
+      Value = (Value & 0x7FFLL) << 10; // instr{21-10} is imm field
+      uint32_t Imm11_0 = static_cast<uint32_t>(Value);
+      *(little32_t *)FixupPtr = RawInstr | Imm11_0;
+      break;
+    }
     }
     return Error::success();
   }
 };
 
 template <typename ELFT>
 class ELFLinkGraphBuilder_aarch64 : public ELFLinkGraphBuilder<ELFT> {
 private:
-  static Expected<aarch64::EdgeKind_aarch64>
-  getRelocationKind(const uint32_t Type) {
+  Expected<aarch64::EdgeKind_aarch64> getRelocationKind(const uint32_t Type) {
     using namespace aarch64;
     switch (Type) {
     case ELF::R_AARCH64_CALL26:
       return EdgeKind_aarch64::R_AARCH64_CALL26;
+    case ELF::R_AARCH64_ADR_PREL_PG_HI21:
+      return EdgeKind_aarch64::R_AARCH64_ADR_PREL_PG_HI21;
+    case ELF::R_AARCH64_ADD_ABS_LO12_NC:
+      return EdgeKind_aarch64::R_AARCH64_ADD_ABS_LO12_NC;
     }
-
     return make_error<JITLinkError>("Unsupported aarch64 relocation:" +
                                     formatv("{0:d}", Type));
   }
 
   Error addRelocations() override {
     LLVM_DEBUG(dbgs() << "Processing relocations:\n");
 
     using Base = ELFLinkGraphBuilder<ELFT>;

llvm/lib/ExecutionEngine/JITLink/aarch64.cpp

 namespace llvm {
 namespace jitlink {
 namespace aarch64 {
 
 const char *getEdgeKindName(Edge::Kind K) {
   switch (K) {
   case R_AARCH64_CALL26:
     return "R_AARCH64_CALL26";
+  case R_AARCH64_ADR_PREL_PG_HI21:
+    return "R_AARCH64_ADR_PREL_PG_HI21";
+  case R_AARCH64_ADD_ABS_LO12_NC:
+    return "R_AARCH64_ADD_ABS_LO12_NC";
   }
   return getGenericEdgeKindName(K);
 }
 } // namespace aarch64
 } // namespace jitlink
 } // namespace llvm

llvm/test/ExecutionEngine/JITLink/AArch64/ELF_aarch64_adrp.s

+# RUN: rm -rf %t && mkdir -p %t
+# RUN: llvm-mc -triple=aarch64-unknown-linux-gnu -relax-relocations=false -position-independent -filetype=obj -o %t/elf_aarch64_adrp.o %s
+# RUN: llvm-jitlink -noexec -check %s %t/elf_aarch64_adrp.o
+
+        .text
+
+# Test Case 1: R_AARCH64_ADR_PREL_PG_HI21
+# jitlink-check: decode_operand(main, 1) = (((str + 0) & 0xFFFFFFFFFFFFF000) - (main & 0xFFFFFFFFFFFFF000))[32:12]
+#
+# Test Case 2: R_AARCH64_ADD_ABS_LO12_NC
+# jitlink-check: decode_operand(main + 4, 2) = (str + 0)[11:0]
+
+
+        .globl    main
+  .p2align  2
+        .type main,@function
+main:
+  adrp  x0, str
+  add x0, x0, :lo12:str
+  ret
+  .size main, .-main
+
+        .globl  str
+  .type str,@object
+str:
+  .asciz  "hello world"
+  .size str, 12