ExecutionEngine: add preliminary support for COFF ARM64

[kaadam]

Hi Everyone,

Currently this patch is WIP, I need to do some clean up, and add a unittest to it. I managed to test it with some example, it seems it works. I will update it early next week.

[mstorsjo]

Looks sensible overall, except for two relocations that have incorrect interpretations. You seem to be missing a couple relocation types as well; check lld/COFF/Chunks.cpp for reference on how to handle the rest of them. Some of them are not very probable to use in JITed code (as TLS code requires space for the variables to be allocated in the TLS section by the system's runtime loader; the SECREL_LOW/HIGH_12A/L relocations are used for that), but most of the remaining ones are pretty straightforward to handle anyway.

lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFAArch64.h

170

No, this is wrong - and this is one of the less obvious details.

If you have an addend stored in the instruction pointed to by IMAGE_REL_ARM64_PAGEBASE_REL21, the addend is expressed in bytes, not in 4096 byte pages.

Consider you have a symbol close to the end of a page, and you want to express an offset by a few bytes (less than a page), that makes the pointed to location in another page.

If the addend would express a number of pages (as this patch expects right now), the addend here would be zero, and you'd end up with this part of the instruction pair pointing at the wrong page.

Therefore, the immediate stored in the instruction before handling relocation is expressed as a number of bytes, even though it means a number of pages after the relocation is done and the instruction is executed.

176

Nit: The indentation of the first comment line is off here

181

Nit: Double spaces between "store" and "or"

189

I guess this check could also be for the relocation type IMAGE_REL_ARM64_PAGEOFFSET_12L?

211

Would it make more sense, stylistically, to extend the ifdef around the debug statement as well? Right now it does look weird to have code referring to variables that don't exist (even though LLVM_DEBUG will make them disappear).

221

The indentation here is weird. Please run clang-format-diff -style LLVM on the changes.

244

This doesn't seem right. IMAGE_REL_ARM64_ADDR64 is a plain 64 bit integer (just like IMAGE_REL_ARM64_ADDR32NB), not a series of instructions that should get immediates added.

288

This switch for checking for alignment is rather verbose - would it make sense to condense it down to a single expression for all alignments?

365

Missing newline at end of file

[kaadam]

Martin, thanks for your thoughts. I'm on to fix them.

lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFAArch64.h

170

Thanks for explaining this. It seems I misunderstood this behavior. To be honest I looked at how MachOAArch64 does, how they decode the addend, I compared to aarch64 reference manual to see how it works, it seemed to me I can do the same thing as MachO does.

189

It is just for IMAGE_REL_ARM64_PAGEOFFSET_12L, we need to determine the shift value only for load/store, since for instructions ADD/ADDS (immediate) we should use zero shift.

211

Yes, you're right, I will fix it.

221

Thanks, I will use it.

244

Yes, you're right. Currently I handle a long branch instruction with this relocation, so when I needed to create stub function (which generates movz/movk instruction), I just set this relocation to detect this case when we have an external symbol. When I tested it with a small examples and Swithshader It worked because I haven't got this relocation type. Of course I should use another one, maybe an internal type.
What do you think?

288

Sure, it could be.

[mstorsjo]

lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFAArch64.h

170

I guess it differs a bit between how the different object file formats encode the relocations and symbol offsets. (IIRC ELF and MachO can say that a relocation is relative to a symbol with offset, while COFF only points at a symbol, and any offset must be applied via the instruction immediates.)

189

Yes, but the relocation names already imply this. The L suffixed relocation is used for loads/stores, and the A suffixed relocation is used for add instructions. For IMAGE_REL_ARM64_PAGEOFFSET_12L we do not need to check whether the instruction is a load/store, but we can go directly to reading out the implicit shift amount, and for IMAGE_REL_ARM64_PAGEOFFSET_12A we should not read any implicit shift amount at all.

See lld/COFF/Chunks.cpp, SectionChunk::applyRelARM64. For IMAGE_REL_ARM64_PAGEOFFSET_12L we call applyArm64Ldr which reads out the shift amount and then calls applyArm64Imm, while applyArm64Imm is called directly for IMAGE_REL_ARM64_PAGEOFFSET_12A.

In general, when a linker (either dynamic or static) resolves a relocation, it should seldom need to inspect the instruction it is applied on, even though it is needed here for reading out the implicit shift amount. In general, the relocation type just encodes a specific action that should be done on that memory location with very little extra logic.

244

Hmm, as I'm not familiar with those bits that generate it, I don't know for sure. As you're already setting a small code model for aarch64/win, doesn't that already achieve this?

Otherwise some variant of adrp+add would normally be used for forming any arbitrary address, instead of a series of mov instructions, unless the value to be formed is a constant.

If necessary I guess one could consider using private internal relocation types, but I'd at least defer those bits to a later patch where it can be discussed properly on its own, and keep this first for the official relocation types.

[kaadam]

Martin, sorry for the delayed response.

lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFAArch64.h

170

Okay, I see now.
By the way I don't necessary need to decode addend in processRelocationRef function, it could be encoded before the relocation applied, am I right? Which is better? Since the inst contains the offset in the immediate part, so it have to be considered (add the offset to 'Value') right before immediate is rewritten.
I update the change later today.

[mstorsjo]

lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFAArch64.h

170

I presume you meant "it could be decoded before the relocation applied", not encoded?

Yes, you could do the decode+update+encode all in one step. When reviewing I noted that this followed such a two-step style, but I presumed this came from general RuntimeDyld design. In lld it's all done in one single function. I'm not familiar with RuntimeDyld to say if there's any specific needs here for it to be this way, but if other parts of RuntimeDyld does it this way (in particular, other COFF architectures) it might be good to match the style.

[kaadam]

lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFAArch64.h

170

Yes, that's what I meant. Thanks for your suggestion. Yes, I follow that style, since RuntimeDyld design follows a two-step style, but as far as I see there is no any style convention, for example in COFF Thumb a few instruction's addend are decoded, most of them are not. I don't see any significance to be this way, but I would like to do a clean approach for this.

[mstorsjo]

Looking pretty good now, just a few details I found. Then this needs tests to get rid of the WIP status.

lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFAArch64.h

47

Leftover commented out code?

58

Is there any better error reporting mechanism available here?

The current form of assert is wrong, as the address of the string will evaluate as nonzero and the assert wouldn't trigger. Maybe assert(0 && "message") in case there's no better way of actually reporting the issue to the caller.

118

I can't say I entirely understand this function (I understand what it tries to do, but fail to follow exactly how the details fit together), but as I see that it is very similar to the corresponding existing code for x86_64, I presume it's correct.

172

Nit: Here and below you have pretty superfluous outer parentheses

181

This doesn't seem to be right?

For instructions B.cond and e.g. cbz, you have 5 least significant bits of the instruction being other data than the immediate, and after that, the resulting value should be left shifted by 2.

So here, it should be Addend = ((orig & 0x00FFFFE0) >> 5) << 2; (for clarity) or Addend = (orig & 0x00FFFFE0) >> 3; (for more straightforward but less obvious code).

Something similar should be done for branch14 below as well.

LLD currently actually ignores the existing immediate in these relocations (which hasn't been an issue so far, but technically is an oversight).

206

Do we need to support reading out the immediate from INTERNAL_REL_ARM64_LONG_BRANCH26 here? Or as the only place that generates it writes a zero immediate I guess it's not necessary?

303

If there actually was a nonzero immediate in the instruction here from before, or32le won't do the right thing. We don't handle this in lld right now, but as this code at least tries to read out the immediate further up, it would be good for consistency to actually clear the immediate from the branch instruction here before or'ing in the final value.

339

Hmm, where does Value end up added to this one?

I do see that the existing COFF targets does it the same way, and that code does seem to be used and have a working test, but I don't see how it works. Do you have any clue?

[kaadam]

Thanks for the lots of comment, it was really helpful. I update the patch, I enclose a unit_test for this. There are tree test failures (adr, b.cond, secrel.), I'm still looking for the reasons.

Expression 'decode_operand(adr1, 1) = (_const[20:0] - adr1[20:0])' is false: 0x2014 != 0x10014
Expression 'decode_operand(bcond, 1)[23:5] = (_foo - bcond)[20:2]' is false: 0x7ffff != 0x7fff8
Expression '*{4}secrel = _foo - section_addr(COFF_AArch64.obj, .text)' is false: 0xfffeffea != 0x4

lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFAArch64.h

118

Yes, the logic is the same as other COFF targets do. Basically we should handle an external symbol which is so far.

So first time when we detected we have an external symbol, generate a stub function (movz/movk [1]). Currently the Value contains the address of the external symbol for the original relocation branch26 (b/bl instruction). We need to replace the Value here to point to the address of the stub instead of the external symbol. That's why we need to call resolveRelocation for the original relocation, but here we will pass the stub address as Value to it. So when original branch26 relocation is resolved that will point to stub. After that we returned with our internal relocation type and stuboffset, create RelocationEntry for these data and we can use the original Value for external symbol. Finally we will resolve the relocation for this internal type, the symbol address will be encode the into movz/movk stub instructions.

all in all we're going to have two jump, bl -> stub -> external symbol.

[1]: https://github.com/llvm-mirror/llvm/blob/master/lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp#L933

206

Yes, it is not necessary since the Addend is always zero this internal type.

303

Yes, you're right . maybe the immediate for these instruction could be cleared when the addends are decoded.

339

Unfortunately I'm not sure what happens under the hood, but it seems it is correct to use only the addend here, which contains the offset of the item from the beginning of its section in this case.

[kaadam]

change is updated, unit_test is attached.

[mstorsjo]

lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFAArch64.h

118

Yes, that much I understand (I implemented the same feature in the lld linker as well) - it was more relating to the exact details of each of the method calls here, but I don't have any concrete question regarding it right now either.

194

You can't do the masking out of the original immediate here; updating orig has no effect at all, as that's a local variable.

Handling masking out the old value within processRelocationRef feels wrong in general (as this function only inspects and gathers info but doesn't update anything yet), I think this should be in resolveRelocation.

So then you can't use or32le there in those cases, but more something like write32le(P, (read32le(P) & ~Mask) | V); (which perhaps can warrant a helper function of its own).

[kaadam]

updated the write method in branch relocations, the unit_test is also updated.

[mstorsjo]

LGTM

Do you need someone to commit this for you? Do you have a preferred form for the git author line in that case ("User Name <email@address>"); with or without non-ascii diacritics?

[kaadam]

Martin, thanks for the review. Yes, I need. May I ask you to commit this change? I use "Adam Kallai <[e-mail]> in the git author line.

[mstorsjo]

test/ExecutionEngine/RuntimeDyld/AArch64/COFF_AArch64.s

128

FWIW, I had to fix up the filename here, to COFF_AArch64.s.tmp.obj to make the tests pass. Pushed with that changed.

[kaadam]

test/ExecutionEngine/RuntimeDyld/AArch64/COFF_AArch64.s

128

That's interesting. I ran tests manually, so I think that's why I didn't noticed this failure. So LLVM test infrastructure use this form of the object file? Thanks for fixing that.
There was no problem with check in line 39?

1. rtdyld-check: decode_operand(brel, 0)[25:0] = (stub_addr(COFF_AArch64.obj/.text, dummy) - brel)[27:2]

[kaadam]

test/ExecutionEngine/RuntimeDyld/AArch64/COFF_AArch64.s

128

Ahh sorry, I see now, you've modified that as well. Thanks again.

[mstorsjo]

test/ExecutionEngine/RuntimeDyld/AArch64/COFF_AArch64.s

128

That's how the test infrastructure expands it yes. The %t expands to a temporary file name based on the current file name, plus .tmp.

You can manually run the tests for just one file, by running bin/llvm-lit -v path/to/test/ExecutionEngine/Foo/file.s, for a subdirectory tree with the same command pointing to a directory, or all of them (with ninja check-llvm).

[kaadam]

My CL is broke the buildbot, a break is necessary in default case. How could I fix it? Shall I need to update this change?

[mstorsjo]

In D69434#1753145, @kaadam wrote:

My CL is broke the buildbot, a break is necessary in default case. How could I fix it? Shall I need to update this change?

I pushed a new commit to fix this now.

You could post a new patch for fixing it, but in this case the fix was trivial enough to just do myself. If fixing takes longer, it might be good to revert the commit in the meantime, and then it might be better to update the full patch so that it can be reapplied in fixed form.

[kaadam]

In D69434#1753153, @mstorsjo wrote:

In D69434#1753145, @kaadam wrote:

My CL is broke the buildbot, a break is necessary in default case. How could I fix it? Shall I need to update this change?

I pushed a new commit to fix this now.

You could post a new patch for fixing it, but in this case the fix was trivial enough to just do myself. If fixing takes longer, it might be good to revert the commit in the meantime, and then it might be better to update the full patch so that it can be reapplied in fixed form.

Okay, I see. Thanks Martin.

[lhames]

Coming to this a bit late -- Thanks for working on this kaadam! And thanks for the review mstorsjo.

Does COFF/AArch64 use a GOT? I couldn't see handling for that here?

lib/ExecutionEngine/RuntimeDyld/RuntimeDyldCOFF.cpp

 //===-- RuntimeDyldCOFF.cpp - Run-time dynamic linker for MC-JIT -*- C++ -*-==//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // Implementation of COFF support for the MC-JIT runtime dynamic linker.
 //
 //===----------------------------------------------------------------------===//
 
 #include "RuntimeDyldCOFF.h"
+#include "Targets/RuntimeDyldCOFFAArch64.h"
 #include "Targets/RuntimeDyldCOFFI386.h"
 #include "Targets/RuntimeDyldCOFFThumb.h"
 #include "Targets/RuntimeDyldCOFFX86_64.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/Triple.h"
 #include "llvm/Object/ObjectFile.h"
 
 using namespace llvm;
llvm::RuntimeDyldCOFF::create(Triple::ArchType Arch,
   switch (Arch) {
   default: llvm_unreachable("Unsupported target for RuntimeDyldCOFF.");
   case Triple::x86:
     return std::make_unique<RuntimeDyldCOFFI386>(MemMgr, Resolver);
   case Triple::thumb:
     return std::make_unique<RuntimeDyldCOFFThumb>(MemMgr, Resolver);
   case Triple::x86_64:
     return std::make_unique<RuntimeDyldCOFFX86_64>(MemMgr, Resolver);
+  case Triple::aarch64:
+    return std::make_unique<RuntimeDyldCOFFAArch64>(MemMgr, Resolver);
   }
 }
 
 std::unique_ptr<RuntimeDyld::LoadedObjectInfo>
 RuntimeDyldCOFF::loadObject(const object::ObjectFile &O) {
   if (auto ObjSectionToIDOrErr = loadObjectImpl(O)) {
     return std::make_unique<LoadedCOFFObjectInfo>(*this, *ObjSectionToIDOrErr);
   } else {

lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFAArch64.h

+//===-- RuntimeDyldCOFFAArch64.h --- COFF/AArch64 specific code ---*- C++
+//-*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// COFF AArch64 support for MC-JIT runtime dynamic linker.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIB_EXECUTIONENGINE_RUNTIMEDYLD_TARGETS_RUNTIMEDYLDCOFFAARCH64_H
+#define LLVM_LIB_EXECUTIONENGINE_RUNTIMEDYLD_TARGETS_RUNTIMEDYLDCOFFAARCH64_H
+
+#include "../RuntimeDyldCOFF.h"
+#include "llvm/BinaryFormat/COFF.h"
+#include "llvm/Object/COFF.h"
+#include "llvm/Support/Endian.h"
+
+#define DEBUG_TYPE "dyld"
+
+using namespace llvm::support::endian;
+
+namespace llvm {
+
+// This relocation type is used for handling long branch instruction
+// throught the Stub.
+enum InternalRelocationType : unsigned {
+  INTERNAL_REL_ARM64_LONG_BRANCH26 = 0x111,
+};
+
+static void add16(uint8_t *p, int16_t v) { write16le(p, read16le(p) + v); }
+static void or32le(void *P, int32_t V) { write32le(P, read32le(P) | V); }
+
+static void write32AArch64Imm(uint8_t *T, uint64_t imm, uint32_t rangeLimit) {
+  uint32_t orig = read32le(T);
+  orig &= ~(0xFFF << 10);
+  write32le(T, orig | ((imm & (0xFFF >> rangeLimit)) << 10));
+}
+
+static void write32AArch64Ldr(uint8_t *T, uint64_t imm) {
+  uint32_t orig = read32le(T);
+  uint32_t size = orig >> 30;
+  // 0x04000000 indicates SIMD/FP registers
+  // 0x00800000 indicates 128 bit
+  if ((orig & 0x04800000) == 0x04800000)
+    size += 4;
+  if ((imm & ((1 << size) - 1)) != 0)
+    assert(0 && "misaligned ldr/str offset");
+  write32AArch64Imm(T, imm >> size, size);
+}
+
+static void write32AArch64Addr(void *T, uint64_t s, uint64_t p, int shift) {
+  uint64_t Imm = (s >> shift) - (p >> shift);
+  uint32_t ImmLo = (Imm & 0x3) << 29;
+  uint32_t ImmHi = (Imm & 0x1FFFFC) << 3;
+  uint64_t Mask = (0x3 << 29) | (0x1FFFFC << 3);
+  write32le(T, (read32le(T) & ~Mask) | ImmLo | ImmHi);
+}
+
+class RuntimeDyldCOFFAArch64 : public RuntimeDyldCOFF {
+
+private:
+  // When a module is loaded we save the SectionID of the unwind
+  // sections in a table until we receive a request to register all
+  // unregisteredEH frame sections with the memory manager.
+  SmallVector<SID, 2> UnregisteredEHFrameSections;
+  SmallVector<SID, 2> RegisteredEHFrameSections;
+  uint64_t ImageBase;
+
+  // Fake an __ImageBase pointer by returning the section with the lowest adress
+  uint64_t getImageBase() {
+    if (!ImageBase) {
+      ImageBase = std::numeric_limits<uint64_t>::max();
+      for (const SectionEntry &Section : Sections)
+        // The Sections list may contain sections that weren't loaded for
+        // whatever reason: they may be debug sections, and ProcessAllSections
+        // is false, or they may be sections that contain 0 bytes. If the
+        // section isn't loaded, the load address will be 0, and it should not
+        // be included in the ImageBase calculation.
+        if (Section.getLoadAddress() != 0)
+          ImageBase = std::min(ImageBase, Section.getLoadAddress());
+    }
+    return ImageBase;
+  }
+
+public:
+  RuntimeDyldCOFFAArch64(RuntimeDyld::MemoryManager &MM,
+                         JITSymbolResolver &Resolver)
+      : RuntimeDyldCOFF(MM, Resolver), ImageBase(0) {}
+
+  unsigned getStubAlignment() override { return 8; }
+
+  unsigned getMaxStubSize() const override { return 20; }
+
+  std::tuple<uint64_t, uint64_t, uint64_t>
+  generateRelocationStub(unsigned SectionID, StringRef TargetName,
+                         uint64_t Offset, uint64_t RelType, uint64_t Addend,
+                         StubMap &Stubs) {
+    uintptr_t StubOffset;
+    SectionEntry &Section = Sections[SectionID];
+
+    RelocationValueRef OriginalRelValueRef;
+    OriginalRelValueRef.SectionID = SectionID;
+    OriginalRelValueRef.Offset = Offset;
+    OriginalRelValueRef.Addend = Addend;
+    OriginalRelValueRef.SymbolName = TargetName.data();
+
+    auto Stub = Stubs.find(OriginalRelValueRef);
+    if (Stub == Stubs.end()) {
+      LLVM_DEBUG(dbgs() << " Create a new stub function for "
+                        << TargetName.data() << "\n");
+
+      StubOffset = Section.getStubOffset();
+      Stubs[OriginalRelValueRef] = StubOffset;
+      createStubFunction(Section.getAddressWithOffset(StubOffset));
+      Section.advanceStubOffset(getMaxStubSize());
+    } else {
+      LLVM_DEBUG(dbgs() << " Stub function found for " << TargetName.data()
+                        << "\n");
+      StubOffset = Stub->second;
+    }
+
+    // Resolve original relocation to stub function.
+    const RelocationEntry RE(SectionID, Offset, RelType, Addend);
+    resolveRelocation(RE, Section.getLoadAddressWithOffset(StubOffset));
+
+    // adjust relocation info so resolution writes to the stub function
+    // Here an internal relocation type is used for resolving long branch via
+    // stub instruction.
+    Addend = 0;
+    Offset = StubOffset;
+    RelType = INTERNAL_REL_ARM64_LONG_BRANCH26;
+
+    return std::make_tuple(Offset, RelType, Addend);
+  }
+
+  Expected<object::relocation_iterator>
+  processRelocationRef(unsigned SectionID, object::relocation_iterator RelI,
+                       const object::ObjectFile &Obj,
+                       ObjSectionToIDMap &ObjSectionToID,
+                       StubMap &Stubs) override {
+
+    auto Symbol = RelI->getSymbol();
+    if (Symbol == Obj.symbol_end())
+      report_fatal_error("Unknown symbol in relocation");
+
+    Expected<StringRef> TargetNameOrErr = Symbol->getName();
+    if (!TargetNameOrErr)
+      return TargetNameOrErr.takeError();
+    StringRef TargetName = *TargetNameOrErr;
+
+    auto SectionOrErr = Symbol->getSection();
+    if (!SectionOrErr)
+      return SectionOrErr.takeError();
+    auto Section = *SectionOrErr;
+
+    uint64_t RelType = RelI->getType();
+    uint64_t Offset = RelI->getOffset();
+
+    // If there is no section, this must be an external reference.
+    const bool IsExtern = Section == Obj.section_end();
+
+    // Determine the Addend used to adjust the relocation value.
+    uint64_t Addend = 0;
+    SectionEntry &AddendSection = Sections[SectionID];
+    uintptr_t ObjTarget = AddendSection.getObjAddress() + Offset;
+    uint8_t *Displacement = (uint8_t *)ObjTarget;
+
+    switch (RelType) {
+    case COFF::IMAGE_REL_ARM64_ADDR32:
+    case COFF::IMAGE_REL_ARM64_ADDR32NB:
+    case COFF::IMAGE_REL_ARM64_REL32:
+    case COFF::IMAGE_REL_ARM64_SECREL:
+      Addend = read32le(Displacement);
+      break;
+    case COFF::IMAGE_REL_ARM64_BRANCH26: {
+      uint32_t orig = read32le(Displacement);
+      Addend = (orig & 0x03FFFFFF) << 2;
+
+      if (IsExtern)
+        std::tie(Offset, RelType, Addend) = generateRelocationStub(
+            SectionID, TargetName, Offset, RelType, Addend, Stubs);
+      break;
+    }
+    case COFF::IMAGE_REL_ARM64_BRANCH19: {
+      uint32_t orig = read32le(Displacement);
+      Addend = (orig & 0x00FFFFE0) >> 3;
+      break;
+    }
+    case COFF::IMAGE_REL_ARM64_BRANCH14: {
+      uint32_t orig = read32le(Displacement);
+      Addend = (orig & 0x000FFFE0) >> 3;
+      break;
+    }
+    case COFF::IMAGE_REL_ARM64_REL21:
+    case COFF::IMAGE_REL_ARM64_PAGEBASE_REL21: {
+      uint32_t orig = read32le(Displacement);
+      Addend = ((orig >> 29) & 0x3) | ((orig >> 3) & 0x1FFFFC);
+      break;
+    }
+    case COFF::IMAGE_REL_ARM64_PAGEOFFSET_12L:
+    case COFF::IMAGE_REL_ARM64_PAGEOFFSET_12A: {
+      uint32_t orig = read32le(Displacement);
+      Addend = ((orig >> 10) & 0xFFF);
+      break;
+    }
+    case COFF::IMAGE_REL_ARM64_ADDR64: {
+      Addend = read64le(Displacement);
+    }
+    default:
+      break;
+    }
+
+#if !defined(NDEBUG)
+    SmallString<32> RelTypeName;
+    RelI->getTypeName(RelTypeName);
+
+    LLVM_DEBUG(dbgs() << "\t\tIn Section " << SectionID << " Offset " << Offset
+                      << " RelType: " << RelTypeName << " TargetName: "
+                      << TargetName << " Addend " << Addend << "\n");
+#endif
+
+    unsigned TargetSectionID = -1;
+    if (IsExtern) {
+      RelocationEntry RE(SectionID, Offset, RelType, Addend);
+      addRelocationForSymbol(RE, TargetName);
+    } else {
+      if (auto TargetSectionIDOrErr = findOrEmitSection(
+              Obj, *Section, Section->isText(), ObjSectionToID)) {
+        TargetSectionID = *TargetSectionIDOrErr;
+      } else
+        return TargetSectionIDOrErr.takeError();
+
+      uint64_t TargetOffset = getSymbolOffset(*Symbol);
+      RelocationEntry RE(SectionID, Offset, RelType, TargetOffset + Addend);
+      addRelocationForSection(RE, TargetSectionID);
+    }
+    return ++RelI;
+  }
+
+  void resolveRelocation(const RelocationEntry &RE, uint64_t Value) override {
+    const auto Section = Sections[RE.SectionID];
+    uint8_t *Target = Section.getAddressWithOffset(RE.Offset);
+    uint64_t FinalAddress = Section.getLoadAddressWithOffset(RE.Offset);
+
+    switch (RE.RelType) {
+    default:
+      llvm_unreachable("unsupported relocation type");
+    case COFF::IMAGE_REL_ARM64_ABSOLUTE: {
+      // This relocation is ignored.
+      break;
+    }
+    case COFF::IMAGE_REL_ARM64_PAGEBASE_REL21: {
+      // The page base of the target, for ADRP instruction.
+      Value += RE.Addend;
+      write32AArch64Addr(Target, Value, FinalAddress, 12);
+      break;
+    }
+    case COFF::IMAGE_REL_ARM64_REL21: {
+      // The 12-bit relative displacement to the target, for instruction ADR
+      Value += RE.Addend;
+      write32AArch64Addr(Target, Value, FinalAddress, 0);
+      break;
+    }
+    case COFF::IMAGE_REL_ARM64_PAGEOFFSET_12A: {
+      // The 12-bit page offset of the target,
+      // for instructions ADD/ADDS (immediate) with zero shift.
+      Value += RE.Addend;
+      write32AArch64Imm(Target, Value & 0xFFF, 0);
+      break;
+    }
+    case COFF::IMAGE_REL_ARM64_PAGEOFFSET_12L: {
+      // The 12-bit page offset of the target,
+      // for instruction LDR (indexed, unsigned immediate).
+      Value += RE.Addend;
+      write32AArch64Ldr(Target, Value & 0xFFF);
+      break;
+    }
+    case COFF::IMAGE_REL_ARM64_ADDR32: {
+      // The 32-bit VA of the target.
+      uint32_t VA = Value + RE.Addend;
+      write32le(Target, VA);
+      break;
+    }
+    case COFF::IMAGE_REL_ARM64_ADDR32NB: {
+      // The target's 32-bit RVA.
+      uint64_t RVA = Value + RE.Addend - getImageBase();
+      write32le(Target, RVA);
+      break;
+    }
+    case INTERNAL_REL_ARM64_LONG_BRANCH26: {
+      // Encode the immadiate value for generated Stub instruction (MOVZ)
+      or32le(Target + 12, ((Value + RE.Addend) & 0xFFFF) << 5);
+      or32le(Target + 8, ((Value + RE.Addend) & 0xFFFF0000) >> 11);
+      or32le(Target + 4, ((Value + RE.Addend) & 0xFFFF00000000) >> 27);
+      or32le(Target + 0, ((Value + RE.Addend) & 0xFFFF000000000000) >> 43);
+      break;
+    }
+    case COFF::IMAGE_REL_ARM64_BRANCH26: {
+      // The 26-bit relative displacement to the target, for B and BL
+      // instructions.
+      uint64_t PCRelVal = Value + RE.Addend - FinalAddress;
+      assert(isInt<28>(PCRelVal) && "Branch target is out of range.");
+      write32le(Target, (read32le(Target) & ~(0x03FFFFFF)) |
+                            (PCRelVal & 0x0FFFFFFC) >> 2);
+      break;
+    }
+    case COFF::IMAGE_REL_ARM64_BRANCH19: {
+      // The 19-bit offset to the relocation target,
+      // for conditional B instruction.
+      uint64_t PCRelVal = Value + RE.Addend - FinalAddress;
+      assert(isInt<21>(PCRelVal) && "Branch target is out of range.");
+      write32le(Target, (read32le(Target) & ~(0x00FFFFE0)) |
+                            (PCRelVal & 0x001FFFFC) << 3);
+      break;
+    }
+    case COFF::IMAGE_REL_ARM64_BRANCH14: {
+      // The 14-bit offset to the relocation target,
+      // for instructions TBZ and TBNZ.
+      uint64_t PCRelVal = Value + RE.Addend - FinalAddress;
+      assert(isInt<16>(PCRelVal) && "Branch target is out of range.");
+      write32le(Target, (read32le(Target) & ~(0x000FFFE0)) |
+                            (PCRelVal & 0x0000FFFC) << 3);
+      break;
+    }
+    case COFF::IMAGE_REL_ARM64_ADDR64: {
+      // The 64-bit VA of the relocation target.
+      write64le(Target, Value + RE.Addend);
+      break;
+    }
+    case COFF::IMAGE_REL_ARM64_SECTION: {
+      // 16-bit section index of the section that contains the target.
+      assert(static_cast<uint32_t>(RE.SectionID) <= UINT16_MAX &&
+             "relocation overflow");
+      add16(Target, RE.SectionID);
+      break;
+    }
+    case COFF::IMAGE_REL_ARM64_SECREL: {
+      // 32-bit offset of the target from the beginning of its section.
+      assert(static_cast<int64_t>(RE.Addend) <= INT32_MAX &&
+             "Relocation overflow");
+      assert(static_cast<int64_t>(RE.Addend) >= INT32_MIN &&
+             "Relocation underflow");
+      write32le(Target, RE.Addend);
+      break;
+    }
+    case COFF::IMAGE_REL_ARM64_REL32: {
+      // The 32-bit relative address from the byte following the relocation.
+      uint64_t Result = Value - FinalAddress - 4;
+      write32le(Target, Result + RE.Addend);
+      break;
+    }
+    }
+  }
+
+  void registerEHFrames() override {}
+};
+
+} // End namespace llvm
+
+#endif

lib/Target/AArch64/AArch64TargetMachine.cpp

if (*CM != CodeModel::Small && *CM != CodeModel::Tiny &&
                            "are allowed on AArch64");
     } else if (*CM == CodeModel::Tiny && !TT.isOSBinFormatELF())
       report_fatal_error("tiny code model is only supported on ELF");
     return *CM;
   }
   // The default MCJIT memory managers make no guarantees about where they can
   // find an executable page; JITed code needs to be able to refer to globals
   // no matter how far away they are.
-  if (JIT)
+  // We should set the CodeModel::Small for Windows ARM64 in JIT mode,
+  // since with large code model LLVM generating 4 MOV instructions, and
+  // Windows doesn't support relocating these long branch (4 MOVs).
+  if (JIT && !TT.isOSWindows())
     return CodeModel::Large;
   return CodeModel::Small;
 }
 
 /// Create an AArch64 architecture model.
 ///
 AArch64TargetMachine::AArch64TargetMachine(const Target &T, const Triple &TT,
                                            StringRef CPU, StringRef FS,

test/ExecutionEngine/RuntimeDyld/AArch64/COFF_AArch64.s

+// RUN: llvm-mc -triple aarch64-windows -filetype obj -o %t.obj %s
+// RUN: llvm-rtdyld -triple aarch64-windows -dummy-extern dummy=0x79563413 -dummy-extern dummyA=0x78566879 -target-addr-start=40960000000000 -verify -check %s %t.obj
+
+  .text
+  .def _bnamed
+  .scl 2
+  .type 32
+  .endef
+
+  .globl _bnamed
+  .align 2
+_bnamed:
+  ret
+
+  .def _foo
+  .scl 2
+  .type 32
+  .endef
+  .globl _foo
+  .align 2
+_foo:
+  movz  w0, #0
+  ret
+
+  .globl _test_adr_relocation
+  .align 2
+
+# IMAGE_REL_ARM64_REL21
+# rtdyld-check:  decode_operand(adr1, 1) = (_const[20:0] - adr1[20:0])
+_test_adr_relocation:
+adr1:
+  adr x0, _const
+  ret
+
+  .globl _test_branch26_reloc
+  .align 2
+
+# IMAGE_REL_ARM64_BRANCH26, test long branch
+# rtdyld-check:  decode_operand(brel, 0)[25:0] = (stub_addr(COFF_AArch64.obj/.text, dummy) - brel)[27:2]
+_test_branch26_reloc:
+brel:
+  b dummy
+  ret
+
+  .globl _test_branch19_reloc
+  .align 2
+
+# IMAGE_REL_ARM64_BRANCH19
+# rtdyld-check:  decode_operand(bcond, 1)[18:0] = (_foo - bcond)[20:2]
+_test_branch19_reloc:
+  mov x0, #3
+  cmp x0, #2
+bcond:
+  bne _foo
+  ret
+
+  .globl _test_branch14_reloc
+  .align 2
+
+# IMAGE_REL_ARM64_BRANCH14
+## rtdyld-check:  decode_operand(tbz_branch, 1) = (_bname - tbz_branch)[15:2]
+_test_branch14_reloc:
+  mov x1, #0
+tbz_branch:
+  tbz x1, #0, _bnamed
+  ret
+
+  .globl  _test_adrp_ldr_reloc
+  .align  2
+
+# IMAGE_REL_ARM64_PAGEBASE_REL21
+# rtdyld-check:  decode_operand(adrp1, 1) = (_const[32:12] - adrp1[32:12])
+_test_adrp_ldr_reloc:
+adrp1:
+  adrp x0, _const
+
+# IMAGE_REL_ARM64_PAGEOFFSET_12L
+# rtdyld-check:  decode_operand(ldr1, 2) = _const[11:3]
+ldr1:
+  ldr  x0, [x0, #:lo12:_const]
+  ret
+
+  .globl  _test_add_reloc
+  .align  2
+
+# IMAGE_REL_ARM64_PAGEOFFSET_12A
+# rtdyld-check: decode_operand(add1, 2) = (tgt+4)[11:0]
+_test_add_reloc:
+add1:
+  add x0, x0, tgt@PAGEOFF+4
+  ret
+
+  .section .data
+  .globl _test_addr64_reloc
+  .align 2
+
+# IMAGE_REL_ARM64_ADDR64
+# rtdyld-check:  *{8}addr64 = tgt+4
+_test_addr64_reloc:
+addr64:
+  .quad tgt+4
+
+# IMAGE_REL_ARM64_ADDR32
+# rtdyld-check:  *{4}_test_addr32_reloc = 0x78566879
+_test_addr32_reloc:
+  .long dummyA
+
+  .globl _relocations
+  .align 2
+
+# IMAGE_REL_ARM64_ADDR32NB, RVA of the target
+# rtdyld-check:  *{4}_relocations = _foo - 40960000000000
+_relocations:
+  .long _foo@IMGREL
+
+# IMAGE_REL_ARM64_ADDR32NB
+# rtdyld-check:  *{4}imgrel2 = _string - 40960000000000+5
+imgrel2:
+  .long _string@IMGREL+5
+
+# IMAGE_REL_ARM64_SECTION
+# rtdyld-check: *{2}secindex = 1
+secindex:
+  .secidx _test_addr32_reloc
+
+# IMAGE_REL_ARM64_SECREL
+# rtdyld-check: *{4}secrel = string - section_addr(COFF_AArch64.obj, .data)
+secrel:
+  .secrel32 string
+
+  .globl _const
+  .align 3
+_const:
+  .quad 4614256650576692846
+
+tgt:
+  .word 1
+  .word 2
+  .word 3
+  .word 4
+  .word 5
+
+  .globl string
+  .align 2
+string:
+  .asciz "Hello World\n"
+
+  .section .rdata,"dr"
+  .globl _string
+  .align 2
+_string:
+  .asciz "Hello World\n"