AArch64: make sure jump table entries can reach entire image

[t.p.northover]

Summary

This patch is addressing two issues:

1. https://bugs.llvm.org/show_bug.cgi?id=47124. A weak function containing a jump table causes a linker warning on Darwin because the entries (LBB - lJTI) refer back to the function and can't be overridden.
2. The AArch64 "small" code model that we default to requires that all statically allocated code and data is within a 4GB window, somewhere in memory. Unfortunately our jump table entries are 32-bit signed constants which only have a 2GB reach, and they're located in a different section (to allow for things like execute-only memory).

Without that second issue I'd prefer to silence the linker warning somehow, and without the first I'd be tempted to let sleeping dogs lie, but together I think they make a compelling enough case to change how we do jump tables. Currently we emit something like

    adrp x0, LJTI0_0@PAGE
    add x0, x0, LJTI0_0@PAGEOFF
    ldrsw x1, [x0, x2, lsl #2]
    add x0, x0, x1
    br x0
    [...]
LJTI0_0:
    .word LBB0_0 - LJTI0_0

If we can no longer use LJTI0_0 as the base we need something within the function that has to be materialized separately. So this changes the sequence to be

    adrp x0, LJTI0_0@PAGE
    add x0, x0, LJTI0_0@PAGEOFF
Ltmp0:
    adr x1, Ltmp0
    ldrsw x2, [x0, x2, lsl #2]
    add x1, x1, x2, lsl #2
    br x1
[...]
LJTI0_0:
    .word (LBB0_0 - Ltmp0)>>2

[t.p.northover]

Decided shifting 32-bit entries was pointless and possibly suboptimal, so skip it. Also noticed that the resize could truncate the SmallVector so guarded against that and added test.

[efriedma]

Given that most jump tables are probably compressed anyway, I don't expect any big performance impact. But it would be nice to have some numbers.

llvm/lib/Target/AArch64/AArch64AsmPrinter.cpp

858

For 32-bit, we sign-extend? I guess this is assuming a function can't be larger than 2GB? (I think you could avoid that assumption by using the compression infrastructure for 32-bit words.)

[efriedma]

LGTM, but I still would like to see some numbers before merge.

llvm/lib/Target/AArch64/AArch64AsmPrinter.cpp

858

On second thought, you're likely to run into other issues well before then, so probably not important.

[t.p.northover]

Thanks Eli, I'll get back to you with numbers.

I've also discovered (well, Ahmed did) that it's not robust in the face of tail duplication. The first adr would steal and emit the label, which might not be in range of the duplicated adr. The fix is easy enough (isNotDuplicable in TableGen) but I still need to write a nice tempting IR file for the test.

llvm/lib/Target/AArch64/AArch64AsmPrinter.cpp

858

Yes, I think the function limit is probably well below 2GB already.

[efriedma]

Instead of trying to make the table relative to the jump/adr, can we use the approach we use for compressed jump tables (using the address of the first destination as the base) unconditionally? Seems less complicated overall. I guess making it work in general costs one extra instruction in general: if you want to handle functions larger than the range of adr, you need to use adrp instead of adr. But functions that large are rare, and the potential upside from duplicating jumptable jumps is pretty large.

[t.p.northover]

Instead of trying to make the table relative to the jump/adr, can we use the approach we use for compressed jump tables (using the address of the first destination as the base) unconditionally? Seems less complicated overall.

As you point out, adr may not have enough range in that case so it would have to be an adrp system. Further, because of MachO limitations on relocation addends (have I said how much I hate that format in the last week? Time to do it again), it can't be anything inside the function, leaving only the function symbol itself.

All in all, I was actually quite happy with this patch on the complexity front. It unifies the three JumpTableDest paths, squeaking in at removing code on average (+50/-62 excluding tests). I don't think the same would be true for a split adrp/adr solution, even if we don't try to optimize and produce adr when possible.

[t.p.northover]

• Mark non-duplicable
• Fix incorrect immediate for ldrsw shift component.

Sorry about the delay in getting back to you with benchmarks, it took me a while to get LNT running reliably again. I don't think there was anything really significant:

CFP2006/453.povray/453.povray.exec	56.5	55.7	-1.423%
CINT2006/473.astar/473.astar.exec	391.1	389.8	-0.326%
CFP2006/470.lbm/470.lbm.exec	241.0	240.7	-0.112%
CINT2006/429.mcf/429.mcf.exec	85.4	85.3	-0.06909%
CINT2006/483.xalancbmk/483.xalancbmk.exec	1271.8	1271.0	-0.06039%
CFP2006/433.milc/433.milc.exec	169.8	169.7	-0.01231%
CINT2006/464.h264ref/464.h264ref.exec	594.7	594.7	+0.001244%
CFP2006/444.namd/444.namd.exec	144.9	144.9	+0.001587%
CINT2006/456.hmmer/456.hmmer.exec	412.0	412.1	+0.01723%
CINT2006/401.bzip2/401.bzip2.exec	336.0	336.2	+0.06515%
CFP2006/447.dealII/447.dealII.exec	568.0	568.6	+0.1038%
CINT2006/458.sjeng/458.sjeng.exec	714.1	715.4	+0.1882%
CINT2006/403.gcc/403.gcc.exec	5.2	5.2	+0.2582%
CINT2006/445.gobmk/445.gobmk.exec	439.2	440.4	+0.2823%
CINT2006/462.libquantum/462.libquantum.exec	10.1	10.1	+0.3071%
CFP2006/482.sphinx3/482.sphinx3.exec	61.6	61.8	+0.3138%
CFP2006/450.soplex/450.soplex.exec	36.7	36.8	+0.3174%
CINT2006/400.perlbench/400.perlbench.exec	139.1	139.6	+0.4139%
CINT2006/471.omnetpp/471.omnetpp.exec	527.3	530.6	+0.6203%

[t.p.northover]

Fix to win64-jumptable.ll hadn't been committed.

[efriedma]

Further, because of MachO limitations on relocation addends (have I said how much I hate that format in the last week? Time to do it again), it can't be anything inside the function, leaving only the function symbol itself.

LLVM seems to accept "adrp x8, (f+1)@PAGE"; does that not do the right thing?

I'm a little concerned about the potential performance impact of the isNotDuplicable marking (that's the one change that impacts compressed jump tables), but your results seem to indicate it isn't significant.

[efriedma]

In D87286#2280553, @efriedma wrote:

Further, because of MachO limitations on relocation addends (have I said how much I hate that format in the last week? Time to do it again), it can't be anything inside the function, leaving only the function symbol itself.

LLVM seems to accept "adrp x8, (f+1)@PAGE"; does that not do the right thing?

Oh, that has limited range; I guess it doesn't help.

In that case, LGTM the way it is, I guess.

[t.p.northover]

I'm a little concerned about the potential performance impact of the isNotDuplicable marking (that's the one change that impacts compressed jump tables), but your results seem to indicate it isn't significant.

I actually suspect that might be positive, or at least not as clear cut. The duplication code is pretty sensitive about the size of the sequence it duplicates, and doesn't know this expands to 3 real instructions (to even get the test working on simple code I had to increase the threshhold via a dev command-line option).

Oh, that has limited range; I guess it doesn't help.

Yep, that's actually what started this whole saga (someone was obfuscating their code and ended up with a huge function with broken jump tables).

Anyway, thanks for the review. It's committed now as 2afe4becec7.

llvm/lib/Target/AArch64/AArch64AsmPrinter.cpp

public:
   /// Wrapper for MCInstLowering.lowerOperand() for the
   /// tblgen'erated pseudo lowering.
   bool lowerOperand(const MachineOperand &MO, MCOperand &MCOp) const {
     return MCInstLowering.lowerOperand(MO, MCOp);
   }
 
   void emitStartOfAsmFile(Module &M) override;
   void emitJumpTableInfo() override;
-  void emitJumpTableEntry(const MachineJumpTableInfo *MJTI,
-                          const MachineBasicBlock *MBB, unsigned JTI);
 
-  void LowerJumpTableDestSmall(MCStreamer &OutStreamer, const MachineInstr &MI);
+  void LowerJumpTableDest(MCStreamer &OutStreamer, const MachineInstr &MI);
 
   void LowerSTACKMAP(MCStreamer &OutStreamer, StackMaps &SM,
                      const MachineInstr &MI);
   void LowerPATCHPOINT(MCStreamer &OutStreamer, StackMaps &SM,
                        const MachineInstr &MI);
 
   void LowerPATCHABLE_FUNCTION_ENTER(const MachineInstr &MI);
   void LowerPATCHABLE_FUNCTION_EXIT(const MachineInstr &MI);
for (unsigned JTI = 0, e = JT.size(); JTI != e; ++JTI) {
 
     // If this jump table was deleted, ignore it.
     if (JTBBs.empty()) continue;
 
     unsigned Size = AFI->getJumpTableEntrySize(JTI);
     emitAlignment(Align(Size));
     OutStreamer->emitLabel(GetJTISymbol(JTI));
 
-    for (auto *JTBB : JTBBs)
-      emitJumpTableEntry(MJTI, JTBB, JTI);
-  }
-}
-
-void AArch64AsmPrinter::emitJumpTableEntry(const MachineJumpTableInfo *MJTI,
-                                           const MachineBasicBlock *MBB,
-                                           unsigned JTI) {
-  const MCExpr *Value = MCSymbolRefExpr::create(MBB->getSymbol(), OutContext);
-  auto AFI = MF->getInfo<AArch64FunctionInfo>();
-  unsigned Size = AFI->getJumpTableEntrySize(JTI);
-
-  if (Size == 4) {
-    // .word LBB - LJTI
-    const TargetLowering *TLI = MF->getSubtarget().getTargetLowering();
-    const MCExpr *Base = TLI->getPICJumpTableRelocBaseExpr(MF, JTI, OutContext);
-    Value = MCBinaryExpr::createSub(Value, Base, OutContext);
-  } else {
-    // .byte (LBB - LBB) >> 2 (or .hword)
-    const MCSymbol *BaseSym = AFI->getJumpTableEntryPCRelSymbol(JTI);
+    const MCSymbol *BaseSym = AArch64FI->getJumpTableEntryPCRelSymbol(JTI);
     const MCExpr *Base = MCSymbolRefExpr::create(BaseSym, OutContext);
+
+    for (auto *JTBB : JTBBs) {
+      const MCExpr *Value =
+          MCSymbolRefExpr::create(JTBB->getSymbol(), OutContext);
+
+      // Each entry is:
+      //     .byte/.hword (LBB - Lbase)>>2
+      // or plain:
+      //     .word LBB - Lbase
       Value = MCBinaryExpr::createSub(Value, Base, OutContext);
+      if (Size != 4)
         Value = MCBinaryExpr::createLShr(
             Value, MCConstantExpr::create(2, OutContext), OutContext);
-  }
 
       OutStreamer->emitValue(Value, Size);
     }
+  }
+}
 
 /// Small jump tables contain an unsigned byte or half, representing the offset
 /// from the lowest-addressed possible destination to the desired basic
 /// block. Since all instructions are 4-byte aligned, this is further compressed
 /// by counting in instructions rather than bytes (i.e. divided by 4). So, to
 /// materialize the correct destination we need:
 ///
 ///             adr xDest, .LBB0_0
 ///             ldrb wScratch, [xTable, xEntry]   (with "lsl #1" for ldrh).
-///             add xDest, xDest, xScratch, lsl #2
-void AArch64AsmPrinter::LowerJumpTableDestSmall(llvm::MCStreamer &OutStreamer,
-                                                const llvm::MachineInstr &MI) {
+///             add xDest, xDest, xScratch (with "lsl #2" for smaller entries)
+void AArch64AsmPrinter::LowerJumpTableDest(llvm::MCStreamer &OutStreamer,
   Register DestReg = MI.getOperand(0).getReg();
   Register ScratchReg = MI.getOperand(1).getReg();
   Register ScratchRegW =
       STI->getRegisterInfo()->getSubReg(ScratchReg, AArch64::sub_32);
   Register TableReg = MI.getOperand(2).getReg();
   Register EntryReg = MI.getOperand(3).getReg();
   int JTIdx = MI.getOperand(4).getIndex();
-  bool IsByteEntry = MI.getOpcode() == AArch64::JumpTableDest8;
+  int Size = AArch64FI->getJumpTableEntrySize(JTIdx);
 
   // This has to be first because the compression pass based its reachability
   // calculations on the start of the JumpTableDest instruction.
   auto Label =
       MF->getInfo<AArch64FunctionInfo>()->getJumpTableEntryPCRelSymbol(JTIdx);
+
+  // If we don't already have a symbol to use as the base, use the ADR
+  // instruction itself.
+  if (!Label) {
+    Label = MF->getContext().createTempSymbol();
+    AArch64FI->setJumpTableEntryInfo(JTIdx, Size, Label);
+    OutStreamer.emitLabel(Label);
+  }
+
+  auto LabelExpr = MCSymbolRefExpr::create(Label, MF->getContext());
   EmitToStreamer(OutStreamer, MCInstBuilder(AArch64::ADR)
                                   .addReg(DestReg)
-                                  .addExpr(MCSymbolRefExpr::create(
-                                      Label, MF->getContext())));
+                                  .addExpr(LabelExpr));
 
   // Load the number of instruction-steps to offset from the label.
-  unsigned LdrOpcode = IsByteEntry ? AArch64::LDRBBroX : AArch64::LDRHHroX;
+  unsigned LdrOpcode;
+  switch (Size) {
+  case 1: LdrOpcode = AArch64::LDRBBroX; break;
+  case 2: LdrOpcode = AArch64::LDRHHroX; break;
+  case 4: LdrOpcode = AArch64::LDRSWroX; break;
+  default:
+    llvm_unreachable("Unknown jump table size");
+  }
+
   EmitToStreamer(OutStreamer, MCInstBuilder(LdrOpcode)
-                                  .addReg(ScratchRegW)
+                                  .addReg(Size == 4 ? ScratchReg : ScratchRegW)
                                   .addReg(TableReg)
                                   .addReg(EntryReg)
                                   .addImm(0)
-                                  .addImm(IsByteEntry ? 0 : 1));
+                                  .addImm(Size == 1 ? 0 : 1));
 
-  // Multiply the steps by 4 and add to the already materialized base label
-  // address.
+  // Add to the already materialized base label address, multiplying by 4 if
+  // compressed.
   EmitToStreamer(OutStreamer, MCInstBuilder(AArch64::ADDXrs)
                                   .addReg(DestReg)
                                   .addReg(DestReg)
                                   .addReg(ScratchReg)
-                                  .addImm(2));
+                                  .addImm(Size == 4 ? 0 : 2));
 }
 
 void AArch64AsmPrinter::LowerSTACKMAP(MCStreamer &OutStreamer, StackMaps &SM,
                                       const MachineInstr &MI) {
   unsigned NumNOPBytes = StackMapOpers(&MI).getNumPatchBytes();
 
   auto &Ctx = OutStreamer.getContext();
   MCSymbol *MILabel = Ctx.createTempSymbol();
case AArch64::TLSDESC_CALLSEQ: {
     MCInst Blr;
     Blr.setOpcode(AArch64::BLR);
     Blr.addOperand(MCOperand::createReg(AArch64::X1));
     EmitToStreamer(*OutStreamer, Blr);
 
     return;
   }
 
-  case AArch64::JumpTableDest32: {
-    // We want:
-    //     ldrsw xScratch, [xTable, xEntry, lsl #2]
-    //     add xDest, xTable, xScratch
-    unsigned DestReg = MI->getOperand(0).getReg(),
-             ScratchReg = MI->getOperand(1).getReg(),
-             TableReg = MI->getOperand(2).getReg(),
-             EntryReg = MI->getOperand(3).getReg();
-    EmitToStreamer(*OutStreamer, MCInstBuilder(AArch64::LDRSWroX)
-                                     .addReg(ScratchReg)
-                                     .addReg(TableReg)
-                                     .addReg(EntryReg)
-                                     .addImm(0)
-                                     .addImm(1));
-    EmitToStreamer(*OutStreamer, MCInstBuilder(AArch64::ADDXrs)
-                                     .addReg(DestReg)
-                                     .addReg(TableReg)
-                                     .addReg(ScratchReg)
-                                     .addImm(0));
-    return;
-  }
+  case AArch64::JumpTableDest32:
   case AArch64::JumpTableDest16:
   case AArch64::JumpTableDest8:
-    LowerJumpTableDestSmall(*OutStreamer, *MI);
+    LowerJumpTableDest(*OutStreamer, *MI);
     return;
 
   case AArch64::FMOVH0:
   case AArch64::FMOVS0:
   case AArch64::FMOVD0:
     EmitFMov0(*MI);
     return;
 

llvm/lib/Target/AArch64/AArch64ISelLowering.cpp

SDValue AArch64TargetLowering::LowerBR_JT(SDValue Op,
                                           SelectionDAG &DAG) const {
   // Jump table entries as PC relative offsets. No additional tweaking
   // is necessary here. Just get the address of the jump table.
   SDLoc DL(Op);
   SDValue JT = Op.getOperand(1);
   SDValue Entry = Op.getOperand(2);
   int JTI = cast<JumpTableSDNode>(JT.getNode())->getIndex();
 
+  auto *AFI = DAG.getMachineFunction().getInfo<AArch64FunctionInfo>();
+  AFI->setJumpTableEntryInfo(JTI, 4, nullptr);
+
   SDNode *Dest =
       DAG.getMachineNode(AArch64::JumpTableDest32, DL, MVT::i64, MVT::i64, JT,
                          Entry, DAG.getTargetJumpTable(JTI, MVT::i32));
   return DAG.getNode(ISD::BRIND, DL, MVT::Other, Op.getOperand(0),
                      SDValue(Dest, 0));
 }
 
 SDValue AArch64TargetLowering::LowerConstantPool(SDValue Op,

llvm/lib/Target/AArch64/AArch64InstrInfo.td

def : Pat<(AArch64LOADgot texternalsym:$addr),
           (LOADgot texternalsym:$addr)>;
 
 def : Pat<(AArch64LOADgot tconstpool:$addr),
           (LOADgot tconstpool:$addr)>;
 
 // 32-bit jump table destination is actually only 2 instructions since we can
 // use the table itself as a PC-relative base. But optimization occurs after
 // branch relaxation so be pessimistic.
-let Size = 12, Constraints = "@earlyclobber $dst,@earlyclobber $scratch" in {
+let Size = 12, Constraints = "@earlyclobber $dst,@earlyclobber $scratch",
+    isNotDuplicable = 1 in {
 def JumpTableDest32 : Pseudo<(outs GPR64:$dst, GPR64sp:$scratch),
                              (ins GPR64:$table, GPR64:$entry, i32imm:$jti), []>,
                       Sched<[]>;
 def JumpTableDest16 : Pseudo<(outs GPR64:$dst, GPR64sp:$scratch),
                              (ins GPR64:$table, GPR64:$entry, i32imm:$jti), []>,
                       Sched<[]>;
 def JumpTableDest8 : Pseudo<(outs GPR64:$dst, GPR64sp:$scratch),
                             (ins GPR64:$table, GPR64:$entry, i32imm:$jti), []>,

llvm/lib/Target/AArch64/AArch64MachineFunctionInfo.h

#endif
 
   unsigned getVarArgsFPRSize() const { return VarArgsFPRSize; }
   void setVarArgsFPRSize(unsigned Size) { VarArgsFPRSize = Size; }
 
   unsigned getSRetReturnReg() const { return SRetReturnReg; }
   void setSRetReturnReg(unsigned Reg) { SRetReturnReg = Reg; }
 
   unsigned getJumpTableEntrySize(int Idx) const {
-    auto It = JumpTableEntryInfo.find(Idx);
-    if (It != JumpTableEntryInfo.end())
-      return It->second.first;
-    return 4;
+    return JumpTableEntryInfo[Idx].first;
   }
   MCSymbol *getJumpTableEntryPCRelSymbol(int Idx) const {
-    return JumpTableEntryInfo.find(Idx)->second.second;
+    return JumpTableEntryInfo[Idx].second;
   }
   void setJumpTableEntryInfo(int Idx, unsigned Size, MCSymbol *PCRelSym) {
+    if ((unsigned)Idx >= JumpTableEntryInfo.size())
+      JumpTableEntryInfo.resize(Idx+1);
     JumpTableEntryInfo[Idx] = std::make_pair(Size, PCRelSym);
   }
 
   using SetOfInstructions = SmallPtrSet<const MachineInstr *, 16>;
 
   const SetOfInstructions &getLOHRelated() const { return LOHRelated; }
 
   // Shortcuts for LOH related types.
void setCalleeSaveBaseToFrameRecordOffset(int Offset) {
     CalleeSaveBaseToFrameRecordOffset = Offset;
   }
 
 private:
   // Hold the lists of LOHs.
   MILOHContainer LOHContainerSet;
   SetOfInstructions LOHRelated;
 
-  DenseMap<int, std::pair<unsigned, MCSymbol *>> JumpTableEntryInfo;
+  SmallVector<std::pair<unsigned, MCSymbol *>, 2> JumpTableEntryInfo;
 };
 
 namespace yaml {
 struct AArch64FunctionInfo final : public yaml::MachineFunctionInfo {
   Optional<bool> HasRedZone;
 
   AArch64FunctionInfo() = default;
   AArch64FunctionInfo(const llvm::AArch64FunctionInfo &MFI);

llvm/lib/Target/AArch64/GISel/AArch64InstructionSelector.cpp

bool AArch64InstructionSelector::selectBrJT(MachineInstr &I,
   assert(I.getOpcode() == TargetOpcode::G_BRJT && "Expected G_BRJT");
   Register JTAddr = I.getOperand(0).getReg();
   unsigned JTI = I.getOperand(1).getIndex();
   Register Index = I.getOperand(2).getReg();
   MachineIRBuilder MIB(I);
 
   Register TargetReg = MRI.createVirtualRegister(&AArch64::GPR64RegClass);
   Register ScratchReg = MRI.createVirtualRegister(&AArch64::GPR64spRegClass);
+
+  MF->getInfo<AArch64FunctionInfo>()->setJumpTableEntryInfo(JTI, 4, nullptr);
   auto JumpTableInst = MIB.buildInstr(AArch64::JumpTableDest32,
                                       {TargetReg, ScratchReg}, {JTAddr, Index})
                            .addJumpTableIndex(JTI);
   // Build the indirect branch.
   MIB.buildInstr(AArch64::BR, {}, {TargetReg});
   I.eraseFromParent();
   return constrainSelectedInstRegOperands(*JumpTableInst, TII, TRI, RBI);
 }

llvm/test/CodeGen/AArch64/jump-table-duplicate.mir

+# RUN: llc -run-pass=tailduplication -tail-dup-size=4 %s -o - | FileCheck %s
+
+# JumpTableDest32 uses an `adr` to a temporary label (itself). If duplicated we
+# cannot guarantee reachability for any uses after the first.
+
+# CHECK: JumpTableDest32
+# CHECK-NOT: JumpTableDest32
+
+
+--- |
+  ; ModuleID = 'jump-table.ll'
+  source_filename = "jump-table.ll"
+  target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128"
+  target triple = "arm64-apple-ios"
+  
+  define i32 @test_jumptable32(i32 %in, i1 %tst) {
+    br i1 %tst, label %true, label %false
+  
+  true:                                             ; preds = %0
+    call void @foo()
+    br label %switch
+  
+  false:                                            ; preds = %0
+    call void @bar()
+    br label %switch
+  
+  lbl1:                                             ; preds = %lbl4, %lbl3, %def, %switch
+    %merge = phi i32 [ 1, %switch ], [ 0, %def ], [ 4, %lbl3 ], [ 8, %lbl4 ]
+    ret i32 %merge
+  
+  switch:                                           ; preds = %false, %true
+    switch i32 %in, label %def [
+      i32 0, label %lbl1
+      i32 1, label %lbl2
+      i32 2, label %lbl3
+      i32 4, label %lbl4
+    ]
+  
+  def:                                              ; preds = %switch
+    br label %lbl1
+  
+  lbl2:                                             ; preds = %switch
+    %1 = call i64 @llvm.aarch64.space(i32 262144, i64 undef)
+    ret i32 2
+  
+  lbl3:                                             ; preds = %switch
+    br label %lbl1
+  
+  lbl4:                                             ; preds = %switch
+    br label %lbl1
+  }
+  
+  declare void @foo()
+  
+  declare void @bar()
+  
+  ; Function Attrs: nounwind
+  declare i64 @llvm.aarch64.space(i32, i64) #0
+  
+  attributes #0 = { nounwind }
+
+...
+---
+name:            test_jumptable32
+alignment:       4
+exposesReturnsTwice: false
+legalized:       false
+regBankSelected: false
+selected:        false
+failedISel:      false
+tracksRegLiveness: true
+hasWinCFI:       false
+registers:       []
+liveins:
+  - { reg: '$w0', virtual-reg: '' }
+  - { reg: '$w1', virtual-reg: '' }
+frameInfo:
+  isFrameAddressTaken: false
+  isReturnAddressTaken: false
+  hasStackMap:     false
+  hasPatchPoint:   false
+  stackSize:       32
+  offsetAdjustment: 0
+  maxAlignment:    8
+  adjustsStack:    true
+  hasCalls:        true
+  stackProtector:  ''
+  maxCallFrameSize: 0
+  cvBytesOfCalleeSavedRegisters: 0
+  hasOpaqueSPAdjustment: false
+  hasVAStart:      false
+  hasMustTailInVarArgFunc: false
+  localFrameSize:  0
+  savePoint:       ''
+  restorePoint:    ''
+fixedStack:      []
+stack:
+  - { id: 0, name: '', type: spill-slot, offset: -8, size: 8, alignment: 8, 
+      stack-id: default, callee-saved-register: '$lr', callee-saved-restored: true, 
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 1, name: '', type: spill-slot, offset: -16, size: 8, alignment: 8, 
+      stack-id: default, callee-saved-register: '$fp', callee-saved-restored: true, 
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 2, name: '', type: spill-slot, offset: -24, size: 8, alignment: 8, 
+      stack-id: default, callee-saved-register: '$x19', callee-saved-restored: true, 
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 3, name: '', type: spill-slot, offset: -32, size: 8, alignment: 8, 
+      stack-id: default, callee-saved-register: '$x20', callee-saved-restored: true, 
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+callSites:       []
+constants:       []
+machineFunctionInfo:
+  hasRedZone:      false
+jumpTable:
+  kind:            label-difference32
+  entries:
+    - id:              0
+      blocks:          [ '%bb.9', '%bb.6', '%bb.7', '%bb.5', '%bb.8' ]
+body:             |
+  bb.0 (%ir-block.0):
+    successors: %bb.1(0x40000000), %bb.2(0x40000000)
+    liveins: $w0, $w1, $x19, $x20, $lr
+  
+    early-clobber $sp = frame-setup STPXpre killed $x20, killed $x19, $sp, -4 :: (store 8 into %stack.3), (store 8 into %stack.2)
+    frame-setup STPXi killed $fp, killed $lr, $sp, 2 :: (store 8 into %stack.1), (store 8 into %stack.0)
+    frame-setup CFI_INSTRUCTION def_cfa_offset 32
+    frame-setup CFI_INSTRUCTION offset $w30, -8
+    frame-setup CFI_INSTRUCTION offset $w29, -16
+    frame-setup CFI_INSTRUCTION offset $w19, -24
+    frame-setup CFI_INSTRUCTION offset $w20, -32
+    renamable $w19 = COPY $w0
+    TBZW killed renamable $w1, 0, %bb.2
+  
+  bb.1.true:
+    successors: %bb.3(0x80000000)
+    liveins: $w19
+  
+    BL @foo, csr_darwin_aarch64_aapcs, implicit-def dead $lr, implicit $sp, implicit-def $sp
+    B %bb.3
+  
+  bb.2.false:
+    successors: %bb.3(0x80000000)
+    liveins: $w19
+  
+    BL @bar, csr_darwin_aarch64_aapcs, implicit-def dead $lr, implicit $sp, implicit-def $sp
+    B %bb.3
+  
+  bb.3.switch:
+    successors: %bb.9(0x1c71c71c), %bb.6(0x1c71c71c), %bb.7(0x1c71c71c), %bb.5(0x0e38e38e), %bb.8(0x1c71c71c)
+    liveins: $w19
+  
+    renamable $w8 = ORRWrs $wzr, killed renamable $w19, 0, implicit-def $x8
+    renamable $x9 = MOVaddrJT target-flags(aarch64-page) %jump-table.0, target-flags(aarch64-pageoff, aarch64-nc) %jump-table.0
+    early-clobber renamable $x10, dead early-clobber renamable $x11 = JumpTableDest32 killed renamable $x9, killed renamable $x8, %jump-table.0
+    BR killed renamable $x10
+  
+  bb.5.def:
+    successors: %bb.9(0x80000000)
+  
+    renamable $w0 = COPY $wzr
+    B %bb.9
+  
+  bb.6.lbl2:
+    successors: %bb.9(0x80000000)
+  
+    dead $xzr = SPACE 262144, undef renamable $x8
+    $w0 = MOVi32imm 2
+    B %bb.9
+  
+  bb.7.lbl3:
+    successors: %bb.9(0x80000000)
+  
+    renamable $w0 = MOVi32imm 4
+    B %bb.9
+  
+  bb.8.lbl4:
+    successors: %bb.9(0x80000000)
+  
+    renamable $w0 = MOVi32imm 8
+  
+  bb.9.lbl1:
+    liveins: $w0
+  
+    $fp, $lr = frame-destroy LDPXi $sp, 2 :: (load 8 from %stack.1), (load 8 from %stack.0)
+    early-clobber $sp, $x20, $x19 = frame-destroy LDPXpost $sp, 4 :: (load 8 from %stack.3), (load 8 from %stack.2)
+    RET_ReallyLR implicit $w0
+
+...

llvm/test/CodeGen/AArch64/jump-table-exynos.ll

 ; RUN: llc -o - %s -mtriple=aarch64-none-linux-gnu -mattr=+force-32bit-jump-tables -aarch64-enable-atomic-cfg-tidy=0 | FileCheck %s
 ; RUN: llc -o - %s -mtriple=aarch64-none-linux-gnu -mcpu=exynos-m3 -aarch64-enable-atomic-cfg-tidy=0 | FileCheck %s
 
 ; Exynos doesn't want jump tables to be compressed for now.
 
 define i32 @test_jumptable(i32 %in)  {
   switch i32 %in, label %def [
     i32 0, label %lbl1
     i32 1, label %lbl2
     i32 2, label %lbl3
     i32 4, label %lbl4
   ]
 ; CHECK-LABEL: test_jumptable:
-; CHECK-NOT: ldrb
+; CHECK:     adrp [[JTPAGE:x[0-9]+]], .LJTI0_0
+; CHECK:     add x[[JT:[0-9]+]], [[JTPAGE]], {{#?}}:lo12:.LJTI0_0
+; CHECK:  [[PCREL_LBL:.Ltmp.*]]:
+; CHECK-NEXT: adr [[PCBASE:x[0-9]+]], [[PCREL_LBL]]
+; CHECK:     ldrsw x[[OFFSET:[0-9]+]], [x[[JT]], {{x[0-9]+}}, lsl #2]
+; CHECK:     add [[DEST:x[0-9]+]], [[PCBASE]], x[[OFFSET]]
+; CHECK:     br [[DEST]]
+
+
+; CHECK: .LJTI0_0:
+; CHECK-NEXT:     .word .LBB{{.*}}-[[PCREL_LBL]]
 
 def:
   ret i32 0
 
 lbl1:
   ret i32 1
 
 lbl2:

llvm/test/CodeGen/AArch64/jump-table.ll

 ; CHECK-IOS-NOT: .data_region
 ; CHECK-IOS: LJTI0_0:
 ; CHECK-IOS-NEXT:     .byte ([[JTBASE]]-[[JTBASE]])>>2
 ; CHECK-IOS-NEXT:     .byte (LBB{{.*}}-[[JTBASE]])>>2
 ; CHECK-IOS-NEXT:     .byte (LBB{{.*}}-[[JTBASE]])>>2
 ; CHECK-IOS-NEXT:     .byte (LBB{{.*}}-[[JTBASE]])>>2
 ; CHECK-IOS-NEXT:     .byte (LBB{{.*}}-[[JTBASE]])>>2
 ; CHECK-IOS-NOT: .end_data_region
+
+; Compressing just the first table has the opportunity to truncate the vector of
+; sizes. Make sure it doesn't.
+define i32 @test_twotables(i32 %in1, i32 %in2) {
+; CHECK-LABEL: test_twotables:
+; CHECK: .LJTI2_0
+; CHECK: .LJTI2_1
+
+  switch i32 %in1, label %def [
+    i32 0, label %lbl1
+    i32 1, label %lbl2
+    i32 2, label %lbl3
+    i32 4, label %lbl4
+  ]
+
+def:
+  ret i32 0
+
+lbl1:
+  ret i32 1
+
+lbl2:
+  ret i32 2
+
+lbl3:
+  ret i32 4
+
+lbl4:
+  switch i32 %in1, label %def [
+    i32 0, label %lbl5
+    i32 1, label %lbl6
+    i32 2, label %lbl7
+    i32 4, label %lbl8
+  ]
+
+lbl5:
+  call i64 @llvm.aarch64.space(i32 262144, i64 undef)
+  ret i32 1
+
+lbl6:
+  call i64 @llvm.aarch64.space(i32 262144, i64 undef)
+  ret i32 2
+
+lbl7:
+  call i64 @llvm.aarch64.space(i32 262144, i64 undef)
+  ret i32 4
+lbl8:
+  call i64 @llvm.aarch64.space(i32 262144, i64 undef)
+  ret i32 8
+
+}
+
+declare i64 @llvm.aarch64.space(i32, i64)

llvm/test/CodeGen/AArch64/win64-jumptable.ll

 
 ; CHECK:    .text
 ; CHECK:    f:
 ; CHECK:    .seh_proc f
 ; CHECK:    b g
 ; CHECK-NEXT:  .seh_endfunclet
 ; CHECK-NEXT: .p2align  2
 ; CHECK-NEXT: .LJTI0_0:
-; CHECK:    .word .LBB0_2-.LJTI0_0
-; CHECK:    .word .LBB0_3-.LJTI0_0
-; CHECK:    .word .LBB0_4-.LJTI0_0
-; CHECK:    .word .LBB0_5-.LJTI0_0
+; CHECK:    .word .LBB0_2-.Ltmp0
+; CHECK:    .word .LBB0_3-.Ltmp0
+; CHECK:    .word .LBB0_4-.Ltmp0
+; CHECK:    .word .LBB0_5-.Ltmp0
 ; CHECK:    .section  .xdata,"dr"
 ; CHECK:    .seh_handlerdata
 ; CHECK:    .text
 ; CHECK:    .seh_endproc
 
 ; Check that we can emit an object file with correct unwind info.
 ; UNWIND: FunctionLength: {{[1-9][0-9]*}}