[AArch64][PAC] Check authenticated LR value during tail call

[atrosinenko]

Summary

When performing a tail call, check the value of LR register after
authentication to prevent the callee from signing and spilling an
untrusted value. This commit implements a few variants of check,
more can be added later.

If it is safe to assume that executable pages are always readable,
LR can be checked just by dereferencing the LR value via LDR.

As an alternative, LR can be checked as follows:

  ; lowered AUT* instruction
  ; <some variant of check that LR contains a valid address>
  b.cond break_block
ret_block:
  ; lowered TCRETURN
break_block:
  brk 0xc471

As the existing methods either break the compatibility with execute-only
memory mappings or can degrade the performance, they are disabled by
default and can be explicitly enabled with a command line option.

Individual subtargets can opt-in to use one of the available methods
by updating AArch64FrameLowering::getAuthenticatedLRCheckMethod().

[atrosinenko]

This patch is inspired by the commit https://github.com/ahmedbougacha/llvm-project/commit/58cf59b84ca4e7930a640480fd5ad1ea194864f5 (and uses the same immediate operand for BRK instruction) but adds the checks during epilogue insertion instead of asm printing.

[kristof.beyls]

llvm/lib/Target/AArch64/AArch64FrameLowering.cpp

261–265

My understanding is that using a load instruction to check the LR can only be done when code segments are not "execute-only".

The commit comment on https://github.com/llvm/llvm-project/commit/a932cd409b861582902211690b497cafc774bee6 suggests that at least LLD assumes that code generated for AArch64 targets is always compatible with execute-only segments. Therefore, I think that defaulting to checking-by-loading-lr is probably the wrong thing to do. It seems to me it would be better to default the other way and only allow checking-by-loading-lr when there is a guarantee that the target environment will not enforce "execute-only" code.

1992

This is just nitpicking:

Would it be useful to have an isTailCallReturn function somewhere, and insert an

assert(!isTailCallReturn(TI->getOpcode())

here? Given how a range of tail call pseudo opcodes have been added recently, it might be likely that a few more could get added in the future, in which case this switch statement needs to be adapted.
I'm just always very cautious when doing a switch on a set of specific opcodes as opcodes tend to evolve over time and such switch statements might be hard to maintain correctly. That's why I tend to prefer having an assert in the default statement that hopefully catches when that happens.

FWIW, https://github.com/llvm/llvm-project/blob/2df05cd01c17f3ef720e554dc7cde43df27e5224/llvm/lib/Target/AArch64/AArch64FrameLowering.cpp#L275 already has a computation of an "IsTailCall" in this file (albeit that it doesn't consider TCRETURNriALL to be a tail call - is that an indication of an instance of the issue I described above?)).

2000

I think this comment would be easier to read if instead of just saying "Turn it into:", it would also clearly indicate the intended effect. For example:

"To avoid generating a signing oracle, generate a code sequence that explicitly checks if the authentication passed or failed, as follows."

With just having written those extra few words now, I think that this extra checking may not be needed if the target core implements FEAT_FPAC (and maybe FEAT_FPACCOMBINE?). If so, maybe a FIXME here would be good to not generate the extra checking code sequences when FEAT_FPAC is implemented by the targeted core?

llvm/test/CodeGen/AArch64/sign-return-address-tailcall.ll

12

nitpick: I think I'd prefer [[AUTIASP]] rather than [[AUT]] as the macro name as that makes it clearer on first read exactly which authenticate operation is expected here. I do like the use of the macro though to hide away the difference between hint #29 and autiasp.

20

nitpick: similarly, I think I'd prefer [[XPACLRI]]

[atrosinenko]

Address the review comments.

[atrosinenko]

Updated the patch.

llvm/lib/Target/AArch64/AArch64FrameLowering.cpp

261–265

Updated. Though, I wonder if using the long snippet by default can introduce noticeable performance regression or not.

Meanwhile, is it permitted to remove a dead load in the machine pipeline (and should I mark it as volatile somehow).

1992

Added an AArch64InstrInfo::isTailCallReturnInst(MachineInstr &) function and redirected there the existing code in getArgumentStackToRestore(). As far as I understand, the new TCRETURNriALL is only used by machine outliner and it seems to be used interchangeably with the existing TCRETURNdi instruction in https://github.com/llvm/llvm-project/blob/feafc2df43545e61a0ba67253284ecbabfd2ba09/llvm/lib/Target/AArch64/AArch64InstrInfo.cpp#L8076C24-L8076C24.

cc: @olista01

2000

Fixed.

There exist separate review items D156784 and D156785 on taking FEAT_FPAC into account. I wonder if the particular CPU models should mention +fpac as supported or should it be only requested explicitly by the user - unlike many other subtarget features, FPAC doesn't make executable code explicitly fail on an unsupported CPU but silently makes it a bit less secure. So, the case "I have CPU X that implements all the instructions that are supported by Y (but not FEAT_FPAC)" may technically be unnoticed.

llvm/test/CodeGen/AArch64/sign-return-address-tailcall.ll

12

Fixed

20

Fixed

[kristof.beyls]

llvm/lib/Target/AArch64/AArch64FrameLowering.cpp

261–265

Yeah, I would expect the long snippet could result in a noticeable performance regression.
As discussed in last Monday's Pointer Authentication sync-up call, most current AArch64-based systems do not enforce execute-only. But that will probably change at some point in the future.
Maybe it would be best to make the code generation dependent on whether the target platform enforces execute-only? That information would probably need to be stored in some kind of TargetInformation class - I'm not sure if that information is currently stored anywhere.

@ab in that meeting also said that there was a 3rd option - checking the values of bits (I think, I don't fully remember) 55 and 56. Would that be a good option to implement/use by default?

It seems to me that at least in principle, later optimization passes are allowed to remove load instructions that they can prove have no side-effects. It may be prudent indeed to mark the load as volatile - or use any other mechanism to indicate that the load does have a side-effect and shouldn't be removed by later optimizations.

2000

Good question!
I don't have a clear answer, I'm afraid.
I'd be interested to hear other people's opinions on this.

[atrosinenko]

llvm/lib/Target/AArch64/AArch64FrameLowering.cpp

261–265

Considering other possible options, IIUC something like this fragment is assumed. I glanced through Optimization Guides for a few CPU cores implementing FEAT_PAuth and it looks like XPAC* instructions are usually faster than other PAuth-related instructions (say, latency is 2, throughput is 1 compared to 5/1 for PAC* and AUT*) - this is somewhat expected as XPAC just clears a range of bits. On the other hand, EOR is still much more efficient.

Thus, it is probably worth implementing yet another option, but I doubt it should be the default because it relies on the particular TBI setting in effect while XPAC "just works" taking into account current settings, as far as I understand.

2000

I think, in D156784 I could just add a TODO at now (to make hasFPAC available for usage) and FeatureFPAC may be added to relevant CPU cores by a later patch, if needed.

[atrosinenko]

Rebased onto current main branch, will upload a few fixes shortly.

[atrosinenko]

Refactoring.

[atrosinenko]

Reworked the patch

• replaced "use a fast checker or not" boolean argument with multiple choices
• created a separate source file and put checkAuthenticatedRegister() function there as it is likely to be later used by other parts of the codegen
• exposed a tail-call-specific AArch64FrameLowering::checkAuthenticatedLR() similarly to signLR() and authenticateLR() so it can be used by outliner callbacks later
• marked a dummy load instruction as volatile similar to Hexagon::PS_crash from HexagonInstrInfo.cpp
• added the third implementation of authenticated address checker, similar to this code. Of course, more checkers can be implemented, but I think it would be better to leave this patch as a common implementation + a few "proofs of concept" and add more checkers via later patches, if needed.

No more changes remain planned on this patch, so I expect it to be ready for further review.

llvm/lib/Target/AArch64/AArch64FrameLowering.cpp

261–265

I reverted this option to using a fast checker by default. As far as I understand, there are no explicit function telling whether this particular subtarget expects execute-only-compatible code as it is expected to "just work" on AArch64. On the other hand, IIUC the support for execute-only mappings was recently dropped at least on Linux because of interference with Privilege Access Never feature.

At now, I just marked this option with a TODO because I expect DummyLoad to be much faster and even if someone want to use it in an execute-only environment, the issue is at least unlikely to be unnoticed.

[kristof.beyls]

llvm/lib/Target/AArch64/AArch64FrameLowering.cpp

261–265

I've done some further investigations and it seems that execute-only enforcement is most likely to get enabled in the not-too-distant future for some very popular AArch64 platforms. Therefore, having something that breaks the execute-only property enabled by default across all AArch64 platforms looks like a no-go. This would break most programs on those platforms.

It seems that this patch also changes code generation for pac-ret (i.e. where only return addresses are protected by pointer authentication)? There are a number of AArch64 platforms that already ship with pac-ret.
Enabling this for pac-ret moves the performance vs code-size vs security hardening trade-off for those platforms for their default code generation. Therefore, it seems to me that a lot of benchmarking would be needed to measure the code size and performance impact of this change before landing it for pac-ret.

With the above 2 observations in mind, I think that:

• using the AuthCheckMethod::DummyLoad cannot be the default across all AArch64 platforms, as it will break most code on AArch64 platforms that plan to enable execute-only; and there is a strong indication that execute-only will be enforced on some of the most popular platforms.
• Protecting against this authentication oracle for pac-ret code generation could only be done by doing substantial amounts of benchmarking to help make a decision on whether this is a worthwhile performance vs code-size vs security hardening trade-off.

I'd recommend to:
(a) not change code generation for pac-ret at all in this patch.
(b) change the default for AuthenticatedLRCheckMethod to something that does not break execute-only.

Ultimately, it seems to me that each platform will have to decide where its default should be in the performance vs code size vs hardening trade-off. With very few software platforms currently choosing to pay the cost for fine-grain control-flow integrity, it seems to me that a default of not hardening against this authentication oracle may be the least bad option available.

[atrosinenko]

• Updated the patch based on the changes from D159357
• Made the check opt-in for subtargets, "none" by default, with command line option to override
• Fixed use after free in createCheckMemOperand function (the same way it is done in Hexagon)

[atrosinenko]

Updated after the D159357 was changed.

[atrosinenko]

Rebased and updated the patch a bit more:

• in getOutliningCandidateInfo, adjusted SequenceSize variable before its first use
• updated code comments
• skipped checking LR if Scadow Call Stack is enabled: the LR value just before TCRETURN* instruction is anyway not the one produced by AUT* (and we cannot check right after AUT* because we cannot be sure which register is usable as a temporary)

[atrosinenko]

Ping.

Here is a summary of the contents of this patch:

• implemented a standalone llvm::AArch64PAuth::checkAuthenticatedRegister utility function to emit one of a number of checks in case a pointer is AUT'ed and not immediately used for memory access
  • placed this function into a sub-namespace instead of making it a static class member, so I don't have to put otherwise irrelevant AArch64PointerAuth class definition to header file
  • note that the checks that are inserted by checkAuthenticatedRegister function are not specific to tail calls (but some of the checks may have restrictions - such as requiring AuthenticatedReg == LR because XPACLRI is encoded as HINT while generic XPAC* instructions require FEAT_PAUTH)
• hooked it to AArch64PointerAuth class via checkAuthenticatedLR method dedicated to hardening tail calls
• in machine outliner, update the costs computed by AArch64InstrInfo::getOutliningCandidateInfo method on a best-effort basis:
  • in MachineOutlinerTailCall outlining mode, we need to insert checks in each caller of OUTLINED_FUNCTION
  • in MachineOutlinerThunk mode, at most a single extra check is inserted in OUTLINED_FUNCTION itself
  • other modes do not introduce new tail calls, so let's just try to account for the checks that would possibly be inserted later into the original candidates by the AArch64PointerAuth pass (including in the two aboves modes)
• factored out isTailCallReturnInst and needsShadowCallStackPrologueEpilogue utility functions

[kristof.beyls]

Thank you for the update!
I haven't looked at Arch64PointerAuth.cpp and the regression test in detail yet.
Most of the other parts of this patch looks fine to me, apart from the 2 nitpicks and the one probably bigger issue with getInstSizeInBytes() - see inline comments.

llvm/lib/Target/AArch64/AArch64InstrInfo.cpp

7412

I had to spend quite a bit of time to understand the logic here.
I think, if possible, it would be easier to understand the logic if SequenceSize was updated much closer to where it is initially computed on line 8133.

Ideally, getInstSizeInBytes(MI) would return the correct number of bytes.

Actually, looking at the documentation of getInstSizeInBytes(MI), at https://github.com/llvm/llvm-project/blob/c4e2fcff788025415b523486efdbdac4f2b08c1e/llvm/lib/Target/AArch64/AArch64InstrInfo.cpp#L83, it states that getInstSizeInBytes is guaranteed to return the maximum number of bytes.
So on a tail call return that might produce lots of bytes, it should return the maximum number.

If it does not produce the maximum number, I seem to remember that that can lead to hard-to-debug compilation failures where the code size estimation for computing whether a constant island is within range could go wrong. (I might be misremembering, it could be something different than a constant island going out of range).

So, I think that getInstSizeInBytes(MI) needs to be adapted so it returns the correct maximum size in bytes for a tail call. When that is implemented, the change on this line in this patch may no longer be needed?

Assuming I remember all of the above correctly, it may be good to add a regression test that verifies that getInstSizeInBytes is calculated correctly for large tail calls. https://reviews.llvm.org/D22870 is a patch that fixed a similar issue on another instruction. The test case that was added there could serve as inspiration.

llvm/lib/Target/AArch64/AArch64PointerAuth.h

88 ↗

(On Diff #557353)

nit pick - feel free to disagree. IIUC, all methods are pre-v8.3 compatible. Is it then worthwhile to call pre-v8.3 compatibility out in the help text? I think I'd remove the "(pre-v8.3 compatible)" part.

94 ↗

(On Diff #557353)

s/succeedes/succeeds/?

[atrosinenko]

Updated the patch, thank you.

llvm/lib/Target/AArch64/AArch64InstrInfo.cpp

7412

On one hand, updating getInstSizeInBytes(MI) should fix not only the computation of SequenceSize but other possible callers of getInstSizeInBytes as well. On the other hand, inserting checker code before TCRETURN* instructions seems like just yet another code transformation (and TCRETURN* instructions are actually four bytes long after AArch64PointerAuth pass processes the function).

Moreover, in getInstSizeInBytes(MI), there is handling for several special cases and a comment stating

// FIXME: We currently only handle pseudoinstructions that don't get expanded
//        before the assembly printer.

Thus, I would rather keep updating SequenceSize ad-hoc, but move the update right after NumBytesToCheckLRInTCEpilogue is computed.

llvm/lib/Target/AArch64/AArch64PointerAuth.h

88 ↗

(On Diff #557353)

There will probably be non v8.2-compatible checkers in the future - for example, if we want to check an *arbitrary* register using XPAC. Though, I agree that it is better to drop the "(pre-v8.3 compatible)" part here: it would be better to add more comprehensible comment like "(requires FEAT_PAUTH)" to those checkers, I think.

94 ↗

(On Diff #557353)

Fixed.

[atrosinenko]

Addressed the comments so far.

[kristof.beyls]

llvm/lib/Target/AArch64/AArch64InstrInfo.cpp

7412

Thanks for pointing to that FIXME - indeed, it seems like as long as the expansion happens early enough, there should be no issue.

llvm/lib/Target/AArch64/AArch64PointerAuth.cpp

147 ↗

(On Diff #557536)

I think this class could use a comment.
Would something like the following be correct?

// AddressCheckPseudoSourceValue represents the memory access inserted by the
// the AuthCheckMethod::DummyLoad method to trigger a run-time error if the
// authentication of a pointer failed.

160 ↗

(On Diff #557536)

Seeing this is a static variable made me think: would it be possible in a single run of any program using the LLVM libraries (such as clang, flang, rust, opt, ...) for MF.getTarget() to potentially return a different result sometimes within the same execution of the that program.
If that could occur, then some from some executions of this function, the CheckPSV variable could be initialized with the wrong TargetMachine reference....

I cannot immediately think of an example, but I am also not sure.
I think it would be more prudent to not have this as a static variable.

191 ↗

(On Diff #557536)

I'm wondering why any/all of the machineinsts created in this function need to have the FrameDestroy flag set? Do you know?

350 ↗

(On Diff #557536)

The convention in the LLVM code base is to use "FIXME" rather than "TODO".

[atrosinenko]

Address the review comments.

[atrosinenko]

llvm/lib/Target/AArch64/AArch64PointerAuth.cpp

147 ↗

(On Diff #557536)

Added your comment, thank you.

160 ↗

(On Diff #557536)

I used the same approach as it is implemented in Hexagon backend: HexagonInstrInfo.cpp. For now TargetMachine is only used by PseudoSourceValue constructor to call TM.getAddressSpaceForPseudoSourceKind(Kind). I agree that using static instance looks suspicious, but I guess allocating new instance from heap on each check is quite wasteful.

350 ↗

(On Diff #557536)

Updated

[atrosinenko]

llvm/lib/Target/AArch64/AArch64PointerAuth.cpp

191 ↗

(On Diff #557536)

Setting MachineInstr::FrameDestroy unconditionally in checkAuthenticatedRegister is definitely a mistake, thank you. Maybe I have to pass MI flags as an argument, but it seems that it works as-is, at least for generating DWARF debug info (see Epilogue Begin marker).

Added an explicit assertion that WinCFI is not requested as I don't yet emit any SEH opcodes.

$ cat /tmp/tail-call.c 
int caller_indirect(int *n, int (fptr)(int*)) {
  asm volatile ("" ::: "lr");
  *n = 42;
  return fptr(n);
}
$ ./bin/clang -O1 -target aarch64-linux-gnu /tmp/tail-call.c -c -o /tmp/tail-call.o -mbranch-protection=pac-ret -mllvm -aarch64-authenticated-lr-check-method=xpac-hint -g
$ dwarfdump -l /tmp/tail-call.o && llvm-objdump -d /tmp/tail-call.o 

.debug_line: line number info for a single cu
Source lines (from CU-DIE at .debug_info offset 0x0000000c):

            NS new statement, BB new basic block, ET end of text sequence
            PE prologue end, EB epilogue begin
            IS=val ISA number, DI=val discriminator value
<pc>        [lno,col] NS BB ET PE EB IS= DI= uri: "filepath"
0x00000000  [   1, 0] NS uri: "/tmp/tail-call.c"
0x0000000c  [   2, 3] NS PE
0x0000000c  [   3, 6] NS
0x00000010  [   4,10] NS ET EB
0x00000030  [   4,10] NS ET


/tmp/tail-call.o:       file format elf64-littleaarch64

Disassembly of section .text:

0000000000000000 <caller_indirect>:
       0: d503233f      paciasp
       4: f81f0ffe      str     x30, [sp, #-0x10]!
       8: 52800548      mov     w8, #0x2a
       c: b9000008      str     w8, [x0]
      10: f84107fe      ldr     x30, [sp], #0x10
      14: d50323bf      autiasp
      18: aa1e03f0      mov     x16, x30
      1c: d50320ff      xpaclri
      20: eb1e021f      cmp     x16, x30
      24: 54000041      b.ne    0x2c <caller_indirect+0x2c>
      28: d61f0020      br      x1
      2c: d4388e20      brk     #0xc471

[kristof.beyls]

llvm/lib/Target/AArch64/AArch64PointerAuth.cpp

160 ↗

(On Diff #557536)

I guess this would be allocating new instances on the stack rather than the heap?
Anyway, I think it would be better to not rely on a static variable where we know this is likely to trigger a hard-to-track-down bug at some point in the future, for some use cases.

If the cost would be too high, I guess it might be possible to "cache" CheckPSV once per MachineFunction being processed? That may require finding an object that is constructed once per MachineFunction and put that cached CheckPSV in there. Not sure if there is such an obvious object at the moment and if so, how much refactoring would be needed to make this possible.

Anyway, I guess this doesn't get called that often (only once per tail call) so presumably it isn't worthwhile to make the code a lot more complicated for the small (would it even be measurable?) compile time gain.

[atrosinenko]

Moved AddressCheckPseudoSourceValue to AArch64SubtargetInfo.

[atrosinenko]

llvm/lib/Target/AArch64/AArch64PointerAuth.cpp

160 ↗

(On Diff #557536)

Unfortunately, MachinePointerInfo stores a pointer to CheckPSV, so allocating it on stack is not possible.

Turned out, each MachineFunction has PseudoSourceValueManager, but it is not subclassed per-target, so caching target-specific PSV would require some refactoring. On the other hand, AddressCheckPseudoSourceValue has quite generic semantics, so moved it to AArch64SubtargetInfo.

[kristof.beyls]

I think that with the last change, all my comments have been addressed.
Please do react if I accidentally missed a comment that needs further work.
Assuming indeed all comments have been addressed: LGTM!

llvm/lib/Target/AArch64/AArch64PointerAuth.cpp

160 ↗

(On Diff #557536)

Thank you, that looks like a neat solution.

[atrosinenko]

Added FIXME mentioning FEAT_FPAC to AArch64PointerAuth::checkAuthenticatedLR.

[atrosinenko]

Thank you for the review. I read through the comments - they are already addressed except maybe for a few FPAC-related notes, so I added a FIXME to checkAuthenticatedLR function and left a note in the discussion of D156784.

I will rebase and retest this patch and land it shortly if everything works.

[chill]

llvm/lib/Target/AArch64/AArch64PointerAuth.cpp

256 ↗

(On Diff #557683)

nit: The check for FEAT_FPAC perhaps can be done in getAuthenticatedLRCheckMethod (and possibly return None), so the logic of deciding whether to emit code or not is kept in one place.

llvm/lib/Target/AArch64/AArch64FrameLowering.cpp

static cl::opt<bool> OrderFrameObjects("aarch64-order-frame-objects",
                                        cl::desc("sort stack allocations"),
                                        cl::init(true), cl::Hidden);
 
 cl::opt<bool> EnableHomogeneousPrologEpilog(
     "homogeneous-prolog-epilog", cl::Hidden,
     cl::desc("Emit homogeneous prologue and epilogue for the size "
              "optimization (default = off)"));
 
+cl::opt<bool> CheckAuthenticatedLRByLoad(
+    "aarch64-check-authenticated-lr-by-load", cl::Hidden, cl::init(false),
+    cl::desc("When performing a tail call with authenticated LR, "
+             "use a load instruction to check the LR"));
+
 STATISTIC(NumRedZoneFunctions, "Number of functions using red zone");
 
 /// Returns how much of the incoming argument stack area (in bytes) we should
 /// clean up in an epilogue. For the C calling convention this will be 0, for
 /// guaranteed tail call conventions it can be positive (a normal return or a
 /// tail call to a function that uses less stack space for arguments) or
 /// negative (for a tail call to a function that needs more stack space than us
 /// for arguments).
 static int64_t getArgumentStackToRestore(MachineFunction &MF,
                                          MachineBasicBlock &MBB) {
   MachineBasicBlock::iterator MBBI = MBB.getLastNonDebugInstr();
-  bool IsTailCallReturn = false;
-  if (MBB.end() != MBBI) {
-    unsigned RetOpcode = MBBI->getOpcode();
-    IsTailCallReturn = RetOpcode == AArch64::TCRETURNdi ||
-                       RetOpcode == AArch64::TCRETURNri ||
-                       RetOpcode == AArch64::TCRETURNriBTI;
-  }
   AArch64FunctionInfo *AFI = MF.getInfo<AArch64FunctionInfo>();
+  bool IsTailCallReturn = (MBB.end() != MBBI)
+                              ? AArch64InstrInfo::isTailCallReturnInst(*MBBI)
+                              : false;
 
   int64_t ArgumentPopSize = 0;
   if (IsTailCallReturn) {
     MachineOperand &StackAdjust = MBBI->getOperand(1);
 
     // For a tail-call in a callee-pops-arguments environment, some or all of
     // the stack may actually be in use for the call's arguments, this is
     // calculated during LowerCall and consumed here...
if (Subtarget.hasPAuth() && TerminatorIsCombinable && !NeedsWinCFI &&
     if (NeedsWinCFI) {
       *HasWinCFI = true;
       BuildMI(MBB, MBBI, DL, TII->get(AArch64::SEH_PACSignLR))
           .setMIFlag(MachineInstr::FrameDestroy);
     }
   }
 }
 
+// Checks authenticated LR just before the TCRETURN* instruction.
+// This function may split MBB - in that case, the returned basic block
+// is the new block containing the return instruction.
+static MachineBasicBlock &checkAuthenticatedLRIfNeeded(MachineFunction &MF,
+                                                       MachineBasicBlock &MBB) {
+  const auto &MFnI = *MF.getInfo<AArch64FunctionInfo>();
+  const auto &Subtarget = MF.getSubtarget<AArch64Subtarget>();
+  const auto *TII = Subtarget.getInstrInfo();
+  const auto *TRI = Subtarget.getRegisterInfo();
+
+  if (!MFnI.shouldSignReturnAddress(MF))
+    return MBB;
+
+  auto TI = MBB.getFirstTerminator();
+  if (TI == MBB.end())
+    return MBB;
+
+  // Only explicitly check LR if we are performing tail call.
+  if (!AArch64InstrInfo::isTailCallReturnInst(*TI))
+    return MBB;
+
+  Register TmpReg =
+      TI->readsRegister(AArch64::X16, TRI) ? AArch64::X17 : AArch64::X16;
+  assert(!TI->readsRegister(TmpReg, TRI) &&
+         "More than a single register is used by TCRETURN");
+
+  // The following code may create a signing oracle:
+  //
+  //   <authenticate LR>
+  //   TCRETURN          ; the callee may sign and spill the LR in its prologue
+  //
+  // To avoid generating a signing oracle, check the authenticated value
+  // before possibly re-signing it in the callee, as follows:
+  //
+  //   <authenticate LR>
+  //   mov tmp, lr
+  //   xpaclri           ; encoded as "hint #7"
+  //   ; Note: at this point, the LR register contains the return address as if
+  //   ; the authentication succeeded and the temporary register contains the
+  //   ; *real* result of authentication.
+  //   cmp tmp, lr
+  //   b.ne break_block
+  // ret_block:
+  //   TCRETURN
+  // break_block:
+  //   brk 0xc471
+  //
+  // or just
+  //
+  //   <authenticate LR>
+  //   ldr tmp, [lr]
+  //   TCRETURN
+
+  const BasicBlock *BB = MBB.getBasicBlock();
+  DebugLoc DL = TI->getDebugLoc();
+
+  if (CheckAuthenticatedLRByLoad) {
+    BuildMI(MBB, TI, DL, TII->get(AArch64::LDRXui), TmpReg)
+        .addReg(AArch64::LR)
+        .addImm(0)
+        .setMIFlags(MachineInstr::FrameDestroy);
+    return MBB;
+  }
+
+  // Auth instruction was previously added to this basic block.
+  assert(TI != MBB.begin() && "Non-terminator instructions expected");
+  auto &LastNonTerminator = *std::prev(TI);
+  MachineBasicBlock *RetBlock = MBB.splitAt(LastNonTerminator);
+
+  MachineBasicBlock *BreakBlock = MF.CreateMachineBasicBlock(BB);
+  MF.push_back(BreakBlock);
+  MBB.splitSuccessor(RetBlock, BreakBlock);
+
+  MachineBasicBlock *AuthBlock = &MBB;
+  BuildMI(AuthBlock, DL, TII->get(TargetOpcode::COPY), TmpReg)
+      .addReg(AArch64::LR)
+      .setMIFlags(MachineInstr::FrameDestroy);
+  BuildMI(AuthBlock, DL, TII->get(AArch64::XPACLRI))
+      .setMIFlags(MachineInstr::FrameDestroy);
+  BuildMI(AuthBlock, DL, TII->get(AArch64::SUBSXrs), AArch64::XZR)
+      .addReg(TmpReg)
+      .addReg(AArch64::LR)
+      .addImm(0)
+      .setMIFlags(MachineInstr::FrameDestroy);
+  BuildMI(AuthBlock, DL, TII->get(AArch64::Bcc))
+      .addImm(AArch64CC::NE)
+      .addMBB(BreakBlock)
+      .setMIFlags(MachineInstr::FrameDestroy);
+  assert(AuthBlock->getFallThrough() == RetBlock);
+
+  BuildMI(BreakBlock, DL, TII->get(AArch64::BRK))
+      .addImm(0xc471)
+      .setMIFlags(MachineInstr::FrameDestroy);
+
+  return *RetBlock;
+}
+
 static bool isFuncletReturnInstr(const MachineInstr &MI) {
   switch (MI.getOpcode()) {
   default:
     return false;
   case AArch64::CATCHRET:
   case AArch64::CLEANUPRET:
     return true;
   }
void AArch64FrameLowering::emitEpilogue(MachineFunction &MF,
   auto WinCFI = make_scope_exit([&]() { assert(HasWinCFI == MF.hasWinCFI()); });
 
   if (MBB.end() != MBBI) {
     DL = MBBI->getDebugLoc();
     IsFunclet = isFuncletReturnInstr(*MBBI);
   }
 
   auto FinishingTouches = make_scope_exit([&]() {
+    MachineBasicBlock *RetMBB = &MBB;
     if (AFI->shouldSignReturnAddress(MF))
       authenticateLR(MF, MBB, NeedsWinCFI, &HasWinCFI);
     if (needsShadowCallStackPrologueEpilogue(MF))
       emitShadowCallStackEpilogue(*TII, MF, MBB, MBB.getFirstTerminator(), DL);
+    else
+      RetMBB = &checkAuthenticatedLRIfNeeded(MF, MBB);
+    // checkAuthenticatedLR() may have split MBB at this point.
     if (EmitCFI)
-      emitCalleeSavedGPRRestores(MBB, MBB.getFirstTerminator());
+      emitCalleeSavedGPRRestores(*RetMBB, RetMBB->getFirstTerminator());
     if (HasWinCFI)
-      BuildMI(MBB, MBB.getFirstTerminator(), DL,
+      BuildMI(*RetMBB, RetMBB->getFirstTerminator(), DL,
               TII->get(AArch64::SEH_EpilogEnd))
           .setMIFlag(MachineInstr::FrameDestroy);
   });
 
   int64_t NumBytes = IsFunclet ? getWinEHFuncletFrameSize(MF)
                                : MFI.getStackSize();
 
   // All calls are tail calls in GHC calling conv, and functions have no

llvm/lib/Target/AArch64/AArch64InstrInfo.h

public:
   static bool isQForm(const MachineInstr &MI);
 
   /// Returns the index for the immediate for a given instruction.
   static unsigned getLoadStoreImmIdx(unsigned Opc);
 
   /// Return true if pairing the given load or store may be paired with another.
   static bool isPairableLdStInst(const MachineInstr &MI);
 
+  /// Returns true if MI is one of the TCRETURN* instructions.
+  static bool isTailCallReturnInst(const MachineInstr &MI);
+
   /// Return the opcode that set flags when possible.  The caller is
   /// responsible for ensuring the opc has a flag setting equivalent.
   static unsigned convertToFlagSettingOpc(unsigned Opc);
 
   /// Return true if this is a load/store that can be potentially paired/merged.
   bool isCandidateToMergeOrPair(const MachineInstr &MI) const;
 
   /// Hint that pairing the given load or store is unprofitable.

llvm/lib/Target/AArch64/AArch64InstrInfo.cpp

bool AArch64InstrInfo::isPairableLdStInst(const MachineInstr &MI) {
   case AArch64::LDRWpre:
   case AArch64::LDURXi:
   case AArch64::LDRXpre:
   case AArch64::LDURSWi:
     return true;
   }
 }
 
+bool AArch64InstrInfo::isTailCallReturnInst(const MachineInstr &MI) {
+  switch (MI.getOpcode()) {
+  default:
+    assert((!MI.isCall() || !MI.isReturn()) &&
+           "Unexpected instruction - was a new tail call opcode introduced?");
+    return false;
+  case AArch64::TCRETURNdi:
+  case AArch64::TCRETURNri:
+  case AArch64::TCRETURNriBTI:
+  case AArch64::TCRETURNriALL:
+    return true;
+  }
+}
+
 unsigned AArch64InstrInfo::convertToFlagSettingOpc(unsigned Opc) {
   switch (Opc) {
   default:
     llvm_unreachable("Opcode has no flag setting equivalent!");
   // 32-bit cases:
   case AArch64::ADDWri:
     return AArch64::ADDSWri;
   case AArch64::ADDWrr:
if (FirstCand.getMF()
     // If the sequence doesn't have enough candidates left, then we're done.
     if (RepeatedSequenceLocs.size() < 2)
       return std::nullopt;
   }
 
   // Properties about candidate MBBs that hold for all of them.
   unsigned FlagsSetInAll = 0xF;
 
   // Compute liveness information for each candidate, and set FlagsSetInAll.
   for (outliner::Candidate &C : RepeatedSequenceLocs)
     FlagsSetInAll &= C.Flags;
 
   unsigned LastInstrOpcode = RepeatedSequenceLocs[0].back()->getOpcode();
 
   // Helper lambda which sets call information for every candidate.
   auto SetCandidateCallInfo =
       [&RepeatedSequenceLocs](unsigned CallID, unsigned NumBytesForCall) {

llvm/test/CodeGen/AArch64/sign-return-address-cfi-negate-ra-state.ll

 ; CHECK-V8A-LABEL: baz_async:
 ; CHECK-V8A:       // %bb.0: // %entry
 ; CHECK-V8A-NEXT:    hint #25
 ; CHECK-V8A-NEXT:    .cfi_negate_ra_state
 ; CHECK-V8A-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-V8A-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-V8A-NEXT:    .cfi_offset w30, -16
 ; CHECK-V8A-NEXT:    .cfi_remember_state
-; CHECK-V8A-NEXT:    cbz w0, .LBB1_2
+; CHECK-V8A-NEXT:    cbz w0, .LBB1_3
 ; CHECK-V8A-NEXT:  // %bb.1: // %if.then
 ; CHECK-V8A-NEXT:    mov w0, wzr
 ; CHECK-V8A-NEXT:    bl _Z3bari
 ; CHECK-V8A-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; CHECK-V8A-NEXT:    .cfi_def_cfa_offset 0
 ; CHECK-V8A-NEXT:    hint #29
 ; CHECK-V8A-NEXT:    .cfi_negate_ra_state
+; CHECK-V8A-NEXT:    mov x16, x30
+; CHECK-V8A-NEXT:    hint #7
+; CHECK-V8A-NEXT:    cmp x16, x30
+; CHECK-V8A-NEXT:    b.ne .LBB1_4
+; CHECK-V8A-NEXT:  // %bb.2: // %if.then
 ; CHECK-V8A-NEXT:    .cfi_restore w30
 ; CHECK-V8A-NEXT:    b _Z3bari
-; CHECK-V8A-NEXT:  .LBB1_2: // %if.else
+; CHECK-V8A-NEXT:  .LBB1_3: // %if.else
 ; CHECK-V8A-NEXT:    .cfi_restore_state
 ; CHECK-V8A-NEXT:    bl _Z4quuxi
 ; CHECK-V8A-NEXT:    add w0, w0, #1
 ; CHECK-V8A-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; CHECK-V8A-NEXT:    .cfi_def_cfa_offset 0
 ; CHECK-V8A-NEXT:    hint #29
 ; CHECK-V8A-NEXT:    .cfi_negate_ra_state
 ; CHECK-V8A-NEXT:    .cfi_restore w30
 ; CHECK-V8A-NEXT:    ret
+; CHECK-V8A-NEXT:  .LBB1_4: // %if.then
+; CHECK-V8A-NEXT:    brk #0xc471
 ;
 ; CHECK-V83A-LABEL: baz_async:
 ; CHECK-V83A:       // %bb.0: // %entry
 ; CHECK-V83A-NEXT:    paciasp
 ; CHECK-V83A-NEXT:    .cfi_negate_ra_state
 ; CHECK-V83A-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-V83A-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-V83A-NEXT:    .cfi_offset w30, -16
 ; CHECK-V83A-NEXT:    .cfi_remember_state
-; CHECK-V83A-NEXT:    cbz w0, .LBB1_2
+; CHECK-V83A-NEXT:    cbz w0, .LBB1_3
 ; CHECK-V83A-NEXT:  // %bb.1: // %if.then
 ; CHECK-V83A-NEXT:    mov w0, wzr
 ; CHECK-V83A-NEXT:    bl _Z3bari
 ; CHECK-V83A-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; CHECK-V83A-NEXT:    .cfi_def_cfa_offset 0
 ; CHECK-V83A-NEXT:    autiasp
 ; CHECK-V83A-NEXT:    .cfi_negate_ra_state
+; CHECK-V83A-NEXT:    mov x16, x30
+; CHECK-V83A-NEXT:    xpaclri
+; CHECK-V83A-NEXT:    cmp x16, x30
+; CHECK-V83A-NEXT:    b.ne .LBB1_4
+; CHECK-V83A-NEXT:  // %bb.2: // %if.then
 ; CHECK-V83A-NEXT:    .cfi_restore w30
 ; CHECK-V83A-NEXT:    b _Z3bari
-; CHECK-V83A-NEXT:  .LBB1_2: // %if.else
+; CHECK-V83A-NEXT:  .LBB1_3: // %if.else
 ; CHECK-V83A-NEXT:    .cfi_restore_state
 ; CHECK-V83A-NEXT:    bl _Z4quuxi
 ; CHECK-V83A-NEXT:    add w0, w0, #1
 ; CHECK-V83A-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; CHECK-V83A-NEXT:    .cfi_def_cfa_offset 0
 ; CHECK-V83A-NEXT:    .cfi_restore w30
 ; CHECK-V83A-NEXT:    retaa
+; CHECK-V83A-NEXT:  .LBB1_4: // %if.then
+; CHECK-V83A-NEXT:    brk #0xc471
 entry:
   %tobool.not = icmp eq i32 %a, 0
   br i1 %tobool.not, label %if.else, label %if.then
 
 if.then:                                          ; preds = %entry
   %call = tail call noundef i32 @_Z3bari(i32 noundef 0)
   %call1 = tail call noundef i32 @_Z3bari(i32 noundef %call)
   br label %return

 define hidden noundef i32 @baz_sync(i32 noundef %a) #0 uwtable(sync) {
 ; CHECK-V8A-LABEL: baz_sync:
 ; CHECK-V8A:       // %bb.0: // %entry
 ; CHECK-V8A-NEXT:    hint #25
 ; CHECK-V8A-NEXT:    .cfi_negate_ra_state
 ; CHECK-V8A-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-V8A-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-V8A-NEXT:    .cfi_offset w30, -16
-; CHECK-V8A-NEXT:    cbz w0, .LBB2_2
+; CHECK-V8A-NEXT:    cbz w0, .LBB2_3
 ; CHECK-V8A-NEXT:  // %bb.1: // %if.then
 ; CHECK-V8A-NEXT:    mov w0, wzr
 ; CHECK-V8A-NEXT:    bl _Z3bari
 ; CHECK-V8A-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; CHECK-V8A-NEXT:    hint #29
+; CHECK-V8A-NEXT:    mov x16, x30
+; CHECK-V8A-NEXT:    hint #7
+; CHECK-V8A-NEXT:    cmp x16, x30
+; CHECK-V8A-NEXT:    b.ne .LBB2_4
+; CHECK-V8A-NEXT:  // %bb.2: // %if.then
 ; CHECK-V8A-NEXT:    b _Z3bari
-; CHECK-V8A-NEXT:  .LBB2_2: // %if.else
+; CHECK-V8A-NEXT:  .LBB2_3: // %if.else
 ; CHECK-V8A-NEXT:    bl _Z4quuxi
 ; CHECK-V8A-NEXT:    add w0, w0, #1
 ; CHECK-V8A-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; CHECK-V8A-NEXT:    hint #29
 ; CHECK-V8A-NEXT:    ret
+; CHECK-V8A-NEXT:  .LBB2_4: // %if.then
+; CHECK-V8A-NEXT:    brk #0xc471
 ;
 ; CHECK-V83A-LABEL: baz_sync:
 ; CHECK-V83A:       // %bb.0: // %entry
 ; CHECK-V83A-NEXT:    paciasp
 ; CHECK-V83A-NEXT:    .cfi_negate_ra_state
 ; CHECK-V83A-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; CHECK-V83A-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-V83A-NEXT:    .cfi_offset w30, -16
-; CHECK-V83A-NEXT:    cbz w0, .LBB2_2
+; CHECK-V83A-NEXT:    cbz w0, .LBB2_3
 ; CHECK-V83A-NEXT:  // %bb.1: // %if.then
 ; CHECK-V83A-NEXT:    mov w0, wzr
 ; CHECK-V83A-NEXT:    bl _Z3bari
 ; CHECK-V83A-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; CHECK-V83A-NEXT:    autiasp
+; CHECK-V83A-NEXT:    mov x16, x30
+; CHECK-V83A-NEXT:    xpaclri
+; CHECK-V83A-NEXT:    cmp x16, x30
+; CHECK-V83A-NEXT:    b.ne .LBB2_4
+; CHECK-V83A-NEXT:  // %bb.2: // %if.then
 ; CHECK-V83A-NEXT:    b _Z3bari
-; CHECK-V83A-NEXT:  .LBB2_2: // %if.else
+; CHECK-V83A-NEXT:  .LBB2_3: // %if.else
 ; CHECK-V83A-NEXT:    bl _Z4quuxi
 ; CHECK-V83A-NEXT:    add w0, w0, #1
 ; CHECK-V83A-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; CHECK-V83A-NEXT:    retaa
+; CHECK-V83A-NEXT:  .LBB2_4: // %if.then
+; CHECK-V83A-NEXT:    brk #0xc471
 entry:
   %tobool.not = icmp eq i32 %a, 0
   br i1 %tobool.not, label %if.else, label %if.then
 
 if.then:                                          ; preds = %entry
   %call = tail call noundef i32 @_Z3bari(i32 noundef 0)
   %call1 = tail call noundef i32 @_Z3bari(i32 noundef %call)
   br label %return

llvm/test/CodeGen/AArch64/sign-return-address-tailcall.ll

+; RUN: llc -mtriple=aarch64 -asm-verbose=0 -aarch64-check-authenticated-lr-by-load              < %s | FileCheck -DAUTIASP="hint #29" --check-prefixes=COMMON,LDR %s
+; RUN: llc -mtriple=aarch64 -asm-verbose=0 -aarch64-check-authenticated-lr-by-load -mattr=v8.3a < %s | FileCheck -DAUTIASP="autiasp"  --check-prefixes=COMMON,LDR %s
+; RUN: llc -mtriple=aarch64 -asm-verbose=0                                                      < %s | FileCheck -DAUTIASP="hint #29" -DXPACLRI="hint #7" --check-prefixes=COMMON,XPAC %s
+; RUN: llc -mtriple=aarch64 -asm-verbose=0                                         -mattr=v8.3a < %s | FileCheck -DAUTIASP="autiasp"  -DXPACLRI="xpaclri" --check-prefixes=COMMON,XPAC %s
+
+define i32 @tailcall_direct() "sign-return-address"="non-leaf" {
+; COMMON-LABEL: tailcall_direct:
+; COMMON:         str x30, [sp, #-16]!
+; COMMON:         ldr x30, [sp], #16
+;
+; LDR-NEXT:       [[AUTIASP]]
+; LDR-NEXT:       ldr x16, [x30]
+; LDR-NEXT:       b callee
+;
+; XPAC-NEXT:      [[AUTIASP]]
+; XPAC-NEXT:      mov x16, x30
+; XPAC-NEXT:      [[XPACLRI]]
+; XPAC-NEXT:      cmp x16, x30
+; XPAC-NEXT:      b.ne .[[FAIL:LBB[_0-9]+]]
+; XPAC-NEXT:      b callee
+; XPAC-NEXT:    .[[FAIL]]:
+; XPAC-NEXT:      brk #0xc471
+  tail call void asm sideeffect "", "~{lr}"()
+  %call = tail call i32 @callee()
+  ret i32 %call
+}
+
+define i32 @tailcall_indirect(ptr %fptr) "sign-return-address"="non-leaf" {
+; COMMON-LABEL: tailcall_indirect:
+; COMMON:         str x30, [sp, #-16]!
+; COMMON:         ldr x30, [sp], #16
+;
+; LDR-NEXT:       [[AUTIASP]]
+; LDR-NEXT:       ldr x16, [x30]
+; LDR-NEXT:       br x0
+;
+; XPAC-NEXT:      [[AUTIASP]]
+; XPAC-NEXT:      mov x16, x30
+; XPAC-NEXT:      [[XPACLRI]]
+; XPAC-NEXT:      cmp x16, x30
+; XPAC-NEXT:      b.ne .[[FAIL:LBB[_0-9]+]]
+; XPAC-NEXT:      br x0
+; XPAC-NEXT:    .[[FAIL]]:
+; XPAC-NEXT:      brk #0xc471
+  tail call void asm sideeffect "", "~{lr}"()
+  %call = tail call i32 %fptr()
+  ret i32 %call
+}
+
+define i32 @tailcall_direct_noframe() "sign-return-address"="non-leaf" {
+; COMMON-LABEL: tailcall_direct_noframe:
+; COMMON-NEXT:    .cfi_startproc
+; COMMON-NEXT:    b callee
+  %call = tail call i32 @callee()
+  ret i32 %call
+}
+
+define i32 @tailcall_indirect_noframe(ptr %fptr) "sign-return-address"="non-leaf" {
+; COMMON-LABEL: tailcall_indirect_noframe:
+; COMMON-NEXT:    .cfi_startproc
+; COMMON-NEXT:    br x0
+  %call = tail call i32 %fptr()
+  ret i32 %call
+}
+
+define i32 @tailcall_direct_noframe_sign_all() "sign-return-address"="all" {
+; COMMON-LABEL: tailcall_direct_noframe_sign_all:
+; COMMON-NOT:     str{{.*}}x30
+; COMMON-NOT:     ldr{{.*}}x30
+;
+; LDR:            [[AUTIASP]]
+; LDR-NEXT:       ldr x16, [x30]
+; LDR-NEXT:       b callee
+;
+; XPAC:           [[AUTIASP]]
+; XPAC-NEXT:      mov x16, x30
+; XPAC-NEXT:      [[XPACLRI]]
+; XPAC-NEXT:      cmp x16, x30
+; XPAC-NEXT:      b.ne .[[FAIL:LBB[_0-9]+]]
+; XPAC-NEXT:      b callee
+; XPAC-NEXT:    .[[FAIL]]:
+; XPAC-NEXT:      brk #0xc471
+  %call = tail call i32 @callee()
+  ret i32 %call
+}
+
+define i32 @tailcall_indirect_noframe_sign_all(ptr %fptr) "sign-return-address"="all" {
+; COMMON-LABEL: tailcall_indirect_noframe_sign_all:
+; COMMON-NOT:     str{{.*}}x30
+; COMMON-NOT:     ldr{{.*}}x30
+;
+; LDR:            [[AUTIASP]]
+; LDR-NEXT:       ldr x16, [x30]
+; LDR-NEXT:       br x0
+;
+; XPAC:           [[AUTIASP]]
+; XPAC-NEXT:      mov x16, x30
+; XPAC-NEXT:      [[XPACLRI]]
+; XPAC-NEXT:      cmp x16, x30
+; XPAC-NEXT:      b.ne .[[FAIL:LBB[_0-9]+]]
+; XPAC-NEXT:      br x0
+; XPAC-NEXT:    .[[FAIL]]:
+; XPAC-NEXT:      brk #0xc471
+  %call = tail call i32 %fptr()
+  ret i32 %call
+}
+
+; Do not emit any LR checks when Shadow Call Stack is enabled
+define i32 @tailcall_scs(ptr %fptr) "sign-return-address"="all" shadowcallstack "target-features"="+reserve-x18" {
+; COMMON-LABEL: tailcall_scs:
+; COMMON:         str x30, [sp, #-16]!
+; COMMON:         ldr x30, [sp], #16
+;
+; COMMON-NOT:     ldr {{.*}}, [x30]
+; COMMON-NOT:     xpac
+; COMMON-NOT:     hint #7
+; COMMON-NOT:     brk
+;
+; Match the end of function:
+; COMMON:         .size tailcall_scs,
+  tail call void asm sideeffect "", "~{lr}"()
+  %call = tail call i32 %fptr()
+  ret i32 %call
+}
+
+declare i32 @callee()

llvm/test/CodeGen/AArch64/sign-return-address.ll

 ; COMPAT-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; COMPAT-NEXT:    .cfi_def_cfa_offset 16
 ; COMPAT-NEXT:    .cfi_offset w30, -16
 ; COMPAT-NEXT:    //APP
 ; COMPAT-NEXT:    mov x30, x0
 ; COMPAT-NEXT:    //NO_APP
 ; COMPAT-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; COMPAT-NEXT:    hint #29
+; COMPAT-NEXT:    mov x16, x30
+; COMPAT-NEXT:    hint #7
+; COMPAT-NEXT:    cmp x16, x30
+; COMPAT-NEXT:    b.ne .LBB9_2
+; COMPAT-NEXT:  // %bb.1:
 ; COMPAT-NEXT:    b bar
+; COMPAT-NEXT:  .LBB9_2:
+; COMPAT-NEXT:    brk #0xc471
 ;
 ; V83A-LABEL: spill_lr_and_tail_call:
 ; V83A:       // %bb.0:
 ; V83A-NEXT:    paciasp
 ; V83A-NEXT:    .cfi_negate_ra_state
 ; V83A-NEXT:    str x30, [sp, #-16]! // 8-byte Folded Spill
 ; V83A-NEXT:    .cfi_def_cfa_offset 16
 ; V83A-NEXT:    .cfi_offset w30, -16
 ; V83A-NEXT:    //APP
 ; V83A-NEXT:    mov x30, x0
 ; V83A-NEXT:    //NO_APP
 ; V83A-NEXT:    ldr x30, [sp], #16 // 8-byte Folded Reload
 ; V83A-NEXT:    autiasp
+; V83A-NEXT:    mov x16, x30
+; V83A-NEXT:    xpaclri
+; V83A-NEXT:    cmp x16, x30
+; V83A-NEXT:    b.ne .LBB9_2
+; V83A-NEXT:  // %bb.1:
 ; V83A-NEXT:    b bar
+; V83A-NEXT:  .LBB9_2:
+; V83A-NEXT:    brk #0xc471
   call void asm sideeffect "mov x30, $0", "r,~{lr}"(i64 %x) #1
   tail call fastcc i64 @bar(i64 %x)
   ret void
 }
 
 define i32 @leaf_sign_all_a_key(i32 %x) "sign-return-address"="all" "sign-return-address-key"="a_key" {
 ; COMPAT-LABEL: leaf_sign_all_a_key:
 ; COMPAT:       // %bb.0: