[AArch64][SVE] Fix handling of stack protection with SVE

[john.brawn]

Summary

Fix a couple of things that were causing stack protection to not work correctly in functions that have scalable vectors on the stack:

• Fix an assertion failure in the StackProtector pass caused by trying to call getTypeAllocSize on a scalable vector.
• When stack protection is enabled move the stack protector location to the top of the SVE locals, so that any overflow in them (or the other locals which are below that) will be detected.

Fixes PR51795.

[sdesmalen]

Thanks for working on this @john.brawn. I've left some suggestions on your patch.

llvm/lib/CodeGen/StackProtector.cpp

355–359

I'd suggest changing the interface HasAddressTaken to take a TypeSize.

llvm/lib/Target/AArch64/AArch64FrameLowering.cpp

110

In your patch you've changed the frame-layout by swapping the areas of fixed-width locals and SVE locals and always requiring/reserving x19 as a base pointer. An alternative idea is to allocate vscale x 16 bytes (i.e. size of one SVE vector) for the stack protector slot, and have that as the first SVE slot after the SVE callee saves:

+-----------------------+
|     Callee Saves      |
+ - - - - - - - - - - - +
|  Frame record (LR/FP) |
+ - - - - - - - - - - - + <- FP
| Neon/SVE callee saves |
+ - - - - - - - - - - - +
|    Stack protector    |
| . . . . . . . . . . . | <- FP - sizeof(Neon/SVE callee saves) - 1*sizeof(SVE vector)
|                       |
| SVE locals/spill-slots|
|                       |
+ - - - - - - - - - - - + 
|                       | 
|        Locals         | 
|                       | 
+------------------------ <- SP/BP

This way, we'd trade a bit of redundant stack space, but we'd avoid having to always reserve x19. We also have the benefit of using the same layout (reducing possibility of bugs being introduced by any new layout which is only enabled through a compile flag). Like with your current patch, we can still access SVE locals directly at <pointer> + <ScalableOffset>, where <pointer> in this case will be FP instead of BP.

I'm not too familiar with the exact requirements for the position of the stack protector slot, but my preliminary understanding is that it must live somewhere below the frame-record and above the locals. The above suggestion would satisfy that requirement.

Can you think of any reason why this might not be feasible?

1126

Could this use hasStackProtectorIndex instead?

[john.brawn]

llvm/lib/Target/AArch64/AArch64FrameLowering.cpp

110

I hadn't though about doing it this way, but it looks like it should be feasible and would definitely simplify things. I'll give it a go and see if I can get it working.

[john.brawn]

Moved the stack protector to above the SVE locals instead of moving SVE locals below the stack protector, and also make use of TypeSize.

[john.brawn]

Ping.

[sdesmalen]

Hi @john.brawn, thanks for updating your patch, this approach seems like a good improvement! Just left a few more comments.

llvm/lib/CodeGen/PrologEpilogInserter.cpp

969 ↗

(On Diff #382687)

nit: unnecessary curly braces

970 ↗

(On Diff #382687)

Do we need to some way to ensure that the LocalStackSlotAllocationPass doesn't pre-allocate the stack protector value if we know it will be allocated to a different StackID?

llvm/lib/CodeGen/StackProtector.cpp

175

This should use TypeSize::isKnownGT(Typesize::getFixed(MemLoc->Size.getValue()), AllocSize).

I doubt it would ever return true when AllocSize is scalable though, becuase from what I can see in the code, MemLoc.hasValue() will return false if I accesses a scalable size, because MemoryLocation will not be precise.

214

If the gep is indexing into a scalable vector type, then OffsetSize cannot be fixed size?

339–340

unnecessary change?

355–360

same here, perhaps just move this into NFC patch and submit it separately.

llvm/lib/Target/AArch64/AArch64FrameLowering.cpp

2984

I think this patch also needs some code to determineSVEStackObjectOffsets to ensure that the stack protector value is ordered before all other objects when allocating their offsets. (this is probably best tested with an MIR test)

llvm/test/CodeGen/AArch64/stack-guard-sve.ll

17

nit: To simplify the test, you could also write:

store <vscale x 4 x float> zeroinitializer, <vscale x 4 x float>* %x, align 16

[john.brawn]

llvm/lib/CodeGen/PrologEpilogInserter.cpp

969 ↗

(On Diff #382687)

That's done for consistency with the preceding if (which has braces because of the nested if/else).

970 ↗

(On Diff #382687)

I'll add an assertion to LocalStackSlotAllocationPass to check for that.

llvm/lib/CodeGen/StackProtector.cpp

214

Indexing into a scalable vector depends only on the size of the element type, which is known.

339–340

A relic of the previous version of this patch, I'll remove.

llvm/lib/Target/AArch64/AArch64FrameLowering.cpp

2984

Will do.

[john.brawn]

Looking into the interaction with LocalStackSlotAllocation it turned out things were more complicated than I thought as it's run before we had decided to put the stack protector in the SVE stack area. I've gone with moving that decision much earlier (instead of attempting to undo what LocalStackSlotAllocation had done).

The getelementptr interaction in HasAddressTaken also wasn't right. Instead of doing "isKnownLT" we have to do "!isKnownGE" as these functions false in the "don't know" case. We also have to be more careful about the subtraction, as you can't subtract a fixed value from a scalable value.

[john.brawn]

Ping.

[sdesmalen]

Thanks for the update, the patch looks structurally good to me. Just left a few nits and a request for an extra test.

llvm/lib/CodeGen/StackProtector.cpp

175

nit: TypeSize::isKnownLT ?

217–218

nit: This can just be:

TypeSize NewAllocSize = TypeSize::Fixed(AllocSize.getKnownMinValue()) - OffsetSize;

Can you also add a test for this case? I guess that would require something like:

%ptr = alloca <vscale x 4 x i32>
%ptr.bc = bitcast <vscale x 4 x i32>* to i32*
%gep = getelementptr i32, i32* %ptr.bc, i64 2 ; no stack protector needed
                                              ; (2*sizeof(i32)) < (vscale x 4 x sizeof(i32))
store i32 %something, i32* %gep
%gep2 = getelementptr i32, i32* %gep, i64 2   ; now a stack protector is needed because
                                              ; it exceeds `sizeof(4 x i32) - 2 x sizeof(i32)`
store i32 %something, i32* %gep2

llvm/lib/Target/AArch64/AArch64ISelLowering.cpp

18431 ↗

(On Diff #388177)

Is it worth adding a FIXME saying that it would be nice to have this in a new interface, something like getStackIDForStackProtectorIndex, rather than doing it in finalizeLowering?

llvm/test/CodeGen/AArch64/stack-guard-reassign-sve.mir

20 ↗

(On Diff #388177)

nit: It would be better to reorder these objects so that the StackGuardSlot is not the first object, so that you can test it still gets allocated at the top of the frame. (because the algorithm processes the objects in order of id)

[john.brawn]

llvm/lib/CodeGen/StackProtector.cpp

175

It needs to be a negated comparison because the TypeSize::isKnownXY functions return false for "I don't know" and in those cases we should assume the access is larger than AllocSize.

[john.brawn]

Adjusted based on review comments.

[sdesmalen]

Thanks, LGTM!

llvm/lib/CodeGen/StackProtector.cpp

bool StackProtector::HasAddressTaken(const Instruction *AI,
                                      uint64_t AllocSize) {
   const DataLayout &DL = M->getDataLayout();
   for (const User *U : AI->users()) {
     const auto *I = cast<Instruction>(U);
     // If this instruction accesses memory make sure it doesn't access beyond
     // the bounds of the allocated object.
     Optional<MemoryLocation> MemLoc = MemoryLocation::getOrNone(I);
     if (MemLoc.hasValue() && MemLoc->Size.hasValue() &&
         MemLoc->Size.getValue() > AllocSize)
       return true;
     switch (I->getOpcode()) {
     case Instruction::Store:
       if (AI == cast<StoreInst>(I)->getValueOperand())
         return true;
       break;
     case Instruction::AtomicCmpXchg:
       // cmpxchg conceptually includes both a load and store from the same
case Instruction::GetElementPtr: {
       // required.
       const GetElementPtrInst *GEP = cast<GetElementPtrInst>(I);
       unsigned TypeSize = DL.getIndexTypeSizeInBits(I->getType());
       APInt Offset(TypeSize, 0);
       APInt MaxOffset(TypeSize, AllocSize);
       if (!GEP->accumulateConstantOffset(DL, Offset) || Offset.ugt(MaxOffset))
         return true;
       // Adjust AllocSize to be the space remaining after this offset.
       if (HasAddressTaken(I, AllocSize - Offset.getLimitedValue()))
         return true;
       break;
     }
     case Instruction::BitCast:
     case Instruction::Select:
     case Instruction::AddrSpaceCast:
       if (HasAddressTaken(I, AllocSize))
         return true;
       break;
     case Instruction::PHI: {
       // Keep track of what PHI nodes we have already visited to ensure
       // they are only visited once.
for (const Instruction &I : BB) {
                                          MachineFrameInfo::SSPLK_LargeArray));
             ORE.emit(RemarkBuilder);
             NeedsProtector = true;
           }
           continue;
         }
 
         bool IsLarge = false;
-        if (ContainsProtectableArray(AI->getAllocatedType(), IsLarge, Strong)) {
+        Type *AllocType = AI->getAllocatedType();
+        if (ContainsProtectableArray(AllocType, IsLarge, Strong)) {
           Layout.insert(std::make_pair(AI, IsLarge
                                        ? MachineFrameInfo::SSPLK_LargeArray
                                        : MachineFrameInfo::SSPLK_SmallArray));
           ORE.emit([&]() {
             return OptimizationRemark(DEBUG_TYPE, "StackProtectorBuffer", &I)
                    << "Stack protection applied to function "
                    << ore::NV("Function", F)
                    << " due to a stack allocated buffer or struct containing a "
                       "buffer";
           });
           NeedsProtector = true;
           continue;
         }
 
-        if (Strong && HasAddressTaken(AI, M->getDataLayout().getTypeAllocSize(
-                                              AI->getAllocatedType()))) {
+        // The exact size of scalable vectors isn't known, so go for the worst
+        // case of a single element.
+        bool isScalable = isa<ScalableVectorType>(AllocType);
+        uint64_t AllocSize = M->getDataLayout().getTypeAllocSize(
+          isScalable ? AllocType->getScalarType() : AllocType);
+        if (Strong && HasAddressTaken(AI, AllocSize)) {
           ++NumAddrTaken;
           Layout.insert(std::make_pair(AI, MachineFrameInfo::SSPLK_AddrOf));
           ORE.emit([&]() {
             return OptimizationRemark(DEBUG_TYPE, "StackProtectorAddressTaken",
                                       &I)
                    << "Stack protection applied to function "
                    << ore::NV("Function", F)
                    << " due to the address of a local variable being taken";

llvm/lib/Target/AArch64/AArch64FrameLowering.cpp

 // The SVE spill/fill instructions have VL-scaled addressing modes such
 // as:
 //    ldr z8, [fp, #-7 mul vl]
 // For SVE the size of the vector length (VL) is not known at compile-time, so
 // '#-7 mul vl' is an offset that can only be evaluated at runtime. With this
 // layout, we don't need to add an unscaled offset to the framepointer before
 // accessing the SVE object in the frame.
 //
+// When stack protection is enabled we need to place SVE stack objects below the
+// the local variables of fixed size, as that's where the stack guard variable
+// will be placed. Therefore they are moved to below the base pointer, as that
+// way we can use VL-scaled addressing below bp to access them.
+//
 // In some cases when a base pointer is not strictly needed, it is generated
 // anyway when offsets from the frame pointer to access local variables become
 // so large that the offset can't be encoded in the immediate fields of loads
 // or stores.
 //
 // Outgoing function arguments must be at the bottom of the stack frame when
 // calling another function. If we do not have variable-sized stack objects, we
 // can allocate a "reserved call frame" area at the bottom of the local
bool needsFrameMoves =
       MF.needsFrameMoves() && !MF.getTarget().getMCAsmInfo()->usesWindowsCFI();
   bool HasFP = hasFP(MF);
   bool NeedsWinCFI = needsWinCFI(MF);
   bool HasWinCFI = false;
   auto Cleanup = make_scope_exit([&]() { MF.setHasWinCFI(HasWinCFI); });
 
   bool IsFunclet = MBB.isEHFuncletEntry();
 
+  bool SVEBelowBP = MFI.getStackProtectorIndex() >= 0;
+
   // At this point, we're going to decide whether or not the function uses a
   // redzone. In most cases, the function doesn't have a redzone so let's
   // assume that's false and set it to true in the case that there's a redzone.
   AFI->setHasRedZone(false);
 
   // Debug location must be unknown since the first debug location is used
   // to determine the end of the prologue.
   DebugLoc DL;
if (NeedsWinCFI) {
       HasWinCFI = true;
       BuildMI(MBB, MBBI, DL, TII->get(AArch64::SEH_StackAlloc))
           .addImm(NumBytes)
           .setMIFlag(MachineInstr::FrameSetup);
     }
     NumBytes = 0;
   }
 
-  StackOffset AllocateBefore = SVEStackSize, AllocateAfter = {};
+  StackOffset AllocateBefore = {}, AllocateAfter = SVEStackSize;
   MachineBasicBlock::iterator CalleeSavesBegin = MBBI, CalleeSavesEnd = MBBI;
 
   // Process the SVE callee-saves to determine what space needs to be
   // allocated.
   if (int64_t CalleeSavedSize = AFI->getSVECalleeSavedStackSize()) {
     // Find callee save instructions in frame.
     CalleeSavesBegin = MBBI;
     assert(IsSVECalleeSave(CalleeSavesBegin) && "Unexpected instruction");
     while (IsSVECalleeSave(MBBI) && MBBI != MBB.getFirstTerminator())
       ++MBBI;
     CalleeSavesEnd = MBBI;
 
     AllocateBefore = StackOffset::getScalable(CalleeSavedSize);
     AllocateAfter = SVEStackSize - AllocateBefore;
   }
 
   // Allocate space for the callee saves (if any).
   emitFrameOffset(MBB, CalleeSavesBegin, DL, AArch64::SP, AArch64::SP,
                   -AllocateBefore, TII,
                   MachineInstr::FrameSetup);
 
+  if (!SVEBelowBP) {
     // Finally allocate remaining SVE stack space.
     emitFrameOffset(MBB, CalleeSavesEnd, DL, AArch64::SP, AArch64::SP,
                     -AllocateAfter, TII,
                     MachineInstr::FrameSetup);
+  }
 
   // Allocate space for the rest of the frame.
   if (NumBytes) {
     // Alignment is required for the parent frame, not the funclet
     const bool NeedsRealignment =
         !IsFunclet && RegInfo->hasStackRealignment(MF);
     unsigned scratchSPReg = AArch64::SP;
 
TII->copyPhysReg(MBB, MBBI, DL, RegInfo->getBaseRegister(), AArch64::SP,
                      false);
     if (NeedsWinCFI) {
       HasWinCFI = true;
       BuildMI(MBB, MBBI, DL, TII->get(AArch64::SEH_Nop))
           .setMIFlag(MachineInstr::FrameSetup);
     }
   }
 
+  if (SVEBelowBP) {
+    // SVE locals are placed below the base pointer.
+    emitFrameOffset(MBB, MBBI, DL, AArch64::SP, AArch64::SP,
+                    -AllocateAfter, TII,
+                    MachineInstr::FrameSetup);
+  }
+
   // The very last FrameSetup instruction indicates the end of prologue. Emit a
   // SEH opcode indicating the prologue end.
   if (NeedsWinCFI && HasWinCFI) {
     BuildMI(MBB, MBBI, DL, TII->get(AArch64::SEH_PrologEnd))
         .setMIFlag(MachineInstr::FrameSetup);
   }
 
   // SEH funclets are passed the frame pointer in X1.  If the parent
StackOffset AArch64FrameLowering::resolveFrameOffsetReference(
   }
 
   assert(
       ((isFixed || isCSR) || !RegInfo->hasStackRealignment(MF) || !UseFP) &&
       "In the presence of dynamic stack pointer realignment, "
       "non-argument/CSR objects cannot be accessed through the frame pointer");
 
   if (isSVE) {
+    if (MFI.getStackProtectorIndex() >= 0) {
+      // SVE locals, but not callee-saved registers, are placed below the base
+      // pointer, and the callee-saved registers shouldn't be accessed.
+      // Therefore the offset is reduced by the callee-saved size, though it's
+      // done as an addition as ObjectOffset is a negative offset.
+      int64_t CalleeSavedSize = AFI->getSVECalleeSavedStackSize();
+      assert(-ObjectOffset >= CalleeSavedSize &&
+             "Unexpected access of callee-saved SVE register");
+      assert(RegInfo->hasBasePointer(MF) &&
+             "Base pointer expected to be available in functions with SVE "
+             "locals and stack protection");
+      FrameReg = RegInfo->getBaseRegister();
+      return StackOffset::getScalable(ObjectOffset + CalleeSavedSize);
+    }
+
     StackOffset FPOffset =
         StackOffset::get(-AFI->getCalleeSaveBaseToFrameRecordOffset(), ObjectOffset);
     StackOffset SPOffset =
         SVEStackSize +
         StackOffset::get(MFI.getStackSize() - AFI->getCalleeSavedStackSize(),
                          ObjectOffset);
     // Always use the FP for SVE spills if available and beneficial.
     if (hasFP(MF) && (SPOffset.getFixed() ||
bool AArch64FrameLowering::assignCalleeSavedSpillSlots(
   auto *AFI = MF.getInfo<AArch64FunctionInfo>();
   for (auto &CS : CSI) {
     Register Reg = CS.getReg();
     const TargetRegisterClass *RC = RegInfo->getMinimalPhysRegClass(Reg);
 
     unsigned Size = RegInfo->getSpillSize(*RC);
     Align Alignment(RegInfo->getSpillAlign(*RC));
     int FrameIdx = MFI.CreateStackObject(Size, Alignment, true);
     CS.setFrameIdx(FrameIdx);
 
     if ((unsigned)FrameIdx < MinCSFrameIndex) MinCSFrameIndex = FrameIdx;
     if ((unsigned)FrameIdx > MaxCSFrameIndex) MaxCSFrameIndex = FrameIdx;
 
     // Grab 8 bytes below FP for the extended asynchronous frame info.
     if (hasFP(MF) && AFI->hasSwiftAsyncContext() && Reg == AArch64::FP) {
       FrameIdx = MFI.CreateStackObject(8, Alignment, true);
       AFI->setSwiftAsyncContextFrameIdx(FrameIdx);

llvm/lib/Target/AArch64/AArch64RegisterInfo.cpp

AArch64RegisterInfo::getCrossCopyRegClass(const TargetRegisterClass *RC) const {
   return RC;
 }
 
 unsigned AArch64RegisterInfo::getBaseRegister() const { return AArch64::X19; }
 
 bool AArch64RegisterInfo::hasBasePointer(const MachineFunction &MF) const {
   const MachineFrameInfo &MFI = MF.getFrameInfo();
 
+  // When stack protection is used SVE locals are placed below the base pointer.
+  if (MF.getInfo<AArch64FunctionInfo>()->getStackSizeSVE() > 0 &&
+      MFI.getStackProtectorIndex() >= 0) {
+    return true;
+  }
+
   // In the presence of variable sized objects or funclets, if the fixed stack
   // size is large enough that referencing from the FP won't result in things
   // being in range relatively often, we can use a base pointer to allow access
   // from the other direction like the SP normally works.
   //
   // Furthermore, if both variable sized objects are present, and the
   // stack needs to be dynamically re-aligned, the base pointer is the only
   // reliable way to reference the locals.

llvm/test/CodeGen/AArch64/stack-guard-sve.ll

+; RUN: llc -mtriple=aarch64--linux-gnu -mattr=+sve < %s | FileCheck %s
+
+declare dso_local void @val_fn(<vscale x 4 x float>)
+declare dso_local void @ptr_fn(<vscale x 4 x float>*)
+
+; An alloca of a scalable vector shouldn't trigger stack protection.
+
+; CHECK-LABEL: call_value:
+; CHECK-NOT: mov x19, sp
+; CHECK: addvl sp, sp, #-1
+; CHECK-NOT: __stack_chk_guard
+; CHECK: st1w { {{z[0-9]+.s}} }, {{p[0-9]+}}, [x29, #-1, mul vl]
+define void @call_value() #0 {
+entry:
+  %x = alloca <vscale x 4 x float>, align 16
+  store <vscale x 4 x float> shufflevector (<vscale x 4 x float> insertelement (<vscale x 4 x float> poison, float 0.000000e+00, i32 0), <vscale x 4 x float> poison, <vscale x 4 x i32> zeroinitializer), <vscale x 4 x float>* %x, align 16
+  %0 = load <vscale x 4 x float>, <vscale x 4 x float>* %x, align 16
+  call void @val_fn(<vscale x 4 x float> %0)
+  ret void
+}
+
+; CHECK-LABEL: call_value_strong:
+; CHECK-NOT: mov x19, sp
+; CHECK: addvl sp, sp, #-1
+; CHECK-NOT: __stack_chk_guard
+; CHECK: st1w { {{z[0-9]+.s}} }, {{p[0-9]+}}, [x29, #-1, mul vl]
+define void @call_value_strong() #1 {
+entry:
+  %x = alloca <vscale x 4 x float>, align 16
+  store <vscale x 4 x float> shufflevector (<vscale x 4 x float> insertelement (<vscale x 4 x float> poison, float 0.000000e+00, i32 0), <vscale x 4 x float> poison, <vscale x 4 x i32> zeroinitializer), <vscale x 4 x float>* %x, align 16
+  %0 = load <vscale x 4 x float>, <vscale x 4 x float>* %x, align 16
+  call void @val_fn(<vscale x 4 x float> %0)
+  ret void
+}
+
+; Address-taking of a scalable vector should trigger stack protection only with
+; sspstrong, and the scalable vector should be be placed below the stack guard.
+
+; CHECK-LABEL: call_ptr:
+; CHECK-NOT: mov x19, sp
+; CHECK: addvl sp, sp, #-1
+; CHECK-NOT: __stack_chk_guard
+; CHECK: addvl x0, x29, #-1
+; CHECK: bl ptr_fn
+define void @call_ptr() #0 {
+entry:
+  %x = alloca <vscale x 4 x float>, align 16
+  call void @ptr_fn(<vscale x 4 x float>* %x)
+  ret void
+}
+
+; CHECK-LABEL: call_ptr_strong:
+; CHECK: sub sp, sp, #16
+; CHECK: mov x19, sp
+; CHECK: addvl sp, sp, #-1
+; CHECK-DAG: ldr [[REG:x[0-9]+]], [{{x[0-9]+}}, :lo12:__stack_chk_guard]
+; CHECK-DAG: str [[REG]], [x19, #8]
+; CHECK-DAG: addvl x0, x19, #-1
+; CHECK: bl ptr_fn
+define void @call_ptr_strong() #1 {
+entry:
+  %x = alloca <vscale x 4 x float>, align 16
+  call void @ptr_fn(<vscale x 4 x float>* %x)
+  ret void
+}
+
+; Check that both variables are addressed in the same way (sp-relative with ssp,
+; bp-relative with sspstrong)
+
+; CHECK-LABEL: call_both:
+; CHECK-NOT: mov x19, sp
+; CHECK: addvl sp, sp, #-2
+; CHECK-NOT: __stack_chk_guard
+; CHECK: st1w { {{z[0-9]+.s}} }, {{p[0-9]+}}, [x29, #-1, mul vl]
+; CHECK: bl val_fn
+; CHECK: addvl x0, x29, #-2
+; CHECK: bl ptr_fn
+define void @call_both() #0 {
+entry:
+  %x = alloca <vscale x 4 x float>, align 16
+  %y = alloca <vscale x 4 x float>, align 16
+  store <vscale x 4 x float> shufflevector (<vscale x 4 x float> insertelement (<vscale x 4 x float> poison, float 0.000000e+00, i32 0), <vscale x 4 x float> poison, <vscale x 4 x i32> zeroinitializer), <vscale x 4 x float>* %x, align 16
+  %0 = load <vscale x 4 x float>, <vscale x 4 x float>* %x, align 16
+  call void @val_fn(<vscale x 4 x float> %0)
+  call void @ptr_fn(<vscale x 4 x float>* %y)
+  ret void
+}
+
+; CHECK-LABEL: call_both_strong:
+; CHECK: sub sp, sp, #16
+; CHECK: mov x19, sp
+; CHECK: addvl sp, sp, #-2
+; CHECK-DAG: ldr [[REG:x[0-9]+]], [{{x[0-9]+}}, :lo12:__stack_chk_guard]
+; CHECK-DAG: str [[REG]], [x19, #8]
+; CHECK-DAG: st1w { {{z[0-9]+.s}} }, {{p[0-9]+}}, [x19, #-1, mul vl]
+; CHECK: bl val_fn
+; CHECK: addvl x0, x19, #-2
+; CHECK: bl ptr_fn
+define void @call_both_strong() #1 {
+entry:
+  %x = alloca <vscale x 4 x float>, align 16
+  %y = alloca <vscale x 4 x float>, align 16
+  store <vscale x 4 x float> shufflevector (<vscale x 4 x float> insertelement (<vscale x 4 x float> poison, float 0.000000e+00, i32 0), <vscale x 4 x float> poison, <vscale x 4 x i32> zeroinitializer), <vscale x 4 x float>* %x, align 16
+  %0 = load <vscale x 4 x float>, <vscale x 4 x float>* %x, align 16
+  call void @val_fn(<vscale x 4 x float> %0)
+  call void @ptr_fn(<vscale x 4 x float>* %y)
+  ret void
+}
+
+; Pushed callee-saved regs should be above the stack guard
+
+; CHECK-LABEL: callee_save:
+; CHECK: mov x29, sp
+; CHECK: addvl sp, sp, #-18
+; CHECK: str {{z[0-9]+}}, [sp, #{{[0-9]+}}, mul vl]
+; CHECK-NOT: mov x19, sp
+; CHECK: addvl sp, sp, #-1
+; CHECK-NOT: __stack_chk_guard
+; CHECK: addvl [[REG:x[0-9]+]], x29, #-11
+; CHECK: st1w { {{z[0-9]+.s}} }, {{p[0-9]+}}, {{\[}}[[REG]], #-8, mul vl]
+define void @callee_save(<vscale x 4 x float> %x) #0 {
+entry:
+  %x.addr = alloca <vscale x 4 x float>, align 16
+  store <vscale x 4 x float> %x, <vscale x 4 x float>* %x.addr, align 16
+  call void @ptr_fn(<vscale x 4 x float>* %x.addr)
+  ret void
+}
+
+; CHECK-LABEL: callee_save_strong:
+; CHECK: mov x29, sp
+; CHECK: addvl sp, sp, #-18
+; CHECK: str {{z[0-9]+}}, [sp, #{{[0-9]+}}, mul vl]
+; CHECK: sub sp, sp, #16
+; CHECK: mov x19, sp
+; CHECK: addvl sp, sp, #-1
+; CHECK-DAG: ldr [[REG:x[0-9]+]], [{{x[0-9]+}}, :lo12:__stack_chk_guard]
+; CHECK-DAG: str [[REG]], [x19, #8]
+; CHECK-DAG: st1w { {{z[0-9]+.s}} }, {{p[0-9]+}}, [x19, #-1, mul vl]
+define void @callee_save_strong(<vscale x 4 x float> %x) #1 {
+entry:
+  %x.addr = alloca <vscale x 4 x float>, align 16
+  store <vscale x 4 x float> %x, <vscale x 4 x float>* %x.addr, align 16
+  call void @ptr_fn(<vscale x 4 x float>* %x.addr)
+  ret void
+}
+
+attributes #0 = { ssp "frame-pointer"="non-leaf" }
+attributes #1 = { sspstrong "frame-pointer"="non-leaf" }