[COFF, ARM64] Fix localaddress to handle stack realignment and variable size objects

[mgrang]

Summary

This fixes using the correct stack registers for SEH when stack realignment is needed or when variable size objects are present.

[efriedma]

lib/Target/AArch64/AArch64ISelLowering.cpp

2750 ↗

(On Diff #183381)

This could use a brief comment to explain why stack realignment matters.

[rnk]

Why isn't SP used in this case? I'd expect that the frame pointer addresses parameters, followed by the alignment gap, followed by the locals area, which is addressable with fixed offsets from SP.

[efriedma]

If there's a call to localaddress in a function without funclets or VLAs, we should use sp, yes. That should be rare in practice, but I guess the testcase is an example.

[mgrang]

In D57183#1370366, @efriedma wrote:

If there's a call to localaddress in a function without funclets or VLAs, we should use sp, yes. That should be rare in practice, but I guess the testcase is an example.

So I guess the logic should be something like this:

if (!hasVarSizedObjects && !hasFunclets) --> use SP
else if (needsStackRealignment) --> use BP
else --> use FP

[efriedma]

Yes, that looks right.

[rnk]

In D57183#1370450, @mgrang wrote:

In D57183#1370366, @efriedma wrote:

If there's a call to localaddress in a function without funclets or VLAs, we should use sp, yes. That should be rare in practice, but I guess the testcase is an example.

So I guess the logic should be something like this:

if (!hasVarSizedObjects && !hasFunclets) --> use SP
else if (needsStackRealignment) --> use BP
else --> use FP

Is that not what the existing code does? RegInfo->getFrameRegister(MF) does this to choose between SP and FP:

unsigned
AArch64RegisterInfo::getFrameRegister(const MachineFunction &MF) const {
  const AArch64FrameLowering *TFI = getFrameLowering(MF);
  return TFI->hasFP(MF) ? AArch64::FP : AArch64::SP;
}

And hasBasePointer checks the same conditions you've listed here.

...

I see, hasFP returns true when there are calls and the stack frame is large, and also in this case:

// Win64 SEH requires frame pointer if funclets are present.
if (MF.hasLocalEscape())
  return true;

Are you sure we still want that there? It sounds like we are trying to allow addressing variables with SP when localescape is present.

[efriedma]

hasBasePointer checks the same conditions

We sometimes emit a base pointer or a frame pointer when it isn't strictly necessary. We don't want to change the result of llvm.localaddress() in that case.

---

The MF.hasLocalEscape() check in hasFP() probably isn't necessary.

[mgrang]

In the existing code, there are several conditions on when FP/BP/SP should be used. I have tried to summarize them here:

use BP:
  if ((hasVarSizedObjects || hasEHFunclets)) && (needsStackRealignment || LocalFrameSize >= 256))

use FP:	
  if (hasEHFunclets || hasVarSizedObjects || needsStackRealignment || hasLocalEscape ||
      hasCalls || isFrameAddressTaken || hasStackMap || hasPatchPoint || !MaxCallFrameSizeComputed ||
      MaxCallFrameSize > DefaultSafeSPDisplacement)

else use SP

Here's my understanding of how the locals should be accessed in various scenarios:

struct S { int x; };

// Use FP to access escaped locals: (hasFunclets = true, hasVarSizedObjects = false, needsStackRealignment = false)
void simple() {
  struct S o;

  __try { o.x; }
  __finally { o.x; }
}

// Use BP to access escaped locals: (hasFunclets = true, hasVarSizedObjects = false, needsStackRealignment = true)
void stack_realignment() {
  struct S __declspec(align(32)) o;

  __try { o.x; }
  __finally { o.x; }
}

// Use BP to access escaped locals: (hasFunclets = true, hasVarSizedObjects = true, needsStackRealignment = false)
void vla_present(int n) {
  int vla[n];

  __try { vla[0]; }
  __finally { vla[0]; }
}

// Use BP to access escaped locals: (hasFunclets = true, hasVarSizedObjects = true, needsStackRealignment = true)
void all(int n) {
  struct S __declspec(align(32)) o;
  int vla[n];
  
  __try { o.x; vla[0]; }
  __finally { o.x; vla[0]; }
}

// Use SP to access locals: (hasFunclets = false, hasVarSizedObjects = false, needsStackRealignment = false)
void non_seh() {
  // call to llvm.localaddress();
}

@rnk @efriedma Could you please comment on if all the scenarios have been captured here and if the behavior is what is expected?

[rnk]

use FP:

if (hasEHFunclets || hasVarSizedObjects || needsStackRealignment || hasLocalEscape ||
    hasCalls || isFrameAddressTaken || hasStackMap || hasPatchPoint || !MaxCallFrameSizeComputed ||
    MaxCallFrameSize > DefaultSafeSPDisplacement)

Should FP be used if needsStackRealignment is true and none of the other conditions are true? I would've expected that SP needs to be used, because FP points to the parameter space before the stack was realigned.

I suppose that it is correct to force a frame to use FP when hasLocalEscape is true, assuming we've already checked the conditions under which BP is needed.

[efriedma]

// Use BP to access escaped locals: (hasFunclets = true, hasVarSizedObjects = true, needsStackRealignment = false)
void vla_present(int n) {

Should still use fp here. (Having both VLAs and funclets isn't really any different from having only one of them.)

[mgrang]

Deleted test/CodeGen/AArch64/seh-localescape.ll as localescape testing is better covered under the updated test/CodeGen/AArch64/seh-finally.ll.

[mgrang]

With this patch, most of the SEH tests in https://github.com/Microsoft/windows_seh_tests/blob/master/src/xcpt4/xcpt4u.c pass.

[TomTan]

In D57183#1378121, @mgrang wrote:

With this patch, most of the SEH tests in https://github.com/Microsoft/windows_seh_tests/blob/master/src/xcpt4/xcpt4u.c pass.

Great to have this. Thanks @mgrang. Curious on what remaining tests fail in xcpt4u.c?

[efriedma]

LGTM. I think this is the correct model for representing frame offsets for localescape/localrecover.

(The code to handle eh_recoverfp correctly is still missing, but I think it's okay to handle that in a separate patch.)

Please give Reid a few days to comment before you merge, though.

[rnk]

I see, we need a separate register selection codepath that ignores frame size considerations. (I could be wrong, I haven't dug that deep, though.)

I think the functionality is good, but I keep insisting that we name these routines after the localescape / localrecover intrinsic set, since they are ostensibly for lambdas as well as SEH. :) Feel free to use your judgement, don't want for me to review it.

include/llvm/CodeGen/TargetFrameLowering.h

267 ↗

(On Diff #184403)

Let's call this getNonLocalFrameIndexReference. In theory, this should have nothing to do with EH. llvm.localescape is a separate LLVM IR feature that happens to support the frontend-outlined try / except / __finally funclets.

lib/Target/AArch64/AArch64RegisterInfo.h

125 ↗

(On Diff #184403)

Similarly, perhaps this should be getLocalAddressRegister so it's associated with the intrinsics.

[mgrang]

In D57183#1378139, @TomTan wrote:

In D57183#1378121, @mgrang wrote:

With this patch, most of the SEH tests in https://github.com/Microsoft/windows_seh_tests/blob/master/src/xcpt4/xcpt4u.c pass.

Great to have this. Thanks @mgrang. Curious on what remaining tests fail in xcpt4u.c?

@TomTan Tests with features which have not yet been implemented (like __leave and asynchronous SEH) are failing.

llvm/trunk/include/llvm/CodeGen/MachineFunction.h

class MachineFunction {
 
   /// CodeView label annotations.
   std::vector<std::pair<MCSymbol *, MDNode *>> CodeViewAnnotations;
 
   bool CallsEHReturn = false;
   bool CallsUnwindInit = false;
   bool HasEHScopes = false;
   bool HasEHFunclets = false;
-  bool HasLocalEscape = false;
 
   /// List of C++ TypeInfo used.
   std::vector<const GlobalValue *> TypeInfos;
 
   /// List of typeids encoding filters used.
   std::vector<unsigned> FilterIds;
 
   /// List of the indices in FilterIds corresponding to filter terminators.
public:
   void setCallsUnwindInit(bool b) { CallsUnwindInit = b; }
 
   bool hasEHScopes() const { return HasEHScopes; }
   void setHasEHScopes(bool V) { HasEHScopes = V; }
 
   bool hasEHFunclets() const { return HasEHFunclets; }
   void setHasEHFunclets(bool V) { HasEHFunclets = V; }
 
-  bool hasLocalEscape() const { return HasLocalEscape; }
-  void setHasLocalEscape(bool V) { HasLocalEscape = V; }
-
   /// Find or create an LandingPadInfo for the specified MachineBasicBlock.
   LandingPadInfo &getOrCreateLandingPadInfo(MachineBasicBlock *LandingPad);
 
   /// Remap landing pad labels and remove any deleted landing pads.
   void tidyLandingPads(DenseMap<MCSymbol *, uintptr_t> *LPMap = nullptr,
                        bool TidyIfNoBeginLabels = true);
 
   /// Return a reference to the landing pad info for the current function.

llvm/trunk/include/llvm/CodeGen/TargetFrameLowering.h

public:
   /// the end of the prologue.
   virtual int getFrameIndexReferencePreferSP(const MachineFunction &MF, int FI,
                                              unsigned &FrameReg,
                                              bool IgnoreSPUpdates) const {
     // Always safe to dispatch to getFrameIndexReference.
     return getFrameIndexReference(MF, FI, FrameReg);
   }
 
+  /// getNonLocalFrameIndexReference - This method returns the offset used to
+  /// reference a frame index location. The offset can be from either FP/BP/SP
+  /// based on which base register is returned by llvm.localaddress.
+  virtual int getNonLocalFrameIndexReference(const MachineFunction &MF,
+                                       int FI) const {
+    // By default, dispatch to getFrameIndexReference. Interested targets can
+    // override this.
+    unsigned FrameReg;
+    return getFrameIndexReference(MF, FI, FrameReg);
+  }
+
   /// This method determines which of the registers reported by
   /// TargetRegisterInfo::getCalleeSavedRegs() should actually get saved.
   /// The default implementation checks populates the \p SavedRegs bitset with
   /// all registers which are modified in the function, targets may override
   /// this function to save additional registers.
   /// This method also sets up the register scavenger ensuring there is a free
   /// register or a frameindex available.
   virtual void determineCalleeSaves(MachineFunction &MF, BitVector &SavedRegs,

llvm/trunk/lib/CodeGen/AsmPrinter/WinException.cpp

void WinException::emitEHRegistrationOffsetLabel(const WinEHFuncInfo &FuncInfo,
   // Compute the parent frame offset. The EHRegNodeFrameIndex will be invalid if
   // after optimization all the invokes were eliminated. We still need to emit
   // the parent frame offset label, but it should be garbage and should never be
   // used.
   int64_t Offset = 0;
   int FI = FuncInfo.EHRegNodeFrameIndex;
   if (FI != INT_MAX) {
     const TargetFrameLowering *TFI = Asm->MF->getSubtarget().getFrameLowering();
-    unsigned UnusedReg;
-    // FIXME: getFrameIndexReference needs to match the behavior of
-    // AArch64RegisterInfo::hasBasePointer in which one of the scenarios where
-    // SP is used is if frame size >= 256.
-    Offset = TFI->getFrameIndexReference(*Asm->MF, FI, UnusedReg);
+    Offset = TFI->getNonLocalFrameIndexReference(*Asm->MF, FI);
   }
 
   MCContext &Ctx = Asm->OutContext;
   MCSymbol *ParentFrameOffset =
       Ctx.getOrCreateParentFrameOffsetSymbol(FLinkageName);
   Asm->OutStreamer->EmitAssignment(ParentFrameOffset,
                                    MCConstantExpr::create(Offset, Ctx));
 }

llvm/trunk/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

for (unsigned Idx = 0, E = I.getNumArgOperands(); Idx < E; ++Idx) {
           MF.getMMI().getContext().getOrCreateFrameAllocSymbol(
               GlobalValue::dropLLVMManglingEscape(MF.getName()), Idx);
       BuildMI(*FuncInfo.MBB, FuncInfo.InsertPt, dl,
               TII->get(TargetOpcode::LOCAL_ESCAPE))
           .addSym(FrameAllocSym)
           .addFrameIndex(FI);
     }
 
-    MF.setHasLocalEscape(true);
-
     return nullptr;
   }
 
   case Intrinsic::localrecover: {
     // i8* @llvm.localrecover(i8* %fn, i8* %fp, i32 %idx)
     MachineFunction &MF = DAG.getMachineFunction();
     MVT PtrVT = TLI.getPointerTy(DAG.getDataLayout(), 0);
 

llvm/trunk/lib/Target/AArch64/AArch64FrameLowering.h

public:
 
   unsigned getWinEHParentFrameOffset(const MachineFunction &MF) const override;
 
   unsigned getWinEHFuncletFrameSize(const MachineFunction &MF) const;
 
   int getFrameIndexReferencePreferSP(const MachineFunction &MF, int FI,
                                      unsigned &FrameReg,
                                      bool IgnoreSPUpdates) const override;
+  int getNonLocalFrameIndexReference(const MachineFunction &MF,
+                               int FI) const override;
+  int getSEHFrameIndexOffset(const MachineFunction &MF, int FI) const;
 
 private:
   bool shouldCombineCSRLocalStackBump(MachineFunction &MF,
                                       unsigned StackBumpBytes) const;
 };
 
 } // End llvm namespace
 
 #endif

llvm/trunk/lib/Target/AArch64/AArch64FrameLowering.cpp

bool AArch64FrameLowering::hasFP(const MachineFunction &MF) const {
   // getReservedReg() -> hasFP in the middle of global isel are too early
   // to know the max call frame size. Hopefully conservatively returning "true"
   // in those cases is fine.
   // DefaultSafeSPDisplacement is fine as we only emergency spill GP regs.
   if (!MFI.isMaxCallFrameSizeComputed() ||
       MFI.getMaxCallFrameSize() > DefaultSafeSPDisplacement)
     return true;
 
-  // Win64 SEH requires frame pointer if funclets are present.
-  if (MF.hasLocalEscape())
-    return true;
-
   return false;
 }
 
 /// hasReservedCallFrame - Under normal circumstances, when a frame pointer is
 /// not required, we reserve argument space for call sites in the function
 /// immediately on entry to the current function.  This eliminates the need for
 /// add/sub sp brackets around call sites.  Returns true if the call frame is
 /// included as part of the stack frame.

 /// references for now.  FIXME: This can go wrong when references are
 /// SP-relative and simple call frames aren't used.
 int AArch64FrameLowering::getFrameIndexReference(const MachineFunction &MF,
                                                  int FI,
                                                  unsigned &FrameReg) const {
   return resolveFrameIndexReference(MF, FI, FrameReg);
 }
 
+int AArch64FrameLowering::getNonLocalFrameIndexReference(
+  const MachineFunction &MF, int FI) const {
+  return getSEHFrameIndexOffset(MF, FI);
+}
+
+static int getFPOffset(const MachineFunction &MF, int FI) {
+  const auto &MFI = MF.getFrameInfo();
+  const auto *AFI = MF.getInfo<AArch64FunctionInfo>();
+  const auto &Subtarget = MF.getSubtarget<AArch64Subtarget>();
+  bool IsWin64 =
+      Subtarget.isCallingConvWin64(MF.getFunction().getCallingConv());
+  unsigned FixedObject = IsWin64 ? alignTo(AFI->getVarArgsGPRSize(), 16) : 0;
+  return MFI.getObjectOffset(FI) + FixedObject + 16;
+}
+
+static int getStackOffset(const MachineFunction &MF, int FI) {
+  const auto &MFI = MF.getFrameInfo();
+  return MFI.getObjectOffset(FI) + MFI.getStackSize();
+}
+
+int AArch64FrameLowering::getSEHFrameIndexOffset(const MachineFunction &MF,
+                                                 int FI) const {
+  const auto *RegInfo = static_cast<const AArch64RegisterInfo *>(
+      MF.getSubtarget().getRegisterInfo());
+  return RegInfo->getLocalAddressRegister(MF) == AArch64::FP ?
+         getFPOffset(MF, FI) : getStackOffset(MF, FI);
+}
+
 int AArch64FrameLowering::resolveFrameIndexReference(const MachineFunction &MF,
                                                      int FI, unsigned &FrameReg,
                                                      bool PreferFP) const {
-  const MachineFrameInfo &MFI = MF.getFrameInfo();
-  const AArch64RegisterInfo *RegInfo = static_cast<const AArch64RegisterInfo *>(
+  const auto &MFI = MF.getFrameInfo();
+  const auto *RegInfo = static_cast<const AArch64RegisterInfo *>(
       MF.getSubtarget().getRegisterInfo());
-  const AArch64FunctionInfo *AFI = MF.getInfo<AArch64FunctionInfo>();
-  const AArch64Subtarget &Subtarget = MF.getSubtarget<AArch64Subtarget>();
-  bool IsWin64 =
-      Subtarget.isCallingConvWin64(MF.getFunction().getCallingConv());
-  unsigned FixedObject = IsWin64 ? alignTo(AFI->getVarArgsGPRSize(), 16) : 0;
-  int FPOffset = MFI.getObjectOffset(FI) + FixedObject + 16;
-  int Offset = MFI.getObjectOffset(FI) + MFI.getStackSize();
+  const auto *AFI = MF.getInfo<AArch64FunctionInfo>();
+  const auto &Subtarget = MF.getSubtarget<AArch64Subtarget>();
+  int FPOffset = getFPOffset(MF, FI);
+  int Offset = getStackOffset(MF, FI);
   bool isFixed = MFI.isFixedObjectIndex(FI);
   bool isCSR = !isFixed && MFI.getObjectOffset(FI) >=
                                -((int)AFI->getCalleeSavedStackSize());
 
   // Use frame pointer to reference fixed objects. Use it for locals if
   // there are VLAs or a dynamically realigned SP (and thus the SP isn't
   // reliable as a base). Make sure useFPForScavengingIndex() does the
   // right thing for the emergency spill slot.

llvm/trunk/lib/Target/AArch64/AArch64ISelLowering.cpp

SDValue AArch64TargetLowering::LowerINTRINSIC_WO_CHAIN(SDValue Op,
   case Intrinsic::aarch64_neon_smin:
     return DAG.getNode(ISD::SMIN, dl, Op.getValueType(),
                        Op.getOperand(1), Op.getOperand(2));
   case Intrinsic::aarch64_neon_umin:
     return DAG.getNode(ISD::UMIN, dl, Op.getValueType(),
                        Op.getOperand(1), Op.getOperand(2));
 
   case Intrinsic::localaddress: {
-    // Returns one of the stack, base, or frame pointer registers, depending on
-    // which is used to reference local variables.
-    MachineFunction &MF = DAG.getMachineFunction();
-    const AArch64RegisterInfo *RegInfo = Subtarget->getRegisterInfo();
-    unsigned Reg;
-    if (RegInfo->hasBasePointer(MF))
-      Reg = RegInfo->getBaseRegister();
-    else // This function handles the SP or FP case.
-      Reg = RegInfo->getFrameRegister(MF);
+    const auto &MF = DAG.getMachineFunction();
+    const auto *RegInfo = Subtarget->getRegisterInfo();
+    unsigned Reg = RegInfo->getLocalAddressRegister(MF);
     return DAG.getCopyFromReg(DAG.getEntryNode(), dl, Reg,
                               Op.getSimpleValueType());
   }
 
   case Intrinsic::eh_recoverfp: {
     // FIXME: This needs to be implemented to correctly handle highly aligned
     // stack objects. For now we simply return the incoming FP. Refer D53541
     // for more details.

llvm/trunk/lib/Target/AArch64/AArch64RegisterInfo.h

public:
   unsigned getFrameRegister(const MachineFunction &MF) const override;
 
   unsigned getRegPressureLimit(const TargetRegisterClass *RC,
                                MachineFunction &MF) const override;
 
   bool trackLivenessAfterRegAlloc(const MachineFunction&) const override {
     return true;
   }
+
+  unsigned getLocalAddressRegister(const MachineFunction &MF) const;
 };
 
 } // end namespace llvm
 
 #endif

llvm/trunk/lib/Target/AArch64/AArch64RegisterInfo.cpp

if (MI.isDebugValue() || MI.getOpcode() == TargetOpcode::STACKMAP ||
     Offset = TFI->resolveFrameIndexReference(MF, FrameIndex, FrameReg,
                                              /*PreferFP=*/true);
     Offset += MI.getOperand(FIOperandNum + 1).getImm();
     MI.getOperand(FIOperandNum).ChangeToRegister(FrameReg, false /*isDef*/);
     MI.getOperand(FIOperandNum + 1).ChangeToImmediate(Offset);
     return;
   }
 
-  // Modify MI as necessary to handle as much of 'Offset' as possible
-  Offset = TFI->resolveFrameIndexReference(MF, FrameIndex, FrameReg);
-
   if (MI.getOpcode() == TargetOpcode::LOCAL_ESCAPE) {
     MachineOperand &FI = MI.getOperand(FIOperandNum);
+    Offset = TFI->getNonLocalFrameIndexReference(MF, FrameIndex);
     FI.ChangeToImmediate(Offset);
     return;
   }
 
+  // Modify MI as necessary to handle as much of 'Offset' as possible
+  Offset = TFI->getFrameIndexReference(MF, FrameIndex, FrameReg);
+
   if (rewriteAArch64FrameIndex(MI, FIOperandNum, FrameReg, Offset, TII))
     return;
 
   assert((!RS || !RS->isScavengingFrameIndex(FrameIndex)) &&
          "Emergency spill slot is out of reach");
 
   // If we get here, the immediate doesn't fit into the instruction.  We folded
   // as much as possible above.  Handle the rest, providing a register that is
unsigned AArch64RegisterInfo::getRegPressureLimit(const TargetRegisterClass *RC,
   case AArch64::QQQRegClassID:
   case AArch64::QQQQRegClassID:
     return 32;
 
   case AArch64::FPR128_loRegClassID:
     return 16;
   }
 }
+
+unsigned AArch64RegisterInfo::getLocalAddressRegister(
+  const MachineFunction &MF) const {
+  const auto &MFI = MF.getFrameInfo();
+  if (!MF.hasEHFunclets() && !MFI.hasVarSizedObjects())
+    return AArch64::SP;
+  else if (needsStackRealignment(MF))
+    return getBaseRegister();
+  return getFrameRegister(MF);
+}

llvm/trunk/test/CodeGen/AArch64/seh-finally.ll

 ; RUN: llc -mtriple arm64-windows -o - %s | FileCheck %s
 
-; Function Attrs: noinline optnone uwtable
-define dso_local i32 @foo() {
+; struct S { int x; };
+; void foo(int n);
+; void foo(struct S o);
+; void simple_seh() {
+;   struct S o;
+; 
+;   __try { foo(o.x); }
+;   __finally { foo(o.x); }
+; }
+; void stack_realign() {
+;   struct S __declspec(align(32)) o;
+; 
+;   __try { foo(o.x); }
+;   __finally { foo(o.x); }
+; }
+; void vla_present(int n) {
+;   int vla[n];
+; 
+;   __try { foo(n); }
+;   __finally { foo(n); }
+; }
+; void vla_and_realign(int n) {
+;   struct S __declspec(align(32)) o;
+;   int vla[n];
+; 
+;   __try { foo(o.x); }
+;   __finally { foo(o.x); }
+; }
+
+%struct.S = type { i32 }
+
+; Test simple SEH (__try/__finally).
+define void @simple_seh() #0 personality i8* bitcast (i32 (...)* @__C_specific_handler to i8*) {
 entry:
-; CHECK-LABEL: foo
-; CHECK: orr     w8, wzr, #0x1
-; CHECK: mov     w0, wzr
-; CHECK: mov     x1, x29
-; CHECK: .set .Lfoo$frame_escape_0, -4
-; CHECK: stur    w8, [x29, #-4]
-; CHECK: bl      "?fin$0@0@foo@@"
-; CHECK: ldur    w0, [x29, #-4]
+; CHECK-LABEL: simple_seh
+; CHECK: add     x29, sp, #16
+; CHECK: orr     x1, xzr, #0xfffffffffffffffe
+; CHECK: stur    x1, [x29, #-16]
+; CHECK: .set .Lsimple_seh$frame_escape_0, -8
+; CHECK: ldur    w0, [x29, #-8]
+; CHECK: bl      foo
+
+  %o = alloca %struct.S, align 4
+  call void (...) @llvm.localescape(%struct.S* %o)
+  %x = getelementptr inbounds %struct.S, %struct.S* %o, i32 0, i32 0
+  %0 = load i32, i32* %x, align 4
+  invoke void @foo(i32 %0) #5
+          to label %invoke.cont unwind label %ehcleanup
 
-  %count = alloca i32, align 4
-  call void (...) @llvm.localescape(i32* %count)
-  store i32 0, i32* %count, align 4
-  %0 = load i32, i32* %count, align 4
-  %add = add nsw i32 %0, 1
-  store i32 %add, i32* %count, align 4
+invoke.cont:                                      ; preds = %entry
   %1 = call i8* @llvm.localaddress()
-  call void @"?fin$0@0@foo@@"(i8 0, i8* %1)
-  %2 = load i32, i32* %count, align 4
-  ret i32 %2
-}
-
-define internal void @"?fin$0@0@foo@@"(i8 %abnormal_termination, i8* %frame_pointer) {
-entry:
-; CHECK-LABEL: @"?fin$0@0@foo@@"
-; CHECK: sub     sp, sp, #16
-; CHECK: str     x1, [sp, #8]
-; CHECK: strb    w0, [sp, #7]
-; CHECK: movz    x8, #:abs_g1_s:.Lfoo$frame_escape_0
-; CHECK: movk    x8, #:abs_g0_nc:.Lfoo$frame_escape_0
-; CHECK: add     x8, x1, x8
-; CHECK: ldr     w9, [x8]
-; CHECK: add     w9, w9, #1
-; CHECK: str     w9, [x8]
+  call void @fin_simple_seh(i8 0, i8* %1)
+  ret void
+
+ehcleanup:                                        ; preds = %entry
+  %2 = cleanuppad within none []
+  %3 = call i8* @llvm.localaddress()
+  call void @fin_simple_seh(i8 1, i8* %3) [ "funclet"(token %2) ]
+  cleanupret from %2 unwind to caller
+}
+
+define void @fin_simple_seh(i8 %abnormal_termination, i8* %frame_pointer) {
+entry:
+; CHECK-LABEL: fin_simple_seh
+; CHECK: movz    x8, #:abs_g1_s:.Lsimple_seh$frame_escape_0
+; CHECK: movk    x8, #:abs_g0_nc:.Lsimple_seh$frame_escape_0
+; CHECK: strb    w0, [sp, #15]
+; CHECK: ldr     w0, [x1, x8]
+; CHECK: bl      foo
 
   %frame_pointer.addr = alloca i8*, align 8
   %abnormal_termination.addr = alloca i8, align 1
-  %0 = call i8* @llvm.localrecover(i8* bitcast (i32 ()* @foo to i8*), i8* %frame_pointer, i32 0)
-  %count = bitcast i8* %0 to i32*
+  %0 = call i8* @llvm.localrecover(i8* bitcast (void ()* @simple_seh to i8*), i8* %frame_pointer, i32 0)
+  %o = bitcast i8* %0 to %struct.S*
   store i8* %frame_pointer, i8** %frame_pointer.addr, align 8
   store i8 %abnormal_termination, i8* %abnormal_termination.addr, align 1
-  %1 = zext i8 %abnormal_termination to i32
-  %cmp = icmp eq i32 %1, 0
-  br i1 %cmp, label %if.then, label %if.end
+  %x = getelementptr inbounds %struct.S, %struct.S* %o, i32 0, i32 0
+  %1 = load i32, i32* %x, align 4
+  call void @foo(i32 %1)
+  ret void
+}
+
+; Test SEH when stack realignment is needed in case highly aligned stack objects are present.
+define void @stack_realign() #0 personality i8* bitcast (i32 (...)* @__C_specific_handler to i8*) {
+entry:
+; CHECK-LABEL: stack_realign
+; CHECK: add     x29, sp, #16
+; CHECK: sub     x9, sp, #64
+; CHECK: and     sp, x9, #0xffffffffffffffe0
+; CHECK: mov     x19, sp
+; CHECK: orr     x1, xzr, #0xfffffffffffffffe
+; CHECK: stur    x1, [x19, #16]
+; CHECK: .set .Lstack_realign$frame_escape_0, 32
+; CHECK: ldr     w0, [x19, #32]
+; CHECK: bl      foo
+
+  %o = alloca %struct.S, align 32
+  call void (...) @llvm.localescape(%struct.S* %o)
+  %x = getelementptr inbounds %struct.S, %struct.S* %o, i32 0, i32 0
+  %0 = load i32, i32* %x, align 32
+  invoke void @foo(i32 %0) #5
+          to label %invoke.cont unwind label %ehcleanup
+
+invoke.cont:                                      ; preds = %entry
+  %1 = call i8* @llvm.localaddress()
+  call void @fin_stack_realign(i8 0, i8* %1)
+  ret void
+
+ehcleanup:                                        ; preds = %entry
+  %2 = cleanuppad within none []
+  %3 = call i8* @llvm.localaddress()
+  call void @fin_stack_realign(i8 1, i8* %3) [ "funclet"(token %2) ]
+  cleanupret from %2 unwind to caller
+}
 
-if.then:                                          ; preds = %entry
-  %2 = load i32, i32* %count, align 4
-  %add = add nsw i32 %2, 1
-  store i32 %add, i32* %count, align 4
-  br label %if.end
+define void @fin_stack_realign(i8 %abnormal_termination, i8* %frame_pointer) {
+entry:
+; CHECK-LABEL: fin_stack_realign
+; CHECK: movz    x8, #:abs_g1_s:.Lstack_realign$frame_escape_0
+; CHECK: movk    x8, #:abs_g0_nc:.Lstack_realign$frame_escape_0
+; CHECK: strb    w0, [sp, #15]
+; CHECK: ldr     w0, [x1, x8]
+; CHECK: bl      foo
 
-if.end:                                           ; preds = %if.then, %entry
+  %frame_pointer.addr = alloca i8*, align 8
+  %abnormal_termination.addr = alloca i8, align 1
+  %0 = call i8* @llvm.localrecover(i8* bitcast (void ()* @stack_realign to i8*), i8* %frame_pointer, i32 0)
+  %o = bitcast i8* %0 to %struct.S*
+  store i8* %frame_pointer, i8** %frame_pointer.addr, align 8
+  store i8 %abnormal_termination, i8* %abnormal_termination.addr, align 1
+  %x = getelementptr inbounds %struct.S, %struct.S* %o, i32 0, i32 0
+  %1 = load i32, i32* %x, align 32
+  call void @foo(i32 %1)
   ret void
 }
 
-; Function Attrs: nounwind readnone
-declare i8* @llvm.localrecover(i8*, i8*, i32)
+; Test SEH when variable size objects are present on the stack. Note: Escaped vla's are current not supported by SEH.
+define void @vla_present(i32 %n) #0 personality i8* bitcast (i32 (...)* @__C_specific_handler to i8*) {
+entry:
+; CHECK-LABEL: vla_present
+; CHECK: add     x29, sp, #32
+; CHECK: orr     x1, xzr, #0xfffffffffffffffe
+; CHECK: stur    x1, [x29, #-32]
+; CHECK: .set .Lvla_present$frame_escape_0, -4
+; CHECK: stur    w0, [x29, #-4]
+; CHECK: ldur    w8, [x29, #-4]
+; CHECK: mov     x9, sp
+; CHECK: stur    x9, [x29, #-16]
+; CHECK: stur    x8, [x29, #-24]
+; CHECK: ldur    w0, [x29, #-4]
+; CHECK: bl      foo
 
-; Function Attrs: nounwind readnone
-declare i8* @llvm.localaddress()
+  %n.addr = alloca i32, align 4
+  %saved_stack = alloca i8*, align 8
+  %__vla_expr0 = alloca i64, align 8
+  call void (...) @llvm.localescape(i32* %n.addr)
+  store i32 %n, i32* %n.addr, align 4
+  %0 = load i32, i32* %n.addr, align 4
+  %1 = zext i32 %0 to i64
+  %2 = call i8* @llvm.stacksave()
+  store i8* %2, i8** %saved_stack, align 8
+  %vla = alloca i32, i64 %1, align 4
+  store i64 %1, i64* %__vla_expr0, align 8
+  %3 = load i32, i32* %n.addr, align 4
+  invoke void @foo(i32 %3) #5
+          to label %invoke.cont unwind label %ehcleanup
+
+invoke.cont:                                      ; preds = %entry
+  %4 = call i8* @llvm.localaddress()
+  call void @fin_vla_present(i8 0, i8* %4)
+  %5 = load i8*, i8** %saved_stack, align 8
+  call void @llvm.stackrestore(i8* %5)
+  ret void
 
-; Function Attrs: nounwind
+ehcleanup:                                        ; preds = %entry
+  %6 = cleanuppad within none []
+  %7 = call i8* @llvm.localaddress()
+  call void @fin_vla_present(i8 1, i8* %7) [ "funclet"(token %6) ]
+  cleanupret from %6 unwind to caller
+}
+
+define void @fin_vla_present(i8 %abnormal_termination, i8* %frame_pointer) {
+entry:
+; CHECK-LABEL: fin_vla_present
+; CHECK: movz    x8, #:abs_g1_s:.Lvla_present$frame_escape_0
+; CHECK: movk    x8, #:abs_g0_nc:.Lvla_present$frame_escape_0
+; CHECK: strb    w0, [sp, #15]
+; CHECK: ldr     w0, [x1, x8]
+; CHECK: bl      foo
+
+  %frame_pointer.addr = alloca i8*, align 8
+  %abnormal_termination.addr = alloca i8, align 1
+  %0 = call i8* @llvm.localrecover(i8* bitcast (void (i32)* @vla_present to i8*), i8* %frame_pointer, i32 0)
+  %n.addr = bitcast i8* %0 to i32*
+  store i8* %frame_pointer, i8** %frame_pointer.addr, align 8
+  store i8 %abnormal_termination, i8* %abnormal_termination.addr, align 1
+  %1 = load i32, i32* %n.addr, align 4
+  call void @foo(i32 %1)
+  ret void
+}
+
+; Test when both vla's and highly aligned objects are present on stack.
+define void @vla_and_realign(i32 %n) #0 personality i8* bitcast (i32 (...)* @__C_specific_handler to i8*) {
+entry:
+; CHECK-LABEL: vla_and_realign
+; CHECK: add     x29, sp, #16
+; CHECK: sub     x9, sp, #64
+; CHECK: and     sp, x9, #0xffffffffffffffe0
+; CHECK: mov     x19, sp
+; CHECK: orr     x1, xzr, #0xfffffffffffffffe
+; CHECK: stur    x1, [x19]
+; CHECK: .set .Lvla_and_realign$frame_escape_0, 32
+; CHECK: stur    w0, [x29, #-4]
+; CHECK: ldur    w8, [x29, #-4]
+; CHECK: mov     x9, sp
+; CHECK: str     x9, [x19, #24]
+; CHECK: str     x8, [x19, #16]
+; CHECK: ldr     w0, [x19, #32]
+; CHECK: bl      foo
+
+  %n.addr = alloca i32, align 4
+  %o = alloca %struct.S, align 32
+  %saved_stack = alloca i8*, align 8
+  %__vla_expr0 = alloca i64, align 8
+  call void (...) @llvm.localescape(%struct.S* %o)
+  store i32 %n, i32* %n.addr, align 4
+  %0 = load i32, i32* %n.addr, align 4
+  %1 = zext i32 %0 to i64
+  %2 = call i8* @llvm.stacksave()
+  store i8* %2, i8** %saved_stack, align 8
+  %vla = alloca i32, i64 %1, align 4
+  store i64 %1, i64* %__vla_expr0, align 8
+  %x = getelementptr inbounds %struct.S, %struct.S* %o, i32 0, i32 0
+  %3 = load i32, i32* %x, align 32
+  invoke void @foo(i32 %3) #5
+          to label %invoke.cont unwind label %ehcleanup
+
+invoke.cont:                                      ; preds = %entry
+  %4 = call i8* @llvm.localaddress()
+  call void @fin_vla_and_realign(i8 0, i8* %4)
+  %5 = load i8*, i8** %saved_stack, align 8
+  call void @llvm.stackrestore(i8* %5)
+  ret void
+
+ehcleanup:                                        ; preds = %entry
+  %6 = cleanuppad within none []
+  %7 = call i8* @llvm.localaddress()
+  call void @fin_vla_and_realign(i8 1, i8* %7) [ "funclet"(token %6) ]
+  cleanupret from %6 unwind to caller
+}
+
+define void @fin_vla_and_realign(i8 %abnormal_termination, i8* %frame_pointer) {
+entry:
+; CHECK-LABEL: fin_vla_and_realign
+; CHECK: movz    x8, #:abs_g1_s:.Lvla_and_realign$frame_escape_0
+; CHECK: movk    x8, #:abs_g0_nc:.Lvla_and_realign$frame_escape_0
+; CHECK: strb    w0, [sp, #15]
+; CHECK: ldr     w0, [x1, x8]
+; CHECK: bl      foo
+
+  %frame_pointer.addr = alloca i8*, align 8
+  %abnormal_termination.addr = alloca i8, align 1
+  %0 = call i8* @llvm.localrecover(i8* bitcast (void (i32)* @vla_and_realign to i8*), i8* %frame_pointer, i32 0)
+  %o = bitcast i8* %0 to %struct.S*
+  store i8* %frame_pointer, i8** %frame_pointer.addr, align 8
+  store i8 %abnormal_termination, i8* %abnormal_termination.addr, align 1
+  %x = getelementptr inbounds %struct.S, %struct.S* %o, i32 0, i32 0
+  %1 = load i32, i32* %x, align 32
+  call void @foo(i32 %1)
+  ret void
+}
+
+declare void @foo(i32)
+declare void @llvm.stackrestore(i8*)
+declare i8* @llvm.stacksave()
+declare i8* @llvm.localrecover(i8*, i8*, i32)
+declare i8* @llvm.localaddress()
 declare void @llvm.localescape(...)
+declare i32 @__C_specific_handler(...)
+
+attributes #0 = { noinline optnone }

llvm/trunk/test/CodeGen/AArch64/seh-localescape.ll

-; RUN: llc -mtriple arm64-windows %s -o - | FileCheck %s
-
-; Function Attrs: noinline nounwind optnone uwtable
-define dso_local i32 @foo() {
-entry:
-; CHECK-LABEL: foo
-; CHECK: .set .Lfoo$frame_escape_0, -4
-
-  %count = alloca i32, align 4
-  call void (...) @llvm.localescape(i32* %count)
-  ret i32 0
-}
-
-define internal i32 @"?filt$0@0@foo@@"(i8* %exception_pointers, i8* %frame_pointer) {
-entry:
-; CHECK-LABEL: @"?filt$0@0@foo@@"
-; CHECK: movz    x8, #:abs_g1_s:.Lfoo$frame_escape_0
-; CHECK: movk    x8, #:abs_g0_nc:.Lfoo$frame_escape_0
-
-  %0 = call i8* @llvm.localrecover(i8* bitcast (i32 ()* @foo to i8*), i8* %frame_pointer, i32 0)
-  %count = bitcast i8* %0 to i32*
-  %1 = load i32, i32* %count, align 4
-  ret i32 %1
-}
-
-; Function Attrs: nounwind readnone
-declare i8* @llvm.localrecover(i8*, i8*, i32) #2
-
-; Function Attrs: nounwind
-declare void @llvm.localescape(...) #3