[ARM64] [Windows] MCLayer support for exception handling

[ssijaric]

Summary

This patch adds the ARM64 unwind opcodes to the MCLayer, as well as SEH directives that will be emitted by the frame lowering in a patch that will follow. This patch, in conjunction with the frame lowering patch, was tested on a couple of SPEC CPU2006 C++ benchmarks that make use of exception handling. We only emit the unwinding opcodes into object files for now.

[ssijaric]

lib/MC/MCAsmStreamer.cpp

1709 ↗

(On Diff #158669)

Will turn asserts into errs, as there are existing test cases that require unwinding info, and will hit these asserts when generating .s files.

[efriedma]

Perhaps I can create a MIR test case with the SEH opcodes, and use llvm-objdump to check the .xdata section?

This seems reasonable.

lib/MC/MCWin64EH.cpp

537 ↗

(On Diff #158669)

Non-deterministic output, I think, since the keys of EpilogMap are pointers.

lib/Target/AArch64/MCTargetDesc/AArch64WinCOFFStreamer.cpp

66 ↗

(On Diff #158669)

Please commit separately? (Although, I'm not sure it matters: IIRC AArch64 assemblers can't relax anything anyway.)

[ssijaric]

Use errs instead of assert MCAsmPrinter for unwind opcodes that are not yet supported. Address Eli's non-determinism comment. Use range-based for loops. Still need to add a test case.

[rnk]

I think it was a mistake to put the Windows x64 .seh_ directives directly in MCStreamer. They should've been implemented similar to the way ARM CFI directives are implemented inside the ARMTargetStreamer class and the way CodeView FPO data is implemented entirely inside lib/Target/X86. Otherwise, we will continue to have a massive proliferation of every target's unwind info directives implemented in our main cross-platform assembler class, MCStreamer.

Does that make sense? You should be able to modify AArch64MCInstrLower.cpp to direct these AArch64-specific pseudo instructions over to the AArch64TargetStreamer class. The .cv_fpo_* directive implementations are probably your best guide.

[ssijaric]

Hi Reid,

Yes, I will move the SEH directives to the AArch64TargetStreamer, etc., and upload a new patch.

Thanks,
Sanjin

[ssijaric]

Move SEH opcode handling to AArch64 target streamers.

Working on MIR test cases.

[ssijaric]

Add a test case and fix a couple of encodings.

[ssijaric]

I also moved the AArch64 specific changes to the Windows exception handling tables to this patch, as they are needed to get the test case to pass (they are required in order to populate .xdata correctly). The test case also has a dependence on https://reviews.llvm.org/D51201, which adds HasWinCFI to the MIRParser.

There's also a dependence on making MachineFunction::HasWinCFI just a plain bool defaulting to false, instead of Optional<bool>. Otherwise, some existing Windows test cases will fail due to not having HasWinCFI set which is checked during .xdata generation. Reid, can you please submit this change as you mentioned it in D50288? Alternatively, I can go through the failing test cases (about 10 in total) and mark the functions with the "nounwind" attribute.

[rnk]

I like the code, but I think this needs more tests, and better testing infrastructure before we can commit it. I would recommend beefing up COFFDumper::printUnwindInfo in llvm-readobj to print the ARM64 unwind opcodes in a human readable format, and then you won't need to write tests with hexadecimal CHECKs like you have now.

lib/MC/MCWin64EH.cpp

30 ↗

(On Diff #162315)

llvm_unreachable() is preferred for this

69 ↗

(On Diff #162315)

ditto

271 ↗

(On Diff #162315)

llvm_unreachable

lib/Target/AArch64/MCTargetDesc/AArch64ELFStreamer.cpp

225 ↗

(On Diff #162315)

This should probably live in AArch64TargetStreamer.cpp of AArch64MCTargetDesc.cpp, and call through a similar factory free function for ELF.

[ssijaric]

Thanks for reviewing, Reid. This update uses llvm_unreachable instead of asserts, and moves createAArch64TargetObjectStreamer to AArch64TargetStreamer.h/.cpp. To do this, I also moved AArch64TargetELFStreamer to the llvm namespace.

I am working on adding support to llvm-objdump to dump the unwind info for Windows ARM64 in a meaningful way. Once that is complete, I will redo the test case, as well as add more test cases.

[ssijaric]

Add test cases. These depend on D53264 and D51201.

[ssijaric]

Re-uploaed with full context and remove an unused declaration of getLabelPlusOne.

[efriedma]

Missing test coverage for SEH_SaveFPLR_X, SEH_SaveReg, SEH_SaveFReg, SEH_SetFP, SEH_AddFP, SEH_Nop, and the alloc_s variant of SEH_StackAlloc. (I'd like to see at least one use of each opcode to show the encoding is correct/consistent with llvm-readobj.)

test/CodeGen/AArch64/wineh5.mir

164 ↗

(On Diff #170697)

It looks like the SEH_StackAlloc is after the PrologEnd here; is that supposed to be valid? If not, can we report_fatal_error on MIR which does that?

Even ignoring that, the opcode sequence here is clearly wrong; you aren't allowed to call another function in the prolog. (From the unwinder's point of view, __chkstk is dynamic allocation, like alloca.) I guess you generated this using clang; that indicates a bug in the frame lowering patch, not this patch. But it would be nice to have valid testcases anyway.

[ssijaric]

test/CodeGen/AArch64/wineh5.mir

164 ↗

(On Diff #170697)

Yes, this is wrong. Cannot come after PrologueEnd. I was doing some experiments with the frame lowering patch and the stack probe, which I clearly got wrong. I will check what cl.exe does.

[ssijaric]

Fix the test case, wineh5.mir, to conform to what cl.exe produces.

[rnk]

In D50166#1273097, @efriedma wrote:

Missing test coverage for SEH_SaveFPLR_X, SEH_SaveReg, SEH_SaveFReg, SEH_SetFP, SEH_AddFP, SEH_Nop, and the alloc_s variant of SEH_StackAlloc. (I'd like to see at least one use of each opcode to show the encoding is correct/consistent with llvm-readobj.)

I'd also like to see a test for shrink-wrapping. You may have to disable it if you cannot yet describe such prologues it with SEH_ opcodes.

[ssijaric]

In D50166#1274871, @rnk wrote:

In D50166#1273097, @efriedma wrote:

Missing test coverage for SEH_SaveFPLR_X, SEH_SaveReg, SEH_SaveFReg, SEH_SetFP, SEH_AddFP, SEH_Nop, and the alloc_s variant of SEH_StackAlloc. (I'd like to see at least one use of each opcode to show the encoding is correct/consistent with llvm-readobj.)

I'd also like to see a test for shrink-wrapping. You may have to disable it if you cannot yet describe such prologues it with SEH_ opcodes.

Hi Reid,

I think shrink wrapping is disabled by default for AArch64. I have a small test case that gets shrink-wrapped, but only if I enable it manually using -enable-shrink-wrap. Do we need a test case if it's disabled by default?

[ssijaric]

In D50166#1276881, @ssijaric wrote:

In D50166#1274871, @rnk wrote:

In D50166#1273097, @efriedma wrote:

Missing test coverage for SEH_SaveFPLR_X, SEH_SaveReg, SEH_SaveFReg, SEH_SetFP, SEH_AddFP, SEH_Nop, and the alloc_s variant of SEH_StackAlloc. (I'd like to see at least one use of each opcode to show the encoding is correct/consistent with llvm-readobj.)

I'd also like to see a test for shrink-wrapping. You may have to disable it if you cannot yet describe such prologues it with SEH_ opcodes.

Hi Reid,

I think shrink wrapping is disabled by default for AArch64. I have a small test case that gets shrink-wrapped, but only if I enable it manually using -enable-shrink-wrap. Do we need a test case if it's disabled by default?

Wrong observation on my part. Shrink wrapping is enabled by default on AArch64, but only if it has no Windows CFI. There is even a comment in Shrink Wrapping that says:

Windows with CFI has some limitations that make it impossible
to use shrink-wrapping.

To handle shrink wrapping, we need to be able to break functions into regions, each with each own .pdata and .xdata. We don't currently support this. But it is definitely something we should handle in the future.

So, I will skip a test case for shrink wrapping. Will add a few more tests to cover remaining the unwind codes.

[ssijaric]

Add test cases for the remaining SEH opcodes. Disable post RA scheduling for the test cases.

A follow on patch will take care of scheduling to make sure that either:

1. SEH_PrologEnd and SEH_EpilogStart/SEH_EpilogEnd act as barriers, or
2. Insert a SEH_Nop for any instruction that ends up between SEH_EpilogStart/SEH_EpilogEnd

Also, need a patch to fix the formatting in llvm-readobj (e.g. the indentation in wineh2.mir is off for floating point saves/restores).

[efriedma]

A follow on patch will take care of scheduling

That seems fine. The fix is likely orthogonal to the other changes. (Maybe you need to override isSchedulingBoundary?)

Also, need a patch to fix the formatting in llvm-readobj (e.g. the indentation in wineh2.mir is off for floating point saves/restores).

FileCheck collapses whitespace; you can fix that in your MIR testcases independently of the readobj fix.

lib/MC/MCWin64EH.cpp

505 ↗

(On Diff #171263)

The extended code word handling is still wrong.

[rnk]

In D50166#1276921, @ssijaric wrote:

So, I will skip a test case for shrink wrapping. Will add a few more tests to cover remaining the unwind codes.

Can you copy a test where shrink wrapping would fire, and validate that it does not for aarch64-windows-gnu? Without the test, someone could remove the logic you found and no tests would fail. We have a test like this for x86_64.

[ssijaric]

In D50166#1277766, @rnk wrote:

In D50166#1276921, @ssijaric wrote:

So, I will skip a test case for shrink wrapping. Will add a few more tests to cover remaining the unwind codes.

Can you copy a test where shrink wrapping would fire, and validate that it does not for aarch64-windows-gnu? Without the test, someone could remove the logic you found and no tests would fail. We have a test like this for x86_64.

Sure, I will add a test case that gets shrink wrapped on aarch64-linux, but doesn't get shrink wrapped on aarch64-windows.

[ssijaric]

Remove spaces in wineh2.mit test case. Fix the extended code words logic that Eli pointed out. Add a test case to make sure that shrink wrapping doesn't kick in.

[efriedma]

LGTM

llvm/trunk/include/llvm/MC/MCStreamer.h

class MCStreamer {
   MCDwarfFrameInfo *getCurrentDwarfFrameInfo();
 
   /// Similar to DwarfFrameInfos, but for SEH unwind info. Chained frames may
   /// refer to each other, so use std::unique_ptr to provide pointer stability.
   std::vector<std::unique_ptr<WinEH::FrameInfo>> WinFrameInfos;
 
   WinEH::FrameInfo *CurrentWinFrameInfo;
 
-  /// Retreive the current frame info if one is available and it is not yet
-  /// closed. Otherwise, issue an error and return null.
-  WinEH::FrameInfo *EnsureValidWinFrameInfo(SMLoc Loc);
-
   /// Tracks an index to represent the order a symbol was emitted in.
   /// Zero means we did not emit that symbol.
   DenseMap<const MCSymbol *, unsigned> SymbolOrdering;
 
   /// This is stack of current and previous section values saved by
   /// PushSection.
   SmallVector<std::pair<MCSectionSubPair, MCSectionSubPair>, 4> SectionStack;
 
   /// The next unique ID to use when creating a WinCFI-related section (.pdata
   /// or .xdata). This ID ensures that we have a one-to-one mapping from
   /// code section to unwind info section, which MSVC's incremental linker
   /// requires.
   unsigned NextWinCFIID = 0;
 
   bool UseAssemblerInfoForParsing;
 
 protected:
   MCStreamer(MCContext &Ctx);
 
   virtual void EmitCFIStartProcImpl(MCDwarfFrameInfo &Frame);
   virtual void EmitCFIEndProcImpl(MCDwarfFrameInfo &CurFrame);
 
-  /// When emitting an object file, create and emit a real label. When emitting
-  /// textual assembly, this should do nothing to avoid polluting our output.
-  virtual MCSymbol *EmitCFILabel();
-
   WinEH::FrameInfo *getCurrentWinFrameInfo() {
     return CurrentWinFrameInfo;
   }
 
   virtual void EmitWindowsUnwindTables();
 
   virtual void EmitRawTextImpl(StringRef String);
 
public:
 
   void setUseAssemblerInfoForParsing(bool v) { UseAssemblerInfoForParsing = v; }
   bool getUseAssemblerInfoForParsing() { return UseAssemblerInfoForParsing; }
 
   MCTargetStreamer *getTargetStreamer() {
     return TargetStreamer.get();
   }
 
+  /// When emitting an object file, create and emit a real label. When emitting
+  /// textual assembly, this should do nothing to avoid polluting our output.
+  virtual MCSymbol *EmitCFILabel();
+
+  /// Retreive the current frame info if one is available and it is not yet
+  /// closed. Otherwise, issue an error and return null.
+  WinEH::FrameInfo *EnsureValidWinFrameInfo(SMLoc Loc);
+
   unsigned getNumFrameInfos() { return DwarfFrameInfos.size(); }
   ArrayRef<MCDwarfFrameInfo> getDwarfFrameInfos() const {
     return DwarfFrameInfos;
   }
 
   bool hasUnfinishedDwarfFrameInfo();
 
   unsigned getNumWinFrameInfos() { return WinFrameInfos.size(); }
public:
   virtual void EmitCFIGnuArgsSize(int64_t Size);
   virtual void EmitCFISignalFrame();
   virtual void EmitCFIUndefined(int64_t Register);
   virtual void EmitCFIRegister(int64_t Register1, int64_t Register2);
   virtual void EmitCFIWindowSave();
 
   virtual void EmitWinCFIStartProc(const MCSymbol *Symbol, SMLoc Loc = SMLoc());
   virtual void EmitWinCFIEndProc(SMLoc Loc = SMLoc());
+  /// This is used on platforms, such as Windows on ARM64, that require function
+  /// or funclet sizes to be emitted in .xdata before the End marker is emitted
+  /// for the frame.  We cannot use the End marker, as it is not set at the
+  /// point of emitting .xdata, in order to indicate that the frame is active.
+  virtual void EmitWinCFIFuncletOrFuncEnd(SMLoc Loc = SMLoc());
   virtual void EmitWinCFIStartChained(SMLoc Loc = SMLoc());
   virtual void EmitWinCFIEndChained(SMLoc Loc = SMLoc());
   virtual void EmitWinCFIPushReg(unsigned Register, SMLoc Loc = SMLoc());
   virtual void EmitWinCFISetFrame(unsigned Register, unsigned Offset,
                                   SMLoc Loc = SMLoc());
   virtual void EmitWinCFIAllocStack(unsigned Size, SMLoc Loc = SMLoc());
   virtual void EmitWinCFISaveReg(unsigned Register, unsigned Offset,
                                  SMLoc Loc = SMLoc());

llvm/trunk/include/llvm/MC/MCWin64EH.h

struct Instruction {
   }
 };
 
 class UnwindEmitter : public WinEH::UnwindEmitter {
 public:
   void Emit(MCStreamer &Streamer) const override;
   void EmitUnwindInfo(MCStreamer &Streamer, WinEH::FrameInfo *FI) const override;
 };
+
+class ARM64UnwindEmitter : public WinEH::UnwindEmitter {
+public:
+  void Emit(MCStreamer &Streamer) const override;
+  void EmitUnwindInfo(MCStreamer &Streamer,
+                      WinEH::FrameInfo *FI) const override;
+};
+
 }
 } // end namespace llvm
 
 #endif

llvm/trunk/include/llvm/MC/MCWinEH.h

 //===- MCWinEH.h - Windows Unwinding Support --------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_MC_MCWINEH_H
 #define LLVM_MC_MCWINEH_H
 
+#include "llvm/ADT/MapVector.h"
 #include <vector>
 
 namespace llvm {
 class MCSection;
 class MCStreamer;
 class MCSymbol;
 
 namespace WinEH {
 struct Instruction {
   const MCSymbol *Label;
-  const unsigned Offset;
-  const unsigned Register;
-  const unsigned Operation;
+  unsigned Offset;
+  unsigned Register;
+  unsigned Operation;
 
   Instruction(unsigned Op, MCSymbol *L, unsigned Reg, unsigned Off)
     : Label(L), Offset(Off), Register(Reg), Operation(Op) {}
 };
 
 struct FrameInfo {
   const MCSymbol *Begin = nullptr;
   const MCSymbol *End = nullptr;
+  const MCSymbol *FuncletOrFuncEnd = nullptr;
   const MCSymbol *ExceptionHandler = nullptr;
   const MCSymbol *Function = nullptr;
   const MCSymbol *PrologEnd = nullptr;
   const MCSymbol *Symbol = nullptr;
   const MCSection *TextSection = nullptr;
 
   bool HandlesUnwind = false;
   bool HandlesExceptions = false;
 
   int LastFrameInst = -1;
   const FrameInfo *ChainedParent = nullptr;
   std::vector<Instruction> Instructions;
+  MapVector<MCSymbol*, std::vector<Instruction>> EpilogMap;
 
   FrameInfo() = default;
   FrameInfo(const MCSymbol *Function, const MCSymbol *BeginFuncEHLabel)
       : Begin(BeginFuncEHLabel), Function(Function) {}
   FrameInfo(const MCSymbol *Function, const MCSymbol *BeginFuncEHLabel,
             const FrameInfo *ChainedParent)
       : Begin(BeginFuncEHLabel), Function(Function),
         ChainedParent(ChainedParent) {}

llvm/trunk/include/llvm/Support/Win64EH.h

enum UnwindOpcodes {
   UOP_PushNonVol = 0,
   UOP_AllocLarge,
   UOP_AllocSmall,
   UOP_SetFPReg,
   UOP_SaveNonVol,
   UOP_SaveNonVolBig,
   UOP_SaveXMM128 = 8,
   UOP_SaveXMM128Big,
-  UOP_PushMachFrame
+  UOP_PushMachFrame,
+  // The following set of unwind opcodes is for ARM64.  They are documented at
+  // https://docs.microsoft.com/en-us/cpp/build/arm64-exception-handling
+  UOP_AllocMedium,
+  UOP_SaveFPLRX,
+  UOP_SaveFPLR,
+  UOP_SaveReg,
+  UOP_SaveRegX,
+  UOP_SaveRegP,
+  UOP_SaveRegPX,
+  UOP_SaveFReg,
+  UOP_SaveFRegX,
+  UOP_SaveFRegP,
+  UOP_SaveFRegPX,
+  UOP_SetFP,
+  UOP_AddFP,
+  UOP_Nop,
+  UOP_End
 };
 
 /// UnwindCode - This union describes a single operation in a function prolog,
 /// or part thereof.
 union UnwindCode {
   struct {
     uint8_t CodeOffset;
     uint8_t UnwindOpAndOpInfo;

llvm/trunk/lib/CodeGen/AsmPrinter/WinException.h

class LLVM_LIBRARY_VISIBILITY WinException : public EHStreamer {
   bool shouldEmitLSDA = false;
 
   /// Per-function flag to indicate if frame moves info should be emitted.
   bool shouldEmitMoves = false;
 
   /// True if this is a 64-bit target and we should use image relative offsets.
   bool useImageRel32 = false;
 
+  /// True if we are generating exception handling on Windows for ARM64.
+  bool isAArch64 = false;
+
   /// Pointer to the current funclet entry BB.
   const MachineBasicBlock *CurrentFuncletEntry = nullptr;
 
   /// The section of the last funclet start.
   MCSection *CurrentFuncletTextSection = nullptr;
 
   void emitCSpecificHandlerTable(const MachineFunction *MF);
 
class LLVM_LIBRARY_VISIBILITY WinException : public EHStreamer {
 
   /// Emits the label used with llvm.x86.seh.recoverfp, which is used by
   /// outlined funclets.
   void emitEHRegistrationOffsetLabel(const WinEHFuncInfo &FuncInfo,
                                      StringRef FLinkageName);
 
   const MCExpr *create32bitRef(const MCSymbol *Value);
   const MCExpr *create32bitRef(const GlobalValue *GV);
-  const MCExpr *getLabelPlusOne(const MCSymbol *Label);
+  const MCExpr *getLabel(const MCSymbol *Label);
   const MCExpr *getOffset(const MCSymbol *OffsetOf, const MCSymbol *OffsetFrom);
   const MCExpr *getOffsetPlusOne(const MCSymbol *OffsetOf,
                                  const MCSymbol *OffsetFrom);
 
   /// Gets the offset that we should use in a table for a stack object with the
   /// given index. For targets using CFI (Win64, etc), this is relative to the
   /// established SP at the end of the prologue. For targets without CFI (Win32
   /// only), it is relative to the frame pointer.

llvm/trunk/lib/CodeGen/AsmPrinter/WinException.cpp

 #include "llvm/Target/TargetLoweringObjectFile.h"
 #include "llvm/Target/TargetOptions.h"
 using namespace llvm;
 
 WinException::WinException(AsmPrinter *A) : EHStreamer(A) {
   // MSVC's EH tables are always composed of 32-bit words.  All known 64-bit
   // platforms use an imagerel32 relocation to refer to symbols.
   useImageRel32 = (A->getDataLayout().getPointerSizeInBits() == 64);
+  isAArch64 = Asm->TM.getTargetTriple().isAArch64();
 }
 
 WinException::~WinException() {}
 
 /// endModule - Emit all exception information that should come after the
 /// content.
 void WinException::endModule() {
   auto &OS = *Asm->OutStreamer;
void WinException::endFunclet() {
 
   const MachineFunction *MF = Asm->MF;
   if (shouldEmitMoves || shouldEmitPersonality) {
     const Function &F = MF->getFunction();
     EHPersonality Per = EHPersonality::Unknown;
     if (F.hasPersonalityFn())
       Per = classifyEHPersonality(F.getPersonalityFn()->stripPointerCasts());
 
+    // On funclet exit, we emit a fake "function" end marker, so that the call
+    // to EmitWinEHHandlerData below can calculate the size of the funclet or
+    // function.
+    if (isAArch64) {
+      Asm->OutStreamer->SwitchSection(CurrentFuncletTextSection);
+      Asm->OutStreamer->EmitWinCFIFuncletOrFuncEnd();
+      MCSection *XData = Asm->OutStreamer->getAssociatedXDataSection(
+          Asm->OutStreamer->getCurrentSectionOnly());
+      Asm->OutStreamer->SwitchSection(XData);
+    }
+
     // Emit an UNWIND_INFO struct describing the prologue.
     Asm->OutStreamer->EmitWinEHHandlerData();
 
     if (Per == EHPersonality::MSVC_CXX && shouldEmitPersonality &&
         !CurrentFuncletEntry->isCleanupFuncletEntry()) {
       // If this is a C++ catch funclet (or the parent function),
       // emit a reference to the LSDA for the parent function.
       StringRef FuncLinkageName = GlobalValue::dropLLVMManglingEscape(F.getName());

 }
 
 const MCExpr *WinException::create32bitRef(const GlobalValue *GV) {
   if (!GV)
     return MCConstantExpr::create(0, Asm->OutContext);
   return create32bitRef(Asm->getSymbol(GV));
 }
 
-const MCExpr *WinException::getLabelPlusOne(const MCSymbol *Label) {
+const MCExpr *WinException::getLabel(const MCSymbol *Label) {
+  if (isAArch64)
+    return MCSymbolRefExpr::create(Label, MCSymbolRefExpr::VK_COFF_IMGREL32,
+                                   Asm->OutContext);
   return MCBinaryExpr::createAdd(create32bitRef(Label),
                                  MCConstantExpr::create(1, Asm->OutContext),
                                  Asm->OutContext);
 }
 
 const MCExpr *WinException::getOffset(const MCSymbol *OffsetOf,
                                       const MCSymbol *OffsetFrom) {
   return MCBinaryExpr::createSub(
void WinException::emitCSpecificHandlerTable(const MachineFunction *MF) {
   OS.EmitLabel(TableEnd);
 }
 
 void WinException::emitSEHActionsForRange(const WinEHFuncInfo &FuncInfo,
                                           const MCSymbol *BeginLabel,
                                           const MCSymbol *EndLabel, int State) {
   auto &OS = *Asm->OutStreamer;
   MCContext &Ctx = Asm->OutContext;
-
   bool VerboseAsm = OS.isVerboseAsm();
   auto AddComment = [&](const Twine &Comment) {
     if (VerboseAsm)
       OS.AddComment(Comment);
   };
 
   assert(BeginLabel && EndLabel);
   while (State != -1) {
     const SEHUnwindMapEntry &UME = FuncInfo.SEHUnwindMap[State];
     const MCExpr *FilterOrFinally;
     const MCExpr *ExceptOrNull;
     auto *Handler = UME.Handler.get<MachineBasicBlock *>();
     if (UME.IsFinally) {
       FilterOrFinally = create32bitRef(getMCSymbolForMBB(Asm, Handler));
       ExceptOrNull = MCConstantExpr::create(0, Ctx);
     } else {
       // For an except, the filter can be 1 (catch-all) or a function
       // label.
       FilterOrFinally = UME.Filter ? create32bitRef(UME.Filter)
                                    : MCConstantExpr::create(1, Ctx);
       ExceptOrNull = create32bitRef(Handler->getSymbol());
     }
 
     AddComment("LabelStart");
-    OS.EmitValue(getLabelPlusOne(BeginLabel), 4);
+    OS.EmitValue(getLabel(BeginLabel), 4);
     AddComment("LabelEnd");
-    OS.EmitValue(getLabelPlusOne(EndLabel), 4);
+    OS.EmitValue(getLabel(EndLabel), 4);
     AddComment(UME.IsFinally ? "FinallyFunclet" : UME.Filter ? "FilterFunction"
                                                              : "CatchAll");
     OS.EmitValue(FilterOrFinally, 4);
     AddComment(UME.IsFinally ? "Null" : "ExceptionHandler");
     OS.EmitValue(ExceptOrNull, 4);
 
     assert(UME.ToState < State && "states should decrease");
     State = UME.ToState;
for (size_t I = 0, E = FuncInfo.TryBlockMap.size(); I != E; ++I) {
       MCSymbol *HandlerMapXData = HandlerMaps[I];
       if (!HandlerMapXData)
         continue;
       // HandlerType {
       //   int32_t         Adjectives;
       //   TypeDescriptor *Type;
       //   int32_t         CatchObjOffset;
       //   void          (*Handler)();
-      //   int32_t         ParentFrameOffset; // x64 only
+      //   int32_t         ParentFrameOffset; // x64 and AArch64 only
       // };
       OS.EmitLabel(HandlerMapXData);
       for (const WinEHHandlerType &HT : TBME.HandlerArray) {
         // Get the frame escape label with the offset of the catch object. If
         // the index is INT_MAX, then there is no catch object, and we should
         // emit an offset of zero, indicating that no copy will occur.
         const MCExpr *FrameAllocOffsetRef = nullptr;
         if (HT.CatchObj.FrameIndex != INT_MAX) {
for (const auto &StateChange : InvokeStateChangeIterator::range(
       // start label for the invoke if we have one, otherwise (this is a call
       // which may unwind to our caller and does not have an EH start label, so)
       // use the previous end label.
       const MCSymbol *ChangeLabel = StateChange.NewStartLabel;
       if (!ChangeLabel)
         ChangeLabel = StateChange.PreviousEndLabel;
       // Emit an entry indicating that PCs after 'Label' have this EH state.
       IPToStateTable.push_back(
-          std::make_pair(getLabelPlusOne(ChangeLabel), StateChange.NewState));
+          std::make_pair(getLabel(ChangeLabel), StateChange.NewState));
       // FIXME: assert that NewState is between CatchLow and CatchHigh.
     }
   }
 }
 
 void WinException::emitEHRegistrationOffsetLabel(const WinEHFuncInfo &FuncInfo,
                                                  StringRef FLinkageName) {
   // Outlined helpers called by the EH runtime need to know the offset of the EH

llvm/trunk/lib/MC/MCAsmStreamer.cpp

public:
   void EmitCFISignalFrame() override;
   void EmitCFIUndefined(int64_t Register) override;
   void EmitCFIRegister(int64_t Register1, int64_t Register2) override;
   void EmitCFIWindowSave() override;
   void EmitCFIReturnColumn(int64_t Register) override;
 
   void EmitWinCFIStartProc(const MCSymbol *Symbol, SMLoc Loc) override;
   void EmitWinCFIEndProc(SMLoc Loc) override;
+  void EmitWinCFIFuncletOrFuncEnd(SMLoc Loc) override;
   void EmitWinCFIStartChained(SMLoc Loc) override;
   void EmitWinCFIEndChained(SMLoc Loc) override;
   void EmitWinCFIPushReg(unsigned Register, SMLoc Loc) override;
   void EmitWinCFISetFrame(unsigned Register, unsigned Offset,
                           SMLoc Loc) override;
   void EmitWinCFIAllocStack(unsigned Size, SMLoc Loc) override;
   void EmitWinCFISaveReg(unsigned Register, unsigned Offset,
                          SMLoc Loc) override;

 
 void MCAsmStreamer::EmitWinCFIEndProc(SMLoc Loc) {
   MCStreamer::EmitWinCFIEndProc(Loc);
 
   OS << "\t.seh_endproc";
   EmitEOL();
 }
 
+// TODO: Implement
+void MCAsmStreamer::EmitWinCFIFuncletOrFuncEnd(SMLoc Loc) {
+}
+
 void MCAsmStreamer::EmitWinCFIStartChained(SMLoc Loc) {
   MCStreamer::EmitWinCFIStartChained(Loc);
 
   OS << "\t.seh_startchained";
   EmitEOL();
 }
 
 void MCAsmStreamer::EmitWinCFIEndChained(SMLoc Loc) {

llvm/trunk/lib/MC/MCStreamer.cpp

if (!CurFrame)
     return;
   if (CurFrame->ChainedParent)
     getContext().reportError(Loc, "Not all chained regions terminated!");
 
   MCSymbol *Label = EmitCFILabel();
   CurFrame->End = Label;
 }
 
+void MCStreamer::EmitWinCFIFuncletOrFuncEnd(SMLoc Loc) {
+  WinEH::FrameInfo *CurFrame = EnsureValidWinFrameInfo(Loc);
+  if (!CurFrame)
+    return;
+  if (CurFrame->ChainedParent)
+    getContext().reportError(Loc, "Not all chained regions terminated!");
+
+  MCSymbol *Label = EmitCFILabel();
+  CurFrame->FuncletOrFuncEnd = Label;
+}
+
 void MCStreamer::EmitWinCFIStartChained(SMLoc Loc) {
   WinEH::FrameInfo *CurFrame = EnsureValidWinFrameInfo(Loc);
   if (!CurFrame)
     return;
 
   MCSymbol *StartProc = EmitCFILabel();
 
   WinFrameInfos.emplace_back(llvm::make_unique<WinEH::FrameInfo>(

llvm/trunk/lib/MC/MCWin64EH.cpp

 //===- lib/MC/MCWin64EH.cpp - MCWin64EH implementation --------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/MC/MCWin64EH.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/MC/MCContext.h"
 #include "llvm/MC/MCExpr.h"
+#include "llvm/MC/MCObjectFileInfo.h"
+#include "llvm/MC/MCObjectStreamer.h"
+#include "llvm/MC/MCSectionCOFF.h"
 #include "llvm/MC/MCStreamer.h"
 #include "llvm/MC/MCSymbol.h"
 #include "llvm/Support/Win64EH.h"
 
 using namespace llvm;
 
 // NOTE: All relocations generated here are 4-byte image-relative.
 
 static uint8_t CountOfUnwindCodes(std::vector<WinEH::Instruction> &Insns) {
   uint8_t Count = 0;
   for (const auto &I : Insns) {
     switch (static_cast<Win64EH::UnwindOpcodes>(I.Operation)) {
+    default:
+      llvm_unreachable("Unsupported unwind code");
     case Win64EH::UOP_PushNonVol:
     case Win64EH::UOP_AllocSmall:
     case Win64EH::UOP_SetFPReg:
     case Win64EH::UOP_PushMachFrame:
       Count += 1;
       break;
     case Win64EH::UOP_SaveNonVol:
     case Win64EH::UOP_SaveXMM128:

 }
 
 static void EmitUnwindCode(MCStreamer &streamer, const MCSymbol *begin,
                            WinEH::Instruction &inst) {
   uint8_t b2;
   uint16_t w;
   b2 = (inst.Operation & 0x0F);
   switch (static_cast<Win64EH::UnwindOpcodes>(inst.Operation)) {
+  default:
+    llvm_unreachable("Unsupported unwind code");
   case Win64EH::UOP_PushNonVol:
     EmitAbsDifference(streamer, inst.Label, begin);
     b2 |= (inst.Register & 0x0F) << 4;
     streamer.EmitIntValue(b2, 1);
     break;
   case Win64EH::UOP_AllocLarge:
     EmitAbsDifference(streamer, inst.Label, begin);
     if (inst.Offset > 512 * 1024 - 8) {
void llvm::Win64EH::UnwindEmitter::EmitUnwindInfo(
   // Switch sections (the static function above is meant to be called from
   // here and from Emit().
   MCSection *XData = Streamer.getAssociatedXDataSection(info->TextSection);
   Streamer.SwitchSection(XData);
 
   ::EmitUnwindInfo(Streamer, info);
 }
 
+static int64_t GetAbsDifference(MCStreamer &Streamer, const MCSymbol *LHS,
+                                const MCSymbol *RHS) {
+  MCContext &Context = Streamer.getContext();
+  const MCExpr *Diff =
+      MCBinaryExpr::createSub(MCSymbolRefExpr::create(LHS, Context),
+                              MCSymbolRefExpr::create(RHS, Context), Context);
+  MCObjectStreamer *OS = (MCObjectStreamer *)(&Streamer);
+  int64_t value;
+  Diff->evaluateAsAbsolute(value, OS->getAssembler());
+  return value;
+}
+
+static uint32_t
+ARM64CountOfUnwindCodes(const std::vector<WinEH::Instruction> &Insns) {
+  uint32_t Count = 0;
+  for (const auto &I : Insns) {
+    switch (static_cast<Win64EH::UnwindOpcodes>(I.Operation)) {
+    default:
+      llvm_unreachable("Unsupported ARM64 unwind code");
+    case Win64EH::UOP_AllocSmall:
+      Count += 1;
+      break;
+    case Win64EH::UOP_AllocMedium:
+      Count += 2;
+      break;
+    case Win64EH::UOP_AllocLarge:
+      Count += 4;
+      break;
+    case Win64EH::UOP_SaveFPLRX:
+      Count += 1;
+      break;
+    case Win64EH::UOP_SaveFPLR:
+      Count += 1;
+      break;
+    case Win64EH::UOP_SaveReg:
+      Count += 2;
+      break;
+    case Win64EH::UOP_SaveRegP:
+      Count += 2;
+      break;
+    case Win64EH::UOP_SaveRegPX:
+      Count += 2;
+      break;
+    case Win64EH::UOP_SaveRegX:
+      Count += 2;
+      break;
+    case Win64EH::UOP_SaveFReg:
+      Count += 2;
+      break;
+    case Win64EH::UOP_SaveFRegP:
+      Count += 2;
+      break;
+    case Win64EH::UOP_SaveFRegX:
+      Count += 2;
+      break;
+    case Win64EH::UOP_SaveFRegPX:
+      Count += 2;
+      break;
+    case Win64EH::UOP_SetFP:
+      Count += 1;
+      break;
+    case Win64EH::UOP_AddFP:
+      Count += 2;
+      break;
+    case Win64EH::UOP_Nop:
+      Count += 1;
+      break;
+    case Win64EH::UOP_End:
+      Count += 1;
+      break;
+    }
+  }
+  return Count;
+}
+
+// Unwind opcode encodings and restrictions are documented at
+// https://docs.microsoft.com/en-us/cpp/build/arm64-exception-handling
+static void ARM64EmitUnwindCode(MCStreamer &streamer, const MCSymbol *begin,
+                                WinEH::Instruction &inst) {
+  uint8_t b, reg;
+  switch (static_cast<Win64EH::UnwindOpcodes>(inst.Operation)) {
+  default:
+    llvm_unreachable("Unsupported ARM64 unwind code");
+  case Win64EH::UOP_AllocSmall:
+    b = (inst.Offset >> 4) & 0x1F;
+    streamer.EmitIntValue(b, 1);
+    break;
+  case Win64EH::UOP_AllocMedium: {
+    uint16_t hw = (inst.Offset >> 4) & 0x7FF;
+    b = 0xC0;
+    b |= (hw >> 8);
+    streamer.EmitIntValue(b, 1);
+    b = hw & 0xFF;
+    streamer.EmitIntValue(b, 1);
+    break;
+  }
+  case Win64EH::UOP_AllocLarge: {
+    uint32_t w;
+    b = 0xE0;
+    streamer.EmitIntValue(b, 1);
+    w = inst.Offset >> 4;
+    b = (w & 0x00FF0000) >> 16;
+    streamer.EmitIntValue(b, 1);
+    b = (w & 0x0000FF00) >> 8;
+    streamer.EmitIntValue(b, 1);
+    b = w & 0x000000FF;
+    streamer.EmitIntValue(b, 1);
+    break;
+  }
+  case Win64EH::UOP_SetFP:
+    b = 0xE1;
+    streamer.EmitIntValue(b, 1);
+    break;
+  case Win64EH::UOP_AddFP:
+    b = 0xE2;
+    streamer.EmitIntValue(b, 1);
+    b = (inst.Offset >> 3);
+    streamer.EmitIntValue(b, 1);
+    break;
+  case Win64EH::UOP_Nop:
+    b = 0xE3;
+    streamer.EmitIntValue(b, 1);
+    break;
+  case Win64EH::UOP_SaveFPLRX:
+    b = 0x80;
+    b |= ((inst.Offset - 1) >> 3) & 0x3F;
+    streamer.EmitIntValue(b, 1);
+    break;
+  case Win64EH::UOP_SaveFPLR:
+    b = 0x40;
+    b |= (inst.Offset >> 3) & 0x3F;
+    streamer.EmitIntValue(b, 1);
+    break;
+  case Win64EH::UOP_SaveReg:
+    assert(inst.Register >= 19 && "Saved reg must be >= 19");
+    reg = inst.Register - 19;
+    b = 0xD0 | ((reg & 0xC) >> 2);
+    streamer.EmitIntValue(b, 1);
+    b = ((reg & 0x3) << 6) | (inst.Offset >> 3);
+    streamer.EmitIntValue(b, 1);
+    break;
+  case Win64EH::UOP_SaveRegX:
+    assert(inst.Register >= 19 && "Saved reg must be >= 19");
+    reg = inst.Register - 19;
+    b = 0xD4 | ((reg & 0x8) >> 3);
+    streamer.EmitIntValue(b, 1);
+    b = ((reg & 0x7) << 5) | ((inst.Offset >> 3) - 1);
+    streamer.EmitIntValue(b, 1);
+    break;
+  case Win64EH::UOP_SaveRegP:
+    assert(inst.Register >= 19 && "Saved registers must be >= 19");
+    reg = inst.Register - 19;
+    b = 0xC8 | ((reg & 0xC) >> 2);
+    streamer.EmitIntValue(b, 1);
+    b = ((reg & 0x3) << 6) | (inst.Offset >> 3);
+    streamer.EmitIntValue(b, 1);
+    break;
+  case Win64EH::UOP_SaveRegPX:
+    assert(inst.Register >= 19 && "Saved registers must be >= 19");
+    reg = inst.Register - 19;
+    b = 0xCC | ((reg & 0xC) >> 2);
+    streamer.EmitIntValue(b, 1);
+    b = ((reg & 0x3) << 6) | ((inst.Offset >> 3) - 1);
+    streamer.EmitIntValue(b, 1);
+    break;
+  case Win64EH::UOP_SaveFReg:
+    assert(inst.Register >= 8 && "Saved dreg must be >= 8");
+    reg = inst.Register - 8;
+    b = 0xDC | ((reg & 0x4) >> 2);
+    streamer.EmitIntValue(b, 1);
+    b = ((reg & 0x3) << 6) | (inst.Offset >> 3);
+    streamer.EmitIntValue(b, 1);
+    break;
+  case Win64EH::UOP_SaveFRegX:
+    assert(inst.Register >= 8 && "Saved dreg must be >= 8");
+    reg = inst.Register - 8;
+    b = 0xDE;
+    streamer.EmitIntValue(b, 1);
+    b = ((reg & 0x7) << 5) | ((inst.Offset >> 3) - 1);
+    streamer.EmitIntValue(b, 1);
+    break;
+  case Win64EH::UOP_SaveFRegP:
+    assert(inst.Register >= 8 && "Saved dregs must be >= 8");
+    reg = inst.Register - 8;
+    b = 0xD8 | ((reg & 0x4) >> 2);
+    streamer.EmitIntValue(b, 1);
+    b = ((reg & 0x3) << 6) | (inst.Offset >> 3);
+    streamer.EmitIntValue(b, 1);
+    break;
+  case Win64EH::UOP_SaveFRegPX:
+    assert(inst.Register >= 8 && "Saved dregs must be >= 8");
+    reg = inst.Register - 8;
+    b = 0xDA | ((reg & 0x4) >> 2);
+    streamer.EmitIntValue(b, 1);
+    b = ((reg & 0x3) << 6) | ((inst.Offset >> 3) - 1);
+    streamer.EmitIntValue(b, 1);
+    break;
+  case Win64EH::UOP_End:
+    b = 0xE4;
+    streamer.EmitIntValue(b, 1);
+    break;
+  }
+}
+
+// Populate the .xdata section.  The format of .xdata on ARM64 is documented at
+// https://docs.microsoft.com/en-us/cpp/build/arm64-exception-handling
+static void ARM64EmitUnwindInfo(MCStreamer &streamer, WinEH::FrameInfo *info) {
+  // If this UNWIND_INFO already has a symbol, it's already been emitted.
+  if (info->Symbol)
+    return;
+
+  MCContext &context = streamer.getContext();
+  MCSymbol *Label = context.createTempSymbol();
+
+  streamer.EmitValueToAlignment(4);
+  streamer.EmitLabel(Label);
+  info->Symbol = Label;
+
+  uint32_t FuncLength = 0x0;
+  FuncLength = (uint32_t)GetAbsDifference(streamer, info->FuncletOrFuncEnd,
+                                          info->Begin);
+  if (FuncLength)
+    FuncLength /= 4;
+  uint32_t PrologCodeBytes = ARM64CountOfUnwindCodes(info->Instructions);
+  uint32_t TotalCodeBytes = PrologCodeBytes;
+
+  // Process epilogs.
+  MapVector<MCSymbol *, uint32_t> EpilogInfo;
+  for (auto &I : info->EpilogMap) {
+    MCSymbol *EpilogStart = I.first;
+    auto &EpilogInstrs = I.second;
+    uint32_t CodeBytes = ARM64CountOfUnwindCodes(EpilogInstrs);
+    EpilogInfo[EpilogStart] = TotalCodeBytes;
+    TotalCodeBytes += CodeBytes;
+  }
+
+  // Code Words, Epilog count, E, X, Vers, Function Length
+  uint32_t row1 = 0x0;
+  uint8_t CodeWords = TotalCodeBytes / 4;
+  uint8_t CodeWordsMod = TotalCodeBytes % 4;
+  if (CodeWordsMod)
+    CodeWords++;
+  uint32_t EpilogCount = info->EpilogMap.size();
+  bool ExtensionWord = EpilogCount > 31 || TotalCodeBytes > 124;
+  if (!ExtensionWord) {
+    row1 |= (EpilogCount & 0x1F) << 22;
+    row1 |= (CodeWords & 0x1F) << 27;
+  }
+  // E is always 0 right now, TODO: packed epilog setup
+  if (info->HandlesExceptions) // X
+    row1 |= 1 << 20;
+  row1 |= FuncLength & 0x3FFFF;
+  streamer.EmitIntValue(row1, 4);
+
+  // Extended Code Words, Extended Epilog Count
+  if (ExtensionWord) {
+    uint32_t row2 = 0x0;
+    row2 |= (CodeWords & 0xFF) << 16;
+    row2 |= (EpilogCount & 0xFFFF);
+    streamer.EmitIntValue(row2, 4);
+  }
+
+  // Epilog Start Index, Epilog Start Offset
+  for (auto &I : EpilogInfo) {
+    MCSymbol *EpilogStart = I.first;
+    uint32_t EpilogIndex = I.second;
+    uint32_t EpilogOffset =
+        (uint32_t)GetAbsDifference(streamer, EpilogStart, info->Begin);
+    if (EpilogOffset)
+      EpilogOffset /= 4;
+    uint32_t row3 = EpilogOffset;
+    row3 |= (EpilogIndex & 0x3FF) << 22;
+    streamer.EmitIntValue(row3, 4);
+  }
+
+  // Emit prolog unwind instructions (in reverse order).
+  uint8_t numInst = info->Instructions.size();
+  for (uint8_t c = 0; c < numInst; ++c) {
+    WinEH::Instruction inst = info->Instructions.back();
+    info->Instructions.pop_back();
+    ARM64EmitUnwindCode(streamer, info->Begin, inst);
+  }
+
+  // Emit epilog unwind instructions
+  for (auto &I : info->EpilogMap) {
+    auto &EpilogInstrs = I.second;
+    for (uint32_t i = 0; i < EpilogInstrs.size(); i++) {
+      WinEH::Instruction inst = EpilogInstrs[i];
+      ARM64EmitUnwindCode(streamer, info->Begin, inst);
+    }
+  }
+
+  int32_t BytesMod = CodeWords * 4 - TotalCodeBytes;
+  assert(BytesMod >= 0);
+  for (int i = 0; i < BytesMod; i++)
+    streamer.EmitIntValue(0xE3, 1);
+
+  if (info->HandlesExceptions)
+    streamer.EmitValue(
+        MCSymbolRefExpr::create(info->ExceptionHandler,
+                                MCSymbolRefExpr::VK_COFF_IMGREL32, context),
+        4);
+}
+
+static void ARM64EmitRuntimeFunction(MCStreamer &streamer,
+                                     const WinEH::FrameInfo *info) {
+  MCContext &context = streamer.getContext();
+
+  streamer.EmitValueToAlignment(4);
+  EmitSymbolRefWithOfs(streamer, info->Function, info->Begin);
+  streamer.EmitValue(MCSymbolRefExpr::create(info->Symbol,
+                                             MCSymbolRefExpr::VK_COFF_IMGREL32,
+                                             context),
+                     4);
+}
+
+void llvm::Win64EH::ARM64UnwindEmitter::Emit(MCStreamer &Streamer) const {
+  // Emit the unwind info structs first.
+  for (const auto &CFI : Streamer.getWinFrameInfos()) {
+    MCSection *XData = Streamer.getAssociatedXDataSection(CFI->TextSection);
+    Streamer.SwitchSection(XData);
+    ARM64EmitUnwindInfo(Streamer, CFI.get());
+  }
+
+  // Now emit RUNTIME_FUNCTION entries.
+  for (const auto &CFI : Streamer.getWinFrameInfos()) {
+    MCSection *PData = Streamer.getAssociatedPDataSection(CFI->TextSection);
+    Streamer.SwitchSection(PData);
+    ARM64EmitRuntimeFunction(Streamer, CFI.get());
+  }
+}
+
+void llvm::Win64EH::ARM64UnwindEmitter::EmitUnwindInfo(
+    MCStreamer &Streamer, WinEH::FrameInfo *info) const {
+  // Switch sections (the static function above is meant to be called from
+  // here and from Emit().
+  MCSection *XData = Streamer.getAssociatedXDataSection(info->TextSection);
+  Streamer.SwitchSection(XData);
+  ARM64EmitUnwindInfo(Streamer, info);
+}

llvm/trunk/lib/Target/AArch64/AArch64AsmPrinter.cpp

 #include "AArch64MCInstLower.h"
 #include "AArch64MachineFunctionInfo.h"
 #include "AArch64RegisterInfo.h"
 #include "AArch64Subtarget.h"
 #include "AArch64TargetObjectFile.h"
 #include "InstPrinter/AArch64InstPrinter.h"
 #include "MCTargetDesc/AArch64AddressingModes.h"
 #include "MCTargetDesc/AArch64MCTargetDesc.h"
+#include "MCTargetDesc/AArch64TargetStreamer.h"
 #include "Utils/AArch64BaseInfo.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/Triple.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/CodeGen/AsmPrinter.h"
 #include "llvm/CodeGen/MachineBasicBlock.h"
void AArch64AsmPrinter::EmitInstruction(const MachineInstr *MI) {
   if (AArch64FI->getLOHRelated().count(MI)) {
     // Generate a label for LOH related instruction
     MCSymbol *LOHLabel = createTempSymbol("loh");
     // Associate the instruction with the label
     LOHInstToLabel[MI] = LOHLabel;
     OutStreamer->EmitLabel(LOHLabel);
   }
 
+  AArch64TargetStreamer *TS =
+    static_cast<AArch64TargetStreamer *>(OutStreamer->getTargetStreamer());
   // Do any manual lowerings.
   switch (MI->getOpcode()) {
   default:
     break;
   case AArch64::MOVIv2d_ns:
     // If the target has <rdar://problem/16473581>, lower this
     // instruction to movi.16b instead.
     if (STI->hasZeroCycleZeroingFPWorkaround() &&
void AArch64AsmPrinter::EmitInstruction(const MachineInstr *MI) {
 
   case TargetOpcode::PATCHABLE_FUNCTION_EXIT:
     LowerPATCHABLE_FUNCTION_EXIT(*MI);
     return;
 
   case TargetOpcode::PATCHABLE_TAIL_CALL:
     LowerPATCHABLE_TAIL_CALL(*MI);
     return;
+
+  case AArch64::SEH_StackAlloc:
+    TS->EmitARM64WinCFIAllocStack(MI->getOperand(0).getImm());
+    return;
+
+  case AArch64::SEH_SaveFPLR:
+    TS->EmitARM64WinCFISaveFPLR(MI->getOperand(0).getImm());
+    return;
+
+  case AArch64::SEH_SaveFPLR_X:
+    assert(MI->getOperand(0).getImm() < 0 &&
+           "Pre increment SEH opcode must have a negative offset");
+    TS->EmitARM64WinCFISaveFPLRX(-MI->getOperand(0).getImm());
+    return;
+
+  case AArch64::SEH_SaveReg:
+    TS->EmitARM64WinCFISaveReg(MI->getOperand(0).getImm(),
+                               MI->getOperand(1).getImm());
+    return;
+
+  case AArch64::SEH_SaveReg_X:
+    assert(MI->getOperand(1).getImm() < 0 &&
+           "Pre increment SEH opcode must have a negative offset");
+    TS->EmitARM64WinCFISaveRegX(MI->getOperand(0).getImm(),
+                    -MI->getOperand(1).getImm());
+    return;
+
+  case AArch64::SEH_SaveRegP:
+    assert((MI->getOperand(1).getImm() - MI->getOperand(0).getImm() == 1) &&
+            "Non-consecutive registers not allowed for save_regp");
+    TS->EmitARM64WinCFISaveRegP(MI->getOperand(0).getImm(),
+                                MI->getOperand(2).getImm());
+    return;
+
+  case AArch64::SEH_SaveRegP_X:
+    assert((MI->getOperand(1).getImm() - MI->getOperand(0).getImm() == 1) &&
+            "Non-consecutive registers not allowed for save_regp_x");
+    assert(MI->getOperand(2).getImm() < 0 &&
+           "Pre increment SEH opcode must have a negative offset");
+    TS->EmitARM64WinCFISaveRegPX(MI->getOperand(0).getImm(),
+                                 -MI->getOperand(2).getImm());
+    return;
+
+  case AArch64::SEH_SaveFReg:
+    TS->EmitARM64WinCFISaveFReg(MI->getOperand(0).getImm(),
+                                MI->getOperand(1).getImm());
+    return;
+
+  case AArch64::SEH_SaveFReg_X:
+    assert(MI->getOperand(1).getImm() < 0 &&
+           "Pre increment SEH opcode must have a negative offset");
+    TS->EmitARM64WinCFISaveFRegX(MI->getOperand(0).getImm(),
+                                 -MI->getOperand(1).getImm());
+    return;
+
+  case AArch64::SEH_SaveFRegP:
+    assert((MI->getOperand(1).getImm() - MI->getOperand(0).getImm() == 1) &&
+            "Non-consecutive registers not allowed for save_regp");
+    TS->EmitARM64WinCFISaveFRegP(MI->getOperand(0).getImm(),
+                                 MI->getOperand(2).getImm());
+    return;
+
+  case AArch64::SEH_SaveFRegP_X:
+    assert((MI->getOperand(1).getImm() - MI->getOperand(0).getImm() == 1) &&
+            "Non-consecutive registers not allowed for save_regp_x");
+    assert(MI->getOperand(2).getImm() < 0 &&
+           "Pre increment SEH opcode must have a negative offset");
+    TS->EmitARM64WinCFISaveFRegPX(MI->getOperand(0).getImm(),
+                                  -MI->getOperand(2).getImm());
+    return;
+
+  case AArch64::SEH_SetFP:
+    TS->EmitARM64WinCFISetFP();
+    return;
+
+  case AArch64::SEH_AddFP:
+    TS->EmitARM64WinCFIAddFP(MI->getOperand(0).getImm());
+    return;
+
+  case AArch64::SEH_Nop:
+    TS->EmitARM64WinCFINop();
+    return;
+
+  case AArch64::SEH_PrologEnd:
+    TS->EmitARM64WinCFIPrologEnd();
+    return;
+
+  case AArch64::SEH_EpilogStart:
+    TS->EmitARM64WinCFIEpilogStart();
+    return;
+
+  case AArch64::SEH_EpilogEnd:
+    TS->EmitARM64WinCFIEpilogEnd();
+    return;
   }
 
   // Finally, do the automated lowerings for everything else.
   MCInst TmpInst;
   MCInstLowering.Lower(MI, TmpInst);
   EmitToStreamer(*OutStreamer, TmpInst);
 }
 
 // Force static initialization.
 extern "C" void LLVMInitializeAArch64AsmPrinter() {
   RegisterAsmPrinter<AArch64AsmPrinter> X(getTheAArch64leTarget());
   RegisterAsmPrinter<AArch64AsmPrinter> Y(getTheAArch64beTarget());
   RegisterAsmPrinter<AArch64AsmPrinter> Z(getTheARM64Target());
 }

llvm/trunk/lib/Target/AArch64/AArch64InstrInfo.td

def F128CSEL : Pseudo<(outs FPR128:$Rd),
                       [(set (f128 FPR128:$Rd),
                             (AArch64csel FPR128:$Rn, FPR128:$Rm,
                                        (i32 imm:$cond), NZCV))]> {
   let Uses = [NZCV];
   let usesCustomInserter = 1;
   let hasNoSchedulingInfo = 1;
 }
 
+//===----------------------------------------------------------------------===//
+// Instructions used for emitting unwind opcodes on ARM64 Windows.
+//===----------------------------------------------------------------------===//
+let isPseudo = 1 in {
+  def SEH_StackAlloc : Pseudo<(outs), (ins i32imm:$size), []>, Sched<[]>;
+  def SEH_SaveFPLR : Pseudo<(outs), (ins i32imm:$offs), []>, Sched<[]>;
+  def SEH_SaveFPLR_X : Pseudo<(outs), (ins i32imm:$offs), []>, Sched<[]>;
+  def SEH_SaveReg : Pseudo<(outs), (ins i32imm:$reg, i32imm:$offs), []>, Sched<[]>;
+  def SEH_SaveReg_X : Pseudo<(outs), (ins i32imm:$reg, i32imm:$offs), []>, Sched<[]>;
+  def SEH_SaveRegP : Pseudo<(outs), (ins i32imm:$reg0, i32imm:$reg1, i32imm:$offs), []>, Sched<[]>;
+  def SEH_SaveRegP_X : Pseudo<(outs), (ins i32imm:$reg0, i32imm:$reg1, i32imm:$offs), []>, Sched<[]>;
+  def SEH_SaveFReg : Pseudo<(outs), (ins i32imm:$reg, i32imm:$offs), []>, Sched<[]>;
+  def SEH_SaveFReg_X :  Pseudo<(outs), (ins i32imm:$reg, i32imm:$offs), []>, Sched<[]>;
+  def SEH_SaveFRegP : Pseudo<(outs), (ins i32imm:$reg0, i32imm:$reg1, i32imm:$offs), []>, Sched<[]>;
+  def SEH_SaveFRegP_X : Pseudo<(outs), (ins i32imm:$reg0, i32imm:$reg1, i32imm:$offs), []>, Sched<[]>;
+  def SEH_SetFP : Pseudo<(outs), (ins), []>, Sched<[]>;
+  def SEH_AddFP : Pseudo<(outs), (ins i32imm:$offs), []>, Sched<[]>;
+  def SEH_Nop : Pseudo<(outs), (ins), []>, Sched<[]>;
+  def SEH_PrologEnd : Pseudo<(outs), (ins), []>, Sched<[]>;
+  def SEH_EpilogStart : Pseudo<(outs), (ins), []>, Sched<[]>;
+  def SEH_EpilogEnd : Pseudo<(outs), (ins), []>, Sched<[]>;
+}
 
 //===----------------------------------------------------------------------===//
 // Floating point immediate move.
 //===----------------------------------------------------------------------===//
 
 let isReMaterializable = 1 in {
 defm FMOV : FPMoveImmediate<"fmov">;
 }

llvm/trunk/lib/Target/AArch64/MCTargetDesc/AArch64ELFStreamer.cpp

 AArch64TargetAsmStreamer::AArch64TargetAsmStreamer(MCStreamer &S,
                                                    formatted_raw_ostream &OS)
   : AArch64TargetStreamer(S), OS(OS) {}
 
 void AArch64TargetAsmStreamer::emitInst(uint32_t Inst) {
   OS << "\t.inst\t0x" << Twine::utohexstr(Inst) << "\n";
 }
 
-class AArch64TargetELFStreamer : public AArch64TargetStreamer {
-private:
-  AArch64ELFStreamer &getStreamer();
-
-  void emitInst(uint32_t Inst) override;
-
-public:
-  AArch64TargetELFStreamer(MCStreamer &S) : AArch64TargetStreamer(S) {}
-};
-
 /// Extend the generic ELFStreamer class so that it can emit mapping symbols at
 /// the appropriate points in the object files. These symbols are defined in the
 /// AArch64 ELF ABI:
 ///    infocenter.arm.com/help/topic/com.arm.doc.ihi0056a/IHI0056A_aaelf64.pdf
 ///
 /// In brief: $x or $d should be emitted at the start of each contiguous region
 /// of A64 code or data in a section. In practice, this emission does not rely
 /// on explicit assembler directives but on inherent properties of the
private:
   int64_t MappingSymbolCounter;
 
   DenseMap<const MCSection *, ElfMappingSymbol> LastMappingSymbols;
   ElfMappingSymbol LastEMS;
 };
 
 } // end anonymous namespace
 
+namespace llvm {
+
 AArch64ELFStreamer &AArch64TargetELFStreamer::getStreamer() {
   return static_cast<AArch64ELFStreamer &>(Streamer);
 }
 
 void AArch64TargetELFStreamer::emitInst(uint32_t Inst) {
   getStreamer().emitInst(Inst);
 }
 
-namespace llvm {
-
 MCTargetStreamer *createAArch64AsmTargetStreamer(MCStreamer &S,
                                                  formatted_raw_ostream &OS,
                                                  MCInstPrinter *InstPrint,
                                                  bool isVerboseAsm) {
   return new AArch64TargetAsmStreamer(S, OS);
 }
 
 MCELFStreamer *createAArch64ELFStreamer(MCContext &Context,
                                         std::unique_ptr<MCAsmBackend> TAB,
                                         std::unique_ptr<MCObjectWriter> OW,
                                         std::unique_ptr<MCCodeEmitter> Emitter,
                                         bool RelaxAll) {
   AArch64ELFStreamer *S = new AArch64ELFStreamer(
       Context, std::move(TAB), std::move(OW), std::move(Emitter));
   if (RelaxAll)
     S->getAssembler().setRelaxAll(true);
   return S;
 }
 
-MCTargetStreamer *
-createAArch64ObjectTargetStreamer(MCStreamer &S, const MCSubtargetInfo &STI) {
-  const Triple &TT = STI.getTargetTriple();
-  if (TT.isOSBinFormatELF())
-    return new AArch64TargetELFStreamer(S);
-  if (TT.isOSBinFormatCOFF())
-    return new AArch64TargetWinCOFFStreamer(S);
-  return nullptr;
-}
-
 } // end namespace llvm

llvm/trunk/lib/Target/AArch64/MCTargetDesc/AArch64MCAsmInfo.cpp

AArch64MCAsmInfoMicrosoftCOFF::AArch64MCAsmInfoMicrosoftCOFF() {
   Data64bitsDirective = "\t.xword\t";
 
   AlignmentIsInBytes = false;
   SupportsDebugInformation = true;
   CodePointerSize = 8;
 
   CommentString = ";";
   ExceptionsType = ExceptionHandling::WinEH;
+  WinEHEncodingType = WinEH::EncodingType::Itanium;
 }
 
 AArch64MCAsmInfoGNUCOFF::AArch64MCAsmInfoGNUCOFF() {
   PrivateGlobalPrefix = ".L";
   PrivateLabelPrefix = ".L";
 
   Data16bitsDirective = "\t.hword\t";
   Data32bitsDirective = "\t.word\t";

llvm/trunk/lib/Target/AArch64/MCTargetDesc/AArch64TargetStreamer.h

 //===-- AArch64TargetStreamer.h - AArch64 Target Streamer ------*- C++ -*--===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_LIB_TARGET_AARCH64_MCTARGETDESC_AARCH64TARGETSTREAMER_H
 #define LLVM_LIB_TARGET_AARCH64_MCTARGETDESC_AARCH64TARGETSTREAMER_H
 
 #include "llvm/MC/MCStreamer.h"
 
+namespace {
+class AArch64ELFStreamer;
+}
+
 namespace llvm {
 
 class AArch64TargetStreamer : public MCTargetStreamer {
 public:
   AArch64TargetStreamer(MCStreamer &S);
   ~AArch64TargetStreamer() override;
 
   void finish() override;
 
   /// Callback used to implement the ldr= pseudo.
   /// Add a new entry to the constant pool for the current section and return an
   /// MCExpr that can be used to refer to the constant pool location.
   const MCExpr *addConstantPoolEntry(const MCExpr *, unsigned Size, SMLoc Loc);
 
   /// Callback used to implemnt the .ltorg directive.
   /// Emit contents of constant pool for the current section.
   void emitCurrentConstantPool();
 
   /// Callback used to implement the .inst directive.
   virtual void emitInst(uint32_t Inst);
 
+  virtual void EmitARM64WinCFIAllocStack(unsigned Size) {}
+  virtual void EmitARM64WinCFISaveFPLR(int Offset) {}
+  virtual void EmitARM64WinCFISaveFPLRX(int Offset) {}
+  virtual void EmitARM64WinCFISaveReg(unsigned Reg, int Offset) {}
+  virtual void EmitARM64WinCFISaveRegX(unsigned Reg, int Offset) {}
+  virtual void EmitARM64WinCFISaveRegP(unsigned Reg, int Offset) {}
+  virtual void EmitARM64WinCFISaveRegPX(unsigned Reg, int Offset) {}
+  virtual void EmitARM64WinCFISaveFReg(unsigned Reg, int Offset) {}
+  virtual void EmitARM64WinCFISaveFRegX(unsigned Reg, int Offset) {}
+  virtual void EmitARM64WinCFISaveFRegP(unsigned Reg, int Offset) {}
+  virtual void EmitARM64WinCFISaveFRegPX(unsigned Reg, int Offset) {}
+  virtual void EmitARM64WinCFISetFP() {}
+  virtual void EmitARM64WinCFIAddFP(unsigned Size) {}
+  virtual void EmitARM64WinCFINop() {}
+  virtual void EmitARM64WinCFIPrologEnd() {}
+  virtual void EmitARM64WinCFIEpilogStart() {}
+  virtual void EmitARM64WinCFIEpilogEnd() {}
+
 private:
   std::unique_ptr<AssemblerConstantPools> ConstantPools;
 };
 
+class AArch64TargetELFStreamer : public AArch64TargetStreamer {
+private:
+  AArch64ELFStreamer &getStreamer();
+
+  void emitInst(uint32_t Inst) override;
+
+public:
+  AArch64TargetELFStreamer(MCStreamer &S) : AArch64TargetStreamer(S) {}
+};
+
+class AArch64TargetWinCOFFStreamer : public llvm::AArch64TargetStreamer {
+private:
+  // True if we are processing SEH directives in an epilogue.
+  bool InEpilogCFI = false;
+
+  // Symbol of the current epilog for which we are processing SEH directives.
+  MCSymbol *CurrentEpilog = nullptr;
+public:
+  AArch64TargetWinCOFFStreamer(llvm::MCStreamer &S)
+    : AArch64TargetStreamer(S) {}
+
+  // The unwind codes on ARM64 Windows are documented at
+  // https://docs.microsoft.com/en-us/cpp/build/arm64-exception-handling
+  void EmitARM64WinCFIAllocStack(unsigned Size) override;
+  void EmitARM64WinCFISaveFPLR(int Offset) override;
+  void EmitARM64WinCFISaveFPLRX(int Offset) override;
+  void EmitARM64WinCFISaveReg(unsigned Reg, int Offset) override;
+  void EmitARM64WinCFISaveRegX(unsigned Reg, int Offset) override;
+  void EmitARM64WinCFISaveRegP(unsigned Reg, int Offset) override;
+  void EmitARM64WinCFISaveRegPX(unsigned Reg, int Offset) override;
+  void EmitARM64WinCFISaveFReg(unsigned Reg, int Offset) override;
+  void EmitARM64WinCFISaveFRegX(unsigned Reg, int Offset) override;
+  void EmitARM64WinCFISaveFRegP(unsigned Reg, int Offset) override;
+  void EmitARM64WinCFISaveFRegPX(unsigned Reg, int Offset) override;
+  void EmitARM64WinCFISetFP() override;
+  void EmitARM64WinCFIAddFP(unsigned Size) override;
+  void EmitARM64WinCFINop() override;
+  void EmitARM64WinCFIPrologEnd() override;
+  void EmitARM64WinCFIEpilogStart() override;
+  void EmitARM64WinCFIEpilogEnd() override;
+private:
+  void EmitARM64WinUnwindCode(unsigned UnwindCode, int Reg, int Offset);
+};
+
+MCTargetStreamer *
+createAArch64ObjectTargetStreamer(MCStreamer &S, const MCSubtargetInfo &STI);
+
 } // end namespace llvm
 
 #endif

llvm/trunk/lib/Target/AArch64/MCTargetDesc/AArch64TargetStreamer.cpp

 //===- AArch64TargetStreamer.cpp - AArch64TargetStreamer class ------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements the AArch64TargetStreamer class.
 //
 //===----------------------------------------------------------------------===//
 
 #include "AArch64TargetStreamer.h"
 #include "llvm/MC/ConstantPools.h"
+#include "llvm/MC/MCSubtargetInfo.h"
 
 using namespace llvm;
 
 //
 // AArch64TargetStreamer Implemenation
 //
 AArch64TargetStreamer::AArch64TargetStreamer(MCStreamer &S)
     : MCTargetStreamer(S), ConstantPools(new AssemblerConstantPools()) {}
void AArch64TargetStreamer::emitInst(uint32_t Inst) {
   // little-endian).
   for (unsigned I = 0; I < 4; ++I) {
     Buffer[I] = uint8_t(Inst);
     Inst >>= 8;
   }
 
   getStreamer().EmitBytes(StringRef(Buffer, 4));
 }
+
+namespace llvm {
+
+MCTargetStreamer *
+createAArch64ObjectTargetStreamer(MCStreamer &S, const MCSubtargetInfo &STI) {
+  const Triple &TT = STI.getTargetTriple();
+  if (TT.isOSBinFormatELF())
+    return new AArch64TargetELFStreamer(S);
+  if (TT.isOSBinFormatCOFF())
+    return new AArch64TargetWinCOFFStreamer(S);
+  return nullptr;
+}
+
+} // end namespace llvm

llvm/trunk/lib/Target/AArch64/MCTargetDesc/AArch64WinCOFFStreamer.h

 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_LIB_TARGET_AARCH64_MCTARGETDESC_AARCH64WINCOFFSTREAMER_H
 #define LLVM_LIB_TARGET_AARCH64_MCTARGETDESC_AARCH64WINCOFFSTREAMER_H
 
 #include "AArch64TargetStreamer.h"
 #include "llvm/MC/MCWinCOFFStreamer.h"
 
-namespace {
-class AArch64WinCOFFStreamer;
-
-class AArch64TargetWinCOFFStreamer : public llvm::AArch64TargetStreamer {
-private:
-  AArch64WinCOFFStreamer &getStreamer();
-
-public:
-  AArch64TargetWinCOFFStreamer(llvm::MCStreamer &S)
-    : AArch64TargetStreamer(S) {}
-};
-
-} // end anonymous namespace
-
 namespace llvm {
 
 MCWinCOFFStreamer *createAArch64WinCOFFStreamer(
     MCContext &Context, std::unique_ptr<MCAsmBackend> TAB,
     std::unique_ptr<MCObjectWriter> OW, std::unique_ptr<MCCodeEmitter> Emitter,
     bool RelaxAll, bool IncrementalLinkerCompatible);
 } // end llvm namespace
 
 #endif

llvm/trunk/lib/Target/AArch64/MCTargetDesc/AArch64WinCOFFStreamer.cpp

 //===-- AArch64WinCOFFStreamer.cpp - ARM Target WinCOFF Streamer ----*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "AArch64WinCOFFStreamer.h"
 #include "llvm/MC/MCAsmBackend.h"
 #include "llvm/MC/MCCodeEmitter.h"
 #include "llvm/MC/MCObjectWriter.h"
+#include "llvm/MC/MCWin64EH.h"
+#include "llvm/MC/MCWinCOFFStreamer.h"
 
 using namespace llvm;
 
 namespace {
 
 class AArch64WinCOFFStreamer : public MCWinCOFFStreamer {
+  Win64EH::ARM64UnwindEmitter EHStreamer;
+
 public:
   friend class AArch64TargetWinCOFFStreamer;
 
   AArch64WinCOFFStreamer(MCContext &C, std::unique_ptr<MCAsmBackend> AB,
                          std::unique_ptr<MCCodeEmitter> CE,
                          std::unique_ptr<MCObjectWriter> OW)
       : MCWinCOFFStreamer(C, std::move(AB), std::move(CE), std::move(OW)) {}
 
+  void EmitWinEHHandlerData(SMLoc Loc) override;
+  void EmitWindowsUnwindTables() override;
   void FinishImpl() override;
 };
 
+void AArch64WinCOFFStreamer::EmitWinEHHandlerData(SMLoc Loc) {
+  MCStreamer::EmitWinEHHandlerData(Loc);
+
+  // We have to emit the unwind info now, because this directive
+  // actually switches to the .xdata section!
+  EHStreamer.EmitUnwindInfo(*this, getCurrentWinFrameInfo());
+}
+
+void AArch64WinCOFFStreamer::EmitWindowsUnwindTables() {
+  if (!getNumWinFrameInfos())
+    return;
+  EHStreamer.Emit(*this);
+}
+
 void AArch64WinCOFFStreamer::FinishImpl() {
   EmitFrames(nullptr);
+  EmitWindowsUnwindTables();
 
   MCWinCOFFStreamer::FinishImpl();
 }
 } // end anonymous namespace
 
 namespace llvm {
+
+// Helper function to common out unwind code setup for those codes that can
+// belong to both prolog and epilog.
+// There are three types of Windows ARM64 SEH codes.  They can
+// 1) take no operands: SEH_Nop, SEH_PrologEnd, SEH_EpilogStart, SEH_EpilogEnd
+// 2) take an offset: SEH_StackAlloc, SEH_SaveFPLR, SEH_SaveFPLR_X
+// 3) take a register and an offset/size: all others
+void AArch64TargetWinCOFFStreamer::EmitARM64WinUnwindCode(unsigned UnwindCode,
+                                                          int Reg,
+                                                          int Offset) {
+  auto &S = getStreamer();
+  WinEH::FrameInfo *CurFrame = S.EnsureValidWinFrameInfo(SMLoc());
+  if (!CurFrame)
+    return;
+  MCSymbol *Label = S.EmitCFILabel();
+  auto Inst = WinEH::Instruction(UnwindCode, Label, Reg, Offset);
+  if (InEpilogCFI)
+    CurFrame->EpilogMap[CurrentEpilog].push_back(Inst);
+  else
+    CurFrame->Instructions.push_back(Inst);
+}
+
+void AArch64TargetWinCOFFStreamer::EmitARM64WinCFIAllocStack(unsigned Size) {
+  unsigned Op = Win64EH::UOP_AllocSmall;
+  if (Size >= 16384)
+    Op = Win64EH::UOP_AllocLarge;
+  else if (Size >= 512)
+    Op = Win64EH::UOP_AllocMedium;
+  EmitARM64WinUnwindCode(Op, -1, Size);
+}
+
+void AArch64TargetWinCOFFStreamer::EmitARM64WinCFISaveFPLR(int Offset) {
+  EmitARM64WinUnwindCode(Win64EH::UOP_SaveFPLR, -1, Offset);
+}
+
+void AArch64TargetWinCOFFStreamer::EmitARM64WinCFISaveFPLRX(int Offset) {
+  EmitARM64WinUnwindCode(Win64EH::UOP_SaveFPLRX, -1, Offset);
+}
+
+void AArch64TargetWinCOFFStreamer::EmitARM64WinCFISaveReg(unsigned Reg,
+                                                          int Offset) {
+  assert(Offset >= 0 && Offset <= 504 &&
+        "Offset for save reg should be >= 0 && <= 504");
+  EmitARM64WinUnwindCode(Win64EH::UOP_SaveReg, Reg, Offset);
+}
+
+void AArch64TargetWinCOFFStreamer::EmitARM64WinCFISaveRegX(unsigned Reg,
+                                                           int Offset) {
+  EmitARM64WinUnwindCode(Win64EH::UOP_SaveRegX, Reg, Offset);
+}
+
+void AArch64TargetWinCOFFStreamer::EmitARM64WinCFISaveRegP(unsigned Reg,
+                                                           int Offset) {
+  EmitARM64WinUnwindCode(Win64EH::UOP_SaveRegP, Reg, Offset);
+}
+
+void AArch64TargetWinCOFFStreamer::EmitARM64WinCFISaveRegPX(unsigned Reg,
+                                                            int Offset) {
+  EmitARM64WinUnwindCode(Win64EH::UOP_SaveRegPX, Reg, Offset);
+}
+
+void AArch64TargetWinCOFFStreamer::EmitARM64WinCFISaveFReg(unsigned Reg,
+                                                           int Offset) {
+  assert(Offset >= 0 && Offset <= 504 &&
+        "Offset for save reg should be >= 0 && <= 504");
+  EmitARM64WinUnwindCode(Win64EH::UOP_SaveFReg, Reg, Offset);
+}
+
+void AArch64TargetWinCOFFStreamer::EmitARM64WinCFISaveFRegX(unsigned Reg,
+                                                            int Offset) {
+  EmitARM64WinUnwindCode(Win64EH::UOP_SaveFRegX, Reg, Offset);
+}
+
+void AArch64TargetWinCOFFStreamer::EmitARM64WinCFISaveFRegP(unsigned Reg,
+                                                            int Offset) {
+  EmitARM64WinUnwindCode(Win64EH::UOP_SaveFRegP, Reg, Offset);
+}
+
+void AArch64TargetWinCOFFStreamer::EmitARM64WinCFISaveFRegPX(unsigned Reg,
+                                                             int Offset) {
+  EmitARM64WinUnwindCode(Win64EH::UOP_SaveFRegPX, Reg, Offset);
+}
+
+void AArch64TargetWinCOFFStreamer::EmitARM64WinCFISetFP() {
+  EmitARM64WinUnwindCode(Win64EH::UOP_SetFP, -1, 0);
+}
+
+void AArch64TargetWinCOFFStreamer::EmitARM64WinCFIAddFP(unsigned Offset) {
+  assert(Offset <= 2040 && "UOP_AddFP must have offset <= 2040");
+  EmitARM64WinUnwindCode(Win64EH::UOP_AddFP, -1, Offset);
+}
+
+void AArch64TargetWinCOFFStreamer::EmitARM64WinCFINop() {
+  EmitARM64WinUnwindCode(Win64EH::UOP_Nop, -1, 0);
+}
+
+// The functions below handle opcodes that can end up in either a prolog or
+// an epilog, but not both.
+void AArch64TargetWinCOFFStreamer::EmitARM64WinCFIPrologEnd() {
+  auto &S = getStreamer();
+  WinEH::FrameInfo *CurFrame = S.EnsureValidWinFrameInfo(SMLoc());
+  if (!CurFrame)
+    return;
+
+  MCSymbol *Label = S.EmitCFILabel();
+  CurFrame->PrologEnd = Label;
+  WinEH::Instruction Inst = WinEH::Instruction(Win64EH::UOP_End, Label, -1, 0);
+  auto it = CurFrame->Instructions.begin();
+  CurFrame->Instructions.insert(it, Inst);
+}
+
+void AArch64TargetWinCOFFStreamer::EmitARM64WinCFIEpilogStart() {
+  auto &S = getStreamer();
+  WinEH::FrameInfo *CurFrame = S.EnsureValidWinFrameInfo(SMLoc());
+  if (!CurFrame)
+    return;
+
+  InEpilogCFI = true;
+  CurrentEpilog = S.EmitCFILabel();
+}
+
+void AArch64TargetWinCOFFStreamer::EmitARM64WinCFIEpilogEnd() {
+  auto &S = getStreamer();
+  WinEH::FrameInfo *CurFrame = S.EnsureValidWinFrameInfo(SMLoc());
+  if (!CurFrame)
+    return;
+
+  InEpilogCFI = false;
+  MCSymbol *Label = S.EmitCFILabel();
+  WinEH::Instruction Inst = WinEH::Instruction(Win64EH::UOP_End, Label, -1, 0);
+  CurFrame->EpilogMap[CurrentEpilog].push_back(Inst);
+  CurrentEpilog = nullptr;
+}
+
 MCWinCOFFStreamer *createAArch64WinCOFFStreamer(
     MCContext &Context, std::unique_ptr<MCAsmBackend> MAB,
     std::unique_ptr<MCObjectWriter> OW, std::unique_ptr<MCCodeEmitter> Emitter,
     bool RelaxAll, bool IncrementalLinkerCompatible) {
   auto *S = new AArch64WinCOFFStreamer(Context, std::move(MAB),
                                        std::move(Emitter), std::move(OW));
   S->getAssembler().setIncrementalLinkerCompatible(IncrementalLinkerCompatible);
   return S;
 }
 
 } // end llvm namespace

llvm/trunk/test/CodeGen/AArch64/wineh1.mir

+# RUN: llc -o - %s -mtriple=aarch64-windows -start-after=prologepilog -filetype=obj -disable-post-ra \
+# RUN:   | llvm-readobj -unwind | FileCheck %s
+# This test case checks the basic validity of the .xdata section.  It's
+# documented at:
+# https://docs.microsoft.com/en-us/cpp/build/arm64-exception-handling
+
+# We expect to see the following in the .xdata section:
+
+# CHECK:   ExceptionData {
+# CHECK-NEXT:      FunctionLength: 92
+# CHECK-NEXT:      Version: 0
+# CHECK-NEXT:      ExceptionData: No
+# CHECK-NEXT:      EpiloguePacked: No
+# CHECK-NEXT:      EpilogueScopes: 1
+# CHECK-NEXT:      ByteCodeLength: 28
+# CHECK-NEXT:      Prologue [
+# CHECK-NEXT:        0xc808              ; stp x19, x20, [sp, #64]
+# CHECK-NEXT:        0xd0c7              ; str x22, [sp, #56]
+# CHECK-NEXT:        0xd086              ; str x21, [sp, #48]
+# CHECK-NEXT:        0xc904              ; stp x23, x24, [sp, #32]
+# CHECK-NEXT:        0xc982              ; stp x25, x26, [sp, #16]
+# CHECK-NEXT:        0xce09              ; stp x27, x28, [sp, #-80]!
+# CHECK-NEXT:        0xe4                ; end
+# CHECK-NEXT:      ]
+# CHECK-NEXT:      EpilogueScopes [
+# CHECK-NEXT:        EpilogueScope {
+# CHECK-NEXT:          StartOffset: 15
+# CHECK-NEXT:          EpilogueStartIndex: 13
+# CHECK-NEXT:          Opcodes [
+# CHECK-NEXT:            0xc808              ; ldp x19, x20, [sp, #64]
+# CHECK-NEXT:            0xd086              ; ldr x21, [sp, #48]
+# CHECK-NEXT:            0xe3                ; nop
+# CHECK-NEXT:            0xd0c7              ; ldr x22, [sp, #56]
+# CHECK-NEXT:            0xc904              ; ldp x23, x24, [sp, #32]
+# CHECK-NEXT:            0xc982              ; ldp x25, x26, [sp, #16]
+# CHECK-NEXT:            0xce09              ; ldp x27, x28, [sp], #80
+# CHECK-NEXT:            0xe4                ; end
+# CHECK-NEXT:          ]
+# CHECK-NEXT:        }
+# CHECK-NEXT:      ]
+# CHECK-NEXT:    }
+...
+---
+name:            test
+alignment:       2
+tracksRegLiveness: true
+hasWinCFI: true
+liveins:
+  - { reg: '$w0' }
+frameInfo:
+  stackSize:       80
+  maxAlignment:    8
+  maxCallFrameSize: 0
+  hasOpaqueSPAdjustment: true
+stack:
+  - { id: 0, type: spill-slot, offset: -8, size: 8, alignment: 8, stack-id: 0,
+      callee-saved-register: '$x19' }
+  - { id: 1, type: spill-slot, offset: -16, size: 8, alignment: 8, stack-id: 0,
+      callee-saved-register: '$x20' }
+  - { id: 2, type: spill-slot, offset: -24, size: 8, alignment: 8, stack-id: 0,
+      callee-saved-register: '$x21' }
+  - { id: 3, type: spill-slot, offset: -32, size: 8, alignment: 8, stack-id: 0,
+      callee-saved-register: '$x22' }
+  - { id: 4, type: spill-slot, offset: -40, size: 8, alignment: 8, stack-id: 0,
+      callee-saved-register: '$x23' }
+  - { id: 5, type: spill-slot, offset: -48, size: 8, alignment: 8, stack-id: 0,
+      callee-saved-register: '$x24' }
+  - { id: 6, type: spill-slot, offset: -56, size: 8, alignment: 8, stack-id: 0,
+      callee-saved-register: '$x25' }
+  - { id: 7, type: spill-slot, offset: -64, size: 8, alignment: 8, stack-id: 0,
+      callee-saved-register: '$x26' }
+  - { id: 8, type: spill-slot, offset: -72, size: 8, alignment: 8, stack-id: 0,
+      callee-saved-register: '$x27' }
+  - { id: 9, type: spill-slot, offset: -80, size: 8, alignment: 8, stack-id: 0,
+      callee-saved-register: '$x28' }
+body:             |
+  bb.0.entry:
+    liveins: $x0, $x1, $x27, $x28, $x25, $x26, $x23, $x24, $x21, $x22, $x19, $x20
+    early-clobber $sp = frame-setup STPXpre killed $x27, killed $x28, $sp, -10 :: (store 8 into %stack.8), (store 8 into %stack.9)
+    frame-setup SEH_SaveRegP_X 27, 28, -80
+    frame-setup STPXi killed $x25, killed $x26, $sp, 2 :: (store 8 into %stack.6), (store 8 into %stack.7)
+    frame-setup SEH_SaveRegP 25, 26, 16
+    frame-setup STPXi killed $x23, killed $x24, $sp, 4 :: (store 8 into %stack.4), (store 8 into %stack.5)
+    frame-setup SEH_SaveRegP 23, 24, 32
+    frame-setup STRXui killed $x21, $sp, 6 :: (store 8 into %stack.2)
+    frame-setup SEH_SaveReg 21, 48
+    frame-setup STRXui killed $x22, $sp, 7 :: (store 8 into %stack.3)
+    frame-setup SEH_SaveReg 22, 56
+    frame-setup STPXi killed $x19, killed $x20, $sp, 8 :: (store 8 into %stack.0), (store 8 into %stack.1)
+    frame-setup SEH_SaveRegP 19, 20, 64
+    frame-setup SEH_PrologEnd
+    $x19 = ADDXrr $x0, killed $x1
+    $x20 = ADDXrr $x19, killed $x0
+    $x21 = ADDXrr $x20, killed $x19
+    $x22 = ADDXrr $x21, killed $x20
+    $x23 = ADDXrr $x22, killed $x21
+    $x24 = ADDXrr $x23, killed $x22
+    $x25 = ADDXrr $x24, killed $x23
+    $x26 = ADDXrr $x25, killed $x24
+    $x27 = ADDXrr $x26, killed $x25
+    $x28 = ADDXrr $x27, killed $x26
+    frame-destroy SEH_EpilogStart
+    $x19, $x20 = frame-destroy LDPXi $sp, 8 :: (load 8 from %stack.0), (load 8 from %stack.1)
+    frame-destroy SEH_SaveRegP 19, 20, 64
+    $x21 = frame-destroy LDRXui $sp, 6 :: (load 8 from %stack.2)
+    frame-destroy SEH_SaveReg 21, 48
+    $x0 = COPY $x28
+    frame-destroy SEH_Nop
+    $x21 = frame-destroy LDRXui $sp, 6 :: (load 8 from %stack.2)
+    frame-destroy SEH_SaveReg 22, 56
+    $x23, $x24 = frame-destroy LDPXi $sp, 4 :: (load 8 from %stack.4), (load 8 from %stack.5)
+    frame-destroy SEH_SaveRegP 23, 24, 32
+    $x25, $x26 = frame-destroy LDPXi $sp, 2 :: (load 8 from %stack.6), (load 8 from %stack.7)
+    frame-destroy SEH_SaveRegP 25, 26, 16
+    early-clobber $sp, $x27, $x28 = frame-destroy LDPXpost $sp, 10 :: (load 8 from %stack.8), (load 8 from %stack.9)
+    frame-destroy SEH_SaveRegP_X 27, 28, -80
+    frame-destroy SEH_EpilogEnd
+    RET_ReallyLR implicit $x0
+
+...

llvm/trunk/test/CodeGen/AArch64/wineh2.mir

+# RUN: llc -o - %s -mtriple=aarch64-windows -start-after=prologepilog \
+# RUN:   -disable-post-ra -filetype=obj | llvm-readobj -unwind | FileCheck %s
+# Test that the pre/post increment save of a flating point register is correct.
+
+# CHECK:        ExceptionData {
+# CHECK-NEXT:      FunctionLength: 136
+# CHECK-NEXT:      Version: 0
+# CHECK-NEXT:      ExceptionData: No
+# CHECK-NEXT:      EpiloguePacked: No
+# CHECK-NEXT:      EpilogueScopes: 1
+# CHECK-NEXT:      ByteCodeLength: 40
+# CHECK-NEXT:      Prologue [
+# CHECK-NEXT:        0xc80e              ; stp x19, x20, [sp, #112]
+# CHECK-NEXT:        0xc88c              ; stp x21, x22, [sp, #96]
+# CHECK-NEXT:        0xc90a              ; stp x23, x24, [sp, #80]
+# CHECK-NEXT:        0xc988              ; stp x25, x26, [sp, #64]
+# CHECK-NEXT:        0xca06              ; stp x27, x28, [sp, #48]
+# CHECK-NEXT:        0xdc45              ; str d9, [sp, #40]
+# CHECK-NEXT:        0xdc04              ; str d8, [sp, #32]
+# CHECK-NEXT:        0xd882              ; stp d10, d11, [sp, #16]
+# CHECK-NEXT:        0xde8f              ; str d12, [sp, #-128]!
+# CHECK-NEXT:        0xe4                ; end
+# CHECK-NEXT:      ]
+# CHECK-NEXT:      EpilogueScopes [
+# CHECK-NEXT:        EpilogueScope {
+# CHECK-NEXT:          StartOffset: 25
+# CHECK-NEXT:          EpilogueStartIndex: 19
+# CHECK-NEXT:          Opcodes [
+# CHECK-NEXT:            0xc80e              ; ldp x19, x20, [sp, #112]
+# CHECK-NEXT:            0xc88c              ; ldp x21, x22, [sp, #96]
+# CHECK-NEXT:            0xc90a              ; ldp x23, x24, [sp, #80]
+# CHECK-NEXT:            0xc988              ; ldp x25, x26, [sp, #64]
+# CHECK-NEXT:            0xca06              ; ldp x27, x28, [sp, #48]
+# CHECK-NEXT:            0xdc04              ; ldr d8, [sp, #32]
+# CHECK-NEXT:            0xdc45              ; ldr d9, [sp, #40]
+# CHECK-NEXT:            0xd882              ; ldp d10, d11, [sp, #16]
+# CHECK-NEXT:            0xde8f              ; ldr d12, [sp], #128
+# CHECK-NEXT:            0xe4                ; end
+# CHECK-NEXT:          ]
+# CHECK-NEXT:        }
+# CHECK-NEXT:      ]
+# CHECK-NEXT:    }
+...
+---
+name:            test
+alignment:       2
+exposesReturnsTwice: false
+legalized:       false
+regBankSelected: false
+selected:        false
+failedISel:      false
+tracksRegLiveness: true
+hasWinCFI:       true
+registers:
+liveins:
+  - { reg: '$w0', virtual-reg: '' }
+frameInfo:
+  isFrameAddressTaken: false
+  isReturnAddressTaken: false
+  hasStackMap:     false
+  hasPatchPoint:   false
+  stackSize:       128
+  offsetAdjustment: 0
+  maxAlignment:    16
+  adjustsStack:    false
+  hasCalls:        false
+  stackProtector:  ''
+  maxCallFrameSize: 0
+  hasOpaqueSPAdjustment: true
+  hasVAStart:      false
+  hasMustTailInVarArgFunc: false
+  localFrameSize:  0
+  savePoint:       ''
+  restorePoint:    ''
+fixedStack:
+stack:
+  - { id: 0, name: '', type: spill-slot, offset: -8, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x19', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 1, name: '', type: spill-slot, offset: -16, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x20', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 2, name: '', type: spill-slot, offset: -24, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x21', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 3, name: '', type: spill-slot, offset: -32, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x22', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 4, name: '', type: spill-slot, offset: -40, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x23', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 5, name: '', type: spill-slot, offset: -48, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x24', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 6, name: '', type: spill-slot, offset: -56, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x25', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 7, name: '', type: spill-slot, offset: -64, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x26', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 8, name: '', type: spill-slot, offset: -72, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x27', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 9, name: '', type: spill-slot, offset: -80, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x28', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 10, name: '', type: spill-slot, offset: -88, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$d8', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 11, name: '', type: spill-slot, offset: -96, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$d9', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 12, name: '', type: spill-slot, offset: -104, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$d10', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 13, name: '', type: spill-slot, offset: -112, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$d11', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 14, name: '', type: spill-slot, offset: -128, size: 8, alignment: 16,
+      stack-id: 0, callee-saved-register: '$d12', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+constants:
+body:             |
+  bb.0.entry:
+    liveins: $x0, $x1, $d0, $d1, $d8, $d9, $d10, $d11, $d12, $x19, $x20, $x21, $x22, $x23, $x24, $x25, $x26, $x27, $x28
+
+    early-clobber $sp = frame-setup STRDpre killed $d12, $sp, -128 :: (store 8 into %stack.14)
+    frame-setup SEH_SaveFReg_X 12, -128
+    frame-setup STPDi killed $d10, killed $d11, $sp, 2 :: (store 8 into %stack.12), (store 8 into %stack.13)
+    frame-setup SEH_SaveFRegP 10, 11, 16
+    frame-setup STRDui killed $d8, $sp, 4 :: (store 8 into %stack.10)
+    frame-setup SEH_SaveFReg 8, 32
+    frame-setup STRDui killed $d9, $sp, 5 :: (store 8 into %stack.11)
+    frame-setup SEH_SaveFReg 9, 40
+    frame-setup STPXi killed $x27, killed $x28, $sp, 6 :: (store 8 into %stack.8), (store 8 into %stack.9)
+    frame-setup SEH_SaveRegP 27, 28, 48
+    frame-setup STPXi killed $x25, killed $x26, $sp, 8 :: (store 8 into %stack.6), (store 8 into %stack.7)
+    frame-setup SEH_SaveRegP 25, 26, 64
+    frame-setup STPXi killed $x23, killed $x24, $sp, 10 :: (store 8 into %stack.4), (store 8 into %stack.5)
+    frame-setup SEH_SaveRegP 23, 24, 80
+    frame-setup STPXi killed $x21, killed $x22, $sp, 12 :: (store 8 into %stack.2), (store 8 into %stack.3)
+    frame-setup SEH_SaveRegP 21, 22, 96
+    frame-setup STPXi killed $x19, killed $x20, $sp, 14 :: (store 8 into %stack.0), (store 8 into %stack.1)
+    frame-setup SEH_SaveRegP 19, 20, 112
+    frame-setup SEH_PrologEnd
+    $x19 = ADDXrr $x0, killed $x1
+    $d8 = FADDDrr killed $d0, $d1
+    $d9 = FADDDrr $d8, $d1
+    $d10 = FADDDrr $d9, $d8
+    $d11 = FADDDrr killed $d9, $d10
+    $d12 = FADDDrr killed $d10, killed $d11
+    $x20 = ADDXrr $x19, killed $x0
+    $x21 = ADDXrr $x20, killed $x19
+    $x22 = ADDXrr $x21, killed $x20
+    $x23 = ADDXrr $x22, killed $x21
+    $x24 = ADDXrr $x23, killed $x22
+    $x25 = ADDXrr $x24, killed $x23
+    $x26 = ADDXrr $x25, killed $x24
+    $x27 = ADDXrr $x26, killed $x25
+    $x28 = ADDXrr $x27, killed $x26
+    $x0 = COPY $d12
+    $x0 = ADDXrr $x0, killed $x28
+    frame-destroy SEH_EpilogStart
+    $x19, $x20 = frame-destroy LDPXi $sp, 14 :: (load 8 from %stack.0), (load 8 from %stack.1)
+    frame-destroy SEH_SaveRegP 19, 20, 112
+    $x21, $x22 = frame-destroy LDPXi $sp, 12 :: (load 8 from %stack.2), (load 8 from %stack.3)
+    frame-destroy SEH_SaveRegP 21, 22, 96
+    $x23, $x24 = frame-destroy LDPXi $sp, 10 :: (load 8 from %stack.4), (load 8 from %stack.5)
+    frame-destroy SEH_SaveRegP 23, 24, 80
+    $x25, $x26 = frame-destroy LDPXi $sp, 8 :: (load 8 from %stack.6), (load 8 from %stack.7)
+    frame-destroy SEH_SaveRegP 25, 26, 64
+    $x27, $x28 = frame-destroy LDPXi $sp, 6 :: (load 8 from %stack.8), (load 8 from %stack.9)
+    frame-destroy SEH_SaveRegP 27, 28, 48
+    $d8 = frame-destroy LDRDui $sp, 4 :: (load 8 from %stack.10)
+    frame-destroy SEH_SaveFReg 8, 32
+    $d9 = frame-destroy LDRDui $sp, 5 :: (load 8 from %stack.11)
+    frame-destroy SEH_SaveFReg 9, 40
+    $d10, $d11 = frame-destroy LDPDi $sp, 2 :: (load 8 from %stack.12), (load 8 from %stack.13)
+    frame-destroy SEH_SaveFRegP 10, 11, 16
+    early-clobber $sp, $d12 = frame-destroy LDRDpost $sp, 128 :: (load 8 from %stack.14)
+    frame-destroy SEH_SaveFReg_X 12, -128
+    frame-destroy SEH_EpilogEnd
+    RET_ReallyLR implicit $x0
+
+...

llvm/trunk/test/CodeGen/AArch64/wineh3.mir

+# RUN: llc -o - %s -mtriple=aarch64-windows -start-after=prologepilog \
+# RUN:   -disable-post-ra -filetype=obj | llvm-readobj -unwind | FileCheck %s
+# Test that the register pairing of both general purpose and floating point
+# registers is correctly saved in the .xdata section, as well as the pre/post
+# increment of floating point register pairs.
+
+# CHECK:        ExceptionData {
+# CHECK-NEXT:      FunctionLength: 124
+# CHECK-NEXT:      Version: 0
+# CHECK-NEXT:      ExceptionData: No
+# CHECK-NEXT:      EpiloguePacked: No
+# CHECK-NEXT:      EpilogueScopes: 1
+# CHECK-NEXT:      ByteCodeLength: 32
+# CHECK-NEXT:      Prologue [
+# CHECK-NEXT:        0xc80c              ; stp x19, x20, [sp, #96]
+# CHECK-NEXT:        0xc88a              ; stp x21, x22, [sp, #80]
+# CHECK-NEXT:        0xc908              ; stp x23, x24, [sp, #64]
+# CHECK-NEXT:        0xc986              ; stp x25, x26, [sp, #48]
+# CHECK-NEXT:        0xca04              ; stp x27, x28, [sp, #32]
+# CHECK-NEXT:        0xd802              ; stp d8, d9, [sp, #16]
+# CHECK-NEXT:        0xda8d              ; stp d10, d11, [sp, #-112]!
+# CHECK-NEXT:        0xe4                ; end
+# CHECK-NEXT:      ]
+# CHECK-NEXT:      EpilogueScopes [
+# CHECK-NEXT:        EpilogueScope {
+# CHECK-NEXT:          StartOffset: 23
+# CHECK-NEXT:          EpilogueStartIndex: 15
+# CHECK-NEXT:          Opcodes [
+# CHECK-NEXT:            0xc80c              ; ldp x19, x20, [sp, #96]
+# CHECK-NEXT:            0xc88a              ; ldp x21, x22, [sp, #80]
+# CHECK-NEXT:            0xc908              ; ldp x23, x24, [sp, #64]
+# CHECK-NEXT:            0xc986              ; ldp x25, x26, [sp, #48]
+# CHECK-NEXT:            0xca04              ; ldp x27, x28, [sp, #32]
+# CHECK-NEXT:            0xd802              ; ldp d8, d9, [sp, #16]
+# CHECK-NEXT:            0xda8d              ; ldp d10, d11, [sp], #112
+# CHECK-NEXT:            0xe4                ; end
+# CHECK-NEXT:          ]
+# CHECK-NEXT:        }
+# CHECK-NEXT:      ]
+# CHECK-NEXT:    }
+...
+---
+name:            test
+alignment:       2
+exposesReturnsTwice: false
+legalized:       false
+regBankSelected: false
+selected:        false
+failedISel:      false
+tracksRegLiveness: true
+hasWinCFI:       true
+registers:
+liveins:
+  - { reg: '$w0', virtual-reg: '' }
+frameInfo:
+  isFrameAddressTaken: false
+  isReturnAddressTaken: false
+  hasStackMap:     false
+  hasPatchPoint:   false
+  stackSize:       112
+  offsetAdjustment: 0
+  maxAlignment:    8
+  adjustsStack:    false
+  hasCalls:        false
+  stackProtector:  ''
+  maxCallFrameSize: 0
+  hasOpaqueSPAdjustment: true
+  hasVAStart:      false
+  hasMustTailInVarArgFunc: false
+  localFrameSize:  0
+  savePoint:       ''
+  restorePoint:    ''
+fixedStack:
+stack:
+  - { id: 0, name: '', type: spill-slot, offset: -8, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x19', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 1, name: '', type: spill-slot, offset: -16, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x20', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 2, name: '', type: spill-slot, offset: -24, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x21', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 3, name: '', type: spill-slot, offset: -32, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x22', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 4, name: '', type: spill-slot, offset: -40, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x23', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 5, name: '', type: spill-slot, offset: -48, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x24', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 6, name: '', type: spill-slot, offset: -56, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x25', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 7, name: '', type: spill-slot, offset: -64, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x26', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 8, name: '', type: spill-slot, offset: -72, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x27', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 9, name: '', type: spill-slot, offset: -80, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x28', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 10, name: '', type: spill-slot, offset: -88, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$d8', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 11, name: '', type: spill-slot, offset: -96, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$d9', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 12, name: '', type: spill-slot, offset: -104, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$d10', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 13, name: '', type: spill-slot, offset: -112, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$d11', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+constants:
+body:             |
+  bb.0.entry:
+    liveins: $x0, $x1, $d0, $d1, $d10, $d11, $d8, $d9, $x27, $x28, $x25, $x26, $x23, $x24, $x21, $x22, $x19, $x20
+
+    early-clobber $sp = frame-setup STPDpre killed $d10, killed $d11, $sp, -14 :: (store 8 into %stack.12), (store 8 into %stack.13)
+    frame-setup SEH_SaveFRegP_X 10, 11, -112
+    frame-setup STPDi killed $d8, killed $d9, $sp, 2 :: (store 8 into %stack.10), (store 8 into %stack.11)
+    frame-setup SEH_SaveFRegP 8, 9, 16
+    frame-setup STPXi killed $x27, killed $x28, $sp, 4 :: (store 8 into %stack.8), (store 8 into %stack.9)
+    frame-setup SEH_SaveRegP 27, 28, 32
+    frame-setup STPXi killed $x25, killed $x26, $sp, 6 :: (store 8 into %stack.6), (store 8 into %stack.7)
+    frame-setup SEH_SaveRegP 25, 26, 48
+    frame-setup STPXi killed $x23, killed $x24, $sp, 8 :: (store 8 into %stack.4), (store 8 into %stack.5)
+    frame-setup SEH_SaveRegP 23, 24, 64
+    frame-setup STPXi killed $x21, killed $x22, $sp, 10 :: (store 8 into %stack.2), (store 8 into %stack.3)
+    frame-setup SEH_SaveRegP 21, 22, 80
+    frame-setup STPXi killed $x19, killed $x20, $sp, 12 :: (store 8 into %stack.0), (store 8 into %stack.1)
+    frame-setup SEH_SaveRegP 19, 20, 96
+    frame-setup SEH_PrologEnd
+    $x19 = ADDXrr $x0, killed $x1
+    $d8 = FADDDrr killed $d0, $d1
+    $d9 = FADDDrr $d8, $d1
+    $d10 = FADDDrr $d9, $d8
+    $d11 = FADDDrr killed $d9, $d10
+    $x20 = ADDXrr $x19, killed $x0
+    $x21 = ADDXrr $x20, killed $x19
+    $x22 = ADDXrr $x21, killed $x20
+    $x23 = ADDXrr $x22, killed $x21
+    $x24 = ADDXrr $x23, killed $x22
+    $x25 = ADDXrr $x24, killed $x23
+    $x26 = ADDXrr $x25, killed $x24
+    $x27 = ADDXrr $x26, killed $x25
+    $x28 = ADDXrr $x27, killed $x26
+    $x0 = COPY $d11
+    $x0 = ADDXrr $x0, killed $x28
+    frame-destroy SEH_EpilogStart
+    $x19, $x20 = frame-destroy LDPXi $sp, 12 :: (load 8 from %stack.0), (load 8 from %stack.1)
+    frame-destroy SEH_SaveRegP 19, 20, 96
+    $x21, $x22 = frame-destroy LDPXi $sp, 10 :: (load 8 from %stack.2), (load 8 from %stack.3)
+    frame-destroy SEH_SaveRegP 21, 22, 80
+    $x23, $x24 = frame-destroy LDPXi $sp, 8 :: (load 8 from %stack.4), (load 8 from %stack.5)
+    frame-destroy SEH_SaveRegP 23, 24, 64
+    $x25, $x26 = frame-destroy LDPXi $sp, 6 :: (load 8 from %stack.6), (load 8 from %stack.7)
+    frame-destroy SEH_SaveRegP 25, 26, 48
+    $x27, $x28 = frame-destroy LDPXi $sp, 4 :: (load 8 from %stack.8), (load 8 from %stack.9)
+    frame-destroy SEH_SaveRegP 27, 28, 32
+    $d8, $d9 = frame-destroy LDPDi $sp, 2 :: (load 8 from %stack.10), (load 8 from %stack.11)
+    frame-destroy SEH_SaveFRegP 8, 9, 16
+    early-clobber $sp, $d10, $d11 = frame-destroy LDPDpost $sp, 14 :: (load 8 from %stack.12), (load 8 from %stack.13)
+    frame-destroy SEH_SaveFRegP_X 10, 11, -112
+    frame-destroy SEH_EpilogEnd
+    RET_ReallyLR implicit $x0
+
+...

llvm/trunk/test/CodeGen/AArch64/wineh4.mir

+# RUN: llc -o - %s -mtriple=aarch64-windows -start-after=prologepilog \
+# RUN:   -disable-branch-fold -disable-post-ra -filetype=obj \
+# RUN: | llvm-readobj -unwind | FileCheck %s
+# Check that multiple epilgoues are correctly placed in .xdata.
+
+# CHECK:        ExceptionData {
+# CHECK-NEXT:      FunctionLength: 164
+# CHECK-NEXT:      Version: 0
+# CHECK-NEXT:      ExceptionData: No
+# CHECK-NEXT:      EpiloguePacked: No
+# CHECK-NEXT:      EpilogueScopes: 2
+# CHECK-NEXT:      ByteCodeLength: 48
+# CHECK-NEXT:      Prologue [
+# CHECK-NEXT:        0xc80c              ; stp x19, x20, [sp, #96]
+# CHECK-NEXT:        0xc88a              ; stp x21, x22, [sp, #80]
+# CHECK-NEXT:        0xc908              ; stp x23, x24, [sp, #64]
+# CHECK-NEXT:        0xc986              ; stp x25, x26, [sp, #48]
+# CHECK-NEXT:        0xca04              ; stp x27, x28, [sp, #32]
+# CHECK-NEXT:        0xd802              ; stp d8, d9, [sp, #16]
+# CHECK-NEXT:        0xda8d              ; stp d10, d11, [sp, #-112]!
+# CHECK-NEXT:        0xe4                ; end
+# CHECK-NEXT:      ]
+# CHECK-NEXT:      EpilogueScopes [
+# CHECK-NEXT:        EpilogueScope {
+# CHECK-NEXT:          StartOffset: 16
+# CHECK-NEXT:          EpilogueStartIndex: 15
+# CHECK-NEXT:          Opcodes [
+# CHECK-NEXT:            0xc80c              ; ldp x19, x20, [sp, #96]
+# CHECK-NEXT:            0xc88a              ; ldp x21, x22, [sp, #80]
+# CHECK-NEXT:            0xc908              ; ldp x23, x24, [sp, #64]
+# CHECK-NEXT:            0xc986              ; ldp x25, x26, [sp, #48]
+# CHECK-NEXT:            0xca04              ; ldp x27, x28, [sp, #32]
+# CHECK-NEXT:            0xd802              ; ldp d8, d9, [sp, #16]
+# CHECK-NEXT:            0xda8d              ; ldp d10, d11, [sp], #112
+# CHECK-NEXT:            0xe4                ; end
+# CHECK-NEXT:          ]
+# CHECK-NEXT:        }
+# CHECK-NEXT:        EpilogueScope {
+# CHECK-NEXT:          StartOffset: 33
+# CHECK-NEXT:          EpilogueStartIndex: 30
+# CHECK-NEXT:          Opcodes [
+# CHECK-NEXT:            0xc80c              ; ldp x19, x20, [sp, #96]
+# CHECK-NEXT:            0xc88a              ; ldp x21, x22, [sp, #80]
+# CHECK-NEXT:            0xc908              ; ldp x23, x24, [sp, #64]
+# CHECK-NEXT:            0xc986              ; ldp x25, x26, [sp, #48]
+# CHECK-NEXT:            0xca04              ; ldp x27, x28, [sp, #32]
+# CHECK-NEXT:            0xd802              ; ldp d8, d9, [sp, #16]
+# CHECK-NEXT:            0xda8d              ; ldp d10, d11, [sp], #112
+# CHECK-NEXT:            0xe4                ; end
+# CHECK-NEXT:          ]
+# CHECK-NEXT:        }
+# CHECK-NEXT:      ]
+# CHECK-NEXT:    }
+...
+---
+name:            test
+alignment:       2
+exposesReturnsTwice: false
+legalized:       false
+regBankSelected: false
+selected:        false
+failedISel:      false
+tracksRegLiveness: true
+hasWinCFI:       true
+registers:
+liveins:
+  - { reg: '$w0', virtual-reg: '' }
+frameInfo:
+  isFrameAddressTaken: false
+  isReturnAddressTaken: false
+  hasStackMap:     false
+  hasPatchPoint:   false
+  stackSize:       112
+  offsetAdjustment: 0
+  maxAlignment:    8
+  adjustsStack:    false
+  hasCalls:        false
+  stackProtector:  ''
+  maxCallFrameSize: 0
+  hasOpaqueSPAdjustment: true
+  hasVAStart:      false
+  hasMustTailInVarArgFunc: false
+  localFrameSize:  0
+  savePoint:       ''
+  restorePoint:    ''
+fixedStack:
+stack:
+  - { id: 0, name: '', type: spill-slot, offset: -8, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x19', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 1, name: '', type: spill-slot, offset: -16, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x20', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 2, name: '', type: spill-slot, offset: -24, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x21', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 3, name: '', type: spill-slot, offset: -32, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x22', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 4, name: '', type: spill-slot, offset: -40, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x23', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 5, name: '', type: spill-slot, offset: -48, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x24', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 6, name: '', type: spill-slot, offset: -56, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x25', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 7, name: '', type: spill-slot, offset: -64, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x26', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 8, name: '', type: spill-slot, offset: -72, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x27', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 9, name: '', type: spill-slot, offset: -80, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x28', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 10, name: '', type: spill-slot, offset: -88, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$d8', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 11, name: '', type: spill-slot, offset: -96, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$d9', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 12, name: '', type: spill-slot, offset: -104, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$d10', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 13, name: '', type: spill-slot, offset: -112, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$d11', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+constants:
+body:             |
+  bb.0.entry:
+    successors: %bb.2(0x40000000), %bb.1(0x40000000)
+    liveins: $x0, $x1, $d0, $d1, $d10, $d11, $d8, $d9, $x27, $x28, $x25, $x26, $x23, $x24, $x21, $x22, $x19, $x20
+
+    early-clobber $sp = frame-setup STPDpre killed $d10, killed $d11, $sp, -14 :: (store 8 into %stack.12), (store 8 into %stack.13)
+    frame-setup SEH_SaveFRegP_X 10, 11, -112
+    frame-setup STPDi killed $d8, killed $d9, $sp, 2 :: (store 8 into %stack.10), (store 8 into %stack.11)
+    frame-setup SEH_SaveFRegP 8, 9, 16
+    frame-setup STPXi killed $x27, killed $x28, $sp, 4 :: (store 8 into %stack.8), (store 8 into %stack.9)
+    frame-setup SEH_SaveRegP 27, 28, 32
+    frame-setup STPXi killed $x25, killed $x26, $sp, 6 :: (store 8 into %stack.6), (store 8 into %stack.7)
+    frame-setup SEH_SaveRegP 25, 26, 48
+    frame-setup STPXi killed $x23, killed $x24, $sp, 8 :: (store 8 into %stack.4), (store 8 into %stack.5)
+    frame-setup SEH_SaveRegP 23, 24, 64
+    frame-setup STPXi killed $x21, killed $x22, $sp, 10 :: (store 8 into %stack.2), (store 8 into %stack.3)
+    frame-setup SEH_SaveRegP 21, 22, 80
+    frame-setup STPXi killed $x19, killed $x20, $sp, 12 :: (store 8 into %stack.0), (store 8 into %stack.1)
+    frame-setup SEH_SaveRegP 19, 20, 96
+    frame-setup SEH_PrologEnd
+    frame-setup CFI_INSTRUCTION def_cfa_offset 112
+    frame-setup CFI_INSTRUCTION offset $w19, -8
+    frame-setup CFI_INSTRUCTION offset $w20, -16
+    frame-setup CFI_INSTRUCTION offset $w21, -24
+    frame-setup CFI_INSTRUCTION offset $w22, -32
+    frame-setup CFI_INSTRUCTION offset $w23, -40
+    frame-setup CFI_INSTRUCTION offset $w24, -48
+    frame-setup CFI_INSTRUCTION offset $w25, -56
+    frame-setup CFI_INSTRUCTION offset $w26, -64
+    frame-setup CFI_INSTRUCTION offset $w27, -72
+    frame-setup CFI_INSTRUCTION offset $w28, -80
+    frame-setup CFI_INSTRUCTION offset $b8, -88
+    frame-setup CFI_INSTRUCTION offset $b9, -96
+    frame-setup CFI_INSTRUCTION offset $b10, -104
+    frame-setup CFI_INSTRUCTION offset $b11, -112
+    $x19 = ADDXrr $x0, killed $x1
+    $d8 = FADDDrr killed $d0, $d1
+    $d9 = FADDDrr $d8, $d1
+    $d10 = FADDDrr $d9, $d8
+    $d11 = FADDDrr killed $d9, $d10
+    $x20 = SUBSXrr $x19, killed $x0, implicit-def $nzcv
+    Bcc 1, %bb.2, implicit killed $nzcv
+    B %bb.1
+
+  bb.1:
+    liveins: $x19, $x20
+
+    $x21 = ADDXrr $x20, killed $x19
+    $x22 = ADDXrr $x21, killed $x20
+    $x23 = ADDXrr $x22, killed $x21
+    $x24 = ADDXrr $x23, killed $x22
+    $x25 = ADDXrr $x24, killed $x23
+    $x26 = ADDXrr $x25, killed $x24
+    $x27 = ADDXrr $x26, killed $x25
+    $x28 = ADDXrr $x27, killed $x26
+    $x0 = COPY $x28
+    frame-destroy SEH_EpilogStart
+    $x19, $x20 = frame-destroy LDPXi $sp, 12 :: (load 8 from %stack.0), (load 8 from %stack.1)
+    frame-destroy SEH_SaveRegP 19, 20, 96
+    $x21, $x22 = frame-destroy LDPXi $sp, 10 :: (load 8 from %stack.2), (load 8 from %stack.3)
+    frame-destroy SEH_SaveRegP 21, 22, 80
+    $x23, $x24 = frame-destroy LDPXi $sp, 8 :: (load 8 from %stack.4), (load 8 from %stack.5)
+    frame-destroy SEH_SaveRegP 23, 24, 64
+    $x25, $x26 = frame-destroy LDPXi $sp, 6 :: (load 8 from %stack.6), (load 8 from %stack.7)
+    frame-destroy SEH_SaveRegP 25, 26, 48
+    $x27, $x28 = frame-destroy LDPXi $sp, 4 :: (load 8 from %stack.8), (load 8 from %stack.9)
+    frame-destroy SEH_SaveRegP 27, 28, 32
+    $d8, $d9 = frame-destroy LDPDi $sp, 2 :: (load 8 from %stack.10), (load 8 from %stack.11)
+    frame-destroy SEH_SaveFRegP 8, 9, 16
+    early-clobber $sp, $d10, $d11 = frame-destroy LDPDpost $sp, 14 :: (load 8 from %stack.12), (load 8 from %stack.13)
+    frame-destroy SEH_SaveFRegP_X 10, 11, -112
+    frame-destroy SEH_EpilogEnd
+    RET_ReallyLR implicit $x0
+
+  bb.2:
+    liveins: $x28, $d11
+
+    $x0 = COPY $d11
+    $x0 = ADDXrr $x0, killed $x28
+    frame-destroy SEH_EpilogStart
+    $x19, $x20 = frame-destroy LDPXi $sp, 12 :: (load 8 from %stack.0), (load 8 from %stack.1)
+    frame-destroy SEH_SaveRegP 19, 20, 96
+    $x21, $x22 = frame-destroy LDPXi $sp, 10 :: (load 8 from %stack.2), (load 8 from %stack.3)
+    frame-destroy SEH_SaveRegP 21, 22, 80
+    $x23, $x24 = frame-destroy LDPXi $sp, 8 :: (load 8 from %stack.4), (load 8 from %stack.5)
+    frame-destroy SEH_SaveRegP 23, 24, 64
+    $x25, $x26 = frame-destroy LDPXi $sp, 6 :: (load 8 from %stack.6), (load 8 from %stack.7)
+    frame-destroy SEH_SaveRegP 25, 26, 48
+    $x27, $x28 = frame-destroy LDPXi $sp, 4 :: (load 8 from %stack.8), (load 8 from %stack.9)
+    frame-destroy SEH_SaveRegP 27, 28, 32
+    $d8, $d9 = frame-destroy LDPDi $sp, 2 :: (load 8 from %stack.10), (load 8 from %stack.11)
+    frame-destroy SEH_SaveFRegP 8, 9, 16
+    early-clobber $sp, $d10, $d11 = frame-destroy LDPDpost $sp, 14 :: (load 8 from %stack.12), (load 8 from %stack.13)
+    frame-destroy SEH_SaveFRegP_X 10, 11, -112
+    frame-destroy SEH_EpilogEnd
+    RET_ReallyLR implicit $x0
+
+...

llvm/trunk/test/CodeGen/AArch64/wineh5.mir

+# RUN: llc -o - %s -mtriple=aarch64-windows -start-after=prologepilog \
+# RUN:   -disable-post-ra -filetype=obj | llvm-readobj -unwind | FileCheck %s
+
+# Check that that the large stack allocation is correctly represented in .xdata.
+
+# CHECK:        ExceptionData {
+# CHECK-NEXT:     FunctionLength: 156
+# CHECK-NEXT:     Version: 0
+# CHECK-NEXT:     ExceptionData: No
+# CHECK-NEXT:     EpiloguePacked: No
+# CHECK-NEXT:     EpilogueScopes: 1
+# CHECK-NEXT:     ByteCodeLength: 20
+# CHECK-NEXT:     Prologue [
+# CHECK-NEXT:       0xe002dac9          ; sub sp, #2993296
+# CHECK-NEXT:       0xe3                ; nop
+# CHECK-NEXT:       0xe3                ; nop
+# CHECK-NEXT:       0x42                ; stp x29, x30, [sp, #16]
+# CHECK-NEXT:       0xd53f              ; str x28, [sp, #256]!
+# CHECK-NEXT:       0xe4                ; end
+# CHECK-NEXT:     ]
+# CHECK-NEXT:     EpilogueScopes [
+# CHECK-NEXT:       EpilogueScope {
+# CHECK-NEXT:         StartOffset: 34
+# CHECK-NEXT:         EpilogueStartIndex: 10
+# CHECK-NEXT:         Opcodes [
+# CHECK-NEXT:           0xe002da00          ; add sp, #2990080
+# CHECK-NEXT:           0xc0c9              ; add sp, #3216
+# CHECK-NEXT:           0x42                ; ldp x29, x30, [sp, #16]
+# CHECK-NEXT:           0xd53f              ; ldr x28, [sp], #256
+# CHECK-NEXT:           0xe4                ; end
+# CHECK-NEXT:         ]
+# CHECK-NEXT:       }
+# CHECK-NEXT:     ]
+# CHECK-NEXT:   }
+
+
+--- |
+  target datalayout = "e-m:w-p:64:64-i32:32-i64:64-i128:128-n32:64-S128"
+  target triple = "aarch64-unknown-windows-msvc19.11.0"
+
+  ; Function Attrs: noinline optnone
+  define dso_local i32 @"?func@@YAHH@Z"(i32 %i) #0 {
+  entry:
+    %retval = alloca i32, align 4
+    %i.addr = alloca i32, align 4
+    %A = alloca [748193 x i32], align 4
+    %a = alloca i32, align 4
+    %B = alloca [123 x i32], align 4
+    store i32 %i, i32* %i.addr, align 4
+    %0 = load i32, i32* %i.addr, align 4
+    %add = add nsw i32 %0, 2
+    store i32 %add, i32* %a, align 4
+    %call = call i32 @"?func2@@YAHXZ"()
+    %1 = load i32, i32* %i.addr, align 4
+    %cmp = icmp sgt i32 %1, 2
+    br i1 %cmp, label %if.then, label %if.else
+
+  if.then:                                          ; preds = %entry
+    %call1 = call i32 @"?func2@@YAHXZ"()
+    store i32 %call1, i32* %retval, align 4
+    br label %return
+
+  if.else:                                          ; preds = %entry
+    %arraydecay = getelementptr inbounds [123 x i32], [123 x i32]* %B, i32 0, i32 0
+    %call2 = call i32 @"?func3@@YAHPEAH@Z"(i32* %arraydecay)
+    store i32 %call2, i32* %retval, align 4
+    br label %return
+
+  return:                                           ; preds = %if.else, %if.then
+    %2 = load i32, i32* %retval, align 4
+    ret i32 %2
+  }
+
+  declare dso_local i32 @"?func2@@YAHXZ"() #1
+
+  declare dso_local i32 @"?func3@@YAHPEAH@Z"(i32*) #1
+
+  ; Function Attrs: nounwind
+  declare void @llvm.stackprotector(i8*, i8**) #2
+
+  attributes #0 = { noinline optnone "correctly-rounded-divide-sqrt-fp-math"="false" "disable-tail-calls"="false" "less-precise-fpmad"="false" "no-frame-pointer-elim"="false" "no-infs-fp-math"="false" "no-jump-tables"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="generic" "target-features"="+neon" "unsafe-fp-math"="false" "use-soft-float"="false" }
+  attributes #1 = { "correctly-rounded-divide-sqrt-fp-math"="false" "disable-tail-calls"="false" "less-precise-fpmad"="false" "no-frame-pointer-elim"="false" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="generic" "target-features"="+neon" "unsafe-fp-math"="false" "use-soft-float"="false" }
+  attributes #2 = { nounwind }
+
+  !llvm.module.flags = !{!0}
+
+  !0 = !{i32 1, !"wchar_size", i32 2}
+
+...
+---
+name:            '?func@@YAHH@Z'
+alignment:       2
+exposesReturnsTwice: false
+legalized:       true
+regBankSelected: true
+selected:        true
+failedISel:      false
+tracksRegLiveness: true
+hasWinCFI:       true
+registers:
+liveins:
+frameInfo:
+  isFrameAddressTaken: false
+  isReturnAddressTaken: false
+  hasStackMap:     false
+  hasPatchPoint:   false
+  stackSize:       2993328
+  offsetAdjustment: 0
+  maxAlignment:    16
+  adjustsStack:    true
+  hasCalls:        true
+  stackProtector:  ''
+  maxCallFrameSize: 0
+  hasOpaqueSPAdjustment: false
+  hasVAStart:      false
+  hasMustTailInVarArgFunc: false
+  localFrameSize:  2993276
+  savePoint:       ''
+  restorePoint:    ''
+fixedStack:
+stack:
+  - { id: 0, name: retval, type: default, offset: -36, size: 4, alignment: 4,
+      stack-id: 0, callee-saved-register: '', callee-saved-restored: true,
+      local-offset: -4, debug-info-variable: '', debug-info-expression: '',
+      debug-info-location: '' }
+  - { id: 1, name: i.addr, type: default, offset: -40, size: 4, alignment: 4,
+      stack-id: 0, callee-saved-register: '', callee-saved-restored: true,
+      local-offset: -8, debug-info-variable: '', debug-info-expression: '',
+      debug-info-location: '' }
+  - { id: 2, name: A, type: default, offset: -2992812, size: 2992772, alignment: 4,
+      stack-id: 0, callee-saved-register: '', callee-saved-restored: true,
+      local-offset: -2992780, debug-info-variable: '', debug-info-expression: '',
+      debug-info-location: '' }
+  - { id: 3, name: a, type: default, offset: -2992816, size: 4, alignment: 4,
+      stack-id: 0, callee-saved-register: '', callee-saved-restored: true,
+      local-offset: -2992784, debug-info-variable: '', debug-info-expression: '',
+      debug-info-location: '' }
+  - { id: 4, name: B, type: default, offset: -2993308, size: 492, alignment: 4,
+      stack-id: 0, callee-saved-register: '', callee-saved-restored: true,
+      local-offset: -2993276, debug-info-variable: '', debug-info-expression: '',
+      debug-info-location: '' }
+  - { id: 5, name: '', type: spill-slot, offset: -2993320, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 6, name: '', type: spill-slot, offset: -2993324, size: 4, alignment: 4,
+      stack-id: 0, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 7, name: '', type: spill-slot, offset: -8, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$fp', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 8, name: '', type: spill-slot, offset: -16, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$lr', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 9, name: '', type: spill-slot, offset: -32, size: 8, alignment: 16,
+      stack-id: 0, callee-saved-register: '$x28', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+constants:
+body:             |
+  bb.1.entry:
+    successors: %bb.2(0x40000000), %bb.3(0x40000000)
+    liveins: $w0, $x28, $fp, $lr
+
+    early-clobber $sp = frame-setup STRXpre killed $x28, $sp, -32 :: (store 8 into %stack.9)
+    frame-setup SEH_SaveReg_X 28, -256
+    frame-setup STPXi killed $fp, killed $lr, $sp, 2 :: (store 8 into %stack.7), (store 8 into %stack.8)
+    frame-setup SEH_SaveFPLR 16
+    $x15 = frame-setup MOVi64imm 187081
+    frame-setup SEH_Nop
+    frame-setup BL &__chkstk, implicit-def $lr, implicit $sp, implicit $x15
+    frame-setup SEH_Nop
+    $sp = frame-setup SUBXrx64 killed $sp, killed $x15, 28
+    frame-setup SEH_StackAlloc 2993296
+    frame-setup SEH_PrologEnd
+    $x8 = ADDXri $sp, 730, 12
+    $x8 = ADDXri $x8, 3208, 0
+    renamable $w9 = MOVi32imm 2
+    STRWui killed renamable $w0, renamable $x8, 0 :: (store 4 into %ir.i.addr)
+    renamable $w0 = LDRWui renamable $x8, 0 :: (load 4 from %ir.i.addr)
+    renamable $w0 = ADDWri killed renamable $w0, 2, 0
+    STRWui killed renamable $w0, $sp, 128 :: (store 4 into %ir.a)
+    STRXui killed $x8, $sp, 1 :: (store 8 into %stack.5)
+    STRWui killed $w9, $sp, 1 :: (store 4 into %stack.6)
+    BL @"?func2@@YAHXZ", csr_aarch64_aapcs, implicit-def $lr, implicit $sp, implicit-def $w0
+    $x8 = LDRXui $sp, 1 :: (load 8 from %stack.5)
+    renamable $w9 = LDRWui killed renamable $x8, 0 :: (load 4 from %ir.i.addr)
+    $w10 = LDRWui $sp, 1 :: (load 4 from %stack.6)
+    $wzr = SUBSWrr killed renamable $w9, killed renamable $w10, implicit-def $nzcv
+    renamable $w9 = CSINCWr $wzr, $wzr, 13, implicit $nzcv
+    TBNZW killed renamable $w9, 0, %bb.2
+    B %bb.3
+
+  bb.2.if.then:
+    successors: %bb.4(0x80000000)
+
+    BL @"?func2@@YAHXZ", csr_aarch64_aapcs, implicit-def $lr, implicit $sp, implicit-def $w0
+    $x8 = LDRXui $sp, 1 :: (load 8 from %stack.5)
+    STRWui killed renamable $w0, killed renamable $x8, 1 :: (store 4 into %ir.retval)
+    B %bb.4
+
+  bb.3.if.else:
+    successors: %bb.4(0x80000000)
+
+    $x8 = ADDXri $sp, 20, 0
+    $x0 = COPY killed renamable $x8
+    BL @"?func3@@YAHPEAH@Z", csr_aarch64_aapcs, implicit-def $lr, implicit $sp, implicit killed $x0, implicit-def $w0
+    $x8 = LDRXui $sp, 1 :: (load 8 from %stack.5)
+    STRWui killed renamable $w0, killed renamable $x8, 1 :: (store 4 into %ir.retval)
+
+  bb.4.return:
+    $x8 = LDRXui $sp, 1 :: (load 8 from %stack.5)
+    renamable $w0 = LDRWui killed renamable $x8, 1 :: (load 4 from %ir.retval)
+    frame-destroy SEH_EpilogStart
+    $sp = frame-destroy ADDXri $sp, 730, 12
+    frame-destroy SEH_StackAlloc 2990080
+    $sp = frame-destroy ADDXri $sp, 3216, 0
+    frame-destroy SEH_StackAlloc 3216
+    $fp, $lr = frame-destroy LDPXi $sp, 2 :: (load 8 from %stack.7), (load 8 from %stack.8)
+    frame-destroy SEH_SaveFPLR 16
+    early-clobber $sp, $x28 = frame-destroy LDRXpost $sp, 32 :: (load 8 from %stack.9)
+    frame-destroy SEH_SaveReg_X 28, -256
+    frame-destroy SEH_EpilogEnd
+    RET_ReallyLR implicit killed $w0
+
+...

llvm/trunk/test/CodeGen/AArch64/wineh6.mir

+# RUN: llc -o - %s -mtriple=aarch64-windows -start-after=prologepilog \
+# RUN:   -disable-post-ra -filetype=obj | llvm-readobj -unwind | FileCheck %s
+# Check save_fplr_x, set_fp, alloc_s
+
+# CHECK:  ExceptionData {
+# CHECK-NEXT:      FunctionLength: 92
+# CHECK-NEXT:      Version: 0
+# CHECK-NEXT:      ExceptionData: No
+# CHECK-NEXT:      EpiloguePacked: No
+# CHECK-NEXT:      EpilogueScopes: 1
+# CHECK-NEXT:      ByteCodeLength: 8
+# CHECK-NEXT:      Prologue [
+# CHECK-NEXT:        0x02                ; sub sp, #32
+# CHECK-NEXT:        0xe1                ; mov fp, sp
+# CHECK-NEXT:        0x81                ; stp x29, x30, [sp, #-16]!
+# CHECK-NEXT:        0xe4                ; end
+# CHECK-NEXT:      ]
+# CHECK-NEXT:      EpilogueScopes [
+# CHECK-NEXT:        EpilogueScope {
+# CHECK-NEXT:          StartOffset: 20
+# CHECK-NEXT:          EpilogueStartIndex: 4
+# CHECK-NEXT:          Opcodes [
+# CHECK-NEXT:            0xe1                ; mov fp, sp
+# CHECK-NEXT:            0x81                ; ldp x29, x30, [sp], #16
+# CHECK-NEXT:            0xe4                ; end
+# CHECK-NEXT:          ]
+# CHECK-NEXT:        }
+# CHECK-NEXT:      ]
+# CHECK-NEXT:    }
+...
+---
+name:            '?func@@YAHHHHH@Z'
+alignment:       3
+exposesReturnsTwice: false
+legalized:       false
+regBankSelected: false
+selected:        false
+failedISel:      false
+tracksRegLiveness: true
+hasWinCFI: true
+registers:
+liveins:
+  - { reg: '$w0', virtual-reg: '' }
+  - { reg: '$w1', virtual-reg: '' }
+  - { reg: '$w2', virtual-reg: '' }
+  - { reg: '$w3', virtual-reg: '' }
+frameInfo:
+  isFrameAddressTaken: false
+  isReturnAddressTaken: false
+  hasStackMap:     false
+  hasPatchPoint:   false
+  stackSize:       48
+  offsetAdjustment: 0
+  maxAlignment:    8
+  adjustsStack:    true
+  hasCalls:        true
+  stackProtector:  ''
+  maxCallFrameSize: 0
+  hasOpaqueSPAdjustment: false
+  hasVAStart:      false
+  hasMustTailInVarArgFunc: false
+  localFrameSize:  24
+  savePoint:       ''
+  restorePoint:    ''
+fixedStack:
+stack:
+  - { id: 0, name: '', type: default, offset: -20, size: 4, alignment: 4,
+      stack-id: 0, callee-saved-register: '', callee-saved-restored: true,
+      local-offset: -4, debug-info-variable: '', debug-info-expression: '',
+      debug-info-location: '' }
+  - { id: 1, name: '', type: default, offset: -24, size: 4, alignment: 4,
+      stack-id: 0, callee-saved-register: '', callee-saved-restored: true,
+      local-offset: -8, debug-info-variable: '', debug-info-expression: '',
+      debug-info-location: '' }
+  - { id: 2, name: '', type: default, offset: -28, size: 4, alignment: 4,
+      stack-id: 0, callee-saved-register: '', callee-saved-restored: true,
+      local-offset: -12, debug-info-variable: '', debug-info-expression: '',
+      debug-info-location: '' }
+  - { id: 3, name: '', type: default, offset: -32, size: 4, alignment: 4,
+      stack-id: 0, callee-saved-register: '', callee-saved-restored: true,
+      local-offset: -16, debug-info-variable: '', debug-info-expression: '',
+      debug-info-location: '' }
+  - { id: 4, name: '', type: default, offset: -40, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '', callee-saved-restored: true,
+      local-offset: -24, debug-info-variable: '', debug-info-expression: '',
+      debug-info-location: '' }
+  - { id: 5, name: '', type: variable-sized, offset: -40,
+      alignment: 1, stack-id: 0, callee-saved-register: '', callee-saved-restored: true,
+      local-offset: -24, debug-info-variable: '', debug-info-expression: '',
+      debug-info-location: '' }
+  - { id: 6, name: '', type: spill-slot, offset: -48, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 7, name: '', type: spill-slot, offset: -8, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$fp', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 8, name: '', type: spill-slot, offset: -16, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$lr', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+constants:
+body:             |
+  bb.0.entry:
+    liveins: $w0, $w1, $w2, $w3, $lr
+
+    early-clobber $sp = frame-setup STPXpre killed $fp, killed $lr, $sp, -2 :: (store 8 into %stack.7), (store 8 into %stack.8)
+    frame-setup SEH_SaveFPLR_X -16
+    $fp = frame-setup ADDXri $sp, 0, 0
+    frame-setup SEH_SetFP
+    $sp = frame-setup SUBXri $sp, 32, 0
+    frame-setup SEH_StackAlloc 32
+    frame-setup SEH_PrologEnd
+    STURWi killed renamable $w3, $fp, -4
+    STURWi killed renamable $w2, $fp, -8
+    STURWi killed renamable $w1, $fp, -12
+    STURWi killed renamable $w0, $fp, -16
+    renamable $x8 = LDURSWi $fp, -16
+    renamable $x8 = ADDXri killed renamable $x8, 15, 0
+    renamable $x8 = UBFMXri killed renamable $x8, 4, 63
+    $x15 = COPY renamable $x8
+    STURXi killed $x8, $fp, -32 :: (store 8 into %stack.6)
+    BL &__chkstk, csr_aarch64_stackprobe_windows, implicit-def dead $lr, implicit $sp, implicit killed $x15
+    renamable $x8 = COPY $sp
+    $x15 = LDURXi $fp, -32 :: (load 8 from %stack.6)
+    renamable $x8 = SUBSXrs killed renamable $x8, killed renamable $x15, 4, implicit-def dead $nzcv
+    $sp = COPY renamable $x8
+    STURXi killed renamable $x8, $fp, -24
+    renamable $x0 = LDURXi $fp, -24
+    renamable $w1 = COPY $wzr
+    $w0 = COPY killed renamable $w1
+    frame-destroy SEH_EpilogStart
+    $sp = frame-destroy ADDXri $fp, 0, 0
+    frame-destroy SEH_SetFP
+    early-clobber $sp, $fp, $lr = frame-destroy LDPXpost $sp, 2
+    frame-destroy SEH_SaveFPLR_X -16
+    frame-destroy SEH_EpilogEnd
+    RET_ReallyLR implicit killed $w0
+
+...

llvm/trunk/test/CodeGen/AArch64/wineh7.mir

+# RUN: llc -o - %s -mtriple=aarch64-windows -start-after=prologepilog \
+# RUN:   -filetype=obj -disable-post-ra | llvm-readobj -unwind | FileCheck %s
+# Check AddFP
+
+# CHECK:   ExceptionData {
+# CHECK-NEXT:      FunctionLength: 72
+# CHECK-NEXT:      Version: 0
+# CHECK-NEXT:      ExceptionData: No
+# CHECK-NEXT:      EpiloguePacked: No
+# CHECK-NEXT:      EpilogueScopes: 1
+# CHECK-NEXT:      ByteCodeLength: 16
+# CHECK-NEXT:      Prologue [
+# CHECK-NEXT:        0xe204              ; add fp, sp, #32
+# CHECK-NEXT:        0x44                ; stp x29, x30, [sp, #32]
+# CHECK-NEXT:        0xc802              ; stp x19, x20, [sp, #16]
+# CHECK-NEXT:        0xcc85              ; stp x21, x22, [sp, #-48]!
+# CHECK-NEXT:        0xe4                ; end
+# CHECK-NEXT:      ]
+# CHECK-NEXT:      EpilogueScopes [
+# CHECK-NEXT:        EpilogueScope {
+# CHECK-NEXT:          StartOffset: 13
+# CHECK-NEXT:          EpilogueStartIndex: 8
+# CHECK-NEXT:          Opcodes [
+# CHECK-NEXT:            0xe204              ; add fp, sp, #32
+# CHECK-NEXT:            0x44                ; ldp x29, x30, [sp, #32]
+# CHECK-NEXT:            0xc802              ; ldp x19, x20, [sp, #16]
+# CHECK-NEXT:            0xcc85              ; ldp x21, x22, [sp], #48
+# CHECK-NEXT:            0xe4                ; end
+# CHECK-NEXT:          ]
+# CHECK-NEXT:        }
+# CHECK-NEXT:      ]
+# CHECK-NEXT:    }
+# CHECK-NEXT:  }
+
+...
+---
+name:            '?func@@YAHHHHH@Z'
+alignment:       3
+exposesReturnsTwice: false
+legalized:       false
+regBankSelected: false
+selected:        false
+failedISel:      false
+tracksRegLiveness: true
+hasWinCFI: true
+registers:
+liveins:
+  - { reg: '$w0', virtual-reg: '' }
+  - { reg: '$w1', virtual-reg: '' }
+  - { reg: '$w2', virtual-reg: '' }
+  - { reg: '$w3', virtual-reg: '' }
+frameInfo:
+  isFrameAddressTaken: false
+  isReturnAddressTaken: false
+  hasStackMap:     false
+  hasPatchPoint:   false
+  stackSize:       48
+  offsetAdjustment: 0
+  maxAlignment:    8
+  adjustsStack:    true
+  hasCalls:        true
+  stackProtector:  ''
+  maxCallFrameSize: 0
+  hasOpaqueSPAdjustment: false
+  hasVAStart:      false
+  hasMustTailInVarArgFunc: false
+  localFrameSize:  0
+  savePoint:       ''
+  restorePoint:    ''
+fixedStack:
+stack:
+  - { id: 0, name: '', type: variable-sized, offset: -48,
+      alignment: 1, stack-id: 0, callee-saved-register: '', callee-saved-restored: true,
+      local-offset: 0, debug-info-variable: '', debug-info-expression: '',
+      debug-info-location: '' }
+  - { id: 1, name: '', type: spill-slot, offset: -8, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$fp', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 2, name: '', type: spill-slot, offset: -16, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$lr', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 3, name: '', type: spill-slot, offset: -24, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x19', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 4, name: '', type: spill-slot, offset: -32, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x20', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 5, name: '', type: spill-slot, offset: -40, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x21', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 6, name: '', type: spill-slot, offset: -48, size: 8, alignment: 8,
+      stack-id: 0, callee-saved-register: '$x22', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+constants:
+body:             |
+  bb.0.entry:
+    liveins: $w0, $w1, $w2, $w3, $x21, $x22, $x19, $x20, $lr
+
+    early-clobber $sp = frame-setup STPXpre killed $x21, killed $x22, $sp, -6 :: (store 8 into %stack.5), (store 8 into %stack.6)
+    frame-setup SEH_SaveRegP_X 21, 22, -48
+    frame-setup STPXi killed $x19, killed $x20, $sp, 2 :: (store 8 into %stack.3), (store 8 into %stack.4)
+    frame-setup SEH_SaveRegP 19, 20, 16
+    frame-setup STPXi killed $fp, killed $lr, $sp, 4 :: (store 8 into %stack.1), (store 8 into %stack.2)
+    frame-setup SEH_SaveFPLR 32
+    $fp = frame-setup ADDXri $sp, 32, 0
+    frame-setup SEH_AddFP 32
+    frame-setup SEH_PrologEnd
+    renamable $w19 = COPY $w3
+    renamable $w0 = KILL $w0, implicit-def $x0
+    renamable $w20 = COPY $w2
+    renamable $w21 = COPY $w1
+    renamable $x8 = SBFMXri killed renamable $x0, 0, 31
+    renamable $x9 = ADDXri killed renamable $x8, 15, 0
+    renamable $x15 = UBFMXri killed renamable $x9, 4, 63
+    renamable $x8 = COPY $sp
+    renamable $x22 = SUBXrs killed renamable $x8, killed renamable $x15, 4
+    $sp = COPY renamable $x22
+    $x0 = COPY renamable $x22
+    renamable $w8 = LDRWroW killed renamable $x22, killed renamable $w21, 1, 1
+    renamable $w9 = ADDWrr killed renamable $w19, killed renamable $w20
+    renamable $w0 = ADDWrr killed renamable $w9, killed renamable $w8
+    frame-destroy SEH_EpilogStart
+    $sp = frame-destroy SUBXri $fp, 32, 0
+    frame-destroy SEH_AddFP 32
+    $fp, $lr = frame-destroy LDPXi $sp, 4 :: (load 8 from %stack.1), (load 8 from %stack.2)
+    frame-destroy SEH_SaveFPLR 32
+    $x19, $x20 = frame-destroy LDPXi $sp, 2 :: (load 8 from %stack.3), (load 8 from %stack.4)
+    frame-destroy SEH_SaveRegP 19, 20, 16
+    early-clobber $sp, $x21, $x22 = frame-destroy LDPXpost $sp, 6 :: (load 8 from %stack.5), (load 8 from %stack.6)
+    frame-destroy SEH_SaveRegP_X 21, 22, -48
+    frame-destroy SEH_EpilogEnd
+    RET_ReallyLR implicit $w0
+
+...

llvm/trunk/test/CodeGen/AArch64/wineh_shrinkwrap.mir

+# RUN: llc -O2 -o - %s -mtriple=aarch64-windows -start-before=shrink-wrap \
+# RUN:   -stop-after=prologepilog | FileCheck %s --check-prefix=WIN64
+# RUN: llc -O2 -o - %s -mtriple=aarch64-linux -start-before=shrink-wrap \
+# RUN:   -stop-after=prologepilog | FileCheck %s --check-prefix=LINUX
+
+# This tests checks that shrink wrapping bails out on Windows AMR64 due to the
+# use of Windows CFI.  We don't currently support fragments for WIndows EH on
+# ARM64.
+# The same test gets shrink wrapped on Linux ARM64.
+
+# WIN64-LABEL: bb.0.entry:
+# WIN64: early-clobber $sp = frame-setup STRXpre killed $x28, $sp, -32
+# WIN64-LABEL: bb.1:
+# WIN64-LABEL: bb.2.if.then:
+
+# LINUX-LABEL: bb.0.entry:
+# LINUX-LABEL: bb.1:
+# LINUX-LABEL: bb.2.if.then:
+# LINUX: early-clobber $sp = frame-setup STRXpre killed $x28, $sp, -32
+--- |
+  ; ModuleID = 'shrink.cpp'
+  target datalayout = "e-m:w-p:64:64-i32:32-i64:64-i128:128-n32:64-S128"
+  target triple = "aarch64-unknown-windows-msvc19.11.0"
+
+  define dso_local i32 @"?func@@YAHHH@Z"(i32 %a, i32 %b) local_unnamed_addr #0 {
+  entry:
+    %A = alloca [1000 x i32], align 4
+    %cmp = icmp sgt i32 %a, 1
+    br i1 %cmp, label %if.then, label %return
+
+  if.then:                                          ; preds = %entry
+    %0 = bitcast [1000 x i32]* %A to i8*
+    call void @llvm.lifetime.start.p0i8(i64 4000, i8* nonnull %0) #3
+    %arraydecay2 = bitcast [1000 x i32]* %A to i32*
+    call void @"?init@@YAXPEAH@Z"(i32* nonnull %arraydecay2)
+    %arrayidx = getelementptr inbounds [1000 x i32], [1000 x i32]* %A, i64 0, i64 100
+    %1 = load i32, i32* %arrayidx, align 4, !tbaa !2
+    %add = add i32 %b, 1
+    %add1 = add i32 %add, %1
+    call void @llvm.lifetime.end.p0i8(i64 4000, i8* nonnull %0) #3
+    br label %return
+
+  return:                                           ; preds = %entry, %if.then
+    %retval.0 = phi i32 [ %add1, %if.then ], [ 0, %entry ]
+    ret i32 %retval.0
+  }
+
+  ; Function Attrs: argmemonly nounwind
+  declare void @llvm.lifetime.start.p0i8(i64, i8* nocapture) #1
+
+  declare dso_local void @"?init@@YAXPEAH@Z"(i32*) local_unnamed_addr #2
+
+  ; Function Attrs: argmemonly nounwind
+  declare void @llvm.lifetime.end.p0i8(i64, i8* nocapture) #1
+
+  ; Function Attrs: nounwind
+  declare void @llvm.stackprotector(i8*, i8**) #3
+
+  attributes #0 = { "correctly-rounded-divide-sqrt-fp-math"="false" "disable-tail-calls"="false" "less-precise-fpmad"="false" "min-legal-vector-width"="0" "no-frame-pointer-elim"="false" "no-infs-fp-math"="false" "no-jump-tables"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="generic" "target-features"="+neon" "unsafe-fp-math"="false" "use-soft-float"="false" }
+  attributes #1 = { argmemonly nounwind }
+  attributes #2 = { "correctly-rounded-divide-sqrt-fp-math"="false" "disable-tail-calls"="false" "less-precise-fpmad"="false" "no-frame-pointer-elim"="false" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="generic" "target-features"="+neon" "unsafe-fp-math"="false" "use-soft-float"="false" }
+  attributes #3 = { nounwind }
+
+  !llvm.module.flags = !{!0}
+  !llvm.ident = !{!1}
+
+  !0 = !{i32 1, !"wchar_size", i32 2}
+  !1 = !{!"clang version 8.0.0"}
+  !2 = !{!3, !3, i64 0}
+  !3 = !{!"int", !4, i64 0}
+  !4 = !{!"omnipotent char", !5, i64 0}
+  !5 = !{!"Simple C++ TBAA"}
+
+...
+---
+name:            '?func@@YAHHH@Z'
+alignment:       2
+exposesReturnsTwice: false
+legalized:       false
+regBankSelected: false
+selected:        false
+failedISel:      false
+tracksRegLiveness: true
+registers:
+liveins:
+  - { reg: '$w0', virtual-reg: '' }
+  - { reg: '$w1', virtual-reg: '' }
+frameInfo:
+  isFrameAddressTaken: false
+  isReturnAddressTaken: false
+  hasStackMap:     false
+  hasPatchPoint:   false
+  stackSize:       0
+  offsetAdjustment: 0
+  maxAlignment:    4
+  adjustsStack:    true
+  hasCalls:        true
+  stackProtector:  ''
+  maxCallFrameSize: 0
+  cvBytesOfCalleeSavedRegisters: 0
+  hasOpaqueSPAdjustment: false
+  hasVAStart:      false
+  hasMustTailInVarArgFunc: false
+  localFrameSize:  4000
+  savePoint:       ''
+  restorePoint:    ''
+fixedStack:
+stack:
+  - { id: 0, name: A, type: default, offset: 0, size: 4000, alignment: 4,
+      stack-id: 0, callee-saved-register: '', callee-saved-restored: true,
+      local-offset: -4000, debug-info-variable: '', debug-info-expression: '',
+      debug-info-location: '' }
+constants:
+body:             |
+  bb.0.entry:
+    successors: %bb.2(0x40000000), %bb.1(0x40000000)
+    liveins: $w0, $w1
+
+    dead $wzr = SUBSWri killed renamable $w0, 2, 0, implicit-def $nzcv
+    Bcc 10, %bb.2, implicit killed $nzcv
+
+  bb.1:
+    successors: %bb.3(0x80000000)
+
+    renamable $w0 = COPY $wzr
+    B %bb.3
+
+  bb.2.if.then:
+    successors: %bb.3(0x80000000)
+    liveins: $w1
+
+    renamable $w19 = COPY $w1
+    ADJCALLSTACKDOWN 0, 0, implicit-def dead $sp, implicit $sp
+    $x0 = ADDXri %stack.0.A, 0, 0
+    BL @"?init@@YAXPEAH@Z", csr_aarch64_aapcs, implicit-def dead $lr, implicit $sp, implicit $x0, implicit-def $sp
+    ADJCALLSTACKUP 0, 0, implicit-def dead $sp, implicit $sp
+    renamable $w8 = LDRWui %stack.0.A, 100 :: (dereferenceable load 4 from %ir.arrayidx, !tbaa !2)
+    renamable $w8 = ADDWrr killed renamable $w19, killed renamable $w8
+    renamable $w0 = ADDWri killed renamable $w8, 1, 0
+
+  bb.3.return:
+    liveins: $w0
+
+    RET_ReallyLR implicit $w0
+
+...