[5/5] [AArch64] Generate and parse SEH assembly directives

[mstorsjo]

Summary

This ensures that you get the same output regardless if generating code directly to an object file or if generating assembly and assembling that.

Add implementations of the EmitARM64WinCFI*() methods in AArch64TargetAsmStreamer, and fill in one blank in MCAsmStreamer.

Add corresponding directive handlers in AArch64AsmParser and COFFAsmParser.

The SEH directives have been picked to match the prior art for SEH assembly directives for x86_64, e.g. the spelling of ".seh_startepilogue" matching the preexisting ".seh_endprologue". For the directives for saving registers, the exact spelling from the arm64 documentation is picked, e.g. ".seh_save_reg" (to follow that naming for all the other ones, e.g. ".seh_save_fregp_x"), while the corresponding one for x86_64 is plain ".seh_savereg" without the second underscore.

Directives in the epilogues have the same names as in prologues, e.g. .seh_savereg, even though the registers are restored, not saved, at that point.

[efriedma]

llvm/test/CodeGen/AArch64/lrint-conv-fp16-win.ll

7

I think this indicates a bug somewhere else: we shouldn't be emitting seh_startepilogue in functions that don't have any other unwind info.

[mstorsjo]

llvm/test/CodeGen/AArch64/lrint-conv-fp16-win.ll

7

No, we do generate full proper directives for start/endproc, prologue and all that. It's just that the current testcase doesn't break if there's changes in the prologue of the function, but the sequence of CHECK-NEXT from fcvtzs to ret requires including these directives here in order not to break the test.

So for these testcases, that focus on testing some other aspect, I did the minimal changes to include the new directives only where they're needed in CHECK-NEXT sequences, but didn't go full out and added checks for all the generated directives. But I'll include one such case as well.

Alternatively, one could add nounwind attributes to the tested functions, to avoid the noise from the added directives. (Is it possible to achieve the same with some llc flag? I tried looking for one but didn't find any.)

[mstorsjo]

Added a testcase that checks the full sequence of generated .seh directives (in the preexisting wineh1.mir), and added a test within seh.s, that makes llvm-mc output assembly, which is fed back to llvm-mc, to make sure that it can parse what it outputs and that it produces the same output as if assembling directly to object files.

[efriedma]

llvm/test/CodeGen/AArch64/lrint-conv-fp16-win.ll

7

Should we be generating any .seh_* directives in this case? If I'm not mistaken, we don't generate any actual unwind data in the object file.

[mstorsjo]

llvm/test/CodeGen/AArch64/lrint-conv-fp16-win.ll

7

The llc command in this testcase does generate unwind data in the generated object file, if it's written to an object file instead of output as asm.

[efriedma]

LGTM

llvm/test/CodeGen/AArch64/lrint-conv-fp16-win.ll

7

Hmm, I see, we emit a trivial xdata with just an "end" opcode. We probably shouldn't, but I guess you don't need to address that in this patch.

[mstorsjo]

llvm/test/CodeGen/AArch64/lrint-conv-fp16-win.ll

7

I think that's correct - it's a leaf function with no actual, prologue/epilogue so unwinding would be just returning to the address in lr. So emitting that sounds right to me, unless we should omit unwind info for leaf functions. (Unwind info for such functions still is needed for unwinding in the event of crashes and such though.)

Curiously though, it never emits .seh_endprologue, only an empty pair of .seh_startepilogue and .seh_endepilogue. So when writing the unwind info, the supposed prologue actually is completely empty (normally it should at least have a terminating UOP_End), so when interpreting the prologue, it implicitly falls through to the epilogue (which consists of one sole UOP_End) which starts at index 0.

[mstorsjo]

Updated the testcase after updated patch 3/5.

Btw, @efriedma - Did you have any opinions on the naming of all the new .seh_* directive that this patch defines? The names of the new directives are pretty much set in stone once we have it in (or at least once third party projects start using them - and I have one usecase lined up).

[efriedma]

llvm/lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp

5197

Looking at these names again, maybe we could add underscores in a few places. The directive names should match the opcode names from https://docs.microsoft.com/en-us/cpp/build/arm64-exception-handling .

For the names that aren't directly derived from the opcodes in the spec, it probably makes sense to match x86. Which I think the current names do; that seems fine.

llvm/test/CodeGen/AArch64/lrint-conv-fp16-win.ll

7

Curiously though, it never emits .seh_endprologue, only an empty pair of .seh_startepilogue and .seh_endepilogue

That's sounds like a bug, then

unless we should omit unwind info for leaf functions

The Microsoft docs say "leaf functions that don't require any local storage, and don't need to save/restore non-volatile registers, do not require a .pdata record"

[mstorsjo]

llvm/lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp

5197

Ok, will change it that way. That does indeed make it more consistent. The only inconsistency is for the simplest form .seh_save_reg vs x86's .seh_savereg, but that's not much of an issue (the actual register names can't be reused across them anyway), and makes it more consistent and nicer across all of them, from simple .seh_save_reg up to .seh_save_fregp_x.

[mstorsjo]

Added underscores in .seh_save_[f]reg[p][_x] and in .seh_set_fp and .seh_add_fp.

[efriedma]

LGTM

llvm/lib/MC/MCAsmStreamer.cpp

 
 void MCAsmStreamer::EmitWinCFIEndProc(SMLoc Loc) {
   MCStreamer::EmitWinCFIEndProc(Loc);
 
   OS << "\t.seh_endproc";
   EmitEOL();
 }
 
-// TODO: Implement
 void MCAsmStreamer::EmitWinCFIFuncletOrFuncEnd(SMLoc Loc) {
+  MCStreamer::EmitWinCFIFuncletOrFuncEnd(Loc);
+
+  OS << "\t.seh_endfunclet";
+  EmitEOL();
 }
 
 void MCAsmStreamer::EmitWinCFIStartChained(SMLoc Loc) {
   MCStreamer::EmitWinCFIStartChained(Loc);
 
   OS << "\t.seh_startchained";
   EmitEOL();
 }

llvm/lib/MC/MCParser/COFFAsmParser.cpp

void Initialize(MCAsmParser &Parser) override {
     addDirectiveHandler<&COFFAsmParser::ParseDirectiveSymbolAttribute>(".weak");
     addDirectiveHandler<&COFFAsmParser::ParseDirectiveCGProfile>(".cg_profile");
 
     // Win64 EH directives.
     addDirectiveHandler<&COFFAsmParser::ParseSEHDirectiveStartProc>(
                                                                    ".seh_proc");
     addDirectiveHandler<&COFFAsmParser::ParseSEHDirectiveEndProc>(
                                                                 ".seh_endproc");
+    addDirectiveHandler<&COFFAsmParser::ParseSEHDirectiveEndFuncletOrFunc>(
+                                                                ".seh_endfunclet");
     addDirectiveHandler<&COFFAsmParser::ParseSEHDirectiveStartChained>(
                                                            ".seh_startchained");
     addDirectiveHandler<&COFFAsmParser::ParseSEHDirectiveEndChained>(
                                                              ".seh_endchained");
     addDirectiveHandler<&COFFAsmParser::ParseSEHDirectiveHandler>(
                                                                 ".seh_handler");
     addDirectiveHandler<&COFFAsmParser::ParseSEHDirectiveHandlerData>(
                                                             ".seh_handlerdata");
class COFFAsmParser : public MCAsmParserExtension {
   bool parseCOMDATType(COFF::COMDATType &Type);
   bool ParseDirectiveLinkOnce(StringRef, SMLoc);
   bool ParseDirectiveRVA(StringRef, SMLoc);
   bool ParseDirectiveCGProfile(StringRef, SMLoc);
 
   // Win64 EH directives.
   bool ParseSEHDirectiveStartProc(StringRef, SMLoc);
   bool ParseSEHDirectiveEndProc(StringRef, SMLoc);
+  bool ParseSEHDirectiveEndFuncletOrFunc(StringRef, SMLoc);
   bool ParseSEHDirectiveStartChained(StringRef, SMLoc);
   bool ParseSEHDirectiveEndChained(StringRef, SMLoc);
   bool ParseSEHDirectiveHandler(StringRef, SMLoc);
   bool ParseSEHDirectiveHandlerData(StringRef, SMLoc);
   bool ParseSEHDirectiveAllocStack(StringRef, SMLoc);
   bool ParseSEHDirectiveEndProlog(StringRef, SMLoc);
 
   bool ParseAtUnwindOrAtExcept(bool &unwind, bool &except);

 }
 
 bool COFFAsmParser::ParseSEHDirectiveEndProc(StringRef, SMLoc Loc) {
   Lex();
   getStreamer().EmitWinCFIEndProc(Loc);
   return false;
 }
 
+bool COFFAsmParser::ParseSEHDirectiveEndFuncletOrFunc(StringRef, SMLoc Loc) {
+  Lex();
+  getStreamer().EmitWinCFIFuncletOrFuncEnd(Loc);
+  return false;
+}
+
 bool COFFAsmParser::ParseSEHDirectiveStartChained(StringRef, SMLoc Loc) {
   Lex();
   getStreamer().EmitWinCFIStartChained(Loc);
   return false;
 }
 
 bool COFFAsmParser::ParseSEHDirectiveEndChained(StringRef, SMLoc Loc) {
   Lex();

llvm/lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp

 //==- AArch64AsmParser.cpp - Parse AArch64 assembly to MCInst instructions -==//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "MCTargetDesc/AArch64AddressingModes.h"
+#include "MCTargetDesc/AArch64InstPrinter.h"
 #include "MCTargetDesc/AArch64MCExpr.h"
 #include "MCTargetDesc/AArch64MCTargetDesc.h"
 #include "MCTargetDesc/AArch64TargetStreamer.h"
 #include "TargetInfo/AArch64TargetInfo.h"
 #include "AArch64InstrInfo.h"
 #include "Utils/AArch64BaseInfo.h"
 #include "llvm/ADT/APFloat.h"
 #include "llvm/ADT/APInt.h"

   bool parseCondCode(OperandVector &Operands, bool invertCondCode);
   unsigned matchRegisterNameAlias(StringRef Name, RegKind Kind);
   bool parseRegister(OperandVector &Operands);
   bool parseSymbolicImmVal(const MCExpr *&ImmVal);
   bool parseNeonVectorList(OperandVector &Operands);
   bool parseOptionalMulOperand(OperandVector &Operands);
   bool parseOperand(OperandVector &Operands, bool isCondCode,
                     bool invertCondCode);
+  bool parseImmExpr(int64_t &Out);
+  bool parseComma();
+  bool parseRegisterInRange(unsigned &Out, unsigned Base, unsigned First,
+                            unsigned Last);
 
   bool showMatchError(SMLoc Loc, unsigned ErrCode, uint64_t ErrorInfo,
                       OperandVector &Operands);
 
   bool parseDirectiveArch(SMLoc L);
   bool parseDirectiveArchExtension(SMLoc L);
   bool parseDirectiveCPU(SMLoc L);
   bool parseDirectiveInst(SMLoc L);
 
   bool parseDirectiveTLSDescCall(SMLoc L);
 
   bool parseDirectiveLOH(StringRef LOH, SMLoc L);
   bool parseDirectiveLtorg(SMLoc L);
 
   bool parseDirectiveReq(StringRef Name, SMLoc L);
   bool parseDirectiveUnreq(SMLoc L);
   bool parseDirectiveCFINegateRAState();
   bool parseDirectiveCFIBKeyFrame();
 
+  bool parseDirectiveSEHAllocStack(SMLoc L);
+  bool parseDirectiveSEHPrologEnd(SMLoc L);
+  bool parseDirectiveSEHSaveFPLR(SMLoc L);
+  bool parseDirectiveSEHSaveFPLRX(SMLoc L);
+  bool parseDirectiveSEHSaveReg(SMLoc L);
+  bool parseDirectiveSEHSaveRegX(SMLoc L);
+  bool parseDirectiveSEHSaveRegP(SMLoc L);
+  bool parseDirectiveSEHSaveRegPX(SMLoc L);
+  bool parseDirectiveSEHSaveFReg(SMLoc L);
+  bool parseDirectiveSEHSaveFRegX(SMLoc L);
+  bool parseDirectiveSEHSaveFRegP(SMLoc L);
+  bool parseDirectiveSEHSaveFRegPX(SMLoc L);
+  bool parseDirectiveSEHSetFP(SMLoc L);
+  bool parseDirectiveSEHAddFP(SMLoc L);
+  bool parseDirectiveSEHNop(SMLoc L);
+  bool parseDirectiveSEHEpilogStart(SMLoc L);
+  bool parseDirectiveSEHEpilogEnd(SMLoc L);
+
   bool validateInstruction(MCInst &Inst, SMLoc &IDLoc,
                            SmallVectorImpl<SMLoc> &Loc);
   bool MatchAndEmitInstruction(SMLoc IDLoc, unsigned &Opcode,
                                OperandVector &Operands, MCStreamer &Out,
                                uint64_t &ErrorInfo,
                                bool MatchingInlineAsm) override;
 /// @name Auto-generated Match Functions
 /// {

     const MCExpr *CPLoc =
         getTargetStreamer().addConstantPoolEntry(SubExprVal, IsXReg ? 8 : 4, Loc);
     Operands.push_back(AArch64Operand::CreateImm(CPLoc, S, E, Ctx));
     return false;
   }
   }
 }
 
+bool AArch64AsmParser::parseImmExpr(int64_t &Out) {
+  const MCExpr *Expr = nullptr;
+  SMLoc L = getLoc();
+  if (check(getParser().parseExpression(Expr), L, "expected expression"))
+    return true;
+  const MCConstantExpr *Value = dyn_cast_or_null<MCConstantExpr>(Expr);
+  if (check(!Value, L, "expected constant expression"))
+    return true;
+  Out = Value->getValue();
+  return false;
+}
+
+bool AArch64AsmParser::parseComma() {
+  if (check(getParser().getTok().isNot(AsmToken::Comma), getLoc(),
+            "expected comma"))
+    return true;
+  // Eat the comma
+  getParser().Lex();
+  return false;
+}
+
+bool AArch64AsmParser::parseRegisterInRange(unsigned &Out, unsigned Base,
+                                            unsigned First, unsigned Last) {
+  unsigned Reg;
+  SMLoc Start, End;
+  if (check(ParseRegister(Reg, Start, End), getLoc(), "expected register"))
+    return true;
+
+  // Special handling for FP and LR; they aren't linearly after x28 in
+  // the registers enum.
+  unsigned RangeEnd = Last;
+  if (Base == AArch64::X0) {
+    if (Last == AArch64::FP) {
+      RangeEnd = AArch64::X28;
+      if (Reg == AArch64::FP) {
+        Out = 29;
+        return false;
+      }
+    }
+    if (Last == AArch64::LR) {
+      RangeEnd = AArch64::X28;
+      if (Reg == AArch64::FP) {
+        Out = 29;
+        return false;
+      } else if (Reg == AArch64::LR) {
+        Out = 30;
+        return false;
+      }
+    }
+  }
+
+  if (check(Reg < First || Reg > RangeEnd, Start,
+            Twine("expected register in range ") +
+                AArch64InstPrinter::getRegisterName(First) + " to " +
+                AArch64InstPrinter::getRegisterName(Last)))
+    return true;
+  Out = Reg - Base;
+  return false;
+}
+
 bool AArch64AsmParser::regsEqual(const MCParsedAsmOperand &Op1,
                                  const MCParsedAsmOperand &Op2) const {
   auto &AOp1 = static_cast<const AArch64Operand&>(Op1);
   auto &AOp2 = static_cast<const AArch64Operand&>(Op2);
   if (AOp1.getRegEqualityTy() == RegConstraintEqualityTy::EqualsReg &&
       AOp2.getRegEqualityTy() == RegConstraintEqualityTy::EqualsReg)
     return MCTargetAsmParser::regsEqual(Op1, Op2);
 

   llvm_unreachable("Implement any new match types added!");
 }
 
 /// ParseDirective parses the arm specific directives
 bool AArch64AsmParser::ParseDirective(AsmToken DirectiveID) {
   const MCObjectFileInfo::Environment Format =
     getContext().getObjectFileInfo()->getObjectFileType();
   bool IsMachO = Format == MCObjectFileInfo::IsMachO;
+  bool IsCOFF = Format == MCObjectFileInfo::IsCOFF;
 
   auto IDVal = DirectiveID.getIdentifier().lower();
   SMLoc Loc = DirectiveID.getLoc();
   if (IDVal == ".arch")
     parseDirectiveArch(Loc);
   else if (IDVal == ".cpu")
     parseDirectiveCPU(Loc);
   else if (IDVal == ".tlsdesccall")

     parseDirectiveCFIBKeyFrame();
   else if (IDVal == ".arch_extension")
     parseDirectiveArchExtension(Loc);
   else if (IsMachO) {
     if (IDVal == MCLOHDirectiveName())
       parseDirectiveLOH(IDVal, Loc);
     else
       return true;
+  } else if (IsCOFF) {
+    if (IDVal == ".seh_stackalloc")
+      parseDirectiveSEHAllocStack(Loc);
+    else if (IDVal == ".seh_endprologue")
+      parseDirectiveSEHPrologEnd(Loc);
+    else if (IDVal == ".seh_save_fplr")
+      parseDirectiveSEHSaveFPLR(Loc);
+    else if (IDVal == ".seh_save_fplr_x")
+      parseDirectiveSEHSaveFPLRX(Loc);
+    else if (IDVal == ".seh_savereg")
+      parseDirectiveSEHSaveReg(Loc);
+    else if (IDVal == ".seh_savereg_x")
+      parseDirectiveSEHSaveRegX(Loc);
+    else if (IDVal == ".seh_saveregp")
+      parseDirectiveSEHSaveRegP(Loc);
+    else if (IDVal == ".seh_saveregp_x")
+      parseDirectiveSEHSaveRegPX(Loc);
+    else if (IDVal == ".seh_savefreg")
+      parseDirectiveSEHSaveFReg(Loc);
+    else if (IDVal == ".seh_savefreg_x")
+      parseDirectiveSEHSaveFRegX(Loc);
+    else if (IDVal == ".seh_savefregp")
+      parseDirectiveSEHSaveFRegP(Loc);
+    else if (IDVal == ".seh_savefregp_x")
+      parseDirectiveSEHSaveFRegPX(Loc);
+    else if (IDVal == ".seh_setfp")
+      parseDirectiveSEHSetFP(Loc);
+    else if (IDVal == ".seh_addfp")
+      parseDirectiveSEHAddFP(Loc);
+    else if (IDVal == ".seh_nop")
+      parseDirectiveSEHNop(Loc);
+    else if (IDVal == ".seh_startepilogue")
+      parseDirectiveSEHEpilogStart(Loc);
+    else if (IDVal == ".seh_endepilogue")
+      parseDirectiveSEHEpilogEnd(Loc);
+    else
+      return true;
   } else
     return true;
   return false;
 }
 
 static void ExpandCryptoAEK(AArch64::ArchKind ArchKind,
                             SmallVector<StringRef, 4> &RequestedExtensions) {
   const bool NoCrypto = llvm::is_contained(RequestedExtensions, "nocrypto");

 bool AArch64AsmParser::parseDirectiveCFIBKeyFrame() {
   if (parseToken(AsmToken::EndOfStatement,
                  "unexpected token in '.cfi_b_key_frame'"))
     return true;
   getStreamer().emitCFIBKeyFrame();
   return false;
 }
 
+/// parseDirectiveSEHAllocStack
+/// ::= .seh_stackalloc
+bool AArch64AsmParser::parseDirectiveSEHAllocStack(SMLoc L) {
+  int64_t Size;
+  if (parseImmExpr(Size))
+    return true;
+  getTargetStreamer().EmitARM64WinCFIAllocStack(Size);
+  return false;
+}
+
+/// parseDirectiveSEHPrologEnd
+/// ::= .seh_endprologue
+bool AArch64AsmParser::parseDirectiveSEHPrologEnd(SMLoc L) {
+  getTargetStreamer().EmitARM64WinCFIPrologEnd();
+  return false;
+}
+
+/// parseDirectiveSEHSaveFPLR
+/// ::= .seh_save_fplr
+bool AArch64AsmParser::parseDirectiveSEHSaveFPLR(SMLoc L) {
+  int64_t Offset;
+  if (parseImmExpr(Offset))
+    return true;
+  getTargetStreamer().EmitARM64WinCFISaveFPLR(Offset);
+  return false;
+}
+
+/// parseDirectiveSEHSaveFPLRX
+/// ::= .seh_save_fplr_x
+bool AArch64AsmParser::parseDirectiveSEHSaveFPLRX(SMLoc L) {
+  int64_t Offset;
+  if (parseImmExpr(Offset))
+    return true;
+  getTargetStreamer().EmitARM64WinCFISaveFPLRX(Offset);
+  return false;
+}
+
+/// parseDirectiveSEHSaveReg
+/// ::= .seh_savereg
+bool AArch64AsmParser::parseDirectiveSEHSaveReg(SMLoc L) {
+  unsigned Reg;
+  int64_t Offset;
+  if (parseRegisterInRange(Reg, AArch64::X0, AArch64::X19, AArch64::LR) ||
+      parseComma() || parseImmExpr(Offset))
+    return true;
+  getTargetStreamer().EmitARM64WinCFISaveReg(Reg, Offset);
+  return false;
+}
+
+/// parseDirectiveSEHSaveRegX
+/// ::= .seh_savereg_x
+bool AArch64AsmParser::parseDirectiveSEHSaveRegX(SMLoc L) {
+  unsigned Reg;
+  int64_t Offset;
+  if (parseRegisterInRange(Reg, AArch64::X0, AArch64::X19, AArch64::LR) ||
+      parseComma() || parseImmExpr(Offset))
+    return true;
+  getTargetStreamer().EmitARM64WinCFISaveRegX(Reg, Offset);
+  return false;
+}
+
+/// parseDirectiveSEHSaveRegP
+/// ::= .seh_saveregp
+bool AArch64AsmParser::parseDirectiveSEHSaveRegP(SMLoc L) {
+  unsigned Reg;
+  int64_t Offset;
+  if (parseRegisterInRange(Reg, AArch64::X0, AArch64::X19, AArch64::LR) ||
+      parseComma() || parseImmExpr(Offset))
+    return true;
+  getTargetStreamer().EmitARM64WinCFISaveRegP(Reg, Offset);
+  return false;
+}
+
+/// parseDirectiveSEHSaveRegPX
+/// ::= .seh_saveregp_x
+bool AArch64AsmParser::parseDirectiveSEHSaveRegPX(SMLoc L) {
+  unsigned Reg;
+  int64_t Offset;
+  if (parseRegisterInRange(Reg, AArch64::X0, AArch64::X19, AArch64::X28) ||
+      parseComma() || parseImmExpr(Offset))
+    return true;
+  getTargetStreamer().EmitARM64WinCFISaveRegPX(Reg, Offset);
+  return false;
+}
+
+/// parseDirectiveSEHSaveFReg
+/// ::= .seh_savefreg
+bool AArch64AsmParser::parseDirectiveSEHSaveFReg(SMLoc L) {
+  unsigned Reg;
+  int64_t Offset;
+  if (parseRegisterInRange(Reg, AArch64::D0, AArch64::D8, AArch64::D15) ||
+      parseComma() || parseImmExpr(Offset))
+    return true;
+  getTargetStreamer().EmitARM64WinCFISaveFReg(Reg, Offset);
+  return false;
+}
+
+/// parseDirectiveSEHSaveFRegX
+/// ::= .seh_savefreg_x
+bool AArch64AsmParser::parseDirectiveSEHSaveFRegX(SMLoc L) {
+  unsigned Reg;
+  int64_t Offset;
+  if (parseRegisterInRange(Reg, AArch64::D0, AArch64::D8, AArch64::D15) ||
+      parseComma() || parseImmExpr(Offset))
+    return true;
+  getTargetStreamer().EmitARM64WinCFISaveFRegX(Reg, Offset);
+  return false;
+}
+
+/// parseDirectiveSEHSaveFRegP
+/// ::= .seh_savefregp
+bool AArch64AsmParser::parseDirectiveSEHSaveFRegP(SMLoc L) {
+  unsigned Reg;
+  int64_t Offset;
+  if (parseRegisterInRange(Reg, AArch64::D0, AArch64::D8, AArch64::D15) ||
+      parseComma() || parseImmExpr(Offset))
+    return true;
+  getTargetStreamer().EmitARM64WinCFISaveFRegP(Reg, Offset);
+  return false;
+}
+
+/// parseDirectiveSEHSaveFRegPX
+/// ::= .seh_savefregp_x
+bool AArch64AsmParser::parseDirectiveSEHSaveFRegPX(SMLoc L) {
+  unsigned Reg;
+  int64_t Offset;
+  if (parseRegisterInRange(Reg, AArch64::D0, AArch64::D8, AArch64::D15) ||
+      parseComma() || parseImmExpr(Offset))
+    return true;
+  getTargetStreamer().EmitARM64WinCFISaveFRegPX(Reg, Offset);
+  return false;
+}
+
+/// parseDirectiveSEHSetFP
+/// ::= .seh_setfp
+bool AArch64AsmParser::parseDirectiveSEHSetFP(SMLoc L) {
+  getTargetStreamer().EmitARM64WinCFISetFP();
+  return false;
+}
+
+/// parseDirectiveSEHAddFP
+/// ::= .seh_addfp
+bool AArch64AsmParser::parseDirectiveSEHAddFP(SMLoc L) {
+  int64_t Size;
+  if (parseImmExpr(Size))
+    return true;
+  getTargetStreamer().EmitARM64WinCFIAddFP(Size);
+  return false;
+}
+
+/// parseDirectiveSEHNop
+/// ::= .seh_nop
+bool AArch64AsmParser::parseDirectiveSEHNop(SMLoc L) {
+  getTargetStreamer().EmitARM64WinCFINop();
+  return false;
+}
+
+/// parseDirectiveSEHEpilogStart
+/// ::= .seh_startepilogue
+bool AArch64AsmParser::parseDirectiveSEHEpilogStart(SMLoc L) {
+  getTargetStreamer().EmitARM64WinCFIEpilogStart();
+  return false;
+}
+
+/// parseDirectiveSEHEpilogEnd
+/// ::= .seh_endepilogue
+bool AArch64AsmParser::parseDirectiveSEHEpilogEnd(SMLoc L) {
+  getTargetStreamer().EmitARM64WinCFIEpilogEnd();
+  return false;
+}
+
 bool
 AArch64AsmParser::classifySymbolRef(const MCExpr *Expr,
                                     AArch64MCExpr::VariantKind &ELFRefKind,
                                     MCSymbolRefExpr::VariantKind &DarwinRefKind,
                                     int64_t &Addend) {
   ELFRefKind = AArch64MCExpr::VK_INVALID;
   DarwinRefKind = MCSymbolRefExpr::VK_None;
   Addend = 0;

llvm/lib/Target/AArch64/MCTargetDesc/AArch64ELFStreamer.cpp

 
 class AArch64ELFStreamer;
 
 class AArch64TargetAsmStreamer : public AArch64TargetStreamer {
   formatted_raw_ostream &OS;
 
   void emitInst(uint32_t Inst) override;
 
+  void EmitARM64WinCFIAllocStack(unsigned Size) override {
+    OS << "\t.seh_stackalloc " << Size << "\n";
+  }
+  void EmitARM64WinCFISaveFPLR(int Offset) override {
+    OS << "\t.seh_save_fplr " << Offset << "\n";
+  }
+  void EmitARM64WinCFISaveFPLRX(int Offset) override {
+    OS << "\t.seh_save_fplr_x " << Offset << "\n";
+  }
+  void EmitARM64WinCFISaveReg(unsigned Reg, int Offset) override {
+    OS << "\t.seh_savereg x" << Reg << ", " << Offset << "\n";
+  }
+  void EmitARM64WinCFISaveRegX(unsigned Reg, int Offset) override {
+    OS << "\t.seh_savereg_x x" << Reg << ", " << Offset << "\n";
+  }
+  void EmitARM64WinCFISaveRegP(unsigned Reg, int Offset) override {
+    OS << "\t.seh_saveregp x" << Reg << ", " << Offset << "\n";
+  }
+  void EmitARM64WinCFISaveRegPX(unsigned Reg, int Offset) override {
+    OS << "\t.seh_saveregp_x x" << Reg << ", " << Offset << "\n";
+  }
+  void EmitARM64WinCFISaveFReg(unsigned Reg, int Offset) override {
+    OS << "\t.seh_savefreg d" << Reg << ", " << Offset << "\n";
+  }
+  void EmitARM64WinCFISaveFRegX(unsigned Reg, int Offset) override {
+    OS << "\t.seh_savefreg_x d" << Reg << ", " << Offset << "\n";
+  }
+  void EmitARM64WinCFISaveFRegP(unsigned Reg, int Offset) override {
+    OS << "\t.seh_savefregp d" << Reg << ", " << Offset << "\n";
+  }
+  void EmitARM64WinCFISaveFRegPX(unsigned Reg, int Offset) override {
+    OS << "\t.seh_savefregp_x d" << Reg << ", " << Offset << "\n";
+  }
+  void EmitARM64WinCFISetFP() override { OS << "\t.seh_setfp\n"; }
+  void EmitARM64WinCFIAddFP(unsigned Size) override {
+    OS << "\t.seh_addfp " << Size << "\n";
+  }
+  void EmitARM64WinCFINop() override { OS << "\t.seh_nop\n"; }
+  void EmitARM64WinCFIPrologEnd() override { OS << "\t.seh_endprologue\n"; }
+  void EmitARM64WinCFIEpilogStart() override { OS << "\t.seh_startepilogue\n"; }
+  void EmitARM64WinCFIEpilogEnd() override { OS << "\t.seh_endepilogue\n"; }
+
 public:
   AArch64TargetAsmStreamer(MCStreamer &S, formatted_raw_ostream &OS);
 };
 
 AArch64TargetAsmStreamer::AArch64TargetAsmStreamer(MCStreamer &S,
                                                    formatted_raw_ostream &OS)
   : AArch64TargetStreamer(S), OS(OS) {}
 

llvm/test/CodeGen/AArch64/arm64-windows-calls.ll

 %struct.S2 = type { i32, i32, i32, i32 }
 define dso_local [2 x i64] @"?f2"() {
 entry:
 ; FIXME: Missed optimization, the entire SP push/pop could be removed
 ; CHECK-LABEL: f2
 ; CHECK:         stp     xzr, xzr, [sp, #-16]!
 ; CHECK-NEXT:    mov     x0, xzr
 ; CHECK-NEXT:    mov     x1, xzr
+; CHECK-NEXT:    .seh_startepilogue
 ; CHECK-NEXT:    add     sp, sp, #16
 
   %retval = alloca %struct.S2, align 4
   %a = getelementptr inbounds %struct.S2, %struct.S2* %retval, i32 0, i32 0
   store i32 0, i32* %a, align 4
   %b = getelementptr inbounds %struct.S2, %struct.S2* %retval, i32 0, i32 1
   store i32 0, i32* %b, align 4
   %c = getelementptr inbounds %struct.S2, %struct.S2* %retval, i32 0, i32 2

llvm/test/CodeGen/AArch64/lrint-conv-fp16-win.ll

 ; RUN: llc < %s -mtriple=aarch64-windows -mattr=+fullfp16 | FileCheck %s
 
 ; CHECK-LABEL: testmhhs:
 ; CHECK:       frintx  h0, h0
 ; CHECK-NEXT:  fcvtzs  w0, h0
+; CHECK-NEXT:  .seh_startepilogue
+; CHECK-NEXT:  .seh_endepilogue
 ; CHECK-NEXT:  ret
 define i16 @testmhhs(half %x) {
 entry:
   %0 = tail call i32 @llvm.lrint.i32.f16(half %x)
   %conv = trunc i32 %0 to i16
   ret i16 %conv
 }
 
 ; CHECK-LABEL: testmhws:
 ; CHECK:       frintx  h0, h0
 ; CHECK-NEXT:  fcvtzs  w0, h0
+; CHECK-NEXT:  .seh_startepilogue
+; CHECK-NEXT:  .seh_endepilogue
 ; CHECK-NEXT:  ret
 define i32 @testmhws(half %x) {
 entry:
   %0 = tail call i32 @llvm.lrint.i32.f16(half %x)
   ret i32 %0
 }
 
 ; CHECK-LABEL: testmhxs:
 ; CHECK:       frintx  h0, h0
 ; CHECK-NEXT:  fcvtzs  w8, h0
 ; CHECK-NEXT:  sxtw    x0, w8
+; CHECK-NEXT:  .seh_startepilogue
+; CHECK-NEXT:  .seh_endepilogue
 ; CHECK-NEXT:  ret
 define i64 @testmhxs(half %x) {
 entry:
   %0 = tail call i32 @llvm.lrint.i32.f16(half %x)
   %conv = sext i32 %0 to i64
   ret i64 %conv
 }
 
 declare i32 @llvm.lrint.i32.f16(half) nounwind readnone

llvm/test/CodeGen/AArch64/lrint-conv-win.ll

 ; RUN: llc < %s -mtriple=aarch64-windows -mattr=+neon | FileCheck %s
 
 ; CHECK-LABEL: testmsxs:
 ; CHECK:       frintx  [[SREG:s[0-9]+]], s0
 ; CHECK-NEXT:  fcvtzs  [[WREG:w[0-9]+]], [[SREG]]
 ; CHECK-NEXT:  sxtw    x0, [[WREG]]
+; CHECK-NEXT:  .seh_startepilogue
+; CHECK-NEXT:  .seh_endepilogue
 ; CHECK-NEXT:  ret
 define i64 @testmsxs(float %x) {
 entry:
   %0 = tail call i32 @llvm.lrint.i32.f32(float %x)
   %conv = sext i32 %0 to i64
   ret i64 %conv
 }
 
 ; CHECK-LABEL: testmsws:
 ; CHECK:       frintx  [[SREG:s[0-9]+]], s0
 ; CHECK-NEXT:  fcvtzs  [[WREG:w[0-9]+]], [[SREG]]
+; CHECK-NEXT:  .seh_startepilogue
+; CHECK-NEXT:  .seh_endepilogue
 ; CHECK-NEXT:  ret
 define i32 @testmsws(float %x) {
 entry:
   %0 = tail call i32 @llvm.lrint.i32.f32(float %x)
   ret i32 %0
 }
 
 ; CHECK-LABEL: testmsxd:
 ; CHECK:       frintx  [[DREG:d[0-9]+]], d0
 ; CHECK-NEXT:  fcvtzs  [[WREG:w[0-9]+]], [[DREG]]
 ; CHECK-NEXT:  sxtw    x0, [[WREG]]
+; CHECK-NEXT:  .seh_startepilogue
+; CHECK-NEXT:  .seh_endepilogue
 ; CHECK-NEXT:  ret
 define i64 @testmsxd(double %x) {
 entry:
   %0 = tail call i32 @llvm.lrint.i32.f64(double %x)
   %conv = sext i32 %0 to i64
   ret i64 %conv
 }
 
 ; CHECK-LABEL: testmswd:
 ; CHECK:       frintx  [[DREG:d[0-9]+]], d0
 ; CHECK-NEXT:  fcvtzs  [[WREG:w[0-9]+]], [[DREG]]
+; CHECK-NEXT:  .seh_startepilogue
+; CHECK-NEXT:  .seh_endepilogue
 ; CHECK-NEXT:  ret
 define i32 @testmswd(double %x) {
 entry:
   %0 = tail call i32 @llvm.lrint.i32.f64(double %x)
   ret i32 %0
 }
 
 declare i32 @llvm.lrint.i32.f32(float) nounwind readnone
 declare i32 @llvm.lrint.i32.f64(double) nounwind readnone

llvm/test/CodeGen/AArch64/lround-conv-fp16-win.ll

 entry:
   %0 = tail call i32 @llvm.lround.i32.f16(half %x)
   ret i32 %0
 }
 
 ; CHECK-LABEL: testmhxs:
 ; CHECK:       fcvtas  w8, h0
 ; CHECK-NEXT:  sxtw    x0, w8
+; CHECK-NEXT:  .seh_startepilogue
+; CHECK-NEXT:  .seh_endepilogue
 ; CHECK-NEXT:  ret
 define i64 @testmhxs(half %x) {
 entry:
   %0 = tail call i32 @llvm.lround.i32.f16(half %x)
   %conv = sext i32 %0 to i64
   ret i64 %conv
 }
 
 declare i32 @llvm.lround.i32.f16(half) nounwind readnone

llvm/test/CodeGen/AArch64/lround-conv-win.ll

 ; RUN: llc < %s -mtriple=aarch64-windows -mattr=+neon | FileCheck %s
 
 ; CHECK-LABEL: testmsxs:
 ; CHECK:       fcvtas  w8, s0
 ; CHECK-NEXT:  sxtw    x0, w8
+; CHECK-NEXT:  .seh_startepilogue
+; CHECK-NEXT:  .seh_endepilogue
 ; CHECK-NEXT:  ret
 define i64 @testmsxs(float %x) {
 entry:
   %0 = tail call i32 @llvm.lround.i32.f32(float %x)
   %conv = sext i32 %0 to i64
   ret i64 %conv
 }
 
 ; CHECK-LABEL: testmsws:
 ; CHECK:       fcvtas  w0, s0
+; CHECK-NEXT:  .seh_startepilogue
+; CHECK-NEXT:  .seh_endepilogue
 ; CHECK-NEXT:  ret
 define i32 @testmsws(float %x) {
 entry:
   %0 = tail call i32 @llvm.lround.i32.f32(float %x)
   ret i32 %0
 }
 
 ; CHECK-LABEL: testmsxd:
 ; CHECK:       fcvtas  w8, d0
 ; CHECK-NEXT:  sxtw    x0, w8
+; CHECK-NEXT:  .seh_startepilogue
+; CHECK-NEXT:  .seh_endepilogue
 ; CHECK-NEXT:  ret
 define i64 @testmsxd(double %x) {
 entry:
   %0 = tail call i32 @llvm.lround.i32.f64(double %x)
   %conv = sext i32 %0 to i64
   ret i64 %conv
 }
 
 ; CHECK-LABEL: testmswd:
 ; CHECK:       fcvtas  w0, d0
+; CHECK-NEXT:  .seh_startepilogue
+; CHECK-NEXT:  .seh_endepilogue
 ; CHECK-NEXT:  ret
 define i32 @testmswd(double %x) {
 entry:
   %0 = tail call i32 @llvm.lround.i32.f64(double %x)
   ret i32 %0
 }
 
 declare i32 @llvm.lround.i32.f32(float) nounwind readnone
 declare i32 @llvm.lround.i32.f64(double) nounwind readnone

llvm/test/CodeGen/AArch64/powi-windows.ll

 ; RUN: llc -mtriple aarch64-windows < %s | FileCheck %s
 
 declare double @llvm.powi.f64(double, i32)
 declare float @llvm.powi.f32(float, i32)
 
 define double @d(double %d, i32 %i) {
 entry:
   %0 = tail call double @llvm.powi.f64(double %d, i32 %i)
   ret double %0
 }
 
 ; CHECK-LABEL: d:
 ; CHECK: scvtf d1, w0
+; CHECK-NEXT:  .seh_startepilogue
+; CHECK-NEXT:  .seh_endepilogue
 ; CHECK-NEXT: b pow
 
 define float @f(float %f, i32 %i) {
 entry:
   %0 = tail call float @llvm.powi.f32(float %f, i32 %i)
   ret float %0
 }
 
 ; CHECK-LABEL: f:
 ; CHECK: scvtf s1, w0
+; CHECK-NEXT:  .seh_startepilogue
+; CHECK-NEXT:  .seh_endepilogue
 ; CHECK-NEXT: b powf
 
 define float @g(double %d, i32 %i) {
 entry:
   %0 = tail call double @llvm.powi.f64(double %d, i32 %i)
   %conv = fptrunc double %0 to float
   ret float %conv
 }

llvm/test/CodeGen/AArch64/seh_funclet_x1.ll

 ; RUN: llc -o - %s -verify-machineinstrs -mtriple=aarch64-windows | FileCheck %s
 
 ; Windows runtime passes the establisher frame as the second argument to the
 ; termination handler.  Check that we copy it into fp.
 
 ; CHECK:      ?dtor$3@?0?main@4HA":
 ; CHECK:      .seh_proc "?dtor$3@?0?main@4HA"
 ; CHECK:      stp     x29, x30, [sp, #-16]!   ; 16-byte Folded Spill
+; CHECK-NEXT: .seh_save_fplr_x 16
+; CHECK-NEXT: .seh_endprologue
 ; CHECK-NEXT: mov     x29, x1
 
 target datalayout = "e-m:w-p:64:64-i32:32-i64:64-i128:128-n32:64-S128"
 target triple = "aarch64-unknown-windows-msvc19.15.26732"
 
 ; Function Attrs: noinline nounwind optnone uwtable
 define dso_local i32 @main() #0 personality i8* bitcast (i32 (...)* @__C_specific_handler to i8*) {
 entry:

llvm/test/CodeGen/AArch64/win64-jumptable.ll

 }
 
 declare void @g(i32, i32)
 
 ; CHECK:    .text
 ; CHECK:    f:
 ; CHECK:    .seh_proc f
 ; CHECK:    b g
+; CHECK-NEXT:  .seh_endfunclet
 ; CHECK-NEXT: .p2align  2
 ; CHECK-NEXT: .LJTI0_0:
 ; CHECK:    .word .LBB0_2-.LJTI0_0
 ; CHECK:    .word .LBB0_3-.LJTI0_0
 ; CHECK:    .word .LBB0_4-.LJTI0_0
 ; CHECK:    .word .LBB0_5-.LJTI0_0
 ; CHECK:    .section  .xdata,"dr"
 ; CHECK:    .seh_handlerdata
 ; CHECK:    .text
 ; CHECK:    .seh_endproc
 
 ; Check that we can emit an object file with correct unwind info.
 ; UNWIND: FunctionLength: {{[1-9][0-9]*}}

llvm/test/CodeGen/AArch64/win_cst_pool.ll

 ; RUN: llc < %s -mtriple=aarch64-win32-msvc | FileCheck %s
 ; RUN: llc < %s -mtriple=aarch64-win32-gnu | FileCheck -check-prefix=MINGW %s
 
 define double @double() {
   ret double 0x2000000000800001
 }
 ; CHECK:              .globl  __real@2000000000800001
 ; CHECK-NEXT:         .section        .rdata,"dr",discard,__real@2000000000800001
 ; CHECK-NEXT:         .p2align  3
 ; CHECK-NEXT: __real@2000000000800001:
 ; CHECK-NEXT:         .xword   0x2000000000800001
 ; CHECK:      double:
 ; CHECK:               adrp    x8, __real@2000000000800001
 ; CHECK-NEXT:          ldr     d0, [x8, __real@2000000000800001]
+; CHECK-NEXT:          .seh_startepilogue
+; CHECK-NEXT:          .seh_endepilogue
 ; CHECK-NEXT:          ret
 
 ; MINGW:              .section        .rdata,"dr"
 ; MINGW-NEXT:         .p2align  3
 ; MINGW-NEXT: [[LABEL:\.LC.*]]:
 ; MINGW-NEXT:         .xword  0x2000000000800001
 ; MINGW:      double:
 ; MINGW:               adrp    x8, [[LABEL]]
 ; MINGW-NEXT:          ldr     d0, [x8, [[LABEL]]]
+; MINGW-NEXT:          .seh_startepilogue
+; MINGW-NEXT:          .seh_endepilogue
 ; MINGW-NEXT:          ret

llvm/test/CodeGen/AArch64/windows-extern-weak.ll

 ; RUN: llc -mtriple aarch64-windows -filetype asm -o - < %s | FileCheck %s
 ; RUN: llc -mtriple aarch64-windows -filetype asm -o - -fast-isel %s | FileCheck %s
 ; RUN: llc -mtriple aarch64-windows -filetype asm -o - -global-isel -global-isel-abort=0 %s | FileCheck %s
 
 define void @func() {
 ; CHECK-LABEL: func:
 ; CHECK:      str x30, [sp, #-16]!
+; CHECK-NEXT: .seh_savereg_x x30, 16
+; CHECK-NEXT: .seh_endprologue
 ; CHECK-NEXT: adrp x8, .refptr.weakfunc
 ; CHECK-NEXT: ldr x8, [x8, .refptr.weakfunc]
 ; CHECK-NEXT: cbz     x8, .LBB0_2
 ; CHECK-NEXT: ; %bb.1:
 ; CHECK-NEXT: blr     x8
 ; CHECK-NEXT: .LBB0_2:
+; CHECK-NEXT: .seh_startepilogue
 ; CHECK-NEXT: ldr x30, [sp], #16
+; CHECK-NEXT: .seh_savereg_x x30, 16
+; CHECK-NEXT: .seh_endepilogue
 ; CHECK-NEXT: ret
 
   br i1 icmp ne (void ()* @weakfunc, void ()* null), label %1, label %2
 
 1:
   call void @weakfunc()
   br label %2
 
 2:
   ret void
 }
 
 declare extern_weak void @weakfunc()

llvm/test/CodeGen/AArch64/wineh-try-catch-cbz.ll

 ; RUN: llc < %s | FileCheck %s
 
 ; Make sure the prologue is sane.  (Doesn't need to exactly match this,
 ; but the original issue only reproduced if the cbz was immediately
 ; after the frame setup.)
 
 ; CHECK: stp     x29, x30, [sp, #-32]!
+; CHECK-NEXT: .seh_save_fplr_x 32
 ; CHECK-NEXT: mov     x29, sp
+; CHECK-NEXT: .seh_setfp
+; CHECK-NEXT: .seh_endprologue
 ; CHECK-NEXT: mov     x1, #-2
 ; CHECK-NEXT: stur    x1, [x29, #16]
 ; CHECK-NEXT: cbz     w0, .LBB0_2
 
 target datalayout = "e-m:w-p:64:64-i32:32-i64:64-i128:128-n32:64-S128"
 target triple = "aarch64-unknown-windows-msvc19.11.0"
 
 ; Function Attrs: uwtable

llvm/test/CodeGen/AArch64/wineh-try-catch-nobase.ll

 ; RUN: llc -o - %s -mtriple=aarch64-windows -verify-machineinstrs | FileCheck %s
 
 ; Make sure we don't have a base pointer.
 ; CHECK-LABEL: "?a@@YAXXZ":
 ; CHECK-NOT: x19
 
 ; Check that we compute the address relative to fp.
 ; CHECK-LABEL: "?catch$2@?0??a@@YAXXZ@4HA":
 ; CHECK:             stp     x29, x30, [sp, #-16]!   ; 16-byte Folded Spill
+; CHECK-NEXT:        .seh_save_fplr_x 16
+; CHECK-NEXT:        .seh_endprologue
 ; CHECK-NEXT:        sub     x0, x29, #16            ; =16
 ; CHECK-NEXT:        mov     x1, xzr
 ; CHECK-NEXT:        bl      "?bb@@YAXPEAHH@Z"
 ; CHECK-NEXT:        adrp    x0, .LBB0_1
 ; CHECK-NEXT:        add     x0, x0, .LBB0_1
+; CHECK-NEXT:        .seh_startepilogue
 ; CHECK-NEXT:        ldp     x29, x30, [sp], #16     ; 16-byte Folded Reload
+; CHECK-NEXT:        .seh_save_fplr_x 16
+; CHECK-NEXT:        .seh_endepilogue
 ; CHECK-NEXT:        ret
 
 target datalayout = "e-m:w-p:64:64-i32:32-i64:64-i128:128-n32:64-S128"
 target triple = "aarch64-unknown-windows-msvc19.11.0"
 
 define dso_local void @"?a@@YAXXZ"(i64 %p1) personality i8* bitcast (i32 (...)* @__CxxFrameHandler3 to i8*) {
 entry:
   %a = alloca i32, align 16

llvm/test/CodeGen/AArch64/wineh-try-catch-realign.ll

 ; RUN: llc -o - %s -mtriple=aarch64-windows -verify-machineinstrs | FileCheck %s
 
 ; Make sure we have a base pointer.
 ; CHECK-LABEL: "?a@@YAXXZ":
 ; CHECK: and     sp, x9, #0xffffffffffffffc0
 ; CHECK: mov     x19, sp
 
 ; Make sure the funclet prologue/epilogue are correct: specifically,
 ; it shouldn't access the parent's frame via sp, and the prologue and
 ; epilogue should be symmetrical.
 ; CHECK-LABEL: "?catch$2@?0??a@@YAXXZ@4HA":
 ; CHECK:      stp     x29, x30, [sp, #-32]!
+; CHECK-NEXT: .seh_save_fplr_x 32
 ; CHECK-NEXT: str     x28, [sp, #16]
+; CHECK-NEXT: .seh_savereg x28, 16
 ; CHECK-NEXT: str     x19, [sp, #24]
+; CHECK-NEXT: .seh_savereg x19, 24
+; CHECK-NEXT: .seh_endprologue
 ; CHECK-NEXT: add     x0, x19, #0
 ; CHECK-NEXT: mov     w1, wzr
 ; CHECK-NEXT: bl      "?bb@@YAXPEAHH@Z"
 ; CHECK-NEXT: adrp    x0, .LBB0_1
 ; CHECK-NEXT: add     x0, x0, .LBB0_1
+; CHECK-NEXT: .seh_startepilogue
 ; CHECK-NEXT: ldr     x19, [sp, #24]
+; CHECK-NEXT: .seh_savereg x19, 24
 ; CHECK-NEXT: ldr     x28, [sp, #16]
+; CHECK-NEXT: .seh_savereg x28, 16
 ; CHECK-NEXT: ldp     x29, x30, [sp], #32
+; CHECK-NEXT: .seh_save_fplr_x 32
+; CHECK-NEXT: .seh_endepilogue
 ; CHECK-NEXT: ret
 
 
 target datalayout = "e-m:w-p:64:64-i32:32-i64:64-i128:128-n32:64-S128"
 target triple = "aarch64-unknown-windows-msvc19.11.0"
 
 define dso_local void @"?a@@YAXXZ"() personality i8* bitcast (i32 (...)* @__CxxFrameHandler3 to i8*) {
 entry:

llvm/test/CodeGen/AArch64/wineh-try-catch-vla.ll

 ; RUN: llc -o - %s -mtriple=aarch64-windows -verify-machineinstrs | FileCheck %s
 
 ; Check we don't have a base pointer.
 ; CHECK-LABEL: "?a@@YAXXZ":
 ; CHECK-NOT: x19
 
 ; Make sure the prologue and epilogue are sane. Make sure the
 ; frame index is relative to the FP, since there is no base pointer.
 ; (Funclets aren't allowed to contain dynamic allocas.)
 ; CHECK-LABEL: "?catch$2@?0??a@@YAXXZ@4HA":
 ; CHECK:      stp     x29, x30, [sp, #-16]!
+; CHECK-NEXT: .seh_save_fplr_x 16
+; CHECK-NEXT: .seh_endprologue
 ; CHECK-NEXT: ldur    x0, [x29, #-8]
 ; CHECK-NEXT: mov     x1, x0
 ; CHECK-NEXT: bl      "?bb@@YAXPEAHH@Z"
 ; CHECK-NEXT: adrp    x0, .LBB0_1
 ; CHECK-NEXT: add     x0, x0, .LBB0_1
+; CHECK-NEXT: .seh_startepilogue
 ; CHECK-NEXT: ldp     x29, x30, [sp], #16
+; CHECK-NEXT: .seh_save_fplr_x 16
+; CHECK-NEXT: .seh_endepilogue
 ; CHECK-NEXT: ret
 
 target datalayout = "e-m:w-p:64:64-i32:32-i64:64-i128:128-n32:64-S128"
 target triple = "aarch64-unknown-windows-msvc19.11.0"
 
 define dso_local void @"?a@@YAXXZ"(i64 %p1) personality i8* bitcast (i32 (...)* @__CxxFrameHandler3 to i8*) {
 entry:
   %a = alloca i32, i64 %p1, align 16

llvm/test/CodeGen/AArch64/wineh1.mir

 # RUN: llc -o - %s -mtriple=aarch64-windows -start-after=prologepilog -filetype=obj  \
 # RUN:   | llvm-readobj --unwind - | FileCheck %s
 # RUN: llc -o - %s -mtriple=aarch64-windows -run-pass=aarch64-ldst-opt \
 # RUN:   | FileCheck %s --check-prefix=CHECK-LDSTOPT
 # This test case checks the basic validity of the .xdata section.  It's
 # documented at:
 # https://docs.microsoft.com/en-us/cpp/build/arm64-exception-handling
 
+# Also test the generated assembler SEH directives.
+# RUN: llc -o - %s -mtriple=aarch64-windows -start-after=prologepilog -filetype=asm  \
+# RUN:   | FileCheck %s --check-prefix=ASM
+
 # We expect to see the following in the .xdata section:
 
 # CHECK:   ExceptionData {
 # CHECK-NEXT:      FunctionLength: 96
 # CHECK-NEXT:      Version: 0
 # CHECK-NEXT:      ExceptionData: No
 # CHECK-NEXT:      EpiloguePacked: No
 # CHECK-NEXT:      EpilogueScopes: 1

 # CHECK-NEXT:      ]
 # CHECK-NEXT:    }
 
 # Check that the load-store optimizer does not merge the two
 # callee-saved stores in the prologue.
 # CHECK-LDSTOPT: name: test
 # CHECK-LDSTOPT: frame-setup STRXui killed $x21, $sp, 6
 # CHECK-LDSTOPT: frame-setup STRXui killed $x22, $sp, 7
+
+# ASM-LABEL: test:
+# ASM: .seh_proc test
+# ASM: .seh_saveregp_x x27, 80
+# ASM: .seh_saveregp x25, 16
+# ASM: .seh_saveregp x23, 32
+# ASM: .seh_savereg x21, 48
+# ASM: .seh_savereg x22, 56
+# ASM: .seh_saveregp x19, 64
+# ASM: .seh_endprologue
+
+# ASM: .seh_startepilogue
+# ASM: .seh_saveregp x19, 64
+# ASM: .seh_savereg x21, 48
+# ASM: .seh_nop
+# ASM: .seh_savereg x22, 56
+# ASM: .seh_saveregp x23, 32
+# ASM: .seh_saveregp x25, 16
+# ASM: .seh_saveregp_x x27, 80
+# ASM: .seh_endepilogue
+
+# ASM: .seh_endfunclet
+# ASM: .section .xdata,"dr"
+# ASM: .seh_handlerdata
+# ASM: .text
+# ASM: .seh_endproc
+
 ...
 ---
 name:            test
 alignment:       4
 tracksRegLiveness: true
 hasWinCFI: true
 liveins:
   - { reg: '$w0' }

llvm/test/MC/AArch64/seh.s

-// This test checks that the SEH directives don't cause the assembler to fail.
-// Checking that llvm-readobj doesn't bail out on the unwind data, but not
-// really checking the contents yet.
+// This test checks that the SEH directives emit the correct unwind data.
 
 // RUN: llvm-mc -triple aarch64-pc-win32 -filetype=obj %s | llvm-readobj -S -r -u - | FileCheck %s
 
+// Check that the output assembler directives also can be parsed, and
+// that they produce equivalent output:
+
+// RUN: llvm-mc -triple aarch64-pc-win32 -filetype=asm %s | llvm-mc -triple aarch64-pc-win32 -filetype=obj - | llvm-readobj -S -r -u - | FileCheck %s
+
 // CHECK:      Sections [
 // CHECK:        Section {
 // CHECK:          Name: .text
 // CHECK:          RelocationCount: 0
 // CHECK:          Characteristics [
 // CHECK-NEXT:       ALIGN_4BYTES
 // CHECK-NEXT:       CNT_CODE
 // CHECK-NEXT:       MEM_EXECUTE
 // CHECK-NEXT:       MEM_READ
 // CHECK-NEXT:     ]
 // CHECK-NEXT:   }
 // CHECK:        Section {
 // CHECK:          Name: .xdata
-// CHECK:          RawDataSize: 32
+// CHECK:          RawDataSize: 60
 // CHECK:          RelocationCount: 2
 // CHECK:          Characteristics [
 // CHECK-NEXT:       ALIGN_4BYTES
 // CHECK-NEXT:       CNT_INITIALIZED_DATA
 // CHECK-NEXT:       MEM_READ
 // CHECK-NEXT:     ]
 // CHECK-NEXT:   }
 // CHECK:        Section {
 // CHECK:          Name: .pdata
 // CHECK:          RelocationCount: 4
 // CHECK:          Characteristics [
 // CHECK-NEXT:       ALIGN_4BYTES
 // CHECK-NEXT:       CNT_INITIALIZED_DATA
 // CHECK-NEXT:       MEM_READ
 // CHECK-NEXT:     ]
 // CHECK-NEXT:   }
 // CHECK-NEXT: ]
 
 // CHECK-NEXT: Relocations [
 // CHECK-NEXT:   Section (4) .xdata {
-// CHECK-NEXT:     0x8 IMAGE_REL_ARM64_ADDR32NB __C_specific_handler
-// CHECK-NEXT:     0x18 IMAGE_REL_ARM64_ADDR32NB __C_specific_handler
+// CHECK-NEXT:     0x24 IMAGE_REL_ARM64_ADDR32NB __C_specific_handler
+// CHECK-NEXT:     0x34 IMAGE_REL_ARM64_ADDR32NB __C_specific_handler
 // CHECK-NEXT:   }
 // CHECK-NEXT:   Section (5) .pdata {
 // CHECK-NEXT:     0x0 IMAGE_REL_ARM64_ADDR32NB func
 // CHECK-NEXT:     0x4 IMAGE_REL_ARM64_ADDR32NB .xdata
 // CHECK-NEXT:     0x8 IMAGE_REL_ARM64_ADDR32NB handlerFunc
 // CHECK-NEXT:     0xC IMAGE_REL_ARM64_ADDR32NB .xdata
 // CHECK-NEXT:   }
 // CHECK-NEXT: ]
 
 // CHECK-NEXT: UnwindInformation [
 // CHECK-NEXT:   RuntimeFunction {
 // CHECK-NEXT:     Function: func
 // CHECK-NEXT:     ExceptionRecord: .xdata
 // CHECK-NEXT:     ExceptionData {
-// CHECK-NEXT:       FunctionLength: 8
-// CHECK:        RuntimeFunction {
+// CHECK-NEXT:       FunctionLength: 72
+// CHECK:            Prologue [
+// CHECK-NEXT:         0xe3                ; nop
+// CHECK-NEXT:         0xe202              ; add fp, sp, #16
+// CHECK-NEXT:         0xdd41              ; str d13, [sp, #8]
+// CHECK-NEXT:         0xde83              ; str d12, [sp, #-32]!
+// CHECK-NEXT:         0xd882              ; stp d10, d11, [sp, #16]
+// CHECK-NEXT:         0xda03              ; stp d8, d9, [sp, #-32]!
+// CHECK-NEXT:         0x83                ; stp x29, x30, [sp, #-32]!
+// CHECK-NEXT:         0x46                ; stp x29, x30, [sp, #48]
+// CHECK-NEXT:         0xd141              ; str x24, [sp, #8]
+// CHECK-NEXT:         0xd483              ; str x23, [sp, #-32]!
+// CHECK-NEXT:         0xc882              ; stp x21, x22, [sp, #16]
+// CHECK-NEXT:         0xcc03              ; stp x19, x20, [sp, #-32]!
+// CHECK-NEXT:         0x83                ; stp x29, x30, [sp, #-32]!
+// CHECK-NEXT:         0xe1                ; mov fp, sp
+// CHECK-NEXT:         0x01                ; sub sp, #16
+// CHECK-NEXT:         0xe4                ; end
+// CHECK-NEXT:       ]
+// CHECK-NEXT:       EpilogueScopes [
+// CHECK-NEXT:         EpilogueScope {
+// CHECK-NEXT:           StartOffset: 16
+// CHECK-NEXT:           EpilogueStartIndex: 25
+// CHECK-NEXT:           Opcodes [
+// CHECK-NEXT:             0x01                ; add sp, #16
+// CHECK-NEXT:             0xe4                ; end
+// CHECK-NEXT:           ]
+// CHECK-NEXT:         }
+// CHECK-NEXT:       ]
+// CHECK-NEXT:       ExceptionHandler [
+// CHECK-NEXT:         Routine: __C_specific_handler (0x0)
+// CHECK-NEXT:         Parameter: 0x0
+// CHECK-NEXT:       ]
+// CHECK-NEXT:     }
+// CHECK-NEXT:   }
+// CHECK-NEXT:   RuntimeFunction {
 // CHECK-NEXT:     Function: handlerFunc
 // CHECK-NEXT:     ExceptionRecord: .xdata
 // CHECK-NEXT:     ExceptionData {
 // CHECK-NEXT:       FunctionLength: 4
+// CHECK-NEXT:       Version: 0
+// CHECK-NEXT:       ExceptionData: Yes
+// CHECK-NEXT:       EpiloguePacked: No
+// CHECK-NEXT:       EpilogueScopes: 0
+// CHECK-NEXT:       ByteCodeLength: 0
+// CHECK-NEXT:       Prologue [
+// CHECK-NEXT:       ]
+// CHECK-NEXT:       EpilogueScopes [
+// CHECK-NEXT:       ]
+// CHECK-NEXT:       ExceptionHandler [
+// CHECK-NEXT:         Routine: __C_specific_handler (0x0)
+// CHECK-NEXT:         Parameter: 0x0
+// CHECK-NEXT:       ]
+// CHECK-NEXT:     }
+// CHECK-NEXT:   }
+// CHECK-NEXT: ]
+
 
     .text
     .globl func
     .def func
     .scl 2
     .type 32
     .endef
     .seh_proc func
 func:
     sub sp, sp, #24
     .seh_stackalloc 24
     mov x29, sp
+    .seh_setfp
+    stp x29, x30, [sp, #-32]!
+    .seh_save_fplr_x 32
+    stp x19, x20, [sp, #-32]!
+    .seh_saveregp_x x19, 32
+    stp x21, x22, [sp, #16]
+    .seh_saveregp x21, 16
+    str x23, [sp, #-32]!
+    .seh_savereg_x x23, 32
+    str x24, [sp, #8]
+    .seh_savereg x24, 8
+    stp x29, x30, [sp, #48]
+    .seh_save_fplr 48
+    stp x29, x30, [sp, #-32]!
+    .seh_save_fplr_x 32
+    stp d8, d9, [sp, #-32]!
+    .seh_savefregp_x d8, 32
+    stp d10, d11, [sp, #16]
+    .seh_savefregp d10, 16
+    str d12, [sp, #-32]!
+    .seh_savefreg_x d12, 32
+    str d13, [sp, #8]
+    .seh_savefreg d13, 8
+    add x29, sp, #16
+    .seh_addfp 16
+    nop
+    .seh_nop
     .seh_endprologue
+    nop
+    .seh_startepilogue
+    add sp, sp, #24
+    .seh_stackalloc 24
+    .seh_endepilogue
+    ret
     .seh_handler __C_specific_handler, @except
     .seh_handlerdata
     .long 0
     .text
-    add sp, sp, #24
-    ret
     .seh_endproc
 
     // Function with no .seh directives; no pdata/xdata entries are
     // generated.
     .globl smallFunc
     .def smallFunc
     .scl 2
     .type 32