[WinEH][ARM64] Split unwind info for functions larger than 1MB

[zzheng]

Summary

Create function segments and emit unwind info of them.

A segment must be less than 1MB and no prolog or epilog can cross segments boundary.

Reference: https://docs.microsoft.com/en-us/cpp/build/arm64-exception-handling#function-fragments

TODOs:

1. Enable packed unwind info (.pdata only) for multi-segment functions. (currently only enabled for functions less than 1MB, that is: functions have one segment)
2. Emit packed unwind info (.pdata only) for segments that have neithor prolog nor epilog.

These two will be done in next patch (WIP).

3. Handle functions that requires more than 255 (0xFF) words to encode all prolog and epilogs unwind op or have more than 65535 (0xFFFF) epilogs.

This one needs some change as we need to move counting code words for epilog earlier in the current process. We may not see either case in practice, IMHO.

4. Shrink Wrapping where some stack ops are moved out of prolog/epilog and put in different segment. See example 2 of above link.
5. Align directive or aligning loops where we cannot reliably compute function size.

4 and 5 are kind of future work, I'm think about what to do as well as getting test cases for them.

[efriedma]

llvm/lib/MC/MCWin64EH.cpp

1099

This FIXME doesn't really make sense; in general, we can't represent segments in pdata-only form. Among other things, pdata-only form is limited to 8KB function length.

---

I'm still not really convinced this "fake epilog" form is really useful, even if the Microsoft documentation suggests using it.

1180

for (auto Inst : llvm::reverse(info->Instructions)

[zzheng]

llvm/lib/MC/MCWin64EH.cpp

1099

Let's keep it this way for now. We can remove it once we verified that MS unwinder is ok with zero epilog scope.

[zzheng]

ping.. @mstorsjo

[mstorsjo]

In D130049#3666718, @zzheng wrote:

ping.. @mstorsjo

Yep, I've noted the patch and I'll definitely have a look, but it'll take a day or two, unfortunately!

[zzheng]

In D130049#3666729, @mstorsjo wrote:

In D130049#3666718, @zzheng wrote:

ping.. @mstorsjo

Yep, I've noted the patch and I'll definitely have a look, but it'll take a day or two, unfortunately!

no problem, we're doing some internal testing on it.

[mstorsjo]

Overall this looks quite good to me, I mostly have a bunch of readability/understandability comments.

llvm/include/llvm/MC/MCWinEH.h

72

Can you add a comment about what the int64_t is here? It's not obvious to me on the first read through the code.

llvm/lib/MC/MCWin64EH.cpp

1010

Should this be >? Isn't it theoretically possible here that we have RemainingLength == SegLimit, we produce one segment and consume all of RemainingLength, and we'd end up adding a zero length segment at the end?

1016

To me, it would be clearer if we'd move the declaration of EpilogsInSegment here, so we don't need to clear it, but we're operating on a new instance of it each time. Or are there (tangible?) performance benefits to doing it like this?

1018

Can we assume that all the epilogs are monotonically ordered? (I think it can be reasonable to do that, but maybe it'd be good to have an assert somewhere that tests it?)

1025

By point 1., I presume you mean "we included all completely contained epilogs, and found another one which is entirely after the current segment" - or would you consider that a third case? It feels confusing to not have that case listed in any case (even if the code probably is correct).

1031

If you don't want to move the declaration of EpilogsInSegment to line 1012 above, it would be clearer to me to do the .clear() here, directly after the std::move, even if that does one unnecessary clean for the last iteration.

1042

Please add the `/*HasProlog*/ comment here for the third parameter.

1144–1147

Isn't SEH unwind data splitting kinda what's being implemented here? It's just that no split is being done for the condition of too many opcode bytes, only for the condition of too large functions. I.e. I'd like to rephrase the error message a bit more.

1180–1181

I'd like to have a more verbose comment here to remind myself about how prologs in !HasProlog segments work - I always end up confused about this, e.g. something like this:

// Even for a segment with !HasProlog, we do need to emit the opcodes for the prolog, for the unwinder to use for unwinding from this segment.
// The end_c opcode at the start indicates to the unwinder that the actual prolog is outside of the current segment, and the unwinder shouldn't try to check for unwinding from a partial prolog.

llvm/test/MC/AArch64/seh-large-func.s

10

Is it meaningful to include all the section headers in the testcase here? They're mostly distracting here I think, and just extra work to update when working on the testcase in my experience.

[zzheng]

Addressed @mstorsjo 's comments

Added an unit test that has multiple epilogs in one segment.

Fixed an oversight in previous version that we only call getARM64OffsetInProlog() for the first segment.

[zzheng]

llvm/lib/MC/MCWin64EH.cpp

1018

I put an assert in the loop that finds and records epilogs' start and end offset (line-990 to line-994), the idea is:
assert(epilog[n].end <= epilog[n+1].start),

llvm/test/MC/AArch64/seh-large-func.s

10

makes sense.
will just keep .pdata and .xdata sections

[zzheng]

llvm/lib/MC/MCWin64EH.cpp

1027

case 1 is when we exit the above while loop with (E == Epilogs.size()), i.e. there's no epilog left to be put into a segment.

1028

case 2 is when we exit the loop above with (E < Epilogs.size) but (Epilogs[E].End >= SegEnd).

1031

this is (E < Epilogs.size()) and (Epilogs[E].end >= SegEnd) when we exit the loop above.

[zzheng]

llvm/lib/MC/MCWin64EH.cpp

1033

this is to find out whether we're in case 2 or 3.

[zzheng]

gentle ping..

[mstorsjo]

LGTM, thanks! Nothing more from me here, if @efriedma is ok with it. A couple very minor stylistic comments remain.

llvm/lib/MC/MCWin64EH.cpp

1034

Nitpick: the indentation of the comment, between the unscoped if() and the statement it controls, is a bit weird. I'd prefer to move the comment to the end of the SegLength = .. line to keep it clearer, or add braces around it (and fix the indentation of the comment).

1145

Super-nitpick: You're missing a space after the comma, before the following text in the next string literal on the next line.

llvm/test/MC/AArch64/seh-large-func.s

24

Super nitpick: You seem to have lost a space before Section { here, making it misaligned.

[efriedma]

LGTM

[mstorsjo]

This triggers failed asserts in two cases in my builds - this one:

clang: ../lib/MC/MCWin64EH.cpp:998: void ARM64FindSegmentsInFunction(llvm::MCStreamer&, llvm::WinEH::FrameInfo*, int64_t): Assertion `(Epilogs.size() == 0 || Offset >= Epilogs.back().End) && "Epilogs should be monotonically ordered"' failed.

It’s reproducible with https://martin.st/temp/wine-corruntimehost.c and https://martin.st/temp/dav1d-thread.c with this command:
clang -target aarch64-windows -c src.c -O2

[mstorsjo]

In D130049#3704064, @mstorsjo wrote:

This triggers failed asserts in two cases in my builds - this one:

clang: ../lib/MC/MCWin64EH.cpp:998: void ARM64FindSegmentsInFunction(llvm::MCStreamer&, llvm::WinEH::FrameInfo*, int64_t): Assertion `(Epilogs.size() == 0 || Offset >= Epilogs.back().End) && "Epilogs should be monotonically ordered"' failed.

It’s reproducible with https://martin.st/temp/wine-corruntimehost.c and https://martin.st/temp/dav1d-thread.c with this command:
clang -target aarch64-windows -c src.c -O2

I ran into this issue in a number of more cases too, hitting compilation of at least 3 different projects.

I had a look at it - see D131393 for a quick tweak that avoids triggering the failed assert in at least the particular case that I looked at.

I also posted D131394, which adds a similar consistency check as for ARM, which triggers and shows the real root cause for this issue. For the dav1d-thread.c testcase, we're currently generating the following code:

$ clang -target aarch64-windows -S dav1d-thread.c -O2 -o -
[...]
dav1d_set_thread_name:
[...]
        ldp     x2, x19, [sp, #8]               // 16-byte Folded Reload
        .seh_startepilogue
        ldr     x30, [sp, #24]                  // 8-byte Folded Reload
        .seh_save_reg   x30, 24
        .seh_save_reg   x19, 16
        add     sp, sp, #32
        .seh_stackalloc 32
        .seh_endepilogue
        br      x2

There's three .seh_ opcodes in the epilogue, even if there's only two actual instructions - while the .seh_save_reg x19 has been merged into an ldp x2, x19 before the start of the epilogue.

[efriedma]

I thought we had resolved all the issues with instruction mismatches... I guess not. We need to suppress the ldp formation transform in that case.

[efriedma]

I thought we had resolved all the issues with instruction mismatches... I guess not. We need to suppress the ldp formation transform in that case.

Did we address this yet?

[mstorsjo]

In D130049#3825496, @efriedma wrote:

I thought we had resolved all the issues with instruction mismatches... I guess not. We need to suppress the ldp formation transform in that case.

Did we address this yet?

I'm not aware of any fix at least; including D131394 should expose a bunch of issues if they're still present. (If there's reason to believe things are fixed, I can try to include that patch in my next round of nightly builds.)

[efriedma]

Posted https://reviews.llvm.org/D134992

llvm/include/llvm/MC/MCWinEH.h

struct FrameInfo {
   const MCSymbol *End = nullptr;
   const MCSymbol *FuncletOrFuncEnd = nullptr;
   const MCSymbol *ExceptionHandler = nullptr;
   const MCSymbol *Function = nullptr;
   const MCSymbol *PrologEnd = nullptr;
   const MCSymbol *Symbol = nullptr;
   MCSection *TextSection = nullptr;
   uint32_t PackedInfo = 0;
+  uint32_t PrologCodeBytes = 0;
 
   bool HandlesUnwind = false;
   bool HandlesExceptions = false;
   bool EmitAttempted = false;
   bool Fragment = false;
 
   int LastFrameInst = -1;
   const FrameInfo *ChainedParent = nullptr;
   std::vector<Instruction> Instructions;
   struct Epilog {
     std::vector<Instruction> Instructions;
     unsigned Condition;
     MCSymbol *End;
   };
   MapVector<MCSymbol *, Epilog> EpilogMap;
 
+  // For splitting unwind info of large functions
+  struct Segment {
+    int64_t Offset;
+    int64_t Length;
+    bool HasProlog;
+    MCSymbol *Symbol;
+    // Map an Epilog's symbol to its offset within the function.
+    MapVector<MCSymbol *, int64_t> Epilogs;
+
+    Segment(int64_t Offset, int64_t Length, bool HasProlog = false)
+        : Offset(Offset), Length(Length), HasProlog(HasProlog) {}
+  };
+
+  std::vector<Segment> Segments;
+
   FrameInfo() = default;
   FrameInfo(const MCSymbol *Function, const MCSymbol *BeginFuncEHLabel)
       : Begin(BeginFuncEHLabel), Function(Function) {}
   FrameInfo(const MCSymbol *Function, const MCSymbol *BeginFuncEHLabel,
             const FrameInfo *ChainedParent)
       : Begin(BeginFuncEHLabel), Function(Function),
         ChainedParent(ChainedParent) {}
 

llvm/lib/MC/MCWin64EH.cpp

case Win64EH::UOP_PushMachFrame:
     EmitAbsDifference(streamer, inst.Label, begin);
     streamer.emitInt8(b2);
     break;
   }
 }
 
 static void EmitSymbolRefWithOfs(MCStreamer &streamer,
                                  const MCSymbol *Base,
+                                 int64_t Offset) {
+  MCContext &Context = streamer.getContext();
+  const MCConstantExpr *OffExpr = MCConstantExpr::create(Offset, Context);
+  const MCSymbolRefExpr *BaseRefRel = MCSymbolRefExpr::create(Base,
+                                              MCSymbolRefExpr::VK_COFF_IMGREL32,
+                                              Context);
+  streamer.emitValue(MCBinaryExpr::createAdd(BaseRefRel, OffExpr, Context), 4);
+}
+
+static void EmitSymbolRefWithOfs(MCStreamer &streamer,
+                                 const MCSymbol *Base,
                                  const MCSymbol *Other) {
   MCContext &Context = streamer.getContext();
   const MCSymbolRefExpr *BaseRef = MCSymbolRefExpr::create(Base, Context);
   const MCSymbolRefExpr *OtherRef = MCSymbolRefExpr::create(Other, Context);
   const MCExpr *Ofs = MCBinaryExpr::createSub(OtherRef, BaseRef, Context);
   const MCSymbolRefExpr *BaseRefRel = MCSymbolRefExpr::create(Base,
                                               MCSymbolRefExpr::VK_COFF_IMGREL32,
                                               Context);
getARM64OffsetInProlog(const std::vector<WinEH::Instruction> &Prolog,
 
   // Check that the epilog actually is a perfect match for the end (backwrds)
   // of the prolog.
   for (int I = Epilog.size() - 1; I >= 0; I--) {
     if (Prolog[I] != Epilog[Epilog.size() - 1 - I])
       return -1;
   }
 
-  // If the epilog was a subset of the prolog, find its offset.
   if (Epilog.size() == Prolog.size())
     return 0;
+
+  // If the epilog was a subset of the prolog, find its offset.
   return ARM64CountOfUnwindCodes(ArrayRef<WinEH::Instruction>(
       &Prolog[Epilog.size()], Prolog.size() - Epilog.size()));
 }
 
 static int checkARM64PackedEpilog(MCStreamer &streamer, WinEH::FrameInfo *info,
+                                  WinEH::FrameInfo::Segment *Seg,
                                   int PrologCodeBytes) {
   // Can only pack if there's one single epilog
-  if (info->EpilogMap.size() != 1)
+  if (Seg->Epilogs.size() != 1)
     return -1;
 
+  MCSymbol *Sym = Seg->Epilogs.begin()->first;
   const std::vector<WinEH::Instruction> &Epilog =
-      info->EpilogMap.begin()->second.Instructions;
+      info->EpilogMap[Sym].Instructions;
 
   // Check that the epilog actually is at the very end of the function,
   // otherwise it can't be packed.
-  uint32_t DistanceFromEnd = (uint32_t)GetAbsDifference(
-      streamer, info->FuncletOrFuncEnd, info->EpilogMap.begin()->first);
+  uint32_t DistanceFromEnd =
+      (uint32_t)(Seg->Offset + Seg->Length - Seg->Epilogs.begin()->second);
   if (DistanceFromEnd / 4 != Epilog.size())
     return -1;
 
   int RetVal = -1;
   // Even if we don't end up sharing opcodes with the prolog, we can still
   // write the offset as a packed offset, if the single epilog is located at
   // the end of the function and the offset (pointing after the prolog) fits
   // as a packed offset.
   if (PrologCodeBytes <= 31 &&
       PrologCodeBytes + ARM64CountOfUnwindCodes(Epilog) <= 124)
     RetVal = PrologCodeBytes;
 
   int Offset = getARM64OffsetInProlog(info->Instructions, Epilog);
   if (Offset < 0)
     return RetVal;
 
   // Check that the offset and prolog size fits in the first word; it's
   // unclear whether the epilog count in the extension word can be taken
   // as packed epilog offset.
   if (Offset > 31 || PrologCodeBytes > 124)
     return RetVal;
 
   // As we choose to express the epilog as part of the prolog, remove the
   // epilog from the map, so we don't try to emit its opcodes.
-  info->EpilogMap.clear();
+  info->EpilogMap.erase(Sym);
   return Offset;
 }
 
 static bool tryARM64PackedUnwind(WinEH::FrameInfo *info, uint32_t FuncLength,
                                  int PackedEpilogOffset) {
   if (PackedEpilogOffset == 0) {
     // Fully symmetric prolog and epilog, should be ok for packed format.
     // For CR=3, the corresponding synthesized epilog actually lacks the
static bool tryARM64PackedUnwind(WinEH::FrameInfo *info, uint32_t FuncLength,
   info->PackedInfo |= (RegF & 0x7) << 13;
   info->PackedInfo |= (RegI & 0xF) << 16;
   info->PackedInfo |= (H & 0x1) << 20;
   info->PackedInfo |= (CR & 0x3) << 21;
   info->PackedInfo |= (FrameSize & 0x1FF) << 23;
   return true;
 }
 
-// Populate the .xdata section.  The format of .xdata on ARM64 is documented at
-// https://docs.microsoft.com/en-us/cpp/build/arm64-exception-handling
-static void ARM64EmitUnwindInfo(MCStreamer &streamer, WinEH::FrameInfo *info,
-                                bool TryPacked = true) {
-  // If this UNWIND_INFO already has a symbol, it's already been emitted.
-  if (info->Symbol)
-    return;
-  // If there's no unwind info here (not even a terminating UOP_End), the
-  // unwind info is considered bogus and skipped. If this was done in
-  // response to an explicit .seh_handlerdata, the associated trailing
-  // handler data is left orphaned in the xdata section.
-  if (info->empty()) {
-    info->EmitAttempted = true;
-    return;
+static void ARM64ProcessEpilogs(WinEH::FrameInfo *info,
+                                WinEH::FrameInfo::Segment *Seg,
+                                uint32_t &TotalCodeBytes,
+                                MapVector<MCSymbol *, uint32_t> &EpilogInfo) {
+
+  std::vector<MCSymbol *> EpilogStarts;
+  for (auto &I : Seg->Epilogs)
+    EpilogStarts.push_back(I.first);
+
+  // Epilogs processed so far.
+  std::vector<MCSymbol *> AddedEpilogs;
+  for (auto S : EpilogStarts) {
+    MCSymbol *EpilogStart = S;
+    auto &EpilogInstrs = info->EpilogMap[S].Instructions;
+    uint32_t CodeBytes = ARM64CountOfUnwindCodes(EpilogInstrs);
+
+    MCSymbol* MatchingEpilog =
+      FindMatchingEpilog(EpilogInstrs, AddedEpilogs, info);
+    int PrologOffset;
+    if (MatchingEpilog) {
+      assert(EpilogInfo.find(MatchingEpilog) != EpilogInfo.end() &&
+             "Duplicate epilog not found");
+      EpilogInfo[EpilogStart] = EpilogInfo.lookup(MatchingEpilog);
+      // Clear the unwind codes in the EpilogMap, so that they don't get output
+      // in ARM64EmitUnwindInfoForSegment().
+      EpilogInstrs.clear();
+    } else if ((PrologOffset = getARM64OffsetInProlog(info->Instructions,
+                                                      EpilogInstrs)) >= 0) {
+      EpilogInfo[EpilogStart] = PrologOffset;
+      // If the segment doesn't have a prolog, an end_c will be emitted before
+      // prolog opcodes. So epilog start index in opcodes array is moved by 1.
+      if (!Seg->HasProlog)
+        EpilogInfo[EpilogStart] += 1;
+      // Clear the unwind codes in the EpilogMap, so that they don't get output
+      // in ARM64EmitUnwindInfoForSegment().
+      EpilogInstrs.clear();
+    } else {
+      EpilogInfo[EpilogStart] = TotalCodeBytes;
+      TotalCodeBytes += CodeBytes;
+      AddedEpilogs.push_back(EpilogStart);
+    }
   }
-  if (info->EmitAttempted) {
-    // If we tried to emit unwind info before (due to an explicit
-    // .seh_handlerdata directive), but skipped it (because there was no
-    // valid information to emit at the time), and it later got valid unwind
-    // opcodes, we can't emit it here, because the trailing handler data
-    // was already emitted elsewhere in the xdata section.
-    streamer.getContext().reportError(
-        SMLoc(), "Earlier .seh_handlerdata for " + info->Function->getName() +
-                     " skipped due to no unwind info at the time "
-                     "(.seh_handlerdata too early?), but the function later "
-                     "did get unwind info that can't be emitted");
-    return;
-  }
 
-  simplifyARM64Opcodes(info->Instructions, false);
-  for (auto &I : info->EpilogMap)
-    simplifyARM64Opcodes(I.second.Instructions, true);
+static void ARM64FindSegmentsInFunction(MCStreamer &streamer,
+                                        WinEH::FrameInfo *info,
+                                        int64_t RawFuncLength) {
+  struct EpilogStartEnd {
+    MCSymbol *Start;
+    int64_t Offset;
+    int64_t End;
+  };
+  // Record Start and End of each epilog.
+  SmallVector<struct EpilogStartEnd, 4> Epilogs;
+  for (auto &I : info->EpilogMap) {
+    MCSymbol *Start = I.first;
+    auto &Instrs = I.second.Instructions;
+    int64_t Offset = GetAbsDifference(streamer, Start, info->Begin);
+    assert((Epilogs.size() == 0 || Offset >= Epilogs.back().End) &&
+           "Epilogs should be monotonically ordered");
+    Epilogs.push_back({Start, Offset, Offset + (int64_t)Instrs.size() * 4});
+  }
+
+  unsigned E = 0;
+  int64_t SegLimit = 0xFFFFC;
+  int64_t SegOffset = 0;
+
+  if (RawFuncLength > SegLimit) {
+
+    int64_t RemainingLength = RawFuncLength;
+
+    while (RemainingLength > SegLimit) {
+      // Try divide the function into segments, requirements:
+      // 1. Segment length <= 0xFFFFC;
+      // 2. Each Prologue or Epilogue must be fully within a segment.
+      int64_t SegLength = SegLimit;
+      int64_t SegEnd = SegOffset + SegLength;
+      // Keep record on symbols and offsets of epilogs in this segment.
+      MapVector<MCSymbol *, int64_t> EpilogsInSegment;
+
+      while (E < Epilogs.size() && Epilogs[E].End < SegEnd) {
+        // Epilogs within current segment.
+        EpilogsInSegment[Epilogs[E].Start] = Epilogs[E].Offset;
+        ++E;
+      }
+
+      // At this point, we have:
+      // 1. Put all epilogs in segments already. No action needed here; or
+      // 2. Found an epilog that will cross segments boundry. We need to
+      //    move back current segment's end boundry, so the epilog is entirely
+      //    in the next segment; or
+      // 3. Left at least one epilog that is entirely after this segment.
+      //    It'll be handled by the next iteration, or the last segment.
+      if (E < Epilogs.size() && Epilogs[E].Offset <= SegEnd)
+        // Move back current Segment's end boundry.
+        SegLength = Epilogs[E].Offset - SegOffset;
+
+      auto Seg = WinEH::FrameInfo::Segment(
+          SegOffset, SegLength, /* HasProlog */!SegOffset);
+      Seg.Epilogs = std::move(EpilogsInSegment);
+      info->Segments.push_back(Seg);
+
+      SegOffset += SegLength;
+      RemainingLength -= SegLength;
+    }
+  }
+
+  // Add the last segment when RawFuncLength > 0xFFFFC,
+  // or the only segment otherwise.
+  auto LastSeg =
+      WinEH::FrameInfo::Segment(SegOffset, RawFuncLength - SegOffset,
+                                /* HasProlog */!SegOffset);
+  for (; E < Epilogs.size(); ++E)
+    LastSeg.Epilogs[Epilogs[E].Start] = Epilogs[E].Offset;
+  info->Segments.push_back(LastSeg);
+}
 
+static void ARM64EmitUnwindInfoForSegment(MCStreamer &streamer,
+                                          WinEH::FrameInfo *info,
+                                          WinEH::FrameInfo::Segment &Seg,
+                                          bool TryPacked = true) {
   MCContext &context = streamer.getContext();
   MCSymbol *Label = context.createTempSymbol();
 
   streamer.emitValueToAlignment(4);
   streamer.emitLabel(Label);
+  Seg.Symbol = Label;
+  // Use the 1st segemnt's label as function's.
+  if (Seg.Offset == 0)
     info->Symbol = Label;
 
-  int64_t RawFuncLength;
-  if (!info->FuncletOrFuncEnd) {
-    report_fatal_error("FuncletOrFuncEnd not set");
-  } else {
-    // FIXME: GetAbsDifference tries to compute the length of the function
-    // immediately, before the whole file is emitted, but in general
-    // that's impossible: the size in bytes of certain assembler directives
-    // like .align and .fill is not known until the whole file is parsed and
-    // relaxations are applied. Currently, GetAbsDifference fails with a fatal
-    // error in that case. (We mostly don't hit this because inline assembly
-    // specifying those directives is rare, and we don't normally try to
-    // align loops on AArch64.)
-    //
-    // There are two potential approaches to delaying the computation. One,
-    // we could emit something like ".word (endfunc-beginfunc)/4+0x10800000",
-    // as long as we have some conservative estimate we could use to prove
-    // that we don't need to split the unwind data. Emitting the constant
-    // is straightforward, but there's no existing code for estimating the
-    // size of the function.
-    //
-    // The other approach would be to use a dedicated, relaxable fragment,
-    // which could grow to accommodate splitting the unwind data if
-    // necessary. This is more straightforward, since it automatically works
-    // without any new infrastructure, and it's consistent with how we handle
-    // relaxation in other contexts.  But it would require some refactoring
-    // to move parts of the pdata/xdata emission into the implementation of
-    // a fragment. We could probably continue to encode the unwind codes
-    // here, but we'd have to emit the pdata, the xdata header, and the
-    // epilogue scopes later, since they depend on whether the we need to
-    // split the unwind data.
-    RawFuncLength = GetAbsDifference(streamer, info->FuncletOrFuncEnd,
-                                     info->Begin);
-  }
-  if (RawFuncLength > 0xFFFFF)
-    report_fatal_error("SEH unwind data splitting not yet implemented");
-  uint32_t FuncLength = (uint32_t)RawFuncLength / 4;
-  uint32_t PrologCodeBytes = ARM64CountOfUnwindCodes(info->Instructions);
-  uint32_t TotalCodeBytes = PrologCodeBytes;
+  bool HasProlog = Seg.HasProlog;
+  bool HasEpilogs = (Seg.Epilogs.size() != 0);
 
-  int PackedEpilogOffset =
-      checkARM64PackedEpilog(streamer, info, PrologCodeBytes);
+  uint32_t SegLength = (uint32_t)Seg.Length / 4;
+  uint32_t PrologCodeBytes = info->PrologCodeBytes;
 
-  if (PackedEpilogOffset >= 0 &&
+  int PackedEpilogOffset = HasEpilogs ?
+      checkARM64PackedEpilog(streamer, info, &Seg, PrologCodeBytes) : -1;
+
+  // TODO:
+  // 1. Enable packed unwind info (.pdata only) for multi-segment functions.
+  // 2. Emit packed unwind info (.pdata only) for segments that have neithor
+  //    prolog nor epilog.
+  if (info->Segments.size() == 1 && PackedEpilogOffset >= 0 &&
       uint32_t(PackedEpilogOffset) < PrologCodeBytes &&
-      !info->HandlesExceptions && FuncLength <= 0x7ff && TryPacked) {
+      !info->HandlesExceptions && SegLength <= 0x7ff && TryPacked) {
     // Matching prolog/epilog and no exception handlers; check if the
     // prolog matches the patterns that can be described by the packed
     // format.
 
     // info->Symbol was already set even if we didn't actually write any
     // unwind info there. Keep using that as indicator that this unwind
     // info has been generated already.
-
-    if (tryARM64PackedUnwind(info, FuncLength, PackedEpilogOffset))
+    if (tryARM64PackedUnwind(info, SegLength, PackedEpilogOffset))
       return;
   }
 
-  // Process epilogs.
-  MapVector<MCSymbol *, uint32_t> EpilogInfo;
-  // Epilogs processed so far.
-  std::vector<MCSymbol *> AddedEpilogs;
+  // If the prolog is not in this segment, we need to emit an end_c, which takes
+  // 1 byte, before prolog unwind ops.
+  if (!HasProlog) {
+    PrologCodeBytes += 1;
+    if (PackedEpilogOffset >= 0)
+      PackedEpilogOffset += 1;
+    // If a segment has neither prolog nor epilog, "With full .xdata record,
+    // Epilog Count = 1. Epilog Start Index points to end_c."
+    // https://docs.microsoft.com/en-us/cpp/build/arm64-exception-handling#function-fragments
+    // TODO: We can remove this if testing shows zero epilog scope is ok with
+    //       MS unwinder.
+    if (!HasEpilogs)
+      // Pack the fake epilog into phantom prolog.
+      PackedEpilogOffset = 0;
+  }
 
-  for (auto &I : info->EpilogMap) {
-    MCSymbol *EpilogStart = I.first;
-    auto &EpilogInstrs = I.second.Instructions;
-    uint32_t CodeBytes = ARM64CountOfUnwindCodes(EpilogInstrs);
+  uint32_t TotalCodeBytes = PrologCodeBytes;
 
-    MCSymbol* MatchingEpilog =
-      FindMatchingEpilog(EpilogInstrs, AddedEpilogs, info);
-    int PrologOffset;
-    if (MatchingEpilog) {
-      assert(EpilogInfo.find(MatchingEpilog) != EpilogInfo.end() &&
-             "Duplicate epilog not found");
-      EpilogInfo[EpilogStart] = EpilogInfo.lookup(MatchingEpilog);
-      // Clear the unwind codes in the EpilogMap, so that they don't get output
-      // in the logic below.
-      EpilogInstrs.clear();
-    } else if ((PrologOffset = getARM64OffsetInProlog(info->Instructions,
-                                                      EpilogInstrs)) >= 0) {
-      EpilogInfo[EpilogStart] = PrologOffset;
-      // Clear the unwind codes in the EpilogMap, so that they don't get output
-      // in the logic below.
-      EpilogInstrs.clear();
-    } else {
-      EpilogInfo[EpilogStart] = TotalCodeBytes;
-      TotalCodeBytes += CodeBytes;
-      AddedEpilogs.push_back(EpilogStart);
-    }
-  }
+  // Process epilogs.
+  MapVector<MCSymbol *, uint32_t> EpilogInfo;
+  ARM64ProcessEpilogs(info, &Seg, TotalCodeBytes, EpilogInfo);
 
   // Code Words, Epilog count, E, X, Vers, Function Length
   uint32_t row1 = 0x0;
   uint32_t CodeWords = TotalCodeBytes / 4;
   uint32_t CodeWordsMod = TotalCodeBytes % 4;
   if (CodeWordsMod)
     CodeWords++;
   uint32_t EpilogCount =
-      PackedEpilogOffset >= 0 ? PackedEpilogOffset : info->EpilogMap.size();
+      PackedEpilogOffset >= 0 ? PackedEpilogOffset : Seg.Epilogs.size();
   bool ExtensionWord = EpilogCount > 31 || TotalCodeBytes > 124;
   if (!ExtensionWord) {
     row1 |= (EpilogCount & 0x1F) << 22;
     row1 |= (CodeWords & 0x1F) << 27;
   }
   if (info->HandlesExceptions) // X
     row1 |= 1 << 20;
   if (PackedEpilogOffset >= 0) // E
     row1 |= 1 << 21;
-  row1 |= FuncLength & 0x3FFFF;
+  row1 |= SegLength & 0x3FFFF;
   streamer.emitInt32(row1);
 
   // Extended Code Words, Extended Epilog Count
   if (ExtensionWord) {
     // FIXME: We should be able to split unwind info into multiple sections.
     if (CodeWords > 0xFF || EpilogCount > 0xFFFF)
-      report_fatal_error("SEH unwind data splitting not yet implemented");
+      report_fatal_error(
+          "SEH unwind data splitting is only implemnted for large functions, "
+          "cases of too many code words or too many epilogs will be done later"
+      );
     uint32_t row2 = 0x0;
     row2 |= (CodeWords & 0xFF) << 16;
     row2 |= (EpilogCount & 0xFFFF);
     streamer.emitInt32(row2);
   }
 
   if (PackedEpilogOffset < 0) {
     // Epilog Start Index, Epilog Start Offset
     for (auto &I : EpilogInfo) {
       MCSymbol *EpilogStart = I.first;
       uint32_t EpilogIndex = I.second;
-      uint32_t EpilogOffset =
-          (uint32_t)GetAbsDifference(streamer, EpilogStart, info->Begin);
+      // Epilog offset within the Segment.
+      uint32_t EpilogOffset = (uint32_t)(Seg.Epilogs[EpilogStart] - Seg.Offset);
       if (EpilogOffset)
         EpilogOffset /= 4;
       uint32_t row3 = EpilogOffset;
       row3 |= (EpilogIndex & 0x3FF) << 22;
       streamer.emitInt32(row3);
     }
   }
 
+  // Note that even for segments that have no prolog, we still need to emit
+  // prolog unwinding opcodes so that the unwinder knows how to unwind from
+  // such a segment.
+  // The end_c opcode at the start indicates to the unwinder that the actual
+  // prolog is outside of the current segment, and the unwinder shouldn't try
+  // to check for unwinding from a partial prolog.
+  if (!HasProlog)
+    // Emit an end_c.
+    streamer.emitInt8((uint8_t)0xE5);
+
   // Emit prolog unwind instructions (in reverse order).
-  uint8_t numInst = info->Instructions.size();
-  for (uint8_t c = 0; c < numInst; ++c) {
-    WinEH::Instruction inst = info->Instructions.back();
-    info->Instructions.pop_back();
-    ARM64EmitUnwindCode(streamer, inst);
-  }
+  for (auto Inst : llvm::reverse(info->Instructions))
+    ARM64EmitUnwindCode(streamer, Inst);
 
   // Emit epilog unwind instructions
-  for (auto &I : info->EpilogMap) {
-    auto &EpilogInstrs = I.second.Instructions;
+  for (auto &I : Seg.Epilogs) {
+    auto &EpilogInstrs = info->EpilogMap[I.first].Instructions;
     for (const WinEH::Instruction &inst : EpilogInstrs)
       ARM64EmitUnwindCode(streamer, inst);
   }
 
   int32_t BytesMod = CodeWords * 4 - TotalCodeBytes;
   assert(BytesMod >= 0);
   for (int i = 0; i < BytesMod; i++)
     streamer.emitInt8(0xE3);
 
   if (info->HandlesExceptions)
     streamer.emitValue(
         MCSymbolRefExpr::create(info->ExceptionHandler,
                                 MCSymbolRefExpr::VK_COFF_IMGREL32, context),
         4);
 }
 
+// Populate the .xdata section.  The format of .xdata on ARM64 is documented at
+// https://docs.microsoft.com/en-us/cpp/build/arm64-exception-handling
+static void ARM64EmitUnwindInfo(MCStreamer &streamer, WinEH::FrameInfo *info,
+                                bool TryPacked = true) {
+  // If this UNWIND_INFO already has a symbol, it's already been emitted.
+  if (info->Symbol)
+    return;
+  // If there's no unwind info here (not even a terminating UOP_End), the
+  // unwind info is considered bogus and skipped. If this was done in
+  // response to an explicit .seh_handlerdata, the associated trailing
+  // handler data is left orphaned in the xdata section.
+  if (info->empty()) {
+    info->EmitAttempted = true;
+    return;
+  }
+  if (info->EmitAttempted) {
+    // If we tried to emit unwind info before (due to an explicit
+    // .seh_handlerdata directive), but skipped it (because there was no
+    // valid information to emit at the time), and it later got valid unwind
+    // opcodes, we can't emit it here, because the trailing handler data
+    // was already emitted elsewhere in the xdata section.
+    streamer.getContext().reportError(
+        SMLoc(), "Earlier .seh_handlerdata for " + info->Function->getName() +
+                     " skipped due to no unwind info at the time "
+                     "(.seh_handlerdata too early?), but the function later "
+                     "did get unwind info that can't be emitted");
+    return;
+  }
+
+  simplifyARM64Opcodes(info->Instructions, false);
+  for (auto &I : info->EpilogMap)
+    simplifyARM64Opcodes(I.second.Instructions, true);
+
+  int64_t RawFuncLength;
+  if (!info->FuncletOrFuncEnd) {
+    report_fatal_error("FuncletOrFuncEnd not set");
+  } else {
+    // FIXME: GetAbsDifference tries to compute the length of the function
+    // immediately, before the whole file is emitted, but in general
+    // that's impossible: the size in bytes of certain assembler directives
+    // like .align and .fill is not known until the whole file is parsed and
+    // relaxations are applied. Currently, GetAbsDifference fails with a fatal
+    // error in that case. (We mostly don't hit this because inline assembly
+    // specifying those directives is rare, and we don't normally try to
+    // align loops on AArch64.)
+    //
+    // There are two potential approaches to delaying the computation. One,
+    // we could emit something like ".word (endfunc-beginfunc)/4+0x10800000",
+    // as long as we have some conservative estimate we could use to prove
+    // that we don't need to split the unwind data. Emitting the constant
+    // is straightforward, but there's no existing code for estimating the
+    // size of the function.
+    //
+    // The other approach would be to use a dedicated, relaxable fragment,
+    // which could grow to accommodate splitting the unwind data if
+    // necessary. This is more straightforward, since it automatically works
+    // without any new infrastructure, and it's consistent with how we handle
+    // relaxation in other contexts.  But it would require some refactoring
+    // to move parts of the pdata/xdata emission into the implementation of
+    // a fragment. We could probably continue to encode the unwind codes
+    // here, but we'd have to emit the pdata, the xdata header, and the
+    // epilogue scopes later, since they depend on whether the we need to
+    // split the unwind data.
+    RawFuncLength = GetAbsDifference(streamer, info->FuncletOrFuncEnd,
+                                     info->Begin);
+  }
+
+  ARM64FindSegmentsInFunction(streamer, info, RawFuncLength);
+
+  info->PrologCodeBytes = ARM64CountOfUnwindCodes(info->Instructions);
+  for (auto &S : info->Segments)
+    ARM64EmitUnwindInfoForSegment(streamer, info, S, TryPacked);
+
+  // Clear prolog instructions after unwind info is emitted for all segments.
+  info->Instructions.clear();
+}
+
 static uint32_t ARMCountOfUnwindCodes(ArrayRef<WinEH::Instruction> Insns) {
   uint32_t Count = 0;
   for (const auto &I : Insns) {
     switch (static_cast<Win64EH::UnwindOpcodes>(I.Operation)) {
     default:
       llvm_unreachable("Unsupported ARM unwind code");
     case Win64EH::UOP_AllocSmall:
       Count += 1;
static void ARMEmitUnwindInfo(MCStreamer &streamer, WinEH::FrameInfo *info,
 
   if (info->HandlesExceptions)
     streamer.emitValue(
         MCSymbolRefExpr::create(info->ExceptionHandler,
                                 MCSymbolRefExpr::VK_COFF_IMGREL32, context),
         4);
 }
 
+static void ARM64EmitRuntimeFunction(MCStreamer &streamer,
+                                     const WinEH::FrameInfo *info) {
+  MCContext &context = streamer.getContext();
+
+  streamer.emitValueToAlignment(4);
+  for (auto &S : info->Segments) {
+    EmitSymbolRefWithOfs(streamer, info->Begin, S.Offset);
+    if (info->PackedInfo)
+      streamer.emitInt32(info->PackedInfo);
+    else
+      streamer.emitValue(
+          MCSymbolRefExpr::create(S.Symbol, MCSymbolRefExpr::VK_COFF_IMGREL32,
+                                  context),
+          4);
+  }
+}
+
+
 static void ARMEmitRuntimeFunction(MCStreamer &streamer,
                                    const WinEH::FrameInfo *info) {
   MCContext &context = streamer.getContext();
 
   streamer.emitValueToAlignment(4);
   EmitSymbolRefWithOfs(streamer, info->Begin, info->Begin);
   if (info->PackedInfo)
     streamer.emitInt32(info->PackedInfo);
for (const auto &CFI : Streamer.getWinFrameInfos()) {
     WinEH::FrameInfo *Info = CFI.get();
     // ARM64EmitUnwindInfo above clears the info struct, so we can't check
     // empty here. But if a Symbol is set, we should create the corresponding
     // pdata entry.
     if (!Info->Symbol)
       continue;
     MCSection *PData = Streamer.getAssociatedPDataSection(CFI->TextSection);
     Streamer.switchSection(PData);
-    ARMEmitRuntimeFunction(Streamer, Info);
+    ARM64EmitRuntimeFunction(Streamer, Info);
   }
 }
 
 void llvm::Win64EH::ARM64UnwindEmitter::EmitUnwindInfo(MCStreamer &Streamer,
                                                        WinEH::FrameInfo *info,
                                                        bool HandlerData) const {
   // Called if there's an .seh_handlerdata directive before the end of the
   // function. This forces writing the xdata record already here - and

llvm/test/MC/AArch64/seh-large-func-multi-epilog.s

+// This test checks that we emit unwind info correctly for epilogs that:
+// 1. mirror the prolog; or
+// 2. are subsequence at the end of the prolog; or
+// 3. neither of above two.
+// in the same segment. 
+// RUN: llvm-mc -triple aarch64-pc-win32 -filetype=obj %s -o %t.o
+// RUN: llvm-readobj -S -r -u %t.o | FileCheck %s
+
+// CHECK:       Section {
+// CHECK:         Number: 4
+// CHECK-NEXT:    Name: .xdata (2E 78 64 61 74 61 00 00)
+// CHECK-NEXT:    VirtualSize: 0x0
+// CHECK-NEXT:    VirtualAddress: 0x0
+// CHECK-NEXT:    RawDataSize: 80
+// CHECK-NEXT:    PointerToRawData: 0x1251AC
+// CHECK-NEXT:    PointerToRelocations: 0x0
+// CHECK-NEXT:    PointerToLineNumbers: 0x0
+// CHECK-NEXT:    RelocationCount: 0
+// CHECK-NEXT:    LineNumberCount: 0
+// CHECK-NEXT:    Characteristics [ (0x40300040)
+// CHECK-NEXT:      IMAGE_SCN_ALIGN_4BYTES (0x300000)
+// CHECK-NEXT:      IMAGE_SCN_CNT_INITIALIZED_DATA (0x40)
+// CHECK-NEXT:      IMAGE_SCN_MEM_READ (0x40000000)
+// CHECK-NEXT:    ]
+// CHECK-NEXT:  }
+// CHECK-NEXT:  Section {
+// CHECK-NEXT:    Number: 5
+// CHECK-NEXT:    Name: .pdata (2E 70 64 61 74 61 00 00)
+// CHECK-NEXT:    VirtualSize: 0x0
+// CHECK-NEXT:    VirtualAddress: 0x0
+// CHECK-NEXT:    RawDataSize: 16
+// CHECK-NEXT:    PointerToRawData: 0x1251FC
+// CHECK-NEXT:    PointerToRelocations: 0x12520C
+// CHECK-NEXT:    PointerToLineNumbers: 0x0
+// CHECK-NEXT:    RelocationCount: 4
+// CHECK-NEXT:    LineNumberCount: 0
+// CHECK-NEXT:    Characteristics [ (0x40300040)
+// CHECK-NEXT:      IMAGE_SCN_ALIGN_4BYTES (0x300000)
+// CHECK-NEXT:      IMAGE_SCN_CNT_INITIALIZED_DATA (0x40)
+// CHECK-NEXT:      IMAGE_SCN_MEM_READ (0x40000000)
+// CHECK-NEXT:    ]
+// CHECK-NEXT:  }
+// CHECK-NEXT:]
+// CHECK-LABEL:Relocations [
+// CHECK-NEXT:  Section (1) .text {
+// CHECK-NEXT:    0x94 IMAGE_REL_ARM64_BRANCH26 foo (12)
+// CHECK-NEXT:    0x125068 IMAGE_REL_ARM64_BRANCH26 foo (12)
+// CHECK-NEXT:  }
+// CHECK-NEXT:  Section (5) .pdata {
+// CHECK-NEXT:    0x0 IMAGE_REL_ARM64_ADDR32NB .text (0)
+// CHECK-NEXT:    0x4 IMAGE_REL_ARM64_ADDR32NB .xdata (7)
+// CHECK-NEXT:    0x8 IMAGE_REL_ARM64_ADDR32NB .text (0)
+// CHECK-NEXT:    0xC IMAGE_REL_ARM64_ADDR32NB .xdata (7)
+// CHECK-NEXT:  }
+// CHECK-NEXT:]
+// CHECK-LABEL:UnwindInformation [
+// CHECK-NEXT:  RuntimeFunction {
+// CHECK-NEXT:    Function: multi_epilog (0x0)
+// CHECK-NEXT:    ExceptionRecord: .xdata (0x0)
+// CHECK-NEXT:    ExceptionData {
+// CHECK-NEXT:      FunctionLength: 1048572
+// CHECK-NEXT:      Version: 0
+// CHECK-NEXT:      ExceptionData: No
+// CHECK-NEXT:      EpiloguePacked: No
+// CHECK-NEXT:      EpilogueScopes: 3
+// CHECK-NEXT:      ByteCodeLength: 24
+// CHECK-NEXT:      Prologue [
+// CHECK-NEXT:        0xe1                ; mov fp, sp
+// CHECK-NEXT:        0xca16              ; stp x27, x28, [sp, #176]
+// CHECK-NEXT:        0xc998              ; stp x25, x26, [sp, #192]
+// CHECK-NEXT:        0xc91a              ; stp x23, x24, [sp, #208]
+// CHECK-NEXT:        0xc89c              ; stp x21, x22, [sp, #224]
+// CHECK-NEXT:        0xc81e              ; stp x19, x20, [sp, #240]
+// CHECK-NEXT:        0x9f                ; stp x29, x30, [sp, #-256]!
+// CHECK-NEXT:        0xe4                ; end
+// CHECK-NEXT:      ]
+// CHECK-NEXT:      EpilogueScopes [
+// CHECK-NEXT:        EpilogueScope {
+// CHECK-NEXT:          StartOffset: 38
+// CHECK-NEXT:          EpilogueStartIndex: 0
+// CHECK-NEXT:          Opcodes [
+// CHECK-NEXT:            0xe1                ; mov sp, fp
+// CHECK-NEXT:            0xca16              ; ldp x27, x28, [sp, #176]
+// CHECK-NEXT:            0xc998              ; ldp x25, x26, [sp, #192]
+// CHECK-NEXT:            0xc91a              ; ldp x23, x24, [sp, #208]
+// CHECK-NEXT:            0xc89c              ; ldp x21, x22, [sp, #224]
+// CHECK-NEXT:            0xc81e              ; ldp x19, x20, [sp, #240]
+// CHECK-NEXT:            0x9f                ; ldp x29, x30, [sp], #256
+// CHECK-NEXT:            0xe4                ; end
+// CHECK-NEXT:          ]
+// CHECK-NEXT:        }
+// CHECK-NEXT:        EpilogueScope {
+// CHECK-NEXT:          StartOffset: 46
+// CHECK-NEXT:          EpilogueStartIndex: 3
+// CHECK-NEXT:          Opcodes [
+// CHECK-NEXT:            0xc998              ; ldp x25, x26, [sp, #192]
+// CHECK-NEXT:            0xc91a              ; ldp x23, x24, [sp, #208]
+// CHECK-NEXT:            0xc89c              ; ldp x21, x22, [sp, #224]
+// CHECK-NEXT:            0xc81e              ; ldp x19, x20, [sp, #240]
+// CHECK-NEXT:            0x9f                ; ldp x29, x30, [sp], #256
+// CHECK-NEXT:            0xe4                ; end
+// CHECK-NEXT:          ]
+// CHECK-NEXT:        }
+// CHECK-NEXT:        EpilogueScope {
+// CHECK-NEXT:          StartOffset: 52
+// CHECK-NEXT:          EpilogueStartIndex: 13
+// CHECK-NEXT:          Opcodes [
+// CHECK-NEXT:            0xe1                ; mov sp, fp
+// CHECK-NEXT:            0xc91a              ; ldp x23, x24, [sp, #208]
+// CHECK-NEXT:            0xc89c              ; ldp x21, x22, [sp, #224]
+// CHECK-NEXT:            0xc81e              ; ldp x19, x20, [sp, #240]
+// CHECK-NEXT:            0x9f                ; ldp x29, x30, [sp], #256
+// CHECK-NEXT:            0xe4                ; end
+// CHECK-NEXT:          ]
+// CHECK-NEXT:        }
+// CHECK-NEXT:      ]
+// CHECK-NEXT:    }
+// CHECK-NEXT:  }
+// CHECK-NEXT:  RuntimeFunction {
+// CHECK-NEXT:    Function: multi_epilog +0xFFFFC (0xFFFFC)
+// CHECK-NEXT:    ExceptionRecord: .xdata +0x28 (0x28)
+// CHECK-NEXT:    ExceptionData {
+// CHECK-NEXT:      FunctionLength: 151744
+// CHECK-NEXT:      Version: 0
+// CHECK-NEXT:      ExceptionData: No
+// CHECK-NEXT:      EpiloguePacked: No
+// CHECK-NEXT:      EpilogueScopes: 3
+// CHECK-NEXT:      ByteCodeLength: 24
+// CHECK-NEXT:      Prologue [
+// CHECK-NEXT:        0xe5                ; end_c
+// CHECK-NEXT:        0xe1                ; mov fp, sp
+// CHECK-NEXT:        0xca16              ; stp x27, x28, [sp, #176]
+// CHECK-NEXT:        0xc998              ; stp x25, x26, [sp, #192]
+// CHECK-NEXT:        0xc91a              ; stp x23, x24, [sp, #208]
+// CHECK-NEXT:        0xc89c              ; stp x21, x22, [sp, #224]
+// CHECK-NEXT:        0xc81e              ; stp x19, x20, [sp, #240]
+// CHECK-NEXT:        0x9f                ; stp x29, x30, [sp, #-256]!
+// CHECK-NEXT:        0xe4                ; end
+// CHECK-NEXT:      ]
+// CHECK-NEXT:      EpilogueScopes [
+// CHECK-NEXT:        EpilogueScope {
+// CHECK-NEXT:          StartOffset: 37916
+// CHECK-NEXT:          EpilogueStartIndex: 1
+// CHECK-NEXT:          Opcodes [
+// CHECK-NEXT:            0xe1                ; mov sp, fp
+// CHECK-NEXT:            0xca16              ; ldp x27, x28, [sp, #176]
+// CHECK-NEXT:            0xc998              ; ldp x25, x26, [sp, #192]
+// CHECK-NEXT:            0xc91a              ; ldp x23, x24, [sp, #208]
+// CHECK-NEXT:            0xc89c              ; ldp x21, x22, [sp, #224]
+// CHECK-NEXT:            0xc81e              ; ldp x19, x20, [sp, #240]
+// CHECK-NEXT:            0x9f                ; ldp x29, x30, [sp], #256
+// CHECK-NEXT:            0xe4                ; end
+// CHECK-NEXT:          ]
+// CHECK-NEXT:        }
+// CHECK-NEXT:        EpilogueScope {
+// CHECK-NEXT:          StartOffset: 37924
+// CHECK-NEXT:          EpilogueStartIndex: 4
+// CHECK-NEXT:          Opcodes [
+// CHECK-NEXT:            0xc998              ; ldp x25, x26, [sp, #192]
+// CHECK-NEXT:            0xc91a              ; ldp x23, x24, [sp, #208]
+// CHECK-NEXT:            0xc89c              ; ldp x21, x22, [sp, #224]
+// CHECK-NEXT:            0xc81e              ; ldp x19, x20, [sp, #240]
+// CHECK-NEXT:            0x9f                ; ldp x29, x30, [sp], #256
+// CHECK-NEXT:            0xe4                ; end
+// CHECK-NEXT:          ]
+// CHECK-NEXT:        }
+// CHECK-NEXT:        EpilogueScope {
+// CHECK-NEXT:          StartOffset: 37930
+// CHECK-NEXT:          EpilogueStartIndex: 14
+// CHECK-NEXT:          Opcodes [
+// CHECK-NEXT:            0xe1                ; mov sp, fp
+// CHECK-NEXT:            0xc91a              ; ldp x23, x24, [sp, #208]
+// CHECK-NEXT:            0xc89c              ; ldp x21, x22, [sp, #224]
+// CHECK-NEXT:            0xc81e              ; ldp x19, x20, [sp, #240]
+// CHECK-NEXT:            0x9f                ; ldp x29, x30, [sp], #256
+// CHECK-NEXT:            0xe4                ; end
+// CHECK-NEXT:          ]
+// CHECK-NEXT:        }
+// CHECK-NEXT:      ]
+// CHECK-NEXT:    }
+// CHECK-NEXT:  }
+// CHECK-NEXT:]
+
+  .text
+  .global multi_epilog
+  .p2align  2
+  .seh_proc multi_epilog
+multi_epilog:
+  stp  x29, lr, [sp, #-256]!
+  .seh_save_fplr_x 256
+  stp  x19, x20, [sp, #240]
+  .seh_save_regp x19, 240
+  stp  x21, x22, [sp, #224]
+  .seh_save_regp x21, 224
+  stp  x23, x24, [sp, #208]
+  .seh_save_regp x23, 208
+  stp  x25, x26, [sp, #192]
+  .seh_save_regp x25, 192
+  stp  x27, x28, [sp, #176]
+  .seh_save_regp x27, 176
+  mov x29, fp
+  .seh_set_fp
+  .seh_endprologue
+        .rept 30
+        nop
+        .endr
+  bl  foo
+// Epilogs 1, 2 and 3 are in the same segment as prolog.
+// epilog1 - mirroring prolog
+  .seh_startepilogue
+  mov sp, x29
+  .seh_set_fp
+  stp  x27, x28, [sp, #176]
+  .seh_save_regp x27, 176
+  stp  x25, x26, [sp, #192]
+  .seh_save_regp x25, 192
+  stp  x23, x24, [sp, #208]
+  .seh_save_regp x23, 208
+  stp  x21, x22, [sp, #224]
+  .seh_save_regp x21, 224
+  ldp  x19, x20, [sp, #240]
+  .seh_save_regp x19, 240
+  ldp x29, lr, [sp], #256
+  .seh_save_fplr_x 256
+  .seh_endepilogue
+  ret
+// epilog2 - a subsequence at the end of prolog, can use prolog's opcodes.
+  .seh_startepilogue
+  stp  x25, x26, [sp, #192]
+  .seh_save_regp x25, 192
+  stp  x23, x24, [sp, #208]
+  .seh_save_regp x23, 208
+  stp  x21, x22, [sp, #224]
+  .seh_save_regp x21, 224
+  ldp  x19, x20, [sp, #240]
+  .seh_save_regp x19, 240
+  ldp x29, lr, [sp], #256
+  .seh_save_fplr_x 256
+  .seh_endepilogue
+  ret
+// epilog3 - cannot use prolog's opcode.
+  .seh_startepilogue
+  mov sp, x29
+  .seh_set_fp
+  stp  x23, x24, [sp, #208]
+  .seh_save_regp x23, 208
+  stp  x21, x22, [sp, #224]
+  .seh_save_regp x21, 224
+  ldp  x19, x20, [sp, #240]
+  .seh_save_regp x19, 240
+  ldp x29, lr, [sp], #256
+  .seh_save_fplr_x 256
+  .seh_endepilogue
+  ret
+        .rept 300000
+        nop
+        .endr
+  bl  foo
+// Epilogs below are in a segment without prolog
+// epilog4 - mirroring prolog, its start index should be 1, counting the end_c. 
+  .seh_startepilogue
+  mov sp, x29
+  .seh_set_fp
+  stp  x27, x28, [sp, #176]
+  .seh_save_regp x27, 176
+  stp  x25, x26, [sp, #192]
+  .seh_save_regp x25, 192
+  stp  x23, x24, [sp, #208]
+  .seh_save_regp x23, 208
+  stp  x21, x22, [sp, #224]
+  .seh_save_regp x21, 224
+  ldp  x19, x20, [sp, #240]
+  .seh_save_regp x19, 240
+  ldp x29, lr, [sp], #256
+  .seh_save_fplr_x 256
+  .seh_endepilogue
+  ret
+// epilog5 - same as epilog2, its start index should be: 1 + epilog2's index.
+  .seh_startepilogue
+  stp  x25, x26, [sp, #192]
+  .seh_save_regp x25, 192
+  stp  x23, x24, [sp, #208]
+  .seh_save_regp x23, 208
+  stp  x21, x22, [sp, #224]
+  .seh_save_regp x21, 224
+  ldp  x19, x20, [sp, #240]
+  .seh_save_regp x19, 240
+  ldp x29, lr, [sp], #256
+  .seh_save_fplr_x 256
+  .seh_endepilogue
+  ret
+// epilog6 - same as epilog3, cannot use prolog's opcode. Again its start index
+//           should be: 1 + epilog3's index.
+  .seh_startepilogue
+  mov sp, x29
+  .seh_set_fp
+  stp  x23, x24, [sp, #208]
+  .seh_save_regp x23, 208
+  stp  x21, x22, [sp, #224]
+  .seh_save_regp x21, 224
+  ldp  x19, x20, [sp, #240]
+  .seh_save_regp x19, 240
+  ldp x29, lr, [sp], #256
+  .seh_save_fplr_x 256
+  .seh_endepilogue
+  ret
+  .seh_endfunclet
+  .seh_endproc

llvm/test/MC/AArch64/seh-large-func.s

+// This test checks that we emit unwind info correctly for functions
+// larger than 1MB.
+// RUN: llvm-mc -triple aarch64-pc-win32 -filetype=obj %s -o %t.o
+// RUN: llvm-readobj -S -r -u %t.o | FileCheck %s
+
+// CHECK:        Section {
+// CHECK:          Number: 4
+// CHECK-NEXT:     Name: .xdata (2E 78 64 61 74 61 00 00)
+// CHECK-NEXT:     VirtualSize: 0x0
+// CHECK-NEXT:     VirtualAddress: 0x0
+// CHECK-NEXT:     RawDataSize: 52
+// CHECK-NEXT:     PointerToRawData: 0x3D0A20
+// CHECK-NEXT:     PointerToRelocations: 0x0
+// CHECK-NEXT:     PointerToLineNumbers: 0x0
+// CHECK-NEXT:     RelocationCount: 0
+// CHECK-NEXT:     LineNumberCount: 0
+// CHECK-NEXT:     Characteristics [ (0x40300040)
+// CHECK-NEXT:       IMAGE_SCN_ALIGN_4BYTES (0x300000)
+// CHECK-NEXT:       IMAGE_SCN_CNT_INITIALIZED_DATA (0x40)
+// CHECK-NEXT:       IMAGE_SCN_MEM_READ (0x40000000)
+// CHECK-NEXT:     ]
+// CHECK-NEXT:   }
+// CHECK-NEXT:   Section {
+// CHECK-NEXT:     Number: 5
+// CHECK-NEXT:     Name: .pdata (2E 70 64 61 74 61 00 00)
+// CHECK-NEXT:     VirtualSize: 0x0
+// CHECK-NEXT:     VirtualAddress: 0x0
+// CHECK-NEXT:     RawDataSize: 40
+// CHECK-NEXT:     PointerToRawData: 0x3D0A54
+// CHECK-NEXT:     PointerToRelocations: 0x3D0A7C
+// CHECK-NEXT:     PointerToLineNumbers: 0x0
+// CHECK-NEXT:     RelocationCount: 10
+// CHECK-NEXT:     LineNumberCount: 0
+// CHECK-NEXT:     Characteristics [ (0x40300040)
+// CHECK-NEXT:       IMAGE_SCN_ALIGN_4BYTES (0x300000)
+// CHECK-NEXT:       IMAGE_SCN_CNT_INITIALIZED_DATA (0x40)
+// CHECK-NEXT:       IMAGE_SCN_MEM_READ (0x40000000)
+// CHECK-NEXT:     ]
+// CHECK-NEXT:   }
+// CHECK-NEXT: ]
+// CHECK-LABEL: Relocations [
+// CHECK-NEXT:   Section (1) .text {
+// CHECK-NEXT:     0x186A04 IMAGE_REL_ARM64_BRANCH26 foo (14)
+// CHECK-NEXT:     0x3D091C IMAGE_REL_ARM64_BRANCH26 foo (14)
+// CHECK-NEXT:   }
+// CHECK-NEXT:   Section (5) .pdata {
+// CHECK-NEXT:     0x0 IMAGE_REL_ARM64_ADDR32NB .text (0)
+// CHECK-NEXT:     0x4 IMAGE_REL_ARM64_ADDR32NB .xdata (9)
+// CHECK-NEXT:     0x8 IMAGE_REL_ARM64_ADDR32NB .text (0)
+// CHECK-NEXT:     0xC IMAGE_REL_ARM64_ADDR32NB .xdata (9)
+// CHECK-NEXT:     0x10 IMAGE_REL_ARM64_ADDR32NB $L.text_1 (2)
+// CHECK-NEXT:     0x14 IMAGE_REL_ARM64_ADDR32NB .xdata (9)
+// CHECK-NEXT:     0x18 IMAGE_REL_ARM64_ADDR32NB $L.text_2 (3)
+// CHECK-NEXT:     0x1C IMAGE_REL_ARM64_ADDR32NB .xdata (9)
+// CHECK-NEXT:     0x20 IMAGE_REL_ARM64_ADDR32NB $L.text_3 (4)
+// CHECK-NEXT:     0x24 IMAGE_REL_ARM64_ADDR32NB .xdata (9)
+// CHECK-NEXT:   }
+// CHECK-NEXT: ]
+// CHECK-LABEL: UnwindInformation [
+// CHECK-NEXT:   RuntimeFunction {
+// CHECK-NEXT:     Function: a (0x0)
+// CHECK-NEXT:     ExceptionRecord: .xdata (0x0)
+// CHECK-NEXT:     ExceptionData {
+// CHECK-NEXT:       FunctionLength: 1048572
+// CHECK-NEXT:       Version: 0
+// CHECK-NEXT:       ExceptionData: No
+// CHECK-NEXT:       EpiloguePacked: No
+// CHECK-NEXT:       EpilogueScopes: 0
+// CHECK-NEXT:       ByteCodeLength: 4
+// CHECK-NEXT:       Prologue [
+// CHECK-NEXT:         0xd561              ; str x30, [sp, #-16]!
+// CHECK-NEXT:         0xe4                ; end
+// CHECK-NEXT:       ]
+// CHECK-NEXT:       EpilogueScopes [
+// CHECK-NEXT:       ]
+// CHECK-NEXT:     }
+// CHECK-NEXT:   }
+// CHECK-NEXT:   RuntimeFunction {
+// CHECK-NEXT:     Function: a +0xFFFFC (0xFFFFC)
+// CHECK-NEXT:     ExceptionRecord: .xdata +0x8 (0x8)
+// CHECK-NEXT:     ExceptionData {
+// CHECK-NEXT:       FunctionLength: 551444
+// CHECK-NEXT:       Version: 0
+// CHECK-NEXT:       ExceptionData: No
+// CHECK-NEXT:       EpiloguePacked: Yes
+// CHECK-NEXT:       EpilogueOffset: 1
+// CHECK-NEXT:       ByteCodeLength: 4
+// CHECK-NEXT:       Prologue [
+// CHECK-NEXT:         0xe5                ; end_c
+// CHECK-NEXT:         0xd561              ; str x30, [sp, #-16]!
+// CHECK-NEXT:         0xe4                ; end
+// CHECK-NEXT:       ]
+// CHECK-NEXT:       Epilogue [
+// CHECK-NEXT:         0xd561              ; ldr x30, [sp], #16
+// CHECK-NEXT:         0xe4                ; end
+// CHECK-NEXT:       ]
+// CHECK-NEXT:     }
+// CHECK-NEXT:   }
+// CHECK-NEXT:   RuntimeFunction {
+// CHECK-NEXT:     Function: b (0x186A10)
+// CHECK-NEXT:     ExceptionRecord: .xdata +0x10 (0x10)
+// CHECK-NEXT:     ExceptionData {
+// CHECK-NEXT:       FunctionLength: 1048572
+// CHECK-NEXT:       Version: 0
+// CHECK-NEXT:       ExceptionData: No
+// CHECK-NEXT:       EpiloguePacked: No
+// CHECK-NEXT:       EpilogueScopes: 0
+// CHECK-NEXT:       ByteCodeLength: 8
+// CHECK-NEXT:       Prologue [
+// CHECK-NEXT:         0xe1                ; mov fp, sp
+// CHECK-NEXT:         0xc81e              ; stp x19, x20, [sp, #240]
+// CHECK-NEXT:         0x9f                ; stp x29, x30, [sp, #-256]!
+// CHECK-NEXT:         0xe4                ; end
+// CHECK-NEXT:       ]
+// CHECK-NEXT:       EpilogueScopes [
+// CHECK-NEXT:       ]
+// CHECK-NEXT:     }
+// CHECK-NEXT:   }
+// CHECK-NEXT:   RuntimeFunction {
+// CHECK-NEXT:     Function: $L.text_2 +0x86A0C (0x286A0C)
+// CHECK-NEXT:     ExceptionRecord: .xdata +0x1C (0x1C)
+// CHECK-NEXT:     ExceptionData {
+// CHECK-NEXT:       FunctionLength: 1048572
+// CHECK-NEXT:       Version: 0
+// CHECK-NEXT:       ExceptionData: No
+// CHECK-NEXT:       EpiloguePacked: Yes
+// CHECK-NEXT:       EpilogueOffset: 0
+// CHECK-NEXT:       ByteCodeLength: 8
+// CHECK-NEXT:       Prologue [
+// CHECK-NEXT:         0xe5                ; end_c
+// CHECK-NEXT:         0xe1                ; mov fp, sp
+// CHECK-NEXT:         0xc81e              ; stp x19, x20, [sp, #240]
+// CHECK-NEXT:         0x9f                ; stp x29, x30, [sp, #-256]!
+// CHECK-NEXT:         0xe4                ; end
+// CHECK-NEXT:       ]
+// CHECK-NEXT:     }
+// CHECK-NEXT:   }
+// CHECK-NEXT:   RuntimeFunction {
+// CHECK-NEXT:     Function: $L.text_3 +0x86A08 (0x386A08)
+// CHECK-NEXT:     ExceptionRecord: .xdata +0x28 (0x28)
+// CHECK-NEXT:     ExceptionData {
+// CHECK-NEXT:       FunctionLength: 302888
+// CHECK-NEXT:       Version: 0
+// CHECK-NEXT:       ExceptionData: No
+// CHECK-NEXT:       EpiloguePacked: Yes
+// CHECK-NEXT:       EpilogueOffset: 1
+// CHECK-NEXT:       ByteCodeLength: 8
+// CHECK-NEXT:       Prologue [
+// CHECK-NEXT:         0xe5                ; end_c
+// CHECK-NEXT:         0xe1                ; mov fp, sp
+// CHECK-NEXT:         0xc81e              ; stp x19, x20, [sp, #240]
+// CHECK-NEXT:         0x9f                ; stp x29, x30, [sp, #-256]!
+// CHECK-NEXT:         0xe4                ; end
+// CHECK-NEXT:       ]
+// CHECK-NEXT:       Epilogue [
+// CHECK-NEXT:         0xe1                ; mov sp, fp
+// CHECK-NEXT:         0xc81e              ; ldp x19, x20, [sp, #240]
+// CHECK-NEXT:         0x9f                ; ldp x29, x30, [sp], #256
+// CHECK-NEXT:         0xe4                ; end
+// CHECK-NEXT:       ]
+// CHECK-NEXT:     }
+// CHECK-NEXT:   }
+// CHECK-NEXT: ]
+
+  .text
+// A simple function with an single epilog mirroring the prolog.
+  .global a
+  .p2align  2
+  .seh_proc a
+a:
+  str x30, [sp, #-16]!
+  .seh_save_reg_x x30, 16
+  .seh_endprologue
+        .rept 400000
+        nop
+        .endr
+  bl  foo
+  .seh_startepilogue
+  ldr x30, [sp], #16
+  .seh_save_reg_x x30, 16
+  .seh_endepilogue
+  ret
+  .seh_endfunclet
+  .seh_endproc
+
+// Example 1 from https://docs.microsoft.com/en-us/cpp/build/arm64-exception-handling#function-fragments
+  .global b
+  .p2align  2
+  .seh_proc b
+b:
+  stp  x29, lr, [sp, #-256]!
+  .seh_save_fplr_x 256
+  stp  x19, x20, [sp, #240]
+  .seh_save_regp x19, 240
+  mov x29, fp
+  .seh_set_fp
+  .seh_endprologue
+        .rept 600000
+        nop
+        .endr
+  bl  foo
+  .seh_startepilogue
+  mov sp, x29
+  .seh_set_fp
+  ldp  x19, x20, [sp, #240]
+  .seh_save_regp x19, 240
+  ldp x29, lr, [sp], #256
+  .seh_save_fplr_x 256
+  .seh_endepilogue
+  ret
+  .seh_endfunclet
+  .seh_endproc

llvm/tools/llvm-readobj/ARMWinEHPrinter.cpp

bool Decoder::opcode_end(const uint8_t *OC, unsigned &Offset, unsigned Length,
   ++Offset;
   return true;
 }
 
 bool Decoder::opcode_end_c(const uint8_t *OC, unsigned &Offset, unsigned Length,
                            bool Prologue) {
   SW.startLine() << format("0x%02x                ; end_c\n", OC[Offset]);
   ++Offset;
-  return true;
+  return false;
 }
 
 bool Decoder::opcode_save_next(const uint8_t *OC, unsigned &Offset,
                                unsigned Length, bool Prologue) {
   if (Prologue)
     SW.startLine() << format("0x%02x                ; save next\n", OC[Offset]);
   else
     SW.startLine() << format("0x%02x                ; restore next\n",