[WIP] Add support for return from an SEH __finally block.

[efriedma]

Summary

MSVC has a warning C4532 for jumping out of a finally block. However, that only specifically covers break, continue, and goto. There's no warning for __leave and return.

It's not completely clear to me why MSVC considers "__leave" and "return" special, compared to other sorts of control flow. My best guess is that return-in-finally is so pervasive across drivers that the warning would be way too noisy. An alternative theory is that MSVC generates a call to _local_unwind in the normal path for certain uses of break/continue/goto, and that's somehow a hazard for unwinding.

In any case, the end result is that return shows up somewhat frequently in __finally blocks in Windows drivers, but I haven't seen any of the others.

This patch generates code that's similar to what MSVC generates: it branches directly to the return block for normal code, and uses _local_unwind to break out of exception handling.

Currently, this only handles return. Adding support for other ways of exiting a __finally blocks should be straightforward on top of this; it's mostly a question of managing the different possible destinations.

To represent the _local_unwind, this patch includes a new intrinsic llvm.seh_localunwind. This generates the necessary arguments to _local_unwind to the nearest catchpad automatically.

Some basic testing shows this seems to do the right thing in simple cases.

Handling __try/__finally inside the __try of a __try/__finally depends on the change to WinException.cpp. Still a bit ugly, but not sure how to make it cleaner.

__try/__finally inside a __finally block is broken because recoverAddrOfEscapedLocal can't recover the address of a variable allocated inside a __finally block. Here specifically, we compute the wrong value for SEHRetNowParent, but it's a general issue affecting any variable declared inside a __finally block. We probably need to store the frame pointer in an alloca in the parent function, and then load it later so we can recover the pointer we want, or something like that.

Based on the patch by Ten Tzen, particularly the code to call _local_unwind. I rewrote a bunch of the code to simplify it, and avoid using _local_unwind in cases where it isn't necessary.

[efriedma]

Figured out the __finally nesting thing; we were missing unwind table entries. (This is a consequence of the fact that our unwind tables don't cover the entire function, just invokes.) Need to think about a cleaner solution.

[efriedma]

I'm currently thinking it makes sense to introduce an intrinsic "void @llvm.localunwind() noreturn". No arguments. Only valid with SEH personality; must unwind to a catchpad with a filter named __IsLocalUnwind, or something like that. It would basically just lower to a call to _local_unwind, but it would compute the arguments in the backend. Primary downside is that it wouldn't be possible to unwind to anything other than the closest catchpad. But not sure if unwinding further is actually useful in practice, given the ways we need to use it.

The other thing I'm thinking about is how to restructure the code to make the catchpad hack in WinException.cpp a bit less hacky.

[efriedma]

Switch to use llvm.seh_localunwind intrinsic.

I'd appreciate any feedback at this point.

[rnk]

The approach makes sense to me, but can you include test cases to illustrate the IR and code that gets generated?

clang/lib/CodeGen/CGException.cpp

1830

We have the option to generalize the naming here to include other kinds of jumps that exit the scope of the finally. I think it's worth implementing __leave eventually, which I think users have asked for at some point. I could go either way, your choice.

2202

If we were to generalize now, we'd want to name this something like HasLocalUnwind.

llvm/lib/CodeGen/AsmPrinter/WinException.cpp

596–606

Perhaps it's time to revisit this.

623–626

I don't understand what this is doing, but I should look at the tests.

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

3040

Since this isn't verifier checked yet, is it possible to emit a diagnostic via the LLVMContext and select this instruction to nothing, or would that leave behind disconnected unreachable MBBs that break too many invariants later?

[efriedma]

clang/lib/CodeGen/CGException.cpp

1830

Probably better to rename once we actually implement that? I'm intentionally not trying to detect other forms of control flow because EnterSEHTryStmt only knows how to return from the function. If we're going to branch elsewhere, we need to return a list of the destinations, not just a boolean "there's a return statement".

llvm/lib/CodeGen/AsmPrinter/WinException.cpp

623–626

I don't remember all the details of this, but as I noted in the commit message, it affects nesting __try/__finally in __try. I think without this, the destination of the _local_unwind somehow ended up inheriting an unrelated SEH state.

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

3040

I can't find any other code trying to report diagnostics that way, but I guess I can look into it.

[efriedma]

Rebased so it builds, and added a couple tests, to unblock anyone wanting to look at this further. Haven't re-done my runtime testing. Still haven't addressed all the review comments.

[efriedma]

Fixup a couple LLVM tests which are failing; I think they're affected by the WinException.cpp changes? Maybe need to look a bit more closely to see if the changes make sense.

clang/lib/CodeGen/CGException.cpp

llvm::BasicBlock *CodeGenFunction::getEHResumeBlock(bool isCleanup) {
   LPadVal = Builder.CreateInsertValue(LPadVal, Sel, 1, "lpad.val");
 
   Builder.CreateResume(LPadVal);
   Builder.restoreIP(SavedIP);
   return EHResumeBlock;
 }
 
 void CodeGenFunction::EmitSEHTryStmt(const SEHTryStmt &S) {
-  EnterSEHTryStmt(S);
+  bool ContainsRetStmt = false;
+  EnterSEHTryStmt(S, ContainsRetStmt);
   {
     JumpDest TryExit = getJumpDestInCurrentScope("__try.__leave");
 
     SEHTryEpilogueStack.push_back(&TryExit);
 
     llvm::BasicBlock *TryBB = nullptr;
     // IsEHa: emit an invoke to _seh_try_begin() runtime for -EHa
     if (getLangOpts().EHAsynch) {
EnterSEHTryStmt(S, ContainsRetStmt);
 
     SEHTryEpilogueStack.pop_back();
 
     if (!TryExit.getBlock()->use_empty())
       EmitBlock(TryExit.getBlock(), /*IsFinished=*/true);
     else
       delete TryExit.getBlock();
   }
-  ExitSEHTryStmt(S);
+  ExitSEHTryStmt(S, ContainsRetStmt);
 }
 
 //  Recursively walk through blocks in a _try
 //      and make all memory instructions volatile
 void CodeGenFunction::VolatilizeTryBlocks(
     llvm::BasicBlock *BB, llvm::SmallPtrSet<llvm::BasicBlock *, 10> &V) {
   if (BB == SEHTryEpilogueStack.back()->getBlock() /* end of Try */ ||
       !V.insert(BB).second /* already visited */ ||
if (TI) {
     for (unsigned I = 0; I < N; I++)
       VolatilizeTryBlocks(TI->getSuccessor(I), V);
   }
 }
 
 namespace {
 struct PerformSEHFinally final : EHScopeStack::Cleanup {
   llvm::Function *OutlinedFinally;
-  PerformSEHFinally(llvm::Function *OutlinedFinally)
-      : OutlinedFinally(OutlinedFinally) {}
+  bool RetFromFinally;
+  PerformSEHFinally(llvm::Function *OutlinedFinally, bool RetFromFinally)
+      : OutlinedFinally(OutlinedFinally), RetFromFinally(RetFromFinally) {}
 
   void Emit(CodeGenFunction &CGF, Flags F) override {
     ASTContext &Context = CGF.getContext();
     CodeGenModule &CGM = CGF.CGM;
 
     CallArgList Args;
 
     // Compute the two argument values.
void Emit(CodeGenFunction &CGF, Flags F) override {
     Args.add(RValue::get(FP), ArgTys[1]);
 
     // Arrange a two-arg function info and type.
     const CGFunctionInfo &FnInfo =
         CGM.getTypes().arrangeBuiltinFunctionCall(Context.VoidTy, Args);
 
     auto Callee = CGCallee::forDirect(OutlinedFinally);
     CGF.EmitCall(FnInfo, Callee, ReturnValueSlot(), Args);
+
+    if (F.isForEHCleanup() && RetFromFinally) {
+      llvm::BasicBlock *AbnormalCont = CGF.createBasicBlock("if.then");
+      llvm::BasicBlock *NormalCont = CGF.createBasicBlock("if.end");
+      llvm::Value *ShouldRetLoad =
+          CGF.Builder.CreateLoad(CGF.SEHRetNowStack.back());
+      llvm::Value *ShouldRet = CGF.Builder.CreateIsNotNull(ShouldRetLoad);
+
+      CGF.Builder.CreateCondBr(ShouldRet, AbnormalCont, NormalCont);
+      CGF.EmitBlock(AbnormalCont);
+      CGF.EmitSEHLocalUnwind();
+      CGF.Builder.CreateUnreachable();
+
+      CGF.EmitBlock(NormalCont);
+    }
   }
 };
 } // end anonymous namespace
 
 namespace {
 /// Find all local variable captures in the statement.
 struct CaptureFinder : ConstStmtVisitor<CaptureFinder> {
   CodeGenFunction &ParentCGF;
   const VarDecl *ParentThis;
   llvm::SmallSetVector<const VarDecl *, 4> Captures;
   Address SEHCodeSlot = Address::invalid();
+  bool ContainsRetStmt = false;
   CaptureFinder(CodeGenFunction &ParentCGF, const VarDecl *ParentThis)
       : ParentCGF(ParentCGF), ParentThis(ParentThis) {}
 
   // Return true if we need to do any capturing work.
   bool foundCaptures() {
-    return !Captures.empty() || SEHCodeSlot.isValid();
+    return !Captures.empty() || SEHCodeSlot.isValid() || ContainsRetStmt;
   }
 
   void Visit(const Stmt *S) {
     // See if this is a capture, then recurse.
     ConstStmtVisitor<CaptureFinder>::Visit(S);
     for (const Stmt *Child : S->children())
       if (Child)
         Visit(Child);
case Builtin::BI_exception_code:
       // This is the simple case where we are the outermost finally. All we
       // have to do here is make sure we escape this and recover it in the
       // outlined handler.
       if (!SEHCodeSlot.isValid())
         SEHCodeSlot = ParentCGF.SEHCodeSlotStack.back();
       break;
     }
   }
+
+  void VisitReturnStmt(const ReturnStmt *) { ContainsRetStmt = true; }
+};
+} // end anonymous namespace
+
+namespace {
+/// Find all local variable captures in the statement.
+struct ReturnStmtFinder : ConstStmtVisitor<ReturnStmtFinder> {
+  bool ContainsRetStmt = false;
+
+  void Visit(const Stmt *S) {
+    // See if this is a capture, then recurse.
+    ConstStmtVisitor::Visit(S);
+    for (const Stmt *Child : S->children())
+      if (Child)
+        Visit(Child);
+  }
+
+  void VisitReturnStmt(const ReturnStmt *) { ContainsRetStmt = true; }
 };
 } // end anonymous namespace
 
 Address CodeGenFunction::recoverAddrOfEscapedLocal(CodeGenFunction &ParentCGF,
                                                    Address ParentVar,
                                                    llvm::Value *ParentFP) {
   llvm::CallInst *RecoverCall = nullptr;
   CGBuilderTy Builder(*this, AllocaInsertPt);
Address CodeGenFunction::recoverAddrOfEscapedLocal(CodeGenFunction &ParentCGF,
   return ParentVar.withPointer(ChildVar, KnownNonNull);
 }
 
 void CodeGenFunction::EmitCapturedLocals(CodeGenFunction &ParentCGF,
                                          const Stmt *OutlinedStmt,
                                          bool IsFilter) {
   // Find all captures in the Stmt.
   CaptureFinder Finder(ParentCGF, ParentCGF.CXXABIThisDecl);
+  if (OutlinedStmt)
     Finder.Visit(OutlinedStmt);
 
   // We can exit early on x86_64 when there are no captures. We just have to
   // save the exception code in filters so that __exception_code() works.
   if (!Finder.foundCaptures() &&
       CGM.getTarget().getTriple().getArch() != llvm::Triple::x86) {
     if (IsFilter)
       EmitSEHExceptionCodeSave(ParentCGF, nullptr, nullptr);
     return;
void CodeGenFunction::EmitCapturedLocals(CodeGenFunction &ParentCGF,
 
   if (Finder.SEHCodeSlot.isValid()) {
     SEHCodeSlotStack.push_back(
         recoverAddrOfEscapedLocal(ParentCGF, Finder.SEHCodeSlot, ParentFP));
   }
 
   if (IsFilter)
     EmitSEHExceptionCodeSave(ParentCGF, ParentFP, EntryFP);
+
+  if (Finder.ContainsRetStmt) {
+    SEHRetNowParent = recoverAddrOfEscapedLocal(
+        ParentCGF, ParentCGF.SEHRetNowStack.back(), ParentFP);
+    Address ParentSEHRetVal =
+        ParentCGF.ParentCGF ? ParentCGF.SEHReturnValue : ParentCGF.ReturnValue;
+    if (ParentSEHRetVal.isValid())
+      SEHReturnValue =
+          recoverAddrOfEscapedLocal(ParentCGF, ParentSEHRetVal, ParentFP);
+  }
 }
 
 /// Arrange a function prototype that can be called by Windows exception
 /// handling personalities. On Win64, the prototype looks like:
 /// RetTy func(void *EHPtrs, void *ParentFP);
 void CodeGenFunction::startOutlinedSEHHelper(CodeGenFunction &ParentCGF,
                                              bool IsFilter,
                                              const Stmt *OutlinedStmt) {
llvm::Value *CodeGenFunction::EmitSEHAbnormalTermination() {
   // Abnormal termination is just the first parameter to the outlined finally
   // helper.
   auto AI = CurFn->arg_begin();
   return Builder.CreateZExt(&*AI, Int32Ty);
 }
 
 void CodeGenFunction::pushSEHCleanup(CleanupKind Kind,
                                      llvm::Function *FinallyFunc) {
-  EHStack.pushCleanup<PerformSEHFinally>(Kind, FinallyFunc);
+  EHStack.pushCleanup<PerformSEHFinally>(Kind, FinallyFunc, false);
 }
 
-void CodeGenFunction::EnterSEHTryStmt(const SEHTryStmt &S) {
+void CodeGenFunction::EnterSEHTryStmt(const SEHTryStmt &S,
+                                      bool &ContainsRetStmt) {
   CodeGenFunction HelperCGF(CGM, /*suppressNewContext=*/true);
   HelperCGF.ParentCGF = this;
   if (const SEHFinallyStmt *Finally = S.getFinallyHandler()) {
+    ReturnStmtFinder Finder;
+    Finder.Visit(Finally);
+    ContainsRetStmt = Finder.ContainsRetStmt;
+    if (ContainsRetStmt) {
+      // Suppose we have something like:
+      // __try {
+      //   f1();
+      // } __finally {
+      //   f2();
+      //   if (z)
+      //     return;
+      //   f3();
+      // }
+      //
+      // We want to generate code something like this, where "StopUnwinding()"
+      // refers to the operation of aborting the unwind, and jumping back
+      // to normal code.
+      //
+      //  int immediate_return = 0;
+      //  __try {
+      //    f1();
+      //  } __finally {
+      //    f2();
+      //    if (z) {
+      //      immediate_return = 1;
+      //      goto end_of_finally;
+      //    }
+      //    f3();
+      //    end_of_finally:
+      //    if (_abnormal_termination())
+      //      StopUnwinding();
+      //  }
+      //  if (immediate_return) {
+      //    return;
+      //  }
+      //
+      // To handle the non-unwind case, we need to synthesize the
+      // "immediate_return" variable, and use it to change control flow
+      // after the finally block.
+      //
+      // To make "StopUnwinding()" work, we use _local_unwind.  This function
+      // tells the SEH unwinder to recompute the unwind action: instead of
+      // using the __except handler that was already computed, stop unwinding
+      // when the unwinder reaches the current function.  (The mechanism used
+      // here is unofficially called a "collided unwind".)
+      //
+      // We represent the destination of _local_unwind with a fake CatchPad:
+      // when the backend sees a filter named "__IsLocalUnwind", it arranges
+      // the unwind tables so that _local_unwind stops at that CatchPad, but
+      // other unwinding ignores it.
+      //
+      // Note that this construct could itself be inside an __try or __finally
+      // block.
+      //
+      // If it's inside the __try of a __try/__finally, the outer __finally
+      // executes before the function returns.
+      //
+      // If it's inside a __finally, we need to jump out of that __finally
+      // in a similar way.
+
+      // Initialize the variable controlling the exception filter.
+      SEHRetNowStack.push_back(
+          CreateTempAlloca(CGM.Int8Ty, CharUnits::fromQuantity(1), "retnow"));
+      Builder.CreateStore(Builder.getInt8(0), SEHRetNowStack.back());
+
+      // Create the exception filter.
+      EHCatchScope *CatchScope = EHStack.pushCatch(1);
+      llvm::Function *FilterFunc = GenerateSEHIsLocalUnwindFunction();
+      llvm::Constant *OpaqueFunc =
+          llvm::ConstantExpr::getBitCast(FilterFunc, Int8PtrTy);
+      CatchScope->setHandler(0, OpaqueFunc, createBasicBlock("__except.ret"));
+    }
     // Outline the finally block.
     llvm::Function *FinallyFunc =
         HelperCGF.GenerateSEHFinallyFunction(*this, *Finally);
 
     // Push a cleanup for __finally blocks.
-    EHStack.pushCleanup<PerformSEHFinally>(NormalAndEHCleanup, FinallyFunc);
+    EHStack.pushCleanup<PerformSEHFinally>(NormalAndEHCleanup, FinallyFunc,
+                                           ContainsRetStmt);
     return;
   }
 
   // Otherwise, we must have an __except block.
   const SEHExceptStmt *Except = S.getExceptHandler();
   assert(Except);
   EHCatchScope *CatchScope = EHStack.pushCatch(1);
   SEHCodeSlotStack.push_back(
void CodeGenFunction::EnterSEHTryStmt(const SEHTryStmt &S,
   // in place of the RTTI typeinfo global that C++ EH uses.
   llvm::Function *FilterFunc =
       HelperCGF.GenerateSEHFilterFunction(*this, *Except);
   llvm::Constant *OpaqueFunc =
       llvm::ConstantExpr::getBitCast(FilterFunc, Int8PtrTy);
   CatchScope->setHandler(0, OpaqueFunc, createBasicBlock("__except.ret"));
 }
 
-void CodeGenFunction::ExitSEHTryStmt(const SEHTryStmt &S) {
+llvm::Function *CodeGenFunction::GenerateSEHIsLocalUnwindFunction() {
+  // IsLocalUnwind is a void dummy func just for readability.
+  if (llvm::Function *F = CGM.getModule().getFunction("__IsLocalUnwind"))
+    return F;
+
+  llvm::LLVMContext &Ctx = getLLVMContext();
+  llvm::Type *ArgTys[] = {llvm::Type::getInt8PtrTy(Ctx),
+                          llvm::Type::getInt8PtrTy(Ctx)};
+  return llvm::Function::Create(
+      llvm::FunctionType::get(llvm::Type::getVoidTy(Ctx), ArgTys, false),
+      llvm::GlobalVariable::ExternalWeakLinkage, "__IsLocalUnwind",
+      &CGM.getModule());
+}
+
+void CodeGenFunction::EmitSEHLocalUnwind() {
+  EmitRuntimeCallOrInvoke(CGM.getIntrinsic(llvm::Intrinsic::seh_localunwind));
+}
+
+void CodeGenFunction::ExitSEHTryStmt(const SEHTryStmt &S,
+                                     bool ContainsRetStmt) {
   // Just pop the cleanup if it's a __finally block.
   if (S.getFinallyHandler()) {
     PopCleanupBlock();
+    if (ContainsRetStmt) {
+      // Create __except block and control flow handling for return from
+      // __finally. See comment in EnterSEHTryStmt.
+      //
+      // First, create the point where we check for a return
+      // from the __finally.
+      llvm::BasicBlock *ContBB = createBasicBlock("__finally.cont");
+      if (HaveInsertPoint())
+        Builder.CreateBr(ContBB);
+
+      EmitBlock(ContBB);
+
+      // On the normal path, check if we have a return-from-finally.
+      llvm::BasicBlock *AbnormalCont = createBasicBlock("if.then");
+      llvm::BasicBlock *NormalCont = createBasicBlock("if.end");
+      llvm::Value *ShouldRetLoad = Builder.CreateLoad(SEHRetNowStack.back());
+      llvm::Value *ShouldRet = Builder.CreateIsNotNull(ShouldRetLoad);
+
+      Builder.CreateCondBr(ShouldRet, AbnormalCont, NormalCont);
+
+      // Check if our filter function returned true.
+      EHCatchScope &CatchScope = cast<EHCatchScope>(*EHStack.begin());
+      emitCatchDispatchBlock(*this, CatchScope);
+
+      // Grab the block before we pop the handler.
+      llvm::BasicBlock *CatchPadBB = CatchScope.getHandler(0).Block;
+      EHStack.popCatch();
+
+      // The catch block only catches return-from-finally.
+      EmitBlockAfterUses(CatchPadBB);
+      llvm::CatchPadInst *CPI =
+          cast<llvm::CatchPadInst>(CatchPadBB->getFirstNonPHI());
+      Builder.CreateCatchRet(CPI, AbnormalCont);
+      EmitBlock(AbnormalCont);
+
+      // If the try block is nested inside a finally block, forward the
+      // return from __finally to the parent function.
+      if (SEHRetNowParent.isValid())
+        Builder.CreateStore(Builder.getInt8(1), SEHRetNowParent);
+      EmitBranchThroughCleanup(ReturnBlock);
+
+      EmitBlock(NormalCont);
+    }
     return;
   }
 
   // IsEHa: emit an invoke _seh_try_end() to mark end of FT flow
   if (getLangOpts().EHAsynch && Builder.GetInsertBlock()) {
     llvm::FunctionCallee SehTryEnd = getSehTryEndFn(CGM);
     EmitRuntimeCallOrInvoke(SehTryEnd);
   }

clang/lib/CodeGen/CGStmt.cpp

auto *SLocPtr =
                                  llvm::GlobalVariable::PrivateLinkage, SLoc);
     SLocPtr->setUnnamedAddr(llvm::GlobalValue::UnnamedAddr::Global);
     CGM.getSanitizerMetadata()->disableSanitizerForGlobal(SLocPtr);
     assert(ReturnLocation.isValid() && "No valid return location");
     Builder.CreateStore(Builder.CreateBitCast(SLocPtr, Int8PtrTy),
                         ReturnLocation);
   }
 
-  // Returning from an outlined SEH helper is UB, and we already warn on it.
+  Address ReturnValue = this->ReturnValue;
   if (IsOutlinedSEHHelper) {
-    Builder.CreateUnreachable();
-    Builder.ClearInsertionPoint();
+    Builder.CreateStore(Builder.getInt8(1), SEHRetNowParent);
+    ReturnValue = SEHReturnValue;
   }
 
   // Emit the result value, even if unused, to evaluate the side effects.
   const Expr *RV = S.getRetValue();
 
   // Record the result expression of the return statement. The recorded
   // expression is used to determine whether a block capture's lifetime should
   // end at the end of the full expression as opposed to the end of the scope

clang/lib/CodeGen/CodeGenFunction.h

public:
   /// write the current selector value into this alloca.
   llvm::AllocaInst *EHSelectorSlot = nullptr;
 
   /// A stack of exception code slots. Entering an __except block pushes a slot
   /// on the stack and leaving pops one. The __exception_code() intrinsic loads
   /// a value from the top of the stack.
   SmallVector<Address, 1> SEHCodeSlotStack;
 
+  /// Variable that indicates abnormal termination from the a child finally
+  /// block.
+  SmallVector<Address, 1> SEHRetNowStack;
+
+  /// Ponter to the parent function's SEHRetNow variable.
+  Address SEHRetNowParent = Address::invalid();
+
+  /// Ponter to the root function's ReturnValue variable.
+  Address SEHReturnValue = Address::invalid();
+
   /// Value returned by __exception_info intrinsic.
   llvm::Value *SEHInfo = nullptr;
 
   /// Emits a landing pad for the current EH stack.
   llvm::BasicBlock *EmitLandingPad();
 
   llvm::BasicBlock *getInvokeDestImpl();
 

   RValue EmitCoroutineIntrinsic(const CallExpr *E, unsigned int IID);
 
   void EnterCXXTryStmt(const CXXTryStmt &S, bool IsFnTryBlock = false);
   void ExitCXXTryStmt(const CXXTryStmt &S, bool IsFnTryBlock = false);
 
   void EmitCXXTryStmt(const CXXTryStmt &S);
   void EmitSEHTryStmt(const SEHTryStmt &S);
   void EmitSEHLeaveStmt(const SEHLeaveStmt &S);
-  void EnterSEHTryStmt(const SEHTryStmt &S);
-  void ExitSEHTryStmt(const SEHTryStmt &S);
+  void EnterSEHTryStmt(const SEHTryStmt &S, bool &ContainsRetStmt);
+  void ExitSEHTryStmt(const SEHTryStmt &S, bool ContainsRetStmt);
   void VolatilizeTryBlocks(llvm::BasicBlock *BB,
                            llvm::SmallPtrSet<llvm::BasicBlock *, 10> &V);
 
+  void EmitSEHLocalUnwind();
+  llvm::Function *GenerateSEHIsLocalUnwindFunction();
+
   void pushSEHCleanup(CleanupKind kind,
                       llvm::Function *FinallyFunc);
   void startOutlinedSEHHelper(CodeGenFunction &ParentCGF, bool IsFilter,
                               const Stmt *OutlinedStmt);
 
   llvm::Function *GenerateSEHFilterFunction(CodeGenFunction &ParentCGF,
                                             const SEHExceptStmt &Except);
 

clang/lib/Sema/SemaStmt.cpp

Sema::ActOnReturnStmt(SourceLocation ReturnLoc, Expr *RetValExp,
   if (R.isInvalid() || ExprEvalContexts.back().isDiscardedStatementContext())
     return R;
 
   VarDecl *VD =
       const_cast<VarDecl *>(cast<ReturnStmt>(R.get())->getNRVOCandidate());
 
   CurScope->updateNRVOCandidate(VD);
 
-  CheckJumpOutOfSEHFinally(*this, ReturnLoc, *CurScope->getFnParent());
-
   return R;
 }
 
 static bool CheckSimplerImplicitMovesMSVCWorkaround(const Sema &S,
                                                     const Expr *E) {
   if (!E || !S.getLangOpts().CPlusPlus23 || !S.getLangOpts().MSVCCompat)
     return false;
   const Decl *D = E->getReferencedDeclOfCallee();

clang/test/CodeGen/exceptions-seh-finally.c

 }
 
 // CHECK-LABEL: define dso_local i32 @nested___finally___finally
 // CHECK: invoke void @"?fin$1@0@nested___finally___finally@@"({{.*}})
 // CHECK:          to label %[[outercont:[^ ]*]] unwind label %[[lpad:[^ ]*]]
 //
 // CHECK: [[outercont]]
 // CHECK: call void @"?fin$0@0@nested___finally___finally@@"({{.*}})
-// CHECK-NEXT: ret i32 0
+// CHECK-NEXT: load i32
+// CHECK-NEXT: switch i32
 //
 // CHECK: [[lpad]]
 // CHECK-NEXT: %[[pad:[^ ]*]] = cleanuppad
 // CHECK: call void @"?fin$0@0@nested___finally___finally@@"({{.*}})
 // CHECK-NEXT: cleanupret from %[[pad]] unwind to caller
 
 // CHECK-LABEL: define internal void @"?fin$0@0@nested___finally___finally@@"({{.*}})
 // CHECK-SAME: [[finally_attrs]]
 // CHECK: ret void
 
 // CHECK-LABEL: define internal void @"?fin$1@0@nested___finally___finally@@"({{.*}})
 // CHECK-SAME: [[finally_attrs]]
-// CHECK: unreachable
+// CHECK: store i32 1
+// CHECK-NEXT: ret void
 
 // FIXME: Our behavior seems suspiciously different.
 
 int nested___finally___finally_with_eh_edge(void) {
   __try {
     __try {
       might_crash();
     } __finally {
       return 899;
     }
   } __finally {
     // Intentionally no return here.
   }
   return 912;
 }
 // CHECK-LABEL: define dso_local i32 @nested___finally___finally_with_eh_edge
 // CHECK: invoke void @might_crash()
 // CHECK-NEXT: to label %[[invokecont:[^ ]*]] unwind label %[[lpad1:[^ ]*]]
 //
 // [[invokecont]]
 // CHECK: invoke void @"?fin$1@0@nested___finally___finally_with_eh_edge@@"({{.*}})
 // CHECK-NEXT:       to label %[[outercont:[^ ]*]] unwind label %[[lpad2:[^ ]*]]
 //
 // CHECK: [[outercont]]
-// CHECK: call void @"?fin$0@0@nested___finally___finally_with_eh_edge@@"({{.*}})
-// CHECK-NEXT: ret i32 912
+// CHECK-NEXT: br label
 //
 // CHECK: [[lpad1]]
 // CHECK-NEXT: %[[innerpad:[^ ]*]] = cleanuppad
 // CHECK: invoke void @"?fin$1@0@nested___finally___finally_with_eh_edge@@"({{.*}})
-// CHECK-NEXT:    label %[[innercleanupretbb:[^ ]*]] unwind label %[[lpad2:[^ ]*]]
+// CHECK-NEXT:    label %[[invokecont2:[^ ]*]] unwind label %[[lpad2:[^ ]*]]
 //
-// CHECK: [[innercleanupretbb]]
+// CHECK: [[invokecont2]]
+// CHECK: br i1 {{.*}}, label %[[ifthen:[^ ]*]], label %[[ifend:[^ ]*]]
+//
+// CHECK: [[ifend]]
 // CHECK-NEXT: cleanupret from %[[innerpad]] unwind label %[[lpad2]]
 //
 // CHECK: [[lpad2]]
+// CHECK: catchswitch {{.*}} unwind label %[[lpad3:[^ ]*]]
+//
+// CHECK: [[lpad3]]
 // CHECK-NEXT: %[[outerpad:[^ ]*]] = cleanuppad
 // CHECK: call void @"?fin$0@0@nested___finally___finally_with_eh_edge@@"({{.*}})
 // CHECK-NEXT: cleanupret from %[[outerpad]] unwind to caller
 
 // CHECK-LABEL: define internal void @"?fin$0@0@nested___finally___finally_with_eh_edge@@"({{.*}})
 // CHECK-SAME: [[finally_attrs]]
 // CHECK: ret void
 
 // CHECK-LABEL: define internal void @"?fin$1@0@nested___finally___finally_with_eh_edge@@"({{.*}})
 // CHECK-SAME: [[finally_attrs]]
-// CHECK: unreachable
+// CHECK: store i32 899
 
 void finally_within_finally(void) {
   __try {
     might_crash();
   } __finally {
     __try {
       might_crash();
     } __finally {

clang/test/CodeGen/exceptions-seh-leave.c

int __leave_with___finally_simple(void) {
   return 1;
 }
 // CHECK-LABEL: define dso_local i32 @__leave_with___finally_simple()
 // CHECK: store i32 15, ptr %myres
 // CHECK-NEXT: br label %[[tryleave:[^ ]*]]
 // CHECK-NOT: store i32 23
 // CHECK: [[tryleave]]
 // CHECK-NEXT: %[[fp:[^ ]*]] = call ptr @llvm.localaddress()
-// CHECK-NEXT: call void @"?fin$0@0@__leave_with___finally_simple@@"(i8 noundef 0, ptr noundef %[[fp]])
+// CHECK-NEXT: invoke void @"?fin$0@0@__leave_with___finally_simple@@"(i8 noundef 0, ptr noundef %[[fp]])
 
 // __finally block doesn't return, __finally.cont doesn't exist.
 int __leave_with___finally_noreturn(void) {
   int myres = 0;
   __try {
     myres = 15;
     __leave;
     myres = 23;

 // CHECK-NEXT:       to label %[[cont:.*]] unwind label %{{.*}}
 // For __finally, there needs to be an explicit __try.__leave, because
 // abnormal.termination.slot needs to be set there.
 // CHECK: [[cont]]
 // CHECK-NEXT: br label %[[tryleave:[^ ]*]]
 // CHECK-NOT: store i32 23
 // CHECK: [[tryleave]]
 // CHECK-NEXT: %[[fp:[^ ]*]] = call ptr @llvm.localaddress()
-// CHECK-NEXT: call void @"?fin$0@0@__leave_with___finally@@"(i8 noundef 0, ptr noundef %[[fp]])
+// CHECK-NEXT: invoke void @"?fin$0@0@__leave_with___finally@@"(i8 noundef 0, ptr noundef %[[fp]])
 
 
 //////////////////////////////////////////////////////////////////////////////
 // Mixed, nested cases.
 
 int nested___except___finally(void) {
   int myres = 0;
   __try {

 // CHECK-LABEL: invoke void @g()
 // CHECK-NEXT:       to label %[[g1_cont1:.*]] unwind label %[[g1_lpad:.*]]
 
 // CHECK: [[g1_cont1]]
 // CHECK-NEXT: %[[fp:[^ ]*]] = call ptr @llvm.localaddress()
 // CHECK-NEXT: invoke void @"?fin$0@0@nested___except___finally@@"(i8 noundef 0, ptr noundef %[[fp]])
 // CHECK-NEXT:       to label %[[fin_cont:.*]] unwind label %[[g2_lpad:.*]]
 
-// CHECK: [[fin_cont]]
-// CHECK: store i32 51, ptr %
-// CHECK-NEXT: br label %[[trycont:[^ ]*]]
-
 // CHECK: [[g1_lpad]]
 // CHECK-NEXT: cleanuppad
 // CHECK-NEXT: %[[fp:[^ ]*]] = call ptr @llvm.localaddress()
 // CHECK-NEXT: invoke void @"?fin$0@0@nested___except___finally@@"(i8 noundef 1, ptr noundef %[[fp]])
 // CHECK-NEXT:       to label %[[g1_resume:.*]] unwind label %[[g2_lpad]]
 // CHECK: cleanupret {{.*}} unwind label %[[g2_lpad]]
 
 // CHECK: [[g2_lpad]]
 // CHECK: catchpad {{.*}} [ptr null]
 // CHECK: catchret
-// CHECK: br label %[[trycont]]
-
-// CHECK: [[trycont]]
-// CHECK-NEXT: ret i32 1
 
 // CHECK-LABEL: define internal void @"?fin$0@0@nested___except___finally@@"(i8 noundef %abnormal_termination, ptr noundef %frame_pointer)
 // CHECK: call void @g()
 // CHECK: unreachable
 
 int nested___except___except(void) {
   int myres = 0;
   __try {

 
 // CHECK: invoke void @g()
 // CHECK-NEXT:       to label %[[g1_cont:.*]] unwind label %[[g1_lpad:.*]]
 
 // CHECK: [[g1_cont]]
 // CHECK: store i32 16, ptr %[[myres:[^ ]*]],
 // CHECK: %[[fp:[^ ]*]] = call ptr @llvm.localaddress()
 // CHECK-NEXT: invoke void @"?fin$1@0@nested___finally___finally@@"(i8 noundef 0, ptr noundef %[[fp]])
-// CHECK-NEXT:       to label %[[finally_cont:.*]] unwind label %[[g2_lpad:.*]]
-
-// CHECK: [[finally_cont]]
-// CHECK: store i32 51, ptr %[[myres]]
-// CHECK: %[[fp:[^ ]*]] = call ptr @llvm.localaddress()
-// CHECK-NEXT: call void @"?fin$0@0@nested___finally___finally@@"(i8 noundef 0, ptr noundef %[[fp]])
-// CHECK-NEXT: ret i32 1
+// CHECK-NEXT:       to label %[[finally_cont:.*]] unwind label %[[dispatch:.*]]
 
 // CHECK: [[g1_lpad]]
 // CHECK-NEXT: %[[padtoken:[^ ]*]] = cleanuppad within none []
 // CHECK-NEXT: %[[fp:[^ ]*]] = call ptr @llvm.localaddress()
 // CHECK-NEXT: invoke void @"?fin$1@0@nested___finally___finally@@"(i8 noundef 1, ptr noundef %[[fp]])
-// CHECK-NEXT:       to label %[[finally_cont2:.*]] unwind label %[[g2_lpad]]
+// CHECK-NEXT:       to label %[[finally_cont2:.*]] unwind label %[[dispatch]]
 // CHECK: [[finally_cont2]]
-// CHECK: cleanupret from %[[padtoken]] unwind label %[[g2_lpad]]
+// CHECK: cleanupret from %[[padtoken]] unwind label %[[dispatch]]
+
+// CHECK: [[dispatch]]
+// CHECK: catchswitch {{.*}} unwind label %[[g2_lpad:.*]]
 
 // CHECK: [[g2_lpad]]
 // CHECK-NEXT: %[[padtoken:[^ ]*]] = cleanuppad within none []
 // CHECK-NEXT: %[[fp:[^ ]*]] = call ptr @llvm.localaddress()
 // CHECK-NEXT: call void @"?fin$0@0@nested___finally___finally@@"(i8 noundef 1, ptr noundef %[[fp]])
 // CHECK: cleanupret from %[[padtoken]] unwind to caller
 
 // CHECK-LABEL: define internal void @"?fin$0@0@nested___finally___finally@@"(i8 noundef %abnormal_termination, ptr noundef %frame_pointer)
 // CHECK: ret void
 
 // CHECK-LABEL: define internal void @"?fin$1@0@nested___finally___finally@@"(i8 noundef %abnormal_termination, ptr noundef %frame_pointer)
 // CHECK: call void @g()
 // CHECK: unreachable

clang/test/CodeGen/windows-seh-return-finally.c

+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py UTC_ARGS: --include-generated-funcs --version 2
+// RUN: %clang_cc1 -triple x86_64-windows -fcxx-exceptions -fexceptions -fms-extensions -S -emit-llvm %s -o - | FileCheck %s
+
+void f1(), f2(), f3();
+void f(int z) {
+  __try {
+    f1();
+  } __finally {
+    f2();
+    if (z)
+      return;
+    f3();
+  }
+}
+// CHECK-LABEL: define dso_local void @f
+// CHECK-SAME: (i32 noundef [[Z:%.*]]) #[[ATTR0:[0-9]+]] personality ptr @__C_specific_handler {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    [[Z_ADDR:%.*]] = alloca i32, align 4
+// CHECK-NEXT:    [[RETNOW:%.*]] = alloca i8, align 1
+// CHECK-NEXT:    call void (...) @llvm.localescape(ptr [[Z_ADDR]], ptr [[RETNOW]])
+// CHECK-NEXT:    store i32 [[Z]], ptr [[Z_ADDR]], align 4
+// CHECK-NEXT:    store i8 0, ptr [[RETNOW]], align 1
+// CHECK-NEXT:    invoke void @f1() #[[ATTR6:[0-9]+]]
+// CHECK-NEXT:    to label [[INVOKE_CONT:%.*]] unwind label [[EHCLEANUP:%.*]]
+// CHECK:       invoke.cont:
+// CHECK-NEXT:    [[TMP0:%.*]] = call ptr @llvm.localaddress()
+// CHECK-NEXT:    invoke void @"?fin$0@0@f@@"(i8 noundef 0, ptr noundef [[TMP0]])
+// CHECK-NEXT:    to label [[INVOKE_CONT1:%.*]] unwind label [[CATCH_DISPATCH:%.*]]
+// CHECK:       invoke.cont1:
+// CHECK-NEXT:    br label [[__FINALLY_CONT:%.*]]
+// CHECK:       __finally.cont:
+// CHECK-NEXT:    [[TMP1:%.*]] = load i8, ptr [[RETNOW]], align 1
+// CHECK-NEXT:    [[TMP2:%.*]] = icmp ne i8 [[TMP1]], 0
+// CHECK-NEXT:    br i1 [[TMP2]], label [[IF_THEN4:%.*]], label [[IF_END5:%.*]]
+// CHECK:       ehcleanup:
+// CHECK-NEXT:    [[TMP3:%.*]] = cleanuppad within none []
+// CHECK-NEXT:    [[TMP4:%.*]] = call ptr @llvm.localaddress()
+// CHECK-NEXT:    invoke void @"?fin$0@0@f@@"(i8 noundef 1, ptr noundef [[TMP4]]) [ "funclet"(token [[TMP3]]) ]
+// CHECK-NEXT:    to label [[INVOKE_CONT2:%.*]] unwind label [[CATCH_DISPATCH]]
+// CHECK:       invoke.cont2:
+// CHECK-NEXT:    [[TMP5:%.*]] = load i8, ptr [[RETNOW]], align 1
+// CHECK-NEXT:    [[TMP6:%.*]] = icmp ne i8 [[TMP5]], 0
+// CHECK-NEXT:    br i1 [[TMP6]], label [[IF_THEN:%.*]], label [[IF_END:%.*]]
+// CHECK:       if.then:
+// CHECK-NEXT:    invoke void @llvm.seh.localunwind()
+// CHECK-NEXT:    to label [[INVOKE_CONT3:%.*]] unwind label [[CATCH_DISPATCH]]
+// CHECK:       invoke.cont3:
+// CHECK-NEXT:    unreachable
+// CHECK:       if.end:
+// CHECK-NEXT:    cleanupret from [[TMP3]] unwind label [[CATCH_DISPATCH]]
+// CHECK:       catch.dispatch:
+// CHECK-NEXT:    [[TMP7:%.*]] = catchswitch within none [label %__except.ret] unwind to caller
+// CHECK:       __except.ret:
+// CHECK-NEXT:    [[TMP8:%.*]] = catchpad within [[TMP7]] [ptr @__IsLocalUnwind]
+// CHECK-NEXT:    catchret from [[TMP8]] to label [[IF_THEN4]]
+// CHECK:       if.then4:
+// CHECK-NEXT:    br label [[IF_END5]]
+// CHECK:       if.end5:
+// CHECK-NEXT:    ret void
+//
+//
+// CHECK-LABEL: define internal void @"?fin$0@0@f@@"
+// CHECK-SAME: (i8 noundef [[ABNORMAL_TERMINATION:%.*]], ptr noundef [[FRAME_POINTER:%.*]]) #[[ATTR1:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    [[FRAME_POINTER_ADDR:%.*]] = alloca ptr, align 8
+// CHECK-NEXT:    [[ABNORMAL_TERMINATION_ADDR:%.*]] = alloca i8, align 1
+// CHECK-NEXT:    [[Z_ADDR:%.*]] = call ptr @llvm.localrecover(ptr @f, ptr [[FRAME_POINTER]], i32 0)
+// CHECK-NEXT:    [[RETNOW:%.*]] = call ptr @llvm.localrecover(ptr @f, ptr [[FRAME_POINTER]], i32 1)
+// CHECK-NEXT:    store ptr [[FRAME_POINTER]], ptr [[FRAME_POINTER_ADDR]], align 8
+// CHECK-NEXT:    store i8 [[ABNORMAL_TERMINATION]], ptr [[ABNORMAL_TERMINATION_ADDR]], align 1
+// CHECK-NEXT:    call void @f2()
+// CHECK-NEXT:    [[TMP0:%.*]] = load i32, ptr [[Z_ADDR]], align 4
+// CHECK-NEXT:    [[TOBOOL:%.*]] = icmp ne i32 [[TMP0]], 0
+// CHECK-NEXT:    br i1 [[TOBOOL]], label [[IF_THEN:%.*]], label [[IF_END:%.*]]
+// CHECK:       if.then:
+// CHECK-NEXT:    store i8 1, ptr [[RETNOW]], align 1
+// CHECK-NEXT:    br label [[RETURN:%.*]]
+// CHECK:       if.end:
+// CHECK-NEXT:    call void @f3()
+// CHECK-NEXT:    br label [[RETURN]]
+// CHECK:       return:
+// CHECK-NEXT:    ret void
+//

llvm/include/llvm/IR/Intrinsics.td

 //    caused by HW exceptions under option -EHa.
 //  calls/invokes to these intrinsics will be discarded during a codegen pass
 //   after EH tables are generated
 def int_seh_try_begin : Intrinsic<[], [], [IntrWriteMem, IntrWillReturn]>;
 def int_seh_try_end : Intrinsic<[], [], [IntrWriteMem, IntrWillReturn]>;
 def int_seh_scope_begin : Intrinsic<[], [], [IntrNoMem]>;
 def int_seh_scope_end : Intrinsic<[], [], [IntrNoMem]>;
 
+// Call _local_unwind to unwind to a local catchpad.
+def int_seh_localunwind : Intrinsic<[], [], [IntrNoReturn]>;
+
 // Note: we treat stacksave/stackrestore as writemem because we don't otherwise
 // model their dependencies on allocas.
 def int_stacksave     : DefaultAttrsIntrinsic<[llvm_ptr_ty]>,
                         ClangBuiltin<"__builtin_stack_save">;
 def int_stackrestore  : DefaultAttrsIntrinsic<[], [llvm_ptr_ty]>,
                         ClangBuiltin<"__builtin_stack_restore">;
 
 def int_get_dynamic_area_offset : DefaultAttrsIntrinsic<[llvm_anyint_ty]>;

llvm/lib/CodeGen/AsmPrinter/WinException.cpp

void WinException::emitCSpecificHandlerTable(const MachineFunction *MF) {
   const MCExpr *LabelDiff = getOffset(TableEnd, TableBegin);
   const MCExpr *EntrySize = MCConstantExpr::create(16, Ctx);
   const MCExpr *EntryCount = MCBinaryExpr::createDiv(LabelDiff, EntrySize, Ctx);
   AddComment("Number of call sites");
   OS.emitValue(EntryCount, 4);
 
   OS.emitLabel(TableBegin);
 
   // Iterate over all the invoke try ranges. Unlike MSVC, LLVM currently only
   // models exceptions from invokes. LLVM also allows arbitrary reordering of
   // the code, so our tables end up looking a bit different. Rather than
   // trying to match MSVC's tables exactly, we emit a denormalized table.  For
   // each range of invokes in the same state, we emit table entries for all
   // the actions that would be taken in that state. This means our tables are
   // slightly bigger, which is OK.
   const MCSymbol *LastStartLabel = nullptr;
   int LastEHState = -1;
   // Break out before we enter into a finally funclet.
   // FIXME: We need to emit separate EH tables for cleanups.
   MachineFunction::const_iterator End = MF->end();
   MachineFunction::const_iterator Stop = std::next(MF->begin());
   while (Stop != End && !Stop->isEHFuncletEntry())
     ++Stop;
   for (const auto &StateChange :
        InvokeStateChangeIterator::range(FuncInfo, MF->begin(), Stop)) {
     // Emit all the actions for the state we just transitioned out of
     // if it was not the null state
     if (LastEHState != -1)
       emitSEHActionsForRange(FuncInfo, LastStartLabel,
                              StateChange.PreviousEndLabel, LastEHState);
     LastStartLabel = StateChange.NewStartLabel;
     LastEHState = StateChange.NewState;
   }
+  for (auto Entry : FuncInfo.SEHUnwindMap) {
+    if (!Entry.IsFinally && Entry.ToState != -1) {
+      // Mark up the destination of _local_unwind so it doesn't unwind
+      // too far.
+      //
+      // FIXME: Can this overlap with the EH_LABEL for an invoke?
+      auto *Handler = Entry.Handler.get<MachineBasicBlock *>();
+      const MCSymbol *Begin = Handler->getSymbol();
+      emitSEHActionsForRange(FuncInfo, Begin, Begin, Entry.ToState);
+    }
+  }
 
   OS.emitLabel(TableEnd);
 }
 
 void WinException::emitSEHActionsForRange(const WinEHFuncInfo &FuncInfo,
                                           const MCSymbol *BeginLabel,
                                           const MCSymbol *EndLabel, int State) {
   auto &OS = *Asm->OutStreamer;

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

case Intrinsic::wasm_rethrow: {
       const TargetLowering &TLI = DAG.getTargetLoweringInfo();
       Ops.push_back(
           DAG.getTargetConstant(Intrinsic::wasm_rethrow, getCurSDLoc(),
                                 TLI.getPointerTy(DAG.getDataLayout())));
       SDVTList VTs = DAG.getVTList(ArrayRef<EVT>({MVT::Other})); // outchain
       DAG.setRoot(DAG.getNode(ISD::INTRINSIC_VOID, getCurSDLoc(), VTs, Ops));
       break;
     }
+    case Intrinsic::seh_localunwind: {
+      if (!isa<CatchSwitchInst>(EHPadBB->getTerminator())) {
+        report_fatal_error("localunwind doesn't point to catchswitch");
+      }
+      auto *CatchSwitch = cast<CatchSwitchInst>(EHPadBB->getTerminator());
+      if (CatchSwitch->getNumHandlers() == 0) {
+        report_fatal_error("catchswitch with no handler");
+      }
+
+      const TargetLowering &TLI = DAG.getTargetLoweringInfo();
+      TargetLowering::ArgListEntry SP, DestBB;
+      Type *PtrTy = PointerType::getInt8PtrTy(*DAG.getContext());
+      EVT PtrVT = TLI.getPointerTy(DAG.getDataLayout());
+      SP.Node = DAG.getNode(ISD::FRAMEADDR, getCurSDLoc(), PtrVT,
+                            DAG.getIntPtrConstant(0, getCurSDLoc()));
+      SP.Ty = PtrTy;
+      // FIXME: Using a BlockAddress here is messy, but there isn't any
+      // node that directly represents taking the address of an MBB.
+      BasicBlock *HandlerBB =
+          const_cast<BasicBlock *>(*CatchSwitch->handler_begin());
+      MachineBasicBlock *HandlerMBB = FuncInfo.MBBMap[HandlerBB];
+      HandlerMBB->setMachineBlockAddressTaken();
+      HandlerMBB->setAddressTakenIRBlock(HandlerBB);
+      DestBB.Node = DAG.getBlockAddress(BlockAddress::get(HandlerBB), PtrVT);
+      DestBB.Ty = PtrTy;
+      TargetLowering::ArgListTy Args{SP, DestBB};
+
+      SDValue Callee = DAG.getExternalSymbol("_local_unwind", PtrVT);
+      TargetLowering::CallLoweringInfo CLI(DAG);
+      CLI.setDebugLoc(getCurSDLoc())
+          .setChain(getRoot())
+          .setCallee(CallingConv::C, Type::getVoidTy(*DAG.getContext()), Callee,
+                     std::move(Args))
+          .setNoReturn();
+      CLI.CB = &I;
+      lowerInvokable(CLI, EHPadBB);
+      break;
+    }
     }
   } else if (I.countOperandBundlesOfType(LLVMContext::OB_deopt)) {
     // Currently we do not lower any intrinsic calls with deopt operand bundles.
     // Eventually we will support lowering the @llvm.experimental.deoptimize
     // intrinsic, and right now there are no plans to support other intrinsics
     // with deopt state.
     LowerCallSiteWithDeoptBundle(&I, getValue(Callee), EHPadBB);
   } else {

llvm/lib/CodeGen/WinEHPrepare.cpp

if (auto *CatchSwitch = dyn_cast<CatchSwitchInst>(FirstNonPHI)) {
     const auto *CatchPad =
         cast<CatchPadInst>((*CatchSwitch->handler_begin())->getFirstNonPHI());
     const BasicBlock *CatchPadBB = CatchPad->getParent();
     const Constant *FilterOrNull =
         cast<Constant>(CatchPad->getArgOperand(0)->stripPointerCasts());
     const Function *Filter = dyn_cast<Function>(FilterOrNull);
     assert((Filter || FilterOrNull->isNullValue()) &&
            "unexpected filter value");
+    // Filters named __IsLocalUnwind are treated specially: we want to catch
+    // unwinds from _local_unwind, but not catchrets in the same funclet.
+    // (They both need to point at the same catchswitch to pass the verifier
+    // checks for nesting.) To make this work, we mess with the state
+    // numbering: the "parent" of any cleanupret pointing to this catchpad is
+    // actually this catchpad's parent.
+    //
+    // Note that _local_unwind looks for unwind table entries for the
+    // catchpad; if there aren't any, it assumes the catchpad doesn't have a
+    // parent.
+    bool IsLocalUnwind =
+        Filter && Filter->getName().startswith("__IsLocalUnwind");
+    if (IsLocalUnwind)
+      Filter = nullptr;
     int TryState = addSEHExcept(FuncInfo, ParentState, Filter, CatchPadBB);
 
     // Everything in the __try block uses TryState as its parent state.
     FuncInfo.EHPadStateMap[CatchSwitch] = TryState;
     FuncInfo.EHPadStateMap[CatchPad] = TryState;
     LLVM_DEBUG(dbgs() << "Assigning state #" << TryState << " to BB "
                       << CatchPadBB->getName() << '\n');
     for (const BasicBlock *PredBlock : predecessors(BB))
       if ((PredBlock = getEHPadFromPredecessor(PredBlock,
                                                CatchSwitch->getParentPad())))
         calculateSEHStateNumbers(FuncInfo, PredBlock->getFirstNonPHI(),
-                                 TryState);
+                                 IsLocalUnwind ? ParentState : TryState);
 
     // Everything in the __except block unwinds to ParentState, just like code
     // outside the __try.
     for (const User *U : CatchPad->users()) {
       const auto *UserI = cast<Instruction>(U);
       if (auto *InnerCatchSwitch = dyn_cast<CatchSwitchInst>(UserI)) {
         BasicBlock *UnwindDest = InnerCatchSwitch->getUnwindDest();
         if (!UnwindDest || UnwindDest == CatchSwitch->getUnwindDest())

llvm/lib/IR/Verifier.cpp

if (Function *F = dyn_cast<Function>(I.getOperand(i))) {
                 F->getIntrinsicID() == Intrinsic::seh_scope_end ||
                 F->getIntrinsicID() == Intrinsic::coro_resume ||
                 F->getIntrinsicID() == Intrinsic::coro_destroy ||
                 F->getIntrinsicID() ==
                     Intrinsic::experimental_patchpoint_void ||
                 F->getIntrinsicID() == Intrinsic::experimental_patchpoint_i64 ||
                 F->getIntrinsicID() == Intrinsic::experimental_gc_statepoint ||
                 F->getIntrinsicID() == Intrinsic::wasm_rethrow ||
+                F->getIntrinsicID() == Intrinsic::seh_localunwind ||
                 IsAttachedCallOperand(F, CBI, i),
             "Cannot invoke an intrinsic other than donothing, patchpoint, "
             "statepoint, coro_resume, coro_destroy or clang.arc.attachedcall",
             &I);
       Check(F->getParent() == &M, "Referencing function in another module!", &I,
             &M, F, F->getParent());
     } else if (BasicBlock *OpBB = dyn_cast<BasicBlock>(I.getOperand(i))) {
       Check(OpBB->getParent() == BB->getParent(),

llvm/test/CodeGen/X86/seh-catchpad.ll

 ; CHECK-NEXT:         .long   .Ltmp2@IMGREL
 ; CHECK-NEXT:         .long   .Ltmp3@IMGREL+1
 ; CHECK-NEXT:         .long   "?filt$0@0@main@@"@IMGREL
 ; CHECK-NEXT:         .long   .LBB1_3@IMGREL
 ; CHECK-NEXT:         .long   .Ltmp6@IMGREL
 ; CHECK-NEXT:         .long   .Ltmp7@IMGREL+1
 ; CHECK-NEXT:         .long   "?filt$0@0@main@@"@IMGREL
 ; CHECK-NEXT:         .long   .LBB1_3@IMGREL
+; CHECK-NEXT:         .long   .LBB1_1@IMGREL
+; CHECK-NEXT:         .long   .LBB1_1@IMGREL+1
+; CHECK-NEXT:         .long   "?filt$0@0@main@@"@IMGREL
+; CHECK-NEXT:         .long   .LBB1_3@IMGREL
 ; CHECK-NEXT: .Llsda_end0:
 
 ; CHECK:         .text
 ; CHECK:         .seh_endproc
 
 ; CHECK: "?dtor$[[finbb]]@?0?main@4HA":
 ; CHECK: .seh_proc "?dtor$[[finbb]]@?0?main@4HA"
 ; CHECK-NOT:         .seh_handler

llvm/test/CodeGen/X86/seh-return-from-finally.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --include-generated-funcs --version 2
+; RUN: llc < %s | FileCheck %s
+target triple = "x86_64-unknown-windows-msvc19.20.0"
+
+; Function Attrs: noinline nounwind optnone uwtable
+define dso_local void @f(i32 noundef %z) #0 personality ptr @__C_specific_handler {
+entry:
+  %z.addr = alloca i32, align 4
+  %retnow = alloca i8, align 1
+  call void (...) @llvm.localescape(ptr %z.addr, ptr %retnow)
+  store i32 %z, ptr %z.addr, align 4
+  store i8 0, ptr %retnow, align 1
+  invoke void @f1() #6
+          to label %invoke.cont unwind label %ehcleanup
+
+invoke.cont:                                      ; preds = %entry
+  %0 = call ptr @llvm.localaddress()
+  invoke void @"?fin$0@0@f@@"(i8 noundef 0, ptr noundef %0)
+          to label %invoke.cont1 unwind label %catch.dispatch
+
+invoke.cont1:                                     ; preds = %invoke.cont
+  br label %__finally.cont
+
+__finally.cont:                                   ; preds = %invoke.cont1
+  %1 = load i8, ptr %retnow, align 1
+  %2 = icmp ne i8 %1, 0
+  br i1 %2, label %if.then4, label %if.end5
+
+ehcleanup:                                        ; preds = %entry
+  %3 = cleanuppad within none []
+  %4 = call ptr @llvm.localaddress()
+  invoke void @"?fin$0@0@f@@"(i8 noundef 1, ptr noundef %4) [ "funclet"(token %3) ]
+          to label %invoke.cont2 unwind label %catch.dispatch
+
+invoke.cont2:                                     ; preds = %ehcleanup
+  %5 = load i8, ptr %retnow, align 1
+  %6 = icmp ne i8 %5, 0
+  br i1 %6, label %if.then, label %if.end
+
+if.then:                                          ; preds = %invoke.cont2
+  invoke void @llvm.seh.localunwind()
+          to label %invoke.cont3 unwind label %catch.dispatch
+
+invoke.cont3:                                     ; preds = %if.then
+  unreachable
+
+if.end:                                           ; preds = %invoke.cont2
+  cleanupret from %3 unwind label %catch.dispatch
+
+catch.dispatch:                                   ; preds = %if.end, %if.then, %ehcleanup, %invoke.cont
+  %7 = catchswitch within none [label %__except.ret] unwind to caller
+
+__except.ret:                                     ; preds = %catch.dispatch
+  %8 = catchpad within %7 [ptr @__IsLocalUnwind]
+  catchret from %8 to label %if.then4
+
+if.then4:                                         ; preds = %__except.ret, %__finally.cont
+  br label %if.end5
+
+if.end5:                                          ; preds = %if.then4, %__finally.cont
+  ret void
+}
+
+declare extern_weak void @__IsLocalUnwind(ptr, ptr)
+
+; Function Attrs: noinline nounwind uwtable
+define internal void @"?fin$0@0@f@@"(i8 noundef %abnormal_termination, ptr noundef %frame_pointer) #1 {
+entry:
+  %frame_pointer.addr = alloca ptr, align 8
+  %abnormal_termination.addr = alloca i8, align 1
+  %z.addr = call ptr @llvm.localrecover(ptr @f, ptr %frame_pointer, i32 0)
+  %retnow = call ptr @llvm.localrecover(ptr @f, ptr %frame_pointer, i32 1)
+  store ptr %frame_pointer, ptr %frame_pointer.addr, align 8
+  store i8 %abnormal_termination, ptr %abnormal_termination.addr, align 1
+  call void @f2()
+  %0 = load i32, ptr %z.addr, align 4
+  %tobool = icmp ne i32 %0, 0
+  br i1 %tobool, label %if.then, label %if.end
+
+if.then:                                          ; preds = %entry
+  store i8 1, ptr %retnow, align 1
+  br label %return
+
+if.end:                                           ; preds = %entry
+  call void @f3()
+  br label %return
+
+return:                                           ; preds = %if.end, %if.then
+  ret void
+}
+
+; Function Attrs: nocallback nofree nosync nounwind willreturn memory(none)
+declare ptr @llvm.localrecover(ptr, ptr, i32 immarg) #2
+
+declare dso_local void @f2(...) #3
+
+declare dso_local void @f3(...) #3
+
+declare dso_local void @f1(...) #3
+
+declare dso_local i32 @__C_specific_handler(...)
+
+; Function Attrs: nocallback nofree nosync nounwind willreturn memory(none)
+declare ptr @llvm.localaddress() #2
+
+; Function Attrs: noreturn nounwind
+declare void @llvm.seh.localunwind() #4
+
+; Function Attrs: nocallback nofree nosync nounwind willreturn
+declare void @llvm.localescape(...) #5
+
+attributes #0 = { noinline nounwind optnone uwtable "min-legal-vector-width"="0" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+cx8,+fxsr,+mmx,+sse,+sse2,+x87" "tune-cpu"="generic" }
+attributes #1 = { noinline nounwind uwtable "min-legal-vector-width"="0" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+cx8,+fxsr,+mmx,+sse,+sse2,+x87" "tune-cpu"="generic" }
+attributes #2 = { nocallback nofree nosync nounwind willreturn memory(none) }
+attributes #3 = { "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+cx8,+fxsr,+mmx,+sse,+sse2,+x87" "tune-cpu"="generic" }
+attributes #4 = { noreturn nounwind }
+attributes #5 = { nocallback nofree nosync nounwind willreturn }
+attributes #6 = { noinline }
+; CHECK-LABEL: f:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    pushq %rbp
+; CHECK-NEXT:    .seh_pushreg %rbp
+; CHECK-NEXT:    subq $48, %rsp
+; CHECK-NEXT:    .seh_stackalloc 48
+; CHECK-NEXT:    leaq {{[0-9]+}}(%rsp), %rbp
+; CHECK-NEXT:    .seh_setframe %rbp, 48
+; CHECK-NEXT:    .seh_endprologue
+; CHECK-NEXT:  .set .Lf$frame_escape_0, -8
+; CHECK-NEXT:  .set .Lf$frame_escape_1, -1
+; CHECK-NEXT:    movl %ecx, -8(%rbp)
+; CHECK-NEXT:    movb $0, -1(%rbp)
+; CHECK-NEXT:  .Ltmp0:
+; CHECK-NEXT:    callq f1
+; CHECK-NEXT:  .Ltmp1:
+; CHECK-NEXT:    jmp .LBB0_1
+; CHECK-NEXT:  .LBB0_1: # %invoke.cont
+; CHECK-NEXT:  .Ltmp6:
+; CHECK-NEXT:    xorl %ecx, %ecx
+; CHECK-NEXT:    movq %rbp, %rdx
+; CHECK-NEXT:    callq "?fin$0@0@f@@"
+; CHECK-NEXT:  .Ltmp7:
+; CHECK-NEXT:    jmp .LBB0_2
+; CHECK-NEXT:  .LBB0_2: # %invoke.cont1
+; CHECK-NEXT:    cmpb $0, -1(%rbp)
+; CHECK-NEXT:    jne .LBB0_9
+; CHECK-NEXT:    jmp .LBB0_10
+; CHECK-NEXT:  .Ltmp8: # Block address taken
+; CHECK-NEXT:  .LBB0_8: # %__except.ret
+; CHECK-NEXT:    jmp .LBB0_9
+; CHECK-NEXT:  .LBB0_9: # %if.then4
+; CHECK-NEXT:  $ehgcr_0_9:
+; CHECK-NEXT:    jmp .LBB0_10
+; CHECK-NEXT:  .LBB0_10: # %if.end5
+; CHECK-NEXT:    addq $48, %rsp
+; CHECK-NEXT:    popq %rbp
+; CHECK-NEXT:    retq
+; CHECK-NEXT:    .seh_handlerdata
+; CHECK-NEXT:  .set .Lf$parent_frame_offset, 48
+; CHECK-NEXT:    .long (.Llsda_end0-.Llsda_begin0)/16 # Number of call sites
+; CHECK-NEXT:  .Llsda_begin0:
+; CHECK-NEXT:    .long .Ltmp0@IMGREL # LabelStart
+; CHECK-NEXT:    .long .Ltmp1@IMGREL+1 # LabelEnd
+; CHECK-NEXT:    .long "?dtor$3@?0?f@4HA"@IMGREL # FinallyFunclet
+; CHECK-NEXT:    .long 0 # Null
+; CHECK-NEXT:    .long .Ltmp6@IMGREL # LabelStart
+; CHECK-NEXT:    .long .Ltmp7@IMGREL+1 # LabelEnd
+; CHECK-NEXT:    .long 1 # CatchAll
+; CHECK-NEXT:    .long .LBB0_8@IMGREL # ExceptionHandler
+; CHECK-NEXT:  .Llsda_end0:
+; CHECK-NEXT:    .text
+; CHECK-NEXT:    .seh_endproc
+; CHECK-NEXT:    .def "?dtor$3@?0?f@4HA";
+; CHECK-NEXT:    .scl 3;
+; CHECK-NEXT:    .type 32;
+; CHECK-NEXT:    .endef
+; CHECK-NEXT:    .p2align 4, 0x90
+; CHECK-NEXT:  "?dtor$3@?0?f@4HA":
+; CHECK-NEXT:  .seh_proc "?dtor$3@?0?f@4HA"
+; CHECK-NEXT:  .LBB0_3: # %ehcleanup
+; CHECK-NEXT:    movq %rdx, {{[0-9]+}}(%rsp)
+; CHECK-NEXT:    pushq %rbp
+; CHECK-NEXT:    .seh_pushreg %rbp
+; CHECK-NEXT:    subq $32, %rsp
+; CHECK-NEXT:    .seh_stackalloc 32
+; CHECK-NEXT:    leaq 48(%rdx), %rbp
+; CHECK-NEXT:    .seh_endprologue
+; CHECK-NEXT:  .Ltmp2:
+; CHECK-NEXT:    movb $1, %cl
+; CHECK-NEXT:    movq %rbp, %rdx
+; CHECK-NEXT:    callq "?fin$0@0@f@@"
+; CHECK-NEXT:  .Ltmp3:
+; CHECK-NEXT:    jmp .LBB0_4
+; CHECK-NEXT:  .LBB0_4: # %invoke.cont2
+; CHECK-NEXT:    cmpb $0, -1(%rbp)
+; CHECK-NEXT:    je .LBB0_7
+; CHECK-NEXT:  # %bb.5: # %if.then
+; CHECK-NEXT:  .Ltmp4:
+; CHECK-NEXT:    leaq -48(%rbp), %rcx
+; CHECK-NEXT:    leaq .Ltmp8(%rip), %rdx
+; CHECK-NEXT:    callq _local_unwind
+; CHECK-NEXT:  .Ltmp5:
+; CHECK-NEXT:    jmp .LBB0_6
+; CHECK-NEXT:  .LBB0_6: # %invoke.cont3
+; CHECK-NEXT:  .LBB0_7: # %if.end
+; CHECK-NEXT:    addq $32, %rsp
+; CHECK-NEXT:    popq %rbp
+; CHECK-NEXT:    retq # CLEANUPRET
+; CHECK-NEXT:  .Lfunc_end0:
+; CHECK-NEXT:    .seh_handlerdata
+; CHECK-NEXT:    .text
+; CHECK-NEXT:    .seh_endproc
+;
+; CHECK-LABEL: "?fin$0@0@f@@":
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    pushq %rsi
+; CHECK-NEXT:    .seh_pushreg %rsi
+; CHECK-NEXT:    subq $48, %rsp
+; CHECK-NEXT:    .seh_stackalloc 48
+; CHECK-NEXT:    .seh_endprologue
+; CHECK-NEXT:    movq %rdx, %rsi
+; CHECK-NEXT:    movq %rdx, {{[0-9]+}}(%rsp)
+; CHECK-NEXT:    movb %cl, {{[0-9]+}}(%rsp)
+; CHECK-NEXT:    callq f2
+; CHECK-NEXT:    cmpl $0, .Lf$frame_escape_0(%rsi)
+; CHECK-NEXT:    je .LBB1_2
+; CHECK-NEXT:  # %bb.1: # %if.then
+; CHECK-NEXT:    leaq .Lf$frame_escape_1(%rsi), %rax
+; CHECK-NEXT:    movb $1, (%rax)
+; CHECK-NEXT:    jmp .LBB1_3
+; CHECK-NEXT:  .LBB1_2: # %if.end
+; CHECK-NEXT:    callq f3
+; CHECK-NEXT:  .LBB1_3: # %return
+; CHECK-NEXT:    nop
+; CHECK-NEXT:    addq $48, %rsp
+; CHECK-NEXT:    popq %rsi
+; CHECK-NEXT:    retq
+; CHECK-NEXT:    .seh_endproc

llvm/test/CodeGen/X86/seh-safe-div.ll

 ; CHECK-NEXT: .long .Ltmp0@IMGREL
 ; CHECK-NEXT: .long .Ltmp1@IMGREL+1
 ; CHECK-NEXT: .long safe_div_filt0@IMGREL
 ; CHECK-NEXT: .long [[handler0]]@IMGREL
 ; CHECK-NEXT: .long .Ltmp0@IMGREL
 ; CHECK-NEXT: .long .Ltmp1@IMGREL+1
 ; CHECK-NEXT: .long safe_div_filt1@IMGREL
 ; CHECK-NEXT: .long [[handler1]]@IMGREL
+; CHECK-NEXT: .long .LBB0_1@IMGREL
+; CHECK-NEXT: .long .LBB0_1@IMGREL+1
+; CHECK-NEXT: .long safe_div_filt1@IMGREL
+; CHECK-NEXT: .long .LBB0_2@IMGREL
 ; CHECK-NEXT: .Llsda_end0:
 ; CHECK: .text
 ; CHECK: .seh_endproc
 
 define void @try_body(ptr %r, ptr %n, ptr %d) {
 entry:
   %0 = load i32, ptr %n, align 4
   %1 = load i32, ptr %d, align 4

llvm/utils/UpdateTestChecks/common.py

 CHECK_RE = re.compile(r'^\s*(?://|[;#])\s*([^:]+?)(?:-NEXT|-NOT|-DAG|-LABEL|-SAME|-EMPTY)?:')
 
 UTC_ARGS_KEY = 'UTC_ARGS:'
 UTC_ARGS_CMD = re.compile(r'.*' + UTC_ARGS_KEY + '\s*(?P<cmd>.*)\s*$')
 UTC_ADVERT = 'NOTE: Assertions have been autogenerated by '
 UNUSED_NOTE = 'NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:'
 
 OPT_FUNCTION_RE = re.compile(
-    r'^(\s*;\s*Function\sAttrs:\s(?P<attrs>[\w\s():,]+?))?\s*define\s+(?P<funcdef_attrs_and_ret>[^@]*)@(?P<func>[\w.$-]+?)\s*'
+    r'^(\s*;\s*Function\sAttrs:\s(?P<attrs>[\w\s():,]+?))?\s*define\s+(?P<funcdef_attrs_and_ret>[^@]*)@(?P<func>([\w.$-]+?|"[\w.$-?@]+?"))\s*'
     r'(?P<args_and_sig>\((\)|(.*?[\w.-]+?)\))[^{]*\{)\n(?P<body>.*?)^\}$',
     flags=(re.M | re.S))
 
 ANALYZE_FUNCTION_RE = re.compile(
     r'^\s*\'(?P<analysis>[\w\s-]+?)\'\s+for\s+function\s+\'(?P<func>[\w.$-]+?)\':'
     r'\s*\n(?P<body>.*)$',
     flags=(re.X | re.S))