[WebAssembly] Handle EH terminate pads for cleanup

[aheejin]

Summary

Terminate pads, cleanup pads with __clang_call_terminate call, have
catch instruction in them because __clang_call_terminate takes an
exception pointer. But these terminate pads should be reached also in
case of foreign exception. So this pass attaches an additional
catch_all BB after every terminate pad BB, with a call to
std::terminate.

[tlively]

LGTM besides that one comment!

llvm/lib/Target/WebAssembly/WebAssemblyHandleEHTerminatePads.cpp

126

Is it possible for one BB to contain multiple calls to __clang_call_terminate? If so, that BB would end up with multiple catch_alls in the current code. Would it be worth asserting that this doesn't happen?

[aheejin]

llvm/lib/Target/WebAssembly/WebAssemblyHandleEHTerminatePads.cpp

126

We ensure that doesn't happen in LateEHPrepare, thanks to you.

If we want to put some assert here it wouldn't be a single line assert, but we need to keep a set of something to see if there is no __clang_call_terminate calls with an EH pad that has already been added, which we are doing in LateEHPrepare anyway. I guess we don't need to duplicate that logic just for assertion..?

[tlively]

llvm/lib/Target/WebAssembly/WebAssemblyHandleEHTerminatePads.cpp

126

Oh, heh, I forgot about that. No need to duplicate that check here, then :)

llvm/lib/Target/WebAssembly/CMakeLists.txt

 add_llvm_target(WebAssemblyCodeGen
   WebAssemblyAddMissingPrototypes.cpp
   WebAssemblyArgumentMove.cpp
   WebAssemblyAsmPrinter.cpp
   WebAssemblyCFGStackify.cpp
   WebAssemblyCFGSort.cpp
   WebAssemblyDebugFixup.cpp
   WebAssemblyDebugValueManager.cpp
+  WebAssemblyHandleEHTerminatePads.cpp
   WebAssemblyLateEHPrepare.cpp
   WebAssemblyExceptionInfo.cpp
   WebAssemblyExplicitLocals.cpp
   WebAssemblyFastISel.cpp
   WebAssemblyFixBrTableDefaults.cpp
   WebAssemblyFixIrreducibleControlFlow.cpp
   WebAssemblyFixFunctionBitcasts.cpp
   WebAssemblyFrameLowering.cpp

llvm/lib/Target/WebAssembly/WebAssembly.h

 FunctionPass *createWebAssemblyMemIntrinsicResults();
 FunctionPass *createWebAssemblyRegStackify();
 FunctionPass *createWebAssemblyRegColoring();
 FunctionPass *createWebAssemblyFixBrTableDefaults();
 FunctionPass *createWebAssemblyFixIrreducibleControlFlow();
 FunctionPass *createWebAssemblyLateEHPrepare();
 FunctionPass *createWebAssemblyCFGSort();
 FunctionPass *createWebAssemblyCFGStackify();
+FunctionPass *createWebAssemblyHandleEHTerminatePads();
 FunctionPass *createWebAssemblyExplicitLocals();
 FunctionPass *createWebAssemblyLowerBrUnless();
 FunctionPass *createWebAssemblyRegNumbering();
 FunctionPass *createWebAssemblyDebugFixup();
 FunctionPass *createWebAssemblyPeephole();
 
 // PassRegistry initialization declarations.
 void initializeWebAssemblyAddMissingPrototypesPass(PassRegistry &);

 void initializeWebAssemblyRegStackifyPass(PassRegistry &);
 void initializeWebAssemblyRegColoringPass(PassRegistry &);
 void initializeWebAssemblyFixBrTableDefaultsPass(PassRegistry &);
 void initializeWebAssemblyFixIrreducibleControlFlowPass(PassRegistry &);
 void initializeWebAssemblyLateEHPreparePass(PassRegistry &);
 void initializeWebAssemblyExceptionInfoPass(PassRegistry &);
 void initializeWebAssemblyCFGSortPass(PassRegistry &);
 void initializeWebAssemblyCFGStackifyPass(PassRegistry &);
+void initializeWebAssemblyHandleEHTerminatePadsPass(PassRegistry &);
 void initializeWebAssemblyExplicitLocalsPass(PassRegistry &);
 void initializeWebAssemblyLowerBrUnlessPass(PassRegistry &);
 void initializeWebAssemblyRegNumberingPass(PassRegistry &);
 void initializeWebAssemblyDebugFixupPass(PassRegistry &);
 void initializeWebAssemblyPeepholePass(PassRegistry &);
 
 namespace WebAssembly {
 enum TargetIndex {

llvm/lib/Target/WebAssembly/WebAssemblyHandleEHTerminatePads.cpp

+// WebAssemblyHandleEHTerminatePads.cpp - WebAssembly Handle EH TerminatePads //
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// \brief Add catch_all blocks to terminate pads.
+///
+/// Terminate pads are cleanup pads with a __clang_call_terminate call. These
+/// are reached when an exception is thrown again in the middle of processing a
+/// thrown exception, to terminate the program. These are cleanup pads that
+/// should run regardless whether the thrown exception is a C++ exception or
+/// not.
+///
+/// Because __clang_call_terminate takes an exception pointer, and
+/// llvm.get.exception intrinsic is selected to 'catch' instruction in
+/// instruction selection, terminate pads have a catch instruction and are in
+/// this form after LateEHPrepare, even though they are cleanup pads:
+/// termpad:
+///   %exn = catch $__cpp_exception
+///   call @__clang_call_terminate(%exn)
+///   unreachable
+///
+/// This pass assumes LateEHPrepare ensured every terminate pad is a single
+/// BB.
+///
+/// __clang_call_terminate is a function generated by clang, in the form of
+/// void __clang_call_terminate(i8* %arg) {
+///   call @__cxa_begin_catch(%arg)
+///   call void @std::terminate()
+///   unreachable
+/// }
+///
+/// To make the terminate pads reachable when a foreign exception is thrown,
+/// this pass attaches an additional catch_all BB after this catch terminate pad
+/// BB, with a call to std::terminate, because foreign exceptions don't have a
+/// valid exception pointer to call __cxa_begin_catch with. So the code example
+/// becomes:
+/// termpad:
+///   %exn = catch $__cpp_exception
+///   call @__clang_call_terminate(%exn)
+///   unreachable
+/// termpad-catchall:
+///   catch_all
+///   call @std::terminate()
+///   unreachable
+///
+/// We do this at the very end of compilation pipeline, even after CFGStackify,
+/// because even though wasm spec allows multiple catch/catch_all blocks per a
+/// try instruction, it has been convenient to maintain the invariant so far
+/// that there has been only a single catch or catch_all attached to a try. This
+/// assumption makes ExceptionInfo generation and CFGStackify simpler, because
+/// we have been always able to assume an EH pad is an end of try block and a
+/// start of catch/catch_all block.
+//===----------------------------------------------------------------------===//
+
+#include "MCTargetDesc/WebAssemblyMCTargetDesc.h"
+#include "WebAssembly.h"
+#include "WebAssemblySubtarget.h"
+#include "WebAssemblyUtilities.h"
+#include "llvm/CodeGen/MachineModuleInfo.h"
+#include "llvm/MC/MCAsmInfo.h"
+#include "llvm/Target/TargetMachine.h"
+using namespace llvm;
+
+#define DEBUG_TYPE "wasm-handle-termpads"
+
+namespace {
+class WebAssemblyHandleEHTerminatePads final : public MachineFunctionPass {
+  StringRef getPassName() const override {
+    return "WebAssembly Handle EH Terminate Pads";
+  }
+
+  bool runOnMachineFunction(MachineFunction &MF) override;
+
+public:
+  static char ID; // Pass identification, replacement for typeid
+  WebAssemblyHandleEHTerminatePads() : MachineFunctionPass(ID) {}
+};
+} // end anonymous namespace
+
+char WebAssemblyHandleEHTerminatePads::ID = 0;
+INITIALIZE_PASS(WebAssemblyHandleEHTerminatePads, DEBUG_TYPE,
+                "WebAssembly Handle EH Terminate Pads", false, false)
+
+FunctionPass *llvm::createWebAssemblyHandleEHTerminatePads() {
+  return new WebAssemblyHandleEHTerminatePads();
+}
+
+bool WebAssemblyHandleEHTerminatePads::runOnMachineFunction(
+    MachineFunction &MF) {
+  LLVM_DEBUG(dbgs() << "********** Handle EH Terminate Pads **********\n"
+                       "********** Function: "
+                    << MF.getName() << '\n');
+
+  if (MF.getTarget().getMCAsmInfo()->getExceptionHandlingType() !=
+          ExceptionHandling::Wasm ||
+      !MF.getFunction().hasPersonalityFn())
+    return false;
+
+  const auto &TII = *MF.getSubtarget<WebAssemblySubtarget>().getInstrInfo();
+
+  // Find calls to __clang_call_terminate()
+  SmallVector<MachineInstr *, 8> ClangCallTerminateCalls;
+  for (auto &MBB : MF) {
+    for (auto &MI : MBB) {
+      if (MI.isCall()) {
+        const MachineOperand &CalleeOp = MI.getOperand(0);
+        if (CalleeOp.isGlobal() && CalleeOp.getGlobal()->getName() ==
+                                       WebAssembly::ClangCallTerminateFn)
+          ClangCallTerminateCalls.push_back(&MI);
+      }
+    }
+  }
+
+  if (ClangCallTerminateCalls.empty())
+    return false;
+
+  for (auto *Call : ClangCallTerminateCalls) {
+    // This should be an EH pad because LateEHPrepare ensures terminate pads are
+    // a single BB.
+    MachineBasicBlock *CatchBB = Call->getParent();
+    assert(CatchBB->isEHPad());
+
+    auto *CatchAllBB = MF.CreateMachineBasicBlock();
+    MF.insert(std::next(CatchBB->getIterator()), CatchAllBB);
+    CatchAllBB->setIsEHPad(true);
+    for (auto *Pred : CatchBB->predecessors())
+      Pred->addSuccessor(CatchAllBB);
+
+    // If the definition of __clang_call_terminate exists in the module, there
+    // should be a declaration of std::terminate within the same module, because
+    // __clang_call_terminate calls it.
+    const auto *StdTerminateFn =
+        MF.getMMI().getModule()->getNamedValue(WebAssembly::StdTerminateFn);
+    assert(StdTerminateFn && "std::terminate() does not exist in the module");
+
+    // Generate a BB in the form of:
+    //   catch_all
+    //   call @std::terminate
+    //   unreachable
+    BuildMI(CatchAllBB, Call->getDebugLoc(), TII.get(WebAssembly::CATCH_ALL));
+    BuildMI(CatchAllBB, Call->getDebugLoc(), TII.get(WebAssembly::CALL))
+        .addGlobalAddress(StdTerminateFn);
+    BuildMI(CatchAllBB, Call->getDebugLoc(), TII.get(WebAssembly::UNREACHABLE));
+  }
+
+  return true;
+}

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.cpp

extern "C" LLVM_EXTERNAL_VISIBILITY void LLVMInitializeWebAssemblyTarget() {
   initializeWebAssemblyMemIntrinsicResultsPass(PR);
   initializeWebAssemblyRegStackifyPass(PR);
   initializeWebAssemblyRegColoringPass(PR);
   initializeWebAssemblyFixIrreducibleControlFlowPass(PR);
   initializeWebAssemblyLateEHPreparePass(PR);
   initializeWebAssemblyExceptionInfoPass(PR);
   initializeWebAssemblyCFGSortPass(PR);
   initializeWebAssemblyCFGStackifyPass(PR);
+  initializeWebAssemblyHandleEHTerminatePadsPass(PR);
   initializeWebAssemblyExplicitLocalsPass(PR);
   initializeWebAssemblyLowerBrUnlessPass(PR);
   initializeWebAssemblyRegNumberingPass(PR);
   initializeWebAssemblyDebugFixupPass(PR);
   initializeWebAssemblyPeepholePass(PR);
 }
 
 //===----------------------------------------------------------------------===//
void WebAssemblyPassConfig::addPreEmitPass() {
 
   // Sort the blocks of the CFG into topological order, a prerequisite for
   // BLOCK and LOOP markers.
   addPass(createWebAssemblyCFGSort());
 
   // Insert BLOCK and LOOP markers.
   addPass(createWebAssemblyCFGStackify());
 
+  // Handle terminate pads for cleanups
+  if (TM->Options.ExceptionModel == ExceptionHandling::Wasm)
+    addPass(createWebAssemblyHandleEHTerminatePads());
+
   // Insert explicit local.get and local.set operators.
   if (!WasmDisableExplicitLocals)
     addPass(createWebAssemblyExplicitLocals());
 
   // Lower br_unless into br_if.
   addPass(createWebAssemblyLowerBrUnless());
 
   // Perform the very last peephole optimizations on the code.

llvm/test/CodeGen/WebAssembly/exception.ll

 ; CHECK:   try
 ; CHECK:     call      foo
 ; CHECK:   catch_all
 ; CHECK:     try
 ; CHECK:       call      __cxa_end_catch
 ; CHECK:     catch     $[[EXN:[0-9]+]]=, __cpp_exception
 ; CHECK:       call      __clang_call_terminate, $[[EXN]]
 ; CHECK:       unreachable
+; CHECK:     catch_all
+; CHECK:       call      _ZSt9terminatev
+; CHECK:       unreachable
 ; CHECK:     end_try
 ; CHECK:     rethrow
 ; CHECK:   end_try
 ; CHECK:   call      __cxa_end_catch
 ; CHECK: end_try
 define void @test_terminatepad() personality i8* bitcast (i32 (...)* @__gxx_wasm_personality_v0 to i8*) {
 entry:
   invoke void @foo()

 declare void @llvm.wasm.throw(i32, i8*)
 declare i8* @llvm.wasm.get.exception(token)
 declare i32 @llvm.wasm.get.ehselector(token)
 declare void @llvm.wasm.rethrow()
 declare i32 @llvm.eh.typeid.for(i8*)
 declare i8* @__cxa_begin_catch(i8*)
 declare void @__cxa_end_catch()
 declare void @__clang_call_terminate(i8*)
+declare void @_ZSt9terminatev()
 declare %struct.Temp* @_ZN4TempD2Ev(%struct.Temp* returned)
 
 ; CHECK: __cpp_exception:
 ; CHECK: .eventtype  __cpp_exception i32