[WebAssembly] Make returns variadic

[tlively]

Summary

This is necessary and sufficient to get simple cases of multiple
return working with multivalue enabled. More complex cases will
require block and loop signatures to be generalized to potentially be
type indices as well.

[sbc100]

llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp

1269

report_fatal_error so end users see this too?

[dschuff]

Should there be a limit on how many returns we are willing to return in a multi? or should we have no fallback to sret at all?

[dschuff]

llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp

1269

By this time we should have legalized everything, so this can be an assert because it should never happen?

[tlively]

In D68684#1702183, @dschuff wrote:

Should there be a limit on how many returns we are willing to return in a multi? or should we have no fallback to sret at all?

My current thinking is that such a limit should be left up to the ABI logic in the frontend and that in the backend we should turn all aggregate returns by value into multivalue returns with no limit. WDYT?

llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp

1269

@sbc100 is right that users should be able to see these errors. These asserts will be triggered when trying to emit nontrivial code with multivalue enabled.

[aheejin]

llvm/lib/Target/WebAssembly/WebAssemblyFastISel.cpp

1309

Why <=2? If multivalues are supported in fastisel later, do we still have that limit?

[tlively]

• Use report_fatal_error to report unimplemented functionality

[tlively]

• Bail instead of asserting in FastISel

[tlively]

I'll land this since it's just a WIP and not ready for users yet. I'm happy to discuss ABI limits and other considerations further and make changes in followups!

llvm/lib/Target/WebAssembly/WebAssemblyFastISel.cpp

1309

Oops, this comment should have been removed. I'm updating this to just bail out of FastISel in the case of multiple return for now. The proper condition to check is Ret->getNumOperands() > 1.

llvm/lib/Target/WebAssembly/MCTargetDesc/WebAssemblyInstPrinter.cpp

 void WebAssemblyInstPrinter::printInst(const MCInst *MI, raw_ostream &OS,
                                        StringRef Annot,
                                        const MCSubtargetInfo &STI) {
   // Print the instruction (this uses the AsmStrings from the .td files).
   printInstruction(MI, OS);
 
   // Print any additional variadic operands.
   const MCInstrDesc &Desc = MII.get(MI->getOpcode());
-  if (Desc.isVariadic())
+  if (Desc.isVariadic()) {
+    if (Desc.getNumOperands() == 0 && MI->getNumOperands() > 0)
+      OS << "\t";
     for (auto I = Desc.getNumOperands(), E = MI->getNumOperands(); I < E; ++I) {
       // FIXME: For CALL_INDIRECT_VOID, don't print a leading comma, because
       // we have an extra flags operand which is not currently printed, for
       // compatiblity reasons.
       if (I != 0 && ((MI->getOpcode() != WebAssembly::CALL_INDIRECT_VOID &&
                       MI->getOpcode() != WebAssembly::CALL_INDIRECT_VOID_S) ||
                      I != Desc.getNumOperands()))
         OS << ", ";
       printOperand(MI, I, OS);
     }
+  }
 
   // Print any added annotation.
   printAnnotation(OS, Annot);
 
   if (CommentStream) {
     // Observe any effects on the control flow stack, for use in annotating
     // control flow label references.
     unsigned Opc = MI->getOpcode();

llvm/lib/Target/WebAssembly/WebAssemblyAsmPrinter.cpp

void WebAssemblyAsmPrinter::EmitInstruction(const MachineInstr *MI) {
   case WebAssembly::ARGUMENT_v2i64_S:
   case WebAssembly::ARGUMENT_v4f32:
   case WebAssembly::ARGUMENT_v4f32_S:
   case WebAssembly::ARGUMENT_v2f64:
   case WebAssembly::ARGUMENT_v2f64_S:
     // These represent values which are live into the function entry, so there's
     // no instruction to emit.
     break;
-  case WebAssembly::FALLTHROUGH_RETURN_I32:
-  case WebAssembly::FALLTHROUGH_RETURN_I32_S:
-  case WebAssembly::FALLTHROUGH_RETURN_I64:
-  case WebAssembly::FALLTHROUGH_RETURN_I64_S:
-  case WebAssembly::FALLTHROUGH_RETURN_F32:
-  case WebAssembly::FALLTHROUGH_RETURN_F32_S:
-  case WebAssembly::FALLTHROUGH_RETURN_F64:
-  case WebAssembly::FALLTHROUGH_RETURN_F64_S:
-  case WebAssembly::FALLTHROUGH_RETURN_v16i8:
-  case WebAssembly::FALLTHROUGH_RETURN_v16i8_S:
-  case WebAssembly::FALLTHROUGH_RETURN_v8i16:
-  case WebAssembly::FALLTHROUGH_RETURN_v8i16_S:
-  case WebAssembly::FALLTHROUGH_RETURN_v4i32:
-  case WebAssembly::FALLTHROUGH_RETURN_v4i32_S:
-  case WebAssembly::FALLTHROUGH_RETURN_v2i64:
-  case WebAssembly::FALLTHROUGH_RETURN_v2i64_S:
-  case WebAssembly::FALLTHROUGH_RETURN_v4f32:
-  case WebAssembly::FALLTHROUGH_RETURN_v4f32_S:
-  case WebAssembly::FALLTHROUGH_RETURN_v2f64:
-  case WebAssembly::FALLTHROUGH_RETURN_v2f64_S: {
+  case WebAssembly::FALLTHROUGH_RETURN: {
     // These instructions represent the implicit return at the end of a
-    // function body. Always pops one value off the stack.
+    // function body.
     if (isVerbose()) {
-      OutStreamer->AddComment("fallthrough-return-value");
+      OutStreamer->AddComment("fallthrough-return");
       OutStreamer->AddBlankLine();
     }
     break;
   }
-  case WebAssembly::FALLTHROUGH_RETURN_VOID:
-  case WebAssembly::FALLTHROUGH_RETURN_VOID_S:
-    // This instruction represents the implicit return at the end of a
-    // function body with no return value.
-    if (isVerbose()) {
-      OutStreamer->AddComment("fallthrough-return-void");
-      OutStreamer->AddBlankLine();
-    }
-    break;
   case WebAssembly::COMPILER_FENCE:
     // This is a compiler barrier that prevents instruction reordering during
     // backend compilation, and should not be emitted.
     break;
   case WebAssembly::EXTRACT_EXCEPTION_I32:
   case WebAssembly::EXTRACT_EXCEPTION_I32_S:
     // These are pseudo instructions that simulates popping values from stack.
     // We print these only when we have -wasm-keep-registers on for assembly

llvm/lib/Target/WebAssembly/WebAssemblyCFGStackify.cpp

 /// because the function ends in an infinite loop or a noreturn call or similar,
 /// it isn't necessary to worry about the function return type at the end of
 /// the function, because it's never reached. However, in WebAssembly, blocks
 /// that end at the function end need to have a return type signature that
 /// matches the function signature, even though it's unreachable. This function
 /// checks for such cases and fixes up the signatures.
 void WebAssemblyCFGStackify::fixEndsAtEndOfFunction(MachineFunction &MF) {
   const auto &MFI = *MF.getInfo<WebAssemblyFunctionInfo>();
-  assert(MFI.getResults().size() <= 1);
 
   if (MFI.getResults().empty())
     return;
 
+  // TODO: Generalize from value types to function types for multivalue
   WebAssembly::ExprType RetType;
   switch (MFI.getResults().front().SimpleTy) {
   case MVT::i32:
     RetType = WebAssembly::ExprType::I32;
     break;
   case MVT::i64:
     RetType = WebAssembly::ExprType::I64;
     break;
default:
     llvm_unreachable("unexpected return type");
   }
 
   for (MachineBasicBlock &MBB : reverse(MF)) {
     for (MachineInstr &MI : reverse(MBB)) {
       if (MI.isPosition() || MI.isDebugInstr())
         continue;
       if (MI.getOpcode() == WebAssembly::END_BLOCK) {
+        assert(MFI.getResults().size() <= 1 &&
+               "Multivalue block signatures not implemented yet");
         EndToBegin[&MI]->getOperand(0).setImm(int32_t(RetType));
         continue;
       }
       if (MI.getOpcode() == WebAssembly::END_LOOP) {
+        assert(MFI.getResults().size() <= 1 &&
+               "Multivalue loop signatures not implemented yet");
         EndToBegin[&MI]->getOperand(0).setImm(int32_t(RetType));
         continue;
       }
       // Something other than an `end`. We're done.
       return;
     }
   }
 }

llvm/lib/Target/WebAssembly/WebAssemblyFastISel.cpp

 bool WebAssemblyFastISel::selectRet(const Instruction *I) {
   if (!FuncInfo.CanLowerReturn)
     return false;
 
   const auto *Ret = cast<ReturnInst>(I);
 
   if (Ret->getNumOperands() == 0) {
     BuildMI(*FuncInfo.MBB, FuncInfo.InsertPt, DbgLoc,
-            TII.get(WebAssembly::RETURN_VOID));
+            TII.get(WebAssembly::RETURN));
     return true;
   }
 
+  // TODO: should probably be <= 2? But be conservative to start...
+  assert(Ret->getNumOperands() < 2 && "Multivalue return not supported yet");
+
   Value *RV = Ret->getOperand(0);
   if (!Subtarget->hasSIMD128() && RV->getType()->isVectorTy())
     return false;
 
-  unsigned Opc;
   switch (getSimpleType(RV->getType())) {
   case MVT::i1:
   case MVT::i8:
   case MVT::i16:
   case MVT::i32:
-    Opc = WebAssembly::RETURN_I32;
-    break;
   case MVT::i64:
-    Opc = WebAssembly::RETURN_I64;
-    break;
   case MVT::f32:
-    Opc = WebAssembly::RETURN_F32;
-    break;
   case MVT::f64:
-    Opc = WebAssembly::RETURN_F64;
-    break;
   case MVT::v16i8:
-    Opc = WebAssembly::RETURN_v16i8;
-    break;
   case MVT::v8i16:
-    Opc = WebAssembly::RETURN_v8i16;
-    break;
   case MVT::v4i32:
-    Opc = WebAssembly::RETURN_v4i32;
-    break;
   case MVT::v2i64:
-    Opc = WebAssembly::RETURN_v2i64;
-    break;
   case MVT::v4f32:
-    Opc = WebAssembly::RETURN_v4f32;
-    break;
   case MVT::v2f64:
-    Opc = WebAssembly::RETURN_v2f64;
-    break;
   case MVT::exnref:
-    Opc = WebAssembly::RETURN_EXNREF;
     break;
   default:
     return false;
   }
 
   unsigned Reg;
   if (FuncInfo.Fn->getAttributes().hasAttribute(0, Attribute::SExt))
     Reg = getRegForSignedValue(RV);
   else if (FuncInfo.Fn->getAttributes().hasAttribute(0, Attribute::ZExt))
     Reg = getRegForUnsignedValue(RV);
   else
     Reg = getRegForValue(RV);
 
   if (Reg == 0)
     return false;
 
-  BuildMI(*FuncInfo.MBB, FuncInfo.InsertPt, DbgLoc, TII.get(Opc)).addReg(Reg);
+  BuildMI(*FuncInfo.MBB, FuncInfo.InsertPt, DbgLoc,
+          TII.get(WebAssembly::RETURN))
+      .addReg(Reg);
   return true;
 }
 
 bool WebAssemblyFastISel::selectUnreachable(const Instruction *I) {
   BuildMI(*FuncInfo.MBB, FuncInfo.InsertPt, DbgLoc,
           TII.get(WebAssembly::UNREACHABLE));
   return true;
 }

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

WebAssemblyTargetLowering::LowerCall(CallLoweringInfo &CLI,
 
   return Chain;
 }
 
 bool WebAssemblyTargetLowering::CanLowerReturn(
     CallingConv::ID /*CallConv*/, MachineFunction & /*MF*/, bool /*IsVarArg*/,
     const SmallVectorImpl<ISD::OutputArg> &Outs,
     LLVMContext & /*Context*/) const {
-  // WebAssembly can't currently handle returning tuples.
-  return Outs.size() <= 1;
+  // WebAssembly can only handle returning tuples with multivalue enabled
+  return Subtarget->hasMultivalue() || Outs.size() <= 1;
 }
 
 SDValue WebAssemblyTargetLowering::LowerReturn(
     SDValue Chain, CallingConv::ID CallConv, bool /*IsVarArg*/,
     const SmallVectorImpl<ISD::OutputArg> &Outs,
     const SmallVectorImpl<SDValue> &OutVals, const SDLoc &DL,
     SelectionDAG &DAG) const {
-  assert(Outs.size() <= 1 && "WebAssembly can only return up to one value");
+  assert(Subtarget->hasMultivalue() ||
+         Outs.size() <= 1 && "MVP WebAssembly can only return up to one value");
   if (!callingConvSupported(CallConv))
     fail(DL, DAG, "WebAssembly doesn't support non-C calling conventions");
 
   SmallVector<SDValue, 4> RetOps(1, Chain);
   RetOps.append(OutVals.begin(), OutVals.end());
   Chain = DAG.getNode(WebAssemblyISD::RETURN, DL, MVT::Other, RetOps);
 
   // Record the number and types of the return values.

llvm/lib/Target/WebAssembly/WebAssemblyInstrControl.td

 defm END_IF    : NRI<(outs), (ins), [], "end_if", 0x0b>;
 // Generic instruction, for disassembler.
 let IsCanonical = 1 in
 defm END       : NRI<(outs), (ins), [], "end", 0x0b>;
 let isTerminator = 1, isBarrier = 1 in
 defm END_FUNCTION : NRI<(outs), (ins), [], "end_function", 0x0b>;
 } // Uses = [VALUE_STACK], Defs = [VALUE_STACK]
 
-multiclass RETURN<WebAssemblyRegClass vt> {
-  defm RETURN_#vt : I<(outs), (ins vt:$val), (outs), (ins),
-                      [(WebAssemblyreturn vt:$val)],
-                      "return  \t$val", "return", 0x0f>;
-  // Equivalent to RETURN_#vt, for use at the end of a function when wasm
-  // semantics return by falling off the end of the block.
-  let isCodeGenOnly = 1 in
-  defm FALLTHROUGH_RETURN_#vt : I<(outs), (ins vt:$val), (outs), (ins), []>;
-}
-
-multiclass SIMD_RETURN<ValueType vt> {
-  defm RETURN_#vt : I<(outs), (ins V128:$val), (outs), (ins),
-                      [(WebAssemblyreturn (vt V128:$val))],
-                      "return  \t$val", "return", 0x0f>,
-                    Requires<[HasSIMD128]>;
-  // Equivalent to RETURN_#vt, for use at the end of a function when wasm
-  // semantics return by falling off the end of the block.
-  let isCodeGenOnly = 1 in
-  defm FALLTHROUGH_RETURN_#vt : I<(outs), (ins V128:$val), (outs), (ins),
-                                  []>,
-                                  Requires<[HasSIMD128]>;
-}
 
 let isTerminator = 1, hasCtrlDep = 1, isBarrier = 1 in {
 
 let isReturn = 1 in {
-  defm "": RETURN<I32>;
-  defm "": RETURN<I64>;
-  defm "": RETURN<F32>;
-  defm "": RETURN<F64>;
-  defm "": RETURN<EXNREF>;
-  defm "": SIMD_RETURN<v16i8>;
-  defm "": SIMD_RETURN<v8i16>;
-  defm "": SIMD_RETURN<v4i32>;
-  defm "": SIMD_RETURN<v2i64>;
-  defm "": SIMD_RETURN<v4f32>;
-  defm "": SIMD_RETURN<v2f64>;
-
-  defm RETURN_VOID : NRI<(outs), (ins), [(WebAssemblyreturn)], "return", 0x0f>;
 
-  // This is to RETURN_VOID what FALLTHROUGH_RETURN_#vt is to RETURN_#vt.
-  let isCodeGenOnly = 1 in
-  defm FALLTHROUGH_RETURN_VOID : NRI<(outs), (ins), []>;
+defm RETURN : I<(outs), (ins variable_ops), (outs), (ins),
+                [(WebAssemblyreturn)],
+                "return", "return", 0x0f>;
+// Equivalent to RETURN, for use at the end of a function when wasm
+// semantics return by falling off the end of the block.
+defm FALLTHROUGH_RETURN : I<(outs), (ins variable_ops), (outs), (ins), []>;
+
 } // isReturn = 1
 
 defm UNREACHABLE : NRI<(outs), (ins), [(trap)], "unreachable", 0x00>;
 } // isTerminator = 1, hasCtrlDep = 1, isBarrier = 1
 
 //===----------------------------------------------------------------------===//
 // Exception handling instructions
 //===----------------------------------------------------------------------===//

llvm/lib/Target/WebAssembly/WebAssemblyInstrInfo.td

def WebAssemblyretcall : SDNode<"WebAssemblyISD::RET_CALL",
                                 SDT_WebAssemblyCall0,
                                 [SDNPHasChain, SDNPVariadic]>;
 def WebAssemblybr_table : SDNode<"WebAssemblyISD::BR_TABLE",
                                  SDT_WebAssemblyBrTable,
                                  [SDNPHasChain, SDNPVariadic]>;
 def WebAssemblyargument : SDNode<"WebAssemblyISD::ARGUMENT",
                                  SDT_WebAssemblyArgument>;
 def WebAssemblyreturn   : SDNode<"WebAssemblyISD::RETURN",
-                                 SDT_WebAssemblyReturn, [SDNPHasChain]>;
+                                 SDT_WebAssemblyReturn,
+                                 [SDNPHasChain, SDNPVariadic]>;
 def WebAssemblywrapper  : SDNode<"WebAssemblyISD::Wrapper",
                                  SDT_WebAssemblyWrapper>;
 def WebAssemblywrapperPIC  : SDNode<"WebAssemblyISD::WrapperPIC",
                                      SDT_WebAssemblyWrapperPIC>;
 def WebAssemblythrow : SDNode<"WebAssemblyISD::THROW", SDT_WebAssemblyThrow,
                               [SDNPHasChain, SDNPVariadic]>;
 
 //===----------------------------------------------------------------------===//

llvm/lib/Target/WebAssembly/WebAssemblyMachineFunctionInfo.cpp

 
 void llvm::computeSignatureVTs(const FunctionType *Ty, const Function &F,
                                const TargetMachine &TM,
                                SmallVectorImpl<MVT> &Params,
                                SmallVectorImpl<MVT> &Results) {
   computeLegalValueVTs(F, TM, Ty->getReturnType(), Results);
 
   MVT PtrVT = MVT::getIntegerVT(TM.createDataLayout().getPointerSizeInBits());
-  if (Results.size() > 1) {
-    // WebAssembly currently can't lower returns of multiple values without
-    // demoting to sret (see WebAssemblyTargetLowering::CanLowerReturn). So
-    // replace multiple return values with a pointer parameter.
+  if (Results.size() > 1 &&
+      !TM.getSubtarget<WebAssemblySubtarget>(F).hasMultivalue()) {
+    // WebAssembly can't lower returns of multiple values without demoting to
+    // sret unless multivalue is enabled (see
+    // WebAssemblyTargetLowering::CanLowerReturn). So replace multiple return
+    // values with a poitner parameter.
     Results.clear();
     Params.push_back(PtrVT);
   }
 
   for (auto *Param : Ty->params())
     computeLegalValueVTs(F, TM, Param, Params);
   if (Ty->isVarArg())
     Params.push_back(PtrVT);

llvm/lib/Target/WebAssembly/WebAssemblyPeephole.cpp

static bool maybeRewriteToDrop(unsigned OldReg, unsigned NewReg,
   }
   return Changed;
 }
 
 static bool maybeRewriteToFallthrough(MachineInstr &MI, MachineBasicBlock &MBB,
                                       const MachineFunction &MF,
                                       WebAssemblyFunctionInfo &MFI,
                                       MachineRegisterInfo &MRI,
-                                      const WebAssemblyInstrInfo &TII,
-                                      unsigned FallthroughOpc,
-                                      unsigned CopyLocalOpc) {
+                                      const WebAssemblyInstrInfo &TII) {
   if (DisableWebAssemblyFallthroughReturnOpt)
     return false;
   if (&MBB != &MF.back())
     return false;
 
   MachineBasicBlock::iterator End = MBB.end();
   --End;
   assert(End->getOpcode() == WebAssembly::END_FUNCTION);
   --End;
   if (&MI != &*End)
     return false;
 
-  if (FallthroughOpc != WebAssembly::FALLTHROUGH_RETURN_VOID) {
-    // If the operand isn't stackified, insert a COPY to read the operand and
-    // stackify it.
-    MachineOperand &MO = MI.getOperand(0);
+  for (auto &MO : MI.explicit_operands()) {
+    // If the operand isn't stackified, insert a COPY to read the operands and
+    // stackify them.
     Register Reg = MO.getReg();
     if (!MFI.isVRegStackified(Reg)) {
-      Register NewReg = MRI.createVirtualRegister(MRI.getRegClass(Reg));
+      unsigned CopyLocalOpc;
+      const TargetRegisterClass *RegClass = MRI.getRegClass(Reg);
+      switch (RegClass->getID()) {
+      case WebAssembly::I32RegClassID:
+        CopyLocalOpc = WebAssembly::COPY_I32;
+        break;
+      case WebAssembly::I64RegClassID:
+        CopyLocalOpc = WebAssembly::COPY_I64;
+        break;
+      case WebAssembly::F32RegClassID:
+        CopyLocalOpc = WebAssembly::COPY_F32;
+        break;
+      case WebAssembly::F64RegClassID:
+        CopyLocalOpc = WebAssembly::COPY_F64;
+        break;
+      case WebAssembly::V128RegClassID:
+        CopyLocalOpc = WebAssembly::COPY_V128;
+        break;
+      case WebAssembly::EXNREFRegClassID:
+        CopyLocalOpc = WebAssembly::COPY_EXNREF;
+        break;
+      default:
+        llvm_unreachable("Unexpected register class for return operand");
+      }
+      Register NewReg = MRI.createVirtualRegister(RegClass);
       BuildMI(MBB, MI, MI.getDebugLoc(), TII.get(CopyLocalOpc), NewReg)
           .addReg(Reg);
       MO.setReg(NewReg);
       MFI.stackifyVReg(NewReg);
     }
   }
 
-  // Rewrite the return.
-  MI.setDesc(TII.get(FallthroughOpc));
+  MI.setDesc(TII.get(WebAssembly::FALLTHROUGH_RETURN));
   return true;
 }
 
 bool WebAssemblyPeephole::runOnMachineFunction(MachineFunction &MF) {
   LLVM_DEBUG({
     dbgs() << "********** Peephole **********\n"
            << "********** Function: " << MF.getName() << '\n';
   });
for (auto &MI : MBB)
                                    "wrong signature, from/to mismatch");
               Changed |= maybeRewriteToDrop(OldReg, NewReg, MO, MFI, MRI);
             }
           }
         }
         break;
       }
       // Optimize away an explicit void return at the end of the function.
-      case WebAssembly::RETURN_I32:
-        Changed |= maybeRewriteToFallthrough(
-            MI, MBB, MF, MFI, MRI, TII, WebAssembly::FALLTHROUGH_RETURN_I32,
-            WebAssembly::COPY_I32);
-        break;
-      case WebAssembly::RETURN_I64:
-        Changed |= maybeRewriteToFallthrough(
-            MI, MBB, MF, MFI, MRI, TII, WebAssembly::FALLTHROUGH_RETURN_I64,
-            WebAssembly::COPY_I64);
-        break;
-      case WebAssembly::RETURN_F32:
-        Changed |= maybeRewriteToFallthrough(
-            MI, MBB, MF, MFI, MRI, TII, WebAssembly::FALLTHROUGH_RETURN_F32,
-            WebAssembly::COPY_F32);
-        break;
-      case WebAssembly::RETURN_F64:
-        Changed |= maybeRewriteToFallthrough(
-            MI, MBB, MF, MFI, MRI, TII, WebAssembly::FALLTHROUGH_RETURN_F64,
-            WebAssembly::COPY_F64);
-        break;
-      case WebAssembly::RETURN_v16i8:
-        Changed |= maybeRewriteToFallthrough(
-            MI, MBB, MF, MFI, MRI, TII, WebAssembly::FALLTHROUGH_RETURN_v16i8,
-            WebAssembly::COPY_V128);
-        break;
-      case WebAssembly::RETURN_v8i16:
-        Changed |= maybeRewriteToFallthrough(
-            MI, MBB, MF, MFI, MRI, TII, WebAssembly::FALLTHROUGH_RETURN_v8i16,
-            WebAssembly::COPY_V128);
-        break;
-      case WebAssembly::RETURN_v4i32:
-        Changed |= maybeRewriteToFallthrough(
-            MI, MBB, MF, MFI, MRI, TII, WebAssembly::FALLTHROUGH_RETURN_v4i32,
-            WebAssembly::COPY_V128);
-        break;
-      case WebAssembly::RETURN_v2i64:
-        Changed |= maybeRewriteToFallthrough(
-            MI, MBB, MF, MFI, MRI, TII, WebAssembly::FALLTHROUGH_RETURN_v2i64,
-            WebAssembly::COPY_V128);
-        break;
-      case WebAssembly::RETURN_v4f32:
-        Changed |= maybeRewriteToFallthrough(
-            MI, MBB, MF, MFI, MRI, TII, WebAssembly::FALLTHROUGH_RETURN_v4f32,
-            WebAssembly::COPY_V128);
-        break;
-      case WebAssembly::RETURN_v2f64:
-        Changed |= maybeRewriteToFallthrough(
-            MI, MBB, MF, MFI, MRI, TII, WebAssembly::FALLTHROUGH_RETURN_v2f64,
-            WebAssembly::COPY_V128);
-        break;
-      case WebAssembly::RETURN_VOID:
-        Changed |= maybeRewriteToFallthrough(
-            MI, MBB, MF, MFI, MRI, TII, WebAssembly::FALLTHROUGH_RETURN_VOID,
-            WebAssembly::INSTRUCTION_LIST_END);
+      case WebAssembly::RETURN:
+        Changed |= maybeRewriteToFallthrough(MI, MBB, MF, MFI, MRI, TII);
         break;
       }
 
   return Changed;
 }

llvm/test/CodeGen/MIR/WebAssembly/int-type-register-class-name.mir

 # RUN: llc -mtriple=wasm32-unknown-unknown -run-pass none -o - %s | FileCheck %s
 # When a register class has a name of an integer type (i32)
 ---
 name: regclass_name_test
 liveins:
   - { reg: '$arguments' }
 body: |
   bb.0:
     liveins: $arguments
     %0:i32 = CONST_I32 0, implicit-def dead $arguments
     ; CHECK: %0:i32 = CONST_I32 0, implicit-def dead $arguments
-    RETURN_VOID implicit-def dead $arguments
+    RETURN implicit-def dead $arguments
 ...

llvm/test/CodeGen/WebAssembly/atomic-fence.mir

bb.0:
     ; CHECK: CALL_VOID @foo
 
     liveins: $arguments
     %0:i32 = CONST_I32 0, implicit-def $arguments
     %1:i32 = ATOMIC_NOTIFY 2, 0, %0:i32, %0:i32, implicit-def $arguments
     COMPILER_FENCE implicit-def $arguments
     %2:i32 = ADD_I32 %0:i32, %0:i32, implicit-def $arguments
     CALL_VOID @foo, %2:i32, %1:i32, implicit-def $arguments
-    RETURN_VOID implicit-def $arguments
+    RETURN implicit-def $arguments
 ...
 
 ---
 # CHECK-LABEL: name: atomic_fence_test
 name: atomic_fence_test
 liveins:
   - { reg: '$arguments' }
 tracksRegLiveness: true
 body: |
   bb.0:
     ; CHECK: %[[REG:[0-9]+]]:i32 = ATOMIC_NOTIFY
     ; CHECK: LOCAL_SET_I32 [[LOCAL:[0-9]+]], %[[REG]]
     ; CHECK: ATOMIC_FENCE
     ; CHECK: ADD_I32
     ; CHECK: LOCAL_GET_I32 [[LOCAL]]
     ; CHECK: CALL_VOID @foo
 
     liveins: $arguments
     %0:i32 = CONST_I32 0, implicit-def $arguments
     %1:i32 = ATOMIC_NOTIFY 2, 0, %0:i32, %0:i32, implicit-def $arguments
     ATOMIC_FENCE 0, implicit-def $arguments
     %2:i32 = ADD_I32 %0:i32, %0:i32, implicit-def $arguments
     CALL_VOID @foo, %2:i32, %1:i32, implicit-def $arguments
-    RETURN_VOID implicit-def $arguments
+    RETURN implicit-def $arguments
 ...
-

llvm/test/CodeGen/WebAssembly/eh-labels.mir

; predecessors: %bb.0
     ; CHECK:      EH_LABEL
     ; CHECK-NEXT: CATCH
     EH_LABEL <mcsymbol .Ltmp2>
     dead %0:i32 = EXTRACT_EXCEPTION_I32 implicit-def dead $arguments
     CATCHRET %bb.2, %bb.0, implicit-def dead $arguments
 
   bb.2:
   ; predecessors: %bb.0, %bb.1
-    RETURN_VOID implicit-def dead $arguments
+    RETURN implicit-def dead $arguments
 ...

llvm/test/CodeGen/WebAssembly/explicit-locals.mir

bb.0:
     successors: %bb.1
     liveins: $arguments
 
   bb.1:
   ; predecessors: %bb.0
     ; CHECK-NOT: dead %{{[0-9]+}}
     ; CHECK: DROP_I32 killed %{{[0-9]+}}
     dead %0:i32 = CONST_I32 0, implicit-def dead $arguments, implicit $sp32, implicit $sp64
-    RETURN_VOID implicit-def dead $arguments
+    RETURN implicit-def dead $arguments
 ...

llvm/test/CodeGen/WebAssembly/function-info.mir

 # RUN: llc -mtriple=wasm32-unknown-unknown -run-pass wasm-cfg-stackify %s -o - | FileCheck %s
 
 # CHECK-LABEL: function_property_test
 # CHECK: machineFunctionInfo:
 # CHECK:   isCFGStackified:   true
 name: function_property_test
 liveins:
   - { reg: '$arguments' }
 body: |
   bb.0:
-    RETURN_VOID implicit-def dead $arguments
+    RETURN implicit-def dead $arguments
 ...

llvm/test/CodeGen/WebAssembly/llround-conv-i32.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=wasm32 | FileCheck %s
 
 define i64 @testmsxs_builtin(float %x) {
 ; CHECK-LABEL: testmsxs_builtin:
 ; CHECK:         .functype testmsxs_builtin (f32) -> (i64)
 ; CHECK-NEXT:  # %bb.0: # %entry
 ; CHECK-NEXT:    local.get 0
 ; CHECK-NEXT:    i64.call llroundf
-; CHECK-NEXT:    # fallthrough-return-value
+; CHECK-NEXT:    # fallthrough-return
 ; CHECK-NEXT:    end_function
 entry:
   %0 = tail call i64 @llvm.llround.f32(float %x)
   ret i64 %0
 }
 
 define i64 @testmsxd_builtin(double %x) {
 ; CHECK-LABEL: testmsxd_builtin:
 ; CHECK:         .functype testmsxd_builtin (f64) -> (i64)
 ; CHECK-NEXT:  # %bb.0: # %entry
 ; CHECK-NEXT:    local.get 0
 ; CHECK-NEXT:    i64.call llround
-; CHECK-NEXT:    # fallthrough-return-value
+; CHECK-NEXT:    # fallthrough-return
 ; CHECK-NEXT:    end_function
 entry:
   %0 = tail call i64 @llvm.llround.f64(double %x)
   ret i64 %0
 }
 
 declare i64 @llvm.llround.f32(float) nounwind readnone
 declare i64 @llvm.llround.f64(double) nounwind readnone

llvm/test/CodeGen/WebAssembly/multivalue.ll

 ; RUN: llc < %s -asm-verbose=false -verify-machineinstrs -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -wasm-keep-registers -mattr=+multivalue | FileCheck %s
 
 ; Test that the multivalue attribute is accepted
 ; TODO(tlively): implement multivalue
 
 target datalayout = "e-m:e-p:32:32-i64:64-n32:64-S128"
 target triple = "wasm32-unknown-unknown"
 
 %pair = type { i32, i32 }
 %packed_pair = type <{ i32, i32 }>
 
-; CHECK-LABEL: sret:
-; CHECK-NEXT: sret (i32, i32, i32) -> ()
-define %pair @sret(%pair %p) {
+; CHECK-LABEL: pair_ident:
+; CHECK-NEXT: pair_ident (i32, i32) -> (i32, i32)
+; CHECK-NEXT: return $0, $1{{$}}
+define %pair @pair_ident(%pair %p) {
   ret %pair %p
 }
 
-; CHECK-LABEL: packed_sret:
-; CHECK-NEXT: packed_sret (i32, i32, i32) -> ()
-define %packed_pair @packed_sret(%packed_pair %p) {
+; CHECK-LABEL: packed_pair_ident:
+; CHECK-NEXT: packed_pair_ident (i32, i32) -> (i32, i32)
+; CHECK-nEXT: return $0, $1{{$}}
+define %packed_pair @packed_pair_ident(%packed_pair %p) {
   ret %packed_pair %p
 }
 
 ; CHECK-LABEL: .section .custom_section.target_features
 ; CHECK-NEXT: .int8 1
 ; CHECK-NEXT: .int8 43
 ; CHECK-NEXT: .int8 10
 ; CHECK-NEXT: .ascii "multivalue"

llvm/test/CodeGen/WebAssembly/reg-argument.mir

 # RUN: llc -mtriple=wasm32-unknown-unknown %s -o - -run-pass wasm-argument-move  | FileCheck %s
 
 # wasm-argument-move pass moves all ARGUMENT instructions to the top of the
 # entry BB.
 ---
 name: argument_i32
 # CHECK-LABEL: argument_i32
 body: |
   ; CHECK-LABEL: bb.0:
   ; CHECK-NEXT: %1:i32 = ARGUMENT_i32 0
   bb.0:
     %0:i32 = CONST_I32 0, implicit-def $arguments
     %1:i32 = ARGUMENT_i32 0, implicit $arguments
-    RETURN_VOID implicit-def $arguments
+    RETURN implicit-def $arguments
 ...
 ---
 name: argument_i64
 # CHECK-LABEL: argument_i64
 body: |
   ; CHECK-LABEL: bb.0:
   ; CHECK-NEXT: %1:i64 = ARGUMENT_i64 0
   bb.0:
     %0:i32 = CONST_I32 0, implicit-def $arguments
     %1:i64 = ARGUMENT_i64 0, implicit $arguments
-    RETURN_VOID implicit-def $arguments
+    RETURN implicit-def $arguments
 ...
 ---
 name: argument_f32
 # CHECK-LABEL: argument_f32
 body: |
   ; CHECK-LABEL: bb.0:
   ; CHECK-NEXT: %1:f32 = ARGUMENT_f32 0
   bb.0:
     %0:i32 = CONST_I32 0, implicit-def $arguments
     %1:f32 = ARGUMENT_f32 0, implicit $arguments
-    RETURN_VOID implicit-def $arguments
+    RETURN implicit-def $arguments
 ...
 ---
 name: argument_f64
 # CHECK-LABEL: argument_f64
 body: |
   ; CHECK-LABEL: bb.0:
   ; CHECK-NEXT: %1:f64 = ARGUMENT_f64 0
   bb.0:
     %0:i32 = CONST_I32 0, implicit-def $arguments
     %1:f64 = ARGUMENT_f64 0, implicit $arguments
-    RETURN_VOID implicit-def $arguments
+    RETURN implicit-def $arguments
 ...
 ---
 name: argument_exnref
 # CHECK-LABEL: argument_exnref
 body: |
   ; CHECK-LABEL: bb.0:
   ; CHECK-NEXT: %1:exnref = ARGUMENT_exnref 0
   bb.0:
     %0:i32 = CONST_I32 0, implicit-def $arguments
     %1:exnref = ARGUMENT_exnref 0, implicit $arguments
-    RETURN_VOID implicit-def $arguments
+    RETURN implicit-def $arguments
 ...

llvm/test/CodeGen/WebAssembly/reg-copy.mir

 # RUN: llc -mtriple=wasm32-unknown-unknown %s -o - -run-pass=postrapseudos | FileCheck %s
 
 ---
 name: copy_i32
 # CHECK-LABEL: copy_i32
 body: |
   ; CHECK-LABEL: bb.0:
   ; CHECK-NEXT: %0:i32 = COPY_I32 %1:i32
-  ; CHECK-NEXT: RETURN_VOID
+  ; CHECK-NEXT: RETURN
   bb.0:
     %0:i32 = COPY %1:i32
-    RETURN_VOID implicit-def $arguments
+    RETURN implicit-def $arguments
 ...
 ---
 name: copy_i64
 # CHECK-LABEL: copy_i64
 body:             |
   ; CHECK-LABEL: bb.0:
   ; CHECK-NEXT: %0:i64 = COPY_I64 %1:i64
-  ; CHECK-NEXT: RETURN_VOID
+  ; CHECK-NEXT: RETURN
   bb.0:
     %0:i64 = COPY %1:i64
-    RETURN_VOID implicit-def $arguments
+    RETURN implicit-def $arguments
 ...
 ---
 name: copy_f32
 # CHECK-LABEL: copy_f32
 body: |
   ; CHECK-LABEL: bb.0:
   ; CHECK-NEXT: %0:f32 = COPY_F32 %1:f32
-  ; CHECK-NEXT: RETURN_VOID
+  ; CHECK-NEXT: RETURN
   bb.0:
     %0:f32 = COPY %1:f32
-    RETURN_VOID implicit-def $arguments
+    RETURN implicit-def $arguments
 ...
 ---
 name: copy_f64
 # CHECK-LABEL: copy_f64
 body: |
   ; CHECK-LABEL: bb.0:
   ; CHECK-NEXT: %0:f64 = COPY_F64 %1:f64
-  ; CHECK-NEXT: RETURN_VOID
+  ; CHECK-NEXT: RETURN
   bb.0:
     %0:f64 = COPY %1:f64
-    RETURN_VOID implicit-def $arguments
+    RETURN implicit-def $arguments
 ...
 ---
 name: copy_v128
 # CHECK-LABEL: copy_v128
 body: |
   ; CHECK-LABEL: bb.0:
   ; CHECK-NEXT: %0:v128 = COPY_V128 %1:v128
-  ; CHECK-NEXT: RETURN_VOID
+  ; CHECK-NEXT: RETURN
   bb.0:
     %0:v128 = COPY %1:v128
-    RETURN_VOID implicit-def $arguments
+    RETURN implicit-def $arguments
 ...
 ---
 name: copy_exnref
 # CHECK-LABEL: copy_exnref
 body: |
   ; CHECK-LABEL: bb.0:
   ; CHECK-NEXT: %0:exnref = COPY_EXNREF %1:exnref
-  ; CHECK-NEXT: RETURN_VOID
+  ; CHECK-NEXT: RETURN
   bb.0:
     %0:exnref = COPY %1:exnref
-    RETURN_VOID implicit-def $arguments
+    RETURN implicit-def $arguments
 ...

llvm/test/DebugInfo/WebAssembly/dbg-value-move-clone.mir

bb.0:
     %0:i64 = ARGUMENT_i64 0, implicit $arguments
     %1:i32 = CONST_I32 0, implicit-def dead $arguments
     DBG_VALUE %1:i32, $noreg, !10, !DIExpression(), debug-location !13; <unknown>:357:12 line no:357
     CALL_VOID @foo, %1:i32, implicit-def dead $arguments, implicit $sp32, implicit $sp64
 
   bb.1:
     CALL_VOID @foo, %1:i32, implicit-def dead $arguments, implicit $sp32, implicit $sp64
     CALL_VOID @foo, %1:i32, implicit-def dead $arguments, implicit $sp32, implicit $sp64
-    RETURN_VOID implicit-def dead $arguments
+    RETURN implicit-def dead $arguments
 
 ...

llvm/test/DebugInfo/WebAssembly/dbg-value-move-reg-stackify.mir

bb.0:
     %0:i64 = ARGUMENT_i64 0, implicit $arguments
     %1:i32 = I32_WRAP_I64 %0:i64, implicit-def dead $arguments
     DBG_VALUE %1:i32, $noreg, !10, !DIExpression(), debug-location !13; <unknown>:357:12 line no:357
     %1:i32 = CALL_i32 @bar, implicit-def dead $arguments, implicit $sp32, implicit $sp64
     DBG_VALUE %1:i32, $noreg, !11, !DIExpression(), debug-location !14; <unknown>:357:12 line no:357
     %1:i32 = CALL_i32 @bar, implicit-def dead $arguments, implicit $sp32, implicit $sp64
     DBG_VALUE %1:i32, $noreg, !12, !DIExpression(), debug-location !15; <unknown>:357:12 line no:357
     CALL_VOID @foo, %1:i32, implicit-def dead $arguments, implicit $sp32, implicit $sp64
-    RETURN_VOID implicit-def dead $arguments
+    RETURN implicit-def dead $arguments
 
 ...