[WebAssembly] Split and recombine multivalue calls for ISel

[tlively]

Summary

Multivalue calls both take and return an arbitrary number of
arguments, but ISel only supports one or the other in a single
instruction. To get around this, calls are modeled as two pseudo
instructions during ISel. These pseudo instructions, CALL_PARAMS and
CALL_RESULTS, are recombined into a single CALL MachineInstr in a
custom emit hook.

RegStackification and the MC layer will additionally need to be made
aware of multivalue calls before the tests will produce correct
output.

[merge_guards_bot]

clang-format: pass.

Build artifacts: console-log.txt, diff.json

[tlively]

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

462

This llvm_unreachable truly is unreachable because it comes after the return. There is a default case at the top of the switch that contains another llvm_unreachable, so this one was both incorrect and redundant.

[aheejin]

Nice! Are there no tests because we don't have the assembly printer support yet?

llvm/lib/Target/WebAssembly/WebAssemblyISelDAGToDAG.cpp

209

Nit: Does this i32 have a meaning? If not, can we use a chain (MVT::Other) instead?

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

433

Is CallParams guaranteed to be the previous node? What we did in ISelDAGToDAG is to make CallParams an operand of CallResult. While they have data dependences and CallResult will be guaranteed to be selected after CallParams, I'm not sure if there can't be any other instructions in between. Can't we get the operand of CallResults?

444

How do we assign which is defs and which is uses in this new CALL instruction?

llvm/lib/Target/WebAssembly/WebAssemblyInstrCall.td

59

CALL->CALL_PARAMS?

63

Sorry for real nit: Can we make comments wrap to 80 cols..? Lines kind of look to short.. I think there should be a shortcut in most editors for doing that.

70

Isn't this also isPseudo = 1?

[tlively]

• Address simple comments

[merge_guards_bot]

Unit tests: pass. 60863 tests passed, 0 failed and 726 were skipped.

clang-format: pass.

Build artifacts: diff.json, CMakeCache.txt, console-log.txt, test-results.xml

[tlively]

llvm/lib/Target/WebAssembly/WebAssemblyISelDAGToDAG.cpp

209

There's no meaning, but I have not been able avoid assertion failures using MTV::Other. I'll investigate further without spending too much time on it.

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

433

Yeah, good point. I didn't see a simple API for getting defs of operands from MachineInstrs, but making this more robust is probably worth some extra code.

444

MachineOperands store this information on themselves and we follow the convention here of having the defs before the uses so things like getNumDefs should still work. This will be a bigger problem at the MC layer, where the def and use information is erased.

[tlively]

• Use MVT::Other for link, rebase

[merge_guards_bot]

Unit tests: pass. 60863 tests passed, 0 failed and 726 were skipped.

clang-format: pass.

Build artifacts: diff.json, CMakeCache.txt, console-log.txt, test-results.xml

[tlively]

llvm/lib/Target/WebAssembly/WebAssemblyISelDAGToDAG.cpp

209

Ok, I've switched this to use MVT::Other. This just required an update to the previous CL to avoid a nullptr dereference.

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

433

Now that I switched to using MVT::Other, there isn't even a shared register to look for. I think we have no choice but to blindly hope that these instructions are emitted next to each other :/

[aheejin]

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

433

Oh, if changing to MTV::Other makes things harder, just don't do it ;( Sorry I was just suggesting.
Oh the other hand, can we use the MTV::Glue thing? I haven't used it myself and I'm not super familiar with it, but the definition of that is to glue nodes together, right? If we can actually guarantee they are glued together that's gonna be good.

[tlively]

• Switch to using MVT::Glue

[tlively]

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

433

MVT::Glue works! There must have been something else wrong when I experimented with it before.

[merge_guards_bot]

Unit tests: fail. 61975 tests passed, 1 failed and 783 were skipped.

failed: LLVM.Bindings/Go/go.test

clang-tidy: unknown.

clang-format: pass.

Build artifacts: diff.json, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[aheejin]

LGTM. Please rebase this onto D71484; this currently contains diff of that CL too.

llvm/include/llvm/Target/TargetMachine.h

public:
   virtual bool isMachineVerifierClean() const { return true; }
 
   /// Adds an AsmPrinter pass to the pipeline that prints assembly or
   /// machine code from the MI representation.
   bool addAsmPrinter(PassManagerBase &PM, raw_pwrite_stream &Out,
                      raw_pwrite_stream *DwoOut, CodeGenFileType FileType,
                      MCContext &Context);
 
-  /// True if the target uses physical regs at Prolog/Epilog insertion
-  /// time. If true (most machines), all vregs must be allocated before
-  /// PEI. If false (virtual-register machines), then callee-save register
-  /// spilling and scavenging are not needed or used.
-  virtual bool usesPhysRegsForPEI() const { return true; }
+  /// True if the target uses physical regs (as nearly all targets do). False
+  /// for stack machines such as WebAssembly and other virtual-register
+  /// machines. If true, all vregs must be allocated before PEI. If false, then
+  /// callee-save register spilling and scavenging are not needed or used. If
+  /// false, implicitly defined registers will still be assumed to be physical
+  /// registers, except that variadic defs will be allocated vregs.
+  virtual bool usesPhysRegsForValues() const { return true; }
 
   /// True if the target wants to use interprocedural register allocation by
   /// default. The -enable-ipra flag can be used to override this.
   virtual bool useIPRA() const {
     return false;
   }
 };
 

llvm/lib/CodeGen/PrologEpilogInserter.cpp

bool PEI::runOnMachineFunction(MachineFunction &MF) {
   calculateSaveRestoreBlocks(MF);
 
   // Stash away DBG_VALUEs that should not be moved by insertion of prolog code.
   SavedDbgValuesMap EntryDbgValues;
   for (MachineBasicBlock *SaveBlock : SaveBlocks)
     stashEntryDbgValues(*SaveBlock, EntryDbgValues);
 
   // Handle CSR spilling and restoring, for targets that need it.
-  if (MF.getTarget().usesPhysRegsForPEI())
+  if (MF.getTarget().usesPhysRegsForValues())
     spillCalleeSavedRegs(MF);
 
   // Allow the target machine to make final modifications to the function
   // before the frame layout is finalized.
   TFI->processFunctionBeforeFrameFinalized(MF, RS);
 
   // Calculate actual frame offsets for all abstract stack objects...
   calculateFrameObjectOffsets(MF);

llvm/lib/CodeGen/SelectionDAG/InstrEmitter.cpp

void InstrEmitter::CreateVirtualRegisters(SDNode *Node,
                                        MachineInstrBuilder &MIB,
                                        const MCInstrDesc &II,
                                        bool IsClone, bool IsCloned,
                                        DenseMap<SDValue, unsigned> &VRBaseMap) {
   assert(Node->getMachineOpcode() != TargetOpcode::IMPLICIT_DEF &&
          "IMPLICIT_DEF should have been handled as a special case elsewhere!");
 
   unsigned NumResults = CountResults(Node);
-  for (unsigned i = 0; i < II.getNumDefs(); ++i) {
+  bool HasVRegVariadicDefs = !MF->getTarget().usesPhysRegsForValues() &&
+                             II.isVariadic() && II.variadicOpsAreDefs();
+  unsigned NumVRegs = HasVRegVariadicDefs ? NumResults : II.getNumDefs();
+  for (unsigned i = 0; i < NumVRegs; ++i) {
     // If the specific node value is only used by a CopyToReg and the dest reg
     // is a vreg in the same register class, use the CopyToReg'd destination
     // register instead of creating a new vreg.
     unsigned VRBase = 0;
     const TargetRegisterClass *RC =
       TRI->getAllocatableClass(TII->getRegClass(II, i, TRI, *MF));
     // Always let the value type influence the used register class. The
     // constraints on the instruction may be too lax to represent the value
if (Opc == TargetOpcode::PATCHPOINT) {
       NumDefs = NumResults;
     }
     ScratchRegs = TLI->getScratchRegisters((CallingConv::ID) CC);
   }
 
   unsigned NumImpUses = 0;
   unsigned NodeOperands =
     countOperands(Node, II.getNumOperands() - NumDefs, NumImpUses);
-  bool HasPhysRegOuts = NumResults > NumDefs && II.getImplicitDefs()!=nullptr;
+  bool HasVRegVariadicDefs = !MF->getTarget().usesPhysRegsForValues() &&
+                             II.isVariadic() && II.variadicOpsAreDefs();
+  bool HasPhysRegOuts = NumResults > NumDefs &&
+                        II.getImplicitDefs() != nullptr && !HasVRegVariadicDefs;
 #ifndef NDEBUG
   unsigned NumMIOperands = NodeOperands + NumResults;
   if (II.isVariadic())
     assert(NumMIOperands >= II.getNumOperands() &&
            "Too few operands for a variadic node!");
   else
     assert(NumMIOperands >= II.getNumOperands() &&
            NumMIOperands <= II.getNumOperands() + II.getNumImplicitDefs() +

llvm/lib/Target/WebAssembly/MCTargetDesc/WebAssemblyMCTargetDesc.h

case WebAssembly::RET_CALL_INDIRECT_S:
     return true;
   default:
     return false;
   }
 }
 
 /// Returns the operand number of a callee, assuming the argument is a call
 /// instruction.
-inline unsigned getCalleeOpNo(unsigned Opc) {
-  switch (Opc) {
+inline const MachineOperand &getCalleeOp(const MachineInstr &MI) {
+  switch (MI.getOpcode()) {
   case WebAssembly::CALL_VOID:
   case WebAssembly::CALL_VOID_S:
   case WebAssembly::CALL_INDIRECT_VOID:
   case WebAssembly::CALL_INDIRECT_VOID_S:
   case WebAssembly::RET_CALL:
   case WebAssembly::RET_CALL_S:
   case WebAssembly::RET_CALL_INDIRECT:
   case WebAssembly::RET_CALL_INDIRECT_S:
-    return 0;
+    return MI.getOperand(0);
   case WebAssembly::CALL_i32:
   case WebAssembly::CALL_i32_S:
   case WebAssembly::CALL_i64:
   case WebAssembly::CALL_i64_S:
   case WebAssembly::CALL_f32:
   case WebAssembly::CALL_f32_S:
   case WebAssembly::CALL_f64:
   case WebAssembly::CALL_f64_S:
inline const MachineOperand &getCalleeOp(const MachineInstr &MI) {
   case WebAssembly::CALL_INDIRECT_v2i64:
   case WebAssembly::CALL_INDIRECT_v2i64_S:
   case WebAssembly::CALL_INDIRECT_v4f32:
   case WebAssembly::CALL_INDIRECT_v4f32_S:
   case WebAssembly::CALL_INDIRECT_v2f64:
   case WebAssembly::CALL_INDIRECT_v2f64_S:
   case WebAssembly::CALL_INDIRECT_exnref:
   case WebAssembly::CALL_INDIRECT_exnref_S:
-    return 1;
+    return MI.getOperand(1);
+  case WebAssembly::CALL:
+  case WebAssembly::CALL_S:
+    return MI.getOperand(MI.getNumDefs());
   default:
     llvm_unreachable("Not a call instruction");
   }
 }
 
 inline bool isMarker(unsigned Opc) {
   switch (Opc) {
   case WebAssembly::BLOCK:

llvm/lib/Target/WebAssembly/WebAssemblyISD.def

 /// This file describes the various WebAssembly ISD node types.
 ///
 //===----------------------------------------------------------------------===//
 
 // NOTE: NO INCLUDE GUARD DESIRED!
 
 HANDLE_NODETYPE(CALL1)
 HANDLE_NODETYPE(CALL0)
+HANDLE_NODETYPE(CALL)
 HANDLE_NODETYPE(RET_CALL)
 HANDLE_NODETYPE(RETURN)
 HANDLE_NODETYPE(ARGUMENT)
 // A wrapper node for TargetExternalSymbol, TargetGlobalAddress, and MCSymbol
 HANDLE_NODETYPE(Wrapper)
 // A special wapper used in PIC code for __memory_base/__table_base relcative
 // access.
 HANDLE_NODETYPE(WrapperPIC)

llvm/lib/Target/WebAssembly/WebAssemblyISelDAGToDAG.cpp

case Intrinsic::wasm_tls_base: {
           CurDAG->getTargetExternalSymbol("__tls_base", PtrVT),
           Node->getOperand(0));
       ReplaceNode(Node, TLSBase);
       return;
     }
     }
     break;
   }
+  case WebAssemblyISD::CALL: {
+    // CALL has both variable operands and variable results, but ISel only
+    // supports one or the other. Split calls into two nodes connected by a
+    // dummy value, one for the operands and one for the results. These two
+    // nodes will be recombined in a custom inserter hook into a single
+    // MachineInstr.
+    SmallVector<SDValue, 16> Ops;
+    for (size_t i = 1; i < Node->getNumOperands(); ++i) {
+      SDValue Op = Node->getOperand(i);
+      if (Op->getOpcode() == WebAssemblyISD::Wrapper)
+        Op = Op->getOperand(0);
+      Ops.push_back(Op);
+    }
+    Ops.push_back(Node->getOperand(0));
+    MachineSDNode *CallParams =
+        CurDAG->getMachineNode(WebAssembly::CALL_PARAMS, DL, MVT::i32, Ops);
+    SDValue Link(CallParams, 0);
+    MachineSDNode *CallResults = CurDAG->getMachineNode(
+        WebAssembly::CALL_RESULTS, DL, Node->getVTList(), Link);
+    ReplaceNode(Node, CallResults);
+    return;
+  }
 
   default:
     break;
   }
 
   // Select the default instruction.
   SelectCode(Node);
 }

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

BuildMI(*DoneMBB, DoneMBB->begin(), DL, TII.get(TargetOpcode::PHI), OutReg)
       .addReg(FalseReg)
       .addMBB(FalseMBB)
       .addReg(TrueReg)
       .addMBB(TrueMBB);
 
   return DoneMBB;
 }
 
+static MachineBasicBlock *LowerCallResults(MachineInstr &CallResults,
+                                           DebugLoc DL, MachineBasicBlock *BB,
+                                           const TargetInstrInfo &TII) {
+  MachineInstr &CallParams = *CallResults.getPrevNode();
+  assert(CallParams.getOpcode() == WebAssembly::CALL_PARAMS);
+  assert(CallResults.getOpcode() == WebAssembly::CALL_RESULTS);
+
+  MachineFunction &MF = *BB->getParent();
+  const MCInstrDesc &MCID = TII.get(WebAssembly::CALL);
+  MachineInstrBuilder MIB(MF, MF.CreateMachineInstr(MCID, DL));
+
+  for (auto Def : CallResults.defs())
+    MIB.add(Def);
+  for (auto Use : CallParams.uses())
+    MIB.add(Use);
+
+  BB->insert(CallResults.getIterator(), MIB);
+  CallParams.eraseFromParent();
+  CallResults.eraseFromParent();
+
+  return BB;
+}
+
 MachineBasicBlock *WebAssemblyTargetLowering::EmitInstrWithCustomInserter(
     MachineInstr &MI, MachineBasicBlock *BB) const {
   const TargetInstrInfo &TII = *Subtarget->getInstrInfo();
   DebugLoc DL = MI.getDebugLoc();
 
   switch (MI.getOpcode()) {
   default:
     llvm_unreachable("Unexpected instr type to insert");
case WebAssembly::FP_TO_UINT_I32_F64:
     return LowerFPToInt(MI, DL, BB, TII, true, false, true,
                         WebAssembly::I32_TRUNC_U_F64);
   case WebAssembly::FP_TO_SINT_I64_F64:
     return LowerFPToInt(MI, DL, BB, TII, false, true, true,
                         WebAssembly::I64_TRUNC_S_F64);
   case WebAssembly::FP_TO_UINT_I64_F64:
     return LowerFPToInt(MI, DL, BB, TII, true, true, true,
                         WebAssembly::I64_TRUNC_U_F64);
-    llvm_unreachable("Unexpected instruction to emit with custom inserter");
+  case WebAssembly::CALL_RESULTS:
+    return LowerCallResults(MI, DL, BB, TII);
   }
 }
 
 const char *
 WebAssemblyTargetLowering::getTargetNodeName(unsigned Opcode) const {
   switch (static_cast<WebAssemblyISD::NodeType>(Opcode)) {
   case WebAssemblyISD::FIRST_NUMBER:
   case WebAssemblyISD::FIRST_MEM_OPCODE:
if (Subtarget->hasTailCall() && !CLI.IsVarArg) {
         } else {
           fail(DL, DAG, "WebAssembly 'tail-call' feature not enabled");
         }
       }
     }
   }
 
   SmallVectorImpl<ISD::InputArg> &Ins = CLI.Ins;
-  if (Ins.size() > 1)
-    fail(DL, DAG, "WebAssembly doesn't support more than 1 returned value yet");
-
   SmallVectorImpl<ISD::OutputArg> &Outs = CLI.Outs;
   SmallVectorImpl<SDValue> &OutVals = CLI.OutVals;
 
   // The generic code may have added an sret argument. If we're lowering an
   // invoke function, the ABI requires that the function pointer be the first
   // argument, so we may have to swap the arguments.
   if (CallConv == CallingConv::WASM_EmscriptenInvoke && Outs.size() >= 2 &&
       Outs[0].Flags.isSRet()) {
WebAssemblyTargetLowering::LowerCall(CallLoweringInfo &CLI,
 
   if (CLI.IsTailCall) {
     // ret_calls do not return values to the current frame
     SDVTList NodeTys = DAG.getVTList(MVT::Other, MVT::Glue);
     return DAG.getNode(WebAssemblyISD::RET_CALL, DL, NodeTys, Ops);
   }
 
   InTys.push_back(MVT::Other);
-  SDVTList InTyList = DAG.getVTList(InTys);
-  SDValue Res =
-      DAG.getNode(Ins.empty() ? WebAssemblyISD::CALL0 : WebAssemblyISD::CALL1,
-                  DL, InTyList, Ops);
-  if (Ins.empty()) {
-    Chain = Res;
-  } else {
-    InVals.push_back(Res);
-    Chain = Res.getValue(1);
+  unsigned Opc;
+  // TODO: Remove CALL0 and CALL1 in favor of CALL
+  switch (Ins.size()) {
+  case 0:
+    Opc = WebAssemblyISD::CALL0;
+    break;
+  case 1:
+    Opc = WebAssemblyISD::CALL1;
+    break;
+  default:
+    Opc = WebAssemblyISD::CALL;
+    break;
   }
+  SDVTList InTyList = DAG.getVTList(InTys);
+  SDValue Res = DAG.getNode(Opc, DL, InTyList, Ops);
 
-  return Chain;
+  for (size_t I = 0; I < Ins.size(); ++I)
+    InVals.push_back(Res.getValue(I));
+
+  // Return the chain
+  return Res.getValue(Ins.size());
 }
 
 bool WebAssemblyTargetLowering::CanLowerReturn(
     CallingConv::ID /*CallConv*/, MachineFunction & /*MF*/, bool /*IsVarArg*/,
     const SmallVectorImpl<ISD::OutputArg> &Outs,
     LLVMContext & /*Context*/) const {
   // WebAssembly can only handle returning tuples with multivalue enabled
   return Subtarget->hasMultivalue() || Outs.size() <= 1;

llvm/lib/Target/WebAssembly/WebAssemblyInstrCall.td

I<(outs rt:$dst),
       (outs), (ins TypeIndex:$type, i32imm:$flags),
       [],
       !strconcat(prefix, "call_indirect\t$dst"),
       !strconcat(prefix, "call_indirect\t$type"),
       0x11>,
     Requires<preds>;
 }
 
+// CALL should take both variadic arguments and produce variadic results, but
+// this is not possible to model directly. Instead, we select calls to a
+// CALL_PARAMS taking variadic aguments linked with a CALL_RESULTS that handles
+// producing the call's variadic results. We recombine the two in a custom
+// inserter hook after DAG ISel, so passes over MachineInstrs will only ever
+// observe CALL nodes with all of the expected variadic uses and defs.
+let isPseudo = 1 in
+defm CALL_PARAMS :
+  I<(outs I32:$link), (ins function32_op:$callee, variable_ops),
+    (outs), (ins function32_op:$callee), [],
+    "call_params\t$callee", "call_params\t$callee", -1>;
+
+let variadicOpsAreDefs = 1, usesCustomInserter = 1, isPseudo = 1 in
+defm CALL_RESULTS :
+  I<(outs), (ins I32:$link, variable_ops), (outs), (ins), [],
+     "call_results", "call_results", -1>;
+
 let Uses = [SP32, SP64], isCall = 1 in {
+
+// TODO: Add an indirect version of the variadic call, delete CALL_*
+defm CALL :
+  I<(outs), (ins function32_op:$callee, variable_ops),
+    (outs), (ins function32_op:$callee), [],
+    "call\t$callee", "call\t$callee", 0x10>;
+
 defm "" : CALL<i32, I32, "i32.">;
 defm "" : CALL<i64, I64, "i64.">;
 defm "" : CALL<f32, F32, "f32.">;
 defm "" : CALL<f64, F64, "f64.">;
 defm "" : CALL<exnref, EXNREF, "exnref.", [HasExceptionHandling]>;
 defm "" : CALL<v16i8, V128, "v128.", [HasSIMD128]>;
 defm "" : CALL<v8i16, V128, "v128.", [HasSIMD128]>;
 defm "" : CALL<v4i32, V128, "v128.", [HasSIMD128]>;
 defm "" : CALL<v2i64, V128, "v128.", [HasSIMD128]>;
 defm "" : CALL<v4f32, V128, "v128.", [HasSIMD128]>;
 defm "" : CALL<v2f64, V128, "v128.", [HasSIMD128]>;
 
 let IsCanonical = 1 in {
+
 defm CALL_VOID :
   I<(outs), (ins function32_op:$callee, variable_ops),
     (outs), (ins function32_op:$callee),
     [(WebAssemblycall0 (i32 imm:$callee))],
     "call    \t$callee", "call\t$callee", 0x10>;
 
 let isReturn = 1 in
 defm RET_CALL :

llvm/lib/Target/WebAssembly/WebAssemblyRegStackify.cpp

static void convertImplicitDefToConstZero(MachineInstr *MI,
   } else {
     llvm_unreachable("Unexpected reg class");
   }
 }
 
 // Determine whether a call to the callee referenced by
 // MI->getOperand(CalleeOpNo) reads memory, writes memory, and/or has side
 // effects.
-static void queryCallee(const MachineInstr &MI, unsigned CalleeOpNo, bool &Read,
-                        bool &Write, bool &Effects, bool &StackPointer) {
+static void queryCallee(const MachineInstr &MI, bool &Read, bool &Write,
+                        bool &Effects, bool &StackPointer) {
   // All calls can use the stack pointer.
   StackPointer = true;
 
-  const MachineOperand &MO = MI.getOperand(CalleeOpNo);
+  const MachineOperand &MO = WebAssembly::getCalleeOp(MI);
   if (MO.isGlobal()) {
     const Constant *GV = MO.getGlobal();
     if (const auto *GA = dyn_cast<GlobalAlias>(GV))
       if (!GA->isInterposable())
         GV = GA->getAliasee();
 
     if (const auto *F = dyn_cast<Function>(GV)) {
       if (!F->doesNotThrow())
static void query(const MachineInstr &MI, AliasAnalysis &AA, bool &Read,
 
   // Check for writes to __stack_pointer global.
   if (MI.getOpcode() == WebAssembly::GLOBAL_SET_I32 &&
       strcmp(MI.getOperand(0).getSymbolName(), "__stack_pointer") == 0)
     StackPointer = true;
 
   // Analyze calls.
   if (MI.isCall()) {
-    unsigned CalleeOpNo = WebAssembly::getCalleeOpNo(MI.getOpcode());
-    queryCallee(MI, CalleeOpNo, Read, Write, Effects, StackPointer);
+    queryCallee(MI, Read, Write, Effects, StackPointer);
   }
 }
 
 // Test whether Def is safe and profitable to rematerialize.
 static bool shouldRematerialize(const MachineInstr &Def, AliasAnalysis &AA,
                                 const WebAssemblyInstrInfo *TII) {
   return Def.isAsCheapAsAMove() && TII->isTriviallyReMaterializable(Def, &AA);
 }

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.h

public:
   TargetPassConfig *createPassConfig(PassManagerBase &PM) override;
 
   TargetLoweringObjectFile *getObjFileLowering() const override {
     return TLOF.get();
   }
 
   TargetTransformInfo getTargetTransformInfo(const Function &F) override;
 
-  bool usesPhysRegsForPEI() const override { return false; }
+  bool usesPhysRegsForValues() const override { return false; }
 
   yaml::MachineFunctionInfo *createDefaultFuncInfoYAML() const override;
   yaml::MachineFunctionInfo *
   convertFuncInfoToYAML(const MachineFunction &MF) const override;
   bool parseMachineFunctionInfo(const yaml::MachineFunctionInfo &,
                                 PerFunctionMIParsingState &PFS,
                                 SMDiagnostic &Error,
                                 SMRange &SourceRange) const override;
 };
 
 } // end namespace llvm
 
 #endif

llvm/lib/Target/WebAssembly/WebAssemblyUtilities.cpp

bool WebAssembly::mayThrow(const MachineInstr &MI) {
   case WebAssembly::RETHROW_S:
     return true;
   }
   if (isCallIndirect(MI.getOpcode()))
     return true;
   if (!MI.isCall())
     return false;
 
-  const MachineOperand &MO = MI.getOperand(getCalleeOpNo(MI.getOpcode()));
+  const MachineOperand &MO = getCalleeOp(MI);
   assert(MO.isGlobal() || MO.isSymbol());
 
   if (MO.isSymbol()) {
     // Some intrinsics are lowered to calls to external symbols, which are then
     // lowered to calls to library functions. Most of libcalls don't throw, but
     // we only list some of them here now.
     // TODO Consider adding 'nounwind' info in TargetLowering::CallLoweringInfo
     // instead for more accurate info.

llvm/test/CodeGen/WebAssembly/multivalue.ll

-; RUN: llc < %s -asm-verbose=false -verify-machineinstrs -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -wasm-keep-registers -mattr=+multivalue | FileCheck %s
-; RUN: llc < %s --filetype=obj -mattr=+multivalue | obj2yaml | FileCheck %s --check-prefix OBJ
+; RUN: llc < %s -asm-verbose=false -verify-machineinstrs -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -wasm-keep-registers -mattr=+multivalue,+tail-call | FileCheck %s
+; RUN: llc < %s --filetype=obj -mattr=+multivalue,+tail-call | obj2yaml | FileCheck %s --check-prefix OBJ
 
-; Test that the multivalue returns, function types, and block types
-; work as expected.
+; Test that the multivalue calls, returns, function types, and block
+; types work as expected.
 
 target datalayout = "e-m:e-p:32:32-i64:64-n32:64-S128"
 target triple = "wasm32-unknown-unknown"
 
-%pair = type { i32, i32 }
-%packed_pair = type <{ i32, i32 }>
+%pair = type { i32, i64 }
+%packed_pair = type <{ i32, i64 }>
+
+
+; CHECK-LABEL: pair_const:
+; CHECK-NEXT: .functype pair_const () -> (i32, i64)
+; CHECK-NEXT: i32.const $push[[L0:[0-9]+]]=, 42{{$}}
+; CHECK-NEXT: i64.const $push[[L1:[0-9]+]]=, 42{{$}}
+; CHECK-NEXT: return $pop[[L0]], $pop[[L1]]{{$}}
+define %pair @pair_const() {
+  ret %pair { i32 42, i64 42 }
+}
+
+; CHECK-LABEL: packed_pair_const:
+; CHECK-NEXT: .functype packed_pair_const () -> (i32, i64)
+; CHECK-NEXT: i32.const $push[[L0:[0-9]+]]=, 42{{$}}
+; CHECK-NEXT: i64.const $push[[L1:[0-9]+]]=, 42{{$}}
+; CHECK-NEXT: return $pop[[L0]], $pop[[L1]]{{$}}
+define %packed_pair @packed_pair_const() {
+  ret %packed_pair <{ i32 42, i64 42 }>
+}
 
 ; CHECK-LABEL: pair_ident:
-; CHECK-NEXT: .functype pair_ident (i32, i32) -> (i32, i32)
+; CHECK-NEXT: .functype pair_ident (i32, i64) -> (i32, i64)
 ; CHECK-NEXT: return $0, $1{{$}}
 define %pair @pair_ident(%pair %p) {
   ret %pair %p
 }
 
 ; CHECK-LABEL: packed_pair_ident:
-; CHECK-NEXT: .functype packed_pair_ident (i32, i32) -> (i32, i32)
+; CHECK-NEXT: .functype packed_pair_ident (i32, i64) -> (i32, i64)
 ; CHECK-NEXT: return $0, $1{{$}}
 define %packed_pair @packed_pair_ident(%packed_pair %p) {
   ret %packed_pair %p
 }
 
+;; TODO: Multivalue calls are a WIP and do not necessarily produce
+;; correct output. For now, just check that they don't cause any
+;; crashes.
+
+define void @pair_call() {
+  %p = call %pair @pair_const()
+  ret void
+}
+
+define void @packed_pair_call() {
+  %p = call %packed_pair @packed_pair_const()
+  ret void
+}
+
+define %pair @pair_call_return() {
+  %p = call %pair @pair_const()
+  ret %pair %p
+}
+
+define %packed_pair @packed_pair_call_return() {
+  %p = call %packed_pair @packed_pair_const()
+  ret %packed_pair %p
+}
+
+define %pair @pair_tail_call() {
+  %p = musttail call %pair @pair_const()
+  ret %pair %p
+}
+
+define %packed_pair @packed_pair_tail_call() {
+  %p = musttail call %packed_pair @packed_pair_const()
+  ret %packed_pair %p
+}
+
+define i32 @pair_call_return_first() {
+  %p = call %pair @pair_const()
+  %v = extractvalue %pair %p, 0
+  ret i32 %v
+}
+
+define i32 @packed_pair_call_return_first() {
+  %p = call %packed_pair @packed_pair_const()
+  %v = extractvalue %packed_pair %p, 0
+  ret i32 %v
+}
+
+define i64 @pair_call_return_second() {
+  %p = call %pair @pair_const()
+  %v = extractvalue %pair %p, 1
+  ret i64 %v
+}
+
+define i64 @packed_pair_call_return_second() {
+  %p = call %packed_pair @packed_pair_const()
+  %v = extractvalue %packed_pair %p, 1
+  ret i64 %v
+}
+
+define %pair @pair_pass_through(%pair %p) {
+  %r = call %pair @pair_ident(%pair %p)
+  ret %pair %r
+}
+
+define %packed_pair @packed_pair_pass_through(%packed_pair %p) {
+  %r = call %packed_pair @packed_pair_ident(%packed_pair %p)
+  ret %packed_pair %r
+}
+
 ; CHECK-LABEL: minimal_loop:
 ; CHECK-NEXT: .functype minimal_loop (i32) -> (i32, i64)
 ; CHECK-NEXT: .LBB{{[0-9]+}}_1:
 ; CHECK-NEXT: loop () -> (i32, i64)
 ; CHECK-NEXT: br 0{{$}}
 ; CHECK-NEXT: .LBB{{[0-9]+}}_2:
 ; CHECK-NEXT: end_loop{{$}}
-define {i32, i64} @minimal_loop(i32* %p) {
+define %pair @minimal_loop(i32* %p) {
 entry:
   br label %loop
 loop:
   br label %loop
 }
 
 ; CHECK-LABEL: .section .custom_section.target_features
-; CHECK-NEXT: .int8 1
+; CHECK-NEXT: .int8 2
 ; CHECK-NEXT: .int8 43
 ; CHECK-NEXT: .int8 10
 ; CHECK-NEXT: .ascii "multivalue"
+; CHECK-NEXT: .int8 43
+; CHECK-NEXT: .int8 9
+; CHECK-NEXT: .ascii "tail-call"
 
 ; OBJ-LABEL:  - Type:            TYPE
 ; OBJ-NEXT:     Signatures:
 ; OBJ-NEXT:       - Index:           0
-; OBJ-NEXT:         ParamTypes:
+; OBJ-NEXT:         ParamTypes:      []
+; OBJ-NEXT:         ReturnTypes:
 ; OBJ-NEXT:           - I32
+; OBJ-NEXT:           - I64
+; OBJ-NEXT:       - Index:           1
+; OBJ-NEXT:         ParamTypes:
 ; OBJ-NEXT:           - I32
+; OBJ-NEXT:           - I64
 ; OBJ-NEXT:         ReturnTypes:
 ; OBJ-NEXT:           - I32
+; OBJ-NEXT:           - I64
+; OBJ-NEXT:       - Index:           2
+; OBJ-NEXT:         ParamTypes:      []
+; OBJ-NEXT:         ReturnTypes:     []
+; OBJ-NEXT:       - Index:           3
+; OBJ-NEXT:         ParamTypes:      []
+; OBJ-NEXT:         ReturnTypes:
 ; OBJ-NEXT:           - I32
-; OBJ-NEXT:       - Index:           1
+; OBJ-NEXT:       - Index:           4
+; OBJ-NEXT:         ParamTypes:      []
+; OBJ-NEXT:         ReturnTypes:
+; OBJ-NEXT:           - I64
+; OBJ-NEXT:       - Index:           5
 ; OBJ-NEXT:         ParamTypes:
 ; OBJ-NEXT:           - I32
 ; OBJ-NEXT:         ReturnTypes:
 ; OBJ-NEXT:           - I32
 ; OBJ-NEXT:           - I64