[WebAssembly][InstrEmitter] Foundation for multivalue call lowering

[tlively]

Summary

WebAssembly is unique among upstream targets in that it does not at
any point use physical registers to store values. Instead, it uses
virtual registers to model positions in its value stack. This means
that some target-independent lowering activities that would use
physical registers need to use virtual registers instead for
WebAssembly and similar downstream targets. This CL generalizes the
existing usesPhysRegsForPEI lowering hook to
usesPhysRegsForValues in preparation for using it in more places.

One such place is in InstrEmitter for instructions that have variadic
defs. On register machines, it only makes sense for these defs to be
physical registers, but for WebAssembly they must be virtual registers
like any other values. This CL changes InstrEmitter to check the new
target lowering hook to determine whether variadic defs should be
physical or virtual registers.

These changes are necessary to support a generalized CALL instruction
for WebAssembly that is capable of returning an arbitrary number of
arguments. Fully implementing that instruction will require additional
changes that are described in comments here but left for a follow up
commit.

[merge_guards_bot]

Unit tests: pass. 60863 tests passed, 0 failed and 726 were skipped.

clang-format: pass.

Build artifacts: console-log.txt, CMakeCache.txt, test-results.xml, diff.json

[sbc100]

In the description you say " .. unique among upstream targets", and later "... WebAssembly and similar downstream targets.". Was it deliberate to use both "upstream" and "downstream", what do these term mean here? Also the "unique" statement seems to contract the "and similar" statement.

llvm/test/CodeGen/WebAssembly/multivalue.ll

2

Is this test also now testing tail call somehow?

[tlively]

In D71484#1783994, @sbc100 wrote:

In the description you say " .. unique among upstream targets", and later "... WebAssembly and similar downstream targets.". Was it deliberate to use both "upstream" and "downstream", what do these term mean here? Also the "unique" statement seems to contract the "and similar" statement.

Yes, this was deliberate. Upstream targets are targets in the LLVM source tree, and downstream targets are targets that people add to LLVM without checking them into the main repo. An example of a downstream target that is similar to WebAssembly is the TVM architecture for the TON blockchain network.

llvm/test/CodeGen/WebAssembly/multivalue.ll

2

Yes, one of the new tests below has a musttail call, which errors out if tail-call is not enabled.

[aheejin]

Mostly LGTM to me

llvm/lib/CodeGen/SelectionDAG/InstrEmitter.cpp

837

Can't variadic-def call instructions also have implicit defs, such as $argument, $sp32, and $sp64?

llvm/lib/Target/WebAssembly/MCTargetDesc/WebAssemblyMCTargetDesc.h

512–513

I think the intention of D64643 was this file only contains methods that can work on unsigned Opc so that they can work equally on MachineInstrs and MCInsts. I now see that this function is not used for MCInsts now, but I think it'd better to make functions in this file consistently take unsigned Opc?

llvm/lib/Target/WebAssembly/WebAssemblyISelDAGToDAG.cpp

219

Why take out the wrapper?

llvm/lib/Target/WebAssembly/WebAssemblyInstrCall.td

63

Just to clarify, even if variable_ops is within (ins), if variadicOpsAreDefs is 1, are they considered as outputs? What happens if we put variable_ops in (outs)?

[tlively]

llvm/lib/CodeGen/SelectionDAG/InstrEmitter.cpp

837

I suppose they probably could, but I think it would complicate the logic to support mixed virtual and physical variadic and implicit arguments. I'm not sure it's worth trying to do when we don't have any instructions that would test that code path. Maybe @dmitry has a need for such an instruction?

llvm/lib/Target/WebAssembly/MCTargetDesc/WebAssemblyMCTargetDesc.h

512–513

We need the MachineInstr in the new case below because the callee op number depends on the number of defs that come before it. Are you suggesting that we move this function to a different file?

llvm/lib/Target/WebAssembly/WebAssemblyISelDAGToDAG.cpp

219

The wrapper is there to enable pattern matching and is removed by the normal tablegen patterns as well.

llvm/lib/Target/WebAssembly/WebAssemblyInstrCall.td

63

That's correct. Tablegen errors out if you try to put the variable_ops in the (outs). This seems like something that could be cleaned up by someone with sufficient motivation, but that person is not me right now!

[tlively]

• Check isVariadic explicitly in InstrEmitter

[merge_guards_bot]

Unit tests: pass. 60863 tests passed, 0 failed and 726 were skipped.

clang-format: pass.

Build artifacts: diff.json, CMakeCache.txt, console-log.txt, test-results.xml

[tlively]

• Add OpInfo check to avoid OOB when there are only variadic ops

[merge_guards_bot]

Unit tests: pass. 60863 tests passed, 0 failed and 726 were skipped.

clang-format: pass.

Build artifacts: diff.json, CMakeCache.txt, console-log.txt, test-results.xml

[aheejin]

llvm/lib/CodeGen/SelectionDAG/InstrEmitter.cpp

837

Oh what I meant was, every wasm instruction implicitly defines ARGUMENT register, which is treated as a physical register. Turns out SP32 and SP64 are implicitly defined by only these two instructions, so I think this should be irrelevant.

llvm/lib/Target/WebAssembly/MCTargetDesc/WebAssemblyMCTargetDesc.h

512–513

Either that, or leave this function here and writing a couple more line in the user side..?

[tlively]

llvm/lib/CodeGen/SelectionDAG/InstrEmitter.cpp

837

So yes, nodes can implicitly define physical registers, but that is not sufficient to make NumResults > NumDefs, since NumResults is the number of explicit outputs of the node. HasPhysRegOuts is only true when there are explicit ouputs of the node that need to be placed in physical registers. That's why it is correct to set it to false when HasVRegVariadicDefs is true. The implicit physical outputs are not affected by any of this.

llvm/lib/Target/WebAssembly/MCTargetDesc/WebAssemblyMCTargetDesc.h

512–513

I would be ok moving this function to a different file, but I'm not sure which file that would be. This file contains all the helper functions for WebAssembly ops as far as I can tell, so I would also be ok leaving it here. It makes sense to generalize over MachineInstr and MCInst where possible, but that's just not possible in this case.

[tlively]

• Rebase on master

[tlively]

@aheejin, there are no changes I am currently planning to make to this, but I am happy to continue the inline discussions.

[merge_guards_bot]

Unit tests: fail. 61975 tests passed, 1 failed and 783 were skipped.

failed: LLVM.Bindings/Go/go.test

clang-tidy: unknown.

clang-format: pass.

Build artifacts: diff.json, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[aheejin]

Sorry for the delayed reply. I somehow assumed this CL was still in progress and didn't know it was finalized long ago! LGTM.

llvm/lib/Target/WebAssembly/MCTargetDesc/WebAssemblyMCTargetDesc.h

512–513

I'd be also OK with leaving this function in this file in the current state, but isn't this function gonna be reduced to ~5 lines after we delete all other CALL_VOID and CALL_type variants? This function may not be really about opcodes by that point then. How about moving this to WebAssemblyUtilities.cpp/h or something?

[tlively]

In D71484#1831640, @aheejin wrote:

Sorry for the delayed reply. I somehow assumed this CL was still in progress and didn't know it was finalized long ago! LGTM.

No problem at all! I wasn't sure whether I would want to change it until I commented.

[tlively]

llvm/lib/Target/WebAssembly/MCTargetDesc/WebAssemblyMCTargetDesc.h

512–513

Yes, it will become much shorter. WebAssemblyUtilities sounds good.

llvm/include/llvm/Target/TargetMachine.h

public:
   virtual bool isMachineVerifierClean() const { return true; }
 
   /// Adds an AsmPrinter pass to the pipeline that prints assembly or
   /// machine code from the MI representation.
   bool addAsmPrinter(PassManagerBase &PM, raw_pwrite_stream &Out,
                      raw_pwrite_stream *DwoOut, CodeGenFileType FileType,
                      MCContext &Context);
 
-  /// True if the target uses physical regs at Prolog/Epilog insertion
-  /// time. If true (most machines), all vregs must be allocated before
-  /// PEI. If false (virtual-register machines), then callee-save register
-  /// spilling and scavenging are not needed or used.
-  virtual bool usesPhysRegsForPEI() const { return true; }
+  /// True if the target uses physical regs (as nearly all targets do). False
+  /// for stack machines such as WebAssembly and other virtual-register
+  /// machines. If true, all vregs must be allocated before PEI. If false, then
+  /// callee-save register spilling and scavenging are not needed or used. If
+  /// false, implicitly defined registers will still be assumed to be physical
+  /// registers, except that variadic defs will be allocated vregs.
+  virtual bool usesPhysRegsForValues() const { return true; }
 
   /// True if the target wants to use interprocedural register allocation by
   /// default. The -enable-ipra flag can be used to override this.
   virtual bool useIPRA() const {
     return false;
   }
 };
 

llvm/lib/CodeGen/PrologEpilogInserter.cpp

bool PEI::runOnMachineFunction(MachineFunction &MF) {
   calculateSaveRestoreBlocks(MF);
 
   // Stash away DBG_VALUEs that should not be moved by insertion of prolog code.
   SavedDbgValuesMap EntryDbgValues;
   for (MachineBasicBlock *SaveBlock : SaveBlocks)
     stashEntryDbgValues(*SaveBlock, EntryDbgValues);
 
   // Handle CSR spilling and restoring, for targets that need it.
-  if (MF.getTarget().usesPhysRegsForPEI())
+  if (MF.getTarget().usesPhysRegsForValues())
     spillCalleeSavedRegs(MF);
 
   // Allow the target machine to make final modifications to the function
   // before the frame layout is finalized.
   TFI->processFunctionBeforeFrameFinalized(MF, RS);
 
   // Calculate actual frame offsets for all abstract stack objects...
   calculateFrameObjectOffsets(MF);

llvm/lib/CodeGen/SelectionDAG/InstrEmitter.cpp

void InstrEmitter::CreateVirtualRegisters(SDNode *Node,
                                        MachineInstrBuilder &MIB,
                                        const MCInstrDesc &II,
                                        bool IsClone, bool IsCloned,
                                        DenseMap<SDValue, unsigned> &VRBaseMap) {
   assert(Node->getMachineOpcode() != TargetOpcode::IMPLICIT_DEF &&
          "IMPLICIT_DEF should have been handled as a special case elsewhere!");
 
   unsigned NumResults = CountResults(Node);
-  for (unsigned i = 0; i < II.getNumDefs(); ++i) {
+  bool HasVRegVariadicDefs = !MF->getTarget().usesPhysRegsForValues() &&
+                             II.isVariadic() && II.variadicOpsAreDefs();
+  unsigned NumVRegs = HasVRegVariadicDefs ? NumResults : II.getNumDefs();
+  for (unsigned i = 0; i < NumVRegs; ++i) {
     // If the specific node value is only used by a CopyToReg and the dest reg
     // is a vreg in the same register class, use the CopyToReg'd destination
     // register instead of creating a new vreg.
     unsigned VRBase = 0;
     const TargetRegisterClass *RC =
       TRI->getAllocatableClass(TII->getRegClass(II, i, TRI, *MF));
     // Always let the value type influence the used register class. The
     // constraints on the instruction may be too lax to represent the value
     // type correctly. For example, a 64-bit float (X86::FR64) can't live in
     // the 32-bit float super-class (X86::FR32).
     if (i < NumResults && TLI->isTypeLegal(Node->getSimpleValueType(i))) {
       const TargetRegisterClass *VTRC = TLI->getRegClassFor(
           Node->getSimpleValueType(i),
           (Node->isDivergent() || (RC && TRI->isDivergentRegClass(RC))));
       if (RC)
         VTRC = TRI->getCommonSubClass(RC, VTRC);
       if (VTRC)
         RC = VTRC;
     }
 
-    if (II.OpInfo[i].isOptionalDef()) {
+    if (II.OpInfo != nullptr && II.OpInfo[i].isOptionalDef()) {
       // Optional def must be a physical register.
       VRBase = cast<RegisterSDNode>(Node->getOperand(i-NumResults))->getReg();
       assert(Register::isPhysicalRegister(VRBase));
       MIB.addReg(VRBase, RegState::Define);
     }
 
     if (!VRBase && !IsClone && !IsCloned)
       for (SDNode *User : Node->uses()) {
if (Opc == TargetOpcode::PATCHPOINT) {
       NumDefs = NumResults;
     }
     ScratchRegs = TLI->getScratchRegisters((CallingConv::ID) CC);
   }
 
   unsigned NumImpUses = 0;
   unsigned NodeOperands =
     countOperands(Node, II.getNumOperands() - NumDefs, NumImpUses);
-  bool HasPhysRegOuts = NumResults > NumDefs && II.getImplicitDefs()!=nullptr;
+  bool HasVRegVariadicDefs = !MF->getTarget().usesPhysRegsForValues() &&
+                             II.isVariadic() && II.variadicOpsAreDefs();
+  bool HasPhysRegOuts = NumResults > NumDefs &&
+                        II.getImplicitDefs() != nullptr && !HasVRegVariadicDefs;
 #ifndef NDEBUG
   unsigned NumMIOperands = NodeOperands + NumResults;
   if (II.isVariadic())
     assert(NumMIOperands >= II.getNumOperands() &&
            "Too few operands for a variadic node!");
   else
     assert(NumMIOperands >= II.getNumOperands() &&
            NumMIOperands <= II.getNumOperands() + II.getNumImplicitDefs() +

llvm/lib/Target/WebAssembly/MCTargetDesc/WebAssemblyMCTargetDesc.h

case WebAssembly::RET_CALL_INDIRECT_S:
     return true;
   default:
     return false;
   }
 }
 
 /// Returns the operand number of a callee, assuming the argument is a call
 /// instruction.
-inline unsigned getCalleeOpNo(unsigned Opc) {
-  switch (Opc) {
+inline const MachineOperand &getCalleeOp(const MachineInstr &MI) {
+  switch (MI.getOpcode()) {
   case WebAssembly::CALL_VOID:
   case WebAssembly::CALL_VOID_S:
   case WebAssembly::CALL_INDIRECT_VOID:
   case WebAssembly::CALL_INDIRECT_VOID_S:
   case WebAssembly::RET_CALL:
   case WebAssembly::RET_CALL_S:
   case WebAssembly::RET_CALL_INDIRECT:
   case WebAssembly::RET_CALL_INDIRECT_S:
-    return 0;
+    return MI.getOperand(0);
   case WebAssembly::CALL_i32:
   case WebAssembly::CALL_i32_S:
   case WebAssembly::CALL_i64:
   case WebAssembly::CALL_i64_S:
   case WebAssembly::CALL_f32:
   case WebAssembly::CALL_f32_S:
   case WebAssembly::CALL_f64:
   case WebAssembly::CALL_f64_S:
inline const MachineOperand &getCalleeOp(const MachineInstr &MI) {
   case WebAssembly::CALL_INDIRECT_v2i64:
   case WebAssembly::CALL_INDIRECT_v2i64_S:
   case WebAssembly::CALL_INDIRECT_v4f32:
   case WebAssembly::CALL_INDIRECT_v4f32_S:
   case WebAssembly::CALL_INDIRECT_v2f64:
   case WebAssembly::CALL_INDIRECT_v2f64_S:
   case WebAssembly::CALL_INDIRECT_exnref:
   case WebAssembly::CALL_INDIRECT_exnref_S:
-    return 1;
+    return MI.getOperand(1);
+  case WebAssembly::CALL:
+  case WebAssembly::CALL_S:
+    return MI.getOperand(MI.getNumDefs());
   default:
     llvm_unreachable("Not a call instruction");
   }
 }
 
 inline bool isMarker(unsigned Opc) {
   switch (Opc) {
   case WebAssembly::BLOCK:

llvm/lib/Target/WebAssembly/WebAssemblyISD.def

 /// This file describes the various WebAssembly ISD node types.
 ///
 //===----------------------------------------------------------------------===//
 
 // NOTE: NO INCLUDE GUARD DESIRED!
 
 HANDLE_NODETYPE(CALL1)
 HANDLE_NODETYPE(CALL0)
+HANDLE_NODETYPE(CALL)
 HANDLE_NODETYPE(RET_CALL)
 HANDLE_NODETYPE(RETURN)
 HANDLE_NODETYPE(ARGUMENT)
 // A wrapper node for TargetExternalSymbol, TargetGlobalAddress, and MCSymbol
 HANDLE_NODETYPE(Wrapper)
 // A special wapper used in PIC code for __memory_base/__table_base relcative
 // access.
 HANDLE_NODETYPE(WrapperPIC)

llvm/lib/Target/WebAssembly/WebAssemblyISelDAGToDAG.cpp

case Intrinsic::wasm_tls_base: {
           CurDAG->getTargetExternalSymbol("__tls_base", PtrVT),
           Node->getOperand(0));
       ReplaceNode(Node, TLSBase);
       return;
     }
     }
     break;
   }
+  case WebAssemblyISD::CALL: {
+    // CALL has both variable operands and variable results, but ISel only
+    // supports one or the other. Split calls into two nodes glued together, one
+    // for the operands and one for the results. These two nodes will be
+    // recombined in a custom inserter hook.
+    // TODO: Split CALL
+    SmallVector<SDValue, 16> Ops;
+    for (size_t i = 1; i < Node->getNumOperands(); ++i) {
+      SDValue Op = Node->getOperand(i);
+      if (Op->getOpcode() == WebAssemblyISD::Wrapper)
+        Op = Op->getOperand(0);
+      Ops.push_back(Op);
+    }
+    Ops.push_back(Node->getOperand(0));
+    MachineSDNode *Call =
+        CurDAG->getMachineNode(WebAssembly::CALL, DL, Node->getVTList(), Ops);
+    ReplaceNode(Node, Call);
+    return;
+  }
 
   default:
     break;
   }
 
   // Select the default instruction.
   SelectCode(Node);
 }

llvm/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

if (Subtarget->hasTailCall() && !CLI.IsVarArg) {
         } else {
           fail(DL, DAG, "WebAssembly 'tail-call' feature not enabled");
         }
       }
     }
   }
 
   SmallVectorImpl<ISD::InputArg> &Ins = CLI.Ins;
-  if (Ins.size() > 1)
-    fail(DL, DAG, "WebAssembly doesn't support more than 1 returned value yet");
-
   SmallVectorImpl<ISD::OutputArg> &Outs = CLI.Outs;
   SmallVectorImpl<SDValue> &OutVals = CLI.OutVals;
 
   // The generic code may have added an sret argument. If we're lowering an
   // invoke function, the ABI requires that the function pointer be the first
   // argument, so we may have to swap the arguments.
   if (CallConv == CallingConv::WASM_EmscriptenInvoke && Outs.size() >= 2 &&
       Outs[0].Flags.isSRet()) {
WebAssemblyTargetLowering::LowerCall(CallLoweringInfo &CLI,
 
   if (CLI.IsTailCall) {
     // ret_calls do not return values to the current frame
     SDVTList NodeTys = DAG.getVTList(MVT::Other, MVT::Glue);
     return DAG.getNode(WebAssemblyISD::RET_CALL, DL, NodeTys, Ops);
   }
 
   InTys.push_back(MVT::Other);
-  SDVTList InTyList = DAG.getVTList(InTys);
-  SDValue Res =
-      DAG.getNode(Ins.empty() ? WebAssemblyISD::CALL0 : WebAssemblyISD::CALL1,
-                  DL, InTyList, Ops);
-  if (Ins.empty()) {
-    Chain = Res;
-  } else {
-    InVals.push_back(Res);
-    Chain = Res.getValue(1);
+  unsigned Opc;
+  // TODO: Remove CALL0 and CALL1 in favor of CALL
+  switch (Ins.size()) {
+  case 0:
+    Opc = WebAssemblyISD::CALL0;
+    break;
+  case 1:
+    Opc = WebAssemblyISD::CALL1;
+    break;
+  default:
+    Opc = WebAssemblyISD::CALL;
+    break;
   }
+  SDVTList InTyList = DAG.getVTList(InTys);
+  SDValue Res = DAG.getNode(Opc, DL, InTyList, Ops);
 
-  return Chain;
+  for (size_t I = 0; I < Ins.size(); ++I)
+    InVals.push_back(Res.getValue(I));
+
+  // Return the chain
+  return Res.getValue(Ins.size());
 }
 
 bool WebAssemblyTargetLowering::CanLowerReturn(
     CallingConv::ID /*CallConv*/, MachineFunction & /*MF*/, bool /*IsVarArg*/,
     const SmallVectorImpl<ISD::OutputArg> &Outs,
     LLVMContext & /*Context*/) const {
   // WebAssembly can only handle returning tuples with multivalue enabled
   return Subtarget->hasMultivalue() || Outs.size() <= 1;

llvm/lib/Target/WebAssembly/WebAssemblyInstrCall.td

I<(outs rt:$dst),
       [],
       !strconcat(prefix, "call_indirect\t$dst"),
       !strconcat(prefix, "call_indirect\t$type"),
       0x11>,
     Requires<preds>;
 }
 
 let Uses = [SP32, SP64], isCall = 1 in {
+
+// TODO: Split CALL into separate nodes for operands and results.
+// TODO: Add an indirect version of the variadic call, delete CALL_*
+let variadicOpsAreDefs = 1 in
+defm CALL :
+  I<(outs), (ins function32_op:$callee, variable_ops),
+    (outs), (ins function32_op:$callee), [],
+    "call    \t$callee", "call\t$callee", 0x10>;
+
 defm "" : CALL<i32, I32, "i32.">;
 defm "" : CALL<i64, I64, "i64.">;
 defm "" : CALL<f32, F32, "f32.">;
 defm "" : CALL<f64, F64, "f64.">;
 defm "" : CALL<exnref, EXNREF, "exnref.", [HasExceptionHandling]>;
 defm "" : CALL<v16i8, V128, "v128.", [HasSIMD128]>;
 defm "" : CALL<v8i16, V128, "v128.", [HasSIMD128]>;
 defm "" : CALL<v4i32, V128, "v128.", [HasSIMD128]>;
 defm "" : CALL<v2i64, V128, "v128.", [HasSIMD128]>;
 defm "" : CALL<v4f32, V128, "v128.", [HasSIMD128]>;
 defm "" : CALL<v2f64, V128, "v128.", [HasSIMD128]>;
 
 let IsCanonical = 1 in {
+
 defm CALL_VOID :
   I<(outs), (ins function32_op:$callee, variable_ops),
     (outs), (ins function32_op:$callee),
     [(WebAssemblycall0 (i32 imm:$callee))],
     "call    \t$callee", "call\t$callee", 0x10>;
 
 let isReturn = 1 in
 defm RET_CALL :

llvm/lib/Target/WebAssembly/WebAssemblyRegStackify.cpp

static void convertImplicitDefToConstZero(MachineInstr *MI,
   } else {
     llvm_unreachable("Unexpected reg class");
   }
 }
 
 // Determine whether a call to the callee referenced by
 // MI->getOperand(CalleeOpNo) reads memory, writes memory, and/or has side
 // effects.
-static void queryCallee(const MachineInstr &MI, unsigned CalleeOpNo, bool &Read,
-                        bool &Write, bool &Effects, bool &StackPointer) {
+static void queryCallee(const MachineInstr &MI, bool &Read, bool &Write,
+                        bool &Effects, bool &StackPointer) {
   // All calls can use the stack pointer.
   StackPointer = true;
 
-  const MachineOperand &MO = MI.getOperand(CalleeOpNo);
+  const MachineOperand &MO = WebAssembly::getCalleeOp(MI);
   if (MO.isGlobal()) {
     const Constant *GV = MO.getGlobal();
     if (const auto *GA = dyn_cast<GlobalAlias>(GV))
       if (!GA->isInterposable())
         GV = GA->getAliasee();
 
     if (const auto *F = dyn_cast<Function>(GV)) {
       if (!F->doesNotThrow())
static void query(const MachineInstr &MI, AliasAnalysis &AA, bool &Read,
 
   // Check for writes to __stack_pointer global.
   if (MI.getOpcode() == WebAssembly::GLOBAL_SET_I32 &&
       strcmp(MI.getOperand(0).getSymbolName(), "__stack_pointer") == 0)
     StackPointer = true;
 
   // Analyze calls.
   if (MI.isCall()) {
-    unsigned CalleeOpNo = WebAssembly::getCalleeOpNo(MI.getOpcode());
-    queryCallee(MI, CalleeOpNo, Read, Write, Effects, StackPointer);
+    queryCallee(MI, Read, Write, Effects, StackPointer);
   }
 }
 
 // Test whether Def is safe and profitable to rematerialize.
 static bool shouldRematerialize(const MachineInstr &Def, AliasAnalysis &AA,
                                 const WebAssemblyInstrInfo *TII) {
   return Def.isAsCheapAsAMove() && TII->isTriviallyReMaterializable(Def, &AA);
 }

llvm/lib/Target/WebAssembly/WebAssemblyTargetMachine.h

public:
   TargetPassConfig *createPassConfig(PassManagerBase &PM) override;
 
   TargetLoweringObjectFile *getObjFileLowering() const override {
     return TLOF.get();
   }
 
   TargetTransformInfo getTargetTransformInfo(const Function &F) override;
 
-  bool usesPhysRegsForPEI() const override { return false; }
+  bool usesPhysRegsForValues() const override { return false; }
 
   yaml::MachineFunctionInfo *createDefaultFuncInfoYAML() const override;
   yaml::MachineFunctionInfo *
   convertFuncInfoToYAML(const MachineFunction &MF) const override;
   bool parseMachineFunctionInfo(const yaml::MachineFunctionInfo &,
                                 PerFunctionMIParsingState &PFS,
                                 SMDiagnostic &Error,
                                 SMRange &SourceRange) const override;
 };
 
 } // end namespace llvm
 
 #endif

llvm/lib/Target/WebAssembly/WebAssemblyUtilities.h

 template <typename T> MachineBasicBlock *getBottom(const T *Unit) {
   MachineBasicBlock *Bottom = Unit->getHeader();
   for (MachineBasicBlock *MBB : Unit->blocks())
     if (MBB->getNumber() > Bottom->getNumber())
       Bottom = MBB;
   return Bottom;
 }
 
+/// Returns the operand number of a callee, assuming the argument is a call
+/// instruction.
+const MachineOperand &getCalleeOp(const MachineInstr &MI);
+
 } // end namespace WebAssembly
 
 } // end namespace llvm
 
 #endif

llvm/lib/Target/WebAssembly/WebAssemblyUtilities.cpp

bool WebAssembly::mayThrow(const MachineInstr &MI) {
   case WebAssembly::RETHROW_S:
     return true;
   }
   if (isCallIndirect(MI.getOpcode()))
     return true;
   if (!MI.isCall())
     return false;
 
-  const MachineOperand &MO = MI.getOperand(getCalleeOpNo(MI.getOpcode()));
+  const MachineOperand &MO = getCalleeOp(MI);
   assert(MO.isGlobal() || MO.isSymbol());
 
   if (MO.isSymbol()) {
     // Some intrinsics are lowered to calls to external symbols, which are then
     // lowered to calls to library functions. Most of libcalls don't throw, but
     // we only list some of them here now.
     // TODO Consider adding 'nounwind' info in TargetLowering::CallLoweringInfo
     // instead for more accurate info.
bool WebAssembly::mayThrow(const MachineInstr &MI) {
   if (F->getName() == CxaBeginCatchFn || F->getName() == PersonalityWrapperFn ||
       F->getName() == ClangCallTerminateFn || F->getName() == StdTerminateFn)
     return false;
 
   // TODO Can we exclude call instructions that are marked as 'nounwind' in the
   // original LLVm IR? (Even when the callee may throw)
   return true;
 }
+
+inline const MachineOperand &getCalleeOp(const MachineInstr &MI) {
+  switch (MI.getOpcode()) {
+  case WebAssembly::CALL_VOID:
+  case WebAssembly::CALL_VOID_S:
+  case WebAssembly::CALL_INDIRECT_VOID:
+  case WebAssembly::CALL_INDIRECT_VOID_S:
+  case WebAssembly::RET_CALL:
+  case WebAssembly::RET_CALL_S:
+  case WebAssembly::RET_CALL_INDIRECT:
+  case WebAssembly::RET_CALL_INDIRECT_S:
+    return MI.getOperand(0);
+  case WebAssembly::CALL_i32:
+  case WebAssembly::CALL_i32_S:
+  case WebAssembly::CALL_i64:
+  case WebAssembly::CALL_i64_S:
+  case WebAssembly::CALL_f32:
+  case WebAssembly::CALL_f32_S:
+  case WebAssembly::CALL_f64:
+  case WebAssembly::CALL_f64_S:
+  case WebAssembly::CALL_v16i8:
+  case WebAssembly::CALL_v16i8_S:
+  case WebAssembly::CALL_v8i16:
+  case WebAssembly::CALL_v8i16_S:
+  case WebAssembly::CALL_v4i32:
+  case WebAssembly::CALL_v4i32_S:
+  case WebAssembly::CALL_v2i64:
+  case WebAssembly::CALL_v2i64_S:
+  case WebAssembly::CALL_v4f32:
+  case WebAssembly::CALL_v4f32_S:
+  case WebAssembly::CALL_v2f64:
+  case WebAssembly::CALL_v2f64_S:
+  case WebAssembly::CALL_exnref:
+  case WebAssembly::CALL_exnref_S:
+  case WebAssembly::CALL_INDIRECT_i32:
+  case WebAssembly::CALL_INDIRECT_i32_S:
+  case WebAssembly::CALL_INDIRECT_i64:
+  case WebAssembly::CALL_INDIRECT_i64_S:
+  case WebAssembly::CALL_INDIRECT_f32:
+  case WebAssembly::CALL_INDIRECT_f32_S:
+  case WebAssembly::CALL_INDIRECT_f64:
+  case WebAssembly::CALL_INDIRECT_f64_S:
+  case WebAssembly::CALL_INDIRECT_v16i8:
+  case WebAssembly::CALL_INDIRECT_v16i8_S:
+  case WebAssembly::CALL_INDIRECT_v8i16:
+  case WebAssembly::CALL_INDIRECT_v8i16_S:
+  case WebAssembly::CALL_INDIRECT_v4i32:
+  case WebAssembly::CALL_INDIRECT_v4i32_S:
+  case WebAssembly::CALL_INDIRECT_v2i64:
+  case WebAssembly::CALL_INDIRECT_v2i64_S:
+  case WebAssembly::CALL_INDIRECT_v4f32:
+  case WebAssembly::CALL_INDIRECT_v4f32_S:
+  case WebAssembly::CALL_INDIRECT_v2f64:
+  case WebAssembly::CALL_INDIRECT_v2f64_S:
+  case WebAssembly::CALL_INDIRECT_exnref:
+  case WebAssembly::CALL_INDIRECT_exnref_S:
+    return MI.getOperand(1);
+  case WebAssembly::CALL:
+  case WebAssembly::CALL_S:
+    return MI.getOperand(MI.getNumDefs());
+  default:
+    llvm_unreachable("Not a call instruction");
+  }
+}

llvm/test/CodeGen/WebAssembly/multivalue.ll

-; RUN: llc < %s -asm-verbose=false -verify-machineinstrs -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -wasm-keep-registers -mattr=+multivalue | FileCheck %s
-; RUN: llc < %s --filetype=obj -mattr=+multivalue | obj2yaml | FileCheck %s --check-prefix OBJ
+; RUN: llc < %s -asm-verbose=false -verify-machineinstrs -disable-wasm-fallthrough-return-opt -wasm-disable-explicit-locals -wasm-keep-registers -mattr=+multivalue,+tail-call | FileCheck %s
+; RUN: llc < %s --filetype=obj -mattr=+multivalue,+tail-call | obj2yaml | FileCheck %s --check-prefix OBJ
 
-; Test that the multivalue returns, function types, and block types
-; work as expected.
+; Test that the multivalue calls, returns, function types, and block
+; types work as expected.
 
 target datalayout = "e-m:e-p:32:32-i64:64-n32:64-S128"
 target triple = "wasm32-unknown-unknown"
 
-%pair = type { i32, i32 }
-%packed_pair = type <{ i32, i32 }>
+%pair = type { i32, i64 }
+%packed_pair = type <{ i32, i64 }>
+
+
+; CHECK-LABEL: pair_const:
+; CHECK-NEXT: .functype pair_const () -> (i32, i64)
+; CHECK-NEXT: i32.const $push[[L0:[0-9]+]]=, 42{{$}}
+; CHECK-NEXT: i64.const $push[[L1:[0-9]+]]=, 42{{$}}
+; CHECK-NEXT: return $pop[[L0]], $pop[[L1]]{{$}}
+define %pair @pair_const() {
+  ret %pair { i32 42, i64 42 }
+}
+
+; CHECK-LABEL: packed_pair_const:
+; CHECK-NEXT: .functype packed_pair_const () -> (i32, i64)
+; CHECK-NEXT: i32.const $push[[L0:[0-9]+]]=, 42{{$}}
+; CHECK-NEXT: i64.const $push[[L1:[0-9]+]]=, 42{{$}}
+; CHECK-NEXT: return $pop[[L0]], $pop[[L1]]{{$}}
+define %packed_pair @packed_pair_const() {
+  ret %packed_pair <{ i32 42, i64 42 }>
+}
 
 ; CHECK-LABEL: pair_ident:
-; CHECK-NEXT: .functype pair_ident (i32, i32) -> (i32, i32)
+; CHECK-NEXT: .functype pair_ident (i32, i64) -> (i32, i64)
 ; CHECK-NEXT: return $0, $1{{$}}
 define %pair @pair_ident(%pair %p) {
   ret %pair %p
 }
 
 ; CHECK-LABEL: packed_pair_ident:
-; CHECK-NEXT: .functype packed_pair_ident (i32, i32) -> (i32, i32)
+; CHECK-NEXT: .functype packed_pair_ident (i32, i64) -> (i32, i64)
 ; CHECK-NEXT: return $0, $1{{$}}
 define %packed_pair @packed_pair_ident(%packed_pair %p) {
   ret %packed_pair %p
 }
 
+;; TODO: Multivalue calls are a WIP and do not necessarily produce
+;; correct output. For now, just check that they don't cause any
+;; crashes.
+
+define void @pair_call() {
+  %p = call %pair @pair_const()
+  ret void
+}
+
+define void @packed_pair_call() {
+  %p = call %packed_pair @packed_pair_const()
+  ret void
+}
+
+define %pair @pair_call_return() {
+  %p = call %pair @pair_const()
+  ret %pair %p
+}
+
+define %packed_pair @packed_pair_call_return() {
+  %p = call %packed_pair @packed_pair_const()
+  ret %packed_pair %p
+}
+
+define %pair @pair_tail_call() {
+  %p = musttail call %pair @pair_const()
+  ret %pair %p
+}
+
+define %packed_pair @packed_pair_tail_call() {
+  %p = musttail call %packed_pair @packed_pair_const()
+  ret %packed_pair %p
+}
+
+define i32 @pair_call_return_first() {
+  %p = call %pair @pair_const()
+  %v = extractvalue %pair %p, 0
+  ret i32 %v
+}
+
+define i32 @packed_pair_call_return_first() {
+  %p = call %packed_pair @packed_pair_const()
+  %v = extractvalue %packed_pair %p, 0
+  ret i32 %v
+}
+
+define i64 @pair_call_return_second() {
+  %p = call %pair @pair_const()
+  %v = extractvalue %pair %p, 1
+  ret i64 %v
+}
+
+define i64 @packed_pair_call_return_second() {
+  %p = call %packed_pair @packed_pair_const()
+  %v = extractvalue %packed_pair %p, 1
+  ret i64 %v
+}
+
+define %pair @pair_pass_through(%pair %p) {
+  %r = call %pair @pair_ident(%pair %p)
+  ret %pair %r
+}
+
+define %packed_pair @packed_pair_pass_through(%packed_pair %p) {
+  %r = call %packed_pair @packed_pair_ident(%packed_pair %p)
+  ret %packed_pair %r
+}
+
 ; CHECK-LABEL: minimal_loop:
 ; CHECK-NEXT: .functype minimal_loop (i32) -> (i32, i64)
 ; CHECK-NEXT: .LBB{{[0-9]+}}_1:
 ; CHECK-NEXT: loop () -> (i32, i64)
 ; CHECK-NEXT: br 0{{$}}
 ; CHECK-NEXT: .LBB{{[0-9]+}}_2:
 ; CHECK-NEXT: end_loop{{$}}
-define {i32, i64} @minimal_loop(i32* %p) {
+define %pair @minimal_loop(i32* %p) {
 entry:
   br label %loop
 loop:
   br label %loop
 }
 
 ; CHECK-LABEL: .section .custom_section.target_features
-; CHECK-NEXT: .int8 1
+; CHECK-NEXT: .int8 2
 ; CHECK-NEXT: .int8 43
 ; CHECK-NEXT: .int8 10
 ; CHECK-NEXT: .ascii "multivalue"
+; CHECK-NEXT: .int8 43
+; CHECK-NEXT: .int8 9
+; CHECK-NEXT: .ascii "tail-call"
 
 ; OBJ-LABEL:  - Type:            TYPE
 ; OBJ-NEXT:     Signatures:
 ; OBJ-NEXT:       - Index:           0
-; OBJ-NEXT:         ParamTypes:
+; OBJ-NEXT:         ParamTypes:      []
+; OBJ-NEXT:         ReturnTypes:
 ; OBJ-NEXT:           - I32
+; OBJ-NEXT:           - I64
+; OBJ-NEXT:       - Index:           1
+; OBJ-NEXT:         ParamTypes:
 ; OBJ-NEXT:           - I32
+; OBJ-NEXT:           - I64
 ; OBJ-NEXT:         ReturnTypes:
 ; OBJ-NEXT:           - I32
+; OBJ-NEXT:           - I64
+; OBJ-NEXT:       - Index:           2
+; OBJ-NEXT:         ParamTypes:      []
+; OBJ-NEXT:         ReturnTypes:     []
+; OBJ-NEXT:       - Index:           3
+; OBJ-NEXT:         ParamTypes:      []
+; OBJ-NEXT:         ReturnTypes:
 ; OBJ-NEXT:           - I32
-; OBJ-NEXT:       - Index:           1
+; OBJ-NEXT:       - Index:           4
+; OBJ-NEXT:         ParamTypes:      []
+; OBJ-NEXT:         ReturnTypes:
+; OBJ-NEXT:           - I64
+; OBJ-NEXT:       - Index:           5
 ; OBJ-NEXT:         ParamTypes:
 ; OBJ-NEXT:           - I32
 ; OBJ-NEXT:         ReturnTypes:
 ; OBJ-NEXT:           - I32
 ; OBJ-NEXT:           - I64