[WASM] SIMD128 support.

[jpp]

Summary

Kicks off the implementation of wasm SIMD128 support (spec: https://github.com/stoklund/portable-simd/blob/master/portable-simd.md), adding support for add, sub, mul for i8x16, i16x8, i32x4, and f32x4.

The spec is WIP, and might change in the near future.

[dschuff]

lib/Target/WebAssembly/WebAssemblyAsmPrinter.cpp

103 ↗

(On Diff #65089)

XXX Do we want to make this helper unconditionally return the SIMD type and put the Subtarget check elsewhere?

lib/Target/WebAssembly/WebAssemblyInstrCall.td

38 ↗

(On Diff #65089)

Should the predicate go outside the multiclass?

lib/Target/WebAssembly/WebAssemblyInstrSIMD.td

15 ↗

(On Diff #65089)

Actually, I'm a bit unclear about Predicates [HasSIMD128] vs Requires<[HasSIMD128]>. Why do we need both here?

lib/Target/WebAssembly/WebAssemblyRegNumbering.cpp

78 ↗

(On Diff #65089)

Should this be an assert instead?

[sunfish]

lib/Target/WebAssembly/WebAssemblySubtarget.cpp

69 ↗

(On Diff #65089)

It's not necessary to check HasSIMD128 here, because we shouldn't have created an ARGUMENT_v16i8 etc. if we don't have HasSIMD128 in the first place :-). And without that check, this function doesn't need to be a member of WebAssemblySubtarget.

The same applies to other hasSIMD128() checks in the patch as well.

[jpp]

lib/Target/WebAssembly/WebAssemblySubtarget.cpp

69 ↗

(On Diff #65089)

I was debating whether I should add all these hasSIMD128() checks everywhere or not, and then I decided to be pedantic. Should I remove them all?

[sunfish]

lib/Target/WebAssembly/WebAssemblySubtarget.cpp

69 ↗

(On Diff #65089)

I favor removing them. We have to think about the subtarget restrictions when creating new instructions, but it's nice for code that's just reading instructions to just handle all of wasm.

[jpp]

lib/Target/WebAssembly/WebAssemblyAsmPrinter.cpp

103 ↗

(On Diff #65089)

The new patch does not check for hasSIMD128 in non-instruction-selection related files.

lib/Target/WebAssembly/WebAssemblyInstrCall.td

38 ↗

(On Diff #65089)

I changed the SIMD instructions to be SIMD_I instead of plain I, and SIM_I Requires<[HasSIMD128]>;

lib/Target/WebAssembly/WebAssemblyRegNumbering.cpp

78 ↗

(On Diff #65089)

not relevant anymore.

[jpp]

Addresses the comments.

[dschuff]

lib/Target/WebAssembly/WebAssemblyInstrCall.td

40 ↗

(On Diff #65576)

The indentation here (and line 45 and 70 and 107) looks funny.

lib/Target/WebAssembly/WebAssemblyInstrInfo.td

113 ↗

(On Diff #65576)

Oh, one other thing that's a little unclear to me. Why do we need ARGUMENT_v16i8 et al instead of just ARGUMENT_V128? I guess it's because it corresponds to MVTs which can be vectors, instead of register classes? Register classes would seem to make more sense (and the other ARGUMENT multiclass is based on WebAssemblyRegClass?)

[jpp]

lib/Target/WebAssembly/WebAssemblyInstrInfo.td

113 ↗

(On Diff #65576)

I tried doing that before, but it upsets tablegen

llvm-tblgen: /usr/local/google/home/jpp/work/llvm/llvm/utils/TableGen/CodeGenDAGPatterns.h:74: llvm::MVT::SimpleValueType llvm::EEVT::TypeSet::getConcrete() const: Assertion `isConcrete() && "Type isn't concrete yet"' failed.

As far as I can tell, the problem here is that the V128 register class accommodates multiple MVT, while the other register classes deal with single MVTs.

[jpp]

lib/Target/WebAssembly/WebAssemblyInstrInfo.td

113 ↗

(On Diff #65804)

If I use ARGUMENT<V128> instead of SIMD_ARGUMENT<V128>, then tablegen compains

In ARGUMENT_V128: Could not infer all types in pattern!

But this is at least not an assertion. :)

[dschuff]

OK, I think this can go in. We can try to optimize the fake ARGUMENT registers as we go. I tried a few random things but didn't find out anything interesting. Not sure if @sunfish has any ideas about that? (see comments in WebAssemblyInstrInfo.td)

[sunfish]

LGTM.

lib/Target/WebAssembly/WebAssemblyInstrInfo.td

113 ↗

(On Diff #65804)

I think your analysis of the problem here is correct. The code here looks reasonable.

lib/Target/WebAssembly/WebAssemblyPeephole.cpp

192 ↗

(On Diff #65804)

These hasSIMD128() checks are also redundant, along the lines of my earlier comment.

llvm/trunk/lib/Target/WebAssembly/InstPrinter/WebAssemblyInstPrinter.cpp

const char *llvm::WebAssembly::TypeToString(MVT Ty) {
   case MVT::i32:
     return "i32";
   case MVT::i64:
     return "i64";
   case MVT::f32:
     return "f32";
   case MVT::f64:
     return "f64";
+  case MVT::v16i8:
+  case MVT::v8i16:
+  case MVT::v4i32:
+  case MVT::v4f32:
+    return "v128";
   default:
     llvm_unreachable("unsupported type");
   }
 }

llvm/trunk/lib/Target/WebAssembly/WebAssembly.td

 //===----------------------------------------------------------------------===//
 
 include "llvm/Target/Target.td"
 
 //===----------------------------------------------------------------------===//
 // WebAssembly Subtarget features.
 //===----------------------------------------------------------------------===//
 
-def FeatureSIMD128 : SubtargetFeature<"simd128", "HasSIMD128", "false",
+def FeatureSIMD128 : SubtargetFeature<"simd128", "HasSIMD128", "true",
                                       "Enable 128-bit SIMD">;
 
 //===----------------------------------------------------------------------===//
 // Architectures.
 //===----------------------------------------------------------------------===//
 
 //===----------------------------------------------------------------------===//
 // Register File Description

llvm/trunk/lib/Target/WebAssembly/WebAssemblyArgumentMove.cpp

 /// respect this is the ScheduleDAG scheduler. This pass is therefore run
 /// immediately after that.
 ///
 /// This is all hopefully a temporary solution until we find a better solution
 /// for describing the live-in nature of arguments.
 ///
 //===----------------------------------------------------------------------===//
 
-#include "WebAssembly.h"
 #include "MCTargetDesc/WebAssemblyMCTargetDesc.h"
+#include "WebAssembly.h"
 #include "WebAssemblyMachineFunctionInfo.h"
+#include "WebAssemblySubtarget.h"
 #include "llvm/CodeGen/MachineBlockFrequencyInfo.h"
 #include "llvm/CodeGen/MachineRegisterInfo.h"
 #include "llvm/CodeGen/Passes.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/raw_ostream.h"
 using namespace llvm;
 
 #define DEBUG_TYPE "wasm-argument-move"

 
 /// Test whether the given instruction is an ARGUMENT.
 static bool IsArgument(const MachineInstr &MI) {
   switch (MI.getOpcode()) {
   case WebAssembly::ARGUMENT_I32:
   case WebAssembly::ARGUMENT_I64:
   case WebAssembly::ARGUMENT_F32:
   case WebAssembly::ARGUMENT_F64:
+  case WebAssembly::ARGUMENT_v16i8:
+  case WebAssembly::ARGUMENT_v8i16:
+  case WebAssembly::ARGUMENT_v4i32:
+  case WebAssembly::ARGUMENT_v4f32:
     return true;
   default:
     return false;
   }
 }
 
 bool WebAssemblyArgumentMove::runOnMachineFunction(MachineFunction &MF) {
   DEBUG({

llvm/trunk/lib/Target/WebAssembly/WebAssemblyAsmPrinter.cpp

 } // end anonymous namespace
 
 //===----------------------------------------------------------------------===//
 // Helpers.
 //===----------------------------------------------------------------------===//
 
 MVT WebAssemblyAsmPrinter::getRegType(unsigned RegNo) const {
   const TargetRegisterClass *TRC = MRI->getRegClass(RegNo);
-  for (MVT T : {MVT::i32, MVT::i64, MVT::f32, MVT::f64})
+  for (MVT T : {MVT::i32, MVT::i64, MVT::f32, MVT::f64, MVT::v16i8, MVT::v8i16,
+                MVT::v4i32, MVT::v4f32})
     if (TRC->hasType(T))
       return T;
   DEBUG(errs() << "Unknown type for register number: " << RegNo);
   llvm_unreachable("Unknown register type");
   return MVT::Other;
 }
 
 const char *WebAssemblyAsmPrinter::toString(MVT VT) const {

 void WebAssemblyAsmPrinter::EmitInstruction(const MachineInstr *MI) {
   DEBUG(dbgs() << "EmitInstruction: " << *MI << '\n');
 
   switch (MI->getOpcode()) {
   case WebAssembly::ARGUMENT_I32:
   case WebAssembly::ARGUMENT_I64:
   case WebAssembly::ARGUMENT_F32:
   case WebAssembly::ARGUMENT_F64:
+  case WebAssembly::ARGUMENT_v16i8:
+  case WebAssembly::ARGUMENT_v8i16:
+  case WebAssembly::ARGUMENT_v4i32:
+  case WebAssembly::ARGUMENT_v4f32:
     // These represent values which are live into the function entry, so there's
     // no instruction to emit.
     break;
   case WebAssembly::FALLTHROUGH_RETURN_I32:
   case WebAssembly::FALLTHROUGH_RETURN_I64:
   case WebAssembly::FALLTHROUGH_RETURN_F32:
-  case WebAssembly::FALLTHROUGH_RETURN_F64: {
+  case WebAssembly::FALLTHROUGH_RETURN_F64:
+  case WebAssembly::FALLTHROUGH_RETURN_v16i8:
+  case WebAssembly::FALLTHROUGH_RETURN_v8i16:
+  case WebAssembly::FALLTHROUGH_RETURN_v4i32:
+  case WebAssembly::FALLTHROUGH_RETURN_v4f32: {
     // These instructions represent the implicit return at the end of a
     // function body. The operand is always a pop.
     assert(MFI->isVRegStackified(MI->getOperand(0).getReg()));
 
     if (isVerbose()) {
       OutStreamer->AddComment("fallthrough-return: $pop" +
                               utostr(MFI->getWARegStackId(
                                   MFI->getWAReg(MI->getOperand(0).getReg()))));

llvm/trunk/lib/Target/WebAssembly/WebAssemblyFastISel.cpp

MVT::SimpleValueType getLegalType(MVT::SimpleValueType VT) {
     case MVT::i8:
     case MVT::i16:
       return MVT::i32;
     case MVT::i32:
     case MVT::i64:
     case MVT::f32:
     case MVT::f64:
       return VT;
+    case MVT::v16i8:
+    case MVT::v8i16:
+    case MVT::v4i32:
+    case MVT::v4f32:
+      if (Subtarget->hasSIMD128())
+        return VT;
+      break;
     default:
       break;
     }
     return MVT::INVALID_SIMPLE_VALUE_TYPE;
   }
   bool computeAddress(const Value *Obj, Address &Addr);
   void materializeLoadStoreOperands(Address &Addr);
   void addLoadStoreOperands(const Address &Addr, const MachineInstrBuilder &MIB,
for (auto const &Arg : F->args()) {
     if (Attrs.hasAttribute(i+1, Attribute::ByVal) ||
         Attrs.hasAttribute(i+1, Attribute::SwiftSelf) ||
         Attrs.hasAttribute(i+1, Attribute::SwiftError) ||
         Attrs.hasAttribute(i+1, Attribute::InAlloca) ||
         Attrs.hasAttribute(i+1, Attribute::Nest))
       return false;
 
     Type *ArgTy = Arg.getType();
-    if (ArgTy->isStructTy() || ArgTy->isArrayTy() || ArgTy->isVectorTy())
+    if (ArgTy->isStructTy() || ArgTy->isArrayTy())
+      return false;
+    if (!Subtarget->hasSIMD128() && ArgTy->isVectorTy())
       return false;
 
     unsigned Opc;
     const TargetRegisterClass *RC;
     switch (getSimpleType(ArgTy)) {
     case MVT::i1:
     case MVT::i8:
     case MVT::i16:
     case MVT::i32:
       Opc = WebAssembly::ARGUMENT_I32;
       RC = &WebAssembly::I32RegClass;
       break;
     case MVT::i64:
       Opc = WebAssembly::ARGUMENT_I64;
       RC = &WebAssembly::I64RegClass;
       break;
     case MVT::f32:
       Opc = WebAssembly::ARGUMENT_F32;
       RC = &WebAssembly::F32RegClass;
       break;
     case MVT::f64:
       Opc = WebAssembly::ARGUMENT_F64;
       RC = &WebAssembly::F64RegClass;
       break;
+    case MVT::v16i8:
+      Opc = WebAssembly::ARGUMENT_v16i8;
+      RC = &WebAssembly::V128RegClass;
+      break;
+    case MVT::v8i16:
+      Opc = WebAssembly::ARGUMENT_v8i16;
+      RC = &WebAssembly::V128RegClass;
+      break;
+    case MVT::v4i32:
+      Opc = WebAssembly::ARGUMENT_v4i32;
+      RC = &WebAssembly::V128RegClass;
+      break;
+    case MVT::v4f32:
+      Opc = WebAssembly::ARGUMENT_v4f32;
+      RC = &WebAssembly::V128RegClass;
+      break;
     default:
       return false;
     }
     unsigned ResultReg = createResultReg(RC);
     BuildMI(*FuncInfo.MBB, FuncInfo.InsertPt, DbgLoc, TII.get(Opc), ResultReg)
       .addImm(i);
     updateValueMap(&Arg, ResultReg);
 
bool WebAssemblyFastISel::selectCall(const Instruction *I) {
   FunctionType *FuncTy = Call->getFunctionType();
   unsigned Opc;
   bool IsDirect = Func != nullptr;
   bool IsVoid = FuncTy->getReturnType()->isVoidTy();
   unsigned ResultReg;
   if (IsVoid) {
     Opc = IsDirect ? WebAssembly::CALL_VOID : WebAssembly::CALL_INDIRECT_VOID;
   } else {
+    if (!Subtarget->hasSIMD128() && Call->getType()->isVectorTy())
+      return false;
+
     MVT::SimpleValueType RetTy = getSimpleType(Call->getType());
     switch (RetTy) {
     case MVT::i1:
     case MVT::i8:
     case MVT::i16:
     case MVT::i32:
       Opc = IsDirect ? WebAssembly::CALL_I32 : WebAssembly::CALL_INDIRECT_I32;
       ResultReg = createResultReg(&WebAssembly::I32RegClass);
       break;
     case MVT::i64:
       Opc = IsDirect ? WebAssembly::CALL_I64 : WebAssembly::CALL_INDIRECT_I64;
       ResultReg = createResultReg(&WebAssembly::I64RegClass);
       break;
     case MVT::f32:
       Opc = IsDirect ? WebAssembly::CALL_F32 : WebAssembly::CALL_INDIRECT_F32;
       ResultReg = createResultReg(&WebAssembly::F32RegClass);
       break;
     case MVT::f64:
       Opc = IsDirect ? WebAssembly::CALL_F64 : WebAssembly::CALL_INDIRECT_F64;
       ResultReg = createResultReg(&WebAssembly::F64RegClass);
       break;
+    case MVT::v16i8:
+      Opc =
+          IsDirect ? WebAssembly::CALL_v16i8 : WebAssembly::CALL_INDIRECT_v16i8;
+      ResultReg = createResultReg(&WebAssembly::V128RegClass);
+      break;
+    case MVT::v8i16:
+      Opc =
+          IsDirect ? WebAssembly::CALL_v8i16 : WebAssembly::CALL_INDIRECT_v8i16;
+      ResultReg = createResultReg(&WebAssembly::V128RegClass);
+      break;
+    case MVT::v4i32:
+      Opc =
+          IsDirect ? WebAssembly::CALL_v4i32 : WebAssembly::CALL_INDIRECT_v4i32;
+      ResultReg = createResultReg(&WebAssembly::V128RegClass);
+      break;
+    case MVT::v4f32:
+      Opc =
+          IsDirect ? WebAssembly::CALL_v4f32 : WebAssembly::CALL_INDIRECT_v4f32;
+      ResultReg = createResultReg(&WebAssembly::V128RegClass);
+      break;
     default:
       return false;
     }
   }
 
   SmallVector<unsigned, 8> Args;
   for (unsigned i = 0, e = Call->getNumArgOperands(); i < e; ++i) {
     Value *V = Call->getArgOperand(i);
bool WebAssemblyFastISel::selectBitCast(const Instruction *I) {
   updateValueMap(I, Reg);
   return true;
 }
 
 bool WebAssemblyFastISel::selectLoad(const Instruction *I) {
   const LoadInst *Load = cast<LoadInst>(I);
   if (Load->isAtomic())
     return false;
+  if (!Subtarget->hasSIMD128() && Load->getType()->isVectorTy())
+    return false;
 
   Address Addr;
   if (!computeAddress(Load->getPointerOperand(), Addr))
     return false;
 
   // TODO: Fold a following sign-/zero-extend into the load instruction.
 
   unsigned Opc;
bool WebAssemblyFastISel::selectLoad(const Instruction *I) {
   updateValueMap(Load, ResultReg);
   return true;
 }
 
 bool WebAssemblyFastISel::selectStore(const Instruction *I) {
   const StoreInst *Store = cast<StoreInst>(I);
   if (Store->isAtomic())
     return false;
+  if (!Subtarget->hasSIMD128() &&
+      Store->getValueOperand()->getType()->isVectorTy())
+    return false;
 
   Address Addr;
   if (!computeAddress(Store->getPointerOperand(), Addr))
     return false;
 
   unsigned Opc;
   const TargetRegisterClass *RC;
   bool VTIsi1 = false;
bool WebAssemblyFastISel::selectBr(const Instruction *I) {
 
   unsigned Opc = WebAssembly::BR_IF;
   if (Not)
     Opc = WebAssembly::BR_UNLESS;
 
   BuildMI(*FuncInfo.MBB, FuncInfo.InsertPt, DbgLoc, TII.get(Opc))
       .addMBB(TBB)
       .addReg(CondReg);
-  
   finishCondBranch(Br->getParent(), TBB, FBB);
   return true;
 }
 
 bool WebAssemblyFastISel::selectRet(const Instruction *I) {
   if (!FuncInfo.CanLowerReturn)
     return false;
 
   const ReturnInst *Ret = cast<ReturnInst>(I);
 
   if (Ret->getNumOperands() == 0) {
     BuildMI(*FuncInfo.MBB, FuncInfo.InsertPt, DbgLoc,
             TII.get(WebAssembly::RETURN_VOID));
     return true;
   }
 
   Value *RV = Ret->getOperand(0);
+  if (!Subtarget->hasSIMD128() && RV->getType()->isVectorTy())
+    return false;
+
   unsigned Opc;
   switch (getSimpleType(RV->getType())) {
   case MVT::i1: case MVT::i8:
   case MVT::i16: case MVT::i32:
     Opc = WebAssembly::RETURN_I32;
     break;
   case MVT::i64:
     Opc = WebAssembly::RETURN_I64;
     break;
-  case MVT::f32: Opc = WebAssembly::RETURN_F32; break;
-  case MVT::f64: Opc = WebAssembly::RETURN_F64; break;
+  case MVT::f32:
+    Opc = WebAssembly::RETURN_F32;
+    break;
+  case MVT::f64:
+    Opc = WebAssembly::RETURN_F64;
+    break;
+  case MVT::v16i8:
+    Opc = WebAssembly::RETURN_v16i8;
+    break;
+  case MVT::v8i16:
+    Opc = WebAssembly::RETURN_v8i16;
+    break;
+  case MVT::v4i32:
+    Opc = WebAssembly::RETURN_v4i32;
+    break;
+  case MVT::v4f32:
+    Opc = WebAssembly::RETURN_v4f32;
+    break;
   default: return false;
   }
 
   unsigned Reg;
   if (FuncInfo.Fn->getAttributes().hasAttribute(0, Attribute::SExt))
     Reg = getRegForSignedValue(RV);
   else if (FuncInfo.Fn->getAttributes().hasAttribute(0, Attribute::ZExt))
     Reg = getRegForUnsignedValue(RV);

llvm/trunk/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

WebAssemblyTargetLowering::WebAssemblyTargetLowering(
   // Tell ISel that we have a stack pointer.
   setStackPointerRegisterToSaveRestore(
       Subtarget->hasAddr64() ? WebAssembly::SP64 : WebAssembly::SP32);
   // Set up the register classes.
   addRegisterClass(MVT::i32, &WebAssembly::I32RegClass);
   addRegisterClass(MVT::i64, &WebAssembly::I64RegClass);
   addRegisterClass(MVT::f32, &WebAssembly::F32RegClass);
   addRegisterClass(MVT::f64, &WebAssembly::F64RegClass);
+  if (Subtarget->hasSIMD128()) {
+    addRegisterClass(MVT::v16i8, &WebAssembly::V128RegClass);
+    addRegisterClass(MVT::v8i16, &WebAssembly::V128RegClass);
+    addRegisterClass(MVT::v4i32, &WebAssembly::V128RegClass);
+    addRegisterClass(MVT::v4f32, &WebAssembly::V128RegClass);
+  }
   // Compute derived properties from the register classes.
   computeRegisterProperties(Subtarget->getRegisterInfo());
 
   setOperationAction(ISD::GlobalAddress, MVTPtr, Custom);
   setOperationAction(ISD::ExternalSymbol, MVTPtr, Custom);
   setOperationAction(ISD::JumpTable, MVTPtr, Custom);
   setOperationAction(ISD::BlockAddress, MVTPtr, Custom);
   setOperationAction(ISD::BRIND, MVT::Other, Custom);

 WebAssemblyTargetLowering::getRegForInlineAsmConstraint(
     const TargetRegisterInfo *TRI, StringRef Constraint, MVT VT) const {
   // First, see if this is a constraint that directly corresponds to a
   // WebAssembly register class.
   if (Constraint.size() == 1) {
     switch (Constraint[0]) {
       case 'r':
         assert(VT != MVT::iPTR && "Pointer MVT not expected here");
+        if (Subtarget->hasSIMD128() && VT.isVector()) {
+          if (VT.getSizeInBits() == 128)
+            return std::make_pair(0U, &WebAssembly::V128RegClass);
+        }
         if (VT.isInteger() && !VT.isVector()) {
           if (VT.getSizeInBits() <= 32)
             return std::make_pair(0U, &WebAssembly::I32RegClass);
           if (VT.getSizeInBits() <= 64)
             return std::make_pair(0U, &WebAssembly::I64RegClass);
         }
         break;
       default:

llvm/trunk/lib/Target/WebAssembly/WebAssemblyInstrCall.td

 multiclass CALL<WebAssemblyRegClass vt, string prefix> {
   def CALL_#vt : I<(outs vt:$dst), (ins i32imm:$callee, variable_ops),
                    [(set vt:$dst, (WebAssemblycall1 (i32 imm:$callee)))],
                    !strconcat(prefix, "call\t$dst, $callee")>;
   def CALL_INDIRECT_#vt : I<(outs vt:$dst), (ins I32:$callee, variable_ops),
                             [(set vt:$dst, (WebAssemblycall1 I32:$callee))],
                             !strconcat(prefix, "call_indirect\t$dst, $callee")>;
 }
+
+multiclass SIMD_CALL<ValueType vt, string prefix> {
+  def CALL_#vt : SIMD_I<(outs V128:$dst), (ins i32imm:$callee, variable_ops),
+                         [(set (vt V128:$dst),
+                               (WebAssemblycall1 (i32 imm:$callee)))],
+                         !strconcat(prefix, "call\t$dst, $callee")>;
+  def CALL_INDIRECT_#vt : SIMD_I<(outs V128:$dst),
+                                  (ins I32:$callee, variable_ops),
+                                  [(set (vt V128:$dst),
+                                        (WebAssemblycall1 I32:$callee))],
+                                  !strconcat(prefix,
+                                             "call_indirect\t$dst, $callee")>;
+}
+
 let Uses = [SP32, SP64], isCall = 1 in {
   defm : CALL<I32, "i32.">;
   defm : CALL<I64, "i64.">;
   defm : CALL<F32, "f32.">;
   defm : CALL<F64, "f64.">;
+  defm : SIMD_CALL<v16i8, "i8x16.">;
+  defm : SIMD_CALL<v8i16, "i16x8.">;
+  defm : SIMD_CALL<v4i32, "i32x4.">;
+  defm : SIMD_CALL<v4f32, "f32x4.">;
 
   def CALL_VOID : I<(outs), (ins i32imm:$callee, variable_ops),
                     [(WebAssemblycall0 (i32 imm:$callee))],
                     "call    \t$callee">;
   def CALL_INDIRECT_VOID : I<(outs), (ins I32:$callee, variable_ops),
                              [(WebAssemblycall0 I32:$callee)],
                              "call_indirect\t$callee">;
 } // Uses = [SP32,SP64], isCall = 1
 
 } // Defs = [ARGUMENTS]
 
 // Patterns for matching a direct call to a global address.
 def : Pat<(i32 (WebAssemblycall1 (WebAssemblywrapper tglobaladdr:$callee))),
           (CALL_I32 tglobaladdr:$callee)>;
 def : Pat<(i64 (WebAssemblycall1 (WebAssemblywrapper tglobaladdr:$callee))),
           (CALL_I64 tglobaladdr:$callee)>;
 def : Pat<(f32 (WebAssemblycall1 (WebAssemblywrapper tglobaladdr:$callee))),
           (CALL_F32 tglobaladdr:$callee)>;
 def : Pat<(f64 (WebAssemblycall1 (WebAssemblywrapper tglobaladdr:$callee))),
           (CALL_F64 tglobaladdr:$callee)>;
+def : Pat<(v16i8 (WebAssemblycall1 (WebAssemblywrapper tglobaladdr:$callee))),
+          (CALL_v16i8 tglobaladdr:$callee)>, Requires<[HasSIMD128]>;
+def : Pat<(v8i16 (WebAssemblycall1 (WebAssemblywrapper tglobaladdr:$callee))),
+          (CALL_v8i16 tglobaladdr:$callee)>, Requires<[HasSIMD128]>;
+def : Pat<(v4i32 (WebAssemblycall1 (WebAssemblywrapper tglobaladdr:$callee))),
+          (CALL_v4i32 tglobaladdr:$callee)>, Requires<[HasSIMD128]>;
+def : Pat<(v4f32 (WebAssemblycall1 (WebAssemblywrapper tglobaladdr:$callee))),
+          (CALL_v4f32 tglobaladdr:$callee)>, Requires<[HasSIMD128]>;
 def : Pat<(WebAssemblycall0 (WebAssemblywrapper tglobaladdr:$callee)),
           (CALL_VOID tglobaladdr:$callee)>;
 
 // Patterns for matching a direct call to an external symbol.
 def : Pat<(i32 (WebAssemblycall1 (WebAssemblywrapper texternalsym:$callee))),
           (CALL_I32 texternalsym:$callee)>;
 def : Pat<(i64 (WebAssemblycall1 (WebAssemblywrapper texternalsym:$callee))),
           (CALL_I64 texternalsym:$callee)>;
 def : Pat<(f32 (WebAssemblycall1 (WebAssemblywrapper texternalsym:$callee))),
           (CALL_F32 texternalsym:$callee)>;
 def : Pat<(f64 (WebAssemblycall1 (WebAssemblywrapper texternalsym:$callee))),
           (CALL_F64 texternalsym:$callee)>;
+def : Pat<(v16i8 (WebAssemblycall1 (WebAssemblywrapper texternalsym:$callee))),
+          (CALL_v16i8 texternalsym:$callee)>, Requires<[HasSIMD128]>;
+def : Pat<(v8i16 (WebAssemblycall1 (WebAssemblywrapper texternalsym:$callee))),
+          (CALL_v8i16 texternalsym:$callee)>, Requires<[HasSIMD128]>;
+def : Pat<(v4i32 (WebAssemblycall1 (WebAssemblywrapper texternalsym:$callee))),
+          (CALL_v4i32 texternalsym:$callee)>, Requires<[HasSIMD128]>;
+def : Pat<(v4f32 (WebAssemblycall1 (WebAssemblywrapper texternalsym:$callee))),
+          (CALL_v4f32 texternalsym:$callee)>, Requires<[HasSIMD128]>;
 def : Pat<(WebAssemblycall0 (WebAssemblywrapper texternalsym:$callee)),
           (CALL_VOID texternalsym:$callee)>;

llvm/trunk/lib/Target/WebAssembly/WebAssemblyInstrControl.td

multiclass RETURN<WebAssemblyRegClass vt> {
   def RETURN_#vt : I<(outs), (ins vt:$val), [(WebAssemblyreturn vt:$val)],
                      "return  \t$val">;
   // Equivalent to RETURN_#vt, for use at the end of a function when wasm
   // semantics return by falling off the end of the block.
   let isCodeGenOnly = 1 in
   def FALLTHROUGH_RETURN_#vt : I<(outs), (ins vt:$val), []>;
 }
 
+multiclass SIMD_RETURN<ValueType vt> {
+  def RETURN_#vt : SIMD_I<(outs), (ins V128:$val),
+                          [(WebAssemblyreturn (vt V128:$val))],
+                          "return  \t$val">;
+  // Equivalent to RETURN_#vt, for use at the end of a function when wasm
+  // semantics return by falling off the end of the block.
+  let isCodeGenOnly = 1 in
+  def FALLTHROUGH_RETURN_#vt : SIMD_I<(outs), (ins V128:$val), []>;
+}
+
 let isTerminator = 1, hasCtrlDep = 1, isBarrier = 1 in {
 let isReturn = 1 in {
   defm : RETURN<I32>;
   defm : RETURN<I64>;
   defm : RETURN<F32>;
   defm : RETURN<F64>;
+  defm : SIMD_RETURN<v16i8>;
+  defm : SIMD_RETURN<v8i16>;
+  defm : SIMD_RETURN<v4i32>;
+  defm : SIMD_RETURN<v4f32>;
+
   def RETURN_VOID : I<(outs), (ins), [(WebAssemblyreturn)], "return">;
 
   // This is to RETURN_VOID what FALLTHROUGH_RETURN_#vt is to RETURN_#vt.
   let isCodeGenOnly = 1 in
   def FALLTHROUGH_RETURN_VOID : I<(outs), (ins), []>;
 } // isReturn = 1
   def UNREACHABLE : I<(outs), (ins), [(trap)], "unreachable">;
 } // isTerminator = 1, hasCtrlDep = 1, isBarrier = 1
 
 } // Defs = [ARGUMENTS]

llvm/trunk/lib/Target/WebAssembly/WebAssemblyInstrFormats.td

 // Normal instructions.
 class I<dag oops, dag iops, list<dag> pattern, string asmstr = "">
     : WebAssemblyInst<asmstr> {
   dag OutOperandList = oops;
   dag InOperandList  = iops;
   let Pattern        = pattern;
 }
 
+class SIMD_I<dag oops, dag iops, list<dag> pattern, string asmstr = "">
+    : I<oops, iops, pattern, asmstr>, Requires<[HasSIMD128]>;
+
 // Unary and binary instructions, for the local types that WebAssembly supports.
 multiclass UnaryInt<SDNode node, string name> {
   def _I32 : I<(outs I32:$dst), (ins I32:$src),
                [(set I32:$dst, (node I32:$src))],
                !strconcat("i32.", !strconcat(name, "\t$dst, $src"))>;
   def _I64 : I<(outs I64:$dst), (ins I64:$src),
                [(set I64:$dst, (node I64:$src))],
                !strconcat("i64.", !strconcat(name, "\t$dst, $src"))>;

 multiclass BinaryFP<SDNode node, string name> {
   def _F32 : I<(outs F32:$dst), (ins F32:$lhs, F32:$rhs),
                [(set F32:$dst, (node F32:$lhs, F32:$rhs))],
                !strconcat("f32.", !strconcat(name, "\t$dst, $lhs, $rhs"))>;
   def _F64 : I<(outs F64:$dst), (ins F64:$lhs, F64:$rhs),
                [(set F64:$dst, (node F64:$lhs, F64:$rhs))],
                !strconcat("f64.", !strconcat(name, "\t$dst, $lhs, $rhs"))>;
 }
+multiclass SIMDBinary<SDNode node, SDNode fnode, string name> {
+  def _I8x16 : SIMD_I<(outs V128:$dst), (ins V128:$lhs, V128:$rhs),
+                      [(set (v16i8 V128:$dst), (node V128:$lhs, V128:$rhs))],
+                      !strconcat("i8x16.", !strconcat(name, "\t$dst, $lhs, $rhs"))>;
+  def _I16x8 : SIMD_I<(outs V128:$dst), (ins V128:$lhs, V128:$rhs),
+                      [(set (v8i16 V128:$dst), (node V128:$lhs, V128:$rhs))],
+                      !strconcat("i16x8.", !strconcat(name, "\t$dst, $lhs, $rhs"))>;
+  def _I32x4 : SIMD_I<(outs V128:$dst), (ins V128:$lhs, V128:$rhs),
+                      [(set (v4i32 V128:$dst), (node V128:$lhs, V128:$rhs))],
+                      !strconcat("i32x4.", !strconcat(name, "\t$dst, $lhs, $rhs"))>;
+  def _F32x4 : SIMD_I<(outs V128:$dst), (ins V128:$lhs, V128:$rhs),
+                      [(set (v4f32 V128:$dst), (fnode V128:$lhs, V128:$rhs))],
+                      !strconcat("f32x4.", !strconcat(name, "\t$dst, $lhs, $rhs"))>;
+
+}
 multiclass ComparisonInt<CondCode cond, string name> {
   def _I32 : I<(outs I32:$dst), (ins I32:$lhs, I32:$rhs),
                [(set I32:$dst, (setcc I32:$lhs, I32:$rhs, cond))],
                !strconcat("i32.", !strconcat(name, "\t$dst, $lhs, $rhs"))>;
   def _I64 : I<(outs I32:$dst), (ins I64:$lhs, I64:$rhs),
                [(set I32:$dst, (setcc I64:$lhs, I64:$rhs, cond))],
                !strconcat("i64.", !strconcat(name, "\t$dst, $lhs, $rhs"))>;
 }
 multiclass ComparisonFP<CondCode cond, string name> {
   def _F32 : I<(outs I32:$dst), (ins F32:$lhs, F32:$rhs),
                [(set I32:$dst, (setcc F32:$lhs, F32:$rhs, cond))],
                !strconcat("f32.", !strconcat(name, "\t$dst, $lhs, $rhs"))>;
   def _F64 : I<(outs I32:$dst), (ins F64:$lhs, F64:$rhs),
                [(set I32:$dst, (setcc F64:$lhs, F64:$rhs, cond))],
                !strconcat("f64.", !strconcat(name, "\t$dst, $lhs, $rhs"))>;
 }

llvm/trunk/lib/Target/WebAssembly/WebAssemblyInstrInfo.td

 // Additional instructions.
 //===----------------------------------------------------------------------===//
 
 multiclass ARGUMENT<WebAssemblyRegClass vt> {
   let hasSideEffects = 1, Uses = [ARGUMENTS], isCodeGenOnly = 1 in
   def ARGUMENT_#vt : I<(outs vt:$res), (ins i32imm:$argno),
                        [(set vt:$res, (WebAssemblyargument timm:$argno))]>;
 }
+multiclass SIMD_ARGUMENT<ValueType vt> {
+  let hasSideEffects = 1, Uses = [ARGUMENTS], isCodeGenOnly = 1 in
+  def ARGUMENT_#vt : SIMD_I<(outs V128:$res), (ins i32imm:$argno),
+                            [(set (vt V128:$res),
+                                  (WebAssemblyargument timm:$argno))]>;
+}
 defm : ARGUMENT<I32>;
 defm : ARGUMENT<I64>;
 defm : ARGUMENT<F32>;
 defm : ARGUMENT<F64>;
+defm : SIMD_ARGUMENT<v16i8>;
+defm : SIMD_ARGUMENT<v8i16>;
+defm : SIMD_ARGUMENT<v4i32>;
+defm : SIMD_ARGUMENT<v4f32>;
 
 let Defs = [ARGUMENTS] in {
 
 // get_local and set_local are not generated by instruction selection; they
 // are implied by virtual register uses and defs.
 multiclass LOCAL<WebAssemblyRegClass vt> {
 let hasSideEffects = 0 in {
   // COPY_LOCAL is not an actual instruction in wasm, but since we allow
let hasSideEffects = 0 in {
   def TEE_LOCAL_#vt : I<(outs vt:$res, vt:$also), (ins vt:$src), [],
                         "tee_local\t$res, $also, $src">;
 } // hasSideEffects = 0
 }
 defm : LOCAL<I32>;
 defm : LOCAL<I64>;
 defm : LOCAL<F32>;
 defm : LOCAL<F64>;
+defm : LOCAL<V128>, Requires<[HasSIMD128]>;
 
 let isMoveImm = 1, isAsCheapAsAMove = 1, isReMaterializable = 1 in {
 def CONST_I32 : I<(outs I32:$res), (ins i32imm:$imm),
                   [(set I32:$res, imm:$imm)],
                   "i32.const\t$res, $imm">;
 def CONST_I64 : I<(outs I64:$res), (ins i64imm:$imm),
                   [(set I64:$res, imm:$imm)],
                   "i64.const\t$res, $imm">;

llvm/trunk/lib/Target/WebAssembly/WebAssemblyInstrSIMD.td

 // WebAssemblyInstrSIMD.td - WebAssembly SIMD codegen support -*- tablegen -*-//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// \brief WebAssembly SIMD operand code-gen constructs.
 ///
 //===----------------------------------------------------------------------===//
 
-// TODO: Implement SIMD instructions.
-// Note: use Requires<[HasSIMD128]>.
+let isCommutable = 1 in {
+defm ADD : SIMDBinary<add, fadd, "add ">;
+defm MUL: SIMDBinary<mul, fmul, "mul ">;
+} // isCommutable = 1
+defm SUB: SIMDBinary<sub, fsub, "sub ">;

llvm/trunk/lib/Target/WebAssembly/WebAssemblyPeephole.cpp

 bool WebAssemblyPeephole::runOnMachineFunction(MachineFunction &MF) {
   DEBUG({
     dbgs() << "********** Peephole **********\n"
            << "********** Function: " << MF.getName() << '\n';
   });
 
   MachineRegisterInfo &MRI = MF.getRegInfo();
   WebAssemblyFunctionInfo &MFI = *MF.getInfo<WebAssemblyFunctionInfo>();
-  const auto &TII = *MF.getSubtarget<WebAssemblySubtarget>().getInstrInfo();
+  const auto &Subtarget = MF.getSubtarget<WebAssemblySubtarget>();
+  const auto &TII = *Subtarget.getInstrInfo();
   const WebAssemblyTargetLowering &TLI =
       *MF.getSubtarget<WebAssemblySubtarget>().getTargetLowering();
   auto &LibInfo = getAnalysis<TargetLibraryInfoWrapperPass>().getTLI();
   bool Changed = false;
 
   for (auto &MBB : MF)
     for (auto &MI : MBB)
       switch (MI.getOpcode()) {
for (auto &MI : MBB)
             MI, MBB, MF, MFI, MRI, TII, WebAssembly::FALLTHROUGH_RETURN_F32,
             WebAssembly::COPY_LOCAL_F32);
         break;
       case WebAssembly::RETURN_F64:
         Changed |= MaybeRewriteToFallthrough(
             MI, MBB, MF, MFI, MRI, TII, WebAssembly::FALLTHROUGH_RETURN_F64,
             WebAssembly::COPY_LOCAL_F64);
         break;
+      case WebAssembly::RETURN_v16i8:
+        Changed |=
+            Subtarget.hasSIMD128() &&
+            MaybeRewriteToFallthrough(MI, MBB, MF, MFI, MRI, TII,
+                                      WebAssembly::FALLTHROUGH_RETURN_v16i8,
+                                      WebAssembly::COPY_LOCAL_V128);
+        break;
+      case WebAssembly::RETURN_v8i16:
+        Changed |=
+            Subtarget.hasSIMD128() &&
+            MaybeRewriteToFallthrough(MI, MBB, MF, MFI, MRI, TII,
+                                      WebAssembly::FALLTHROUGH_RETURN_v8i16,
+                                      WebAssembly::COPY_LOCAL_V128);
+        break;
+      case WebAssembly::RETURN_v4i32:
+        Changed |=
+            Subtarget.hasSIMD128() &&
+            MaybeRewriteToFallthrough(MI, MBB, MF, MFI, MRI, TII,
+                                      WebAssembly::FALLTHROUGH_RETURN_v4i32,
+                                      WebAssembly::COPY_LOCAL_V128);
+        break;
+      case WebAssembly::RETURN_v4f32:
+        Changed |=
+            Subtarget.hasSIMD128() &&
+            MaybeRewriteToFallthrough(MI, MBB, MF, MFI, MRI, TII,
+                                      WebAssembly::FALLTHROUGH_RETURN_v4f32,
+                                      WebAssembly::COPY_LOCAL_V128);
+        break;
       case WebAssembly::RETURN_VOID:
         if (!DisableWebAssemblyFallthroughReturnOpt &&
             &MBB == &MF.back() && &MI == &MBB.back())
           MI.setDesc(TII.get(WebAssembly::FALLTHROUGH_RETURN_VOID));
         break;
       }
 
   return Changed;
 }

llvm/trunk/lib/Target/WebAssembly/WebAssemblyPrepareForLiveIntervals.cpp

 
 /// Test whether the given instruction is an ARGUMENT.
 static bool IsArgument(const MachineInstr *MI) {
   switch (MI->getOpcode()) {
   case WebAssembly::ARGUMENT_I32:
   case WebAssembly::ARGUMENT_I64:
   case WebAssembly::ARGUMENT_F32:
   case WebAssembly::ARGUMENT_F64:
+  case WebAssembly::ARGUMENT_v16i8:
+  case WebAssembly::ARGUMENT_v8i16:
+  case WebAssembly::ARGUMENT_v4i32:
+  case WebAssembly::ARGUMENT_v4f32:
     return true;
   default:
     return false;
   }
 }
 
 // Test whether the given register has an ARGUMENT def.
 static bool HasArgumentDef(unsigned Reg, const MachineRegisterInfo &MRI) {
-  for (auto &Def : MRI.def_instructions(Reg))
+  for (const auto &Def : MRI.def_instructions(Reg))
     if (IsArgument(&Def))
       return true;
   return false;
 }
 
 bool WebAssemblyPrepareForLiveIntervals::runOnMachineFunction(MachineFunction &MF) {
   DEBUG({
     dbgs() << "********** Prepare For LiveIntervals **********\n"

llvm/trunk/lib/Target/WebAssembly/WebAssemblyRegNumbering.cpp

bool WebAssemblyRegNumbering::runOnMachineFunction(MachineFunction &MF) {
   // WebAssembly argument registers are in the same index space as local
   // variables. Assign the numbers for them first.
   MachineBasicBlock &EntryMBB = MF.front();
   for (MachineInstr &MI : EntryMBB) {
     switch (MI.getOpcode()) {
     case WebAssembly::ARGUMENT_I32:
     case WebAssembly::ARGUMENT_I64:
     case WebAssembly::ARGUMENT_F32:
-    case WebAssembly::ARGUMENT_F64: {
+    case WebAssembly::ARGUMENT_F64:
+    case WebAssembly::ARGUMENT_v16i8:
+    case WebAssembly::ARGUMENT_v8i16:
+    case WebAssembly::ARGUMENT_v4i32:
+    case WebAssembly::ARGUMENT_v4f32: {
       int64_t Imm = MI.getOperand(1).getImm();
       DEBUG(dbgs() << "Arg VReg " << MI.getOperand(0).getReg() << " -> WAReg "
                    << Imm << "\n");
       MFI.setWAReg(MI.getOperand(0).getReg(), Imm);
       break;
     }
     default:
       break;

llvm/trunk/lib/Target/WebAssembly/WebAssemblyRegStackify.cpp

static unsigned GetTeeLocalOpcode(const TargetRegisterClass *RC) {
   if (RC == &WebAssembly::I32RegClass)
     return WebAssembly::TEE_LOCAL_I32;
   if (RC == &WebAssembly::I64RegClass)
     return WebAssembly::TEE_LOCAL_I64;
   if (RC == &WebAssembly::F32RegClass)
     return WebAssembly::TEE_LOCAL_F32;
   if (RC == &WebAssembly::F64RegClass)
     return WebAssembly::TEE_LOCAL_F64;
+  if (RC == &WebAssembly::V128RegClass)
+    return WebAssembly::TEE_LOCAL_V128;
   llvm_unreachable("Unexpected register class");
 }
 
 // Shrink LI to its uses, cleaning up LI.
 static void ShrinkToUses(LiveInterval &LI, LiveIntervals &LIS) {
   if (LIS.shrinkToUses(&LI)) {
     SmallVector<LiveInterval*, 4> SplitLIs;
     LIS.splitSeparateComponents(LI, SplitLIs);
for (auto MII = MBB.rbegin(); MII != MBB.rend(); ++MII) {
         if (Def->getOpcode() == TargetOpcode::INLINEASM)
           continue;
 
         // Argument instructions represent live-in registers and not real
         // instructions.
         if (Def->getOpcode() == WebAssembly::ARGUMENT_I32 ||
             Def->getOpcode() == WebAssembly::ARGUMENT_I64 ||
             Def->getOpcode() == WebAssembly::ARGUMENT_F32 ||
-            Def->getOpcode() == WebAssembly::ARGUMENT_F64)
+            Def->getOpcode() == WebAssembly::ARGUMENT_F64 ||
+            Def->getOpcode() == WebAssembly::ARGUMENT_v16i8 ||
+            Def->getOpcode() == WebAssembly::ARGUMENT_v8i16 ||
+            Def->getOpcode() == WebAssembly::ARGUMENT_v4i32 ||
+            Def->getOpcode() == WebAssembly::ARGUMENT_v4f32)
           continue;
 
         // Decide which strategy to take. Prefer to move a single-use value
         // over cloning it, and prefer cloning over introducing a tee_local.
         // For moving, we require the def to be in the same block as the use;
         // this makes things simpler (LiveIntervals' handleMove function only
         // supports intra-block moves) and it's MachineSink's job to catch all
         // the sinking opportunities anyway.

llvm/trunk/lib/Target/WebAssembly/WebAssemblyRegisterInfo.td

 def SP64 : WebAssemblyReg<"%SP64">;
 
 // The register allocation framework requires register classes have at least
 // one register, so we define a few for the floating point register classes
 // since we otherwise don't need a physical register in those classes.
 def F32_0 : WebAssemblyReg<"%f32.0">;
 def F64_0 : WebAssemblyReg<"%f64.0">;
 
+def V128_0: WebAssemblyReg<"%v128">;
+
 // The expression stack "register". This is an opaque entity which serves to
 // order uses and defs that must remain in LIFO order.
 def EXPR_STACK : WebAssemblyReg<"STACK">;
 
 // The incoming arguments "register". This is an opaque entity which serves to
 // order the ARGUMENT instructions that are emulating live-in registers and
 // must not be scheduled below other instructions.
 def ARGUMENTS : WebAssemblyReg<"ARGUMENTS">;
 
 //===----------------------------------------------------------------------===//
 //  Register classes
 //===----------------------------------------------------------------------===//
 
 def I32 : WebAssemblyRegClass<[i32], 32, (add FP32, SP32)>;
 def I64 : WebAssemblyRegClass<[i64], 64, (add FP64, SP64)>;
 def F32 : WebAssemblyRegClass<[f32], 32, (add F32_0)>;
 def F64 : WebAssemblyRegClass<[f64], 64, (add F64_0)>;
+def V128 : WebAssemblyRegClass<[v4f32, v4i32, v16i8, v8i16], 128, (add V128_0)>;
+

llvm/trunk/test/CodeGen/WebAssembly/simd-arith.ll

+; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -mattr=+simd128 | FileCheck %s --check-prefixes CHECK,SIMD128
+; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -mattr=+simd128 -fast-isel | FileCheck %s --check-prefixes CHECK,SIMD128
+; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -mattr=-simd128 | FileCheck %s --check-prefixes CHECK,NO-SIMD128
+; RUN: llc < %s -asm-verbose=false -disable-wasm-fallthrough-return-opt -mattr=-simd128 -fast-isel | FileCheck %s --check-prefixes CHECK,NO-SIMD128
+
+; Test that basic SIMD128 arithmetic operations assemble as expected.
+
+target datalayout = "e-m:e-p:32:32-i64:64-n32:64-S128"
+target triple = "wasm32-unknown-unknown"
+
+declare i32 @llvm.ctlz.i32(i32, i1)
+declare i32 @llvm.cttz.i32(i32, i1)
+declare i32 @llvm.ctpop.i32(i32)
+
+; ==============================================================================
+; 16 x i8
+; ==============================================================================
+; CHECK-LABEL: add_v16i8
+; NO-SIMD128-NOT: i8x16
+; SIMD128: .param v128, v128{{$}}
+; SIMD128: .result v128{{$}}
+; SIMD128: i8x16.add $push0=, $0, $1{{$}}
+; SIMD128: return $pop0{{$}}
+define <16 x i8> @add_v16i8(<16 x i8> %x, <16 x i8> %y) {
+  %a = add <16 x i8> %x, %y
+  ret <16 x i8> %a
+}
+
+; CHECK-LABEL: sub_v16i8
+; NO-SIMD128-NOT: i8x16
+; SIMD128: .param v128, v128{{$}}
+; SIMD128: .result v128{{$}}
+; SIMD128: i8x16.sub $push0=, $0, $1{{$}}
+; SIMD128: return $pop0{{$}}
+define <16 x i8> @sub_v16i8(<16 x i8> %x, <16 x i8> %y) {
+  %a = sub <16 x i8> %x, %y
+  ret <16 x i8> %a
+}
+
+; CHECK-LABEL: mul_v16i8
+; NO-SIMD128-NOT: i8x16
+; SIMD128: .param v128, v128{{$}}
+; SIMD128: .result v128{{$}}
+; SIMD128: i8x16.mul $push0=, $0, $1{{$}}
+; SIMD128: return $pop0{{$}}
+define <16 x i8> @mul_v16i8(<16 x i8> %x, <16 x i8> %y) {
+  %a = mul <16 x i8> %x, %y
+  ret <16 x i8> %a
+}
+
+; ==============================================================================
+; 8 x i16
+; ==============================================================================
+; CHECK-LABEL: add_v8i16
+; NO-SIMD128-NOT: i16x8
+; SIMD128: .param v128, v128{{$}}
+; SIMD128: .result v128{{$}}
+; SIMD128: i16x8.add $push0=, $0, $1{{$}}
+; SIMD128: return $pop0{{$}}
+define <8 x i16> @add_v8i16(<8 x i16> %x, <8 x i16> %y) {
+  %a = add <8 x i16> %x, %y
+  ret <8 x i16> %a
+}
+
+; CHECK-LABEL: sub_v8i16
+; NO-SIMD128-NOT: i16x8
+; SIMD128: .param v128, v128{{$}}
+; SIMD128: .result v128{{$}}
+; SIMD128: i16x8.sub $push0=, $0, $1{{$}}
+; SIMD128: return $pop0{{$}}
+define <8 x i16> @sub_v8i16(<8 x i16> %x, <8 x i16> %y) {
+  %a = sub <8 x i16> %x, %y
+  ret <8 x i16> %a
+}
+
+; CHECK-LABEL: mul_v8i16
+; NO-SIMD128-NOT: i16x8
+; SIMD128: .param v128, v128{{$}}
+; SIMD128: .result v128{{$}}
+; SIMD128: i16x8.mul $push0=, $0, $1{{$}}
+; SIMD128: return $pop0{{$}}
+define <8 x i16> @mul_v8i16(<8 x i16> %x, <8 x i16> %y) {
+  %a = mul <8 x i16> %x, %y
+  ret <8 x i16> %a
+}
+
+; ==============================================================================
+; 4 x i32
+; ==============================================================================
+; CHECK-LABEL: add_v4i32
+; NO-SIMD128-NOT: i32x4
+; SIMD128: .param v128, v128{{$}}
+; SIMD128: .result v128{{$}}
+; SIMD128: i32x4.add $push0=, $0, $1{{$}}
+; SIMD128: return $pop0{{$}}
+define <4 x i32> @add_v4i32(<4 x i32> %x, <4 x i32> %y) {
+  %a = add <4 x i32> %x, %y
+  ret <4 x i32> %a
+}
+
+; CHECK-LABEL: sub_v4i32
+; NO-SIMD128-NOT: i32x4
+; SIMD128: .param v128, v128{{$}}
+; SIMD128: .result v128{{$}}
+; SIMD128: i32x4.sub $push0=, $0, $1{{$}}
+; SIMD128: return $pop0{{$}}
+define <4 x i32> @sub_v4i32(<4 x i32> %x, <4 x i32> %y) {
+  %a = sub <4 x i32> %x, %y
+  ret <4 x i32> %a
+}
+
+; CHECK-LABEL: mul_v4i32
+; NO-SIMD128-NOT: i32x4
+; SIMD128: .param v128, v128{{$}}
+; SIMD128: .result v128{{$}}
+; SIMD128: i32x4.mul $push0=, $0, $1{{$}}
+; SIMD128: return $pop0{{$}}
+define <4 x i32> @mul_v4i32(<4 x i32> %x, <4 x i32> %y) {
+  %a = mul <4 x i32> %x, %y
+  ret <4 x i32> %a
+}
+
+; ==============================================================================
+; 4 x float
+; ==============================================================================
+; CHECK-LABEL: add_v4f32
+; NO-SIMD128-NOT: f32x4
+; SIMD128: .param v128, v128{{$}}
+; SIMD128: .result v128{{$}}
+; SIMD128: f32x4.add $push0=, $0, $1{{$}}
+; SIMD128: return $pop0{{$}}
+define <4 x float> @add_v4f32(<4 x float> %x, <4 x float> %y) {
+  %a = fadd <4 x float> %x, %y
+  ret <4 x float> %a
+}
+
+; CHECK-LABEL: sub_v4f32
+; NO-SIMD128-NOT: f32x4
+; SIMD128: .param v128, v128{{$}}
+; SIMD128: .result v128{{$}}
+; SIMD128: f32x4.sub $push0=, $0, $1{{$}}
+; SIMD128: return $pop0{{$}}
+define <4 x float> @sub_v4f32(<4 x float> %x, <4 x float> %y) {
+  %a = fsub <4 x float> %x, %y
+  ret <4 x float> %a
+}
+
+; CHECK-LABEL: mul_v4f32
+; NO-SIMD128-NOT: f32x4
+; SIMD128: .param v128, v128{{$}}
+; SIMD128: .result v128{{$}}
+; SIMD128: f32x4.mul $push0=, $0, $1{{$}}
+; SIMD128: return $pop0{{$}}
+define <4 x float> @mul_v4f32(<4 x float> %x, <4 x float> %y) {
+  %a = fmul <4 x float> %x, %y
+  ret <4 x float> %a
+}
+