[PPC64] Add vector quadword add/sub instructions for POWER8

[kbarton]

Summary

This patch adds support for the vector quadword add/sub instructions introduced in POWER8:

• vadduqm
• vaddeuqm
• vaddcuq
• vaddecuq
• vsubuqm
• vsubeuqm
• vsubcuq
• vsubecuq

In addition to adding the instructions themselves, it also adds support for the v1i128 type for intrinsics (Intrinsics.td, Function.cpp, and IntrinsicEmitter.cpp).

[wschmidt]

A couple of inline notes.

lib/Target/PowerPC/PPCCallingConv.td

65

Looks like a line-too-long.

lib/Target/PowerPC/PPCISelLowering.cpp

4863

This is a lot more complicated than that. You've just asserted you can pass a 128-bit value in a single 64-bit register, which will bring about wailing and gnashing of teeth.

An i128 needs to be passed in two GPRs, and furthermore you may need to first skip a GPR to ensure that the image of the i128 in the parameter save area is aligned on a 16-byte boundary. (If the parameter's address is taken, it will be forced to its home location in memory, and that must be properly aligned.) I've verified that in gcc we have rs6000_function_arg_advance_1 which certainly makes this adjustment, as does rs6000_function_arg.

I believe you can borrow some of the necessarily logic from the varargs handling of vector types, below; it has to pass varargs parameters in consecutive GPRs as well. However, I'm bothered that I don't see anything there for alignment. The ABI document is wishy-washy about this for varargs: "Alignment requirements such as those for vector types may require the va_list pointer to first be aligned before accessing a value." Note the "may." But GCC's varargs handling uses rs6000_function_arg_advance_1 as well, so I believe to be compatible we have to have the same alignment done here.

CCing Uli for his thoughts, since he spent a lot of his life in this ABI code. At one time this passed the gcc ABI compatibility tests, so maybe I'm missing something.

[wschmidt]

One more general comment -- you don't have any testing for the ABI changes. Also, the ABI changes should probably be a separate patch from the quadword instruction patch....sorry. :/

[nemanjai]

include/llvm/IR/IntrinsicsPowerPC.td

537

Do we not want these guarded for Power8 Altivec feature? It would seem that we don't want these available in the back end for CPU's that don't have the feature.

test/CodeGen/PowerPC/vec_add_sub_quadword.ll

9

I don't think all the registers are ABI-required registers. Probably ok to use a regex for the other(s). Just making the same comment I received previously and I think it works well. A regex can be something like {{[0-9]+}}.

It may or may not make sense, but you can check that the value being added is loaded into a register and that that register is used for the add (at least for the immediate). Although I don't know if that's a good idea for commutative operations. Maybe someone more knowledgeable can comment on that.

15

1. Why the star at the end?
2. I think that in a large test case, it helps to have CHECK-LABEL for every function so that if something fails, it is easier to find where the failure is. The tool won't go past a check if the previous one did not match.

71

Would it make sense to check that the right parameter is in the right register?

[kbarton]

In D9081#157856, @wschmidt wrote:

One more general comment -- you don't have any testing for the ABI changes. Also, the ABI changes should probably be a separate patch from the quadword instruction patch....sorry. :/

Agreed. I will refactor these changes and take a closer look at your comments in PPCISelLowering.cpp and submit a new patch for review.

[kbarton]

include/llvm/IR/IntrinsicsPowerPC.td

537

We don't have any other guards in this file, as far as I can tell.
I believe this is fine to define these intrinsics as is, and let other diagnostic messages determine whether they are valid for the target architecture.

test/CodeGen/PowerPC/vec_add_sub_quadword.ll

9

Agreed. Will change.

15

The star is a typo.
I've also added the check label for the first several functions. Good call.

71

I don't know how this would be done, short of actually running the test case and making sure the output is correct.

[kbarton]

test/CodeGen/PowerPC/vec_add_sub_quadword.ll

9

Actually, on second thought I think it's better to leave these tests as is.
The ABI specifies which registers the parameters to the function are passed in and given these are extremely short functions there is no reason to not reuse the same registers for the specific instructions being tested. So, by using the same registers as the parameters, we are testing to ensure no unnecessary register moves are being added to the code.

If these were more complicated test cases, then we would need to use regular expressions because we couldn't guarantee in all cases the same registers would be used.

[kbarton]

The changes related to the ABI have been refactored and put into a different patch.
This patch now contains only the 8 new instructions to be added, and the necessary support for the v1i128 type in the intrinsics code.

[wschmidt]

LGTM. Sorry I missed this with the email server problem!

[kbarton]

Committed revision 238144.

include/llvm/IR/Intrinsics.td

 def llvm_v8i32_ty      : LLVMType<v8i32>;    //  8 x i32
 def llvm_v16i32_ty     : LLVMType<v16i32>;   // 16 x i32
 def llvm_v1i64_ty      : LLVMType<v1i64>;    //  1 x i64
 def llvm_v2i64_ty      : LLVMType<v2i64>;    //  2 x i64
 def llvm_v4i64_ty      : LLVMType<v4i64>;    //  4 x i64
 def llvm_v8i64_ty      : LLVMType<v8i64>;    //  8 x i64
 def llvm_v16i64_ty     : LLVMType<v16i64>;   // 16 x i64
 
+def llvm_v1i128_ty     : LLVMType<v1i128>;   //  1 x i128
+
 def llvm_v2f16_ty      : LLVMType<v2f16>;    //  2 x half (__fp16)
 def llvm_v4f16_ty      : LLVMType<v4f16>;    //  4 x half (__fp16)
 def llvm_v8f16_ty      : LLVMType<v8f16>;    //  8 x half (__fp16)
 def llvm_v1f32_ty      : LLVMType<v1f32>;    //  1 x float
 def llvm_v2f32_ty      : LLVMType<v2f32>;    //  2 x float
 def llvm_v4f32_ty      : LLVMType<v4f32>;    //  4 x float
 def llvm_v8f32_ty      : LLVMType<v8f32>;    //  8 x float
 def llvm_v16f32_ty     : LLVMType<v16f32>;   // 16 x float

include/llvm/IR/IntrinsicsPowerPC.td

 
 /// PowerPC_Vec_DDD_Intrinsic - A PowerPC intrinsic that takes two v2i64
 /// vectors and returns one.  These intrinsics have no side effects.
 class PowerPC_Vec_DDD_Intrinsic<string GCCIntSuffix>
   : PowerPC_Vec_Intrinsic<GCCIntSuffix,
                           [llvm_v2i64_ty], [llvm_v2i64_ty, llvm_v2i64_ty],
                           [IntrNoMem]>;
 
+/// PowerPC_Vec_QQQ_Intrinsic - A PowerPC intrinsic that takes two v1i128
+/// vectors and returns one. These intrinsics have no side effects.
+class PowerPC_Vec_QQQ_Intrinsic<string GCCIntSuffix>
+  : PowerPC_Vec_Intrinsic<GCCIntSuffix,
+                         [llvm_v1i128_ty], [llvm_v1i128_ty, llvm_v1i128_ty],
+                         [IntrNoMem]>;
+
 //===----------------------------------------------------------------------===//
 // PowerPC VSX Intrinsic Class Definitions.
 //
 
 /// PowerPC_VSX_Vec_DDD_Intrinsic - A PowerPC intrinsic that takes two v2f64
 /// vectors and returns one.  These intrinsics have no side effects.
 class PowerPC_VSX_Vec_DDD_Intrinsic<string GCCIntSuffix>
   : PowerPC_VSX_Intrinsic<GCCIntSuffix,

 // Saturating adds.
 def int_ppc_altivec_vaddubs : PowerPC_Vec_BBB_Intrinsic<"vaddubs">;
 def int_ppc_altivec_vaddsbs : PowerPC_Vec_BBB_Intrinsic<"vaddsbs">;
 def int_ppc_altivec_vadduhs : PowerPC_Vec_HHH_Intrinsic<"vadduhs">;
 def int_ppc_altivec_vaddshs : PowerPC_Vec_HHH_Intrinsic<"vaddshs">;
 def int_ppc_altivec_vadduws : PowerPC_Vec_WWW_Intrinsic<"vadduws">;
 def int_ppc_altivec_vaddsws : PowerPC_Vec_WWW_Intrinsic<"vaddsws">;
 def int_ppc_altivec_vaddcuw : PowerPC_Vec_WWW_Intrinsic<"vaddcuw">;
+def int_ppc_altivec_vaddcuq : PowerPC_Vec_QQQ_Intrinsic<"vaddcuq">;
 
 // Saturating subs.
 def int_ppc_altivec_vsububs : PowerPC_Vec_BBB_Intrinsic<"vsububs">;
 def int_ppc_altivec_vsubsbs : PowerPC_Vec_BBB_Intrinsic<"vsubsbs">;
 def int_ppc_altivec_vsubuhs : PowerPC_Vec_HHH_Intrinsic<"vsubuhs">;
 def int_ppc_altivec_vsubshs : PowerPC_Vec_HHH_Intrinsic<"vsubshs">;
 def int_ppc_altivec_vsubuws : PowerPC_Vec_WWW_Intrinsic<"vsubuws">;
 def int_ppc_altivec_vsubsws : PowerPC_Vec_WWW_Intrinsic<"vsubsws">;
 def int_ppc_altivec_vsubcuw : PowerPC_Vec_WWW_Intrinsic<"vsubcuw">;
+def int_ppc_altivec_vsubcuq : PowerPC_Vec_QQQ_Intrinsic<"vsubcuq">;
 
 let TargetPrefix = "ppc" in {  // All PPC intrinsics start with "llvm.ppc.".
   // Saturating multiply-adds.
   def int_ppc_altivec_vmhaddshs : GCCBuiltin<"__builtin_altivec_vmhaddshs">,
               Intrinsic<[llvm_v8i16_ty], [llvm_v8i16_ty,
                          llvm_v8i16_ty, llvm_v8i16_ty], [IntrNoMem]>;
   def int_ppc_altivec_vmhraddshs : GCCBuiltin<"__builtin_altivec_vmhraddshs">,
               Intrinsic<[llvm_v8i16_ty], [llvm_v8i16_ty,
let TargetPrefix = "ppc" in {  // All PPC intrinsics start with "llvm.ppc.".
   def int_ppc_altivec_vrfim : GCCBuiltin<"__builtin_altivec_vrfim">,
               Intrinsic<[llvm_v4f32_ty], [llvm_v4f32_ty], [IntrNoMem]>;
   def int_ppc_altivec_vrfin : GCCBuiltin<"__builtin_altivec_vrfin">,
               Intrinsic<[llvm_v4f32_ty], [llvm_v4f32_ty], [IntrNoMem]>;
   def int_ppc_altivec_vrfip : GCCBuiltin<"__builtin_altivec_vrfip">,
               Intrinsic<[llvm_v4f32_ty], [llvm_v4f32_ty], [IntrNoMem]>;
   def int_ppc_altivec_vrfiz : GCCBuiltin<"__builtin_altivec_vrfiz">,
               Intrinsic<[llvm_v4f32_ty], [llvm_v4f32_ty], [IntrNoMem]>;
+
+  // Add Extended Quadword
+  def int_ppc_altivec_vaddeuqm : GCCBuiltin<"__builtin_altivec_vaddeuqm">,
+              Intrinsic<[llvm_v1i128_ty], 
+                        [llvm_v1i128_ty, llvm_v1i128_ty, llvm_v1i128_ty],
+                        [IntrNoMem]>;
+  def int_ppc_altivec_vaddecuq : GCCBuiltin<"__builtin_altivec_vaddecuq">,
+              Intrinsic<[llvm_v1i128_ty], 
+                        [llvm_v1i128_ty, llvm_v1i128_ty, llvm_v1i128_ty],
+                        [IntrNoMem]>;
+
+  // Sub Extended Quadword
+  def int_ppc_altivec_vsubeuqm : GCCBuiltin<"__builtin_altivec_vsubeuqm">,
+              Intrinsic<[llvm_v1i128_ty], 
+                        [llvm_v1i128_ty, llvm_v1i128_ty, llvm_v1i128_ty],
+                        [IntrNoMem]>;
+  def int_ppc_altivec_vsubecuq : GCCBuiltin<"__builtin_altivec_vsubecuq">,
+              Intrinsic<[llvm_v1i128_ty], 
+                        [llvm_v1i128_ty, llvm_v1i128_ty, llvm_v1i128_ty],
+                        [IntrNoMem]>;
 }
 
 def int_ppc_altivec_vsl   : PowerPC_Vec_WWW_Intrinsic<"vsl">;
 def int_ppc_altivec_vslo  : PowerPC_Vec_WWW_Intrinsic<"vslo">;
 
 def int_ppc_altivec_vslb  : PowerPC_Vec_BBB_Intrinsic<"vslb">;
 def int_ppc_altivec_vslh  : PowerPC_Vec_HHH_Intrinsic<"vslh">;
 def int_ppc_altivec_vslw  : PowerPC_Vec_WWW_Intrinsic<"vslw">;

lib/IR/Function.cpp

enum IIT_Info {
   IIT_EXTEND_ARG = 24,
   IIT_TRUNC_ARG = 25,
   IIT_ANYPTR = 26,
   IIT_V1   = 27,
   IIT_VARARG = 28,
   IIT_HALF_VEC_ARG = 29,
   IIT_SAME_VEC_WIDTH_ARG = 30,
   IIT_PTR_TO_ARG = 31,
-  IIT_VEC_OF_PTRS_TO_ELT = 32
+  IIT_VEC_OF_PTRS_TO_ELT = 32,
+  IIT_I128 = 33
 };
 
 
 static void DecodeIITType(unsigned &NextElt, ArrayRef<unsigned char> Infos,
                       SmallVectorImpl<Intrinsic::IITDescriptor> &OutputTable) {
   IIT_Info Info = IIT_Info(Infos[NextElt++]);
   unsigned StructElts = 2;
   using namespace Intrinsic;
case IIT_I16:
     OutputTable.push_back(IITDescriptor::get(IITDescriptor::Integer,16));
     return;
   case IIT_I32:
     OutputTable.push_back(IITDescriptor::get(IITDescriptor::Integer, 32));
     return;
   case IIT_I64:
     OutputTable.push_back(IITDescriptor::get(IITDescriptor::Integer, 64));
     return;
+  case IIT_I128:
+    OutputTable.push_back(IITDescriptor::get(IITDescriptor::Integer, 128));
+    return;
   case IIT_V1:
     OutputTable.push_back(IITDescriptor::get(IITDescriptor::Vector, 1));
     DecodeIITType(NextElt, Infos, OutputTable);
     return;
   case IIT_V2:
     OutputTable.push_back(IITDescriptor::get(IITDescriptor::Vector, 2));
     DecodeIITType(NextElt, Infos, OutputTable);
     return;

lib/Target/PowerPC/PPCCallingConv.td

def RetCC_PPC : CallingConv<[
   CCIfType<[f64], CCAssignToReg<[F1, F2, F3, F4, F5, F6, F7, F8]>>,
 
   // QPX vectors are returned in QF1 and QF2. 
   CCIfType<[v4f64, v4f32, v4i1],
            CCIfSubtarget<"hasQPX()", CCAssignToReg<[QF1, QF2]>>>,
  
   // Vector types returned as "direct" go into V2 .. V9; note that only the
   // ELFv2 ABI fully utilizes all these registers.
-  CCIfType<[v16i8, v8i16, v4i32, v4f32], CCIfSubtarget<"hasAltivec()",
+  CCIfType<[v16i8, v8i16, v4i32, v2i64, v1i128, v4f32], CCIfSubtarget<"hasAltivec()",
            CCAssignToReg<[V2, V3, V4, V5, V6, V7, V8, V9]>>>,
   CCIfType<[v2f64, v2i64], CCIfSubtarget<"hasVSX()",
            CCAssignToReg<[VSH2, VSH3, VSH4, VSH5, VSH6, VSH7, VSH8, VSH9]>>>
 ]>;
 
 // No explicit register is specified for the AnyReg calling convention. The
 // register allocator may assign the arguments to any free register.
 //
def RetCC_PPC64_ELF_FIS : CallingConv<[
   CCIfType<[i16],  CCPromoteToType<i64>>,
   CCIfType<[i32],  CCPromoteToType<i64>>,
   CCIfType<[i64],  CCAssignToReg<[X3, X4]>>,
   CCIfType<[i128], CCAssignToReg<[X3, X4, X5, X6]>>,
   CCIfType<[f32],  CCAssignToReg<[F1, F2, F3, F4, F5, F6, F7, F8]>>,
   CCIfType<[f64],  CCAssignToReg<[F1, F2, F3, F4, F5, F6, F7, F8]>>,
   CCIfType<[v4f64, v4f32, v4i1],
            CCIfSubtarget<"hasQPX()", CCAssignToReg<[QF1, QF2]>>>,
-  CCIfType<[v16i8, v8i16, v4i32, v4f32], CCIfSubtarget<"hasAltivec()",
+  CCIfType<[v16i8, v8i16, v4i32, v2i64, v1i128, v4f32], 
+           CCIfSubtarget<"hasAltivec()",
            CCAssignToReg<[V2, V3, V4, V5, V6, V7, V8, V9]>>>,
   CCIfType<[v2f64, v2i64], CCIfSubtarget<"hasVSX()",
            CCAssignToReg<[VSH2, VSH3, VSH4, VSH5, VSH6, VSH7, VSH8, VSH9]>>>
 ]>;
 
 //===----------------------------------------------------------------------===//
 // PowerPC System V Release 4 32-bit ABI
 //===----------------------------------------------------------------------===//

 // In contrast to CC_PPC32_SVR4_VarArg, this calling convention first tries to
 // put vector arguments in vector registers before putting them on the stack.
 def CC_PPC32_SVR4 : CallingConv<[
   // QPX vectors mirror the scalar FP convention.
   CCIfType<[v4f64, v4f32, v4i1], CCIfSubtarget<"hasQPX()",
     CCAssignToReg<[QF1, QF2, QF3, QF4, QF5, QF6, QF7, QF8]>>>,
 
   // The first 12 Vector arguments are passed in AltiVec registers.
-  CCIfType<[v16i8, v8i16, v4i32, v4f32], CCIfSubtarget<"hasAltivec()",
-           CCAssignToReg<[V2, V3, V4, V5, V6, V7, V8, V9,
-                          V10, V11, V12, V13]>>>,
+  CCIfType<[v16i8, v8i16, v4i32, v2i64, v1i128, v4f32], 
+           CCIfSubtarget<"hasAltivec()", CCAssignToReg<[V2, V3, V4, V5, V6, V7,
+                          V8, V9, V10, V11, V12, V13]>>>,
   CCIfType<[v2f64, v2i64], CCIfSubtarget<"hasVSX()",
            CCAssignToReg<[VSH2, VSH3, VSH4, VSH5, VSH6, VSH7, VSH8, VSH9,
                           VSH10, VSH11, VSH12, VSH13]>>>,
            
   CCDelegateTo<CC_PPC32_SVR4_Common>
 ]>;  
 
 // Helper "calling convention" to handle aggregate by value arguments.

lib/Target/PowerPC/PPCISelLowering.cpp

if (Subtarget.hasAltivec()) {
     // First set operation action for all vector types to expand. Then we
     // will selectively turn on ones that can be effectively codegen'd.
     for (MVT VT : MVT::vector_valuetypes()) {
       // add/sub are legal for all supported vector VT's.
       setOperationAction(ISD::ADD , VT, Legal);
       setOperationAction(ISD::SUB , VT, Legal);
 
       // Vector instructions introduced in P8
-      if (Subtarget.hasP8Altivec()) {
+      if (Subtarget.hasP8Altivec() && (VT.SimpleTy != MVT::v1i128)) {
         setOperationAction(ISD::CTPOP, VT, Legal);
         setOperationAction(ISD::CTLZ, VT, Legal);
       }
       else {
         setOperationAction(ISD::CTPOP, VT, Expand);
         setOperationAction(ISD::CTLZ, VT, Expand);
       }
 
if (Subtarget.hasVSX()) {
       setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::v2i64, Legal);
       setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::v2i32, Legal);
       setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::v2i16, Custom);
       setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::v2i8, Custom);
 
       addRegisterClass(MVT::v2i64, &PPC::VSRCRegClass);
     }
 
-    if (Subtarget.hasP8Altivec()) 
+    if (Subtarget.hasP8Altivec()) {
       addRegisterClass(MVT::v2i64, &PPC::VRRCRegClass);
+      addRegisterClass(MVT::v1i128, &PPC::VRRCRegClass);
+    }
   }
 
   if (Subtarget.hasQPX()) {
     setOperationAction(ISD::FADD, MVT::v4f64, Legal);
     setOperationAction(ISD::FSUB, MVT::v4f64, Legal);
     setOperationAction(ISD::FMUL, MVT::v4f64, Legal);
     setOperationAction(ISD::FREM, MVT::v4f64, Expand);
 

 static unsigned CalculateStackSlotAlignment(EVT ArgVT, EVT OrigVT,
                                             ISD::ArgFlagsTy Flags,
                                             unsigned PtrByteSize) {
   unsigned Align = PtrByteSize;
 
   // Altivec parameters are padded to a 16 byte boundary.
   if (ArgVT == MVT::v4f32 || ArgVT == MVT::v4i32 ||
       ArgVT == MVT::v8i16 || ArgVT == MVT::v16i8 ||
-      ArgVT == MVT::v2f64 || ArgVT == MVT::v2i64)
+      ArgVT == MVT::v2f64 || ArgVT == MVT::v2i64 ||
+      ArgVT == MVT::v1i128)
     Align = 16;
   // QPX vector types stored in double-precision are padded to a 32 byte
   // boundary.
   else if (ArgVT == MVT::v4f64 || ArgVT == MVT::v4i1)
     Align = 32;
 
   // ByVal parameters are aligned as requested.
   if (Flags.isByVal()) {
if (ArgVT == MVT::f32 || ArgVT == MVT::f64 ||
                     ArgVT == MVT::v4f64 ||
                     ArgVT == MVT::v4i1)))
       if (AvailableFPRs > 0) {
         --AvailableFPRs;
         return false;
       }
     if (ArgVT == MVT::v4f32 || ArgVT == MVT::v4i32 ||
         ArgVT == MVT::v8i16 || ArgVT == MVT::v16i8 ||
-        ArgVT == MVT::v2f64 || ArgVT == MVT::v2i64)
+        ArgVT == MVT::v2f64 || ArgVT == MVT::v2i64 ||
+        ArgVT == MVT::v1i128)
       if (AvailableVRs > 0) {
         --AvailableVRs;
         return false;
       }
   }
 
   return UseMemory;
 }
case MVT::f64:
       }
       break;
     case MVT::v4f32:
     case MVT::v4i32:
     case MVT::v8i16:
     case MVT::v16i8:
     case MVT::v2f64:
     case MVT::v2i64:
+    case MVT::v1i128:
       if (!Subtarget.hasQPX()) {
       // These can be scalar arguments or elements of a vector array type
       // passed directly.  The latter are used to implement ELFv2 homogenous
       // vector aggregates.
       if (VR_idx != Num_VR_Regs) {
         unsigned VReg = (ObjectVT == MVT::v2f64 || ObjectVT == MVT::v2i64) ?
                         MF.addLiveIn(VSRH[VR_idx], &PPC::VSHRCRegClass) :
                         MF.addLiveIn(VR[VR_idx], &PPC::VRRCRegClass);
if (CallConv == CallingConv::Fast) {
           if (++NumGPRsUsed <= NumGPRs)
             continue;
           break;
         case MVT::v4i32:
         case MVT::v8i16:
         case MVT::v16i8:
         case MVT::v2f64:
         case MVT::v2i64:
+        case MVT::v1i128:
           if (++NumVRsUsed <= NumVRs)
             continue;
           break;
         case MVT::v4f32:
     // When using QPX, this is handled like a FP register, otherwise, it
     // is an Altivec register.
           if (Subtarget.hasQPX()) {
             if (++NumFPRsUsed <= NumFPRs)
if (Flags.isByVal()) {
       continue;
     }
 
     switch (Arg.getSimpleValueType().SimpleTy) {
     default: llvm_unreachable("Unexpected ValueType for argument!");
     case MVT::i1:
     case MVT::i32:
     case MVT::i64:
+    case MVT::i128:
       // These can be scalar arguments or elements of an integer array type
       // passed directly.  Clang may use those instead of "byval" aggregate
       // types to avoid forcing arguments to memory unnecessarily.
       if (GPR_idx != NumGPRs) {
         RegsToPass.push_back(std::make_pair(GPR[GPR_idx++], Arg));
       } else {
         if (CallConv == CallingConv::Fast)
           ComputePtrOff();
 
         LowerMemOpCallTo(DAG, MF, Chain, Arg, PtrOff, SPDiff, ArgOffset,
                          true, isTailCall, false, MemOpChains,
                          TailCallArguments, dl);
         if (CallConv == CallingConv::Fast)
case MVT::f64: {
       break;
     }
     case MVT::v4f32:
     case MVT::v4i32:
     case MVT::v8i16:
     case MVT::v16i8:
     case MVT::v2f64:
     case MVT::v2i64:
+    case MVT::v1i128:
       if (!Subtarget.hasQPX()) {
       // These can be scalar arguments or elements of a vector array type
       // passed directly.  The latter are used to implement ELFv2 homogenous
       // vector aggregates.
 
       // For a varargs call, named arguments go into VRs or on the stack as
       // usual; unnamed arguments always go to the stack or the corresponding
       // GPRs when within range.  For now, we always put the value in both

lib/Target/PowerPC/PPCInstrAltivec.td

 def : Pat<(store v4i32:$rS, xoaddr:$dst),
           (STVX $rS, xoaddr:$dst)>;
 
 // Bit conversions.
 def : Pat<(v16i8 (bitconvert (v8i16 VRRC:$src))), (v16i8 VRRC:$src)>;
 def : Pat<(v16i8 (bitconvert (v4i32 VRRC:$src))), (v16i8 VRRC:$src)>;
 def : Pat<(v16i8 (bitconvert (v4f32 VRRC:$src))), (v16i8 VRRC:$src)>;
 def : Pat<(v16i8 (bitconvert (v2i64 VRRC:$src))), (v16i8 VRRC:$src)>;
+def : Pat<(v16i8 (bitconvert (v1i128 VRRC:$src))), (v16i8 VRRC:$src)>;
 
 def : Pat<(v8i16 (bitconvert (v16i8 VRRC:$src))), (v8i16 VRRC:$src)>;
 def : Pat<(v8i16 (bitconvert (v4i32 VRRC:$src))), (v8i16 VRRC:$src)>;
 def : Pat<(v8i16 (bitconvert (v4f32 VRRC:$src))), (v8i16 VRRC:$src)>;
 def : Pat<(v8i16 (bitconvert (v2i64 VRRC:$src))), (v8i16 VRRC:$src)>;
+def : Pat<(v8i16 (bitconvert (v1i128 VRRC:$src))), (v8i16 VRRC:$src)>;
 
 def : Pat<(v4i32 (bitconvert (v16i8 VRRC:$src))), (v4i32 VRRC:$src)>;
 def : Pat<(v4i32 (bitconvert (v8i16 VRRC:$src))), (v4i32 VRRC:$src)>;
 def : Pat<(v4i32 (bitconvert (v4f32 VRRC:$src))), (v4i32 VRRC:$src)>;
 def : Pat<(v4i32 (bitconvert (v2i64 VRRC:$src))), (v4i32 VRRC:$src)>;
+def : Pat<(v4i32 (bitconvert (v1i128 VRRC:$src))), (v4i32 VRRC:$src)>;
 
 def : Pat<(v4f32 (bitconvert (v16i8 VRRC:$src))), (v4f32 VRRC:$src)>;
 def : Pat<(v4f32 (bitconvert (v8i16 VRRC:$src))), (v4f32 VRRC:$src)>;
 def : Pat<(v4f32 (bitconvert (v4i32 VRRC:$src))), (v4f32 VRRC:$src)>;
 def : Pat<(v4f32 (bitconvert (v2i64 VRRC:$src))), (v4f32 VRRC:$src)>;
+def : Pat<(v4f32 (bitconvert (v1i128 VRRC:$src))), (v4f32 VRRC:$src)>;
 
 def : Pat<(v2i64 (bitconvert (v16i8 VRRC:$src))), (v2i64 VRRC:$src)>;
 def : Pat<(v2i64 (bitconvert (v8i16 VRRC:$src))), (v2i64 VRRC:$src)>;
 def : Pat<(v2i64 (bitconvert (v4i32 VRRC:$src))), (v2i64 VRRC:$src)>;
 def : Pat<(v2i64 (bitconvert (v4f32 VRRC:$src))), (v2i64 VRRC:$src)>;
+def : Pat<(v2i64 (bitconvert (v1i128 VRRC:$src))), (v2i64 VRRC:$src)>;
+
+def : Pat<(v1i128 (bitconvert (v16i8 VRRC:$src))), (v1i128 VRRC:$src)>;
+def : Pat<(v1i128 (bitconvert (v8i16 VRRC:$src))), (v1i128 VRRC:$src)>;
+def : Pat<(v1i128 (bitconvert (v4i32 VRRC:$src))), (v1i128 VRRC:$src)>;
+def : Pat<(v1i128 (bitconvert (v4f32 VRRC:$src))), (v1i128 VRRC:$src)>;
+def : Pat<(v1i128 (bitconvert (v2i64 VRRC:$src))), (v1i128 VRRC:$src)>;
 
 // Shuffles.
 
 // Match vsldoi(x,x), vpkuwum(x,x), vpkuhum(x,x)
 def:Pat<(vsldoi_unary_shuffle:$in v16i8:$vA, undef),
         (VSLDOI $vA, $vA, (VSLDOI_unary_get_imm $in))>;
 def:Pat<(vpkuwum_unary_shuffle v16i8:$vA, undef),
         (VPKUWUM $vA, $vA)>;
def VSRAD : VXForm_1<964, (outs vrrc:$vD), (ins vrrc:$vA, vrrc:$vB),
                     "vsrad $vD, $vA, $vB", IIC_VecGeneral,
                     [(set v2i64:$vD, (sra v2i64:$vA, v2i64:$vB))]>;
 
 // Vector Integer Arithmetic Instructions
 let isCommutable = 1 in {
 def VADDUDM : VXForm_1<192, (outs vrrc:$vD), (ins vrrc:$vA, vrrc:$vB),
                        "vaddudm $vD, $vA, $vB", IIC_VecGeneral,
                        [(set v2i64:$vD, (add v2i64:$vA, v2i64:$vB))]>;
+def VADDUQM : VXForm_1<256, (outs vrrc:$vD), (ins vrrc:$vA, vrrc:$vB),
+                       "vadduqm $vD, $vA, $vB", IIC_VecGeneral,
+                       [(set v1i128:$vD, (add v1i128:$vA, v1i128:$vB))]>;
 } // isCommutable
 
+// Vector Quadword Add
+def VADDEUQM : VA1a_Int_Ty<60, "vaddeuqm", int_ppc_altivec_vaddeuqm, v1i128>;
+def VADDCUQ  : VX1_Int_Ty<320, "vaddcuq", int_ppc_altivec_vaddcuq, v1i128>;
+def VADDECUQ : VA1a_Int_Ty<61, "vaddecuq", int_ppc_altivec_vaddecuq, v1i128>;
+
+// Vector Doubleword Subtract
 def VSUBUDM : VXForm_1<1216, (outs vrrc:$vD), (ins vrrc:$vA, vrrc:$vB),
                        "vsubudm $vD, $vA, $vB", IIC_VecGeneral,
                        [(set v2i64:$vD, (sub v2i64:$vA, v2i64:$vB))]>;
 
+// Vector Quadword Subtract
+def VSUBUQM : VXForm_1<1280, (outs vrrc:$vD), (ins vrrc:$vA, vrrc:$vB),
+                       "vsubuqm $vD, $vA, $vB", IIC_VecGeneral,
+                       [(set v1i128:$vD, (sub v1i128:$vA, v1i128:$vB))]>;
+def VSUBEUQM : VA1a_Int_Ty<62, "vsubeuqm", int_ppc_altivec_vsubeuqm, v1i128>;
+def VSUBCUQ  : VX1_Int_Ty<1344, "vsubcuq", int_ppc_altivec_vsubcuq, v1i128>;
+def VSUBECUQ : VA1a_Int_Ty<63, "vsubecuq", int_ppc_altivec_vsubecuq, v1i128>;
+
 // Count Leading Zeros
 def VCLZB : VXForm_2<1794, (outs vrrc:$vD), (ins vrrc:$vB),
                      "vclzb $vD, $vB", IIC_VecGeneral,
                      [(set v16i8:$vD, (ctlz v16i8:$vB))]>;
 def VCLZH : VXForm_2<1858, (outs vrrc:$vD), (ins vrrc:$vB),
                      "vclzh $vD, $vB", IIC_VecGeneral,
                      [(set v8i16:$vD, (ctlz v8i16:$vB))]>;
 def VCLZW : VXForm_2<1922, (outs vrrc:$vD), (ins vrrc:$vB),

lib/Target/PowerPC/PPCInstrVSX.td

 def : Pat<(v16i8 (bitconvert v2i64:$A)),
           (COPY_TO_REGCLASS $A, VRRC)>;
 
 def : Pat<(v2f64 (bitconvert v2i64:$A)),
           (COPY_TO_REGCLASS $A, VRRC)>;
 def : Pat<(v2i64 (bitconvert v2f64:$A)),
           (COPY_TO_REGCLASS $A, VRRC)>;
 
+def : Pat<(v2f64 (bitconvert v1i128:$A)),
+          (COPY_TO_REGCLASS $A, VRRC)>;
+def : Pat<(v1i128 (bitconvert v2f64:$A)),
+          (COPY_TO_REGCLASS $A, VRRC)>;
+
 // sign extension patterns
 // To extend "in place" from v2i32 to v2i64, we have input data like:
 // | undef | i32 | undef | i32 |
 // but xvcvsxwdp expects the input in big-Endian format:
 // | i32 | undef | i32 | undef |
 // so we need to shift everything to the left by one i32 (word) before
 // the conversion.
 def : Pat<(sext_inreg v2i64:$C, v2i32),

lib/Target/PowerPC/PPCRegisterInfo.td

 // register N is spilled to offset 8 * (32 - N) below the back chain word of the
 // previous stack frame. By allocating non-volatiles in reverse order we make
 // sure that the Floating-point register save area is always as small as
 // possible because there aren't any unused spill slots.
 def F8RC : RegisterClass<"PPC", [f64], 64, (add (sequence "F%u", 0, 13),
                                                 (sequence "F%u", 31, 14))>;
 def F4RC : RegisterClass<"PPC", [f32], 32, (add F8RC)>;
 
-def VRRC : RegisterClass<"PPC", [v16i8,v8i16,v4i32,v2i64,v4f32], 128,
+def VRRC : RegisterClass<"PPC", [v16i8,v8i16,v4i32,v2i64,v1i128,v4f32], 128,
                          (add V2, V3, V4, V5, V0, V1, V6, V7, V8, V9, V10, V11,
                              V12, V13, V14, V15, V16, V17, V18, V19, V31, V30,
                              V29, V28, V27, V26, V25, V24, V23, V22, V21, V20)>;
 
 // VSX register classes (the allocation order mirrors that of the corresponding
 // subregister classes).
 def VSLRC : RegisterClass<"PPC", [v4i32,v4f32,v2f64,v2i64], 128,
                           (add (sequence "VSL%u", 0, 13),

test/CodeGen/PowerPC/vec_add_sub_quadword.ll

+; Check VMX 128-bit integer operations
+;
+; RUN: llc -mtriple=powerpc64-unknown-linux-gnu -mcpu=pwr8 < %s | FileCheck %s
+; RUN: llc -mtriple=powerpc64-unknown-linux-gnu -mcpu=pwr8 -mattr=-vsx < %s | FileCheck %s
+
+define <1 x i128> @test_add(<1 x i128> %x, <1 x i128> %y) nounwind {
+       %result = add <1 x i128> %x, %y
+       ret <1 x i128> %result
+; CHECK: vadduqm 2, 2, 3
+}
+
+define <1 x i128> @increment_by_one(<1 x i128> %x) nounwind {
+       %result = add <1 x i128> %x, <i128 1>
+       ret <1 x i128> %result
+; CHECK vadduqm 2, 2, 3*
+}
+
+define <1 x i128> @increment_by_val(<1 x i128> %x, i128 %val) nounwind {
+       %tmpvec = insertelement <1 x i128> <i128 0>, i128 %val, i32 0
+       %tmpvec2 = insertelement <1 x i128> %tmpvec, i128 %val, i32 1
+       %result = add <1 x i128> %x, %tmpvec2
+       ret <1 x i128> %result
+; CHECK: vadduqm 2, 2, 3
+}
+
+define <1 x i128> @test_sub(<1 x i128> %x, <1 x i128> %y) nounwind {
+       %result = sub <1 x i128> %x, %y
+       ret <1 x i128> %result
+; CHECK: vsubuqm 2, 2, 3
+}
+
+define <1 x i128> @decrement_by_one(<1 x i128> %x) nounwind {
+       %result = sub <1 x i128> %x, <i128 1>
+       ret <1 x i128> %result
+; CHECK vsubuqm 2, 2, 3
+}
+
+define <1 x i128> @decrement_by_val(<1 x i128> %x, i128 %val) nounwind {
+       %tmpvec = insertelement <1 x i128> <i128 0>, i128 %val, i32 0
+       %tmpvec2 = insertelement <1 x i128> %tmpvec, i128 %val, i32 1
+       %result = sub <1 x i128> %x, %tmpvec2
+       ret <1 x i128> %result
+; CHECK vsubuqm   2, 2, 3
+}
+
+declare <1 x i128> @llvm.ppc.altivec.vaddeuqm(<1 x i128> %x, 
+                                              <1 x i128> %y, 
+                                              <1 x i128> %z) nounwind readnone
+declare <1 x i128> @llvm.ppc.altivec.vaddcuq(<1 x i128> %x, 
+                                             <1 x i128> %y) nounwind readnone
+declare <1 x i128> @llvm.ppc.altivec.vaddecuq(<1 x i128> %x, 
+                                              <1 x i128> %y, 
+                                              <1 x i128> %z) nounwind readnone
+declare <1 x i128> @llvm.ppc.altivec.vsubeuqm(<1 x i128> %x, 
+                                              <1 x i128> %y, 
+                                              <1 x i128> %z) nounwind readnone
+declare <1 x i128> @llvm.ppc.altivec.vsubcuq(<1 x i128> %x, 
+                                             <1 x i128> %y) nounwind readnone
+declare <1 x i128> @llvm.ppc.altivec.vsubecuq(<1 x i128> %x, 
+                                              <1 x i128> %y, 
+                                              <1 x i128> %z) nounwind readnone
+
+define <1 x i128> @test_vaddeuqm(<1 x i128> %x, 
+                               <1 x i128> %y, 
+                                 <1 x i128> %z) nounwind {
+  %tmp = tail call <1 x i128> @llvm.ppc.altivec.vaddeuqm(<1 x i128> %x,
+                                                         <1 x i128> %y,
+                                                         <1 x i128> %z)
+  ret <1 x i128> %tmp
+; CHECK-LABEL: test_vaddeuqm
+; CHECK: vaddeuqm 2, 2, 3, 4
+}
+
+define <1 x i128> @test_vaddcuq(<1 x i128> %x, 
+                               <1 x i128> %y) nounwind {
+  %tmp = tail call <1 x i128> @llvm.ppc.altivec.vaddcuq(<1 x i128> %x,
+                                                        <1 x i128> %y)
+  ret <1 x i128> %tmp
+; CHECK-LABEL: test_vaddcuq
+; CHECK: vaddcuq 2, 2, 3
+}
+
+define <1 x i128> @test_vaddecuq(<1 x i128> %x, 
+                               <1 x i128> %y, 
+                                 <1 x i128> %z) nounwind {
+  %tmp = tail call <1 x i128> @llvm.ppc.altivec.vaddecuq(<1 x i128> %x,
+                                                         <1 x i128> %y,
+                                                         <1 x i128> %z)
+  ret <1 x i128> %tmp
+; CHECK-LABEL: test_vaddecuq
+; CHECK: vaddecuq 2, 2, 3, 4
+}
+
+define <1 x i128> @test_vsubeuqm(<1 x i128> %x, 
+                               <1 x i128> %y, 
+                                 <1 x i128> %z) nounwind {
+  %tmp = tail call <1 x i128> @llvm.ppc.altivec.vsubeuqm(<1 x i128> %x,
+                                                         <1 x i128> %y,
+                                                         <1 x i128> %z)
+  ret <1 x i128> %tmp
+; CHECK-LABEL: test_vsubeuqm
+; CHECK: vsubeuqm 2, 2, 3, 4
+}
+
+define <1 x i128> @test_vsubcuq(<1 x i128> %x, 
+                               <1 x i128> %y) nounwind {
+  %tmp = tail call <1 x i128> @llvm.ppc.altivec.vsubcuq(<1 x i128> %x,
+                                                        <1 x i128> %y)
+  ret <1 x i128> %tmp
+; CHECK-LABEL: test_vsubcuq
+; CHECK: vsubcuq 2, 2, 3
+}
+
+define <1 x i128> @test_vsubecuq(<1 x i128> %x, 
+                               <1 x i128> %y, 
+                                 <1 x i128> %z) nounwind {
+  %tmp = tail call <1 x i128> @llvm.ppc.altivec.vsubecuq(<1 x i128> %x,
+                                                         <1 x i128> %y,
+                                                         <1 x i128> %z)
+  ret <1 x i128> %tmp
+; CHECK-LABEL: test_vsubecuq
+; CHECK: vsubecuq 2, 2, 3, 4
+}
+

test/MC/Disassembler/PowerPC/ppc64-encoding-vmx.txt

 0x10 0x43 0x22 0x00
 
 # CHECK: vadduhs 2, 3, 4                 
 0x10 0x43 0x22 0x40
 
 # CHECK: vadduws 2, 3, 4                 
 0x10 0x43 0x22 0x80
 
+# CHECK: vadduqm 2, 3, 4
+0x10 0x43 0x21 0x00
+
+# CHECK: vaddeuqm 2, 3, 4, 5
+0x10 0x43 0x21 0x7c
+
+# CHECK: vaddcuq 2, 3, 4
+0x10 0x43 0x21 0x40
+
+# CHECK: vaddecuq 2, 3, 4, 5
+0x10 0x43 0x21 0x7d
+
 # CHECK: vsubcuw 2, 3, 4                 
 0x10 0x43 0x25 0x80
 
 # CHECK: vsubsbs 2, 3, 4                 
 0x10 0x43 0x27 0x00
 
 # CHECK: vsubshs 2, 3, 4                 
 0x10 0x43 0x27 0x40

 0x10 0x43 0x26 0x00
 
 # CHECK: vsubuhs 2, 3, 4                 
 0x10 0x43 0x26 0x40
 
 # CHECK: vsubuws 2, 3, 4                 
 0x10 0x43 0x26 0x80
 
+# CHECK: vsubuqm 2, 3, 4
+0x10 0x43 0x25 0x00
+
+# CHECK: vsubeuqm 2, 3, 4, 5
+0x10 0x43 0x21 0x7e
+
+# CHECK: vsubcuq 2, 3, 4
+0x10 0x43 0x25 0x40
+
+# CHECK: vsubecuq 2, 3, 4, 5
+0x10 0x43 0x21 0x7f
+
 # CHECK: vmulesb 2, 3, 4                 
 0x10 0x43 0x23 0x08
 
 # CHECK: vmulesh 2, 3, 4                 
 0x10 0x43 0x23 0x48
 
 # CHECK: vmulesw 2, 3, 4
 0x10 0x43 0x23 0x88

test/MC/PowerPC/ppc64-encoding-vmx.s

 # CHECK-LE: vaddubs 2, 3, 4                 # encoding: [0x00,0x22,0x43,0x10]
             vaddubs 2, 3, 4
 # CHECK-BE: vadduhs 2, 3, 4                 # encoding: [0x10,0x43,0x22,0x40]
 # CHECK-LE: vadduhs 2, 3, 4                 # encoding: [0x40,0x22,0x43,0x10]
             vadduhs 2, 3, 4
 # CHECK-BE: vadduws 2, 3, 4                 # encoding: [0x10,0x43,0x22,0x80]
 # CHECK-LE: vadduws 2, 3, 4                 # encoding: [0x80,0x22,0x43,0x10]
             vadduws 2, 3, 4
+# CHECK-BE: vadduqm 2, 3, 4                 # encoding: [0x10,0x43,0x21,0x00]
+# CHECK-LE: vadduqm 2, 3, 4                 # encoding: [0x00,0x21,0x43,0x10]
+            vadduqm 2, 3, 4
+# CHECK-BE: vaddeuqm 2, 3, 4, 5             # encoding: [0x10,0x43,0x21,0x7c]
+# CHECK-LE: vaddeuqm 2, 3, 4, 5             # encoding: [0x7c,0x21,0x43,0x10]
+            vaddeuqm 2, 3, 4, 5
+# CHECK-BE: vaddcuq 2, 3, 4                 # encoding: [0x10,0x43,0x21,0x40]
+# CHECK-LE: vaddcuq 2, 3, 4                 # encoding: [0x40,0x21,0x43,0x10]
+            vaddcuq 2, 3, 4
+# CHECK-BE: vaddecuq 2, 3, 4, 5             # encoding: [0x10,0x43,0x21,0x7d]
+# CHECK-LE: vaddecuq 2, 3, 4, 5             # encoding: [0x7d,0x21,0x43,0x10]
+            vaddecuq 2, 3, 4, 5   
   
 # CHECK-BE: vsubcuw 2, 3, 4                 # encoding: [0x10,0x43,0x25,0x80]
 # CHECK-LE: vsubcuw 2, 3, 4                 # encoding: [0x80,0x25,0x43,0x10]
             vsubcuw 2, 3, 4
 # CHECK-BE: vsubsbs 2, 3, 4                 # encoding: [0x10,0x43,0x27,0x00]
 # CHECK-LE: vsubsbs 2, 3, 4                 # encoding: [0x00,0x27,0x43,0x10]
             vsubsbs 2, 3, 4
 # CHECK-BE: vsubshs 2, 3, 4                 # encoding: [0x10,0x43,0x27,0x40]
 # CHECK-LE: vsubshs 2, 3, 4                 # encoding: [0x40,0x27,0x43,0x10]

 # CHECK-LE: vsububs 2, 3, 4                 # encoding: [0x00,0x26,0x43,0x10]
             vsububs 2, 3, 4
 # CHECK-BE: vsubuhs 2, 3, 4                 # encoding: [0x10,0x43,0x26,0x40]
 # CHECK-LE: vsubuhs 2, 3, 4                 # encoding: [0x40,0x26,0x43,0x10]
             vsubuhs 2, 3, 4
 # CHECK-BE: vsubuws 2, 3, 4                 # encoding: [0x10,0x43,0x26,0x80]
 # CHECK-LE: vsubuws 2, 3, 4                 # encoding: [0x80,0x26,0x43,0x10]
             vsubuws 2, 3, 4
+# CHECK-BE: vsubuqm 2, 3, 4                 # encoding: [0x10,0x43,0x25,0x00]
+# CHECK-LE: vsubuqm 2, 3, 4                 # encoding: [0x00,0x25,0x43,0x10]
+            vsubuqm 2, 3, 4
+# CHECK-BE: vsubeuqm 2, 3, 4, 5             # encoding: [0x10,0x43,0x21,0x7e]
+# CHECK-LE: vsubeuqm 2, 3, 4, 5             # encoding: [0x7e,0x21,0x43,0x10]
+            vsubeuqm 2, 3, 4, 5
+# CHECK-BE: vsubcuq 2, 3, 4                 # encoding: [0x10,0x43,0x25,0x40] 
+# CHECK-LE: vsubcuq 2, 3, 4                 # encoding: [0x40,0x25,0x43,0x10]
+            vsubcuq 2, 3, 4
+# CHECK-BE: vsubecuq 2, 3, 4, 5             # encoding: [0x10,0x43,0x21,0x7f]
+# CHECK-LE: vsubecuq 2, 3, 4, 5             # encoding: [0x7f,0x21,0x43,0x10]
+            vsubecuq 2, 3, 4, 5 
   
 # CHECK-BE: vmulesb 2, 3, 4                 # encoding: [0x10,0x43,0x23,0x08]
 # CHECK-LE: vmulesb 2, 3, 4                 # encoding: [0x08,0x23,0x43,0x10]
             vmulesb 2, 3, 4
 # CHECK-BE: vmulesh 2, 3, 4                 # encoding: [0x10,0x43,0x23,0x48]
 # CHECK-LE: vmulesh 2, 3, 4                 # encoding: [0x48,0x23,0x43,0x10]
             vmulesh 2, 3, 4
 # CHECK-BE: vmulesw 2, 3, 4                 # encoding: [0x10,0x43,0x23,0x88]
 # CHECK-LE: vmulesw 2, 3, 4                 # encoding: [0x88,0x23,0x43,0x10]

utils/TableGen/IntrinsicEmitter.cpp

enum IIT_Info {
   IIT_EXTEND_ARG = 24,
   IIT_TRUNC_ARG = 25,
   IIT_ANYPTR = 26,
   IIT_V1   = 27,
   IIT_VARARG = 28,
   IIT_HALF_VEC_ARG = 29,
   IIT_SAME_VEC_WIDTH_ARG = 30,
   IIT_PTR_TO_ARG = 31,
-  IIT_VEC_OF_PTRS_TO_ELT = 32
+  IIT_VEC_OF_PTRS_TO_ELT = 32,
+  IIT_I128 = 33
 };
 
 
 static void EncodeFixedValueType(MVT::SimpleValueType VT,
                                  std::vector<unsigned char> &Sig) {
   if (MVT(VT).isInteger()) {
     unsigned BitWidth = MVT(VT).getSizeInBits();
     switch (BitWidth) {
     default: PrintFatalError("unhandled integer type width in intrinsic!");
     case 1: return Sig.push_back(IIT_I1);
     case 8: return Sig.push_back(IIT_I8);
     case 16: return Sig.push_back(IIT_I16);
     case 32: return Sig.push_back(IIT_I32);
     case 64: return Sig.push_back(IIT_I64);
+    case 128: return Sig.push_back(IIT_I128);  
     }
   }
 
   switch (VT) {
   default: PrintFatalError("unhandled MVT in intrinsic!");
   case MVT::f16: return Sig.push_back(IIT_F16);
   case MVT::f32: return Sig.push_back(IIT_F32);
   case MVT::f64: return Sig.push_back(IIT_F64);