[X86][SSE] Improve cost model for i64 vector comparisons on pre-SSE42 targets

[RKSimon]

Summary

As discussed on PR24888, until SSE42 we don't have access to PCMPGTQ for v2i64 comparisons, but the cost models don't reflect this, resulting in over-optimistic vectorizaton.

This patch adds SSE2 'base level' costs that match what a typical target is capable of and only reduces the v2i64 costs at SSE42.

Technically SSE41 provides a PCMPEQQ v2i64 equality test, but as getCmpSelInstrCost doesn't give us a way to discriminate between comparision test types we can't easily make use of this, otherwise we could split the cost of integer equality and greater-than tests to give better costings of each.

[spatel]

LGTM.

Looks like another case that suggests the correct way to solve these kinds of problems in general: derive the TTI cost model directly from the sched machine model.

lib/Target/X86/X86TargetTransformInfo.cpp

int X86TTIImpl::getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy) {
   // Legalize the type.
   std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, ValTy);
 
   MVT MTy = LT.second;
 
   int ISD = TLI->InstructionOpcodeToISD(Opcode);
   assert(ISD && "Invalid opcode");
 
+  static const CostTblEntry SSE2CostTbl[] = {
+    { ISD::SETCC,   MVT::v2i64,   8 },
+    { ISD::SETCC,   MVT::v4i32,   1 },
+    { ISD::SETCC,   MVT::v8i16,   1 },
+    { ISD::SETCC,   MVT::v16i8,   1 },
+  };
+
   static const CostTblEntry SSE42CostTbl[] = {
     { ISD::SETCC,   MVT::v2f64,   1 },
     { ISD::SETCC,   MVT::v4f32,   1 },
     { ISD::SETCC,   MVT::v2i64,   1 },
-    { ISD::SETCC,   MVT::v4i32,   1 },
-    { ISD::SETCC,   MVT::v8i16,   1 },
-    { ISD::SETCC,   MVT::v16i8,   1 },
   };
 
   static const CostTblEntry AVX1CostTbl[] = {
     { ISD::SETCC,   MVT::v4f64,   1 },
     { ISD::SETCC,   MVT::v8f32,   1 },
     // AVX1 does not support 8-wide integer compare.
     { ISD::SETCC,   MVT::v4i64,   4 },
     { ISD::SETCC,   MVT::v8i32,   4 },
int X86TTIImpl::getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy) {
   if (ST->hasAVX())
     if (const auto *Entry = CostTableLookup(AVX1CostTbl, ISD, MTy))
       return LT.first * Entry->Cost;
 
   if (ST->hasSSE42())
     if (const auto *Entry = CostTableLookup(SSE42CostTbl, ISD, MTy))
       return LT.first * Entry->Cost;
 
+  if (ST->hasSSE2())
+    if (const auto *Entry = CostTableLookup(SSE2CostTbl, ISD, MTy))
+      return LT.first * Entry->Cost;
+
   return BaseT::getCmpSelInstrCost(Opcode, ValTy, CondTy);
 }
 
 int X86TTIImpl::getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index) {
   assert(Val->isVectorTy() && "This must be a vector type");
 
   if (Index != -1U) {
     // Legalize the type.

test/Analysis/CostModel/X86/cmp.ll

define i32 @cmp(i32 %arg) {
   ;SSE2:  cost of 1 {{.*}} icmp
   ;SSE3:  cost of 1 {{.*}} icmp
   ;SSSE3: cost of 1 {{.*}} icmp
   ;SSE41: cost of 1 {{.*}} icmp
   ;SSE42: cost of 1 {{.*}} icmp
   ;AVX:   cost of 1 {{.*}} icmp
   %H = icmp eq <4 x i32> undef, undef
 
-  ;SSE2:  cost of 1 {{.*}} icmp
-  ;SSE3:  cost of 1 {{.*}} icmp
-  ;SSSE3: cost of 1 {{.*}} icmp
-  ;SSE41: cost of 1 {{.*}} icmp
+  ;SSE2:  cost of 8 {{.*}} icmp
+  ;SSE3:  cost of 8 {{.*}} icmp
+  ;SSSE3: cost of 8 {{.*}} icmp
+  ;SSE41: cost of 8 {{.*}} icmp
   ;SSE42: cost of 1 {{.*}} icmp
   ;AVX:   cost of 1 {{.*}} icmp
   %I = icmp eq <2 x i64> undef, undef
 
-  ;SSE2:  cost of 2 {{.*}} icmp
-  ;SSE3:  cost of 2 {{.*}} icmp
-  ;SSSE3: cost of 2 {{.*}} icmp
-  ;SSE41: cost of 2 {{.*}} icmp
+  ;SSE2:  cost of 16 {{.*}} icmp
+  ;SSE3:  cost of 16 {{.*}} icmp
+  ;SSSE3: cost of 16 {{.*}} icmp
+  ;SSE41: cost of 16 {{.*}} icmp
   ;SSE42: cost of 2 {{.*}} icmp
   ;AVX1:  cost of 4 {{.*}} icmp
   ;AVX2:  cost of 1 {{.*}} icmp
   %J = icmp eq <4 x i64> undef, undef
 
   ;SSE2:  cost of 2 {{.*}} icmp
   ;SSE3:  cost of 2 {{.*}} icmp
   ;SSSE3: cost of 2 {{.*}} icmp