[SystemZ::TTI] i8/i16 operands extension cost revisited

[jonpa]

Summary

I have in this patch looked at the extra costs added for i8/16 instructions that requires the operands to be sign/zero extended.

Three minor changes:

• The operands extension cost for divides could be removed with NFC on spec. This should be because we now have such a high cost already for the divide (20).
• I started looking at the costs for lhsr/ashr and added a check if the shifted value is a load, in which case it should not be needed. This seemed like an improvement... However, since nearly all the cases here were loads, I tried removing this entirely, so that all lhsr/ashr instructions get a cost of 1 no matter the bitwidth. This was nearly the same result - just one file changed by SLP and probably for the better. So it seemed this could be removed, which I did.
• For ICmp instructions, instead of adding 2 all the time for extending each operand, this is only done if that operand is neither a load or an immediate. Again, this seemed like an improvement, but I also tried removing the extra cost entirely. There were a few more changes there on benchmark, so I ended up not removing it.

I am however not sure this extra function is really motivated and could perhaps look in more detail what happens if I remove it. It seemed at least in the one case I looked at where things changed (after removing all extra costs) that there actually was not any extra extension of the operand, so it seems again that this does not happen all the time.

I could remove the check for a comparison of memory with a small immediate, since those cases have a 0 operands extension cost, which is equivalent to checking the operands per this patch.

I am not really sure anymore about considering these instructions to require extensions of the operands. It's hard to tell when this is really needed or not. This patch adds the awareness of cases when this should not be needed, for instance in the case of a load, so in that sense it's an improvement at least.

[uweigand]

LGTM, thanks!

[jonpa]

r347961

lib/Target/SystemZ/SystemZTargetTransformInfo.cpp

else {  // Scalar:
     if (Opcode == Instruction::FAdd || Opcode == Instruction::FSub ||
         Opcode == Instruction::FMul || Opcode == Instruction::FDiv)
       return 1;
 
     // There is no native support for FRem.
     if (Opcode == Instruction::FRem)
       return LIBCALL_COST;
 
-    if (Opcode == Instruction::LShr || Opcode == Instruction::AShr)
-      return (ScalarBits >= 32 ? 1 : 2 /*ext*/);
-
     // Or requires one instruction, although it has custom handling for i64.
     if (Opcode == Instruction::Or)
       return 1;
 
     if (Opcode == Instruction::Xor && ScalarBits == 1) {
       if (ST->hasLoadStoreOnCond2())
         return 5; // 2 * (li 0; loc 1); xor
       return 7; // 2 * ipm sequences ; xor ; shift ; compare
     }
 
     if (DivRemConstPow2)
       return (SignedDivRem ? SDivPow2Cost : 1);
     if (DivRemConst)
       return DivMulSeqCost;
-    if (SignedDivRem)
-      // sext of op(s) for narrow types
-      return DivInstrCost + (ScalarBits < 32 ? 3 : (ScalarBits == 32 ? 1 : 0));
-    if (UnsignedDivRem)
-      // Clearing of low 64 bit reg + sext of op(s) for narrow types + dl[g]r
-      return DivInstrCost + (ScalarBits < 32 ? 3 : 1);
+    if (SignedDivRem || UnsignedDivRem)
+      return DivInstrCost;
   }
 
   // Fallback to the default implementation.
   return BaseT::getArithmeticInstrCost(Opcode, Ty, Op1Info, Op2Info,
                                        Opd1PropInfo, Opd2PropInfo, Args);
 }
 
 int SystemZTTIImpl::getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index,
if ((Opcode == Instruction::ZExt || Opcode == Instruction::SExt) &&
         Cost++;
       return Cost;
     }
   }
 
   return BaseT::getCastInstrCost(Opcode, Dst, Src, I);
 }
 
+// Scalar i8 / i16 operations will typically be made after first extending
+// the operands to i32.
+static unsigned getOperandsExtensionCost(const Instruction *I) {
+  unsigned ExtCost = 0;
+  for (Value *Op : I->operands())
+    // A load of i8 or i16 sign/zero extends to i32.
+    if (!isa<LoadInst>(Op) && !isa<ConstantInt>(Op))
+      ExtCost++;
+
+  return ExtCost;
+}
+
 int SystemZTTIImpl::getCmpSelInstrCost(unsigned Opcode, Type *ValTy,
                                        Type *CondTy, const Instruction *I) {
   if (ValTy->isVectorTy()) {
     assert (ST->hasVector() && "getCmpSelInstrCost() called with vector type.");
     unsigned VF = ValTy->getVectorNumElements();
 
     // Called with a compare instruction.
     if (Opcode == Instruction::ICmp || Opcode == Instruction::FCmp) {
else { // Called with a select instruction.
 
       return getNumVectorRegs(ValTy) /*vsel*/ + PackCost;
     }
   }
   else { // Scalar
     switch (Opcode) {
     case Instruction::ICmp: {
       unsigned Cost = 1;
-      if (ValTy->isIntegerTy() && ValTy->getScalarSizeInBits() <= 16) {
-        if (I != nullptr) {
-          // Single instruction for comparison of memory with a small immediate.
-          if (const LoadInst* Ld = dyn_cast<LoadInst>(I->getOperand(0))) {
-            const Instruction *FoldedValue = nullptr;
-            if (isFoldableLoad(Ld, FoldedValue))
-              return Cost;
-          }
-        }
-        Cost += 2; // extend both operands
-      }
+      if (ValTy->isIntegerTy() && ValTy->getScalarSizeInBits() <= 16)
+        Cost += (I != nullptr ? getOperandsExtensionCost(I) : 2);
       return Cost;
     }
     case Instruction::Select:
       if (ValTy->isFloatingPointTy())
         return 4; // No load on condition for FP - costs a conditional jump.
       return 1; // Load On Condition.
     }
   }

test/Analysis/CostModel/SystemZ/divrem-reg.ll

define i64 @fun0(i64 %a, i64 %b) {
   %r = sdiv i64 %a, %b
   ret i64 %r
 ; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %r = sdiv i64
 }
 
 define i32 @fun1(i32 %a, i32 %b) {
   %r = sdiv i32 %a, %b
   ret i32 %r
-; CHECK: Cost Model: Found an estimated cost of 21 for instruction:   %r = sdiv i32 %a, %b
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %r = sdiv i32 %a, %b
 }
 
 define i16 @fun2(i16 %a, i16 %b) {
   %r = sdiv i16 %a, %b
   ret i16 %r
-; CHECK: Cost Model: Found an estimated cost of 23 for instruction:   %r = sdiv i16 %a, %b
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %r = sdiv i16 %a, %b
 }
 
 define i8 @fun3(i8 %a, i8 %b) {
   %r = sdiv i8 %a, %b
   ret i8 %r
-; CHECK: Cost Model: Found an estimated cost of 23 for instruction:   %r = sdiv i8 %a, %b
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %r = sdiv i8 %a, %b
 }
 
 ; Vector sdiv
 
 define <2 x i64> @fun4(<2 x i64> %a, <2 x i64> %b) {
   %r = sdiv <2 x i64> %a, %b
   ret <2 x i64> %r
 ; CHECK: Cost Model: Found an estimated cost of 47 for instruction:   %r = sdiv <2 x i64>
 }
 
 define <4 x i32> @fun5(<4 x i32> %a, <4 x i32> %b) {
   %r = sdiv <4 x i32> %a, %b
   ret <4 x i32> %r
-; CHECK: Cost Model: Found an estimated cost of 98 for instruction:   %r = sdiv <4 x i32>
+; CHECK: Cost Model: Found an estimated cost of 94 for instruction:   %r = sdiv <4 x i32>
 }
 
 define <2 x i32> @fun6(<2 x i32> %a, <2 x i32> %b) {
   %r = sdiv <2 x i32> %a, %b
   ret <2 x i32> %r
-; CHECK: Cost Model: Found an estimated cost of 50 for instruction:   %r = sdiv <2 x i32>
+; CHECK: Cost Model: Found an estimated cost of 48 for instruction:   %r = sdiv <2 x i32>
 }
 
 define <8 x i16> @fun7(<8 x i16> %a, <8 x i16> %b) {
   %r = sdiv <8 x i16> %a, %b
   ret <8 x i16> %r
 ; CHECK: Cost Model: Found an estimated cost of 1000 for instruction:   %r = sdiv <8 x i16>
 }
 
 define <4 x i16> @fun8(<4 x i16> %a, <4 x i16> %b) {
   %r = sdiv <4 x i16> %a, %b
   ret <4 x i16> %r
-; CHECK: Cost Model: Found an estimated cost of 106 for instruction:   %r = sdiv <4 x i16>
+; CHECK: Cost Model: Found an estimated cost of 94 for instruction:   %r = sdiv <4 x i16>
 }
 
 define <16 x i8> @fun9(<16 x i8> %a, <16 x i8> %b) {
   %r = sdiv <16 x i8> %a, %b
   ret <16 x i8> %r
 ; CHECK: Cost Model: Found an estimated cost of 1000 for instruction:   %r = sdiv <16 x i8>
 }
 
 define <8 x i8> @fun10(<8 x i8> %a, <8 x i8> %b) {
   %r = sdiv <8 x i8> %a, %b
   ret <8 x i8> %r
 ; CHECK: Cost Model: Found an estimated cost of 1000 for instruction:   %r = sdiv <8 x i8>
 }
 
 ; Scalar udiv
 
 define i64 @fun11(i64 %a, i64 %b) {
   %r = udiv i64 %a, %b
   ret i64 %r
-; CHECK: Cost Model: Found an estimated cost of 21 for instruction:   %r = udiv i64 %a, %b
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %r = udiv i64 %a, %b
 }
 
 define i32 @fun12(i32 %a, i32 %b) {
   %r = udiv i32 %a, %b
   ret i32 %r
-; CHECK: Cost Model: Found an estimated cost of 21 for instruction:   %r = udiv i32
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %r = udiv i32
 }
 
 define i16 @fun13(i16 %a, i16 %b) {
   %r = udiv i16 %a, %b
   ret i16 %r
-; CHECK: Cost Model: Found an estimated cost of 23 for instruction:   %r = udiv i16
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %r = udiv i16
 }
 
 define i8 @fun14(i8 %a, i8 %b) {
   %r = udiv i8 %a, %b
   ret i8 %r
-; CHECK: Cost Model: Found an estimated cost of 23 for instruction:   %r = udiv i8
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %r = udiv i8
 }
 
 ; Vector udiv
 
 define <2 x i64> @fun15(<2 x i64> %a, <2 x i64> %b) {
   %r = udiv <2 x i64> %a, %b
   ret <2 x i64> %r
-; CHECK: Cost Model: Found an estimated cost of 49 for instruction:   %r = udiv <2 x i64>
+; CHECK: Cost Model: Found an estimated cost of 47 for instruction:   %r = udiv <2 x i64>
 }
 
 define <4 x i32> @fun16(<4 x i32> %a, <4 x i32> %b) {
   %r = udiv <4 x i32> %a, %b
   ret <4 x i32> %r
-; CHECK: Cost Model: Found an estimated cost of 98 for instruction:   %r = udiv <4 x i32>
+; CHECK: Cost Model: Found an estimated cost of 94 for instruction:   %r = udiv <4 x i32>
 }
 
 define <2 x i32> @fun17(<2 x i32> %a, <2 x i32> %b) {
   %r = udiv <2 x i32> %a, %b
   ret <2 x i32> %r
-; CHECK: Cost Model: Found an estimated cost of 50 for instruction:   %r = udiv <2 x i32>
+; CHECK: Cost Model: Found an estimated cost of 48 for instruction:   %r = udiv <2 x i32>
 }
 
 define <8 x i16> @fun18(<8 x i16> %a, <8 x i16> %b) {
   %r = udiv <8 x i16> %a, %b
   ret <8 x i16> %r
 ; CHECK: Cost Model: Found an estimated cost of 1000 for instruction:   %r = udiv <8 x i16>
 }
 
 define <4 x i16> @fun19(<4 x i16> %a, <4 x i16> %b) {
   %r = udiv <4 x i16> %a, %b
   ret <4 x i16> %r
-; CHECK: Cost Model: Found an estimated cost of 106 for instruction:   %r = udiv <4 x i16>
+; CHECK: Cost Model: Found an estimated cost of 94 for instruction:   %r = udiv <4 x i16>
 }
 
 define <16 x i8> @fun20(<16 x i8> %a, <16 x i8> %b) {
   %r = udiv <16 x i8> %a, %b
   ret <16 x i8> %r
 ; CHECK: Cost Model: Found an estimated cost of 1000 for instruction:   %r = udiv <16 x i8>
 }
 
define i64 @fun22(i64 %a, i64 %b) {
   %r = srem i64 %a, %b
   ret i64 %r
 ; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %r = srem i64
 }
 
 define i32 @fun23(i32 %a, i32 %b) {
   %r = srem i32 %a, %b
   ret i32 %r
-; CHECK: Cost Model: Found an estimated cost of 21 for instruction:   %r = srem i32
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %r = srem i32
 }
 
 define i16 @fun24(i16 %a, i16 %b) {
   %r = srem i16 %a, %b
   ret i16 %r
-; CHECK: Cost Model: Found an estimated cost of 23 for instruction:   %r = srem i16
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %r = srem i16
 }
 
 define i8 @fun25(i8 %a, i8 %b) {
   %r = srem i8 %a, %b
   ret i8 %r
-; CHECK: Cost Model: Found an estimated cost of 23 for instruction:   %r = srem i8
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %r = srem i8
 }
 
 ; Vector srem
 
 define <2 x i64> @fun26(<2 x i64> %a, <2 x i64> %b) {
   %r = srem <2 x i64> %a, %b
   ret <2 x i64> %r
 ; CHECK: Cost Model: Found an estimated cost of 47 for instruction:   %r = srem <2 x i64>
 }
 
 define <4 x i32> @fun27(<4 x i32> %a, <4 x i32> %b) {
   %r = srem <4 x i32> %a, %b
   ret <4 x i32> %r
-; CHECK: Cost Model: Found an estimated cost of 98 for instruction:   %r = srem <4 x i32>
+; CHECK: Cost Model: Found an estimated cost of 94 for instruction:   %r = srem <4 x i32>
 }
 
 define <2 x i32> @fun28(<2 x i32> %a, <2 x i32> %b) {
   %r = srem <2 x i32> %a, %b
   ret <2 x i32> %r
-; CHECK: Cost Model: Found an estimated cost of 50 for instruction:   %r = srem <2 x i32>
+; CHECK: Cost Model: Found an estimated cost of 48 for instruction:   %r = srem <2 x i32>
 }
 
 define <8 x i16> @fun29(<8 x i16> %a, <8 x i16> %b) {
   %r = srem <8 x i16> %a, %b
   ret <8 x i16> %r
 ; CHECK: ost Model: Found an estimated cost of 1000 for instruction:   %r = srem <8 x i16>
 }
 
 define <4 x i16> @fun30(<4 x i16> %a, <4 x i16> %b) {
   %r = srem <4 x i16> %a, %b
   ret <4 x i16> %r
-; CHECK: Cost Model: Found an estimated cost of 106 for instruction:   %r = srem <4 x i16>
+; CHECK: Cost Model: Found an estimated cost of 94 for instruction:   %r = srem <4 x i16>
 }
 
 define <16 x i8> @fun31(<16 x i8> %a, <16 x i8> %b) {
   %r = srem <16 x i8> %a, %b
   ret <16 x i8> %r
 ; CHECK: Cost Model: Found an estimated cost of 1000 for instruction:   %r = srem <16 x i8>
 }
 
 define <8 x i8> @fun32(<8 x i8> %a, <8 x i8> %b) {
   %r = srem <8 x i8> %a, %b
   ret <8 x i8> %r
 ; CHECK: Cost Model: Found an estimated cost of 1000 for instruction:   %r = srem <8 x i8>
 }
 
 ; Scalar urem
 
 define i64 @fun33(i64 %a, i64 %b) {
   %r = urem i64 %a, %b
   ret i64 %r
-; CHECK: Cost Model: Found an estimated cost of 21 for instruction:   %r = urem i64
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %r = urem i64
 }
 
 define i32 @fun34(i32 %a, i32 %b) {
   %r = urem i32 %a, %b
   ret i32 %r
-; CHECK: Cost Model: Found an estimated cost of 21 for instruction:   %r = urem i32
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %r = urem i32
 }
 
 define i16 @fun35(i16 %a, i16 %b) {
   %r = urem i16 %a, %b
   ret i16 %r
-; CHECK: Cost Model: Found an estimated cost of 23 for instruction:   %r = urem i16
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %r = urem i16
 }
 
 define i8 @fun36(i8 %a, i8 %b) {
   %r = urem i8 %a, %b
   ret i8 %r
-; CHECK: Cost Model: Found an estimated cost of 23 for instruction:   %r = urem i8
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %r = urem i8
 }
 
 ; Vector urem
 
 define <2 x i64> @fun37(<2 x i64> %a, <2 x i64> %b) {
   %r = urem <2 x i64> %a, %b
   ret <2 x i64> %r
-; CHECK: Cost Model: Found an estimated cost of 49 for instruction:   %r = urem <2 x i64>
+; CHECK: Cost Model: Found an estimated cost of 47 for instruction:   %r = urem <2 x i64>
 }
 
 define <4 x i32> @fun38(<4 x i32> %a, <4 x i32> %b) {
   %r = urem <4 x i32> %a, %b
   ret <4 x i32> %r
-; CHECK: Cost Model: Found an estimated cost of 98 for instruction:   %r = urem <4 x i32>
+; CHECK: Cost Model: Found an estimated cost of 94 for instruction:   %r = urem <4 x i32>
 }
 
 define <2 x i32> @fun39(<2 x i32> %a, <2 x i32> %b) {
   %r = urem <2 x i32> %a, %b
   ret <2 x i32> %r
-; CHECK: Cost Model: Found an estimated cost of 50 for instruction:   %r = urem <2 x i32>
+; CHECK: Cost Model: Found an estimated cost of 48 for instruction:   %r = urem <2 x i32>
 }
 
 define <8 x i16> @fun40(<8 x i16> %a, <8 x i16> %b) {
   %r = urem <8 x i16> %a, %b
   ret <8 x i16> %r
 ; CHECK: Cost Model: Found an estimated cost of 1000 for instruction:   %r = urem <8 x i16>
 }
 
 define <4 x i16> @fun41(<4 x i16> %a, <4 x i16> %b) {
   %r = urem <4 x i16> %a, %b
   ret <4 x i16> %r
-; CHECK: Cost Model: Found an estimated cost of 106 for instruction:   %r = urem <4 x i16>
+; CHECK: Cost Model: Found an estimated cost of 94 for instruction:   %r = urem <4 x i16>
 }
 
 define <16 x i8> @fun42(<16 x i8> %a, <16 x i8> %b) {
   %r = urem <16 x i8> %a, %b
   ret <16 x i8> %r
 ; CHECK: Cost Model: Found an estimated cost of 1000 for instruction:   %r = urem <16 x i8>
 }
 
 define <8 x i8> @fun43(<8 x i8> %a, <8 x i8> %b) {
   %r = urem <8 x i8> %a, %b
   ret <8 x i8> %r
 ; CHECK: Cost Model: Found an estimated cost of 1000 for instruction:   %r = urem <8 x i8>
 }

test/Analysis/CostModel/SystemZ/int-operands-extcost.ll

+; RUN: opt < %s -cost-model -analyze -mtriple=systemz-unknown -mcpu=z13 \
+; RUN:  | FileCheck %s
+;
+; Test that i8/i16 operands get extra costs for extensions to i32 only in
+; cases where this is needed.
+
+define void @icmp() {
+  %li8_0 = load i8, i8* undef
+  %li8_1 = load i8, i8* undef
+  icmp slt i8 %li8_0, %li8_1
+
+  %a0 = add i8 %li8_0, 1
+  %a1 = add i8 %li8_1, 1
+  icmp slt i8 %a0, %a1
+
+  icmp slt i8 %a0, 123
+
+  %li16_0 = load i16, i16* undef
+  %li16_1 = load i16, i16* undef
+  icmp slt i16 %li16_0, %li16_1
+
+  %a2 = add i16 %li16_0, 1
+  %a3 = add i16 %li16_1, 1
+  icmp slt i16 %a2, %a3
+
+  icmp slt i16 %a2, 123
+
+  ret void;
+; CHECK: Printing analysis 'Cost Model Analysis' for function 'icmp':
+; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %li8_0 = load i8, i8* undef
+; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %li8_1 = load i8, i8* undef
+; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %1 = icmp slt i8 %li8_0, %li8_1
+; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %a0 = add i8 %li8_0, 1
+; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %a1 = add i8 %li8_1, 1
+; CHECK: Cost Model: Found an estimated cost of 3 for instruction:   %2 = icmp slt i8 %a0, %a1
+; CHECK: Cost Model: Found an estimated cost of 2 for instruction:   %3 = icmp slt i8 %a0, 123
+; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %li16_0 = load i16, i16* undef
+; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %li16_1 = load i16, i16* undef
+; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %4 = icmp slt i16 %li16_0, %li16_1
+; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %a2 = add i16 %li16_0, 1
+; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %a3 = add i16 %li16_1, 1
+; CHECK: Cost Model: Found an estimated cost of 3 for instruction:   %5 = icmp slt i16 %a2, %a3
+; CHECK: Cost Model: Found an estimated cost of 2 for instruction:   %6 = icmp slt i16 %a2, 123
+; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   ret void
+}

test/Analysis/CostModel/SystemZ/logical.ll

define void @ashr() {
   %res13 = ashr <8 x i16> undef, undef
   %res14 = ashr <8 x i32> undef, undef
   %res15 = ashr <8 x i64> undef, undef
   %res16 = ashr <16 x i8> undef, undef
   %res17 = ashr <16 x i16> undef, undef
   %res18 = ashr <16 x i32> undef, undef
   %res19 = ashr <16 x i64> undef, undef
 
-; CHECK: Cost Model: Found an estimated cost of 2 for instruction:   %res0 = ashr i8 undef, undef
-; CHECK: Cost Model: Found an estimated cost of 2 for instruction:   %res1 = ashr i16 undef, undef
+; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res0 = ashr i8 undef, undef
+; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res1 = ashr i16 undef, undef
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res2 = ashr i32 undef, undef
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res3 = ashr i64 undef, undef
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res4 = ashr <2 x i8> undef, undef
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res5 = ashr <2 x i16> undef, undef
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res6 = ashr <2 x i32> undef, undef
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res7 = ashr <2 x i64> undef, undef
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res8 = ashr <4 x i8> undef, undef
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res9 = ashr <4 x i16> undef, undef
define void @lshr() {
   %res13 = lshr <8 x i16> undef, undef
   %res14 = lshr <8 x i32> undef, undef
   %res15 = lshr <8 x i64> undef, undef
   %res16 = lshr <16 x i8> undef, undef
   %res17 = lshr <16 x i16> undef, undef
   %res18 = lshr <16 x i32> undef, undef
   %res19 = lshr <16 x i64> undef, undef
 
-; CHECK: Cost Model: Found an estimated cost of 2 for instruction:   %res0 = lshr i8 undef, undef
-; CHECK: Cost Model: Found an estimated cost of 2 for instruction:   %res1 = lshr i16 undef, undef
+; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res0 = lshr i8 undef, undef
+; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res1 = lshr i16 undef, undef
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res2 = lshr i32 undef, undef
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res3 = lshr i64 undef, undef
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res4 = lshr <2 x i8> undef, undef
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res5 = lshr <2 x i16> undef, undef
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res6 = lshr <2 x i32> undef, undef
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res7 = lshr <2 x i64> undef, undef
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res8 = lshr <4 x i8> undef, undef
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %res9 = lshr <4 x i16> undef, undef

test/Analysis/CostModel/SystemZ/memop-folding-int-arith.ll

define void @sdiv_lhs(i32 %arg32, i64 %arg64) {
   %sext_1 = sext i32 %li32_3 to i64
   %sext_2 = sext i32 %li32_3 to i64
   sdiv i64 %sext_1, undef
 
   ret void;
 
 ; An sdiv loaded dividend (lhs) operand is *not* foldable.
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %li32 = load i32, i32* undef
-; CHECK: Cost Model: Found an estimated cost of 21 for instruction:   %1 = sdiv i32 %li32, %arg32
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %1 = sdiv i32 %li32, %arg32
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %li32_0 = load i32, i32* undef
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %li32_1 = load i32, i32* undef
-; CHECK: Cost Model: Found an estimated cost of 21 for instruction:   %2 = sdiv i32 %li32_0, %li32_1
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %2 = sdiv i32 %li32_0, %li32_1
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %li64 = load i64, i64* undef
 ; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %3 = sdiv i64 %li64, %arg64
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %li64_0 = load i64, i64* undef
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %li64_1 = load i64, i64* undef
 ; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %4 = sdiv i64 %li64_0, %li64_1
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %li64_2 = load i64, i64* undef
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %tr = trunc i64 %li64_2 to i32
 ; CHECK: Cost Model: Found an estimated cost of 10 for instruction:   %5 = sdiv i32 %tr, undef
define void @sdiv_rhs(i32 %arg32, i64 %arg64) {
   %sext_1 = sext i32 %li32_3 to i64
   %sext_2 = sext i32 %li32_3 to i64
   sdiv i64 undef, %sext_1
 
   ret void;
 
 ; An sdiv loaded divisor (rhs) operand is foldable.
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %li32 = load i32, i32* undef
-; CHECK: Cost Model: Found an estimated cost of 21 for instruction:   %1 = sdiv i32 %arg32, %li32
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %1 = sdiv i32 %arg32, %li32
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %li64 = load i64, i64* undef
 ; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %2 = sdiv i64 %arg64, %li64
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %li64_2 = load i64, i64* undef
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %tr = trunc i64 %li64_2 to i32
-; CHECK: Cost Model: Found an estimated cost of 21 for instruction:   %3 = sdiv i32 undef, %tr
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %3 = sdiv i32 undef, %tr
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %li32_2 = load i32, i32* undef
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %sext_0 = sext i32 %li32_2 to i64
 ; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %4 = sdiv i64 undef, %sext_0
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %li32_3 = load i32, i32* undef
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %sext_1 = sext i32 %li32_3 to i64
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %sext_2 = sext i32 %li32_3 to i64
 ; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %5 = sdiv i64 undef, %sext_1
 }
define void @udiv_lhs(i32 %arg32, i64 %arg64) {
   %li64_3 = load i64, i64* undef
   %tr_1 = trunc i64 %li64_3 to i32
   udiv i64 %li64_3, undef
 
   ret void;
 
 ; An udiv loaded dividend (lhs) operand is *not* foldable.
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %li32 = load i32, i32* undef
-; CHECK: Cost Model: Found an estimated cost of 21 for instruction:   %1 = udiv i32 %li32, %arg32
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %1 = udiv i32 %li32, %arg32
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %li32_0 = load i32, i32* undef
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %li32_1 = load i32, i32* undef
-; CHECK: Cost Model: Found an estimated cost of 21 for instruction:   %2 = udiv i32 %li32_0, %li32_1
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %2 = udiv i32 %li32_0, %li32_1
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %li64 = load i64, i64* undef
-; CHECK: Cost Model: Found an estimated cost of 21 for instruction:   %3 = udiv i64 %li64, %arg64
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %3 = udiv i64 %li64, %arg64
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %li64_0 = load i64, i64* undef
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %li64_1 = load i64, i64* undef
-; CHECK: Cost Model: Found an estimated cost of 21 for instruction:   %4 = udiv i64 %li64_0, %li64_1
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %4 = udiv i64 %li64_0, %li64_1
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %li64_2 = load i64, i64* undef
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %tr_0 = trunc i64 %li64_2 to i32
 ; CHECK: Cost Model: Found an estimated cost of 10 for instruction:   %5 = udiv i32 %tr_0, undef
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %li64_3 = load i64, i64* undef
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %tr_1 = trunc i64 %li64_3 to i32
 ; CHECK: Cost Model: Found an estimated cost of 10 for instruction:   %6 = udiv i64 %li64_3, undef
 }
 
define void @udiv_rhs(i32 %arg32, i64 %arg64) {
   %li64_3 = load i64, i64* undef
   %tr_1 = trunc i64 %li64_3 to i32
   udiv i64 undef, %li64_3
 
   ret void;
 
 ; An udiv loaded divisor (rhs) operand is foldable.
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %li32 = load i32, i32* undef
-; CHECK: Cost Model: Found an estimated cost of 21 for instruction:   %1 = udiv i32 %arg32, %li32
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %1 = udiv i32 %arg32, %li32
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %li64 = load i64, i64* undef
-; CHECK: Cost Model: Found an estimated cost of 21 for instruction:   %2 = udiv i64 %arg64, %li64
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %2 = udiv i64 %arg64, %li64
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %li64_2 = load i64, i64* undef
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %tr_0 = trunc i64 %li64_2 to i32
-; CHECK: Cost Model: Found an estimated cost of 21 for instruction:   %3 = udiv i32 undef, %tr_0
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %3 = udiv i32 undef, %tr_0
 ; CHECK: Cost Model: Found an estimated cost of 1 for instruction:   %li64_3 = load i64, i64* undef
 ; CHECK: Cost Model: Found an estimated cost of 0 for instruction:   %tr_1 = trunc i64 %li64_3 to i32
-; CHECK: Cost Model: Found an estimated cost of 21 for instruction:   %4 = udiv i64 undef, %li64_3
+; CHECK: Cost Model: Found an estimated cost of 20 for instruction:   %4 = udiv i64 undef, %li64_3
 }
 
 define void @and() {
   %li32 = load i32, i32* undef
   and i32 %li32, undef
 
   %li32_0 = load i32, i32* undef
   %li32_1 = load i32, i32* undef