[InstCombine] Teach canEvaluateTruncated and EvaluateInDifferentType to handle expression tree with multi-used nodes.

[aaboud]

Summary

Added support for patterns like this: http://rise4fun.com/Alive/85d
Where truncate expression contains nodes with multi usages, however all these usages are dominated by the same truncate instruction.

Solution: Instead of returning false when hitting such a node with more than one usage, simply record all the usages that were not visited yet, so they can be checked later against all visited nodes.

[aaboud]

fixed some typos.

[craig.topper]

lib/Transforms/InstCombine/InstCombineCasts.cpp

706

I think a better way to name this is like this

canEvaluateTruncated->canEvaluateTruncatedImpl
canEvaluateTruncatedWrapper->canEvaluateTruncated

I think the Impl name will be more obvious that it shouldn't be called directly.

test/Transforms/InstCombine/cast.ll

1370

tranc->trunc

[aaboud]

Thanks Craig for the comments.
Will upload an updated patch soon.

lib/Transforms/InstCombine/InstCombineCasts.cpp

706

I thought to do that, but then tried to reduce the number of changed lines :)
I will update the patch accordingly.

[aaboud]

Fixed typo and naming according to Craig comments.

[spatel]

I didn't look closely at the implementation, but extending instcombine with visitation/evaluation maps may cross the boundary of what instcombine should handle. Adding some more potential reviewers.

[aaboud]

In D37195#855376, @spatel wrote:

I didn't look closely at the implementation, but extending instcombine with visitation/evaluation maps may cross the boundary of what instcombine should handle. Adding some more potential reviewers.

We could implement this by passing a clean set of visitedInsts/ToVisitInsts per call to these functions, instead of having a member instance in InstCombine.
I do not see why it is much different that passing the demandedBits map and update it recursively!

The only thing that we need to be aware of regarding these two functions I modified, is that they have no "depth" guard, like many others!

[spatel]

In D37195#855385, @aaboud wrote:

I do not see why it is much different that passing the demandedBits map and update it recursively!

That may be true, but there's also an argument that doing demandedBits analysis within instcombine already went over the line. :)
There have been a few recent llvm-dev threads about the responsibilities and limits of various passes. Here's one:
http://lists.llvm.org/pipermail/llvm-dev/2017-May/113212.html

I don't have enough experience with this, so I'll defer to others for their guidance. Just thought it was worth mentioning.

[aaboud]

So, how do you wish to proceed from here?
Do you think that such optimization should be moved to separate new pass? Though it will be doing very similar thing as InstCombine, just will catch more cases that it does today?

Please, share with me your opinion on this optimization.

Thanks,
Amjad

[spatel]

In D37195#860696, @aaboud wrote:

So, how do you wish to proceed from here?
Do you think that such optimization should be moved to separate new pass? Though it will be doing very similar thing as InstCombine, just will catch more cases that it does today?

Please, share with me your opinion on this optimization.

Not sure if the question is for me, but since I raised the doubt...yes, a separate pass that just does evaluation in a narrower type (a subset of demanded bits analysis?) seems like a reasonable thing to implement. Another option is to extend BDCE since that's also using demanded bits. Either way, that would let us lift this task out of instcombine where it is likely costing more than is necessary because instcombine runs so many times. Note that I raised a similar vector possibility for myself in https://reviews.llvm.org/D37236 (vector demanded elements).

That patch does suggest another option. As you've noted, we already do demanded bits analysis (and it even has some allowance for multiple uses) in instcombine, so why doesn't that work? The big fix is to extract that whole thing out of instcombine as its own pass so we can better control it, but (yet another) quick fix is to adjust demanded bits to catch these cases?

As I said, I don't have enough experience to say what's best. Post to llvm-dev to get more opinions?

[aaboud]

New approach was uploaded to D38313.

lib/Transforms/InstCombine/InstCombineCasts.cpp

if (!AI.hasOneUse()) {
     // New is the allocation instruction, pointer typed. AI is the original
     // allocation instruction, also pointer typed. Thus, cast to use is BitCast.
     Value *NewCast = AllocaBuilder.CreateBitCast(New, AI.getType(), "tmpcast");
     replaceInstUsesWith(AI, NewCast);
   }
   return replaceInstUsesWith(CI, New);
 }
 
-/// Given an expression that CanEvaluateTruncated or CanEvaluateSExtd returns
-/// true for, actually insert the code to evaluate the expression.
+/// Given an expression that CanEvaluate{Truncated, SExtd, ZExtd}, returns a new
+/// Value representing the inserted code to evaluate the expression. 
 Value *InstCombiner::EvaluateInDifferentType(Value *V, Type *Ty,
                                                     bool isSigned) {
+  EvaluatedInstMap.clear();
+  return EvaluateInDifferentTypeImpl(V, Ty, isSigned);
+}
+
+/// Recursive helper function for EvaluateInDifferentType.
+/// This function should not be called directly!
+Value *InstCombiner::EvaluateInDifferentTypeImpl(Value *V, Type *Ty,
+                                                 bool isSigned) {
   if (Constant *C = dyn_cast<Constant>(V)) {
     C = ConstantExpr::getIntegerCast(C, Ty, isSigned /*Sext or ZExt*/);
     // If we got a constantexpr back, try to simplify it with DL info.
     if (Constant *FoldedC = ConstantFoldConstant(C, DL, &TLI))
       C = FoldedC;
     return C;
   }
 
   // Otherwise, it must be an instruction.
   Instruction *I = cast<Instruction>(V);
+
+  // Check if this instruction have already been evaluated.
+  if (EvaluatedInstMap.count(I))
+    return EvaluatedInstMap[I];
+
   Instruction *Res = nullptr;
   unsigned Opc = I->getOpcode();
   switch (Opc) {
   case Instruction::Add:
   case Instruction::Sub:
   case Instruction::Mul:
   case Instruction::And:
   case Instruction::Or:
   case Instruction::Xor:
   case Instruction::AShr:
   case Instruction::LShr:
   case Instruction::Shl:
   case Instruction::UDiv:
   case Instruction::URem: {
-    Value *LHS = EvaluateInDifferentType(I->getOperand(0), Ty, isSigned);
-    Value *RHS = EvaluateInDifferentType(I->getOperand(1), Ty, isSigned);
+    Value *LHS = EvaluateInDifferentTypeImpl(I->getOperand(0), Ty, isSigned);
+    Value *RHS = EvaluateInDifferentTypeImpl(I->getOperand(1), Ty, isSigned);
     Res = BinaryOperator::Create((Instruction::BinaryOps)Opc, LHS, RHS);
     break;
   }
   case Instruction::Trunc:
   case Instruction::ZExt:
   case Instruction::SExt:
     // If the source type of the cast is the type we're trying for then we can
     // just return the source.  There's no need to insert it because it is not
     // new.
     if (I->getOperand(0)->getType() == Ty)
       return I->getOperand(0);
 
     // Otherwise, must be the same type of cast, so just reinsert a new one.
     // This also handles the case of zext(trunc(x)) -> zext(x).
     Res = CastInst::CreateIntegerCast(I->getOperand(0), Ty,
                                       Opc == Instruction::SExt);
     break;
   case Instruction::Select: {
-    Value *True = EvaluateInDifferentType(I->getOperand(1), Ty, isSigned);
-    Value *False = EvaluateInDifferentType(I->getOperand(2), Ty, isSigned);
+    Value *True = EvaluateInDifferentTypeImpl(I->getOperand(1), Ty, isSigned);
+    Value *False = EvaluateInDifferentTypeImpl(I->getOperand(2), Ty, isSigned);
     Res = SelectInst::Create(I->getOperand(0), True, False);
     break;
   }
   case Instruction::PHI: {
     PHINode *OPN = cast<PHINode>(I);
     PHINode *NPN = PHINode::Create(Ty, OPN->getNumIncomingValues());
+    // Must update the evaluated instruction map here to prevent infinite loop.
+    EvaluatedInstMap[I] = NPN;
     for (unsigned i = 0, e = OPN->getNumIncomingValues(); i != e; ++i) {
       Value *V =
-          EvaluateInDifferentType(OPN->getIncomingValue(i), Ty, isSigned);
+          EvaluateInDifferentTypeImpl(OPN->getIncomingValue(i), Ty, isSigned);
       NPN->addIncoming(V, OPN->getIncomingBlock(i));
     }
     Res = NPN;
     break;
   }
   default:
     // TODO: Can handle more cases here.
     llvm_unreachable("Unreachable!");
   }
 
+  EvaluatedInstMap[I] = Res;
   Res->takeName(I);
   return InsertNewInstWith(Res, *I);
 }
 
 Instruction::CastOps InstCombiner::isEliminableCastPair(const CastInst *CI1,
                                                         const CastInst *CI2) {
   Type *SrcTy = CI1->getSrcTy();
   Type *MidTy = CI1->getDestTy();

 ///
 /// Ty will always be a type smaller than V.  We should return true if trunc(V)
 /// can be computed by computing V in the smaller type.  If V is an instruction,
 /// then trunc(inst(x,y)) can be computed as inst(trunc(x),trunc(y)), which only
 /// makes sense if x and y can be efficiently truncated.
 ///
 /// This function works on both vectors and scalars.
 ///
-static bool canEvaluateTruncated(Value *V, Type *Ty, InstCombiner &IC,
+bool InstCombiner::canEvaluateTruncated(Value *V, Type *Ty, InstCombiner &IC,
+                                        Instruction *CxtI) {
+  VisitedInsts.clear();
+  ToVisitInsts.clear();
+  if (!canEvaluateTruncatedImpl(V, Ty, IC, CxtI))
+    return false;
+  // We can truncate the given expression tree only if we had visited all
+  // instructions that are expected to be visited during expression evaluation.
+  for (auto *I : ToVisitInsts)
+    if (!VisitedInsts.count(I))
+      return false;
+  return true;
+}
+
+/// Recursive helper function for canEvaluateTruncated.
+/// This function should not be called directly!
+bool InstCombiner::canEvaluateTruncatedImpl(Value *V, Type *Ty,
+                                            InstCombiner &IC,
                                             Instruction *CxtI) {
   // We can always evaluate constants in another type.
   if (isa<Constant>(V))
     return true;
 
   Instruction *I = dyn_cast<Instruction>(V);
   if (!I) return false;
 
+  if (VisitedInsts.count(I))
+    return true;
+
+  VisitedInsts.insert(I);
+
   Type *OrigTy = V->getType();
 
   // If this is an extension from the dest type, we can eliminate it, even if it
   // has multiple uses.
   if ((isa<ZExtInst>(I) || isa<SExtInst>(I)) &&
       I->getOperand(0)->getType() == Ty)
     return true;
 
-  // We can't extend or shrink something that has multiple uses: doing so would
-  // require duplicating the instruction in general, which isn't profitable.
-  if (!I->hasOneUse()) return false;
+  // We don't want to duplicate instructiosn, which isn't profitable. Thus, we
+  // can't extend or shrink something that has multiple uses, unless all users
+  // are dominated by the trunc instruction and will be visited during the
+  // expresion evaluation. Mark all users that were not visited yet to be
+  // checked later against visited list.
+  if (!I->hasOneUse()) {
+    for (auto *U : I->users()) {
+      auto *UI = dyn_cast<Instruction>(U);
+      if (UI && !VisitedInsts.count(UI))
+        ToVisitInsts.insert(UI);
+    }
+  }
 
   unsigned Opc = I->getOpcode();
   switch (Opc) {
   case Instruction::Add:
   case Instruction::Sub:
   case Instruction::Mul:
   case Instruction::And:
   case Instruction::Or:
   case Instruction::Xor:
     // These operators can all arbitrarily be extended or truncated.
-    return canEvaluateTruncated(I->getOperand(0), Ty, IC, CxtI) &&
-           canEvaluateTruncated(I->getOperand(1), Ty, IC, CxtI);
+    return canEvaluateTruncatedImpl(I->getOperand(0), Ty, IC, CxtI) &&
+           canEvaluateTruncatedImpl(I->getOperand(1), Ty, IC, CxtI);
 
   case Instruction::UDiv:
   case Instruction::URem: {
     // UDiv and URem can be truncated if all the truncated bits are zero.
     uint32_t OrigBitWidth = OrigTy->getScalarSizeInBits();
     uint32_t BitWidth = Ty->getScalarSizeInBits();
     if (BitWidth < OrigBitWidth) {
       APInt Mask = APInt::getHighBitsSet(OrigBitWidth, OrigBitWidth-BitWidth);
       if (IC.MaskedValueIsZero(I->getOperand(0), Mask, 0, CxtI) &&
           IC.MaskedValueIsZero(I->getOperand(1), Mask, 0, CxtI)) {
-        return canEvaluateTruncated(I->getOperand(0), Ty, IC, CxtI) &&
-               canEvaluateTruncated(I->getOperand(1), Ty, IC, CxtI);
+        return canEvaluateTruncatedImpl(I->getOperand(0), Ty, IC, CxtI) &&
+               canEvaluateTruncatedImpl(I->getOperand(1), Ty, IC, CxtI);
       }
     }
     break;
   }
   case Instruction::Shl: {
     // If we are truncating the result of this SHL, and if it's a shift of a
     // constant amount, we can always perform a SHL in a smaller type.
     const APInt *Amt;
     if (match(I->getOperand(1), m_APInt(Amt))) {
       uint32_t BitWidth = Ty->getScalarSizeInBits();
       if (Amt->getLimitedValue(BitWidth) < BitWidth)
-        return canEvaluateTruncated(I->getOperand(0), Ty, IC, CxtI);
+        return canEvaluateTruncatedImpl(I->getOperand(0), Ty, IC, CxtI);
     }
     break;
   }
   case Instruction::LShr: {
     // If this is a truncate of a logical shr, we can truncate it to a smaller
     // lshr iff we know that the bits we would otherwise be shifting in are
     // already zeros.
     const APInt *Amt;
     if (match(I->getOperand(1), m_APInt(Amt))) {
       uint32_t OrigBitWidth = OrigTy->getScalarSizeInBits();
       uint32_t BitWidth = Ty->getScalarSizeInBits();
       if (IC.MaskedValueIsZero(I->getOperand(0),
             APInt::getHighBitsSet(OrigBitWidth, OrigBitWidth-BitWidth), 0, CxtI) &&
           Amt->getLimitedValue(BitWidth) < BitWidth) {
-        return canEvaluateTruncated(I->getOperand(0), Ty, IC, CxtI);
+        return canEvaluateTruncatedImpl(I->getOperand(0), Ty, IC, CxtI);
       }
     }
     break;
   }
   case Instruction::AShr: {
     // If this is a truncate of an arithmetic shr, we can truncate it to a
     // smaller ashr iff we know that all the bits from the sign bit of the
     // original type and the sign bit of the truncate type are similar.
     // TODO: It is enough to check that the bits we would be shifting in are
     //       similar to sign bit of the truncate type.
     const APInt *Amt;
     if (match(I->getOperand(1), m_APInt(Amt))) {
       uint32_t OrigBitWidth = OrigTy->getScalarSizeInBits();
       uint32_t BitWidth = Ty->getScalarSizeInBits();
       if (Amt->getLimitedValue(BitWidth) < BitWidth &&
           OrigBitWidth - BitWidth <
               IC.ComputeNumSignBits(I->getOperand(0), 0, CxtI))
-        return canEvaluateTruncated(I->getOperand(0), Ty, IC, CxtI);
+        return canEvaluateTruncatedImpl(I->getOperand(0), Ty, IC, CxtI);
     }
     break;
   }
   case Instruction::Trunc:
     // trunc(trunc(x)) -> trunc(x)
     return true;
   case Instruction::ZExt:
   case Instruction::SExt:
     // trunc(ext(x)) -> ext(x) if the source type is smaller than the new dest
     // trunc(ext(x)) -> trunc(x) if the source type is larger than the new dest
     return true;
   case Instruction::Select: {
     SelectInst *SI = cast<SelectInst>(I);
-    return canEvaluateTruncated(SI->getTrueValue(), Ty, IC, CxtI) &&
-           canEvaluateTruncated(SI->getFalseValue(), Ty, IC, CxtI);
+    return canEvaluateTruncatedImpl(SI->getTrueValue(), Ty, IC, CxtI) &&
+           canEvaluateTruncatedImpl(SI->getFalseValue(), Ty, IC, CxtI);
   }
   case Instruction::PHI: {
     // We can change a phi if we can change all operands.  Note that we never
     // get into trouble with cyclic PHIs here because we only consider
     // instructions with a single use.
     PHINode *PN = cast<PHINode>(I);
     for (Value *IncValue : PN->incoming_values())
-      if (!canEvaluateTruncated(IncValue, Ty, IC, CxtI))
+      if (!canEvaluateTruncatedImpl(IncValue, Ty, IC, CxtI))
         return false;
     return true;
   }
   default:
     // TODO: Can handle more cases here.
     break;
   }
 
Instruction *InstCombiner::visitTrunc(TruncInst &CI) {
   Value *Src = CI.getOperand(0);
   Type *DestTy = CI.getType(), *SrcTy = Src->getType();
 
   // Attempt to truncate the entire input expression tree to the destination
   // type.   Only do this if the dest type is a simple type, don't convert the
   // expression tree to something weird like i93 unless the source is also
   // strange.
   if ((DestTy->isVectorTy() || shouldChangeType(SrcTy, DestTy)) &&
       canEvaluateTruncated(Src, DestTy, *this, &CI)) {
 
     // If this cast is a truncate, evaluting in a different type always
     // eliminates the cast, so it is always a win.
     DEBUG(dbgs() << "ICE: EvaluateInDifferentType converting expression type"
           " to avoid cast: " << CI << '\n');
     Value *Res = EvaluateInDifferentType(Src, DestTy, false);
     assert(Res->getType() == DestTy);
     return replaceInstUsesWith(CI, Res);

lib/Transforms/InstCombine/InstCombineInternal.h

private:
   const SimplifyQuery SQ;
   OptimizationRemarkEmitter &ORE;
   // Optional analyses. When non-null, these can both be used to do better
   // combining and will be updated to reflect any changes.
   LoopInfo *LI;
 
   bool MadeIRChange;
 
+  // Map evaluated instruction to its new instruction.
+  DenseMap<Instruction*, Instruction*> EvaluatedInstMap;
+  // Set of instructions that have been visited during expression evaluation.
+  SmallPtrSet<Instruction*, 4> VisitedInsts;
+  // Set of instruction expected to be visited during expression evaluation.
+  SmallPtrSet<Instruction*, 4> ToVisitInsts;
+
 public:
   InstCombiner(InstCombineWorklist &Worklist, BuilderTy &Builder,
                bool MinimizeSize, bool ExpensiveCombines, AliasAnalysis *AA,
                AssumptionCache &AC, TargetLibraryInfo &TLI, DominatorTree &DT,
                OptimizationRemarkEmitter &ORE, const DataLayout &DL,
                LoopInfo *LI)
       : Worklist(Worklist), Builder(Builder), MinimizeSize(MinimizeSize),
         ExpensiveCombines(ExpensiveCombines), AA(AA), AC(AC), TLI(TLI), DT(DT),
private:
   bool SimplifyStoreAtEndOfBlock(StoreInst &SI);
 
   Instruction *
   SimplifyElementUnorderedAtomicMemCpy(ElementUnorderedAtomicMemCpyInst *AMI);
   Instruction *SimplifyMemTransfer(MemIntrinsic *MI);
   Instruction *SimplifyMemSet(MemSetInst *MI);
 
   Value *EvaluateInDifferentType(Value *V, Type *Ty, bool isSigned);
+  Value *EvaluateInDifferentTypeImpl(Value *V, Type *Ty, bool isSigned);
+  bool canEvaluateTruncated(Value *V, Type *Ty, InstCombiner &IC,
+                            Instruction *CxtI);
+  bool canEvaluateTruncatedImpl(Value *V, Type *Ty, InstCombiner &IC,
+                                Instruction *CxtI);
 
   /// \brief Returns a value X such that Val = X * Scale, or null if none.
   ///
   /// If the multiplication is known not to overflow then NoSignedWrap is set.
   Value *Descale(Value *Val, APInt Scale, bool &NoSignedWrap);
 };
 
 } // end namespace llvm.
 
 #undef DEBUG_TYPE
 
 #endif

test/Transforms/InstCombine/cast.ll

 ; CHECK-NEXT:    ret i16 [[TMP1]]
 ;
   %a = sext i16 %v to i32
   %s = ashr i32 %a, 18
   %t = trunc i32 %s to i16
   ret i16 %t
 }
 
+; Cases with truncate expression tree where not all nodes have a single usage.
+; However, all the nodes are dominated by the trunc instruction.
+; Case 1: same basic block
+define i16 @test_trunc_multi_use_case1(i16 %v) {
+; CHECK-LABEL: @test_trunc_multi_use_case1(
+; CHECK-NEXT:    [[TMP1:%.*]] = ashr i16 [[V:%.*]], 1
+; CHECK-NEXT:    [[S:%.*]] = mul i16 [[TMP1]], [[TMP1]]
+; CHECK-NEXT:    ret i16 [[S]]
+;
+  %a = sext i16 %v to i32
+  %b = ashr i32 %a, 1
+  %s = mul i32 %b, %b
+  %t = trunc i32 %s to i16
+  ret i16 %t
+}
+
+; Case 2: cross basic block
+define i16 @test_trunc_multi_use_case2(i16 %v, i1 %cond) {
+; CHECK-LABEL: @test_trunc_multi_use_case2(
+; CHECK-NEXT:  BB0:
+; CHECK-NEXT:    [[B:%.*]] = shl i16 [[V:%.*]], 1
+; CHECK-NEXT:    br i1 [[COND:%.*]], label [[BB1:%.*]], label [[BB2:%.*]]
+; CHECK:       BB1:
+; CHECK-NEXT:    [[D:%.*]] = mul i16 [[B]], [[B]]
+; CHECK-NEXT:    br label [[BB2]]
+; CHECK:       BB2:
+; CHECK-NEXT:    [[S:%.*]] = phi i16 [ [[B]], [[BB0:%.*]] ], [ [[D]], [[BB1]] ]
+; CHECK-NEXT:    ret i16 [[S]]
+;
+BB0:
+  %a = sext i16 %v to i32
+  %b = shl i32 %a, 1
+  br i1 %cond, label %BB1, label %BB2
+BB1:
+  %d = mul i32 %b, %b
+  br label %BB2
+BB2:
+  %s = phi i32 [ %b, %BB0 ], [ %d, %BB1 ]
+  %t = trunc i32 %s to i16
+  ret i16 %t
+}
+
+; Case 3: cross basic block - not all nodes are dominated by trunc instruction.
+define i16 @test_trunc_multi_use_case3(i16 %v) {
+; CHECK-LABEL: @test_trunc_multi_use_case3(
+; CHECK-NEXT:  BB0:
+; CHECK-NEXT:    [[A:%.*]] = sext i16 [[V:%.*]] to i32
+; CHECK-NEXT:    [[B:%.*]] = shl nsw i32 [[A]], 1
+; CHECK-NEXT:    [[COND:%.*]] = icmp ult i32 [[B]], 777
+; CHECK-NEXT:    br i1 [[COND]], label [[BB1:%.*]], label [[BB2:%.*]]
+; CHECK:       BB1:
+; CHECK-NEXT:    [[D:%.*]] = mul nsw i32 [[A]], 1554
+; CHECK-NEXT:    br label [[BB2]]
+; CHECK:       BB2:
+; CHECK-NEXT:    [[S:%.*]] = phi i32 [ [[B]], [[BB0:%.*]] ], [ [[D]], [[BB1]] ]
+; CHECK-NEXT:    [[T:%.*]] = trunc i32 [[S]] to i16
+; CHECK-NEXT:    ret i16 [[T]]
+;
+BB0:
+  %a = sext i16 %v to i32
+  %b = shl i32 %a, 1
+  %cond = icmp ult i32 %b, 777
+  br i1 %cond, label %BB1, label %BB2
+BB1:
+  %d = mul i32 %b, 777
+  br label %BB2
+BB2:
+  %s = phi i32 [ %b, %BB0 ], [ %d, %BB1 ]
+  %t = trunc i32 %s to i16
+  ret i16 %t
+}
+
 ; Overflow on a float to int or int to float conversion is undefined (PR21130).
 
 define i8 @overflow_fptosi() {
 ; CHECK-LABEL: @overflow_fptosi(
 ; CHECK-NEXT:    ret i8 undef
 ;
   %i = fptosi double 1.56e+02 to i8
   ret i8 %i

test/Transforms/InstCombine/pr33765.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -S %s -instcombine | FileCheck %s
 
 @glob = external global i16
 
 define void @patatino(i8 %beth) {
 ; CHECK-LABEL: @patatino(
-; CHECK-NEXT:    [[CONV:%.*]] = zext i8 [[BETH:%.*]] to i32
+; CHECK-NEXT:    [[CONV:%.*]] = zext i8 [[BETH:%.*]] to i16
 ; CHECK-NEXT:    br i1 false, label [[IF_THEN9:%.*]], label [[IF_THEN9]]
 ; CHECK:       if.then9:
-; CHECK-NEXT:    [[MUL:%.*]] = mul nuw nsw i32 [[CONV]], [[CONV]]
+; CHECK-NEXT:    [[MUL:%.*]] = mul nuw i16 [[CONV]], [[CONV]]
 ; CHECK-NEXT:    [[TINKY:%.*]] = load i16, i16* @glob, align 2
-; CHECK-NEXT:    [[CONV131:%.*]] = zext i16 [[TINKY]] to i32
-; CHECK-NEXT:    [[AND:%.*]] = and i32 [[MUL]], [[CONV131]]
-; CHECK-NEXT:    [[CONV14:%.*]] = trunc i32 [[AND]] to i16
-; CHECK-NEXT:    store i16 [[CONV14]], i16* @glob, align 2
+; CHECK-NEXT:    [[AND:%.*]] = and i16 [[MUL]], [[TINKY]]
+; CHECK-NEXT:    store i16 [[AND]], i16* @glob, align 2
 ; CHECK-NEXT:    ret void
 ;
   %conv = zext i8 %beth to i32
   %mul = mul nuw nsw i32 %conv, %conv
   %conv3 = and i32 %mul, 255
   %tobool8 = icmp ne i32 %mul, %conv3
   br i1 %tobool8, label %if.then9, label %if.then9
 
 if.then9:
   %tinky = load i16, i16* @glob
   %conv13 = sext i16 %tinky to i32
   %and = and i32 %mul, %conv13
   %conv14 = trunc i32 %and to i16
   store i16 %conv14, i16* @glob
   ret void
 }