[InstCombine] PR37603: low bit mask canonicalization

[lebedev.ri]

Summary

This is PR37603.

https://godbolt.org/g/VCMNpS
https://rise4fun.com/Alive/idM

When doing bit manipulations, it is quite common to calculate some bit mask,
and apply it to some value via and.

The typical C code looks like:

int mask_signed_add(int nbits) {
    return (1 << nbits) - 1;
}

which is translated into (with -O3)

define dso_local i32 @mask_signed_add(int)(i32) local_unnamed_addr #0 {
  %2 = shl i32 1, %0
  %3 = add nsw i32 %2, -1
  ret i32 %3
}

But there is a second, less readable variant:

int mask_signed_xor(int nbits) {
    return ~(-(1 << nbits));
}

which is translated into (with -O3)

define dso_local i32 @mask_signed_xor(int)(i32) local_unnamed_addr #0 {
  %2 = shl i32 -1, %0
  %3 = xor i32 %2, -1
  ret i32 %3
}

Since we created such a mask, it is quite likely that we will use it in and next.
And then we may get rid of not op by folding into andn.

But now that i have actually looked:
https://godbolt.org/g/VTUDmU
_some_ backend changes will be needed too.
We clearly loose bzhi recognition.

[lebedev.ri]

Actually run $ ninja check-llvm -- update test/Transforms/InstCombine/rem.ll

[lebedev.ri]

test/Transforms/InstCombine/set-lowbits-mask-canonicalize.ll

20–21 ↗

(On Diff #148763)

Note: i'm not *sure* we want to go this way.
Maybe we want to canonicalize the other way around?

(Another unrelated question could be,
do we want to "canonicalize" all add nuw i32 %val, -1 to xor i32 %val, -1?
I'm not sure here, because we can't go from not to dec as per alive,
and the latter does not produce lea)

[spatel]

This is a good IR canonicalization regardless of what happens in the backend because a 'not' is better for bit-tracking analysis and other transforms than an 'add'.

I don't understand the nuw question:
https://rise4fun.com/Alive/ydD

We're missing this in instsimplify?

[lebedev.ri]

In D47428#1117675, @spatel wrote:

This is a good IR canonicalization regardless of what happens in the backend because a 'not' is better for bit-tracking analysis and other transforms than an 'add'.

Yep, i thought so too..

I don't understand the nuw question:
https://rise4fun.com/Alive/ydD

We're missing this in instsimplify?

I think we do? https://godbolt.org/g/rT3k7v
But i was only thinking about replacing and nuw with xor there.

[spatel]

In D47428#1117688, @lebedev.ri wrote:

In D47428#1117675, @spatel wrote:

This is a good IR canonicalization regardless of what happens in the backend because a 'not' is better for bit-tracking analysis and other transforms than an 'add'.

Yep, i thought so too..

I don't understand the nuw question:
https://rise4fun.com/Alive/ydD

We're missing this in instsimplify?

I think we do? https://godbolt.org/g/rT3k7v
But i was only thinking about replacing and nuw with xor there.

I'm still not clear on the nuw question:
https://godbolt.org/g/mTHjhc

Are you trying to transform with a constant other than -1? If -1, then we should have simplified the IR to a constant.

[lebedev.ri]

Ping.

[spatel]

We didn't resolve the 'nuw' question - am I not seeing the scenario that you asked about?

lib/Transforms/InstCombine/InstCombineAddSub.cpp

1102 ↗

(On Diff #148763)

The comment should match what the code is doing:
~(-1 << NBits)

1103 ↗

(On Diff #148763)

The IR motivation is what I'd prefer to state here, so something like:
Because a 'not' is better for bit-tracking analysis and other transforms than an 'add'.

1112 ↗

(On Diff #148763)

Is this always safe to cast rather than dyn_cast? What happens if NBits is a constant expression?

[lebedev.ri]

Address review notes.

In D47428#1120820, @spatel wrote:

We didn't resolve the 'nuw' question - am I not seeing the scenario that you asked about?

True.
It was just a passing-by thought.
Here i don't particularly care about that fold.
I was just thinking about profitability of transform to xor %x, -1,
and thought whether we could always do that if the input is add %x, -1.

I can submit that fold as a follow-up.

[lebedev.ri]

This comment has been deleted.

lib/Transforms/InstCombine/InstCombineAddSub.cpp

1112 ↗

(On Diff #148763)

New test with constant seems to work.
So in this case, the constant folding seems to already happen,
but i agree i guess it is better to be proactively safer here.

[spatel]

In D47428#1120867, @lebedev.ri wrote:

I was just thinking about profitability of transform to xor %x, -1,
and thought whether we could always do that if the input is add %x, -1.

I can submit that fold as a follow-up.

Sounds good. This patch LGTM.

[lebedev.ri]

In D47428#1120886, @spatel wrote:

In D47428#1120867, @lebedev.ri wrote:

I was just thinking about profitability of transform to xor %x, -1,
and thought whether we could always do that if the input is add %x, -1.

I can submit that fold as a follow-up.

Sounds good. This patch LGTM.

Thank you for the review!
To not degrade x86 backend, this can't land until after D47453.

[spatel]

test/Transforms/InstCombine/set-lowbits-mask-canonicalize.ll

204–211 ↗

(On Diff #149752)

I didn't see this test initially - it's ok to add it here, but it's better to include under test/Transforms/InstSimplify instead (if there's nothing like it already there).

But this doesn't exercise the case that I was thinking of. We should always constant fold simple constants like this. It's *constant expressions* that we (or at least me...multiple times!) forget and then result in fuzzer failures some time later. So the test will need a global variable or some other magic, but now that the code is dyn_cast'ing, it's probably a moot point.

llvm/trunk/lib/Transforms/InstCombine/InstCombineAddSub.cpp

if (MatchRem(MulOpV, RemOpV, C1, Rem2IsSigned) &&
                         : Builder.CreateURem(X, NewDivisor, "urem");
       }
     }
   }
 
   return nullptr;
 }
 
+/// Fold
+///   (1 << NBits) - 1
+/// Into:
+///   ~(-(1 << NBits))
+/// Because a 'not' is better for bit-tracking analysis and other transforms
+/// than an 'add'. The new shl is always nsw, and is nuw if old `and` was.
+static Instruction *canonicalizeLowbitMask(BinaryOperator &I,
+                                           InstCombiner::BuilderTy &Builder) {
+  Value *NBits;
+  if (!match(&I, m_Add(m_OneUse(m_Shl(m_One(), m_Value(NBits))), m_AllOnes())))
+    return nullptr;
+
+  Constant *MinusOne = Constant::getAllOnesValue(NBits->getType());
+  Value *NotMask = Builder.CreateShl(MinusOne, NBits, "notmask");
+  // Be wary of constant folding.
+  if (auto *BOp = dyn_cast<BinaryOperator>(NotMask)) {
+    // Always NSW. But NUW propagates from `add`.
+    BOp->setHasNoSignedWrap();
+    BOp->setHasNoUnsignedWrap(I.hasNoUnsignedWrap());
+  }
+
+  return BinaryOperator::CreateNot(NotMask, I.getName());
+}
+
 Instruction *InstCombiner::visitAdd(BinaryOperator &I) {
   bool Changed = SimplifyAssociativeOrCommutative(I);
   Value *LHS = I.getOperand(0), *RHS = I.getOperand(1);
   if (Value *V =
           SimplifyAddInst(LHS, RHS, I.hasNoSignedWrap(), I.hasNoUnsignedWrap(),
                           SQ.getWithInstruction(&I)))
     return replaceInstUsesWith(I, V);
 
if (!I.hasNoSignedWrap() && willNotOverflowSignedAdd(LHS, RHS, I)) {
     Changed = true;
     I.setHasNoSignedWrap(true);
   }
   if (!I.hasNoUnsignedWrap() && willNotOverflowUnsignedAdd(LHS, RHS, I)) {
     Changed = true;
     I.setHasNoUnsignedWrap(true);
   }
 
+  if (Instruction *V = canonicalizeLowbitMask(I, Builder))
+    return V;
+
   return Changed ? &I : nullptr;
 }
 
 Instruction *InstCombiner::visitFAdd(BinaryOperator &I) {
   bool Changed = SimplifyAssociativeOrCommutative(I);
   Value *LHS = I.getOperand(0), *RHS = I.getOperand(1);
   if (Value *V = SimplifyFAddInst(LHS, RHS, I.getFastMathFlags(),
                                   SQ.getWithInstruction(&I)))

llvm/trunk/test/Transforms/InstCombine/rem.ll

;
   %shl = shl i32 1, %y
   %zext = zext i32 %shl to i64
   %urem = urem i64 %x, %zext
   ret i64 %urem
 }
 
 define i64 @test15(i32 %x, i32 %y) {
 ; CHECK-LABEL: @test15(
-; CHECK-NEXT:    [[SHL:%.*]] = shl nuw i32 1, [[Y:%.*]]
-; CHECK-NEXT:    [[TMP1:%.*]] = add i32 [[SHL]], -1
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nsw i32 -1, [[Y:%.*]]
+; CHECK-NEXT:    [[TMP1:%.*]] = xor i32 [[NOTMASK]], -1
 ; CHECK-NEXT:    [[TMP2:%.*]] = and i32 [[TMP1]], [[X:%.*]]
 ; CHECK-NEXT:    [[UREM:%.*]] = zext i32 [[TMP2]] to i64
 ; CHECK-NEXT:    ret i64 [[UREM]]
 ;
   %shl = shl i32 1, %y
   %zext0 = zext i32 %shl to i64
   %zext1 = zext i32 %x to i64
   %urem = urem i64 %zext1, %zext0

llvm/trunk/test/Transforms/InstCombine/set-lowbits-mask-canonicalize.ll

 ; ============================================================================ ;
 ; Most basic positive tests
 ; ============================================================================ ;
 
 ; No no-wrap tags on shl
 
 define i32 @shl_add(i32 %NBits) {
 ; CHECK-LABEL: @shl_add(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl i32 1, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add i32 [[SETBIT]], -1
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nsw i32 -1, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor i32 [[NOTMASK]], -1
 ; CHECK-NEXT:    ret i32 [[RET]]
 ;
   %setbit = shl i32 1, %NBits
   %ret = add i32 %setbit, -1
   ret i32 %ret
 }
 
 define i32 @shl_add_nsw(i32 %NBits) {
 ; CHECK-LABEL: @shl_add_nsw(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl i32 1, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add nsw i32 [[SETBIT]], -1
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nsw i32 -1, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor i32 [[NOTMASK]], -1
 ; CHECK-NEXT:    ret i32 [[RET]]
 ;
   %setbit = shl i32 1, %NBits
   %ret = add nsw i32 %setbit, -1
   ret i32 %ret
 }
 
 define i32 @shl_add_nuw(i32 %NBits) {
 ; CHECK-LABEL: @shl_add_nuw(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl i32 1, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add nuw i32 [[SETBIT]], -1
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nuw nsw i32 -1, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor i32 [[NOTMASK]], -1
 ; CHECK-NEXT:    ret i32 [[RET]]
 ;
   %setbit = shl i32 1, %NBits
   %ret = add nuw i32 %setbit, -1
   ret i32 %ret
 }
 
 define i32 @shl_add_nsw_nuw(i32 %NBits) {
 ; CHECK-LABEL: @shl_add_nsw_nuw(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl i32 1, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add nuw nsw i32 [[SETBIT]], -1
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nuw nsw i32 -1, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor i32 [[NOTMASK]], -1
 ; CHECK-NEXT:    ret i32 [[RET]]
 ;
   %setbit = shl i32 1, %NBits
   %ret = add nuw nsw i32 %setbit, -1
   ret i32 %ret
 }
 
 ; shl is nsw
 
 define i32 @shl_nsw_add(i32 %NBits) {
 ; CHECK-LABEL: @shl_nsw_add(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl nsw i32 1, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add i32 [[SETBIT]], -1
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nsw i32 -1, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor i32 [[NOTMASK]], -1
 ; CHECK-NEXT:    ret i32 [[RET]]
 ;
   %setbit = shl nsw i32 1, %NBits
   %ret = add i32 %setbit, -1
   ret i32 %ret
 }
 
 define i32 @shl_nsw_add_nsw(i32 %NBits) {
 ; CHECK-LABEL: @shl_nsw_add_nsw(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl nsw i32 1, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add nsw i32 [[SETBIT]], -1
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nsw i32 -1, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor i32 [[NOTMASK]], -1
 ; CHECK-NEXT:    ret i32 [[RET]]
 ;
   %setbit = shl nsw i32 1, %NBits
   %ret = add nsw i32 %setbit, -1
   ret i32 %ret
 }
 
 define i32 @shl_nsw_add_nuw(i32 %NBits) {
 ; CHECK-LABEL: @shl_nsw_add_nuw(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl nsw i32 1, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add nuw i32 [[SETBIT]], -1
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nuw nsw i32 -1, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor i32 [[NOTMASK]], -1
 ; CHECK-NEXT:    ret i32 [[RET]]
 ;
   %setbit = shl nsw i32 1, %NBits
   %ret = add nuw i32 %setbit, -1
   ret i32 %ret
 }
 
 define i32 @shl_nsw_add_nsw_nuw(i32 %NBits) {
 ; CHECK-LABEL: @shl_nsw_add_nsw_nuw(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl nsw i32 1, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add nuw nsw i32 [[SETBIT]], -1
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nuw nsw i32 -1, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor i32 [[NOTMASK]], -1
 ; CHECK-NEXT:    ret i32 [[RET]]
 ;
   %setbit = shl nsw i32 1, %NBits
   %ret = add nuw nsw i32 %setbit, -1
   ret i32 %ret
 }
 
 ; shl is nuw
 
 define i32 @shl_nuw_add(i32 %NBits) {
 ; CHECK-LABEL: @shl_nuw_add(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl nuw i32 1, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add i32 [[SETBIT]], -1
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nsw i32 -1, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor i32 [[NOTMASK]], -1
 ; CHECK-NEXT:    ret i32 [[RET]]
 ;
   %setbit = shl nuw i32 1, %NBits
   %ret = add i32 %setbit, -1
   ret i32 %ret
 }
 
 define i32 @shl_nuw_add_nsw(i32 %NBits) {
 ; CHECK-LABEL: @shl_nuw_add_nsw(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl nuw i32 1, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add nsw i32 [[SETBIT]], -1
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nsw i32 -1, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor i32 [[NOTMASK]], -1
 ; CHECK-NEXT:    ret i32 [[RET]]
 ;
   %setbit = shl nuw i32 1, %NBits
   %ret = add nsw i32 %setbit, -1
   ret i32 %ret
 }
 
 define i32 @shl_nuw_add_nuw(i32 %NBits) {
 ; CHECK-LABEL: @shl_nuw_add_nuw(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl nuw i32 1, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add nuw i32 [[SETBIT]], -1
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nuw nsw i32 -1, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor i32 [[NOTMASK]], -1
 ; CHECK-NEXT:    ret i32 [[RET]]
 ;
   %setbit = shl nuw i32 1, %NBits
   %ret = add nuw i32 %setbit, -1
   ret i32 %ret
 }
 
 define i32 @shl_nuw_add_nsw_nuw(i32 %NBits) {
 ; CHECK-LABEL: @shl_nuw_add_nsw_nuw(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl nuw i32 1, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add nuw nsw i32 [[SETBIT]], -1
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nuw nsw i32 -1, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor i32 [[NOTMASK]], -1
 ; CHECK-NEXT:    ret i32 [[RET]]
 ;
   %setbit = shl nuw i32 1, %NBits
   %ret = add nuw nsw i32 %setbit, -1
   ret i32 %ret
 }
 
 ; shl is nuw nsw
 
 define i32 @shl_nsw_nuw_add(i32 %NBits) {
 ; CHECK-LABEL: @shl_nsw_nuw_add(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl nuw nsw i32 1, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add i32 [[SETBIT]], -1
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nsw i32 -1, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor i32 [[NOTMASK]], -1
 ; CHECK-NEXT:    ret i32 [[RET]]
 ;
   %setbit = shl nuw nsw i32 1, %NBits
   %ret = add i32 %setbit, -1
   ret i32 %ret
 }
 
 define i32 @shl_nsw_nuw_add_nsw(i32 %NBits) {
 ; CHECK-LABEL: @shl_nsw_nuw_add_nsw(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl nuw nsw i32 1, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add nsw i32 [[SETBIT]], -1
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nsw i32 -1, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor i32 [[NOTMASK]], -1
 ; CHECK-NEXT:    ret i32 [[RET]]
 ;
   %setbit = shl nuw nsw i32 1, %NBits
   %ret = add nsw i32 %setbit, -1
   ret i32 %ret
 }
 
 define i32 @shl_nsw_nuw_add_nuw(i32 %NBits) {
 ; CHECK-LABEL: @shl_nsw_nuw_add_nuw(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl nuw nsw i32 1, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add nuw i32 [[SETBIT]], -1
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nuw nsw i32 -1, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor i32 [[NOTMASK]], -1
 ; CHECK-NEXT:    ret i32 [[RET]]
 ;
   %setbit = shl nuw nsw i32 1, %NBits
   %ret = add nuw i32 %setbit, -1
   ret i32 %ret
 }
 
 define i32 @shl_nsw_nuw_add_nsw_nuw(i32 %NBits) {
 ; CHECK-LABEL: @shl_nsw_nuw_add_nsw_nuw(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl nuw nsw i32 1, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add nuw nsw i32 [[SETBIT]], -1
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nuw nsw i32 -1, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor i32 [[NOTMASK]], -1
 ; CHECK-NEXT:    ret i32 [[RET]]
 ;
   %setbit = shl nuw nsw i32 1, %NBits
   %ret = add nuw nsw i32 %setbit, -1
   ret i32 %ret
 }
 
 ; ============================================================================ ;
 ; Vectors
 ; ============================================================================ ;
 
 define <2 x i32> @shl_add_vec(<2 x i32> %NBits) {
 ; CHECK-LABEL: @shl_add_vec(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl <2 x i32> <i32 1, i32 1>, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add <2 x i32> [[SETBIT]], <i32 -1, i32 -1>
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nsw <2 x i32> <i32 -1, i32 -1>, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor <2 x i32> [[NOTMASK]], <i32 -1, i32 -1>
 ; CHECK-NEXT:    ret <2 x i32> [[RET]]
 ;
   %setbit = shl <2 x i32> <i32 1, i32 1>, %NBits
   %ret = add <2 x i32> %setbit, <i32 -1, i32 -1>
   ret <2 x i32> %ret
 }
 
 define <3 x i32> @shl_add_vec_undef0(<3 x i32> %NBits) {
 ; CHECK-LABEL: @shl_add_vec_undef0(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl <3 x i32> <i32 1, i32 undef, i32 1>, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add <3 x i32> [[SETBIT]], <i32 -1, i32 -1, i32 -1>
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nsw <3 x i32> <i32 -1, i32 -1, i32 -1>, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor <3 x i32> [[NOTMASK]], <i32 -1, i32 -1, i32 -1>
 ; CHECK-NEXT:    ret <3 x i32> [[RET]]
 ;
   %setbit = shl <3 x i32> <i32 1, i32 undef, i32 1>, %NBits
   %ret = add <3 x i32> %setbit, <i32 -1, i32 -1, i32 -1>
   ret <3 x i32> %ret
 }
 
 define <3 x i32> @shl_add_vec_undef1(<3 x i32> %NBits) {
 ; CHECK-LABEL: @shl_add_vec_undef1(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl <3 x i32> <i32 1, i32 1, i32 1>, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add <3 x i32> [[SETBIT]], <i32 -1, i32 undef, i32 -1>
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nsw <3 x i32> <i32 -1, i32 -1, i32 -1>, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor <3 x i32> [[NOTMASK]], <i32 -1, i32 -1, i32 -1>
 ; CHECK-NEXT:    ret <3 x i32> [[RET]]
 ;
   %setbit = shl <3 x i32> <i32 1, i32 1, i32 1>, %NBits
   %ret = add <3 x i32> %setbit, <i32 -1, i32 undef, i32 -1>
   ret <3 x i32> %ret
 }
 
 define <3 x i32> @shl_add_vec_undef2(<3 x i32> %NBits) {
 ; CHECK-LABEL: @shl_add_vec_undef2(
-; CHECK-NEXT:    [[SETBIT:%.*]] = shl <3 x i32> <i32 1, i32 undef, i32 1>, [[NBITS:%.*]]
-; CHECK-NEXT:    [[RET:%.*]] = add <3 x i32> [[SETBIT]], <i32 -1, i32 undef, i32 -1>
+; CHECK-NEXT:    [[NOTMASK:%.*]] = shl nsw <3 x i32> <i32 -1, i32 -1, i32 -1>, [[NBITS:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = xor <3 x i32> [[NOTMASK]], <i32 -1, i32 -1, i32 -1>
 ; CHECK-NEXT:    ret <3 x i32> [[RET]]
 ;
   %setbit = shl <3 x i32> <i32 1, i32 undef, i32 1>, %NBits
   %ret = add <3 x i32> %setbit, <i32 -1, i32 undef, i32 -1>
   ret <3 x i32> %ret
 }
 
 ; ============================================================================ ;