[X86] Improve instruction ordering of constant `srl/shl` with `and` to get better and-masks

[goldstein.w.n]

Summary

This moves some logic from combineShiftLeft and generalizes is to
work for shl and and. It also improves the non-mask constant
generation (reducing based on getSignificantBits instead of
countTrailingOnes) so that we can catch some cases where imm64 ->
sign-extended imm32/imm8 or imm32 -> sign-extended imm8

[pengfei]

llvm/test/CodeGen/X86/bitreverse.ll

538

IIRC, LEA is expensive, so this looks like a good deal.

llvm/test/CodeGen/X86/bmi-x86_64.ll

131–132

Remove BEXTR-SLOW to eliminate the message.

llvm/test/CodeGen/X86/btc_bts_btr.ll

988 ↗

(On Diff #488861)

One more andb

1011 ↗

(On Diff #488861)

ditto.

llvm/test/CodeGen/X86/combine-bitreverse.ll

241

One more shrl?

300

ditto.

325

ditto.

llvm/test/CodeGen/X86/const-shift-of-constmasked.ll

657

You can add option --no_x86_scrub_sp when updating test.

1194

ditto.

llvm/test/CodeGen/X86/const-shift-with-and.ll

5 ↗

(On Diff #488861)

What's these tests used for? They are not changed in this patch?
Besides, do you still need new test case for this patch? Are these covered by the other changes already?

[goldstein.w.n]

llvm/test/CodeGen/X86/btc_bts_btr.ll

988 ↗

(On Diff #488861)

One more andb

Sorry, didn't see earlier. Think this breaks a few combine patterns that probably relied on semi-canonicalizarion of (and (shift x, y), z) instead of the other way around.

Fixing the cases caught here is doable, but is it possible to maybe move this to later in the process? Currently it's guarded behind AfterLegalize (for the exact reason of not breaking other patterns), but is there a higher level or different pass this might be better placed in?

[craig.topper]

llvm/test/CodeGen/X86/bitreverse.ll

538

I thought LEA was only expensive if it uses 3 sources.

[craig.topper]

llvm/lib/Target/X86/X86ISelLowering.cpp

47494

You could check !VT.isScalarInteger() instead.

47554

Drop else after return.

56345

Why checking both operands of the shift? Isn't only the first one interesting? An AND on the shift amount is very different.

[pengfei]

llvm/lib/Target/X86/X86ISelLowering.cpp

47679

Add one space after if.

49129

ditto.

llvm/test/CodeGen/X86/bitreverse.ll

538

You are right. SOM says only 3 operand LEA is slow. But I found MCA cannot reflect the difference.
So this is yet another regression.
old: https://godbolt.org/z/dPW93v8zT
new: https://godbolt.org/z/hrEovKW4f

[pengfei]

llvm/lib/Target/X86/X86ISelLowering.cpp

47438

results

47438

One more space. You may run clang-format to help solvoing the format nits.

47533

Use curly braces for consistency.

[craig.topper]

llvm/test/CodeGen/X86/bitreverse.ll

538

Looks like the scheduler models make the distinction for AMD CPUs but not Intel

Small snippet of the predicate check:

X86SchedPredicates.td
61:def IsThreeOperandsLEAFn :

X86ScheduleBdVer2.td
560:    IsThreeOperandsLEAFn,

X86ScheduleBtVer2.td
945:    IsThreeOperandsLEAFn,

X86ScheduleZnver3.td
590:    IsThreeOperandsLEAFn,

[pengfei]

llvm/test/CodeGen/X86/bitreverse.ll

538

Thanks for the information! We may need to do the same thing for Intel. Filed #60043 for it.

[goldstein.w.n]

Fix many a nit

[goldstein.w.n]

llvm/test/CodeGen/X86/btc_bts_btr.ll

988 ↗

(On Diff #488861)

Do you know if there is a later stage I can move this transform too so that it won't break any other patterns?

[pengfei]

llvm/test/CodeGen/X86/btc_bts_btr.ll

988 ↗

(On Diff #488861)

No, I don't. Maybe do it in a peephole pass after ISel?

[craig.topper]

llvm/test/CodeGen/X86/btc_bts_btr.ll

988 ↗

(On Diff #488861)

We have something similarish implemented during isel in X86DAGToDAGISel::tryShrinkShlLogicImm.

[goldstein.w.n]

llvm/test/CodeGen/X86/btc_bts_btr.ll

988 ↗

(On Diff #488861)

Do you know if there is a later stage I can move this transform too so that it won't break any other patterns?

988 ↗

(On Diff #488861)

We have something similarish implemented during isel in X86DAGToDAGISel::tryShrinkShlLogicImm.

I did in fact try moving it there and it did clean up some of the missed optimizations but added some (more severe) other missed optimizations.

988 ↗

(On Diff #488861)

No, I don't. Maybe do it in a peephole pass after ISel?

Is there an example file / pass you could point to for me to emulate?

[RKSimon]

llvm/lib/Target/X86/X86ISelLowering.cpp

47444

a lot of the if-else complexity is coming from you trying to handle and(shift(x,c1),c2) and shift(and(x,c2),c3) permutations in the same code - can you not have 2 variants in separate wrapper functions and then have a simpler core function that both call?

47454

(style) assert message

47480

(style) assert messages

[goldstein.w.n]

llvm/lib/Target/X86/X86ISelLowering.cpp

47444

a lot of the if-else complexity is coming from you trying to handle and(shift(x,c1),c2) and shift(and(x,c2),c3) permutations in the same code - can you not have 2 variants in separate wrapper functions and then have a simpler core function that both call?

It seemed to me there was enough duplication / not "too" complex to justify one function, but can ofc split.

[goldstein.w.n]

Fix regressions + more format issues

[goldstein.w.n]

@pengfei fixed the added and and shift in btc_bts_btr.ll and combine-bitreverse.ll respectively. The diffs are gone so your comments don't show up anymore.

[goldstein.w.n]

Fix andn + remove bextr + no-scrub-sp

[pengfei]

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

9353

So it is only profitable when MASK is a constant? Why don't use c3 in the comments?

9361

Duplicated define.

llvm/lib/Target/X86/X86ISelLowering.cpp

47459–47460

You could use ISD::isExtOpcode(Opc).

47479

I'd say the implementation is too complicated to me to understand. And the overall improvement doesn't look worth the complexity. Not to mention there's still regression in bitreverse.ll
I may not try to understand the code here, leave it for other reviewers.

[goldstein.w.n]

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

9353

So it is only profitable when MASK is a constant? Why don't use c3 in the comments?

llvm/lib/Target/X86/X86ISelLowering.cpp

47479

I'd say the implementation is too complicated to me to understand. And the overall improvement doesn't look worth the complexity. Not to mention there's still regression in bitreverse.ll

I think the bitreverse.ll regressions have been fixed (that was the point of the new patterns in DAGCombiner.cpp).

I may not try to understand the code here, leave it for other reviewers.

Okay, I will split into 2-functions as Simon suggested which I think will decrease the complexity.

[goldstein.w.n]

Split and/shift cases (readability). Fix some nits

[pengfei]

llvm/lib/Target/X86/X86ISelLowering.cpp

47479

I think the bitreverse.ll regressions have been fixed (that was the point of the new patterns in DAGCombiner.cpp).

I think you fixed combine-bitreverse.ll rather than bitreverse.ll
The problem in bitreverse.ll is 2 lea is replaced by 2 mov + and + shl. I think it is a minor regression due to the increased 2 mov, so I didn't insist on that.
Is the TODO in the file to address this problem?

[goldstein.w.n]

More cleanup

[goldstein.w.n]

llvm/lib/Target/X86/X86ISelLowering.cpp

47479

I'd say the implementation is too complicated to me to understand. And the overall improvement doesn't look worth the complexity. Not to mention there's still regression in bitreverse.ll
I may not try to understand the code here, leave it for other reviewers.

As simon suggested, split the two implementations. Think its a lot more readable now.

47479

I think the bitreverse.ll regressions have been fixed (that was the point of the new patterns in DAGCombiner.cpp).

I think you fixed combine-bitreverse.ll rather than bitreverse.ll
The problem in bitreverse.ll is 2 lea is replaced by 2 mov + and + shl. I think it is a minor regression due to the increased 2 mov, so I didn't insist on that.

I see. I think this is b.c of the reordering (and (shl X..)) -> (shl (and X..)) when X persists (then we need a mov).
We could prevent it with a guard in combineAndWithLogicalShift that only did the transform if this was the last use of X the shiftop was shl 1/2/3 (can become lea). Is there a way to query "isLastUse(X)" for an SDValue or do we just need to do hasOneUse?

Also note other places in the patch get the inverse behavior i.e llvm/test/CodeGen/X86/combine-rotates.ll:rotl_merge_i5.

Is the TODO in the file to address this problem?

Not directly, no. Maybe by accident.

[goldstein.w.n]

Fix regression in bitreverse

[goldstein.w.n]

llvm/lib/Target/X86/X86ISelLowering.cpp

47479

I think the bitreverse.ll regressions have been fixed (that was the point of the new patterns in DAGCombiner.cpp).

I think you fixed combine-bitreverse.ll rather than bitreverse.ll
The problem in bitreverse.ll is 2 lea is replaced by 2 mov + and + shl. I think it is a minor regression due to the increased 2 mov, so I didn't insist on that.
Is the TODO in the file to address this problem?

Fixed.

[pengfei]

The new code is a bit easier to understand now, at least to me. The idea masks sense to me though I still think the logic is a bit verbose.
I'm not going to block this patch, but please wait some days for other reviewers.

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

9440

Unintended change.

llvm/lib/Target/X86/X86ISelLowering.cpp

47663

I found we prefer to const APInt.

47674

countTrailingOnes?

47676

Should it just need to check MaskCnt == 8/16/32. Or even AndMask == 0xff/0xffff/0xffffffff?

47706

countTrailingOnes?

47735

Duplicated.

47735

it's

47847

is

56681

Add a comment to explain this is for combineLogicalShiftWithAnd.
Why combineAndWithLogicalShift not need this?

[goldstein.w.n]

llvm/lib/Target/X86/X86ISelLowering.cpp

56681

Add a comment to explain this is for combineLogicalShiftWithAnd.
Why combineAndWithLogicalShift not need this?

this is only called for shift ops (i.e VisitSHL and visitShiftByConstant).

[goldstein.w.n]

Fix nits + improve comments

[goldstein.w.n]

Update tests for all backends

[goldstein.w.n]

llvm/test/CodeGen/AArch64/arm64-bitfield-extract.ll

942 ↗

(On Diff #491035)

@t.p.northover the changes to the DagCombiner (allowing shl/shr to combine through an and) seems to cause regression on Aarch64. Going to update with TargetLowering flag unless this is in fact preferable.

[goldstein.w.n]

Put the DAGCombiner changes behind TLI flag as it seems to cause
minor regression on aarch64

[goldstein.w.n]

llvm/test/CodeGen/AArch64/arm64-bitfield-extract.ll

942 ↗

(On Diff #491035)

@t.p.northover the changes to the DagCombiner (allowing shl/shr to combine through an and) seems to cause regression on Aarch64. Going to update with TargetLowering flag unless this is in fact preferable.

I put it behind TLI.isDesirableToCombineShiftsAcross which is disabled by default so should be no issue.

[goldstein.w.n]

Made some changes (TLI flag) since this was accepted. Can someone do a quick pass of the most recent changes to re-verify?

[pengfei]

llvm/include/llvm/CodeGen/TargetLowering.h

4020–4022

Is Shift1 inner and Shift2 outer?

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

9440

This still no changed.

llvm/lib/Target/X86/X86ISelLowering.cpp

56685–56686

The arguments are not used. Leave it void for now?

[goldstein.w.n]

llvm/include/llvm/CodeGen/TargetLowering.h

4020–4022

Is Shift1 inner and Shift2 outer?

Yes, renamed to OuterShift and InnerShift to be clearer.

[goldstein.w.n]

Fix style, issue, rename TLI helper, void variables

[pengfei]

llvm/include/llvm/CodeGen/TargetLowering.h

4020

Is the comment still incorrect?

[goldstein.w.n]

Fix comment

[goldstein.w.n]

llvm/include/llvm/CodeGen/TargetLowering.h

4020

Is the comment still incorrect?

Oof, fixed.

[RKSimon]

@goldstein.w.n What is happening with this patch? After they leave my "Ready to Review" list I tend to lose track......

[goldstein.w.n]

In D141653#4198903, @RKSimon wrote:

@goldstein.w.n What is happening with this patch? After they leave my "Ready to Review" list I tend to lose track......

When I tested on bootstrap build it caused on infinite loop. I think this approach is inherently brittle and susceptible
to that sort of bug. I haven't quite abandoned it as I think some of the shl/shr improvements can be salvaged. But
the imm improvedments I want to move to a new pass that runs at the way of end DAG lowering.

llvm/include/llvm/CodeGen/TargetLowering.h

public:
   /// Return true if it is profitable to combine an XOR of a logical shift
   /// to create a logical shift of NOT. This transformation may not be desirable
   /// if it disrupts a particularly auspicious target-specific tree (e.g.
   /// BIC on ARM/AArch64). By default, it returns true.
   virtual bool isDesirableToCommuteXorWithShift(const SDNode *N) const {
     return true;
   }
 
+  /// Return true if it is profitable to combine the two shifts Shift1 and
+  /// Shift2 across Op at Level. For example  fold (shl (and (srl x, c1), c3),
+  /// c2) -> (and (srl x, (sub c1, c2), (c3 << c2)).
+  ///
+  /// @param OuterShift outer shift Node
+  /// @param Op to combine shifts across
+  /// @param InnerShift inner shift Node
+  /// @param Level the current DAGCombine legalization level.
+  virtual bool isDesirableToCombineShiftsAcross(const SDNode *OuterShift,
+                                                const SDNode *Op,
+                                                const SDNode *InnerShift,
+                                                CombineLevel Level) const {
+    return false;
+  }
+
   /// Return true if the target has native support for the specified value type
   /// and it is 'desirable' to use the type for the given node type. e.g. On x86
   /// i16 is legal, but undesirable since i16 instruction encodings are longer
   /// and some i16 instructions are slow.
   virtual bool isTypeDesirableForOp(unsigned /*Opc*/, EVT VT) const {
     // By default, assume all legal types are desirable.
     return isTypeLegal(VT);
   }

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

if (ISD::matchBinaryPredicate(InnerShiftAmt, N1, MatchEqual,
       EVT InnerShiftAmtVT = N0Op0.getOperand(1).getValueType();
       SDValue NewSHL = DAG.getZExtOrTrunc(N1, DL, InnerShiftAmtVT);
       NewSHL = DAG.getNode(ISD::SHL, DL, N0Op0.getValueType(), N0Op0, NewSHL);
       AddToWorklist(NewSHL.getNode());
       return DAG.getNode(ISD::ZERO_EXTEND, SDLoc(N0), VT, NewSHL);
     }
   }
 
-  if (N0.getOpcode() == ISD::SRL || N0.getOpcode() == ISD::SRA) {
+  if (N0.getOpcode() == ISD::SRL || N0.getOpcode() == ISD::SRA ||
+      N0.getOpcode() == ISD::AND) {
     auto MatchShiftAmount = [OpSizeInBits](ConstantSDNode *LHS,
                                            ConstantSDNode *RHS) {
       const APInt &LHSC = LHS->getAPIntValue();
       const APInt &RHSC = RHS->getAPIntValue();
       return LHSC.ult(OpSizeInBits) && RHSC.ult(OpSizeInBits) &&
              LHSC.getZExtValue() <= RHSC.getZExtValue();
     };
 
     SDLoc DL(N);
 
     // fold (shl (sr[la] exact X,  C1), C2) -> (shl    X, (C2-C1)) if C1 <= C2
     // fold (shl (sr[la] exact X,  C1), C2) -> (sr[la] X, (C2-C1)) if C1 >= C2
-    if (N0->getFlags().hasExact()) {
+    if (N0.getOpcode() != ISD::AND && N0->getFlags().hasExact()) {
       if (ISD::matchBinaryPredicate(N0.getOperand(1), N1, MatchShiftAmount,
                                     /*AllowUndefs*/ false,
                                     /*AllowTypeMismatch*/ true)) {
         SDValue N01 = DAG.getZExtOrTrunc(N0.getOperand(1), DL, ShiftVT);
         SDValue Diff = DAG.getNode(ISD::SUB, DL, ShiftVT, N1, N01);
         return DAG.getNode(ISD::SHL, DL, VT, N0.getOperand(0), Diff);
       }
       if (ISD::matchBinaryPredicate(N1, N0.getOperand(1), MatchShiftAmount,
                                     /*AllowUndefs*/ false,
                                     /*AllowTypeMismatch*/ true)) {
         SDValue N01 = DAG.getZExtOrTrunc(N0.getOperand(1), DL, ShiftVT);
         SDValue Diff = DAG.getNode(ISD::SUB, DL, ShiftVT, N01, N1);
         return DAG.getNode(N0.getOpcode(), DL, VT, N0.getOperand(0), Diff);
       }
     }
 
     // fold (shl (srl x, c1), c2) -> (and (shl x, (sub c2, c1), MASK) or
     //                               (and (srl x, (sub c1, c2), MASK)
     // Only fold this if the inner shift has no other uses -- if it does,
     // folding this will increase the total number of instructions.
     if (N0.getOpcode() == ISD::SRL &&
         (N0.getOperand(1) == N1 || N0.hasOneUse()) &&
+        // TODO: If there is already an existing mask (either use of N or
+        // operand of N0) then we should do this fold regardless of
+        // `shouldFoldConstantShiftPairToMask`.
         TLI.shouldFoldConstantShiftPairToMask(N, Level)) {
       if (ISD::matchBinaryPredicate(N1, N0.getOperand(1), MatchShiftAmount,
                                     /*AllowUndefs*/ false,
                                     /*AllowTypeMismatch*/ true)) {
         SDValue N01 = DAG.getZExtOrTrunc(N0.getOperand(1), DL, ShiftVT);
         SDValue Diff = DAG.getNode(ISD::SUB, DL, ShiftVT, N01, N1);
         SDValue Mask = DAG.getAllOnesConstant(DL, VT);
         Mask = DAG.getNode(ISD::SHL, DL, VT, Mask, N01);
         Mask = DAG.getNode(ISD::SRL, DL, VT, Mask, Diff);
         SDValue Shift = DAG.getNode(ISD::SRL, DL, VT, N0.getOperand(0), Diff);
         return DAG.getNode(ISD::AND, DL, VT, Shift, Mask);
       }
       if (ISD::matchBinaryPredicate(N0.getOperand(1), N1, MatchShiftAmount,
                                     /*AllowUndefs*/ false,
                                     /*AllowTypeMismatch*/ true)) {
         SDValue N01 = DAG.getZExtOrTrunc(N0.getOperand(1), DL, ShiftVT);
         SDValue Diff = DAG.getNode(ISD::SUB, DL, ShiftVT, N1, N01);
         SDValue Mask = DAG.getAllOnesConstant(DL, VT);
         Mask = DAG.getNode(ISD::SHL, DL, VT, Mask, N1);
         SDValue Shift = DAG.getNode(ISD::SHL, DL, VT, N0.getOperand(0), Diff);
         return DAG.getNode(ISD::AND, DL, VT, Shift, Mask);
       }
     }
+
+    // fold (shl (and (srl x, c1), c3), c2) ->
+    //            (and (srl x, (sub c1, c2), (c3 << c2)) or
+    //            (and (shl x, (sub c2, c1), (c3 << c2))
+    if (N0.getOpcode() == ISD::AND && N0.hasOneUse()) {
+      if (dyn_cast<ConstantSDNode>(N0.getOperand(1))) {
+        SDValue N00 = N0.getOperand(0);
+        if (N00.getOpcode() == ISD::SRL && N00.hasOneUse() &&
+            TLI.isDesirableToCombineShiftsAcross(N, N0.getNode(), N00.getNode(),
+                                                 Level)) {
+          SDLoc DL(N);
+          SDValue N001 = DAG.getZExtOrTrunc(N00.getOperand(1), DL, ShiftVT);
+          if (ISD::matchBinaryPredicate(N1, N001, MatchShiftAmount,
+                                        /*AllowUndefs*/ false,
+                                        /*AllowTypeMismatch*/ true)) {
+            SDValue Diff = DAG.getNode(ISD::SUB, DL, ShiftVT, N001, N1);
+            SDValue Mask = DAG.getNode(ISD::SHL, DL, VT, N0.getOperand(1), N1);
+            SDValue Shift =
+                DAG.getNode(ISD::SRL, DL, VT, N00.getOperand(0), Diff);
+            return DAG.getNode(ISD::AND, DL, VT, Shift, Mask);
+          }
+          if (ISD::matchBinaryPredicate(N001, N1, MatchShiftAmount,
+                                        /*AllowUndefs*/ false,
+                                        /*AllowTypeMismatch*/ true)) {
+            SDValue Diff = DAG.getNode(ISD::SUB, DL, ShiftVT, N1, N001);
+            SDValue Mask = DAG.getNode(ISD::SHL, DL, VT, N0.getOperand(1), N1);
+            SDValue Shift =
+                DAG.getNode(ISD::SHL, DL, VT, N00.getOperand(0), Diff);
+            return DAG.getNode(ISD::AND, DL, VT, Shift, Mask);
+          }
+        }
+      }
+    }
   }
 
   // fold (shl (sra x, c1), c1) -> (and x, (shl -1, c1))
   if (N0.getOpcode() == ISD::SRA && N1 == N0.getOperand(1) &&
       isConstantOrConstantVector(N1, /* No Opaques */ true)) {
     SDLoc DL(N);
     SDValue AllBits = DAG.getAllOnesConstant(DL, VT);
     SDValue HiBitsMask = DAG.getNode(ISD::SHL, DL, VT, AllBits, N1);
if (N0.getOpcode() == ISD::STEP_VECTOR &&
     const APInt &C0 = N0.getConstantOperandAPInt(0);
     if (ShlVal.ult(C0.getBitWidth())) {
       APInt NewStep = C0 << ShlVal;
       return DAG.getStepVector(SDLoc(N), VT, NewStep);
     }
   }
 
   return SDValue();
 }
 
 // Transform a right shift of a multiply into a multiply-high.
 // Examples:
 // (srl (mul (zext i32:$a to i64), (zext i32:$a to i64)), 32) -> (mulhu $a, $b)
 // (sra (mul (sext i32:$a to i64), (sext i32:$a to i64)), 32) -> (mulhs $a, $b)
 static SDValue combineShiftToMULH(SDNode *N, SelectionDAG &DAG,
                                   const TargetLowering &TLI) {
   assert((N->getOpcode() == ISD::SRL || N->getOpcode() == ISD::SRA) &&
if (auto *N001C = isConstOrConstSplat(InnerShift.getOperand(1))) {
         SDValue And = DAG.getNode(ISD::AND, DL, InnerShiftVT, NewShift, Mask);
         return DAG.getNode(ISD::TRUNCATE, DL, VT, And);
       }
     }
   }
 
   // fold (srl (shl x, c1), c2) -> (and (shl x, (sub c1, c2), MASK) or
   //                               (and (srl x, (sub c2, c1), MASK)
-  if (N0.getOpcode() == ISD::SHL &&
-      (N0.getOperand(1) == N1 || N0->hasOneUse()) &&
-      TLI.shouldFoldConstantShiftPairToMask(N, Level)) {
+  if (N0.getOpcode() == ISD::SHL || N0.getOpcode() == ISD::AND) {
     auto MatchShiftAmount = [OpSizeInBits](ConstantSDNode *LHS,
                                            ConstantSDNode *RHS) {
       const APInt &LHSC = LHS->getAPIntValue();
       const APInt &RHSC = RHS->getAPIntValue();
       return LHSC.ult(OpSizeInBits) && RHSC.ult(OpSizeInBits) &&
              LHSC.getZExtValue() <= RHSC.getZExtValue();
     };
+    SDLoc DL(N);
+    if (N0.getOpcode() == ISD::SHL &&
+        (N0.getOperand(1) == N1 || N0->hasOneUse()) &&
+        // TODO: If there is already an existing mask (either use of N or
+        // operand of N0) then we should do this fold regardless of
+        // `shouldFoldConstantShiftPairToMask`.
+        TLI.shouldFoldConstantShiftPairToMask(N, Level)) {
       if (ISD::matchBinaryPredicate(N1, N0.getOperand(1), MatchShiftAmount,
                                     /*AllowUndefs*/ false,
                                     /*AllowTypeMismatch*/ true)) {
-      SDLoc DL(N);
         SDValue N01 = DAG.getZExtOrTrunc(N0.getOperand(1), DL, ShiftVT);
         SDValue Diff = DAG.getNode(ISD::SUB, DL, ShiftVT, N01, N1);
         SDValue Mask = DAG.getAllOnesConstant(DL, VT);
         Mask = DAG.getNode(ISD::SRL, DL, VT, Mask, N01);
         Mask = DAG.getNode(ISD::SHL, DL, VT, Mask, Diff);
         SDValue Shift = DAG.getNode(ISD::SHL, DL, VT, N0.getOperand(0), Diff);
         return DAG.getNode(ISD::AND, DL, VT, Shift, Mask);
       }
       if (ISD::matchBinaryPredicate(N0.getOperand(1), N1, MatchShiftAmount,
                                     /*AllowUndefs*/ false,
                                     /*AllowTypeMismatch*/ true)) {
-      SDLoc DL(N);
         SDValue N01 = DAG.getZExtOrTrunc(N0.getOperand(1), DL, ShiftVT);
         SDValue Diff = DAG.getNode(ISD::SUB, DL, ShiftVT, N1, N01);
         SDValue Mask = DAG.getAllOnesConstant(DL, VT);
         Mask = DAG.getNode(ISD::SRL, DL, VT, Mask, N1);
         SDValue Shift = DAG.getNode(ISD::SRL, DL, VT, N0.getOperand(0), Diff);
         return DAG.getNode(ISD::AND, DL, VT, Shift, Mask);
       }
     }
 
+    // fold (srl (and (shl x, c1), c3), c2) ->
+    //            (and (shl x, (sub c1, c2), (c3 >> c2)) or
+    //            (and (srl x, (sub c2, c1), (c3 >> c2))
+    if (N0.getOpcode() == ISD::AND && N0.hasOneUse()) {
+      if (dyn_cast<ConstantSDNode>(N0.getOperand(1))) {
+        SDValue N00 = N0.getOperand(0);
+        if (N00.getOpcode() == ISD::SHL && N00.hasOneUse() &&
+            TLI.isDesirableToCombineShiftsAcross(N, N0.getNode(), N00.getNode(),
+                                                 Level)) {
+          SDValue N001 = DAG.getZExtOrTrunc(N00.getOperand(1), DL, ShiftVT);
+          if (ISD::matchBinaryPredicate(N1, N001, MatchShiftAmount,
+                                        /*AllowUndefs*/ false,
+                                        /*AllowTypeMismatch*/ true)) {
+            SDValue Diff = DAG.getNode(ISD::SUB, DL, ShiftVT, N001, N1);
+            SDValue Mask = DAG.getNode(ISD::SRL, DL, VT, N0.getOperand(1), N1);
+            SDValue Shift =
+                DAG.getNode(ISD::SHL, DL, VT, N00.getOperand(0), Diff);
+            return DAG.getNode(ISD::AND, DL, VT, Shift, Mask);
+          }
+          if (ISD::matchBinaryPredicate(N001, N1, MatchShiftAmount,
+                                        /*AllowUndefs*/ false,
+                                        /*AllowTypeMismatch*/ true)) {
+            SDValue Diff = DAG.getNode(ISD::SUB, DL, ShiftVT, N1, N001);
+            SDValue Mask = DAG.getNode(ISD::SRL, DL, VT, N0.getOperand(1), N1);
+            SDValue Shift =
+                DAG.getNode(ISD::SRL, DL, VT, N00.getOperand(0), Diff);
+            return DAG.getNode(ISD::AND, DL, VT, Shift, Mask);
+          }
+        }
+      }
+    }
+  }
+
   // fold (srl (anyextend x), c) -> (and (anyextend (srl x, c)), mask)
   // TODO - support non-uniform vector shift amounts.
   if (N1C && N0.getOpcode() == ISD::ANY_EXTEND) {
     // Shifting in all undef bits?
     EVT SmallVT = N0.getOperand(0).getValueType();
     unsigned BitSize = SmallVT.getScalarSizeInBits();
     if (N1C->getAPIntValue().uge(BitSize))
       return DAG.getUNDEF(VT);

llvm/lib/Target/X86/X86ISelLowering.h

public:
     /// Replace the results of node with an illegal result
     /// type with new values built out of custom code.
     ///
     void ReplaceNodeResults(SDNode *N, SmallVectorImpl<SDValue>&Results,
                             SelectionDAG &DAG) const override;
 
     SDValue PerformDAGCombine(SDNode *N, DAGCombinerInfo &DCI) const override;
 
+    bool isDesirableToCombineShiftsAcross(const SDNode *OuterShift,
+                                          const SDNode *Op,
+                                          const SDNode *InnerShift,
+                                          CombineLevel Level) const override;
+
+    bool isDesirableToCommuteWithShift(const SDNode *N,
+                                       CombineLevel Level) const override;
+
     /// Return true if the target has native support for
     /// the specified value type and it is 'desirable' to use the type for the
     /// given node type. e.g. On x86 i16 is legal, but undesirable since i16
     /// instruction encodings are longer and some i16 instructions are slow.
     bool isTypeDesirableForOp(unsigned Opc, EVT VT) const override;
 
     /// Return true if the target has native support for the
     /// specified value type and it is 'desirable' to use the type. e.g. On x86

llvm/lib/Target/X86/X86ISelLowering.cpp

     return true;
 
   // Make sure the new mask can be represented by a combination of mask bits
   // and non-demanded bits.
   if (!ZeroExtendMask.isSubsetOf(Mask | ~DemandedBits))
     return false;
 
   // Replace the constant with the zero extend mask.
+  // TODO: We should also look into sign-extending here.
   SDLoc DL(Op);
   SDValue NewC = TLO.DAG.getConstant(ZeroExtendMask, DL, VT);
   SDValue NewOp = TLO.DAG.getNode(ISD::AND, DL, VT, Op.getOperand(0), NewC);
   return TLO.CombineTo(Op, NewOp);
 }
 
 void X86TargetLowering::computeKnownBitsForTargetNode(const SDValue Op,
                                                       KnownBits &Known,

     APInt Mask17 = APInt::getHighBitsSet(32, 17);
     if (DAG.MaskedValueIsZero(Op, Mask17))
       return Op;
     // Mask off upper 16-bits of sign-extended constants.
     if (ISD::isBuildVectorOfConstantSDNodes(Op.getNode()))
       return DAG.getNode(ISD::AND, SDLoc(N), VT, Op,
                          DAG.getConstant(0xFFFF, SDLoc(N), VT));
     if (Op.getOpcode() == ISD::SIGN_EXTEND && N->isOnlyUserOf(Op.getNode())) {
       SDValue Src = Op.getOperand(0);
       // Convert sext(vXi16) to zext(vXi16).
       if (Src.getScalarValueSizeInBits() == 16 && VT.getSizeInBits() <= 128)
         return DAG.getNode(ISD::ZERO_EXTEND, SDLoc(N), VT, Src);
       // Convert sext(vXi8) to zext(vXi16 sext(vXi8)) on pre-SSE41 targets
       // which will expand the extension.
       if (Src.getScalarValueSizeInBits() < 16 && !Subtarget.hasSSE41()) {
         EVT ExtVT = VT.changeVectorElementType(MVT::i16);
         Src = DAG.getNode(ISD::SIGN_EXTEND, SDLoc(N), ExtVT, Src);
         return DAG.getNode(ISD::ZERO_EXTEND, SDLoc(N), VT, Src);
       }
     }
     // Convert SIGN_EXTEND_VECTOR_INREG to ZEXT_EXTEND_VECTOR_INREG.
     if (Op.getOpcode() == ISD::SIGN_EXTEND_VECTOR_INREG &&
         N->isOnlyUserOf(Op.getNode())) {
       SDValue Src = Op.getOperand(0);
       if (Src.getScalarValueSizeInBits() == 16)
         return DAG.getNode(ISD::ZERO_EXTEND_VECTOR_INREG, SDLoc(N), VT, Src);
     }
     // Convert VSRAI(Op, 16) to VSRLI(Op, 16).
     if (Op.getOpcode() == X86ISD::VSRAI && Op.getConstantOperandVal(1) == 16 &&
         N->isOnlyUserOf(Op.getNode())) {
       return DAG.getNode(X86ISD::VSRLI, SDLoc(N), VT, Op.getOperand(0),
                          Op.getOperand(1));
     }
     return SDValue();
   };
   SDValue ZeroN0 = GetZeroableOp(N0);
   SDValue ZeroN1 = GetZeroableOp(N1);
   if (!ZeroN0 && !ZeroN1)
     return SDValue();
   N0 = ZeroN0 ? ZeroN0 : N0;
   N1 = ZeroN1 ? ZeroN1 : N1;
 
   // Use SplitOpsAndApply to handle AVX splitting.
   auto PMADDWDBuilder = [](SelectionDAG &DAG, const SDLoc &DL,
                            ArrayRef<SDValue> Ops) {
     MVT ResVT = MVT::getVectorVT(MVT::i32, Ops[0].getValueSizeInBits() / 32);
     MVT OpVT = MVT::getVectorVT(MVT::i16, Ops[0].getValueSizeInBits() / 16);
     return DAG.getNode(X86ISD::VPMADDWD, DL, ResVT,
                        DAG.getBitcast(OpVT, Ops[0]),
                        DAG.getBitcast(OpVT, Ops[1]));
   };
   return SplitOpsAndApply(DAG, Subtarget, SDLoc(N), VT, {N0, N1},
                           PMADDWDBuilder);
 }
 
 static SDValue combineMulToPMULDQ(SDNode *N, SelectionDAG &DAG,
                                   const X86Subtarget &Subtarget) {
   if (!Subtarget.hasSSE2())
     return SDValue();
 
   EVT VT = N->getValueType(0);
 
   // Only support vXi64 vectors.
   if (!VT.isVector() || VT.getVectorElementType() != MVT::i64 ||
       VT.getVectorNumElements() < 2 ||
       !isPowerOf2_32(VT.getVectorNumElements()))
     return SDValue();
 
   SDValue N0 = N->getOperand(0);
   SDValue N1 = N->getOperand(1);
 
   // MULDQ returns the 64-bit result of the signed multiplication of the lower
   // 32-bits. We can lower with this if the sign bits stretch that far.

 }
 
 static SDValue combineMul(SDNode *N, SelectionDAG &DAG,
                           TargetLowering::DAGCombinerInfo &DCI,
                           const X86Subtarget &Subtarget) {
   EVT VT = N->getValueType(0);
 
   if (SDValue V = combineMulToPMADDWD(N, DAG, Subtarget))
     return V;
 
   if (SDValue V = combineMulToPMULDQ(N, DAG, Subtarget))
     return V;
 
   if (DCI.isBeforeLegalize() && VT.isVector())
     return reduceVMULWidth(N, DAG, Subtarget);
 
   // Optimize a single multiply with constant into two operations in order to
   // implement it with two cheaper instructions, e.g. LEA + SHL, LEA + LEA.
   if (!MulConstantOptimization)
     return SDValue();
 
   // An imul is usually smaller than the alternative sequence.
   if (DAG.getMachineFunction().getFunction().hasMinSize())
     return SDValue();
 
   if (DCI.isBeforeLegalize() || DCI.isCalledByLegalizer())
     return SDValue();
 
   if (VT != MVT::i64 && VT != MVT::i32)
     return SDValue();
 
   ConstantSDNode *C = dyn_cast<ConstantSDNode>(N->getOperand(1));
   if (!C)
     return SDValue();
   if (isPowerOf2_64(C->getZExtValue()))
     return SDValue();
 
   int64_t SignMulAmt = C->getSExtValue();

                            DAG.getConstant(Log2_64(AbsMulAmt - 2),
                                            DL, MVT::i8));
       NewMul = DAG.getNode(
           ISD::ADD, DL, VT, NewMul,
           DAG.getNode(ISD::ADD, DL, VT, N->getOperand(0), N->getOperand(0)));
     } else if (SignMulAmt >= 0 && isPowerOf2_64(AbsMulAmt + 2)) {
       // (mul x, 2^N - 2) => (sub (shl x, N), (add x, x))
       NewMul = DAG.getNode(ISD::SHL, DL, VT, N->getOperand(0),
                            DAG.getConstant(Log2_64(AbsMulAmt + 2),
                                            DL, MVT::i8));
       NewMul = DAG.getNode(
           ISD::SUB, DL, VT, NewMul,
           DAG.getNode(ISD::ADD, DL, VT, N->getOperand(0), N->getOperand(0)));
     }
   }
 
   return NewMul;
 }
 
 // Try to form a MULHU or MULHS node by looking for
 // (srl (mul ext, ext), 16)
 // TODO: This is X86 specific because we want to be able to handle wide types
 // before type legalization. But we can only do it if the vector will be
 // legalized via widening/splitting. Type legalization can't handle promotion
 // of a MULHU/MULHS. There isn't a way to convey this to the generic DAG
 // combiner.
 static SDValue combineShiftToPMULH(SDNode *N, SelectionDAG &DAG,
                                    const X86Subtarget &Subtarget) {
   assert((N->getOpcode() == ISD::SRL || N->getOpcode() == ISD::SRA) &&
            "SRL or SRA node is required here!");
   SDLoc DL(N);
 
   if (!Subtarget.hasSSE2())

     return SDValue();
 
   // Need a shift by 16.
   APInt ShiftAmt;
   if (!ISD::isConstantSplatVector(N->getOperand(1).getNode(), ShiftAmt) ||
       ShiftAmt != 16)
     return SDValue();
 
   SDValue LHS = ShiftOperand.getOperand(0);
   SDValue RHS = ShiftOperand.getOperand(1);
 
   unsigned ExtOpc = LHS.getOpcode();
   if ((ExtOpc != ISD::SIGN_EXTEND && ExtOpc != ISD::ZERO_EXTEND) ||
       RHS.getOpcode() != ExtOpc)
     return SDValue();
 
   // Peek through the extends.
   LHS = LHS.getOperand(0);
   RHS = RHS.getOperand(0);
 
   // Ensure the input types match.
   EVT MulVT = LHS.getValueType();
   if (MulVT.getVectorElementType() != MVT::i16 || RHS.getValueType() != MulVT)
     return SDValue();
 
   unsigned Opc = ExtOpc == ISD::SIGN_EXTEND ? ISD::MULHS : ISD::MULHU;
   SDValue Mulh = DAG.getNode(Opc, DL, MulVT, LHS, RHS);
 
   ExtOpc = N->getOpcode() == ISD::SRA ? ISD::SIGN_EXTEND : ISD::ZERO_EXTEND;
   return DAG.getNode(ExtOpc, DL, VT, Mulh);
 }
 
-static SDValue combineShiftLeft(SDNode *N, SelectionDAG &DAG) {
+static unsigned findImplicitModUsage(SDNode *N,
+                                     SmallPtrSet<SDNode *, 8> *SeenNodes) {
+  if (!SeenNodes->insert(N).second)
+    return 0;
+  for (auto U : N->uses()) {
+    unsigned Opc = U->getOpcode();
+    if ((Opc == ISD::SHL || Opc == ISD::SRL || Opc == ISD::SRA ||
+         Opc == X86ISD::BT) &&
+        N == U->getOperand(1).getNode())
+      return U->getOperand(0).getValueSizeInBits() - 1;
+
+    if (ISD::isExtOpcode(Opc) || Opc == ISD::TRUNCATE || Opc == ISD::BITCAST) {
+      if (unsigned ModSize = findImplicitModUsage(U, SeenNodes))
+        return ModSize;
+    }
+  }
+
+  return 0;
+}
+
+static unsigned getImplicitModMask(SDNode *N) {
+  SmallPtrSet<SDNode *, 8> SeenNodes;
+  return findImplicitModUsage(N, &SeenNodes);
+}
+
+static bool isMaskAlreadyIdeal(unsigned DesiredMask, bool PreserveANDN,
+                               const APInt &AndMask) {
+  if (PreserveANDN && AndMask.getSignificantBits() <= 32)
+    return true;
+
+  if (DesiredMask) {
+    // Check if we already have the desired (shift-amount) mask.
+    if (AndMask == DesiredMask)
+      return true;
+    if ((AndMask | (AndMask - 1)) == DesiredMask)
+      return true;
+  } else if (AndMask == UINT8_MAX || AndMask == UINT16_MAX ||
+             AndMask == UINT32_MAX) {
+    // Already have ideal mask
+    return true;
+  }
+
+  return false;
+}
+
+static bool isNewAndMaskPreferable(unsigned DesiredMask, bool PreserveANDN,
+                                   bool PreferShrunk, const APInt &OldAndMask,
+                                   const APInt &NewAndMask) {
+  // Several conditions for whether we want to do the swap:
+  //  1. NewAndMask == DesiredMask
+  //  2. PreserveANDN and we save a movabs
+  //  3. !PreserveANDN and we are able to create `movl/movzwl/movzbl` form mask.
+  //  4. !PreserveANDN and we are able to shrink the mask (potentially saving
+  //     `movabs` or getting imm8 form).
+
+  // Case 1: NewAndMask == DesiredMask
+  if (DesiredMask && NewAndMask == DesiredMask)
+    return true;
+
+  // Case 2: PreserveANDN and we save a movabs
+  if (NewAndMask.getSignificantBits() <= 32 &&
+      OldAndMask.getSignificantBits() > 32)
+    return true;
+
+  if (PreserveANDN)
+    return false;
+
+  // Case 3: We are able to create `movl/movzwl/movzbl` form mask.
+  if (NewAndMask.isMask()) {
+    if (NewAndMask == UINT8_MAX || NewAndMask == UINT16_MAX ||
+        NewAndMask == UINT32_MAX)
+      return true;
+  }
+
+  // Case 4: We are able to shrink the mask.
+  return PreferShrunk &&
+         NewAndMask.getSignificantBits() < OldAndMask.getSignificantBits();
+}
+
+// Try and re-ordering (srl/shl (and X, C0), C1) -> (and (srl/shl X, C1), C0
+// <</>> C1) if the new mask (C0 <</>> C1) can be more efficiently represented
+// than the original C0. Note this only tries to optimize by re-ordering.
+static SDValue
+combineLogicalShiftWithAnd(SDNode *N, SelectionDAG &DAG,
+                           TargetLowering::DAGCombinerInfo &DCI) {
+  // Only do this on the last DAG combine as it can interfere with other
+  // combines. This is also necessary to avoid and infinite loop between this
+  // and `DAGCombiner::visitShiftByConstant`.
+  if (!DCI.isAfterLegalizeDAG())
+    return SDValue();
+
+  SDNode *AndOp;
+  SDValue RawVal;
+  unsigned Opc = N->getOpcode();
+  bool PreserveANDN = false;
+  assert((Opc == ISD::SRL || Opc == ISD::SHL) &&
+         "Invalid node to combine shift with and");
+
+  // Get the shift amount, only proceed if it's constant.
+  auto *ShiftC = dyn_cast<ConstantSDNode>(N->getOperand(1));
+  if (!ShiftC)
+    return SDValue();
+
+  // Get AndOp and RawVal (RawVal being the shifted amount assuming no
+  // and).
+  AndOp = N->getOperand(0).getNode();
+  if (AndOp->getOpcode() != ISD::AND || !AndOp->hasOneUse())
+    return SDValue();
+
+  // Get the `and` mask and RawVal if we didn't get it earlier.
+  ConstantSDNode *AndC;
+  for (unsigned Idx = 0; Idx < 2; ++Idx) {
+    AndC = dyn_cast<ConstantSDNode>(AndOp->getOperand(Idx));
+    if (AndC) {
+      RawVal = AndOp->getOperand(1 - Idx);
+      break;
+    }
+  }
+
+  // Only proceeed if we have a constant mask.
+  if (!AndC)
+    return SDValue();
+
+  EVT VT = RawVal.getValueType();
+  // TODO: Makes sense to do this on vector types if it allows us to use a mask
+  // thats easier to create. Note, if this changes to include vector types
+  // update `isDesirableToCommuteWithShift`.
+  if (!VT.isScalarInteger())
+    return SDValue();
+
+  SDLoc DL(N);
+
+  // Check if the `and` is used as a shift-amount. If so, try and generate a
+  // shift-amount mask (which can be eliminated).
+  unsigned DesiredMask = getImplicitModMask(N);
+  APInt AndMask = AndC->getAPIntValue();
+  unsigned ShiftCnt = ShiftC->getZExtValue();
+
+  // Don't reorder if we have an `andn` pattern unless we will be able to
+  // eliminate a `movabs`.
+  PreserveANDN =
+      isBitwiseNot(RawVal) && DAG.getTargetLoweringInfo().hasAndNot(RawVal);
+
+  // Check if mask is already good. Either in `movl/movzwl/movzbl` form, setup
+  // for `andn` or already equal to `DesiredMask`.
+  if (isMaskAlreadyIdeal(DesiredMask, PreserveANDN, AndMask))
+    return SDValue();
+
+  // Determine Mask if we swap order of `srl/shl` and `and`.
+  APInt NewAndMask;
+  if (Opc == ISD::SHL) {
+    NewAndMask = AndMask.shl(ShiftCnt);
+    NewAndMask.setLowBits(ShiftCnt);
+    // Set high bits for fair comparison.
+    APInt SExtMask = AndMask;
+    SExtMask.setHighBits(ShiftCnt);
+    if (SExtMask.getSignificantBits() < AndMask.getSignificantBits())
+      AndMask = SExtMask;
+  } else {
+    // TODO: This can/should be `ashr`, but there are several other places that
+    // undo sign-extended constants so making a replacement that relies on
+    // sign-extension can end up as a net negative.
+    NewAndMask = AndMask.lshr(ShiftCnt);
+  }
+
+  if (isNewAndMaskPreferable(DesiredMask, PreserveANDN,
+                             /* PreferShrunk */ true, AndMask, NewAndMask))
+    return DAG.getNode(
+        ISD::AND, DL, VT,
+        DAG.getNode(Opc, DL, VT, RawVal, DAG.getConstant(ShiftCnt, DL, VT)),
+        DAG.getConstant(NewAndMask, DL, VT));
+
+  return SDValue();
+}
+
+// Try and re-ordering (and (srl/shl X, C0), C1) -> (srl/shl (and X, C1 <</>>
+// C0), C0) if the new mask (C1 <</>> C0) can be more efficiently represented
+// than the original C1. Note this only tries to optimize by re-ordering.
+static SDValue
+combineAndWithLogicalShift(SDNode *N, SelectionDAG &DAG,
+                           TargetLowering::DAGCombinerInfo &DCI) {
+  // Only do this on the last DAG combine as it can interfere with other
+  // combines. This is also necessary to avoid and infinite loop between this
+  // and `DAGCombiner::visitShiftByConstant`.
+  if (!DCI.isAfterLegalizeDAG())
+    return SDValue();
+
+  SDNode *ShiftOp;
+  SDValue RawVal;
+  assert(N->getOpcode() == ISD::AND &&
+         "Invalid node to combine shift with and");
+
+  // Get ShiftOp, AndOp and RawVal (RawVal being the shifted amount assuming
+  // no and).
+  unsigned Idx;
+  ConstantSDNode *AndC;
+  for (Idx = 0; Idx < 2; ++Idx) {
+    ShiftOp = N->getOperand(Idx).getNode();
+    if ((ShiftOp->getOpcode() == ISD::SRL ||
+         ShiftOp->getOpcode() == ISD::SHL) &&
+        ShiftOp->hasOneUse()) {
+      AndC = dyn_cast<ConstantSDNode>(N->getOperand(1 - Idx));
+      RawVal = ShiftOp->getOperand(0);
+      break;
+    }
+  }
+
+  // We either could not find a shift or the and mask is non-constant.
+  if (Idx == 2 || !AndC)
+    return SDValue();
+
+  // Only proceed if shift-amount is constant.
+  auto *ShiftC = dyn_cast<ConstantSDNode>(ShiftOp->getOperand(1));
+  if (!ShiftC)
+    return SDValue();
+
+  EVT VT = RawVal.getValueType();
+  // TODO: Makes sense to do this on vector types if it allows us to use a
+  // mask thats easier to create.
+  if (!VT.isScalarInteger())
+    return SDValue();
+
+  SDLoc DL(N);
+  unsigned ShiftCnt = ShiftC->getZExtValue();
+  unsigned DesiredMask = getImplicitModMask(N);
+  APInt AndMask = AndC->getAPIntValue();
+
+  // This is a hack. Avoid reordering if RawVal is potentially alive after the
+  // and/shift and the shift can be emitted as a LEA. This avoid causing a
+  // slight scheduling regression by emitting mov + shift instead of just LEA.
+  // Note, its possible to save some move instructions by adding similiar logic
+  // in 'combineLogicalShiftWithAnd'.
+  // TODO: Add a pass after ISel to reorder shifts if they can be transformed to
+  // LEA to save move instructions and remove this hack.
+  bool PreferShrunk = !(ShiftOp->getOpcode() == ISD::SHL && ShiftCnt <= 3 &&
+                        !RawVal.hasOneUse());
+
+  // Check if mask is already good. Either in `movl/movzwl/movzbl` form, setup
+  // for `andn` or already equal to `DesiredMask`.
+  if (isMaskAlreadyIdeal(DesiredMask, /* PreserveANDN */ false, AndMask))
+    return SDValue();
+
+  // Determine Mask if we swap order of `srl/shl` and `and`.
+  APInt NewAndMask;
+  if (ShiftOp->getOpcode() == ISD::SHL) {
+    // TODO: This can/should be `ashr`, but there are several other places that
+    // undo sign-extended constants so making a replacement that relies on
+    // sign-extension can end up as a net negative.
+    NewAndMask = AndMask.lshr(ShiftCnt);
+  } else {
+    NewAndMask = AndMask.shl(ShiftCnt);
+    NewAndMask.setLowBits(ShiftCnt);
+    // Set high bits for fair comparison.
+    APInt SExtMask = AndMask;
+    SExtMask.setHighBits(ShiftCnt);
+    if (SExtMask.getSignificantBits() < AndMask.getSignificantBits())
+      AndMask = SExtMask;
+  }
+
+  if (isNewAndMaskPreferable(DesiredMask, /* PreserveANDN */ false,
+                             PreferShrunk, AndMask, NewAndMask))
+    return DAG.getNode(ShiftOp->getOpcode(), DL, VT,
+                       DAG.getNode(ISD::AND, DL, VT, RawVal,
+                                   DAG.getConstant(NewAndMask, DL, VT)),
+                       DAG.getConstant(ShiftCnt, DL, VT));
+
+  return SDValue();
+}
+
+static SDValue combineShiftLeft(SDNode *N, SelectionDAG &DAG,
+                                TargetLowering::DAGCombinerInfo &DCI) {
   SDValue N0 = N->getOperand(0);
   SDValue N1 = N->getOperand(1);
   ConstantSDNode *N1C = dyn_cast<ConstantSDNode>(N1);
   EVT VT = N0.getValueType();
 
   // fold (shl (and (setcc_c), c1), c2) -> (and setcc_c, (c1 << c2))
   // since the result of setcc_c is all zero's or all ones.
   if (VT.isInteger() && !VT.isVector() &&

       MaskOK = Mask.isIntN(N00.getOperand(0).getValueSizeInBits());
     }
     if (MaskOK && Mask != 0) {
       SDLoc DL(N);
       return DAG.getNode(ISD::AND, DL, VT, N00, DAG.getConstant(Mask, DL, VT));
     }
   }
 
+  if (SDValue V = combineLogicalShiftWithAnd(N, DAG, DCI))
+    return V;
+
   return SDValue();
 }
 
 static SDValue combineShiftRightArithmetic(SDNode *N, SelectionDAG &DAG,
                                            const X86Subtarget &Subtarget) {
   SDValue N0 = N->getOperand(0);
   SDValue N1 = N->getOperand(1);
   EVT VT = N0.getValueType();

                        DAG.getConstant(SarConst, DL, CVT));
   }
   return SDValue();
 }
 
 static SDValue combineShiftRightLogical(SDNode *N, SelectionDAG &DAG,
                                         TargetLowering::DAGCombinerInfo &DCI,
                                         const X86Subtarget &Subtarget) {
-  SDValue N0 = N->getOperand(0);
-  SDValue N1 = N->getOperand(1);
-  EVT VT = N0.getValueType();
-
   if (SDValue V = combineShiftToPMULH(N, DAG, Subtarget))
     return V;
 
   // Only do this on the last DAG combine as it can interfere with other
   // combines.
   if (!DCI.isAfterLegalizeDAG())
     return SDValue();
 
-  // Try to improve a sequence of srl (and X, C1), C2 by inverting the order.
-  // TODO: This is a generic DAG combine that became an x86-only combine to
-  // avoid shortcomings in other folds such as bswap, bit-test ('bt'), and
-  // and-not ('andn').
-  if (N0.getOpcode() != ISD::AND || !N0.hasOneUse())
-    return SDValue();
-
-  auto *ShiftC = dyn_cast<ConstantSDNode>(N1);
-  auto *AndC = dyn_cast<ConstantSDNode>(N0.getOperand(1));
-  if (!ShiftC || !AndC)
-    return SDValue();
-
-  // If we can shrink the constant mask below 8-bits or 32-bits, then this
-  // transform should reduce code size. It may also enable secondary transforms
-  // from improved known-bits analysis or instruction selection.
-  APInt MaskVal = AndC->getAPIntValue();
-
-  // If this can be matched by a zero extend, don't optimize.
-  if (MaskVal.isMask()) {
-    unsigned TO = MaskVal.countTrailingOnes();
-    if (TO >= 8 && isPowerOf2_32(TO))
-      return SDValue();
-  }
+  if (SDValue V = combineLogicalShiftWithAnd(N, DAG, DCI))
+    return V;
 
-  APInt NewMaskVal = MaskVal.lshr(ShiftC->getAPIntValue());
-  unsigned OldMaskSize = MaskVal.getMinSignedBits();
-  unsigned NewMaskSize = NewMaskVal.getMinSignedBits();
-  if ((OldMaskSize > 8 && NewMaskSize <= 8) ||
-      (OldMaskSize > 32 && NewMaskSize <= 32)) {
-    // srl (and X, AndC), ShiftC --> and (srl X, ShiftC), (AndC >> ShiftC)
-    SDLoc DL(N);
-    SDValue NewMask = DAG.getConstant(NewMaskVal, DL, VT);
-    SDValue NewShift = DAG.getNode(ISD::SRL, DL, VT, N0.getOperand(0), N1);
-    return DAG.getNode(ISD::AND, DL, VT, NewShift, NewMask);
-  }
   return SDValue();
 }
 
 static SDValue combineHorizOpWithShuffle(SDNode *N, SelectionDAG &DAG,
                                          const X86Subtarget &Subtarget) {
   unsigned Opcode = N->getOpcode();
   assert(isHorizOp(Opcode) && "Unexpected hadd/hsub/pack opcode");
 

     return SDValue();
 
   // Don't prevent creation of ANDN.
   if (isBitwiseNot(Op0))
     return SDValue();
 
   if (!supportedVectorShiftWithImm(VT.getSimpleVT(), Subtarget, ISD::SRL))
     return SDValue();
 
   unsigned EltBitWidth = VT.getScalarSizeInBits();
   if (EltBitWidth != DAG.ComputeNumSignBits(Op0))
     return SDValue();
 
   SDLoc DL(N);
   unsigned ShiftVal = SplatVal.countTrailingOnes();
   SDValue ShAmt = DAG.getTargetConstant(EltBitWidth - ShiftVal, DL, MVT::i8);
   SDValue Shift = DAG.getNode(X86ISD::VSRLI, DL, VT, Op0, ShAmt);

               X86::MaxShuffleCombineDepth,
               /*HasVarMask*/ false, /*AllowVarCrossLaneMask*/ true,
               /*AllowVarPerLaneMask*/ true, DAG, Subtarget))
         return DAG.getNode(ISD::EXTRACT_VECTOR_ELT, dl, VT, Shuffle,
                            N0.getOperand(1));
     }
   }
 
+  if (SDValue V = combineAndWithLogicalShift(N, DAG, DCI))
+    return V;
+
   return SDValue();
 }
 
 // Canonicalize OR(AND(X,C),AND(Y,~C)) -> OR(AND(X,C),ANDNP(C,Y))
 static SDValue canonicalizeBitSelect(SDNode *N, SelectionDAG &DAG,
                                      const X86Subtarget &Subtarget) {
   assert(N->getOpcode() == ISD::OR && "Unexpected Opcode");
 

   }
 
   // Attempt to combine as a shuffle on SSE41+ targets.
   if ((Opcode == ISD::ANY_EXTEND_VECTOR_INREG ||
        Opcode == ISD::ZERO_EXTEND_VECTOR_INREG) &&
       Subtarget.hasSSE41()) {
     SDValue Op(N, 0);
     if (TLI.isTypeLegal(VT) && TLI.isTypeLegal(In.getValueType()))
       if (SDValue Res = combineX86ShufflesRecursively(Op, DAG, Subtarget))
         return Res;
   }
 
   return SDValue();
 }
 
 static SDValue combineKSHIFT(SDNode *N, SelectionDAG &DAG,
                              TargetLowering::DAGCombinerInfo &DCI) {

   case X86ISD::CMP:         return combineCMP(N, DAG);
   case ISD::ADD:            return combineAdd(N, DAG, DCI, Subtarget);
   case ISD::SUB:            return combineSub(N, DAG, DCI, Subtarget);
   case X86ISD::ADD:
   case X86ISD::SUB:         return combineX86AddSub(N, DAG, DCI);
   case X86ISD::SBB:         return combineSBB(N, DAG);
   case X86ISD::ADC:         return combineADC(N, DAG, DCI);
   case ISD::MUL:            return combineMul(N, DAG, DCI, Subtarget);
-  case ISD::SHL:            return combineShiftLeft(N, DAG);
+  case ISD::SHL:            return combineShiftLeft(N, DAG, DCI);
   case ISD::SRA:            return combineShiftRightArithmetic(N, DAG, Subtarget);
   case ISD::SRL:            return combineShiftRightLogical(N, DAG, DCI, Subtarget);
   case ISD::AND:            return combineAnd(N, DAG, DCI, Subtarget);
   case ISD::OR:             return combineOr(N, DAG, DCI, Subtarget);
   case ISD::XOR:            return combineXor(N, DAG, DCI, Subtarget);
   case X86ISD::BEXTR:
   case X86ISD::BEXTRI:      return combineBEXTR(N, DAG, DCI, Subtarget);
   case ISD::LOAD:           return combineLoad(N, DAG, DCI, Subtarget);

   case ISD::SIGN_EXTEND_VECTOR_INREG:
   case ISD::ZERO_EXTEND_VECTOR_INREG:
     return combineEXTEND_VECTOR_INREG(N, DAG, DCI, Subtarget);
   case ISD::SETCC:          return combineSetCC(N, DAG, DCI, Subtarget);
   case X86ISD::SETCC:       return combineX86SetCC(N, DAG, Subtarget);
   case X86ISD::BRCOND:      return combineBrCond(N, DAG, Subtarget);
   case X86ISD::PACKSS:
   case X86ISD::PACKUS:      return combineVectorPack(N, DAG, DCI, Subtarget);
   case X86ISD::HADD:
   case X86ISD::HSUB:
   case X86ISD::FHADD:
   case X86ISD::FHSUB:       return combineVectorHADDSUB(N, DAG, DCI, Subtarget);
   case X86ISD::VSHL:
   case X86ISD::VSRA:
   case X86ISD::VSRL:
     return combineVectorShiftVar(N, DAG, DCI, Subtarget);
   case X86ISD::VSHLI:
   case X86ISD::VSRAI:
   case X86ISD::VSRLI:
     return combineVectorShiftImm(N, DAG, DCI, Subtarget);
   case ISD::INSERT_VECTOR_ELT:
   case X86ISD::PINSRB:

   case X86ISD::SUBV_BROADCAST_LOAD: return combineBROADCAST_LOAD(N, DAG, DCI);
   case X86ISD::MOVDQ2Q:     return combineMOVDQ2Q(N, DAG);
   case X86ISD::PDEP:        return combinePDEP(N, DAG, DCI);
   }
 
   return SDValue();
 }
 
+// At the moment, always worth it to save the instruction. Maybe some edge cases
+// where a small shift left could fold with complex-addressing, but so far not
+// seen.
+bool X86TargetLowering::isDesirableToCombineShiftsAcross(
+    const SDNode *OuterShift, const SDNode *Op, const SDNode *InnerShift,
+    CombineLevel Level) const {
+  (void)OuterShift;
+  (void)Op;
+  (void)InnerShift;
+  (void)Level;
+  return true;
+}
+
+// In `combineLogicalShiftWithAnd` reorder (shift (and x, c0), c1) -> (and
+// (shift x, c1), c0 <</>> c1) if the new constant `c0 <</>> c1` is "better"
+// than the original `c1`. This is done after legalize dag, so if we are at
+// that level don't allow commute if its a pattern we already handle (otherwise
+// we can either get an infinite loop or worse codegen).
+bool X86TargetLowering::isDesirableToCommuteWithShift(
+    const SDNode *N, CombineLevel Level) const {
+  if (Level < AfterLegalizeDAG)
+    return true;
+
+  if (N->getOpcode() != ISD::SRL && N->getOpcode() != ISD::SHL)
+    return true;
+  if (dyn_cast<ConstantSDNode>(N->getOperand(1)) == nullptr)
+    return true;
+
+  SDNode *AndOp = N->getOperand(0).getNode();
+  if (AndOp->getOpcode() != ISD::AND || !AndOp->hasOneUse() ||
+      !AndOp->getValueType(0).isScalarInteger())
+    return true;
+
+  for (unsigned Idx = 0; Idx < 2; ++Idx) {
+    if (dyn_cast<ConstantSDNode>(AndOp->getOperand(Idx)) != nullptr)
+      return false;
+  }
+
+  return true;
+}
+
 bool X86TargetLowering::isTypeDesirableForOp(unsigned Opc, EVT VT) const {
   if (!isTypeLegal(VT))
     return false;
 
   // There are no vXi8 shifts.
   if (Opc == ISD::SHL && VT.isVector() && VT.getVectorElementType() == MVT::i8)
     return false;
 

llvm/test/CodeGen/X86/avx512-calling-conv.ll

 ; KNL-NEXT:    andl $1, %r9d
 ; KNL-NEXT:    shll $4, %r9d
 ; KNL-NEXT:    orl %ecx, %r9d
 ; KNL-NEXT:    andl $1, %r8d
 ; KNL-NEXT:    shll $5, %r8d
 ; KNL-NEXT:    orl %r9d, %r8d
 ; KNL-NEXT:    andl $1, %r10d
 ; KNL-NEXT:    shll $6, %r10d
-; KNL-NEXT:    andl $1, %r11d
 ; KNL-NEXT:    shll $7, %r11d
-; KNL-NEXT:    orl %r10d, %r11d
+; KNL-NEXT:    movzbl %r11b, %ecx
+; KNL-NEXT:    orl %r10d, %ecx
 ; KNL-NEXT:    andl $1, %ebx
 ; KNL-NEXT:    shll $8, %ebx
-; KNL-NEXT:    orl %r11d, %ebx
+; KNL-NEXT:    orl %ecx, %ebx
 ; KNL-NEXT:    andl $1, %r14d
 ; KNL-NEXT:    shll $9, %r14d
 ; KNL-NEXT:    orl %ebx, %r14d
 ; KNL-NEXT:    andl $1, %ebp
 ; KNL-NEXT:    shll $10, %ebp
 ; KNL-NEXT:    orl %r14d, %ebp
 ; KNL-NEXT:    orl %r8d, %ebp
 ; KNL-NEXT:    andl $1, %r15d
 ; KNL-NEXT:    shll $11, %r15d
 ; KNL-NEXT:    andl $1, %r12d
 ; KNL-NEXT:    shll $12, %r12d
 ; KNL-NEXT:    orl %r15d, %r12d
 ; KNL-NEXT:    andl $1, %r13d
 ; KNL-NEXT:    shll $13, %r13d
 ; KNL-NEXT:    orl %r12d, %r13d
 ; KNL-NEXT:    andl $1, %edx
 ; KNL-NEXT:    shll $14, %edx
 ; KNL-NEXT:    orl %r13d, %edx
-; KNL-NEXT:    andl $1, %esi
 ; KNL-NEXT:    shll $15, %esi
-; KNL-NEXT:    orl %edx, %esi
-; KNL-NEXT:    orl %ebp, %esi
-; KNL-NEXT:    movw %si, (%rax)
+; KNL-NEXT:    movzwl %si, %ecx
+; KNL-NEXT:    orl %edx, %ecx
+; KNL-NEXT:    orl %ebp, %ecx
+; KNL-NEXT:    movw %cx, (%rax)
 ; KNL-NEXT:    popq %rbx
 ; KNL-NEXT:    popq %r12
 ; KNL-NEXT:    popq %r13
 ; KNL-NEXT:    popq %r14
 ; KNL-NEXT:    popq %r15
 ; KNL-NEXT:    popq %rbp
 ; KNL-NEXT:    retq
 ;

 ; SKX-NEXT:    andl $1, %r9d
 ; SKX-NEXT:    shll $4, %r9d
 ; SKX-NEXT:    orl %ecx, %r9d
 ; SKX-NEXT:    andl $1, %r8d
 ; SKX-NEXT:    shll $5, %r8d
 ; SKX-NEXT:    orl %r9d, %r8d
 ; SKX-NEXT:    andl $1, %r10d
 ; SKX-NEXT:    shll $6, %r10d
-; SKX-NEXT:    andl $1, %r11d
 ; SKX-NEXT:    shll $7, %r11d
-; SKX-NEXT:    orl %r10d, %r11d
+; SKX-NEXT:    movzbl %r11b, %ecx
+; SKX-NEXT:    orl %r10d, %ecx
 ; SKX-NEXT:    andl $1, %ebx
 ; SKX-NEXT:    shll $8, %ebx
-; SKX-NEXT:    orl %r11d, %ebx
+; SKX-NEXT:    orl %ecx, %ebx
 ; SKX-NEXT:    andl $1, %r14d
 ; SKX-NEXT:    shll $9, %r14d
 ; SKX-NEXT:    orl %ebx, %r14d
 ; SKX-NEXT:    andl $1, %ebp
 ; SKX-NEXT:    shll $10, %ebp
 ; SKX-NEXT:    orl %r14d, %ebp
 ; SKX-NEXT:    orl %r8d, %ebp
 ; SKX-NEXT:    andl $1, %r15d
 ; SKX-NEXT:    shll $11, %r15d
 ; SKX-NEXT:    andl $1, %r12d
 ; SKX-NEXT:    shll $12, %r12d
 ; SKX-NEXT:    orl %r15d, %r12d
 ; SKX-NEXT:    andl $1, %r13d
 ; SKX-NEXT:    shll $13, %r13d
 ; SKX-NEXT:    orl %r12d, %r13d
 ; SKX-NEXT:    andl $1, %edx
 ; SKX-NEXT:    shll $14, %edx
 ; SKX-NEXT:    orl %r13d, %edx
-; SKX-NEXT:    andl $1, %esi
 ; SKX-NEXT:    shll $15, %esi
-; SKX-NEXT:    orl %edx, %esi
-; SKX-NEXT:    orl %ebp, %esi
-; SKX-NEXT:    movw %si, (%rax)
+; SKX-NEXT:    movzwl %si, %ecx
+; SKX-NEXT:    orl %edx, %ecx
+; SKX-NEXT:    orl %ebp, %ecx
+; SKX-NEXT:    movw %cx, (%rax)
 ; SKX-NEXT:    popq %rbx
 ; SKX-NEXT:    popq %r12
 ; SKX-NEXT:    popq %r13
 ; SKX-NEXT:    popq %r14
 ; SKX-NEXT:    popq %r15
 ; SKX-NEXT:    popq %rbp
 ; SKX-NEXT:    retq
 ;

 ; KNL_X32-NEXT:    kmovw %edx, %k1
 ; KNL_X32-NEXT:    kshiftlw $15, %k1, %k1
 ; KNL_X32-NEXT:    korw %k1, %k0, %k0
 ; KNL_X32-NEXT:    kmovw %ecx, %k1
 ; KNL_X32-NEXT:    kmovw (%esp), %k2 ## 2-byte Reload
 ; KNL_X32-NEXT:    kandw %k2, %k0, %k0
 ; KNL_X32-NEXT:    kmovw %eax, %k2
 ; KNL_X32-NEXT:    kandw %k1, %k2, %k1
-; KNL_X32-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; KNL_X32-NEXT:    movl {{[0-9]+}}(%esp), %ebp
 ; KNL_X32-NEXT:    kmovw %k1, %ebx
 ; KNL_X32-NEXT:    kshiftrw $1, %k0, %k1
-; KNL_X32-NEXT:    kmovw %k1, %ebp
+; KNL_X32-NEXT:    kmovw %k1, %eax
 ; KNL_X32-NEXT:    kshiftrw $2, %k0, %k1
-; KNL_X32-NEXT:    kmovw %k1, %esi
-; KNL_X32-NEXT:    kshiftrw $3, %k0, %k1
 ; KNL_X32-NEXT:    kmovw %k1, %edi
+; KNL_X32-NEXT:    kshiftrw $3, %k0, %k1
+; KNL_X32-NEXT:    kmovw %k1, %esi
 ; KNL_X32-NEXT:    kshiftrw $4, %k0, %k1
 ; KNL_X32-NEXT:    kmovw %k1, %edx
 ; KNL_X32-NEXT:    kshiftrw $5, %k0, %k1
 ; KNL_X32-NEXT:    kmovw %k1, %ecx
 ; KNL_X32-NEXT:    kshiftrw $6, %k0, %k1
 ; KNL_X32-NEXT:    andl $1, %ebx
-; KNL_X32-NEXT:    movb %bl, 2(%eax)
+; KNL_X32-NEXT:    movb %bl, 2(%ebp)
 ; KNL_X32-NEXT:    kmovw %k0, %ebx
 ; KNL_X32-NEXT:    andl $1, %ebx
-; KNL_X32-NEXT:    andl $1, %ebp
-; KNL_X32-NEXT:    leal (%ebx,%ebp,2), %ebx
-; KNL_X32-NEXT:    kmovw %k1, %ebp
+; KNL_X32-NEXT:    andl $1, %eax
+; KNL_X32-NEXT:    leal (%ebx,%eax,2), %eax
+; KNL_X32-NEXT:    kmovw %k1, %ebx
 ; KNL_X32-NEXT:    kshiftrw $7, %k0, %k1
+; KNL_X32-NEXT:    andl $1, %edi
+; KNL_X32-NEXT:    leal (%eax,%edi,4), %edi
+; KNL_X32-NEXT:    kmovw %k1, %eax
+; KNL_X32-NEXT:    kshiftrw $8, %k0, %k1
 ; KNL_X32-NEXT:    andl $1, %esi
-; KNL_X32-NEXT:    leal (%ebx,%esi,4), %ebx
+; KNL_X32-NEXT:    leal (%edi,%esi,8), %edi
 ; KNL_X32-NEXT:    kmovw %k1, %esi
-; KNL_X32-NEXT:    kshiftrw $8, %k0, %k1
-; KNL_X32-NEXT:    andl $1, %edi
-; KNL_X32-NEXT:    leal (%ebx,%edi,8), %ebx
-; KNL_X32-NEXT:    kmovw %k1, %edi
 ; KNL_X32-NEXT:    kshiftrw $9, %k0, %k1
 ; KNL_X32-NEXT:    andl $1, %edx
 ; KNL_X32-NEXT:    shll $4, %edx
-; KNL_X32-NEXT:    orl %ebx, %edx
-; KNL_X32-NEXT:    kmovw %k1, %ebx
+; KNL_X32-NEXT:    orl %edi, %edx
+; KNL_X32-NEXT:    kmovw %k1, %edi
 ; KNL_X32-NEXT:    kshiftrw $10, %k0, %k1
 ; KNL_X32-NEXT:    andl $1, %ecx
 ; KNL_X32-NEXT:    shll $5, %ecx
 ; KNL_X32-NEXT:    orl %edx, %ecx
 ; KNL_X32-NEXT:    kmovw %k1, %edx
 ; KNL_X32-NEXT:    kshiftrw $11, %k0, %k1
-; KNL_X32-NEXT:    andl $1, %ebp
-; KNL_X32-NEXT:    shll $6, %ebp
+; KNL_X32-NEXT:    andl $1, %ebx
+; KNL_X32-NEXT:    shll $6, %ebx
+; KNL_X32-NEXT:    shll $7, %eax
+; KNL_X32-NEXT:    movzbl %al, %eax
+; KNL_X32-NEXT:    orl %ebx, %eax
+; KNL_X32-NEXT:    kmovw %k1, %ebx
+; KNL_X32-NEXT:    kshiftrw $12, %k0, %k1
 ; KNL_X32-NEXT:    andl $1, %esi
-; KNL_X32-NEXT:    shll $7, %esi
-; KNL_X32-NEXT:    orl %ebp, %esi
+; KNL_X32-NEXT:    shll $8, %esi
+; KNL_X32-NEXT:    orl %eax, %esi
 ; KNL_X32-NEXT:    kmovw %k1, %ebp
-; KNL_X32-NEXT:    kshiftrw $12, %k0, %k1
+; KNL_X32-NEXT:    kshiftrw $13, %k0, %k1
 ; KNL_X32-NEXT:    andl $1, %edi
-; KNL_X32-NEXT:    shll $8, %edi
+; KNL_X32-NEXT:    shll $9, %edi
 ; KNL_X32-NEXT:    orl %esi, %edi
-; KNL_X32-NEXT:    kmovw %k1, %esi
-; KNL_X32-NEXT:    kshiftrw $13, %k0, %k1
-; KNL_X32-NEXT:    andl $1, %ebx
-; KNL_X32-NEXT:    shll $9, %ebx
-; KNL_X32-NEXT:    orl %edi, %ebx
-; KNL_X32-NEXT:    kmovw %k1, %edi
+; KNL_X32-NEXT:    kmovw %k1, %eax
 ; KNL_X32-NEXT:    kshiftrw $14, %k0, %k1
 ; KNL_X32-NEXT:    andl $1, %edx
 ; KNL_X32-NEXT:    shll $10, %edx
-; KNL_X32-NEXT:    orl %ebx, %edx
-; KNL_X32-NEXT:    kmovw %k1, %ebx
+; KNL_X32-NEXT:    orl %edi, %edx
+; KNL_X32-NEXT:    kmovw %k1, %esi
 ; KNL_X32-NEXT:    kshiftrw $15, %k0, %k0
 ; KNL_X32-NEXT:    orl %ecx, %edx
 ; KNL_X32-NEXT:    kmovw %k0, %ecx
+; KNL_X32-NEXT:    andl $1, %ebx
+; KNL_X32-NEXT:    shll $11, %ebx
 ; KNL_X32-NEXT:    andl $1, %ebp
-; KNL_X32-NEXT:    shll $11, %ebp
+; KNL_X32-NEXT:    shll $12, %ebp
+; KNL_X32-NEXT:    orl %ebx, %ebp
+; KNL_X32-NEXT:    andl $1, %eax
+; KNL_X32-NEXT:    shll $13, %eax
+; KNL_X32-NEXT:    orl %ebp, %eax
 ; KNL_X32-NEXT:    andl $1, %esi
-; KNL_X32-NEXT:    shll $12, %esi
-; KNL_X32-NEXT:    orl %ebp, %esi
-; KNL_X32-NEXT:    andl $1, %edi
-; KNL_X32-NEXT:    shll $13, %edi
-; KNL_X32-NEXT:    orl %esi, %edi
-; KNL_X32-NEXT:    andl $1, %ebx
-; KNL_X32-NEXT:    shll $14, %ebx
-; KNL_X32-NEXT:    orl %edi, %ebx
-; KNL_X32-NEXT:    andl $1, %ecx
+; KNL_X32-NEXT:    shll $14, %esi
+; KNL_X32-NEXT:    orl %eax, %esi
 ; KNL_X32-NEXT:    shll $15, %ecx
-; KNL_X32-NEXT:    orl %ebx, %ecx
+; KNL_X32-NEXT:    movzwl %cx, %ecx
+; KNL_X32-NEXT:    orl %esi, %ecx
 ; KNL_X32-NEXT:    orl %edx, %ecx
+; KNL_X32-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; KNL_X32-NEXT:    movw %cx, (%eax)
 ; KNL_X32-NEXT:    addl $16, %esp
 ; KNL_X32-NEXT:    popl %esi
 ; KNL_X32-NEXT:    popl %edi
 ; KNL_X32-NEXT:    popl %ebx
 ; KNL_X32-NEXT:    popl %ebp
 ; KNL_X32-NEXT:    retl $4
 ;

 ; FASTISEL-NEXT:    andl $1, %r9d
 ; FASTISEL-NEXT:    shll $4, %r9d
 ; FASTISEL-NEXT:    orl %ecx, %r9d
 ; FASTISEL-NEXT:    andl $1, %r8d
 ; FASTISEL-NEXT:    shll $5, %r8d
 ; FASTISEL-NEXT:    orl %r9d, %r8d
 ; FASTISEL-NEXT:    andl $1, %r10d
 ; FASTISEL-NEXT:    shll $6, %r10d
-; FASTISEL-NEXT:    andl $1, %r11d
 ; FASTISEL-NEXT:    shll $7, %r11d
-; FASTISEL-NEXT:    orl %r10d, %r11d
+; FASTISEL-NEXT:    movzbl %r11b, %ecx
+; FASTISEL-NEXT:    orl %r10d, %ecx
 ; FASTISEL-NEXT:    andl $1, %ebx
 ; FASTISEL-NEXT:    shll $8, %ebx
-; FASTISEL-NEXT:    orl %r11d, %ebx
+; FASTISEL-NEXT:    orl %ecx, %ebx
 ; FASTISEL-NEXT:    andl $1, %r14d
 ; FASTISEL-NEXT:    shll $9, %r14d
 ; FASTISEL-NEXT:    orl %ebx, %r14d
 ; FASTISEL-NEXT:    andl $1, %ebp
 ; FASTISEL-NEXT:    shll $10, %ebp
 ; FASTISEL-NEXT:    orl %r14d, %ebp
 ; FASTISEL-NEXT:    orl %r8d, %ebp
 ; FASTISEL-NEXT:    andl $1, %r15d
 ; FASTISEL-NEXT:    shll $11, %r15d
 ; FASTISEL-NEXT:    andl $1, %r12d
 ; FASTISEL-NEXT:    shll $12, %r12d
 ; FASTISEL-NEXT:    orl %r15d, %r12d
 ; FASTISEL-NEXT:    andl $1, %r13d
 ; FASTISEL-NEXT:    shll $13, %r13d
 ; FASTISEL-NEXT:    orl %r12d, %r13d
 ; FASTISEL-NEXT:    andl $1, %edx
 ; FASTISEL-NEXT:    shll $14, %edx
 ; FASTISEL-NEXT:    orl %r13d, %edx
-; FASTISEL-NEXT:    andl $1, %esi
 ; FASTISEL-NEXT:    shll $15, %esi
-; FASTISEL-NEXT:    orl %edx, %esi
-; FASTISEL-NEXT:    orl %ebp, %esi
-; FASTISEL-NEXT:    movw %si, (%rax)
+; FASTISEL-NEXT:    movzwl %si, %ecx
+; FASTISEL-NEXT:    orl %edx, %ecx
+; FASTISEL-NEXT:    orl %ebp, %ecx
+; FASTISEL-NEXT:    movw %cx, (%rax)
 ; FASTISEL-NEXT:    popq %rbx
 ; FASTISEL-NEXT:    popq %r12
 ; FASTISEL-NEXT:    popq %r13
 ; FASTISEL-NEXT:    popq %r14
 ; FASTISEL-NEXT:    popq %r15
 ; FASTISEL-NEXT:    popq %rbp
 ; FASTISEL-NEXT:    retq
   %c = and <17 x i1> %a, %b

llvm/test/CodeGen/X86/bitreverse.ll

; GFNI-NEXT:    retq
   ret i8 %b
 }
 
 declare i4 @llvm.bitreverse.i4(i4) readnone
 
 define i4 @test_bitreverse_i4(i4 %a) {
 ; X86-LABEL: test_bitreverse_i4:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT:    movl %ecx, %eax
-; X86-NEXT:    andb $8, %al
-; X86-NEXT:    movl %ecx, %edx
-; X86-NEXT:    addb %cl, %dl
-; X86-NEXT:    andb $4, %dl
-; X86-NEXT:    movb %cl, %ah
-; X86-NEXT:    shlb $3, %ah
-; X86-NEXT:    andb $8, %ah
-; X86-NEXT:    orb %dl, %ah
+; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl %eax, %ecx
+; X86-NEXT:    addb %al, %cl
+; X86-NEXT:    andb $4, %cl
+; X86-NEXT:    movl %eax, %edx
+; X86-NEXT:    shlb $3, %dl
+; X86-NEXT:    andb $8, %dl
+; X86-NEXT:    orb %cl, %dl
+; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    shrb %cl
 ; X86-NEXT:    andb $2, %cl
-; X86-NEXT:    orb %ah, %cl
+; X86-NEXT:    orb %dl, %cl
 ; X86-NEXT:    shrb $3, %al
+; X86-NEXT:    andb $1, %al
 ; X86-NEXT:    orb %cl, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_bitreverse_i4:
 ; X64:       # %bb.0:
 ; X64-NEXT:    # kill: def $edi killed $edi def $rdi
+; X64-NEXT:    leal (%rdi,%rdi), %eax
+; X64-NEXT:    andb $4, %al
+; X64-NEXT:    leal (,%rdi,8), %ecx
+; X64-NEXT:    andb $8, %cl
+; X64-NEXT:    orb %al, %cl
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andb $8, %al
-; X64-NEXT:    leal (%rdi,%rdi), %ecx
-; X64-NEXT:    andb $4, %cl
-; X64-NEXT:    leal (,%rdi,8), %edx
-; X64-NEXT:    andb $8, %dl
-; X64-NEXT:    orb %cl, %dl
-; X64-NEXT:    shrb %dil
-; X64-NEXT:    andb $2, %dil
-; X64-NEXT:    orb %dil, %dl
-; X64-NEXT:    shrb $3, %al
-; X64-NEXT:    orb %dl, %al
+; X64-NEXT:    shrb %al
+; X64-NEXT:    andb $2, %al
+; X64-NEXT:    orb %cl, %al
+; X64-NEXT:    shrb $3, %dil
+; X64-NEXT:    andb $1, %dil
+; X64-NEXT:    orb %dil, %al
 ; X64-NEXT:    retq
 ;
 ; X86XOP-LABEL: test_bitreverse_i4:
 ; X86XOP:       # %bb.0:
 ; X86XOP-NEXT:    vmovd {{.*#+}} xmm0 = mem[0],zero,zero,zero
 ; X86XOP-NEXT:    vpperm {{\.?LCPI[0-9]+_[0-9]+}}, %xmm0, %xmm0, %xmm0
 ; X86XOP-NEXT:    vmovd %xmm0, %eax
 ; X86XOP-NEXT:    shrb $4, %al
 ; X86XOP-NEXT:    # kill: def $al killed $al killed $eax
 ; X86XOP-NEXT:    retl
 ;
 ; GFNI-LABEL: test_bitreverse_i4:
 ; GFNI:       # %bb.0:
 ; GFNI-NEXT:    # kill: def $edi killed $edi def $rdi
+; GFNI-NEXT:    leal (%rdi,%rdi), %eax
+; GFNI-NEXT:    andb $4, %al
+; GFNI-NEXT:    leal (,%rdi,8), %ecx
+; GFNI-NEXT:    andb $8, %cl
+; GFNI-NEXT:    orb %al, %cl
 ; GFNI-NEXT:    movl %edi, %eax
-; GFNI-NEXT:    andb $8, %al
-; GFNI-NEXT:    leal (%rdi,%rdi), %ecx
-; GFNI-NEXT:    andb $4, %cl
-; GFNI-NEXT:    leal (,%rdi,8), %edx
-; GFNI-NEXT:    andb $8, %dl
-; GFNI-NEXT:    orb %cl, %dl
-; GFNI-NEXT:    shrb %dil
-; GFNI-NEXT:    andb $2, %dil
-; GFNI-NEXT:    orb %dil, %dl
-; GFNI-NEXT:    shrb $3, %al
-; GFNI-NEXT:    orb %dl, %al
+; GFNI-NEXT:    shrb %al
+; GFNI-NEXT:    andb $2, %al
+; GFNI-NEXT:    orb %cl, %al
+; GFNI-NEXT:    shrb $3, %dil
+; GFNI-NEXT:    andb $1, %dil
+; GFNI-NEXT:    orb %dil, %al
 ; GFNI-NEXT:    retq
   %b = call i4 @llvm.bitreverse.i4(i4 %a)
   ret i4 %b
 }
 
 ; These tests check that bitreverse(constant) calls are folded
 
 define <2 x i16> @fold_v2i16() {

llvm/test/CodeGen/X86/bmi-x86_64.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+bmi | FileCheck %s --check-prefixes=CHECK,BEXTR-SLOW,BMI1-SLOW
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+bmi,+bmi2 | FileCheck %s --check-prefixes=CHECK,BEXTR-SLOW,BMI2-SLOW
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+bmi | FileCheck %s --check-prefixes=CHECK,BMI1-SLOW
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+bmi,+bmi2 | FileCheck %s --check-prefixes=CHECK,BMI2-SLOW
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+bmi,+fast-bextr | FileCheck %s --check-prefixes=CHECK,BEXTR-FAST
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+bmi,+bmi2,+fast-bextr | FileCheck %s --check-prefixes=CHECK,BEXTR-FAST
 
 declare i64 @llvm.x86.bmi.bextr.64(i64, i64)
 
 define i64 @bextr64(i64 %x, i64 %y)   {
 ; CHECK-LABEL: bextr64:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    bextrq %rsi, %rdi, %rax
 ; CHECK-NEXT:    retq
   %tmp = tail call i64 @llvm.x86.bmi.bextr.64(i64 %x, i64 %y)
   ret i64 %tmp
 }
 
 define i64 @bextr64b(i64 %x)  uwtable  ssp {
-; BEXTR-SLOW-LABEL: bextr64b:
-; BEXTR-SLOW:       # %bb.0:
-; BEXTR-SLOW-NEXT:    movq %rdi, %rax
-; BEXTR-SLOW-NEXT:    shrl $4, %eax
-; BEXTR-SLOW-NEXT:    andl $4095, %eax # imm = 0xFFF
-; BEXTR-SLOW-NEXT:    retq
-;
-; BEXTR-FAST-LABEL: bextr64b:
-; BEXTR-FAST:       # %bb.0:
-; BEXTR-FAST-NEXT:    movl $3076, %eax # imm = 0xC04
-; BEXTR-FAST-NEXT:    bextrl %eax, %edi, %eax
-; BEXTR-FAST-NEXT:    retq
+; CHECK-LABEL: bextr64b:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    movzwl %di, %eax
+; CHECK-NEXT:    shrl $4, %eax
+; CHECK-NEXT:    retq
   %1 = lshr i64 %x, 4
   %2 = and i64 %1, 4095
   ret i64 %2
 }
 
 ; Make sure we still use the AH subreg trick to extract 15:8
 define i64 @bextr64_subreg(i64 %x)  uwtable  ssp {
 ; CHECK-LABEL: bextr64_subreg:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movq %rdi, %rax
 ; CHECK-NEXT:    movzbl %ah, %eax
 ; CHECK-NEXT:    retq
   %1 = lshr i64 %x, 8
   %2 = and i64 %1, 255
   ret i64 %2
 }
 
 define i64 @bextr64b_load(ptr %x) {
-; BEXTR-SLOW-LABEL: bextr64b_load:
-; BEXTR-SLOW:       # %bb.0:
-; BEXTR-SLOW-NEXT:    movl (%rdi), %eax
-; BEXTR-SLOW-NEXT:    shrl $4, %eax
-; BEXTR-SLOW-NEXT:    andl $4095, %eax # imm = 0xFFF
-; BEXTR-SLOW-NEXT:    retq
-;
-; BEXTR-FAST-LABEL: bextr64b_load:
-; BEXTR-FAST:       # %bb.0:
-; BEXTR-FAST-NEXT:    movl $3076, %eax # imm = 0xC04
-; BEXTR-FAST-NEXT:    bextrl %eax, (%rdi), %eax
-; BEXTR-FAST-NEXT:    retq
+; CHECK-LABEL: bextr64b_load:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    movzwl (%rdi), %eax
+; CHECK-NEXT:    shrl $4, %eax
+; CHECK-NEXT:    retq
   %1 = load i64, ptr %x, align 8
   %2 = lshr i64 %1, 4
   %3 = and i64 %2, 4095
   ret i64 %3
 }
 
 ; PR34042
 define i64 @bextr64c(i64 %x, i32 %y) {

 ; CHECK-NEXT:    shrq $2, %rdi
 ; CHECK-NEXT:    movabsq $8589934590, %rax # imm = 0x1FFFFFFFE
 ; CHECK-NEXT:    andq %rdi, %rax
 ; CHECK-NEXT:    retq
 entry:
   %shr = lshr i64 %x, 2
   %and = and i64 %shr, 8589934590
   ret i64 %and
 }

llvm/test/CodeGen/X86/bmi.ll

 ; X64-NEXT:    bextrl %esi, (%rdi), %eax
 ; X64-NEXT:    retq
   %x1 = load i32, ptr %x
   %tmp = tail call i32 @llvm.x86.bmi.bextr.32(i32 %x1, i32 %y)
   ret i32 %tmp
 }
 
 define i32 @bextr32b(i32 %x)  uwtable  ssp {
-; X86-SLOW-BEXTR-LABEL: bextr32b:
+; X86-LABEL: bextr32b:
+; X86:       # %bb.0:
+; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    shrl $4, %eax
+; X86-NEXT:    retl
+;
+; X64-LABEL: bextr32b:
+; X64:       # %bb.0:
+; X64-NEXT:    movzwl %di, %eax
+; X64-NEXT:    shrl $4, %eax
+; X64-NEXT:    retq
+  %1 = lshr i32 %x, 4
+  %2 = and i32 %1, 4095
+  ret i32 %2
+}
+
+define i32 @bextr32b_no_mov(i32 %x)  uwtable  ssp {
+; X86-SLOW-BEXTR-LABEL: bextr32b_no_mov:
 ; X86-SLOW-BEXTR:       # %bb.0:
 ; X86-SLOW-BEXTR-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-SLOW-BEXTR-NEXT:    shrl $4, %eax
+; X86-SLOW-BEXTR-NEXT:    shrl $3, %eax
 ; X86-SLOW-BEXTR-NEXT:    andl $4095, %eax # imm = 0xFFF
 ; X86-SLOW-BEXTR-NEXT:    retl
 ;
-; X64-SLOW-BEXTR-LABEL: bextr32b:
+; X64-SLOW-BEXTR-LABEL: bextr32b_no_mov:
 ; X64-SLOW-BEXTR:       # %bb.0:
 ; X64-SLOW-BEXTR-NEXT:    movl %edi, %eax
-; X64-SLOW-BEXTR-NEXT:    shrl $4, %eax
+; X64-SLOW-BEXTR-NEXT:    shrl $3, %eax
 ; X64-SLOW-BEXTR-NEXT:    andl $4095, %eax # imm = 0xFFF
 ; X64-SLOW-BEXTR-NEXT:    retq
 ;
-; X86-FAST-BEXTR-LABEL: bextr32b:
+; X86-FAST-BEXTR-LABEL: bextr32b_no_mov:
 ; X86-FAST-BEXTR:       # %bb.0:
-; X86-FAST-BEXTR-NEXT:    movl $3076, %eax # imm = 0xC04
+; X86-FAST-BEXTR-NEXT:    movl $3075, %eax # imm = 0xC03
 ; X86-FAST-BEXTR-NEXT:    bextrl %eax, {{[0-9]+}}(%esp), %eax
 ; X86-FAST-BEXTR-NEXT:    retl
 ;
-; X64-FAST-BEXTR-LABEL: bextr32b:
+; X64-FAST-BEXTR-LABEL: bextr32b_no_mov:
 ; X64-FAST-BEXTR:       # %bb.0:
-; X64-FAST-BEXTR-NEXT:    movl $3076, %eax # imm = 0xC04
+; X64-FAST-BEXTR-NEXT:    movl $3075, %eax # imm = 0xC03
 ; X64-FAST-BEXTR-NEXT:    bextrl %eax, %edi, %eax
 ; X64-FAST-BEXTR-NEXT:    retq
-  %1 = lshr i32 %x, 4
+  %1 = lshr i32 %x, 3
   %2 = and i32 %1, 4095
   ret i32 %2
 }
 
 ; Make sure we still use AH subreg trick to extract 15:8
 define i32 @bextr32_subreg(i32 %x)  uwtable  ssp {
 ; X86-LABEL: bextr32_subreg:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: bextr32_subreg:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    movzbl %ah, %eax
 ; X64-NEXT:    retq
   %1 = lshr i32 %x, 8
   %2 = and i32 %1, 255
   ret i32 %2
 }
 
 define i32 @bextr32b_load(ptr %x)  uwtable  ssp {
-; X86-SLOW-BEXTR-LABEL: bextr32b_load:
+; X86-LABEL: bextr32b_load:
+; X86:       # %bb.0:
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzwl (%eax), %eax
+; X86-NEXT:    shrl $4, %eax
+; X86-NEXT:    retl
+;
+; X64-LABEL: bextr32b_load:
+; X64:       # %bb.0:
+; X64-NEXT:    movzwl (%rdi), %eax
+; X64-NEXT:    shrl $4, %eax
+; X64-NEXT:    retq
+  %1 = load i32, ptr %x
+  %2 = lshr i32 %1, 4
+  %3 = and i32 %2, 4095
+  ret i32 %3
+}
+
+define i32 @bextr32_load_no_mov(ptr %x)  uwtable  ssp {
+; X86-SLOW-BEXTR-LABEL: bextr32_load_no_mov:
 ; X86-SLOW-BEXTR:       # %bb.0:
 ; X86-SLOW-BEXTR-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-SLOW-BEXTR-NEXT:    movl (%eax), %eax
-; X86-SLOW-BEXTR-NEXT:    shrl $4, %eax
+; X86-SLOW-BEXTR-NEXT:    shrl $3, %eax
 ; X86-SLOW-BEXTR-NEXT:    andl $4095, %eax # imm = 0xFFF
 ; X86-SLOW-BEXTR-NEXT:    retl
 ;
-; X64-SLOW-BEXTR-LABEL: bextr32b_load:
+; X64-SLOW-BEXTR-LABEL: bextr32_load_no_mov:
 ; X64-SLOW-BEXTR:       # %bb.0:
 ; X64-SLOW-BEXTR-NEXT:    movl (%rdi), %eax
-; X64-SLOW-BEXTR-NEXT:    shrl $4, %eax
+; X64-SLOW-BEXTR-NEXT:    shrl $3, %eax
 ; X64-SLOW-BEXTR-NEXT:    andl $4095, %eax # imm = 0xFFF
 ; X64-SLOW-BEXTR-NEXT:    retq
 ;
-; X86-FAST-BEXTR-LABEL: bextr32b_load:
+; X86-FAST-BEXTR-LABEL: bextr32_load_no_mov:
 ; X86-FAST-BEXTR:       # %bb.0:
 ; X86-FAST-BEXTR-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-FAST-BEXTR-NEXT:    movl $3076, %ecx # imm = 0xC04
+; X86-FAST-BEXTR-NEXT:    movl $3075, %ecx # imm = 0xC03
 ; X86-FAST-BEXTR-NEXT:    bextrl %ecx, (%eax), %eax
 ; X86-FAST-BEXTR-NEXT:    retl
 ;
-; X64-FAST-BEXTR-LABEL: bextr32b_load:
+; X64-FAST-BEXTR-LABEL: bextr32_load_no_mov:
 ; X64-FAST-BEXTR:       # %bb.0:
-; X64-FAST-BEXTR-NEXT:    movl $3076, %eax # imm = 0xC04
+; X64-FAST-BEXTR-NEXT:    movl $3075, %eax # imm = 0xC03
 ; X64-FAST-BEXTR-NEXT:    bextrl %eax, (%rdi), %eax
 ; X64-FAST-BEXTR-NEXT:    retq
   %1 = load i32, ptr %x
-  %2 = lshr i32 %1, 4
+  %2 = lshr i32 %1, 3
   %3 = and i32 %2, 4095
   ret i32 %3
 }
 
 ; PR34042
 define i32 @bextr32c(i32 %x, i16 zeroext %y) {
 ; X86-LABEL: bextr32c:
 ; X86:       # %bb.0:
; X64-NEXT:    retq
   %tmp2 = and i32 %x1, %tmp
   ret i32 %tmp2
 }
 
 define i32 @blsi32_z(i32 %a, i32 %b) nounwind {
 ; X86-LABEL: blsi32_z:
 ; X86:       # %bb.0:
 ; X86-NEXT:    blsil {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    jne .LBB25_2
+; X86-NEXT:    jne .LBB27_2
 ; X86-NEXT:  # %bb.1:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:  .LBB25_2:
+; X86-NEXT:  .LBB27_2:
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: blsi32_z:
 ; X64:       # %bb.0:
 ; X64-NEXT:    blsil %edi, %eax
 ; X64-NEXT:    cmovel %esi, %eax
 ; X64-NEXT:    retq
   %t0 = sub i32 0, %a

 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    negl %eax
 ; X86-NEXT:    sbbl %esi, %edx
 ; X86-NEXT:    andl %esi, %edx
 ; X86-NEXT:    andl %ecx, %eax
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    orl %edx, %ecx
-; X86-NEXT:    jne .LBB29_2
+; X86-NEXT:    jne .LBB31_2
 ; X86-NEXT:  # %bb.1:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:  .LBB29_2:
+; X86-NEXT:  .LBB31_2:
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: blsi64_z:
 ; X64:       # %bb.0:
 ; X64-NEXT:    blsiq %rdi, %rax
 ; X64-NEXT:    cmoveq %rsi, %rax
 ; X64-NEXT:    retq
; X64-NEXT:    retq
   %tmp2 = xor i32 %x1, %tmp
   ret i32 %tmp2
 }
 
 define i32 @blsmsk32_z(i32 %a, i32 %b) nounwind {
 ; X86-LABEL: blsmsk32_z:
 ; X86:       # %bb.0:
 ; X86-NEXT:    blsmskl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    jne .LBB34_2
+; X86-NEXT:    jne .LBB36_2
 ; X86-NEXT:  # %bb.1:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:  .LBB34_2:
+; X86-NEXT:  .LBB36_2:
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: blsmsk32_z:
 ; X64:       # %bb.0:
 ; X64-NEXT:    blsmskl %edi, %eax
 ; X64-NEXT:    cmovel %esi, %eax
 ; X64-NEXT:    retq
   %t0 = sub i32 %a, 1

 ; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    addl $-1, %eax
 ; X86-NEXT:    movl %esi, %edx
 ; X86-NEXT:    adcl $-1, %edx
 ; X86-NEXT:    xorl %ecx, %eax
 ; X86-NEXT:    xorl %esi, %edx
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    orl %edx, %ecx
-; X86-NEXT:    jne .LBB38_2
+; X86-NEXT:    jne .LBB40_2
 ; X86-NEXT:  # %bb.1:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT:  .LBB38_2:
+; X86-NEXT:  .LBB40_2:
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: blsmsk64_z:
 ; X64:       # %bb.0:
 ; X64-NEXT:    blsmskq %rdi, %rax
 ; X64-NEXT:    cmoveq %rsi, %rax
 ; X64-NEXT:    retq
; X64-NEXT:    retq
   %tmp2 = and i32 %x1, %tmp
   ret i32 %tmp2
 }
 
 define i32 @blsr32_z(i32 %a, i32 %b) nounwind {
 ; X86-LABEL: blsr32_z:
 ; X86:       # %bb.0:
 ; X86-NEXT:    blsrl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    jne .LBB43_2
+; X86-NEXT:    jne .LBB45_2
 ; X86-NEXT:  # %bb.1:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:  .LBB43_2:
+; X86-NEXT:  .LBB45_2:
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: blsr32_z:
 ; X64:       # %bb.0:
 ; X64-NEXT:    blsrl %edi, %eax
 ; X64-NEXT:    cmovel %esi, %eax
 ; X64-NEXT:    retq
   %t0 = sub i32 %a, 1

 ; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    addl $-1, %eax
 ; X86-NEXT:    movl %esi, %edx
 ; X86-NEXT:    adcl $-1, %edx
 ; X86-NEXT:    andl %ecx, %eax
 ; X86-NEXT:    andl %esi, %edx
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    orl %edx, %ecx
-; X86-NEXT:    jne .LBB47_2
+; X86-NEXT:    jne .LBB49_2
 ; X86-NEXT:  # %bb.1:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT:  .LBB47_2:
+; X86-NEXT:  .LBB49_2:
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: blsr64_z:
 ; X64:       # %bb.0:
 ; X64-NEXT:    blsrq %rdi, %rax
 ; X64-NEXT:    cmoveq %rsi, %rax
 ; X64-NEXT:    retq

 }
 
 define void @pr40060(i32, i32) {
 ; X86-LABEL: pr40060:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    bextrl %eax, {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    testl %eax, %eax
-; X86-NEXT:    js .LBB52_1
+; X86-NEXT:    js .LBB54_1
 ; X86-NEXT:  # %bb.2:
 ; X86-NEXT:    jmp bar # TAILCALL
-; X86-NEXT:  .LBB52_1:
+; X86-NEXT:  .LBB54_1:
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: pr40060:
 ; X64:       # %bb.0:
 ; X64-NEXT:    bextrl %esi, %edi, %eax
 ; X64-NEXT:    testl %eax, %eax
-; X64-NEXT:    js .LBB52_1
+; X64-NEXT:    js .LBB54_1
 ; X64-NEXT:  # %bb.2:
 ; X64-NEXT:    jmp bar # TAILCALL
-; X64-NEXT:  .LBB52_1:
+; X64-NEXT:  .LBB54_1:
 ; X64-NEXT:    retq
   %3 = tail call i32 @llvm.x86.bmi.bextr.32(i32 %0, i32 %1)
   %4 = icmp sgt i32 %3, -1
   br i1 %4, label %5, label %6
 
   tail call void @bar()
   br label %6
 
   ret void
 }
 
 define i32 @blsr32_branch(i32 %x) {
 ; X86-LABEL: blsr32_branch:
 ; X86:       # %bb.0:
 ; X86-NEXT:    pushl %esi
 ; X86-NEXT:    .cfi_def_cfa_offset 8
 ; X86-NEXT:    .cfi_offset %esi, -8
 ; X86-NEXT:    blsrl {{[0-9]+}}(%esp), %esi
-; X86-NEXT:    jne .LBB53_2
+; X86-NEXT:    jne .LBB55_2
 ; X86-NEXT:  # %bb.1:
 ; X86-NEXT:    calll bar
-; X86-NEXT:  .LBB53_2:
+; X86-NEXT:  .LBB55_2:
 ; X86-NEXT:    movl %esi, %eax
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    .cfi_def_cfa_offset 4
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: blsr32_branch:
 ; X64:       # %bb.0:
 ; X64-NEXT:    pushq %rbx
 ; X64-NEXT:    .cfi_def_cfa_offset 16
 ; X64-NEXT:    .cfi_offset %rbx, -16
 ; X64-NEXT:    blsrl %edi, %ebx
-; X64-NEXT:    jne .LBB53_2
+; X64-NEXT:    jne .LBB55_2
 ; X64-NEXT:  # %bb.1:
 ; X64-NEXT:    callq bar
-; X64-NEXT:  .LBB53_2:
+; X64-NEXT:  .LBB55_2:
 ; X64-NEXT:    movl %ebx, %eax
 ; X64-NEXT:    popq %rbx
 ; X64-NEXT:    .cfi_def_cfa_offset 8
 ; X64-NEXT:    retq
   %tmp = sub i32 %x, 1
   %tmp2 = and i32 %x, %tmp
   %cmp = icmp eq i32 %tmp2, 0
   br i1 %cmp, label %1, label %2

 ; X86-NEXT:    movl %eax, %esi
 ; X86-NEXT:    addl $-1, %esi
 ; X86-NEXT:    movl %ecx, %edi
 ; X86-NEXT:    adcl $-1, %edi
 ; X86-NEXT:    andl %eax, %esi
 ; X86-NEXT:    andl %ecx, %edi
 ; X86-NEXT:    movl %esi, %eax
 ; X86-NEXT:    orl %edi, %eax
-; X86-NEXT:    jne .LBB54_2
+; X86-NEXT:    jne .LBB56_2
 ; X86-NEXT:  # %bb.1:
 ; X86-NEXT:    calll bar
-; X86-NEXT:  .LBB54_2:
+; X86-NEXT:  .LBB56_2:
 ; X86-NEXT:    movl %esi, %eax
 ; X86-NEXT:    movl %edi, %edx
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    .cfi_def_cfa_offset 8
 ; X86-NEXT:    popl %edi
 ; X86-NEXT:    .cfi_def_cfa_offset 4
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: blsr64_branch:
 ; X64:       # %bb.0:
 ; X64-NEXT:    pushq %rbx
 ; X64-NEXT:    .cfi_def_cfa_offset 16
 ; X64-NEXT:    .cfi_offset %rbx, -16
 ; X64-NEXT:    blsrq %rdi, %rbx
-; X64-NEXT:    jne .LBB54_2
+; X64-NEXT:    jne .LBB56_2
 ; X64-NEXT:  # %bb.1:
 ; X64-NEXT:    callq bar
-; X64-NEXT:  .LBB54_2:
+; X64-NEXT:  .LBB56_2:
 ; X64-NEXT:    movq %rbx, %rax
 ; X64-NEXT:    popq %rbx
 ; X64-NEXT:    .cfi_def_cfa_offset 8
 ; X64-NEXT:    retq
   %tmp = sub i64 %x, 1
   %tmp2 = and i64 %x, %tmp
   %cmp = icmp eq i64 %tmp2, 0
   br i1 %cmp, label %1, label %2
 
   tail call void @bar()
   br label %2
   ret i64 %tmp2
 }
 
 define i32 @blsi32_branch(i32 %x) {
 ; X86-LABEL: blsi32_branch:
 ; X86:       # %bb.0:
 ; X86-NEXT:    pushl %esi
 ; X86-NEXT:    .cfi_def_cfa_offset 8
 ; X86-NEXT:    .cfi_offset %esi, -8
 ; X86-NEXT:    blsil {{[0-9]+}}(%esp), %esi
-; X86-NEXT:    jne .LBB55_2
+; X86-NEXT:    jne .LBB57_2
 ; X86-NEXT:  # %bb.1:
 ; X86-NEXT:    calll bar
-; X86-NEXT:  .LBB55_2:
+; X86-NEXT:  .LBB57_2:
 ; X86-NEXT:    movl %esi, %eax
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    .cfi_def_cfa_offset 4
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: blsi32_branch:
 ; X64:       # %bb.0:
 ; X64-NEXT:    pushq %rbx
 ; X64-NEXT:    .cfi_def_cfa_offset 16
 ; X64-NEXT:    .cfi_offset %rbx, -16
 ; X64-NEXT:    blsil %edi, %ebx
-; X64-NEXT:    jne .LBB55_2
+; X64-NEXT:    jne .LBB57_2
 ; X64-NEXT:  # %bb.1:
 ; X64-NEXT:    callq bar
-; X64-NEXT:  .LBB55_2:
+; X64-NEXT:  .LBB57_2:
 ; X64-NEXT:    movl %ebx, %eax
 ; X64-NEXT:    popq %rbx
 ; X64-NEXT:    .cfi_def_cfa_offset 8
 ; X64-NEXT:    retq
   %tmp = sub i32 0, %x
   %tmp2 = and i32 %x, %tmp
   %cmp = icmp eq i32 %tmp2, 0
   br i1 %cmp, label %1, label %2

 ; X86-NEXT:    xorl %esi, %esi
 ; X86-NEXT:    movl %eax, %edi
 ; X86-NEXT:    negl %edi
 ; X86-NEXT:    sbbl %ecx, %esi
 ; X86-NEXT:    andl %ecx, %esi
 ; X86-NEXT:    andl %eax, %edi
 ; X86-NEXT:    movl %edi, %eax
 ; X86-NEXT:    orl %esi, %eax
-; X86-NEXT:    jne .LBB56_2
+; X86-NEXT:    jne .LBB58_2
 ; X86-NEXT:  # %bb.1:
 ; X86-NEXT:    calll bar
-; X86-NEXT:  .LBB56_2:
+; X86-NEXT:  .LBB58_2:
 ; X86-NEXT:    movl %edi, %eax
 ; X86-NEXT:    movl %esi, %edx
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    .cfi_def_cfa_offset 8
 ; X86-NEXT:    popl %edi
 ; X86-NEXT:    .cfi_def_cfa_offset 4
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: blsi64_branch:
 ; X64:       # %bb.0:
 ; X64-NEXT:    pushq %rbx
 ; X64-NEXT:    .cfi_def_cfa_offset 16
 ; X64-NEXT:    .cfi_offset %rbx, -16
 ; X64-NEXT:    blsiq %rdi, %rbx
-; X64-NEXT:    jne .LBB56_2
+; X64-NEXT:    jne .LBB58_2
 ; X64-NEXT:  # %bb.1:
 ; X64-NEXT:    callq bar
-; X64-NEXT:  .LBB56_2:
+; X64-NEXT:  .LBB58_2:
 ; X64-NEXT:    movq %rbx, %rax
 ; X64-NEXT:    popq %rbx
 ; X64-NEXT:    .cfi_def_cfa_offset 8
 ; X64-NEXT:    retq
   %tmp = sub i64 0, %x
   %tmp2 = and i64 %x, %tmp
   %cmp = icmp eq i64 %tmp2, 0
   br i1 %cmp, label %1, label %2
 
   tail call void @bar()
   br label %2
   ret i64 %tmp2
 }
 
 declare dso_local void @bar()
 
 define void @pr42118_i32(i32 %x) {
 ; X86-LABEL: pr42118_i32:
 ; X86:       # %bb.0:
 ; X86-NEXT:    blsrl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    jne .LBB57_1
+; X86-NEXT:    jne .LBB59_1
 ; X86-NEXT:  # %bb.2:
 ; X86-NEXT:    jmp bar # TAILCALL
-; X86-NEXT:  .LBB57_1:
+; X86-NEXT:  .LBB59_1:
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: pr42118_i32:
 ; X64:       # %bb.0:
 ; X64-NEXT:    blsrl %edi, %eax
-; X64-NEXT:    jne .LBB57_1
+; X64-NEXT:    jne .LBB59_1
 ; X64-NEXT:  # %bb.2:
 ; X64-NEXT:    jmp bar # TAILCALL
-; X64-NEXT:  .LBB57_1:
+; X64-NEXT:  .LBB59_1:
 ; X64-NEXT:    retq
   %tmp = sub i32 0, %x
   %tmp1 = and i32 %tmp, %x
   %cmp = icmp eq i32 %tmp1, %x
   br i1 %cmp, label %1, label %2
 
   tail call void @bar()
   br label %2

 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    movl %eax, %edx
 ; X86-NEXT:    addl $-1, %edx
 ; X86-NEXT:    movl %ecx, %esi
 ; X86-NEXT:    adcl $-1, %esi
 ; X86-NEXT:    andl %eax, %edx
 ; X86-NEXT:    andl %ecx, %esi
 ; X86-NEXT:    orl %edx, %esi
-; X86-NEXT:    jne .LBB58_1
+; X86-NEXT:    jne .LBB60_1
 ; X86-NEXT:  # %bb.2:
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    .cfi_def_cfa_offset 4
 ; X86-NEXT:    jmp bar # TAILCALL
-; X86-NEXT:  .LBB58_1:
+; X86-NEXT:  .LBB60_1:
 ; X86-NEXT:    .cfi_def_cfa_offset 8
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    .cfi_def_cfa_offset 4
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: pr42118_i64:
 ; X64:       # %bb.0:
 ; X64-NEXT:    blsrq %rdi, %rax
-; X64-NEXT:    jne .LBB58_1
+; X64-NEXT:    jne .LBB60_1
 ; X64-NEXT:  # %bb.2:
 ; X64-NEXT:    jmp bar # TAILCALL
-; X64-NEXT:  .LBB58_1:
+; X64-NEXT:  .LBB60_1:
 ; X64-NEXT:    retq
   %tmp = sub i64 0, %x
   %tmp1 = and i64 %tmp, %x
   %cmp = icmp eq i64 %tmp1, %x
   br i1 %cmp, label %1, label %2
 
   tail call void @bar()
   br label %2
 
   ret void
 }
 
 define i32 @blsi_cflag_32(i32 %x, i32 %y) nounwind {
 ; X86-LABEL: blsi_cflag_32:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    testl %eax, %eax
-; X86-NEXT:    jne .LBB59_1
+; X86-NEXT:    jne .LBB61_1
 ; X86-NEXT:  # %bb.2:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    retl
-; X86-NEXT:  .LBB59_1:
+; X86-NEXT:  .LBB61_1:
 ; X86-NEXT:    blsil %eax, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: blsi_cflag_32:
 ; X64:       # %bb.0:
 ; X64-NEXT:    blsil %edi, %eax
 ; X64-NEXT:    cmovael %esi, %eax
 ; X64-NEXT:    retq

 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    negl %eax
 ; X86-NEXT:    sbbl %esi, %edx
 ; X86-NEXT:    movl %ecx, %edi
 ; X86-NEXT:    orl %esi, %edi
-; X86-NEXT:    jne .LBB60_1
+; X86-NEXT:    jne .LBB62_1
 ; X86-NEXT:  # %bb.2:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    jmp .LBB60_3
-; X86-NEXT:  .LBB60_1:
+; X86-NEXT:    jmp .LBB62_3
+; X86-NEXT:  .LBB62_1:
 ; X86-NEXT:    andl %esi, %edx
 ; X86-NEXT:    andl %ecx, %eax
-; X86-NEXT:  .LBB60_3:
+; X86-NEXT:  .LBB62_3:
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    popl %edi
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: blsi_cflag_64:
 ; X64:       # %bb.0:
 ; X64-NEXT:    blsiq %rdi, %rax
 ; X64-NEXT:    cmovaeq %rsi, %rax
 ; X64-NEXT:    retq
   %tobool = icmp eq i64 %x, 0
   %sub = sub nsw i64 0, %x
   %and = and i64 %sub, %x
   %cond = select i1 %tobool, i64 %y, i64 %and
   ret i64 %cond
 }

llvm/test/CodeGen/X86/combine-bitreverse.ll

 ; X86-NEXT:    orl %ecx, %eax
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    andl $858993459, %ecx # imm = 0x33333333
 ; X86-NEXT:    shrl $2, %eax
 ; X86-NEXT:    andl $858993459, %eax # imm = 0x33333333
 ; X86-NEXT:    leal (%eax,%ecx,4), %ecx
 ; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    andl $5592405, %eax # imm = 0x555555
+; X86-NEXT:    andl $44739242, %ecx # imm = 0x2AAAAAA
 ; X86-NEXT:    shll $6, %ecx
-; X86-NEXT:    andl $-1431655808, %ecx # imm = 0xAAAAAA80
 ; X86-NEXT:    shll $8, %eax
 ; X86-NEXT:    orl %ecx, %eax
 ; X86-NEXT:    bswapl %eax
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    andl $986895, %ecx # imm = 0xF0F0F
 ; X86-NEXT:    shll $4, %ecx
 ; X86-NEXT:    shrl $4, %eax
 ; X86-NEXT:    andl $135204623, %eax # imm = 0x80F0F0F
 ; X86-NEXT:    orl %ecx, %eax

 ; X64-NEXT:    orl %eax, %edi
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andl $858993459, %eax # imm = 0x33333333
 ; X64-NEXT:    shrl $2, %edi
 ; X64-NEXT:    andl $858993459, %edi # imm = 0x33333333
 ; X64-NEXT:    leal (%rdi,%rax,4), %eax
 ; X64-NEXT:    movl %eax, %ecx
 ; X64-NEXT:    andl $5592405, %ecx # imm = 0x555555
+; X64-NEXT:    andl $44739242, %eax # imm = 0x2AAAAAA
 ; X64-NEXT:    shll $6, %eax
-; X64-NEXT:    andl $-1431655808, %eax # imm = 0xAAAAAA80
 ; X64-NEXT:    shll $8, %ecx
 ; X64-NEXT:    orl %eax, %ecx
 ; X64-NEXT:    bswapl %ecx
 ; X64-NEXT:    movl %ecx, %eax
 ; X64-NEXT:    andl $986895, %eax # imm = 0xF0F0F
 ; X64-NEXT:    shll $4, %eax
 ; X64-NEXT:    shrl $4, %ecx
 ; X64-NEXT:    andl $135204623, %ecx # imm = 0x80F0F0F
 ; X64-NEXT:    orl %eax, %ecx
 ; X64-NEXT:    movl %ecx, %eax
 ; X64-NEXT:    andl $3355443, %eax # imm = 0x333333
 ; X64-NEXT:    shrl $2, %ecx
 ; X64-NEXT:    andl $36909875, %ecx # imm = 0x2333333
 ; X64-NEXT:    leal (%rcx,%rax,4), %eax
 ; X64-NEXT:    movl %eax, %ecx
 ; X64-NEXT:    andl $1431655765, %ecx # imm = 0x55555555
 ; X64-NEXT:    shrl %eax
 ; X64-NEXT:    andl $1431655765, %eax # imm = 0x55555555
 ; X64-NEXT:    leal (%rax,%rcx,2), %eax
 ; X64-NEXT:    retq
   %b = call i32 @llvm.bitreverse.i32(i32 %a0)
   %c = shl i32 %b, 7
   %d = call i32 @llvm.bitreverse.i32(i32 %c)
   ret i32 %d
 }
 
 define i64 @test_bitreverse_shli_bitreverse_i64(i64 %a) nounwind {
 ; X86-LABEL: test_bitreverse_shli_bitreverse_i64:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    bswapl %eax
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    andl $252645135, %ecx # imm = 0xF0F0F0F
 ; X86-NEXT:    shll $4, %ecx
 ; X86-NEXT:    shrl $4, %eax
 ; X86-NEXT:    andl $252645135, %eax # imm = 0xF0F0F0F
 ; X86-NEXT:    orl %ecx, %eax
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    andl $858993459, %ecx # imm = 0x33333333
 ; X86-NEXT:    shrl $2, %eax
 ; X86-NEXT:    andl $858993459, %eax # imm = 0x33333333
 ; X86-NEXT:    leal (%eax,%ecx,4), %eax
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    andl $357913941, %ecx # imm = 0x15555555
 ; X86-NEXT:    andl $-1431655766, %eax # imm = 0xAAAAAAAA
 ; X86-NEXT:    leal (%eax,%ecx,4), %eax
 ; X86-NEXT:    bswapl %eax
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    andl $235867919, %ecx # imm = 0xE0F0F0F
 ; X86-NEXT:    shll $4, %ecx
 ; X86-NEXT:    shrl $4, %eax
 ; X86-NEXT:    andl $252645135, %eax # imm = 0xF0F0F0F
 ; X86-NEXT:    orl %ecx, %eax

llvm/test/CodeGen/X86/combine-rotates.ll

; CHECK-NEXT:    retq
   ret i32 %4
 }
 
 ; Ensure we normalize the inner rotation before adding the results.
 define i5 @rotl_merge_i5(i5 %x) {
 ; CHECK-LABEL: rotl_merge_i5:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    # kill: def $edi killed $edi def $rdi
-; CHECK-NEXT:    leal (,%rdi,4), %ecx
-; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    andb $24, %al
-; CHECK-NEXT:    shrb $3, %al
-; CHECK-NEXT:    orb %cl, %al
+; CHECK-NEXT:    leal (,%rdi,4), %eax
+; CHECK-NEXT:    shrb $3, %dil
+; CHECK-NEXT:    andb $3, %dil
+; CHECK-NEXT:    orb %dil, %al
+; CHECK-NEXT:    # kill: def $al killed $al killed $eax
 ; CHECK-NEXT:    retq
   %r1 = call i5 @llvm.fshl.i5(i5 %x, i5 %x, i5 -1)
   %r2 = call i5 @llvm.fshl.i5(i5 %r1, i5 %r1, i5 1)
   ret i5 %r2
 }
 declare i5 @llvm.fshl.i5(i5, i5, i5)
 
 declare <4 x i32> @llvm.fshl.v4i32(<4 x i32>, <4 x i32>, <4 x i32>)
 declare <4 x i32> @llvm.fshr.v4i32(<4 x i32>, <4 x i32>, <4 x i32>)

llvm/test/CodeGen/X86/const-shift-of-constmasked.ll

-; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --no_x86_scrub_sp
 ; RUN: llc < %s -mtriple=i686-unknown-unknown | FileCheck %s --check-prefix=X86
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown | FileCheck %s --check-prefix=X64
 
 ; The mask is all-ones, potentially shifted.
-
+; NB: Update with `--no_x86_scrub_sp` as for i686 the loads and shift
+; can fold making the exact SP value relevant to correctness.
 ;------------------------------------------------------------------------------;
 ; 8-bit
 ;------------------------------------------------------------------------------;
 
 ; lshr
 
 define i8 @test_i8_7_mask_lshr_1(i8 %a0) {
 ; X86-LABEL: test_i8_7_mask_lshr_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andb $6, %al
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    shrb %al
+; X86-NEXT:    andb $3, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_7_mask_lshr_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andb $6, %al
 ; X64-NEXT:    shrb %al
+; X64-NEXT:    andb $3, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 7
   %t1 = lshr i8 %t0, 1
   ret i8 %t1
 }
 
 define i8 @test_i8_28_mask_lshr_1(i8 %a0) {
 ; X86-LABEL: test_i8_28_mask_lshr_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andb $28, %al
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    shrb %al
+; X86-NEXT:    andb $14, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_28_mask_lshr_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andb $28, %al
 ; X64-NEXT:    shrb %al
+; X64-NEXT:    andb $14, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 28
   %t1 = lshr i8 %t0, 1
   ret i8 %t1
 }
 define i8 @test_i8_28_mask_lshr_2(i8 %a0) {
 ; X86-LABEL: test_i8_28_mask_lshr_2:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andb $28, %al
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    shrb $2, %al
+; X86-NEXT:    andb $7, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_28_mask_lshr_2:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andb $28, %al
 ; X64-NEXT:    shrb $2, %al
+; X64-NEXT:    andb $7, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 28
   %t1 = lshr i8 %t0, 2
   ret i8 %t1
 }
 define i8 @test_i8_28_mask_lshr_3(i8 %a0) {
 ; X86-LABEL: test_i8_28_mask_lshr_3:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andb $24, %al
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    shrb $3, %al
+; X86-NEXT:    andb $3, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_28_mask_lshr_3:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andb $24, %al
 ; X64-NEXT:    shrb $3, %al
+; X64-NEXT:    andb $3, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 28
   %t1 = lshr i8 %t0, 3
   ret i8 %t1
 }
 define i8 @test_i8_28_mask_lshr_4(i8 %a0) {
 ; X86-LABEL: test_i8_28_mask_lshr_4:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andb $16, %al
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    shrb $4, %al
+; X86-NEXT:    andb $1, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_28_mask_lshr_4:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andb $16, %al
 ; X64-NEXT:    shrb $4, %al
+; X64-NEXT:    andb $1, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 28
   %t1 = lshr i8 %t0, 4
   ret i8 %t1
 }
 
 define i8 @test_i8_224_mask_lshr_1(i8 %a0) {
 ; X86-LABEL: test_i8_224_mask_lshr_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    andb $-32, %al
 ; X86-NEXT:    shrb %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_224_mask_lshr_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andb $-32, %al
 ; X64-NEXT:    shrb %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 224
   %t1 = lshr i8 %t0, 1
   ret i8 %t1
 }
 define i8 @test_i8_224_mask_lshr_4(i8 %a0) {
 ; X86-LABEL: test_i8_224_mask_lshr_4:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andb $-32, %al
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    shrb $4, %al
+; X86-NEXT:    andb $14, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_224_mask_lshr_4:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andb $-32, %al
 ; X64-NEXT:    shrb $4, %al
+; X64-NEXT:    andb $14, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 224
   %t1 = lshr i8 %t0, 4
   ret i8 %t1
 }
 define i8 @test_i8_224_mask_lshr_5(i8 %a0) {
 ; X86-LABEL: test_i8_224_mask_lshr_5:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    shrb $5, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_224_mask_lshr_5:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shrb $5, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 224
   %t1 = lshr i8 %t0, 5
   ret i8 %t1
 }
 define i8 @test_i8_224_mask_lshr_6(i8 %a0) {
 ; X86-LABEL: test_i8_224_mask_lshr_6:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    shrb $6, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_224_mask_lshr_6:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shrb $6, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 224
   %t1 = lshr i8 %t0, 6
   ret i8 %t1
 }
 
 ; ashr
 
 define i8 @test_i8_7_mask_ashr_1(i8 %a0) {
 ; X86-LABEL: test_i8_7_mask_ashr_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andb $6, %al
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    shrb %al
+; X86-NEXT:    andb $3, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_7_mask_ashr_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andb $6, %al
 ; X64-NEXT:    shrb %al
+; X64-NEXT:    andb $3, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 7
   %t1 = ashr i8 %t0, 1
   ret i8 %t1
 }
 
 define i8 @test_i8_28_mask_ashr_1(i8 %a0) {
 ; X86-LABEL: test_i8_28_mask_ashr_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andb $28, %al
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    shrb %al
+; X86-NEXT:    andb $14, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_28_mask_ashr_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andb $28, %al
 ; X64-NEXT:    shrb %al
+; X64-NEXT:    andb $14, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 28
   %t1 = ashr i8 %t0, 1
   ret i8 %t1
 }
 define i8 @test_i8_28_mask_ashr_2(i8 %a0) {
 ; X86-LABEL: test_i8_28_mask_ashr_2:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andb $28, %al
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    shrb $2, %al
+; X86-NEXT:    andb $7, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_28_mask_ashr_2:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andb $28, %al
 ; X64-NEXT:    shrb $2, %al
+; X64-NEXT:    andb $7, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 28
   %t1 = ashr i8 %t0, 2
   ret i8 %t1
 }
 define i8 @test_i8_28_mask_ashr_3(i8 %a0) {
 ; X86-LABEL: test_i8_28_mask_ashr_3:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andb $24, %al
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    shrb $3, %al
+; X86-NEXT:    andb $3, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_28_mask_ashr_3:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andb $24, %al
 ; X64-NEXT:    shrb $3, %al
+; X64-NEXT:    andb $3, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 28
   %t1 = ashr i8 %t0, 3
   ret i8 %t1
 }
 define i8 @test_i8_28_mask_ashr_4(i8 %a0) {
 ; X86-LABEL: test_i8_28_mask_ashr_4:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andb $16, %al
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    shrb $4, %al
+; X86-NEXT:    andb $1, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_28_mask_ashr_4:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andb $16, %al
 ; X64-NEXT:    shrb $4, %al
+; X64-NEXT:    andb $1, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 28
   %t1 = ashr i8 %t0, 4
   ret i8 %t1
 }
 
 define i8 @test_i8_224_mask_ashr_1(i8 %a0) {
 ; X86-LABEL: test_i8_224_mask_ashr_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    andb $-32, %al
 ; X86-NEXT:    sarb %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_224_mask_ashr_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andb $-32, %al
 ; X64-NEXT:    sarb %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 224
   %t1 = ashr i8 %t0, 1
   ret i8 %t1
 }
 define i8 @test_i8_224_mask_ashr_4(i8 %a0) {
 ; X86-LABEL: test_i8_224_mask_ashr_4:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    andb $-32, %al
 ; X86-NEXT:    sarb $4, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_224_mask_ashr_4:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andb $-32, %al
 ; X64-NEXT:    sarb $4, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 224
   %t1 = ashr i8 %t0, 4
   ret i8 %t1
 }
 define i8 @test_i8_224_mask_ashr_5(i8 %a0) {
 ; X86-LABEL: test_i8_224_mask_ashr_5:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    sarb $5, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_224_mask_ashr_5:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    sarb $5, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 224
   %t1 = ashr i8 %t0, 5
   ret i8 %t1
 }
 define i8 @test_i8_224_mask_ashr_6(i8 %a0) {
 ; X86-LABEL: test_i8_224_mask_ashr_6:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    sarb $6, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_224_mask_ashr_6:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    sarb $6, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 224
   %t1 = ashr i8 %t0, 6
   ret i8 %t1
 }
 
 ; shl
 
 define i8 @test_i8_7_mask_shl_1(i8 %a0) {
 ; X86-LABEL: test_i8_7_mask_shl_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    andb $7, %al
 ; X86-NEXT:    addb %al, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_7_mask_shl_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    # kill: def $edi killed $edi def $rdi
 ; X64-NEXT:    andb $7, %dil
 ; X64-NEXT:    leal (%rdi,%rdi), %eax
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 7
   %t1 = shl i8 %t0, 1
   ret i8 %t1
 }
 define i8 @test_i8_7_mask_shl_4(i8 %a0) {
 ; X86-LABEL: test_i8_7_mask_shl_4:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    andb $7, %al
 ; X86-NEXT:    shlb $4, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_7_mask_shl_4:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andb $7, %al
 ; X64-NEXT:    shlb $4, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 7
   %t1 = shl i8 %t0, 4
   ret i8 %t1
 }
 define i8 @test_i8_7_mask_shl_5(i8 %a0) {
 ; X86-LABEL: test_i8_7_mask_shl_5:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    shlb $5, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_7_mask_shl_5:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shlb $5, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 7
   %t1 = shl i8 %t0, 5
   ret i8 %t1
 }
 define i8 @test_i8_7_mask_shl_6(i8 %a0) {
 ; X86-LABEL: test_i8_7_mask_shl_6:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    shlb $6, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_7_mask_shl_6:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shlb $6, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 7
   %t1 = shl i8 %t0, 6
   ret i8 %t1
 }
 
 define i8 @test_i8_28_mask_shl_1(i8 %a0) {
 ; X86-LABEL: test_i8_28_mask_shl_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    andb $28, %al
 ; X86-NEXT:    addb %al, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_28_mask_shl_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    # kill: def $edi killed $edi def $rdi
 ; X64-NEXT:    andb $28, %dil
 ; X64-NEXT:    leal (%rdi,%rdi), %eax
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 28
   %t1 = shl i8 %t0, 1
   ret i8 %t1
 }
 define i8 @test_i8_28_mask_shl_2(i8 %a0) {
 ; X86-LABEL: test_i8_28_mask_shl_2:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    andb $28, %al
 ; X86-NEXT:    shlb $2, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_28_mask_shl_2:
 ; X64:       # %bb.0:
 ; X64-NEXT:    # kill: def $edi killed $edi def $rdi
 ; X64-NEXT:    andb $28, %dil
 ; X64-NEXT:    leal (,%rdi,4), %eax
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 28
   %t1 = shl i8 %t0, 2
   ret i8 %t1
 }
 define i8 @test_i8_28_mask_shl_3(i8 %a0) {
 ; X86-LABEL: test_i8_28_mask_shl_3:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    andb $28, %al
 ; X86-NEXT:    shlb $3, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_28_mask_shl_3:
 ; X64:       # %bb.0:
 ; X64-NEXT:    # kill: def $edi killed $edi def $rdi
 ; X64-NEXT:    andb $28, %dil
 ; X64-NEXT:    leal (,%rdi,8), %eax
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 28
   %t1 = shl i8 %t0, 3
   ret i8 %t1
 }
 define i8 @test_i8_28_mask_shl_4(i8 %a0) {
 ; X86-LABEL: test_i8_28_mask_shl_4:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    andb $12, %al
 ; X86-NEXT:    shlb $4, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_28_mask_shl_4:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andb $12, %al
 ; X64-NEXT:    shlb $4, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %t0 = and i8 %a0, 28
   %t1 = shl i8 %t0, 4
   ret i8 %t1
 }
 
 define i8 @test_i8_224_mask_shl_1(i8 %a0) {
 ; X86-LABEL: test_i8_224_mask_shl_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzbl 4(%esp), %eax
 ; X86-NEXT:    andb $96, %al
 ; X86-NEXT:    addb %al, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i8_224_mask_shl_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    # kill: def $edi killed $edi def $rdi
 ; X64-NEXT:    andb $96, %dil

 ; 16-bit
 ;------------------------------------------------------------------------------;
 
 ; lshr
 
 define i16 @test_i16_127_mask_lshr_1(i16 %a0) {
 ; X86-LABEL: test_i16_127_mask_lshr_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andl $126, %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl %eax
+; X86-NEXT:    andl $63, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_127_mask_lshr_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andl $126, %eax
 ; X64-NEXT:    shrl %eax
+; X64-NEXT:    andl $63, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 127
   %t1 = lshr i16 %t0, 1
   ret i16 %t1
 }
 
 define i16 @test_i16_2032_mask_lshr_3(i16 %a0) {
 ; X86-LABEL: test_i16_2032_mask_lshr_3:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andl $2032, %eax # imm = 0x7F0
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $3, %eax
+; X86-NEXT:    andl $254, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_2032_mask_lshr_3:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andl $2032, %eax # imm = 0x7F0
 ; X64-NEXT:    shrl $3, %eax
+; X64-NEXT:    andl $254, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 2032
   %t1 = lshr i16 %t0, 3
   ret i16 %t1
 }
 define i16 @test_i16_2032_mask_lshr_4(i16 %a0) {
 ; X86-LABEL: test_i16_2032_mask_lshr_4:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $4, %eax
 ; X86-NEXT:    andl $127, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_2032_mask_lshr_4:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shrl $4, %eax
 ; X64-NEXT:    andl $127, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 2032
   %t1 = lshr i16 %t0, 4
   ret i16 %t1
 }
 define i16 @test_i16_2032_mask_lshr_5(i16 %a0) {
 ; X86-LABEL: test_i16_2032_mask_lshr_5:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $5, %eax
 ; X86-NEXT:    andl $63, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_2032_mask_lshr_5:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shrl $5, %eax
 ; X64-NEXT:    andl $63, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 2032
   %t1 = lshr i16 %t0, 5
   ret i16 %t1
 }
 define i16 @test_i16_2032_mask_lshr_6(i16 %a0) {
 ; X86-LABEL: test_i16_2032_mask_lshr_6:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $6, %eax
 ; X86-NEXT:    andl $31, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_2032_mask_lshr_6:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shrl $6, %eax
 ; X64-NEXT:    andl $31, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 2032
   %t1 = lshr i16 %t0, 6
   ret i16 %t1
 }
 
 define i16 @test_i16_65024_mask_lshr_1(i16 %a0) {
 ; X86-LABEL: test_i16_65024_mask_lshr_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andl $65024, %eax # imm = 0xFE00
+; X86-NEXT:    movzwl 4(%esp), %eax
 ; X86-NEXT:    shrl %eax
+; X86-NEXT:    andl $32512, %eax # imm = 0x7F00
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_65024_mask_lshr_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andl $65024, %eax # imm = 0xFE00
 ; X64-NEXT:    shrl %eax
+; X64-NEXT:    andl $32512, %eax # imm = 0x7F00
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 65024
   %t1 = lshr i16 %t0, 1
   ret i16 %t1
 }
+
 define i16 @test_i16_65024_mask_lshr_8(i16 %a0) {
 ; X86-LABEL: test_i16_65024_mask_lshr_8:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andl $65024, %eax # imm = 0xFE00
-; X86-NEXT:    shrl $8, %eax
+; X86-NEXT:    movzbl 5(%esp), %eax
+; X86-NEXT:    andl $-2, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_65024_mask_lshr_8:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andl $65024, %eax # imm = 0xFE00
 ; X64-NEXT:    shrl $8, %eax
+; X64-NEXT:    andl $254, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 65024
   %t1 = lshr i16 %t0, 8
   ret i16 %t1
 }
 define i16 @test_i16_65024_mask_lshr_9(i16 %a0) {
 ; X86-LABEL: test_i16_65024_mask_lshr_9:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzwl 4(%esp), %eax
 ; X86-NEXT:    shrl $9, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_65024_mask_lshr_9:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movzwl %di, %eax
 ; X64-NEXT:    shrl $9, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 65024
   %t1 = lshr i16 %t0, 9
   ret i16 %t1
 }
 define i16 @test_i16_65024_mask_lshr_10(i16 %a0) {
 ; X86-LABEL: test_i16_65024_mask_lshr_10:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzwl 4(%esp), %eax
 ; X86-NEXT:    shrl $10, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_65024_mask_lshr_10:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movzwl %di, %eax
 ; X64-NEXT:    shrl $10, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 65024
   %t1 = lshr i16 %t0, 10
   ret i16 %t1
 }
 
 ; ashr
 
 define i16 @test_i16_127_mask_ashr_1(i16 %a0) {
 ; X86-LABEL: test_i16_127_mask_ashr_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andl $126, %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl %eax
+; X86-NEXT:    andl $63, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_127_mask_ashr_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andl $126, %eax
 ; X64-NEXT:    shrl %eax
+; X64-NEXT:    andl $63, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 127
   %t1 = ashr i16 %t0, 1
   ret i16 %t1
 }
 
 define i16 @test_i16_2032_mask_ashr_3(i16 %a0) {
 ; X86-LABEL: test_i16_2032_mask_ashr_3:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andl $2032, %eax # imm = 0x7F0
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $3, %eax
+; X86-NEXT:    andl $254, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_2032_mask_ashr_3:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andl $2032, %eax # imm = 0x7F0
 ; X64-NEXT:    shrl $3, %eax
+; X64-NEXT:    andl $254, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 2032
   %t1 = ashr i16 %t0, 3
   ret i16 %t1
 }
 define i16 @test_i16_2032_mask_ashr_4(i16 %a0) {
 ; X86-LABEL: test_i16_2032_mask_ashr_4:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $4, %eax
 ; X86-NEXT:    andl $127, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_2032_mask_ashr_4:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shrl $4, %eax
 ; X64-NEXT:    andl $127, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 2032
   %t1 = ashr i16 %t0, 4
   ret i16 %t1
 }
 define i16 @test_i16_2032_mask_ashr_5(i16 %a0) {
 ; X86-LABEL: test_i16_2032_mask_ashr_5:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $5, %eax
 ; X86-NEXT:    andl $63, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_2032_mask_ashr_5:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shrl $5, %eax
 ; X64-NEXT:    andl $63, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 2032
   %t1 = ashr i16 %t0, 5
   ret i16 %t1
 }
 define i16 @test_i16_2032_mask_ashr_6(i16 %a0) {
 ; X86-LABEL: test_i16_2032_mask_ashr_6:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $6, %eax
 ; X86-NEXT:    andl $31, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_2032_mask_ashr_6:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shrl $6, %eax
 ; X64-NEXT:    andl $31, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 2032
   %t1 = ashr i16 %t0, 6
   ret i16 %t1
 }
 
 define i16 @test_i16_65024_mask_ashr_1(i16 %a0) {
 ; X86-LABEL: test_i16_65024_mask_ashr_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzwl 4(%esp), %eax
 ; X86-NEXT:    andl $65024, %eax # imm = 0xFE00
 ; X86-NEXT:    cwtl
 ; X86-NEXT:    shrl %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_65024_mask_ashr_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    andl $65024, %edi # imm = 0xFE00
 ; X64-NEXT:    movswl %di, %eax
 ; X64-NEXT:    shrl %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 65024
   %t1 = ashr i16 %t0, 1
   ret i16 %t1
 }
 define i16 @test_i16_65024_mask_ashr_8(i16 %a0) {
 ; X86-LABEL: test_i16_65024_mask_ashr_8:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzwl 4(%esp), %eax
 ; X86-NEXT:    andl $65024, %eax # imm = 0xFE00
 ; X86-NEXT:    cwtl
 ; X86-NEXT:    shrl $8, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_65024_mask_ashr_8:
 ; X64:       # %bb.0:
 ; X64-NEXT:    andl $65024, %edi # imm = 0xFE00
 ; X64-NEXT:    movswl %di, %eax
 ; X64-NEXT:    shrl $8, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 65024
   %t1 = ashr i16 %t0, 8
   ret i16 %t1
 }
 define i16 @test_i16_65024_mask_ashr_9(i16 %a0) {
 ; X86-LABEL: test_i16_65024_mask_ashr_9:
 ; X86:       # %bb.0:
-; X86-NEXT:    movswl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movswl 4(%esp), %eax
 ; X86-NEXT:    shrl $9, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_65024_mask_ashr_9:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movswl %di, %eax
 ; X64-NEXT:    shrl $9, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 65024
   %t1 = ashr i16 %t0, 9
   ret i16 %t1
 }
 define i16 @test_i16_65024_mask_ashr_10(i16 %a0) {
 ; X86-LABEL: test_i16_65024_mask_ashr_10:
 ; X86:       # %bb.0:
-; X86-NEXT:    movswl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movswl 4(%esp), %eax
 ; X86-NEXT:    shrl $10, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_65024_mask_ashr_10:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movswl %di, %eax
 ; X64-NEXT:    shrl $10, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 65024
   %t1 = ashr i16 %t0, 10
   ret i16 %t1
 }
 
 ; shl
 
 define i16 @test_i16_127_mask_shl_1(i16 %a0) {
 ; X86-LABEL: test_i16_127_mask_shl_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andl $127, %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    addl %eax, %eax
+; X86-NEXT:    movzbl %al, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_127_mask_shl_1:
 ; X64:       # %bb.0:
-; X64-NEXT:    # kill: def $edi killed $edi def $rdi
-; X64-NEXT:    andl $127, %edi
-; X64-NEXT:    leal (%rdi,%rdi), %eax
+; X64-NEXT:    addl %edi, %edi
+; X64-NEXT:    movzbl %dil, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 127
   %t1 = shl i16 %t0, 1
   ret i16 %t1
 }
 define i16 @test_i16_127_mask_shl_8(i16 %a0) {
 ; X86-LABEL: test_i16_127_mask_shl_8:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzwl 4(%esp), %eax
 ; X86-NEXT:    andl $127, %eax
 ; X86-NEXT:    shll $8, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_127_mask_shl_8:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andl $127, %eax
 ; X64-NEXT:    shll $8, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 127
   %t1 = shl i16 %t0, 8
   ret i16 %t1
 }
 define i16 @test_i16_127_mask_shl_9(i16 %a0) {
 ; X86-LABEL: test_i16_127_mask_shl_9:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shll $9, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_127_mask_shl_9:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shll $9, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 127
   %t1 = shl i16 %t0, 9
   ret i16 %t1
 }
 define i16 @test_i16_127_mask_shl_10(i16 %a0) {
 ; X86-LABEL: test_i16_127_mask_shl_10:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shll $10, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_127_mask_shl_10:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shll $10, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 127
   %t1 = shl i16 %t0, 10
   ret i16 %t1
 }
 
 define i16 @test_i16_2032_mask_shl_3(i16 %a0) {
 ; X86-LABEL: test_i16_2032_mask_shl_3:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzwl 4(%esp), %eax
 ; X86-NEXT:    andl $2032, %eax # imm = 0x7F0
 ; X86-NEXT:    shll $3, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_2032_mask_shl_3:
 ; X64:       # %bb.0:
 ; X64-NEXT:    # kill: def $edi killed $edi def $rdi
 ; X64-NEXT:    andl $2032, %edi # imm = 0x7F0
 ; X64-NEXT:    leal (,%rdi,8), %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 2032
   %t1 = shl i16 %t0, 3
   ret i16 %t1
 }
 define i16 @test_i16_2032_mask_shl_4(i16 %a0) {
 ; X86-LABEL: test_i16_2032_mask_shl_4:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzwl 4(%esp), %eax
 ; X86-NEXT:    andl $2032, %eax # imm = 0x7F0
 ; X86-NEXT:    shll $4, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_2032_mask_shl_4:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andl $2032, %eax # imm = 0x7F0
 ; X64-NEXT:    shll $4, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 2032
   %t1 = shl i16 %t0, 4
   ret i16 %t1
 }
 define i16 @test_i16_2032_mask_shl_5(i16 %a0) {
 ; X86-LABEL: test_i16_2032_mask_shl_5:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzwl 4(%esp), %eax
 ; X86-NEXT:    andl $2032, %eax # imm = 0x7F0
 ; X86-NEXT:    shll $5, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_2032_mask_shl_5:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andl $2032, %eax # imm = 0x7F0
 ; X64-NEXT:    shll $5, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 2032
   %t1 = shl i16 %t0, 5
   ret i16 %t1
 }
 define i16 @test_i16_2032_mask_shl_6(i16 %a0) {
 ; X86-LABEL: test_i16_2032_mask_shl_6:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzwl 4(%esp), %eax
 ; X86-NEXT:    andl $1008, %eax # imm = 0x3F0
 ; X86-NEXT:    shll $6, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_2032_mask_shl_6:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andl $1008, %eax # imm = 0x3F0
 ; X64-NEXT:    shll $6, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %t0 = and i16 %a0, 2032
   %t1 = shl i16 %t0, 6
   ret i16 %t1
 }
 
 define i16 @test_i16_65024_mask_shl_1(i16 %a0) {
 ; X86-LABEL: test_i16_65024_mask_shl_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzwl 4(%esp), %eax
 ; X86-NEXT:    andl $32256, %eax # imm = 0x7E00
 ; X86-NEXT:    addl %eax, %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_65024_mask_shl_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    # kill: def $edi killed $edi def $rdi

 ; 32-bit
 ;------------------------------------------------------------------------------;
 
 ; lshr
 
 define i32 @test_i32_32767_mask_lshr_1(i32 %a0) {
 ; X86-LABEL: test_i32_32767_mask_lshr_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl $32766, %eax # imm = 0x7FFE
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl %eax
+; X86-NEXT:    andl $16383, %eax # imm = 0x3FFF
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_32767_mask_lshr_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andl $32766, %eax # imm = 0x7FFE
 ; X64-NEXT:    shrl %eax
+; X64-NEXT:    andl $16383, %eax # imm = 0x3FFF
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 32767
   %t1 = lshr i32 %t0, 1
   ret i32 %t1
 }
 
 define i32 @test_i32_8388352_mask_lshr_7(i32 %a0) {
 ; X86-LABEL: test_i32_8388352_mask_lshr_7:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl $8388352, %eax # imm = 0x7FFF00
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $7, %eax
+; X86-NEXT:    andl $65534, %eax # imm = 0xFFFE
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_8388352_mask_lshr_7:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andl $8388352, %eax # imm = 0x7FFF00
 ; X64-NEXT:    shrl $7, %eax
+; X64-NEXT:    andl $65534, %eax # imm = 0xFFFE
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 8388352
   %t1 = lshr i32 %t0, 7
   ret i32 %t1
 }
 define i32 @test_i32_8388352_mask_lshr_8(i32 %a0) {
 ; X86-LABEL: test_i32_8388352_mask_lshr_8:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl $8388352, %eax # imm = 0x7FFF00
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $8, %eax
+; X86-NEXT:    andl $32767, %eax # imm = 0x7FFF
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_8388352_mask_lshr_8:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andl $8388352, %eax # imm = 0x7FFF00
 ; X64-NEXT:    shrl $8, %eax
+; X64-NEXT:    andl $32767, %eax # imm = 0x7FFF
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 8388352
   %t1 = lshr i32 %t0, 8
   ret i32 %t1
 }
 define i32 @test_i32_8388352_mask_lshr_9(i32 %a0) {
 ; X86-LABEL: test_i32_8388352_mask_lshr_9:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl $8388096, %eax # imm = 0x7FFE00
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $9, %eax
+; X86-NEXT:    andl $16383, %eax # imm = 0x3FFF
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_8388352_mask_lshr_9:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andl $8388096, %eax # imm = 0x7FFE00
 ; X64-NEXT:    shrl $9, %eax
+; X64-NEXT:    andl $16383, %eax # imm = 0x3FFF
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 8388352
   %t1 = lshr i32 %t0, 9
   ret i32 %t1
 }
 define i32 @test_i32_8388352_mask_lshr_10(i32 %a0) {
 ; X86-LABEL: test_i32_8388352_mask_lshr_10:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl $8387584, %eax # imm = 0x7FFC00
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $10, %eax
+; X86-NEXT:    andl $8191, %eax # imm = 0x1FFF
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_8388352_mask_lshr_10:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andl $8387584, %eax # imm = 0x7FFC00
 ; X64-NEXT:    shrl $10, %eax
+; X64-NEXT:    andl $8191, %eax # imm = 0x1FFF
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 8388352
   %t1 = lshr i32 %t0, 10
   ret i32 %t1
 }
 
 define i32 @test_i32_4294836224_mask_lshr_1(i32 %a0) {
 ; X86-LABEL: test_i32_4294836224_mask_lshr_1:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl $-131072, %eax # imm = 0xFFFE0000
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    andl 4(%esp), %eax
 ; X86-NEXT:    shrl %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_4294836224_mask_lshr_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andl $-131072, %eax # imm = 0xFFFE0000
 ; X64-NEXT:    shrl %eax
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 4294836224
   %t1 = lshr i32 %t0, 1
   ret i32 %t1
 }
+
+
 define i32 @test_i32_4294836224_mask_lshr_16(i32 %a0) {
 ; X86-LABEL: test_i32_4294836224_mask_lshr_16:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl $-131072, %eax # imm = 0xFFFE0000
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    shrl $16, %eax
+; X86-NEXT:    movzwl 6(%esp), %eax
+; X86-NEXT:    andl $-2, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_4294836224_mask_lshr_16:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andl $-131072, %eax # imm = 0xFFFE0000
 ; X64-NEXT:    shrl $16, %eax
+; X64-NEXT:    andl $-2, %eax
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 4294836224
   %t1 = lshr i32 %t0, 16
   ret i32 %t1
 }
 define i32 @test_i32_4294836224_mask_lshr_17(i32 %a0) {
 ; X86-LABEL: test_i32_4294836224_mask_lshr_17:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $17, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_4294836224_mask_lshr_17:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shrl $17, %eax
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 4294836224
   %t1 = lshr i32 %t0, 17
   ret i32 %t1
 }
 define i32 @test_i32_4294836224_mask_lshr_18(i32 %a0) {
 ; X86-LABEL: test_i32_4294836224_mask_lshr_18:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $18, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_4294836224_mask_lshr_18:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shrl $18, %eax
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 4294836224
   %t1 = lshr i32 %t0, 18
   ret i32 %t1
 }
 
 ; ashr
 
 define i32 @test_i32_32767_mask_ashr_1(i32 %a0) {
 ; X86-LABEL: test_i32_32767_mask_ashr_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl $32766, %eax # imm = 0x7FFE
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl %eax
+; X86-NEXT:    andl $16383, %eax # imm = 0x3FFF
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_32767_mask_ashr_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andl $32766, %eax # imm = 0x7FFE
 ; X64-NEXT:    shrl %eax
+; X64-NEXT:    andl $16383, %eax # imm = 0x3FFF
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 32767
   %t1 = ashr i32 %t0, 1
   ret i32 %t1
 }
 
 define i32 @test_i32_8388352_mask_ashr_7(i32 %a0) {
 ; X86-LABEL: test_i32_8388352_mask_ashr_7:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl $8388352, %eax # imm = 0x7FFF00
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $7, %eax
+; X86-NEXT:    andl $65534, %eax # imm = 0xFFFE
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_8388352_mask_ashr_7:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andl $8388352, %eax # imm = 0x7FFF00
 ; X64-NEXT:    shrl $7, %eax
+; X64-NEXT:    andl $65534, %eax # imm = 0xFFFE
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 8388352
   %t1 = ashr i32 %t0, 7
   ret i32 %t1
 }
 define i32 @test_i32_8388352_mask_ashr_8(i32 %a0) {
 ; X86-LABEL: test_i32_8388352_mask_ashr_8:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl $8388352, %eax # imm = 0x7FFF00
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $8, %eax
+; X86-NEXT:    andl $32767, %eax # imm = 0x7FFF
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_8388352_mask_ashr_8:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andl $8388352, %eax # imm = 0x7FFF00
 ; X64-NEXT:    shrl $8, %eax
+; X64-NEXT:    andl $32767, %eax # imm = 0x7FFF
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 8388352
   %t1 = ashr i32 %t0, 8
   ret i32 %t1
 }
 define i32 @test_i32_8388352_mask_ashr_9(i32 %a0) {
 ; X86-LABEL: test_i32_8388352_mask_ashr_9:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl $8388096, %eax # imm = 0x7FFE00
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $9, %eax
+; X86-NEXT:    andl $16383, %eax # imm = 0x3FFF
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_8388352_mask_ashr_9:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andl $8388096, %eax # imm = 0x7FFE00
 ; X64-NEXT:    shrl $9, %eax
+; X64-NEXT:    andl $16383, %eax # imm = 0x3FFF
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 8388352
   %t1 = ashr i32 %t0, 9
   ret i32 %t1
 }
 define i32 @test_i32_8388352_mask_ashr_10(i32 %a0) {
 ; X86-LABEL: test_i32_8388352_mask_ashr_10:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl $8387584, %eax # imm = 0x7FFC00
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $10, %eax
+; X86-NEXT:    andl $8191, %eax # imm = 0x1FFF
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_8388352_mask_ashr_10:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    andl $8387584, %eax # imm = 0x7FFC00
 ; X64-NEXT:    shrl $10, %eax
+; X64-NEXT:    andl $8191, %eax # imm = 0x1FFF
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 8388352
   %t1 = ashr i32 %t0, 10
   ret i32 %t1
 }
 
 define i32 @test_i32_4294836224_mask_ashr_1(i32 %a0) {
 ; X86-LABEL: test_i32_4294836224_mask_ashr_1:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl $-131072, %eax # imm = 0xFFFE0000
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    andl 4(%esp), %eax
 ; X86-NEXT:    sarl %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_4294836224_mask_ashr_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andl $-131072, %eax # imm = 0xFFFE0000
 ; X64-NEXT:    sarl %eax
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 4294836224
   %t1 = ashr i32 %t0, 1
   ret i32 %t1
 }
 define i32 @test_i32_4294836224_mask_ashr_16(i32 %a0) {
 ; X86-LABEL: test_i32_4294836224_mask_ashr_16:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl $-131072, %eax # imm = 0xFFFE0000
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    andl 4(%esp), %eax
 ; X86-NEXT:    sarl $16, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_4294836224_mask_ashr_16:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andl $-131072, %eax # imm = 0xFFFE0000
 ; X64-NEXT:    sarl $16, %eax
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 4294836224
   %t1 = ashr i32 %t0, 16
   ret i32 %t1
 }
 define i32 @test_i32_4294836224_mask_ashr_17(i32 %a0) {
 ; X86-LABEL: test_i32_4294836224_mask_ashr_17:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    sarl $17, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_4294836224_mask_ashr_17:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    sarl $17, %eax
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 4294836224
   %t1 = ashr i32 %t0, 17
   ret i32 %t1
 }
 define i32 @test_i32_4294836224_mask_ashr_18(i32 %a0) {
 ; X86-LABEL: test_i32_4294836224_mask_ashr_18:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    sarl $18, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_4294836224_mask_ashr_18:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    sarl $18, %eax
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 4294836224
   %t1 = ashr i32 %t0, 18
   ret i32 %t1
 }
 
 ; shl
 
 define i32 @test_i32_32767_mask_shl_1(i32 %a0) {
 ; X86-LABEL: test_i32_32767_mask_shl_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl $32767, %eax # imm = 0x7FFF
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    addl %eax, %eax
+; X86-NEXT:    movzwl %ax, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_32767_mask_shl_1:
 ; X64:       # %bb.0:
-; X64-NEXT:    # kill: def $edi killed $edi def $rdi
-; X64-NEXT:    andl $32767, %edi # imm = 0x7FFF
-; X64-NEXT:    leal (%rdi,%rdi), %eax
+; X64-NEXT:    addl %edi, %edi
+; X64-NEXT:    movzwl %di, %eax
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 32767
   %t1 = shl i32 %t0, 1
   ret i32 %t1
 }
 define i32 @test_i32_32767_mask_shl_16(i32 %a0) {
 ; X86-LABEL: test_i32_32767_mask_shl_16:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl $32767, %eax # imm = 0x7FFF
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    andl 4(%esp), %eax
 ; X86-NEXT:    shll $16, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_32767_mask_shl_16:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andl $32767, %eax # imm = 0x7FFF
 ; X64-NEXT:    shll $16, %eax
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 32767
   %t1 = shl i32 %t0, 16
   ret i32 %t1
 }
 define i32 @test_i32_32767_mask_shl_17(i32 %a0) {
 ; X86-LABEL: test_i32_32767_mask_shl_17:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shll $17, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_32767_mask_shl_17:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shll $17, %eax
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 32767
   %t1 = shl i32 %t0, 17
   ret i32 %t1
 }
 define i32 @test_i32_32767_mask_shl_18(i32 %a0) {
 ; X86-LABEL: test_i32_32767_mask_shl_18:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shll $18, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_32767_mask_shl_18:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shll $18, %eax
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 32767
   %t1 = shl i32 %t0, 18
   ret i32 %t1
 }
 
 define i32 @test_i32_8388352_mask_shl_7(i32 %a0) {
 ; X86-LABEL: test_i32_8388352_mask_shl_7:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl $8388352, %eax # imm = 0x7FFF00
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    andl 4(%esp), %eax
 ; X86-NEXT:    shll $7, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_8388352_mask_shl_7:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andl $8388352, %eax # imm = 0x7FFF00
 ; X64-NEXT:    shll $7, %eax
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 8388352
   %t1 = shl i32 %t0, 7
   ret i32 %t1
 }
 define i32 @test_i32_8388352_mask_shl_8(i32 %a0) {
 ; X86-LABEL: test_i32_8388352_mask_shl_8:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl $8388352, %eax # imm = 0x7FFF00
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    andl 4(%esp), %eax
 ; X86-NEXT:    shll $8, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_8388352_mask_shl_8:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andl $8388352, %eax # imm = 0x7FFF00
 ; X64-NEXT:    shll $8, %eax
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 8388352
   %t1 = shl i32 %t0, 8
   ret i32 %t1
 }
 define i32 @test_i32_8388352_mask_shl_9(i32 %a0) {
 ; X86-LABEL: test_i32_8388352_mask_shl_9:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl $8388352, %eax # imm = 0x7FFF00
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    andl 4(%esp), %eax
 ; X86-NEXT:    shll $9, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_8388352_mask_shl_9:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andl $8388352, %eax # imm = 0x7FFF00
 ; X64-NEXT:    shll $9, %eax
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 8388352
   %t1 = shl i32 %t0, 9
   ret i32 %t1
 }
 define i32 @test_i32_8388352_mask_shl_10(i32 %a0) {
 ; X86-LABEL: test_i32_8388352_mask_shl_10:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl $4194048, %eax # imm = 0x3FFF00
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    andl 4(%esp), %eax
 ; X86-NEXT:    shll $10, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_8388352_mask_shl_10:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andl $4194048, %eax # imm = 0x3FFF00
 ; X64-NEXT:    shll $10, %eax
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 8388352
   %t1 = shl i32 %t0, 10
   ret i32 %t1
 }
 
 define i32 @test_i32_4294836224_mask_shl_1(i32 %a0) {
 ; X86-LABEL: test_i32_4294836224_mask_shl_1:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl $2147352576, %eax # imm = 0x7FFE0000
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    andl 4(%esp), %eax
 ; X86-NEXT:    addl %eax, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i32_4294836224_mask_shl_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    # kill: def $edi killed $edi def $rdi
 ; X64-NEXT:    andl $2147352576, %edi # imm = 0x7FFE0000
 ; X64-NEXT:    leal (%rdi,%rdi), %eax
 ; X64-NEXT:    retq
   %t0 = and i32 %a0, 4294836224
   %t1 = shl i32 %t0, 1
   ret i32 %t1
 }
 
 ;------------------------------------------------------------------------------;
 ; 64-bit
 ;------------------------------------------------------------------------------;
 
 ; lshr
 
 define i64 @test_i64_2147483647_mask_lshr_1(i64 %a0) {
 ; X86-LABEL: test_i64_2147483647_mask_lshr_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl $2147483646, %eax # imm = 0x7FFFFFFE
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl %eax
+; X86-NEXT:    andl $1073741823, %eax # imm = 0x3FFFFFFF
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_2147483647_mask_lshr_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
-; X64-NEXT:    andl $2147483646, %eax # imm = 0x7FFFFFFE
-; X64-NEXT:    shrq %rax
+; X64-NEXT:    shrl %eax
+; X64-NEXT:    andl $1073741823, %eax # imm = 0x3FFFFFFF
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 2147483647
   %t1 = lshr i64 %t0, 1
   ret i64 %t1
 }
 
 define i64 @test_i64_140737488289792_mask_lshr_15(i64 %a0) {
 ; X86-LABEL: test_i64_140737488289792_mask_lshr_15:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT:    movzwl 6(%esp), %ecx
 ; X86-NEXT:    shll $16, %ecx
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 8(%esp), %eax
 ; X86-NEXT:    shldl $17, %ecx, %eax
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_140737488289792_mask_lshr_15:
 ; X64:       # %bb.0:
-; X64-NEXT:    movabsq $140737488289792, %rax # imm = 0x7FFFFFFF0000
-; X64-NEXT:    andq %rdi, %rax
+; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    shrq $15, %rax
+; X64-NEXT:    andl $-2, %eax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 140737488289792
   %t1 = lshr i64 %t0, 15
   ret i64 %t1
 }
 define i64 @test_i64_140737488289792_mask_lshr_16(i64 %a0) {
 ; X86-LABEL: test_i64_140737488289792_mask_lshr_16:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT:    movl 4(%esp), %ecx
 ; X86-NEXT:    movl $32767, %eax # imm = 0x7FFF
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    andl 8(%esp), %eax
 ; X86-NEXT:    shldl $16, %ecx, %eax
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_140737488289792_mask_lshr_16:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    shrq $16, %rax
 ; X64-NEXT:    andl $2147483647, %eax # imm = 0x7FFFFFFF
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 140737488289792
   %t1 = lshr i64 %t0, 16
   ret i64 %t1
 }
 define i64 @test_i64_140737488289792_mask_lshr_17(i64 %a0) {
 ; X86-LABEL: test_i64_140737488289792_mask_lshr_17:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT:    movl 4(%esp), %ecx
 ; X86-NEXT:    movl $32767, %eax # imm = 0x7FFF
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    andl 8(%esp), %eax
 ; X86-NEXT:    shldl $15, %ecx, %eax
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_140737488289792_mask_lshr_17:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    shrq $17, %rax
 ; X64-NEXT:    andl $1073741823, %eax # imm = 0x3FFFFFFF
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 140737488289792
   %t1 = lshr i64 %t0, 17
   ret i64 %t1
 }
 define i64 @test_i64_140737488289792_mask_lshr_18(i64 %a0) {
 ; X86-LABEL: test_i64_140737488289792_mask_lshr_18:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT:    movl 4(%esp), %ecx
 ; X86-NEXT:    movl $32767, %eax # imm = 0x7FFF
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    andl 8(%esp), %eax
 ; X86-NEXT:    shldl $14, %ecx, %eax
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_140737488289792_mask_lshr_18:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    shrq $18, %rax
 ; X64-NEXT:    andl $536870911, %eax # imm = 0x1FFFFFFF
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 140737488289792
   %t1 = lshr i64 %t0, 18
   ret i64 %t1
 }
 
 define i64 @test_i64_18446744065119617024_mask_lshr_1(i64 %a0) {
 ; X86-LABEL: test_i64_18446744065119617024_mask_lshr_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    movl 8(%esp), %edx
 ; X86-NEXT:    shrl %edx
 ; X86-NEXT:    xorl %eax, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_18446744065119617024_mask_lshr_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movabsq $-8589934592, %rax # imm = 0xFFFFFFFE00000000
 ; X64-NEXT:    andq %rdi, %rax
 ; X64-NEXT:    shrq %rax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 18446744065119617024
   %t1 = lshr i64 %t0, 1
   ret i64 %t1
 }
 define i64 @test_i64_18446744065119617024_mask_lshr_32(i64 %a0) {
 ; X86-LABEL: test_i64_18446744065119617024_mask_lshr_32:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 8(%esp), %eax
 ; X86-NEXT:    andl $-2, %eax
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_18446744065119617024_mask_lshr_32:
 ; X64:       # %bb.0:
-; X64-NEXT:    movabsq $-8589934592, %rax # imm = 0xFFFFFFFE00000000
-; X64-NEXT:    andq %rdi, %rax
+; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    shrq $32, %rax
+; X64-NEXT:    andl $-2, %eax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 18446744065119617024
   %t1 = lshr i64 %t0, 32
   ret i64 %t1
 }
 define i64 @test_i64_18446744065119617024_mask_lshr_33(i64 %a0) {
 ; X86-LABEL: test_i64_18446744065119617024_mask_lshr_33:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 8(%esp), %eax
 ; X86-NEXT:    shrl %eax
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_18446744065119617024_mask_lshr_33:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    shrq $33, %rax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 18446744065119617024
   %t1 = lshr i64 %t0, 33
   ret i64 %t1
 }
 define i64 @test_i64_18446744065119617024_mask_lshr_34(i64 %a0) {
 ; X86-LABEL: test_i64_18446744065119617024_mask_lshr_34:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 8(%esp), %eax
 ; X86-NEXT:    shrl $2, %eax
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_18446744065119617024_mask_lshr_34:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    shrq $34, %rax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 18446744065119617024
   %t1 = lshr i64 %t0, 34
   ret i64 %t1
 }
 
 ; ashr
 
 define i64 @test_i64_2147483647_mask_ashr_1(i64 %a0) {
 ; X86-LABEL: test_i64_2147483647_mask_ashr_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl $2147483646, %eax # imm = 0x7FFFFFFE
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl %eax
+; X86-NEXT:    andl $1073741823, %eax # imm = 0x3FFFFFFF
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_2147483647_mask_ashr_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
-; X64-NEXT:    andl $2147483646, %eax # imm = 0x7FFFFFFE
-; X64-NEXT:    shrq %rax
+; X64-NEXT:    shrl %eax
+; X64-NEXT:    andl $1073741823, %eax # imm = 0x3FFFFFFF
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 2147483647
   %t1 = ashr i64 %t0, 1
   ret i64 %t1
 }
 
 define i64 @test_i64_140737488289792_mask_ashr_15(i64 %a0) {
 ; X86-LABEL: test_i64_140737488289792_mask_ashr_15:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT:    movzwl 6(%esp), %ecx
 ; X86-NEXT:    shll $16, %ecx
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 8(%esp), %eax
 ; X86-NEXT:    shldl $17, %ecx, %eax
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_140737488289792_mask_ashr_15:
 ; X64:       # %bb.0:
-; X64-NEXT:    movabsq $140737488289792, %rax # imm = 0x7FFFFFFF0000
-; X64-NEXT:    andq %rdi, %rax
+; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    shrq $15, %rax
+; X64-NEXT:    andl $-2, %eax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 140737488289792
   %t1 = ashr i64 %t0, 15
   ret i64 %t1
 }
 define i64 @test_i64_140737488289792_mask_ashr_16(i64 %a0) {
 ; X86-LABEL: test_i64_140737488289792_mask_ashr_16:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT:    movl 4(%esp), %ecx
 ; X86-NEXT:    movl $32767, %eax # imm = 0x7FFF
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    andl 8(%esp), %eax
 ; X86-NEXT:    shldl $16, %ecx, %eax
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_140737488289792_mask_ashr_16:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    shrq $16, %rax
 ; X64-NEXT:    andl $2147483647, %eax # imm = 0x7FFFFFFF
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 140737488289792
   %t1 = ashr i64 %t0, 16
   ret i64 %t1
 }
 define i64 @test_i64_140737488289792_mask_ashr_17(i64 %a0) {
 ; X86-LABEL: test_i64_140737488289792_mask_ashr_17:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT:    movl 4(%esp), %ecx
 ; X86-NEXT:    movl $32767, %eax # imm = 0x7FFF
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    andl 8(%esp), %eax
 ; X86-NEXT:    shldl $15, %ecx, %eax
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_140737488289792_mask_ashr_17:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    shrq $17, %rax
 ; X64-NEXT:    andl $1073741823, %eax # imm = 0x3FFFFFFF
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 140737488289792
   %t1 = ashr i64 %t0, 17
   ret i64 %t1
 }
 define i64 @test_i64_140737488289792_mask_ashr_18(i64 %a0) {
 ; X86-LABEL: test_i64_140737488289792_mask_ashr_18:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT:    movl 4(%esp), %ecx
 ; X86-NEXT:    movl $32767, %eax # imm = 0x7FFF
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    andl 8(%esp), %eax
 ; X86-NEXT:    shldl $14, %ecx, %eax
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_140737488289792_mask_ashr_18:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    shrq $18, %rax
 ; X64-NEXT:    andl $536870911, %eax # imm = 0x1FFFFFFF
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 140737488289792
   %t1 = ashr i64 %t0, 18
   ret i64 %t1
 }
 
 define i64 @test_i64_18446744065119617024_mask_ashr_1(i64 %a0) {
 ; X86-LABEL: test_i64_18446744065119617024_mask_ashr_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    movl 8(%esp), %edx
 ; X86-NEXT:    sarl %edx
 ; X86-NEXT:    xorl %eax, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_18446744065119617024_mask_ashr_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movabsq $-8589934592, %rax # imm = 0xFFFFFFFE00000000
 ; X64-NEXT:    andq %rdi, %rax
 ; X64-NEXT:    sarq %rax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 18446744065119617024
   %t1 = ashr i64 %t0, 1
   ret i64 %t1
 }
 define i64 @test_i64_18446744065119617024_mask_ashr_32(i64 %a0) {
 ; X86-LABEL: test_i64_18446744065119617024_mask_ashr_32:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    movl 8(%esp), %edx
 ; X86-NEXT:    movl %edx, %eax
 ; X86-NEXT:    andl $-2, %eax
 ; X86-NEXT:    sarl $31, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_18446744065119617024_mask_ashr_32:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movabsq $-8589934592, %rax # imm = 0xFFFFFFFE00000000
 ; X64-NEXT:    andq %rdi, %rax
 ; X64-NEXT:    sarq $32, %rax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 18446744065119617024
   %t1 = ashr i64 %t0, 32
   ret i64 %t1
 }
 define i64 @test_i64_18446744065119617024_mask_ashr_33(i64 %a0) {
 ; X86-LABEL: test_i64_18446744065119617024_mask_ashr_33:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    movl 8(%esp), %edx
 ; X86-NEXT:    movl %edx, %eax
 ; X86-NEXT:    sarl %eax
 ; X86-NEXT:    sarl $31, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_18446744065119617024_mask_ashr_33:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    sarq $33, %rax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 18446744065119617024
   %t1 = ashr i64 %t0, 33
   ret i64 %t1
 }
 define i64 @test_i64_18446744065119617024_mask_ashr_34(i64 %a0) {
 ; X86-LABEL: test_i64_18446744065119617024_mask_ashr_34:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    movl 8(%esp), %edx
 ; X86-NEXT:    movl %edx, %eax
 ; X86-NEXT:    sarl $2, %eax
 ; X86-NEXT:    sarl $31, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_18446744065119617024_mask_ashr_34:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    sarq $34, %rax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 18446744065119617024
   %t1 = ashr i64 %t0, 34
   ret i64 %t1
 }
 
 ; shl
 
 define i64 @test_i64_2147483647_mask_shl_1(i64 %a0) {
 ; X86-LABEL: test_i64_2147483647_mask_shl_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    addl %eax, %eax
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_2147483647_mask_shl_1:
 ; X64:       # %bb.0:
-; X64-NEXT:    andl $2147483647, %edi # imm = 0x7FFFFFFF
-; X64-NEXT:    leaq (%rdi,%rdi), %rax
+; X64-NEXT:    leal (%rdi,%rdi), %eax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 2147483647
   %t1 = shl i64 %t0, 1
   ret i64 %t1
 }
 define i64 @test_i64_2147483647_mask_shl_32(i64 %a0) {
 ; X86-LABEL: test_i64_2147483647_mask_shl_32:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl $2147483647, %edx # imm = 0x7FFFFFFF
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    andl 4(%esp), %edx
 ; X86-NEXT:    xorl %eax, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_2147483647_mask_shl_32:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    andl $2147483647, %eax # imm = 0x7FFFFFFF
 ; X64-NEXT:    shlq $32, %rax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 2147483647
   %t1 = shl i64 %t0, 32
   ret i64 %t1
 }
 define i64 @test_i64_2147483647_mask_shl_33(i64 %a0) {
 ; X86-LABEL: test_i64_2147483647_mask_shl_33:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    movl 4(%esp), %edx
 ; X86-NEXT:    addl %edx, %edx
 ; X86-NEXT:    xorl %eax, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_2147483647_mask_shl_33:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    shlq $33, %rax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 2147483647
   %t1 = shl i64 %t0, 33
   ret i64 %t1
 }
 define i64 @test_i64_2147483647_mask_shl_34(i64 %a0) {
 ; X86-LABEL: test_i64_2147483647_mask_shl_34:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    movl 4(%esp), %edx
 ; X86-NEXT:    shll $2, %edx
 ; X86-NEXT:    xorl %eax, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_2147483647_mask_shl_34:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    shlq $34, %rax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 2147483647
   %t1 = shl i64 %t0, 34
   ret i64 %t1
 }
 
 define i64 @test_i64_140737488289792_mask_shl_15(i64 %a0) {
 ; X86-LABEL: test_i64_140737488289792_mask_shl_15:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    movl $32767, %edx # imm = 0x7FFF
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    andl 8(%esp), %edx
 ; X86-NEXT:    shldl $15, %eax, %edx
 ; X86-NEXT:    andl $65536, %eax # imm = 0x10000
 ; X86-NEXT:    shll $15, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_140737488289792_mask_shl_15:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movabsq $140737488289792, %rax # imm = 0x7FFFFFFF0000
 ; X64-NEXT:    andq %rdi, %rax
 ; X64-NEXT:    shlq $15, %rax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 140737488289792
   %t1 = shl i64 %t0, 15
   ret i64 %t1
 }
 define i64 @test_i64_140737488289792_mask_shl_16(i64 %a0) {
 ; X86-LABEL: test_i64_140737488289792_mask_shl_16:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    movl $32767, %edx # imm = 0x7FFF
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    andl 8(%esp), %edx
 ; X86-NEXT:    shldl $16, %eax, %edx
 ; X86-NEXT:    xorl %eax, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_140737488289792_mask_shl_16:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movabsq $140737488289792, %rax # imm = 0x7FFFFFFF0000
 ; X64-NEXT:    andq %rdi, %rax
 ; X64-NEXT:    shlq $16, %rax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 140737488289792
   %t1 = shl i64 %t0, 16
   ret i64 %t1
 }
 define i64 @test_i64_140737488289792_mask_shl_17(i64 %a0) {
 ; X86-LABEL: test_i64_140737488289792_mask_shl_17:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzwl 6(%esp), %eax
 ; X86-NEXT:    shll $16, %eax
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    movl 8(%esp), %edx
 ; X86-NEXT:    shldl $17, %eax, %edx
 ; X86-NEXT:    xorl %eax, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_140737488289792_mask_shl_17:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movabsq $140737488289792, %rax # imm = 0x7FFFFFFF0000
 ; X64-NEXT:    andq %rdi, %rax
 ; X64-NEXT:    shlq $17, %rax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 140737488289792
   %t1 = shl i64 %t0, 17
   ret i64 %t1
 }
 define i64 @test_i64_140737488289792_mask_shl_18(i64 %a0) {
 ; X86-LABEL: test_i64_140737488289792_mask_shl_18:
 ; X86:       # %bb.0:
-; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzwl 6(%esp), %eax
 ; X86-NEXT:    shll $16, %eax
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    movl 8(%esp), %edx
 ; X86-NEXT:    shldl $18, %eax, %edx
 ; X86-NEXT:    xorl %eax, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_140737488289792_mask_shl_18:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movabsq $70368744112128, %rax # imm = 0x3FFFFFFF0000
 ; X64-NEXT:    andq %rdi, %rax
 ; X64-NEXT:    shlq $18, %rax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 140737488289792
   %t1 = shl i64 %t0, 18
   ret i64 %t1
 }
 
 define i64 @test_i64_18446744065119617024_mask_shl_1(i64 %a0) {
 ; X86-LABEL: test_i64_18446744065119617024_mask_shl_1:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl $2147483646, %edx # imm = 0x7FFFFFFE
-; X86-NEXT:    andl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    andl 8(%esp), %edx
 ; X86-NEXT:    addl %edx, %edx
 ; X86-NEXT:    xorl %eax, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i64_18446744065119617024_mask_shl_1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movabsq $9223372028264841216, %rax # imm = 0x7FFFFFFE00000000
 ; X64-NEXT:    andq %rdi, %rax
 ; X64-NEXT:    addq %rax, %rax
 ; X64-NEXT:    retq
   %t0 = and i64 %a0, 18446744065119617024
   %t1 = shl i64 %t0, 1
   ret i64 %t1
 }

llvm/test/CodeGen/X86/fold-and-shift.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=i686-- | FileCheck %s
 
 define i32 @t1(ptr %X, i32 %i) {
 ; CHECK-LABEL: t1:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; CHECK-NEXT:    movzbl %cl, %ecx
-; CHECK-NEXT:    movl (%eax,%ecx,4), %eax
+; CHECK-NEXT:    movl (%ecx,%eax,4), %eax
 ; CHECK-NEXT:    retl
 
 entry:
   %tmp2 = shl i32 %i, 2
   %tmp4 = and i32 %tmp2, 1020
   %tmp7 = getelementptr i8, ptr %X, i32 %tmp4
   %tmp9 = load i32, ptr %tmp7
   ret i32 %tmp9
 }
 
 define i32 @t2(ptr %X, i32 %i) {
 ; CHECK-LABEL: t2:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; CHECK-NEXT:    movzwl %cx, %ecx
-; CHECK-NEXT:    movl (%eax,%ecx,4), %eax
+; CHECK-NEXT:    movzwl {{[0-9]+}}(%esp), %ecx
+; CHECK-NEXT:    addl %ecx, %ecx
+; CHECK-NEXT:    movl (%eax,%ecx,2), %eax
 ; CHECK-NEXT:    retl
 
 entry:
   %tmp2 = shl i32 %i, 1
   %tmp4 = and i32 %tmp2, 131070
   %tmp7 = getelementptr i16, ptr %X, i32 %tmp4
   %tmp9 = load i32, ptr %tmp7
   ret i32 %tmp9

llvm/test/CodeGen/X86/limited-prec.ll

 ; precision6-LABEL: f4:
 ; precision6:       # %bb.0: # %entry
 ; precision6-NEXT:    subl $8, %esp
 ; precision6-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; precision6-NEXT:    movl %eax, %ecx
 ; precision6-NEXT:    andl $8388607, %ecx # imm = 0x7FFFFF
 ; precision6-NEXT:    orl $1065353216, %ecx # imm = 0x3F800000
 ; precision6-NEXT:    movl %ecx, (%esp)
-; precision6-NEXT:    andl $2139095040, %eax # imm = 0x7F800000
 ; precision6-NEXT:    shrl $23, %eax
+; precision6-NEXT:    movzbl %al, %eax
 ; precision6-NEXT:    addl $-127, %eax
 ; precision6-NEXT:    movl %eax, {{[0-9]+}}(%esp)
 ; precision6-NEXT:    flds (%esp)
 ; precision6-NEXT:    fld %st(0)
 ; precision6-NEXT:    fmuls {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision6-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision6-NEXT:    fmulp %st, %st(1)
 ; precision6-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision6-NEXT:    fildl {{[0-9]+}}(%esp)
 ; precision6-NEXT:    fmuls {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision6-NEXT:    faddp %st, %st(1)
 ; precision6-NEXT:    addl $8, %esp
 ; precision6-NEXT:    retl
 ;
 ; precision12-LABEL: f4:
 ; precision12:       # %bb.0: # %entry
 ; precision12-NEXT:    subl $8, %esp
 ; precision12-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; precision12-NEXT:    movl %eax, %ecx
 ; precision12-NEXT:    andl $8388607, %ecx # imm = 0x7FFFFF
 ; precision12-NEXT:    orl $1065353216, %ecx # imm = 0x3F800000
 ; precision12-NEXT:    movl %ecx, (%esp)
-; precision12-NEXT:    andl $2139095040, %eax # imm = 0x7F800000
 ; precision12-NEXT:    shrl $23, %eax
+; precision12-NEXT:    movzbl %al, %eax
 ; precision12-NEXT:    addl $-127, %eax
 ; precision12-NEXT:    movl %eax, {{[0-9]+}}(%esp)
 ; precision12-NEXT:    flds (%esp)
 ; precision12-NEXT:    fld %st(0)
 ; precision12-NEXT:    fmuls {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision12-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision12-NEXT:    fmul %st(1), %st
 ; precision12-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}

 ; precision18-LABEL: f4:
 ; precision18:       # %bb.0: # %entry
 ; precision18-NEXT:    subl $8, %esp
 ; precision18-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; precision18-NEXT:    movl %eax, %ecx
 ; precision18-NEXT:    andl $8388607, %ecx # imm = 0x7FFFFF
 ; precision18-NEXT:    orl $1065353216, %ecx # imm = 0x3F800000
 ; precision18-NEXT:    movl %ecx, (%esp)
-; precision18-NEXT:    andl $2139095040, %eax # imm = 0x7F800000
 ; precision18-NEXT:    shrl $23, %eax
+; precision18-NEXT:    movzbl %al, %eax
 ; precision18-NEXT:    addl $-127, %eax
 ; precision18-NEXT:    movl %eax, {{[0-9]+}}(%esp)
 ; precision18-NEXT:    flds (%esp)
 ; precision18-NEXT:    fld %st(0)
 ; precision18-NEXT:    fmuls {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision18-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision18-NEXT:    fmul %st(1), %st
 ; precision18-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}

 ; precision6-LABEL: f5:
 ; precision6:       # %bb.0: # %entry
 ; precision6-NEXT:    subl $8, %esp
 ; precision6-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; precision6-NEXT:    movl %eax, %ecx
 ; precision6-NEXT:    andl $8388607, %ecx # imm = 0x7FFFFF
 ; precision6-NEXT:    orl $1065353216, %ecx # imm = 0x3F800000
 ; precision6-NEXT:    movl %ecx, (%esp)
-; precision6-NEXT:    andl $2139095040, %eax # imm = 0x7F800000
 ; precision6-NEXT:    shrl $23, %eax
+; precision6-NEXT:    movzbl %al, %eax
 ; precision6-NEXT:    addl $-127, %eax
 ; precision6-NEXT:    movl %eax, {{[0-9]+}}(%esp)
 ; precision6-NEXT:    flds (%esp)
 ; precision6-NEXT:    fld %st(0)
 ; precision6-NEXT:    fmuls {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision6-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision6-NEXT:    fmulp %st, %st(1)
 ; precision6-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision6-NEXT:    fiaddl {{[0-9]+}}(%esp)
 ; precision6-NEXT:    addl $8, %esp
 ; precision6-NEXT:    retl
 ;
 ; precision12-LABEL: f5:
 ; precision12:       # %bb.0: # %entry
 ; precision12-NEXT:    subl $8, %esp
 ; precision12-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; precision12-NEXT:    movl %eax, %ecx
 ; precision12-NEXT:    andl $8388607, %ecx # imm = 0x7FFFFF
 ; precision12-NEXT:    orl $1065353216, %ecx # imm = 0x3F800000
 ; precision12-NEXT:    movl %ecx, (%esp)
-; precision12-NEXT:    andl $2139095040, %eax # imm = 0x7F800000
 ; precision12-NEXT:    shrl $23, %eax
+; precision12-NEXT:    movzbl %al, %eax
 ; precision12-NEXT:    addl $-127, %eax
 ; precision12-NEXT:    movl %eax, {{[0-9]+}}(%esp)
 ; precision12-NEXT:    flds (%esp)
 ; precision12-NEXT:    fld %st(0)
 ; precision12-NEXT:    fmuls {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision12-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision12-NEXT:    fmul %st(1), %st
 ; precision12-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision12-NEXT:    fmul %st(1), %st
 ; precision12-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision12-NEXT:    fmulp %st, %st(1)
 ; precision12-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision12-NEXT:    fiaddl {{[0-9]+}}(%esp)
 ; precision12-NEXT:    addl $8, %esp
 ; precision12-NEXT:    retl
 ;
 ; precision18-LABEL: f5:
 ; precision18:       # %bb.0: # %entry
 ; precision18-NEXT:    subl $8, %esp
 ; precision18-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; precision18-NEXT:    movl %eax, %ecx
 ; precision18-NEXT:    andl $8388607, %ecx # imm = 0x7FFFFF
 ; precision18-NEXT:    orl $1065353216, %ecx # imm = 0x3F800000
 ; precision18-NEXT:    movl %ecx, (%esp)
-; precision18-NEXT:    andl $2139095040, %eax # imm = 0x7F800000
 ; precision18-NEXT:    shrl $23, %eax
+; precision18-NEXT:    movzbl %al, %eax
 ; precision18-NEXT:    addl $-127, %eax
 ; precision18-NEXT:    movl %eax, {{[0-9]+}}(%esp)
 ; precision18-NEXT:    flds (%esp)
 ; precision18-NEXT:    fld %st(0)
 ; precision18-NEXT:    fmuls {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision18-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision18-NEXT:    fmul %st(1), %st
 ; precision18-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}

 ; precision6-LABEL: f6:
 ; precision6:       # %bb.0: # %entry
 ; precision6-NEXT:    subl $8, %esp
 ; precision6-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; precision6-NEXT:    movl %eax, %ecx
 ; precision6-NEXT:    andl $8388607, %ecx # imm = 0x7FFFFF
 ; precision6-NEXT:    orl $1065353216, %ecx # imm = 0x3F800000
 ; precision6-NEXT:    movl %ecx, (%esp)
-; precision6-NEXT:    andl $2139095040, %eax # imm = 0x7F800000
 ; precision6-NEXT:    shrl $23, %eax
+; precision6-NEXT:    movzbl %al, %eax
 ; precision6-NEXT:    addl $-127, %eax
 ; precision6-NEXT:    movl %eax, {{[0-9]+}}(%esp)
 ; precision6-NEXT:    flds (%esp)
 ; precision6-NEXT:    fld %st(0)
 ; precision6-NEXT:    fmuls {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision6-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision6-NEXT:    fmulp %st, %st(1)
 ; precision6-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision6-NEXT:    fildl {{[0-9]+}}(%esp)
 ; precision6-NEXT:    fmuls {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision6-NEXT:    faddp %st, %st(1)
 ; precision6-NEXT:    addl $8, %esp
 ; precision6-NEXT:    retl
 ;
 ; precision12-LABEL: f6:
 ; precision12:       # %bb.0: # %entry
 ; precision12-NEXT:    subl $8, %esp
 ; precision12-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; precision12-NEXT:    movl %eax, %ecx
 ; precision12-NEXT:    andl $8388607, %ecx # imm = 0x7FFFFF
 ; precision12-NEXT:    orl $1065353216, %ecx # imm = 0x3F800000
 ; precision12-NEXT:    movl %ecx, (%esp)
-; precision12-NEXT:    andl $2139095040, %eax # imm = 0x7F800000
 ; precision12-NEXT:    shrl $23, %eax
+; precision12-NEXT:    movzbl %al, %eax
 ; precision12-NEXT:    addl $-127, %eax
 ; precision12-NEXT:    movl %eax, {{[0-9]+}}(%esp)
 ; precision12-NEXT:    flds (%esp)
 ; precision12-NEXT:    fld %st(0)
 ; precision12-NEXT:    fmuls {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision12-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision12-NEXT:    fmul %st(1), %st
 ; precision12-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision12-NEXT:    fmulp %st, %st(1)
 ; precision12-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision12-NEXT:    fildl {{[0-9]+}}(%esp)
 ; precision12-NEXT:    fmuls {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision12-NEXT:    faddp %st, %st(1)
 ; precision12-NEXT:    addl $8, %esp
 ; precision12-NEXT:    retl
 ;
 ; precision18-LABEL: f6:
 ; precision18:       # %bb.0: # %entry
 ; precision18-NEXT:    subl $8, %esp
 ; precision18-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; precision18-NEXT:    movl %eax, %ecx
 ; precision18-NEXT:    andl $8388607, %ecx # imm = 0x7FFFFF
 ; precision18-NEXT:    orl $1065353216, %ecx # imm = 0x3F800000
 ; precision18-NEXT:    movl %ecx, (%esp)
-; precision18-NEXT:    andl $2139095040, %eax # imm = 0x7F800000
 ; precision18-NEXT:    shrl $23, %eax
+; precision18-NEXT:    movzbl %al, %eax
 ; precision18-NEXT:    addl $-127, %eax
 ; precision18-NEXT:    movl %eax, {{[0-9]+}}(%esp)
 ; precision18-NEXT:    flds (%esp)
 ; precision18-NEXT:    fld %st(0)
 ; precision18-NEXT:    fmuls {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision18-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}
 ; precision18-NEXT:    fmul %st(1), %st
 ; precision18-NEXT:    fadds {{\.?LCPI[0-9]+_[0-9]+}}

llvm/test/CodeGen/X86/movmsk-cmp.ll

 ; SSE-LABEL: movmsk_v16i8:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    pcmpeqb %xmm1, %xmm0
 ; SSE-NEXT:    pmovmskb %xmm0, %eax
 ; SSE-NEXT:    movl %eax, %ecx
 ; SSE-NEXT:    shrl $15, %ecx
 ; SSE-NEXT:    movl %eax, %edx
 ; SSE-NEXT:    shrl $8, %edx
-; SSE-NEXT:    andl $1, %edx
-; SSE-NEXT:    andl $8, %eax
 ; SSE-NEXT:    shrl $3, %eax
 ; SSE-NEXT:    xorl %edx, %eax
 ; SSE-NEXT:    andl %ecx, %eax
 ; SSE-NEXT:    # kill: def $al killed $al killed $eax
 ; SSE-NEXT:    retq
 ;
 ; AVX1OR2-LABEL: movmsk_v16i8:
 ; AVX1OR2:       # %bb.0:
 ; AVX1OR2-NEXT:    vpcmpeqb %xmm1, %xmm0, %xmm0
 ; AVX1OR2-NEXT:    vpmovmskb %xmm0, %eax
 ; AVX1OR2-NEXT:    movl %eax, %ecx
 ; AVX1OR2-NEXT:    shrl $15, %ecx
 ; AVX1OR2-NEXT:    movl %eax, %edx
 ; AVX1OR2-NEXT:    shrl $8, %edx
-; AVX1OR2-NEXT:    andl $1, %edx
-; AVX1OR2-NEXT:    andl $8, %eax
 ; AVX1OR2-NEXT:    shrl $3, %eax
 ; AVX1OR2-NEXT:    xorl %edx, %eax
 ; AVX1OR2-NEXT:    andl %ecx, %eax
 ; AVX1OR2-NEXT:    # kill: def $al killed $al killed $eax
 ; AVX1OR2-NEXT:    retq
 ;
 ; KNL-LABEL: movmsk_v16i8:
 ; KNL:       # %bb.0:

 ; TODO: Replace shift+mask chain with AND+CMP.
 define i1 @movmsk_v4i32(<4 x i32> %x, <4 x i32> %y) {
 ; SSE-LABEL: movmsk_v4i32:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    pcmpgtd %xmm0, %xmm1
 ; SSE-NEXT:    movmskps %xmm1, %eax
 ; SSE-NEXT:    movl %eax, %ecx
 ; SSE-NEXT:    shrb $3, %cl
-; SSE-NEXT:    andb $4, %al
 ; SSE-NEXT:    shrb $2, %al
+; SSE-NEXT:    andb $1, %al
 ; SSE-NEXT:    xorb %cl, %al
 ; SSE-NEXT:    # kill: def $al killed $al killed $eax
 ; SSE-NEXT:    retq
 ;
 ; AVX1OR2-LABEL: movmsk_v4i32:
 ; AVX1OR2:       # %bb.0:
 ; AVX1OR2-NEXT:    vpcmpgtd %xmm0, %xmm1, %xmm0
 ; AVX1OR2-NEXT:    vmovmskps %xmm0, %eax
 ; AVX1OR2-NEXT:    movl %eax, %ecx
 ; AVX1OR2-NEXT:    shrb $3, %cl
-; AVX1OR2-NEXT:    andb $4, %al
 ; AVX1OR2-NEXT:    shrb $2, %al
+; AVX1OR2-NEXT:    andb $1, %al
 ; AVX1OR2-NEXT:    xorb %cl, %al
 ; AVX1OR2-NEXT:    # kill: def $al killed $al killed $eax
 ; AVX1OR2-NEXT:    retq
 ;
 ; KNL-LABEL: movmsk_v4i32:
 ; KNL:       # %bb.0:
 ; KNL-NEXT:    # kill: def $xmm1 killed $xmm1 def $zmm1
 ; KNL-NEXT:    # kill: def $xmm0 killed $xmm0 def $zmm0

llvm/test/CodeGen/X86/pr15267.ll

; CHECK-NEXT:    retq
   ret <4 x i64> %sext
 }
 
 define <16 x i4> @test4(ptr %in) nounwind {
 ; CHECK-LABEL: test4:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movq (%rdi), %rax
 ; CHECK-NEXT:    movl %eax, %ecx
-; CHECK-NEXT:    shrl $4, %ecx
 ; CHECK-NEXT:    andl $15, %ecx
-; CHECK-NEXT:    movl %eax, %edx
-; CHECK-NEXT:    andl $15, %edx
-; CHECK-NEXT:    vmovd %edx, %xmm0
+; CHECK-NEXT:    vmovd %ecx, %xmm0
+; CHECK-NEXT:    movzbl %al, %ecx
+; CHECK-NEXT:    shrl $4, %ecx
 ; CHECK-NEXT:    vpinsrb $1, %ecx, %xmm0, %xmm0
 ; CHECK-NEXT:    movl %eax, %ecx
 ; CHECK-NEXT:    shrl $8, %ecx
 ; CHECK-NEXT:    andl $15, %ecx
 ; CHECK-NEXT:    vpinsrb $2, %ecx, %xmm0, %xmm0
-; CHECK-NEXT:    movl %eax, %ecx
+; CHECK-NEXT:    movzwl %ax, %ecx
 ; CHECK-NEXT:    shrl $12, %ecx
-; CHECK-NEXT:    andl $15, %ecx
 ; CHECK-NEXT:    vpinsrb $3, %ecx, %xmm0, %xmm0
 ; CHECK-NEXT:    movl %eax, %ecx
 ; CHECK-NEXT:    shrl $16, %ecx
 ; CHECK-NEXT:    andl $15, %ecx
 ; CHECK-NEXT:    vpinsrb $4, %ecx, %xmm0, %xmm0
 ; CHECK-NEXT:    movl %eax, %ecx
 ; CHECK-NEXT:    shrl $20, %ecx
 ; CHECK-NEXT:    andl $15, %ecx

llvm/test/CodeGen/X86/pr45995.ll

 ; CHECK-NEXT:    .cfi_offset rbx, -32
 ; CHECK-NEXT:    .cfi_offset r14, -24
 ; CHECK-NEXT:    .cfi_offset rbp, -16
 ; CHECK-NEXT:    vpslld xmm0, xmm0, 31
 ; CHECK-NEXT:    vmovmskps edi, xmm0
 ; CHECK-NEXT:    mov ebx, edi
 ; CHECK-NEXT:    shr bl, 3
 ; CHECK-NEXT:    mov ebp, edi
-; CHECK-NEXT:    and bpl, 4
 ; CHECK-NEXT:    shr bpl, 2
+; CHECK-NEXT:    and bpl, 1
 ; CHECK-NEXT:    mov r14d, edi
-; CHECK-NEXT:    and r14b, 2
 ; CHECK-NEXT:    shr r14b
+; CHECK-NEXT:    and r14b, 1
 ; CHECK-NEXT:    call print_i1@PLT
 ; CHECK-NEXT:    movzx edi, r14b
 ; CHECK-NEXT:    call print_i1@PLT
 ; CHECK-NEXT:    movzx edi, bpl
 ; CHECK-NEXT:    call print_i1@PLT
 ; CHECK-NEXT:    movzx edi, bl
 ; CHECK-NEXT:    call print_i1@PLT
 ; CHECK-NEXT:    pop rbx

 ; CHECK-NEXT:    .cfi_offset r15, -24
 ; CHECK-NEXT:    .cfi_offset rbp, -16
 ; CHECK-NEXT:    vpslld xmm1, xmm1, 31
 ; CHECK-NEXT:    vmovmskps ebx, xmm1
 ; CHECK-NEXT:    mov eax, ebx
 ; CHECK-NEXT:    shr al, 3
 ; CHECK-NEXT:    mov byte ptr [rsp + 7], al # 1-byte Spill
 ; CHECK-NEXT:    mov r14d, ebx
-; CHECK-NEXT:    and r14b, 4
 ; CHECK-NEXT:    shr r14b, 2
+; CHECK-NEXT:    and r14b, 1
 ; CHECK-NEXT:    mov r15d, ebx
-; CHECK-NEXT:    and r15b, 2
 ; CHECK-NEXT:    shr r15b
+; CHECK-NEXT:    and r15b, 1
 ; CHECK-NEXT:    vpslld xmm0, xmm0, 31
 ; CHECK-NEXT:    vmovmskps edi, xmm0
 ; CHECK-NEXT:    mov r12d, edi
 ; CHECK-NEXT:    shr r12b, 3
 ; CHECK-NEXT:    mov r13d, edi
-; CHECK-NEXT:    and r13b, 4
 ; CHECK-NEXT:    shr r13b, 2
+; CHECK-NEXT:    and r13b, 1
 ; CHECK-NEXT:    mov ebp, edi
-; CHECK-NEXT:    and bpl, 2
 ; CHECK-NEXT:    shr bpl
+; CHECK-NEXT:    and bpl, 1
 ; CHECK-NEXT:    call print_i1@PLT
 ; CHECK-NEXT:    movzx edi, bpl
 ; CHECK-NEXT:    call print_i1@PLT
 ; CHECK-NEXT:    movzx edi, r13b
 ; CHECK-NEXT:    call print_i1@PLT
 ; CHECK-NEXT:    movzx edi, r12b
 ; CHECK-NEXT:    call print_i1@PLT
 ; CHECK-NEXT:    mov edi, ebx

llvm/test/CodeGen/X86/pull-binop-through-shift.ll

 ; X64-NEXT:    shrl $8, %eax
 ; X64-NEXT:    andl $8388352, %eax # imm = 0x7FFF00
 ; X64-NEXT:    movl %eax, (%rsi)
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: and_nosignbit_lshr:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl 8(%esp), %ecx
-; X86-NEXT:    movl $2147418112, %eax # imm = 0x7FFF0000
-; X86-NEXT:    andl 4(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $8, %eax
+; X86-NEXT:    andl $8388352, %eax # imm = 0x7FFF00
 ; X86-NEXT:    movl %eax, (%ecx)
 ; X86-NEXT:    retl
   %t0 = and i32 %x, 2147418112 ; 0x7FFF0000
   %r = lshr i32 %t0, 8
   store i32 %r, ptr %dst
   ret i32 %r
 }
 

 ; X64-NEXT:    shrl $8, %eax
 ; X64-NEXT:    andl $8388352, %eax # imm = 0x7FFF00
 ; X64-NEXT:    movl %eax, (%rsi)
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: and_nosignbit_ashr:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl 8(%esp), %ecx
-; X86-NEXT:    movl $2147418112, %eax # imm = 0x7FFF0000
-; X86-NEXT:    andl 4(%esp), %eax
+; X86-NEXT:    movl 4(%esp), %eax
 ; X86-NEXT:    shrl $8, %eax
+; X86-NEXT:    andl $8388352, %eax # imm = 0x7FFF00
 ; X86-NEXT:    movl %eax, (%ecx)
 ; X86-NEXT:    retl
   %t0 = and i32 %x, 2147418112 ; 0x7FFF0000
   %r = ashr i32 %t0, 8
   store i32 %r, ptr %dst
   ret i32 %r
 }
 

llvm/test/CodeGen/X86/rev16.ll

; X64-NEXT:    retq
   %mask_r8 = and i32 %r8, 16711935
   %tmp = or i32 %mask_l8, %mask_r8
   ret i32 %tmp
 }
 
 define i32 @not_rev16(i32 %a) {
 ; X86-LABEL: not_rev16:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT:    movl %ecx, %eax
-; X86-NEXT:    shll $8, %eax
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    shrl $8, %ecx
 ; X86-NEXT:    andl $65280, %ecx # imm = 0xFF00
-; X86-NEXT:    andl $16711680, %eax # imm = 0xFF0000
+; X86-NEXT:    andl $65280, %eax # imm = 0xFF00
+; X86-NEXT:    shll $8, %eax
 ; X86-NEXT:    orl %ecx, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: not_rev16:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    shll $8, %eax
-; X64-NEXT:    shrl $8, %edi
+; X64-NEXT:    shrl $8, %eax
+; X64-NEXT:    andl $65280, %eax # imm = 0xFF00
 ; X64-NEXT:    andl $65280, %edi # imm = 0xFF00
-; X64-NEXT:    andl $16711680, %eax # imm = 0xFF0000
+; X64-NEXT:    shll $8, %edi
 ; X64-NEXT:    orl %edi, %eax
 ; X64-NEXT:    retq
   %l8 = shl i32 %a, 8
   %r8 = lshr i32 %a, 8
   %mask_r8 = and i32 %r8, 4278255360
   %mask_l8 = and i32 %l8, 16711935
   %tmp = or i32 %mask_r8, %mask_l8
   ret i32 %tmp
; X64-NEXT:    retq
   %mask_r8 = and i32 %r8, 16711935
   %tmp = or i32 %mask_r8, %mask_l8
   ret i32 %tmp
 }
 
 define i32 @different_shift_amount(i32 %a) {
 ; X86-LABEL: different_shift_amount:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    movl %eax, %ecx
-; X86-NEXT:    shll $9, %ecx
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    shrl $8, %eax
-; X86-NEXT:    andl $-16712192, %ecx # imm = 0xFF00FE00
+; X86-NEXT:    andl $8355967, %ecx # imm = 0x7F807F
+; X86-NEXT:    shll $9, %ecx
 ; X86-NEXT:    andl $16711935, %eax # imm = 0xFF00FF
 ; X86-NEXT:    orl %ecx, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: different_shift_amount:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    shll $9, %eax
-; X64-NEXT:    shrl $8, %edi
-; X64-NEXT:    andl $-16712192, %eax # imm = 0xFF00FE00
-; X64-NEXT:    andl $16711935, %edi # imm = 0xFF00FF
+; X64-NEXT:    shrl $8, %eax
+; X64-NEXT:    andl $8355967, %edi # imm = 0x7F807F
+; X64-NEXT:    shll $9, %edi
+; X64-NEXT:    andl $16711935, %eax # imm = 0xFF00FF
 ; X64-NEXT:    orl %edi, %eax
 ; X64-NEXT:    retq
   %l8 = shl i32 %a, 9
   %r8 = lshr i32 %a, 8
   %mask_l8 = and i32 %l8, 4278255360
   %mask_r8 = and i32 %r8, 16711935
   %tmp = or i32 %mask_l8, %mask_r8
   ret i32 %tmp

llvm/test/CodeGen/X86/rotate-extract.ll

 
 ; Result would overshift
 define i32 @no_extract_shrl(i32 %i) nounwind {
 ; X86-LABEL: no_extract_shrl:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    shrl $9, %ecx
-; X86-NEXT:    andl $-8, %eax
+; X86-NEXT:    andl $120, %eax
 ; X86-NEXT:    shll $25, %eax
 ; X86-NEXT:    orl %ecx, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: no_extract_shrl:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shrl $9, %eax
-; X64-NEXT:    andl $-8, %edi
+; X64-NEXT:    andl $120, %edi
 ; X64-NEXT:    shll $25, %edi
 ; X64-NEXT:    orl %edi, %eax
 ; X64-NEXT:    retq
   %lhs_div = lshr i32 %i, 3
   %rhs_div = lshr i32 %i, 9
   %lhs_shift = shl i32 %lhs_div, 28
   %out = or i32 %lhs_shift, %rhs_div
   ret i32 %out

llvm/test/CodeGen/X86/selectcc-to-shiftand.ll

 ; ANY-NEXT:    retq
   %shl = select i1 %t, i8 128, i8 0
   ret i8 %shl
 }
 
 define i16 @sel_shift_bool_i16(i1 %t) {
 ; ANY-LABEL: sel_shift_bool_i16:
 ; ANY:       # %bb.0:
-; ANY-NEXT:    movl %edi, %eax
-; ANY-NEXT:    andl $1, %eax
-; ANY-NEXT:    shll $7, %eax
+; ANY-NEXT:    shll $7, %edi
+; ANY-NEXT:    movzbl %dil, %eax
 ; ANY-NEXT:    # kill: def $ax killed $ax killed $eax
 ; ANY-NEXT:    retq
   %shl = select i1 %t, i16 128, i16 0
   ret i16 %shl
 }
 
 define i32 @sel_shift_bool_i32(i1 %t) {
 ; ANY-LABEL: sel_shift_bool_i32:

llvm/test/CodeGen/X86/setcc.ll

; X64-NEXT:    retq
   %6 = xor i32 %5, 1
   ret i32 %6
 }
 
 define i16 @shift_and(i16 %a) {
 ; X86-LABEL: shift_and:
 ; X86:       ## %bb.0:
 ; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    andb $4, %al
-; X86-NEXT:    shrb $2, %al
-; X86-NEXT:    movzbl %al, %eax
+; X86-NEXT:    shrl $2, %eax
+; X86-NEXT:    andl $1, %eax
 ; X86-NEXT:    ## kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-NOTBM-LABEL: shift_and:
 ; X64-NOTBM:       ## %bb.0:
 ; X64-NOTBM-NEXT:    movl %edi, %eax
 ; X64-NOTBM-NEXT:    shrl $10, %eax
 ; X64-NOTBM-NEXT:    andl $1, %eax

llvm/test/CodeGen/X86/shift-amount-mod.ll

; X64-NEXT:    retq
   %negaaddbitwidthaddb = add i64 %negaaddbitwidth, %b
   %shifted = lshr i64 %val, %negaaddbitwidthaddb
   ret i64 %shifted
 }
 
 define i16 @sh_trunc_sh(i64 %x) {
 ; X32-LABEL: sh_trunc_sh:
 ; X32:       # %bb.0:
-; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X32-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    shrl $4, %eax
-; X32-NEXT:    andl $15, %eax
 ; X32-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: sh_trunc_sh:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    shrq $36, %rax
 ; X64-NEXT:    andl $15, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $rax
 ; X64-NEXT:    retq
   %s = lshr i64 %x, 24
   %t = trunc i64 %s to i16
   %r = lshr i16 %t, 12
   ret i16 %r
 }

llvm/test/CodeGen/X86/shift-mask.ll

; X64-NEXT:    retq
   %1 = lshr i16 %a0, 3
   %2 = shl i16 %1, 3
   ret i16 %2
 }
 
 define i16 @test_i16_shl_lshr_1(i16 %a0) {
 ; X86-LABEL: test_i16_shl_lshr_1:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    andl $16376, %eax # imm = 0x3FF8
 ; X86-NEXT:    shll $2, %eax
-; X86-NEXT:    andl $65504, %eax # imm = 0xFFE0
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-MASK-LABEL: test_i16_shl_lshr_1:
 ; X64-MASK:       # %bb.0:
 ; X64-MASK-NEXT:    # kill: def $edi killed $edi def $rdi
+; X64-MASK-NEXT:    andl $16376, %edi # imm = 0x3FF8
 ; X64-MASK-NEXT:    leal (,%rdi,4), %eax
-; X64-MASK-NEXT:    andl $65504, %eax # imm = 0xFFE0
 ; X64-MASK-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-MASK-NEXT:    retq
 ;
 ; X64-SHIFT-LABEL: test_i16_shl_lshr_1:
 ; X64-SHIFT:       # %bb.0:
 ; X64-SHIFT-NEXT:    movzwl %di, %eax
 ; X64-SHIFT-NEXT:    shrl $3, %eax
 ; X64-SHIFT-NEXT:    shll $5, %eax
; X64-SHIFT-NEXT:    retq
   %2 = lshr i8 %1, 5
   ret i8 %2
 }
 
 define i8 @test_i8_lshr_lshr_2(i8 %a0) {
 ; X86-LABEL: test_i8_lshr_lshr_2:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    andb $7, %al
 ; X86-NEXT:    shlb $2, %al
-; X86-NEXT:    andb $28, %al
 ; X86-NEXT:    retl
 ;
 ; X64-MASK-LABEL: test_i8_lshr_lshr_2:
 ; X64-MASK:       # %bb.0:
 ; X64-MASK-NEXT:    # kill: def $edi killed $edi def $rdi
+; X64-MASK-NEXT:    andb $7, %dil
 ; X64-MASK-NEXT:    leal (,%rdi,4), %eax
-; X64-MASK-NEXT:    andb $28, %al
 ; X64-MASK-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-MASK-NEXT:    retq
 ;
 ; X64-SHIFT-LABEL: test_i8_lshr_lshr_2:
 ; X64-SHIFT:       # %bb.0:
 ; X64-SHIFT-NEXT:    movl %edi, %eax
 ; X64-SHIFT-NEXT:    shlb $5, %al
 ; X64-SHIFT-NEXT:    shrb $3, %al
; X64-BMI-NEXT:    retq
   %1 = shl i16 %a0, 3
   %2 = lshr i16 %1, 5
   ret i16 %2
 }
 
 define i16 @test_i16_lshr_lshr_2(i16 %a0) {
 ; X86-LABEL: test_i16_lshr_lshr_2:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    andl $2047, %eax # imm = 0x7FF
 ; X86-NEXT:    shll $2, %eax
-; X86-NEXT:    andl $8188, %eax # imm = 0x1FFC
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_i16_lshr_lshr_2:
 ; X64:       # %bb.0:
 ; X64-NEXT:    # kill: def $edi killed $edi def $rdi
+; X64-NEXT:    andl $2047, %edi # imm = 0x7FF
 ; X64-NEXT:    leal (,%rdi,4), %eax
-; X64-NEXT:    andl $8188, %eax # imm = 0x1FFC
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %1 = shl i16 %a0, 5
   %2 = lshr i16 %1, 3
   ret i16 %2
 }
 
 define i32 @test_i32_lshr_lshr_0(i32 %a0) {
; X64-SHIFT-NEXT:    retq
   %1 = shl i32 %a0, 3
   %2 = lshr i32 %1, 5
   ret i32 %2
 }
 
 define i32 @test_i32_lshr_lshr_2(i32 %a0) {
 ; X86-LABEL: test_i32_lshr_lshr_2:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movl $134217727, %eax # imm = 0x7FFFFFF
+; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    shll $2, %eax
-; X86-NEXT:    andl $536870908, %eax # imm = 0x1FFFFFFC
 ; X86-NEXT:    retl
 ;
 ; X64-MASK-LABEL: test_i32_lshr_lshr_2:
 ; X64-MASK:       # %bb.0:
 ; X64-MASK-NEXT:    # kill: def $edi killed $edi def $rdi
+; X64-MASK-NEXT:    andl $134217727, %edi # imm = 0x7FFFFFF
 ; X64-MASK-NEXT:    leal (,%rdi,4), %eax
-; X64-MASK-NEXT:    andl $536870908, %eax # imm = 0x1FFFFFFC
 ; X64-MASK-NEXT:    retq
 ;
 ; X64-SHIFT-LABEL: test_i32_lshr_lshr_2:
 ; X64-SHIFT:       # %bb.0:
 ; X64-SHIFT-NEXT:    movl %edi, %eax
 ; X64-SHIFT-NEXT:    shll $5, %eax
 ; X64-SHIFT-NEXT:    shrl $3, %eax
 ; X64-SHIFT-NEXT:    retq

 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
 ; X86-NEXT:    shldl $2, %eax, %edx
 ; X86-NEXT:    shll $2, %eax
 ; X86-NEXT:    andl $536870911, %edx # imm = 0x1FFFFFFF
 ; X86-NEXT:    retl
 ;
 ; X64-MASK-LABEL: test_i64_lshr_lshr_2:
 ; X64-MASK:       # %bb.0:
-; X64-MASK-NEXT:    leaq (,%rdi,4), %rcx
-; X64-MASK-NEXT:    movabsq $2305843009213693948, %rax # imm = 0x1FFFFFFFFFFFFFFC
-; X64-MASK-NEXT:    andq %rcx, %rax
+; X64-MASK-NEXT:    movabsq $576460752303423487, %rax # imm = 0x7FFFFFFFFFFFFFF
+; X64-MASK-NEXT:    andq %rdi, %rax
+; X64-MASK-NEXT:    shlq $2, %rax
 ; X64-MASK-NEXT:    retq
 ;
 ; X64-SHIFT-LABEL: test_i64_lshr_lshr_2:
 ; X64-SHIFT:       # %bb.0:
 ; X64-SHIFT-NEXT:    movq %rdi, %rax
 ; X64-SHIFT-NEXT:    shlq $5, %rax
 ; X64-SHIFT-NEXT:    shrq $3, %rax
 ; X64-SHIFT-NEXT:    retq
   %1 = shl i64 %a0, 5
   %2 = lshr i64 %1, 3
   ret i64 %2
 }

llvm/test/CodeGen/X86/sttni.ll

 ; X86-NEXT:    pcmpestri $24, %xmm1, %xmm0
 ; X86-NEXT:    cmpl $16, %ecx
 ; X86-NEXT:    jne .LBB8_2
 ; X86-NEXT:  # %bb.1:
 ; X86-NEXT:    xorl %eax, %eax
 ; X86-NEXT:    jmp .LBB8_3
 ; X86-NEXT:  .LBB8_2: # %compare
 ; X86-NEXT:    movdqa %xmm0, (%esp)
-; X86-NEXT:    addl %ecx, %ecx
-; X86-NEXT:    andl $14, %ecx
-; X86-NEXT:    movzwl (%esp,%ecx), %eax
+; X86-NEXT:    andl $7, %ecx
+; X86-NEXT:    movzwl (%esp,%ecx,2), %eax
 ; X86-NEXT:    movdqa %xmm1, {{[0-9]+}}(%esp)
-; X86-NEXT:    subw 16(%esp,%ecx), %ax
+; X86-NEXT:    subw 16(%esp,%ecx,2), %ax
 ; X86-NEXT:  .LBB8_3: # %exit
 ; X86-NEXT:    movzwl %ax, %eax
 ; X86-NEXT:    movl %ebp, %esp
 ; X86-NEXT:    popl %ebp
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: pcmpestri_reg_diff_i16:
 ; X64:       # %bb.0: # %entry

 ; X86-NEXT:    pcmpestri $25, %xmm0, %xmm1
 ; X86-NEXT:    cmpl $8, %ecx
 ; X86-NEXT:    jne .LBB11_2
 ; X86-NEXT:  # %bb.1:
 ; X86-NEXT:    xorl %eax, %eax
 ; X86-NEXT:    jmp .LBB11_3
 ; X86-NEXT:  .LBB11_2: # %compare
 ; X86-NEXT:    movdqa %xmm1, (%esp)
-; X86-NEXT:    addl %ecx, %ecx
-; X86-NEXT:    andl $14, %ecx
-; X86-NEXT:    movzwl (%esp,%ecx), %eax
+; X86-NEXT:    andl $7, %ecx
+; X86-NEXT:    movzwl (%esp,%ecx,2), %eax
 ; X86-NEXT:    movdqa %xmm0, {{[0-9]+}}(%esp)
-; X86-NEXT:    subw 16(%esp,%ecx), %ax
+; X86-NEXT:    subw 16(%esp,%ecx,2), %ax
 ; X86-NEXT:  .LBB11_3: # %exit
 ; X86-NEXT:    movzwl %ax, %eax
 ; X86-NEXT:    leal -4(%ebp), %esp
 ; X86-NEXT:    popl %esi
 ; X86-NEXT:    popl %ebp
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: pcmpestri_mem_diff_i16:

 ; X86-NEXT:    movzwl %ax, %eax
 ; X86-NEXT:    retl
 ; X86-NEXT:  .LBB20_2: # %compare
 ; X86-NEXT:    pushl %ebp
 ; X86-NEXT:    movl %esp, %ebp
 ; X86-NEXT:    andl $-16, %esp
 ; X86-NEXT:    subl $48, %esp
 ; X86-NEXT:    movdqa %xmm0, (%esp)
-; X86-NEXT:    addl %ecx, %ecx
-; X86-NEXT:    andl $14, %ecx
-; X86-NEXT:    movzwl (%esp,%ecx), %eax
+; X86-NEXT:    andl $7, %ecx
+; X86-NEXT:    movzwl (%esp,%ecx,2), %eax
 ; X86-NEXT:    movdqa %xmm1, {{[0-9]+}}(%esp)
-; X86-NEXT:    subw 16(%esp,%ecx), %ax
+; X86-NEXT:    subw 16(%esp,%ecx,2), %ax
 ; X86-NEXT:    movl %ebp, %esp
 ; X86-NEXT:    popl %ebp
 ; X86-NEXT:    movzwl %ax, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: pcmpistri_reg_diff_i16:
 ; X64:       # %bb.0: # %entry
 ; X64-NEXT:    pcmpistri $24, %xmm1, %xmm0

 ; X86-NEXT:    pcmpistri $25, %xmm0, %xmm1
 ; X86-NEXT:    cmpl $8, %ecx
 ; X86-NEXT:    jne .LBB23_2
 ; X86-NEXT:  # %bb.1:
 ; X86-NEXT:    xorl %eax, %eax
 ; X86-NEXT:    jmp .LBB23_3
 ; X86-NEXT:  .LBB23_2: # %compare
 ; X86-NEXT:    movdqa %xmm1, (%esp)
-; X86-NEXT:    addl %ecx, %ecx
-; X86-NEXT:    andl $14, %ecx
-; X86-NEXT:    movzwl (%esp,%ecx), %eax
+; X86-NEXT:    andl $7, %ecx
+; X86-NEXT:    movzwl (%esp,%ecx,2), %eax
 ; X86-NEXT:    movdqa %xmm0, {{[0-9]+}}(%esp)
-; X86-NEXT:    subw 16(%esp,%ecx), %ax
+; X86-NEXT:    subw 16(%esp,%ecx,2), %ax
 ; X86-NEXT:  .LBB23_3: # %exit
 ; X86-NEXT:    movzwl %ax, %eax
 ; X86-NEXT:    movl %ebp, %esp
 ; X86-NEXT:    popl %ebp
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: pcmpistri_mem_diff_i16:
 ; X64:       # %bb.0: # %entry

llvm/test/CodeGen/X86/tbm_patterns.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=x86_64-unknown-unknown -mattr=+tbm < %s | FileCheck %s
 
 define i32 @test_x86_tbm_bextri_u32(i32 %a) nounwind {
 ; CHECK-LABEL: test_x86_tbm_bextri_u32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    bextrl $3076, %edi, %eax # imm = 0xC04
+; CHECK-NEXT:    movzwl %di, %eax
+; CHECK-NEXT:    shrl $4, %eax
 ; CHECK-NEXT:    retq
   %t0 = lshr i32 %a, 4
   %t1 = and i32 %t0, 4095
   ret i32 %t1
 }
 
 ; Make sure we still use AH subreg trick for extracting bits 15:8
 define i32 @test_x86_tbm_bextri_u32_subreg(i32 %a) nounwind {
 ; CHECK-LABEL: test_x86_tbm_bextri_u32_subreg:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movl %edi, %eax
 ; CHECK-NEXT:    movzbl %ah, %eax
 ; CHECK-NEXT:    retq
   %t0 = lshr i32 %a, 8
   %t1 = and i32 %t0, 255
   ret i32 %t1
 }
 
 define i32 @test_x86_tbm_bextri_u32_m(ptr nocapture %a) nounwind {
 ; CHECK-LABEL: test_x86_tbm_bextri_u32_m:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    bextrl $3076, (%rdi), %eax # imm = 0xC04
+; CHECK-NEXT:    movzwl (%rdi), %eax
+; CHECK-NEXT:    shrl $4, %eax
 ; CHECK-NEXT:    retq
   %t0 = load i32, ptr %a
   %t1 = lshr i32 %t0, 4
   %t2 = and i32 %t1, 4095
   ret i32 %t2
 }
 
 define i32 @test_x86_tbm_bextri_u32_z(i32 %a, i32 %b) nounwind {
 ; CHECK-LABEL: test_x86_tbm_bextri_u32_z:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    bextrl $3076, %edi, %eax # imm = 0xC04
+; CHECK-NEXT:    movzwl %di, %eax
+; CHECK-NEXT:    shrl $4, %eax
 ; CHECK-NEXT:    cmovel %esi, %eax
 ; CHECK-NEXT:    retq
   %t0 = lshr i32 %a, 4
   %t1 = and i32 %t0, 4095
   %t2 = icmp eq i32 %t1, 0
   %t3 = select i1 %t2, i32 %b, i32 %t1
   ret i32 %t3
 }
 
 define i32 @test_x86_tbm_bextri_u32_z2(i32 %a, i32 %b, i32 %c) nounwind {
 ; CHECK-LABEL: test_x86_tbm_bextri_u32_z2:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movl %esi, %eax
-; CHECK-NEXT:    bextrl $3076, %edi, %ecx # imm = 0xC04
+; CHECK-NEXT:    testl $65520, %edi # imm = 0xFFF0
 ; CHECK-NEXT:    cmovnel %edx, %eax
 ; CHECK-NEXT:    retq
   %t0 = lshr i32 %a, 4
   %t1 = and i32 %t0, 4095
   %t2 = icmp eq i32 %t1, 0
   %t3 = select i1 %t2, i32 %b, i32 %c
   ret i32 %t3
 }
 
 define i32 @test_x86_tbm_bextri_u32_sle(i32 %a, i32 %b, i32 %c) nounwind {
 ; CHECK-LABEL: test_x86_tbm_bextri_u32_sle:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movl %esi, %eax
-; CHECK-NEXT:    bextrl $3076, %edi, %ecx # imm = 0xC04
+; CHECK-NEXT:    movzwl %di, %ecx
+; CHECK-NEXT:    shrl $4, %ecx
 ; CHECK-NEXT:    testl %ecx, %ecx
 ; CHECK-NEXT:    cmovgl %edx, %eax
 ; CHECK-NEXT:    retq
   %t0 = lshr i32 %a, 4
   %t1 = and i32 %t0, 4095
   %t2 = icmp sle i32 %t1, 0
   %t3 = select i1 %t2, i32 %b, i32 %c
   ret i32 %t3
 }
 
 define i64 @test_x86_tbm_bextri_u64(i64 %a) nounwind {
 ; CHECK-LABEL: test_x86_tbm_bextri_u64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    bextrl $3076, %edi, %eax # imm = 0xC04
+; CHECK-NEXT:    movzwl %di, %eax
+; CHECK-NEXT:    shrl $4, %eax
 ; CHECK-NEXT:    retq
   %t0 = lshr i64 %a, 4
   %t1 = and i64 %t0, 4095
   ret i64 %t1
 }
 
 ; Make sure we still use AH subreg trick for extracting bits 15:8
 define i64 @test_x86_tbm_bextri_u64_subreg(i64 %a) nounwind {
 ; CHECK-LABEL: test_x86_tbm_bextri_u64_subreg:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movq %rdi, %rax
 ; CHECK-NEXT:    movzbl %ah, %eax
 ; CHECK-NEXT:    retq
   %t0 = lshr i64 %a, 8
   %t1 = and i64 %t0, 255
   ret i64 %t1
 }
 
 define i64 @test_x86_tbm_bextri_u64_m(ptr nocapture %a) nounwind {
 ; CHECK-LABEL: test_x86_tbm_bextri_u64_m:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    bextrl $3076, (%rdi), %eax # imm = 0xC04
+; CHECK-NEXT:    movzwl (%rdi), %eax
+; CHECK-NEXT:    shrl $4, %eax
 ; CHECK-NEXT:    retq
   %t0 = load i64, ptr %a
   %t1 = lshr i64 %t0, 4
   %t2 = and i64 %t1, 4095
   ret i64 %t2
 }
 
 define i64 @test_x86_tbm_bextri_u64_z(i64 %a, i64 %b) nounwind {
 ; CHECK-LABEL: test_x86_tbm_bextri_u64_z:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    bextrl $3076, %edi, %eax # imm = 0xC04
+; CHECK-NEXT:    movzwl %di, %eax
+; CHECK-NEXT:    shrl $4, %eax
 ; CHECK-NEXT:    cmoveq %rsi, %rax
 ; CHECK-NEXT:    retq
   %t0 = lshr i64 %a, 4
   %t1 = and i64 %t0, 4095
   %t2 = icmp eq i64 %t1, 0
   %t3 = select i1 %t2, i64 %b, i64 %t1
   ret i64 %t3
 }
 
 define i64 @test_x86_tbm_bextri_u64_z2(i64 %a, i64 %b, i64 %c) nounwind {
 ; CHECK-LABEL: test_x86_tbm_bextri_u64_z2:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movq %rsi, %rax
-; CHECK-NEXT:    bextrl $3076, %edi, %ecx # imm = 0xC04
+; CHECK-NEXT:    testl $65520, %edi # imm = 0xFFF0
 ; CHECK-NEXT:    cmovneq %rdx, %rax
 ; CHECK-NEXT:    retq
   %t0 = lshr i64 %a, 4
   %t1 = and i64 %t0, 4095
   %t2 = icmp eq i64 %t1, 0
   %t3 = select i1 %t2, i64 %b, i64 %c
   ret i64 %t3
 }
 
 define i64 @test_x86_tbm_bextri_u64_sle(i64 %a, i64 %b, i64 %c) nounwind {
 ; CHECK-LABEL: test_x86_tbm_bextri_u64_sle:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movq %rsi, %rax
-; CHECK-NEXT:    bextrl $3076, %edi, %ecx # imm = 0xC04
+; CHECK-NEXT:    movzwl %di, %ecx
+; CHECK-NEXT:    shrl $4, %ecx
 ; CHECK-NEXT:    testq %rcx, %rcx
 ; CHECK-NEXT:    cmovgq %rdx, %rax
 ; CHECK-NEXT:    retq
   %t0 = lshr i64 %a, 4
   %t1 = and i64 %t0, 4095
   %t2 = icmp sle i64 %t1, 0
   %t3 = select i1 %t2, i64 %b, i64 %c
   ret i64 %t3

llvm/test/CodeGen/X86/udiv_fix.ll

 ; X64-NEXT:    cwtl
 ; X64-NEXT:    shrl %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: func3:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    addl %eax, %eax
-; X86-NEXT:    movzbl %cl, %ecx
+; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    shll $4, %ecx
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    divw %cx
 ; X86-NEXT:    # kill: def $ax killed $ax def $eax
 ; X86-NEXT:    addl %eax, %eax
 ; X86-NEXT:    cwtl
 ; X86-NEXT:    shrl %eax

llvm/test/CodeGen/X86/udiv_fix_sat.ll

 ; X64-NEXT:    cmovael %ecx, %eax
 ; X64-NEXT:    shrl %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: func:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    movzwl %ax, %eax
+; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    shll $8, %eax
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    divl %ecx
 ; X86-NEXT:    cmpl $131071, %eax # imm = 0x1FFFF
 ; X86-NEXT:    movl $131071, %ecx # imm = 0x1FFFF
 ; X86-NEXT:    cmovael %ecx, %eax
 ; X86-NEXT:    shrl %eax
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax

 ; X64-NEXT:    movswl %cx, %eax
 ; X64-NEXT:    shrl %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: func3:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    addl %eax, %eax
-; X86-NEXT:    movzbl %cl, %ecx
+; X86-NEXT:    movzbl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    shll $4, %ecx
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    divw %cx
 ; X86-NEXT:    # kill: def $ax killed $ax def $eax
 ; X86-NEXT:    movzwl %ax, %ecx
 ; X86-NEXT:    cmpl $32767, %ecx # imm = 0x7FFF
 ; X86-NEXT:    movl $32767, %ecx # imm = 0x7FFF

 ; X64-NEXT:    cmovaeq %rcx, %rax
 ; X64-NEXT:    shrl %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $rax
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: func7:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    movl %ecx, %edx
-; X86-NEXT:    shll $17, %edx
-; X86-NEXT:    shrl $15, %ecx
-; X86-NEXT:    andl $1, %ecx
+; X86-NEXT:    shrl $15, %edx
+; X86-NEXT:    shll $17, %ecx
 ; X86-NEXT:    pushl $0
 ; X86-NEXT:    pushl %eax
-; X86-NEXT:    pushl %ecx
 ; X86-NEXT:    pushl %edx
+; X86-NEXT:    pushl %ecx
 ; X86-NEXT:    calll __udivdi3
 ; X86-NEXT:    addl $16, %esp
 ; X86-NEXT:    cmpl $131071, %eax # imm = 0x1FFFF
 ; X86-NEXT:    movl $131071, %ecx # imm = 0x1FFFF
 ; X86-NEXT:    cmovael %ecx, %eax
 ; X86-NEXT:    testl %edx, %edx
 ; X86-NEXT:    cmovnel %ecx, %eax
 ; X86-NEXT:    shrl %eax

llvm/test/CodeGen/X86/urem-seteq-illegal-types.ll

 }
 
 define i1 @test_urem_even(i27 %X) nounwind {
 ; X86-LABEL: test_urem_even:
 ; X86:       # %bb.0:
 ; X86-NEXT:    imull $115043767, {{[0-9]+}}(%esp), %eax # imm = 0x6DB6DB7
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    shll $26, %ecx
-; X86-NEXT:    andl $134217726, %eax # imm = 0x7FFFFFE
 ; X86-NEXT:    shrl %eax
+; X86-NEXT:    andl $67108863, %eax # imm = 0x3FFFFFF
 ; X86-NEXT:    orl %ecx, %eax
 ; X86-NEXT:    andl $134217727, %eax # imm = 0x7FFFFFF
 ; X86-NEXT:    cmpl $9586981, %eax # imm = 0x924925
 ; X86-NEXT:    setb %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_urem_even:
 ; X64:       # %bb.0:
 ; X64-NEXT:    imull $115043767, %edi, %eax # imm = 0x6DB6DB7
 ; X64-NEXT:    movl %eax, %ecx
 ; X64-NEXT:    shll $26, %ecx
-; X64-NEXT:    andl $134217726, %eax # imm = 0x7FFFFFE
 ; X64-NEXT:    shrl %eax
+; X64-NEXT:    andl $67108863, %eax # imm = 0x3FFFFFF
 ; X64-NEXT:    orl %ecx, %eax
 ; X64-NEXT:    andl $134217727, %eax # imm = 0x7FFFFFF
 ; X64-NEXT:    cmpl $9586981, %eax # imm = 0x924925
 ; X64-NEXT:    setb %al
 ; X64-NEXT:    retq
   %urem = urem i27 %X, 14
   %cmp = icmp eq i27 %urem, 0
   ret i1 %cmp

 }
 
 define <3 x i1> @test_urem_vec(<3 x i11> %X) nounwind {
 ; X86-LABEL: test_urem_vec:
 ; X86:       # %bb.0:
 ; X86-NEXT:    imull $683, {{[0-9]+}}(%esp), %eax # imm = 0x2AB
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    shll $10, %ecx
-; X86-NEXT:    andl $2046, %eax # imm = 0x7FE
 ; X86-NEXT:    shrl %eax
+; X86-NEXT:    andl $1023, %eax # imm = 0x3FF
 ; X86-NEXT:    orl %ecx, %eax
 ; X86-NEXT:    andl $2047, %eax # imm = 0x7FF
 ; X86-NEXT:    cmpl $342, %eax # imm = 0x156
 ; X86-NEXT:    setae %al
 ; X86-NEXT:    imull $1463, {{[0-9]+}}(%esp), %ecx # imm = 0x5B7
 ; X86-NEXT:    addl $-1463, %ecx # imm = 0xFA49
 ; X86-NEXT:    andl $2047, %ecx # imm = 0x7FF
 ; X86-NEXT:    cmpl $293, %ecx # imm = 0x125

llvm/test/CodeGen/X86/vselect.ll

 ; This test case previously crashed after r363802, r363850, and r363856 due
 ; any_extend_vector_inreg not being handled by the X86 backend.
 define i64 @vselect_any_extend_vector_inreg_crash(ptr %x) {
 ; SSE-LABEL: vselect_any_extend_vector_inreg_crash:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    movq {{.*#+}} xmm0 = mem[0],zero
 ; SSE-NEXT:    pcmpeqb {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
 ; SSE-NEXT:    movq %xmm0, %rax
-; SSE-NEXT:    andl $1, %eax
-; SSE-NEXT:    shlq $15, %rax
+; SSE-NEXT:    shll $15, %eax
+; SSE-NEXT:    movzwl %ax, %eax
 ; SSE-NEXT:    retq
 ;
 ; AVX-LABEL: vselect_any_extend_vector_inreg_crash:
 ; AVX:       # %bb.0:
 ; AVX-NEXT:    vmovq {{.*#+}} xmm0 = mem[0],zero
 ; AVX-NEXT:    vpcmpeqb {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0, %xmm0
 ; AVX-NEXT:    vmovq %xmm0, %rax
-; AVX-NEXT:    andl $1, %eax
-; AVX-NEXT:    shlq $15, %rax
+; AVX-NEXT:    shll $15, %eax
+; AVX-NEXT:    movzwl %ax, %eax
 ; AVX-NEXT:    retq
 0:
   %1 = load <8 x i8>, ptr %x
   %2 = icmp eq <8 x i8> %1, <i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49>
   %3 = select <8 x i1> %2, <8 x i64> <i64 32768, i64 16384, i64 8192, i64 4096, i64 2048, i64 1024, i64 512, i64 256>, <8 x i64> zeroinitializer
   %4 = extractelement <8 x i64> %3, i32 0
   ret i64 %4
 }
 

llvm/test/CodeGen/X86/zext-logicop-shift-load.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=i686-unknown-unknown | FileCheck %s --check-prefix=X86
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown | FileCheck %s --check-prefix=X64
 
 define i64 @test1(ptr %data) {
 ; X86-LABEL: test1:
 ; X86:       # %bb.0: # %entry
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    movzbl (%eax), %eax
+; X86-NEXT:    andl $15, %eax
 ; X86-NEXT:    shll $2, %eax
-; X86-NEXT:    andl $60, %eax
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test1:
 ; X64:       # %bb.0: # %entry
 ; X64-NEXT:    movl (%rdi), %eax
 ; X64-NEXT:    shll $2, %eax
 ; X64-NEXT:    andl $60, %eax