[X86] Remove patterns for shift/rotate with immediate 1 and optimize during MC lowering

[skan]

Summary

It's first suggested by @craig.topper in D150068. I think there are at least three pros

1. This can reduce the patterns during ISEL, as a result, reducing the bytes in X86GenDAGISel.inc
2. The patterns for shift/rotate with immediate 1 look quite similar to shift/rotate with immediate 8. So this can be seen as eliminating "duplicate" code.
3. Delay the optimization from imm8 to imm1, so that the previous optimization passes do not need to handle the version of imm1

It improves fast isel code and makes X86DomainReassignment work for shifts by 1, but regressed global isel, though no one should care.

[RKSimon]

What is the motivation here?

[skan]

In D150107#4326328, @RKSimon wrote:

What is the motivation here?

It's first suggested by craig in D150068. I think there are at least three pros

1. This can reduce the patterns during ISEL, as a result, reducing the bytes in X86GenDAGISel.inc
2. The patterns for shift/rotate with immediate 1 look quite similar to shift/rotate with immediate 8. So this can be seen as eliminating "duplicate" code.
3. Delay the optimization from imm8 to imm1, so that the previous optimization passes do not need to handle the version of imm1

[craig.topper]

llvm/test/tools/llvm-mca/X86/AlderlakeP/resources-x86_64.s

1518 ↗

(On Diff #520315)

I think you need to let mayLoad = 0

[craig.topper]

llvm/test/tools/llvm-mca/X86/AlderlakeP/resources-x86_64.s

1518 ↗

(On Diff #520315)

Err nevermind

[craig.topper]

llvm/test/tools/llvm-mca/X86/AlderlakeP/resources-x86_64.s

1518 ↗

(On Diff #520315)

This should really be a different patch. This is an existing bug independent of whether removing the patterns gets approved.

[craig.topper]

In D150107#4326423, @skan wrote:

In D150107#4326328, @RKSimon wrote:

What is the motivation here?

It's first suggested by craig in D150068. I think there are at least three pros

1. This can reduce the patterns during ISEL, as a result, reducing the bytes in X86GenDAGISel.inc
2. The patterns for shift/rotate with immediate 1 look quite similar to shift/rotate with immediate 8. So this can be seen as eliminating "duplicate" code.
3. Delay the optimization from imm8 to imm1, so that the previous optimization passes do not need to handle the version of imm1

Yeah I thought since the code had been moved to a function it was easy to share it and get an isel table reduction.

Looks like it improved fast isel code and made X86DomainReassignment work for shifts by 1. But regressed global isel, though I guess no one should care.

[skan]

In D150107#4328315, @craig.topper wrote:

In D150107#4326423, @skan wrote:

In D150107#4326328, @RKSimon wrote:

What is the motivation here?

It's first suggested by craig in D150068. I think there are at least three pros

1. This can reduce the patterns during ISEL, as a result, reducing the bytes in X86GenDAGISel.inc
2. The patterns for shift/rotate with immediate 1 look quite similar to shift/rotate with immediate 8. So this can be seen as eliminating "duplicate" code.
3. Delay the optimization from imm8 to imm1, so that the previous optimization passes do not need to handle the version of imm1

Yeah I thought since the code had been moved to a function it was easy to share it and get an isel table reduction.

Looks like it improved fast isel code and made X86DomainReassignment work for shifts by 1. But regressed global isel, though I guess no one should care.

Yes, and thank you for the illustration! The code generated by global isel is far from expected in this test, so I agree no one should care.

llvm/test/tools/llvm-mca/X86/AlderlakeP/resources-x86_64.s

1518 ↗

(On Diff #520315)

I thought so. But when I tried to remove the patterns only in the TD, these tests changed too.

If I did not add mayLoad =1, mayStore=1 explicitly for shift/rotate[m1], these instructions would show mayLoad=0, mayStore=0, hasUnModeledSideEffect=1 in these tests.

That's why I updated these bits. It's weird to me if I set mayLoad=1 for m1 version but set mayLoad=0 for mi version.

[craig.topper]

llvm/lib/Target/X86/X86InstrShiftRotate.td

365

I think this line is the one that affected the llvm-mca tests. You can go ahead and commit that and the the llvm-mca test changes.

[skan]

Address review comments: commit unrelated change first rGc60461e3f815

[skan]

Remove more patterns

[craig.topper]

llvm/lib/Target/X86/X86MCInstLower.cpp

508

Missing if?

[skan]

Address review comments

llvm/lib/Target/X86/X86MCInstLower.cpp

508

Good catch! Done.

[craig.topper]

llvm/lib/Target/X86/MCTargetDesc/X86EncodingOptimization.cpp

139

Why not check the opcode first?

[skan]

Address review comments: check opcode first in optimizeRotateWithImmediate

[skan]

Friendly ping

[craig.topper]

llvm/lib/Target/X86/MCTargetDesc/X86EncodingOptimization.cpp

151

Are the flags the same? I don't think we ever use the flags from rotate, but since we're treating this like almost an assembler optimization we should probably know if we'll break any future flag usages.

[skan]

llvm/lib/Target/X86/MCTargetDesc/X86EncodingOptimization.cpp

151

Are the flags the same? I don't think we ever use the flags from rotate, but since we're treating this like almost an assembler optimization we should probably know if we'll break any future flag usages.

Good point! The SPEC says

For ROL and ROR instructions, if
the masked count is 0, the flags are not affected. If the masked count is 1, then the OF flag is affected, otherwise
(masked count is greater than 1) the OF flag is undefined.

So they have different impact on the flags. I will revert the related changes.

[skan]

Revert incorrect change about ROL/ROR

[craig.topper]

LGTM

[skan]

Rebase

[hans]

skan mentioned this in rG2ef8ae134828: [X86] Remove patterns for ADC/SBB with immediate 8 and optimize during MC…

That caused assert in Chromium builds. See https://bugs.chromium.org/p/chromium/issues/detail?id=1446973#c2 for a reproducer.

[hans]

In D150107#4356102, @hans wrote:

skan mentioned this in rG2ef8ae134828: [X86] Remove patterns for ADC/SBB with immediate 8 and optimize during MC…

That caused assert in Chromium builds. See https://bugs.chromium.org/p/chromium/issues/detail?id=1446973#c2 for a reproducer.

I'll revert while that's investigated.

[skan]

In D150107#4356140, @hans wrote:

In D150107#4356102, @hans wrote:

skan mentioned this in rG2ef8ae134828: [X86] Remove patterns for ADC/SBB with immediate 8 and optimize during MC…

That caused assert in Chromium builds. See https://bugs.chromium.org/p/chromium/issues/detail?id=1446973#c2 for a reproducer.

I'll revert while that's investigated.

@hans Thanks for the report! If you retested the reproducer before the revert, you would find it already passed after 5586bc539acb26cb94e461438de01a5080513401. So I reapplied the patch.

It failed before b/c the last operand of MCInst ADC/SBBri could be an expression instead of an immediate.

[hans]

In D150107#4356347, @skan wrote:

In D150107#4356140, @hans wrote:

In D150107#4356102, @hans wrote:

skan mentioned this in rG2ef8ae134828: [X86] Remove patterns for ADC/SBB with immediate 8 and optimize during MC…

That caused assert in Chromium builds. See https://bugs.chromium.org/p/chromium/issues/detail?id=1446973#c2 for a reproducer.

I'll revert while that's investigated.

@hans Thanks for the report! If you retested the reproducer before the revert, you would find it already passed after 5586bc539acb26cb94e461438de01a5080513401. So I reapplied the patch.

It failed before b/c the last operand of MCInst ADC/SBBri could be an expression instead of an immediate.

Sorry about that. I didn't realize 5586bc539acb26cb94e461438de01a5080513401 was a bug fix, and I obviously failed to re-run the test at head.

llvm/lib/Target/X86/MCTargetDesc/X86EncodingOptimization.h

 
 #ifndef LLVM_LIB_TARGET_X86_X86ENCODINGOPTIMIZATION_H
 #define LLVM_LIB_TARGET_X86_X86ENCODINGOPTIMIZATION_H
 namespace llvm {
 class MCInst;
 namespace X86 {
 bool optimizeInstFromVEX3ToVEX2(MCInst &MI);
 bool optimizeShiftRotateWithImmediateOne(MCInst &MI);
+bool optimizeRotateWithImmediate(MCInst &MI);
 } // namespace X86
 } // namespace llvm
 #endif

llvm/lib/Target/X86/MCTargetDesc/X86EncodingOptimization.cpp

default:
     TO_IMM1(SHR8m)
     TO_IMM1(SHR16m)
     TO_IMM1(SHR32m)
     TO_IMM1(SHR64m)
     TO_IMM1(SHL8m)
     TO_IMM1(SHL16m)
     TO_IMM1(SHL32m)
     TO_IMM1(SHL64m)
+#undef TO_IMM1
   }
   MCOperand &LastOp = MI.getOperand(MI.getNumOperands() - 1);
   if (!LastOp.isImm() || LastOp.getImm() != 1)
     return false;
   MI.setOpcode(NewOpc);
   MI.erase(&LastOp);
   return true;
 }
+
+bool X86::optimizeRotateWithImmediate(MCInst &MI) {
+  unsigned NumOps = MI.getNumOperands();
+  if (!NumOps)
+    return false;
+  MCOperand &LastOp = MI.getOperand(MI.getNumOperands() - 1);
+  if (!LastOp.isImm())
+    return false;
+  int64_t Imm = LastOp.getImm();
+  if (Imm == 1)
+    return false;
+  unsigned NewOpc;
+#define FROM_TO(FROM, TO, N)                                                   \
+  case X86::FROM:                                                              \
+    if (Imm != N)                                                              \
+      return false;                                                            \
+    NewOpc = X86::TO;                                                          \
+    break;
+  switch (MI.getOpcode()) {
+  default:
+    return false;
+    FROM_TO(ROL8ri, ROR8r1, 7)
+    FROM_TO(ROR8ri, ROL8r1, 7)
+    FROM_TO(ROL8mi, ROR8m1, 7)
+    FROM_TO(ROR8mi, ROL8m1, 7)
+    FROM_TO(ROL16ri, ROR16r1, 15)
+    FROM_TO(ROR16ri, ROL16r1, 15)
+    FROM_TO(ROL16mi, ROR16m1, 15)
+    FROM_TO(ROR16mi, ROL16m1, 15)
+    FROM_TO(ROL32ri, ROR32r1, 31)
+    FROM_TO(ROL32mi, ROR32m1, 31)
+    FROM_TO(ROR32ri, ROL32r1, 31)
+    FROM_TO(ROR32mi, ROL32m1, 31)
+    FROM_TO(ROL64ri, ROR64r1, 63)
+    FROM_TO(ROR64ri, ROL64r1, 63)
+    FROM_TO(ROL64mi, ROR64m1, 63)
+    FROM_TO(ROR64mi, ROL64m1, 63)
+  }
+  MI.setOpcode(NewOpc);
+  MI.erase(&LastOp);
+  return true;
+}

llvm/lib/Target/X86/X86ISelLowering.cpp

       .addImm(8);
 
   // Jump if the result of the shift is zero.
   BuildMI(fixShadowMBB, DL, TII->get(X86::JCC_1)).addMBB(sinkMBB).addImm(X86::COND_E);
   fixShadowMBB->addSuccessor(sinkMBB);
   fixShadowMBB->addSuccessor(fixShadowLoopPrepareMBB);
 
   // Do a single shift left.
-  unsigned ShlR1Opc = (PVT == MVT::i64) ? X86::SHL64r1 : X86::SHL32r1;
+  unsigned ShlR1Opc = (PVT == MVT::i64) ? X86::SHL64ri : X86::SHL32ri;
   Register SspAfterShlReg = MRI.createVirtualRegister(PtrRC);
   BuildMI(fixShadowLoopPrepareMBB, DL, TII->get(ShlR1Opc), SspAfterShlReg)
-      .addReg(SspSecondShrReg);
+      .addReg(SspSecondShrReg)
+      .addImm(1);
 
   // Save the value 128 to a register (will be used next with incssp).
   Register Value128InReg = MRI.createVirtualRegister(PtrRC);
   unsigned MovRIOpc = (PVT == MVT::i64) ? X86::MOV64ri32 : X86::MOV32ri;
   BuildMI(fixShadowLoopPrepareMBB, DL, TII->get(MovRIOpc), Value128InReg)
       .addImm(128);
   fixShadowLoopPrepareMBB->addSuccessor(fixShadowLoopMBB);
 

llvm/lib/Target/X86/X86InstrInfo.cpp

inline static bool isDefConvertible(const MachineInstr &MI, bool &NoSignFlag,
   case X86::ADC16rr:   case X86::ADC8rr:   case X86::ADC64rm:
   case X86::ADC32rm:   case X86::ADC16rm:  case X86::ADC8rm:
   case X86::SBB64ri32: case X86::SBB64ri8: case X86::SBB32ri:
   case X86::SBB32ri8:  case X86::SBB16ri:  case X86::SBB16ri8:
   case X86::SBB8ri:    case X86::SBB64rr:  case X86::SBB32rr:
   case X86::SBB16rr:   case X86::SBB8rr:   case X86::SBB64rm:
   case X86::SBB32rm:   case X86::SBB16rm:  case X86::SBB8rm:
   case X86::NEG8r:     case X86::NEG16r:   case X86::NEG32r: case X86::NEG64r:
-  case X86::SAR8r1:    case X86::SAR16r1:  case X86::SAR32r1:case X86::SAR64r1:
-  case X86::SHR8r1:    case X86::SHR16r1:  case X86::SHR32r1:case X86::SHR64r1:
-  case X86::SHL8r1:    case X86::SHL16r1:  case X86::SHL32r1:case X86::SHL64r1:
   case X86::LZCNT16rr: case X86::LZCNT16rm:
   case X86::LZCNT32rr: case X86::LZCNT32rm:
   case X86::LZCNT64rr: case X86::LZCNT64rm:
   case X86::POPCNT16rr:case X86::POPCNT16rm:
   case X86::POPCNT32rr:case X86::POPCNT32rm:
   case X86::POPCNT64rr:case X86::POPCNT64rm:
   case X86::TZCNT16rr: case X86::TZCNT16rm:
   case X86::TZCNT32rr: case X86::TZCNT32rm:

llvm/lib/Target/X86/X86InstrShiftRotate.td

 //===-- X86InstrShiftRotate.td - Shift and Rotate Instrs ---*- tablegen -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file describes the shift and rotate instructions.
 //
 //===----------------------------------------------------------------------===//
 
 // FIXME: Someone needs to smear multipattern goodness all over this file.
 
-let Defs = [EFLAGS] in {
+let Defs = [EFLAGS], hasSideEffects = 0 in {
 
 let Constraints = "$src1 = $dst" in {
 let Uses = [CL], SchedRW = [WriteShiftCL] in {
 def SHL8rCL  : I<0xD2, MRM4r, (outs GR8 :$dst), (ins GR8 :$src1),
                  "shl{b}\t{%cl, $dst|$dst, cl}",
                  [(set GR8:$dst, (shl GR8:$src1, CL))]>;
 def SHL16rCL : I<0xD3, MRM4r, (outs GR16:$dst), (ins GR16:$src1),
                  "shl{w}\t{%cl, $dst|$dst, cl}",
def SHL32ri  : Ii8<0xC1, MRM4r, (outs GR32:$dst), (ins GR32:$src1, u8imm:$src2),
                    [(set GR32:$dst, (shl GR32:$src1, (i8 imm:$src2)))]>,
                    OpSize32;
 def SHL64ri  : RIi8<0xC1, MRM4r, (outs GR64:$dst),
                     (ins GR64:$src1, u8imm:$src2),
                     "shl{q}\t{$src2, $dst|$dst, $src2}",
                     [(set GR64:$dst, (shl GR64:$src1, (i8 imm:$src2)))]>;
 } // isConvertibleToThreeAddress = 1
 
-// NOTE: We don't include patterns for shifts of a register by one, because
-// 'add reg,reg' is cheaper (and we have a Pat pattern for shift-by-one).
-let hasSideEffects = 0 in {
 def SHL8r1   : I<0xD0, MRM4r, (outs GR8:$dst), (ins GR8:$src1),
                  "shl{b}\t$dst", []>;
 def SHL16r1  : I<0xD1, MRM4r, (outs GR16:$dst), (ins GR16:$src1),
                  "shl{w}\t$dst", []>, OpSize16;
 def SHL32r1  : I<0xD1, MRM4r, (outs GR32:$dst), (ins GR32:$src1),
                  "shl{l}\t$dst", []>, OpSize32;
 def SHL64r1  : RI<0xD1, MRM4r, (outs GR64:$dst), (ins GR64:$src1),
                  "shl{q}\t$dst", []>;
-} // hasSideEffects = 0
 } // SchedRW
 } // Constraints = "$src = $dst"
 
 // FIXME: Why do we need an explicit "Uses = [CL]" when the instr has a pattern
 // using CL?
 let Uses = [CL], SchedRW = [WriteShiftCLLd, WriteRMW] in {
 def SHL8mCL  : I<0xD2, MRM4m, (outs), (ins i8mem :$dst),
                  "shl{b}\t{%cl, $dst|$dst, cl}",
                  [(store (shl (loadi8 addr:$dst), CL), addr:$dst)]>;
 def SHL16mCL : I<0xD3, MRM4m, (outs), (ins i16mem:$dst),
                  "shl{w}\t{%cl, $dst|$dst, cl}",
                  [(store (shl (loadi16 addr:$dst), CL), addr:$dst)]>,
                  OpSize16;
 def SHL32mCL : I<0xD3, MRM4m, (outs), (ins i32mem:$dst),
                  "shl{l}\t{%cl, $dst|$dst, cl}",
                  [(store (shl (loadi32 addr:$dst), CL), addr:$dst)]>,
                  OpSize32;
 def SHL64mCL : RI<0xD3, MRM4m, (outs), (ins i64mem:$dst),
                   "shl{q}\t{%cl, $dst|$dst, cl}",
                   [(store (shl (loadi64 addr:$dst), CL), addr:$dst)]>,
                   Requires<[In64BitMode]>;
 } // Uses, SchedRW
 
-let SchedRW = [WriteShiftLd, WriteRMW] in {
+let SchedRW = [WriteShiftLd, WriteRMW], mayLoad = 1, mayStore = 1 in {
 def SHL8mi   : Ii8<0xC0, MRM4m, (outs), (ins i8mem :$dst, u8imm:$src),
                    "shl{b}\t{$src, $dst|$dst, $src}",
                 [(store (shl (loadi8 addr:$dst), (i8 imm:$src)), addr:$dst)]>;
 def SHL16mi  : Ii8<0xC1, MRM4m, (outs), (ins i16mem:$dst, u8imm:$src),
                    "shl{w}\t{$src, $dst|$dst, $src}",
                [(store (shl (loadi16 addr:$dst), (i8 imm:$src)), addr:$dst)]>,
                OpSize16;
 def SHL32mi  : Ii8<0xC1, MRM4m, (outs), (ins i32mem:$dst, u8imm:$src),
                    "shl{l}\t{$src, $dst|$dst, $src}",
                [(store (shl (loadi32 addr:$dst), (i8 imm:$src)), addr:$dst)]>,
                OpSize32;
 def SHL64mi : RIi8<0xC1, MRM4m, (outs), (ins i64mem:$dst, u8imm:$src),
                   "shl{q}\t{$src, $dst|$dst, $src}",
                   [(store (shl (loadi64 addr:$dst), (i8 imm:$src)), addr:$dst)]>,
                   Requires<[In64BitMode]>;
 
 // Shift by 1
 def SHL8m1   : I<0xD0, MRM4m, (outs), (ins i8mem :$dst),
-                 "shl{b}\t$dst",
-                [(store (shl (loadi8 addr:$dst), (i8 1)), addr:$dst)]>;
+                 "shl{b}\t$dst", []>;
 def SHL16m1  : I<0xD1, MRM4m, (outs), (ins i16mem:$dst),
-                 "shl{w}\t$dst",
-                 [(store (shl (loadi16 addr:$dst), (i8 1)), addr:$dst)]>,
-                 OpSize16;
+                 "shl{w}\t$dst", []>, OpSize16;
 def SHL32m1  : I<0xD1, MRM4m, (outs), (ins i32mem:$dst),
-                 "shl{l}\t$dst",
-                 [(store (shl (loadi32 addr:$dst), (i8 1)), addr:$dst)]>,
-                 OpSize32;
+                 "shl{l}\t$dst", []>, OpSize32;
 def SHL64m1 : RI<0xD1, MRM4m, (outs), (ins i64mem:$dst),
-                  "shl{q}\t$dst",
-                 [(store (shl (loadi64 addr:$dst), (i8 1)), addr:$dst)]>,
-                 Requires<[In64BitMode]>;
-} // SchedRW
+                  "shl{q}\t$dst", []>, Requires<[In64BitMode]>;
+} // SchedRW, mayLoad, mayStore
 
 let Constraints = "$src1 = $dst" in {
 let Uses = [CL], SchedRW = [WriteShiftCL] in {
 def SHR8rCL  : I<0xD2, MRM5r, (outs GR8 :$dst), (ins GR8 :$src1),
                  "shr{b}\t{%cl, $dst|$dst, cl}",
                  [(set GR8:$dst, (srl GR8:$src1, CL))]>;
 def SHR16rCL : I<0xD3, MRM5r, (outs GR16:$dst), (ins GR16:$src1),
                  "shr{w}\t{%cl, $dst|$dst, cl}",
def SHR32ri  : Ii8<0xC1, MRM5r, (outs GR32:$dst), (ins GR32:$src1, u8imm:$src2),
                    [(set GR32:$dst, (srl GR32:$src1, (i8 imm:$src2)))]>,
                    OpSize32;
 def SHR64ri : RIi8<0xC1, MRM5r, (outs GR64:$dst), (ins GR64:$src1, u8imm:$src2),
                   "shr{q}\t{$src2, $dst|$dst, $src2}",
                   [(set GR64:$dst, (srl GR64:$src1, (i8 imm:$src2)))]>;
 
 // Shift right by 1
 def SHR8r1   : I<0xD0, MRM5r, (outs GR8:$dst), (ins GR8:$src1),
-                 "shr{b}\t$dst",
-                 [(set GR8:$dst, (srl GR8:$src1, (i8 1)))]>;
+                 "shr{b}\t$dst", []>;
 def SHR16r1  : I<0xD1, MRM5r, (outs GR16:$dst), (ins GR16:$src1),
-                 "shr{w}\t$dst",
-                 [(set GR16:$dst, (srl GR16:$src1, (i8 1)))]>, OpSize16;
+                 "shr{w}\t$dst", []>, OpSize16;
 def SHR32r1  : I<0xD1, MRM5r, (outs GR32:$dst), (ins GR32:$src1),
-                 "shr{l}\t$dst",
-                 [(set GR32:$dst, (srl GR32:$src1, (i8 1)))]>, OpSize32;
+                 "shr{l}\t$dst", []>, OpSize32;
 def SHR64r1  : RI<0xD1, MRM5r, (outs GR64:$dst), (ins GR64:$src1),
-                 "shr{q}\t$dst",
-                 [(set GR64:$dst, (srl GR64:$src1, (i8 1)))]>;
+                 "shr{q}\t$dst", []>;
 } // SchedRW
 } // Constraints = "$src = $dst"
 
 
 let Uses = [CL], SchedRW = [WriteShiftCLLd, WriteRMW] in {
 def SHR8mCL  : I<0xD2, MRM5m, (outs), (ins i8mem :$dst),
                  "shr{b}\t{%cl, $dst|$dst, cl}",
                  [(store (srl (loadi8 addr:$dst), CL), addr:$dst)]>;
 def SHR16mCL : I<0xD3, MRM5m, (outs), (ins i16mem:$dst),
                  "shr{w}\t{%cl, $dst|$dst, cl}",
                  [(store (srl (loadi16 addr:$dst), CL), addr:$dst)]>,
                  OpSize16;
 def SHR32mCL : I<0xD3, MRM5m, (outs), (ins i32mem:$dst),
                  "shr{l}\t{%cl, $dst|$dst, cl}",
                  [(store (srl (loadi32 addr:$dst), CL), addr:$dst)]>,
                  OpSize32;
 def SHR64mCL : RI<0xD3, MRM5m, (outs), (ins i64mem:$dst),
                   "shr{q}\t{%cl, $dst|$dst, cl}",
                   [(store (srl (loadi64 addr:$dst), CL), addr:$dst)]>,
                   Requires<[In64BitMode]>;
 } // Uses, SchedRW
 
-let SchedRW = [WriteShiftLd, WriteRMW] in {
+let SchedRW = [WriteShiftLd, WriteRMW], mayLoad = 1, mayStore = 1 in {
 def SHR8mi   : Ii8<0xC0, MRM5m, (outs), (ins i8mem :$dst, u8imm:$src),
                    "shr{b}\t{$src, $dst|$dst, $src}",
                 [(store (srl (loadi8 addr:$dst), (i8 imm:$src)), addr:$dst)]>;
 def SHR16mi  : Ii8<0xC1, MRM5m, (outs), (ins i16mem:$dst, u8imm:$src),
                    "shr{w}\t{$src, $dst|$dst, $src}",
                [(store (srl (loadi16 addr:$dst), (i8 imm:$src)), addr:$dst)]>,
                OpSize16;
 def SHR32mi  : Ii8<0xC1, MRM5m, (outs), (ins i32mem:$dst, u8imm:$src),
                    "shr{l}\t{$src, $dst|$dst, $src}",
                [(store (srl (loadi32 addr:$dst), (i8 imm:$src)), addr:$dst)]>,
                OpSize32;
 def SHR64mi : RIi8<0xC1, MRM5m, (outs), (ins i64mem:$dst, u8imm:$src),
                   "shr{q}\t{$src, $dst|$dst, $src}",
                  [(store (srl (loadi64 addr:$dst), (i8 imm:$src)), addr:$dst)]>,
                  Requires<[In64BitMode]>;
 
 // Shift by 1
 def SHR8m1   : I<0xD0, MRM5m, (outs), (ins i8mem :$dst),
-                 "shr{b}\t$dst",
-                 [(store (srl (loadi8 addr:$dst), (i8 1)), addr:$dst)]>;
+                 "shr{b}\t$dst", []>;
 def SHR16m1  : I<0xD1, MRM5m, (outs), (ins i16mem:$dst),
-                 "shr{w}\t$dst",
-                 [(store (srl (loadi16 addr:$dst), (i8 1)), addr:$dst)]>,
-                 OpSize16;
+                 "shr{w}\t$dst", []>, OpSize16;
 def SHR32m1  : I<0xD1, MRM5m, (outs), (ins i32mem:$dst),
-                 "shr{l}\t$dst",
-                 [(store (srl (loadi32 addr:$dst), (i8 1)), addr:$dst)]>,
-                 OpSize32;
+                 "shr{l}\t$dst", []>, OpSize32;
 def SHR64m1 : RI<0xD1, MRM5m, (outs), (ins i64mem:$dst),
-                  "shr{q}\t$dst",
-                 [(store (srl (loadi64 addr:$dst), (i8 1)), addr:$dst)]>,
-                 Requires<[In64BitMode]>;
-} // SchedRW
+                  "shr{q}\t$dst", []>, Requires<[In64BitMode]>;
+} // SchedRW, mayLoad, mayStore
+
 
 let Constraints = "$src1 = $dst" in {
 let Uses = [CL], SchedRW = [WriteShiftCL] in {
 def SAR8rCL  : I<0xD2, MRM7r, (outs GR8 :$dst), (ins GR8 :$src1),
                  "sar{b}\t{%cl, $dst|$dst, cl}",
                  [(set GR8:$dst, (sra GR8:$src1, CL))]>;
 def SAR16rCL : I<0xD3, MRM7r, (outs GR16:$dst), (ins GR16:$src1),
                  "sar{w}\t{%cl, $dst|$dst, cl}",
def SAR32ri  : Ii8<0xC1, MRM7r, (outs GR32:$dst), (ins GR32:$src1, u8imm:$src2),
                    OpSize32;
 def SAR64ri  : RIi8<0xC1, MRM7r, (outs GR64:$dst),
                     (ins GR64:$src1, u8imm:$src2),
                     "sar{q}\t{$src2, $dst|$dst, $src2}",
                     [(set GR64:$dst, (sra GR64:$src1, (i8 imm:$src2)))]>;
 
 // Shift by 1
 def SAR8r1   : I<0xD0, MRM7r, (outs GR8 :$dst), (ins GR8 :$src1),
-                 "sar{b}\t$dst",
-                 [(set GR8:$dst, (sra GR8:$src1, (i8 1)))]>;
+                 "sar{b}\t$dst", []>;
 def SAR16r1  : I<0xD1, MRM7r, (outs GR16:$dst), (ins GR16:$src1),
-                 "sar{w}\t$dst",
-                 [(set GR16:$dst, (sra GR16:$src1, (i8 1)))]>, OpSize16;
+                 "sar{w}\t$dst", []>, OpSize16;
 def SAR32r1  : I<0xD1, MRM7r, (outs GR32:$dst), (ins GR32:$src1),
-                 "sar{l}\t$dst",
-                 [(set GR32:$dst, (sra GR32:$src1, (i8 1)))]>, OpSize32;
+                 "sar{l}\t$dst", []>, OpSize32;
 def SAR64r1  : RI<0xD1, MRM7r, (outs GR64:$dst), (ins GR64:$src1),
-                  "sar{q}\t$dst",
-                  [(set GR64:$dst, (sra GR64:$src1, (i8 1)))]>;
+                  "sar{q}\t$dst", []>;
 } // SchedRW
 } // Constraints = "$src = $dst"
 
 
 let Uses = [CL], SchedRW = [WriteShiftCLLd, WriteRMW] in {
 def SAR8mCL  : I<0xD2, MRM7m, (outs), (ins i8mem :$dst),
                  "sar{b}\t{%cl, $dst|$dst, cl}",
                  [(store (sra (loadi8 addr:$dst), CL), addr:$dst)]>;
 def SAR16mCL : I<0xD3, MRM7m, (outs), (ins i16mem:$dst),
                  "sar{w}\t{%cl, $dst|$dst, cl}",
                  [(store (sra (loadi16 addr:$dst), CL), addr:$dst)]>,
                  OpSize16;
 def SAR32mCL : I<0xD3, MRM7m, (outs), (ins i32mem:$dst),
                  "sar{l}\t{%cl, $dst|$dst, cl}",
                  [(store (sra (loadi32 addr:$dst), CL), addr:$dst)]>,
                  OpSize32;
 def SAR64mCL : RI<0xD3, MRM7m, (outs), (ins i64mem:$dst),
                  "sar{q}\t{%cl, $dst|$dst, cl}",
                  [(store (sra (loadi64 addr:$dst), CL), addr:$dst)]>,
                  Requires<[In64BitMode]>;
 } // Uses, SchedRW
 
-let SchedRW = [WriteShiftLd, WriteRMW] in {
+let SchedRW = [WriteShiftLd, WriteRMW], mayLoad = 1, mayStore = 1 in {
 def SAR8mi   : Ii8<0xC0, MRM7m, (outs), (ins i8mem :$dst, u8imm:$src),
                    "sar{b}\t{$src, $dst|$dst, $src}",
                 [(store (sra (loadi8 addr:$dst), (i8 imm:$src)), addr:$dst)]>;
 def SAR16mi  : Ii8<0xC1, MRM7m, (outs), (ins i16mem:$dst, u8imm:$src),
                    "sar{w}\t{$src, $dst|$dst, $src}",
                [(store (sra (loadi16 addr:$dst), (i8 imm:$src)), addr:$dst)]>,
                OpSize16;
 def SAR32mi  : Ii8<0xC1, MRM7m, (outs), (ins i32mem:$dst, u8imm:$src),
                    "sar{l}\t{$src, $dst|$dst, $src}",
                [(store (sra (loadi32 addr:$dst), (i8 imm:$src)), addr:$dst)]>,
                OpSize32;
 def SAR64mi  : RIi8<0xC1, MRM7m, (outs), (ins i64mem:$dst, u8imm:$src),
                     "sar{q}\t{$src, $dst|$dst, $src}",
                  [(store (sra (loadi64 addr:$dst), (i8 imm:$src)), addr:$dst)]>,
                  Requires<[In64BitMode]>;
 
 // Shift by 1
 def SAR8m1   : I<0xD0, MRM7m, (outs), (ins i8mem :$dst),
-                 "sar{b}\t$dst",
-                [(store (sra (loadi8 addr:$dst), (i8 1)), addr:$dst)]>;
+                 "sar{b}\t$dst", []>;
 def SAR16m1  : I<0xD1, MRM7m, (outs), (ins i16mem:$dst),
-                 "sar{w}\t$dst",
-               [(store (sra (loadi16 addr:$dst), (i8 1)), addr:$dst)]>,
-               OpSize16;
+                 "sar{w}\t$dst", []>, OpSize16;
 def SAR32m1  : I<0xD1, MRM7m, (outs), (ins i32mem:$dst),
-                 "sar{l}\t$dst",
-               [(store (sra (loadi32 addr:$dst), (i8 1)), addr:$dst)]>,
-               OpSize32;
+                 "sar{l}\t$dst", []>, OpSize32;
 def SAR64m1 : RI<0xD1, MRM7m, (outs), (ins i64mem:$dst),
-                  "sar{q}\t$dst",
-                 [(store (sra (loadi64 addr:$dst), (i8 1)), addr:$dst)]>,
-                 Requires<[In64BitMode]>;
+                  "sar{q}\t$dst", []>, Requires<[In64BitMode]>;
 } // SchedRW
 
 //===----------------------------------------------------------------------===//
 // Rotate instructions
 //===----------------------------------------------------------------------===//
 
-let hasSideEffects = 0 in {
 let Constraints = "$src1 = $dst" in {
 
 let Uses = [CL, EFLAGS], SchedRW = [WriteRotateCL] in {
 def RCL8rCL : I<0xD2, MRM2r, (outs GR8:$dst), (ins GR8:$src1),
                 "rcl{b}\t{%cl, $dst|$dst, cl}", []>;
 def RCL16rCL : I<0xD3, MRM2r, (outs GR16:$dst), (ins GR16:$src1),
                  "rcl{w}\t{%cl, $dst|$dst, cl}", []>, OpSize16;
 def RCL32rCL : I<0xD3, MRM2r, (outs GR32:$dst), (ins GR32:$src1),
def RCR32r1 : I<0xD1, MRM3r, (outs GR32:$dst), (ins GR32:$src1),
                 "rcr{l}\t$dst", []>, OpSize32;
 def RCR32ri : Ii8<0xC1, MRM3r, (outs GR32:$dst), (ins GR32:$src1, u8imm:$cnt),
                   "rcr{l}\t{$cnt, $dst|$dst, $cnt}", []>, OpSize32;
 def RCR64r1 : RI<0xD1, MRM3r, (outs GR64:$dst), (ins GR64:$src1),
                  "rcr{q}\t$dst", []>;
 def RCR64ri : RIi8<0xC1, MRM3r, (outs GR64:$dst), (ins GR64:$src1, u8imm:$cnt),
                    "rcr{q}\t{$cnt, $dst|$dst, $cnt}", []>;
 } // Uses = [EFLAGS], SchedRW
-
 } // Constraints = "$src = $dst"
 
 let mayLoad = 1, mayStore = 1 in {
 let Uses = [EFLAGS], SchedRW = [WriteRotateLd, WriteRMW] in {
 def RCL8m1 : I<0xD0, MRM2m, (outs), (ins i8mem:$dst),
                "rcl{b}\t$dst", []>;
 def RCL8mi : Ii8<0xC0, MRM2m, (outs), (ins i8mem:$dst, u8imm:$cnt),
                  "rcl{b}\t{$cnt, $dst|$dst, $cnt}", []>;
 def RCL16m1 : I<0xD1, MRM2m, (outs), (ins i16mem:$dst),
                 "rcl{w}\t$dst", []>, OpSize16;
 def RCL16mi : Ii8<0xC1, MRM2m, (outs), (ins i16mem:$dst, u8imm:$cnt),
def RCR16mCL : I<0xD3, MRM3m, (outs), (ins i16mem:$dst),
                  "rcr{w}\t{%cl, $dst|$dst, cl}", []>, OpSize16;
 def RCR32mCL : I<0xD3, MRM3m, (outs), (ins i32mem:$dst),
                  "rcr{l}\t{%cl, $dst|$dst, cl}", []>, OpSize32;
 def RCR64mCL : RI<0xD3, MRM3m, (outs), (ins i64mem:$dst),
                   "rcr{q}\t{%cl, $dst|$dst, cl}", []>,
                   Requires<[In64BitMode]>;
 } // Uses = [CL, EFLAGS], SchedRW
 } // mayLoad, mayStore
-} // hasSideEffects = 0
 
 let Constraints = "$src1 = $dst" in {
 // FIXME: provide shorter instructions when imm8 == 1
 let Uses = [CL], SchedRW = [WriteRotateCL] in {
 def ROL8rCL  : I<0xD2, MRM0r, (outs GR8 :$dst), (ins GR8 :$src1),
                  "rol{b}\t{%cl, $dst|$dst, cl}",
                  [(set GR8:$dst, (rotl GR8:$src1, CL))]>;
 def ROL16rCL : I<0xD3, MRM0r, (outs GR16:$dst), (ins GR16:$src1),
def ROL32ri  : Ii8<0xC1, MRM0r, (outs GR32:$dst), (ins GR32:$src1, u8imm:$src2),
                    OpSize32;
 def ROL64ri  : RIi8<0xC1, MRM0r, (outs GR64:$dst),
                     (ins GR64:$src1, u8imm:$src2),
                     "rol{q}\t{$src2, $dst|$dst, $src2}",
                     [(set GR64:$dst, (rotl GR64:$src1, (i8 imm:$src2)))]>;
 
 // Rotate by 1
 def ROL8r1   : I<0xD0, MRM0r, (outs GR8 :$dst), (ins GR8 :$src1),
-                 "rol{b}\t$dst",
-                 [(set GR8:$dst, (rotl GR8:$src1, (i8 1)))]>;
+                 "rol{b}\t$dst", []>;
 def ROL16r1  : I<0xD1, MRM0r, (outs GR16:$dst), (ins GR16:$src1),
-                 "rol{w}\t$dst",
-                 [(set GR16:$dst, (rotl GR16:$src1, (i8 1)))]>, OpSize16;
+                 "rol{w}\t$dst", []>, OpSize16;
 def ROL32r1  : I<0xD1, MRM0r, (outs GR32:$dst), (ins GR32:$src1),
-                 "rol{l}\t$dst",
-                 [(set GR32:$dst, (rotl GR32:$src1, (i8 1)))]>, OpSize32;
+                 "rol{l}\t$dst", []>, OpSize32;
 def ROL64r1  : RI<0xD1, MRM0r, (outs GR64:$dst), (ins GR64:$src1),
-                  "rol{q}\t$dst",
-                  [(set GR64:$dst, (rotl GR64:$src1, (i8 1)))]>;
+                  "rol{q}\t$dst", []>;
 } // SchedRW
 } // Constraints = "$src = $dst"
 
 let Uses = [CL], SchedRW = [WriteRotateCLLd, WriteRMW] in {
 def ROL8mCL  : I<0xD2, MRM0m, (outs), (ins i8mem :$dst),
                  "rol{b}\t{%cl, $dst|$dst, cl}",
                  [(store (rotl (loadi8 addr:$dst), CL), addr:$dst)]>;
 def ROL16mCL : I<0xD3, MRM0m, (outs), (ins i16mem:$dst),
                  "rol{w}\t{%cl, $dst|$dst, cl}",
                  [(store (rotl (loadi16 addr:$dst), CL), addr:$dst)]>, OpSize16;
 def ROL32mCL : I<0xD3, MRM0m, (outs), (ins i32mem:$dst),
                  "rol{l}\t{%cl, $dst|$dst, cl}",
                  [(store (rotl (loadi32 addr:$dst), CL), addr:$dst)]>, OpSize32;
 def ROL64mCL :  RI<0xD3, MRM0m, (outs), (ins i64mem:$dst),
                    "rol{q}\t{%cl, $dst|$dst, cl}",
                    [(store (rotl (loadi64 addr:$dst), CL), addr:$dst)]>,
                    Requires<[In64BitMode]>;
 } // Uses, SchedRW
 
-let SchedRW = [WriteRotateLd, WriteRMW] in {
+let SchedRW = [WriteRotateLd, WriteRMW], mayLoad = 1, mayStore = 1 in {
 def ROL8mi   : Ii8<0xC0, MRM0m, (outs), (ins i8mem :$dst, u8imm:$src1),
                    "rol{b}\t{$src1, $dst|$dst, $src1}",
                [(store (rotl (loadi8 addr:$dst), (i8 imm:$src1)), addr:$dst)]>;
 def ROL16mi  : Ii8<0xC1, MRM0m, (outs), (ins i16mem:$dst, u8imm:$src1),
                    "rol{w}\t{$src1, $dst|$dst, $src1}",
               [(store (rotl (loadi16 addr:$dst), (i8 imm:$src1)), addr:$dst)]>,
               OpSize16;
 def ROL32mi  : Ii8<0xC1, MRM0m, (outs), (ins i32mem:$dst, u8imm:$src1),
                    "rol{l}\t{$src1, $dst|$dst, $src1}",
               [(store (rotl (loadi32 addr:$dst), (i8 imm:$src1)), addr:$dst)]>,
               OpSize32;
 def ROL64mi  : RIi8<0xC1, MRM0m, (outs), (ins i64mem:$dst, u8imm:$src1),
                     "rol{q}\t{$src1, $dst|$dst, $src1}",
                 [(store (rotl (loadi64 addr:$dst), (i8 imm:$src1)), addr:$dst)]>,
                 Requires<[In64BitMode]>;
 
 // Rotate by 1
 def ROL8m1   : I<0xD0, MRM0m, (outs), (ins i8mem :$dst),
-                 "rol{b}\t$dst",
-                 [(store (rotl (loadi8 addr:$dst), (i8 1)), addr:$dst)]>;
+                 "rol{b}\t$dst", []>;
 def ROL16m1  : I<0xD1, MRM0m, (outs), (ins i16mem:$dst),
-                 "rol{w}\t$dst",
-                 [(store (rotl (loadi16 addr:$dst), (i8 1)), addr:$dst)]>,
-                 OpSize16;
+                 "rol{w}\t$dst", []>, OpSize16;
 def ROL32m1  : I<0xD1, MRM0m, (outs), (ins i32mem:$dst),
-                 "rol{l}\t$dst",
-                 [(store (rotl (loadi32 addr:$dst), (i8 1)), addr:$dst)]>,
-                 OpSize32;
+                 "rol{l}\t$dst", []>, OpSize32;
 def ROL64m1  : RI<0xD1, MRM0m, (outs), (ins i64mem:$dst),
-                 "rol{q}\t$dst",
-                 [(store (rotl (loadi64 addr:$dst), (i8 1)), addr:$dst)]>,
-                 Requires<[In64BitMode]>;
-} // SchedRW
+                 "rol{q}\t$dst", []>, Requires<[In64BitMode]>;
+} // SchedRW, mayLoad, mayStore
 
 let Constraints = "$src1 = $dst" in {
 let Uses = [CL], SchedRW = [WriteRotateCL] in {
 def ROR8rCL  : I<0xD2, MRM1r, (outs GR8 :$dst), (ins GR8 :$src1),
                  "ror{b}\t{%cl, $dst|$dst, cl}",
                  [(set GR8:$dst, (rotr GR8:$src1, CL))]>;
 def ROR16rCL : I<0xD3, MRM1r, (outs GR16:$dst), (ins GR16:$src1),
                  "ror{w}\t{%cl, $dst|$dst, cl}",
def ROR32ri  : Ii8<0xC1, MRM1r, (outs GR32:$dst), (ins GR32:$src1, u8imm:$src2),
                    OpSize32;
 def ROR64ri  : RIi8<0xC1, MRM1r, (outs GR64:$dst),
                     (ins GR64:$src1, u8imm:$src2),
                     "ror{q}\t{$src2, $dst|$dst, $src2}",
                     [(set GR64:$dst, (rotr GR64:$src1, (i8 imm:$src2)))]>;
 
 // Rotate by 1
 def ROR8r1   : I<0xD0, MRM1r, (outs GR8 :$dst), (ins GR8 :$src1),
-                 "ror{b}\t$dst",
-                 [(set GR8:$dst, (rotr GR8:$src1, (i8 1)))]>;
+                 "ror{b}\t$dst", []>;
 def ROR16r1  : I<0xD1, MRM1r, (outs GR16:$dst), (ins GR16:$src1),
-                 "ror{w}\t$dst",
-                 [(set GR16:$dst, (rotr GR16:$src1, (i8 1)))]>, OpSize16;
+                 "ror{w}\t$dst", []>, OpSize16;
 def ROR32r1  : I<0xD1, MRM1r, (outs GR32:$dst), (ins GR32:$src1),
-                 "ror{l}\t$dst",
-                 [(set GR32:$dst, (rotr GR32:$src1, (i8 1)))]>, OpSize32;
+                 "ror{l}\t$dst", []>, OpSize32;
 def ROR64r1  : RI<0xD1, MRM1r, (outs GR64:$dst), (ins GR64:$src1),
-                  "ror{q}\t$dst",
-                  [(set GR64:$dst, (rotr GR64:$src1, (i8 1)))]>;
+                  "ror{q}\t$dst", []>;
 } // SchedRW
 } // Constraints = "$src = $dst", SchedRW
 
 let Uses = [CL], SchedRW = [WriteRotateCLLd, WriteRMW] in {
 def ROR8mCL  : I<0xD2, MRM1m, (outs), (ins i8mem :$dst),
                  "ror{b}\t{%cl, $dst|$dst, cl}",
                  [(store (rotr (loadi8 addr:$dst), CL), addr:$dst)]>;
 def ROR16mCL : I<0xD3, MRM1m, (outs), (ins i16mem:$dst),
                  "ror{w}\t{%cl, $dst|$dst, cl}",
                  [(store (rotr (loadi16 addr:$dst), CL), addr:$dst)]>, OpSize16;
 def ROR32mCL : I<0xD3, MRM1m, (outs), (ins i32mem:$dst),
                  "ror{l}\t{%cl, $dst|$dst, cl}",
                  [(store (rotr (loadi32 addr:$dst), CL), addr:$dst)]>, OpSize32;
 def ROR64mCL : RI<0xD3, MRM1m, (outs), (ins i64mem:$dst),
                   "ror{q}\t{%cl, $dst|$dst, cl}",
                   [(store (rotr (loadi64 addr:$dst), CL), addr:$dst)]>,
                   Requires<[In64BitMode]>;
 } // Uses, SchedRW
 
-let SchedRW = [WriteRotateLd, WriteRMW] in {
+let SchedRW = [WriteRotateLd, WriteRMW], mayLoad = 1, mayStore =1 in {
 def ROR8mi   : Ii8<0xC0, MRM1m, (outs), (ins i8mem :$dst, u8imm:$src),
                    "ror{b}\t{$src, $dst|$dst, $src}",
                    [(store (rotr (loadi8 addr:$dst), (i8 imm:$src)), addr:$dst)]>;
 def ROR16mi  : Ii8<0xC1, MRM1m, (outs), (ins i16mem:$dst, u8imm:$src),
                    "ror{w}\t{$src, $dst|$dst, $src}",
                    [(store (rotr (loadi16 addr:$dst), (i8 imm:$src)), addr:$dst)]>,
                    OpSize16;
 def ROR32mi  : Ii8<0xC1, MRM1m, (outs), (ins i32mem:$dst, u8imm:$src),
                    "ror{l}\t{$src, $dst|$dst, $src}",
                    [(store (rotr (loadi32 addr:$dst), (i8 imm:$src)), addr:$dst)]>,
                    OpSize32;
 def ROR64mi  : RIi8<0xC1, MRM1m, (outs), (ins i64mem:$dst, u8imm:$src),
                     "ror{q}\t{$src, $dst|$dst, $src}",
                     [(store (rotr (loadi64 addr:$dst), (i8 imm:$src)), addr:$dst)]>,
                     Requires<[In64BitMode]>;
 
 // Rotate by 1
 def ROR8m1   : I<0xD0, MRM1m, (outs), (ins i8mem :$dst),
-                 "ror{b}\t$dst",
-                 [(store (rotr (loadi8 addr:$dst), (i8 1)), addr:$dst)]>;
+                 "ror{b}\t$dst", []>;
 def ROR16m1  : I<0xD1, MRM1m, (outs), (ins i16mem:$dst),
-                 "ror{w}\t$dst",
-                 [(store (rotr (loadi16 addr:$dst), (i8 1)), addr:$dst)]>,
-                 OpSize16;
+                 "ror{w}\t$dst", []>, OpSize16;
 def ROR32m1  : I<0xD1, MRM1m, (outs), (ins i32mem:$dst),
-                 "ror{l}\t$dst",
-                 [(store (rotr (loadi32 addr:$dst), (i8 1)), addr:$dst)]>,
+                 "ror{l}\t$dst", []>,
                  OpSize32;
 def ROR64m1  : RI<0xD1, MRM1m, (outs), (ins i64mem:$dst),
-                 "ror{q}\t$dst",
-                 [(store (rotr (loadi64 addr:$dst), (i8 1)), addr:$dst)]>,
-                 Requires<[In64BitMode]>;
-} // SchedRW
+                 "ror{q}\t$dst", []>, Requires<[In64BitMode]>;
+} // SchedRW, mayLoad, mayStore
 
 
 //===----------------------------------------------------------------------===//
 // Double shift instructions (generalizations of rotate)
 //===----------------------------------------------------------------------===//
 
 let Constraints = "$src1 = $dst" in {
 

 def SHRD64mri8 : RIi8<0xAC, MRMDestMem,
                       (outs), (ins i64mem:$dst, GR64:$src2, u8imm:$src3),
                       "shrd{q}\t{$src3, $src2, $dst|$dst, $src2, $src3}",
                       [(store (fshr GR64:$src2, (loadi64 addr:$dst),
                                     (i8 imm:$src3)), addr:$dst)]>,
                  TB;
 } // SchedRW
 
-} // Defs = [EFLAGS]
-
-// Use the opposite rotate if allows us to use the rotate by 1 instruction.
-def : Pat<(rotl GR8:$src1,  (i8 7)),  (ROR8r1  GR8:$src1)>;
-def : Pat<(rotl GR16:$src1, (i8 15)), (ROR16r1 GR16:$src1)>;
-def : Pat<(rotl GR32:$src1, (i8 31)), (ROR32r1 GR32:$src1)>;
-def : Pat<(rotl GR64:$src1, (i8 63)), (ROR64r1 GR64:$src1)>;
-def : Pat<(rotr GR8:$src1,  (i8 7)),  (ROL8r1  GR8:$src1)>;
-def : Pat<(rotr GR16:$src1, (i8 15)), (ROL16r1 GR16:$src1)>;
-def : Pat<(rotr GR32:$src1, (i8 31)), (ROL32r1 GR32:$src1)>;
-def : Pat<(rotr GR64:$src1, (i8 63)), (ROL64r1 GR64:$src1)>;
-
-def : Pat<(store (rotl (loadi8 addr:$dst), (i8 7)), addr:$dst),
-          (ROR8m1 addr:$dst)>;
-def : Pat<(store (rotl (loadi16 addr:$dst), (i8 15)), addr:$dst),
-          (ROR16m1 addr:$dst)>;
-def : Pat<(store (rotl (loadi32 addr:$dst), (i8 31)), addr:$dst),
-          (ROR32m1 addr:$dst)>;
-def : Pat<(store (rotl (loadi64 addr:$dst), (i8 63)), addr:$dst),
-          (ROR64m1 addr:$dst)>, Requires<[In64BitMode]>;
-
-def : Pat<(store (rotr (loadi8 addr:$dst), (i8 7)), addr:$dst),
-          (ROL8m1 addr:$dst)>;
-def : Pat<(store (rotr (loadi16 addr:$dst), (i8 15)), addr:$dst),
-          (ROL16m1 addr:$dst)>;
-def : Pat<(store (rotr (loadi32 addr:$dst), (i8 31)), addr:$dst),
-          (ROL32m1 addr:$dst)>;
-def : Pat<(store (rotr (loadi64 addr:$dst), (i8 63)), addr:$dst),
-          (ROL64m1 addr:$dst)>, Requires<[In64BitMode]>;
+} // Defs = [EFLAGS], hasSideEffects
 
 // Sandy Bridge and newer Intel processors support faster rotates using
 // SHLD to avoid a partial flag update on the normal rotate instructions.
 // Use a pseudo so that TwoInstructionPass and register allocation will see
 // this as unary instruction.
 let Predicates = [HasFastSHLDRotate], AddedComplexity = 5,
     Defs = [EFLAGS], isPseudo = 1, SchedRW = [WriteSHDrri],
     Constraints = "$src1 = $dst" in {

llvm/lib/Target/X86/X86MCInstLower.cpp

void X86MCInstLower::Lower(const MachineInstr *MI, MCInst &OutMI) const {
 
   for (const MachineOperand &MO : MI->operands())
     if (auto MaybeMCOp = LowerMachineOperand(MI, MO))
       OutMI.addOperand(*MaybeMCOp);
 
   if (X86::optimizeInstFromVEX3ToVEX2(OutMI))
     return;
 
+  X86::optimizeRotateWithImmediate(OutMI);
+
+  if (X86::optimizeShiftRotateWithImmediateOne(OutMI))
+    return;
+
   // Handle a few special cases to eliminate operand modifiers.
   switch (OutMI.getOpcode()) {
   case X86::LEA64_32r:
   case X86::LEA64r:
   case X86::LEA16r:
   case X86::LEA32r:
     // LEA should have a segment register, but it must be empty.
     assert(OutMI.getNumOperands() == 1 + X86::AddrNumOperands &&

llvm/test/CodeGen/X86/GlobalISel/ashr-scalar.ll

 }
 
 define i1 @test_ashr_i1_imm1(i32 %arg1) {
 ; X64-LABEL: test_ashr_i1_imm1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shlb $7, %al
 ; X64-NEXT:    sarb $7, %al
-; X64-NEXT:    sarb %al
+; X64-NEXT:    movb $1, %cl
+; X64-NEXT:    sarb %cl, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %a = trunc i32 %arg1 to i1
   %res = ashr i1 %a, 1
   ret i1 %res
 }

llvm/test/CodeGen/X86/GlobalISel/lshr-scalar.ll

; X64-NEXT:    retq
   ret i1 %res
 }
 
 define i1 @test_lshr_i1_imm1(i32 %arg1) {
 ; X64-LABEL: test_lshr_i1_imm1:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andb $1, %al
-; X64-NEXT:    shrb %al
+; X64-NEXT:    movb $1, %cl
+; X64-NEXT:    shrb %cl, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %a = trunc i32 %arg1 to i1
   %res = lshr i1 %a, 1
   ret i1 %res
 }

llvm/test/CodeGen/X86/GlobalISel/select-ashr-scalar.mir

 # NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
 # RUN: llc -mtriple=x86_64-linux-gnu -run-pass=instruction-select -verify-machineinstrs %s -o - | FileCheck %s --check-prefix=ALL
 --- |
 
   define i64 @test_ashr_i64(i64 %arg1, i64 %arg2) {
     %res = ashr i64 %arg1, %arg2
     ret i64 %res
   }
 
   define i64 @test_ashr_i64_imm(i64 %arg1) {
     %res = ashr i64 %arg1, 5
     ret i64 %res
   }
 
-  define i64 @test_ashr_i64_imm1(i64 %arg1) {
-    %res = ashr i64 %arg1, 1
-    ret i64 %res
-  }
-
   define i32 @test_ashr_i32(i32 %arg1, i32 %arg2) {
     %res = ashr i32 %arg1, %arg2
     ret i32 %res
   }
 
   define i32 @test_ashr_i32_imm(i32 %arg1) {
     %res = ashr i32 %arg1, 5
     ret i32 %res
   }
 
-  define i32 @test_ashr_i32_imm1(i32 %arg1) {
-    %res = ashr i32 %arg1, 1
-    ret i32 %res
-  }
-
   define i16 @test_ashr_i16(i32 %arg1, i32 %arg2) {
     %a = trunc i32 %arg1 to i16
     %a2 = trunc i32 %arg2 to i16
     %res = ashr i16 %a, %a2
     ret i16 %res
   }
 
   define i16 @test_ashr_i16_imm(i32 %arg1) {
     %a = trunc i32 %arg1 to i16
     %res = ashr i16 %a, 5
     ret i16 %res
   }
 
-  define i16 @test_ashr_i16_imm1(i32 %arg1) {
-    %a = trunc i32 %arg1 to i16
-    %res = ashr i16 %a, 1
-    ret i16 %res
-  }
-
   define i8 @test_ashr_i8(i32 %arg1, i32 %arg2) {
     %a = trunc i32 %arg1 to i8
     %a2 = trunc i32 %arg2 to i8
     %res = ashr i8 %a, %a2
     ret i8 %res
   }
 
   define i8 @test_ashr_i8_imm(i32 %arg1) {
     %a = trunc i32 %arg1 to i8
     %res = ashr i8 %a, 5
     ret i8 %res
   }
-
-  define i8 @test_ashr_i8_imm1(i32 %arg1) {
-    %a = trunc i32 %arg1 to i8
-    %res = ashr i8 %a, 1
-    ret i8 %res
-  }
 ...
 ---
 name:            test_ashr_i64
 alignment:       16
 legalized:       true
 regBankSelected: true
 tracksRegLiveness: true
 registers:
bb.1 (%ir-block.0):
     %0(s64) = COPY $rdi
     %1(s8) = G_CONSTANT i8 5
     %2(s64) = G_ASHR %0, %1
     $rax = COPY %2(s64)
     RET 0, implicit $rax
 
 ...
 ---
-name:            test_ashr_i64_imm1
-alignment:       16
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-registers:
-  - { id: 0, class: gpr, preferred-register: '' }
-  - { id: 1, class: gpr, preferred-register: '' }
-  - { id: 2, class: gpr, preferred-register: '' }
-liveins:
-fixedStack:
-stack:
-constants:
-body:             |
-  bb.1 (%ir-block.0):
-    liveins: $rdi
-
-    ; ALL-LABEL: name: test_ashr_i64_imm1
-    ; ALL: liveins: $rdi
-    ; ALL: [[COPY:%[0-9]+]]:gr64 = COPY $rdi
-    ; ALL: [[SAR64r1_:%[0-9]+]]:gr64 = SAR64r1 [[COPY]], implicit-def $eflags
-    ; ALL: $rax = COPY [[SAR64r1_]]
-    ; ALL: RET 0, implicit $rax
-    %0(s64) = COPY $rdi
-    %1(s8) = G_CONSTANT i8 1
-    %2(s64) = G_ASHR %0, %1
-    $rax = COPY %2(s64)
-    RET 0, implicit $rax
-
-...
----
 name:            test_ashr_i32
 alignment:       16
 legalized:       true
 regBankSelected: true
 tracksRegLiveness: true
 registers:
   - { id: 0, class: gpr, preferred-register: '' }
   - { id: 1, class: gpr, preferred-register: '' }
bb.1 (%ir-block.0):
     %0(s32) = COPY $edi
     %1(s8) = G_CONSTANT i8 5
     %2(s32) = G_ASHR %0, %1
     $eax = COPY %2(s32)
     RET 0, implicit $eax
 
 ...
 ---
-name:            test_ashr_i32_imm1
-alignment:       16
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-registers:
-  - { id: 0, class: gpr, preferred-register: '' }
-  - { id: 1, class: gpr, preferred-register: '' }
-  - { id: 2, class: gpr, preferred-register: '' }
-liveins:
-fixedStack:
-stack:
-constants:
-body:             |
-  bb.1 (%ir-block.0):
-    liveins: $edi
-
-    ; ALL-LABEL: name: test_ashr_i32_imm1
-    ; ALL: liveins: $edi
-    ; ALL: [[COPY:%[0-9]+]]:gr32 = COPY $edi
-    ; ALL: [[SAR32r1_:%[0-9]+]]:gr32 = SAR32r1 [[COPY]], implicit-def $eflags
-    ; ALL: $eax = COPY [[SAR32r1_]]
-    ; ALL: RET 0, implicit $eax
-    %0(s32) = COPY $edi
-    %1(s8) = G_CONSTANT i8 1
-    %2(s32) = G_ASHR %0, %1
-    $eax = COPY %2(s32)
-    RET 0, implicit $eax
-
-...
----
 name:            test_ashr_i16
 alignment:       16
 legalized:       true
 regBankSelected: true
 tracksRegLiveness: true
 registers:
   - { id: 0, class: gpr, preferred-register: '' }
   - { id: 1, class: gpr, preferred-register: '' }
bb.1 (%ir-block.0):
     %2(s8) = G_CONSTANT i8 5
     %1(s16) = G_TRUNC %0(s32)
     %3(s16) = G_ASHR %1, %2
     $ax = COPY %3(s16)
     RET 0, implicit $ax
 
 ...
 ---
-name:            test_ashr_i16_imm1
-alignment:       16
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-registers:
-  - { id: 0, class: gpr, preferred-register: '' }
-  - { id: 1, class: gpr, preferred-register: '' }
-  - { id: 2, class: gpr, preferred-register: '' }
-  - { id: 3, class: gpr, preferred-register: '' }
-liveins:
-fixedStack:
-stack:
-constants:
-body:             |
-  bb.1 (%ir-block.0):
-    liveins: $edi
-
-    ; ALL-LABEL: name: test_ashr_i16_imm1
-    ; ALL: liveins: $edi
-    ; ALL: [[COPY:%[0-9]+]]:gr32 = COPY $edi
-    ; ALL: [[COPY1:%[0-9]+]]:gr16 = COPY [[COPY]].sub_16bit
-    ; ALL: [[SAR16r1_:%[0-9]+]]:gr16 = SAR16r1 [[COPY1]], implicit-def $eflags
-    ; ALL: $ax = COPY [[SAR16r1_]]
-    ; ALL: RET 0, implicit $ax
-    %0(s32) = COPY $edi
-    %2(s8) = G_CONSTANT i8 1
-    %1(s16) = G_TRUNC %0(s32)
-    %3(s16) = G_ASHR %1, %2
-    $ax = COPY %3(s16)
-    RET 0, implicit $ax
-
-...
----
 name:            test_ashr_i8
 alignment:       16
 legalized:       true
 regBankSelected: true
 tracksRegLiveness: true
 registers:
   - { id: 0, class: gpr, preferred-register: '' }
   - { id: 1, class: gpr, preferred-register: '' }
bb.1 (%ir-block.0):
     %0(s32) = COPY $edi
     %2(s8) = G_CONSTANT i8 5
     %1(s8) = G_TRUNC %0(s32)
     %3(s8) = G_ASHR %1, %2
     $al = COPY %3(s8)
     RET 0, implicit $al
 
 ...
----
-name:            test_ashr_i8_imm1
-alignment:       16
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-registers:
-  - { id: 0, class: gpr, preferred-register: '' }
-  - { id: 1, class: gpr, preferred-register: '' }
-  - { id: 2, class: gpr, preferred-register: '' }
-  - { id: 3, class: gpr, preferred-register: '' }
-liveins:
-fixedStack:
-stack:
-constants:
-body:             |
-  bb.1 (%ir-block.0):
-    liveins: $edi
-
-    ; ALL-LABEL: name: test_ashr_i8_imm1
-    ; ALL: liveins: $edi
-    ; ALL: [[COPY:%[0-9]+]]:gr32 = COPY $edi
-    ; ALL: [[COPY1:%[0-9]+]]:gr8 = COPY [[COPY]].sub_8bit
-    ; ALL: [[SAR8r1_:%[0-9]+]]:gr8 = SAR8r1 [[COPY1]], implicit-def $eflags
-    ; ALL: $al = COPY [[SAR8r1_]]
-    ; ALL: RET 0, implicit $al
-    %0(s32) = COPY $edi
-    %2(s8) = G_CONSTANT i8 1
-    %1(s8) = G_TRUNC %0(s32)
-    %3(s8) = G_ASHR %1, %2
-    $al = COPY %3(s8)
-    RET 0, implicit $al
-
-...

llvm/test/CodeGen/X86/GlobalISel/select-lshr-scalar.mir

 # NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
 # RUN: llc -mtriple=x86_64-linux-gnu -run-pass=instruction-select -verify-machineinstrs %s -o - | FileCheck %s --check-prefix=ALL
 --- |
 
   define i64 @test_lshr_i64(i64 %arg1, i64 %arg2) {
     %res = lshr i64 %arg1, %arg2
     ret i64 %res
   }
 
   define i64 @test_lshr_i64_imm(i64 %arg1) {
     %res = lshr i64 %arg1, 5
     ret i64 %res
   }
 
-  define i64 @test_lshr_i64_imm1(i64 %arg1) {
-    %res = lshr i64 %arg1, 1
-    ret i64 %res
-  }
-
   define i32 @test_lshr_i32(i32 %arg1, i32 %arg2) {
     %res = lshr i32 %arg1, %arg2
     ret i32 %res
   }
 
   define i32 @test_lshr_i32_imm(i32 %arg1) {
     %res = lshr i32 %arg1, 5
     ret i32 %res
   }
 
-  define i32 @test_lshr_i32_imm1(i32 %arg1) {
-    %res = lshr i32 %arg1, 1
-    ret i32 %res
-  }
-
   define i16 @test_lshr_i16(i32 %arg1, i32 %arg2) {
     %a = trunc i32 %arg1 to i16
     %a2 = trunc i32 %arg2 to i16
     %res = lshr i16 %a, %a2
     ret i16 %res
   }
 
   define i16 @test_lshr_i16_imm(i32 %arg1) {
     %a = trunc i32 %arg1 to i16
     %res = lshr i16 %a, 5
     ret i16 %res
   }
 
-  define i16 @test_lshr_i16_imm1(i32 %arg1) {
-    %a = trunc i32 %arg1 to i16
-    %res = lshr i16 %a, 1
-    ret i16 %res
-  }
-
   define i8 @test_lshr_i8(i32 %arg1, i32 %arg2) {
     %a = trunc i32 %arg1 to i8
     %a2 = trunc i32 %arg2 to i8
     %res = lshr i8 %a, %a2
     ret i8 %res
   }
 
   define i8 @test_lshr_i8_imm(i32 %arg1) {
     %a = trunc i32 %arg1 to i8
     %res = lshr i8 %a, 5
     ret i8 %res
   }
 
-  define i8 @test_lshr_i8_imm1(i32 %arg1) {
-    %a = trunc i32 %arg1 to i8
-    %res = lshr i8 %a, 1
-    ret i8 %res
-  }
 ...
 ---
 name:            test_lshr_i64
 alignment:       16
 legalized:       true
 regBankSelected: true
 tracksRegLiveness: true
 registers:
bb.1 (%ir-block.0):
     %0(s64) = COPY $rdi
     %1(s8) = G_CONSTANT i8 5
     %2(s64) = G_LSHR %0, %1
     $rax = COPY %2(s64)
     RET 0, implicit $rax
 
 ...
 ---
-name:            test_lshr_i64_imm1
-alignment:       16
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-registers:
-  - { id: 0, class: gpr, preferred-register: '' }
-  - { id: 1, class: gpr, preferred-register: '' }
-  - { id: 2, class: gpr, preferred-register: '' }
-liveins:
-fixedStack:
-stack:
-constants:
-body:             |
-  bb.1 (%ir-block.0):
-    liveins: $rdi
-
-    ; ALL-LABEL: name: test_lshr_i64_imm1
-    ; ALL: liveins: $rdi
-    ; ALL: [[COPY:%[0-9]+]]:gr64 = COPY $rdi
-    ; ALL: [[SHR64r1_:%[0-9]+]]:gr64 = SHR64r1 [[COPY]], implicit-def $eflags
-    ; ALL: $rax = COPY [[SHR64r1_]]
-    ; ALL: RET 0, implicit $rax
-    %0(s64) = COPY $rdi
-    %1(s8) = G_CONSTANT i8 1
-    %2(s64) = G_LSHR %0, %1
-    $rax = COPY %2(s64)
-    RET 0, implicit $rax
-
-...
----
 name:            test_lshr_i32
 alignment:       16
 legalized:       true
 regBankSelected: true
 tracksRegLiveness: true
 registers:
   - { id: 0, class: gpr, preferred-register: '' }
   - { id: 1, class: gpr, preferred-register: '' }
bb.1 (%ir-block.0):
     %0(s32) = COPY $edi
     %1(s8) = G_CONSTANT i8 5
     %2(s32) = G_LSHR %0, %1
     $eax = COPY %2(s32)
     RET 0, implicit $eax
 
 ...
 ---
-name:            test_lshr_i32_imm1
-alignment:       16
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-registers:
-  - { id: 0, class: gpr, preferred-register: '' }
-  - { id: 1, class: gpr, preferred-register: '' }
-  - { id: 2, class: gpr, preferred-register: '' }
-liveins:
-fixedStack:
-stack:
-constants:
-body:             |
-  bb.1 (%ir-block.0):
-    liveins: $edi
-
-    ; ALL-LABEL: name: test_lshr_i32_imm1
-    ; ALL: liveins: $edi
-    ; ALL: [[COPY:%[0-9]+]]:gr32 = COPY $edi
-    ; ALL: [[SHR32r1_:%[0-9]+]]:gr32 = SHR32r1 [[COPY]], implicit-def $eflags
-    ; ALL: $eax = COPY [[SHR32r1_]]
-    ; ALL: RET 0, implicit $eax
-    %0(s32) = COPY $edi
-    %1(s8) = G_CONSTANT i8 1
-    %2(s32) = G_LSHR %0, %1
-    $eax = COPY %2(s32)
-    RET 0, implicit $eax
-
-...
----
 name:            test_lshr_i16
 alignment:       16
 legalized:       true
 regBankSelected: true
 tracksRegLiveness: true
 registers:
   - { id: 0, class: gpr, preferred-register: '' }
   - { id: 1, class: gpr, preferred-register: '' }
bb.1 (%ir-block.0):
     %2(s8) = G_CONSTANT i8 5
     %1(s16) = G_TRUNC %0(s32)
     %3(s16) = G_LSHR %1, %2
     $ax = COPY %3(s16)
     RET 0, implicit $ax
 
 ...
 ---
-name:            test_lshr_i16_imm1
-alignment:       16
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-registers:
-  - { id: 0, class: gpr, preferred-register: '' }
-  - { id: 1, class: gpr, preferred-register: '' }
-  - { id: 2, class: gpr, preferred-register: '' }
-  - { id: 3, class: gpr, preferred-register: '' }
-liveins:
-fixedStack:
-stack:
-constants:
-body:             |
-  bb.1 (%ir-block.0):
-    liveins: $edi
-
-    ; ALL-LABEL: name: test_lshr_i16_imm1
-    ; ALL: liveins: $edi
-    ; ALL: [[COPY:%[0-9]+]]:gr32 = COPY $edi
-    ; ALL: [[COPY1:%[0-9]+]]:gr16 = COPY [[COPY]].sub_16bit
-    ; ALL: [[SHR16r1_:%[0-9]+]]:gr16 = SHR16r1 [[COPY1]], implicit-def $eflags
-    ; ALL: $ax = COPY [[SHR16r1_]]
-    ; ALL: RET 0, implicit $ax
-    %0(s32) = COPY $edi
-    %2(s8) = G_CONSTANT i8 1
-    %1(s16) = G_TRUNC %0(s32)
-    %3(s16) = G_LSHR %1, %2
-    $ax = COPY %3(s16)
-    RET 0, implicit $ax
-
-...
----
 name:            test_lshr_i8
 alignment:       16
 legalized:       true
 regBankSelected: true
 tracksRegLiveness: true
 registers:
   - { id: 0, class: gpr, preferred-register: '' }
   - { id: 1, class: gpr, preferred-register: '' }
bb.1 (%ir-block.0):
     %0(s32) = COPY $edi
     %2(s8) = G_CONSTANT i8 5
     %1(s8) = G_TRUNC %0(s32)
     %3(s8) = G_LSHR %1, %2
     $al = COPY %3(s8)
     RET 0, implicit $al
 
 ...
----
-name:            test_lshr_i8_imm1
-alignment:       16
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-registers:
-  - { id: 0, class: gpr, preferred-register: '' }
-  - { id: 1, class: gpr, preferred-register: '' }
-  - { id: 2, class: gpr, preferred-register: '' }
-  - { id: 3, class: gpr, preferred-register: '' }
-liveins:
-fixedStack:
-stack:
-constants:
-body:             |
-  bb.1 (%ir-block.0):
-    liveins: $edi
-
-    ; ALL-LABEL: name: test_lshr_i8_imm1
-    ; ALL: liveins: $edi
-    ; ALL: [[COPY:%[0-9]+]]:gr32 = COPY $edi
-    ; ALL: [[COPY1:%[0-9]+]]:gr8 = COPY [[COPY]].sub_8bit
-    ; ALL: [[SHR8r1_:%[0-9]+]]:gr8 = SHR8r1 [[COPY1]], implicit-def $eflags
-    ; ALL: $al = COPY [[SHR8r1_]]
-    ; ALL: RET 0, implicit $al
-    %0(s32) = COPY $edi
-    %2(s8) = G_CONSTANT i8 1
-    %1(s8) = G_TRUNC %0(s32)
-    %3(s8) = G_LSHR %1, %2
-    $al = COPY %3(s8)
-    RET 0, implicit $al
-
-...

llvm/test/CodeGen/X86/fast-isel-shift.ll

; CHECK-NEXT:    retq
   %c = shl i64 %a, 1
   ret i64 %c
 }
 
 define i8 @lshr_imm1_i8(i8 %a) {
 ; CHECK-LABEL: lshr_imm1_i8:
 ; CHECK:       ## %bb.0:
 ; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    shrb $1, %al
+; CHECK-NEXT:    shrb %al
 ; CHECK-NEXT:    ## kill: def $al killed $al killed $eax
 ; CHECK-NEXT:    retq
   %c = lshr i8 %a, 1
   ret i8 %c
 }
 
 define i16 @lshr_imm1_i16(i16 %a) {
 ; CHECK-LABEL: lshr_imm1_i16:
 ; CHECK:       ## %bb.0:
 ; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    shrw $1, %ax
+; CHECK-NEXT:    shrw %ax
 ; CHECK-NEXT:    ## kill: def $ax killed $ax killed $eax
 ; CHECK-NEXT:    retq
   %c = lshr i16 %a, 1
   ret i16 %c
 }
 
 define i32 @lshr_imm1_i32(i32 %a) {
 ; CHECK-LABEL: lshr_imm1_i32:
 ; CHECK:       ## %bb.0:
 ; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    shrl $1, %eax
+; CHECK-NEXT:    shrl %eax
 ; CHECK-NEXT:    retq
   %c = lshr i32 %a, 1
   ret i32 %c
 }
 
 define i64 @lshr_imm1_i64(i64 %a) {
 ; CHECK-LABEL: lshr_imm1_i64:
 ; CHECK:       ## %bb.0:
 ; CHECK-NEXT:    movq %rdi, %rax
-; CHECK-NEXT:    shrq $1, %rax
+; CHECK-NEXT:    shrq %rax
 ; CHECK-NEXT:    retq
   %c = lshr i64 %a, 1
   ret i64 %c
 }
 
 define i8 @ashr_imm1_i8(i8 %a) {
 ; CHECK-LABEL: ashr_imm1_i8:
 ; CHECK:       ## %bb.0:
 ; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    sarb $1, %al
+; CHECK-NEXT:    sarb %al
 ; CHECK-NEXT:    ## kill: def $al killed $al killed $eax
 ; CHECK-NEXT:    retq
   %c = ashr i8 %a, 1
   ret i8 %c
 }
 
 define i16 @ashr_imm1_i16(i16 %a) {
 ; CHECK-LABEL: ashr_imm1_i16:
 ; CHECK:       ## %bb.0:
 ; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    sarw $1, %ax
+; CHECK-NEXT:    sarw %ax
 ; CHECK-NEXT:    ## kill: def $ax killed $ax killed $eax
 ; CHECK-NEXT:    retq
   %c = ashr i16 %a, 1
   ret i16 %c
 }
 
 define i32 @ashr_imm1_i32(i32 %a) {
 ; CHECK-LABEL: ashr_imm1_i32:
 ; CHECK:       ## %bb.0:
 ; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    sarl $1, %eax
+; CHECK-NEXT:    sarl %eax
 ; CHECK-NEXT:    retq
   %c = ashr i32 %a, 1
   ret i32 %c
 }
 
 define i64 @ashr_imm1_i64(i64 %a) {
 ; CHECK-LABEL: ashr_imm1_i64:
 ; CHECK:       ## %bb.0:
 ; CHECK-NEXT:    movq %rdi, %rax
-; CHECK-NEXT:    sarq $1, %rax
+; CHECK-NEXT:    sarq %rax
 ; CHECK-NEXT:    retq
   %c = ashr i64 %a, 1
   ret i64 %c
 }
 
 define i8 @shl_imm4_i8(i8 %a) {
 ; CHECK-LABEL: shl_imm4_i8:
 ; CHECK:       ## %bb.0:

llvm/test/CodeGen/X86/gpr-to-mask.ll

 define void @test_shr1(i1 %cond, ptr %ptr1, ptr %ptr2, <8 x float> %fvec1, <8 x float> %fvec2, ptr %fptrvec) {
 ; X86-64-LABEL: test_shr1:
 ; X86-64:       # %bb.0: # %entry
 ; X86-64-NEXT:    # kill: def $ymm1 killed $ymm1 def $zmm1
 ; X86-64-NEXT:    # kill: def $ymm0 killed $ymm0 def $zmm0
 ; X86-64-NEXT:    testb $1, %dil
 ; X86-64-NEXT:    je .LBB6_2
 ; X86-64-NEXT:  # %bb.1: # %if
-; X86-64-NEXT:    movzbl (%rsi), %eax
-; X86-64-NEXT:    shrb %al
+; X86-64-NEXT:    kmovb (%rsi), %k0
+; X86-64-NEXT:    kshiftrb $1, %k0, %k1
 ; X86-64-NEXT:    jmp .LBB6_3
 ; X86-64-NEXT:  .LBB6_2: # %else
-; X86-64-NEXT:    movzbl (%rdx), %eax
+; X86-64-NEXT:    kmovb (%rdx), %k1
 ; X86-64-NEXT:  .LBB6_3: # %exit
-; X86-64-NEXT:    kmovd %eax, %k1
 ; X86-64-NEXT:    vmovaps %zmm0, %zmm1 {%k1}
 ; X86-64-NEXT:    vmovaps %ymm1, (%rcx)
 ; X86-64-NEXT:    vzeroupper
 ; X86-64-NEXT:    retq
 ;
 ; X86-32-LABEL: test_shr1:
 ; X86-32:       # %bb.0: # %entry
 ; X86-32-NEXT:    # kill: def $ymm1 killed $ymm1 def $zmm1
 ; X86-32-NEXT:    # kill: def $ymm0 killed $ymm0 def $zmm0
 ; X86-32-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-32-NEXT:    testb $1, {{[0-9]+}}(%esp)
 ; X86-32-NEXT:    je .LBB6_2
 ; X86-32-NEXT:  # %bb.1: # %if
 ; X86-32-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X86-32-NEXT:    movzbl (%ecx), %ecx
-; X86-32-NEXT:    shrb %cl
+; X86-32-NEXT:    kmovb (%ecx), %k0
+; X86-32-NEXT:    kshiftrb $1, %k0, %k1
 ; X86-32-NEXT:    jmp .LBB6_3
 ; X86-32-NEXT:  .LBB6_2: # %else
 ; X86-32-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X86-32-NEXT:    movzbl (%ecx), %ecx
+; X86-32-NEXT:    kmovb (%ecx), %k1
 ; X86-32-NEXT:  .LBB6_3: # %exit
-; X86-32-NEXT:    kmovd %ecx, %k1
 ; X86-32-NEXT:    vmovaps %zmm0, %zmm1 {%k1}
 ; X86-32-NEXT:    vmovaps %ymm1, (%eax)
 ; X86-32-NEXT:    vzeroupper
 ; X86-32-NEXT:    retl
 entry:
   br i1 %cond, label %if, label %else
 
 if:

llvm/test/CodeGen/X86/is_fpclass.ll

 ; CHECK-32-NEXT:    andl {{[0-9]+}}(%esp), %eax
 ; CHECK-32-NEXT:    orl {{[0-9]+}}(%esp), %eax
 ; CHECK-32-NEXT:    sete %al
 ; CHECK-32-NEXT:    retl
 ;
 ; CHECK-64-LABEL: iszero_d_strictfp:
 ; CHECK-64:       # %bb.0: # %entry
 ; CHECK-64-NEXT:    movq %xmm0, %rax
-; CHECK-64-NEXT:    shlq $1, %rax
+; CHECK-64-NEXT:    shlq %rax
 ; CHECK-64-NEXT:    testq %rax, %rax
 ; CHECK-64-NEXT:    sete %al
 ; CHECK-64-NEXT:    retq
 entry:
   %0 = tail call i1 @llvm.is.fpclass.f64(double %x, i32 96)  ; 0x60 = "zero"
   ret i1 %0
 }