[AVR] Optimize 16-bit shifts

[benshi001]

Summary

Remove redundant RORs when shift amount >= 8.

[benshi001]

For int >> 9, the following instructions are genreated

mov r24, r25
clr r25
lsr r25
ror r24

However the ror is unecessary, it can be simpilfied to

mov r24, r25
clr r25
lsr r24

[aykevl]

I should have explained this a while ago, sorry for neglecting this.

In D90092#2368009, @benshi001 wrote:

In D90092#2366711, @aykevl wrote:

See my comment D89047#2366709.
I think this optimization would be implemented better in SelectionDAG instead of using pseudo-instructions. That would also make it possible to optimize things like (short)x << 12.

Could you please explain more about be implemented better in SelectionDAG instead of using pseudo-instructions ? I can not figure out a way using SelectionDAG. Because I think SelectionDAG requires data flow dependency and looks like a tree structure. But The mov/clr pair seems can not be put into a tree structure.

Sorry I meant in the MachineIR phase. I think the ideal way to expand bit shift operations is as follows:

1. Lower lsl/lshr/ashr operations in SelectionDAG to pseudo instructions that have a constant immediate operand. You will only need six pseudo instructions this way: for lsl/lshr/ashr and for both 8 bit and 16 bit versions. These pseudo instructions should have the usesCustomInserter flag set.
2. Expand these instructions in AVRTargetLowering::EmitInstrWithCustomInserter. There are already a few existing expansions in there.

The big advantage is that you can do shifts before register allocation and I think it is also easier to structure the code. Also, this means you don't need as many instructions to expand in the pseudo instruction expansion pass.

Recently I have been thinking a lot about constant shifts on AVR and I've written a blog post here: https://aykevl.nl/2021/02/avr-bitshift
I think you will find it useful.

To be clear: I really appreciate your work on the AVR backend. There are many things that need to be improved, such as these bit shift operations. But I think there is a different and possibly better way to implement these expansions.

[benshi001]

In D96506#2585207, @aykevl wrote:

I should have explained this a while ago, sorry for neglecting this.

In D90092#2368009, @benshi001 wrote:

In D90092#2366711, @aykevl wrote:

See my comment D89047#2366709.
I think this optimization would be implemented better in SelectionDAG instead of using pseudo-instructions. That would also make it possible to optimize things like (short)x << 12.

Could you please explain more about be implemented better in SelectionDAG instead of using pseudo-instructions ? I can not figure out a way using SelectionDAG. Because I think SelectionDAG requires data flow dependency and looks like a tree structure. But The mov/clr pair seems can not be put into a tree structure.

Sorry I meant in the MachineIR phase. I think the ideal way to expand bit shift operations is as follows:

1. Lower lsl/lshr/ashr operations in SelectionDAG to pseudo instructions that have a constant immediate operand. You will only need six pseudo instructions this way: for lsl/lshr/ashr and for both 8 bit and 16 bit versions. These pseudo instructions should have the usesCustomInserter flag set.
2. Expand these instructions in AVRTargetLowering::EmitInstrWithCustomInserter. There are already a few existing expansions in there.

The big advantage is that you can do shifts before register allocation and I think it is also easier to structure the code. Also, this means you don't need as many instructions to expand in the pseudo instruction expansion pass.

Recently I have been thinking a lot about constant shifts on AVR and I've written a blog post here: https://aykevl.nl/2021/02/avr-bitshift
I think you will find it useful.

To be clear: I really appreciate your work on the AVR backend. There are many things that need to be improved, such as these bit shift operations. But I think there is a different and possibly better way to implement these expansions.

Thanks for your explanation. I will study your way later.

[benshi001]

In D96506#2585207, @aykevl wrote:

I should have explained this a while ago, sorry for neglecting this.

In D90092#2368009, @benshi001 wrote:

In D90092#2366711, @aykevl wrote:

See my comment D89047#2366709.
I think this optimization would be implemented better in SelectionDAG instead of using pseudo-instructions. That would also make it possible to optimize things like (short)x << 12.

Could you please explain more about be implemented better in SelectionDAG instead of using pseudo-instructions ? I can not figure out a way using SelectionDAG. Because I think SelectionDAG requires data flow dependency and looks like a tree structure. But The mov/clr pair seems can not be put into a tree structure.

Sorry I meant in the MachineIR phase. I think the ideal way to expand bit shift operations is as follows:

1. Lower lsl/lshr/ashr operations in SelectionDAG to pseudo instructions that have a constant immediate operand. You will only need six pseudo instructions this way: for lsl/lshr/ashr and for both 8 bit and 16 bit versions. These pseudo instructions should have the usesCustomInserter flag set.
2. Expand these instructions in AVRTargetLowering::EmitInstrWithCustomInserter. There are already a few existing expansions in there.

The big advantage is that you can do shifts before register allocation and I think it is also easier to structure the code. Also, this means you don't need as many instructions to expand in the pseudo instruction expansion pass.

Recently I have been thinking a lot about constant shifts on AVR and I've written a blog post here: https://aykevl.nl/2021/02/avr-bitshift
I think you will find it useful.

To be clear: I really appreciate your work on the AVR backend. There are many things that need to be improved, such as these bit shift operations. But I think there is a different and possibly better way to implement these expansions.

Thanks. I have submitted a new patch with a compromised solution against your suggestion. Your suggestion introducing 6 pseudo instructions with an extra shift amount operand is really good. https://reviews.llvm.org/D98335

But I still like to do the pseudo expansion in the old way, which requires the least change.

llvm/lib/Target/AVR/AVRExpandPseudoInsts.cpp

bool AVRExpandPseudo::expand<AVR::LSLW12Rd>(Block &MBB, BlockIt MBBI) {
   if (ImpIsDead)
     MI1->getOperand(3).setIsDead();
 
   MI.eraseFromParent();
   return true;
 }
 
 template <>
+bool AVRExpandPseudo::expand<AVR::LSLWHIRd>(Block &MBB, BlockIt MBBI) {
+  MachineInstr &MI = *MBBI;
+  Register DstLoReg, DstHiReg;
+  Register DstReg = MI.getOperand(0).getReg();
+  bool DstIsDead = MI.getOperand(0).isDead();
+  bool DstIsKill = MI.getOperand(1).isKill();
+  bool ImpIsDead = MI.getOperand(2).isDead();
+  TRI->splitReg(DstReg, DstLoReg, DstHiReg);
+
+  // Only left shift high byte.
+  auto MIBHI =
+      buildMI(MBB, MBBI, AVR::ADDRdRr)
+          .addReg(DstHiReg, RegState::Define | getDeadRegState(DstIsDead))
+          .addReg(DstHiReg)
+          .addReg(DstHiReg, getKillRegState(DstIsKill));
+
+  if (ImpIsDead)
+    MIBHI->getOperand(3).setIsDead();
+
+  MI.eraseFromParent();
+  return true;
+}
+
+template <>
 bool AVRExpandPseudo::expand<AVR::LSRWRd>(Block &MBB, BlockIt MBBI) {
   MachineInstr &MI = *MBBI;
   Register DstLoReg, DstHiReg;
   Register DstReg = MI.getOperand(0).getReg();
   bool DstIsDead = MI.getOperand(0).isDead();
   bool DstIsKill = MI.getOperand(1).isKill();
   bool ImpIsDead = MI.getOperand(2).isDead();
   unsigned OpLo = AVR::RORRd;
bool AVRExpandPseudo::expand<AVR::LSRW12Rd>(Block &MBB, BlockIt MBBI) {
   if (ImpIsDead)
     MIBHI->getOperand(3).setIsDead();
 
   MI.eraseFromParent();
   return true;
 }
 
 template <>
+bool AVRExpandPseudo::expand<AVR::LSRWLORd>(Block &MBB, BlockIt MBBI) {
+  MachineInstr &MI = *MBBI;
+  Register DstLoReg, DstHiReg;
+  Register DstReg = MI.getOperand(0).getReg();
+  bool DstIsDead = MI.getOperand(0).isDead();
+  bool DstIsKill = MI.getOperand(1).isKill();
+  bool ImpIsDead = MI.getOperand(2).isDead();
+  TRI->splitReg(DstReg, DstLoReg, DstHiReg);
+
+  // Only logical right shift lower byte.
+  auto MIBLO =
+      buildMI(MBB, MBBI, AVR::LSRRd)
+          .addReg(DstLoReg, RegState::Define | getDeadRegState(DstIsDead))
+          .addReg(DstLoReg, getKillRegState(DstIsKill));
+
+  if (ImpIsDead)
+    MIBLO->getOperand(2).setIsDead();
+
+  MI.eraseFromParent();
+  return true;
+}
+
+template <>
 bool AVRExpandPseudo::expand<AVR::RORWRd>(Block &MBB, BlockIt MBBI) {
   llvm_unreachable("RORW unimplemented");
   return false;
 }
 
 template <>
 bool AVRExpandPseudo::expand<AVR::ROLWRd>(Block &MBB, BlockIt MBBI) {
   llvm_unreachable("ROLW unimplemented");
bool AVRExpandPseudo::expand<AVR::ASRW8Rd>(Block &MBB, BlockIt MBBI) {
   if (ImpIsDead)
     MIBHI->getOperand(3).setIsDead();
 
   MI.eraseFromParent();
   return true;
 }
 
 template <>
+bool AVRExpandPseudo::expand<AVR::ASRWLORd>(Block &MBB, BlockIt MBBI) {
+  MachineInstr &MI = *MBBI;
+  Register DstLoReg, DstHiReg;
+  Register DstReg = MI.getOperand(0).getReg();
+  bool DstIsDead = MI.getOperand(0).isDead();
+  bool DstIsKill = MI.getOperand(1).isKill();
+  bool ImpIsDead = MI.getOperand(2).isDead();
+  TRI->splitReg(DstReg, DstLoReg, DstHiReg);
+
+  // Only arithmetic right shift lower byte.
+  auto MIBLO =
+      buildMI(MBB, MBBI, AVR::ASRRd)
+          .addReg(DstLoReg, RegState::Define | getDeadRegState(DstIsDead))
+          .addReg(DstLoReg, getKillRegState(DstIsKill));
+
+  if (ImpIsDead)
+    MIBLO->getOperand(2).setIsDead();
+
+  MI.eraseFromParent();
+  return true;
+}
+
+template <>
 bool AVRExpandPseudo::expand<AVR::LSLB7Rd>(Block &MBB, BlockIt MBBI) {
   MachineInstr &MI = *MBBI;
   Register DstReg = MI.getOperand(0).getReg();
   bool DstIsDead = MI.getOperand(0).isDead();
   bool DstIsKill = MI.getOperand(1).isKill();
   bool ImpIsDead = MI.getOperand(2).isDead();
 
   // ror r24
case AVR::LDDWRdYQ: //:FIXME: remove this once PR13375 gets fixed
     EXPAND(AVR::INWRdA);
     EXPAND(AVR::OUTWARr);
     EXPAND(AVR::PUSHWRr);
     EXPAND(AVR::POPWRd);
     EXPAND(AVR::ROLBRd);
     EXPAND(AVR::RORBRd);
     EXPAND(AVR::LSLWRd);
     EXPAND(AVR::LSLW4Rd);
+    EXPAND(AVR::LSLB7Rd);
     EXPAND(AVR::LSLW8Rd);
     EXPAND(AVR::LSLW12Rd);
+    EXPAND(AVR::LSLWHIRd);
     EXPAND(AVR::LSRWRd);
     EXPAND(AVR::LSRW4Rd);
+    EXPAND(AVR::LSRB7Rd);
     EXPAND(AVR::LSRW8Rd);
     EXPAND(AVR::LSRW12Rd);
+    EXPAND(AVR::LSRWLORd);
     EXPAND(AVR::RORWRd);
     EXPAND(AVR::ROLWRd);
     EXPAND(AVR::ASRWRd);
-    EXPAND(AVR::ASRW8Rd);
-    EXPAND(AVR::LSLB7Rd);
-    EXPAND(AVR::LSRB7Rd);
     EXPAND(AVR::ASRB7Rd);
+    EXPAND(AVR::ASRW8Rd);
+    EXPAND(AVR::ASRWLORd);
     EXPAND(AVR::SEXT);
     EXPAND(AVR::ZEXT);
     EXPAND(AVR::SPREAD);
     EXPAND(AVR::SPWRITE);
   }
 #undef EXPAND
   return false;
 }

llvm/lib/Target/AVR/AVRISelLowering.h

enum NodeType {
   /// Represents an abstract call instruction,
   /// which includes a bunch of information.
   CALL,
   /// A wrapper node for TargetConstantPool,
   /// TargetExternalSymbol, and TargetGlobalAddress.
   WRAPPER,
   LSL,     ///< Logical shift left.
   LSL4,    ///< Logical shift left 4 bits.
+  LSL7,    ///< Logical shift left 7 bits.
   LSL8,    ///< Logical shift left 8 bits.
   LSL12,   ///< Logical shift left 12 bits.
+  LSLHI,   ///< Logical shift left higher byte only.
   LSR,     ///< Logical shift right.
   LSR4,    ///< Logical shift right 4 bits.
+  LSR7,    ///< Logical shift right 7 bits.
   LSR8,    ///< Logical shift right 8 bits.
   LSR12,   ///< Logical shift right 12 bits.
+  LSRLO,   ///< Logical shift right lower byte only.
   ASR,     ///< Arithmetic shift right.
-  ASR8,    ///< Arithmetic shift right 8 bits.
-  LSL7,    ///< Logical shift left 7 bits.
-  LSR7,    ///< Logical shift right 7 bits.
   ASR7,    ///< Arithmetic shift right 7 bits.
+  ASR8,    ///< Arithmetic shift right 8 bits.
+  ASRLO,   ///< Arithmetic shift right lower byte only.
   ROR,     ///< Bit rotate right.
   ROL,     ///< Bit rotate left.
   LSLLOOP, ///< A loop of single logical shift left instructions.
   LSRLOOP, ///< A loop of single logical shift right instructions.
   ROLLOOP, ///< A loop of single left bit rotate instructions.
   RORLOOP, ///< A loop of single right bit rotate instructions.
   ASRLOOP, ///< A loop of single arithmetic shift right instructions.
   /// AVR conditional branches. Operand 0 is the chain operand, operand 1

llvm/lib/Target/AVR/AVRISelLowering.cpp

 
 EVT AVRTargetLowering::getSetCCResultType(const DataLayout &DL, LLVMContext &,
                                           EVT VT) const {
   assert(!VT.isVector() && "No AVR SetCC type for vectors!");
   return MVT::i8;
 }
 
 SDValue AVRTargetLowering::LowerShifts(SDValue Op, SelectionDAG &DAG) const {
-  //:TODO: this function has to be completely rewritten to produce optimal
-  // code, for now it's producing very long but correct code.
   unsigned Opc8;
   const SDNode *N = Op.getNode();
   EVT VT = Op.getValueType();
   SDLoc dl(N);
   assert(isPowerOf2_32(VT.getSizeInBits()) &&
          "Expected power-of-2 shift amount");
 
   // Expand non-constant shifts to loops.
if (4 <= ShiftAmount && ShiftAmount < 8)
         break;
       default:
         break;
       }
     else if (8 <= ShiftAmount && ShiftAmount < 12)
       switch (Op.getOpcode()) {
       case ISD::SHL:
         Victim = DAG.getNode(AVRISD::LSL8, dl, VT, Victim);
+        // Only left shift the higher byte.
+        Opc8 = AVRISD::LSLHI;
         ShiftAmount -= 8;
         break;
       case ISD::SRL:
         Victim = DAG.getNode(AVRISD::LSR8, dl, VT, Victim);
+        // Only logical right shift the lower byte.
+        Opc8 = AVRISD::LSRLO;
         ShiftAmount -= 8;
         break;
       case ISD::SRA:
         Victim = DAG.getNode(AVRISD::ASR8, dl, VT, Victim);
+        // Only arithmetic right shift the lower byte.
+        Opc8 = AVRISD::ASRLO;
         ShiftAmount -= 8;
         break;
       default:
         break;
       }
     else if (12 <= ShiftAmount)
       switch (Op.getOpcode()) {
       case ISD::SHL:
         Victim = DAG.getNode(AVRISD::LSL12, dl, VT, Victim);
+        // Only left shift the higher byte.
+        Opc8 = AVRISD::LSLHI;
         ShiftAmount -= 12;
         break;
       case ISD::SRL:
         Victim = DAG.getNode(AVRISD::LSR12, dl, VT, Victim);
+        // Only logical right shift the lower byte.
+        Opc8 = AVRISD::LSRLO;
         ShiftAmount -= 12;
         break;
       default:
         break;
       }
   }
 
   while (ShiftAmount--) {

llvm/lib/Target/AVR/AVRInstrInfo.td

 def AVRlsr12 : SDNode<"AVRISD::LSR12", SDTIntUnaryOp>;
 def AVRrol : SDNode<"AVRISD::ROL", SDTIntUnaryOp>;
 def AVRror : SDNode<"AVRISD::ROR", SDTIntUnaryOp>;
 def AVRasr : SDNode<"AVRISD::ASR", SDTIntUnaryOp>;
 def AVRasr8 : SDNode<"AVRISD::ASR8", SDTIntUnaryOp>;
 def AVRlsl7 : SDNode<"AVRISD::LSL7", SDTIntUnaryOp>;
 def AVRlsr7 : SDNode<"AVRISD::LSR7", SDTIntUnaryOp>;
 def AVRasr7 : SDNode<"AVRISD::ASR7", SDTIntUnaryOp>;
+def AVRlslhi : SDNode<"AVRISD::LSLHI", SDTIntUnaryOp>;
+def AVRlsrlo : SDNode<"AVRISD::LSRLO", SDTIntUnaryOp>;
+def AVRasrlo : SDNode<"AVRISD::ASRLO", SDTIntUnaryOp>;
 
 // Pseudo shift nodes for non-constant shift amounts.
 def AVRlslLoop : SDNode<"AVRISD::LSLLOOP", SDTIntShiftOp>;
 def AVRlsrLoop : SDNode<"AVRISD::LSRLOOP", SDTIntShiftOp>;
 def AVRrolLoop : SDNode<"AVRISD::ROLLOOP", SDTIntShiftOp>;
 def AVRrorLoop : SDNode<"AVRISD::RORLOOP", SDTIntShiftOp>;
 def AVRasrLoop : SDNode<"AVRISD::ASRLOOP", SDTIntShiftOp>;
 
def LSLW4Rd : Pseudo<(outs DREGS:$rd),
                        [(set i16:$rd, (AVRlsl4 i16:$src)), (implicit SREG)]>;
 
   def LSLW8Rd : Pseudo<(outs DREGS:$rd),
                        (ins DREGS:$src),
                        "lslw8\t$rd",
                        [(set i16:$rd, (AVRlsl8 i16:$src)), (implicit SREG)]>;
 
   def LSLW12Rd : Pseudo<(outs DREGS:$rd),
                         (ins DREGS:$src),
                         "lslw12\t$rd",
                         [(set i16:$rd, (AVRlsl12 i16:$src)), (implicit SREG)]>;
 
+  def LSLWHIRd : Pseudo<(outs DREGS:$rd),
+                        (ins DREGS:$src),
+                        "lslwhi\t$rd",
+                        [(set i16:$rd, (AVRlslhi i16:$src)), (implicit SREG)]>;
+
   def LSRRd : FRd<0b1001,
                   0b0100110,
                   (outs GPR8:$rd),
                   (ins GPR8:$src),
                   "lsr\t$rd",
                   [(set i8:$rd, (AVRlsr i8:$src)), (implicit SREG)]>;
 
   def LSRB7Rd : Pseudo<(outs GPR8:$rd),
def LSRW4Rd : Pseudo<(outs DREGS:$rd),
                        [(set i16:$rd, (AVRlsr4 i16:$src)), (implicit SREG)]>;
 
   def LSRW8Rd : Pseudo<(outs DREGS:$rd),
                        (ins DREGS:$src),
                        "lsrw8\t$rd",
                        [(set i16:$rd, (AVRlsr8 i16:$src)), (implicit SREG)]>;
 
   def LSRW12Rd : Pseudo<(outs DREGS:$rd),
                         (ins DREGS:$src),
                         "lsrw12\t$rd",
                         [(set i16:$rd, (AVRlsr12 i16:$src)), (implicit SREG)]>;
 
+  def LSRWLORd : Pseudo<(outs DREGS:$rd),
+                        (ins DREGS:$src),
+                        "lsrwlo\t$rd",
+                        [(set i16:$rd, (AVRlsrlo i16:$src)), (implicit SREG)]>;
+
   def ASRRd : FRd<0b1001,
                   0b0100101,
                   (outs GPR8:$rd),
                   (ins GPR8:$src),
                   "asr\t$rd",
                   [(set i8:$rd, (AVRasr i8:$src)), (implicit SREG)]>;
 
   def ASRB7Rd : Pseudo<(outs GPR8:$rd),
                        (ins GPR8:$src),
                        "asrb7\t$rd",
                        [(set i8:$rd, (AVRasr7 i8:$src)), (implicit SREG)]>;
 
   def ASRWRd : Pseudo<(outs DREGS:$rd),
                       (ins DREGS:$src),
                       "asrw\t$rd",
                       [(set i16:$rd, (AVRasr i16:$src)), (implicit SREG)]>;
 
   def ASRW8Rd : Pseudo<(outs DREGS:$rd),
                        (ins DREGS:$src),
                        "asrw8\t$rd",
                        [(set i16:$rd, (AVRasr8 i16:$src)), (implicit SREG)]>;
 
+  def ASRWLORd : Pseudo<(outs DREGS:$rd),
+                        (ins DREGS:$src),
+                        "asrwlo\t$rd",
+                        [(set i16:$rd, (AVRasrlo i16:$src)), (implicit SREG)]>;
+
   // Bit rotate operations.
   let Uses = [SREG] in
   {
     // 8-bit ROL is an alias of ADC Rd, Rd
 
     def ROLBRd : Pseudo<(outs GPR8:$rd),
                         (ins GPR8:$src),
                         "rolb\t$rd",

llvm/test/CodeGen/AVR/shift.ll

-; RUN: llc < %s -march=avr | FileCheck %s
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc < %s -mtriple=avr | FileCheck %s
 
 ; Optimize for speed.
 ; CHECK-LABEL: shift_i8_i8_speed
 define i8 @shift_i8_i8_speed(i8 %a, i8 %b) {
-  ; CHECK:        dec r22
-  ; CHECK-NEXT:   brmi .LBB0_2
-  ; CHECK-NEXT: .LBB0_1:
-  ; CHECK-NEXT:   lsl r24
-  ; CHECK-NEXT:   dec r22
-  ; CHECK-NEXT:   brpl .LBB0_1
-  ; CHECK-NEXT: .LBB0_2:
-  ; CHECK-NEXT:   ret
+; CHECK-LABEL: shift_i8_i8_speed:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    dec r22
+; CHECK-NEXT:    brmi .LBB0_2
+; CHECK-NEXT:  .LBB0_1: ; =>This Inner Loop Header: Depth=1
+; CHECK-NEXT:    lsl r24
+; CHECK-NEXT:    dec r22
+; CHECK-NEXT:    brpl .LBB0_1
+; CHECK-NEXT:  .LBB0_2:
+; CHECK-NEXT:    ret
   %result = shl i8 %a, %b
   ret i8 %result
 }
 
 ; Optimize for size (producing slightly smaller code).
 ; CHECK-LABEL: shift_i8_i8_size
 define i8 @shift_i8_i8_size(i8 %a, i8 %b) optsize {
-  ; CHECK:      .LBB1_1:
-  ; CHECK-NEXT:   dec r22
-  ; CHECK-NEXT:   brmi .LBB1_3
-  ; CHECK:        lsl r24
-  ; CHECK-NEXT:   rjmp .LBB1_1
-  ; CHECK-NEXT: .LBB1_3:
-  ; CHECK-NEXT:   ret
+; CHECK-LABEL: shift_i8_i8_size:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:  .LBB1_1: ; =>This Inner Loop Header: Depth=1
+; CHECK-NEXT:    dec r22
+; CHECK-NEXT:    brmi .LBB1_3
+; CHECK-NEXT:  ; %bb.2: ; in Loop: Header=BB1_1 Depth=1
+; CHECK-NEXT:    lsl r24
+; CHECK-NEXT:    rjmp .LBB1_1
+; CHECK-NEXT:  .LBB1_3:
+; CHECK-NEXT:    ret
   %result = shl i8 %a, %b
   ret i8 %result
 }
 
 ; CHECK-LABEL: shift_i16_i16
 define i16 @shift_i16_i16(i16 %a, i16 %b) {
-  ; CHECK:        dec r22
-  ; CHECK-NEXT:   brmi .LBB2_2
-  ; CHECK-NEXT: .LBB2_1:
-  ; CHECK-NEXT:   lsl r24
-  ; CHECK-NEXT:   rol r25
-  ; CHECK-NEXT:   dec r22
-  ; CHECK-NEXT:   brpl .LBB2_1
-  ; CHECK-NEXT: .LBB2_2:
-  ; CHECK-NEXT:   ret
+; CHECK-LABEL: shift_i16_i16:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    dec r22
+; CHECK-NEXT:    brmi .LBB2_2
+; CHECK-NEXT:  .LBB2_1: ; =>This Inner Loop Header: Depth=1
+; CHECK-NEXT:    lsl r24
+; CHECK-NEXT:    rol r25
+; CHECK-NEXT:    dec r22
+; CHECK-NEXT:    brpl .LBB2_1
+; CHECK-NEXT:  .LBB2_2:
+; CHECK-NEXT:    ret
   %result = shl i16 %a, %b
   ret i16 %result
 }
 
 ; CHECK-LABEL: shift_i64_i64
 define i64 @shift_i64_i64(i64 %a, i64 %b) {
-  ; CHECK: call    __ashldi3
+; CHECK-LABEL: shift_i64_i64:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    push r16
+; CHECK-NEXT:    mov r16, r10
+; CHECK-NEXT:    call __ashldi3
+; CHECK-NEXT:    pop r16
+; CHECK-NEXT:    ret
   %result = shl i64 %a, %b
   ret i64 %result
 }
 
 define i8 @lsl_i8_1(i8 %a) {
 ; CHECK-LABEL: lsl_i8_1:
-; CHECK:       lsl r24
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    lsl r24
+; CHECK-NEXT:    ret
   %res = shl i8 %a, 1
   ret i8 %res
 }
 
 define i8 @lsl_i8_2(i8 %a) {
 ; CHECK-LABEL: lsl_i8_2:
-; CHECK:       lsl r24
-; CHECK-NEXT:  lsl r24
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    lsl r24
+; CHECK-NEXT:    lsl r24
+; CHECK-NEXT:    ret
   %res = shl i8 %a, 2
   ret i8 %res
 }
 
 define i8 @lsl_i8_3(i8 %a) {
 ; CHECK-LABEL: lsl_i8_3:
-; CHECK:       lsl r24
-; CHECK-NEXT:  lsl r24
-; CHECK-NEXT:  lsl r24
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    lsl r24
+; CHECK-NEXT:    lsl r24
+; CHECK-NEXT:    lsl r24
+; CHECK-NEXT:    ret
   %res = shl i8 %a, 3
   ret i8 %res
 }
 
 define i8 @lsl_i8_4(i8 %a) {
 ; CHECK-LABEL: lsl_i8_4:
-; CHECK:       swap r24
-; CHECK-NEXT:  andi r24, -16
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    swap r24
+; CHECK-NEXT:    andi r24, -16
+; CHECK-NEXT:    ret
   %res = shl i8 %a, 4
   ret i8 %res
 }
 
 define i8 @lsl_i8_5(i8 %a) {
 ; CHECK-LABEL: lsl_i8_5:
-; CHECK:       swap r24
-; CHECK-NEXT:  andi r24, -16
-; CHECK-NEXT:  lsl r24
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    swap r24
+; CHECK-NEXT:    andi r24, -16
+; CHECK-NEXT:    lsl r24
+; CHECK-NEXT:    ret
   %res = shl i8 %a, 5
   ret i8 %res
 }
 
 define i8 @lsl_i8_6(i8 %a) {
 ; CHECK-LABEL: lsl_i8_6:
-; CHECK:       swap r24
-; CHECK-NEXT:  andi r24, -16
-; CHECK-NEXT:  lsl r24
-; CHECK-NEXT:  lsl r24
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    swap r24
+; CHECK-NEXT:    andi r24, -16
+; CHECK-NEXT:    lsl r24
+; CHECK-NEXT:    lsl r24
+; CHECK-NEXT:    ret
   %res = shl i8 %a, 6
   ret i8 %res
 }
 
 define i8 @lsr_i8_1(i8 %a) {
 ; CHECK-LABEL: lsr_i8_1:
-; CHECK:       lsr r24
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    lsr r24
+; CHECK-NEXT:    ret
   %res = lshr i8 %a, 1
   ret i8 %res
 }
 
 define i8 @lsr_i8_2(i8 %a) {
 ; CHECK-LABEL: lsr_i8_2:
-; CHECK:       lsr r24
-; CHECK-NEXT:  lsr r24
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    lsr r24
+; CHECK-NEXT:    lsr r24
+; CHECK-NEXT:    ret
   %res = lshr i8 %a, 2
   ret i8 %res
 }
 
 define i8 @lsr_i8_3(i8 %a) {
 ; CHECK-LABEL: lsr_i8_3:
-; CHECK:       lsr r24
-; CHECK-NEXT:  lsr r24
-; CHECK-NEXT:  lsr r24
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    lsr r24
+; CHECK-NEXT:    lsr r24
+; CHECK-NEXT:    lsr r24
+; CHECK-NEXT:    ret
   %res = lshr i8 %a, 3
   ret i8 %res
 }
 
 define i8 @lsr_i8_4(i8 %a) {
 ; CHECK-LABEL: lsr_i8_4:
-; CHECK:       swap r24
-; CHECK-NEXT:  andi r24, 15
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    swap r24
+; CHECK-NEXT:    andi r24, 15
+; CHECK-NEXT:    ret
   %res = lshr i8 %a, 4
   ret i8 %res
 }
 
 define i8 @lsr_i8_5(i8 %a) {
 ; CHECK-LABEL: lsr_i8_5:
-; CHECK:       swap r24
-; CHECK-NEXT:  andi r24, 15
-; CHECK-NEXT:  lsr r24
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    swap r24
+; CHECK-NEXT:    andi r24, 15
+; CHECK-NEXT:    lsr r24
+; CHECK-NEXT:    ret
   %res = lshr i8 %a, 5
   ret i8 %res
 }
 
 define i8 @lsr_i8_6(i8 %a) {
 ; CHECK-LABEL: lsr_i8_6:
-; CHECK:       swap r24
-; CHECK-NEXT:  andi r24, 15
-; CHECK-NEXT:  lsr r24
-; CHECK-NEXT:  lsr r24
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    swap r24
+; CHECK-NEXT:    andi r24, 15
+; CHECK-NEXT:    lsr r24
+; CHECK-NEXT:    lsr r24
+; CHECK-NEXT:    ret
   %res = lshr i8 %a, 6
   ret i8 %res
 }
 
 define i8 @lsl_i8_7(i8 %a) {
-; CHECK-LABEL: lsl_i8_7
-; CHECK:      ror r24
-; CHECK-NEXT: clr r24
-; CHECK-NEXT: ror r24
+; CHECK-LABEL: lsl_i8_7:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    ror r24
+; CHECK-NEXT:    clr r24
+; CHECK-NEXT:    ror r24
+; CHECK-NEXT:    ret
   %result = shl i8 %a, 7
   ret i8 %result
 }
 
 define i8 @lsr_i8_7(i8 %a) {
-; CHECK-LABEL: lsr_i8_7
-; CHECK:      rol r24
-; CHECK-NEXT: clr r24
-; CHECK-NEXT: rol r24
+; CHECK-LABEL: lsr_i8_7:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    rol r24
+; CHECK-NEXT:    clr r24
+; CHECK-NEXT:    rol r24
+; CHECK-NEXT:    ret
   %result = lshr i8 %a, 7
   ret i8 %result
 }
 
 define i8 @asr_i8_7(i8 %a) {
-; CHECK-LABEL: asr_i8_7
-; CHECK:      lsl r24
-; CHECK-NEXT: sbc r24, r24
+; CHECK-LABEL: asr_i8_7:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    lsl r24
+; CHECK-NEXT:    sbc r24, r24
+; CHECK-NEXT:    ret
   %result = ashr i8 %a, 7
   ret i8 %result
 }
 
 define i16 @lsl_i16_5(i16 %a) {
-; CHECK-LABEL: lsl_i16_5
-; CHECK:       swap r25
-; CHECK-NEXT:  swap r24
-; CHECK-NEXT:  andi r25, 240
-; CHECK-NEXT:  eor r25, r24
-; CHECK-NEXT:  andi r24, 240
-; CHECK-NEXT:  eor r25, r24
-; CHECK-NEXT:  lsl r24
-; CHECK-NEXT:  rol r25
-; CHECK-NEXT:  ret
+; CHECK-LABEL: lsl_i16_5:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    swap r25
+; CHECK-NEXT:    swap r24
+; CHECK-NEXT:    andi r25, 240
+; CHECK-NEXT:    eor r25, r24
+; CHECK-NEXT:    andi r24, 240
+; CHECK-NEXT:    eor r25, r24
+; CHECK-NEXT:    lsl r24
+; CHECK-NEXT:    rol r25
+; CHECK-NEXT:    ret
   %result = shl i16 %a, 5
   ret i16 %result
 }
 
 define i16 @lsl_i16_9(i16 %a) {
-; CHECK-LABEL: lsl_i16_9
-; CHECK:       mov r25, r24
-; CHECK-NEXT:  clr r24
-; CHECK-NEXT:  lsl r24
-; CHECK-NEXT:  rol r25
-; CHECK-NEXT:  ret
+; CHECK-LABEL: lsl_i16_9:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    mov r25, r24
+; CHECK-NEXT:    clr r24
+; CHECK-NEXT:    lsl r25
+; CHECK-NEXT:    ret
   %result = shl i16 %a, 9
   ret i16 %result
 }
 
 define i16 @lsl_i16_13(i16 %a) {
-; CHECK-LABEL: lsl_i16_13
-; CHECK:       mov r25, r24
-; CHECK-NEXT:  swap r25
-; CHECK-NEXT:  andi r25, 240
-; CHECK-NEXT:  clr r24
-; CHECK-NEXT:  lsl r24
-; CHECK-NEXT:  rol r25
-; CHECK-NEXT:  ret
+; CHECK-LABEL: lsl_i16_13:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    mov r25, r24
+; CHECK-NEXT:    swap r25
+; CHECK-NEXT:    andi r25, 240
+; CHECK-NEXT:    clr r24
+; CHECK-NEXT:    lsl r25
+; CHECK-NEXT:    ret
   %result = shl i16 %a, 13
   ret i16 %result
 }
 
 define i16 @lsr_i16_5(i16 %a) {
-; CHECK-LABEL: lsr_i16_5
-; CHECK:       swap r25
-; CHECK-NEXT:  swap r24
-; CHECK-NEXT:  andi r24, 15
-; CHECK-NEXT:  eor r24, r25
-; CHECK-NEXT:  andi r25, 15
-; CHECK-NEXT:  eor r24, r25
-; CHECK-NEXT:  lsr r25
-; CHECK-NEXT:  ror r24
-; CHECK-NEXT:  ret
+; CHECK-LABEL: lsr_i16_5:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    swap r25
+; CHECK-NEXT:    swap r24
+; CHECK-NEXT:    andi r24, 15
+; CHECK-NEXT:    eor r24, r25
+; CHECK-NEXT:    andi r25, 15
+; CHECK-NEXT:    eor r24, r25
+; CHECK-NEXT:    lsr r25
+; CHECK-NEXT:    ror r24
+; CHECK-NEXT:    ret
   %result = lshr i16 %a, 5
   ret i16 %result
 }
 
 define i16 @lsr_i16_9(i16 %a) {
-; CHECK-LABEL: lsr_i16_9
-; CHECK:       mov r24, r25
-; CHECK-NEXT:  clr r25
-; CHECK-NEXT:  lsr r25
-; CHECK-NEXT:  ror r24
-; CHECK-NEXT:  ret
+; CHECK-LABEL: lsr_i16_9:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    mov r24, r25
+; CHECK-NEXT:    clr r25
+; CHECK-NEXT:    lsr r24
+; CHECK-NEXT:    ret
   %result = lshr i16 %a, 9
   ret i16 %result
 }
 
 define i16 @lsr_i16_13(i16 %a) {
-; CHECK-LABEL: lsr_i16_13
-; CHECK:       mov r24, r25
-; CHECK-NEXT:  swap r24
-; CHECK-NEXT:  andi r24, 15
-; CHECK-NEXT:  clr r25
-; CHECK-NEXT:  lsr r25
-; CHECK-NEXT:  ror r24
-; CHECK-NEXT:  ret
+; CHECK-LABEL: lsr_i16_13:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    mov r24, r25
+; CHECK-NEXT:    swap r24
+; CHECK-NEXT:    andi r24, 15
+; CHECK-NEXT:    clr r25
+; CHECK-NEXT:    lsr r24
+; CHECK-NEXT:    ret
   %result = lshr i16 %a, 13
   ret i16 %result
 }
 
 define i16 @asr_i16_9(i16 %a) {
-; CHECK-LABEL: asr_i16_9
-; CHECK:       mov r24, r25
-; CHECK-NEXT:  lsl r25
-; CHECK-NEXT:  sbc r25, r25
-; CHECK-NEXT:  asr r25
-; CHECK-NEXT:  ror r24
-; CHECK-NEXT:  ret
+; CHECK-LABEL: asr_i16_9:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    mov r24, r25
+; CHECK-NEXT:    lsl r25
+; CHECK-NEXT:    sbc r25, r25
+; CHECK-NEXT:    asr r24
+; CHECK-NEXT:    ret
   %result = ashr i16 %a, 9
   ret i16 %result
 }