[RISCV] Legalize select when Zbt extension available

[mundaym]

Summary

The custom expansion of select operations in the RISC-V backend
interferes with the matching of cmov instructions. Legalizing
select when the Zbt extension is available solves that problem.

[craig.topper]

llvm/lib/Target/RISCV/RISCVInstrInfoB.td

780

Is there a difference between doing it this way versus just adding more riscv_selectcc patterns for other condition codes?

[mundaym]

llvm/lib/Target/RISCV/RISCVInstrInfoB.td

780

No I think either would work. Using riscv_selectcc would have the advantage that we'd only encounter normalised condition codes so we'd need fewer patterns. Do you have a preference?

[craig.topper]

llvm/lib/Target/RISCV/RISCVInstrInfoB.td

780

I think my preference is towards fewer patterns.

[lenary]

The code generation improvements here are looking good, let's hope they stay with a set of riscv_selectcc patterns.

llvm/lib/Target/RISCV/RISCVInstrInfoB.td

780

@mundaym and I discussed this today, and fewer patterns makes sense, though there's a slight hiccup with using CondCode directly in the riscv_selectcc pattern that he's looking into.

[mundaym]

llvm/lib/Target/RISCV/RISCVInstrInfoB.td

780

After further investigation it looks like using riscv_selectcc will end up requiring roughly the same number of patterns because we need to lower each legal comparison operation (SETEQ, SETGE, etc.) individually whereas with select we only need to do it for comparison operations that benefit from optimization.

I think we should therefore stick with the original approach I took of legalizing select with some additional optimization patterns. This approach also bypasses the issue @lenary mentioned where the comparison operation (e.g. SETEQ) currently has to be encoded as an immediate in riscv_selectcc (e.g. XLenVT 17) which is a bit fragile, especially if more patterns are added.

[craig.topper]

LGTM

[lenary]

LGTM!

I'm going to look into removing more of the xori … 1 which are used by the selects in some of these cases.

[lenary]

We've found some issues in testing, so for the moment I've reverted this patch in rG7c9c2a2ea5e3760d7310309c96c9a4ce41fa4d9b.

[mundaym]

Fix inverted patterns.

[craig.topper]

LGTM

[luismarques]

LGTM. The issues we detected are now solved.

[craig.topper]

@mundaym do you have commit access yet? I think Sam had been committing your previous patches?

[mundaym]

@craig.topper No, I don't have commit access yet. (Thanks for reviewing this by the way!)

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM,
   for (auto N : {ISD::EXTLOAD, ISD::SEXTLOAD, ISD::ZEXTLOAD})
     setLoadExtAction(N, XLenVT, MVT::i1, Promote);
 
   // TODO: add all necessary setOperationAction calls.
   setOperationAction(ISD::DYNAMIC_STACKALLOC, XLenVT, Expand);
 
   setOperationAction(ISD::BR_JT, MVT::Other, Expand);
   setOperationAction(ISD::BR_CC, XLenVT, Expand);
-  setOperationAction(ISD::SELECT, XLenVT, Custom);
   setOperationAction(ISD::SELECT_CC, XLenVT, Expand);
 
   setOperationAction(ISD::STACKSAVE, MVT::Other, Expand);
   setOperationAction(ISD::STACKRESTORE, MVT::Other, Expand);
 
   setOperationAction(ISD::VASTART, MVT::Other, Custom);
   setOperationAction(ISD::VAARG, MVT::Other, Expand);
   setOperationAction(ISD::VACOPY, MVT::Other, Expand);
if (Subtarget.hasStdExtZbb()) {
     setOperationAction(ISD::CTTZ, XLenVT, Expand);
     setOperationAction(ISD::CTLZ, XLenVT, Expand);
     setOperationAction(ISD::CTPOP, XLenVT, Expand);
   }
 
   if (Subtarget.hasStdExtZbt()) {
     setOperationAction(ISD::FSHL, XLenVT, Legal);
     setOperationAction(ISD::FSHR, XLenVT, Legal);
+    setOperationAction(ISD::SELECT, XLenVT, Legal);
 
     if (Subtarget.is64Bit()) {
       setOperationAction(ISD::FSHL, MVT::i32, Custom);
       setOperationAction(ISD::FSHR, MVT::i32, Custom);
     }
+  } else {
+    setOperationAction(ISD::SELECT, XLenVT, Custom);
   }
 
   ISD::CondCode FPCCToExpand[] = {
       ISD::SETOGT, ISD::SETOGE, ISD::SETONE, ISD::SETUEQ, ISD::SETUGT,
       ISD::SETUGE, ISD::SETULT, ISD::SETULE, ISD::SETUNE, ISD::SETGT,
       ISD::SETGE,  ISD::SETNE,  ISD::SETO,   ISD::SETUO};
 
   ISD::NodeType FPOpToExpand[] = {

llvm/lib/Target/RISCV/RISCVInstrInfoB.td

 let Predicates = [HasStdExtZbp, IsRV32] in {
 def : Pat<(rotr (riscv_grevi GPR:$rs1, (i32 24)), (i32 16)), (GREVI GPR:$rs1, 8)>;
 def : Pat<(rotl (riscv_grevi GPR:$rs1, (i32 24)), (i32 16)), (GREVI GPR:$rs1, 8)>;
 } // Predicates = [HasStdExtZbp, IsRV32]
 
 let Predicates = [HasStdExtZbt] in {
 def : Pat<(or (and (not GPR:$rs2), GPR:$rs3), (and GPR:$rs2, GPR:$rs1)),
           (CMIX GPR:$rs1, GPR:$rs2, GPR:$rs3)>;
-def : Pat<(riscv_selectcc GPR:$rs2, (XLenVT 0), (XLenVT 17), GPR:$rs3, GPR:$rs1),
+def : Pat<(select (XLenVT (setne GPR:$rs2, 0)), GPR:$rs1, GPR:$rs3),
+          (CMOV GPR:$rs1, GPR:$rs2, GPR:$rs3)>;
+def : Pat<(select (XLenVT (seteq GPR:$rs2, 0)), GPR:$rs3, GPR:$rs1),
+          (CMOV GPR:$rs1, GPR:$rs2, GPR:$rs3)>;
+def : Pat<(select (XLenVT (seteq GPR:$x, GPR:$y)), GPR:$rs1, GPR:$rs3),
+          (CMOV GPR:$rs1, (XOR GPR:$x, GPR:$y), GPR:$rs3)>;
+def : Pat<(select (XLenVT (setne GPR:$x, GPR:$y)), GPR:$rs3, GPR:$rs1),
+          (CMOV GPR:$rs1, (XOR GPR:$x, GPR:$y), GPR:$rs3)>;
+def : Pat<(select (XLenVT (setuge GPR:$x, GPR:$y)), GPR:$rs1, GPR:$rs3),
+          (CMOV GPR:$rs1, (SLTU GPR:$x, GPR:$y), GPR:$rs3)>;
+def : Pat<(select (XLenVT (setule GPR:$y, GPR:$x)), GPR:$rs1, GPR:$rs3),
+          (CMOV GPR:$rs1, (SLTU GPR:$x, GPR:$y), GPR:$rs3)>;
+def : Pat<(select (XLenVT (setge GPR:$x, GPR:$y)), GPR:$rs1, GPR:$rs3),
+          (CMOV GPR:$rs1, (SLT GPR:$x, GPR:$y), GPR:$rs3)>;
+def : Pat<(select (XLenVT (setle GPR:$y, GPR:$x)), GPR:$rs1, GPR:$rs3),
+          (CMOV GPR:$rs1, (SLT GPR:$x, GPR:$y), GPR:$rs3)>;
+def : Pat<(select GPR:$rs2, GPR:$rs3, GPR:$rs1),
           (CMOV GPR:$rs1, GPR:$rs2, GPR:$rs3)>;
 } // Predicates = [HasStdExtZbt]
 
 // fshl and fshr concatenate their operands in the same order. fsr and fsl
 // instruction use different orders. fshl will return its first operand for
 // shift of zero, fshr will return its second operand. fsl and fsr both return
 // $rs1 so the patterns need to have different operand orders.
 //

llvm/test/CodeGen/RISCV/rv32Zbb.ll

 ; RV32I-NEXT:    sll a2, a0, a2
 ; RV32I-NEXT:  .LBB1_3:
 ; RV32I-NEXT:    not a1, a1
 ; RV32I-NEXT:    not a0, a2
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: slo_i64:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    addi a3, a2, -32
 ; RV32IB-NEXT:    not a0, a0
-; RV32IB-NEXT:    bltz a3, .LBB1_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    mv a2, zero
-; RV32IB-NEXT:    sll a1, a0, a3
-; RV32IB-NEXT:    j .LBB1_3
-; RV32IB-NEXT:  .LBB1_2:
 ; RV32IB-NEXT:    not a1, a1
 ; RV32IB-NEXT:    sll a1, a1, a2
 ; RV32IB-NEXT:    addi a3, zero, 31
 ; RV32IB-NEXT:    sub a3, a3, a2
 ; RV32IB-NEXT:    srli a4, a0, 1
 ; RV32IB-NEXT:    srl a3, a4, a3
 ; RV32IB-NEXT:    or a1, a1, a3
-; RV32IB-NEXT:    sll a2, a0, a2
-; RV32IB-NEXT:  .LBB1_3:
+; RV32IB-NEXT:    addi a3, a2, -32
+; RV32IB-NEXT:    sll a4, a0, a3
+; RV32IB-NEXT:    slti a5, a3, 0
+; RV32IB-NEXT:    cmov a1, a5, a4, a1
+; RV32IB-NEXT:    sll a0, a0, a2
+; RV32IB-NEXT:    srai a2, a3, 31
+; RV32IB-NEXT:    and a0, a2, a0
 ; RV32IB-NEXT:    not a1, a1
-; RV32IB-NEXT:    not a0, a2
+; RV32IB-NEXT:    not a0, a0
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBB-LABEL: slo_i64:
 ; RV32IBB:       # %bb.0:
 ; RV32IBB-NEXT:    addi a3, a2, -32
 ; RV32IBB-NEXT:    not a0, a0
 ; RV32IBB-NEXT:    bltz a3, .LBB1_2
 ; RV32IBB-NEXT:  # %bb.1:

 ; RV32I-NEXT:    srl a2, a1, a2
 ; RV32I-NEXT:  .LBB3_3:
 ; RV32I-NEXT:    not a0, a0
 ; RV32I-NEXT:    not a1, a2
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: sro_i64:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    addi a3, a2, -32
 ; RV32IB-NEXT:    not a1, a1
-; RV32IB-NEXT:    bltz a3, .LBB3_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    mv a2, zero
-; RV32IB-NEXT:    srl a0, a1, a3
-; RV32IB-NEXT:    j .LBB3_3
-; RV32IB-NEXT:  .LBB3_2:
 ; RV32IB-NEXT:    not a0, a0
 ; RV32IB-NEXT:    srl a0, a0, a2
 ; RV32IB-NEXT:    addi a3, zero, 31
 ; RV32IB-NEXT:    sub a3, a3, a2
 ; RV32IB-NEXT:    slli a4, a1, 1
 ; RV32IB-NEXT:    sll a3, a4, a3
 ; RV32IB-NEXT:    or a0, a0, a3
-; RV32IB-NEXT:    srl a2, a1, a2
-; RV32IB-NEXT:  .LBB3_3:
+; RV32IB-NEXT:    addi a3, a2, -32
+; RV32IB-NEXT:    srl a4, a1, a3
+; RV32IB-NEXT:    slti a5, a3, 0
+; RV32IB-NEXT:    cmov a0, a5, a4, a0
+; RV32IB-NEXT:    srl a1, a1, a2
+; RV32IB-NEXT:    srai a2, a3, 31
+; RV32IB-NEXT:    and a1, a2, a1
 ; RV32IB-NEXT:    not a0, a0
-; RV32IB-NEXT:    not a1, a2
+; RV32IB-NEXT:    not a1, a1
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBB-LABEL: sro_i64:
 ; RV32IBB:       # %bb.0:
 ; RV32IBB-NEXT:    addi a3, a2, -32
 ; RV32IBB-NEXT:    not a1, a1
 ; RV32IBB-NEXT:    bltz a3, .LBB3_2
 ; RV32IBB-NEXT:  # %bb.1:

 ; RV32I-NEXT:    lw s1, 20(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s0, 24(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw ra, 28(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: ctlz_i64:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    bnez a1, .LBB9_2
-; RV32IB-NEXT:  # %bb.1:
+; RV32IB-NEXT:    clz a2, a1
 ; RV32IB-NEXT:    clz a0, a0
 ; RV32IB-NEXT:    addi a0, a0, 32
-; RV32IB-NEXT:    mv a1, zero
-; RV32IB-NEXT:    ret
-; RV32IB-NEXT:  .LBB9_2:
-; RV32IB-NEXT:    clz a0, a1
+; RV32IB-NEXT:    cmov a0, a1, a2, a0
 ; RV32IB-NEXT:    mv a1, zero
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBB-LABEL: ctlz_i64:
 ; RV32IBB:       # %bb.0:
 ; RV32IBB-NEXT:    bnez a1, .LBB9_2
 ; RV32IBB-NEXT:  # %bb.1:
 ; RV32IBB-NEXT:    clz a0, a0

 ; RV32I-NEXT:    lw s1, 20(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s0, 24(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw ra, 28(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: cttz_i64:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    bnez a0, .LBB11_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    ctz a0, a1
-; RV32IB-NEXT:    addi a0, a0, 32
-; RV32IB-NEXT:    mv a1, zero
-; RV32IB-NEXT:    ret
-; RV32IB-NEXT:  .LBB11_2:
-; RV32IB-NEXT:    ctz a0, a0
+; RV32IB-NEXT:    ctz a2, a0
+; RV32IB-NEXT:    ctz a1, a1
+; RV32IB-NEXT:    addi a1, a1, 32
+; RV32IB-NEXT:    cmov a0, a0, a2, a1
 ; RV32IB-NEXT:    mv a1, zero
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBB-LABEL: cttz_i64:
 ; RV32IBB:       # %bb.0:
 ; RV32IBB-NEXT:    bnez a0, .LBB11_2
 ; RV32IBB-NEXT:  # %bb.1:
 ; RV32IBB-NEXT:    ctz a0, a1

 ; RV32I-NEXT:  .LBB19_3:
 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    mv a1, a3
 ; RV32I-NEXT:  .LBB19_4:
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: min_i64:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    mv a4, a0
-; RV32IB-NEXT:    bge a1, a3, .LBB19_3
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    beq a1, a3, .LBB19_4
-; RV32IB-NEXT:  .LBB19_2:
-; RV32IB-NEXT:    min a1, a1, a3
-; RV32IB-NEXT:    ret
-; RV32IB-NEXT:  .LBB19_3:
-; RV32IB-NEXT:    mv a0, a2
-; RV32IB-NEXT:    bne a1, a3, .LBB19_2
-; RV32IB-NEXT:  .LBB19_4:
-; RV32IB-NEXT:    minu a0, a4, a2
+; RV32IB-NEXT:    slt a4, a1, a3
+; RV32IB-NEXT:    cmov a4, a4, a2, a0
+; RV32IB-NEXT:    xor a5, a1, a3
+; RV32IB-NEXT:    minu a0, a0, a2
+; RV32IB-NEXT:    cmov a0, a5, a0, a4
 ; RV32IB-NEXT:    min a1, a1, a3
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBB-LABEL: min_i64:
 ; RV32IBB:       # %bb.0:
 ; RV32IBB-NEXT:    mv a4, a0
 ; RV32IBB-NEXT:    bge a1, a3, .LBB19_3
 ; RV32IBB-NEXT:  # %bb.1:

 ; RV32I-NEXT:  .LBB21_3:
 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    mv a1, a3
 ; RV32I-NEXT:  .LBB21_4:
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: max_i64:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    mv a4, a0
-; RV32IB-NEXT:    bge a3, a1, .LBB21_3
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    beq a1, a3, .LBB21_4
-; RV32IB-NEXT:  .LBB21_2:
-; RV32IB-NEXT:    max a1, a1, a3
-; RV32IB-NEXT:    ret
-; RV32IB-NEXT:  .LBB21_3:
-; RV32IB-NEXT:    mv a0, a2
-; RV32IB-NEXT:    bne a1, a3, .LBB21_2
-; RV32IB-NEXT:  .LBB21_4:
-; RV32IB-NEXT:    maxu a0, a4, a2
+; RV32IB-NEXT:    slt a4, a3, a1
+; RV32IB-NEXT:    cmov a4, a4, a2, a0
+; RV32IB-NEXT:    xor a5, a1, a3
+; RV32IB-NEXT:    maxu a0, a0, a2
+; RV32IB-NEXT:    cmov a0, a5, a0, a4
 ; RV32IB-NEXT:    max a1, a1, a3
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBB-LABEL: max_i64:
 ; RV32IBB:       # %bb.0:
 ; RV32IBB-NEXT:    mv a4, a0
 ; RV32IBB-NEXT:    bge a3, a1, .LBB21_3
 ; RV32IBB-NEXT:  # %bb.1:

 ; RV32I-NEXT:  .LBB23_3:
 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    mv a1, a3
 ; RV32I-NEXT:  .LBB23_4:
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: minu_i64:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    mv a4, a0
-; RV32IB-NEXT:    bgeu a1, a3, .LBB23_3
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    beq a1, a3, .LBB23_4
-; RV32IB-NEXT:  .LBB23_2:
-; RV32IB-NEXT:    minu a1, a1, a3
-; RV32IB-NEXT:    ret
-; RV32IB-NEXT:  .LBB23_3:
-; RV32IB-NEXT:    mv a0, a2
-; RV32IB-NEXT:    bne a1, a3, .LBB23_2
-; RV32IB-NEXT:  .LBB23_4:
-; RV32IB-NEXT:    minu a0, a4, a2
+; RV32IB-NEXT:    sltu a4, a1, a3
+; RV32IB-NEXT:    cmov a4, a4, a2, a0
+; RV32IB-NEXT:    xor a5, a1, a3
+; RV32IB-NEXT:    minu a0, a0, a2
+; RV32IB-NEXT:    cmov a0, a5, a0, a4
 ; RV32IB-NEXT:    minu a1, a1, a3
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBB-LABEL: minu_i64:
 ; RV32IBB:       # %bb.0:
 ; RV32IBB-NEXT:    mv a4, a0
 ; RV32IBB-NEXT:    bgeu a1, a3, .LBB23_3
 ; RV32IBB-NEXT:  # %bb.1:

 ; RV32I-NEXT:  .LBB25_3:
 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    mv a1, a3
 ; RV32I-NEXT:  .LBB25_4:
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: maxu_i64:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    mv a4, a0
-; RV32IB-NEXT:    bgeu a3, a1, .LBB25_3
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    beq a1, a3, .LBB25_4
-; RV32IB-NEXT:  .LBB25_2:
-; RV32IB-NEXT:    maxu a1, a1, a3
-; RV32IB-NEXT:    ret
-; RV32IB-NEXT:  .LBB25_3:
-; RV32IB-NEXT:    mv a0, a2
-; RV32IB-NEXT:    bne a1, a3, .LBB25_2
-; RV32IB-NEXT:  .LBB25_4:
-; RV32IB-NEXT:    maxu a0, a4, a2
+; RV32IB-NEXT:    sltu a4, a3, a1
+; RV32IB-NEXT:    cmov a4, a4, a2, a0
+; RV32IB-NEXT:    xor a5, a1, a3
+; RV32IB-NEXT:    maxu a0, a0, a2
+; RV32IB-NEXT:    cmov a0, a5, a0, a4
 ; RV32IB-NEXT:    maxu a1, a1, a3
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBB-LABEL: maxu_i64:
 ; RV32IBB:       # %bb.0:
 ; RV32IBB-NEXT:    mv a4, a0
 ; RV32IBB-NEXT:    bgeu a3, a1, .LBB25_3
 ; RV32IBB-NEXT:  # %bb.1:

 ; RV32I-NEXT:    neg a0, a0
 ; RV32I-NEXT:    add a1, a1, a2
 ; RV32I-NEXT:    neg a1, a1
 ; RV32I-NEXT:  .LBB27_2:
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: abs_i64:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    bgez a1, .LBB27_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    snez a2, a0
+; RV32IB-NEXT:    neg a2, a0
+; RV32IB-NEXT:    slti a3, a1, 0
+; RV32IB-NEXT:    cmov a2, a3, a0, a2
+; RV32IB-NEXT:    snez a0, a0
+; RV32IB-NEXT:    add a0, a1, a0
 ; RV32IB-NEXT:    neg a0, a0
-; RV32IB-NEXT:    add a1, a1, a2
-; RV32IB-NEXT:    neg a1, a1
-; RV32IB-NEXT:  .LBB27_2:
+; RV32IB-NEXT:    cmov a1, a3, a1, a0
+; RV32IB-NEXT:    mv a0, a2
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBB-LABEL: abs_i64:
 ; RV32IBB:       # %bb.0:
 ; RV32IBB-NEXT:    bgez a1, .LBB27_2
 ; RV32IBB-NEXT:  # %bb.1:
 ; RV32IBB-NEXT:    snez a2, a0
 ; RV32IBB-NEXT:    neg a0, a0
 ; RV32IBB-NEXT:    add a1, a1, a2
 ; RV32IBB-NEXT:    neg a1, a1
 ; RV32IBB-NEXT:  .LBB27_2:
 ; RV32IBB-NEXT:    ret
   %abs = tail call i64 @llvm.abs.i64(i64 %x, i1 true)
   ret i64 %abs
 }

llvm/test/CodeGen/RISCV/rv32Zbbp.ll

 ; RV32I-NEXT:    or a1, a7, t0
 ; RV32I-NEXT:    bltz t1, .LBB7_6
 ; RV32I-NEXT:  .LBB7_9:
 ; RV32I-NEXT:    or a0, zero, a4
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: rol_i64:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    andi a3, a2, 63
-; RV32IB-NEXT:    addi t1, a3, -32
+; RV32IB-NEXT:    sll a7, a1, a2
+; RV32IB-NEXT:    andi a4, a2, 63
 ; RV32IB-NEXT:    addi a6, zero, 31
-; RV32IB-NEXT:    bltz t1, .LBB7_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    sll a7, a0, t1
-; RV32IB-NEXT:    j .LBB7_3
-; RV32IB-NEXT:  .LBB7_2:
-; RV32IB-NEXT:    sll a4, a1, a2
-; RV32IB-NEXT:    sub a3, a6, a3
-; RV32IB-NEXT:    srli a5, a0, 1
-; RV32IB-NEXT:    srl a3, a5, a3
-; RV32IB-NEXT:    or a7, a4, a3
-; RV32IB-NEXT:  .LBB7_3:
-; RV32IB-NEXT:    neg a4, a2
-; RV32IB-NEXT:    andi a5, a4, 63
-; RV32IB-NEXT:    addi a3, a5, -32
-; RV32IB-NEXT:    bltz a3, .LBB7_7
-; RV32IB-NEXT:  # %bb.4:
-; RV32IB-NEXT:    mv t0, zero
-; RV32IB-NEXT:    bgez a3, .LBB7_8
-; RV32IB-NEXT:  .LBB7_5:
-; RV32IB-NEXT:    srl a3, a0, a4
-; RV32IB-NEXT:    sub a4, a6, a5
-; RV32IB-NEXT:    slli a1, a1, 1
-; RV32IB-NEXT:    sll a1, a1, a4
-; RV32IB-NEXT:    or a4, a3, a1
-; RV32IB-NEXT:    or a1, a7, t0
-; RV32IB-NEXT:    bgez t1, .LBB7_9
-; RV32IB-NEXT:  .LBB7_6:
+; RV32IB-NEXT:    sub a5, a6, a4
+; RV32IB-NEXT:    srli a3, a0, 1
+; RV32IB-NEXT:    srl a3, a3, a5
+; RV32IB-NEXT:    or a7, a7, a3
+; RV32IB-NEXT:    addi t1, a4, -32
+; RV32IB-NEXT:    sll a5, a0, t1
+; RV32IB-NEXT:    slti a3, t1, 0
+; RV32IB-NEXT:    cmov a7, a3, a5, a7
+; RV32IB-NEXT:    neg a5, a2
+; RV32IB-NEXT:    srl t0, a1, a5
+; RV32IB-NEXT:    andi t2, a5, 63
+; RV32IB-NEXT:    addi a4, t2, -32
+; RV32IB-NEXT:    srai a3, a4, 31
+; RV32IB-NEXT:    and a3, a3, t0
+; RV32IB-NEXT:    or a7, a7, a3
+; RV32IB-NEXT:    srl t0, a0, a5
+; RV32IB-NEXT:    sub a5, a6, t2
+; RV32IB-NEXT:    slli a3, a1, 1
+; RV32IB-NEXT:    sll a3, a3, a5
+; RV32IB-NEXT:    or a3, t0, a3
+; RV32IB-NEXT:    srl a1, a1, a4
+; RV32IB-NEXT:    slti a4, a4, 0
+; RV32IB-NEXT:    cmov a1, a4, a1, a3
 ; RV32IB-NEXT:    sll a0, a0, a2
-; RV32IB-NEXT:    or a0, a0, a4
-; RV32IB-NEXT:    ret
-; RV32IB-NEXT:  .LBB7_7:
-; RV32IB-NEXT:    srl t0, a1, a4
-; RV32IB-NEXT:    bltz a3, .LBB7_5
-; RV32IB-NEXT:  .LBB7_8:
-; RV32IB-NEXT:    srl a4, a1, a3
-; RV32IB-NEXT:    or a1, a7, t0
-; RV32IB-NEXT:    bltz t1, .LBB7_6
-; RV32IB-NEXT:  .LBB7_9:
-; RV32IB-NEXT:    or a0, zero, a4
+; RV32IB-NEXT:    srai a2, t1, 31
+; RV32IB-NEXT:    and a0, a2, a0
+; RV32IB-NEXT:    or a0, a0, a1
+; RV32IB-NEXT:    mv a1, a7
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBB-LABEL: rol_i64:
 ; RV32IBB:       # %bb.0:
 ; RV32IBB-NEXT:    andi a3, a2, 63
 ; RV32IBB-NEXT:    addi t1, a3, -32
 ; RV32IBB-NEXT:    addi a6, zero, 31
 ; RV32IBB-NEXT:    bltz t1, .LBB7_2

 ; RV32I-NEXT:    or a0, a7, t0
 ; RV32I-NEXT:    bltz t1, .LBB9_6
 ; RV32I-NEXT:  .LBB9_9:
 ; RV32I-NEXT:    or a1, zero, a4
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: ror_i64:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    andi a3, a2, 63
-; RV32IB-NEXT:    addi t1, a3, -32
+; RV32IB-NEXT:    srl a7, a0, a2
+; RV32IB-NEXT:    andi a4, a2, 63
 ; RV32IB-NEXT:    addi a6, zero, 31
-; RV32IB-NEXT:    bltz t1, .LBB9_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    srl a7, a1, t1
-; RV32IB-NEXT:    j .LBB9_3
-; RV32IB-NEXT:  .LBB9_2:
-; RV32IB-NEXT:    srl a4, a0, a2
-; RV32IB-NEXT:    sub a3, a6, a3
-; RV32IB-NEXT:    slli a5, a1, 1
-; RV32IB-NEXT:    sll a3, a5, a3
-; RV32IB-NEXT:    or a7, a4, a3
-; RV32IB-NEXT:  .LBB9_3:
-; RV32IB-NEXT:    neg a4, a2
-; RV32IB-NEXT:    andi a5, a4, 63
-; RV32IB-NEXT:    addi a3, a5, -32
-; RV32IB-NEXT:    bltz a3, .LBB9_7
-; RV32IB-NEXT:  # %bb.4:
-; RV32IB-NEXT:    mv t0, zero
-; RV32IB-NEXT:    bgez a3, .LBB9_8
-; RV32IB-NEXT:  .LBB9_5:
-; RV32IB-NEXT:    sll a3, a1, a4
-; RV32IB-NEXT:    sub a4, a6, a5
-; RV32IB-NEXT:    srli a0, a0, 1
-; RV32IB-NEXT:    srl a0, a0, a4
-; RV32IB-NEXT:    or a4, a3, a0
-; RV32IB-NEXT:    or a0, a7, t0
-; RV32IB-NEXT:    bgez t1, .LBB9_9
-; RV32IB-NEXT:  .LBB9_6:
+; RV32IB-NEXT:    sub a5, a6, a4
+; RV32IB-NEXT:    slli a3, a1, 1
+; RV32IB-NEXT:    sll a3, a3, a5
+; RV32IB-NEXT:    or a7, a7, a3
+; RV32IB-NEXT:    addi t1, a4, -32
+; RV32IB-NEXT:    srl a5, a1, t1
+; RV32IB-NEXT:    slti a3, t1, 0
+; RV32IB-NEXT:    cmov a7, a3, a5, a7
+; RV32IB-NEXT:    neg a5, a2
+; RV32IB-NEXT:    sll t0, a0, a5
+; RV32IB-NEXT:    andi t2, a5, 63
+; RV32IB-NEXT:    addi a4, t2, -32
+; RV32IB-NEXT:    srai a3, a4, 31
+; RV32IB-NEXT:    and a3, a3, t0
+; RV32IB-NEXT:    or a7, a7, a3
+; RV32IB-NEXT:    sll t0, a1, a5
+; RV32IB-NEXT:    sub a5, a6, t2
+; RV32IB-NEXT:    srli a3, a0, 1
+; RV32IB-NEXT:    srl a3, a3, a5
+; RV32IB-NEXT:    or a3, t0, a3
+; RV32IB-NEXT:    sll a0, a0, a4
+; RV32IB-NEXT:    slti a4, a4, 0
+; RV32IB-NEXT:    cmov a0, a4, a0, a3
 ; RV32IB-NEXT:    srl a1, a1, a2
-; RV32IB-NEXT:    or a1, a1, a4
-; RV32IB-NEXT:    ret
-; RV32IB-NEXT:  .LBB9_7:
-; RV32IB-NEXT:    sll t0, a0, a4
-; RV32IB-NEXT:    bltz a3, .LBB9_5
-; RV32IB-NEXT:  .LBB9_8:
-; RV32IB-NEXT:    sll a4, a0, a3
-; RV32IB-NEXT:    or a0, a7, t0
-; RV32IB-NEXT:    bltz t1, .LBB9_6
-; RV32IB-NEXT:  .LBB9_9:
-; RV32IB-NEXT:    or a1, zero, a4
+; RV32IB-NEXT:    srai a2, t1, 31
+; RV32IB-NEXT:    and a1, a2, a1
+; RV32IB-NEXT:    or a1, a1, a0
+; RV32IB-NEXT:    mv a0, a7
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBB-LABEL: ror_i64:
 ; RV32IBB:       # %bb.0:
 ; RV32IBB-NEXT:    andi a3, a2, 63
 ; RV32IBB-NEXT:    addi t1, a3, -32
 ; RV32IBB-NEXT:    addi a6, zero, 31
 ; RV32IBB-NEXT:    bltz t1, .LBB9_2

llvm/test/CodeGen/RISCV/rv32Zbs.ll

 ; RV32I-NEXT:    and a0, a2, a0
 ; RV32I-NEXT:    and a1, a3, a1
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: sbclr_i64:
 ; RV32IB:       # %bb.0:
 ; RV32IB-NEXT:    andi a3, a2, 63
 ; RV32IB-NEXT:    addi a3, a3, -32
-; RV32IB-NEXT:    bltz a3, .LBB2_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    mv a2, zero
-; RV32IB-NEXT:    sbset a3, zero, a3
-; RV32IB-NEXT:    j .LBB2_3
-; RV32IB-NEXT:  .LBB2_2:
-; RV32IB-NEXT:    mv a3, zero
+; RV32IB-NEXT:    slti a4, a3, 0
+; RV32IB-NEXT:    sbset a5, zero, a3
+; RV32IB-NEXT:    cmov a4, a4, a5, zero
 ; RV32IB-NEXT:    sbset a2, zero, a2
-; RV32IB-NEXT:  .LBB2_3:
+; RV32IB-NEXT:    srai a3, a3, 31
+; RV32IB-NEXT:    and a2, a3, a2
+; RV32IB-NEXT:    andn a1, a1, a4
 ; RV32IB-NEXT:    andn a0, a0, a2
-; RV32IB-NEXT:    andn a1, a1, a3
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBS-LABEL: sbclr_i64:
 ; RV32IBS:       # %bb.0:
 ; RV32IBS-NEXT:    andi a3, a2, 63
 ; RV32IBS-NEXT:    addi a3, a3, -32
 ; RV32IBS-NEXT:    bltz a3, .LBB2_2
 ; RV32IBS-NEXT:  # %bb.1:

 ; RV32I-NEXT:    ret
 ; RV32I-NEXT:  .LBB7_2:
 ; RV32I-NEXT:    mv a1, zero
 ; RV32I-NEXT:    sll a0, a2, a0
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: sbset_i64_zero:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    addi a1, a0, -32
-; RV32IB-NEXT:    bltz a1, .LBB7_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    mv a0, zero
-; RV32IB-NEXT:    sbset a1, zero, a1
-; RV32IB-NEXT:    ret
-; RV32IB-NEXT:  .LBB7_2:
-; RV32IB-NEXT:    mv a1, zero
+; RV32IB-NEXT:    addi a2, a0, -32
+; RV32IB-NEXT:    slti a1, a2, 0
+; RV32IB-NEXT:    sbset a3, zero, a2
+; RV32IB-NEXT:    cmov a1, a1, a3, zero
 ; RV32IB-NEXT:    sbset a0, zero, a0
+; RV32IB-NEXT:    srai a2, a2, 31
+; RV32IB-NEXT:    and a0, a2, a0
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBS-LABEL: sbset_i64_zero:
 ; RV32IBS:       # %bb.0:
 ; RV32IBS-NEXT:    addi a1, a0, -32
 ; RV32IBS-NEXT:    bltz a1, .LBB7_2
 ; RV32IBS-NEXT:  # %bb.1:
 ; RV32IBS-NEXT:    mv a0, zero

 ; RV32I-NEXT:    or a0, a0, a1
 ; RV32I-NEXT:  .LBB12_3:
 ; RV32I-NEXT:    andi a0, a0, 1
 ; RV32I-NEXT:    mv a1, zero
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: sbext_i64:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    andi a3, a2, 63
-; RV32IB-NEXT:    addi a4, a3, -32
-; RV32IB-NEXT:    bltz a4, .LBB12_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    srl a0, a1, a4
-; RV32IB-NEXT:    j .LBB12_3
-; RV32IB-NEXT:  .LBB12_2:
 ; RV32IB-NEXT:    srl a0, a0, a2
-; RV32IB-NEXT:    addi a2, zero, 31
-; RV32IB-NEXT:    sub a2, a2, a3
-; RV32IB-NEXT:    slli a1, a1, 1
-; RV32IB-NEXT:    sll a1, a1, a2
-; RV32IB-NEXT:    or a0, a0, a1
-; RV32IB-NEXT:  .LBB12_3:
+; RV32IB-NEXT:    andi a2, a2, 63
+; RV32IB-NEXT:    addi a3, zero, 31
+; RV32IB-NEXT:    sub a3, a3, a2
+; RV32IB-NEXT:    slli a4, a1, 1
+; RV32IB-NEXT:    sll a3, a4, a3
+; RV32IB-NEXT:    or a0, a0, a3
+; RV32IB-NEXT:    addi a2, a2, -32
+; RV32IB-NEXT:    srl a1, a1, a2
+; RV32IB-NEXT:    slti a2, a2, 0
+; RV32IB-NEXT:    cmov a0, a2, a1, a0
 ; RV32IB-NEXT:    andi a0, a0, 1
 ; RV32IB-NEXT:    mv a1, zero
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBS-LABEL: sbext_i64:
 ; RV32IBS:       # %bb.0:
 ; RV32IBS-NEXT:    andi a3, a2, 63
 ; RV32IBS-NEXT:    addi a4, a3, -32

llvm/test/CodeGen/RISCV/rv32Zbt.ll

 ; RV32I-NEXT:  # %bb.1:
 ; RV32I-NEXT:    mv a3, a0
 ; RV32I-NEXT:  .LBB3_2:
 ; RV32I-NEXT:    mv a0, a3
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: cmov_sle_i32:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    bge a2, a1, .LBB3_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    mv a3, a0
-; RV32IB-NEXT:  .LBB3_2:
-; RV32IB-NEXT:    mv a0, a3
+; RV32IB-NEXT:    slt a1, a2, a1
+; RV32IB-NEXT:    cmov a0, a1, a3, a0
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBT-LABEL: cmov_sle_i32:
 ; RV32IBT:       # %bb.0:
-; RV32IBT-NEXT:    bge a2, a1, .LBB3_2
-; RV32IBT-NEXT:  # %bb.1:
-; RV32IBT-NEXT:    mv a3, a0
-; RV32IBT-NEXT:  .LBB3_2:
-; RV32IBT-NEXT:    mv a0, a3
+; RV32IBT-NEXT:    slt a1, a2, a1
+; RV32IBT-NEXT:    cmov a0, a1, a3, a0
 ; RV32IBT-NEXT:    ret
   %tobool = icmp sle i32 %b, %c
   %cond = select i1 %tobool, i32 %d, i32 %a
   ret i32 %cond
 }
 
 define i32 @cmov_sge_i32(i32 %a, i32 %b, i32 %c, i32 %d) nounwind {
 ; RV32I-LABEL: cmov_sge_i32:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    bge a1, a2, .LBB4_2
 ; RV32I-NEXT:  # %bb.1:
 ; RV32I-NEXT:    mv a3, a0
 ; RV32I-NEXT:  .LBB4_2:
 ; RV32I-NEXT:    mv a0, a3
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: cmov_sge_i32:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    bge a1, a2, .LBB4_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    mv a3, a0
-; RV32IB-NEXT:  .LBB4_2:
-; RV32IB-NEXT:    mv a0, a3
+; RV32IB-NEXT:    slt a1, a1, a2
+; RV32IB-NEXT:    cmov a0, a1, a3, a0
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBT-LABEL: cmov_sge_i32:
 ; RV32IBT:       # %bb.0:
-; RV32IBT-NEXT:    bge a1, a2, .LBB4_2
-; RV32IBT-NEXT:  # %bb.1:
-; RV32IBT-NEXT:    mv a3, a0
-; RV32IBT-NEXT:  .LBB4_2:
-; RV32IBT-NEXT:    mv a0, a3
+; RV32IBT-NEXT:    slt a1, a1, a2
+; RV32IBT-NEXT:    cmov a0, a1, a3, a0
 ; RV32IBT-NEXT:    ret
   %tobool = icmp sge i32 %b, %c
   %cond = select i1 %tobool, i32 %d, i32 %a
   ret i32 %cond
 }
 
 define i32 @cmov_ule_i32(i32 %a, i32 %b, i32 %c, i32 %d) nounwind {
 ; RV32I-LABEL: cmov_ule_i32:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    bgeu a2, a1, .LBB5_2
 ; RV32I-NEXT:  # %bb.1:
 ; RV32I-NEXT:    mv a3, a0
 ; RV32I-NEXT:  .LBB5_2:
 ; RV32I-NEXT:    mv a0, a3
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: cmov_ule_i32:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    bgeu a2, a1, .LBB5_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    mv a3, a0
-; RV32IB-NEXT:  .LBB5_2:
-; RV32IB-NEXT:    mv a0, a3
+; RV32IB-NEXT:    sltu a1, a2, a1
+; RV32IB-NEXT:    cmov a0, a1, a3, a0
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBT-LABEL: cmov_ule_i32:
 ; RV32IBT:       # %bb.0:
-; RV32IBT-NEXT:    bgeu a2, a1, .LBB5_2
-; RV32IBT-NEXT:  # %bb.1:
-; RV32IBT-NEXT:    mv a3, a0
-; RV32IBT-NEXT:  .LBB5_2:
-; RV32IBT-NEXT:    mv a0, a3
+; RV32IBT-NEXT:    sltu a1, a2, a1
+; RV32IBT-NEXT:    cmov a0, a1, a3, a0
 ; RV32IBT-NEXT:    ret
   %tobool = icmp ule i32 %b, %c
   %cond = select i1 %tobool, i32 %d, i32 %a
   ret i32 %cond
 }
 
 define i32 @cmov_uge_i32(i32 %a, i32 %b, i32 %c, i32 %d) nounwind {
 ; RV32I-LABEL: cmov_uge_i32:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    bgeu a1, a2, .LBB6_2
 ; RV32I-NEXT:  # %bb.1:
 ; RV32I-NEXT:    mv a3, a0
 ; RV32I-NEXT:  .LBB6_2:
 ; RV32I-NEXT:    mv a0, a3
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: cmov_uge_i32:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    bgeu a1, a2, .LBB6_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    mv a3, a0
-; RV32IB-NEXT:  .LBB6_2:
-; RV32IB-NEXT:    mv a0, a3
+; RV32IB-NEXT:    sltu a1, a1, a2
+; RV32IB-NEXT:    cmov a0, a1, a3, a0
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBT-LABEL: cmov_uge_i32:
 ; RV32IBT:       # %bb.0:
-; RV32IBT-NEXT:    bgeu a1, a2, .LBB6_2
-; RV32IBT-NEXT:  # %bb.1:
-; RV32IBT-NEXT:    mv a3, a0
-; RV32IBT-NEXT:  .LBB6_2:
-; RV32IBT-NEXT:    mv a0, a3
+; RV32IBT-NEXT:    sltu a1, a1, a2
+; RV32IBT-NEXT:    cmov a0, a1, a3, a0
 ; RV32IBT-NEXT:    ret
   %tobool = icmp uge i32 %b, %c
   %cond = select i1 %tobool, i32 %d, i32 %a
   ret i32 %cond
 }
 
 define i64 @cmov_i64(i64 %a, i64 %b, i64 %c) nounwind {
 ; RV32I-LABEL: cmov_i64:

 ; RV32I-NEXT:    mv a7, a1
 ; RV32I-NEXT:  .LBB8_4:
 ; RV32I-NEXT:    mv a0, a6
 ; RV32I-NEXT:    mv a1, a7
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: cmov_sle_i64:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    beq a3, a5, .LBB8_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    slt a2, a5, a3
-; RV32IB-NEXT:    xori a2, a2, 1
-; RV32IB-NEXT:    beqz a2, .LBB8_3
-; RV32IB-NEXT:    j .LBB8_4
-; RV32IB-NEXT:  .LBB8_2:
+; RV32IB-NEXT:    xor t0, a3, a5
+; RV32IB-NEXT:    slt a3, a5, a3
+; RV32IB-NEXT:    xori a3, a3, 1
 ; RV32IB-NEXT:    sltu a2, a4, a2
 ; RV32IB-NEXT:    xori a2, a2, 1
-; RV32IB-NEXT:    bnez a2, .LBB8_4
-; RV32IB-NEXT:  .LBB8_3:
-; RV32IB-NEXT:    mv a6, a0
-; RV32IB-NEXT:    mv a7, a1
-; RV32IB-NEXT:  .LBB8_4:
-; RV32IB-NEXT:    mv a0, a6
-; RV32IB-NEXT:    mv a1, a7
+; RV32IB-NEXT:    cmov a2, t0, a2, a3
+; RV32IB-NEXT:    cmov a0, a2, a0, a6
+; RV32IB-NEXT:    cmov a1, a2, a1, a7
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBT-LABEL: cmov_sle_i64:
 ; RV32IBT:       # %bb.0:
-; RV32IBT-NEXT:    beq a3, a5, .LBB8_2
-; RV32IBT-NEXT:  # %bb.1:
-; RV32IBT-NEXT:    slt a2, a5, a3
-; RV32IBT-NEXT:    xori a2, a2, 1
-; RV32IBT-NEXT:    beqz a2, .LBB8_3
-; RV32IBT-NEXT:    j .LBB8_4
-; RV32IBT-NEXT:  .LBB8_2:
+; RV32IBT-NEXT:    xor t0, a3, a5
+; RV32IBT-NEXT:    slt a3, a5, a3
+; RV32IBT-NEXT:    xori a3, a3, 1
 ; RV32IBT-NEXT:    sltu a2, a4, a2
 ; RV32IBT-NEXT:    xori a2, a2, 1
-; RV32IBT-NEXT:    bnez a2, .LBB8_4
-; RV32IBT-NEXT:  .LBB8_3:
-; RV32IBT-NEXT:    mv a6, a0
-; RV32IBT-NEXT:    mv a7, a1
-; RV32IBT-NEXT:  .LBB8_4:
-; RV32IBT-NEXT:    mv a0, a6
-; RV32IBT-NEXT:    mv a1, a7
+; RV32IBT-NEXT:    cmov a2, t0, a2, a3
+; RV32IBT-NEXT:    cmov a0, a2, a0, a6
+; RV32IBT-NEXT:    cmov a1, a2, a1, a7
 ; RV32IBT-NEXT:    ret
   %tobool = icmp sle i64 %b, %c
   %cond = select i1 %tobool, i64 %d, i64 %a
   ret i64 %cond
 }
 
 define i64 @cmov_sge_i64(i64 %a, i64 %b, i64 %c, i64 %d) nounwind {
 ; RV32I-LABEL: cmov_sge_i64:

 ; RV32I-NEXT:    mv a7, a1
 ; RV32I-NEXT:  .LBB9_4:
 ; RV32I-NEXT:    mv a0, a6
 ; RV32I-NEXT:    mv a1, a7
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: cmov_sge_i64:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    beq a3, a5, .LBB9_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    slt a2, a3, a5
-; RV32IB-NEXT:    xori a2, a2, 1
-; RV32IB-NEXT:    beqz a2, .LBB9_3
-; RV32IB-NEXT:    j .LBB9_4
-; RV32IB-NEXT:  .LBB9_2:
+; RV32IB-NEXT:    xor t0, a3, a5
+; RV32IB-NEXT:    slt a3, a3, a5
+; RV32IB-NEXT:    xori a3, a3, 1
 ; RV32IB-NEXT:    sltu a2, a2, a4
 ; RV32IB-NEXT:    xori a2, a2, 1
-; RV32IB-NEXT:    bnez a2, .LBB9_4
-; RV32IB-NEXT:  .LBB9_3:
-; RV32IB-NEXT:    mv a6, a0
-; RV32IB-NEXT:    mv a7, a1
-; RV32IB-NEXT:  .LBB9_4:
-; RV32IB-NEXT:    mv a0, a6
-; RV32IB-NEXT:    mv a1, a7
+; RV32IB-NEXT:    cmov a2, t0, a2, a3
+; RV32IB-NEXT:    cmov a0, a2, a0, a6
+; RV32IB-NEXT:    cmov a1, a2, a1, a7
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBT-LABEL: cmov_sge_i64:
 ; RV32IBT:       # %bb.0:
-; RV32IBT-NEXT:    beq a3, a5, .LBB9_2
-; RV32IBT-NEXT:  # %bb.1:
-; RV32IBT-NEXT:    slt a2, a3, a5
-; RV32IBT-NEXT:    xori a2, a2, 1
-; RV32IBT-NEXT:    beqz a2, .LBB9_3
-; RV32IBT-NEXT:    j .LBB9_4
-; RV32IBT-NEXT:  .LBB9_2:
+; RV32IBT-NEXT:    xor t0, a3, a5
+; RV32IBT-NEXT:    slt a3, a3, a5
+; RV32IBT-NEXT:    xori a3, a3, 1
 ; RV32IBT-NEXT:    sltu a2, a2, a4
 ; RV32IBT-NEXT:    xori a2, a2, 1
-; RV32IBT-NEXT:    bnez a2, .LBB9_4
-; RV32IBT-NEXT:  .LBB9_3:
-; RV32IBT-NEXT:    mv a6, a0
-; RV32IBT-NEXT:    mv a7, a1
-; RV32IBT-NEXT:  .LBB9_4:
-; RV32IBT-NEXT:    mv a0, a6
-; RV32IBT-NEXT:    mv a1, a7
+; RV32IBT-NEXT:    cmov a2, t0, a2, a3
+; RV32IBT-NEXT:    cmov a0, a2, a0, a6
+; RV32IBT-NEXT:    cmov a1, a2, a1, a7
 ; RV32IBT-NEXT:    ret
   %tobool = icmp sge i64 %b, %c
   %cond = select i1 %tobool, i64 %d, i64 %a
   ret i64 %cond
 }
 
 define i64 @cmov_ule_i64(i64 %a, i64 %b, i64 %c, i64 %d) nounwind {
 ; RV32I-LABEL: cmov_ule_i64:

 ; RV32I-NEXT:    mv a7, a1
 ; RV32I-NEXT:  .LBB10_4:
 ; RV32I-NEXT:    mv a0, a6
 ; RV32I-NEXT:    mv a1, a7
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: cmov_ule_i64:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    beq a3, a5, .LBB10_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    sltu a2, a5, a3
-; RV32IB-NEXT:    xori a2, a2, 1
-; RV32IB-NEXT:    beqz a2, .LBB10_3
-; RV32IB-NEXT:    j .LBB10_4
-; RV32IB-NEXT:  .LBB10_2:
+; RV32IB-NEXT:    xor t0, a3, a5
+; RV32IB-NEXT:    sltu a3, a5, a3
+; RV32IB-NEXT:    xori a3, a3, 1
 ; RV32IB-NEXT:    sltu a2, a4, a2
 ; RV32IB-NEXT:    xori a2, a2, 1
-; RV32IB-NEXT:    bnez a2, .LBB10_4
-; RV32IB-NEXT:  .LBB10_3:
-; RV32IB-NEXT:    mv a6, a0
-; RV32IB-NEXT:    mv a7, a1
-; RV32IB-NEXT:  .LBB10_4:
-; RV32IB-NEXT:    mv a0, a6
-; RV32IB-NEXT:    mv a1, a7
+; RV32IB-NEXT:    cmov a2, t0, a2, a3
+; RV32IB-NEXT:    cmov a0, a2, a0, a6
+; RV32IB-NEXT:    cmov a1, a2, a1, a7
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBT-LABEL: cmov_ule_i64:
 ; RV32IBT:       # %bb.0:
-; RV32IBT-NEXT:    beq a3, a5, .LBB10_2
-; RV32IBT-NEXT:  # %bb.1:
-; RV32IBT-NEXT:    sltu a2, a5, a3
-; RV32IBT-NEXT:    xori a2, a2, 1
-; RV32IBT-NEXT:    beqz a2, .LBB10_3
-; RV32IBT-NEXT:    j .LBB10_4
-; RV32IBT-NEXT:  .LBB10_2:
+; RV32IBT-NEXT:    xor t0, a3, a5
+; RV32IBT-NEXT:    sltu a3, a5, a3
+; RV32IBT-NEXT:    xori a3, a3, 1
 ; RV32IBT-NEXT:    sltu a2, a4, a2
 ; RV32IBT-NEXT:    xori a2, a2, 1
-; RV32IBT-NEXT:    bnez a2, .LBB10_4
-; RV32IBT-NEXT:  .LBB10_3:
-; RV32IBT-NEXT:    mv a6, a0
-; RV32IBT-NEXT:    mv a7, a1
-; RV32IBT-NEXT:  .LBB10_4:
-; RV32IBT-NEXT:    mv a0, a6
-; RV32IBT-NEXT:    mv a1, a7
+; RV32IBT-NEXT:    cmov a2, t0, a2, a3
+; RV32IBT-NEXT:    cmov a0, a2, a0, a6
+; RV32IBT-NEXT:    cmov a1, a2, a1, a7
 ; RV32IBT-NEXT:    ret
   %tobool = icmp ule i64 %b, %c
   %cond = select i1 %tobool, i64 %d, i64 %a
   ret i64 %cond
 }
 
 define i64 @cmov_uge_i64(i64 %a, i64 %b, i64 %c, i64 %d) nounwind {
 ; RV32I-LABEL: cmov_uge_i64:

 ; RV32I-NEXT:    mv a7, a1
 ; RV32I-NEXT:  .LBB11_4:
 ; RV32I-NEXT:    mv a0, a6
 ; RV32I-NEXT:    mv a1, a7
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: cmov_uge_i64:
 ; RV32IB:       # %bb.0:
-; RV32IB-NEXT:    beq a3, a5, .LBB11_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    sltu a2, a3, a5
-; RV32IB-NEXT:    xori a2, a2, 1
-; RV32IB-NEXT:    beqz a2, .LBB11_3
-; RV32IB-NEXT:    j .LBB11_4
-; RV32IB-NEXT:  .LBB11_2:
+; RV32IB-NEXT:    xor t0, a3, a5
+; RV32IB-NEXT:    sltu a3, a3, a5
+; RV32IB-NEXT:    xori a3, a3, 1
 ; RV32IB-NEXT:    sltu a2, a2, a4
 ; RV32IB-NEXT:    xori a2, a2, 1
-; RV32IB-NEXT:    bnez a2, .LBB11_4
-; RV32IB-NEXT:  .LBB11_3:
-; RV32IB-NEXT:    mv a6, a0
-; RV32IB-NEXT:    mv a7, a1
-; RV32IB-NEXT:  .LBB11_4:
-; RV32IB-NEXT:    mv a0, a6
-; RV32IB-NEXT:    mv a1, a7
+; RV32IB-NEXT:    cmov a2, t0, a2, a3
+; RV32IB-NEXT:    cmov a0, a2, a0, a6
+; RV32IB-NEXT:    cmov a1, a2, a1, a7
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBT-LABEL: cmov_uge_i64:
 ; RV32IBT:       # %bb.0:
-; RV32IBT-NEXT:    beq a3, a5, .LBB11_2
-; RV32IBT-NEXT:  # %bb.1:
-; RV32IBT-NEXT:    sltu a2, a3, a5
-; RV32IBT-NEXT:    xori a2, a2, 1
-; RV32IBT-NEXT:    beqz a2, .LBB11_3
-; RV32IBT-NEXT:    j .LBB11_4
-; RV32IBT-NEXT:  .LBB11_2:
+; RV32IBT-NEXT:    xor t0, a3, a5
+; RV32IBT-NEXT:    sltu a3, a3, a5
+; RV32IBT-NEXT:    xori a3, a3, 1
 ; RV32IBT-NEXT:    sltu a2, a2, a4
 ; RV32IBT-NEXT:    xori a2, a2, 1
-; RV32IBT-NEXT:    bnez a2, .LBB11_4
-; RV32IBT-NEXT:  .LBB11_3:
-; RV32IBT-NEXT:    mv a6, a0
-; RV32IBT-NEXT:    mv a7, a1
-; RV32IBT-NEXT:  .LBB11_4:
-; RV32IBT-NEXT:    mv a0, a6
-; RV32IBT-NEXT:    mv a1, a7
+; RV32IBT-NEXT:    cmov a2, t0, a2, a3
+; RV32IBT-NEXT:    cmov a0, a2, a0, a6
+; RV32IBT-NEXT:    cmov a1, a2, a1, a7
 ; RV32IBT-NEXT:    ret
   %tobool = icmp uge i64 %b, %c
   %cond = select i1 %tobool, i64 %d, i64 %a
   ret i64 %cond
 }
 
 declare i32 @llvm.fshl.i32(i32, i32, i32)
 

 ; RV32I-NEXT:    or a1, a7, t0
 ; RV32I-NEXT:    bltz t1, .LBB13_6
 ; RV32I-NEXT:  .LBB13_9:
 ; RV32I-NEXT:    or a0, zero, a2
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: fshl_i64:
 ; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    sll a7, a1, a4
 ; RV32IB-NEXT:    andi a5, a4, 63
-; RV32IB-NEXT:    addi t2, a5, -32
 ; RV32IB-NEXT:    addi a6, zero, 31
-; RV32IB-NEXT:    bltz t2, .LBB13_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    sll a7, a0, t2
-; RV32IB-NEXT:    j .LBB13_3
-; RV32IB-NEXT:  .LBB13_2:
-; RV32IB-NEXT:    sll a7, a1, a4
-; RV32IB-NEXT:    sub a5, a6, a5
+; RV32IB-NEXT:    sub t0, a6, a5
 ; RV32IB-NEXT:    srli a1, a0, 1
-; RV32IB-NEXT:    srl a1, a1, a5
+; RV32IB-NEXT:    srl a1, a1, t0
 ; RV32IB-NEXT:    or a7, a7, a1
-; RV32IB-NEXT:  .LBB13_3:
-; RV32IB-NEXT:    not t1, a4
+; RV32IB-NEXT:    addi t1, a5, -32
+; RV32IB-NEXT:    sll t0, a0, t1
+; RV32IB-NEXT:    slti a1, t1, 0
+; RV32IB-NEXT:    cmov t0, a1, t0, a7
+; RV32IB-NEXT:    not a7, a4
+; RV32IB-NEXT:    srli t4, a3, 1
+; RV32IB-NEXT:    srl t2, t4, a7
 ; RV32IB-NEXT:    addi a1, zero, 63
-; RV32IB-NEXT:    andn a5, a1, a4
-; RV32IB-NEXT:    addi a1, a5, -32
-; RV32IB-NEXT:    srli t3, a3, 1
-; RV32IB-NEXT:    bltz a1, .LBB13_7
-; RV32IB-NEXT:  # %bb.4:
-; RV32IB-NEXT:    mv t0, zero
-; RV32IB-NEXT:    bgez a1, .LBB13_8
-; RV32IB-NEXT:  .LBB13_5:
-; RV32IB-NEXT:    fsri a1, a2, a3, 1
-; RV32IB-NEXT:    srl a1, a1, t1
-; RV32IB-NEXT:    sub a2, a6, a5
-; RV32IB-NEXT:    slli a3, t3, 1
-; RV32IB-NEXT:    sll a2, a3, a2
-; RV32IB-NEXT:    or a2, a1, a2
-; RV32IB-NEXT:    or a1, a7, t0
-; RV32IB-NEXT:    bgez t2, .LBB13_9
-; RV32IB-NEXT:  .LBB13_6:
+; RV32IB-NEXT:    andn t3, a1, a4
+; RV32IB-NEXT:    addi a5, t3, -32
+; RV32IB-NEXT:    srai a1, a5, 31
+; RV32IB-NEXT:    and a1, a1, t2
+; RV32IB-NEXT:    or a1, t0, a1
+; RV32IB-NEXT:    fsri a2, a2, a3, 1
+; RV32IB-NEXT:    srl a7, a2, a7
+; RV32IB-NEXT:    sub a3, a6, t3
+; RV32IB-NEXT:    slli a2, t4, 1
+; RV32IB-NEXT:    sll a2, a2, a3
+; RV32IB-NEXT:    or a2, a7, a2
+; RV32IB-NEXT:    srl a3, t4, a5
+; RV32IB-NEXT:    slti a5, a5, 0
+; RV32IB-NEXT:    cmov a2, a5, a3, a2
 ; RV32IB-NEXT:    sll a0, a0, a4
+; RV32IB-NEXT:    srai a3, t1, 31
+; RV32IB-NEXT:    and a0, a3, a0
 ; RV32IB-NEXT:    or a0, a0, a2
 ; RV32IB-NEXT:    ret
-; RV32IB-NEXT:  .LBB13_7:
-; RV32IB-NEXT:    srl t0, t3, t1
-; RV32IB-NEXT:    bltz a1, .LBB13_5
-; RV32IB-NEXT:  .LBB13_8:
-; RV32IB-NEXT:    srl a2, t3, a1
-; RV32IB-NEXT:    or a1, a7, t0
-; RV32IB-NEXT:    bltz t2, .LBB13_6
-; RV32IB-NEXT:  .LBB13_9:
-; RV32IB-NEXT:    or a0, zero, a2
-; RV32IB-NEXT:    ret
 ;
 ; RV32IBT-LABEL: fshl_i64:
 ; RV32IBT:       # %bb.0:
+; RV32IBT-NEXT:    sll a7, a1, a4
 ; RV32IBT-NEXT:    andi a5, a4, 63
-; RV32IBT-NEXT:    addi t1, a5, -32
 ; RV32IBT-NEXT:    addi a6, zero, 31
-; RV32IBT-NEXT:    bltz t1, .LBB13_2
-; RV32IBT-NEXT:  # %bb.1:
-; RV32IBT-NEXT:    sll a7, a0, t1
-; RV32IBT-NEXT:    j .LBB13_3
-; RV32IBT-NEXT:  .LBB13_2:
-; RV32IBT-NEXT:    sll a7, a1, a4
-; RV32IBT-NEXT:    sub a5, a6, a5
+; RV32IBT-NEXT:    sub t0, a6, a5
 ; RV32IBT-NEXT:    srli a1, a0, 1
-; RV32IBT-NEXT:    srl a1, a1, a5
+; RV32IBT-NEXT:    srl a1, a1, t0
 ; RV32IBT-NEXT:    or a7, a7, a1
-; RV32IBT-NEXT:  .LBB13_3:
-; RV32IBT-NEXT:    not a1, a4
-; RV32IBT-NEXT:    andi t3, a1, 63
-; RV32IBT-NEXT:    addi a5, t3, -32
-; RV32IBT-NEXT:    srli t2, a3, 1
-; RV32IBT-NEXT:    bltz a5, .LBB13_7
-; RV32IBT-NEXT:  # %bb.4:
-; RV32IBT-NEXT:    mv t0, zero
-; RV32IBT-NEXT:    bgez a5, .LBB13_8
-; RV32IBT-NEXT:  .LBB13_5:
+; RV32IBT-NEXT:    addi t1, a5, -32
+; RV32IBT-NEXT:    sll t0, a0, t1
+; RV32IBT-NEXT:    slti a1, t1, 0
+; RV32IBT-NEXT:    cmov t0, a1, t0, a7
+; RV32IBT-NEXT:    not a5, a4
+; RV32IBT-NEXT:    srli a7, a3, 1
+; RV32IBT-NEXT:    srl t4, a7, a5
+; RV32IBT-NEXT:    andi t2, a5, 63
+; RV32IBT-NEXT:    addi t3, t2, -32
+; RV32IBT-NEXT:    srai a1, t3, 31
+; RV32IBT-NEXT:    and a1, a1, t4
+; RV32IBT-NEXT:    or a1, t0, a1
 ; RV32IBT-NEXT:    fsri a2, a2, a3, 1
-; RV32IBT-NEXT:    srl a1, a2, a1
-; RV32IBT-NEXT:    sub a2, a6, t3
-; RV32IBT-NEXT:    slli a3, t2, 1
-; RV32IBT-NEXT:    sll a2, a3, a2
-; RV32IBT-NEXT:    or a2, a1, a2
-; RV32IBT-NEXT:    or a1, a7, t0
-; RV32IBT-NEXT:    bgez t1, .LBB13_9
-; RV32IBT-NEXT:  .LBB13_6:
+; RV32IBT-NEXT:    srl a2, a2, a5
+; RV32IBT-NEXT:    sub a3, a6, t2
+; RV32IBT-NEXT:    slli a5, a7, 1
+; RV32IBT-NEXT:    sll a3, a5, a3
+; RV32IBT-NEXT:    or a2, a2, a3
+; RV32IBT-NEXT:    srl a3, a7, t3
+; RV32IBT-NEXT:    slti a5, t3, 0
+; RV32IBT-NEXT:    cmov a2, a5, a3, a2
 ; RV32IBT-NEXT:    sll a0, a0, a4
+; RV32IBT-NEXT:    srai a3, t1, 31
+; RV32IBT-NEXT:    and a0, a3, a0
 ; RV32IBT-NEXT:    or a0, a0, a2
 ; RV32IBT-NEXT:    ret
-; RV32IBT-NEXT:  .LBB13_7:
-; RV32IBT-NEXT:    srl t0, t2, a1
-; RV32IBT-NEXT:    bltz a5, .LBB13_5
-; RV32IBT-NEXT:  .LBB13_8:
-; RV32IBT-NEXT:    srl a2, t2, a5
-; RV32IBT-NEXT:    or a1, a7, t0
-; RV32IBT-NEXT:    bltz t1, .LBB13_6
-; RV32IBT-NEXT:  .LBB13_9:
-; RV32IBT-NEXT:    or a0, zero, a2
-; RV32IBT-NEXT:    ret
   %1 = tail call i64 @llvm.fshl.i64(i64 %a, i64 %b, i64 %c)
   ret i64 %1
 }
 
 declare i32 @llvm.fshr.i32(i32, i32, i32)
 
 define i32 @fshr_i32(i32 %a, i32 %b, i32 %c) nounwind {
 ; RV32I-LABEL: fshr_i32:

 ; RV32I-NEXT:    or a0, t0, a7
 ; RV32I-NEXT:    bltz t1, .LBB15_6
 ; RV32I-NEXT:  .LBB15_9:
 ; RV32I-NEXT:    or a1, a1, zero
 ; RV32I-NEXT:    ret
 ;
 ; RV32IB-LABEL: fshr_i64:
 ; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    srl a7, a2, a4
 ; RV32IB-NEXT:    andi a5, a4, 63
-; RV32IB-NEXT:    addi t2, a5, -32
 ; RV32IB-NEXT:    addi a6, zero, 31
-; RV32IB-NEXT:    bltz t2, .LBB15_2
-; RV32IB-NEXT:  # %bb.1:
-; RV32IB-NEXT:    srl a7, a3, t2
-; RV32IB-NEXT:    j .LBB15_3
-; RV32IB-NEXT:  .LBB15_2:
-; RV32IB-NEXT:    srl a7, a2, a4
-; RV32IB-NEXT:    sub a5, a6, a5
+; RV32IB-NEXT:    sub t0, a6, a5
 ; RV32IB-NEXT:    slli a2, a3, 1
-; RV32IB-NEXT:    sll a2, a2, a5
+; RV32IB-NEXT:    sll a2, a2, t0
 ; RV32IB-NEXT:    or a7, a7, a2
-; RV32IB-NEXT:  .LBB15_3:
-; RV32IB-NEXT:    not t1, a4
-; RV32IB-NEXT:    addi a2, zero, 63
-; RV32IB-NEXT:    andn a2, a2, a4
-; RV32IB-NEXT:    addi a5, a2, -32
-; RV32IB-NEXT:    slli t3, a0, 1
-; RV32IB-NEXT:    bltz a5, .LBB15_7
-; RV32IB-NEXT:  # %bb.4:
-; RV32IB-NEXT:    mv t0, zero
-; RV32IB-NEXT:    bgez a5, .LBB15_8
-; RV32IB-NEXT:  .LBB15_5:
+; RV32IB-NEXT:    addi t2, a5, -32
+; RV32IB-NEXT:    srl t0, a3, t2
+; RV32IB-NEXT:    slti a2, t2, 0
+; RV32IB-NEXT:    cmov a7, a2, t0, a7
+; RV32IB-NEXT:    not t3, a4
+; RV32IB-NEXT:    slli t0, a0, 1
+; RV32IB-NEXT:    sll t1, t0, t3
+; RV32IB-NEXT:    addi a5, zero, 63
+; RV32IB-NEXT:    andn t4, a5, a4
+; RV32IB-NEXT:    addi a2, t4, -32
+; RV32IB-NEXT:    srai a5, a2, 31
+; RV32IB-NEXT:    and a5, a5, t1
+; RV32IB-NEXT:    or a7, a5, a7
 ; RV32IB-NEXT:    fsri a1, a0, a1, 31
-; RV32IB-NEXT:    sll a1, a1, t1
-; RV32IB-NEXT:    sub a2, a6, a2
+; RV32IB-NEXT:    sll a1, a1, t3
+; RV32IB-NEXT:    sub a5, a6, t4
 ; RV32IB-NEXT:    sbclri a0, a0, 31
-; RV32IB-NEXT:    srl a0, a0, a2
-; RV32IB-NEXT:    or a1, a1, a0
-; RV32IB-NEXT:    or a0, t0, a7
-; RV32IB-NEXT:    bgez t2, .LBB15_9
-; RV32IB-NEXT:  .LBB15_6:
-; RV32IB-NEXT:    srl a2, a3, a4
-; RV32IB-NEXT:    or a1, a1, a2
-; RV32IB-NEXT:    ret
-; RV32IB-NEXT:  .LBB15_7:
-; RV32IB-NEXT:    sll t0, t3, t1
-; RV32IB-NEXT:    bltz a5, .LBB15_5
-; RV32IB-NEXT:  .LBB15_8:
-; RV32IB-NEXT:    sll a1, t3, a5
-; RV32IB-NEXT:    or a0, t0, a7
-; RV32IB-NEXT:    bltz t2, .LBB15_6
-; RV32IB-NEXT:  .LBB15_9:
-; RV32IB-NEXT:    or a1, a1, zero
+; RV32IB-NEXT:    srl a0, a0, a5
+; RV32IB-NEXT:    or a0, a1, a0
+; RV32IB-NEXT:    sll a1, t0, a2
+; RV32IB-NEXT:    slti a2, a2, 0
+; RV32IB-NEXT:    cmov a0, a2, a1, a0
+; RV32IB-NEXT:    srl a1, a3, a4
+; RV32IB-NEXT:    srai a2, t2, 31
+; RV32IB-NEXT:    and a1, a2, a1
+; RV32IB-NEXT:    or a1, a0, a1
+; RV32IB-NEXT:    mv a0, a7
 ; RV32IB-NEXT:    ret
 ;
 ; RV32IBT-LABEL: fshr_i64:
 ; RV32IBT:       # %bb.0:
+; RV32IBT-NEXT:    srl a7, a2, a4
 ; RV32IBT-NEXT:    andi a5, a4, 63
-; RV32IBT-NEXT:    addi t1, a5, -32
 ; RV32IBT-NEXT:    addi a6, zero, 31
-; RV32IBT-NEXT:    bltz t1, .LBB15_2
-; RV32IBT-NEXT:  # %bb.1:
-; RV32IBT-NEXT:    srl a7, a3, t1
-; RV32IBT-NEXT:    j .LBB15_3
-; RV32IBT-NEXT:  .LBB15_2:
-; RV32IBT-NEXT:    srl a7, a2, a4
-; RV32IBT-NEXT:    sub a5, a6, a5
+; RV32IBT-NEXT:    sub t0, a6, a5
 ; RV32IBT-NEXT:    slli a2, a3, 1
-; RV32IBT-NEXT:    sll a2, a2, a5
+; RV32IBT-NEXT:    sll a2, a2, t0
 ; RV32IBT-NEXT:    or a7, a7, a2
-; RV32IBT-NEXT:  .LBB15_3:
-; RV32IBT-NEXT:    not a2, a4
-; RV32IBT-NEXT:    andi t2, a2, 63
-; RV32IBT-NEXT:    addi a5, t2, -32
-; RV32IBT-NEXT:    slli t3, a0, 1
-; RV32IBT-NEXT:    bltz a5, .LBB15_7
-; RV32IBT-NEXT:  # %bb.4:
-; RV32IBT-NEXT:    mv t0, zero
-; RV32IBT-NEXT:    bgez a5, .LBB15_8
-; RV32IBT-NEXT:  .LBB15_5:
-; RV32IBT-NEXT:    lui a5, 524288
-; RV32IBT-NEXT:    addi a5, a5, -1
-; RV32IBT-NEXT:    and t3, a0, a5
-; RV32IBT-NEXT:    sub a5, a6, t2
-; RV32IBT-NEXT:    srl a5, t3, a5
+; RV32IBT-NEXT:    addi t2, a5, -32
+; RV32IBT-NEXT:    srl t0, a3, t2
+; RV32IBT-NEXT:    slti a2, t2, 0
+; RV32IBT-NEXT:    cmov a7, a2, t0, a7
+; RV32IBT-NEXT:    not t4, a4
+; RV32IBT-NEXT:    slli t0, a0, 1
+; RV32IBT-NEXT:    sll t1, t0, t4
+; RV32IBT-NEXT:    andi t3, t4, 63
+; RV32IBT-NEXT:    addi a5, t3, -32
+; RV32IBT-NEXT:    srai a2, a5, 31
+; RV32IBT-NEXT:    and a2, a2, t1
+; RV32IBT-NEXT:    or a7, a2, a7
+; RV32IBT-NEXT:    lui a2, 524288
+; RV32IBT-NEXT:    addi a2, a2, -1
+; RV32IBT-NEXT:    and t1, a0, a2
+; RV32IBT-NEXT:    sub a2, a6, t3
+; RV32IBT-NEXT:    srl a2, t1, a2
 ; RV32IBT-NEXT:    fsri a0, a0, a1, 31
-; RV32IBT-NEXT:    sll a0, a0, a2
-; RV32IBT-NEXT:    or a1, a0, a5
-; RV32IBT-NEXT:    or a0, t0, a7
-; RV32IBT-NEXT:    bgez t1, .LBB15_9
-; RV32IBT-NEXT:  .LBB15_6:
-; RV32IBT-NEXT:    srl a2, a3, a4
-; RV32IBT-NEXT:    or a1, a1, a2
-; RV32IBT-NEXT:    ret
-; RV32IBT-NEXT:  .LBB15_7:
-; RV32IBT-NEXT:    sll t0, t3, a2
-; RV32IBT-NEXT:    bltz a5, .LBB15_5
-; RV32IBT-NEXT:  .LBB15_8:
-; RV32IBT-NEXT:    sll a1, t3, a5
-; RV32IBT-NEXT:    or a0, t0, a7
-; RV32IBT-NEXT:    bltz t1, .LBB15_6
-; RV32IBT-NEXT:  .LBB15_9:
-; RV32IBT-NEXT:    or a1, a1, zero
+; RV32IBT-NEXT:    sll a0, a0, t4
+; RV32IBT-NEXT:    or a0, a0, a2
+; RV32IBT-NEXT:    sll a1, t0, a5
+; RV32IBT-NEXT:    slti a2, a5, 0
+; RV32IBT-NEXT:    cmov a0, a2, a1, a0
+; RV32IBT-NEXT:    srl a1, a3, a4
+; RV32IBT-NEXT:    srai a2, t2, 31
+; RV32IBT-NEXT:    and a1, a2, a1
+; RV32IBT-NEXT:    or a1, a0, a1
+; RV32IBT-NEXT:    mv a0, a7
 ; RV32IBT-NEXT:    ret
   %1 = tail call i64 @llvm.fshr.i64(i64 %a, i64 %b, i64 %c)
   ret i64 %1
 }
 
 define i32 @fshri_i32(i32 %a, i32 %b) nounwind {
 ; RV32I-LABEL: fshri_i32:
 ; RV32I:       # %bb.0:

llvm/test/CodeGen/RISCV/rv64Zbt.ll

 ; RV64I-NEXT:  # %bb.1:
 ; RV64I-NEXT:    mv a3, a0
 ; RV64I-NEXT:  .LBB3_2:
 ; RV64I-NEXT:    mv a0, a3
 ; RV64I-NEXT:    ret
 ;
 ; RV64IB-LABEL: cmov_sle_i32:
 ; RV64IB:       # %bb.0:
-; RV64IB-NEXT:    bge a2, a1, .LBB3_2
-; RV64IB-NEXT:  # %bb.1:
-; RV64IB-NEXT:    mv a3, a0
-; RV64IB-NEXT:  .LBB3_2:
-; RV64IB-NEXT:    mv a0, a3
+; RV64IB-NEXT:    slt a1, a2, a1
+; RV64IB-NEXT:    cmov a0, a1, a3, a0
 ; RV64IB-NEXT:    ret
 ;
 ; RV64IBT-LABEL: cmov_sle_i32:
 ; RV64IBT:       # %bb.0:
-; RV64IBT-NEXT:    bge a2, a1, .LBB3_2
-; RV64IBT-NEXT:  # %bb.1:
-; RV64IBT-NEXT:    mv a3, a0
-; RV64IBT-NEXT:  .LBB3_2:
-; RV64IBT-NEXT:    mv a0, a3
+; RV64IBT-NEXT:    slt a1, a2, a1
+; RV64IBT-NEXT:    cmov a0, a1, a3, a0
 ; RV64IBT-NEXT:    ret
   %tobool = icmp sle i32 %b, %c
   %cond = select i1 %tobool, i32 %d, i32 %a
   ret i32 %cond
 }
 
 define signext i32 @cmov_sge_i32(i32 signext %a, i32 signext %b, i32 signext %c, i32 signext %d) nounwind {
 ; RV64I-LABEL: cmov_sge_i32:
 ; RV64I:       # %bb.0:
 ; RV64I-NEXT:    bge a1, a2, .LBB4_2
 ; RV64I-NEXT:  # %bb.1:
 ; RV64I-NEXT:    mv a3, a0
 ; RV64I-NEXT:  .LBB4_2:
 ; RV64I-NEXT:    mv a0, a3
 ; RV64I-NEXT:    ret
 ;
 ; RV64IB-LABEL: cmov_sge_i32:
 ; RV64IB:       # %bb.0:
-; RV64IB-NEXT:    bge a1, a2, .LBB4_2
-; RV64IB-NEXT:  # %bb.1:
-; RV64IB-NEXT:    mv a3, a0
-; RV64IB-NEXT:  .LBB4_2:
-; RV64IB-NEXT:    mv a0, a3
+; RV64IB-NEXT:    slt a1, a1, a2
+; RV64IB-NEXT:    cmov a0, a1, a3, a0
 ; RV64IB-NEXT:    ret
 ;
 ; RV64IBT-LABEL: cmov_sge_i32:
 ; RV64IBT:       # %bb.0:
-; RV64IBT-NEXT:    bge a1, a2, .LBB4_2
-; RV64IBT-NEXT:  # %bb.1:
-; RV64IBT-NEXT:    mv a3, a0
-; RV64IBT-NEXT:  .LBB4_2:
-; RV64IBT-NEXT:    mv a0, a3
+; RV64IBT-NEXT:    slt a1, a1, a2
+; RV64IBT-NEXT:    cmov a0, a1, a3, a0
 ; RV64IBT-NEXT:    ret
   %tobool = icmp sge i32 %b, %c
   %cond = select i1 %tobool, i32 %d, i32 %a
   ret i32 %cond
 }
 
 define signext i32 @cmov_ule_i32(i32 signext %a, i32 signext %b, i32 signext %c, i32 signext %d) nounwind {
 ; RV64I-LABEL: cmov_ule_i32:
 ; RV64I:       # %bb.0:
 ; RV64I-NEXT:    bgeu a2, a1, .LBB5_2
 ; RV64I-NEXT:  # %bb.1:
 ; RV64I-NEXT:    mv a3, a0
 ; RV64I-NEXT:  .LBB5_2:
 ; RV64I-NEXT:    mv a0, a3
 ; RV64I-NEXT:    ret
 ;
 ; RV64IB-LABEL: cmov_ule_i32:
 ; RV64IB:       # %bb.0:
-; RV64IB-NEXT:    bgeu a2, a1, .LBB5_2
-; RV64IB-NEXT:  # %bb.1:
-; RV64IB-NEXT:    mv a3, a0
-; RV64IB-NEXT:  .LBB5_2:
-; RV64IB-NEXT:    mv a0, a3
+; RV64IB-NEXT:    sltu a1, a2, a1
+; RV64IB-NEXT:    cmov a0, a1, a3, a0
 ; RV64IB-NEXT:    ret
 ;
 ; RV64IBT-LABEL: cmov_ule_i32:
 ; RV64IBT:       # %bb.0:
-; RV64IBT-NEXT:    bgeu a2, a1, .LBB5_2
-; RV64IBT-NEXT:  # %bb.1:
-; RV64IBT-NEXT:    mv a3, a0
-; RV64IBT-NEXT:  .LBB5_2:
-; RV64IBT-NEXT:    mv a0, a3
+; RV64IBT-NEXT:    sltu a1, a2, a1
+; RV64IBT-NEXT:    cmov a0, a1, a3, a0
 ; RV64IBT-NEXT:    ret
   %tobool = icmp ule i32 %b, %c
   %cond = select i1 %tobool, i32 %d, i32 %a
   ret i32 %cond
 }
 
 define signext i32 @cmov_uge_i32(i32 signext %a, i32 signext %b, i32 signext %c, i32 signext %d) nounwind {
 ; RV64I-LABEL: cmov_uge_i32:
 ; RV64I:       # %bb.0:
 ; RV64I-NEXT:    bgeu a1, a2, .LBB6_2
 ; RV64I-NEXT:  # %bb.1:
 ; RV64I-NEXT:    mv a3, a0
 ; RV64I-NEXT:  .LBB6_2:
 ; RV64I-NEXT:    mv a0, a3
 ; RV64I-NEXT:    ret
 ;
 ; RV64IB-LABEL: cmov_uge_i32:
 ; RV64IB:       # %bb.0:
-; RV64IB-NEXT:    bgeu a1, a2, .LBB6_2
-; RV64IB-NEXT:  # %bb.1:
-; RV64IB-NEXT:    mv a3, a0
-; RV64IB-NEXT:  .LBB6_2:
-; RV64IB-NEXT:    mv a0, a3
+; RV64IB-NEXT:    sltu a1, a1, a2
+; RV64IB-NEXT:    cmov a0, a1, a3, a0
 ; RV64IB-NEXT:    ret
 ;
 ; RV64IBT-LABEL: cmov_uge_i32:
 ; RV64IBT:       # %bb.0:
-; RV64IBT-NEXT:    bgeu a1, a2, .LBB6_2
-; RV64IBT-NEXT:  # %bb.1:
-; RV64IBT-NEXT:    mv a3, a0
-; RV64IBT-NEXT:  .LBB6_2:
-; RV64IBT-NEXT:    mv a0, a3
+; RV64IBT-NEXT:    sltu a1, a1, a2
+; RV64IBT-NEXT:    cmov a0, a1, a3, a0
 ; RV64IBT-NEXT:    ret
   %tobool = icmp uge i32 %b, %c
   %cond = select i1 %tobool, i32 %d, i32 %a
   ret i32 %cond
 }
 
 define i64 @cmov_i64(i64 %a, i64 %b, i64 %c) nounwind {
 ; RV64I-LABEL: cmov_i64:

 ; RV64I-NEXT:  # %bb.1:
 ; RV64I-NEXT:    mv a3, a0
 ; RV64I-NEXT:  .LBB8_2:
 ; RV64I-NEXT:    mv a0, a3
 ; RV64I-NEXT:    ret
 ;
 ; RV64IB-LABEL: cmov_sle_i64:
 ; RV64IB:       # %bb.0:
-; RV64IB-NEXT:    bge a2, a1, .LBB8_2
-; RV64IB-NEXT:  # %bb.1:
-; RV64IB-NEXT:    mv a3, a0
-; RV64IB-NEXT:  .LBB8_2:
-; RV64IB-NEXT:    mv a0, a3
+; RV64IB-NEXT:    slt a1, a2, a1
+; RV64IB-NEXT:    cmov a0, a1, a3, a0
 ; RV64IB-NEXT:    ret
 ;
 ; RV64IBT-LABEL: cmov_sle_i64:
 ; RV64IBT:       # %bb.0:
-; RV64IBT-NEXT:    bge a2, a1, .LBB8_2
-; RV64IBT-NEXT:  # %bb.1:
-; RV64IBT-NEXT:    mv a3, a0
-; RV64IBT-NEXT:  .LBB8_2:
-; RV64IBT-NEXT:    mv a0, a3
+; RV64IBT-NEXT:    slt a1, a2, a1
+; RV64IBT-NEXT:    cmov a0, a1, a3, a0
 ; RV64IBT-NEXT:    ret
   %tobool = icmp sle i64 %b, %c
   %cond = select i1 %tobool, i64 %d, i64 %a
   ret i64 %cond
 }
 
 define i64 @cmov_sge_i64(i64 %a, i64 %b, i64 %c, i64 %d) nounwind {
 ; RV64I-LABEL: cmov_sge_i64:
 ; RV64I:       # %bb.0:
 ; RV64I-NEXT:    bge a1, a2, .LBB9_2
 ; RV64I-NEXT:  # %bb.1:
 ; RV64I-NEXT:    mv a3, a0
 ; RV64I-NEXT:  .LBB9_2:
 ; RV64I-NEXT:    mv a0, a3
 ; RV64I-NEXT:    ret
 ;
 ; RV64IB-LABEL: cmov_sge_i64:
 ; RV64IB:       # %bb.0:
-; RV64IB-NEXT:    bge a1, a2, .LBB9_2
-; RV64IB-NEXT:  # %bb.1:
-; RV64IB-NEXT:    mv a3, a0
-; RV64IB-NEXT:  .LBB9_2:
-; RV64IB-NEXT:    mv a0, a3
+; RV64IB-NEXT:    slt a1, a1, a2
+; RV64IB-NEXT:    cmov a0, a1, a3, a0
 ; RV64IB-NEXT:    ret
 ;
 ; RV64IBT-LABEL: cmov_sge_i64:
 ; RV64IBT:       # %bb.0:
-; RV64IBT-NEXT:    bge a1, a2, .LBB9_2
-; RV64IBT-NEXT:  # %bb.1:
-; RV64IBT-NEXT:    mv a3, a0
-; RV64IBT-NEXT:  .LBB9_2:
-; RV64IBT-NEXT:    mv a0, a3
+; RV64IBT-NEXT:    slt a1, a1, a2
+; RV64IBT-NEXT:    cmov a0, a1, a3, a0
 ; RV64IBT-NEXT:    ret
   %tobool = icmp sge i64 %b, %c
   %cond = select i1 %tobool, i64 %d, i64 %a
   ret i64 %cond
 }
 
 define i64 @cmov_ule_i64(i64 %a, i64 %b, i64 %c, i64 %d) nounwind {
 ; RV64I-LABEL: cmov_ule_i64:
 ; RV64I:       # %bb.0:
 ; RV64I-NEXT:    bgeu a2, a1, .LBB10_2
 ; RV64I-NEXT:  # %bb.1:
 ; RV64I-NEXT:    mv a3, a0
 ; RV64I-NEXT:  .LBB10_2:
 ; RV64I-NEXT:    mv a0, a3
 ; RV64I-NEXT:    ret
 ;
 ; RV64IB-LABEL: cmov_ule_i64:
 ; RV64IB:       # %bb.0:
-; RV64IB-NEXT:    bgeu a2, a1, .LBB10_2
-; RV64IB-NEXT:  # %bb.1:
-; RV64IB-NEXT:    mv a3, a0
-; RV64IB-NEXT:  .LBB10_2:
-; RV64IB-NEXT:    mv a0, a3
+; RV64IB-NEXT:    sltu a1, a2, a1
+; RV64IB-NEXT:    cmov a0, a1, a3, a0
 ; RV64IB-NEXT:    ret
 ;
 ; RV64IBT-LABEL: cmov_ule_i64:
 ; RV64IBT:       # %bb.0:
-; RV64IBT-NEXT:    bgeu a2, a1, .LBB10_2
-; RV64IBT-NEXT:  # %bb.1:
-; RV64IBT-NEXT:    mv a3, a0
-; RV64IBT-NEXT:  .LBB10_2:
-; RV64IBT-NEXT:    mv a0, a3
+; RV64IBT-NEXT:    sltu a1, a2, a1
+; RV64IBT-NEXT:    cmov a0, a1, a3, a0
 ; RV64IBT-NEXT:    ret
   %tobool = icmp ule i64 %b, %c
   %cond = select i1 %tobool, i64 %d, i64 %a
   ret i64 %cond
 }
 
 define i64 @cmov_uge_i64(i64 %a, i64 %b, i64 %c, i64 %d) nounwind {
 ; RV64I-LABEL: cmov_uge_i64:
 ; RV64I:       # %bb.0:
 ; RV64I-NEXT:    bgeu a1, a2, .LBB11_2
 ; RV64I-NEXT:  # %bb.1:
 ; RV64I-NEXT:    mv a3, a0
 ; RV64I-NEXT:  .LBB11_2:
 ; RV64I-NEXT:    mv a0, a3
 ; RV64I-NEXT:    ret
 ;
 ; RV64IB-LABEL: cmov_uge_i64:
 ; RV64IB:       # %bb.0:
-; RV64IB-NEXT:    bgeu a1, a2, .LBB11_2
-; RV64IB-NEXT:  # %bb.1:
-; RV64IB-NEXT:    mv a3, a0
-; RV64IB-NEXT:  .LBB11_2:
-; RV64IB-NEXT:    mv a0, a3
+; RV64IB-NEXT:    sltu a1, a1, a2
+; RV64IB-NEXT:    cmov a0, a1, a3, a0
 ; RV64IB-NEXT:    ret
 ;
 ; RV64IBT-LABEL: cmov_uge_i64:
 ; RV64IBT:       # %bb.0:
-; RV64IBT-NEXT:    bgeu a1, a2, .LBB11_2
-; RV64IBT-NEXT:  # %bb.1:
-; RV64IBT-NEXT:    mv a3, a0
-; RV64IBT-NEXT:  .LBB11_2:
-; RV64IBT-NEXT:    mv a0, a3
+; RV64IBT-NEXT:    sltu a1, a1, a2
+; RV64IBT-NEXT:    cmov a0, a1, a3, a0
 ; RV64IBT-NEXT:    ret
   %tobool = icmp uge i64 %b, %c
   %cond = select i1 %tobool, i64 %d, i64 %a
   ret i64 %cond
 }
 
 declare i32 @llvm.fshl.i32(i32, i32, i32)