[AArch64][GlobalISel] Move imm adjustment for G_ICMP to post-legalizer lowering

[paquette]

Summary

Move the code which adjusts the immediate/predicate on a G_ICMP to AArch64PostLegalizerLowering.

This

• Reduces the number of places we need to test for optimized compares in the selector. We know that the compare should have been simplified by the time it hits the selector, so we can avoid testing this in selects, brconds, etc.

• Allows us to potentially fold more compares (previously, this optimization was only done after calling tryFoldCompare, this may allow us to hit some more TST cases)

• Simplifies the selection code in emitIntegerCompare significantly; we can just use an emitSUBS function.

• Allows us to avoid checking that the predicate has been updated after emitIntegerCompare.

Also add a utility header file for things that may be useful in the selector and various combiners. No need for an implementation file at this point, since it's just one constexpr function for now. I've run into a couple cases where having one of these would be handy, so might as well add it here. There are a couple functions in the selector that can probably be factored out into here.

[aemerson]

llvm/lib/Target/AArch64/GISel/AArch64PostLegalizerLowering.cpp

519–520

debug left

llvm/test/CodeGen/AArch64/GlobalISel/select-arith-immed-compare.mir

31

Can we have some end-to-end tests that show that the MIR that we previously had being selected to CSINCWr still selects to the same output if we now go through lowering->select? Don't have to cover all the cases but to show the mechanism still works.

Maybe we'll need to use .ll tests for this, not sure.

[paquette]

llvm/test/CodeGen/AArch64/GlobalISel/select-arith-immed-compare.mir

31

To avoid a .ll test we could do something like this:

llc -mtriple=aarch64 -start-before=aarch64-postlegalizer-lowering -stop-after=instruction-select -verify-machineinstrs

A slightly hackier, but more pointed version would be:

llc -mtriple=aarch64 -run-pass=aarch64-postlegalizer-lowering -run-pass=instruction-select -verify-machineinstrs

(IIRC this works)

From the perspective that we'd only run aarch64-postlegalizer-lowering and instruction-select, I guess this is better. We'd have to use regbankselected MIR though, which is a little wonky. Not sure if that's something that we'd want to support in the future.

Preferences? (I could also just take the .ll test route, or add a GISel checkline to an existing .ll test somewhere probably)

[aemerson]

llvm/test/CodeGen/AArch64/GlobalISel/select-arith-immed-compare.mir

31

I didn't realize that -start-before and -stop-after actually worked without crashing. That's the ideal route, otherwise adding check lines to an existing .ll is fine too.

[paquette]

• Remove leftover debug code
• Make the test also show that we select the right thing
• Improve a couple comments
• Add G_SELECT testcase showing that still works
• Add ands/tst testcase showing we can now fold constants into the tst/ands case

[aemerson]

Nice.

llvm/lib/Target/AArch64/AArch64Combine.td

 def vashr_vlshr_imm_matchdata : GIDefMatchData<"int64_t">;
 def vashr_vlshr_imm : GICombineRule<
   (defs root:$root, vashr_vlshr_imm_matchdata:$matchinfo),
   (match (wip_match_opcode G_ASHR, G_LSHR):$root,
           [{ return matchVAshrLshrImm(*${root}, MRI, ${matchinfo}); }]),
   (apply [{ applyVAshrLshrImm(*${root}, MRI, ${matchinfo}); }])
 >;
 
+def adjust_icmp_imm_matchdata :
+  GIDefMatchData<"std::pair<uint64_t, CmpInst::Predicate>">;
+def adjust_icmp_imm : GICombineRule <
+  (defs root:$root, adjust_icmp_imm_matchdata:$matchinfo),
+  (match (wip_match_opcode G_ICMP):$root,
+          [{ return matchAdjustICmpImmAndPred(*${root}, MRI, ${matchinfo}); }]),
+  (apply [{ applyAdjustICmpImmAndPred(*${root}, ${matchinfo}, B, Observer); }])
+>;
+
+def icmp_lowering : GICombineGroup<[adjust_icmp_imm]>;
+
 // Post-legalization combines which should happen at all optimization levels.
 // (E.g. ones that facilitate matching for the selector) For example, matching
 // pseudos.
 def AArch64PostLegalizerLoweringHelper
     : GICombinerHelper<"AArch64GenPostLegalizerLoweringHelper",
-                       [shuffle_vector_pseudos, vashr_vlshr_imm]> {
+                       [shuffle_vector_pseudos, vashr_vlshr_imm,
+                        icmp_lowering]> {
   let DisableRuleOption = "aarch64postlegalizerlowering-disable-rule";
 }
 
 // Post-legalization combines which are primarily optimizations.
 def AArch64PostLegalizerCombinerHelper
     : GICombinerHelper<"AArch64GenPostLegalizerCombinerHelper",
                        [copy_prop, erase_undef_store, combines_for_extload,
                         sext_trunc_sextload,
                         hoist_logic_op_with_same_opcode_hands,
                         and_trivial_mask, xor_of_and_with_same_reg]> {
   let DisableRuleOption = "aarch64postlegalizercombiner-disable-rule";
 }

llvm/lib/Target/AArch64/GISel/AArch64GlobalISelUtils.h

+//===- AArch64GlobalISelUtils ------------------------------------*- C++ -*-==//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+/// \file APIs for AArch64-specific helper functions used in the GlobalISel
+/// pipeline.
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIB_TARGET_AARCH64_GISEL_AARCH64GLOBALISELUTILS_H
+#define LLVM_LIB_TARGET_AARCH64_GISEL_AARCH64GLOBALISELUTILS_H
+
+#include <cstdint>
+
+namespace llvm {
+namespace AArch64GISelUtils {
+
+/// \returns true if \p C is a legal immediate operand for an arithmetic
+/// instruction.
+constexpr bool isLegalArithImmed(const uint64_t C) {
+  return (C >> 12 == 0) || ((C & 0xFFFULL) == 0 && C >> 24 == 0);
+}
+
+} // namespace AArch64GISelUtils
+} // namespace llvm
+
+#endif

llvm/lib/Target/AArch64/GISel/AArch64InstructionSelector.cpp

MachineInstr *emitLoadFromConstantPool(const Constant *CPVal,
                                          MachineIRBuilder &MIRBuilder) const;
 
   // Emit a vector concat operation.
   MachineInstr *emitVectorConcat(Optional<Register> Dst, Register Op1,
                                  Register Op2,
                                  MachineIRBuilder &MIRBuilder) const;
 
   // Emit an integer compare between LHS and RHS, which checks for Predicate.
-  //
-  // This returns the produced compare instruction, and the predicate which
-  // was ultimately used in the compare. The predicate may differ from what
-  // is passed in \p Predicate due to optimization.
-  std::pair<MachineInstr *, CmpInst::Predicate>
-  emitIntegerCompare(MachineOperand &LHS, MachineOperand &RHS,
+  MachineInstr *emitIntegerCompare(MachineOperand &LHS, MachineOperand &RHS,
                                    MachineOperand &Predicate,
                                    MachineIRBuilder &MIRBuilder) const;
 
   /// Emit a floating point comparison between \p LHS and \p RHS.
   MachineInstr *emitFPCompare(Register LHS, Register RHS,
                               MachineIRBuilder &MIRBuilder) const;
 
   MachineInstr *emitInstr(unsigned Opcode,
                           std::initializer_list<llvm::DstOp> DstOps,
                           std::initializer_list<llvm::SrcOp> SrcOps,
MachineInstr *emitBinOp(
       const std::array<std::array<unsigned, 2>, 3> &AddrModeAndSizeToOpcode,
       Register Dst, MachineOperand &LHS, MachineOperand &RHS,
       MachineIRBuilder &MIRBuilder) const;
   MachineInstr *emitADD(Register DefReg, MachineOperand &LHS,
                         MachineOperand &RHS,
                         MachineIRBuilder &MIRBuilder) const;
   MachineInstr *emitADDS(Register Dst, MachineOperand &LHS, MachineOperand &RHS,
                          MachineIRBuilder &MIRBuilder) const;
+  MachineInstr *emitSUBS(Register Dst, MachineOperand &LHS, MachineOperand &RHS,
+                         MachineIRBuilder &MIRBuilder) const;
   MachineInstr *emitCMN(MachineOperand &LHS, MachineOperand &RHS,
                         MachineIRBuilder &MIRBuilder) const;
   MachineInstr *emitTST(MachineOperand &LHS, MachineOperand &RHS,
                         MachineIRBuilder &MIRBuilder) const;
   MachineInstr *emitExtractVectorElt(Optional<Register> DstReg,
                                      const RegisterBank &DstRB, LLT ScalarTy,
                                      Register VecReg, unsigned LaneIdx,
                                      MachineIRBuilder &MIRBuilder) const;
private:
   void materializeLargeCMVal(MachineInstr &I, const Value *V,
                              unsigned OpFlags) const;
 
   // Optimization methods.
   bool tryOptSelect(MachineInstr &MI) const;
   MachineInstr *tryFoldIntegerCompare(MachineOperand &LHS, MachineOperand &RHS,
                                       MachineOperand &Predicate,
                                       MachineIRBuilder &MIRBuilder) const;
-  MachineInstr *tryOptArithImmedIntegerCompare(MachineOperand &LHS,
-                                               MachineOperand &RHS,
-                                               CmpInst::Predicate &Predicate,
-                                               MachineIRBuilder &MIB) const;
-  MachineInstr *tryOptArithShiftedCompare(MachineOperand &LHS,
-                                          MachineOperand &RHS,
-                                          MachineIRBuilder &MIB) const;
 
   /// Return true if \p MI is a load or store of \p NumBytes bytes.
   bool isLoadStoreOfNumBytes(const MachineInstr &MI, unsigned NumBytes) const;
 
   /// Returns true if \p MI is guaranteed to have the high-half of a 64-bit
   /// register zeroed out. In other words, the result of MI has been explicitly
   /// zero extended.
   bool isDef32(const MachineInstr &MI) const;
bool AArch64InstructionSelector::selectCompareBranch(
   if (!VRegAndVal) {
     std::swap(RHS, LHS);
     VRegAndVal = getConstantVRegValWithLookThrough(RHS, MRI);
     LHSMI = getDefIgnoringCopies(LHS, MRI);
   }
 
   if (!VRegAndVal || VRegAndVal->Value != 0) {
     // If we can't select a CBZ then emit a cmp + Bcc.
-    MachineInstr *Cmp;
-    std::tie(Cmp, Pred) = emitIntegerCompare(
-        CCMI->getOperand(2), CCMI->getOperand(3), CCMI->getOperand(1), MIB);
+    auto Pred =
+        static_cast<CmpInst::Predicate>(CCMI->getOperand(1).getPredicate());
+    emitIntegerCompare(CCMI->getOperand(2), CCMI->getOperand(3),
+                       CCMI->getOperand(1), MIB);
     const AArch64CC::CondCode CC = changeICMPPredToAArch64CC(Pred);
     MIB.buildInstr(AArch64::Bcc, {}, {}).addImm(CC).addMBB(DestMBB);
     I.eraseFromParent();
     return true;
   }
 
   // Try to emit a TB(N)Z for an eq or ne condition.
   if (tryOptAndIntoCompareBranch(LHSMI, VRegAndVal->Value, Pred, DestMBB,
case TargetOpcode::G_ICMP: {
 
     if (Ty != LLT::scalar(32)) {
       LLVM_DEBUG(dbgs() << "G_ICMP result has type: " << Ty
                         << ", expected: " << LLT::scalar(32) << '\n');
       return false;
     }
 
     MachineIRBuilder MIRBuilder(I);
-    MachineInstr *Cmp;
-    CmpInst::Predicate Pred;
-    std::tie(Cmp, Pred) = emitIntegerCompare(I.getOperand(2), I.getOperand(3),
+    auto Pred = static_cast<CmpInst::Predicate>(I.getOperand(1).getPredicate());
+    MachineInstr *Cmp = emitIntegerCompare(I.getOperand(2), I.getOperand(3),
                                              I.getOperand(1), MIRBuilder);
     emitCSetForICMP(I.getOperand(0).getReg(), Pred, MIRBuilder);
     I.eraseFromParent();
     return true;
   }
 
   case TargetOpcode::G_FCMP: {
     MachineIRBuilder MIRBuilder(I);

 
 MachineInstr *AArch64InstructionSelector::emitBinOp(
     const std::array<std::array<unsigned, 2>, 3> &AddrModeAndSizeToOpcode,
     Register Dst, MachineOperand &LHS, MachineOperand &RHS,
     MachineIRBuilder &MIRBuilder) const {
   MachineRegisterInfo &MRI = MIRBuilder.getMF().getRegInfo();
   assert(LHS.isReg() && RHS.isReg() && "Expected register operands?");
   auto Ty = MRI.getType(LHS.getReg());
-  assert(Ty.isScalar() && "Expected a scalar?");
+  assert(!Ty.isVector() && "Expected a scalar or pointer?");
   unsigned Size = Ty.getSizeInBits();
   assert((Size == 32 || Size == 64) && "Expected a 32-bit or 64-bit type only");
   bool Is32Bit = Size == 32;
   if (auto Fns = selectArithImmed(RHS))
     return emitInstr(AddrModeAndSizeToOpcode[0][Is32Bit], {Dst}, {LHS},
                      MIRBuilder, Fns);
   if (auto Fns = selectShiftedRegister(RHS))
     return emitInstr(AddrModeAndSizeToOpcode[1][Is32Bit], {Dst}, {LHS},
AArch64InstructionSelector::emitADDS(Register Dst, MachineOperand &LHS,
   const std::array<std::array<unsigned, 2>, 3> OpcTable{
       {{AArch64::ADDSXri, AArch64::ADDSWri},
        {AArch64::ADDSXrs, AArch64::ADDSWrs},
        {AArch64::ADDSXrr, AArch64::ADDSWrr}}};
   return emitBinOp(OpcTable, Dst, LHS, RHS, MIRBuilder);
 }
 
 MachineInstr *
+AArch64InstructionSelector::emitSUBS(Register Dst, MachineOperand &LHS,
+                                     MachineOperand &RHS,
+                                     MachineIRBuilder &MIRBuilder) const {
+  const std::array<std::array<unsigned, 2>, 3> OpcTable{
+      {{AArch64::SUBSXri, AArch64::SUBSWri},
+       {AArch64::SUBSXrs, AArch64::SUBSWrs},
+       {AArch64::SUBSXrr, AArch64::SUBSWrr}}};
+  return emitBinOp(OpcTable, Dst, LHS, RHS, MIRBuilder);
+}
+
+MachineInstr *
 AArch64InstructionSelector::emitCMN(MachineOperand &LHS, MachineOperand &RHS,
                                     MachineIRBuilder &MIRBuilder) const {
   MachineRegisterInfo &MRI = MIRBuilder.getMF().getRegInfo();
   bool Is32Bit = (MRI.getType(LHS.getReg()).getSizeInBits() == 32);
   return emitADDS(Is32Bit ? AArch64::WZR : AArch64::XZR, LHS, RHS, MIRBuilder);
 }
 
 MachineInstr *
if (auto ValAndVReg = getConstantVRegValWithLookThrough(RHS.getReg(), MRI)) {
     }
   }
 
   if (auto Fns = selectLogicalShiftedRegister(RHS))
     return emitInstr(OpcTable[1][Is32Bit], {Ty}, {LHS}, MIRBuilder, Fns);
   return emitInstr(OpcTable[2][Is32Bit], {Ty}, {LHS, RHS}, MIRBuilder);
 }
 
-std::pair<MachineInstr *, CmpInst::Predicate>
-AArch64InstructionSelector::emitIntegerCompare(
+MachineInstr *AArch64InstructionSelector::emitIntegerCompare(
     MachineOperand &LHS, MachineOperand &RHS, MachineOperand &Predicate,
     MachineIRBuilder &MIRBuilder) const {
   assert(LHS.isReg() && RHS.isReg() && "Expected LHS and RHS to be registers!");
   assert(Predicate.isPredicate() && "Expected predicate?");
   MachineRegisterInfo &MRI = MIRBuilder.getMF().getRegInfo();
   LLT CmpTy = MRI.getType(LHS.getReg());
   assert(!CmpTy.isVector() && "Expected scalar or pointer");
   unsigned Size = CmpTy.getSizeInBits();
   assert((Size == 32 || Size == 64) && "Expected a 32-bit or 64-bit LHS/RHS?");
-  auto P = static_cast<CmpInst::Predicate>(Predicate.getPredicate());
-
   // Fold the compare into a cmn or tst if possible.
   if (auto FoldCmp = tryFoldIntegerCompare(LHS, RHS, Predicate, MIRBuilder))
-    return {FoldCmp, P};
-
-  // Compares need special handling for their shifted/immediate forms. We
-  // may be able to modify the predicate or an illegal constant to perform
-  // some folding.
-  if (auto ImmedCmp = tryOptArithImmedIntegerCompare(LHS, RHS, P, MIRBuilder))
-    return {ImmedCmp, P};
-  if (auto ShiftedCmp = tryOptArithShiftedCompare(LHS, RHS, MIRBuilder))
-    return {ShiftedCmp, P};
-  unsigned CmpOpc = Size == 32 ? AArch64::SUBSWrr : AArch64::SUBSXrr;
-  auto CmpMI = MIRBuilder.buildInstr(
-      CmpOpc, {MRI.cloneVirtualRegister(LHS.getReg())}, {LHS, RHS});
-  constrainSelectedInstRegOperands(*CmpMI, TII, TRI, RBI);
-  return {&*CmpMI, P};
+    return FoldCmp;
+  auto Dst = MRI.cloneVirtualRegister(LHS.getReg());
+  return emitSUBS(Dst, LHS, RHS, MIRBuilder);
 }
 
 MachineInstr *AArch64InstructionSelector::emitCSetForFCmp(
     Register Dst, CmpInst::Predicate Pred, MachineIRBuilder &MIRBuilder) const {
   MachineRegisterInfo &MRI = *MIRBuilder.getMRI();
 #ifndef NDEBUG
   LLT Ty = MRI.getType(Dst);
   assert(!Ty.isVector() && Ty.getSizeInBits() == 32 &&
if (!CondDef)
     return false;
 
   unsigned CondOpc = CondDef->getOpcode();
   if (CondOpc != TargetOpcode::G_ICMP && CondOpc != TargetOpcode::G_FCMP)
     return false;
 
   AArch64CC::CondCode CondCode;
   if (CondOpc == TargetOpcode::G_ICMP) {
-    MachineInstr *Cmp;
-    CmpInst::Predicate Pred;
-
-    std::tie(Cmp, Pred) =
-        emitIntegerCompare(CondDef->getOperand(2), CondDef->getOperand(3),
-                           CondDef->getOperand(1), MIB);
-
-    // Have to collect the CondCode after emitIntegerCompare, since it can
-    // update the predicate.
-    CondCode = changeICMPPredToAArch64CC(Pred);
+    auto Pred =
+        static_cast<CmpInst::Predicate>(CondDef->getOperand(1).getPredicate());
+    CondCode = changeICMPPredToAArch64CC(Pred);
   } else {
     // Get the condition code for the select.
     CmpInst::Predicate Pred =
         static_cast<CmpInst::Predicate>(CondDef->getOperand(1).getPredicate());
     AArch64CC::CondCode CondCode2;
     changeFCMPPredToAArch64CC(Pred, CondCode, CondCode2);
 
     // changeFCMPPredToAArch64CC sets CondCode2 to AL when we require two
if (!isUnsignedICMPPred(P) && LHSDef &&
 
     return emitTST(LHSDef->getOperand(1),
                    LHSDef->getOperand(2), MIRBuilder);
   }
 
   return nullptr;
 }
 
-MachineInstr *AArch64InstructionSelector::tryOptArithImmedIntegerCompare(
-    MachineOperand &LHS, MachineOperand &RHS, CmpInst::Predicate &P,
-    MachineIRBuilder &MIB) const {
-  // Attempt to select the immediate form of an integer compare.
-  MachineRegisterInfo &MRI = *MIB.getMRI();
-  auto Ty = MRI.getType(LHS.getReg());
-  assert(!Ty.isVector() && "Expected scalar or pointer only?");
-  unsigned Size = Ty.getSizeInBits();
-  assert((Size == 32 || Size == 64) &&
-         "Expected 32 bit or 64 bit compare only?");
-
-  // Check if this is a case we can already handle.
-  InstructionSelector::ComplexRendererFns ImmFns;
-  ImmFns = selectArithImmed(RHS);
-
-  if (!ImmFns) {
-    // We didn't get a rendering function, but we may still have a constant.
-    auto MaybeImmed = getImmedFromMO(RHS);
-    if (!MaybeImmed)
-      return nullptr;
-
-    // We have a constant, but it doesn't fit. Try adjusting it by one and
-    // updating the predicate if possible.
-    uint64_t C = *MaybeImmed;
-    CmpInst::Predicate NewP;
-    switch (P) {
-    default:
-      return nullptr;
-    case CmpInst::ICMP_SLT:
-    case CmpInst::ICMP_SGE:
-      // Check for
-      //
-      // x slt c => x sle c - 1
-      // x sge c => x sgt c - 1
-      //
-      // When c is not the smallest possible negative number.
-      if ((Size == 64 && static_cast<int64_t>(C) == INT64_MIN) ||
-          (Size == 32 && static_cast<int32_t>(C) == INT32_MIN))
-        return nullptr;
-      NewP = (P == CmpInst::ICMP_SLT) ? CmpInst::ICMP_SLE : CmpInst::ICMP_SGT;
-      C -= 1;
-      break;
-    case CmpInst::ICMP_ULT:
-    case CmpInst::ICMP_UGE:
-      // Check for
-      //
-      // x ult c => x ule c - 1
-      // x uge c => x ugt c - 1
-      //
-      // When c is not zero.
-      if (C == 0)
-        return nullptr;
-      NewP = (P == CmpInst::ICMP_ULT) ? CmpInst::ICMP_ULE : CmpInst::ICMP_UGT;
-      C -= 1;
-      break;
-    case CmpInst::ICMP_SLE:
-    case CmpInst::ICMP_SGT:
-      // Check for
-      //
-      // x sle c => x slt c + 1
-      // x sgt c => s sge c + 1
-      //
-      // When c is not the largest possible signed integer.
-      if ((Size == 32 && static_cast<int32_t>(C) == INT32_MAX) ||
-          (Size == 64 && static_cast<int64_t>(C) == INT64_MAX))
-        return nullptr;
-      NewP = (P == CmpInst::ICMP_SLE) ? CmpInst::ICMP_SLT : CmpInst::ICMP_SGE;
-      C += 1;
-      break;
-    case CmpInst::ICMP_ULE:
-    case CmpInst::ICMP_UGT:
-      // Check for
-      //
-      // x ule c => x ult c + 1
-      // x ugt c => s uge c + 1
-      //
-      // When c is not the largest possible unsigned integer.
-      if ((Size == 32 && static_cast<uint32_t>(C) == UINT32_MAX) ||
-          (Size == 64 && C == UINT64_MAX))
-        return nullptr;
-      NewP = (P == CmpInst::ICMP_ULE) ? CmpInst::ICMP_ULT : CmpInst::ICMP_UGE;
-      C += 1;
-      break;
-    }
-
-    // Check if the new constant is valid.
-    if (Size == 32)
-      C = static_cast<uint32_t>(C);
-    ImmFns = select12BitValueWithLeftShift(C);
-    if (!ImmFns)
-      return nullptr;
-    P = NewP;
-  }
-
-  // At this point, we know we can select an immediate form. Go ahead and do
-  // that.
-  unsigned Opc;
-  if (Size == 32) {
-    Opc = AArch64::SUBSWri;
-  } else {
-    Opc = AArch64::SUBSXri;
-  }
-
-  auto CmpMI = MIB.buildInstr(Opc, {Ty}, {LHS.getReg()});
-  for (auto &RenderFn : *ImmFns)
-    RenderFn(CmpMI);
-  constrainSelectedInstRegOperands(*CmpMI, TII, TRI, RBI);
-  return &*CmpMI;
-}
-
-MachineInstr *AArch64InstructionSelector::tryOptArithShiftedCompare(
-    MachineOperand &LHS, MachineOperand &RHS, MachineIRBuilder &MIB) const {
-  // We are looking for the following pattern:
-  //
-  // shift = G_SHL/ASHR/LHSR y, c
-  // ...
-  // cmp = G_ICMP pred, something, shift
-  //
-  // Since we will select the G_ICMP to a SUBS, we can potentially fold the
-  // shift into the subtract.
-  static const unsigned OpcTable[2] = {AArch64::SUBSWrs, AArch64::SUBSXrs};
-  auto ImmFns = selectShiftedRegister(RHS);
-  if (!ImmFns)
-    return nullptr;
-  MachineRegisterInfo &MRI = *MIB.getMRI();
-  auto Ty = MRI.getType(LHS.getReg());
-  assert(!Ty.isVector() && "Expected scalar or pointer only?");
-  unsigned Size = Ty.getSizeInBits();
-  bool Idx = (Size == 64);
-  unsigned Opc = OpcTable[Idx];
-  auto CmpMI = MIB.buildInstr(Opc, {Ty}, {LHS.getReg()});
-  for (auto &RenderFn : *ImmFns)
-    RenderFn(CmpMI);
-  constrainSelectedInstRegOperands(*CmpMI, TII, TRI, RBI);
-  return &*CmpMI;
-}
-
 bool AArch64InstructionSelector::tryOptShuffleDupLane(
     MachineInstr &I, LLT DstTy, LLT SrcTy, ArrayRef<int> Mask,
     MachineRegisterInfo &MRI) const {
   assert(I.getOpcode() == TargetOpcode::G_SHUFFLE_VECTOR);
 
   // We assume that scalar->vector splats have been been handled in the
   // post-legalizer combiner to G_DUP. However splats of a source vector's
   // lane don't fit that pattern, detect it here:

llvm/lib/Target/AArch64/GISel/AArch64PostLegalizerLowering.cpp

  //=== lib/CodeGen/GlobalISel/AArch64PostLegalizerLowering.cpp ------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // Post-legalization lowering for instructions. (E.g. pseudos).
 //
 //===----------------------------------------------------------------------===//
 
 #include "AArch64TargetMachine.h"
+#include "AArch64GlobalISelUtils.h"
 #include "MCTargetDesc/AArch64MCTargetDesc.h"
 #include "llvm/CodeGen/GlobalISel/Combiner.h"
 #include "llvm/CodeGen/GlobalISel/CombinerHelper.h"
 #include "llvm/CodeGen/GlobalISel/CombinerInfo.h"
 #include "llvm/CodeGen/GlobalISel/GISelKnownBits.h"
 #include "llvm/CodeGen/GlobalISel/MIPatternMatch.h"
 #include "llvm/CodeGen/GlobalISel/MachineIRBuilder.h"
 #include "llvm/CodeGen/GlobalISel/Utils.h"
 #include "llvm/CodeGen/MachineDominators.h"
 #include "llvm/CodeGen/MachineFunctionPass.h"
 #include "llvm/CodeGen/MachineInstrBuilder.h"
 #include "llvm/CodeGen/MachineRegisterInfo.h"
 #include "llvm/CodeGen/TargetOpcodes.h"
 #include "llvm/CodeGen/TargetPassConfig.h"
 #include "llvm/Support/Debug.h"
 
 #define DEBUG_TYPE "aarch64-postlegalizer-lowering"
 
 using namespace llvm;
 using namespace MIPatternMatch;
+using namespace AArch64GISelUtils;
 
 /// Represents a pseudo instruction which replaces a G_SHUFFLE_VECTOR.
 ///
 /// Used for matching target-supported shuffles before codegen.
 struct ShuffleVectorPseudo {
   unsigned Opc; ///< Opcode for the instruction. (E.g. G_ZIP1)
   Register Dst; ///< Destination register.
   SmallVector<SrcOp, 2> SrcOps; ///< Source registers.
unsigned NewOpc =
       Opc == TargetOpcode::G_ASHR ? AArch64::G_VASHR : AArch64::G_VLSHR;
   MachineIRBuilder MIB(MI);
   auto ImmDef = MIB.buildConstant(LLT::scalar(32), Imm);
   MIB.buildInstr(NewOpc, {MI.getOperand(0)}, {MI.getOperand(1), ImmDef});
   MI.eraseFromParent();
   return true;
 }
 
+/// Determine if it is possible to modify the \p RHS and predicate \p P of a
+/// comparison instruction such that the right-hand side is an arithmetic
+/// immediate.
+///
+/// \returns A pair containing the updated immediate and predicate which may
+/// be used to optimize the instruction.
+///
+/// \note This assumes that the comparison has been legalized.
+Optional<std::pair<uint64_t, CmpInst::Predicate>>
+tryAdjustICmpImmAndPred(Register RHS, CmpInst::Predicate P,
+                          const MachineRegisterInfo &MRI) {
+  const auto &Ty = MRI.getType(RHS);
+  if (Ty.isVector())
+    return None;
+  unsigned Size = Ty.getSizeInBits();
+  assert((Size == 32 || Size == 64) && "Expected 32 or 64 bit compare only?");
+
+  // If the RHS is not a constant, or the RHS is already a valid arithmetic
+  // immediate, then there is nothing to change.
+  auto ValAndVReg = getConstantVRegValWithLookThrough(RHS, MRI);
+  if (!ValAndVReg)
+    return None;
+  uint64_t C = ValAndVReg->Value;
+  if (isLegalArithImmed(C))
+    return None;
+
+  // We have a non-arithmetic immediate. Check if adjusting the immediate and
+  // adjusting the predicate will result in a legal arithmetic immediate.
+  switch (P) {
+  default:
+    return None;
+  case CmpInst::ICMP_SLT:
+  case CmpInst::ICMP_SGE:
+    // Check for
+    //
+    // x slt c => x sle c - 1
+    // x sge c => x sgt c - 1
+    //
+    // When c is not the smallest possible negative number.
+    if ((Size == 64 && static_cast<int64_t>(C) == INT64_MIN) ||
+        (Size == 32 && static_cast<int32_t>(C) == INT32_MIN))
+      return None;
+    P = (P == CmpInst::ICMP_SLT) ? CmpInst::ICMP_SLE : CmpInst::ICMP_SGT;
+    C -= 1;
+    break;
+  case CmpInst::ICMP_ULT:
+  case CmpInst::ICMP_UGE:
+    // Check for
+    //
+    // x ult c => x ule c - 1
+    // x uge c => x ugt c - 1
+    //
+    // When c is not zero.
+    if (C == 0)
+      return None;
+    P = (P == CmpInst::ICMP_ULT) ? CmpInst::ICMP_ULE : CmpInst::ICMP_UGT;
+    C -= 1;
+    break;
+  case CmpInst::ICMP_SLE:
+  case CmpInst::ICMP_SGT:
+    // Check for
+    //
+    // x sle c => x slt c + 1
+    // x sgt c => s sge c + 1
+    //
+    // When c is not the largest possible signed integer.
+    if ((Size == 32 && static_cast<int32_t>(C) == INT32_MAX) ||
+        (Size == 64 && static_cast<int64_t>(C) == INT64_MAX))
+      return None;
+    P = (P == CmpInst::ICMP_SLE) ? CmpInst::ICMP_SLT : CmpInst::ICMP_SGE;
+    C += 1;
+    break;
+  case CmpInst::ICMP_ULE:
+  case CmpInst::ICMP_UGT:
+    // Check for
+    //
+    // x ule c => x ult c + 1
+    // x ugt c => s uge c + 1
+    //
+    // When c is not the largest possible unsigned integer.
+    if ((Size == 32 && static_cast<uint32_t>(C) == UINT32_MAX) ||
+        (Size == 64 && C == UINT64_MAX))
+      return None;
+    P = (P == CmpInst::ICMP_ULE) ? CmpInst::ICMP_ULT : CmpInst::ICMP_UGE;
+    C += 1;
+    break;
+  }
+
+  // Check if the new constant is valid, and return the updated constant and
+  // predicate if it is.
+  if (Size == 32)
+    C = static_cast<uint32_t>(C);
+  if (!isLegalArithImmed(C))
+    return None;
+  return {{C, P}};
+}
+
+/// \returns true if it is possible to update the RHS + predicate of a compare
+/// such that the RHS will be selected as an arithmetic immediate.
+bool matchAdjustICmpImmAndPred(
+    MachineInstr &MI, const MachineRegisterInfo &MRI,
+    std::pair<uint64_t, CmpInst::Predicate> &MatchInfo) {
+  Register RHS = MI.getOperand(3).getReg();
+  auto Pred = static_cast<CmpInst::Predicate>(MI.getOperand(1).getPredicate());
+  if (auto MaybeNewImmAndPred = tryAdjustICmpImmAndPred(RHS, Pred, MRI)) {
+    MatchInfo = *MaybeNewImmAndPred;
+    errs() << "Optimizing: ";
+    MI.dump();
+    return true;
+  }
+  return false;
+}
+
+bool applyAdjustICmpImmAndPred(
+    MachineInstr &MI, std::pair<uint64_t, CmpInst::Predicate> &MatchInfo,
+    MachineIRBuilder &MIB, GISelChangeObserver &Observer) {
+  MIB.setInstrAndDebugLoc(MI);
+  MachineOperand &RHS = MI.getOperand(3);
+  MachineRegisterInfo &MRI = *MIB.getMRI();
+  auto Cst = MIB.buildConstant(MRI.cloneVirtualRegister(RHS.getReg()),
+                               MatchInfo.first);
+  Observer.changingInstr(MI);
+  RHS.setReg(Cst->getOperand(0).getReg());
+  MI.getOperand(1).setPredicate(MatchInfo.second);
+  Observer.changedInstr(MI);
+  return true;
+}
+
 #define AARCH64POSTLEGALIZERLOWERINGHELPER_GENCOMBINERHELPER_DEPS
 #include "AArch64GenPostLegalizeGILowering.inc"
 #undef AARCH64POSTLEGALIZERLOWERINGHELPER_GENCOMBINERHELPER_DEPS
 
 namespace {
 #define AARCH64POSTLEGALIZERLOWERINGHELPER_GENCOMBINERHELPER_H
 #include "AArch64GenPostLegalizeGILowering.inc"
 #undef AARCH64POSTLEGALIZERLOWERINGHELPER_GENCOMBINERHELPER_H

llvm/test/CodeGen/AArch64/GlobalISel/fold-select.mir

bb.0:
     %7:gpr(s32) = G_FCMP floatpred(oeq), %2(s32), %3
     %4:gpr(s1) = G_TRUNC %7(s32)
     %5:gpr(s32) = G_SELECT %4(s1), %6, %1
     $w0 = COPY %5(s32)
     RET_ReallyLR implicit $w0
 
 ...
 ---
-name:            check_update_predicate
-alignment:       4
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $w0, $w1
-    ; The G_ICMP is optimized here to be a slt comparison by adding 1 to the
-    ; constant. So, the CSELWr should use the predicate code 11, rather than
-    ; 13.
-
-    ; CHECK-LABEL: name: check_update_predicate
-    ; CHECK: liveins: $w0, $w1
-    ; CHECK: %copy1:gpr32sp = COPY $w0
-    ; CHECK: %copy2:gpr32 = COPY $w1
-    ; CHECK: %cst:gpr32 = MOVi32imm -1
-    ; CHECK: [[SUBSWri:%[0-9]+]]:gpr32 = SUBSWri %copy1, 0, 0, implicit-def $nzcv
-    ; CHECK: %select:gpr32 = CSELWr %cst, %copy2, 11, implicit $nzcv
-    ; CHECK: $w0 = COPY %select
-    ; CHECK: RET_ReallyLR implicit $w0
-    %copy1:gpr(s32) = COPY $w0
-    %copy2:gpr(s32) = COPY $w1
-    %cst:gpr(s32) = G_CONSTANT i32 -1
-    %cmp:gpr(s32) = G_ICMP intpred(sle), %copy1(s32), %cst
-    %trunc:gpr(s1) = G_TRUNC %cmp(s32)
-    %select:gpr(s32) = G_SELECT %trunc(s1), %cst, %copy2
-    $w0 = COPY %select(s32)
-    RET_ReallyLR implicit $w0
-...
----
 name:            csinc
 alignment:       4
 legalized:       true
 regBankSelected: true
 tracksRegLiveness: true
 body:             |
   bb.0:
     liveins: $w0, $w1

llvm/test/CodeGen/AArch64/GlobalISel/postlegalizer-lowering-adjust-icmp-imm.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=aarch64 -run-pass=aarch64-postlegalizer-lowering -verify-machineinstrs %s -o - | FileCheck %s
+
+...
+---
+name:            slt_to_sle_s32
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $w0
+
+    ; x slt c => x sle c - 1
+    ;
+    ; log_2(4096) == 12, so we can represent this as a 12 bit value with a
+    ; left shift.
+
+    ; CHECK-LABEL: name: slt_to_sle_s32
+    ; CHECK: liveins: $w0
+    ; CHECK: %reg:_(s32) = COPY $w0
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 4096
+    ; CHECK: %cmp:_(s32) = G_ICMP intpred(sle), %reg(s32), [[C]]
+    ; CHECK: $w0 = COPY %cmp(s32)
+    ; CHECK: RET_ReallyLR implicit $w0
+    %reg:_(s32) = COPY $w0
+    %cst:_(s32) = G_CONSTANT i32 4097
+    %cmp:_(s32) = G_ICMP intpred(slt), %reg(s32), %cst
+    $w0 = COPY %cmp(s32)
+    RET_ReallyLR implicit $w0
+
+...
+---
+name:            slt_to_sle_s64
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $x0
+
+    ; x slt c => x sle c - 1
+    ;
+    ; log_2(4096) == 12, so we can represent this as a 12 bit value with a
+    ; left shift.
+
+    ; CHECK-LABEL: name: slt_to_sle_s64
+    ; CHECK: liveins: $x0
+    ; CHECK: %reg:_(s64) = COPY $x0
+    ; CHECK: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 4096
+    ; CHECK: %cmp:_(s32) = G_ICMP intpred(sle), %reg(s64), [[C]]
+    ; CHECK: %ext:_(s64) = G_ZEXT %cmp(s32)
+    ; CHECK: $x0 = COPY %ext(s64)
+    ; CHECK: RET_ReallyLR implicit $x0
+    %reg:_(s64) = COPY $x0
+    %cst:_(s64) = G_CONSTANT i64 4097
+    %cmp:_(s32) = G_ICMP intpred(slt), %reg(s64), %cst
+    %ext:_(s64) = G_ZEXT %cmp(s32)
+    $x0 = COPY %ext(s64)
+    RET_ReallyLR implicit $x0
+
+...
+---
+name:            sge_to_sgt_s32
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $w0
+
+    ; x sge c => x sgt c - 1
+    ;
+    ; log_2(4096) == 12, so we can represent this as a 12 bit value with a
+    ; left shift.
+
+    ; CHECK-LABEL: name: sge_to_sgt_s32
+    ; CHECK: liveins: $w0
+    ; CHECK: %reg:_(s32) = COPY $w0
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 4096
+    ; CHECK: %cmp:_(s32) = G_ICMP intpred(sgt), %reg(s32), [[C]]
+    ; CHECK: $w0 = COPY %cmp(s32)
+    ; CHECK: RET_ReallyLR implicit $w0
+    %reg:_(s32) = COPY $w0
+    %cst:_(s32) = G_CONSTANT i32 4097
+    %cmp:_(s32) = G_ICMP intpred(sge), %reg(s32), %cst
+    $w0 = COPY %cmp(s32)
+    RET_ReallyLR implicit $w0
+
+...
+---
+name:            sge_to_sgt_s64
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $x0
+
+    ; x sge c => x sgt c - 1
+    ;
+    ; log_2(4096) == 12, so we can represent this as a 12 bit value with a
+    ; left shift.
+
+    ; CHECK-LABEL: name: sge_to_sgt_s64
+    ; CHECK: liveins: $x0
+    ; CHECK: %reg:_(s64) = COPY $x0
+    ; CHECK: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 4096
+    ; CHECK: %cmp:_(s32) = G_ICMP intpred(sgt), %reg(s64), [[C]]
+    ; CHECK: %ext:_(s64) = G_ZEXT %cmp(s32)
+    ; CHECK: $x0 = COPY %ext(s64)
+    ; CHECK: RET_ReallyLR implicit $x0
+    %reg:_(s64) = COPY $x0
+    %cst:_(s64) = G_CONSTANT i64 4097
+    %cmp:_(s32) = G_ICMP intpred(sge), %reg(s64), %cst
+    %ext:_(s64) = G_ZEXT %cmp(s32)
+    $x0 = COPY %ext(s64)
+    RET_ReallyLR implicit $x0
+
+...
+---
+name:            ult_to_ule_s32
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $w0
+
+    ; x ult c => x ule c - 1
+    ;
+    ; log_2(4096) == 12, so we can represent this as a 12 bit value with a
+    ; left shift.
+
+    ; CHECK-LABEL: name: ult_to_ule_s32
+    ; CHECK: liveins: $w0
+    ; CHECK: %reg:_(s32) = COPY $w0
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 4096
+    ; CHECK: %cmp:_(s32) = G_ICMP intpred(ule), %reg(s32), [[C]]
+    ; CHECK: $w0 = COPY %cmp(s32)
+    ; CHECK: RET_ReallyLR implicit $w0
+    %reg:_(s32) = COPY $w0
+    %cst:_(s32) = G_CONSTANT i32 4097
+    %cmp:_(s32) = G_ICMP intpred(ult), %reg(s32), %cst
+    $w0 = COPY %cmp(s32)
+    RET_ReallyLR implicit $w0
+
+...
+---
+name:            ult_to_ule_s64
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $x0
+
+    ; x ult c => x ule c - 1
+    ;
+    ; log_2(4096) == 12, so we can represent this as a 12 bit value with a
+    ; left shift.
+
+    ; CHECK-LABEL: name: ult_to_ule_s64
+    ; CHECK: liveins: $x0
+    ; CHECK: %reg:_(s64) = COPY $x0
+    ; CHECK: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 4096
+    ; CHECK: %cmp:_(s32) = G_ICMP intpred(ule), %reg(s64), [[C]]
+    ; CHECK: %ext:_(s64) = G_ANYEXT %cmp(s32)
+    ; CHECK: $x0 = COPY %ext(s64)
+    ; CHECK: RET_ReallyLR implicit $x0
+    %reg:_(s64) = COPY $x0
+    %cst:_(s64) = G_CONSTANT i64 4097
+    %cmp:_(s32) = G_ICMP intpred(ult), %reg(s64), %cst
+    %ext:_(s64) = G_ANYEXT %cmp(s32)
+    $x0 = COPY %ext(s64)
+    RET_ReallyLR implicit $x0
+
+...
+---
+name:            uge_to_ugt_s32
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $w0
+
+    ; x uge c => x ugt c - 1
+    ;
+    ; log_2(4096) == 12, so we can represent this as a 12 bit value with a
+    ; left shift.
+
+    ; CHECK-LABEL: name: uge_to_ugt_s32
+    ; CHECK: liveins: $w0
+    ; CHECK: %reg:_(s32) = COPY $w0
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 4096
+    ; CHECK: %cmp:_(s32) = G_ICMP intpred(ugt), %reg(s32), [[C]]
+    ; CHECK: $w0 = COPY %cmp(s32)
+    ; CHECK: RET_ReallyLR implicit $w0
+    %reg:_(s32) = COPY $w0
+    %cst:_(s32) = G_CONSTANT i32 4097
+    %cmp:_(s32) = G_ICMP intpred(uge), %reg(s32), %cst
+    $w0 = COPY %cmp(s32)
+    RET_ReallyLR implicit $w0
+
+...
+---
+name:            uge_to_ugt_s64
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $x0
+
+    ; x uge c => x ugt c - 1
+    ;
+    ; log_2(4096) == 12, so we can represent this as a 12 bit value with a
+    ; left shift.
+
+    ; CHECK-LABEL: name: uge_to_ugt_s64
+    ; CHECK: liveins: $x0
+    ; CHECK: %reg:_(s64) = COPY $x0
+    ; CHECK: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 4096
+    ; CHECK: %cmp:_(s32) = G_ICMP intpred(ugt), %reg(s64), [[C]]
+    ; CHECK: %ext:_(s64) = G_ZEXT %cmp(s32)
+    ; CHECK: $x0 = COPY %ext(s64)
+    ; CHECK: RET_ReallyLR implicit $x0
+    %reg:_(s64) = COPY $x0
+    %cst:_(s64) = G_CONSTANT i64 4097
+    %cmp:_(s32) = G_ICMP intpred(uge), %reg(s64), %cst
+    %ext:_(s64) = G_ZEXT %cmp(s32)
+    $x0 = COPY %ext(s64)
+    RET_ReallyLR implicit $x0
+
+...
+---
+name:            sle_to_slt_s32
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $w0
+
+    ; x sle c => x slt c + 1
+    ;
+    ; log_2(8192) == 13, so we can represent this as a 12 bit value with a
+    ; left shift.
+    ;
+    ; (We can't use 4095 here, because that's a legal arithmetic immediate.)
+
+    ; CHECK-LABEL: name: sle_to_slt_s32
+    ; CHECK: liveins: $w0
+    ; CHECK: %reg:_(s32) = COPY $w0
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 8192
+    ; CHECK: %cmp:_(s32) = G_ICMP intpred(slt), %reg(s32), [[C]]
+    ; CHECK: $w0 = COPY %cmp(s32)
+    ; CHECK: RET_ReallyLR implicit $w0
+    %reg:_(s32) = COPY $w0
+    %cst:_(s32) = G_CONSTANT i32 8191
+    %cmp:_(s32) = G_ICMP intpred(sle), %reg(s32), %cst
+    $w0 = COPY %cmp(s32)
+    RET_ReallyLR implicit $w0
+
+...
+---
+name:            sle_to_slt_s64
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $x0
+
+    ; x sle c => x slt c + 1
+    ;
+    ; log_2(8192) == 13, so we can represent this as a 12 bit value with a
+    ; left shift.
+
+    ; CHECK-LABEL: name: sle_to_slt_s64
+    ; CHECK: liveins: $x0
+    ; CHECK: %reg:_(s64) = COPY $x0
+    ; CHECK: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 8192
+    ; CHECK: %cmp:_(s32) = G_ICMP intpred(slt), %reg(s64), [[C]]
+    ; CHECK: %ext:_(s64) = G_ZEXT %cmp(s32)
+    ; CHECK: $x0 = COPY %ext(s64)
+    ; CHECK: RET_ReallyLR implicit $x0
+    %reg:_(s64) = COPY $x0
+    %cst:_(s64) = G_CONSTANT i64 8191
+    %cmp:_(s32) = G_ICMP intpred(sle), %reg(s64), %cst
+    %ext:_(s64) = G_ZEXT %cmp(s32)
+    $x0 = COPY %ext(s64)
+    RET_ReallyLR implicit $x0
+
+...
+---
+name:            sgt_to_sge_s32
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $w0
+
+    ; x sgt c => s sge c + 1
+    ;
+    ; log_2(8192) == 13, so we can represent this as a 12 bit value with a
+    ; left shift.
+
+    ; CHECK-LABEL: name: sgt_to_sge_s32
+    ; CHECK: liveins: $w0
+    ; CHECK: %reg:_(s32) = COPY $w0
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 8192
+    ; CHECK: %cmp:_(s32) = G_ICMP intpred(sge), %reg(s32), [[C]]
+    ; CHECK: $w0 = COPY %cmp(s32)
+    ; CHECK: RET_ReallyLR implicit $w0
+    %reg:_(s32) = COPY $w0
+    %cst:_(s32) = G_CONSTANT i32 8191
+    %cmp:_(s32) = G_ICMP intpred(sgt), %reg(s32), %cst
+    $w0 = COPY %cmp(s32)
+    RET_ReallyLR implicit $w0
+
+...
+---
+name:            sgt_to_sge_s64
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $x0
+
+    ; x sgt c => s sge c + 1
+    ;
+    ; log_2(8192) == 13, so we can represent this as a 12 bit value with a
+    ; left shift.
+
+    ; CHECK-LABEL: name: sgt_to_sge_s64
+    ; CHECK: liveins: $x0
+    ; CHECK: %reg:_(s64) = COPY $x0
+    ; CHECK: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 8192
+    ; CHECK: %cmp:_(s32) = G_ICMP intpred(sge), %reg(s64), [[C]]
+    ; CHECK: %ext:_(s64) = G_ANYEXT %cmp(s32)
+    ; CHECK: $x0 = COPY %ext(s64)
+    ; CHECK: RET_ReallyLR implicit $x0
+    %reg:_(s64) = COPY $x0
+    %cst:_(s64) = G_CONSTANT i64 8191
+    %cmp:_(s32) = G_ICMP intpred(sgt), %reg(s64), %cst
+    %ext:_(s64) = G_ANYEXT %cmp(s32)
+    $x0 = COPY %ext(s64)
+    RET_ReallyLR implicit $x0
+
+...
+---
+name:            no_opt_int32_min
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $w0
+
+    ; The cmp should not change.
+    ;
+    ; If we subtract 1 from the constant, it will wrap around, and so it's not
+    ; true that
+    ;
+    ; x slt c => x sle c - 1
+    ; x sge c => x sgt c - 1
+
+    ; CHECK-LABEL: name: no_opt_int32_min
+    ; CHECK: liveins: $w0
+    ; CHECK: %reg:_(s32) = COPY $w0
+    ; CHECK: %cst:_(s32) = G_CONSTANT i32 -2147483648
+    ; CHECK: %cmp:_(s32) = G_ICMP intpred(slt), %reg(s32), %cst
+    ; CHECK: $w0 = COPY %cmp(s32)
+    ; CHECK: RET_ReallyLR implicit $w0
+    %reg:_(s32) = COPY $w0
+    %cst:_(s32) = G_CONSTANT i32 -2147483648
+    %cmp:_(s32) = G_ICMP intpred(slt), %reg(s32), %cst
+    $w0 = COPY %cmp(s32)
+    RET_ReallyLR implicit $w0
+
+...
+---
+name:            no_opt_int64_min
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $x0
+
+    ; The cmp should not change.
+    ;
+    ; If we subtract 1 from the constant, it will wrap around, and so it's not
+    ; true that
+    ;
+    ; x slt c => x sle c - 1
+    ; x sge c => x sgt c - 1
+
+    ; CHECK-LABEL: name: no_opt_int64_min
+    ; CHECK: liveins: $x0
+    ; CHECK: %reg:_(s64) = COPY $x0
+    ; CHECK: %cst:_(s64) = G_CONSTANT i64 -9223372036854775808
+    ; CHECK: %cmp:_(s32) = G_ICMP intpred(slt), %reg(s64), %cst
+    ; CHECK: %ext:_(s64) = G_ANYEXT %cmp(s32)
+    ; CHECK: $x0 = COPY %ext(s64)
+    ; CHECK: RET_ReallyLR implicit $x0
+    %reg:_(s64) = COPY $x0
+    %cst:_(s64) = G_CONSTANT i64 -9223372036854775808
+    %cmp:_(s32) = G_ICMP intpred(slt), %reg(s64), %cst
+    %ext:_(s64) = G_ANYEXT %cmp(s32)
+    $x0 = COPY %ext(s64)
+    RET_ReallyLR implicit $x0
+
+...
+---
+name:            no_opt_int32_max
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $w0
+
+    ; The cmp should not change.
+    ;
+    ; If we add 1 to the constant, it will wrap around, and so it's not true
+    ; that
+    ;
+    ; x slt c => x sle c - 1
+    ; x sge c => x sgt c - 1
+
+    ; CHECK-LABEL: name: no_opt_int32_max
+    ; CHECK: liveins: $w0
+    ; CHECK: %reg:_(s32) = COPY $w0
+    ; CHECK: %cst:_(s32) = G_CONSTANT i32 2147483647
+    ; CHECK: %cmp:_(s32) = G_ICMP intpred(sle), %reg(s32), %cst
+    ; CHECK: $w0 = COPY %cmp(s32)
+    ; CHECK: RET_ReallyLR implicit $w0
+    %reg:_(s32) = COPY $w0
+    %cst:_(s32) = G_CONSTANT i32 2147483647
+    %cmp:_(s32) = G_ICMP intpred(sle), %reg(s32), %cst
+    $w0 = COPY %cmp(s32)
+    RET_ReallyLR implicit $w0
+
+...
+---
+name:            no_opt_int64_max
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $x0
+
+    ; The cmp should not change.
+    ;
+    ; If we add 1 to the constant, it will wrap around, and so it's not true
+    ; that
+    ;
+    ; x slt c => x sle c - 1
+    ; x sge c => x sgt c - 1
+
+
+    ; CHECK-LABEL: name: no_opt_int64_max
+    ; CHECK: liveins: $x0
+    ; CHECK: %reg:_(s64) = COPY $x0
+    ; CHECK: %cst:_(s64) = G_CONSTANT i64 9223372036854775807
+    ; CHECK: %cmp:_(s32) = G_ICMP intpred(sle), %reg(s64), %cst
+    ; CHECK: %ext:_(s64) = G_ZEXT %cmp(s32)
+    ; CHECK: $x0 = COPY %ext(s64)
+    ; CHECK: RET_ReallyLR implicit $x0
+    %reg:_(s64) = COPY $x0
+    %cst:_(s64) = G_CONSTANT i64 9223372036854775807
+    %cmp:_(s32) = G_ICMP intpred(sle), %reg(s64), %cst
+    %ext:_(s64) = G_ZEXT %cmp(s32)
+    $x0 = COPY %ext(s64)
+    RET_ReallyLR implicit $x0
+
+...
+---
+name:            no_opt_zero
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $x0
+
+    ; The cmp should not change.
+    ;
+    ; This is an unsigned comparison, so when the constant is 0, the following
+    ; does not hold:
+    ;
+    ; x slt c => x sle c - 1
+    ; x sge c => x sgt c - 1
+
+    ; CHECK-LABEL: name: no_opt_zero
+    ; CHECK: liveins: $x0
+    ; CHECK: %reg:_(s64) = COPY $x0
+    ; CHECK: %cst:_(s64) = G_CONSTANT i64 0
+    ; CHECK: %cmp:_(s32) = G_ICMP intpred(ult), %reg(s64), %cst
+    ; CHECK: %ext:_(s64) = G_ZEXT %cmp(s32)
+    ; CHECK: $x0 = COPY %ext(s64)
+    ; CHECK: RET_ReallyLR implicit $x0
+    %reg:_(s64) = COPY $x0
+    %cst:_(s64) = G_CONSTANT i64 0
+    %cmp:_(s32) = G_ICMP intpred(ult), %reg(s64), %cst
+    %ext:_(s64) = G_ZEXT %cmp(s32)
+    $x0 = COPY %ext(s64)
+    RET_ReallyLR implicit $x0
+

llvm/test/CodeGen/AArch64/GlobalISel/select-arith-immed-compare.mir

-# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
-# RUN: llc -mtriple=aarch64 -run-pass=instruction-select -verify-machineinstrs %s -o - | FileCheck %s
-
-name:            slt_to_sle_s32
-alignment:       4
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $w0
-
-    ; x slt c => x sle c - 1
-    ;
-    ; We should not have a MOV here. We can subtract 1 from the constant and
-    ; change the condition code.
-    ;
-    ; log_2(4096) == 12, so we can represent this as a 12 bit value with a
-    ; left shift.
-
-    ; CHECK-LABEL: name: slt_to_sle_s32
-    ; CHECK: liveins: $w0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr32sp = COPY $w0
-    ; CHECK: SUBSWri [[COPY]], 1, 12, implicit-def $nzcv
-    ; CHECK: [[CSINCWr:%[0-9]+]]:gpr32 = CSINCWr $wzr, $wzr, 12, implicit $nzcv
-    ; CHECK: [[ANDWri:%[0-9]+]]:gpr32sp = ANDWri [[CSINCWr]], 0
-    ; CHECK: $w0 = COPY [[ANDWri]]
-    ; CHECK: RET_ReallyLR implicit $w0
-    %0:gpr(s32) = COPY $w0
-    %1:gpr(s32) = G_CONSTANT i32 4097
-    %4:gpr(s32) = G_ICMP intpred(slt), %0(s32), %1
-    %5:gpr(s32) = G_CONSTANT i32 1
-    %3:gpr(s32) = G_AND %4, %5
-    $w0 = COPY %3(s32)
-    RET_ReallyLR implicit $w0
-
-...
----
-name:            slt_to_sle_s64
-alignment:       4
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $x0
-
-    ; x slt c => x sle c - 1
-    ;
-    ; We should not have a MOV here. We can subtract 1 from the constant and
-    ; change the condition code.
-    ;
-    ; log_2(4096) == 12, so we can represent this as a 12 bit value with a
-    ; left shift.
-
-    ; CHECK-LABEL: name: slt_to_sle_s64
-    ; CHECK: liveins: $x0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr64sp = COPY $x0
-    ; CHECK: [[SUBSXri:%[0-9]+]]:gpr64 = SUBSXri [[COPY]], 1, 12, implicit-def $nzcv
-    ; CHECK: [[CSINCWr:%[0-9]+]]:gpr32 = CSINCWr $wzr, $wzr, 12, implicit $nzcv
-    ; CHECK: [[DEF:%[0-9]+]]:gpr64all = IMPLICIT_DEF
-    ; CHECK: [[INSERT_SUBREG:%[0-9]+]]:gpr64 = INSERT_SUBREG [[DEF]], [[CSINCWr]], %subreg.sub_32
-    ; CHECK: [[ANDXri:%[0-9]+]]:gpr64sp = ANDXri [[INSERT_SUBREG]], 4096
-    ; CHECK: $x0 = COPY [[ANDXri]]
-    ; CHECK: RET_ReallyLR implicit $x0
-    %0:gpr(s64) = COPY $x0
-    %1:gpr(s64) = G_CONSTANT i64 4097
-    %4:gpr(s32) = G_ICMP intpred(slt), %0(s64), %1
-    %6:gpr(s64) = G_ANYEXT %4(s32)
-    %5:gpr(s64) = G_CONSTANT i64 1
-    %3:gpr(s64) = G_AND %6, %5
-    $x0 = COPY %3(s64)
-    RET_ReallyLR implicit $x0
-
-...
----
-name:            sge_to_sgt_s32
-alignment:       4
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $w0
-
-    ; x sge c => x sgt c - 1
-    ;
-    ; We should not have a MOV here. We can subtract 1 from the constant and
-    ; change the condition code.
-    ;
-    ; log_2(4096) == 12, so we can represent this as a 12 bit value with a
-    ; left shift.
-
-    ; CHECK-LABEL: name: sge_to_sgt_s32
-    ; CHECK: liveins: $w0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr32sp = COPY $w0
-    ; CHECK: [[SUBSWri:%[0-9]+]]:gpr32 = SUBSWri [[COPY]], 1, 12, implicit-def $nzcv
-    ; CHECK: [[CSINCWr:%[0-9]+]]:gpr32 = CSINCWr $wzr, $wzr, 13, implicit $nzcv
-    ; CHECK: [[ANDWri:%[0-9]+]]:gpr32sp = ANDWri [[CSINCWr]], 0
-    ; CHECK: $w0 = COPY [[ANDWri]]
-    ; CHECK: RET_ReallyLR implicit $w0
-    %0:gpr(s32) = COPY $w0
-    %1:gpr(s32) = G_CONSTANT i32 4097
-    %4:gpr(s32) = G_ICMP intpred(sge), %0(s32), %1
-    %5:gpr(s32) = G_CONSTANT i32 1
-    %3:gpr(s32) = G_AND %4, %5
-    $w0 = COPY %3(s32)
-    RET_ReallyLR implicit $w0
-
-...
----
-name:            sge_to_sgt_s64
-alignment:       4
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $x0
-
-    ; x sge c => x sgt c - 1
-    ;
-    ; We should not have a MOV here. We can subtract 1 from the constant and
-    ; change the condition code.
-    ;
-    ; log_2(4096) == 12, so we can represent this as a 12 bit value with a
-    ; left shift.
-
-    ; CHECK-LABEL: name: sge_to_sgt_s64
-    ; CHECK: liveins: $x0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr64sp = COPY $x0
-    ; CHECK: [[SUBSXri:%[0-9]+]]:gpr64 = SUBSXri [[COPY]], 1, 12, implicit-def $nzcv
-    ; CHECK: [[CSINCWr:%[0-9]+]]:gpr32 = CSINCWr $wzr, $wzr, 13, implicit $nzcv
-    ; CHECK: [[DEF:%[0-9]+]]:gpr64all = IMPLICIT_DEF
-    ; CHECK: [[INSERT_SUBREG:%[0-9]+]]:gpr64 = INSERT_SUBREG [[DEF]], [[CSINCWr]], %subreg.sub_32
-    ; CHECK: [[ANDXri:%[0-9]+]]:gpr64sp = ANDXri [[INSERT_SUBREG]], 4096
-    ; CHECK: $x0 = COPY [[ANDXri]]
-    ; CHECK: RET_ReallyLR implicit $x0
-    %0:gpr(s64) = COPY $x0
-    %1:gpr(s64) = G_CONSTANT i64 4097
-    %4:gpr(s32) = G_ICMP intpred(sge), %0(s64), %1
-    %6:gpr(s64) = G_ANYEXT %4(s32)
-    %5:gpr(s64) = G_CONSTANT i64 1
-    %3:gpr(s64) = G_AND %6, %5
-    $x0 = COPY %3(s64)
-    RET_ReallyLR implicit $x0
-
-...
----
-name:            ult_to_ule_s32
-alignment:       4
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $w0
-
-    ; x ult c => x ule c - 1
-    ;
-    ; We should not have a MOV here. We can subtract 1 from the constant and
-    ; change the condition code.
-    ;
-    ; log_2(4096) == 12, so we can represent this as a 12 bit value with a
-    ; left shift.
-
-    ; CHECK-LABEL: name: ult_to_ule_s32
-    ; CHECK: liveins: $w0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr32sp = COPY $w0
-    ; CHECK: [[SUBSWri:%[0-9]+]]:gpr32 = SUBSWri [[COPY]], 1, 12, implicit-def $nzcv
-    ; CHECK: [[CSINCWr:%[0-9]+]]:gpr32 = CSINCWr $wzr, $wzr, 8, implicit $nzcv
-    ; CHECK: [[ANDWri:%[0-9]+]]:gpr32sp = ANDWri [[CSINCWr]], 0
-    ; CHECK: $w0 = COPY [[ANDWri]]
-    ; CHECK: RET_ReallyLR implicit $w0
-    %0:gpr(s32) = COPY $w0
-    %1:gpr(s32) = G_CONSTANT i32 4097
-    %4:gpr(s32) = G_ICMP intpred(ult), %0(s32), %1
-    %5:gpr(s32) = G_CONSTANT i32 1
-    %3:gpr(s32) = G_AND %4, %5
-    $w0 = COPY %3(s32)
-    RET_ReallyLR implicit $w0
-
-...
----
-name:            ult_to_ule_s64
-alignment:       4
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $x0
-
-    ; x ult c => x ule c - 1
-    ;
-    ; We should not have a MOV here. We can subtract 1 from the constant and
-    ; change the condition code.
-    ;
-    ; log_2(4096) == 12, so we can represent this as a 12 bit value with a
-    ; left shift.
-
-    ; CHECK-LABEL: name: ult_to_ule_s64
-    ; CHECK: liveins: $x0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr64sp = COPY $x0
-    ; CHECK: [[SUBSXri:%[0-9]+]]:gpr64 = SUBSXri [[COPY]], 1, 12, implicit-def $nzcv
-    ; CHECK: [[CSINCWr:%[0-9]+]]:gpr32 = CSINCWr $wzr, $wzr, 8, implicit $nzcv
-    ; CHECK: [[DEF:%[0-9]+]]:gpr64all = IMPLICIT_DEF
-    ; CHECK: [[INSERT_SUBREG:%[0-9]+]]:gpr64 = INSERT_SUBREG [[DEF]], [[CSINCWr]], %subreg.sub_32
-    ; CHECK: [[ANDXri:%[0-9]+]]:gpr64sp = ANDXri [[INSERT_SUBREG]], 4096
-    ; CHECK: $x0 = COPY [[ANDXri]]
-    ; CHECK: RET_ReallyLR implicit $x0
-    %0:gpr(s64) = COPY $x0
-    %1:gpr(s64) = G_CONSTANT i64 4097
-    %4:gpr(s32) = G_ICMP intpred(ult), %0(s64), %1
-    %6:gpr(s64) = G_ANYEXT %4(s32)
-    %5:gpr(s64) = G_CONSTANT i64 1
-    %3:gpr(s64) = G_AND %6, %5
-    $x0 = COPY %3(s64)
-    RET_ReallyLR implicit $x0
-
-...
----
-name:            uge_to_ugt_s32
-alignment:       4
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $w0
-
-    ; x uge c => x ugt c - 1
-    ;
-    ; We should not have a MOV here. We can subtract 1 from the constant and
-    ; change the condition code.
-    ;
-    ; log_2(4096) == 12, so we can represent this as a 12 bit value with a
-    ; left shift.
-
-    ; CHECK-LABEL: name: uge_to_ugt_s32
-    ; CHECK: liveins: $w0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr32sp = COPY $w0
-    ; CHECK: [[SUBSWri:%[0-9]+]]:gpr32 = SUBSWri [[COPY]], 1, 12, implicit-def $nzcv
-    ; CHECK: [[CSINCWr:%[0-9]+]]:gpr32 = CSINCWr $wzr, $wzr, 9, implicit $nzcv
-    ; CHECK: [[ANDWri:%[0-9]+]]:gpr32sp = ANDWri [[CSINCWr]], 0
-    ; CHECK: $w0 = COPY [[ANDWri]]
-    ; CHECK: RET_ReallyLR implicit $w0
-    %0:gpr(s32) = COPY $w0
-    %1:gpr(s32) = G_CONSTANT i32 4097
-    %4:gpr(s32) = G_ICMP intpred(uge), %0(s32), %1
-    %5:gpr(s32) = G_CONSTANT i32 1
-    %3:gpr(s32) = G_AND %4, %5
-    $w0 = COPY %3(s32)
-    RET_ReallyLR implicit $w0
-
-...
----
-name:            uge_to_ugt_s64
-alignment:       4
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $x0
-
-    ; x uge c => x ugt c - 1
-    ;
-    ; We should not have a MOV here. We can subtract 1 from the constant and
-    ; change the condition code.
-    ;
-    ; log_2(4096) == 12, so we can represent this as a 12 bit value with a
-    ; left shift.
-
-    ; CHECK-LABEL: name: uge_to_ugt_s64
-    ; CHECK: liveins: $x0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr64sp = COPY $x0
-    ; CHECK: [[SUBSXri:%[0-9]+]]:gpr64 = SUBSXri [[COPY]], 1, 12, implicit-def $nzcv
-    ; CHECK: [[CSINCWr:%[0-9]+]]:gpr32 = CSINCWr $wzr, $wzr, 9, implicit $nzcv
-    ; CHECK: [[DEF:%[0-9]+]]:gpr64all = IMPLICIT_DEF
-    ; CHECK: [[INSERT_SUBREG:%[0-9]+]]:gpr64 = INSERT_SUBREG [[DEF]], [[CSINCWr]], %subreg.sub_32
-    ; CHECK: [[ANDXri:%[0-9]+]]:gpr64sp = ANDXri [[INSERT_SUBREG]], 4096
-    ; CHECK: $x0 = COPY [[ANDXri]]
-    ; CHECK: RET_ReallyLR implicit $x0
-    %0:gpr(s64) = COPY $x0
-    %1:gpr(s64) = G_CONSTANT i64 4097
-    %4:gpr(s32) = G_ICMP intpred(uge), %0(s64), %1
-    %6:gpr(s64) = G_ANYEXT %4(s32)
-    %5:gpr(s64) = G_CONSTANT i64 1
-    %3:gpr(s64) = G_AND %6, %5
-    $x0 = COPY %3(s64)
-    RET_ReallyLR implicit $x0
-
-...
----
-name:            sle_to_slt_s32
-alignment:       4
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $w0
-
-    ; x sle c => x slt c + 1
-    ;
-    ; We should not have a MOV here. We can add 1 to the constant and change
-    ; the condition code.
-    ;
-    ; log_2(8192) == 13, so we can represent this as a 12 bit value with a
-    ; left shift.
-    ;
-    ; (We can't use 4095 here, because that's a legal arithmetic immediate.)
-
-    ; CHECK-LABEL: name: sle_to_slt_s32
-    ; CHECK: liveins: $w0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr32sp = COPY $w0
-    ; CHECK: [[SUBSWri:%[0-9]+]]:gpr32 = SUBSWri [[COPY]], 2, 12, implicit-def $nzcv
-    ; CHECK: [[CSINCWr:%[0-9]+]]:gpr32 = CSINCWr $wzr, $wzr, 10, implicit $nzcv
-    ; CHECK: [[ANDWri:%[0-9]+]]:gpr32sp = ANDWri [[CSINCWr]], 0
-    ; CHECK: $w0 = COPY [[ANDWri]]
-    ; CHECK: RET_ReallyLR implicit $w0
-    %0:gpr(s32) = COPY $w0
-    %1:gpr(s32) = G_CONSTANT i32 8191
-    %4:gpr(s32) = G_ICMP intpred(sle), %0(s32), %1
-    %5:gpr(s32) = G_CONSTANT i32 1
-    %3:gpr(s32) = G_AND %4, %5
-    $w0 = COPY %3(s32)
-    RET_ReallyLR implicit $w0
-
-...
----
-name:            sle_to_slt_s64
-alignment:       4
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $x0
-
-    ; x sle c => x slt c + 1
-    ;
-    ; We should not have a MOV here. We can add 1 to the constant and change
-    ; the condition code.
-    ;
-    ; log_2(8192) == 13, so we can represent this as a 12 bit value with a
-    ; left shift.
-
-    ; CHECK-LABEL: name: sle_to_slt_s64
-    ; CHECK: liveins: $x0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr64sp = COPY $x0
-    ; CHECK: [[SUBSXri:%[0-9]+]]:gpr64 = SUBSXri [[COPY]], 2, 12, implicit-def $nzcv
-    ; CHECK: [[CSINCWr:%[0-9]+]]:gpr32 = CSINCWr $wzr, $wzr, 10, implicit $nzcv
-    ; CHECK: [[DEF:%[0-9]+]]:gpr64all = IMPLICIT_DEF
-    ; CHECK: [[INSERT_SUBREG:%[0-9]+]]:gpr64 = INSERT_SUBREG [[DEF]], [[CSINCWr]], %subreg.sub_32
-    ; CHECK: [[ANDXri:%[0-9]+]]:gpr64sp = ANDXri [[INSERT_SUBREG]], 4096
-    ; CHECK: $x0 = COPY [[ANDXri]]
-    ; CHECK: RET_ReallyLR implicit $x0
-    %0:gpr(s64) = COPY $x0
-    %1:gpr(s64) = G_CONSTANT i64 8191
-    %4:gpr(s32) = G_ICMP intpred(sle), %0(s64), %1
-    %6:gpr(s64) = G_ANYEXT %4(s32)
-    %5:gpr(s64) = G_CONSTANT i64 1
-    %3:gpr(s64) = G_AND %6, %5
-    $x0 = COPY %3(s64)
-    RET_ReallyLR implicit $x0
-
-...
----
-name:            sgt_to_sge_s32
-alignment:       4
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $w0
-
-    ; x sgt c => s sge c + 1
-    ;
-    ; We should not have a MOV here. We can add 1 to the constant and change
-    ; the condition code.
-    ;
-    ; log_2(8192) == 13, so we can represent this as a 12 bit value with a
-    ; left shift.
-
-    ; CHECK-LABEL: name: sgt_to_sge_s32
-    ; CHECK: liveins: $w0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr32sp = COPY $w0
-    ; CHECK: [[SUBSWri:%[0-9]+]]:gpr32 = SUBSWri [[COPY]], 2, 12, implicit-def $nzcv
-    ; CHECK: [[CSINCWr:%[0-9]+]]:gpr32 = CSINCWr $wzr, $wzr, 11, implicit $nzcv
-    ; CHECK: [[ANDWri:%[0-9]+]]:gpr32sp = ANDWri [[CSINCWr]], 0
-    ; CHECK: $w0 = COPY [[ANDWri]]
-    ; CHECK: RET_ReallyLR implicit $w0
-    %0:gpr(s32) = COPY $w0
-    %1:gpr(s32) = G_CONSTANT i32 8191
-    %4:gpr(s32) = G_ICMP intpred(sgt), %0(s32), %1
-    %5:gpr(s32) = G_CONSTANT i32 1
-    %3:gpr(s32) = G_AND %4, %5
-    $w0 = COPY %3(s32)
-    RET_ReallyLR implicit $w0
-
-...
----
-name:            sgt_to_sge_s64
-alignment:       4
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $x0
-
-    ; x sgt c => s sge c + 1
-    ;
-    ; We should not have a MOV here. We can add 1 to the constant and change
-    ; the condition code.
-    ;
-    ; log_2(8192) == 13, so we can represent this as a 12 bit value with a
-    ; left shift.
-
-    ; CHECK-LABEL: name: sgt_to_sge_s64
-    ; CHECK: liveins: $x0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr64sp = COPY $x0
-    ; CHECK: [[SUBSXri:%[0-9]+]]:gpr64 = SUBSXri [[COPY]], 2, 12, implicit-def $nzcv
-    ; CHECK: [[CSINCWr:%[0-9]+]]:gpr32 = CSINCWr $wzr, $wzr, 11, implicit $nzcv
-    ; CHECK: [[DEF:%[0-9]+]]:gpr64all = IMPLICIT_DEF
-    ; CHECK: [[INSERT_SUBREG:%[0-9]+]]:gpr64 = INSERT_SUBREG [[DEF]], [[CSINCWr]], %subreg.sub_32
-    ; CHECK: [[ANDXri:%[0-9]+]]:gpr64sp = ANDXri [[INSERT_SUBREG]], 4096
-    ; CHECK: $x0 = COPY [[ANDXri]]
-    ; CHECK: RET_ReallyLR implicit $x0
-    %0:gpr(s64) = COPY $x0
-    %1:gpr(s64) = G_CONSTANT i64 8191
-    %4:gpr(s32) = G_ICMP intpred(sgt), %0(s64), %1
-    %6:gpr(s64) = G_ANYEXT %4(s32)
-    %5:gpr(s64) = G_CONSTANT i64 1
-    %3:gpr(s64) = G_AND %6, %5
-    $x0 = COPY %3(s64)
-    RET_ReallyLR implicit $x0
-
-...
----
-name:            no_opt_int32_min
-alignment:       4
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $w0
-
-    ; This one should contain a MOV.
-    ;
-    ; If we subtract 1 from the constant, it will wrap around, and so it's not
-    ; true that
-    ;
-    ; x slt c => x sle c - 1
-    ; x sge c => x sgt c - 1
-
-    ; CHECK-LABEL: name: no_opt_int32_min
-    ; CHECK: liveins: $w0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr32 = COPY $w0
-    ; CHECK: [[MOVi32imm:%[0-9]+]]:gpr32 = MOVi32imm -2147483648
-    ; CHECK: [[SUBSWrr:%[0-9]+]]:gpr32 = SUBSWrr [[COPY]], [[MOVi32imm]], implicit-def $nzcv
-    ; CHECK: [[CSINCWr:%[0-9]+]]:gpr32 = CSINCWr $wzr, $wzr, 10, implicit $nzcv
-    ; CHECK: [[ANDWri:%[0-9]+]]:gpr32sp = ANDWri [[CSINCWr]], 0
-    ; CHECK: $w0 = COPY [[ANDWri]]
-    ; CHECK: RET_ReallyLR implicit $w0
-    %0:gpr(s32) = COPY $w0
-    %1:gpr(s32) = G_CONSTANT i32 -2147483648
-    %4:gpr(s32) = G_ICMP intpred(slt), %0(s32), %1
-    %5:gpr(s32) = G_CONSTANT i32 1
-    %3:gpr(s32) = G_AND %4, %5
-    $w0 = COPY %3(s32)
-    RET_ReallyLR implicit $w0
-
-...
----
-name:            no_opt_int64_min
-alignment:       4
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $x0
-
-    ; This one should contain a MOV.
-    ;
-    ; If we subtract 1 from the constant, it will wrap around, and so it's not
-    ; true that
-    ;
-    ; x slt c => x sle c - 1
-    ; x sge c => x sgt c - 1
-
-    ; CHECK-LABEL: name: no_opt_int64_min
-    ; CHECK: liveins: $x0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr64 = COPY $x0
-    ; CHECK: [[MOVi64imm:%[0-9]+]]:gpr64 = MOVi64imm -9223372036854775808
-    ; CHECK: [[SUBSXrr:%[0-9]+]]:gpr64 = SUBSXrr [[COPY]], [[MOVi64imm]], implicit-def $nzcv
-    ; CHECK: [[CSINCWr:%[0-9]+]]:gpr32 = CSINCWr $wzr, $wzr, 10, implicit $nzcv
-    ; CHECK: [[DEF:%[0-9]+]]:gpr64all = IMPLICIT_DEF
-    ; CHECK: [[INSERT_SUBREG:%[0-9]+]]:gpr64 = INSERT_SUBREG [[DEF]], [[CSINCWr]], %subreg.sub_32
-    ; CHECK: [[ANDXri:%[0-9]+]]:gpr64sp = ANDXri [[INSERT_SUBREG]], 4096
-    ; CHECK: $x0 = COPY [[ANDXri]]
-    ; CHECK: RET_ReallyLR implicit $x0
-    %0:gpr(s64) = COPY $x0
-    %1:gpr(s64) = G_CONSTANT i64 -9223372036854775808
-    %4:gpr(s32) = G_ICMP intpred(slt), %0(s64), %1
-    %6:gpr(s64) = G_ANYEXT %4(s32)
-    %5:gpr(s64) = G_CONSTANT i64 1
-    %3:gpr(s64) = G_AND %6, %5
-    $x0 = COPY %3(s64)
-    RET_ReallyLR implicit $x0
-
-...
----
-name:            no_opt_int32_max
-alignment:       4
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $w0
-
-    ; This one should contain a MOV.
-    ;
-    ; If we add 1 to the constant, it will wrap around, and so it's not true
-    ; that
-    ;
-    ; x slt c => x sle c - 1
-    ; x sge c => x sgt c - 1
-
-    ; CHECK-LABEL: name: no_opt_int32_max
-    ; CHECK: liveins: $w0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr32 = COPY $w0
-    ; CHECK: [[MOVi32imm:%[0-9]+]]:gpr32 = MOVi32imm 2147483647
-    ; CHECK: [[SUBSWrr:%[0-9]+]]:gpr32 = SUBSWrr [[COPY]], [[MOVi32imm]], implicit-def $nzcv
-    ; CHECK: [[CSINCWr:%[0-9]+]]:gpr32 = CSINCWr $wzr, $wzr, 12, implicit $nzcv
-    ; CHECK: [[ANDWri:%[0-9]+]]:gpr32sp = ANDWri [[CSINCWr]], 0
-    ; CHECK: $w0 = COPY [[ANDWri]]
-    ; CHECK: RET_ReallyLR implicit $w0
-    %0:gpr(s32) = COPY $w0
-    %1:gpr(s32) = G_CONSTANT i32 2147483647
-    %4:gpr(s32) = G_ICMP intpred(sle), %0(s32), %1
-    %5:gpr(s32) = G_CONSTANT i32 1
-    %3:gpr(s32) = G_AND %4, %5
-    $w0 = COPY %3(s32)
-    RET_ReallyLR implicit $w0
-
-...
----
-name:            no_opt_int64_max
-alignment:       4
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $x0
-
-    ; This one should contain a MOV.
-    ;
-    ; If we add 1 to the constant, it will wrap around, and so it's not true
-    ; that
-    ;
-    ; x slt c => x sle c - 1
-    ; x sge c => x sgt c - 1
-
-
-    ; CHECK-LABEL: name: no_opt_int64_max
-    ; CHECK: liveins: $x0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr64 = COPY $x0
-    ; CHECK: [[MOVi64imm:%[0-9]+]]:gpr64 = MOVi64imm 9223372036854775807
-    ; CHECK: [[SUBSXrr:%[0-9]+]]:gpr64 = SUBSXrr [[COPY]], [[MOVi64imm]], implicit-def $nzcv
-    ; CHECK: [[CSINCWr:%[0-9]+]]:gpr32 = CSINCWr $wzr, $wzr, 12, implicit $nzcv
-    ; CHECK: [[DEF:%[0-9]+]]:gpr64all = IMPLICIT_DEF
-    ; CHECK: [[INSERT_SUBREG:%[0-9]+]]:gpr64 = INSERT_SUBREG [[DEF]], [[CSINCWr]], %subreg.sub_32
-    ; CHECK: [[ANDXri:%[0-9]+]]:gpr64sp = ANDXri [[INSERT_SUBREG]], 4096
-    ; CHECK: $x0 = COPY [[ANDXri]]
-    ; CHECK: RET_ReallyLR implicit $x0
-    %0:gpr(s64) = COPY $x0
-    %1:gpr(s64) = G_CONSTANT i64 9223372036854775807
-    %4:gpr(s32) = G_ICMP intpred(sle), %0(s64), %1
-    %6:gpr(s64) = G_ANYEXT %4(s32)
-    %5:gpr(s64) = G_CONSTANT i64 1
-    %3:gpr(s64) = G_AND %6, %5
-    $x0 = COPY %3(s64)
-    RET_ReallyLR implicit $x0
-
-...
----
-name:            no_opt_zero
-alignment:       4
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $x0
-
-    ; This one should contain a MOV.
-    ;
-    ; This is an unsigned comparison, so when the constant is 0, the following
-    ; does not hold:
-    ;
-    ; x slt c => x sle c - 1
-    ; x sge c => x sgt c - 1
-
-    ; CHECK-LABEL: name: no_opt_zero
-    ; CHECK: liveins: $x0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr64sp = COPY $x0
-    ; CHECK: [[SUBSXri:%[0-9]+]]:gpr64 = SUBSXri [[COPY]], 0, 0, implicit-def $nzcv
-    ; CHECK: [[CSINCWr:%[0-9]+]]:gpr32 = CSINCWr $wzr, $wzr, 2, implicit $nzcv
-    ; CHECK: [[DEF:%[0-9]+]]:gpr64all = IMPLICIT_DEF
-    ; CHECK: [[INSERT_SUBREG:%[0-9]+]]:gpr64 = INSERT_SUBREG [[DEF]], [[CSINCWr]], %subreg.sub_32
-    ; CHECK: [[ANDXri:%[0-9]+]]:gpr64sp = ANDXri [[INSERT_SUBREG]], 4096
-    ; CHECK: $x0 = COPY [[ANDXri]]
-    ; CHECK: RET_ReallyLR implicit $x0
-    %0:gpr(s64) = COPY $x0
-    %1:gpr(s64) = G_CONSTANT i64 0
-    %4:gpr(s32) = G_ICMP intpred(ult), %0(s64), %1
-    %6:gpr(s64) = G_ANYEXT %4(s32)
-    %5:gpr(s64) = G_CONSTANT i64 1
-    %3:gpr(s64) = G_AND %6, %5
-    $x0 = COPY %3(s64)
-    RET_ReallyLR implicit $x0
-
-...
----
-name:            more_than_one_use_select
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $x0, $x1, $x2
-
-    ; Both of these selects use the same compare.
-    ;
-    ; They should both be optimized in the same way, so the SUBS produced for
-    ; each CSEL should be the same.
-
-    ; CHECK-LABEL: name: more_than_one_use_select
-    ; CHECK: liveins: $x0, $x1, $x2
-    ; CHECK: %a:gpr64common = COPY $x0
-    ; CHECK: %b:gpr64 = COPY $x1
-    ; CHECK: %c:gpr64 = COPY $x2
-    ; CHECK: [[SUBSXri:%[0-9]+]]:gpr64 = SUBSXri %a, 0, 0, implicit-def $nzcv
-    ; CHECK: %select1:gpr64 = CSELXr %a, %b, 11, implicit $nzcv
-    ; CHECK: [[SUBSXri1:%[0-9]+]]:gpr64 = SUBSXri %a, 0, 0, implicit-def $nzcv
-    ; CHECK: %select2:gpr64 = CSELXr %b, %c, 11, implicit $nzcv
-    ; CHECK: %add:gpr64 = ADDXrr %select1, %select2
-    ; CHECK: $x0 = COPY %add
-    ; CHECK: RET_ReallyLR implicit $x0
-    %a:gpr(s64) = COPY $x0
-    %b:gpr(s64) = COPY $x1
-    %c:gpr(s64) = COPY $x2
-    %cst:gpr(s64) = G_CONSTANT i64 -1
-    %cmp:gpr(s32) = G_ICMP intpred(sle), %a(s64), %cst
-    %trunc_cmp:gpr(s1) = G_TRUNC %cmp(s32)
-    %select1:gpr(s64) = G_SELECT %trunc_cmp(s1), %a, %b
-    %select2:gpr(s64) = G_SELECT %trunc_cmp(s1), %b, %c
-    %add:gpr(s64) = G_ADD %select1, %select2
-    $x0 = COPY %add(s64)
-    RET_ReallyLR implicit $x0
-...
----
-name:            more_than_one_use_select_no_opt
-legalized:       true
-regBankSelected: true
-tracksRegLiveness: true
-body:             |
-  bb.0:
-    liveins: $x0, $x1, $x2
-
-    ; When we don't end up doing the optimization, we should not change the
-    ; predicate.
-    ;
-    ; In this case, the CSELXrs should both have predicate code 13.
-
-    ; CHECK-LABEL: name: more_than_one_use_select_no_opt
-    ; CHECK: liveins: $x0, $x1, $x2
-    ; CHECK: %a:gpr64 = COPY $x0
-    ; CHECK: %b:gpr64 = COPY $x1
-    ; CHECK: %c:gpr64 = COPY $x2
-    ; CHECK: %cst:gpr64 = MOVi64imm 922337203685477580
-    ; CHECK: [[SUBSXrr:%[0-9]+]]:gpr64 = SUBSXrr %a, %cst, implicit-def $nzcv
-    ; CHECK: %select1:gpr64 = CSELXr %a, %b, 13, implicit $nzcv
-    ; CHECK: [[SUBSXrr1:%[0-9]+]]:gpr64 = SUBSXrr %a, %cst, implicit-def $nzcv
-    ; CHECK: %select2:gpr64 = CSELXr %b, %c, 13, implicit $nzcv
-    ; CHECK: %add:gpr64 = ADDXrr %select1, %select2
-    ; CHECK: $x0 = COPY %add
-    ; CHECK: RET_ReallyLR implicit $x0
-    %a:gpr(s64) = COPY $x0
-    %b:gpr(s64) = COPY $x1
-    %c:gpr(s64) = COPY $x2
-    %cst:gpr(s64) = G_CONSTANT i64 922337203685477580
-    %cmp:gpr(s32) = G_ICMP intpred(sle), %a(s64), %cst
-    %trunc_cmp:gpr(s1) = G_TRUNC %cmp(s32)
-    %select1:gpr(s64) = G_SELECT %trunc_cmp(s1), %a, %b
-    %select2:gpr(s64) = G_SELECT %trunc_cmp(s1), %b, %c
-    %add:gpr(s64) = G_ADD %select1, %select2
-    $x0 = COPY %add(s64)
-    RET_ReallyLR implicit $x0
-...

llvm/test/CodeGen/AArch64/GlobalISel/select-cbz.mir

body:             |
 
   bb.2:
     %6:gpr(s64) = G_CONSTANT i64 0
     G_STORE %6(s64), %0(p0) :: (store 8)
 
   bb.3:
     RET_ReallyLR
 
-...
----
-name:            update_pred_minus_one
-legalized:       true
-regBankSelected: true
 
-body:             |
-  ; CHECK-LABEL: name: update_pred_minus_one
-  ; CHECK: bb.0:
-  ; CHECK:   successors: %bb.0(0x40000000), %bb.1(0x40000000)
-  ; CHECK:   [[COPY:%[0-9]+]]:gpr32sp = COPY $w0
-  ; CHECK:   [[SUBSWri:%[0-9]+]]:gpr32 = SUBSWri [[COPY]], 0, 0, implicit-def $nzcv
-  ; CHECK:   Bcc 11, %bb.1, implicit $nzcv
-  ; CHECK:   B %bb.0
-  ; CHECK: bb.1:
-  ; The G_ICMP here will be optimized into a slt against 0.
-  ; The branch should inherit this change, so we should have Bcc 11 rather than
-  ; Bcc 13.
-
-  bb.0:
-    liveins: $w0
-    successors: %bb.0, %bb.1
-
-    %0:gpr(s32) = COPY $w0
-    %1:gpr(s32) = G_CONSTANT i32 -1
-    %2:gpr(s32) = G_ICMP intpred(sle), %0, %1
-    %3:gpr(s1) = G_TRUNC %2(s32)
-    G_BRCOND %3(s1), %bb.1
-    G_BR %bb.0
-
-  bb.1:
-...

llvm/test/CodeGen/AArch64/GlobalISel/tbz-sgt.mir

 alignment:       4
 legalized:       true
 regBankSelected: true
 body:             |
   ; CHECK-LABEL: name: dont_fold_and
   ; CHECK: bb.0:
   ; CHECK:   successors: %bb.0(0x40000000), %bb.1(0x40000000)
   ; CHECK:   %copy:gpr64 = COPY $x0
-  ; CHECK:   %and:gpr64sp = ANDXri %copy, 8000
-  ; CHECK:   [[SUBSXri:%[0-9]+]]:gpr64 = SUBSXri %and, 0, 0, implicit-def $nzcv
-  ; CHECK:   Bcc 10, %bb.1, implicit $nzcv
+  ; CHECK:   %negative_one:gpr64 = MOVi64imm -1
+  ; CHECK:   %and:gpr64common = ANDXri %copy, 8000
+  ; CHECK:   [[SUBSXrr:%[0-9]+]]:gpr64 = SUBSXrr %and, %negative_one, implicit-def $nzcv
+  ; CHECK:   Bcc 12, %bb.1, implicit $nzcv
   ; CHECK:   B %bb.0
   ; CHECK: bb.1:
   ; CHECK:   RET_ReallyLR
   bb.0:
     successors: %bb.0, %bb.1
     liveins: $x0
     %copy:gpr(s64) = COPY $x0
     %bit:gpr(s64) = G_CONSTANT i64 8