[LoopPredication] Widen checks if condition operands constant ranges are known

[dmakogon]

Summary

This enables LoopPredication to widen checks of the following form:

x < guardLimit,

where x and guardLimit have known constant ranges (meaning they are not
full-set). If they are reasonably small and max(x) <= max(guardLimit), we
can predicate the loop with the condition max(x) <= guardLimit.

[mkazantsev]

llvm/lib/Transforms/Scalar/LoopPredication.cpp

780

Need to make it clear that Guard always belongs to loop, but here we are filtering away guards from nested loops. Assert on L.contains(Guard->getParent())?

800

Isn't [this] enough?

Can it be const Value and const Instruction?

811

Not very informative, can we instead print one message, in the end, and make it verbose? Like, "Speculating LHS = ... less than RHS = ... because max(LHS) < max(RHS)" or something?

843

if (IsSigned)
  LHSMaxIsGreaterThanRHSMax = LHSMaxValue.sgt(RHSMaxValue);
else
  HSMaxIsGreaterThanRHSMax = LHSMaxValue.ugt(RHSMaxValue);

?

897

Should we have a separate statistics for that?

[yrouban]

llvm/lib/Transforms/Scalar/LoopPredication.cpp

831

do it right after RHSRange = ComputeRange() and only then calculate LHSRange.

843

or

bool LHSMaxIsGreaterThanRHSMax = IsSigned
                               ? LHSMaxValue.sgt(RHSMaxValue)
                               : LHSMaxValue.ugt(RHSMaxValue);

or even a bigger change without introduced LHSMaxValue and RHSMaxValue:

bool IsGreaterThanRHSMax = IsSigned
                         ? LHSRange.getSignedMax().sgt(RHSRange.getSignedMax())
                         : LHSRange.getUnsignedMax().ugt(RHSRange.getUnsignedMax());

[apilipenko]

llvm/lib/Transforms/Scalar/LoopPredication.cpp

251–252

401–402

This is a non-canonical way to get an analysis. This way it will not benefit from analysis manager caching. I guess you have to do this because LVI is a function analysis but a part of LoopStandardAnalysisResults.

I'm not sure how we should go about it, but I suggest splitting the LVI off from the initial patch. You can still get the constant range from ValueTracking to get the basic version of this optimization.

808–822

Please, spell the list of supported predicated explicitly here.

841

Why do you need to check this?

[dmakogon]

Addressed comments

[dmakogon]

llvm/lib/Transforms/Scalar/LoopPredication.cpp

401–402

Removed LVI from this patch. Please note the test changes. I explicitly inserted assumes of dominating branches conditions to allow for the widening to happen

[apilipenko]

llvm/lib/Transforms/Scalar/LoopPredication.cpp

808–822

Could you please, spell the list of *supported* predicates explicitly?

[dmakogon]

Rebase, address comments

[dmakogon]

llvm/lib/Transforms/Scalar/LoopPredication.cpp

841

Well, this is the speculation condition we decided to use. The idea here is the following:
Assume LHS is an index and RHS is array length.
So we'd like to speculate only if the index possible maximum value isn't greater that the array length maximum value.
Otherwise we assume that index would take the value greater than the array length in most cases, so we'd deoptimize.
We don't have exact statistics that such cases prevail, it's just our expectations I guess. Also here we cut off the case when LHS has a full-set range.

[mkazantsev]

llvm/lib/Transforms/Scalar/LoopPredication.cpp

820

In the switch above:

  case CmpInst::ICMP_SGE:
  case CmpInst::ICMP_UGT:
  case CmpInst::ICMP_UGE:
  case CmpInst::ICMP_SGT:
    std::swap(LHS, RHS);
    Pred = ICmpInst::getSwappedPredicate(Pred);
    LLVM_FALLTHROUGH
// Allowed predicates:
    break;
default:
  return nullptr

[mkazantsev]

Marking to remove it from my review list, please update if you have plans on it.

llvm/lib/Transforms/Scalar/LoopPredication.cpp

 //     guardStart u< guardLimit && latchLimit u> 1.
 //   For sge condition the widened condition is:
 //     guardStart u< guardLimit && latchLimit s> 1.
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Transforms/Scalar/LoopPredication.h"
 #include "llvm/ADT/Statistic.h"
 #include "llvm/Analysis/AliasAnalysis.h"
+#include "llvm/Analysis/AssumptionCache.h"
 #include "llvm/Analysis/BranchProbabilityInfo.h"
 #include "llvm/Analysis/GuardUtils.h"
 #include "llvm/Analysis/LoopInfo.h"
 #include "llvm/Analysis/LoopPass.h"
 #include "llvm/Analysis/MemorySSA.h"
 #include "llvm/Analysis/MemorySSAUpdater.h"
 #include "llvm/Analysis/ScalarEvolution.h"
 #include "llvm/Analysis/ScalarEvolutionExpressions.h"
+#include "llvm/Analysis/ValueTracking.h"
 #include "llvm/IR/Function.h"
+#include "llvm/IR/Instructions.h"
 #include "llvm/IR/IntrinsicInst.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/PatternMatch.h"
 #include "llvm/IR/ProfDataUtils.h"
 #include "llvm/InitializePasses.h"
 #include "llvm/Pass.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Debug.h"
+#include "llvm/Support/KnownBits.h"
 #include "llvm/Transforms/Scalar.h"
 #include "llvm/Transforms/Utils/GuardUtils.h"
 #include "llvm/Transforms/Utils/Local.h"
 #include "llvm/Transforms/Utils/LoopUtils.h"
 #include "llvm/Transforms/Utils/ScalarEvolutionExpander.h"
 #include <optional>
 
 #define DEBUG_TYPE "loop-predication"
 
 STATISTIC(TotalConsidered, "Number of guards considered");
 STATISTIC(TotalWidened, "Number of checks widened");
+STATISTIC(NumWidenedUsingConstantRanges,
+          "Number of checks widened using constant ranges analysis");
 
 using namespace llvm;
 
 static cl::opt<bool> EnableIVTruncation("loop-predication-enable-iv-truncation",
                                         cl::Hidden, cl::init(true));
 
 static cl::opt<bool> EnableCountDownLoop("loop-predication-enable-count-down-loop",
                                         cl::Hidden, cl::init(true));

 
 static cl::opt<bool> InsertAssumesOfPredicatedGuardsConditions(
     "loop-predication-insert-assumes-of-predicated-guards-conditions",
     cl::Hidden,
     cl::desc("Whether or not we should insert assumes of conditions of "
              "predicated guards"),
     cl::init(true));
 
+static cl::opt<bool> WidenChecksUsingConstantRanges(
+    "loop-predication-widen-checks-using-contant-ranges", cl::Hidden,
+    cl::desc(
+        "Whether or not we should widen checks using constant range analysis"),
+    cl::init(true));
+
 namespace {
 /// Represents an induction variable check:
 ///   icmp Pred, <induction variable>, <loop invariant limit>
 struct LoopICmp {
   ICmpInst::Predicate Pred;
   const SCEVAddRecExpr *IV;
   const SCEV *Limit;
   LoopICmp(ICmpInst::Predicate Pred, const SCEVAddRecExpr *IV,
            const SCEV *Limit)
     : Pred(Pred), IV(IV), Limit(Limit) {}
   LoopICmp() = default;
   void dump() {
     dbgs() << "LoopICmp Pred = " << Pred << ", IV = " << *IV
            << ", Limit = " << *Limit << "\n";
   }
 };
 
 class LoopPredication {
   AliasAnalysis *AA;
   DominatorTree *DT;
   ScalarEvolution *SE;
   LoopInfo *LI;
   MemorySSAUpdater *MSSAU;
+  AssumptionCache *AC;
 
   Loop *L;
   const DataLayout *DL;
   BasicBlock *Preheader;
   LoopICmp LatchCheck;
 
   bool isSupportedStep(const SCEV* Step);
   std::optional<LoopICmp> parseLoopICmp(ICmpInst *ICI);
class LoopPredication {
 
   Value *expandCheck(SCEVExpander &Expander, Instruction *Guard,
                      ICmpInst::Predicate Pred, const SCEV *LHS,
                      const SCEV *RHS);
 
   std::optional<Value *> widenICmpRangeCheck(ICmpInst *ICI,
                                              SCEVExpander &Expander,
                                              Instruction *Guard);
+
+  std::optional<Value *> widenRangeCheckUsingConstantRange(ICmpInst *ICI,
+                                                           Instruction *Guard);
+
   std::optional<Value *>
   widenICmpRangeCheckIncrementingLoop(LoopICmp LatchCheck, LoopICmp RangeCheck,
                                       SCEVExpander &Expander,
                                       Instruction *Guard);
+
   std::optional<Value *>
   widenICmpRangeCheckDecrementingLoop(LoopICmp LatchCheck, LoopICmp RangeCheck,
                                       SCEVExpander &Expander,
                                       Instruction *Guard);
+
   unsigned collectChecks(SmallVectorImpl<Value *> &Checks, Value *Condition,
                          SCEVExpander &Expander, Instruction *Guard);
   bool widenGuardConditions(IntrinsicInst *II, SCEVExpander &Expander);
   bool widenWidenableBranchGuardConditions(BranchInst *Guard, SCEVExpander &Expander);
   // If the loop always exits through another block in the loop, we should not
   // predicate based on the latch check. For example, the latch check can be a
   // very coarse grained check and there can be more fine grained exit checks
   // within the loop.
   bool isLoopProfitableToPredicate();
 
   bool predicateLoopExits(Loop *L, SCEVExpander &Rewriter);
 
 public:
   LoopPredication(AliasAnalysis *AA, DominatorTree *DT, ScalarEvolution *SE,
-                  LoopInfo *LI, MemorySSAUpdater *MSSAU)
-      : AA(AA), DT(DT), SE(SE), LI(LI), MSSAU(MSSAU){};
+                  LoopInfo *LI, MemorySSAUpdater *MSSAU, AssumptionCache *AC)
+      : AA(AA), DT(DT), SE(SE), LI(LI), MSSAU(MSSAU), AC(AC){};
   bool runOnLoop(Loop *L);
 };
 
 class LoopPredicationLegacyPass : public LoopPass {
 public:
   static char ID;
   LoopPredicationLegacyPass() : LoopPass(ID) {
     initializeLoopPredicationLegacyPassPass(*PassRegistry::getPassRegistry());
   }
 
   void getAnalysisUsage(AnalysisUsage &AU) const override {
     AU.addRequired<BranchProbabilityInfoWrapperPass>();
     getLoopAnalysisUsage(AU);
     AU.addPreserved<MemorySSAWrapperPass>();
   }
 
   bool runOnLoop(Loop *L, LPPassManager &LPM) override {
     if (skipLoop(L))
       return false;
+    Function &F = *L->getHeader()->getParent();
+
     auto *SE = &getAnalysis<ScalarEvolutionWrapperPass>().getSE();
     auto *LI = &getAnalysis<LoopInfoWrapperPass>().getLoopInfo();
     auto *DT = &getAnalysis<DominatorTreeWrapperPass>().getDomTree();
     auto *MSSAWP = getAnalysisIfAvailable<MemorySSAWrapperPass>();
     std::unique_ptr<MemorySSAUpdater> MSSAU;
     if (MSSAWP)
       MSSAU = std::make_unique<MemorySSAUpdater>(&MSSAWP->getMSSA());
     auto *AA = &getAnalysis<AAResultsWrapperPass>().getAAResults();
-    LoopPredication LP(AA, DT, SE, LI, MSSAU ? MSSAU.get() : nullptr);
+    auto *AC = &getAnalysis<AssumptionCacheTracker>().getAssumptionCache(F);
+    LoopPredication LP(AA, DT, SE, LI, MSSAU ? MSSAU.get() : nullptr, AC);
     return LP.runOnLoop(L);
   }
 };
 
 char LoopPredicationLegacyPass::ID = 0;
 } // end namespace
 
 INITIALIZE_PASS_BEGIN(LoopPredicationLegacyPass, "loop-predication",

 
 PreservedAnalyses LoopPredicationPass::run(Loop &L, LoopAnalysisManager &AM,
                                            LoopStandardAnalysisResults &AR,
                                            LPMUpdater &U) {
   std::unique_ptr<MemorySSAUpdater> MSSAU;
   if (AR.MSSA)
     MSSAU = std::make_unique<MemorySSAUpdater>(AR.MSSA);
   LoopPredication LP(&AR.AA, &AR.DT, &AR.SE, &AR.LI,
-                     MSSAU ? MSSAU.get() : nullptr);
+                     MSSAU ? MSSAU.get() : nullptr, &AR.AC);
   if (!LP.runOnLoop(&L))
     return PreservedAnalyses::all();
 
   auto PA = getLoopPassPreservedAnalyses();
   if (AR.MSSA)
     PA.preserve<MemorySSAAnalysis>();
   return PA;
 }
 
return widenICmpRangeCheckIncrementingLoop(CurrLatchCheck, *RangeCheck,
                                                Expander, Guard);
   else {
     assert(Step->isAllOnesValue() && "Step should be -1!");
     return widenICmpRangeCheckDecrementingLoop(CurrLatchCheck, *RangeCheck,
                                                Expander, Guard);
   }
 }
 
+std::optional<Value *>
+LoopPredication::widenRangeCheckUsingConstantRange(ICmpInst *ICI,
+                                                   Instruction *Guard) {
+  if (!WidenChecksUsingConstantRanges || !L->contains(Guard->getParent()))
+    return std::nullopt;
+  LLVM_DEBUG(dbgs() << "Analyzing ICmpInst condition using constant range:\n");
+  LLVM_DEBUG(ICI->dump());
+
+  // We are looking for the range checks of the form:
+  //   x < guardLimit,
+  // where x and guardLimit have known constant ranges (meaning they are not
+  // full-set). If they are reasonably small and max(x) <= max(guardLimit), we
+  // can predicate the loop with the condition max(x) <= guardLimit.
+  auto Pred = ICI->getPredicate();
+  auto *LHS = ICI->getOperand(0);
+  auto *RHS = ICI->getOperand(1);
+
+  // Both operands are loop variant, bail out.
+  if (!L->isLoopInvariant(LHS) && !L->isLoopInvariant(RHS)) {
+    LLVM_DEBUG(dbgs() << "LHS and RHS are loop variant\n");
+    return std::nullopt;
+  }
+
+  auto ComputeRange = [this](const Value *V, bool IsSigned,
+                             const Instruction *CtxI) {
+    auto Range = ConstantRange::fromKnownBits(
+        computeKnownBits(V, *DL, 0, AC, CtxI, DT), IsSigned);
+    return Range.intersectWith(
+        computeConstantRange(V, IsSigned, true, AC, CtxI, DT));
+  };
+
+  switch (Pred) {
+  case CmpInst::ICMP_UGT:
+  case CmpInst::ICMP_UGE:
+  case CmpInst::ICMP_SGT:
+  case CmpInst::ICMP_SGE:
+    std::swap(LHS, RHS);
+    Pred = ICmpInst::getSwappedPredicate(Pred);
+    break;
+  default:
+    break;
+  }
+
+  // We don't support equality predicates currently.
+  if (Pred != CmpInst::ICMP_ULT && Pred != CmpInst::ICMP_ULE &&
+      Pred != CmpInst::ICMP_SLT && Pred != CmpInst::ICMP_SLE)
+    return std::nullopt;
+
+  bool IsSigned = ICmpInst::isSigned(Pred);
+  auto RHSRange = ComputeRange(RHS, IsSigned, Guard);
+  if (RHSRange.isFullSet()) {
+    LLVM_DEBUG(dbgs() << "RHS range is full-set\n");
+    return std::nullopt;
+  }
+
+  auto LHSRange = ComputeRange(LHS, IsSigned, Guard);
+
+  auto LHSMaxValue =
+      IsSigned ? LHSRange.getSignedMax() : LHSRange.getUnsignedMax();
+  bool LHSMaxIsGreaterThanRHSMax =
+      IsSigned ? LHSMaxValue.sgt(RHSRange.getSignedMax())
+               : LHSMaxValue.ugt(RHSRange.getUnsignedMax());
+
+  if (LHSMaxIsGreaterThanRHSMax) {
+    LLVM_DEBUG(dbgs() << "LHS max value is not greater than RHS maximum\n");
+    return std::nullopt;
+  }
+
+  ICmpInst::Predicate NewPred = ICmpInst::getNonStrictPredicate(Pred);
+  LLVM_DEBUG(dbgs() << "Speculating " << LHSMaxValue << " "
+                    << ICmpInst::getPredicateName(NewPred) << " " << *RHS
+                    << " because max(LHS) < max(RHS)\n");
+  auto *MaxCI = ConstantInt::get(LHS->getType(), LHSMaxValue);
+  IRBuilder<> Builder(findInsertPt(Guard, {RHS}));
+  auto *NewICI = Builder.CreateICmp(NewPred, MaxCI, RHS);
+  return NewICI;
+}
+
 unsigned LoopPredication::collectChecks(SmallVectorImpl<Value *> &Checks,
                                         Value *Condition,
                                         SCEVExpander &Expander,
                                         Instruction *Guard) {
   unsigned NumWidened = 0;
   // The guard condition is expected to be in form of:
   //   cond1 && cond2 && cond3 ...
   // Iterate over subconditions looking for icmp conditions which can be
do {
 
     if (ICmpInst *ICI = dyn_cast<ICmpInst>(Condition)) {
       if (auto NewRangeCheck = widenICmpRangeCheck(ICI, Expander,
                                                    Guard)) {
         Checks.push_back(*NewRangeCheck);
         NumWidened++;
         continue;
       }
+      if (auto RC = widenRangeCheckUsingConstantRange(ICI, Guard)) {
+        Checks.push_back(*RC);
+        NumWidened++;
+        NumWidenedUsingConstantRanges++;
+        continue;
+      }
     }
 
     // Save the condition as is if we can't widen it
     Checks.push_back(Condition);
   } while (!Worklist.empty());
   // At the moment, our matching logic for wideable conditions implicitly
   // assumes we preserve the form: (br (and Cond, WC())).  FIXME
   // Note that if there were multiple calls to wideable condition in the

llvm/test/Transforms/LoopPredication/constant-ranges.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -S -passes='loop-mssa(loop-predication)' -loop-predication-insert-assumes-of-predicated-guards-conditions=false < %s 2>&1 | FileCheck %s
+; RUN: opt -S -passes='loop-mssa(loop-predication)' -loop-predication-widen-checks-using-contant-ranges=true -loop-predication-insert-assumes-of-predicated-guards-conditions=false < %s 2>&1 | FileCheck %s
 
 define i32 @test_ult(ptr noundef %p, i32 noundef %n, ptr noundef %arr, ptr noundef %len_p, i32 noundef %c) {
 ; CHECK-LABEL: @test_ult(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, ptr [[LEN_P:%.*]], align 4, !range [[RNG0:![0-9]+]], !noundef !1
+; CHECK-NEXT:    [[TMP0:%.*]] = icmp ule i32 127, [[LEN]]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[BACKEDGE:%.*]] ]
 ; CHECK-NEXT:    [[EL_PTR:%.*]] = getelementptr i32, ptr [[P:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[EL:%.*]] = load i32, ptr [[EL_PTR]], align 4
 ; CHECK-NEXT:    [[BOUND_CHECK:%.*]] = icmp ult i32 [[EL]], 128
 ; CHECK-NEXT:    br i1 [[BOUND_CHECK]], label [[DO_RANGE_CHECK:%.*]], label [[BOUND_CHECK_FAILED:%.*]]
 ; CHECK:       do_range_check:
 ; CHECK-NEXT:    call void @llvm.assume(i1 [[BOUND_CHECK]])
-; CHECK-NEXT:    [[RANGE_CHECK_ULT:%.*]] = icmp ult i32 [[EL]], [[LEN]]
 ; CHECK-NEXT:    [[WC:%.*]] = call i1 @llvm.experimental.widenable.condition()
-; CHECK-NEXT:    [[EXPLICIT_GUARD_COND:%.*]] = and i1 [[RANGE_CHECK_ULT]], [[WC]]
-; CHECK-NEXT:    br i1 [[EXPLICIT_GUARD_COND]], label [[BACKEDGE]], label [[DEOPT:%.*]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i1 [[TMP0]], [[WC]]
+; CHECK-NEXT:    br i1 [[TMP1]], label [[BACKEDGE]], label [[DEOPT:%.*]]
 ; CHECK:       backedge:
 ; CHECK-NEXT:    [[ARR_PTR:%.*]] = getelementptr i32, ptr [[ARR:%.*]], i32 [[EL]]
 ; CHECK-NEXT:    store i32 [[IV]], ptr [[ARR_PTR]], align 4
 ; CHECK-NEXT:    [[IV_NEXT]] = add i32 [[IV]], 1
 ; CHECK-NEXT:    [[LOOP_COND:%.*]] = icmp slt i32 [[IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       deopt:
 ; CHECK-NEXT:    [[DEOPT_RES:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"() ]

 bound_check_failed:
   ret i32 -1
 }
 
 define i32 @test_slt(ptr noundef %p, i32 noundef %n, ptr noundef %arr, ptr noundef %len_p, i32 noundef %c) {
 ; CHECK-LABEL: @test_slt(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, ptr [[LEN_P:%.*]], align 4, !range [[RNG0]], !noundef !1
+; CHECK-NEXT:    [[TMP0:%.*]] = icmp sle i32 127, [[LEN]]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[BACKEDGE:%.*]] ]
 ; CHECK-NEXT:    [[EL_PTR:%.*]] = getelementptr i32, ptr [[P:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[EL:%.*]] = load i32, ptr [[EL_PTR]], align 4
 ; CHECK-NEXT:    [[BOUND_CHECK:%.*]] = icmp slt i32 [[EL]], 128
 ; CHECK-NEXT:    br i1 [[BOUND_CHECK]], label [[DO_RANGE_CHECK:%.*]], label [[BOUND_CHECK_FAILED:%.*]]
 ; CHECK:       do_range_check:
 ; CHECK-NEXT:    call void @llvm.assume(i1 [[BOUND_CHECK]])
-; CHECK-NEXT:    [[RANGE_CHECK_SLT:%.*]] = icmp slt i32 [[EL]], [[LEN]]
 ; CHECK-NEXT:    [[WC:%.*]] = call i1 @llvm.experimental.widenable.condition()
-; CHECK-NEXT:    [[EXPLICIT_GUARD_COND:%.*]] = and i1 [[RANGE_CHECK_SLT]], [[WC]]
-; CHECK-NEXT:    br i1 [[EXPLICIT_GUARD_COND]], label [[BACKEDGE]], label [[DEOPT:%.*]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i1 [[TMP0]], [[WC]]
+; CHECK-NEXT:    br i1 [[TMP1]], label [[BACKEDGE]], label [[DEOPT:%.*]]
 ; CHECK:       backedge:
 ; CHECK-NEXT:    [[ARR_PTR:%.*]] = getelementptr i32, ptr [[ARR:%.*]], i32 [[EL]]
 ; CHECK-NEXT:    store i32 [[IV]], ptr [[ARR_PTR]], align 4
 ; CHECK-NEXT:    [[IV_NEXT]] = add i32 [[IV]], 1
 ; CHECK-NEXT:    [[LOOP_COND:%.*]] = icmp ult i32 [[IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       deopt:
 ; CHECK-NEXT:    [[DEOPT_RES:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"() ]

 bound_check_failed:
   ret i32 -1
 }
 
 define i32 @test_ule(ptr noundef %p, i32 noundef %n, ptr noundef %arr, ptr noundef %len_p, i32 noundef %c) {
 ; CHECK-LABEL: @test_ule(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, ptr [[LEN_P:%.*]], align 4, !range [[RNG0]], !noundef !1
+; CHECK-NEXT:    [[TMP0:%.*]] = icmp ule i32 127, [[LEN]]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[BACKEDGE:%.*]] ]
 ; CHECK-NEXT:    [[EL_PTR:%.*]] = getelementptr i32, ptr [[P:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[EL:%.*]] = load i32, ptr [[EL_PTR]], align 4
 ; CHECK-NEXT:    [[BOUND_CHECK:%.*]] = icmp ult i32 [[EL]], 128
 ; CHECK-NEXT:    br i1 [[BOUND_CHECK]], label [[DO_RANGE_CHECK:%.*]], label [[BOUND_CHECK_FAILED:%.*]]
 ; CHECK:       do_range_check:
 ; CHECK-NEXT:    call void @llvm.assume(i1 [[BOUND_CHECK]])
-; CHECK-NEXT:    [[RANGE_CHECK_ULE:%.*]] = icmp ule i32 [[EL]], [[LEN]]
 ; CHECK-NEXT:    [[WC:%.*]] = call i1 @llvm.experimental.widenable.condition()
-; CHECK-NEXT:    [[EXPLICIT_GUARD_COND:%.*]] = and i1 [[RANGE_CHECK_ULE]], [[WC]]
-; CHECK-NEXT:    br i1 [[EXPLICIT_GUARD_COND]], label [[BACKEDGE]], label [[DEOPT:%.*]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i1 [[TMP0]], [[WC]]
+; CHECK-NEXT:    br i1 [[TMP1]], label [[BACKEDGE]], label [[DEOPT:%.*]]
 ; CHECK:       backedge:
 ; CHECK-NEXT:    [[ARR_PTR:%.*]] = getelementptr i32, ptr [[ARR:%.*]], i32 [[EL]]
 ; CHECK-NEXT:    store i32 [[IV]], ptr [[ARR_PTR]], align 4
 ; CHECK-NEXT:    [[IV_NEXT]] = add i32 [[IV]], 1
 ; CHECK-NEXT:    [[LOOP_COND:%.*]] = icmp ult i32 [[IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       deopt:
 ; CHECK-NEXT:    [[DEOPT_RES:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"() ]

 bound_check_failed:
   ret i32 -1
 }
 
 define i32 @test_sle(ptr noundef %p, i32 noundef %n, ptr noundef %arr, ptr noundef %len_p, i32 noundef %c) {
 ; CHECK-LABEL: @test_sle(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, ptr [[LEN_P:%.*]], align 4, !range [[RNG0]], !noundef !1
+; CHECK-NEXT:    [[TMP0:%.*]] = icmp sle i32 127, [[LEN]]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[BACKEDGE:%.*]] ]
 ; CHECK-NEXT:    [[EL_PTR:%.*]] = getelementptr i32, ptr [[P:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[EL:%.*]] = load i32, ptr [[EL_PTR]], align 4
 ; CHECK-NEXT:    [[BOUND_CHECK:%.*]] = icmp slt i32 [[EL]], 128
 ; CHECK-NEXT:    br i1 [[BOUND_CHECK]], label [[DO_RANGE_CHECK:%.*]], label [[BOUND_CHECK_FAILED:%.*]]
 ; CHECK:       do_range_check:
 ; CHECK-NEXT:    call void @llvm.assume(i1 [[BOUND_CHECK]])
-; CHECK-NEXT:    [[RANGE_CHECK_SLE:%.*]] = icmp sle i32 [[EL]], [[LEN]]
 ; CHECK-NEXT:    [[WC:%.*]] = call i1 @llvm.experimental.widenable.condition()
-; CHECK-NEXT:    [[EXPLICIT_GUARD_COND:%.*]] = and i1 [[RANGE_CHECK_SLE]], [[WC]]
-; CHECK-NEXT:    br i1 [[EXPLICIT_GUARD_COND]], label [[BACKEDGE]], label [[DEOPT:%.*]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i1 [[TMP0]], [[WC]]
+; CHECK-NEXT:    br i1 [[TMP1]], label [[BACKEDGE]], label [[DEOPT:%.*]]
 ; CHECK:       backedge:
 ; CHECK-NEXT:    [[ARR_PTR:%.*]] = getelementptr i32, ptr [[ARR:%.*]], i32 [[EL]]
 ; CHECK-NEXT:    store i32 [[IV]], ptr [[ARR_PTR]], align 4
 ; CHECK-NEXT:    [[IV_NEXT]] = add i32 [[IV]], 1
 ; CHECK-NEXT:    [[LOOP_COND:%.*]] = icmp ult i32 [[IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       deopt:
 ; CHECK-NEXT:    [[DEOPT_RES:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"() ]

 bound_check_failed:
   ret i32 -1
 }
 
 define i32 @test_ugt(ptr noundef %p, i32 noundef %n, ptr noundef %arr, ptr noundef %len_p, i32 noundef %c) {
 ; CHECK-LABEL: @test_ugt(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, ptr [[LEN_P:%.*]], align 4, !range [[RNG0]], !noundef !1
+; CHECK-NEXT:    [[TMP0:%.*]] = icmp ule i32 127, [[LEN]]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[BACKEDGE:%.*]] ]
 ; CHECK-NEXT:    [[EL_PTR:%.*]] = getelementptr i32, ptr [[P:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[EL:%.*]] = load i32, ptr [[EL_PTR]], align 4
 ; CHECK-NEXT:    [[BOUND_CHECK:%.*]] = icmp ult i32 [[EL]], 128
 ; CHECK-NEXT:    br i1 [[BOUND_CHECK]], label [[DO_RANGE_CHECK:%.*]], label [[BOUND_CHECK_FAILED:%.*]]
 ; CHECK:       do_range_check:
 ; CHECK-NEXT:    call void @llvm.assume(i1 [[BOUND_CHECK]])
-; CHECK-NEXT:    [[RANGE_CHECK_SLE:%.*]] = icmp ugt i32 [[LEN]], [[EL]]
 ; CHECK-NEXT:    [[WC:%.*]] = call i1 @llvm.experimental.widenable.condition()
-; CHECK-NEXT:    [[EXPLICIT_GUARD_COND:%.*]] = and i1 [[RANGE_CHECK_SLE]], [[WC]]
-; CHECK-NEXT:    br i1 [[EXPLICIT_GUARD_COND]], label [[BACKEDGE]], label [[DEOPT:%.*]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i1 [[TMP0]], [[WC]]
+; CHECK-NEXT:    br i1 [[TMP1]], label [[BACKEDGE]], label [[DEOPT:%.*]]
 ; CHECK:       backedge:
 ; CHECK-NEXT:    [[ARR_PTR:%.*]] = getelementptr i32, ptr [[ARR:%.*]], i32 [[EL]]
 ; CHECK-NEXT:    store i32 [[IV]], ptr [[ARR_PTR]], align 4
 ; CHECK-NEXT:    [[IV_NEXT]] = add i32 [[IV]], 1
 ; CHECK-NEXT:    [[LOOP_COND:%.*]] = icmp ult i32 [[IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       deopt:
 ; CHECK-NEXT:    [[DEOPT_RES:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"() ]

 bound_check_failed:
   ret i32 -1
 }
 
 define i32 @test_uge(ptr noundef %p, i32 noundef %n, ptr noundef %arr, ptr noundef %len_p, i32 noundef %c) {
 ; CHECK-LABEL: @test_uge(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, ptr [[LEN_P:%.*]], align 4, !range [[RNG0]], !noundef !1
+; CHECK-NEXT:    [[TMP0:%.*]] = icmp ule i32 127, [[LEN]]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[BACKEDGE:%.*]] ]
 ; CHECK-NEXT:    [[EL_PTR:%.*]] = getelementptr i32, ptr [[P:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[EL:%.*]] = load i32, ptr [[EL_PTR]], align 4
 ; CHECK-NEXT:    [[BOUND_CHECK:%.*]] = icmp ult i32 [[EL]], 128
 ; CHECK-NEXT:    br i1 [[BOUND_CHECK]], label [[DO_RANGE_CHECK:%.*]], label [[BOUND_CHECK_FAILED:%.*]]
 ; CHECK:       do_range_check:
 ; CHECK-NEXT:    call void @llvm.assume(i1 [[BOUND_CHECK]])
-; CHECK-NEXT:    [[RANGE_CHECK_SLE:%.*]] = icmp uge i32 [[LEN]], [[EL]]
 ; CHECK-NEXT:    [[WC:%.*]] = call i1 @llvm.experimental.widenable.condition()
-; CHECK-NEXT:    [[EXPLICIT_GUARD_COND:%.*]] = and i1 [[RANGE_CHECK_SLE]], [[WC]]
-; CHECK-NEXT:    br i1 [[EXPLICIT_GUARD_COND]], label [[BACKEDGE]], label [[DEOPT:%.*]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i1 [[TMP0]], [[WC]]
+; CHECK-NEXT:    br i1 [[TMP1]], label [[BACKEDGE]], label [[DEOPT:%.*]]
 ; CHECK:       backedge:
 ; CHECK-NEXT:    [[ARR_PTR:%.*]] = getelementptr i32, ptr [[ARR:%.*]], i32 [[EL]]
 ; CHECK-NEXT:    store i32 [[IV]], ptr [[ARR_PTR]], align 4
 ; CHECK-NEXT:    [[IV_NEXT]] = add i32 [[IV]], 1
 ; CHECK-NEXT:    [[LOOP_COND:%.*]] = icmp ult i32 [[IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       deopt:
 ; CHECK-NEXT:    [[DEOPT_RES:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"() ]

 bound_check_failed:
   ret i32 -1
 }
 
 define i32 @test_sgt(ptr noundef %p, i32 noundef %n, ptr noundef %arr, ptr noundef %len_p, i32 noundef %c) {
 ; CHECK-LABEL: @test_sgt(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, ptr [[LEN_P:%.*]], align 4, !range [[RNG0]], !noundef !1
+; CHECK-NEXT:    [[TMP0:%.*]] = icmp sle i32 127, [[LEN]]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[BACKEDGE:%.*]] ]
 ; CHECK-NEXT:    [[EL_PTR:%.*]] = getelementptr i32, ptr [[P:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[EL:%.*]] = load i32, ptr [[EL_PTR]], align 4
 ; CHECK-NEXT:    [[BOUND_CHECK:%.*]] = icmp slt i32 [[EL]], 128
 ; CHECK-NEXT:    br i1 [[BOUND_CHECK]], label [[DO_RANGE_CHECK:%.*]], label [[BOUND_CHECK_FAILED:%.*]]
 ; CHECK:       do_range_check:
 ; CHECK-NEXT:    call void @llvm.assume(i1 [[BOUND_CHECK]])
-; CHECK-NEXT:    [[RANGE_CHECK_SLE:%.*]] = icmp sgt i32 [[LEN]], [[EL]]
 ; CHECK-NEXT:    [[WC:%.*]] = call i1 @llvm.experimental.widenable.condition()
-; CHECK-NEXT:    [[EXPLICIT_GUARD_COND:%.*]] = and i1 [[RANGE_CHECK_SLE]], [[WC]]
-; CHECK-NEXT:    br i1 [[EXPLICIT_GUARD_COND]], label [[BACKEDGE]], label [[DEOPT:%.*]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i1 [[TMP0]], [[WC]]
+; CHECK-NEXT:    br i1 [[TMP1]], label [[BACKEDGE]], label [[DEOPT:%.*]]
 ; CHECK:       backedge:
 ; CHECK-NEXT:    [[ARR_PTR:%.*]] = getelementptr i32, ptr [[ARR:%.*]], i32 [[EL]]
 ; CHECK-NEXT:    store i32 [[IV]], ptr [[ARR_PTR]], align 4
 ; CHECK-NEXT:    [[IV_NEXT]] = add i32 [[IV]], 1
 ; CHECK-NEXT:    [[LOOP_COND:%.*]] = icmp ult i32 [[IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       deopt:
 ; CHECK-NEXT:    [[DEOPT_RES:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"() ]

 bound_check_failed:
   ret i32 -1
 }
 
 define i32 @test_sge(ptr noundef %p, i32 noundef %n, ptr noundef %arr, ptr noundef %len_p, i32 noundef %c) {
 ; CHECK-LABEL: @test_sge(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, ptr [[LEN_P:%.*]], align 4, !range [[RNG0]], !noundef !1
+; CHECK-NEXT:    [[TMP0:%.*]] = icmp sle i32 127, [[LEN]]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[BACKEDGE:%.*]] ]
 ; CHECK-NEXT:    [[EL_PTR:%.*]] = getelementptr i32, ptr [[P:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[EL:%.*]] = load i32, ptr [[EL_PTR]], align 4
 ; CHECK-NEXT:    [[BOUND_CHECK:%.*]] = icmp slt i32 [[EL]], 128
 ; CHECK-NEXT:    br i1 [[BOUND_CHECK]], label [[DO_RANGE_CHECK:%.*]], label [[BOUND_CHECK_FAILED:%.*]]
 ; CHECK:       do_range_check:
 ; CHECK-NEXT:    call void @llvm.assume(i1 [[BOUND_CHECK]])
-; CHECK-NEXT:    [[RANGE_CHECK_SLE:%.*]] = icmp sge i32 [[LEN]], [[EL]]
 ; CHECK-NEXT:    [[WC:%.*]] = call i1 @llvm.experimental.widenable.condition()
-; CHECK-NEXT:    [[EXPLICIT_GUARD_COND:%.*]] = and i1 [[RANGE_CHECK_SLE]], [[WC]]
-; CHECK-NEXT:    br i1 [[EXPLICIT_GUARD_COND]], label [[BACKEDGE]], label [[DEOPT:%.*]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i1 [[TMP0]], [[WC]]
+; CHECK-NEXT:    br i1 [[TMP1]], label [[BACKEDGE]], label [[DEOPT:%.*]]
 ; CHECK:       backedge:
 ; CHECK-NEXT:    [[ARR_PTR:%.*]] = getelementptr i32, ptr [[ARR:%.*]], i32 [[EL]]
 ; CHECK-NEXT:    store i32 [[IV]], ptr [[ARR_PTR]], align 4
 ; CHECK-NEXT:    [[IV_NEXT]] = add i32 [[IV]], 1
 ; CHECK-NEXT:    [[LOOP_COND:%.*]] = icmp ult i32 [[IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       deopt:
 ; CHECK-NEXT:    [[DEOPT_RES:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"() ]