[LoopPredication] Widen checks if condition operands constant ranges are known

[dmakogon]

Summary

This enables LoopPredication to widen checks of the following form:

x < guardLimit,

where x and guardLimit have known constant ranges (meaning they are not
full-set). If they are reasonably small and max(x) <= max(guardLimit), we
can predicate the loop with the condition max(x) <= guardLimit.

[mkazantsev]

llvm/lib/Transforms/Scalar/LoopPredication.cpp

789

Need to make it clear that Guard always belongs to loop, but here we are filtering away guards from nested loops. Assert on L.contains(Guard->getParent())?

809

Isn't [this] enough?

Can it be const Value and const Instruction?

820

Not very informative, can we instead print one message, in the end, and make it verbose? Like, "Speculating LHS = ... less than RHS = ... because max(LHS) < max(RHS)" or something?

852

if (IsSigned)
  LHSMaxIsGreaterThanRHSMax = LHSMaxValue.sgt(RHSMaxValue);
else
  HSMaxIsGreaterThanRHSMax = LHSMaxValue.ugt(RHSMaxValue);

?

909

Should we have a separate statistics for that?

[yrouban]

llvm/lib/Transforms/Scalar/LoopPredication.cpp

840

do it right after RHSRange = ComputeRange() and only then calculate LHSRange.

852

or

bool LHSMaxIsGreaterThanRHSMax = IsSigned
                               ? LHSMaxValue.sgt(RHSMaxValue)
                               : LHSMaxValue.ugt(RHSMaxValue);

or even a bigger change without introduced LHSMaxValue and RHSMaxValue:

bool IsGreaterThanRHSMax = IsSigned
                         ? LHSRange.getSignedMax().sgt(RHSRange.getSignedMax())
                         : LHSRange.getUnsignedMax().ugt(RHSRange.getUnsignedMax());

[apilipenko]

llvm/lib/Transforms/Scalar/LoopPredication.cpp

248–249

404–405

This is a non-canonical way to get an analysis. This way it will not benefit from analysis manager caching. I guess you have to do this because LVI is a function analysis but a part of LoopStandardAnalysisResults.

I'm not sure how we should go about it, but I suggest splitting the LVI off from the initial patch. You can still get the constant range from ValueTracking to get the basic version of this optimization.

817–831

Please, spell the list of supported predicated explicitly here.

850

Why do you need to check this?

[dmakogon]

Addressed comments

[dmakogon]

llvm/lib/Transforms/Scalar/LoopPredication.cpp

404–405

Removed LVI from this patch. Please note the test changes. I explicitly inserted assumes of dominating branches conditions to allow for the widening to happen

[apilipenko]

llvm/lib/Transforms/Scalar/LoopPredication.cpp

817–831

Could you please, spell the list of *supported* predicates explicitly?

[dmakogon]

Rebase, address comments

[dmakogon]

llvm/lib/Transforms/Scalar/LoopPredication.cpp

850

Well, this is the speculation condition we decided to use. The idea here is the following:
Assume LHS is an index and RHS is array length.
So we'd like to speculate only if the index possible maximum value isn't greater that the array length maximum value.
Otherwise we assume that index would take the value greater than the array length in most cases, so we'd deoptimize.
We don't have exact statistics that such cases prevail, it's just our expectations I guess. Also here we cut off the case when LHS has a full-set range.

[mkazantsev]

llvm/lib/Transforms/Scalar/LoopPredication.cpp

829

In the switch above:

  case CmpInst::ICMP_SGE:
  case CmpInst::ICMP_UGT:
  case CmpInst::ICMP_UGE:
  case CmpInst::ICMP_SGT:
    std::swap(LHS, RHS);
    Pred = ICmpInst::getSwappedPredicate(Pred);
    LLVM_FALLTHROUGH
// Allowed predicates:
    break;
default:
  return nullptr

[mkazantsev]

Marking to remove it from my review list, please update if you have plans on it.

llvm/lib/Transforms/Scalar/LoopPredication.cpp

 //     guardStart u< guardLimit && latchLimit u> 1.
 //   For sge condition the widened condition is:
 //     guardStart u< guardLimit && latchLimit s> 1.
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Transforms/Scalar/LoopPredication.h"
 #include "llvm/ADT/Statistic.h"
 #include "llvm/Analysis/AliasAnalysis.h"
+#include "llvm/Analysis/AssumptionCache.h"
 #include "llvm/Analysis/BranchProbabilityInfo.h"
 #include "llvm/Analysis/GuardUtils.h"
+#include "llvm/Analysis/LazyValueInfo.h"
 #include "llvm/Analysis/LoopInfo.h"
 #include "llvm/Analysis/LoopPass.h"
 #include "llvm/Analysis/MemorySSA.h"
 #include "llvm/Analysis/MemorySSAUpdater.h"
 #include "llvm/Analysis/ScalarEvolution.h"
 #include "llvm/Analysis/ScalarEvolutionExpressions.h"
+#include "llvm/Analysis/ValueTracking.h"
 #include "llvm/IR/Function.h"
+#include "llvm/IR/Instructions.h"
 #include "llvm/IR/IntrinsicInst.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/PatternMatch.h"
 #include "llvm/InitializePasses.h"
 #include "llvm/Pass.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Debug.h"
+#include "llvm/Support/KnownBits.h"
 #include "llvm/Transforms/Scalar.h"
 #include "llvm/Transforms/Utils/GuardUtils.h"
 #include "llvm/Transforms/Utils/Local.h"
 #include "llvm/Transforms/Utils/LoopUtils.h"
 #include "llvm/Transforms/Utils/ScalarEvolutionExpander.h"
 
 #define DEBUG_TYPE "loop-predication"
 

 
 static cl::opt<bool> InsertAssumesOfPredicatedGuardsConditions(
     "loop-predication-insert-assumes-of-predicated-guards-conditions",
     cl::Hidden,
     cl::desc("Whether or not we should insert assumes of conditions of "
              "predicated guards"),
     cl::init(true));
 
+static cl::opt<bool> WidenChecksUsingConstantRange(
+    "loop-predication-widen-checks-using-contants-range", cl::Hidden,
+    cl::desc(
+        "Whether or not we should widen checks using constant range analysis"),
+    cl::init(true));
+
+static cl::opt<unsigned> MaxConstantRangeValue(
+    "loop-predication-max-constant-range-value", cl::Hidden,
+    cl::desc("The maximum value of constant range for a value in check when "
+             "widening using constant range analysis"),
+    cl::init(4096));
+
 namespace {
 /// Represents an induction variable check:
 ///   icmp Pred, <induction variable>, <loop invariant limit>
 struct LoopICmp {
   ICmpInst::Predicate Pred;
   const SCEVAddRecExpr *IV;
   const SCEV *Limit;
   LoopICmp(ICmpInst::Predicate Pred, const SCEVAddRecExpr *IV,
            const SCEV *Limit)
     : Pred(Pred), IV(IV), Limit(Limit) {}
   LoopICmp() = default;
   void dump() {
     dbgs() << "LoopICmp Pred = " << Pred << ", IV = " << *IV
            << ", Limit = " << *Limit << "\n";
   }
 };
 
 class LoopPredication {
   AliasAnalysis *AA;
   DominatorTree *DT;
   ScalarEvolution *SE;
   LoopInfo *LI;
   MemorySSAUpdater *MSSAU;
+  AssumptionCache *AC;
+  LazyValueInfo *LVI;
 
   Loop *L;
   const DataLayout *DL;
   BasicBlock *Preheader;
   LoopICmp LatchCheck;
 
   bool isSupportedStep(const SCEV* Step);
   Optional<LoopICmp> parseLoopICmp(ICmpInst *ICI);
class LoopPredication {
   bool isLoopInvariantValue(const SCEV* S);
 
   Value *expandCheck(SCEVExpander &Expander, Instruction *Guard,
                      ICmpInst::Predicate Pred, const SCEV *LHS,
                      const SCEV *RHS);
 
   Optional<Value *> widenICmpRangeCheck(ICmpInst *ICI, SCEVExpander &Expander,
                                         Instruction *Guard);
+  Optional<Value *> widenRangeCheckUsingConstantRange(ICmpInst *ICI,
+                                                      Instruction *Guard);
   Optional<Value *> widenICmpRangeCheckIncrementingLoop(LoopICmp LatchCheck,
                                                         LoopICmp RangeCheck,
                                                         SCEVExpander &Expander,
                                                         Instruction *Guard);
   Optional<Value *> widenICmpRangeCheckDecrementingLoop(LoopICmp LatchCheck,
                                                         LoopICmp RangeCheck,
                                                         SCEVExpander &Expander,
                                                         Instruction *Guard);
   unsigned collectChecks(SmallVectorImpl<Value *> &Checks, Value *Condition,
                          SCEVExpander &Expander, Instruction *Guard);
   bool widenGuardConditions(IntrinsicInst *II, SCEVExpander &Expander);
   bool widenWidenableBranchGuardConditions(BranchInst *Guard, SCEVExpander &Expander);
   // If the loop always exits through another block in the loop, we should not
   // predicate based on the latch check. For example, the latch check can be a
   // very coarse grained check and there can be more fine grained exit checks
   // within the loop.
   bool isLoopProfitableToPredicate();
 
   bool predicateLoopExits(Loop *L, SCEVExpander &Rewriter);
 
 public:
   LoopPredication(AliasAnalysis *AA, DominatorTree *DT, ScalarEvolution *SE,
-                  LoopInfo *LI, MemorySSAUpdater *MSSAU)
-      : AA(AA), DT(DT), SE(SE), LI(LI), MSSAU(MSSAU){};
+                  LoopInfo *LI, MemorySSAUpdater *MSSAU, AssumptionCache *AC,
+                  LazyValueInfo *LVI)
+      : AA(AA), DT(DT), SE(SE), LI(LI), MSSAU(MSSAU), AC(AC), LVI(LVI) {};
   bool runOnLoop(Loop *L);
 };
 
 class LoopPredicationLegacyPass : public LoopPass {
 public:
   static char ID;
   LoopPredicationLegacyPass() : LoopPass(ID) {
     initializeLoopPredicationLegacyPassPass(*PassRegistry::getPassRegistry());
   }
 
   void getAnalysisUsage(AnalysisUsage &AU) const override {
     AU.addRequired<BranchProbabilityInfoWrapperPass>();
+    AU.addRequired<AssumptionCacheTracker>();
+    AU.addRequired<LazyValueInfoWrapperPass>();
     getLoopAnalysisUsage(AU);
     AU.addPreserved<MemorySSAWrapperPass>();
   }
 
   bool runOnLoop(Loop *L, LPPassManager &LPM) override {
     if (skipLoop(L))
       return false;
+    Function &F = *L->getHeader()->getParent();
+
     auto *SE = &getAnalysis<ScalarEvolutionWrapperPass>().getSE();
     auto *LI = &getAnalysis<LoopInfoWrapperPass>().getLoopInfo();
     auto *DT = &getAnalysis<DominatorTreeWrapperPass>().getDomTree();
     auto *MSSAWP = getAnalysisIfAvailable<MemorySSAWrapperPass>();
+    auto *AC = &getAnalysis<AssumptionCacheTracker>().getAssumptionCache(F);
+    auto *LVI = &getAnalysis<LazyValueInfoWrapperPass>().getLVI();
     std::unique_ptr<MemorySSAUpdater> MSSAU;
     if (MSSAWP)
       MSSAU = std::make_unique<MemorySSAUpdater>(&MSSAWP->getMSSA());
     auto *AA = &getAnalysis<AAResultsWrapperPass>().getAAResults();
-    LoopPredication LP(AA, DT, SE, LI, MSSAU ? MSSAU.get() : nullptr);
+    LoopPredication LP(AA, DT, SE, LI, MSSAU ? MSSAU.get() : nullptr, AC, LVI);
     return LP.runOnLoop(L);
   }
 };
 
 char LoopPredicationLegacyPass::ID = 0;
 } // end namespace
 
 INITIALIZE_PASS_BEGIN(LoopPredicationLegacyPass, "loop-predication",
                       "Loop predication", false, false)
 INITIALIZE_PASS_DEPENDENCY(BranchProbabilityInfoWrapperPass)
 INITIALIZE_PASS_DEPENDENCY(LoopPass)
 INITIALIZE_PASS_END(LoopPredicationLegacyPass, "loop-predication",
                     "Loop predication", false, false)
 
 Pass *llvm::createLoopPredicationPass() {
   return new LoopPredicationLegacyPass();
 }
 
 PreservedAnalyses LoopPredicationPass::run(Loop &L, LoopAnalysisManager &AM,
                                            LoopStandardAnalysisResults &AR,
                                            LPMUpdater &U) {
   std::unique_ptr<MemorySSAUpdater> MSSAU;
   if (AR.MSSA)
     MSSAU = std::make_unique<MemorySSAUpdater>(AR.MSSA);
+
+  auto &DL = L.getHeader()->getModule()->getDataLayout();
+  LazyValueInfo LVI(&AR.AC, &DL, &AR.TLI);
+
   LoopPredication LP(&AR.AA, &AR.DT, &AR.SE, &AR.LI,
-                     MSSAU ? MSSAU.get() : nullptr);
+                     MSSAU ? MSSAU.get() : nullptr, &AR.AC, &LVI);
   if (!LP.runOnLoop(&L))
     return PreservedAnalyses::all();
 
   auto PA = getLoopPassPreservedAnalyses();
   if (AR.MSSA)
     PA.preserve<MemorySSAAnalysis>();
   return PA;
 }
return widenICmpRangeCheckIncrementingLoop(CurrLatchCheck, *RangeCheck,
                                                Expander, Guard);
   else {
     assert(Step->isAllOnesValue() && "Step should be -1!");
     return widenICmpRangeCheckDecrementingLoop(CurrLatchCheck, *RangeCheck,
                                                Expander, Guard);
   }
 }
 
+Optional<Value *>
+LoopPredication::widenRangeCheckUsingConstantRange(ICmpInst *ICI,
+                                                   Instruction *Guard) {
+  if (!WidenChecksUsingConstantRange || LI->getLoopFor(Guard->getParent()) != L)
+    return None;
+  LLVM_DEBUG(dbgs() << "Analyzing ICmpInst condition using constant range:\n");
+  LLVM_DEBUG(ICI->dump());
+
+  // We are looking for the range checks of the form:
+  //   x < guardLimit,
+  // where x and guardLimit have known constant ranges (meaning they are not
+  // full-set). If they are reasonably small and max(x) <= max(guardLimit), we
+  // can predicate the loop with the condition max(x) <= guardLimit.
+  auto Pred = ICI->getPredicate();
+  auto *LHS = ICI->getOperand(0);
+  auto *RHS = ICI->getOperand(1);
+
+  // Both operands are loop variant, bail out.
+  if (!L->isLoopInvariant(LHS) && !L->isLoopInvariant(RHS)) {
+    LLVM_DEBUG(dbgs() << "LHS and RHS are loop variant\n");
+    return None;
+  }
+
+  auto ComputeRange = [&](Value *V, bool IsSigned, Instruction *CtxI) {
+    auto Range = ConstantRange::fromKnownBits(
+        computeKnownBits(V, *DL, 0, AC, CtxI, DT), IsSigned);
+    Range = Range.intersectWith(
+        computeConstantRange(V, IsSigned, true, AC, CtxI, DT));
+    return Range.intersectWith(LVI->getConstantRange(V, CtxI));
+  };
+
+  switch (Pred) {
+  case CmpInst::ICMP_EQ:
+  case CmpInst::ICMP_NE:
+    LLVM_DEBUG(dbgs() << "Unsupported predicate\n");
+    return None;
+  case CmpInst::ICMP_UGT:
+  case CmpInst::ICMP_UGE:
+  case CmpInst::ICMP_SGT:
+  case CmpInst::ICMP_SGE:
+    std::swap(LHS, RHS);
+    Pred = ICmpInst::getSwappedPredicate(Pred);
+    break;
+  default:
+    break;
+  }
+
+  bool IsSigned = ICmpInst::isSigned(Pred);
+
+  auto LHSRange = ComputeRange(LHS, IsSigned, Guard);
+  auto RHSRange = ComputeRange(RHS, IsSigned, Guard);
+
+  ICmpInst::Predicate NewPred = ICmpInst::getNonStrictPredicate(Pred);
+
+  if (RHSRange.isFullSet()) {
+    LLVM_DEBUG(dbgs() << "RHS range is full-set\n");
+    return None;
+  }
+
+  auto LHSMaxValue =
+      IsSigned ? LHSRange.getSignedMax() : LHSRange.getUnsignedMax();
+  auto RHSMaxValue =
+      IsSigned ? RHSRange.getSignedMax() : RHSRange.getUnsignedMax();
+
+  bool LHSMaxIsGreaterThanRHSMax = LHSMaxValue.ugt(RHSMaxValue);
+  if (IsSigned)
+    LHSMaxIsGreaterThanRHSMax = LHSMaxValue.sgt(RHSMaxValue);
+  if (LHSMaxIsGreaterThanRHSMax) {
+    LLVM_DEBUG(dbgs() << "LHS max value is not greater than RHS maximum\n");
+    return None;
+  }
+
+  LLVM_DEBUG(dbgs() << "LHS: " << LHSMaxValue << "\n");
+  LLVM_DEBUG(dbgs() << "RHS: " << *RHS << "\n");
+  LLVM_DEBUG(dbgs() << "Pred: " << ICmpInst::getPredicateName(NewPred) << "\n");
+  auto *MaxCI = ConstantInt::get(LHS->getType(), LHSMaxValue);
+  IRBuilder<> Builder(findInsertPt(Guard, {RHS}));
+  auto *NewICI = Builder.CreateICmp(NewPred, MaxCI, RHS);
+  return NewICI;
+}
+
 unsigned LoopPredication::collectChecks(SmallVectorImpl<Value *> &Checks,
                                         Value *Condition,
                                         SCEVExpander &Expander,
                                         Instruction *Guard) {
   unsigned NumWidened = 0;
   // The guard condition is expected to be in form of:
   //   cond1 && cond2 && cond3 ...
   // Iterate over subconditions looking for icmp conditions which can be
do {
 
     if (ICmpInst *ICI = dyn_cast<ICmpInst>(Condition)) {
       if (auto NewRangeCheck = widenICmpRangeCheck(ICI, Expander,
                                                    Guard)) {
         Checks.push_back(*NewRangeCheck);
         NumWidened++;
         continue;
       }
+      if (auto RC = widenRangeCheckUsingConstantRange(ICI, Guard)) {
+        Checks.push_back(*RC);
+        NumWidened++;
+        continue;
+      }
     }
 
     // Save the condition as is if we can't widen it
     Checks.push_back(Condition);
   } while (!Worklist.empty());
   // At the moment, our matching logic for wideable conditions implicitly
   // assumes we preserve the form: (br (and Cond, WC())).  FIXME
   // Note that if there were multiple calls to wideable condition in the

llvm/test/Transforms/LoopPredication/constant-ranges.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -S -passes='loop-mssa(loop-predication)' -loop-predication-insert-assumes-of-predicated-guards-conditions=false < %s 2>&1 | FileCheck %s
+; RUN: opt -S -passes='loop-mssa(loop-predication)' -loop-predication-widen-checks-using-contants-range=true -loop-predication-insert-assumes-of-predicated-guards-conditions=false < %s 2>&1 | FileCheck %s
 
 define i32 @test_ult(ptr noundef %p, i32 noundef %n, ptr noundef %arr, ptr noundef %len_p, i32 noundef %c) {
 ; CHECK-LABEL: @test_ult(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, ptr [[LEN_P:%.*]], align 4, !range [[RNG0:![0-9]+]], !noundef !1
+; CHECK-NEXT:    [[TMP0:%.*]] = icmp ule i32 127, [[LEN]]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[BACKEDGE:%.*]] ]
 ; CHECK-NEXT:    [[EL_PTR:%.*]] = getelementptr i32, ptr [[P:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[EL:%.*]] = load i32, ptr [[EL_PTR]], align 4
 ; CHECK-NEXT:    [[BOUND_CHECK:%.*]] = icmp ult i32 [[EL]], 128
 ; CHECK-NEXT:    br i1 [[BOUND_CHECK]], label [[DO_RANGE_CHECK:%.*]], label [[BOUND_CHECK_FAILED:%.*]]
 ; CHECK:       do_range_check:
-; CHECK-NEXT:    [[RANGE_CHECK_ULT:%.*]] = icmp ult i32 [[EL]], [[LEN]]
 ; CHECK-NEXT:    [[WC:%.*]] = call i1 @llvm.experimental.widenable.condition()
-; CHECK-NEXT:    [[EXPLICIT_GUARD_COND:%.*]] = and i1 [[RANGE_CHECK_ULT]], [[WC]]
-; CHECK-NEXT:    br i1 [[EXPLICIT_GUARD_COND]], label [[BACKEDGE]], label [[DEOPT:%.*]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i1 [[TMP0]], [[WC]]
+; CHECK-NEXT:    br i1 [[TMP1]], label [[BACKEDGE]], label [[DEOPT:%.*]]
 ; CHECK:       backedge:
 ; CHECK-NEXT:    [[ARR_PTR:%.*]] = getelementptr i32, ptr [[ARR:%.*]], i32 [[EL]]
 ; CHECK-NEXT:    store i32 [[IV]], ptr [[ARR_PTR]], align 4
 ; CHECK-NEXT:    [[IV_NEXT]] = add i32 [[IV]], 1
 ; CHECK-NEXT:    [[LOOP_COND:%.*]] = icmp slt i32 [[IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       deopt:
 ; CHECK-NEXT:    [[DEOPT_RES:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"() ]

 bound_check_failed:
   ret i32 -1
 }
 
 define i32 @test_slt(ptr noundef %p, i32 noundef %n, ptr noundef %arr, ptr noundef %len_p, i32 noundef %c) {
 ; CHECK-LABEL: @test_slt(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, ptr [[LEN_P:%.*]], align 4, !range [[RNG0]], !noundef !1
+; CHECK-NEXT:    [[TMP0:%.*]] = icmp sle i32 127, [[LEN]]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[BACKEDGE:%.*]] ]
 ; CHECK-NEXT:    [[EL_PTR:%.*]] = getelementptr i32, ptr [[P:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[EL:%.*]] = load i32, ptr [[EL_PTR]], align 4
 ; CHECK-NEXT:    [[BOUND_CHECK:%.*]] = icmp slt i32 [[EL]], 128
 ; CHECK-NEXT:    br i1 [[BOUND_CHECK]], label [[DO_RANGE_CHECK:%.*]], label [[BOUND_CHECK_FAILED:%.*]]
 ; CHECK:       do_range_check:
-; CHECK-NEXT:    [[RANGE_CHECK_SLT:%.*]] = icmp slt i32 [[EL]], [[LEN]]
 ; CHECK-NEXT:    [[WC:%.*]] = call i1 @llvm.experimental.widenable.condition()
-; CHECK-NEXT:    [[EXPLICIT_GUARD_COND:%.*]] = and i1 [[RANGE_CHECK_SLT]], [[WC]]
-; CHECK-NEXT:    br i1 [[EXPLICIT_GUARD_COND]], label [[BACKEDGE]], label [[DEOPT:%.*]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i1 [[TMP0]], [[WC]]
+; CHECK-NEXT:    br i1 [[TMP1]], label [[BACKEDGE]], label [[DEOPT:%.*]]
 ; CHECK:       backedge:
 ; CHECK-NEXT:    [[ARR_PTR:%.*]] = getelementptr i32, ptr [[ARR:%.*]], i32 [[EL]]
 ; CHECK-NEXT:    store i32 [[IV]], ptr [[ARR_PTR]], align 4
 ; CHECK-NEXT:    [[IV_NEXT]] = add i32 [[IV]], 1
 ; CHECK-NEXT:    [[LOOP_COND:%.*]] = icmp ult i32 [[IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       deopt:
 ; CHECK-NEXT:    [[DEOPT_RES:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"() ]

 bound_check_failed:
   ret i32 -1
 }
 
 define i32 @test_ule(ptr noundef %p, i32 noundef %n, ptr noundef %arr, ptr noundef %len_p, i32 noundef %c) {
 ; CHECK-LABEL: @test_ule(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, ptr [[LEN_P:%.*]], align 4, !range [[RNG0]], !noundef !1
+; CHECK-NEXT:    [[TMP0:%.*]] = icmp ule i32 127, [[LEN]]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[BACKEDGE:%.*]] ]
 ; CHECK-NEXT:    [[EL_PTR:%.*]] = getelementptr i32, ptr [[P:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[EL:%.*]] = load i32, ptr [[EL_PTR]], align 4
 ; CHECK-NEXT:    [[BOUND_CHECK:%.*]] = icmp ult i32 [[EL]], 128
 ; CHECK-NEXT:    br i1 [[BOUND_CHECK]], label [[DO_RANGE_CHECK:%.*]], label [[BOUND_CHECK_FAILED:%.*]]
 ; CHECK:       do_range_check:
-; CHECK-NEXT:    [[RANGE_CHECK_ULE:%.*]] = icmp ule i32 [[EL]], [[LEN]]
 ; CHECK-NEXT:    [[WC:%.*]] = call i1 @llvm.experimental.widenable.condition()
-; CHECK-NEXT:    [[EXPLICIT_GUARD_COND:%.*]] = and i1 [[RANGE_CHECK_ULE]], [[WC]]
-; CHECK-NEXT:    br i1 [[EXPLICIT_GUARD_COND]], label [[BACKEDGE]], label [[DEOPT:%.*]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i1 [[TMP0]], [[WC]]
+; CHECK-NEXT:    br i1 [[TMP1]], label [[BACKEDGE]], label [[DEOPT:%.*]]
 ; CHECK:       backedge:
 ; CHECK-NEXT:    [[ARR_PTR:%.*]] = getelementptr i32, ptr [[ARR:%.*]], i32 [[EL]]
 ; CHECK-NEXT:    store i32 [[IV]], ptr [[ARR_PTR]], align 4
 ; CHECK-NEXT:    [[IV_NEXT]] = add i32 [[IV]], 1
 ; CHECK-NEXT:    [[LOOP_COND:%.*]] = icmp ult i32 [[IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       deopt:
 ; CHECK-NEXT:    [[DEOPT_RES:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"() ]

 bound_check_failed:
   ret i32 -1
 }
 
 define i32 @test_sle(ptr noundef %p, i32 noundef %n, ptr noundef %arr, ptr noundef %len_p, i32 noundef %c) {
 ; CHECK-LABEL: @test_sle(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, ptr [[LEN_P:%.*]], align 4, !range [[RNG0]], !noundef !1
+; CHECK-NEXT:    [[TMP0:%.*]] = icmp sle i32 127, [[LEN]]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[BACKEDGE:%.*]] ]
 ; CHECK-NEXT:    [[EL_PTR:%.*]] = getelementptr i32, ptr [[P:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[EL:%.*]] = load i32, ptr [[EL_PTR]], align 4
 ; CHECK-NEXT:    [[BOUND_CHECK:%.*]] = icmp slt i32 [[EL]], 128
 ; CHECK-NEXT:    br i1 [[BOUND_CHECK]], label [[DO_RANGE_CHECK:%.*]], label [[BOUND_CHECK_FAILED:%.*]]
 ; CHECK:       do_range_check:
-; CHECK-NEXT:    [[RANGE_CHECK_SLE:%.*]] = icmp sle i32 [[EL]], [[LEN]]
 ; CHECK-NEXT:    [[WC:%.*]] = call i1 @llvm.experimental.widenable.condition()
-; CHECK-NEXT:    [[EXPLICIT_GUARD_COND:%.*]] = and i1 [[RANGE_CHECK_SLE]], [[WC]]
-; CHECK-NEXT:    br i1 [[EXPLICIT_GUARD_COND]], label [[BACKEDGE]], label [[DEOPT:%.*]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i1 [[TMP0]], [[WC]]
+; CHECK-NEXT:    br i1 [[TMP1]], label [[BACKEDGE]], label [[DEOPT:%.*]]
 ; CHECK:       backedge:
 ; CHECK-NEXT:    [[ARR_PTR:%.*]] = getelementptr i32, ptr [[ARR:%.*]], i32 [[EL]]
 ; CHECK-NEXT:    store i32 [[IV]], ptr [[ARR_PTR]], align 4
 ; CHECK-NEXT:    [[IV_NEXT]] = add i32 [[IV]], 1
 ; CHECK-NEXT:    [[LOOP_COND:%.*]] = icmp ult i32 [[IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       deopt:
 ; CHECK-NEXT:    [[DEOPT_RES:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"() ]

 bound_check_failed:
   ret i32 -1
 }
 
 define i32 @test_ugt(ptr noundef %p, i32 noundef %n, ptr noundef %arr, ptr noundef %len_p, i32 noundef %c) {
 ; CHECK-LABEL: @test_ugt(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, ptr [[LEN_P:%.*]], align 4, !range [[RNG0]], !noundef !1
+; CHECK-NEXT:    [[TMP0:%.*]] = icmp ule i32 127, [[LEN]]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[BACKEDGE:%.*]] ]
 ; CHECK-NEXT:    [[EL_PTR:%.*]] = getelementptr i32, ptr [[P:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[EL:%.*]] = load i32, ptr [[EL_PTR]], align 4
 ; CHECK-NEXT:    [[BOUND_CHECK:%.*]] = icmp ult i32 [[EL]], 128
 ; CHECK-NEXT:    br i1 [[BOUND_CHECK]], label [[DO_RANGE_CHECK:%.*]], label [[BOUND_CHECK_FAILED:%.*]]
 ; CHECK:       do_range_check:
-; CHECK-NEXT:    [[RANGE_CHECK_SLE:%.*]] = icmp ugt i32 [[LEN]], [[EL]]
 ; CHECK-NEXT:    [[WC:%.*]] = call i1 @llvm.experimental.widenable.condition()
-; CHECK-NEXT:    [[EXPLICIT_GUARD_COND:%.*]] = and i1 [[RANGE_CHECK_SLE]], [[WC]]
-; CHECK-NEXT:    br i1 [[EXPLICIT_GUARD_COND]], label [[BACKEDGE]], label [[DEOPT:%.*]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i1 [[TMP0]], [[WC]]
+; CHECK-NEXT:    br i1 [[TMP1]], label [[BACKEDGE]], label [[DEOPT:%.*]]
 ; CHECK:       backedge:
 ; CHECK-NEXT:    [[ARR_PTR:%.*]] = getelementptr i32, ptr [[ARR:%.*]], i32 [[EL]]
 ; CHECK-NEXT:    store i32 [[IV]], ptr [[ARR_PTR]], align 4
 ; CHECK-NEXT:    [[IV_NEXT]] = add i32 [[IV]], 1
 ; CHECK-NEXT:    [[LOOP_COND:%.*]] = icmp ult i32 [[IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       deopt:
 ; CHECK-NEXT:    [[DEOPT_RES:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"() ]

 bound_check_failed:
   ret i32 -1
 }
 
 define i32 @test_uge(ptr noundef %p, i32 noundef %n, ptr noundef %arr, ptr noundef %len_p, i32 noundef %c) {
 ; CHECK-LABEL: @test_uge(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, ptr [[LEN_P:%.*]], align 4, !range [[RNG0]], !noundef !1
+; CHECK-NEXT:    [[TMP0:%.*]] = icmp ule i32 127, [[LEN]]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[BACKEDGE:%.*]] ]
 ; CHECK-NEXT:    [[EL_PTR:%.*]] = getelementptr i32, ptr [[P:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[EL:%.*]] = load i32, ptr [[EL_PTR]], align 4
 ; CHECK-NEXT:    [[BOUND_CHECK:%.*]] = icmp ult i32 [[EL]], 128
 ; CHECK-NEXT:    br i1 [[BOUND_CHECK]], label [[DO_RANGE_CHECK:%.*]], label [[BOUND_CHECK_FAILED:%.*]]
 ; CHECK:       do_range_check:
-; CHECK-NEXT:    [[RANGE_CHECK_SLE:%.*]] = icmp uge i32 [[LEN]], [[EL]]
 ; CHECK-NEXT:    [[WC:%.*]] = call i1 @llvm.experimental.widenable.condition()
-; CHECK-NEXT:    [[EXPLICIT_GUARD_COND:%.*]] = and i1 [[RANGE_CHECK_SLE]], [[WC]]
-; CHECK-NEXT:    br i1 [[EXPLICIT_GUARD_COND]], label [[BACKEDGE]], label [[DEOPT:%.*]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i1 [[TMP0]], [[WC]]
+; CHECK-NEXT:    br i1 [[TMP1]], label [[BACKEDGE]], label [[DEOPT:%.*]]
 ; CHECK:       backedge:
 ; CHECK-NEXT:    [[ARR_PTR:%.*]] = getelementptr i32, ptr [[ARR:%.*]], i32 [[EL]]
 ; CHECK-NEXT:    store i32 [[IV]], ptr [[ARR_PTR]], align 4
 ; CHECK-NEXT:    [[IV_NEXT]] = add i32 [[IV]], 1
 ; CHECK-NEXT:    [[LOOP_COND:%.*]] = icmp ult i32 [[IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       deopt:
 ; CHECK-NEXT:    [[DEOPT_RES:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"() ]

 bound_check_failed:
   ret i32 -1
 }
 
 define i32 @test_sgt(ptr noundef %p, i32 noundef %n, ptr noundef %arr, ptr noundef %len_p, i32 noundef %c) {
 ; CHECK-LABEL: @test_sgt(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, ptr [[LEN_P:%.*]], align 4, !range [[RNG0]], !noundef !1
+; CHECK-NEXT:    [[TMP0:%.*]] = icmp sle i32 127, [[LEN]]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[BACKEDGE:%.*]] ]
 ; CHECK-NEXT:    [[EL_PTR:%.*]] = getelementptr i32, ptr [[P:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[EL:%.*]] = load i32, ptr [[EL_PTR]], align 4
 ; CHECK-NEXT:    [[BOUND_CHECK:%.*]] = icmp slt i32 [[EL]], 128
 ; CHECK-NEXT:    br i1 [[BOUND_CHECK]], label [[DO_RANGE_CHECK:%.*]], label [[BOUND_CHECK_FAILED:%.*]]
 ; CHECK:       do_range_check:
-; CHECK-NEXT:    [[RANGE_CHECK_SLE:%.*]] = icmp sgt i32 [[LEN]], [[EL]]
 ; CHECK-NEXT:    [[WC:%.*]] = call i1 @llvm.experimental.widenable.condition()
-; CHECK-NEXT:    [[EXPLICIT_GUARD_COND:%.*]] = and i1 [[RANGE_CHECK_SLE]], [[WC]]
-; CHECK-NEXT:    br i1 [[EXPLICIT_GUARD_COND]], label [[BACKEDGE]], label [[DEOPT:%.*]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i1 [[TMP0]], [[WC]]
+; CHECK-NEXT:    br i1 [[TMP1]], label [[BACKEDGE]], label [[DEOPT:%.*]]
 ; CHECK:       backedge:
 ; CHECK-NEXT:    [[ARR_PTR:%.*]] = getelementptr i32, ptr [[ARR:%.*]], i32 [[EL]]
 ; CHECK-NEXT:    store i32 [[IV]], ptr [[ARR_PTR]], align 4
 ; CHECK-NEXT:    [[IV_NEXT]] = add i32 [[IV]], 1
 ; CHECK-NEXT:    [[LOOP_COND:%.*]] = icmp ult i32 [[IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       deopt:
 ; CHECK-NEXT:    [[DEOPT_RES:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"() ]

 bound_check_failed:
   ret i32 -1
 }
 
 define i32 @test_sge(ptr noundef %p, i32 noundef %n, ptr noundef %arr, ptr noundef %len_p, i32 noundef %c) {
 ; CHECK-LABEL: @test_sge(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[LEN:%.*]] = load i32, ptr [[LEN_P:%.*]], align 4, !range [[RNG0]], !noundef !1
+; CHECK-NEXT:    [[TMP0:%.*]] = icmp sle i32 127, [[LEN]]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[BACKEDGE:%.*]] ]
 ; CHECK-NEXT:    [[EL_PTR:%.*]] = getelementptr i32, ptr [[P:%.*]], i32 [[IV]]
 ; CHECK-NEXT:    [[EL:%.*]] = load i32, ptr [[EL_PTR]], align 4
 ; CHECK-NEXT:    [[BOUND_CHECK:%.*]] = icmp slt i32 [[EL]], 128
 ; CHECK-NEXT:    br i1 [[BOUND_CHECK]], label [[DO_RANGE_CHECK:%.*]], label [[BOUND_CHECK_FAILED:%.*]]
 ; CHECK:       do_range_check:
-; CHECK-NEXT:    [[RANGE_CHECK_SLE:%.*]] = icmp sge i32 [[LEN]], [[EL]]
 ; CHECK-NEXT:    [[WC:%.*]] = call i1 @llvm.experimental.widenable.condition()
-; CHECK-NEXT:    [[EXPLICIT_GUARD_COND:%.*]] = and i1 [[RANGE_CHECK_SLE]], [[WC]]
-; CHECK-NEXT:    br i1 [[EXPLICIT_GUARD_COND]], label [[BACKEDGE]], label [[DEOPT:%.*]]
+; CHECK-NEXT:    [[TMP1:%.*]] = and i1 [[TMP0]], [[WC]]
+; CHECK-NEXT:    br i1 [[TMP1]], label [[BACKEDGE]], label [[DEOPT:%.*]]
 ; CHECK:       backedge:
 ; CHECK-NEXT:    [[ARR_PTR:%.*]] = getelementptr i32, ptr [[ARR:%.*]], i32 [[EL]]
 ; CHECK-NEXT:    store i32 [[IV]], ptr [[ARR_PTR]], align 4
 ; CHECK-NEXT:    [[IV_NEXT]] = add i32 [[IV]], 1
 ; CHECK-NEXT:    [[LOOP_COND:%.*]] = icmp ult i32 [[IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       deopt:
 ; CHECK-NEXT:    [[DEOPT_RES:%.*]] = call i32 (...) @llvm.experimental.deoptimize.i32() [ "deopt"() ]