[SCEV] Drop cached ranges of AddRecs after flag update

[mkazantsev]

Summary

Our range computation methods benefit from no-wrap flags. But if the ranges
were first computed before the flags were set, the cached range will be too
pessimistic.

We need to drop cached ranges whenever we sharpen AddRec's no wrap flags.

[fhahn]

llvm/include/llvm/Analysis/ScalarEvolution.h

1491

Is there a reason we don't do that for all SCEV expressions with wrapping flags? Might be good to document that this also invalidates cached info for AddRec, if it updates the flags and this should be use d from inside SE instead of AddRec->setNoWrapFlags. Not sure if there's a nice way to enforce users in SE to use SE::setNoWrapFlags.

[nikic]

llvm/lib/Analysis/ScalarEvolution.cpp

5360

Could it happen that other ranges have been computed in the meantime, based on the now outdated addrec range?

[mkazantsev]

llvm/lib/Analysis/ScalarEvolution.cpp

5360

Unfortunately yes. But tracking them would be expensive.

[mkazantsev]

llvm/include/llvm/Analysis/ScalarEvolution.h

1491

Sure, it can be done for adds/muls too. Can do as follow-up.

[fhahn]

llvm/lib/Analysis/ScalarEvolution.cpp

5360

Agreed, I do not think tracking this would be feasible unfortunately, but missing updates should be fine, as this should only refine the flags.

Can we assert here that Flags is a refinement of the existing flags on the AddREc?

[mkazantsev]

Added elaborate comment, added assert that we don't pessimize the flags.

For some reason, it's no longer helpimg srem.ll. There has been a lot of related patches merged, one of them broke it?

[fhahn]

In D89847#2381786, @mkazantsev wrote:

Added elaborate comment, added assert that we don't pessimize the flags.

For some reason, it's no longer helpimg srem.ll. There has been a lot of related patches merged, one of them broke it?

I'm not sure, but I think @nikic had some patches that delay the flag computation in some cases, so maybe this caused the changes there?

Patch LGTM, thanks!

[mkazantsev]

I tried reverting that patch, but it didn't help. We can figure it out later I think.

llvm/test/Analysis/ScalarEvolution/srem.ll

32

Not sure w

[mkazantsev]

llvm/lib/Analysis/ScalarEvolution.cpp

5358

I realized it's not the right way to check it since the assert may never fail.

[mkazantsev]

llvm/lib/Analysis/ScalarEvolution.cpp

5358

Actually, setNoWrapFlags does not drop previous flags, so this assert is just pointless. I'll remove it.

[mkazantsev]

Problem with srem solved after rebase. Weird...

[fhahn]

llvm/lib/Analysis/ScalarEvolution.cpp

5358

Ah right it already encodes this safety check. I guess it's not worth worrying about it here then, thanks!

llvm/include/llvm/Analysis/ScalarEvolution.h

DenseMap<const SCEV *, ConstantRange> &Cache =
         Hint == HINT_RANGE_UNSIGNED ? UnsignedRanges : SignedRanges;
 
     auto Pair = Cache.try_emplace(S, std::move(CR));
     if (!Pair.second)
       Pair.first->second = std::move(CR);
     return Pair.first->second;
   }
 
+  /// Update no-wrap flags of an AddRec.
+  void setNoWrapFlags(SCEVAddRecExpr *AddRec, SCEV::NoWrapFlags Flags);
+
   /// Determine the range for a particular SCEV.
   /// NOTE: This returns a reference to an entry in a cache. It must be
   /// copied if its needed for longer.
   const ConstantRange &getRangeRef(const SCEV *S, RangeSignHint Hint);
 
   /// Determines the range for the affine SCEVAddRecExpr {\p Start,+,\p Stop}.
   /// Helper for \c getRange.
   ConstantRange getRangeForAffineAR(const SCEV *Start, const SCEV *Stop,

llvm/lib/Analysis/ScalarEvolution.cpp

if (const SCEVAddRecExpr *AR = dyn_cast<SCEVAddRecExpr>(Op))
     if (AR->isAffine()) {
       const SCEV *Start = AR->getStart();
       const SCEV *Step = AR->getStepRecurrence(*this);
       unsigned BitWidth = getTypeSizeInBits(AR->getType());
       const Loop *L = AR->getLoop();
 
       if (!AR->hasNoUnsignedWrap()) {
         auto NewFlags = proveNoWrapViaConstantRanges(AR);
-        const_cast<SCEVAddRecExpr *>(AR)->setNoWrapFlags(NewFlags);
+        setNoWrapFlags(const_cast<SCEVAddRecExpr *>(AR), NewFlags);
       }
 
       // If we have special knowledge that this addrec won't overflow,
       // we don't need to do any further analysis.
       if (AR->hasNoUnsignedWrap())
         return getAddRecExpr(
             getExtendAddRecStart<SCEVZeroExtendExpr>(AR, Ty, this, Depth + 1),
             getZeroExtendExpr(Step, Ty, Depth + 1), L, AR->getNoWrapFlags());
if (AR->isAffine()) {
           const SCEV *OperandExtendedAdd =
             getAddExpr(WideStart,
                        getMulExpr(WideMaxBECount,
                                   getZeroExtendExpr(Step, WideTy, Depth + 1),
                                   SCEV::FlagAnyWrap, Depth + 1),
                        SCEV::FlagAnyWrap, Depth + 1);
           if (ZAdd == OperandExtendedAdd) {
             // Cache knowledge of AR NUW, which is propagated to this AddRec.
-            const_cast<SCEVAddRecExpr *>(AR)->setNoWrapFlags(SCEV::FlagNUW);
+            setNoWrapFlags(const_cast<SCEVAddRecExpr *>(AR), SCEV::FlagNUW);
             // Return the expression with the addrec on the outside.
             return getAddRecExpr(
                 getExtendAddRecStart<SCEVZeroExtendExpr>(AR, Ty, this,
                                                          Depth + 1),
                 getZeroExtendExpr(Step, Ty, Depth + 1), L,
                 AR->getNoWrapFlags());
           }
           // Similar to above, only this time treat the step value as signed.
           // This covers loops that count down.
           OperandExtendedAdd =
             getAddExpr(WideStart,
                        getMulExpr(WideMaxBECount,
                                   getSignExtendExpr(Step, WideTy, Depth + 1),
                                   SCEV::FlagAnyWrap, Depth + 1),
                        SCEV::FlagAnyWrap, Depth + 1);
           if (ZAdd == OperandExtendedAdd) {
             // Cache knowledge of AR NW, which is propagated to this AddRec.
             // Negative step causes unsigned wrap, but it still can't self-wrap.
-            const_cast<SCEVAddRecExpr *>(AR)->setNoWrapFlags(SCEV::FlagNW);
+            setNoWrapFlags(const_cast<SCEVAddRecExpr *>(AR), SCEV::FlagNW);
             // Return the expression with the addrec on the outside.
             return getAddRecExpr(
                 getExtendAddRecStart<SCEVZeroExtendExpr>(AR, Ty, this,
                                                          Depth + 1),
                 getSignExtendExpr(Step, Ty, Depth + 1), L,
                 AR->getNoWrapFlags());
           }
         }
if (AR->isAffine()) {
         // is safe.
         if (isKnownPositive(Step)) {
           const SCEV *N = getConstant(APInt::getMinValue(BitWidth) -
                                       getUnsignedRangeMax(Step));
           if (isLoopBackedgeGuardedByCond(L, ICmpInst::ICMP_ULT, AR, N) ||
               isKnownOnEveryIteration(ICmpInst::ICMP_ULT, AR, N)) {
             // Cache knowledge of AR NUW, which is propagated to this
             // AddRec.
-            const_cast<SCEVAddRecExpr *>(AR)->setNoWrapFlags(SCEV::FlagNUW);
+            setNoWrapFlags(const_cast<SCEVAddRecExpr *>(AR), SCEV::FlagNUW);
             // Return the expression with the addrec on the outside.
             return getAddRecExpr(
                 getExtendAddRecStart<SCEVZeroExtendExpr>(AR, Ty, this,
                                                          Depth + 1),
                 getZeroExtendExpr(Step, Ty, Depth + 1), L,
                 AR->getNoWrapFlags());
           }
         } else if (isKnownNegative(Step)) {
           const SCEV *N = getConstant(APInt::getMaxValue(BitWidth) -
                                       getSignedRangeMin(Step));
           if (isLoopBackedgeGuardedByCond(L, ICmpInst::ICMP_UGT, AR, N) ||
               isKnownOnEveryIteration(ICmpInst::ICMP_UGT, AR, N)) {
             // Cache knowledge of AR NW, which is propagated to this
             // AddRec.  Negative step causes unsigned wrap, but it
             // still can't self-wrap.
-            const_cast<SCEVAddRecExpr *>(AR)->setNoWrapFlags(SCEV::FlagNW);
+            setNoWrapFlags(const_cast<SCEVAddRecExpr *>(AR), SCEV::FlagNW);
             // Return the expression with the addrec on the outside.
             return getAddRecExpr(
                 getExtendAddRecStart<SCEVZeroExtendExpr>(AR, Ty, this,
                                                          Depth + 1),
                 getSignExtendExpr(Step, Ty, Depth + 1), L,
                 AR->getNoWrapFlags());
           }
         }
if (AR->isAffine()) {
           const SCEV *SZExtR = getZeroExtendExpr(SResidual, Ty, Depth + 1);
           return getAddExpr(SZExtD, SZExtR,
                             (SCEV::NoWrapFlags)(SCEV::FlagNSW | SCEV::FlagNUW),
                             Depth + 1);
         }
       }
 
       if (proveNoWrapByVaryingStart<SCEVZeroExtendExpr>(Start, Step, L)) {
-        const_cast<SCEVAddRecExpr *>(AR)->setNoWrapFlags(SCEV::FlagNUW);
+        setNoWrapFlags(const_cast<SCEVAddRecExpr *>(AR), SCEV::FlagNUW);
         return getAddRecExpr(
             getExtendAddRecStart<SCEVZeroExtendExpr>(AR, Ty, this, Depth + 1),
             getZeroExtendExpr(Step, Ty, Depth + 1), L, AR->getNoWrapFlags());
       }
     }
 
   // zext(A % B) --> zext(A) % zext(B)
   {
if (const SCEVAddRecExpr *AR = dyn_cast<SCEVAddRecExpr>(Op))
     if (AR->isAffine()) {
       const SCEV *Start = AR->getStart();
       const SCEV *Step = AR->getStepRecurrence(*this);
       unsigned BitWidth = getTypeSizeInBits(AR->getType());
       const Loop *L = AR->getLoop();
 
       if (!AR->hasNoSignedWrap()) {
         auto NewFlags = proveNoWrapViaConstantRanges(AR);
-        const_cast<SCEVAddRecExpr *>(AR)->setNoWrapFlags(NewFlags);
+        setNoWrapFlags(const_cast<SCEVAddRecExpr *>(AR), NewFlags);
       }
 
       // If we have special knowledge that this addrec won't overflow,
       // we don't need to do any further analysis.
       if (AR->hasNoSignedWrap())
         return getAddRecExpr(
             getExtendAddRecStart<SCEVSignExtendExpr>(AR, Ty, this, Depth + 1),
             getSignExtendExpr(Step, Ty, Depth + 1), L, SCEV::FlagNSW);
if (AR->isAffine()) {
           const SCEV *OperandExtendedAdd =
             getAddExpr(WideStart,
                        getMulExpr(WideMaxBECount,
                                   getSignExtendExpr(Step, WideTy, Depth + 1),
                                   SCEV::FlagAnyWrap, Depth + 1),
                        SCEV::FlagAnyWrap, Depth + 1);
           if (SAdd == OperandExtendedAdd) {
             // Cache knowledge of AR NSW, which is propagated to this AddRec.
-            const_cast<SCEVAddRecExpr *>(AR)->setNoWrapFlags(SCEV::FlagNSW);
+            setNoWrapFlags(const_cast<SCEVAddRecExpr *>(AR), SCEV::FlagNSW);
             // Return the expression with the addrec on the outside.
             return getAddRecExpr(
                 getExtendAddRecStart<SCEVSignExtendExpr>(AR, Ty, this,
                                                          Depth + 1),
                 getSignExtendExpr(Step, Ty, Depth + 1), L,
                 AR->getNoWrapFlags());
           }
           // Similar to above, only this time treat the step value as unsigned.
           // This covers loops that count up with an unsigned step.
           OperandExtendedAdd =
             getAddExpr(WideStart,
                        getMulExpr(WideMaxBECount,
                                   getZeroExtendExpr(Step, WideTy, Depth + 1),
                                   SCEV::FlagAnyWrap, Depth + 1),
                        SCEV::FlagAnyWrap, Depth + 1);
           if (SAdd == OperandExtendedAdd) {
             // If AR wraps around then
             //
             //    abs(Step) * MaxBECount > unsigned-max(AR->getType())
             // => SAdd != OperandExtendedAdd
             //
             // Thus (AR is not NW => SAdd != OperandExtendedAdd) <=>
             // (SAdd == OperandExtendedAdd => AR is NW)
 
-            const_cast<SCEVAddRecExpr *>(AR)->setNoWrapFlags(SCEV::FlagNW);
+            setNoWrapFlags(const_cast<SCEVAddRecExpr *>(AR), SCEV::FlagNW);
 
             // Return the expression with the addrec on the outside.
             return getAddRecExpr(
                 getExtendAddRecStart<SCEVSignExtendExpr>(AR, Ty, this,
                                                          Depth + 1),
                 getZeroExtendExpr(Step, Ty, Depth + 1), L,
                 AR->getNoWrapFlags());
           }
if (AR->isAffine()) {
         // is safe.
         ICmpInst::Predicate Pred;
         const SCEV *OverflowLimit =
             getSignedOverflowLimitForStep(Step, &Pred, this);
         if (OverflowLimit &&
             (isLoopBackedgeGuardedByCond(L, Pred, AR, OverflowLimit) ||
              isKnownOnEveryIteration(Pred, AR, OverflowLimit))) {
           // Cache knowledge of AR NSW, then propagate NSW to the wide AddRec.
-          const_cast<SCEVAddRecExpr *>(AR)->setNoWrapFlags(SCEV::FlagNSW);
+          setNoWrapFlags(const_cast<SCEVAddRecExpr *>(AR), SCEV::FlagNSW);
           return getAddRecExpr(
               getExtendAddRecStart<SCEVSignExtendExpr>(AR, Ty, this, Depth + 1),
               getSignExtendExpr(Step, Ty, Depth + 1), L, AR->getNoWrapFlags());
         }
       }
 
       // sext({C,+,Step}) --> (sext(D) + sext({C-D,+,Step}))<nuw><nsw>
       // if D + (C - D + Step * n) could be proven to not signed wrap
       // where D maximizes the number of trailing zeros of (C - D + Step * n)
       if (const auto *SC = dyn_cast<SCEVConstant>(Start)) {
         const APInt &C = SC->getAPInt();
         const APInt &D = extractConstantWithoutWrapping(*this, C, Step);
         if (D != 0) {
           const SCEV *SSExtD = getSignExtendExpr(getConstant(D), Ty, Depth);
           const SCEV *SResidual =
               getAddRecExpr(getConstant(C - D), Step, L, AR->getNoWrapFlags());
           const SCEV *SSExtR = getSignExtendExpr(SResidual, Ty, Depth + 1);
           return getAddExpr(SSExtD, SSExtR,
                             (SCEV::NoWrapFlags)(SCEV::FlagNSW | SCEV::FlagNUW),
                             Depth + 1);
         }
       }
 
       if (proveNoWrapByVaryingStart<SCEVSignExtendExpr>(Start, Step, L)) {
-        const_cast<SCEVAddRecExpr *>(AR)->setNoWrapFlags(SCEV::FlagNSW);
+        setNoWrapFlags(const_cast<SCEVAddRecExpr *>(AR), SCEV::FlagNSW);
         return getAddRecExpr(
             getExtendAddRecStart<SCEVSignExtendExpr>(AR, Ty, this, Depth + 1),
             getSignExtendExpr(Step, Ty, Depth + 1), L, AR->getNoWrapFlags());
       }
     }
 
   // If the input value is provably positive and we could not simplify
   // away the sext build a zext instead.
ScalarEvolution::getOrCreateAddRecExpr(ArrayRef<const SCEV *> Ops,
   if (!S) {
     const SCEV **O = SCEVAllocator.Allocate<const SCEV *>(Ops.size());
     std::uninitialized_copy(Ops.begin(), Ops.end(), O);
     S = new (SCEVAllocator)
         SCEVAddRecExpr(ID.Intern(SCEVAllocator), O, Ops.size(), L);
     UniqueSCEVs.InsertNode(S, IP);
     addToLoopUseLists(S);
   }
-  S->setNoWrapFlags(Flags);
+  setNoWrapFlags(S, Flags);
   return S;
 }
 
 const SCEV *
 ScalarEvolution::getOrCreateMulExpr(ArrayRef<const SCEV *> Ops,
                                     SCEV::NoWrapFlags Flags) {
   FoldingSetNodeID ID;
   ID.AddInteger(scMulExpr);

 static Optional<ConstantRange> GetRangeFromMetadata(Value *V) {
   if (Instruction *I = dyn_cast<Instruction>(V))
     if (MDNode *MD = I->getMetadata(LLVMContext::MD_range))
       return getConstantRangeFromMetadata(*MD);
 
   return None;
 }
 
+void ScalarEvolution::setNoWrapFlags(SCEVAddRecExpr *AddRec,
+                                     SCEV::NoWrapFlags Flags) {
+  if (AddRec->getNoWrapFlags(Flags) != Flags) {
+    AddRec->setNoWrapFlags(Flags);
+    UnsignedRanges.erase(AddRec);
+    SignedRanges.erase(AddRec);
+  }
+}
+
 /// Determine the range for a particular SCEV.  If SignHint is
 /// HINT_RANGE_UNSIGNED (resp. HINT_RANGE_SIGNED) then getRange prefers ranges
 /// with a "cleaner" unsigned (resp. signed) representation.
 const ConstantRange &
 ScalarEvolution::getRangeRef(const SCEV *S,
                              ScalarEvolution::RangeSignHint SignHint) {
   DenseMap<const SCEV *, ConstantRange> &Cache =
       SignHint == ScalarEvolution::HINT_RANGE_UNSIGNED ? UnsignedRanges

llvm/test/Analysis/ScalarEvolution/srem.ll

 ; CHECK-NEXT:    --> %1 U: full-set S: full-set Exits: <<Unknown>> LoopDispositions: { %for.cond: Variant }
 ; CHECK-NEXT:    %call = call i32 @_Z3adji(i32 %1)
 ; CHECK-NEXT:    --> %call U: full-set S: full-set Exits: <<Unknown>> LoopDispositions: { %for.cond: Variant }
 ; CHECK-NEXT:    %2 = load i32, i32* %arrayidx, align 4
 ; CHECK-NEXT:    --> %2 U: full-set S: full-set Exits: <<Unknown>> LoopDispositions: { %for.cond: Variant }
 ; CHECK-NEXT:    %add = add nsw i32 %2, %call
 ; CHECK-NEXT:    --> (%2 + %call) U: full-set S: full-set Exits: <<Unknown>> LoopDispositions: { %for.cond: Variant }
 ; CHECK-NEXT:    %inc = add nsw i32 %i.0, 1
-; CHECK-NEXT:    --> {1,+,1}<nuw><%for.cond> U: full-set S: full-set Exits: (1 + %width) LoopDispositions: { %for.cond: Computable }
+; CHECK-NEXT:    --> {1,+,1}<nuw><%for.cond> U: [1,0) S: [1,0) Exits: (1 + %width) LoopDispositions: { %for.cond: Computable }
 ; CHECK-NEXT:  Determining loop execution counts for: @_Z4loopi
 ; CHECK-NEXT:  Loop %for.cond: backedge-taken count is %width
 ; CHECK-NEXT:  Loop %for.cond: max backedge-taken count is -1
 ; CHECK-NEXT:  Loop %for.cond: Predicated backedge-taken count is %width
 ; CHECK-NEXT:   Predicates:
 ; CHECK:       Loop %for.cond: Trip multiple is 1
 ;
 entry:

llvm/test/Transforms/IndVarSimplify/floating-point-iv.ll

 ; CHECK-NEXT:  bb4:
 ; CHECK-NEXT:    br label [[BB8:%.*]]
 ; CHECK:       bb8:
 ; CHECK-NEXT:    [[TMP10:%.*]] = phi double [ 0.000000e+00, [[BB4:%.*]] ], [ [[TMP12:%.*]], [[BB22:%.*]] ]
 ; CHECK-NEXT:    [[TMP11_INT:%.*]] = phi i32 [ 0, [[BB4]] ], [ [[TMP13_INT:%.*]], [[BB22]] ]
 ; CHECK-NEXT:    [[INDVAR_CONV:%.*]] = sitofp i32 [[TMP11_INT]] to double
 ; CHECK-NEXT:    [[TMP12]] = fadd double [[TMP10]], [[INDVAR_CONV]]
 ; CHECK-NEXT:    [[TMP13_INT]] = add nuw nsw i32 [[TMP11_INT]], 1
-; CHECK-NEXT:    [[TMP14:%.*]] = icmp slt i32 [[TMP13_INT]], 99999
+; CHECK-NEXT:    [[TMP14:%.*]] = icmp ult i32 [[TMP13_INT]], 99999
 ; CHECK-NEXT:    br i1 [[TMP14]], label [[BB22]], label [[BB6:%.*]]
 ; CHECK:       bb22:
 ; CHECK-NEXT:    br i1 true, label [[BB8]], label [[BB6]]
 ; CHECK:       bb6:
 ; CHECK-NEXT:    [[TMP12_LCSSA:%.*]] = phi double [ [[TMP12]], [[BB22]] ], [ [[TMP12]], [[BB8]] ]
 ; CHECK-NEXT:    ret double [[TMP12_LCSSA]]
 ;
 bb4:

 ; CHECK-NEXT:  bb4:
 ; CHECK-NEXT:    br label [[BB8:%.*]]
 ; CHECK:       bb8:
 ; CHECK-NEXT:    [[TMP10:%.*]] = phi float [ 0.000000e+00, [[BB4:%.*]] ], [ [[TMP12:%.*]], [[BB22:%.*]] ]
 ; CHECK-NEXT:    [[TMP11_INT:%.*]] = phi i32 [ 0, [[BB4]] ], [ [[TMP13_INT:%.*]], [[BB22]] ]
 ; CHECK-NEXT:    [[INDVAR_CONV:%.*]] = sitofp i32 [[TMP11_INT]] to float
 ; CHECK-NEXT:    [[TMP12]] = fadd float [[TMP10]], [[INDVAR_CONV]]
 ; CHECK-NEXT:    [[TMP13_INT]] = add nuw nsw i32 [[TMP11_INT]], 1
-; CHECK-NEXT:    [[TMP14:%.*]] = icmp slt i32 [[TMP13_INT]], 99999
+; CHECK-NEXT:    [[TMP14:%.*]] = icmp ult i32 [[TMP13_INT]], 99999
 ; CHECK-NEXT:    br i1 [[TMP14]], label [[BB22]], label [[BB6:%.*]]
 ; CHECK:       bb22:
 ; CHECK-NEXT:    br i1 true, label [[BB8]], label [[BB6]]
 ; CHECK:       bb6:
 ; CHECK-NEXT:    [[TMP12_LCSSA:%.*]] = phi float [ [[TMP12]], [[BB22]] ], [ [[TMP12]], [[BB8]] ]
 ; CHECK-NEXT:    ret float [[TMP12_LCSSA]]
 ;
 bb4:

 ; CHECK-NEXT:  bb4:
 ; CHECK-NEXT:    br label [[BB8:%.*]]
 ; CHECK:       bb8:
 ; CHECK-NEXT:    [[TMP10:%.*]] = phi float [ 0.000000e+00, [[BB4:%.*]] ], [ [[TMP12:%.*]], [[BB22:%.*]] ]
 ; CHECK-NEXT:    [[TMP11_INT:%.*]] = phi i32 [ 0, [[BB4]] ], [ [[TMP13_INT:%.*]], [[BB22]] ]
 ; CHECK-NEXT:    [[INDVAR_CONV:%.*]] = sitofp i32 [[TMP11_INT]] to float
 ; CHECK-NEXT:    [[TMP12]] = fadd float [[TMP10]], [[INDVAR_CONV]]
 ; CHECK-NEXT:    [[TMP13_INT]] = add nuw nsw i32 [[TMP11_INT]], 1
-; CHECK-NEXT:    [[TMP14:%.*]] = icmp slt i32 [[TMP13_INT]], 99999
+; CHECK-NEXT:    [[TMP14:%.*]] = icmp ult i32 [[TMP13_INT]], 99999
 ; CHECK-NEXT:    br i1 [[TMP14]], label [[BB22]], label [[BB6:%.*]]
 ; CHECK:       bb22:
 ; CHECK-NEXT:    br i1 true, label [[BB8]], label [[BB6]]
 ; CHECK:       bb6:
 ; CHECK-NEXT:    [[TMP12_LCSSA:%.*]] = phi float [ [[TMP12]], [[BB22]] ], [ [[TMP12]], [[BB8]] ]
 ; CHECK-NEXT:    ret float [[TMP12_LCSSA]]
 ;
 bb4: