[SimplifyCFG] Two entry phi select costs

[samparker]

Summary

In same manner as D82438, but in FoldTwoEntryPHINode, estimate the cost needed to insert the required selects and count this against the given budget. This will prevent some high cost conversions which IfConversion can later evaluate with better accuracy.

[lebedev.ri]

llvm/lib/Transforms/Utils/SimplifyCFG.cpp

98–102

You need to bump it by that new cmp+sel cost, so +2.

2000

Put this into D86346?

2011

I'm not sure this does what you think it does..

[samparker]

llvm/lib/Transforms/Utils/SimplifyCFG.cpp

98–102

Where is the cmp coming from?

2011

Do you think this is wrong? What do you think it should be doing?

[lebedev.ri]

llvm/lib/Transforms/Utils/SimplifyCFG.cpp

98–102

Looking at getCmpSelInstrCost(), right, only the select, so +1.

[lebedev.ri]

llvm/lib/Transforms/Utils/SimplifyCFG.cpp

2011

Well, the function takes BudgetRemaining by value,
so this doesn't actually change the BudgetRemaining in the caller function.

[samparker]

llvm/lib/Transforms/Utils/SimplifyCFG.cpp

2011

Right, this patch will need to be changed to work with D86346, but as a standalone patch it does what it should.

[samparker]

Rebased after moving the select cost and validation into a standalone function. I've kept the threshold at 4 as a default, because this causes far fewer codegen changes (X86, AArch64, Arm) than trying to adjust it to accommodate the select by bumping it up to 5.

[lebedev.ri]

So we have two thresholds at play here?
That doesn't seem right to me..

llvm/lib/Transforms/Utils/SimplifyCFG.cpp

1997–2018

here

1997–2018

and here

[lebedev.ri]

Hold on, how does this interact with D86346?

[lebedev.ri]

This review seems to be stuck/dead, consider abandoning if no longer relevant.

llvm/lib/Transforms/Utils/SimplifyCFG.cpp

 // a select, so the "clamp" idiom (of a min followed by a max) will be caught.
 // To catch this, we need to fold a compare and a select, hence '2' being the
 // minimum reasonable default.
 static cl::opt<unsigned> PHINodeFoldingThreshold(
     "phi-node-folding-threshold", cl::Hidden, cl::init(2),
     cl::desc(
         "Control the amount of phi node folding to perform (default = 2)"));
 
 static cl::opt<unsigned> TwoEntryPHINodeFoldingThreshold(
     "two-entry-phi-node-folding-threshold", cl::Hidden, cl::init(4),
     cl::desc("Control the maximal total instruction cost that we are willing "
              "to speculatively execute to fold a 2-entry PHI node into a "
              "select (default = 4)"));
 
 static cl::opt<bool> DupRet(
     "simplifycfg-dup-ret", cl::Hidden, cl::init(false),
     cl::desc("Duplicate return instructions into unconditional branches"));
 
 static cl::opt<bool>
     HoistCommon("simplifycfg-hoist-common", cl::Hidden, cl::init(true),
                 cl::desc("Hoist common instructions up to the parent block"));
if (auto *SI = dyn_cast<StoreInst>(&CurI)) {
         // Found the previous store, return its value operand.
         return SI->getValueOperand();
       return nullptr; // Unknown store.
     }
   }
 
   return nullptr;
 }
 
+/// Check if the cost of the necessary selects will be within the given
+/// budget.
+static bool selectCostsWithinBudget(BasicBlock *BB, BasicBlock *ThenBB,
+                                    BasicBlock *EndBB, int BudgetRemaining,
+                                    const TargetTransformInfo &TTI) {
+  TargetTransformInfo::TargetCostKind CostKind =
+    BB->getParent()->hasMinSize()
+    ? TargetTransformInfo::TCK_CodeSize
+    : TargetTransformInfo::TCK_SizeAndLatency;
+  for (PHINode &PN : EndBB->phis()) {
+    Value *OrigV = PN.getIncomingValueForBlock(BB);
+    Value *ThenV = PN.getIncomingValueForBlock(ThenBB);
+    if (OrigV != ThenV)
+      BudgetRemaining -=
+          TTI.getCmpSelInstrCost(Instruction::Select, PN.getType(), nullptr,
+                                 CostKind);
+  }
+  return BudgetRemaining >= 0;
+}
+
 /// Speculate a conditional basic block flattening the CFG.
 ///
 /// Note that this is a very risky transform currently. Speculating
 /// instructions like this is most often not desirable. Instead, there is an MI
 /// pass which can do it with full awareness of the resource constraints.
 /// However, some cases are "obvious" and we should do directly. An example of
 /// this is speculating a single, reasonably cheap instruction.
 ///
 /// There is only one distinct advantage to flattening the CFG at the IR level:
bool SimplifyCFGOpt::SpeculativelyExecuteBB(BranchInst *BI, BasicBlock *ThenBB,
   // Be conservative for now. FP select instruction can often be expensive.
   Value *BrCond = BI->getCondition();
   if (isa<FCmpInst>(BrCond))
     return false;
 
   BasicBlock *BB = BI->getParent();
   BasicBlock *EndBB = ThenBB->getTerminator()->getSuccessor(0);
 
-  TargetTransformInfo::TargetCostKind CostKind =
-    BI->getFunction()->hasMinSize()
-    ? TargetTransformInfo::TCK_CodeSize
-    : TargetTransformInfo::TCK_SizeAndLatency;
   // Check how expensive it will be to insert the necessary selects.
   int BudgetRemaining =
       PHINodeFoldingThreshold * TargetTransformInfo::TCC_Basic;
-  for (PHINode &PN : EndBB->phis()) {
-    unsigned OrigI = PN.getBasicBlockIndex(BB);
-    unsigned ThenI = PN.getBasicBlockIndex(ThenBB);
-    Value *OrigV = PN.getIncomingValue(OrigI);
-    Value *ThenV = PN.getIncomingValue(ThenI);
-    if (OrigV != ThenV)
-      BudgetRemaining -=
-          TTI.getCmpSelInstrCost(Instruction::Select, PN.getType(), nullptr,
-                                 CostKind);
-  }
-  if (BudgetRemaining < 0)
+
+  if (!selectCostsWithinBudget(BB, ThenBB, EndBB, BudgetRemaining, TTI))
     return false;
 
   // If ThenBB is actually on the false edge of the conditional branch, remember
   // to swap the select operands later.
   bool Invert = false;
   if (ThenBB != BI->getSuccessor(0)) {
     assert(ThenBB == BI->getSuccessor(1) && "No edge from 'if' block?");
     Invert = true;
static bool FoldTwoEntryPHINode(PHINode *PN, const TargetTransformInfo &TTI,
 
   // Move all 'aggressive' instructions, which are defined in the
   // conditional parts of the if's up to the dominating block.
   if (IfBlock1)
     hoistAllInstructionsInto(DomBlock, InsertPt, IfBlock1);
   if (IfBlock2)
     hoistAllInstructionsInto(DomBlock, InsertPt, IfBlock2);
 
+  // Check whether the cost of the required selects will surpass the budget.
+  if (!selectCostsWithinBudget(IfTrue, IfFalse, BB, BudgetRemaining, TTI))
+    return Changed;
+
   // Propagate fast-math-flags from phi nodes to replacement selects.
   IRBuilder<>::FastMathFlagGuard FMFGuard(Builder);
   while (PHINode *PN = dyn_cast<PHINode>(BB->begin())) {
     if (isa<FPMathOperator>(PN))
       Builder.setFastMathFlags(PN->getFastMathFlags());
 
     // Change the PHI node into a select instruction.
     Value *TrueVal = PN->getIncomingValue(PN->getIncomingBlock(0) == IfFalse);

llvm/test/Transforms/PGOProfile/chr.ll

 ; CHECK-NEXT:    [[TMP11:%.*]] = add i32 [[SUM1_NONCHR]], 43
 ; CHECK-NEXT:    [[SUM2_NONCHR:%.*]] = select i1 [[TMP10]], i32 [[SUM1_NONCHR]], i32 [[TMP11]], !prof !16
 ; CHECK-NEXT:    [[TMP12:%.*]] = and i32 [[TMP0]], 4
 ; CHECK-NEXT:    [[TMP13:%.*]] = icmp eq i32 [[TMP12]], 0
 ; CHECK-NEXT:    [[TMP14:%.*]] = and i32 [[TMP0]], 8
 ; CHECK-NEXT:    [[TMP15:%.*]] = icmp eq i32 [[TMP14]], 0
 ; CHECK-NEXT:    [[SUM4_NONCHR_V:%.*]] = select i1 [[TMP15]], i32 44, i32 88
 ; CHECK-NEXT:    [[SUM4_NONCHR:%.*]] = add i32 [[SUM2_NONCHR]], [[SUM4_NONCHR_V]]
-; CHECK-NEXT:    [[SUM5_NONCHR:%.*]] = select i1 [[TMP13]], i32 [[SUM2_NONCHR]], i32 [[SUM4_NONCHR]], !prof !16
-; CHECK-NEXT:    br label [[BB3]]
+; CHECK-NEXT:    [[SPEC_SELECT:%.*]] = select i1 [[TMP13]], i32 [[SUM2_NONCHR]], i32 [[SUM4_NONCHR]], !prof !16
+; CHECK-NEXT:    ret i32 [[SPEC_SELECT]]
 ; CHECK:       bb3:
-; CHECK-NEXT:    [[SUM6:%.*]] = phi i32 [ [[TMP4]], [[BB0]] ], [ [[SUM0]], [[ENTRY_SPLIT_NONCHR]] ], [ [[SUM5_NONCHR]], [[BB0_NONCHR]] ]
+; CHECK-NEXT:    [[SUM6:%.*]] = phi i32 [ [[TMP4]], [[BB0]] ], [ [[SUM0]], [[ENTRY_SPLIT_NONCHR]] ]
 ; CHECK-NEXT:    ret i32 [[SUM6]]
 ;
 entry:
   %0 = load i32, i32* %i
   %1 = and i32 %0, 255
   %2 = icmp eq i32 %1, 0
   br i1 %2, label %bb3, label %bb0, !prof !15
 

 ; CHECK-NEXT:    [[TMP14:%.*]] = add i32 [[SUM1_NONCHR]], 43
 ; CHECK-NEXT:    [[SUM2_NONCHR:%.*]] = select i1 [[TMP13]], i32 [[SUM1_NONCHR]], i32 [[TMP14]], !prof !16
 ; CHECK-NEXT:    [[TMP15:%.*]] = and i32 [[SUM0]], 4
 ; CHECK-NEXT:    [[TMP16:%.*]] = icmp eq i32 [[TMP15]], 0
 ; CHECK-NEXT:    [[TMP17:%.*]] = and i32 [[TMP0]], 8
 ; CHECK-NEXT:    [[TMP18:%.*]] = icmp eq i32 [[TMP17]], 0
 ; CHECK-NEXT:    [[SUM4_NONCHR_V:%.*]] = select i1 [[TMP18]], i32 44, i32 88
 ; CHECK-NEXT:    [[SUM4_NONCHR:%.*]] = add i32 [[SUM2_NONCHR]], [[SUM4_NONCHR_V]]
-; CHECK-NEXT:    [[SUM5_NONCHR:%.*]] = select i1 [[TMP16]], i32 [[SUM2_NONCHR]], i32 [[SUM4_NONCHR]], !prof !16
-; CHECK-NEXT:    br label [[BB3]]
+; CHECK-NEXT:    [[SPEC_SELECT:%.*]] = select i1 [[TMP16]], i32 [[SUM2_NONCHR]], i32 [[SUM4_NONCHR]], !prof !16
+; CHECK-NEXT:    ret i32 [[SPEC_SELECT]]
 ; CHECK:       bb3:
-; CHECK-NEXT:    [[SUM6:%.*]] = phi i32 [ [[TMP7]], [[BB0]] ], [ [[SUM0]], [[ENTRY_SPLIT_NONCHR]] ], [ [[SUM5_NONCHR]], [[BB0_NONCHR]] ]
+; CHECK-NEXT:    [[SUM6:%.*]] = phi i32 [ [[TMP7]], [[BB0]] ], [ [[SUM0]], [[ENTRY_SPLIT_NONCHR]] ]
 ; CHECK-NEXT:    ret i32 [[SUM6]]
 ;
 entry:
   %0 = load i32, i32* %i
   %1 = and i32 %0, 255
   %2 = icmp eq i32 %1, 0
   br i1 %2, label %bb3, label %bb0, !prof !15
 

 ; CHECK-NEXT:    [[V8_NONCHR:%.*]] = add i32 [[SUM0]], 43
 ; CHECK-NEXT:    [[SUM2_NONCHR:%.*]] = select i1 [[V4_NONCHR]], i32 [[SUM0]], i32 [[V8_NONCHR]], !prof !16
 ; CHECK-NEXT:    [[V9_NONCHR:%.*]] = and i32 [[J0]], 4
 ; CHECK-NEXT:    [[V10_NONCHR:%.*]] = icmp eq i32 [[V9_NONCHR]], 0
 ; CHECK-NEXT:    [[V11_NONCHR:%.*]] = and i32 [[I0]], 8
 ; CHECK-NEXT:    [[V12_NONCHR:%.*]] = icmp eq i32 [[V11_NONCHR]], 0
 ; CHECK-NEXT:    [[SUM4_NONCHR_V:%.*]] = select i1 [[V12_NONCHR]], i32 44, i32 88
 ; CHECK-NEXT:    [[SUM4_NONCHR:%.*]] = add i32 [[SUM2_NONCHR]], [[SUM4_NONCHR_V]]
-; CHECK-NEXT:    [[SUM5_NONCHR:%.*]] = select i1 [[V10_NONCHR]], i32 [[SUM2_NONCHR]], i32 [[SUM4_NONCHR]], !prof !16
-; CHECK-NEXT:    br label [[BB3]]
+; CHECK-NEXT:    [[SPEC_SELECT:%.*]] = select i1 [[V10_NONCHR]], i32 [[SUM2_NONCHR]], i32 [[SUM4_NONCHR]], !prof !16
+; CHECK-NEXT:    ret i32 [[SPEC_SELECT]]
 ; CHECK:       bb3:
-; CHECK-NEXT:    [[SUM6:%.*]] = phi i32 [ [[V13]], [[BB0]] ], [ [[SUM0]], [[ENTRY_SPLIT_NONCHR]] ], [ [[SUM5_NONCHR]], [[BB0_NONCHR]] ]
+; CHECK-NEXT:    [[SUM6:%.*]] = phi i32 [ [[V13]], [[BB0]] ], [ [[SUM0]], [[ENTRY_SPLIT_NONCHR]] ]
 ; CHECK-NEXT:    ret i32 [[SUM6]]
 ;
 entry:
   %i0 = load i32, i32* %i
   %j0 = load i32, i32* %j
   %v1 = and i32 %i0, 255
   %v2 = icmp eq i32 %v1, 0
   br i1 %v2, label %bb3, label %bb0, !prof !15

 ; CHECK-NEXT:    [[TMP6:%.*]] = and i32 [[TMP0]], 1
 ; CHECK-NEXT:    [[TMP7:%.*]] = icmp eq i32 [[TMP6]], 0
 ; CHECK-NEXT:    [[TMP8:%.*]] = add i32 [[SUM0]], 85
 ; CHECK-NEXT:    [[SUM2_NONCHR:%.*]] = select i1 [[TMP7]], i32 [[SUM0]], i32 [[TMP8]], !prof !16
 ; CHECK-NEXT:    [[TMP9:%.*]] = and i32 [[TMP0]], 8
 ; CHECK-NEXT:    [[TMP10:%.*]] = icmp eq i32 [[TMP9]], 0
 ; CHECK-NEXT:    [[SUM4_NONCHR_V:%.*]] = select i1 [[TMP10]], i32 44, i32 88
 ; CHECK-NEXT:    [[SUM4_NONCHR:%.*]] = add i32 [[SUM2_NONCHR]], [[SUM4_NONCHR_V]]
-; CHECK-NEXT:    [[SUM5_NONCHR:%.*]] = select i1 [[TMP7]], i32 [[SUM2_NONCHR]], i32 [[SUM4_NONCHR]], !prof !16
-; CHECK-NEXT:    br label [[BB3]]
+; CHECK-NEXT:    [[SPEC_SELECT:%.*]] = select i1 [[TMP7]], i32 [[SUM2_NONCHR]], i32 [[SUM4_NONCHR]], !prof !16
+; CHECK-NEXT:    ret i32 [[SPEC_SELECT]]
 ; CHECK:       bb3:
-; CHECK-NEXT:    [[SUM6:%.*]] = phi i32 [ [[TMP4]], [[BB0]] ], [ [[SUM0]], [[ENTRY_SPLIT_NONCHR]] ], [ [[SUM5_NONCHR]], [[BB0_NONCHR]] ]
+; CHECK-NEXT:    [[SUM6:%.*]] = phi i32 [ [[TMP4]], [[BB0]] ], [ [[SUM0]], [[ENTRY_SPLIT_NONCHR]] ]
 ; CHECK-NEXT:    ret i32 [[SUM6]]
 ;
 entry:
   %0 = load i32, i32* %i
   %1 = and i32 %0, 255
   %2 = icmp eq i32 %1, 0
   br i1 %2, label %bb3, label %bb0, !prof !15
 

llvm/test/Transforms/SimplifyCFG/ARM/phi-eliminate.ll

 ; CHECK-V8M-TWO-FOLD-4-LABEL: @test_i32(
 ; CHECK-V8M-TWO-FOLD-4-NEXT:  entry:
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
 ; CHECK-V8M-TWO-FOLD-4:       O:
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[IAJ:%.*]] = add i32 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[IAJAK:%.*]] = add i32 [[IAJ]], [[K:%.*]]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[IXJ:%.*]] = xor i32 [[I]], [[J]]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[IXJXK:%.*]] = xor i32 [[IXJ]], [[K]]
-; CHECK-V8M-TWO-FOLD-4-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i32 [[IAJAK]], i32 [[IXJXK]]
+; CHECK-V8M-TWO-FOLD-4-NEXT:    br i1 [[B:%.*]], label [[P:%.*]], label [[Q:%.*]]
+; CHECK-V8M-TWO-FOLD-4:       P:
+; CHECK-V8M-TWO-FOLD-4-NEXT:    br label [[N:%.*]]
+; CHECK-V8M-TWO-FOLD-4:       Q:
+; CHECK-V8M-TWO-FOLD-4-NEXT:    br label [[N]]
+; CHECK-V8M-TWO-FOLD-4:       N:
+; CHECK-V8M-TWO-FOLD-4-NEXT:    [[WP:%.*]] = phi i32 [ [[IAJAK]], [[P]] ], [ [[IXJXK]], [[Q]] ]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[WP2:%.*]] = add i32 [[WP]], [[WP]]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    br label [[M]]
 ; CHECK-V8M-TWO-FOLD-4:       M:
-; CHECK-V8M-TWO-FOLD-4-NEXT:    [[W:%.*]] = phi i32 [ [[WP2]], [[O]] ], [ 2, [[ENTRY:%.*]] ]
+; CHECK-V8M-TWO-FOLD-4-NEXT:    [[W:%.*]] = phi i32 [ [[WP2]], [[N]] ], [ 2, [[ENTRY:%.*]] ]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[R:%.*]] = add i32 [[W]], 1
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    ret i32 [[R]]
 ;
 ; CHECK-V8A-TWO-FOLD-4-LABEL: @test_i32(
 ; CHECK-V8A-TWO-FOLD-4-NEXT:  entry:
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
 ; CHECK-V8A-TWO-FOLD-4:       O:
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[IAJ:%.*]] = add i32 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[IAJAK:%.*]] = add i32 [[IAJ]], [[K:%.*]]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[IXJ:%.*]] = xor i32 [[I]], [[J]]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[IXJXK:%.*]] = xor i32 [[IXJ]], [[K]]
-; CHECK-V8A-TWO-FOLD-4-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i32 [[IAJAK]], i32 [[IXJXK]]
+; CHECK-V8A-TWO-FOLD-4-NEXT:    br i1 [[B:%.*]], label [[P:%.*]], label [[Q:%.*]]
+; CHECK-V8A-TWO-FOLD-4:       P:
+; CHECK-V8A-TWO-FOLD-4-NEXT:    br label [[N:%.*]]
+; CHECK-V8A-TWO-FOLD-4:       Q:
+; CHECK-V8A-TWO-FOLD-4-NEXT:    br label [[N]]
+; CHECK-V8A-TWO-FOLD-4:       N:
+; CHECK-V8A-TWO-FOLD-4-NEXT:    [[WP:%.*]] = phi i32 [ [[IAJAK]], [[P]] ], [ [[IXJXK]], [[Q]] ]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[WP2:%.*]] = add i32 [[WP]], [[WP]]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    br label [[M]]
 ; CHECK-V8A-TWO-FOLD-4:       M:
-; CHECK-V8A-TWO-FOLD-4-NEXT:    [[W:%.*]] = phi i32 [ [[WP2]], [[O]] ], [ 2, [[ENTRY:%.*]] ]
+; CHECK-V8A-TWO-FOLD-4-NEXT:    [[W:%.*]] = phi i32 [ [[WP2]], [[N]] ], [ 2, [[ENTRY:%.*]] ]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[R:%.*]] = add i32 [[W]], 1
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    ret i32 [[R]]
 ;
 ; CHECK-V8M-TWO-FOLD-5-LABEL: @test_i32(
 ; CHECK-V8M-TWO-FOLD-5-NEXT:  entry:
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
 ; CHECK-V8M-TWO-FOLD-5:       O:
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    [[IAJ:%.*]] = add i32 [[I:%.*]], [[J:%.*]]

 ; CHECK-V8M-TWO-FOLD-6-LABEL: @test_i32(
 ; CHECK-V8M-TWO-FOLD-6-NEXT:  entry:
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[IAJ:%.*]] = add i32 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[IAJAK:%.*]] = add i32 [[IAJ]], [[K:%.*]]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[IXJ:%.*]] = xor i32 [[I]], [[J]]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[IXJXK:%.*]] = xor i32 [[IXJ]], [[K]]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i32 [[IAJAK]], i32 [[IXJXK]]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[WP2:%.*]] = add i32 [[WP]], [[WP]]
-; CHECK-V8M-TWO-FOLD-6-NEXT:    [[W:%.*]] = select i1 [[A:%.*]], i32 2, i32 [[WP2]]
-; CHECK-V8M-TWO-FOLD-6-NEXT:    [[R:%.*]] = add i32 [[W]], 1
+; CHECK-V8M-TWO-FOLD-6-NEXT:    [[SPEC_SELECT:%.*]] = select i1 [[A:%.*]], i32 2, i32 [[WP2]]
+; CHECK-V8M-TWO-FOLD-6-NEXT:    [[R:%.*]] = add i32 [[SPEC_SELECT]], 1
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    ret i32 [[R]]
 ;
 ; CHECK-V8A-TWO-FOLD-6-LABEL: @test_i32(
 ; CHECK-V8A-TWO-FOLD-6-NEXT:  entry:
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[IAJ:%.*]] = add i32 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[IAJAK:%.*]] = add i32 [[IAJ]], [[K:%.*]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[IXJ:%.*]] = xor i32 [[I]], [[J]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[IXJXK:%.*]] = xor i32 [[IXJ]], [[K]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i32 [[IAJAK]], i32 [[IXJXK]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[WP2:%.*]] = add i32 [[WP]], [[WP]]
-; CHECK-V8A-TWO-FOLD-6-NEXT:    [[W:%.*]] = select i1 [[A:%.*]], i32 2, i32 [[WP2]]
-; CHECK-V8A-TWO-FOLD-6-NEXT:    [[R:%.*]] = add i32 [[W]], 1
+; CHECK-V8A-TWO-FOLD-6-NEXT:    [[SPEC_SELECT:%.*]] = select i1 [[A:%.*]], i32 2, i32 [[WP2]]
+; CHECK-V8A-TWO-FOLD-6-NEXT:    [[R:%.*]] = add i32 [[SPEC_SELECT]], 1
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    ret i32 [[R]]
 ;
 entry:
   br i1 %a, label %M, label %O
 O:
   br i1 %b, label %P, label %Q
 P:
   %iaj = add i32 %i, %j

 ; CHECK-V8M-TWO-FOLD-4-LABEL: @test_i32_minsize(
 ; CHECK-V8M-TWO-FOLD-4-NEXT:  entry:
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
 ; CHECK-V8M-TWO-FOLD-4:       O:
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[IAJ:%.*]] = add i32 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[IAJAK:%.*]] = add i32 [[IAJ]], [[K:%.*]]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[IXJ:%.*]] = xor i32 [[I]], [[J]]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[IXJXK:%.*]] = xor i32 [[IXJ]], [[K]]
-; CHECK-V8M-TWO-FOLD-4-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i32 [[IAJAK]], i32 [[IXJXK]]
+; CHECK-V8M-TWO-FOLD-4-NEXT:    br i1 [[B:%.*]], label [[P:%.*]], label [[Q:%.*]]
+; CHECK-V8M-TWO-FOLD-4:       P:
+; CHECK-V8M-TWO-FOLD-4-NEXT:    br label [[N:%.*]]
+; CHECK-V8M-TWO-FOLD-4:       Q:
+; CHECK-V8M-TWO-FOLD-4-NEXT:    br label [[N]]
+; CHECK-V8M-TWO-FOLD-4:       N:
+; CHECK-V8M-TWO-FOLD-4-NEXT:    [[WP:%.*]] = phi i32 [ [[IAJAK]], [[P]] ], [ [[IXJXK]], [[Q]] ]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[WP2:%.*]] = add i32 [[WP]], [[WP]]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    br label [[M]]
 ; CHECK-V8M-TWO-FOLD-4:       M:
-; CHECK-V8M-TWO-FOLD-4-NEXT:    [[W:%.*]] = phi i32 [ [[WP2]], [[O]] ], [ 2, [[ENTRY:%.*]] ]
+; CHECK-V8M-TWO-FOLD-4-NEXT:    [[W:%.*]] = phi i32 [ [[WP2]], [[N]] ], [ 2, [[ENTRY:%.*]] ]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[R:%.*]] = add i32 [[W]], 1
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    ret i32 [[R]]
 ;
 ; CHECK-V8A-TWO-FOLD-4-LABEL: @test_i32_minsize(
 ; CHECK-V8A-TWO-FOLD-4-NEXT:  entry:
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
 ; CHECK-V8A-TWO-FOLD-4:       O:
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[IAJ:%.*]] = add i32 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[IAJAK:%.*]] = add i32 [[IAJ]], [[K:%.*]]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[IXJ:%.*]] = xor i32 [[I]], [[J]]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[IXJXK:%.*]] = xor i32 [[IXJ]], [[K]]
-; CHECK-V8A-TWO-FOLD-4-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i32 [[IAJAK]], i32 [[IXJXK]]
+; CHECK-V8A-TWO-FOLD-4-NEXT:    br i1 [[B:%.*]], label [[P:%.*]], label [[Q:%.*]]
+; CHECK-V8A-TWO-FOLD-4:       P:
+; CHECK-V8A-TWO-FOLD-4-NEXT:    br label [[N:%.*]]
+; CHECK-V8A-TWO-FOLD-4:       Q:
+; CHECK-V8A-TWO-FOLD-4-NEXT:    br label [[N]]
+; CHECK-V8A-TWO-FOLD-4:       N:
+; CHECK-V8A-TWO-FOLD-4-NEXT:    [[WP:%.*]] = phi i32 [ [[IAJAK]], [[P]] ], [ [[IXJXK]], [[Q]] ]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[WP2:%.*]] = add i32 [[WP]], [[WP]]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    br label [[M]]
 ; CHECK-V8A-TWO-FOLD-4:       M:
-; CHECK-V8A-TWO-FOLD-4-NEXT:    [[W:%.*]] = phi i32 [ [[WP2]], [[O]] ], [ 2, [[ENTRY:%.*]] ]
+; CHECK-V8A-TWO-FOLD-4-NEXT:    [[W:%.*]] = phi i32 [ [[WP2]], [[N]] ], [ 2, [[ENTRY:%.*]] ]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[R:%.*]] = add i32 [[W]], 1
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    ret i32 [[R]]
 ;
 ; CHECK-V8M-TWO-FOLD-5-LABEL: @test_i32_minsize(
 ; CHECK-V8M-TWO-FOLD-5-NEXT:  entry:
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
 ; CHECK-V8M-TWO-FOLD-5:       O:
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    [[IAJ:%.*]] = add i32 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    [[IAJAK:%.*]] = add i32 [[IAJ]], [[K:%.*]]
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    [[IXJ:%.*]] = xor i32 [[I]], [[J]]
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    [[IXJXK:%.*]] = xor i32 [[IXJ]], [[K]]
-; CHECK-V8M-TWO-FOLD-5-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i32 [[IAJAK]], i32 [[IXJXK]]
+; CHECK-V8M-TWO-FOLD-5-NEXT:    br i1 [[B:%.*]], label [[P:%.*]], label [[Q:%.*]]
+; CHECK-V8M-TWO-FOLD-5:       P:
+; CHECK-V8M-TWO-FOLD-5-NEXT:    br label [[N:%.*]]
+; CHECK-V8M-TWO-FOLD-5:       Q:
+; CHECK-V8M-TWO-FOLD-5-NEXT:    br label [[N]]
+; CHECK-V8M-TWO-FOLD-5:       N:
+; CHECK-V8M-TWO-FOLD-5-NEXT:    [[WP:%.*]] = phi i32 [ [[IAJAK]], [[P]] ], [ [[IXJXK]], [[Q]] ]
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    [[WP2:%.*]] = add i32 [[WP]], [[WP]]
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    br label [[M]]
 ; CHECK-V8M-TWO-FOLD-5:       M:
-; CHECK-V8M-TWO-FOLD-5-NEXT:    [[W:%.*]] = phi i32 [ [[WP2]], [[O]] ], [ 2, [[ENTRY:%.*]] ]
+; CHECK-V8M-TWO-FOLD-5-NEXT:    [[W:%.*]] = phi i32 [ [[WP2]], [[N]] ], [ 2, [[ENTRY:%.*]] ]
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    [[R:%.*]] = add i32 [[W]], 1
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    ret i32 [[R]]
 ;
 ; CHECK-V8A-TWO-FOLD-5-LABEL: @test_i32_minsize(
 ; CHECK-V8A-TWO-FOLD-5-NEXT:  entry:
 ; CHECK-V8A-TWO-FOLD-5-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
 ; CHECK-V8A-TWO-FOLD-5:       O:
 ; CHECK-V8A-TWO-FOLD-5-NEXT:    [[IAJ:%.*]] = add i32 [[I:%.*]], [[J:%.*]]

 ; CHECK-V8M-TWO-FOLD-6-LABEL: @test_i32_minsize(
 ; CHECK-V8M-TWO-FOLD-6-NEXT:  entry:
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[IAJ:%.*]] = add i32 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[IAJAK:%.*]] = add i32 [[IAJ]], [[K:%.*]]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[IXJ:%.*]] = xor i32 [[I]], [[J]]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[IXJXK:%.*]] = xor i32 [[IXJ]], [[K]]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i32 [[IAJAK]], i32 [[IXJXK]]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[WP2:%.*]] = add i32 [[WP]], [[WP]]
-; CHECK-V8M-TWO-FOLD-6-NEXT:    [[W:%.*]] = select i1 [[A:%.*]], i32 2, i32 [[WP2]]
-; CHECK-V8M-TWO-FOLD-6-NEXT:    [[R:%.*]] = add i32 [[W]], 1
+; CHECK-V8M-TWO-FOLD-6-NEXT:    [[SPEC_SELECT:%.*]] = select i1 [[A:%.*]], i32 2, i32 [[WP2]]
+; CHECK-V8M-TWO-FOLD-6-NEXT:    [[R:%.*]] = add i32 [[SPEC_SELECT]], 1
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    ret i32 [[R]]
 ;
 ; CHECK-V8A-TWO-FOLD-6-LABEL: @test_i32_minsize(
 ; CHECK-V8A-TWO-FOLD-6-NEXT:  entry:
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[IAJ:%.*]] = add i32 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[IAJAK:%.*]] = add i32 [[IAJ]], [[K:%.*]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[IXJ:%.*]] = xor i32 [[I]], [[J]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[IXJXK:%.*]] = xor i32 [[IXJ]], [[K]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i32 [[IAJAK]], i32 [[IXJXK]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[WP2:%.*]] = add i32 [[WP]], [[WP]]
-; CHECK-V8A-TWO-FOLD-6-NEXT:    [[W:%.*]] = select i1 [[A:%.*]], i32 2, i32 [[WP2]]
-; CHECK-V8A-TWO-FOLD-6-NEXT:    [[R:%.*]] = add i32 [[W]], 1
+; CHECK-V8A-TWO-FOLD-6-NEXT:    [[SPEC_SELECT:%.*]] = select i1 [[A:%.*]], i32 2, i32 [[WP2]]
+; CHECK-V8A-TWO-FOLD-6-NEXT:    [[R:%.*]] = add i32 [[SPEC_SELECT]], 1
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    ret i32 [[R]]
 ;
 entry:
   br i1 %a, label %M, label %O
 O:
   br i1 %b, label %P, label %Q
 P:
   %iaj = add i32 %i, %j

 ; CHECK-V8M-TWO-FOLD-4-LABEL: @test_i64(
 ; CHECK-V8M-TWO-FOLD-4-NEXT:  entry:
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
 ; CHECK-V8M-TWO-FOLD-4:       O:
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[IAJ:%.*]] = add i64 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[IAJAK:%.*]] = add i64 [[IAJ]], [[K:%.*]]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[IXJ:%.*]] = xor i64 [[I]], [[J]]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[IXJXK:%.*]] = xor i64 [[IXJ]], [[K]]
-; CHECK-V8M-TWO-FOLD-4-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i64 [[IAJAK]], i64 [[IXJXK]]
+; CHECK-V8M-TWO-FOLD-4-NEXT:    br i1 [[B:%.*]], label [[P:%.*]], label [[Q:%.*]]
+; CHECK-V8M-TWO-FOLD-4:       P:
+; CHECK-V8M-TWO-FOLD-4-NEXT:    br label [[N:%.*]]
+; CHECK-V8M-TWO-FOLD-4:       Q:
+; CHECK-V8M-TWO-FOLD-4-NEXT:    br label [[N]]
+; CHECK-V8M-TWO-FOLD-4:       N:
+; CHECK-V8M-TWO-FOLD-4-NEXT:    [[WP:%.*]] = phi i64 [ [[IAJAK]], [[P]] ], [ [[IXJXK]], [[Q]] ]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[WP2:%.*]] = add i64 [[WP]], [[WP]]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    br label [[M]]
 ; CHECK-V8M-TWO-FOLD-4:       M:
-; CHECK-V8M-TWO-FOLD-4-NEXT:    [[W:%.*]] = phi i64 [ [[WP2]], [[O]] ], [ 2, [[ENTRY:%.*]] ]
+; CHECK-V8M-TWO-FOLD-4-NEXT:    [[W:%.*]] = phi i64 [ [[WP2]], [[N]] ], [ 2, [[ENTRY:%.*]] ]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[R:%.*]] = add i64 [[W]], 1
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    ret i64 [[R]]
 ;
 ; CHECK-V8A-TWO-FOLD-4-LABEL: @test_i64(
 ; CHECK-V8A-TWO-FOLD-4-NEXT:  entry:
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
 ; CHECK-V8A-TWO-FOLD-4:       O:
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[IAJ:%.*]] = add i64 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[IAJAK:%.*]] = add i64 [[IAJ]], [[K:%.*]]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[IXJ:%.*]] = xor i64 [[I]], [[J]]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[IXJXK:%.*]] = xor i64 [[IXJ]], [[K]]
-; CHECK-V8A-TWO-FOLD-4-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i64 [[IAJAK]], i64 [[IXJXK]]
+; CHECK-V8A-TWO-FOLD-4-NEXT:    br i1 [[B:%.*]], label [[P:%.*]], label [[Q:%.*]]
+; CHECK-V8A-TWO-FOLD-4:       P:
+; CHECK-V8A-TWO-FOLD-4-NEXT:    br label [[N:%.*]]
+; CHECK-V8A-TWO-FOLD-4:       Q:
+; CHECK-V8A-TWO-FOLD-4-NEXT:    br label [[N]]
+; CHECK-V8A-TWO-FOLD-4:       N:
+; CHECK-V8A-TWO-FOLD-4-NEXT:    [[WP:%.*]] = phi i64 [ [[IAJAK]], [[P]] ], [ [[IXJXK]], [[Q]] ]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[WP2:%.*]] = add i64 [[WP]], [[WP]]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    br label [[M]]
 ; CHECK-V8A-TWO-FOLD-4:       M:
-; CHECK-V8A-TWO-FOLD-4-NEXT:    [[W:%.*]] = phi i64 [ [[WP2]], [[O]] ], [ 2, [[ENTRY:%.*]] ]
+; CHECK-V8A-TWO-FOLD-4-NEXT:    [[W:%.*]] = phi i64 [ [[WP2]], [[N]] ], [ 2, [[ENTRY:%.*]] ]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[R:%.*]] = add i64 [[W]], 1
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    ret i64 [[R]]
 ;
 ; CHECK-V8M-TWO-FOLD-5-LABEL: @test_i64(
 ; CHECK-V8M-TWO-FOLD-5-NEXT:  entry:
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
 ; CHECK-V8M-TWO-FOLD-5:       O:
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    [[IAJ:%.*]] = add i64 [[I:%.*]], [[J:%.*]]

 ; CHECK-V8M-TWO-FOLD-6-LABEL: @test_i64(
 ; CHECK-V8M-TWO-FOLD-6-NEXT:  entry:
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[IAJ:%.*]] = add i64 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[IAJAK:%.*]] = add i64 [[IAJ]], [[K:%.*]]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[IXJ:%.*]] = xor i64 [[I]], [[J]]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[IXJXK:%.*]] = xor i64 [[IXJ]], [[K]]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i64 [[IAJAK]], i64 [[IXJXK]]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[WP2:%.*]] = add i64 [[WP]], [[WP]]
-; CHECK-V8M-TWO-FOLD-6-NEXT:    [[W:%.*]] = select i1 [[A:%.*]], i64 2, i64 [[WP2]]
-; CHECK-V8M-TWO-FOLD-6-NEXT:    [[R:%.*]] = add i64 [[W]], 1
+; CHECK-V8M-TWO-FOLD-6-NEXT:    [[SPEC_SELECT:%.*]] = select i1 [[A:%.*]], i64 2, i64 [[WP2]]
+; CHECK-V8M-TWO-FOLD-6-NEXT:    [[R:%.*]] = add i64 [[SPEC_SELECT]], 1
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    ret i64 [[R]]
 ;
 ; CHECK-V8A-TWO-FOLD-6-LABEL: @test_i64(
 ; CHECK-V8A-TWO-FOLD-6-NEXT:  entry:
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[IAJ:%.*]] = add i64 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[IAJAK:%.*]] = add i64 [[IAJ]], [[K:%.*]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[IXJ:%.*]] = xor i64 [[I]], [[J]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[IXJXK:%.*]] = xor i64 [[IXJ]], [[K]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i64 [[IAJAK]], i64 [[IXJXK]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[WP2:%.*]] = add i64 [[WP]], [[WP]]
-; CHECK-V8A-TWO-FOLD-6-NEXT:    [[W:%.*]] = select i1 [[A:%.*]], i64 2, i64 [[WP2]]
-; CHECK-V8A-TWO-FOLD-6-NEXT:    [[R:%.*]] = add i64 [[W]], 1
+; CHECK-V8A-TWO-FOLD-6-NEXT:    [[SPEC_SELECT:%.*]] = select i1 [[A:%.*]], i64 2, i64 [[WP2]]
+; CHECK-V8A-TWO-FOLD-6-NEXT:    [[R:%.*]] = add i64 [[SPEC_SELECT]], 1
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    ret i64 [[R]]
 ;
 entry:
   br i1 %a, label %M, label %O
 O:
   br i1 %b, label %P, label %Q
 P:
   %iaj = add i64 %i, %j

 ; CHECK-V8M-TWO-FOLD-4-LABEL: @test_i64_minsize(
 ; CHECK-V8M-TWO-FOLD-4-NEXT:  entry:
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
 ; CHECK-V8M-TWO-FOLD-4:       O:
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[IAJ:%.*]] = add i64 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[IAJAK:%.*]] = add i64 [[IAJ]], [[K:%.*]]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[IXJ:%.*]] = xor i64 [[I]], [[J]]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[IXJXK:%.*]] = xor i64 [[IXJ]], [[K]]
-; CHECK-V8M-TWO-FOLD-4-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i64 [[IAJAK]], i64 [[IXJXK]]
+; CHECK-V8M-TWO-FOLD-4-NEXT:    br i1 [[B:%.*]], label [[P:%.*]], label [[Q:%.*]]
+; CHECK-V8M-TWO-FOLD-4:       P:
+; CHECK-V8M-TWO-FOLD-4-NEXT:    br label [[N:%.*]]
+; CHECK-V8M-TWO-FOLD-4:       Q:
+; CHECK-V8M-TWO-FOLD-4-NEXT:    br label [[N]]
+; CHECK-V8M-TWO-FOLD-4:       N:
+; CHECK-V8M-TWO-FOLD-4-NEXT:    [[WP:%.*]] = phi i64 [ [[IAJAK]], [[P]] ], [ [[IXJXK]], [[Q]] ]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[WP2:%.*]] = add i64 [[WP]], [[WP]]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    br label [[M]]
 ; CHECK-V8M-TWO-FOLD-4:       M:
-; CHECK-V8M-TWO-FOLD-4-NEXT:    [[W:%.*]] = phi i64 [ [[WP2]], [[O]] ], [ 2, [[ENTRY:%.*]] ]
+; CHECK-V8M-TWO-FOLD-4-NEXT:    [[W:%.*]] = phi i64 [ [[WP2]], [[N]] ], [ 2, [[ENTRY:%.*]] ]
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    [[R:%.*]] = add i64 [[W]], 1
 ; CHECK-V8M-TWO-FOLD-4-NEXT:    ret i64 [[R]]
 ;
 ; CHECK-V8A-TWO-FOLD-4-LABEL: @test_i64_minsize(
 ; CHECK-V8A-TWO-FOLD-4-NEXT:  entry:
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
 ; CHECK-V8A-TWO-FOLD-4:       O:
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[IAJ:%.*]] = add i64 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[IAJAK:%.*]] = add i64 [[IAJ]], [[K:%.*]]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[IXJ:%.*]] = xor i64 [[I]], [[J]]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[IXJXK:%.*]] = xor i64 [[IXJ]], [[K]]
-; CHECK-V8A-TWO-FOLD-4-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i64 [[IAJAK]], i64 [[IXJXK]]
+; CHECK-V8A-TWO-FOLD-4-NEXT:    br i1 [[B:%.*]], label [[P:%.*]], label [[Q:%.*]]
+; CHECK-V8A-TWO-FOLD-4:       P:
+; CHECK-V8A-TWO-FOLD-4-NEXT:    br label [[N:%.*]]
+; CHECK-V8A-TWO-FOLD-4:       Q:
+; CHECK-V8A-TWO-FOLD-4-NEXT:    br label [[N]]
+; CHECK-V8A-TWO-FOLD-4:       N:
+; CHECK-V8A-TWO-FOLD-4-NEXT:    [[WP:%.*]] = phi i64 [ [[IAJAK]], [[P]] ], [ [[IXJXK]], [[Q]] ]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[WP2:%.*]] = add i64 [[WP]], [[WP]]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    br label [[M]]
 ; CHECK-V8A-TWO-FOLD-4:       M:
-; CHECK-V8A-TWO-FOLD-4-NEXT:    [[W:%.*]] = phi i64 [ [[WP2]], [[O]] ], [ 2, [[ENTRY:%.*]] ]
+; CHECK-V8A-TWO-FOLD-4-NEXT:    [[W:%.*]] = phi i64 [ [[WP2]], [[N]] ], [ 2, [[ENTRY:%.*]] ]
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    [[R:%.*]] = add i64 [[W]], 1
 ; CHECK-V8A-TWO-FOLD-4-NEXT:    ret i64 [[R]]
 ;
 ; CHECK-V8M-TWO-FOLD-5-LABEL: @test_i64_minsize(
 ; CHECK-V8M-TWO-FOLD-5-NEXT:  entry:
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
 ; CHECK-V8M-TWO-FOLD-5:       O:
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    [[IAJ:%.*]] = add i64 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    [[IAJAK:%.*]] = add i64 [[IAJ]], [[K:%.*]]
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    [[IXJ:%.*]] = xor i64 [[I]], [[J]]
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    [[IXJXK:%.*]] = xor i64 [[IXJ]], [[K]]
-; CHECK-V8M-TWO-FOLD-5-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i64 [[IAJAK]], i64 [[IXJXK]]
+; CHECK-V8M-TWO-FOLD-5-NEXT:    br i1 [[B:%.*]], label [[P:%.*]], label [[Q:%.*]]
+; CHECK-V8M-TWO-FOLD-5:       P:
+; CHECK-V8M-TWO-FOLD-5-NEXT:    br label [[N:%.*]]
+; CHECK-V8M-TWO-FOLD-5:       Q:
+; CHECK-V8M-TWO-FOLD-5-NEXT:    br label [[N]]
+; CHECK-V8M-TWO-FOLD-5:       N:
+; CHECK-V8M-TWO-FOLD-5-NEXT:    [[WP:%.*]] = phi i64 [ [[IAJAK]], [[P]] ], [ [[IXJXK]], [[Q]] ]
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    [[WP2:%.*]] = add i64 [[WP]], [[WP]]
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    br label [[M]]
 ; CHECK-V8M-TWO-FOLD-5:       M:
-; CHECK-V8M-TWO-FOLD-5-NEXT:    [[W:%.*]] = phi i64 [ [[WP2]], [[O]] ], [ 2, [[ENTRY:%.*]] ]
+; CHECK-V8M-TWO-FOLD-5-NEXT:    [[W:%.*]] = phi i64 [ [[WP2]], [[N]] ], [ 2, [[ENTRY:%.*]] ]
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    [[R:%.*]] = add i64 [[W]], 1
 ; CHECK-V8M-TWO-FOLD-5-NEXT:    ret i64 [[R]]
 ;
 ; CHECK-V8A-TWO-FOLD-5-LABEL: @test_i64_minsize(
 ; CHECK-V8A-TWO-FOLD-5-NEXT:  entry:
 ; CHECK-V8A-TWO-FOLD-5-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
 ; CHECK-V8A-TWO-FOLD-5:       O:
 ; CHECK-V8A-TWO-FOLD-5-NEXT:    [[IAJ:%.*]] = add i64 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8A-TWO-FOLD-5-NEXT:    [[IAJAK:%.*]] = add i64 [[IAJ]], [[K:%.*]]
 ; CHECK-V8A-TWO-FOLD-5-NEXT:    [[IXJ:%.*]] = xor i64 [[I]], [[J]]
 ; CHECK-V8A-TWO-FOLD-5-NEXT:    [[IXJXK:%.*]] = xor i64 [[IXJ]], [[K]]
 ; CHECK-V8A-TWO-FOLD-5-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i64 [[IAJAK]], i64 [[IXJXK]]
 ; CHECK-V8A-TWO-FOLD-5-NEXT:    [[WP2:%.*]] = add i64 [[WP]], [[WP]]
 ; CHECK-V8A-TWO-FOLD-5-NEXT:    br label [[M]]
 ; CHECK-V8A-TWO-FOLD-5:       M:
 ; CHECK-V8A-TWO-FOLD-5-NEXT:    [[W:%.*]] = phi i64 [ [[WP2]], [[O]] ], [ 2, [[ENTRY:%.*]] ]
 ; CHECK-V8A-TWO-FOLD-5-NEXT:    [[R:%.*]] = add i64 [[W]], 1
 ; CHECK-V8A-TWO-FOLD-5-NEXT:    ret i64 [[R]]
 ;
 ; CHECK-V8M-TWO-FOLD-6-LABEL: @test_i64_minsize(
 ; CHECK-V8M-TWO-FOLD-6-NEXT:  entry:
+; CHECK-V8M-TWO-FOLD-6-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
+; CHECK-V8M-TWO-FOLD-6:       O:
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[IAJ:%.*]] = add i64 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[IAJAK:%.*]] = add i64 [[IAJ]], [[K:%.*]]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[IXJ:%.*]] = xor i64 [[I]], [[J]]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[IXJXK:%.*]] = xor i64 [[IXJ]], [[K]]
-; CHECK-V8M-TWO-FOLD-6-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i64 [[IAJAK]], i64 [[IXJXK]]
+; CHECK-V8M-TWO-FOLD-6-NEXT:    br i1 [[B:%.*]], label [[P:%.*]], label [[Q:%.*]]
+; CHECK-V8M-TWO-FOLD-6:       P:
+; CHECK-V8M-TWO-FOLD-6-NEXT:    br label [[N:%.*]]
+; CHECK-V8M-TWO-FOLD-6:       Q:
+; CHECK-V8M-TWO-FOLD-6-NEXT:    br label [[N]]
+; CHECK-V8M-TWO-FOLD-6:       N:
+; CHECK-V8M-TWO-FOLD-6-NEXT:    [[WP:%.*]] = phi i64 [ [[IAJAK]], [[P]] ], [ [[IXJXK]], [[Q]] ]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[WP2:%.*]] = add i64 [[WP]], [[WP]]
-; CHECK-V8M-TWO-FOLD-6-NEXT:    [[W:%.*]] = select i1 [[A:%.*]], i64 2, i64 [[WP2]]
+; CHECK-V8M-TWO-FOLD-6-NEXT:    br label [[M]]
+; CHECK-V8M-TWO-FOLD-6:       M:
+; CHECK-V8M-TWO-FOLD-6-NEXT:    [[W:%.*]] = phi i64 [ [[WP2]], [[N]] ], [ 2, [[ENTRY:%.*]] ]
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    [[R:%.*]] = add i64 [[W]], 1
 ; CHECK-V8M-TWO-FOLD-6-NEXT:    ret i64 [[R]]
 ;
 ; CHECK-V8A-TWO-FOLD-6-LABEL: @test_i64_minsize(
 ; CHECK-V8A-TWO-FOLD-6-NEXT:  entry:
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[IAJ:%.*]] = add i64 [[I:%.*]], [[J:%.*]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[IAJAK:%.*]] = add i64 [[IAJ]], [[K:%.*]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[IXJ:%.*]] = xor i64 [[I]], [[J]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[IXJXK:%.*]] = xor i64 [[IXJ]], [[K]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i64 [[IAJAK]], i64 [[IXJXK]]
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    [[WP2:%.*]] = add i64 [[WP]], [[WP]]
-; CHECK-V8A-TWO-FOLD-6-NEXT:    [[W:%.*]] = select i1 [[A:%.*]], i64 2, i64 [[WP2]]
-; CHECK-V8A-TWO-FOLD-6-NEXT:    [[R:%.*]] = add i64 [[W]], 1
+; CHECK-V8A-TWO-FOLD-6-NEXT:    [[SPEC_SELECT:%.*]] = select i1 [[A:%.*]], i64 2, i64 [[WP2]]
+; CHECK-V8A-TWO-FOLD-6-NEXT:    [[R:%.*]] = add i64 [[SPEC_SELECT]], 1
 ; CHECK-V8A-TWO-FOLD-6-NEXT:    ret i64 [[R]]
 ;
 entry:
   br i1 %a, label %M, label %O
 O:
   br i1 %b, label %P, label %Q
 P:
   %iaj = add i64 %i, %j

llvm/test/Transforms/SimplifyCFG/ARM/speculate-math.ll

 declare float @llvm.minimum.f32(float, float) nounwind readonly
 declare float @llvm.maximum.f32(float, float) nounwind readonly
 
 define double @fdiv_test(double %a, double %b) {
 ; CHECK-MVE-LABEL: @fdiv_test(
 ; CHECK-MVE-NEXT:  entry:
 ; CHECK-MVE-NEXT:    [[CMP:%.*]] = fcmp ogt double [[A:%.*]], 0.000000e+00
 ; CHECK-MVE-NEXT:    [[DIV:%.*]] = fdiv double [[B:%.*]], [[A]]
-; CHECK-MVE-NEXT:    [[COND:%.*]] = select nsz i1 [[CMP]], double [[DIV]], double 0.000000e+00
+; CHECK-MVE-NEXT:    br i1 [[CMP]], label [[COND_TRUE:%.*]], label [[COND_END:%.*]]
+; CHECK-MVE:       cond.true:
+; CHECK-MVE-NEXT:    br label [[COND_END]]
+; CHECK-MVE:       cond.end:
+; CHECK-MVE-NEXT:    [[COND:%.*]] = phi nsz double [ [[DIV]], [[COND_TRUE]] ], [ 0.000000e+00, [[ENTRY:%.*]] ]
 ; CHECK-MVE-NEXT:    ret double [[COND]]
 ;
 ; CHECK-V8M-MAIN-LABEL: @fdiv_test(
 ; CHECK-V8M-MAIN-NEXT:  entry:
 ; CHECK-V8M-MAIN-NEXT:    [[CMP:%.*]] = fcmp ogt double [[A:%.*]], 0.000000e+00
 ; CHECK-V8M-MAIN-NEXT:    [[DIV:%.*]] = fdiv double [[B:%.*]], [[A]]
-; CHECK-V8M-MAIN-NEXT:    [[COND:%.*]] = select nsz i1 [[CMP]], double [[DIV]], double 0.000000e+00
+; CHECK-V8M-MAIN-NEXT:    br i1 [[CMP]], label [[COND_TRUE:%.*]], label [[COND_END:%.*]]
+; CHECK-V8M-MAIN:       cond.true:
+; CHECK-V8M-MAIN-NEXT:    br label [[COND_END]]
+; CHECK-V8M-MAIN:       cond.end:
+; CHECK-V8M-MAIN-NEXT:    [[COND:%.*]] = phi nsz double [ [[DIV]], [[COND_TRUE]] ], [ 0.000000e+00, [[ENTRY:%.*]] ]
 ; CHECK-V8M-MAIN-NEXT:    ret double [[COND]]
 ;
 ; CHECK-V8M-BASE-LABEL: @fdiv_test(
 ; CHECK-V8M-BASE-NEXT:  entry:
 ; CHECK-V8M-BASE-NEXT:    [[CMP:%.*]] = fcmp ogt double [[A:%.*]], 0.000000e+00
 ; CHECK-V8M-BASE-NEXT:    [[DIV:%.*]] = fdiv double [[B:%.*]], [[A]]
-; CHECK-V8M-BASE-NEXT:    [[COND:%.*]] = select nsz i1 [[CMP]], double [[DIV]], double 0.000000e+00
+; CHECK-V8M-BASE-NEXT:    br i1 [[CMP]], label [[COND_TRUE:%.*]], label [[COND_END:%.*]]
+; CHECK-V8M-BASE:       cond.true:
+; CHECK-V8M-BASE-NEXT:    br label [[COND_END]]
+; CHECK-V8M-BASE:       cond.end:
+; CHECK-V8M-BASE-NEXT:    [[COND:%.*]] = phi nsz double [ [[DIV]], [[COND_TRUE]] ], [ 0.000000e+00, [[ENTRY:%.*]] ]
 ; CHECK-V8M-BASE-NEXT:    ret double [[COND]]
 ;
 entry:
   %cmp = fcmp ogt double %a, 0.0
   br i1 %cmp, label %cond.true, label %cond.end
 
 cond.true:
   %div = fdiv double %b, %a
   br label %cond.end
 
 cond.end:
   %cond = phi nsz double [ %div, %cond.true ], [ 0.0, %entry ]
   ret double %cond
 }
 
 define void @sqrt_test(float addrspace(1)* noalias nocapture %out, float %a) nounwind {
 ; CHECK-MVE-LABEL: @sqrt_test(
 ; CHECK-MVE-NEXT:  entry:
 ; CHECK-MVE-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-MVE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.sqrt.f32(float [[A]]) #3
+; CHECK-MVE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.sqrt.f32(float [[A]]) [[ATTR3:#.*]]
 ; CHECK-MVE-NEXT:    [[COND_I:%.*]] = select afn i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-MVE-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-MVE-NEXT:    ret void
 ;
 ; CHECK-V8M-MAIN-LABEL: @sqrt_test(
 ; CHECK-V8M-MAIN-NEXT:  entry:
 ; CHECK-V8M-MAIN-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-V8M-MAIN-NEXT:    [[TMP0:%.*]] = tail call float @llvm.sqrt.f32(float [[A]]) #2
+; CHECK-V8M-MAIN-NEXT:    [[TMP0:%.*]] = tail call float @llvm.sqrt.f32(float [[A]]) [[ATTR2:#.*]]
 ; CHECK-V8M-MAIN-NEXT:    [[COND_I:%.*]] = select afn i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-V8M-MAIN-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-V8M-MAIN-NEXT:    ret void
 ;
 ; CHECK-V8M-BASE-LABEL: @sqrt_test(
 ; CHECK-V8M-BASE-NEXT:  entry:
 ; CHECK-V8M-BASE-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-V8M-BASE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.sqrt.f32(float [[A]]) #2
+; CHECK-V8M-BASE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.sqrt.f32(float [[A]]) [[ATTR2:#.*]]
 ; CHECK-V8M-BASE-NEXT:    [[COND_I:%.*]] = select afn i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-V8M-BASE-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-V8M-BASE-NEXT:    ret void
 ;
 entry:
   %cmp.i = fcmp olt float %a, 0.000000e+00
   br i1 %cmp.i, label %test_sqrt.exit, label %cond.else.i
 
 cond.else.i:                                      ; preds = %entry
   %0 = tail call float @llvm.sqrt.f32(float %a) nounwind readnone
   br label %test_sqrt.exit
 
 test_sqrt.exit:                                   ; preds = %cond.else.i, %entry
   %cond.i = phi afn float [ %0, %cond.else.i ], [ 0x7FF8000000000000, %entry ]
   store float %cond.i, float addrspace(1)* %out, align 4
   ret void
 }
 
 define void @fabs_test(float addrspace(1)* noalias nocapture %out, float %a) nounwind {
 ; CHECK-MVE-LABEL: @fabs_test(
 ; CHECK-MVE-NEXT:  entry:
 ; CHECK-MVE-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-MVE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fabs.f32(float [[A]]) #3
+; CHECK-MVE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fabs.f32(float [[A]]) [[ATTR3]]
 ; CHECK-MVE-NEXT:    [[COND_I:%.*]] = select reassoc i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-MVE-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-MVE-NEXT:    ret void
 ;
 ; CHECK-V8M-MAIN-LABEL: @fabs_test(
 ; CHECK-V8M-MAIN-NEXT:  entry:
 ; CHECK-V8M-MAIN-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-V8M-MAIN-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fabs.f32(float [[A]]) #2
+; CHECK-V8M-MAIN-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fabs.f32(float [[A]]) [[ATTR2]]
 ; CHECK-V8M-MAIN-NEXT:    [[COND_I:%.*]] = select reassoc i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-V8M-MAIN-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-V8M-MAIN-NEXT:    ret void
 ;
 ; CHECK-V8M-BASE-LABEL: @fabs_test(
 ; CHECK-V8M-BASE-NEXT:  entry:
 ; CHECK-V8M-BASE-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-V8M-BASE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fabs.f32(float [[A]]) #2
+; CHECK-V8M-BASE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fabs.f32(float [[A]]) [[ATTR2]]
 ; CHECK-V8M-BASE-NEXT:    [[COND_I:%.*]] = select reassoc i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-V8M-BASE-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-V8M-BASE-NEXT:    ret void
 ;
 entry:
   %cmp.i = fcmp olt float %a, 0.000000e+00
   br i1 %cmp.i, label %test_fabs.exit, label %cond.else.i
 
 cond.else.i:                                      ; preds = %entry
   %0 = tail call float @llvm.fabs.f32(float %a) nounwind readnone
   br label %test_fabs.exit
 
 test_fabs.exit:                                   ; preds = %cond.else.i, %entry
   %cond.i = phi reassoc float [ %0, %cond.else.i ], [ 0x7FF8000000000000, %entry ]
   store float %cond.i, float addrspace(1)* %out, align 4
   ret void
 }
 
 define void @fma_test(float addrspace(1)* noalias nocapture %out, float %a, float %b, float %c) nounwind {
 ; CHECK-MVE-LABEL: @fma_test(
 ; CHECK-MVE-NEXT:  entry:
 ; CHECK-MVE-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-MVE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fma.f32(float [[A]], float [[B:%.*]], float [[C:%.*]]) #3
+; CHECK-MVE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fma.f32(float [[A]], float [[B:%.*]], float [[C:%.*]]) [[ATTR3]]
 ; CHECK-MVE-NEXT:    [[COND_I:%.*]] = select reassoc nsz i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-MVE-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-MVE-NEXT:    ret void
 ;
 ; CHECK-V8M-MAIN-LABEL: @fma_test(
 ; CHECK-V8M-MAIN-NEXT:  entry:
 ; CHECK-V8M-MAIN-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-V8M-MAIN-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fma.f32(float [[A]], float [[B:%.*]], float [[C:%.*]]) #2
+; CHECK-V8M-MAIN-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fma.f32(float [[A]], float [[B:%.*]], float [[C:%.*]]) [[ATTR2]]
 ; CHECK-V8M-MAIN-NEXT:    [[COND_I:%.*]] = select reassoc nsz i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-V8M-MAIN-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-V8M-MAIN-NEXT:    ret void
 ;
 ; CHECK-V8M-BASE-LABEL: @fma_test(
 ; CHECK-V8M-BASE-NEXT:  entry:
 ; CHECK-V8M-BASE-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-V8M-BASE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fma.f32(float [[A]], float [[B:%.*]], float [[C:%.*]]) #2
+; CHECK-V8M-BASE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fma.f32(float [[A]], float [[B:%.*]], float [[C:%.*]]) [[ATTR2]]
 ; CHECK-V8M-BASE-NEXT:    [[COND_I:%.*]] = select reassoc nsz i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-V8M-BASE-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-V8M-BASE-NEXT:    ret void
 ;
 entry:
   %cmp.i = fcmp olt float %a, 0.000000e+00
   br i1 %cmp.i, label %test_fma.exit, label %cond.else.i
 
 cond.else.i:                                      ; preds = %entry
   %0 = tail call float @llvm.fma.f32(float %a, float %b, float %c) nounwind readnone
   br label %test_fma.exit
 
 test_fma.exit:                                   ; preds = %cond.else.i, %entry
   %cond.i = phi nsz reassoc float [ %0, %cond.else.i ], [ 0x7FF8000000000000, %entry ]
   store float %cond.i, float addrspace(1)* %out, align 4
   ret void
 }
 
 define void @fmuladd_test(float addrspace(1)* noalias nocapture %out, float %a, float %b, float %c) nounwind {
 ; CHECK-MVE-LABEL: @fmuladd_test(
 ; CHECK-MVE-NEXT:  entry:
 ; CHECK-MVE-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-MVE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fmuladd.f32(float [[A]], float [[B:%.*]], float [[C:%.*]]) #3
+; CHECK-MVE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fmuladd.f32(float [[A]], float [[B:%.*]], float [[C:%.*]]) [[ATTR3]]
 ; CHECK-MVE-NEXT:    [[COND_I:%.*]] = select ninf i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-MVE-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-MVE-NEXT:    ret void
 ;
 ; CHECK-V8M-MAIN-LABEL: @fmuladd_test(
 ; CHECK-V8M-MAIN-NEXT:  entry:
 ; CHECK-V8M-MAIN-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-V8M-MAIN-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fmuladd.f32(float [[A]], float [[B:%.*]], float [[C:%.*]]) #2
+; CHECK-V8M-MAIN-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fmuladd.f32(float [[A]], float [[B:%.*]], float [[C:%.*]]) [[ATTR2]]
 ; CHECK-V8M-MAIN-NEXT:    [[COND_I:%.*]] = select ninf i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-V8M-MAIN-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-V8M-MAIN-NEXT:    ret void
 ;
 ; CHECK-V8M-BASE-LABEL: @fmuladd_test(
 ; CHECK-V8M-BASE-NEXT:  entry:
 ; CHECK-V8M-BASE-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-V8M-BASE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fmuladd.f32(float [[A]], float [[B:%.*]], float [[C:%.*]]) #2
+; CHECK-V8M-BASE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fmuladd.f32(float [[A]], float [[B:%.*]], float [[C:%.*]]) [[ATTR2]]
 ; CHECK-V8M-BASE-NEXT:    [[COND_I:%.*]] = select ninf i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-V8M-BASE-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-V8M-BASE-NEXT:    ret void
 ;
 entry:
   %cmp.i = fcmp olt float %a, 0.000000e+00
   br i1 %cmp.i, label %test_fmuladd.exit, label %cond.else.i
 
 cond.else.i:                                      ; preds = %entry
   %0 = tail call float @llvm.fmuladd.f32(float %a, float %b, float %c) nounwind readnone
   br label %test_fmuladd.exit
 
 test_fmuladd.exit:                                   ; preds = %cond.else.i, %entry
   %cond.i = phi ninf float [ %0, %cond.else.i ], [ 0x7FF8000000000000, %entry ]
   store float %cond.i, float addrspace(1)* %out, align 4
   ret void
 }
 
 define void @minnum_test(float addrspace(1)* noalias nocapture %out, float %a, float %b) nounwind {
 ; CHECK-MVE-LABEL: @minnum_test(
 ; CHECK-MVE-NEXT:  entry:
 ; CHECK-MVE-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-MVE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.minnum.f32(float [[A]], float [[B:%.*]]) #3
+; CHECK-MVE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.minnum.f32(float [[A]], float [[B:%.*]]) [[ATTR3]]
 ; CHECK-MVE-NEXT:    [[COND_I:%.*]] = select i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-MVE-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-MVE-NEXT:    ret void
 ;
 ; CHECK-V8M-MAIN-LABEL: @minnum_test(
 ; CHECK-V8M-MAIN-NEXT:  entry:
 ; CHECK-V8M-MAIN-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-V8M-MAIN-NEXT:    [[TMP0:%.*]] = tail call float @llvm.minnum.f32(float [[A]], float [[B:%.*]]) #2
+; CHECK-V8M-MAIN-NEXT:    [[TMP0:%.*]] = tail call float @llvm.minnum.f32(float [[A]], float [[B:%.*]]) [[ATTR2]]
 ; CHECK-V8M-MAIN-NEXT:    [[COND_I:%.*]] = select i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-V8M-MAIN-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-V8M-MAIN-NEXT:    ret void
 ;
 ; CHECK-V8M-BASE-LABEL: @minnum_test(
 ; CHECK-V8M-BASE-NEXT:  entry:
 ; CHECK-V8M-BASE-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-V8M-BASE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.minnum.f32(float [[A]], float [[B:%.*]]) #2
+; CHECK-V8M-BASE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.minnum.f32(float [[A]], float [[B:%.*]]) [[ATTR2]]
 ; CHECK-V8M-BASE-NEXT:    [[COND_I:%.*]] = select i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-V8M-BASE-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-V8M-BASE-NEXT:    ret void
 ;
 entry:
   %cmp.i = fcmp olt float %a, 0.000000e+00
   br i1 %cmp.i, label %test_minnum.exit, label %cond.else.i
 
 cond.else.i:                                      ; preds = %entry
   %0 = tail call float @llvm.minnum.f32(float %a, float %b) nounwind readnone
   br label %test_minnum.exit
 
 test_minnum.exit:                                   ; preds = %cond.else.i, %entry
   %cond.i = phi float [ %0, %cond.else.i ], [ 0x7FF8000000000000, %entry ]
   store float %cond.i, float addrspace(1)* %out, align 4
   ret void
 }
 
 define void @maxnum_test(float addrspace(1)* noalias nocapture %out, float %a, float %b) nounwind {
 ; CHECK-MVE-LABEL: @maxnum_test(
 ; CHECK-MVE-NEXT:  entry:
 ; CHECK-MVE-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-MVE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.maxnum.f32(float [[A]], float [[B:%.*]]) #3
+; CHECK-MVE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.maxnum.f32(float [[A]], float [[B:%.*]]) [[ATTR3]]
 ; CHECK-MVE-NEXT:    [[COND_I:%.*]] = select ninf nsz i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-MVE-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-MVE-NEXT:    ret void
 ;
 ; CHECK-V8M-MAIN-LABEL: @maxnum_test(
 ; CHECK-V8M-MAIN-NEXT:  entry:
 ; CHECK-V8M-MAIN-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-V8M-MAIN-NEXT:    [[TMP0:%.*]] = tail call float @llvm.maxnum.f32(float [[A]], float [[B:%.*]]) #2
+; CHECK-V8M-MAIN-NEXT:    [[TMP0:%.*]] = tail call float @llvm.maxnum.f32(float [[A]], float [[B:%.*]]) [[ATTR2]]
 ; CHECK-V8M-MAIN-NEXT:    [[COND_I:%.*]] = select ninf nsz i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-V8M-MAIN-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-V8M-MAIN-NEXT:    ret void
 ;
 ; CHECK-V8M-BASE-LABEL: @maxnum_test(
 ; CHECK-V8M-BASE-NEXT:  entry:
 ; CHECK-V8M-BASE-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-V8M-BASE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.maxnum.f32(float [[A]], float [[B:%.*]]) #2
+; CHECK-V8M-BASE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.maxnum.f32(float [[A]], float [[B:%.*]]) [[ATTR2]]
 ; CHECK-V8M-BASE-NEXT:    [[COND_I:%.*]] = select ninf nsz i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-V8M-BASE-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-V8M-BASE-NEXT:    ret void
 ;
 entry:
   %cmp.i = fcmp olt float %a, 0.000000e+00
   br i1 %cmp.i, label %test_maxnum.exit, label %cond.else.i
 
 cond.else.i:                                      ; preds = %entry
   %0 = tail call float @llvm.maxnum.f32(float %a, float %b) nounwind readnone
   br label %test_maxnum.exit
 
 test_maxnum.exit:                                   ; preds = %cond.else.i, %entry
   %cond.i = phi ninf nsz float [ %0, %cond.else.i ], [ 0x7FF8000000000000, %entry ]
   store float %cond.i, float addrspace(1)* %out, align 4
   ret void
 }
 
 define void @minimum_test(float addrspace(1)* noalias nocapture %out, float %a, float %b) nounwind {
 ; CHECK-MVE-LABEL: @minimum_test(
 ; CHECK-MVE-NEXT:  entry:
 ; CHECK-MVE-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-MVE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.minimum.f32(float [[A]], float [[B:%.*]]) #3
+; CHECK-MVE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.minimum.f32(float [[A]], float [[B:%.*]]) [[ATTR3]]
 ; CHECK-MVE-NEXT:    [[COND_I:%.*]] = select reassoc i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-MVE-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-MVE-NEXT:    ret void
 ;
 ; CHECK-V8M-MAIN-LABEL: @minimum_test(
 ; CHECK-V8M-MAIN-NEXT:  entry:
 ; CHECK-V8M-MAIN-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-V8M-MAIN-NEXT:    [[TMP0:%.*]] = tail call float @llvm.minimum.f32(float [[A]], float [[B:%.*]]) #2
+; CHECK-V8M-MAIN-NEXT:    [[TMP0:%.*]] = tail call float @llvm.minimum.f32(float [[A]], float [[B:%.*]]) [[ATTR2]]
 ; CHECK-V8M-MAIN-NEXT:    [[COND_I:%.*]] = select reassoc i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-V8M-MAIN-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-V8M-MAIN-NEXT:    ret void
 ;
 ; CHECK-V8M-BASE-LABEL: @minimum_test(
 ; CHECK-V8M-BASE-NEXT:  entry:
 ; CHECK-V8M-BASE-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-V8M-BASE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.minimum.f32(float [[A]], float [[B:%.*]]) #2
+; CHECK-V8M-BASE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.minimum.f32(float [[A]], float [[B:%.*]]) [[ATTR2]]
 ; CHECK-V8M-BASE-NEXT:    [[COND_I:%.*]] = select reassoc i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-V8M-BASE-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-V8M-BASE-NEXT:    ret void
 ;
 entry:
   %cmp.i = fcmp olt float %a, 0.000000e+00
   br i1 %cmp.i, label %test_minimum.exit, label %cond.else.i
 
 cond.else.i:                                      ; preds = %entry
   %0 = tail call float @llvm.minimum.f32(float %a, float %b) nounwind readnone
   br label %test_minimum.exit
 
 test_minimum.exit:                                   ; preds = %cond.else.i, %entry
   %cond.i = phi reassoc float [ %0, %cond.else.i ], [ 0x7FF8000000000000, %entry ]
   store float %cond.i, float addrspace(1)* %out, align 4
   ret void
 }
 
 define void @maximum_test(float addrspace(1)* noalias nocapture %out, float %a, float %b) nounwind {
 ; CHECK-MVE-LABEL: @maximum_test(
 ; CHECK-MVE-NEXT:  entry:
 ; CHECK-MVE-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-MVE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.maximum.f32(float [[A]], float [[B:%.*]]) #3
+; CHECK-MVE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.maximum.f32(float [[A]], float [[B:%.*]]) [[ATTR3]]
 ; CHECK-MVE-NEXT:    [[COND_I:%.*]] = select nsz i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-MVE-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-MVE-NEXT:    ret void
 ;
 ; CHECK-V8M-MAIN-LABEL: @maximum_test(
 ; CHECK-V8M-MAIN-NEXT:  entry:
 ; CHECK-V8M-MAIN-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-V8M-MAIN-NEXT:    [[TMP0:%.*]] = tail call float @llvm.maximum.f32(float [[A]], float [[B:%.*]]) #2
+; CHECK-V8M-MAIN-NEXT:    [[TMP0:%.*]] = tail call float @llvm.maximum.f32(float [[A]], float [[B:%.*]]) [[ATTR2]]
 ; CHECK-V8M-MAIN-NEXT:    [[COND_I:%.*]] = select nsz i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-V8M-MAIN-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-V8M-MAIN-NEXT:    ret void
 ;
 ; CHECK-V8M-BASE-LABEL: @maximum_test(
 ; CHECK-V8M-BASE-NEXT:  entry:
 ; CHECK-V8M-BASE-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; CHECK-V8M-BASE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.maximum.f32(float [[A]], float [[B:%.*]]) #2
+; CHECK-V8M-BASE-NEXT:    [[TMP0:%.*]] = tail call float @llvm.maximum.f32(float [[A]], float [[B:%.*]]) [[ATTR2]]
 ; CHECK-V8M-BASE-NEXT:    [[COND_I:%.*]] = select nsz i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; CHECK-V8M-BASE-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; CHECK-V8M-BASE-NEXT:    ret void
 ;
 entry:
   %cmp.i = fcmp olt float %a, 0.000000e+00
   br i1 %cmp.i, label %test_maximum.exit, label %cond.else.i
 

llvm/test/Transforms/SimplifyCFG/PhiEliminate3.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt < %s -simplifycfg -S -phi-node-folding-threshold=1 | FileCheck %s --check-prefix=ALL --check-prefix=CHECK-ONE
 ; RUN: opt < %s -simplifycfg -S -phi-node-folding-threshold=2 | FileCheck %s --check-prefix=ALL --check-prefix=CHECK-TWO
 ; RUN: opt < %s -simplifycfg -S -phi-node-folding-threshold=7 | FileCheck %s --check-prefix=ALL --check-prefix=CHECK-SEVEN
+; RUN: opt < %s -simplifycfg -S -two-entry-phi-node-folding-threshold=5 | FileCheck %s --check-prefix=ALL --check-prefix=CHECK-TWO-FOLD-5
+; RUN: opt < %s -simplifycfg -S -two-entry-phi-node-folding-threshold=6 | FileCheck %s --check-prefix=ALL --check-prefix=CHECK-TWO-FOLD-6
 
 ; Test merging of blocks containing complex expressions,
 ; with various folding thresholds
 
 define i32 @test(i1 %a, i1 %b, i32 %i, i32 %j, i32 %k) {
-; ALL-LABEL: @test(
-; ALL-NEXT:  entry:
-; ALL-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
-; ALL:       O:
-; ALL-NEXT:    [[IAJ:%.*]] = add i32 [[I:%.*]], [[J:%.*]]
-; ALL-NEXT:    [[IAJAK:%.*]] = add i32 [[IAJ]], [[K:%.*]]
-; ALL-NEXT:    [[IXJ:%.*]] = xor i32 [[I]], [[J]]
-; ALL-NEXT:    [[IXJXK:%.*]] = xor i32 [[IXJ]], [[K]]
-; ALL-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i32 [[IAJAK]], i32 [[IXJXK]]
-; ALL-NEXT:    [[WP2:%.*]] = add i32 [[WP]], [[WP]]
-; ALL-NEXT:    br label [[M]]
-; ALL:       M:
-; ALL-NEXT:    [[W:%.*]] = phi i32 [ [[WP2]], [[O]] ], [ 2, [[ENTRY:%.*]] ]
-; ALL-NEXT:    [[R:%.*]] = add i32 [[W]], 1
-; ALL-NEXT:    ret i32 [[R]]
+; CHECK-ONE-LABEL: @test(
+; CHECK-ONE-NEXT:  entry:
+; CHECK-ONE-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
+; CHECK-ONE:       O:
+; CHECK-ONE-NEXT:    [[IAJ:%.*]] = add i32 [[I:%.*]], [[J:%.*]]
+; CHECK-ONE-NEXT:    [[IAJAK:%.*]] = add i32 [[IAJ]], [[K:%.*]]
+; CHECK-ONE-NEXT:    [[IXJ:%.*]] = xor i32 [[I]], [[J]]
+; CHECK-ONE-NEXT:    [[IXJXK:%.*]] = xor i32 [[IXJ]], [[K]]
+; CHECK-ONE-NEXT:    br i1 [[B:%.*]], label [[P:%.*]], label [[Q:%.*]]
+; CHECK-ONE:       P:
+; CHECK-ONE-NEXT:    br label [[N:%.*]]
+; CHECK-ONE:       Q:
+; CHECK-ONE-NEXT:    br label [[N]]
+; CHECK-ONE:       N:
+; CHECK-ONE-NEXT:    [[WP:%.*]] = phi i32 [ [[IAJAK]], [[P]] ], [ [[IXJXK]], [[Q]] ]
+; CHECK-ONE-NEXT:    [[WP2:%.*]] = add i32 [[WP]], [[WP]]
+; CHECK-ONE-NEXT:    br label [[M]]
+; CHECK-ONE:       M:
+; CHECK-ONE-NEXT:    [[W:%.*]] = phi i32 [ [[WP2]], [[N]] ], [ 2, [[ENTRY:%.*]] ]
+; CHECK-ONE-NEXT:    [[R:%.*]] = add i32 [[W]], 1
+; CHECK-ONE-NEXT:    ret i32 [[R]]
+;
+; CHECK-TWO-LABEL: @test(
+; CHECK-TWO-NEXT:  entry:
+; CHECK-TWO-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
+; CHECK-TWO:       O:
+; CHECK-TWO-NEXT:    [[IAJ:%.*]] = add i32 [[I:%.*]], [[J:%.*]]
+; CHECK-TWO-NEXT:    [[IAJAK:%.*]] = add i32 [[IAJ]], [[K:%.*]]
+; CHECK-TWO-NEXT:    [[IXJ:%.*]] = xor i32 [[I]], [[J]]
+; CHECK-TWO-NEXT:    [[IXJXK:%.*]] = xor i32 [[IXJ]], [[K]]
+; CHECK-TWO-NEXT:    br i1 [[B:%.*]], label [[P:%.*]], label [[Q:%.*]]
+; CHECK-TWO:       P:
+; CHECK-TWO-NEXT:    br label [[N:%.*]]
+; CHECK-TWO:       Q:
+; CHECK-TWO-NEXT:    br label [[N]]
+; CHECK-TWO:       N:
+; CHECK-TWO-NEXT:    [[WP:%.*]] = phi i32 [ [[IAJAK]], [[P]] ], [ [[IXJXK]], [[Q]] ]
+; CHECK-TWO-NEXT:    [[WP2:%.*]] = add i32 [[WP]], [[WP]]
+; CHECK-TWO-NEXT:    br label [[M]]
+; CHECK-TWO:       M:
+; CHECK-TWO-NEXT:    [[W:%.*]] = phi i32 [ [[WP2]], [[N]] ], [ 2, [[ENTRY:%.*]] ]
+; CHECK-TWO-NEXT:    [[R:%.*]] = add i32 [[W]], 1
+; CHECK-TWO-NEXT:    ret i32 [[R]]
+;
+; CHECK-SEVEN-LABEL: @test(
+; CHECK-SEVEN-NEXT:  entry:
+; CHECK-SEVEN-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
+; CHECK-SEVEN:       O:
+; CHECK-SEVEN-NEXT:    [[IAJ:%.*]] = add i32 [[I:%.*]], [[J:%.*]]
+; CHECK-SEVEN-NEXT:    [[IAJAK:%.*]] = add i32 [[IAJ]], [[K:%.*]]
+; CHECK-SEVEN-NEXT:    [[IXJ:%.*]] = xor i32 [[I]], [[J]]
+; CHECK-SEVEN-NEXT:    [[IXJXK:%.*]] = xor i32 [[IXJ]], [[K]]
+; CHECK-SEVEN-NEXT:    br i1 [[B:%.*]], label [[P:%.*]], label [[Q:%.*]]
+; CHECK-SEVEN:       P:
+; CHECK-SEVEN-NEXT:    br label [[N:%.*]]
+; CHECK-SEVEN:       Q:
+; CHECK-SEVEN-NEXT:    br label [[N]]
+; CHECK-SEVEN:       N:
+; CHECK-SEVEN-NEXT:    [[WP:%.*]] = phi i32 [ [[IAJAK]], [[P]] ], [ [[IXJXK]], [[Q]] ]
+; CHECK-SEVEN-NEXT:    [[WP2:%.*]] = add i32 [[WP]], [[WP]]
+; CHECK-SEVEN-NEXT:    br label [[M]]
+; CHECK-SEVEN:       M:
+; CHECK-SEVEN-NEXT:    [[W:%.*]] = phi i32 [ [[WP2]], [[N]] ], [ 2, [[ENTRY:%.*]] ]
+; CHECK-SEVEN-NEXT:    [[R:%.*]] = add i32 [[W]], 1
+; CHECK-SEVEN-NEXT:    ret i32 [[R]]
+;
+; CHECK-TWO-FOLD-5-LABEL: @test(
+; CHECK-TWO-FOLD-5-NEXT:  entry:
+; CHECK-TWO-FOLD-5-NEXT:    br i1 [[A:%.*]], label [[M:%.*]], label [[O:%.*]]
+; CHECK-TWO-FOLD-5:       O:
+; CHECK-TWO-FOLD-5-NEXT:    [[IAJ:%.*]] = add i32 [[I:%.*]], [[J:%.*]]
+; CHECK-TWO-FOLD-5-NEXT:    [[IAJAK:%.*]] = add i32 [[IAJ]], [[K:%.*]]
+; CHECK-TWO-FOLD-5-NEXT:    [[IXJ:%.*]] = xor i32 [[I]], [[J]]
+; CHECK-TWO-FOLD-5-NEXT:    [[IXJXK:%.*]] = xor i32 [[IXJ]], [[K]]
+; CHECK-TWO-FOLD-5-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i32 [[IAJAK]], i32 [[IXJXK]]
+; CHECK-TWO-FOLD-5-NEXT:    [[WP2:%.*]] = add i32 [[WP]], [[WP]]
+; CHECK-TWO-FOLD-5-NEXT:    br label [[M]]
+; CHECK-TWO-FOLD-5:       M:
+; CHECK-TWO-FOLD-5-NEXT:    [[W:%.*]] = phi i32 [ [[WP2]], [[O]] ], [ 2, [[ENTRY:%.*]] ]
+; CHECK-TWO-FOLD-5-NEXT:    [[R:%.*]] = add i32 [[W]], 1
+; CHECK-TWO-FOLD-5-NEXT:    ret i32 [[R]]
+;
+; CHECK-TWO-FOLD-6-LABEL: @test(
+; CHECK-TWO-FOLD-6-NEXT:  entry:
+; CHECK-TWO-FOLD-6-NEXT:    [[IAJ:%.*]] = add i32 [[I:%.*]], [[J:%.*]]
+; CHECK-TWO-FOLD-6-NEXT:    [[IAJAK:%.*]] = add i32 [[IAJ]], [[K:%.*]]
+; CHECK-TWO-FOLD-6-NEXT:    [[IXJ:%.*]] = xor i32 [[I]], [[J]]
+; CHECK-TWO-FOLD-6-NEXT:    [[IXJXK:%.*]] = xor i32 [[IXJ]], [[K]]
+; CHECK-TWO-FOLD-6-NEXT:    [[WP:%.*]] = select i1 [[B:%.*]], i32 [[IAJAK]], i32 [[IXJXK]]
+; CHECK-TWO-FOLD-6-NEXT:    [[WP2:%.*]] = add i32 [[WP]], [[WP]]
+; CHECK-TWO-FOLD-6-NEXT:    [[SPEC_SELECT:%.*]] = select i1 [[A:%.*]], i32 2, i32 [[WP2]]
+; CHECK-TWO-FOLD-6-NEXT:    [[R:%.*]] = add i32 [[SPEC_SELECT]], 1
+; CHECK-TWO-FOLD-6-NEXT:    ret i32 [[R]]
 ;
 entry:
   br i1 %a, label %M, label %O
 O:
   br i1 %b, label %P, label %Q
 P:
   %iaj = add i32 %i, %j
   %iajak = add i32 %iaj, %k

llvm/test/Transforms/SimplifyCFG/speculate-math.ll

 declare float @llvm.minimum.f32(float, float) nounwind readonly
 declare float @llvm.maximum.f32(float, float) nounwind readonly
 
 define double @fdiv_test(double %a, double %b) {
 ; ALL-LABEL: @fdiv_test(
 ; ALL-NEXT:  entry:
 ; ALL-NEXT:    [[CMP:%.*]] = fcmp ogt double [[A:%.*]], 0.000000e+00
 ; ALL-NEXT:    [[DIV:%.*]] = fdiv double [[B:%.*]], [[A]]
-; ALL-NEXT:    [[COND:%.*]] = select nsz i1 [[CMP]], double [[DIV]], double 0.000000e+00
+; ALL-NEXT:    br i1 [[CMP]], label [[COND_TRUE:%.*]], label [[COND_END:%.*]]
+; ALL:       cond.true:
+; ALL-NEXT:    br label [[COND_END]]
+; ALL:       cond.end:
+; ALL-NEXT:    [[COND:%.*]] = phi nsz double [ [[DIV]], [[COND_TRUE]] ], [ 0.000000e+00, [[ENTRY:%.*]] ]
 ; ALL-NEXT:    ret double [[COND]]
 ;
 entry:
   %cmp = fcmp ogt double %a, 0.0
   br i1 %cmp, label %cond.true, label %cond.end
 
 cond.true:
   %div = fdiv double %b, %a
   br label %cond.end
 
 cond.end:
   %cond = phi nsz double [ %div, %cond.true ], [ 0.0, %entry ]
   ret double %cond
 }
 
 define void @sqrt_test(float addrspace(1)* noalias nocapture %out, float %a) nounwind {
 ; ALL-LABEL: @sqrt_test(
 ; ALL-NEXT:  entry:
 ; ALL-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; ALL-NEXT:    [[TMP0:%.*]] = tail call float @llvm.sqrt.f32(float [[A]]) #2
+; ALL-NEXT:    [[TMP0:%.*]] = tail call float @llvm.sqrt.f32(float [[A]]) [[ATTR2:#.*]]
 ; ALL-NEXT:    [[COND_I:%.*]] = select afn i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; ALL-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; ALL-NEXT:    ret void
 ;
 entry:
   %cmp.i = fcmp olt float %a, 0.000000e+00
   br i1 %cmp.i, label %test_sqrt.exit, label %cond.else.i
 
 cond.else.i:                                      ; preds = %entry
   %0 = tail call float @llvm.sqrt.f32(float %a) nounwind readnone
   br label %test_sqrt.exit
 
 test_sqrt.exit:                                   ; preds = %cond.else.i, %entry
   %cond.i = phi afn float [ %0, %cond.else.i ], [ 0x7FF8000000000000, %entry ]
   store float %cond.i, float addrspace(1)* %out, align 4
   ret void
 }
 
 define void @fabs_test(float addrspace(1)* noalias nocapture %out, float %a) nounwind {
 ; ALL-LABEL: @fabs_test(
 ; ALL-NEXT:  entry:
 ; ALL-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; ALL-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fabs.f32(float [[A]]) #2
+; ALL-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fabs.f32(float [[A]]) [[ATTR2]]
 ; ALL-NEXT:    [[COND_I:%.*]] = select reassoc i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; ALL-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; ALL-NEXT:    ret void
 ;
 entry:
   %cmp.i = fcmp olt float %a, 0.000000e+00
   br i1 %cmp.i, label %test_fabs.exit, label %cond.else.i
 
 cond.else.i:                                      ; preds = %entry
   %0 = tail call float @llvm.fabs.f32(float %a) nounwind readnone
   br label %test_fabs.exit
 
 test_fabs.exit:                                   ; preds = %cond.else.i, %entry
   %cond.i = phi reassoc float [ %0, %cond.else.i ], [ 0x7FF8000000000000, %entry ]
   store float %cond.i, float addrspace(1)* %out, align 4
   ret void
 }
 
 define void @fma_test(float addrspace(1)* noalias nocapture %out, float %a, float %b, float %c) nounwind {
 ; ALL-LABEL: @fma_test(
 ; ALL-NEXT:  entry:
 ; ALL-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; ALL-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fma.f32(float [[A]], float [[B:%.*]], float [[C:%.*]]) #2
+; ALL-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fma.f32(float [[A]], float [[B:%.*]], float [[C:%.*]]) [[ATTR2]]
 ; ALL-NEXT:    [[COND_I:%.*]] = select reassoc nsz i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; ALL-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; ALL-NEXT:    ret void
 ;
 entry:
   %cmp.i = fcmp olt float %a, 0.000000e+00
   br i1 %cmp.i, label %test_fma.exit, label %cond.else.i
 
 cond.else.i:                                      ; preds = %entry
   %0 = tail call float @llvm.fma.f32(float %a, float %b, float %c) nounwind readnone
   br label %test_fma.exit
 
 test_fma.exit:                                   ; preds = %cond.else.i, %entry
   %cond.i = phi nsz reassoc float [ %0, %cond.else.i ], [ 0x7FF8000000000000, %entry ]
   store float %cond.i, float addrspace(1)* %out, align 4
   ret void
 }
 
 define void @fmuladd_test(float addrspace(1)* noalias nocapture %out, float %a, float %b, float %c) nounwind {
 ; ALL-LABEL: @fmuladd_test(
 ; ALL-NEXT:  entry:
 ; ALL-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; ALL-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fmuladd.f32(float [[A]], float [[B:%.*]], float [[C:%.*]]) #2
+; ALL-NEXT:    [[TMP0:%.*]] = tail call float @llvm.fmuladd.f32(float [[A]], float [[B:%.*]], float [[C:%.*]]) [[ATTR2]]
 ; ALL-NEXT:    [[COND_I:%.*]] = select ninf i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; ALL-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; ALL-NEXT:    ret void
 ;
 entry:
   %cmp.i = fcmp olt float %a, 0.000000e+00
   br i1 %cmp.i, label %test_fmuladd.exit, label %cond.else.i
 
 cond.else.i:                                      ; preds = %entry
   %0 = tail call float @llvm.fmuladd.f32(float %a, float %b, float %c) nounwind readnone
   br label %test_fmuladd.exit
 
 test_fmuladd.exit:                                   ; preds = %cond.else.i, %entry
   %cond.i = phi ninf float [ %0, %cond.else.i ], [ 0x7FF8000000000000, %entry ]
   store float %cond.i, float addrspace(1)* %out, align 4
   ret void
 }
 
 define void @minnum_test(float addrspace(1)* noalias nocapture %out, float %a, float %b) nounwind {
 ; ALL-LABEL: @minnum_test(
 ; ALL-NEXT:  entry:
 ; ALL-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; ALL-NEXT:    [[TMP0:%.*]] = tail call float @llvm.minnum.f32(float [[A]], float [[B:%.*]]) #2
+; ALL-NEXT:    [[TMP0:%.*]] = tail call float @llvm.minnum.f32(float [[A]], float [[B:%.*]]) [[ATTR2]]
 ; ALL-NEXT:    [[COND_I:%.*]] = select i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; ALL-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; ALL-NEXT:    ret void
 ;
 entry:
   %cmp.i = fcmp olt float %a, 0.000000e+00
   br i1 %cmp.i, label %test_minnum.exit, label %cond.else.i
 
 cond.else.i:                                      ; preds = %entry
   %0 = tail call float @llvm.minnum.f32(float %a, float %b) nounwind readnone
   br label %test_minnum.exit
 
 test_minnum.exit:                                   ; preds = %cond.else.i, %entry
   %cond.i = phi float [ %0, %cond.else.i ], [ 0x7FF8000000000000, %entry ]
   store float %cond.i, float addrspace(1)* %out, align 4
   ret void
 }
 
 define void @maxnum_test(float addrspace(1)* noalias nocapture %out, float %a, float %b) nounwind {
 ; ALL-LABEL: @maxnum_test(
 ; ALL-NEXT:  entry:
 ; ALL-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; ALL-NEXT:    [[TMP0:%.*]] = tail call float @llvm.maxnum.f32(float [[A]], float [[B:%.*]]) #2
+; ALL-NEXT:    [[TMP0:%.*]] = tail call float @llvm.maxnum.f32(float [[A]], float [[B:%.*]]) [[ATTR2]]
 ; ALL-NEXT:    [[COND_I:%.*]] = select ninf nsz i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; ALL-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; ALL-NEXT:    ret void
 ;
 entry:
   %cmp.i = fcmp olt float %a, 0.000000e+00
   br i1 %cmp.i, label %test_maxnum.exit, label %cond.else.i
 
 cond.else.i:                                      ; preds = %entry
   %0 = tail call float @llvm.maxnum.f32(float %a, float %b) nounwind readnone
   br label %test_maxnum.exit
 
 test_maxnum.exit:                                   ; preds = %cond.else.i, %entry
   %cond.i = phi ninf nsz float [ %0, %cond.else.i ], [ 0x7FF8000000000000, %entry ]
   store float %cond.i, float addrspace(1)* %out, align 4
   ret void
 }
 
 define void @minimum_test(float addrspace(1)* noalias nocapture %out, float %a, float %b) nounwind {
 ; ALL-LABEL: @minimum_test(
 ; ALL-NEXT:  entry:
 ; ALL-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; ALL-NEXT:    [[TMP0:%.*]] = tail call float @llvm.minimum.f32(float [[A]], float [[B:%.*]]) #2
+; ALL-NEXT:    [[TMP0:%.*]] = tail call float @llvm.minimum.f32(float [[A]], float [[B:%.*]]) [[ATTR2]]
 ; ALL-NEXT:    [[COND_I:%.*]] = select reassoc i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; ALL-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; ALL-NEXT:    ret void
 ;
 entry:
   %cmp.i = fcmp olt float %a, 0.000000e+00
   br i1 %cmp.i, label %test_minimum.exit, label %cond.else.i
 
 cond.else.i:                                      ; preds = %entry
   %0 = tail call float @llvm.minimum.f32(float %a, float %b) nounwind readnone
   br label %test_minimum.exit
 
 test_minimum.exit:                                   ; preds = %cond.else.i, %entry
   %cond.i = phi reassoc float [ %0, %cond.else.i ], [ 0x7FF8000000000000, %entry ]
   store float %cond.i, float addrspace(1)* %out, align 4
   ret void
 }
 
 define void @maximum_test(float addrspace(1)* noalias nocapture %out, float %a, float %b) nounwind {
 ; ALL-LABEL: @maximum_test(
 ; ALL-NEXT:  entry:
 ; ALL-NEXT:    [[CMP_I:%.*]] = fcmp olt float [[A:%.*]], 0.000000e+00
-; ALL-NEXT:    [[TMP0:%.*]] = tail call float @llvm.maximum.f32(float [[A]], float [[B:%.*]]) #2
+; ALL-NEXT:    [[TMP0:%.*]] = tail call float @llvm.maximum.f32(float [[A]], float [[B:%.*]]) [[ATTR2]]
 ; ALL-NEXT:    [[COND_I:%.*]] = select nsz i1 [[CMP_I]], float 0x7FF8000000000000, float [[TMP0]]
 ; ALL-NEXT:    store float [[COND_I]], float addrspace(1)* [[OUT:%.*]], align 4
 ; ALL-NEXT:    ret void
 ;
 entry:
   %cmp.i = fcmp olt float %a, 0.000000e+00
   br i1 %cmp.i, label %test_maximum.exit, label %cond.else.i