[Pipeline] Remove GlobalCleanupPM

[aeubanks]

Summary

This is mostly unnecessary given we're about to run the simplification pipeline anyway.

A couple of remarks, some to do with not regressing PhaseOrdering tests.
The main issue with just removing GlobalCleanupPM is that we no longer run InstCombine before the inliner.
Another issue is that CGSCC passes before the function simplification pipeline may have less precise IR to work with.
SROA -> EarlyCSE -> InstCombine -> SimplifyCFG should be the passes at the top of the function simplification pipeline.
InstCombine/SimplifyCFG need to run before JumpThreading, so move JumpThreading after those.
This also adds an extra EarlyCSE in the -O1 pipeline to match where GVN runs in other pipelines, otherwise we never run EarlyCSE after InstCombine.

Fairly substantial compile time wins:
https://llvm-compile-time-tracker.com/compare.php?from=a5595e9f1feb5960132ffcef539ab425abbe97cc&to=2baa33baa9bfa722329a728a100610daba60c559&stat=instructions:u

IR diffs on llvm-test-suite:
https://github.com/aeubanks/llvm-ir-diff/commit/f833481abce7d2310c84029a5df08d4d9bd943e5
The diffs mostly look neutral except for some gcc torture suite files which are arguably unreasonable.

This regresses D117091 because we now do not run instcombine before the inliner, losing return value alignment info.
This regresses pr52289.ll which was a fuzzed test case to begin with (and never properly fixed).

[aeubanks]

D145210 fixes cgscc-devirt-iteration.ll

@spatel @anton-afanasyev for the instcombine issue

I see some regressions on internal benchmarks from this, investigating those

llvm/test/Transforms/PhaseOrdering/X86/pr52289.ll

9

this is an instcombine issue related to https://github.com/llvm/llvm-project/issues/51631 which has some bugs split out from it that are still open

[aeubanks]

rebase, update tests

[aeubanks]

llvm/test/Transforms/PhaseOrdering/X86/pr52289.ll

9

https://github.com/llvm/llvm-project/issues/51744 and https://github.com/llvm/llvm-project/issues/51748 are the open issues for this. is it ok to regress this for now given that the original motivation for this was found via fuzzing and not real world code?

[aeubanks]

add FIXME to phase ordering test

[nikic]

I think this (removing GlobalCleanupPM) makes sense at a high level, but this is a pretty substantial pipeline change and will need more thorough evaluation (e.g. looking at IR diffs on test suite). For example, it is not at all obvious to me that moving InstCombine before EarlyCSE is harmless, given the impact this may have on one-use heuristics.

[aeubanks]

got IR diffs for llvm-test-suite by dumping the IR in clang after running the optimization pipeline (-O3), IR file name is md5 of the module identifier

@@@ -1093,6 -1093,6 +1096,16 @@@ void EmitAssemblyHelper::EmitAssembly(B
  
    std::unique_ptr<llvm::ToolOutputFile> ThinLinkOS, DwoOS;
    RunOptimizationPipeline(Action, OS, ThinLinkOS);
++  if (!TheModule->getModuleIdentifier().empty()) {
++    std::error_code EC;
++    std::string FileName = "/usr/local/google/home/aeubanks/tmp/wow3/";
++    FileName += llvm::utohexstr(llvm::MD5Hash(TheModule->getModuleIdentifier()), false, 16);
++    FileName += ".ll";
++    llvm::raw_fd_ostream OS(FileName, EC);
++    if (EC)
++      report_fatal_error(createStringError(EC, "???"));
++    TheModule->print(OS, nullptr);
++  }
    RunCodegenPipeline(Action, OS, DwoOS);

all IR diffs: https://github.com/aeubanks/llvm-ir-diff/commit/ca19575407a94933d10e9b493784cce47751d0aa
diffs with <50 lines changed: https://github.com/aeubanks/llvm-ir-diff/commit/5592d7e71191501b6029af1daa1ee438076f4913

I'm going to be out for a couple weeks, but a couple initial observations:
4508D45179E820C5.ll: we don't run instcombine anymore before getting to the function simplification pipeline, and that causes argpromo to fail. perhaps we should add back in an early instcombine somewhere. (but the forced noinline in the file prevents obvious inlining)
1A365B06B0D46C86.ll: select i1 %37, i32 %38, i32 %35 becomes call i32 @llvm.smin.i32(i32 %36, i32 4095)
1ACE620D6E75417F.ll: extra memset
A4B000DC4289863A.ll: missed used once global -> alloca

will see if running an early instcombine resolves most of these issues

[asbirlea]

Nit: the diffs would be easier to analyze with "-fno-discard-value-names".

[aeubanks]

update

[aeubanks]

fix clang test

[aeubanks]

In D145265#4225332, @asbirlea wrote:

Nit: the diffs would be easier to analyze with "-fno-discard-value-names".

done, see commit description

the IR diffs look pretty good now, I'm not seeing any major regressions

[aeubanks]

this regresses clang/test/Headers/mm_malloc.c (added here) because we now don't run instcombine before the inliner, and we lose return value alignment info when inlining. perhaps this is still a tradeoff worth making

[aeubanks]

rebase

keep the existing EarlyCSE/InstCombine ordering

[nikic]

llvm/lib/Passes/PassBuilderPipelines.cpp

419

Why the extra EarlyCSE pass in the O1 pipeline?

495

Any particular reason why InstCombine and AggressiveInstCombine are no longer directly next to each other?

[aeubanks]

update commit message

[aeubanks]

llvm/lib/Passes/PassBuilderPipelines.cpp

419

forgot to update the commit message, see that

495

we need InstCombine -> SimplifyCFG, so swapped them (see updated commit message). I don't necessarily see why AIC needs to be right after IC

[asbirlea]

This went through a few rounds of testing on our internal benchmarks and it's at a point where there are no meaningful run-time regressions observed, but the compile-time improvements remain and are significant enough to move fwd with the change.
Approving to move things forward. If folks do encounter regressions from this change before or after landing, please flag them here.

[nikic]

Unfortunately I'm seeing a number of Rust optimization regressions with this change. I'll try to reduce those to some PhaseOrdering tests.

[nikic]

Here are two inputs that currently fold down to constants and no longer do so with this patch: https://gist.github.com/nikic/4a714ea550bf2252543570585f642af2 These need further reduction for PhaseOrdering tests, but should be a good starting point for analysis...

[aeubanks]

rebase

[aeubanks]

@nikic could you try running this over the rust tests again?

[nikic]

In D145265#4478621, @aeubanks wrote:

@nikic could you try running this over the rust tests again?

Ignoring some unrelated powerpc data layout failures, these codegen tests fail:

[codegen] tests/codegen/issues/issue-45222.rs
[codegen] tests/codegen/issues/issue-45466.rs
[codegen] tests/codegen/issues/issue-69101-bounds-check.rs
[codegen] tests/codegen/iter-repeat-n-trivial-drop.rs
[codegen] tests/codegen/slice-as_chunks.rs

The iter-repeat-n-trivial-drop.rs and slice-as_chunks.rs failures are cosmetic.

issues-45222.rs is the same as last time (loop not folded to constant).

issue-45466.rs fails to convert a loop into a memset 0 (this looks like the simplest test case, because it has essentially just one function in initial IR).

issue-69101-bounds-check.rs leaves behind panic_bounds_checks in already_sliced_no_bounds_check.

I've put the input IR for these three failure up at https://gist.github.com/nikic/9ec0bc02d451e7abd0f8d78800c2206c.

llvm/lib/Passes/PassBuilderPipelines.cpp

PassBuilder::buildO1FunctionSimplificationPipeline(OptimizationLevel Level,
 
   if (AreStatisticsEnabled())
     FPM.addPass(CountVisitsPass());
 
   // Form SSA out of local memory accesses after breaking apart aggregates into
   // scalars.
   FPM.addPass(SROAPass(SROAOptions::ModifyCFG));
 
-  // Catch trivial redundancies
-  FPM.addPass(EarlyCSEPass(true /* Enable mem-ssa. */));
-
-  // Hoisting of scalars and load expressions.
+  FPM.addPass(InstCombinePass());
   FPM.addPass(
       SimplifyCFGPass(SimplifyCFGOptions().convertSwitchRangeToICmp(true)));
-  FPM.addPass(InstCombinePass());
+
+  // Catch trivial redundancies
+  FPM.addPass(EarlyCSEPass(true /* Enable mem-ssa. */));
 
   FPM.addPass(LibCallsShrinkWrapPass());
 
   invokePeepholeEPCallbacks(FPM, Level);
 
-  FPM.addPass(
-      SimplifyCFGPass(SimplifyCFGOptions().convertSwitchRangeToICmp(true)));
-
   // Form canonically associated expression trees, and simplify the trees using
   // basic mathematical properties. For example, this will form (nearly)
   // minimal multiplication trees.
   FPM.addPass(ReassociatePass());
 
   // Add the primary loop simplification pipeline.
   // FIXME: Currently this is split into two loop pass pipelines because we run
   // some function passes in between them. These can and should be removed
PassBuilder::buildO1FunctionSimplificationPipeline(OptimizationLevel Level,
   // *All* loop passes must preserve it, in order to be able to use it.
   FPM.addPass(createFunctionToLoopPassAdaptor(std::move(LPM2),
                                               /*UseMemorySSA=*/false,
                                               /*UseBlockFrequencyInfo=*/false));
 
   // Delete small array after loop unroll.
   FPM.addPass(SROAPass(SROAOptions::ModifyCFG));
 
   // Specially optimize memory movement as it doesn't look like dataflow in SSA.
   FPM.addPass(MemCpyOptPass());
 
   // Sparse conditional constant propagation.
   // FIXME: It isn't clear why we do this *after* loop passes rather than
   // before...
   FPM.addPass(SCCPPass());
 
   // Delete dead bit computations (instcombine runs after to fold away the dead
PassBuilder::buildFunctionSimplificationPipeline(OptimizationLevel Level,
 
   if (AreStatisticsEnabled())
     FPM.addPass(CountVisitsPass());
 
   // Form SSA out of local memory accesses after breaking apart aggregates into
   // scalars.
   FPM.addPass(SROAPass(SROAOptions::ModifyCFG));
 
+  FPM.addPass(InstCombinePass());
+  FPM.addPass(
+      SimplifyCFGPass(SimplifyCFGOptions().convertSwitchRangeToICmp(true)));
+
   // Catch trivial redundancies
   FPM.addPass(EarlyCSEPass(true /* Enable mem-ssa. */));
   if (EnableKnowledgeRetention)
     FPM.addPass(AssumeSimplifyPass());
 
   // Hoisting of scalars and load expressions.
   if (EnableGVNHoist)
     FPM.addPass(GVNHoistPass());
 
   // Global value numbering based sinking.
   if (EnableGVNSink) {
     FPM.addPass(GVNSinkPass());
     FPM.addPass(
         SimplifyCFGPass(SimplifyCFGOptions().convertSwitchRangeToICmp(true)));
   }
 
   // Speculative execution if the target has divergent branches; otherwise nop.
   FPM.addPass(SpeculativeExecutionPass(/* OnlyIfDivergentTarget =*/true));
 
   // Optimize based on known information about branches, and cleanup afterward.
   FPM.addPass(JumpThreadingPass());
   FPM.addPass(CorrelatedValuePropagationPass());
 
-  FPM.addPass(
-      SimplifyCFGPass(SimplifyCFGOptions().convertSwitchRangeToICmp(true)));
-  FPM.addPass(InstCombinePass());
   if (Level == OptimizationLevel::O3)
     FPM.addPass(AggressiveInstCombinePass());
 
   if (EnableConstraintElimination)
     FPM.addPass(ConstraintEliminationPass());
 
   if (!Level.isOptimizingForSize())
     FPM.addPass(LibCallsShrinkWrapPass());
PassBuilder::buildModuleSimplificationPipeline(OptimizationLevel Level,
 
   // Attach metadata to indirect call sites indicating the set of functions
   // they may target at run-time. This should follow IPSCCP.
   MPM.addPass(CalledValuePropagationPass());
 
   // Optimize globals to try and fold them into constants.
   MPM.addPass(GlobalOptPass());
 
-  // Create a small function pass pipeline to cleanup after all the global
-  // optimizations.
-  FunctionPassManager GlobalCleanupPM;
-  // FIXME: Should this instead by a run of SROA?
-  GlobalCleanupPM.addPass(PromotePass());
-  GlobalCleanupPM.addPass(InstCombinePass());
-  invokePeepholeEPCallbacks(GlobalCleanupPM, Level);
-  GlobalCleanupPM.addPass(
-      SimplifyCFGPass(SimplifyCFGOptions().convertSwitchRangeToICmp(true)));
-  MPM.addPass(createModuleToFunctionPassAdaptor(std::move(GlobalCleanupPM),
-                                                PTO.EagerlyInvalidateAnalyses));
-
   // Add all the requested passes for instrumentation PGO, if requested.
   if (PGOOpt && Phase != ThinOrFullLTOPhase::ThinLTOPostLink &&
       (PGOOpt->Action == PGOOptions::IRInstr ||
        PGOOpt->Action == PGOOptions::IRUse)) {
     addPGOInstrPasses(MPM, Level,
                       /* RunProfileGen */ PGOOpt->Action == PGOOptions::IRInstr,
                       /* IsCS */ false, PGOOpt->ProfileFile,
                       PGOOpt->ProfileRemappingFile, Phase, PGOOpt->FS);

llvm/test/Transforms/Coroutines/coro-async.ll

 ; Make sure we update the async function pointer
 ; CHECK: @my_async_function_fp = constant <{ i32, i32 }> <{ {{.*}}, i32 176 }
 ; CHECK: @my_async_function_pa_fp = constant <{ i32, i32 }> <{ {{.*}}, i32 176 }
 ; CHECK: @my_async_function2_fp = constant <{ i32, i32 }> <{ {{.*}}, i32 176 }
 
 ; CHECK-LABEL: define swiftcc void @my_async_function(i8* swiftasync %async.ctxt, %async.task* %task, %async.actor* %actor)
 ; CHECK-O0-LABEL: define swiftcc void @my_async_function(i8* swiftasync %async.ctxt, %async.task* %task, %async.actor* %actor)
 ; CHECK-SAME: !dbg ![[SP1:[0-9]+]] {
-; CHECK: coro.return:
+; CHECK: entry:
 ; CHECK:   [[FRAMEPTR:%.*]] = getelementptr inbounds i8, i8* %async.ctxt, i64 128
 ; CHECK:   [[ACTOR_SPILL_ADDR:%.*]] = getelementptr inbounds i8, i8* %async.ctxt, i64 152
 ; CHECK:   [[CAST1:%.*]] = bitcast i8* [[ACTOR_SPILL_ADDR]] to %async.actor**
 ; CHECK:   store %async.actor* %actor, %async.actor** [[CAST1]]
 ; CHECK:   [[ADDR1:%.*]]  = getelementptr inbounds i8, i8* %async.ctxt, i64 144
 ; CHECK:   [[ASYNC_CTXT_SPILL_ADDR:%.*]] = bitcast i8* [[ADDR1]] to i8**
 ; CHECK:   store i8* %async.ctxt, i8** [[ASYNC_CTXT_SPILL_ADDR]]
 ; CHECK:   [[ALLOCA_PRJ1:%.*]] = bitcast i8* [[FRAMEPTR]] to i64*

llvm/test/Transforms/Coroutines/coro-retcon-alloca-opaque-ptr.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 
 ; RUN: opt < %s -passes='default<O2>' -opaque-pointers=1 -S | FileCheck %s
 
 target datalayout = "p:64:64:64"
 
 declare {ptr, ptr, i32} @prototype_f(ptr, i1)
 define {ptr, ptr, i32} @f(ptr %buffer, i32 %n, { i32 } %dummy) {
 ; CHECK-LABEL: @f(
-; CHECK-NEXT:  coro.return:
+; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[N_VAL_SPILL_ADDR:%.*]] = getelementptr inbounds [[F_FRAME:%.*]], ptr [[BUFFER:%.*]], i64 0, i32 1
 ; CHECK-NEXT:    store i32 [[N:%.*]], ptr [[N_VAL_SPILL_ADDR]], align 4
 ; CHECK-NEXT:    [[TMP0:%.*]] = tail call ptr @allocate(i32 [[N]])
 ; CHECK-NEXT:    store ptr [[TMP0]], ptr [[BUFFER]], align 8
 ; CHECK-NEXT:    [[TMP1:%.*]] = insertvalue { ptr, ptr, i32 } { ptr @f.resume.0, ptr undef, i32 undef }, ptr [[TMP0]], 1
 ; CHECK-NEXT:    [[TMP2:%.*]] = insertvalue { ptr, ptr, i32 } [[TMP1]], i32 [[N]], 2
 ; CHECK-NEXT:    ret { ptr, ptr, i32 } [[TMP2]]
 ;
cleanup:
   call i1 @llvm.coro.end(ptr %hdl, i1 0)
   unreachable
 }
 
 
 declare {ptr, i32} @prototype_g(ptr, i1)
 define {ptr, i32} @g(ptr %buffer, i32 %n) {
 ; CHECK-LABEL: @g(
-; CHECK-NEXT:  coro.return:
+; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    store i32 [[N:%.*]], ptr [[BUFFER:%.*]], align 4
 ; CHECK-NEXT:    [[TMP0:%.*]] = zext i32 [[N]] to i64
 ; CHECK-NEXT:    [[TMP1:%.*]] = alloca i8, i64 [[TMP0]], align 8
 ; CHECK-NEXT:    tail call void @use(ptr nonnull [[TMP1]])
 ; CHECK-NEXT:    [[TMP2:%.*]] = insertvalue { ptr, i32 } { ptr @g.resume.0, i32 undef }, i32 [[N]], 1
 ; CHECK-NEXT:    ret { ptr, i32 } [[TMP2]]
 ;
 entry:

llvm/test/Transforms/Coroutines/coro-retcon-alloca.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -opaque-pointers=0 < %s -passes='default<O2>' -S | FileCheck %s
 
 target datalayout = "p:64:64:64"
 
 declare {i8*, i8*, i32} @prototype_f(i8*, i1)
 define {i8*, i8*, i32} @f(i8* %buffer, i32 %n) {
 ; CHECK-LABEL: @f(
-; CHECK-NEXT:  coro.return:
+; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[N_VAL_SPILL_ADDR:%.*]] = getelementptr inbounds i8, i8* [[BUFFER:%.*]], i64 8
 ; CHECK-NEXT:    [[TMP0:%.*]] = bitcast i8* [[N_VAL_SPILL_ADDR]] to i32*
 ; CHECK-NEXT:    store i32 [[N:%.*]], i32* [[TMP0]], align 4
 ; CHECK-NEXT:    [[TMP1:%.*]] = tail call i8* @allocate(i32 [[N]])
 ; CHECK-NEXT:    [[DOTSPILL_ADDR:%.*]] = bitcast i8* [[BUFFER]] to i8**
 ; CHECK-NEXT:    store i8* [[TMP1]], i8** [[DOTSPILL_ADDR]], align 8
 ; CHECK-NEXT:    [[TMP2:%.*]] = insertvalue { i8*, i8*, i32 } { i8* bitcast ({ i8*, i8*, i32 } (i8*, i1)* @f.resume.0 to i8*), i8* undef, i32 undef }, i8* [[TMP1]], 1
 ; CHECK-NEXT:    [[TMP3:%.*]] = insertvalue { i8*, i8*, i32 } [[TMP2]], i32 [[N]], 2
cleanup:
   unreachable
 }
 
 
 
 declare {i8*, i32} @prototype_g(i8*, i1)
 define {i8*, i32} @g(i8* %buffer, i32 %n) {
 ; CHECK-LABEL: @g(
-; CHECK-NEXT:  coro.return:
+; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[N_VAL_SPILL_ADDR:%.*]] = bitcast i8* [[BUFFER:%.*]] to i32*
 ; CHECK-NEXT:    store i32 [[N:%.*]], i32* [[N_VAL_SPILL_ADDR]], align 4
 ; CHECK-NEXT:    [[TMP0:%.*]] = zext i32 [[N]] to i64
 ; CHECK-NEXT:    [[TMP1:%.*]] = alloca i8, i64 [[TMP0]], align 8
 ; CHECK-NEXT:    tail call void @use(i8* nonnull [[TMP1]])
 ; CHECK-NEXT:    [[TMP2:%.*]] = insertvalue { i8*, i32 } { i8* bitcast ({ i8*, i32 } (i8*, i1)* @g.resume.0 to i8*), i32 undef }, i32 [[N]], 1
 ; CHECK-NEXT:    ret { i8*, i32 } [[TMP2]]
 ;
cleanup:
   unreachable
 }
 
 
 
 declare {i8*, i32} @prototype_h(i8*, i1)
 define {i8*, i32} @h(i8* %buffer, i32 %n) {
 ; CHECK-LABEL: @h(
-; CHECK-NEXT:  coro.return:
+; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[N_SPILL_ADDR:%.*]] = bitcast i8* [[BUFFER:%.*]] to i32*
 ; CHECK-NEXT:    store i32 [[N:%.*]], i32* [[N_SPILL_ADDR]], align 4
 ; CHECK-NEXT:    [[TMP0:%.*]] = insertvalue { i8*, i32 } { i8* bitcast ({ i8*, i32 } (i8*, i1)* @h.resume.0 to i8*), i32 undef }, i32 [[N]], 1
 ; CHECK-NEXT:    ret { i8*, i32 } [[TMP0]]
 ;
 entry:
   %id = call token @llvm.coro.id.retcon(i32 1024, i32 8, i8* %buffer, i8* bitcast ({i8*, i32} (i8*, i1)* @prototype_h to i8*), i8* bitcast (i8* (i32)* @allocate to i8*), i8* bitcast (void (i8*)* @deallocate to i8*))
   %hdl = call i8* @llvm.coro.begin(token %id, i8* null)
cleanup:
   unreachable
 }
 
 
 
 declare {i8*, i32} @prototype_i(i8*)
 define {i8*, i32} @i(i8* %buffer, i32 %n) {
 ; CHECK-LABEL: @i(
-; CHECK-NEXT:  coro.return:
+; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[N_VAL_SPILL_ADDR:%.*]] = bitcast i8* [[BUFFER:%.*]] to i32*
 ; CHECK-NEXT:    store i32 [[N:%.*]], i32* [[N_VAL_SPILL_ADDR]], align 4
 ; CHECK-NEXT:    [[TMP0:%.*]] = insertvalue { i8*, i32 } { i8* bitcast ({ i8*, i32 } (i8*)* @i.resume.0 to i8*), i32 undef }, i32 [[N]], 1
 ; CHECK-NEXT:    ret { i8*, i32 } [[TMP0]]
 ;
 entry:
   %id = call token @llvm.coro.id.retcon(i32 1024, i32 8, i8* %buffer, i8* bitcast ({i8*, i32} (i8*)* @prototype_i to i8*), i8* bitcast (i8* (i32)* @allocate to i8*), i8* bitcast (void (i8*)* @deallocate to i8*))
   %hdl = call i8* @llvm.coro.begin(token %id, i8* null)
loop2:
   br i1 %cmp, label %loop2, label %loop
 }
 
 
 
 declare {i8*, i32} @prototype_j(i8*)
 define {i8*, i32} @j(i8* %buffer, i32 %n) {
 ; CHECK-LABEL: @j(
-; CHECK-NEXT:  coro.return:
+; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[N_VAL_SPILL_ADDR:%.*]] = bitcast i8* [[BUFFER:%.*]] to i32*
 ; CHECK-NEXT:    store i32 [[N:%.*]], i32* [[N_VAL_SPILL_ADDR]], align 4
 ; CHECK-NEXT:    [[TMP0:%.*]] = insertvalue { i8*, i32 } { i8* bitcast ({ i8*, i32 } (i8*)* @j.resume.0 to i8*), i32 undef }, i32 [[N]], 1
 ; CHECK-NEXT:    ret { i8*, i32 } [[TMP0]]
 ;
 entry:
   %id = call token @llvm.coro.id.retcon(i32 1024, i32 8, i8* %buffer, i8* bitcast ({i8*, i32} (i8*)* @prototype_j to i8*), i8* bitcast (i8* (i32)* @allocate to i8*), i8* bitcast (void (i8*)* @deallocate to i8*))
   %hdl = call i8* @llvm.coro.begin(token %id, i8* null)

 end:
   call i1 @llvm.coro.end(i8* %hdl, i1 0)
   unreachable
 }
 
 declare i32 @getSize()
 define {i8*, i32} @k(i8* %buffer, i32 %n, i1 %cond) {
 ; CHECK-LABEL: @k(
-; CHECK-NEXT:  PostSpill:
+; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[SIZE:%.*]] = tail call i32 @getSize()
 ; CHECK-NEXT:    br i1 [[COND:%.*]], label [[ALLOCA_BLOCK:%.*]], label [[CORO_RETURN:%.*]]
 ; CHECK:       coro.return:
 ; CHECK-NEXT:    [[TMP0:%.*]] = insertvalue { i8*, i32 } { i8* bitcast ({ i8*, i32 } (i8*, i1)* @k.resume.0 to i8*), i32 undef }, i32 [[N:%.*]], 1
 ; CHECK-NEXT:    ret { i8*, i32 } [[TMP0]]
 ; CHECK:       alloca_block:
 ; CHECK-NEXT:    [[TMP1:%.*]] = zext i32 [[SIZE]] to i64
 ; CHECK-NEXT:    [[TMP2:%.*]] = alloca i8, i64 [[TMP1]], align 8

llvm/test/Transforms/Coroutines/coro-retcon-once-value.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -opaque-pointers=0 < %s -passes='default<O2>' -S | FileCheck %s
 
 target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-apple-macosx10.12.0"
 
 define {i8*, i32} @f(i8* %buffer, i32* %array) {
 ; CHECK-LABEL: @f(
-; CHECK-NEXT:  PostSpill:
+; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[ARRAY_SPILL_ADDR:%.*]] = bitcast i8* [[BUFFER:%.*]] to i32**
 ; CHECK-NEXT:    store i32* [[ARRAY:%.*]], i32** [[ARRAY_SPILL_ADDR]], align 8
 ; CHECK-NEXT:    [[LOAD:%.*]] = load i32, i32* [[ARRAY]], align 4
 ; CHECK-NEXT:    [[LOAD_POS:%.*]] = icmp sgt i32 [[LOAD]], 0
-; CHECK-NEXT:    [[TMP0:%.*]] = tail call i32 @llvm.smax.i32(i32 [[LOAD]], i32 0)
-; CHECK-NEXT:    [[TMP1:%.*]] = select i1 [[LOAD_POS]], i8* bitcast (void (i8*, i1)* @f.resume.0 to i8*), i8* bitcast (void (i8*, i1)* @f.resume.1 to i8*)
-; CHECK-NEXT:    [[TMP2:%.*]] = insertvalue { i8*, i32 } undef, i8* [[TMP1]], 0
-; CHECK-NEXT:    [[TMP3:%.*]] = insertvalue { i8*, i32 } [[TMP2]], i32 [[TMP0]], 1
+; CHECK-NEXT:    [[TMP0:%.*]] = select i1 [[LOAD_POS]], i8* bitcast (void (i8*, i1)* @f.resume.0 to i8*), i8* bitcast (void (i8*, i1)* @f.resume.1 to i8*)
+; CHECK-NEXT:    [[TMP1:%.*]] = tail call i32 @llvm.smax.i32(i32 [[LOAD]], i32 0)
+; CHECK-NEXT:    [[TMP2:%.*]] = insertvalue { i8*, i32 } undef, i8* [[TMP0]], 0
+; CHECK-NEXT:    [[TMP3:%.*]] = insertvalue { i8*, i32 } [[TMP2]], i32 [[TMP1]], 1
 ; CHECK-NEXT:    ret { i8*, i32 } [[TMP3]]
 ;
 entry:
   %id = call token @llvm.coro.id.retcon.once(i32 8, i32 8, i8* %buffer, i8* bitcast (void (i8*, i1)* @prototype to i8*), i8* bitcast (i8* (i32)* @allocate to i8*), i8* bitcast (void (i8*)* @deallocate to i8*))
   %hdl = call i8* @llvm.coro.begin(token %id, i8* null)
   %load = load i32, i32* %array
   %load.pos = icmp sgt i32 %load, 0
   br i1 %load.pos, label %pos, label %neg

 cleanup:
   call i1 @llvm.coro.end(i8* %hdl, i1 0)
   unreachable
 }
 
 define void @test(i32* %array) {
 ; CHECK-LABEL: @test(
 ; CHECK-NEXT:  entry:
-; CHECK-NEXT:    [[TMP0:%.*]] = alloca i32*, align 8
-; CHECK-NEXT:    [[DOTSUB:%.*]] = bitcast i32** [[TMP0]] to i8*
-; CHECK-NEXT:    store i32* [[ARRAY:%.*]], i32** [[TMP0]], align 8
+; CHECK-NEXT:    [[BUFFER1:%.*]] = alloca i32*, align 8
+; CHECK-NEXT:    [[BUFFER1_SUB:%.*]] = bitcast i32** [[BUFFER1]] to i8*
+; CHECK-NEXT:    store i32* [[ARRAY:%.*]], i32** [[BUFFER1]], align 8
 ; CHECK-NEXT:    [[LOAD_I:%.*]] = load i32, i32* [[ARRAY]], align 4
 ; CHECK-NEXT:    [[LOAD_POS_I:%.*]] = icmp sgt i32 [[LOAD_I]], 0
-; CHECK-NEXT:    [[TMP1:%.*]] = tail call i32 @llvm.smax.i32(i32 [[LOAD_I]], i32 0)
-; CHECK-NEXT:    tail call void @print(i32 [[TMP1]])
+; CHECK-NEXT:    [[TMP0:%.*]] = tail call i32 @llvm.smax.i32(i32 [[LOAD_I]], i32 0)
+; CHECK-NEXT:    tail call void @print(i32 [[TMP0]])
 ; CHECK-NEXT:    [[CONT_CAST:%.*]] = select i1 [[LOAD_POS_I]], void (i8*, i1)* @f.resume.0, void (i8*, i1)* @f.resume.1
-; CHECK-NEXT:    call void [[CONT_CAST]](i8* nonnull [[DOTSUB]], i1 zeroext false)
+; CHECK-NEXT:    call void [[CONT_CAST]](i8* nonnull [[BUFFER1_SUB]], i1 zeroext false)
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %0 = alloca [8 x i8], align 8
   %buffer = bitcast [8 x i8]* %0 to i8*
   %prepare = call i8* @llvm.coro.prepare.retcon(i8* bitcast ({i8*, i32} (i8*, i32*)* @f to i8*))
   %f = bitcast i8* %prepare to {i8*, i32} (i8*, i32*)*
   %result = call {i8*, i32} %f(i8* %buffer, i32* %array)

llvm/test/Transforms/Coroutines/coro-retcon-opaque-ptr.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt < %s -passes='default<O2>' -opaque-pointers -S | FileCheck %s
 
 ; Same test as coro-retcon.ll, but with opaque pointers enabled.
 
 define ptr @f(ptr %buffer, i32 %n) {
 ; CHECK-LABEL: @f(
-; CHECK-NEXT:  coro.return:
+; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    store i32 [[N:%.*]], ptr [[BUFFER:%.*]], align 4
 ; CHECK-NEXT:    tail call void @print(i32 [[N]])
 ; CHECK-NEXT:    ret ptr @f.resume.0
 ;
 entry:
   %id = call token @llvm.coro.id.retcon(i32 8, i32 4, ptr %buffer, ptr @prototype, ptr @allocate, ptr @deallocate)
   %hdl = call ptr @llvm.coro.begin(token %id, ptr null)
   br label %loop

 cleanup:                                          ; preds = %loop
   %0 = call i1 @llvm.coro.end(ptr %hdl, i1 false)
   unreachable
 }
 
 define i32 @main() {
 ; CHECK-LABEL: @main(
 ; CHECK-NEXT:  entry:
-; CHECK-NEXT:    call void @print(i32 4)
-; CHECK-NEXT:    call void @print(i32 5), !noalias !0
-; CHECK-NEXT:    call void @print(i32 6), !noalias !3
+; CHECK-NEXT:    tail call void @print(i32 4)
+; CHECK-NEXT:    tail call void @print(i32 5), !noalias !0
+; CHECK-NEXT:    tail call void @print(i32 6), !noalias !3
 ; CHECK-NEXT:    ret i32 0
 ;
 entry:
   %0 = alloca [8 x i8], align 4
   %prepare = call ptr @llvm.coro.prepare.retcon(ptr @f)
   %cont0 = call ptr %prepare(ptr %0, i32 4)
   %cont1 = call ptr %cont0(ptr %0, i1 zeroext false)
   %cont2 = call ptr %cont1(ptr %0, i1 zeroext false)
   %1 = call ptr %cont2(ptr %0, i1 zeroext true)
   ret i32 0
 }
 
 define hidden { ptr, ptr } @g(ptr %buffer, ptr %ptr) {
 ; CHECK-LABEL: @g(
-; CHECK-NEXT:  coro.return:
+; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[TMP0:%.*]] = tail call ptr @allocate(i32 8)
 ; CHECK-NEXT:    store ptr [[TMP0]], ptr [[BUFFER:%.*]], align 8
 ; CHECK-NEXT:    store ptr [[PTR:%.*]], ptr [[TMP0]], align 8
 ; CHECK-NEXT:    [[TMP1:%.*]] = insertvalue { ptr, ptr } { ptr @g.resume.0, ptr undef }, ptr [[PTR]], 1
 ; CHECK-NEXT:    ret { ptr, ptr } [[TMP1]]
 ;
 entry:
   %id = call token @llvm.coro.id.retcon(i32 8, i32 4, ptr %buffer, ptr @g_prototype, ptr @allocate, ptr @deallocate)

llvm/test/Transforms/Coroutines/coro-retcon-resume-values.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -opaque-pointers=0 < %s -passes="default<O2>" -aa-pipeline=default -S | FileCheck %s
 
 define i8* @f(i8* %buffer, i32 %n) {
 ; CHECK-LABEL: @f(
-; CHECK-NEXT:  coro.return:
+; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[TMP0:%.*]] = tail call i8* @allocate(i32 12)
 ; CHECK-NEXT:    [[TMP1:%.*]] = bitcast i8* [[BUFFER:%.*]] to i8**
 ; CHECK-NEXT:    store i8* [[TMP0]], i8** [[TMP1]], align 8
 ; CHECK-NEXT:    [[N_SPILL_ADDR:%.*]] = bitcast i8* [[TMP0]] to i32*
 ; CHECK-NEXT:    store i32 [[N:%.*]], i32* [[N_SPILL_ADDR]], align 4
 ; CHECK-NEXT:    ret i8* bitcast (i8* (i8*, i32, i1)* @f.resume.0 to i8*)
 ;
 entry:

 ; CHECK-NEXT:    [[INPUT_RELOAD_ADDR13_I:%.*]] = getelementptr inbounds i8, i8* [[TMP0]], i64 8
 ; CHECK-NEXT:    [[TMP3:%.*]] = bitcast i8* [[INPUT_RELOAD_ADDR13_I]] to i32*
 ; CHECK-NEXT:    [[N_VAL3_RELOAD_ADDR11_I:%.*]] = getelementptr inbounds i8, i8* [[TMP0]], i64 4
 ; CHECK-NEXT:    [[TMP4:%.*]] = bitcast i8* [[N_VAL3_RELOAD_ADDR11_I]] to i32*
 ; CHECK-NEXT:    [[N_VAL3_RELOAD12_I:%.*]] = load i32, i32* [[TMP4]], align 4, !noalias !3
 ; CHECK-NEXT:    [[SUM7_I:%.*]] = add i32 [[N_VAL3_RELOAD12_I]], 2
 ; CHECK-NEXT:    store i32 [[SUM7_I]], i32* [[TMP4]], align 4, !noalias !3
 ; CHECK-NEXT:    store i32 4, i32* [[TMP3]], align 4, !noalias !3
-; CHECK-NEXT:    [[SUM7_I7:%.*]] = add i32 [[N_VAL3_RELOAD12_I]], 6
-; CHECK-NEXT:    tail call void @print(i32 [[SUM7_I7]]), !noalias !6
+; CHECK-NEXT:    [[SUM7_I8:%.*]] = add i32 [[N_VAL3_RELOAD12_I]], 6
+; CHECK-NEXT:    tail call void @print(i32 [[SUM7_I8]]), !noalias !6
 ; CHECK-NEXT:    tail call void @deallocate(i8* [[TMP0]]), !noalias !6
 ; CHECK-NEXT:    ret i32 0
 ;
 entry:
   %0 = alloca [8 x i8], align 4
   %buffer = bitcast [8 x i8]* %0 to i8*
   %prepare = call i8* @llvm.coro.prepare.retcon(i8* bitcast (i8* (i8*, i32)* @f to i8*))
   %f = bitcast i8* %prepare to i8* (i8*, i32)*

llvm/test/Transforms/Coroutines/coro-retcon-value.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; First example from Doc/Coroutines.rst (two block loop) converted to retcon
 ; RUN: opt < %s -passes='default<O2>' -S | FileCheck %s
 
 define {ptr, i32} @f(ptr %buffer, i32 %n) {
 ; CHECK-LABEL: @f(
-; CHECK-NEXT:  coro.return:
+; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    store i32 [[N:%.*]], ptr [[BUFFER:%.*]], align 4
 ; CHECK-NEXT:    [[TMP0:%.*]] = insertvalue { ptr, i32 } { ptr @f.resume.0, i32 undef }, i32 [[N]], 1
 ; CHECK-NEXT:    ret { ptr, i32 } [[TMP0]]
 ;
 entry:
   %id = call token @llvm.coro.id.retcon(i32 8, i32 4, ptr %buffer, ptr @prototype, ptr @allocate, ptr @deallocate)
   %hdl = call ptr @llvm.coro.begin(token %id, ptr null)
   br label %loop
cleanup:
   unreachable
 }
 
 
 
 define i32 @main() {
 ; CHECK-LABEL: @main(
 ; CHECK-NEXT:  entry:
-; CHECK-NEXT:    call void @print(i32 4)
-; CHECK-NEXT:    call void @print(i32 5)
-; CHECK-NEXT:    call void @print(i32 6)
+; CHECK-NEXT:    tail call void @print(i32 4)
+; CHECK-NEXT:    tail call void @print(i32 5)
+; CHECK-NEXT:    tail call void @print(i32 6)
 ; CHECK-NEXT:    ret i32 0
 ;
 entry:
   %0 = alloca [8 x i8], align 4
   %prepare = call ptr @llvm.coro.prepare.retcon(ptr @f)
   %result0 = call {ptr, i32} %prepare(ptr %0, i32 4)
   %value0 = extractvalue {ptr, i32} %result0, 1
   call void @print(i32 %value0)

llvm/test/Transforms/Coroutines/coro-retcon.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; First example from Doc/Coroutines.rst (two block loop) converted to retcon
 ; RUN: opt -opaque-pointers=0 < %s -passes='default<O2>' -S | FileCheck %s
 ; RUN: opt -opaque-pointers=0 < %s -passes='module(coro-early),cgscc(coro-split),module(coro-cleanup)' -S | FileCheck --check-prefix=CORO %s
 
 define i8* @f(i8* %buffer, i32 %n) {
 ; CHECK-LABEL: @f(
-; CHECK-NEXT:  coro.return:
+; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[N_VAL_SPILL_ADDR:%.*]] = bitcast i8* [[BUFFER:%.*]] to i32*
 ; CHECK-NEXT:    store i32 [[N:%.*]], i32* [[N_VAL_SPILL_ADDR]], align 4
 ; CHECK-NEXT:    tail call void @print(i32 [[N]])
 ; CHECK-NEXT:    ret i8* bitcast (i8* (i8*, i1)* @f.resume.0 to i8*)
 ;
 ; CORO-LABEL: @f(
 ; CORO-NEXT:  entry:
 ; CORO-NEXT:    [[FRAMEPTR:%.*]] = bitcast i8* [[BUFFER:%.*]] to %f.Frame*
entry:
   ret i32 0
 }
 
 ;   Unfortunately, we don't seem to fully optimize this right now due
 ;   to some sort of phase-ordering thing.
 
 define hidden { i8*, i8* } @g(i8* %buffer, i16* %ptr) {
 ; CHECK-LABEL: @g(
-; CHECK-NEXT:  coro.return:
+; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[TMP0:%.*]] = tail call i8* @allocate(i32 8)
 ; CHECK-NEXT:    [[TMP1:%.*]] = bitcast i8* [[BUFFER:%.*]] to i8**
 ; CHECK-NEXT:    store i8* [[TMP0]], i8** [[TMP1]], align 8
 ; CHECK-NEXT:    [[PTR_SPILL_ADDR:%.*]] = bitcast i8* [[TMP0]] to i16**
 ; CHECK-NEXT:    store i16* [[PTR:%.*]], i16** [[PTR_SPILL_ADDR]], align 8
-; CHECK-NEXT:    [[TMP2:%.*]] = bitcast i16* [[PTR]] to i8*
-; CHECK-NEXT:    [[TMP3:%.*]] = insertvalue { i8*, i8* } { i8* bitcast ({ i8*, i8* } (i8*, i1)* @g.resume.0 to i8*), i8* undef }, i8* [[TMP2]], 1
-; CHECK-NEXT:    ret { i8*, i8* } [[TMP3]]
+; CHECK-NEXT:    [[PTR_RELOAD1_CAST:%.*]] = bitcast i16* [[PTR]] to i8*
+; CHECK-NEXT:    [[TMP2:%.*]] = insertvalue { i8*, i8* } { i8* bitcast ({ i8*, i8* } (i8*, i1)* @g.resume.0 to i8*), i8* undef }, i8* [[PTR_RELOAD1_CAST]], 1
+; CHECK-NEXT:    ret { i8*, i8* } [[TMP2]]
 ;
 ; CORO-LABEL: @g(
 ; CORO-NEXT:  entry:
 ; CORO-NEXT:    [[TMP0:%.*]] = call i8* @allocate(i32 8)
 ; CORO-NEXT:    [[TMP1:%.*]] = bitcast i8* [[BUFFER:%.*]] to i8**
 ; CORO-NEXT:    store i8* [[TMP0]], i8** [[TMP1]], align 8
 ; CORO-NEXT:    [[FRAMEPTR:%.*]] = bitcast i8* [[TMP0]] to %g.Frame*
 ; CORO-NEXT:    [[PTR_SPILL_ADDR:%.*]] = getelementptr inbounds [[G_FRAME:%.*]], %g.Frame* [[FRAMEPTR]], i32 0, i32 0

llvm/test/Transforms/Coroutines/coro-swifterror.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -opaque-pointers=0 < %s -passes='default<O2>' -S | FileCheck %s
 target datalayout = "E-p:32:32"
 
 define i8* @f(i8* %buffer, i32 %n, i8** swifterror %errorslot) {
 ; CHECK-LABEL: @f(
-; CHECK-NEXT:  coro.return:
+; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[N_VAL_SPILL_ADDR:%.*]] = bitcast i8* [[BUFFER:%.*]] to i32*
 ; CHECK-NEXT:    store i32 [[N:%.*]], i32* [[N_VAL_SPILL_ADDR]], align 4
 ; CHECK-NEXT:    tail call void @print(i32 [[N]])
 ; CHECK-NEXT:    store i8* null, i8** [[ERRORSLOT:%.*]], align 4
 ; CHECK-NEXT:    tail call void @maybeThrow(i8** nonnull swifterror [[ERRORSLOT]])
 ; CHECK-NEXT:    [[TMP0:%.*]] = load i8*, i8** [[ERRORSLOT]], align 4
 ; CHECK-NEXT:    tail call void @logError(i8* [[TMP0]])
 ; CHECK-NEXT:    store i8* [[TMP0]], i8** [[ERRORSLOT]], align 4
cleanup:
   unreachable
 }
 
 ;   TODO: figure out a way to eliminate this
 
 
 define i8* @g(i8* %buffer, i32 %n) {
 ; CHECK-LABEL: @g(
-; CHECK-NEXT:  coro.return:
+; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[TMP0:%.*]] = alloca swifterror i8*, align 4
 ; CHECK-NEXT:    [[N_VAL_SPILL_ADDR:%.*]] = bitcast i8* [[BUFFER:%.*]] to i32*
 ; CHECK-NEXT:    store i32 [[N:%.*]], i32* [[N_VAL_SPILL_ADDR]], align 4
 ; CHECK-NEXT:    tail call void @print(i32 [[N]])
 ; CHECK-NEXT:    store i8* null, i8** [[TMP0]], align 4
 ; CHECK-NEXT:    call void @maybeThrow(i8** nonnull swifterror [[TMP0]])
 ; CHECK-NEXT:    [[TMP1:%.*]] = load i8*, i8** [[TMP0]], align 4
 ; CHECK-NEXT:    [[DOTSPILL_ADDR:%.*]] = getelementptr inbounds i8, i8* [[BUFFER]], i32 4

llvm/test/Transforms/PhaseOrdering/X86/pr52289.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -O3 -S < %s | FileCheck %s
 
 target datalayout = "n8:16:32:64"
 
 define i32 @main(i32 %a) {
 ; CHECK-LABEL: @main(
 ; CHECK-NEXT:  if.end:
+; CHECK-NEXT:    [[TMP0:%.*]] = add i32 [[A:%.*]], 1
+; CHECK-NEXT:    [[CONV:%.*]] = and i32 [[TMP0]], 8
+; CHECK-NEXT:    [[CMP_I_NOT:%.*]] = icmp eq i32 [[CONV]], 0
+; CHECK-NEXT:    [[SHL_I:%.*]] = select i1 [[CMP_I_NOT]], i32 7, i32 0
+; CHECK-NEXT:    [[COND_I:%.*]] = shl nuw nsw i32 [[CONV]], [[SHL_I]]
+; CHECK-NEXT:    [[CONV4_I:%.*]] = zext i32 [[COND_I]] to i64
+; CHECK-NEXT:    [[SEXT:%.*]] = shl i64 [[CONV4_I]], 56
+; CHECK-NEXT:    [[CONV1:%.*]] = ashr exact i64 [[SEXT]], 56
+; CHECK-NEXT:    [[TMP1:%.*]] = trunc i64 [[CONV1]] to i32
+; CHECK-NEXT:    [[SEXT1:%.*]] = mul i32 [[TMP1]], 1355350016
+; CHECK-NEXT:    [[TOBOOL:%.*]] = icmp ne i32 [[SEXT1]], 65536
+; CHECK-NEXT:    tail call void @llvm.assume(i1 [[TOBOOL]])
 ; CHECK-NEXT:    ret i32 0
 ;
   %inc = add nsw i32 %a, 1
   %and = and i32 %inc, 8
   %conv = trunc i32 %and to i8
   %call = call fastcc i8 @a(i8 %conv, i32 7)
   %conv1 = sext i8 %call to i64
   %call2 = call fastcc i64 @d(i64 %conv1)

llvm/test/Transforms/PhaseOrdering/inlining-alignment-assumptions.ll

 
 define internal i1 @callee1(i1 %c, ptr align 8 %ptr) {
   store volatile i64 0, ptr %ptr
   ret i1 %c
 }
 
 define void @caller1(i1 %c, ptr align 1 %ptr) {
 ; ASSUMPTIONS-OFF-LABEL: @caller1(
-; ASSUMPTIONS-OFF-NEXT:    br i1 [[C:%.*]], label [[COMMON_RET:%.*]], label [[FALSE2:%.*]]
+; ASSUMPTIONS-OFF-NEXT:    br i1 [[C:%.*]], label [[COMMON_RET:%.*]], label [[FALSE1:%.*]]
+; ASSUMPTIONS-OFF:       false1:
+; ASSUMPTIONS-OFF-NEXT:    store volatile i64 1, ptr [[PTR:%.*]], align 4
+; ASSUMPTIONS-OFF-NEXT:    br label [[COMMON_RET]]
 ; ASSUMPTIONS-OFF:       common.ret:
-; ASSUMPTIONS-OFF-NEXT:    [[DOTSINK:%.*]] = phi i64 [ 3, [[FALSE2]] ], [ 2, [[TMP0:%.*]] ]
-; ASSUMPTIONS-OFF-NEXT:    store volatile i64 0, ptr [[PTR:%.*]], align 8
+; ASSUMPTIONS-OFF-NEXT:    [[DOTSINK:%.*]] = phi i64 [ 3, [[FALSE1]] ], [ 2, [[TMP0:%.*]] ]
+; ASSUMPTIONS-OFF-NEXT:    store volatile i64 0, ptr [[PTR]], align 8
 ; ASSUMPTIONS-OFF-NEXT:    store volatile i64 -1, ptr [[PTR]], align 4
 ; ASSUMPTIONS-OFF-NEXT:    store volatile i64 -1, ptr [[PTR]], align 4
 ; ASSUMPTIONS-OFF-NEXT:    store volatile i64 -1, ptr [[PTR]], align 4
 ; ASSUMPTIONS-OFF-NEXT:    store volatile i64 -1, ptr [[PTR]], align 4
 ; ASSUMPTIONS-OFF-NEXT:    store volatile i64 -1, ptr [[PTR]], align 4
 ; ASSUMPTIONS-OFF-NEXT:    store volatile i64 [[DOTSINK]], ptr [[PTR]], align 4
 ; ASSUMPTIONS-OFF-NEXT:    ret void
-; ASSUMPTIONS-OFF:       false2:
-; ASSUMPTIONS-OFF-NEXT:    store volatile i64 1, ptr [[PTR]], align 4
-; ASSUMPTIONS-OFF-NEXT:    br label [[COMMON_RET]]
 ;
 ; ASSUMPTIONS-ON-LABEL: @caller1(
-; ASSUMPTIONS-ON-NEXT:    br i1 [[C:%.*]], label [[COMMON_RET:%.*]], label [[FALSE2:%.*]]
+; ASSUMPTIONS-ON-NEXT:    br i1 [[C:%.*]], label [[COMMON_RET:%.*]], label [[FALSE1:%.*]]
+; ASSUMPTIONS-ON:       false1:
+; ASSUMPTIONS-ON-NEXT:    store volatile i64 1, ptr [[PTR:%.*]], align 4
+; ASSUMPTIONS-ON-NEXT:    br label [[COMMON_RET]]
 ; ASSUMPTIONS-ON:       common.ret:
-; ASSUMPTIONS-ON-NEXT:    [[DOTSINK:%.*]] = phi i64 [ 3, [[FALSE2]] ], [ 2, [[TMP0:%.*]] ]
-; ASSUMPTIONS-ON-NEXT:    call void @llvm.assume(i1 true) [ "align"(ptr [[PTR:%.*]], i64 8) ]
+; ASSUMPTIONS-ON-NEXT:    [[DOTSINK:%.*]] = phi i64 [ 3, [[FALSE1]] ], [ 2, [[TMP0:%.*]] ]
+; ASSUMPTIONS-ON-NEXT:    call void @llvm.assume(i1 true) [ "align"(ptr [[PTR]], i64 8) ]
 ; ASSUMPTIONS-ON-NEXT:    store volatile i64 0, ptr [[PTR]], align 8
 ; ASSUMPTIONS-ON-NEXT:    store volatile i64 -1, ptr [[PTR]], align 8
 ; ASSUMPTIONS-ON-NEXT:    store volatile i64 -1, ptr [[PTR]], align 8
 ; ASSUMPTIONS-ON-NEXT:    store volatile i64 -1, ptr [[PTR]], align 8
 ; ASSUMPTIONS-ON-NEXT:    store volatile i64 -1, ptr [[PTR]], align 8
 ; ASSUMPTIONS-ON-NEXT:    store volatile i64 -1, ptr [[PTR]], align 8
 ; ASSUMPTIONS-ON-NEXT:    store volatile i64 [[DOTSINK]], ptr [[PTR]], align 8
 ; ASSUMPTIONS-ON-NEXT:    ret void
-; ASSUMPTIONS-ON:       false2:
-; ASSUMPTIONS-ON-NEXT:    store volatile i64 1, ptr [[PTR]], align 4
-; ASSUMPTIONS-ON-NEXT:    br label [[COMMON_RET]]
 ;
   br i1 %c, label %true1, label %false1
 
 true1:
   %c2 = call i1 @callee1(i1 %c, ptr %ptr)
   store volatile i64 -1, ptr %ptr
   store volatile i64 -1, ptr %ptr
   store volatile i64 -1, ptr %ptr