[CallGraph][FIX] Ensure generic intrinsics are represented in the CG

[jdoerfert]

Summary

Intrinsics have historically been excluded from the call graph with an
exception of 3 special ones added at some point. This meant that passes
depending on the call graph needed to handle intrinsics explicitly as
the underlying assumption, namely that intrinsics can't call or modify
things, doesn't hold. We are slowly moving away from special handling of
intrinsics, or at least towards explicitly checking what intrinsics we
want to handle differently.

This patch:

• Includes most intrinsics in the call graph. "Assume-like" intrinsics are excluded. What assume-like means is a question for a later time.
• Fixes the annotations on patchpoint intrinsics as they can (and will) callback into the module.
• Removes the special handling of intrinsics in the GlobalsAA pass.
• Removes the IntrinsicInst::isLeaf method.

Properly
Fixes: https://github.com/llvm/llvm-project/issues/52706

See also:
https://discourse.llvm.org/t/intrinsics-are-not-special-stop-pretending-i-mean-it/67545

[arsenm]

Should include the GVN testcase

llvm/lib/Analysis/CallGraph.cpp

102

This is only tangentially related but this misses call targets from constantexpr casted functions, and aliases. I was debugging some issue related to this not long ago

[jdoerfert]

In D141190#4033426, @arsenm wrote:

Should include the GVN testcase

Good point, I'll add https://godbolt.org/z/qn3YhWEKo

llvm/lib/Analysis/CallGraph.cpp

102

Please fix that independently. Let's keep this about intrinsics.

[jdoerfert]

Add test case, fix oversight.

[arsenm]

The previous handling seemed like a weak attempt to define nocallback

[nikic]

Can you please a) land the changes to patchpoint attributes and b) the conversion from isLeaf to !NoCallback separately, and then rebase this? Those two changes are obviously correct and desirable, but I'm not sure about the rest of the intrinsic intrinsic modelling here. It doesn't really seem the job of the call graph to model nocallback intrinsics?

[nikic]

llvm/lib/Analysis/CallGraph.cpp

37

So this means that for example llvm.umax is part of the call graph but llvm.objectsize is not -- the intended logic here isn't clear to me.

llvm/test/Transforms/GVN/intrinsics_in_cg.ll

1

Precommit test.

9

intrinsic

[jdoerfert]

In D141190#4037328, @nikic wrote:

Can you please a) land the changes to patchpoint attributes and b) the conversion from isLeaf to !NoCallback separately, and then rebase this? Those two changes are obviously correct and desirable, but I'm not sure about the rest of the intrinsic intrinsic modelling here. It doesn't really seem the job of the call graph to model nocallback intrinsics?

a) Will do, also the test.
b) This is not as simple as there are two isLeaf uses that mean different things. See below.

W/o modeling intrinsics we try to make up for them in other places, e.g., GlobalsAA. That is arguably undesirable. It is also not about modeling nocallback intrinsics per se.
Right now, we add most intrinsics to the CG but only add *incoming* egdges for isLeaf intrinsics and never outgoing edges. Which means our CG is totally broken and mostly useless wrt. intrinsics.
Even the leaf ones are not represented reasonably because they are called but themselves call nothing (according to the CG). That works for some other magic reasons.

With this patch we add incoming and outgoing edges to intrinsic nodes where they make sense. So if the node is called, it has an incoming edge, if it can call back, it has an outgoing edge.
These are two separate changes and both needed to make the CG sane.

TLDR; Intrinsics are not special in any way. We should treat them like function declarations, which this patch does.

llvm/lib/Analysis/CallGraph.cpp

37

It is as clear as it was before. We added both of the above to the call graph, but not dbg stuff. And then we did not add a single edge connecting the intrinsics we added (except 3), which means adding them was a waste to begin with and totally unhelpful. I'll roll back to !isDbgInfo but that makes as little sense to me as the above.

[jdoerfert]

Precommit test, only exclude dbg intrinsics from the CG (as before), precommit the patchpoint attribute change.

[nikic]

In D141190#4037719, @jdoerfert wrote:

Right now, we add most intrinsics to the CG but only add *incoming* egdges for isLeaf intrinsics and never outgoing edges. Which means our CG is totally broken and mostly useless wrt. intrinsics.
Even the leaf ones are not represented reasonably because they are called but themselves call nothing (according to the CG). That works for some other magic reasons.

I don't think this is quite right: For !isLeaf intrinsics an edge to CallsExternalNode is added, not to the intrinsic. This seems consistent with the current model where intrinsics themselves are not part of the call graph, so only the call that may be done by the intrinsic is represented in the CG.

With this patch we add incoming and outgoing edges to intrinsic nodes where they make sense. So if the node is called, it has an incoming edge, if it can call back, it has an outgoing edge.
These are two separate changes and both needed to make the CG sane.

TLDR; Intrinsics are not special in any way. We should treat them like function declarations, which this patch does.

I don't really agree that intrinsics aren't special. Intrinsics are basically an extensible mechanism for custom instructions. Some intrinsics may be call like (those that are !nocallback), some intrinsics may be load/store like (those with memory effects) and some may be more like arithmetic instructions (most intrinsics). With that in mind, I don't think it's unreasonable for a call graph to only model the "call like" intrinsics.

All that being said, as we do happen to encode intrinsics as calls, it's also not unreasonable to model all intrinsics in the call graph either. And it does look like this makes the implementation simpler overall, so that seems fine as long as it doesn't regress optimization. But see the note inline -- I don't think your change to GlobalsAA is right.

llvm/lib/Analysis/GlobalsModRef.cpp

609

This change doesn't look right to me: Now you're going to handle intrinsic calls both via the call graph, and via the crude code directly below. After your changes, shouldn't be be doing a continue on all calls instead? Why is this not causing any test failures -- are we missing coverage? Or am I completely misunderstanding something here?

[jdoerfert]

In D141190#4039570, @nikic wrote:

In D141190#4037719, @jdoerfert wrote:

Right now, we add most intrinsics to the CG but only add *incoming* egdges for isLeaf intrinsics and never outgoing edges. Which means our CG is totally broken and mostly useless wrt. intrinsics.
Even the leaf ones are not represented reasonably because they are called but themselves call nothing (according to the CG). That works for some other magic reasons.

I don't think this is quite right: For !isLeaf intrinsics an edge to CallsExternalNode is added, not to the intrinsic. This seems consistent with the current model where intrinsics themselves are not part of the call graph, so only the call that may be done by the intrinsic is represented in the CG.

You are right, we don't call the intrinsic but the external node. That is even worse. Why do we have the intrinsics in the CG in the first place (among other things) if we don't connect them at all?

With this patch we add incoming and outgoing edges to intrinsic nodes where they make sense. So if the node is called, it has an incoming edge, if it can call back, it has an outgoing edge.
These are two separate changes and both needed to make the CG sane.

TLDR; Intrinsics are not special in any way. We should treat them like function declarations, which this patch does.

I don't really agree that intrinsics aren't special. Intrinsics are basically an extensible mechanism for custom instructions. Some intrinsics may be call like (those that are !nocallback), some intrinsics may be load/store like (those with memory effects) and some may be more like arithmetic instructions (most intrinsics). With that in mind, I don't think it's unreasonable for a call graph to only model the "call like" intrinsics.

FWIW, an intrinsic is (in our encoding) a call. In our model it will execute some code.
Given a call to an intrinsic llvm.abc [attrs] and a function declaration abc [attrs], all we know is [attrs].
The details, e.g., "call like", is part of [attrs] in both cases. If we stop distinguishing them above we will remove bugs and special handling code.

All that being said, as we do happen to encode intrinsics as calls, it's also not unreasonable to model all intrinsics in the call graph either. And it does look like this makes the implementation simpler overall, so that seems fine as long as it doesn't regress optimization. But see the note inline -- I don't think your change to GlobalsAA is right.

Will be addressed before I commit this.

---

I'm assuming with the fix to GlobalsAA you are OK with this now. I'll commit at some point today.

llvm/lib/Analysis/GlobalsModRef.cpp

609

Agreed, we should do continue. I'll fix that pre-commit.

For intrinsics and declarations the change didn't matter since the code below should just read the attributes again. That's all we knew about them to begin with.
For definitions we could know more then the attributes, but that's somewhat unlikely at this stage since GlobalsAA doesn't really do much deduction, just summarizes the body. Assuming the attributes already summarize the body properly, there would not be a change in the MRI computed for the function.
Still worth skipping them.

[aeubanks]

we should really replace all the remaining (only 4 passes AFAICT) uses of this old CallGraph in favor of LazyCallGraph

[jdoerfert]

In D141190#4040628, @aeubanks wrote:

we should really replace all the remaining (only 4 passes AFAICT) uses of this old CallGraph in favor of LazyCallGraph

That would be great. We'll have a proper GVN test for the problem this is solving by then :)

[jdoerfert]

Closed by https://reviews.llvm.org/rG10410534696e1922cbed56b229d123b2db4acd8e, missed a 0 when copying the revision ID...

llvm/include/llvm/Analysis/CallGraph.h

public:
   void stealCalledFunctionsFrom(CallGraphNode *N) {
     assert(CalledFunctions.empty() &&
            "Cannot steal callsite information if I already have some");
     std::swap(CalledFunctions, N->CalledFunctions);
   }
 
   /// Adds a function to the list of functions called by this one.
   void addCalledFunction(CallBase *Call, CallGraphNode *M) {
-    assert(!Call || !Call->getCalledFunction() ||
-           !Call->getCalledFunction()->isIntrinsic() ||
-           !Intrinsic::isLeaf(Call->getCalledFunction()->getIntrinsicID()));
     CalledFunctions.emplace_back(Call ? std::optional<WeakTrackingVH>(Call)
                                       : std::optional<WeakTrackingVH>(),
                                  M);
     M->AddRef();
   }
 
   void removeCallEdge(iterator I) {
     I->second->DropRef();

llvm/include/llvm/IR/Intrinsics.h

#undef GET_INTRINSIC_ENUM_VALUES
 
   /// Return the function type for an intrinsic.
   FunctionType *getType(LLVMContext &Context, ID id,
                         ArrayRef<Type *> Tys = std::nullopt);
 
   /// Returns true if the intrinsic can be overloaded.
   bool isOverloaded(ID id);
 
-  /// Returns true if the intrinsic is a leaf, i.e. it does not make any calls
-  /// itself.  Most intrinsics are leafs, the exceptions being the patchpoint
-  /// and statepoint intrinsics. These call (or invoke) their "target" argument.
-  bool isLeaf(ID id);
-
   /// Return the attributes for an intrinsic.
   AttributeList getAttributes(LLVMContext &C, ID id);
 
   /// Create or insert an LLVM Function declaration for an intrinsic, and return
   /// it.
   ///
   /// The Tys parameter is for intrinsics with overloaded types (e.g., those
   /// using iAny, fAny, vAny, or iPTRAny).  For a declaration of an overloaded

llvm/lib/Analysis/CallGraph.cpp

 //===- CallGraph.cpp - Build a Module's call graph ------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Analysis/CallGraph.h"
 #include "llvm/ADT/SCCIterator.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/Config/llvm-config.h"
 #include "llvm/IR/AbstractCallSite.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/IntrinsicInst.h"
-#include "llvm/IR/Intrinsics.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/PassManager.h"
 #include "llvm/InitializePasses.h"
 #include "llvm/Pass.h"
 #include "llvm/Support/Compiler.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/raw_ostream.h"
 #include <cassert>
 
 using namespace llvm;
 
 //===----------------------------------------------------------------------===//
 // Implementations of the CallGraph class methods.
 //
 
 CallGraph::CallGraph(Module &M)
     : M(M), ExternalCallingNode(getOrInsertFunction(nullptr)),
       CallsExternalNode(std::make_unique<CallGraphNode>(this, nullptr)) {
   // Add every interesting function to the call graph.
   for (Function &F : M)
     if (!isDbgInfoIntrinsic(F.getIntrinsicID()))
       addToCallGraph(&F);
 }
 
 CallGraph::CallGraph(CallGraph &&Arg)
     : M(Arg.M), FunctionMap(std::move(Arg.FunctionMap)),
       ExternalCallingNode(Arg.ExternalCallingNode),
       CallsExternalNode(std::move(Arg.CallsExternalNode)) {
   Arg.FunctionMap.clear();
void CallGraph::addToCallGraph(Function *F) {
   populateCallGraphNode(Node);
 }
 
 void CallGraph::populateCallGraphNode(CallGraphNode *Node) {
   Function *F = Node->getFunction();
 
   // If this function is not defined in this translation unit, it could call
   // anything.
-  if (F->isDeclaration() && !F->isIntrinsic())
+  if (F->isDeclaration() && !F->hasFnAttribute(Attribute::NoCallback))
     Node->addCalledFunction(nullptr, CallsExternalNode.get());
 
   // Look for calls by this function.
   for (BasicBlock &BB : *F)
     for (Instruction &I : BB) {
       if (auto *Call = dyn_cast<CallBase>(&I)) {
         const Function *Callee = Call->getCalledFunction();
-        if (!Callee || !Intrinsic::isLeaf(Callee->getIntrinsicID()))
-          // Indirect calls of intrinsics are not allowed so no need to check.
-          // We can be more precise here by using TargetArg returned by
-          // Intrinsic::isLeaf.
+        if (!Callee)
           Node->addCalledFunction(Call, CallsExternalNode.get());
-        else if (!Callee->isIntrinsic())
+        else if (!isDbgInfoIntrinsic(Callee->getIntrinsicID()))
           Node->addCalledFunction(Call, getOrInsertFunction(Callee));
 
         // Add reference to callback functions.
         forEachCallbackFunction(*Call, [=](Function *CB) {
           Node->addCalledFunction(nullptr, getOrInsertFunction(CB));
         });
       }
     }

llvm/lib/Analysis/CallGraphSCCPass.cpp

for (CallGraphNode::iterator I = CGN->begin(); I != CGNEnd;) {
 
       // If this call site is null, then the function pass deleted the call
       // entirely and the WeakTrackingVH nulled it out.
       auto *Call = dyn_cast_or_null<CallBase>(*I->first);
       if (!Call ||
           // If we've already seen this call site, then the FunctionPass RAUW'd
           // one call with another, which resulted in two "uses" in the edge
           // list of the same call.
-          Calls.count(Call) ||
-
-          // If the call edge is not from a call or invoke, or it is a
-          // intrinsic call, then the function pass RAUW'd a call with
-          // another value. This can happen when constant folding happens
-          // of well known functions etc.
-          (Call->getCalledFunction() &&
-           Call->getCalledFunction()->isIntrinsic() &&
-           Intrinsic::isLeaf(Call->getCalledFunction()->getIntrinsicID()))) {
+          Calls.count(Call)) {
         assert(!CheckingMode &&
                "CallGraphSCCPass did not update the CallGraph correctly!");
 
         // If this was an indirect call site, count it.
         if (!I->second->getFunction())
           ++NumIndirectRemoved;
         else
           ++NumDirectRemoved;

llvm/lib/Analysis/GlobalsModRef.cpp

 #include "llvm/ADT/SmallPtrSet.h"
 #include "llvm/ADT/Statistic.h"
 #include "llvm/Analysis/CallGraph.h"
 #include "llvm/Analysis/MemoryBuiltins.h"
 #include "llvm/Analysis/TargetLibraryInfo.h"
 #include "llvm/Analysis/ValueTracking.h"
 #include "llvm/IR/InstIterator.h"
 #include "llvm/IR/Instructions.h"
-#include "llvm/IR/IntrinsicInst.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/PassManager.h"
 #include "llvm/InitializePasses.h"
 #include "llvm/Pass.h"
 #include "llvm/Support/CommandLine.h"
 
 using namespace llvm;
 
for (auto *Node : SCC) {
       // above.
       if (Node->getFunction()->hasOptNone())
         continue;
 
       for (Instruction &I : instructions(Node->getFunction())) {
         if (isModAndRefSet(FI.getModRefInfo()))
           break; // The mod/ref lattice saturates here.
 
-        // We handle calls specially because the graph-relevant aspects are
-        // handled above.
-        if (auto *Call = dyn_cast<CallBase>(&I)) {
-          if (Function *Callee = Call->getCalledFunction()) {
-            // The callgraph doesn't include intrinsic calls.
-            if (Callee->isIntrinsic()) {
-              if (isa<DbgInfoIntrinsic>(Call))
-                // Don't let dbg intrinsics affect alias info.
-                continue;
-
-              MemoryEffects Behaviour = AAResultBase::getMemoryEffects(Callee);
-              FI.addModRefInfo(Behaviour.getModRef());
-            }
-          }
-          continue;
-        }
-
         // All non-call instructions we use the primary predicates for whether
         // they read or write memory.
         if (I.mayReadFromMemory())
           FI.addModRefInfo(ModRefInfo::Ref);
         if (I.mayWriteToMemory())
           FI.addModRefInfo(ModRefInfo::Mod);
       }
     }

llvm/lib/IR/Function.cpp

 }
 
 bool Intrinsic::isOverloaded(ID id) {
 #define GET_INTRINSIC_OVERLOAD_TABLE
 #include "llvm/IR/IntrinsicImpl.inc"
 #undef GET_INTRINSIC_OVERLOAD_TABLE
 }
 
-bool Intrinsic::isLeaf(ID id) {
-  switch (id) {
-  default:
-    return true;
-
-  case Intrinsic::experimental_gc_statepoint:
-  case Intrinsic::experimental_patchpoint_void:
-  case Intrinsic::experimental_patchpoint_i64:
-    return false;
-  }
-}
-
 /// This defines the "Intrinsic::getAttributes(ID id)" method.
 #define GET_INTRINSIC_ATTRIBUTES
 #include "llvm/IR/IntrinsicImpl.inc"
 #undef GET_INTRINSIC_ATTRIBUTES
 
 Function *Intrinsic::getDeclaration(Module *M, ID id, ArrayRef<Type*> Tys) {
   // There can never be multiple globals with the same name of different types,
   // because intrinsics must be a specific type.

llvm/test/Analysis/CallGraph/ignore-assumelike-calls.ll

 ; RUN: opt < %s -passes=print-callgraph -disable-output 2>&1 | FileCheck %s
 ; CHECK: Call graph node <<null function>><<{{.*}}>>  #uses=0
 ; CHECK-NEXT:   CS<None> calls function 'other_intrinsic_use'
 ; CHECK-NEXT:   CS<None> calls function 'other_cast_intrinsic_use'
 ; CHECK-NEXT:   CS<None> calls function 'llvm.lifetime.start.p0'
 ; CHECK-NEXT:   CS<None> calls function 'llvm.memset.p0.i64'
 ; CHECK-NEXT:   CS<None> calls function 'llvm.memset.p1.i64'
 ; CHECK-EMPTY:
 ; CHECK-NEXT:   Call graph node for function: 'addrspacecast_only'<<{{.*}}>>  #uses=0
 ; CHECK-EMPTY:
 ; CHECK-NEXT:   Call graph node for function: 'bitcast_only'<<{{.*}}>>  #uses=0
 ; CHECK-EMPTY:
-; CHECK-NEXT:   Call graph node for function: 'llvm.lifetime.start.p0'<<{{.*}}>>  #uses=1
+; CHECK-NEXT:   Call graph node for function: 'llvm.lifetime.start.p0'<<{{.*}}>>  #uses=3
 ; CHECK-EMPTY:
-; CHECK-NEXT:   Call graph node for function: 'llvm.memset.p0.i64'<<{{.*}}>>  #uses=1
+; CHECK-NEXT:   Call graph node for function: 'llvm.memset.p0.i64'<<{{.*}}>>  #uses=2
 ; CHECK-EMPTY:
-; CHECK-NEXT:   Call graph node for function: 'llvm.memset.p1.i64'<<{{.*}}>>  #uses=1
+; CHECK-NEXT:   Call graph node for function: 'llvm.memset.p1.i64'<<{{.*}}>>  #uses=2
 ; CHECK-EMPTY:
 ; CHECK-NEXT:   Call graph node for function: 'other_cast_intrinsic_use'<<{{.*}}>>  #uses=1
+; CHECK-NEXT:   CS<{{.*}}> calls function 'llvm.memset.p1.i64'
 ; CHECK-EMPTY:
 ; CHECK-NEXT:   Call graph node for function: 'other_intrinsic_use'<<{{.*}}>>  #uses=1
+; CHECK-NEXT:   CS<{{.*}}> calls function 'llvm.memset.p0.i64'
 ; CHECK-EMPTY:
 ; CHECK-NEXT:   Call graph node for function: 'used_by_lifetime'<<{{.*}}>>  #uses=0
+; CHECK-NEXT:   CS<{{.*}}> calls function 'llvm.lifetime.start.p0'
 ; CHECK-EMPTY:
 ; CHECK-NEXT:   Call graph node for function: 'used_by_lifetime_cast'<<{{.*}}>>  #uses=0
+; CHECK-NEXT:   CS<{{.*}}> calls function 'llvm.lifetime.start.p0'
 ; CHECK-EMPTY:
 
 define internal void @used_by_lifetime() {
 entry:
   call void @llvm.lifetime.start.p0(i64 4, ptr @used_by_lifetime)
   ret void
 }
 

llvm/test/Analysis/CallGraph/non-leaf-intrinsics.ll

entry:
   ret void
 }
 
 
 ; CHECK: Call graph node <<null function>>
 ; CHECK:  CS<None> calls function 'f'
 
 ; CHECK: Call graph node for function: 'calls_patchpoint'
-; CHECK-NEXT:  CS<[[addr_1:[^>]+]]> calls external node
+; CS<{{.*}}> calls function 'llvm.experimental.patchpoint.void'
 
 ; CHECK: Call graph node for function: 'calls_statepoint'
-; CHECK-NEXT:  CS<[[addr_0:[^>]+]]> calls external node
+; CS<{{.*}}> calls function 'llvm.experimental.gc.statepoint.p0'
+
+; CHECK: Call graph node for function: 'llvm.experimental.gc.statepoint.p0'<<{{.*}}>>  #uses=2
+; CHECK-NEXT:  CS<[[addr_1:[^>]+]]> calls external node
+
+; CHECK: Call graph node for function: 'llvm.experimental.patchpoint.void'<<{{.*}}>>  #uses=2
+; CHECK-NEXT:  CS<[[addr_1:[^>]+]]> calls external node

llvm/test/Transforms/GVN/intrinsics_in_cg.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --function-signature
 ; RUN: opt < %s -passes='require<globals-aa>,gvn' -S | FileCheck %s
 
 ; Ensure we do not hoist the load over the call.
-; FIXME: Currently broken until D141190 or similar lands.
 
 @G1 = internal global i32 1
 @G2 = internal global i32 1
 
 define i32 @direct_intrinsic(i1 %c) {
 ; CHECK-LABEL: define {{[^@]+}}@direct_intrinsic
 ; CHECK-SAME: (i1 [[C:%.*]]) {
 ; CHECK-NEXT:    br i1 [[C]], label [[INIT:%.*]], label [[CHECK:%.*]]
 ; CHECK:       init:
 ; CHECK-NEXT:    store i32 0, ptr @G1, align 4
 ; CHECK-NEXT:    br label [[CHECK]]
 ; CHECK:       check:
 ; CHECK-NEXT:    call void @llvm.unknown()
 ; CHECK-NEXT:    [[V:%.*]] = load i32, ptr @G1, align 4
 ; CHECK-NEXT:    ret i32 [[V]]
 ;
   br i1 %c, label %init, label %check
 init:
   store i32 0, ptr @G1
   br label %check
 check:
   call void @llvm.unknown()
   %v = load i32, ptr @G1
   ret i32 %v
 }
 
 define i32 @indirect_intrinsic(i1 %c) {
 ; CHECK-LABEL: define {{[^@]+}}@indirect_intrinsic
 ; CHECK-SAME: (i1 [[C:%.*]]) {
-; CHECK-NEXT:    br i1 [[C]], label [[INIT:%.*]], label [[DOTCHECK_CRIT_EDGE:%.*]]
-; CHECK:       .check_crit_edge:
-; CHECK-NEXT:    [[V_PRE:%.*]] = load i32, ptr @G2, align 4
-; CHECK-NEXT:    br label [[CHECK:%.*]]
+; CHECK-NEXT:    br i1 [[C]], label [[INIT:%.*]], label [[CHECK:%.*]]
 ; CHECK:       init:
 ; CHECK-NEXT:    store i32 0, ptr @G2, align 4
 ; CHECK-NEXT:    br label [[CHECK]]
 ; CHECK:       check:
-; CHECK-NEXT:    [[V:%.*]] = phi i32 [ [[V_PRE]], [[DOTCHECK_CRIT_EDGE]] ], [ 0, [[INIT]] ]
 ; CHECK-NEXT:    call void @intrinsic_caller()
+; CHECK-NEXT:    [[V:%.*]] = load i32, ptr @G2, align 4
 ; CHECK-NEXT:    ret i32 [[V]]
 ;
   br i1 %c, label %init, label %check
 init:
   store i32 0, ptr @G2
   br label %check
 check:
   call void @intrinsic_caller()