Index: docs/LangRef.rst =================================================================== --- docs/LangRef.rst +++ docs/LangRef.rst @@ -15118,6 +15118,143 @@ ``@llvm.experimental.guard`` cannot be invoked. +'``llvm.experimental.widenable.condition``' Intrinsic +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Syntax: +""""""" + +:: + + declare i1 @llvm.experimental.widenable.condition() + +Overview: +""""""""" + +This intrinsic represents a "widenable condition" which is +boolean expressions with the following property: whether this +expression is `true` or `false`, the program is correct and +well-defined. + +Together with :ref:`deoptimization operand bundles `, +``@llvm.experimental.widenable.condition`` allows frontends to +express guards or checks on optimistic assumptions made during +compilation and represent them as branch instructions on special +conditions. + +It is somewhat close to `undef` in definition, but is deliberately +used to perform guard widening and similar transforms and does not +have such properties of `undef` as ability to change value during the +live range. + +Arguments: +"""""""""" + +None. + +Semantics: +"""""""""" + +The intrinsic ``@llvm.experimental.widenable.condition()`` +always non-deterministically returns `true` or `false`. It is +not like `undef` in terms that the result of every call of this +intrinsic is well-defined and cannot change after it is computed. + +When used in a branch condition, it allows us to choose between +two alternative correct solutions for the same problem, like +in example below: + +.. code-block:: text + + %cond = call i1 @llvm.experimental.widenable.condition() + br i1 %cond, label %solution_1, label %solution_2 + + label %fast_path: + ; Apply memory-consuming but fast solution for a task. + + label %slow_path: + ; Cheap in memory but slow solution. + +Whether the result of intrinsic's call is `true` or `false`, +it should be correct to pick either solution. We can switch +between them by replacing the result of +``@llvm.experimental.widenable.condition`` with different +`i1` expressions. + +This is how it can be used to represent guards as widenable branches: + +.. code-block:: text + + block: + ; Unguarded instructions + call void @llvm.experimental.guard(i1 %cond, ) ["deopt"()] + ; Guarded instructions + +Can be expressed in an alternative equivalent form of explicit branch using +``@llvm.experimental.widenable.condition``: + +.. code-block:: text + + block: + ; Unguarded instructions + %widenable_condition = call i1 @llvm.experimental.widenable.condition() + %guard_condition = and i1 %cond, %widenable_condition + br i1 %guard_condition, label %guarded, label %deopt + + guarded: + ; Guarded instructions + + deopt: + call type @llvm.experimental.deoptimize() [ "deopt"() ] + +So the block `guarded` is only reachable when `%cond` is `true`, +and it should be valid to go to the block `deopt` whenever `%cond` +is `true` or `false`. + +``@llvm.experimental.widenable.condition`` will never throw, thus +it cannot be invoked. + +Guard widening: +""""""""""""""" + +When ``@llvm.experimental.widenable.condition()`` is used in +condition of a guard represented as explicit branch, it is +legal to widen the guard's condition with any additional +conditions. + +Guard widening looks like replacement of + +.. code-block:: text + + %widenable_cond = call i1 @llvm.experimental.widenable.condition() + %guard_cond = and i1 %cond, %widenable_cond + br i1 %guard_cond, label %guarded, label %deopt + +with + +.. code-block:: text + + %widenable_cond = call i1 @llvm.experimental.widenable.condition() + %new_cond = and i1 %any_other_cond, %widenable_cond + %new_guard_cond = and i1 %cond, %new_cond + br i1 %new_guard_cond, label %guarded, label %deopt + +for this branch. Here `%any_other_cond` is an arbitrarily chosen +well-defined `i1` value. By making guard widening, we may +impose stricter conditions on `guarded` block and bail to the +deopt when the new condition is not met. + +Lowering: +""""""""" + +It is always correct to replace all uses of result of +call of ``@llvm.experimental.widenable.condition`` +with any well-defined `i1` expression. Default +lowering strategy is replacing it with constant `true`. +Use cases for which it is not profitable performance-wise +may use other lowering strategies. + + '``llvm.load.relative``' Intrinsic ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Index: include/llvm/IR/Intrinsics.td =================================================================== --- include/llvm/IR/Intrinsics.td +++ include/llvm/IR/Intrinsics.td @@ -838,6 +838,10 @@ def int_experimental_guard : Intrinsic<[], [llvm_i1_ty, llvm_vararg_ty], [Throws]>; +// Supports widenable conditions for guards represented as explicit branches. +def int_experimental_widenable_condition : Intrinsic<[llvm_i1_ty], [], + [IntrInaccessibleMemOnly]>; + // NOP: calls/invokes to this intrinsic are removed by codegen def int_donothing : Intrinsic<[], [], [IntrNoMem]>; Index: include/llvm/InitializePasses.h =================================================================== --- include/llvm/InitializePasses.h +++ include/llvm/InitializePasses.h @@ -140,6 +140,7 @@ void initializeExpandMemCmpPassPass(PassRegistry&); void initializeExpandPostRAPass(PassRegistry&); void initializeExpandReductionsPass(PassRegistry&); +void initializeMakeGuardsExplicitLegacyPassPass(PassRegistry&); void initializeExternalAAWrapperPassPass(PassRegistry&); void initializeFEntryInserterPass(PassRegistry&); void initializeFinalizeMachineBundlesPass(PassRegistry&); Index: include/llvm/Transforms/Scalar/MakeGuardsExplicit.h =================================================================== --- include/llvm/Transforms/Scalar/MakeGuardsExplicit.h +++ include/llvm/Transforms/Scalar/MakeGuardsExplicit.h @@ -0,0 +1,47 @@ +//===-- MakeGuardsExplicit.h - Turn guard intrinsics into guard branches --===// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===----------------------------------------------------------------------===// +// +// This pass lowers the @llvm.experimental.guard intrinsic to the new form of +// guard represented as widenable explicit branch to the deopt block. The +// difference between this pass and LowerGuardIntrinsic is that after this pass +// the guard represented as intrinsic: +// +// call void(i1, ...) @llvm.experimental.guard(i1 %old_cond) [ "deopt"() ] +// +// transforms to a guard represented as widenable explicit branch: +// +// %widenable_cond = call i1 @llvm.experimental.widenable.condition() +// br i1 (%old_cond & %widenable_cond), label %guarded, label %deopt +// +// Here: +// - The semantics of @llvm.experimental.widenable.condition allows to replace +// %widenable_cond with the construction (%widenable_cond & %any_other_cond) +// without loss of correctness; +// - %guarded is the lower part of old guard intrinsic's parent block split by +// the intrinsic call; +// - %deopt is a block containing a sole call to @llvm.experimental.deoptimize +// intrinsic. +// +// Therefore, this branch preserves the property of widenability. +// +//===----------------------------------------------------------------------===// +#ifndef LLVM_TRANSFORMS_SCALAR_MAKEGUARDSEXPLICIT_H +#define LLVM_TRANSFORMS_SCALAR_MAKEGUARDSEXPLICIT_H + +#include "llvm/IR/PassManager.h" + +namespace llvm { + +struct MakeGuardsExplicitPass : public PassInfoMixin { + PreservedAnalyses run(Function &F, FunctionAnalysisManager &AM); +}; + +} // namespace llvm + +#endif //LLVM_TRANSFORMS_SCALAR_MAKEGUARDSEXPLICIT_H Index: lib/Passes/PassBuilder.cpp =================================================================== --- lib/Passes/PassBuilder.cpp +++ lib/Passes/PassBuilder.cpp @@ -130,6 +130,7 @@ #include "llvm/Transforms/Scalar/LowerAtomic.h" #include "llvm/Transforms/Scalar/LowerExpectIntrinsic.h" #include "llvm/Transforms/Scalar/LowerGuardIntrinsic.h" +#include "llvm/Transforms/Scalar/MakeGuardsExplicit.h" #include "llvm/Transforms/Scalar/MemCpyOptimizer.h" #include "llvm/Transforms/Scalar/MergedLoadStoreMotion.h" #include "llvm/Transforms/Scalar/NaryReassociate.h" Index: lib/Passes/PassRegistry.def =================================================================== --- lib/Passes/PassRegistry.def +++ lib/Passes/PassRegistry.def @@ -164,6 +164,7 @@ FUNCTION_PASS("early-cse", EarlyCSEPass(/*UseMemorySSA=*/false)) FUNCTION_PASS("early-cse-memssa", EarlyCSEPass(/*UseMemorySSA=*/true)) FUNCTION_PASS("ee-instrument", EntryExitInstrumenterPass(/*PostInlining=*/false)) +FUNCTION_PASS("make-guards-explicit", MakeGuardsExplicitPass()) FUNCTION_PASS("post-inline-ee-instrument", EntryExitInstrumenterPass(/*PostInlining=*/true)) FUNCTION_PASS("gvn-hoist", GVNHoistPass()) FUNCTION_PASS("instcombine", InstCombinePass()) Index: lib/Transforms/Scalar/CMakeLists.txt =================================================================== --- lib/Transforms/Scalar/CMakeLists.txt +++ lib/Transforms/Scalar/CMakeLists.txt @@ -45,6 +45,7 @@ LowerAtomic.cpp LowerExpectIntrinsic.cpp LowerGuardIntrinsic.cpp + MakeGuardsExplicit.cpp MemCpyOptimizer.cpp MergeICmps.cpp MergedLoadStoreMotion.cpp Index: lib/Transforms/Scalar/MakeGuardsExplicit.cpp =================================================================== --- lib/Transforms/Scalar/MakeGuardsExplicit.cpp +++ lib/Transforms/Scalar/MakeGuardsExplicit.cpp @@ -0,0 +1,120 @@ +//===- MakeGuardsExplicit.cpp - Turn guard intrinsics into guard branches -===// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===----------------------------------------------------------------------===// +// +// This pass lowers the @llvm.experimental.guard intrinsic to the new form of +// guard represented as widenable explicit branch to the deopt block. The +// difference between this pass and LowerGuardIntrinsic is that after this pass +// the guard represented as intrinsic: +// +// call void(i1, ...) @llvm.experimental.guard(i1 %old_cond) [ "deopt"() ] +// +// transforms to a guard represented as widenable explicit branch: +// +// %widenable_cond = call i1 @llvm.experimental.widenable.condition() +// br i1 (%old_cond & %widenable_cond), label %guarded, label %deopt +// +// Here: +// - The semantics of @llvm.experimental.widenable.condition allows to replace +// %widenable_cond with the construction (%widenable_cond & %any_other_cond) +// without loss of correctness; +// - %guarded is the lower part of old guard intrinsic's parent block split by +// the intrinsic call; +// - %deopt is a block containing a sole call to @llvm.experimental.deoptimize +// intrinsic. +// +// Therefore, this branch preserves the property of widenability. +// +//===----------------------------------------------------------------------===// + +#include "llvm/Transforms/Scalar/MakeGuardsExplicit.h" +#include "llvm/Analysis/GuardUtils.h" +#include "llvm/IR/InstIterator.h" +#include "llvm/IR/IntrinsicInst.h" +#include "llvm/IR/Intrinsics.h" +#include "llvm/IR/IRBuilder.h" +#include "llvm/Pass.h" +#include "llvm/Transforms/Scalar.h" +#include "llvm/Transforms/Utils/GuardUtils.h" + +using namespace llvm; + +namespace { +struct MakeGuardsExplicitLegacyPass : public FunctionPass { + static char ID; + MakeGuardsExplicitLegacyPass() : FunctionPass(ID) { + initializeMakeGuardsExplicitLegacyPassPass(*PassRegistry::getPassRegistry()); + } + + bool runOnFunction(Function &F) override; +}; +} + +static void turnToExplicitForm(CallInst *Guard, Function *DeoptIntrinsic) { + // Replace the guard with an explicit branch (just like in GuardWidening). + BasicBlock *BB = Guard->getParent(); + makeGuardControlFlowExplicit(DeoptIntrinsic, Guard); + BranchInst *ExplicitGuard = cast(BB->getTerminator()); + assert(ExplicitGuard->isConditional() && "Must be!"); + + // We want the guard to be expressed as explicit control flow, but still be + // widenable. For that, we add Widenable Condition intrinsic call to the + // guard's condition. + IRBuilder<> B(ExplicitGuard); + auto *WidenableCondition = + B.CreateIntrinsic(Intrinsic::experimental_widenable_condition, + ExplicitGuard, "widenable_cond"); + WidenableCondition->setCallingConv(Guard->getCallingConv()); + auto *NewCond = + B.CreateAnd(ExplicitGuard->getCondition(), WidenableCondition); + NewCond->setName("exiplicit_guard_cond"); + ExplicitGuard->setCondition(NewCond); + Guard->eraseFromParent(); +} + +static bool explicifyGuards(Function &F) { + // Check if we can cheaply rule out the possibility of not having any work to + // do. + auto *GuardDecl = F.getParent()->getFunction( + Intrinsic::getName(Intrinsic::experimental_guard)); + if (!GuardDecl || GuardDecl->use_empty()) + return false; + + SmallVector GuardIntrinsics; + for (auto &I : instructions(F)) + if (isGuard(&I)) + GuardIntrinsics.push_back(cast(&I)); + + if (GuardIntrinsics.empty()) + return false; + + auto *DeoptIntrinsic = Intrinsic::getDeclaration( + F.getParent(), Intrinsic::experimental_deoptimize, {F.getReturnType()}); + DeoptIntrinsic->setCallingConv(GuardDecl->getCallingConv()); + + for (auto *Guard : GuardIntrinsics) + turnToExplicitForm(Guard, DeoptIntrinsic); + + return true; +} + +bool MakeGuardsExplicitLegacyPass::runOnFunction(Function &F) { + return explicifyGuards(F); +} + +char MakeGuardsExplicitLegacyPass::ID = 0; +INITIALIZE_PASS(MakeGuardsExplicitLegacyPass, "make-guards-explicit", + "Lower the guard intrinsic to explicit control flow form", + false, false) + +PreservedAnalyses MakeGuardsExplicitPass::run(Function &F, + FunctionAnalysisManager &) { + if (explicifyGuards(F)) + return PreservedAnalyses::none(); + return PreservedAnalyses::all(); +} Index: lib/Transforms/Scalar/Scalar.cpp =================================================================== --- lib/Transforms/Scalar/Scalar.cpp +++ lib/Transforms/Scalar/Scalar.cpp @@ -51,6 +51,7 @@ initializeNewGVNLegacyPassPass(Registry); initializeEarlyCSELegacyPassPass(Registry); initializeEarlyCSEMemSSALegacyPassPass(Registry); + initializeMakeGuardsExplicitLegacyPassPass(Registry); initializeGVNHoistLegacyPassPass(Registry); initializeGVNSinkLegacyPassPass(Registry); initializeFlattenCFGPassPass(Registry); Index: test/Transforms/LICM/explicit_guards.ll =================================================================== --- test/Transforms/LICM/explicit_guards.ll +++ test/Transforms/LICM/explicit_guards.ll @@ -0,0 +1,82 @@ +; NOTE: Assertions have been autogenerated by utils/update_test_checks.py +; RUN: opt -S -make-guards-explicit -basicaa -licm < %s | FileCheck %s +; RUN: opt -S -aa-pipeline=basic-aa -passes='require,make-guards-explicit,loop(licm)' < %s | FileCheck %s + +; Test interaction between explicit guards and LICM: make sure that we do not +; hoist explicit conditions while we can hoist invariant loads in presence of +; explicit guards. + +declare void @llvm.experimental.guard(i1,...) + +; Make sure that we do not hoist widenable_cond out of loop. +define void @do_not_hoist_widenable_cond(i1 %cond, i32 %N, i32 %M) { +; CHECK-LABEL: @do_not_hoist_widenable_cond( +; CHECK-NEXT: entry: +; CHECK-NEXT: br label [[LOOP:%.*]] +; CHECK: loop: +; CHECK-NEXT: [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[GUARDED:%.*]] ] +; CHECK-NEXT: [[GUARD_COND:%.*]] = icmp slt i32 [[IV]], [[N:%.*]] +; CHECK-NEXT: [[WIDENABLE_COND:%.*]] = call i1 @llvm.experimental.widenable.condition() +; CHECK-NEXT: [[EXIPLICIT_GUARD_COND:%.*]] = and i1 [[GUARD_COND]], [[WIDENABLE_COND]] +; CHECK-NEXT: br i1 [[EXIPLICIT_GUARD_COND]], label [[GUARDED]], label [[DEOPT:%.*]], !prof !0 +; CHECK: deopt: +; CHECK-NEXT: call void (...) @llvm.experimental.deoptimize.isVoid() [ "deopt"() ] +; CHECK-NEXT: ret void +; CHECK: guarded: +; CHECK-NEXT: [[LOOP_COND:%.*]] = icmp slt i32 [[IV]], [[M:%.*]] +; CHECK-NEXT: [[IV_NEXT]] = add i32 [[IV]], 1 +; CHECK-NEXT: br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]] +; CHECK: exit: +; CHECK-NEXT: ret void +; +entry: + br label %loop + +loop: + %iv = phi i32 [ 0, %entry ], [ %iv.next, %loop ] + %guard_cond = icmp slt i32 %iv, %N + call void(i1, ...) @llvm.experimental.guard(i1 %guard_cond) [ "deopt"() ] + %loop_cond = icmp slt i32 %iv, %M + %iv.next = add i32 %iv, 1 + br i1 %loop_cond, label %loop, label %exit + +exit: + ret void +} + +define void @hoist_invariant_load(i1 %cond, i32* %np, i32 %M) { +; CHECK-LABEL: @hoist_invariant_load( +; CHECK-NEXT: entry: +; CHECK-NEXT: [[N:%.*]] = load i32, i32* [[NP:%.*]] +; CHECK-NEXT: br label [[LOOP:%.*]] +; CHECK: loop: +; CHECK-NEXT: [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[GUARDED:%.*]] ] +; CHECK-NEXT: [[GUARD_COND:%.*]] = icmp slt i32 [[IV]], [[N]] +; CHECK-NEXT: [[WIDENABLE_COND:%.*]] = call i1 @llvm.experimental.widenable.condition() +; CHECK-NEXT: [[EXIPLICIT_GUARD_COND:%.*]] = and i1 [[GUARD_COND]], [[WIDENABLE_COND]] +; CHECK-NEXT: br i1 [[EXIPLICIT_GUARD_COND]], label [[GUARDED]], label [[DEOPT:%.*]], !prof !0 +; CHECK: deopt: +; CHECK-NEXT: call void (...) @llvm.experimental.deoptimize.isVoid() [ "deopt"() ] +; CHECK-NEXT: ret void +; CHECK: guarded: +; CHECK-NEXT: [[LOOP_COND:%.*]] = icmp slt i32 [[IV]], [[M:%.*]] +; CHECK-NEXT: [[IV_NEXT]] = add i32 [[IV]], 1 +; CHECK-NEXT: br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]] +; CHECK: exit: +; CHECK-NEXT: ret void +; +entry: + br label %loop + +loop: + %iv = phi i32 [ 0, %entry ], [ %iv.next, %loop ] + %N = load i32, i32* %np + %guard_cond = icmp slt i32 %iv, %N + call void(i1, ...) @llvm.experimental.guard(i1 %guard_cond) [ "deopt"() ] + %loop_cond = icmp slt i32 %iv, %M + %iv.next = add i32 %iv, 1 + br i1 %loop_cond, label %loop, label %exit + +exit: + ret void +} Index: test/Transforms/MakeGuardsExplicit/basic.ll =================================================================== --- test/Transforms/MakeGuardsExplicit/basic.ll +++ test/Transforms/MakeGuardsExplicit/basic.ll @@ -0,0 +1,135 @@ +; NOTE: Assertions have been autogenerated by utils/update_test_checks.py +; RUN: opt -S -make-guards-explicit < %s | FileCheck %s +; RUN: opt -S -passes=make-guards-explicit < %s | FileCheck %s + +declare void @llvm.experimental.guard(i1,...) + +; Check that a sole guard can be turned into explicit guards form. +define void @trivial_guard(i1 %cond) { +; CHECK-LABEL: @trivial_guard( +; CHECK-NEXT: entry: +; CHECK-NEXT: [[WIDENABLE_COND:%.*]] = call i1 @llvm.experimental.widenable.condition() +; CHECK-NEXT: [[EXIPLICIT_GUARD_COND:%.*]] = and i1 [[COND:%.*]], [[WIDENABLE_COND]] +; CHECK-NEXT: br i1 [[EXIPLICIT_GUARD_COND]], label [[GUARDED:%.*]], label [[DEOPT:%.*]], !prof !0 +; CHECK: deopt: +; CHECK-NEXT: call void (...) @llvm.experimental.deoptimize.isVoid() [ "deopt"(i32 123, i64 456) ] +; CHECK-NEXT: ret void +; CHECK: guarded: +; CHECK-NEXT: ret void +; +entry: + call void(i1, ...) @llvm.experimental.guard(i1 %cond) [ "deopt"(i32 123, i64 456) ] + ret void +} + +; Check that a sequence of guards can be turned into explicit guards form. +define void @trivial_guard_sequence(i1 %cond1, i1 %cond2, i1 %cond3) { +; CHECK-LABEL: @trivial_guard_sequence( +; CHECK-NEXT: entry: +; CHECK-NEXT: [[WIDENABLE_COND:%.*]] = call i1 @llvm.experimental.widenable.condition() +; CHECK-NEXT: [[EXIPLICIT_GUARD_COND:%.*]] = and i1 [[COND1:%.*]], [[WIDENABLE_COND]] +; CHECK-NEXT: br i1 [[EXIPLICIT_GUARD_COND]], label [[GUARDED:%.*]], label [[DEOPT:%.*]], !prof !0 +; CHECK: deopt: +; CHECK-NEXT: call void (...) @llvm.experimental.deoptimize.isVoid() [ "deopt"(i32 123, i64 456) ] +; CHECK-NEXT: ret void +; CHECK: guarded: +; CHECK-NEXT: [[WIDENABLE_COND3:%.*]] = call i1 @llvm.experimental.widenable.condition() +; CHECK-NEXT: [[EXIPLICIT_GUARD_COND4:%.*]] = and i1 [[COND2:%.*]], [[WIDENABLE_COND3]] +; CHECK-NEXT: br i1 [[EXIPLICIT_GUARD_COND4]], label [[GUARDED1:%.*]], label [[DEOPT2:%.*]], !prof !0 +; CHECK: deopt2: +; CHECK-NEXT: call void (...) @llvm.experimental.deoptimize.isVoid() [ "deopt"(i32 789, i64 123) ] +; CHECK-NEXT: ret void +; CHECK: guarded1: +; CHECK-NEXT: [[WIDENABLE_COND7:%.*]] = call i1 @llvm.experimental.widenable.condition() +; CHECK-NEXT: [[EXIPLICIT_GUARD_COND8:%.*]] = and i1 [[COND3:%.*]], [[WIDENABLE_COND7]] +; CHECK-NEXT: br i1 [[EXIPLICIT_GUARD_COND8]], label [[GUARDED5:%.*]], label [[DEOPT6:%.*]], !prof !0 +; CHECK: deopt6: +; CHECK-NEXT: call void (...) @llvm.experimental.deoptimize.isVoid() [ "deopt"(i32 456, i64 789) ] +; CHECK-NEXT: ret void +; CHECK: guarded5: +; CHECK-NEXT: ret void +; +entry: + call void(i1, ...) @llvm.experimental.guard(i1 %cond1) [ "deopt"(i32 123, i64 456) ] + call void(i1, ...) @llvm.experimental.guard(i1 %cond2) [ "deopt"(i32 789, i64 123) ] + call void(i1, ...) @llvm.experimental.guard(i1 %cond3) [ "deopt"(i32 456, i64 789) ] + ret void +} + +; Check that all instructions between the guards preserve. +define void @split_block_contents(i1 %cond1, i1 %cond2, i1 %cond3, i32* %p) { +; CHECK-LABEL: @split_block_contents( +; CHECK-NEXT: entry: +; CHECK-NEXT: store i32 0, i32* [[P:%.*]] +; CHECK-NEXT: [[WIDENABLE_COND:%.*]] = call i1 @llvm.experimental.widenable.condition() +; CHECK-NEXT: [[EXIPLICIT_GUARD_COND:%.*]] = and i1 [[COND1:%.*]], [[WIDENABLE_COND]] +; CHECK-NEXT: br i1 [[EXIPLICIT_GUARD_COND]], label [[GUARDED:%.*]], label [[DEOPT:%.*]], !prof !0 +; CHECK: deopt: +; CHECK-NEXT: call void (...) @llvm.experimental.deoptimize.isVoid() [ "deopt"(i32 123, i64 456) ] +; CHECK-NEXT: ret void +; CHECK: guarded: +; CHECK-NEXT: store i32 1, i32* [[P]] +; CHECK-NEXT: [[WIDENABLE_COND3:%.*]] = call i1 @llvm.experimental.widenable.condition() +; CHECK-NEXT: [[EXIPLICIT_GUARD_COND4:%.*]] = and i1 [[COND2:%.*]], [[WIDENABLE_COND3]] +; CHECK-NEXT: br i1 [[EXIPLICIT_GUARD_COND4]], label [[GUARDED1:%.*]], label [[DEOPT2:%.*]], !prof !0 +; CHECK: deopt2: +; CHECK-NEXT: call void (...) @llvm.experimental.deoptimize.isVoid() [ "deopt"(i32 789, i64 123) ] +; CHECK-NEXT: ret void +; CHECK: guarded1: +; CHECK-NEXT: store i32 2, i32* [[P]] +; CHECK-NEXT: [[WIDENABLE_COND7:%.*]] = call i1 @llvm.experimental.widenable.condition() +; CHECK-NEXT: [[EXIPLICIT_GUARD_COND8:%.*]] = and i1 [[COND3:%.*]], [[WIDENABLE_COND7]] +; CHECK-NEXT: br i1 [[EXIPLICIT_GUARD_COND8]], label [[GUARDED5:%.*]], label [[DEOPT6:%.*]], !prof !0 +; CHECK: deopt6: +; CHECK-NEXT: call void (...) @llvm.experimental.deoptimize.isVoid() [ "deopt"(i32 456, i64 789) ] +; CHECK-NEXT: ret void +; CHECK: guarded5: +; CHECK-NEXT: store i32 3, i32* [[P]] +; CHECK-NEXT: ret void +; +entry: + store i32 0, i32* %p + call void(i1, ...) @llvm.experimental.guard(i1 %cond1) [ "deopt"(i32 123, i64 456) ] + store i32 1, i32* %p + call void(i1, ...) @llvm.experimental.guard(i1 %cond2) [ "deopt"(i32 789, i64 123) ] + store i32 2, i32* %p + call void(i1, ...) @llvm.experimental.guard(i1 %cond3) [ "deopt"(i32 456, i64 789) ] + store i32 3, i32* %p + ret void +} + +; Check that the guard can split a loop properly. +define void @split_loop(i1 %cond, i32 %N, i32 %M) { +; CHECK-LABEL: @split_loop( +; CHECK-NEXT: entry: +; CHECK-NEXT: br label [[LOOP:%.*]] +; CHECK: loop: +; CHECK-NEXT: [[IV:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[GUARDED:%.*]] ] +; CHECK-NEXT: [[GUARD_COND:%.*]] = icmp slt i32 [[IV]], [[N:%.*]] +; CHECK-NEXT: [[WIDENABLE_COND:%.*]] = call i1 @llvm.experimental.widenable.condition() +; CHECK-NEXT: [[EXIPLICIT_GUARD_COND:%.*]] = and i1 [[GUARD_COND]], [[WIDENABLE_COND]] +; CHECK-NEXT: br i1 [[EXIPLICIT_GUARD_COND]], label [[GUARDED]], label [[DEOPT:%.*]], !prof !0 +; CHECK: deopt: +; CHECK-NEXT: call void (...) @llvm.experimental.deoptimize.isVoid() [ "deopt"(i32 123, i64 456) ] +; CHECK-NEXT: ret void +; CHECK: guarded: +; CHECK-NEXT: [[LOOP_COND:%.*]] = icmp slt i32 [[IV]], [[M:%.*]] +; CHECK-NEXT: [[IV_NEXT]] = add i32 [[IV]], 1 +; CHECK-NEXT: br i1 [[LOOP_COND]], label [[LOOP]], label [[EXIT:%.*]] +; CHECK: exit: +; CHECK-NEXT: ret void +; +entry: + br label %loop + +loop: + %iv = phi i32 [ 0, %entry ], [ %iv.next, %loop ] + %guard_cond = icmp slt i32 %iv, %N + call void(i1, ...) @llvm.experimental.guard(i1 %guard_cond) [ "deopt"(i32 123, i64 456) ] + %loop_cond = icmp slt i32 %iv, %M + %iv.next = add i32 %iv, 1 + br i1 %loop_cond, label %loop, label %exit + +exit: + ret void +}