This is an archive of the discontinued LLVM Phabricator instance.

Differential D159547

[LV] Relax high loop trip count threshold for deciding to interleave the loop
DraftPublic

Authored by nilanjana_basu on Sep 25 2023, 6:30 PM.
This is a draft revision that has not yet been submitted for review.

Details

Reviewers
None
Summary

A set of microbenchmarks in llvm-test-suite (https://github.com/llvm/llvm-test-suite/pull/26), when tested on a AArch64 platform, demonstrates that loop interleaving is beneficial once the interleaved part of the loop runs at least twice. The performance improves as trip count increases and is best for trip counts that only run the vectorized &/or interleaved loop. For example, if VW is the vectorization width & IC is the interleaving count, loops with trip count TC > VW*IC*2 performs better with interleaving and the performance is best when TC is a multiple of VW*IC.

The current trip count threshold to allow loop interleaving is 128 which seems arbitrarily high & uncorrelated with factors like VW, IC, register pressure.

We have also found example in an application benchmark that was compiled with PGO where a hot loop with trip count less than 24 shows a 40% regression since it didn't get interleaved because the profile-driven trip count was less than 128. Therefore, it seems reasonable to get rid of this threshold and use the trip count for computing interleaving count instead.

Diff Detail

Event Timeline

nilanjana_basu created this revision.Sep 25 2023, 6:30 PM
Herald added a project: Restricted Project. · View Herald TranscriptSep 25 2023, 6:30 PM
Herald added subscribers: artagnon, wlei, wenlei, hiraditya. · View Herald Transcript
Harbormaster completed remote builds in B257598: Diff 557344.Sep 25 2023, 7:17 PM
nilanjana_basu updated this revision to Diff 557384.Sep 26 2023, 6:59 PM

Updated all unit tests that failed because of this change.

Herald added a subscriber: zzheng. · View Herald TranscriptSep 26 2023, 6:59 PM
Harbormaster completed remote builds in B257622: Diff 557384.Sep 26 2023, 7:20 PM
nilanjana_basu edited the summary of this revision. (Show Details)Sep 27 2023, 9:38 AM
nilanjana_basu edited the summary of this revision. (Show Details)Sep 27 2023, 1:39 PM
Herald added a subscriber: kristof.beyls. · View Herald TranscriptSep 27 2023, 1:39 PM
nilanjana_basu updated this revision to Diff 557423.Sep 27 2023, 2:12 PM

Updated commit message

Harbormaster completed remote builds in B257645: Diff 557423.Sep 27 2023, 4:18 PM
nilanjana_basu updated this revision to Diff 557432.Sep 28 2023, 12:23 AM

Removed the option of -tiny-trip-count-interleave-threshold which will no longer be used, and updated interleave_short_tc.ll accordingly

nilanjana_basu updated this revision to Diff 557433.Sep 28 2023, 12:29 AM

Removed tiny-trip-count-interleave-threshold from zero_unroll.ll

Harbormaster completed remote builds in B257653: Diff 557433.Sep 28 2023, 1:00 AM
nilanjana_basu updated this revision to Diff 557449.Sep 28 2023, 11:08 AM

Updated test case comments

Harbormaster completed remote builds in B257663: Diff 557449.Sep 28 2023, 1:12 PM

Revision Contents

PathSize
llvm/
lib/
Transforms/
Vectorize/
28 lines
test/
Transforms/
LoopDistribute/
2 lines
LoopVectorize/
AArch64/
161 lines
SystemZ/
2 lines
X86/
96 lines
15 lines
81 lines
691 lines
1658 lines
18 lines
202 lines
160 lines
41 lines
2 lines
2 lines
PhaseOrdering/
X86/
178 lines

Diff 557449

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 258 Lines • ▼ Show 20 Linesstatic cl::opt<bool> EnableInterleavedMemAccesses(
cl::desc("Enable vectorization on interleaved memory accesses in a loop")); cl::desc("Enable vectorization on interleaved memory accesses in a loop"));
/// An interleave-group may need masking if it resides in a block that needs /// An interleave-group may need masking if it resides in a block that needs
/// predication, or in order to mask away gaps. /// predication, or in order to mask away gaps.
static cl::opt<bool> EnableMaskedInterleavedMemAccesses( static cl::opt<bool> EnableMaskedInterleavedMemAccesses(
"enable-masked-interleaved-mem-accesses", cl::init(false), cl::Hidden, "enable-masked-interleaved-mem-accesses", cl::init(false), cl::Hidden,
cl::desc("Enable vectorization on masked interleaved memory accesses in a loop")); cl::desc("Enable vectorization on masked interleaved memory accesses in a loop"));
static cl::opt<unsigned> TinyTripCountInterleaveThreshold(
"tiny-trip-count-interleave-threshold", cl::init(128), cl::Hidden,
cl::desc("We don't interleave loops with a estimated constant trip count "
"below this number"));
static cl::opt<unsigned> ForceTargetNumScalarRegs( static cl::opt<unsigned> ForceTargetNumScalarRegs(
"force-target-num-scalar-regs", cl::init(0), cl::Hidden, "force-target-num-scalar-regs", cl::init(0), cl::Hidden,
cl::desc("A flag that overrides the target's number of scalar registers.")); cl::desc("A flag that overrides the target's number of scalar registers."));
static cl::opt<unsigned> ForceTargetNumVectorRegs( static cl::opt<unsigned> ForceTargetNumVectorRegs(
"force-target-num-vector-regs", cl::init(0), cl::Hidden, "force-target-num-vector-regs", cl::init(0), cl::Hidden,
cl::desc("A flag that overrides the target's number of vector registers.")); cl::desc("A flag that overrides the target's number of vector registers."));
▲ Show 20 Lines • Show All 5,379 Lines • ▼ Show 20 Lines if (!isScalarEpilogueAllowed())
return 1; return 1;
// We used the distance for the interleave count. // We used the distance for the interleave count.
if (!Legal->isSafeForAnyVectorWidth()) if (!Legal->isSafeForAnyVectorWidth())
return 1; return 1;
auto BestKnownTC = getSmallBestKnownTC(*PSE.getSE(), TheLoop); auto BestKnownTC = getSmallBestKnownTC(*PSE.getSE(), TheLoop);
const bool HasReductions = !Legal->getReductionVars().empty(); const bool HasReductions = !Legal->getReductionVars().empty();
// Do not interleave loops with a relatively small known or estimated trip
// count. But we will interleave when InterleaveSmallLoopScalarReduction is
// enabled, and the code has scalar reductions(HasReductions && VF = 1),
// because with the above conditions interleaving can expose ILP and break
// cross iteration dependences for reductions.
if (BestKnownTC && (*BestKnownTC < TinyTripCountInterleaveThreshold) &&
!(InterleaveSmallLoopScalarReduction && HasReductions && VF.isScalar()))
return 1;
// If we did not calculate the cost for VF (because the user selected the VF) // If we did not calculate the cost for VF (because the user selected the VF)
// then we calculate the cost of VF here. // then we calculate the cost of VF here.
if (LoopCost == 0) { if (LoopCost == 0) {
LoopCost = expectedCost(VF).first; LoopCost = expectedCost(VF).first;
assert(LoopCost.isValid() && "Expected to have chosen a VF with valid cost"); assert(LoopCost.isValid() && "Expected to have chosen a VF with valid cost");
// Loop body is free and there is no need for interleaving. // Loop body is free and there is no need for interleaving.
▲ Show 20 Lines • Show All 57 Lines • ▼ Show 20 Lines if (VF.isScalar()) {
if (ForceTargetMaxScalarInterleaveFactor.getNumOccurrences() > 0) if (ForceTargetMaxScalarInterleaveFactor.getNumOccurrences() > 0)
MaxInterleaveCount = ForceTargetMaxScalarInterleaveFactor; MaxInterleaveCount = ForceTargetMaxScalarInterleaveFactor;
} else { } else {
if (ForceTargetMaxVectorInterleaveFactor.getNumOccurrences() > 0) if (ForceTargetMaxVectorInterleaveFactor.getNumOccurrences() > 0)
MaxInterleaveCount = ForceTargetMaxVectorInterleaveFactor; MaxInterleaveCount = ForceTargetMaxVectorInterleaveFactor;
} }
// If trip count is known or estimated compile time constant, limit the // If trip count is known or estimated compile time constant, limit the
// interleave count to be less than the trip count divided by VF, provided it // interleave count to be less than the trip count divided by VF * 2,
// is at least 1. // provided VF is at least 1, such that the vector loop runs at least twice
// to make interleaving seem profitable. When
// InterleaveSmallLoopScalarReduction is true, we allow interleaving even when
// the vector loop runs once.
// //
// For scalable vectors we can't know if interleaving is beneficial. It may // For scalable vectors we can't know if interleaving is beneficial. It may
// not be beneficial for small loops if none of the lanes in the second vector // not be beneficial for small loops if none of the lanes in the second vector
// iterations is enabled. However, for larger loops, there is likely to be a // iterations is enabled. However, for larger loops, there is likely to be a
// similar benefit as for fixed-width vectors. For now, we choose to leave // similar benefit as for fixed-width vectors. For now, we choose to leave
// the InterleaveCount as if vscale is '1', although if some information about // the InterleaveCount as if vscale is '1', although if some information about
// the vector is known (e.g. min vector size), we can make a better decision. // the vector is known (e.g. min vector size), we can make a better decision.
if (BestKnownTC) { if (BestKnownTC) {
if (InterleaveSmallLoopScalarReduction)
MaxInterleaveCount = MaxInterleaveCount =
std::min(*BestKnownTC / VF.getKnownMinValue(), MaxInterleaveCount); std::min(*BestKnownTC / VF.getKnownMinValue(), MaxInterleaveCount);
else
MaxInterleaveCount = std::min(*BestKnownTC / (VF.getKnownMinValue() * 2),
MaxInterleaveCount);
// Make sure MaxInterleaveCount is greater than 0. // Make sure MaxInterleaveCount is greater than 0.
MaxInterleaveCount = std::max(1u, MaxInterleaveCount); MaxInterleaveCount = std::max(1u, MaxInterleaveCount);
} }
assert(MaxInterleaveCount > 0 && assert(MaxInterleaveCount > 0 &&
"Maximum interleave count must be greater than 0"); "Maximum interleave count must be greater than 0");
// Clamp the calculated IC to be between the 1 and the max interleave count // Clamp the calculated IC to be between the 1 and the max interleave count
▲ Show 20 Lines • Show All 4,568 Lines • Show Last 20 Lines

llvm/test/Transforms/LoopDistribute/basic-with-memchecks.ll

Show First 20 Lines • Show All 73 Lines • ▼ Show 20 Lines
; CHECK: for.body.ph: ; CHECK: for.body.ph:
; CHECK: br label %for.body ; CHECK: br label %for.body
; CHECK: for.body: ; CHECK: for.body:
; CHECK: %mulC = mul i32 %loadD, %loadE ; CHECK: %mulC = mul i32 %loadD, %loadE
; CHECK: for.end: ; CHECK: for.end:
; VECTORIZE: mul <4 x i32> ; VECTORIZE: mul <4 x i32>
; VECTORIZE: mul <4 x i32>
; VECTORIZE-NOT: mul <4 x i32>
for.body: ; preds = %for.body, %entry for.body: ; preds = %for.body, %entry
%ind = phi i64 [ 0, %entry ], [ %add, %for.body ] %ind = phi i64 [ 0, %entry ], [ %add, %for.body ]
%arrayidxA = getelementptr inbounds i32, ptr %a, i64 %ind %arrayidxA = getelementptr inbounds i32, ptr %a, i64 %ind
%loadA = load i32, ptr %arrayidxA, align 4 %loadA = load i32, ptr %arrayidxA, align 4
%arrayidxB = getelementptr inbounds i32, ptr %b, i64 %ind %arrayidxB = getelementptr inbounds i32, ptr %b, i64 %ind
▲ Show 20 Lines • Show All 199 Lines • Show Last 20 Lines

llvm/test/Transforms/LoopVectorize/AArch64/sve-inductions-unusual-types.ll

; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 3
; REQUIRES: asserts ; REQUIRES: asserts
; RUN: opt -passes=loop-vectorize -S < %s -debug -prefer-predicate-over-epilogue=scalar-epilogue 2>%t | FileCheck %s ; RUN: opt -passes=loop-vectorize -S < %s -debug -prefer-predicate-over-epilogue=scalar-epilogue 2>%t | FileCheck %s
; RUN: cat %t | FileCheck %s --check-prefix=DEBUG ; RUN: cat %t | FileCheck %s --check-prefix=DEBUG
target triple = "aarch64-unknown-linux-gnu" target triple = "aarch64-unknown-linux-gnu"
; DEBUG: Found an estimated cost of Invalid for VF vscale x 1 For instruction: %indvars.iv1294 = phi i7 [ %indvars.iv.next1295, %for.body ], [ 0, %entry ] ; DEBUG: Found an estimated cost of Invalid for VF vscale x 1 For instruction: %indvars.iv1294 = phi i7 [ %indvars.iv.next1295, %for.body ], [ 0, %entry ]
; DEBUG: Found an estimated cost of Invalid for VF vscale x 1 For instruction: %addi7 = add i7 %indvars.iv1294, 0 ; DEBUG: Found an estimated cost of Invalid for VF vscale x 1 For instruction: %addi7 = add i7 %indvars.iv1294, 0
; DEBUG: Found an estimated cost of Invalid for VF vscale x 1 For instruction: %indvars.iv.next1295 = add i7 %indvars.iv1294, 1 ; DEBUG: Found an estimated cost of Invalid for VF vscale x 1 For instruction: %indvars.iv.next1295 = add i7 %indvars.iv1294, 1
define void @induction_i7(ptr %dst) #0 { define void @induction_i7(ptr %dst) #0 {
; CHECK-LABEL: @induction_i7( ; CHECK-LABEL: define void @induction_i7(
; CHECK-SAME: ptr [[DST:%.*]]) #[[ATTR0:[0-9]+]] {
; CHECK-NEXT: entry:
; CHECK-NEXT: [[TMP0:%.*]] = call i64 @llvm.vscale.i64()
; CHECK-NEXT: [[TMP1:%.*]] = mul i64 [[TMP0]], 4
; CHECK-NEXT: [[MIN_ITERS_CHECK:%.*]] = icmp ult i64 64, [[TMP1]]
; CHECK-NEXT: br i1 [[MIN_ITERS_CHECK]], label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]]
; CHECK: vector.ph: ; CHECK: vector.ph:
; CHECK: [[TMP4:%.*]] = call <vscale x 2 x i8> @llvm.experimental.stepvector.nxv2i8() ; CHECK-NEXT: [[TMP2:%.*]] = call i64 @llvm.vscale.i64()
; CHECK: [[TMP5:%.*]] = trunc <vscale x 2 x i8> [[TMP4]] to <vscale x 2 x i7> ; CHECK-NEXT: [[TMP3:%.*]] = mul i64 [[TMP2]], 4
; CHECK-NEXT: [[N_MOD_VF:%.*]] = urem i64 64, [[TMP3]]
; CHECK-NEXT: [[N_VEC:%.*]] = sub i64 64, [[N_MOD_VF]]
; CHECK-NEXT: [[IND_END:%.*]] = trunc i64 [[N_VEC]] to i7
; CHECK-NEXT: [[TMP4:%.*]] = call <vscale x 2 x i8> @llvm.experimental.stepvector.nxv2i8()
; CHECK-NEXT: [[TMP5:%.*]] = trunc <vscale x 2 x i8> [[TMP4]] to <vscale x 2 x i7>
; CHECK-NEXT: [[TMP6:%.*]] = add <vscale x 2 x i7> [[TMP5]], zeroinitializer ; CHECK-NEXT: [[TMP6:%.*]] = add <vscale x 2 x i7> [[TMP5]], zeroinitializer
; CHECK-NEXT: [[TMP7:%.*]] = mul <vscale x 2 x i7> [[TMP6]], shufflevector (<vscale x 2 x i7> insertelement (<vscale x 2 x i7> poison, i7 1, i64 0), <vscale x 2 x i7> poison, <vscale x 2 x i32> zeroinitializer) ; CHECK-NEXT: [[TMP7:%.*]] = mul <vscale x 2 x i7> [[TMP6]], shufflevector (<vscale x 2 x i7> insertelement (<vscale x 2 x i7> poison, i7 1, i64 0), <vscale x 2 x i7> poison, <vscale x 2 x i32> zeroinitializer)
; CHECK-NEXT: [[INDUCTION:%.*]] = add <vscale x 2 x i7> zeroinitializer, [[TMP7]] ; CHECK-NEXT: [[INDUCTION:%.*]] = add <vscale x 2 x i7> zeroinitializer, [[TMP7]]
; CHECK-NEXT: [[TMP8:%.*]] = call i7 @llvm.vscale.i7()
; CHECK-NEXT: [[TMP9:%.*]] = mul i7 [[TMP8]], 2
; CHECK-NEXT: [[TMP10:%.*]] = mul i7 1, [[TMP9]]
; CHECK-NEXT: [[DOTSPLATINSERT:%.*]] = insertelement <vscale x 2 x i7> poison, i7 [[TMP10]], i64 0
; CHECK-NEXT: [[DOTSPLAT:%.*]] = shufflevector <vscale x 2 x i7> [[DOTSPLATINSERT]], <vscale x 2 x i7> poison, <vscale x 2 x i32> zeroinitializer
; CHECK-NEXT: br label [[VECTOR_BODY:%.*]]
; CHECK: vector.body: ; CHECK: vector.body:
; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, %vector.ph ], [ [[INDEX_NEXT:%.*]], %vector.body ] ; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[VEC_IND:%.*]] = phi <vscale x 2 x i7> [ [[INDUCTION]], %vector.ph ], [ [[VEC_IND_NEXT:%.*]], %vector.body ] ; CHECK-NEXT: [[VEC_IND:%.*]] = phi <vscale x 2 x i7> [ [[INDUCTION]], [[VECTOR_PH]] ], [ [[VEC_IND_NEXT:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[TMP10:%.*]] = add i64 [[INDEX]], 0 ; CHECK-NEXT: [[STEP_ADD:%.*]] = add <vscale x 2 x i7> [[VEC_IND]], [[DOTSPLAT]]
; CHECK-NEXT: [[TMP11:%.*]] = add <vscale x 2 x i7> [[VEC_IND]], zeroinitializer ; CHECK-NEXT: [[TMP11:%.*]] = add i64 [[INDEX]], 0
; CHECK-NEXT: [[TMP12:%.*]] = getelementptr inbounds i64, ptr [[DST:%.*]], i64 [[TMP10]] ; CHECK-NEXT: [[TMP12:%.*]] = call i64 @llvm.vscale.i64()
; CHECK-NEXT: [[EXT:%.+]] = zext <vscale x 2 x i7> [[TMP11]] to <vscale x 2 x i64> ; CHECK-NEXT: [[TMP13:%.*]] = mul i64 [[TMP12]], 2
; CHECK-NEXT: [[TMP13:%.*]] = getelementptr inbounds i64, ptr [[TMP12]], i32 0 ; CHECK-NEXT: [[TMP14:%.*]] = add i64 [[TMP13]], 0
; CHECK-NEXT: store <vscale x 2 x i64> [[EXT]], ptr [[TMP13]], align 8 ; CHECK-NEXT: [[TMP15:%.*]] = mul i64 [[TMP14]], 1
; CHECK-NEXT: [[TMP15:%.*]] = call i64 @llvm.vscale.i64() ; CHECK-NEXT: [[TMP16:%.*]] = add i64 [[INDEX]], [[TMP15]]
; CHECK-NEXT: [[TMP16:%.*]] = mul i64 [[TMP15]], 2 ; CHECK-NEXT: [[TMP17:%.*]] = add <vscale x 2 x i7> [[VEC_IND]], zeroinitializer
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], [[TMP16]] ; CHECK-NEXT: [[TMP18:%.*]] = add <vscale x 2 x i7> [[STEP_ADD]], zeroinitializer
; CHECK-NEXT: [[VEC_IND_NEXT]] = add <vscale x 2 x i7> [[VEC_IND]], ; CHECK-NEXT: [[TMP19:%.*]] = getelementptr inbounds i64, ptr [[DST]], i64 [[TMP11]]
; CHECK-NEXT: [[TMP20:%.*]] = getelementptr inbounds i64, ptr [[DST]], i64 [[TMP16]]
; CHECK-NEXT: [[TMP21:%.*]] = zext <vscale x 2 x i7> [[TMP17]] to <vscale x 2 x i64>
; CHECK-NEXT: [[TMP22:%.*]] = zext <vscale x 2 x i7> [[TMP18]] to <vscale x 2 x i64>
; CHECK-NEXT: [[TMP23:%.*]] = getelementptr inbounds i64, ptr [[TMP19]], i32 0
; CHECK-NEXT: store <vscale x 2 x i64> [[TMP21]], ptr [[TMP23]], align 8
; CHECK-NEXT: [[TMP24:%.*]] = call i64 @llvm.vscale.i64()
; CHECK-NEXT: [[TMP25:%.*]] = mul i64 [[TMP24]], 2
; CHECK-NEXT: [[TMP26:%.*]] = getelementptr inbounds i64, ptr [[TMP19]], i64 [[TMP25]]
; CHECK-NEXT: store <vscale x 2 x i64> [[TMP22]], ptr [[TMP26]], align 8
; CHECK-NEXT: [[TMP27:%.*]] = call i64 @llvm.vscale.i64()
; CHECK-NEXT: [[TMP28:%.*]] = mul i64 [[TMP27]], 4
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], [[TMP28]]
; CHECK-NEXT: [[VEC_IND_NEXT]] = add <vscale x 2 x i7> [[STEP_ADD]], [[DOTSPLAT]]
; CHECK-NEXT: [[TMP29:%.*]] = icmp eq i64 [[INDEX_NEXT]], [[N_VEC]]
; CHECK-NEXT: br i1 [[TMP29]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP0:![0-9]+]]
; CHECK: middle.block:
; CHECK-NEXT: [[CMP_N:%.*]] = icmp eq i64 64, [[N_VEC]]
; CHECK-NEXT: br i1 [[CMP_N]], label [[FOR_END:%.*]], label [[SCALAR_PH]]
; CHECK: scalar.ph:
; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i7 [ [[IND_END]], [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]
; CHECK-NEXT: [[BC_RESUME_VAL1:%.*]] = phi i64 [ [[N_VEC]], [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY]] ]
; CHECK-NEXT: br label [[FOR_BODY:%.*]]
; CHECK: for.body:
; CHECK-NEXT: [[INDVARS_IV1294:%.*]] = phi i7 [ [[INDVARS_IV_NEXT1295:%.*]], [[FOR_BODY]] ], [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ]
; CHECK-NEXT: [[INDVARS_IV1286:%.*]] = phi i64 [ [[INDVARS_IV_NEXT1287:%.*]], [[FOR_BODY]] ], [ [[BC_RESUME_VAL1]], [[SCALAR_PH]] ]
; CHECK-NEXT: [[ADDI7:%.*]] = add i7 [[INDVARS_IV1294]], 0
; CHECK-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds i64, ptr [[DST]], i64 [[INDVARS_IV1286]]
; CHECK-NEXT: [[EXT:%.*]] = zext i7 [[ADDI7]] to i64
; CHECK-NEXT: store i64 [[EXT]], ptr [[ARRAYIDX]], align 8
; CHECK-NEXT: [[INDVARS_IV_NEXT1287]] = add nuw nsw i64 [[INDVARS_IV1286]], 1
; CHECK-NEXT: [[INDVARS_IV_NEXT1295]] = add i7 [[INDVARS_IV1294]], 1
; CHECK-NEXT: [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT1287]], 64
; CHECK-NEXT: br i1 [[EXITCOND]], label [[FOR_END]], label [[FOR_BODY]], !llvm.loop [[LOOP3:![0-9]+]]
; CHECK: for.end:
; CHECK-NEXT: ret void
; ;
entry: entry:
br label %for.body br label %for.body
for.body: ; preds = %for.body, %entry for.body: ; preds = %for.body, %entry
%indvars.iv1294 = phi i7 [ %indvars.iv.next1295, %for.body ], [ 0, %entry ] %indvars.iv1294 = phi i7 [ %indvars.iv.next1295, %for.body ], [ 0, %entry ]
%indvars.iv1286 = phi i64 [ %indvars.iv.next1287, %for.body ], [ 0, %entry ] %indvars.iv1286 = phi i64 [ %indvars.iv.next1287, %for.body ], [ 0, %entry ]
%addi7 = add i7 %indvars.iv1294, 0 %addi7 = add i7 %indvars.iv1294, 0
Show All 10 Lines
} }
; DEBUG: Found an estimated cost of Invalid for VF vscale x 1 For instruction: %indvars.iv1294 = phi i3 [ %indvars.iv.next1295, %for.body ], [ 0, %entry ] ; DEBUG: Found an estimated cost of Invalid for VF vscale x 1 For instruction: %indvars.iv1294 = phi i3 [ %indvars.iv.next1295, %for.body ], [ 0, %entry ]
; DEBUG: Found an estimated cost of Invalid for VF vscale x 1 For instruction: %zexti3 = zext i3 %indvars.iv1294 to i64 ; DEBUG: Found an estimated cost of Invalid for VF vscale x 1 For instruction: %zexti3 = zext i3 %indvars.iv1294 to i64
; DEBUG: Found an estimated cost of Invalid for VF vscale x 1 For instruction: %indvars.iv.next1295 = add i3 %indvars.iv1294, 1 ; DEBUG: Found an estimated cost of Invalid for VF vscale x 1 For instruction: %indvars.iv.next1295 = add i3 %indvars.iv1294, 1
define void @induction_i3_zext(ptr %dst) #0 { define void @induction_i3_zext(ptr %dst) #0 {
; CHECK-LABEL: @induction_i3_zext( ; CHECK-LABEL: define void @induction_i3_zext(
; CHECK-SAME: ptr [[DST:%.*]]) #[[ATTR0]] {
; CHECK-NEXT: entry:
; CHECK-NEXT: [[TMP0:%.*]] = call i64 @llvm.vscale.i64()
; CHECK-NEXT: [[TMP1:%.*]] = mul i64 [[TMP0]], 4
; CHECK-NEXT: [[MIN_ITERS_CHECK:%.*]] = icmp ult i64 64, [[TMP1]]
; CHECK-NEXT: br i1 [[MIN_ITERS_CHECK]], label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]]
; CHECK: vector.ph: ; CHECK: vector.ph:
; CHECK: [[TMP4:%.*]] = call <vscale x 2 x i8> @llvm.experimental.stepvector.nxv2i8() ; CHECK-NEXT: [[TMP2:%.*]] = call i64 @llvm.vscale.i64()
; CHECK: [[TMP5:%.*]] = trunc <vscale x 2 x i8> [[TMP4]] to <vscale x 2 x i3> ; CHECK-NEXT: [[TMP3:%.*]] = mul i64 [[TMP2]], 4
; CHECK-NEXT: [[N_MOD_VF:%.*]] = urem i64 64, [[TMP3]]
; CHECK-NEXT: [[N_VEC:%.*]] = sub i64 64, [[N_MOD_VF]]
; CHECK-NEXT: [[IND_END:%.*]] = trunc i64 [[N_VEC]] to i3
; CHECK-NEXT: [[TMP4:%.*]] = call <vscale x 2 x i8> @llvm.experimental.stepvector.nxv2i8()
; CHECK-NEXT: [[TMP5:%.*]] = trunc <vscale x 2 x i8> [[TMP4]] to <vscale x 2 x i3>
; CHECK-NEXT: [[TMP6:%.*]] = add <vscale x 2 x i3> [[TMP5]], zeroinitializer ; CHECK-NEXT: [[TMP6:%.*]] = add <vscale x 2 x i3> [[TMP5]], zeroinitializer
; CHECK-NEXT: [[TMP7:%.*]] = mul <vscale x 2 x i3> [[TMP6]], shufflevector (<vscale x 2 x i3> insertelement (<vscale x 2 x i3> poison, i3 1, i64 0), <vscale x 2 x i3> poison, <vscale x 2 x i32> zeroinitializer) ; CHECK-NEXT: [[TMP7:%.*]] = mul <vscale x 2 x i3> [[TMP6]], shufflevector (<vscale x 2 x i3> insertelement (<vscale x 2 x i3> poison, i3 1, i64 0), <vscale x 2 x i3> poison, <vscale x 2 x i32> zeroinitializer)
; CHECK-NEXT: [[INDUCTION:%.*]] = add <vscale x 2 x i3> zeroinitializer, [[TMP7]] ; CHECK-NEXT: [[INDUCTION:%.*]] = add <vscale x 2 x i3> zeroinitializer, [[TMP7]]
; CHECK-NEXT: [[TMP8:%.*]] = call i3 @llvm.vscale.i3()
; CHECK-NEXT: [[TMP9:%.*]] = mul i3 [[TMP8]], 2
; CHECK-NEXT: [[TMP10:%.*]] = mul i3 1, [[TMP9]]
; CHECK-NEXT: [[DOTSPLATINSERT:%.*]] = insertelement <vscale x 2 x i3> poison, i3 [[TMP10]], i64 0
; CHECK-NEXT: [[DOTSPLAT:%.*]] = shufflevector <vscale x 2 x i3> [[DOTSPLATINSERT]], <vscale x 2 x i3> poison, <vscale x 2 x i32> zeroinitializer
; CHECK-NEXT: br label [[VECTOR_BODY:%.*]]
; CHECK: vector.body: ; CHECK: vector.body:
; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, %vector.ph ], [ [[INDEX_NEXT:%.*]], %vector.body ] ; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[VEC_IND:%.*]] = phi <vscale x 2 x i3> [ [[INDUCTION]], %vector.ph ], [ [[VEC_IND_NEXT:%.*]], %vector.body ] ; CHECK-NEXT: [[VEC_IND:%.*]] = phi <vscale x 2 x i3> [ [[INDUCTION]], [[VECTOR_PH]] ], [ [[VEC_IND_NEXT:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[TMP9:%.*]] = add i64 [[INDEX]], 0 ; CHECK-NEXT: [[STEP_ADD:%.*]] = add <vscale x 2 x i3> [[VEC_IND]], [[DOTSPLAT]]
; CHECK-NEXT: [[TMP10:%.*]] = zext <vscale x 2 x i3> [[VEC_IND]] to <vscale x 2 x i64> ; CHECK-NEXT: [[TMP11:%.*]] = add i64 [[INDEX]], 0
; CHECK-NEXT: [[TMP12:%.*]] = getelementptr inbounds i64, ptr [[DST:%.*]], i64 [[TMP9]] ; CHECK-NEXT: [[TMP12:%.*]] = call i64 @llvm.vscale.i64()
; CHECK-NEXT: [[TMP13:%.*]] = getelementptr inbounds i64, ptr [[TMP12]], i32 0 ; CHECK-NEXT: [[TMP13:%.*]] = mul i64 [[TMP12]], 2
; CHECK-NEXT: store <vscale x 2 x i64> [[TMP10]], ptr [[TMP13]], align 8 ; CHECK-NEXT: [[TMP14:%.*]] = add i64 [[TMP13]], 0
; CHECK-NEXT: [[TMP15:%.*]] = call i64 @llvm.vscale.i64() ; CHECK-NEXT: [[TMP15:%.*]] = mul i64 [[TMP14]], 1
; CHECK-NEXT: [[TMP16:%.*]] = mul i64 [[TMP15]], 2 ; CHECK-NEXT: [[TMP16:%.*]] = add i64 [[INDEX]], [[TMP15]]
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], [[TMP16]] ; CHECK-NEXT: [[TMP17:%.*]] = zext <vscale x 2 x i3> [[VEC_IND]] to <vscale x 2 x i64>
; CHECK-NEXT: [[VEC_IND_NEXT]] = add <vscale x 2 x i3> [[VEC_IND]], ; CHECK-NEXT: [[TMP18:%.*]] = zext <vscale x 2 x i3> [[STEP_ADD]] to <vscale x 2 x i64>
; CHECK-NEXT: [[TMP19:%.*]] = getelementptr inbounds i64, ptr [[DST]], i64 [[TMP11]]
; CHECK-NEXT: [[TMP20:%.*]] = getelementptr inbounds i64, ptr [[DST]], i64 [[TMP16]]
; CHECK-NEXT: [[TMP21:%.*]] = getelementptr inbounds i64, ptr [[TMP19]], i32 0
; CHECK-NEXT: store <vscale x 2 x i64> [[TMP17]], ptr [[TMP21]], align 8
; CHECK-NEXT: [[TMP22:%.*]] = call i64 @llvm.vscale.i64()
; CHECK-NEXT: [[TMP23:%.*]] = mul i64 [[TMP22]], 2
; CHECK-NEXT: [[TMP24:%.*]] = getelementptr inbounds i64, ptr [[TMP19]], i64 [[TMP23]]
; CHECK-NEXT: store <vscale x 2 x i64> [[TMP18]], ptr [[TMP24]], align 8
; CHECK-NEXT: [[TMP25:%.*]] = call i64 @llvm.vscale.i64()
; CHECK-NEXT: [[TMP26:%.*]] = mul i64 [[TMP25]], 4
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], [[TMP26]]
; CHECK-NEXT: [[VEC_IND_NEXT]] = add <vscale x 2 x i3> [[STEP_ADD]], [[DOTSPLAT]]
; CHECK-NEXT: [[TMP27:%.*]] = icmp eq i64 [[INDEX_NEXT]], [[N_VEC]]
; CHECK-NEXT: br i1 [[TMP27]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP4:![0-9]+]]
; CHECK: middle.block:
; CHECK-NEXT: [[CMP_N:%.*]] = icmp eq i64 64, [[N_VEC]]
; CHECK-NEXT: br i1 [[CMP_N]], label [[FOR_END:%.*]], label [[SCALAR_PH]]
; CHECK: scalar.ph:
; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i3 [ [[IND_END]], [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]
; CHECK-NEXT: [[BC_RESUME_VAL1:%.*]] = phi i64 [ [[N_VEC]], [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY]] ]
; CHECK-NEXT: br label [[FOR_BODY:%.*]]
; CHECK: for.body:
; CHECK-NEXT: [[INDVARS_IV1294:%.*]] = phi i3 [ [[INDVARS_IV_NEXT1295:%.*]], [[FOR_BODY]] ], [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ]
; CHECK-NEXT: [[INDVARS_IV1286:%.*]] = phi i64 [ [[INDVARS_IV_NEXT1287:%.*]], [[FOR_BODY]] ], [ [[BC_RESUME_VAL1]], [[SCALAR_PH]] ]
; CHECK-NEXT: [[ZEXTI3:%.*]] = zext i3 [[INDVARS_IV1294]] to i64
; CHECK-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds i64, ptr [[DST]], i64 [[INDVARS_IV1286]]
; CHECK-NEXT: store i64 [[ZEXTI3]], ptr [[ARRAYIDX]], align 8
; CHECK-NEXT: [[INDVARS_IV_NEXT1287]] = add nuw nsw i64 [[INDVARS_IV1286]], 1
; CHECK-NEXT: [[INDVARS_IV_NEXT1295]] = add i3 [[INDVARS_IV1294]], 1
; CHECK-NEXT: [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT1287]], 64
; CHECK-NEXT: br i1 [[EXITCOND]], label [[FOR_END]], label [[FOR_BODY]], !llvm.loop [[LOOP5:![0-9]+]]
; CHECK: for.end:
; CHECK-NEXT: ret void
; ;
entry: entry:
br label %for.body br label %for.body
for.body: ; preds = %for.body, %entry for.body: ; preds = %for.body, %entry
%indvars.iv1294 = phi i3 [ %indvars.iv.next1295, %for.body ], [ 0, %entry ] %indvars.iv1294 = phi i3 [ %indvars.iv.next1295, %for.body ], [ 0, %entry ]
%indvars.iv1286 = phi i64 [ %indvars.iv.next1287, %for.body ], [ 0, %entry ] %indvars.iv1286 = phi i64 [ %indvars.iv.next1287, %for.body ], [ 0, %entry ]
%zexti3 = zext i3 %indvars.iv1294 to i64 %zexti3 = zext i3 %indvars.iv1294 to i64
Show All 13 Lines

llvm/test/Transforms/LoopVectorize/SystemZ/zero_unroll.ll

; RUN: opt -S -passes=loop-vectorize -mtriple=s390x-linux-gnu -tiny-trip-count-interleave-threshold=4 -vectorizer-min-trip-count=8 < %s | FileCheck %s ; RUN: opt -S -passes=loop-vectorize -mtriple=s390x-linux-gnu -vectorizer-min-trip-count=8 < %s | FileCheck %s
define i32 @main(i32 %arg, ptr nocapture readnone %arg1) #0 { define i32 @main(i32 %arg, ptr nocapture readnone %arg1) #0 {
;CHECK: vector.body: ;CHECK: vector.body:
entry: entry:
%0 = alloca i8, align 1 %0 = alloca i8, align 1
br label %loop br label %loop
loop: loop:
Show All 12 Lines

llvm/test/Transforms/LoopVectorize/X86/imprecise-through-phis.ll

Show First 20 Lines • Show All 67 Lines • ▼ Show 20 Lines
; SSE-LABEL: @sumIfVector( ; SSE-LABEL: @sumIfVector(
; SSE-NEXT: entry: ; SSE-NEXT: entry:
; SSE-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]] ; SSE-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]]
; SSE: vector.ph: ; SSE: vector.ph:
; SSE-NEXT: br label [[VECTOR_BODY:%.*]] ; SSE-NEXT: br label [[VECTOR_BODY:%.*]]
; SSE: vector.body: ; SSE: vector.body:
; SSE-NEXT: [[INDEX:%.*]] = phi i32 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ] ; SSE-NEXT: [[INDEX:%.*]] = phi i32 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
; SSE-NEXT: [[VEC_PHI:%.*]] = phi <2 x double> [ zeroinitializer, [[VECTOR_PH]] ], [ [[PREDPHI:%.*]], [[VECTOR_BODY]] ] ; SSE-NEXT: [[VEC_PHI:%.*]] = phi <2 x double> [ zeroinitializer, [[VECTOR_PH]] ], [ [[PREDPHI:%.*]], [[VECTOR_BODY]] ]
; SSE-NEXT: [[VEC_PHI1:%.*]] = phi <2 x double> [ zeroinitializer, [[VECTOR_PH]] ], [ [[PREDPHI3:%.*]], [[VECTOR_BODY]] ]
; SSE-NEXT: [[TMP0:%.*]] = add i32 [[INDEX]], 0 ; SSE-NEXT: [[TMP0:%.*]] = add i32 [[INDEX]], 0
; SSE-NEXT: [[TMP1:%.*]] = getelementptr double, ptr [[ARR:%.*]], i32 [[TMP0]] ; SSE-NEXT: [[TMP1:%.*]] = add i32 [[INDEX]], 2
; SSE-NEXT: [[TMP2:%.*]] = getelementptr double, ptr [[TMP1]], i32 0 ; SSE-NEXT: [[TMP2:%.*]] = getelementptr double, ptr [[ARR:%.*]], i32 [[TMP0]]
; SSE-NEXT: [[WIDE_LOAD:%.*]] = load <2 x double>, ptr [[TMP2]], align 8 ; SSE-NEXT: [[TMP3:%.*]] = getelementptr double, ptr [[ARR]], i32 [[TMP1]]
; SSE-NEXT: [[TMP3:%.*]] = fcmp fast une <2 x double> [[WIDE_LOAD]], <double 4.200000e+01, double 4.200000e+01> ; SSE-NEXT: [[TMP4:%.*]] = getelementptr double, ptr [[TMP2]], i32 0
; SSE-NEXT: [[TMP4:%.*]] = fadd fast <2 x double> [[VEC_PHI]], [[WIDE_LOAD]] ; SSE-NEXT: [[WIDE_LOAD:%.*]] = load <2 x double>, ptr [[TMP4]], align 8
; SSE-NEXT: [[TMP5:%.*]] = xor <2 x i1> [[TMP3]], <i1 true, i1 true> ; SSE-NEXT: [[TMP5:%.*]] = getelementptr double, ptr [[TMP2]], i32 2
; SSE-NEXT: [[PREDPHI]] = select <2 x i1> [[TMP3]], <2 x double> [[TMP4]], <2 x double> [[VEC_PHI]] ; SSE-NEXT: [[WIDE_LOAD2:%.*]] = load <2 x double>, ptr [[TMP5]], align 8
; SSE-NEXT: [[INDEX_NEXT]] = add nuw i32 [[INDEX]], 2 ; SSE-NEXT: [[TMP6:%.*]] = fcmp fast une <2 x double> [[WIDE_LOAD]], <double 4.200000e+01, double 4.200000e+01>
; SSE-NEXT: [[TMP6:%.*]] = icmp eq i32 [[INDEX_NEXT]], 32 ; SSE-NEXT: [[TMP7:%.*]] = fcmp fast une <2 x double> [[WIDE_LOAD2]], <double 4.200000e+01, double 4.200000e+01>
; SSE-NEXT: br i1 [[TMP6]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP0:![0-9]+]] ; SSE-NEXT: [[TMP8:%.*]] = fadd fast <2 x double> [[VEC_PHI]], [[WIDE_LOAD]]
; SSE-NEXT: [[TMP9:%.*]] = fadd fast <2 x double> [[VEC_PHI1]], [[WIDE_LOAD2]]
; SSE-NEXT: [[TMP10:%.*]] = xor <2 x i1> [[TMP6]], <i1 true, i1 true>
; SSE-NEXT: [[TMP11:%.*]] = xor <2 x i1> [[TMP7]], <i1 true, i1 true>
; SSE-NEXT: [[PREDPHI]] = select <2 x i1> [[TMP6]], <2 x double> [[TMP8]], <2 x double> [[VEC_PHI]]
; SSE-NEXT: [[PREDPHI3]] = select <2 x i1> [[TMP7]], <2 x double> [[TMP9]], <2 x double> [[VEC_PHI1]]
; SSE-NEXT: [[INDEX_NEXT]] = add nuw i32 [[INDEX]], 4
; SSE-NEXT: [[TMP12:%.*]] = icmp eq i32 [[INDEX_NEXT]], 32
; SSE-NEXT: br i1 [[TMP12]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP0:![0-9]+]]
; SSE: middle.block: ; SSE: middle.block:
; SSE-NEXT: [[TMP7:%.*]] = call fast double @llvm.vector.reduce.fadd.v2f64(double -0.000000e+00, <2 x double> [[PREDPHI]]) ; SSE-NEXT: [[BIN_RDX:%.*]] = fadd fast <2 x double> [[PREDPHI3]], [[PREDPHI]]
; SSE-NEXT: [[TMP13:%.*]] = call fast double @llvm.vector.reduce.fadd.v2f64(double -0.000000e+00, <2 x double> [[BIN_RDX]])
; SSE-NEXT: br i1 true, label [[DONE:%.*]], label [[SCALAR_PH]] ; SSE-NEXT: br i1 true, label [[DONE:%.*]], label [[SCALAR_PH]]
; SSE: scalar.ph: ; SSE: scalar.ph:
; SSE-NEXT: [[BC_RESUME_VAL:%.*]] = phi i32 [ 32, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ] ; SSE-NEXT: [[BC_RESUME_VAL:%.*]] = phi i32 [ 32, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]
; SSE-NEXT: [[BC_MERGE_RDX:%.*]] = phi double [ 0.000000e+00, [[ENTRY]] ], [ [[TMP7]], [[MIDDLE_BLOCK]] ] ; SSE-NEXT: [[BC_MERGE_RDX:%.*]] = phi double [ 0.000000e+00, [[ENTRY]] ], [ [[TMP13]], [[MIDDLE_BLOCK]] ]
; SSE-NEXT: br label [[LOOP:%.*]] ; SSE-NEXT: br label [[LOOP:%.*]]
; SSE: loop: ; SSE: loop:
; SSE-NEXT: [[I:%.*]] = phi i32 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[I_NEXT:%.*]], [[NEXT_ITER:%.*]] ] ; SSE-NEXT: [[I:%.*]] = phi i32 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[I_NEXT:%.*]], [[NEXT_ITER:%.*]] ]
; SSE-NEXT: [[TOT:%.*]] = phi double [ [[BC_MERGE_RDX]], [[SCALAR_PH]] ], [ [[TOT_NEXT:%.*]], [[NEXT_ITER]] ] ; SSE-NEXT: [[TOT:%.*]] = phi double [ [[BC_MERGE_RDX]], [[SCALAR_PH]] ], [ [[TOT_NEXT:%.*]], [[NEXT_ITER]] ]
; SSE-NEXT: [[ADDR:%.*]] = getelementptr double, ptr [[ARR]], i32 [[I]] ; SSE-NEXT: [[ADDR:%.*]] = getelementptr double, ptr [[ARR]], i32 [[I]]
; SSE-NEXT: [[NEXTVAL:%.*]] = load double, ptr [[ADDR]], align 8 ; SSE-NEXT: [[NEXTVAL:%.*]] = load double, ptr [[ADDR]], align 8
; SSE-NEXT: [[TST:%.*]] = fcmp fast une double [[NEXTVAL]], 4.200000e+01 ; SSE-NEXT: [[TST:%.*]] = fcmp fast une double [[NEXTVAL]], 4.200000e+01
; SSE-NEXT: br i1 [[TST]], label [[DO_ADD:%.*]], label [[NO_ADD:%.*]] ; SSE-NEXT: br i1 [[TST]], label [[DO_ADD:%.*]], label [[NO_ADD:%.*]]
; SSE: do.add: ; SSE: do.add:
; SSE-NEXT: [[TOT_NEW:%.*]] = fadd fast double [[TOT]], [[NEXTVAL]] ; SSE-NEXT: [[TOT_NEW:%.*]] = fadd fast double [[TOT]], [[NEXTVAL]]
; SSE-NEXT: br label [[NEXT_ITER]] ; SSE-NEXT: br label [[NEXT_ITER]]
; SSE: no.add: ; SSE: no.add:
; SSE-NEXT: br label [[NEXT_ITER]] ; SSE-NEXT: br label [[NEXT_ITER]]
; SSE: next.iter: ; SSE: next.iter:
; SSE-NEXT: [[TOT_NEXT]] = phi double [ [[TOT]], [[NO_ADD]] ], [ [[TOT_NEW]], [[DO_ADD]] ] ; SSE-NEXT: [[TOT_NEXT]] = phi double [ [[TOT]], [[NO_ADD]] ], [ [[TOT_NEW]], [[DO_ADD]] ]
; SSE-NEXT: [[I_NEXT]] = add i32 [[I]], 1 ; SSE-NEXT: [[I_NEXT]] = add i32 [[I]], 1
; SSE-NEXT: [[AGAIN:%.*]] = icmp ult i32 [[I_NEXT]], 32 ; SSE-NEXT: [[AGAIN:%.*]] = icmp ult i32 [[I_NEXT]], 32
; SSE-NEXT: br i1 [[AGAIN]], label [[LOOP]], label [[DONE]], !llvm.loop [[LOOP2:![0-9]+]] ; SSE-NEXT: br i1 [[AGAIN]], label [[LOOP]], label [[DONE]], !llvm.loop [[LOOP3:![0-9]+]]
; SSE: done: ; SSE: done:
; SSE-NEXT: [[TOT_NEXT_LCSSA:%.*]] = phi double [ [[TOT_NEXT]], [[NEXT_ITER]] ], [ [[TMP7]], [[MIDDLE_BLOCK]] ] ; SSE-NEXT: [[TOT_NEXT_LCSSA:%.*]] = phi double [ [[TOT_NEXT]], [[NEXT_ITER]] ], [ [[TMP13]], [[MIDDLE_BLOCK]] ]
; SSE-NEXT: ret double [[TOT_NEXT_LCSSA]] ; SSE-NEXT: ret double [[TOT_NEXT_LCSSA]]
; ;
; AVX-LABEL: @sumIfVector( ; AVX-LABEL: @sumIfVector(
; AVX-NEXT: entry: ; AVX-NEXT: entry:
; AVX-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]] ; AVX-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]]
; AVX: vector.ph: ; AVX: vector.ph:
; AVX-NEXT: br label [[VECTOR_BODY:%.*]] ; AVX-NEXT: br label [[VECTOR_BODY:%.*]]
; AVX: vector.body: ; AVX: vector.body:
; AVX-NEXT: [[INDEX:%.*]] = phi i32 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ] ; AVX-NEXT: [[INDEX:%.*]] = phi i32 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
; AVX-NEXT: [[VEC_PHI:%.*]] = phi <4 x double> [ zeroinitializer, [[VECTOR_PH]] ], [ [[PREDPHI:%.*]], [[VECTOR_BODY]] ] ; AVX-NEXT: [[VEC_PHI:%.*]] = phi <4 x double> [ zeroinitializer, [[VECTOR_PH]] ], [ [[PREDPHI:%.*]], [[VECTOR_BODY]] ]
; AVX-NEXT: [[VEC_PHI1:%.*]] = phi <4 x double> [ zeroinitializer, [[VECTOR_PH]] ], [ [[PREDPHI7:%.*]], [[VECTOR_BODY]] ]
; AVX-NEXT: [[VEC_PHI2:%.*]] = phi <4 x double> [ zeroinitializer, [[VECTOR_PH]] ], [ [[PREDPHI8:%.*]], [[VECTOR_BODY]] ]
; AVX-NEXT: [[VEC_PHI3:%.*]] = phi <4 x double> [ zeroinitializer, [[VECTOR_PH]] ], [ [[PREDPHI9:%.*]], [[VECTOR_BODY]] ]
; AVX-NEXT: [[TMP0:%.*]] = add i32 [[INDEX]], 0 ; AVX-NEXT: [[TMP0:%.*]] = add i32 [[INDEX]], 0
; AVX-NEXT: [[TMP1:%.*]] = getelementptr double, ptr [[ARR:%.*]], i32 [[TMP0]] ; AVX-NEXT: [[TMP1:%.*]] = add i32 [[INDEX]], 4
; AVX-NEXT: [[TMP2:%.*]] = getelementptr double, ptr [[TMP1]], i32 0 ; AVX-NEXT: [[TMP2:%.*]] = add i32 [[INDEX]], 8
; AVX-NEXT: [[WIDE_LOAD:%.*]] = load <4 x double>, ptr [[TMP2]], align 8 ; AVX-NEXT: [[TMP3:%.*]] = add i32 [[INDEX]], 12
; AVX-NEXT: [[TMP3:%.*]] = fcmp fast une <4 x double> [[WIDE_LOAD]], <double 4.200000e+01, double 4.200000e+01, double 4.200000e+01, double 4.200000e+01> ; AVX-NEXT: [[TMP4:%.*]] = getelementptr double, ptr [[ARR:%.*]], i32 [[TMP0]]
; AVX-NEXT: [[TMP4:%.*]] = fadd fast <4 x double> [[VEC_PHI]], [[WIDE_LOAD]] ; AVX-NEXT: [[TMP5:%.*]] = getelementptr double, ptr [[ARR]], i32 [[TMP1]]
; AVX-NEXT: [[TMP5:%.*]] = xor <4 x i1> [[TMP3]], <i1 true, i1 true, i1 true, i1 true> ; AVX-NEXT: [[TMP6:%.*]] = getelementptr double, ptr [[ARR]], i32 [[TMP2]]
; AVX-NEXT: [[PREDPHI]] = select <4 x i1> [[TMP3]], <4 x double> [[TMP4]], <4 x double> [[VEC_PHI]] ; AVX-NEXT: [[TMP7:%.*]] = getelementptr double, ptr [[ARR]], i32 [[TMP3]]
; AVX-NEXT: [[INDEX_NEXT]] = add nuw i32 [[INDEX]], 4 ; AVX-NEXT: [[TMP8:%.*]] = getelementptr double, ptr [[TMP4]], i32 0
; AVX-NEXT: [[TMP6:%.*]] = icmp eq i32 [[INDEX_NEXT]], 32 ; AVX-NEXT: [[WIDE_LOAD:%.*]] = load <4 x double>, ptr [[TMP8]], align 8
; AVX-NEXT: br i1 [[TMP6]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP0:![0-9]+]] ; AVX-NEXT: [[TMP9:%.*]] = getelementptr double, ptr [[TMP4]], i32 4
; AVX-NEXT: [[WIDE_LOAD4:%.*]] = load <4 x double>, ptr [[TMP9]], align 8
; AVX-NEXT: [[TMP10:%.*]] = getelementptr double, ptr [[TMP4]], i32 8
; AVX-NEXT: [[WIDE_LOAD5:%.*]] = load <4 x double>, ptr [[TMP10]], align 8
; AVX-NEXT: [[TMP11:%.*]] = getelementptr double, ptr [[TMP4]], i32 12
; AVX-NEXT: [[WIDE_LOAD6:%.*]] = load <4 x double>, ptr [[TMP11]], align 8
; AVX-NEXT: [[TMP12:%.*]] = fcmp fast une <4 x double> [[WIDE_LOAD]], <double 4.200000e+01, double 4.200000e+01, double 4.200000e+01, double 4.200000e+01>
; AVX-NEXT: [[TMP13:%.*]] = fcmp fast une <4 x double> [[WIDE_LOAD4]], <double 4.200000e+01, double 4.200000e+01, double 4.200000e+01, double 4.200000e+01>
; AVX-NEXT: [[TMP14:%.*]] = fcmp fast une <4 x double> [[WIDE_LOAD5]], <double 4.200000e+01, double 4.200000e+01, double 4.200000e+01, double 4.200000e+01>
; AVX-NEXT: [[TMP15:%.*]] = fcmp fast une <4 x double> [[WIDE_LOAD6]], <double 4.200000e+01, double 4.200000e+01, double 4.200000e+01, double 4.200000e+01>
; AVX-NEXT: [[TMP16:%.*]] = fadd fast <4 x double> [[VEC_PHI]], [[WIDE_LOAD]]
; AVX-NEXT: [[TMP17:%.*]] = fadd fast <4 x double> [[VEC_PHI1]], [[WIDE_LOAD4]]
; AVX-NEXT: [[TMP18:%.*]] = fadd fast <4 x double> [[VEC_PHI2]], [[WIDE_LOAD5]]
; AVX-NEXT: [[TMP19:%.*]] = fadd fast <4 x double> [[VEC_PHI3]], [[WIDE_LOAD6]]
; AVX-NEXT: [[TMP20:%.*]] = xor <4 x i1> [[TMP12]], <i1 true, i1 true, i1 true, i1 true>
; AVX-NEXT: [[TMP21:%.*]] = xor <4 x i1> [[TMP13]], <i1 true, i1 true, i1 true, i1 true>
; AVX-NEXT: [[TMP22:%.*]] = xor <4 x i1> [[TMP14]], <i1 true, i1 true, i1 true, i1 true>
; AVX-NEXT: [[TMP23:%.*]] = xor <4 x i1> [[TMP15]], <i1 true, i1 true, i1 true, i1 true>
; AVX-NEXT: [[PREDPHI]] = select <4 x i1> [[TMP12]], <4 x double> [[TMP16]], <4 x double> [[VEC_PHI]]
; AVX-NEXT: [[PREDPHI7]] = select <4 x i1> [[TMP13]], <4 x double> [[TMP17]], <4 x double> [[VEC_PHI1]]
; AVX-NEXT: [[PREDPHI8]] = select <4 x i1> [[TMP14]], <4 x double> [[TMP18]], <4 x double> [[VEC_PHI2]]
; AVX-NEXT: [[PREDPHI9]] = select <4 x i1> [[TMP15]], <4 x double> [[TMP19]], <4 x double> [[VEC_PHI3]]
; AVX-NEXT: [[INDEX_NEXT]] = add nuw i32 [[INDEX]], 16
; AVX-NEXT: [[TMP24:%.*]] = icmp eq i32 [[INDEX_NEXT]], 32
; AVX-NEXT: br i1 [[TMP24]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP0:![0-9]+]]
; AVX: middle.block: ; AVX: middle.block:
; AVX-NEXT: [[TMP7:%.*]] = call fast double @llvm.vector.reduce.fadd.v4f64(double -0.000000e+00, <4 x double> [[PREDPHI]]) ; AVX-NEXT: [[BIN_RDX:%.*]] = fadd fast <4 x double> [[PREDPHI7]], [[PREDPHI]]
; AVX-NEXT: [[BIN_RDX10:%.*]] = fadd fast <4 x double> [[PREDPHI8]], [[BIN_RDX]]
; AVX-NEXT: [[BIN_RDX11:%.*]] = fadd fast <4 x double> [[PREDPHI9]], [[BIN_RDX10]]
; AVX-NEXT: [[TMP25:%.*]] = call fast double @llvm.vector.reduce.fadd.v4f64(double -0.000000e+00, <4 x double> [[BIN_RDX11]])
; AVX-NEXT: br i1 true, label [[DONE:%.*]], label [[SCALAR_PH]] ; AVX-NEXT: br i1 true, label [[DONE:%.*]], label [[SCALAR_PH]]
; AVX: scalar.ph: ; AVX: scalar.ph:
; AVX-NEXT: [[BC_RESUME_VAL:%.*]] = phi i32 [ 32, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ] ; AVX-NEXT: [[BC_RESUME_VAL:%.*]] = phi i32 [ 32, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]
; AVX-NEXT: [[BC_MERGE_RDX:%.*]] = phi double [ 0.000000e+00, [[ENTRY]] ], [ [[TMP7]], [[MIDDLE_BLOCK]] ] ; AVX-NEXT: [[BC_MERGE_RDX:%.*]] = phi double [ 0.000000e+00, [[ENTRY]] ], [ [[TMP25]], [[MIDDLE_BLOCK]] ]
; AVX-NEXT: br label [[LOOP:%.*]] ; AVX-NEXT: br label [[LOOP:%.*]]
; AVX: loop: ; AVX: loop:
; AVX-NEXT: [[I:%.*]] = phi i32 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[I_NEXT:%.*]], [[NEXT_ITER:%.*]] ] ; AVX-NEXT: [[I:%.*]] = phi i32 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[I_NEXT:%.*]], [[NEXT_ITER:%.*]] ]
; AVX-NEXT: [[TOT:%.*]] = phi double [ [[BC_MERGE_RDX]], [[SCALAR_PH]] ], [ [[TOT_NEXT:%.*]], [[NEXT_ITER]] ] ; AVX-NEXT: [[TOT:%.*]] = phi double [ [[BC_MERGE_RDX]], [[SCALAR_PH]] ], [ [[TOT_NEXT:%.*]], [[NEXT_ITER]] ]
; AVX-NEXT: [[ADDR:%.*]] = getelementptr double, ptr [[ARR]], i32 [[I]] ; AVX-NEXT: [[ADDR:%.*]] = getelementptr double, ptr [[ARR]], i32 [[I]]
; AVX-NEXT: [[NEXTVAL:%.*]] = load double, ptr [[ADDR]], align 8 ; AVX-NEXT: [[NEXTVAL:%.*]] = load double, ptr [[ADDR]], align 8
; AVX-NEXT: [[TST:%.*]] = fcmp fast une double [[NEXTVAL]], 4.200000e+01 ; AVX-NEXT: [[TST:%.*]] = fcmp fast une double [[NEXTVAL]], 4.200000e+01
; AVX-NEXT: br i1 [[TST]], label [[DO_ADD:%.*]], label [[NO_ADD:%.*]] ; AVX-NEXT: br i1 [[TST]], label [[DO_ADD:%.*]], label [[NO_ADD:%.*]]
; AVX: do.add: ; AVX: do.add:
; AVX-NEXT: [[TOT_NEW:%.*]] = fadd fast double [[TOT]], [[NEXTVAL]] ; AVX-NEXT: [[TOT_NEW:%.*]] = fadd fast double [[TOT]], [[NEXTVAL]]
; AVX-NEXT: br label [[NEXT_ITER]] ; AVX-NEXT: br label [[NEXT_ITER]]
; AVX: no.add: ; AVX: no.add:
; AVX-NEXT: br label [[NEXT_ITER]] ; AVX-NEXT: br label [[NEXT_ITER]]
; AVX: next.iter: ; AVX: next.iter:
; AVX-NEXT: [[TOT_NEXT]] = phi double [ [[TOT]], [[NO_ADD]] ], [ [[TOT_NEW]], [[DO_ADD]] ] ; AVX-NEXT: [[TOT_NEXT]] = phi double [ [[TOT]], [[NO_ADD]] ], [ [[TOT_NEW]], [[DO_ADD]] ]
; AVX-NEXT: [[I_NEXT]] = add i32 [[I]], 1 ; AVX-NEXT: [[I_NEXT]] = add i32 [[I]], 1
; AVX-NEXT: [[AGAIN:%.*]] = icmp ult i32 [[I_NEXT]], 32 ; AVX-NEXT: [[AGAIN:%.*]] = icmp ult i32 [[I_NEXT]], 32
; AVX-NEXT: br i1 [[AGAIN]], label [[LOOP]], label [[DONE]], !llvm.loop [[LOOP2:![0-9]+]] ; AVX-NEXT: br i1 [[AGAIN]], label [[LOOP]], label [[DONE]], !llvm.loop [[LOOP3:![0-9]+]]
; AVX: done: ; AVX: done:
; AVX-NEXT: [[TOT_NEXT_LCSSA:%.*]] = phi double [ [[TOT_NEXT]], [[NEXT_ITER]] ], [ [[TMP7]], [[MIDDLE_BLOCK]] ] ; AVX-NEXT: [[TOT_NEXT_LCSSA:%.*]] = phi double [ [[TOT_NEXT]], [[NEXT_ITER]] ], [ [[TMP25]], [[MIDDLE_BLOCK]] ]
; AVX-NEXT: ret double [[TOT_NEXT_LCSSA]] ; AVX-NEXT: ret double [[TOT_NEXT_LCSSA]]
; ;
entry: entry:
br label %loop br label %loop
loop: loop:
%i = phi i32 [0, %entry], [%i.next, %next.iter] %i = phi i32 [0, %entry], [%i.next, %next.iter]
%tot = phi double [0.0, %entry], [%tot.next, %next.iter] %tot = phi double [0.0, %entry], [%tot.next, %next.iter]
Show All 24 Lines

llvm/test/Transforms/LoopVectorize/X86/interleave_short_tc.ll

; Check that we won't interleave by more than "best known" estimated trip count. ; Check that we won't interleave by more than half the "best known" estimated trip count.
; The loop is expected to be vectorized by 4 and interleaving suppresed due to
; short trip count which is controled by "tiny-trip-count-interleave-threshold".
; RUN: opt -passes=loop-vectorize -force-vector-width=4 -vectorizer-min-trip-count=4 -S < %s | FileCheck %s
;
; The loop is expected to be vectorized by 4 and computed interleaving factor is 1. ; The loop is expected to be vectorized by 4 and computed interleaving factor is 1.
; Thus the resulting step is 4. ; Thus the resulting step is 4.
; RUN: opt -passes=loop-vectorize -force-vector-width=4 -vectorizer-min-trip-count=4 -tiny-trip-count-interleave-threshold=4 -S < %s | FileCheck %s ; RUN: opt -passes=loop-vectorize -force-vector-width=4 -vectorizer-min-trip-count=4 -S < %s | FileCheck %s
; The loop is expected to be vectorized by 2 and computed interleaving factor is 2. ; The loop is expected to be vectorized by 2 and computed interleaving factor is 2.
; Thus the resulting step is 4. ; Thus the resulting step is 4.
; RUN: opt -passes=loop-vectorize -force-vector-width=2 -vectorizer-min-trip-count=4 -tiny-trip-count-interleave-threshold=4 -S < %s | FileCheck %s ; RUN: opt -passes=loop-vectorize -force-vector-width=2 -vectorizer-min-trip-count=4 -S < %s | FileCheck %s
; Check that we won't interleave by more than "best known" estimated trip count. ; Check that we won't interleave by more than half the "best known" estimated trip count.
target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128" target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu" target triple = "x86_64-unknown-linux-gnu"
@a = dso_local global [5 x i32] zeroinitializer, align 16 @a = dso_local global [5 x i32] zeroinitializer, align 16
@b = dso_local global [5 x i32] zeroinitializer, align 16 @b = dso_local global [5 x i32] zeroinitializer, align 16
; Function Attrs: nofree norecurse nounwind uwtable ; Function Attrs: nofree norecurse nounwind uwtable
Show All 27 Linesfor.body: ; preds = %for.body, %for.body.preheader
%2 = load i32, ptr %arrayidx2, align 4 %2 = load i32, ptr %arrayidx2, align 4
%add = add nsw i32 %2, %mul %add = add nsw i32 %2, %mul
store i32 %add, ptr %arrayidx2, align 4 store i32 %add, ptr %arrayidx2, align 4
%indvars.iv.next = add nuw nsw i64 %indvars.iv, 1 %indvars.iv.next = add nuw nsw i64 %indvars.iv, 1
%exitcond = icmp eq i64 %indvars.iv.next, %wide.trip.count %exitcond = icmp eq i64 %indvars.iv.next, %wide.trip.count
br i1 %exitcond, label %for.cond.cleanup.loopexit, label %for.body, !prof !1 br i1 %exitcond, label %for.cond.cleanup.loopexit, label %for.body, !prof !1
} }
!1 = !{!"branch_weights", i32 1, i32 5} ; Trip count is set to 8
!1 = !{!"branch_weights", i32 1, i32 7}

llvm/test/Transforms/LoopVectorize/X86/limit-vf-by-tripcount.ll

Show First 20 Lines • Show All 79 Lines • ▼ Show 20 Lines
; CHECK: middle.block: ; CHECK: middle.block:
; CHECK-NEXT: br i1 false, label [[EXIT:%.*]], label [[VEC_EPILOG_ITER_CHECK:%.*]] ; CHECK-NEXT: br i1 false, label [[EXIT:%.*]], label [[VEC_EPILOG_ITER_CHECK:%.*]]
; CHECK: vec.epilog.iter.check: ; CHECK: vec.epilog.iter.check:
; CHECK-NEXT: br i1 false, label [[VEC_EPILOG_SCALAR_PH]], label [[VEC_EPILOG_PH]] ; CHECK-NEXT: br i1 false, label [[VEC_EPILOG_SCALAR_PH]], label [[VEC_EPILOG_PH]]
; CHECK: vec.epilog.ph: ; CHECK: vec.epilog.ph:
; CHECK-NEXT: [[VEC_EPILOG_RESUME_VAL:%.*]] = phi i64 [ 16, [[VEC_EPILOG_ITER_CHECK]] ], [ 0, [[VECTOR_MAIN_LOOP_ITER_CHECK]] ] ; CHECK-NEXT: [[VEC_EPILOG_RESUME_VAL:%.*]] = phi i64 [ 16, [[VEC_EPILOG_ITER_CHECK]] ], [ 0, [[VECTOR_MAIN_LOOP_ITER_CHECK]] ]
; CHECK-NEXT: br label [[VEC_EPILOG_VECTOR_BODY:%.*]] ; CHECK-NEXT: br label [[VEC_EPILOG_VECTOR_BODY:%.*]]
; CHECK: vec.epilog.vector.body: ; CHECK: vec.epilog.vector.body:
; CHECK-NEXT: [[INDEX2:%.*]] = phi i64 [ [[VEC_EPILOG_RESUME_VAL]], [[VEC_EPILOG_PH]] ], [ [[INDEX_NEXT4:%.*]], [[VEC_EPILOG_VECTOR_BODY]] ] ; CHECK-NEXT: [[INDEX1:%.*]] = phi i64 [ [[VEC_EPILOG_RESUME_VAL]], [[VEC_EPILOG_PH]] ], [ [[INDEX_NEXT3:%.*]], [[VEC_EPILOG_VECTOR_BODY]] ]
; CHECK-NEXT: [[TMP6:%.*]] = add i64 [[INDEX2]], 0 ; CHECK-NEXT: [[TMP6:%.*]] = add i64 [[INDEX1]], 0
; CHECK-NEXT: [[TMP7:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i64 [[TMP6]] ; CHECK-NEXT: [[TMP7:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i64 [[TMP6]]
; CHECK-NEXT: [[TMP8:%.*]] = getelementptr inbounds i8, ptr [[TMP7]], i32 0 ; CHECK-NEXT: [[TMP8:%.*]] = getelementptr inbounds i8, ptr [[TMP7]], i32 0
; CHECK-NEXT: [[WIDE_LOAD3:%.*]] = load <2 x i8>, ptr [[TMP8]], align 64 ; CHECK-NEXT: [[WIDE_LOAD2:%.*]] = load <2 x i8>, ptr [[TMP8]], align 64
; CHECK-NEXT: [[TMP9:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 [[TMP6]] ; CHECK-NEXT: [[TMP9:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 [[TMP6]]
; CHECK-NEXT: [[TMP10:%.*]] = getelementptr inbounds i8, ptr [[TMP9]], i32 0 ; CHECK-NEXT: [[TMP10:%.*]] = getelementptr inbounds i8, ptr [[TMP9]], i32 0
; CHECK-NEXT: store <2 x i8> [[WIDE_LOAD3]], ptr [[TMP10]], align 64 ; CHECK-NEXT: store <2 x i8> [[WIDE_LOAD2]], ptr [[TMP10]], align 64
; CHECK-NEXT: [[INDEX_NEXT4]] = add nuw i64 [[INDEX2]], 2 ; CHECK-NEXT: [[INDEX_NEXT3]] = add nuw i64 [[INDEX1]], 2
; CHECK-NEXT: [[TMP11:%.*]] = icmp eq i64 [[INDEX_NEXT4]], 18 ; CHECK-NEXT: [[TMP11:%.*]] = icmp eq i64 [[INDEX_NEXT3]], 18
; CHECK-NEXT: br i1 [[TMP11]], label [[VEC_EPILOG_MIDDLE_BLOCK:%.*]], label [[VEC_EPILOG_VECTOR_BODY]], !llvm.loop [[LOOP5:![0-9]+]] ; CHECK-NEXT: br i1 [[TMP11]], label [[VEC_EPILOG_MIDDLE_BLOCK:%.*]], label [[VEC_EPILOG_VECTOR_BODY]], !llvm.loop [[LOOP5:![0-9]+]]
; CHECK: vec.epilog.middle.block: ; CHECK: vec.epilog.middle.block:
; CHECK-NEXT: br i1 true, label [[EXIT]], label [[VEC_EPILOG_SCALAR_PH]] ; CHECK-NEXT: br i1 true, label [[EXIT]], label [[VEC_EPILOG_SCALAR_PH]]
; CHECK: vec.epilog.scalar.ph: ; CHECK: vec.epilog.scalar.ph:
; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 18, [[VEC_EPILOG_MIDDLE_BLOCK]] ], [ 16, [[VEC_EPILOG_ITER_CHECK]] ], [ 0, [[ITER_CHECK:%.*]] ] ; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 18, [[VEC_EPILOG_MIDDLE_BLOCK]] ], [ 16, [[VEC_EPILOG_ITER_CHECK]] ], [ 0, [[ITER_CHECK:%.*]] ]
; CHECK-NEXT: br label [[LOOP:%.*]] ; CHECK-NEXT: br label [[LOOP:%.*]]
; CHECK: loop: ; CHECK: loop:
; CHECK-NEXT: [[I:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[VEC_EPILOG_SCALAR_PH]] ], [ [[I_NEXT:%.*]], [[LOOP]] ] ; CHECK-NEXT: [[I:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[VEC_EPILOG_SCALAR_PH]] ], [ [[I_NEXT:%.*]], [[LOOP]] ]
▲ Show 20 Lines • Show All 47 Lines • ▼ Show 20 Lines
; CHECK: middle.block: ; CHECK: middle.block:
; CHECK-NEXT: br i1 false, label [[EXIT:%.*]], label [[VEC_EPILOG_ITER_CHECK:%.*]] ; CHECK-NEXT: br i1 false, label [[EXIT:%.*]], label [[VEC_EPILOG_ITER_CHECK:%.*]]
; CHECK: vec.epilog.iter.check: ; CHECK: vec.epilog.iter.check:
; CHECK-NEXT: br i1 false, label [[VEC_EPILOG_SCALAR_PH]], label [[VEC_EPILOG_PH]] ; CHECK-NEXT: br i1 false, label [[VEC_EPILOG_SCALAR_PH]], label [[VEC_EPILOG_PH]]
; CHECK: vec.epilog.ph: ; CHECK: vec.epilog.ph:
; CHECK-NEXT: [[VEC_EPILOG_RESUME_VAL:%.*]] = phi i64 [ 16, [[VEC_EPILOG_ITER_CHECK]] ], [ 0, [[VECTOR_MAIN_LOOP_ITER_CHECK]] ] ; CHECK-NEXT: [[VEC_EPILOG_RESUME_VAL:%.*]] = phi i64 [ 16, [[VEC_EPILOG_ITER_CHECK]] ], [ 0, [[VECTOR_MAIN_LOOP_ITER_CHECK]] ]
; CHECK-NEXT: br label [[VEC_EPILOG_VECTOR_BODY:%.*]] ; CHECK-NEXT: br label [[VEC_EPILOG_VECTOR_BODY:%.*]]
; CHECK: vec.epilog.vector.body: ; CHECK: vec.epilog.vector.body:
; CHECK-NEXT: [[INDEX2:%.*]] = phi i64 [ [[VEC_EPILOG_RESUME_VAL]], [[VEC_EPILOG_PH]] ], [ [[INDEX_NEXT4:%.*]], [[VEC_EPILOG_VECTOR_BODY]] ] ; CHECK-NEXT: [[INDEX1:%.*]] = phi i64 [ [[VEC_EPILOG_RESUME_VAL]], [[VEC_EPILOG_PH]] ], [ [[INDEX_NEXT3:%.*]], [[VEC_EPILOG_VECTOR_BODY]] ]
; CHECK-NEXT: [[TMP6:%.*]] = add i64 [[INDEX2]], 0 ; CHECK-NEXT: [[TMP6:%.*]] = add i64 [[INDEX1]], 0
; CHECK-NEXT: [[TMP7:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i64 [[TMP6]] ; CHECK-NEXT: [[TMP7:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i64 [[TMP6]]
; CHECK-NEXT: [[TMP8:%.*]] = getelementptr inbounds i8, ptr [[TMP7]], i32 0 ; CHECK-NEXT: [[TMP8:%.*]] = getelementptr inbounds i8, ptr [[TMP7]], i32 0
; CHECK-NEXT: [[WIDE_LOAD3:%.*]] = load <2 x i8>, ptr [[TMP8]], align 64 ; CHECK-NEXT: [[WIDE_LOAD2:%.*]] = load <2 x i8>, ptr [[TMP8]], align 64
; CHECK-NEXT: [[TMP9:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 [[TMP6]] ; CHECK-NEXT: [[TMP9:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 [[TMP6]]
; CHECK-NEXT: [[TMP10:%.*]] = getelementptr inbounds i8, ptr [[TMP9]], i32 0 ; CHECK-NEXT: [[TMP10:%.*]] = getelementptr inbounds i8, ptr [[TMP9]], i32 0
; CHECK-NEXT: store <2 x i8> [[WIDE_LOAD3]], ptr [[TMP10]], align 64 ; CHECK-NEXT: store <2 x i8> [[WIDE_LOAD2]], ptr [[TMP10]], align 64
; CHECK-NEXT: [[INDEX_NEXT4]] = add nuw i64 [[INDEX2]], 2 ; CHECK-NEXT: [[INDEX_NEXT3]] = add nuw i64 [[INDEX1]], 2
; CHECK-NEXT: [[TMP11:%.*]] = icmp eq i64 [[INDEX_NEXT4]], 18 ; CHECK-NEXT: [[TMP11:%.*]] = icmp eq i64 [[INDEX_NEXT3]], 18
; CHECK-NEXT: br i1 [[TMP11]], label [[VEC_EPILOG_MIDDLE_BLOCK:%.*]], label [[VEC_EPILOG_VECTOR_BODY]], !llvm.loop [[LOOP8:![0-9]+]] ; CHECK-NEXT: br i1 [[TMP11]], label [[VEC_EPILOG_MIDDLE_BLOCK:%.*]], label [[VEC_EPILOG_VECTOR_BODY]], !llvm.loop [[LOOP8:![0-9]+]]
; CHECK: vec.epilog.middle.block: ; CHECK: vec.epilog.middle.block:
; CHECK-NEXT: br i1 false, label [[EXIT]], label [[VEC_EPILOG_SCALAR_PH]] ; CHECK-NEXT: br i1 false, label [[EXIT]], label [[VEC_EPILOG_SCALAR_PH]]
; CHECK: vec.epilog.scalar.ph: ; CHECK: vec.epilog.scalar.ph:
; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 18, [[VEC_EPILOG_MIDDLE_BLOCK]] ], [ 16, [[VEC_EPILOG_ITER_CHECK]] ], [ 0, [[ITER_CHECK:%.*]] ] ; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 18, [[VEC_EPILOG_MIDDLE_BLOCK]] ], [ 16, [[VEC_EPILOG_ITER_CHECK]] ], [ 0, [[ITER_CHECK:%.*]] ]
; CHECK-NEXT: br label [[LOOP:%.*]] ; CHECK-NEXT: br label [[LOOP:%.*]]
; CHECK: loop: ; CHECK: loop:
; CHECK-NEXT: [[I:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[VEC_EPILOG_SCALAR_PH]] ], [ [[I_NEXT:%.*]], [[LOOP]] ] ; CHECK-NEXT: [[I:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[VEC_EPILOG_SCALAR_PH]] ], [ [[I_NEXT:%.*]], [[LOOP]] ]
Show All 28 Lines
; CHECK-LABEL: @test_tc_20( ; CHECK-LABEL: @test_tc_20(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]] ; CHECK-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]]
; CHECK: vector.ph: ; CHECK: vector.ph:
; CHECK-NEXT: br label [[VECTOR_BODY:%.*]] ; CHECK-NEXT: br label [[VECTOR_BODY:%.*]]
; CHECK: vector.body: ; CHECK: vector.body:
; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ] ; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[TMP0:%.*]] = add i64 [[INDEX]], 0 ; CHECK-NEXT: [[TMP0:%.*]] = add i64 [[INDEX]], 0
; CHECK-NEXT: [[TMP1:%.*]] = getelementptr inbounds i8, ptr [[SRC:%.*]], i64 [[TMP0]] ; CHECK-NEXT: [[TMP1:%.*]] = add i64 [[INDEX]], 4
; CHECK-NEXT: [[TMP2:%.*]] = getelementptr inbounds i8, ptr [[TMP1]], i32 0 ; CHECK-NEXT: [[TMP2:%.*]] = getelementptr inbounds i8, ptr [[SRC:%.*]], i64 [[TMP0]]
; CHECK-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i8>, ptr [[TMP2]], align 64 ; CHECK-NEXT: [[TMP3:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i64 [[TMP1]]
; CHECK-NEXT: [[TMP3:%.*]] = getelementptr inbounds i8, ptr [[DST:%.*]], i64 [[TMP0]] ; CHECK-NEXT: [[TMP4:%.*]] = getelementptr inbounds i8, ptr [[TMP2]], i32 0
; CHECK-NEXT: [[TMP4:%.*]] = getelementptr inbounds i8, ptr [[TMP3]], i32 0 ; CHECK-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i8>, ptr [[TMP4]], align 64
; CHECK-NEXT: store <4 x i8> [[WIDE_LOAD]], ptr [[TMP4]], align 64 ; CHECK-NEXT: [[TMP5:%.*]] = getelementptr inbounds i8, ptr [[TMP2]], i32 4
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 4 ; CHECK-NEXT: [[WIDE_LOAD1:%.*]] = load <4 x i8>, ptr [[TMP5]], align 64
; CHECK-NEXT: [[TMP5:%.*]] = icmp eq i64 [[INDEX_NEXT]], 20 ; CHECK-NEXT: [[TMP6:%.*]] = getelementptr inbounds i8, ptr [[DST:%.*]], i64 [[TMP0]]
; CHECK-NEXT: br i1 [[TMP5]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP10:![0-9]+]] ; CHECK-NEXT: [[TMP7:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 [[TMP1]]
; CHECK-NEXT: [[TMP8:%.*]] = getelementptr inbounds i8, ptr [[TMP6]], i32 0
; CHECK-NEXT: store <4 x i8> [[WIDE_LOAD]], ptr [[TMP8]], align 64
; CHECK-NEXT: [[TMP9:%.*]] = getelementptr inbounds i8, ptr [[TMP6]], i32 4
; CHECK-NEXT: store <4 x i8> [[WIDE_LOAD1]], ptr [[TMP9]], align 64
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 8
; CHECK-NEXT: [[TMP10:%.*]] = icmp eq i64 [[INDEX_NEXT]], 16
; CHECK-NEXT: br i1 [[TMP10]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP10:![0-9]+]]
; CHECK: middle.block: ; CHECK: middle.block:
; CHECK-NEXT: br i1 true, label [[EXIT:%.*]], label [[SCALAR_PH]] ; CHECK-NEXT: br i1 false, label [[EXIT:%.*]], label [[SCALAR_PH]]
; CHECK: scalar.ph: ; CHECK: scalar.ph:
; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 20, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ] ; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 16, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]
; CHECK-NEXT: br label [[LOOP:%.*]] ; CHECK-NEXT: br label [[LOOP:%.*]]
; CHECK: loop: ; CHECK: loop:
; CHECK-NEXT: [[I:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[I_NEXT:%.*]], [[LOOP]] ] ; CHECK-NEXT: [[I:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[I_NEXT:%.*]], [[LOOP]] ]
; CHECK-NEXT: [[LDADDR:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i64 [[I]] ; CHECK-NEXT: [[LDADDR:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i64 [[I]]
; CHECK-NEXT: [[VAL:%.*]] = load i8, ptr [[LDADDR]], align 64 ; CHECK-NEXT: [[VAL:%.*]] = load i8, ptr [[LDADDR]], align 64
; CHECK-NEXT: [[STADDR:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 [[I]] ; CHECK-NEXT: [[STADDR:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 [[I]]
; CHECK-NEXT: store i8 [[VAL]], ptr [[STADDR]], align 64 ; CHECK-NEXT: store i8 [[VAL]], ptr [[STADDR]], align 64
; CHECK-NEXT: [[I_NEXT]] = add i64 [[I]], 1 ; CHECK-NEXT: [[I_NEXT]] = add i64 [[I]], 1
Show All 23 Lines
; CHECK-LABEL: @limit_main_loop_vf_to_avoid_dead_main_vector_loop( ; CHECK-LABEL: @limit_main_loop_vf_to_avoid_dead_main_vector_loop(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]] ; CHECK-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]]
; CHECK: vector.ph: ; CHECK: vector.ph:
; CHECK-NEXT: br label [[VECTOR_BODY:%.*]] ; CHECK-NEXT: br label [[VECTOR_BODY:%.*]]
; CHECK: vector.body: ; CHECK: vector.body:
; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ] ; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[TMP0:%.*]] = add i64 [[INDEX]], 0 ; CHECK-NEXT: [[TMP0:%.*]] = add i64 [[INDEX]], 0
; CHECK-NEXT: [[TMP1:%.*]] = getelementptr inbounds [3 x i8], ptr [[SRC:%.*]], i64 [[TMP0]], i64 0 ; CHECK-NEXT: [[TMP1:%.*]] = add i64 [[INDEX]], 8
; CHECK-NEXT: [[TMP2:%.*]] = getelementptr inbounds i8, ptr [[TMP1]], i32 0 ; CHECK-NEXT: [[TMP2:%.*]] = getelementptr inbounds [3 x i8], ptr [[SRC:%.*]], i64 [[TMP0]], i64 0
; CHECK-NEXT: [[WIDE_VEC:%.*]] = load <24 x i8>, ptr [[TMP2]], align 1 ; CHECK-NEXT: [[TMP3:%.*]] = getelementptr inbounds [3 x i8], ptr [[SRC]], i64 [[TMP1]], i64 0
; CHECK-NEXT: [[TMP4:%.*]] = getelementptr inbounds i8, ptr [[TMP2]], i32 0
; CHECK-NEXT: [[TMP5:%.*]] = getelementptr inbounds i8, ptr [[TMP3]], i32 0
; CHECK-NEXT: [[WIDE_VEC:%.*]] = load <24 x i8>, ptr [[TMP4]], align 1
; CHECK-NEXT: [[WIDE_VEC1:%.*]] = load <24 x i8>, ptr [[TMP5]], align 1
; CHECK-NEXT: [[STRIDED_VEC:%.*]] = shufflevector <24 x i8> [[WIDE_VEC]], <24 x i8> poison, <8 x i32> <i32 0, i32 3, i32 6, i32 9, i32 12, i32 15, i32 18, i32 21> ; CHECK-NEXT: [[STRIDED_VEC:%.*]] = shufflevector <24 x i8> [[WIDE_VEC]], <24 x i8> poison, <8 x i32> <i32 0, i32 3, i32 6, i32 9, i32 12, i32 15, i32 18, i32 21>
; CHECK-NEXT: [[TMP3:%.*]] = getelementptr inbounds i8, ptr [[DST:%.*]], i64 [[TMP0]] ; CHECK-NEXT: [[STRIDED_VEC2:%.*]] = shufflevector <24 x i8> [[WIDE_VEC1]], <24 x i8> poison, <8 x i32> <i32 0, i32 3, i32 6, i32 9, i32 12, i32 15, i32 18, i32 21>
; CHECK-NEXT: [[TMP4:%.*]] = getelementptr inbounds i8, ptr [[TMP3]], i32 0 ; CHECK-NEXT: [[TMP6:%.*]] = getelementptr inbounds i8, ptr [[DST:%.*]], i64 [[TMP0]]
; CHECK-NEXT: store <8 x i8> [[STRIDED_VEC]], ptr [[TMP4]], align 1 ; CHECK-NEXT: [[TMP7:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 [[TMP1]]
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 8 ; CHECK-NEXT: [[TMP8:%.*]] = getelementptr inbounds i8, ptr [[TMP6]], i32 0
; CHECK-NEXT: [[TMP5:%.*]] = icmp eq i64 [[INDEX_NEXT]], 24 ; CHECK-NEXT: store <8 x i8> [[STRIDED_VEC]], ptr [[TMP8]], align 1
; CHECK-NEXT: br i1 [[TMP5]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP12:![0-9]+]] ; CHECK-NEXT: [[TMP9:%.*]] = getelementptr inbounds i8, ptr [[TMP6]], i32 8
; CHECK-NEXT: store <8 x i8> [[STRIDED_VEC2]], ptr [[TMP9]], align 1
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 16
; CHECK-NEXT: [[TMP10:%.*]] = icmp eq i64 [[INDEX_NEXT]], 16
; CHECK-NEXT: br i1 [[TMP10]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP12:![0-9]+]]
; CHECK: middle.block: ; CHECK: middle.block:
; CHECK-NEXT: br label [[SCALAR_PH]] ; CHECK-NEXT: br label [[SCALAR_PH]]
; CHECK: scalar.ph: ; CHECK: scalar.ph:
; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 24, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ] ; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 16, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]
; CHECK-NEXT: br label [[LOOP:%.*]] ; CHECK-NEXT: br label [[LOOP:%.*]]
; CHECK: loop: ; CHECK: loop:
; CHECK-NEXT: [[IV:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[IV_NEXT:%.*]], [[LOOP]] ] ; CHECK-NEXT: [[IV:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[IV_NEXT:%.*]], [[LOOP]] ]
; CHECK-NEXT: [[GEP_SRC:%.*]] = getelementptr inbounds [3 x i8], ptr [[SRC]], i64 [[IV]], i64 0 ; CHECK-NEXT: [[GEP_SRC:%.*]] = getelementptr inbounds [3 x i8], ptr [[SRC]], i64 [[IV]], i64 0
; CHECK-NEXT: [[L:%.*]] = load i8, ptr [[GEP_SRC]], align 1 ; CHECK-NEXT: [[L:%.*]] = load i8, ptr [[GEP_SRC]], align 1
; CHECK-NEXT: [[GEP_DST:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 [[IV]] ; CHECK-NEXT: [[GEP_DST:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 [[IV]]
; CHECK-NEXT: store i8 [[L]], ptr [[GEP_DST]], align 1 ; CHECK-NEXT: store i8 [[L]], ptr [[GEP_DST]], align 1
; CHECK-NEXT: [[IV_NEXT]] = add nuw nsw i64 [[IV]], 1 ; CHECK-NEXT: [[IV_NEXT]] = add nuw nsw i64 [[IV]], 1
Show All 21 Lines

llvm/test/Transforms/LoopVectorize/X86/load-deref-pred.ll

Show First 20 Lines • Show All 2,501 Lines • ▼ Show 20 Lines
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[ALLOCA:%.*]] = alloca [105 x i32], align 4 ; CHECK-NEXT: [[ALLOCA:%.*]] = alloca [105 x i32], align 4
; CHECK-NEXT: call void @init(ptr [[ALLOCA]]) ; CHECK-NEXT: call void @init(ptr [[ALLOCA]])
; CHECK-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]] ; CHECK-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]]
; CHECK: vector.ph: ; CHECK: vector.ph:
; CHECK-NEXT: br label [[VECTOR_BODY:%.*]] ; CHECK-NEXT: br label [[VECTOR_BODY:%.*]]
; CHECK: vector.body: ; CHECK: vector.body:
; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ] ; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[VEC_PHI:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP29:%.*]], [[VECTOR_BODY]] ] ; CHECK-NEXT: [[VEC_PHI:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP116:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[VEC_PHI1:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP117:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[VEC_PHI2:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP118:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[VEC_PHI3:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP119:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[OFFSET_IDX:%.*]] = mul i64 [[INDEX]], 3 ; CHECK-NEXT: [[OFFSET_IDX:%.*]] = mul i64 [[INDEX]], 3
; CHECK-NEXT: [[TMP0:%.*]] = add i64 [[OFFSET_IDX]], 0 ; CHECK-NEXT: [[TMP0:%.*]] = add i64 [[OFFSET_IDX]], 0
; CHECK-NEXT: [[TMP1:%.*]] = add i64 [[OFFSET_IDX]], 3 ; CHECK-NEXT: [[TMP1:%.*]] = add i64 [[OFFSET_IDX]], 3
; CHECK-NEXT: [[TMP2:%.*]] = add i64 [[OFFSET_IDX]], 6 ; CHECK-NEXT: [[TMP2:%.*]] = add i64 [[OFFSET_IDX]], 6
; CHECK-NEXT: [[TMP3:%.*]] = add i64 [[OFFSET_IDX]], 9 ; CHECK-NEXT: [[TMP3:%.*]] = add i64 [[OFFSET_IDX]], 9
; CHECK-NEXT: [[TMP4:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE:%.*]], i64 [[TMP0]] ; CHECK-NEXT: [[TMP4:%.*]] = add i64 [[OFFSET_IDX]], 12
; CHECK-NEXT: [[TMP5:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP1]] ; CHECK-NEXT: [[TMP5:%.*]] = add i64 [[OFFSET_IDX]], 15
; CHECK-NEXT: [[TMP6:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP2]] ; CHECK-NEXT: [[TMP6:%.*]] = add i64 [[OFFSET_IDX]], 18
; CHECK-NEXT: [[TMP7:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP3]] ; CHECK-NEXT: [[TMP7:%.*]] = add i64 [[OFFSET_IDX]], 21
; CHECK-NEXT: [[TMP8:%.*]] = load i1, ptr [[TMP4]], align 1 ; CHECK-NEXT: [[TMP8:%.*]] = add i64 [[OFFSET_IDX]], 24
; CHECK-NEXT: [[TMP9:%.*]] = load i1, ptr [[TMP5]], align 1 ; CHECK-NEXT: [[TMP9:%.*]] = add i64 [[OFFSET_IDX]], 27
; CHECK-NEXT: [[TMP10:%.*]] = load i1, ptr [[TMP6]], align 1 ; CHECK-NEXT: [[TMP10:%.*]] = add i64 [[OFFSET_IDX]], 30
; CHECK-NEXT: [[TMP11:%.*]] = load i1, ptr [[TMP7]], align 1 ; CHECK-NEXT: [[TMP11:%.*]] = add i64 [[OFFSET_IDX]], 33
; CHECK-NEXT: [[TMP12:%.*]] = insertelement <4 x i1> poison, i1 [[TMP8]], i32 0 ; CHECK-NEXT: [[TMP12:%.*]] = add i64 [[OFFSET_IDX]], 36
; CHECK-NEXT: [[TMP13:%.*]] = insertelement <4 x i1> [[TMP12]], i1 [[TMP9]], i32 1 ; CHECK-NEXT: [[TMP13:%.*]] = add i64 [[OFFSET_IDX]], 39
; CHECK-NEXT: [[TMP14:%.*]] = insertelement <4 x i1> [[TMP13]], i1 [[TMP10]], i32 2 ; CHECK-NEXT: [[TMP14:%.*]] = add i64 [[OFFSET_IDX]], 42
; CHECK-NEXT: [[TMP15:%.*]] = insertelement <4 x i1> [[TMP14]], i1 [[TMP11]], i32 3 ; CHECK-NEXT: [[TMP15:%.*]] = add i64 [[OFFSET_IDX]], 45
; CHECK-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP0]] ; CHECK-NEXT: [[TMP16:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE:%.*]], i64 [[TMP0]]
; CHECK-NEXT: [[TMP17:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP1]] ; CHECK-NEXT: [[TMP17:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP1]]
; CHECK-NEXT: [[TMP18:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP2]] ; CHECK-NEXT: [[TMP18:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP2]]
; CHECK-NEXT: [[TMP19:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP3]] ; CHECK-NEXT: [[TMP19:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP3]]
; CHECK-NEXT: [[TMP20:%.*]] = load i32, ptr [[TMP16]], align 4 ; CHECK-NEXT: [[TMP20:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP4]]
; CHECK-NEXT: [[TMP21:%.*]] = load i32, ptr [[TMP17]], align 4 ; CHECK-NEXT: [[TMP21:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP5]]
; CHECK-NEXT: [[TMP22:%.*]] = load i32, ptr [[TMP18]], align 4 ; CHECK-NEXT: [[TMP22:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP6]]
; CHECK-NEXT: [[TMP23:%.*]] = load i32, ptr [[TMP19]], align 4 ; CHECK-NEXT: [[TMP23:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP7]]
; CHECK-NEXT: [[TMP24:%.*]] = insertelement <4 x i32> poison, i32 [[TMP20]], i32 0 ; CHECK-NEXT: [[TMP24:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP8]]
; CHECK-NEXT: [[TMP25:%.*]] = insertelement <4 x i32> [[TMP24]], i32 [[TMP21]], i32 1 ; CHECK-NEXT: [[TMP25:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP9]]
; CHECK-NEXT: [[TMP26:%.*]] = insertelement <4 x i32> [[TMP25]], i32 [[TMP22]], i32 2 ; CHECK-NEXT: [[TMP26:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP10]]
; CHECK-NEXT: [[TMP27:%.*]] = insertelement <4 x i32> [[TMP26]], i32 [[TMP23]], i32 3 ; CHECK-NEXT: [[TMP27:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP11]]
; CHECK-NEXT: [[TMP28:%.*]] = xor <4 x i1> [[TMP15]], <i1 true, i1 true, i1 true, i1 true> ; CHECK-NEXT: [[TMP28:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP12]]
; CHECK-NEXT: [[PREDPHI:%.*]] = select <4 x i1> [[TMP15]], <4 x i32> [[TMP27]], <4 x i32> zeroinitializer ; CHECK-NEXT: [[TMP29:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP13]]
; CHECK-NEXT: [[TMP29]] = add <4 x i32> [[VEC_PHI]], [[PREDPHI]] ; CHECK-NEXT: [[TMP30:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP14]]
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 4 ; CHECK-NEXT: [[TMP31:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP15]]
; CHECK-NEXT: [[TMP30:%.*]] = icmp eq i64 [[INDEX_NEXT]], 32 ; CHECK-NEXT: [[TMP32:%.*]] = load i1, ptr [[TMP16]], align 1
; CHECK-NEXT: br i1 [[TMP30]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP30:![0-9]+]] ; CHECK-NEXT: [[TMP33:%.*]] = load i1, ptr [[TMP17]], align 1
; CHECK-NEXT: [[TMP34:%.*]] = load i1, ptr [[TMP18]], align 1
; CHECK-NEXT: [[TMP35:%.*]] = load i1, ptr [[TMP19]], align 1
; CHECK-NEXT: [[TMP36:%.*]] = insertelement <4 x i1> poison, i1 [[TMP32]], i32 0
; CHECK-NEXT: [[TMP37:%.*]] = insertelement <4 x i1> [[TMP36]], i1 [[TMP33]], i32 1
; CHECK-NEXT: [[TMP38:%.*]] = insertelement <4 x i1> [[TMP37]], i1 [[TMP34]], i32 2
; CHECK-NEXT: [[TMP39:%.*]] = insertelement <4 x i1> [[TMP38]], i1 [[TMP35]], i32 3
; CHECK-NEXT: [[TMP40:%.*]] = load i1, ptr [[TMP20]], align 1
; CHECK-NEXT: [[TMP41:%.*]] = load i1, ptr [[TMP21]], align 1
; CHECK-NEXT: [[TMP42:%.*]] = load i1, ptr [[TMP22]], align 1
; CHECK-NEXT: [[TMP43:%.*]] = load i1, ptr [[TMP23]], align 1
; CHECK-NEXT: [[TMP44:%.*]] = insertelement <4 x i1> poison, i1 [[TMP40]], i32 0
; CHECK-NEXT: [[TMP45:%.*]] = insertelement <4 x i1> [[TMP44]], i1 [[TMP41]], i32 1
; CHECK-NEXT: [[TMP46:%.*]] = insertelement <4 x i1> [[TMP45]], i1 [[TMP42]], i32 2
; CHECK-NEXT: [[TMP47:%.*]] = insertelement <4 x i1> [[TMP46]], i1 [[TMP43]], i32 3
; CHECK-NEXT: [[TMP48:%.*]] = load i1, ptr [[TMP24]], align 1
; CHECK-NEXT: [[TMP49:%.*]] = load i1, ptr [[TMP25]], align 1
; CHECK-NEXT: [[TMP50:%.*]] = load i1, ptr [[TMP26]], align 1
; CHECK-NEXT: [[TMP51:%.*]] = load i1, ptr [[TMP27]], align 1
; CHECK-NEXT: [[TMP52:%.*]] = insertelement <4 x i1> poison, i1 [[TMP48]], i32 0
; CHECK-NEXT: [[TMP53:%.*]] = insertelement <4 x i1> [[TMP52]], i1 [[TMP49]], i32 1
; CHECK-NEXT: [[TMP54:%.*]] = insertelement <4 x i1> [[TMP53]], i1 [[TMP50]], i32 2
; CHECK-NEXT: [[TMP55:%.*]] = insertelement <4 x i1> [[TMP54]], i1 [[TMP51]], i32 3
; CHECK-NEXT: [[TMP56:%.*]] = load i1, ptr [[TMP28]], align 1
; CHECK-NEXT: [[TMP57:%.*]] = load i1, ptr [[TMP29]], align 1
; CHECK-NEXT: [[TMP58:%.*]] = load i1, ptr [[TMP30]], align 1
; CHECK-NEXT: [[TMP59:%.*]] = load i1, ptr [[TMP31]], align 1
; CHECK-NEXT: [[TMP60:%.*]] = insertelement <4 x i1> poison, i1 [[TMP56]], i32 0
; CHECK-NEXT: [[TMP61:%.*]] = insertelement <4 x i1> [[TMP60]], i1 [[TMP57]], i32 1
; CHECK-NEXT: [[TMP62:%.*]] = insertelement <4 x i1> [[TMP61]], i1 [[TMP58]], i32 2
; CHECK-NEXT: [[TMP63:%.*]] = insertelement <4 x i1> [[TMP62]], i1 [[TMP59]], i32 3
; CHECK-NEXT: [[TMP64:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP0]]
; CHECK-NEXT: [[TMP65:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP1]]
; CHECK-NEXT: [[TMP66:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP2]]
; CHECK-NEXT: [[TMP67:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP3]]
; CHECK-NEXT: [[TMP68:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP4]]
; CHECK-NEXT: [[TMP69:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP5]]
; CHECK-NEXT: [[TMP70:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP6]]
; CHECK-NEXT: [[TMP71:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP7]]
; CHECK-NEXT: [[TMP72:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP8]]
; CHECK-NEXT: [[TMP73:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP9]]
; CHECK-NEXT: [[TMP74:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP10]]
; CHECK-NEXT: [[TMP75:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP11]]
; CHECK-NEXT: [[TMP76:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP12]]
; CHECK-NEXT: [[TMP77:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP13]]
; CHECK-NEXT: [[TMP78:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP14]]
; CHECK-NEXT: [[TMP79:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP15]]
; CHECK-NEXT: [[TMP80:%.*]] = load i32, ptr [[TMP64]], align 4
; CHECK-NEXT: [[TMP81:%.*]] = load i32, ptr [[TMP65]], align 4
; CHECK-NEXT: [[TMP82:%.*]] = load i32, ptr [[TMP66]], align 4
; CHECK-NEXT: [[TMP83:%.*]] = load i32, ptr [[TMP67]], align 4
; CHECK-NEXT: [[TMP84:%.*]] = insertelement <4 x i32> poison, i32 [[TMP80]], i32 0
; CHECK-NEXT: [[TMP85:%.*]] = insertelement <4 x i32> [[TMP84]], i32 [[TMP81]], i32 1
; CHECK-NEXT: [[TMP86:%.*]] = insertelement <4 x i32> [[TMP85]], i32 [[TMP82]], i32 2
; CHECK-NEXT: [[TMP87:%.*]] = insertelement <4 x i32> [[TMP86]], i32 [[TMP83]], i32 3
; CHECK-NEXT: [[TMP88:%.*]] = load i32, ptr [[TMP68]], align 4
; CHECK-NEXT: [[TMP89:%.*]] = load i32, ptr [[TMP69]], align 4
; CHECK-NEXT: [[TMP90:%.*]] = load i32, ptr [[TMP70]], align 4
; CHECK-NEXT: [[TMP91:%.*]] = load i32, ptr [[TMP71]], align 4
; CHECK-NEXT: [[TMP92:%.*]] = insertelement <4 x i32> poison, i32 [[TMP88]], i32 0
; CHECK-NEXT: [[TMP93:%.*]] = insertelement <4 x i32> [[TMP92]], i32 [[TMP89]], i32 1
; CHECK-NEXT: [[TMP94:%.*]] = insertelement <4 x i32> [[TMP93]], i32 [[TMP90]], i32 2
; CHECK-NEXT: [[TMP95:%.*]] = insertelement <4 x i32> [[TMP94]], i32 [[TMP91]], i32 3
; CHECK-NEXT: [[TMP96:%.*]] = load i32, ptr [[TMP72]], align 4
; CHECK-NEXT: [[TMP97:%.*]] = load i32, ptr [[TMP73]], align 4
; CHECK-NEXT: [[TMP98:%.*]] = load i32, ptr [[TMP74]], align 4
; CHECK-NEXT: [[TMP99:%.*]] = load i32, ptr [[TMP75]], align 4
; CHECK-NEXT: [[TMP100:%.*]] = insertelement <4 x i32> poison, i32 [[TMP96]], i32 0
; CHECK-NEXT: [[TMP101:%.*]] = insertelement <4 x i32> [[TMP100]], i32 [[TMP97]], i32 1
; CHECK-NEXT: [[TMP102:%.*]] = insertelement <4 x i32> [[TMP101]], i32 [[TMP98]], i32 2
; CHECK-NEXT: [[TMP103:%.*]] = insertelement <4 x i32> [[TMP102]], i32 [[TMP99]], i32 3
; CHECK-NEXT: [[TMP104:%.*]] = load i32, ptr [[TMP76]], align 4
; CHECK-NEXT: [[TMP105:%.*]] = load i32, ptr [[TMP77]], align 4
; CHECK-NEXT: [[TMP106:%.*]] = load i32, ptr [[TMP78]], align 4
; CHECK-NEXT: [[TMP107:%.*]] = load i32, ptr [[TMP79]], align 4
; CHECK-NEXT: [[TMP108:%.*]] = insertelement <4 x i32> poison, i32 [[TMP104]], i32 0
; CHECK-NEXT: [[TMP109:%.*]] = insertelement <4 x i32> [[TMP108]], i32 [[TMP105]], i32 1
; CHECK-NEXT: [[TMP110:%.*]] = insertelement <4 x i32> [[TMP109]], i32 [[TMP106]], i32 2
; CHECK-NEXT: [[TMP111:%.*]] = insertelement <4 x i32> [[TMP110]], i32 [[TMP107]], i32 3
; CHECK-NEXT: [[TMP112:%.*]] = xor <4 x i1> [[TMP39]], <i1 true, i1 true, i1 true, i1 true>
; CHECK-NEXT: [[TMP113:%.*]] = xor <4 x i1> [[TMP47]], <i1 true, i1 true, i1 true, i1 true>
; CHECK-NEXT: [[TMP114:%.*]] = xor <4 x i1> [[TMP55]], <i1 true, i1 true, i1 true, i1 true>
; CHECK-NEXT: [[TMP115:%.*]] = xor <4 x i1> [[TMP63]], <i1 true, i1 true, i1 true, i1 true>
; CHECK-NEXT: [[PREDPHI:%.*]] = select <4 x i1> [[TMP39]], <4 x i32> [[TMP87]], <4 x i32> zeroinitializer
; CHECK-NEXT: [[PREDPHI4:%.*]] = select <4 x i1> [[TMP47]], <4 x i32> [[TMP95]], <4 x i32> zeroinitializer
; CHECK-NEXT: [[PREDPHI5:%.*]] = select <4 x i1> [[TMP55]], <4 x i32> [[TMP103]], <4 x i32> zeroinitializer
; CHECK-NEXT: [[PREDPHI6:%.*]] = select <4 x i1> [[TMP63]], <4 x i32> [[TMP111]], <4 x i32> zeroinitializer
; CHECK-NEXT: [[TMP116]] = add <4 x i32> [[VEC_PHI]], [[PREDPHI]]
; CHECK-NEXT: [[TMP117]] = add <4 x i32> [[VEC_PHI1]], [[PREDPHI4]]
; CHECK-NEXT: [[TMP118]] = add <4 x i32> [[VEC_PHI2]], [[PREDPHI5]]
; CHECK-NEXT: [[TMP119]] = add <4 x i32> [[VEC_PHI3]], [[PREDPHI6]]
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 16
; CHECK-NEXT: [[TMP120:%.*]] = icmp eq i64 [[INDEX_NEXT]], 32
; CHECK-NEXT: br i1 [[TMP120]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP30:![0-9]+]]
; CHECK: middle.block: ; CHECK: middle.block:
; CHECK-NEXT: [[TMP31:%.*]] = call i32 @llvm.vector.reduce.add.v4i32(<4 x i32> [[TMP29]]) ; CHECK-NEXT: [[BIN_RDX:%.*]] = add <4 x i32> [[TMP117]], [[TMP116]]
; CHECK-NEXT: [[BIN_RDX7:%.*]] = add <4 x i32> [[TMP118]], [[BIN_RDX]]
; CHECK-NEXT: [[BIN_RDX8:%.*]] = add <4 x i32> [[TMP119]], [[BIN_RDX7]]
; CHECK-NEXT: [[TMP121:%.*]] = call i32 @llvm.vector.reduce.add.v4i32(<4 x i32> [[BIN_RDX8]])
; CHECK-NEXT: br i1 false, label [[LOOP_EXIT:%.*]], label [[SCALAR_PH]] ; CHECK-NEXT: br i1 false, label [[LOOP_EXIT:%.*]], label [[SCALAR_PH]]
; CHECK: scalar.ph: ; CHECK: scalar.ph:
; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 96, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ] ; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 96, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]
; CHECK-NEXT: [[BC_MERGE_RDX:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[TMP31]], [[MIDDLE_BLOCK]] ] ; CHECK-NEXT: [[BC_MERGE_RDX:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[TMP121]], [[MIDDLE_BLOCK]] ]
; CHECK-NEXT: br label [[LOOP:%.*]] ; CHECK-NEXT: br label [[LOOP:%.*]]
; CHECK: loop: ; CHECK: loop:
; CHECK-NEXT: [[IV:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[IV_NEXT:%.*]], [[LATCH:%.*]] ] ; CHECK-NEXT: [[IV:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[IV_NEXT:%.*]], [[LATCH:%.*]] ]
; CHECK-NEXT: [[ACCUM:%.*]] = phi i32 [ [[BC_MERGE_RDX]], [[SCALAR_PH]] ], [ [[ACCUM_NEXT:%.*]], [[LATCH]] ] ; CHECK-NEXT: [[ACCUM:%.*]] = phi i32 [ [[BC_MERGE_RDX]], [[SCALAR_PH]] ], [ [[ACCUM_NEXT:%.*]], [[LATCH]] ]
; CHECK-NEXT: [[IV_NEXT]] = add i64 [[IV]], 3 ; CHECK-NEXT: [[IV_NEXT]] = add i64 [[IV]], 3
; CHECK-NEXT: [[TEST_ADDR:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[IV]] ; CHECK-NEXT: [[TEST_ADDR:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[IV]]
; CHECK-NEXT: [[EARLYCND:%.*]] = load i1, ptr [[TEST_ADDR]], align 1 ; CHECK-NEXT: [[EARLYCND:%.*]] = load i1, ptr [[TEST_ADDR]], align 1
; CHECK-NEXT: br i1 [[EARLYCND]], label [[PRED:%.*]], label [[LATCH]] ; CHECK-NEXT: br i1 [[EARLYCND]], label [[PRED:%.*]], label [[LATCH]]
; CHECK: pred: ; CHECK: pred:
; CHECK-NEXT: [[ADDR:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[IV]] ; CHECK-NEXT: [[ADDR:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[IV]]
; CHECK-NEXT: [[VAL:%.*]] = load i32, ptr [[ADDR]], align 4 ; CHECK-NEXT: [[VAL:%.*]] = load i32, ptr [[ADDR]], align 4
; CHECK-NEXT: br label [[LATCH]] ; CHECK-NEXT: br label [[LATCH]]
; CHECK: latch: ; CHECK: latch:
; CHECK-NEXT: [[VAL_PHI:%.*]] = phi i32 [ 0, [[LOOP]] ], [ [[VAL]], [[PRED]] ] ; CHECK-NEXT: [[VAL_PHI:%.*]] = phi i32 [ 0, [[LOOP]] ], [ [[VAL]], [[PRED]] ]
; CHECK-NEXT: [[ACCUM_NEXT]] = add i32 [[ACCUM]], [[VAL_PHI]] ; CHECK-NEXT: [[ACCUM_NEXT]] = add i32 [[ACCUM]], [[VAL_PHI]]
; CHECK-NEXT: [[EXIT:%.*]] = icmp ugt i64 [[IV]], 100 ; CHECK-NEXT: [[EXIT:%.*]] = icmp ugt i64 [[IV]], 100
; CHECK-NEXT: br i1 [[EXIT]], label [[LOOP_EXIT]], label [[LOOP]], !llvm.loop [[LOOP31:![0-9]+]] ; CHECK-NEXT: br i1 [[EXIT]], label [[LOOP_EXIT]], label [[LOOP]], !llvm.loop [[LOOP31:![0-9]+]]
; CHECK: loop_exit: ; CHECK: loop_exit:
; CHECK-NEXT: [[ACCUM_NEXT_LCSSA:%.*]] = phi i32 [ [[ACCUM_NEXT]], [[LATCH]] ], [ [[TMP31]], [[MIDDLE_BLOCK]] ] ; CHECK-NEXT: [[ACCUM_NEXT_LCSSA:%.*]] = phi i32 [ [[ACCUM_NEXT]], [[LATCH]] ], [ [[TMP121]], [[MIDDLE_BLOCK]] ]
; CHECK-NEXT: ret i32 [[ACCUM_NEXT_LCSSA]] ; CHECK-NEXT: ret i32 [[ACCUM_NEXT_LCSSA]]
; ;
entry: entry:
%alloca = alloca [105 x i32] %alloca = alloca [105 x i32]
call void @init(ptr %alloca) call void @init(ptr %alloca)
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %latch ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %latch ]
Show All 21 Lines
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[ALLOCA:%.*]] = alloca [108 x i32], align 4 ; CHECK-NEXT: [[ALLOCA:%.*]] = alloca [108 x i32], align 4
; CHECK-NEXT: call void @init(ptr [[ALLOCA]]) ; CHECK-NEXT: call void @init(ptr [[ALLOCA]])
; CHECK-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]] ; CHECK-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]]
; CHECK: vector.ph: ; CHECK: vector.ph:
; CHECK-NEXT: br label [[VECTOR_BODY:%.*]] ; CHECK-NEXT: br label [[VECTOR_BODY:%.*]]
; CHECK: vector.body: ; CHECK: vector.body:
; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ] ; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[VEC_PHI:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP29:%.*]], [[VECTOR_BODY]] ] ; CHECK-NEXT: [[VEC_PHI:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP87:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[VEC_PHI1:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP88:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[VEC_PHI2:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP89:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[OFFSET_IDX:%.*]] = mul i64 [[INDEX]], 4 ; CHECK-NEXT: [[OFFSET_IDX:%.*]] = mul i64 [[INDEX]], 4
; CHECK-NEXT: [[TMP0:%.*]] = add i64 [[OFFSET_IDX]], 0 ; CHECK-NEXT: [[TMP0:%.*]] = add i64 [[OFFSET_IDX]], 0
; CHECK-NEXT: [[TMP1:%.*]] = add i64 [[OFFSET_IDX]], 4 ; CHECK-NEXT: [[TMP1:%.*]] = add i64 [[OFFSET_IDX]], 4
; CHECK-NEXT: [[TMP2:%.*]] = add i64 [[OFFSET_IDX]], 8 ; CHECK-NEXT: [[TMP2:%.*]] = add i64 [[OFFSET_IDX]], 8
; CHECK-NEXT: [[TMP3:%.*]] = add i64 [[OFFSET_IDX]], 12 ; CHECK-NEXT: [[TMP3:%.*]] = add i64 [[OFFSET_IDX]], 12
; CHECK-NEXT: [[TMP4:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE:%.*]], i64 [[TMP0]] ; CHECK-NEXT: [[TMP4:%.*]] = add i64 [[OFFSET_IDX]], 16
; CHECK-NEXT: [[TMP5:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP1]] ; CHECK-NEXT: [[TMP5:%.*]] = add i64 [[OFFSET_IDX]], 20
; CHECK-NEXT: [[TMP6:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP2]] ; CHECK-NEXT: [[TMP6:%.*]] = add i64 [[OFFSET_IDX]], 24
; CHECK-NEXT: [[TMP7:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP3]] ; CHECK-NEXT: [[TMP7:%.*]] = add i64 [[OFFSET_IDX]], 28
; CHECK-NEXT: [[TMP8:%.*]] = load i1, ptr [[TMP4]], align 1 ; CHECK-NEXT: [[TMP8:%.*]] = add i64 [[OFFSET_IDX]], 32
; CHECK-NEXT: [[TMP9:%.*]] = load i1, ptr [[TMP5]], align 1 ; CHECK-NEXT: [[TMP9:%.*]] = add i64 [[OFFSET_IDX]], 36
; CHECK-NEXT: [[TMP10:%.*]] = load i1, ptr [[TMP6]], align 1 ; CHECK-NEXT: [[TMP10:%.*]] = add i64 [[OFFSET_IDX]], 40
; CHECK-NEXT: [[TMP11:%.*]] = load i1, ptr [[TMP7]], align 1 ; CHECK-NEXT: [[TMP11:%.*]] = add i64 [[OFFSET_IDX]], 44
; CHECK-NEXT: [[TMP12:%.*]] = insertelement <4 x i1> poison, i1 [[TMP8]], i32 0 ; CHECK-NEXT: [[TMP12:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE:%.*]], i64 [[TMP0]]
; CHECK-NEXT: [[TMP13:%.*]] = insertelement <4 x i1> [[TMP12]], i1 [[TMP9]], i32 1 ; CHECK-NEXT: [[TMP13:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP1]]
; CHECK-NEXT: [[TMP14:%.*]] = insertelement <4 x i1> [[TMP13]], i1 [[TMP10]], i32 2 ; CHECK-NEXT: [[TMP14:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP2]]
; CHECK-NEXT: [[TMP15:%.*]] = insertelement <4 x i1> [[TMP14]], i1 [[TMP11]], i32 3 ; CHECK-NEXT: [[TMP15:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP3]]
; CHECK-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP0]] ; CHECK-NEXT: [[TMP16:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP4]]
; CHECK-NEXT: [[TMP17:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP1]] ; CHECK-NEXT: [[TMP17:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP5]]
; CHECK-NEXT: [[TMP18:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP2]] ; CHECK-NEXT: [[TMP18:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP6]]
; CHECK-NEXT: [[TMP19:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP3]] ; CHECK-NEXT: [[TMP19:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP7]]
; CHECK-NEXT: [[TMP20:%.*]] = load i32, ptr [[TMP16]], align 4 ; CHECK-NEXT: [[TMP20:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP8]]
; CHECK-NEXT: [[TMP21:%.*]] = load i32, ptr [[TMP17]], align 4 ; CHECK-NEXT: [[TMP21:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP9]]
; CHECK-NEXT: [[TMP22:%.*]] = load i32, ptr [[TMP18]], align 4 ; CHECK-NEXT: [[TMP22:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP10]]
; CHECK-NEXT: [[TMP23:%.*]] = load i32, ptr [[TMP19]], align 4 ; CHECK-NEXT: [[TMP23:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP11]]
; CHECK-NEXT: [[TMP24:%.*]] = insertelement <4 x i32> poison, i32 [[TMP20]], i32 0 ; CHECK-NEXT: [[TMP24:%.*]] = load i1, ptr [[TMP12]], align 1
; CHECK-NEXT: [[TMP25:%.*]] = insertelement <4 x i32> [[TMP24]], i32 [[TMP21]], i32 1 ; CHECK-NEXT: [[TMP25:%.*]] = load i1, ptr [[TMP13]], align 1
; CHECK-NEXT: [[TMP26:%.*]] = insertelement <4 x i32> [[TMP25]], i32 [[TMP22]], i32 2 ; CHECK-NEXT: [[TMP26:%.*]] = load i1, ptr [[TMP14]], align 1
; CHECK-NEXT: [[TMP27:%.*]] = insertelement <4 x i32> [[TMP26]], i32 [[TMP23]], i32 3 ; CHECK-NEXT: [[TMP27:%.*]] = load i1, ptr [[TMP15]], align 1
; CHECK-NEXT: [[TMP28:%.*]] = xor <4 x i1> [[TMP15]], <i1 true, i1 true, i1 true, i1 true> ; CHECK-NEXT: [[TMP28:%.*]] = insertelement <4 x i1> poison, i1 [[TMP24]], i32 0
; CHECK-NEXT: [[PREDPHI:%.*]] = select <4 x i1> [[TMP15]], <4 x i32> [[TMP27]], <4 x i32> zeroinitializer ; CHECK-NEXT: [[TMP29:%.*]] = insertelement <4 x i1> [[TMP28]], i1 [[TMP25]], i32 1
; CHECK-NEXT: [[TMP29]] = add <4 x i32> [[VEC_PHI]], [[PREDPHI]] ; CHECK-NEXT: [[TMP30:%.*]] = insertelement <4 x i1> [[TMP29]], i1 [[TMP26]], i32 2
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 4 ; CHECK-NEXT: [[TMP31:%.*]] = insertelement <4 x i1> [[TMP30]], i1 [[TMP27]], i32 3
; CHECK-NEXT: [[TMP30:%.*]] = icmp eq i64 [[INDEX_NEXT]], 24 ; CHECK-NEXT: [[TMP32:%.*]] = load i1, ptr [[TMP16]], align 1
; CHECK-NEXT: br i1 [[TMP30]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP32:![0-9]+]] ; CHECK-NEXT: [[TMP33:%.*]] = load i1, ptr [[TMP17]], align 1
; CHECK-NEXT: [[TMP34:%.*]] = load i1, ptr [[TMP18]], align 1
; CHECK-NEXT: [[TMP35:%.*]] = load i1, ptr [[TMP19]], align 1
; CHECK-NEXT: [[TMP36:%.*]] = insertelement <4 x i1> poison, i1 [[TMP32]], i32 0
; CHECK-NEXT: [[TMP37:%.*]] = insertelement <4 x i1> [[TMP36]], i1 [[TMP33]], i32 1
; CHECK-NEXT: [[TMP38:%.*]] = insertelement <4 x i1> [[TMP37]], i1 [[TMP34]], i32 2
; CHECK-NEXT: [[TMP39:%.*]] = insertelement <4 x i1> [[TMP38]], i1 [[TMP35]], i32 3
; CHECK-NEXT: [[TMP40:%.*]] = load i1, ptr [[TMP20]], align 1
; CHECK-NEXT: [[TMP41:%.*]] = load i1, ptr [[TMP21]], align 1
; CHECK-NEXT: [[TMP42:%.*]] = load i1, ptr [[TMP22]], align 1
; CHECK-NEXT: [[TMP43:%.*]] = load i1, ptr [[TMP23]], align 1
; CHECK-NEXT: [[TMP44:%.*]] = insertelement <4 x i1> poison, i1 [[TMP40]], i32 0
; CHECK-NEXT: [[TMP45:%.*]] = insertelement <4 x i1> [[TMP44]], i1 [[TMP41]], i32 1
; CHECK-NEXT: [[TMP46:%.*]] = insertelement <4 x i1> [[TMP45]], i1 [[TMP42]], i32 2
; CHECK-NEXT: [[TMP47:%.*]] = insertelement <4 x i1> [[TMP46]], i1 [[TMP43]], i32 3
; CHECK-NEXT: [[TMP48:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP0]]
; CHECK-NEXT: [[TMP49:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP1]]
; CHECK-NEXT: [[TMP50:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP2]]
; CHECK-NEXT: [[TMP51:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP3]]
; CHECK-NEXT: [[TMP52:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP4]]
; CHECK-NEXT: [[TMP53:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP5]]
; CHECK-NEXT: [[TMP54:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP6]]
; CHECK-NEXT: [[TMP55:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP7]]
; CHECK-NEXT: [[TMP56:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP8]]
; CHECK-NEXT: [[TMP57:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP9]]
; CHECK-NEXT: [[TMP58:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP10]]
; CHECK-NEXT: [[TMP59:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP11]]
; CHECK-NEXT: [[TMP60:%.*]] = load i32, ptr [[TMP48]], align 4
; CHECK-NEXT: [[TMP61:%.*]] = load i32, ptr [[TMP49]], align 4
; CHECK-NEXT: [[TMP62:%.*]] = load i32, ptr [[TMP50]], align 4
; CHECK-NEXT: [[TMP63:%.*]] = load i32, ptr [[TMP51]], align 4
; CHECK-NEXT: [[TMP64:%.*]] = insertelement <4 x i32> poison, i32 [[TMP60]], i32 0
; CHECK-NEXT: [[TMP65:%.*]] = insertelement <4 x i32> [[TMP64]], i32 [[TMP61]], i32 1
; CHECK-NEXT: [[TMP66:%.*]] = insertelement <4 x i32> [[TMP65]], i32 [[TMP62]], i32 2
; CHECK-NEXT: [[TMP67:%.*]] = insertelement <4 x i32> [[TMP66]], i32 [[TMP63]], i32 3
; CHECK-NEXT: [[TMP68:%.*]] = load i32, ptr [[TMP52]], align 4
; CHECK-NEXT: [[TMP69:%.*]] = load i32, ptr [[TMP53]], align 4
; CHECK-NEXT: [[TMP70:%.*]] = load i32, ptr [[TMP54]], align 4
; CHECK-NEXT: [[TMP71:%.*]] = load i32, ptr [[TMP55]], align 4
; CHECK-NEXT: [[TMP72:%.*]] = insertelement <4 x i32> poison, i32 [[TMP68]], i32 0
; CHECK-NEXT: [[TMP73:%.*]] = insertelement <4 x i32> [[TMP72]], i32 [[TMP69]], i32 1
; CHECK-NEXT: [[TMP74:%.*]] = insertelement <4 x i32> [[TMP73]], i32 [[TMP70]], i32 2
; CHECK-NEXT: [[TMP75:%.*]] = insertelement <4 x i32> [[TMP74]], i32 [[TMP71]], i32 3
; CHECK-NEXT: [[TMP76:%.*]] = load i32, ptr [[TMP56]], align 4
; CHECK-NEXT: [[TMP77:%.*]] = load i32, ptr [[TMP57]], align 4
; CHECK-NEXT: [[TMP78:%.*]] = load i32, ptr [[TMP58]], align 4
; CHECK-NEXT: [[TMP79:%.*]] = load i32, ptr [[TMP59]], align 4
; CHECK-NEXT: [[TMP80:%.*]] = insertelement <4 x i32> poison, i32 [[TMP76]], i32 0
; CHECK-NEXT: [[TMP81:%.*]] = insertelement <4 x i32> [[TMP80]], i32 [[TMP77]], i32 1
; CHECK-NEXT: [[TMP82:%.*]] = insertelement <4 x i32> [[TMP81]], i32 [[TMP78]], i32 2
; CHECK-NEXT: [[TMP83:%.*]] = insertelement <4 x i32> [[TMP82]], i32 [[TMP79]], i32 3
; CHECK-NEXT: [[TMP84:%.*]] = xor <4 x i1> [[TMP31]], <i1 true, i1 true, i1 true, i1 true>
; CHECK-NEXT: [[TMP85:%.*]] = xor <4 x i1> [[TMP39]], <i1 true, i1 true, i1 true, i1 true>
; CHECK-NEXT: [[TMP86:%.*]] = xor <4 x i1> [[TMP47]], <i1 true, i1 true, i1 true, i1 true>
; CHECK-NEXT: [[PREDPHI:%.*]] = select <4 x i1> [[TMP31]], <4 x i32> [[TMP67]], <4 x i32> zeroinitializer
; CHECK-NEXT: [[PREDPHI3:%.*]] = select <4 x i1> [[TMP39]], <4 x i32> [[TMP75]], <4 x i32> zeroinitializer
; CHECK-NEXT: [[PREDPHI4:%.*]] = select <4 x i1> [[TMP47]], <4 x i32> [[TMP83]], <4 x i32> zeroinitializer
; CHECK-NEXT: [[TMP87]] = add <4 x i32> [[VEC_PHI]], [[PREDPHI]]
; CHECK-NEXT: [[TMP88]] = add <4 x i32> [[VEC_PHI1]], [[PREDPHI3]]
; CHECK-NEXT: [[TMP89]] = add <4 x i32> [[VEC_PHI2]], [[PREDPHI4]]
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 12
; CHECK-NEXT: [[TMP90:%.*]] = icmp eq i64 [[INDEX_NEXT]], 24
; CHECK-NEXT: br i1 [[TMP90]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP32:![0-9]+]]
; CHECK: middle.block: ; CHECK: middle.block:
; CHECK-NEXT: [[TMP31:%.*]] = call i32 @llvm.vector.reduce.add.v4i32(<4 x i32> [[TMP29]]) ; CHECK-NEXT: [[BIN_RDX:%.*]] = add <4 x i32> [[TMP88]], [[TMP87]]
; CHECK-NEXT: [[BIN_RDX5:%.*]] = add <4 x i32> [[TMP89]], [[BIN_RDX]]
; CHECK-NEXT: [[TMP91:%.*]] = call i32 @llvm.vector.reduce.add.v4i32(<4 x i32> [[BIN_RDX5]])
; CHECK-NEXT: br i1 false, label [[LOOP_EXIT:%.*]], label [[SCALAR_PH]] ; CHECK-NEXT: br i1 false, label [[LOOP_EXIT:%.*]], label [[SCALAR_PH]]
; CHECK: scalar.ph: ; CHECK: scalar.ph:
; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 96, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ] ; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 96, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]
; CHECK-NEXT: [[BC_MERGE_RDX:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[TMP31]], [[MIDDLE_BLOCK]] ] ; CHECK-NEXT: [[BC_MERGE_RDX:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[TMP91]], [[MIDDLE_BLOCK]] ]
; CHECK-NEXT: br label [[LOOP:%.*]] ; CHECK-NEXT: br label [[LOOP:%.*]]
; CHECK: loop: ; CHECK: loop:
; CHECK-NEXT: [[IV:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[IV_NEXT:%.*]], [[LATCH:%.*]] ] ; CHECK-NEXT: [[IV:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[IV_NEXT:%.*]], [[LATCH:%.*]] ]
; CHECK-NEXT: [[ACCUM:%.*]] = phi i32 [ [[BC_MERGE_RDX]], [[SCALAR_PH]] ], [ [[ACCUM_NEXT:%.*]], [[LATCH]] ] ; CHECK-NEXT: [[ACCUM:%.*]] = phi i32 [ [[BC_MERGE_RDX]], [[SCALAR_PH]] ], [ [[ACCUM_NEXT:%.*]], [[LATCH]] ]
; CHECK-NEXT: [[IV_NEXT]] = add i64 [[IV]], 4 ; CHECK-NEXT: [[IV_NEXT]] = add i64 [[IV]], 4
; CHECK-NEXT: [[TEST_ADDR:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[IV]] ; CHECK-NEXT: [[TEST_ADDR:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[IV]]
; CHECK-NEXT: [[EARLYCND:%.*]] = load i1, ptr [[TEST_ADDR]], align 1 ; CHECK-NEXT: [[EARLYCND:%.*]] = load i1, ptr [[TEST_ADDR]], align 1
; CHECK-NEXT: br i1 [[EARLYCND]], label [[PRED:%.*]], label [[LATCH]] ; CHECK-NEXT: br i1 [[EARLYCND]], label [[PRED:%.*]], label [[LATCH]]
; CHECK: pred: ; CHECK: pred:
; CHECK-NEXT: [[ADDR:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[IV]] ; CHECK-NEXT: [[ADDR:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[IV]]
; CHECK-NEXT: [[VAL:%.*]] = load i32, ptr [[ADDR]], align 4 ; CHECK-NEXT: [[VAL:%.*]] = load i32, ptr [[ADDR]], align 4
; CHECK-NEXT: br label [[LATCH]] ; CHECK-NEXT: br label [[LATCH]]
; CHECK: latch: ; CHECK: latch:
; CHECK-NEXT: [[VAL_PHI:%.*]] = phi i32 [ 0, [[LOOP]] ], [ [[VAL]], [[PRED]] ] ; CHECK-NEXT: [[VAL_PHI:%.*]] = phi i32 [ 0, [[LOOP]] ], [ [[VAL]], [[PRED]] ]
; CHECK-NEXT: [[ACCUM_NEXT]] = add i32 [[ACCUM]], [[VAL_PHI]] ; CHECK-NEXT: [[ACCUM_NEXT]] = add i32 [[ACCUM]], [[VAL_PHI]]
; CHECK-NEXT: [[EXIT:%.*]] = icmp ugt i64 [[IV]], 100 ; CHECK-NEXT: [[EXIT:%.*]] = icmp ugt i64 [[IV]], 100
; CHECK-NEXT: br i1 [[EXIT]], label [[LOOP_EXIT]], label [[LOOP]], !llvm.loop [[LOOP33:![0-9]+]] ; CHECK-NEXT: br i1 [[EXIT]], label [[LOOP_EXIT]], label [[LOOP]], !llvm.loop [[LOOP33:![0-9]+]]
; CHECK: loop_exit: ; CHECK: loop_exit:
; CHECK-NEXT: [[ACCUM_NEXT_LCSSA:%.*]] = phi i32 [ [[ACCUM_NEXT]], [[LATCH]] ], [ [[TMP31]], [[MIDDLE_BLOCK]] ] ; CHECK-NEXT: [[ACCUM_NEXT_LCSSA:%.*]] = phi i32 [ [[ACCUM_NEXT]], [[LATCH]] ], [ [[TMP91]], [[MIDDLE_BLOCK]] ]
; CHECK-NEXT: ret i32 [[ACCUM_NEXT_LCSSA]] ; CHECK-NEXT: ret i32 [[ACCUM_NEXT_LCSSA]]
; ;
entry: entry:
%alloca = alloca [108 x i32] %alloca = alloca [108 x i32]
call void @init(ptr %alloca) call void @init(ptr %alloca)
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %latch ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %latch ]
Show All 21 Lines
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[ALLOCA:%.*]] = alloca [110 x i32], align 4 ; CHECK-NEXT: [[ALLOCA:%.*]] = alloca [110 x i32], align 4
; CHECK-NEXT: call void @init(ptr [[ALLOCA]]) ; CHECK-NEXT: call void @init(ptr [[ALLOCA]])
; CHECK-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]] ; CHECK-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]]
; CHECK: vector.ph: ; CHECK: vector.ph:
; CHECK-NEXT: br label [[VECTOR_BODY:%.*]] ; CHECK-NEXT: br label [[VECTOR_BODY:%.*]]
; CHECK: vector.body: ; CHECK: vector.body:
; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ] ; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[VEC_PHI:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP29:%.*]], [[VECTOR_BODY]] ] ; CHECK-NEXT: [[VEC_PHI:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP58:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[VEC_PHI1:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP59:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[OFFSET_IDX:%.*]] = mul i64 [[INDEX]], 5 ; CHECK-NEXT: [[OFFSET_IDX:%.*]] = mul i64 [[INDEX]], 5
; CHECK-NEXT: [[TMP0:%.*]] = add i64 [[OFFSET_IDX]], 0 ; CHECK-NEXT: [[TMP0:%.*]] = add i64 [[OFFSET_IDX]], 0
; CHECK-NEXT: [[TMP1:%.*]] = add i64 [[OFFSET_IDX]], 5 ; CHECK-NEXT: [[TMP1:%.*]] = add i64 [[OFFSET_IDX]], 5
; CHECK-NEXT: [[TMP2:%.*]] = add i64 [[OFFSET_IDX]], 10 ; CHECK-NEXT: [[TMP2:%.*]] = add i64 [[OFFSET_IDX]], 10
; CHECK-NEXT: [[TMP3:%.*]] = add i64 [[OFFSET_IDX]], 15 ; CHECK-NEXT: [[TMP3:%.*]] = add i64 [[OFFSET_IDX]], 15
; CHECK-NEXT: [[TMP4:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE:%.*]], i64 [[TMP0]] ; CHECK-NEXT: [[TMP4:%.*]] = add i64 [[OFFSET_IDX]], 20
; CHECK-NEXT: [[TMP5:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP1]] ; CHECK-NEXT: [[TMP5:%.*]] = add i64 [[OFFSET_IDX]], 25
; CHECK-NEXT: [[TMP6:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP2]] ; CHECK-NEXT: [[TMP6:%.*]] = add i64 [[OFFSET_IDX]], 30
; CHECK-NEXT: [[TMP7:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP3]] ; CHECK-NEXT: [[TMP7:%.*]] = add i64 [[OFFSET_IDX]], 35
; CHECK-NEXT: [[TMP8:%.*]] = load i1, ptr [[TMP4]], align 1 ; CHECK-NEXT: [[TMP8:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE:%.*]], i64 [[TMP0]]
; CHECK-NEXT: [[TMP9:%.*]] = load i1, ptr [[TMP5]], align 1 ; CHECK-NEXT: [[TMP9:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP1]]
; CHECK-NEXT: [[TMP10:%.*]] = load i1, ptr [[TMP6]], align 1 ; CHECK-NEXT: [[TMP10:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP2]]
; CHECK-NEXT: [[TMP11:%.*]] = load i1, ptr [[TMP7]], align 1 ; CHECK-NEXT: [[TMP11:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP3]]
; CHECK-NEXT: [[TMP12:%.*]] = insertelement <4 x i1> poison, i1 [[TMP8]], i32 0 ; CHECK-NEXT: [[TMP12:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP4]]
; CHECK-NEXT: [[TMP13:%.*]] = insertelement <4 x i1> [[TMP12]], i1 [[TMP9]], i32 1 ; CHECK-NEXT: [[TMP13:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP5]]
; CHECK-NEXT: [[TMP14:%.*]] = insertelement <4 x i1> [[TMP13]], i1 [[TMP10]], i32 2 ; CHECK-NEXT: [[TMP14:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP6]]
; CHECK-NEXT: [[TMP15:%.*]] = insertelement <4 x i1> [[TMP14]], i1 [[TMP11]], i32 3 ; CHECK-NEXT: [[TMP15:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP7]]
; CHECK-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP0]] ; CHECK-NEXT: [[TMP16:%.*]] = load i1, ptr [[TMP8]], align 1
; CHECK-NEXT: [[TMP17:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP1]] ; CHECK-NEXT: [[TMP17:%.*]] = load i1, ptr [[TMP9]], align 1
; CHECK-NEXT: [[TMP18:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP2]] ; CHECK-NEXT: [[TMP18:%.*]] = load i1, ptr [[TMP10]], align 1
; CHECK-NEXT: [[TMP19:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP3]] ; CHECK-NEXT: [[TMP19:%.*]] = load i1, ptr [[TMP11]], align 1
; CHECK-NEXT: [[TMP20:%.*]] = load i32, ptr [[TMP16]], align 4 ; CHECK-NEXT: [[TMP20:%.*]] = insertelement <4 x i1> poison, i1 [[TMP16]], i32 0
; CHECK-NEXT: [[TMP21:%.*]] = load i32, ptr [[TMP17]], align 4 ; CHECK-NEXT: [[TMP21:%.*]] = insertelement <4 x i1> [[TMP20]], i1 [[TMP17]], i32 1
; CHECK-NEXT: [[TMP22:%.*]] = load i32, ptr [[TMP18]], align 4 ; CHECK-NEXT: [[TMP22:%.*]] = insertelement <4 x i1> [[TMP21]], i1 [[TMP18]], i32 2
; CHECK-NEXT: [[TMP23:%.*]] = load i32, ptr [[TMP19]], align 4 ; CHECK-NEXT: [[TMP23:%.*]] = insertelement <4 x i1> [[TMP22]], i1 [[TMP19]], i32 3
; CHECK-NEXT: [[TMP24:%.*]] = insertelement <4 x i32> poison, i32 [[TMP20]], i32 0 ; CHECK-NEXT: [[TMP24:%.*]] = load i1, ptr [[TMP12]], align 1
; CHECK-NEXT: [[TMP25:%.*]] = insertelement <4 x i32> [[TMP24]], i32 [[TMP21]], i32 1 ; CHECK-NEXT: [[TMP25:%.*]] = load i1, ptr [[TMP13]], align 1
; CHECK-NEXT: [[TMP26:%.*]] = insertelement <4 x i32> [[TMP25]], i32 [[TMP22]], i32 2 ; CHECK-NEXT: [[TMP26:%.*]] = load i1, ptr [[TMP14]], align 1
; CHECK-NEXT: [[TMP27:%.*]] = insertelement <4 x i32> [[TMP26]], i32 [[TMP23]], i32 3 ; CHECK-NEXT: [[TMP27:%.*]] = load i1, ptr [[TMP15]], align 1
; CHECK-NEXT: [[TMP28:%.*]] = xor <4 x i1> [[TMP15]], <i1 true, i1 true, i1 true, i1 true> ; CHECK-NEXT: [[TMP28:%.*]] = insertelement <4 x i1> poison, i1 [[TMP24]], i32 0
; CHECK-NEXT: [[PREDPHI:%.*]] = select <4 x i1> [[TMP15]], <4 x i32> [[TMP27]], <4 x i32> zeroinitializer ; CHECK-NEXT: [[TMP29:%.*]] = insertelement <4 x i1> [[TMP28]], i1 [[TMP25]], i32 1
; CHECK-NEXT: [[TMP29]] = add <4 x i32> [[VEC_PHI]], [[PREDPHI]] ; CHECK-NEXT: [[TMP30:%.*]] = insertelement <4 x i1> [[TMP29]], i1 [[TMP26]], i32 2
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 4 ; CHECK-NEXT: [[TMP31:%.*]] = insertelement <4 x i1> [[TMP30]], i1 [[TMP27]], i32 3
; CHECK-NEXT: [[TMP30:%.*]] = icmp eq i64 [[INDEX_NEXT]], 20 ; CHECK-NEXT: [[TMP32:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP0]]
; CHECK-NEXT: br i1 [[TMP30]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP34:![0-9]+]] ; CHECK-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP1]]
; CHECK-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP2]]
; CHECK-NEXT: [[TMP35:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP3]]
; CHECK-NEXT: [[TMP36:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP4]]
; CHECK-NEXT: [[TMP37:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP5]]
; CHECK-NEXT: [[TMP38:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP6]]
; CHECK-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP7]]
; CHECK-NEXT: [[TMP40:%.*]] = load i32, ptr [[TMP32]], align 4
; CHECK-NEXT: [[TMP41:%.*]] = load i32, ptr [[TMP33]], align 4
; CHECK-NEXT: [[TMP42:%.*]] = load i32, ptr [[TMP34]], align 4
; CHECK-NEXT: [[TMP43:%.*]] = load i32, ptr [[TMP35]], align 4
; CHECK-NEXT: [[TMP44:%.*]] = insertelement <4 x i32> poison, i32 [[TMP40]], i32 0
; CHECK-NEXT: [[TMP45:%.*]] = insertelement <4 x i32> [[TMP44]], i32 [[TMP41]], i32 1
; CHECK-NEXT: [[TMP46:%.*]] = insertelement <4 x i32> [[TMP45]], i32 [[TMP42]], i32 2
; CHECK-NEXT: [[TMP47:%.*]] = insertelement <4 x i32> [[TMP46]], i32 [[TMP43]], i32 3
; CHECK-NEXT: [[TMP48:%.*]] = load i32, ptr [[TMP36]], align 4
; CHECK-NEXT: [[TMP49:%.*]] = load i32, ptr [[TMP37]], align 4
; CHECK-NEXT: [[TMP50:%.*]] = load i32, ptr [[TMP38]], align 4
; CHECK-NEXT: [[TMP51:%.*]] = load i32, ptr [[TMP39]], align 4
; CHECK-NEXT: [[TMP52:%.*]] = insertelement <4 x i32> poison, i32 [[TMP48]], i32 0
; CHECK-NEXT: [[TMP53:%.*]] = insertelement <4 x i32> [[TMP52]], i32 [[TMP49]], i32 1
; CHECK-NEXT: [[TMP54:%.*]] = insertelement <4 x i32> [[TMP53]], i32 [[TMP50]], i32 2
; CHECK-NEXT: [[TMP55:%.*]] = insertelement <4 x i32> [[TMP54]], i32 [[TMP51]], i32 3
; CHECK-NEXT: [[TMP56:%.*]] = xor <4 x i1> [[TMP23]], <i1 true, i1 true, i1 true, i1 true>
; CHECK-NEXT: [[TMP57:%.*]] = xor <4 x i1> [[TMP31]], <i1 true, i1 true, i1 true, i1 true>
; CHECK-NEXT: [[PREDPHI:%.*]] = select <4 x i1> [[TMP23]], <4 x i32> [[TMP47]], <4 x i32> zeroinitializer
; CHECK-NEXT: [[PREDPHI2:%.*]] = select <4 x i1> [[TMP31]], <4 x i32> [[TMP55]], <4 x i32> zeroinitializer
; CHECK-NEXT: [[TMP58]] = add <4 x i32> [[VEC_PHI]], [[PREDPHI]]
; CHECK-NEXT: [[TMP59]] = add <4 x i32> [[VEC_PHI1]], [[PREDPHI2]]
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 8
; CHECK-NEXT: [[TMP60:%.*]] = icmp eq i64 [[INDEX_NEXT]], 16
; CHECK-NEXT: br i1 [[TMP60]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP34:![0-9]+]]
; CHECK: middle.block: ; CHECK: middle.block:
; CHECK-NEXT: [[TMP31:%.*]] = call i32 @llvm.vector.reduce.add.v4i32(<4 x i32> [[TMP29]]) ; CHECK-NEXT: [[BIN_RDX:%.*]] = add <4 x i32> [[TMP59]], [[TMP58]]
; CHECK-NEXT: [[TMP61:%.*]] = call i32 @llvm.vector.reduce.add.v4i32(<4 x i32> [[BIN_RDX]])
; CHECK-NEXT: br i1 false, label [[LOOP_EXIT:%.*]], label [[SCALAR_PH]] ; CHECK-NEXT: br i1 false, label [[LOOP_EXIT:%.*]], label [[SCALAR_PH]]
; CHECK: scalar.ph: ; CHECK: scalar.ph:
; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 100, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ] ; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 80, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]
; CHECK-NEXT: [[BC_MERGE_RDX:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[TMP31]], [[MIDDLE_BLOCK]] ] ; CHECK-NEXT: [[BC_MERGE_RDX:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[TMP61]], [[MIDDLE_BLOCK]] ]
; CHECK-NEXT: br label [[LOOP:%.*]] ; CHECK-NEXT: br label [[LOOP:%.*]]
; CHECK: loop: ; CHECK: loop:
; CHECK-NEXT: [[IV:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[IV_NEXT:%.*]], [[LATCH:%.*]] ] ; CHECK-NEXT: [[IV:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[IV_NEXT:%.*]], [[LATCH:%.*]] ]
; CHECK-NEXT: [[ACCUM:%.*]] = phi i32 [ [[BC_MERGE_RDX]], [[SCALAR_PH]] ], [ [[ACCUM_NEXT:%.*]], [[LATCH]] ] ; CHECK-NEXT: [[ACCUM:%.*]] = phi i32 [ [[BC_MERGE_RDX]], [[SCALAR_PH]] ], [ [[ACCUM_NEXT:%.*]], [[LATCH]] ]
; CHECK-NEXT: [[IV_NEXT]] = add i64 [[IV]], 5 ; CHECK-NEXT: [[IV_NEXT]] = add i64 [[IV]], 5
; CHECK-NEXT: [[TEST_ADDR:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[IV]] ; CHECK-NEXT: [[TEST_ADDR:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[IV]]
; CHECK-NEXT: [[EARLYCND:%.*]] = load i1, ptr [[TEST_ADDR]], align 1 ; CHECK-NEXT: [[EARLYCND:%.*]] = load i1, ptr [[TEST_ADDR]], align 1
; CHECK-NEXT: br i1 [[EARLYCND]], label [[PRED:%.*]], label [[LATCH]] ; CHECK-NEXT: br i1 [[EARLYCND]], label [[PRED:%.*]], label [[LATCH]]
; CHECK: pred: ; CHECK: pred:
; CHECK-NEXT: [[ADDR:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[IV]] ; CHECK-NEXT: [[ADDR:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[IV]]
; CHECK-NEXT: [[VAL:%.*]] = load i32, ptr [[ADDR]], align 4 ; CHECK-NEXT: [[VAL:%.*]] = load i32, ptr [[ADDR]], align 4
; CHECK-NEXT: br label [[LATCH]] ; CHECK-NEXT: br label [[LATCH]]
; CHECK: latch: ; CHECK: latch:
; CHECK-NEXT: [[VAL_PHI:%.*]] = phi i32 [ 0, [[LOOP]] ], [ [[VAL]], [[PRED]] ] ; CHECK-NEXT: [[VAL_PHI:%.*]] = phi i32 [ 0, [[LOOP]] ], [ [[VAL]], [[PRED]] ]
; CHECK-NEXT: [[ACCUM_NEXT]] = add i32 [[ACCUM]], [[VAL_PHI]] ; CHECK-NEXT: [[ACCUM_NEXT]] = add i32 [[ACCUM]], [[VAL_PHI]]
; CHECK-NEXT: [[EXIT:%.*]] = icmp ugt i64 [[IV]], 100 ; CHECK-NEXT: [[EXIT:%.*]] = icmp ugt i64 [[IV]], 100
; CHECK-NEXT: br i1 [[EXIT]], label [[LOOP_EXIT]], label [[LOOP]], !llvm.loop [[LOOP35:![0-9]+]] ; CHECK-NEXT: br i1 [[EXIT]], label [[LOOP_EXIT]], label [[LOOP]], !llvm.loop [[LOOP35:![0-9]+]]
; CHECK: loop_exit: ; CHECK: loop_exit:
; CHECK-NEXT: [[ACCUM_NEXT_LCSSA:%.*]] = phi i32 [ [[ACCUM_NEXT]], [[LATCH]] ], [ [[TMP31]], [[MIDDLE_BLOCK]] ] ; CHECK-NEXT: [[ACCUM_NEXT_LCSSA:%.*]] = phi i32 [ [[ACCUM_NEXT]], [[LATCH]] ], [ [[TMP61]], [[MIDDLE_BLOCK]] ]
; CHECK-NEXT: ret i32 [[ACCUM_NEXT_LCSSA]] ; CHECK-NEXT: ret i32 [[ACCUM_NEXT_LCSSA]]
; ;
entry: entry:
%alloca = alloca [110 x i32] %alloca = alloca [110 x i32]
call void @init(ptr %alloca) call void @init(ptr %alloca)
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %latch ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %latch ]
Show All 20 Lines
; CHECK-LABEL: @neg_test_non_unit_stride_off_by_four_bytes( ; CHECK-LABEL: @neg_test_non_unit_stride_off_by_four_bytes(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[ALLOCA:%.*]] = alloca [103 x i32], align 4 ; CHECK-NEXT: [[ALLOCA:%.*]] = alloca [103 x i32], align 4
; CHECK-NEXT: call void @init(ptr [[ALLOCA]]) ; CHECK-NEXT: call void @init(ptr [[ALLOCA]])
; CHECK-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]] ; CHECK-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]]
; CHECK: vector.ph: ; CHECK: vector.ph:
; CHECK-NEXT: br label [[VECTOR_BODY:%.*]] ; CHECK-NEXT: br label [[VECTOR_BODY:%.*]]
; CHECK: vector.body: ; CHECK: vector.body:
; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[PRED_LOAD_CONTINUE6:%.*]] ] ; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[PRED_LOAD_CONTINUE33:%.*]] ]
; CHECK-NEXT: [[VEC_PHI:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP37:%.*]], [[PRED_LOAD_CONTINUE6]] ] ; CHECK-NEXT: [[VEC_PHI:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP148:%.*]], [[PRED_LOAD_CONTINUE33]] ]
; CHECK-NEXT: [[VEC_PHI1:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP149:%.*]], [[PRED_LOAD_CONTINUE33]] ]
; CHECK-NEXT: [[VEC_PHI2:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP150:%.*]], [[PRED_LOAD_CONTINUE33]] ]
; CHECK-NEXT: [[VEC_PHI3:%.*]] = phi <4 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP151:%.*]], [[PRED_LOAD_CONTINUE33]] ]
; CHECK-NEXT: [[OFFSET_IDX:%.*]] = mul i64 [[INDEX]], 2 ; CHECK-NEXT: [[OFFSET_IDX:%.*]] = mul i64 [[INDEX]], 2
; CHECK-NEXT: [[TMP0:%.*]] = add i64 [[OFFSET_IDX]], 0 ; CHECK-NEXT: [[TMP0:%.*]] = add i64 [[OFFSET_IDX]], 0
; CHECK-NEXT: [[TMP1:%.*]] = add i64 [[OFFSET_IDX]], 2 ; CHECK-NEXT: [[TMP1:%.*]] = add i64 [[OFFSET_IDX]], 2
; CHECK-NEXT: [[TMP2:%.*]] = add i64 [[OFFSET_IDX]], 4 ; CHECK-NEXT: [[TMP2:%.*]] = add i64 [[OFFSET_IDX]], 4
; CHECK-NEXT: [[TMP3:%.*]] = add i64 [[OFFSET_IDX]], 6 ; CHECK-NEXT: [[TMP3:%.*]] = add i64 [[OFFSET_IDX]], 6
; CHECK-NEXT: [[TMP4:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE:%.*]], i64 [[TMP0]] ; CHECK-NEXT: [[TMP4:%.*]] = add i64 [[OFFSET_IDX]], 8
; CHECK-NEXT: [[TMP5:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP1]] ; CHECK-NEXT: [[TMP5:%.*]] = add i64 [[OFFSET_IDX]], 10
; CHECK-NEXT: [[TMP6:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP2]] ; CHECK-NEXT: [[TMP6:%.*]] = add i64 [[OFFSET_IDX]], 12
; CHECK-NEXT: [[TMP7:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP3]] ; CHECK-NEXT: [[TMP7:%.*]] = add i64 [[OFFSET_IDX]], 14
; CHECK-NEXT: [[TMP8:%.*]] = load i1, ptr [[TMP4]], align 1 ; CHECK-NEXT: [[TMP8:%.*]] = add i64 [[OFFSET_IDX]], 16
; CHECK-NEXT: [[TMP9:%.*]] = load i1, ptr [[TMP5]], align 1 ; CHECK-NEXT: [[TMP9:%.*]] = add i64 [[OFFSET_IDX]], 18
; CHECK-NEXT: [[TMP10:%.*]] = load i1, ptr [[TMP6]], align 1 ; CHECK-NEXT: [[TMP10:%.*]] = add i64 [[OFFSET_IDX]], 20
; CHECK-NEXT: [[TMP11:%.*]] = load i1, ptr [[TMP7]], align 1 ; CHECK-NEXT: [[TMP11:%.*]] = add i64 [[OFFSET_IDX]], 22
; CHECK-NEXT: [[TMP12:%.*]] = insertelement <4 x i1> poison, i1 [[TMP8]], i32 0 ; CHECK-NEXT: [[TMP12:%.*]] = add i64 [[OFFSET_IDX]], 24
; CHECK-NEXT: [[TMP13:%.*]] = insertelement <4 x i1> [[TMP12]], i1 [[TMP9]], i32 1 ; CHECK-NEXT: [[TMP13:%.*]] = add i64 [[OFFSET_IDX]], 26
; CHECK-NEXT: [[TMP14:%.*]] = insertelement <4 x i1> [[TMP13]], i1 [[TMP10]], i32 2 ; CHECK-NEXT: [[TMP14:%.*]] = add i64 [[OFFSET_IDX]], 28
; CHECK-NEXT: [[TMP15:%.*]] = insertelement <4 x i1> [[TMP14]], i1 [[TMP11]], i32 3 ; CHECK-NEXT: [[TMP15:%.*]] = add i64 [[OFFSET_IDX]], 30
; CHECK-NEXT: [[TMP16:%.*]] = extractelement <4 x i1> [[TMP15]], i32 0 ; CHECK-NEXT: [[TMP16:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE:%.*]], i64 [[TMP0]]
; CHECK-NEXT: br i1 [[TMP16]], label [[PRED_LOAD_IF:%.*]], label [[PRED_LOAD_CONTINUE:%.*]] ; CHECK-NEXT: [[TMP17:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP1]]
; CHECK-NEXT: [[TMP18:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP2]]
; CHECK-NEXT: [[TMP19:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP3]]
; CHECK-NEXT: [[TMP20:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP4]]
; CHECK-NEXT: [[TMP21:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP5]]
; CHECK-NEXT: [[TMP22:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP6]]
; CHECK-NEXT: [[TMP23:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP7]]
; CHECK-NEXT: [[TMP24:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP8]]
; CHECK-NEXT: [[TMP25:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP9]]
; CHECK-NEXT: [[TMP26:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP10]]
; CHECK-NEXT: [[TMP27:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP11]]
; CHECK-NEXT: [[TMP28:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP12]]
; CHECK-NEXT: [[TMP29:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP13]]
; CHECK-NEXT: [[TMP30:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP14]]
; CHECK-NEXT: [[TMP31:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[TMP15]]
; CHECK-NEXT: [[TMP32:%.*]] = load i1, ptr [[TMP16]], align 1
; CHECK-NEXT: [[TMP33:%.*]] = load i1, ptr [[TMP17]], align 1
; CHECK-NEXT: [[TMP34:%.*]] = load i1, ptr [[TMP18]], align 1
; CHECK-NEXT: [[TMP35:%.*]] = load i1, ptr [[TMP19]], align 1
; CHECK-NEXT: [[TMP36:%.*]] = insertelement <4 x i1> poison, i1 [[TMP32]], i32 0
; CHECK-NEXT: [[TMP37:%.*]] = insertelement <4 x i1> [[TMP36]], i1 [[TMP33]], i32 1
; CHECK-NEXT: [[TMP38:%.*]] = insertelement <4 x i1> [[TMP37]], i1 [[TMP34]], i32 2
; CHECK-NEXT: [[TMP39:%.*]] = insertelement <4 x i1> [[TMP38]], i1 [[TMP35]], i32 3
; CHECK-NEXT: [[TMP40:%.*]] = load i1, ptr [[TMP20]], align 1
; CHECK-NEXT: [[TMP41:%.*]] = load i1, ptr [[TMP21]], align 1
; CHECK-NEXT: [[TMP42:%.*]] = load i1, ptr [[TMP22]], align 1
; CHECK-NEXT: [[TMP43:%.*]] = load i1, ptr [[TMP23]], align 1
; CHECK-NEXT: [[TMP44:%.*]] = insertelement <4 x i1> poison, i1 [[TMP40]], i32 0
; CHECK-NEXT: [[TMP45:%.*]] = insertelement <4 x i1> [[TMP44]], i1 [[TMP41]], i32 1
; CHECK-NEXT: [[TMP46:%.*]] = insertelement <4 x i1> [[TMP45]], i1 [[TMP42]], i32 2
; CHECK-NEXT: [[TMP47:%.*]] = insertelement <4 x i1> [[TMP46]], i1 [[TMP43]], i32 3
; CHECK-NEXT: [[TMP48:%.*]] = load i1, ptr [[TMP24]], align 1
; CHECK-NEXT: [[TMP49:%.*]] = load i1, ptr [[TMP25]], align 1
; CHECK-NEXT: [[TMP50:%.*]] = load i1, ptr [[TMP26]], align 1
; CHECK-NEXT: [[TMP51:%.*]] = load i1, ptr [[TMP27]], align 1
; CHECK-NEXT: [[TMP52:%.*]] = insertelement <4 x i1> poison, i1 [[TMP48]], i32 0
; CHECK-NEXT: [[TMP53:%.*]] = insertelement <4 x i1> [[TMP52]], i1 [[TMP49]], i32 1
; CHECK-NEXT: [[TMP54:%.*]] = insertelement <4 x i1> [[TMP53]], i1 [[TMP50]], i32 2
; CHECK-NEXT: [[TMP55:%.*]] = insertelement <4 x i1> [[TMP54]], i1 [[TMP51]], i32 3
; CHECK-NEXT: [[TMP56:%.*]] = load i1, ptr [[TMP28]], align 1
; CHECK-NEXT: [[TMP57:%.*]] = load i1, ptr [[TMP29]], align 1
; CHECK-NEXT: [[TMP58:%.*]] = load i1, ptr [[TMP30]], align 1
; CHECK-NEXT: [[TMP59:%.*]] = load i1, ptr [[TMP31]], align 1
; CHECK-NEXT: [[TMP60:%.*]] = insertelement <4 x i1> poison, i1 [[TMP56]], i32 0
; CHECK-NEXT: [[TMP61:%.*]] = insertelement <4 x i1> [[TMP60]], i1 [[TMP57]], i32 1
; CHECK-NEXT: [[TMP62:%.*]] = insertelement <4 x i1> [[TMP61]], i1 [[TMP58]], i32 2
; CHECK-NEXT: [[TMP63:%.*]] = insertelement <4 x i1> [[TMP62]], i1 [[TMP59]], i32 3
; CHECK-NEXT: [[TMP64:%.*]] = extractelement <4 x i1> [[TMP39]], i32 0
; CHECK-NEXT: br i1 [[TMP64]], label [[PRED_LOAD_IF:%.*]], label [[PRED_LOAD_CONTINUE:%.*]]
; CHECK: pred.load.if: ; CHECK: pred.load.if:
; CHECK-NEXT: [[TMP17:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP0]] ; CHECK-NEXT: [[TMP65:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP0]]
; CHECK-NEXT: [[TMP18:%.*]] = load i32, ptr [[TMP17]], align 4 ; CHECK-NEXT: [[TMP66:%.*]] = load i32, ptr [[TMP65]], align 4
; CHECK-NEXT: [[TMP19:%.*]] = insertelement <4 x i32> poison, i32 [[TMP18]], i32 0 ; CHECK-NEXT: [[TMP67:%.*]] = insertelement <4 x i32> poison, i32 [[TMP66]], i32 0
; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE]] ; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE]]
; CHECK: pred.load.continue: ; CHECK: pred.load.continue:
; CHECK-NEXT: [[TMP20:%.*]] = phi <4 x i32> [ poison, [[VECTOR_BODY]] ], [ [[TMP19]], [[PRED_LOAD_IF]] ] ; CHECK-NEXT: [[TMP68:%.*]] = phi <4 x i32> [ poison, [[VECTOR_BODY]] ], [ [[TMP67]], [[PRED_LOAD_IF]] ]
; CHECK-NEXT: [[TMP21:%.*]] = extractelement <4 x i1> [[TMP15]], i32 1 ; CHECK-NEXT: [[TMP69:%.*]] = extractelement <4 x i1> [[TMP39]], i32 1
; CHECK-NEXT: br i1 [[TMP21]], label [[PRED_LOAD_IF1:%.*]], label [[PRED_LOAD_CONTINUE2:%.*]] ; CHECK-NEXT: br i1 [[TMP69]], label [[PRED_LOAD_IF4:%.*]], label [[PRED_LOAD_CONTINUE5:%.*]]
; CHECK: pred.load.if1: ; CHECK: pred.load.if4:
; CHECK-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP1]] ; CHECK-NEXT: [[TMP70:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP1]]
; CHECK-NEXT: [[TMP23:%.*]] = load i32, ptr [[TMP22]], align 4 ; CHECK-NEXT: [[TMP71:%.*]] = load i32, ptr [[TMP70]], align 4
; CHECK-NEXT: [[TMP24:%.*]] = insertelement <4 x i32> [[TMP20]], i32 [[TMP23]], i32 1 ; CHECK-NEXT: [[TMP72:%.*]] = insertelement <4 x i32> [[TMP68]], i32 [[TMP71]], i32 1
; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE2]] ; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE5]]
; CHECK: pred.load.continue2: ; CHECK: pred.load.continue5:
; CHECK-NEXT: [[TMP25:%.*]] = phi <4 x i32> [ [[TMP20]], [[PRED_LOAD_CONTINUE]] ], [ [[TMP24]], [[PRED_LOAD_IF1]] ] ; CHECK-NEXT: [[TMP73:%.*]] = phi <4 x i32> [ [[TMP68]], [[PRED_LOAD_CONTINUE]] ], [ [[TMP72]], [[PRED_LOAD_IF4]] ]
; CHECK-NEXT: [[TMP26:%.*]] = extractelement <4 x i1> [[TMP15]], i32 2 ; CHECK-NEXT: [[TMP74:%.*]] = extractelement <4 x i1> [[TMP39]], i32 2
; CHECK-NEXT: br i1 [[TMP26]], label [[PRED_LOAD_IF3:%.*]], label [[PRED_LOAD_CONTINUE4:%.*]] ; CHECK-NEXT: br i1 [[TMP74]], label [[PRED_LOAD_IF6:%.*]], label [[PRED_LOAD_CONTINUE7:%.*]]
; CHECK: pred.load.if3: ; CHECK: pred.load.if6:
; CHECK-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP2]] ; CHECK-NEXT: [[TMP75:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP2]]
; CHECK-NEXT: [[TMP28:%.*]] = load i32, ptr [[TMP27]], align 4 ; CHECK-NEXT: [[TMP76:%.*]] = load i32, ptr [[TMP75]], align 4
; CHECK-NEXT: [[TMP29:%.*]] = insertelement <4 x i32> [[TMP25]], i32 [[TMP28]], i32 2 ; CHECK-NEXT: [[TMP77:%.*]] = insertelement <4 x i32> [[TMP73]], i32 [[TMP76]], i32 2
; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE4]] ; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE7]]
; CHECK: pred.load.continue4: ; CHECK: pred.load.continue7:
; CHECK-NEXT: [[TMP30:%.*]] = phi <4 x i32> [ [[TMP25]], [[PRED_LOAD_CONTINUE2]] ], [ [[TMP29]], [[PRED_LOAD_IF3]] ] ; CHECK-NEXT: [[TMP78:%.*]] = phi <4 x i32> [ [[TMP73]], [[PRED_LOAD_CONTINUE5]] ], [ [[TMP77]], [[PRED_LOAD_IF6]] ]
; CHECK-NEXT: [[TMP31:%.*]] = extractelement <4 x i1> [[TMP15]], i32 3 ; CHECK-NEXT: [[TMP79:%.*]] = extractelement <4 x i1> [[TMP39]], i32 3
; CHECK-NEXT: br i1 [[TMP31]], label [[PRED_LOAD_IF5:%.*]], label [[PRED_LOAD_CONTINUE6]] ; CHECK-NEXT: br i1 [[TMP79]], label [[PRED_LOAD_IF8:%.*]], label [[PRED_LOAD_CONTINUE9:%.*]]
; CHECK: pred.load.if5: ; CHECK: pred.load.if8:
; CHECK-NEXT: [[TMP32:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP3]] ; CHECK-NEXT: [[TMP80:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP3]]
; CHECK-NEXT: [[TMP33:%.*]] = load i32, ptr [[TMP32]], align 4 ; CHECK-NEXT: [[TMP81:%.*]] = load i32, ptr [[TMP80]], align 4
; CHECK-NEXT: [[TMP34:%.*]] = insertelement <4 x i32> [[TMP30]], i32 [[TMP33]], i32 3 ; CHECK-NEXT: [[TMP82:%.*]] = insertelement <4 x i32> [[TMP78]], i32 [[TMP81]], i32 3
; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE6]] ; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE9]]
; CHECK: pred.load.continue6: ; CHECK: pred.load.continue9:
; CHECK-NEXT: [[TMP35:%.*]] = phi <4 x i32> [ [[TMP30]], [[PRED_LOAD_CONTINUE4]] ], [ [[TMP34]], [[PRED_LOAD_IF5]] ] ; CHECK-NEXT: [[TMP83:%.*]] = phi <4 x i32> [ [[TMP78]], [[PRED_LOAD_CONTINUE7]] ], [ [[TMP82]], [[PRED_LOAD_IF8]] ]
; CHECK-NEXT: [[TMP36:%.*]] = xor <4 x i1> [[TMP15]], <i1 true, i1 true, i1 true, i1 true> ; CHECK-NEXT: [[TMP84:%.*]] = extractelement <4 x i1> [[TMP47]], i32 0
; CHECK-NEXT: [[PREDPHI:%.*]] = select <4 x i1> [[TMP15]], <4 x i32> [[TMP35]], <4 x i32> zeroinitializer ; CHECK-NEXT: br i1 [[TMP84]], label [[PRED_LOAD_IF10:%.*]], label [[PRED_LOAD_CONTINUE11:%.*]]
; CHECK-NEXT: [[TMP37]] = add <4 x i32> [[VEC_PHI]], [[PREDPHI]] ; CHECK: pred.load.if10:
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 4 ; CHECK-NEXT: [[TMP85:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP4]]
; CHECK-NEXT: [[TMP38:%.*]] = icmp eq i64 [[INDEX_NEXT]], 52 ; CHECK-NEXT: [[TMP86:%.*]] = load i32, ptr [[TMP85]], align 4
; CHECK-NEXT: br i1 [[TMP38]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP36:![0-9]+]] ; CHECK-NEXT: [[TMP87:%.*]] = insertelement <4 x i32> poison, i32 [[TMP86]], i32 0
; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE11]]
; CHECK: pred.load.continue11:
; CHECK-NEXT: [[TMP88:%.*]] = phi <4 x i32> [ poison, [[PRED_LOAD_CONTINUE9]] ], [ [[TMP87]], [[PRED_LOAD_IF10]] ]
; CHECK-NEXT: [[TMP89:%.*]] = extractelement <4 x i1> [[TMP47]], i32 1
; CHECK-NEXT: br i1 [[TMP89]], label [[PRED_LOAD_IF12:%.*]], label [[PRED_LOAD_CONTINUE13:%.*]]
; CHECK: pred.load.if12:
; CHECK-NEXT: [[TMP90:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP5]]
; CHECK-NEXT: [[TMP91:%.*]] = load i32, ptr [[TMP90]], align 4
; CHECK-NEXT: [[TMP92:%.*]] = insertelement <4 x i32> [[TMP88]], i32 [[TMP91]], i32 1
; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE13]]
; CHECK: pred.load.continue13:
; CHECK-NEXT: [[TMP93:%.*]] = phi <4 x i32> [ [[TMP88]], [[PRED_LOAD_CONTINUE11]] ], [ [[TMP92]], [[PRED_LOAD_IF12]] ]
; CHECK-NEXT: [[TMP94:%.*]] = extractelement <4 x i1> [[TMP47]], i32 2
; CHECK-NEXT: br i1 [[TMP94]], label [[PRED_LOAD_IF14:%.*]], label [[PRED_LOAD_CONTINUE15:%.*]]
; CHECK: pred.load.if14:
; CHECK-NEXT: [[TMP95:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP6]]
; CHECK-NEXT: [[TMP96:%.*]] = load i32, ptr [[TMP95]], align 4
; CHECK-NEXT: [[TMP97:%.*]] = insertelement <4 x i32> [[TMP93]], i32 [[TMP96]], i32 2
; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE15]]
; CHECK: pred.load.continue15:
; CHECK-NEXT: [[TMP98:%.*]] = phi <4 x i32> [ [[TMP93]], [[PRED_LOAD_CONTINUE13]] ], [ [[TMP97]], [[PRED_LOAD_IF14]] ]
; CHECK-NEXT: [[TMP99:%.*]] = extractelement <4 x i1> [[TMP47]], i32 3
; CHECK-NEXT: br i1 [[TMP99]], label [[PRED_LOAD_IF16:%.*]], label [[PRED_LOAD_CONTINUE17:%.*]]
; CHECK: pred.load.if16:
; CHECK-NEXT: [[TMP100:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP7]]
; CHECK-NEXT: [[TMP101:%.*]] = load i32, ptr [[TMP100]], align 4
; CHECK-NEXT: [[TMP102:%.*]] = insertelement <4 x i32> [[TMP98]], i32 [[TMP101]], i32 3
; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE17]]
; CHECK: pred.load.continue17:
; CHECK-NEXT: [[TMP103:%.*]] = phi <4 x i32> [ [[TMP98]], [[PRED_LOAD_CONTINUE15]] ], [ [[TMP102]], [[PRED_LOAD_IF16]] ]
; CHECK-NEXT: [[TMP104:%.*]] = extractelement <4 x i1> [[TMP55]], i32 0
; CHECK-NEXT: br i1 [[TMP104]], label [[PRED_LOAD_IF18:%.*]], label [[PRED_LOAD_CONTINUE19:%.*]]
; CHECK: pred.load.if18:
; CHECK-NEXT: [[TMP105:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP8]]
; CHECK-NEXT: [[TMP106:%.*]] = load i32, ptr [[TMP105]], align 4
; CHECK-NEXT: [[TMP107:%.*]] = insertelement <4 x i32> poison, i32 [[TMP106]], i32 0
; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE19]]
; CHECK: pred.load.continue19:
; CHECK-NEXT: [[TMP108:%.*]] = phi <4 x i32> [ poison, [[PRED_LOAD_CONTINUE17]] ], [ [[TMP107]], [[PRED_LOAD_IF18]] ]
; CHECK-NEXT: [[TMP109:%.*]] = extractelement <4 x i1> [[TMP55]], i32 1
; CHECK-NEXT: br i1 [[TMP109]], label [[PRED_LOAD_IF20:%.*]], label [[PRED_LOAD_CONTINUE21:%.*]]
; CHECK: pred.load.if20:
; CHECK-NEXT: [[TMP110:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP9]]
; CHECK-NEXT: [[TMP111:%.*]] = load i32, ptr [[TMP110]], align 4
; CHECK-NEXT: [[TMP112:%.*]] = insertelement <4 x i32> [[TMP108]], i32 [[TMP111]], i32 1
; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE21]]
; CHECK: pred.load.continue21:
; CHECK-NEXT: [[TMP113:%.*]] = phi <4 x i32> [ [[TMP108]], [[PRED_LOAD_CONTINUE19]] ], [ [[TMP112]], [[PRED_LOAD_IF20]] ]
; CHECK-NEXT: [[TMP114:%.*]] = extractelement <4 x i1> [[TMP55]], i32 2
; CHECK-NEXT: br i1 [[TMP114]], label [[PRED_LOAD_IF22:%.*]], label [[PRED_LOAD_CONTINUE23:%.*]]
; CHECK: pred.load.if22:
; CHECK-NEXT: [[TMP115:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP10]]
; CHECK-NEXT: [[TMP116:%.*]] = load i32, ptr [[TMP115]], align 4
; CHECK-NEXT: [[TMP117:%.*]] = insertelement <4 x i32> [[TMP113]], i32 [[TMP116]], i32 2
; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE23]]
; CHECK: pred.load.continue23:
; CHECK-NEXT: [[TMP118:%.*]] = phi <4 x i32> [ [[TMP113]], [[PRED_LOAD_CONTINUE21]] ], [ [[TMP117]], [[PRED_LOAD_IF22]] ]
; CHECK-NEXT: [[TMP119:%.*]] = extractelement <4 x i1> [[TMP55]], i32 3
; CHECK-NEXT: br i1 [[TMP119]], label [[PRED_LOAD_IF24:%.*]], label [[PRED_LOAD_CONTINUE25:%.*]]
; CHECK: pred.load.if24:
; CHECK-NEXT: [[TMP120:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP11]]
; CHECK-NEXT: [[TMP121:%.*]] = load i32, ptr [[TMP120]], align 4
; CHECK-NEXT: [[TMP122:%.*]] = insertelement <4 x i32> [[TMP118]], i32 [[TMP121]], i32 3
; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE25]]
; CHECK: pred.load.continue25:
; CHECK-NEXT: [[TMP123:%.*]] = phi <4 x i32> [ [[TMP118]], [[PRED_LOAD_CONTINUE23]] ], [ [[TMP122]], [[PRED_LOAD_IF24]] ]
; CHECK-NEXT: [[TMP124:%.*]] = extractelement <4 x i1> [[TMP63]], i32 0
; CHECK-NEXT: br i1 [[TMP124]], label [[PRED_LOAD_IF26:%.*]], label [[PRED_LOAD_CONTINUE27:%.*]]
; CHECK: pred.load.if26:
; CHECK-NEXT: [[TMP125:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP12]]
; CHECK-NEXT: [[TMP126:%.*]] = load i32, ptr [[TMP125]], align 4
; CHECK-NEXT: [[TMP127:%.*]] = insertelement <4 x i32> poison, i32 [[TMP126]], i32 0
; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE27]]
; CHECK: pred.load.continue27:
; CHECK-NEXT: [[TMP128:%.*]] = phi <4 x i32> [ poison, [[PRED_LOAD_CONTINUE25]] ], [ [[TMP127]], [[PRED_LOAD_IF26]] ]
; CHECK-NEXT: [[TMP129:%.*]] = extractelement <4 x i1> [[TMP63]], i32 1
; CHECK-NEXT: br i1 [[TMP129]], label [[PRED_LOAD_IF28:%.*]], label [[PRED_LOAD_CONTINUE29:%.*]]
; CHECK: pred.load.if28:
; CHECK-NEXT: [[TMP130:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP13]]
; CHECK-NEXT: [[TMP131:%.*]] = load i32, ptr [[TMP130]], align 4
; CHECK-NEXT: [[TMP132:%.*]] = insertelement <4 x i32> [[TMP128]], i32 [[TMP131]], i32 1
; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE29]]
; CHECK: pred.load.continue29:
; CHECK-NEXT: [[TMP133:%.*]] = phi <4 x i32> [ [[TMP128]], [[PRED_LOAD_CONTINUE27]] ], [ [[TMP132]], [[PRED_LOAD_IF28]] ]
; CHECK-NEXT: [[TMP134:%.*]] = extractelement <4 x i1> [[TMP63]], i32 2
; CHECK-NEXT: br i1 [[TMP134]], label [[PRED_LOAD_IF30:%.*]], label [[PRED_LOAD_CONTINUE31:%.*]]
; CHECK: pred.load.if30:
; CHECK-NEXT: [[TMP135:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP14]]
; CHECK-NEXT: [[TMP136:%.*]] = load i32, ptr [[TMP135]], align 4
; CHECK-NEXT: [[TMP137:%.*]] = insertelement <4 x i32> [[TMP133]], i32 [[TMP136]], i32 2
; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE31]]
; CHECK: pred.load.continue31:
; CHECK-NEXT: [[TMP138:%.*]] = phi <4 x i32> [ [[TMP133]], [[PRED_LOAD_CONTINUE29]] ], [ [[TMP137]], [[PRED_LOAD_IF30]] ]
; CHECK-NEXT: [[TMP139:%.*]] = extractelement <4 x i1> [[TMP63]], i32 3
; CHECK-NEXT: br i1 [[TMP139]], label [[PRED_LOAD_IF32:%.*]], label [[PRED_LOAD_CONTINUE33]]
; CHECK: pred.load.if32:
; CHECK-NEXT: [[TMP140:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[TMP15]]
; CHECK-NEXT: [[TMP141:%.*]] = load i32, ptr [[TMP140]], align 4
; CHECK-NEXT: [[TMP142:%.*]] = insertelement <4 x i32> [[TMP138]], i32 [[TMP141]], i32 3
; CHECK-NEXT: br label [[PRED_LOAD_CONTINUE33]]
; CHECK: pred.load.continue33:
; CHECK-NEXT: [[TMP143:%.*]] = phi <4 x i32> [ [[TMP138]], [[PRED_LOAD_CONTINUE31]] ], [ [[TMP142]], [[PRED_LOAD_IF32]] ]
; CHECK-NEXT: [[TMP144:%.*]] = xor <4 x i1> [[TMP39]], <i1 true, i1 true, i1 true, i1 true>
; CHECK-NEXT: [[TMP145:%.*]] = xor <4 x i1> [[TMP47]], <i1 true, i1 true, i1 true, i1 true>
; CHECK-NEXT: [[TMP146:%.*]] = xor <4 x i1> [[TMP55]], <i1 true, i1 true, i1 true, i1 true>
; CHECK-NEXT: [[TMP147:%.*]] = xor <4 x i1> [[TMP63]], <i1 true, i1 true, i1 true, i1 true>
; CHECK-NEXT: [[PREDPHI:%.*]] = select <4 x i1> [[TMP39]], <4 x i32> [[TMP83]], <4 x i32> zeroinitializer
; CHECK-NEXT: [[PREDPHI34:%.*]] = select <4 x i1> [[TMP47]], <4 x i32> [[TMP103]], <4 x i32> zeroinitializer
; CHECK-NEXT: [[PREDPHI35:%.*]] = select <4 x i1> [[TMP55]], <4 x i32> [[TMP123]], <4 x i32> zeroinitializer
; CHECK-NEXT: [[PREDPHI36:%.*]] = select <4 x i1> [[TMP63]], <4 x i32> [[TMP143]], <4 x i32> zeroinitializer
; CHECK-NEXT: [[TMP148]] = add <4 x i32> [[VEC_PHI]], [[PREDPHI]]
; CHECK-NEXT: [[TMP149]] = add <4 x i32> [[VEC_PHI1]], [[PREDPHI34]]
; CHECK-NEXT: [[TMP150]] = add <4 x i32> [[VEC_PHI2]], [[PREDPHI35]]
; CHECK-NEXT: [[TMP151]] = add <4 x i32> [[VEC_PHI3]], [[PREDPHI36]]
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 16
; CHECK-NEXT: [[TMP152:%.*]] = icmp eq i64 [[INDEX_NEXT]], 48
; CHECK-NEXT: br i1 [[TMP152]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP36:![0-9]+]]
; CHECK: middle.block: ; CHECK: middle.block:
; CHECK-NEXT: [[TMP39:%.*]] = call i32 @llvm.vector.reduce.add.v4i32(<4 x i32> [[TMP37]]) ; CHECK-NEXT: [[BIN_RDX:%.*]] = add <4 x i32> [[TMP149]], [[TMP148]]
; CHECK-NEXT: br i1 true, label [[LOOP_EXIT:%.*]], label [[SCALAR_PH]] ; CHECK-NEXT: [[BIN_RDX37:%.*]] = add <4 x i32> [[TMP150]], [[BIN_RDX]]
; CHECK-NEXT: [[BIN_RDX38:%.*]] = add <4 x i32> [[TMP151]], [[BIN_RDX37]]
; CHECK-NEXT: [[TMP153:%.*]] = call i32 @llvm.vector.reduce.add.v4i32(<4 x i32> [[BIN_RDX38]])
; CHECK-NEXT: br i1 false, label [[LOOP_EXIT:%.*]], label [[SCALAR_PH]]
; CHECK: scalar.ph: ; CHECK: scalar.ph:
; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 104, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ] ; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 96, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]
; CHECK-NEXT: [[BC_MERGE_RDX:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[TMP39]], [[MIDDLE_BLOCK]] ] ; CHECK-NEXT: [[BC_MERGE_RDX:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[TMP153]], [[MIDDLE_BLOCK]] ]
; CHECK-NEXT: br label [[LOOP:%.*]] ; CHECK-NEXT: br label [[LOOP:%.*]]
; CHECK: loop: ; CHECK: loop:
; CHECK-NEXT: [[IV:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[IV_NEXT:%.*]], [[LATCH:%.*]] ] ; CHECK-NEXT: [[IV:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[IV_NEXT:%.*]], [[LATCH:%.*]] ]
; CHECK-NEXT: [[ACCUM:%.*]] = phi i32 [ [[BC_MERGE_RDX]], [[SCALAR_PH]] ], [ [[ACCUM_NEXT:%.*]], [[LATCH]] ] ; CHECK-NEXT: [[ACCUM:%.*]] = phi i32 [ [[BC_MERGE_RDX]], [[SCALAR_PH]] ], [ [[ACCUM_NEXT:%.*]], [[LATCH]] ]
; CHECK-NEXT: [[IV_NEXT]] = add i64 [[IV]], 2 ; CHECK-NEXT: [[IV_NEXT]] = add i64 [[IV]], 2
; CHECK-NEXT: [[TEST_ADDR:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[IV]] ; CHECK-NEXT: [[TEST_ADDR:%.*]] = getelementptr inbounds i1, ptr [[TEST_BASE]], i64 [[IV]]
; CHECK-NEXT: [[EARLYCND:%.*]] = load i1, ptr [[TEST_ADDR]], align 1 ; CHECK-NEXT: [[EARLYCND:%.*]] = load i1, ptr [[TEST_ADDR]], align 1
; CHECK-NEXT: br i1 [[EARLYCND]], label [[PRED:%.*]], label [[LATCH]] ; CHECK-NEXT: br i1 [[EARLYCND]], label [[PRED:%.*]], label [[LATCH]]
; CHECK: pred: ; CHECK: pred:
; CHECK-NEXT: [[ADDR:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[IV]] ; CHECK-NEXT: [[ADDR:%.*]] = getelementptr inbounds i32, ptr [[ALLOCA]], i64 [[IV]]
; CHECK-NEXT: [[VAL:%.*]] = load i32, ptr [[ADDR]], align 4 ; CHECK-NEXT: [[VAL:%.*]] = load i32, ptr [[ADDR]], align 4
; CHECK-NEXT: br label [[LATCH]] ; CHECK-NEXT: br label [[LATCH]]
; CHECK: latch: ; CHECK: latch:
; CHECK-NEXT: [[VAL_PHI:%.*]] = phi i32 [ 0, [[LOOP]] ], [ [[VAL]], [[PRED]] ] ; CHECK-NEXT: [[VAL_PHI:%.*]] = phi i32 [ 0, [[LOOP]] ], [ [[VAL]], [[PRED]] ]
; CHECK-NEXT: [[ACCUM_NEXT]] = add i32 [[ACCUM]], [[VAL_PHI]] ; CHECK-NEXT: [[ACCUM_NEXT]] = add i32 [[ACCUM]], [[VAL_PHI]]
; CHECK-NEXT: [[EXIT:%.*]] = icmp ugt i64 [[IV]], 100 ; CHECK-NEXT: [[EXIT:%.*]] = icmp ugt i64 [[IV]], 100
; CHECK-NEXT: br i1 [[EXIT]], label [[LOOP_EXIT]], label [[LOOP]], !llvm.loop [[LOOP37:![0-9]+]] ; CHECK-NEXT: br i1 [[EXIT]], label [[LOOP_EXIT]], label [[LOOP]], !llvm.loop [[LOOP37:![0-9]+]]
; CHECK: loop_exit: ; CHECK: loop_exit:
; CHECK-NEXT: [[ACCUM_NEXT_LCSSA:%.*]] = phi i32 [ [[ACCUM_NEXT]], [[LATCH]] ], [ [[TMP39]], [[MIDDLE_BLOCK]] ] ; CHECK-NEXT: [[ACCUM_NEXT_LCSSA:%.*]] = phi i32 [ [[ACCUM_NEXT]], [[LATCH]] ], [ [[TMP153]], [[MIDDLE_BLOCK]] ]
; CHECK-NEXT: ret i32 [[ACCUM_NEXT_LCSSA]] ; CHECK-NEXT: ret i32 [[ACCUM_NEXT_LCSSA]]
; ;
entry: entry:
%alloca = alloca [103 x i32] %alloca = alloca [103 x i32]
call void @init(ptr %alloca) call void @init(ptr %alloca)
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %latch ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %latch ]
▲ Show 20 Lines • Show All 232 Lines • Show Last 20 Lines

llvm/test/Transforms/LoopVectorize/X86/metadata-enable.ll

Show All 16 Lines
target triple = "x86_64-unknown-linux-gnu" target triple = "x86_64-unknown-linux-gnu"
define i32 @enabled(ptr noalias nocapture %a, ptr noalias nocapture readonly %b, i32 %N) { define i32 @enabled(ptr noalias nocapture %a, ptr noalias nocapture readonly %b, i32 %N) {
; O1-LABEL: @enabled( ; O1-LABEL: @enabled(
; O1-NEXT: entry: ; O1-NEXT: entry:
; O1-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0 ; O1-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0
; O1-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer ; O1-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer
; O1-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4 ; O1-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4
; O1-NEXT: [[TMP0:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]] ; O1-NEXT: [[TMP0:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4
; O1-NEXT: store <4 x i32> [[TMP0]], ptr [[A:%.*]], align 4 ; O1-NEXT: [[WIDE_LOAD1:%.*]] = load <4 x i32>, ptr [[TMP0]], align 4
; O1-NEXT: [[TMP1:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4 ; O1-NEXT: [[TMP1:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]]
; O1-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP1]], align 4 ; O1-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1]], [[BROADCAST_SPLAT]]
; O1-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]] ; O1-NEXT: store <4 x i32> [[TMP1]], ptr [[A:%.*]], align 4
; O1-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4 ; O1-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4
; O1-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4 ; O1-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4
; O1-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8 ; O1-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8
; O1-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4 ; O1-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4
; O1-NEXT: [[TMP5:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]] ; O1-NEXT: [[TMP5:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12
; O1-NEXT: [[TMP6:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8 ; O1-NEXT: [[WIDE_LOAD1_1:%.*]] = load <4 x i32>, ptr [[TMP5]], align 4
; O1-NEXT: store <4 x i32> [[TMP5]], ptr [[TMP6]], align 4 ; O1-NEXT: [[TMP6:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]]
; O1-NEXT: [[TMP7:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12 ; O1-NEXT: [[TMP7:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_1]], [[BROADCAST_SPLAT]]
; O1-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP7]], align 4 ; O1-NEXT: [[TMP8:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8
; O1-NEXT: [[TMP8:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]] ; O1-NEXT: store <4 x i32> [[TMP6]], ptr [[TMP8]], align 4
; O1-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12 ; O1-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12
; O1-NEXT: store <4 x i32> [[TMP8]], ptr [[TMP9]], align 4 ; O1-NEXT: store <4 x i32> [[TMP7]], ptr [[TMP9]], align 4
; O1-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16 ; O1-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16
; O1-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4 ; O1-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4
; O1-NEXT: [[TMP11:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]] ; O1-NEXT: [[TMP11:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20
; O1-NEXT: [[TMP12:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16 ; O1-NEXT: [[WIDE_LOAD1_2:%.*]] = load <4 x i32>, ptr [[TMP11]], align 4
; O1-NEXT: store <4 x i32> [[TMP11]], ptr [[TMP12]], align 4 ; O1-NEXT: [[TMP12:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]]
; O1-NEXT: [[TMP13:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20 ; O1-NEXT: [[TMP13:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_2]], [[BROADCAST_SPLAT]]
; O1-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP13]], align 4 ; O1-NEXT: [[TMP14:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16
; O1-NEXT: [[TMP14:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]] ; O1-NEXT: store <4 x i32> [[TMP12]], ptr [[TMP14]], align 4
; O1-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20 ; O1-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20
; O1-NEXT: store <4 x i32> [[TMP14]], ptr [[TMP15]], align 4 ; O1-NEXT: store <4 x i32> [[TMP13]], ptr [[TMP15]], align 4
; O1-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24 ; O1-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24
; O1-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4 ; O1-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4
; O1-NEXT: [[TMP17:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]] ; O1-NEXT: [[TMP17:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28
; O1-NEXT: [[TMP18:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24 ; O1-NEXT: [[WIDE_LOAD1_3:%.*]] = load <4 x i32>, ptr [[TMP17]], align 4
; O1-NEXT: store <4 x i32> [[TMP17]], ptr [[TMP18]], align 4 ; O1-NEXT: [[TMP18:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]]
; O1-NEXT: [[TMP19:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28 ; O1-NEXT: [[TMP19:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_3]], [[BROADCAST_SPLAT]]
; O1-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP19]], align 4 ; O1-NEXT: [[TMP20:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24
; O1-NEXT: [[TMP20:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]] ; O1-NEXT: store <4 x i32> [[TMP18]], ptr [[TMP20]], align 4
; O1-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28 ; O1-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28
; O1-NEXT: store <4 x i32> [[TMP20]], ptr [[TMP21]], align 4 ; O1-NEXT: store <4 x i32> [[TMP19]], ptr [[TMP21]], align 4
; O1-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32 ; O1-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32
; O1-NEXT: [[WIDE_LOAD_8:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4 ; O1-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4
; O1-NEXT: [[TMP23:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_8]], [[BROADCAST_SPLAT]] ; O1-NEXT: [[TMP23:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36
; O1-NEXT: [[TMP24:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32 ; O1-NEXT: [[WIDE_LOAD1_4:%.*]] = load <4 x i32>, ptr [[TMP23]], align 4
; O1-NEXT: store <4 x i32> [[TMP23]], ptr [[TMP24]], align 4 ; O1-NEXT: [[TMP24:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]]
; O1-NEXT: [[TMP25:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36 ; O1-NEXT: [[TMP25:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_4]], [[BROADCAST_SPLAT]]
; O1-NEXT: [[WIDE_LOAD_9:%.*]] = load <4 x i32>, ptr [[TMP25]], align 4 ; O1-NEXT: [[TMP26:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32
; O1-NEXT: [[TMP26:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_9]], [[BROADCAST_SPLAT]] ; O1-NEXT: store <4 x i32> [[TMP24]], ptr [[TMP26]], align 4
; O1-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36 ; O1-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36
; O1-NEXT: store <4 x i32> [[TMP26]], ptr [[TMP27]], align 4 ; O1-NEXT: store <4 x i32> [[TMP25]], ptr [[TMP27]], align 4
; O1-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40 ; O1-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40
; O1-NEXT: [[WIDE_LOAD_10:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4 ; O1-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4
; O1-NEXT: [[TMP29:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_10]], [[BROADCAST_SPLAT]] ; O1-NEXT: [[TMP29:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44
; O1-NEXT: [[TMP30:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40 ; O1-NEXT: [[WIDE_LOAD1_5:%.*]] = load <4 x i32>, ptr [[TMP29]], align 4
; O1-NEXT: store <4 x i32> [[TMP29]], ptr [[TMP30]], align 4 ; O1-NEXT: [[TMP30:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]]
; O1-NEXT: [[TMP31:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44 ; O1-NEXT: [[TMP31:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_5]], [[BROADCAST_SPLAT]]
; O1-NEXT: [[WIDE_LOAD_11:%.*]] = load <4 x i32>, ptr [[TMP31]], align 4 ; O1-NEXT: [[TMP32:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40
; O1-NEXT: [[TMP32:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_11]], [[BROADCAST_SPLAT]] ; O1-NEXT: store <4 x i32> [[TMP30]], ptr [[TMP32]], align 4
; O1-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44 ; O1-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44
; O1-NEXT: store <4 x i32> [[TMP32]], ptr [[TMP33]], align 4 ; O1-NEXT: store <4 x i32> [[TMP31]], ptr [[TMP33]], align 4
; O1-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48 ; O1-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48
; O1-NEXT: [[WIDE_LOAD_12:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4 ; O1-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4
; O1-NEXT: [[TMP35:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_12]], [[BROADCAST_SPLAT]] ; O1-NEXT: [[TMP35:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52
; O1-NEXT: [[TMP36:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48 ; O1-NEXT: [[WIDE_LOAD1_6:%.*]] = load <4 x i32>, ptr [[TMP35]], align 4
; O1-NEXT: store <4 x i32> [[TMP35]], ptr [[TMP36]], align 4 ; O1-NEXT: [[TMP36:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]]
; O1-NEXT: [[TMP37:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52 ; O1-NEXT: [[TMP37:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_6]], [[BROADCAST_SPLAT]]
; O1-NEXT: [[WIDE_LOAD_13:%.*]] = load <4 x i32>, ptr [[TMP37]], align 4 ; O1-NEXT: [[TMP38:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48
; O1-NEXT: [[TMP38:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_13]], [[BROADCAST_SPLAT]] ; O1-NEXT: store <4 x i32> [[TMP36]], ptr [[TMP38]], align 4
; O1-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52 ; O1-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52
; O1-NEXT: store <4 x i32> [[TMP38]], ptr [[TMP39]], align 4 ; O1-NEXT: store <4 x i32> [[TMP37]], ptr [[TMP39]], align 4
; O1-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56 ; O1-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56
; O1-NEXT: [[WIDE_LOAD_14:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4 ; O1-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4
; O1-NEXT: [[TMP41:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_14]], [[BROADCAST_SPLAT]] ; O1-NEXT: [[TMP41:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60
; O1-NEXT: [[TMP42:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56 ; O1-NEXT: [[WIDE_LOAD1_7:%.*]] = load <4 x i32>, ptr [[TMP41]], align 4
; O1-NEXT: store <4 x i32> [[TMP41]], ptr [[TMP42]], align 4 ; O1-NEXT: [[TMP42:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]]
; O1-NEXT: [[TMP43:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60 ; O1-NEXT: [[TMP43:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_7]], [[BROADCAST_SPLAT]]
; O1-NEXT: [[WIDE_LOAD_15:%.*]] = load <4 x i32>, ptr [[TMP43]], align 4 ; O1-NEXT: [[TMP44:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56
; O1-NEXT: [[TMP44:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_15]], [[BROADCAST_SPLAT]] ; O1-NEXT: store <4 x i32> [[TMP42]], ptr [[TMP44]], align 4
; O1-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60 ; O1-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60
; O1-NEXT: store <4 x i32> [[TMP44]], ptr [[TMP45]], align 4 ; O1-NEXT: store <4 x i32> [[TMP43]], ptr [[TMP45]], align 4
; O1-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4 ; O1-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4
; O1-NEXT: ret i32 [[TMP46]] ; O1-NEXT: ret i32 [[TMP46]]
; ;
; O2-LABEL: @enabled( ; O2-LABEL: @enabled(
; O2-NEXT: entry: ; O2-NEXT: entry:
; O2-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0 ; O2-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0
; O2-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer ; O2-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer
; O2-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4 ; O2-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4
; O2-NEXT: [[TMP0:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]] ; O2-NEXT: [[TMP0:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4
; O2-NEXT: store <4 x i32> [[TMP0]], ptr [[A:%.*]], align 4 ; O2-NEXT: [[WIDE_LOAD1:%.*]] = load <4 x i32>, ptr [[TMP0]], align 4
; O2-NEXT: [[TMP1:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4 ; O2-NEXT: [[TMP1:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP1]], align 4 ; O2-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]] ; O2-NEXT: store <4 x i32> [[TMP1]], ptr [[A:%.*]], align 4
; O2-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4 ; O2-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4
; O2-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4 ; O2-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4
; O2-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8 ; O2-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8
; O2-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4 ; O2-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4
; O2-NEXT: [[TMP5:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]] ; O2-NEXT: [[TMP5:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12
; O2-NEXT: [[TMP6:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8 ; O2-NEXT: [[WIDE_LOAD1_1:%.*]] = load <4 x i32>, ptr [[TMP5]], align 4
; O2-NEXT: store <4 x i32> [[TMP5]], ptr [[TMP6]], align 4 ; O2-NEXT: [[TMP6:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[TMP7:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12 ; O2-NEXT: [[TMP7:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_1]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP7]], align 4 ; O2-NEXT: [[TMP8:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8
; O2-NEXT: [[TMP8:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]] ; O2-NEXT: store <4 x i32> [[TMP6]], ptr [[TMP8]], align 4
; O2-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12 ; O2-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12
; O2-NEXT: store <4 x i32> [[TMP8]], ptr [[TMP9]], align 4 ; O2-NEXT: store <4 x i32> [[TMP7]], ptr [[TMP9]], align 4
; O2-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16 ; O2-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16
; O2-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4 ; O2-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4
; O2-NEXT: [[TMP11:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]] ; O2-NEXT: [[TMP11:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20
; O2-NEXT: [[TMP12:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16 ; O2-NEXT: [[WIDE_LOAD1_2:%.*]] = load <4 x i32>, ptr [[TMP11]], align 4
; O2-NEXT: store <4 x i32> [[TMP11]], ptr [[TMP12]], align 4 ; O2-NEXT: [[TMP12:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[TMP13:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20 ; O2-NEXT: [[TMP13:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_2]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP13]], align 4 ; O2-NEXT: [[TMP14:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16
; O2-NEXT: [[TMP14:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]] ; O2-NEXT: store <4 x i32> [[TMP12]], ptr [[TMP14]], align 4
; O2-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20 ; O2-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20
; O2-NEXT: store <4 x i32> [[TMP14]], ptr [[TMP15]], align 4 ; O2-NEXT: store <4 x i32> [[TMP13]], ptr [[TMP15]], align 4
; O2-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24 ; O2-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24
; O2-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4 ; O2-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4
; O2-NEXT: [[TMP17:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]] ; O2-NEXT: [[TMP17:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28
; O2-NEXT: [[TMP18:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24 ; O2-NEXT: [[WIDE_LOAD1_3:%.*]] = load <4 x i32>, ptr [[TMP17]], align 4
; O2-NEXT: store <4 x i32> [[TMP17]], ptr [[TMP18]], align 4 ; O2-NEXT: [[TMP18:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[TMP19:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28 ; O2-NEXT: [[TMP19:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_3]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP19]], align 4 ; O2-NEXT: [[TMP20:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24
; O2-NEXT: [[TMP20:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]] ; O2-NEXT: store <4 x i32> [[TMP18]], ptr [[TMP20]], align 4
; O2-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28 ; O2-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28
; O2-NEXT: store <4 x i32> [[TMP20]], ptr [[TMP21]], align 4 ; O2-NEXT: store <4 x i32> [[TMP19]], ptr [[TMP21]], align 4
; O2-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32 ; O2-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32
; O2-NEXT: [[WIDE_LOAD_8:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4 ; O2-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4
; O2-NEXT: [[TMP23:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_8]], [[BROADCAST_SPLAT]] ; O2-NEXT: [[TMP23:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36
; O2-NEXT: [[TMP24:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32 ; O2-NEXT: [[WIDE_LOAD1_4:%.*]] = load <4 x i32>, ptr [[TMP23]], align 4
; O2-NEXT: store <4 x i32> [[TMP23]], ptr [[TMP24]], align 4 ; O2-NEXT: [[TMP24:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[TMP25:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36 ; O2-NEXT: [[TMP25:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_4]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[WIDE_LOAD_9:%.*]] = load <4 x i32>, ptr [[TMP25]], align 4 ; O2-NEXT: [[TMP26:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32
; O2-NEXT: [[TMP26:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_9]], [[BROADCAST_SPLAT]] ; O2-NEXT: store <4 x i32> [[TMP24]], ptr [[TMP26]], align 4
; O2-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36 ; O2-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36
; O2-NEXT: store <4 x i32> [[TMP26]], ptr [[TMP27]], align 4 ; O2-NEXT: store <4 x i32> [[TMP25]], ptr [[TMP27]], align 4
; O2-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40 ; O2-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40
; O2-NEXT: [[WIDE_LOAD_10:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4 ; O2-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4
; O2-NEXT: [[TMP29:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_10]], [[BROADCAST_SPLAT]] ; O2-NEXT: [[TMP29:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44
; O2-NEXT: [[TMP30:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40 ; O2-NEXT: [[WIDE_LOAD1_5:%.*]] = load <4 x i32>, ptr [[TMP29]], align 4
; O2-NEXT: store <4 x i32> [[TMP29]], ptr [[TMP30]], align 4 ; O2-NEXT: [[TMP30:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[TMP31:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44 ; O2-NEXT: [[TMP31:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_5]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[WIDE_LOAD_11:%.*]] = load <4 x i32>, ptr [[TMP31]], align 4 ; O2-NEXT: [[TMP32:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40
; O2-NEXT: [[TMP32:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_11]], [[BROADCAST_SPLAT]] ; O2-NEXT: store <4 x i32> [[TMP30]], ptr [[TMP32]], align 4
; O2-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44 ; O2-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44
; O2-NEXT: store <4 x i32> [[TMP32]], ptr [[TMP33]], align 4 ; O2-NEXT: store <4 x i32> [[TMP31]], ptr [[TMP33]], align 4
; O2-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48 ; O2-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48
; O2-NEXT: [[WIDE_LOAD_12:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4 ; O2-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4
; O2-NEXT: [[TMP35:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_12]], [[BROADCAST_SPLAT]] ; O2-NEXT: [[TMP35:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52
; O2-NEXT: [[TMP36:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48 ; O2-NEXT: [[WIDE_LOAD1_6:%.*]] = load <4 x i32>, ptr [[TMP35]], align 4
; O2-NEXT: store <4 x i32> [[TMP35]], ptr [[TMP36]], align 4 ; O2-NEXT: [[TMP36:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[TMP37:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52 ; O2-NEXT: [[TMP37:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_6]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[WIDE_LOAD_13:%.*]] = load <4 x i32>, ptr [[TMP37]], align 4 ; O2-NEXT: [[TMP38:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48
; O2-NEXT: [[TMP38:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_13]], [[BROADCAST_SPLAT]] ; O2-NEXT: store <4 x i32> [[TMP36]], ptr [[TMP38]], align 4
; O2-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52 ; O2-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52
; O2-NEXT: store <4 x i32> [[TMP38]], ptr [[TMP39]], align 4 ; O2-NEXT: store <4 x i32> [[TMP37]], ptr [[TMP39]], align 4
; O2-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56 ; O2-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56
; O2-NEXT: [[WIDE_LOAD_14:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4 ; O2-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4
; O2-NEXT: [[TMP41:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_14]], [[BROADCAST_SPLAT]] ; O2-NEXT: [[TMP41:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60
; O2-NEXT: [[TMP42:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56 ; O2-NEXT: [[WIDE_LOAD1_7:%.*]] = load <4 x i32>, ptr [[TMP41]], align 4
; O2-NEXT: store <4 x i32> [[TMP41]], ptr [[TMP42]], align 4 ; O2-NEXT: [[TMP42:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[TMP43:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60 ; O2-NEXT: [[TMP43:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_7]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[WIDE_LOAD_15:%.*]] = load <4 x i32>, ptr [[TMP43]], align 4 ; O2-NEXT: [[TMP44:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56
; O2-NEXT: [[TMP44:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_15]], [[BROADCAST_SPLAT]] ; O2-NEXT: store <4 x i32> [[TMP42]], ptr [[TMP44]], align 4
; O2-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60 ; O2-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60
; O2-NEXT: store <4 x i32> [[TMP44]], ptr [[TMP45]], align 4 ; O2-NEXT: store <4 x i32> [[TMP43]], ptr [[TMP45]], align 4
; O2-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4 ; O2-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4
; O2-NEXT: ret i32 [[TMP46]] ; O2-NEXT: ret i32 [[TMP46]]
; ;
; O3-LABEL: @enabled( ; O3-LABEL: @enabled(
; O3-NEXT: entry: ; O3-NEXT: entry:
; O3-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0 ; O3-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0
; O3-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer ; O3-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer
; O3-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4 ; O3-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4
; O3-NEXT: [[TMP0:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]] ; O3-NEXT: [[TMP0:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4
; O3-NEXT: store <4 x i32> [[TMP0]], ptr [[A:%.*]], align 4 ; O3-NEXT: [[WIDE_LOAD1:%.*]] = load <4 x i32>, ptr [[TMP0]], align 4
; O3-NEXT: [[TMP1:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4 ; O3-NEXT: [[TMP1:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP1]], align 4 ; O3-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]] ; O3-NEXT: store <4 x i32> [[TMP1]], ptr [[A:%.*]], align 4
; O3-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4 ; O3-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4
; O3-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4 ; O3-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4
; O3-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8 ; O3-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8
; O3-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4 ; O3-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4
; O3-NEXT: [[TMP5:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]] ; O3-NEXT: [[TMP5:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12
; O3-NEXT: [[TMP6:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8 ; O3-NEXT: [[WIDE_LOAD1_1:%.*]] = load <4 x i32>, ptr [[TMP5]], align 4
; O3-NEXT: store <4 x i32> [[TMP5]], ptr [[TMP6]], align 4 ; O3-NEXT: [[TMP6:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[TMP7:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12 ; O3-NEXT: [[TMP7:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_1]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP7]], align 4 ; O3-NEXT: [[TMP8:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8
; O3-NEXT: [[TMP8:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]] ; O3-NEXT: store <4 x i32> [[TMP6]], ptr [[TMP8]], align 4
; O3-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12 ; O3-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12
; O3-NEXT: store <4 x i32> [[TMP8]], ptr [[TMP9]], align 4 ; O3-NEXT: store <4 x i32> [[TMP7]], ptr [[TMP9]], align 4
; O3-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16 ; O3-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16
; O3-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4 ; O3-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4
; O3-NEXT: [[TMP11:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]] ; O3-NEXT: [[TMP11:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20
; O3-NEXT: [[TMP12:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16 ; O3-NEXT: [[WIDE_LOAD1_2:%.*]] = load <4 x i32>, ptr [[TMP11]], align 4
; O3-NEXT: store <4 x i32> [[TMP11]], ptr [[TMP12]], align 4 ; O3-NEXT: [[TMP12:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[TMP13:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20 ; O3-NEXT: [[TMP13:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_2]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP13]], align 4 ; O3-NEXT: [[TMP14:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16
; O3-NEXT: [[TMP14:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]] ; O3-NEXT: store <4 x i32> [[TMP12]], ptr [[TMP14]], align 4
; O3-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20 ; O3-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20
; O3-NEXT: store <4 x i32> [[TMP14]], ptr [[TMP15]], align 4 ; O3-NEXT: store <4 x i32> [[TMP13]], ptr [[TMP15]], align 4
; O3-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24 ; O3-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24
; O3-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4 ; O3-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4
; O3-NEXT: [[TMP17:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]] ; O3-NEXT: [[TMP17:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28
; O3-NEXT: [[TMP18:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24 ; O3-NEXT: [[WIDE_LOAD1_3:%.*]] = load <4 x i32>, ptr [[TMP17]], align 4
; O3-NEXT: store <4 x i32> [[TMP17]], ptr [[TMP18]], align 4 ; O3-NEXT: [[TMP18:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[TMP19:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28 ; O3-NEXT: [[TMP19:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_3]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP19]], align 4 ; O3-NEXT: [[TMP20:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24
; O3-NEXT: [[TMP20:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]] ; O3-NEXT: store <4 x i32> [[TMP18]], ptr [[TMP20]], align 4
; O3-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28 ; O3-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28
; O3-NEXT: store <4 x i32> [[TMP20]], ptr [[TMP21]], align 4 ; O3-NEXT: store <4 x i32> [[TMP19]], ptr [[TMP21]], align 4
; O3-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32 ; O3-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32
; O3-NEXT: [[WIDE_LOAD_8:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4 ; O3-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4
; O3-NEXT: [[TMP23:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_8]], [[BROADCAST_SPLAT]] ; O3-NEXT: [[TMP23:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36
; O3-NEXT: [[TMP24:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32 ; O3-NEXT: [[WIDE_LOAD1_4:%.*]] = load <4 x i32>, ptr [[TMP23]], align 4
; O3-NEXT: store <4 x i32> [[TMP23]], ptr [[TMP24]], align 4 ; O3-NEXT: [[TMP24:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[TMP25:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36 ; O3-NEXT: [[TMP25:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_4]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[WIDE_LOAD_9:%.*]] = load <4 x i32>, ptr [[TMP25]], align 4 ; O3-NEXT: [[TMP26:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32
; O3-NEXT: [[TMP26:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_9]], [[BROADCAST_SPLAT]] ; O3-NEXT: store <4 x i32> [[TMP24]], ptr [[TMP26]], align 4
; O3-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36 ; O3-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36
; O3-NEXT: store <4 x i32> [[TMP26]], ptr [[TMP27]], align 4 ; O3-NEXT: store <4 x i32> [[TMP25]], ptr [[TMP27]], align 4
; O3-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40 ; O3-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40
; O3-NEXT: [[WIDE_LOAD_10:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4 ; O3-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4
; O3-NEXT: [[TMP29:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_10]], [[BROADCAST_SPLAT]] ; O3-NEXT: [[TMP29:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44
; O3-NEXT: [[TMP30:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40 ; O3-NEXT: [[WIDE_LOAD1_5:%.*]] = load <4 x i32>, ptr [[TMP29]], align 4
; O3-NEXT: store <4 x i32> [[TMP29]], ptr [[TMP30]], align 4 ; O3-NEXT: [[TMP30:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[TMP31:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44 ; O3-NEXT: [[TMP31:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_5]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[WIDE_LOAD_11:%.*]] = load <4 x i32>, ptr [[TMP31]], align 4 ; O3-NEXT: [[TMP32:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40
; O3-NEXT: [[TMP32:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_11]], [[BROADCAST_SPLAT]] ; O3-NEXT: store <4 x i32> [[TMP30]], ptr [[TMP32]], align 4
; O3-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44 ; O3-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44
; O3-NEXT: store <4 x i32> [[TMP32]], ptr [[TMP33]], align 4 ; O3-NEXT: store <4 x i32> [[TMP31]], ptr [[TMP33]], align 4
; O3-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48 ; O3-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48
; O3-NEXT: [[WIDE_LOAD_12:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4 ; O3-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4
; O3-NEXT: [[TMP35:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_12]], [[BROADCAST_SPLAT]] ; O3-NEXT: [[TMP35:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52
; O3-NEXT: [[TMP36:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48 ; O3-NEXT: [[WIDE_LOAD1_6:%.*]] = load <4 x i32>, ptr [[TMP35]], align 4
; O3-NEXT: store <4 x i32> [[TMP35]], ptr [[TMP36]], align 4 ; O3-NEXT: [[TMP36:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[TMP37:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52 ; O3-NEXT: [[TMP37:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_6]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[WIDE_LOAD_13:%.*]] = load <4 x i32>, ptr [[TMP37]], align 4 ; O3-NEXT: [[TMP38:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48
; O3-NEXT: [[TMP38:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_13]], [[BROADCAST_SPLAT]] ; O3-NEXT: store <4 x i32> [[TMP36]], ptr [[TMP38]], align 4
; O3-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52 ; O3-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52
; O3-NEXT: store <4 x i32> [[TMP38]], ptr [[TMP39]], align 4 ; O3-NEXT: store <4 x i32> [[TMP37]], ptr [[TMP39]], align 4
; O3-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56 ; O3-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56
; O3-NEXT: [[WIDE_LOAD_14:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4 ; O3-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4
; O3-NEXT: [[TMP41:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_14]], [[BROADCAST_SPLAT]] ; O3-NEXT: [[TMP41:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60
; O3-NEXT: [[TMP42:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56 ; O3-NEXT: [[WIDE_LOAD1_7:%.*]] = load <4 x i32>, ptr [[TMP41]], align 4
; O3-NEXT: store <4 x i32> [[TMP41]], ptr [[TMP42]], align 4 ; O3-NEXT: [[TMP42:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[TMP43:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60 ; O3-NEXT: [[TMP43:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_7]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[WIDE_LOAD_15:%.*]] = load <4 x i32>, ptr [[TMP43]], align 4 ; O3-NEXT: [[TMP44:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56
; O3-NEXT: [[TMP44:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_15]], [[BROADCAST_SPLAT]] ; O3-NEXT: store <4 x i32> [[TMP42]], ptr [[TMP44]], align 4
; O3-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60 ; O3-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60
; O3-NEXT: store <4 x i32> [[TMP44]], ptr [[TMP45]], align 4 ; O3-NEXT: store <4 x i32> [[TMP43]], ptr [[TMP45]], align 4
; O3-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4 ; O3-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4
; O3-NEXT: ret i32 [[TMP46]] ; O3-NEXT: ret i32 [[TMP46]]
; ;
; O3DEFAULT-LABEL: @enabled( ; O3DEFAULT-LABEL: @enabled(
; O3DEFAULT-NEXT: entry: ; O3DEFAULT-NEXT: entry:
; O3DEFAULT-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0 ; O3DEFAULT-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0
; O3DEFAULT-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer ; O3DEFAULT-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer
; O3DEFAULT-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4 ; O3DEFAULT-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4
; O3DEFAULT-NEXT: [[TMP0:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: [[TMP0:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4
; O3DEFAULT-NEXT: store <4 x i32> [[TMP0]], ptr [[A:%.*]], align 4 ; O3DEFAULT-NEXT: [[WIDE_LOAD1:%.*]] = load <4 x i32>, ptr [[TMP0]], align 4
; O3DEFAULT-NEXT: [[TMP1:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4 ; O3DEFAULT-NEXT: [[TMP1:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP1]], align 4 ; O3DEFAULT-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: store <4 x i32> [[TMP1]], ptr [[A:%.*]], align 4
; O3DEFAULT-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4 ; O3DEFAULT-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4
; O3DEFAULT-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4 ; O3DEFAULT-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4
; O3DEFAULT-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8 ; O3DEFAULT-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8
; O3DEFAULT-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4 ; O3DEFAULT-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4
; O3DEFAULT-NEXT: [[TMP5:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: [[TMP5:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12
; O3DEFAULT-NEXT: [[TMP6:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8 ; O3DEFAULT-NEXT: [[WIDE_LOAD1_1:%.*]] = load <4 x i32>, ptr [[TMP5]], align 4
; O3DEFAULT-NEXT: store <4 x i32> [[TMP5]], ptr [[TMP6]], align 4 ; O3DEFAULT-NEXT: [[TMP6:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[TMP7:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12 ; O3DEFAULT-NEXT: [[TMP7:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_1]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP7]], align 4 ; O3DEFAULT-NEXT: [[TMP8:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8
; O3DEFAULT-NEXT: [[TMP8:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: store <4 x i32> [[TMP6]], ptr [[TMP8]], align 4
; O3DEFAULT-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12 ; O3DEFAULT-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12
; O3DEFAULT-NEXT: store <4 x i32> [[TMP8]], ptr [[TMP9]], align 4 ; O3DEFAULT-NEXT: store <4 x i32> [[TMP7]], ptr [[TMP9]], align 4
; O3DEFAULT-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16 ; O3DEFAULT-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16
; O3DEFAULT-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4 ; O3DEFAULT-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4
; O3DEFAULT-NEXT: [[TMP11:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: [[TMP11:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20
; O3DEFAULT-NEXT: [[TMP12:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16 ; O3DEFAULT-NEXT: [[WIDE_LOAD1_2:%.*]] = load <4 x i32>, ptr [[TMP11]], align 4
; O3DEFAULT-NEXT: store <4 x i32> [[TMP11]], ptr [[TMP12]], align 4 ; O3DEFAULT-NEXT: [[TMP12:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[TMP13:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20 ; O3DEFAULT-NEXT: [[TMP13:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_2]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP13]], align 4 ; O3DEFAULT-NEXT: [[TMP14:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16
; O3DEFAULT-NEXT: [[TMP14:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: store <4 x i32> [[TMP12]], ptr [[TMP14]], align 4
; O3DEFAULT-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20 ; O3DEFAULT-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20
; O3DEFAULT-NEXT: store <4 x i32> [[TMP14]], ptr [[TMP15]], align 4 ; O3DEFAULT-NEXT: store <4 x i32> [[TMP13]], ptr [[TMP15]], align 4
; O3DEFAULT-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24 ; O3DEFAULT-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24
; O3DEFAULT-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4 ; O3DEFAULT-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4
; O3DEFAULT-NEXT: [[TMP17:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: [[TMP17:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28
; O3DEFAULT-NEXT: [[TMP18:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24 ; O3DEFAULT-NEXT: [[WIDE_LOAD1_3:%.*]] = load <4 x i32>, ptr [[TMP17]], align 4
; O3DEFAULT-NEXT: store <4 x i32> [[TMP17]], ptr [[TMP18]], align 4 ; O3DEFAULT-NEXT: [[TMP18:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[TMP19:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28 ; O3DEFAULT-NEXT: [[TMP19:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_3]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP19]], align 4 ; O3DEFAULT-NEXT: [[TMP20:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24
; O3DEFAULT-NEXT: [[TMP20:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: store <4 x i32> [[TMP18]], ptr [[TMP20]], align 4
; O3DEFAULT-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28 ; O3DEFAULT-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28
; O3DEFAULT-NEXT: store <4 x i32> [[TMP20]], ptr [[TMP21]], align 4 ; O3DEFAULT-NEXT: store <4 x i32> [[TMP19]], ptr [[TMP21]], align 4
; O3DEFAULT-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32 ; O3DEFAULT-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32
; O3DEFAULT-NEXT: [[WIDE_LOAD_8:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4 ; O3DEFAULT-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4
; O3DEFAULT-NEXT: [[TMP23:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_8]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: [[TMP23:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36
; O3DEFAULT-NEXT: [[TMP24:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32 ; O3DEFAULT-NEXT: [[WIDE_LOAD1_4:%.*]] = load <4 x i32>, ptr [[TMP23]], align 4
; O3DEFAULT-NEXT: store <4 x i32> [[TMP23]], ptr [[TMP24]], align 4 ; O3DEFAULT-NEXT: [[TMP24:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[TMP25:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36 ; O3DEFAULT-NEXT: [[TMP25:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_4]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[WIDE_LOAD_9:%.*]] = load <4 x i32>, ptr [[TMP25]], align 4 ; O3DEFAULT-NEXT: [[TMP26:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32
; O3DEFAULT-NEXT: [[TMP26:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_9]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: store <4 x i32> [[TMP24]], ptr [[TMP26]], align 4
; O3DEFAULT-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36 ; O3DEFAULT-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36
; O3DEFAULT-NEXT: store <4 x i32> [[TMP26]], ptr [[TMP27]], align 4 ; O3DEFAULT-NEXT: store <4 x i32> [[TMP25]], ptr [[TMP27]], align 4
; O3DEFAULT-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40 ; O3DEFAULT-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40
; O3DEFAULT-NEXT: [[WIDE_LOAD_10:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4 ; O3DEFAULT-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4
; O3DEFAULT-NEXT: [[TMP29:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_10]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: [[TMP29:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44
; O3DEFAULT-NEXT: [[TMP30:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40 ; O3DEFAULT-NEXT: [[WIDE_LOAD1_5:%.*]] = load <4 x i32>, ptr [[TMP29]], align 4
; O3DEFAULT-NEXT: store <4 x i32> [[TMP29]], ptr [[TMP30]], align 4 ; O3DEFAULT-NEXT: [[TMP30:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[TMP31:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44 ; O3DEFAULT-NEXT: [[TMP31:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_5]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[WIDE_LOAD_11:%.*]] = load <4 x i32>, ptr [[TMP31]], align 4 ; O3DEFAULT-NEXT: [[TMP32:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40
; O3DEFAULT-NEXT: [[TMP32:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_11]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: store <4 x i32> [[TMP30]], ptr [[TMP32]], align 4
; O3DEFAULT-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44 ; O3DEFAULT-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44
; O3DEFAULT-NEXT: store <4 x i32> [[TMP32]], ptr [[TMP33]], align 4 ; O3DEFAULT-NEXT: store <4 x i32> [[TMP31]], ptr [[TMP33]], align 4
; O3DEFAULT-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48 ; O3DEFAULT-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48
; O3DEFAULT-NEXT: [[WIDE_LOAD_12:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4 ; O3DEFAULT-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4
; O3DEFAULT-NEXT: [[TMP35:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_12]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: [[TMP35:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52
; O3DEFAULT-NEXT: [[TMP36:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48 ; O3DEFAULT-NEXT: [[WIDE_LOAD1_6:%.*]] = load <4 x i32>, ptr [[TMP35]], align 4
; O3DEFAULT-NEXT: store <4 x i32> [[TMP35]], ptr [[TMP36]], align 4 ; O3DEFAULT-NEXT: [[TMP36:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[TMP37:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52 ; O3DEFAULT-NEXT: [[TMP37:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_6]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[WIDE_LOAD_13:%.*]] = load <4 x i32>, ptr [[TMP37]], align 4 ; O3DEFAULT-NEXT: [[TMP38:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48
; O3DEFAULT-NEXT: [[TMP38:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_13]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: store <4 x i32> [[TMP36]], ptr [[TMP38]], align 4
; O3DEFAULT-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52 ; O3DEFAULT-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52
; O3DEFAULT-NEXT: store <4 x i32> [[TMP38]], ptr [[TMP39]], align 4 ; O3DEFAULT-NEXT: store <4 x i32> [[TMP37]], ptr [[TMP39]], align 4
; O3DEFAULT-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56 ; O3DEFAULT-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56
; O3DEFAULT-NEXT: [[WIDE_LOAD_14:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4 ; O3DEFAULT-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4
; O3DEFAULT-NEXT: [[TMP41:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_14]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: [[TMP41:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60
; O3DEFAULT-NEXT: [[TMP42:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56 ; O3DEFAULT-NEXT: [[WIDE_LOAD1_7:%.*]] = load <4 x i32>, ptr [[TMP41]], align 4
; O3DEFAULT-NEXT: store <4 x i32> [[TMP41]], ptr [[TMP42]], align 4 ; O3DEFAULT-NEXT: [[TMP42:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[TMP43:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60 ; O3DEFAULT-NEXT: [[TMP43:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_7]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[WIDE_LOAD_15:%.*]] = load <4 x i32>, ptr [[TMP43]], align 4 ; O3DEFAULT-NEXT: [[TMP44:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56
; O3DEFAULT-NEXT: [[TMP44:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_15]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: store <4 x i32> [[TMP42]], ptr [[TMP44]], align 4
; O3DEFAULT-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60 ; O3DEFAULT-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60
; O3DEFAULT-NEXT: store <4 x i32> [[TMP44]], ptr [[TMP45]], align 4 ; O3DEFAULT-NEXT: store <4 x i32> [[TMP43]], ptr [[TMP45]], align 4
; O3DEFAULT-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4 ; O3DEFAULT-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4
; O3DEFAULT-NEXT: ret i32 [[TMP46]] ; O3DEFAULT-NEXT: ret i32 [[TMP46]]
; ;
; Os-LABEL: @enabled( ; Os-LABEL: @enabled(
; Os-NEXT: entry: ; Os-NEXT: entry:
; Os-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0 ; Os-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0
; Os-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer ; Os-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer
; Os-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4 ; Os-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4
; Os-NEXT: [[TMP0:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]] ; Os-NEXT: [[TMP0:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4
; Os-NEXT: store <4 x i32> [[TMP0]], ptr [[A:%.*]], align 4 ; Os-NEXT: [[WIDE_LOAD1:%.*]] = load <4 x i32>, ptr [[TMP0]], align 4
; Os-NEXT: [[TMP1:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4 ; Os-NEXT: [[TMP1:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP1]], align 4 ; Os-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]] ; Os-NEXT: store <4 x i32> [[TMP1]], ptr [[A:%.*]], align 4
; Os-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4 ; Os-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4
; Os-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4 ; Os-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4
; Os-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8 ; Os-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8
; Os-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4 ; Os-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4
; Os-NEXT: [[TMP5:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]] ; Os-NEXT: [[TMP5:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12
; Os-NEXT: [[TMP6:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8 ; Os-NEXT: [[WIDE_LOAD1_1:%.*]] = load <4 x i32>, ptr [[TMP5]], align 4
; Os-NEXT: store <4 x i32> [[TMP5]], ptr [[TMP6]], align 4 ; Os-NEXT: [[TMP6:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[TMP7:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12 ; Os-NEXT: [[TMP7:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_1]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP7]], align 4 ; Os-NEXT: [[TMP8:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8
; Os-NEXT: [[TMP8:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]] ; Os-NEXT: store <4 x i32> [[TMP6]], ptr [[TMP8]], align 4
; Os-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12 ; Os-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12
; Os-NEXT: store <4 x i32> [[TMP8]], ptr [[TMP9]], align 4 ; Os-NEXT: store <4 x i32> [[TMP7]], ptr [[TMP9]], align 4
; Os-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16 ; Os-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16
; Os-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4 ; Os-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4
; Os-NEXT: [[TMP11:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]] ; Os-NEXT: [[TMP11:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20
; Os-NEXT: [[TMP12:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16 ; Os-NEXT: [[WIDE_LOAD1_2:%.*]] = load <4 x i32>, ptr [[TMP11]], align 4
; Os-NEXT: store <4 x i32> [[TMP11]], ptr [[TMP12]], align 4 ; Os-NEXT: [[TMP12:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[TMP13:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20 ; Os-NEXT: [[TMP13:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_2]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP13]], align 4 ; Os-NEXT: [[TMP14:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16
; Os-NEXT: [[TMP14:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]] ; Os-NEXT: store <4 x i32> [[TMP12]], ptr [[TMP14]], align 4
; Os-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20 ; Os-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20
; Os-NEXT: store <4 x i32> [[TMP14]], ptr [[TMP15]], align 4 ; Os-NEXT: store <4 x i32> [[TMP13]], ptr [[TMP15]], align 4
; Os-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24 ; Os-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24
; Os-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4 ; Os-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4
; Os-NEXT: [[TMP17:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]] ; Os-NEXT: [[TMP17:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28
; Os-NEXT: [[TMP18:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24 ; Os-NEXT: [[WIDE_LOAD1_3:%.*]] = load <4 x i32>, ptr [[TMP17]], align 4
; Os-NEXT: store <4 x i32> [[TMP17]], ptr [[TMP18]], align 4 ; Os-NEXT: [[TMP18:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[TMP19:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28 ; Os-NEXT: [[TMP19:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_3]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP19]], align 4 ; Os-NEXT: [[TMP20:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24
; Os-NEXT: [[TMP20:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]] ; Os-NEXT: store <4 x i32> [[TMP18]], ptr [[TMP20]], align 4
; Os-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28 ; Os-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28
; Os-NEXT: store <4 x i32> [[TMP20]], ptr [[TMP21]], align 4 ; Os-NEXT: store <4 x i32> [[TMP19]], ptr [[TMP21]], align 4
; Os-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32 ; Os-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32
; Os-NEXT: [[WIDE_LOAD_8:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4 ; Os-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4
; Os-NEXT: [[TMP23:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_8]], [[BROADCAST_SPLAT]] ; Os-NEXT: [[TMP23:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36
; Os-NEXT: [[TMP24:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32 ; Os-NEXT: [[WIDE_LOAD1_4:%.*]] = load <4 x i32>, ptr [[TMP23]], align 4
; Os-NEXT: store <4 x i32> [[TMP23]], ptr [[TMP24]], align 4 ; Os-NEXT: [[TMP24:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[TMP25:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36 ; Os-NEXT: [[TMP25:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_4]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[WIDE_LOAD_9:%.*]] = load <4 x i32>, ptr [[TMP25]], align 4 ; Os-NEXT: [[TMP26:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32
; Os-NEXT: [[TMP26:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_9]], [[BROADCAST_SPLAT]] ; Os-NEXT: store <4 x i32> [[TMP24]], ptr [[TMP26]], align 4
; Os-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36 ; Os-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36
; Os-NEXT: store <4 x i32> [[TMP26]], ptr [[TMP27]], align 4 ; Os-NEXT: store <4 x i32> [[TMP25]], ptr [[TMP27]], align 4
; Os-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40 ; Os-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40
; Os-NEXT: [[WIDE_LOAD_10:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4 ; Os-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4
; Os-NEXT: [[TMP29:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_10]], [[BROADCAST_SPLAT]] ; Os-NEXT: [[TMP29:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44
; Os-NEXT: [[TMP30:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40 ; Os-NEXT: [[WIDE_LOAD1_5:%.*]] = load <4 x i32>, ptr [[TMP29]], align 4
; Os-NEXT: store <4 x i32> [[TMP29]], ptr [[TMP30]], align 4 ; Os-NEXT: [[TMP30:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[TMP31:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44 ; Os-NEXT: [[TMP31:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_5]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[WIDE_LOAD_11:%.*]] = load <4 x i32>, ptr [[TMP31]], align 4 ; Os-NEXT: [[TMP32:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40
; Os-NEXT: [[TMP32:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_11]], [[BROADCAST_SPLAT]] ; Os-NEXT: store <4 x i32> [[TMP30]], ptr [[TMP32]], align 4
; Os-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44 ; Os-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44
; Os-NEXT: store <4 x i32> [[TMP32]], ptr [[TMP33]], align 4 ; Os-NEXT: store <4 x i32> [[TMP31]], ptr [[TMP33]], align 4
; Os-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48 ; Os-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48
; Os-NEXT: [[WIDE_LOAD_12:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4 ; Os-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4
; Os-NEXT: [[TMP35:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_12]], [[BROADCAST_SPLAT]] ; Os-NEXT: [[TMP35:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52
; Os-NEXT: [[TMP36:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48 ; Os-NEXT: [[WIDE_LOAD1_6:%.*]] = load <4 x i32>, ptr [[TMP35]], align 4
; Os-NEXT: store <4 x i32> [[TMP35]], ptr [[TMP36]], align 4 ; Os-NEXT: [[TMP36:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[TMP37:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52 ; Os-NEXT: [[TMP37:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_6]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[WIDE_LOAD_13:%.*]] = load <4 x i32>, ptr [[TMP37]], align 4 ; Os-NEXT: [[TMP38:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48
; Os-NEXT: [[TMP38:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_13]], [[BROADCAST_SPLAT]] ; Os-NEXT: store <4 x i32> [[TMP36]], ptr [[TMP38]], align 4
; Os-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52 ; Os-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52
; Os-NEXT: store <4 x i32> [[TMP38]], ptr [[TMP39]], align 4 ; Os-NEXT: store <4 x i32> [[TMP37]], ptr [[TMP39]], align 4
; Os-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56 ; Os-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56
; Os-NEXT: [[WIDE_LOAD_14:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4 ; Os-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4
; Os-NEXT: [[TMP41:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_14]], [[BROADCAST_SPLAT]] ; Os-NEXT: [[TMP41:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60
; Os-NEXT: [[TMP42:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56 ; Os-NEXT: [[WIDE_LOAD1_7:%.*]] = load <4 x i32>, ptr [[TMP41]], align 4
; Os-NEXT: store <4 x i32> [[TMP41]], ptr [[TMP42]], align 4 ; Os-NEXT: [[TMP42:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[TMP43:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60 ; Os-NEXT: [[TMP43:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_7]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[WIDE_LOAD_15:%.*]] = load <4 x i32>, ptr [[TMP43]], align 4 ; Os-NEXT: [[TMP44:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56
; Os-NEXT: [[TMP44:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_15]], [[BROADCAST_SPLAT]] ; Os-NEXT: store <4 x i32> [[TMP42]], ptr [[TMP44]], align 4
; Os-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60 ; Os-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60
; Os-NEXT: store <4 x i32> [[TMP44]], ptr [[TMP45]], align 4 ; Os-NEXT: store <4 x i32> [[TMP43]], ptr [[TMP45]], align 4
; Os-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4 ; Os-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4
; Os-NEXT: ret i32 [[TMP46]] ; Os-NEXT: ret i32 [[TMP46]]
; ;
; Oz-LABEL: @enabled( ; Oz-LABEL: @enabled(
; Oz-NEXT: entry: ; Oz-NEXT: entry:
; Oz-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0 ; Oz-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0
; Oz-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer ; Oz-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer
; Oz-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4 ; Oz-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4
; Oz-NEXT: [[TMP0:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]] ; Oz-NEXT: [[TMP0:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4
; Oz-NEXT: store <4 x i32> [[TMP0]], ptr [[A:%.*]], align 4 ; Oz-NEXT: [[WIDE_LOAD1:%.*]] = load <4 x i32>, ptr [[TMP0]], align 4
; Oz-NEXT: [[TMP1:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4 ; Oz-NEXT: [[TMP1:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]]
; Oz-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP1]], align 4 ; Oz-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1]], [[BROADCAST_SPLAT]]
; Oz-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]] ; Oz-NEXT: store <4 x i32> [[TMP1]], ptr [[A:%.*]], align 4
; Oz-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4 ; Oz-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4
; Oz-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4 ; Oz-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4
; Oz-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8 ; Oz-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8
; Oz-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4 ; Oz-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4
; Oz-NEXT: [[TMP5:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]] ; Oz-NEXT: [[TMP5:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12
; Oz-NEXT: [[TMP6:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8 ; Oz-NEXT: [[WIDE_LOAD1_1:%.*]] = load <4 x i32>, ptr [[TMP5]], align 4
; Oz-NEXT: store <4 x i32> [[TMP5]], ptr [[TMP6]], align 4 ; Oz-NEXT: [[TMP6:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]]
; Oz-NEXT: [[TMP7:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12 ; Oz-NEXT: [[TMP7:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_1]], [[BROADCAST_SPLAT]]
; Oz-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP7]], align 4 ; Oz-NEXT: [[TMP8:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8
; Oz-NEXT: [[TMP8:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]] ; Oz-NEXT: store <4 x i32> [[TMP6]], ptr [[TMP8]], align 4
; Oz-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12 ; Oz-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12
; Oz-NEXT: store <4 x i32> [[TMP8]], ptr [[TMP9]], align 4 ; Oz-NEXT: store <4 x i32> [[TMP7]], ptr [[TMP9]], align 4
; Oz-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16 ; Oz-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16
; Oz-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4 ; Oz-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4
; Oz-NEXT: [[TMP11:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]] ; Oz-NEXT: [[TMP11:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20
; Oz-NEXT: [[TMP12:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16 ; Oz-NEXT: [[WIDE_LOAD1_2:%.*]] = load <4 x i32>, ptr [[TMP11]], align 4
; Oz-NEXT: store <4 x i32> [[TMP11]], ptr [[TMP12]], align 4 ; Oz-NEXT: [[TMP12:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]]
; Oz-NEXT: [[TMP13:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20 ; Oz-NEXT: [[TMP13:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_2]], [[BROADCAST_SPLAT]]
; Oz-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP13]], align 4 ; Oz-NEXT: [[TMP14:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16
; Oz-NEXT: [[TMP14:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]] ; Oz-NEXT: store <4 x i32> [[TMP12]], ptr [[TMP14]], align 4
; Oz-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20 ; Oz-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20
; Oz-NEXT: store <4 x i32> [[TMP14]], ptr [[TMP15]], align 4 ; Oz-NEXT: store <4 x i32> [[TMP13]], ptr [[TMP15]], align 4
; Oz-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24 ; Oz-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24
; Oz-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4 ; Oz-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4
; Oz-NEXT: [[TMP17:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]] ; Oz-NEXT: [[TMP17:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28
; Oz-NEXT: [[TMP18:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24 ; Oz-NEXT: [[WIDE_LOAD1_3:%.*]] = load <4 x i32>, ptr [[TMP17]], align 4
; Oz-NEXT: store <4 x i32> [[TMP17]], ptr [[TMP18]], align 4 ; Oz-NEXT: [[TMP18:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]]
; Oz-NEXT: [[TMP19:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28 ; Oz-NEXT: [[TMP19:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_3]], [[BROADCAST_SPLAT]]
; Oz-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP19]], align 4 ; Oz-NEXT: [[TMP20:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24
; Oz-NEXT: [[TMP20:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]] ; Oz-NEXT: store <4 x i32> [[TMP18]], ptr [[TMP20]], align 4
; Oz-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28 ; Oz-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28
; Oz-NEXT: store <4 x i32> [[TMP20]], ptr [[TMP21]], align 4 ; Oz-NEXT: store <4 x i32> [[TMP19]], ptr [[TMP21]], align 4
; Oz-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32 ; Oz-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32
; Oz-NEXT: [[WIDE_LOAD_8:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4 ; Oz-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4
; Oz-NEXT: [[TMP23:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_8]], [[BROADCAST_SPLAT]] ; Oz-NEXT: [[TMP23:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36
; Oz-NEXT: [[TMP24:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32 ; Oz-NEXT: [[WIDE_LOAD1_4:%.*]] = load <4 x i32>, ptr [[TMP23]], align 4
; Oz-NEXT: store <4 x i32> [[TMP23]], ptr [[TMP24]], align 4 ; Oz-NEXT: [[TMP24:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]]
; Oz-NEXT: [[TMP25:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36 ; Oz-NEXT: [[TMP25:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_4]], [[BROADCAST_SPLAT]]
; Oz-NEXT: [[WIDE_LOAD_9:%.*]] = load <4 x i32>, ptr [[TMP25]], align 4 ; Oz-NEXT: [[TMP26:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32
; Oz-NEXT: [[TMP26:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_9]], [[BROADCAST_SPLAT]] ; Oz-NEXT: store <4 x i32> [[TMP24]], ptr [[TMP26]], align 4
; Oz-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36 ; Oz-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36
; Oz-NEXT: store <4 x i32> [[TMP26]], ptr [[TMP27]], align 4 ; Oz-NEXT: store <4 x i32> [[TMP25]], ptr [[TMP27]], align 4
; Oz-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40 ; Oz-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40
; Oz-NEXT: [[WIDE_LOAD_10:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4 ; Oz-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4
; Oz-NEXT: [[TMP29:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_10]], [[BROADCAST_SPLAT]] ; Oz-NEXT: [[TMP29:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44
; Oz-NEXT: [[TMP30:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40 ; Oz-NEXT: [[WIDE_LOAD1_5:%.*]] = load <4 x i32>, ptr [[TMP29]], align 4
; Oz-NEXT: store <4 x i32> [[TMP29]], ptr [[TMP30]], align 4 ; Oz-NEXT: [[TMP30:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]]
; Oz-NEXT: [[TMP31:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44 ; Oz-NEXT: [[TMP31:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_5]], [[BROADCAST_SPLAT]]
; Oz-NEXT: [[WIDE_LOAD_11:%.*]] = load <4 x i32>, ptr [[TMP31]], align 4 ; Oz-NEXT: [[TMP32:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40
; Oz-NEXT: [[TMP32:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_11]], [[BROADCAST_SPLAT]] ; Oz-NEXT: store <4 x i32> [[TMP30]], ptr [[TMP32]], align 4
; Oz-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44 ; Oz-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44
; Oz-NEXT: store <4 x i32> [[TMP32]], ptr [[TMP33]], align 4 ; Oz-NEXT: store <4 x i32> [[TMP31]], ptr [[TMP33]], align 4
; Oz-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48 ; Oz-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48
; Oz-NEXT: [[WIDE_LOAD_12:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4 ; Oz-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4
; Oz-NEXT: [[TMP35:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_12]], [[BROADCAST_SPLAT]] ; Oz-NEXT: [[TMP35:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52
; Oz-NEXT: [[TMP36:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48 ; Oz-NEXT: [[WIDE_LOAD1_6:%.*]] = load <4 x i32>, ptr [[TMP35]], align 4
; Oz-NEXT: store <4 x i32> [[TMP35]], ptr [[TMP36]], align 4 ; Oz-NEXT: [[TMP36:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]]
; Oz-NEXT: [[TMP37:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52 ; Oz-NEXT: [[TMP37:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_6]], [[BROADCAST_SPLAT]]
; Oz-NEXT: [[WIDE_LOAD_13:%.*]] = load <4 x i32>, ptr [[TMP37]], align 4 ; Oz-NEXT: [[TMP38:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48
; Oz-NEXT: [[TMP38:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_13]], [[BROADCAST_SPLAT]] ; Oz-NEXT: store <4 x i32> [[TMP36]], ptr [[TMP38]], align 4
; Oz-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52 ; Oz-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52
; Oz-NEXT: store <4 x i32> [[TMP38]], ptr [[TMP39]], align 4 ; Oz-NEXT: store <4 x i32> [[TMP37]], ptr [[TMP39]], align 4
; Oz-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56 ; Oz-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56
; Oz-NEXT: [[WIDE_LOAD_14:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4 ; Oz-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4
; Oz-NEXT: [[TMP41:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_14]], [[BROADCAST_SPLAT]] ; Oz-NEXT: [[TMP41:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60
; Oz-NEXT: [[TMP42:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56 ; Oz-NEXT: [[WIDE_LOAD1_7:%.*]] = load <4 x i32>, ptr [[TMP41]], align 4
; Oz-NEXT: store <4 x i32> [[TMP41]], ptr [[TMP42]], align 4 ; Oz-NEXT: [[TMP42:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]]
; Oz-NEXT: [[TMP43:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60 ; Oz-NEXT: [[TMP43:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_7]], [[BROADCAST_SPLAT]]
; Oz-NEXT: [[WIDE_LOAD_15:%.*]] = load <4 x i32>, ptr [[TMP43]], align 4 ; Oz-NEXT: [[TMP44:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56
; Oz-NEXT: [[TMP44:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_15]], [[BROADCAST_SPLAT]] ; Oz-NEXT: store <4 x i32> [[TMP42]], ptr [[TMP44]], align 4
; Oz-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60 ; Oz-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60
; Oz-NEXT: store <4 x i32> [[TMP44]], ptr [[TMP45]], align 4 ; Oz-NEXT: store <4 x i32> [[TMP43]], ptr [[TMP45]], align 4
; Oz-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4 ; Oz-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4
; Oz-NEXT: ret i32 [[TMP46]] ; Oz-NEXT: ret i32 [[TMP46]]
; ;
; O1VEC2-LABEL: @enabled( ; O1VEC2-LABEL: @enabled(
; O1VEC2-NEXT: entry: ; O1VEC2-NEXT: entry:
; O1VEC2-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0 ; O1VEC2-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0
; O1VEC2-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer ; O1VEC2-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer
; O1VEC2-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4 ; O1VEC2-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4
; O1VEC2-NEXT: [[TMP0:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]] ; O1VEC2-NEXT: [[TMP0:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4
; O1VEC2-NEXT: store <4 x i32> [[TMP0]], ptr [[A:%.*]], align 4 ; O1VEC2-NEXT: [[WIDE_LOAD1:%.*]] = load <4 x i32>, ptr [[TMP0]], align 4
; O1VEC2-NEXT: [[TMP1:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4 ; O1VEC2-NEXT: [[TMP1:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]]
; O1VEC2-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP1]], align 4 ; O1VEC2-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1]], [[BROADCAST_SPLAT]]
; O1VEC2-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]] ; O1VEC2-NEXT: store <4 x i32> [[TMP1]], ptr [[A:%.*]], align 4
; O1VEC2-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4 ; O1VEC2-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4
; O1VEC2-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4 ; O1VEC2-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4
; O1VEC2-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8 ; O1VEC2-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8
; O1VEC2-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4 ; O1VEC2-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4
; O1VEC2-NEXT: [[TMP5:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]] ; O1VEC2-NEXT: [[TMP5:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12
; O1VEC2-NEXT: [[TMP6:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8 ; O1VEC2-NEXT: [[WIDE_LOAD1_1:%.*]] = load <4 x i32>, ptr [[TMP5]], align 4
; O1VEC2-NEXT: store <4 x i32> [[TMP5]], ptr [[TMP6]], align 4 ; O1VEC2-NEXT: [[TMP6:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]]
; O1VEC2-NEXT: [[TMP7:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12 ; O1VEC2-NEXT: [[TMP7:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_1]], [[BROADCAST_SPLAT]]
; O1VEC2-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP7]], align 4 ; O1VEC2-NEXT: [[TMP8:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8
; O1VEC2-NEXT: [[TMP8:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]] ; O1VEC2-NEXT: store <4 x i32> [[TMP6]], ptr [[TMP8]], align 4
; O1VEC2-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12 ; O1VEC2-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12
; O1VEC2-NEXT: store <4 x i32> [[TMP8]], ptr [[TMP9]], align 4 ; O1VEC2-NEXT: store <4 x i32> [[TMP7]], ptr [[TMP9]], align 4
; O1VEC2-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16 ; O1VEC2-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16
; O1VEC2-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4 ; O1VEC2-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4
; O1VEC2-NEXT: [[TMP11:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]] ; O1VEC2-NEXT: [[TMP11:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20
; O1VEC2-NEXT: [[TMP12:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16 ; O1VEC2-NEXT: [[WIDE_LOAD1_2:%.*]] = load <4 x i32>, ptr [[TMP11]], align 4
; O1VEC2-NEXT: store <4 x i32> [[TMP11]], ptr [[TMP12]], align 4 ; O1VEC2-NEXT: [[TMP12:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]]
; O1VEC2-NEXT: [[TMP13:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20 ; O1VEC2-NEXT: [[TMP13:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_2]], [[BROADCAST_SPLAT]]
; O1VEC2-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP13]], align 4 ; O1VEC2-NEXT: [[TMP14:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16
; O1VEC2-NEXT: [[TMP14:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]] ; O1VEC2-NEXT: store <4 x i32> [[TMP12]], ptr [[TMP14]], align 4
; O1VEC2-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20 ; O1VEC2-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20
; O1VEC2-NEXT: store <4 x i32> [[TMP14]], ptr [[TMP15]], align 4 ; O1VEC2-NEXT: store <4 x i32> [[TMP13]], ptr [[TMP15]], align 4
; O1VEC2-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24 ; O1VEC2-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24
; O1VEC2-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4 ; O1VEC2-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4
; O1VEC2-NEXT: [[TMP17:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]] ; O1VEC2-NEXT: [[TMP17:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28
; O1VEC2-NEXT: [[TMP18:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24 ; O1VEC2-NEXT: [[WIDE_LOAD1_3:%.*]] = load <4 x i32>, ptr [[TMP17]], align 4
; O1VEC2-NEXT: store <4 x i32> [[TMP17]], ptr [[TMP18]], align 4 ; O1VEC2-NEXT: [[TMP18:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]]
; O1VEC2-NEXT: [[TMP19:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28 ; O1VEC2-NEXT: [[TMP19:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_3]], [[BROADCAST_SPLAT]]
; O1VEC2-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP19]], align 4 ; O1VEC2-NEXT: [[TMP20:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24
; O1VEC2-NEXT: [[TMP20:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]] ; O1VEC2-NEXT: store <4 x i32> [[TMP18]], ptr [[TMP20]], align 4
; O1VEC2-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28 ; O1VEC2-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28
; O1VEC2-NEXT: store <4 x i32> [[TMP20]], ptr [[TMP21]], align 4 ; O1VEC2-NEXT: store <4 x i32> [[TMP19]], ptr [[TMP21]], align 4
; O1VEC2-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32 ; O1VEC2-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32
; O1VEC2-NEXT: [[WIDE_LOAD_8:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4 ; O1VEC2-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4
; O1VEC2-NEXT: [[TMP23:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_8]], [[BROADCAST_SPLAT]] ; O1VEC2-NEXT: [[TMP23:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36
; O1VEC2-NEXT: [[TMP24:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32 ; O1VEC2-NEXT: [[WIDE_LOAD1_4:%.*]] = load <4 x i32>, ptr [[TMP23]], align 4
; O1VEC2-NEXT: store <4 x i32> [[TMP23]], ptr [[TMP24]], align 4 ; O1VEC2-NEXT: [[TMP24:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]]
; O1VEC2-NEXT: [[TMP25:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36 ; O1VEC2-NEXT: [[TMP25:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_4]], [[BROADCAST_SPLAT]]
; O1VEC2-NEXT: [[WIDE_LOAD_9:%.*]] = load <4 x i32>, ptr [[TMP25]], align 4 ; O1VEC2-NEXT: [[TMP26:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32
; O1VEC2-NEXT: [[TMP26:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_9]], [[BROADCAST_SPLAT]] ; O1VEC2-NEXT: store <4 x i32> [[TMP24]], ptr [[TMP26]], align 4
; O1VEC2-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36 ; O1VEC2-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36
; O1VEC2-NEXT: store <4 x i32> [[TMP26]], ptr [[TMP27]], align 4 ; O1VEC2-NEXT: store <4 x i32> [[TMP25]], ptr [[TMP27]], align 4
; O1VEC2-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40 ; O1VEC2-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40
; O1VEC2-NEXT: [[WIDE_LOAD_10:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4 ; O1VEC2-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4
; O1VEC2-NEXT: [[TMP29:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_10]], [[BROADCAST_SPLAT]] ; O1VEC2-NEXT: [[TMP29:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44
; O1VEC2-NEXT: [[TMP30:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40 ; O1VEC2-NEXT: [[WIDE_LOAD1_5:%.*]] = load <4 x i32>, ptr [[TMP29]], align 4
; O1VEC2-NEXT: store <4 x i32> [[TMP29]], ptr [[TMP30]], align 4 ; O1VEC2-NEXT: [[TMP30:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]]
; O1VEC2-NEXT: [[TMP31:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44 ; O1VEC2-NEXT: [[TMP31:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_5]], [[BROADCAST_SPLAT]]
; O1VEC2-NEXT: [[WIDE_LOAD_11:%.*]] = load <4 x i32>, ptr [[TMP31]], align 4 ; O1VEC2-NEXT: [[TMP32:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40
; O1VEC2-NEXT: [[TMP32:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_11]], [[BROADCAST_SPLAT]] ; O1VEC2-NEXT: store <4 x i32> [[TMP30]], ptr [[TMP32]], align 4
; O1VEC2-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44 ; O1VEC2-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44
; O1VEC2-NEXT: store <4 x i32> [[TMP32]], ptr [[TMP33]], align 4 ; O1VEC2-NEXT: store <4 x i32> [[TMP31]], ptr [[TMP33]], align 4
; O1VEC2-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48 ; O1VEC2-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48
; O1VEC2-NEXT: [[WIDE_LOAD_12:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4 ; O1VEC2-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4
; O1VEC2-NEXT: [[TMP35:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_12]], [[BROADCAST_SPLAT]] ; O1VEC2-NEXT: [[TMP35:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52
; O1VEC2-NEXT: [[TMP36:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48 ; O1VEC2-NEXT: [[WIDE_LOAD1_6:%.*]] = load <4 x i32>, ptr [[TMP35]], align 4
; O1VEC2-NEXT: store <4 x i32> [[TMP35]], ptr [[TMP36]], align 4 ; O1VEC2-NEXT: [[TMP36:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]]
; O1VEC2-NEXT: [[TMP37:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52 ; O1VEC2-NEXT: [[TMP37:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_6]], [[BROADCAST_SPLAT]]
; O1VEC2-NEXT: [[WIDE_LOAD_13:%.*]] = load <4 x i32>, ptr [[TMP37]], align 4 ; O1VEC2-NEXT: [[TMP38:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48
; O1VEC2-NEXT: [[TMP38:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_13]], [[BROADCAST_SPLAT]] ; O1VEC2-NEXT: store <4 x i32> [[TMP36]], ptr [[TMP38]], align 4
; O1VEC2-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52 ; O1VEC2-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52
; O1VEC2-NEXT: store <4 x i32> [[TMP38]], ptr [[TMP39]], align 4 ; O1VEC2-NEXT: store <4 x i32> [[TMP37]], ptr [[TMP39]], align 4
; O1VEC2-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56 ; O1VEC2-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56
; O1VEC2-NEXT: [[WIDE_LOAD_14:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4 ; O1VEC2-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4
; O1VEC2-NEXT: [[TMP41:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_14]], [[BROADCAST_SPLAT]] ; O1VEC2-NEXT: [[TMP41:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60
; O1VEC2-NEXT: [[TMP42:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56 ; O1VEC2-NEXT: [[WIDE_LOAD1_7:%.*]] = load <4 x i32>, ptr [[TMP41]], align 4
; O1VEC2-NEXT: store <4 x i32> [[TMP41]], ptr [[TMP42]], align 4 ; O1VEC2-NEXT: [[TMP42:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]]
; O1VEC2-NEXT: [[TMP43:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60 ; O1VEC2-NEXT: [[TMP43:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_7]], [[BROADCAST_SPLAT]]
; O1VEC2-NEXT: [[WIDE_LOAD_15:%.*]] = load <4 x i32>, ptr [[TMP43]], align 4 ; O1VEC2-NEXT: [[TMP44:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56
; O1VEC2-NEXT: [[TMP44:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_15]], [[BROADCAST_SPLAT]] ; O1VEC2-NEXT: store <4 x i32> [[TMP42]], ptr [[TMP44]], align 4
; O1VEC2-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60 ; O1VEC2-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60
; O1VEC2-NEXT: store <4 x i32> [[TMP44]], ptr [[TMP45]], align 4 ; O1VEC2-NEXT: store <4 x i32> [[TMP43]], ptr [[TMP45]], align 4
; O1VEC2-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4 ; O1VEC2-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4
; O1VEC2-NEXT: ret i32 [[TMP46]] ; O1VEC2-NEXT: ret i32 [[TMP46]]
; ;
; OzVEC2-LABEL: @enabled( ; OzVEC2-LABEL: @enabled(
; OzVEC2-NEXT: entry: ; OzVEC2-NEXT: entry:
; OzVEC2-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0 ; OzVEC2-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0
; OzVEC2-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer ; OzVEC2-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer
; OzVEC2-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4 ; OzVEC2-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4
; OzVEC2-NEXT: [[TMP0:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]] ; OzVEC2-NEXT: [[TMP0:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4
; OzVEC2-NEXT: store <4 x i32> [[TMP0]], ptr [[A:%.*]], align 4 ; OzVEC2-NEXT: [[WIDE_LOAD1:%.*]] = load <4 x i32>, ptr [[TMP0]], align 4
; OzVEC2-NEXT: [[TMP1:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4 ; OzVEC2-NEXT: [[TMP1:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]]
; OzVEC2-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP1]], align 4 ; OzVEC2-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1]], [[BROADCAST_SPLAT]]
; OzVEC2-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]] ; OzVEC2-NEXT: store <4 x i32> [[TMP1]], ptr [[A:%.*]], align 4
; OzVEC2-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4 ; OzVEC2-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4
; OzVEC2-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4 ; OzVEC2-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4
; OzVEC2-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8 ; OzVEC2-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8
; OzVEC2-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4 ; OzVEC2-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4
; OzVEC2-NEXT: [[TMP5:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]] ; OzVEC2-NEXT: [[TMP5:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12
; OzVEC2-NEXT: [[TMP6:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8 ; OzVEC2-NEXT: [[WIDE_LOAD1_1:%.*]] = load <4 x i32>, ptr [[TMP5]], align 4
; OzVEC2-NEXT: store <4 x i32> [[TMP5]], ptr [[TMP6]], align 4 ; OzVEC2-NEXT: [[TMP6:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]]
; OzVEC2-NEXT: [[TMP7:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12 ; OzVEC2-NEXT: [[TMP7:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_1]], [[BROADCAST_SPLAT]]
; OzVEC2-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP7]], align 4 ; OzVEC2-NEXT: [[TMP8:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8
; OzVEC2-NEXT: [[TMP8:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]] ; OzVEC2-NEXT: store <4 x i32> [[TMP6]], ptr [[TMP8]], align 4
; OzVEC2-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12 ; OzVEC2-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12
; OzVEC2-NEXT: store <4 x i32> [[TMP8]], ptr [[TMP9]], align 4 ; OzVEC2-NEXT: store <4 x i32> [[TMP7]], ptr [[TMP9]], align 4
; OzVEC2-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16 ; OzVEC2-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16
; OzVEC2-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4 ; OzVEC2-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4
; OzVEC2-NEXT: [[TMP11:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]] ; OzVEC2-NEXT: [[TMP11:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20
; OzVEC2-NEXT: [[TMP12:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16 ; OzVEC2-NEXT: [[WIDE_LOAD1_2:%.*]] = load <4 x i32>, ptr [[TMP11]], align 4
; OzVEC2-NEXT: store <4 x i32> [[TMP11]], ptr [[TMP12]], align 4 ; OzVEC2-NEXT: [[TMP12:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]]
; OzVEC2-NEXT: [[TMP13:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20 ; OzVEC2-NEXT: [[TMP13:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_2]], [[BROADCAST_SPLAT]]
; OzVEC2-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP13]], align 4 ; OzVEC2-NEXT: [[TMP14:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16
; OzVEC2-NEXT: [[TMP14:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]] ; OzVEC2-NEXT: store <4 x i32> [[TMP12]], ptr [[TMP14]], align 4
; OzVEC2-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20 ; OzVEC2-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20
; OzVEC2-NEXT: store <4 x i32> [[TMP14]], ptr [[TMP15]], align 4 ; OzVEC2-NEXT: store <4 x i32> [[TMP13]], ptr [[TMP15]], align 4
; OzVEC2-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24 ; OzVEC2-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24
; OzVEC2-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4 ; OzVEC2-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4
; OzVEC2-NEXT: [[TMP17:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]] ; OzVEC2-NEXT: [[TMP17:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28
; OzVEC2-NEXT: [[TMP18:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24 ; OzVEC2-NEXT: [[WIDE_LOAD1_3:%.*]] = load <4 x i32>, ptr [[TMP17]], align 4
; OzVEC2-NEXT: store <4 x i32> [[TMP17]], ptr [[TMP18]], align 4 ; OzVEC2-NEXT: [[TMP18:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]]
; OzVEC2-NEXT: [[TMP19:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28 ; OzVEC2-NEXT: [[TMP19:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_3]], [[BROADCAST_SPLAT]]
; OzVEC2-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP19]], align 4 ; OzVEC2-NEXT: [[TMP20:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24
; OzVEC2-NEXT: [[TMP20:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]] ; OzVEC2-NEXT: store <4 x i32> [[TMP18]], ptr [[TMP20]], align 4
; OzVEC2-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28 ; OzVEC2-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28
; OzVEC2-NEXT: store <4 x i32> [[TMP20]], ptr [[TMP21]], align 4 ; OzVEC2-NEXT: store <4 x i32> [[TMP19]], ptr [[TMP21]], align 4
; OzVEC2-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32 ; OzVEC2-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32
; OzVEC2-NEXT: [[WIDE_LOAD_8:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4 ; OzVEC2-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4
; OzVEC2-NEXT: [[TMP23:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_8]], [[BROADCAST_SPLAT]] ; OzVEC2-NEXT: [[TMP23:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36
; OzVEC2-NEXT: [[TMP24:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32 ; OzVEC2-NEXT: [[WIDE_LOAD1_4:%.*]] = load <4 x i32>, ptr [[TMP23]], align 4
; OzVEC2-NEXT: store <4 x i32> [[TMP23]], ptr [[TMP24]], align 4 ; OzVEC2-NEXT: [[TMP24:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]]
; OzVEC2-NEXT: [[TMP25:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36 ; OzVEC2-NEXT: [[TMP25:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_4]], [[BROADCAST_SPLAT]]
; OzVEC2-NEXT: [[WIDE_LOAD_9:%.*]] = load <4 x i32>, ptr [[TMP25]], align 4 ; OzVEC2-NEXT: [[TMP26:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32
; OzVEC2-NEXT: [[TMP26:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_9]], [[BROADCAST_SPLAT]] ; OzVEC2-NEXT: store <4 x i32> [[TMP24]], ptr [[TMP26]], align 4
; OzVEC2-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36 ; OzVEC2-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36
; OzVEC2-NEXT: store <4 x i32> [[TMP26]], ptr [[TMP27]], align 4 ; OzVEC2-NEXT: store <4 x i32> [[TMP25]], ptr [[TMP27]], align 4
; OzVEC2-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40 ; OzVEC2-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40
; OzVEC2-NEXT: [[WIDE_LOAD_10:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4 ; OzVEC2-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4
; OzVEC2-NEXT: [[TMP29:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_10]], [[BROADCAST_SPLAT]] ; OzVEC2-NEXT: [[TMP29:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44
; OzVEC2-NEXT: [[TMP30:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40 ; OzVEC2-NEXT: [[WIDE_LOAD1_5:%.*]] = load <4 x i32>, ptr [[TMP29]], align 4
; OzVEC2-NEXT: store <4 x i32> [[TMP29]], ptr [[TMP30]], align 4 ; OzVEC2-NEXT: [[TMP30:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]]
; OzVEC2-NEXT: [[TMP31:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44 ; OzVEC2-NEXT: [[TMP31:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_5]], [[BROADCAST_SPLAT]]
; OzVEC2-NEXT: [[WIDE_LOAD_11:%.*]] = load <4 x i32>, ptr [[TMP31]], align 4 ; OzVEC2-NEXT: [[TMP32:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40
; OzVEC2-NEXT: [[TMP32:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_11]], [[BROADCAST_SPLAT]] ; OzVEC2-NEXT: store <4 x i32> [[TMP30]], ptr [[TMP32]], align 4
; OzVEC2-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44 ; OzVEC2-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44
; OzVEC2-NEXT: store <4 x i32> [[TMP32]], ptr [[TMP33]], align 4 ; OzVEC2-NEXT: store <4 x i32> [[TMP31]], ptr [[TMP33]], align 4
; OzVEC2-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48 ; OzVEC2-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48
; OzVEC2-NEXT: [[WIDE_LOAD_12:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4 ; OzVEC2-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4
; OzVEC2-NEXT: [[TMP35:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_12]], [[BROADCAST_SPLAT]] ; OzVEC2-NEXT: [[TMP35:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52
; OzVEC2-NEXT: [[TMP36:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48 ; OzVEC2-NEXT: [[WIDE_LOAD1_6:%.*]] = load <4 x i32>, ptr [[TMP35]], align 4
; OzVEC2-NEXT: store <4 x i32> [[TMP35]], ptr [[TMP36]], align 4 ; OzVEC2-NEXT: [[TMP36:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]]
; OzVEC2-NEXT: [[TMP37:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52 ; OzVEC2-NEXT: [[TMP37:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_6]], [[BROADCAST_SPLAT]]
; OzVEC2-NEXT: [[WIDE_LOAD_13:%.*]] = load <4 x i32>, ptr [[TMP37]], align 4 ; OzVEC2-NEXT: [[TMP38:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48
; OzVEC2-NEXT: [[TMP38:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_13]], [[BROADCAST_SPLAT]] ; OzVEC2-NEXT: store <4 x i32> [[TMP36]], ptr [[TMP38]], align 4
; OzVEC2-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52 ; OzVEC2-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52
; OzVEC2-NEXT: store <4 x i32> [[TMP38]], ptr [[TMP39]], align 4 ; OzVEC2-NEXT: store <4 x i32> [[TMP37]], ptr [[TMP39]], align 4
; OzVEC2-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56 ; OzVEC2-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56
; OzVEC2-NEXT: [[WIDE_LOAD_14:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4 ; OzVEC2-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4
; OzVEC2-NEXT: [[TMP41:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_14]], [[BROADCAST_SPLAT]] ; OzVEC2-NEXT: [[TMP41:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60
; OzVEC2-NEXT: [[TMP42:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56 ; OzVEC2-NEXT: [[WIDE_LOAD1_7:%.*]] = load <4 x i32>, ptr [[TMP41]], align 4
; OzVEC2-NEXT: store <4 x i32> [[TMP41]], ptr [[TMP42]], align 4 ; OzVEC2-NEXT: [[TMP42:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]]
; OzVEC2-NEXT: [[TMP43:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60 ; OzVEC2-NEXT: [[TMP43:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_7]], [[BROADCAST_SPLAT]]
; OzVEC2-NEXT: [[WIDE_LOAD_15:%.*]] = load <4 x i32>, ptr [[TMP43]], align 4 ; OzVEC2-NEXT: [[TMP44:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56
; OzVEC2-NEXT: [[TMP44:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_15]], [[BROADCAST_SPLAT]] ; OzVEC2-NEXT: store <4 x i32> [[TMP42]], ptr [[TMP44]], align 4
; OzVEC2-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60 ; OzVEC2-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60
; OzVEC2-NEXT: store <4 x i32> [[TMP44]], ptr [[TMP45]], align 4 ; OzVEC2-NEXT: store <4 x i32> [[TMP43]], ptr [[TMP45]], align 4
; OzVEC2-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4 ; OzVEC2-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4
; OzVEC2-NEXT: ret i32 [[TMP46]] ; OzVEC2-NEXT: ret i32 [[TMP46]]
; ;
; O3DIS-LABEL: @enabled( ; O3DIS-LABEL: @enabled(
; O3DIS-NEXT: entry: ; O3DIS-NEXT: entry:
; O3DIS-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0 ; O3DIS-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0
; O3DIS-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer ; O3DIS-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer
; O3DIS-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4 ; O3DIS-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4
; O3DIS-NEXT: [[TMP0:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]] ; O3DIS-NEXT: [[TMP0:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4
; O3DIS-NEXT: store <4 x i32> [[TMP0]], ptr [[A:%.*]], align 4 ; O3DIS-NEXT: [[WIDE_LOAD1:%.*]] = load <4 x i32>, ptr [[TMP0]], align 4
; O3DIS-NEXT: [[TMP1:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4 ; O3DIS-NEXT: [[TMP1:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]]
; O3DIS-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP1]], align 4 ; O3DIS-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1]], [[BROADCAST_SPLAT]]
; O3DIS-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]] ; O3DIS-NEXT: store <4 x i32> [[TMP1]], ptr [[A:%.*]], align 4
; O3DIS-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4 ; O3DIS-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4
; O3DIS-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4 ; O3DIS-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4
; O3DIS-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8 ; O3DIS-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8
; O3DIS-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4 ; O3DIS-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4
; O3DIS-NEXT: [[TMP5:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]] ; O3DIS-NEXT: [[TMP5:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12
; O3DIS-NEXT: [[TMP6:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8 ; O3DIS-NEXT: [[WIDE_LOAD1_1:%.*]] = load <4 x i32>, ptr [[TMP5]], align 4
; O3DIS-NEXT: store <4 x i32> [[TMP5]], ptr [[TMP6]], align 4 ; O3DIS-NEXT: [[TMP6:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]]
; O3DIS-NEXT: [[TMP7:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12 ; O3DIS-NEXT: [[TMP7:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_1]], [[BROADCAST_SPLAT]]
; O3DIS-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP7]], align 4 ; O3DIS-NEXT: [[TMP8:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8
; O3DIS-NEXT: [[TMP8:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]] ; O3DIS-NEXT: store <4 x i32> [[TMP6]], ptr [[TMP8]], align 4
; O3DIS-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12 ; O3DIS-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12
; O3DIS-NEXT: store <4 x i32> [[TMP8]], ptr [[TMP9]], align 4 ; O3DIS-NEXT: store <4 x i32> [[TMP7]], ptr [[TMP9]], align 4
; O3DIS-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16 ; O3DIS-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16
; O3DIS-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4 ; O3DIS-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4
; O3DIS-NEXT: [[TMP11:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]] ; O3DIS-NEXT: [[TMP11:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20
; O3DIS-NEXT: [[TMP12:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16 ; O3DIS-NEXT: [[WIDE_LOAD1_2:%.*]] = load <4 x i32>, ptr [[TMP11]], align 4
; O3DIS-NEXT: store <4 x i32> [[TMP11]], ptr [[TMP12]], align 4 ; O3DIS-NEXT: [[TMP12:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]]
; O3DIS-NEXT: [[TMP13:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20 ; O3DIS-NEXT: [[TMP13:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_2]], [[BROADCAST_SPLAT]]
; O3DIS-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP13]], align 4 ; O3DIS-NEXT: [[TMP14:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16
; O3DIS-NEXT: [[TMP14:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]] ; O3DIS-NEXT: store <4 x i32> [[TMP12]], ptr [[TMP14]], align 4
; O3DIS-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20 ; O3DIS-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20
; O3DIS-NEXT: store <4 x i32> [[TMP14]], ptr [[TMP15]], align 4 ; O3DIS-NEXT: store <4 x i32> [[TMP13]], ptr [[TMP15]], align 4
; O3DIS-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24 ; O3DIS-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24
; O3DIS-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4 ; O3DIS-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4
; O3DIS-NEXT: [[TMP17:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]] ; O3DIS-NEXT: [[TMP17:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28
; O3DIS-NEXT: [[TMP18:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24 ; O3DIS-NEXT: [[WIDE_LOAD1_3:%.*]] = load <4 x i32>, ptr [[TMP17]], align 4
; O3DIS-NEXT: store <4 x i32> [[TMP17]], ptr [[TMP18]], align 4 ; O3DIS-NEXT: [[TMP18:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]]
; O3DIS-NEXT: [[TMP19:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28 ; O3DIS-NEXT: [[TMP19:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_3]], [[BROADCAST_SPLAT]]
; O3DIS-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP19]], align 4 ; O3DIS-NEXT: [[TMP20:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24
; O3DIS-NEXT: [[TMP20:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]] ; O3DIS-NEXT: store <4 x i32> [[TMP18]], ptr [[TMP20]], align 4
; O3DIS-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28 ; O3DIS-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28
; O3DIS-NEXT: store <4 x i32> [[TMP20]], ptr [[TMP21]], align 4 ; O3DIS-NEXT: store <4 x i32> [[TMP19]], ptr [[TMP21]], align 4
; O3DIS-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32 ; O3DIS-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32
; O3DIS-NEXT: [[WIDE_LOAD_8:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4 ; O3DIS-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4
; O3DIS-NEXT: [[TMP23:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_8]], [[BROADCAST_SPLAT]] ; O3DIS-NEXT: [[TMP23:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36
; O3DIS-NEXT: [[TMP24:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32 ; O3DIS-NEXT: [[WIDE_LOAD1_4:%.*]] = load <4 x i32>, ptr [[TMP23]], align 4
; O3DIS-NEXT: store <4 x i32> [[TMP23]], ptr [[TMP24]], align 4 ; O3DIS-NEXT: [[TMP24:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]]
; O3DIS-NEXT: [[TMP25:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36 ; O3DIS-NEXT: [[TMP25:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_4]], [[BROADCAST_SPLAT]]
; O3DIS-NEXT: [[WIDE_LOAD_9:%.*]] = load <4 x i32>, ptr [[TMP25]], align 4 ; O3DIS-NEXT: [[TMP26:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32
; O3DIS-NEXT: [[TMP26:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_9]], [[BROADCAST_SPLAT]] ; O3DIS-NEXT: store <4 x i32> [[TMP24]], ptr [[TMP26]], align 4
; O3DIS-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36 ; O3DIS-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36
; O3DIS-NEXT: store <4 x i32> [[TMP26]], ptr [[TMP27]], align 4 ; O3DIS-NEXT: store <4 x i32> [[TMP25]], ptr [[TMP27]], align 4
; O3DIS-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40 ; O3DIS-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40
; O3DIS-NEXT: [[WIDE_LOAD_10:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4 ; O3DIS-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4
; O3DIS-NEXT: [[TMP29:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_10]], [[BROADCAST_SPLAT]] ; O3DIS-NEXT: [[TMP29:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44
; O3DIS-NEXT: [[TMP30:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40 ; O3DIS-NEXT: [[WIDE_LOAD1_5:%.*]] = load <4 x i32>, ptr [[TMP29]], align 4
; O3DIS-NEXT: store <4 x i32> [[TMP29]], ptr [[TMP30]], align 4 ; O3DIS-NEXT: [[TMP30:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]]
; O3DIS-NEXT: [[TMP31:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44 ; O3DIS-NEXT: [[TMP31:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_5]], [[BROADCAST_SPLAT]]
; O3DIS-NEXT: [[WIDE_LOAD_11:%.*]] = load <4 x i32>, ptr [[TMP31]], align 4 ; O3DIS-NEXT: [[TMP32:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40
; O3DIS-NEXT: [[TMP32:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_11]], [[BROADCAST_SPLAT]] ; O3DIS-NEXT: store <4 x i32> [[TMP30]], ptr [[TMP32]], align 4
; O3DIS-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44 ; O3DIS-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44
; O3DIS-NEXT: store <4 x i32> [[TMP32]], ptr [[TMP33]], align 4 ; O3DIS-NEXT: store <4 x i32> [[TMP31]], ptr [[TMP33]], align 4
; O3DIS-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48 ; O3DIS-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48
; O3DIS-NEXT: [[WIDE_LOAD_12:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4 ; O3DIS-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4
; O3DIS-NEXT: [[TMP35:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_12]], [[BROADCAST_SPLAT]] ; O3DIS-NEXT: [[TMP35:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52
; O3DIS-NEXT: [[TMP36:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48 ; O3DIS-NEXT: [[WIDE_LOAD1_6:%.*]] = load <4 x i32>, ptr [[TMP35]], align 4
; O3DIS-NEXT: store <4 x i32> [[TMP35]], ptr [[TMP36]], align 4 ; O3DIS-NEXT: [[TMP36:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]]
; O3DIS-NEXT: [[TMP37:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52 ; O3DIS-NEXT: [[TMP37:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_6]], [[BROADCAST_SPLAT]]
; O3DIS-NEXT: [[WIDE_LOAD_13:%.*]] = load <4 x i32>, ptr [[TMP37]], align 4 ; O3DIS-NEXT: [[TMP38:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48
; O3DIS-NEXT: [[TMP38:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_13]], [[BROADCAST_SPLAT]] ; O3DIS-NEXT: store <4 x i32> [[TMP36]], ptr [[TMP38]], align 4
; O3DIS-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52 ; O3DIS-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52
; O3DIS-NEXT: store <4 x i32> [[TMP38]], ptr [[TMP39]], align 4 ; O3DIS-NEXT: store <4 x i32> [[TMP37]], ptr [[TMP39]], align 4
; O3DIS-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56 ; O3DIS-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56
; O3DIS-NEXT: [[WIDE_LOAD_14:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4 ; O3DIS-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4
; O3DIS-NEXT: [[TMP41:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_14]], [[BROADCAST_SPLAT]] ; O3DIS-NEXT: [[TMP41:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60
; O3DIS-NEXT: [[TMP42:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56 ; O3DIS-NEXT: [[WIDE_LOAD1_7:%.*]] = load <4 x i32>, ptr [[TMP41]], align 4
; O3DIS-NEXT: store <4 x i32> [[TMP41]], ptr [[TMP42]], align 4 ; O3DIS-NEXT: [[TMP42:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]]
; O3DIS-NEXT: [[TMP43:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60 ; O3DIS-NEXT: [[TMP43:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_7]], [[BROADCAST_SPLAT]]
; O3DIS-NEXT: [[WIDE_LOAD_15:%.*]] = load <4 x i32>, ptr [[TMP43]], align 4 ; O3DIS-NEXT: [[TMP44:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56
; O3DIS-NEXT: [[TMP44:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_15]], [[BROADCAST_SPLAT]] ; O3DIS-NEXT: store <4 x i32> [[TMP42]], ptr [[TMP44]], align 4
; O3DIS-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60 ; O3DIS-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60
; O3DIS-NEXT: store <4 x i32> [[TMP44]], ptr [[TMP45]], align 4 ; O3DIS-NEXT: store <4 x i32> [[TMP43]], ptr [[TMP45]], align 4
; O3DIS-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4 ; O3DIS-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4
; O3DIS-NEXT: ret i32 [[TMP46]] ; O3DIS-NEXT: ret i32 [[TMP46]]
; ;
entry: entry:
br label %for.body br label %for.body
for.body: ; preds = %for.body, %entry for.body: ; preds = %for.body, %entry
%indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %for.body ] %indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %for.body ]
Show All 29 Lines
; O1-NEXT: [[TMP1:%.*]] = load i32, ptr [[A]], align 4 ; O1-NEXT: [[TMP1:%.*]] = load i32, ptr [[A]], align 4
; O1-NEXT: ret i32 [[TMP1]] ; O1-NEXT: ret i32 [[TMP1]]
; ;
; O2-LABEL: @nopragma( ; O2-LABEL: @nopragma(
; O2-NEXT: entry: ; O2-NEXT: entry:
; O2-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0 ; O2-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0
; O2-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer ; O2-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer
; O2-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4 ; O2-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4
; O2-NEXT: [[TMP0:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]] ; O2-NEXT: [[TMP0:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4
; O2-NEXT: store <4 x i32> [[TMP0]], ptr [[A:%.*]], align 4 ; O2-NEXT: [[WIDE_LOAD1:%.*]] = load <4 x i32>, ptr [[TMP0]], align 4
; O2-NEXT: [[TMP1:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4 ; O2-NEXT: [[TMP1:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP1]], align 4 ; O2-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]] ; O2-NEXT: store <4 x i32> [[TMP1]], ptr [[A:%.*]], align 4
; O2-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4 ; O2-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4
; O2-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4 ; O2-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4
; O2-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8 ; O2-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8
; O2-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4 ; O2-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4
; O2-NEXT: [[TMP5:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]] ; O2-NEXT: [[TMP5:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12
; O2-NEXT: [[TMP6:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8 ; O2-NEXT: [[WIDE_LOAD1_1:%.*]] = load <4 x i32>, ptr [[TMP5]], align 4
; O2-NEXT: store <4 x i32> [[TMP5]], ptr [[TMP6]], align 4 ; O2-NEXT: [[TMP6:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[TMP7:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12 ; O2-NEXT: [[TMP7:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_1]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP7]], align 4 ; O2-NEXT: [[TMP8:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8
; O2-NEXT: [[TMP8:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]] ; O2-NEXT: store <4 x i32> [[TMP6]], ptr [[TMP8]], align 4
; O2-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12 ; O2-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12
; O2-NEXT: store <4 x i32> [[TMP8]], ptr [[TMP9]], align 4 ; O2-NEXT: store <4 x i32> [[TMP7]], ptr [[TMP9]], align 4
; O2-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16 ; O2-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16
; O2-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4 ; O2-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4
; O2-NEXT: [[TMP11:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]] ; O2-NEXT: [[TMP11:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20
; O2-NEXT: [[TMP12:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16 ; O2-NEXT: [[WIDE_LOAD1_2:%.*]] = load <4 x i32>, ptr [[TMP11]], align 4
; O2-NEXT: store <4 x i32> [[TMP11]], ptr [[TMP12]], align 4 ; O2-NEXT: [[TMP12:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[TMP13:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20 ; O2-NEXT: [[TMP13:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_2]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP13]], align 4 ; O2-NEXT: [[TMP14:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16
; O2-NEXT: [[TMP14:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]] ; O2-NEXT: store <4 x i32> [[TMP12]], ptr [[TMP14]], align 4
; O2-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20 ; O2-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20
; O2-NEXT: store <4 x i32> [[TMP14]], ptr [[TMP15]], align 4 ; O2-NEXT: store <4 x i32> [[TMP13]], ptr [[TMP15]], align 4
; O2-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24 ; O2-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24
; O2-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4 ; O2-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4
; O2-NEXT: [[TMP17:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]] ; O2-NEXT: [[TMP17:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28
; O2-NEXT: [[TMP18:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24 ; O2-NEXT: [[WIDE_LOAD1_3:%.*]] = load <4 x i32>, ptr [[TMP17]], align 4
; O2-NEXT: store <4 x i32> [[TMP17]], ptr [[TMP18]], align 4 ; O2-NEXT: [[TMP18:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[TMP19:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28 ; O2-NEXT: [[TMP19:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_3]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP19]], align 4 ; O2-NEXT: [[TMP20:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24
; O2-NEXT: [[TMP20:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]] ; O2-NEXT: store <4 x i32> [[TMP18]], ptr [[TMP20]], align 4
; O2-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28 ; O2-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28
; O2-NEXT: store <4 x i32> [[TMP20]], ptr [[TMP21]], align 4 ; O2-NEXT: store <4 x i32> [[TMP19]], ptr [[TMP21]], align 4
; O2-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32 ; O2-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32
; O2-NEXT: [[WIDE_LOAD_8:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4 ; O2-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4
; O2-NEXT: [[TMP23:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_8]], [[BROADCAST_SPLAT]] ; O2-NEXT: [[TMP23:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36
; O2-NEXT: [[TMP24:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32 ; O2-NEXT: [[WIDE_LOAD1_4:%.*]] = load <4 x i32>, ptr [[TMP23]], align 4
; O2-NEXT: store <4 x i32> [[TMP23]], ptr [[TMP24]], align 4 ; O2-NEXT: [[TMP24:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[TMP25:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36 ; O2-NEXT: [[TMP25:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_4]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[WIDE_LOAD_9:%.*]] = load <4 x i32>, ptr [[TMP25]], align 4 ; O2-NEXT: [[TMP26:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32
; O2-NEXT: [[TMP26:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_9]], [[BROADCAST_SPLAT]] ; O2-NEXT: store <4 x i32> [[TMP24]], ptr [[TMP26]], align 4
; O2-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36 ; O2-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36
; O2-NEXT: store <4 x i32> [[TMP26]], ptr [[TMP27]], align 4 ; O2-NEXT: store <4 x i32> [[TMP25]], ptr [[TMP27]], align 4
; O2-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40 ; O2-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40
; O2-NEXT: [[WIDE_LOAD_10:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4 ; O2-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4
; O2-NEXT: [[TMP29:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_10]], [[BROADCAST_SPLAT]] ; O2-NEXT: [[TMP29:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44
; O2-NEXT: [[TMP30:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40 ; O2-NEXT: [[WIDE_LOAD1_5:%.*]] = load <4 x i32>, ptr [[TMP29]], align 4
; O2-NEXT: store <4 x i32> [[TMP29]], ptr [[TMP30]], align 4 ; O2-NEXT: [[TMP30:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[TMP31:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44 ; O2-NEXT: [[TMP31:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_5]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[WIDE_LOAD_11:%.*]] = load <4 x i32>, ptr [[TMP31]], align 4 ; O2-NEXT: [[TMP32:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40
; O2-NEXT: [[TMP32:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_11]], [[BROADCAST_SPLAT]] ; O2-NEXT: store <4 x i32> [[TMP30]], ptr [[TMP32]], align 4
; O2-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44 ; O2-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44
; O2-NEXT: store <4 x i32> [[TMP32]], ptr [[TMP33]], align 4 ; O2-NEXT: store <4 x i32> [[TMP31]], ptr [[TMP33]], align 4
; O2-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48 ; O2-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48
; O2-NEXT: [[WIDE_LOAD_12:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4 ; O2-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4
; O2-NEXT: [[TMP35:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_12]], [[BROADCAST_SPLAT]] ; O2-NEXT: [[TMP35:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52
; O2-NEXT: [[TMP36:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48 ; O2-NEXT: [[WIDE_LOAD1_6:%.*]] = load <4 x i32>, ptr [[TMP35]], align 4
; O2-NEXT: store <4 x i32> [[TMP35]], ptr [[TMP36]], align 4 ; O2-NEXT: [[TMP36:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[TMP37:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52 ; O2-NEXT: [[TMP37:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_6]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[WIDE_LOAD_13:%.*]] = load <4 x i32>, ptr [[TMP37]], align 4 ; O2-NEXT: [[TMP38:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48
; O2-NEXT: [[TMP38:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_13]], [[BROADCAST_SPLAT]] ; O2-NEXT: store <4 x i32> [[TMP36]], ptr [[TMP38]], align 4
; O2-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52 ; O2-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52
; O2-NEXT: store <4 x i32> [[TMP38]], ptr [[TMP39]], align 4 ; O2-NEXT: store <4 x i32> [[TMP37]], ptr [[TMP39]], align 4
; O2-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56 ; O2-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56
; O2-NEXT: [[WIDE_LOAD_14:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4 ; O2-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4
; O2-NEXT: [[TMP41:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_14]], [[BROADCAST_SPLAT]] ; O2-NEXT: [[TMP41:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60
; O2-NEXT: [[TMP42:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56 ; O2-NEXT: [[WIDE_LOAD1_7:%.*]] = load <4 x i32>, ptr [[TMP41]], align 4
; O2-NEXT: store <4 x i32> [[TMP41]], ptr [[TMP42]], align 4 ; O2-NEXT: [[TMP42:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[TMP43:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60 ; O2-NEXT: [[TMP43:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_7]], [[BROADCAST_SPLAT]]
; O2-NEXT: [[WIDE_LOAD_15:%.*]] = load <4 x i32>, ptr [[TMP43]], align 4 ; O2-NEXT: [[TMP44:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56
; O2-NEXT: [[TMP44:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_15]], [[BROADCAST_SPLAT]] ; O2-NEXT: store <4 x i32> [[TMP42]], ptr [[TMP44]], align 4
; O2-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60 ; O2-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60
; O2-NEXT: store <4 x i32> [[TMP44]], ptr [[TMP45]], align 4 ; O2-NEXT: store <4 x i32> [[TMP43]], ptr [[TMP45]], align 4
; O2-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4 ; O2-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4
; O2-NEXT: ret i32 [[TMP46]] ; O2-NEXT: ret i32 [[TMP46]]
; ;
; O3-LABEL: @nopragma( ; O3-LABEL: @nopragma(
; O3-NEXT: entry: ; O3-NEXT: entry:
; O3-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0 ; O3-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0
; O3-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer ; O3-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer
; O3-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4 ; O3-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4
; O3-NEXT: [[TMP0:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]] ; O3-NEXT: [[TMP0:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4
; O3-NEXT: store <4 x i32> [[TMP0]], ptr [[A:%.*]], align 4 ; O3-NEXT: [[WIDE_LOAD1:%.*]] = load <4 x i32>, ptr [[TMP0]], align 4
; O3-NEXT: [[TMP1:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4 ; O3-NEXT: [[TMP1:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP1]], align 4 ; O3-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]] ; O3-NEXT: store <4 x i32> [[TMP1]], ptr [[A:%.*]], align 4
; O3-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4 ; O3-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4
; O3-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4 ; O3-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4
; O3-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8 ; O3-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8
; O3-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4 ; O3-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4
; O3-NEXT: [[TMP5:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]] ; O3-NEXT: [[TMP5:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12
; O3-NEXT: [[TMP6:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8 ; O3-NEXT: [[WIDE_LOAD1_1:%.*]] = load <4 x i32>, ptr [[TMP5]], align 4
; O3-NEXT: store <4 x i32> [[TMP5]], ptr [[TMP6]], align 4 ; O3-NEXT: [[TMP6:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[TMP7:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12 ; O3-NEXT: [[TMP7:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_1]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP7]], align 4 ; O3-NEXT: [[TMP8:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8
; O3-NEXT: [[TMP8:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]] ; O3-NEXT: store <4 x i32> [[TMP6]], ptr [[TMP8]], align 4
; O3-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12 ; O3-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12
; O3-NEXT: store <4 x i32> [[TMP8]], ptr [[TMP9]], align 4 ; O3-NEXT: store <4 x i32> [[TMP7]], ptr [[TMP9]], align 4
; O3-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16 ; O3-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16
; O3-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4 ; O3-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4
; O3-NEXT: [[TMP11:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]] ; O3-NEXT: [[TMP11:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20
; O3-NEXT: [[TMP12:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16 ; O3-NEXT: [[WIDE_LOAD1_2:%.*]] = load <4 x i32>, ptr [[TMP11]], align 4
; O3-NEXT: store <4 x i32> [[TMP11]], ptr [[TMP12]], align 4 ; O3-NEXT: [[TMP12:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[TMP13:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20 ; O3-NEXT: [[TMP13:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_2]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP13]], align 4 ; O3-NEXT: [[TMP14:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16
; O3-NEXT: [[TMP14:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]] ; O3-NEXT: store <4 x i32> [[TMP12]], ptr [[TMP14]], align 4
; O3-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20 ; O3-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20
; O3-NEXT: store <4 x i32> [[TMP14]], ptr [[TMP15]], align 4 ; O3-NEXT: store <4 x i32> [[TMP13]], ptr [[TMP15]], align 4
; O3-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24 ; O3-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24
; O3-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4 ; O3-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4
; O3-NEXT: [[TMP17:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]] ; O3-NEXT: [[TMP17:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28
; O3-NEXT: [[TMP18:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24 ; O3-NEXT: [[WIDE_LOAD1_3:%.*]] = load <4 x i32>, ptr [[TMP17]], align 4
; O3-NEXT: store <4 x i32> [[TMP17]], ptr [[TMP18]], align 4 ; O3-NEXT: [[TMP18:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[TMP19:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28 ; O3-NEXT: [[TMP19:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_3]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP19]], align 4 ; O3-NEXT: [[TMP20:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24
; O3-NEXT: [[TMP20:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]] ; O3-NEXT: store <4 x i32> [[TMP18]], ptr [[TMP20]], align 4
; O3-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28 ; O3-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28
; O3-NEXT: store <4 x i32> [[TMP20]], ptr [[TMP21]], align 4 ; O3-NEXT: store <4 x i32> [[TMP19]], ptr [[TMP21]], align 4
; O3-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32 ; O3-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32
; O3-NEXT: [[WIDE_LOAD_8:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4 ; O3-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4
; O3-NEXT: [[TMP23:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_8]], [[BROADCAST_SPLAT]] ; O3-NEXT: [[TMP23:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36
; O3-NEXT: [[TMP24:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32 ; O3-NEXT: [[WIDE_LOAD1_4:%.*]] = load <4 x i32>, ptr [[TMP23]], align 4
; O3-NEXT: store <4 x i32> [[TMP23]], ptr [[TMP24]], align 4 ; O3-NEXT: [[TMP24:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[TMP25:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36 ; O3-NEXT: [[TMP25:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_4]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[WIDE_LOAD_9:%.*]] = load <4 x i32>, ptr [[TMP25]], align 4 ; O3-NEXT: [[TMP26:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32
; O3-NEXT: [[TMP26:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_9]], [[BROADCAST_SPLAT]] ; O3-NEXT: store <4 x i32> [[TMP24]], ptr [[TMP26]], align 4
; O3-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36 ; O3-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36
; O3-NEXT: store <4 x i32> [[TMP26]], ptr [[TMP27]], align 4 ; O3-NEXT: store <4 x i32> [[TMP25]], ptr [[TMP27]], align 4
; O3-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40 ; O3-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40
; O3-NEXT: [[WIDE_LOAD_10:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4 ; O3-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4
; O3-NEXT: [[TMP29:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_10]], [[BROADCAST_SPLAT]] ; O3-NEXT: [[TMP29:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44
; O3-NEXT: [[TMP30:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40 ; O3-NEXT: [[WIDE_LOAD1_5:%.*]] = load <4 x i32>, ptr [[TMP29]], align 4
; O3-NEXT: store <4 x i32> [[TMP29]], ptr [[TMP30]], align 4 ; O3-NEXT: [[TMP30:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[TMP31:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44 ; O3-NEXT: [[TMP31:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_5]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[WIDE_LOAD_11:%.*]] = load <4 x i32>, ptr [[TMP31]], align 4 ; O3-NEXT: [[TMP32:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40
; O3-NEXT: [[TMP32:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_11]], [[BROADCAST_SPLAT]] ; O3-NEXT: store <4 x i32> [[TMP30]], ptr [[TMP32]], align 4
; O3-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44 ; O3-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44
; O3-NEXT: store <4 x i32> [[TMP32]], ptr [[TMP33]], align 4 ; O3-NEXT: store <4 x i32> [[TMP31]], ptr [[TMP33]], align 4
; O3-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48 ; O3-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48
; O3-NEXT: [[WIDE_LOAD_12:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4 ; O3-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4
; O3-NEXT: [[TMP35:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_12]], [[BROADCAST_SPLAT]] ; O3-NEXT: [[TMP35:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52
; O3-NEXT: [[TMP36:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48 ; O3-NEXT: [[WIDE_LOAD1_6:%.*]] = load <4 x i32>, ptr [[TMP35]], align 4
; O3-NEXT: store <4 x i32> [[TMP35]], ptr [[TMP36]], align 4 ; O3-NEXT: [[TMP36:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[TMP37:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52 ; O3-NEXT: [[TMP37:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_6]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[WIDE_LOAD_13:%.*]] = load <4 x i32>, ptr [[TMP37]], align 4 ; O3-NEXT: [[TMP38:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48
; O3-NEXT: [[TMP38:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_13]], [[BROADCAST_SPLAT]] ; O3-NEXT: store <4 x i32> [[TMP36]], ptr [[TMP38]], align 4
; O3-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52 ; O3-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52
; O3-NEXT: store <4 x i32> [[TMP38]], ptr [[TMP39]], align 4 ; O3-NEXT: store <4 x i32> [[TMP37]], ptr [[TMP39]], align 4
; O3-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56 ; O3-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56
; O3-NEXT: [[WIDE_LOAD_14:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4 ; O3-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4
; O3-NEXT: [[TMP41:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_14]], [[BROADCAST_SPLAT]] ; O3-NEXT: [[TMP41:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60
; O3-NEXT: [[TMP42:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56 ; O3-NEXT: [[WIDE_LOAD1_7:%.*]] = load <4 x i32>, ptr [[TMP41]], align 4
; O3-NEXT: store <4 x i32> [[TMP41]], ptr [[TMP42]], align 4 ; O3-NEXT: [[TMP42:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[TMP43:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60 ; O3-NEXT: [[TMP43:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_7]], [[BROADCAST_SPLAT]]
; O3-NEXT: [[WIDE_LOAD_15:%.*]] = load <4 x i32>, ptr [[TMP43]], align 4 ; O3-NEXT: [[TMP44:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56
; O3-NEXT: [[TMP44:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_15]], [[BROADCAST_SPLAT]] ; O3-NEXT: store <4 x i32> [[TMP42]], ptr [[TMP44]], align 4
; O3-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60 ; O3-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60
; O3-NEXT: store <4 x i32> [[TMP44]], ptr [[TMP45]], align 4 ; O3-NEXT: store <4 x i32> [[TMP43]], ptr [[TMP45]], align 4
; O3-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4 ; O3-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4
; O3-NEXT: ret i32 [[TMP46]] ; O3-NEXT: ret i32 [[TMP46]]
; ;
; O3DEFAULT-LABEL: @nopragma( ; O3DEFAULT-LABEL: @nopragma(
; O3DEFAULT-NEXT: entry: ; O3DEFAULT-NEXT: entry:
; O3DEFAULT-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0 ; O3DEFAULT-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0
; O3DEFAULT-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer ; O3DEFAULT-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer
; O3DEFAULT-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4 ; O3DEFAULT-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4
; O3DEFAULT-NEXT: [[TMP0:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: [[TMP0:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4
; O3DEFAULT-NEXT: store <4 x i32> [[TMP0]], ptr [[A:%.*]], align 4 ; O3DEFAULT-NEXT: [[WIDE_LOAD1:%.*]] = load <4 x i32>, ptr [[TMP0]], align 4
; O3DEFAULT-NEXT: [[TMP1:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4 ; O3DEFAULT-NEXT: [[TMP1:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP1]], align 4 ; O3DEFAULT-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: store <4 x i32> [[TMP1]], ptr [[A:%.*]], align 4
; O3DEFAULT-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4 ; O3DEFAULT-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4
; O3DEFAULT-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4 ; O3DEFAULT-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4
; O3DEFAULT-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8 ; O3DEFAULT-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8
; O3DEFAULT-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4 ; O3DEFAULT-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4
; O3DEFAULT-NEXT: [[TMP5:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: [[TMP5:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12
; O3DEFAULT-NEXT: [[TMP6:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8 ; O3DEFAULT-NEXT: [[WIDE_LOAD1_1:%.*]] = load <4 x i32>, ptr [[TMP5]], align 4
; O3DEFAULT-NEXT: store <4 x i32> [[TMP5]], ptr [[TMP6]], align 4 ; O3DEFAULT-NEXT: [[TMP6:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[TMP7:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12 ; O3DEFAULT-NEXT: [[TMP7:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_1]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP7]], align 4 ; O3DEFAULT-NEXT: [[TMP8:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8
; O3DEFAULT-NEXT: [[TMP8:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: store <4 x i32> [[TMP6]], ptr [[TMP8]], align 4
; O3DEFAULT-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12 ; O3DEFAULT-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12
; O3DEFAULT-NEXT: store <4 x i32> [[TMP8]], ptr [[TMP9]], align 4 ; O3DEFAULT-NEXT: store <4 x i32> [[TMP7]], ptr [[TMP9]], align 4
; O3DEFAULT-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16 ; O3DEFAULT-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16
; O3DEFAULT-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4 ; O3DEFAULT-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4
; O3DEFAULT-NEXT: [[TMP11:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: [[TMP11:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20
; O3DEFAULT-NEXT: [[TMP12:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16 ; O3DEFAULT-NEXT: [[WIDE_LOAD1_2:%.*]] = load <4 x i32>, ptr [[TMP11]], align 4
; O3DEFAULT-NEXT: store <4 x i32> [[TMP11]], ptr [[TMP12]], align 4 ; O3DEFAULT-NEXT: [[TMP12:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[TMP13:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20 ; O3DEFAULT-NEXT: [[TMP13:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_2]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP13]], align 4 ; O3DEFAULT-NEXT: [[TMP14:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16
; O3DEFAULT-NEXT: [[TMP14:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: store <4 x i32> [[TMP12]], ptr [[TMP14]], align 4
; O3DEFAULT-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20 ; O3DEFAULT-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20
; O3DEFAULT-NEXT: store <4 x i32> [[TMP14]], ptr [[TMP15]], align 4 ; O3DEFAULT-NEXT: store <4 x i32> [[TMP13]], ptr [[TMP15]], align 4
; O3DEFAULT-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24 ; O3DEFAULT-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24
; O3DEFAULT-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4 ; O3DEFAULT-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4
; O3DEFAULT-NEXT: [[TMP17:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: [[TMP17:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28
; O3DEFAULT-NEXT: [[TMP18:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24 ; O3DEFAULT-NEXT: [[WIDE_LOAD1_3:%.*]] = load <4 x i32>, ptr [[TMP17]], align 4
; O3DEFAULT-NEXT: store <4 x i32> [[TMP17]], ptr [[TMP18]], align 4 ; O3DEFAULT-NEXT: [[TMP18:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[TMP19:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28 ; O3DEFAULT-NEXT: [[TMP19:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_3]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP19]], align 4 ; O3DEFAULT-NEXT: [[TMP20:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24
; O3DEFAULT-NEXT: [[TMP20:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: store <4 x i32> [[TMP18]], ptr [[TMP20]], align 4
; O3DEFAULT-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28 ; O3DEFAULT-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28
; O3DEFAULT-NEXT: store <4 x i32> [[TMP20]], ptr [[TMP21]], align 4 ; O3DEFAULT-NEXT: store <4 x i32> [[TMP19]], ptr [[TMP21]], align 4
; O3DEFAULT-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32 ; O3DEFAULT-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32
; O3DEFAULT-NEXT: [[WIDE_LOAD_8:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4 ; O3DEFAULT-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4
; O3DEFAULT-NEXT: [[TMP23:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_8]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: [[TMP23:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36
; O3DEFAULT-NEXT: [[TMP24:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32 ; O3DEFAULT-NEXT: [[WIDE_LOAD1_4:%.*]] = load <4 x i32>, ptr [[TMP23]], align 4
; O3DEFAULT-NEXT: store <4 x i32> [[TMP23]], ptr [[TMP24]], align 4 ; O3DEFAULT-NEXT: [[TMP24:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[TMP25:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36 ; O3DEFAULT-NEXT: [[TMP25:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_4]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[WIDE_LOAD_9:%.*]] = load <4 x i32>, ptr [[TMP25]], align 4 ; O3DEFAULT-NEXT: [[TMP26:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32
; O3DEFAULT-NEXT: [[TMP26:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_9]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: store <4 x i32> [[TMP24]], ptr [[TMP26]], align 4
; O3DEFAULT-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36 ; O3DEFAULT-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36
; O3DEFAULT-NEXT: store <4 x i32> [[TMP26]], ptr [[TMP27]], align 4 ; O3DEFAULT-NEXT: store <4 x i32> [[TMP25]], ptr [[TMP27]], align 4
; O3DEFAULT-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40 ; O3DEFAULT-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40
; O3DEFAULT-NEXT: [[WIDE_LOAD_10:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4 ; O3DEFAULT-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4
; O3DEFAULT-NEXT: [[TMP29:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_10]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: [[TMP29:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44
; O3DEFAULT-NEXT: [[TMP30:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40 ; O3DEFAULT-NEXT: [[WIDE_LOAD1_5:%.*]] = load <4 x i32>, ptr [[TMP29]], align 4
; O3DEFAULT-NEXT: store <4 x i32> [[TMP29]], ptr [[TMP30]], align 4 ; O3DEFAULT-NEXT: [[TMP30:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[TMP31:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44 ; O3DEFAULT-NEXT: [[TMP31:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_5]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[WIDE_LOAD_11:%.*]] = load <4 x i32>, ptr [[TMP31]], align 4 ; O3DEFAULT-NEXT: [[TMP32:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40
; O3DEFAULT-NEXT: [[TMP32:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_11]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: store <4 x i32> [[TMP30]], ptr [[TMP32]], align 4
; O3DEFAULT-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44 ; O3DEFAULT-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44
; O3DEFAULT-NEXT: store <4 x i32> [[TMP32]], ptr [[TMP33]], align 4 ; O3DEFAULT-NEXT: store <4 x i32> [[TMP31]], ptr [[TMP33]], align 4
; O3DEFAULT-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48 ; O3DEFAULT-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48
; O3DEFAULT-NEXT: [[WIDE_LOAD_12:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4 ; O3DEFAULT-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4
; O3DEFAULT-NEXT: [[TMP35:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_12]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: [[TMP35:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52
; O3DEFAULT-NEXT: [[TMP36:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48 ; O3DEFAULT-NEXT: [[WIDE_LOAD1_6:%.*]] = load <4 x i32>, ptr [[TMP35]], align 4
; O3DEFAULT-NEXT: store <4 x i32> [[TMP35]], ptr [[TMP36]], align 4 ; O3DEFAULT-NEXT: [[TMP36:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[TMP37:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52 ; O3DEFAULT-NEXT: [[TMP37:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_6]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[WIDE_LOAD_13:%.*]] = load <4 x i32>, ptr [[TMP37]], align 4 ; O3DEFAULT-NEXT: [[TMP38:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48
; O3DEFAULT-NEXT: [[TMP38:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_13]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: store <4 x i32> [[TMP36]], ptr [[TMP38]], align 4
; O3DEFAULT-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52 ; O3DEFAULT-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52
; O3DEFAULT-NEXT: store <4 x i32> [[TMP38]], ptr [[TMP39]], align 4 ; O3DEFAULT-NEXT: store <4 x i32> [[TMP37]], ptr [[TMP39]], align 4
; O3DEFAULT-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56 ; O3DEFAULT-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56
; O3DEFAULT-NEXT: [[WIDE_LOAD_14:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4 ; O3DEFAULT-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4
; O3DEFAULT-NEXT: [[TMP41:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_14]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: [[TMP41:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60
; O3DEFAULT-NEXT: [[TMP42:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56 ; O3DEFAULT-NEXT: [[WIDE_LOAD1_7:%.*]] = load <4 x i32>, ptr [[TMP41]], align 4
; O3DEFAULT-NEXT: store <4 x i32> [[TMP41]], ptr [[TMP42]], align 4 ; O3DEFAULT-NEXT: [[TMP42:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[TMP43:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60 ; O3DEFAULT-NEXT: [[TMP43:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_7]], [[BROADCAST_SPLAT]]
; O3DEFAULT-NEXT: [[WIDE_LOAD_15:%.*]] = load <4 x i32>, ptr [[TMP43]], align 4 ; O3DEFAULT-NEXT: [[TMP44:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56
; O3DEFAULT-NEXT: [[TMP44:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_15]], [[BROADCAST_SPLAT]] ; O3DEFAULT-NEXT: store <4 x i32> [[TMP42]], ptr [[TMP44]], align 4
; O3DEFAULT-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60 ; O3DEFAULT-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60
; O3DEFAULT-NEXT: store <4 x i32> [[TMP44]], ptr [[TMP45]], align 4 ; O3DEFAULT-NEXT: store <4 x i32> [[TMP43]], ptr [[TMP45]], align 4
; O3DEFAULT-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4 ; O3DEFAULT-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4
; O3DEFAULT-NEXT: ret i32 [[TMP46]] ; O3DEFAULT-NEXT: ret i32 [[TMP46]]
; ;
; Os-LABEL: @nopragma( ; Os-LABEL: @nopragma(
; Os-NEXT: entry: ; Os-NEXT: entry:
; Os-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0 ; Os-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0
; Os-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer ; Os-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer
; Os-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4 ; Os-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[B:%.*]], align 4
; Os-NEXT: [[TMP0:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]] ; Os-NEXT: [[TMP0:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4
; Os-NEXT: store <4 x i32> [[TMP0]], ptr [[A:%.*]], align 4 ; Os-NEXT: [[WIDE_LOAD1:%.*]] = load <4 x i32>, ptr [[TMP0]], align 4
; Os-NEXT: [[TMP1:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 4 ; Os-NEXT: [[TMP1:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP1]], align 4 ; Os-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[TMP2:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]] ; Os-NEXT: store <4 x i32> [[TMP1]], ptr [[A:%.*]], align 4
; Os-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4 ; Os-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 4
; Os-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4 ; Os-NEXT: store <4 x i32> [[TMP2]], ptr [[TMP3]], align 4
; Os-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8 ; Os-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 8
; Os-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4 ; Os-NEXT: [[WIDE_LOAD_1:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4
; Os-NEXT: [[TMP5:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]] ; Os-NEXT: [[TMP5:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12
; Os-NEXT: [[TMP6:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8 ; Os-NEXT: [[WIDE_LOAD1_1:%.*]] = load <4 x i32>, ptr [[TMP5]], align 4
; Os-NEXT: store <4 x i32> [[TMP5]], ptr [[TMP6]], align 4 ; Os-NEXT: [[TMP6:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_1]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[TMP7:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 12 ; Os-NEXT: [[TMP7:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_1]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP7]], align 4 ; Os-NEXT: [[TMP8:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 8
; Os-NEXT: [[TMP8:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]] ; Os-NEXT: store <4 x i32> [[TMP6]], ptr [[TMP8]], align 4
; Os-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12 ; Os-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 12
; Os-NEXT: store <4 x i32> [[TMP8]], ptr [[TMP9]], align 4 ; Os-NEXT: store <4 x i32> [[TMP7]], ptr [[TMP9]], align 4
; Os-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16 ; Os-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 16
; Os-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4 ; Os-NEXT: [[WIDE_LOAD_2:%.*]] = load <4 x i32>, ptr [[TMP10]], align 4
; Os-NEXT: [[TMP11:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]] ; Os-NEXT: [[TMP11:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20
; Os-NEXT: [[TMP12:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16 ; Os-NEXT: [[WIDE_LOAD1_2:%.*]] = load <4 x i32>, ptr [[TMP11]], align 4
; Os-NEXT: store <4 x i32> [[TMP11]], ptr [[TMP12]], align 4 ; Os-NEXT: [[TMP12:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_2]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[TMP13:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 20 ; Os-NEXT: [[TMP13:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_2]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP13]], align 4 ; Os-NEXT: [[TMP14:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 16
; Os-NEXT: [[TMP14:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]] ; Os-NEXT: store <4 x i32> [[TMP12]], ptr [[TMP14]], align 4
; Os-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20 ; Os-NEXT: [[TMP15:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 20
; Os-NEXT: store <4 x i32> [[TMP14]], ptr [[TMP15]], align 4 ; Os-NEXT: store <4 x i32> [[TMP13]], ptr [[TMP15]], align 4
; Os-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24 ; Os-NEXT: [[TMP16:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 24
; Os-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4 ; Os-NEXT: [[WIDE_LOAD_3:%.*]] = load <4 x i32>, ptr [[TMP16]], align 4
; Os-NEXT: [[TMP17:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]] ; Os-NEXT: [[TMP17:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28
; Os-NEXT: [[TMP18:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24 ; Os-NEXT: [[WIDE_LOAD1_3:%.*]] = load <4 x i32>, ptr [[TMP17]], align 4
; Os-NEXT: store <4 x i32> [[TMP17]], ptr [[TMP18]], align 4 ; Os-NEXT: [[TMP18:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_3]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[TMP19:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 28 ; Os-NEXT: [[TMP19:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_3]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP19]], align 4 ; Os-NEXT: [[TMP20:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 24
; Os-NEXT: [[TMP20:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]] ; Os-NEXT: store <4 x i32> [[TMP18]], ptr [[TMP20]], align 4
; Os-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28 ; Os-NEXT: [[TMP21:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 28
; Os-NEXT: store <4 x i32> [[TMP20]], ptr [[TMP21]], align 4 ; Os-NEXT: store <4 x i32> [[TMP19]], ptr [[TMP21]], align 4
; Os-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32 ; Os-NEXT: [[TMP22:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 32
; Os-NEXT: [[WIDE_LOAD_8:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4 ; Os-NEXT: [[WIDE_LOAD_4:%.*]] = load <4 x i32>, ptr [[TMP22]], align 4
; Os-NEXT: [[TMP23:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_8]], [[BROADCAST_SPLAT]] ; Os-NEXT: [[TMP23:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36
; Os-NEXT: [[TMP24:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32 ; Os-NEXT: [[WIDE_LOAD1_4:%.*]] = load <4 x i32>, ptr [[TMP23]], align 4
; Os-NEXT: store <4 x i32> [[TMP23]], ptr [[TMP24]], align 4 ; Os-NEXT: [[TMP24:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_4]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[TMP25:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 36 ; Os-NEXT: [[TMP25:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_4]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[WIDE_LOAD_9:%.*]] = load <4 x i32>, ptr [[TMP25]], align 4 ; Os-NEXT: [[TMP26:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 32
; Os-NEXT: [[TMP26:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_9]], [[BROADCAST_SPLAT]] ; Os-NEXT: store <4 x i32> [[TMP24]], ptr [[TMP26]], align 4
; Os-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36 ; Os-NEXT: [[TMP27:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 36
; Os-NEXT: store <4 x i32> [[TMP26]], ptr [[TMP27]], align 4 ; Os-NEXT: store <4 x i32> [[TMP25]], ptr [[TMP27]], align 4
; Os-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40 ; Os-NEXT: [[TMP28:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 40
; Os-NEXT: [[WIDE_LOAD_10:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4 ; Os-NEXT: [[WIDE_LOAD_5:%.*]] = load <4 x i32>, ptr [[TMP28]], align 4
; Os-NEXT: [[TMP29:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_10]], [[BROADCAST_SPLAT]] ; Os-NEXT: [[TMP29:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44
; Os-NEXT: [[TMP30:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40 ; Os-NEXT: [[WIDE_LOAD1_5:%.*]] = load <4 x i32>, ptr [[TMP29]], align 4
; Os-NEXT: store <4 x i32> [[TMP29]], ptr [[TMP30]], align 4 ; Os-NEXT: [[TMP30:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_5]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[TMP31:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 44 ; Os-NEXT: [[TMP31:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_5]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[WIDE_LOAD_11:%.*]] = load <4 x i32>, ptr [[TMP31]], align 4 ; Os-NEXT: [[TMP32:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 40
; Os-NEXT: [[TMP32:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_11]], [[BROADCAST_SPLAT]] ; Os-NEXT: store <4 x i32> [[TMP30]], ptr [[TMP32]], align 4
; Os-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44 ; Os-NEXT: [[TMP33:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 44
; Os-NEXT: store <4 x i32> [[TMP32]], ptr [[TMP33]], align 4 ; Os-NEXT: store <4 x i32> [[TMP31]], ptr [[TMP33]], align 4
; Os-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48 ; Os-NEXT: [[TMP34:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 48
; Os-NEXT: [[WIDE_LOAD_12:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4 ; Os-NEXT: [[WIDE_LOAD_6:%.*]] = load <4 x i32>, ptr [[TMP34]], align 4
; Os-NEXT: [[TMP35:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_12]], [[BROADCAST_SPLAT]] ; Os-NEXT: [[TMP35:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52
; Os-NEXT: [[TMP36:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48 ; Os-NEXT: [[WIDE_LOAD1_6:%.*]] = load <4 x i32>, ptr [[TMP35]], align 4
; Os-NEXT: store <4 x i32> [[TMP35]], ptr [[TMP36]], align 4 ; Os-NEXT: [[TMP36:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_6]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[TMP37:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 52 ; Os-NEXT: [[TMP37:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_6]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[WIDE_LOAD_13:%.*]] = load <4 x i32>, ptr [[TMP37]], align 4 ; Os-NEXT: [[TMP38:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 48
; Os-NEXT: [[TMP38:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_13]], [[BROADCAST_SPLAT]] ; Os-NEXT: store <4 x i32> [[TMP36]], ptr [[TMP38]], align 4
; Os-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52 ; Os-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 52
; Os-NEXT: store <4 x i32> [[TMP38]], ptr [[TMP39]], align 4 ; Os-NEXT: store <4 x i32> [[TMP37]], ptr [[TMP39]], align 4
; Os-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56 ; Os-NEXT: [[TMP40:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 56
; Os-NEXT: [[WIDE_LOAD_14:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4 ; Os-NEXT: [[WIDE_LOAD_7:%.*]] = load <4 x i32>, ptr [[TMP40]], align 4
; Os-NEXT: [[TMP41:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_14]], [[BROADCAST_SPLAT]] ; Os-NEXT: [[TMP41:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60
; Os-NEXT: [[TMP42:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56 ; Os-NEXT: [[WIDE_LOAD1_7:%.*]] = load <4 x i32>, ptr [[TMP41]], align 4
; Os-NEXT: store <4 x i32> [[TMP41]], ptr [[TMP42]], align 4 ; Os-NEXT: [[TMP42:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_7]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[TMP43:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 60 ; Os-NEXT: [[TMP43:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1_7]], [[BROADCAST_SPLAT]]
; Os-NEXT: [[WIDE_LOAD_15:%.*]] = load <4 x i32>, ptr [[TMP43]], align 4 ; Os-NEXT: [[TMP44:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 56
; Os-NEXT: [[TMP44:%.*]] = add nsw <4 x i32> [[WIDE_LOAD_15]], [[BROADCAST_SPLAT]] ; Os-NEXT: store <4 x i32> [[TMP42]], ptr [[TMP44]], align 4
; Os-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60 ; Os-NEXT: [[TMP45:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 60
; Os-NEXT: store <4 x i32> [[TMP44]], ptr [[TMP45]], align 4 ; Os-NEXT: store <4 x i32> [[TMP43]], ptr [[TMP45]], align 4
; Os-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4 ; Os-NEXT: [[TMP46:%.*]] = load i32, ptr [[A]], align 4
; Os-NEXT: ret i32 [[TMP46]] ; Os-NEXT: ret i32 [[TMP46]]
; ;
; Oz-LABEL: @nopragma( ; Oz-LABEL: @nopragma(
; Oz-NEXT: entry: ; Oz-NEXT: entry:
; Oz-NEXT: br label [[FOR_BODY:%.*]] ; Oz-NEXT: br label [[FOR_BODY:%.*]]
; Oz: for.body: ; Oz: for.body:
; Oz-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY]] ] ; Oz-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY]] ]
Show All 14 Lines
; O1VEC2-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]] ; O1VEC2-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]]
; O1VEC2: vector.ph: ; O1VEC2: vector.ph:
; O1VEC2-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0 ; O1VEC2-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0
; O1VEC2-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer ; O1VEC2-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer
; O1VEC2-NEXT: br label [[VECTOR_BODY:%.*]] ; O1VEC2-NEXT: br label [[VECTOR_BODY:%.*]]
; O1VEC2: vector.body: ; O1VEC2: vector.body:
; O1VEC2-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ] ; O1VEC2-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
; O1VEC2-NEXT: [[TMP0:%.*]] = add i64 [[INDEX]], 0 ; O1VEC2-NEXT: [[TMP0:%.*]] = add i64 [[INDEX]], 0
; O1VEC2-NEXT: [[TMP1:%.*]] = getelementptr inbounds i32, ptr [[B:%.*]], i64 [[TMP0]] ; O1VEC2-NEXT: [[TMP1:%.*]] = add i64 [[INDEX]], 4
; O1VEC2-NEXT: [[TMP2:%.*]] = getelementptr inbounds i32, ptr [[TMP1]], i32 0 ; O1VEC2-NEXT: [[TMP2:%.*]] = getelementptr inbounds i32, ptr [[B:%.*]], i64 [[TMP0]]
; O1VEC2-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[TMP2]], align 4 ; O1VEC2-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 [[TMP1]]
; O1VEC2-NEXT: [[TMP3:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]] ; O1VEC2-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[TMP2]], i32 0
; O1VEC2-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[TMP0]] ; O1VEC2-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4
; O1VEC2-NEXT: [[TMP5:%.*]] = getelementptr inbounds i32, ptr [[TMP4]], i32 0 ; O1VEC2-NEXT: [[TMP5:%.*]] = getelementptr inbounds i32, ptr [[TMP2]], i32 4
; O1VEC2-NEXT: store <4 x i32> [[TMP3]], ptr [[TMP5]], align 4 ; O1VEC2-NEXT: [[WIDE_LOAD1:%.*]] = load <4 x i32>, ptr [[TMP5]], align 4
; O1VEC2-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 4 ; O1VEC2-NEXT: [[TMP6:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]]
; O1VEC2-NEXT: [[TMP6:%.*]] = icmp eq i64 [[INDEX_NEXT]], 64 ; O1VEC2-NEXT: [[TMP7:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1]], [[BROADCAST_SPLAT]]
; O1VEC2-NEXT: br i1 [[TMP6]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP0:![0-9]+]] ; O1VEC2-NEXT: [[TMP8:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[TMP0]]
; O1VEC2-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 [[TMP1]]
; O1VEC2-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[TMP8]], i32 0
; O1VEC2-NEXT: store <4 x i32> [[TMP6]], ptr [[TMP10]], align 4
; O1VEC2-NEXT: [[TMP11:%.*]] = getelementptr inbounds i32, ptr [[TMP8]], i32 4
; O1VEC2-NEXT: store <4 x i32> [[TMP7]], ptr [[TMP11]], align 4
; O1VEC2-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 8
; O1VEC2-NEXT: [[TMP12:%.*]] = icmp eq i64 [[INDEX_NEXT]], 64
; O1VEC2-NEXT: br i1 [[TMP12]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP0:![0-9]+]]
; O1VEC2: middle.block: ; O1VEC2: middle.block:
; O1VEC2-NEXT: br i1 true, label [[FOR_END:%.*]], label [[SCALAR_PH]] ; O1VEC2-NEXT: br i1 true, label [[FOR_END:%.*]], label [[SCALAR_PH]]
; O1VEC2: scalar.ph: ; O1VEC2: scalar.ph:
; O1VEC2-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 64, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ] ; O1VEC2-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 64, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]
; O1VEC2-NEXT: br label [[FOR_BODY:%.*]] ; O1VEC2-NEXT: br label [[FOR_BODY:%.*]]
; O1VEC2: for.body: ; O1VEC2: for.body:
; O1VEC2-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY]] ] ; O1VEC2-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY]] ]
; O1VEC2-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 [[INDVARS_IV]] ; O1VEC2-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 [[INDVARS_IV]]
; O1VEC2-NEXT: [[TMP7:%.*]] = load i32, ptr [[ARRAYIDX]], align 4 ; O1VEC2-NEXT: [[TMP13:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
; O1VEC2-NEXT: [[ADD:%.*]] = add nsw i32 [[TMP7]], [[N]] ; O1VEC2-NEXT: [[ADD:%.*]] = add nsw i32 [[TMP13]], [[N]]
; O1VEC2-NEXT: [[ARRAYIDX2:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 [[INDVARS_IV]] ; O1VEC2-NEXT: [[ARRAYIDX2:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 [[INDVARS_IV]]
; O1VEC2-NEXT: store i32 [[ADD]], ptr [[ARRAYIDX2]], align 4 ; O1VEC2-NEXT: store i32 [[ADD]], ptr [[ARRAYIDX2]], align 4
; O1VEC2-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1 ; O1VEC2-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1
; O1VEC2-NEXT: [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], 64 ; O1VEC2-NEXT: [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], 64
; O1VEC2-NEXT: br i1 [[EXITCOND]], label [[FOR_END]], label [[FOR_BODY]], !llvm.loop [[LOOP3:![0-9]+]] ; O1VEC2-NEXT: br i1 [[EXITCOND]], label [[FOR_END]], label [[FOR_BODY]], !llvm.loop [[LOOP3:![0-9]+]]
; O1VEC2: for.end: ; O1VEC2: for.end:
; O1VEC2-NEXT: [[TMP8:%.*]] = load i32, ptr [[A]], align 4 ; O1VEC2-NEXT: [[TMP14:%.*]] = load i32, ptr [[A]], align 4
; O1VEC2-NEXT: ret i32 [[TMP8]] ; O1VEC2-NEXT: ret i32 [[TMP14]]
; ;
; OzVEC2-LABEL: @nopragma( ; OzVEC2-LABEL: @nopragma(
; OzVEC2-NEXT: entry: ; OzVEC2-NEXT: entry:
; OzVEC2-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]] ; OzVEC2-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]]
; OzVEC2: vector.ph: ; OzVEC2: vector.ph:
; OzVEC2-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0 ; OzVEC2-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <4 x i32> poison, i32 [[N:%.*]], i64 0
; OzVEC2-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer ; OzVEC2-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <4 x i32> [[BROADCAST_SPLATINSERT]], <4 x i32> poison, <4 x i32> zeroinitializer
; OzVEC2-NEXT: br label [[VECTOR_BODY:%.*]] ; OzVEC2-NEXT: br label [[VECTOR_BODY:%.*]]
; OzVEC2: vector.body: ; OzVEC2: vector.body:
; OzVEC2-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ] ; OzVEC2-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
; OzVEC2-NEXT: [[TMP0:%.*]] = add i64 [[INDEX]], 0 ; OzVEC2-NEXT: [[TMP0:%.*]] = add i64 [[INDEX]], 0
; OzVEC2-NEXT: [[TMP1:%.*]] = getelementptr inbounds i32, ptr [[B:%.*]], i64 [[TMP0]] ; OzVEC2-NEXT: [[TMP1:%.*]] = add i64 [[INDEX]], 4
; OzVEC2-NEXT: [[TMP2:%.*]] = getelementptr inbounds i32, ptr [[TMP1]], i32 0 ; OzVEC2-NEXT: [[TMP2:%.*]] = getelementptr inbounds i32, ptr [[B:%.*]], i64 [[TMP0]]
; OzVEC2-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[TMP2]], align 4 ; OzVEC2-NEXT: [[TMP3:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 [[TMP1]]
; OzVEC2-NEXT: [[TMP3:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]] ; OzVEC2-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[TMP2]], i32 0
; OzVEC2-NEXT: [[TMP4:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[TMP0]] ; OzVEC2-NEXT: [[WIDE_LOAD:%.*]] = load <4 x i32>, ptr [[TMP4]], align 4
; OzVEC2-NEXT: [[TMP5:%.*]] = getelementptr inbounds i32, ptr [[TMP4]], i32 0 ; OzVEC2-NEXT: [[TMP5:%.*]] = getelementptr inbounds i32, ptr [[TMP2]], i32 4
; OzVEC2-NEXT: store <4 x i32> [[TMP3]], ptr [[TMP5]], align 4 ; OzVEC2-NEXT: [[WIDE_LOAD1:%.*]] = load <4 x i32>, ptr [[TMP5]], align 4
; OzVEC2-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 4 ; OzVEC2-NEXT: [[TMP6:%.*]] = add nsw <4 x i32> [[WIDE_LOAD]], [[BROADCAST_SPLAT]]
; OzVEC2-NEXT: [[TMP6:%.*]] = icmp eq i64 [[INDEX_NEXT]], 64 ; OzVEC2-NEXT: [[TMP7:%.*]] = add nsw <4 x i32> [[WIDE_LOAD1]], [[BROADCAST_SPLAT]]
; OzVEC2-NEXT: br i1 [[TMP6]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP0:![0-9]+]] ; OzVEC2-NEXT: [[TMP8:%.*]] = getelementptr inbounds i32, ptr [[A:%.*]], i64 [[TMP0]]
; OzVEC2-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 [[TMP1]]
; OzVEC2-NEXT: [[TMP10:%.*]] = getelementptr inbounds i32, ptr [[TMP8]], i32 0
; OzVEC2-NEXT: store <4 x i32> [[TMP6]], ptr [[TMP10]], align 4
; OzVEC2-NEXT: [[TMP11:%.*]] = getelementptr inbounds i32, ptr [[TMP8]], i32 4
; OzVEC2-NEXT: store <4 x i32> [[TMP7]], ptr [[TMP11]], align 4
; OzVEC2-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 8
; OzVEC2-NEXT: [[TMP12:%.*]] = icmp eq i64 [[INDEX_NEXT]], 64
; OzVEC2-NEXT: br i1 [[TMP12]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP0:![0-9]+]]
; OzVEC2: middle.block: ; OzVEC2: middle.block:
; OzVEC2-NEXT: br i1 true, label [[FOR_END:%.*]], label [[SCALAR_PH]] ; OzVEC2-NEXT: br i1 true, label [[FOR_END:%.*]], label [[SCALAR_PH]]
; OzVEC2: scalar.ph: ; OzVEC2: scalar.ph:
; OzVEC2-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 64, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ] ; OzVEC2-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 64, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]
; OzVEC2-NEXT: br label [[FOR_BODY:%.*]] ; OzVEC2-NEXT: br label [[FOR_BODY:%.*]]
; OzVEC2: for.body: ; OzVEC2: for.body:
; OzVEC2-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY]] ] ; OzVEC2-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY]] ]
; OzVEC2-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 [[INDVARS_IV]] ; OzVEC2-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[B]], i64 [[INDVARS_IV]]
; OzVEC2-NEXT: [[TMP7:%.*]] = load i32, ptr [[ARRAYIDX]], align 4 ; OzVEC2-NEXT: [[TMP13:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
; OzVEC2-NEXT: [[ADD:%.*]] = add nsw i32 [[TMP7]], [[N]] ; OzVEC2-NEXT: [[ADD:%.*]] = add nsw i32 [[TMP13]], [[N]]
; OzVEC2-NEXT: [[ARRAYIDX2:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 [[INDVARS_IV]] ; OzVEC2-NEXT: [[ARRAYIDX2:%.*]] = getelementptr inbounds i32, ptr [[A]], i64 [[INDVARS_IV]]
; OzVEC2-NEXT: store i32 [[ADD]], ptr [[ARRAYIDX2]], align 4 ; OzVEC2-NEXT: store i32 [[ADD]], ptr [[ARRAYIDX2]], align 4
; OzVEC2-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1 ; OzVEC2-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1
; OzVEC2-NEXT: [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], 64 ; OzVEC2-NEXT: [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], 64
; OzVEC2-NEXT: br i1 [[EXITCOND]], label [[FOR_END]], label [[FOR_BODY]], !llvm.loop [[LOOP3:![0-9]+]] ; OzVEC2-NEXT: br i1 [[EXITCOND]], label [[FOR_END]], label [[FOR_BODY]], !llvm.loop [[LOOP3:![0-9]+]]
; OzVEC2: for.end: ; OzVEC2: for.end:
; OzVEC2-NEXT: [[TMP8:%.*]] = load i32, ptr [[A]], align 4 ; OzVEC2-NEXT: [[TMP14:%.*]] = load i32, ptr [[A]], align 4
; OzVEC2-NEXT: ret i32 [[TMP8]] ; OzVEC2-NEXT: ret i32 [[TMP14]]
; ;
; O3DIS-LABEL: @nopragma( ; O3DIS-LABEL: @nopragma(
; O3DIS-NEXT: entry: ; O3DIS-NEXT: entry:
; O3DIS-NEXT: br label [[FOR_BODY:%.*]] ; O3DIS-NEXT: br label [[FOR_BODY:%.*]]
; O3DIS: for.body: ; O3DIS: for.body:
; O3DIS-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY]] ] ; O3DIS-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY]] ]
; O3DIS-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[B:%.*]], i64 [[INDVARS_IV]] ; O3DIS-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[B:%.*]], i64 [[INDVARS_IV]]
; O3DIS-NEXT: [[TMP0:%.*]] = load i32, ptr [[ARRAYIDX]], align 4 ; O3DIS-NEXT: [[TMP0:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
▲ Show 20 Lines • Show All 254 Lines • Show Last 20 Lines

llvm/test/Transforms/LoopVectorize/X86/pr42674.ll

; NOTE: Assertions have been autogenerated by utils/update_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
; RUN: opt %s -passes=loop-vectorize,instcombine,simplifycfg -simplifycfg-require-and-preserve-domtree=1 -mtriple=x86_64-unknown-linux-gnu -mattr=avx512vl,avx512dq,avx512bw -S | FileCheck %s ; RUN: opt %s -passes=loop-vectorize,instcombine,simplifycfg -simplifycfg-require-and-preserve-domtree=1 -mtriple=x86_64-unknown-linux-gnu -mattr=avx512vl,avx512dq,avx512bw -S | FileCheck %s
@bytes = global [128 x i8] zeroinitializer, align 16 @bytes = global [128 x i8] zeroinitializer, align 16
; Make sure we end up with vector code for this loop. We used to try to create ; Make sure we end up with vector code for this loop. We used to try to create
; a VF=64,UF=4 loop, but the scalar trip count is only 128 so ; a VF=64,UF=4 loop, but the scalar trip count is only 128 so
; the vector loop was dead code leaving only a scalar remainder. ; the vector loop was dead code leaving only a scalar remainder.
define zeroext i8 @sum() { define zeroext i8 @sum() {
; CHECK-LABEL: @sum( ; CHECK-LABEL: @sum(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[WIDE_LOAD2:%.*]] = load <64 x i8>, ptr getelementptr inbounds ([128 x i8], ptr @bytes, i64 0, i64 64), align 1 ; CHECK-NEXT: br label [[VECTOR_BODY:%.*]]
; CHECK-NEXT: [[WIDE_LOAD:%.*]] = load <64 x i8>, ptr @bytes, align 1 ; CHECK: vector.body:
; CHECK-NEXT: [[BIN_RDX:%.*]] = add <64 x i8> [[WIDE_LOAD2]], [[WIDE_LOAD]] ; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[TMP0:%.*]] = call i8 @llvm.vector.reduce.add.v64i8(<64 x i8> [[BIN_RDX]]) ; CHECK-NEXT: [[VEC_PHI:%.*]] = phi <64 x i8> [ zeroinitializer, [[ENTRY]] ], [ [[TMP1:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: ret i8 [[TMP0]] ; CHECK-NEXT: [[TMP0:%.*]] = getelementptr inbounds [128 x i8], ptr @bytes, i64 0, i64 [[INDEX]]
; CHECK-NEXT: [[WIDE_LOAD:%.*]] = load <64 x i8>, ptr [[TMP0]], align 1
; CHECK-NEXT: [[TMP1]] = add <64 x i8> [[WIDE_LOAD]], [[VEC_PHI]]
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 64
; CHECK-NEXT: [[TMP2:%.*]] = icmp eq i64 [[INDEX_NEXT]], 128
; CHECK-NEXT: br i1 [[TMP2]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP0:![0-9]+]]
; CHECK: middle.block:
; CHECK-NEXT: [[TMP3:%.*]] = call i8 @llvm.vector.reduce.add.v64i8(<64 x i8> [[TMP1]])
; CHECK-NEXT: ret i8 [[TMP3]]
; ;
entry: entry:
br label %for.body br label %for.body
for.body: ; preds = %for.body, %entry for.body: ; preds = %for.body, %entry
%indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %for.body ] %indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %for.body ]
%r.010 = phi i8 [ 0, %entry ], [ %add, %for.body ] %r.010 = phi i8 [ 0, %entry ], [ %add, %for.body ]
%arrayidx = getelementptr inbounds [128 x i8], ptr @bytes, i64 0, i64 %indvars.iv %arrayidx = getelementptr inbounds [128 x i8], ptr @bytes, i64 0, i64 %indvars.iv
Show All 10 Lines

llvm/test/Transforms/LoopVectorize/X86/strided_load_cost.ll

Show All 12 Lines
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[IDXPROM:%.*]] = sext i32 [[I:%.*]] to i64 ; CHECK-NEXT: [[IDXPROM:%.*]] = sext i32 [[I:%.*]] to i64
; CHECK-NEXT: [[IDXPROM5:%.*]] = sext i32 [[J:%.*]] to i64 ; CHECK-NEXT: [[IDXPROM5:%.*]] = sext i32 [[J:%.*]] to i64
; CHECK-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]] ; CHECK-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]]
; CHECK: vector.ph: ; CHECK: vector.ph:
; CHECK-NEXT: br label [[VECTOR_BODY:%.*]] ; CHECK-NEXT: br label [[VECTOR_BODY:%.*]]
; CHECK: vector.body: ; CHECK: vector.body:
; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ] ; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[VEC_PHI:%.*]] = phi <8 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP36:%.*]], [[VECTOR_BODY]] ] ; CHECK-NEXT: [[VEC_PHI:%.*]] = phi <8 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP144:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[VEC_PHI1:%.*]] = phi <8 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP145:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[VEC_PHI2:%.*]] = phi <8 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP146:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[VEC_PHI3:%.*]] = phi <8 x i32> [ zeroinitializer, [[VECTOR_PH]] ], [ [[TMP147:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[TMP0:%.*]] = add i64 [[INDEX]], 0 ; CHECK-NEXT: [[TMP0:%.*]] = add i64 [[INDEX]], 0
; CHECK-NEXT: [[TMP1:%.*]] = add i64 [[INDEX]], 1 ; CHECK-NEXT: [[TMP1:%.*]] = add i64 [[INDEX]], 1
; CHECK-NEXT: [[TMP2:%.*]] = add i64 [[INDEX]], 2 ; CHECK-NEXT: [[TMP2:%.*]] = add i64 [[INDEX]], 2
; CHECK-NEXT: [[TMP3:%.*]] = add i64 [[INDEX]], 3 ; CHECK-NEXT: [[TMP3:%.*]] = add i64 [[INDEX]], 3
; CHECK-NEXT: [[TMP4:%.*]] = add i64 [[INDEX]], 4 ; CHECK-NEXT: [[TMP4:%.*]] = add i64 [[INDEX]], 4
; CHECK-NEXT: [[TMP5:%.*]] = add i64 [[INDEX]], 5 ; CHECK-NEXT: [[TMP5:%.*]] = add i64 [[INDEX]], 5
; CHECK-NEXT: [[TMP6:%.*]] = add i64 [[INDEX]], 6 ; CHECK-NEXT: [[TMP6:%.*]] = add i64 [[INDEX]], 6
; CHECK-NEXT: [[TMP7:%.*]] = add i64 [[INDEX]], 7 ; CHECK-NEXT: [[TMP7:%.*]] = add i64 [[INDEX]], 7
; CHECK-NEXT: [[TMP8:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA:%.*]], i64 [[IDXPROM]], i64 [[TMP0]] ; CHECK-NEXT: [[TMP8:%.*]] = add i64 [[INDEX]], 8
; CHECK-NEXT: [[TMP9:%.*]] = getelementptr inbounds i32, ptr [[TMP8]], i32 0 ; CHECK-NEXT: [[TMP9:%.*]] = add i64 [[INDEX]], 9
; CHECK-NEXT: [[WIDE_LOAD:%.*]] = load <8 x i32>, ptr [[TMP9]], align 4, !tbaa [[TBAA1:![0-9]+]] ; CHECK-NEXT: [[TMP10:%.*]] = add i64 [[INDEX]], 10
; CHECK-NEXT: [[TMP10:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP0]], i64 [[IDXPROM5]] ; CHECK-NEXT: [[TMP11:%.*]] = add i64 [[INDEX]], 11
; CHECK-NEXT: [[TMP11:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP1]], i64 [[IDXPROM5]] ; CHECK-NEXT: [[TMP12:%.*]] = add i64 [[INDEX]], 12
; CHECK-NEXT: [[TMP12:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP2]], i64 [[IDXPROM5]] ; CHECK-NEXT: [[TMP13:%.*]] = add i64 [[INDEX]], 13
; CHECK-NEXT: [[TMP13:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP3]], i64 [[IDXPROM5]] ; CHECK-NEXT: [[TMP14:%.*]] = add i64 [[INDEX]], 14
; CHECK-NEXT: [[TMP14:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP4]], i64 [[IDXPROM5]] ; CHECK-NEXT: [[TMP15:%.*]] = add i64 [[INDEX]], 15
; CHECK-NEXT: [[TMP15:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP5]], i64 [[IDXPROM5]] ; CHECK-NEXT: [[TMP16:%.*]] = add i64 [[INDEX]], 16
; CHECK-NEXT: [[TMP16:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP6]], i64 [[IDXPROM5]] ; CHECK-NEXT: [[TMP17:%.*]] = add i64 [[INDEX]], 17
; CHECK-NEXT: [[TMP17:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP7]], i64 [[IDXPROM5]] ; CHECK-NEXT: [[TMP18:%.*]] = add i64 [[INDEX]], 18
; CHECK-NEXT: [[TMP18:%.*]] = load i32, ptr [[TMP10]], align 4, !tbaa [[TBAA1]] ; CHECK-NEXT: [[TMP19:%.*]] = add i64 [[INDEX]], 19
; CHECK-NEXT: [[TMP19:%.*]] = load i32, ptr [[TMP11]], align 4, !tbaa [[TBAA1]] ; CHECK-NEXT: [[TMP20:%.*]] = add i64 [[INDEX]], 20
; CHECK-NEXT: [[TMP20:%.*]] = load i32, ptr [[TMP12]], align 4, !tbaa [[TBAA1]] ; CHECK-NEXT: [[TMP21:%.*]] = add i64 [[INDEX]], 21
; CHECK-NEXT: [[TMP21:%.*]] = load i32, ptr [[TMP13]], align 4, !tbaa [[TBAA1]] ; CHECK-NEXT: [[TMP22:%.*]] = add i64 [[INDEX]], 22
; CHECK-NEXT: [[TMP22:%.*]] = load i32, ptr [[TMP14]], align 4, !tbaa [[TBAA1]] ; CHECK-NEXT: [[TMP23:%.*]] = add i64 [[INDEX]], 23
; CHECK-NEXT: [[TMP23:%.*]] = load i32, ptr [[TMP15]], align 4, !tbaa [[TBAA1]] ; CHECK-NEXT: [[TMP24:%.*]] = add i64 [[INDEX]], 24
; CHECK-NEXT: [[TMP24:%.*]] = load i32, ptr [[TMP16]], align 4, !tbaa [[TBAA1]] ; CHECK-NEXT: [[TMP25:%.*]] = add i64 [[INDEX]], 25
; CHECK-NEXT: [[TMP25:%.*]] = load i32, ptr [[TMP17]], align 4, !tbaa [[TBAA1]] ; CHECK-NEXT: [[TMP26:%.*]] = add i64 [[INDEX]], 26
; CHECK-NEXT: [[TMP26:%.*]] = insertelement <8 x i32> poison, i32 [[TMP18]], i32 0 ; CHECK-NEXT: [[TMP27:%.*]] = add i64 [[INDEX]], 27
; CHECK-NEXT: [[TMP27:%.*]] = insertelement <8 x i32> [[TMP26]], i32 [[TMP19]], i32 1 ; CHECK-NEXT: [[TMP28:%.*]] = add i64 [[INDEX]], 28
; CHECK-NEXT: [[TMP28:%.*]] = insertelement <8 x i32> [[TMP27]], i32 [[TMP20]], i32 2 ; CHECK-NEXT: [[TMP29:%.*]] = add i64 [[INDEX]], 29
; CHECK-NEXT: [[TMP29:%.*]] = insertelement <8 x i32> [[TMP28]], i32 [[TMP21]], i32 3 ; CHECK-NEXT: [[TMP30:%.*]] = add i64 [[INDEX]], 30
; CHECK-NEXT: [[TMP30:%.*]] = insertelement <8 x i32> [[TMP29]], i32 [[TMP22]], i32 4 ; CHECK-NEXT: [[TMP31:%.*]] = add i64 [[INDEX]], 31
; CHECK-NEXT: [[TMP31:%.*]] = insertelement <8 x i32> [[TMP30]], i32 [[TMP23]], i32 5 ; CHECK-NEXT: [[TMP32:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA:%.*]], i64 [[IDXPROM]], i64 [[TMP0]]
; CHECK-NEXT: [[TMP32:%.*]] = insertelement <8 x i32> [[TMP31]], i32 [[TMP24]], i32 6 ; CHECK-NEXT: [[TMP33:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[IDXPROM]], i64 [[TMP8]]
; CHECK-NEXT: [[TMP33:%.*]] = insertelement <8 x i32> [[TMP32]], i32 [[TMP25]], i32 7 ; CHECK-NEXT: [[TMP34:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[IDXPROM]], i64 [[TMP16]]
; CHECK-NEXT: [[TMP34:%.*]] = mul nsw <8 x i32> [[TMP33]], [[WIDE_LOAD]] ; CHECK-NEXT: [[TMP35:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[IDXPROM]], i64 [[TMP24]]
; CHECK-NEXT: [[TMP35:%.*]] = add <8 x i32> [[VEC_PHI]], <i32 4, i32 4, i32 4, i32 4, i32 4, i32 4, i32 4, i32 4> ; CHECK-NEXT: [[TMP36:%.*]] = getelementptr inbounds i32, ptr [[TMP32]], i32 0
; CHECK-NEXT: [[TMP36]] = add <8 x i32> [[TMP35]], [[TMP34]] ; CHECK-NEXT: [[WIDE_LOAD:%.*]] = load <8 x i32>, ptr [[TMP36]], align 4, !tbaa [[TBAA1:![0-9]+]]
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 8 ; CHECK-NEXT: [[TMP37:%.*]] = getelementptr inbounds i32, ptr [[TMP32]], i32 8
; CHECK-NEXT: [[TMP37:%.*]] = icmp eq i64 [[INDEX_NEXT]], 96 ; CHECK-NEXT: [[WIDE_LOAD4:%.*]] = load <8 x i32>, ptr [[TMP37]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: br i1 [[TMP37]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP5:![0-9]+]] ; CHECK-NEXT: [[TMP38:%.*]] = getelementptr inbounds i32, ptr [[TMP32]], i32 16
; CHECK-NEXT: [[WIDE_LOAD5:%.*]] = load <8 x i32>, ptr [[TMP38]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP39:%.*]] = getelementptr inbounds i32, ptr [[TMP32]], i32 24
; CHECK-NEXT: [[WIDE_LOAD6:%.*]] = load <8 x i32>, ptr [[TMP39]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP40:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP0]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP41:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP1]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP42:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP2]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP43:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP3]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP44:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP4]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP45:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP5]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP46:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP6]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP47:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP7]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP48:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP8]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP49:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP9]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP50:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP10]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP51:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP11]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP52:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP12]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP53:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP13]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP54:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP14]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP55:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP15]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP56:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP16]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP57:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP17]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP58:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP18]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP59:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP19]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP60:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP20]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP61:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP21]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP62:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP22]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP63:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP23]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP64:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP24]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP65:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP25]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP66:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP26]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP67:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP27]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP68:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP28]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP69:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP29]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP70:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP30]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP71:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[TMP31]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP72:%.*]] = load i32, ptr [[TMP40]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP73:%.*]] = load i32, ptr [[TMP41]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP74:%.*]] = load i32, ptr [[TMP42]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP75:%.*]] = load i32, ptr [[TMP43]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP76:%.*]] = load i32, ptr [[TMP44]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP77:%.*]] = load i32, ptr [[TMP45]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP78:%.*]] = load i32, ptr [[TMP46]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP79:%.*]] = load i32, ptr [[TMP47]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP80:%.*]] = insertelement <8 x i32> poison, i32 [[TMP72]], i32 0
; CHECK-NEXT: [[TMP81:%.*]] = insertelement <8 x i32> [[TMP80]], i32 [[TMP73]], i32 1
; CHECK-NEXT: [[TMP82:%.*]] = insertelement <8 x i32> [[TMP81]], i32 [[TMP74]], i32 2
; CHECK-NEXT: [[TMP83:%.*]] = insertelement <8 x i32> [[TMP82]], i32 [[TMP75]], i32 3
; CHECK-NEXT: [[TMP84:%.*]] = insertelement <8 x i32> [[TMP83]], i32 [[TMP76]], i32 4
; CHECK-NEXT: [[TMP85:%.*]] = insertelement <8 x i32> [[TMP84]], i32 [[TMP77]], i32 5
; CHECK-NEXT: [[TMP86:%.*]] = insertelement <8 x i32> [[TMP85]], i32 [[TMP78]], i32 6
; CHECK-NEXT: [[TMP87:%.*]] = insertelement <8 x i32> [[TMP86]], i32 [[TMP79]], i32 7
; CHECK-NEXT: [[TMP88:%.*]] = load i32, ptr [[TMP48]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP89:%.*]] = load i32, ptr [[TMP49]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP90:%.*]] = load i32, ptr [[TMP50]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP91:%.*]] = load i32, ptr [[TMP51]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP92:%.*]] = load i32, ptr [[TMP52]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP93:%.*]] = load i32, ptr [[TMP53]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP94:%.*]] = load i32, ptr [[TMP54]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP95:%.*]] = load i32, ptr [[TMP55]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP96:%.*]] = insertelement <8 x i32> poison, i32 [[TMP88]], i32 0
; CHECK-NEXT: [[TMP97:%.*]] = insertelement <8 x i32> [[TMP96]], i32 [[TMP89]], i32 1
; CHECK-NEXT: [[TMP98:%.*]] = insertelement <8 x i32> [[TMP97]], i32 [[TMP90]], i32 2
; CHECK-NEXT: [[TMP99:%.*]] = insertelement <8 x i32> [[TMP98]], i32 [[TMP91]], i32 3
; CHECK-NEXT: [[TMP100:%.*]] = insertelement <8 x i32> [[TMP99]], i32 [[TMP92]], i32 4
; CHECK-NEXT: [[TMP101:%.*]] = insertelement <8 x i32> [[TMP100]], i32 [[TMP93]], i32 5
; CHECK-NEXT: [[TMP102:%.*]] = insertelement <8 x i32> [[TMP101]], i32 [[TMP94]], i32 6
; CHECK-NEXT: [[TMP103:%.*]] = insertelement <8 x i32> [[TMP102]], i32 [[TMP95]], i32 7
; CHECK-NEXT: [[TMP104:%.*]] = load i32, ptr [[TMP56]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP105:%.*]] = load i32, ptr [[TMP57]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP106:%.*]] = load i32, ptr [[TMP58]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP107:%.*]] = load i32, ptr [[TMP59]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP108:%.*]] = load i32, ptr [[TMP60]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP109:%.*]] = load i32, ptr [[TMP61]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP110:%.*]] = load i32, ptr [[TMP62]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP111:%.*]] = load i32, ptr [[TMP63]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP112:%.*]] = insertelement <8 x i32> poison, i32 [[TMP104]], i32 0
; CHECK-NEXT: [[TMP113:%.*]] = insertelement <8 x i32> [[TMP112]], i32 [[TMP105]], i32 1
; CHECK-NEXT: [[TMP114:%.*]] = insertelement <8 x i32> [[TMP113]], i32 [[TMP106]], i32 2
; CHECK-NEXT: [[TMP115:%.*]] = insertelement <8 x i32> [[TMP114]], i32 [[TMP107]], i32 3
; CHECK-NEXT: [[TMP116:%.*]] = insertelement <8 x i32> [[TMP115]], i32 [[TMP108]], i32 4
; CHECK-NEXT: [[TMP117:%.*]] = insertelement <8 x i32> [[TMP116]], i32 [[TMP109]], i32 5
; CHECK-NEXT: [[TMP118:%.*]] = insertelement <8 x i32> [[TMP117]], i32 [[TMP110]], i32 6
; CHECK-NEXT: [[TMP119:%.*]] = insertelement <8 x i32> [[TMP118]], i32 [[TMP111]], i32 7
; CHECK-NEXT: [[TMP120:%.*]] = load i32, ptr [[TMP64]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP121:%.*]] = load i32, ptr [[TMP65]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP122:%.*]] = load i32, ptr [[TMP66]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP123:%.*]] = load i32, ptr [[TMP67]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP124:%.*]] = load i32, ptr [[TMP68]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP125:%.*]] = load i32, ptr [[TMP69]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP126:%.*]] = load i32, ptr [[TMP70]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP127:%.*]] = load i32, ptr [[TMP71]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[TMP128:%.*]] = insertelement <8 x i32> poison, i32 [[TMP120]], i32 0
; CHECK-NEXT: [[TMP129:%.*]] = insertelement <8 x i32> [[TMP128]], i32 [[TMP121]], i32 1
; CHECK-NEXT: [[TMP130:%.*]] = insertelement <8 x i32> [[TMP129]], i32 [[TMP122]], i32 2
; CHECK-NEXT: [[TMP131:%.*]] = insertelement <8 x i32> [[TMP130]], i32 [[TMP123]], i32 3
; CHECK-NEXT: [[TMP132:%.*]] = insertelement <8 x i32> [[TMP131]], i32 [[TMP124]], i32 4
; CHECK-NEXT: [[TMP133:%.*]] = insertelement <8 x i32> [[TMP132]], i32 [[TMP125]], i32 5
; CHECK-NEXT: [[TMP134:%.*]] = insertelement <8 x i32> [[TMP133]], i32 [[TMP126]], i32 6
; CHECK-NEXT: [[TMP135:%.*]] = insertelement <8 x i32> [[TMP134]], i32 [[TMP127]], i32 7
; CHECK-NEXT: [[TMP136:%.*]] = mul nsw <8 x i32> [[TMP87]], [[WIDE_LOAD]]
; CHECK-NEXT: [[TMP137:%.*]] = mul nsw <8 x i32> [[TMP103]], [[WIDE_LOAD4]]
; CHECK-NEXT: [[TMP138:%.*]] = mul nsw <8 x i32> [[TMP119]], [[WIDE_LOAD5]]
; CHECK-NEXT: [[TMP139:%.*]] = mul nsw <8 x i32> [[TMP135]], [[WIDE_LOAD6]]
; CHECK-NEXT: [[TMP140:%.*]] = add <8 x i32> [[VEC_PHI]], <i32 4, i32 4, i32 4, i32 4, i32 4, i32 4, i32 4, i32 4>
; CHECK-NEXT: [[TMP141:%.*]] = add <8 x i32> [[VEC_PHI1]], <i32 4, i32 4, i32 4, i32 4, i32 4, i32 4, i32 4, i32 4>
; CHECK-NEXT: [[TMP142:%.*]] = add <8 x i32> [[VEC_PHI2]], <i32 4, i32 4, i32 4, i32 4, i32 4, i32 4, i32 4, i32 4>
; CHECK-NEXT: [[TMP143:%.*]] = add <8 x i32> [[VEC_PHI3]], <i32 4, i32 4, i32 4, i32 4, i32 4, i32 4, i32 4, i32 4>
; CHECK-NEXT: [[TMP144]] = add <8 x i32> [[TMP140]], [[TMP136]]
; CHECK-NEXT: [[TMP145]] = add <8 x i32> [[TMP141]], [[TMP137]]
; CHECK-NEXT: [[TMP146]] = add <8 x i32> [[TMP142]], [[TMP138]]
; CHECK-NEXT: [[TMP147]] = add <8 x i32> [[TMP143]], [[TMP139]]
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 32
; CHECK-NEXT: [[TMP148:%.*]] = icmp eq i64 [[INDEX_NEXT]], 96
; CHECK-NEXT: br i1 [[TMP148]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP5:![0-9]+]]
; CHECK: middle.block: ; CHECK: middle.block:
; CHECK-NEXT: [[TMP38:%.*]] = call i32 @llvm.vector.reduce.add.v8i32(<8 x i32> [[TMP36]]) ; CHECK-NEXT: [[BIN_RDX:%.*]] = add <8 x i32> [[TMP145]], [[TMP144]]
; CHECK-NEXT: [[BIN_RDX7:%.*]] = add <8 x i32> [[TMP146]], [[BIN_RDX]]
; CHECK-NEXT: [[BIN_RDX8:%.*]] = add <8 x i32> [[TMP147]], [[BIN_RDX7]]
; CHECK-NEXT: [[TMP149:%.*]] = call i32 @llvm.vector.reduce.add.v8i32(<8 x i32> [[BIN_RDX8]])
; CHECK-NEXT: br i1 false, label [[FOR_COND_CLEANUP:%.*]], label [[SCALAR_PH]] ; CHECK-NEXT: br i1 false, label [[FOR_COND_CLEANUP:%.*]], label [[SCALAR_PH]]
; CHECK: scalar.ph: ; CHECK: scalar.ph:
; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 96, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ] ; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 96, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]
; CHECK-NEXT: [[BC_MERGE_RDX:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[TMP38]], [[MIDDLE_BLOCK]] ] ; CHECK-NEXT: [[BC_MERGE_RDX:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[TMP149]], [[MIDDLE_BLOCK]] ]
; CHECK-NEXT: br label [[FOR_BODY:%.*]] ; CHECK-NEXT: br label [[FOR_BODY:%.*]]
; CHECK: for.cond.cleanup: ; CHECK: for.cond.cleanup:
; CHECK-NEXT: [[ADD7_LCSSA:%.*]] = phi i32 [ [[ADD7:%.*]], [[FOR_BODY]] ], [ [[TMP38]], [[MIDDLE_BLOCK]] ] ; CHECK-NEXT: [[ADD7_LCSSA:%.*]] = phi i32 [ [[ADD7:%.*]], [[FOR_BODY]] ], [ [[TMP149]], [[MIDDLE_BLOCK]] ]
; CHECK-NEXT: ret i32 [[ADD7_LCSSA]] ; CHECK-NEXT: ret i32 [[ADD7_LCSSA]]
; CHECK: for.body: ; CHECK: for.body:
; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY]] ] ; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY]] ]
; CHECK-NEXT: [[SUM_015:%.*]] = phi i32 [ [[BC_MERGE_RDX]], [[SCALAR_PH]] ], [ [[ADD7]], [[FOR_BODY]] ] ; CHECK-NEXT: [[SUM_015:%.*]] = phi i32 [ [[BC_MERGE_RDX]], [[SCALAR_PH]] ], [ [[ADD7]], [[FOR_BODY]] ]
; CHECK-NEXT: [[ARRAYIDX2:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[IDXPROM]], i64 [[INDVARS_IV]] ; CHECK-NEXT: [[ARRAYIDX2:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[IDXPROM]], i64 [[INDVARS_IV]]
; CHECK-NEXT: [[TMP39:%.*]] = load i32, ptr [[ARRAYIDX2]], align 4, !tbaa [[TBAA1]] ; CHECK-NEXT: [[TMP150:%.*]] = load i32, ptr [[ARRAYIDX2]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[ARRAYIDX6:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[INDVARS_IV]], i64 [[IDXPROM5]] ; CHECK-NEXT: [[ARRAYIDX6:%.*]] = getelementptr inbounds [100 x i32], ptr [[DATA]], i64 [[INDVARS_IV]], i64 [[IDXPROM5]]
; CHECK-NEXT: [[TMP40:%.*]] = load i32, ptr [[ARRAYIDX6]], align 4, !tbaa [[TBAA1]] ; CHECK-NEXT: [[TMP151:%.*]] = load i32, ptr [[ARRAYIDX6]], align 4, !tbaa [[TBAA1]]
; CHECK-NEXT: [[MUL:%.*]] = mul nsw i32 [[TMP40]], [[TMP39]] ; CHECK-NEXT: [[MUL:%.*]] = mul nsw i32 [[TMP151]], [[TMP150]]
; CHECK-NEXT: [[ADD:%.*]] = add i32 [[SUM_015]], 4 ; CHECK-NEXT: [[ADD:%.*]] = add i32 [[SUM_015]], 4
; CHECK-NEXT: [[ADD7]] = add i32 [[ADD]], [[MUL]] ; CHECK-NEXT: [[ADD7]] = add i32 [[ADD]], [[MUL]]
; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1
; CHECK-NEXT: [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], 100 ; CHECK-NEXT: [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], 100
; CHECK-NEXT: br i1 [[EXITCOND]], label [[FOR_COND_CLEANUP]], label [[FOR_BODY]], !llvm.loop [[LOOP7:![0-9]+]] ; CHECK-NEXT: br i1 [[EXITCOND]], label [[FOR_COND_CLEANUP]], label [[FOR_BODY]], !llvm.loop [[LOOP8:![0-9]+]]
; ;
entry: entry:
%idxprom = sext i32 %i to i64 %idxprom = sext i32 %i to i64
%idxprom5 = sext i32 %j to i64 %idxprom5 = sext i32 %j to i64
br label %for.body br label %for.body
for.cond.cleanup: ; preds = %for.body for.cond.cleanup: ; preds = %for.body
ret i32 %add7 ret i32 %add7
Show All 27 Lines

llvm/test/Transforms/LoopVectorize/X86/unroll-small-loops.ll

; RUN: opt < %s -passes=loop-vectorize,dce -mtriple=x86_64-apple-macosx10.8.0 -mcpu=corei7-avx -force-vector-width=4 -force-vector-interleave=0 -S \ ; RUN: opt < %s -passes=loop-vectorize,dce -mtriple=x86_64-apple-macosx10.8.0 -mcpu=corei7-avx -force-vector-width=4 -force-vector-interleave=0 -S \
; RUN: | FileCheck %s --check-prefix=CHECK-VECTOR ; RUN: | FileCheck %s --check-prefix=CHECK-VECTOR
; RUN: opt < %s -passes=loop-vectorize,dce -mtriple=x86_64-apple-macosx10.8.0 -mcpu=corei7-avx -force-vector-width=1 -force-vector-interleave=0 -S \ ; RUN: opt < %s -passes=loop-vectorize,dce -mtriple=x86_64-apple-macosx10.8.0 -mcpu=corei7-avx -force-vector-width=1 -force-vector-interleave=0 -S \
; RUN: | FileCheck %s --check-prefix=CHECK-SCALAR ; RUN: | FileCheck %s --check-prefix=CHECK-SCALAR
target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128" target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
target triple = "x86_64-apple-macosx10.8.0" target triple = "x86_64-apple-macosx10.8.0"
; We don't unroll this loop because it has a small constant trip count. ; We don't unroll this loop because it has a small constant trip count.
; ;
; CHECK-VECTOR-LABEL: @foo( ; CHECK-VECTOR-LABEL: @foo_trip_count_8(
; CHECK-VECTOR: load <4 x i32> ; CHECK-VECTOR: load <4 x i32>
; CHECK-VECTOR-NOT: load <4 x i32> ; CHECK-VECTOR-NOT: load <4 x i32>
; CHECK-VECTOR: store <4 x i32> ; CHECK-VECTOR: store <4 x i32>
; CHECK-VECTOR-NOT: store <4 x i32> ; CHECK-VECTOR-NOT: store <4 x i32>
; CHECK-VECTOR: ret ; CHECK-VECTOR: ret
; ;
; CHECK-SCALAR-LABEL: @foo( ; CHECK-SCALAR-LABEL: @foo_trip_count_8(
; CHECK-SCALAR: load i32, ptr ; CHECK-SCALAR: load i32, ptr
; CHECK-SCALAR-NOT: load i32, ptr ; CHECK-SCALAR-NOT: load i32, ptr
; CHECK-SCALAR: store i32 ; CHECK-SCALAR: store i32
; CHECK-SCALAR-NOT: store i32 ; CHECK-SCALAR-NOT: store i32
; CHECK-SCALAR: ret ; CHECK-SCALAR: ret
define i32 @foo(ptr nocapture %A) nounwind uwtable ssp { define i32 @foo_trip_count_8(ptr nocapture %A) nounwind uwtable ssp {
br label %1
; <label>:1 ; preds = %1, %0
%indvars.iv = phi i64 [ 0, %0 ], [ %indvars.iv.next, %1 ]
%2 = getelementptr inbounds i32, ptr %A, i64 %indvars.iv
%3 = load i32, ptr %2, align 4
%4 = add nsw i32 %3, 6
store i32 %4, ptr %2, align 4
%indvars.iv.next = add i64 %indvars.iv, 1
%lftr.wideiv = trunc i64 %indvars.iv.next to i32
%exitcond = icmp eq i32 %lftr.wideiv, 8
br i1 %exitcond, label %5, label %1
; <label>:5 ; preds = %1
ret i32 undef
}
; We unroll this loop twice to keep it profitable for a trip count of 16
; and VF of 4 by allowing the vectorized loop to run at least twice
;
; CHECK-VECTOR-LABEL: @foo_trip_count_16(
; CHECK-VECTOR: load <4 x i32>
; CHECK-VECTOR: load <4 x i32>
; CHECK-VECTOR-NOT: load <4 x i32>
; CHECK-VECTOR: store <4 x i32>
; CHECK-VECTOR: store <4 x i32>
; CHECK-VECTOR-NOT: store <4 x i32>
; CHECK-VECTOR: ret
;
; CHECK-SCALAR-LABEL: @foo_trip_count_16(
; CHECK-SCALAR: load i32, ptr
; CHECK-SCALAR-NOT: load i32, ptr
; CHECK-SCALAR: store i32
; CHECK-SCALAR-NOT: store i32
; CHECK-SCALAR: ret
define i32 @foo_trip_count_16(ptr nocapture %A) nounwind uwtable ssp {
br label %1
; <label>:1 ; preds = %1, %0
%indvars.iv = phi i64 [ 0, %0 ], [ %indvars.iv.next, %1 ]
%2 = getelementptr inbounds i32, ptr %A, i64 %indvars.iv
%3 = load i32, ptr %2, align 4
%4 = add nsw i32 %3, 6
store i32 %4, ptr %2, align 4
%indvars.iv.next = add i64 %indvars.iv, 1
%lftr.wideiv = trunc i64 %indvars.iv.next to i32
%exitcond = icmp eq i32 %lftr.wideiv, 16
br i1 %exitcond, label %5, label %1
; <label>:5 ; preds = %1
ret i32 undef
}
; We unroll this loop thrice to keep it profitable for a trip count of 24
; and VF of 4 by allowing the vectorized loop to run at least twice
;
; CHECK-VECTOR-LABEL: @foo_trip_count_24(
; CHECK-VECTOR: load <4 x i32>
; CHECK-VECTOR: load <4 x i32>
; CHECK-VECTOR: load <4 x i32>
; CHECK-VECTOR-NOT: load <4 x i32>
; CHECK-VECTOR: store <4 x i32>
; CHECK-VECTOR: store <4 x i32>
; CHECK-VECTOR: store <4 x i32>
; CHECK-VECTOR-NOT: store <4 x i32>
; CHECK-VECTOR: ret
;
; CHECK-SCALAR-LABEL: @foo_trip_count_24(
; CHECK-SCALAR: load i32, ptr
; CHECK-SCALAR-NOT: load i32, ptr
; CHECK-SCALAR: store i32
; CHECK-SCALAR-NOT: store i32
; CHECK-SCALAR: ret
define i32 @foo_trip_count_24(ptr nocapture %A) nounwind uwtable ssp {
br label %1
; <label>:1 ; preds = %1, %0
%indvars.iv = phi i64 [ 0, %0 ], [ %indvars.iv.next, %1 ]
%2 = getelementptr inbounds i32, ptr %A, i64 %indvars.iv
%3 = load i32, ptr %2, align 4
%4 = add nsw i32 %3, 6
store i32 %4, ptr %2, align 4
%indvars.iv.next = add i64 %indvars.iv, 1
%lftr.wideiv = trunc i64 %indvars.iv.next to i32
%exitcond = icmp eq i32 %lftr.wideiv, 24
br i1 %exitcond, label %5, label %1
; <label>:5 ; preds = %1
ret i32 undef
}
; We unroll this loop 4 times to keep it profitable for a trip count of 32
; and VF of 4 by allowing the vectorized loop to run at least twice
;
; CHECK-VECTOR-LABEL: @foo_trip_count_32(
; CHECK-VECTOR: load <4 x i32>
; CHECK-VECTOR: load <4 x i32>
; CHECK-VECTOR: load <4 x i32>
; CHECK-VECTOR: load <4 x i32>
; CHECK-VECTOR-NOT: load <4 x i32>
; CHECK-VECTOR: store <4 x i32>
; CHECK-VECTOR: store <4 x i32>
; CHECK-VECTOR: store <4 x i32>
; CHECK-VECTOR: store <4 x i32>
; CHECK-VECTOR-NOT: store <4 x i32>
; CHECK-VECTOR: ret
;
; CHECK-SCALAR-LABEL: @foo_trip_count_32(
; CHECK-SCALAR: load i32, ptr
; CHECK-SCALAR-NOT: load i32, ptr
; CHECK-SCALAR: store i32
; CHECK-SCALAR-NOT: store i32
; CHECK-SCALAR: ret
define i32 @foo_trip_count_32(ptr nocapture %A) nounwind uwtable ssp {
br label %1
; <label>:1 ; preds = %1, %0
%indvars.iv = phi i64 [ 0, %0 ], [ %indvars.iv.next, %1 ]
%2 = getelementptr inbounds i32, ptr %A, i64 %indvars.iv
%3 = load i32, ptr %2, align 4
%4 = add nsw i32 %3, 6
store i32 %4, ptr %2, align 4
%indvars.iv.next = add i64 %indvars.iv, 1
%lftr.wideiv = trunc i64 %indvars.iv.next to i32
%exitcond = icmp eq i32 %lftr.wideiv, 32
br i1 %exitcond, label %5, label %1
; <label>:5 ; preds = %1
ret i32 undef
}
; We unroll this loop 4 times to keep it profitable for a trip count of 100
; and VF of 4 by allowing the vectorized loop to run at least twice
;
; CHECK-VECTOR-LABEL: @foo_trip_count_100(
; CHECK-VECTOR: load <4 x i32>
; CHECK-VECTOR: load <4 x i32>
; CHECK-VECTOR: load <4 x i32>
; CHECK-VECTOR: load <4 x i32>
; CHECK-VECTOR-NOT: load <4 x i32>
; CHECK-VECTOR: store <4 x i32>
; CHECK-VECTOR: store <4 x i32>
; CHECK-VECTOR: store <4 x i32>
; CHECK-VECTOR: store <4 x i32>
; CHECK-VECTOR-NOT: store <4 x i32>
; CHECK-VECTOR: ret
;
; CHECK-SCALAR-LABEL: @foo_trip_count_100(
; CHECK-SCALAR: load i32, ptr
; CHECK-SCALAR-NOT: load i32, ptr
; CHECK-SCALAR: store i32
; CHECK-SCALAR-NOT: store i32
; CHECK-SCALAR: ret
define i32 @foo_trip_count_100(ptr nocapture %A) nounwind uwtable ssp {
br label %1 br label %1
; <label>:1 ; preds = %1, %0 ; <label>:1 ; preds = %1, %0
%indvars.iv = phi i64 [ 0, %0 ], [ %indvars.iv.next, %1 ] %indvars.iv = phi i64 [ 0, %0 ], [ %indvars.iv.next, %1 ]
%2 = getelementptr inbounds i32, ptr %A, i64 %indvars.iv %2 = getelementptr inbounds i32, ptr %A, i64 %indvars.iv
%3 = load i32, ptr %2, align 4 %3 = load i32, ptr %2, align 4
%4 = add nsw i32 %3, 6 %4 = add nsw i32 %3, 6
store i32 %4, ptr %2, align 4 store i32 %4, ptr %2, align 4
▲ Show 20 Lines • Show All 70 Lines • Show Last 20 Lines

llvm/test/Transforms/LoopVectorize/X86/vect.omp.force.small-tc.ll

Show All 21 Lines
; CHECK-LABEL: @vectorized( ; CHECK-LABEL: @vectorized(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]] ; CHECK-NEXT: br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]]
; CHECK: vector.ph: ; CHECK: vector.ph:
; CHECK-NEXT: br label [[VECTOR_BODY:%.*]] ; CHECK-NEXT: br label [[VECTOR_BODY:%.*]]
; CHECK: vector.body: ; CHECK: vector.body:
; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ] ; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[TMP0:%.*]] = add i64 [[INDEX]], 0 ; CHECK-NEXT: [[TMP0:%.*]] = add i64 [[INDEX]], 0
; CHECK-NEXT: [[TMP1:%.*]] = getelementptr inbounds float, ptr [[B:%.*]], i64 [[TMP0]] ; CHECK-NEXT: [[TMP1:%.*]] = add i64 [[INDEX]], 4
; CHECK-NEXT: [[TMP2:%.*]] = getelementptr inbounds float, ptr [[TMP1]], i32 0 ; CHECK-NEXT: [[TMP2:%.*]] = getelementptr inbounds float, ptr [[B:%.*]], i64 [[TMP0]]
; CHECK-NEXT: [[WIDE_LOAD:%.*]] = load <4 x float>, ptr [[TMP2]], align 4, !llvm.access.group [[ACC_GRP0:![0-9]+]] ; CHECK-NEXT: [[TMP3:%.*]] = getelementptr inbounds float, ptr [[B]], i64 [[TMP1]]
; CHECK-NEXT: [[TMP3:%.*]] = getelementptr inbounds float, ptr [[A:%.*]], i64 [[TMP0]] ; CHECK-NEXT: [[TMP4:%.*]] = getelementptr inbounds float, ptr [[TMP2]], i32 0
; CHECK-NEXT: [[TMP4:%.*]] = getelementptr inbounds float, ptr [[TMP3]], i32 0 ; CHECK-NEXT: [[WIDE_LOAD:%.*]] = load <4 x float>, ptr [[TMP4]], align 4, !llvm.access.group [[ACC_GRP0:![0-9]+]]
; CHECK-NEXT: [[WIDE_LOAD1:%.*]] = load <4 x float>, ptr [[TMP4]], align 4, !llvm.access.group [[ACC_GRP0]] ; CHECK-NEXT: [[TMP5:%.*]] = getelementptr inbounds float, ptr [[TMP2]], i32 4
; CHECK-NEXT: [[TMP5:%.*]] = fadd fast <4 x float> [[WIDE_LOAD]], [[WIDE_LOAD1]] ; CHECK-NEXT: [[WIDE_LOAD1:%.*]] = load <4 x float>, ptr [[TMP5]], align 4, !llvm.access.group [[ACC_GRP0]]
; CHECK-NEXT: store <4 x float> [[TMP5]], ptr [[TMP4]], align 4, !llvm.access.group [[ACC_GRP0]] ; CHECK-NEXT: [[TMP6:%.*]] = getelementptr inbounds float, ptr [[A:%.*]], i64 [[TMP0]]
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 4 ; CHECK-NEXT: [[TMP7:%.*]] = getelementptr inbounds float, ptr [[A]], i64 [[TMP1]]
; CHECK-NEXT: [[TMP6:%.*]] = icmp eq i64 [[INDEX_NEXT]], 20 ; CHECK-NEXT: [[TMP8:%.*]] = getelementptr inbounds float, ptr [[TMP6]], i32 0
; CHECK-NEXT: br i1 [[TMP6]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP1:![0-9]+]] ; CHECK-NEXT: [[WIDE_LOAD2:%.*]] = load <4 x float>, ptr [[TMP8]], align 4, !llvm.access.group [[ACC_GRP0]]
; CHECK-NEXT: [[TMP9:%.*]] = getelementptr inbounds float, ptr [[TMP6]], i32 4
; CHECK-NEXT: [[WIDE_LOAD3:%.*]] = load <4 x float>, ptr [[TMP9]], align 4, !llvm.access.group [[ACC_GRP0]]
; CHECK-NEXT: [[TMP10:%.*]] = fadd fast <4 x float> [[WIDE_LOAD]], [[WIDE_LOAD2]]
; CHECK-NEXT: [[TMP11:%.*]] = fadd fast <4 x float> [[WIDE_LOAD1]], [[WIDE_LOAD3]]
; CHECK-NEXT: store <4 x float> [[TMP10]], ptr [[TMP8]], align 4, !llvm.access.group [[ACC_GRP0]]
; CHECK-NEXT: store <4 x float> [[TMP11]], ptr [[TMP9]], align 4, !llvm.access.group [[ACC_GRP0]]
; CHECK-NEXT: [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 8
; CHECK-NEXT: [[TMP12:%.*]] = icmp eq i64 [[INDEX_NEXT]], 16
; CHECK-NEXT: br i1 [[TMP12]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP1:![0-9]+]]
; CHECK: middle.block: ; CHECK: middle.block:
; CHECK-NEXT: br i1 true, label [[FOR_END:%.*]], label [[SCALAR_PH]] ; CHECK-NEXT: br i1 false, label [[FOR_END:%.*]], label [[SCALAR_PH]]
; CHECK: scalar.ph: ; CHECK: scalar.ph:
; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 20, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ] ; CHECK-NEXT: [[BC_RESUME_VAL:%.*]] = phi i64 [ 16, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]
; CHECK-NEXT: br label [[FOR_BODY:%.*]] ; CHECK-NEXT: br label [[FOR_BODY:%.*]]
; CHECK: for.body: ; CHECK: for.body:
; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY]] ] ; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY]] ]
; CHECK-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds float, ptr [[B]], i64 [[INDVARS_IV]] ; CHECK-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds float, ptr [[B]], i64 [[INDVARS_IV]]
; CHECK-NEXT: [[TMP7:%.*]] = load float, ptr [[ARRAYIDX]], align 4, !llvm.access.group [[ACC_GRP0]] ; CHECK-NEXT: [[TMP13:%.*]] = load float, ptr [[ARRAYIDX]], align 4, !llvm.access.group [[ACC_GRP0]]
; CHECK-NEXT: [[ARRAYIDX2:%.*]] = getelementptr inbounds float, ptr [[A]], i64 [[INDVARS_IV]] ; CHECK-NEXT: [[ARRAYIDX2:%.*]] = getelementptr inbounds float, ptr [[A]], i64 [[INDVARS_IV]]
; CHECK-NEXT: [[TMP8:%.*]] = load float, ptr [[ARRAYIDX2]], align 4, !llvm.access.group [[ACC_GRP0]] ; CHECK-NEXT: [[TMP14:%.*]] = load float, ptr [[ARRAYIDX2]], align 4, !llvm.access.group [[ACC_GRP0]]
; CHECK-NEXT: [[ADD:%.*]] = fadd fast float [[TMP7]], [[TMP8]] ; CHECK-NEXT: [[ADD:%.*]] = fadd fast float [[TMP13]], [[TMP14]]
; CHECK-NEXT: store float [[ADD]], ptr [[ARRAYIDX2]], align 4, !llvm.access.group [[ACC_GRP0]] ; CHECK-NEXT: store float [[ADD]], ptr [[ARRAYIDX2]], align 4, !llvm.access.group [[ACC_GRP0]]
; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1
; CHECK-NEXT: [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], 20 ; CHECK-NEXT: [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], 20
; CHECK-NEXT: br i1 [[EXITCOND]], label [[FOR_END]], label [[FOR_BODY]], !llvm.loop [[LOOP5:![0-9]+]] ; CHECK-NEXT: br i1 [[EXITCOND]], label [[FOR_END]], label [[FOR_BODY]], !llvm.loop [[LOOP5:![0-9]+]]
; CHECK: for.end: ; CHECK: for.end:
; CHECK-NEXT: ret void ; CHECK-NEXT: ret void
; ;
entry: entry:
▲ Show 20 Lines • Show All 155 Lines • Show Last 20 Lines

llvm/test/Transforms/LoopVectorize/X86/vectorization-remarks-loopid-dbg.ll

; RUN: opt < %s -passes=loop-vectorize -mtriple=x86_64-unknown-linux -S -pass-remarks='loop-vectorize' 2>&1 | FileCheck -check-prefix=VECTORIZED %s ; RUN: opt < %s -passes=loop-vectorize -mtriple=x86_64-unknown-linux -S -pass-remarks='loop-vectorize' 2>&1 | FileCheck -check-prefix=VECTORIZED %s
; RUN: opt < %s -passes=loop-vectorize -force-vector-width=1 -force-vector-interleave=4 -mtriple=x86_64-unknown-linux -S -pass-remarks='loop-vectorize' 2>&1 | FileCheck -check-prefix=UNROLLED %s ; RUN: opt < %s -passes=loop-vectorize -force-vector-width=1 -force-vector-interleave=4 -mtriple=x86_64-unknown-linux -S -pass-remarks='loop-vectorize' 2>&1 | FileCheck -check-prefix=UNROLLED %s
; RUN: opt < %s -passes=loop-vectorize -force-vector-width=1 -force-vector-interleave=1 -mtriple=x86_64-unknown-linux -S -pass-remarks-analysis='loop-vectorize' 2>&1 | FileCheck -check-prefix=NONE %s ; RUN: opt < %s -passes=loop-vectorize -force-vector-width=1 -force-vector-interleave=1 -mtriple=x86_64-unknown-linux -S -pass-remarks-analysis='loop-vectorize' 2>&1 | FileCheck -check-prefix=NONE %s
; RUN: llc < %s -mtriple x86_64-pc-linux-gnu -o - | FileCheck -check-prefix=DEBUG-OUTPUT %s ; RUN: llc < %s -mtriple x86_64-pc-linux-gnu -o - | FileCheck -check-prefix=DEBUG-OUTPUT %s
; DEBUG-OUTPUT-NOT: .loc ; DEBUG-OUTPUT-NOT: .loc
; DEBUG-OUTPUT-NOT: {{.*}}.debug_info ; DEBUG-OUTPUT-NOT: {{.*}}.debug_info
; VECTORIZED: remark: vectorization-remarks.c:17:8: vectorized loop (vectorization width: 4, interleaved count: 1) ; VECTORIZED: remark: vectorization-remarks.c:17:8: vectorized loop (vectorization width: 4, interleaved count: 2)
; UNROLLED: remark: vectorization-remarks.c:17:8: interleaved loop (interleaved count: 4) ; UNROLLED: remark: vectorization-remarks.c:17:8: interleaved loop (interleaved count: 4)
; NONE: remark: vectorization-remarks.c:17:8: loop not vectorized: vectorization and interleaving are explicitly disabled, or the loop has already been vectorized ; NONE: remark: vectorization-remarks.c:17:8: loop not vectorized: vectorization and interleaving are explicitly disabled, or the loop has already been vectorized
target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128" target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
define i32 @foo(i32 %n) #0 !dbg !4 { define i32 @foo(i32 %n) #0 !dbg !4 {
entry: entry:
%diff = alloca i32, align 4 %diff = alloca i32, align 4
▲ Show 20 Lines • Show All 57 Lines • Show Last 20 Lines

llvm/test/Transforms/LoopVectorize/X86/vectorization-remarks.ll

; RUN: opt < %s -passes=loop-vectorize -mtriple=x86_64-unknown-linux -S -pass-remarks='loop-vectorize' 2>&1 | FileCheck -check-prefix=VECTORIZED %s ; RUN: opt < %s -passes=loop-vectorize -mtriple=x86_64-unknown-linux -S -pass-remarks='loop-vectorize' 2>&1 | FileCheck -check-prefix=VECTORIZED %s
; RUN: opt < %s -passes=loop-vectorize -force-vector-width=1 -force-vector-interleave=4 -mtriple=x86_64-unknown-linux -S -pass-remarks='loop-vectorize' 2>&1 | FileCheck -check-prefix=UNROLLED %s ; RUN: opt < %s -passes=loop-vectorize -force-vector-width=1 -force-vector-interleave=4 -mtriple=x86_64-unknown-linux -S -pass-remarks='loop-vectorize' 2>&1 | FileCheck -check-prefix=UNROLLED %s
; RUN: opt < %s -passes=loop-vectorize -force-vector-width=1 -force-vector-interleave=1 -mtriple=x86_64-unknown-linux -S -pass-remarks-analysis='loop-vectorize' 2>&1 | FileCheck -check-prefix=NONE %s ; RUN: opt < %s -passes=loop-vectorize -force-vector-width=1 -force-vector-interleave=1 -mtriple=x86_64-unknown-linux -S -pass-remarks-analysis='loop-vectorize' 2>&1 | FileCheck -check-prefix=NONE %s
; RUN: llc < %s -mtriple x86_64-pc-linux-gnu -o - | FileCheck -check-prefix=DEBUG-OUTPUT %s ; RUN: llc < %s -mtriple x86_64-pc-linux-gnu -o - | FileCheck -check-prefix=DEBUG-OUTPUT %s
; DEBUG-OUTPUT-NOT: .loc ; DEBUG-OUTPUT-NOT: .loc
; DEBUG-OUTPUT-NOT: {{.*}}.debug_info ; DEBUG-OUTPUT-NOT: {{.*}}.debug_info
; VECTORIZED: remark: vectorization-remarks.c:17:8: vectorized loop (vectorization width: 4, interleaved count: 1) ; VECTORIZED: remark: vectorization-remarks.c:17:8: vectorized loop (vectorization width: 4, interleaved count: 2)
; UNROLLED: remark: vectorization-remarks.c:17:8: interleaved loop (interleaved count: 4) ; UNROLLED: remark: vectorization-remarks.c:17:8: interleaved loop (interleaved count: 4)
; NONE: remark: vectorization-remarks.c:17:8: loop not vectorized: vectorization and interleaving are explicitly disabled, or the loop has already been vectorized ; NONE: remark: vectorization-remarks.c:17:8: loop not vectorized: vectorization and interleaving are explicitly disabled, or the loop has already been vectorized
target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128" target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
define i32 @foo(i32 %n) #0 !dbg !4 { define i32 @foo(i32 %n) #0 !dbg !4 {
entry: entry:
%diff = alloca i32, align 4 %diff = alloca i32, align 4
▲ Show 20 Lines • Show All 56 Lines • Show Last 20 Lines

llvm/test/Transforms/PhaseOrdering/X86/excessive-unrolling.ll

; NOTE: Assertions have been autogenerated by utils/update_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
; RUN: opt -passes='default<O3>' -unroll-runtime -S %s | FileCheck %s ; RUN: opt -passes='default<O3>' -unroll-runtime -S %s | FileCheck %s
target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128" target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-apple-macosx" target triple = "x86_64-apple-macosx"
@b = global [58 x double] zeroinitializer, align 16 @b = global [58 x double] zeroinitializer, align 16
@c = global [58 x double] zeroinitializer, align 16 @c = global [58 x double] zeroinitializer, align 16
@a = global [58 x double] zeroinitializer, align 16 @a = global [58 x double] zeroinitializer, align 16
; Test case for #42332, showing excessive unrolling of vector loop. ; Test case for #42332, showing excessive unrolling of vector loop.
define void @test_known_trip_count() { define void @test_known_trip_count() {
; CHECK-LABEL: @test_known_trip_count( ; CHECK-LABEL: @test_known_trip_count(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[WIDE_LOAD:%.*]] = load <2 x double>, ptr @b, align 16 ; CHECK-NEXT: [[WIDE_LOAD:%.*]] = load <2 x double>, ptr @b, align 16
; CHECK-NEXT: [[WIDE_LOAD3:%.*]] = load <2 x double>, ptr @c, align 16 ; CHECK-NEXT: [[WIDE_LOAD3:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 2), align 16
; CHECK-NEXT: [[TMP0:%.*]] = fadd <2 x double> [[WIDE_LOAD]], [[WIDE_LOAD3]] ; CHECK-NEXT: [[WIDE_LOAD4:%.*]] = load <2 x double>, ptr @c, align 16
; CHECK-NEXT: [[WIDE_LOAD5:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 2), align 16
; CHECK-NEXT: [[TMP0:%.*]] = fadd <2 x double> [[WIDE_LOAD]], [[WIDE_LOAD4]]
; CHECK-NEXT: [[TMP1:%.*]] = fadd <2 x double> [[WIDE_LOAD3]], [[WIDE_LOAD5]]
; CHECK-NEXT: store <2 x double> [[TMP0]], ptr @a, align 16 ; CHECK-NEXT: store <2 x double> [[TMP0]], ptr @a, align 16
; CHECK-NEXT: [[WIDE_LOAD_1:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 2), align 16
; CHECK-NEXT: [[WIDE_LOAD3_1:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 2), align 16
; CHECK-NEXT: [[TMP1:%.*]] = fadd <2 x double> [[WIDE_LOAD_1]], [[WIDE_LOAD3_1]]
; CHECK-NEXT: store <2 x double> [[TMP1]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 2), align 16 ; CHECK-NEXT: store <2 x double> [[TMP1]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 2), align 16
; CHECK-NEXT: [[WIDE_LOAD_2:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 4), align 16 ; CHECK-NEXT: [[WIDE_LOAD_1:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 4), align 16
; CHECK-NEXT: [[WIDE_LOAD3_2:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 4), align 16 ; CHECK-NEXT: [[WIDE_LOAD3_1:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 6), align 16
; CHECK-NEXT: [[TMP2:%.*]] = fadd <2 x double> [[WIDE_LOAD_2]], [[WIDE_LOAD3_2]] ; CHECK-NEXT: [[WIDE_LOAD4_1:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 4), align 16
; CHECK-NEXT: [[WIDE_LOAD5_1:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 6), align 16
; CHECK-NEXT: [[TMP2:%.*]] = fadd <2 x double> [[WIDE_LOAD_1]], [[WIDE_LOAD4_1]]
; CHECK-NEXT: [[TMP3:%.*]] = fadd <2 x double> [[WIDE_LOAD3_1]], [[WIDE_LOAD5_1]]
; CHECK-NEXT: store <2 x double> [[TMP2]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 4), align 16 ; CHECK-NEXT: store <2 x double> [[TMP2]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 4), align 16
; CHECK-NEXT: [[WIDE_LOAD_3:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 6), align 16
; CHECK-NEXT: [[WIDE_LOAD3_3:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 6), align 16
; CHECK-NEXT: [[TMP3:%.*]] = fadd <2 x double> [[WIDE_LOAD_3]], [[WIDE_LOAD3_3]]
; CHECK-NEXT: store <2 x double> [[TMP3]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 6), align 16 ; CHECK-NEXT: store <2 x double> [[TMP3]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 6), align 16
; CHECK-NEXT: [[WIDE_LOAD_4:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 8), align 16 ; CHECK-NEXT: [[WIDE_LOAD_2:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 8), align 16
; CHECK-NEXT: [[WIDE_LOAD3_4:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 8), align 16 ; CHECK-NEXT: [[WIDE_LOAD3_2:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 10), align 16
; CHECK-NEXT: [[TMP4:%.*]] = fadd <2 x double> [[WIDE_LOAD_4]], [[WIDE_LOAD3_4]] ; CHECK-NEXT: [[WIDE_LOAD4_2:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 8), align 16
; CHECK-NEXT: [[WIDE_LOAD5_2:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 10), align 16
; CHECK-NEXT: [[TMP4:%.*]] = fadd <2 x double> [[WIDE_LOAD_2]], [[WIDE_LOAD4_2]]
; CHECK-NEXT: [[TMP5:%.*]] = fadd <2 x double> [[WIDE_LOAD3_2]], [[WIDE_LOAD5_2]]
; CHECK-NEXT: store <2 x double> [[TMP4]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 8), align 16 ; CHECK-NEXT: store <2 x double> [[TMP4]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 8), align 16
; CHECK-NEXT: [[WIDE_LOAD_5:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 10), align 16
; CHECK-NEXT: [[WIDE_LOAD3_5:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 10), align 16
; CHECK-NEXT: [[TMP5:%.*]] = fadd <2 x double> [[WIDE_LOAD_5]], [[WIDE_LOAD3_5]]
; CHECK-NEXT: store <2 x double> [[TMP5]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 10), align 16 ; CHECK-NEXT: store <2 x double> [[TMP5]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 10), align 16
; CHECK-NEXT: [[WIDE_LOAD_6:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 12), align 16 ; CHECK-NEXT: [[WIDE_LOAD_3:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 12), align 16
; CHECK-NEXT: [[WIDE_LOAD3_6:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 12), align 16 ; CHECK-NEXT: [[WIDE_LOAD3_3:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 14), align 16
; CHECK-NEXT: [[TMP6:%.*]] = fadd <2 x double> [[WIDE_LOAD_6]], [[WIDE_LOAD3_6]] ; CHECK-NEXT: [[WIDE_LOAD4_3:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 12), align 16
; CHECK-NEXT: [[WIDE_LOAD5_3:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 14), align 16
; CHECK-NEXT: [[TMP6:%.*]] = fadd <2 x double> [[WIDE_LOAD_3]], [[WIDE_LOAD4_3]]
; CHECK-NEXT: [[TMP7:%.*]] = fadd <2 x double> [[WIDE_LOAD3_3]], [[WIDE_LOAD5_3]]
; CHECK-NEXT: store <2 x double> [[TMP6]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 12), align 16 ; CHECK-NEXT: store <2 x double> [[TMP6]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 12), align 16
; CHECK-NEXT: [[WIDE_LOAD_7:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 14), align 16
; CHECK-NEXT: [[WIDE_LOAD3_7:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 14), align 16
; CHECK-NEXT: [[TMP7:%.*]] = fadd <2 x double> [[WIDE_LOAD_7]], [[WIDE_LOAD3_7]]
; CHECK-NEXT: store <2 x double> [[TMP7]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 14), align 16 ; CHECK-NEXT: store <2 x double> [[TMP7]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 14), align 16
; CHECK-NEXT: [[WIDE_LOAD_8:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 16), align 16 ; CHECK-NEXT: [[WIDE_LOAD_4:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 16), align 16
; CHECK-NEXT: [[WIDE_LOAD3_8:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 16), align 16 ; CHECK-NEXT: [[WIDE_LOAD3_4:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 18), align 16
; CHECK-NEXT: [[TMP8:%.*]] = fadd <2 x double> [[WIDE_LOAD_8]], [[WIDE_LOAD3_8]] ; CHECK-NEXT: [[WIDE_LOAD4_4:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 16), align 16
; CHECK-NEXT: [[WIDE_LOAD5_4:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 18), align 16
; CHECK-NEXT: [[TMP8:%.*]] = fadd <2 x double> [[WIDE_LOAD_4]], [[WIDE_LOAD4_4]]
; CHECK-NEXT: [[TMP9:%.*]] = fadd <2 x double> [[WIDE_LOAD3_4]], [[WIDE_LOAD5_4]]
; CHECK-NEXT: store <2 x double> [[TMP8]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 16), align 16 ; CHECK-NEXT: store <2 x double> [[TMP8]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 16), align 16
; CHECK-NEXT: [[WIDE_LOAD_9:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 18), align 16
; CHECK-NEXT: [[WIDE_LOAD3_9:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 18), align 16
; CHECK-NEXT: [[TMP9:%.*]] = fadd <2 x double> [[WIDE_LOAD_9]], [[WIDE_LOAD3_9]]
; CHECK-NEXT: store <2 x double> [[TMP9]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 18), align 16 ; CHECK-NEXT: store <2 x double> [[TMP9]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 18), align 16
; CHECK-NEXT: [[WIDE_LOAD_10:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 20), align 16 ; CHECK-NEXT: [[WIDE_LOAD_5:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 20), align 16
; CHECK-NEXT: [[WIDE_LOAD3_10:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 20), align 16 ; CHECK-NEXT: [[WIDE_LOAD3_5:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 22), align 16
; CHECK-NEXT: [[TMP10:%.*]] = fadd <2 x double> [[WIDE_LOAD_10]], [[WIDE_LOAD3_10]] ; CHECK-NEXT: [[WIDE_LOAD4_5:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 20), align 16
; CHECK-NEXT: [[WIDE_LOAD5_5:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 22), align 16
; CHECK-NEXT: [[TMP10:%.*]] = fadd <2 x double> [[WIDE_LOAD_5]], [[WIDE_LOAD4_5]]
; CHECK-NEXT: [[TMP11:%.*]] = fadd <2 x double> [[WIDE_LOAD3_5]], [[WIDE_LOAD5_5]]
; CHECK-NEXT: store <2 x double> [[TMP10]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 20), align 16 ; CHECK-NEXT: store <2 x double> [[TMP10]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 20), align 16
; CHECK-NEXT: [[WIDE_LOAD_11:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 22), align 16
; CHECK-NEXT: [[WIDE_LOAD3_11:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 22), align 16
; CHECK-NEXT: [[TMP11:%.*]] = fadd <2 x double> [[WIDE_LOAD_11]], [[WIDE_LOAD3_11]]
; CHECK-NEXT: store <2 x double> [[TMP11]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 22), align 16 ; CHECK-NEXT: store <2 x double> [[TMP11]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 22), align 16
; CHECK-NEXT: [[WIDE_LOAD_12:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 24), align 16 ; CHECK-NEXT: [[WIDE_LOAD_6:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 24), align 16
; CHECK-NEXT: [[WIDE_LOAD3_12:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 24), align 16 ; CHECK-NEXT: [[WIDE_LOAD3_6:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 26), align 16
; CHECK-NEXT: [[TMP12:%.*]] = fadd <2 x double> [[WIDE_LOAD_12]], [[WIDE_LOAD3_12]] ; CHECK-NEXT: [[WIDE_LOAD4_6:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 24), align 16
; CHECK-NEXT: [[WIDE_LOAD5_6:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 26), align 16
; CHECK-NEXT: [[TMP12:%.*]] = fadd <2 x double> [[WIDE_LOAD_6]], [[WIDE_LOAD4_6]]
; CHECK-NEXT: [[TMP13:%.*]] = fadd <2 x double> [[WIDE_LOAD3_6]], [[WIDE_LOAD5_6]]
; CHECK-NEXT: store <2 x double> [[TMP12]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 24), align 16 ; CHECK-NEXT: store <2 x double> [[TMP12]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 24), align 16
; CHECK-NEXT: [[WIDE_LOAD_13:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 26), align 16
; CHECK-NEXT: [[WIDE_LOAD3_13:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 26), align 16
; CHECK-NEXT: [[TMP13:%.*]] = fadd <2 x double> [[WIDE_LOAD_13]], [[WIDE_LOAD3_13]]
; CHECK-NEXT: store <2 x double> [[TMP13]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 26), align 16 ; CHECK-NEXT: store <2 x double> [[TMP13]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 26), align 16
; CHECK-NEXT: [[WIDE_LOAD_14:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 28), align 16 ; CHECK-NEXT: [[WIDE_LOAD_7:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 28), align 16
; CHECK-NEXT: [[WIDE_LOAD3_14:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 28), align 16 ; CHECK-NEXT: [[WIDE_LOAD3_7:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 30), align 16
; CHECK-NEXT: [[TMP14:%.*]] = fadd <2 x double> [[WIDE_LOAD_14]], [[WIDE_LOAD3_14]] ; CHECK-NEXT: [[WIDE_LOAD4_7:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 28), align 16
; CHECK-NEXT: [[WIDE_LOAD5_7:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 30), align 16
; CHECK-NEXT: [[TMP14:%.*]] = fadd <2 x double> [[WIDE_LOAD_7]], [[WIDE_LOAD4_7]]
; CHECK-NEXT: [[TMP15:%.*]] = fadd <2 x double> [[WIDE_LOAD3_7]], [[WIDE_LOAD5_7]]
; CHECK-NEXT: store <2 x double> [[TMP14]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 28), align 16 ; CHECK-NEXT: store <2 x double> [[TMP14]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 28), align 16
; CHECK-NEXT: [[WIDE_LOAD_15:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 30), align 16
; CHECK-NEXT: [[WIDE_LOAD3_15:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 30), align 16
; CHECK-NEXT: [[TMP15:%.*]] = fadd <2 x double> [[WIDE_LOAD_15]], [[WIDE_LOAD3_15]]
; CHECK-NEXT: store <2 x double> [[TMP15]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 30), align 16 ; CHECK-NEXT: store <2 x double> [[TMP15]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 30), align 16
; CHECK-NEXT: [[WIDE_LOAD_16:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 32), align 16 ; CHECK-NEXT: [[WIDE_LOAD_8:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 32), align 16
; CHECK-NEXT: [[WIDE_LOAD3_16:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 32), align 16 ; CHECK-NEXT: [[WIDE_LOAD3_8:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 34), align 16
; CHECK-NEXT: [[TMP16:%.*]] = fadd <2 x double> [[WIDE_LOAD_16]], [[WIDE_LOAD3_16]] ; CHECK-NEXT: [[WIDE_LOAD4_8:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 32), align 16
; CHECK-NEXT: [[WIDE_LOAD5_8:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 34), align 16
; CHECK-NEXT: [[TMP16:%.*]] = fadd <2 x double> [[WIDE_LOAD_8]], [[WIDE_LOAD4_8]]
; CHECK-NEXT: [[TMP17:%.*]] = fadd <2 x double> [[WIDE_LOAD3_8]], [[WIDE_LOAD5_8]]
; CHECK-NEXT: store <2 x double> [[TMP16]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 32), align 16 ; CHECK-NEXT: store <2 x double> [[TMP16]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 32), align 16
; CHECK-NEXT: [[WIDE_LOAD_17:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 34), align 16
; CHECK-NEXT: [[WIDE_LOAD3_17:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 34), align 16
; CHECK-NEXT: [[TMP17:%.*]] = fadd <2 x double> [[WIDE_LOAD_17]], [[WIDE_LOAD3_17]]
; CHECK-NEXT: store <2 x double> [[TMP17]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 34), align 16 ; CHECK-NEXT: store <2 x double> [[TMP17]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 34), align 16
; CHECK-NEXT: [[WIDE_LOAD_18:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 36), align 16 ; CHECK-NEXT: [[WIDE_LOAD_9:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 36), align 16
; CHECK-NEXT: [[WIDE_LOAD3_18:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 36), align 16 ; CHECK-NEXT: [[WIDE_LOAD3_9:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 38), align 16
; CHECK-NEXT: [[TMP18:%.*]] = fadd <2 x double> [[WIDE_LOAD_18]], [[WIDE_LOAD3_18]] ; CHECK-NEXT: [[WIDE_LOAD4_9:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 36), align 16
; CHECK-NEXT: [[WIDE_LOAD5_9:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 38), align 16
; CHECK-NEXT: [[TMP18:%.*]] = fadd <2 x double> [[WIDE_LOAD_9]], [[WIDE_LOAD4_9]]
; CHECK-NEXT: [[TMP19:%.*]] = fadd <2 x double> [[WIDE_LOAD3_9]], [[WIDE_LOAD5_9]]
; CHECK-NEXT: store <2 x double> [[TMP18]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 36), align 16 ; CHECK-NEXT: store <2 x double> [[TMP18]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 36), align 16
; CHECK-NEXT: [[WIDE_LOAD_19:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 38), align 16
; CHECK-NEXT: [[WIDE_LOAD3_19:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 38), align 16
; CHECK-NEXT: [[TMP19:%.*]] = fadd <2 x double> [[WIDE_LOAD_19]], [[WIDE_LOAD3_19]]
; CHECK-NEXT: store <2 x double> [[TMP19]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 38), align 16 ; CHECK-NEXT: store <2 x double> [[TMP19]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 38), align 16
; CHECK-NEXT: [[WIDE_LOAD_20:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 40), align 16 ; CHECK-NEXT: [[WIDE_LOAD_10:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 40), align 16
; CHECK-NEXT: [[WIDE_LOAD3_20:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 40), align 16 ; CHECK-NEXT: [[WIDE_LOAD3_10:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 42), align 16
; CHECK-NEXT: [[TMP20:%.*]] = fadd <2 x double> [[WIDE_LOAD_20]], [[WIDE_LOAD3_20]] ; CHECK-NEXT: [[WIDE_LOAD4_10:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 40), align 16
; CHECK-NEXT: [[WIDE_LOAD5_10:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 42), align 16
; CHECK-NEXT: [[TMP20:%.*]] = fadd <2 x double> [[WIDE_LOAD_10]], [[WIDE_LOAD4_10]]
; CHECK-NEXT: [[TMP21:%.*]] = fadd <2 x double> [[WIDE_LOAD3_10]], [[WIDE_LOAD5_10]]
; CHECK-NEXT: store <2 x double> [[TMP20]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 40), align 16 ; CHECK-NEXT: store <2 x double> [[TMP20]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 40), align 16
; CHECK-NEXT: [[WIDE_LOAD_21:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 42), align 16
; CHECK-NEXT: [[WIDE_LOAD3_21:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 42), align 16
; CHECK-NEXT: [[TMP21:%.*]] = fadd <2 x double> [[WIDE_LOAD_21]], [[WIDE_LOAD3_21]]
; CHECK-NEXT: store <2 x double> [[TMP21]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 42), align 16 ; CHECK-NEXT: store <2 x double> [[TMP21]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 42), align 16
; CHECK-NEXT: [[WIDE_LOAD_22:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 44), align 16 ; CHECK-NEXT: [[WIDE_LOAD_11:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 44), align 16
; CHECK-NEXT: [[WIDE_LOAD3_22:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 44), align 16 ; CHECK-NEXT: [[WIDE_LOAD3_11:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 46), align 16
; CHECK-NEXT: [[TMP22:%.*]] = fadd <2 x double> [[WIDE_LOAD_22]], [[WIDE_LOAD3_22]] ; CHECK-NEXT: [[WIDE_LOAD4_11:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 44), align 16
; CHECK-NEXT: [[WIDE_LOAD5_11:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 46), align 16
; CHECK-NEXT: [[TMP22:%.*]] = fadd <2 x double> [[WIDE_LOAD_11]], [[WIDE_LOAD4_11]]
; CHECK-NEXT: [[TMP23:%.*]] = fadd <2 x double> [[WIDE_LOAD3_11]], [[WIDE_LOAD5_11]]
; CHECK-NEXT: store <2 x double> [[TMP22]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 44), align 16 ; CHECK-NEXT: store <2 x double> [[TMP22]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 44), align 16
; CHECK-NEXT: [[WIDE_LOAD_23:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 46), align 16
; CHECK-NEXT: [[WIDE_LOAD3_23:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 46), align 16
; CHECK-NEXT: [[TMP23:%.*]] = fadd <2 x double> [[WIDE_LOAD_23]], [[WIDE_LOAD3_23]]
; CHECK-NEXT: store <2 x double> [[TMP23]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 46), align 16 ; CHECK-NEXT: store <2 x double> [[TMP23]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 46), align 16
; CHECK-NEXT: [[WIDE_LOAD_24:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 48), align 16 ; CHECK-NEXT: [[WIDE_LOAD_12:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 48), align 16
; CHECK-NEXT: [[WIDE_LOAD3_24:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 48), align 16 ; CHECK-NEXT: [[WIDE_LOAD3_12:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 50), align 16
; CHECK-NEXT: [[TMP24:%.*]] = fadd <2 x double> [[WIDE_LOAD_24]], [[WIDE_LOAD3_24]] ; CHECK-NEXT: [[WIDE_LOAD4_12:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 48), align 16
; CHECK-NEXT: [[WIDE_LOAD5_12:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 50), align 16
; CHECK-NEXT: [[TMP24:%.*]] = fadd <2 x double> [[WIDE_LOAD_12]], [[WIDE_LOAD4_12]]
; CHECK-NEXT: [[TMP25:%.*]] = fadd <2 x double> [[WIDE_LOAD3_12]], [[WIDE_LOAD5_12]]
; CHECK-NEXT: store <2 x double> [[TMP24]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 48), align 16 ; CHECK-NEXT: store <2 x double> [[TMP24]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 48), align 16
; CHECK-NEXT: [[WIDE_LOAD_25:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 50), align 16
; CHECK-NEXT: [[WIDE_LOAD3_25:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 50), align 16
; CHECK-NEXT: [[TMP25:%.*]] = fadd <2 x double> [[WIDE_LOAD_25]], [[WIDE_LOAD3_25]]
; CHECK-NEXT: store <2 x double> [[TMP25]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 50), align 16 ; CHECK-NEXT: store <2 x double> [[TMP25]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 50), align 16
; CHECK-NEXT: [[WIDE_LOAD_26:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 52), align 16 ; CHECK-NEXT: [[WIDE_LOAD_13:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 52), align 16
; CHECK-NEXT: [[WIDE_LOAD3_26:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 52), align 16 ; CHECK-NEXT: [[WIDE_LOAD3_13:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 54), align 16
; CHECK-NEXT: [[TMP26:%.*]] = fadd <2 x double> [[WIDE_LOAD_26]], [[WIDE_LOAD3_26]] ; CHECK-NEXT: [[WIDE_LOAD4_13:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 52), align 16
; CHECK-NEXT: [[WIDE_LOAD5_13:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 54), align 16
; CHECK-NEXT: [[TMP26:%.*]] = fadd <2 x double> [[WIDE_LOAD_13]], [[WIDE_LOAD4_13]]
; CHECK-NEXT: [[TMP27:%.*]] = fadd <2 x double> [[WIDE_LOAD3_13]], [[WIDE_LOAD5_13]]
; CHECK-NEXT: store <2 x double> [[TMP26]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 52), align 16 ; CHECK-NEXT: store <2 x double> [[TMP26]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 52), align 16
; CHECK-NEXT: [[WIDE_LOAD_27:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 54), align 16
; CHECK-NEXT: [[WIDE_LOAD3_27:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 54), align 16
; CHECK-NEXT: [[TMP27:%.*]] = fadd <2 x double> [[WIDE_LOAD_27]], [[WIDE_LOAD3_27]]
; CHECK-NEXT: store <2 x double> [[TMP27]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 54), align 16 ; CHECK-NEXT: store <2 x double> [[TMP27]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 54), align 16
; CHECK-NEXT: [[WIDE_LOAD_28:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 56), align 16 ; CHECK-NEXT: [[WIDE_LOAD_14:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 0, i64 56), align 16
; CHECK-NEXT: [[WIDE_LOAD3_28:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 56), align 16 ; CHECK-NEXT: [[WIDE_LOAD3_14:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 1, i64 0), align 16
; CHECK-NEXT: [[TMP28:%.*]] = fadd <2 x double> [[WIDE_LOAD_28]], [[WIDE_LOAD3_28]] ; CHECK-NEXT: [[WIDE_LOAD4_14:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 0, i64 56), align 16
; CHECK-NEXT: [[WIDE_LOAD5_14:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 1, i64 0), align 16
; CHECK-NEXT: [[TMP28:%.*]] = fadd <2 x double> [[WIDE_LOAD_14]], [[WIDE_LOAD4_14]]
; CHECK-NEXT: [[TMP29:%.*]] = fadd <2 x double> [[WIDE_LOAD3_14]], [[WIDE_LOAD5_14]]
; CHECK-NEXT: store <2 x double> [[TMP28]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 56), align 16 ; CHECK-NEXT: store <2 x double> [[TMP28]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 0, i64 56), align 16
; CHECK-NEXT: [[WIDE_LOAD_29:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @b, i64 1, i64 0), align 16
; CHECK-NEXT: [[WIDE_LOAD3_29:%.*]] = load <2 x double>, ptr getelementptr inbounds ([58 x double], ptr @c, i64 1, i64 0), align 16
; CHECK-NEXT: [[TMP29:%.*]] = fadd <2 x double> [[WIDE_LOAD_29]], [[WIDE_LOAD3_29]]
; CHECK-NEXT: store <2 x double> [[TMP29]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 1, i64 0), align 16 ; CHECK-NEXT: store <2 x double> [[TMP29]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 1, i64 0), align 16
; CHECK-NEXT: [[TMP30:%.*]] = load double, ptr getelementptr inbounds ([58 x double], ptr @b, i64 1, i64 2), align 16 ; CHECK-NEXT: [[TMP30:%.*]] = load double, ptr getelementptr inbounds ([58 x double], ptr @b, i64 1, i64 2), align 16
; CHECK-NEXT: [[TMP31:%.*]] = load double, ptr getelementptr inbounds ([58 x double], ptr @c, i64 1, i64 2), align 16 ; CHECK-NEXT: [[TMP31:%.*]] = load double, ptr getelementptr inbounds ([58 x double], ptr @c, i64 1, i64 2), align 16
; CHECK-NEXT: [[ADD:%.*]] = fadd double [[TMP30]], [[TMP31]] ; CHECK-NEXT: [[ADD:%.*]] = fadd double [[TMP30]], [[TMP31]]
; CHECK-NEXT: store double [[ADD]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 1, i64 2), align 16 ; CHECK-NEXT: store double [[ADD]], ptr getelementptr inbounds ([58 x double], ptr @a, i64 1, i64 2), align 16
; CHECK-NEXT: ret void ; CHECK-NEXT: ret void
; ;
entry: entry:
▲ Show 20 Lines • Show All 103 Lines • Show Last 20 Lines