This is an archive of the discontinued LLVM Phabricator instance.

Differential D99192

[NFC] Add tests for scalable vectorization of loops with large stride acesses
ClosedPublic

Authored by david-arm on Mar 23 2021, 8:26 AM.

Details

Reviewers
sdesmalen
peterwaller-arm
kmclaughlin
fhahn
CarolineConcatto
Commits
rGe3a13304fc03: [NFC] Add tests for scalable vectorization of loops with large stride acesses
Summary

This patch just adds tests that we can vectorize loop such as these:

for (i = 0; i < n; i++)
  dst[i * 7] += 1;

and

for (i = 0; i < n; i++)
  if (cond[i])
    dst[i * 7] += 1;

using scalable vectors, where we expect to use gathers and scatters in the
vectorized loop. The vector of pointers used for the gather is identical
to those used for the scatter so there should be no memory dependences.

Tests are added here:

Transforms/LoopVectorize/AArch64/sve-large-strides.ll

Diff Detail

Event Timeline

david-arm created this revision.Mar 23 2021, 8:26 AM
Herald added a subscriber: kristof.beyls. · View Herald TranscriptMar 23 2021, 8:26 AM
david-arm requested review of this revision.Mar 23 2021, 8:26 AM
Herald added a project: Restricted Project. · View Herald TranscriptMar 23 2021, 8:26 AM
Herald added a subscriber: llvm-commits. · View Herald Transcript
david-arm added a reviewer: fhahn.Mar 23 2021, 8:27 AM
david-arm added a parent revision: D98715: [LoopVectorize] Add support for scalable vectorization of induction variables.Mar 23 2021, 9:47 AM
Harbormaster completed remote builds in B95278: Diff 332684.Mar 23 2021, 12:00 PM
david-arm added a reviewer: CarolineConcatto.Mar 30 2021, 3:20 AM
peterwaller-arm accepted this revision.Mar 31 2021, 2:07 AM

LGTM.

llvm/test/Transforms/LoopVectorize/AArch64/sve-large-strides.ll
2

Optional: I'd understood it was preferred to put the triples and attrs into the IR to make invoking opt manually on the file a little easier.

This revision is now accepted and ready to land.Mar 31 2021, 2:07 AM
CarolineConcatto added a comment.Mar 31 2021, 11:55 AM

Hey David,
I believe I have some nit. :)

llvm/test/Transforms/LoopVectorize/AArch64/sve-large-strides.ll
6

Why do many spaces between CHECK and the instructions?

12

Is there any reason to be more than 1 space?

33

Could this be label too?

david-arm added inline comments.Apr 1 2021, 1:11 AM
llvm/test/Transforms/LoopVectorize/AArch64/sve-large-strides.ll
6

I'm just trying to align the starts of all the instructions, i.e. ensure that '%' lines up in the same column. I think it's a bit easier to read this way. This also matches the behaviour of the automated CHECK line writing tool - utils/update_test_checks.py. For example, I used tests like this as a reference:

test/Transforms/LoopVectorize/X86/invariant-store-vectorization.ll
12

Again, just trying to follow same convention as utils/update_test_checks.py that's all.

33

I think in such cases we normally seem to use CHECK:, at least for tests generated by utils/update_test_checks.py in llvm/test/Transforms/LoopVectorize/X86/

CarolineConcatto added inline comments.Apr 1 2021, 1:24 AM
llvm/test/Transforms/LoopVectorize/AArch64/sve-large-strides.ll
12

Oh...that is nice...so I will not have any more problems with formatting tests

CarolineConcatto accepted this revision.Apr 1 2021, 2:24 AM

LGMT David!
Learning new things every day!

This revision was landed with ongoing or failed builds.Apr 1 2021, 2:26 AM
Closed by commit rGe3a13304fc03: [NFC] Add tests for scalable vectorization of loops with large stride acesses (authored by david-arm). · Explain Why
This revision was automatically updated to reflect the committed changes.
david-arm added a commit: rGe3a13304fc03: [NFC] Add tests for scalable vectorization of loops with large stride acesses.

Revision Contents

PathSize
llvm/
test/
Transforms/
LoopVectorize/
AArch64/
101 lines

Diff 334625

llvm/test/Transforms/LoopVectorize/AArch64/sve-large-strides.ll

  • This file was added.
; RUN: opt -mtriple aarch64-linux-gnu -mattr=+sve -loop-vectorize -dce -instcombine -S <%s | FileCheck %s
peterwaller-armUnsubmitted
Not Done
ReplyInline Actions

Optional: I'd understood it was preferred to put the triples and attrs into the IR to make invoking opt manually on the file a little easier.

peterwaller-arm: Optional: I'd understood it was preferred to put the triples and attrs into the IR to make…
define void @stride7_i32(i32* noalias nocapture %dst, i64 %n) {
; CHECK-LABEL: @stride7_i32(
; CHECK: vector.body
; CHECK: %[[VEC_IND:.*]] = phi <vscale x 4 x i64> [ %{{.*}}, %vector.ph ], [ %{{.*}}, %vector.body ]
CarolineConcattoUnsubmitted
Not Done
ReplyInline Actions

Why do many spaces between CHECK and the instructions?

CarolineConcatto: Why do many spaces between CHECK and the instructions?
david-armAuthorUnsubmitted
Done
ReplyInline Actions

I'm just trying to align the starts of all the instructions, i.e. ensure that '%' lines up in the same column. I think it's a bit easier to read this way. This also matches the behaviour of the automated CHECK line writing tool - utils/update_test_checks.py. For example, I used tests like this as a reference:

test/Transforms/LoopVectorize/X86/invariant-store-vectorization.ll
david-arm: I'm just trying to align the starts of all the instructions, i.e. ensure that '%' lines up in…
; CHECK-NEXT: %[[PTR_INDICES:.*]] = mul nuw nsw <vscale x 4 x i64> %[[VEC_IND]], shufflevector (<vscale x 4 x i64> insertelement (<vscale x 4 x i64> poison, i64 7, i32 0), <vscale x 4 x i64> poison, <vscale x 4 x i32> zeroinitializer)
; CHECK-NEXT: %[[PTRS:.*]] = getelementptr inbounds i32, i32* %dst, <vscale x 4 x i64> %[[PTR_INDICES]]
; CHECK-NEXT: %[[GLOAD:.*]] = call <vscale x 4 x i32> @llvm.masked.gather.nxv4i32.nxv4p0i32(<vscale x 4 x i32*> %[[PTRS]]
; CHECK-NEXT: %[[VALS:.*]] = add nsw <vscale x 4 x i32> %[[GLOAD]],
; CHECK-NEXT: call void @llvm.masked.scatter.nxv4i32.nxv4p0i32(<vscale x 4 x i32> %[[VALS]], <vscale x 4 x i32*> %[[PTRS]]
entry:
CarolineConcattoUnsubmitted
Not Done
ReplyInline Actions

Is there any reason to be more than 1 space?

CarolineConcatto: Is there any reason to be more than 1 space?
david-armAuthorUnsubmitted
Done
ReplyInline Actions

Again, just trying to follow same convention as utils/update_test_checks.py that's all.

david-arm: Again, just trying to follow same convention as utils/update_test_checks.py that's all.
CarolineConcattoUnsubmitted
Not Done
ReplyInline Actions

Oh...that is nice...so I will not have any more problems with formatting tests

CarolineConcatto: Oh...that is nice...so I will not have any more problems with formatting tests
br label %for.body
for.body: ; preds = %entry, %for.body
%i.05 = phi i64 [ %inc, %for.body ], [ 0, %entry ]
%mul = mul nuw nsw i64 %i.05, 7
%arrayidx = getelementptr inbounds i32, i32* %dst, i64 %mul
%0 = load i32, i32* %arrayidx, align 4
%add = add nsw i32 %0, 3
store i32 %add, i32* %arrayidx, align 4
%inc = add nuw nsw i64 %i.05, 1
%exitcond.not = icmp eq i64 %inc, %n
br i1 %exitcond.not, label %for.end, label %for.body, !llvm.loop !0
for.end: ; preds = %for.end.loopexit, %entry
ret void
}
define void @stride7_f64(double* noalias nocapture %dst, i64 %n) {
; CHECK-LABEL: @stride7_f64(
; CHECK: vector.body
; CHECK: %[[VEC_IND:.*]] = phi <vscale x 2 x i64> [ %{{.*}}, %vector.ph ], [ %{{.*}}, %vector.body ]
CarolineConcattoUnsubmitted
Not Done
ReplyInline Actions

Could this be label too?

CarolineConcatto: Could this be label too?
david-armAuthorUnsubmitted
Done
ReplyInline Actions

I think in such cases we normally seem to use CHECK:, at least for tests generated by utils/update_test_checks.py in llvm/test/Transforms/LoopVectorize/X86/

david-arm: I think in such cases we normally seem to use `CHECK:`, at least for tests generated by…
; CHECK-NEXT: %[[PTR_INDICES:.*]] = mul nuw nsw <vscale x 2 x i64> %[[VEC_IND]], shufflevector (<vscale x 2 x i64> insertelement (<vscale x 2 x i64> poison, i64 7, i32 0), <vscale x 2 x i64> poison, <vscale x 2 x i32> zeroinitializer)
; CHECK-NEXT: %[[PTRS:.*]] = getelementptr inbounds double, double* %dst, <vscale x 2 x i64> %[[PTR_INDICES]]
; CHECK-NEXT: %[[GLOAD:.*]] = call <vscale x 2 x double> @llvm.masked.gather.nxv2f64.nxv2p0f64(<vscale x 2 x double*> %[[PTRS]],
; CHECK-NEXT: %[[VALS:.*]] = fadd <vscale x 2 x double> %[[GLOAD]],
; CHECK-NEXT: call void @llvm.masked.scatter.nxv2f64.nxv2p0f64(<vscale x 2 x double> %[[VALS]], <vscale x 2 x double*> %[[PTRS]],
entry:
br label %for.body
for.body: ; preds = %entry, %for.body
%i.05 = phi i64 [ %inc, %for.body ], [ 0, %entry ]
%mul = mul nuw nsw i64 %i.05, 7
%arrayidx = getelementptr inbounds double, double* %dst, i64 %mul
%0 = load double, double* %arrayidx, align 8
%add = fadd double %0, 1.000000e+00
store double %add, double* %arrayidx, align 8
%inc = add nuw nsw i64 %i.05, 1
%exitcond.not = icmp eq i64 %inc, %n
br i1 %exitcond.not, label %for.end, label %for.body, !llvm.loop !6
for.end: ; preds = %for.end.loopexit, %entry
ret void
}
define void @cond_stride7_f64(double* noalias nocapture %dst, i64* noalias nocapture readonly %cond, i64 %n) {
; CHECK-LABEL: @cond_stride7_f64(
; CHECK: vector.body
; CHECK: %[[MASK:.*]] = icmp ne <vscale x 2 x i64>
; CHECK: %[[PTRS:.*]] = getelementptr inbounds double, double* %dst, <vscale x 2 x i64> %{{.*}}
; CHECK-NEXT: %[[GLOAD:.*]] = call <vscale x 2 x double> @llvm.masked.gather.nxv2f64.nxv2p0f64(<vscale x 2 x double*> %[[PTRS]], i32 8, <vscale x 2 x i1> %[[MASK]]
; CHECK-NEXT: %[[VALS:.*]] = fadd <vscale x 2 x double> %[[GLOAD]],
; CHECK-NEXT: call void @llvm.masked.scatter.nxv2f64.nxv2p0f64(<vscale x 2 x double> %[[VALS]], <vscale x 2 x double*> %[[PTRS]], i32 8, <vscale x 2 x i1> %[[MASK]])
entry:
br label %for.body
for.body: ; preds = %entry, %for.inc
%i.07 = phi i64 [ %inc, %for.inc ], [ 0, %entry ]
%arrayidx = getelementptr inbounds i64, i64* %cond, i64 %i.07
%0 = load i64, i64* %arrayidx, align 8
%tobool.not = icmp eq i64 %0, 0
br i1 %tobool.not, label %for.inc, label %if.then
if.then: ; preds = %for.body
%mul = mul nsw i64 %i.07, 7
%arrayidx1 = getelementptr inbounds double, double* %dst, i64 %mul
%1 = load double, double* %arrayidx1, align 8
%add = fadd double %1, 1.000000e+00
store double %add, double* %arrayidx1, align 8
br label %for.inc
for.inc: ; preds = %for.body, %if.then
%inc = add nuw nsw i64 %i.07, 1
%exitcond.not = icmp eq i64 %inc, %n
br i1 %exitcond.not, label %for.end, label %for.body, !llvm.loop !6
for.end: ; preds = %for.end.loopexit, %entry
ret void
}
!0 = distinct !{!0, !1, !2, !3, !4, !5}
!1 = !{!"llvm.loop.mustprogress"}
!2 = !{!"llvm.loop.vectorize.width", i32 4}
!3 = !{!"llvm.loop.vectorize.scalable.enable", i1 true}
!4 = !{!"llvm.loop.interleave.count", i32 1}
!5 = !{!"llvm.loop.vectorize.enable", i1 true}
!6 = distinct !{!6, !1, !7, !3, !4, !5}
!7 = !{!"llvm.loop.vectorize.width", i32 2}