This is an archive of the discontinued LLVM Phabricator instance.

Differential D106677

[LV] Don't let ForceTargetInstructionCost override Invalid cost.
ClosedPublic

Authored by sdesmalen on Jul 23 2021, 9:15 AM.

Details

Reviewers
c-rhodes
david-arm
Commits
rG13ccb097258a: [LV] Don't let ForceTargetInstructionCost override Invalid cost.
Summary

Invalid costs can be used to avoid vectorization with a given VF, which is
used for scalable vectors to avoid things that the code-generator cannot
handle. If we override the cost using the -force-target-instruction-cost
option of the LV, we would override this mechanism, rendering the flag useless.

This change ensures the cost is only overriden when the original cost that
was calculated is valid. That allows the flag to be used in combination
with the -scalable-vectorization option.

Diff Detail

Unit TestsFailed

TimeTest
100 msx64 debian > LLVM.Transforms/LoopVectorize/AArch64::sve-inductions.ll
2,890 msx64 debian > libarcher.barrier::barrier.c
2,910 msx64 debian > libarcher.critical::critical.c
3,010 msx64 debian > libarcher.parallel::parallel-simple2.c
2,960 msx64 debian > libarcher.races::critical-unrelated.c
View Full Test Results (24 Failed)

Event Timeline

sdesmalen created this revision.Jul 23 2021, 9:15 AM
Herald added a subscriber: hiraditya. · View Herald TranscriptJul 23 2021, 9:15 AM
sdesmalen requested review of this revision.Jul 23 2021, 9:15 AM
Herald added a project: Restricted Project. · View Herald TranscriptJul 23 2021, 9:15 AM
Herald added a subscriber: llvm-commits. · View Herald Transcript
sdesmalen added reviewers: c-rhodes, david-arm.Jul 23 2021, 9:15 AM
Harbormaster completed remote builds in B115880: Diff 361255.Jul 23 2021, 9:57 AM
david-arm accepted this revision.Jul 26 2021, 3:12 AM

LGTM!

This revision is now accepted and ready to land.Jul 26 2021, 3:12 AM
sdesmalen updated this revision to Diff 361749.Jul 26 2021, 12:10 PM
sdesmalen edited the summary of this revision. (Show Details)

Split some of the tests from sve-inductions.ll to a non-aarch64-specific test using the -force-target-supports-scalable-vectors flag (partly because one of the tests requires VF=vscale x 1. The other tests moved seemed non-AArch64-specific.

Herald added a subscriber: kristof.beyls. · View Herald TranscriptJul 26 2021, 12:10 PM
sdesmalen updated this revision to Diff 361759.Jul 26 2021, 12:27 PM
sdesmalen edited the summary of this revision. (Show Details)

Moved changes for sve-inductions.ll to a separate NFC patch.

This revision was landed with ongoing or failed builds.Jul 26 2021, 12:29 PM
Closed by commit rG13ccb097258a: [LV] Don't let ForceTargetInstructionCost override Invalid cost. (authored by sdesmalen). · Explain Why
This revision was automatically updated to reflect the committed changes.
sdesmalen added a commit: rG13ccb097258a: [LV] Don't let ForceTargetInstructionCost override Invalid cost..
Harbormaster completed remote builds in B116246: Diff 361759.Jul 26 2021, 2:47 PM

Revision Contents

PathSize
llvm/
lib/
Transforms/
Vectorize/
3 lines
test/
Transforms/
LoopVectorize/
AArch64/
3 lines

Diff 361255

llvm/lib/Transforms/Vectorize/LoopVectorize.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 6,889 Lines • ▼ Show 20 Lines for (Instruction &I : BB->instructionsWithoutDebug()) {
// Skip ignored values. // Skip ignored values.
if (ValuesToIgnore.count(&I) || if (ValuesToIgnore.count(&I) ||
(VF.isVector() && VecValuesToIgnore.count(&I))) (VF.isVector() && VecValuesToIgnore.count(&I)))
continue; continue;
VectorizationCostTy C = getInstructionCost(&I, VF); VectorizationCostTy C = getInstructionCost(&I, VF);
// Check if we should override the cost. // Check if we should override the cost.
if (ForceTargetInstructionCost.getNumOccurrences() > 0) if (C.first.isValid() &&
ForceTargetInstructionCost.getNumOccurrences() > 0)
C.first = InstructionCost(ForceTargetInstructionCost); C.first = InstructionCost(ForceTargetInstructionCost);
// Keep a list of instructions with invalid costs. // Keep a list of instructions with invalid costs.
if (Invalid && !C.first.isValid()) if (Invalid && !C.first.isValid())
Invalid->emplace_back(&I, VF); Invalid->emplace_back(&I, VF);
BlockCost.first += C.first; BlockCost.first += C.first;
BlockCost.second |= C.second; BlockCost.second |= C.second;
▲ Show 20 Lines • Show All 3,530 Lines • Show Last 20 Lines

llvm/test/Transforms/LoopVectorize/AArch64/scalable-call.ll

; RUN: opt -S -loop-vectorize -force-vector-interleave=1 -instcombine -mattr=+sve -mtriple aarch64-unknown-linux-gnu -scalable-vectorization=on \ ; RUN: opt -S -loop-vectorize -force-vector-interleave=1 -instcombine -mattr=+sve -mtriple aarch64-unknown-linux-gnu -scalable-vectorization=on \
; RUN: -pass-remarks-missed=loop-vectorize < %s 2>%t | FileCheck %s ; RUN: -pass-remarks-missed=loop-vectorize < %s 2>%t | FileCheck %s
; RUN: cat %t | FileCheck %s --check-prefix=CHECK-REMARKS ; RUN: cat %t | FileCheck %s --check-prefix=CHECK-REMARKS
; RUN: opt -S -loop-vectorize -force-vector-interleave=1 -force-target-instruction-cost=1 -instcombine -mattr=+sve -mtriple aarch64-unknown-linux-gnu \
; RUN: -scalable-vectorization=on -pass-remarks-missed=loop-vectorize < %s 2>%t | FileCheck %s
; RUN: cat %t | FileCheck %s --check-prefix=CHECK-REMARKS
define void @vec_load(i64 %N, double* nocapture %a, double* nocapture readonly %b) { define void @vec_load(i64 %N, double* nocapture %a, double* nocapture readonly %b) {
; CHECK-LABEL: @vec_load ; CHECK-LABEL: @vec_load
; CHECK: vector.body: ; CHECK: vector.body:
; CHECK: %[[LOAD:.*]] = load <vscale x 2 x double>, <vscale x 2 x double>* ; CHECK: %[[LOAD:.*]] = load <vscale x 2 x double>, <vscale x 2 x double>*
; CHECK: call <vscale x 2 x double> @foo_vec(<vscale x 2 x double> %[[LOAD]]) ; CHECK: call <vscale x 2 x double> @foo_vec(<vscale x 2 x double> %[[LOAD]])
entry: entry:
%cmp7 = icmp sgt i64 %N, 0 %cmp7 = icmp sgt i64 %N, 0
▲ Show 20 Lines • Show All 236 Lines • Show Last 20 Lines