Page MenuHomePhabricator

[VP,Integer,#2] ExpandVectorPredication pass
Needs ReviewPublic

Authored by simoll on Apr 15 2020, 6:38 AM.

Details

Summary

This patch implements expansion of llvm.vp.* intrinsics (https://llvm.org/docs/LangRef.html#vector-predication-intrinsics).

VP expansion is required for targets that do not implement VP code generation. Since expansion is controllable with TTI, targets can switch on the VP intrinsics they do support in their backend offering a smooth transition strategy for VP code generation (VE, RISC-V V, ARM SVE, AVX512, ..).

ExpandVectorPredication

The ExpandVectorPredication pass that folds the EVL parameter into the vector mask and/or lowers a VP intrinsics to a standard IR instruction.

Diff Detail

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
lenary removed a subscriber: lenary.Apr 15 2020, 7:48 AM
vkmr added a subscriber: vkmr.Apr 16 2020, 3:56 AM
arsenm added inline comments.Apr 16 2020, 3:37 PM
llvm/include/llvm/CodeGen/ExpandVectorPredication.h
2

C++ mode comment

llvm/lib/CodeGen/ExpandVectorPredication.cpp
198–199

Shouldn't be true for FP divisions

272–273

Combine into one LLVM_DEBUG

simoll updated this revision to Diff 258250.Apr 17 2020, 1:27 AM
simoll marked 4 inline comments as done.
  • fixed sanitizeStrategy
  • arsenm's comments (thx!)
  • camelCaseForFunctionNames
  • rebased
simoll added inline comments.Apr 17 2020, 1:28 AM
llvm/lib/CodeGen/ExpandVectorPredication.cpp
198–199

Yep. At least not before we have vector-predicated, constrained fdiv.

reames resigned from this revision.May 22 2020, 3:26 PM
simoll planned changes to this revision.May 26 2020, 5:23 AM

This will use llvm.get.active.lane.mask to expand VP intrinsics for scalable vectors on targets that do not support the %evl parameter (D79100).

simoll updated this revision to Diff 268204.Jun 3 2020, 8:17 AM
simoll edited the summary of this revision. (Show Details)
  • Use llvm.get.active.lane.mask to fold the %evl parameter into the vector mask parameter for scalable VP ops.
  • Never discard %evl if the operation may cause UB (eg for sdiv, udiv).
  • Added tests for VP ops with scalable types.
craig.topper added inline comments.
llvm/include/llvm/IR/PredicatedInst.h
34 ↗(On Diff #268204)

Can this inherit from Instruction? Its a little surprising that it has Instruction in its name but has to be casted to Instruction.

50 ↗(On Diff #268204)

Can we move this above copyIRFlags to keep all the deleted things together and away from code that does stuff.

llvm/lib/CodeGen/ExpandVectorPredication.cpp
46

Should this start with lower case 'i'?

71

Add /* Vector */ comment to the true.

77

Save the register bit width instead of querying it again?

99

This isn't safe for sdiv/srem is it? INT_MIN/-1 is overflow and will trap.

248

Do we need the nullptr here? I had to lookup what argument that is.

llvm/lib/IR/PredicatedInst.cpp
28 ↗(On Diff #268204)

I think we prefer "auto *" for pointers?

simoll planned changes to this revision.Jun 4 2020, 1:33 AM
simoll marked 12 inline comments as done.

@craig.topper Thanks for your comments! I'll update shortly.

llvm/include/llvm/IR/PredicatedInst.h
34 ↗(On Diff #268204)

Conceptually, PredicatedInstruction is a super class of Instruction: An Instruction is a PredicatedInstruction with the mask/evl parameters hard-wired to "all lanes true". That's why we can't make Instruction an ancestor of this one.
I'll clarify this with a comment.

llvm/lib/CodeGen/ExpandVectorPredication.cpp
71

This really should have its own enum.

77

Moved this function in to the CachingVPExpander class and added a lane bits cache there.

99

Good point. I'll fix this.

llvm/lib/IR/PredicatedInst.cpp
28 ↗(On Diff #268204)

Turns out we do. Fixed.

simoll updated this revision to Diff 268387.Jun 4 2020, 1:50 AM
simoll marked 4 inline comments as done.
simoll added a reviewer: craig.topper.

Addressed @craig.topper 's comments

simoll marked an inline comment as done.Jun 4 2020, 2:11 AM
simoll added inline comments.
llvm/lib/IR/IntrinsicInst.cpp
304–307

This unrelated change fixes a bug (ParamFactor does not make sense as a return value here and is not initialized!). I'll push a fix in a separate commit along with a test for this case.

simoll updated this revision to Diff 268436.Jun 4 2020, 5:29 AM

Fixed VPIntrinsic::canIgnoreVectorLength separately (a0dfdda4e5e8) and rebased onto that patch.

There are still more "auto" variables that are pointers in this patch.

llvm/include/llvm/IR/PredicatedInst.h
34 ↗(On Diff #268204)

But a VPIntrinsic is also an Instruction isn't it? At least the CallInst for the vp intrinsic is an Instruction.

simoll planned changes to this revision.Jun 5 2020, 1:17 AM
simoll marked an inline comment as done.

Will fix remaining auto* cases.

llvm/include/llvm/IR/PredicatedInst.h
34 ↗(On Diff #268204)

The whole reason for having the PredicatedInstruction class is that we want to add a new perspective on the IR where all instructions may be predicated (through a mask and evl). There are no VPIntrinsics in that perspective.

In the standard IR perspective, using the Instruction type, VPIntrinsics are just regular CallInsts calling intrinsics.

simoll updated this revision to Diff 268724.Jun 5 2020, 3:00 AM
  • Use auto*
  • Rebased
craig.topper added inline comments.Jun 5 2020, 3:13 PM
llvm/include/llvm/IR/PredicatedInst.h
34 ↗(On Diff #268204)

I was just wanting to get rid of the need to cast to Instruction in copyIRFlags and getParent and replaceOperation etc. Everything this class represents must be an Instruction whether its predicated or not. No type safety comes from this class inheriting from User instead of Instruction since a CallInst to a VP intrinsic is also an Instruction.

simoll planned changes to this revision.Jun 8 2020, 12:56 AM
simoll marked an inline comment as done.

Will rephrase this patch with VPIntrinsic instead of PredicatedInstruction since the purpose of the latter will only become clear with generalized pattern matching/lifting optimizations to VP.

llvm/include/llvm/IR/PredicatedInst.h
34 ↗(On Diff #268204)

That's exactly the point: a PredicatedInstruction cannot implicitly cast to an Instruction because that would be an implicit change of perspective.

simoll updated this revision to Diff 269123.Jun 8 2020, 1:54 AM
  • Removed PredicatedInstruction, which isn't strictly necessary for this patch.
  • Rebased.
simoll edited the summary of this revision. (Show Details)Jun 8 2020, 2:01 AM
khchen added a subscriber: khchen.Sep 5 2020, 12:09 AM
simoll updated this revision to Diff 296121.Oct 5 2020, 2:49 AM
  • adapted to get.active.mask changes (ult instead of ule)
  • rebased
simoll updated this revision to Diff 296136.Oct 5 2020, 3:17 AM

fixed for privatized ElementCount members.

Any more feedback for this patch?

Aside from my nitpicks, I was wondering if it's possible to add tests for the other strategies. Even if that involved a strategy that you could force on the command-line, since I know we don't have real TTIs just now. It would be nice to see that the pass behaves in the various ways it's meant to.

llvm/lib/CodeGen/ExpandVectorPredication.cpp
313

stray lower-case l in EVL (or should it be %evl?)

318

typo (reassess), also preferably with a capital R to match the style of the other comments

352

typo: strictly

365

Comment trails off here.

387

You shouldn't need the parentheses in these switch cases, also the formatting with the breaks looks a little off to me.

Linking this to the SelectionDAG patch: D91441

simoll planned changes to this revision.Mon, Nov 16, 5:23 AM
simoll marked 3 inline comments as done.

Aside from my nitpicks, I was wondering if it's possible to add tests for the other strategies. Even if that involved a strategy that you could force on the command-line, since I know we don't have real TTIs just now. It would be nice to see that the pass behaves in the various ways it's meant to.

I like that idea! I'll add some cl::opts to override the target strategy for tests. Thx :)

simoll updated this revision to Diff 306066.Wed, Nov 18, 5:22 AM
simoll marked 2 inline comments as done.
  • Added override options to override the expansion strategy.
  • Added expansion tests for fixed and scalable types and all possible expansion strategies.
  • Rebased.
simoll added inline comments.Wed, Nov 18, 5:23 AM
llvm/include/llvm/IR/Intrinsics.td
1302 ↗(On Diff #306066)

FYI I'll commit the attribute changes to the VP intrinsics separately.

simoll updated this revision to Diff 306142.Wed, Nov 18, 9:52 AM

Rebased onto "speculatable" VP intrinsic patch.