This is an archive of the discontinued LLVM Phabricator instance.

Differential D110797

[MLIR] Replace std ops with arith dialect ops
ClosedPublic

Authored by Mogball on Sep 29 2021, 6:46 PM.

Details

Reviewers
antiagainst
rriddle
aartbik
ftynse
sjarus
clementval
nicolasvasilache
herhut
jsetoain
jpienaar
sscalpone
Commits
rGa54f4eae0e1d: [MLIR] Replace std ops with arith dialect ops
Summary

Precursor: https://reviews.llvm.org/D110200

Removed redundant ops from the standard dialect that were moved to the
arith or math dialects.

Renamed all instances of operations in the codebase and in tests.

Diff Detail

Event Timeline

Mogball created this revision.Sep 29 2021, 6:46 PM
Herald added a reviewer: antiagainst. · View Herald TranscriptSep 29 2021, 6:46 PM
Herald added a reviewer: rriddle. · View Herald Transcript
Herald added subscribers: wenzhicui, wrengr, Chia-hungDuan and 31 others. · View Herald Transcript
Herald added a reviewer: aartbik. · View Herald TranscriptSep 29 2021, 6:46 PM
Herald added a reviewer: ftynse. · View Herald Transcript
Herald added a reviewer: aartbik. · View Herald Transcript
Herald added a reviewer: sjarus. · View Herald Transcript
Herald added a reviewer: clementval. · View Herald Transcript
Mogball requested review of this revision.Sep 29 2021, 6:46 PM
Herald added a reviewer: nicolasvasilache. · View Herald TranscriptSep 29 2021, 6:46 PM
Herald added a reviewer: herhut. · View Herald Transcript
Herald added a project: Restricted Project. · View Herald Transcript
Herald added subscribers: stephenneuendorffer, nicolasvasilache. · View Herald Transcript
Harbormaster completed remote builds in B126482: Diff 376086.Sep 29 2021, 6:47 PM
Herald added a reviewer: jsetoain. · View Herald TranscriptSep 29 2021, 6:47 PM
Mogball added a comment.Sep 29 2021, 6:47 PM

All tests are passing but I still have some cleanup to do

rriddle added inline comments.Sep 29 2021, 7:07 PM
mlir/lib/Transforms/Utils/FoldUtils.cpp
63–64

Can you use the dialect hook instead? The only reason ConstantOp is here is because the standard dialect hasn't been split yet.

Mogball added inline comments.Sep 29 2021, 8:49 PM
mlir/lib/Transforms/Utils/FoldUtils.cpp
63–64

Yeah this is part of the "cleanup" I still have to do.

Mogball updated this revision to Diff 376103.Sep 29 2021, 10:17 PM

rename arith.const -> arith.constant

Harbormaster completed remote builds in B126493: Diff 376103.Sep 29 2021, 10:18 PM
rriddle requested changes to this revision.Sep 30 2021, 5:58 PM

Seems mostly mechanical, thanks!

mlir/docs/Tutorials/Toy/Ch-5.md
65–66

This likely needs an update (possibly same for Ch.6)

mlir/include/mlir/Conversion/SPIRVCommon/Pattern.h
25 ↗(On Diff #376103)

Please use the updated matchAndRewrite that takes an OpAdaptor.

mlir/lib/Analysis/AffineAnalysis.cpp
57–61

For reference, you should be able to do this.

mlir/lib/Analysis/CMakeLists.txt
39–45

It'd be really nice to remove all dialect dependencies from this target.

mlir/lib/Conversion/ArithmeticToSPIRV/ArithmeticToSPIRV.cpp
752–754

Functions should be fully namespace resolved and at the top-level.

mlir/lib/Conversion/StandardToSPIRV/StandardToSPIRV.cpp
13

In general, if you want to share pieces of code solely within the lib, you should place the include in the lib/ folder. include/ should *only* contain things that are meant to be outwardly exposed via the API, of which implementation utilities generally aren't. GPUCommon is an example: https://github.com/llvm/llvm-project/tree/main/mlir/lib/Conversion/GPUCommon

mlir/lib/Dialect/Arithmetic/IR/ArithmeticDialect.cpp
32–33

Only classes should really go in .cpp namespaces.

https://llvm.org/docs/CodingStandards.html#anonymous-namespaces
https://llvm.org/docs/CodingStandards.html#use-namespace-qualifiers-to-implement-previously-declared-functions

mlir/lib/Dialect/Arithmetic/IR/ArithmeticOps.cpp
556–559

std::tuple<Ts...> *?

633

Please use proper ||

697

Drop the newline here.

mlir/lib/Dialect/StandardOps/IR/Ops.cpp
138

Please remove usernames from TODO comments.

mlir/lib/Transforms/Utils/FoldUtils.cpp
63–64

Is this done or still pending?

mlir/test/IR/core-ops.mlir
32

Can you move these tests to a file in the Arith dialect folder?

mlir/test/IR/invalid-ops.mlir
4

Same comment as core-ops.mlir

This revision now requires changes to proceed.Sep 30 2021, 5:58 PM
Mogball marked 15 inline comments as done.Oct 1 2021, 10:10 AM
Mogball added inline comments.
mlir/lib/Dialect/Arithmetic/IR/ArithmeticDialect.cpp
32–33

Yeah this is one I need to drill myself to remember.

mlir/lib/Dialect/Arithmetic/IR/ArithmeticOps.cpp
556–559

This was one "reviewer bait" because I didn't stumble across and existing example of something like this in the LLVM/MLIR codebase.

Mogball updated this revision to Diff 376566.Oct 1 2021, 10:15 AM
Mogball marked 2 inline comments as done.

Code cleanup

Herald added a reviewer: jpienaar. · View Herald TranscriptOct 1 2021, 10:15 AM
Harbormaster completed remote builds in B126722: Diff 376566.Oct 1 2021, 10:15 AM
Mogball updated this revision to Diff 376605.Oct 1 2021, 12:27 PM

Rebase against upstream

Harbormaster completed remote builds in B126739: Diff 376605.Oct 1 2021, 12:27 PM
Mogball updated this revision to Diff 376660.Oct 1 2021, 7:04 PM

Fixing bazel build and removing a stray file..

Harbormaster completed remote builds in B126759: Diff 376660.Oct 1 2021, 7:04 PM
Mogball updated this revision to Diff 376676.Oct 2 2021, 12:27 AM

Defer splitting StandardTo* and *ToStandard passes until later

Harbormaster completed remote builds in B126767: Diff 376676.Oct 2 2021, 12:27 AM
rriddle accepted this revision.Oct 5 2021, 11:45 AM

Did a pass through of the resolved comments, and looks good.

mlir/lib/Transforms/Utils/FoldUtils.cpp
16

Are either of these headers(Arith and Standard) necessary now?

If not can you see about closing https://llvm.org/PR48490?

rriddle added inline comments.Oct 5 2021, 11:47 AM
mlir/lib/Transforms/Utils/FoldUtils.cpp
16

Sorry linked the wrong bug, meant closing PR44866.

Mogball marked 2 inline comments as done.Oct 6 2021, 10:11 AM
Mogball updated this revision to Diff 377589.Oct 6 2021, 10:12 AM

Remove FoldUtils dependency on StandardOps/Arithmetic. Fixes PR44866

Harbormaster completed remote builds in B127329: Diff 377589.Oct 6 2021, 10:12 AM
jpienaar accepted this revision.Oct 6 2021, 10:45 AM

Thanks!

This revision is now accepted and ready to land.Oct 6 2021, 10:45 AM
Mogball updated this revision to Diff 377719.Oct 6 2021, 4:14 PM
  • rebase against upstream
Harbormaster completed remote builds in B127423: Diff 377719.Oct 6 2021, 5:02 PM
Mogball updated this revision to Diff 377734.Oct 6 2021, 7:18 PM
  • rename ops in flang
Herald added a project: Restricted Project. · View Herald TranscriptOct 6 2021, 7:18 PM
Harbormaster completed remote builds in B127436: Diff 377734.Oct 6 2021, 7:31 PM
Herald added a subscriber: jdoerfert. · View Herald TranscriptOct 6 2021, 7:31 PM
Mogball updated this revision to Diff 377737.Oct 6 2021, 8:14 PM
  • clang-format
Herald added a reviewer: sscalpone. · View Herald TranscriptOct 6 2021, 8:14 PM
Harbormaster completed remote builds in B127437: Diff 377737.Oct 6 2021, 8:29 PM
Mogball updated this revision to Diff 377936.Oct 7 2021, 11:50 AM
  • rebase against upstream because that's my life now
Harbormaster completed remote builds in B127577: Diff 377936.Oct 7 2021, 12:10 PM
Mogball updated this revision to Diff 377982.Oct 7 2021, 1:01 PM
  • clang-format
Harbormaster completed remote builds in B127608: Diff 377982.Oct 7 2021, 1:38 PM
Mogball updated this revision to Diff 377993.Oct 7 2021, 1:39 PM
  • update python bindings
Harbormaster completed remote builds in B127618: Diff 377993.Oct 7 2021, 4:25 PM
Mogball updated this revision to Diff 379223.Oct 12 2021, 4:53 PM

updates with HEAD

Mogball updated this revision to Diff 379224.Oct 12 2021, 4:55 PM

wrong diff

Mogball updated this revision to Diff 379261.Oct 12 2021, 8:06 PM

update to HEAD

This revision was landed with ongoing or failed builds.Oct 12 2021, 8:07 PM
Closed by commit rGa54f4eae0e1d: [MLIR] Replace std ops with arith dialect ops (authored by Mogball). · Explain Why
This revision was automatically updated to reflect the committed changes.
Mogball added a commit: rGa54f4eae0e1d: [MLIR] Replace std ops with arith dialect ops.
Harbormaster completed remote builds in B128527: Diff 379261.Oct 12 2021, 8:25 PM
marbre added a subscriber: marbre.Oct 28 2021, 11:39 AM

I recently noticed that arith::ConstantOp support was added to the Cpp emitter with this commit. @jpienaar and @mehdi_amini, do we really want to add support for such ops to the printer? The alternative would be to add conversions, like the arith.constant to emitc.constant one which I have implemented in our mlir-emitc repo: https://github.com/iml130/mlir-emitc/blob/c3b7bf093417ecd50e4b246ffd85079dbd75e359/lib/Dialect/EmitC/Conversion/ArithToEmitC.cpp#L27-L38.

Mogball added a comment.Oct 29 2021, 9:48 AM
In D110797#3094354, @marbre wrote:

I recently noticed that arith::ConstantOp support was added to the Cpp emitter with this commit. @jpienaar and @mehdi_amini, do we really want to add support for such ops to the printer? The alternative would be to add conversions, like the arith.constant to emitc.constant one which I have implemented in our mlir-emitc repo: https://github.com/iml130/mlir-emitc/blob/c3b7bf093417ecd50e4b246ffd85079dbd75e359/lib/Dialect/EmitC/Conversion/ArithToEmitC.cpp#L27-L38.

I wasn't sure in what ways arith.constant superseded std.constant in EmitC. I had assumed that the C printer was wholly handling std.constant and not a subset of its functionality.

Large Diff

This large diff affects 810 files. Files without inline comments have been collapsed. Expand All Files

Revision Contents

PathSize
flang/
include/
flang/
Lower/
8 lines
Support/
6 lines
Optimizer/
Dialect/
3 lines
14 lines
Support/
4 lines
6 lines
Transforms/
5 lines
lib/
Lower/
22 lines
18 lines
14 lines
2 lines
7 lines
33 lines
95 lines
Optimizer/
CodeGen/
26 lines
Dialect/
1 line
24 lines
Transforms/
3 lines
1 line
41 lines
10 lines
56 lines
test/
Fir/
46 lines
18 lines
40 lines
8 lines
8 lines
4 lines
2 lines
284 lines
140 lines
208 lines
50 lines
mlir/
docs/
Bindings/
303 lines
234 lines
146 lines
63 lines
2 lines
2 lines
Dialects/
114 lines
Linalg/
48 lines
20 lines
612 lines
48 lines
397 lines
Rationale/
30 lines
102 lines
313 lines
20 lines
20 lines
41 lines
Tutorials/
Toy/
127 lines
27 lines
includes/
img/
2 lines
2 lines
6 lines
2 lines
examples/
standalone/
standalone-opt/
1 line
5 lines
test/
Standalone/
2 lines
toy/
Ch5/
mlir/
19 lines
Ch6/
mlir/
19 lines
11 lines
Ch7/
mlir/
19 lines
11 lines
include/
mlir/
Conversion/
ArithmeticToLLVM/
28 lines
ArithmeticToSPIRV/
28 lines
2 lines
72 lines
VectorToSCF/
2 lines
Dialect/
Affine/
IR/
9 lines
4 lines
Arithmetic/
1 line
IR/
59 lines
2 lines
68 lines
Transforms/
5 lines
42 lines
26 lines
ArmSVE/
22 lines
Async/
6 lines
Complex/
IR/
2 lines
3 lines
GPU/
2 lines
1 line
2 lines
Linalg/
IR/
9 lines
8 lines
1 line
2 lines
Utils/
2 lines
MemRef/
IR/
1 line
2 lines
OpenMP/
2 lines
SCF/
8 lines
7 lines
11 lines
Shape/
IR/
1 line
2 lines
SparseTensor/
Transforms/
6 lines
StandardOps/
IR/
67 lines
1239 lines
46 lines
Transforms/
2 lines
Utils/
3 lines
Tensor/
IR/
1 line
1 line
Vector/
5 lines
4 lines
X86Vector/
2 lines
IR/
4 lines
2 lines
2 lines
Transforms/
3 lines
29 lines
lib/
Analysis/
14 lines
5 lines
5 lines
1 line
3 lines
9 lines
Conversion/
AffineToStandard/
93 lines
1 line
ArithmeticToLLVM/
304 lines
16 lines
ArithmeticToSPIRV/
826 lines
16 lines
AsyncToLLVM/
10 lines
1 line
2 lines
ComplexToLLVM/
2 lines
ComplexToStandard/
1 line
462 lines
GPUCommon/
1 line
2 lines
GPUToNVVM/
2 lines
16 lines
GPUToROCDL/
1 line
15 lines
GPUToSPIRV/
2 lines
2 lines
LinalgToStandard/
3 lines
MathToLLVM/
12 lines
MathToLibm/
1 line
7 lines
MathToSPIRV/
49 lines
OpenACCToSCF/
3 lines
3 lines
OpenMPToLLVM/
1 line
2 lines
4 lines
SCFToGPU/
1 line
61 lines
1 line
SCFToOpenMP/
1 line
31 lines
SCFToSPIRV/
1 line
2 lines
SCFToStandard/
1 line
7 lines
SPIRVCommon/
45 lines
SPIRVToLLVM/
1 line
3 lines
ShapeToStandard/
1 line
125 lines
StandardToLLVM/
1 line
202 lines
StandardToSPIRV/
3 lines
527 lines
9 lines
TosaToLinalg/
1 line
889 lines
7 lines
TosaToStandard/
1 line
60 lines
2 lines
VectorToGPU/
1 line
13 lines
VectorToLLVM/
1 line
17 lines
3 lines
VectorToSCF/
1 line
35 lines
Dialect/
Affine/
IR/
12 lines
1 line
Transforms/
3 lines
3 lines
1 line
112 lines
Arithmetic/
1 line
IR/
11 lines
308 lines
Transforms/
65 lines
17 lines
150 lines
ArmSVE/
IR/
12 lines
Async/
Transforms/
97 lines
11 lines
3 lines
4 lines
Complex/
IR/
2 lines
18 lines
EmitC/
IR/
2 lines
GPU/
2 lines
IR/
10 lines
Transforms/
83 lines
4 lines
9 lines
Linalg/
IR/
1 line
9 lines
78 lines
Transforms/
6 lines
1 line
10 lines
2 lines
7 lines
2 lines
7 lines
2 lines
4 lines
4 lines
3 lines
4 lines
27 lines
16 lines
21 lines
48 lines
Utils/
1 line
26 lines
Math/
Transforms/
12 lines
1 line
28 lines
207 lines
MemRef/
IR/
1 line
38 lines
Transforms/
1 line
3 lines
3 lines
OpenACC/
1 line
IR/
5 lines
Quant/
1 line
Transforms/
5 lines
SCF/
1 line
57 lines
Transforms/
1 line
9 lines
29 lines
7 lines
16 lines
48 lines
4 lines
Shape/
IR/
1 line
14 lines
Transforms/
1 line
4 lines
SparseTensor/
IR/
1 line
5 lines
Transforms/
1 line
83 lines
5 lines
75 lines
Utils/
1 line
131 lines
StandardOps/
1 line
IR/
1318 lines
Transforms/
19 lines
1 line
170 lines
23 lines
Utils/
30 lines
Tensor/
IR/
1 line
15 lines
Transforms/
18 lines
1 line
Vector/
1 line
14 lines
64 lines
149 lines
14 lines
X86Vector/
Transforms/
1 line
7 lines
Dialect/
Arithmetic/
Transforms/
Transforms/
16 lines
Target/
Cpp/
2 lines
6 lines
11 lines
Transforms/
1 line
4 lines
Utils/
1 line
17 lines
156 lines
3 lines
python/
9 lines
mlir/
dialects/
15 lines
70 lines
52 lines
5 lines
linalg/
opdsl/
lang/
40 lines
test/
Analysis/
8 lines
12 lines
156 lines
30 lines
16 lines
20 lines
CAPI/
6 lines
40 lines
18 lines
Conversion/
AffineToStandard/
24 lines
590 lines
ArithmeticToLLVM/
365 lines
ArithmeticToLLVM/
StandardToLLVM/
67 lines
ArithmeticToSPIRV/
StandardToSPIRV/
arithmetic-to-spirv.mlir
std-ops-to-spirv.mlir
428 lines
AsyncToLLVM/
14 lines
36 lines
ComplexToLLVM/
16 lines
ComplexToStandard/
356 lines
2 lines
GPUCommon/
6 lines
8 lines
GPUToNVVM/
26 lines
20 lines
GPUToROCDL/
14 lines
GPUToSPIRV/
18 lines
24 lines
2 lines
4 lines
GPUToVulkan/
4 lines
LinalgToSPIRV/
8 lines
LinalgToVector/
14 lines
MathToLibm/
8 lines
MemRefToLLVM/
16 lines
10 lines
14 lines
4 lines
MemRefToSPIRV/
2 lines
OpenACCToLLVM/
10 lines
OpenMPToLLVM/
6 lines
SCFToGPU/
24 lines
108 lines
60 lines
12 lines
SCFToOpenMP/
58 lines
SCFToSPIRV/
18 lines
24 lines
SCFToStandard/
178 lines
SPIRVToLLVM/
2 lines
ShapeToStandard/
268 lines
StandardToLLVM/
func-memref-return.mlir
func-memref.mlir
131 lines
92 lines
23 lines
497 lines
StandardToSPIRV/
324 lines
TosaToLinalg/
978 lines
TosaToStandard/
94 lines
VectorToGPU/
26 lines
VectorToLLVM/
26 lines
210 lines
VectorToROCDL/
6 lines
VectorToSCF/
2 lines
4 lines
12 lines
8 lines
130 lines
Dialect/
AMX/
4 lines
4 lines
6 lines
Affine/
SuperVectorize/
18 lines
40 lines
278 lines
58 lines
6 lines
6 lines
12 lines
132 lines
6 lines
12 lines
16 lines
248 lines
26 lines
90 lines
26 lines
32 lines
22 lines
6 lines
28 lines
30 lines
34 lines
2 lines
4 lines
24 lines
32 lines
46 lines
174 lines
108 lines
8 lines
68 lines
94 lines
Arithmetic/
14 lines
388 lines
56 lines
555 lines
706 lines
ArmSVE/
6 lines
2 lines
Async/
26 lines
16 lines
4 lines
4 lines
4 lines
20 lines
32 lines
12 lines
4 lines
Complex/
16 lines
GPU/
106 lines
106 lines
44 lines
28 lines
12 lines
2 lines
20 lines
74 lines
20 lines
LLVMIR/
8 lines
Linalg/
9 lines
34 lines
6 lines
160 lines
144 lines
40 lines
12 lines
72 lines
20 lines
24 lines
12 lines
48 lines
12 lines
12 lines
10 lines
16 lines
10 lines
86 lines
34 lines
16 lines
268 lines
14 lines
50 lines
80 lines
26 lines
30 lines
64 lines
156 lines
32 lines
72 lines
22 lines
58 lines
28 lines
6 lines
18 lines
44 lines
6 lines
8 lines
192 lines
8 lines
32 lines
18 lines
20 lines
2 lines
16 lines
120 lines
14 lines
12 lines
90 lines
62 lines
4 lines
14 lines
90 lines
84 lines
10 lines
100 lines
8 lines
20 lines
36 lines
28 lines
16 lines
20 lines
34 lines
76 lines
60 lines
84 lines
138 lines
238 lines
Math/
44 lines
284 lines
MemRef/
52 lines
16 lines
4 lines
6 lines
OpenACC/
12 lines
20 lines
204 lines
OpenMP/
54 lines
14 lines
Quant/
10 lines
66 lines
SCF/
6 lines
232 lines
144 lines
110 lines
14 lines
78 lines
78 lines
220 lines
128 lines
166 lines
128 lines
134 lines
20 lines
112 lines
94 lines
SPIRV/
IR/
4 lines
2 lines
Shape/
76 lines
2 lines
6 lines
2 lines
SparseTensor/
112 lines
78 lines
2 lines
16 lines
8 lines
716 lines
546 lines
516 lines
50 lines
100 lines
170 lines
52 lines
40 lines
40 lines
40 lines
48 lines
48 lines
8 lines
32 lines
34 lines
38 lines
34 lines
262 lines
24 lines
Standard/
13 lines
18 lines
405 lines
101 lines
24 lines
8 lines
52 lines
24 lines
12 lines
Tensor/
18 lines
126 lines
14 lines
10 lines
34 lines
Tosa/
6 lines
6 lines
Vector/
182 lines
208 lines
84 lines
176 lines
76 lines
44 lines
38 lines
62 lines
120 lines
50 lines
82 lines
40 lines
92 lines
Examples/
Toy/
Ch5/
28 lines
Ch6/
28 lines
Ch7/
28 lines
IR/
536 lines
4 lines
8 lines
536 lines
34 lines
6 lines
20 lines
14 lines
14 lines
2 lines
162 lines
8 lines
204 lines
20 lines
6 lines
Integration/
Dialect/
Async/
CPU/
14 lines
18 lines
36 lines
36 lines
Linalg/
CPU/
38 lines
28 lines
14 lines
2 lines
14 lines
12 lines
16 lines
16 lines
16 lines
16 lines
16 lines
6 lines
2 lines
8 lines
4 lines
4 lines
2 lines
6 lines
SparseTensor/
CPU/
6 lines
70 lines
10 lines
28 lines
10 lines
12 lines
18 lines
38 lines
6 lines
14 lines
26 lines
22 lines
26 lines
10 lines
22 lines
8 lines
6 lines
Standard/
CPU/
38 lines
Vector/
CPU/
AMX/
22 lines
24 lines
16 lines
30 lines
12 lines
X86Vector/
10 lines
8 lines
4 lines
178 lines
14 lines
16 lines
30 lines
20 lines
10 lines
10 lines
22 lines
10 lines
16 lines
16 lines
6 lines
32 lines
8 lines
8 lines
20 lines
28 lines
16 lines
16 lines
14 lines
14 lines
70 lines
6 lines
20 lines
6 lines
20 lines
20 lines
2 lines
20 lines
3 lines
3 lines
56 lines
12 lines
4 lines
70 lines
70 lines
58 lines
36 lines
54 lines
30 lines
28 lines
28 lines
20 lines
28 lines
GPU/
CUDA/
TensorCore/
14 lines
12 lines
36 lines
36 lines
36 lines
18 lines
36 lines
10 lines
36 lines
10 lines
12 lines
36 lines
14 lines
8 lines
ROCM/
8 lines
8 lines
14 lines
28 lines
Interfaces/
InferShapedTypeOpInterface/
28 lines
Pass/
2 lines
2 lines
Target/
Cpp/
2 lines
10 lines
2 lines
4 lines
20 lines
LLVMIR/
4 lines
Transforms/
34 lines
38 lines
14 lines
14 lines
8 lines
16 lines
460 lines
14 lines
604 lines
146 lines
38 lines
154 lines
122 lines
208 lines
22 lines
46 lines
20 lines
10 lines
112 lines
136 lines
22 lines
116 lines
68 lines
4 lines
24 lines
60 lines
84 lines
44 lines
16 lines
8 lines
68 lines
46 lines
48 lines
34 lines
38 lines
4 lines
20 lines
4 lines
4 lines
4 lines
lib/
Dialect/
GPU/
4 lines
Linalg/
8 lines
Math/
4 lines
SCF/
6 lines
Test/
5 lines
7 lines
Vector/
27 lines
IR/
54 lines
Transforms/
6 lines
mlir-cpu-runner/
7 lines
5 lines
9 lines
21 lines
26 lines
14 lines
16 lines
98 lines
26 lines
18 lines
34 lines
19 lines
22 lines
mlir-lsp-server/
2 lines
2 lines
8 lines
2 lines
mlir-opt/
1 line
mlir-spirv-cpu-runner/
32 lines
16 lines
mlir-tblgen/
4 lines
mlir-vulkan-runner/
18 lines
12 lines
14 lines
18 lines
20 lines
24 lines
python/
dialects/
linalg/
opdsl/
102 lines
66 lines
18 lines
13 lines
38 lines
291 lines
integration/
dialects/
linalg/
40 lines
ir/
8 lines
62 lines
tools/
mlir-spirv-cpu-runner/
1 line
7 lines
mlir-vulkan-runner/
1 line
7 lines
unittests/
ExecutionEngine/
1 line
18 lines
Interfaces/
9 lines
utils/
bazel/
llvm-project-overlay/
mlir/
161 lines
test/
8 lines

Diff 379262

flang/include/flang/Lower/CharacterRuntime.h

flang/include/flang/Lower/Support/Utils.h

flang/include/flang/Optimizer/Dialect/FIROps.h

flang/include/flang/Optimizer/Dialect/FIROps.td

flang/include/flang/Optimizer/Support/InitFIR.h

flang/include/flang/Optimizer/Support/Utils.h

flang/include/flang/Optimizer/Transforms/RewritePatterns.td

flang/lib/Lower/CharacterExpr.cpp

flang/lib/Lower/CharacterRuntime.cpp

flang/lib/Lower/ComplexExpr.cpp

flang/lib/Lower/DoLoopHelper.cpp

flang/lib/Lower/FIRBuilder.cpp

flang/lib/Lower/IO.cpp

flang/lib/Lower/IntrinsicCall.cpp

flang/lib/Optimizer/CodeGen/PreCGRewrite.cpp

flang/lib/Optimizer/Dialect/CMakeLists.txt

flang/lib/Optimizer/Dialect/FIROps.cpp

flang/lib/Optimizer/Transforms/AbstractResult.cpp

flang/lib/Optimizer/Transforms/AffineDemotion.cpp

flang/lib/Optimizer/Transforms/AffinePromotion.cpp

flang/lib/Optimizer/Transforms/CharacterConversion.cpp

flang/lib/Optimizer/Transforms/RewriteLoop.cpp

flang/test/Fir/abstract-results.fir

flang/test/Fir/affine-demotion.fir

flang/test/Fir/affine-promotion.fir

flang/test/Fir/cg-ops.fir

flang/test/Fir/char-conversion.fir

flang/test/Fir/convert-fold.fir

flang/test/Fir/external-mangling.fir

flang/test/Fir/fir-ops.fir

flang/test/Fir/invalid.fir

flang/test/Fir/loop01.fir

flang/test/Fir/loop02.fir

mlir/docs/Bindings/Python.md

mlir/docs/BufferDeallocationInternals.md

mlir/docs/Bufferization.md

mlir/docs/DeclarativeRewrites.md

mlir/docs/Diagnostics.md

mlir/docs/DialectConversion.md

mlir/docs/Dialects/Affine.md

mlir/docs/Dialects/Linalg/_index.md

mlir/docs/Dialects/MemRef.md

mlir/docs/Dialects/Vector.md

mlir/docs/Dialects/emitc.md

mlir/docs/LangRef.md

mlir/docs/Rationale/MLIRForGraphAlgorithms.md

mlir/docs/Rationale/Rationale.md

mlir/docs/SPIRVToLLVMDialectConversion.md

mlir/docs/SymbolsAndSymbolTables.md

mlir/docs/TargetLLVMIR.md

mlir/docs/Traits.md

mlir/docs/Tutorials/Toy/Ch-5.md

Show All 9 Lines
To make it more interesting, in this chapter we will consider that we want to To make it more interesting, in this chapter we will consider that we want to
reuse existing optimizations implemented in a dialect optimizing affine reuse existing optimizations implemented in a dialect optimizing affine
transformations: `Affine`. This dialect is tailored to the computation-heavy transformations: `Affine`. This dialect is tailored to the computation-heavy
part of the program and is limited: it doesn't support representing our part of the program and is limited: it doesn't support representing our
`toy.print` builtin, for instance, neither should it! Instead, we can target `toy.print` builtin, for instance, neither should it! Instead, we can target
`Affine` for the computation heavy part of Toy, and in the `Affine` for the computation heavy part of Toy, and in the
[next chapter](Ch-6.md) directly target the `LLVM IR` dialect for lowering [next chapter](Ch-6.md) directly target the `LLVM IR` dialect for lowering
`print`. As part of this lowering, we will be lowering from the `print`. As part of this lowering, we will be lowering from the
[TensorType](../../Dialects/Builtin.md/#rankedtensortype) that `Toy` [TensorType](../../Dialects/Builtin.md/#rankedtensortype) that `Toy` operates on
operates on to the [MemRefType](../../Dialects/Builtin.md/#memreftype) that is to the [MemRefType](../../Dialects/Builtin.md/#memreftype) that is indexed via
indexed via an affine loop-nest. Tensors represent an abstract value-typed an affine loop-nest. Tensors represent an abstract value-typed sequence of data,
sequence of data, meaning that they don't live in any memory. MemRefs, on the meaning that they don't live in any memory. MemRefs, on the other hand,
other hand, represent lower level buffer access, as they are concrete represent lower level buffer access, as they are concrete references to a region
references to a region of memory. of memory.
# Dialect Conversions # Dialect Conversions
MLIR has many different dialects, so it is important to have a unified framework MLIR has many different dialects, so it is important to have a unified framework
for [converting](../../../getting_started/Glossary.md/#conversion) between them. This is where the for [converting](../../../getting_started/Glossary.md/#conversion) between them.
`DialectConversion` framework comes into play. This framework allows for This is where the `DialectConversion` framework comes into play. This framework
transforming a set of *illegal* operations to a set of *legal* ones. To use this allows for transforming a set of *illegal* operations to a set of *legal* ones.
framework, we need to provide two things (and an optional third): To use this framework, we need to provide two things (and an optional third):
* A [Conversion Target](../../DialectConversion.md/#conversion-target) * A [Conversion Target](../../DialectConversion.md/#conversion-target)
- This is the formal specification of what operations or dialects are - This is the formal specification of what operations or dialects are
legal for the conversion. Operations that aren't legal will require legal for the conversion. Operations that aren't legal will require
rewrite patterns to perform rewrite patterns to perform
[legalization](../../../getting_started/Glossary.md/#legalization). [legalization](../../../getting_started/Glossary.md/#legalization).
* A set of * A set of
[Rewrite Patterns](../../DialectConversion.md/#rewrite-pattern-specification) [Rewrite Patterns](../../DialectConversion.md/#rewrite-pattern-specification)
- This is the set of [patterns](../QuickstartRewrites.md) used to - This is the set of [patterns](../QuickstartRewrites.md) used to convert
convert *illegal* operations into a set of zero or more *legal* ones. *illegal* operations into a set of zero or more *legal* ones.
* Optionally, a [Type Converter](../../DialectConversion.md/#type-conversion). * Optionally, a [Type Converter](../../DialectConversion.md/#type-conversion).
- If provided, this is used to convert the types of block arguments. We - If provided, this is used to convert the types of block arguments. We
won't be needing this for our conversion. won't be needing this for our conversion.
## Conversion Target ## Conversion Target
For our purposes, we want to convert the compute-intensive `Toy` operations into For our purposes, we want to convert the compute-intensive `Toy` operations into
a combination of operations from the `Affine`, `MemRef` and `Standard` dialects a combination of operations from the `Affine`, `MemRef` and `Standard` dialects
for further optimization. To start off the lowering, we first define our for further optimization. To start off the lowering, we first define our
conversion target: conversion target:
```c++ ```c++
void ToyToAffineLoweringPass::runOnFunction() { void ToyToAffineLoweringPass::runOnFunction() {
// The first thing to define is the conversion target. This will define the // The first thing to define is the conversion target. This will define the
// final target for this lowering. // final target for this lowering.
mlir::ConversionTarget target(getContext()); mlir::ConversionTarget target(getContext());
// We define the specific operations, or dialects, that are legal targets for // We define the specific operations, or dialects, that are legal targets for
// this lowering. In our case, we are lowering to a combination of the // this lowering. In our case, we are lowering to a combination of the
// `Affine`, `MemRef` and `Standard` dialects. // `Affine`, `Arithmetic`, `MemRef`, and `Standard` dialects.
rriddleUnsubmitted
Done
ReplyInline Actions

This likely needs an update (possibly same for Ch.6)

rriddle: This likely needs an update (possibly same for Ch.6)
target.addLegalDialect<mlir::AffineDialect, mlir::memref::MemRefDialect, target.addLegalDialect<AffineDialect, arith::ArithmeticDialect,
mlir::StandardOpsDialect>(); memref::MemRefDialect, StandardOpsDialect>();
// We also define the Toy dialect as Illegal so that the conversion will fail // We also define the Toy dialect as Illegal so that the conversion will fail
// if any of these operations are *not* converted. Given that we actually want // if any of these operations are *not* converted. Given that we actually want
// a partial lowering, we explicitly mark the Toy operations that don't want // a partial lowering, we explicitly mark the Toy operations that don't want
// to lower, `toy.print`, as *legal*. // to lower, `toy.print`, as *legal*.
target.addIllegalDialect<ToyDialect>(); target.addIllegalDialect<ToyDialect>();
target.addLegalOp<PrintOp>(); target.addLegalOp<PrintOp>();
... ...
} }
``` ```
Above, we first set the toy dialect to illegal, and then the print operation Above, we first set the toy dialect to illegal, and then the print operation as
as legal. We could have done this the other way around. legal. We could have done this the other way around. Individual operations
Individual operations always take precedence over the (more generic) dialect always take precedence over the (more generic) dialect definitions, so the order
definitions, so the order doesn't matter. See `ConversionTarget::getOpInfo` doesn't matter. See `ConversionTarget::getOpInfo` for the details.
for the details.
## Conversion Patterns ## Conversion Patterns
After the conversion target has been defined, we can define how to convert the After the conversion target has been defined, we can define how to convert the
*illegal* operations into *legal* ones. Similarly to the canonicalization *illegal* operations into *legal* ones. Similarly to the canonicalization
framework introduced in [chapter 3](Ch-3.md), the framework introduced in [chapter 3](Ch-3.md), the
[`DialectConversion` framework](../../DialectConversion.md) also uses [`DialectConversion` framework](../../DialectConversion.md) also uses
[RewritePatterns](../QuickstartRewrites.md) to perform the conversion logic. [RewritePatterns](../QuickstartRewrites.md) to perform the conversion logic.
These patterns may be the `RewritePatterns` seen before or a new type of pattern These patterns may be the `RewritePatterns` seen before or a new type of pattern
specific to the conversion framework `ConversionPattern`. `ConversionPatterns` specific to the conversion framework `ConversionPattern`. `ConversionPatterns`
are different from traditional `RewritePatterns` in that they accept an are different from traditional `RewritePatterns` in that they accept an
additional `operands` parameter containing operands that have been additional `operands` parameter containing operands that have been
remapped/replaced. This is used when dealing with type conversions, as the remapped/replaced. This is used when dealing with type conversions, as the
pattern will want to operate on values of the new type but match against the pattern will want to operate on values of the new type but match against the
old. For our lowering, this invariant will be useful as it translates from the old. For our lowering, this invariant will be useful as it translates from the
[TensorType](../../Dialects/Builtin.md/#rankedtensortype) currently [TensorType](../../Dialects/Builtin.md/#rankedtensortype) currently being
being operated on to the [MemRefType](../../Dialects/Builtin.md/#memreftype). operated on to the [MemRefType](../../Dialects/Builtin.md/#memreftype). Let's
Let's look at a snippet of lowering the `toy.transpose` operation: look at a snippet of lowering the `toy.transpose` operation:
```c++ ```c++
/// Lower the `toy.transpose` operation to an affine loop nest. /// Lower the `toy.transpose` operation to an affine loop nest.
struct TransposeOpLowering : public mlir::ConversionPattern { struct TransposeOpLowering : public mlir::ConversionPattern {
TransposeOpLowering(mlir::MLIRContext *ctx) TransposeOpLowering(mlir::MLIRContext *ctx)
: mlir::ConversionPattern(TransposeOp::getOperationName(), 1, ctx) {} : mlir::ConversionPattern(TransposeOp::getOperationName(), 1, ctx) {}
/// Match and rewrite the given `toy.transpose` operation, with the given /// Match and rewrite the given `toy.transpose` operation, with the given
▲ Show 20 Lines • Show All 69 Lines • ▼ Show 20 Lines
potential design considerations when it comes to partial lowering. In our potential design considerations when it comes to partial lowering. In our
lowering, we transform from a value-type, TensorType, to an allocated lowering, we transform from a value-type, TensorType, to an allocated
(buffer-like) type, MemRefType. However, given that we do not lower the (buffer-like) type, MemRefType. However, given that we do not lower the
`toy.print` operation, we need to temporarily bridge these two worlds. There are `toy.print` operation, we need to temporarily bridge these two worlds. There are
many ways to go about this, each with their own tradeoffs: many ways to go about this, each with their own tradeoffs:
* Generate `load` operations from the buffer * Generate `load` operations from the buffer
One option is to generate `load` operations from the buffer type to materialize One option is to generate `load` operations from the buffer type to
an instance of the value type. This allows for the definition of the `toy.print` materialize an instance of the value type. This allows for the definition of
operation to remain unchanged. The downside to this approach is that the the `toy.print` operation to remain unchanged. The downside to this approach
optimizations on the `affine` dialect are limited, because the `load` will is that the optimizations on the `affine` dialect are limited, because the
actually involve a full copy that is only visible *after* our optimizations have `load` will actually involve a full copy that is only visible *after* our
been performed. optimizations have been performed.
* Generate a new version of `toy.print` that operates on the lowered type * Generate a new version of `toy.print` that operates on the lowered type
Another option would be to have another, lowered, variant of `toy.print` that Another option would be to have another, lowered, variant of `toy.print`
operates on the lowered type. The benefit of this option is that there is no that operates on the lowered type. The benefit of this option is that there
hidden, unnecessary copy to the optimizer. The downside is that another is no hidden, unnecessary copy to the optimizer. The downside is that
operation definition is needed that may duplicate many aspects of the first. another operation definition is needed that may duplicate many aspects of
Defining a base class in [ODS](../../OpDefinitions.md) may simplify this, but the first. Defining a base class in [ODS](../../OpDefinitions.md) may
you still need to treat these operations separately. simplify this, but you still need to treat these operations separately.
* Update `toy.print` to allow for operating on the lowered type * Update `toy.print` to allow for operating on the lowered type
A third option is to update the current definition of `toy.print` to allow for A third option is to update the current definition of `toy.print` to allow
operating the on the lowered type. The benefit of this approach is that it is for operating the on the lowered type. The benefit of this approach is that
simple, does not introduce an additional hidden copy, and does not require it is simple, does not introduce an additional hidden copy, and does not
another operation definition. The downside to this option is that it requires require another operation definition. The downside to this option is that it
mixing abstraction levels in the `Toy` dialect. requires mixing abstraction levels in the `Toy` dialect.
For the sake of simplicity, we will use the third option for this lowering. This For the sake of simplicity, we will use the third option for this lowering. This
involves updating the type constraints on the PrintOp in the operation involves updating the type constraints on the PrintOp in the operation
definition file: definition file:
```tablegen ```tablegen
def PrintOp : Toy_Op<"print"> { def PrintOp : Toy_Op<"print"> {
... ...
Show All 17 Linesfunc @main() {
toy.return toy.return
} }
``` ```
With affine lowering added to our pipeline, we can now generate: With affine lowering added to our pipeline, we can now generate:
```mlir ```mlir
func @main() { func @main() {
%cst = constant 1.000000e+00 : f64 %cst = arith.constant 1.000000e+00 : f64
%cst_0 = constant 2.000000e+00 : f64 %cst_0 = arith.constant 2.000000e+00 : f64
%cst_1 = constant 3.000000e+00 : f64 %cst_1 = arith.constant 3.000000e+00 : f64
%cst_2 = constant 4.000000e+00 : f64 %cst_2 = arith.constant 4.000000e+00 : f64
%cst_3 = constant 5.000000e+00 : f64 %cst_3 = arith.constant 5.000000e+00 : f64
%cst_4 = constant 6.000000e+00 : f64 %cst_4 = arith.constant 6.000000e+00 : f64
// Allocating buffers for the inputs and outputs. // Allocating buffers for the inputs and outputs.
%0 = alloc() : memref<3x2xf64> %0 = memref.alloc() : memref<3x2xf64>
%1 = alloc() : memref<3x2xf64> %1 = memref.alloc() : memref<3x2xf64>
%2 = alloc() : memref<2x3xf64> %2 = memref.alloc() : memref<2x3xf64>
// Initialize the input buffer with the constant values. // Initialize the input buffer with the constant values.
affine.store %cst, %2[0, 0] : memref<2x3xf64> affine.store %cst, %2[0, 0] : memref<2x3xf64>
affine.store %cst_0, %2[0, 1] : memref<2x3xf64> affine.store %cst_0, %2[0, 1] : memref<2x3xf64>
affine.store %cst_1, %2[0, 2] : memref<2x3xf64> affine.store %cst_1, %2[0, 2] : memref<2x3xf64>
affine.store %cst_2, %2[1, 0] : memref<2x3xf64> affine.store %cst_2, %2[1, 0] : memref<2x3xf64>
affine.store %cst_3, %2[1, 1] : memref<2x3xf64> affine.store %cst_3, %2[1, 1] : memref<2x3xf64>
affine.store %cst_4, %2[1, 2] : memref<2x3xf64> affine.store %cst_4, %2[1, 2] : memref<2x3xf64>
// Load the transpose value from the input buffer and store it into the // Load the transpose value from the input buffer and store it into the
// next input buffer. // next input buffer.
affine.for %arg0 = 0 to 3 { affine.for %arg0 = 0 to 3 {
affine.for %arg1 = 0 to 2 { affine.for %arg1 = 0 to 2 {
%3 = affine.load %2[%arg1, %arg0] : memref<2x3xf64> %3 = affine.load %2[%arg1, %arg0] : memref<2x3xf64>
affine.store %3, %1[%arg0, %arg1] : memref<3x2xf64> affine.store %3, %1[%arg0, %arg1] : memref<3x2xf64>
} }
} }
// Multiply and store into the output buffer. // Multiply and store into the output buffer.
affine.for %arg0 = 0 to 3 { affine.for %arg0 = 0 to 3 {
affine.for %arg1 = 0 to 2 { affine.for %arg1 = 0 to 2 {
%3 = affine.load %1[%arg0, %arg1] : memref<3x2xf64> %3 = affine.load %1[%arg0, %arg1] : memref<3x2xf64>
%4 = affine.load %1[%arg0, %arg1] : memref<3x2xf64> %4 = affine.load %1[%arg0, %arg1] : memref<3x2xf64>
%5 = mulf %3, %4 : f64 %5 = arith.mulf %3, %4 : f64
affine.store %5, %0[%arg0, %arg1] : memref<3x2xf64> affine.store %5, %0[%arg0, %arg1] : memref<3x2xf64>
} }
} }
// Print the value held by the buffer. // Print the value held by the buffer.
toy.print %0 : memref<3x2xf64> toy.print %0 : memref<3x2xf64>
dealloc %2 : memref<2x3xf64> memref.dealloc %2 : memref<2x3xf64>
dealloc %1 : memref<3x2xf64> memref.dealloc %1 : memref<3x2xf64>
dealloc %0 : memref<3x2xf64> memref.dealloc %0 : memref<3x2xf64>
return return
} }
``` ```
## Taking Advantage of Affine Optimization ## Taking Advantage of Affine Optimization
Our naive lowering is correct, but it leaves a lot to be desired with regards to Our naive lowering is correct, but it leaves a lot to be desired with regards to
efficiency. For example, the lowering of `toy.mul` has generated some redundant efficiency. For example, the lowering of `toy.mul` has generated some redundant
loads. Let's look at how adding a few existing optimizations to the pipeline can loads. Let's look at how adding a few existing optimizations to the pipeline can
help clean this up. Adding the `LoopFusion` and `MemRefDataFlowOpt` passes to help clean this up. Adding the `LoopFusion` and `MemRefDataFlowOpt` passes to
the pipeline gives the following result: the pipeline gives the following result:
```mlir ```mlir
func @main() { func @main() {
%cst = constant 1.000000e+00 : f64 %cst = arith.constant 1.000000e+00 : f64
%cst_0 = constant 2.000000e+00 : f64 %cst_0 = arith.constant 2.000000e+00 : f64
%cst_1 = constant 3.000000e+00 : f64 %cst_1 = arith.constant 3.000000e+00 : f64
%cst_2 = constant 4.000000e+00 : f64 %cst_2 = arith.constant 4.000000e+00 : f64
%cst_3 = constant 5.000000e+00 : f64 %cst_3 = arith.constant 5.000000e+00 : f64
%cst_4 = constant 6.000000e+00 : f64 %cst_4 = arith.constant 6.000000e+00 : f64
// Allocating buffers for the inputs and outputs. // Allocating buffers for the inputs and outputs.
%0 = alloc() : memref<3x2xf64> %0 = memref.alloc() : memref<3x2xf64>
%1 = alloc() : memref<2x3xf64> %1 = memref.alloc() : memref<2x3xf64>
// Initialize the input buffer with the constant values. // Initialize the input buffer with the constant values.
affine.store %cst, %1[0, 0] : memref<2x3xf64> affine.store %cst, %1[0, 0] : memref<2x3xf64>
affine.store %cst_0, %1[0, 1] : memref<2x3xf64> affine.store %cst_0, %1[0, 1] : memref<2x3xf64>
affine.store %cst_1, %1[0, 2] : memref<2x3xf64> affine.store %cst_1, %1[0, 2] : memref<2x3xf64>
affine.store %cst_2, %1[1, 0] : memref<2x3xf64> affine.store %cst_2, %1[1, 0] : memref<2x3xf64>
affine.store %cst_3, %1[1, 1] : memref<2x3xf64> affine.store %cst_3, %1[1, 1] : memref<2x3xf64>
affine.store %cst_4, %1[1, 2] : memref<2x3xf64> affine.store %cst_4, %1[1, 2] : memref<2x3xf64>
affine.for %arg0 = 0 to 3 { affine.for %arg0 = 0 to 3 {
affine.for %arg1 = 0 to 2 { affine.for %arg1 = 0 to 2 {
// Load the transpose value from the input buffer. // Load the transpose value from the input buffer.
%2 = affine.load %1[%arg1, %arg0] : memref<2x3xf64> %2 = affine.load %1[%arg1, %arg0] : memref<2x3xf64>
// Multiply and store into the output buffer. // Multiply and store into the output buffer.
%3 = mulf %2, %2 : f64 %3 = arith.mulf %2, %2 : f64
affine.store %3, %0[%arg0, %arg1] : memref<3x2xf64> affine.store %3, %0[%arg0, %arg1] : memref<3x2xf64>
} }
} }
// Print the value held by the buffer. // Print the value held by the buffer.
toy.print %0 : memref<3x2xf64> toy.print %0 : memref<3x2xf64>
dealloc %1 : memref<2x3xf64> memref.dealloc %1 : memref<2x3xf64>
dealloc %0 : memref<3x2xf64> memref.dealloc %0 : memref<3x2xf64>
return return
} }
``` ```
Here, we can see that a redundant allocation was removed, the two loop nests Here, we can see that a redundant allocation was removed, the two loop nests
were fused, and some unnecessary `load`s were removed. You can build `toyc-ch5` were fused, and some unnecessary `load`s were removed. You can build `toyc-ch5`
and try yourself: `toyc-ch5 test/Examples/Toy/Ch5/affine-lowering.mlir and try yourself: `toyc-ch5 test/Examples/Toy/Ch5/affine-lowering.mlir
-emit=mlir-affine`. We can also check our optimizations by adding `-opt`. -emit=mlir-affine`. We can also check our optimizations by adding `-opt`.
In this chapter we explored some aspects of partial lowering, with the intent to In this chapter we explored some aspects of partial lowering, with the intent to
optimize. In the [next chapter](Ch-6.md) we will continue the discussion about optimize. In the [next chapter](Ch-6.md) we will continue the discussion about
dialect conversion by targeting LLVM for code generation. dialect conversion by targeting LLVM for code generation.

mlir/docs/Tutorials/Toy/Ch-6.md

mlir/docs/includes/img/branch_example_post_move.svg

mlir/docs/includes/img/branch_example_pre_move.svg

mlir/docs/includes/img/nested_branch_example_post_move.svg

mlir/docs/includes/img/nested_branch_example_pre_move.svg

mlir/examples/standalone/standalone-opt/CMakeLists.txt

mlir/examples/standalone/standalone-opt/standalone-opt.cpp

mlir/examples/standalone/test/Standalone/dummy.mlir

mlir/examples/toy/Ch5/mlir/LowerToAffineLoops.cpp

mlir/examples/toy/Ch6/mlir/LowerToAffineLoops.cpp

mlir/examples/toy/Ch6/mlir/LowerToLLVM.cpp

mlir/examples/toy/Ch7/mlir/LowerToAffineLoops.cpp

mlir/examples/toy/Ch7/mlir/LowerToLLVM.cpp

mlir/include/mlir/Conversion/ArithmeticToLLVM/ArithmeticToLLVM.h

mlir/include/mlir/Conversion/ArithmeticToSPIRV/ArithmeticToSPIRV.h

mlir/include/mlir/Conversion/Passes.h

mlir/include/mlir/Conversion/Passes.td

mlir/include/mlir/Conversion/VectorToSCF/VectorToSCF.h

mlir/include/mlir/Dialect/Affine/IR/AffineOps.td

mlir/include/mlir/Dialect/Affine/Passes.td

mlir/include/mlir/Dialect/Arithmetic/CMakeLists.txt

mlir/include/mlir/Dialect/Arithmetic/IR/Arithmetic.h

mlir/include/mlir/Dialect/Arithmetic/IR/ArithmeticBase.td

mlir/include/mlir/Dialect/Arithmetic/IR/ArithmeticOps.td

mlir/include/mlir/Dialect/Arithmetic/Transforms/CMakeLists.txt

mlir/include/mlir/Dialect/Arithmetic/Transforms/Passes.h

mlir/include/mlir/Dialect/Arithmetic/Transforms/Passes.td

mlir/include/mlir/Dialect/ArmSVE/ArmSVE.td

mlir/include/mlir/Dialect/Async/Passes.td

mlir/include/mlir/Dialect/Complex/IR/Complex.h

mlir/include/mlir/Dialect/Complex/IR/ComplexBase.td

mlir/include/mlir/Dialect/GPU/GPUBase.td

mlir/include/mlir/Dialect/GPU/GPUDialect.h

mlir/include/mlir/Dialect/GPU/GPUOps.td

mlir/include/mlir/Dialect/Linalg/IR/LinalgBase.td

mlir/include/mlir/Dialect/Linalg/IR/LinalgStructuredOps.td

mlir/include/mlir/Dialect/Linalg/IR/LinalgTypes.h

mlir/include/mlir/Dialect/Linalg/Passes.td

mlir/include/mlir/Dialect/Linalg/Utils/Utils.h

mlir/include/mlir/Dialect/MemRef/IR/MemRef.h

mlir/include/mlir/Dialect/MemRef/IR/MemRefBase.td

mlir/include/mlir/Dialect/OpenMP/OpenMPOps.td

mlir/include/mlir/Dialect/SCF/Passes.td

mlir/include/mlir/Dialect/SCF/SCF.h

mlir/include/mlir/Dialect/SCF/SCFOps.td

mlir/include/mlir/Dialect/Shape/IR/Shape.h

mlir/include/mlir/Dialect/Shape/IR/ShapeBase.td

mlir/include/mlir/Dialect/SparseTensor/Transforms/Passes.td

mlir/include/mlir/Dialect/StandardOps/IR/Ops.h

mlir/include/mlir/Dialect/StandardOps/IR/Ops.td

mlir/include/mlir/Dialect/StandardOps/IR/StandardOpsBase.td

mlir/include/mlir/Dialect/StandardOps/Transforms/Passes.h

mlir/include/mlir/Dialect/StandardOps/Utils/Utils.h

mlir/include/mlir/Dialect/Tensor/IR/Tensor.h

mlir/include/mlir/Dialect/Tensor/IR/TensorBase.td

mlir/include/mlir/Dialect/Vector/VectorOps.td

mlir/include/mlir/Dialect/Vector/VectorTransforms.h

mlir/include/mlir/Dialect/X86Vector/X86Vector.td

mlir/include/mlir/IR/OpDefinition.h

mlir/include/mlir/InitAllDialects.h

mlir/include/mlir/InitAllPasses.h

mlir/include/mlir/Transforms/BufferUtils.h

mlir/include/mlir/Transforms/Passes.td

mlir/lib/Analysis/AffineAnalysis.cpp

Show All 11 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/AffineAnalysis.h" #include "mlir/Analysis/AffineAnalysis.h"
#include "mlir/Analysis/LoopAnalysis.h" #include "mlir/Analysis/LoopAnalysis.h"
#include "mlir/Analysis/SliceAnalysis.h" #include "mlir/Analysis/SliceAnalysis.h"
#include "mlir/Analysis/Utils.h" #include "mlir/Analysis/Utils.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Affine/IR/AffineValueMap.h" #include "mlir/Dialect/Affine/IR/AffineValueMap.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/AffineExprVisitor.h" #include "mlir/IR/AffineExprVisitor.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/IntegerSet.h" #include "mlir/IR/IntegerSet.h"
#include "mlir/Support/MathExtras.h" #include "mlir/Support/MathExtras.h"
#include "llvm/ADT/DenseMap.h" #include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/TypeSwitch.h" #include "llvm/ADT/TypeSwitch.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
Show All 20 Linesstatic Value getSupportedReduction(AffineForOp forOp, unsigned pos,
// Expected only one combiner operation. // Expected only one combiner operation.
if (combinerOps.size() > 1) if (combinerOps.size() > 1)
return nullptr; return nullptr;
Operation *combinerOp = combinerOps.back(); Operation *combinerOp = combinerOps.back();
Optional<AtomicRMWKind> maybeKind = Optional<AtomicRMWKind> maybeKind =
TypeSwitch<Operation *, Optional<AtomicRMWKind>>(combinerOp) TypeSwitch<Operation *, Optional<AtomicRMWKind>>(combinerOp)
.Case<AddFOp>([](Operation *) { return AtomicRMWKind::addf; }) .Case([](arith::AddFOp) { return AtomicRMWKind::addf; })
.Case<MulFOp>([](Operation *) { return AtomicRMWKind::mulf; }) .Case([](arith::MulFOp) { return AtomicRMWKind::mulf; })
.Case<AddIOp>([](Operation *) { return AtomicRMWKind::addi; }) .Case([](arith::AddIOp) { return AtomicRMWKind::addi; })
.Case<MulIOp>([](Operation *) { return AtomicRMWKind::muli; }) .Case([](arith::MulIOp) { return AtomicRMWKind::muli; })
.Default([](Operation *) -> Optional<AtomicRMWKind> { .Default([](Operation *) -> Optional<AtomicRMWKind> {
rriddleUnsubmitted
Done
ReplyInline Actions
TypeSwitch<Operation *, Optional<AtomicRMWKind>>(combinerOp)
- .Case<arith::AddFOp>([](Operation *) { return AtomicRMWKind::addf; })
- .Case<arith::MulFOp>([](Operation *) { return AtomicRMWKind::mulf; })
- .Case<arith::AddIOp>([](Operation *) { return AtomicRMWKind::addi; })
- .Case<arith::MulIOp>([](Operation *) { return AtomicRMWKind::muli; })
+ .Case([](arith::AddFOp) { return AtomicRMWKind::addf; })
+ .Case([](arith::MulFOp) { return AtomicRMWKind::mulf; })
+ .Case([](arith::AddIOp) { return AtomicRMWKind::addi; })
+ .Case([](arith::MulIOp) { return AtomicRMWKind::muli; })
.Default([](Operation *) -> Optional<AtomicRMWKind> {

For reference, you should be able to do this.

rriddle: For reference, you should be able to do this.
// TODO: AtomicRMW supports other kinds of reductions this is // TODO: AtomicRMW supports other kinds of reductions this is
// currently not detecting, add those when the need arises. // currently not detecting, add those when the need arises.
return llvm::None; return llvm::None;
}); });
if (!maybeKind) if (!maybeKind)
return nullptr; return nullptr;
kind = *maybeKind; kind = *maybeKind;
▲ Show 20 Lines • Show All 566 Lines • ▼ Show 20 LinesaddMemRefAccessConstraints(const AffineValueMap &srcAccessMap,
// Add equality constraints for any operands that are defined by constant ops. // Add equality constraints for any operands that are defined by constant ops.
auto addEqForConstOperands = [&](ArrayRef<Value> operands) { auto addEqForConstOperands = [&](ArrayRef<Value> operands) {
for (unsigned i = 0, e = operands.size(); i < e; ++i) { for (unsigned i = 0, e = operands.size(); i < e; ++i) {
if (isForInductionVar(operands[i])) if (isForInductionVar(operands[i]))
continue; continue;
auto symbol = operands[i]; auto symbol = operands[i];
assert(isValidSymbol(symbol)); assert(isValidSymbol(symbol));
// Check if the symbol is a constant. // Check if the symbol is a constant.
if (auto cOp = symbol.getDefiningOp<ConstantIndexOp>()) if (auto cOp = symbol.getDefiningOp<arith::ConstantIndexOp>())
dependenceDomain->addBound(FlatAffineConstraints::EQ, dependenceDomain->addBound(FlatAffineConstraints::EQ,
valuePosMap.getSymPos(symbol), valuePosMap.getSymPos(symbol), cOp.value());
cOp.getValue());
} }
}; };
// Add equality constraints for any src symbols defined by constant ops. // Add equality constraints for any src symbols defined by constant ops.
addEqForConstOperands(srcOperands); addEqForConstOperands(srcOperands);
// Add equality constraints for any dst symbols defined by constant ops. // Add equality constraints for any dst symbols defined by constant ops.
addEqForConstOperands(dstOperands); addEqForConstOperands(dstOperands);
▲ Show 20 Lines • Show All 448 Lines • Show Last 20 Lines

mlir/lib/Analysis/AffineStructures.cpp

mlir/lib/Analysis/CMakeLists.txt

Show All 30 Linesadd_mlir_library(MLIRAnalysis
AliasAnalysis/LocalAliasAnalysis.cpp AliasAnalysis/LocalAliasAnalysis.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Analysis ${MLIR_MAIN_INCLUDE_DIR}/mlir/Analysis
DEPENDS DEPENDS
mlir-headers mlir-headers
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRAffine
MLIRCallInterfaces MLIRCallInterfaces
MLIRControlFlowInterfaces MLIRControlFlowInterfaces
MLIRDataLayoutInterfaces MLIRDataLayoutInterfaces
MLIRInferTypeOpInterface MLIRInferTypeOpInterface
MLIRLinalg MLIRSideEffectInterfaces
MLIRSCF MLIRViewLikeInterface
rriddleUnsubmitted
Done
ReplyInline Actions

It'd be really nice to remove all dialect dependencies from this target.

rriddle: It'd be really nice to remove all dialect dependencies from this target.
) )
add_mlir_library(MLIRLoopAnalysis add_mlir_library(MLIRLoopAnalysis
AffineAnalysis.cpp AffineAnalysis.cpp
AffineStructures.cpp AffineStructures.cpp
LinearTransform.cpp LinearTransform.cpp
LoopAnalysis.cpp LoopAnalysis.cpp
NestedMatcher.cpp NestedMatcher.cpp
Show All 20 Lines

mlir/lib/Analysis/NumberOfExecutions.cpp

mlir/lib/Analysis/SliceAnalysis.cpp

mlir/lib/Analysis/Utils.cpp

mlir/lib/Conversion/AffineToStandard/AffineToStandard.cpp

mlir/lib/Conversion/AffineToStandard/CMakeLists.txt

mlir/lib/Conversion/ArithmeticToLLVM/ArithmeticToLLVM.cpp

mlir/lib/Conversion/ArithmeticToLLVM/CMakeLists.txt

mlir/lib/Conversion/ArithmeticToSPIRV/ArithmeticToSPIRV.cpp

  • This file was added.
//===- ArithmeticToSPIRV.cpp - Arithmetic to SPIRV dialect conversion -----===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
#include "mlir/Conversion/ArithmeticToSPIRV/ArithmeticToSPIRV.h"
#include "../PassDetail.h"
#include "../SPIRVCommon/Pattern.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h"
#include "mlir/Dialect/SPIRV/IR/SPIRVOps.h"
#include "mlir/Dialect/SPIRV/Transforms/SPIRVConversion.h"
#include "llvm/Support/Debug.h"
#define DEBUG_TYPE "arith-to-spirv-pattern"
using namespace mlir;
//===----------------------------------------------------------------------===//
// Operation Conversion
//===----------------------------------------------------------------------===//
namespace {
/// Converts composite arith.constant operation to spv.Constant.
struct ConstantCompositeOpPattern final
: public OpConversionPattern<arith::ConstantOp> {
using OpConversionPattern<arith::ConstantOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(arith::ConstantOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts scalar arith.constant operation to spv.Constant.
struct ConstantScalarOpPattern final
: public OpConversionPattern<arith::ConstantOp> {
using OpConversionPattern<arith::ConstantOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(arith::ConstantOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts arith.remsi to SPIR-V ops.
///
/// This cannot be merged into the template unary/binary pattern due to Vulkan
/// restrictions over spv.SRem and spv.SMod.
struct RemSIOpPattern final : public OpConversionPattern<arith::RemSIOp> {
using OpConversionPattern<arith::RemSIOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(arith::RemSIOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts bitwise operations to SPIR-V operations. This is a special pattern
/// other than the BinaryOpPatternPattern because if the operands are boolean
/// values, SPIR-V uses different operations (`SPIRVLogicalOp`). For
/// non-boolean operands, SPIR-V should use `SPIRVBitwiseOp`.
template <typename Op, typename SPIRVLogicalOp, typename SPIRVBitwiseOp>
struct BitwiseOpPattern final : public OpConversionPattern<Op> {
using OpConversionPattern<Op>::OpConversionPattern;
LogicalResult
matchAndRewrite(Op op, typename Op::Adaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts arith.xori to SPIR-V operations.
struct XOrIOpLogicalPattern final : public OpConversionPattern<arith::XOrIOp> {
using OpConversionPattern<arith::XOrIOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(arith::XOrIOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts arith.xori to SPIR-V operations if the type of source is i1 or
/// vector of i1.
struct XOrIOpBooleanPattern final : public OpConversionPattern<arith::XOrIOp> {
using OpConversionPattern<arith::XOrIOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(arith::XOrIOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts arith.uitofp to spv.Select if the type of source is i1 or vector of
/// i1.
struct UIToFPI1Pattern final : public OpConversionPattern<arith::UIToFPOp> {
using OpConversionPattern<arith::UIToFPOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(arith::UIToFPOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts arith.extui to spv.Select if the type of source is i1 or vector of
/// i1.
struct ExtUII1Pattern final : public OpConversionPattern<arith::ExtUIOp> {
using OpConversionPattern<arith::ExtUIOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(arith::ExtUIOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts arith.trunci to spv.Select if the type of result is i1 or vector of
/// i1.
struct TruncII1Pattern final : public OpConversionPattern<arith::TruncIOp> {
using OpConversionPattern<arith::TruncIOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(arith::TruncIOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts type-casting standard operations to SPIR-V operations.
template <typename Op, typename SPIRVOp>
struct TypeCastingOpPattern final : public OpConversionPattern<Op> {
using OpConversionPattern<Op>::OpConversionPattern;
LogicalResult
matchAndRewrite(Op op, typename Op::Adaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts integer compare operation on i1 type operands to SPIR-V ops.
class CmpIOpBooleanPattern final : public OpConversionPattern<arith::CmpIOp> {
public:
using OpConversionPattern<arith::CmpIOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(arith::CmpIOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts integer compare operation to SPIR-V ops.
class CmpIOpPattern final : public OpConversionPattern<arith::CmpIOp> {
public:
using OpConversionPattern<arith::CmpIOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(arith::CmpIOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts floating-point comparison operations to SPIR-V ops.
class CmpFOpPattern final : public OpConversionPattern<arith::CmpFOp> {
public:
using OpConversionPattern<arith::CmpFOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(arith::CmpFOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts floating point NaN check to SPIR-V ops. This pattern requires
/// Kernel capability.
class CmpFOpNanKernelPattern final : public OpConversionPattern<arith::CmpFOp> {
public:
using OpConversionPattern<arith::CmpFOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(arith::CmpFOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts floating point NaN check to SPIR-V ops. This pattern does not
/// require additional capability.
class CmpFOpNanNonePattern final : public OpConversionPattern<arith::CmpFOp> {
public:
using OpConversionPattern<arith::CmpFOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(arith::CmpFOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
} // end anonymous namespace
//===----------------------------------------------------------------------===//
// Conversion Helpers
//===----------------------------------------------------------------------===//
/// Converts the given `srcAttr` into a boolean attribute if it holds an
/// integral value. Returns null attribute if conversion fails.
static BoolAttr convertBoolAttr(Attribute srcAttr, Builder builder) {
if (auto boolAttr = srcAttr.dyn_cast<BoolAttr>())
return boolAttr;
if (auto intAttr = srcAttr.dyn_cast<IntegerAttr>())
return builder.getBoolAttr(intAttr.getValue().getBoolValue());
return BoolAttr();
}
/// Converts the given `srcAttr` to a new attribute of the given `dstType`.
/// Returns null attribute if conversion fails.
static IntegerAttr convertIntegerAttr(IntegerAttr srcAttr, IntegerType dstType,
Builder builder) {
// If the source number uses less active bits than the target bitwidth, then
// it should be safe to convert.
if (srcAttr.getValue().isIntN(dstType.getWidth()))
return builder.getIntegerAttr(dstType, srcAttr.getInt());
// XXX: Try again by interpreting the source number as a signed value.
// Although integers in the standard dialect are signless, they can represent
// a signed number. It's the operation decides how to interpret. This is
// dangerous, but it seems there is no good way of handling this if we still
// want to change the bitwidth. Emit a message at least.
if (srcAttr.getValue().isSignedIntN(dstType.getWidth())) {
auto dstAttr = builder.getIntegerAttr(dstType, srcAttr.getInt());
LLVM_DEBUG(llvm::dbgs() << "attribute '" << srcAttr << "' converted to '"
<< dstAttr << "' for type '" << dstType << "'\n");
return dstAttr;
}
LLVM_DEBUG(llvm::dbgs() << "attribute '" << srcAttr
<< "' illegal: cannot fit into target type '"
<< dstType << "'\n");
return IntegerAttr();
}
/// Converts the given `srcAttr` to a new attribute of the given `dstType`.
/// Returns null attribute if `dstType` is not 32-bit or conversion fails.
static FloatAttr convertFloatAttr(FloatAttr srcAttr, FloatType dstType,
Builder builder) {
// Only support converting to float for now.
if (!dstType.isF32())
return FloatAttr();
// Try to convert the source floating-point number to single precision.
APFloat dstVal = srcAttr.getValue();
bool losesInfo = false;
APFloat::opStatus status =
dstVal.convert(APFloat::IEEEsingle(), APFloat::rmTowardZero, &losesInfo);
if (status != APFloat::opOK || losesInfo) {
LLVM_DEBUG(llvm::dbgs()
<< srcAttr << " illegal: cannot fit into converted type '"
<< dstType << "'\n");
return FloatAttr();
}
return builder.getF32FloatAttr(dstVal.convertToFloat());
}
/// Returns true if the given `type` is a boolean scalar or vector type.
static bool isBoolScalarOrVector(Type type) {
if (type.isInteger(1))
return true;
if (auto vecType = type.dyn_cast<VectorType>())
return vecType.getElementType().isInteger(1);
return false;
}
//===----------------------------------------------------------------------===//
// ConstantOp with composite type
//===----------------------------------------------------------------------===//
LogicalResult ConstantCompositeOpPattern::matchAndRewrite(
arith::ConstantOp constOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
auto srcType = constOp.getType().dyn_cast<ShapedType>();
if (!srcType)
return failure();
// std.constant should only have vector or tenor types.
assert((srcType.isa<VectorType, RankedTensorType>()));
auto dstType = getTypeConverter()->convertType(srcType);
if (!dstType)
return failure();
auto dstElementsAttr = constOp.value().dyn_cast<DenseElementsAttr>();
ShapedType dstAttrType = dstElementsAttr.getType();
if (!dstElementsAttr)
return failure();
// If the composite type has more than one dimensions, perform linearization.
if (srcType.getRank() > 1) {
if (srcType.isa<RankedTensorType>()) {
dstAttrType = RankedTensorType::get(srcType.getNumElements(),
srcType.getElementType());
dstElementsAttr = dstElementsAttr.reshape(dstAttrType);
} else {
// TODO: add support for large vectors.
return failure();
}
}
Type srcElemType = srcType.getElementType();
Type dstElemType;
// Tensor types are converted to SPIR-V array types; vector types are
// converted to SPIR-V vector/array types.
if (auto arrayType = dstType.dyn_cast<spirv::ArrayType>())
dstElemType = arrayType.getElementType();
else
dstElemType = dstType.cast<VectorType>().getElementType();
// If the source and destination element types are different, perform
// attribute conversion.
if (srcElemType != dstElemType) {
SmallVector<Attribute, 8> elements;
if (srcElemType.isa<FloatType>()) {
for (FloatAttr srcAttr : dstElementsAttr.getValues<FloatAttr>()) {
FloatAttr dstAttr =
convertFloatAttr(srcAttr, dstElemType.cast<FloatType>(), rewriter);
if (!dstAttr)
return failure();
elements.push_back(dstAttr);
}
} else if (srcElemType.isInteger(1)) {
return failure();
} else {
for (IntegerAttr srcAttr : dstElementsAttr.getValues<IntegerAttr>()) {
IntegerAttr dstAttr = convertIntegerAttr(
srcAttr, dstElemType.cast<IntegerType>(), rewriter);
if (!dstAttr)
return failure();
elements.push_back(dstAttr);
}
}
// Unfortunately, we cannot use dialect-specific types for element
// attributes; element attributes only works with builtin types. So we need
// to prepare another converted builtin types for the destination elements
// attribute.
if (dstAttrType.isa<RankedTensorType>())
dstAttrType = RankedTensorType::get(dstAttrType.getShape(), dstElemType);
else
dstAttrType = VectorType::get(dstAttrType.getShape(), dstElemType);
dstElementsAttr = DenseElementsAttr::get(dstAttrType, elements);
}
rewriter.replaceOpWithNewOp<spirv::ConstantOp>(constOp, dstType,
dstElementsAttr);
return success();
}
//===----------------------------------------------------------------------===//
// ConstantOp with scalar type
//===----------------------------------------------------------------------===//
LogicalResult ConstantScalarOpPattern::matchAndRewrite(
arith::ConstantOp constOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
Type srcType = constOp.getType();
if (!srcType.isIntOrIndexOrFloat())
return failure();
Type dstType = getTypeConverter()->convertType(srcType);
if (!dstType)
return failure();
// Floating-point types.
if (srcType.isa<FloatType>()) {
auto srcAttr = constOp.value().cast<FloatAttr>();
auto dstAttr = srcAttr;
// Floating-point types not supported in the target environment are all
// converted to float type.
if (srcType != dstType) {
dstAttr = convertFloatAttr(srcAttr, dstType.cast<FloatType>(), rewriter);
if (!dstAttr)
return failure();
}
rewriter.replaceOpWithNewOp<spirv::ConstantOp>(constOp, dstType, dstAttr);
return success();
}
// Bool type.
if (srcType.isInteger(1)) {
// std.constant can use 0/1 instead of true/false for i1 values. We need to
// handle that here.
auto dstAttr = convertBoolAttr(constOp.value(), rewriter);
if (!dstAttr)
return failure();
rewriter.replaceOpWithNewOp<spirv::ConstantOp>(constOp, dstType, dstAttr);
return success();
}
// IndexType or IntegerType. Index values are converted to 32-bit integer
// values when converting to SPIR-V.
auto srcAttr = constOp.value().cast<IntegerAttr>();
auto dstAttr =
convertIntegerAttr(srcAttr, dstType.cast<IntegerType>(), rewriter);
if (!dstAttr)
return failure();
rewriter.replaceOpWithNewOp<spirv::ConstantOp>(constOp, dstType, dstAttr);
return success();
}
//===----------------------------------------------------------------------===//
// RemSIOpPattern
//===----------------------------------------------------------------------===//
/// Returns signed remainder for `lhs` and `rhs` and lets the result follow
/// the sign of `signOperand`.
///
/// Note that this is needed for Vulkan. Per the Vulkan's SPIR-V environment
/// spec, "for the OpSRem and OpSMod instructions, if either operand is negative
/// the result is undefined." So we cannot directly use spv.SRem/spv.SMod
/// if either operand can be negative. Emulate it via spv.UMod.
static Value emulateSignedRemainder(Location loc, Value lhs, Value rhs,
Value signOperand, OpBuilder &builder) {
assert(lhs.getType() == rhs.getType());
assert(lhs == signOperand || rhs == signOperand);
Type type = lhs.getType();
// Calculate the remainder with spv.UMod.
Value lhsAbs = builder.create<spirv::GLSLSAbsOp>(loc, type, lhs);
Value rhsAbs = builder.create<spirv::GLSLSAbsOp>(loc, type, rhs);
Value abs = builder.create<spirv::UModOp>(loc, lhsAbs, rhsAbs);
// Fix the sign.
Value isPositive;
if (lhs == signOperand)
isPositive = builder.create<spirv::IEqualOp>(loc, lhs, lhsAbs);
else
isPositive = builder.create<spirv::IEqualOp>(loc, rhs, rhsAbs);
Value absNegate = builder.create<spirv::SNegateOp>(loc, type, abs);
return builder.create<spirv::SelectOp>(loc, type, isPositive, abs, absNegate);
}
LogicalResult
RemSIOpPattern::matchAndRewrite(arith::RemSIOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
Value result = emulateSignedRemainder(op.getLoc(), adaptor.getOperands()[0],
adaptor.getOperands()[1],
adaptor.getOperands()[0], rewriter);
rewriter.replaceOp(op, result);
return success();
}
//===----------------------------------------------------------------------===//
// BitwiseOpPattern
//===----------------------------------------------------------------------===//
template <typename Op, typename SPIRVLogicalOp, typename SPIRVBitwiseOp>
LogicalResult
BitwiseOpPattern<Op, SPIRVLogicalOp, SPIRVBitwiseOp>::matchAndRewrite(
Op op, typename Op::Adaptor adaptor,
ConversionPatternRewriter &rewriter) const {
assert(adaptor.getOperands().size() == 2);
auto dstType =
this->getTypeConverter()->convertType(op.getResult().getType());
if (!dstType)
return failure();
if (isBoolScalarOrVector(adaptor.getOperands().front().getType())) {
rewriter.template replaceOpWithNewOp<SPIRVLogicalOp>(op, dstType,
adaptor.getOperands());
} else {
rewriter.template replaceOpWithNewOp<SPIRVBitwiseOp>(op, dstType,
adaptor.getOperands());
}
return success();
}
//===----------------------------------------------------------------------===//
// XOrIOpLogicalPattern
//===----------------------------------------------------------------------===//
LogicalResult XOrIOpLogicalPattern::matchAndRewrite(
arith::XOrIOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
assert(adaptor.getOperands().size() == 2);
if (isBoolScalarOrVector(adaptor.getOperands().front().getType()))
return failure();
auto dstType = getTypeConverter()->convertType(op.getType());
if (!dstType)
return failure();
rewriter.replaceOpWithNewOp<spirv::BitwiseXorOp>(op, dstType,
adaptor.getOperands());
return success();
}
//===----------------------------------------------------------------------===//
// XOrIOpBooleanPattern
//===----------------------------------------------------------------------===//
LogicalResult XOrIOpBooleanPattern::matchAndRewrite(
arith::XOrIOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
assert(adaptor.getOperands().size() == 2);
if (!isBoolScalarOrVector(adaptor.getOperands().front().getType()))
return failure();
auto dstType = getTypeConverter()->convertType(op.getType());
if (!dstType)
return failure();
rewriter.replaceOpWithNewOp<spirv::LogicalNotEqualOp>(op, dstType,
adaptor.getOperands());
return success();
}
//===----------------------------------------------------------------------===//
// UIToFPI1Pattern
//===----------------------------------------------------------------------===//
LogicalResult
UIToFPI1Pattern::matchAndRewrite(arith::UIToFPOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
auto srcType = adaptor.getOperands().front().getType();
if (!isBoolScalarOrVector(srcType))
return failure();
auto dstType =
this->getTypeConverter()->convertType(op.getResult().getType());
Location loc = op.getLoc();
Value zero = spirv::ConstantOp::getZero(dstType, loc, rewriter);
Value one = spirv::ConstantOp::getOne(dstType, loc, rewriter);
rewriter.template replaceOpWithNewOp<spirv::SelectOp>(
op, dstType, adaptor.getOperands().front(), one, zero);
return success();
}
//===----------------------------------------------------------------------===//
// ExtUII1Pattern
//===----------------------------------------------------------------------===//
LogicalResult
ExtUII1Pattern::matchAndRewrite(arith::ExtUIOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
auto srcType = adaptor.getOperands().front().getType();
if (!isBoolScalarOrVector(srcType))
return failure();
auto dstType =
this->getTypeConverter()->convertType(op.getResult().getType());
Location loc = op.getLoc();
Value zero = spirv::ConstantOp::getZero(dstType, loc, rewriter);
Value one = spirv::ConstantOp::getOne(dstType, loc, rewriter);
rewriter.template replaceOpWithNewOp<spirv::SelectOp>(
op, dstType, adaptor.getOperands().front(), one, zero);
return success();
}
//===----------------------------------------------------------------------===//
// TruncII1Pattern
//===----------------------------------------------------------------------===//
LogicalResult
TruncII1Pattern::matchAndRewrite(arith::TruncIOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
auto dstType =
this->getTypeConverter()->convertType(op.getResult().getType());
if (!isBoolScalarOrVector(dstType))
return failure();
Location loc = op.getLoc();
auto srcType = adaptor.getOperands().front().getType();
// Check if (x & 1) == 1.
Value mask = spirv::ConstantOp::getOne(srcType, loc, rewriter);
Value maskedSrc = rewriter.create<spirv::BitwiseAndOp>(
loc, srcType, adaptor.getOperands()[0], mask);
Value isOne = rewriter.create<spirv::IEqualOp>(loc, maskedSrc, mask);
Value zero = spirv::ConstantOp::getZero(dstType, loc, rewriter);
Value one = spirv::ConstantOp::getOne(dstType, loc, rewriter);
rewriter.replaceOpWithNewOp<spirv::SelectOp>(op, dstType, isOne, one, zero);
return success();
}
//===----------------------------------------------------------------------===//
// TypeCastingOpPattern
//===----------------------------------------------------------------------===//
template <typename Op, typename SPIRVOp>
LogicalResult TypeCastingOpPattern<Op, SPIRVOp>::matchAndRewrite(
Op op, typename Op::Adaptor adaptor,
ConversionPatternRewriter &rewriter) const {
assert(adaptor.getOperands().size() == 1);
auto srcType = adaptor.getOperands().front().getType();
auto dstType =
this->getTypeConverter()->convertType(op.getResult().getType());
if (isBoolScalarOrVector(srcType) || isBoolScalarOrVector(dstType))
return failure();
if (dstType == srcType) {
// Due to type conversion, we are seeing the same source and target type.
// Then we can just erase this operation by forwarding its operand.
rewriter.replaceOp(op, adaptor.getOperands().front());
} else {
rewriter.template replaceOpWithNewOp<SPIRVOp>(op, dstType,
adaptor.getOperands());
}
return success();
}
//===----------------------------------------------------------------------===//
// CmpIOpBooleanPattern
//===----------------------------------------------------------------------===//
LogicalResult CmpIOpBooleanPattern::matchAndRewrite(
arith::CmpIOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
Type operandType = op.lhs().getType();
if (!isBoolScalarOrVector(operandType))
return failure();
switch (op.getPredicate()) {
#define DISPATCH(cmpPredicate, spirvOp) \
case cmpPredicate: \
rewriter.replaceOpWithNewOp<spirvOp>(op, op.getResult().getType(), \
adaptor.lhs(), adaptor.rhs()); \
return success();
DISPATCH(arith::CmpIPredicate::eq, spirv::LogicalEqualOp);
DISPATCH(arith::CmpIPredicate::ne, spirv::LogicalNotEqualOp);
#undef DISPATCH
default:;
}
return failure();
}
//===----------------------------------------------------------------------===//
// CmpIOpPattern
//===----------------------------------------------------------------------===//
LogicalResult
CmpIOpPattern::matchAndRewrite(arith::CmpIOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
Type operandType = op.lhs().getType();
if (isBoolScalarOrVector(operandType))
return failure();
switch (op.getPredicate()) {
#define DISPATCH(cmpPredicate, spirvOp) \
case cmpPredicate: \
if (spirvOp::template hasTrait<OpTrait::spirv::UnsignedOp>() && \
operandType != this->getTypeConverter()->convertType(operandType)) { \
return op.emitError( \
"bitwidth emulation is not implemented yet on unsigned op"); \
} \
rewriter.replaceOpWithNewOp<spirvOp>(op, op.getResult().getType(), \
adaptor.lhs(), adaptor.rhs()); \
return success();
DISPATCH(arith::CmpIPredicate::eq, spirv::IEqualOp);
DISPATCH(arith::CmpIPredicate::ne, spirv::INotEqualOp);
DISPATCH(arith::CmpIPredicate::slt, spirv::SLessThanOp);
DISPATCH(arith::CmpIPredicate::sle, spirv::SLessThanEqualOp);
DISPATCH(arith::CmpIPredicate::sgt, spirv::SGreaterThanOp);
DISPATCH(arith::CmpIPredicate::sge, spirv::SGreaterThanEqualOp);
DISPATCH(arith::CmpIPredicate::ult, spirv::ULessThanOp);
DISPATCH(arith::CmpIPredicate::ule, spirv::ULessThanEqualOp);
DISPATCH(arith::CmpIPredicate::ugt, spirv::UGreaterThanOp);
DISPATCH(arith::CmpIPredicate::uge, spirv::UGreaterThanEqualOp);
#undef DISPATCH
}
return failure();
}
//===----------------------------------------------------------------------===//
// CmpFOpPattern
//===----------------------------------------------------------------------===//
LogicalResult
CmpFOpPattern::matchAndRewrite(arith::CmpFOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
switch (op.getPredicate()) {
#define DISPATCH(cmpPredicate, spirvOp) \
case cmpPredicate: \
rewriter.replaceOpWithNewOp<spirvOp>(op, op.getResult().getType(), \
adaptor.lhs(), adaptor.rhs()); \
return success();
// Ordered.
DISPATCH(arith::CmpFPredicate::OEQ, spirv::FOrdEqualOp);
DISPATCH(arith::CmpFPredicate::OGT, spirv::FOrdGreaterThanOp);
DISPATCH(arith::CmpFPredicate::OGE, spirv::FOrdGreaterThanEqualOp);
DISPATCH(arith::CmpFPredicate::OLT, spirv::FOrdLessThanOp);
DISPATCH(arith::CmpFPredicate::OLE, spirv::FOrdLessThanEqualOp);
DISPATCH(arith::CmpFPredicate::ONE, spirv::FOrdNotEqualOp);
// Unordered.
DISPATCH(arith::CmpFPredicate::UEQ, spirv::FUnordEqualOp);
DISPATCH(arith::CmpFPredicate::UGT, spirv::FUnordGreaterThanOp);
DISPATCH(arith::CmpFPredicate::UGE, spirv::FUnordGreaterThanEqualOp);
DISPATCH(arith::CmpFPredicate::ULT, spirv::FUnordLessThanOp);
DISPATCH(arith::CmpFPredicate::ULE, spirv::FUnordLessThanEqualOp);
DISPATCH(arith::CmpFPredicate::UNE, spirv::FUnordNotEqualOp);
#undef DISPATCH
default:
break;
}
return failure();
}
//===----------------------------------------------------------------------===//
// CmpFOpNanKernelPattern
//===----------------------------------------------------------------------===//
LogicalResult CmpFOpNanKernelPattern::matchAndRewrite(
arith::CmpFOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
if (op.getPredicate() == arith::CmpFPredicate::ORD) {
rewriter.replaceOpWithNewOp<spirv::OrderedOp>(op, adaptor.lhs(),
adaptor.rhs());
return success();
}
if (op.getPredicate() == arith::CmpFPredicate::UNO) {
rewriter.replaceOpWithNewOp<spirv::UnorderedOp>(op, adaptor.lhs(),
adaptor.rhs());
return success();
}
return failure();
}
//===----------------------------------------------------------------------===//
// CmpFOpNanNonePattern
//===----------------------------------------------------------------------===//
LogicalResult CmpFOpNanNonePattern::matchAndRewrite(
arith::CmpFOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
if (op.getPredicate() != arith::CmpFPredicate::ORD &&
op.getPredicate() != arith::CmpFPredicate::UNO)
return failure();
Location loc = op.getLoc();
Value lhsIsNan = rewriter.create<spirv::IsNanOp>(loc, adaptor.lhs());
Value rhsIsNan = rewriter.create<spirv::IsNanOp>(loc, adaptor.rhs());
Value replace = rewriter.create<spirv::LogicalOrOp>(loc, lhsIsNan, rhsIsNan);
if (op.getPredicate() == arith::CmpFPredicate::ORD)
replace = rewriter.create<spirv::LogicalNotOp>(loc, replace);
rewriter.replaceOp(op, replace);
return success();
}
//===----------------------------------------------------------------------===//
// Pattern Population
//===----------------------------------------------------------------------===//
void mlir::arith::populateArithmeticToSPIRVPatterns(
SPIRVTypeConverter &typeConverter, RewritePatternSet &patterns) {
rriddleUnsubmitted
Done
ReplyInline Actions
//===----------------------------------------------------------------------===//
- namespace mlir {
- void populateArithmeticToSPIRVPatterns(SPIRVTypeConverter &typeConverter,
- RewritePatternSet &patterns) {
+ void mlir::populateArithmeticToSPIRVPatterns(SPIRVTypeConverter &typeConverter,
+ RewritePatternSet &patterns) {
// clang-format off

Functions should be fully namespace resolved and at the top-level.

rriddle: Functions should be fully namespace resolved and at the top-level.
// clang-format off
patterns.add<
ConstantCompositeOpPattern,
ConstantScalarOpPattern,
spirv::UnaryAndBinaryOpPattern<arith::AddIOp, spirv::IAddOp>,
spirv::UnaryAndBinaryOpPattern<arith::SubIOp, spirv::ISubOp>,
spirv::UnaryAndBinaryOpPattern<arith::MulIOp, spirv::IMulOp>,
spirv::UnaryAndBinaryOpPattern<arith::DivUIOp, spirv::UDivOp>,
spirv::UnaryAndBinaryOpPattern<arith::DivSIOp, spirv::SDivOp>,
spirv::UnaryAndBinaryOpPattern<arith::RemUIOp, spirv::UModOp>,
RemSIOpPattern,
BitwiseOpPattern<arith::AndIOp, spirv::LogicalAndOp, spirv::BitwiseAndOp>,
BitwiseOpPattern<arith::OrIOp, spirv::LogicalOrOp, spirv::BitwiseOrOp>,
XOrIOpLogicalPattern, XOrIOpBooleanPattern,
spirv::UnaryAndBinaryOpPattern<arith::ShLIOp, spirv::ShiftLeftLogicalOp>,
spirv::UnaryAndBinaryOpPattern<arith::ShRUIOp, spirv::ShiftRightLogicalOp>,
spirv::UnaryAndBinaryOpPattern<arith::ShRSIOp, spirv::ShiftRightArithmeticOp>,
spirv::UnaryAndBinaryOpPattern<arith::NegFOp, spirv::FNegateOp>,
spirv::UnaryAndBinaryOpPattern<arith::AddFOp, spirv::FAddOp>,
spirv::UnaryAndBinaryOpPattern<arith::SubFOp, spirv::FSubOp>,
spirv::UnaryAndBinaryOpPattern<arith::MulFOp, spirv::FMulOp>,
spirv::UnaryAndBinaryOpPattern<arith::DivFOp, spirv::FDivOp>,
spirv::UnaryAndBinaryOpPattern<arith::RemFOp, spirv::FRemOp>,
TypeCastingOpPattern<arith::ExtUIOp, spirv::UConvertOp>, ExtUII1Pattern,
TypeCastingOpPattern<arith::ExtSIOp, spirv::SConvertOp>,
TypeCastingOpPattern<arith::ExtFOp, spirv::FConvertOp>,
TypeCastingOpPattern<arith::TruncIOp, spirv::SConvertOp>, TruncII1Pattern,
TypeCastingOpPattern<arith::TruncFOp, spirv::FConvertOp>,
TypeCastingOpPattern<arith::UIToFPOp, spirv::ConvertUToFOp>, UIToFPI1Pattern,
TypeCastingOpPattern<arith::SIToFPOp, spirv::ConvertSToFOp>,
TypeCastingOpPattern<arith::FPToSIOp, spirv::ConvertFToSOp>,
TypeCastingOpPattern<arith::IndexCastOp, spirv::SConvertOp>,
CmpIOpBooleanPattern, CmpIOpPattern,
CmpFOpNanNonePattern, CmpFOpPattern
>(typeConverter, patterns.getContext());
// clang-format on
// Give CmpFOpNanKernelPattern a higher benefit so it can prevail when Kernel
// capability is available.
patterns.add<CmpFOpNanKernelPattern>(typeConverter, patterns.getContext(),
/*benefit=*/2);
}
//===----------------------------------------------------------------------===//
// Pass Definition
//===----------------------------------------------------------------------===//
namespace {
struct ConvertArithmeticToSPIRVPass
: public ConvertArithmeticToSPIRVBase<ConvertArithmeticToSPIRVPass> {
void runOnFunction() override {
auto module = getOperation()->getParentOfType<ModuleOp>();
auto targetAttr = spirv::lookupTargetEnvOrDefault(module);
auto target = SPIRVConversionTarget::get(targetAttr);
SPIRVTypeConverter::Options options;
options.emulateNon32BitScalarTypes = this->emulateNon32BitScalarTypes;
SPIRVTypeConverter typeConverter(targetAttr, options);
RewritePatternSet patterns(&getContext());
mlir::arith::populateArithmeticToSPIRVPatterns(typeConverter, patterns);
if (failed(applyPartialConversion(getOperation(), *target,
std::move(patterns))))
signalPassFailure();
}
};
} // end anonymous namespace
std::unique_ptr<Pass> mlir::arith::createConvertArithmeticToSPIRVPass() {
return std::make_unique<ConvertArithmeticToSPIRVPass>();
}

mlir/lib/Conversion/ArithmeticToSPIRV/CMakeLists.txt

mlir/lib/Conversion/AsyncToLLVM/AsyncToLLVM.cpp

mlir/lib/Conversion/AsyncToLLVM/CMakeLists.txt

mlir/lib/Conversion/CMakeLists.txt

mlir/lib/Conversion/ComplexToLLVM/ComplexToLLVM.cpp

mlir/lib/Conversion/ComplexToStandard/CMakeLists.txt

mlir/lib/Conversion/ComplexToStandard/ComplexToStandard.cpp

mlir/lib/Conversion/GPUCommon/CMakeLists.txt

mlir/lib/Conversion/GPUCommon/GPUToLLVMConversion.cpp

mlir/lib/Conversion/GPUToNVVM/CMakeLists.txt

mlir/lib/Conversion/GPUToNVVM/LowerGpuOpsToNVVMOps.cpp

mlir/lib/Conversion/GPUToROCDL/CMakeLists.txt

mlir/lib/Conversion/GPUToROCDL/LowerGpuOpsToROCDLOps.cpp

mlir/lib/Conversion/GPUToSPIRV/CMakeLists.txt

mlir/lib/Conversion/GPUToSPIRV/GPUToSPIRVPass.cpp

mlir/lib/Conversion/LinalgToStandard/LinalgToStandard.cpp

mlir/lib/Conversion/MathToLLVM/MathToLLVM.cpp

mlir/lib/Conversion/MathToLibm/CMakeLists.txt

mlir/lib/Conversion/MathToLibm/MathToLibm.cpp

mlir/lib/Conversion/MathToSPIRV/MathToSPIRV.cpp

mlir/lib/Conversion/OpenACCToSCF/CMakeLists.txt

mlir/lib/Conversion/OpenACCToSCF/OpenACCToSCF.cpp

mlir/lib/Conversion/OpenMPToLLVM/CMakeLists.txt

mlir/lib/Conversion/OpenMPToLLVM/OpenMPToLLVM.cpp

mlir/lib/Conversion/PassDetail.h

mlir/lib/Conversion/SCFToGPU/CMakeLists.txt

mlir/lib/Conversion/SCFToGPU/SCFToGPU.cpp

mlir/lib/Conversion/SCFToGPU/SCFToGPUPass.cpp

mlir/lib/Conversion/SCFToOpenMP/CMakeLists.txt

mlir/lib/Conversion/SCFToOpenMP/SCFToOpenMP.cpp

mlir/lib/Conversion/SCFToSPIRV/CMakeLists.txt

mlir/lib/Conversion/SCFToSPIRV/SCFToSPIRVPass.cpp

mlir/lib/Conversion/SCFToStandard/CMakeLists.txt

mlir/lib/Conversion/SCFToStandard/SCFToStandard.cpp

mlir/lib/Conversion/SPIRVCommon/Pattern.h

mlir/lib/Conversion/SPIRVToLLVM/CMakeLists.txt

mlir/lib/Conversion/SPIRVToLLVM/ConvertLaunchFuncToLLVMCalls.cpp

mlir/lib/Conversion/ShapeToStandard/CMakeLists.txt

mlir/lib/Conversion/ShapeToStandard/ShapeToStandard.cpp

mlir/lib/Conversion/StandardToLLVM/CMakeLists.txt

mlir/lib/Conversion/StandardToLLVM/StandardToLLVM.cpp

mlir/lib/Conversion/StandardToSPIRV/CMakeLists.txt

mlir/lib/Conversion/StandardToSPIRV/StandardToSPIRV.cpp

//===- StandardToSPIRV.cpp - Standard to SPIR-V Patterns ------------------===// //===- StandardToSPIRV.cpp - Standard to SPIR-V Patterns ------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements patterns to convert standard dialect to SPIR-V dialect. // This file implements patterns to convert standard dialect to SPIR-V dialect.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "../SPIRVCommon/Pattern.h"
rriddleUnsubmitted
Done
ReplyInline Actions

In general, if you want to share pieces of code solely within the lib, you should place the include in the lib/ folder. include/ should *only* contain things that are meant to be outwardly exposed via the API, of which implementation utilities generally aren't. GPUCommon is an example: https://github.com/llvm/llvm-project/tree/main/mlir/lib/Conversion/GPUCommon

rriddle: In general, if you want to share pieces of code solely within the lib, you should place the…
#include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h" #include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h"
#include "mlir/Dialect/SPIRV/IR/SPIRVOps.h" #include "mlir/Dialect/SPIRV/IR/SPIRVOps.h"
#include "mlir/Dialect/SPIRV/Transforms/SPIRVConversion.h" #include "mlir/Dialect/SPIRV/Transforms/SPIRVConversion.h"
#include "mlir/Dialect/SPIRV/Utils/LayoutUtils.h" #include "mlir/Dialect/SPIRV/Utils/LayoutUtils.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/IR/AffineMap.h" #include "mlir/IR/AffineMap.h"
#include "mlir/Support/LogicalResult.h" #include "mlir/Support/LogicalResult.h"
#include "llvm/ADT/SetVector.h" #include "llvm/ADT/SetVector.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#define DEBUG_TYPE "std-to-spirv-pattern" #define DEBUG_TYPE "std-to-spirv-pattern"
using namespace mlir; using namespace mlir;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Utility functions // Utility functions
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
/// Returns true if the given `type` is a boolean scalar or vector type.
static bool isBoolScalarOrVector(Type type) {
if (type.isInteger(1))
return true;
if (auto vecType = type.dyn_cast<VectorType>())
return vecType.getElementType().isInteger(1);
return false;
}
/// Converts the given `srcAttr` into a boolean attribute if it holds an /// Converts the given `srcAttr` into a boolean attribute if it holds an
/// integral value. Returns null attribute if conversion fails. /// integral value. Returns null attribute if conversion fails.
static BoolAttr convertBoolAttr(Attribute srcAttr, Builder builder) { static BoolAttr convertBoolAttr(Attribute srcAttr, Builder builder) {
if (auto boolAttr = srcAttr.dyn_cast<BoolAttr>()) if (auto boolAttr = srcAttr.dyn_cast<BoolAttr>())
return boolAttr; return boolAttr;
if (auto intAttr = srcAttr.dyn_cast<IntegerAttr>()) if (auto intAttr = srcAttr.dyn_cast<IntegerAttr>())
return builder.getBoolAttr(intAttr.getValue().getBoolValue()); return builder.getBoolAttr(intAttr.getValue().getBoolValue());
return BoolAttr(); return BoolAttr();
▲ Show 20 Lines • Show All 44 Lines • ▼ Show 20 Lines LLVM_DEBUG(llvm::dbgs()
<< srcAttr << " illegal: cannot fit into converted type '" << srcAttr << " illegal: cannot fit into converted type '"
<< dstType << "'\n"); << dstType << "'\n");
return FloatAttr(); return FloatAttr();
} }
return builder.getF32FloatAttr(dstVal.convertToFloat()); return builder.getF32FloatAttr(dstVal.convertToFloat());
} }
/// Returns signed remainder for `lhs` and `rhs` and lets the result follow
/// the sign of `signOperand`.
///
/// Note that this is needed for Vulkan. Per the Vulkan's SPIR-V environment
/// spec, "for the OpSRem and OpSMod instructions, if either operand is negative
/// the result is undefined." So we cannot directly use spv.SRem/spv.SMod
/// if either operand can be negative. Emulate it via spv.UMod.
static Value emulateSignedRemainder(Location loc, Value lhs, Value rhs,
Value signOperand, OpBuilder &builder) {
assert(lhs.getType() == rhs.getType());
assert(lhs == signOperand || rhs == signOperand);
Type type = lhs.getType();
// Calculate the remainder with spv.UMod.
Value lhsAbs = builder.create<spirv::GLSLSAbsOp>(loc, type, lhs);
Value rhsAbs = builder.create<spirv::GLSLSAbsOp>(loc, type, rhs);
Value abs = builder.create<spirv::UModOp>(loc, lhsAbs, rhsAbs);
// Fix the sign.
Value isPositive;
if (lhs == signOperand)
isPositive = builder.create<spirv::IEqualOp>(loc, lhs, lhsAbs);
else
isPositive = builder.create<spirv::IEqualOp>(loc, rhs, rhsAbs);
Value absNegate = builder.create<spirv::SNegateOp>(loc, type, abs);
return builder.create<spirv::SelectOp>(loc, type, isPositive, abs, absNegate);
}
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Operation conversion // Operation conversion
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Note that DRR cannot be used for the patterns in this file: we may need to // Note that DRR cannot be used for the patterns in this file: we may need to
// convert type along the way, which requires ConversionPattern. DRR generates // convert type along the way, which requires ConversionPattern. DRR generates
// normal RewritePattern. // normal RewritePattern.
namespace { namespace {
/// Converts unary and binary standard operations to SPIR-V operations.
template <typename StdOp, typename SPIRVOp>
class UnaryAndBinaryOpPattern final : public OpConversionPattern<StdOp> {
public:
using OpConversionPattern<StdOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(StdOp operation, typename StdOp::Adaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
assert(adaptor.getOperands().size() <= 2);
auto dstType = this->getTypeConverter()->convertType(operation.getType());
if (!dstType)
return failure();
if (SPIRVOp::template hasTrait<OpTrait::spirv::UnsignedOp>() &&
dstType != operation.getType()) {
return operation.emitError(
"bitwidth emulation is not implemented yet on unsigned op");
}
rewriter.template replaceOpWithNewOp<SPIRVOp>(operation, dstType,
adaptor.getOperands());
return success();
}
};
/// Converts std.remi_signed to SPIR-V ops.
///
/// This cannot be merged into the template unary/binary pattern due to
/// Vulkan restrictions over spv.SRem and spv.SMod.
class SignedRemIOpPattern final : public OpConversionPattern<SignedRemIOp> {
public:
using OpConversionPattern<SignedRemIOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(SignedRemIOp remOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts bitwise standard operations to SPIR-V operations. This is a special
/// pattern other than the BinaryOpPatternPattern because if the operands are
/// boolean values, SPIR-V uses different operations (`SPIRVLogicalOp`). For
/// non-boolean operands, SPIR-V should use `SPIRVBitwiseOp`.
template <typename StdOp, typename SPIRVLogicalOp, typename SPIRVBitwiseOp>
class BitwiseOpPattern final : public OpConversionPattern<StdOp> {
public:
using OpConversionPattern<StdOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(StdOp operation, typename StdOp::Adaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
assert(adaptor.getOperands().size() == 2);
auto dstType =
this->getTypeConverter()->convertType(operation.getResult().getType());
if (!dstType)
return failure();
if (isBoolScalarOrVector(adaptor.getOperands().front().getType())) {
rewriter.template replaceOpWithNewOp<SPIRVLogicalOp>(
operation, dstType, adaptor.getOperands());
} else {
rewriter.template replaceOpWithNewOp<SPIRVBitwiseOp>(
operation, dstType, adaptor.getOperands());
}
return success();
}
};
/// Converts composite std.constant operation to spv.Constant. /// Converts composite std.constant operation to spv.Constant.
class ConstantCompositeOpPattern final class ConstantCompositeOpPattern final
: public OpConversionPattern<ConstantOp> { : public OpConversionPattern<ConstantOp> {
public: public:
using OpConversionPattern<ConstantOp>::OpConversionPattern; using OpConversionPattern<ConstantOp>::OpConversionPattern;
LogicalResult LogicalResult
matchAndRewrite(ConstantOp constOp, OpAdaptor adaptor, matchAndRewrite(ConstantOp constOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override; ConversionPatternRewriter &rewriter) const override;
}; };
/// Converts scalar std.constant operation to spv.Constant. /// Converts scalar std.constant operation to spv.Constant.
class ConstantScalarOpPattern final : public OpConversionPattern<ConstantOp> { class ConstantScalarOpPattern final : public OpConversionPattern<ConstantOp> {
public: public:
using OpConversionPattern<ConstantOp>::OpConversionPattern; using OpConversionPattern<ConstantOp>::OpConversionPattern;
LogicalResult LogicalResult
matchAndRewrite(ConstantOp constOp, OpAdaptor adaptor, matchAndRewrite(ConstantOp constOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override; ConversionPatternRewriter &rewriter) const override;
}; };
/// Converts floating-point comparison operations to SPIR-V ops.
class CmpFOpPattern final : public OpConversionPattern<CmpFOp> {
public:
using OpConversionPattern<CmpFOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(CmpFOp cmpFOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts floating point NaN check to SPIR-V ops. This pattern requires
/// Kernel capability.
class CmpFOpNanKernelPattern final : public OpConversionPattern<CmpFOp> {
public:
using OpConversionPattern<CmpFOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(CmpFOp cmpFOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts floating point NaN check to SPIR-V ops. This pattern does not
/// require additional capability.
class CmpFOpNanNonePattern final : public OpConversionPattern<CmpFOp> {
public:
using OpConversionPattern<CmpFOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(CmpFOp cmpFOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts integer compare operation on i1 type operands to SPIR-V ops.
class BoolCmpIOpPattern final : public OpConversionPattern<CmpIOp> {
public:
using OpConversionPattern<CmpIOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(CmpIOp cmpIOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts integer compare operation to SPIR-V ops.
class CmpIOpPattern final : public OpConversionPattern<CmpIOp> {
public:
using OpConversionPattern<CmpIOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(CmpIOp cmpIOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts std.return to spv.Return. /// Converts std.return to spv.Return.
class ReturnOpPattern final : public OpConversionPattern<ReturnOp> { class ReturnOpPattern final : public OpConversionPattern<ReturnOp> {
public: public:
using OpConversionPattern<ReturnOp>::OpConversionPattern; using OpConversionPattern<ReturnOp>::OpConversionPattern;
LogicalResult LogicalResult
matchAndRewrite(ReturnOp returnOp, OpAdaptor adaptor, matchAndRewrite(ReturnOp returnOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override; ConversionPatternRewriter &rewriter) const override;
Show All 13 Lines
public: public:
using OpConversionPattern<SplatOp>::OpConversionPattern; using OpConversionPattern<SplatOp>::OpConversionPattern;
LogicalResult LogicalResult
matchAndRewrite(SplatOp op, OpAdaptor adaptor, matchAndRewrite(SplatOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override; ConversionPatternRewriter &rewriter) const override;
}; };
/// Converts std.zexti to spv.Select if the type of source is i1 or vector of
/// i1.
class ZeroExtendI1Pattern final : public OpConversionPattern<ZeroExtendIOp> {
public:
using OpConversionPattern<ZeroExtendIOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(ZeroExtendIOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
auto srcType = adaptor.getOperands().front().getType();
if (!isBoolScalarOrVector(srcType))
return failure();
auto dstType =
this->getTypeConverter()->convertType(op.getResult().getType());
Location loc = op.getLoc();
Value zero = spirv::ConstantOp::getZero(dstType, loc, rewriter);
Value one = spirv::ConstantOp::getOne(dstType, loc, rewriter);
rewriter.template replaceOpWithNewOp<spirv::SelectOp>(
op, dstType, adaptor.getOperands().front(), one, zero);
return success();
}
};
/// Converts tensor.extract into loading using access chains from SPIR-V local /// Converts tensor.extract into loading using access chains from SPIR-V local
/// variables. /// variables.
class TensorExtractPattern final class TensorExtractPattern final
: public OpConversionPattern<tensor::ExtractOp> { : public OpConversionPattern<tensor::ExtractOp> {
public: public:
TensorExtractPattern(TypeConverter &typeConverter, MLIRContext *context, TensorExtractPattern(TypeConverter &typeConverter, MLIRContext *context,
int64_t threshold, PatternBenefit benefit = 1) int64_t threshold, PatternBenefit benefit = 1)
: OpConversionPattern(typeConverter, context, benefit), : OpConversionPattern(typeConverter, context, benefit),
▲ Show 20 Lines • Show All 45 Lines • ▼ Show 20 Lines matchAndRewrite(tensor::ExtractOp extractOp, OpAdaptor adaptor,
return success(); return success();
} }
private: private:
int64_t byteCountThreshold; int64_t byteCountThreshold;
}; };
/// Converts std.trunci to spv.Select if the type of result is i1 or vector of
/// i1.
class TruncI1Pattern final : public OpConversionPattern<TruncateIOp> {
public:
using OpConversionPattern<TruncateIOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(TruncateIOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
auto dstType =
this->getTypeConverter()->convertType(op.getResult().getType());
if (!isBoolScalarOrVector(dstType))
return failure();
Location loc = op.getLoc();
auto srcType = adaptor.getOperands().front().getType();
// Check if (x & 1) == 1.
Value mask = spirv::ConstantOp::getOne(srcType, loc, rewriter);
Value maskedSrc = rewriter.create<spirv::BitwiseAndOp>(
loc, srcType, adaptor.getOperands()[0], mask);
Value isOne = rewriter.create<spirv::IEqualOp>(loc, maskedSrc, mask);
Value zero = spirv::ConstantOp::getZero(dstType, loc, rewriter);
Value one = spirv::ConstantOp::getOne(dstType, loc, rewriter);
rewriter.replaceOpWithNewOp<spirv::SelectOp>(op, dstType, isOne, one, zero);
return success();
}
};
/// Converts std.uitofp to spv.Select if the type of source is i1 or vector of
/// i1.
class UIToFPI1Pattern final : public OpConversionPattern<UIToFPOp> {
public:
using OpConversionPattern<UIToFPOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(UIToFPOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
auto srcType = adaptor.getOperands().front().getType();
if (!isBoolScalarOrVector(srcType))
return failure();
auto dstType =
this->getTypeConverter()->convertType(op.getResult().getType());
Location loc = op.getLoc();
Value zero = spirv::ConstantOp::getZero(dstType, loc, rewriter);
Value one = spirv::ConstantOp::getOne(dstType, loc, rewriter);
rewriter.template replaceOpWithNewOp<spirv::SelectOp>(
op, dstType, adaptor.getOperands().front(), one, zero);
return success();
}
};
/// Converts type-casting standard operations to SPIR-V operations.
template <typename StdOp, typename SPIRVOp>
class TypeCastingOpPattern final : public OpConversionPattern<StdOp> {
public:
using OpConversionPattern<StdOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(StdOp operation, typename StdOp::Adaptor adaptor,
ConversionPatternRewriter &rewriter) const override {
assert(adaptor.getOperands().size() == 1);
auto srcType = adaptor.getOperands().front().getType();
auto dstType =
this->getTypeConverter()->convertType(operation.getResult().getType());
if (isBoolScalarOrVector(srcType) || isBoolScalarOrVector(dstType))
return failure();
if (dstType == srcType) {
// Due to type conversion, we are seeing the same source and target type.
// Then we can just erase this operation by forwarding its operand.
rewriter.replaceOp(operation, adaptor.getOperands().front());
} else {
rewriter.template replaceOpWithNewOp<SPIRVOp>(operation, dstType,
adaptor.getOperands());
}
return success();
}
};
/// Converts std.xor to SPIR-V operations.
class XOrOpPattern final : public OpConversionPattern<XOrOp> {
public:
using OpConversionPattern<XOrOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(XOrOp xorOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
/// Converts std.xor to SPIR-V operations if the type of source is i1 or vector
/// of i1.
class BoolXOrOpPattern final : public OpConversionPattern<XOrOp> {
public:
using OpConversionPattern<XOrOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(XOrOp xorOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override;
};
} // namespace } // namespace
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// SignedRemIOpPattern
//===----------------------------------------------------------------------===//
LogicalResult SignedRemIOpPattern::matchAndRewrite(
SignedRemIOp remOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
Value result = emulateSignedRemainder(
remOp.getLoc(), adaptor.getOperands()[0], adaptor.getOperands()[1],
adaptor.getOperands()[0], rewriter);
rewriter.replaceOp(remOp, result);
return success();
}
//===----------------------------------------------------------------------===//
// ConstantOp with composite type. // ConstantOp with composite type.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// TODO: This probably should be split into the vector case and tensor case, // TODO: This probably should be split into the vector case and tensor case,
// so that the tensor case can be moved to TensorToSPIRV conversion. But, // so that the tensor case can be moved to TensorToSPIRV conversion. But,
// std.constant is for the standard dialect though. // std.constant is for the standard dialect though.
LogicalResult ConstantCompositeOpPattern::matchAndRewrite( LogicalResult ConstantCompositeOpPattern::matchAndRewrite(
ConstantOp constOp, OpAdaptor adaptor, ConstantOp constOp, OpAdaptor adaptor,
▲ Show 20 Lines • Show All 126 Lines • ▼ Show 20 Lines auto dstAttr =
convertIntegerAttr(srcAttr, dstType.cast<IntegerType>(), rewriter); convertIntegerAttr(srcAttr, dstType.cast<IntegerType>(), rewriter);
if (!dstAttr) if (!dstAttr)
return failure(); return failure();
rewriter.replaceOpWithNewOp<spirv::ConstantOp>(constOp, dstType, dstAttr); rewriter.replaceOpWithNewOp<spirv::ConstantOp>(constOp, dstType, dstAttr);
return success(); return success();
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// CmpFOp
//===----------------------------------------------------------------------===//
LogicalResult
CmpFOpPattern::matchAndRewrite(CmpFOp cmpFOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
switch (cmpFOp.getPredicate()) {
#define DISPATCH(cmpPredicate, spirvOp) \
case cmpPredicate: \
rewriter.replaceOpWithNewOp<spirvOp>(cmpFOp, cmpFOp.getResult().getType(), \
adaptor.lhs(), adaptor.rhs()); \
return success();
// Ordered.
DISPATCH(CmpFPredicate::OEQ, spirv::FOrdEqualOp);
DISPATCH(CmpFPredicate::OGT, spirv::FOrdGreaterThanOp);
DISPATCH(CmpFPredicate::OGE, spirv::FOrdGreaterThanEqualOp);
DISPATCH(CmpFPredicate::OLT, spirv::FOrdLessThanOp);
DISPATCH(CmpFPredicate::OLE, spirv::FOrdLessThanEqualOp);
DISPATCH(CmpFPredicate::ONE, spirv::FOrdNotEqualOp);
// Unordered.
DISPATCH(CmpFPredicate::UEQ, spirv::FUnordEqualOp);
DISPATCH(CmpFPredicate::UGT, spirv::FUnordGreaterThanOp);
DISPATCH(CmpFPredicate::UGE, spirv::FUnordGreaterThanEqualOp);
DISPATCH(CmpFPredicate::ULT, spirv::FUnordLessThanOp);
DISPATCH(CmpFPredicate::ULE, spirv::FUnordLessThanEqualOp);
DISPATCH(CmpFPredicate::UNE, spirv::FUnordNotEqualOp);
#undef DISPATCH
default:
break;
}
return failure();
}
LogicalResult CmpFOpNanKernelPattern::matchAndRewrite(
CmpFOp cmpFOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
if (cmpFOp.getPredicate() == CmpFPredicate::ORD) {
rewriter.replaceOpWithNewOp<spirv::OrderedOp>(cmpFOp, adaptor.lhs(),
adaptor.rhs());
return success();
}
if (cmpFOp.getPredicate() == CmpFPredicate::UNO) {
rewriter.replaceOpWithNewOp<spirv::UnorderedOp>(cmpFOp, adaptor.lhs(),
adaptor.rhs());
return success();
}
return failure();
}
LogicalResult CmpFOpNanNonePattern::matchAndRewrite(
CmpFOp cmpFOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
if (cmpFOp.getPredicate() != CmpFPredicate::ORD &&
cmpFOp.getPredicate() != CmpFPredicate::UNO)
return failure();
Location loc = cmpFOp.getLoc();
Value lhsIsNan = rewriter.create<spirv::IsNanOp>(loc, adaptor.lhs());
Value rhsIsNan = rewriter.create<spirv::IsNanOp>(loc, adaptor.rhs());
Value replace = rewriter.create<spirv::LogicalOrOp>(loc, lhsIsNan, rhsIsNan);
if (cmpFOp.getPredicate() == CmpFPredicate::ORD)
replace = rewriter.create<spirv::LogicalNotOp>(loc, replace);
rewriter.replaceOp(cmpFOp, replace);
return success();
}
//===----------------------------------------------------------------------===//
// CmpIOp
//===----------------------------------------------------------------------===//
LogicalResult
BoolCmpIOpPattern::matchAndRewrite(CmpIOp cmpIOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
Type operandType = cmpIOp.lhs().getType();
if (!isBoolScalarOrVector(operandType))
return failure();
switch (cmpIOp.getPredicate()) {
#define DISPATCH(cmpPredicate, spirvOp) \
case cmpPredicate: \
rewriter.replaceOpWithNewOp<spirvOp>(cmpIOp, cmpIOp.getResult().getType(), \
adaptor.lhs(), adaptor.rhs()); \
return success();
DISPATCH(CmpIPredicate::eq, spirv::LogicalEqualOp);
DISPATCH(CmpIPredicate::ne, spirv::LogicalNotEqualOp);
#undef DISPATCH
default:;
}
return failure();
}
LogicalResult
CmpIOpPattern::matchAndRewrite(CmpIOp cmpIOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
Type operandType = cmpIOp.lhs().getType();
if (isBoolScalarOrVector(operandType))
return failure();
switch (cmpIOp.getPredicate()) {
#define DISPATCH(cmpPredicate, spirvOp) \
case cmpPredicate: \
if (spirvOp::template hasTrait<OpTrait::spirv::UnsignedOp>() && \
operandType != this->getTypeConverter()->convertType(operandType)) { \
return cmpIOp.emitError( \
"bitwidth emulation is not implemented yet on unsigned op"); \
} \
rewriter.replaceOpWithNewOp<spirvOp>(cmpIOp, cmpIOp.getResult().getType(), \
adaptor.lhs(), adaptor.rhs()); \
return success();
DISPATCH(CmpIPredicate::eq, spirv::IEqualOp);
DISPATCH(CmpIPredicate::ne, spirv::INotEqualOp);
DISPATCH(CmpIPredicate::slt, spirv::SLessThanOp);
DISPATCH(CmpIPredicate::sle, spirv::SLessThanEqualOp);
DISPATCH(CmpIPredicate::sgt, spirv::SGreaterThanOp);
DISPATCH(CmpIPredicate::sge, spirv::SGreaterThanEqualOp);
DISPATCH(CmpIPredicate::ult, spirv::ULessThanOp);
DISPATCH(CmpIPredicate::ule, spirv::ULessThanEqualOp);
DISPATCH(CmpIPredicate::ugt, spirv::UGreaterThanOp);
DISPATCH(CmpIPredicate::uge, spirv::UGreaterThanEqualOp);
#undef DISPATCH
}
return failure();
}
//===----------------------------------------------------------------------===//
// ReturnOp // ReturnOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
LogicalResult LogicalResult
ReturnOpPattern::matchAndRewrite(ReturnOp returnOp, OpAdaptor adaptor, ReturnOpPattern::matchAndRewrite(ReturnOp returnOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const { ConversionPatternRewriter &rewriter) const {
if (returnOp.getNumOperands() > 1) if (returnOp.getNumOperands() > 1)
return failure(); return failure();
Show All 31 Lines if (!dstVecType || !spirv::CompositeType::isValid(dstVecType))
return failure(); return failure();
SmallVector<Value, 4> source(dstVecType.getNumElements(), adaptor.input()); SmallVector<Value, 4> source(dstVecType.getNumElements(), adaptor.input());
rewriter.replaceOpWithNewOp<spirv::CompositeConstructOp>(op, dstVecType, rewriter.replaceOpWithNewOp<spirv::CompositeConstructOp>(op, dstVecType,
source); source);
return success(); return success();
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// XorOp
//===----------------------------------------------------------------------===//
LogicalResult
XOrOpPattern::matchAndRewrite(XOrOp xorOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
assert(adaptor.getOperands().size() == 2);
if (isBoolScalarOrVector(adaptor.getOperands().front().getType()))
return failure();
auto dstType = getTypeConverter()->convertType(xorOp.getType());
if (!dstType)
return failure();
rewriter.replaceOpWithNewOp<spirv::BitwiseXorOp>(xorOp, dstType,
adaptor.getOperands());
return success();
}
LogicalResult
BoolXOrOpPattern::matchAndRewrite(XOrOp xorOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const {
assert(adaptor.getOperands().size() == 2);
if (!isBoolScalarOrVector(adaptor.getOperands().front().getType()))
return failure();
auto dstType = getTypeConverter()->convertType(xorOp.getType());
if (!dstType)
return failure();
rewriter.replaceOpWithNewOp<spirv::LogicalNotEqualOp>(xorOp, dstType,
adaptor.getOperands());
return success();
}
//===----------------------------------------------------------------------===//
// Pattern population // Pattern population
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
namespace mlir { namespace mlir {
void populateStandardToSPIRVPatterns(SPIRVTypeConverter &typeConverter, void populateStandardToSPIRVPatterns(SPIRVTypeConverter &typeConverter,
RewritePatternSet &patterns) { RewritePatternSet &patterns) {
MLIRContext *context = patterns.getContext(); MLIRContext *context = patterns.getContext();
patterns.add< patterns.add<
// Unary and binary patterns // Unary and binary patterns
BitwiseOpPattern<AndOp, spirv::LogicalAndOp, spirv::BitwiseAndOp>, spirv::UnaryAndBinaryOpPattern<MaxFOp, spirv::GLSLFMaxOp>,
BitwiseOpPattern<OrOp, spirv::LogicalOrOp, spirv::BitwiseOrOp>, spirv::UnaryAndBinaryOpPattern<MaxSIOp, spirv::GLSLSMaxOp>,
UnaryAndBinaryOpPattern<AbsFOp, spirv::GLSLFAbsOp>, spirv::UnaryAndBinaryOpPattern<MaxUIOp, spirv::GLSLUMaxOp>,
UnaryAndBinaryOpPattern<AddFOp, spirv::FAddOp>, spirv::UnaryAndBinaryOpPattern<MinFOp, spirv::GLSLFMinOp>,
UnaryAndBinaryOpPattern<AddIOp, spirv::IAddOp>, spirv::UnaryAndBinaryOpPattern<MinSIOp, spirv::GLSLSMinOp>,
UnaryAndBinaryOpPattern<CeilFOp, spirv::GLSLCeilOp>, spirv::UnaryAndBinaryOpPattern<MinUIOp, spirv::GLSLUMinOp>,
UnaryAndBinaryOpPattern<DivFOp, spirv::FDivOp>,
UnaryAndBinaryOpPattern<FloorFOp, spirv::GLSLFloorOp>,
UnaryAndBinaryOpPattern<MaxFOp, spirv::GLSLFMaxOp>,
UnaryAndBinaryOpPattern<MaxSIOp, spirv::GLSLSMaxOp>,
UnaryAndBinaryOpPattern<MaxUIOp, spirv::GLSLUMaxOp>,
UnaryAndBinaryOpPattern<MinFOp, spirv::GLSLFMinOp>,
UnaryAndBinaryOpPattern<MinSIOp, spirv::GLSLSMinOp>,
UnaryAndBinaryOpPattern<MinUIOp, spirv::GLSLUMinOp>,
UnaryAndBinaryOpPattern<MulFOp, spirv::FMulOp>,
UnaryAndBinaryOpPattern<MulIOp, spirv::IMulOp>,
UnaryAndBinaryOpPattern<NegFOp, spirv::FNegateOp>,
UnaryAndBinaryOpPattern<RemFOp, spirv::FRemOp>,
UnaryAndBinaryOpPattern<ShiftLeftOp, spirv::ShiftLeftLogicalOp>,
UnaryAndBinaryOpPattern<SignedDivIOp, spirv::SDivOp>,
UnaryAndBinaryOpPattern<SignedShiftRightOp,
spirv::ShiftRightArithmeticOp>,
UnaryAndBinaryOpPattern<SubIOp, spirv::ISubOp>,
UnaryAndBinaryOpPattern<SubFOp, spirv::FSubOp>,
UnaryAndBinaryOpPattern<UnsignedDivIOp, spirv::UDivOp>,
UnaryAndBinaryOpPattern<UnsignedRemIOp, spirv::UModOp>,
UnaryAndBinaryOpPattern<UnsignedShiftRightOp, spirv::ShiftRightLogicalOp>,
SignedRemIOpPattern, XOrOpPattern, BoolXOrOpPattern,
// Comparison patterns
BoolCmpIOpPattern, CmpFOpPattern, CmpFOpNanNonePattern, CmpIOpPattern,
// Constant patterns // Constant patterns
ConstantCompositeOpPattern, ConstantScalarOpPattern, ConstantCompositeOpPattern, ConstantScalarOpPattern,
ReturnOpPattern, SelectOpPattern, SplatPattern, ReturnOpPattern, SelectOpPattern, SplatPattern>(typeConverter, context);
// Type cast patterns
UIToFPI1Pattern, ZeroExtendI1Pattern, TruncI1Pattern,
TypeCastingOpPattern<IndexCastOp, spirv::SConvertOp>,
TypeCastingOpPattern<SIToFPOp, spirv::ConvertSToFOp>,
TypeCastingOpPattern<UIToFPOp, spirv::ConvertUToFOp>,
TypeCastingOpPattern<SignExtendIOp, spirv::SConvertOp>,
TypeCastingOpPattern<ZeroExtendIOp, spirv::UConvertOp>,
TypeCastingOpPattern<TruncateIOp, spirv::SConvertOp>,
TypeCastingOpPattern<FPToSIOp, spirv::ConvertFToSOp>,
TypeCastingOpPattern<FPExtOp, spirv::FConvertOp>,
TypeCastingOpPattern<FPTruncOp, spirv::FConvertOp>>(typeConverter,
context);
// Give CmpFOpNanKernelPattern a higher benefit so it can prevail when Kernel
// capability is available.
patterns.add<CmpFOpNanKernelPattern>(typeConverter, context,
/*benefit=*/2);
} }
void populateTensorToSPIRVPatterns(SPIRVTypeConverter &typeConverter, void populateTensorToSPIRVPatterns(SPIRVTypeConverter &typeConverter,
int64_t byteCountThreshold, int64_t byteCountThreshold,
RewritePatternSet &patterns) { RewritePatternSet &patterns) {
patterns.add<TensorExtractPattern>(typeConverter, patterns.getContext(), patterns.add<TensorExtractPattern>(typeConverter, patterns.getContext(),
byteCountThreshold); byteCountThreshold);
} }
} // namespace mlir } // namespace mlir

mlir/lib/Conversion/StandardToSPIRV/StandardToSPIRVPass.cpp

mlir/lib/Conversion/TosaToLinalg/CMakeLists.txt

mlir/lib/Conversion/TosaToLinalg/TosaToLinalg.cpp

mlir/lib/Conversion/TosaToLinalg/TosaToLinalgPass.cpp

mlir/lib/Conversion/TosaToStandard/CMakeLists.txt

mlir/lib/Conversion/TosaToStandard/TosaToStandard.cpp

mlir/lib/Conversion/TosaToStandard/TosaToStandardPass.cpp

mlir/lib/Conversion/VectorToGPU/CMakeLists.txt

mlir/lib/Conversion/VectorToGPU/VectorToGPU.cpp

mlir/lib/Conversion/VectorToLLVM/CMakeLists.txt

mlir/lib/Conversion/VectorToLLVM/ConvertVectorToLLVM.cpp

mlir/lib/Conversion/VectorToLLVM/ConvertVectorToLLVMPass.cpp

mlir/lib/Conversion/VectorToSCF/CMakeLists.txt

mlir/lib/Conversion/VectorToSCF/VectorToSCF.cpp

mlir/lib/Dialect/Affine/IR/AffineOps.cpp

mlir/lib/Dialect/Affine/IR/CMakeLists.txt

mlir/lib/Dialect/Affine/Transforms/AffineDataCopyGeneration.cpp

mlir/lib/Dialect/Affine/Transforms/AffineLoopInvariantCodeMotion.cpp

mlir/lib/Dialect/Affine/Transforms/CMakeLists.txt

mlir/lib/Dialect/Affine/Transforms/SuperVectorize.cpp

mlir/lib/Dialect/Arithmetic/CMakeLists.txt

mlir/lib/Dialect/Arithmetic/IR/ArithmeticDialect.cpp

//===- ArithmeticDialect.cpp - MLIR Arithmetic dialect implementation -----===// //===- ArithmeticDialect.cpp - MLIR Arithmetic dialect implementation -----===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/IR/Builders.h"
#include "mlir/Transforms/InliningUtils.h" #include "mlir/Transforms/InliningUtils.h"
using namespace mlir; using namespace mlir;
using namespace mlir::arith; using namespace mlir::arith;
#include "mlir/Dialect/Arithmetic/IR/ArithmeticOpsDialect.cpp.inc" #include "mlir/Dialect/Arithmetic/IR/ArithmeticOpsDialect.cpp.inc"
namespace { namespace {
/// This class defines the interface for handling inlining for arithmetic /// This class defines the interface for handling inlining for arithmetic
/// dialect operations. /// dialect operations.
struct ArithmeticInlinerInterface : public DialectInlinerInterface { struct ArithmeticInlinerInterface : public DialectInlinerInterface {
using DialectInlinerInterface::DialectInlinerInterface; using DialectInlinerInterface::DialectInlinerInterface;
/// All arithmetic dialect ops can be inlined. /// All arithmetic dialect ops can be inlined.
bool isLegalToInline(Operation *, Region *, bool, bool isLegalToInline(Operation *, Region *, bool,
BlockAndValueMapping &) const final { BlockAndValueMapping &) const final {
return true; return true;
} }
}; };
} // end anonymous namespace } // end anonymous namespace
void mlir::arith::ArithmeticDialect::initialize() { void arith::ArithmeticDialect::initialize() {
addOperations< addOperations<
rriddleUnsubmitted
Done
ReplyInline Actions
} // end anonymous namespace
- namespace mlir {
-
void arith::ArithmeticDialect::initialize() {

Only classes should really go in .cpp namespaces.

https://llvm.org/docs/CodingStandards.html#anonymous-namespaces
https://llvm.org/docs/CodingStandards.html#use-namespace-qualifiers-to-implement-previously-declared-functions

rriddle: Only classes should really go in .cpp namespaces. https://llvm.org/docs/CodingStandards.
MogballAuthorUnsubmitted
Done
ReplyInline Actions

Yeah this is one I need to drill myself to remember.

Mogball: Yeah this is one I need to drill myself to remember.
#define GET_OP_LIST #define GET_OP_LIST
#include "mlir/Dialect/Arithmetic/IR/ArithmeticOps.cpp.inc" #include "mlir/Dialect/Arithmetic/IR/ArithmeticOps.cpp.inc"
>(); >();
addInterfaces<ArithmeticInlinerInterface>(); addInterfaces<ArithmeticInlinerInterface>();
} }
/// Materialize an integer or floating point constant.
Operation *arith::ArithmeticDialect::materializeConstant(OpBuilder &builder,
Attribute value,
Type type,
Location loc) {
return builder.create<arith::ConstantOp>(loc, value, type);
}

mlir/lib/Dialect/Arithmetic/IR/ArithmeticOps.cpp

Show First 20 Lines • Show All 70 Lines • ▼ Show 20 Lines
// TableGen'd canonicalization patterns // TableGen'd canonicalization patterns
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
namespace { namespace {
#include "ArithmeticCanonicalization.inc" #include "ArithmeticCanonicalization.inc"
} // end anonymous namespace } // end anonymous namespace
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// ConstantOp
//===----------------------------------------------------------------------===//
void arith::ConstantOp::getAsmResultNames(
function_ref<void(Value, StringRef)> setNameFn) {
auto type = getType();
if (auto intCst = value().dyn_cast<IntegerAttr>()) {
auto intType = type.dyn_cast<IntegerType>();
// Sugar i1 constants with 'true' and 'false'.
if (intType && intType.getWidth() == 1)
return setNameFn(getResult(), (intCst.getInt() ? "true" : "false"));
// Otherwise, build a compex name with the value and type.
SmallString<32> specialNameBuffer;
llvm::raw_svector_ostream specialName(specialNameBuffer);
specialName << 'c' << intCst.getInt();
if (intType)
specialName << '_' << type;
setNameFn(getResult(), specialName.str());
} else {
setNameFn(getResult(), "cst");
}
}
/// TODO: disallow arith.constant to return anything other than signless integer
/// or float like.
static LogicalResult verify(arith::ConstantOp op) {
auto type = op.getType();
// The value's type must match the return type.
if (op.value().getType() != type) {
return op.emitOpError() << "value type " << op.value().getType()
<< " must match return type: " << type;
}
// Integer values must be signless.
if (type.isa<IntegerType>() && !type.cast<IntegerType>().isSignless())
return op.emitOpError("integer return type must be signless");
// Any float or elements attribute are acceptable.
if (!op.value().isa<IntegerAttr, FloatAttr, ElementsAttr>()) {
return op.emitOpError(
"value must be an integer, float, or elements attribute");
}
return success();
}
bool arith::ConstantOp::isBuildableWith(Attribute value, Type type) {
// The value's type must be the same as the provided type.
if (value.getType() != type)
return false;
// Integer values must be signless.
if (type.isa<IntegerType>() && !type.cast<IntegerType>().isSignless())
return false;
// Integer, float, and element attributes are buildable.
return value.isa<IntegerAttr, FloatAttr, ElementsAttr>();
}
OpFoldResult arith::ConstantOp::fold(ArrayRef<Attribute> operands) {
return value();
}
void arith::ConstantIntOp::build(OpBuilder &builder, OperationState &result,
int64_t value, unsigned width) {
auto type = builder.getIntegerType(width);
arith::ConstantOp::build(builder, result, type,
builder.getIntegerAttr(type, value));
}
void arith::ConstantIntOp::build(OpBuilder &builder, OperationState &result,
int64_t value, Type type) {
assert(type.isSignlessInteger() &&
"ConstantIntOp can only have signless integer type values");
arith::ConstantOp::build(builder, result, type,
builder.getIntegerAttr(type, value));
}
bool arith::ConstantIntOp::classof(Operation *op) {
if (auto constOp = dyn_cast_or_null<arith::ConstantOp>(op))
return constOp.getType().isSignlessInteger();
return false;
}
void arith::ConstantFloatOp::build(OpBuilder &builder, OperationState &result,
const APFloat &value, FloatType type) {
arith::ConstantOp::build(builder, result, type,
builder.getFloatAttr(type, value));
}
bool arith::ConstantFloatOp::classof(Operation *op) {
if (auto constOp = dyn_cast_or_null<arith::ConstantOp>(op))
return constOp.getType().isa<FloatType>();
return false;
}
void arith::ConstantIndexOp::build(OpBuilder &builder, OperationState &result,
int64_t value) {
arith::ConstantOp::build(builder, result, builder.getIndexType(),
builder.getIndexAttr(value));
}
bool arith::ConstantIndexOp::classof(Operation *op) {
if (auto constOp = dyn_cast_or_null<arith::ConstantOp>(op))
return constOp.getType().isIndex();
return false;
}
//===----------------------------------------------------------------------===//
// AddIOp // AddIOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
OpFoldResult arith::AddIOp::fold(ArrayRef<Attribute> operands) { OpFoldResult arith::AddIOp::fold(ArrayRef<Attribute> operands) {
// addi(x, 0) -> x // addi(x, 0) -> x
if (matchPattern(rhs(), m_Zero())) if (matchPattern(rhs(), m_Zero()))
return lhs(); return lhs();
▲ Show 20 Lines • Show All 285 Lines • ▼ Show 20 Lines
OpFoldResult arith::OrIOp::fold(ArrayRef<Attribute> operands) { OpFoldResult arith::OrIOp::fold(ArrayRef<Attribute> operands) {
/// or(x, 0) -> x /// or(x, 0) -> x
if (matchPattern(rhs(), m_Zero())) if (matchPattern(rhs(), m_Zero()))
return lhs(); return lhs();
/// or(x, x) -> x /// or(x, x) -> x
if (lhs() == rhs()) if (lhs() == rhs())
return rhs(); return rhs();
/// or(x, <all ones>) -> <all ones>
if (auto rhsAttr = operands[1].dyn_cast_or_null<IntegerAttr>())
if (rhsAttr.getValue().isAllOnes())
return rhsAttr;
return constFoldBinaryOp<IntegerAttr>(operands, return constFoldBinaryOp<IntegerAttr>(operands,
[](APInt a, APInt b) { return a | b; }); [](APInt a, APInt b) { return a | b; });
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// XOrIOp // XOrIOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 47 Lines • ▼ Show 20 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
OpFoldResult arith::DivFOp::fold(ArrayRef<Attribute> operands) { OpFoldResult arith::DivFOp::fold(ArrayRef<Attribute> operands) {
return constFoldBinaryOp<FloatAttr>( return constFoldBinaryOp<FloatAttr>(
operands, [](APFloat a, APFloat b) { return a / b; }); operands, [](APFloat a, APFloat b) { return a / b; });
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Utility functions for verifying cast ops
//===----------------------------------------------------------------------===//
template <typename... Types>
using type_list = std::tuple<Types...> *;
/// Returns a non-null type only if the provided type is one of the allowed
rriddleUnsubmitted
Done
ReplyInline Actions

std::tuple<Ts...> *?

rriddle: `std::tuple<Ts...> *`?
MogballAuthorUnsubmitted
Done
ReplyInline Actions

This was one "reviewer bait" because I didn't stumble across and existing example of something like this in the LLVM/MLIR codebase.

Mogball: This was one "reviewer bait" because I didn't stumble across and existing example of something…
/// types or one of the allowed shaped types of the allowed types. Returns the
/// element type if a valid shaped type is provided.
template <typename... ShapedTypes, typename... ElementTypes>
static Type getUnderlyingType(Type type, type_list<ShapedTypes...>,
type_list<ElementTypes...>) {
if (type.isa<ShapedType>() && !type.isa<ShapedTypes...>())
return {};
auto underlyingType = getElementTypeOrSelf(type);
if (!underlyingType.isa<ElementTypes...>())
return {};
return underlyingType;
}
/// Get allowed underlying types for vectors and tensors.
template <typename... ElementTypes>
static Type getTypeIfLike(Type type) {
return getUnderlyingType(type, type_list<VectorType, TensorType>(),
type_list<ElementTypes...>());
}
/// Get allowed underlying types for vectors, tensors, and memrefs.
template <typename... ElementTypes>
static Type getTypeIfLikeOrMemRef(Type type) {
return getUnderlyingType(type,
type_list<VectorType, TensorType, MemRefType>(),
type_list<ElementTypes...>());
}
static bool areValidCastInputsAndOutputs(TypeRange inputs, TypeRange outputs) {
return inputs.size() == 1 && outputs.size() == 1 &&
succeeded(verifyCompatibleShapes(inputs.front(), outputs.front()));
}
//===----------------------------------------------------------------------===//
// Verifiers for integer and floating point extension/truncation ops // Verifiers for integer and floating point extension/truncation ops
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Extend ops can only extend to a wider type. // Extend ops can only extend to a wider type.
template <typename ValType, typename Op> template <typename ValType, typename Op>
static LogicalResult verifyExtOp(Op op) { static LogicalResult verifyExtOp(Op op) {
Type srcType = getElementTypeOrSelf(op.in().getType()); Type srcType = getElementTypeOrSelf(op.in().getType());
Type dstType = getElementTypeOrSelf(op.getType()); Type dstType = getElementTypeOrSelf(op.getType());
Show All 13 Linesstatic LogicalResult verifyTruncateOp(Op op) {
if (srcType.cast<ValType>().getWidth() <= dstType.cast<ValType>().getWidth()) if (srcType.cast<ValType>().getWidth() <= dstType.cast<ValType>().getWidth())
return op.emitError("result type ") return op.emitError("result type ")
<< dstType << " must be shorter than operand type " << srcType; << dstType << " must be shorter than operand type " << srcType;
return success(); return success();
} }
/// Validate a cast that changes the width of a type.
template <template <typename> class WidthComparator, typename... ElementTypes>
static bool checkWidthChangeCast(TypeRange inputs, TypeRange outputs) {
if (!areValidCastInputsAndOutputs(inputs, outputs))
return false;
auto srcType = getTypeIfLike<ElementTypes...>(inputs.front());
auto dstType = getTypeIfLike<ElementTypes...>(outputs.front());
if (!srcType || !dstType)
rriddleUnsubmitted
Done
ReplyInline Actions

Please use proper ||

rriddle: Please use proper ||
return false;
return WidthComparator<unsigned>()(dstType.getIntOrFloatBitWidth(),
srcType.getIntOrFloatBitWidth());
}
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// ExtUIOp // ExtUIOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
OpFoldResult arith::ExtUIOp::fold(ArrayRef<Attribute> operands) { OpFoldResult arith::ExtUIOp::fold(ArrayRef<Attribute> operands) {
if (auto lhs = operands[0].dyn_cast_or_null<IntegerAttr>()) if (auto lhs = operands[0].dyn_cast_or_null<IntegerAttr>())
return IntegerAttr::get( return IntegerAttr::get(
getType(), lhs.getValue().zext(getType().getIntOrFloatBitWidth())); getType(), lhs.getValue().zext(getType().getIntOrFloatBitWidth()));
return {}; return {};
} }
bool arith::ExtUIOp::areCastCompatible(TypeRange inputs, TypeRange outputs) {
return checkWidthChangeCast<std::greater, IntegerType>(inputs, outputs);
}
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// ExtSIOp // ExtSIOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
OpFoldResult arith::ExtSIOp::fold(ArrayRef<Attribute> operands) { OpFoldResult arith::ExtSIOp::fold(ArrayRef<Attribute> operands) {
if (auto lhs = operands[0].dyn_cast_or_null<IntegerAttr>()) if (auto lhs = operands[0].dyn_cast_or_null<IntegerAttr>())
return IntegerAttr::get( return IntegerAttr::get(
getType(), lhs.getValue().sext(getType().getIntOrFloatBitWidth())); getType(), lhs.getValue().sext(getType().getIntOrFloatBitWidth()));
return {}; return {};
} }
// TODO temporary fixes until second patch is in bool arith::ExtSIOp::areCastCompatible(TypeRange inputs, TypeRange outputs) {
OpFoldResult arith::TruncFOp::fold(ArrayRef<Attribute> operands) { return checkWidthChangeCast<std::greater, IntegerType>(inputs, outputs);
return {};
} }
bool arith::TruncFOp::areCastCompatible(TypeRange inputs, TypeRange outputs) { //===----------------------------------------------------------------------===//
return true; // ExtFOp
//===----------------------------------------------------------------------===//
bool arith::ExtFOp::areCastCompatible(TypeRange inputs, TypeRange outputs) {
return checkWidthChangeCast<std::greater, FloatType>(inputs, outputs);
} }
//===----------------------------------------------------------------------===//
// TruncIOp
//===----------------------------------------------------------------------===//
OpFoldResult arith::TruncIOp::fold(ArrayRef<Attribute> operands) { OpFoldResult arith::TruncIOp::fold(ArrayRef<Attribute> operands) {
// trunci(zexti(a)) -> a
// trunci(sexti(a)) -> a
if (matchPattern(getOperand(), m_Op<arith::ExtUIOp>()) ||
matchPattern(getOperand(), m_Op<arith::ExtSIOp>()))
return getOperand().getDefiningOp()->getOperand(0);
assert(operands.size() == 1 && "unary operation takes one operand");
if (!operands[0])
return {}; return {};
}
bool arith::TruncIOp::areCastCompatible(TypeRange inputs, TypeRange outputs) { if (auto lhs = operands[0].dyn_cast<IntegerAttr>()) {
return true; return IntegerAttr::get(
rriddleUnsubmitted
Done
ReplyInline Actions

Drop the newline here.

rriddle: Drop the newline here.
getType(), lhs.getValue().trunc(getType().getIntOrFloatBitWidth()));
} }
bool arith::ExtUIOp::areCastCompatible(TypeRange inputs, TypeRange outputs) { return {};
return true;
} }
bool arith::ExtSIOp::areCastCompatible(TypeRange inputs, TypeRange outputs) { bool arith::TruncIOp::areCastCompatible(TypeRange inputs, TypeRange outputs) {
return true; return checkWidthChangeCast<std::less, IntegerType>(inputs, outputs);
} }
bool arith::ExtFOp::areCastCompatible(TypeRange inputs, TypeRange outputs) { //===----------------------------------------------------------------------===//
return true; // TruncFOp
} //===----------------------------------------------------------------------===//
/// Perform safe const propagation for truncf, i.e. only propagate if FP value
/// can be represented without precision loss or rounding.
OpFoldResult arith::TruncFOp::fold(ArrayRef<Attribute> operands) {
assert(operands.size() == 1 && "unary operation takes one operand");
auto constOperand = operands.front();
if (!constOperand || !constOperand.isa<FloatAttr>())
return {};
// Convert to target type via 'double'.
double sourceValue =
constOperand.dyn_cast<FloatAttr>().getValue().convertToDouble();
auto targetAttr = FloatAttr::get(getType(), sourceValue);
// Propagate if constant's value does not change after truncation.
if (sourceValue == targetAttr.getValue().convertToDouble())
return targetAttr;
OpFoldResult arith::ConstantOp::fold(ArrayRef<Attribute> operands) {
return {}; return {};
} }
bool arith::SIToFPOp::areCastCompatible(TypeRange inputs, TypeRange outputs) { bool arith::TruncFOp::areCastCompatible(TypeRange inputs, TypeRange outputs) {
return true; return checkWidthChangeCast<std::less, FloatType>(inputs, outputs);
} }
//===----------------------------------------------------------------------===//
// Verifiers for casts between integers and floats.
//===----------------------------------------------------------------------===//
template <typename From, typename To>
static bool checkIntFloatCast(TypeRange inputs, TypeRange outputs) {
if (!areValidCastInputsAndOutputs(inputs, outputs))
return false;
auto srcType = getTypeIfLike<From>(inputs.front());
auto dstType = getTypeIfLike<To>(outputs.back());
return srcType && dstType;
}
//===----------------------------------------------------------------------===//
// UIToFPOp
//===----------------------------------------------------------------------===//
bool arith::UIToFPOp::areCastCompatible(TypeRange inputs, TypeRange outputs) { bool arith::UIToFPOp::areCastCompatible(TypeRange inputs, TypeRange outputs) {
return true; return checkIntFloatCast<IntegerType, FloatType>(inputs, outputs);
} }
bool arith::FPToSIOp::areCastCompatible(TypeRange inputs, TypeRange outputs) { //===----------------------------------------------------------------------===//
return true; // SIToFPOp
//===----------------------------------------------------------------------===//
bool arith::SIToFPOp::areCastCompatible(TypeRange inputs, TypeRange outputs) {
return checkIntFloatCast<IntegerType, FloatType>(inputs, outputs);
} }
//===----------------------------------------------------------------------===//
// FPToUIOp
//===----------------------------------------------------------------------===//
bool arith::FPToUIOp::areCastCompatible(TypeRange inputs, TypeRange outputs) { bool arith::FPToUIOp::areCastCompatible(TypeRange inputs, TypeRange outputs) {
return true; return checkIntFloatCast<FloatType, IntegerType>(inputs, outputs);
}
//===----------------------------------------------------------------------===//
// FPToSIOp
//===----------------------------------------------------------------------===//
bool arith::FPToSIOp::areCastCompatible(TypeRange inputs, TypeRange outputs) {
return checkIntFloatCast<FloatType, IntegerType>(inputs, outputs);
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// IndexCastOp // IndexCastOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
bool arith::IndexCastOp::areCastCompatible(TypeRange inputs, bool arith::IndexCastOp::areCastCompatible(TypeRange inputs,
TypeRange outputs) { TypeRange outputs) {
assert(inputs.size() == 1 && outputs.size() == 1 && if (!areValidCastInputsAndOutputs(inputs, outputs))
"index_cast op expects one result and one result"); return false;
// Shape equivalence is guaranteed by op traits. auto srcType = getTypeIfLikeOrMemRef<IntegerType, IndexType>(inputs.front());
auto srcType = getElementTypeOrSelf(inputs.front()); auto dstType = getTypeIfLikeOrMemRef<IntegerType, IndexType>(outputs.front());
auto dstType = getElementTypeOrSelf(outputs.front()); if (!srcType || !dstType)
return false;
return (srcType.isIndex() && dstType.isSignlessInteger()) || return (srcType.isIndex() && dstType.isSignlessInteger()) ||
(srcType.isSignlessInteger() && dstType.isIndex()); (srcType.isSignlessInteger() && dstType.isIndex());
} }
OpFoldResult arith::IndexCastOp::fold(ArrayRef<Attribute> operands) { OpFoldResult arith::IndexCastOp::fold(ArrayRef<Attribute> operands) {
// index_cast(constant) -> constant // index_cast(constant) -> constant
// A little hack because we go through int. Otherwise, the size of the // A little hack because we go through int. Otherwise, the size of the
Show All 9 Linesvoid arith::IndexCastOp::getCanonicalizationPatterns(
patterns.insert<IndexCastOfIndexCast, IndexCastOfExtSI>(context); patterns.insert<IndexCastOfIndexCast, IndexCastOfExtSI>(context);
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// BitcastOp // BitcastOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
bool arith::BitcastOp::areCastCompatible(TypeRange inputs, TypeRange outputs) { bool arith::BitcastOp::areCastCompatible(TypeRange inputs, TypeRange outputs) {
assert(inputs.size() == 1 && outputs.size() == 1 && if (!areValidCastInputsAndOutputs(inputs, outputs))
"bitcast op expects one operand and one result"); return false;
// Shape equivalence is guaranteed by op traits. auto srcType =
auto srcType = getElementTypeOrSelf(inputs.front()); getTypeIfLikeOrMemRef<IntegerType, IndexType, FloatType>(inputs.front());
auto dstType = getElementTypeOrSelf(outputs.front()); auto dstType =
getTypeIfLikeOrMemRef<IntegerType, IndexType, FloatType>(outputs.front());
if (!srcType || !dstType)
return false;
// Types are guarnateed to be integers or floats by constraints.
return srcType.getIntOrFloatBitWidth() == dstType.getIntOrFloatBitWidth(); return srcType.getIntOrFloatBitWidth() == dstType.getIntOrFloatBitWidth();
} }
OpFoldResult arith::BitcastOp::fold(ArrayRef<Attribute> operands) { OpFoldResult arith::BitcastOp::fold(ArrayRef<Attribute> operands) {
assert(operands.size() == 1 && "bitcast op expects 1 operand"); assert(operands.size() == 1 && "bitcast op expects 1 operand");
auto resType = getType(); auto resType = getType();
auto operand = operands[0]; auto operand = operands[0];
▲ Show 20 Lines • Show All 189 Lines • Show Last 20 Lines

mlir/lib/Dialect/Arithmetic/Transforms/Bufferize.cpp

mlir/lib/Dialect/Arithmetic/Transforms/CMakeLists.txt

mlir/lib/Dialect/Arithmetic/Transforms/ExpandOps.cpp

mlir/lib/Dialect/Arithmetic/Transforms/PassDetail.h

mlir/lib/Dialect/ArmSVE/IR/ArmSVEDialect.cpp

mlir/lib/Dialect/Async/Transforms/AsyncParallelFor.cpp

mlir/lib/Dialect/Async/Transforms/AsyncToAsyncRuntime.cpp

mlir/lib/Dialect/Async/Transforms/CMakeLists.txt

mlir/lib/Dialect/Async/Transforms/PassDetail.h

mlir/lib/Dialect/Complex/IR/CMakeLists.txt

mlir/lib/Dialect/Complex/IR/ComplexDialect.cpp

mlir/lib/Dialect/EmitC/IR/EmitC.cpp

mlir/lib/Dialect/GPU/CMakeLists.txt

mlir/lib/Dialect/GPU/IR/GPUDialect.cpp

mlir/lib/Dialect/GPU/Transforms/AllReduceLowering.cpp

mlir/lib/Dialect/GPU/Transforms/KernelOutlining.cpp

mlir/lib/Dialect/GPU/Transforms/MemoryPromotion.cpp

mlir/lib/Dialect/Linalg/IR/CMakeLists.txt

mlir/lib/Dialect/Linalg/IR/LinalgInterfaces.cpp

mlir/lib/Dialect/Linalg/IR/LinalgOps.cpp

mlir/lib/Dialect/Linalg/Transforms/Bufferize.cpp

mlir/lib/Dialect/Linalg/Transforms/CMakeLists.txt

mlir/lib/Dialect/Linalg/Transforms/ComprehensiveBufferize.cpp

mlir/lib/Dialect/Linalg/Transforms/Detensorize.cpp

mlir/lib/Dialect/Linalg/Transforms/DropUnitDims.cpp

mlir/lib/Dialect/Linalg/Transforms/ElementwiseOpFusion.cpp

mlir/lib/Dialect/Linalg/Transforms/Fusion.cpp

mlir/lib/Dialect/Linalg/Transforms/FusionOnTensors.cpp

mlir/lib/Dialect/Linalg/Transforms/Hoisting.cpp

mlir/lib/Dialect/Linalg/Transforms/InlineScalarOperands.cpp

mlir/lib/Dialect/Linalg/Transforms/Loops.cpp

mlir/lib/Dialect/Linalg/Transforms/PassDetail.h

mlir/lib/Dialect/Linalg/Transforms/Promotion.cpp

mlir/lib/Dialect/Linalg/Transforms/Tiling.cpp

mlir/lib/Dialect/Linalg/Transforms/Transforms.cpp

mlir/lib/Dialect/Linalg/Transforms/Vectorization.cpp

mlir/lib/Dialect/Linalg/Utils/CMakeLists.txt

mlir/lib/Dialect/Linalg/Utils/Utils.cpp

mlir/lib/Dialect/Math/Transforms/AlgebraicSimplification.cpp

mlir/lib/Dialect/Math/Transforms/CMakeLists.txt

mlir/lib/Dialect/Math/Transforms/ExpandTanh.cpp

mlir/lib/Dialect/Math/Transforms/PolynomialApproximation.cpp

mlir/lib/Dialect/MemRef/IR/CMakeLists.txt

mlir/lib/Dialect/MemRef/IR/MemRefOps.cpp

mlir/lib/Dialect/MemRef/Transforms/CMakeLists.txt

mlir/lib/Dialect/MemRef/Transforms/FoldSubViewOps.cpp

mlir/lib/Dialect/MemRef/Transforms/ResolveShapedTypeResultDims.cpp

mlir/lib/Dialect/OpenACC/CMakeLists.txt

mlir/lib/Dialect/OpenACC/IR/OpenACC.cpp

mlir/lib/Dialect/Quant/CMakeLists.txt

mlir/lib/Dialect/Quant/Transforms/ConvertConst.cpp

mlir/lib/Dialect/SCF/CMakeLists.txt

mlir/lib/Dialect/SCF/SCF.cpp

mlir/lib/Dialect/SCF/Transforms/CMakeLists.txt

mlir/lib/Dialect/SCF/Transforms/ForToWhile.cpp

mlir/lib/Dialect/SCF/Transforms/LoopPipelining.cpp

mlir/lib/Dialect/SCF/Transforms/LoopRangeFolding.cpp

mlir/lib/Dialect/SCF/Transforms/LoopSpecialization.cpp

mlir/lib/Dialect/SCF/Transforms/ParallelLoopTiling.cpp

mlir/lib/Dialect/SCF/Transforms/PassDetail.h

mlir/lib/Dialect/Shape/IR/CMakeLists.txt

mlir/lib/Dialect/Shape/IR/Shape.cpp

mlir/lib/Dialect/Shape/Transforms/CMakeLists.txt

mlir/lib/Dialect/Shape/Transforms/ShapeToShapeLowering.cpp

mlir/lib/Dialect/SparseTensor/IR/CMakeLists.txt

mlir/lib/Dialect/SparseTensor/IR/SparseTensorDialect.cpp

mlir/lib/Dialect/SparseTensor/Transforms/CMakeLists.txt

mlir/lib/Dialect/SparseTensor/Transforms/SparseTensorConversion.cpp

mlir/lib/Dialect/SparseTensor/Transforms/SparseTensorPasses.cpp

mlir/lib/Dialect/SparseTensor/Transforms/Sparsification.cpp

mlir/lib/Dialect/SparseTensor/Utils/CMakeLists.txt

mlir/lib/Dialect/SparseTensor/Utils/Merger.cpp

mlir/lib/Dialect/StandardOps/CMakeLists.txt

mlir/lib/Dialect/StandardOps/IR/Ops.cpp

//===- Ops.cpp - Standard MLIR Operations ---------------------------------===// //===- Ops.cpp - Standard MLIR Operations ---------------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/CommonFolders.h" #include "mlir/Dialect/CommonFolders.h"
#include "mlir/Dialect/StandardOps/Utils/Utils.h" #include "mlir/Dialect/StandardOps/Utils/Utils.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/AffineMap.h" #include "mlir/IR/AffineMap.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/BuiltinTypes.h" #include "mlir/IR/BuiltinTypes.h"
▲ Show 20 Lines • Show All 75 Lines • ▼ Show 20 Linesstruct StdInlinerInterface : public DialectInlinerInterface {
} }
}; };
} // end anonymous namespace } // end anonymous namespace
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// StandardOpsDialect // StandardOpsDialect
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
/// A custom unary operation printer that omits the "std." prefix from the
/// operation names.
static void printStandardUnaryOp(Operation *op, OpAsmPrinter &p) {
assert(op->getNumOperands() == 1 && "unary op should have one operand");
assert(op->getNumResults() == 1 && "unary op should have one result");
p << ' ' << op->getOperand(0);
p.printOptionalAttrDict(op->getAttrs());
p << " : " << op->getOperand(0).getType();
}
/// A custom binary operation printer that omits the "std." prefix from the /// A custom binary operation printer that omits the "std." prefix from the
/// operation names. /// operation names.
static void printStandardBinaryOp(Operation *op, OpAsmPrinter &p) { static void printStandardBinaryOp(Operation *op, OpAsmPrinter &p) {
assert(op->getNumOperands() == 2 && "binary op should have two operands"); assert(op->getNumOperands() == 2 && "binary op should have two operands");
assert(op->getNumResults() == 1 && "binary op should have one result"); assert(op->getNumResults() == 1 && "binary op should have one result");
// If not all the operand and result types are the same, just use the // If not all the operand and result types are the same, just use the
// generic assembly form to avoid omitting information in printing. // generic assembly form to avoid omitting information in printing.
auto resultType = op->getResult(0).getType(); auto resultType = op->getResult(0).getType();
if (op->getOperand(0).getType() != resultType || if (op->getOperand(0).getType() != resultType ||
op->getOperand(1).getType() != resultType) { op->getOperand(1).getType() != resultType) {
p.printGenericOp(op); p.printGenericOp(op);
return; return;
} }
p << ' ' << op->getOperand(0) << ", " << op->getOperand(1); p << ' ' << op->getOperand(0) << ", " << op->getOperand(1);
p.printOptionalAttrDict(op->getAttrs()); p.printOptionalAttrDict(op->getAttrs());
// Now we can output only one type for all operands and the result. // Now we can output only one type for all operands and the result.
p << " : " << op->getResult(0).getType(); p << " : " << op->getResult(0).getType();
} }
/// A custom ternary operation printer that omits the "std." prefix from the
/// operation names.
static void printStandardTernaryOp(Operation *op, OpAsmPrinter &p) {
assert(op->getNumOperands() == 3 && "ternary op should have three operands");
assert(op->getNumResults() == 1 && "ternary op should have one result");
// If not all the operand and result types are the same, just use the
// generic assembly form to avoid omitting information in printing.
auto resultType = op->getResult(0).getType();
if (op->getOperand(0).getType() != resultType ||
op->getOperand(1).getType() != resultType ||
op->getOperand(2).getType() != resultType) {
p.printGenericOp(op);
return;
}
p << ' ' << op->getOperand(0) << ", " << op->getOperand(1) << ", "
<< op->getOperand(2);
p.printOptionalAttrDict(op->getAttrs());
// Now we can output only one type for all operands and the result.
p << " : " << op->getResult(0).getType();
}
/// A custom cast operation printer that omits the "std." prefix from the
/// operation names.
static void printStandardCastOp(Operation *op, OpAsmPrinter &p) {
p << ' ' << op->getOperand(0) << " : " << op->getOperand(0).getType()
<< " to " << op->getResult(0).getType();
}
void StandardOpsDialect::initialize() { void StandardOpsDialect::initialize() {
addOperations< addOperations<
#define GET_OP_LIST #define GET_OP_LIST
#include "mlir/Dialect/StandardOps/IR/Ops.cpp.inc" #include "mlir/Dialect/StandardOps/IR/Ops.cpp.inc"
>(); >();
addInterfaces<StdInlinerInterface>(); addInterfaces<StdInlinerInterface>();
} }
/// Materialize a single constant operation from a given attribute value with /// Materialize a single constant operation from a given attribute value with
/// the desired resultant type. /// the desired resultant type.
Operation *StandardOpsDialect::materializeConstant(OpBuilder &builder, Operation *StandardOpsDialect::materializeConstant(OpBuilder &builder,
Attribute value, Type type, Attribute value, Type type,
Location loc) { Location loc) {
if (arith::ConstantOp::isBuildableWith(value, type))
rriddleUnsubmitted
Done
ReplyInline Actions

Please remove usernames from TODO comments.

rriddle: Please remove usernames from TODO comments.
return builder.create<arith::ConstantOp>(loc, type, value);
return builder.create<ConstantOp>(loc, type, value); return builder.create<ConstantOp>(loc, type, value);
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Common cast compatibility check for vector types.
//===----------------------------------------------------------------------===//
/// This method checks for cast compatibility of vector types.
/// If 'a' and 'b' are vector types, and they are cast compatible,
/// it calls the 'areElementsCastCompatible' function to check for
/// element cast compatibility.
/// Returns 'true' if the vector types are cast compatible, and 'false'
/// otherwise.
static bool areVectorCastSimpleCompatible(
Type a, Type b,
function_ref<bool(TypeRange, TypeRange)> areElementsCastCompatible) {
if (auto va = a.dyn_cast<VectorType>())
if (auto vb = b.dyn_cast<VectorType>())
return va.getShape().equals(vb.getShape()) &&
areElementsCastCompatible(va.getElementType(),
vb.getElementType());
return false;
}
//===----------------------------------------------------------------------===//
// AddFOp
//===----------------------------------------------------------------------===//
OpFoldResult AddFOp::fold(ArrayRef<Attribute> operands) {
return constFoldBinaryOp<FloatAttr>(
operands, [](APFloat a, APFloat b) { return a + b; });
}
//===----------------------------------------------------------------------===//
// AddIOp
//===----------------------------------------------------------------------===//
OpFoldResult AddIOp::fold(ArrayRef<Attribute> operands) {
/// addi(x, 0) -> x
if (matchPattern(rhs(), m_Zero()))
return lhs();
return constFoldBinaryOp<IntegerAttr>(operands,
[](APInt a, APInt b) { return a + b; });
}
/// Canonicalize a sum of a constant and (constant - something) to simply be
/// a sum of constants minus something. This transformation does similar
/// transformations for additions of a constant with a subtract/add of
/// a constant. This may result in some operations being reordered (but should
/// remain equivalent).
struct AddConstantReorder : public OpRewritePattern<AddIOp> {
using OpRewritePattern<AddIOp>::OpRewritePattern;
LogicalResult matchAndRewrite(AddIOp addop,
PatternRewriter &rewriter) const override {
for (int i = 0; i < 2; i++) {
APInt origConst;
APInt midConst;
if (matchPattern(addop.getOperand(i), m_ConstantInt(&origConst))) {
if (auto midAddOp = addop.getOperand(1 - i).getDefiningOp<AddIOp>()) {
for (int j = 0; j < 2; j++) {
if (matchPattern(midAddOp.getOperand(j),
m_ConstantInt(&midConst))) {
auto nextConstant = rewriter.create<ConstantOp>(
addop.getLoc(), rewriter.getIntegerAttr(
addop.getType(), origConst + midConst));
rewriter.replaceOpWithNewOp<AddIOp>(addop, nextConstant,
midAddOp.getOperand(1 - j));
return success();
}
}
}
if (auto midSubOp = addop.getOperand(1 - i).getDefiningOp<SubIOp>()) {
if (matchPattern(midSubOp.getOperand(0), m_ConstantInt(&midConst))) {
auto nextConstant = rewriter.create<ConstantOp>(
addop.getLoc(),
rewriter.getIntegerAttr(addop.getType(), origConst + midConst));
rewriter.replaceOpWithNewOp<SubIOp>(addop, nextConstant,
midSubOp.getOperand(1));
return success();
}
if (matchPattern(midSubOp.getOperand(1), m_ConstantInt(&midConst))) {
auto nextConstant = rewriter.create<ConstantOp>(
addop.getLoc(),
rewriter.getIntegerAttr(addop.getType(), origConst - midConst));
rewriter.replaceOpWithNewOp<AddIOp>(addop, nextConstant,
midSubOp.getOperand(0));
return success();
}
}
}
}
return failure();
}
};
void AddIOp::getCanonicalizationPatterns(OwningRewritePatternList &results,
MLIRContext *context) {
results.insert<AddConstantReorder>(context);
}
//===----------------------------------------------------------------------===//
// AndOp
//===----------------------------------------------------------------------===//
OpFoldResult AndOp::fold(ArrayRef<Attribute> operands) {
/// and(x, 0) -> 0
if (matchPattern(rhs(), m_Zero()))
return rhs();
/// and(x, allOnes) -> x
APInt intValue;
if (matchPattern(rhs(), m_ConstantInt(&intValue)) && intValue.isAllOnes())
return lhs();
/// and(x,x) -> x
if (lhs() == rhs())
return rhs();
return constFoldBinaryOp<IntegerAttr>(operands,
[](APInt a, APInt b) { return a & b; });
}
//===----------------------------------------------------------------------===//
// AssertOp // AssertOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
LogicalResult AssertOp::canonicalize(AssertOp op, PatternRewriter &rewriter) { LogicalResult AssertOp::canonicalize(AssertOp op, PatternRewriter &rewriter) {
// Erase assertion if argument is constant true. // Erase assertion if argument is constant true.
if (matchPattern(op.arg(), m_One())) { if (matchPattern(op.arg(), m_One())) {
rewriter.eraseOp(op); rewriter.eraseOp(op);
return success(); return success();
▲ Show 20 Lines • Show All 77 Lines • ▼ Show 20 LinesAttribute mlir::getIdentityValueAttr(AtomicRMWKind kind, Type resultType,
} }
return nullptr; return nullptr;
} }
/// Returns the identity value associated with an AtomicRMWKind op. /// Returns the identity value associated with an AtomicRMWKind op.
Value mlir::getIdentityValue(AtomicRMWKind op, Type resultType, Value mlir::getIdentityValue(AtomicRMWKind op, Type resultType,
OpBuilder &builder, Location loc) { OpBuilder &builder, Location loc) {
Attribute attr = getIdentityValueAttr(op, resultType, builder, loc); Attribute attr = getIdentityValueAttr(op, resultType, builder, loc);
return builder.create<ConstantOp>(loc, attr); return builder.create<arith::ConstantOp>(loc, attr);
} }
/// Return the value obtained by applying the reduction operation kind /// Return the value obtained by applying the reduction operation kind
/// associated with a binary AtomicRMWKind op to `lhs` and `rhs`. /// associated with a binary AtomicRMWKind op to `lhs` and `rhs`.
Value mlir::getReductionOp(AtomicRMWKind op, OpBuilder &builder, Location loc, Value mlir::getReductionOp(AtomicRMWKind op, OpBuilder &builder, Location loc,
Value lhs, Value rhs) { Value lhs, Value rhs) {
switch (op) { switch (op) {
case AtomicRMWKind::addf: case AtomicRMWKind::addf:
return builder.create<AddFOp>(loc, lhs, rhs); return builder.create<arith::AddFOp>(loc, lhs, rhs);
case AtomicRMWKind::addi: case AtomicRMWKind::addi:
return builder.create<AddIOp>(loc, lhs, rhs); return builder.create<arith::AddIOp>(loc, lhs, rhs);
case AtomicRMWKind::mulf: case AtomicRMWKind::mulf:
return builder.create<MulFOp>(loc, lhs, rhs); return builder.create<arith::MulFOp>(loc, lhs, rhs);
case AtomicRMWKind::muli: case AtomicRMWKind::muli:
return builder.create<MulIOp>(loc, lhs, rhs); return builder.create<arith::MulIOp>(loc, lhs, rhs);
case AtomicRMWKind::maxf: case AtomicRMWKind::maxf:
return builder.create<SelectOp>( return builder.create<SelectOp>(
loc, builder.create<CmpFOp>(loc, CmpFPredicate::OGT, lhs, rhs), lhs, loc,
rhs); builder.create<arith::CmpFOp>(loc, arith::CmpFPredicate::OGT, lhs, rhs),
lhs, rhs);
case AtomicRMWKind::minf: case AtomicRMWKind::minf:
return builder.create<SelectOp>( return builder.create<SelectOp>(
loc, builder.create<CmpFOp>(loc, CmpFPredicate::OLT, lhs, rhs), lhs, loc,
rhs); builder.create<arith::CmpFOp>(loc, arith::CmpFPredicate::OLT, lhs, rhs),
lhs, rhs);
case AtomicRMWKind::maxs: case AtomicRMWKind::maxs:
return builder.create<SelectOp>( return builder.create<SelectOp>(
loc, builder.create<CmpIOp>(loc, CmpIPredicate::sgt, lhs, rhs), lhs, loc,
rhs); builder.create<arith::CmpIOp>(loc, arith::CmpIPredicate::sgt, lhs, rhs),
lhs, rhs);
case AtomicRMWKind::mins: case AtomicRMWKind::mins:
return builder.create<SelectOp>( return builder.create<SelectOp>(
loc, builder.create<CmpIOp>(loc, CmpIPredicate::slt, lhs, rhs), lhs, loc,
rhs); builder.create<arith::CmpIOp>(loc, arith::CmpIPredicate::slt, lhs, rhs),
lhs, rhs);
case AtomicRMWKind::maxu: case AtomicRMWKind::maxu:
return builder.create<SelectOp>( return builder.create<SelectOp>(
loc, builder.create<CmpIOp>(loc, CmpIPredicate::ugt, lhs, rhs), lhs, loc,
rhs); builder.create<arith::CmpIOp>(loc, arith::CmpIPredicate::ugt, lhs, rhs),
lhs, rhs);
case AtomicRMWKind::minu: case AtomicRMWKind::minu:
return builder.create<SelectOp>( return builder.create<SelectOp>(
loc, builder.create<CmpIOp>(loc, CmpIPredicate::ult, lhs, rhs), lhs, loc,
rhs); builder.create<arith::CmpIOp>(loc, arith::CmpIPredicate::ult, lhs, rhs),
lhs, rhs);
// TODO: Add remaining reduction operations. // TODO: Add remaining reduction operations.
default: default:
(void)emitOptionalError(loc, "Reduction operation type not supported"); (void)emitOptionalError(loc, "Reduction operation type not supported");
break; break;
} }
return nullptr; return nullptr;
} }
▲ Show 20 Lines • Show All 75 Lines • ▼ Show 20 Linesstatic LogicalResult verify(AtomicYieldOp op) {
Type resultType = op.result().getType(); Type resultType = op.result().getType();
if (parentType != resultType) if (parentType != resultType)
return op.emitOpError() << "types mismatch between yield op: " << resultType return op.emitOpError() << "types mismatch between yield op: " << resultType
<< " and its parent: " << parentType; << " and its parent: " << parentType;
return success(); return success();
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// BitcastOp
//===----------------------------------------------------------------------===//
bool BitcastOp::areCastCompatible(TypeRange inputs, TypeRange outputs) {
assert(inputs.size() == 1 && outputs.size() == 1 &&
"bitcast op expects one operand and result");
Type a = inputs.front(), b = outputs.front();
if (a.isSignlessIntOrFloat() && b.isSignlessIntOrFloat())
return a.getIntOrFloatBitWidth() == b.getIntOrFloatBitWidth();
return areVectorCastSimpleCompatible(a, b, areCastCompatible);
}
OpFoldResult BitcastOp::fold(ArrayRef<Attribute> operands) {
assert(operands.size() == 1 && "bitcastop expects 1 operand");
// Bitcast of bitcast
auto *sourceOp = getOperand().getDefiningOp();
if (auto sourceBitcast = dyn_cast_or_null<BitcastOp>(sourceOp)) {
setOperand(sourceBitcast.getOperand());
return getResult();
}
auto operand = operands[0];
if (!operand)
return {};
Type resType = getResult().getType();
if (auto denseAttr = operand.dyn_cast<DenseElementsAttr>())
return denseAttr.bitcast(resType.cast<ShapedType>().getElementType());
APInt bits;
if (auto floatAttr = operand.dyn_cast<FloatAttr>())
bits = floatAttr.getValue().bitcastToAPInt();
else if (auto intAttr = operand.dyn_cast<IntegerAttr>())
bits = intAttr.getValue();
else
return {};
if (resType.isa<IntegerType>())
return IntegerAttr::get(resType, bits);
if (auto resFloatType = resType.dyn_cast<FloatType>())
return FloatAttr::get(resType,
APFloat(resFloatType.getFloatSemantics(), bits));
return {};
}
//===----------------------------------------------------------------------===//
// BranchOp // BranchOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
/// Given a successor, try to collapse it to a new destination if it only /// Given a successor, try to collapse it to a new destination if it only
/// contains a passthrough unconditional branch. If the successor is /// contains a passthrough unconditional branch. If the successor is
/// collapsable, `successor` and `successorOperands` are updated to reference /// collapsable, `successor` and `successorOperands` are updated to reference
/// the new destination and values. `argStorage` is used as storage if operands /// the new destination and values. `argStorage` is used as storage if operands
/// to the collapsed successor need to be remapped. It must outlive uses of /// to the collapsed successor need to be remapped. It must outlive uses of
▲ Show 20 Lines • Show All 173 Lines • ▼ Show 20 Linesstatic Type getI1SameShape(Type type) {
if (type.isa<UnrankedTensorType>()) if (type.isa<UnrankedTensorType>())
return UnrankedTensorType::get(i1Type); return UnrankedTensorType::get(i1Type);
if (auto vectorType = type.dyn_cast<VectorType>()) if (auto vectorType = type.dyn_cast<VectorType>())
return VectorType::get(vectorType.getShape(), i1Type); return VectorType::get(vectorType.getShape(), i1Type);
return i1Type; return i1Type;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// CmpIOp
//===----------------------------------------------------------------------===//
static void buildCmpIOp(OpBuilder &build, OperationState &result,
CmpIPredicate predicate, Value lhs, Value rhs) {
result.addOperands({lhs, rhs});
result.types.push_back(getI1SameShape(lhs.getType()));
result.addAttribute(CmpIOp::getPredicateAttrName(),
build.getI64IntegerAttr(static_cast<int64_t>(predicate)));
}
// Compute `lhs` `pred` `rhs`, where `pred` is one of the known integer
// comparison predicates.
bool mlir::applyCmpPredicate(CmpIPredicate predicate, const APInt &lhs,
const APInt &rhs) {
switch (predicate) {
case CmpIPredicate::eq:
return lhs.eq(rhs);
case CmpIPredicate::ne:
return lhs.ne(rhs);
case CmpIPredicate::slt:
return lhs.slt(rhs);
case CmpIPredicate::sle:
return lhs.sle(rhs);
case CmpIPredicate::sgt:
return lhs.sgt(rhs);
case CmpIPredicate::sge:
return lhs.sge(rhs);
case CmpIPredicate::ult:
return lhs.ult(rhs);
case CmpIPredicate::ule:
return lhs.ule(rhs);
case CmpIPredicate::ugt:
return lhs.ugt(rhs);
case CmpIPredicate::uge:
return lhs.uge(rhs);
}
llvm_unreachable("unknown comparison predicate");
}
// Returns true if the predicate is true for two equal operands.
static bool applyCmpPredicateToEqualOperands(CmpIPredicate predicate) {
switch (predicate) {
case CmpIPredicate::eq:
case CmpIPredicate::sle:
case CmpIPredicate::sge:
case CmpIPredicate::ule:
case CmpIPredicate::uge:
return true;
case CmpIPredicate::ne:
case CmpIPredicate::slt:
case CmpIPredicate::sgt:
case CmpIPredicate::ult:
case CmpIPredicate::ugt:
return false;
}
llvm_unreachable("unknown comparison predicate");
}
// Constant folding hook for comparisons.
OpFoldResult CmpIOp::fold(ArrayRef<Attribute> operands) {
assert(operands.size() == 2 && "cmpi takes two arguments");
if (lhs() == rhs()) {
auto val = applyCmpPredicateToEqualOperands(getPredicate());
return BoolAttr::get(getContext(), val);
}
auto lhs = operands.front().dyn_cast_or_null<IntegerAttr>();
auto rhs = operands.back().dyn_cast_or_null<IntegerAttr>();
if (!lhs || !rhs)
return {};
auto val = applyCmpPredicate(getPredicate(), lhs.getValue(), rhs.getValue());
return BoolAttr::get(getContext(), val);
}
//===----------------------------------------------------------------------===//
// CmpFOp
//===----------------------------------------------------------------------===//
static void buildCmpFOp(OpBuilder &build, OperationState &result,
CmpFPredicate predicate, Value lhs, Value rhs) {
result.addOperands({lhs, rhs});
result.types.push_back(getI1SameShape(lhs.getType()));
result.addAttribute(CmpFOp::getPredicateAttrName(),
build.getI64IntegerAttr(static_cast<int64_t>(predicate)));
}
/// Compute `lhs` `pred` `rhs`, where `pred` is one of the known floating point
/// comparison predicates.
bool mlir::applyCmpPredicate(CmpFPredicate predicate, const APFloat &lhs,
const APFloat &rhs) {
auto cmpResult = lhs.compare(rhs);
switch (predicate) {
case CmpFPredicate::AlwaysFalse:
return false;
case CmpFPredicate::OEQ:
return cmpResult == APFloat::cmpEqual;
case CmpFPredicate::OGT:
return cmpResult == APFloat::cmpGreaterThan;
case CmpFPredicate::OGE:
return cmpResult == APFloat::cmpGreaterThan ||
cmpResult == APFloat::cmpEqual;
case CmpFPredicate::OLT:
return cmpResult == APFloat::cmpLessThan;
case CmpFPredicate::OLE:
return cmpResult == APFloat::cmpLessThan || cmpResult == APFloat::cmpEqual;
case CmpFPredicate::ONE:
return cmpResult != APFloat::cmpUnordered && cmpResult != APFloat::cmpEqual;
case CmpFPredicate::ORD:
return cmpResult != APFloat::cmpUnordered;
case CmpFPredicate::UEQ:
return cmpResult == APFloat::cmpUnordered || cmpResult == APFloat::cmpEqual;
case CmpFPredicate::UGT:
return cmpResult == APFloat::cmpUnordered ||
cmpResult == APFloat::cmpGreaterThan;
case CmpFPredicate::UGE:
return cmpResult == APFloat::cmpUnordered ||
cmpResult == APFloat::cmpGreaterThan ||
cmpResult == APFloat::cmpEqual;
case CmpFPredicate::ULT:
return cmpResult == APFloat::cmpUnordered ||
cmpResult == APFloat::cmpLessThan;
case CmpFPredicate::ULE:
return cmpResult == APFloat::cmpUnordered ||
cmpResult == APFloat::cmpLessThan || cmpResult == APFloat::cmpEqual;
case CmpFPredicate::UNE:
return cmpResult != APFloat::cmpEqual;
case CmpFPredicate::UNO:
return cmpResult == APFloat::cmpUnordered;
case CmpFPredicate::AlwaysTrue:
return true;
}
llvm_unreachable("unknown comparison predicate");
}
// Constant folding hook for comparisons.
OpFoldResult CmpFOp::fold(ArrayRef<Attribute> operands) {
assert(operands.size() == 2 && "cmpf takes two arguments");
auto lhs = operands.front().dyn_cast_or_null<FloatAttr>();
auto rhs = operands.back().dyn_cast_or_null<FloatAttr>();
// TODO: We could actually do some intelligent things if we know only one
// of the operands, but it's inf or nan.
if (!lhs || !rhs)
return {};
auto val = applyCmpPredicate(getPredicate(), lhs.getValue(), rhs.getValue());
return IntegerAttr::get(IntegerType::get(getContext(), 1), APInt(1, val));
}
//===----------------------------------------------------------------------===//
// CondBranchOp // CondBranchOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
namespace { namespace {
/// cond_br true, ^bb1, ^bb2 /// cond_br true, ^bb1, ^bb2
/// -> br ^bb1 /// -> br ^bb1
/// cond_br false, ^bb1, ^bb2 /// cond_br false, ^bb1, ^bb2
/// -> br ^bb2 /// -> br ^bb2
▲ Show 20 Lines • Show All 187 Lines • ▼ Show 20 Lines LogicalResult matchAndRewrite(CondBranchOp condbr,
// op. // op.
if (condbr.getTrueDest()->getSinglePredecessor()) { if (condbr.getTrueDest()->getSinglePredecessor()) {
for (OpOperand &use : for (OpOperand &use :
llvm::make_early_inc_range(condbr.condition().getUses())) { llvm::make_early_inc_range(condbr.condition().getUses())) {
if (use.getOwner()->getBlock() == condbr.getTrueDest()) { if (use.getOwner()->getBlock() == condbr.getTrueDest()) {
replaced = true; replaced = true;
if (!constantTrue) if (!constantTrue)
constantTrue = rewriter.create<mlir::ConstantOp>( constantTrue = rewriter.create<arith::ConstantOp>(
condbr.getLoc(), ty, rewriter.getBoolAttr(true)); condbr.getLoc(), ty, rewriter.getBoolAttr(true));
rewriter.updateRootInPlace(use.getOwner(), rewriter.updateRootInPlace(use.getOwner(),
[&] { use.set(constantTrue); }); [&] { use.set(constantTrue); });
} }
} }
} }
if (condbr.getFalseDest()->getSinglePredecessor()) { if (condbr.getFalseDest()->getSinglePredecessor()) {
for (OpOperand &use : for (OpOperand &use :
llvm::make_early_inc_range(condbr.condition().getUses())) { llvm::make_early_inc_range(condbr.condition().getUses())) {
if (use.getOwner()->getBlock() == condbr.getFalseDest()) { if (use.getOwner()->getBlock() == condbr.getFalseDest()) {
replaced = true; replaced = true;
if (!constantFalse) if (!constantFalse)
constantFalse = rewriter.create<mlir::ConstantOp>( constantFalse = rewriter.create<arith::ConstantOp>(
condbr.getLoc(), ty, rewriter.getBoolAttr(false)); condbr.getLoc(), ty, rewriter.getBoolAttr(false));
rewriter.updateRootInPlace(use.getOwner(), rewriter.updateRootInPlace(use.getOwner(),
[&] { use.set(constantFalse); }); [&] { use.set(constantFalse); });
} }
} }
} }
return success(replaced); return success(replaced);
Show All 18 Lines
Block *CondBranchOp::getSuccessorForOperands(ArrayRef<Attribute> operands) { Block *CondBranchOp::getSuccessorForOperands(ArrayRef<Attribute> operands) {
if (IntegerAttr condAttr = operands.front().dyn_cast_or_null<IntegerAttr>()) if (IntegerAttr condAttr = operands.front().dyn_cast_or_null<IntegerAttr>())
return condAttr.getValue().isOneValue() ? trueDest() : falseDest(); return condAttr.getValue().isOneValue() ? trueDest() : falseDest();
return nullptr; return nullptr;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Constant*Op // ConstantOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
static void print(OpAsmPrinter &p, ConstantOp &op) { static void print(OpAsmPrinter &p, ConstantOp &op) {
p << " "; p << " ";
p.printOptionalAttrDict(op->getAttrs(), /*elidedAttrs=*/{"value"}); p.printOptionalAttrDict(op->getAttrs(), /*elidedAttrs=*/{"value"});
if (op->getAttrs().size() > 1) if (op->getAttrs().size() > 1)
p << ' '; p << ' ';
Show All 30 Linesstatic LogicalResult verify(ConstantOp &op) {
if (!value) if (!value)
return op.emitOpError("requires a 'value' attribute"); return op.emitOpError("requires a 'value' attribute");
Type type = op.getType(); Type type = op.getType();
if (!value.getType().isa<NoneType>() && type != value.getType()) if (!value.getType().isa<NoneType>() && type != value.getType())
return op.emitOpError() << "requires attribute's type (" << value.getType() return op.emitOpError() << "requires attribute's type (" << value.getType()
<< ") to match op's return type (" << type << ")"; << ") to match op's return type (" << type << ")";
if (auto intAttr = value.dyn_cast<IntegerAttr>()) {
if (type.isa<IndexType>() || value.isa<BoolAttr>())
return success();
IntegerType intType = type.cast<IntegerType>();
if (!intType.isSignless())
return op.emitOpError("requires integer result types to be signless");
// If the type has a known bitwidth we verify that the value can be
// represented with the given bitwidth.
unsigned bitwidth = intType.getWidth();
APInt intVal = intAttr.getValue();
if (!intVal.isSignedIntN(bitwidth) && !intVal.isIntN(bitwidth))
return op.emitOpError("requires 'value' to be an integer within the "
"range of the integer result type");
return success();
}
if (auto complexTy = type.dyn_cast<ComplexType>()) { if (auto complexTy = type.dyn_cast<ComplexType>()) {
auto arrayAttr = value.dyn_cast<ArrayAttr>(); auto arrayAttr = value.dyn_cast<ArrayAttr>();
if (!complexTy || arrayAttr.size() != 2) if (!complexTy || arrayAttr.size() != 2)
return op.emitOpError( return op.emitOpError(
"requires 'value' to be a complex constant, represented as array of " "requires 'value' to be a complex constant, represented as array of "
"two values"); "two values");
auto complexEltTy = complexTy.getElementType(); auto complexEltTy = complexTy.getElementType();
if (complexEltTy != arrayAttr[0].getType() || if (complexEltTy != arrayAttr[0].getType() ||
complexEltTy != arrayAttr[1].getType()) { complexEltTy != arrayAttr[1].getType()) {
return op.emitOpError() return op.emitOpError()
<< "requires attribute's element types (" << arrayAttr[0].getType() << "requires attribute's element types (" << arrayAttr[0].getType()
<< ", " << arrayAttr[1].getType() << ", " << arrayAttr[1].getType()
<< ") to match the element type of the op's return type (" << ") to match the element type of the op's return type ("
<< complexEltTy << ")"; << complexEltTy << ")";
} }
return success(); return success();
} }
if (type.isa<FloatType>()) {
if (!value.isa<FloatAttr>())
return op.emitOpError("requires 'value' to be a floating point constant");
return success();
}
if (type.isa<ShapedType>()) {
if (!value.isa<ElementsAttr>())
return op.emitOpError("requires 'value' to be a shaped constant");
return success();
}
if (type.isa<FunctionType>()) { if (type.isa<FunctionType>()) {
auto fnAttr = value.dyn_cast<FlatSymbolRefAttr>(); auto fnAttr = value.dyn_cast<FlatSymbolRefAttr>();
if (!fnAttr) if (!fnAttr)
return op.emitOpError("requires 'value' to be a function reference"); return op.emitOpError("requires 'value' to be a function reference");
// Try to find the referenced function. // Try to find the referenced function.
auto fn = auto fn =
op->getParentOfType<ModuleOp>().lookupSymbol<FuncOp>(fnAttr.getValue()); op->getParentOfType<ModuleOp>().lookupSymbol<FuncOp>(fnAttr.getValue());
Show All 17 Lines
OpFoldResult ConstantOp::fold(ArrayRef<Attribute> operands) { OpFoldResult ConstantOp::fold(ArrayRef<Attribute> operands) {
assert(operands.empty() && "constant has no operands"); assert(operands.empty() && "constant has no operands");
return getValue(); return getValue();
} }
void ConstantOp::getAsmResultNames( void ConstantOp::getAsmResultNames(
function_ref<void(Value, StringRef)> setNameFn) { function_ref<void(Value, StringRef)> setNameFn) {
Type type = getType(); Type type = getType();
if (auto intCst = getValue().dyn_cast<IntegerAttr>()) { if (type.isa<FunctionType>()) {
IntegerType intTy = type.dyn_cast<IntegerType>();
// Sugar i1 constants with 'true' and 'false'.
if (intTy && intTy.getWidth() == 1)
return setNameFn(getResult(), (intCst.getInt() ? "true" : "false"));
// Otherwise, build a complex name with the value and type.
SmallString<32> specialNameBuffer;
llvm::raw_svector_ostream specialName(specialNameBuffer);
specialName << 'c' << intCst.getInt();
if (intTy)
specialName << '_' << type;
setNameFn(getResult(), specialName.str());
} else if (type.isa<FunctionType>()) {
setNameFn(getResult(), "f"); setNameFn(getResult(), "f");
} else { } else {
setNameFn(getResult(), "cst"); setNameFn(getResult(), "cst");
} }
} }
/// Returns true if a constant operation can be built with the given value and /// Returns true if a constant operation can be built with the given value and
/// result type. /// result type.
bool ConstantOp::isBuildableWith(Attribute value, Type type) { bool ConstantOp::isBuildableWith(Attribute value, Type type) {
// SymbolRefAttr can only be used with a function type. // SymbolRefAttr can only be used with a function type.
if (value.isa<SymbolRefAttr>()) if (value.isa<SymbolRefAttr>())
return type.isa<FunctionType>(); return type.isa<FunctionType>();
// The attribute must have the same type as 'type'. // The attribute must have the same type as 'type'.
if (!value.getType().isa<NoneType>() && value.getType() != type) if (!value.getType().isa<NoneType>() && value.getType() != type)
return false; return false;
// If the type is an integer type, it must be signless.
if (IntegerType integerTy = type.dyn_cast<IntegerType>())
if (!integerTy.isSignless())
return false;
// Finally, check that the attribute kind is handled. // Finally, check that the attribute kind is handled.
if (auto arrAttr = value.dyn_cast<ArrayAttr>()) { if (auto arrAttr = value.dyn_cast<ArrayAttr>()) {
auto complexTy = type.dyn_cast<ComplexType>(); auto complexTy = type.dyn_cast<ComplexType>();
if (!complexTy) if (!complexTy)
return false; return false;
auto complexEltTy = complexTy.getElementType(); auto complexEltTy = complexTy.getElementType();
return arrAttr.size() == 2 && arrAttr[0].getType() == complexEltTy && return arrAttr.size() == 2 && arrAttr[0].getType() == complexEltTy &&
arrAttr[1].getType() == complexEltTy; arrAttr[1].getType() == complexEltTy;
} }
return value.isa<IntegerAttr, FloatAttr, ElementsAttr, UnitAttr>(); return value.isa<UnitAttr>();
}
void ConstantFloatOp::build(OpBuilder &builder, OperationState &result,
const APFloat &value, FloatType type) {
ConstantOp::build(builder, result, type, builder.getFloatAttr(type, value));
}
bool ConstantFloatOp::classof(Operation *op) {
return ConstantOp::classof(op) && op->getResult(0).getType().isa<FloatType>();
}
/// ConstantIntOp only matches values whose result type is an IntegerType.
bool ConstantIntOp::classof(Operation *op) {
return ConstantOp::classof(op) &&
op->getResult(0).getType().isSignlessInteger();
}
void ConstantIntOp::build(OpBuilder &builder, OperationState &result,
int64_t value, unsigned width) {
Type type = builder.getIntegerType(width);
ConstantOp::build(builder, result, type, builder.getIntegerAttr(type, value));
}
/// Build a constant int op producing an integer with the specified type,
/// which must be an integer type.
void ConstantIntOp::build(OpBuilder &builder, OperationState &result,
int64_t value, Type type) {
assert(type.isSignlessInteger() &&
"ConstantIntOp can only have signless integer type");
ConstantOp::build(builder, result, type, builder.getIntegerAttr(type, value));
}
/// ConstantIndexOp only matches values whose result type is Index.
bool ConstantIndexOp::classof(Operation *op) {
return ConstantOp::classof(op) && op->getResult(0).getType().isIndex();
}
void ConstantIndexOp::build(OpBuilder &builder, OperationState &result,
int64_t value) {
Type type = builder.getIndexType();
ConstantOp::build(builder, result, type, builder.getIntegerAttr(type, value));
}
// ---------------------------------------------------------------------------
// DivFOp
// ---------------------------------------------------------------------------
OpFoldResult DivFOp::fold(ArrayRef<Attribute> operands) {
return constFoldBinaryOp<FloatAttr>(
operands, [](APFloat a, APFloat b) { return a / b; });
}
//===----------------------------------------------------------------------===//
// FPExtOp
//===----------------------------------------------------------------------===//
bool FPExtOp::areCastCompatible(TypeRange inputs, TypeRange outputs) {
if (inputs.size() != 1 || outputs.size() != 1)
return false;
Type a = inputs.front(), b = outputs.front();
if (auto fa = a.dyn_cast<FloatType>())
if (auto fb = b.dyn_cast<FloatType>())
return fa.getWidth() < fb.getWidth();
return areVectorCastSimpleCompatible(a, b, areCastCompatible);
}
//===----------------------------------------------------------------------===//
// FPToSIOp
//===----------------------------------------------------------------------===//
bool FPToSIOp::areCastCompatible(TypeRange inputs, TypeRange outputs) {
if (inputs.size() != 1 || outputs.size() != 1)
return false;
Type a = inputs.front(), b = outputs.front();
if (a.isa<FloatType>() && b.isSignlessInteger())
return true;
return areVectorCastSimpleCompatible(a, b, areCastCompatible);
}
//===----------------------------------------------------------------------===//
// FPToUIOp
//===----------------------------------------------------------------------===//
bool FPToUIOp::areCastCompatible(TypeRange inputs, TypeRange outputs) {
if (inputs.size() != 1 || outputs.size() != 1)
return false;
Type a = inputs.front(), b = outputs.front();
if (a.isa<FloatType>() && b.isSignlessInteger())
return true;
return areVectorCastSimpleCompatible(a, b, areCastCompatible);
}
//===----------------------------------------------------------------------===//
// FPTruncOp
//===----------------------------------------------------------------------===//
bool FPTruncOp::areCastCompatible(TypeRange inputs, TypeRange outputs) {
if (inputs.size() != 1 || outputs.size() != 1)
return false;
Type a = inputs.front(), b = outputs.front();
if (auto fa = a.dyn_cast<FloatType>())
if (auto fb = b.dyn_cast<FloatType>())
return fa.getWidth() > fb.getWidth();
return areVectorCastSimpleCompatible(a, b, areCastCompatible);
}
/// Perform safe const propagation for fptrunc, i.e. only propagate
/// if FP value can be represented without precision loss or rounding.
OpFoldResult FPTruncOp::fold(ArrayRef<Attribute> operands) {
assert(operands.size() == 1 && "unary operation takes one operand");
auto constOperand = operands.front();
if (!constOperand || !constOperand.isa<FloatAttr>())
return {};
// Convert to target type via 'double'.
double sourceValue =
constOperand.dyn_cast<FloatAttr>().getValue().convertToDouble();
auto targetAttr = FloatAttr::get(getType(), sourceValue);
// Propagate if constant's value does not change after truncation.
if (sourceValue == targetAttr.getValue().convertToDouble())
return targetAttr;
return {};
}
//===----------------------------------------------------------------------===//
// IndexCastOp
//===----------------------------------------------------------------------===//
// Index cast is applicable from index to integer and backwards.
bool IndexCastOp::areCastCompatible(TypeRange inputs, TypeRange outputs) {
if (inputs.size() != 1 || outputs.size() != 1)
return false;
Type a = inputs.front(), b = outputs.front();
if (a.isa<ShapedType>() && b.isa<ShapedType>()) {
auto aShaped = a.cast<ShapedType>();
auto bShaped = b.cast<ShapedType>();
return (aShaped.getShape() == bShaped.getShape()) &&
areCastCompatible(aShaped.getElementType(),
bShaped.getElementType());
}
return (a.isIndex() && b.isSignlessInteger()) ||
(a.isSignlessInteger() && b.isIndex());
}
OpFoldResult IndexCastOp::fold(ArrayRef<Attribute> cstOperands) {
// Fold IndexCast(IndexCast(x)) -> x
auto cast = getOperand().getDefiningOp<IndexCastOp>();
if (cast && cast.getOperand().getType() == getType())
return cast.getOperand();
// Fold IndexCast(constant) -> constant
// A little hack because we go through int. Otherwise, the size
// of the constant might need to change.
if (auto value = cstOperands[0].dyn_cast_or_null<IntegerAttr>())
return IntegerAttr::get(getType(), value.getInt());
return {};
}
namespace {
/// index_cast(sign_extend x) => index_cast(x)
struct IndexCastOfSExt : public OpRewritePattern<IndexCastOp> {
using OpRewritePattern<IndexCastOp>::OpRewritePattern;
LogicalResult matchAndRewrite(IndexCastOp op,
PatternRewriter &rewriter) const override {
if (auto extop = op.getOperand().getDefiningOp<SignExtendIOp>()) {
op.setOperand(extop.getOperand());
return success();
}
return failure();
}
};
} // namespace
void IndexCastOp::getCanonicalizationPatterns(OwningRewritePatternList &results,
MLIRContext *context) {
results.insert<IndexCastOfSExt>(context);
}
//===----------------------------------------------------------------------===//
// MulFOp
//===----------------------------------------------------------------------===//
OpFoldResult MulFOp::fold(ArrayRef<Attribute> operands) {
return constFoldBinaryOp<FloatAttr>(
operands, [](APFloat a, APFloat b) { return a * b; });
}
//===----------------------------------------------------------------------===//
// MulIOp
//===----------------------------------------------------------------------===//
OpFoldResult MulIOp::fold(ArrayRef<Attribute> operands) {
/// muli(x, 0) -> 0
if (matchPattern(rhs(), m_Zero()))
return rhs();
/// muli(x, 1) -> x
if (matchPattern(rhs(), m_One()))
return getOperand(0);
// TODO: Handle the overflow case.
return constFoldBinaryOp<IntegerAttr>(operands,
[](APInt a, APInt b) { return a * b; });
}
//===----------------------------------------------------------------------===//
// OrOp
//===----------------------------------------------------------------------===//
OpFoldResult OrOp::fold(ArrayRef<Attribute> operands) {
/// or(x, 0) -> x
if (matchPattern(rhs(), m_Zero()))
return lhs();
/// or(x, x) -> x
if (lhs() == rhs())
return rhs();
/// or(x, <all ones>) -> <all ones>
if (auto rhsAttr = operands[1].dyn_cast_or_null<IntegerAttr>())
if (rhsAttr.getValue().isAllOnes())
return rhsAttr;
return constFoldBinaryOp<IntegerAttr>(operands,
[](APInt a, APInt b) { return a | b; });
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// RankOp // RankOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
OpFoldResult RankOp::fold(ArrayRef<Attribute> operands) { OpFoldResult RankOp::fold(ArrayRef<Attribute> operands) {
// Constant fold rank when the rank of the operand is known. // Constant fold rank when the rank of the operand is known.
▲ Show 20 Lines • Show All 50 Lines • ▼ Show 20 Lines if (!matchPattern(op.getTrueValue(), m_Zero()))
return failure(); return failure();
if (!matchPattern(op.getFalseValue(), m_One())) if (!matchPattern(op.getFalseValue(), m_One()))
return failure(); return failure();
if (!op.getType().isInteger(1)) if (!op.getType().isInteger(1))
return failure(); return failure();
rewriter.replaceOpWithNewOp<XOrOp>(op, op.condition(), op.getFalseValue()); rewriter.replaceOpWithNewOp<arith::XOrIOp>(op, op.condition(),
op.getFalseValue());
return success(); return success();
} }
}; };
void SelectOp::getCanonicalizationPatterns(OwningRewritePatternList &results, void SelectOp::getCanonicalizationPatterns(OwningRewritePatternList &results,
MLIRContext *context) { MLIRContext *context) {
results.insert<SelectToNot>(context); results.insert<SelectToNot>(context);
} }
Show All 9 LinesOpFoldResult SelectOp::fold(ArrayRef<Attribute> operands) {
// select true, %0, %1 => %0 // select true, %0, %1 => %0
if (matchPattern(condition, m_One())) if (matchPattern(condition, m_One()))
return trueVal; return trueVal;
// select false, %0, %1 => %1 // select false, %0, %1 => %1
if (matchPattern(condition, m_Zero())) if (matchPattern(condition, m_Zero()))
return falseVal; return falseVal;
if (auto cmp = dyn_cast_or_null<CmpIOp>(condition.getDefiningOp())) { if (auto cmp = dyn_cast_or_null<arith::CmpIOp>(condition.getDefiningOp())) {
auto pred = cmp.predicate(); auto pred = cmp.predicate();
if (pred == mlir::CmpIPredicate::eq || pred == mlir::CmpIPredicate::ne) { if (pred == arith::CmpIPredicate::eq || pred == arith::CmpIPredicate::ne) {
auto cmpLhs = cmp.lhs(); auto cmpLhs = cmp.lhs();
auto cmpRhs = cmp.rhs(); auto cmpRhs = cmp.rhs();
// %0 = cmpi eq, %arg0, %arg1 // %0 = arith.cmpi eq, %arg0, %arg1
// %1 = select %0, %arg0, %arg1 => %arg1 // %1 = select %0, %arg0, %arg1 => %arg1
// %0 = cmpi ne, %arg0, %arg1 // %0 = arith.cmpi ne, %arg0, %arg1
// %1 = select %0, %arg0, %arg1 => %arg0 // %1 = select %0, %arg0, %arg1 => %arg0
if ((cmpLhs == trueVal && cmpRhs == falseVal) || if ((cmpLhs == trueVal && cmpRhs == falseVal) ||
(cmpRhs == trueVal && cmpLhs == falseVal)) (cmpRhs == trueVal && cmpLhs == falseVal))
return pred == mlir::CmpIPredicate::ne ? trueVal : falseVal; return pred == arith::CmpIPredicate::ne ? trueVal : falseVal;
} }
} }
return nullptr; return nullptr;
} }
static void print(OpAsmPrinter &p, SelectOp op) { static void print(OpAsmPrinter &p, SelectOp op) {
p << " " << op.getOperands(); p << " " << op.getOperands();
p.printOptionalAttrDict(op->getAttrs()); p.printOptionalAttrDict(op->getAttrs());
▲ Show 20 Lines • Show All 43 Lines • ▼ Show 20 Lines if (conditionType != shapedConditionType)
return op.emitOpError() return op.emitOpError()
<< "expected condition type to have the same shape " << "expected condition type to have the same shape "
"as the result type, expected " "as the result type, expected "
<< shapedConditionType << ", but got " << conditionType; << shapedConditionType << ", but got " << conditionType;
return success(); return success();
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// SignExtendIOp
//===----------------------------------------------------------------------===//
static LogicalResult verify(SignExtendIOp op) {
// Get the scalar type (which is either directly the type of the operand
// or the vector's/tensor's element type.
auto srcType = getElementTypeOrSelf(op.getOperand().getType());
auto dstType = getElementTypeOrSelf(op.getType());
// For now, index is forbidden for the source and the destination type.
if (srcType.isa<IndexType>())
return op.emitError() << srcType << " is not a valid operand type";
if (dstType.isa<IndexType>())
return op.emitError() << dstType << " is not a valid result type";
if (srcType.cast<IntegerType>().getWidth() >=
dstType.cast<IntegerType>().getWidth())
return op.emitError("result type ")
<< dstType << " must be wider than operand type " << srcType;
return success();
}
OpFoldResult SignExtendIOp::fold(ArrayRef<Attribute> operands) {
assert(operands.size() == 1 && "unary operation takes one operand");
if (!operands[0])
return {};
if (auto lhs = operands[0].dyn_cast<IntegerAttr>()) {
return IntegerAttr::get(
getType(), lhs.getValue().sext(getType().getIntOrFloatBitWidth()));
}
return {};
}
//===----------------------------------------------------------------------===//
// SignedDivIOp
//===----------------------------------------------------------------------===//
OpFoldResult SignedDivIOp::fold(ArrayRef<Attribute> operands) {
assert(operands.size() == 2 && "binary operation takes two operands");
// Don't fold if it would overflow or if it requires a division by zero.
bool overflowOrDiv0 = false;
auto result = constFoldBinaryOp<IntegerAttr>(operands, [&](APInt a, APInt b) {
if (overflowOrDiv0 || !b) {
overflowOrDiv0 = true;
return a;
}
return a.sdiv_ov(b, overflowOrDiv0);
});
// Fold out division by one. Assumes all tensors of all ones are splats.
if (auto rhs = operands[1].dyn_cast_or_null<IntegerAttr>()) {
if (rhs.getValue() == 1)
return lhs();
} else if (auto rhs = operands[1].dyn_cast_or_null<SplatElementsAttr>()) {
if (rhs.getSplatValue<IntegerAttr>().getValue() == 1)
return lhs();
}
return overflowOrDiv0 ? Attribute() : result;
}
//===----------------------------------------------------------------------===//
// SignedFloorDivIOp
//===----------------------------------------------------------------------===//
static APInt signedCeilNonnegInputs(APInt a, APInt b, bool &overflow) {
// Returns (a-1)/b + 1
APInt one(a.getBitWidth(), 1, true); // Signed value 1.
APInt val = a.ssub_ov(one, overflow).sdiv_ov(b, overflow);
return val.sadd_ov(one, overflow);
}
OpFoldResult SignedFloorDivIOp::fold(ArrayRef<Attribute> operands) {
assert(operands.size() == 2 && "binary operation takes two operands");
// Don't fold if it would overflow or if it requires a division by zero.
bool overflowOrDiv0 = false;
auto result = constFoldBinaryOp<IntegerAttr>(operands, [&](APInt a, APInt b) {
if (overflowOrDiv0 || !b) {
overflowOrDiv0 = true;
return a;
}
unsigned bits = a.getBitWidth();
APInt zero = APInt::getZero(bits);
if (a.sge(zero) && b.sgt(zero)) {
// Both positive (or a is zero), return a / b.
return a.sdiv_ov(b, overflowOrDiv0);
} else if (a.sle(zero) && b.slt(zero)) {
// Both negative (or a is zero), return -a / -b.
APInt posA = zero.ssub_ov(a, overflowOrDiv0);
APInt posB = zero.ssub_ov(b, overflowOrDiv0);
return posA.sdiv_ov(posB, overflowOrDiv0);
} else if (a.slt(zero) && b.sgt(zero)) {
// A is negative, b is positive, return - ceil(-a, b).
APInt posA = zero.ssub_ov(a, overflowOrDiv0);
APInt ceil = signedCeilNonnegInputs(posA, b, overflowOrDiv0);
return zero.ssub_ov(ceil, overflowOrDiv0);
} else {
// A is positive, b is negative, return - ceil(a, -b).
APInt posB = zero.ssub_ov(b, overflowOrDiv0);
APInt ceil = signedCeilNonnegInputs(a, posB, overflowOrDiv0);
return zero.ssub_ov(ceil, overflowOrDiv0);
}
});
// Fold out floor division by one. Assumes all tensors of all ones are
// splats.
if (auto rhs = operands[1].dyn_cast_or_null<IntegerAttr>()) {
if (rhs.getValue() == 1)
return lhs();
} else if (auto rhs = operands[1].dyn_cast_or_null<SplatElementsAttr>()) {
if (rhs.getSplatValue<IntegerAttr>().getValue() == 1)
return lhs();
}
return overflowOrDiv0 ? Attribute() : result;
}
//===----------------------------------------------------------------------===//
// SignedCeilDivIOp
//===----------------------------------------------------------------------===//
OpFoldResult SignedCeilDivIOp::fold(ArrayRef<Attribute> operands) {
assert(operands.size() == 2 && "binary operation takes two operands");
// Don't fold if it would overflow or if it requires a division by zero.
bool overflowOrDiv0 = false;
auto result = constFoldBinaryOp<IntegerAttr>(operands, [&](APInt a, APInt b) {
if (overflowOrDiv0 || !b) {
overflowOrDiv0 = true;
return a;
}
unsigned bits = a.getBitWidth();
APInt zero = APInt::getZero(bits);
if (a.sgt(zero) && b.sgt(zero)) {
// Both positive, return ceil(a, b).
return signedCeilNonnegInputs(a, b, overflowOrDiv0);
} else if (a.slt(zero) && b.slt(zero)) {
// Both negative, return ceil(-a, -b).
APInt posA = zero.ssub_ov(a, overflowOrDiv0);
APInt posB = zero.ssub_ov(b, overflowOrDiv0);
return signedCeilNonnegInputs(posA, posB, overflowOrDiv0);
} else if (a.slt(zero) && b.sgt(zero)) {
// A is negative, b is positive, return - ( -a / b).
APInt posA = zero.ssub_ov(a, overflowOrDiv0);
APInt div = posA.sdiv_ov(b, overflowOrDiv0);
return zero.ssub_ov(div, overflowOrDiv0);
} else {
// A is positive (or zero), b is negative, return - (a / -b).
APInt posB = zero.ssub_ov(b, overflowOrDiv0);
APInt div = a.sdiv_ov(posB, overflowOrDiv0);
return zero.ssub_ov(div, overflowOrDiv0);
}
});
// Fold out floor division by one. Assumes all tensors of all ones are
// splats.
if (auto rhs = operands[1].dyn_cast_or_null<IntegerAttr>()) {
if (rhs.getValue() == 1)
return lhs();
} else if (auto rhs = operands[1].dyn_cast_or_null<SplatElementsAttr>()) {
if (rhs.getSplatValue<IntegerAttr>().getValue() == 1)
return lhs();
}
return overflowOrDiv0 ? Attribute() : result;
}
//===----------------------------------------------------------------------===//
// SignedRemIOp
//===----------------------------------------------------------------------===//
OpFoldResult SignedRemIOp::fold(ArrayRef<Attribute> operands) {
assert(operands.size() == 2 && "remi_signed takes two operands");
auto rhs = operands.back().dyn_cast_or_null<IntegerAttr>();
if (!rhs)
return {};
auto rhsValue = rhs.getValue();
// x % 1 = 0
if (rhsValue.isOneValue())
return IntegerAttr::get(rhs.getType(), APInt(rhsValue.getBitWidth(), 0));
// Don't fold if it requires division by zero.
if (rhsValue.isNullValue())
return {};
auto lhs = operands.front().dyn_cast_or_null<IntegerAttr>();
if (!lhs)
return {};
return IntegerAttr::get(lhs.getType(), lhs.getValue().srem(rhsValue));
}
//===----------------------------------------------------------------------===//
// SIToFPOp
//===----------------------------------------------------------------------===//
// sitofp is applicable from integer types to float types.
bool SIToFPOp::areCastCompatible(TypeRange inputs, TypeRange outputs) {
if (inputs.size() != 1 || outputs.size() != 1)
return false;
Type a = inputs.front(), b = outputs.front();
if (a.isSignlessInteger() && b.isa<FloatType>())
return true;
return areVectorCastSimpleCompatible(a, b, areCastCompatible);
}
//===----------------------------------------------------------------------===//
// SplatOp // SplatOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
static LogicalResult verify(SplatOp op) { static LogicalResult verify(SplatOp op) {
// TODO: we could replace this by a trait. // TODO: we could replace this by a trait.
if (op.getOperand().getType() != if (op.getOperand().getType() !=
op.getType().cast<ShapedType>().getElementType()) op.getType().cast<ShapedType>().getElementType())
return op.emitError("operand should be of elemental type of result type"); return op.emitError("operand should be of elemental type of result type");
Show All 13 LinesOpFoldResult SplatOp::fold(ArrayRef<Attribute> operands) {
assert(shapedType.getElementType() == constOperand.getType() && assert(shapedType.getElementType() == constOperand.getType() &&
"incorrect input attribute type for folding"); "incorrect input attribute type for folding");
// SplatElementsAttr::get treats single value for second arg as being a splat. // SplatElementsAttr::get treats single value for second arg as being a splat.
return SplatElementsAttr::get(shapedType, {constOperand}); return SplatElementsAttr::get(shapedType, {constOperand});
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// SubFOp
//===----------------------------------------------------------------------===//
OpFoldResult SubFOp::fold(ArrayRef<Attribute> operands) {
return constFoldBinaryOp<FloatAttr>(
operands, [](APFloat a, APFloat b) { return a - b; });
}
//===----------------------------------------------------------------------===//
// SubIOp
//===----------------------------------------------------------------------===//
OpFoldResult SubIOp::fold(ArrayRef<Attribute> operands) {
// subi(x,x) -> 0
if (getOperand(0) == getOperand(1))
return Builder(getContext()).getZeroAttr(getType());
// subi(x,0) -> x
if (matchPattern(rhs(), m_Zero()))
return lhs();
return constFoldBinaryOp<IntegerAttr>(operands,
[](APInt a, APInt b) { return a - b; });
}
/// Canonicalize a sub of a constant and (constant +/- something) to simply be
/// a single operation that merges the two constants.
struct SubConstantReorder : public OpRewritePattern<SubIOp> {
using OpRewritePattern<SubIOp>::OpRewritePattern;
LogicalResult matchAndRewrite(SubIOp subOp,
PatternRewriter &rewriter) const override {
APInt origConst;
APInt midConst;
if (matchPattern(subOp.getOperand(0), m_ConstantInt(&origConst))) {
if (auto midAddOp = subOp.getOperand(1).getDefiningOp<AddIOp>()) {
// origConst - (midConst + something) == (origConst - midConst) -
// something
for (int j = 0; j < 2; j++) {
if (matchPattern(midAddOp.getOperand(j), m_ConstantInt(&midConst))) {
auto nextConstant = rewriter.create<ConstantOp>(
subOp.getLoc(),
rewriter.getIntegerAttr(subOp.getType(), origConst - midConst));
rewriter.replaceOpWithNewOp<SubIOp>(subOp, nextConstant,
midAddOp.getOperand(1 - j));
return success();
}
}
}
if (auto midSubOp = subOp.getOperand(0).getDefiningOp<SubIOp>()) {
if (matchPattern(midSubOp.getOperand(0), m_ConstantInt(&midConst))) {
// (midConst - something) - origConst == (midConst - origConst) -
// something
auto nextConstant = rewriter.create<ConstantOp>(
subOp.getLoc(),
rewriter.getIntegerAttr(subOp.getType(), midConst - origConst));
rewriter.replaceOpWithNewOp<SubIOp>(subOp, nextConstant,
midSubOp.getOperand(1));
return success();
}
if (matchPattern(midSubOp.getOperand(1), m_ConstantInt(&midConst))) {
// (something - midConst) - origConst == something - (origConst +
// midConst)
auto nextConstant = rewriter.create<ConstantOp>(
subOp.getLoc(),
rewriter.getIntegerAttr(subOp.getType(), origConst + midConst));
rewriter.replaceOpWithNewOp<SubIOp>(subOp, midSubOp.getOperand(0),
nextConstant);
return success();
}
}
if (auto midSubOp = subOp.getOperand(1).getDefiningOp<SubIOp>()) {
if (matchPattern(midSubOp.getOperand(0), m_ConstantInt(&midConst))) {
// origConst - (midConst - something) == (origConst - midConst) +
// something
auto nextConstant = rewriter.create<ConstantOp>(
subOp.getLoc(),
rewriter.getIntegerAttr(subOp.getType(), origConst - midConst));
rewriter.replaceOpWithNewOp<AddIOp>(subOp, nextConstant,
midSubOp.getOperand(1));
return success();
}
if (matchPattern(midSubOp.getOperand(1), m_ConstantInt(&midConst))) {
// origConst - (something - midConst) == (origConst + midConst) -
// something
auto nextConstant = rewriter.create<ConstantOp>(
subOp.getLoc(),
rewriter.getIntegerAttr(subOp.getType(), origConst + midConst));
rewriter.replaceOpWithNewOp<SubIOp>(subOp, nextConstant,
midSubOp.getOperand(0));
return success();
}
}
}
if (matchPattern(subOp.getOperand(1), m_ConstantInt(&origConst))) {
if (auto midAddOp = subOp.getOperand(0).getDefiningOp<AddIOp>()) {
// (midConst + something) - origConst == (midConst - origConst) +
// something
for (int j = 0; j < 2; j++) {
if (matchPattern(midAddOp.getOperand(j), m_ConstantInt(&midConst))) {
auto nextConstant = rewriter.create<ConstantOp>(
subOp.getLoc(),
rewriter.getIntegerAttr(subOp.getType(), midConst - origConst));
rewriter.replaceOpWithNewOp<AddIOp>(subOp, nextConstant,
midAddOp.getOperand(1 - j));
return success();
}
}
}
if (auto midSubOp = subOp.getOperand(0).getDefiningOp<SubIOp>()) {
if (matchPattern(midSubOp.getOperand(0), m_ConstantInt(&midConst))) {
// (midConst - something) - origConst == (midConst - origConst) -
// something
auto nextConstant = rewriter.create<ConstantOp>(
subOp.getLoc(),
rewriter.getIntegerAttr(subOp.getType(), midConst - origConst));
rewriter.replaceOpWithNewOp<SubIOp>(subOp, nextConstant,
midSubOp.getOperand(1));
return success();
}
if (matchPattern(midSubOp.getOperand(1), m_ConstantInt(&midConst))) {
// (something - midConst) - origConst == something - (midConst +
// origConst)
auto nextConstant = rewriter.create<ConstantOp>(
subOp.getLoc(),
rewriter.getIntegerAttr(subOp.getType(), midConst + origConst));
rewriter.replaceOpWithNewOp<SubIOp>(subOp, midSubOp.getOperand(0),
nextConstant);
return success();
}
}
if (auto midSubOp = subOp.getOperand(1).getDefiningOp<SubIOp>()) {
if (matchPattern(midSubOp.getOperand(0), m_ConstantInt(&midConst))) {
// origConst - (midConst - something) == (origConst - midConst) +
// something
auto nextConstant = rewriter.create<ConstantOp>(
subOp.getLoc(),
rewriter.getIntegerAttr(subOp.getType(), origConst - midConst));
rewriter.replaceOpWithNewOp<AddIOp>(subOp, nextConstant,
midSubOp.getOperand(1));
return success();
}
if (matchPattern(midSubOp.getOperand(1), m_ConstantInt(&midConst))) {
// origConst - (something - midConst) == (origConst - midConst) -
// something
auto nextConstant = rewriter.create<ConstantOp>(
subOp.getLoc(),
rewriter.getIntegerAttr(subOp.getType(), origConst - midConst));
rewriter.replaceOpWithNewOp<SubIOp>(subOp, nextConstant,
midSubOp.getOperand(0));
return success();
}
}
}
return failure();
}
};
void SubIOp::getCanonicalizationPatterns(OwningRewritePatternList &results,
MLIRContext *context) {
results.insert<SubConstantReorder>(context);
}
//===----------------------------------------------------------------------===//
// UIToFPOp
//===----------------------------------------------------------------------===//
// uitofp is applicable from integer types to float types.
bool UIToFPOp::areCastCompatible(TypeRange inputs, TypeRange outputs) {
if (inputs.size() != 1 || outputs.size() != 1)
return false;
Type a = inputs.front(), b = outputs.front();
if (a.isSignlessInteger() && b.isa<FloatType>())
return true;
return areVectorCastSimpleCompatible(a, b, areCastCompatible);
}
//===----------------------------------------------------------------------===//
// SwitchOp // SwitchOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
void SwitchOp::build(OpBuilder &builder, OperationState &result, Value value, void SwitchOp::build(OpBuilder &builder, OperationState &result, Value value,
Block *defaultDestination, ValueRange defaultOperands, Block *defaultDestination, ValueRange defaultOperands,
DenseIntElementsAttr caseValues, DenseIntElementsAttr caseValues,
BlockRange caseDestinations, BlockRange caseDestinations,
ArrayRef<ValueRange> caseOperands) { ArrayRef<ValueRange> caseOperands) {
▲ Show 20 Lines • Show All 421 Lines • ▼ Show 20 Lines results.add(&simplifySwitchWithOnlyDefault)
.add(&dropSwitchCasesThatMatchDefault) .add(&dropSwitchCasesThatMatchDefault)
.add(&simplifyConstSwitchValue) .add(&simplifyConstSwitchValue)
.add(&simplifyPassThroughSwitch) .add(&simplifyPassThroughSwitch)
.add(&simplifySwitchFromSwitchOnSameCondition) .add(&simplifySwitchFromSwitchOnSameCondition)
.add(&simplifySwitchFromDefaultSwitchOnSameCondition); .add(&simplifySwitchFromDefaultSwitchOnSameCondition);
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// TruncateIOp
//===----------------------------------------------------------------------===//
static LogicalResult verify(TruncateIOp op) {
auto srcType = getElementTypeOrSelf(op.getOperand().getType());
auto dstType = getElementTypeOrSelf(op.getType());
if (srcType.isa<IndexType>())
return op.emitError() << srcType << " is not a valid operand type";
if (dstType.isa<IndexType>())
return op.emitError() << dstType << " is not a valid result type";
if (srcType.cast<IntegerType>().getWidth() <=
dstType.cast<IntegerType>().getWidth())
return op.emitError("operand type ")
<< srcType << " must be wider than result type " << dstType;
return success();
}
OpFoldResult TruncateIOp::fold(ArrayRef<Attribute> operands) {
// trunci(zexti(a)) -> a
// trunci(sexti(a)) -> a
if (matchPattern(getOperand(), m_Op<ZeroExtendIOp>()) ||
matchPattern(getOperand(), m_Op<SignExtendIOp>()))
return getOperand().getDefiningOp()->getOperand(0);
assert(operands.size() == 1 && "unary operation takes one operand");
if (!operands[0])
return {};
if (auto lhs = operands[0].dyn_cast<IntegerAttr>()) {
return IntegerAttr::get(
getType(), lhs.getValue().trunc(getType().getIntOrFloatBitWidth()));
}
return {};
}
//===----------------------------------------------------------------------===//
// UnsignedDivIOp
//===----------------------------------------------------------------------===//
OpFoldResult UnsignedDivIOp::fold(ArrayRef<Attribute> operands) {
assert(operands.size() == 2 && "binary operation takes two operands");
// Don't fold if it would require a division by zero.
bool div0 = false;
auto result = constFoldBinaryOp<IntegerAttr>(operands, [&](APInt a, APInt b) {
if (div0 || !b) {
div0 = true;
return a;
}
return a.udiv(b);
});
// Fold out division by one. Assumes all tensors of all ones are splats.
if (auto rhs = operands[1].dyn_cast_or_null<IntegerAttr>()) {
if (rhs.getValue() == 1)
return lhs();
} else if (auto rhs = operands[1].dyn_cast_or_null<SplatElementsAttr>()) {
if (rhs.getSplatValue<IntegerAttr>().getValue() == 1)
return lhs();
}
return div0 ? Attribute() : result;
}
//===----------------------------------------------------------------------===//
// UnsignedRemIOp
//===----------------------------------------------------------------------===//
OpFoldResult UnsignedRemIOp::fold(ArrayRef<Attribute> operands) {
assert(operands.size() == 2 && "remi_unsigned takes two operands");
auto rhs = operands.back().dyn_cast_or_null<IntegerAttr>();
if (!rhs)
return {};
auto rhsValue = rhs.getValue();
// x % 1 = 0
if (rhsValue.isOneValue())
return IntegerAttr::get(rhs.getType(), APInt(rhsValue.getBitWidth(), 0));
// Don't fold if it requires division by zero.
if (rhsValue.isNullValue())
return {};
auto lhs = operands.front().dyn_cast_or_null<IntegerAttr>();
if (!lhs)
return {};
return IntegerAttr::get(lhs.getType(), lhs.getValue().urem(rhsValue));
}
//===----------------------------------------------------------------------===//
// XOrOp
//===----------------------------------------------------------------------===//
OpFoldResult XOrOp::fold(ArrayRef<Attribute> operands) {
/// xor(x, 0) -> x
if (matchPattern(rhs(), m_Zero()))
return lhs();
/// xor(x,x) -> 0
if (lhs() == rhs())
return Builder(getContext()).getZeroAttr(getType());
return constFoldBinaryOp<IntegerAttr>(operands,
[](APInt a, APInt b) { return a ^ b; });
}
namespace {
/// Replace a not of a comparison operation, for example: not(cmp eq A, B) =>
/// cmp ne A, B. Note that a logical not is implemented as xor 1, val.
struct NotICmp : public OpRewritePattern<XOrOp> {
using OpRewritePattern<XOrOp>::OpRewritePattern;
LogicalResult matchAndRewrite(XOrOp op,
PatternRewriter &rewriter) const override {
// Commutative ops (such as xor) have the constant appear second, which
// we assume here.
APInt constValue;
if (!matchPattern(op.getOperand(1), m_ConstantInt(&constValue)))
return failure();
if (constValue != 1)
return failure();
auto prev = op.getOperand(0).getDefiningOp<CmpIOp>();
if (!prev)
return failure();
switch (prev.predicate()) {
case CmpIPredicate::eq:
rewriter.replaceOpWithNewOp<CmpIOp>(op, CmpIPredicate::ne, prev.lhs(),
prev.rhs());
return success();
case CmpIPredicate::ne:
rewriter.replaceOpWithNewOp<CmpIOp>(op, CmpIPredicate::eq, prev.lhs(),
prev.rhs());
return success();
case CmpIPredicate::slt:
rewriter.replaceOpWithNewOp<CmpIOp>(op, CmpIPredicate::sge, prev.lhs(),
prev.rhs());
return success();
case CmpIPredicate::sle:
rewriter.replaceOpWithNewOp<CmpIOp>(op, CmpIPredicate::sgt, prev.lhs(),
prev.rhs());
return success();
case CmpIPredicate::sgt:
rewriter.replaceOpWithNewOp<CmpIOp>(op, CmpIPredicate::sle, prev.lhs(),
prev.rhs());
return success();
case CmpIPredicate::sge:
rewriter.replaceOpWithNewOp<CmpIOp>(op, CmpIPredicate::slt, prev.lhs(),
prev.rhs());
return success();
case CmpIPredicate::ult:
rewriter.replaceOpWithNewOp<CmpIOp>(op, CmpIPredicate::uge, prev.lhs(),
prev.rhs());
return success();
case CmpIPredicate::ule:
rewriter.replaceOpWithNewOp<CmpIOp>(op, CmpIPredicate::ugt, prev.lhs(),
prev.rhs());
return success();
case CmpIPredicate::ugt:
rewriter.replaceOpWithNewOp<CmpIOp>(op, CmpIPredicate::ule, prev.lhs(),
prev.rhs());
return success();
case CmpIPredicate::uge:
rewriter.replaceOpWithNewOp<CmpIOp>(op, CmpIPredicate::ult, prev.lhs(),
prev.rhs());
return success();
}
return failure();
}
};
} // namespace
void XOrOp::getCanonicalizationPatterns(OwningRewritePatternList &results,
MLIRContext *context) {
results.insert<NotICmp>(context);
}
//===----------------------------------------------------------------------===//
// ZeroExtendIOp
//===----------------------------------------------------------------------===//
static LogicalResult verify(ZeroExtendIOp op) {
auto srcType = getElementTypeOrSelf(op.getOperand().getType());
auto dstType = getElementTypeOrSelf(op.getType());
if (srcType.isa<IndexType>())
return op.emitError() << srcType << " is not a valid operand type";
if (dstType.isa<IndexType>())
return op.emitError() << dstType << " is not a valid result type";
if (srcType.cast<IntegerType>().getWidth() >=
dstType.cast<IntegerType>().getWidth())
return op.emitError("result type ")
<< dstType << " must be wider than operand type " << srcType;
return success();
}
//===----------------------------------------------------------------------===//
// TableGen'd op method definitions // TableGen'd op method definitions
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#define GET_OP_CLASSES #define GET_OP_CLASSES
#include "mlir/Dialect/StandardOps/IR/Ops.cpp.inc" #include "mlir/Dialect/StandardOps/IR/Ops.cpp.inc"

mlir/lib/Dialect/StandardOps/Transforms/Bufferize.cpp

mlir/lib/Dialect/StandardOps/Transforms/CMakeLists.txt

mlir/lib/Dialect/StandardOps/Transforms/ExpandOps.cpp

mlir/lib/Dialect/StandardOps/Transforms/TensorConstantBufferize.cpp

mlir/lib/Dialect/StandardOps/Utils/Utils.cpp

mlir/lib/Dialect/Tensor/IR/CMakeLists.txt

mlir/lib/Dialect/Tensor/IR/TensorOps.cpp

mlir/lib/Dialect/Tensor/Transforms/Bufferize.cpp

mlir/lib/Dialect/Tensor/Transforms/CMakeLists.txt

mlir/lib/Dialect/Vector/CMakeLists.txt

mlir/lib/Dialect/Vector/VectorMultiDimReductionTransforms.cpp

mlir/lib/Dialect/Vector/VectorOps.cpp

mlir/lib/Dialect/Vector/VectorTransforms.cpp

mlir/lib/Dialect/Vector/VectorUtils.cpp

mlir/lib/Dialect/X86Vector/Transforms/CMakeLists.txt

mlir/lib/Dialect/X86Vector/Transforms/LegalizeForLLVMExport.cpp

mlir/lib/Target/Cpp/CMakeLists.txt

mlir/lib/Target/Cpp/TranslateRegistration.cpp

mlir/lib/Target/Cpp/TranslateToCpp.cpp

mlir/lib/Transforms/LoopCoalescing.cpp

mlir/lib/Transforms/PassDetail.h

mlir/lib/Transforms/Utils/CMakeLists.txt

mlir/lib/Transforms/Utils/FoldUtils.cpp

//===- FoldUtils.cpp ---- Fold Utilities ----------------------------------===// //===- FoldUtils.cpp ---- Fold Utilities ----------------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file defines various operation fold utilities. These utilities are // This file defines various operation fold utilities. These utilities are
// intended to be used by passes to unify and simply their logic. // intended to be used by passes to unify and simply their logic.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Transforms/FoldUtils.h" #include "mlir/Transforms/FoldUtils.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
rriddleUnsubmitted
Done
ReplyInline Actions

Are either of these headers(Arith and Standard) necessary now?

If not can you see about closing https://llvm.org/PR48490?

rriddle: Are either of these headers(Arith and Standard) necessary now? If not can you see about…
rriddleUnsubmitted
Done
ReplyInline Actions

Sorry linked the wrong bug, meant closing PR44866.

rriddle: Sorry linked the wrong bug, meant closing PR44866.
#include "mlir/IR/Matchers.h" #include "mlir/IR/Matchers.h"
#include "mlir/IR/Operation.h" #include "mlir/IR/Operation.h"
using namespace mlir; using namespace mlir;
/// Given an operation, find the parent region that folded constants should be /// Given an operation, find the parent region that folded constants should be
/// inserted into. /// inserted into.
static Region * static Region *
Show All 30 Linesstatic Operation *materializeConstant(Dialect *dialect, OpBuilder &builder,
// Ask the dialect to materialize a constant operation for this value. // Ask the dialect to materialize a constant operation for this value.
if (auto *constOp = dialect->materializeConstant(builder, value, type, loc)) { if (auto *constOp = dialect->materializeConstant(builder, value, type, loc)) {
assert(insertPt == builder.getInsertionPoint()); assert(insertPt == builder.getInsertionPoint());
assert(matchPattern(constOp, m_Constant())); assert(matchPattern(constOp, m_Constant()));
return constOp; return constOp;
} }
// TODO: To facilitate splitting the std dialect (PR48490), have a special
// case for falling back to std.constant. Eventually, we will have separate
// ops tensor.constant, int.constant, float.constant, etc. that live in their
// respective dialects, which will allow each dialect to implement the
// materializeConstant hook above.
//
// The special case is needed because in the interim state while we are
// splitting out those dialects from std, the std dialect depends on the
// tensor dialect, which makes it impossible for the tensor dialect to use
// std.constant (it would be a cyclic dependency) as part of its
// materializeConstant hook.
//
// If the dialect is unable to materialize a constant, check to see if the
// standard constant can be used.
if (ConstantOp::isBuildableWith(value, type))
return builder.create<ConstantOp>(loc, type, value);
return nullptr; return nullptr;
} }
rriddleUnsubmitted
Done
ReplyInline Actions

Can you use the dialect hook instead? The only reason ConstantOp is here is because the standard dialect hasn't been split yet.

rriddle: Can you use the dialect hook instead? The only reason ConstantOp is here is because the…
MogballAuthorUnsubmitted
Done
ReplyInline Actions

Yeah this is part of the "cleanup" I still have to do.

Mogball: Yeah this is part of the "cleanup" I still have to do.
rriddleUnsubmitted
Done
ReplyInline Actions

Is this done or still pending?

rriddle: Is this done or still pending?
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// OperationFolder // OperationFolder
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
LogicalResult OperationFolder::tryToFold( LogicalResult OperationFolder::tryToFold(
Operation *op, function_ref<void(Operation *)> processGeneratedConstants, Operation *op, function_ref<void(Operation *)> processGeneratedConstants,
function_ref<void(Operation *)> preReplaceAction, bool *inPlaceUpdate) { function_ref<void(Operation *)> preReplaceAction, bool *inPlaceUpdate) {
▲ Show 20 Lines • Show All 214 Lines • Show Last 20 Lines

mlir/lib/Transforms/Utils/LoopUtils.cpp

mlir/lib/Transforms/Utils/Utils.cpp

mlir/python/CMakeLists.txt

mlir/python/mlir/dialects/ArithmeticOps.td

mlir/python/mlir/dialects/_arith_ops_ext.py

mlir/python/mlir/dialects/_std_ops_ext.py

mlir/python/mlir/dialects/arith.py

mlir/python/mlir/dialects/linalg/opdsl/lang/emitter.py

mlir/test/Analysis/test-alias-analysis.mlir

mlir/test/Analysis/test-dominance.mlir

mlir/test/Analysis/test-liveness.mlir

mlir/test/Analysis/test-match-reduction.mlir

mlir/test/Analysis/test-number-of-block-executions.mlir

mlir/test/Analysis/test-number-of-operation-executions.mlir

mlir/test/CAPI/execution_engine.c

mlir/test/CAPI/ir.c

mlir/test/CAPI/pass.c

mlir/test/Conversion/AffineToStandard/lower-affine-to-vector.mlir

mlir/test/Conversion/AffineToStandard/lower-affine.mlir

mlir/test/Conversion/ArithmeticToLLVM/arith-to-llvm.mlir

mlir/test/Conversion/ArithmeticToLLVM/convert-nd-vector-to-llvmir.mlir

mlir/test/Conversion/ArithmeticToSPIRV/arithmetic-to-spirv.mlir

mlir/test/Conversion/AsyncToLLVM/convert-runtime-to-llvm.mlir

mlir/test/Conversion/AsyncToLLVM/convert-to-llvm.mlir

mlir/test/Conversion/ComplexToLLVM/convert-to-llvm.mlir

mlir/test/Conversion/ComplexToStandard/convert-to-standard.mlir

mlir/test/Conversion/ComplexToStandard/full-conversion.mlir

mlir/test/Conversion/GPUCommon/lower-launch-func-to-gpu-runtime-calls.mlir

mlir/test/Conversion/GPUCommon/memory-attrbution.mlir

mlir/test/Conversion/GPUToNVVM/gpu-to-nvvm.mlir

mlir/test/Conversion/GPUToNVVM/wmma-ops-to-nvvm.mlir

mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl.mlir

mlir/test/Conversion/GPUToSPIRV/builtins.mlir

mlir/test/Conversion/GPUToSPIRV/load-store.mlir

mlir/test/Conversion/GPUToSPIRV/module-structure-opencl.mlir

mlir/test/Conversion/GPUToSPIRV/simple.mlir

mlir/test/Conversion/GPUToVulkan/lower-gpu-launch-vulkan-launch.mlir

mlir/test/Conversion/LinalgToSPIRV/linalg-to-spirv.mlir

mlir/test/Conversion/LinalgToVector/linalg-to-vector.mlir

mlir/test/Conversion/MathToLibm/convert-to-libm.mlir

mlir/test/Conversion/MemRefToLLVM/convert-alloca-scope.mlir

mlir/test/Conversion/MemRefToLLVM/convert-dynamic-memref-ops.mlir

mlir/test/Conversion/MemRefToLLVM/convert-static-memref-ops.mlir

mlir/test/Conversion/MemRefToLLVM/memref-to-llvm.mlir

mlir/test/Conversion/MemRefToSPIRV/memref-to-spirv.mlir

mlir/test/Conversion/OpenACCToLLVM/convert-data-operands-to-llvmir.mlir

mlir/test/Conversion/OpenMPToLLVM/convert-to-llvmir.mlir

mlir/test/Conversion/SCFToGPU/no_blocks_no_threads.mlir

mlir/test/Conversion/SCFToGPU/parallel_loop.mlir

mlir/test/Conversion/SCFToGPU/step_one.mlir

mlir/test/Conversion/SCFToGPU/step_positive.mlir

mlir/test/Conversion/SCFToOpenMP/reductions.mlir

mlir/test/Conversion/SCFToSPIRV/for.mlir

mlir/test/Conversion/SCFToSPIRV/if.mlir

mlir/test/Conversion/SCFToStandard/convert-to-cfg.mlir

mlir/test/Conversion/SPIRVToLLVM/lower-host-to-llvm-calls.mlir

mlir/test/Conversion/ShapeToStandard/shape-to-standard.mlir

mlir/test/Conversion/StandardToLLVM/convert-nd-vector-to-llvmir.mlir

mlir/test/Conversion/StandardToLLVM/func-memref-return.mlir

mlir/test/Conversion/StandardToLLVM/func-memref.mlir

mlir/test/Conversion/StandardToLLVM/rank.mlir

mlir/test/Conversion/StandardToLLVM/standard-to-llvm.mlir

mlir/test/Conversion/StandardToSPIRV/std-ops-to-spirv.mlir

mlir/test/Conversion/TosaToLinalg/tosa-to-linalg.mlir

mlir/test/Conversion/TosaToStandard/tosa-to-standard.mlir

mlir/test/Conversion/VectorToGPU/vector-to-mma-ops.mlir

mlir/test/Conversion/VectorToLLVM/vector-mask-to-llvm.mlir

mlir/test/Conversion/VectorToLLVM/vector-to-llvm.mlir

mlir/test/Conversion/VectorToROCDL/vector-to-rocdl.mlir

mlir/test/Conversion/VectorToSCF/tensor-transfer-ops.mlir

mlir/test/Conversion/VectorToSCF/unrolled-tensor-transfer-ops.mlir

mlir/test/Conversion/VectorToSCF/unrolled-vector-to-loops.mlir

mlir/test/Conversion/VectorToSCF/vector-to-scf-mask-and-permutation-map.mlir

mlir/test/Conversion/VectorToSCF/vector-to-scf.mlir

mlir/test/Dialect/AMX/invalid.mlir

mlir/test/Dialect/AMX/legalize-for-llvm.mlir

mlir/test/Dialect/AMX/roundtrip.mlir

mlir/test/Dialect/Affine/SuperVectorize/uniform_divergent.mlir

mlir/test/Dialect/Affine/SuperVectorize/vector_utils.mlir

mlir/test/Dialect/Affine/SuperVectorize/vectorize_1d.mlir

mlir/test/Dialect/Affine/SuperVectorize/vectorize_2d.mlir

mlir/test/Dialect/Affine/SuperVectorize/vectorize_3d.mlir

mlir/test/Dialect/Affine/SuperVectorize/vectorize_outer_loop_2d.mlir

mlir/test/Dialect/Affine/SuperVectorize/vectorize_outer_loop_transpose_2d.mlir

mlir/test/Dialect/Affine/SuperVectorize/vectorize_reduction.mlir

mlir/test/Dialect/Affine/SuperVectorize/vectorize_reduction_2d.mlir

mlir/test/Dialect/Affine/SuperVectorize/vectorize_transpose_2d.mlir

mlir/test/Dialect/Affine/affine-data-copy.mlir

mlir/test/Dialect/Affine/affine-loop-invariant-code-motion.mlir

mlir/test/Dialect/Affine/affine-loop-normalize.mlir

mlir/test/Dialect/Affine/canonicalize.mlir

mlir/test/Dialect/Affine/constant-fold.mlir

mlir/test/Dialect/Affine/dma-generate.mlir

mlir/test/Dialect/Affine/dma.mlir

mlir/test/Dialect/Affine/inlining.mlir

mlir/test/Dialect/Affine/invalid.mlir

mlir/test/Dialect/Affine/load-store-invalid.mlir

mlir/test/Dialect/Affine/loop-tiling-parametric.mlir

mlir/test/Dialect/Affine/loop-tiling-unsupported.mlir

mlir/test/Dialect/Affine/loop-tiling-validity.mlir

mlir/test/Dialect/Affine/loop-tiling.mlir

mlir/test/Dialect/Affine/ops.mlir

mlir/test/Dialect/Affine/parallelize.mlir

mlir/test/Dialect/Affine/scalrep.mlir

mlir/test/Dialect/Affine/simplify-affine-structures.mlir

mlir/test/Dialect/Affine/slicing-utils.mlir

mlir/test/Dialect/Affine/unroll-jam.mlir

mlir/test/Dialect/Affine/unroll.mlir

mlir/test/Dialect/Arithmetic/bufferize.mlir

mlir/test/Dialect/Arithmetic/canonicalize.mlir

mlir/test/Dialect/Arithmetic/expand-ops.mlir

mlir/test/Dialect/Arithmetic/invalid.mlir

mlir/test/Dialect/Arithmetic/ops.mlir

mlir/test/Dialect/ArmSVE/memcpy.mlir

mlir/test/Dialect/ArmSVE/roundtrip.mlir

mlir/test/Dialect/Async/async-parallel-for-async-dispatch.mlir

mlir/test/Dialect/Async/async-parallel-for-canonicalize.mlir

mlir/test/Dialect/Async/async-parallel-for-compute-fn.mlir

mlir/test/Dialect/Async/async-parallel-for-seq-dispatch.mlir

mlir/test/Dialect/Async/async-runtime-ref-counting.mlir

mlir/test/Dialect/Async/async-to-async-runtime-eliminate-blocking.mlir

mlir/test/Dialect/Async/async-to-async-runtime.mlir

mlir/test/Dialect/Async/ops.mlir

mlir/test/Dialect/Async/runtime.mlir

mlir/test/Dialect/Complex/canonicalize.mlir

mlir/test/Dialect/GPU/all-reduce-max.mlir

mlir/test/Dialect/GPU/all-reduce.mlir

mlir/test/Dialect/GPU/canonicalize.mlir

mlir/test/Dialect/GPU/invalid.mlir

mlir/test/Dialect/GPU/mapping.mlir

mlir/test/Dialect/GPU/multiple-all-reduce.mlir

mlir/test/Dialect/GPU/ops.mlir

mlir/test/Dialect/GPU/outlining.mlir

mlir/test/Dialect/GPU/promotion.mlir

mlir/test/Dialect/LLVMIR/canonicalize.mlir

mlir/test/Dialect/Linalg/affine.mlir

mlir/test/Dialect/Linalg/bufferize.mlir

mlir/test/Dialect/Linalg/canonicalize-duplicate-inputs.mlir

mlir/test/Dialect/Linalg/canonicalize.mlir

mlir/test/Dialect/Linalg/comprehensive-bufferize-analysis-2fill-extract-matmul-all-perms.mlir

mlir/test/Dialect/Linalg/comprehensive-module-bufferize-analysis.mlir

mlir/test/Dialect/Linalg/comprehensive-module-bufferize-invalid.mlir

mlir/test/Dialect/Linalg/comprehensive-module-bufferize.mlir

mlir/test/Dialect/Linalg/convert-elementwise-to-linalg.mlir

mlir/test/Dialect/Linalg/detensorize_0d.mlir

mlir/test/Dialect/Linalg/detensorize_br_operands.mlir

mlir/test/Dialect/Linalg/detensorize_if.mlir

mlir/test/Dialect/Linalg/detensorize_trivial.mlir

mlir/test/Dialect/Linalg/detensorize_while.mlir

mlir/test/Dialect/Linalg/detensorize_while_impure_cf.mlir

mlir/test/Dialect/Linalg/detensorize_while_pure_cf.mlir

mlir/test/Dialect/Linalg/distribute-tiled-loop.mlir

mlir/test/Dialect/Linalg/drop-unit-extent-dims.mlir

mlir/test/Dialect/Linalg/forward-vector-transfers.mlir

mlir/test/Dialect/Linalg/fusion-2-level.mlir

mlir/test/Dialect/Linalg/fusion-elementwise-ops.mlir

mlir/test/Dialect/Linalg/fusion-elementwise-options.mlir

mlir/test/Dialect/Linalg/fusion-indexed.mlir

mlir/test/Dialect/Linalg/fusion-pattern.mlir

mlir/test/Dialect/Linalg/fusion-push-reshape.mlir

mlir/test/Dialect/Linalg/fusion-sequence.mlir

mlir/test/Dialect/Linalg/fusion-tensor-pattern.mlir

mlir/test/Dialect/Linalg/fusion.mlir

mlir/test/Dialect/Linalg/generalize-named-ops.mlir

mlir/test/Dialect/Linalg/generalize-named-polymorphic-ops.mlir

mlir/test/Dialect/Linalg/generalize-pad-tensor.mlir

mlir/test/Dialect/Linalg/hoist-padding.mlir

mlir/test/Dialect/Linalg/hoisting.mlir

mlir/test/Dialect/Linalg/inline-scalar-operands.mlir

mlir/test/Dialect/Linalg/interchange.mlir

mlir/test/Dialect/Linalg/invalid.mlir

mlir/test/Dialect/Linalg/library-calls.mlir

mlir/test/Dialect/Linalg/llvm.mlir

mlir/test/Dialect/Linalg/loops.mlir

mlir/test/Dialect/Linalg/lower-pad-tensor.mlir

mlir/test/Dialect/Linalg/named-ops.mlir

mlir/test/Dialect/Linalg/parallel-loops.mlir

mlir/test/Dialect/Linalg/promote.mlir

mlir/test/Dialect/Linalg/promotion_options.mlir

mlir/test/Dialect/Linalg/reshape_control_fusion.mlir

mlir/test/Dialect/Linalg/reshape_fusion.mlir

mlir/test/Dialect/Linalg/reshape_linearization_fusion.mlir

mlir/test/Dialect/Linalg/reshape_linearization_fusion_with_unit_dims.mlir

mlir/test/Dialect/Linalg/resolve-shaped-type-result-dims.mlir

mlir/test/Dialect/Linalg/roundtrip.mlir

mlir/test/Dialect/Linalg/subtensor-of-padtensor.mlir

mlir/test/Dialect/Linalg/tile-and-distribute.mlir

mlir/test/Dialect/Linalg/tile-and-fuse-on-tensors.mlir

mlir/test/Dialect/Linalg/tile-and-fuse-tensors.mlir

mlir/test/Dialect/Linalg/tile-and-pad-tensors.mlir

mlir/test/Dialect/Linalg/tile-and-peel-tensors.mlir

mlir/test/Dialect/Linalg/tile-conv.mlir

mlir/test/Dialect/Linalg/tile-indexed.mlir

mlir/test/Dialect/Linalg/tile-pad-tensor-op.mlir

mlir/test/Dialect/Linalg/tile-parallel-reduce.mlir

mlir/test/Dialect/Linalg/tile-parallel.mlir

mlir/test/Dialect/Linalg/tile-scalarize-dynamic-dims.mlir

mlir/test/Dialect/Linalg/tile-tensors.mlir

mlir/test/Dialect/Linalg/tile.mlir

mlir/test/Dialect/Linalg/tiled-loop-peeling.mlir

mlir/test/Dialect/Linalg/tiled-loop-to-scf.mlir

mlir/test/Dialect/Linalg/transform-patterns.mlir

mlir/test/Dialect/Linalg/vectorization.mlir

mlir/test/Dialect/Math/algebraic-simplification.mlir

mlir/test/Dialect/Math/polynomial-approximation.mlir

mlir/test/Dialect/MemRef/canonicalize.mlir

mlir/test/Dialect/MemRef/fold-subview-ops.mlir

mlir/test/Dialect/MemRef/invalid.mlir

mlir/test/Dialect/MemRef/ops.mlir

mlir/test/Dialect/OpenACC/canonicalize.mlir

mlir/test/Dialect/OpenACC/invalid.mlir

mlir/test/Dialect/OpenACC/ops.mlir

mlir/test/Dialect/OpenMP/invalid.mlir

mlir/test/Dialect/OpenMP/ops.mlir

mlir/test/Dialect/Quant/canonicalize.mlir

mlir/test/Dialect/Quant/convert-const.mlir

mlir/test/Dialect/SCF/bufferize.mlir

mlir/test/Dialect/SCF/canonicalize.mlir

mlir/test/Dialect/SCF/for-loop-canonicalization.mlir

mlir/test/Dialect/SCF/for-loop-peeling.mlir

mlir/test/Dialect/SCF/for-loop-specialization.mlir

mlir/test/Dialect/SCF/for-loop-to-while-loop.mlir

mlir/test/Dialect/SCF/invalid.mlir

mlir/test/Dialect/SCF/loop-pipelining.mlir

mlir/test/Dialect/SCF/loop-range.mlir

mlir/test/Dialect/SCF/loop-unroll.mlir

mlir/test/Dialect/SCF/ops.mlir

mlir/test/Dialect/SCF/parallel-loop-fusion.mlir

mlir/test/Dialect/SCF/parallel-loop-specialization.mlir

mlir/test/Dialect/SCF/parallel-loop-tiling-inbound-check.mlir

mlir/test/Dialect/SCF/parallel-loop-tiling.mlir

mlir/test/Dialect/SPIRV/IR/memory-ops.mlir

mlir/test/Dialect/SPIRV/IR/target-and-abi.mlir

mlir/test/Dialect/Shape/canonicalize.mlir

mlir/test/Dialect/Shape/invalid.mlir

mlir/test/Dialect/Shape/ops.mlir

mlir/test/Dialect/Shape/shape-to-shape.mlir

mlir/test/Dialect/SparseTensor/conversion.mlir

mlir/test/Dialect/SparseTensor/dense.mlir

mlir/test/Dialect/SparseTensor/fold.mlir

mlir/test/Dialect/SparseTensor/invalid.mlir

mlir/test/Dialect/SparseTensor/roundtrip.mlir

mlir/test/Dialect/SparseTensor/sparse_1d.mlir

mlir/test/Dialect/SparseTensor/sparse_2d.mlir

mlir/test/Dialect/SparseTensor/sparse_3d.mlir

mlir/test/Dialect/SparseTensor/sparse_affine.mlir

mlir/test/Dialect/SparseTensor/sparse_fp_ops.mlir

mlir/test/Dialect/SparseTensor/sparse_int_ops.mlir

mlir/test/Dialect/SparseTensor/sparse_kernels.mlir

mlir/test/Dialect/SparseTensor/sparse_lower.mlir

mlir/test/Dialect/SparseTensor/sparse_lower_col.mlir

mlir/test/Dialect/SparseTensor/sparse_lower_inplace.mlir

mlir/test/Dialect/SparseTensor/sparse_nd.mlir

mlir/test/Dialect/SparseTensor/sparse_out.mlir

mlir/test/Dialect/SparseTensor/sparse_parallel.mlir

mlir/test/Dialect/SparseTensor/sparse_perm.mlir

mlir/test/Dialect/SparseTensor/sparse_perm_lower.mlir

mlir/test/Dialect/SparseTensor/sparse_scalars.mlir

mlir/test/Dialect/SparseTensor/sparse_storage.mlir

mlir/test/Dialect/SparseTensor/sparse_vector.mlir

mlir/test/Dialect/SparseTensor/sparse_vector_peeled.mlir

mlir/test/Dialect/Standard/bufferize.mlir

mlir/test/Dialect/Standard/canonicalize-cf.mlir

mlir/test/Dialect/Standard/canonicalize.mlir

mlir/test/Dialect/Standard/expand-ops.mlir

mlir/test/Dialect/Standard/expand-tanh.mlir

mlir/test/Dialect/Standard/func-bufferize.mlir

mlir/test/Dialect/Standard/invalid.mlir

mlir/test/Dialect/Standard/ops.mlir

mlir/test/Dialect/Standard/tensor-constant-bufferize.mlir

mlir/test/Dialect/Tensor/bufferize.mlir

mlir/test/Dialect/Tensor/canonicalize.mlir

mlir/test/Dialect/Tensor/invalid.mlir

mlir/test/Dialect/Tensor/ops.mlir

mlir/test/Dialect/Tensor/resolve-shaped-type-result-dims.mlir

mlir/test/Dialect/Tosa/canonicalize.mlir

mlir/test/Dialect/Tosa/tosa-infer-shapes.mlir

mlir/test/Dialect/Vector/canonicalize.mlir

mlir/test/Dialect/Vector/invalid.mlir

mlir/test/Dialect/Vector/ops.mlir

mlir/test/Dialect/Vector/vector-contract-transforms.mlir

mlir/test/Dialect/Vector/vector-distribution.mlir

mlir/test/Dialect/Vector/vector-mem-transforms.mlir

mlir/test/Dialect/Vector/vector-multi-reduction-lowering.mlir

mlir/test/Dialect/Vector/vector-multi-reduction-outer-lowering.mlir

mlir/test/Dialect/Vector/vector-transfer-full-partial-split.mlir

mlir/test/Dialect/Vector/vector-transfer-to-vector-load-store.mlir

mlir/test/Dialect/Vector/vector-transfer-unroll.mlir

mlir/test/Dialect/Vector/vector-transferop-opt.mlir

mlir/test/Dialect/Vector/vector-transforms.mlir

mlir/test/Examples/Toy/Ch5/affine-lowering.mlir

mlir/test/Examples/Toy/Ch6/affine-lowering.mlir

mlir/test/Examples/Toy/Ch7/affine-lowering.mlir

mlir/test/IR/core-ops.mlir

Show All 23 Lines
// CHECK-DAG: #[[$SUBVIEW_MAP8:map[0-9]+]] = affine_map<(d0, d1, d2, d3)[s0, s1, s2, s3, s4] -> (d0 * s1 + s0 + d1 * s2 + d2 * s3 + d3 * s4)> // CHECK-DAG: #[[$SUBVIEW_MAP8:map[0-9]+]] = affine_map<(d0, d1, d2, d3)[s0, s1, s2, s3, s4] -> (d0 * s1 + s0 + d1 * s2 + d2 * s3 + d3 * s4)>
// CHECK-DAG: #[[$SUBVIEW_MAP9:map[0-9]+]] = affine_map<(d0, d1) -> (d0 * 3 + d1 + 6)> // CHECK-DAG: #[[$SUBVIEW_MAP9:map[0-9]+]] = affine_map<(d0, d1) -> (d0 * 3 + d1 + 6)>
// CHECK-DAG: #[[$SUBVIEW_MAP10:map[0-9]+]] = affine_map<(d0) -> (d0 + 3)> // CHECK-DAG: #[[$SUBVIEW_MAP10:map[0-9]+]] = affine_map<(d0) -> (d0 + 3)>
// CHECK-DAG: #[[$SUBVIEW_MAP11:map[0-9]+]] = affine_map<() -> (4)> // CHECK-DAG: #[[$SUBVIEW_MAP11:map[0-9]+]] = affine_map<() -> (4)>
// CHECK-DAG: #[[$SUBVIEW_MAP12:map[0-9]+]] = affine_map<()[s0] -> (s0)> // CHECK-DAG: #[[$SUBVIEW_MAP12:map[0-9]+]] = affine_map<()[s0] -> (s0)>
// CHECK-LABEL: func @func_with_ops // CHECK-LABEL: func @func_with_ops
// CHECK-SAME: %[[ARG:.*]]: f32 // CHECK-SAME: %[[ARG:.*]]: f32
func @func_with_ops(f32) { func @func_with_ops(f32) {
rriddleUnsubmitted
Done
ReplyInline Actions

Can you move these tests to a file in the Arith dialect folder?

rriddle: Can you move these tests to a file in the Arith dialect folder?
^bb0(%a : f32): ^bb0(%a : f32):
// CHECK: %[[T:.*]] = "getTensor"() : () -> tensor<4x4x?xf32> // CHECK: %[[T:.*]] = "getTensor"() : () -> tensor<4x4x?xf32>
%t = "getTensor"() : () -> tensor<4x4x?xf32> %t = "getTensor"() : () -> tensor<4x4x?xf32>
// CHECK: %[[C2:.*]] = constant 2 : index // CHECK: %[[C2:.*]] = arith.constant 2 : index
// CHECK-NEXT: %{{.*}} = tensor.dim %[[T]], %[[C2]] : tensor<4x4x?xf32> // CHECK-NEXT: %{{.*}} = tensor.dim %[[T]], %[[C2]] : tensor<4x4x?xf32>
%c2 = constant 2 : index %c2 = arith.constant 2 : index
%t2 = "tensor.dim"(%t, %c2) : (tensor<4x4x?xf32>, index) -> index %t2 = "tensor.dim"(%t, %c2) : (tensor<4x4x?xf32>, index) -> index
// CHECK: %{{.*}} = addf %[[ARG]], %[[ARG]] : f32 // CHECK: %{{.*}} = arith.addf %[[ARG]], %[[ARG]] : f32
%x = "std.addf"(%a, %a) : (f32,f32) -> (f32) %x = "arith.addf"(%a, %a) : (f32,f32) -> (f32)
// CHECK: return // CHECK: return
return return
} }
// CHECK-LABEL: func @standard_instrs(%arg0: tensor<4x4x?xf32>, %arg1: f32, %arg2: i32, %arg3: index, %arg4: i64, %arg5: f16) { // CHECK-LABEL: func @standard_instrs(%arg0: tensor<4x4x?xf32>, %arg1: f32, %arg2: i32, %arg3: index, %arg4: i64, %arg5: f16) {
func @standard_instrs(tensor<4x4x?xf32>, f32, i32, index, i64, f16) { func @standard_instrs(tensor<4x4x?xf32>, f32, i32, index, i64, f16) {
^bb42(%t: tensor<4x4x?xf32>, %f: f32, %i: i32, %idx : index, %j: i64, %half: f16): ^bb42(%t: tensor<4x4x?xf32>, %f: f32, %i: i32, %idx : index, %j: i64, %half: f16):
// CHECK: %[[C2:.*]] = constant 2 : index // CHECK: %[[C2:.*]] = arith.constant 2 : index
// CHECK: %[[A2:.*]] = tensor.dim %arg0, %[[C2]] : tensor<4x4x?xf32> // CHECK: %[[A2:.*]] = tensor.dim %arg0, %[[C2]] : tensor<4x4x?xf32>
%c2 = constant 2 : index %c2 = arith.constant 2 : index
%a2 = tensor.dim %t, %c2 : tensor<4x4x?xf32> %a2 = tensor.dim %t, %c2 : tensor<4x4x?xf32>
// CHECK: %[[F2:.*]] = addf %arg1, %arg1 : f32
%f2 = "std.addf"(%f, %f) : (f32,f32) -> f32
// CHECK: %[[F3:.*]] = addf %[[F2]], %[[F2]] : f32
%f3 = addf %f2, %f2 : f32
// CHECK: %[[I2:.*]] = addi %arg2, %arg2 : i32
%i2 = "std.addi"(%i, %i) : (i32,i32) -> i32
// CHECK: %[[I3:.*]] = addi %[[I2]], %[[I2]] : i32
%i3 = addi %i2, %i2 : i32
// CHECK: %[[IDX1:.*]] = addi %arg3, %arg3 : index
%idx1 = addi %idx, %idx : index
// CHECK: %[[IDX2:.*]] = addi %arg3, %[[IDX1]] : index
%idx2 = "std.addi"(%idx, %idx1) : (index, index) -> index
// CHECK: %[[F4:.*]] = subf %arg1, %arg1 : f32
%f4 = "std.subf"(%f, %f) : (f32,f32) -> f32
// CHECK: %[[F5:.*]] = subf %[[F4]], %[[F4]] : f32
%f5 = subf %f4, %f4 : f32
// CHECK: %[[I4:.*]] = subi %arg2, %arg2 : i32
%i4 = "std.subi"(%i, %i) : (i32,i32) -> i32
// CHECK: %[[I5:.*]] = subi %[[I4]], %[[I4]] : i32
%i5 = subi %i4, %i4 : i32
// CHECK: %[[F6:.*]] = mulf %[[F2]], %[[F2]] : f32
%f6 = mulf %f2, %f2 : f32
// CHECK: %[[I6:.*]] = muli %[[I2]], %[[I2]] : i32
%i6 = muli %i2, %i2 : i32
// CHECK: %c42_i32 = constant 42 : i32
%x = "std.constant"(){value = 42 : i32} : () -> i32
// CHECK: %c42_i32_0 = constant 42 : i32
%7 = constant 42 : i32
// CHECK: %c43 = constant {crazy = "std.foo"} 43 : index
%8 = constant {crazy = "std.foo"} 43: index
// CHECK: %cst = constant 4.300000e+01 : bf16
%9 = constant 43.0 : bf16
// CHECK: %f = constant @func_with_ops : (f32) -> () // CHECK: %f = constant @func_with_ops : (f32) -> ()
%10 = constant @func_with_ops : (f32) -> () %10 = constant @func_with_ops : (f32) -> ()
// CHECK: %f_1 = constant @affine_apply : () -> () // CHECK: %f_0 = constant @affine_apply : () -> ()
%11 = constant @affine_apply : () -> () %11 = constant @affine_apply : () -> ()
// CHECK: %f_2 = constant @affine_apply : () -> () // CHECK: %[[I2:.*]] = arith.addi
%12 = constant @affine_apply : () -> () %i2 = arith.addi %i, %i: i32
// CHECK: %[[I3:.*]] = arith.addi
// CHECK: %cst_3 = constant dense<0> : vector<4xi32> %i3 = arith.addi %i2, %i : i32
%13 = constant dense<0> : vector<4 x i32> // CHECK: %[[I4:.*]] = arith.addi
%i4 = arith.addi %i2, %i3 : i32
// CHECK: %cst_4 = constant dense<0> : tensor<42xi32> // CHECK: %[[F3:.*]] = arith.addf
%tci32 = constant dense<0> : tensor<42 x i32> %f3 = arith.addf %f, %f : f32
// CHECK: %[[F4:.*]] = arith.addf
// CHECK: %cst_5 = constant dense<0> : vector<42xi32> %f4 = arith.addf %f, %f3 : f32
%vci32 = constant dense<0> : vector<42 x i32>
%true = arith.constant true
// CHECK: %{{[0-9]+}} = cmpi eq, %{{[0-9]+}}, %{{[0-9]+}} : i32 %tci32 = arith.constant dense<0> : tensor<42xi32>
%14 = cmpi eq, %i3, %i4 : i32 %vci32 = arith.constant dense<0> : vector<42xi32>
%tci1 = arith.constant dense<1> : tensor<42xi1>
// Predicate 1 means inequality comparison. %vci1 = arith.constant dense<1> : vector<42xi1>
// CHECK: %{{[0-9]+}} = cmpi ne, %{{[0-9]+}}, %{{[0-9]+}} : i32
%15 = "std.cmpi"(%i3, %i4) {predicate = 1} : (i32, i32) -> i1 // CHECK: %{{.*}} = select %{{.*}}, %arg3, %arg3 : index
%21 = select %true, %idx, %idx : index
// CHECK: %{{[0-9]+}} = cmpi slt, %cst_3, %cst_3 : vector<4xi32>
%16 = cmpi slt, %13, %13 : vector<4 x i32> // CHECK: %{{.*}} = select %{{.*}}, %{{.*}}, %{{.*}} : tensor<42xi1>, tensor<42xi32>
%22 = select %tci1, %tci32, %tci32 : tensor<42 x i1>, tensor<42 x i32>
// CHECK: %{{[0-9]+}} = cmpi ne, %cst_3, %cst_3 : vector<4xi32>
%17 = "std.cmpi"(%13, %13) {predicate = 1} : (vector<4 x i32>, vector<4 x i32>) -> vector<4 x i1> // CHECK: %{{.*}} = select %{{.*}}, %{{.*}}, %{{.*}} : vector<42xi1>, vector<42xi32>
%23 = select %vci1, %vci32, %vci32 : vector<42 x i1>, vector<42 x i32>
// CHECK: %{{[0-9]+}} = cmpi slt, %arg3, %arg3 : index
%18 = cmpi slt, %idx, %idx : index // CHECK: %{{.*}} = select %{{.*}}, %arg3, %arg3 : index
%24 = "std.select"(%true, %idx, %idx) : (i1, index, index) -> index
// CHECK: %{{[0-9]+}} = cmpi eq, %cst_4, %cst_4 : tensor<42xi32>
%19 = cmpi eq, %tci32, %tci32 : tensor<42 x i32> // CHECK: %{{.*}} = select %{{.*}}, %{{.*}}, %{{.*}} : tensor<42xi32>
%25 = std.select %true, %tci32, %tci32 : tensor<42 x i32>
// CHECK: %{{[0-9]+}} = cmpi eq, %cst_5, %cst_5 : vector<42xi32>
%20 = cmpi eq, %vci32, %vci32 : vector<42 x i32> %64 = arith.constant dense<0.> : vector<4 x f32>
%tcf32 = arith.constant dense<0.> : tensor<42 x f32>
// CHECK: %{{[0-9]+}} = select %{{[0-9]+}}, %arg3, %arg3 : index %vcf32 = arith.constant dense<0.> : vector<4 x f32>
%21 = select %18, %idx, %idx : index
// CHECK: %{{[0-9]+}} = select %{{[0-9]+}}, %cst_4, %cst_4 : tensor<42xi1>, tensor<42xi32>
%22 = select %19, %tci32, %tci32 : tensor<42 x i1>, tensor<42 x i32>
// CHECK: %{{[0-9]+}} = select %{{[0-9]+}}, %cst_5, %cst_5 : vector<42xi1>, vector<42xi32>
%23 = select %20, %vci32, %vci32 : vector<42 x i1>, vector<42 x i32>
// CHECK: %{{[0-9]+}} = select %{{[0-9]+}}, %arg3, %arg3 : index
%24 = "std.select"(%18, %idx, %idx) : (i1, index, index) -> index
// CHECK: %{{[0-9]+}} = select %{{[0-9]+}}, %cst_4, %cst_4 : tensor<42xi32>
%25 = std.select %18, %tci32, %tci32 : tensor<42 x i32>
// CHECK: %{{[0-9]+}} = divi_signed %arg2, %arg2 : i32
%26 = divi_signed %i, %i : i32
// CHECK: %{{[0-9]+}} = divi_signed %arg3, %arg3 : index
%27 = divi_signed %idx, %idx : index
// CHECK: %{{[0-9]+}} = divi_signed %cst_5, %cst_5 : vector<42xi32>
%28 = divi_signed %vci32, %vci32 : vector<42 x i32>
// CHECK: %{{[0-9]+}} = divi_signed %cst_4, %cst_4 : tensor<42xi32>
%29 = divi_signed %tci32, %tci32 : tensor<42 x i32>
// CHECK: %{{[0-9]+}} = divi_signed %arg2, %arg2 : i32
%30 = "std.divi_signed"(%i, %i) : (i32, i32) -> i32
// CHECK: %{{[0-9]+}} = divi_unsigned %arg2, %arg2 : i32
%31 = divi_unsigned %i, %i : i32
// CHECK: %{{[0-9]+}} = divi_unsigned %arg3, %arg3 : index
%32 = divi_unsigned %idx, %idx : index
// CHECK: %{{[0-9]+}} = divi_unsigned %cst_5, %cst_5 : vector<42xi32>
%33 = divi_unsigned %vci32, %vci32 : vector<42 x i32>
// CHECK: %{{[0-9]+}} = divi_unsigned %cst_4, %cst_4 : tensor<42xi32>
%34 = divi_unsigned %tci32, %tci32 : tensor<42 x i32>
// CHECK: %{{[0-9]+}} = divi_unsigned %arg2, %arg2 : i32
%35 = "std.divi_unsigned"(%i, %i) : (i32, i32) -> i32
// CHECK: %{{[0-9]+}} = remi_signed %arg2, %arg2 : i32
%36 = remi_signed %i, %i : i32
// CHECK: %{{[0-9]+}} = remi_signed %arg3, %arg3 : index
%37 = remi_signed %idx, %idx : index
// CHECK: %{{[0-9]+}} = remi_signed %cst_5, %cst_5 : vector<42xi32>
%38 = remi_signed %vci32, %vci32 : vector<42 x i32>
// CHECK: %{{[0-9]+}} = remi_signed %cst_4, %cst_4 : tensor<42xi32>
%39 = remi_signed %tci32, %tci32 : tensor<42 x i32>
// CHECK: %{{[0-9]+}} = remi_signed %arg2, %arg2 : i32
%40 = "std.remi_signed"(%i, %i) : (i32, i32) -> i32
// CHECK: %{{[0-9]+}} = remi_unsigned %arg2, %arg2 : i32
%41 = remi_unsigned %i, %i : i32
// CHECK: %{{[0-9]+}} = remi_unsigned %arg3, %arg3 : index
%42 = remi_unsigned %idx, %idx : index
// CHECK: %{{[0-9]+}} = remi_unsigned %cst_5, %cst_5 : vector<42xi32>
%43 = remi_unsigned %vci32, %vci32 : vector<42 x i32>
// CHECK: %{{[0-9]+}} = remi_unsigned %cst_4, %cst_4 : tensor<42xi32>
%44 = remi_unsigned %tci32, %tci32 : tensor<42 x i32>
// CHECK: %{{[0-9]+}} = remi_unsigned %arg2, %arg2 : i32
%45 = "std.remi_unsigned"(%i, %i) : (i32, i32) -> i32
// CHECK: %{{[0-9]+}} = divf %arg1, %arg1 : f32
%46 = "std.divf"(%f, %f) : (f32,f32) -> f32
// CHECK: %{{[0-9]+}} = divf %arg1, %arg1 : f32
%47 = divf %f, %f : f32
// CHECK: %{{[0-9]+}} = divf %arg0, %arg0 : tensor<4x4x?xf32>
%48 = divf %t, %t : tensor<4x4x?xf32>
// CHECK: %{{[0-9]+}} = remf %arg1, %arg1 : f32
%49 = "std.remf"(%f, %f) : (f32,f32) -> f32
// CHECK: %{{[0-9]+}} = remf %arg1, %arg1 : f32
%50 = remf %f, %f : f32
// CHECK: %{{[0-9]+}} = remf %arg0, %arg0 : tensor<4x4x?xf32>
%51 = remf %t, %t : tensor<4x4x?xf32>
// CHECK: %{{[0-9]+}} = and %arg2, %arg2 : i32
%52 = "std.and"(%i, %i) : (i32,i32) -> i32
// CHECK: %{{[0-9]+}} = and %arg2, %arg2 : i32
%53 = and %i, %i : i32
// CHECK: %{{[0-9]+}} = and %cst_5, %cst_5 : vector<42xi32>
%54 = std.and %vci32, %vci32 : vector<42 x i32>
// CHECK: %{{[0-9]+}} = and %cst_4, %cst_4 : tensor<42xi32>
%55 = and %tci32, %tci32 : tensor<42 x i32>
// CHECK: %{{[0-9]+}} = or %arg2, %arg2 : i32
%56 = "std.or"(%i, %i) : (i32,i32) -> i32
// CHECK: %{{[0-9]+}} = or %arg2, %arg2 : i32 // CHECK: %{{.*}} = arith.cmpf ogt, %{{.*}}, %{{.*}} : f32
%57 = or %i, %i : i32 %65 = arith.cmpf ogt, %f3, %f4 : f32
// CHECK: %{{[0-9]+}} = or %cst_5, %cst_5 : vector<42xi32>
%58 = std.or %vci32, %vci32 : vector<42 x i32>
// CHECK: %{{[0-9]+}} = or %cst_4, %cst_4 : tensor<42xi32>
%59 = or %tci32, %tci32 : tensor<42 x i32>
// CHECK: %{{[0-9]+}} = xor %arg2, %arg2 : i32
%60 = "std.xor"(%i, %i) : (i32,i32) -> i32
// CHECK: %{{[0-9]+}} = xor %arg2, %arg2 : i32
%61 = xor %i, %i : i32
// CHECK: %{{[0-9]+}} = xor %cst_5, %cst_5 : vector<42xi32>
%62 = std.xor %vci32, %vci32 : vector<42 x i32>
// CHECK: %{{[0-9]+}} = xor %cst_4, %cst_4 : tensor<42xi32>
%63 = xor %tci32, %tci32 : tensor<42 x i32>
%64 = constant dense<0.> : vector<4 x f32>
%tcf32 = constant dense<0.> : tensor<42 x f32>
%vcf32 = constant dense<0.> : vector<4 x f32>
// CHECK: %{{[0-9]+}} = cmpf ogt, %{{[0-9]+}}, %{{[0-9]+}} : f32
%65 = cmpf ogt, %f3, %f4 : f32
// Predicate 0 means ordered equality comparison. // Predicate 0 means ordered equality comparison.
// CHECK: %{{[0-9]+}} = cmpf oeq, %{{[0-9]+}}, %{{[0-9]+}} : f32 // CHECK: %{{.*}} = arith.cmpf oeq, %{{.*}}, %{{.*}} : f32
%66 = "std.cmpf"(%f3, %f4) {predicate = 1} : (f32, f32) -> i1 %66 = "arith.cmpf"(%f3, %f4) {predicate = 1} : (f32, f32) -> i1
// CHECK: %{{[0-9]+}} = cmpf olt, %cst_8, %cst_8 : vector<4xf32> // CHECK: %{{.*}} = arith.cmpf olt, %{{.*}}, %{{.*}}: vector<4xf32>
%67 = cmpf olt, %vcf32, %vcf32 : vector<4 x f32> %67 = arith.cmpf olt, %vcf32, %vcf32 : vector<4 x f32>
// CHECK: %{{[0-9]+}} = cmpf oeq, %cst_8, %cst_8 : vector<4xf32> // CHECK: %{{.*}} = arith.cmpf oeq, %{{.*}}, %{{.*}}: vector<4xf32>
%68 = "std.cmpf"(%vcf32, %vcf32) {predicate = 1} : (vector<4 x f32>, vector<4 x f32>) -> vector<4 x i1> %68 = "arith.cmpf"(%vcf32, %vcf32) {predicate = 1} : (vector<4 x f32>, vector<4 x f32>) -> vector<4 x i1>
// CHECK: %{{[0-9]+}} = cmpf oeq, %cst_7, %cst_7 : tensor<42xf32> // CHECK: %{{.*}} = arith.cmpf oeq, %{{.*}}, %{{.*}}: tensor<42xf32>
%69 = cmpf oeq, %tcf32, %tcf32 : tensor<42 x f32> %69 = arith.cmpf oeq, %tcf32, %tcf32 : tensor<42 x f32>
// CHECK: %{{[0-9]+}} = cmpf oeq, %cst_8, %cst_8 : vector<4xf32> // CHECK: %{{.*}} = arith.cmpf oeq, %{{.*}}, %{{.*}}: vector<4xf32>
%70 = cmpf oeq, %vcf32, %vcf32 : vector<4 x f32> %70 = arith.cmpf oeq, %vcf32, %vcf32 : vector<4 x f32>
// CHECK: %{{[0-9]+}} = rank %arg0 : tensor<4x4x?xf32> // CHECK: %{{.*}} = rank %arg0 : tensor<4x4x?xf32>
%71 = "std.rank"(%t) : (tensor<4x4x?xf32>) -> index %71 = "std.rank"(%t) : (tensor<4x4x?xf32>) -> index
// CHECK: %{{[0-9]+}} = rank %arg0 : tensor<4x4x?xf32> // CHECK: %{{.*}} = rank %arg0 : tensor<4x4x?xf32>
%72 = rank %t : tensor<4x4x?xf32> %72 = rank %t : tensor<4x4x?xf32>
// CHECK: = constant unit // CHECK: = constant unit
%73 = constant unit %73 = constant unit
// CHECK: constant true // CHECK: arith.constant true
%74 = constant true %74 = arith.constant true
// CHECK: constant false
%75 = constant false
// CHECK: = index_cast {{.*}} : index to i64
%76 = index_cast %idx : index to i64
// CHECK: = index_cast {{.*}} : i32 to index
%77 = index_cast %i : i32 to index
// CHECK: = sitofp {{.*}} : i32 to f32
%78 = sitofp %i : i32 to f32
// CHECK: = sitofp {{.*}} : i32 to f64
%79 = sitofp %i : i32 to f64
// CHECK: = sitofp {{.*}} : i64 to f32
%80 = sitofp %j : i64 to f32
// CHECK: = sitofp {{.*}} : i64 to f64
%81 = sitofp %j : i64 to f64
// CHECK: = sexti %arg2 : i32 to i64
%82 = "std.sexti"(%i) : (i32) -> i64
// CHECK: = sexti %arg2 : i32 to i64
%83 = sexti %i : i32 to i64
// CHECK: %{{[0-9]+}} = sexti %cst_5 : vector<42xi32>
%84 = sexti %vci32 : vector<42 x i32> to vector<42 x i64>
// CHECK: %{{[0-9]+}} = sexti %cst_4 : tensor<42xi32>
%85 = sexti %tci32 : tensor<42 x i32> to tensor<42 x i64>
// CHECK: = zexti %arg2 : i32 to i64
%86 = "std.zexti"(%i) : (i32) -> i64
// CHECK: = zexti %arg2 : i32 to i64
%87 = zexti %i : i32 to i64
// CHECK: %{{[0-9]+}} = zexti %cst_5 : vector<42xi32>
%88 = zexti %vci32 : vector<42 x i32> to vector<42 x i64>
// CHECK: %{{[0-9]+}} = zexti %cst_4 : tensor<42xi32>
%89 = zexti %tci32 : tensor<42 x i32> to tensor<42 x i64>
// CHECK: = trunci %arg2 : i32 to i16
%90 = "std.trunci"(%i) : (i32) -> i16
// CHECK: = trunci %arg2 : i32 to i16
%91 = trunci %i : i32 to i16
// CHECK: %{{[0-9]+}} = trunci %cst_5 : vector<42xi32>
%92 = trunci %vci32 : vector<42 x i32> to vector<42 x i16>
// CHECK: %{{[0-9]+}} = trunci %cst_4 : tensor<42xi32>
%93 = trunci %tci32 : tensor<42 x i32> to tensor<42 x i16>
// CHECK: = fpext {{.*}} : f16 to f32
%94 = fpext %half : f16 to f32
// CHECK: = fptrunc {{.*}} : f32 to f16
%95 = fptrunc %f : f32 to f16
// CHECK: %{{[0-9]+}} = absf %arg1 : f32
%100 = "std.absf"(%f) : (f32) -> f32
// CHECK: %{{[0-9]+}} = absf %arg1 : f32
%101 = absf %f : f32
// CHECK: %{{[0-9]+}} = absf %cst_8 : vector<4xf32> // CHECK: arith.constant false
%102 = absf %vcf32 : vector<4xf32> %75 = arith.constant false
// CHECK: %{{[0-9]+}} = absf %arg0 : tensor<4x4x?xf32> // CHECK: %{{.*}} = math.abs %arg1 : f32
%103 = absf %t : tensor<4x4x?xf32> %100 = "math.abs"(%f) : (f32) -> f32
// CHECK: %{{[0-9]+}} = ceilf %arg1 : f32 // CHECK: %{{.*}} = math.abs %arg1 : f32
%104 = "std.ceilf"(%f) : (f32) -> f32 %101 = math.abs %f : f32
// CHECK: %{{[0-9]+}} = ceilf %arg1 : f32 // CHECK: %{{.*}} = math.abs %{{.*}}: vector<4xf32>
%105 = ceilf %f : f32 %102 = math.abs %vcf32 : vector<4xf32>
// CHECK: %{{[0-9]+}} = ceilf %cst_8 : vector<4xf32> // CHECK: %{{.*}} = math.abs %arg0 : tensor<4x4x?xf32>
%106 = ceilf %vcf32 : vector<4xf32> %103 = math.abs %t : tensor<4x4x?xf32>
// CHECK: %{{[0-9]+}} = ceilf %arg0 : tensor<4x4x?xf32> // CHECK: %{{.*}} = math.ceil %arg1 : f32
%107 = ceilf %t : tensor<4x4x?xf32> %104 = "math.ceil"(%f) : (f32) -> f32
// CHECK: %{{[0-9]+}} = negf %arg1 : f32 // CHECK: %{{.*}} = math.ceil %arg1 : f32
%112 = "std.negf"(%f) : (f32) -> f32 %105 = math.ceil %f : f32
// CHECK: %{{[0-9]+}} = negf %arg1 : f32 // CHECK: %{{.*}} = math.ceil %{{.*}}: vector<4xf32>
%113 = negf %f : f32 %106 = math.ceil %vcf32 : vector<4xf32>
// CHECK: %{{[0-9]+}} = negf %cst_8 : vector<4xf32> // CHECK: %{{.*}} = math.ceil %arg0 : tensor<4x4x?xf32>
%114 = negf %vcf32 : vector<4xf32> %107 = math.ceil %t : tensor<4x4x?xf32>
// CHECK: %{{[0-9]+}} = negf %arg0 : tensor<4x4x?xf32> // CHECK: %{{.*}} = math.copysign %arg1, %arg1 : f32
%115 = negf %t : tensor<4x4x?xf32> %116 = "math.copysign"(%f, %f) : (f32, f32) -> f32
// CHECK: %{{[0-9]+}} = copysign %arg1, %arg1 : f32 // CHECK: %{{.*}} = math.copysign %arg1, %arg1 : f32
%116 = "std.copysign"(%f, %f) : (f32, f32) -> f32 %117 = math.copysign %f, %f : f32
// CHECK: %{{[0-9]+}} = copysign %arg1, %arg1 : f32 // CHECK: %{{.*}} = math.copysign %{{.*}}, %{{.*}}: vector<4xf32>
%117 = copysign %f, %f : f32 %118 = math.copysign %vcf32, %vcf32 : vector<4xf32>
// CHECK: %{{[0-9]+}} = copysign %cst_8, %cst_8 : vector<4xf32> // CHECK: %{{.*}} = math.copysign %arg0, %arg0 : tensor<4x4x?xf32>
%118 = copysign %vcf32, %vcf32 : vector<4xf32> %119 = math.copysign %t, %t : tensor<4x4x?xf32>
// CHECK: %{{[0-9]+}} = copysign %arg0, %arg0 : tensor<4x4x?xf32> // CHECK: %{{.*}} = math.rsqrt %arg1 : f32
%119 = copysign %t, %t : tensor<4x4x?xf32>
// CHECK: %{{[0-9]+}} = shift_left %arg2, %arg2 : i32
%124 = "std.shift_left"(%i, %i) : (i32, i32) -> i32
// CHECK:%{{[0-9]+}} = shift_left %[[I2]], %[[I2]] : i32
%125 = shift_left %i2, %i2 : i32
// CHECK: %{{[0-9]+}} = shift_left %arg3, %arg3 : index
%126 = shift_left %idx, %idx : index
// CHECK: %{{[0-9]+}} = shift_left %cst_5, %cst_5 : vector<42xi32>
%127 = shift_left %vci32, %vci32 : vector<42 x i32>
// CHECK: %{{[0-9]+}} = shift_left %cst_4, %cst_4 : tensor<42xi32>
%128 = shift_left %tci32, %tci32 : tensor<42 x i32>
// CHECK: %{{[0-9]+}} = shift_right_signed %arg2, %arg2 : i32
%129 = "std.shift_right_signed"(%i, %i) : (i32, i32) -> i32
// CHECK:%{{[0-9]+}} = shift_right_signed %[[I2]], %[[I2]] : i32
%130 = shift_right_signed %i2, %i2 : i32
// CHECK: %{{[0-9]+}} = shift_right_signed %arg3, %arg3 : index
%131 = shift_right_signed %idx, %idx : index
// CHECK: %{{[0-9]+}} = shift_right_signed %cst_5, %cst_5 : vector<42xi32>
%132 = shift_right_signed %vci32, %vci32 : vector<42 x i32>
// CHECK: %{{[0-9]+}} = shift_right_signed %cst_4, %cst_4 : tensor<42xi32>
%133 = shift_right_signed %tci32, %tci32 : tensor<42 x i32>
// CHECK: %{{[0-9]+}} = shift_right_unsigned %arg2, %arg2 : i32
%134 = "std.shift_right_unsigned"(%i, %i) : (i32, i32) -> i32
// CHECK:%{{[0-9]+}} = shift_right_unsigned %[[I2]], %[[I2]] : i32
%135 = shift_right_unsigned %i2, %i2 : i32
// CHECK: %{{[0-9]+}} = shift_right_unsigned %arg3, %arg3 : index
%136 = shift_right_unsigned %idx, %idx : index
// CHECK: %{{[0-9]+}} = shift_right_unsigned %cst_5, %cst_5 : vector<42xi32>
%137 = shift_right_unsigned %vci32, %vci32 : vector<42 x i32>
// CHECK: %{{[0-9]+}} = shift_right_unsigned %cst_4, %cst_4 : tensor<42xi32>
%138 = shift_right_unsigned %tci32, %tci32 : tensor<42 x i32>
// CHECK: = fpext {{.*}} : vector<4xf32> to vector<4xf64>
%143 = fpext %vcf32 : vector<4xf32> to vector<4xf64>
// CHECK: = fptrunc {{.*}} : vector<4xf32> to vector<4xf16>
%144 = fptrunc %vcf32 : vector<4xf32> to vector<4xf16>
// CHECK: %{{[0-9]+}} = math.rsqrt %arg1 : f32
%145 = math.rsqrt %f : f32 %145 = math.rsqrt %f : f32
// CHECK: = fptosi {{.*}} : f32 to i32 // CHECK: math.floor %arg1 : f32
%159 = fptosi %f : f32 to i32 %163 = "math.floor"(%f) : (f32) -> f32
// CHECK: = fptosi {{.*}} : f32 to i64
%160 = fptosi %f : f32 to i64
// CHECK: = fptosi {{.*}} : f16 to i32
%161 = fptosi %half : f16 to i32
// CHECK: = fptosi {{.*}} : f16 to i64
%162 = fptosi %half : f16 to i64
// CHECK: floorf %arg1 : f32
%163 = "std.floorf"(%f) : (f32) -> f32
// CHECK: %{{[0-9]+}} = floorf %arg1 : f32
%164 = floorf %f : f32
// CHECK: %{{[0-9]+}} = floorf %cst_8 : vector<4xf32>
%165 = floorf %vcf32 : vector<4xf32>
// CHECK: %{{[0-9]+}} = floorf %arg0 : tensor<4x4x?xf32>
%166 = floorf %t : tensor<4x4x?xf32>
// CHECK: %{{[0-9]+}} = floordivi_signed %arg2, %arg2 : i32
%167 = floordivi_signed %i, %i : i32
// CHECK: %{{[0-9]+}} = floordivi_signed %arg3, %arg3 : index
%168 = floordivi_signed %idx, %idx : index
// CHECK: %{{[0-9]+}} = floordivi_signed %cst_5, %cst_5 : vector<42xi32>
%169 = floordivi_signed %vci32, %vci32 : vector<42 x i32>
// CHECK: %{{[0-9]+}} = floordivi_signed %cst_4, %cst_4 : tensor<42xi32>
%170 = floordivi_signed %tci32, %tci32 : tensor<42 x i32>
// CHECK: %{{[0-9]+}} = ceildivi_signed %arg2, %arg2 : i32
%171 = ceildivi_signed %i, %i : i32
// CHECK: %{{[0-9]+}} = ceildivi_signed %arg3, %arg3 : index // CHECK: %{{.*}} = math.floor %arg1 : f32
%172 = ceildivi_signed %idx, %idx : index %164 = math.floor %f : f32
// CHECK: %{{[0-9]+}} = ceildivi_signed %cst_5, %cst_5 : vector<42xi32> // CHECK: %{{.*}} = math.floor %{{.*}}: vector<4xf32>
%173 = ceildivi_signed %vci32, %vci32 : vector<42 x i32> %165 = math.floor %vcf32 : vector<4xf32>
// CHECK: %{{[0-9]+}} = ceildivi_signed %cst_4, %cst_4 : tensor<42xi32> // CHECK: %{{.*}} = math.floor %arg0 : tensor<4x4x?xf32>
%174 = ceildivi_signed %tci32, %tci32 : tensor<42 x i32> %166 = math.floor %t : tensor<4x4x?xf32>
return return
} }
// CHECK-LABEL: func @affine_apply() { // CHECK-LABEL: func @affine_apply() {
func @affine_apply() { func @affine_apply() {
%i = "std.constant"() {value = 0: index} : () -> index %i = "arith.constant"() {value = 0: index} : () -> index
%j = "std.constant"() {value = 1: index} : () -> index %j = "arith.constant"() {value = 1: index} : () -> index
// CHECK: affine.apply #map0(%c0) // CHECK: affine.apply #map0(%c0)
%a = "affine.apply" (%i) { map = affine_map<(d0) -> (d0 + 1)> } : %a = "affine.apply" (%i) { map = affine_map<(d0) -> (d0 + 1)> } :
(index) -> (index) (index) -> (index)
// CHECK: affine.apply #map1()[%c0] // CHECK: affine.apply #map1()[%c0]
%b = affine.apply affine_map<()[x] -> (x+1)>()[%i] %b = affine.apply affine_map<()[x] -> (x+1)>()[%i]
▲ Show 20 Lines • Show All 96 Lines • ▼ Show 20 Linesfunc @memref_view(%arg0 : index, %arg1 : index, %arg2 : index) {
%1 = memref.view %0[%arg2][%arg0, %arg1] : memref<2048xi8> to memref<?x?xf32> %1 = memref.view %0[%arg2][%arg0, %arg1] : memref<2048xi8> to memref<?x?xf32>
// Test one dynamic size and dynamic offset. // Test one dynamic size and dynamic offset.
// CHECK: %{{.*}} = memref.view %0[%arg2][%arg1] : memref<2048xi8> to memref<4x?xf32> // CHECK: %{{.*}} = memref.view %0[%arg2][%arg1] : memref<2048xi8> to memref<4x?xf32>
%3 = memref.view %0[%arg2][%arg1] : memref<2048xi8> to memref<4x?xf32> %3 = memref.view %0[%arg2][%arg1] : memref<2048xi8> to memref<4x?xf32>
// Test static sizes and static offset. // Test static sizes and static offset.
// CHECK: %{{.*}} = memref.view %0[{{.*}}][] : memref<2048xi8> to memref<64x4xf32> // CHECK: %{{.*}} = memref.view %0[{{.*}}][] : memref<2048xi8> to memref<64x4xf32>
%c0 = constant 0: index %c0 = arith.constant 0: index
%5 = memref.view %0[%c0][] : memref<2048xi8> to memref<64x4xf32> %5 = memref.view %0[%c0][] : memref<2048xi8> to memref<64x4xf32>
return return
} }
// CHECK-LABEL: func @memref_subview(%arg0 // CHECK-LABEL: func @memref_subview(%arg0
func @memref_subview(%arg0 : index, %arg1 : index, %arg2 : index) { func @memref_subview(%arg0 : index, %arg1 : index, %arg2 : index) {
%c0 = constant 0 : index %c0 = arith.constant 0 : index
%c1 = constant 1 : index %c1 = arith.constant 1 : index
%0 = memref.alloc() : memref<8x16x4xf32, affine_map<(d0, d1, d2) -> (d0 * 64 + d1 * 4 + d2)>> %0 = memref.alloc() : memref<8x16x4xf32, affine_map<(d0, d1, d2) -> (d0 * 64 + d1 * 4 + d2)>>
// CHECK: subview %0[%c0, %c0, %c0] [%arg0, %arg1, %arg2] [%c1, %c1, %c1] : // CHECK: subview %0[%c0, %c0, %c0] [%arg0, %arg1, %arg2] [%c1, %c1, %c1] :
// CHECK-SAME: memref<8x16x4xf32, #[[$BASE_MAP0]]> // CHECK-SAME: memref<8x16x4xf32, #[[$BASE_MAP0]]>
// CHECK-SAME: to memref<?x?x?xf32, #[[$BASE_MAP3]]> // CHECK-SAME: to memref<?x?x?xf32, #[[$BASE_MAP3]]>
%1 = memref.subview %0[%c0, %c0, %c0][%arg0, %arg1, %arg2][%c1, %c1, %c1] %1 = memref.subview %0[%c0, %c0, %c0][%arg0, %arg1, %arg2][%c1, %c1, %c1]
: memref<8x16x4xf32, offset:0, strides: [64, 4, 1]> to : memref<8x16x4xf32, offset:0, strides: [64, 4, 1]> to
memref<?x?x?xf32, offset: ?, strides: [?, ?, ?]> memref<?x?x?xf32, offset: ?, strides: [?, ?, ?]>
▲ Show 20 Lines • Show All 93 Lines • ▼ Show 20 Linesfunc @memref_subview(%arg0 : index, %arg1 : index, %arg2 : index) {
%30 = memref.subview %a30[0, %arg1][%arg2, 1][1, 1] : memref<?x?xf32> to memref<?xf32, affine_map<(d0)[s0, s1] -> (d0 * s1 + s0)>> %30 = memref.subview %a30[0, %arg1][%arg2, 1][1, 1] : memref<?x?xf32> to memref<?xf32, affine_map<(d0)[s0, s1] -> (d0 * s1 + s0)>>
return return
} }
// CHECK-LABEL: func @test_dimop // CHECK-LABEL: func @test_dimop
// CHECK-SAME: %[[ARG:.*]]: tensor<4x4x?xf32> // CHECK-SAME: %[[ARG:.*]]: tensor<4x4x?xf32>
func @test_dimop(%arg0: tensor<4x4x?xf32>) { func @test_dimop(%arg0: tensor<4x4x?xf32>) {
// CHECK: %[[C2:.*]] = constant 2 : index // CHECK: %[[C2:.*]] = arith.constant 2 : index
// CHECK: %{{.*}} = tensor.dim %[[ARG]], %[[C2]] : tensor<4x4x?xf32> // CHECK: %{{.*}} = tensor.dim %[[ARG]], %[[C2]] : tensor<4x4x?xf32>
%c2 = constant 2 : index %c2 = arith.constant 2 : index
%0 = tensor.dim %arg0, %c2 : tensor<4x4x?xf32> %0 = tensor.dim %arg0, %c2 : tensor<4x4x?xf32>
// use dim as an index to ensure type correctness // use dim as an index to ensure type correctness
%1 = affine.apply affine_map<(d0) -> (d0)>(%0) %1 = affine.apply affine_map<(d0) -> (d0)>(%0)
return return
} }
// CHECK-LABEL: func @test_splat_op // CHECK-LABEL: func @test_splat_op
// CHECK-SAME: [[S:%arg[0-9]+]]: f32 // CHECK-SAME: [[S:%arg[0-9]+]]: f32
Show All 34 Lines
} }
// CHECK-LABEL: func @generic_atomic_rmw // CHECK-LABEL: func @generic_atomic_rmw
// CHECK-SAME: ([[BUF:%.*]]: memref<1x2xf32>, [[I:%.*]]: index, [[J:%.*]]: index) // CHECK-SAME: ([[BUF:%.*]]: memref<1x2xf32>, [[I:%.*]]: index, [[J:%.*]]: index)
func @generic_atomic_rmw(%I: memref<1x2xf32>, %i : index, %j : index) { func @generic_atomic_rmw(%I: memref<1x2xf32>, %i : index, %j : index) {
%x = generic_atomic_rmw %I[%i, %j] : memref<1x2xf32> { %x = generic_atomic_rmw %I[%i, %j] : memref<1x2xf32> {
// CHECK-NEXT: generic_atomic_rmw [[BUF]]{{\[}}[[I]], [[J]]] : memref // CHECK-NEXT: generic_atomic_rmw [[BUF]]{{\[}}[[I]], [[J]]] : memref
^bb0(%old_value : f32): ^bb0(%old_value : f32):
%c1 = constant 1.0 : f32 %c1 = arith.constant 1.0 : f32
%out = addf %c1, %old_value : f32 %out = arith.addf %c1, %old_value : f32
atomic_yield %out : f32 atomic_yield %out : f32
// CHECK: index_attr = 8 : index // CHECK: index_attr = 8 : index
} { index_attr = 8 : index } } { index_attr = 8 : index }
return return
} }
// CHECK-LABEL: func @assume_alignment // CHECK-LABEL: func @assume_alignment
// CHECK-SAME: %[[MEMREF:.*]]: memref<4x4xf16> // CHECK-SAME: %[[MEMREF:.*]]: memref<4x4xf16>
func @assume_alignment(%0: memref<4x4xf16>) { func @assume_alignment(%0: memref<4x4xf16>) {
// CHECK: memref.assume_alignment %[[MEMREF]], 16 : memref<4x4xf16> // CHECK: memref.assume_alignment %[[MEMREF]], 16 : memref<4x4xf16>
memref.assume_alignment %0, 16 : memref<4x4xf16> memref.assume_alignment %0, 16 : memref<4x4xf16>
return return
} }
// CHECK-LABEL: func @slice({{.*}}) { // CHECK-LABEL: func @slice({{.*}}) {
func @slice(%t: tensor<8x16x4xf32>, %idx : index) { func @slice(%t: tensor<8x16x4xf32>, %idx : index) {
%c0 = constant 0 : index %c0 = arith.constant 0 : index
%c1 = constant 1 : index %c1 = arith.constant 1 : index
// CHECK: tensor.extract_slice // CHECK: tensor.extract_slice
// CHECK-SAME: tensor<8x16x4xf32> to tensor<?x?x?xf32> // CHECK-SAME: tensor<8x16x4xf32> to tensor<?x?x?xf32>
%1 = tensor.extract_slice %t[%c0, %c0, %c0][%idx, %idx, %idx][%c1, %c1, %c1] %1 = tensor.extract_slice %t[%c0, %c0, %c0][%idx, %idx, %idx][%c1, %c1, %c1]
: tensor<8x16x4xf32> to tensor<?x?x?xf32> : tensor<8x16x4xf32> to tensor<?x?x?xf32>
// CHECK: tensor.extract_slice // CHECK: tensor.extract_slice
// CHECK-SAME: tensor<8x16x4xf32> to tensor<4x4x4xf32> // CHECK-SAME: tensor<8x16x4xf32> to tensor<4x4x4xf32>
Show All 9 Lines
} }
// CHECK-LABEL: func @insert_slice({{.*}}) { // CHECK-LABEL: func @insert_slice({{.*}}) {
func @insert_slice( func @insert_slice(
%t: tensor<8x16x4xf32>, %t: tensor<8x16x4xf32>,
%t2: tensor<16x32x8xf32>, %t2: tensor<16x32x8xf32>,
%t3: tensor<4x4xf32>, %t3: tensor<4x4xf32>,
%idx : index) { %idx : index) {
%c0 = constant 0 : index %c0 = arith.constant 0 : index
%c1 = constant 1 : index %c1 = arith.constant 1 : index
// CHECK: tensor.insert_slice // CHECK: tensor.insert_slice
// CHECK-SAME: tensor<8x16x4xf32> into tensor<16x32x8xf32> // CHECK-SAME: tensor<8x16x4xf32> into tensor<16x32x8xf32>
%1 = tensor.insert_slice %t into %t2[%c0, %c0, %c0][%idx, %idx, %idx][%c1, %c1, %c1] %1 = tensor.insert_slice %t into %t2[%c0, %c0, %c0][%idx, %idx, %idx][%c1, %c1, %c1]
: tensor<8x16x4xf32> into tensor<16x32x8xf32> : tensor<8x16x4xf32> into tensor<16x32x8xf32>
// CHECK: tensor.insert_slice // CHECK: tensor.insert_slice
// CHECK-SAME: tensor<8x16x4xf32> into tensor<16x32x8xf32> // CHECK-SAME: tensor<8x16x4xf32> into tensor<16x32x8xf32>
Show All 10 Lines

mlir/test/IR/diagnostic-handler.mlir

mlir/test/IR/elements-attr-interface.mlir

mlir/test/IR/invalid-ops.mlir

// RUN: mlir-opt -allow-unregistered-dialect %s -split-input-file -verify-diagnostics // RUN: mlir-opt -allow-unregistered-dialect %s -split-input-file -verify-diagnostics
func @dim(%arg : tensor<1x?xf32>) { func @dim(%arg : tensor<1x?xf32>) {
%c2 = constant 2 : index %c2 = arith.constant 2 : index
rriddleUnsubmitted
Done
ReplyInline Actions

Same comment as core-ops.mlir

rriddle: Same comment as core-ops.mlir
tensor.dim %arg, %c2 : tensor<1x?xf32> // expected-error {{'tensor.dim' op index is out of range}} tensor.dim %arg, %c2 : tensor<1x?xf32> // expected-error {{'tensor.dim' op index is out of range}}
return return
} }
// ----- // -----
func @rank(f32) { func @rank(f32) {
^bb(%0: f32): ^bb(%0: f32):
"std.rank"(%0): (f32)->index // expected-error {{'std.rank' op operand #0 must be any memref or tensor type}} "std.rank"(%0): (f32)->index // expected-error {{'std.rank' op operand #0 must be any memref or tensor type}}
return return
} }
// ----- // -----
func @constant() {
^bb:
%x = "std.constant"(){value = "xyz"} : () -> i32 // expected-error {{unsupported 'value' attribute}}
return
}
// -----
func @constant_out_of_range() {
^bb:
%x = "std.constant"(){value = 100} : () -> i1 // expected-error {{requires attribute's type ('i64') to match op's return type ('i1')}}
return
}
// -----
func @constant_wrong_type() {
^bb:
%x = "std.constant"(){value = 10.} : () -> f32 // expected-error {{requires attribute's type ('f64') to match op's return type ('f32')}}
return
}
// -----
func @affine_apply_no_map() { func @affine_apply_no_map() {
^bb0: ^bb0:
%i = constant 0 : index %i = arith.constant 0 : index
%x = "affine.apply" (%i) { } : (index) -> (index) // expected-error {{requires attribute 'map'}} %x = "affine.apply" (%i) { } : (index) -> (index) // expected-error {{requires attribute 'map'}}
return return
} }
// ----- // -----
func @affine_apply_wrong_operand_count() { func @affine_apply_wrong_operand_count() {
^bb0: ^bb0:
%i = constant 0 : index %i = arith.constant 0 : index
%x = "affine.apply" (%i) {map = affine_map<(d0, d1) -> ((d0 + 1), (d1 + 2))>} : (index) -> (index) // expected-error {{'affine.apply' op operand count and affine map dimension and symbol count must match}} %x = "affine.apply" (%i) {map = affine_map<(d0, d1) -> ((d0 + 1), (d1 + 2))>} : (index) -> (index) // expected-error {{'affine.apply' op operand count and affine map dimension and symbol count must match}}
return return
} }
// ----- // -----
func @affine_apply_wrong_result_count() { func @affine_apply_wrong_result_count() {
^bb0: ^bb0:
%i = constant 0 : index %i = arith.constant 0 : index
%j = constant 1 : index %j = arith.constant 1 : index
%x = "affine.apply" (%i, %j) {map = affine_map<(d0, d1) -> ((d0 + 1), (d1 + 2))>} : (index,index) -> (index) // expected-error {{'affine.apply' op mapping must produce one value}} %x = "affine.apply" (%i, %j) {map = affine_map<(d0, d1) -> ((d0 + 1), (d1 + 2))>} : (index,index) -> (index) // expected-error {{'affine.apply' op mapping must produce one value}}
return return
} }
// ----- // -----
func @unknown_custom_op() { func @unknown_custom_op() {
^bb0: ^bb0:
%i = test.crazyThing() {value = 0} : () -> index // expected-error {{custom op 'test.crazyThing' is unknown}} %i = test.crazyThing() {value = 0} : () -> index // expected-error {{custom op 'test.crazyThing' is unknown}}
return return
} }
// ----- // -----
func @unknown_std_op() { func @unknown_std_op() {
// expected-error@+1 {{unregistered operation 'std.foo_bar_op' found in dialect ('std') that does not allow unknown operations}} // expected-error@+1 {{unregistered operation 'std.foo_bar_op' found in dialect ('std') that does not allow unknown operations}}
%0 = "std.foo_bar_op"() : () -> index %0 = "std.foo_bar_op"() : () -> index
return return
} }
// ----- // -----
func @bad_alloc_wrong_dynamic_dim_count() { func @bad_alloc_wrong_dynamic_dim_count() {
^bb0: ^bb0:
%0 = constant 7 : index %0 = arith.constant 7 : index
// Test alloc with wrong number of dynamic dimensions. // Test alloc with wrong number of dynamic dimensions.
// expected-error@+1 {{dimension operand count does not equal memref dynamic dimension count}} // expected-error@+1 {{dimension operand count does not equal memref dynamic dimension count}}
%1 = memref.alloc(%0)[%0] : memref<2x4xf32, affine_map<(d0, d1)[s0] -> ((d0 + s0), d1)>, 1> %1 = memref.alloc(%0)[%0] : memref<2x4xf32, affine_map<(d0, d1)[s0] -> ((d0 + s0), d1)>, 1>
return return
} }
// ----- // -----
func @bad_alloc_wrong_symbol_count() { func @bad_alloc_wrong_symbol_count() {
^bb0: ^bb0:
%0 = constant 7 : index %0 = arith.constant 7 : index
// Test alloc with wrong number of symbols // Test alloc with wrong number of symbols
// expected-error@+1 {{symbol operand count does not equal memref symbol count}} // expected-error@+1 {{symbol operand count does not equal memref symbol count}}
%1 = memref.alloc(%0) : memref<2x?xf32, affine_map<(d0, d1)[s0] -> ((d0 + s0), d1)>, 1> %1 = memref.alloc(%0) : memref<2x?xf32, affine_map<(d0, d1)[s0] -> ((d0 + s0), d1)>, 1>
return return
} }
// ----- // -----
func @test_store_zero_results() { func @test_store_zero_results() {
^bb0: ^bb0:
%0 = memref.alloc() : memref<1024x64xf32, affine_map<(d0, d1) -> (d0, d1)>, 1> %0 = memref.alloc() : memref<1024x64xf32, affine_map<(d0, d1) -> (d0, d1)>, 1>
%1 = constant 0 : index %1 = arith.constant 0 : index
%2 = constant 1 : index %2 = arith.constant 1 : index
%3 = memref.load %0[%1, %2] : memref<1024x64xf32, affine_map<(d0, d1) -> (d0, d1)>, 1> %3 = memref.load %0[%1, %2] : memref<1024x64xf32, affine_map<(d0, d1) -> (d0, d1)>, 1>
// Test that store returns zero results. // Test that store returns zero results.
%4 = memref.store %3, %0[%1, %2] : memref<1024x64xf32, affine_map<(d0, d1) -> (d0, d1)>, 1> // expected-error {{cannot name an operation with no results}} %4 = memref.store %3, %0[%1, %2] : memref<1024x64xf32, affine_map<(d0, d1) -> (d0, d1)>, 1> // expected-error {{cannot name an operation with no results}}
return return
} }
// ----- // -----
func @test_store_zero_results2(%x: i32, %p: memref<i32>) { func @test_store_zero_results2(%x: i32, %p: memref<i32>) {
"memref.store"(%x,%p) : (i32, memref<i32>) -> i32 // expected-error {{'memref.store' op requires zero results}} "memref.store"(%x,%p) : (i32, memref<i32>) -> i32 // expected-error {{'memref.store' op requires zero results}}
return return
} }
// ----- // -----
func @test_alloc_memref_map_rank_mismatch() { func @test_alloc_memref_map_rank_mismatch() {
^bb0: ^bb0:
%0 = memref.alloc() : memref<1024x64xf32, affine_map<(d0) -> (d0)>, 1> // expected-error {{memref affine map dimension mismatch}} %0 = memref.alloc() : memref<1024x64xf32, affine_map<(d0) -> (d0)>, 1> // expected-error {{memref affine map dimension mismatch}}
return return
} }
// ----- // -----
func @intlimit2() {
^bb:
%0 = "std.constant"() {value = 0} : () -> i16777215
%1 = "std.constant"() {value = 1} : () -> i16777216 // expected-error {{integer bitwidth is limited to 16777215 bits}}
return
}
// -----
func @calls(%arg0: i32) { func @calls(%arg0: i32) {
%x = call @calls() : () -> i32 // expected-error {{incorrect number of operands for callee}} %x = call @calls() : () -> i32 // expected-error {{incorrect number of operands for callee}}
return return
} }
// ----- // -----
func @func_with_ops(f32) {
^bb0(%a : f32):
%sf = addf %a, %a, %a : f32 // expected-error {{'std.addf' op expected 2 operands}}
}
// -----
func @func_with_ops(f32) {
^bb0(%a : f32):
%sf = addf(%a, %a) : f32 // expected-error {{'std.addf' expected function type}}
}
// -----
func @func_with_ops(f32) {
^bb0(%a : f32):
%sf = addf{%a, %a} : f32 // expected-error {{expected attribute name}}
}
// -----
func @func_with_ops(f32) {
^bb0(%a : f32):
// expected-error@+1 {{'std.addi' op operand #0 must be signless-integer-like}}
%sf = addi %a, %a : f32
}
// -----
func @func_with_ops(i32) {
^bb0(%a : i32):
%sf = addf %a, %a : i32 // expected-error {{'std.addf' op operand #0 must be floating-point-like}}
}
// -----
func @func_with_ops(i32) {
^bb0(%a : i32):
// expected-error@+1 {{failed to satisfy constraint: allowed 64-bit signless integer cases: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9}}
%r = "std.cmpi"(%a, %a) {predicate = 42} : (i32, i32) -> i1
}
// -----
// Comparison are defined for arguments of the same type.
func @func_with_ops(i32, i64) {
^bb0(%a : i32, %b : i64): // expected-note {{prior use here}}
%r = cmpi eq, %a, %b : i32 // expected-error {{use of value '%b' expects different type than prior uses}}
}
// -----
// Comparisons must have the "predicate" attribute.
func @func_with_ops(i32, i32) {
^bb0(%a : i32, %b : i32):
%r = cmpi %a, %b : i32 // expected-error {{expected string or keyword containing one of the following enum values}}
}
// -----
// Integer comparisons are not recognized for float types.
func @func_with_ops(f32, f32) {
^bb0(%a : f32, %b : f32):
%r = cmpi eq, %a, %b : f32 // expected-error {{'lhs' must be signless-integer-like, but got 'f32'}}
}
// -----
// Result type must be boolean like.
func @func_with_ops(i32, i32) {
^bb0(%a : i32, %b : i32):
%r = "std.cmpi"(%a, %b) {predicate = 0} : (i32, i32) -> i32 // expected-error {{op result #0 must be bool-like}}
}
// -----
func @func_with_ops(i32, i32) {
^bb0(%a : i32, %b : i32):
// expected-error@+1 {{requires attribute 'predicate'}}
%r = "std.cmpi"(%a, %b) {foo = 1} : (i32, i32) -> i1
}
// -----
func @func_with_ops() {
^bb0:
%c = constant dense<0> : vector<42 x i32>
// expected-error@+1 {{all non-scalar operands/results must have the same shape and base type}}
%r = "std.cmpi"(%c, %c) {predicate = 0} : (vector<42 x i32>, vector<42 x i32>) -> vector<41 x i1>
}
// -----
func @func_with_ops(i32, i32, i32) { func @func_with_ops(i32, i32, i32) {
^bb0(%cond : i32, %t : i32, %f : i32): ^bb0(%cond : i32, %t : i32, %f : i32):
// expected-error@+2 {{different type than prior uses}} // expected-error@+2 {{different type than prior uses}}
// expected-note@-2 {{prior use here}} // expected-note@-2 {{prior use here}}
%r = select %cond, %t, %f : i32 %r = select %cond, %t, %f : i32
} }
// ----- // -----
Show All 25 Lines
func @func_with_ops(tensor<12xi1>, tensor<42xi32>, tensor<42xi32>) { func @func_with_ops(tensor<12xi1>, tensor<42xi32>, tensor<42xi32>) {
^bb0(%cond : tensor<12xi1>, %t : tensor<42xi32>, %f : tensor<42xi32>): ^bb0(%cond : tensor<12xi1>, %t : tensor<42xi32>, %f : tensor<42xi32>):
// expected-error@+1 {{all non-scalar operands/results must have the same shape and base type}} // expected-error@+1 {{all non-scalar operands/results must have the same shape and base type}}
%r = "std.select"(%cond, %t, %f) : (tensor<12xi1>, tensor<42xi32>, tensor<42xi32>) -> tensor<42xi32> %r = "std.select"(%cond, %t, %f) : (tensor<12xi1>, tensor<42xi32>, tensor<42xi32>) -> tensor<42xi32>
} }
// ----- // -----
func @invalid_cmp_shape(%idx : () -> ()) {
// expected-error@+1 {{'lhs' must be signless-integer-like, but got '() -> ()'}}
%cmp = cmpi eq, %idx, %idx : () -> ()
// -----
func @invalid_cmp_attr(%idx : i32) {
// expected-error@+1 {{expected string or keyword containing one of the following enum values}}
%cmp = cmpi i1, %idx, %idx : i32
// -----
func @cmpf_generic_invalid_predicate_value(%a : f32) {
// expected-error@+1 {{attribute 'predicate' failed to satisfy constraint: allowed 64-bit signless integer cases}}
%r = "std.cmpf"(%a, %a) {predicate = 42} : (f32, f32) -> i1
}
// -----
func @cmpf_canonical_invalid_predicate_value(%a : f32) {
// expected-error@+1 {{expected string or keyword containing one of the following enum values}}
%r = cmpf foo, %a, %a : f32
}
// -----
func @cmpf_canonical_invalid_predicate_value_signed(%a : f32) {
// expected-error@+1 {{expected string or keyword containing one of the following enum values}}
%r = cmpf sge, %a, %a : f32
}
// -----
func @cmpf_canonical_invalid_predicate_value_no_order(%a : f32) {
// expected-error@+1 {{expected string or keyword containing one of the following enum values}}
%r = cmpf eq, %a, %a : f32
}
// -----
func @cmpf_canonical_no_predicate_attr(%a : f32, %b : f32) {
%r = cmpf %a, %b : f32 // expected-error {{}}
}
// -----
func @cmpf_generic_no_predicate_attr(%a : f32, %b : f32) {
// expected-error@+1 {{requires attribute 'predicate'}}
%r = "std.cmpf"(%a, %b) {foo = 1} : (f32, f32) -> i1
}
// -----
func @cmpf_wrong_type(%a : i32, %b : i32) {
%r = cmpf oeq, %a, %b : i32 // expected-error {{must be floating-point-like}}
}
// -----
func @cmpf_generic_wrong_result_type(%a : f32, %b : f32) {
// expected-error@+1 {{result #0 must be bool-like}}
%r = "std.cmpf"(%a, %b) {predicate = 0} : (f32, f32) -> f32
}
// -----
func @cmpf_canonical_wrong_result_type(%a : f32, %b : f32) -> f32 {
%r = cmpf oeq, %a, %b : f32 // expected-note {{prior use here}}
// expected-error@+1 {{use of value '%r' expects different type than prior uses}}
return %r : f32
}
// -----
func @cmpf_result_shape_mismatch(%a : vector<42xf32>) {
// expected-error@+1 {{all non-scalar operands/results must have the same shape and base type}}
%r = "std.cmpf"(%a, %a) {predicate = 0} : (vector<42 x f32>, vector<42 x f32>) -> vector<41 x i1>
}
// -----
func @cmpf_operand_shape_mismatch(%a : vector<42xf32>, %b : vector<41xf32>) {
// expected-error@+1 {{op requires all operands to have the same type}}
%r = "std.cmpf"(%a, %b) {predicate = 0} : (vector<42 x f32>, vector<41 x f32>) -> vector<42 x i1>
}
// -----
func @cmpf_generic_operand_type_mismatch(%a : f32, %b : f64) {
// expected-error@+1 {{op requires all operands to have the same type}}
%r = "std.cmpf"(%a, %b) {predicate = 0} : (f32, f64) -> i1
}
// -----
func @cmpf_canonical_type_mismatch(%a : f32, %b : f64) { // expected-note {{prior use here}}
// expected-error@+1 {{use of value '%b' expects different type than prior uses}}
%r = cmpf oeq, %a, %b : f32
}
// -----
func @index_cast_index_to_index(%arg0: index) {
// expected-error@+1 {{are cast incompatible}}
%0 = index_cast %arg0: index to index
return
}
// -----
func @index_cast_float(%arg0: index, %arg1: f32) {
// expected-error@+1 {{are cast incompatible}}
%0 = index_cast %arg0 : index to f32
return
}
// -----
func @index_cast_float_to_index(%arg0: f32) {
// expected-error@+1 {{are cast incompatible}}
%0 = index_cast %arg0 : f32 to index
return
}
// -----
func @sitofp_i32_to_i64(%arg0 : i32) {
// expected-error@+1 {{are cast incompatible}}
%0 = sitofp %arg0 : i32 to i64
return
}
// -----
func @sitofp_f32_to_i32(%arg0 : f32) {
// expected-error@+1 {{are cast incompatible}}
%0 = sitofp %arg0 : f32 to i32
return
}
// -----
func @fpext_f32_to_f16(%arg0 : f32) {
// expected-error@+1 {{are cast incompatible}}
%0 = fpext %arg0 : f32 to f16
return
}
// -----
func @fpext_f16_to_f16(%arg0 : f16) {
// expected-error@+1 {{are cast incompatible}}
%0 = fpext %arg0 : f16 to f16
return
}
// -----
func @fpext_i32_to_f32(%arg0 : i32) {
// expected-error@+1 {{are cast incompatible}}
%0 = fpext %arg0 : i32 to f32
return
}
// -----
func @fpext_f32_to_i32(%arg0 : f32) {
// expected-error@+1 {{are cast incompatible}}
%0 = fpext %arg0 : f32 to i32
return
}
// -----
func @fpext_vec(%arg0 : vector<2xf16>) {
// expected-error@+1 {{all non-scalar operands/results must have the same shape and base type}}
%0 = fpext %arg0 : vector<2xf16> to vector<3xf32>
return
}
// -----
func @fpext_vec_f32_to_f16(%arg0 : vector<2xf32>) {
// expected-error@+1 {{are cast incompatible}}
%0 = fpext %arg0 : vector<2xf32> to vector<2xf16>
return
}
// -----
func @fpext_vec_f16_to_f16(%arg0 : vector<2xf16>) {
// expected-error@+1 {{are cast incompatible}}
%0 = fpext %arg0 : vector<2xf16> to vector<2xf16>
return
}
// -----
func @fpext_vec_i32_to_f32(%arg0 : vector<2xi32>) {
// expected-error@+1 {{are cast incompatible}}
%0 = fpext %arg0 : vector<2xi32> to vector<2xf32>
return
}
// -----
func @fpext_vec_f32_to_i32(%arg0 : vector<2xf32>) {
// expected-error@+1 {{are cast incompatible}}
%0 = fpext %arg0 : vector<2xf32> to vector<2xi32>
return
}
// -----
func @fptrunc_f16_to_f32(%arg0 : f16) {
// expected-error@+1 {{are cast incompatible}}
%0 = fptrunc %arg0 : f16 to f32
return
}
// -----
func @fptrunc_f32_to_f32(%arg0 : f32) {
// expected-error@+1 {{are cast incompatible}}
%0 = fptrunc %arg0 : f32 to f32
return
}
// -----
func @fptrunc_i32_to_f32(%arg0 : i32) {
// expected-error@+1 {{are cast incompatible}}
%0 = fptrunc %arg0 : i32 to f32
return
}
// -----
func @fptrunc_f32_to_i32(%arg0 : f32) {
// expected-error@+1 {{are cast incompatible}}
%0 = fptrunc %arg0 : f32 to i32
return
}
// -----
func @fptrunc_vec(%arg0 : vector<2xf16>) {
// expected-error@+1 {{all non-scalar operands/results must have the same shape and base type}}
%0 = fptrunc %arg0 : vector<2xf16> to vector<3xf32>
return
}
// -----
func @fptrunc_vec_f16_to_f32(%arg0 : vector<2xf16>) {
// expected-error@+1 {{are cast incompatible}}
%0 = fptrunc %arg0 : vector<2xf16> to vector<2xf32>
return
}
// -----
func @fptrunc_vec_f32_to_f32(%arg0 : vector<2xf32>) {
// expected-error@+1 {{are cast incompatible}}
%0 = fptrunc %arg0 : vector<2xf32> to vector<2xf32>
return
}
// -----
func @fptrunc_vec_i32_to_f32(%arg0 : vector<2xi32>) {
// expected-error@+1 {{are cast incompatible}}
%0 = fptrunc %arg0 : vector<2xi32> to vector<2xf32>
return
}
// -----
func @fptrunc_vec_f32_to_i32(%arg0 : vector<2xf32>) {
// expected-error@+1 {{are cast incompatible}}
%0 = fptrunc %arg0 : vector<2xf32> to vector<2xi32>
return
}
// -----
func @sexti_index_as_operand(%arg0 : index) {
// expected-error@+1 {{'index' is not a valid operand type}}
%0 = sexti %arg0 : index to i128
return
}
// -----
func @zexti_index_as_operand(%arg0 : index) {
// expected-error@+1 {{'index' is not a valid operand type}}
%0 = zexti %arg0 : index to i128
return
}
// -----
func @trunci_index_as_operand(%arg0 : index) {
// expected-error@+1 {{'index' is not a valid operand type}}
%2 = trunci %arg0 : index to i128
return
}
// -----
func @sexti_index_as_result(%arg0 : i1) {
// expected-error@+1 {{'index' is not a valid result type}}
%0 = sexti %arg0 : i1 to index
return
}
// -----
func @zexti_index_as_operand(%arg0 : i1) {
// expected-error@+1 {{'index' is not a valid result type}}
%0 = zexti %arg0 : i1 to index
return
}
// -----
func @trunci_index_as_result(%arg0 : i128) {
// expected-error@+1 {{'index' is not a valid result type}}
%2 = trunci %arg0 : i128 to index
return
}
// -----
func @sexti_cast_to_narrower(%arg0 : i16) {
// expected-error@+1 {{must be wider}}
%0 = sexti %arg0 : i16 to i15
return
}
// -----
func @zexti_cast_to_narrower(%arg0 : i16) {
// expected-error@+1 {{must be wider}}
%0 = zexti %arg0 : i16 to i15
return
}
// -----
func @trunci_cast_to_wider(%arg0 : i16) {
// expected-error@+1 {{must be wider}}
%0 = trunci %arg0 : i16 to i17
return
}
// -----
func @sexti_cast_to_same_width(%arg0 : i16) {
// expected-error@+1 {{must be wider}}
%0 = sexti %arg0 : i16 to i16
return
}
// -----
func @zexti_cast_to_same_width(%arg0 : i16) {
// expected-error@+1 {{must be wider}}
%0 = zexti %arg0 : i16 to i16
return
}
// -----
func @trunci_cast_to_same_width(%arg0 : i16) {
// expected-error@+1 {{must be wider}}
%0 = trunci %arg0 : i16 to i16
return
}
// -----
func @return_not_in_function() { func @return_not_in_function() {
"foo.region"() ({ "foo.region"() ({
// expected-error@+1 {{'std.return' op expects parent op 'builtin.func'}} // expected-error@+1 {{'std.return' op expects parent op 'builtin.func'}}
return return
}): () -> () }): () -> ()
return return
} }
▲ Show 20 Lines • Show All 283 Lines • ▼ Show 20 Lines
} }
// ----- // -----
func @generic_atomic_rmw_wrong_arg_num(%I: memref<10xf32>, %i : index) { func @generic_atomic_rmw_wrong_arg_num(%I: memref<10xf32>, %i : index) {
// expected-error@+1 {{expected single number of entry block arguments}} // expected-error@+1 {{expected single number of entry block arguments}}
%x = generic_atomic_rmw %I[%i] : memref<10xf32> { %x = generic_atomic_rmw %I[%i] : memref<10xf32> {
^bb0(%arg0 : f32, %arg1 : f32): ^bb0(%arg0 : f32, %arg1 : f32):
%c1 = constant 1.0 : f32 %c1 = arith.constant 1.0 : f32
atomic_yield %c1 : f32 atomic_yield %c1 : f32
} }
return return
} }
// ----- // -----
func @generic_atomic_rmw_wrong_arg_type(%I: memref<10xf32>, %i : index) { func @generic_atomic_rmw_wrong_arg_type(%I: memref<10xf32>, %i : index) {
// expected-error@+1 {{expected block argument of the same type result type}} // expected-error@+1 {{expected block argument of the same type result type}}
%x = generic_atomic_rmw %I[%i] : memref<10xf32> { %x = generic_atomic_rmw %I[%i] : memref<10xf32> {
^bb0(%old_value : i32): ^bb0(%old_value : i32):
%c1 = constant 1.0 : f32 %c1 = arith.constant 1.0 : f32
atomic_yield %c1 : f32 atomic_yield %c1 : f32
} }
return return
} }
// ----- // -----
func @generic_atomic_rmw_result_type_mismatch(%I: memref<10xf32>, %i : index) { func @generic_atomic_rmw_result_type_mismatch(%I: memref<10xf32>, %i : index) {
// expected-error@+1 {{failed to verify that result type matches element type of memref}} // expected-error@+1 {{failed to verify that result type matches element type of memref}}
%0 = "std.generic_atomic_rmw"(%I, %i) ( { %0 = "std.generic_atomic_rmw"(%I, %i) ( {
^bb0(%old_value: f32): ^bb0(%old_value: f32):
%c1 = constant 1.0 : f32 %c1 = arith.constant 1.0 : f32
atomic_yield %c1 : f32 atomic_yield %c1 : f32
}) : (memref<10xf32>, index) -> i32 }) : (memref<10xf32>, index) -> i32
return return
} }
// ----- // -----
func @generic_atomic_rmw_has_side_effects(%I: memref<10xf32>, %i : index) { func @generic_atomic_rmw_has_side_effects(%I: memref<10xf32>, %i : index) {
// expected-error@+4 {{should contain only operations with no side effects}} // expected-error@+4 {{should contain only operations with no side effects}}
%x = generic_atomic_rmw %I[%i] : memref<10xf32> { %x = generic_atomic_rmw %I[%i] : memref<10xf32> {
^bb0(%old_value : f32): ^bb0(%old_value : f32):
%c1 = constant 1.0 : f32 %c1 = arith.constant 1.0 : f32
%buf = memref.alloc() : memref<2048xf32> %buf = memref.alloc() : memref<2048xf32>
atomic_yield %c1 : f32 atomic_yield %c1 : f32
} }
} }
// ----- // -----
func @atomic_yield_type_mismatch(%I: memref<10xf32>, %i : index) { func @atomic_yield_type_mismatch(%I: memref<10xf32>, %i : index) {
// expected-error@+4 {{op types mismatch between yield op: 'i32' and its parent: 'f32'}} // expected-error@+4 {{op types mismatch between yield op: 'i32' and its parent: 'f32'}}
%x = generic_atomic_rmw %I[%i] : memref<10xf32> { %x = generic_atomic_rmw %I[%i] : memref<10xf32> {
^bb0(%old_value : f32): ^bb0(%old_value : f32):
%c1 = constant 1 : i32 %c1 = arith.constant 1 : i32
atomic_yield %c1 : i32 atomic_yield %c1 : i32
} }
return return
} }
// ----- // -----
// alignment is not power of 2. // alignment is not power of 2.
▲ Show 20 Lines • Show All 50 Lines • Show Last 20 Lines

mlir/test/IR/invalid.mlir

mlir/test/IR/locations.mlir

mlir/test/IR/memory-ops.mlir

mlir/test/IR/op-stats.mlir

mlir/test/IR/opaque_locations.mlir

mlir/test/IR/operand.mlir

mlir/test/IR/parser.mlir

mlir/test/IR/pretty-locations.mlir

mlir/test/IR/repro_b120295301.mlir

mlir/test/IR/test-matchers.mlir

mlir/test/IR/visitors.mlir

mlir/test/Integration/Dialect/Async/CPU/microbench-linalg-async-parallel-for.mlir

mlir/test/Integration/Dialect/Async/CPU/microbench-scf-async-parallel-for.mlir

mlir/test/Integration/Dialect/Async/CPU/test-async-parallel-for-1d.mlir

mlir/test/Integration/Dialect/Async/CPU/test-async-parallel-for-2d.mlir

mlir/test/Integration/Dialect/Linalg/CPU/benchmark_matmul.mlir

mlir/test/Integration/Dialect/Linalg/CPU/matmul-vs-matvec.mlir

mlir/test/Integration/Dialect/Linalg/CPU/rank-reducing-subview.mlir

mlir/test/Integration/Dialect/Linalg/CPU/test-collapse-tensor.mlir

mlir/test/Integration/Dialect/Linalg/CPU/test-comprehensive-bufferize.mlir

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-1d-call.mlir

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-1d-nwc-wcf-call.mlir

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-2d-call.mlir

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-2d-nhwc-hwcf-call.mlir

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-3d-call.mlir

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-3d-ndhwc-dhwcf-call.mlir

mlir/test/Integration/Dialect/Linalg/CPU/test-elementwise.mlir

mlir/test/Integration/Dialect/Linalg/CPU/test-expand-tensor.mlir

mlir/test/Integration/Dialect/Linalg/CPU/test-padtensor.mlir

mlir/test/Integration/Dialect/Linalg/CPU/test-subtensor-insert-multiple-uses.mlir

mlir/test/Integration/Dialect/Linalg/CPU/test-subtensor-insert.mlir

mlir/test/Integration/Dialect/Linalg/CPU/test-tensor-e2e.mlir

mlir/test/Integration/Dialect/Linalg/CPU/test-tensor-matmul.mlir

mlir/test/Integration/Dialect/SparseTensor/CPU/dense_output.mlir

mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_cast.mlir

mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_constant_to_sparse_tensor.mlir

mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_conversion.mlir

mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_filter_conv2d.mlir

mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_flatten.mlir

mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_matvec.mlir

mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_mttkrp.mlir

mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_out_simple.mlir

mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_quantized_matmul.mlir

mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_reductions.mlir

mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_sampled_matmul.mlir

mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_sampled_mm_fusion.mlir

mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_scale.mlir

mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_spmm.mlir

mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_storage.mlir

mlir/test/Integration/Dialect/SparseTensor/CPU/sparse_sum.mlir

mlir/test/Integration/Dialect/Standard/CPU/test-ceil-floor-pos-neg.mlir

mlir/test/Integration/Dialect/Vector/CPU/AMX/test-mulf.mlir

mlir/test/Integration/Dialect/Vector/CPU/AMX/test-muli-ext.mlir

mlir/test/Integration/Dialect/Vector/CPU/AMX/test-muli.mlir

mlir/test/Integration/Dialect/Vector/CPU/AMX/test-tilezero-block.mlir

mlir/test/Integration/Dialect/Vector/CPU/AMX/test-tilezero.mlir

mlir/test/Integration/Dialect/Vector/CPU/X86Vector/test-dot.mlir

mlir/test/Integration/Dialect/Vector/CPU/X86Vector/test-mask-compress.mlir

mlir/test/Integration/Dialect/Vector/CPU/X86Vector/test-rsqrt.mlir

mlir/test/Integration/Dialect/Vector/CPU/X86Vector/test-sparse-dot-product.mlir

mlir/test/Integration/Dialect/Vector/CPU/X86Vector/test-vp2intersect-i32.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-broadcast.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-compress.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-contraction.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-create-mask-v4i1.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-create-mask.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-expand.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-extract-strided-slice.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-flat-transpose-col.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-flat-transpose-row.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-fma.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-gather.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-index-vectors.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-insert-strided-slice.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-maskedload.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-maskedstore.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-matrix-multiply-col.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-matrix-multiply-row.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-outerproduct-f32.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-outerproduct-i64.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-print-int.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-reductions-f32-reassoc.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-reductions-f32.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-reductions-f64-reassoc.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-reductions-f64.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-reductions-i32.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-reductions-i4.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-reductions-i64.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-reductions-si4.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-reductions-ui4.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-scatter.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-shape-cast.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-shuffle.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-sparse-dot-matvec.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-sparse-saxpy-jagged-matvec.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-transfer-read-1d.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-transfer-read-2d.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-transfer-read-3d.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-transfer-read.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-transfer-to-loops.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-transfer-write.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-transpose.mlir

mlir/test/Integration/Dialect/Vector/CPU/test-vector-distribute.mlir

mlir/test/Integration/GPU/CUDA/TensorCore/wmma-matmul-f16.mlir

mlir/test/Integration/GPU/CUDA/TensorCore/wmma-matmul-f32.mlir

mlir/test/Integration/GPU/CUDA/all-reduce-and.mlir

mlir/test/Integration/GPU/CUDA/all-reduce-max.mlir

mlir/test/Integration/GPU/CUDA/all-reduce-min.mlir

mlir/test/Integration/GPU/CUDA/all-reduce-op.mlir

mlir/test/Integration/GPU/CUDA/all-reduce-or.mlir

mlir/test/Integration/GPU/CUDA/all-reduce-region.mlir

mlir/test/Integration/GPU/CUDA/all-reduce-xor.mlir

mlir/test/Integration/GPU/CUDA/async.mlir

mlir/test/Integration/GPU/CUDA/gpu-to-cubin.mlir

mlir/test/Integration/GPU/CUDA/multiple-all-reduce.mlir

mlir/test/Integration/GPU/CUDA/shuffle.mlir

mlir/test/Integration/GPU/CUDA/two-modules.mlir

mlir/test/Integration/GPU/ROCM/gpu-to-hsaco.mlir

mlir/test/Integration/GPU/ROCM/two-modules.mlir

mlir/test/Integration/GPU/ROCM/vecadd.mlir

mlir/test/Integration/GPU/ROCM/vector-transferops.mlir

mlir/test/Interfaces/InferShapedTypeOpInterface/resolve-shaped-type-result-dims.mlir

mlir/test/Pass/ir-printing.mlir

mlir/test/Pass/run-reproducer.mlir

mlir/test/Target/Cpp/call.mlir

mlir/test/Target/Cpp/for.mlir

mlir/test/Target/Cpp/if.mlir

mlir/test/Target/Cpp/invalid.mlir

mlir/test/Target/Cpp/stdops.mlir

mlir/test/Target/LLVMIR/vector-to-llvm-ir.mlir

mlir/test/Transforms/buffer-deallocation.mlir

mlir/test/Transforms/buffer-hoisting.mlir

mlir/test/Transforms/buffer-loop-hoisting.mlir

mlir/test/Transforms/canonicalize-block-merge.mlir

mlir/test/Transforms/canonicalize-dce.mlir

mlir/test/Transforms/canonicalize-td.mlir

mlir/test/Transforms/canonicalize.mlir

mlir/test/Transforms/compose-subview.mlir

mlir/test/Transforms/constant-fold.mlir

mlir/test/Transforms/cse.mlir

mlir/test/Transforms/inlining.mlir

mlir/test/Transforms/loop-coalescing.mlir

mlir/test/Transforms/loop-fusion-2.mlir

mlir/test/Transforms/loop-fusion-3.mlir

mlir/test/Transforms/loop-fusion-4.mlir

mlir/test/Transforms/loop-fusion-dependence-check.mlir

mlir/test/Transforms/loop-fusion-slice-computation.mlir

mlir/test/Transforms/loop-fusion-transformation.mlir

mlir/test/Transforms/loop-fusion.mlir

mlir/test/Transforms/loop-invariant-code-motion.mlir

mlir/test/Transforms/memref-bound-check.mlir

mlir/test/Transforms/memref-dependence-check.mlir

mlir/test/Transforms/normalize-memrefs-ops-dynamic.mlir

mlir/test/Transforms/normalize-memrefs-ops.mlir

mlir/test/Transforms/normalize-memrefs.mlir

mlir/test/Transforms/parallel-loop-collapsing.mlir

mlir/test/Transforms/parametric-tiling.mlir

mlir/test/Transforms/pipeline-data-transfer.mlir

mlir/test/Transforms/print-op-graph.mlir

mlir/test/Transforms/promote-buffers-to-stack.mlir

mlir/test/Transforms/sccp-callgraph.mlir

mlir/test/Transforms/sccp-structured.mlir

mlir/test/Transforms/sccp.mlir

mlir/test/Transforms/scf-loop-unroll.mlir

mlir/test/Transforms/single-parallel-loop-collapsing.mlir

mlir/test/Transforms/strip-debuginfo.mlir

mlir/test/Transforms/test-canonicalize.mlir

mlir/test/Transforms/test-inlining.mlir

mlir/test/Transforms/test-operation-folder.mlir

mlir/test/Transforms/test-pattern-selective-replacement.mlir

mlir/test/lib/Dialect/GPU/TestGpuRewrite.cpp

mlir/test/lib/Dialect/Linalg/TestLinalgTransforms.cpp

mlir/test/lib/Dialect/Math/TestPolynomialApproximation.cpp

mlir/test/lib/Dialect/SCF/TestSCFUtils.cpp

mlir/test/lib/Dialect/Test/TestDialect.cpp

mlir/test/lib/Dialect/Test/TestPatterns.cpp

mlir/test/lib/Dialect/Vector/TestVectorTransforms.cpp

mlir/test/lib/IR/TestMatchers.cpp

mlir/test/lib/Transforms/TestLoopUnrolling.cpp

mlir/test/mlir-cpu-runner/async-error.mlir

mlir/test/mlir-cpu-runner/async-group.mlir

mlir/test/mlir-cpu-runner/async-value.mlir

mlir/test/mlir-cpu-runner/async.mlir

mlir/test/mlir-cpu-runner/bare_ptr_call_conv.mlir

mlir/test/mlir-cpu-runner/copy.mlir

mlir/test/mlir-cpu-runner/global_memref.mlir

mlir/test/mlir-cpu-runner/math_polynomial_approx.mlir

mlir/test/mlir-cpu-runner/memref_reinterpret_cast.mlir

mlir/test/mlir-cpu-runner/memref_reshape.mlir

mlir/test/mlir-cpu-runner/sgemm_naive_codegen.mlir

mlir/test/mlir-cpu-runner/unranked_memref.mlir

mlir/test/mlir-cpu-runner/utils.mlir

mlir/test/mlir-lsp-server/definition-split-file.test

mlir/test/mlir-lsp-server/definition.test

mlir/test/mlir-lsp-server/hover.test

mlir/test/mlir-lsp-server/references.test

mlir/test/mlir-opt/commandline.mlir

mlir/test/mlir-spirv-cpu-runner/double.mlir

mlir/test/mlir-spirv-cpu-runner/simple_add.mlir

mlir/test/mlir-tblgen/return-types.mlir

mlir/test/mlir-vulkan-runner/addf.mlir

mlir/test/mlir-vulkan-runner/addi.mlir

mlir/test/mlir-vulkan-runner/addi8.mlir

mlir/test/mlir-vulkan-runner/mulf.mlir

mlir/test/mlir-vulkan-runner/subf.mlir

mlir/test/mlir-vulkan-runner/time.mlir

mlir/test/python/dialects/linalg/opdsl/emit_structured_generic.py

mlir/test/python/dialects/linalg/ops.py

mlir/test/python/dialects/memref.py

mlir/test/python/dialects/scf.py

mlir/test/python/dialects/std.py

mlir/test/python/execution_engine.py

mlir/test/python/integration/dialects/linalg/opsrun.py

mlir/test/python/ir/dialects.py

mlir/test/python/ir/operation.py

mlir/tools/mlir-spirv-cpu-runner/CMakeLists.txt

mlir/tools/mlir-spirv-cpu-runner/mlir-spirv-cpu-runner.cpp

mlir/tools/mlir-vulkan-runner/CMakeLists.txt

mlir/tools/mlir-vulkan-runner/mlir-vulkan-runner.cpp

mlir/unittests/ExecutionEngine/CMakeLists.txt

mlir/unittests/ExecutionEngine/Invoke.cpp

mlir/unittests/Interfaces/InferTypeOpInterfaceTest.cpp

utils/bazel/llvm-project-overlay/mlir/BUILD.bazel

utils/bazel/llvm-project-overlay/mlir/test/BUILD.bazel