This is an archive of the discontinued LLVM Phabricator instance.

Differential D134045

[mlir] use strided layout in structured codegen-related tests
ClosedPublic

Authored by ftynse on Sep 16 2022, 6:54 AM.

Details

Reviewers
nicolasvasilache
aartbik
herhut
dcaballe
Commits
rGf3fae035c7a1: [mlir] use strided layout in structured codegen-related tests
Summary

All relevant operations have been switched to primarily use the strided
layout, but still support the affine map layout. Update the relevant
tests to use the strided format instead for compatibility with how ops
now print by default.

Diff Detail

Event Timeline

ftynse created this revision.Sep 16 2022, 6:54 AM
Herald added a reviewer: aartbik. · View Herald TranscriptSep 16 2022, 6:54 AM
Herald added a project: Restricted Project. · View Herald Transcript
Herald added subscribers: bzcheeseman, ThomasRaoux, awarzynski and 25 others. · View Herald Transcript
ftynse requested review of this revision.Sep 16 2022, 6:54 AM
Herald added a reviewer: herhut. · View Herald TranscriptSep 16 2022, 6:54 AM
Herald added a reviewer: dcaballe. · View Herald Transcript
Herald added a project: Restricted Project. · View Herald Transcript
Herald added a subscriber: stephenneuendorffer. · View Herald Transcript
Harbormaster completed remote builds in B187134: Diff 460736.Sep 16 2022, 7:19 AM
nicolasvasilache accepted this revision.Sep 16 2022, 8:50 AM
This revision is now accepted and ready to land.Sep 16 2022, 8:50 AM
Closed by commit rGf3fae035c7a1: [mlir] use strided layout in structured codegen-related tests (authored by ftynse). · Explain WhySep 16 2022, 11:11 PM
This revision was automatically updated to reflect the committed changes.
ftynse added a commit: rGf3fae035c7a1: [mlir] use strided layout in structured codegen-related tests.

Revision Contents

PathSize
mlir/
docs/
5 lines
Dialects/
Linalg/
39 lines
include/
mlir/
Dialect/
Bufferization/
IR/
4 lines
MemRef/
Transforms/
6 lines
test/
Conversion/
BufferizationToMemRef/
25 lines
MemRefToLLVM/
10 lines
SCFToGPU/
68 lines
Dialect/
Bufferization/
Transforms/
31 lines
Linalg/
11 lines
27 lines
16 lines
15 lines
MemRef/
73 lines
5 lines
Vector/
13 lines
12 lines
19 lines
Integration/
Dialect/
Vector/
CPU/
11 lines
Transforms/
12 lines

Diff 460970

mlir/docs/Bufferization.md

Show First 20 Lines • Show All 311 Lines • ▼ Show 20 Lines
%0 = "my_dialect.unbufferizable_op(%t) : (tensor<?x?xf32>) -> (tensor<?x?xf32>) %0 = "my_dialect.unbufferizable_op(%t) : (tensor<?x?xf32>) -> (tensor<?x?xf32>)
%1 = tensor.extract %0[%idx1, %idx2] : tensor<?xf32> %1 = tensor.extract %0[%idx1, %idx2] : tensor<?xf32>
``` ```
When bufferizing the above IR, One-Shot Bufferize inserts a `to_memref` ops with When bufferizing the above IR, One-Shot Bufferize inserts a `to_memref` ops with
dynamic offset and strides: dynamic offset and strides:
```mlir ```mlir
#map = affine_map<(d0, d1)[s0, s1, s2] -> (d0 * s1 + s0 + d1 * s2)>
%0 = "my_dialect.unbufferizable_op(%t) : (tensor<?x?xf32>) -> (tensor<?x?xf32>) %0 = "my_dialect.unbufferizable_op(%t) : (tensor<?x?xf32>) -> (tensor<?x?xf32>)
%0_m = bufferization.to_memref %0 : memref<?x?xf32, #map> %0_m = bufferization.to_memref %0 : memref<?x?xf32, strided<[?, ?], offset: ?>>
%1 = memref.load %0_m[%idx1, %idx2] : memref<?x?xf32, #map> %1 = memref.load %0_m[%idx1, %idx2] : memref<?x?xf32, strided<[?, ?], offset: ?>>
``` ```
All users of `%0` have fully dynamic layout maps. This ensures that the All users of `%0` have fully dynamic layout maps. This ensures that the
bufferized IR composes well with future bufferizations of `unbufferizable_op` bufferized IR composes well with future bufferizations of `unbufferizable_op`
(maybe bufferized by another pass), regardless of the exact memref type of the (maybe bufferized by another pass), regardless of the exact memref type of the
future bufferization. If the op turns out to be bufferized to an op with a future bufferization. If the op turns out to be bufferized to an op with a
simpler memref type (e.g., identity layout map), we expect that canonicalization simpler memref type (e.g., identity layout map), we expect that canonicalization
patterns would clean up unnecessarily dynamic layout maps. (Some of these patterns would clean up unnecessarily dynamic layout maps. (Some of these
▲ Show 20 Lines • Show All 363 Lines • Show Last 20 Lines

mlir/docs/Dialects/Linalg/_index.md

Show First 20 Lines • Show All 93 Lines • ▼ Show 20 Lines#accesses = [
affine_map<(m) -> (m)> affine_map<(m) -> (m)>
] ]
#attrs = { #attrs = {
indexing_maps = #accesses, indexing_maps = #accesses,
iterator_types = ["parallel"] iterator_types = ["parallel"]
} }
// memory layouts func.func @example(%A: memref<?xf32, strided<[1]>>,
#identity = affine_map<(d0) -> (d0)>
func.func @example(%A: memref<?xf32, #identity>,
%B: memref<?xvector<4xf32>, strided<[2], offset: 1>>) { %B: memref<?xvector<4xf32>, strided<[2], offset: 1>>) {
linalg.generic #attrs linalg.generic #attrs
ins(%A: memref<?xf32, #identity>) ins(%A: memref<?xf32, strided<[1]>>)
outs(%B: memref<?xvector<4xf32>, strided<[2], offset: 1>>) { outs(%B: memref<?xvector<4xf32>, strided<[2], offset: 1>>) {
^bb0(%a: f32, %b: vector<4xf32>): ^bb0(%a: f32, %b: vector<4xf32>):
%c = "some_compute"(%a, %b): (f32, vector<4xf32>) -> (vector<4xf32>) %c = "some_compute"(%a, %b): (f32, vector<4xf32>) -> (vector<4xf32>)
linalg.yield %c: vector<4xf32> linalg.yield %c: vector<4xf32>
} }
return return
} }
``` ```
The property "*Input and Output Operands Define The Iteration Space*" is The property "*Input and Output Operands Define The Iteration Space*" is
materialized by a lowering into a form that will resemble: materialized by a lowering into a form that will resemble:
```mlir ```mlir
// Run: mlir-opt example1.mlir -allow-unregistered-dialect -convert-linalg-to-loops // Run: mlir-opt example1.mlir -allow-unregistered-dialect -convert-linalg-to-loops
// This converted representation is in the `scf` dialect. // This converted representation is in the `scf` dialect.
// It's syntax can be found here: https://mlir.llvm.org/docs/Dialects/SCFDialect/ // It's syntax can be found here: https://mlir.llvm.org/docs/Dialects/SCFDialect/
#map0 = affine_map<(d0) -> (d0 * 2 + 1)>
func.func @example(%arg0: memref<?xf32>, %arg1: memref<?xvector<4xf32>, #map0>) { func.func @example(%arg0: memref<?xf32>,
%arg1: memref<?xvector<4xf32>, strided<[2], offset: 1>>) {
%c0 = arith.constant 0 : index %c0 = arith.constant 0 : index
%c1 = arith.constant 1 : index %c1 = arith.constant 1 : index
%0 = memref.dim %arg0, %c0 : memref<?xf32> %0 = memref.dim %arg0, %c0 : memref<?xf32>
scf.for %arg2 = %c0 to %0 step %c1 { scf.for %arg2 = %c0 to %0 step %c1 {
%1 = memref.load %arg0[%arg2] : memref<?xf32> %1 = memref.load %arg0[%arg2] : memref<?xf32>
%2 = memref.load %arg1[%arg2] : memref<?xvector<4xf32>, #map0> %2 = memref.load %arg1[%arg2]
: memref<?xvector<4xf32>, strided<[2], offset: 1>>
%3 = "some_compute"(%1, %2) : (f32, vector<4xf32>) -> vector<4xf32> %3 = "some_compute"(%1, %2) : (f32, vector<4xf32>) -> vector<4xf32>
memref.store %3, %arg1[%arg2] : memref<?xvector<4xf32>, #map0> memref.store %3, %arg1[%arg2]
: memref<?xvector<4xf32>, strided<[2], offset: 1>>
} }
return return
} }
``` ```
The property participates in simplifying analyses and transformations. For The property participates in simplifying analyses and transformations. For
instance, it guarantees no out-of bounds access can occur by construction instance, it guarantees no out-of bounds access can occur by construction
(assuming dynamic operand dimensions agree with each other, which is the purpose (assuming dynamic operand dimensions agree with each other, which is the purpose
▲ Show 20 Lines • Show All 55 Lines • ▼ Show 20 Lines
} }
``` ```
The property "*Reversible Mappings Between Control and Data Structures*" is The property "*Reversible Mappings Between Control and Data Structures*" is
materialized by a lowering into a form that will resemble: materialized by a lowering into a form that will resemble:
```mlir ```mlir
// Run: mlir-opt example2.mlir -allow-unregistered-dialect -convert-linalg-to-loops // Run: mlir-opt example2.mlir -allow-unregistered-dialect -convert-linalg-to-loops
#map0 = affine_map<(d0, d1) -> (d0 * 2 + d1 * 2)>
func.func @example(%arg0: memref<8x?xf32, #map0>, %arg1: memref<?xvector<4xf32>>) { func.func @example(%arg0: memref<8x?xf32, strided<[2, 2]>>, %arg1: memref<?xvector<4xf32>>) {
%c8 = arith.constant 8 : index %c8 = arith.constant 8 : index
%c0 = arith.constant 0 : index %c0 = arith.constant 0 : index
%c1 = arith.constant 1 : index %c1 = arith.constant 1 : index
%0 = memref.dim %arg0, %c1 : memref<8x?xf32, #map0> %0 = memref.dim %arg0, %c1 : memref<8x?xf32, strided<[2, 2]>>
scf.for %arg2 = %c0 to %0 step %c1 { scf.for %arg2 = %c0 to %0 step %c1 {
scf.for %arg3 = %c0 to %c8 step %c1 { scf.for %arg3 = %c0 to %c8 step %c1 {
%1 = memref.load %arg0[%arg3, %arg2] : memref<8x?xf32, #map0> %1 = memref.load %arg0[%arg3, %arg2] : memref<8x?xf32, strided<[2, 2]>>
%2 = memref.load %arg1[%arg3] : memref<?xvector<4xf32>> %2 = memref.load %arg1[%arg3] : memref<?xvector<4xf32>>
%3 = "some_compute"(%1, %2) : (f32, vector<4xf32>) -> vector<4xf32> %3 = "some_compute"(%1, %2) : (f32, vector<4xf32>) -> vector<4xf32>
memref.store %3, %arg1[%arg3] : memref<?xvector<4xf32>> memref.store %3, %arg1[%arg3] : memref<?xvector<4xf32>>
} }
} }
return return
} }
``` ```
▲ Show 20 Lines • Show All 170 Lines • ▼ Show 20 Lines
``` ```
The property "*Map To an External Library Call*" is materialized by a lowering The property "*Map To an External Library Call*" is materialized by a lowering
into a form that will resemble: into a form that will resemble:
```mlir ```mlir
// Run: mlir-opt example4.mlir -convert-linalg-to-std // Run: mlir-opt example4.mlir -convert-linalg-to-std
#map0 = affine_map<(d0, d1)[s0, s1, s2] -> (d0 * s1 + s0 + d1 * s2)>
func.func @example(%arg0: memref<?x?xf32>, %arg1: memref<?x?xf32>, %arg2: memref<?x?xf32>) { func.func @example(%arg0: memref<?x?xf32>, %arg1: memref<?x?xf32>, %arg2: memref<?x?xf32>) {
%0 = memref.cast %arg0 : memref<?x?xf32> to memref<?x?xf32, #map0> %0 = memref.cast %arg0 : memref<?x?xf32> to memref<?x?xf32, strided<[?, ?], offset: ?>>
%1 = memref.cast %arg1 : memref<?x?xf32> to memref<?x?xf32, #map0> %1 = memref.cast %arg1 : memref<?x?xf32> to memref<?x?xf32, strided<[?, ?], offset: ?>>
%2 = memref.cast %arg2 : memref<?x?xf32> to memref<?x?xf32, #map0> %2 = memref.cast %arg2 : memref<?x?xf32> to memref<?x?xf32, strided<[?, ?], offset: ?>>
call @pointwise_add(%0, %1, %2) : (memref<?x?xf32, #map0>, memref<?x?xf32, #map0>, memref<?x?xf32, #map0>) -> () call @pointwise_add(%0, %1, %2) : (memref<?x?xf32, strided<[?, ?], offset: ?>>,
memref<?x?xf32, strided<[?, ?], offset: ?>>, memref<?x?xf32, strided<[?, ?], offset: ?>>) -> ()
return return
} }
func.func @pointwise_add(memref<?x?xf32, #map0>, memref<?x?xf32, #map0>, memref<?x?xf32, #map0>) attributes {llvm.emit_c_interface} func.func @pointwise_add(memref<?x?xf32, strided<[?, ?], offset: ?>>,
memref<?x?xf32, strided<[?, ?], offset: ?>>,
memref<?x?xf32, strided<[?, ?], offset: ?>>) attributes {llvm.emit_c_interface}
``` ```
Which, after lowering to LLVM resembles: Which, after lowering to LLVM resembles:
```mlir ```mlir
// Run: mlir-opt example4.mlir -convert-linalg-to-std | mlir-opt -convert-func-to-llvm // Run: mlir-opt example4.mlir -convert-linalg-to-std | mlir-opt -convert-func-to-llvm
// Some generated code are omitted here. // Some generated code are omitted here.
func.func @example(%arg0: !llvm<"float*">, ...) { func.func @example(%arg0: !llvm<"float*">, ...) {
▲ Show 20 Lines • Show All 278 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Bufferization/IR/BufferizationOps.td

Show First 20 Lines • Show All 349 Lines • ▼ Show 20 Lines TypesMatchWith<"type of 'tensor' is the tensor equivalent of 'memref'",
"memref", "tensor", "memref", "tensor",
"memref::getTensorTypeFromMemRefType($_self)"> "memref::getTensorTypeFromMemRefType($_self)">
]> { ]> {
let summary = "tensor to memref cast operation"; let summary = "tensor to memref cast operation";
let description = [{ let description = [{
Casts a tensor to a memref. Casts a tensor to a memref.
```mlir ```mlir
// Result type is memref<4x?xf32, #map0, 42> // Result type is memref<4x?xf32, #layout, 42>
%12 = bufferization.to_memref %10 : memref<4x?xf32, #map0, 42> %12 = bufferization.to_memref %10 : memref<4x?xf32, #layout, 42>
``` ```
Note, that mutating the result of the `to_memref` operation leads to Note, that mutating the result of the `to_memref` operation leads to
undefined behavior. undefined behavior.
This operation is a specialized variant of the built-in This operation is a specialized variant of the built-in
`unrealized_conversion_cast` and is intended for use in the context of `unrealized_conversion_cast` and is intended for use in the context of
gradual bufferization. gradual bufferization.
▲ Show 20 Lines • Show All 50 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/MemRef/Transforms/Passes.h

Show First 20 Lines • Show All 74 Lines • ▼ Show 20 Lines
/// into: /// into:
/// ``` /// ```
/// %0 = memref.alloc() : memref<5x4x128xf32> /// %0 = memref.alloc() : memref<5x4x128xf32>
/// scf.for %iv = %c1 to %c1024 step %c3 { /// scf.for %iv = %c1 to %c1024 step %c3 {
/// %s = arith.subi %iv, %c1 : index /// %s = arith.subi %iv, %c1 : index
/// %d = arith.divsi %s, %c3 : index /// %d = arith.divsi %s, %c3 : index
/// %i = arith.remsi %d, %c5 : index /// %i = arith.remsi %d, %c5 : index
/// %sv = memref.subview %0[%i, 0, 0] [1, 4, 128] [1, 1, 1] : /// %sv = memref.subview %0[%i, 0, 0] [1, 4, 128] [1, 1, 1] :
/// memref<5x4x128xf32> to memref<4x128xf32, #map0> /// memref<5x4x128xf32> to memref<4x128xf32, strided<[128, 1], offset: ?>>
/// memref.copy %1, %sv : memref<4x128xf32> to memref<4x128xf32, #map0> /// memref.copy %1, %sv : memref<4x128xf32> to memref<4x128xf32, strided<...>>
/// "some_use"(%sv) : (memref<4x128xf32, $map0>) -> () /// "some_use"(%sv) : (memref<4x128xf32, strided<...>) -> ()
/// } /// }
/// ``` /// ```
LogicalResult multiBuffer(memref::AllocOp allocOp, unsigned multiplier); LogicalResult multiBuffer(memref::AllocOp allocOp, unsigned multiplier);
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Passes // Passes
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 48 Lines • Show Last 20 Lines

mlir/test/Conversion/BufferizationToMemRef/bufferization-to-memref.mlir

Show All 33 Lines
// expected-error@+1 {{failed to legalize operation 'bufferization.clone' that was explicitly marked illegal}} // expected-error@+1 {{failed to legalize operation 'bufferization.clone' that was explicitly marked illegal}}
%1 = bufferization.clone %arg0 : memref<*xf32> to memref<*xf32> %1 = bufferization.clone %arg0 : memref<*xf32> to memref<*xf32>
memref.dealloc %arg0 : memref<*xf32> memref.dealloc %arg0 : memref<*xf32>
return %1 : memref<*xf32> return %1 : memref<*xf32>
} }
// ----- // -----
// CHECK: #[[$MAP:.*]] = affine_map<(d0)[s0, s1] -> (d0 * s1 + s0)>
#map = affine_map<(d0)[s0, s1] -> (d0 * s1 + s0)>
// CHECK-LABEL: func @conversion_with_layout_map( // CHECK-LABEL: func @conversion_with_layout_map(
// CHECK-SAME: %[[ARG:.*]]: memref<?xf32, #[[$MAP]]> // CHECK-SAME: %[[ARG:.*]]: memref<?xf32, strided<[?], offset: ?>>
// CHECK: %[[C0:.*]] = arith.constant 0 : index // CHECK: %[[C0:.*]] = arith.constant 0 : index
// CHECK: %[[DIM:.*]] = memref.dim %[[ARG]], %[[C0]] // CHECK: %[[DIM:.*]] = memref.dim %[[ARG]], %[[C0]]
// CHECK: %[[ALLOC:.*]] = memref.alloc(%[[DIM]]) : memref<?xf32> // CHECK: %[[ALLOC:.*]] = memref.alloc(%[[DIM]]) : memref<?xf32>
// CHECK: %[[CASTED:.*]] = memref.cast %[[ALLOC]] : memref<?xf32> to memref<?xf32, #[[$MAP]]> // CHECK: %[[CASTED:.*]] = memref.cast %[[ALLOC]] : memref<?xf32> to memref<?xf32, strided<[?], offset: ?>>
// CHECK: memref.copy // CHECK: memref.copy
// CHECK: memref.dealloc // CHECK: memref.dealloc
// CHECK: return %[[CASTED]] // CHECK: return %[[CASTED]]
func.func @conversion_with_layout_map(%arg0 : memref<?xf32, #map>) -> memref<?xf32, #map> { func.func @conversion_with_layout_map(%arg0 : memref<?xf32, strided<[?], offset: ?>>) -> memref<?xf32, strided<[?], offset: ?>> {
%1 = bufferization.clone %arg0 : memref<?xf32, #map> to memref<?xf32, #map> %1 = bufferization.clone %arg0 : memref<?xf32, strided<[?], offset: ?>> to memref<?xf32, strided<[?], offset: ?>>
memref.dealloc %arg0 : memref<?xf32, #map> memref.dealloc %arg0 : memref<?xf32, strided<[?], offset: ?>>
return %1 : memref<?xf32, #map> return %1 : memref<?xf32, strided<[?], offset: ?>>
} }
// ----- // -----
// This bufferization.clone cannot be lowered because a buffer with this layout // This bufferization.clone cannot be lowered because a buffer with this layout
// map cannot be allocated (or casted to). // map cannot be allocated (or casted to).
#map2 = affine_map<(d0)[s0] -> (d0 * 10 + s0)> func.func @conversion_with_invalid_layout_map(%arg0 : memref<?xf32, strided<[10], offset: ?>>)
func.func @conversion_with_invalid_layout_map(%arg0 : memref<?xf32, #map2>) -> memref<?xf32, strided<[10], offset: ?>> {
-> memref<?xf32, #map2> {
// expected-error@+1 {{failed to legalize operation 'bufferization.clone' that was explicitly marked illegal}} // expected-error@+1 {{failed to legalize operation 'bufferization.clone' that was explicitly marked illegal}}
%1 = bufferization.clone %arg0 : memref<?xf32, #map2> to memref<?xf32, #map2> %1 = bufferization.clone %arg0 : memref<?xf32, strided<[10], offset: ?>> to memref<?xf32, strided<[10], offset: ?>>
memref.dealloc %arg0 : memref<?xf32, #map2> memref.dealloc %arg0 : memref<?xf32, strided<[10], offset: ?>>
return %1 : memref<?xf32, #map2> return %1 : memref<?xf32, strided<[10], offset: ?>>
} }

mlir/test/Conversion/MemRefToLLVM/memref-to-llvm.mlir

Show First 20 Lines • Show All 1,036 Lines • ▼ Show 20 Linesfunc.func @memref_copy_ranked() {
// CHECK: "llvm.intr.memcpy"([[GEP2]], [[GEP1]], [[SIZE]], [[VOLATILE]]) : (!llvm.ptr<f32>, !llvm.ptr<f32>, i64, i1) -> () // CHECK: "llvm.intr.memcpy"([[GEP2]], [[GEP1]], [[SIZE]], [[VOLATILE]]) : (!llvm.ptr<f32>, !llvm.ptr<f32>, i64, i1) -> ()
return return
} }
// ----- // -----
// CHECK-LABEL: func @memref_copy_contiguous // CHECK-LABEL: func @memref_copy_contiguous
#map = affine_map<(d0, d1)[s0] -> (d0 * 2 + s0 + d1)>
func.func @memref_copy_contiguous(%in: memref<16x2xi32>, %offset: index) { func.func @memref_copy_contiguous(%in: memref<16x2xi32>, %offset: index) {
%buf = memref.alloc() : memref<1x2xi32> %buf = memref.alloc() : memref<1x2xi32>
%sub = memref.subview %in[%offset, 0] [1, 2] [1, 1] : memref<16x2xi32> to memref<1x2xi32, #map> %sub = memref.subview %in[%offset, 0] [1, 2] [1, 1] : memref<16x2xi32> to memref<1x2xi32, strided<[2, 1], offset: ?>>
memref.copy %sub, %buf : memref<1x2xi32, #map> to memref<1x2xi32> memref.copy %sub, %buf : memref<1x2xi32, strided<[2, 1], offset: ?>> to memref<1x2xi32>
// CHECK: [[EXTRACT0:%.*]] = llvm.extractvalue {{%.*}}[3, 0] : !llvm.struct<(ptr<i32>, ptr<i32>, i64, array<2 x i64>, array<2 x i64>)> // CHECK: [[EXTRACT0:%.*]] = llvm.extractvalue {{%.*}}[3, 0] : !llvm.struct<(ptr<i32>, ptr<i32>, i64, array<2 x i64>, array<2 x i64>)>
// CHECK: [[MUL1:%.*]] = llvm.mul {{.*}}, [[EXTRACT0]] : i64 // CHECK: [[MUL1:%.*]] = llvm.mul {{.*}}, [[EXTRACT0]] : i64
// CHECK: [[EXTRACT1:%.*]] = llvm.extractvalue {{%.*}}[3, 1] : !llvm.struct<(ptr<i32>, ptr<i32>, i64, array<2 x i64>, array<2 x i64>)> // CHECK: [[EXTRACT1:%.*]] = llvm.extractvalue {{%.*}}[3, 1] : !llvm.struct<(ptr<i32>, ptr<i32>, i64, array<2 x i64>, array<2 x i64>)>
// CHECK: [[MUL2:%.*]] = llvm.mul [[MUL1]], [[EXTRACT1]] : i64 // CHECK: [[MUL2:%.*]] = llvm.mul [[MUL1]], [[EXTRACT1]] : i64
// CHECK: [[NULL:%.*]] = llvm.mlir.null : !llvm.ptr<i32> // CHECK: [[NULL:%.*]] = llvm.mlir.null : !llvm.ptr<i32>
// CHECK: [[GEP:%.*]] = llvm.getelementptr [[NULL]][1] : (!llvm.ptr<i32>) -> !llvm.ptr<i32> // CHECK: [[GEP:%.*]] = llvm.getelementptr [[NULL]][1] : (!llvm.ptr<i32>) -> !llvm.ptr<i32>
// CHECK: [[PTRTOINT:%.*]] = llvm.ptrtoint [[GEP]] : !llvm.ptr<i32> to i64 // CHECK: [[PTRTOINT:%.*]] = llvm.ptrtoint [[GEP]] : !llvm.ptr<i32> to i64
// CHECK: [[SIZE:%.*]] = llvm.mul [[MUL2]], [[PTRTOINT]] : i64 // CHECK: [[SIZE:%.*]] = llvm.mul [[MUL2]], [[PTRTOINT]] : i64
Show All 18 Linesfunc.func @memref_copy_0d_offset(%in: memref<2xi32>) {
memref.copy %scalar, %buf : memref<i32, strided<[], offset: 1>> to memref<i32> memref.copy %scalar, %buf : memref<i32, strided<[], offset: 1>> to memref<i32>
// CHECK: llvm.intr.memcpy // CHECK: llvm.intr.memcpy
return return
} }
// ----- // -----
// CHECK-LABEL: func @memref_copy_noncontiguous // CHECK-LABEL: func @memref_copy_noncontiguous
#map = affine_map<(d0, d1)[s0] -> (d0 * 2 + s0 + d1)>
func.func @memref_copy_noncontiguous(%in: memref<16x2xi32>, %offset: index) { func.func @memref_copy_noncontiguous(%in: memref<16x2xi32>, %offset: index) {
%buf = memref.alloc() : memref<2x1xi32> %buf = memref.alloc() : memref<2x1xi32>
%sub = memref.subview %in[%offset, 0] [2, 1] [1, 1] : memref<16x2xi32> to memref<2x1xi32, #map> %sub = memref.subview %in[%offset, 0] [2, 1] [1, 1] : memref<16x2xi32> to memref<2x1xi32, strided<[2, 1], offset: ?>>
memref.copy %sub, %buf : memref<2x1xi32, #map> to memref<2x1xi32> memref.copy %sub, %buf : memref<2x1xi32, strided<[2, 1], offset: ?>> to memref<2x1xi32>
// CHECK: llvm.call @memrefCopy // CHECK: llvm.call @memrefCopy
return return
} }
// ----- // -----
// CHECK-LABEL: func @memref_copy_unranked // CHECK-LABEL: func @memref_copy_unranked
func.func @memref_copy_unranked() { func.func @memref_copy_unranked() {
Show All 26 Lines

mlir/test/Conversion/SCFToGPU/parallel_loop.mlir

Show First 20 Lines • Show All 191 Lines • ▼ Show 20 Lines
// CHECK: gpu.terminator // CHECK: gpu.terminator
// CHECK: } // CHECK: }
// CHECK: return // CHECK: return
// CHECK: } // CHECK: }
// CHECK: } // CHECK: }
// ----- // -----
#map0 = affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>
#map1 = affine_map<(d0)[s0] -> (2, -d0 + s0)> #map1 = affine_map<(d0)[s0] -> (2, -d0 + s0)>
#map2 = affine_map<(d0)[s0] -> (3, -d0 + s0)> #map2 = affine_map<(d0)[s0] -> (3, -d0 + s0)>
#map3 = affine_map<(d0, d1)[s0, s1, s2] -> (d0 * s1 + s0 + d1 * s2)>
module { module {
func.func @sum(%arg0: memref<?x?xf32, #map0>, %arg1: memref<?x?xf32, #map0>, %arg2: memref<?x?xf32, #map0>) { func.func @sum(%arg0: memref<?x?xf32, strided<[?, 1], offset: ?>>, %arg1: memref<?x?xf32, strided<[?, 1], offset: ?>>, %arg2: memref<?x?xf32, strided<[?, 1], offset: ?>>) {
%c1 = arith.constant 1 : index %c1 = arith.constant 1 : index
%c0 = arith.constant 0 : index %c0 = arith.constant 0 : index
%c3 = arith.constant 3 : index %c3 = arith.constant 3 : index
%c2 = arith.constant 2 : index %c2 = arith.constant 2 : index
%0 = memref.dim %arg0, %c0 : memref<?x?xf32, #map0> %0 = memref.dim %arg0, %c0 : memref<?x?xf32, strided<[?, 1], offset: ?>>
%1 = memref.dim %arg0, %c1 : memref<?x?xf32, #map0> %1 = memref.dim %arg0, %c1 : memref<?x?xf32, strided<[?, 1], offset: ?>>
scf.parallel (%arg3, %arg4) = (%c0, %c0) to (%0, %1) step (%c2, %c3) { scf.parallel (%arg3, %arg4) = (%c0, %c0) to (%0, %1) step (%c2, %c3) {
%2 = memref.dim %arg0, %c0 : memref<?x?xf32, #map0> %2 = memref.dim %arg0, %c0 : memref<?x?xf32, strided<[?, 1], offset: ?>>
%3 = affine.min #map1(%arg3)[%2] %3 = affine.min #map1(%arg3)[%2]
%squared_min = arith.muli %3, %3 : index %squared_min = arith.muli %3, %3 : index
%4 = memref.dim %arg0, %c1 : memref<?x?xf32, #map0> %4 = memref.dim %arg0, %c1 : memref<?x?xf32, strided<[?, 1], offset: ?>>
%d = arith.subi %4, %arg4 : index %d = arith.subi %4, %arg4 : index
%5 = arith.minsi %c3, %d : index %5 = arith.minsi %c3, %d : index
%6 = memref.subview %arg0[%arg3, %arg4][%squared_min, %5][%c1, %c1] : memref<?x?xf32, #map0> to memref<?x?xf32, #map3> %6 = memref.subview %arg0[%arg3, %arg4][%squared_min, %5][%c1, %c1] : memref<?x?xf32, strided<[?, 1], offset: ?>> to memref<?x?xf32, strided<[?, ?], offset: ?>>
%7 = memref.dim %arg1, %c0 : memref<?x?xf32, #map0> %7 = memref.dim %arg1, %c0 : memref<?x?xf32, strided<[?, 1], offset: ?>>
%8 = affine.min #map1(%arg3)[%7] %8 = affine.min #map1(%arg3)[%7]
%9 = memref.dim %arg1, %c1 : memref<?x?xf32, #map0> %9 = memref.dim %arg1, %c1 : memref<?x?xf32, strided<[?, 1], offset: ?>>
%10 = affine.min #map2(%arg4)[%9] %10 = affine.min #map2(%arg4)[%9]
%11 = memref.subview %arg1[%arg3, %arg4][%8, %10][%c1, %c1] : memref<?x?xf32, #map0> to memref<?x?xf32, #map3> %11 = memref.subview %arg1[%arg3, %arg4][%8, %10][%c1, %c1] : memref<?x?xf32, strided<[?, 1], offset: ?>> to memref<?x?xf32, strided<[?, ?], offset: ?>>
%12 = memref.dim %arg2, %c0 : memref<?x?xf32, #map0> %12 = memref.dim %arg2, %c0 : memref<?x?xf32, strided<[?, 1], offset: ?>>
%13 = affine.min #map1(%arg3)[%12] %13 = affine.min #map1(%arg3)[%12]
%14 = memref.dim %arg2, %c1 : memref<?x?xf32, #map0> %14 = memref.dim %arg2, %c1 : memref<?x?xf32, strided<[?, 1], offset: ?>>
%15 = affine.min #map2(%arg4)[%14] %15 = affine.min #map2(%arg4)[%14]
%16 = memref.subview %arg2[%arg3, %arg4][%13, %15][%c1, %c1] : memref<?x?xf32, #map0> to memref<?x?xf32, #map3> %16 = memref.subview %arg2[%arg3, %arg4][%13, %15][%c1, %c1] : memref<?x?xf32, strided<[?, 1], offset: ?>> to memref<?x?xf32, strided<[?, ?], offset: ?>>
scf.parallel (%arg5, %arg6) = (%c0, %c0) to (%squared_min, %5) step (%c1, %c1) { scf.parallel (%arg5, %arg6) = (%c0, %c0) to (%squared_min, %5) step (%c1, %c1) {
%17 = memref.load %6[%arg5, %arg6] : memref<?x?xf32, #map3> %17 = memref.load %6[%arg5, %arg6] : memref<?x?xf32, strided<[?, ?], offset: ?>>
%18 = memref.load %11[%arg5, %arg6] : memref<?x?xf32, #map3> %18 = memref.load %11[%arg5, %arg6] : memref<?x?xf32, strided<[?, ?], offset: ?>>
%19 = memref.load %16[%arg5, %arg6] : memref<?x?xf32, #map3> %19 = memref.load %16[%arg5, %arg6] : memref<?x?xf32, strided<[?, ?], offset: ?>>
%20 = arith.addf %17, %18 : f32 %20 = arith.addf %17, %18 : f32
memref.store %20, %16[%arg5, %arg6] : memref<?x?xf32, #map3> memref.store %20, %16[%arg5, %arg6] : memref<?x?xf32, strided<[?, ?], offset: ?>>
scf.yield scf.yield
} {mapping = [#gpu.loop_dim_map<bound = (d0) -> (d0), map = (d0) -> (d0), processor = thread_x>, #gpu.loop_dim_map<bound = (d0) -> (d0), map = (d0) -> (d0), processor = thread_y>]} } {mapping = [#gpu.loop_dim_map<bound = (d0) -> (d0), map = (d0) -> (d0), processor = thread_x>, #gpu.loop_dim_map<bound = (d0) -> (d0), map = (d0) -> (d0), processor = thread_y>]}
scf.yield scf.yield
} {mapping = [#gpu.loop_dim_map<bound = (d0) -> (d0), map = (d0) -> (d0), processor = block_x>, #gpu.loop_dim_map<bound = (d0) -> (d0), map = (d0) -> (d0), processor = block_y>]} } {mapping = [#gpu.loop_dim_map<bound = (d0) -> (d0), map = (d0) -> (d0), processor = block_x>, #gpu.loop_dim_map<bound = (d0) -> (d0), map = (d0) -> (d0), processor = block_y>]}
return return
} }
} }
// CHECK-DAG: #[[$MAP0:.*]] = affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>
// CHECK-DAG: #[[$MAP1:.*]] = affine_map<(d0)[s0, s1] -> ((d0 - s0) ceildiv s1)> // CHECK-DAG: #[[$MAP1:.*]] = affine_map<(d0)[s0, s1] -> ((d0 - s0) ceildiv s1)>
// CHECK-DAG: #[[$MAP2:.*]] = affine_map<(d0)[s0, s1] -> (d0 * s0 + s1)> // CHECK-DAG: #[[$MAP2:.*]] = affine_map<(d0)[s0, s1] -> (d0 * s0 + s1)>
// CHECK-DAG: #[[$MAP3:.*]] = affine_map<(d0)[s0] -> (2, -d0 + s0)> // CHECK-DAG: #[[$MAP3:.*]] = affine_map<(d0)[s0] -> (2, -d0 + s0)>
// CHECK-DAG: #[[$MAP4:.*]] = affine_map<(d0)[s0] -> (3, -d0 + s0)> // CHECK-DAG: #[[$MAP4:.*]] = affine_map<(d0)[s0] -> (3, -d0 + s0)>
// CHECK-DAG: #[[$MAP5:.*]] = affine_map<(d0, d1)[s0, s1, s2] -> (d0 * s1 + s0 + d1 * s2)>
// CHECK: module { // CHECK: module {
// CHECK-LABEL: func @sum( // CHECK-LABEL: func @sum(
// CHECK-SAME: [[VAL_0:%.*]]: memref<?x?xf32, #[[$MAP0]]>, [[VAL_1:%.*]]: memref<?x?xf32, #[[$MAP0]]>, [[VAL_2:%.*]]: memref<?x?xf32, #[[$MAP0]]>) { // CHECK-SAME: [[VAL_0:%.*]]: memref<?x?xf32, strided<[?, 1], offset: ?>>, [[VAL_1:%.*]]: memref<?x?xf32, strided<[?, 1], offset: ?>>, [[VAL_2:%.*]]: memref<?x?xf32, strided<[?, 1], offset: ?>>) {
// CHECK: %[[C1:.*]] = arith.constant 1 : index // CHECK: %[[C1:.*]] = arith.constant 1 : index
// CHECK: %[[C0:.*]] = arith.constant 0 : index // CHECK: %[[C0:.*]] = arith.constant 0 : index
// CHECK: %[[C3:.*]] = arith.constant 3 : index // CHECK: %[[C3:.*]] = arith.constant 3 : index
// CHECK: %[[C2:.*]] = arith.constant 2 : index // CHECK: %[[C2:.*]] = arith.constant 2 : index
// CHECK: [[VAL_7:%.*]] = memref.dim [[VAL_0]], %[[C0]] : memref<?x?xf32, #[[$MAP0]]> // CHECK: [[VAL_7:%.*]] = memref.dim [[VAL_0]], %[[C0]] : memref<?x?xf32, strided<[?, 1], offset: ?>>
// CHECK: [[VAL_8:%.*]] = memref.dim [[VAL_0]], %[[C1]] : memref<?x?xf32, #[[$MAP0]]> // CHECK: [[VAL_8:%.*]] = memref.dim [[VAL_0]], %[[C1]] : memref<?x?xf32, strided<[?, 1], offset: ?>>
// CHECK: [[VAL_9:%.*]] = arith.constant 1 : index // CHECK: [[VAL_9:%.*]] = arith.constant 1 : index
// CHECK: [[VAL_10:%.*]] = affine.apply #[[$MAP1]]([[VAL_7]]){{\[}}%[[C0]], %[[C2]]] // CHECK: [[VAL_10:%.*]] = affine.apply #[[$MAP1]]([[VAL_7]]){{\[}}%[[C0]], %[[C2]]]
// CHECK: [[VAL_11:%.*]] = affine.apply #[[$MAP1]]([[VAL_8]]){{\[}}%[[C0]], %[[C3]]] // CHECK: [[VAL_11:%.*]] = affine.apply #[[$MAP1]]([[VAL_8]]){{\[}}%[[C0]], %[[C3]]]
// CHECK: [[VAL_12:%.*]] = arith.constant 4 : index // CHECK: [[VAL_12:%.*]] = arith.constant 4 : index
// CHECK: [[VAL_13:%.*]] = affine.apply #[[$MAP1]]([[VAL_12]]){{\[}}%[[C0]], %[[C1]]] // CHECK: [[VAL_13:%.*]] = affine.apply #[[$MAP1]]([[VAL_12]]){{\[}}%[[C0]], %[[C1]]]
// CHECK: [[VAL_15:%.*]] = affine.apply #[[$MAP1]](%[[C3]]){{\[}}%[[C0]], %[[C1]]] // CHECK: [[VAL_15:%.*]] = affine.apply #[[$MAP1]](%[[C3]]){{\[}}%[[C0]], %[[C1]]]
// CHECK: gpu.launch blocks([[VAL_16:%.*]], [[VAL_17:%.*]], [[VAL_18:%.*]]) in ([[VAL_19:%.*]] = [[VAL_10]], [[VAL_20:%.*]] = [[VAL_11]], [[VAL_21:%.*]] = [[VAL_9]]) threads([[VAL_22:%.*]], [[VAL_23:%.*]], [[VAL_24:%.*]]) in ([[VAL_25:%.*]] = [[VAL_13]], [[VAL_26:%.*]] = [[VAL_15]], [[VAL_27:%.*]] = [[VAL_9]]) { // CHECK: gpu.launch blocks([[VAL_16:%.*]], [[VAL_17:%.*]], [[VAL_18:%.*]]) in ([[VAL_19:%.*]] = [[VAL_10]], [[VAL_20:%.*]] = [[VAL_11]], [[VAL_21:%.*]] = [[VAL_9]]) threads([[VAL_22:%.*]], [[VAL_23:%.*]], [[VAL_24:%.*]]) in ([[VAL_25:%.*]] = [[VAL_13]], [[VAL_26:%.*]] = [[VAL_15]], [[VAL_27:%.*]] = [[VAL_9]]) {
// CHECK: [[VAL_28:%.*]] = affine.apply #[[$MAP2]]([[VAL_16]]){{\[}}%[[C2]], %[[C0]]] // CHECK: [[VAL_28:%.*]] = affine.apply #[[$MAP2]]([[VAL_16]]){{\[}}%[[C2]], %[[C0]]]
// CHECK: [[VAL_29:%.*]] = affine.apply #[[$MAP2]]([[VAL_17]]){{\[}}%[[C3]], %[[C0]]] // CHECK: [[VAL_29:%.*]] = affine.apply #[[$MAP2]]([[VAL_17]]){{\[}}%[[C3]], %[[C0]]]
// CHECK: [[VAL_30:%.*]] = memref.dim [[VAL_0]], %[[C0]] : memref<?x?xf32, #[[$MAP0]]> // CHECK: [[VAL_30:%.*]] = memref.dim [[VAL_0]], %[[C0]] : memref<?x?xf32, strided<[?, 1], offset: ?>>
// CHECK: [[VAL_31:%.*]] = affine.min #[[$MAP3]]([[VAL_28]]){{\[}}[[VAL_30]]] // CHECK: [[VAL_31:%.*]] = affine.min #[[$MAP3]]([[VAL_28]]){{\[}}[[VAL_30]]]
// CHECK: [[VAL_31_SQUARED:%.*]] = arith.muli [[VAL_31]], [[VAL_31]] : index // CHECK: [[VAL_31_SQUARED:%.*]] = arith.muli [[VAL_31]], [[VAL_31]] : index
// CHECK: [[VAL_32:%.*]] = memref.dim [[VAL_0]], %[[C1]] : memref<?x?xf32, #[[$MAP0]]> // CHECK: [[VAL_32:%.*]] = memref.dim [[VAL_0]], %[[C1]] : memref<?x?xf32, strided<[?, 1], offset: ?>>
// CHECK: [[VAL_D:%.*]] = arith.subi [[VAL_32]], [[VAL_29]] : index // CHECK: [[VAL_D:%.*]] = arith.subi [[VAL_32]], [[VAL_29]] : index
// CHECK: [[VAL_33:%.*]] = arith.minsi %[[C3]], [[VAL_D]] : index // CHECK: [[VAL_33:%.*]] = arith.minsi %[[C3]], [[VAL_D]] : index
// CHECK: [[VAL_34:%.*]] = memref.subview [[VAL_0]]{{\[}}[[VAL_28]], [[VAL_29]]] {{\[}}[[VAL_31_SQUARED]], [[VAL_33]]] {{\[}}%[[C1]], %[[C1]]] : memref<?x?xf32, #[[$MAP0]]> to memref<?x?xf32, #[[$MAP5]]> // CHECK: [[VAL_34:%.*]] = memref.subview [[VAL_0]]{{\[}}[[VAL_28]], [[VAL_29]]] {{\[}}[[VAL_31_SQUARED]], [[VAL_33]]] {{\[}}%[[C1]], %[[C1]]] : memref<?x?xf32, strided<[?, 1], offset: ?>> to memref<?x?xf32, strided<[?, ?], offset: ?>>
// CHECK: [[VAL_35:%.*]] = memref.dim [[VAL_1]], %[[C0]] : memref<?x?xf32, #[[$MAP0]]> // CHECK: [[VAL_35:%.*]] = memref.dim [[VAL_1]], %[[C0]] : memref<?x?xf32, strided<[?, 1], offset: ?>>
// CHECK: [[VAL_36:%.*]] = affine.min #[[$MAP3]]([[VAL_28]]){{\[}}[[VAL_35]]] // CHECK: [[VAL_36:%.*]] = affine.min #[[$MAP3]]([[VAL_28]]){{\[}}[[VAL_35]]]
// CHECK: [[VAL_37:%.*]] = memref.dim [[VAL_1]], %[[C1]] : memref<?x?xf32, #[[$MAP0]]> // CHECK: [[VAL_37:%.*]] = memref.dim [[VAL_1]], %[[C1]] : memref<?x?xf32, strided<[?, 1], offset: ?>>
// CHECK: [[VAL_38:%.*]] = affine.min #[[$MAP4]]([[VAL_29]]){{\[}}[[VAL_37]]] // CHECK: [[VAL_38:%.*]] = affine.min #[[$MAP4]]([[VAL_29]]){{\[}}[[VAL_37]]]
// CHECK: [[VAL_39:%.*]] = memref.subview [[VAL_1]]{{\[}}[[VAL_28]], [[VAL_29]]] {{\[}}[[VAL_36]], [[VAL_38]]] {{\[}}%[[C1]], %[[C1]]] : memref<?x?xf32, #[[$MAP0]]> to memref<?x?xf32, #[[$MAP5]]> // CHECK: [[VAL_39:%.*]] = memref.subview [[VAL_1]]{{\[}}[[VAL_28]], [[VAL_29]]] {{\[}}[[VAL_36]], [[VAL_38]]] {{\[}}%[[C1]], %[[C1]]] : memref<?x?xf32, strided<[?, 1], offset: ?>> to memref<?x?xf32, strided<[?, ?], offset: ?>>
// CHECK: [[VAL_40:%.*]] = memref.dim [[VAL_2]], %[[C0]] : memref<?x?xf32, #[[$MAP0]]> // CHECK: [[VAL_40:%.*]] = memref.dim [[VAL_2]], %[[C0]] : memref<?x?xf32, strided<[?, 1], offset: ?>>
// CHECK: [[VAL_41:%.*]] = affine.min #[[$MAP3]]([[VAL_28]]){{\[}}[[VAL_40]]] // CHECK: [[VAL_41:%.*]] = affine.min #[[$MAP3]]([[VAL_28]]){{\[}}[[VAL_40]]]
// CHECK: [[VAL_42:%.*]] = memref.dim [[VAL_2]], %[[C1]] : memref<?x?xf32, #[[$MAP0]]> // CHECK: [[VAL_42:%.*]] = memref.dim [[VAL_2]], %[[C1]] : memref<?x?xf32, strided<[?, 1], offset: ?>>
// CHECK: [[VAL_43:%.*]] = affine.min #[[$MAP4]]([[VAL_29]]){{\[}}[[VAL_42]]] // CHECK: [[VAL_43:%.*]] = affine.min #[[$MAP4]]([[VAL_29]]){{\[}}[[VAL_42]]]
// CHECK: [[VAL_44:%.*]] = memref.subview [[VAL_2]]{{\[}}[[VAL_28]], [[VAL_29]]] {{\[}}[[VAL_41]], [[VAL_43]]] {{\[}}%[[C1]], %[[C1]]] : memref<?x?xf32, #[[$MAP0]]> to memref<?x?xf32, #[[$MAP5]]> // CHECK: [[VAL_44:%.*]] = memref.subview [[VAL_2]]{{\[}}[[VAL_28]], [[VAL_29]]] {{\[}}[[VAL_41]], [[VAL_43]]] {{\[}}%[[C1]], %[[C1]]] : memref<?x?xf32, strided<[?, 1], offset: ?>> to memref<?x?xf32, strided<[?, ?], offset: ?>>
// CHECK: [[VAL_45:%.*]] = affine.apply #[[$MAP2]]([[VAL_22]]){{\[}}%[[C1]], %[[C0]]] // CHECK: [[VAL_45:%.*]] = affine.apply #[[$MAP2]]([[VAL_22]]){{\[}}%[[C1]], %[[C0]]]
// CHECK: [[VAL_46:%.*]] = arith.cmpi slt, [[VAL_45]], [[VAL_31_SQUARED]] : index // CHECK: [[VAL_46:%.*]] = arith.cmpi slt, [[VAL_45]], [[VAL_31_SQUARED]] : index
// CHECK: scf.if [[VAL_46]] { // CHECK: scf.if [[VAL_46]] {
// CHECK: [[VAL_47:%.*]] = affine.apply #[[$MAP2]]([[VAL_23]]){{\[}}%[[C1]], %[[C0]]] // CHECK: [[VAL_47:%.*]] = affine.apply #[[$MAP2]]([[VAL_23]]){{\[}}%[[C1]], %[[C0]]]
// CHECK: [[VAL_48:%.*]] = arith.cmpi slt, [[VAL_47]], [[VAL_33]] : index // CHECK: [[VAL_48:%.*]] = arith.cmpi slt, [[VAL_47]], [[VAL_33]] : index
// CHECK: scf.if [[VAL_48]] { // CHECK: scf.if [[VAL_48]] {
// CHECK: [[VAL_49:%.*]] = memref.load [[VAL_34]]{{\[}}[[VAL_45]], [[VAL_47]]] : memref<?x?xf32, #[[$MAP5]]> // CHECK: [[VAL_49:%.*]] = memref.load [[VAL_34]]{{\[}}[[VAL_45]], [[VAL_47]]] : memref<?x?xf32, strided<[?, ?], offset: ?>>
// CHECK: [[VAL_50:%.*]] = memref.load [[VAL_39]]{{\[}}[[VAL_45]], [[VAL_47]]] : memref<?x?xf32, #[[$MAP5]]> // CHECK: [[VAL_50:%.*]] = memref.load [[VAL_39]]{{\[}}[[VAL_45]], [[VAL_47]]] : memref<?x?xf32, strided<[?, ?], offset: ?>>
// CHECK: [[VAL_51:%.*]] = memref.load [[VAL_44]]{{\[}}[[VAL_45]], [[VAL_47]]] : memref<?x?xf32, #[[$MAP5]]> // CHECK: [[VAL_51:%.*]] = memref.load [[VAL_44]]{{\[}}[[VAL_45]], [[VAL_47]]] : memref<?x?xf32, strided<[?, ?], offset: ?>>
// CHECK: [[VAL_52:%.*]] = arith.addf [[VAL_49]], [[VAL_50]] : f32 // CHECK: [[VAL_52:%.*]] = arith.addf [[VAL_49]], [[VAL_50]] : f32
// CHECK: memref.store [[VAL_52]], [[VAL_44]]{{\[}}[[VAL_45]], [[VAL_47]]] : memref<?x?xf32, #[[$MAP5]]> // CHECK: memref.store [[VAL_52]], [[VAL_44]]{{\[}}[[VAL_45]], [[VAL_47]]] : memref<?x?xf32, strided<[?, ?], offset: ?>>
// CHECK: } // CHECK: }
// CHECK: } // CHECK: }
// CHECK: gpu.terminator // CHECK: gpu.terminator
// CHECK: } // CHECK: }
// CHECK: return // CHECK: return
// CHECK: } // CHECK: }
// CHECK: } // CHECK: }
▲ Show 20 Lines • Show All 85 Lines • Show Last 20 Lines

mlir/test/Dialect/Bufferization/Transforms/finalizing-bufferize.mlir

Show All 23 Linesfunc.func @unable_to_convert_lone_tensor_load(%arg0: memref<f32>) {
%0 = bufferization.to_tensor %arg0 : memref<f32> %0 = bufferization.to_tensor %arg0 : memref<f32>
// expected-error @+1 {{failed to legalize operation 'test.sink'}} // expected-error @+1 {{failed to legalize operation 'test.sink'}}
"test.sink"(%0) : (tensor<f32>) -> () "test.sink"(%0) : (tensor<f32>) -> ()
return return
} }
// ----- // -----
// CHECK: #[[$map1:.*]] = affine_map<(d0)[s0] -> (d0 + s0)>
// CHECK-LABEL: func @dyn_layout_to_no_layout_cast( // CHECK-LABEL: func @dyn_layout_to_no_layout_cast(
// CHECK-SAME: %[[arg:.*]]: memref<?xf32, #[[$map1]]>) // CHECK-SAME: %[[arg:.*]]: memref<?xf32, strided<[1], offset: ?>>)
// CHECK: %[[c0:.*]] = arith.constant 0 : index // CHECK: %[[c0:.*]] = arith.constant 0 : index
// CHECK: %[[dim:.*]] = memref.dim %[[arg]], %[[c0]] // CHECK: %[[dim:.*]] = memref.dim %[[arg]], %[[c0]]
// CHECK: %[[alloc:.*]] = memref.alloc(%[[dim]]) : memref<?xf32> // CHECK: %[[alloc:.*]] = memref.alloc(%[[dim]]) : memref<?xf32>
// CHECK: memref.copy %[[arg]], %[[alloc]] // CHECK: memref.copy %[[arg]], %[[alloc]]
// CHECK: return %[[alloc]] // CHECK: return %[[alloc]]
#map1 = affine_map<(d0)[s0] -> (d0 + s0)> func.func @dyn_layout_to_no_layout_cast(%m: memref<?xf32, strided<[1], offset: ?>>) -> memref<?xf32> {
func.func @dyn_layout_to_no_layout_cast(%m: memref<?xf32, #map1>) -> memref<?xf32> { %0 = bufferization.to_tensor %m : memref<?xf32, strided<[1], offset: ?>>
%0 = bufferization.to_tensor %m : memref<?xf32, #map1>
%1 = bufferization.to_memref %0 : memref<?xf32> %1 = bufferization.to_memref %0 : memref<?xf32>
return %1 : memref<?xf32> return %1 : memref<?xf32>
} }
// ----- // -----
// CHECK: #[[$map2:.*]] = affine_map<(d0)[s0] -> (d0 * 100 + s0)>
// CHECK-LABEL: func @fancy_layout_to_no_layout_cast( // CHECK-LABEL: func @fancy_layout_to_no_layout_cast(
// CHECK-SAME: %[[arg:.*]]: memref<?xf32, #[[$map2]]>) // CHECK-SAME: %[[arg:.*]]: memref<?xf32, strided<[100], offset: ?>>)
// CHECK: %[[c0:.*]] = arith.constant 0 : index // CHECK: %[[c0:.*]] = arith.constant 0 : index
// CHECK: %[[dim:.*]] = memref.dim %[[arg]], %[[c0]] // CHECK: %[[dim:.*]] = memref.dim %[[arg]], %[[c0]]
// CHECK: %[[alloc:.*]] = memref.alloc(%[[dim]]) : memref<?xf32> // CHECK: %[[alloc:.*]] = memref.alloc(%[[dim]]) : memref<?xf32>
// CHECK: memref.copy %[[arg]], %[[alloc]] // CHECK: memref.copy %[[arg]], %[[alloc]]
// CHECK: return %[[alloc]] // CHECK: return %[[alloc]]
#map2 = affine_map<(d0)[s0] -> (d0 * 100 + s0)> func.func @fancy_layout_to_no_layout_cast(%m: memref<?xf32, strided<[100], offset: ?>>) -> memref<?xf32> {
func.func @fancy_layout_to_no_layout_cast(%m: memref<?xf32, #map2>) -> memref<?xf32> { %0 = bufferization.to_tensor %m : memref<?xf32, strided<[100], offset: ?>>
%0 = bufferization.to_tensor %m : memref<?xf32, #map2>
%1 = bufferization.to_memref %0 : memref<?xf32> %1 = bufferization.to_memref %0 : memref<?xf32>
return %1 : memref<?xf32> return %1 : memref<?xf32>
} }
// ----- // -----
// CHECK: #[[$map3:.*]] = affine_map<(d0)[s0] -> (d0 + 25)>
// CHECK-LABEL: func @static_layout_to_no_layout_cast( // CHECK-LABEL: func @static_layout_to_no_layout_cast(
// CHECK-SAME: %[[arg:.*]]: memref<?xf32, #[[$map3]]>) // CHECK-SAME: %[[arg:.*]]: memref<?xf32, strided<[1], offset: 25>>)
// CHECK: %[[c0:.*]] = arith.constant 0 : index // CHECK: %[[c0:.*]] = arith.constant 0 : index
// CHECK: %[[dim:.*]] = memref.dim %[[arg]], %[[c0]] // CHECK: %[[dim:.*]] = memref.dim %[[arg]], %[[c0]]
// CHECK: %[[alloc:.*]] = memref.alloc(%[[dim]]) : memref<?xf32> // CHECK: %[[alloc:.*]] = memref.alloc(%[[dim]]) : memref<?xf32>
// CHECK: memref.copy %[[arg]], %[[alloc]] // CHECK: memref.copy %[[arg]], %[[alloc]]
// CHECK: return %[[alloc]] // CHECK: return %[[alloc]]
#map3 = affine_map<(d0)[s0] -> (d0 + 25)> func.func @static_layout_to_no_layout_cast(%m: memref<?xf32, strided<[1], offset: 25>>) -> memref<?xf32> {
func.func @static_layout_to_no_layout_cast(%m: memref<?xf32, #map3>) -> memref<?xf32> { %0 = bufferization.to_tensor %m : memref<?xf32, strided<[1], offset: 25>>
%0 = bufferization.to_tensor %m : memref<?xf32, #map3>
%1 = bufferization.to_memref %0 : memref<?xf32> %1 = bufferization.to_memref %0 : memref<?xf32>
return %1 : memref<?xf32> return %1 : memref<?xf32>
} }
// ----- // -----
// TODO: to_memref with layout maps not supported yet. This should fold to a // TODO: to_memref with layout maps not supported yet. This should fold to a
// memref.cast. // memref.cast.
#map4 = affine_map<(d0)[s0] -> (d0 + s0)> func.func @no_layout_to_dyn_layout_cast(%m: memref<?xf32>) -> memref<?xf32, strided<[1], offset: ?>> {
func.func @no_layout_to_dyn_layout_cast(%m: memref<?xf32>) -> memref<?xf32, #map4> {
%0 = bufferization.to_tensor %m : memref<?xf32> %0 = bufferization.to_tensor %m : memref<?xf32>
// expected-error @+1 {{failed to materialize conversion for result #0 of operation 'bufferization.to_memref' that remained live after conversion}} // expected-error @+1 {{failed to materialize conversion for result #0 of operation 'bufferization.to_memref' that remained live after conversion}}
%1 = bufferization.to_memref %0 : memref<?xf32, #map4> %1 = bufferization.to_memref %0 : memref<?xf32, strided<[1], offset: ?>>
// expected-note @+1 {{see existing live user here}} // expected-note @+1 {{see existing live user here}}
return %1 : memref<?xf32, #map4> return %1 : memref<?xf32, strided<[1], offset: ?>>
} }
// ----- // -----
func.func @illegal_unranked_to_rank(%m: memref<*xf32>) -> memref<?xf32> { func.func @illegal_unranked_to_rank(%m: memref<*xf32>) -> memref<?xf32> {
// expected-note @+1 {{prior use here}} // expected-note @+1 {{prior use here}}
%0 = bufferization.to_tensor %m : memref<*xf32> %0 = bufferization.to_tensor %m : memref<*xf32>
// expected-error @+1 {{expects different type than prior uses: 'tensor<?xf32>' vs 'tensor<*xf32>'}} // expected-error @+1 {{expects different type than prior uses: 'tensor<?xf32>' vs 'tensor<*xf32>'}}
%1 = bufferization.to_memref %0 : memref<?xf32> %1 = bufferization.to_memref %0 : memref<?xf32>
return %1 : memref<?xf32> return %1 : memref<?xf32>
} }

mlir/test/Dialect/Linalg/drop-unit-extent-dims.mlir

Show First 20 Lines • Show All 747 Lines • ▼ Show 20 Lines
// CHECK-SAME: outs(%[[INIT_RESHAPE]] : memref<f32>) // CHECK-SAME: outs(%[[INIT_RESHAPE]] : memref<f32>)
// CHECK: return %[[INIT:.+]] : memref<1xf32> // CHECK: return %[[INIT:.+]] : memref<1xf32>
// ----- // -----
// Test that nothing changes and no assertions are fired for memrefs with affine // Test that nothing changes and no assertions are fired for memrefs with affine
// maps while still changing the other operations. // maps while still changing the other operations.
#map0 = affine_map<(d0, d1, d2)[s0] -> (d0 * s0 + d1 + d2)>
#accesses = [ #accesses = [
affine_map<(i, j, k, l, m) -> (i, k, m)>, affine_map<(i, j, k, l, m) -> (i, k, m)>,
affine_map<(i, j, k, l, m) -> ()>, affine_map<(i, j, k, l, m) -> ()>,
affine_map<(i, j, k, l, m) -> (i, k, j, l, m)> affine_map<(i, j, k, l, m) -> (i, k, j, l, m)>
] ]
#trait = { #trait = {
iterator_types = ["parallel", "parallel", "parallel", "parallel", "parallel"], iterator_types = ["parallel", "parallel", "parallel", "parallel", "parallel"],
indexing_maps = #accesses, indexing_maps = #accesses,
library_call = "some_external_func" library_call = "some_external_func"
} }
func.func @input_stays_same(%arg0 : memref<?x1x?xf32, #map0>, %arg1 : f32, %shape: memref<?x1x?x1x?xf32>) -> memref<?x1x?x1x?xf32> { func.func @input_stays_same(%arg0 : memref<?x1x?xf32, strided<[?, 1, 1]>>, %arg1 : f32, %shape: memref<?x1x?x1x?xf32>) -> memref<?x1x?x1x?xf32> {
linalg.generic #trait linalg.generic #trait
ins(%arg0, %arg1 : memref<?x1x?xf32, #map0>, f32) ins(%arg0, %arg1 : memref<?x1x?xf32, strided<[?, 1, 1]>>, f32)
outs(%shape : memref<?x1x?x1x?xf32>) { outs(%shape : memref<?x1x?x1x?xf32>) {
^bb0(%arg2 : f32, %arg3 : f32, %arg4 : f32) : ^bb0(%arg2 : f32, %arg3 : f32, %arg4 : f32) :
linalg.yield %arg3 : f32 linalg.yield %arg3 : f32
} }
return %shape : memref<?x1x?x1x?xf32> return %shape : memref<?x1x?x1x?xf32>
} }
// CHECK: #[[MAP0:.*]] = affine_map<(d0, d1, d2)[s0] -> (d0 * s0 + d1 + d2)>
// CHECK: #[[MAP1:.*]] = affine_map<(d0, d1, d2) -> (d0, 0, d2)> // CHECK: #[[MAP1:.*]] = affine_map<(d0, d1, d2) -> (d0, 0, d2)>
// CHECK: #[[MAP2:.*]] = affine_map<(d0, d1, d2) -> ()> // CHECK: #[[MAP2:.*]] = affine_map<(d0, d1, d2) -> ()>
// CHECK: #[[MAP3:.*]] = affine_map<(d0, d1, d2) -> (d0, d1, d2)> // CHECK: #[[MAP3:.*]] = affine_map<(d0, d1, d2) -> (d0, d1, d2)>
// CHECK: func @input_stays_same( // CHECK: func @input_stays_same(
// CHECK-SAME: %[[ARG0:.*]]: memref<?x1x?xf32, #[[MAP0]]>, // CHECK-SAME: %[[ARG0:.*]]: memref<?x1x?xf32, strided<[?, 1, 1]>>,
// CHECK-SAME: %[[ARG1:.*]]: f32, %[[ARG2:.*]]: memref<?x1x?x1x?xf32>) // CHECK-SAME: %[[ARG1:.*]]: f32, %[[ARG2:.*]]: memref<?x1x?x1x?xf32>)
// CHECK-SAME -> memref<?x1x?x1x?xf32> { // CHECK-SAME -> memref<?x1x?x1x?xf32> {
// CHECK: %[[OUT:.*]] = memref.collapse_shape %[[ARG2]] {{\[}}[0, 1], [2, 3], [4]] // CHECK: %[[OUT:.*]] = memref.collapse_shape %[[ARG2]] {{\[}}[0, 1], [2, 3], [4]]
// CHECK-SAME: : memref<?x1x?x1x?xf32> into memref<?x?x?xf32> // CHECK-SAME: : memref<?x1x?x1x?xf32> into memref<?x?x?xf32>
// CHECK: linalg.generic // CHECK: linalg.generic
// CHECK-SAME: {indexing_maps = [#[[MAP1]], #[[MAP2]], #[[MAP3]]], // CHECK-SAME: {indexing_maps = [#[[MAP1]], #[[MAP2]], #[[MAP3]]],
// CHECK-SAME: iterator_types = ["parallel", "parallel", "parallel"]} // CHECK-SAME: iterator_types = ["parallel", "parallel", "parallel"]}
// CHECK-SAME: ins(%[[ARG0]], %[[ARG1]] : memref<?x1x?xf32, #[[MAP0]]>, f32) // CHECK-SAME: ins(%[[ARG0]], %[[ARG1]] : memref<?x1x?xf32, strided<[?, 1, 1]>>, f32)
// CHECK-SAME: outs(%[[OUT]] : memref<?x?x?xf32>) { // CHECK-SAME: outs(%[[OUT]] : memref<?x?x?xf32>) {
// CHECK: ^bb0(%{{.*}}: f32, %[[ARG:.*]]: f32, %{{.*}}: f32): // CHECK: ^bb0(%{{.*}}: f32, %[[ARG:.*]]: f32, %{{.*}}: f32):
// CHECK: linalg.yield %[[ARG]] : f32 // CHECK: linalg.yield %[[ARG]] : f32
// CHECK: } // CHECK: }
// CHECK: return %[[ARG2]] : memref<?x1x?x1x?xf32> // CHECK: return %[[ARG2]] : memref<?x1x?x1x?xf32>
// ----- // -----
▲ Show 20 Lines • Show All 74 Lines • Show Last 20 Lines

mlir/test/Dialect/Linalg/fusion-indexed.mlir

// RUN: mlir-opt %s -test-linalg-greedy-fusion -split-input-file | FileCheck %s // RUN: mlir-opt %s -test-linalg-greedy-fusion -split-input-file | FileCheck %s
#map = affine_map<(d0, d1)[s0, s1, s2] -> (d0 * s1 + s0 + d1 * s2)>
#id_2d = affine_map<(d0, d1) -> (d0, d1)> #id_2d = affine_map<(d0, d1) -> (d0, d1)>
#pointwise_2d_trait = { #pointwise_2d_trait = {
indexing_maps = [#id_2d, #id_2d, #id_2d], indexing_maps = [#id_2d, #id_2d, #id_2d],
iterator_types = ["parallel", "parallel"] iterator_types = ["parallel", "parallel"]
} }
func.func @fuse_indexed_consumer(%A: memref<?x?xf32>, func.func @fuse_indexed_consumer(%A: memref<?x?xf32>,
%B: memref<?x?xf32>, %B: memref<?x?xf32>,
%C: memref<?x?xf32>, %C: memref<?x?xf32>,
Show All 11 Linesfunc.func @fuse_indexed_consumer(%A: memref<?x?xf32>,
%c10 = arith.constant 10 : index %c10 = arith.constant 10 : index
%0 = memref.dim %C, %c0 : memref<?x?xf32> %0 = memref.dim %C, %c0 : memref<?x?xf32>
%1 = memref.dim %C, %c1 : memref<?x?xf32> %1 = memref.dim %C, %c1 : memref<?x?xf32>
%2 = memref.dim %D, %c0 : memref<?x?xf32> %2 = memref.dim %D, %c0 : memref<?x?xf32>
%3 = memref.dim %D, %c1 : memref<?x?xf32> %3 = memref.dim %D, %c1 : memref<?x?xf32>
scf.for %arg2 = %c0 to %0 step %c10 { scf.for %arg2 = %c0 to %0 step %c10 {
scf.for %arg3 = %c0 to %1 step %c25 { scf.for %arg3 = %c0 to %1 step %c25 {
%4 = memref.subview %C[%arg2, %arg3][%c10, %c25][%c1, %c1] : %4 = memref.subview %C[%arg2, %arg3][%c10, %c25][%c1, %c1] :
memref<?x?xf32> to memref<?x?xf32, #map> memref<?x?xf32> to memref<?x?xf32, strided<[?, ?], offset: ?>>
%5 = memref.subview %D[%arg2, %arg3][%c10, %c25][%c1, %c1] : %5 = memref.subview %D[%arg2, %arg3][%c10, %c25][%c1, %c1] :
memref<?x?xf32> to memref<?x?xf32, #map> memref<?x?xf32> to memref<?x?xf32, strided<[?, ?], offset: ?>>
linalg.generic { linalg.generic {
indexing_maps = [#id_2d, #id_2d], indexing_maps = [#id_2d, #id_2d],
iterator_types = ["parallel", "parallel"]} iterator_types = ["parallel", "parallel"]}
ins(%4 : memref<?x?xf32, #map>) ins(%4 : memref<?x?xf32, strided<[?, ?], offset: ?>>)
outs(%5 : memref<?x?xf32, #map>) { outs(%5 : memref<?x?xf32, strided<[?, ?], offset: ?>>) {
^bb0(%arg4: f32, %arg5: f32): ^bb0(%arg4: f32, %arg5: f32):
%idx0 = linalg.index 0 : index %idx0 = linalg.index 0 : index
%idx1 = linalg.index 1 : index %idx1 = linalg.index 1 : index
%6 = arith.addi %idx0, %arg2 : index %6 = arith.addi %idx0, %arg2 : index
%7 = arith.addi %idx1, %arg3 : index %7 = arith.addi %idx1, %arg3 : index
%8 = arith.index_cast %6 : index to i32 %8 = arith.index_cast %6 : index to i32
%9 = arith.sitofp %8 : i32 to f32 %9 = arith.sitofp %8 : i32 to f32
%10 = arith.index_cast %7 : index to i32 %10 = arith.index_cast %7 : index to i32
Show All 13 Lines
// CHECK: linalg.generic // CHECK: linalg.generic
// CHECK-NOT: affine.apply // CHECK-NOT: affine.apply
// CHECK: arith.addf // CHECK: arith.addf
// CHECK: linalg.generic // CHECK: linalg.generic
// CHECK: arith.index_cast // CHECK: arith.index_cast
// ----- // -----
#map = affine_map<(d0, d1)[s0, s1, s2] -> (d0 * s1 + s0 + d1 * s2)>
func.func @fuse_indexed_producer(%A: memref<?x?xindex>, func.func @fuse_indexed_producer(%A: memref<?x?xindex>,
%B: memref<?x?xindex>) { %B: memref<?x?xindex>) {
%c1 = arith.constant 1 : index %c1 = arith.constant 1 : index
%c0 = arith.constant 0 : index %c0 = arith.constant 0 : index
%c25 = arith.constant 25 : index %c25 = arith.constant 25 : index
%c10 = arith.constant 10 : index %c10 = arith.constant 10 : index
linalg.generic { linalg.generic {
indexing_maps = [affine_map<(i, j) -> (j, i)>], indexing_maps = [affine_map<(i, j) -> (j, i)>],
iterator_types = ["parallel", "parallel"]} iterator_types = ["parallel", "parallel"]}
outs(%A : memref<?x?xindex>) { outs(%A : memref<?x?xindex>) {
^bb0(%a: index): ^bb0(%a: index):
%idx0 = linalg.index 0 : index %idx0 = linalg.index 0 : index
%idx1 = linalg.index 1 : index %idx1 = linalg.index 1 : index
%0 = arith.addi %idx0, %idx1 : index %0 = arith.addi %idx0, %idx1 : index
linalg.yield %0 : index linalg.yield %0 : index
} }
%A_X = memref.dim %A, %c0 : memref<?x?xindex> %A_X = memref.dim %A, %c0 : memref<?x?xindex>
%A_Y = memref.dim %A, %c1 : memref<?x?xindex> %A_Y = memref.dim %A, %c1 : memref<?x?xindex>
scf.parallel (%arg2, %arg3) = (%c0, %c0) to (%A_X, %A_Y) step (%c10, %c25) { scf.parallel (%arg2, %arg3) = (%c0, %c0) to (%A_X, %A_Y) step (%c10, %c25) {
%A_view = memref.subview %A[%arg2, %arg3][%c10, %c25][%c1, %c1] : %A_view = memref.subview %A[%arg2, %arg3][%c10, %c25][%c1, %c1] :
memref<?x?xindex> to memref<?x?xindex, #map> memref<?x?xindex> to memref<?x?xindex, strided<[?, ?], offset: ?>>
%B_view = memref.subview %B[%arg2, %arg3][%c10, %c25][%c1, %c1] : %B_view = memref.subview %B[%arg2, %arg3][%c10, %c25][%c1, %c1] :
memref<?x?xindex> to memref<?x?xindex, #map> memref<?x?xindex> to memref<?x?xindex, strided<[?, ?], offset: ?>>
linalg.generic { linalg.generic {
indexing_maps = [affine_map<(i, j) -> (i, j)>, indexing_maps = [affine_map<(i, j) -> (i, j)>,
affine_map<(i, j) -> (i, j)>], affine_map<(i, j) -> (i, j)>],
iterator_types = ["parallel", "parallel"]} iterator_types = ["parallel", "parallel"]}
ins(%A_view : memref<?x?xindex, #map>) ins(%A_view : memref<?x?xindex, strided<[?, ?], offset: ?>>)
outs(%B_view : memref<?x?xindex, #map>) { outs(%B_view : memref<?x?xindex, strided<[?, ?], offset: ?>>) {
^bb0(%a: index, %b: index): ^bb0(%a: index, %b: index):
linalg.yield %a : index linalg.yield %a : index
} }
} }
return return
} }
// CHECK: [[$MAP:#[a-zA-Z0-9_]*]] = affine_map<(d0, d1) -> (d0 + d1)> // CHECK: [[$MAP:#[a-zA-Z0-9_]*]] = affine_map<(d0, d1) -> (d0 + d1)>
// CHECK-LABEL: func @fuse_indexed_producer // CHECK-LABEL: func @fuse_indexed_producer
// CHECK: scf.parallel ([[I:%.*]], [[J:%.*]]) = // CHECK: scf.parallel ([[I:%.*]], [[J:%.*]]) =
// CHECK: linalg.generic // CHECK: linalg.generic
// CHECK: [[idx0:%.*]] = linalg.index 0 : index // CHECK: [[idx0:%.*]] = linalg.index 0 : index
// CHECK: [[i_new:%.*]] = affine.apply [[$MAP]]([[idx0]], [[J]]) // CHECK: [[i_new:%.*]] = affine.apply [[$MAP]]([[idx0]], [[J]])
// CHECK: [[idx1:%.*]] = linalg.index 1 : index // CHECK: [[idx1:%.*]] = linalg.index 1 : index
// CHECK: [[j_new:%.*]] = affine.apply [[$MAP]]([[idx1]], [[I]]) // CHECK: [[j_new:%.*]] = affine.apply [[$MAP]]([[idx1]], [[I]])
// CHECK: [[sum:%.*]] = arith.addi [[i_new]], [[j_new]] : index // CHECK: [[sum:%.*]] = arith.addi [[i_new]], [[j_new]] : index
// CHECK: linalg.yield [[sum]] : index // CHECK: linalg.yield [[sum]] : index
// CHECK: linalg.generic // CHECK: linalg.generic
// ----- // -----
#map = affine_map<(d0, d1)[s0, s1, s2] -> (d0 * s1 + s0 + d1 * s2)>
func.func @fuse_indexed_producer_tiled_second_dim_only(%A: memref<?x?xindex>, func.func @fuse_indexed_producer_tiled_second_dim_only(%A: memref<?x?xindex>,
%B: memref<?x?xindex>) { %B: memref<?x?xindex>) {
%c1 = arith.constant 1 : index %c1 = arith.constant 1 : index
%c0 = arith.constant 0 : index %c0 = arith.constant 0 : index
%c25 = arith.constant 25 : index %c25 = arith.constant 25 : index
linalg.generic { linalg.generic {
indexing_maps = [affine_map<(i, j) -> (i, j)>], indexing_maps = [affine_map<(i, j) -> (i, j)>],
iterator_types = ["parallel", "parallel"]} iterator_types = ["parallel", "parallel"]}
outs(%A : memref<?x?xindex>) { outs(%A : memref<?x?xindex>) {
^bb0(%a: index): ^bb0(%a: index):
%idx0 = linalg.index 0 : index %idx0 = linalg.index 0 : index
%idx1 = linalg.index 1 : index %idx1 = linalg.index 1 : index
%0 = arith.addi %idx0, %idx1 : index %0 = arith.addi %idx0, %idx1 : index
linalg.yield %0 : index linalg.yield %0 : index
} }
%A_X = memref.dim %A, %c0 : memref<?x?xindex> %A_X = memref.dim %A, %c0 : memref<?x?xindex>
%A_Y = memref.dim %A, %c1 : memref<?x?xindex> %A_Y = memref.dim %A, %c1 : memref<?x?xindex>
scf.parallel (%arg3) = (%c0) to (%A_Y) step (%c25) { scf.parallel (%arg3) = (%c0) to (%A_Y) step (%c25) {
%A_view = memref.subview %A[%c0, %arg3][%A_X, %c25][%c1, %c1] : %A_view = memref.subview %A[%c0, %arg3][%A_X, %c25][%c1, %c1] :
memref<?x?xindex> to memref<?x?xindex, #map> memref<?x?xindex> to memref<?x?xindex, strided<[?, ?], offset: ?>>
%B_view = memref.subview %B[%c0, %arg3][%A_X, %c25][%c1, %c1] : %B_view = memref.subview %B[%c0, %arg3][%A_X, %c25][%c1, %c1] :
memref<?x?xindex> to memref<?x?xindex, #map> memref<?x?xindex> to memref<?x?xindex, strided<[?, ?], offset: ?>>
linalg.generic { linalg.generic {
indexing_maps = [affine_map<(i, j) -> (i, j)>, indexing_maps = [affine_map<(i, j) -> (i, j)>,
affine_map<(i, j) -> (i, j)>], affine_map<(i, j) -> (i, j)>],
iterator_types = ["parallel", "parallel"]} iterator_types = ["parallel", "parallel"]}
ins(%A_view : memref<?x?xindex, #map>) ins(%A_view : memref<?x?xindex, strided<[?, ?], offset: ?>>)
outs(%B_view : memref<?x?xindex, #map>) { outs(%B_view : memref<?x?xindex, strided<[?, ?], offset: ?>>) {
^bb0(%a: index, %b: index): ^bb0(%a: index, %b: index):
linalg.yield %a : index linalg.yield %a : index
} }
} }
return return
} }
// CHECK: [[$MAP:#[a-zA-Z0-9_]*]] = affine_map<(d0, d1) -> (d0 + d1)> // CHECK: [[$MAP:#[a-zA-Z0-9_]*]] = affine_map<(d0, d1) -> (d0 + d1)>
// CHECK-LABEL: func @fuse_indexed_producer_tiled_second_dim_only // CHECK-LABEL: func @fuse_indexed_producer_tiled_second_dim_only
Show All 9 Lines

mlir/test/Dialect/Linalg/fusion.mlir

Show First 20 Lines • Show All 593 Lines • ▼ Show 20 Lines
// CHECK: linalg.generic // CHECK: linalg.generic
// CHECK: arith.mulf // CHECK: arith.mulf
// ----- // -----
#map0 = affine_map<(d0, d1) -> (d0)> #map0 = affine_map<(d0, d1) -> (d0)>
#map1 = affine_map<(d0, d1) -> (d0, d1)> #map1 = affine_map<(d0, d1) -> (d0, d1)>
#map2 = affine_map<(d0, d1)[s0, s1, s2] -> (d0 * s1 + s0 + d1 * s2)>
func.func @fusion_of_three(%arg0: memref<100x10xf32>, func.func @fusion_of_three(%arg0: memref<100x10xf32>,
%arg1: memref<100xf32>, %arg1: memref<100xf32>,
%arg2: memref<100x10xf32>) { %arg2: memref<100x10xf32>) {
%c0 = arith.constant 0 : index %c0 = arith.constant 0 : index
%c1 = arith.constant 1 : index %c1 = arith.constant 1 : index
%0 = memref.alloc() {temp = true} : memref<100x10xf32> %0 = memref.alloc() {temp = true} : memref<100x10xf32>
linalg.generic { linalg.generic {
Show All 17 Linesfunc.func @fusion_of_three(%arg0: memref<100x10xf32>,
memref.dealloc %0 : memref<100x10xf32> memref.dealloc %0 : memref<100x10xf32>
%2 = memref.dim %1, %c0 : memref<100x10xf32> %2 = memref.dim %1, %c0 : memref<100x10xf32>
%3 = memref.dim %1, %c1 : memref<100x10xf32> %3 = memref.dim %1, %c1 : memref<100x10xf32>
%4 = memref.dim %arg2, %c0 : memref<100x10xf32> %4 = memref.dim %arg2, %c0 : memref<100x10xf32>
%5 = memref.dim %arg2, %c1 : memref<100x10xf32> %5 = memref.dim %arg2, %c1 : memref<100x10xf32>
scf.for %i = %c0 to %2 step %c1 { scf.for %i = %c0 to %2 step %c1 {
scf.for %j = %c0 to %3 step %c1 { scf.for %j = %c0 to %3 step %c1 {
%6 = memref.subview %1[%i, %j][%c1, %c1][%c1, %c1] : %6 = memref.subview %1[%i, %j][%c1, %c1][%c1, %c1] :
memref<100x10xf32> to memref<?x?xf32, #map2> memref<100x10xf32> to memref<?x?xf32, strided<[?, ?], offset: ?>>
%7 = memref.subview %arg2[%i, %j][%c1, %c1][%c1, %c1] : %7 = memref.subview %arg2[%i, %j][%c1, %c1][%c1, %c1] :
memref<100x10xf32> to memref<?x?xf32, #map2> memref<100x10xf32> to memref<?x?xf32, strided<[?, ?], offset: ?>>
linalg.generic { linalg.generic {
indexing_maps = [#map1, #map1], indexing_maps = [#map1, #map1],
iterator_types = ["parallel", "parallel"]} iterator_types = ["parallel", "parallel"]}
ins(%6 : memref<?x?xf32, #map2>) ins(%6 : memref<?x?xf32, strided<[?, ?], offset: ?>>)
outs(%7 : memref<?x?xf32, #map2>) { outs(%7 : memref<?x?xf32, strided<[?, ?], offset: ?>>) {
^bb0(%arg3: f32, %arg4: f32): ^bb0(%arg3: f32, %arg4: f32):
%8 = math.exp %arg3 : f32 %8 = math.exp %arg3 : f32
linalg.yield %8 : f32 linalg.yield %8 : f32
} }
} }
} }
memref.dealloc %1 : memref<100x10xf32> memref.dealloc %1 : memref<100x10xf32>
return return
Show All 12 Lines
// CHECK: exp // CHECK: exp
// CHECK: linalg.yield // CHECK: linalg.yield
// ----- // -----
#map0 = affine_map<(d0)[s0] -> (2, -d0 + s0)> #map0 = affine_map<(d0)[s0] -> (2, -d0 + s0)>
#map1 = affine_map<(d0)[s0] -> (3, -d0 + s0)> #map1 = affine_map<(d0)[s0] -> (3, -d0 + s0)>
#map2 = affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>
#map3 = affine_map<(d0)[s0, s1] -> (s0 + 1, -d0 + s0 + s1)> #map3 = affine_map<(d0)[s0, s1] -> (s0 + 1, -d0 + s0 + s1)>
#map4 = affine_map<(d0)[s0, s1] -> (s0 + 2, -d0 + s0 + s1)> #map4 = affine_map<(d0)[s0, s1] -> (s0 + 2, -d0 + s0 + s1)>
func.func @fill_and_conv(%arg0: memref<?x?xf32>, %arg1: memref<?x?xf32>, %arg2: memref<?x?xf32>) { func.func @fill_and_conv(%arg0: memref<?x?xf32>, %arg1: memref<?x?xf32>, %arg2: memref<?x?xf32>) {
%cst = arith.constant 0.000000e+00 : f32 %cst = arith.constant 0.000000e+00 : f32
%c2 = arith.constant 2 : index %c2 = arith.constant 2 : index
%c3 = arith.constant 3 : index %c3 = arith.constant 3 : index
%c0 = arith.constant 0 : index %c0 = arith.constant 0 : index
%c1 = arith.constant 1 : index %c1 = arith.constant 1 : index
linalg.fill ins(%cst : f32) outs(%arg0 : memref<?x?xf32>) linalg.fill ins(%cst : f32) outs(%arg0 : memref<?x?xf32>)
%2 = memref.dim %arg1, %c0 : memref<?x?xf32> %2 = memref.dim %arg1, %c0 : memref<?x?xf32>
%3 = memref.dim %arg1, %c1 : memref<?x?xf32> %3 = memref.dim %arg1, %c1 : memref<?x?xf32>
%4 = memref.dim %arg2, %c0 : memref<?x?xf32> %4 = memref.dim %arg2, %c0 : memref<?x?xf32>
%5 = memref.dim %arg2, %c1 : memref<?x?xf32> %5 = memref.dim %arg2, %c1 : memref<?x?xf32>
scf.for %arg3 = %c0 to %4 step %c2 { scf.for %arg3 = %c0 to %4 step %c2 {
scf.for %arg4 = %c0 to %5 step %c3 { scf.for %arg4 = %c0 to %5 step %c3 {
%6 = affine.min #map3(%arg3)[%2, %4] %6 = affine.min #map3(%arg3)[%2, %4]
%7 = affine.min #map4(%arg4)[%3, %5] %7 = affine.min #map4(%arg4)[%3, %5]
%8 = memref.subview %arg0[%arg3, %arg4] [%6, %7] [1, 1] : memref<?x?xf32> to memref<?x?xf32, #map2> %8 = memref.subview %arg0[%arg3, %arg4] [%6, %7] [1, 1] : memref<?x?xf32> to memref<?x?xf32, strided<[?, 1], offset: ?>>
%9 = affine.min #map0(%arg3)[%4] %9 = affine.min #map0(%arg3)[%4]
%10 = affine.min #map1(%arg4)[%5] %10 = affine.min #map1(%arg4)[%5]
%11 = memref.subview %arg2[%arg3, %arg4] [%9, %10] [1, 1] : memref<?x?xf32> to memref<?x?xf32, #map2> %11 = memref.subview %arg2[%arg3, %arg4] [%9, %10] [1, 1] : memref<?x?xf32> to memref<?x?xf32, strided<[?, 1], offset: ?>>
linalg.conv_2d ins(%8, %arg1 : memref<?x?xf32, #map2>, memref<?x?xf32>) outs(%11 : memref<?x?xf32, #map2>) linalg.conv_2d ins(%8, %arg1 : memref<?x?xf32, strided<[?, 1], offset: ?>>, memref<?x?xf32>) outs(%11 : memref<?x?xf32, strided<[?, 1], offset: ?>>)
} }
} }
return return
} }
// CHECK-LABEL: func @fill_and_conv // CHECK-LABEL: func @fill_and_conv
// CHECK: scf.for // CHECK: scf.for
// CHECK: scf.for // CHECK: scf.for
// CHECK: linalg.fill // CHECK: linalg.fill
▲ Show 20 Lines • Show All 44 Lines • Show Last 20 Lines

mlir/test/Dialect/Linalg/promote.mlir

Show First 20 Lines • Show All 145 Lines • ▼ Show 20 Lines ^bb0(%arg1: !pdl.operation):
%0 = transform.structured.match ops{["linalg.matmul"]} in %arg1 %0 = transform.structured.match ops{["linalg.matmul"]} in %arg1
%1 = transform.structured.promote %0 %1 = transform.structured.promote %0
} }
} }
// ----- // -----
#map0 = affine_map<(d0, d1, d2, d3)[s0, s1, s2, s3, s4] -> (d0 * s1 + s0 + d1 * s2 + d2 * s3 + d3 * s4)>
#map2 = affine_map<(d0, d1)[s0] -> (d0 * 128 + s0 + d1)>
#map5 = affine_map<(d0, d1)[s0, s1, s2] -> (d0 * s1 + s0 + d1 * s2)>
#map6 = affine_map<(d0, d1, d2) -> (d0, d2)> #map6 = affine_map<(d0, d1, d2) -> (d0, d2)>
#map7 = affine_map<(d0, d1, d2) -> (d1, d2)> #map7 = affine_map<(d0, d1, d2) -> (d1, d2)>
#map8 = affine_map<(d0, d1, d2) -> (d0, d1)> #map8 = affine_map<(d0, d1, d2) -> (d0, d1)>
// CHECK: promote_rank_reducing_subviews(%[[arg0:.+]]: memref<{{.*}}>, %[[arg1:.+]]: memref<{{.*}}>, %[[arg2:.+]]: memref<{{.*}}>, %[[lb1:.+]]: index, %[[lb2:.+]]: index, %[[lb3:.+]]: index, %[[lb4:.+]]: index, %[[lb5:.+]]: index, %[[lb6:.+]]: index, %[[ub1:.+]]: index, %[[ub2:.+]]: index // CHECK: promote_rank_reducing_subviews(%[[arg0:.+]]: memref<{{.*}}>, %[[arg1:.+]]: memref<{{.*}}>, %[[arg2:.+]]: memref<{{.*}}>, %[[lb1:.+]]: index, %[[lb2:.+]]: index, %[[lb3:.+]]: index, %[[lb4:.+]]: index, %[[lb5:.+]]: index, %[[lb6:.+]]: index, %[[ub1:.+]]: index, %[[ub2:.+]]: index
func.func @promote_rank_reducing_subviews(%arg0: memref<?x?x?x64xf32, #map0>, %arg1: memref<128x3x3x64xf32, #map0>, %arg2: memref<?x?x?x128xf32>, func.func @promote_rank_reducing_subviews(%arg0: memref<?x?x?x64xf32, strided<[?, ?, ?, ?], offset: ?>>, %arg1: memref<128x3x3x64xf32, strided<[?, ?, ?, ?], offset: ?>>, %arg2: memref<?x?x?x128xf32>,
%arg3: index, %arg4: index, %arg5: index, %arg6: index, %arg7: index, %arg8: index, %ub1: index, %ub2: index) { %arg3: index, %arg4: index, %arg5: index, %arg6: index, %arg7: index, %arg8: index, %ub1: index, %ub2: index) {
%13 = memref.subview %arg0[%arg3, 0, %arg4, %arg8] [1, 1, %ub1, 32] [1, 1, 1, 1] : memref<?x?x?x64xf32, #map0> to memref<?x32xf32, #map5> %13 = memref.subview %arg0[%arg3, 0, %arg4, %arg8] [1, 1, %ub1, 32] [1, 1, 1, 1] : memref<?x?x?x64xf32, strided<[?, ?, ?, ?], offset: ?>> to memref<?x32xf32, strided<[?, ?], offset: ?>>
%14 = memref.subview %arg1[0, %arg6, %arg7, %arg8] [128, 1, 1, 32] [1, 1, 1, 1] : memref<128x3x3x64xf32, #map0> to memref<128x32xf32, #map5> %14 = memref.subview %arg1[0, %arg6, %arg7, %arg8] [128, 1, 1, 32] [1, 1, 1, 1] : memref<128x3x3x64xf32, strided<[?, ?, ?, ?], offset: ?>> to memref<128x32xf32, strided<[?, ?], offset: ?>>
%9 = memref.subview %arg2[%arg3, %arg4, %arg5, 0] [1, 1, %ub2, 128] [1, 1, 1, 1] : memref<?x?x?x128xf32> to memref<?x128xf32, #map2> %9 = memref.subview %arg2[%arg3, %arg4, %arg5, 0] [1, 1, %ub2, 128] [1, 1, 1, 1] : memref<?x?x?x128xf32> to memref<?x128xf32, strided<[128, 1], offset: ?>>
// CHECK: %[[a_alloc:.+]] = memref.alloc // CHECK: %[[a_alloc:.+]] = memref.alloc
// CHECK: %[[a_view:.+]] = memref.view %[[a_alloc]]{{.*}} // CHECK: %[[a_view:.+]] = memref.view %[[a_alloc]]{{.*}}
// CHECK: %[[a_pro_subview:.+]] = memref.subview %[[a_view]][0, 0] [%[[ub1]], {{.+}}] [1, 1] // CHECK: %[[a_pro_subview:.+]] = memref.subview %[[a_view]][0, 0] [%[[ub1]], {{.+}}] [1, 1]
// CHECK: memref.alloc // CHECK: memref.alloc
// CHECK: %[[b_view:.+]] = memref.view // CHECK: %[[b_view:.+]] = memref.view
// CHECK: %[[b_pro_subview:.+]] = memref.subview %[[b_view]] // CHECK: %[[b_pro_subview:.+]] = memref.subview %[[b_view]]
// CHECK: memref.alloc // CHECK: memref.alloc
// CHECK: %[[c_view:.+]] = memref.view // CHECK: %[[c_view:.+]] = memref.view
// CHECK: %[[c_pro_subview:.+]] = memref.subview %[[c_view]] // CHECK: %[[c_pro_subview:.+]] = memref.subview %[[c_view]]
// CHECK-COUNT-3: memref.copy // CHECK-COUNT-3: memref.copy
// CHECK: linalg.generic // CHECK: linalg.generic
// CHECK-SAME: ins(%[[a_pro_subview]], %[[b_pro_subview]] // CHECK-SAME: ins(%[[a_pro_subview]], %[[b_pro_subview]]
// CHECK-SAME: outs(%[[c_pro_subview]] // CHECK-SAME: outs(%[[c_pro_subview]]
linalg.generic {indexing_maps = [#map6, #map7, #map8], iterator_types = ["parallel", "parallel", "reduction"]} ins(%13, %14 : memref<?x32xf32, #map5>, memref<128x32xf32, #map5>) outs(%9 : memref<?x128xf32, #map2>) { linalg.generic {indexing_maps = [#map6, #map7, #map8], iterator_types = ["parallel", "parallel", "reduction"]} ins(%13, %14 : memref<?x32xf32, strided<[?, ?], offset: ?>>, memref<128x32xf32, strided<[?, ?], offset: ?>>) outs(%9 : memref<?x128xf32, strided<[128, 1], offset: ?>>) {
^bb0(%arg9: f32, %arg10: f32, %arg11: f32): ^bb0(%arg9: f32, %arg10: f32, %arg11: f32):
%15 = arith.mulf %arg9, %arg10 : f32 %15 = arith.mulf %arg9, %arg10 : f32
%16 = arith.addf %arg11, %15 : f32 %16 = arith.addf %arg11, %15 : f32
linalg.yield %16 : f32 linalg.yield %16 : f32
} }
return return
} }
Show All 9 Lines

mlir/test/Dialect/MemRef/canonicalize.mlir

Show All 36 Lines
// CHECK: return %[[ARG0]] : memref<4x6x16x32xi8> // CHECK: return %[[ARG0]] : memref<4x6x16x32xi8>
func.func @subview_of_static_full_size(%arg0 : memref<4x6x16x32xi8>) -> memref<4x6x16x32xi8> { func.func @subview_of_static_full_size(%arg0 : memref<4x6x16x32xi8>) -> memref<4x6x16x32xi8> {
%0 = memref.subview %arg0[0, 0, 0, 0] [4, 6, 16, 32] [1, 1, 1, 1] : memref<4x6x16x32xi8> to memref<4x6x16x32xi8> %0 = memref.subview %arg0[0, 0, 0, 0] [4, 6, 16, 32] [1, 1, 1, 1] : memref<4x6x16x32xi8> to memref<4x6x16x32xi8>
return %0 : memref<4x6x16x32xi8> return %0 : memref<4x6x16x32xi8>
} }
// ----- // -----
#map0 = affine_map<(d0, d1, d2)[s0, s1, s2, s3] -> (d0 * s1 + s0 + d1 * s2 + d2 * s3)>
func.func @subview_canonicalize(%arg0 : memref<?x?x?xf32>, %arg1 : index, func.func @subview_canonicalize(%arg0 : memref<?x?x?xf32>, %arg1 : index,
%arg2 : index) -> memref<?x?x?xf32, #map0> %arg2 : index) -> memref<?x?x?xf32, strided<[?, ?, ?], offset: ?>>
{ {
%c0 = arith.constant 0 : index %c0 = arith.constant 0 : index
%c1 = arith.constant 1 : index %c1 = arith.constant 1 : index
%c4 = arith.constant 4 : index %c4 = arith.constant 4 : index
%0 = memref.subview %arg0[%c0, %arg1, %c1] [%c4, %c1, %arg2] [%c1, %c1, %c1] : memref<?x?x?xf32> to memref<?x?x?xf32, #map0> %0 = memref.subview %arg0[%c0, %arg1, %c1] [%c4, %c1, %arg2] [%c1, %c1, %c1] : memref<?x?x?xf32> to memref<?x?x?xf32, strided<[?, ?, ?], offset: ?>>
return %0 : memref<?x?x?xf32, #map0> return %0 : memref<?x?x?xf32, strided<[?, ?, ?], offset: ?>>
} }
// CHECK-LABEL: func @subview_canonicalize // CHECK-LABEL: func @subview_canonicalize
// CHECK-SAME: %[[ARG0:.+]]: memref<?x?x?xf32> // CHECK-SAME: %[[ARG0:.+]]: memref<?x?x?xf32>
// CHECK: %[[SUBVIEW:.+]] = memref.subview %[[ARG0]][0, %{{[a-zA-Z0-9_]+}}, 1] // CHECK: %[[SUBVIEW:.+]] = memref.subview %[[ARG0]][0, %{{[a-zA-Z0-9_]+}}, 1]
// CHECK-SAME: [4, 1, %{{[a-zA-Z0-9_]+}}] [1, 1, 1] // CHECK-SAME: [4, 1, %{{[a-zA-Z0-9_]+}}] [1, 1, 1]
// CHECK-SAME: : memref<?x?x?xf32> to memref<4x1x?xf32 // CHECK-SAME: : memref<?x?x?xf32> to memref<4x1x?xf32
// CHECK: %[[RESULT:.+]] = memref.cast %[[SUBVIEW]] // CHECK: %[[RESULT:.+]] = memref.cast %[[SUBVIEW]]
// CHEKC: return %[[RESULT]] // CHEKC: return %[[RESULT]]
// ----- // -----
#map0 = affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>
func.func @rank_reducing_subview_canonicalize(%arg0 : memref<?x?x?xf32>, %arg1 : index, func.func @rank_reducing_subview_canonicalize(%arg0 : memref<?x?x?xf32>, %arg1 : index,
%arg2 : index) -> memref<?x?xf32, #map0> %arg2 : index) -> memref<?x?xf32, strided<[?, 1], offset: ?>>
{ {
%c0 = arith.constant 0 : index %c0 = arith.constant 0 : index
%c1 = arith.constant 1 : index %c1 = arith.constant 1 : index
%c4 = arith.constant 4 : index %c4 = arith.constant 4 : index
%0 = memref.subview %arg0[%c0, %arg1, %c1] [%c4, 1, %arg2] [%c1, %c1, %c1] : memref<?x?x?xf32> to memref<?x?xf32, #map0> %0 = memref.subview %arg0[%c0, %arg1, %c1] [%c4, 1, %arg2] [%c1, %c1, %c1] : memref<?x?x?xf32> to memref<?x?xf32, strided<[?, 1], offset: ?>>
return %0 : memref<?x?xf32, #map0> return %0 : memref<?x?xf32, strided<[?, 1], offset: ?>>
} }
// CHECK-LABEL: func @rank_reducing_subview_canonicalize // CHECK-LABEL: func @rank_reducing_subview_canonicalize
// CHECK-SAME: %[[ARG0:.+]]: memref<?x?x?xf32> // CHECK-SAME: %[[ARG0:.+]]: memref<?x?x?xf32>
// CHECK: %[[SUBVIEW:.+]] = memref.subview %[[ARG0]][0, %{{[a-zA-Z0-9_]+}}, 1] // CHECK: %[[SUBVIEW:.+]] = memref.subview %[[ARG0]][0, %{{[a-zA-Z0-9_]+}}, 1]
// CHECK-SAME: [4, 1, %{{[a-zA-Z0-9_]+}}] [1, 1, 1] // CHECK-SAME: [4, 1, %{{[a-zA-Z0-9_]+}}] [1, 1, 1]
// CHECK-SAME: : memref<?x?x?xf32> to memref<4x?xf32 // CHECK-SAME: : memref<?x?x?xf32> to memref<4x?xf32
// CHECK: %[[RESULT:.+]] = memref.cast %[[SUBVIEW]] // CHECK: %[[RESULT:.+]] = memref.cast %[[SUBVIEW]]
// CHECK: return %[[RESULT]] // CHECK: return %[[RESULT]]
▲ Show 20 Lines • Show All 167 Lines • ▼ Show 20 Linesfunc.func @alloc_alignment_const_fold() -> memref<?xf32> {
// CHECK-NEXT: return %1 : memref<?xf32> // CHECK-NEXT: return %1 : memref<?xf32>
return %a : memref<?xf32> return %a : memref<?xf32>
} }
// ----- // -----
// CHECK-LABEL: func @alloc_const_fold_with_symbols1( // CHECK-LABEL: func @alloc_const_fold_with_symbols1(
// CHECK: %[[c1:.+]] = arith.constant 1 : index // CHECK: %[[c1:.+]] = arith.constant 1 : index
// CHECK: %[[mem1:.+]] = memref.alloc({{.*}})[%[[c1]], %[[c1]]] : memref<?xi32, #map> // CHECK: %[[mem1:.+]] = memref.alloc({{.*}})[%[[c1]], %[[c1]]] : memref<?xi32, strided{{.*}}>
// CHECK: return %[[mem1]] : memref<?xi32, #map> // CHECK: return %[[mem1]] : memref<?xi32, strided{{.*}}>
#map0 = affine_map<(d0)[s0, s1] -> (d0 * s1 + s0)> func.func @alloc_const_fold_with_symbols1(%arg0 : index) -> memref<?xi32, strided<[?], offset: ?>> {
func.func @alloc_const_fold_with_symbols1(%arg0 : index) -> memref<?xi32, #map0> {
%c1 = arith.constant 1 : index %c1 = arith.constant 1 : index
%0 = memref.alloc(%arg0)[%c1, %c1] : memref<?xi32, #map0> %0 = memref.alloc(%arg0)[%c1, %c1] : memref<?xi32, strided<[?], offset: ?>>
return %0 : memref<?xi32, #map0> return %0 : memref<?xi32, strided<[?], offset: ?>>
} }
// ----- // -----
// CHECK-LABEL: func @alloc_const_fold_with_symbols2( // CHECK-LABEL: func @alloc_const_fold_with_symbols2(
// CHECK: %[[c1:.+]] = arith.constant 1 : index // CHECK: %[[c1:.+]] = arith.constant 1 : index
// CHECK: %[[mem1:.+]] = memref.alloc()[%[[c1]], %[[c1]]] : memref<1xi32, #map> // CHECK: %[[mem1:.+]] = memref.alloc()[%[[c1]], %[[c1]]] : memref<1xi32, strided{{.*}}>
// CHECK: %[[mem2:.+]] = memref.cast %[[mem1]] : memref<1xi32, #map> to memref<?xi32, #map> // CHECK: %[[mem2:.+]] = memref.cast %[[mem1]] : memref<1xi32, strided{{.*}}> to memref<?xi32, strided{{.*}}>
// CHECK: return %[[mem2]] : memref<?xi32, #map> // CHECK: return %[[mem2]] : memref<?xi32, strided{{.*}}>
#map0 = affine_map<(d0)[s0, s1] -> (d0 * s1 + s0)> func.func @alloc_const_fold_with_symbols2() -> memref<?xi32, strided<[?], offset: ?>> {
func.func @alloc_const_fold_with_symbols2() -> memref<?xi32, #map0> {
%c1 = arith.constant 1 : index %c1 = arith.constant 1 : index
%0 = memref.alloc(%c1)[%c1, %c1] : memref<?xi32, #map0> %0 = memref.alloc(%c1)[%c1, %c1] : memref<?xi32, strided<[?], offset: ?>>
return %0 : memref<?xi32, #map0> return %0 : memref<?xi32, strided<[?], offset: ?>>
} }
// ----- // -----
// CHECK-LABEL: func @allocator // CHECK-LABEL: func @allocator
// CHECK: %[[alloc:.+]] = memref.alloc // CHECK: %[[alloc:.+]] = memref.alloc
// CHECK: memref.store %[[alloc:.+]], %arg0 // CHECK: memref.store %[[alloc:.+]], %arg0
func.func @allocator(%arg0 : memref<memref<?xi32>>, %arg1 : index) { func.func @allocator(%arg0 : memref<memref<?xi32>>, %arg1 : index) {
%0 = memref.alloc(%arg1) : memref<?xi32> %0 = memref.alloc(%arg1) : memref<?xi32>
▲ Show 20 Lines • Show All 184 Lines • ▼ Show 20 Linesfunc.func @fold_rank_memref(%arg0 : memref<?x?xf32>) -> (index) {
%rank_0 = memref.rank %arg0 : memref<?x?xf32> %rank_0 = memref.rank %arg0 : memref<?x?xf32>
// CHECK-NEXT: return [[C2]] // CHECK-NEXT: return [[C2]]
return %rank_0 : index return %rank_0 : index
} }
// ----- // -----
#map = affine_map<(d0, d1) -> (d0 * 42 + d1)> func.func @fold_no_op_subview(%arg0 : memref<20x42xf32>) -> memref<20x42xf32, strided<[42, 1]>> {
func.func @fold_no_op_subview(%arg0 : memref<20x42xf32>) -> memref<20x42xf32, #map> { %0 = memref.subview %arg0[0, 0] [20, 42] [1, 1] : memref<20x42xf32> to memref<20x42xf32, strided<[42, 1]>>
%0 = memref.subview %arg0[0, 0] [20, 42] [1, 1] : memref<20x42xf32> to memref<20x42xf32, #map> return %0 : memref<20x42xf32, strided<[42, 1]>>
return %0 : memref<20x42xf32, #map>
} }
// CHECK-LABEL: func @fold_no_op_subview( // CHECK-LABEL: func @fold_no_op_subview(
// CHECK: %[[ARG0:.+]]: memref<20x42xf32>) // CHECK: %[[ARG0:.+]]: memref<20x42xf32>)
// CHECK: %[[CAST:.+]] = memref.cast %[[ARG0]] // CHECK: %[[CAST:.+]] = memref.cast %[[ARG0]]
// CHECK: return %[[CAST]] // CHECK: return %[[CAST]]
// ----- // -----
#map = affine_map<(d0, d1) -> (d0 * 42 + d1 + 1)> func.func @no_fold_subview_with_non_zero_offset(%arg0 : memref<20x42xf32>) -> memref<20x42xf32, strided<[42, 1], offset: 1>> {
func.func @no_fold_subview_with_non_zero_offset(%arg0 : memref<20x42xf32>) -> memref<20x42xf32, #map> { %0 = memref.subview %arg0[0, 1] [20, 42] [1, 1] : memref<20x42xf32> to memref<20x42xf32, strided<[42, 1], offset: 1>>
%0 = memref.subview %arg0[0, 1] [20, 42] [1, 1] : memref<20x42xf32> to memref<20x42xf32, #map> return %0 : memref<20x42xf32, strided<[42, 1], offset: 1>>
return %0 : memref<20x42xf32, #map>
} }
// CHECK-LABEL: func @no_fold_subview_with_non_zero_offset( // CHECK-LABEL: func @no_fold_subview_with_non_zero_offset(
// CHECK: %[[SUBVIEW:.+]] = memref.subview // CHECK: %[[SUBVIEW:.+]] = memref.subview
// CHECK: return %[[SUBVIEW]] // CHECK: return %[[SUBVIEW]]
// ----- // -----
#map = affine_map<(d0, d1) -> (d0 * 42 + d1 * 2)> func.func @no_fold_subview_with_non_unit_stride(%arg0 : memref<20x42xf32>) -> memref<20x42xf32, strided<[42, 2]>> {
func.func @no_fold_subview_with_non_unit_stride(%arg0 : memref<20x42xf32>) -> memref<20x42xf32, #map> { %0 = memref.subview %arg0[0, 0] [20, 42] [1, 2] : memref<20x42xf32> to memref<20x42xf32, strided<[42, 2]>>
%0 = memref.subview %arg0[0, 0] [20, 42] [1, 2] : memref<20x42xf32> to memref<20x42xf32, #map> return %0 : memref<20x42xf32, strided<[42, 2]>>
return %0 : memref<20x42xf32, #map>
} }
// CHECK-LABEL: func @no_fold_subview_with_non_unit_stride( // CHECK-LABEL: func @no_fold_subview_with_non_unit_stride(
// CHECK: %[[SUBVIEW:.+]] = memref.subview // CHECK: %[[SUBVIEW:.+]] = memref.subview
// CHECK: return %[[SUBVIEW]] // CHECK: return %[[SUBVIEW]]
// ----- // -----
#map = affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + d1 + s0)> func.func @no_fold_dynamic_no_op_subview(%arg0 : memref<?x?xf32>) -> memref<?x?xf32, strided<[?, 1], offset: ?>> {
func.func @no_fold_dynamic_no_op_subview(%arg0 : memref<?x?xf32>) -> memref<?x?xf32, #map> {
%c0 = arith.constant 0 : index %c0 = arith.constant 0 : index
%c1 = arith.constant 1 : index %c1 = arith.constant 1 : index
%0 = memref.dim %arg0, %c0 : memref<?x?xf32> %0 = memref.dim %arg0, %c0 : memref<?x?xf32>
%1 = memref.dim %arg0, %c1 : memref<?x?xf32> %1 = memref.dim %arg0, %c1 : memref<?x?xf32>
%2 = memref.subview %arg0[0, 0] [%0, %1] [1, 1] : memref<?x?xf32> to memref<?x?xf32, #map> %2 = memref.subview %arg0[0, 0] [%0, %1] [1, 1] : memref<?x?xf32> to memref<?x?xf32, strided<[?, 1], offset: ?>>
return %2 : memref<?x?xf32, #map> return %2 : memref<?x?xf32, strided<[?, 1], offset: ?>>
} }
// CHECK-LABEL: func @no_fold_dynamic_no_op_subview( // CHECK-LABEL: func @no_fold_dynamic_no_op_subview(
// CHECK: %[[SUBVIEW:.+]] = memref.subview // CHECK: %[[SUBVIEW:.+]] = memref.subview
// CHECK: return %[[SUBVIEW]] // CHECK: return %[[SUBVIEW]]
// ----- // -----
func.func @atomicrmw_cast_fold(%arg0 : f32, %arg1 : memref<4xf32>, %c : index) { func.func @atomicrmw_cast_fold(%arg0 : f32, %arg1 : memref<4xf32>, %c : index) {
%v = memref.cast %arg1 : memref<4xf32> to memref<?xf32> %v = memref.cast %arg1 : memref<4xf32> to memref<?xf32>
%a = memref.atomic_rmw addf %arg0, %v[%c] : (f32, memref<?xf32>) -> f32 %a = memref.atomic_rmw addf %arg0, %v[%c] : (f32, memref<?xf32>) -> f32
return return
} }
// CHECK-LABEL: func @atomicrmw_cast_fold // CHECK-LABEL: func @atomicrmw_cast_fold
// CHECK-NEXT: memref.atomic_rmw addf %arg0, %arg1[%arg2] : (f32, memref<4xf32>) -> f32 // CHECK-NEXT: memref.atomic_rmw addf %arg0, %arg1[%arg2] : (f32, memref<4xf32>) -> f32
// ----- // -----
#map = affine_map<(d0)[s0, s1] -> (d0 * s1 + s0)>
func.func @copy_of_cast(%m1: memref<?xf32>, %m2: memref<*xf32>) { func.func @copy_of_cast(%m1: memref<?xf32>, %m2: memref<*xf32>) {
%casted1 = memref.cast %m1 : memref<?xf32> to memref<?xf32, #map> %casted1 = memref.cast %m1 : memref<?xf32> to memref<?xf32, strided<[?], offset: ?>>
%casted2 = memref.cast %m2 : memref<*xf32> to memref<?xf32, #map> %casted2 = memref.cast %m2 : memref<*xf32> to memref<?xf32, strided<[?], offset: ?>>
memref.copy %casted1, %casted2 : memref<?xf32, #map> to memref<?xf32, #map> memref.copy %casted1, %casted2 : memref<?xf32, strided<[?], offset: ?>> to memref<?xf32, strided<[?], offset: ?>>
return return
} }
// CHECK-LABEL: func @copy_of_cast( // CHECK-LABEL: func @copy_of_cast(
// CHECK-SAME: %[[m1:.*]]: memref<?xf32>, %[[m2:.*]]: memref<*xf32> // CHECK-SAME: %[[m1:.*]]: memref<?xf32>, %[[m2:.*]]: memref<*xf32>
// CHECK: %[[casted2:.*]] = memref.cast %[[m2]] // CHECK: %[[casted2:.*]] = memref.cast %[[m2]]
// CHECK: memref.copy %[[m1]], %[[casted2]] // CHECK: memref.copy %[[m1]], %[[casted2]]
▲ Show 20 Lines • Show All 248 Lines • Show Last 20 Lines

mlir/test/Dialect/MemRef/simplify-extract-strided-metadata.mlir

Show First 20 Lines • Show All 208 Lines • ▼ Show 20 Lines
// CHECK-DAG: %[[C128:.*]] = arith.constant 128 : index // CHECK-DAG: %[[C128:.*]] = arith.constant 128 : index
// CHECK-DAG: %[[C64:.*]] = arith.constant 64 : index // CHECK-DAG: %[[C64:.*]] = arith.constant 64 : index
// CHECK-DAG: %[[C1:.*]] = arith.constant 1 : index // CHECK-DAG: %[[C1:.*]] = arith.constant 1 : index
// CHECK-DAG: %[[BASE:.*]], %[[OFFSET:.*]], %[[SIZES:.*]]:2, %[[STRIDES:.*]]:2 = memref.extract_strided_metadata %[[ARG]] // CHECK-DAG: %[[BASE:.*]], %[[OFFSET:.*]], %[[SIZES:.*]]:2, %[[STRIDES:.*]]:2 = memref.extract_strided_metadata %[[ARG]]
// //
// CHECK-DAG: %[[FINAL_OFFSET:.*]] = affine.apply #[[$OFFSETS_MAP]]()[%[[DYN_OFFSET0]], %[[DYN_OFFSET1]]] // CHECK-DAG: %[[FINAL_OFFSET:.*]] = affine.apply #[[$OFFSETS_MAP]]()[%[[DYN_OFFSET0]], %[[DYN_OFFSET1]]]
// //
// CHECK: return %[[BASE]], %[[FINAL_OFFSET]], %[[C64]], %[[C64]], %[[C128]], %[[C1]] // CHECK: return %[[BASE]], %[[FINAL_OFFSET]], %[[C64]], %[[C64]], %[[C128]], %[[C1]]
#map0 = affine_map<(d0, d1)[s0] -> (d0 * 128 + s0 + d1)>
func.func @extract_strided_metadata_of_subview_w_variable_offset( func.func @extract_strided_metadata_of_subview_w_variable_offset(
%arg0: memref<384x128xf32>, %arg1 : index, %arg2 : index) %arg0: memref<384x128xf32>, %arg1 : index, %arg2 : index)
-> (memref<f32>, index, index, index, index, index) { -> (memref<f32>, index, index, index, index, index) {
%subview = memref.subview %arg0[%arg1, %arg2] [64, 64] [1, 1] : %subview = memref.subview %arg0[%arg1, %arg2] [64, 64] [1, 1] :
memref<384x128xf32> to memref<64x64xf32, #map0> memref<384x128xf32> to memref<64x64xf32, strided<[128, 1], offset: ?>>
%base_buffer, %offset, %sizes:2, %strides:2 = memref.extract_strided_metadata %subview : %base_buffer, %offset, %sizes:2, %strides:2 = memref.extract_strided_metadata %subview :
memref<64x64xf32, #map0> -> memref<f32>, index, index, index, index, index memref<64x64xf32, strided<[128, 1], offset: ?>> -> memref<f32>, index, index, index, index, index
return %base_buffer, %offset, %sizes#0, %sizes#1, %strides#0, %strides#1 : return %base_buffer, %offset, %sizes#0, %sizes#1, %strides#0, %strides#1 :
memref<f32>, index, index, index, index, index memref<f32>, index, index, index, index, index
} }
// ----- // -----
// Check that all the math is correct for all types of computations. // Check that all the math is correct for all types of computations.
▲ Show 20 Lines • Show All 49 Lines • Show Last 20 Lines

mlir/test/Dialect/Vector/vector-transfer-collapse-inner-most-dims.mlir

// RUN: mlir-opt %s -test-vector-transfer-collapse-inner-most-dims -split-input-file | FileCheck %s // RUN: mlir-opt %s -test-vector-transfer-collapse-inner-most-dims -split-input-file | FileCheck %s
#map1 = affine_map<(d0, d1, d2, d3)[s0] -> (d0 * 3072 + s0 + d1 * 8 + d2 + d3)> func.func @contiguous_inner_most_view(%in: memref<1x1x8x1xf32, strided<[3072, 8, 1, 1], offset: ?>>) -> vector<1x8x1xf32>{
func.func @contiguous_inner_most_view(%in: memref<1x1x8x1xf32, #map1>) -> vector<1x8x1xf32>{
%c0 = arith.constant 0 : index %c0 = arith.constant 0 : index
%cst = arith.constant 0.0 : f32 %cst = arith.constant 0.0 : f32
%0 = vector.transfer_read %in[%c0, %c0, %c0, %c0], %cst {in_bounds = [true, true, true]} : memref<1x1x8x1xf32, #map1>, vector<1x8x1xf32> %0 = vector.transfer_read %in[%c0, %c0, %c0, %c0], %cst {in_bounds = [true, true, true]} : memref<1x1x8x1xf32, strided<[3072, 8, 1, 1], offset: ?>>, vector<1x8x1xf32>
return %0 : vector<1x8x1xf32> return %0 : vector<1x8x1xf32>
} }
// CHECK-DAG: #[[MAP0:.+]] = affine_map<(d0, d1, d2, d3)[s0] -> (d0 * 3072 + s0 + d1 * 8 + d2 + d3)> // CHECK: func @contiguous_inner_most_view(%[[SRC:.+]]: memref<1x1x8x1xf32, strided<[3072, 8, 1, 1], offset: ?>>
// CHECK-DAG: #[[MAP1:.+]] = affine_map<(d0, d1, d2)[s0] -> (d0 * 3072 + s0 + d1 * 8 + d2)>
// CHECK: func @contiguous_inner_most_view(%[[SRC:.+]]: memref<1x1x8x1xf32, #[[MAP0]]>
// CHECK: %[[SRC_0:.+]] = memref.subview %[[SRC]] // CHECK: %[[SRC_0:.+]] = memref.subview %[[SRC]]
// CHECK-SAME: memref<1x1x8x1xf32, #[[MAP0]]> to memref<1x1x8xf32, #[[MAP1]]> // CHECK-SAME: memref<1x1x8x1xf32, strided<[3072, 8, 1, 1], offset: ?>> to memref<1x1x8xf32
// CHECK: %[[VEC:.+]] = vector.transfer_read %[[SRC_0]] // CHECK: %[[VEC:.+]] = vector.transfer_read %[[SRC_0]]
// CHECK-SAME: memref<1x1x8xf32, #[[MAP1]]>, vector<1x8xf32> // CHECK-SAME: memref<1x1x8xf32, {{.*}}>, vector<1x8xf32>
// CHECK: %[[RESULT:.+]] = vector.shape_cast %[[VEC]] // CHECK: %[[RESULT:.+]] = vector.shape_cast %[[VEC]]
// CHECK: return %[[RESULT]] // CHECK: return %[[RESULT]]
// ----- // -----
func.func @contiguous_inner_most_dim(%A: memref<16x1xf32>, %i:index, %j:index) -> (vector<8x1xf32>) { func.func @contiguous_inner_most_dim(%A: memref<16x1xf32>, %i:index, %j:index) -> (vector<8x1xf32>) {
%c0 = arith.constant 0 : index %c0 = arith.constant 0 : index
%f0 = arith.constant 0.0 : f32 %f0 = arith.constant 0.0 : f32
▲ Show 20 Lines • Show All 41 Lines • Show Last 20 Lines

mlir/test/Dialect/Vector/vector-transfer-flatten.mlir

Show First 20 Lines • Show All 56 Lines • ▼ Show 20 Lines
// CHECK-LABEL: func @transfer_read_0d // CHECK-LABEL: func @transfer_read_0d
// CHECK-SAME: %[[ARG:.+]]: memref<i8> // CHECK-SAME: %[[ARG:.+]]: memref<i8>
// CHECK: %[[CST:.+]] = arith.constant 0 : i8 // CHECK: %[[CST:.+]] = arith.constant 0 : i8
// CHECK: %[[READ:.+]] = vector.transfer_read %[[ARG]][], %[[CST]] : memref<i8> // CHECK: %[[READ:.+]] = vector.transfer_read %[[ARG]][], %[[CST]] : memref<i8>
// CHECK: return %[[READ]] // CHECK: return %[[READ]]
// ----- // -----
#map0 = affine_map<(d0, d1, d2, d3)[s0, s1] -> (d0 * s1 + s0 + d1 * 32 + d2 * 4 + d3)> func.func @transfer_read_flattenable_with_dynamic_dims_and_indices(%arg0 : memref<?x?x8x4xi8, strided<[?, 32, 4, 1], offset: ?>>, %arg1 : index, %arg2 : index) -> vector<8x4xi8> {
func.func @transfer_read_flattenable_with_dynamic_dims_and_indices(%arg0 : memref<?x?x8x4xi8, #map0>, %arg1 : index, %arg2 : index) -> vector<8x4xi8> {
%c0_i8 = arith.constant 0 : i8 %c0_i8 = arith.constant 0 : i8
%c0 = arith.constant 0 : index %c0 = arith.constant 0 : index
%result = vector.transfer_read %arg0[%arg1, %arg2, %c0, %c0], %c0_i8 {in_bounds = [true, true]} : memref<?x?x8x4xi8, #map0>, vector<8x4xi8> %result = vector.transfer_read %arg0[%arg1, %arg2, %c0, %c0], %c0_i8 {in_bounds = [true, true]} : memref<?x?x8x4xi8, strided<[?, 32, 4, 1], offset: ?>>, vector<8x4xi8>
return %result : vector<8x4xi8> return %result : vector<8x4xi8>
} }
// CHECK-LABEL: func @transfer_read_flattenable_with_dynamic_dims_and_indices // CHECK-LABEL: func @transfer_read_flattenable_with_dynamic_dims_and_indices
// CHECK-SAME: %[[ARG0:.+]]: memref<?x?x8x4xi8, {{.+}}>, %[[ARG1:.+]]: index, %[[ARG2:.+]]: index // CHECK-SAME: %[[ARG0:.+]]: memref<?x?x8x4xi8, {{.+}}>, %[[ARG1:.+]]: index, %[[ARG2:.+]]: index
// CHECK: %[[C0_I8:.+]] = arith.constant 0 : i8 // CHECK: %[[C0_I8:.+]] = arith.constant 0 : i8
// CHECK: %[[C0:.+]] = arith.constant 0 : index // CHECK: %[[C0:.+]] = arith.constant 0 : index
// CHECK: %[[COLLAPSED:.+]] = memref.collapse_shape %[[ARG0]] {{\[}}[0], [1], [2, 3]{{\]}} // CHECK: %[[COLLAPSED:.+]] = memref.collapse_shape %[[ARG0]] {{\[}}[0], [1], [2, 3]{{\]}}
// CHECK-SAME: : memref<?x?x8x4xi8, {{.+}}> into memref<?x?x32xi8, {{.+}}> // CHECK-SAME: : memref<?x?x8x4xi8, {{.+}}> into memref<?x?x32xi8, {{.+}}>
// CHECK: %[[VEC1D:.+]] = vector.transfer_read %[[COLLAPSED]] // CHECK: %[[VEC1D:.+]] = vector.transfer_read %[[COLLAPSED]]
// CHECK-SAME: [%[[ARG1]], %[[ARG2]], %[[C0]]], %[[C0_I8]] // CHECK-SAME: [%[[ARG1]], %[[ARG2]], %[[C0]]], %[[C0_I8]]
// CHECK-SAME: {in_bounds = [true]} // CHECK-SAME: {in_bounds = [true]}
// CHECK-SAME: : memref<?x?x32xi8, {{.+}}>, vector<32xi8> // CHECK-SAME: : memref<?x?x32xi8, {{.+}}>, vector<32xi8>
// CHECK: %[[VEC2D:.+]] = vector.shape_cast %[[VEC1D]] : vector<32xi8> to vector<8x4xi8> // CHECK: %[[VEC2D:.+]] = vector.shape_cast %[[VEC1D]] : vector<32xi8> to vector<8x4xi8>
// CHECK: return %[[VEC2D]] : vector<8x4xi8> // CHECK: return %[[VEC2D]] : vector<8x4xi8>
// ----- // -----
#map0 = affine_map<(d0, d1, d2, d3)[s0, s1] -> (d0 * s1 + s0 + d1 * 32 + d2 * 4 + d3)> func.func @transfer_write_flattenable_with_dynamic_dims_and_indices(%vec : vector<8x4xi8>, %dst : memref<?x?x8x4xi8, strided<[?, 32, 4, 1], offset: ?>>, %arg1 : index, %arg2 : index) {
func.func @transfer_write_flattenable_with_dynamic_dims_and_indices(%vec : vector<8x4xi8>, %dst : memref<?x?x8x4xi8, #map0>, %arg1 : index, %arg2 : index) {
%c0 = arith.constant 0 : index %c0 = arith.constant 0 : index
vector.transfer_write %vec, %dst[%arg1, %arg2, %c0, %c0] {in_bounds = [true, true]} : vector<8x4xi8>, memref<?x?x8x4xi8, #map0> vector.transfer_write %vec, %dst[%arg1, %arg2, %c0, %c0] {in_bounds = [true, true]} : vector<8x4xi8>, memref<?x?x8x4xi8, strided<[?, 32, 4, 1], offset: ?>>
return return
} }
// CHECK-LABEL: func @transfer_write_flattenable_with_dynamic_dims_and_indices // CHECK-LABEL: func @transfer_write_flattenable_with_dynamic_dims_and_indices
// CHECK-SAME: %[[ARG0:.+]]: vector<8x4xi8>, %[[ARG1:.+]]: memref<?x?x8x4xi8, {{.+}}>, %[[ARG2:.+]]: index, %[[ARG3:.+]]: index // CHECK-SAME: %[[ARG0:.+]]: vector<8x4xi8>, %[[ARG1:.+]]: memref<?x?x8x4xi8, {{.+}}>, %[[ARG2:.+]]: index, %[[ARG3:.+]]: index
// CHECK: %[[C0:.+]] = arith.constant 0 : index // CHECK: %[[C0:.+]] = arith.constant 0 : index
// CHECK: %[[COLLAPSED:.+]] = memref.collapse_shape %[[ARG1]] {{\[}}[0], [1], [2, 3]{{\]}} // CHECK: %[[COLLAPSED:.+]] = memref.collapse_shape %[[ARG1]] {{\[}}[0], [1], [2, 3]{{\]}}
// CHECK-SAME: : memref<?x?x8x4xi8, {{.+}}> into memref<?x?x32xi8, {{.+}}> // CHECK-SAME: : memref<?x?x8x4xi8, {{.+}}> into memref<?x?x32xi8, {{.+}}>
Show All 33 Lines

mlir/test/Dialect/Vector/vector-warp-distribute.mlir

Show First 20 Lines • Show All 79 Lines • ▼ Show 20 Lines
// CHECK-DIST-AND-PROP: vector.transfer_read {{.*}} vector<1xf32> // CHECK-DIST-AND-PROP: vector.transfer_read {{.*}} vector<1xf32>
// CHECK-DIST-AND-PROP: vector.transfer_read {{.*}} vector<2xf32> // CHECK-DIST-AND-PROP: vector.transfer_read {{.*}} vector<2xf32>
// CHECK-DIST-AND-PROP: vector.transfer_read {{.*}} vector<2xf32> // CHECK-DIST-AND-PROP: vector.transfer_read {{.*}} vector<2xf32>
// CHECK-DIST-AND-PROP: arith.addf {{.*}} : vector<1xf32> // CHECK-DIST-AND-PROP: arith.addf {{.*}} : vector<1xf32>
// CHECK-DIST-AND-PROP: arith.addf {{.*}} : vector<2xf32> // CHECK-DIST-AND-PROP: arith.addf {{.*}} : vector<2xf32>
// CHECK-DIST-AND-PROP: vector.transfer_write {{.*}} : vector<1xf32> // CHECK-DIST-AND-PROP: vector.transfer_write {{.*}} : vector<1xf32>
// CHECK-DIST-AND-PROP: vector.transfer_write {{.*}} : vector<2xf32> // CHECK-DIST-AND-PROP: vector.transfer_write {{.*}} : vector<2xf32>
#map0 = affine_map<(d0)[s0] -> (d0 + s0)>
func.func @warp(%laneid: index, %arg1: memref<1024xf32>, %arg2: memref<1024xf32>, func.func @warp(%laneid: index, %arg1: memref<1024xf32>, %arg2: memref<1024xf32>,
%arg3: memref<1024xf32>, %gid : index) { %arg3: memref<1024xf32>, %gid : index) {
vector.warp_execute_on_lane_0(%laneid)[32] { vector.warp_execute_on_lane_0(%laneid)[32] {
%sa = memref.subview %arg1[%gid] [128] [1] : memref<1024xf32> to memref<128xf32, #map0> %sa = memref.subview %arg1[%gid] [128] [1] : memref<1024xf32> to memref<128xf32, strided<[1], offset: ?>>
%sb = memref.subview %arg2[%gid] [128] [1] : memref<1024xf32> to memref<128xf32, #map0> %sb = memref.subview %arg2[%gid] [128] [1] : memref<1024xf32> to memref<128xf32, strided<[1], offset: ?>>
%sc = memref.subview %arg3[%gid] [128] [1] : memref<1024xf32> to memref<128xf32, #map0> %sc = memref.subview %arg3[%gid] [128] [1] : memref<1024xf32> to memref<128xf32, strided<[1], offset: ?>>
%c0 = arith.constant 0 : index %c0 = arith.constant 0 : index
%c32 = arith.constant 32 : index %c32 = arith.constant 32 : index
%cst = arith.constant 0.000000e+00 : f32 %cst = arith.constant 0.000000e+00 : f32
%2 = vector.transfer_read %sa[%c0], %cst : memref<128xf32, #map0>, vector<32xf32> %2 = vector.transfer_read %sa[%c0], %cst : memref<128xf32, strided<[1], offset: ?>>, vector<32xf32>
%3 = vector.transfer_read %sa[%c32], %cst : memref<128xf32, #map0>, vector<32xf32> %3 = vector.transfer_read %sa[%c32], %cst : memref<128xf32, strided<[1], offset: ?>>, vector<32xf32>
%4 = vector.transfer_read %sb[%c0], %cst : memref<128xf32, #map0>, vector<64xf32> %4 = vector.transfer_read %sb[%c0], %cst : memref<128xf32, strided<[1], offset: ?>>, vector<64xf32>
%5 = vector.transfer_read %sb[%c32], %cst : memref<128xf32, #map0>, vector<64xf32> %5 = vector.transfer_read %sb[%c32], %cst : memref<128xf32, strided<[1], offset: ?>>, vector<64xf32>
%6 = arith.addf %2, %3 : vector<32xf32> %6 = arith.addf %2, %3 : vector<32xf32>
%7 = arith.addf %4, %5 : vector<64xf32> %7 = arith.addf %4, %5 : vector<64xf32>
vector.transfer_write %6, %sc[%c0] : vector<32xf32>, memref<128xf32, #map0> vector.transfer_write %6, %sc[%c0] : vector<32xf32>, memref<128xf32, strided<[1], offset: ?>>
vector.transfer_write %7, %sc[%c32] : vector<64xf32>, memref<128xf32, #map0> vector.transfer_write %7, %sc[%c32] : vector<64xf32>, memref<128xf32, strided<[1], offset: ?>>
} }
return return
} }
// ----- // -----
// CHECK-D-LABEL: func @warp_extract( // CHECK-D-LABEL: func @warp_extract(
// CHECK-D: %[[WARPOP:.*]]:2 = vector.warp_execute_on_lane_0(%{{.*}})[32] -> (vector<1xf32>, vector<1x1xf32>) // CHECK-D: %[[WARPOP:.*]]:2 = vector.warp_execute_on_lane_0(%{{.*}})[32] -> (vector<1xf32>, vector<1x1xf32>)
▲ Show 20 Lines • Show All 665 Lines • Show Last 20 Lines

mlir/test/Integration/Dialect/Vector/CPU/test-transfer-read-1d.mlir

Show All 31 Linesfunc.func @transfer_read_1d(%A : memref<?x?xf32>, %base1 : index, %base2 : index) {
%fm42 = arith.constant -42.0: f32 %fm42 = arith.constant -42.0: f32
%f = vector.transfer_read %A[%base1, %base2], %fm42 %f = vector.transfer_read %A[%base1, %base2], %fm42
{permutation_map = affine_map<(d0, d1) -> (d0)>} {permutation_map = affine_map<(d0, d1) -> (d0)>}
: memref<?x?xf32>, vector<9xf32> : memref<?x?xf32>, vector<9xf32>
vector.print %f: vector<9xf32> vector.print %f: vector<9xf32>
return return
} }
#map0 = affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>
#map1 = affine_map<(d0, d1) -> (6 * d0 + 2 * d1)>
// Vector load with unit stride only on last dim. // Vector load with unit stride only on last dim.
func.func @transfer_read_1d_unit_stride(%A : memref<?x?xf32>) { func.func @transfer_read_1d_unit_stride(%A : memref<?x?xf32>) {
%c0 = arith.constant 0 : index %c0 = arith.constant 0 : index
%c1 = arith.constant 1 : index %c1 = arith.constant 1 : index
%c2 = arith.constant 2 : index %c2 = arith.constant 2 : index
%c3 = arith.constant 3 : index %c3 = arith.constant 3 : index
%c4 = arith.constant 4 : index %c4 = arith.constant 4 : index
%c5 = arith.constant 5 : index %c5 = arith.constant 5 : index
%c6 = arith.constant 6 : index %c6 = arith.constant 6 : index
%fm42 = arith.constant -42.0: f32 %fm42 = arith.constant -42.0: f32
scf.for %arg2 = %c1 to %c5 step %c2 { scf.for %arg2 = %c1 to %c5 step %c2 {
scf.for %arg3 = %c0 to %c6 step %c3 { scf.for %arg3 = %c0 to %c6 step %c3 {
%0 = memref.subview %A[%arg2, %arg3] [1, 2] [1, 1] %0 = memref.subview %A[%arg2, %arg3] [1, 2] [1, 1]
: memref<?x?xf32> to memref<1x2xf32, #map0> : memref<?x?xf32> to memref<1x2xf32, strided<[?, 1], offset: ?>>
%1 = vector.transfer_read %0[%c0, %c0], %fm42 {in_bounds=[true]} %1 = vector.transfer_read %0[%c0, %c0], %fm42 {in_bounds=[true]}
: memref<1x2xf32, #map0>, vector<2xf32> : memref<1x2xf32, strided<[?, 1], offset: ?>>, vector<2xf32>
vector.print %1 : vector<2xf32> vector.print %1 : vector<2xf32>
} }
} }
return return
} }
// Vector load with unit stride only on last dim. Strides are not static, so // Vector load with unit stride only on last dim. Strides are not static, so
// codegen must go through VectorToSCF 1D lowering. // codegen must go through VectorToSCF 1D lowering.
func.func @transfer_read_1d_non_static_unit_stride(%A : memref<?x?xf32>) { func.func @transfer_read_1d_non_static_unit_stride(%A : memref<?x?xf32>) {
%c1 = arith.constant 1 : index %c1 = arith.constant 1 : index
%c2 = arith.constant 2 : index %c2 = arith.constant 2 : index
%c6 = arith.constant 6 : index %c6 = arith.constant 6 : index
%fm42 = arith.constant -42.0: f32 %fm42 = arith.constant -42.0: f32
%1 = memref.reinterpret_cast %A to offset: [%c6], sizes: [%c1, %c2], strides: [%c6, %c1] %1 = memref.reinterpret_cast %A to offset: [%c6], sizes: [%c1, %c2], strides: [%c6, %c1]
: memref<?x?xf32> to memref<?x?xf32, strided<[?, ?], offset: ?>> : memref<?x?xf32> to memref<?x?xf32, strided<[?, ?], offset: ?>>
%2 = vector.transfer_read %1[%c2, %c1], %fm42 {in_bounds=[true]} %2 = vector.transfer_read %1[%c2, %c1], %fm42 {in_bounds=[true]}
: memref<?x?xf32, strided<[?, ?], offset: ?>>, vector<4xf32> : memref<?x?xf32, strided<[?, ?], offset: ?>>, vector<4xf32>
vector.print %2 : vector<4xf32> vector.print %2 : vector<4xf32>
return return
} }
// Vector load where last dim has non-unit stride. // Vector load where last dim has non-unit stride.
func.func @transfer_read_1d_non_unit_stride(%A : memref<?x?xf32>) { func.func @transfer_read_1d_non_unit_stride(%A : memref<?x?xf32>) {
%B = memref.reinterpret_cast %A to offset: [0], sizes: [4, 3], strides: [6, 2] %B = memref.reinterpret_cast %A to offset: [0], sizes: [4, 3], strides: [6, 2]
: memref<?x?xf32> to memref<4x3xf32, #map1> : memref<?x?xf32> to memref<4x3xf32, strided<[6, 2]>>
%c1 = arith.constant 1 : index %c1 = arith.constant 1 : index
%c2 = arith.constant 2 : index %c2 = arith.constant 2 : index
%fm42 = arith.constant -42.0: f32 %fm42 = arith.constant -42.0: f32
%vec = vector.transfer_read %B[%c2, %c1], %fm42 {in_bounds=[false]} : memref<4x3xf32, #map1>, vector<3xf32> %vec = vector.transfer_read %B[%c2, %c1], %fm42 {in_bounds=[false]} : memref<4x3xf32, strided<[6, 2]>>, vector<3xf32>
vector.print %vec : vector<3xf32> vector.print %vec : vector<3xf32>
return return
} }
// Broadcast. // Broadcast.
func.func @transfer_read_1d_broadcast( func.func @transfer_read_1d_broadcast(
%A : memref<?x?xf32>, %base1 : index, %base2 : index) { %A : memref<?x?xf32>, %base1 : index, %base2 : index) {
%fm42 = arith.constant -42.0: f32 %fm42 = arith.constant -42.0: f32
▲ Show 20 Lines • Show All 133 Lines • Show Last 20 Lines

mlir/test/Transforms/canonicalize.mlir

Show First 20 Lines • Show All 468 Lines • ▼ Show 20 Lines affine.for %j = 0 to 10 {
%v = memref.load %a[%i, %j] : memref<?x?xf32> %v = memref.load %a[%i, %j] : memref<?x?xf32>
memref.store %v, %b[%zero, %zero, %i, %j, %zero] : memref<4x?x8x?x?xf32> memref.store %v, %b[%zero, %zero, %i, %j, %zero] : memref<4x?x8x?x?xf32>
} }
} }
return %b, %c, %d, %e : memref<4 x ? x 8 x ? x ? x f32>, memref<? x ? x i32>, memref<? x ? x f32>, memref<4 x ? x 8 x ? x ? x f32> return %b, %c, %d, %e : memref<4 x ? x 8 x ? x ? x f32>, memref<? x ? x i32>, memref<? x ? x f32>, memref<4 x ? x 8 x ? x ? x f32>
} }
#map1 = affine_map<(d0, d1)[s0, s1, s2] -> (d0 * s1 + s0 + d1 * s2)>
#map2 = affine_map<(d0, d1, d2)[s0, s1, s2] -> (d0 * s2 + d1 * s1 + d2 + s0)>
#map3 = affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>
// CHECK-LABEL: func @dim_op_fold( // CHECK-LABEL: func @dim_op_fold(
// CHECK-SAME: %[[ARG0:[a-z0-9]*]]: index // CHECK-SAME: %[[ARG0:[a-z0-9]*]]: index
// CHECK-SAME: %[[ARG1:[a-z0-9]*]]: index // CHECK-SAME: %[[ARG1:[a-z0-9]*]]: index
// CHECK-SAME: %[[ARG2:[a-z0-9]*]]: index // CHECK-SAME: %[[ARG2:[a-z0-9]*]]: index
// CHECK-SAME: %[[BUF:[a-z0-9]*]]: memref<?xi8> // CHECK-SAME: %[[BUF:[a-z0-9]*]]: memref<?xi8>
func.func @dim_op_fold(%arg0: index, %arg1: index, %arg2: index, %BUF: memref<?xi8>, %M : index, %N : index, %K : index) { func.func @dim_op_fold(%arg0: index, %arg1: index, %arg2: index, %BUF: memref<?xi8>, %M : index, %N : index, %K : index) {
// CHECK-SAME: [[M:arg[0-9]+]]: index // CHECK-SAME: [[M:arg[0-9]+]]: index
// CHECK-SAME: [[N:arg[0-9]+]]: index // CHECK-SAME: [[N:arg[0-9]+]]: index
// CHECK-SAME: [[K:arg[0-9]+]]: index // CHECK-SAME: [[K:arg[0-9]+]]: index
%c0 = arith.constant 0 : index %c0 = arith.constant 0 : index
%c1 = arith.constant 1 : index %c1 = arith.constant 1 : index
%c2 = arith.constant 2 : index %c2 = arith.constant 2 : index
%0 = memref.alloc(%arg0, %arg1) : memref<?x?xf32> %0 = memref.alloc(%arg0, %arg1) : memref<?x?xf32>
%1 = memref.alloc(%arg1, %arg2) : memref<?x8x?xf32> %1 = memref.alloc(%arg1, %arg2) : memref<?x8x?xf32>
%2 = memref.dim %1, %c2 : memref<?x8x?xf32> %2 = memref.dim %1, %c2 : memref<?x8x?xf32>
affine.for %arg3 = 0 to %2 { affine.for %arg3 = 0 to %2 {
%3 = memref.alloc(%arg0) : memref<?xi8> %3 = memref.alloc(%arg0) : memref<?xi8>
%ub = memref.dim %3, %c0 : memref<?xi8> %ub = memref.dim %3, %c0 : memref<?xi8>
affine.for %arg4 = 0 to %ub { affine.for %arg4 = 0 to %ub {
%s = memref.dim %0, %c0 : memref<?x?xf32> %s = memref.dim %0, %c0 : memref<?x?xf32>
%v = memref.view %3[%c0][%arg4, %s] : memref<?xi8> to memref<?x?xf32> %v = memref.view %3[%c0][%arg4, %s] : memref<?xi8> to memref<?x?xf32>
%sv = memref.subview %0[%c0, %c0][%s,%arg4][%c1,%c1] : memref<?x?xf32> to memref<?x?xf32, #map1> %sv = memref.subview %0[%c0, %c0][%s,%arg4][%c1,%c1] : memref<?x?xf32> to memref<?x?xf32, strided<[?, ?], offset: ?>>
%l = memref.dim %v, %c1 : memref<?x?xf32> %l = memref.dim %v, %c1 : memref<?x?xf32>
%u = memref.dim %sv, %c0 : memref<?x?xf32, #map1> %u = memref.dim %sv, %c0 : memref<?x?xf32, strided<[?, ?], offset: ?>>
affine.for %arg5 = %l to %u { affine.for %arg5 = %l to %u {
"foo"() : () -> () "foo"() : () -> ()
} }
%sv2 = memref.subview %0[0, 0][17, %arg4][1, 1] : memref<?x?xf32> to memref<17x?xf32, #map3> %sv2 = memref.subview %0[0, 0][17, %arg4][1, 1] : memref<?x?xf32> to memref<17x?xf32, strided<[?, 1], offset: ?>>
%l2 = memref.dim %v, %c1 : memref<?x?xf32> %l2 = memref.dim %v, %c1 : memref<?x?xf32>
%u2 = memref.dim %sv2, %c1 : memref<17x?xf32, #map3> %u2 = memref.dim %sv2, %c1 : memref<17x?xf32, strided<[?, 1], offset: ?>>
scf.for %arg5 = %l2 to %u2 step %c1 { scf.for %arg5 = %l2 to %u2 step %c1 {
"foo"() : () -> () "foo"() : () -> ()
} }
} }
} }
// CHECK: affine.for %[[I:.*]] = 0 to %[[ARG2]] { // CHECK: affine.for %[[I:.*]] = 0 to %[[ARG2]] {
// CHECK-NEXT: affine.for %[[J:.*]] = 0 to %[[ARG0]] { // CHECK-NEXT: affine.for %[[J:.*]] = 0 to %[[ARG0]] {
// CHECK-NEXT: affine.for %[[K:.*]] = %[[ARG0]] to %[[ARG0]] { // CHECK-NEXT: affine.for %[[K:.*]] = %[[ARG0]] to %[[ARG0]] {
▲ Show 20 Lines • Show All 717 Lines • Show Last 20 Lines