This is an archive of the discontinued LLVM Phabricator instance.

Differential D104356

[mlir][linalg] Change the pretty printed FillOp operand order.
ClosedPublic

Authored by gysit on Jun 15 2021, 11:31 PM.

Details

Reviewers
aartbik
sjarus
nicolasvasilache
pifon2a
Commits
rGa21a6f51bcd5: [mlir][linalg] Change the pretty printed FillOp operand order.
Summary

The patch changes the pretty printed FillOp operand order from output, value to value, output. The change is a follow up to https://reviews.llvm.org/D104121 that passes the fill value using a scalar input instead of the former capture semantics.

Diff Detail

Event Timeline

gysit created this revision.Jun 15 2021, 11:31 PM
Herald added a reviewer: aartbik. · View Herald TranscriptJun 15 2021, 11:31 PM
Herald added a reviewer: sjarus. · View Herald Transcript
Herald added subscribers: armkevincheng, shabalin, jsmolens and 21 others. · View Herald Transcript
gysit requested review of this revision.Jun 15 2021, 11:31 PM
Herald added a reviewer: nicolasvasilache. · View Herald TranscriptJun 15 2021, 11:31 PM
Herald added a project: Restricted Project. · View Herald Transcript
Herald added subscribers: limo1996, stephenneuendorffer, nicolasvasilache. · View Herald Transcript
Harbormaster completed remote builds in B109461: Diff 352350.Jun 15 2021, 11:31 PM
gysit mentioned this in D104359: [mlir][linalg] Adapt the FillOp builder signature..Jun 16 2021, 12:09 AM
gysit added a reviewer: pifon2a.Jun 17 2021, 11:14 PM
nicolasvasilache accepted this revision.Jun 21 2021, 12:16 AM
This revision is now accepted and ready to land.Jun 21 2021, 12:16 AM
gysit updated this revision to Diff 353852.Jun 22 2021, 11:26 PM

Rebase.

Harbormaster completed remote builds in B110549: Diff 353852.Jun 22 2021, 11:57 PM
This revision was landed with ongoing or failed builds.Jun 23 2021, 12:06 AM
Closed by commit rGa21a6f51bcd5: [mlir][linalg] Change the pretty printed FillOp operand order. (authored by gysit). · Explain Why
This revision was automatically updated to reflect the committed changes.
gysit added a commit: rGa21a6f51bcd5: [mlir][linalg] Change the pretty printed FillOp operand order..
gysit mentioned this in rG7cef24ee83cd: [mlir][linalg] Adapt the FillOp builder signature..Jun 23 2021, 1:07 AM

Revision Contents

PathSize
mlir/
include/
mlir/
Dialect/
Linalg/
IR/
2 lines
6 lines
lib/
Dialect/
Linalg/
IR/
4 lines
Transforms/
2 lines
Vector/
2 lines
test/
Conversion/
TosaToLinalg/
30 lines
Dialect/
Linalg/
4 lines
16 lines
6 lines
24 lines
16 lines
6 lines
18 lines
16 lines
6 lines
2 lines
2 lines
8 lines
6 lines
16 lines
4 lines
16 lines
10 lines
12 lines
4 lines
16 lines
6 lines
Vector/
4 lines
Integration/
Dialect/
Async/
CPU/
4 lines
Linalg/
CPU/
12 lines
12 lines
12 lines
8 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
mlir-cpu-runner/
2 lines
6 lines
10 lines
4 lines
python/
dialects/
linalg/
4 lines
6 lines

Diff 353858

mlir/include/mlir/Dialect/Linalg/IR/LinalgOps.h

Show First 20 Lines • Show All 66 Lines • ▼ Show 20 Lines
/// type names: /// type names:
/// 1. form a string which is the concatenation of the linalg op name with all /// 1. form a string which is the concatenation of the linalg op name with all
/// the operand type names, separate by underscores; /// the operand type names, separate by underscores;
/// 2. drop the `linalg.` prefix, and the `<`, `>`, `?` symbols from the type. /// 2. drop the `linalg.` prefix, and the `<`, `>`, `?` symbols from the type.
/// Assumes `op` is a LinalgOp. /// Assumes `op` is a LinalgOp.
/// ///
/// Examples: /// Examples:
/// ///
/// 1. linalg.fill(%A, %f) : memref<f32>, f32 /// 1. linalg.fill(%f, %A) : f32, memref<f32>
/// name mangles into `linalg_fill_viewf32_f32_impl` /// name mangles into `linalg_fill_viewf32_f32_impl`
/// ///
/// 2. linalg.dot %A, %B, %C : /// 2. linalg.dot %A, %B, %C :
/// (memref<?xf32, stride_specification>, /// (memref<?xf32, stride_specification>,
/// memref<?xf32, stride_specification>, memref<f32>) /// memref<?xf32, stride_specification>, memref<f32>)
/// name mangles into `linalg_dot_viewxf32_viewxf32_viewf32_impl` /// name mangles into `linalg_dot_viewxf32_viewxf32_viewf32_impl`
/// ///
/// 3. linalg.matmul(...) : /// 3. linalg.matmul(...) :
▲ Show 20 Lines • Show All 48 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Linalg/IR/LinalgStructuredOps.td

Show First 20 Lines • Show All 205 Lines • ▼ Show 20 Lines static std::function<
void(ImplicitLocOpBuilder &b, Block &block, ValueRange captures)> void(ImplicitLocOpBuilder &b, Block &block, ValueRange captures)>
getRegionBuilder() { getRegionBuilder() {
return &regionBuilder; return &regionBuilder;
} }
static unsigned getNumRegionArgs() { return 2; } static unsigned getNumRegionArgs() { return 2; }
}]; }];
let assemblyFormat = [{ let assemblyFormat = [{
`(` $output `,` $value `)` attr-dict `:` `(` $value `,` $output `)` attr-dict `:`
type($output) `,` type($value) (`->` type($result)^)? type($value) `,` type($output) (`->` type($result)^)?
custom<FillOpRegion>($region, ref(type($output)), ref(type($value))) custom<FillOpRegion>($region, ref(type($value)), ref(type($output)))
}]; }];
let builders = [ let builders = [
OpBuilder<(ins "Value":$output, "Value":$value)> OpBuilder<(ins "Value":$output, "Value":$value)>
]; ];
let verifier = [{ return ::verify(*this); }]; let verifier = [{ return ::verify(*this); }];
▲ Show 20 Lines • Show All 453 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/IR/LinalgOps.cpp

Show First 20 Lines • Show All 429 Lines • ▼ Show 20 Linesvoid FillOp::build(OpBuilder &builder, OperationState &result, Value output,
Value value) { Value value) {
build(builder, result, output.getType().dyn_cast<RankedTensorType>(), value, build(builder, result, output.getType().dyn_cast<RankedTensorType>(), value,
output); output);
fillStructuredOpRegion<FillOp>(builder, *result.regions.front(), fillStructuredOpRegion<FillOp>(builder, *result.regions.front(),
TypeRange{value.getType()}, TypeRange{value.getType()},
TypeRange{output.getType()}, {}); TypeRange{output.getType()}, {});
} }
ParseResult parseFillOpRegion(OpAsmParser &parser, Region &r, Type outputType, ParseResult parseFillOpRegion(OpAsmParser &parser, Region &r, Type valueType,
Type valueType) { Type outputType) {
OpBuilder opBuilder(parser.getBuilder().getContext()); OpBuilder opBuilder(parser.getBuilder().getContext());
fillStructuredOpRegion<FillOp>(opBuilder, r, TypeRange{valueType}, fillStructuredOpRegion<FillOp>(opBuilder, r, TypeRange{valueType},
TypeRange{outputType}); TypeRange{outputType});
return success(); return success();
} }
/// FillOp region is elided when printing. /// FillOp region is elided when printing.
void printFillOpRegion(OpAsmPrinter &, Operation *, Region &, Type, Type) {} void printFillOpRegion(OpAsmPrinter &, Operation *, Region &, Type, Type) {}
▲ Show 20 Lines • Show All 2,825 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/Transforms/Fusion.cpp

Show First 20 Lines • Show All 642 Lines • ▼ Show 20 Lines
/// the fused subview) has update semantics. /// the fused subview) has update semantics.
/// ///
/// Since an inverse computation is needed, we need to consider the projection /// Since an inverse computation is needed, we need to consider the projection
/// of the producerIndexMap w.r.t the parallel loops. The actual fusable loops /// of the producerIndexMap w.r.t the parallel loops. The actual fusable loops
/// are the dimensions of the consumerLoopToProducerLoop map that correspond to /// are the dimensions of the consumerLoopToProducerLoop map that correspond to
/// parallel loops and appear in the result of the map /// parallel loops and appear in the result of the map
/// ///
/// Example 1: /// Example 1:
/// linalg.fill(%c, %cst) /// linalg.fill(%cst, %c)
/// linalg.matmul ins(%a, %b) outs(%c) /// linalg.matmul ins(%a, %b) outs(%c)
/// Number of parallel loops : 2 /// Number of parallel loops : 2
/// producerIndexMap = affine_map<(i, j) ->(i , j)> /// producerIndexMap = affine_map<(i, j) ->(i , j)>
/// consumerIndexMap = affine_map<(i, j, k) -> (i, j)> /// consumerIndexMap = affine_map<(i, j, k) -> (i, j)>
/// consumerLoopToProducerLoop = affine_map<(i, j, k) -> (i, j)> /// consumerLoopToProducerLoop = affine_map<(i, j, k) -> (i, j)>
/// Fused dimensions : i, j /// Fused dimensions : i, j
/// ///
/// Example 2: /// Example 2:
▲ Show 20 Lines • Show All 324 Lines • Show Last 20 Lines

mlir/lib/Dialect/Vector/VectorTransforms.cpp

Show First 20 Lines • Show All 2,407 Lines • ▼ Show 20 Lines
/// 1. `inBoundsCond` and a `compatibleMemRefType` have been computed. /// 1. `inBoundsCond` and a `compatibleMemRefType` have been computed.
/// 2. a memref of single vector `alloc` has been allocated. /// 2. a memref of single vector `alloc` has been allocated.
/// Produce IR resembling: /// Produce IR resembling:
/// ``` /// ```
/// %1:3 = scf.if (%inBounds) { /// %1:3 = scf.if (%inBounds) {
/// memref.cast %A: memref<A...> to compatibleMemRefType /// memref.cast %A: memref<A...> to compatibleMemRefType
/// scf.yield %view, ... : compatibleMemRefType, index, index /// scf.yield %view, ... : compatibleMemRefType, index, index
/// } else { /// } else {
/// %2 = linalg.fill(%alloc, %pad) /// %2 = linalg.fill(%pad, %alloc)
/// %3 = subview %view [...][...][...] /// %3 = subview %view [...][...][...]
/// linalg.copy(%3, %alloc) /// linalg.copy(%3, %alloc)
/// memref.cast %alloc: memref<B...> to compatibleMemRefType /// memref.cast %alloc: memref<B...> to compatibleMemRefType
/// scf.yield %4, ... : compatibleMemRefType, index, index /// scf.yield %4, ... : compatibleMemRefType, index, index
/// } /// }
/// ``` /// ```
/// Return the produced scf::IfOp. /// Return the produced scf::IfOp.
static scf::IfOp static scf::IfOp
▲ Show 20 Lines • Show All 1,765 Lines • Show Last 20 Lines

mlir/test/Conversion/TosaToLinalg/tosa-to-linalg.mlir

Show First 20 Lines • Show All 538 Lines • ▼ Show 20 Lines
// CHECK-DAG: #[[$MAP1:.*]] = affine_map<(d0, d1) -> (d1)> // CHECK-DAG: #[[$MAP1:.*]] = affine_map<(d0, d1) -> (d1)>
// CHECK-DAG: #[[$MAP2:.*]] = affine_map<(d0, d1) -> (d0)> // CHECK-DAG: #[[$MAP2:.*]] = affine_map<(d0, d1) -> (d0)>
// CHECK-LABEL: @reduce_float // CHECK-LABEL: @reduce_float
// CHECK-SAME: [[ARG0:%.+]]: tensor<5x4xf32> // CHECK-SAME: [[ARG0:%.+]]: tensor<5x4xf32>
func @reduce_float(%arg0: tensor<5x4xf32>) -> () { func @reduce_float(%arg0: tensor<5x4xf32>) -> () {
// CHECK: [[INIT:%.+]] = linalg.init_tensor [4] // CHECK: [[INIT:%.+]] = linalg.init_tensor [4]
// CHECK: [[CST0:%.+]] = constant 0.0 // CHECK: [[CST0:%.+]] = constant 0.0
// CHECK: [[FILL:%.+]] = linalg.fill([[INIT]], [[CST0]]) // CHECK: [[FILL:%.+]] = linalg.fill([[CST0]], [[INIT]])
// CHECK: [[GENERIC:%.+]] = linalg.generic {indexing_maps = [#[[$MAP0]], #[[$MAP1]]], iterator_types = ["reduction", "parallel"]} ins([[ARG0]] : tensor<5x4xf32>) outs([[FILL]] : tensor<4xf32>) // CHECK: [[GENERIC:%.+]] = linalg.generic {indexing_maps = [#[[$MAP0]], #[[$MAP1]]], iterator_types = ["reduction", "parallel"]} ins([[ARG0]] : tensor<5x4xf32>) outs([[FILL]] : tensor<4xf32>)
// CHECK: ^bb0(%arg1: f32, %arg2: f32) // CHECK: ^bb0(%arg1: f32, %arg2: f32)
// CHECK: [[RES:%.+]] = addf %arg1, %arg2 : f32 // CHECK: [[RES:%.+]] = addf %arg1, %arg2 : f32
// CHECK: linalg.yield [[RES]] : f32 // CHECK: linalg.yield [[RES]] : f32
// CHECK: linalg.tensor_expand_shape [[GENERIC]] {{\[}}[0, 1]] : tensor<4xf32> into tensor<1x4xf32> // CHECK: linalg.tensor_expand_shape [[GENERIC]] {{\[}}[0, 1]] : tensor<4xf32> into tensor<1x4xf32>
%0 = "tosa.reduce_sum"(%arg0) {axis = 0 : i64} : (tensor<5x4xf32>) -> tensor<1x4xf32> %0 = "tosa.reduce_sum"(%arg0) {axis = 0 : i64} : (tensor<5x4xf32>) -> tensor<1x4xf32>
// CHECK: [[INIT:%.+]] = linalg.init_tensor [5] // CHECK: [[INIT:%.+]] = linalg.init_tensor [5]
// CHECK: [[CST0:%.+]] = constant 0.0 // CHECK: [[CST0:%.+]] = constant 0.0
// CHECK: [[FILL:%.+]] = linalg.fill([[INIT]], [[CST0]]) // CHECK: [[FILL:%.+]] = linalg.fill([[CST0]], [[INIT]])
// CHECK: [[GENERIC:%.+]] = linalg.generic {indexing_maps = [#[[$MAP0]], #[[$MAP2]]], iterator_types = ["parallel", "reduction"]} ins([[ARG0]] : tensor<5x4xf32>) outs([[FILL]] : tensor<5xf32>) // CHECK: [[GENERIC:%.+]] = linalg.generic {indexing_maps = [#[[$MAP0]], #[[$MAP2]]], iterator_types = ["parallel", "reduction"]} ins([[ARG0]] : tensor<5x4xf32>) outs([[FILL]] : tensor<5xf32>)
// CHECK: ^bb0(%arg1: f32, %arg2: f32) // CHECK: ^bb0(%arg1: f32, %arg2: f32)
// CHECK: [[RES:%.+]] = addf %arg1, %arg2 : f32 // CHECK: [[RES:%.+]] = addf %arg1, %arg2 : f32
// CHECK: linalg.yield [[RES]] : f32 // CHECK: linalg.yield [[RES]] : f32
// CHECK: linalg.tensor_expand_shape [[GENERIC]] {{\[}}[0, 1]] : tensor<5xf32> into tensor<5x1xf32> // CHECK: linalg.tensor_expand_shape [[GENERIC]] {{\[}}[0, 1]] : tensor<5xf32> into tensor<5x1xf32>
%1 = "tosa.reduce_sum"(%arg0) {axis = 1 : i64} : (tensor<5x4xf32>) -> tensor<5x1xf32> %1 = "tosa.reduce_sum"(%arg0) {axis = 1 : i64} : (tensor<5x4xf32>) -> tensor<5x1xf32>
// CHECK: constant 1.0 // CHECK: constant 1.0
Show All 24 Lines
// CHECK: #[[$MAP1:.*]] = affine_map<(d0, d1) -> (d1)> // CHECK: #[[$MAP1:.*]] = affine_map<(d0, d1) -> (d1)>
// CHECK: #[[$MAP2:.*]] = affine_map<(d0, d1) -> (d0)> // CHECK: #[[$MAP2:.*]] = affine_map<(d0, d1) -> (d0)>
// CHECK-LABEL: @reduce_int // CHECK-LABEL: @reduce_int
// CHECK-SAME: [[ARG0:%.+]]: tensor<5x4xi32> // CHECK-SAME: [[ARG0:%.+]]: tensor<5x4xi32>
func @reduce_int(%arg0: tensor<5x4xi32>) -> () { func @reduce_int(%arg0: tensor<5x4xi32>) -> () {
// CHECK: [[INIT:%.+]] = linalg.init_tensor [4] // CHECK: [[INIT:%.+]] = linalg.init_tensor [4]
// CHECK: [[CST0:%.+]] = constant 0 // CHECK: [[CST0:%.+]] = constant 0
// CHECK: [[FILL:%.+]] = linalg.fill([[INIT]], [[CST0]]) // CHECK: [[FILL:%.+]] = linalg.fill([[CST0]], [[INIT]])
// CHECK: [[GENERIC:%.+]] = linalg.generic {indexing_maps = [#[[$MAP0]], #[[$MAP1]]], iterator_types = ["reduction", "parallel"]} ins([[ARG0]] : tensor<5x4xi32>) outs([[FILL]] : tensor<4xi32>) // CHECK: [[GENERIC:%.+]] = linalg.generic {indexing_maps = [#[[$MAP0]], #[[$MAP1]]], iterator_types = ["reduction", "parallel"]} ins([[ARG0]] : tensor<5x4xi32>) outs([[FILL]] : tensor<4xi32>)
// CHECK: ^bb0(%arg1: i32, %arg2: i32) // CHECK: ^bb0(%arg1: i32, %arg2: i32)
// CHECK: [[RES:%.+]] = addi %arg1, %arg2 : i32 // CHECK: [[RES:%.+]] = addi %arg1, %arg2 : i32
// CHECK: linalg.yield [[RES]] : i32 // CHECK: linalg.yield [[RES]] : i32
// CHECK: linalg.tensor_expand_shape [[GENERIC]] {{\[}}[0, 1]] : tensor<4xi32> into tensor<1x4xi32> // CHECK: linalg.tensor_expand_shape [[GENERIC]] {{\[}}[0, 1]] : tensor<4xi32> into tensor<1x4xi32>
%0 = "tosa.reduce_sum"(%arg0) {axis = 0 : i64} : (tensor<5x4xi32>) -> tensor<1x4xi32> %0 = "tosa.reduce_sum"(%arg0) {axis = 0 : i64} : (tensor<5x4xi32>) -> tensor<1x4xi32>
// CHECK: [[INIT:%.+]] = linalg.init_tensor [5] // CHECK: [[INIT:%.+]] = linalg.init_tensor [5]
// CHECK: [[CST0:%.+]] = constant 0 // CHECK: [[CST0:%.+]] = constant 0
// CHECK: [[FILL:%.+]] = linalg.fill([[INIT]], [[CST0]]) // CHECK: [[FILL:%.+]] = linalg.fill([[CST0]], [[INIT]])
// CHECK: [[GENERIC:%.+]] = linalg.generic {indexing_maps = [#[[$MAP0]], #[[$MAP2]]], iterator_types = ["parallel", "reduction"]} ins([[ARG0]] : tensor<5x4xi32>) outs([[FILL]] : tensor<5xi32>) // CHECK: [[GENERIC:%.+]] = linalg.generic {indexing_maps = [#[[$MAP0]], #[[$MAP2]]], iterator_types = ["parallel", "reduction"]} ins([[ARG0]] : tensor<5x4xi32>) outs([[FILL]] : tensor<5xi32>)
// CHECK: ^bb0(%arg1: i32, %arg2: i32) // CHECK: ^bb0(%arg1: i32, %arg2: i32)
// CHECK: [[RES:%.+]] = addi %arg1, %arg2 : i32 // CHECK: [[RES:%.+]] = addi %arg1, %arg2 : i32
// CHECK: linalg.yield [[RES]] : i32 // CHECK: linalg.yield [[RES]] : i32
// CHECK: linalg.tensor_expand_shape [[GENERIC]] {{\[}}[0, 1]] : tensor<5xi32> into tensor<5x1xi32> // CHECK: linalg.tensor_expand_shape [[GENERIC]] {{\[}}[0, 1]] : tensor<5xi32> into tensor<5x1xi32>
%1 = "tosa.reduce_sum"(%arg0) {axis = 1 : i64} : (tensor<5x4xi32>) -> tensor<5x1xi32> %1 = "tosa.reduce_sum"(%arg0) {axis = 1 : i64} : (tensor<5x4xi32>) -> tensor<5x1xi32>
// CHECK: constant 1 // CHECK: constant 1
Show All 23 Lines
// CHECK: #[[$MAP0:.*]] = affine_map<(d0, d1) -> (d0, d1)> // CHECK: #[[$MAP0:.*]] = affine_map<(d0, d1) -> (d0, d1)>
// CHECK: #[[$MAP1:.*]] = affine_map<(d0, d1) -> (d1)> // CHECK: #[[$MAP1:.*]] = affine_map<(d0, d1) -> (d1)>
// CHECK-LABEL: @reduce_bool // CHECK-LABEL: @reduce_bool
// CHECK-SAME: [[ARG0:%.+]]: tensor<5x4xi1> // CHECK-SAME: [[ARG0:%.+]]: tensor<5x4xi1>
func @reduce_bool(%arg0: tensor<5x4xi1>) -> () { func @reduce_bool(%arg0: tensor<5x4xi1>) -> () {
// CHECK: [[INIT:%.+]] = linalg.init_tensor [4] // CHECK: [[INIT:%.+]] = linalg.init_tensor [4]
// CHECK: [[CST0:%.+]] = constant true // CHECK: [[CST0:%.+]] = constant true
// CHECK: [[FILL:%.+]] = linalg.fill([[INIT]], [[CST0]]) // CHECK: [[FILL:%.+]] = linalg.fill([[CST0]], [[INIT]])
// CHECK: [[GENERIC:%.+]] = linalg.generic {indexing_maps = [#[[$MAP0]], #[[$MAP1]]], iterator_types = ["reduction", "parallel"]} ins([[ARG0]] : tensor<5x4xi1>) outs([[FILL]] : tensor<4xi1>) // CHECK: [[GENERIC:%.+]] = linalg.generic {indexing_maps = [#[[$MAP0]], #[[$MAP1]]], iterator_types = ["reduction", "parallel"]} ins([[ARG0]] : tensor<5x4xi1>) outs([[FILL]] : tensor<4xi1>)
// CHECK: ^bb0(%arg1: i1, %arg2: i1) // CHECK: ^bb0(%arg1: i1, %arg2: i1)
// CHECK: [[RES:%.+]] = and %arg1, %arg2 : i1 // CHECK: [[RES:%.+]] = and %arg1, %arg2 : i1
// CHECK: linalg.yield [[RES]] : i1 // CHECK: linalg.yield [[RES]] : i1
// CHECK: linalg.tensor_expand_shape [[GENERIC]] {{\[}}[0, 1]] : tensor<4xi1> into tensor<1x4xi1> // CHECK: linalg.tensor_expand_shape [[GENERIC]] {{\[}}[0, 1]] : tensor<4xi1> into tensor<1x4xi1>
%0 = "tosa.reduce_all"(%arg0) {axis = 0 : i64} : (tensor<5x4xi1>) -> tensor<1x4xi1> %0 = "tosa.reduce_all"(%arg0) {axis = 0 : i64} : (tensor<5x4xi1>) -> tensor<1x4xi1>
// CHECK: constant false // CHECK: constant false
Show All 15 Linesfunc @concat(%arg0: tensor<5x1xf32>, %arg1: tensor<6x1xf32>) -> () {
// CHECK: [[IDX0:%.+]] = constant 0 : index // CHECK: [[IDX0:%.+]] = constant 0 : index
// CHECK: [[ARG0_DIM0:%.+]] = memref.dim %arg0, [[IDX0]] // CHECK: [[ARG0_DIM0:%.+]] = memref.dim %arg0, [[IDX0]]
// CHECK: [[IDX1:%.+]] = constant 1 : index // CHECK: [[IDX1:%.+]] = constant 1 : index
// CHECK: [[ARG0_DIM1:%.+]] = memref.dim %arg0, [[IDX1]] // CHECK: [[ARG0_DIM1:%.+]] = memref.dim %arg0, [[IDX1]]
// CHECK: [[ARG1_AXIS:%.+]] = memref.dim %arg1, [[AXIS]] // CHECK: [[ARG1_AXIS:%.+]] = memref.dim %arg1, [[AXIS]]
// CHECK: [[RESULT_AXIS:%.+]] = addi [[ARG0_DIM0]], [[ARG1_AXIS]] // CHECK: [[RESULT_AXIS:%.+]] = addi [[ARG0_DIM0]], [[ARG1_AXIS]]
// CHECK: [[INIT:%.+]] = linalg.init_tensor [11, 1] // CHECK: [[INIT:%.+]] = linalg.init_tensor [11, 1]
// CHECK: [[CST:%.+]] = constant 0.0 // CHECK: [[CST:%.+]] = constant 0.0
// CHECK: [[FILL:%.+]] = linalg.fill([[INIT]], [[CST]]) // CHECK: [[FILL:%.+]] = linalg.fill([[CST]], [[INIT]])
// CHECK: [[ARG0_DIM0:%.+]] = memref.dim %arg0, [[AXIS]] // CHECK: [[ARG0_DIM0:%.+]] = memref.dim %arg0, [[AXIS]]
// CHECK: [[INSERT0:%.+]] = tensor.insert_slice %arg0 into [[FILL]]{{\[}}[[OFFSET]], [[OFFSET]]] {{\[}}[[ARG0_DIM0]], [[ARG0_DIM1]]] {{\[}}[[STRIDE]], [[STRIDE]]] // CHECK: [[INSERT0:%.+]] = tensor.insert_slice %arg0 into [[FILL]]{{\[}}[[OFFSET]], [[OFFSET]]] {{\[}}[[ARG0_DIM0]], [[ARG0_DIM1]]] {{\[}}[[STRIDE]], [[STRIDE]]]
// CHECK: [[NEW_OFFSET:%.+]] = addi [[OFFSET]], [[ARG0_DIM0]] // CHECK: [[NEW_OFFSET:%.+]] = addi [[OFFSET]], [[ARG0_DIM0]]
// CHECK: [[ARG1_DIM0:%.+]] = memref.dim %arg1, [[AXIS]] // CHECK: [[ARG1_DIM0:%.+]] = memref.dim %arg1, [[AXIS]]
// CHECK: [[INSERT1:%.+]] = tensor.insert_slice %arg1 into [[INSERT0]]{{\[}}[[NEW_OFFSET]], [[OFFSET]]] {{\[}}[[ARG1_DIM0]], [[ARG0_DIM1]]] {{\[}}[[STRIDE]], [[STRIDE]]] // CHECK: [[INSERT1:%.+]] = tensor.insert_slice %arg1 into [[INSERT0]]{{\[}}[[NEW_OFFSET]], [[OFFSET]]] {{\[}}[[ARG1_DIM0]], [[ARG0_DIM1]]] {{\[}}[[STRIDE]], [[STRIDE]]]
%0 = "tosa.concat"(%arg0, %arg1) { axis = 0 : i64} : (tensor<5x1xf32>, tensor<6x1xf32>) -> (tensor<11x1xf32>) %0 = "tosa.concat"(%arg0, %arg1) { axis = 0 : i64} : (tensor<5x1xf32>, tensor<6x1xf32>) -> (tensor<11x1xf32>)
// CHECK: [[AXIS:%.+]] = constant 1 // CHECK: [[AXIS:%.+]] = constant 1
// CHECK: [[STRIDE:%.+]] = constant 1 // CHECK: [[STRIDE:%.+]] = constant 1
// CHECK: [[OFFSET:%.+]] = constant 0 : index // CHECK: [[OFFSET:%.+]] = constant 0 : index
// CHECK: [[IDX0:%.+]] = constant 0 : index // CHECK: [[IDX0:%.+]] = constant 0 : index
// CHECK: [[ARG0_DIM0:%.+]] = memref.dim %arg0, [[IDX0]] // CHECK: [[ARG0_DIM0:%.+]] = memref.dim %arg0, [[IDX0]]
// CHECK: [[IDX1:%.+]] = constant 1 : index // CHECK: [[IDX1:%.+]] = constant 1 : index
// CHECK: [[ARG0_DIM1:%.+]] = memref.dim %arg0, [[IDX1]] // CHECK: [[ARG0_DIM1:%.+]] = memref.dim %arg0, [[IDX1]]
// CHECK: [[ARG1_AXIS:%.+]] = memref.dim %arg0, [[AXIS]] // CHECK: [[ARG1_AXIS:%.+]] = memref.dim %arg0, [[AXIS]]
// CHECK: [[RESULT_AXIS:%.+]] = addi [[ARG0_DIM1]], [[ARG1_AXIS]] // CHECK: [[RESULT_AXIS:%.+]] = addi [[ARG0_DIM1]], [[ARG1_AXIS]]
// CHECK: [[INIT:%.+]] = linalg.init_tensor [5, 2] // CHECK: [[INIT:%.+]] = linalg.init_tensor [5, 2]
// CHECK: [[CST:%.+]] = constant 0.0 // CHECK: [[CST:%.+]] = constant 0.0
// CHECK: [[FILL:%.+]] = linalg.fill([[INIT]], [[CST]]) // CHECK: [[FILL:%.+]] = linalg.fill([[CST]], [[INIT]])
// CHECK: [[ARG0_DIM1:%.+]] = memref.dim %arg0, [[AXIS]] // CHECK: [[ARG0_DIM1:%.+]] = memref.dim %arg0, [[AXIS]]
// CHECK: [[INSERT0:%.+]] = tensor.insert_slice %arg0 into [[FILL]]{{\[}}[[OFFSET]], [[OFFSET]]] {{\[}}[[ARG0_DIM0]], [[ARG0_DIM1]]] {{\[}}[[STRIDE]], [[STRIDE]]] // CHECK: [[INSERT0:%.+]] = tensor.insert_slice %arg0 into [[FILL]]{{\[}}[[OFFSET]], [[OFFSET]]] {{\[}}[[ARG0_DIM0]], [[ARG0_DIM1]]] {{\[}}[[STRIDE]], [[STRIDE]]]
// CHECK: [[NEW_OFFSET:%.+]] = addi [[OFFSET]], [[ARG0_DIM1]] // CHECK: [[NEW_OFFSET:%.+]] = addi [[OFFSET]], [[ARG0_DIM1]]
// CHECK: [[ARG1_DIM1:%.+]] = memref.dim %arg0, [[AXIS]] // CHECK: [[ARG1_DIM1:%.+]] = memref.dim %arg0, [[AXIS]]
// CHECK: [[INSERT1:%.+]] = tensor.insert_slice %arg0 into [[INSERT0]]{{\[}}[[OFFSET]], [[NEW_OFFSET]]] {{\[}}[[ARG0_DIM0]], [[ARG1_DIM1]]] {{\[}}[[STRIDE]], [[STRIDE]]] // CHECK: [[INSERT1:%.+]] = tensor.insert_slice %arg0 into [[INSERT0]]{{\[}}[[OFFSET]], [[NEW_OFFSET]]] {{\[}}[[ARG0_DIM0]], [[ARG1_DIM1]]] {{\[}}[[STRIDE]], [[STRIDE]]]
%1 = "tosa.concat"(%arg0, %arg0) { axis = 1 : i64} : (tensor<5x1xf32>, tensor<5x1xf32>) -> (tensor<5x2xf32>) %1 = "tosa.concat"(%arg0, %arg0) { axis = 1 : i64} : (tensor<5x1xf32>, tensor<5x1xf32>) -> (tensor<5x2xf32>)
return return
} }
▲ Show 20 Lines • Show All 134 Lines • ▼ Show 20 Lines
// ----- // -----
// CHECK-LABEL: @matmul // CHECK-LABEL: @matmul
func @matmul(%arg0: tensor<1x5x3xf32>, %arg1: tensor<1x3x6xf32>, %arg2: tensor<1x6xf32>) -> (tensor<1x5x6xf32>) { func @matmul(%arg0: tensor<1x5x3xf32>, %arg1: tensor<1x3x6xf32>, %arg2: tensor<1x6xf32>) -> (tensor<1x5x6xf32>) {
// CHECK: [[C0:%.+]] = constant 0 // CHECK: [[C0:%.+]] = constant 0
// CHECK: [[INIT:%.+]] = linalg.init_tensor [1, 5, 6] // CHECK: [[INIT:%.+]] = linalg.init_tensor [1, 5, 6]
// CHECK: [[FILLED:%.+]] = linalg.fill([[INIT]], [[C0]]) : tensor<1x5x6xf32>, f32 -> tensor<1x5x6xf32> // CHECK: [[FILLED:%.+]] = linalg.fill([[C0]], [[INIT]]) : f32, tensor<1x5x6xf32> -> tensor<1x5x6xf32>
// CHECK: linalg.batch_matmul ins(%arg0, %arg1 : tensor<1x5x3xf32>, tensor<1x3x6xf32>) outs([[FILLED]] : tensor<1x5x6xf32>) -> tensor<1x5x6xf32> // CHECK: linalg.batch_matmul ins(%arg0, %arg1 : tensor<1x5x3xf32>, tensor<1x3x6xf32>) outs([[FILLED]] : tensor<1x5x6xf32>) -> tensor<1x5x6xf32>
%0 = "tosa.matmul"(%arg0, %arg1) : (tensor<1x5x3xf32>, tensor<1x3x6xf32>) -> (tensor<1x5x6xf32>) %0 = "tosa.matmul"(%arg0, %arg1) : (tensor<1x5x3xf32>, tensor<1x3x6xf32>) -> (tensor<1x5x6xf32>)
return %0 : tensor<1x5x6xf32> return %0 : tensor<1x5x6xf32>
} }
// ----- // -----
// CHECK: #[[$MAP0:.*]] = affine_map<(d0, d1) -> (d1)> // CHECK: #[[$MAP0:.*]] = affine_map<(d0, d1) -> (d1)>
▲ Show 20 Lines • Show All 64 Lines • ▼ Show 20 Lines
// CHECK: #[[$MAP1:.*]] = affine_map<(d0, d1) -> (d1)> // CHECK: #[[$MAP1:.*]] = affine_map<(d0, d1) -> (d1)>
// CHECK: #[[$MAP2:.*]] = affine_map<(d0, d1) -> (d0)> // CHECK: #[[$MAP2:.*]] = affine_map<(d0, d1) -> (d0)>
// CHECK: #[[$MAP3:.*]] = affine_map<(d0) -> (d0)> // CHECK: #[[$MAP3:.*]] = affine_map<(d0) -> (d0)>
// CHECK: #[[$MAP4:.*]] = affine_map<(d0) -> ()> // CHECK: #[[$MAP4:.*]] = affine_map<(d0) -> ()>
func @argmax(%arg0 : tensor<3x2xi32>, %arg1 : tensor<6xf32>) -> () { func @argmax(%arg0 : tensor<3x2xi32>, %arg1 : tensor<6xf32>) -> () {
// CHECK: [[IDX_INIT:%.+]] = linalg.init_tensor [2] // CHECK: [[IDX_INIT:%.+]] = linalg.init_tensor [2]
// CHECK: [[IDX_MIN:%.+]] = constant 0 : i32 // CHECK: [[IDX_MIN:%.+]] = constant 0 : i32
// CHECK: [[IDX_FILL:%.+]] = linalg.fill([[IDX_INIT]], [[IDX_MIN]]) // CHECK: [[IDX_FILL:%.+]] = linalg.fill([[IDX_MIN]], [[IDX_INIT]])
// CHECK: [[VAL_INIT:%.+]] = linalg.init_tensor [2] // CHECK: [[VAL_INIT:%.+]] = linalg.init_tensor [2]
// CHECK: [[VAL_MIN:%.+]] = constant -2147483648 // CHECK: [[VAL_MIN:%.+]] = constant -2147483648
// CHECK: [[VAL_FILL:%.+]] = linalg.fill([[VAL_INIT]], [[VAL_MIN]]) // CHECK: [[VAL_FILL:%.+]] = linalg.fill([[VAL_MIN]], [[VAL_INIT]])
// CHECK: linalg.generic {indexing_maps = [#[[$MAP0]], #[[$MAP1]], #[[$MAP1]]], iterator_types = ["reduction", "parallel"]} ins(%arg0 : tensor<3x2xi32>) outs([[IDX_FILL]], [[VAL_FILL]] : tensor<2xi32>, tensor<2xi32>) // CHECK: linalg.generic {indexing_maps = [#[[$MAP0]], #[[$MAP1]], #[[$MAP1]]], iterator_types = ["reduction", "parallel"]} ins(%arg0 : tensor<3x2xi32>) outs([[IDX_FILL]], [[VAL_FILL]] : tensor<2xi32>, tensor<2xi32>)
// CHECK: [[IDX:%.+]] = linalg.index 0 // CHECK: [[IDX:%.+]] = linalg.index 0
// CHECK: [[CAST:%.+]] = index_cast [[IDX]] // CHECK: [[CAST:%.+]] = index_cast [[IDX]]
// CHECK: [[CMP:%.+]] = cmpi sgt, %arg2, %arg4 // CHECK: [[CMP:%.+]] = cmpi sgt, %arg2, %arg4
// CHECK: [[SELECT_VAL:%.+]] = select [[CMP]], %arg2, %arg4 // CHECK: [[SELECT_VAL:%.+]] = select [[CMP]], %arg2, %arg4
// CHECK: [[SELECT_IDX:%.+]] = select [[CMP]], [[CAST]], %arg3 // CHECK: [[SELECT_IDX:%.+]] = select [[CMP]], [[CAST]], %arg3
// CHECK: linalg.yield [[SELECT_IDX]], [[SELECT_VAL]] // CHECK: linalg.yield [[SELECT_IDX]], [[SELECT_VAL]]
%0 = "tosa.argmax"(%arg0) { axis = 0 : i64} : (tensor<3x2xi32>) -> (tensor<2xi32>) %0 = "tosa.argmax"(%arg0) { axis = 0 : i64} : (tensor<3x2xi32>) -> (tensor<2xi32>)
// CHECK: [[IDX_INIT:%.+]] = linalg.init_tensor [3] // CHECK: [[IDX_INIT:%.+]] = linalg.init_tensor [3]
// CHECK: [[IDX_MIN:%.+]] = constant 0 : i32 // CHECK: [[IDX_MIN:%.+]] = constant 0 : i32
// CHECK: [[IDX_FILL:%.+]] = linalg.fill([[IDX_INIT]], [[IDX_MIN]]) // CHECK: [[IDX_FILL:%.+]] = linalg.fill([[IDX_MIN]], [[IDX_INIT]])
// CHECK: [[VAL_INIT:%.+]] = linalg.init_tensor [3] // CHECK: [[VAL_INIT:%.+]] = linalg.init_tensor [3]
// CHECK: [[VAL_MIN:%.+]] = constant -2147483648 // CHECK: [[VAL_MIN:%.+]] = constant -2147483648
// CHECK: [[VAL_FILL:%.+]] = linalg.fill([[VAL_INIT]], [[VAL_MIN]]) // CHECK: [[VAL_FILL:%.+]] = linalg.fill([[VAL_MIN]], [[VAL_INIT]])
// CHECK: linalg.generic {indexing_maps = [#map0, #map2, #map2], iterator_types = ["parallel", "reduction"]} ins(%arg0 : tensor<3x2xi32>) outs([[IDX_FILL]], [[VAL_FILL]] : tensor<3xi32>, tensor<3xi32>) // CHECK: linalg.generic {indexing_maps = [#map0, #map2, #map2], iterator_types = ["parallel", "reduction"]} ins(%arg0 : tensor<3x2xi32>) outs([[IDX_FILL]], [[VAL_FILL]] : tensor<3xi32>, tensor<3xi32>)
// CHECK: [[IDX:%.+]] = linalg.index 1 // CHECK: [[IDX:%.+]] = linalg.index 1
// CHECK: [[CAST:%.+]] = index_cast [[IDX]] // CHECK: [[CAST:%.+]] = index_cast [[IDX]]
// CHECK: [[CMP:%.+]] = cmpi sgt, %arg2, %arg4 // CHECK: [[CMP:%.+]] = cmpi sgt, %arg2, %arg4
// CHECK: [[SELECT_VAL:%.+]] = select [[CMP]], %arg2, %arg4 // CHECK: [[SELECT_VAL:%.+]] = select [[CMP]], %arg2, %arg4
// CHECK: [[SELECT_IDX:%.+]] = select [[CMP]], [[CAST]], %arg3 // CHECK: [[SELECT_IDX:%.+]] = select [[CMP]], [[CAST]], %arg3
// CHECK: linalg.yield [[SELECT_IDX]], [[SELECT_VAL]] // CHECK: linalg.yield [[SELECT_IDX]], [[SELECT_VAL]]
%1 = "tosa.argmax"(%arg0) { axis = 1 : i64} : (tensor<3x2xi32>) -> (tensor<3xi32>) %1 = "tosa.argmax"(%arg0) { axis = 1 : i64} : (tensor<3x2xi32>) -> (tensor<3xi32>)
▲ Show 20 Lines • Show All 84 Lines • ▼ Show 20 Lines
} }
// ----- // -----
// CHECK-LABEL: @max_pool // CHECK-LABEL: @max_pool
func @max_pool(%arg0: tensor<1x6x34x62xf32>) -> () { func @max_pool(%arg0: tensor<1x6x34x62xf32>) -> () {
// CHECK-DAG: [[CONST:%.+]] = constant -3.40282347E+38 // CHECK-DAG: [[CONST:%.+]] = constant -3.40282347E+38
// CHECK-DAG: [[INIT:%.+]] = linalg.init_tensor [1, 4, 32, 62] // CHECK-DAG: [[INIT:%.+]] = linalg.init_tensor [1, 4, 32, 62]
// CHECK-DAG: [[FILL:%.+]] = linalg.fill([[INIT]], [[CONST]]) // CHECK-DAG: [[FILL:%.+]] = linalg.fill([[CONST]], [[INIT]])
// CHECK-DAG: [[KERNEL:%.+]] = linalg.init_tensor [3, 3] // CHECK-DAG: [[KERNEL:%.+]] = linalg.init_tensor [3, 3]
// CHECK: linalg.pooling_nhwc_max {dilations = dense<1> : vector<2xi64>, strides = dense<1> : vector<2xi64>} ins(%arg0, [[KERNEL]] : tensor<1x6x34x62xf32>, tensor<3x3xf32>) outs([[FILL]] : tensor<1x4x32x62xf32>) // CHECK: linalg.pooling_nhwc_max {dilations = dense<1> : vector<2xi64>, strides = dense<1> : vector<2xi64>} ins(%arg0, [[KERNEL]] : tensor<1x6x34x62xf32>, tensor<3x3xf32>) outs([[FILL]] : tensor<1x4x32x62xf32>)
%0 = "tosa.max_pool2d"(%arg0) {pad = [0, 0, 0, 0], kernel = [3, 3], stride = [1, 1]} : (tensor<1x6x34x62xf32>) -> (tensor<1x4x32x62xf32>) %0 = "tosa.max_pool2d"(%arg0) {pad = [0, 0, 0, 0], kernel = [3, 3], stride = [1, 1]} : (tensor<1x6x34x62xf32>) -> (tensor<1x4x32x62xf32>)
return return
} }
// CHECK-LABEL: @max_pool_padded // CHECK-LABEL: @max_pool_padded
func @max_pool_padded(%arg0: tensor<1x6x34x62xf32>) -> () { func @max_pool_padded(%arg0: tensor<1x6x34x62xf32>) -> () {
// CHECK-DAG: [[CONST:%.+]] = constant -3.40282347E+38 : f32 // CHECK-DAG: [[CONST:%.+]] = constant -3.40282347E+38 : f32
// CHECK-DAG: [[PAD:%.+]] = linalg.pad_tensor %arg0 low[0, 0, 0, 0] high[0, 0, 1, 0] // CHECK-DAG: [[PAD:%.+]] = linalg.pad_tensor %arg0 low[0, 0, 0, 0] high[0, 0, 1, 0]
// CHECK-DAG: linalg.yield [[CONST]] // CHECK-DAG: linalg.yield [[CONST]]
// CHECK-DAG: [[INITVAL:%.+]] = constant -3.40282347E+38 : f32 // CHECK-DAG: [[INITVAL:%.+]] = constant -3.40282347E+38 : f32
// CHECK-DAG: [[INIT:%.+]] = linalg.init_tensor [1, 4, 33, 62] // CHECK-DAG: [[INIT:%.+]] = linalg.init_tensor [1, 4, 33, 62]
// CHECK-DAG: [[FILL:%.+]] = linalg.fill([[INIT]], [[INITVAL]]) // CHECK-DAG: [[FILL:%.+]] = linalg.fill([[INITVAL]], [[INIT]])
// CHECK-DAG: [[KERNEL:%.+]] = linalg.init_tensor [3, 3] // CHECK-DAG: [[KERNEL:%.+]] = linalg.init_tensor [3, 3]
// CHECK: linalg.pooling_nhwc_max {dilations = dense<1> : vector<2xi64>, strides = dense<1> : vector<2xi64>} ins([[PAD]], [[KERNEL]] : tensor<1x6x35x62xf32>, tensor<3x3xf32>) outs([[FILL]] : tensor<1x4x33x62xf32>) // CHECK: linalg.pooling_nhwc_max {dilations = dense<1> : vector<2xi64>, strides = dense<1> : vector<2xi64>} ins([[PAD]], [[KERNEL]] : tensor<1x6x35x62xf32>, tensor<3x3xf32>) outs([[FILL]] : tensor<1x4x33x62xf32>)
%0 = "tosa.max_pool2d"(%arg0) {pad = [0, 0, 0, 1], kernel = [3, 3], stride = [1, 1]} : (tensor<1x6x34x62xf32>) -> (tensor<1x4x33x62xf32>) %0 = "tosa.max_pool2d"(%arg0) {pad = [0, 0, 0, 1], kernel = [3, 3], stride = [1, 1]} : (tensor<1x6x34x62xf32>) -> (tensor<1x4x33x62xf32>)
return return
} }
// CHECK-LABEL: @max_pool_i8 // CHECK-LABEL: @max_pool_i8
func @max_pool_i8(%arg0: tensor<1x6x34x62xi8>) -> () { func @max_pool_i8(%arg0: tensor<1x6x34x62xi8>) -> () {
Show All 22 Lines
// CHECK-LABEL: @avg_pool // CHECK-LABEL: @avg_pool
func @avg_pool(%arg0: tensor<1x6x34x62xf32>) -> (tensor<1x5x33x62xf32>) { func @avg_pool(%arg0: tensor<1x6x34x62xf32>) -> (tensor<1x5x33x62xf32>) {
// Initial piece computes the sum of the pooling region, with appropriate padding. // Initial piece computes the sum of the pooling region, with appropriate padding.
// CHECK: [[CONST:%.+]] = constant 0 // CHECK: [[CONST:%.+]] = constant 0
// CHECK: [[PAD:%.+]] = linalg.pad_tensor %arg0 low[0, 1, 1, 0] high[0, 1, 1, 0] // CHECK: [[PAD:%.+]] = linalg.pad_tensor %arg0 low[0, 1, 1, 0] high[0, 1, 1, 0]
// CHECK: [[CONST:%.+]] = constant 0 // CHECK: [[CONST:%.+]] = constant 0
// CHECK: [[INIT:%.+]] = linalg.init_tensor [1, 5, 33, 62] // CHECK: [[INIT:%.+]] = linalg.init_tensor [1, 5, 33, 62]
// CHECK: [[FILL:%.+]] = linalg.fill([[INIT]], [[CONST]]) // CHECK: [[FILL:%.+]] = linalg.fill([[CONST]], [[INIT]])
// CHECK: [[KERNEL:%.+]] = linalg.init_tensor [4, 4] // CHECK: [[KERNEL:%.+]] = linalg.init_tensor [4, 4]
// CHECK: [[POOL:%.+]] = linalg.pooling_nhwc_sum {dilations = dense<1> : vector<2xi64>, strides = dense<1> : vector<2xi64>} ins([[PAD]], [[KERNEL]] : tensor<1x8x36x62xf32>, tensor<4x4xf32>) outs([[FILL]] : tensor<1x5x33x62xf32>) // CHECK: [[POOL:%.+]] = linalg.pooling_nhwc_sum {dilations = dense<1> : vector<2xi64>, strides = dense<1> : vector<2xi64>} ins([[PAD]], [[KERNEL]] : tensor<1x8x36x62xf32>, tensor<4x4xf32>) outs([[FILL]] : tensor<1x5x33x62xf32>)
// CHECK: [[GENERIC:%.+]] = linalg.generic {indexing_maps = [#map], iterator_types = ["parallel", "parallel", "parallel", "parallel"]} outs([[POOL]] : tensor<1x5x33x62xf32>) // CHECK: [[GENERIC:%.+]] = linalg.generic {indexing_maps = [#map], iterator_types = ["parallel", "parallel", "parallel", "parallel"]} outs([[POOL]] : tensor<1x5x33x62xf32>)
// CHECK: [[ZERO:%.0]] = constant 0 // CHECK: [[ZERO:%.0]] = constant 0
// CHECK: [[ONE:%.+]] = constant 1 // CHECK: [[ONE:%.+]] = constant 1
// CHECK: [[HEIGHT:%.+]] = constant 4 // CHECK: [[HEIGHT:%.+]] = constant 4
// CHECK: [[WIDTH:%.+]] = constant 32 // CHECK: [[WIDTH:%.+]] = constant 32
// CHECK: [[IDX1:%.+]] = linalg.index 1 // CHECK: [[IDX1:%.+]] = linalg.index 1
▲ Show 20 Lines • Show All 390 Lines • Show Last 20 Lines

mlir/test/Dialect/Linalg/bufferize.mlir

Show First 20 Lines • Show All 238 Lines • ▼ Show 20 Lines
// ----- // -----
// CHECK-LABEL: func @bufferize_fill( // CHECK-LABEL: func @bufferize_fill(
// CHECK-SAME: %[[IN:.*]]: tensor<?xf32> // CHECK-SAME: %[[IN:.*]]: tensor<?xf32>
func @bufferize_fill(%arg0: tensor<?xf32>) -> tensor<?xf32> { func @bufferize_fill(%arg0: tensor<?xf32>) -> tensor<?xf32> {
%c0 = constant 0.0 : f32 %c0 = constant 0.0 : f32
// CHECK: %[[MEMREF:.*]] = memref.buffer_cast %[[IN]] : memref<?xf32> // CHECK: %[[MEMREF:.*]] = memref.buffer_cast %[[IN]] : memref<?xf32>
// CHECK: linalg.fill(%[[MEMREF]], %cst) : memref<?xf32>, f32 // CHECK: linalg.fill(%cst, %[[MEMREF]]) : f32, memref<?xf32>
// CHECK: %[[TENSOR:.*]] = memref.tensor_load %[[MEMREF]] : memref<?xf32> // CHECK: %[[TENSOR:.*]] = memref.tensor_load %[[MEMREF]] : memref<?xf32>
// CHECK: return %[[TENSOR]] // CHECK: return %[[TENSOR]]
%0 = linalg.fill(%arg0, %c0) : tensor<?xf32>, f32 -> tensor<?xf32> %0 = linalg.fill(%c0, %arg0) : f32, tensor<?xf32> -> tensor<?xf32>
return %0 : tensor<?xf32> return %0 : tensor<?xf32>
} }
// ----- // -----
// CHECK-LABEL: func @bufferize_tensor_collapse_shape( // CHECK-LABEL: func @bufferize_tensor_collapse_shape(
// CHECK-SAME: %[[IN:.*]]: tensor<4x5xf32> // CHECK-SAME: %[[IN:.*]]: tensor<4x5xf32>
func @bufferize_tensor_collapse_shape(%arg0: tensor<4x5xf32>) -> tensor<20xf32> { func @bufferize_tensor_collapse_shape(%arg0: tensor<4x5xf32>) -> tensor<20xf32> {
Show All 9 Lines

mlir/test/Dialect/Linalg/canonicalize.mlir

Show All 29 Linesfunc @memref_cast_into_tiled_loop(%arg0: memref<192xf32>) {
%c192 = constant 192 : index %c192 = constant 192 : index
// CHECK: linalg.tiled_loop // CHECK: linalg.tiled_loop
// CHECK-SAME: outs (%{{.*}} = %{{.*}}: memref<192xf32>) // CHECK-SAME: outs (%{{.*}} = %{{.*}}: memref<192xf32>)
linalg.tiled_loop (%arg3) = (%c0) to (%c192) step (%c24) linalg.tiled_loop (%arg3) = (%c0) to (%c192) step (%c24)
outs (%out = %0: memref<192xf32, #map>) { outs (%out = %0: memref<192xf32, #map>) {
%14 = affine.min affine_map<(d0) -> (-d0 + 192, 24)>(%arg3) %14 = affine.min affine_map<(d0) -> (-d0 + 192, 24)>(%arg3)
%16 = memref.subview %out[%arg3] [%14] [1] %16 = memref.subview %out[%arg3] [%14] [1]
: memref<192xf32, #map> to memref<?xf32, #map> : memref<192xf32, #map> to memref<?xf32, #map>
linalg.fill(%16, %cst) : memref<?xf32, #map>, f32 linalg.fill(%cst, %16) : f32, memref<?xf32, #map>
linalg.yield linalg.yield
} }
return return
} }
// ----- // -----
// CHECK-LABEL: zero_rank_reshape_multi // CHECK-LABEL: zero_rank_reshape_multi
▲ Show 20 Lines • Show All 632 Lines • ▼ Show 20 Lines
// CHECK-NOT: linalg.generic // CHECK-NOT: linalg.generic
// CHECK-NOT: linalg.pad_tensor // CHECK-NOT: linalg.pad_tensor
// CHECK: return // CHECK: return
func @dead_linalg_tensor(%arg0 : tensor<7x7xi32>, %arg1 : tensor<7x7xf32>, func @dead_linalg_tensor(%arg0 : tensor<7x7xi32>, %arg1 : tensor<7x7xf32>,
%arg2: tensor<?x?xf32>, %high : index) { %arg2: tensor<?x?xf32>, %high : index) {
%c0_i32 = constant 0 : i32 %c0_i32 = constant 0 : i32
%c0 = constant 0 : index %c0 = constant 0 : index
%cst = constant 0.000000e+00 : f32 %cst = constant 0.000000e+00 : f32
%0 = linalg.fill(%arg0, %c0_i32) : tensor<7x7xi32>, i32 -> tensor<7x7xi32> %0 = linalg.fill(%c0_i32, %arg0) : i32, tensor<7x7xi32> -> tensor<7x7xi32>
%1 = linalg.matmul ins(%arg1, %arg1: tensor<7x7xf32>, tensor<7x7xf32>) %1 = linalg.matmul ins(%arg1, %arg1: tensor<7x7xf32>, tensor<7x7xf32>)
outs(%arg1: tensor<7x7xf32>) -> tensor<7x7xf32> outs(%arg1: tensor<7x7xf32>) -> tensor<7x7xf32>
%2 = linalg.generic #trait outs(%arg0 : tensor<7x7xi32>) { %2 = linalg.generic #trait outs(%arg0 : tensor<7x7xi32>) {
^bb(%3: i32) : ^bb(%3: i32) :
linalg.yield %3 : i32 linalg.yield %3 : i32
} -> tensor<7x7xi32> } -> tensor<7x7xi32>
%3 = linalg.pad_tensor %arg2 low[%c0, %c0] high[%high, %high] { %3 = linalg.pad_tensor %arg2 low[%c0, %c0] high[%high, %high] {
^bb0(%arg9: index, %arg10: index): // no predecessors ^bb0(%arg9: index, %arg10: index): // no predecessors
Show All 19 Lines
} }
func @propogate_casts(%arg0 : tensor<?x?xf32>, %arg1 : f32, %arg2 : index, func @propogate_casts(%arg0 : tensor<?x?xf32>, %arg1 : f32, %arg2 : index,
%arg3 : index) -> tensor<?x?xf32> { %arg3 : index) -> tensor<?x?xf32> {
%c0 = constant 0 : index %c0 = constant 0 : index
%c1 = constant 1 : index %c1 = constant 1 : index
%c21 = constant 21 : index %c21 = constant 21 : index
%c42 = constant 42 : index %c42 = constant 42 : index
%0 = linalg.init_tensor [%c21, %c42] : tensor<?x?xf32> %0 = linalg.init_tensor [%c21, %c42] : tensor<?x?xf32>
%1 = linalg.fill(%0, %arg1) : tensor<?x?xf32>, f32 -> tensor<?x?xf32> %1 = linalg.fill(%arg1, %0) : f32, tensor<?x?xf32> -> tensor<?x?xf32>
%2 = memref.dim %arg0, %c0 : tensor<?x?xf32> %2 = memref.dim %arg0, %c0 : tensor<?x?xf32>
%3 = memref.dim %arg0, %c1 : tensor<?x?xf32> %3 = memref.dim %arg0, %c1 : tensor<?x?xf32>
%4 = tensor.insert_slice %arg0 into %1[%arg2, %arg3] [%2, %3] [1, 1] : tensor<?x?xf32> into tensor<?x?xf32> %4 = tensor.insert_slice %arg0 into %1[%arg2, %arg3] [%2, %3] [1, 1] : tensor<?x?xf32> into tensor<?x?xf32>
return %4 : tensor<?x?xf32> return %4 : tensor<?x?xf32>
} }
// CHECK-LABEL: func @propogate_casts // CHECK-LABEL: func @propogate_casts
// CHECK: %[[INIT:.+]] = linalg.init_tensor [21, 42] // CHECK: %[[INIT:.+]] = linalg.init_tensor [21, 42]
// CHECK: %[[FILL:.+]] = linalg.fill(%[[INIT]], %{{.+}}) // CHECK: %[[FILL:.+]] = linalg.fill(%{{.+}}, %[[INIT]])
// CHECK: %[[INSERTED:.+]] = tensor.insert_slice %{{.+}} into %[[FILL]] // CHECK: %[[INSERTED:.+]] = tensor.insert_slice %{{.+}} into %[[FILL]]
// CHECK: %[[RESULT:.+]] = tensor.cast %[[INSERTED]] // CHECK: %[[RESULT:.+]] = tensor.cast %[[INSERTED]]
// CHECK: return %[[RESULT]] // CHECK: return %[[RESULT]]
// ----- // -----
// CHECK-LABEL: @self_copy // CHECK-LABEL: @self_copy
func @self_copy(%arg0 : memref<2x3x?x4xf32>) { func @self_copy(%arg0 : memref<2x3x?x4xf32>) {
// CHECK-NOT: linalg.copy // CHECK-NOT: linalg.copy
linalg.copy(%arg0, %arg0): memref<2x3x?x4xf32>, memref<2x3x?x4xf32> linalg.copy(%arg0, %arg0): memref<2x3x?x4xf32>, memref<2x3x?x4xf32>
// CHECK: return // CHECK: return
return return
} }
// ----- // -----
// CHECK-LABEL: func @fold_fill_reshape() // CHECK-LABEL: func @fold_fill_reshape()
func @fold_fill_reshape() -> tensor<6x4xf32> { func @fold_fill_reshape() -> tensor<6x4xf32> {
%zero = constant 0.0 : f32 %zero = constant 0.0 : f32
// CHECK: %[[INIT:.+]] = linalg.init_tensor [6, 4] : tensor<6x4xf32> // CHECK: %[[INIT:.+]] = linalg.init_tensor [6, 4] : tensor<6x4xf32>
%init = linalg.init_tensor [1, 2, 3, 4] : tensor<1x2x3x4xf32> %init = linalg.init_tensor [1, 2, 3, 4] : tensor<1x2x3x4xf32>
// CHECK: %[[FILL:.+]] = linalg.fill(%[[INIT]], %cst) : tensor<6x4xf32>, f32 -> tensor<6x4xf32> // CHECK: %[[FILL:.+]] = linalg.fill(%cst, %[[INIT]]) : f32, tensor<6x4xf32> -> tensor<6x4xf32>
%fill = linalg.fill(%init, %zero) : tensor<1x2x3x4xf32>, f32 -> tensor<1x2x3x4xf32> %fill = linalg.fill(%zero, %init) : f32, tensor<1x2x3x4xf32> -> tensor<1x2x3x4xf32>
%reshape = linalg.tensor_collapse_shape %fill [[0, 1, 2], [3]] %reshape = linalg.tensor_collapse_shape %fill [[0, 1, 2], [3]]
: tensor<1x2x3x4xf32> into tensor<6x4xf32> : tensor<1x2x3x4xf32> into tensor<6x4xf32>
// CHECK: return %[[FILL]] : tensor<6x4xf32> // CHECK: return %[[FILL]] : tensor<6x4xf32>
return %reshape : tensor<6x4xf32> return %reshape : tensor<6x4xf32>
} }
// ----- // -----
// CHECK: func @fold_fill_reshape_dynamic // CHECK: func @fold_fill_reshape_dynamic
// CHECK-SAME: %[[ARG0:.+]]: tensor<?x?x?x?x?xf32> // CHECK-SAME: %[[ARG0:.+]]: tensor<?x?x?x?x?xf32>
func @fold_fill_reshape_dynamic(%arg0 : tensor<?x?x?x?x?xf32>) -> tensor<?x?xf32> { func @fold_fill_reshape_dynamic(%arg0 : tensor<?x?x?x?x?xf32>) -> tensor<?x?xf32> {
%zero = constant 0.0 : f32 %zero = constant 0.0 : f32
// CHECK: %[[RESHAPE:.+]] = linalg.tensor_collapse_shape %[[ARG0]] // CHECK: %[[RESHAPE:.+]] = linalg.tensor_collapse_shape %[[ARG0]]
%0 = linalg.fill(%arg0, %zero) : tensor<?x?x?x?x?xf32>, f32 -> tensor<?x?x?x?x?xf32> %0 = linalg.fill(%zero, %arg0) : f32, tensor<?x?x?x?x?xf32> -> tensor<?x?x?x?x?xf32>
// CHECK: %[[RESULT:.+]] = linalg.fill(%[[RESHAPE]], %{{.+}}) // CHECK: %[[RESULT:.+]] = linalg.fill(%{{.+}}, %[[RESHAPE]])
%1 = linalg.tensor_collapse_shape %0 [[0, 1, 2], [3, 4]] %1 = linalg.tensor_collapse_shape %0 [[0, 1, 2], [3, 4]]
: tensor<?x?x?x?x?xf32> into tensor<?x?xf32> : tensor<?x?x?x?x?xf32> into tensor<?x?xf32>
// CHECK: return %[[RESULT]] // CHECK: return %[[RESULT]]
return %1 : tensor<?x?xf32> return %1 : tensor<?x?xf32>
} }
// ----- // -----
▲ Show 20 Lines • Show All 96 Lines • Show Last 20 Lines

mlir/test/Dialect/Linalg/comprehensive-func-bufferize-analysis.mlir

Show First 20 Lines • Show All 349 Lines • ▼ Show 20 Linesfunc @nested_extract_slice_and_insert(
// CHECK-NEXT: fill // CHECK-NEXT: fill
// CHECK-SAME: {__inplace_results_attr__ = ["true"]} // CHECK-SAME: {__inplace_results_attr__ = ["true"]}
// CHECK-NEXT: tensor.insert_slice // CHECK-NEXT: tensor.insert_slice
// CHECK-SAME: {__inplace_results_attr__ = ["true"]} // CHECK-SAME: {__inplace_results_attr__ = ["true"]}
// CHECK-NEXT: tensor.insert_slice // CHECK-NEXT: tensor.insert_slice
// CHECK-SAME: {__inplace_results_attr__ = ["false"]} // CHECK-SAME: {__inplace_results_attr__ = ["false"]}
%sA = tensor.extract_slice %A[0, 0][%idx, %idx][1, 1] : tensor<?x?xf32> to tensor<?x?xf32> %sA = tensor.extract_slice %A[0, 0][%idx, %idx][1, 1] : tensor<?x?xf32> to tensor<?x?xf32>
%ssA = tensor.extract_slice %sA[0, 0][4, 4][1, 1] : tensor<?x?xf32> to tensor<4x4xf32> %ssA = tensor.extract_slice %sA[0, 0][4, 4][1, 1] : tensor<?x?xf32> to tensor<4x4xf32>
%FA = linalg.fill(%ssA, %f0) : tensor<4x4xf32>, f32 -> tensor<4x4xf32> %FA = linalg.fill(%f0, %ssA) : f32, tensor<4x4xf32> -> tensor<4x4xf32>
%rsA = tensor.insert_slice %FA into %sA[0, 0][4, 4][1, 1] : tensor<4x4xf32> into tensor<?x?xf32> %rsA = tensor.insert_slice %FA into %sA[0, 0][4, 4][1, 1] : tensor<4x4xf32> into tensor<?x?xf32>
%rA = tensor.insert_slice %rsA into %A[0, 0][%idx, %idx][1, 1] : tensor<?x?xf32> into tensor<?x?xf32> %rA = tensor.insert_slice %rsA into %A[0, 0][%idx, %idx][1, 1] : tensor<?x?xf32> into tensor<?x?xf32>
// 3-level matching tensor.extract_slice / tensor.insert_slice into // 3-level matching tensor.extract_slice / tensor.insert_slice into
// inplaceable %B. // inplaceable %B.
// CHECK-NEXT: tensor.extract_slice // CHECK-NEXT: tensor.extract_slice
// CHECK-SAME: {__inplace_results_attr__ = ["true"]} // CHECK-SAME: {__inplace_results_attr__ = ["true"]}
// CHECK-NEXT: tensor.extract_slice // CHECK-NEXT: tensor.extract_slice
Show All 10 Linesfunc @nested_extract_slice_and_insert(
// CHECK-SAME: {__inplace_results_attr__ = ["true"]} // CHECK-SAME: {__inplace_results_attr__ = ["true"]}
// CHECK-NEXT: tensor.insert_slice // CHECK-NEXT: tensor.insert_slice
// CHECK-SAME: {__inplace_results_attr__ = ["true"]} // CHECK-SAME: {__inplace_results_attr__ = ["true"]}
// CHECK-NEXT: tensor.insert_slice // CHECK-NEXT: tensor.insert_slice
// CHECK-SAME: {__inplace_results_attr__ = ["true"]} // CHECK-SAME: {__inplace_results_attr__ = ["true"]}
%sB = tensor.extract_slice %B[0, 0][%idx, %idx][1, 1] : tensor<?x?xf32> to tensor<?x?xf32> %sB = tensor.extract_slice %B[0, 0][%idx, %idx][1, 1] : tensor<?x?xf32> to tensor<?x?xf32>
%ssB = tensor.extract_slice %sB[0, 0][4, %idx][1, 1] : tensor<?x?xf32> to tensor<4x?xf32> %ssB = tensor.extract_slice %sB[0, 0][4, %idx][1, 1] : tensor<?x?xf32> to tensor<4x?xf32>
%sssB = tensor.extract_slice %ssB[0, 0][4, 4][1, 1] : tensor<4x?xf32> to tensor<4x4xf32> %sssB = tensor.extract_slice %ssB[0, 0][4, 4][1, 1] : tensor<4x?xf32> to tensor<4x4xf32>
%FB = linalg.fill(%sssB, %f0) : tensor<4x4xf32>, f32 -> tensor<4x4xf32> %FB = linalg.fill(%f0, %sssB) : f32, tensor<4x4xf32> -> tensor<4x4xf32>
%rssB = tensor.insert_slice %FB into %ssB[0, 0][4, 4][1, 1] : tensor<4x4xf32> into tensor<4x?xf32> %rssB = tensor.insert_slice %FB into %ssB[0, 0][4, 4][1, 1] : tensor<4x4xf32> into tensor<4x?xf32>
%rsB = tensor.insert_slice %rssB into %sB[0, 0][4, %idx][1, 1] : tensor<4x?xf32> into tensor<?x?xf32> %rsB = tensor.insert_slice %rssB into %sB[0, 0][4, %idx][1, 1] : tensor<4x?xf32> into tensor<?x?xf32>
%rB = tensor.insert_slice %rsB into %B[0, 0][%idx, %idx][1, 1] : tensor<?x?xf32> into tensor<?x?xf32> %rB = tensor.insert_slice %rsB into %B[0, 0][%idx, %idx][1, 1] : tensor<?x?xf32> into tensor<?x?xf32>
// 2-level matching tensor.extract_slice / tensor.insert_slice into // 2-level matching tensor.extract_slice / tensor.insert_slice into
// inplaceable %C with a twist. // inplaceable %C with a twist.
// Throw a wrench in the system: %rsC production sizes do not match %ssC. // Throw a wrench in the system: %rsC production sizes do not match %ssC.
// CHECK-NEXT: tensor.extract_slice // CHECK-NEXT: tensor.extract_slice
// CHECK-SAME: {__inplace_results_attr__ = ["true"]} // CHECK-SAME: {__inplace_results_attr__ = ["true"]}
// The tensor.insert_slice that would be candidate for matching does not actually // The tensor.insert_slice that would be candidate for matching does not actually
// match. That tensor.insert_slice can still be bufferized inplace nonetheless // match. That tensor.insert_slice can still be bufferized inplace nonetheless
// but this tensor.extract_slice, which bufferizes to an inplace write, cannot. // but this tensor.extract_slice, which bufferizes to an inplace write, cannot.
// CHECK-NEXT: tensor.extract_slice // CHECK-NEXT: tensor.extract_slice
// CHECK-SAME: {__inplace_results_attr__ = ["false"]} // CHECK-SAME: {__inplace_results_attr__ = ["false"]}
// CHECK-NEXT: fill // CHECK-NEXT: fill
// CHECK-SAME: {__inplace_results_attr__ = ["true"]} // CHECK-SAME: {__inplace_results_attr__ = ["true"]}
// CHECK-NEXT: tensor.insert_slice // CHECK-NEXT: tensor.insert_slice
// CHECK-SAME: {__inplace_results_attr__ = ["true"]} // CHECK-SAME: {__inplace_results_attr__ = ["true"]}
// CHECK-NEXT: tensor.insert_slice // CHECK-NEXT: tensor.insert_slice
// CHECK-SAME: {__inplace_results_attr__ = ["true"]} // CHECK-SAME: {__inplace_results_attr__ = ["true"]}
%sC = tensor.extract_slice %C[0, 0][%idx, %idx][1, 1] : tensor<?x?xf32> to tensor<?x?xf32> %sC = tensor.extract_slice %C[0, 0][%idx, %idx][1, 1] : tensor<?x?xf32> to tensor<?x?xf32>
%ssC = tensor.extract_slice %sC[0, 0][4, 4][1, 1] : tensor<?x?xf32> to tensor<4x4xf32> %ssC = tensor.extract_slice %sC[0, 0][4, 4][1, 1] : tensor<?x?xf32> to tensor<4x4xf32>
%FC = linalg.fill(%ssC, %f0) : tensor<4x4xf32>, f32 -> tensor<4x4xf32> %FC = linalg.fill(%f0, %ssC) : f32, tensor<4x4xf32> -> tensor<4x4xf32>
%rsC = tensor.insert_slice %FC into %sC[0, 0][12345, 67890][1, 1] : tensor<4x4xf32> into tensor<?x?xf32> %rsC = tensor.insert_slice %FC into %sC[0, 0][12345, 67890][1, 1] : tensor<4x4xf32> into tensor<?x?xf32>
%rC = tensor.insert_slice %rsC into %C[0, 0][%idx, %idx][1, 1] : tensor<?x?xf32> into tensor<?x?xf32> %rC = tensor.insert_slice %rsC into %C[0, 0][%idx, %idx][1, 1] : tensor<?x?xf32> into tensor<?x?xf32>
return %rA, %rB, %rC: tensor<?x?xf32>, tensor<?x?xf32>, tensor<?x?xf32> return %rA, %rB, %rC: tensor<?x?xf32>, tensor<?x?xf32>, tensor<?x?xf32>
} }

mlir/test/Dialect/Linalg/comprehensive-func-bufferize.mlir

// RUN: mlir-opt %s -linalg-comprehensive-func-bufferize -split-input-file | FileCheck %s // RUN: mlir-opt %s -linalg-comprehensive-func-bufferize -split-input-file | FileCheck %s
// CHECK-DAG: #[[$map_2d_dyn:.*]] = affine_map<(d0)[s0, s1] -> (d0 * s1 + s0)> // CHECK-DAG: #[[$map_2d_dyn:.*]] = affine_map<(d0)[s0, s1] -> (d0 * s1 + s0)>
// CHECK-LABEL: func @fill_inplace( // CHECK-LABEL: func @fill_inplace(
// CHECK-SAME: %[[A:[a-zA-Z0-9]*]]: tensor<?xf32> {linalg.inplaceable = true}) // CHECK-SAME: %[[A:[a-zA-Z0-9]*]]: tensor<?xf32> {linalg.inplaceable = true})
func @fill_inplace(%A : tensor<?xf32> {linalg.inplaceable = true}) -> tensor<?xf32> { func @fill_inplace(%A : tensor<?xf32> {linalg.inplaceable = true}) -> tensor<?xf32> {
// CHECK: %[[I:.*]] = memref.buffer_cast %[[A]] : memref<?xf32, #[[$map_2d_dyn]]> // CHECK: %[[I:.*]] = memref.buffer_cast %[[A]] : memref<?xf32, #[[$map_2d_dyn]]>
// CHECK: %[[F0:.*]] = constant 0.000000e+00 : f32 // CHECK: %[[F0:.*]] = constant 0.000000e+00 : f32
%f0 = constant 0.0 : f32 %f0 = constant 0.0 : f32
/// Inplaceable, no alloc /// Inplaceable, no alloc
// CHECK-NOT: alloc // CHECK-NOT: alloc
// CHECK: linalg.fill(%[[I]], %[[F0]]) : memref<?xf32, #[[$map_2d_dyn]]>, f32 // CHECK: linalg.fill(%[[F0]], %[[I]]) : f32, memref<?xf32, #[[$map_2d_dyn]]>
%r = linalg.fill(%A, %f0) : tensor<?xf32>, f32 -> tensor<?xf32> %r = linalg.fill(%f0, %A) : f32, tensor<?xf32> -> tensor<?xf32>
// CHECK: %[[R:.*]] = memref.tensor_load %[[I]] : memref<?xf32, #[[$map_2d_dyn]]> // CHECK: %[[R:.*]] = memref.tensor_load %[[I]] : memref<?xf32, #[[$map_2d_dyn]]>
// CHECK: return %[[R]] : tensor<?xf32> // CHECK: return %[[R]] : tensor<?xf32>
return %r: tensor<?xf32> return %r: tensor<?xf32>
} }
// ----- // -----
// CHECK-DAG: #[[$map_2d_dyn:.*]] = affine_map<(d0)[s0, s1] -> (d0 * s1 + s0)> // CHECK-DAG: #[[$map_2d_dyn:.*]] = affine_map<(d0)[s0, s1] -> (d0 * s1 + s0)>
/// No linalg.inplaceable flag, must allocate. /// No linalg.inplaceable flag, must allocate.
// CHECK-LABEL: func @not_inplace( // CHECK-LABEL: func @not_inplace(
// CHECK-SAME: %[[A:[a-zA-Z0-9]*]]: tensor<?xf32>) // CHECK-SAME: %[[A:[a-zA-Z0-9]*]]: tensor<?xf32>)
func @not_inplace(%A : tensor<?xf32>) -> tensor<?xf32> { func @not_inplace(%A : tensor<?xf32>) -> tensor<?xf32> {
// CHECK: %[[I:.*]] = memref.buffer_cast %[[A]] : memref<?xf32, #[[$map_2d_dyn]]> // CHECK: %[[I:.*]] = memref.buffer_cast %[[A]] : memref<?xf32, #[[$map_2d_dyn]]>
// CHECK: %[[D0:.*]] = memref.dim %[[I]], {{.*}} : memref<?xf32, #[[$map_2d_dyn]]> // CHECK: %[[D0:.*]] = memref.dim %[[I]], {{.*}} : memref<?xf32, #[[$map_2d_dyn]]>
// CHECK: %[[ALLOC:.*]] = memref.alloc(%[[D0]]) : memref<?xf32> // CHECK: %[[ALLOC:.*]] = memref.alloc(%[[D0]]) : memref<?xf32>
// CHECK: %[[I2:.*]] = memref.cast %[[ALLOC]] : memref<?xf32> to memref<?xf32, #map> // CHECK: %[[I2:.*]] = memref.cast %[[ALLOC]] : memref<?xf32> to memref<?xf32, #map>
// CHECK: %[[F0:.*]] = constant 0.000000e+00 : f32 // CHECK: %[[F0:.*]] = constant 0.000000e+00 : f32
%f0 = constant 0.0 : f32 %f0 = constant 0.0 : f32
// CHECK: linalg.fill(%[[I2]], %[[F0]]) : memref<?xf32, #[[$map_2d_dyn]]>, f32 // CHECK: linalg.fill(%[[F0]], %[[I2]]) : f32, memref<?xf32, #[[$map_2d_dyn]]>
%r = linalg.fill(%A, %f0) : tensor<?xf32>, f32 -> tensor<?xf32> %r = linalg.fill(%f0, %A) : f32, tensor<?xf32> -> tensor<?xf32>
// CHECK: dealloc %[[ALLOC]] : memref<?xf32> // CHECK: dealloc %[[ALLOC]] : memref<?xf32>
// CHECK: %[[R:.*]] = memref.tensor_load %[[I2]] : memref<?xf32, #[[$map_2d_dyn]]> // CHECK: %[[R:.*]] = memref.tensor_load %[[I2]] : memref<?xf32, #[[$map_2d_dyn]]>
// CHECK: return %[[R]] : tensor<?xf32> // CHECK: return %[[R]] : tensor<?xf32>
return %r: tensor<?xf32> return %r: tensor<?xf32>
} }
// ----- // -----
// CHECK-LABEL: func @not_inplace // CHECK-LABEL: func @not_inplace
// CHECK-SAME: %[[A:[a-zA-Z0-9]*]]: tensor<?x?xf32> // CHECK-SAME: %[[A:[a-zA-Z0-9]*]]: tensor<?x?xf32>
func @not_inplace(%A : tensor<?x?xf32> {linalg.inplaceable = true}) -> tensor<?x?xf32> { func @not_inplace(%A : tensor<?x?xf32> {linalg.inplaceable = true}) -> tensor<?x?xf32> {
%f0 = constant 0.0 : f32 %f0 = constant 0.0 : f32
// CHECK: %[[BUFFER_CAST:.*]] = memref.buffer_cast %[[A]] : memref<?x?xf32 // CHECK: %[[BUFFER_CAST:.*]] = memref.buffer_cast %[[A]] : memref<?x?xf32
/// Cross-op multiple uses of %A, the first op which has interfering reads must alloc. /// Cross-op multiple uses of %A, the first op which has interfering reads must alloc.
// CHECK: %[[ALLOC:.*]] = memref.alloc // CHECK: %[[ALLOC:.*]] = memref.alloc
// CHECK: %[[CAST:.*]] = memref.cast %[[ALLOC]] // CHECK: %[[CAST:.*]] = memref.cast %[[ALLOC]]
// CHECK: linalg.fill(%[[CAST]] // CHECK: linalg.fill({{.*}}, %[[CAST]]
%f = linalg.fill(%A, %f0) : tensor<?x?xf32>, f32 -> tensor<?x?xf32> %f = linalg.fill(%f0, %A) : f32, tensor<?x?xf32> -> tensor<?x?xf32>
/// The second op has no interfering reads and can reuse. /// The second op has no interfering reads and can reuse.
// CHECK-NOT: alloc // CHECK-NOT: alloc
// CHECK: linalg.matmul{{.*}}outs(%[[BUFFER_CAST]] // CHECK: linalg.matmul{{.*}}outs(%[[BUFFER_CAST]]
%r = linalg.matmul ins(%f, %f: tensor<?x?xf32>, tensor<?x?xf32>) %r = linalg.matmul ins(%f, %f: tensor<?x?xf32>, tensor<?x?xf32>)
outs(%A: tensor<?x?xf32>) outs(%A: tensor<?x?xf32>)
-> tensor<?x?xf32> -> tensor<?x?xf32>
return %r: tensor<?x?xf32> return %r: tensor<?x?xf32>
▲ Show 20 Lines • Show All 99 Lines • ▼ Show 20 Linesfunc @insert_slice_fun(%A : tensor<?xf32> {linalg.inplaceable = true}, %t : tensor<4xf32>)
// CHECK: %[[BUFFER_CAST_B:.*]] = memref.buffer_cast {{.*}} : memref<4xf32 // CHECK: %[[BUFFER_CAST_B:.*]] = memref.buffer_cast {{.*}} : memref<4xf32
// CHECK-NOT: alloc // CHECK-NOT: alloc
// CHECK: %[[SV:.*]] = memref.subview %[[BUFFER_CAST_A]] // CHECK: %[[SV:.*]] = memref.subview %[[BUFFER_CAST_A]]
// CHECK: linalg.copy(%[[BUFFER_CAST_B]], %[[SV]]) // CHECK: linalg.copy(%[[BUFFER_CAST_B]], %[[SV]])
%r0 = tensor.insert_slice %t into %A[0][4][1] : tensor<4xf32> into tensor<?xf32> %r0 = tensor.insert_slice %t into %A[0][4][1] : tensor<4xf32> into tensor<?xf32>
/// Overwrite BUFFER_CAST_A inplace. /// Overwrite BUFFER_CAST_A inplace.
// CHECK: linalg.fill(%[[BUFFER_CAST_A]] // CHECK: linalg.fill({{.*}}, %[[BUFFER_CAST_A]]
%r1 = linalg.fill(%r0, %f0) : tensor<?xf32>, f32 -> tensor<?xf32> %r1 = linalg.fill(%f0, %r0) : f32, tensor<?xf32> -> tensor<?xf32>
return %r1: tensor<?xf32> return %r1: tensor<?xf32>
} }
// ----- // -----
// CHECK-LABEL: func @insert_slice_fun // CHECK-LABEL: func @insert_slice_fun
func @insert_slice_fun(%A : tensor<?xf32> {linalg.inplaceable = true}, %t : tensor<4xf32>) func @insert_slice_fun(%A : tensor<?xf32> {linalg.inplaceable = true}, %t : tensor<4xf32>)
-> tensor<?xf32> -> tensor<?xf32>
{ {
%f0 = constant 0.0 : f32 %f0 = constant 0.0 : f32
// CHECK: %[[BUFFER_CAST_A:.*]] = memref.buffer_cast {{.*}} : memref<?xf32 // CHECK: %[[BUFFER_CAST_A:.*]] = memref.buffer_cast {{.*}} : memref<?xf32
// CHECK: %[[BUFFER_CAST_B:.*]] = memref.buffer_cast {{.*}} : memref<4xf32 // CHECK: %[[BUFFER_CAST_B:.*]] = memref.buffer_cast {{.*}} : memref<4xf32
// CHECK: linalg.fill(%[[BUFFER_CAST_A]] // CHECK: linalg.fill({{.*}}, %[[BUFFER_CAST_A]]
%r0 = linalg.fill(%A, %f0) : tensor<?xf32>, f32 -> tensor<?xf32> %r0 = linalg.fill(%f0, %A) : f32, tensor<?xf32> -> tensor<?xf32>
// CHECK-NOT: alloc // CHECK-NOT: alloc
// CHECK: %[[SV:.*]] = memref.subview %[[BUFFER_CAST_A]] // CHECK: %[[SV:.*]] = memref.subview %[[BUFFER_CAST_A]]
/// Overwrite BUFFER_CAST_A inplace by copying into the subview. /// Overwrite BUFFER_CAST_A inplace by copying into the subview.
// CHECK: linalg.copy(%[[BUFFER_CAST_B]], %[[SV]]) // CHECK: linalg.copy(%[[BUFFER_CAST_B]], %[[SV]])
%r1 = tensor.insert_slice %t into %r0[0][4][1] : tensor<4xf32> into tensor<?xf32> %r1 = tensor.insert_slice %t into %r0[0][4][1] : tensor<4xf32> into tensor<?xf32>
return %r1: tensor<?xf32> return %r1: tensor<?xf32>
Show All 32 Linesfunc @insert_slice_fun_not_inplace(%A : tensor<?xf32> {linalg.inplaceable = true}, %t : tensor<4xf32>)
// CHECK-DAG: %[[SV:.*]] = memref.subview %[[BUFFER_CAST_A]][0] [4] [1] : memref<?xf32, {{.*}}> to memref<4xf32, {{.*}}> // CHECK-DAG: %[[SV:.*]] = memref.subview %[[BUFFER_CAST_A]][0] [4] [1] : memref<?xf32, {{.*}}> to memref<4xf32, {{.*}}>
// CHECK-DAG: linalg.copy(%[[BUFFER_CAST_B]], %[[SV]]) : memref<4xf32, {{.*}}>, memref<4xf32, {{.*}}> // CHECK-DAG: linalg.copy(%[[BUFFER_CAST_B]], %[[SV]]) : memref<4xf32, {{.*}}>, memref<4xf32, {{.*}}>
%r0 = tensor.insert_slice %t into %A[0][4][1] : tensor<4xf32> into tensor<?xf32> %r0 = tensor.insert_slice %t into %A[0][4][1] : tensor<4xf32> into tensor<?xf32>
// fill would interfere with %r0 that is also being returned. // fill would interfere with %r0 that is also being returned.
// So we need to bufferize it out of place and make a new alloc. // So we need to bufferize it out of place and make a new alloc.
// CHECK-DAG: %[[ALLOC:.*]] = memref.alloc({{.*}}) : memref<?xf32> // CHECK-DAG: %[[ALLOC:.*]] = memref.alloc({{.*}}) : memref<?xf32>
// CHECK-DAG: %[[ALLOC_CAST_DYNAMIC:.*]] = memref.cast %[[ALLOC]] : memref<?xf32> to memref<?xf32, {{.*}} // CHECK-DAG: %[[ALLOC_CAST_DYNAMIC:.*]] = memref.cast %[[ALLOC]] : memref<?xf32> to memref<?xf32, {{.*}}
// CHECK: linalg.fill(%[[ALLOC_CAST_DYNAMIC]] // CHECK: linalg.fill(%{{.*}}, %[[ALLOC_CAST_DYNAMIC]]
// CHECK: memref.dealloc %[[ALLOC]] : memref<?xf32> // CHECK: memref.dealloc %[[ALLOC]] : memref<?xf32>
%r1 = linalg.fill(%A, %f0) : tensor<?xf32>, f32 -> tensor<?xf32> %r1 = linalg.fill(%f0, %A) : f32, tensor<?xf32> -> tensor<?xf32>
// CHECK-DAG: %[[RET_A:.*]] = memref.tensor_load %[[BUFFER_CAST_A]] : memref<?xf32, {{.*}} // CHECK-DAG: %[[RET_A:.*]] = memref.tensor_load %[[BUFFER_CAST_A]] : memref<?xf32, {{.*}}
// CHECK-DAG: %[[RET_B:.*]] = memref.tensor_load %[[ALLOC_CAST_DYNAMIC]] : memref<?xf32, {{.*}} // CHECK-DAG: %[[RET_B:.*]] = memref.tensor_load %[[ALLOC_CAST_DYNAMIC]] : memref<?xf32, {{.*}}
// CHECK: return %[[RET_B]], %[[RET_A]] // CHECK: return %[[RET_B]], %[[RET_A]]
return %r1, %r0: tensor<?xf32>, tensor<?xf32> return %r1, %r0: tensor<?xf32>, tensor<?xf32>
} }
// ----- // -----
Show All 21 Lines

mlir/test/Dialect/Linalg/drop-unit-extent-dims.mlir

Show First 20 Lines • Show All 260 Lines • ▼ Show 20 Lines
// CHECK: %[[RESULT:.+]] = linalg.generic // CHECK: %[[RESULT:.+]] = linalg.generic
// CHECK: return %[[RESULT]] // CHECK: return %[[RESULT]]
// ----- // -----
func @fold_unit_dim_for_init_tensor(%input: tensor<1x1000xf32>) -> tensor<1xf32> { func @fold_unit_dim_for_init_tensor(%input: tensor<1x1000xf32>) -> tensor<1xf32> {
%cst = constant 0.0 : f32 %cst = constant 0.0 : f32
%init = linalg.init_tensor [1] : tensor<1xf32> %init = linalg.init_tensor [1] : tensor<1xf32>
%fill = linalg.fill(%init, %cst) : tensor<1xf32>, f32 -> tensor<1xf32> %fill = linalg.fill(%cst, %init) : f32, tensor<1xf32> -> tensor<1xf32>
%add = linalg.generic { %add = linalg.generic {
indexing_maps = [affine_map<(d0, d1) -> (d0, d1)>, affine_map<(d0, d1) -> (d0)>], indexing_maps = [affine_map<(d0, d1) -> (d0, d1)>, affine_map<(d0, d1) -> (d0)>],
iterator_types = ["parallel", "reduction"]} iterator_types = ["parallel", "reduction"]}
ins(%input : tensor<1x1000xf32>)outs(%fill : tensor<1xf32>) { ins(%input : tensor<1x1000xf32>)outs(%fill : tensor<1xf32>) {
^bb0(%arg1: f32, %arg2: f32): ^bb0(%arg1: f32, %arg2: f32):
%1823 = addf %arg1, %arg2 : f32 %1823 = addf %arg1, %arg2 : f32
linalg.yield %1823 : f32 linalg.yield %1823 : f32
} -> tensor<1xf32> } -> tensor<1xf32>
return %add : tensor<1xf32> return %add : tensor<1xf32>
} }
// CHECK-DAG: #[[MAP1:.+]] = affine_map<(d0) -> (d0)> // CHECK-DAG: #[[MAP1:.+]] = affine_map<(d0) -> (d0)>
// CHECK-DAG: #[[MAP2:.+]] = affine_map<(d0) -> ()> // CHECK-DAG: #[[MAP2:.+]] = affine_map<(d0) -> ()>
// CHECK: func @fold_unit_dim_for_init_tensor // CHECK: func @fold_unit_dim_for_init_tensor
// CHECK: %[[INPUT_RESHAPE:.+]] = linalg.tensor_collapse_shape %{{.+}} {{\[}}[0, 1]] : tensor<1x1000xf32> into tensor<1000xf32> // CHECK: %[[INPUT_RESHAPE:.+]] = linalg.tensor_collapse_shape %{{.+}} {{\[}}[0, 1]] : tensor<1x1000xf32> into tensor<1000xf32>
// CHECK: %[[INIT:.+]] = linalg.init_tensor [] : tensor<f32> // CHECK: %[[INIT:.+]] = linalg.init_tensor [] : tensor<f32>
// CHECK: %[[FILL:.+]] = linalg.fill(%[[INIT]], %cst) : tensor<f32>, f32 -> tensor<f32> // CHECK: %[[FILL:.+]] = linalg.fill(%cst, %[[INIT]]) : f32, tensor<f32> -> tensor<f32>
// CHECK: %[[GENERIC:.+]] = linalg.generic // CHECK: %[[GENERIC:.+]] = linalg.generic
// CHECK-SAME: indexing_maps = [#[[MAP1]], #[[MAP2]]] // CHECK-SAME: indexing_maps = [#[[MAP1]], #[[MAP2]]]
// CHECK-SAME: iterator_types = ["reduction"] // CHECK-SAME: iterator_types = ["reduction"]
// CHECK-SAME: ins(%[[INPUT_RESHAPE]] : tensor<1000xf32>) // CHECK-SAME: ins(%[[INPUT_RESHAPE]] : tensor<1000xf32>)
// CHECK-SAME: outs(%[[FILL]] : tensor<f32>) // CHECK-SAME: outs(%[[FILL]] : tensor<f32>)
// CHECK: %[[GENERIC_RESHAPE:.+]] = linalg.tensor_expand_shape %[[GENERIC]] [] : tensor<f32> into tensor<1xf32> // CHECK: %[[GENERIC_RESHAPE:.+]] = linalg.tensor_expand_shape %[[GENERIC]] [] : tensor<f32> into tensor<1xf32>
// CHECK: return %[[GENERIC_RESHAPE:.+]] : tensor<1xf32> // CHECK: return %[[GENERIC_RESHAPE:.+]] : tensor<1xf32>
Show All 27 Lines
// ----- // -----
func @unit_dim_for_reduction(%arg0: tensor<1x?x1x?xf32>) -> tensor<1x?xf32> { func @unit_dim_for_reduction(%arg0: tensor<1x?x1x?xf32>) -> tensor<1x?xf32> {
%cst = constant 1.000000e+00 : f32 %cst = constant 1.000000e+00 : f32
%c3 = constant 3 : index %c3 = constant 3 : index
%0 = memref.dim %arg0, %c3 : tensor<1x?x1x?xf32> %0 = memref.dim %arg0, %c3 : tensor<1x?x1x?xf32>
%1 = linalg.init_tensor [1, %0] : tensor<1x?xf32> %1 = linalg.init_tensor [1, %0] : tensor<1x?xf32>
%2 = linalg.fill(%1, %cst) : tensor<1x?xf32>, f32 -> tensor<1x?xf32> %2 = linalg.fill(%cst, %1) : f32, tensor<1x?xf32> -> tensor<1x?xf32>
%3 = linalg.generic { %3 = linalg.generic {
indexing_maps = [affine_map<(d0, d1, d2, d3) -> (d0, d1, d2, d3)>, indexing_maps = [affine_map<(d0, d1, d2, d3) -> (d0, d1, d2, d3)>,
affine_map<(d0, d1, d2, d3) -> (d0, d1)>], affine_map<(d0, d1, d2, d3) -> (d0, d1)>],
iterator_types = ["parallel", "parallel", "reduction", "reduction"]} iterator_types = ["parallel", "parallel", "reduction", "reduction"]}
ins(%arg0 : tensor<1x?x1x?xf32>) ins(%arg0 : tensor<1x?x1x?xf32>)
outs(%2 : tensor<1x?xf32>) { outs(%2 : tensor<1x?xf32>) {
^bb0(%arg1: f32, %arg2: f32): // no predecessors ^bb0(%arg1: f32, %arg2: f32): // no predecessors
%4 = addf %arg1, %arg2 : f32 %4 = addf %arg1, %arg2 : f32
linalg.yield %4 : f32 linalg.yield %4 : f32
} -> tensor<1x?xf32> } -> tensor<1x?xf32>
return %3 : tensor<1x?xf32> return %3 : tensor<1x?xf32>
} }
// CHECK-DAG: #[[MAP2:.+]] = affine_map<(d0, d1) -> (d0, d1)> // CHECK-DAG: #[[MAP2:.+]] = affine_map<(d0, d1) -> (d0, d1)>
// CHECK-DAG: #[[MAP3:.+]] = affine_map<(d0, d1) -> (d0)> // CHECK-DAG: #[[MAP3:.+]] = affine_map<(d0, d1) -> (d0)>
// CHECK: func @unit_dim_for_reduction // CHECK: func @unit_dim_for_reduction
// CHECK-SAME: %[[ARG0:.+]]: tensor<1x?x1x?xf32> // CHECK-SAME: %[[ARG0:.+]]: tensor<1x?x1x?xf32>
// CHECK-DAG: %[[RESHAPE:.+]] = linalg.tensor_collapse_shape %[[ARG0]] {{\[}}[0, 1, 2], [3]] // CHECK-DAG: %[[RESHAPE:.+]] = linalg.tensor_collapse_shape %[[ARG0]] {{\[}}[0, 1, 2], [3]]
// CHECK: %[[INIT:.+]] = linalg.init_tensor [%{{.+}}] : tensor<?xf32> // CHECK: %[[INIT:.+]] = linalg.init_tensor [%{{.+}}] : tensor<?xf32>
// CHECK: %[[FILL:.+]] = linalg.fill(%[[INIT]], %{{.+}}) // CHECK: %[[FILL:.+]] = linalg.fill(%{{.+}}, %[[INIT]])
// CHECK: %[[RESULT:.+]] = linalg.generic // CHECK: %[[RESULT:.+]] = linalg.generic
// CHECK-SAME: indexing_maps = [#[[MAP2]], #[[MAP3]]] // CHECK-SAME: indexing_maps = [#[[MAP2]], #[[MAP3]]]
// CHECK-SAME: iterator_types = ["parallel", "reduction"] // CHECK-SAME: iterator_types = ["parallel", "reduction"]
// CHECK-SAME: ins(%[[RESHAPE]] : tensor<?x?xf32>) // CHECK-SAME: ins(%[[RESHAPE]] : tensor<?x?xf32>)
// CHECK-SAME: outs(%[[FILL]] : tensor<?xf32>) // CHECK-SAME: outs(%[[FILL]] : tensor<?xf32>)
// CHECK: %[[RESULT_RESHAPE:.+]] = linalg.tensor_expand_shape %[[RESULT]] {{\[}}[0, 1]] // CHECK: %[[RESULT_RESHAPE:.+]] = linalg.tensor_expand_shape %[[RESULT]] {{\[}}[0, 1]]
// CHECK: return %[[RESULT_RESHAPE]] // CHECK: return %[[RESULT_RESHAPE]]
// ----- // -----
func @unit_dim_for_reduction_keep_one(%arg0: tensor<1x?x1x1xf32>) -> tensor<1x1xf32> { func @unit_dim_for_reduction_keep_one(%arg0: tensor<1x?x1x1xf32>) -> tensor<1x1xf32> {
%cst = constant 1.000000e+00 : f32 %cst = constant 1.000000e+00 : f32
%c3 = constant 3 : index %c3 = constant 3 : index
%1 = linalg.init_tensor [1, 1] : tensor<1x1xf32> %1 = linalg.init_tensor [1, 1] : tensor<1x1xf32>
%2 = linalg.fill(%1, %cst) : tensor<1x1xf32>, f32 -> tensor<1x1xf32> %2 = linalg.fill(%cst, %1) : f32, tensor<1x1xf32> -> tensor<1x1xf32>
%3 = linalg.generic { %3 = linalg.generic {
indexing_maps = [affine_map<(d0, d1, d2, d3) -> (d0, d1, d2, d3)>, indexing_maps = [affine_map<(d0, d1, d2, d3) -> (d0, d1, d2, d3)>,
affine_map<(d0, d1, d2, d3) -> (d0, d1)>], affine_map<(d0, d1, d2, d3) -> (d0, d1)>],
iterator_types = ["parallel", "parallel", "reduction", "reduction"]} iterator_types = ["parallel", "parallel", "reduction", "reduction"]}
ins(%arg0 : tensor<1x?x1x1xf32>) ins(%arg0 : tensor<1x?x1x1xf32>)
outs(%2 : tensor<1x1xf32>) { outs(%2 : tensor<1x1xf32>) {
^bb0(%arg1: f32, %arg2: f32): // no predecessors ^bb0(%arg1: f32, %arg2: f32): // no predecessors
%4 = addf %arg1, %arg2 : f32 %4 = addf %arg1, %arg2 : f32
linalg.yield %4 : f32 linalg.yield %4 : f32
} -> tensor<1x1xf32> } -> tensor<1x1xf32>
return %3 : tensor<1x1xf32> return %3 : tensor<1x1xf32>
} }
// CHECK-DAG: #[[MAP2:.+]] = affine_map<(d0, d1) -> (d0, d1)> // CHECK-DAG: #[[MAP2:.+]] = affine_map<(d0, d1) -> (d0, d1)>
// CHECK-DAG: #[[MAP3:.+]] = affine_map<(d0, d1) -> (d0)> // CHECK-DAG: #[[MAP3:.+]] = affine_map<(d0, d1) -> (d0)>
// CHECK: func @unit_dim_for_reduction_keep_one // CHECK: func @unit_dim_for_reduction_keep_one
// CHECK-SAME: %[[ARG0:.+]]: tensor<1x?x1x1xf32> // CHECK-SAME: %[[ARG0:.+]]: tensor<1x?x1x1xf32>
// CHECK-DAG: %[[RESHAPE:.+]] = linalg.tensor_collapse_shape %[[ARG0]] {{\[}}[0, 1, 2], [3]] // CHECK-DAG: %[[RESHAPE:.+]] = linalg.tensor_collapse_shape %[[ARG0]] {{\[}}[0, 1, 2], [3]]
// CHECK: %[[INIT:.+]] = linalg.init_tensor [1] : tensor<1xf32> // CHECK: %[[INIT:.+]] = linalg.init_tensor [1] : tensor<1xf32>
// CHECK: %[[FILL:.+]] = linalg.fill(%[[INIT]], %{{.+}}) // CHECK: %[[FILL:.+]] = linalg.fill(%{{.+}}, %[[INIT]])
// CHECK: %[[RESULT:.+]] = linalg.generic // CHECK: %[[RESULT:.+]] = linalg.generic
// CHECK-SAME: indexing_maps = [#[[MAP2]], #[[MAP3]]] // CHECK-SAME: indexing_maps = [#[[MAP2]], #[[MAP3]]]
// CHECK-SAME: iterator_types = ["parallel", "reduction"] // CHECK-SAME: iterator_types = ["parallel", "reduction"]
// CHECK-SAME: ins(%[[RESHAPE]] : tensor<?x1xf32>) // CHECK-SAME: ins(%[[RESHAPE]] : tensor<?x1xf32>)
// CHECK-SAME: outs(%[[FILL]] : tensor<1xf32>) // CHECK-SAME: outs(%[[FILL]] : tensor<1xf32>)
// CHECK: %[[RESULT_RESHAPE:.+]] = linalg.tensor_expand_shape %[[RESULT]] {{\[}}[0, 1]] // CHECK: %[[RESULT_RESHAPE:.+]] = linalg.tensor_expand_shape %[[RESULT]] {{\[}}[0, 1]]
// CHECK: return %[[RESULT_RESHAPE]] // CHECK: return %[[RESULT_RESHAPE]]
// ----- // -----
func @unit_dim_for_reduction_inner(%arg0: tensor<?x1x?x1xf32>) -> tensor<?x1xf32> { func @unit_dim_for_reduction_inner(%arg0: tensor<?x1x?x1xf32>) -> tensor<?x1xf32> {
%cst = constant 1.000000e+00 : f32 %cst = constant 1.000000e+00 : f32
%c2 = constant 2 : index %c2 = constant 2 : index
%0 = memref.dim %arg0, %c2 : tensor<?x1x?x1xf32> %0 = memref.dim %arg0, %c2 : tensor<?x1x?x1xf32>
%1 = linalg.init_tensor [%0, 1] : tensor<?x1xf32> %1 = linalg.init_tensor [%0, 1] : tensor<?x1xf32>
%2 = linalg.fill(%1, %cst) : tensor<?x1xf32>, f32 -> tensor<?x1xf32> %2 = linalg.fill(%cst, %1) : f32, tensor<?x1xf32> -> tensor<?x1xf32>
%3 = linalg.generic { %3 = linalg.generic {
indexing_maps = [affine_map<(d0, d1, d2, d3) -> (d0, d1, d2, d3)>, indexing_maps = [affine_map<(d0, d1, d2, d3) -> (d0, d1, d2, d3)>,
affine_map<(d0, d1, d2, d3) -> (d0, d1)>], affine_map<(d0, d1, d2, d3) -> (d0, d1)>],
iterator_types = ["parallel", "parallel", "reduction", "reduction"]} iterator_types = ["parallel", "parallel", "reduction", "reduction"]}
ins(%arg0 : tensor<?x1x?x1xf32>) ins(%arg0 : tensor<?x1x?x1xf32>)
outs(%2 : tensor<?x1xf32>) { outs(%2 : tensor<?x1xf32>) {
^bb0(%arg1: f32, %arg2: f32): // no predecessors ^bb0(%arg1: f32, %arg2: f32): // no predecessors
%4 = addf %arg1, %arg2 : f32 %4 = addf %arg1, %arg2 : f32
linalg.yield %4 : f32 linalg.yield %4 : f32
} -> tensor<?x1xf32> } -> tensor<?x1xf32>
return %3 : tensor<?x1xf32> return %3 : tensor<?x1xf32>
} }
// CHECK-DAG: #[[MAP2:.+]] = affine_map<(d0, d1) -> (d0, d1)> // CHECK-DAG: #[[MAP2:.+]] = affine_map<(d0, d1) -> (d0, d1)>
// CHECK-DAG: #[[MAP3:.+]] = affine_map<(d0, d1) -> (d0)> // CHECK-DAG: #[[MAP3:.+]] = affine_map<(d0, d1) -> (d0)>
// CHECK: func @unit_dim_for_reduction_inner // CHECK: func @unit_dim_for_reduction_inner
// CHECK-SAME: %[[ARG0:.+]]: tensor<?x1x?x1xf32> // CHECK-SAME: %[[ARG0:.+]]: tensor<?x1x?x1xf32>
// CHECK-DAG: %[[RESHAPE:.+]] = linalg.tensor_collapse_shape %[[ARG0]] {{\[}}[0, 1], [2, 3]] // CHECK-DAG: %[[RESHAPE:.+]] = linalg.tensor_collapse_shape %[[ARG0]] {{\[}}[0, 1], [2, 3]]
// CHECK: %[[INIT:.+]] = linalg.init_tensor [%{{.+}}] : tensor<?xf32> // CHECK: %[[INIT:.+]] = linalg.init_tensor [%{{.+}}] : tensor<?xf32>
// CHECK: %[[FILL:.+]] = linalg.fill(%[[INIT]], %{{.+}}) // CHECK: %[[FILL:.+]] = linalg.fill(%{{.+}}, %[[INIT]])
// CHECK: %[[RESULT:.+]] = linalg.generic // CHECK: %[[RESULT:.+]] = linalg.generic
// CHECK-SAME: indexing_maps = [#[[MAP2]], #[[MAP3]]] // CHECK-SAME: indexing_maps = [#[[MAP2]], #[[MAP3]]]
// CHECK-SAME: iterator_types = ["parallel", "reduction"] // CHECK-SAME: iterator_types = ["parallel", "reduction"]
// CHECK-SAME: ins(%[[RESHAPE]] : tensor<?x?xf32>) // CHECK-SAME: ins(%[[RESHAPE]] : tensor<?x?xf32>)
// CHECK-SAME: outs(%[[FILL]] : tensor<?xf32>) // CHECK-SAME: outs(%[[FILL]] : tensor<?xf32>)
// CHECK: %[[RESULT_RESHAPE:.+]] = linalg.tensor_expand_shape %[[RESULT]] {{\[}}[0, 1]] // CHECK: %[[RESULT_RESHAPE:.+]] = linalg.tensor_expand_shape %[[RESULT]] {{\[}}[0, 1]]
// CHECK: return %[[RESULT_RESHAPE]] // CHECK: return %[[RESULT_RESHAPE]]
▲ Show 20 Lines • Show All 323 Lines • Show Last 20 Lines

mlir/test/Dialect/Linalg/forward-vector-transfers.mlir

Show All 23 Lines
// CHECK-NOT: linalg.copy // CHECK-NOT: linalg.copy
// CHECK: %[[ALLOC:.*]] = memref.alloc // CHECK: %[[ALLOC:.*]] = memref.alloc
// CHECK: vector.transfer_read %[[ARG0]] // CHECK: vector.transfer_read %[[ARG0]]
// CHECK-NOT: in_bounds // CHECK-NOT: in_bounds
func @testAllocFillRead(%in: memref<? x f32>) -> vector<32 x f32> { func @testAllocFillRead(%in: memref<? x f32>) -> vector<32 x f32> {
%c0 = constant 0: index %c0 = constant 0: index
%f0 = constant 0.0: f32 %f0 = constant 0.0: f32
%alloc = memref.alloc() : memref<32 x f32> %alloc = memref.alloc() : memref<32 x f32>
linalg.fill(%alloc, %f0): memref<32 x f32>, f32 linalg.fill(%f0, %alloc) : f32, memref<32 x f32>
%subview = memref.subview %alloc[0][16][1] : memref<32 x f32> to memref<16 x f32> %subview = memref.subview %alloc[0][16][1] : memref<32 x f32> to memref<16 x f32>
linalg.copy(%in, %subview): memref<? x f32>, memref<16 x f32> linalg.copy(%in, %subview): memref<? x f32>, memref<16 x f32>
%0 = vector.transfer_read %alloc[%c0], %f0 {in_bounds = [true]} : memref<32 x f32>, vector<32 x f32> %0 = vector.transfer_read %alloc[%c0], %f0 {in_bounds = [true]} : memref<32 x f32>, vector<32 x f32>
memref.dealloc %alloc : memref<32 x f32> memref.dealloc %alloc : memref<32 x f32>
return %0: vector<32 x f32> return %0: vector<32 x f32>
} }
// CHECK-LABEL: testViewRead // CHECK-LABEL: testViewRead
Show All 23 Lines
// CHECK: vector.transfer_read %[[ARG0]] // CHECK: vector.transfer_read %[[ARG0]]
// CHECK-NOT: in_bounds // CHECK-NOT: in_bounds
func @testViewFillRead(%in: memref<? x f32>) -> vector<32 x f32> { func @testViewFillRead(%in: memref<? x f32>) -> vector<32 x f32> {
%c0 = constant 0: index %c0 = constant 0: index
%f0 = constant 0.0: f32 %f0 = constant 0.0: f32
%alloc = memref.alloc() : memref<128 x i8> %alloc = memref.alloc() : memref<128 x i8>
%view = memref.view %alloc[%c0][] : memref<128 x i8> to memref<32 x f32> %view = memref.view %alloc[%c0][] : memref<128 x i8> to memref<32 x f32>
%subview = memref.subview %view[0][16][1] : memref<32 x f32> to memref<16 x f32> %subview = memref.subview %view[0][16][1] : memref<32 x f32> to memref<16 x f32>
linalg.fill(%view, %f0): memref<32 x f32>, f32 linalg.fill(%f0, %view) : f32, memref<32 x f32>
linalg.copy(%in, %subview): memref<? x f32>, memref<16 x f32> linalg.copy(%in, %subview): memref<? x f32>, memref<16 x f32>
%0 = vector.transfer_read %view[%c0], %f0 {in_bounds = [true]} : memref<32 x f32>, vector<32 x f32> %0 = vector.transfer_read %view[%c0], %f0 {in_bounds = [true]} : memref<32 x f32>, vector<32 x f32>
memref.dealloc %alloc : memref<128 x i8> memref.dealloc %alloc : memref<128 x i8>
return %0: vector<32 x f32> return %0: vector<32 x f32>
} }
// CHECK-LABEL: testAllocWrite // CHECK-LABEL: testAllocWrite
// CHECK-SAME: %[[ARG0:[0-9a-zA-Z]*]]: vector // CHECK-SAME: %[[ARG0:[0-9a-zA-Z]*]]: vector
▲ Show 20 Lines • Show All 43 Lines • ▼ Show 20 Lines
// CHECK: %[[ALLOC:.*]] = memref.alloc // CHECK: %[[ALLOC:.*]] = memref.alloc
// CHECK: linalg.copy // CHECK: linalg.copy
// CHECK: vector.transfer_read %[[ALLOC]] // CHECK: vector.transfer_read %[[ALLOC]]
func @failAllocFillRead(%in: memref<? x f32>) -> vector<32 x f32> { func @failAllocFillRead(%in: memref<? x f32>) -> vector<32 x f32> {
%c0 = constant 0: index %c0 = constant 0: index
%f0 = constant 0.0: f32 %f0 = constant 0.0: f32
%f1 = constant 1.0: f32 %f1 = constant 1.0: f32
%alloc = memref.alloc() : memref<32 x f32> %alloc = memref.alloc() : memref<32 x f32>
linalg.fill(%alloc, %f0): memref<32 x f32>, f32 linalg.fill(%f0, %alloc) : f32, memref<32 x f32>
%subview = memref.subview %alloc[0][16][1] : memref<32 x f32> to memref<16 x f32> %subview = memref.subview %alloc[0][16][1] : memref<32 x f32> to memref<16 x f32>
linalg.copy(%in, %subview): memref<? x f32>, memref<16 x f32> linalg.copy(%in, %subview): memref<? x f32>, memref<16 x f32>
"some_interleaved_use"(%subview) : (memref<16 x f32>) -> () "some_interleaved_use"(%subview) : (memref<16 x f32>) -> ()
%0 = vector.transfer_read %alloc[%c0], %f1: memref<32 x f32>, vector<32 x f32> %0 = vector.transfer_read %alloc[%c0], %f1: memref<32 x f32>, vector<32 x f32>
memref.dealloc %alloc : memref<32 x f32> memref.dealloc %alloc : memref<32 x f32>
return %0: vector<32 x f32> return %0: vector<32 x f32>
} }
Show All 19 Lines

mlir/test/Dialect/Linalg/fusion-pattern.mlir

// RUN: mlir-opt %s -test-linalg-fusion-transform-patterns -canonicalize -cse -split-input-file | FileCheck %s // RUN: mlir-opt %s -test-linalg-fusion-transform-patterns -canonicalize -cse -split-input-file | FileCheck %s
module { module {
func @basic_fusion(%arg0: memref<?x?xf32>, %arg1: memref<?x?xf32>, func @basic_fusion(%arg0: memref<?x?xf32>, %arg1: memref<?x?xf32>,
%arg2: memref<?x?xf32>) { %arg2: memref<?x?xf32>) {
%cst = constant 0.000000e+00 : f32 %cst = constant 0.000000e+00 : f32
linalg.fill(%arg2, %cst) : memref<?x?xf32>, f32 linalg.fill(%cst, %arg2) : f32, memref<?x?xf32>
linalg.matmul {__internal_linalg_transform__ = "basic_fusion"} linalg.matmul {__internal_linalg_transform__ = "basic_fusion"}
ins(%arg0, %arg1 : memref<?x?xf32>, memref<?x?xf32>) ins(%arg0, %arg1 : memref<?x?xf32>, memref<?x?xf32>)
outs(%arg2 : memref<?x?xf32>) outs(%arg2 : memref<?x?xf32>)
return return
} }
} }
// CHECK-DAG: #[[MAP0:.+]] = affine_map<(d0)[s0] -> (32, -d0 + s0)> // CHECK-DAG: #[[MAP0:.+]] = affine_map<(d0)[s0] -> (32, -d0 + s0)>
// CHECK-DAG: #[[MAP1:.+]] = affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)> // CHECK-DAG: #[[MAP1:.+]] = affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>
// CHECK-DAG: #[[MAP2:.+]] = affine_map<(d0)[s0] -> (64, -d0 + s0)> // CHECK-DAG: #[[MAP2:.+]] = affine_map<(d0)[s0] -> (64, -d0 + s0)>
// CHECK-DAG: #[[MAP3:.+]] = affine_map<(d0)[s0] -> (16, -d0 + s0)> // CHECK-DAG: #[[MAP3:.+]] = affine_map<(d0)[s0] -> (16, -d0 + s0)>
// CHECK-DAG: #[[MAP4:.+]] = affine_map<(d0)[s0, s1] -> (-d0 + s0, 32, -d0 + s1)> // CHECK-DAG: #[[MAP4:.+]] = affine_map<(d0)[s0, s1] -> (-d0 + s0, 32, -d0 + s1)>
// CHECK-DAG: #[[MAP5:.+]] = affine_map<(d0)[s0, s1] -> (-d0 + s0, 64, -d0 + s1)> // CHECK-DAG: #[[MAP5:.+]] = affine_map<(d0)[s0, s1] -> (-d0 + s0, 64, -d0 + s1)>
// CHECK: func @basic_fusion // CHECK: func @basic_fusion
// CHECK-SAME: %[[ARG0:[a-zA-Z0-9_]+]]: memref<?x?xf32> // CHECK-SAME: %[[ARG0:[a-zA-Z0-9_]+]]: memref<?x?xf32>
// CHECK-SAME: %[[ARG1:[a-zA-Z0-9_]+]]: memref<?x?xf32> // CHECK-SAME: %[[ARG1:[a-zA-Z0-9_]+]]: memref<?x?xf32>
// CHECK-SAME: %[[ARG2:[a-zA-Z0-9_]+]]: memref<?x?xf32> // CHECK-SAME: %[[ARG2:[a-zA-Z0-9_]+]]: memref<?x?xf32>
// CHECK-DAG: %[[C0:.+]] = constant 0 : index // CHECK-DAG: %[[C0:.+]] = constant 0 : index
// CHECK-DAG: %[[C1:.+]] = constant 1 : index // CHECK-DAG: %[[C1:.+]] = constant 1 : index
// CHECK-DAG: %[[C32:.+]] = constant 32 : index // CHECK-DAG: %[[C32:.+]] = constant 32 : index
// CHECK-DAG: %[[C64:.+]] = constant 64 : index // CHECK-DAG: %[[C64:.+]] = constant 64 : index
// CHECK-DAG: %[[C16:.+]] = constant 16 : index // CHECK-DAG: %[[C16:.+]] = constant 16 : index
// CHECK-DAG: %[[CST:.+]] = constant 0.0{{.*}} : f32 // CHECK-DAG: %[[CST:.+]] = constant 0.0{{.*}} : f32
// CHECK-DAG: linalg.fill(%[[ARG2]], %[[CST]]) // CHECK-DAG: linalg.fill(%[[CST]], %[[ARG2]])
// CHECK-SAME: __internal_linalg_transform__ = "after_basic_fusion_original" // CHECK-SAME: __internal_linalg_transform__ = "after_basic_fusion_original"
// CHECK-DAG: %[[M:.+]] = memref.dim %[[ARG0]], %[[C0]] // CHECK-DAG: %[[M:.+]] = memref.dim %[[ARG0]], %[[C0]]
// CHECK-DAG: %[[N:.+]] = memref.dim %[[ARG1]], %[[C1]] // CHECK-DAG: %[[N:.+]] = memref.dim %[[ARG1]], %[[C1]]
// CHECK: scf.parallel (%[[IV0:.+]], %[[IV1:.+]]) = // CHECK: scf.parallel (%[[IV0:.+]], %[[IV1:.+]]) =
// CHECK-SAME: to (%[[M]], %[[N]]) // CHECK-SAME: to (%[[M]], %[[N]])
// CHECK-SAME: step (%[[C32]], %[[C64]]) { // CHECK-SAME: step (%[[C32]], %[[C64]]) {
// CHECK: %[[TILE_M:.+]] = affine.min #[[MAP0]](%[[IV0]])[%[[M]]] // CHECK: %[[TILE_M:.+]] = affine.min #[[MAP0]](%[[IV0]])[%[[M]]]
// CHECK: %[[K:.+]] = memref.dim %[[ARG0]], %[[C1]] // CHECK: %[[K:.+]] = memref.dim %[[ARG0]], %[[C1]]
// CHECK: %[[SV1:.+]] = memref.subview %[[ARG0]][%[[IV0]], 0] // CHECK: %[[SV1:.+]] = memref.subview %[[ARG0]][%[[IV0]], 0]
// CHECK-SAME: [%[[TILE_M]], %[[K]]] // CHECK-SAME: [%[[TILE_M]], %[[K]]]
// CHECK: %[[K_2:.+]] = memref.dim %[[ARG1]], %[[C0]] // CHECK: %[[K_2:.+]] = memref.dim %[[ARG1]], %[[C0]]
// CHECK: %[[TILE_N:.+]] = affine.min #[[MAP2]](%[[IV1]])[%[[N]]] // CHECK: %[[TILE_N:.+]] = affine.min #[[MAP2]](%[[IV1]])[%[[N]]]
// CHECK: %[[SV2:.+]] = memref.subview %[[ARG1]][0, %[[IV1]]] // CHECK: %[[SV2:.+]] = memref.subview %[[ARG1]][0, %[[IV1]]]
// CHECK-SAME: %[[K_2]], %[[TILE_N]] // CHECK-SAME: %[[K_2]], %[[TILE_N]]
// CHECK: %[[M_2:.+]] = memref.dim %[[ARG2]], %[[C0]] // CHECK: %[[M_2:.+]] = memref.dim %[[ARG2]], %[[C0]]
// CHECK: %[[TILE_M_2:.+]] = affine.min #[[MAP0]](%[[IV0]])[%[[M_2]]] // CHECK: %[[TILE_M_2:.+]] = affine.min #[[MAP0]](%[[IV0]])[%[[M_2]]]
// CHECK: %[[N_2:.+]] = memref.dim %[[ARG2]], %[[C1]] // CHECK: %[[N_2:.+]] = memref.dim %[[ARG2]], %[[C1]]
// CHECK: %[[TILE_N_2:.+]] = affine.min #[[MAP2]](%[[IV1]])[%[[N_2]]] // CHECK: %[[TILE_N_2:.+]] = affine.min #[[MAP2]](%[[IV1]])[%[[N_2]]]
// CHECK: %[[SV3:.+]] = memref.subview %[[ARG2]][%[[IV0]], %[[IV1]]] // CHECK: %[[SV3:.+]] = memref.subview %[[ARG2]][%[[IV0]], %[[IV1]]]
// CHECK-SAME: [%[[TILE_M_2]], %[[TILE_N_2]]] // CHECK-SAME: [%[[TILE_M_2]], %[[TILE_N_2]]]
// CHECK: %[[TILE_M_3:.+]] = affine.min #[[MAP4]](%[[IV0]])[%[[M_2]], %[[M]]] // CHECK: %[[TILE_M_3:.+]] = affine.min #[[MAP4]](%[[IV0]])[%[[M_2]], %[[M]]]
// CHECK: %[[TILE_N_3:.+]] = affine.min #[[MAP5]](%[[IV1]])[%[[N_2]], %[[N]]] // CHECK: %[[TILE_N_3:.+]] = affine.min #[[MAP5]](%[[IV1]])[%[[N_2]], %[[N]]]
// CHECK: %[[SV3_2:.+]] = memref.subview %[[ARG2]][%[[IV0]], %[[IV1]]] // CHECK: %[[SV3_2:.+]] = memref.subview %[[ARG2]][%[[IV0]], %[[IV1]]]
// CHECK-SAME: [%[[TILE_M_3]], %[[TILE_N_3]]] // CHECK-SAME: [%[[TILE_M_3]], %[[TILE_N_3]]]
// CHECK: linalg.fill(%[[SV3_2]], %[[CST]]) // CHECK: linalg.fill(%[[CST]], %[[SV3_2]])
// CHECK-SAME: __internal_linalg_transform__ = "after_basic_fusion_producer" // CHECK-SAME: __internal_linalg_transform__ = "after_basic_fusion_producer"
// CHECK: scf.for %[[IV2:.+]] = %[[C0]] to %[[K]] step %[[C16]] { // CHECK: scf.for %[[IV2:.+]] = %[[C0]] to %[[K]] step %[[C16]] {
// CHECK: %[[TILE_K:.+]] = affine.min #[[MAP3]](%[[IV2]])[%[[K]]] // CHECK: %[[TILE_K:.+]] = affine.min #[[MAP3]](%[[IV2]])[%[[K]]]
// CHECK: %[[SV4:.+]] = memref.subview %[[SV1]][0, %[[IV2]]] // CHECK: %[[SV4:.+]] = memref.subview %[[SV1]][0, %[[IV2]]]
// CHECK-SAME: [%[[TILE_M]], %[[TILE_K]]] // CHECK-SAME: [%[[TILE_M]], %[[TILE_K]]]
// CHECK: %[[TILE_K_2:.+]] = affine.min #[[MAP3]](%[[IV2]])[%[[K_2]]] // CHECK: %[[TILE_K_2:.+]] = affine.min #[[MAP3]](%[[IV2]])[%[[K_2]]]
// CHECK: %[[SV5:.+]] = memref.subview %[[SV2]][%[[IV2]], 0] // CHECK: %[[SV5:.+]] = memref.subview %[[SV2]][%[[IV2]], 0]
// CHECK-SAME: [%[[TILE_K_2]], %[[TILE_N]]] // CHECK-SAME: [%[[TILE_K_2]], %[[TILE_N]]]
Show All 9 Lines
// ----- // -----
module { module {
func @rhs_fusion(%arg0: memref<?x?xf32>, %arg1: memref<?x?xf32>, func @rhs_fusion(%arg0: memref<?x?xf32>, %arg1: memref<?x?xf32>,
%arg2: memref<?x?xf32>, %arg3: memref<?x?xf32>) { %arg2: memref<?x?xf32>, %arg3: memref<?x?xf32>) {
%cst = constant 0.000000e+00 : f32 %cst = constant 0.000000e+00 : f32
linalg.copy(%arg1, %arg2) : memref<?x?xf32>, memref<?x?xf32> linalg.copy(%arg1, %arg2) : memref<?x?xf32>, memref<?x?xf32>
linalg.fill(%arg3, %cst) : memref<?x?xf32>, f32 linalg.fill(%cst, %arg3) : f32, memref<?x?xf32>
linalg.matmul {__internal_linalg_transform__ = "rhs_fusion"} linalg.matmul {__internal_linalg_transform__ = "rhs_fusion"}
ins(%arg0, %arg2 : memref<?x?xf32>, memref<?x?xf32>) ins(%arg0, %arg2 : memref<?x?xf32>, memref<?x?xf32>)
outs(%arg3 : memref<?x?xf32>) outs(%arg3 : memref<?x?xf32>)
return return
} }
} }
// CHECK-DAG: #[[MAP0:.+]] = affine_map<(d0)[s0] -> (64, -d0 + s0)> // CHECK-DAG: #[[MAP0:.+]] = affine_map<(d0)[s0] -> (64, -d0 + s0)>
// CHECK-DAG: #[[MAP1:.+]] = affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)> // CHECK-DAG: #[[MAP1:.+]] = affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>
▲ Show 20 Lines • Show All 65 Lines • ▼ Show 20 Lines
// ----- // -----
module { module {
func @two_operand_fusion(%arg0: memref<?x?xf32>, %arg1: memref<?x?xf32>, func @two_operand_fusion(%arg0: memref<?x?xf32>, %arg1: memref<?x?xf32>,
%arg2: memref<?x?xf32>, %arg3: memref<?x?xf32>) { %arg2: memref<?x?xf32>, %arg3: memref<?x?xf32>) {
%cst = constant 0.000000e+00 : f32 %cst = constant 0.000000e+00 : f32
linalg.copy(%arg0, %arg1) : memref<?x?xf32>, memref<?x?xf32> linalg.copy(%arg0, %arg1) : memref<?x?xf32>, memref<?x?xf32>
linalg.fill(%arg3, %cst) : memref<?x?xf32>, f32 linalg.fill(%cst, %arg3) : f32, memref<?x?xf32>
linalg.matmul {__internal_linalg_transform__ = "two_operand_fusion"} linalg.matmul {__internal_linalg_transform__ = "two_operand_fusion"}
ins(%arg1, %arg2 : memref<?x?xf32>, memref<?x?xf32>) ins(%arg1, %arg2 : memref<?x?xf32>, memref<?x?xf32>)
outs(%arg3 : memref<?x?xf32>) outs(%arg3 : memref<?x?xf32>)
return return
} }
} }
// CHECK-DAG: #[[MAP0:.+]] = affine_map<(d0)[s0] -> (32, -d0 + s0)> // CHECK-DAG: #[[MAP0:.+]] = affine_map<(d0)[s0] -> (32, -d0 + s0)>
// CHECK-DAG: #[[MAP1:.+]] = affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)> // CHECK-DAG: #[[MAP1:.+]] = affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>
// CHECK-DAG: #[[MAP2:.+]] = affine_map<(d0)[s0] -> (16, -d0 + s0)> // CHECK-DAG: #[[MAP2:.+]] = affine_map<(d0)[s0] -> (16, -d0 + s0)>
// CHECK-DAG: #[[MAP3:.+]] = affine_map<(d0)[s0] -> (64, -d0 + s0)> // CHECK-DAG: #[[MAP3:.+]] = affine_map<(d0)[s0] -> (64, -d0 + s0)>
// CHECK-DAG: #[[MAP4:.+]] = affine_map<(d0)[s0, s1] -> (-d0 + s0, 32, -d0 + s1)> // CHECK-DAG: #[[MAP4:.+]] = affine_map<(d0)[s0, s1] -> (-d0 + s0, 32, -d0 + s1)>
// CHECK: func @two_operand_fusion // CHECK: func @two_operand_fusion
// CHECK-SAME: %[[ARG0:[a-zA-Z0-9_]+]]: memref<?x?xf32> // CHECK-SAME: %[[ARG0:[a-zA-Z0-9_]+]]: memref<?x?xf32>
// CHECK-SAME: %[[ARG1:[a-zA-Z0-9_]+]]: memref<?x?xf32> // CHECK-SAME: %[[ARG1:[a-zA-Z0-9_]+]]: memref<?x?xf32>
// CHECK-SAME: %[[ARG2:[a-zA-Z0-9_]+]]: memref<?x?xf32> // CHECK-SAME: %[[ARG2:[a-zA-Z0-9_]+]]: memref<?x?xf32>
// CHECK-SAME: %[[ARG3:[a-zA-Z0-9_]+]]: memref<?x?xf32> // CHECK-SAME: %[[ARG3:[a-zA-Z0-9_]+]]: memref<?x?xf32>
// CHECK-DAG: %[[C0:.+]] = constant 0 : index // CHECK-DAG: %[[C0:.+]] = constant 0 : index
// CHECK-DAG: %[[C1:.+]] = constant 1 : index // CHECK-DAG: %[[C1:.+]] = constant 1 : index
// CHECK-DAG: %[[C32:.+]] = constant 32 : index // CHECK-DAG: %[[C32:.+]] = constant 32 : index
// CHECK-DAG: %[[C64:.+]] = constant 64 : index // CHECK-DAG: %[[C64:.+]] = constant 64 : index
// CHECK-DAG: %[[C16:.+]] = constant 16 : index // CHECK-DAG: %[[C16:.+]] = constant 16 : index
// CHECK-DAG: %[[CST:.+]] = constant 0.0{{.*}} : f32 // CHECK-DAG: %[[CST:.+]] = constant 0.0{{.*}} : f32
// CHECK: linalg.copy(%[[ARG0]], %[[ARG1]]) // CHECK: linalg.copy(%[[ARG0]], %[[ARG1]])
// CHECK-SAME: __internal_linalg_transform__ = "after_two_operand_fusion_original" // CHECK-SAME: __internal_linalg_transform__ = "after_two_operand_fusion_original"
// CHECK: linalg.fill(%[[ARG3]], %[[CST]]) // CHECK: linalg.fill(%[[CST]], %[[ARG3]])
// CHECK-SAME: __internal_linalg_transform__ = "after_two_operand_fusion_original" // CHECK-SAME: __internal_linalg_transform__ = "after_two_operand_fusion_original"
// CHECK-DAG: %[[M:.+]] = memref.dim %[[ARG1]], %[[C0]] // CHECK-DAG: %[[M:.+]] = memref.dim %[[ARG1]], %[[C0]]
// CHECK: scf.parallel (%[[IV0:.+]]) = // CHECK: scf.parallel (%[[IV0:.+]]) =
// CHECK-SAME: (%[[C0]]) to (%[[M]]) step (%[[C32]]) { // CHECK-SAME: (%[[C0]]) to (%[[M]]) step (%[[C32]]) {
// CHECK: %[[TILE_M:.+]] = affine.min #[[MAP0]](%[[IV0]])[%[[M]]] // CHECK: %[[TILE_M:.+]] = affine.min #[[MAP0]](%[[IV0]])[%[[M]]]
// CHECK: %[[K:.+]] = memref.dim %[[ARG1]], %[[C1]] // CHECK: %[[K:.+]] = memref.dim %[[ARG1]], %[[C1]]
// CHECK: %[[SV1:.+]] = memref.subview %[[ARG1]][%[[IV0]], 0] // CHECK: %[[SV1:.+]] = memref.subview %[[ARG1]][%[[IV0]], 0]
// CHECK-SAME: [%[[TILE_M]], %[[K]]] // CHECK-SAME: [%[[TILE_M]], %[[K]]]
Show All 10 Lines
// CHECK: %[[K_3:.+]] = memref.dim %[[ARG0]], %[[C1]] // CHECK: %[[K_3:.+]] = memref.dim %[[ARG0]], %[[C1]]
// CHECK: %[[SV3:.+]] = memref.subview %[[ARG0]][%[[IV0]], 0] // CHECK: %[[SV3:.+]] = memref.subview %[[ARG0]][%[[IV0]], 0]
// CHECK-SAME: [%[[TILE_M_4]], %[[K_3]]] // CHECK-SAME: [%[[TILE_M_4]], %[[K_3]]]
// CHECK: %[[TILE_M_5:.+]] = affine.min #[[MAP4]](%[[IV0]])[%[[M]], %[[M]]] // CHECK: %[[TILE_M_5:.+]] = affine.min #[[MAP4]](%[[IV0]])[%[[M]], %[[M]]]
// CHECK: %[[SV3_2:.+]] = memref.subview %[[ARG1]][%[[IV0]], 0] // CHECK: %[[SV3_2:.+]] = memref.subview %[[ARG1]][%[[IV0]], 0]
// CHECK-SAME: [%[[TILE_M_5]], %[[K]]] // CHECK-SAME: [%[[TILE_M_5]], %[[K]]]
// CHECK: linalg.copy(%[[SV3]], %[[SV3_2]]) // CHECK: linalg.copy(%[[SV3]], %[[SV3_2]])
// CHECK-SAME: __internal_linalg_transform__ = "after_two_operand_fusion_producer" // CHECK-SAME: __internal_linalg_transform__ = "after_two_operand_fusion_producer"
// CHECK: linalg.fill(%[[SV2_2]], %[[CST]]) // CHECK: linalg.fill(%[[CST]], %[[SV2_2]])
// CHECK-SAME: __internal_linalg_transform__ = "after_two_operand_fusion_producer" // CHECK-SAME: __internal_linalg_transform__ = "after_two_operand_fusion_producer"
// CHECK-DAG: %[[N_2:.+]] = memref.dim %[[ARG2]], %[[C1]] // CHECK-DAG: %[[N_2:.+]] = memref.dim %[[ARG2]], %[[C1]]
// CHECK: scf.parallel (%[[IV1:.+]]) = // CHECK: scf.parallel (%[[IV1:.+]]) =
// CHECK-SAME: (%[[C0]]) to (%[[N_2]]) step (%[[C64]]) { // CHECK-SAME: (%[[C0]]) to (%[[N_2]]) step (%[[C64]]) {
// CHECK-NEXT: scf.for %[[IV2:.+]] = %[[C0]] to %[[K]] step %[[C16]] { // CHECK-NEXT: scf.for %[[IV2:.+]] = %[[C0]] to %[[K]] step %[[C16]] {
// CHECK: %[[TILE_K:.+]] = affine.min #[[MAP2]](%[[IV2]])[%[[K]]] // CHECK: %[[TILE_K:.+]] = affine.min #[[MAP2]](%[[IV2]])[%[[K]]]
// CHECK: %[[SV4:.+]] = memref.subview %[[SV1]][0, %[[IV2]]] // CHECK: %[[SV4:.+]] = memref.subview %[[SV1]][0, %[[IV2]]]
// CHECK-SAME: [%[[TILE_M]], %[[TILE_K]]] // CHECK-SAME: [%[[TILE_M]], %[[TILE_K]]]
▲ Show 20 Lines • Show All 198 Lines • ▼ Show 20 Lines func @basic_no_fusion(%arg0: memref<?x?xf32>, %arg1: memref<?x?xf32>,
%arg2: memref<?x?xf32>) { %arg2: memref<?x?xf32>) {
%c0 = constant 0 : index %c0 = constant 0 : index
%c1 = constant 1 : index %c1 = constant 1 : index
%c2 = constant 2 : index %c2 = constant 2 : index
%c32 = constant 32 : index %c32 = constant 32 : index
%c64 = constant 64 : index %c64 = constant 64 : index
%c16 = constant 16 : index %c16 = constant 16 : index
%cst = constant 0.000000e+00 : f32 %cst = constant 0.000000e+00 : f32
linalg.fill(%arg2, %cst) : memref<?x?xf32>, f32 linalg.fill(%cst, %arg2) : f32, memref<?x?xf32>
%0 = memref.dim %arg0, %c0 : memref<?x?xf32> %0 = memref.dim %arg0, %c0 : memref<?x?xf32>
%1 = memref.dim %arg1, %c1 : memref<?x?xf32> %1 = memref.dim %arg1, %c1 : memref<?x?xf32>
%2 = memref.dim %arg0, %c1 : memref<?x?xf32> %2 = memref.dim %arg0, %c1 : memref<?x?xf32>
scf.parallel (%arg3, %arg4) = (%c0, %c0) to (%0, %1) step (%c32, %c64) { scf.parallel (%arg3, %arg4) = (%c0, %c0) to (%0, %1) step (%c32, %c64) {
scf.for %arg5 = %c0 to %2 step %c16 { scf.for %arg5 = %c0 to %2 step %c16 {
%3 = affine.min #map0(%arg3)[%0] %3 = affine.min #map0(%arg3)[%0]
%4 = affine.min #map1(%arg4)[%1] %4 = affine.min #map1(%arg4)[%1]
%5 = affine.min #map2(%arg5)[%2] %5 = affine.min #map2(%arg5)[%2]
Show All 18 Lines
// CHECK: linalg.matmul // CHECK: linalg.matmul
// ----- // -----
module { module {
func @basic_conv_fusion(%arg0: memref<?x?x?x?xf32>, %arg1: memref<?x?x?x?xf32>, func @basic_conv_fusion(%arg0: memref<?x?x?x?xf32>, %arg1: memref<?x?x?x?xf32>,
%arg2: memref<?x?x?x?xf32>) { %arg2: memref<?x?x?x?xf32>) {
%cst = constant 0.000000e+00 : f32 %cst = constant 0.000000e+00 : f32
linalg.fill(%arg2, %cst) : memref<?x?x?x?xf32>, f32 linalg.fill(%cst, %arg2) : f32, memref<?x?x?x?xf32>
linalg.conv(%arg0, %arg1, %arg2) { linalg.conv(%arg0, %arg1, %arg2) {
dilations = [1, 1], strides = [1, 1], dilations = [1, 1], strides = [1, 1],
__internal_linalg_transform__ = "basic_fusion"} : __internal_linalg_transform__ = "basic_fusion"} :
memref<?x?x?x?xf32>, memref<?x?x?x?xf32>, memref<?x?x?x?xf32> memref<?x?x?x?xf32>, memref<?x?x?x?xf32>, memref<?x?x?x?xf32>
return return
} }
} }
// CHECK: func @basic_conv_fusion // CHECK: func @basic_conv_fusion
Show All 11 Lines

mlir/test/Dialect/Linalg/fusion-sequence.mlir

// RUN: mlir-opt -pass-pipeline="func(test-linalg-tile-and-fuse{tile-sizes=16,32,64}),resolve-shaped-type-result-dims,canonicalize,cse" -split-input-file %s | FileCheck %s // RUN: mlir-opt -pass-pipeline="func(test-linalg-tile-and-fuse{tile-sizes=16,32,64}),resolve-shaped-type-result-dims,canonicalize,cse" -split-input-file %s | FileCheck %s
module { module {
func @three_op_fusion(%arg0: memref<?x?xf32>, %arg1: memref<?x?xf32>, func @three_op_fusion(%arg0: memref<?x?xf32>, %arg1: memref<?x?xf32>,
%arg2: memref<?xf32>, %arg3 : memref<?x?xf32>) { %arg2: memref<?xf32>, %arg3 : memref<?x?xf32>) {
%cst = constant 0.000000e+00 : f32 %cst = constant 0.000000e+00 : f32
%c0 = constant 0 : index %c0 = constant 0 : index
%c1 = constant 1 : index %c1 = constant 1 : index
%d0 = memref.dim %arg0, %c0 : memref<?x?xf32> %d0 = memref.dim %arg0, %c0 : memref<?x?xf32>
%d1 = memref.dim %arg1, %c1 : memref<?x?xf32> %d1 = memref.dim %arg1, %c1 : memref<?x?xf32>
%0 = memref.alloc(%d0, %d1) : memref<?x?xf32> %0 = memref.alloc(%d0, %d1) : memref<?x?xf32>
linalg.fill(%0, %cst) : memref<?x?xf32>, f32 linalg.fill(%cst, %0) : f32, memref<?x?xf32>
linalg.matmul ins(%arg0, %arg1 : memref<?x?xf32>, memref<?x?xf32>) linalg.matmul ins(%arg0, %arg1 : memref<?x?xf32>, memref<?x?xf32>)
outs(%0 : memref<?x?xf32>) outs(%0 : memref<?x?xf32>)
linalg.generic linalg.generic
{indexing_maps = [affine_map<(d0, d1) -> (d0, d1)>, {indexing_maps = [affine_map<(d0, d1) -> (d0, d1)>,
affine_map<(d0, d1) -> (d1)>, affine_map<(d0, d1) -> (d1)>,
affine_map<(d0, d1) -> (d0, d1)>], affine_map<(d0, d1) -> (d0, d1)>],
iterator_types = ["parallel", "parallel"]} iterator_types = ["parallel", "parallel"]}
ins(%0, %arg2 : memref<?x?xf32>, memref<?xf32>) ins(%0, %arg2 : memref<?x?xf32>, memref<?xf32>)
Show All 16 Lines
// CHECK: %[[TEMP:.+]] = memref.alloc(%{{.*}}, %{{.*}}) : memref<?x?xf32> // CHECK: %[[TEMP:.+]] = memref.alloc(%{{.*}}, %{{.*}}) : memref<?x?xf32>
// CHECK: scf.parallel (%[[IV0:.+]], %[[IV1:.+]]) = {{.*}} { // CHECK: scf.parallel (%[[IV0:.+]], %[[IV1:.+]]) = {{.*}} {
// CHECK: %[[SV_TEMP_1:.+]] = memref.subview %[[TEMP]][%[[IV0]], %[[IV1]]] // CHECK: %[[SV_TEMP_1:.+]] = memref.subview %[[TEMP]][%[[IV0]], %[[IV1]]]
// CHECK-DAG: %[[SV_ARG2:.+]] = memref.subview %[[ARG2]][%[[IV1]]] // CHECK-DAG: %[[SV_ARG2:.+]] = memref.subview %[[ARG2]][%[[IV1]]]
// CHECK-DAG: %[[SV_ARG3:.+]] = memref.subview %[[ARG3]][%[[IV0]], %[[IV1]]] // CHECK-DAG: %[[SV_ARG3:.+]] = memref.subview %[[ARG3]][%[[IV0]], %[[IV1]]]
// CHECK-DAG: %[[SV_ARG0:.+]] = memref.subview %[[ARG0]][%[[IV0]], 0] // CHECK-DAG: %[[SV_ARG0:.+]] = memref.subview %[[ARG0]][%[[IV0]], 0]
// CHECK-DAG: %[[SV_ARG1:.+]] = memref.subview %[[ARG1]][0, %[[IV1]]] // CHECK-DAG: %[[SV_ARG1:.+]] = memref.subview %[[ARG1]][0, %[[IV1]]]
// CHECK: %[[SV_TEMP_2:.+]] = memref.subview %[[TEMP]][%[[IV0]], %[[IV1]]] // CHECK: %[[SV_TEMP_2:.+]] = memref.subview %[[TEMP]][%[[IV0]], %[[IV1]]]
// CHECK: linalg.fill(%[[SV_TEMP_2]], %{{.+}}) // CHECK: linalg.fill(%{{.+}}, %[[SV_TEMP_2]])
// CHECK: linalg.matmul // CHECK: linalg.matmul
// CHECK-SAME: ins(%[[SV_ARG0]], %[[SV_ARG1]] // CHECK-SAME: ins(%[[SV_ARG0]], %[[SV_ARG1]]
// CHECK-SAME: : memref<?x?xf32, #[[MAP2]]>, memref<?x?xf32, #[[MAP2]]>) // CHECK-SAME: : memref<?x?xf32, #[[MAP2]]>, memref<?x?xf32, #[[MAP2]]>)
// CHECK-SAME: outs(%[[SV_TEMP_2]] : memref<?x?xf32, #[[MAP2]]>) // CHECK-SAME: outs(%[[SV_TEMP_2]] : memref<?x?xf32, #[[MAP2]]>)
// CHECK: linalg.generic // CHECK: linalg.generic
// CHECK-SAME: ins(%[[SV_TEMP_1]], %[[SV_ARG2]] // CHECK-SAME: ins(%[[SV_TEMP_1]], %[[SV_ARG2]]
// CHECK-SAME: : memref<?x?xf32, #[[MAP2]]>, memref<?xf32, #[[MAP3]]>) // CHECK-SAME: : memref<?x?xf32, #[[MAP2]]>, memref<?xf32, #[[MAP3]]>)
// CHECK-SAME: outs(%[[SV_ARG3]] : memref<?x?xf32, #[[MAP2]]>) // CHECK-SAME: outs(%[[SV_ARG3]] : memref<?x?xf32, #[[MAP2]]>)
Show All 10 Lines func @sequence_of_matmul(%arg0: memref<?x?xf32>, %arg1: memref<?x?xf32>,
%c0 = constant 0 : index %c0 = constant 0 : index
%c1 = constant 1 : index %c1 = constant 1 : index
%m = memref.dim %arg0, %c0 : memref<?x?xf32> %m = memref.dim %arg0, %c0 : memref<?x?xf32>
%n1 = memref.dim %arg1, %c1 : memref<?x?xf32> %n1 = memref.dim %arg1, %c1 : memref<?x?xf32>
%n2 = memref.dim %arg2, %c1 : memref<?x?xf32> %n2 = memref.dim %arg2, %c1 : memref<?x?xf32>
%n3 = memref.dim %arg3, %c1 : memref<?x?xf32> %n3 = memref.dim %arg3, %c1 : memref<?x?xf32>
%0 = memref.alloc(%m, %n1) : memref<?x?xf32> %0 = memref.alloc(%m, %n1) : memref<?x?xf32>
%1 = memref.alloc(%m, %n2) : memref<?x?xf32> %1 = memref.alloc(%m, %n2) : memref<?x?xf32>
linalg.fill(%0, %cst) : memref<?x?xf32>, f32 linalg.fill(%cst, %0) : f32, memref<?x?xf32>
linalg.matmul ins(%arg0, %arg1 : memref<?x?xf32>, memref<?x?xf32>) linalg.matmul ins(%arg0, %arg1 : memref<?x?xf32>, memref<?x?xf32>)
outs(%0 : memref<?x?xf32>) outs(%0 : memref<?x?xf32>)
linalg.fill(%1, %cst) : memref<?x?xf32>, f32 linalg.fill(%cst, %1) : f32, memref<?x?xf32>
linalg.matmul ins(%0, %arg2 : memref<?x?xf32>, memref<?x?xf32>) linalg.matmul ins(%0, %arg2 : memref<?x?xf32>, memref<?x?xf32>)
outs(%1 : memref<?x?xf32>) outs(%1 : memref<?x?xf32>)
linalg.fill(%arg4, %cst) : memref<?x?xf32>, f32 linalg.fill(%cst, %arg4) : f32, memref<?x?xf32>
linalg.matmul ins(%1, %arg3 : memref<?x?xf32>, memref<?x?xf32>) linalg.matmul ins(%1, %arg3 : memref<?x?xf32>, memref<?x?xf32>)
outs(%arg4 : memref<?x?xf32>) outs(%arg4 : memref<?x?xf32>)
return return
} }
} }
// CHECK-DAG: #[[MAP0:.+]] = affine_map<(d0)[s0] -> (16, -d0 + s0)> // CHECK-DAG: #[[MAP0:.+]] = affine_map<(d0)[s0] -> (16, -d0 + s0)>
// CHECK-DAG: #[[MAP1:.+]] = affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)> // CHECK-DAG: #[[MAP1:.+]] = affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>
Show All 32 Lines
// CHECK: %[[SV_ALLOC1:.+]] = memref.subview %[[ALLOC1]][%[[IV0]], 0] // CHECK: %[[SV_ALLOC1:.+]] = memref.subview %[[ALLOC1]][%[[IV0]], 0]
// CHECK-SAME: [%[[TILE_M_4]], %[[N1]]] // CHECK-SAME: [%[[TILE_M_4]], %[[N1]]]
// CHECK: %[[SV_ALLOC2:.+]] = memref.subview %[[ALLOC2]][%[[IV0]], 0] // CHECK: %[[SV_ALLOC2:.+]] = memref.subview %[[ALLOC2]][%[[IV0]], 0]
// CHECK-SAME: [%[[TILE_M_4]], %[[N2]]] // CHECK-SAME: [%[[TILE_M_4]], %[[N2]]]
// CHECK: %[[TILE_M_5:.+]] = affine.min #[[MAP2]](%[[IV0]])[%[[M]], %[[M]]] // CHECK: %[[TILE_M_5:.+]] = affine.min #[[MAP2]](%[[IV0]])[%[[M]], %[[M]]]
// CHECK: %[[N0:.+]] = memref.dim %[[ARG0]], %[[C1]] // CHECK: %[[N0:.+]] = memref.dim %[[ARG0]], %[[C1]]
// CHECK: %[[SV_ARG0:.+]] = memref.subview %[[ARG0]][%[[IV0]], 0] // CHECK: %[[SV_ARG0:.+]] = memref.subview %[[ARG0]][%[[IV0]], 0]
// CHECK-SAME: [%[[TILE_M_5]], %[[N0]]] // CHECK-SAME: [%[[TILE_M_5]], %[[N0]]]
// CHECK: linalg.fill(%[[SV_ALLOC1]], %{{.+}}) // CHECK: linalg.fill(%{{.+}}, %[[SV_ALLOC1]])
// CHECK: linalg.matmul ins(%[[SV_ARG0]], %[[ARG1]] // CHECK: linalg.matmul ins(%[[SV_ARG0]], %[[ARG1]]
// CHECK-SAME: : memref<?x?xf32, #[[MAP1]]>, memref<?x?xf32>) // CHECK-SAME: : memref<?x?xf32, #[[MAP1]]>, memref<?x?xf32>)
// CHECK-SAME: outs(%[[SV_ALLOC1]] : memref<?x?xf32, #[[MAP1]]>) // CHECK-SAME: outs(%[[SV_ALLOC1]] : memref<?x?xf32, #[[MAP1]]>)
// CHECK: linalg.fill(%[[SV_ALLOC2]], %{{.+}}) // CHECK: linalg.fill(%{{.+}}, %[[SV_ALLOC2]])
// CHECK: linalg.matmul ins(%[[SV_ALLOC1]], %[[ARG2]] // CHECK: linalg.matmul ins(%[[SV_ALLOC1]], %[[ARG2]]
// CHECK-SAME: : memref<?x?xf32, #[[MAP1]]>, memref<?x?xf32>) // CHECK-SAME: : memref<?x?xf32, #[[MAP1]]>, memref<?x?xf32>)
// CHECK-SAME: outs(%[[SV_ALLOC2]] : memref<?x?xf32, #[[MAP1]]>) // CHECK-SAME: outs(%[[SV_ALLOC2]] : memref<?x?xf32, #[[MAP1]]>)
// CHECK: linalg.fill(%[[SV_ARG4_2]], %{{.+}}) // CHECK: linalg.fill(%{{.+}}, %[[SV_ARG4_2]])
// CHECK: linalg.matmul ins(%[[SV_ALLOC3]], %[[ARG3]] // CHECK: linalg.matmul ins(%[[SV_ALLOC3]], %[[ARG3]]
// CHECK-SAME: : memref<?x?xf32, #[[MAP1]]>, memref<?x?xf32>) // CHECK-SAME: : memref<?x?xf32, #[[MAP1]]>, memref<?x?xf32>)
// CHECK-SAME: outs(%[[SV_ARG4]] : memref<?x?xf32, #[[MAP1]]>) // CHECK-SAME: outs(%[[SV_ARG4]] : memref<?x?xf32, #[[MAP1]]>)
// CHECK: scf.yield // CHECK: scf.yield
// CHECK: } // CHECK: }
// ----- // -----
▲ Show 20 Lines • Show All 117 Lines • Show Last 20 Lines

mlir/test/Dialect/Linalg/fusion-tensor-pattern.mlir

Show First 20 Lines • Show All 245 Lines • ▼ Show 20 Lines
// TLOOP: return %[[RESULT]] : [[TY]] // TLOOP: return %[[RESULT]] : [[TY]]
// ----- // -----
module { module {
func @matmul_out_fusion(%arg0: tensor<?x?xf32>, %arg1: tensor<?x?xf32>, func @matmul_out_fusion(%arg0: tensor<?x?xf32>, %arg1: tensor<?x?xf32>,
%arg2: tensor<?x?xf32>) -> tensor<?x?xf32> { %arg2: tensor<?x?xf32>) -> tensor<?x?xf32> {
%c0 = constant 0.0 : f32 %c0 = constant 0.0 : f32
%0 = linalg.fill(%arg0, %c0) : tensor<?x?xf32>, f32 -> tensor<?x?xf32> %0 = linalg.fill(%c0, %arg0) : f32, tensor<?x?xf32> -> tensor<?x?xf32>
%1 = linalg.matmul {__internal_linalg_transform__ = "out_fusion"} %1 = linalg.matmul {__internal_linalg_transform__ = "out_fusion"}
ins(%arg1, %arg2 : tensor<?x?xf32>, tensor<?x?xf32>) ins(%arg1, %arg2 : tensor<?x?xf32>, tensor<?x?xf32>)
outs(%0 : tensor<?x?xf32>) -> tensor<?x?xf32> outs(%0 : tensor<?x?xf32>) -> tensor<?x?xf32>
return %1 : tensor<?x?xf32> return %1 : tensor<?x?xf32>
} }
} }
// CHECK-LABEL: func @matmul_out_fusion( // CHECK-LABEL: func @matmul_out_fusion(
// CHECK-SAME: %[[ARG0:[a-zA-Z0-9_]+]]: tensor<?x?xf32> // CHECK-SAME: %[[ARG0:[a-zA-Z0-9_]+]]: tensor<?x?xf32>
// CHECK-SAME: %[[ARG1:[a-zA-Z0-9_]+]]: tensor<?x?xf32> // CHECK-SAME: %[[ARG1:[a-zA-Z0-9_]+]]: tensor<?x?xf32>
// CHECK-SAME: %[[ARG2:[a-zA-Z0-9_]+]]: tensor<?x?xf32> // CHECK-SAME: %[[ARG2:[a-zA-Z0-9_]+]]: tensor<?x?xf32>
// CHECK: %[[C0:.*]] = constant 0.0{{.*}} : f32 // CHECK: %[[C0:.*]] = constant 0.0{{.*}} : f32
// CHECK-NOT: fill // CHECK-NOT: fill
// CHECK: scf.for %[[I:.*]]{{.*}}iter_args(%{{.*}} = %[[ARG0]]) -> (tensor<?x?xf32>) { // CHECK: scf.for %[[I:.*]]{{.*}}iter_args(%{{.*}} = %[[ARG0]]) -> (tensor<?x?xf32>) {
// CHECK: scf.for %[[J:.*]] // CHECK: scf.for %[[J:.*]]
// CHECK: %[[ST:.*]] = tensor.extract_slice %[[ARG0]] // CHECK: %[[ST:.*]] = tensor.extract_slice %[[ARG0]]
// CHECK: %[[ST_FILL:.*]] = linalg.fill(%[[ST]], %[[C0]]) {__internal_linalg_transform__ = "after_out_fusion_producer"} : tensor<?x?xf32>, f32 -> tensor<?x?xf32> // CHECK: %[[ST_FILL:.*]] = linalg.fill(%[[C0]], %[[ST]]) {__internal_linalg_transform__ = "after_out_fusion_producer"} : f32, tensor<?x?xf32> -> tensor<?x?xf32>
// CHECK: %[[ST_MM_RES:.*]] = scf.for %[[K:.*]]{{.*}}iter_args(%[[BB:.*]] = %[[ST_FILL]]) -> (tensor<?x?xf32>) { // CHECK: %[[ST_MM_RES:.*]] = scf.for %[[K:.*]]{{.*}}iter_args(%[[BB:.*]] = %[[ST_FILL]]) -> (tensor<?x?xf32>) {
// CHECK-NOT: fill // CHECK-NOT: fill
// CHECK: %[[ST_MM:.*]] = linalg.matmul {__internal_linalg_transform__ = "after_out_fusion"} ins(%{{.*}}, %{{.*}} : tensor<?x?xf32>, tensor<?x?xf32>) outs(%[[BB]] : tensor<?x?xf32>) -> tensor<?x?xf32> // CHECK: %[[ST_MM:.*]] = linalg.matmul {__internal_linalg_transform__ = "after_out_fusion"} ins(%{{.*}}, %{{.*}} : tensor<?x?xf32>, tensor<?x?xf32>) outs(%[[BB]] : tensor<?x?xf32>) -> tensor<?x?xf32>
// CHECK: scf.yield %[[ST_MM]] : tensor<?x?xf32> // CHECK: scf.yield %[[ST_MM]] : tensor<?x?xf32>
// CHECK: %[[MM:.*]] = tensor.insert_slice %[[ST_MM_RES]] into {{.*}} // CHECK: %[[MM:.*]] = tensor.insert_slice %[[ST_MM_RES]] into {{.*}}
// CHECK: scf.yield %[[MM]] : tensor<?x?xf32> // CHECK: scf.yield %[[MM]] : tensor<?x?xf32>
Show All 19 Lines
// TLOOP-SAME: %[[B_:.*]] = %[[B]]: [[TY]], // TLOOP-SAME: %[[B_:.*]] = %[[B]]: [[TY]],
// TLOOP-SAME: %[[C0_F32_:.*]] = %[[C0_F32]] // TLOOP-SAME: %[[C0_F32_:.*]] = %[[C0_F32]]
// TLOOP-SAME: outs (%[[OUT_:.*]] = %[[OUT]]: [[TY]]) { // TLOOP-SAME: outs (%[[OUT_:.*]] = %[[OUT]]: [[TY]]) {
// TLOOP: %[[DIM_A__1:.*]] = memref.dim %[[A_]], %[[C1]] : [[TY]] // TLOOP: %[[DIM_A__1:.*]] = memref.dim %[[A_]], %[[C1]] : [[TY]]
// TLOOP: %[[A_SUB:.*]] = tensor.extract_slice %[[A_]][%[[I]], 0] // TLOOP: %[[A_SUB:.*]] = tensor.extract_slice %[[A_]][%[[I]], 0]
// TLOOP: %[[B_SUB:.*]] = tensor.extract_slice %[[B_]][0, %[[J]]] // TLOOP: %[[B_SUB:.*]] = tensor.extract_slice %[[B_]][0, %[[J]]]
// TLOOP: %[[OUT_SUB:.*]] = tensor.extract_slice %[[OUT_]][%[[I]], %[[J]]] // TLOOP: %[[OUT_SUB:.*]] = tensor.extract_slice %[[OUT_]][%[[I]], %[[J]]]
// TLOOP: %[[INIT_SUB:.*]] = linalg.fill(%[[OUT_SUB]], %[[C0_F32_]]) // TLOOP: %[[INIT_SUB:.*]] = linalg.fill(%[[C0_F32_]], %[[OUT_SUB]])
// TLOOP: %[[AB_SUB:.*]] = linalg.tiled_loop (%[[K:.*]]) = (%[[C0]]) // TLOOP: %[[AB_SUB:.*]] = linalg.tiled_loop (%[[K:.*]]) = (%[[C0]])
// TLOOP-SAME: to (%[[DIM_A__1]]) step (%[[C16]]) // TLOOP-SAME: to (%[[DIM_A__1]]) step (%[[C16]])
// TLOOP-SAME: ins (%[[A_SUB_:.*]] = %[[A_SUB]]: [[TY]], // TLOOP-SAME: ins (%[[A_SUB_:.*]] = %[[A_SUB]]: [[TY]],
// TLOOP-SAME: %[[B_SUB_:.*]] = %[[B_SUB]]: [[TY]]) // TLOOP-SAME: %[[B_SUB_:.*]] = %[[B_SUB]]: [[TY]])
// TLOOP-SAME: outs (%[[INIT_SUB_:.*]] = %[[INIT_SUB]]: [[TY]]) // TLOOP-SAME: outs (%[[INIT_SUB_:.*]] = %[[INIT_SUB]]: [[TY]])
// TLOOP-SAME: iterators["reduction"] { // TLOOP-SAME: iterators["reduction"] {
▲ Show 20 Lines • Show All 87 Lines • Show Last 20 Lines

mlir/test/Dialect/Linalg/fusion.mlir

Show First 20 Lines • Show All 672 Lines • ▼ Show 20 Lines
// ----- // -----
#map0 = affine_map<(d0, d1, d2) -> (d0, d1 - d2)> #map0 = affine_map<(d0, d1, d2) -> (d0, d1 - d2)>
#map1 = affine_map<(d0, d1, d2, d3)[s0, s1, s2, s3, s4] -> (d0 * s1 + s0 + d1 * s2 + d2 * s3 + d3 * s4)> #map1 = affine_map<(d0, d1, d2, d3)[s0, s1, s2, s3, s4] -> (d0 * s1 + s0 + d1 * s2 + d2 * s3 + d3 * s4)>
#map2 = affine_map<()[s0] -> (s0 + 3)> #map2 = affine_map<()[s0] -> (s0 + 3)>
func @fill_and_conv(%arg0: memref<?x?x?x?xf32>, %arg1: memref<2x3x1x1xf32>, %arg2: memref<?x?x?x?xf32>) { func @fill_and_conv(%arg0: memref<?x?x?x?xf32>, %arg1: memref<2x3x1x1xf32>, %arg2: memref<?x?x?x?xf32>) {
%cst = constant 0.000000e+00 : f32 %cst = constant 0.000000e+00 : f32
linalg.fill(%arg2, %cst) : memref<?x?x?x?xf32>, f32 linalg.fill(%cst, %arg2) : f32, memref<?x?x?x?xf32>
%c4 = constant 4 : index %c4 = constant 4 : index
%c1 = constant 1 : index %c1 = constant 1 : index
%c0 = constant 0 : index %c0 = constant 0 : index
%c2 = constant 2 : index %c2 = constant 2 : index
%c3 = constant 3 : index %c3 = constant 3 : index
%4 = memref.dim %arg1, %c0 : memref<2x3x1x1xf32> %4 = memref.dim %arg1, %c0 : memref<2x3x1x1xf32>
%5 = memref.dim %arg1, %c1 : memref<2x3x1x1xf32> %5 = memref.dim %arg1, %c1 : memref<2x3x1x1xf32>
▲ Show 20 Lines • Show All 75 Lines • Show Last 20 Lines

mlir/test/Dialect/Linalg/generalize-named-ops.mlir

Show First 20 Lines • Show All 466 Lines • ▼ Show 20 Lines
// CHECK: ^{{.+}}(%[[BBARG0:.+]]: f32, %[[BBARG1:.+]]: f32, %[[BBARG2:.+]]: f32) // CHECK: ^{{.+}}(%[[BBARG0:.+]]: f32, %[[BBARG1:.+]]: f32, %[[BBARG2:.+]]: f32)
// CHECK-NEXT: %[[CMP:.+]] = cmpf olt, %[[BBARG0]], %[[BBARG2]] : f32 // CHECK-NEXT: %[[CMP:.+]] = cmpf olt, %[[BBARG0]], %[[BBARG2]] : f32
// CHECK-NEXT: %[[RES:.+]] = select %[[CMP]], %[[BBARG0]], %[[BBARG2]] : f32 // CHECK-NEXT: %[[RES:.+]] = select %[[CMP]], %[[BBARG0]], %[[BBARG2]] : f32
// CHECK-NEXT: linalg.yield %[[RES]] : f32 // CHECK-NEXT: linalg.yield %[[RES]] : f32
// ----- // -----
func @generalize_fill(%output: memref<?x?xf32>, %value : f32) { func @generalize_fill(%output: memref<?x?xf32>, %value : f32) {
linalg.fill(%output, %value) : memref<?x?xf32>, f32 linalg.fill(%value, %output) : f32, memref<?x?xf32>
return return
} }
// CHECK-DAG: #[[MAP0:.+]] = affine_map<(d0, d1) -> ()> // CHECK-DAG: #[[MAP0:.+]] = affine_map<(d0, d1) -> ()>
// CHECK-DAG: #[[MAP1:.+]] = affine_map<(d0, d1) -> (d0, d1)> // CHECK-DAG: #[[MAP1:.+]] = affine_map<(d0, d1) -> (d0, d1)>
// CHECK: func @generalize_fill // CHECK: func @generalize_fill
// CHECK-SAME: (%[[ARG0:.+]]: memref<?x?xf32>, %[[VAL:.+]]: f32) // CHECK-SAME: (%[[ARG0:.+]]: memref<?x?xf32>, %[[VAL:.+]]: f32)
Show All 9 Lines

mlir/test/Dialect/Linalg/invalid.mlir

Show First 20 Lines • Show All 635 Lines • ▼ Show 20 Lines
} }
// ----- // -----
func @illegal_fill_tensor_no_return(%arg0 : index, %arg1 : index, %arg2 : f32) func @illegal_fill_tensor_no_return(%arg0 : index, %arg1 : index, %arg2 : f32)
{ {
%0 = linalg.init_tensor [%arg0, %arg1] : tensor<?x?xf32> %0 = linalg.init_tensor [%arg0, %arg1] : tensor<?x?xf32>
// expected-error @+1 {{expected fill op with no result value to use memref type}} // expected-error @+1 {{expected fill op with no result value to use memref type}}
linalg.fill(%0, %arg2) : tensor<?x?xf32>, f32 linalg.fill(%arg2, %0) : f32, tensor<?x?xf32>
} }
// ----- // -----
func @illegal_fill_memref_with_return(%arg0 : memref<?x?xf32>, %arg1 : f32) -> memref<?x?xf32> func @illegal_fill_memref_with_return(%arg0 : memref<?x?xf32>, %arg1 : f32) -> memref<?x?xf32>
{ {
// expected-error @+1 {{unexpected #results > #outputs}} // expected-error @+1 {{unexpected #results > #outputs}}
%0 = linalg.fill(%arg0, %arg1) : memref<?x?xf32>, f32 -> memref<?x?xf32> %0 = linalg.fill(%arg1, %arg0) : f32, memref<?x?xf32> -> memref<?x?xf32>
return %0 : memref<?x?xf32> return %0 : memref<?x?xf32>
} }
// ----- // -----
func @illegal_fill_memref_with_tensor_return func @illegal_fill_memref_with_tensor_return
(%arg0 : memref<?x?xf32>, %arg1 : f32) -> tensor<?x?xf32> (%arg0 : memref<?x?xf32>, %arg1 : f32) -> tensor<?x?xf32>
{ {
// expected-error @+1 {{unexpected #results > #outputs}} // expected-error @+1 {{unexpected #results > #outputs}}
%0 = linalg.fill(%arg0, %arg1) : memref<?x?xf32>, f32 -> tensor<?x?xf32> %0 = linalg.fill(%arg1, %arg0) : f32, memref<?x?xf32> -> tensor<?x?xf32>
return %0 : tensor<?x?xf32> return %0 : tensor<?x?xf32>
} }
// ----- // -----
func @illegal_fill_tensor_with_memref_return func @illegal_fill_tensor_with_memref_return
(%arg0 : tensor<?x?xf32>, %arg1 : f32) -> memref<?x?xf32> (%arg0 : tensor<?x?xf32>, %arg1 : f32) -> memref<?x?xf32>
{ {
// expected-error @+1 {{expected type of operand #1 ('tensor<?x?xf32>') to match type of corresponding result ('memref<?x?xf32>')}} // expected-error @+1 {{expected type of operand #1 ('tensor<?x?xf32>') to match type of corresponding result ('memref<?x?xf32>')}}
%0 = linalg.fill(%arg0, %arg1) : tensor<?x?xf32>, f32 -> memref<?x?xf32> %0 = linalg.fill(%arg1, %arg0) : f32, tensor<?x?xf32> -> memref<?x?xf32>
return %0 : memref<?x?xf32> return %0 : memref<?x?xf32>
} }
// ----- // -----
func @invalid_static_matmul(%arg0: memref<2x4xf32>, %arg1: memref<3x4xf32>, %arg2: memref<2x4xf32>) { func @invalid_static_matmul(%arg0: memref<2x4xf32>, %arg1: memref<3x4xf32>, %arg2: memref<2x4xf32>) {
// expected-error @+1 {{inferred input/output operand #1 has shape's dimension #0 to be 4, but found 3}} // expected-error @+1 {{inferred input/output operand #1 has shape's dimension #0 to be 4, but found 3}}
linalg.matmul ins(%arg0, %arg1 : memref<2x4xf32>, memref<3x4xf32>) linalg.matmul ins(%arg0, %arg1 : memref<2x4xf32>, memref<3x4xf32>)
▲ Show 20 Lines • Show All 137 Lines • Show Last 20 Lines

mlir/test/Dialect/Linalg/loops.mlir

Show First 20 Lines • Show All 176 Lines • ▼ Show 20 Lines
// CHECKPARALLEL-DAG: %[[a:.*]] = memref.load %arg0[%{{.*}}] : memref<?xf32, #[[$strided1D]]> // CHECKPARALLEL-DAG: %[[a:.*]] = memref.load %arg0[%{{.*}}] : memref<?xf32, #[[$strided1D]]>
// CHECKPARALLEL-DAG: %[[b:.*]] = memref.load %{{.*}}[%{{.*}}] : memref<?xf32, #[[$strided1D]]> // CHECKPARALLEL-DAG: %[[b:.*]] = memref.load %{{.*}}[%{{.*}}] : memref<?xf32, #[[$strided1D]]>
// CHECKPARALLEL-DAG: %[[inc:.*]] = mulf %[[a]], %[[b]] : f32 // CHECKPARALLEL-DAG: %[[inc:.*]] = mulf %[[a]], %[[b]] : f32
// CHECKPARALLEL-DAG: %[[c:.*]] = memref.load %{{.*}}[] : memref<f32> // CHECKPARALLEL-DAG: %[[c:.*]] = memref.load %{{.*}}[] : memref<f32>
// CHECKPARALLEL-DAG: %[[res:.*]] = addf %[[c]], %[[inc]] : f32 // CHECKPARALLEL-DAG: %[[res:.*]] = addf %[[c]], %[[inc]] : f32
// CHECKPARALLEL: store %[[res]], %{{.*}}[] : memref<f32> // CHECKPARALLEL: store %[[res]], %{{.*}}[] : memref<f32>
func @fill_view(%arg0: memref<?xf32, offset: ?, strides: [1]>, %arg1: f32) { func @fill_view(%arg0: memref<?xf32, offset: ?, strides: [1]>, %arg1: f32) {
linalg.fill(%arg0, %arg1) : memref<?xf32, offset: ?, strides: [1]>, f32 linalg.fill(%arg1, %arg0) : f32, memref<?xf32, offset: ?, strides: [1]>
return return
} }
// CHECK-LABEL: func @fill_view( // CHECK-LABEL: func @fill_view(
// CHECK: %{{.*}}: memref<?xf32, #[[$strided1D]]>, %{{.*}}: f32) { // CHECK: %{{.*}}: memref<?xf32, #[[$strided1D]]>, %{{.*}}: f32) {
// CHECK: scf.for {{.*}} to %{{.*}} // CHECK: scf.for {{.*}} to %{{.*}}
// CHECK: store %{{.*}}, %{{.*}}[%{{.*}}] : memref<?xf32, #[[$strided1D]]> // CHECK: store %{{.*}}, %{{.*}}[%{{.*}}] : memref<?xf32, #[[$strided1D]]>
// CHECKPARALLEL-LABEL: func @fill_view( // CHECKPARALLEL-LABEL: func @fill_view(
// CHECKPARALLEL: %{{.*}}: memref<?xf32, #[[$strided1D]]>, %{{.*}}: f32) { // CHECKPARALLEL: %{{.*}}: memref<?xf32, #[[$strided1D]]>, %{{.*}}: f32) {
// CHECKPARALLEL: scf.parallel (%{{.*}}) = (%{{.*}}) to (%{{.*}}) step (%{{.*}}) { // CHECKPARALLEL: scf.parallel (%{{.*}}) = (%{{.*}}) to (%{{.*}}) step (%{{.*}}) {
// CHECKPARALLEL: store %{{.*}}, %{{.*}}[%{{.*}}] : memref<?xf32, #[[$strided1D]]> // CHECKPARALLEL: store %{{.*}}, %{{.*}}[%{{.*}}] : memref<?xf32, #[[$strided1D]]>
func @fill_view0(%arg0: memref<f32>, %arg1: f32) { func @fill_view0(%arg0: memref<f32>, %arg1: f32) {
linalg.fill(%arg0, %arg1) : memref<f32>, f32 linalg.fill(%arg1, %arg0) : f32, memref<f32>
return return
} }
// CHECK-LABEL: func @fill_view0(%{{.*}}: memref<f32>, %{{.*}}: f32) { // CHECK-LABEL: func @fill_view0(%{{.*}}: memref<f32>, %{{.*}}: f32) {
// CHECK: store %{{.*}}, %{{.*}}[] : memref<f32> // CHECK: store %{{.*}}, %{{.*}}[] : memref<f32>
// CHECKPARALLEL-LABEL: func @fill_view0(%{{.*}}: memref<f32>, %{{.*}}: f32) { // CHECKPARALLEL-LABEL: func @fill_view0(%{{.*}}: memref<f32>, %{{.*}}: f32) {
// CHECKPARALLEL: store %{{.*}}, %{{.*}}[] : memref<f32> // CHECKPARALLEL: store %{{.*}}, %{{.*}}[] : memref<f32>
func @fill_view3(%arg0: memref<?x?x?xf32, offset: ?, strides: [?, ?, 1]>, %arg1: f32) { func @fill_view3(%arg0: memref<?x?x?xf32, offset: ?, strides: [?, ?, 1]>, %arg1: f32) {
linalg.fill(%arg0, %arg1) : memref<?x?x?xf32, offset: ?, strides: [?, ?, 1]>, f32 linalg.fill(%arg1, %arg0) : f32, memref<?x?x?xf32, offset: ?, strides: [?, ?, 1]>
return return
} }
// CHECK-LABEL: func @fill_view3( // CHECK-LABEL: func @fill_view3(
// CHECK: %{{.*}}: memref<?x?x?xf32, #[[$strided3D]]>, %{{.*}}: f32) { // CHECK: %{{.*}}: memref<?x?x?xf32, #[[$strided3D]]>, %{{.*}}: f32) {
// CHECK: scf.for {{.*}} to %{{.*}} // CHECK: scf.for {{.*}} to %{{.*}}
// CHECK: scf.for {{.*}} to %{{.*}} // CHECK: scf.for {{.*}} to %{{.*}}
// CHECK: scf.for {{.*}} to %{{.*}} // CHECK: scf.for {{.*}} to %{{.*}}
// CHECK: store %{{.*}}, {{.*}} : memref<?x?x?xf32, #[[$strided3D]]> // CHECK: store %{{.*}}, {{.*}} : memref<?x?x?xf32, #[[$strided3D]]>
▲ Show 20 Lines • Show All 1,203 Lines • Show Last 20 Lines

mlir/test/Dialect/Linalg/named-ops.mlir

// RUN: mlir-opt -split-input-file -verify-diagnostics %s | FileCheck %s // RUN: mlir-opt -split-input-file -verify-diagnostics %s | FileCheck %s
// CHECK-LABEL: func @depthwise_conv_2d_input_nhwc_filter_hwcf_tensor // CHECK-LABEL: func @depthwise_conv_2d_input_nhwc_filter_hwcf_tensor
func @depthwise_conv_2d_input_nhwc_filter_hwcf_tensor(%input: tensor<2x4x5x2xf32>, %filter: tensor<2x2x2x3xf32>) -> tensor<2x3x4x2x3xf32> { func @depthwise_conv_2d_input_nhwc_filter_hwcf_tensor(%input: tensor<2x4x5x2xf32>, %filter: tensor<2x2x2x3xf32>) -> tensor<2x3x4x2x3xf32> {
%zero = constant 0.000000e+00 : f32 %zero = constant 0.000000e+00 : f32
%init = linalg.init_tensor [2, 3, 4, 2, 3] : tensor<2x3x4x2x3xf32> %init = linalg.init_tensor [2, 3, 4, 2, 3] : tensor<2x3x4x2x3xf32>
%fill = linalg.fill(%init, %zero) : tensor<2x3x4x2x3xf32>, f32 -> tensor<2x3x4x2x3xf32> %fill = linalg.fill(%zero, %init) : f32, tensor<2x3x4x2x3xf32> -> tensor<2x3x4x2x3xf32>
// CHECK: %{{.+}} = linalg.depthwise_conv_2d_input_nhwc_filter_hwcf // CHECK: %{{.+}} = linalg.depthwise_conv_2d_input_nhwc_filter_hwcf
// CHECK-SAME: {dilations = dense<1> : tensor<2xi64>, strides = dense<1> : tensor<2xi64>} // CHECK-SAME: {dilations = dense<1> : tensor<2xi64>, strides = dense<1> : tensor<2xi64>}
// CHECK-SAME: ins(%{{.+}}, %{{.+}} : tensor<2x4x5x2xf32>, tensor<2x2x2x3xf32>) // CHECK-SAME: ins(%{{.+}}, %{{.+}} : tensor<2x4x5x2xf32>, tensor<2x2x2x3xf32>)
// CHECK-SAME: outs(%{{.+}} : tensor<2x3x4x2x3xf32>) // CHECK-SAME: outs(%{{.+}} : tensor<2x3x4x2x3xf32>)
%0 = linalg.depthwise_conv_2d_input_nhwc_filter_hwcf %0 = linalg.depthwise_conv_2d_input_nhwc_filter_hwcf
{ dilations = dense<1> : tensor<2xi64>, strides = dense<1> : tensor<2xi64> } { dilations = dense<1> : tensor<2xi64>, strides = dense<1> : tensor<2xi64> }
ins(%input, %filter : tensor<2x4x5x2xf32>, tensor<2x2x2x3xf32>) ins(%input, %filter : tensor<2x4x5x2xf32>, tensor<2x2x2x3xf32>)
outs(%fill : tensor<2x3x4x2x3xf32>) -> tensor<2x3x4x2x3xf32> outs(%fill : tensor<2x3x4x2x3xf32>) -> tensor<2x3x4x2x3xf32>
Show All 36 Lines linalg.depthwise_conv_2d_input_nhwc_filter_hwc {dilations = dense<1> : vector<2xi64>, strides = dense<2> : vector<2xi64>}
ins(%input, %filter: memref<1x113x113x96xf32>, memref<3x3x96xf32>) ins(%input, %filter: memref<1x113x113x96xf32>, memref<3x3x96xf32>)
outs(%output: memref<1x56x56x96xf32>) outs(%output: memref<1x56x56x96xf32>)
return return
} }
func @depthwise_conv_2d_input_nhwc_filter_hwcf_tensor_dilated(%input: tensor<2x8x9x2xf32>, %filter: tensor<2x2x2x3xf32>) -> tensor<2x6x7x2x3xf32> { func @depthwise_conv_2d_input_nhwc_filter_hwcf_tensor_dilated(%input: tensor<2x8x9x2xf32>, %filter: tensor<2x2x2x3xf32>) -> tensor<2x6x7x2x3xf32> {
%zero = constant 0.000000e+00 : f32 %zero = constant 0.000000e+00 : f32
%init = linalg.init_tensor [2, 6, 7, 2, 3] : tensor<2x6x7x2x3xf32> %init = linalg.init_tensor [2, 6, 7, 2, 3] : tensor<2x6x7x2x3xf32>
%fill = linalg.fill(%init, %zero) : tensor<2x6x7x2x3xf32>, f32 -> tensor<2x6x7x2x3xf32> %fill = linalg.fill(%zero, %init) : f32, tensor<2x6x7x2x3xf32> -> tensor<2x6x7x2x3xf32>
// CHECK: %{{.+}} = linalg.depthwise_conv_2d_input_nhwc_filter_hwcf // CHECK: %{{.+}} = linalg.depthwise_conv_2d_input_nhwc_filter_hwcf
// CHECK-SAME: {dilations = dense<2> : tensor<2xi64>, strides = dense<1> : tensor<2xi64>} // CHECK-SAME: {dilations = dense<2> : tensor<2xi64>, strides = dense<1> : tensor<2xi64>}
// CHECK-SAME: ins(%{{.+}}, %{{.+}} : tensor<2x8x9x2xf32>, tensor<2x2x2x3xf32>) // CHECK-SAME: ins(%{{.+}}, %{{.+}} : tensor<2x8x9x2xf32>, tensor<2x2x2x3xf32>)
// CHECK-SAME: outs(%{{.+}} : tensor<2x6x7x2x3xf32>) // CHECK-SAME: outs(%{{.+}} : tensor<2x6x7x2x3xf32>)
%0 = linalg.depthwise_conv_2d_input_nhwc_filter_hwcf %0 = linalg.depthwise_conv_2d_input_nhwc_filter_hwcf
{ dilations = dense<2> : tensor<2xi64>, strides = dense<1> : tensor<2xi64> } { dilations = dense<2> : tensor<2xi64>, strides = dense<1> : tensor<2xi64> }
ins(%input, %filter : tensor<2x8x9x2xf32>, tensor<2x2x2x3xf32>) ins(%input, %filter : tensor<2x8x9x2xf32>, tensor<2x2x2x3xf32>)
outs(%fill : tensor<2x6x7x2x3xf32>) -> tensor<2x6x7x2x3xf32> outs(%fill : tensor<2x6x7x2x3xf32>) -> tensor<2x6x7x2x3xf32>
▲ Show 20 Lines • Show All 252 Lines • ▼ Show 20 Lines
// CHECK-SAME: dilations = dense<1> : tensor<2xi64> // CHECK-SAME: dilations = dense<1> : tensor<2xi64>
// CHECK-SAME: strides = dense<1> : tensor<2xi64> // CHECK-SAME: strides = dense<1> : tensor<2xi64>
// CHECK-SAME: ins(%{{.+}}, %{{.+}} : tensor<1x4x4x1xf32>, tensor<3x3xf32>) // CHECK-SAME: ins(%{{.+}}, %{{.+}} : tensor<1x4x4x1xf32>, tensor<3x3xf32>)
// CHECK-SAME: outs(%{{.+}} : tensor<1x2x2x1xf32>) -> tensor<1x2x2x1xf32> // CHECK-SAME: outs(%{{.+}} : tensor<1x2x2x1xf32>) -> tensor<1x2x2x1xf32>
func @pooling_nhwc_sum_tensor(%input: tensor<1x4x4x1xf32>) -> tensor<1x2x2x1xf32> { func @pooling_nhwc_sum_tensor(%input: tensor<1x4x4x1xf32>) -> tensor<1x2x2x1xf32> {
%fake = linalg.init_tensor [3, 3] : tensor<3x3xf32> %fake = linalg.init_tensor [3, 3] : tensor<3x3xf32>
%init = linalg.init_tensor [1, 2, 2, 1] : tensor<1x2x2x1xf32> %init = linalg.init_tensor [1, 2, 2, 1] : tensor<1x2x2x1xf32>
%cst = constant 0.000000e+00 : f32 %cst = constant 0.000000e+00 : f32
%fill = linalg.fill(%init, %cst) : tensor<1x2x2x1xf32>, f32 -> tensor<1x2x2x1xf32> %fill = linalg.fill(%cst, %init) : f32, tensor<1x2x2x1xf32> -> tensor<1x2x2x1xf32>
%res = linalg.pooling_nhwc_sum {dilations = dense<1> : tensor<2xi64>, strides = dense<1> : tensor<2xi64>} %res = linalg.pooling_nhwc_sum {dilations = dense<1> : tensor<2xi64>, strides = dense<1> : tensor<2xi64>}
ins(%input, %fake: tensor<1x4x4x1xf32>, tensor<3x3xf32>) ins(%input, %fake: tensor<1x4x4x1xf32>, tensor<3x3xf32>)
outs(%fill: tensor<1x2x2x1xf32>) -> tensor<1x2x2x1xf32> outs(%fill: tensor<1x2x2x1xf32>) -> tensor<1x2x2x1xf32>
return %res : tensor<1x2x2x1xf32> return %res : tensor<1x2x2x1xf32>
} }
// ----- // -----
Show All 17 Lines
// CHECK-SAME: dilations = dense<1> : tensor<2xi64> // CHECK-SAME: dilations = dense<1> : tensor<2xi64>
// CHECK-SAME: strides = dense<1> : tensor<2xi64> // CHECK-SAME: strides = dense<1> : tensor<2xi64>
// CHECK-SAME: ins(%{{.+}}, %{{.+}} : tensor<1x4x4x1xf32>, tensor<3x3xf32>) // CHECK-SAME: ins(%{{.+}}, %{{.+}} : tensor<1x4x4x1xf32>, tensor<3x3xf32>)
// CHECK-SAME: outs(%{{.+}} : tensor<1x2x2x1xf32>) -> tensor<1x2x2x1xf32> // CHECK-SAME: outs(%{{.+}} : tensor<1x2x2x1xf32>) -> tensor<1x2x2x1xf32>
func @pooling_nhwc_max_tensor(%input: tensor<1x4x4x1xf32>) -> tensor<1x2x2x1xf32> { func @pooling_nhwc_max_tensor(%input: tensor<1x4x4x1xf32>) -> tensor<1x2x2x1xf32> {
%fake = linalg.init_tensor [3, 3] : tensor<3x3xf32> %fake = linalg.init_tensor [3, 3] : tensor<3x3xf32>
%init = linalg.init_tensor [1, 2, 2, 1] : tensor<1x2x2x1xf32> %init = linalg.init_tensor [1, 2, 2, 1] : tensor<1x2x2x1xf32>
%cst = constant 0.000000e+00 : f32 %cst = constant 0.000000e+00 : f32
%fill = linalg.fill(%init, %cst) : tensor<1x2x2x1xf32>, f32 -> tensor<1x2x2x1xf32> %fill = linalg.fill(%cst, %init) : f32, tensor<1x2x2x1xf32> -> tensor<1x2x2x1xf32>
%res = linalg.pooling_nhwc_max {dilations = dense<1> : tensor<2xi64>, strides = dense<1> : tensor<2xi64>} %res = linalg.pooling_nhwc_max {dilations = dense<1> : tensor<2xi64>, strides = dense<1> : tensor<2xi64>}
ins(%input, %fake: tensor<1x4x4x1xf32>, tensor<3x3xf32>) ins(%input, %fake: tensor<1x4x4x1xf32>, tensor<3x3xf32>)
outs(%fill: tensor<1x2x2x1xf32>) -> tensor<1x2x2x1xf32> outs(%fill: tensor<1x2x2x1xf32>) -> tensor<1x2x2x1xf32>
return %res : tensor<1x2x2x1xf32> return %res : tensor<1x2x2x1xf32>
} }
// ----- // -----
Show All 17 Lines
// CHECK-SAME: dilations = dense<1> : tensor<2xi64> // CHECK-SAME: dilations = dense<1> : tensor<2xi64>
// CHECK-SAME: strides = dense<1> : tensor<2xi64> // CHECK-SAME: strides = dense<1> : tensor<2xi64>
// CHECK-SAME: ins(%{{.+}}, %{{.+}} : tensor<1x4x4x1xi8>, tensor<3x3xi8>) // CHECK-SAME: ins(%{{.+}}, %{{.+}} : tensor<1x4x4x1xi8>, tensor<3x3xi8>)
// CHECK-SAME: outs(%{{.+}} : tensor<1x2x2x1xi8>) -> tensor<1x2x2x1xi8> // CHECK-SAME: outs(%{{.+}} : tensor<1x2x2x1xi8>) -> tensor<1x2x2x1xi8>
func @pooling_nhwc_i8_max_tensor(%input: tensor<1x4x4x1xi8>) -> tensor<1x2x2x1xi8> { func @pooling_nhwc_i8_max_tensor(%input: tensor<1x4x4x1xi8>) -> tensor<1x2x2x1xi8> {
%fake = linalg.init_tensor [3, 3] : tensor<3x3xi8> %fake = linalg.init_tensor [3, 3] : tensor<3x3xi8>
%init = linalg.init_tensor [1, 2, 2, 1] : tensor<1x2x2x1xi8> %init = linalg.init_tensor [1, 2, 2, 1] : tensor<1x2x2x1xi8>
%cst = constant 0 : i8 %cst = constant 0 : i8
%fill = linalg.fill(%init, %cst) : tensor<1x2x2x1xi8>, i8 -> tensor<1x2x2x1xi8> %fill = linalg.fill(%cst, %init) : i8, tensor<1x2x2x1xi8> -> tensor<1x2x2x1xi8>
%res = linalg.pooling_nhwc_i8_max {dilations = dense<1> : tensor<2xi64>, strides = dense<1> : tensor<2xi64>} %res = linalg.pooling_nhwc_i8_max {dilations = dense<1> : tensor<2xi64>, strides = dense<1> : tensor<2xi64>}
ins(%input, %fake: tensor<1x4x4x1xi8>, tensor<3x3xi8>) ins(%input, %fake: tensor<1x4x4x1xi8>, tensor<3x3xi8>)
outs(%fill: tensor<1x2x2x1xi8>) -> tensor<1x2x2x1xi8> outs(%fill: tensor<1x2x2x1xi8>) -> tensor<1x2x2x1xi8>
return %res : tensor<1x2x2x1xi8> return %res : tensor<1x2x2x1xi8>
} }
// ----- // -----
Show All 17 Lines
// CHECK-SAME: dilations = dense<1> : tensor<2xi64> // CHECK-SAME: dilations = dense<1> : tensor<2xi64>
// CHECK-SAME: strides = dense<1> : tensor<2xi64> // CHECK-SAME: strides = dense<1> : tensor<2xi64>
// CHECK-SAME: ins(%{{.+}}, %{{.+}} : tensor<1x4x4x1xi16>, tensor<3x3xi16>) // CHECK-SAME: ins(%{{.+}}, %{{.+}} : tensor<1x4x4x1xi16>, tensor<3x3xi16>)
// CHECK-SAME: outs(%{{.+}} : tensor<1x2x2x1xi16>) -> tensor<1x2x2x1xi16> // CHECK-SAME: outs(%{{.+}} : tensor<1x2x2x1xi16>) -> tensor<1x2x2x1xi16>
func @pooling_nhwc_i16_max_tensor(%input: tensor<1x4x4x1xi16>) -> tensor<1x2x2x1xi16> { func @pooling_nhwc_i16_max_tensor(%input: tensor<1x4x4x1xi16>) -> tensor<1x2x2x1xi16> {
%fake = linalg.init_tensor [3, 3] : tensor<3x3xi16> %fake = linalg.init_tensor [3, 3] : tensor<3x3xi16>
%init = linalg.init_tensor [1, 2, 2, 1] : tensor<1x2x2x1xi16> %init = linalg.init_tensor [1, 2, 2, 1] : tensor<1x2x2x1xi16>
%cst = constant 0 : i16 %cst = constant 0 : i16
%fill = linalg.fill(%init, %cst) : tensor<1x2x2x1xi16>, i16 -> tensor<1x2x2x1xi16> %fill = linalg.fill(%cst, %init) : i16, tensor<1x2x2x1xi16> -> tensor<1x2x2x1xi16>
%res = linalg.pooling_nhwc_i16_max {dilations = dense<1> : tensor<2xi64>, strides = dense<1> : tensor<2xi64>} %res = linalg.pooling_nhwc_i16_max {dilations = dense<1> : tensor<2xi64>, strides = dense<1> : tensor<2xi64>}
ins(%input, %fake: tensor<1x4x4x1xi16>, tensor<3x3xi16>) ins(%input, %fake: tensor<1x4x4x1xi16>, tensor<3x3xi16>)
outs(%fill: tensor<1x2x2x1xi16>) -> tensor<1x2x2x1xi16> outs(%fill: tensor<1x2x2x1xi16>) -> tensor<1x2x2x1xi16>
return %res : tensor<1x2x2x1xi16> return %res : tensor<1x2x2x1xi16>
} }
// ----- // -----
Show All 17 Lines
// CHECK-SAME: dilations = dense<1> : tensor<2xi64> // CHECK-SAME: dilations = dense<1> : tensor<2xi64>
// CHECK-SAME: strides = dense<1> : tensor<2xi64> // CHECK-SAME: strides = dense<1> : tensor<2xi64>
// CHECK-SAME: ins(%{{.+}}, %{{.+}} : tensor<1x4x4x1xi32>, tensor<3x3xi32>) // CHECK-SAME: ins(%{{.+}}, %{{.+}} : tensor<1x4x4x1xi32>, tensor<3x3xi32>)
// CHECK-SAME: outs(%{{.+}} : tensor<1x2x2x1xi32>) -> tensor<1x2x2x1xi32> // CHECK-SAME: outs(%{{.+}} : tensor<1x2x2x1xi32>) -> tensor<1x2x2x1xi32>
func @pooling_nhwc_i32_max_tensor(%input: tensor<1x4x4x1xi32>) -> tensor<1x2x2x1xi32> { func @pooling_nhwc_i32_max_tensor(%input: tensor<1x4x4x1xi32>) -> tensor<1x2x2x1xi32> {
%fake = linalg.init_tensor [3, 3] : tensor<3x3xi32> %fake = linalg.init_tensor [3, 3] : tensor<3x3xi32>
%init = linalg.init_tensor [1, 2, 2, 1] : tensor<1x2x2x1xi32> %init = linalg.init_tensor [1, 2, 2, 1] : tensor<1x2x2x1xi32>
%cst = constant 0 : i32 %cst = constant 0 : i32
%fill = linalg.fill(%init, %cst) : tensor<1x2x2x1xi32>, i32 -> tensor<1x2x2x1xi32> %fill = linalg.fill(%cst, %init) : i32, tensor<1x2x2x1xi32> -> tensor<1x2x2x1xi32>
%res = linalg.pooling_nhwc_i32_max {dilations = dense<1> : tensor<2xi64>, strides = dense<1> : tensor<2xi64>} %res = linalg.pooling_nhwc_i32_max {dilations = dense<1> : tensor<2xi64>, strides = dense<1> : tensor<2xi64>}
ins(%input, %fake: tensor<1x4x4x1xi32>, tensor<3x3xi32>) ins(%input, %fake: tensor<1x4x4x1xi32>, tensor<3x3xi32>)
outs(%fill: tensor<1x2x2x1xi32>) -> tensor<1x2x2x1xi32> outs(%fill: tensor<1x2x2x1xi32>) -> tensor<1x2x2x1xi32>
return %res : tensor<1x2x2x1xi32> return %res : tensor<1x2x2x1xi32>
} }
// ----- // -----
Show All 18 Lines
// CHECK-SAME: dilations = dense<1> : tensor<2xi64> // CHECK-SAME: dilations = dense<1> : tensor<2xi64>
// CHECK-SAME: strides = dense<1> : tensor<2xi64> // CHECK-SAME: strides = dense<1> : tensor<2xi64>
// CHECK-SAME: ins(%{{.+}}, %{{.+}} : tensor<1x4x4x1xf32>, tensor<3x3xf32>) // CHECK-SAME: ins(%{{.+}}, %{{.+}} : tensor<1x4x4x1xf32>, tensor<3x3xf32>)
// CHECK-SAME: outs(%{{.+}} : tensor<1x2x2x1xf32>) -> tensor<1x2x2x1xf32> // CHECK-SAME: outs(%{{.+}} : tensor<1x2x2x1xf32>) -> tensor<1x2x2x1xf32>
func @pooling_nhwc_min_tensor(%input: tensor<1x4x4x1xf32>) -> tensor<1x2x2x1xf32> { func @pooling_nhwc_min_tensor(%input: tensor<1x4x4x1xf32>) -> tensor<1x2x2x1xf32> {
%fake = linalg.init_tensor [3, 3] : tensor<3x3xf32> %fake = linalg.init_tensor [3, 3] : tensor<3x3xf32>
%init = linalg.init_tensor [1, 2, 2, 1] : tensor<1x2x2x1xf32> %init = linalg.init_tensor [1, 2, 2, 1] : tensor<1x2x2x1xf32>
%cst = constant 0.000000e+00 : f32 %cst = constant 0.000000e+00 : f32
%fill = linalg.fill(%init, %cst) : tensor<1x2x2x1xf32>, f32 -> tensor<1x2x2x1xf32> %fill = linalg.fill(%cst, %init) : f32, tensor<1x2x2x1xf32> -> tensor<1x2x2x1xf32>
%res = linalg.pooling_nhwc_min {dilations = dense<1> : tensor<2xi64>, strides = dense<1> : tensor<2xi64>} %res = linalg.pooling_nhwc_min {dilations = dense<1> : tensor<2xi64>, strides = dense<1> : tensor<2xi64>}
ins(%input, %fake: tensor<1x4x4x1xf32>, tensor<3x3xf32>) ins(%input, %fake: tensor<1x4x4x1xf32>, tensor<3x3xf32>)
outs(%fill: tensor<1x2x2x1xf32>) -> tensor<1x2x2x1xf32> outs(%fill: tensor<1x2x2x1xf32>) -> tensor<1x2x2x1xf32>
return %res : tensor<1x2x2x1xf32> return %res : tensor<1x2x2x1xf32>
} }
// ----- // -----
Show All 12 Lines

mlir/test/Dialect/Linalg/promotion_options.mlir

Show All 17 Lines
// CHECK: scf.for // CHECK: scf.for
// CHECK: %[[T7:.+]] = memref.subview %[[ARG0]] // CHECK: %[[T7:.+]] = memref.subview %[[ARG0]]
// CHECK: %[[T12:.+]] = memref.subview %[[ARG1]] // CHECK: %[[T12:.+]] = memref.subview %[[ARG1]]
// CHECK: %[[T17:.+]] = memref.subview %[[ARG2]] // CHECK: %[[T17:.+]] = memref.subview %[[ARG2]]
// CHECK: %[[T18:.+]] = memref.alloc(%{{.*}}, %{{.*}}) : memref<?x?xf32, 3> // CHECK: %[[T18:.+]] = memref.alloc(%{{.*}}, %{{.*}}) : memref<?x?xf32, 3>
// CHECK: %[[T19:.+]] = memref.subview %[[T18]] // CHECK: %[[T19:.+]] = memref.subview %[[T18]]
// CHECK: %[[T20:.+]] = memref.alloc(%{{.*}}, %{{.*}}) : memref<?x?xf32, 3> // CHECK: %[[T20:.+]] = memref.alloc(%{{.*}}, %{{.*}}) : memref<?x?xf32, 3>
// CHECK: %[[T21:.+]] = memref.subview %[[T20]] // CHECK: %[[T21:.+]] = memref.subview %[[T20]]
// CHECK: linalg.fill(%[[T19]], %[[C42]]) // CHECK: linalg.fill(%[[C42]], %[[T19]])
// CHECK: linalg.copy(%[[T7]], %[[T19]]) // CHECK: linalg.copy(%[[T7]], %[[T19]])
// CHECK: linalg.fill(%[[T21]], %[[C42]]) // CHECK: linalg.fill(%[[C42]], %[[T21]])
// CHECK: linalg.copy(%[[T17]], %[[T21]]) // CHECK: linalg.copy(%[[T17]], %[[T21]])
// CHECK: linalg.matmul ins(%[[T19]], %[[T12]]{{.*}} outs(%[[T21]] // CHECK: linalg.matmul ins(%[[T19]], %[[T12]]{{.*}} outs(%[[T21]]
// CHECK-NOT: linalg.fill // CHECK-NOT: linalg.fill
// CHECK: linalg.copy(%[[T21]], %[[T17]]) // CHECK: linalg.copy(%[[T21]], %[[T17]])
// CHECK: memref.dealloc %[[T18]] // CHECK: memref.dealloc %[[T18]]
// CHECK: memref.dealloc %[[T20]] // CHECK: memref.dealloc %[[T20]]

mlir/test/Dialect/Linalg/roundtrip.mlir

Show First 20 Lines • Show All 147 Lines • ▼ Show 20 Lines
// CHECK-SAME: ins(%{{.*}}, %{{.*}}: memref<?xf32, #[[$strided1D]]>, // CHECK-SAME: ins(%{{.*}}, %{{.*}}: memref<?xf32, #[[$strided1D]]>,
// CHECK-SAME: memref<?xf32, #[[$strided1D]]>) // CHECK-SAME: memref<?xf32, #[[$strided1D]]>)
// CHECK-SAME: outs(%{{.*}}: memref<f32>) // CHECK-SAME: outs(%{{.*}}: memref<f32>)
// ----- // -----
func @fill_view(%arg0: memref<?xf32, offset: ?, strides: [1]>, %arg1: f32) { func @fill_view(%arg0: memref<?xf32, offset: ?, strides: [1]>, %arg1: f32) {
linalg.fill(%arg0, %arg1) : memref<?xf32, offset: ?, strides: [1]>, f32 linalg.fill(%arg1, %arg0) : f32, memref<?xf32, offset: ?, strides: [1]>
return return
} }
// CHECK-LABEL: func @fill_view( // CHECK-LABEL: func @fill_view(
// CHECK: %{{.*}}: memref<?xf32, #[[$strided1D]]>, %{{.*}}: f32) { // CHECK: %{{.*}}: memref<?xf32, #[[$strided1D]]>, %{{.*}}: f32) {
// CHECK: linalg.fill(%{{.*}}, %{{.*}}) : memref<?xf32, #[[$strided1D]]>, f32 // CHECK: linalg.fill(%{{.*}}, %{{.*}}) : f32, memref<?xf32, #[[$strided1D]]>
// ----- // -----
func @transpose(%arg0: memref<?x?x?xf32, offset: ?, strides: [?, ?, 1]>) { func @transpose(%arg0: memref<?x?x?xf32, offset: ?, strides: [?, ?, 1]>) {
%0 = memref.transpose %arg0 (i, j, k) -> (k, j, i) : memref<?x?x?xf32, offset: ?, strides: [?, ?, 1]> to memref<?x?x?xf32, affine_map<(d0, d1, d2)[s0, s1, s2] -> (d2 * s1 + s0 + d1 * s2 + d0)>> %0 = memref.transpose %arg0 (i, j, k) -> (k, j, i) : memref<?x?x?xf32, offset: ?, strides: [?, ?, 1]> to memref<?x?x?xf32, affine_map<(d0, d1, d2)[s0, s1, s2] -> (d2 * s1 + s0 + d1 * s2 + d0)>>
return return
} }
// CHECK-LABEL: func @transpose // CHECK-LABEL: func @transpose
// CHECK: memref.transpose %{{.*}} ([[i:.*]], [[j:.*]], [[k:.*]]) -> ([[k]], [[j]], [[i]]) : // CHECK: memref.transpose %{{.*}} ([[i:.*]], [[j:.*]], [[k:.*]]) -> ([[k]], [[j]], [[i]]) :
// CHECK-SAME: memref<?x?x?xf32, #[[$strided3D]]> to memref<?x?x?xf32, #[[$strided3DT]]> // CHECK-SAME: memref<?x?x?xf32, #[[$strided3D]]> to memref<?x?x?xf32, #[[$strided3DT]]>
// ----- // -----
func @fill_view3(%arg0: memref<?x?x?xf32, offset: ?, strides: [?, ?, 1]>, %arg1: f32) { func @fill_view3(%arg0: memref<?x?x?xf32, offset: ?, strides: [?, ?, 1]>, %arg1: f32) {
linalg.fill(%arg0, %arg1) : memref<?x?x?xf32, offset: ?, strides: [?, ?, 1]>, f32 linalg.fill(%arg1, %arg0) : f32, memref<?x?x?xf32, offset: ?, strides: [?, ?, 1]>
return return
} }
// CHECK-LABEL: func @fill_view3( // CHECK-LABEL: func @fill_view3(
// CHECK: %{{.*}}: memref<?x?x?xf32, #[[$strided3D]]>, %{{.*}}: f32) { // CHECK: %{{.*}}: memref<?x?x?xf32, #[[$strided3D]]>, %{{.*}}: f32) {
// CHECK: linalg.fill(%{{.*}}, %{{.*}}) : memref<?x?x?xf32, #[[$strided3D]]>, f32 // CHECK: linalg.fill(%{{.*}}, %{{.*}}) : f32, memref<?x?x?xf32, #[[$strided3D]]>
// ----- // -----
func @copy_view(%arg0: memref<?xf32, offset: ?, strides: [1]>, func @copy_view(%arg0: memref<?xf32, offset: ?, strides: [1]>,
%arg1: memref<?xf32, offset: ?, strides: [1]>) { %arg1: memref<?xf32, offset: ?, strides: [1]>) {
linalg.copy(%arg0, %arg1) : memref<?xf32, offset: ?, strides: [1]>, linalg.copy(%arg0, %arg1) : memref<?xf32, offset: ?, strides: [1]>,
memref<?xf32, offset: ?, strides: [1]> memref<?xf32, offset: ?, strides: [1]>
▲ Show 20 Lines • Show All 233 Lines • ▼ Show 20 Lines
// ----- // -----
func @generic_with_multiple_tensor_outputs( func @generic_with_multiple_tensor_outputs(
%arg0: tensor<?xi32>, %arg1: tensor<?xi32>, %arg2: i32) %arg0: tensor<?xi32>, %arg1: tensor<?xi32>, %arg2: i32)
-> (tensor<i32>, tensor<i32>) { -> (tensor<i32>, tensor<i32>) {
%c0 = constant 0 : index %c0 = constant 0 : index
%0 = linalg.init_tensor [] : tensor<i32> %0 = linalg.init_tensor [] : tensor<i32>
%1 = linalg.fill(%0, %arg2) : tensor<i32>, i32 -> tensor<i32> %1 = linalg.fill(%arg2, %0) : i32, tensor<i32> -> tensor<i32>
%2 = linalg.init_tensor [] : tensor<i32> %2 = linalg.init_tensor [] : tensor<i32>
%3 = linalg.fill(%2, %arg2) : tensor<i32>, i32 -> tensor<i32> %3 = linalg.fill(%arg2, %2) : i32, tensor<i32> -> tensor<i32>
%4:2 = linalg.generic { %4:2 = linalg.generic {
indexing_maps = [affine_map<(d0) -> (d0)>, affine_map<(d0) -> (d0)>, affine_map<(d0) -> ()>, affine_map<(d0) -> ()>], indexing_maps = [affine_map<(d0) -> (d0)>, affine_map<(d0) -> (d0)>, affine_map<(d0) -> ()>, affine_map<(d0) -> ()>],
iterator_types = ["reduction"]} iterator_types = ["reduction"]}
ins(%arg0, %arg1 : tensor<?xi32>, tensor<?xi32>) ins(%arg0, %arg1 : tensor<?xi32>, tensor<?xi32>)
outs(%1, %3 : tensor<i32>, tensor<i32>) { outs(%1, %3 : tensor<i32>, tensor<i32>) {
^bb0(%arg3: i32, %arg4: i32, %arg5: i32, %arg6: i32): // no predecessors ^bb0(%arg3: i32, %arg4: i32, %arg5: i32, %arg6: i32): // no predecessors
%5 = cmpi sge, %arg3, %arg5 : i32 %5 = cmpi sge, %arg3, %arg5 : i32
%6 = select %5, %arg3, %arg5 : i32 %6 = select %5, %arg3, %arg5 : i32
▲ Show 20 Lines • Show All 256 Lines • ▼ Show 20 Lines
// CHECK: func @legal_collapsing_reshape_dynamic_memref // CHECK: func @legal_collapsing_reshape_dynamic_memref
// CHECK: linalg.collapse_shape // CHECK: linalg.collapse_shape
// CHECK-SAME: [0], [1], [2, 3, 4] // CHECK-SAME: [0], [1], [2, 3, 4]
// ----- // -----
func @fill_tensor(%arg0 : index, %arg1 : index, %arg2 : f32) -> tensor<?x?xf32> { func @fill_tensor(%arg0 : index, %arg1 : index, %arg2 : f32) -> tensor<?x?xf32> {
%0 = linalg.init_tensor [%arg0, %arg1] : tensor<?x?xf32> %0 = linalg.init_tensor [%arg0, %arg1] : tensor<?x?xf32>
%1 = linalg.fill(%0, %arg2) : tensor<?x?xf32>, f32 -> tensor<?x?xf32> %1 = linalg.fill(%arg2, %0) : f32, tensor<?x?xf32> -> tensor<?x?xf32>
return %1 : tensor<?x?xf32> return %1 : tensor<?x?xf32>
} }
// CHECK: %{{.+}} = linalg.fill(%{{.+}}, %{{.+}}) : tensor<?x?xf32>, f32 -> tensor<?x?xf32> // CHECK: %{{.+}} = linalg.fill(%{{.+}}, %{{.+}}) : f32, tensor<?x?xf32> -> tensor<?x?xf32>
// ----- // -----
#accesses_4 = [ #accesses_4 = [
affine_map<(i, j) -> (i, j)>, affine_map<(i, j) -> (i, j)>,
affine_map<(i, j) -> (i, j)>, affine_map<(i, j) -> (i, j)>,
affine_map<(i, j) -> (i, j)> affine_map<(i, j) -> (i, j)>
] ]
▲ Show 20 Lines • Show All 162 Lines • Show Last 20 Lines

mlir/test/Dialect/Linalg/tile-and-fuse-tensors.mlir

Show First 20 Lines • Show All 71 Lines • ▼ Show 20 Linesfunc @conv_tensors_static(%input: tensor<1x225x225x3xf32>, %filter: tensor<3x3x3x32xf32>, %elementwise: tensor<1x112x112x32xf32>) -> tensor<1x112x112x32xf32> {
%c32 = constant 32 : index %c32 = constant 32 : index
%c16 = constant 16 : index %c16 = constant 16 : index
%c8 = constant 8 : index %c8 = constant 8 : index
%c4 = constant 4 : index %c4 = constant 4 : index
%c0 = constant 0 : index %c0 = constant 0 : index
%cst = constant 0.0 : f32 %cst = constant 0.0 : f32
%init = linalg.init_tensor [1, 112, 112, 32] : tensor<1x112x112x32xf32> %init = linalg.init_tensor [1, 112, 112, 32] : tensor<1x112x112x32xf32>
%fill = linalg.fill(%init, %cst) : tensor<1x112x112x32xf32>, f32 -> tensor<1x112x112x32xf32> %fill = linalg.fill(%cst, %init) : f32, tensor<1x112x112x32xf32> -> tensor<1x112x112x32xf32>
%conv = linalg.conv_2d_input_nhwc_filter_hwcf %conv = linalg.conv_2d_input_nhwc_filter_hwcf
{dilations = dense<1> : tensor<2xi64>, strides = dense<2> : tensor<2xi64>} {dilations = dense<1> : tensor<2xi64>, strides = dense<2> : tensor<2xi64>}
ins(%input, %filter : tensor<1x225x225x3xf32>, tensor<3x3x3x32xf32>) ins(%input, %filter : tensor<1x225x225x3xf32>, tensor<3x3x3x32xf32>)
outs(%fill : tensor<1x112x112x32xf32>) -> tensor<1x112x112x32xf32> outs(%fill : tensor<1x112x112x32xf32>) -> tensor<1x112x112x32xf32>
%for0 = scf.for %iv0 = %c0 to %c112 step %c8 iter_args(%arg0 = %fill) -> tensor<1x112x112x32xf32> { %for0 = scf.for %iv0 = %c0 to %c112 step %c8 iter_args(%arg0 = %fill) -> tensor<1x112x112x32xf32> {
%for1 = scf.for %iv1 = %c0 to %c112 step %c16 iter_args(%arg1 = %arg0) -> tensor<1x112x112x32xf32> { %for1 = scf.for %iv1 = %c0 to %c112 step %c16 iter_args(%arg1 = %arg0) -> tensor<1x112x112x32xf32> {
Show All 27 Lines
// CHECK: #[[MAP0:.+]] = affine_map<(d0) -> (d0 * 2)> // CHECK: #[[MAP0:.+]] = affine_map<(d0) -> (d0 * 2)>
// CHECK: #[[MAP1:.+]] = affine_map<(d0, d1, d2, d3) -> (d0, d1, d2, d3)> // CHECK: #[[MAP1:.+]] = affine_map<(d0, d1, d2, d3) -> (d0, d1, d2, d3)>
// CHECK: func @conv_tensors_static // CHECK: func @conv_tensors_static
// CHECK-SAME: (%[[INPUT:.+]]: tensor<1x225x225x3xf32>, %[[FILTER:.+]]: tensor<3x3x3x32xf32>, %[[ELEM:.+]]: tensor<1x112x112x32xf32>) // CHECK-SAME: (%[[INPUT:.+]]: tensor<1x225x225x3xf32>, %[[FILTER:.+]]: tensor<3x3x3x32xf32>, %[[ELEM:.+]]: tensor<1x112x112x32xf32>)
// CHECK: %[[INIT:.+]] = linalg.init_tensor [1, 112, 112, 32] : tensor<1x112x112x32xf32> // CHECK: %[[INIT:.+]] = linalg.init_tensor [1, 112, 112, 32] : tensor<1x112x112x32xf32>
// CHECK-NEXT: %[[FILL:.+]] = linalg.fill(%[[INIT]], %cst) : tensor<1x112x112x32xf32>, f32 -> tensor<1x112x112x32xf32> // CHECK-NEXT: %[[FILL:.+]] = linalg.fill(%cst, %[[INIT]]) : f32, tensor<1x112x112x32xf32> -> tensor<1x112x112x32xf32>
// CHECK-NEXT: scf.for %[[IV0:.+]] = %{{.+}} to %{{.+}} step %{{.+}} iter_args(%[[ARG0:.+]] = %[[FILL]]) // CHECK-NEXT: scf.for %[[IV0:.+]] = %{{.+}} to %{{.+}} step %{{.+}} iter_args(%[[ARG0:.+]] = %[[FILL]])
// CHECK-NEXT: %[[OFFSET_H:.+]] = affine.apply #[[MAP0]](%[[IV0]]) // CHECK-NEXT: %[[OFFSET_H:.+]] = affine.apply #[[MAP0]](%[[IV0]])
// CHECK-NEXT: scf.for %[[IV1:.+]] = %{{.+}} to %{{.+}} step %{{.+}} iter_args(%[[ARG1:.+]] = %[[ARG0]]) // CHECK-NEXT: scf.for %[[IV1:.+]] = %{{.+}} to %{{.+}} step %{{.+}} iter_args(%[[ARG1:.+]] = %[[ARG0]])
// CHECK-NEXT: %[[OFFSET_W:.+]] = affine.apply #[[MAP0]](%[[IV1]]) // CHECK-NEXT: %[[OFFSET_W:.+]] = affine.apply #[[MAP0]](%[[IV1]])
// CHECK-NEXT: %[[ST_INPUT:.+]] = tensor.extract_slice %arg0[0, %[[OFFSET_H]], %[[OFFSET_W]], 0] [1, 17, 33, 3] [1, 1, 1, 1] : tensor<1x225x225x3xf32> to tensor<1x17x33x3xf32> // CHECK-NEXT: %[[ST_INPUT:.+]] = tensor.extract_slice %arg0[0, %[[OFFSET_H]], %[[OFFSET_W]], 0] [1, 17, 33, 3] [1, 1, 1, 1] : tensor<1x225x225x3xf32> to tensor<1x17x33x3xf32>
// CHECK-NEXT: scf.for %[[IV2:.+]] = %{{.+}} to %{{.+}} step %{{.+}} iter_args(%[[ARG2:.+]] = %[[ARG1]]) // CHECK-NEXT: scf.for %[[IV2:.+]] = %{{.+}} to %{{.+}} step %{{.+}} iter_args(%[[ARG2:.+]] = %[[ARG1]])
// CHECK-NEXT: %[[ST_ELEM:.+]] = tensor.extract_slice %[[ELEM]][0, %[[IV0]], %[[IV1]], %[[IV2]]] [1, 8, 16, 4] [1, 1, 1, 1] : tensor<1x112x112x32xf32> to tensor<1x8x16x4xf32> // CHECK-NEXT: %[[ST_ELEM:.+]] = tensor.extract_slice %[[ELEM]][0, %[[IV0]], %[[IV1]], %[[IV2]]] [1, 8, 16, 4] [1, 1, 1, 1] : tensor<1x112x112x32xf32> to tensor<1x8x16x4xf32>
Show All 21 Linesfunc @conv_tensors_dynamic(%input: tensor<?x?x?x?xf32>, %filter: tensor<?x?x?x?xf32>, %elementwise: tensor<?x?x?x?xf32>) -> tensor<?x?x?x?xf32> {
%c16 = constant 16 : index %c16 = constant 16 : index
%n = memref.dim %elementwise, %c0 : tensor<?x?x?x?xf32> %n = memref.dim %elementwise, %c0 : tensor<?x?x?x?xf32>
%oh = memref.dim %elementwise, %c1 : tensor<?x?x?x?xf32> %oh = memref.dim %elementwise, %c1 : tensor<?x?x?x?xf32>
%ow = memref.dim %elementwise, %c2 : tensor<?x?x?x?xf32> %ow = memref.dim %elementwise, %c2 : tensor<?x?x?x?xf32>
%oc = memref.dim %elementwise, %c3 : tensor<?x?x?x?xf32> %oc = memref.dim %elementwise, %c3 : tensor<?x?x?x?xf32>
%init = linalg.init_tensor [%n, %oh, %ow, %oc] : tensor<?x?x?x?xf32> %init = linalg.init_tensor [%n, %oh, %ow, %oc] : tensor<?x?x?x?xf32>
%fill = linalg.fill(%init, %cst) : tensor<?x?x?x?xf32>, f32 -> tensor<?x?x?x?xf32> %fill = linalg.fill(%cst, %init) : f32, tensor<?x?x?x?xf32> -> tensor<?x?x?x?xf32>
%conv = linalg.conv_2d_input_nhwc_filter_hwcf %conv = linalg.conv_2d_input_nhwc_filter_hwcf
{dilations = dense<1> : tensor<2xi64>, strides = dense<2> : tensor<2xi64>} {dilations = dense<1> : tensor<2xi64>, strides = dense<2> : tensor<2xi64>}
ins(%input, %filter : tensor<?x?x?x?xf32>, tensor<?x?x?x?xf32>) ins(%input, %filter : tensor<?x?x?x?xf32>, tensor<?x?x?x?xf32>)
outs(%fill : tensor<?x?x?x?xf32>) -> tensor<?x?x?x?xf32> outs(%fill : tensor<?x?x?x?xf32>) -> tensor<?x?x?x?xf32>
%for0 = scf.for %iv0 = %c0 to %n step %c8 iter_args(%arg0 = %fill) -> tensor<?x?x?x?xf32> { %for0 = scf.for %iv0 = %c0 to %n step %c8 iter_args(%arg0 = %fill) -> tensor<?x?x?x?xf32> {
%for1 = scf.for %iv1 = %c0 to %oh step %c16 iter_args(%arg1 = %arg0) -> tensor<?x?x?x?xf32> { %for1 = scf.for %iv1 = %c0 to %oh step %c16 iter_args(%arg1 = %arg0) -> tensor<?x?x?x?xf32> {
▲ Show 20 Lines • Show All 52 Lines • ▼ Show 20 Lines
// CHECK-DAG: %[[C3:.+]] = constant 3 : index // CHECK-DAG: %[[C3:.+]] = constant 3 : index
// CHECK-DAG: %[[ELEM_N:.+]] = memref.dim %[[ELEM]], %[[C0]] : tensor<?x?x?x?xf32> // CHECK-DAG: %[[ELEM_N:.+]] = memref.dim %[[ELEM]], %[[C0]] : tensor<?x?x?x?xf32>
// CHECK-DAG: %[[ELEM_OH:.+]] = memref.dim %[[ELEM]], %[[C1]] : tensor<?x?x?x?xf32> // CHECK-DAG: %[[ELEM_OH:.+]] = memref.dim %[[ELEM]], %[[C1]] : tensor<?x?x?x?xf32>
// CHECK-DAG: %[[ELEM_OW:.+]] = memref.dim %[[ELEM]], %[[C2]] : tensor<?x?x?x?xf32> // CHECK-DAG: %[[ELEM_OW:.+]] = memref.dim %[[ELEM]], %[[C2]] : tensor<?x?x?x?xf32>
// CHECK-DAG: %[[ELEM_OC:.+]] = memref.dim %[[ELEM]], %[[C3]] : tensor<?x?x?x?xf32> // CHECK-DAG: %[[ELEM_OC:.+]] = memref.dim %[[ELEM]], %[[C3]] : tensor<?x?x?x?xf32>
// CHECK: %[[INIT:.+]] = linalg.init_tensor [%[[ELEM_N]], %[[ELEM_OH]], %[[ELEM_OW]], %[[ELEM_OC]]] : tensor<?x?x?x?xf32> // CHECK: %[[INIT:.+]] = linalg.init_tensor [%[[ELEM_N]], %[[ELEM_OH]], %[[ELEM_OW]], %[[ELEM_OC]]] : tensor<?x?x?x?xf32>
// CHECK: %[[FILL:.+]] = linalg.fill(%[[INIT]], %cst) : tensor<?x?x?x?xf32>, f32 -> tensor<?x?x?x?xf32> // CHECK: %[[FILL:.+]] = linalg.fill(%cst, %[[INIT]]) : f32, tensor<?x?x?x?xf32> -> tensor<?x?x?x?xf32>
// CHECK-DAG: %[[FILTER_H:.+]] = memref.dim %[[FILTER]], %[[C0]] : tensor<?x?x?x?xf32> // CHECK-DAG: %[[FILTER_H:.+]] = memref.dim %[[FILTER]], %[[C0]] : tensor<?x?x?x?xf32>
// CHECK-DAG: %[[FILTER_W:.+]] = memref.dim %[[FILTER]], %[[C1]] : tensor<?x?x?x?xf32> // CHECK-DAG: %[[FILTER_W:.+]] = memref.dim %[[FILTER]], %[[C1]] : tensor<?x?x?x?xf32>
// CHECK-DAG: %[[INPUT_N:.+]] = memref.dim %[[INPUT]], %[[C0]] : tensor<?x?x?x?xf32> // CHECK-DAG: %[[INPUT_N:.+]] = memref.dim %[[INPUT]], %[[C0]] : tensor<?x?x?x?xf32>
// CHECK-DAG: %[[INPUT_H:.+]] = memref.dim %[[INPUT]], %[[C1]] : tensor<?x?x?x?xf32> // CHECK-DAG: %[[INPUT_H:.+]] = memref.dim %[[INPUT]], %[[C1]] : tensor<?x?x?x?xf32>
// CHECK-DAG: %[[INPUT_W:.+]] = memref.dim %[[INPUT]], %[[C2]] : tensor<?x?x?x?xf32> // CHECK-DAG: %[[INPUT_W:.+]] = memref.dim %[[INPUT]], %[[C2]] : tensor<?x?x?x?xf32>
// CHECK-DAG: %[[INPUT_C:.+]] = memref.dim %[[INPUT]], %[[C3]] : tensor<?x?x?x?xf32> // CHECK-DAG: %[[INPUT_C:.+]] = memref.dim %[[INPUT]], %[[C3]] : tensor<?x?x?x?xf32>
// CHECK-DAG: %[[FILTER_IC:.+]] = memref.dim %[[FILTER]], %[[C2]] : tensor<?x?x?x?xf32> // CHECK-DAG: %[[FILTER_IC:.+]] = memref.dim %[[FILTER]], %[[C2]] : tensor<?x?x?x?xf32>
▲ Show 20 Lines • Show All 74 Lines • ▼ Show 20 Linesfunc @pad_generic_static(%small_input: tensor<58x1xf32>, %large_input: tensor<64x128xf32>) -> tensor<64x128xf32> {
%d0 = memref.dim %large_input, %c0 : tensor<64x128xf32> %d0 = memref.dim %large_input, %c0 : tensor<64x128xf32>
%d1 = memref.dim %large_input, %c1 : tensor<64x128xf32> %d1 = memref.dim %large_input, %c1 : tensor<64x128xf32>
%pad = linalg.pad_tensor %small_input low[4, 60] high[2, 67] { %pad = linalg.pad_tensor %small_input low[4, 60] high[2, 67] {
^bb0(%arg0: index, %arg1: index): ^bb0(%arg0: index, %arg1: index):
linalg.yield %zero : f32 linalg.yield %zero : f32
} : tensor<58x1xf32> to tensor<64x128xf32> } : tensor<58x1xf32> to tensor<64x128xf32>
%fill = linalg.fill(%large_input, %zero) : tensor<64x128xf32>, f32 -> tensor<64x128xf32> %fill = linalg.fill(%zero, %large_input) : f32, tensor<64x128xf32> -> tensor<64x128xf32>
%for0 = scf.for %iv0 = %c0 to %d0 step %c16 iter_args(%arg0 = %fill) -> tensor<64x128xf32> { %for0 = scf.for %iv0 = %c0 to %d0 step %c16 iter_args(%arg0 = %fill) -> tensor<64x128xf32> {
%for1 = scf.for %iv1 = %c0 to %d1 step %c32 iter_args(%arg1 = %arg0) -> tensor<64x128xf32> { %for1 = scf.for %iv1 = %c0 to %d1 step %c32 iter_args(%arg1 = %arg0) -> tensor<64x128xf32> {
%0 = tensor.extract_slice %pad[%iv0, %iv1][16, 32][1, 1] : tensor<64x128xf32> to tensor<16x32xf32> %0 = tensor.extract_slice %pad[%iv0, %iv1][16, 32][1, 1] : tensor<64x128xf32> to tensor<16x32xf32>
%1 = tensor.extract_slice %large_input[%iv0, %iv1][16, 32][1, 1] : tensor<64x128xf32> to tensor<16x32xf32> %1 = tensor.extract_slice %large_input[%iv0, %iv1][16, 32][1, 1] : tensor<64x128xf32> to tensor<16x32xf32>
%2 = tensor.extract_slice %arg1[%iv0, %iv1][16, 32][1, 1] : tensor<64x128xf32> to tensor<16x32xf32> %2 = tensor.extract_slice %arg1[%iv0, %iv1][16, 32][1, 1] : tensor<64x128xf32> to tensor<16x32xf32>
%add = linalg.generic %add = linalg.generic
Show All 14 Lines

mlir/test/Dialect/Linalg/tile.mlir

Show First 20 Lines • Show All 282 Lines • ▼ Show 20 Lines
// TILE-234: %[[szM:.*]] = affine.min #[[$bound_map_2]](%[[I]])[%[[localM]]] // TILE-234: %[[szM:.*]] = affine.min #[[$bound_map_2]](%[[I]])[%[[localM]]]
// TILE-234: %[[sAi:.*]] = memref.subview %{{.*}}[%[[I]]] [%[[szM]]] [1] : memref<?xf32, #[[$strided1D]]> to memref<?xf32, #[[$strided1D]]> // TILE-234: %[[sAi:.*]] = memref.subview %{{.*}}[%[[I]]] [%[[szM]]] [1] : memref<?xf32, #[[$strided1D]]> to memref<?xf32, #[[$strided1D]]>
// TILE-234: %[[localM:.*]] = memref.dim %{{.*}}, %c0 // TILE-234: %[[localM:.*]] = memref.dim %{{.*}}, %c0
// TILE-234: %[[szM:.*]] = affine.min #[[$bound_map_2]](%[[I]])[%[[localM]]] // TILE-234: %[[szM:.*]] = affine.min #[[$bound_map_2]](%[[I]])[%[[localM]]]
// TILE-234: %[[sBi:.*]] = memref.subview %{{.*}}[%[[I]]] [%[[szM]]] [1] : memref<?xf32, #[[$strided1D]]> to memref<?xf32, #[[$strided1D]]> // TILE-234: %[[sBi:.*]] = memref.subview %{{.*}}[%[[I]]] [%[[szM]]] [1] : memref<?xf32, #[[$strided1D]]> to memref<?xf32, #[[$strided1D]]>
// TILE-234: linalg.dot ins(%[[sAi]], %[[sBi]]{{.*}} outs( // TILE-234: linalg.dot ins(%[[sAi]], %[[sBi]]{{.*}} outs(
func @fill_static(%arg0: memref<127x99xf32>, %arg1: f32) { func @fill_static(%arg0: memref<127x99xf32>, %arg1: f32) {
linalg.fill(%arg0, %arg1) : memref<127x99xf32>, f32 linalg.fill(%arg1, %arg0) : f32, memref<127x99xf32>
return return
} }
// TILE-2-LABEL: func @fill_static // TILE-2-LABEL: func @fill_static
// TILE-2: for // TILE-2: for
// TILE-2-NOT: for // TILE-2-NOT: for
// TILE-2: memref.subview{{.*}} : memref<127x99xf32> // TILE-2: memref.subview{{.*}} : memref<127x99xf32>
// TILE-2: linalg.fill{{.*}} : memref<?x99xf32, #[[$stride_99_1_layout_map]]>, f32 // TILE-2: linalg.fill{{.*}} : f32, memref<?x99xf32, #[[$stride_99_1_layout_map]]>
// TILE-02-LABEL: func @fill_static // TILE-02-LABEL: func @fill_static
// TILE-02: for // TILE-02: for
// TILE-02-NOT: for // TILE-02-NOT: for
// TILE-02: memref.subview{{.*}} : memref<127x99xf32> // TILE-02: memref.subview{{.*}} : memref<127x99xf32>
// TILE-02: linalg.fill{{.*}} : memref<127x?xf32, #[[$stride_99_1_layout_map]]>, f32 // TILE-02: linalg.fill{{.*}} : f32, memref<127x?xf32, #[[$stride_99_1_layout_map]]>
// TILE-002-LABEL: func @fill_static // TILE-002-LABEL: func @fill_static
// TILE-002-NOT: for // TILE-002-NOT: for
// TILE-002: linalg.fill{{.*}} memref<127x99xf32>, f32 // TILE-002: linalg.fill{{.*}} f32, memref<127x99xf32>
// TILE-234-LABEL: func @fill_static // TILE-234-LABEL: func @fill_static
// TILE-234: for // TILE-234: for
// TILE-234: for // TILE-234: for
// TILE-234-NOT: for // TILE-234-NOT: for
// TILE-234: memref.subview{{.*}} : memref<127x99xf32> // TILE-234: memref.subview{{.*}} : memref<127x99xf32>
// TILE-234: linalg.fill{{.*}} : memref<?x?xf32, #[[$stride_99_1_layout_map]]>, f32 // TILE-234: linalg.fill{{.*}} : f32, memref<?x?xf32, #[[$stride_99_1_layout_map]]>
func @fill(%arg0: memref<?x?xf32, offset: ?, strides: [?, 1]>, %arg1: f32) { func @fill(%arg0: memref<?x?xf32, offset: ?, strides: [?, 1]>, %arg1: f32) {
linalg.fill(%arg0, %arg1) : memref<?x?xf32, offset: ?, strides: [?, 1]>, f32 linalg.fill(%arg1, %arg0) : f32, memref<?x?xf32, offset: ?, strides: [?, 1]>
return return
} }
// TILE-2-LABEL: func @fill // TILE-2-LABEL: func @fill
// TILE-2: for // TILE-2: for
// TILE-2-NOT: for // TILE-2-NOT: for
// TILE-2: fill{{.*}} f32 // TILE-2: fill{{.*}} f32
// TILE-02-LABEL: func @fill // TILE-02-LABEL: func @fill
▲ Show 20 Lines • Show All 52 Lines • Show Last 20 Lines

mlir/test/Dialect/Linalg/tiled-loops.mlir

Show All 19 Lines linalg.tiled_loop (%i, %j) = (%c0, %c0) to (%c192, %c192) step (%c24, %c16)
%0 = affine.min #map0(%i) %0 = affine.min #map0(%i)
%1 = memref.subview %A_[%i, 0] [%0, 192] [1, 1] %1 = memref.subview %A_[%i, 0] [%0, 192] [1, 1]
: memref<192x192xf32> to memref<?x192xf32, #map1> : memref<192x192xf32> to memref<?x192xf32, #map1>
%2 = affine.min #map2(%j) %2 = affine.min #map2(%j)
%3 = memref.subview %B_[0, %j] [192, %2] [1, 1] %3 = memref.subview %B_[0, %j] [192, %2] [1, 1]
: memref<192x192xf32> to memref<192x?xf32, #map1> : memref<192x192xf32> to memref<192x?xf32, #map1>
%4 = memref.subview %C_[%i, %j] [%0, %2] [1, 1] %4 = memref.subview %C_[%i, %j] [%0, %2] [1, 1]
: memref<192x192xf32> to memref<?x?xf32, #map1> : memref<192x192xf32> to memref<?x?xf32, #map1>
linalg.fill(%4, %cst) : memref<?x?xf32, #map1>, f32 linalg.fill(%cst, %4) : f32, memref<?x?xf32, #map1>
linalg.matmul ins(%1, %3 : memref<?x192xf32, #map1>, linalg.matmul ins(%1, %3 : memref<?x192xf32, #map1>,
memref<192x?xf32, #map1>) memref<192x?xf32, #map1>)
outs(%4 : memref<?x?xf32, #map1>) outs(%4 : memref<?x?xf32, #map1>)
linalg.yield linalg.yield
} }
return return
} }
Show All 21 Linesfunc @tiled_loop_reduction(%A: memref<192x192xf32>,
%c0 = constant 0 : index %c0 = constant 0 : index
%c192 = constant 192 : index %c192 = constant 192 : index
%cst = constant 0.000000e+00 : f32 %cst = constant 0.000000e+00 : f32
linalg.tiled_loop (%i, %j) = (%c0, %c0) to (%c192, %c192) step (%c24, %c16) linalg.tiled_loop (%i, %j) = (%c0, %c0) to (%c192, %c192) step (%c24, %c16)
ins (%A_ = %A: memref<192x192xf32>, %B_ = %B: memref<192x192xf32>) ins (%A_ = %A: memref<192x192xf32>, %B_ = %B: memref<192x192xf32>)
outs (%C_ = %C: memref<f32>) outs (%C_ = %C: memref<f32>)
iterators["reduction", "reduction"] { iterators["reduction", "reduction"] {
linalg.fill(%A_, %cst) : memref<192x192xf32>, f32 linalg.fill(%cst, %A_) : f32, memref<192x192xf32>
linalg.yield linalg.yield
} }
return return
} }
// CHECK-LABEL: @tiled_loop_reduction // CHECK-LABEL: @tiled_loop_reduction
// CHECK: %[[C24:.*]] = constant 24 : index // CHECK: %[[C24:.*]] = constant 24 : index
// CHECK: %[[C16:.*]] = constant 16 : index // CHECK: %[[C16:.*]] = constant 16 : index
// CHECK: %[[C0:.*]] = constant 0 : index // CHECK: %[[C0:.*]] = constant 0 : index
// CHECK: %[[C192:.*]] = constant 192 : index // CHECK: %[[C192:.*]] = constant 192 : index
// CHECK: scf.for %{{.*}} = %[[C0]] to %[[C192]] step %[[C24]] // CHECK: scf.for %{{.*}} = %[[C0]] to %[[C192]] step %[[C24]]
// CHECK: scf.for %{{.*}} = %[[C0]] to %[[C192]] step %[[C16]] // CHECK: scf.for %{{.*}} = %[[C0]] to %[[C192]] step %[[C16]]
// CHECK: linalg.fill // CHECK: linalg.fill

mlir/test/Dialect/Linalg/transform-patterns.mlir

Show First 20 Lines • Show All 290 Lines • ▼ Show 20 Lines
func @aligned_promote_fill(%arg0: memref<?x?xf32, offset: ?, strides: [?, 1]>) { func @aligned_promote_fill(%arg0: memref<?x?xf32, offset: ?, strides: [?, 1]>) {
%c2000 = constant 2000 : index %c2000 = constant 2000 : index
%c4000 = constant 4000 : index %c4000 = constant 4000 : index
%c0 = constant 0 : index %c0 = constant 0 : index
%c1 = constant 1 : index %c1 = constant 1 : index
%cf = constant 1.0 : f32 %cf = constant 1.0 : f32
%3 = memref.subview %arg0[%c0, %c0][%c2000, %c4000][%c1, %c1] : %3 = memref.subview %arg0[%c0, %c0][%c2000, %c4000][%c1, %c1] :
memref<?x?xf32, offset: ?, strides: [?, 1]> to memref<?x?xf32, offset: ?, strides: [?, ?]> memref<?x?xf32, offset: ?, strides: [?, 1]> to memref<?x?xf32, offset: ?, strides: [?, ?]>
linalg.fill(%3, %cf) { __internal_linalg_transform__ = "_promote_views_aligned_"} linalg.fill(%cf, %3) { __internal_linalg_transform__ = "_promote_views_aligned_"}
: memref<?x?xf32, offset: ?, strides: [?, ?]>, f32 : f32, memref<?x?xf32, offset: ?, strides: [?, ?]>
return return
} }
// CHECK-LABEL: func @aligned_promote_fill // CHECK-LABEL: func @aligned_promote_fill
// CHECK: %[[cf:.*]] = constant {{.*}} : f32 // CHECK: %[[cf:.*]] = constant {{.*}} : f32
// CHECK: %[[s0:.*]] = memref.subview {{.*}}: memref<?x?xf32, #map{{.*}}> to memref<?x?xf32, #map{{.*}}> // CHECK: %[[s0:.*]] = memref.subview {{.*}}: memref<?x?xf32, #map{{.*}}> to memref<?x?xf32, #map{{.*}}>
// CHECK: %[[a0:.*]] = memref.alloc() {alignment = 32 : i64} : memref<32000000xi8> // CHECK: %[[a0:.*]] = memref.alloc() {alignment = 32 : i64} : memref<32000000xi8>
// CHECK: %[[v0:.*]] = memref.view %[[a0]]{{.*}} : memref<32000000xi8> to memref<?x?xf32> // CHECK: %[[v0:.*]] = memref.view %[[a0]]{{.*}} : memref<32000000xi8> to memref<?x?xf32>
// CHECK: %[[l0:.*]] = memref.subview %[[v0]][0, 0] [%{{.*}}, %{{.*}}] [1, 1] : memref<?x?xf32> to memref<?x?xf32, #[[$STRIDED_2D_u_1]]> // CHECK: %[[l0:.*]] = memref.subview %[[v0]][0, 0] [%{{.*}}, %{{.*}}] [1, 1] : memref<?x?xf32> to memref<?x?xf32, #[[$STRIDED_2D_u_1]]>
// CHECK: linalg.fill(%[[v0]], {{.*}}) : memref<?x?xf32>, f32 // CHECK: linalg.fill({{.*}}, %[[v0]]) : f32, memref<?x?xf32>
// CHECK: linalg.copy(%[[s0]], %[[l0]]) : memref<?x?xf32, #map{{.*}}>, memref<?x?xf32, #map{{.*}}> // CHECK: linalg.copy(%[[s0]], %[[l0]]) : memref<?x?xf32, #map{{.*}}>, memref<?x?xf32, #map{{.*}}>
// CHECK: linalg.fill(%[[v0]], %[[cf]]) : memref<?x?xf32>, f32 // CHECK: linalg.fill(%[[cf]], %[[v0]]) : f32, memref<?x?xf32>
func @aligned_promote_fill_complex(%arg0: memref<?x?xcomplex<f32>, offset: ?, strides: [?, 1]>) { func @aligned_promote_fill_complex(%arg0: memref<?x?xcomplex<f32>, offset: ?, strides: [?, 1]>) {
%c2000 = constant 2000 : index %c2000 = constant 2000 : index
%c4000 = constant 4000 : index %c4000 = constant 4000 : index
%c0 = constant 0 : index %c0 = constant 0 : index
%c1 = constant 1 : index %c1 = constant 1 : index
%cf = constant 1.0 : f32 %cf = constant 1.0 : f32
%cc = complex.create %cf, %cf : complex<f32> %cc = complex.create %cf, %cf : complex<f32>
%3 = memref.subview %arg0[%c0, %c0][%c2000, %c4000][%c1, %c1] : %3 = memref.subview %arg0[%c0, %c0][%c2000, %c4000][%c1, %c1] :
memref<?x?xcomplex<f32>, offset: ?, strides: [?, 1]> to memref<?x?xcomplex<f32>, offset: ?, strides: [?, ?]> memref<?x?xcomplex<f32>, offset: ?, strides: [?, 1]> to memref<?x?xcomplex<f32>, offset: ?, strides: [?, ?]>
linalg.fill(%3, %cc) { __internal_linalg_transform__ = "_promote_views_aligned_"} linalg.fill(%cc, %3) { __internal_linalg_transform__ = "_promote_views_aligned_"}
: memref<?x?xcomplex<f32>, offset: ?, strides: [?, ?]>, complex<f32> : complex<f32>, memref<?x?xcomplex<f32>, offset: ?, strides: [?, ?]>
return return
} }
// CHECK-LABEL: func @aligned_promote_fill_complex // CHECK-LABEL: func @aligned_promote_fill_complex
// CHECK: %[[cc:.*]] = complex.create {{.*}} : complex<f32> // CHECK: %[[cc:.*]] = complex.create {{.*}} : complex<f32>
// CHECK: %[[s0:.*]] = memref.subview {{.*}}: memref<?x?xcomplex<f32>, #map{{.*}}> to memref<?x?xcomplex<f32>, #map{{.*}}> // CHECK: %[[s0:.*]] = memref.subview {{.*}}: memref<?x?xcomplex<f32>, #map{{.*}}> to memref<?x?xcomplex<f32>, #map{{.*}}>
// CHECK: %[[a0:.*]] = memref.alloc() {alignment = 32 : i64} : memref<64000000xi8> // CHECK: %[[a0:.*]] = memref.alloc() {alignment = 32 : i64} : memref<64000000xi8>
// CHECK: %[[v0:.*]] = memref.view %[[a0]]{{.*}} : memref<64000000xi8> to memref<?x?xcomplex<f32>> // CHECK: %[[v0:.*]] = memref.view %[[a0]]{{.*}} : memref<64000000xi8> to memref<?x?xcomplex<f32>>
// CHECK: %[[l0:.*]] = memref.subview %[[v0]][0, 0] [%{{.*}}, %{{.*}}] [1, 1] : memref<?x?xcomplex<f32>> to memref<?x?xcomplex<f32>, #[[$STRIDED_2D_u_1]]> // CHECK: %[[l0:.*]] = memref.subview %[[v0]][0, 0] [%{{.*}}, %{{.*}}] [1, 1] : memref<?x?xcomplex<f32>> to memref<?x?xcomplex<f32>, #[[$STRIDED_2D_u_1]]>
// CHECK: linalg.fill(%[[v0]], {{.*}}) : memref<?x?xcomplex<f32>>, complex<f32> // CHECK: linalg.fill({{.*}}, %[[v0]]) : complex<f32>, memref<?x?xcomplex<f32>>
// CHECK: linalg.copy(%[[s0]], %[[l0]]) : memref<?x?xcomplex<f32>, #map{{.*}}>, memref<?x?xcomplex<f32>, #map{{.*}}> // CHECK: linalg.copy(%[[s0]], %[[l0]]) : memref<?x?xcomplex<f32>, #map{{.*}}>, memref<?x?xcomplex<f32>, #map{{.*}}>
// CHECK: linalg.fill(%[[v0]], %[[cc]]) : memref<?x?xcomplex<f32>>, complex<f32> // CHECK: linalg.fill(%[[cc]], %[[v0]]) : complex<f32>, memref<?x?xcomplex<f32>>
func @tile_permute_parallel_loop(%arg0: memref<?x?xf32>, func @tile_permute_parallel_loop(%arg0: memref<?x?xf32>,
%arg1: memref<?x?xf32>, %arg1: memref<?x?xf32>,
%arg2: memref<?x?xf32>) { %arg2: memref<?x?xf32>) {
linalg.matmul {__internal_linalg_transform__ = "par__with_perm__"} linalg.matmul {__internal_linalg_transform__ = "par__with_perm__"}
ins(%arg0, %arg1: memref<?x?xf32>, memref<?x?xf32>) ins(%arg0, %arg1: memref<?x?xf32>, memref<?x?xf32>)
outs(%arg2: memref<?x?xf32>) outs(%arg2: memref<?x?xf32>)
return return
Show All 15 Lines

mlir/test/Dialect/Linalg/vectorization.mlir

Show First 20 Lines • Show All 151 Lines • ▼ Show 20 Lines
} }
// ----- // -----
// CHECK-LABEL: func @test_vectorize_fill // CHECK-LABEL: func @test_vectorize_fill
func @test_vectorize_fill(%A : memref<8x16xf32>, %arg0 : f32) { func @test_vectorize_fill(%A : memref<8x16xf32>, %arg0 : f32) {
// CHECK: %[[V:.*]] = vector.broadcast {{.*}} : f32 to vector<8x16xf32> // CHECK: %[[V:.*]] = vector.broadcast {{.*}} : f32 to vector<8x16xf32>
// CHECK: vector.transfer_write %[[V]], {{.*}} : vector<8x16xf32>, memref<8x16xf32> // CHECK: vector.transfer_write %[[V]], {{.*}} : vector<8x16xf32>, memref<8x16xf32>
linalg.fill(%A, %arg0) : memref<8x16xf32>, f32 linalg.fill(%arg0, %A) : f32, memref<8x16xf32>
return return
} }
// ----- // -----
// CHECK-LABEL: func @test_vectorize_fill // CHECK-LABEL: func @test_vectorize_fill
func @test_vectorize_fill_scalar(%A : memref<f32>, %arg0 : f32) { func @test_vectorize_fill_scalar(%A : memref<f32>, %arg0 : f32) {
// CHECK-SAME: (%[[M:.*]]: memref<f32>, %[[V:.*]]: f32) // CHECK-SAME: (%[[M:.*]]: memref<f32>, %[[V:.*]]: f32)
// CHECK: store %[[V]], %[[M]][] : memref<f32> // CHECK: store %[[V]], %[[M]][] : memref<f32>
linalg.fill(%A, %arg0) : memref<f32>, f32 linalg.fill(%arg0, %A) : f32, memref<f32>
return return
} }
// ----- // -----
// CHECK-LABEL: func @test_vectorize_copy // CHECK-LABEL: func @test_vectorize_copy
func @test_vectorize_copy(%A : memref<8x16xf32>, %B : memref<8x16xf32>) { func @test_vectorize_copy(%A : memref<8x16xf32>, %B : memref<8x16xf32>) {
// CHECK: %[[V:.*]] = vector.transfer_read {{.*}} : memref<8x16xf32>, vector<8x16xf32> // CHECK: %[[V:.*]] = vector.transfer_read {{.*}} : memref<8x16xf32>, vector<8x16xf32>
▲ Show 20 Lines • Show All 400 Lines • ▼ Show 20 Lines
// CHECK-DAG: %[[C5:.*]] = constant 5 : index // CHECK-DAG: %[[C5:.*]] = constant 5 : index
// CHECK: %[[V0:.*]] = addi %[[LOW]], %[[C2]] : index // CHECK: %[[V0:.*]] = addi %[[LOW]], %[[C2]] : index
// CHECK: %[[V1:.*]] = addi %[[V0]], %[[C3]] : index // CHECK: %[[V1:.*]] = addi %[[V0]], %[[C3]] : index
// CHECK: %[[V2:.*]] = addi %[[HIGH]], %[[C5]] : index // CHECK: %[[V2:.*]] = addi %[[HIGH]], %[[C5]] : index
// CHECK: %[[DIM3:.*]] = memref.dim %[[SRC]], %[[C3]] : tensor<1x2x2x?xf32> // CHECK: %[[DIM3:.*]] = memref.dim %[[SRC]], %[[C3]] : tensor<1x2x2x?xf32>
// CHECK: %[[V4:.*]] = addi %[[DIM3]], %[[C3]] : index // CHECK: %[[V4:.*]] = addi %[[DIM3]], %[[C3]] : index
// CHECK: %[[V5:.*]] = addi %[[V4]], %[[C2]] : index // CHECK: %[[V5:.*]] = addi %[[V4]], %[[C2]] : index
// CHECK: %[[INIT:.*]] = linalg.init_tensor [6, %[[V1]], %[[V2]], %[[V5]]] : tensor<6x?x?x?xf32> // CHECK: %[[INIT:.*]] = linalg.init_tensor [6, %[[V1]], %[[V2]], %[[V5]]] : tensor<6x?x?x?xf32>
// CHECK: %[[FILL:.*]] = linalg.fill(%[[INIT]], %{{.*}}) : tensor<6x?x?x?xf32>, f32 -> tensor<6x?x?x?xf32> // CHECK: %[[FILL:.*]] = linalg.fill(%{{.*}}, %[[INIT]]) : f32, tensor<6x?x?x?xf32> -> tensor<6x?x?x?xf32>
// CHECK: %[[SRCDIM:.*]] = memref.dim %[[SRC]], %[[C3]] : tensor<1x2x2x?xf32> // CHECK: %[[SRCDIM:.*]] = memref.dim %[[SRC]], %[[C3]] : tensor<1x2x2x?xf32>
// CHECK: %[[RESULT:.*]] = tensor.insert_slice %[[SRC]] into %[[FILL]][2, %[[LOW]], 3, 3] [1, 2, 2, %[[SRCDIM]]] [1, 1, 1, 1] : tensor<1x2x2x?xf32> into tensor<6x?x?x?xf32> // CHECK: %[[RESULT:.*]] = tensor.insert_slice %[[SRC]] into %[[FILL]][2, %[[LOW]], 3, 3] [1, 2, 2, %[[SRCDIM]]] [1, 1, 1, 1] : tensor<1x2x2x?xf32> into tensor<6x?x?x?xf32>
// CHECK: return %[[RESULT]] // CHECK: return %[[RESULT]]
func @pad_static_dynamic(%arg0: tensor<1x2x2x?xf32>, %low: index, %high: index, func @pad_static_dynamic(%arg0: tensor<1x2x2x?xf32>, %low: index, %high: index,
%pad_value: f32) -> tensor<6x?x?x?xf32> { %pad_value: f32) -> tensor<6x?x?x?xf32> {
%0 = linalg.pad_tensor %arg0 low[2, %low, 3, 3] high[3, 3, %high, 2] { %0 = linalg.pad_tensor %arg0 low[2, %low, 3, 3] high[3, 3, %high, 2] {
^bb0(%arg1: index, %arg2: index, %arg3: index, %arg4: index): ^bb0(%arg1: index, %arg2: index, %arg3: index, %arg4: index):
linalg.yield %pad_value : f32 linalg.yield %pad_value : f32
▲ Show 20 Lines • Show All 193 Lines • Show Last 20 Lines

mlir/test/Dialect/Vector/vector-transfer-full-partial-split.mlir

Show First 20 Lines • Show All 69 Lines • ▼ Show 20 Linesfunc @split_vector_transfer_read_2d(%A: memref<?x8xf32>, %i: index, %j: index) -> vector<4x8xf32> {
// LINALG: %[[cmp1:.*]] = cmpi sle, %[[idx1]], %[[c8]] : index // LINALG: %[[cmp1:.*]] = cmpi sle, %[[idx1]], %[[c8]] : index
// are both conds true // are both conds true
// LINALG: %[[cond:.*]] = and %[[cmp0]], %[[cmp1]] : i1 // LINALG: %[[cond:.*]] = and %[[cmp0]], %[[cmp1]] : i1
// LINALG: %[[ifres:.*]]:3 = scf.if %[[cond]] -> (memref<?x8xf32>, index, index) { // LINALG: %[[ifres:.*]]:3 = scf.if %[[cond]] -> (memref<?x8xf32>, index, index) {
// inBounds, just yield %A // inBounds, just yield %A
// LINALG: scf.yield %[[A]], %[[i]], %[[j]] : memref<?x8xf32>, index, index // LINALG: scf.yield %[[A]], %[[i]], %[[j]] : memref<?x8xf32>, index, index
// LINALG: } else { // LINALG: } else {
// slow path, fill tmp alloc and yield a memref_casted version of it // slow path, fill tmp alloc and yield a memref_casted version of it
// LINALG: linalg.fill(%[[alloc]], %cst) : memref<4x8xf32>, f32 // LINALG: linalg.fill(%cst, %[[alloc]]) : f32, memref<4x8xf32>
// LINALG: %[[d0:.*]] = memref.dim %[[A]], %[[c0]] : memref<?x8xf32> // LINALG: %[[d0:.*]] = memref.dim %[[A]], %[[c0]] : memref<?x8xf32>
// LINALG: %[[sv0:.*]] = affine.min #[[$bounds_map_4]](%[[d0]], %[[i]], %[[c4]]) // LINALG: %[[sv0:.*]] = affine.min #[[$bounds_map_4]](%[[d0]], %[[i]], %[[c4]])
// LINALG: %[[sv1:.*]] = affine.min #[[$bounds_map_8]](%[[c8]], %[[j]], %[[c8]]) // LINALG: %[[sv1:.*]] = affine.min #[[$bounds_map_8]](%[[c8]], %[[j]], %[[c8]])
// LINALG: %[[sv:.*]] = memref.subview %[[A]][%[[i]], %[[j]]] [%[[sv0]], %[[sv1]]] [1, 1] // LINALG: %[[sv:.*]] = memref.subview %[[A]][%[[i]], %[[j]]] [%[[sv0]], %[[sv1]]] [1, 1]
// LINALG-SAME: memref<?x8xf32> to memref<?x?xf32, #[[$map_2d_stride_8x1]]> // LINALG-SAME: memref<?x8xf32> to memref<?x?xf32, #[[$map_2d_stride_8x1]]>
// LINALG: linalg.copy(%[[sv]], %[[alloc]]) : memref<?x?xf32, #[[$map_2d_stride_8x1]]>, memref<4x8xf32> // LINALG: linalg.copy(%[[sv]], %[[alloc]]) : memref<?x?xf32, #[[$map_2d_stride_8x1]]>, memref<4x8xf32>
// LINALG: %[[yielded:.*]] = memref.cast %[[alloc]] : // LINALG: %[[yielded:.*]] = memref.cast %[[alloc]] :
// LINALG-SAME: memref<4x8xf32> to memref<?x8xf32> // LINALG-SAME: memref<4x8xf32> to memref<?x8xf32>
▲ Show 20 Lines • Show All 75 Lines • ▼ Show 20 Linesfunc @split_vector_transfer_read_strided_2d(
// LINALG: %[[ifres:.*]]:3 = scf.if %[[cond]] -> (memref<?x8xf32, #[[$map_2d_stride_1]]>, index, index) { // LINALG: %[[ifres:.*]]:3 = scf.if %[[cond]] -> (memref<?x8xf32, #[[$map_2d_stride_1]]>, index, index) {
// inBounds but not cast-compatible: yield a memref_casted form of %A // inBounds but not cast-compatible: yield a memref_casted form of %A
// LINALG: %[[casted:.*]] = memref.cast %arg0 : // LINALG: %[[casted:.*]] = memref.cast %arg0 :
// LINALG-SAME: memref<7x8xf32, #[[$map_2d_stride_1]]> to memref<?x8xf32, #[[$map_2d_stride_1]]> // LINALG-SAME: memref<7x8xf32, #[[$map_2d_stride_1]]> to memref<?x8xf32, #[[$map_2d_stride_1]]>
// LINALG: scf.yield %[[casted]], %[[i]], %[[j]] : // LINALG: scf.yield %[[casted]], %[[i]], %[[j]] :
// LINALG-SAME: memref<?x8xf32, #[[$map_2d_stride_1]]>, index, index // LINALG-SAME: memref<?x8xf32, #[[$map_2d_stride_1]]>, index, index
// LINALG: } else { // LINALG: } else {
// slow path, fill tmp alloc and yield a memref_casted version of it // slow path, fill tmp alloc and yield a memref_casted version of it
// LINALG: linalg.fill(%[[alloc]], %cst) : memref<4x8xf32>, f32 // LINALG: linalg.fill(%cst, %[[alloc]]) : f32, memref<4x8xf32>
// LINALG: %[[sv0:.*]] = affine.min #[[$bounds_map_4]](%[[c7]], %[[i]], %[[c4]]) // LINALG: %[[sv0:.*]] = affine.min #[[$bounds_map_4]](%[[c7]], %[[i]], %[[c4]])
// LINALG: %[[sv1:.*]] = affine.min #[[$bounds_map_8]](%[[c8]], %[[j]], %[[c8]]) // LINALG: %[[sv1:.*]] = affine.min #[[$bounds_map_8]](%[[c8]], %[[j]], %[[c8]])
// LINALG: %[[sv:.*]] = memref.subview %[[A]][%[[i]], %[[j]]] [%[[sv0]], %[[sv1]]] [1, 1] // LINALG: %[[sv:.*]] = memref.subview %[[A]][%[[i]], %[[j]]] [%[[sv0]], %[[sv1]]] [1, 1]
// LINALG-SAME: memref<7x8xf32, #[[$map_2d_stride_1]]> to memref<?x?xf32, #[[$map_2d_stride_1]]> // LINALG-SAME: memref<7x8xf32, #[[$map_2d_stride_1]]> to memref<?x?xf32, #[[$map_2d_stride_1]]>
// LINALG: linalg.copy(%[[sv]], %[[alloc]]) : memref<?x?xf32, #[[$map_2d_stride_1]]>, memref<4x8xf32> // LINALG: linalg.copy(%[[sv]], %[[alloc]]) : memref<?x?xf32, #[[$map_2d_stride_1]]>, memref<4x8xf32>
// LINALG: %[[yielded:.*]] = memref.cast %[[alloc]] : // LINALG: %[[yielded:.*]] = memref.cast %[[alloc]] :
// LINALG-SAME: memref<4x8xf32> to memref<?x8xf32, #[[$map_2d_stride_1]]> // LINALG-SAME: memref<4x8xf32> to memref<?x8xf32, #[[$map_2d_stride_1]]>
// LINALG: scf.yield %[[yielded]], %[[c0]], %[[c0]] : // LINALG: scf.yield %[[yielded]], %[[c0]], %[[c0]] :
▲ Show 20 Lines • Show All 213 Lines • Show Last 20 Lines

mlir/test/Integration/Dialect/Async/CPU/microbench-linalg-async-parallel-for.mlir

Show First 20 Lines • Show All 59 Lines • ▼ Show 20 Linesfunc @entry() {
// //
// Sanity check for the function under test. // Sanity check for the function under test.
// //
%LHS10 = memref.alloc() {alignment = 64} : memref<1x10xf32> %LHS10 = memref.alloc() {alignment = 64} : memref<1x10xf32>
%RHS10 = memref.alloc() {alignment = 64} : memref<1x10xf32> %RHS10 = memref.alloc() {alignment = 64} : memref<1x10xf32>
%DST10 = memref.alloc() {alignment = 64} : memref<1x10xf32> %DST10 = memref.alloc() {alignment = 64} : memref<1x10xf32>
linalg.fill(%LHS10, %f1) : memref<1x10xf32>, f32 linalg.fill(%f1, %LHS10) : f32, memref<1x10xf32>
linalg.fill(%RHS10, %f1) : memref<1x10xf32>, f32 linalg.fill(%f1, %RHS10) : f32, memref<1x10xf32>
%LHS = memref.cast %LHS10 : memref<1x10xf32> to memref<?x?xf32> %LHS = memref.cast %LHS10 : memref<1x10xf32> to memref<?x?xf32>
%RHS = memref.cast %RHS10 : memref<1x10xf32> to memref<?x?xf32> %RHS = memref.cast %RHS10 : memref<1x10xf32> to memref<?x?xf32>
%DST = memref.cast %DST10 : memref<1x10xf32> to memref<?x?xf32> %DST = memref.cast %DST10 : memref<1x10xf32> to memref<?x?xf32>
call @linalg_generic(%LHS, %RHS, %DST) call @linalg_generic(%LHS, %RHS, %DST)
: (memref<?x?xf32>, memref<?x?xf32>, memref<?x?xf32>) -> () : (memref<?x?xf32>, memref<?x?xf32>, memref<?x?xf32>) -> ()
▲ Show 20 Lines • Show All 54 Lines • Show Last 20 Lines

mlir/test/Integration/Dialect/Linalg/CPU/benchmark_matmul.mlir

Show First 20 Lines • Show All 53 Lines • ▼ Show 20 Lines
func @main() { func @main() {
%v0 = constant 0.0 : !elem_type_a %v0 = constant 0.0 : !elem_type_a
%v1 = constant 1.0 : !elem_type_a %v1 = constant 1.0 : !elem_type_a
%A = memref.alloc() : !row_major_A %A = memref.alloc() : !row_major_A
%B = memref.alloc() : !row_major_B %B = memref.alloc() : !row_major_B
%C = memref.alloc() : !row_major_C %C = memref.alloc() : !row_major_C
linalg.fill(%A, %v1) : !row_major_A, !elem_type_a linalg.fill(%v1, %A) : !elem_type_a, !row_major_A
linalg.fill(%B, %v1) : !row_major_B, !elem_type_b linalg.fill(%v1, %B) : !elem_type_b, !row_major_B
linalg.fill(%C, %v0) : !row_major_C, !elem_type_c linalg.fill(%v0, %C) : !elem_type_c, !row_major_C
%c0 = constant 0: index %c0 = constant 0: index
%c1 = constant 1: index %c1 = constant 1: index
%iters = constant ${ITERS}: index %iters = constant ${ITERS}: index
/// Run and dump performance for matmul. /// Run and dump performance for matmul.
/// Preheating run: /// Preheating run:
scf.for %arg0 = %c0 to %iters step %c1 { scf.for %arg0 = %c0 to %iters step %c1 {
%z = constant 0.0 : !elem_type_c %z = constant 0.0 : !elem_type_c
linalg.fill(%C, %z) : !row_major_C, !elem_type_c linalg.fill(%z, %C) : !elem_type_c, !row_major_C
call @matmul(%A, %B, %C) : (!row_major_A, !row_major_B, !row_major_C) -> () call @matmul(%A, %B, %C) : (!row_major_A, !row_major_B, !row_major_C) -> ()
} }
%t_start_matmul = call @rtclock() : () -> f64 %t_start_matmul = call @rtclock() : () -> f64
scf.for %arg0 = %c0 to %iters step %c1 { scf.for %arg0 = %c0 to %iters step %c1 {
// linalg.matmul writes %C in place, need to reset it to zero every time. // linalg.matmul writes %C in place, need to reset it to zero every time.
// This is accounts for about 10-15% perf hit on small sizes. // This is accounts for about 10-15% perf hit on small sizes.
// Once linalg on tensors is ready, fusing fill at the register level will // Once linalg on tensors is ready, fusing fill at the register level will
// be easy. // be easy.
%z = constant 0.0 : !elem_type_c %z = constant 0.0 : !elem_type_c
linalg.fill(%C, %z) : !row_major_C, !elem_type_c linalg.fill(%z, %C) : !elem_type_c, !row_major_C
call @matmul(%A, %B, %C) : (!row_major_A, !row_major_B, !row_major_C) -> () call @matmul(%A, %B, %C) : (!row_major_A, !row_major_B, !row_major_C) -> ()
} }
%t_end_matmul = call @rtclock() : () -> f64 %t_end_matmul = call @rtclock() : () -> f64
%tmatmul = subf %t_end_matmul, %t_start_matmul: f64 %tmatmul = subf %t_end_matmul, %t_start_matmul: f64
call @print_perf(%iters, %tmatmul) : (index, f64) -> () call @print_perf(%iters, %tmatmul) : (index, f64) -> ()
// CHECK: {{^0$}} // CHECK: {{^0$}}
%C_ref = memref.alloc() : !row_major_C %C_ref = memref.alloc() : !row_major_C
linalg.fill(%C_ref, %v0) : !row_major_C, !elem_type_c linalg.fill(%v0, %C_ref) : !elem_type_c, !row_major_C
linalg.matmul ins(%A, %B : !row_major_A, !row_major_B) linalg.matmul ins(%A, %B : !row_major_A, !row_major_B)
outs(%C_ref: !row_major_C) outs(%C_ref: !row_major_C)
%act = memref.cast %C : !row_major_C to memref<*xf32> %act = memref.cast %C : !row_major_C to memref<*xf32>
%exp = memref.cast %C_ref : !row_major_C to memref<*xf32> %exp = memref.cast %C_ref : !row_major_C to memref<*xf32>
%errors = call @verifyMemRefF32(%act, %exp) : (memref<*xf32>, memref<*xf32>) -> i64 %errors = call @verifyMemRefF32(%act, %exp) : (memref<*xf32>, memref<*xf32>) -> i64
vector.print %errors : i64 vector.print %errors : i64
memref.dealloc %C_ref : !row_major_C memref.dealloc %C_ref : !row_major_C
Show All 12 Lines

mlir/test/Integration/Dialect/Linalg/CPU/benchmark_matmul_column_major_as_row_major.mlir

Show First 20 Lines • Show All 63 Lines • ▼ Show 20 Lines
func @main() { func @main() {
%f0 = constant 0.0 : !elem_type_c %f0 = constant 0.0 : !elem_type_c
%f1 = constant 1.0 : !elem_type_a %f1 = constant 1.0 : !elem_type_a
%cA = memref.alloc() : !column_major_A %cA = memref.alloc() : !column_major_A
%cB = memref.alloc() : !column_major_B %cB = memref.alloc() : !column_major_B
%cC = memref.alloc() : !column_major_C %cC = memref.alloc() : !column_major_C
linalg.fill(%cA, %f1) : !column_major_A, !elem_type_a linalg.fill(%f1, %cA) : !elem_type_a, !column_major_A
linalg.fill(%cB, %f1) : !column_major_B, !elem_type_b linalg.fill(%f1, %cB) : !elem_type_b, !column_major_B
linalg.fill(%cC, %f0) : !column_major_C, !elem_type_c linalg.fill(%f0, %cC) : !elem_type_c, !column_major_C
%c0 = constant 0: index %c0 = constant 0: index
%c1 = constant 1: index %c1 = constant 1: index
%iters = constant ${ITERS}: index %iters = constant ${ITERS}: index
/// Run and dump performance for matmul_column_major as a row-major /// Run and dump performance for matmul_column_major as a row-major
%A = memref.alloc() : !row_major_A %A = memref.alloc() : !row_major_A
%B = memref.alloc() : !row_major_B %B = memref.alloc() : !row_major_B
%C = memref.alloc() : !row_major_C %C = memref.alloc() : !row_major_C
%t_start_matmul_column_major_as_row_major = call @rtclock() : () -> f64 %t_start_matmul_column_major_as_row_major = call @rtclock() : () -> f64
scf.for %arg0 = %c0 to %iters step %c1 { scf.for %arg0 = %c0 to %iters step %c1 {
// linalg.matmul writes %C in place, need to reset it to zero every time. // linalg.matmul writes %C in place, need to reset it to zero every time.
// This is accounts for about 10-15% perf hit on small sizes. // This is accounts for about 10-15% perf hit on small sizes.
// Once linalg on tensors is ready, fusing fill at the register level will // Once linalg on tensors is ready, fusing fill at the register level will
// be easy. // be easy.
linalg.fill(%C, %f0) : !row_major_C, !elem_type_c linalg.fill(%f0, %C) : !elem_type_c, !row_major_C
call @matmul_column_major_as_row_major(%cA, %cB, %cC, %A, %B, %C) : call @matmul_column_major_as_row_major(%cA, %cB, %cC, %A, %B, %C) :
(!column_major_A, !column_major_B, !column_major_C, (!column_major_A, !column_major_B, !column_major_C,
!row_major_A, !row_major_B, !row_major_C) -> () !row_major_A, !row_major_B, !row_major_C) -> ()
} }
%t_end_matmul_column_major_as_row_major = call @rtclock() : () -> f64 %t_end_matmul_column_major_as_row_major = call @rtclock() : () -> f64
%tmatmul_column_major_as_row_major = subf %t_end_matmul_column_major_as_row_major, %t_start_matmul_column_major_as_row_major: f64 %tmatmul_column_major_as_row_major = subf %t_end_matmul_column_major_as_row_major, %t_start_matmul_column_major_as_row_major: f64
call @print_perf(%iters, %tmatmul_column_major_as_row_major) : (index, f64) -> () call @print_perf(%iters, %tmatmul_column_major_as_row_major) : (index, f64) -> ()
// CHECK: {{^0$}} // CHECK: {{^0$}}
%cC_ref = memref.alloc() : !column_major_C %cC_ref = memref.alloc() : !column_major_C
linalg.fill(%cC_ref, %f0) : !column_major_C, !elem_type_c linalg.fill(%f0, %cC_ref) : !elem_type_c, !column_major_C
linalg.matmul_column_major ins(%cA, %cB : !column_major_A, !column_major_B) linalg.matmul_column_major ins(%cA, %cB : !column_major_A, !column_major_B)
outs(%cC_ref: !column_major_C) outs(%cC_ref: !column_major_C)
%act1 = memref.cast %cC : !column_major_C to memref<*xf32> %act1 = memref.cast %cC : !column_major_C to memref<*xf32>
%exp1 = memref.cast %cC_ref : !column_major_C to memref<*xf32> %exp1 = memref.cast %cC_ref : !column_major_C to memref<*xf32>
%errors1 = call @verifyMemRefF32(%act1, %exp1) : (memref<*xf32>, memref<*xf32>) -> i64 %errors1 = call @verifyMemRefF32(%act1, %exp1) : (memref<*xf32>, memref<*xf32>) -> i64
vector.print %errors1 : i64 vector.print %errors1 : i64
memref.dealloc %cC_ref : !column_major_C memref.dealloc %cC_ref : !column_major_C
// CHECK: {{^0$}} // CHECK: {{^0$}}
%C_ref = memref.alloc() : !row_major_C %C_ref = memref.alloc() : !row_major_C
linalg.fill(%C_ref, %f0) : !row_major_C, !elem_type_c linalg.fill(%f0, %C_ref) : !elem_type_c, !row_major_C
linalg.matmul ins(%A, %B : !row_major_A, !row_major_B) linalg.matmul ins(%A, %B : !row_major_A, !row_major_B)
outs(%C_ref: !row_major_C) outs(%C_ref: !row_major_C)
%act2 = memref.cast %C : !row_major_C to memref<*xf32> %act2 = memref.cast %C : !row_major_C to memref<*xf32>
%exp2 = memref.cast %C_ref : !row_major_C to memref<*xf32> %exp2 = memref.cast %C_ref : !row_major_C to memref<*xf32>
%errors2 = call @verifyMemRefF32(%act2, %exp2) : (memref<*xf32>, memref<*xf32>) -> i64 %errors2 = call @verifyMemRefF32(%act2, %exp2) : (memref<*xf32>, memref<*xf32>) -> i64
vector.print %errors2 : i64 vector.print %errors2 : i64
memref.dealloc %C_ref : !row_major_C memref.dealloc %C_ref : !row_major_C
Show All 16 Lines

mlir/test/Integration/Dialect/Linalg/CPU/benchmark_matmul_i8_i8_i32.mlir

Show First 20 Lines • Show All 53 Lines • ▼ Show 20 Lines
func @main() { func @main() {
%v0 = constant 0 : !elem_type_c %v0 = constant 0 : !elem_type_c
%v1 = constant 1 : !elem_type_a %v1 = constant 1 : !elem_type_a
%A = memref.alloc() : !row_major_A %A = memref.alloc() : !row_major_A
%B = memref.alloc() : !row_major_B %B = memref.alloc() : !row_major_B
%C = memref.alloc() : !row_major_C %C = memref.alloc() : !row_major_C
linalg.fill(%A, %v1) : !row_major_A, !elem_type_a linalg.fill(%v1, %A) : !elem_type_a, !row_major_A
linalg.fill(%B, %v1) : !row_major_B, !elem_type_b linalg.fill(%v1, %B) : !elem_type_b, !row_major_B
linalg.fill(%C, %v0) : !row_major_C, !elem_type_c linalg.fill(%v0, %C) : !elem_type_c, !row_major_C
%c0 = constant 0: index %c0 = constant 0: index
%c1 = constant 1: index %c1 = constant 1: index
%iters = constant 100: index %iters = constant 100: index
/// Run and dump performance for matmul. /// Run and dump performance for matmul.
/// Preheating run: /// Preheating run:
scf.for %arg0 = %c0 to %iters step %c1 { scf.for %arg0 = %c0 to %iters step %c1 {
linalg.fill(%C, %v0) : !row_major_C, !elem_type_c linalg.fill(%v0, %C) : !elem_type_c, !row_major_C
call @matmul(%A, %B, %C) : (!row_major_A, !row_major_B, !row_major_C) -> () call @matmul(%A, %B, %C) : (!row_major_A, !row_major_B, !row_major_C) -> ()
} }
%t_start_matmul = call @rtclock() : () -> f64 %t_start_matmul = call @rtclock() : () -> f64
scf.for %arg0 = %c0 to %iters step %c1 { scf.for %arg0 = %c0 to %iters step %c1 {
// linalg.matmul writes %C in place, need to reset it to zero every time. // linalg.matmul writes %C in place, need to reset it to zero every time.
// This is accounts for about 10-15% perf hit on small sizes. // This is accounts for about 10-15% perf hit on small sizes.
// Once linalg on tensors is ready, fusing fill at the register level will // Once linalg on tensors is ready, fusing fill at the register level will
// be easy. // be easy.
linalg.fill(%C, %v0) : !row_major_C, !elem_type_c linalg.fill(%v0, %C) : !elem_type_c, !row_major_C
call @matmul(%A, %B, %C) : (!row_major_A, !row_major_B, !row_major_C) -> () call @matmul(%A, %B, %C) : (!row_major_A, !row_major_B, !row_major_C) -> ()
} }
%t_end_matmul = call @rtclock() : () -> f64 %t_end_matmul = call @rtclock() : () -> f64
%tmatmul = subf %t_end_matmul, %t_start_matmul: f64 %tmatmul = subf %t_end_matmul, %t_start_matmul: f64
call @print_perf(%iters, %tmatmul) : (index, f64) -> () call @print_perf(%iters, %tmatmul) : (index, f64) -> ()
// CHECK: {{^0$}} // CHECK: {{^0$}}
%C_ref = memref.alloc() : !row_major_C %C_ref = memref.alloc() : !row_major_C
linalg.fill(%C_ref, %v0) : !row_major_C, !elem_type_c linalg.fill(%v0, %C_ref) : !elem_type_c, !row_major_C
linalg.matmul_i8_i8_i32 ins(%A, %B : !row_major_A, !row_major_B) linalg.matmul_i8_i8_i32 ins(%A, %B : !row_major_A, !row_major_B)
outs(%C_ref: !row_major_C) outs(%C_ref: !row_major_C)
%res = memref.cast %C : !row_major_C to memref<*xi32> %res = memref.cast %C : !row_major_C to memref<*xi32>
%exp = memref.cast %C_ref : !row_major_C to memref<*xi32> %exp = memref.cast %C_ref : !row_major_C to memref<*xi32>
%errors = call @verifyMemRefI32(%res, %exp) : (memref<*xi32>, memref<*xi32>) -> i64 %errors = call @verifyMemRefI32(%res, %exp) : (memref<*xi32>, memref<*xi32>) -> i64
vector.print %errors : i64 vector.print %errors : i64
memref.dealloc %C_ref : !row_major_C memref.dealloc %C_ref : !row_major_C
Show All 12 Lines

mlir/test/Integration/Dialect/Linalg/CPU/matmul-vs-matvec.mlir

// RUN: mlir-opt %s -convert-linalg-to-loops -convert-scf-to-std -convert-linalg-to-llvm -convert-std-to-llvm | \ // RUN: mlir-opt %s -convert-linalg-to-loops -convert-scf-to-std -convert-linalg-to-llvm -convert-std-to-llvm | \
// RUN: mlir-cpu-runner -O3 -e main -entry-point-result=void \ // RUN: mlir-cpu-runner -O3 -e main -entry-point-result=void \
// RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \ // RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \
// RUN: | FileCheck %s // RUN: | FileCheck %s
func private @print_memref_f32(memref<*xf32>) func private @print_memref_f32(memref<*xf32>)
func @matmul(%A: memref<?x?xf32>, %B: memref<?x?xf32>) -> (memref<?x?xf32>) { func @matmul(%A: memref<?x?xf32>, %B: memref<?x?xf32>) -> (memref<?x?xf32>) {
%c0 = constant 0 : index %c0 = constant 0 : index
%c1 = constant 1 : index %c1 = constant 1 : index
%f0 = constant 0.0 : f32 %f0 = constant 0.0 : f32
%x = memref.dim %A, %c0 : memref<?x?xf32> %x = memref.dim %A, %c0 : memref<?x?xf32>
%y = memref.dim %B, %c1 : memref<?x?xf32> %y = memref.dim %B, %c1 : memref<?x?xf32>
%C = memref.alloc(%x, %y) : memref<?x?xf32> %C = memref.alloc(%x, %y) : memref<?x?xf32>
linalg.fill(%C, %f0) : memref<?x?xf32>, f32 linalg.fill(%f0, %C) : f32, memref<?x?xf32>
linalg.matmul ins(%A, %B: memref<?x?xf32>, memref<?x?xf32>) linalg.matmul ins(%A, %B: memref<?x?xf32>, memref<?x?xf32>)
outs(%C: memref<?x?xf32>) outs(%C: memref<?x?xf32>)
return %C : memref<?x?xf32> return %C : memref<?x?xf32>
} }
func @matvec(%A: memref<?x?xf32>, %B: memref<?x?xf32>) -> (memref<?x?xf32>) { func @matvec(%A: memref<?x?xf32>, %B: memref<?x?xf32>) -> (memref<?x?xf32>) {
%c0 = constant 0 : index %c0 = constant 0 : index
%c1 = constant 1 : index %c1 = constant 1 : index
%f0 = constant 0.0 : f32 %f0 = constant 0.0 : f32
%m = memref.dim %A, %c0 : memref<?x?xf32> %m = memref.dim %A, %c0 : memref<?x?xf32>
%x = memref.dim %A, %c1 : memref<?x?xf32> %x = memref.dim %A, %c1 : memref<?x?xf32>
%n = memref.dim %B, %c1 : memref<?x?xf32> %n = memref.dim %B, %c1 : memref<?x?xf32>
%C = memref.alloc(%m, %n) : memref<?x?xf32> %C = memref.alloc(%m, %n) : memref<?x?xf32>
linalg.fill(%C, %f0) : memref<?x?xf32>, f32 linalg.fill(%f0, %C) : f32, memref<?x?xf32>
scf.for %i = %c0 to %n step %c1 { scf.for %i = %c0 to %n step %c1 {
%b = memref.subview %B[0, %i][%x, 1][1, 1] : memref<?x?xf32> to memref<?xf32, offset: ?, strides: [?]> %b = memref.subview %B[0, %i][%x, 1][1, 1] : memref<?x?xf32> to memref<?xf32, offset: ?, strides: [?]>
%c = memref.subview %C[0, %i][%m, 1][1, 1] : memref<?x?xf32> to memref<?xf32, offset: ?, strides: [?]> %c = memref.subview %C[0, %i][%m, 1][1, 1] : memref<?x?xf32> to memref<?xf32, offset: ?, strides: [?]>
linalg.matvec ins(%A, %b: memref<?x?xf32>, memref<?xf32, offset: ?, strides: [?]>) linalg.matvec ins(%A, %b: memref<?x?xf32>, memref<?xf32, offset: ?, strides: [?]>)
outs(%c: memref<?xf32, offset: ?, strides: [?]>) outs(%c: memref<?xf32, offset: ?, strides: [?]>)
} }
return %C : memref<?x?xf32> return %C : memref<?x?xf32>
} }
func @main() { func @main() {
%c0 = constant 0 : index %c0 = constant 0 : index
%c1 = constant 1 : index %c1 = constant 1 : index
%m = constant 5 : index %m = constant 5 : index
%x = constant 3 : index %x = constant 3 : index
%n = constant 2 : index %n = constant 2 : index
%val1 = constant 13.0 : f32 %val1 = constant 13.0 : f32
%val2 = constant 17.0 : f32 %val2 = constant 17.0 : f32
%A = memref.alloc(%m, %x) : memref<?x?xf32> %A = memref.alloc(%m, %x) : memref<?x?xf32>
%B = memref.alloc(%x, %n) : memref<?x?xf32> %B = memref.alloc(%x, %n) : memref<?x?xf32>
linalg.fill(%A, %val1) : memref<?x?xf32>, f32 linalg.fill(%val1, %A) : f32, memref<?x?xf32>
linalg.fill(%B, %val2) : memref<?x?xf32>, f32 linalg.fill(%val2, %B) : f32, memref<?x?xf32>
memref.store %val1, %B[%c0, %c0] : memref<?x?xf32> memref.store %val1, %B[%c0, %c0] : memref<?x?xf32>
%C1 = call @matmul(%A, %B) : (memref<?x?xf32>, memref<?x?xf32>) -> memref<?x?xf32> %C1 = call @matmul(%A, %B) : (memref<?x?xf32>, memref<?x?xf32>) -> memref<?x?xf32>
%C2 = call @matvec(%A, %B) : (memref<?x?xf32>, memref<?x?xf32>) -> memref<?x?xf32> %C2 = call @matvec(%A, %B) : (memref<?x?xf32>, memref<?x?xf32>) -> memref<?x?xf32>
scf.for %i = %c0 to %m step %c1 { scf.for %i = %c0 to %m step %c1 {
scf.for %j = %c0 to %n step %c1 { scf.for %j = %c0 to %n step %c1 {
%e1 = memref.load %C1[%i, %j] : memref<?x?xf32> %e1 = memref.load %C1[%i, %j] : memref<?x?xf32>
%e2 = memref.load %C2[%i, %j] : memref<?x?xf32> %e2 = memref.load %C2[%i, %j] : memref<?x?xf32>
%c = cmpf oeq, %e1, %e2 : f32 %c = cmpf oeq, %e1, %e2 : f32
Show All 16 Lines

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-1d-call.mlir

Show All 19 Lines
// RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \ // RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \
// RUN: | FileCheck %s // RUN: | FileCheck %s
func private @print_memref_f32(memref<*xf32>) func private @print_memref_f32(memref<*xf32>)
// Creates and returns a 1-D buffer of size %s1 filled with the value %f // Creates and returns a 1-D buffer of size %s1 filled with the value %f
func @alloc_1d_filled_f32(%s1 : index, %f : f32) -> memref<?xf32> { func @alloc_1d_filled_f32(%s1 : index, %f : f32) -> memref<?xf32> {
%buf = memref.alloc(%s1) : memref<?xf32> %buf = memref.alloc(%s1) : memref<?xf32>
linalg.fill(%buf, %f) : memref<?xf32>, f32 linalg.fill(%f, %buf) : f32, memref<?xf32>
return %buf : memref<?xf32> return %buf : memref<?xf32>
} }
func @conv_1d(%arg0: memref<?xf32>, %arg1: memref<?xf32>, %arg2: memref<?xf32>) { func @conv_1d(%arg0: memref<?xf32>, %arg1: memref<?xf32>, %arg2: memref<?xf32>) {
linalg.conv_1d ins (%arg0, %arg1: memref<?xf32>, memref<?xf32>) linalg.conv_1d ins (%arg0, %arg1: memref<?xf32>, memref<?xf32>)
outs (%arg2: memref<?xf32>) outs (%arg2: memref<?xf32>)
return return
} }
Show All 26 Lines

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-1d-input-ncw-filter-wcf-call.mlir

Show All 19 Lines
// RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \ // RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \
// RUN: | FileCheck %s // RUN: | FileCheck %s
func private @print_memref_f32(memref<*xf32>) func private @print_memref_f32(memref<*xf32>)
// Creates and returns 3-D buffer of size (%s1, %s2, %s3) filled with the value %f // Creates and returns 3-D buffer of size (%s1, %s2, %s3) filled with the value %f
func @alloc_3d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %f : f32) -> memref<?x?x?xf32> { func @alloc_3d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %f : f32) -> memref<?x?x?xf32> {
%buf = memref.alloc(%s1, %s2, %s3) : memref<?x?x?xf32> %buf = memref.alloc(%s1, %s2, %s3) : memref<?x?x?xf32>
linalg.fill(%buf, %f) : memref<?x?x?xf32>, f32 linalg.fill(%f, %buf) : f32, memref<?x?x?xf32>
return %buf : memref<?x?x?xf32> return %buf : memref<?x?x?xf32>
} }
func @conv_1d_input_ncw_filter_wcf(%arg0: memref<?x?x?xf32>, %arg1: memref<?x?x?xf32>, %arg2: memref<?x?x?xf32>) { func @conv_1d_input_ncw_filter_wcf(%arg0: memref<?x?x?xf32>, %arg1: memref<?x?x?xf32>, %arg2: memref<?x?x?xf32>) {
linalg.conv_1d_input_ncw_filter_wcf {dilations = dense<1> : tensor<1xi64>, linalg.conv_1d_input_ncw_filter_wcf {dilations = dense<1> : tensor<1xi64>,
strides = dense<1> : tensor<1xi64>} strides = dense<1> : tensor<1xi64>}
ins (%arg0, %arg1: memref<?x?x?xf32>, memref<?x?x?xf32>) ins (%arg0, %arg1: memref<?x?x?xf32>, memref<?x?x?xf32>)
outs (%arg2: memref<?x?x?xf32>) outs (%arg2: memref<?x?x?xf32>)
Show All 34 Lines

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-1d-input-nwc-filter-wcf-call.mlir

Show All 19 Lines
// RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \ // RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \
// RUN: | FileCheck %s // RUN: | FileCheck %s
func private @print_memref_f32(memref<*xf32>) func private @print_memref_f32(memref<*xf32>)
// Creates and returns 3-D buffer of size (%s1, %s2, %s3) filled with the value %f // Creates and returns 3-D buffer of size (%s1, %s2, %s3) filled with the value %f
func @alloc_3d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %f : f32) -> memref<?x?x?xf32> { func @alloc_3d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %f : f32) -> memref<?x?x?xf32> {
%buf = memref.alloc(%s1, %s2, %s3) : memref<?x?x?xf32> %buf = memref.alloc(%s1, %s2, %s3) : memref<?x?x?xf32>
linalg.fill(%buf, %f) : memref<?x?x?xf32>, f32 linalg.fill(%f, %buf) : f32, memref<?x?x?xf32>
return %buf : memref<?x?x?xf32> return %buf : memref<?x?x?xf32>
} }
func @conv_1d_input_nwc_filter_wcf(%arg0: memref<?x?x?xf32>, %arg1: memref<?x?x?xf32>, %arg2: memref<?x?x?xf32>) { func @conv_1d_input_nwc_filter_wcf(%arg0: memref<?x?x?xf32>, %arg1: memref<?x?x?xf32>, %arg2: memref<?x?x?xf32>) {
linalg.conv_1d_input_nwc_filter_wcf {dilations = dense<1> : tensor<1xi64>, linalg.conv_1d_input_nwc_filter_wcf {dilations = dense<1> : tensor<1xi64>,
strides = dense<1> : tensor<1xi64>} strides = dense<1> : tensor<1xi64>}
ins (%arg0, %arg1: memref<?x?x?xf32>, memref<?x?x?xf32>) ins (%arg0, %arg1: memref<?x?x?xf32>, memref<?x?x?xf32>)
outs (%arg2: memref<?x?x?xf32>) outs (%arg2: memref<?x?x?xf32>)
▲ Show 20 Lines • Show All 45 Lines • Show Last 20 Lines

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-1d-ncw-call.mlir

Show All 19 Lines
// RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \ // RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \
// RUN: | FileCheck %s // RUN: | FileCheck %s
func private @print_memref_f32(memref<*xf32>) func private @print_memref_f32(memref<*xf32>)
// Creates and returns 3-D buffer of size (%s1, %s2, %s3) filled with the value %f // Creates and returns 3-D buffer of size (%s1, %s2, %s3) filled with the value %f
func @alloc_3d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %f : f32) -> memref<?x?x?xf32> { func @alloc_3d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %f : f32) -> memref<?x?x?xf32> {
%buf = memref.alloc(%s1, %s2, %s3) : memref<?x?x?xf32> %buf = memref.alloc(%s1, %s2, %s3) : memref<?x?x?xf32>
linalg.fill(%buf, %f) : memref<?x?x?xf32>, f32 linalg.fill(%f, %buf) : f32, memref<?x?x?xf32>
return %buf : memref<?x?x?xf32> return %buf : memref<?x?x?xf32>
} }
func @conv_1d_ncw(%arg0: memref<?x?x?xf32>, %arg1: memref<?x?x?xf32>, %arg2: memref<?x?x?xf32>) { func @conv_1d_ncw(%arg0: memref<?x?x?xf32>, %arg1: memref<?x?x?xf32>, %arg2: memref<?x?x?xf32>) {
linalg.conv_1d_ncw ins (%arg0, %arg1: memref<?x?x?xf32>, memref<?x?x?xf32>) linalg.conv_1d_ncw ins (%arg0, %arg1: memref<?x?x?xf32>, memref<?x?x?xf32>)
outs (%arg2: memref<?x?x?xf32>) outs (%arg2: memref<?x?x?xf32>)
return return
} }
Show All 32 Lines

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-1d-nwc-call.mlir

Show All 19 Lines
// RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \ // RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \
// RUN: | FileCheck %s // RUN: | FileCheck %s
func private @print_memref_f32(memref<*xf32>) func private @print_memref_f32(memref<*xf32>)
// Creates and returns 3-D buffer of size (%s1, %s2, %s3) filled with the value %f // Creates and returns 3-D buffer of size (%s1, %s2, %s3) filled with the value %f
func @alloc_3d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %f : f32) -> memref<?x?x?xf32> { func @alloc_3d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %f : f32) -> memref<?x?x?xf32> {
%buf = memref.alloc(%s1, %s2, %s3) : memref<?x?x?xf32> %buf = memref.alloc(%s1, %s2, %s3) : memref<?x?x?xf32>
linalg.fill(%buf, %f) : memref<?x?x?xf32>, f32 linalg.fill(%f, %buf) : f32, memref<?x?x?xf32>
return %buf : memref<?x?x?xf32> return %buf : memref<?x?x?xf32>
} }
func @conv_1d_nwc(%arg0: memref<?x?x?xf32>, %arg1: memref<?x?x?xf32>, %arg2: memref<?x?x?xf32>) { func @conv_1d_nwc(%arg0: memref<?x?x?xf32>, %arg1: memref<?x?x?xf32>, %arg2: memref<?x?x?xf32>) {
linalg.conv_1d_nwc ins (%arg0, %arg1: memref<?x?x?xf32>, memref<?x?x?xf32>) linalg.conv_1d_nwc ins (%arg0, %arg1: memref<?x?x?xf32>, memref<?x?x?xf32>)
outs (%arg2: memref<?x?x?xf32>) outs (%arg2: memref<?x?x?xf32>)
return return
} }
▲ Show 20 Lines • Show All 43 Lines • Show Last 20 Lines

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-2d-call.mlir

Show All 19 Lines
// RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \ // RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \
// RUN: | FileCheck %s // RUN: | FileCheck %s
func private @print_memref_f32(memref<*xf32>) func private @print_memref_f32(memref<*xf32>)
// Creates and returns a 2-D buffer of size (%s1, %s2) filled with the value %f // Creates and returns a 2-D buffer of size (%s1, %s2) filled with the value %f
func @alloc_2d_filled_f32(%s1 : index, %s2 : index, %f : f32) -> memref<?x?xf32> { func @alloc_2d_filled_f32(%s1 : index, %s2 : index, %f : f32) -> memref<?x?xf32> {
%buf = memref.alloc(%s1, %s2) : memref<?x?xf32> %buf = memref.alloc(%s1, %s2) : memref<?x?xf32>
linalg.fill(%buf, %f) : memref<?x?xf32>, f32 linalg.fill(%f, %buf) : f32, memref<?x?xf32>
return %buf : memref<?x?xf32> return %buf : memref<?x?xf32>
} }
func @conv_2d(%arg0: memref<?x?xf32>, %arg1: memref<?x?xf32>, %arg2: memref<?x?xf32>) { func @conv_2d(%arg0: memref<?x?xf32>, %arg1: memref<?x?xf32>, %arg2: memref<?x?xf32>) {
linalg.conv_2d ins (%arg0, %arg1: memref<?x?xf32>, memref<?x?xf32>) linalg.conv_2d ins (%arg0, %arg1: memref<?x?xf32>, memref<?x?xf32>)
outs (%arg2: memref<?x?xf32>) outs (%arg2: memref<?x?xf32>)
return return
} }
Show All 31 Lines

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-2d-input-nchw-filter-hwcf-call.mlir

Show All 19 Lines
// RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \ // RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \
// RUN: | FileCheck %s // RUN: | FileCheck %s
func private @print_memref_f32(memref<*xf32>) func private @print_memref_f32(memref<*xf32>)
// Creates and returns 4-D buffer of size (%s1, %s2, %s3, %s4) filled with the value %f // Creates and returns 4-D buffer of size (%s1, %s2, %s3, %s4) filled with the value %f
func @alloc_4d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %s4 : index, %f : f32) -> memref<?x?x?x?xf32> { func @alloc_4d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %s4 : index, %f : f32) -> memref<?x?x?x?xf32> {
%buf = memref.alloc(%s1, %s2, %s3, %s4) : memref<?x?x?x?xf32> %buf = memref.alloc(%s1, %s2, %s3, %s4) : memref<?x?x?x?xf32>
linalg.fill(%buf, %f) : memref<?x?x?x?xf32>, f32 linalg.fill(%f, %buf) : f32, memref<?x?x?x?xf32>
return %buf : memref<?x?x?x?xf32> return %buf : memref<?x?x?x?xf32>
} }
func @conv_2d_input_nchw_filter_hwcf(%arg0: memref<?x?x?x?xf32>, %arg1: memref<?x?x?x?xf32>, %arg2: memref<?x?x?x?xf32>) { func @conv_2d_input_nchw_filter_hwcf(%arg0: memref<?x?x?x?xf32>, %arg1: memref<?x?x?x?xf32>, %arg2: memref<?x?x?x?xf32>) {
linalg.conv_2d_input_nchw_filter_hwcf {dilations = dense<1> : tensor<2xi64>, linalg.conv_2d_input_nchw_filter_hwcf {dilations = dense<1> : tensor<2xi64>,
strides = dense<1> : tensor<2xi64>} strides = dense<1> : tensor<2xi64>}
ins (%arg0, %arg1: memref<?x?x?x?xf32>, memref<?x?x?x?xf32>) ins (%arg0, %arg1: memref<?x?x?x?xf32>, memref<?x?x?x?xf32>)
outs (%arg2: memref<?x?x?x?xf32>) outs (%arg2: memref<?x?x?x?xf32>)
▲ Show 20 Lines • Show All 47 Lines • Show Last 20 Lines

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-2d-input-nhwc-filter-hwcf-call.mlir

Show All 19 Lines
// RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \ // RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \
// RUN: | FileCheck %s // RUN: | FileCheck %s
func private @print_memref_f32(memref<*xf32>) func private @print_memref_f32(memref<*xf32>)
// Creates and returns 4-D buffer of size (%s1, %s2, %s3, %s4) filled with the value %f // Creates and returns 4-D buffer of size (%s1, %s2, %s3, %s4) filled with the value %f
func @alloc_4d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %s4 : index, %f : f32) -> memref<?x?x?x?xf32> { func @alloc_4d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %s4 : index, %f : f32) -> memref<?x?x?x?xf32> {
%buf = memref.alloc(%s1, %s2, %s3, %s4) : memref<?x?x?x?xf32> %buf = memref.alloc(%s1, %s2, %s3, %s4) : memref<?x?x?x?xf32>
linalg.fill(%buf, %f) : memref<?x?x?x?xf32>, f32 linalg.fill(%f, %buf) : f32, memref<?x?x?x?xf32>
return %buf : memref<?x?x?x?xf32> return %buf : memref<?x?x?x?xf32>
} }
func @conv_2d_input_nhwc_filter_hwcf(%arg0: memref<?x?x?x?xf32>, %arg1: memref<?x?x?x?xf32>, %arg2: memref<?x?x?x?xf32>) { func @conv_2d_input_nhwc_filter_hwcf(%arg0: memref<?x?x?x?xf32>, %arg1: memref<?x?x?x?xf32>, %arg2: memref<?x?x?x?xf32>) {
linalg.conv_2d_input_nhwc_filter_hwcf {dilations = dense<1> : tensor<2xi64>, linalg.conv_2d_input_nhwc_filter_hwcf {dilations = dense<1> : tensor<2xi64>,
strides = dense<1> : tensor<2xi64>} strides = dense<1> : tensor<2xi64>}
ins (%arg0, %arg1: memref<?x?x?x?xf32>, memref<?x?x?x?xf32>) ins (%arg0, %arg1: memref<?x?x?x?xf32>, memref<?x?x?x?xf32>)
outs (%arg2: memref<?x?x?x?xf32>) outs (%arg2: memref<?x?x?x?xf32>)
▲ Show 20 Lines • Show All 93 Lines • Show Last 20 Lines

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-2d-nchw-call.mlir

Show All 19 Lines
// RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \ // RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \
// RUN: | FileCheck %s // RUN: | FileCheck %s
func private @print_memref_f32(memref<*xf32>) func private @print_memref_f32(memref<*xf32>)
// Creates and returns 4-D buffer of size (%s1, %s2, %s3, %s4) filled with the value %f // Creates and returns 4-D buffer of size (%s1, %s2, %s3, %s4) filled with the value %f
func @alloc_4d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %s4 : index, %f : f32) -> memref<?x?x?x?xf32> { func @alloc_4d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %s4 : index, %f : f32) -> memref<?x?x?x?xf32> {
%buf = memref.alloc(%s1, %s2, %s3, %s4) : memref<?x?x?x?xf32> %buf = memref.alloc(%s1, %s2, %s3, %s4) : memref<?x?x?x?xf32>
linalg.fill(%buf, %f) : memref<?x?x?x?xf32>, f32 linalg.fill(%f, %buf) : f32, memref<?x?x?x?xf32>
return %buf : memref<?x?x?x?xf32> return %buf : memref<?x?x?x?xf32>
} }
func @conv_2d_nchw(%arg0: memref<?x?x?x?xf32>, %arg1: memref<?x?x?x?xf32>, %arg2: memref<?x?x?x?xf32>) { func @conv_2d_nchw(%arg0: memref<?x?x?x?xf32>, %arg1: memref<?x?x?x?xf32>, %arg2: memref<?x?x?x?xf32>) {
linalg.conv_2d_nchw ins (%arg0, %arg1: memref<?x?x?x?xf32>, memref<?x?x?x?xf32>) linalg.conv_2d_nchw ins (%arg0, %arg1: memref<?x?x?x?xf32>, memref<?x?x?x?xf32>)
outs (%arg2: memref<?x?x?x?xf32>) outs (%arg2: memref<?x?x?x?xf32>)
return return
} }
▲ Show 20 Lines • Show All 45 Lines • Show Last 20 Lines

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-2d-nhwc-call.mlir

Show All 19 Lines
// RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \ // RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \
// RUN: | FileCheck %s // RUN: | FileCheck %s
func private @print_memref_f32(memref<*xf32>) func private @print_memref_f32(memref<*xf32>)
// Creates and returns 4-D buffer of size (%s1, %s2, %s3, %s4) filled with the value %f // Creates and returns 4-D buffer of size (%s1, %s2, %s3, %s4) filled with the value %f
func @alloc_4d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %s4 : index, %f : f32) -> memref<?x?x?x?xf32> { func @alloc_4d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %s4 : index, %f : f32) -> memref<?x?x?x?xf32> {
%buf = memref.alloc(%s1, %s2, %s3, %s4) : memref<?x?x?x?xf32> %buf = memref.alloc(%s1, %s2, %s3, %s4) : memref<?x?x?x?xf32>
linalg.fill(%buf, %f) : memref<?x?x?x?xf32>, f32 linalg.fill(%f, %buf) : f32, memref<?x?x?x?xf32>
return %buf : memref<?x?x?x?xf32> return %buf : memref<?x?x?x?xf32>
} }
func @conv_2d_nhwc(%arg0: memref<?x?x?x?xf32>, %arg1: memref<?x?x?x?xf32>, %arg2: memref<?x?x?x?xf32>) { func @conv_2d_nhwc(%arg0: memref<?x?x?x?xf32>, %arg1: memref<?x?x?x?xf32>, %arg2: memref<?x?x?x?xf32>) {
linalg.conv_2d_nhwc ins (%arg0, %arg1: memref<?x?x?x?xf32>, memref<?x?x?x?xf32>) linalg.conv_2d_nhwc ins (%arg0, %arg1: memref<?x?x?x?xf32>, memref<?x?x?x?xf32>)
outs (%arg2: memref<?x?x?x?xf32>) outs (%arg2: memref<?x?x?x?xf32>)
return return
} }
▲ Show 20 Lines • Show All 91 Lines • Show Last 20 Lines

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-3d-call.mlir

Show All 19 Lines
// RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \ // RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \
// RUN: | FileCheck %s // RUN: | FileCheck %s
func private @print_memref_f32(memref<*xf32>) func private @print_memref_f32(memref<*xf32>)
// Creates and returns 3-D buffer of size (%s1, %s2, %s3) filled with the value %f // Creates and returns 3-D buffer of size (%s1, %s2, %s3) filled with the value %f
func @alloc_3d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %f : f32) -> memref<?x?x?xf32> { func @alloc_3d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %f : f32) -> memref<?x?x?xf32> {
%buf = memref.alloc(%s1, %s2, %s3) : memref<?x?x?xf32> %buf = memref.alloc(%s1, %s2, %s3) : memref<?x?x?xf32>
linalg.fill(%buf, %f) : memref<?x?x?xf32>, f32 linalg.fill(%f, %buf) : f32, memref<?x?x?xf32>
return %buf : memref<?x?x?xf32> return %buf : memref<?x?x?xf32>
} }
func @conv_3d(%arg0: memref<?x?x?xf32>, %arg1: memref<?x?x?xf32>, %arg2: memref<?x?x?xf32>) { func @conv_3d(%arg0: memref<?x?x?xf32>, %arg1: memref<?x?x?xf32>, %arg2: memref<?x?x?xf32>) {
linalg.conv_3d ins (%arg0, %arg1: memref<?x?x?xf32>, memref<?x?x?xf32>) linalg.conv_3d ins (%arg0, %arg1: memref<?x?x?xf32>, memref<?x?x?xf32>)
outs (%arg2: memref<?x?x?xf32>) outs (%arg2: memref<?x?x?xf32>)
return return
} }
▲ Show 20 Lines • Show All 48 Lines • Show Last 20 Lines

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-3d-input-ncdhw-filter-dhwcf-call.mlir

Show All 19 Lines
// RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \ // RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \
// RUN: | FileCheck %s // RUN: | FileCheck %s
func private @print_memref_f32(memref<*xf32>) func private @print_memref_f32(memref<*xf32>)
// Creates and returns 5-D buffer of size (%s1, %s2, %s3, %s4, %s5) filled with the value %f // Creates and returns 5-D buffer of size (%s1, %s2, %s3, %s4, %s5) filled with the value %f
func @alloc_5d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %s4 : index, %s5 : index, %f : f32) -> memref<?x?x?x?x?xf32> { func @alloc_5d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %s4 : index, %s5 : index, %f : f32) -> memref<?x?x?x?x?xf32> {
%buf = memref.alloc(%s1, %s2, %s3, %s4, %s5) : memref<?x?x?x?x?xf32> %buf = memref.alloc(%s1, %s2, %s3, %s4, %s5) : memref<?x?x?x?x?xf32>
linalg.fill(%buf, %f) : memref<?x?x?x?x?xf32>, f32 linalg.fill(%f, %buf) : f32, memref<?x?x?x?x?xf32>
return %buf : memref<?x?x?x?x?xf32> return %buf : memref<?x?x?x?x?xf32>
} }
func @conv_3d_input_ncdhw_filter_dhwcf(%arg0: memref<?x?x?x?x?xf32>, %arg1: memref<?x?x?x?x?xf32>, %arg2: memref<?x?x?x?x?xf32>) { func @conv_3d_input_ncdhw_filter_dhwcf(%arg0: memref<?x?x?x?x?xf32>, %arg1: memref<?x?x?x?x?xf32>, %arg2: memref<?x?x?x?x?xf32>) {
linalg.conv_3d_input_ncdhw_filter_dhwcf {dilations = dense<1> : tensor<3xi64>, linalg.conv_3d_input_ncdhw_filter_dhwcf {dilations = dense<1> : tensor<3xi64>,
strides = dense<1> : tensor<3xi64>} strides = dense<1> : tensor<3xi64>}
ins (%arg0, %arg1: memref<?x?x?x?x?xf32>, memref<?x?x?x?x?xf32>) ins (%arg0, %arg1: memref<?x?x?x?x?xf32>, memref<?x?x?x?x?xf32>)
outs (%arg2: memref<?x?x?x?x?xf32>) outs (%arg2: memref<?x?x?x?x?xf32>)
▲ Show 20 Lines • Show All 54 Lines • Show Last 20 Lines

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-3d-input-ndhwc-filter-dhwcf-call.mlir

Show All 19 Lines
// RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \ // RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \
// RUN: | FileCheck %s // RUN: | FileCheck %s
func private @print_memref_f32(memref<*xf32>) func private @print_memref_f32(memref<*xf32>)
// Creates and returns 5-D buffer of size (%s1, %s2, %s3, %s4, %s5) filled with the value %f // Creates and returns 5-D buffer of size (%s1, %s2, %s3, %s4, %s5) filled with the value %f
func @alloc_5d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %s4 : index, %s5 : index, %f : f32) -> memref<?x?x?x?x?xf32> { func @alloc_5d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %s4 : index, %s5 : index, %f : f32) -> memref<?x?x?x?x?xf32> {
%buf = memref.alloc(%s1, %s2, %s3, %s4, %s5) : memref<?x?x?x?x?xf32> %buf = memref.alloc(%s1, %s2, %s3, %s4, %s5) : memref<?x?x?x?x?xf32>
linalg.fill(%buf, %f) : memref<?x?x?x?x?xf32>, f32 linalg.fill(%f, %buf) : f32, memref<?x?x?x?x?xf32>
return %buf : memref<?x?x?x?x?xf32> return %buf : memref<?x?x?x?x?xf32>
} }
func @conv_3d_input_ndhwc_filter_dhwcf(%arg0: memref<?x?x?x?x?xf32>, %arg1: memref<?x?x?x?x?xf32>, %arg2: memref<?x?x?x?x?xf32>) { func @conv_3d_input_ndhwc_filter_dhwcf(%arg0: memref<?x?x?x?x?xf32>, %arg1: memref<?x?x?x?x?xf32>, %arg2: memref<?x?x?x?x?xf32>) {
linalg.conv_3d_input_ndhwc_filter_dhwcf {dilations = dense<1> : tensor<3xi64>, linalg.conv_3d_input_ndhwc_filter_dhwcf {dilations = dense<1> : tensor<3xi64>,
strides = dense<1> : tensor<3xi64>} strides = dense<1> : tensor<3xi64>}
ins (%arg0, %arg1: memref<?x?x?x?x?xf32>, memref<?x?x?x?x?xf32>) ins (%arg0, %arg1: memref<?x?x?x?x?xf32>, memref<?x?x?x?x?xf32>)
outs (%arg2: memref<?x?x?x?x?xf32>) outs (%arg2: memref<?x?x?x?x?xf32>)
▲ Show 20 Lines • Show All 156 Lines • Show Last 20 Lines

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-3d-ncdhw-call.mlir

Show All 19 Lines
// RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \ // RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \
// RUN: | FileCheck %s // RUN: | FileCheck %s
func private @print_memref_f32(memref<*xf32>) func private @print_memref_f32(memref<*xf32>)
// Creates and returns 5-D buffer of size (%s1, %s2, %s3, %s4, %s5) filled with the value %f // Creates and returns 5-D buffer of size (%s1, %s2, %s3, %s4, %s5) filled with the value %f
func @alloc_5d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %s4 : index, %s5 : index, %f : f32) -> memref<?x?x?x?x?xf32> { func @alloc_5d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %s4 : index, %s5 : index, %f : f32) -> memref<?x?x?x?x?xf32> {
%buf = memref.alloc(%s1, %s2, %s3, %s4, %s5) : memref<?x?x?x?x?xf32> %buf = memref.alloc(%s1, %s2, %s3, %s4, %s5) : memref<?x?x?x?x?xf32>
linalg.fill(%buf, %f) : memref<?x?x?x?x?xf32>, f32 linalg.fill(%f, %buf) : f32, memref<?x?x?x?x?xf32>
return %buf : memref<?x?x?x?x?xf32> return %buf : memref<?x?x?x?x?xf32>
} }
func @conv_3d_ncdhw(%arg0: memref<?x?x?x?x?xf32>, %arg1: memref<?x?x?x?x?xf32>, %arg2: memref<?x?x?x?x?xf32>) { func @conv_3d_ncdhw(%arg0: memref<?x?x?x?x?xf32>, %arg1: memref<?x?x?x?x?xf32>, %arg2: memref<?x?x?x?x?xf32>) {
linalg.conv_3d_ncdhw ins (%arg0, %arg1: memref<?x?x?x?x?xf32>, memref<?x?x?x?x?xf32>) linalg.conv_3d_ncdhw ins (%arg0, %arg1: memref<?x?x?x?x?xf32>, memref<?x?x?x?x?xf32>)
outs (%arg2: memref<?x?x?x?x?xf32>) outs (%arg2: memref<?x?x?x?x?xf32>)
return return
} }
▲ Show 20 Lines • Show All 52 Lines • Show Last 20 Lines

mlir/test/Integration/Dialect/Linalg/CPU/test-conv-3d-ndhwc-call.mlir

Show All 19 Lines
// RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \ // RUN: -shared-libs=%mlir_integration_test_dir/libmlir_runner_utils%shlibext \
// RUN: | FileCheck %s // RUN: | FileCheck %s
func private @print_memref_f32(memref<*xf32>) func private @print_memref_f32(memref<*xf32>)
// Creates and returns 5-D buffer of size (%s1, %s2, %s3, %s4, %s5) filled with the value %f // Creates and returns 5-D buffer of size (%s1, %s2, %s3, %s4, %s5) filled with the value %f
func @alloc_5d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %s4 : index, %s5 : index, %f : f32) -> memref<?x?x?x?x?xf32> { func @alloc_5d_filled_f32(%s1 : index, %s2 : index, %s3 : index, %s4 : index, %s5 : index, %f : f32) -> memref<?x?x?x?x?xf32> {
%buf = memref.alloc(%s1, %s2, %s3, %s4, %s5) : memref<?x?x?x?x?xf32> %buf = memref.alloc(%s1, %s2, %s3, %s4, %s5) : memref<?x?x?x?x?xf32>
linalg.fill(%buf, %f) : memref<?x?x?x?x?xf32>, f32 linalg.fill(%f, %buf) : f32, memref<?x?x?x?x?xf32>
return %buf : memref<?x?x?x?x?xf32> return %buf : memref<?x?x?x?x?xf32>
} }
func @conv_3d_ndhwc(%arg0: memref<?x?x?x?x?xf32>, %arg1: memref<?x?x?x?x?xf32>, %arg2: memref<?x?x?x?x?xf32>) { func @conv_3d_ndhwc(%arg0: memref<?x?x?x?x?xf32>, %arg1: memref<?x?x?x?x?xf32>, %arg2: memref<?x?x?x?x?xf32>) {
linalg.conv_3d_ndhwc ins (%arg0, %arg1: memref<?x?x?x?x?xf32>, memref<?x?x?x?x?xf32>) linalg.conv_3d_ndhwc ins (%arg0, %arg1: memref<?x?x?x?x?xf32>, memref<?x?x?x?x?xf32>)
outs (%arg2: memref<?x?x?x?x?xf32>) outs (%arg2: memref<?x?x?x?x?xf32>)
return return
} }
▲ Show 20 Lines • Show All 154 Lines • Show Last 20 Lines

mlir/test/mlir-cpu-runner/async.mlir

Show All 20 Linesfunc @main() {
%c0 = constant 0.0 : f32 %c0 = constant 0.0 : f32
%c1 = constant 1.0 : f32 %c1 = constant 1.0 : f32
%c2 = constant 2.0 : f32 %c2 = constant 2.0 : f32
%c3 = constant 3.0 : f32 %c3 = constant 3.0 : f32
%c4 = constant 4.0 : f32 %c4 = constant 4.0 : f32
%A = memref.alloc() : memref<4xf32> %A = memref.alloc() : memref<4xf32>
linalg.fill(%A, %c0) : memref<4xf32>, f32 linalg.fill(%c0, %A) : f32, memref<4xf32>
// CHECK: [0, 0, 0, 0] // CHECK: [0, 0, 0, 0]
%U = memref.cast %A : memref<4xf32> to memref<*xf32> %U = memref.cast %A : memref<4xf32> to memref<*xf32>
call @print_memref_f32(%U): (memref<*xf32>) -> () call @print_memref_f32(%U): (memref<*xf32>) -> ()
// CHECK: Current thread id: [[MAIN:.*]] // CHECK: Current thread id: [[MAIN:.*]]
// CHECK: [1, 0, 0, 0] // CHECK: [1, 0, 0, 0]
memref.store %c1, %A[%i0]: memref<4xf32> memref.store %c1, %A[%i0]: memref<4xf32>
▲ Show 20 Lines • Show All 51 Lines • Show Last 20 Lines

mlir/test/mlir-cpu-runner/sgemm_naive_codegen.mlir

// RUN: mlir-opt -convert-linalg-to-loops -lower-affine -convert-scf-to-std -convert-vector-to-llvm -convert-std-to-llvm %s | mlir-cpu-runner -O3 -e main -entry-point-result=void -shared-libs=%mlir_runner_utils_dir/libmlir_c_runner_utils%shlibext | FileCheck %s // RUN: mlir-opt -convert-linalg-to-loops -lower-affine -convert-scf-to-std -convert-vector-to-llvm -convert-std-to-llvm %s | mlir-cpu-runner -O3 -e main -entry-point-result=void -shared-libs=%mlir_runner_utils_dir/libmlir_c_runner_utils%shlibext | FileCheck %s
func @main() { func @main() {
%A = memref.alloc() : memref<16x16xf32> %A = memref.alloc() : memref<16x16xf32>
%B = memref.alloc() : memref<16x16xf32> %B = memref.alloc() : memref<16x16xf32>
%C = memref.alloc() : memref<16x16xf32> %C = memref.alloc() : memref<16x16xf32>
%cf1 = constant 1.00000e+00 : f32 %cf1 = constant 1.00000e+00 : f32
linalg.fill(%A, %cf1) : memref<16x16xf32>, f32 linalg.fill(%cf1, %A) : f32, memref<16x16xf32>
linalg.fill(%B, %cf1) : memref<16x16xf32>, f32 linalg.fill(%cf1, %B) : f32, memref<16x16xf32>
%reps = constant 1 : index %reps = constant 1 : index
%t_start = call @rtclock() : () -> f64 %t_start = call @rtclock() : () -> f64
affine.for %arg0 = 0 to 5 { affine.for %arg0 = 0 to 5 {
linalg.fill(%C, %cf1) : memref<16x16xf32>, f32 linalg.fill(%cf1, %C) : f32, memref<16x16xf32>
call @sgemm_naive(%A, %B, %C) : (memref<16x16xf32>, memref<16x16xf32>, memref<16x16xf32>) -> () call @sgemm_naive(%A, %B, %C) : (memref<16x16xf32>, memref<16x16xf32>, memref<16x16xf32>) -> ()
} }
%t_end = call @rtclock() : () -> f64 %t_end = call @rtclock() : () -> f64
%t = subf %t_end, %t_start : f64 %t = subf %t_end, %t_start : f64
%res = affine.load %C[0, 0]: memref<16x16xf32> %res = affine.load %C[0, 0]: memref<16x16xf32>
vector.print %res: f32 vector.print %res: f32
▲ Show 20 Lines • Show All 48 Lines • Show Last 20 Lines

mlir/test/mlir-cpu-runner/unranked_memref.mlir

Show All 39 Lines
// CHECK-COUNT-4: [1, 1, 1] // CHECK-COUNT-4: [1, 1, 1]
func @main() -> () { func @main() -> () {
%A = memref.alloc() : memref<10x3xf32, 0> %A = memref.alloc() : memref<10x3xf32, 0>
%f2 = constant 2.00000e+00 : f32 %f2 = constant 2.00000e+00 : f32
%f5 = constant 5.00000e+00 : f32 %f5 = constant 5.00000e+00 : f32
%f10 = constant 10.00000e+00 : f32 %f10 = constant 10.00000e+00 : f32
%V = memref.cast %A : memref<10x3xf32, 0> to memref<?x?xf32> %V = memref.cast %A : memref<10x3xf32, 0> to memref<?x?xf32>
linalg.fill(%V, %f10) : memref<?x?xf32, 0>, f32 linalg.fill(%f10, %V) : f32, memref<?x?xf32, 0>
%U = memref.cast %A : memref<10x3xf32, 0> to memref<*xf32> %U = memref.cast %A : memref<10x3xf32, 0> to memref<*xf32>
call @print_memref_f32(%U) : (memref<*xf32>) -> () call @print_memref_f32(%U) : (memref<*xf32>) -> ()
%V2 = memref.cast %U : memref<*xf32> to memref<?x?xf32> %V2 = memref.cast %U : memref<*xf32> to memref<?x?xf32>
linalg.fill(%V2, %f5) : memref<?x?xf32, 0>, f32 linalg.fill(%f5, %V2) : f32, memref<?x?xf32, 0>
%U2 = memref.cast %V2 : memref<?x?xf32, 0> to memref<*xf32> %U2 = memref.cast %V2 : memref<?x?xf32, 0> to memref<*xf32>
call @print_memref_f32(%U2) : (memref<*xf32>) -> () call @print_memref_f32(%U2) : (memref<*xf32>) -> ()
%V3 = memref.cast %V2 : memref<?x?xf32> to memref<*xf32> %V3 = memref.cast %V2 : memref<?x?xf32> to memref<*xf32>
%V4 = memref.cast %V3 : memref<*xf32> to memref<?x?xf32> %V4 = memref.cast %V3 : memref<*xf32> to memref<?x?xf32>
linalg.fill(%V4, %f2) : memref<?x?xf32, 0>, f32 linalg.fill(%f2, %V4) : f32, memref<?x?xf32, 0>
%U3 = memref.cast %V2 : memref<?x?xf32> to memref<*xf32> %U3 = memref.cast %V2 : memref<?x?xf32> to memref<*xf32>
call @print_memref_f32(%U3) : (memref<*xf32>) -> () call @print_memref_f32(%U3) : (memref<*xf32>) -> ()
// 122 is ASCII for 'z'. // 122 is ASCII for 'z'.
%i8_z = constant 122 : i8 %i8_z = constant 122 : i8
%I8 = memref.alloc() : memref<i8> %I8 = memref.alloc() : memref<i8>
memref.store %i8_z, %I8[]: memref<i8> memref.store %i8_z, %I8[]: memref<i8>
%U4 = memref.cast %I8 : memref<i8> to memref<*xi8> %U4 = memref.cast %I8 : memref<i8> to memref<*xi8>
call @print_memref_i8(%U4) : (memref<*xi8>) -> () call @print_memref_i8(%U4) : (memref<*xi8>) -> ()
memref.dealloc %A : memref<10x3xf32, 0> memref.dealloc %A : memref<10x3xf32, 0>
call @return_var_memref_caller() : () -> () call @return_var_memref_caller() : () -> ()
call @return_two_var_memref_caller() : () -> () call @return_two_var_memref_caller() : () -> ()
call @dim_op_of_unranked() : () -> () call @dim_op_of_unranked() : () -> ()
return return
} }
func private @print_memref_i8(memref<*xi8>) attributes { llvm.emit_c_interface } func private @print_memref_i8(memref<*xi8>) attributes { llvm.emit_c_interface }
func private @print_memref_f32(memref<*xf32>) attributes { llvm.emit_c_interface } func private @print_memref_f32(memref<*xf32>) attributes { llvm.emit_c_interface }
func @return_two_var_memref_caller() { func @return_two_var_memref_caller() {
%0 = memref.alloca() : memref<4x3xf32> %0 = memref.alloca() : memref<4x3xf32>
%c0f32 = constant 1.0 : f32 %c0f32 = constant 1.0 : f32
linalg.fill(%0, %c0f32) : memref<4x3xf32>, f32 linalg.fill(%c0f32, %0) : f32, memref<4x3xf32>
%1:2 = call @return_two_var_memref(%0) : (memref<4x3xf32>) -> (memref<*xf32>, memref<*xf32>) %1:2 = call @return_two_var_memref(%0) : (memref<4x3xf32>) -> (memref<*xf32>, memref<*xf32>)
call @print_memref_f32(%1#0) : (memref<*xf32>) -> () call @print_memref_f32(%1#0) : (memref<*xf32>) -> ()
call @print_memref_f32(%1#1) : (memref<*xf32>) -> () call @print_memref_f32(%1#1) : (memref<*xf32>) -> ()
return return
} }
func @return_two_var_memref(%arg0: memref<4x3xf32>) -> (memref<*xf32>, memref<*xf32>) { func @return_two_var_memref(%arg0: memref<4x3xf32>) -> (memref<*xf32>, memref<*xf32>) {
%0 = memref.cast %arg0 : memref<4x3xf32> to memref<*xf32> %0 = memref.cast %arg0 : memref<4x3xf32> to memref<*xf32>
return %0, %0 : memref<*xf32>, memref<*xf32> return %0, %0 : memref<*xf32>, memref<*xf32>
} }
func @return_var_memref_caller() { func @return_var_memref_caller() {
%0 = memref.alloca() : memref<4x3xf32> %0 = memref.alloca() : memref<4x3xf32>
%c0f32 = constant 1.0 : f32 %c0f32 = constant 1.0 : f32
linalg.fill(%0, %c0f32) : memref<4x3xf32>, f32 linalg.fill(%c0f32, %0) : f32, memref<4x3xf32>
%1 = call @return_var_memref(%0) : (memref<4x3xf32>) -> memref<*xf32> %1 = call @return_var_memref(%0) : (memref<4x3xf32>) -> memref<*xf32>
call @print_memref_f32(%1) : (memref<*xf32>) -> () call @print_memref_f32(%1) : (memref<*xf32>) -> ()
return return
} }
func @return_var_memref(%arg0: memref<4x3xf32>) -> memref<*xf32> { func @return_var_memref(%arg0: memref<4x3xf32>) -> memref<*xf32> {
%0 = memref.cast %arg0: memref<4x3xf32> to memref<*xf32> %0 = memref.cast %arg0: memref<4x3xf32> to memref<*xf32>
return %0 : memref<*xf32> return %0 : memref<*xf32>
Show All 23 Lines

mlir/test/mlir-cpu-runner/utils.mlir

Show All 13 Lines
} }
// PRINT-0D: Unranked Memref base@ = {{.*}} rank = 0 offset = 0 sizes = [] strides = [] data = // PRINT-0D: Unranked Memref base@ = {{.*}} rank = 0 offset = 0 sizes = [] strides = [] data =
// PRINT-0D: [2] // PRINT-0D: [2]
func @print_1d() { func @print_1d() {
%f = constant 2.00000e+00 : f32 %f = constant 2.00000e+00 : f32
%A = memref.alloc() : memref<16xf32> %A = memref.alloc() : memref<16xf32>
%B = memref.cast %A: memref<16xf32> to memref<?xf32> %B = memref.cast %A: memref<16xf32> to memref<?xf32>
linalg.fill(%B, %f) : memref<?xf32>, f32 linalg.fill(%f, %B) : f32, memref<?xf32>
%U = memref.cast %B : memref<?xf32> to memref<*xf32> %U = memref.cast %B : memref<?xf32> to memref<*xf32>
call @print_memref_f32(%U): (memref<*xf32>) -> () call @print_memref_f32(%U): (memref<*xf32>) -> ()
memref.dealloc %A : memref<16xf32> memref.dealloc %A : memref<16xf32>
return return
} }
// PRINT-1D: Unranked Memref base@ = {{.*}} rank = 1 offset = 0 sizes = [16] strides = [1] data = // PRINT-1D: Unranked Memref base@ = {{.*}} rank = 1 offset = 0 sizes = [16] strides = [1] data =
// PRINT-1D-NEXT: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] // PRINT-1D-NEXT: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2]
func @print_3d() { func @print_3d() {
%f = constant 2.00000e+00 : f32 %f = constant 2.00000e+00 : f32
%f4 = constant 4.00000e+00 : f32 %f4 = constant 4.00000e+00 : f32
%A = memref.alloc() : memref<3x4x5xf32> %A = memref.alloc() : memref<3x4x5xf32>
%B = memref.cast %A: memref<3x4x5xf32> to memref<?x?x?xf32> %B = memref.cast %A: memref<3x4x5xf32> to memref<?x?x?xf32>
linalg.fill(%B, %f) : memref<?x?x?xf32>, f32 linalg.fill(%f, %B) : f32, memref<?x?x?xf32>
%c2 = constant 2 : index %c2 = constant 2 : index
memref.store %f4, %B[%c2, %c2, %c2]: memref<?x?x?xf32> memref.store %f4, %B[%c2, %c2, %c2]: memref<?x?x?xf32>
%U = memref.cast %B : memref<?x?x?xf32> to memref<*xf32> %U = memref.cast %B : memref<?x?x?xf32> to memref<*xf32>
call @print_memref_f32(%U): (memref<*xf32>) -> () call @print_memref_f32(%U): (memref<*xf32>) -> ()
memref.dealloc %A : memref<3x4x5xf32> memref.dealloc %A : memref<3x4x5xf32>
return return
} }
Show All 29 Lines

mlir/test/python/dialects/linalg/ops.py

Show First 20 Lines • Show All 53 Lines • ▼ Show 20 Lines
def testFill(): def testFill():
with Context() as ctx, Location.unknown(): with Context() as ctx, Location.unknown():
module = Module.create() module = Module.create()
f32 = F32Type.get() f32 = F32Type.get()
with InsertionPoint(module.body): with InsertionPoint(module.body):
# CHECK-LABEL: func @fill_tensor # CHECK-LABEL: func @fill_tensor
# CHECK-SAME: %[[OUT:[0-9a-z]+]]: tensor<12x?xf32> # CHECK-SAME: %[[OUT:[0-9a-z]+]]: tensor<12x?xf32>
# CHECK-NEXT: %[[CST:.*]] = constant 0.0{{.*}} : f32 # CHECK-NEXT: %[[CST:.*]] = constant 0.0{{.*}} : f32
# CHECK-NEXT: %[[RES:.*]] = linalg.fill(%[[OUT]], %[[CST]]) : tensor<12x?xf32>, f32 -> tensor<12x?xf32> # CHECK-NEXT: %[[RES:.*]] = linalg.fill(%[[CST]], %[[OUT]]) : f32, tensor<12x?xf32> -> tensor<12x?xf32>
# CHECK-NEXT: return %[[RES]] : tensor<12x?xf32> # CHECK-NEXT: return %[[RES]] : tensor<12x?xf32>
@builtin.FuncOp.from_py_func( @builtin.FuncOp.from_py_func(
RankedTensorType.get((12, -1), f32)) RankedTensorType.get((12, -1), f32))
def fill_tensor(out): def fill_tensor(out):
zero = std.ConstantOp(value=FloatAttr.get(f32, 0.), result=f32).result zero = std.ConstantOp(value=FloatAttr.get(f32, 0.), result=f32).result
# TODO: FillOp.result is None. When len(results) == 1 we expect it to # TODO: FillOp.result is None. When len(results) == 1 we expect it to
# be results[0] as per _linalg_ops_gen.py. This seems like an # be results[0] as per _linalg_ops_gen.py. This seems like an
# orthogonal bug in the generator of _linalg_ops_gen.py. # orthogonal bug in the generator of _linalg_ops_gen.py.
return linalg.FillOp(output=out, value=zero).results[0] return linalg.FillOp(output=out, value=zero).results[0]
# CHECK-LABEL: func @fill_buffer # CHECK-LABEL: func @fill_buffer
# CHECK-SAME: %[[OUT:[0-9a-z]+]]: memref<12x?xf32> # CHECK-SAME: %[[OUT:[0-9a-z]+]]: memref<12x?xf32>
# CHECK-NEXT: %[[CST:.*]] = constant 0.0{{.*}} : f32 # CHECK-NEXT: %[[CST:.*]] = constant 0.0{{.*}} : f32
# CHECK-NEXT: linalg.fill(%[[OUT]], %[[CST]]) : memref<12x?xf32>, f32 # CHECK-NEXT: linalg.fill(%[[CST]], %[[OUT]]) : f32, memref<12x?xf32>
# CHECK-NEXT: return # CHECK-NEXT: return
@builtin.FuncOp.from_py_func( @builtin.FuncOp.from_py_func(
MemRefType.get((12, -1), f32)) MemRefType.get((12, -1), f32))
def fill_buffer(out): def fill_buffer(out):
zero = std.ConstantOp(value=FloatAttr.get(f32, 0.), result=f32).result zero = std.ConstantOp(value=FloatAttr.get(f32, 0.), result=f32).result
linalg.FillOp(output=out, value=zero) linalg.FillOp(output=out, value=zero)
print(module) print(module)
▲ Show 20 Lines • Show All 103 Lines • Show Last 20 Lines

mlir/test/python/dialects/linalg/opsrun.py

Show All 19 Lines
func @main() -> f32 attributes {llvm.emit_c_interface} { func @main() -> f32 attributes {llvm.emit_c_interface} {
%v0 = constant 0.0 : f32 %v0 = constant 0.0 : f32
%v1 = constant 1.0 : f32 %v1 = constant 1.0 : f32
%v2 = constant 2.0 : f32 %v2 = constant 2.0 : f32
%A = memref.alloc() : memref<4x16xf32> %A = memref.alloc() : memref<4x16xf32>
%B = memref.alloc() : memref<16x8xf32> %B = memref.alloc() : memref<16x8xf32>
%C = memref.alloc() : memref<4x8xf32> %C = memref.alloc() : memref<4x8xf32>
linalg.fill(%A, %v1) : memref<4x16xf32>, f32 linalg.fill(%v1, %A) : f32, memref<4x16xf32>
linalg.fill(%B, %v2) : memref<16x8xf32>, f32 linalg.fill(%v2, %B) : f32, memref<16x8xf32>
linalg.fill(%C, %v0) : memref<4x8xf32>, f32 linalg.fill(%v0, %C) : f32, memref<4x8xf32>
call @matmul_on_buffers(%A, %B, %C) : call @matmul_on_buffers(%A, %B, %C) :
(memref<4x16xf32>, memref<16x8xf32>, memref<4x8xf32>) -> () (memref<4x16xf32>, memref<16x8xf32>, memref<4x8xf32>) -> ()
%c0 = constant 0 : index %c0 = constant 0 : index
%0 = memref.load %C[%c0, %c0] : memref<4x8xf32> %0 = memref.load %C[%c0, %c0] : memref<4x8xf32>
// TODO: FFI-based solution to allow testing and printing with python code. // TODO: FFI-based solution to allow testing and printing with python code.
▲ Show 20 Lines • Show All 147 Lines • Show Last 20 Lines