This is an archive of the discontinued LLVM Phabricator instance.

Differential D120624

[mlir] Rename the Standard dialect to the Func dialect
ClosedPublic

Authored by rriddle on Feb 26 2022, 2:51 PM.

Details

Reviewers
lebedev.ri
antiagainst
aartbik
ftynse
sjarus
clementval
bondhugula
nicolasvasilache
herhut
jsetoain
lattner
Commits
rG23aa5a744666: [mlir] Rename the Standard dialect to the Func dialect
Summary

The last remaining operations in the standard dialect all revolve around
FuncOp/function related constructs. This patch simply handles the initial
renaming (which by itself is already huge), but there are a large number
of cleanups unlocked/necessary afterwards:

  • Removing a bunch of unnecessary dependencies on Func
  • Cleaning up the From/ToStandard conversion passes
  • Preparing for the move of FuncOp to the Func dialect

See the discussion at https://discourse.llvm.org/t/standard-dialect-the-final-chapter/6061

Diff Detail

Event Timeline

rriddle created this revision.Feb 26 2022, 2:51 PM
Herald added a reviewer: lebedev.ri. · View Herald TranscriptFeb 26 2022, 2:51 PM
Herald added a reviewer: antiagainst. · View Herald Transcript
Herald added a reviewer: aartbik. · View Herald Transcript
Herald added a reviewer: ftynse. · View Herald Transcript
Herald added a reviewer: aartbik. · View Herald Transcript
Herald added a reviewer: sjarus. · View Herald Transcript
Herald added a reviewer: clementval. · View Herald Transcript
Herald added a reviewer: bondhugula. · View Herald Transcript
Herald added subscribers: awarzynski, arjunp, sdasgup3 and 35 others. · View Herald Transcript
rriddle requested review of this revision.Feb 26 2022, 2:51 PM
Herald added a reviewer: nicolasvasilache. · View Herald TranscriptFeb 26 2022, 2:51 PM
Herald added a reviewer: herhut. · View Herald Transcript
Herald added a project: Restricted Project. · View Herald Transcript
Herald added subscribers: stephenneuendorffer, nicolasvasilache. · View Herald Transcript
Harbormaster completed remote builds in B151639: Diff 411646.Feb 26 2022, 3:02 PM
Herald added a reviewer: jsetoain. · View Herald TranscriptFeb 26 2022, 3:02 PM
lattner accepted this revision.Feb 26 2022, 4:59 PM
This revision is now accepted and ready to land.Feb 26 2022, 4:59 PM
ftynse added inline comments.Feb 28 2022, 2:59 AM
mlir/tools/mlir-spirv-cpu-runner/mlir-spirv-cpu-runner.cpp
102

This needs the ::func namespace.

mlir/tools/mlir-vulkan-runner/mlir-vulkan-runner.cpp
78

This needs the ::func namespace.

ftynse requested changes to this revision.Feb 28 2022, 3:05 AM

Looks good in general.

There are a couple of build failures on less frequently used targets (namely spirv and vulkan runner), and in flang. Some Python updates are also missing, in particular there are imports that are not of the form import mlir.dialects.std that may not have been caught by the rename regexp.

mlir/test/python/dialects/scf.py
67

There's a name conflict between the imported func module and the func variable defined at line 62.

This revision now requires changes to proceed.Feb 28 2022, 3:05 AM
rriddle requested review of this revision.Feb 28 2022, 9:25 AM
rriddle updated this revision to Diff 411821.
rriddle marked 3 inline comments as done.
Herald added a project: Restricted Project. · View Herald TranscriptFeb 28 2022, 9:25 AM
Harbormaster completed remote builds in B151764: Diff 411821.Feb 28 2022, 9:47 AM
Herald added a subscriber: jdoerfert. · View Herald TranscriptFeb 28 2022, 9:47 AM
rriddle updated this revision to Diff 411848.Feb 28 2022, 11:07 AM
rriddle added a comment.Feb 28 2022, 11:07 AM

Thanks @ftynse , everything should be good now.

Harbormaster completed remote builds in B151783: Diff 411848.Feb 28 2022, 11:26 AM
rriddle updated this revision to Diff 411907.Feb 28 2022, 3:09 PM
Harbormaster completed remote builds in B151823: Diff 411907.Feb 28 2022, 3:30 PM
ftynse accepted this revision.Mar 1 2022, 2:09 AM
This revision is now accepted and ready to land.Mar 1 2022, 2:09 AM
This revision was landed with ongoing or failed builds.Mar 1 2022, 12:10 PM
Closed by commit rG23aa5a744666: [mlir] Rename the Standard dialect to the Func dialect (authored by rriddle). · Explain Why
This revision was automatically updated to reflect the committed changes.
rriddle added a commit: rG23aa5a744666: [mlir] Rename the Standard dialect to the Func dialect.
Herald added a project: Restricted Project. · View Herald TranscriptMar 1 2022, 12:10 PM

Revision Contents

PathSize
flang/
include/
flang/
Lower/
Support/
2 lines
Optimizer/
Builder/
2 lines
Runtime/
2 lines
2 lines
2 lines
2 lines
Dialect/
1 line
2 lines
4 lines
Support/
2 lines
2 lines
Transforms/
10 lines
lib/
Lower/
4 lines
2 lines
2 lines
Optimizer/
Builder/
Runtime/
2 lines
2 lines
2 lines
2 lines
CodeGen/
2 lines
2 lines
8 lines
Dialect/
2 lines
2 lines
Transforms/
14 lines
4 lines
4 lines
2 lines
4 lines
2 lines
6 lines
2 lines
4 lines
mlir/
benchmark/
python/
10 lines
docs/
Bindings/
10 lines
2 lines
16 lines
Dialects/
2 lines
15 lines
2 lines
8 lines
2 lines
2 lines
8 lines
2 lines
4 lines
Tutorials/
2 lines
Toy/
4 lines
examples/
standalone/
standalone-opt/
2 lines
toy/
Ch5/
mlir/
12 lines
Ch6/
mlir/
12 lines
11 lines
Ch7/
mlir/
12 lines
11 lines
include/
mlir-c/
Dialect/
Func.h
Standard.h
12 lines
2 lines
mlir/
Conversion/
LLVMCommon/
4 lines
2 lines
LinalgToStandard/
2 lines
12 lines
VectorToSCF/
2 lines
Dialect/
Affine/
Analysis/
2 lines
Arithmetic/
Transforms/
2 lines
ArmSVE/
2 lines
2 lines
ControlFlow/
IR/
6 lines
Func/
StandardOps/
IR/
26 lines
87 lines
Transforms/
6 lines
10 lines
12 lines
14 lines
Func/
IR/
8 lines
Transforms/
5 lines
Linalg/
Transforms/
4 lines
Utils/
2 lines
Math/
IR/
6 lines
MemRef/
IR/
2 lines
Transforms/
4 lines
4 lines
PDL/
IR/
2 lines
StandardOps/
IR/
Transforms/
Tosa/
Transforms/
2 lines
2 lines
Utils/
2 lines
4 lines
4 lines
lib/
CAPI/
Dialect/
6 lines
Func.cpp
Standard.cpp
8 lines
Conversion/
AffineToStandard/
8 lines
2 lines
AsyncToLLVM/
76 lines
2 lines
ComplexToLLVM/
2 lines
2 lines
ComplexToStandard/
2 lines
4 lines
ControlFlowToLLVM/
2 lines
GPUCommon/
2 lines
2 lines
4 lines
GPUToVulkan/
2 lines
4 lines
LinalgToSPIRV/
2 lines
LinalgToStandard/
10 lines
MathToLibm/
2 lines
8 lines
MemRefToLLVM/
2 lines
OpenACCToSCF/
2 lines
5 lines
SCFToGPU/
2 lines
2 lines
2 lines
SCFToOpenMP/
2 lines
2 lines
SCFToSPIRV/
2 lines
SPIRVToLLVM/
2 lines
2 lines
ShapeToStandard/
2 lines
4 lines
StandardToLLVM/
2 lines
23 lines
StandardToSPIRV/
2 lines
12 lines
TosaToLinalg/
2 lines
2 lines
8 lines
8 lines
TosaToSCF/
2 lines
TosaToStandard/
2 lines
4 lines
4 lines
VectorToLLVM/
2 lines
4 lines
VectorToROCDL/
2 lines
Dialect/
AMX/
Transforms/
2 lines
Affine/
Analysis/
2 lines
2 lines
2 lines
2 lines
Transforms/
2 lines
2 lines
2 lines
Arithmetic/
Transforms/
4 lines
6 lines
ArmSVE/
Transforms/
2 lines
6 lines
Async/
Transforms/
36 lines
4 lines
4 lines
18 lines
2 lines
Bufferization/
Transforms/
12 lines
7 lines
2 lines
6 lines
2 lines
Func/
IR/
17 lines
Func/
StandardOps/
IR/
45 lines
Transforms/
8 lines
5 lines
13 lines
11 lines
10 lines
GPU/
2 lines
Transforms/
6 lines
Linalg/
Analysis/
2 lines
2 lines
ComprehensiveBufferize/
4 lines
50 lines
Transforms/
6 lines
4 lines
2 lines
2 lines
2 lines
4 lines
2 lines
6 lines
Utils/
2 lines
2 lines
Math/
Transforms/
2 lines
2 lines
MemRef/
IR/
1 line
Transforms/
2 lines
4 lines
15 lines
5 lines
SCF/
2 lines
2 lines
Transforms/
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
Utils/
2 lines
9 lines
Shape/
Transforms/
6 lines
SparseTensor/
Pipelines/
2 lines
4 lines
Transforms/
2 lines
29 lines
9 lines
2 lines
StandardOps/
IR/
Transforms/
Tensor/
Transforms/
2 lines
Tosa/
Transforms/
2 lines
8 lines
Vector/
Utils/
2 lines
4 lines
X86Vector/
Transforms/
1 line
ExecutionEngine/
2 lines
IR/
4 lines
Parser/
5 lines
Target/
Cpp/
2 lines
4 lines
22 lines
lib/
Dialect/
Func/
include/
mlir/
Dialect/
StandardOps/
python/
20 lines
mlir/
dialects/
FuncOps.td
StandardOps.td
12 lines
8 lines
_func_ops_ext.py
_std_ops_ext.py
4 lines
func.py
std.py
2 lines
test/
CAPI/
2 lines
22 lines
6 lines
Conversion/
ArithmeticToLLVM/
8 lines
2 lines
GPUToNVVM/
46 lines
GPUToROCDL/
42 lines
MathToLLVM/
30 lines
MemRefToLLVM/
2 lines
ShapeToStandard/
2 lines
StandardToLLVM/
10 lines
StandardToSPIRV/
6 lines
Dialect/
Arithmetic/
4 lines
Tensor/
2 lines
IR/
6 lines
6 lines
2 lines
6 lines
6 lines
2 lines
14 lines
2 lines
20 lines
Integration/
Dialect/
SparseTensor/
python/
4 lines
taco/
tools/
2 lines
Transforms/
30 lines
lib/
Conversion/
StandardToLLVM/
6 lines
Dialect/
Affine/
2 lines
2 lines
Func/
StandardOps/
6 lines
10 lines
GPU/
2 lines
4 lines
4 lines
Linalg/
4 lines
4 lines
4 lines
4 lines
SCF/
4 lines
4 lines
StandardOps/
Test/
4 lines
6 lines
10 lines
Tosa/
2 lines
Vector/
2 lines
4 lines
IR/
2 lines
1 line
1 line
1 line
4 lines
Transforms/
7 lines
mlir-opt/
2 lines
python/
dialects/
12 lines
func.py
std.py
10 lines
8 lines
8 lines
4 lines
ir/
7 lines
22 lines
6 lines
2 lines
tools/
mlir-opt/
2 lines
mlir-spirv-cpu-runner/
2 lines
4 lines
mlir-vulkan-runner/
2 lines
4 lines
unittests/
Conversion/
PDLToPDLInterp/
2 lines
Interfaces/
2 lines
4 lines
utils/
vim/
syntax/
1 line

Diff 411821

flang/include/flang/Lower/Support/Utils.h

Show All 10 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef FORTRAN_LOWER_SUPPORT_UTILS_H #ifndef FORTRAN_LOWER_SUPPORT_UTILS_H
#define FORTRAN_LOWER_SUPPORT_UTILS_H #define FORTRAN_LOWER_SUPPORT_UTILS_H
#include "flang/Common/indirection.h" #include "flang/Common/indirection.h"
#include "flang/Parser/char-block.h" #include "flang/Parser/char-block.h"
#include "flang/Semantics/tools.h" #include "flang/Semantics/tools.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/BuiltinAttributes.h" #include "mlir/IR/BuiltinAttributes.h"
#include "llvm/ADT/StringRef.h" #include "llvm/ADT/StringRef.h"
#include <cstdint> #include <cstdint>
namespace Fortran::lower { namespace Fortran::lower {
using SomeExpr = Fortran::evaluate::Expr<Fortran::evaluate::SomeType>; using SomeExpr = Fortran::evaluate::Expr<Fortran::evaluate::SomeType>;
} }
Show All 33 Lines

flang/include/flang/Optimizer/Builder/Factory.h

Show All 11 Lines
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef FORTRAN_OPTIMIZER_BUILDER_FACTORY_H #ifndef FORTRAN_OPTIMIZER_BUILDER_FACTORY_H
#define FORTRAN_OPTIMIZER_BUILDER_FACTORY_H #define FORTRAN_OPTIMIZER_BUILDER_FACTORY_H
#include "flang/Optimizer/Dialect/FIROps.h" #include "flang/Optimizer/Dialect/FIROps.h"
#include "flang/Optimizer/Dialect/FIRType.h" #include "flang/Optimizer/Dialect/FIRType.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "llvm/ADT/iterator_range.h" #include "llvm/ADT/iterator_range.h"
namespace mlir { namespace mlir {
class Location; class Location;
class Value; class Value;
} // namespace mlir } // namespace mlir
namespace fir::factory { namespace fir::factory {
▲ Show 20 Lines • Show All 83 Lines • Show Last 20 Lines

flang/include/flang/Optimizer/Builder/Runtime/Character.h

//===-- Character.h -- generate calls to character runtime API --*- C++ -*-===// //===-- Character.h -- generate calls to character runtime API --*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef FORTRAN_OPTIMIZER_BUILDER_RUNTIME_CHARACTER_H #ifndef FORTRAN_OPTIMIZER_BUILDER_RUNTIME_CHARACTER_H
#define FORTRAN_OPTIMIZER_BUILDER_RUNTIME_CHARACTER_H #define FORTRAN_OPTIMIZER_BUILDER_RUNTIME_CHARACTER_H
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
namespace fir { namespace fir {
class ExtendedValue; class ExtendedValue;
class FirOpBuilder; class FirOpBuilder;
} // namespace fir } // namespace fir
namespace fir::runtime { namespace fir::runtime {
▲ Show 20 Lines • Show All 104 Lines • Show Last 20 Lines

flang/include/flang/Optimizer/Builder/Runtime/Numeric.h

//===-- Numeric.h -- generate numeric intrinsics runtime calls --*- C++ -*-===// //===-- Numeric.h -- generate numeric intrinsics runtime calls --*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef FORTRAN_OPTIMIZER_BUILDER_RUNTIME_NUMERIC_H #ifndef FORTRAN_OPTIMIZER_BUILDER_RUNTIME_NUMERIC_H
#define FORTRAN_OPTIMIZER_BUILDER_RUNTIME_NUMERIC_H #define FORTRAN_OPTIMIZER_BUILDER_RUNTIME_NUMERIC_H
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
namespace fir { namespace fir {
class ExtendedValue; class ExtendedValue;
class FirOpBuilder; class FirOpBuilder;
} // namespace fir } // namespace fir
namespace fir::runtime { namespace fir::runtime {
Show All 30 Lines

flang/include/flang/Optimizer/Builder/Runtime/Reduction.h

Show All 10 Lines
// have a specialization where the value is returned as a scalar value, e.g. // have a specialization where the value is returned as a scalar value, e.g.
// `genAll` is a specialization of `genAllDescriptor`. // `genAll` is a specialization of `genAllDescriptor`.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef FORTRAN_OPTIMIZER_BUILDER_RUNTIME_REDUCTION_H #ifndef FORTRAN_OPTIMIZER_BUILDER_RUNTIME_REDUCTION_H
#define FORTRAN_OPTIMIZER_BUILDER_RUNTIME_REDUCTION_H #define FORTRAN_OPTIMIZER_BUILDER_RUNTIME_REDUCTION_H
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
namespace fir { namespace fir {
class ExtendedValue; class ExtendedValue;
class FirOpBuilder; class FirOpBuilder;
} // namespace fir } // namespace fir
namespace fir::runtime { namespace fir::runtime {
▲ Show 20 Lines • Show All 130 Lines • Show Last 20 Lines

flang/include/flang/Optimizer/Builder/Runtime/Transformational.h

//===-- Transformational.h --------------------------------------*- C++ -*-===// //===-- Transformational.h --------------------------------------*- C++ -*-===//
// Generate transformational intrinsic runtime API calls. // Generate transformational intrinsic runtime API calls.
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef FORTRAN_OPTIMIZER_BUILDER_RUNTIME_TRANSFORMATIONAL_H #ifndef FORTRAN_OPTIMIZER_BUILDER_RUNTIME_TRANSFORMATIONAL_H
#define FORTRAN_OPTIMIZER_BUILDER_RUNTIME_TRANSFORMATIONAL_H #define FORTRAN_OPTIMIZER_BUILDER_RUNTIME_TRANSFORMATIONAL_H
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
namespace fir { namespace fir {
class ExtendedValue; class ExtendedValue;
class FirOpBuilder; class FirOpBuilder;
} // namespace fir } // namespace fir
namespace fir::runtime { namespace fir::runtime {
▲ Show 20 Lines • Show All 42 Lines • Show Last 20 Lines

flang/include/flang/Optimizer/Dialect/CanonicalizationPatterns.td

Show All 10 Lines
/// ///
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef FORTRAN_FIR_REWRITE_PATTERNS #ifndef FORTRAN_FIR_REWRITE_PATTERNS
#define FORTRAN_FIR_REWRITE_PATTERNS #define FORTRAN_FIR_REWRITE_PATTERNS
include "mlir/IR/OpBase.td" include "mlir/IR/OpBase.td"
include "mlir/Dialect/Arithmetic/IR/ArithmeticOps.td" include "mlir/Dialect/Arithmetic/IR/ArithmeticOps.td"
include "mlir/Dialect/StandardOps/IR/Ops.td"
include "flang/Optimizer/Dialect/FIROps.td" include "flang/Optimizer/Dialect/FIROps.td"
def IdenticalTypePred : Constraint<CPred<"$0.getType() == $1.getType()">>; def IdenticalTypePred : Constraint<CPred<"$0.getType() == $1.getType()">>;
def IntegerTypePred : Constraint<CPred<"fir::isa_integer($0.getType())">>; def IntegerTypePred : Constraint<CPred<"fir::isa_integer($0.getType())">>;
def IndexTypePred : Constraint<CPred<"$0.getType().isa<mlir::IndexType>()">>; def IndexTypePred : Constraint<CPred<"$0.getType().isa<mlir::IndexType>()">>;
def SmallerWidthPred def SmallerWidthPred
: Constraint<CPred<"$0.getType().getIntOrFloatBitWidth() " : Constraint<CPred<"$0.getType().getIntOrFloatBitWidth() "
Show All 34 Lines

flang/include/flang/Optimizer/Dialect/FIROps.h

//===-- Optimizer/Dialect/FIROps.h - FIR operations -------------*- C++ -*-===// //===-- Optimizer/Dialect/FIROps.h - FIR operations -------------*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef FORTRAN_OPTIMIZER_DIALECT_FIROPS_H #ifndef FORTRAN_OPTIMIZER_DIALECT_FIROPS_H
#define FORTRAN_OPTIMIZER_DIALECT_FIROPS_H #define FORTRAN_OPTIMIZER_DIALECT_FIROPS_H
#include "flang/Optimizer/Dialect/FIRType.h" #include "flang/Optimizer/Dialect/FIRType.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Interfaces/LoopLikeInterface.h" #include "mlir/Interfaces/LoopLikeInterface.h"
#include "mlir/Interfaces/SideEffectInterfaces.h" #include "mlir/Interfaces/SideEffectInterfaces.h"
using namespace mlir; using namespace mlir;
namespace fir { namespace fir {
class FirEndOp; class FirEndOp;
Show All 28 Lines

flang/include/flang/Optimizer/Dialect/FIROpsSupport.h

//===-- Optimizer/Dialect/FIROpsSupport.h -- FIR op support -----*- C++ -*-===// //===-- Optimizer/Dialect/FIROpsSupport.h -- FIR op support -----*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef FORTRAN_OPTIMIZER_DIALECT_FIROPSSUPPORT_H #ifndef FORTRAN_OPTIMIZER_DIALECT_FIROPSSUPPORT_H
#define FORTRAN_OPTIMIZER_DIALECT_FIROPSSUPPORT_H #define FORTRAN_OPTIMIZER_DIALECT_FIROPSSUPPORT_H
#include "flang/Optimizer/Dialect/FIROps.h" #include "flang/Optimizer/Dialect/FIROps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
namespace fir { namespace fir {
/// return true iff the Operation is a non-volatile LoadOp /// return true iff the Operation is a non-volatile LoadOp
inline bool nonVolatileLoad(mlir::Operation *op) { inline bool nonVolatileLoad(mlir::Operation *op) {
if (auto load = dyn_cast<fir::LoadOp>(op)) if (auto load = dyn_cast<fir::LoadOp>(op))
return !load->getAttr("volatile"); return !load->getAttr("volatile");
return false; return false;
} }
/// return true iff the Operation is a call /// return true iff the Operation is a call
inline bool isaCall(mlir::Operation *op) { inline bool isaCall(mlir::Operation *op) {
return isa<fir::CallOp>(op) || isa<fir::DispatchOp>(op) || return isa<fir::CallOp>(op) || isa<fir::DispatchOp>(op) ||
isa<mlir::CallOp>(op) || isa<mlir::CallIndirectOp>(op); isa<mlir::func::CallOp>(op) || isa<mlir::func::CallIndirectOp>(op);
} }
/// return true iff the Operation is a fir::CallOp, fir::DispatchOp, /// return true iff the Operation is a fir::CallOp, fir::DispatchOp,
/// mlir::CallOp, or mlir::CallIndirectOp and not pure /// mlir::CallOp, or mlir::CallIndirectOp and not pure
/// NB: this is not the same as `!pureCall(op)` /// NB: this is not the same as `!pureCall(op)`
inline bool impureCall(mlir::Operation *op) { inline bool impureCall(mlir::Operation *op) {
// Should we also auto-detect that the called function is pure if its // Should we also auto-detect that the called function is pure if its
// arguments are not references? For now, rely on a "pure" attribute. // arguments are not references? For now, rely on a "pure" attribute.
▲ Show 20 Lines • Show All 66 Lines • Show Last 20 Lines

flang/include/flang/Optimizer/Support/InitFIR.h

Show All 22 Lines
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
namespace fir::support { namespace fir::support {
#define FLANG_NONCODEGEN_DIALECT_LIST \ #define FLANG_NONCODEGEN_DIALECT_LIST \
mlir::AffineDialect, FIROpsDialect, mlir::acc::OpenACCDialect, \ mlir::AffineDialect, FIROpsDialect, mlir::acc::OpenACCDialect, \
mlir::omp::OpenMPDialect, mlir::scf::SCFDialect, \ mlir::omp::OpenMPDialect, mlir::scf::SCFDialect, \
mlir::arith::ArithmeticDialect, mlir::cf::ControlFlowDialect, \ mlir::arith::ArithmeticDialect, mlir::cf::ControlFlowDialect, \
mlir::StandardOpsDialect, mlir::vector::VectorDialect mlir::func::FuncDialect, mlir::vector::VectorDialect
// The definitive list of dialects used by flang. // The definitive list of dialects used by flang.
#define FLANG_DIALECT_LIST \ #define FLANG_DIALECT_LIST \
FLANG_NONCODEGEN_DIALECT_LIST, FIRCodeGenDialect, mlir::LLVM::LLVMDialect FLANG_NONCODEGEN_DIALECT_LIST, FIRCodeGenDialect, mlir::LLVM::LLVMDialect
inline void registerNonCodegenDialects(mlir::DialectRegistry &registry) { inline void registerNonCodegenDialects(mlir::DialectRegistry &registry) {
registry.insert<FLANG_NONCODEGEN_DIALECT_LIST>(); registry.insert<FLANG_NONCODEGEN_DIALECT_LIST>();
} }
▲ Show 20 Lines • Show All 51 Lines • Show Last 20 Lines

flang/include/flang/Optimizer/Support/Utils.h

Show All 9 Lines
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef FORTRAN_OPTIMIZER_SUPPORT_UTILS_H #ifndef FORTRAN_OPTIMIZER_SUPPORT_UTILS_H
#define FORTRAN_OPTIMIZER_SUPPORT_UTILS_H #define FORTRAN_OPTIMIZER_SUPPORT_UTILS_H
#include "flang/Common/default-kinds.h" #include "flang/Common/default-kinds.h"
#include "flang/Optimizer/Dialect/FIRType.h" #include "flang/Optimizer/Dialect/FIRType.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/BuiltinAttributes.h" #include "mlir/IR/BuiltinAttributes.h"
namespace fir { namespace fir {
/// Return the integer value of a arith::ConstantOp. /// Return the integer value of a arith::ConstantOp.
inline std::int64_t toInt(mlir::arith::ConstantOp cop) { inline std::int64_t toInt(mlir::arith::ConstantOp cop) {
return cop.getValue().cast<mlir::IntegerAttr>().getValue().getSExtValue(); return cop.getValue().cast<mlir::IntegerAttr>().getValue().getSExtValue();
} }
Show All 18 Lines

flang/include/flang/Optimizer/Transforms/Passes.td

Show All 19 Linesdef AbstractResultOpt : Pass<"abstract-result-opt", "mlir::FuncOp"> {
let summary = "Convert fir.array, fir.box and fir.rec function result to " let summary = "Convert fir.array, fir.box and fir.rec function result to "
"function argument"; "function argument";
let description = [{ let description = [{
This pass is required before code gen to the LLVM IR dialect, This pass is required before code gen to the LLVM IR dialect,
including the pre-cg rewrite pass. including the pre-cg rewrite pass.
}]; }];
let constructor = "::fir::createAbstractResultOptPass()"; let constructor = "::fir::createAbstractResultOptPass()";
let dependentDialects = [ let dependentDialects = [
"fir::FIROpsDialect", "mlir::StandardOpsDialect" "fir::FIROpsDialect", "mlir::func::FuncDialect"
]; ];
let options = [ let options = [
Option<"passResultAsBox", "abstract-result-as-box", Option<"passResultAsBox", "abstract-result-as-box",
"bool", /*default=*/"false", "bool", /*default=*/"false",
"Pass fir.array<T> result as fir.box<fir.array<T>> argument instead" "Pass fir.array<T> result as fir.box<fir.array<T>> argument instead"
" of fir.ref<fir.array<T>>."> " of fir.ref<fir.array<T>>.">
]; ];
} }
Show All 14 Lines let description = [{
`fir.load` (`fir.store`) will be converted to `affine.load` (`affine.store`) `fir.load` (`fir.store`) will be converted to `affine.load` (`affine.store`)
where possible. This conversion includes adding a dummy `fir.convert` cast where possible. This conversion includes adding a dummy `fir.convert` cast
to adapt values of type `!fir.ref<!fir.array>` to `memref`. This is done to adapt values of type `!fir.ref<!fir.array>` to `memref`. This is done
because the affine dialect presently only understands the `memref` type. because the affine dialect presently only understands the `memref` type.
}]; }];
let constructor = "::fir::createPromoteToAffinePass()"; let constructor = "::fir::createPromoteToAffinePass()";
let dependentDialects = [ let dependentDialects = [
"fir::FIROpsDialect", "mlir::StandardOpsDialect", "mlir::AffineDialect" "fir::FIROpsDialect", "mlir::func::FuncDialect", "mlir::AffineDialect"
]; ];
} }
def AffineDialectDemotion : Pass<"demote-affine", "::mlir::FuncOp"> { def AffineDialectDemotion : Pass<"demote-affine", "::mlir::FuncOp"> {
let summary = "Converts `affine.{load,store}` back to fir operations"; let summary = "Converts `affine.{load,store}` back to fir operations";
let description = [{ let description = [{
Affine dialect's default lowering for loads and stores is different from Affine dialect's default lowering for loads and stores is different from
fir as it uses the `memref` type. The `memref` type is not compatible with fir as it uses the `memref` type. The `memref` type is not compatible with
the Fortran runtime. Therefore, conversion of memory operations back to the Fortran runtime. Therefore, conversion of memory operations back to
`fir.load` and `fir.store` with `!fir.ref<?>` types is required. `fir.load` and `fir.store` with `!fir.ref<?>` types is required.
}]; }];
let constructor = "::fir::createAffineDemotionPass()"; let constructor = "::fir::createAffineDemotionPass()";
let dependentDialects = [ let dependentDialects = [
"fir::FIROpsDialect", "mlir::StandardOpsDialect", "mlir::AffineDialect" "fir::FIROpsDialect", "mlir::func::FuncDialect", "mlir::AffineDialect"
]; ];
} }
def ArrayValueCopy : Pass<"array-value-copy", "::mlir::FuncOp"> { def ArrayValueCopy : Pass<"array-value-copy", "::mlir::FuncOp"> {
let summary = "Convert array value operations to memory operations."; let summary = "Convert array value operations to memory operations.";
let description = [{ let description = [{
Transform the set of array value primitives to a memory-based array Transform the set of array value primitives to a memory-based array
representation. representation.
Show All 35 Lines let description = [{
Transform the `fir.do_loop`, `fir.if`, and `fir.iterate_while` ops into Transform the `fir.do_loop`, `fir.if`, and `fir.iterate_while` ops into
plain old test and branch operations. Removing the high-level control plain old test and branch operations. Removing the high-level control
structures can enable other optimizations. structures can enable other optimizations.
This pass is required before code gen to the LLVM IR dialect. This pass is required before code gen to the LLVM IR dialect.
}]; }];
let constructor = "::fir::createFirToCfgPass()"; let constructor = "::fir::createFirToCfgPass()";
let dependentDialects = [ let dependentDialects = [
"fir::FIROpsDialect", "mlir::StandardOpsDialect" "fir::FIROpsDialect", "mlir::func::FuncDialect"
]; ];
let options = [ let options = [
Option<"forceLoopToExecuteOnce", "always-execute-loop-body", "bool", Option<"forceLoopToExecuteOnce", "always-execute-loop-body", "bool",
/*default=*/"false", /*default=*/"false",
"force the body of a loop to execute at least once"> "force the body of a loop to execute at least once">
]; ];
} }
Show All 9 Linesdef MemRefDataFlowOpt : Pass<"fir-memref-dataflow-opt", "::mlir::FuncOp"> {
let summary = let summary =
"Perform store/load forwarding and potentially removing dead stores."; "Perform store/load forwarding and potentially removing dead stores.";
let description = [{ let description = [{
This pass performs store to load forwarding to eliminate memory accesses and This pass performs store to load forwarding to eliminate memory accesses and
potentially the entire allocation if all the accesses are forwarded. potentially the entire allocation if all the accesses are forwarded.
}]; }];
let constructor = "::fir::createMemDataFlowOptPass()"; let constructor = "::fir::createMemDataFlowOptPass()";
let dependentDialects = [ let dependentDialects = [
"fir::FIROpsDialect", "mlir::StandardOpsDialect" "fir::FIROpsDialect", "mlir::func::FuncDialect"
]; ];
} }
def MemoryAllocationOpt : Pass<"memory-allocation-opt", "mlir::FuncOp"> { def MemoryAllocationOpt : Pass<"memory-allocation-opt", "mlir::FuncOp"> {
let summary = "Convert stack to heap allocations and vice versa."; let summary = "Convert stack to heap allocations and vice versa.";
let description = [{ let description = [{
Convert stack allocations to heap allocations and vice versa based on Convert stack allocations to heap allocations and vice versa based on
estimated size, lifetime, usage patterns, the call tree, etc. estimated size, lifetime, usage patterns, the call tree, etc.
Show All 14 Lines

flang/lib/Lower/Bridge.cpp

Show First 20 Lines • Show All 421 Lines • ▼ Show 20 Linesprivate:
// Termination of symbolically referenced execution units // Termination of symbolically referenced execution units
//===--------------------------------------------------------------------===// //===--------------------------------------------------------------------===//
/// END of program /// END of program
/// ///
/// Generate the cleanup block before the program exits /// Generate the cleanup block before the program exits
void genExitRoutine() { void genExitRoutine() {
if (blockIsUnterminated()) if (blockIsUnterminated())
builder->create<mlir::ReturnOp>(toLocation()); builder->create<mlir::func::ReturnOp>(toLocation());
} }
void genFIR(const Fortran::parser::EndProgramStmt &) { genExitRoutine(); } void genFIR(const Fortran::parser::EndProgramStmt &) { genExitRoutine(); }
/// END of procedure-like constructs /// END of procedure-like constructs
/// ///
/// Generate the cleanup block before the procedure exits /// Generate the cleanup block before the procedure exits
void genReturnSymbol(const Fortran::semantics::Symbol &functionSymbol) { void genReturnSymbol(const Fortran::semantics::Symbol &functionSymbol) {
const Fortran::semantics::Symbol &resultSym = const Fortran::semantics::Symbol &resultSym =
Show All 15 Lines mlir::Value resultVal = resultSymBox.match(
mlir::Type resultRefType = builder->getRefType(resultType); mlir::Type resultRefType = builder->getRefType(resultType);
// A function with multiple entry points returning different types // A function with multiple entry points returning different types
// tags all result variables with one of the largest types to allow // tags all result variables with one of the largest types to allow
// them to share the same storage. Convert this to the actual type. // them to share the same storage. Convert this to the actual type.
if (resultRef.getType() != resultRefType) if (resultRef.getType() != resultRefType)
TODO(loc, "Convert to actual type"); TODO(loc, "Convert to actual type");
return builder->create<fir::LoadOp>(loc, resultRef); return builder->create<fir::LoadOp>(loc, resultRef);
}); });
builder->create<mlir::ReturnOp>(loc, resultVal); builder->create<mlir::func::ReturnOp>(loc, resultVal);
} }
void genFIRProcedureExit(Fortran::lower::pft::FunctionLikeUnit &funit, void genFIRProcedureExit(Fortran::lower::pft::FunctionLikeUnit &funit,
const Fortran::semantics::Symbol &symbol) { const Fortran::semantics::Symbol &symbol) {
if (mlir::Block *finalBlock = funit.finalBlock) { if (mlir::Block *finalBlock = funit.finalBlock) {
// The current block must end with a terminator. // The current block must end with a terminator.
if (blockIsUnterminated()) if (blockIsUnterminated())
builder->create<mlir::cf::BranchOp>(toLocation(), finalBlock); builder->create<mlir::cf::BranchOp>(toLocation(), finalBlock);
▲ Show 20 Lines • Show All 662 Lines • Show Last 20 Lines

flang/lib/Lower/CMakeLists.txt

Show All 31 Linesadd_flang_library(FortranLower
FIRSupport FIRSupport
FIRTransforms FIRTransforms
${dialect_libs} ${dialect_libs}
FortranCommon FortranCommon
FortranParser FortranParser
FortranEvaluate FortranEvaluate
FortranSemantics FortranSemantics
MLIRAffineToStandard MLIRAffineToStandard
MLIRFunc
MLIRLLVMIR MLIRLLVMIR
MLIRSCFToControlFlow MLIRSCFToControlFlow
MLIRStandard
LINK_COMPONENTS LINK_COMPONENTS
Support Support
) )

flang/lib/Lower/ConvertExpr.cpp

Show All 25 Lines
#include "flang/Optimizer/Builder/Complex.h" #include "flang/Optimizer/Builder/Complex.h"
#include "flang/Optimizer/Builder/Factory.h" #include "flang/Optimizer/Builder/Factory.h"
#include "flang/Optimizer/Builder/MutableBox.h" #include "flang/Optimizer/Builder/MutableBox.h"
#include "flang/Optimizer/Dialect/FIROpsSupport.h" #include "flang/Optimizer/Dialect/FIROpsSupport.h"
#include "flang/Semantics/expression.h" #include "flang/Semantics/expression.h"
#include "flang/Semantics/symbol.h" #include "flang/Semantics/symbol.h"
#include "flang/Semantics/tools.h" #include "flang/Semantics/tools.h"
#include "flang/Semantics/type.h" #include "flang/Semantics/type.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#define DEBUG_TYPE "flang-lower-expr" #define DEBUG_TYPE "flang-lower-expr"
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// The composition and structure of Fortran::evaluate::Expr is defined in // The composition and structure of Fortran::evaluate::Expr is defined in
// the various header files in include/flang/Evaluate. You are referred // the various header files in include/flang/Evaluate. You are referred
// there for more information on these data structures. Generally speaking, // there for more information on these data structures. Generally speaking,
▲ Show 20 Lines • Show All 2,677 Lines • Show Last 20 Lines

flang/lib/Optimizer/Builder/Runtime/Character.cpp

//===-- Character.cpp -- runtime for CHARACTER type entities --------------===// //===-- Character.cpp -- runtime for CHARACTER type entities --------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "flang/Optimizer/Builder/Runtime/Character.h" #include "flang/Optimizer/Builder/Runtime/Character.h"
#include "flang/Lower/Todo.h" #include "flang/Lower/Todo.h"
#include "flang/Optimizer/Builder/BoxValue.h" #include "flang/Optimizer/Builder/BoxValue.h"
#include "flang/Optimizer/Builder/Character.h" #include "flang/Optimizer/Builder/Character.h"
#include "flang/Optimizer/Builder/FIRBuilder.h" #include "flang/Optimizer/Builder/FIRBuilder.h"
#include "flang/Optimizer/Builder/Runtime/RTBuilder.h" #include "flang/Optimizer/Builder/Runtime/RTBuilder.h"
#include "flang/Runtime/character.h" #include "flang/Runtime/character.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
using namespace Fortran::runtime; using namespace Fortran::runtime;
/// Generate calls to string handling intrinsics such as index, scan, and /// Generate calls to string handling intrinsics such as index, scan, and
/// verify. These are the descriptor based implementations that take four /// verify. These are the descriptor based implementations that take four
/// arguments (string1, string2, back, kind). /// arguments (string1, string2, back, kind).
template <typename FN> template <typename FN>
static void genCharacterSearch(FN func, fir::FirOpBuilder &builder, static void genCharacterSearch(FN func, fir::FirOpBuilder &builder,
▲ Show 20 Lines • Show All 254 Lines • Show Last 20 Lines

flang/lib/Optimizer/Builder/Runtime/Numeric.cpp

//===-- Numeric.cpp -- runtime API for numeric intrinsics -----------------===// //===-- Numeric.cpp -- runtime API for numeric intrinsics -----------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "flang/Optimizer/Builder/Runtime/Numeric.h" #include "flang/Optimizer/Builder/Runtime/Numeric.h"
#include "flang/Optimizer/Builder/BoxValue.h" #include "flang/Optimizer/Builder/BoxValue.h"
#include "flang/Optimizer/Builder/Character.h" #include "flang/Optimizer/Builder/Character.h"
#include "flang/Optimizer/Builder/FIRBuilder.h" #include "flang/Optimizer/Builder/FIRBuilder.h"
#include "flang/Optimizer/Builder/Runtime/RTBuilder.h" #include "flang/Optimizer/Builder/Runtime/RTBuilder.h"
#include "flang/Runtime/numeric.h" #include "flang/Runtime/numeric.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
using namespace Fortran::runtime; using namespace Fortran::runtime;
// The real*10 and real*16 placeholders below are used to force the // The real*10 and real*16 placeholders below are used to force the
// compilation of the real*10 and real*16 method names on systems that // compilation of the real*10 and real*16 method names on systems that
// may not have them in their runtime library. This can occur in the // may not have them in their runtime library. This can occur in the
// case of cross compilation, for example. // case of cross compilation, for example.
▲ Show 20 Lines • Show All 376 Lines • Show Last 20 Lines

flang/lib/Optimizer/Builder/Runtime/Reduction.cpp

//===-- Reduction.cpp -- generate reduction intrinsics runtime calls- -----===// //===-- Reduction.cpp -- generate reduction intrinsics runtime calls- -----===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "flang/Optimizer/Builder/Runtime/Reduction.h" #include "flang/Optimizer/Builder/Runtime/Reduction.h"
#include "flang/Lower/Todo.h" #include "flang/Lower/Todo.h"
#include "flang/Optimizer/Builder/BoxValue.h" #include "flang/Optimizer/Builder/BoxValue.h"
#include "flang/Optimizer/Builder/Character.h" #include "flang/Optimizer/Builder/Character.h"
#include "flang/Optimizer/Builder/FIRBuilder.h" #include "flang/Optimizer/Builder/FIRBuilder.h"
#include "flang/Optimizer/Builder/Runtime/RTBuilder.h" #include "flang/Optimizer/Builder/Runtime/RTBuilder.h"
#include "flang/Runtime/reduction.h" #include "flang/Runtime/reduction.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
using namespace Fortran::runtime; using namespace Fortran::runtime;
/// Placeholder for real*10 version of Maxval Intrinsic /// Placeholder for real*10 version of Maxval Intrinsic
struct ForcedMaxvalReal10 { struct ForcedMaxvalReal10 {
static constexpr const char *name = ExpandAndQuoteKey(RTNAME(MaxvalReal10)); static constexpr const char *name = ExpandAndQuoteKey(RTNAME(MaxvalReal10));
static constexpr fir::runtime::FuncTypeBuilderFunc getTypeModel() { static constexpr fir::runtime::FuncTypeBuilderFunc getTypeModel() {
return [](mlir::MLIRContext *ctx) { return [](mlir::MLIRContext *ctx) {
▲ Show 20 Lines • Show All 904 Lines • Show Last 20 Lines

flang/lib/Optimizer/Builder/Runtime/Transformational.cpp

Show All 9 Lines
#include "flang/Optimizer/Builder/Runtime/Transformational.h" #include "flang/Optimizer/Builder/Runtime/Transformational.h"
#include "flang/Lower/Todo.h" #include "flang/Lower/Todo.h"
#include "flang/Optimizer/Builder/BoxValue.h" #include "flang/Optimizer/Builder/BoxValue.h"
#include "flang/Optimizer/Builder/Character.h" #include "flang/Optimizer/Builder/Character.h"
#include "flang/Optimizer/Builder/FIRBuilder.h" #include "flang/Optimizer/Builder/FIRBuilder.h"
#include "flang/Optimizer/Builder/Runtime/RTBuilder.h" #include "flang/Optimizer/Builder/Runtime/RTBuilder.h"
#include "flang/Runtime/matmul.h" #include "flang/Runtime/matmul.h"
#include "flang/Runtime/transformational.h" #include "flang/Runtime/transformational.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
using namespace Fortran::runtime; using namespace Fortran::runtime;
/// Generate call to Cshift intrinsic /// Generate call to Cshift intrinsic
void fir::runtime::genCshift(fir::FirOpBuilder &builder, mlir::Location loc, void fir::runtime::genCshift(fir::FirOpBuilder &builder, mlir::Location loc,
mlir::Value resultBox, mlir::Value arrayBox, mlir::Value resultBox, mlir::Value arrayBox,
mlir::Value shiftBox, mlir::Value dimBox) { mlir::Value shiftBox, mlir::Value dimBox) {
auto cshiftFunc = fir::runtime::getRuntimeFunc<mkRTKey(Cshift)>(loc, builder); auto cshiftFunc = fir::runtime::getRuntimeFunc<mkRTKey(Cshift)>(loc, builder);
▲ Show 20 Lines • Show All 150 Lines • Show Last 20 Lines

flang/lib/Optimizer/CodeGen/CGOps.h

//===-- CGOps.h -------------------------------------------------*- C++ -*-===// //===-- CGOps.h -------------------------------------------------*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// Coding style: https://mlir.llvm.org/getting_started/DeveloperGuide/ // Coding style: https://mlir.llvm.org/getting_started/DeveloperGuide/
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef OPTIMIZER_CODEGEN_CGOPS_H #ifndef OPTIMIZER_CODEGEN_CGOPS_H
#define OPTIMIZER_CODEGEN_CGOPS_H #define OPTIMIZER_CODEGEN_CGOPS_H
#include "flang/Optimizer/Dialect/FIRType.h" #include "flang/Optimizer/Dialect/FIRType.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
using namespace mlir; using namespace mlir;
#define GET_OP_CLASSES #define GET_OP_CLASSES
#include "flang/Optimizer/CodeGen/CGOps.h.inc" #include "flang/Optimizer/CodeGen/CGOps.h.inc"
#endif #endif

flang/lib/Optimizer/CodeGen/PreCGRewrite.cpp

Show First 20 Lines • Show All 256 Lines • ▼ Show 20 Lines
class CodeGenRewrite : public CodeGenRewriteBase<CodeGenRewrite> { class CodeGenRewrite : public CodeGenRewriteBase<CodeGenRewrite> {
public: public:
void runOnOperation() override final { void runOnOperation() override final {
auto op = getOperation(); auto op = getOperation();
auto &context = getContext(); auto &context = getContext();
mlir::OpBuilder rewriter(&context); mlir::OpBuilder rewriter(&context);
mlir::ConversionTarget target(context); mlir::ConversionTarget target(context);
target.addLegalDialect<mlir::arith::ArithmeticDialect, FIROpsDialect, target.addLegalDialect<mlir::arith::ArithmeticDialect, FIROpsDialect,
FIRCodeGenDialect, mlir::StandardOpsDialect>(); FIRCodeGenDialect, mlir::func::FuncDialect>();
target.addIllegalOp<ArrayCoorOp>(); target.addIllegalOp<ArrayCoorOp>();
target.addIllegalOp<ReboxOp>(); target.addIllegalOp<ReboxOp>();
target.addDynamicallyLegalOp<EmboxOp>([](EmboxOp embox) { target.addDynamicallyLegalOp<EmboxOp>([](EmboxOp embox) {
return !(embox.getShape() || return !(embox.getShape() ||
embox.getType().cast<BoxType>().getEleTy().isa<SequenceType>()); embox.getType().cast<BoxType>().getEleTy().isa<SequenceType>());
}); });
mlir::RewritePatternSet patterns(&context); mlir::RewritePatternSet patterns(&context);
patterns.insert<EmboxConversion, ArrayCoorConversion, ReboxConversion>( patterns.insert<EmboxConversion, ArrayCoorConversion, ReboxConversion>(
Show All 15 Lines

flang/lib/Optimizer/CodeGen/TargetRewrite.cpp

Show First 20 Lines • Show All 593 Lines • ▼ Show 20 Lines if (!func.empty()) {
} }
} break; } break;
case FixupTy::Codes::ReturnAsStore: { case FixupTy::Codes::ReturnAsStore: {
// The value being returned is now being returned in memory (callee // The value being returned is now being returned in memory (callee
// stack space) through a hidden reference argument. // stack space) through a hidden reference argument.
auto newArg = func.front().insertArgument(fixup.index, auto newArg = func.front().insertArgument(fixup.index,
newInTys[fixup.index], loc); newInTys[fixup.index], loc);
offset++; offset++;
func.walk([&](mlir::ReturnOp ret) { func.walk([&](mlir::func::ReturnOp ret) {
rewriter->setInsertionPoint(ret); rewriter->setInsertionPoint(ret);
auto oldOper = ret.getOperand(0); auto oldOper = ret.getOperand(0);
auto oldOperTy = ReferenceType::get(oldOper.getType()); auto oldOperTy = ReferenceType::get(oldOper.getType());
auto cast = rewriter->create<ConvertOp>(loc, oldOperTy, newArg); auto cast = rewriter->create<ConvertOp>(loc, oldOperTy, newArg);
rewriter->create<fir::StoreOp>(loc, oldOper, cast); rewriter->create<fir::StoreOp>(loc, oldOper, cast);
rewriter->create<mlir::ReturnOp>(loc); rewriter->create<mlir::func::ReturnOp>(loc);
ret.erase(); ret.erase();
}); });
} break; } break;
case FixupTy::Codes::ReturnType: { case FixupTy::Codes::ReturnType: {
// The function is still returning a value, but its type has likely // The function is still returning a value, but its type has likely
// changed to suit the target ABI convention. // changed to suit the target ABI convention.
func.walk([&](mlir::ReturnOp ret) { func.walk([&](mlir::func::ReturnOp ret) {
rewriter->setInsertionPoint(ret); rewriter->setInsertionPoint(ret);
auto oldOper = ret.getOperand(0); auto oldOper = ret.getOperand(0);
auto oldOperTy = ReferenceType::get(oldOper.getType()); auto oldOperTy = ReferenceType::get(oldOper.getType());
auto mem = auto mem =
rewriter->create<fir::AllocaOp>(loc, newResTys[fixup.index]); rewriter->create<fir::AllocaOp>(loc, newResTys[fixup.index]);
auto cast = rewriter->create<ConvertOp>(loc, oldOperTy, mem); auto cast = rewriter->create<ConvertOp>(loc, oldOperTy, mem);
rewriter->create<fir::StoreOp>(loc, oldOper, cast); rewriter->create<fir::StoreOp>(loc, oldOper, cast);
mlir::Value load = rewriter->create<fir::LoadOp>(loc, mem); mlir::Value load = rewriter->create<fir::LoadOp>(loc, mem);
rewriter->create<mlir::ReturnOp>(loc, load); rewriter->create<mlir::func::ReturnOp>(loc, load);
ret.erase(); ret.erase();
}); });
} break; } break;
case FixupTy::Codes::Split: { case FixupTy::Codes::Split: {
// The FIR argument has been split into a pair of distinct arguments // The FIR argument has been split into a pair of distinct arguments
// that are in juxtaposition to each other. (For COMPLEX value.) // that are in juxtaposition to each other. (For COMPLEX value.)
auto newArg = func.front().insertArgument(fixup.index, auto newArg = func.front().insertArgument(fixup.index,
newInTys[fixup.index], loc); newInTys[fixup.index], loc);
▲ Show 20 Lines • Show All 171 Lines • Show Last 20 Lines

flang/lib/Optimizer/Dialect/FIRDialect.cpp

Show All 36 Linesstruct FIRInlinerInterface : public mlir::DialectInlinerInterface {
} }
/// This hook is called when a terminator operation has been inlined. /// This hook is called when a terminator operation has been inlined.
/// We handle the return (a Fortran FUNCTION) by replacing the values /// We handle the return (a Fortran FUNCTION) by replacing the values
/// previously returned by the call operation with the operands of the /// previously returned by the call operation with the operands of the
/// return. /// return.
void handleTerminator(mlir::Operation *op, void handleTerminator(mlir::Operation *op,
llvm::ArrayRef<mlir::Value> valuesToRepl) const final { llvm::ArrayRef<mlir::Value> valuesToRepl) const final {
auto returnOp = cast<mlir::ReturnOp>(op); auto returnOp = cast<mlir::func::ReturnOp>(op);
assert(returnOp.getNumOperands() == valuesToRepl.size()); assert(returnOp.getNumOperands() == valuesToRepl.size());
for (const auto &it : llvm::enumerate(returnOp.getOperands())) for (const auto &it : llvm::enumerate(returnOp.getOperands()))
valuesToRepl[it.index()].replaceAllUsesWith(it.value()); valuesToRepl[it.index()].replaceAllUsesWith(it.value());
} }
mlir::Operation *materializeCallConversion(mlir::OpBuilder &builder, mlir::Operation *materializeCallConversion(mlir::OpBuilder &builder,
mlir::Value input, mlir::Value input,
mlir::Type resultType, mlir::Type resultType,
▲ Show 20 Lines • Show All 41 Lines • Show Last 20 Lines

flang/lib/Optimizer/Dialect/FIROps.cpp

Show All 10 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "flang/Optimizer/Dialect/FIROps.h" #include "flang/Optimizer/Dialect/FIROps.h"
#include "flang/Optimizer/Dialect/FIRAttr.h" #include "flang/Optimizer/Dialect/FIRAttr.h"
#include "flang/Optimizer/Dialect/FIROpsSupport.h" #include "flang/Optimizer/Dialect/FIROpsSupport.h"
#include "flang/Optimizer/Dialect/FIRType.h" #include "flang/Optimizer/Dialect/FIRType.h"
#include "flang/Optimizer/Support/Utils.h" #include "flang/Optimizer/Support/Utils.h"
#include "mlir/Dialect/CommonFolders.h" #include "mlir/Dialect/CommonFolders.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/BuiltinAttributes.h" #include "mlir/IR/BuiltinAttributes.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/Diagnostics.h" #include "mlir/IR/Diagnostics.h"
#include "mlir/IR/Matchers.h" #include "mlir/IR/Matchers.h"
#include "mlir/IR/OpDefinition.h" #include "mlir/IR/OpDefinition.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "llvm/ADT/STLExtras.h" #include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/SmallVector.h" #include "llvm/ADT/SmallVector.h"
▲ Show 20 Lines • Show All 3,318 Lines • Show Last 20 Lines

flang/lib/Optimizer/Transforms/AbstractResult.cpp

//===- AbstractResult.cpp - Conversion of Abstract Function Result --------===// //===- AbstractResult.cpp - Conversion of Abstract Function Result --------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "flang/Optimizer/Dialect/FIRDialect.h" #include "flang/Optimizer/Dialect/FIRDialect.h"
#include "flang/Optimizer/Dialect/FIROps.h" #include "flang/Optimizer/Dialect/FIROps.h"
#include "flang/Optimizer/Dialect/FIRType.h" #include "flang/Optimizer/Dialect/FIRType.h"
#include "flang/Optimizer/Transforms/Passes.h" #include "flang/Optimizer/Transforms/Passes.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/Diagnostics.h" #include "mlir/IR/Diagnostics.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
#include "llvm/ADT/TypeSwitch.h" #include "llvm/ADT/TypeSwitch.h"
#define DEBUG_TYPE "flang-abstract-result-opt" #define DEBUG_TYPE "flang-abstract-result-opt"
▲ Show 20 Lines • Show All 121 Lines • ▼ Show 20 Linespublic:
mlir::LogicalResult mlir::LogicalResult
matchAndRewrite(fir::SaveResultOp op, matchAndRewrite(fir::SaveResultOp op,
mlir::PatternRewriter &rewriter) const override { mlir::PatternRewriter &rewriter) const override {
rewriter.eraseOp(op); rewriter.eraseOp(op);
return mlir::success(); return mlir::success();
} }
}; };
class ReturnOpConversion : public mlir::OpRewritePattern<mlir::ReturnOp> { class ReturnOpConversion : public mlir::OpRewritePattern<mlir::func::ReturnOp> {
public: public:
using OpRewritePattern::OpRewritePattern; using OpRewritePattern::OpRewritePattern;
ReturnOpConversion(mlir::MLIRContext *context, ReturnOpConversion(mlir::MLIRContext *context,
const AbstractResultOptions &opt) const AbstractResultOptions &opt)
: OpRewritePattern(context), options{opt} {} : OpRewritePattern(context), options{opt} {}
mlir::LogicalResult mlir::LogicalResult
matchAndRewrite(mlir::ReturnOp ret, matchAndRewrite(mlir::func::ReturnOp ret,
mlir::PatternRewriter &rewriter) const override { mlir::PatternRewriter &rewriter) const override {
rewriter.setInsertionPoint(ret); rewriter.setInsertionPoint(ret);
auto returnedValue = ret.getOperand(0); auto returnedValue = ret.getOperand(0);
bool replacedStorage = false; bool replacedStorage = false;
if (auto *op = returnedValue.getDefiningOp()) if (auto *op = returnedValue.getDefiningOp())
if (auto load = mlir::dyn_cast<fir::LoadOp>(op)) { if (auto load = mlir::dyn_cast<fir::LoadOp>(op)) {
auto resultStorage = load.getMemref(); auto resultStorage = load.getMemref();
load.getMemref().replaceAllUsesWith(options.newArg); load.getMemref().replaceAllUsesWith(options.newArg);
replacedStorage = true; replacedStorage = true;
if (auto *alloc = resultStorage.getDefiningOp()) if (auto *alloc = resultStorage.getDefiningOp())
if (alloc->use_empty()) if (alloc->use_empty())
rewriter.eraseOp(alloc); rewriter.eraseOp(alloc);
} }
// The result storage may have been optimized out by a memory to // The result storage may have been optimized out by a memory to
// register pass, this is possible for fir.box results, or fir.record // register pass, this is possible for fir.box results, or fir.record
// with no length parameters. Simply store the result in the result storage. // with no length parameters. Simply store the result in the result storage.
// at the return point. // at the return point.
if (!replacedStorage) if (!replacedStorage)
rewriter.create<fir::StoreOp>(ret.getLoc(), returnedValue, rewriter.create<fir::StoreOp>(ret.getLoc(), returnedValue,
options.newArg); options.newArg);
rewriter.replaceOpWithNewOp<mlir::ReturnOp>(ret); rewriter.replaceOpWithNewOp<mlir::func::ReturnOp>(ret);
return mlir::success(); return mlir::success();
} }
private: private:
const AbstractResultOptions &options; const AbstractResultOptions &options;
}; };
class AddrOfOpConversion : public mlir::OpRewritePattern<fir::AddrOfOp> { class AddrOfOpConversion : public mlir::OpRewritePattern<fir::AddrOfOp> {
▲ Show 20 Lines • Show All 45 Lines • ▼ Show 20 Lines if (mustConvertCallOrFunc(funcTy)) {
options.newArg = func.front().insertArgument(zero, argTy, loc); options.newArg = func.front().insertArgument(zero, argTy, loc);
if (mustEmboxResult(resultType, options)) { if (mustEmboxResult(resultType, options)) {
auto bufferType = fir::ReferenceType::get(resultType); auto bufferType = fir::ReferenceType::get(resultType);
rewriter.setInsertionPointToStart(&func.front()); rewriter.setInsertionPointToStart(&func.front());
options.newArg = options.newArg =
rewriter.create<fir::BoxAddrOp>(loc, bufferType, options.newArg); rewriter.create<fir::BoxAddrOp>(loc, bufferType, options.newArg);
} }
patterns.insert<ReturnOpConversion>(context, options); patterns.insert<ReturnOpConversion>(context, options);
target.addDynamicallyLegalOp<mlir::ReturnOp>( target.addDynamicallyLegalOp<mlir::func::ReturnOp>(
[](mlir::ReturnOp ret) { return ret.operands().empty(); }); [](mlir::func::ReturnOp ret) { return ret.operands().empty(); });
} }
} }
if (func.empty()) if (func.empty())
return; return;
// Convert the calls and, if needed, the ReturnOp in the function body. // Convert the calls and, if needed, the ReturnOp in the function body.
target.addLegalDialect<fir::FIROpsDialect, mlir::arith::ArithmeticDialect, target.addLegalDialect<fir::FIROpsDialect, mlir::arith::ArithmeticDialect,
mlir::StandardOpsDialect>(); mlir::func::FuncDialect>();
target.addIllegalOp<fir::SaveResultOp>(); target.addIllegalOp<fir::SaveResultOp>();
target.addDynamicallyLegalOp<fir::CallOp>([](fir::CallOp call) { target.addDynamicallyLegalOp<fir::CallOp>([](fir::CallOp call) {
return !mustConvertCallOrFunc(call.getFunctionType()); return !mustConvertCallOrFunc(call.getFunctionType());
}); });
target.addDynamicallyLegalOp<fir::AddrOfOp>([](fir::AddrOfOp addrOf) { target.addDynamicallyLegalOp<fir::AddrOfOp>([](fir::AddrOfOp addrOf) {
if (auto funTy = addrOf.getType().dyn_cast<mlir::FunctionType>()) if (auto funTy = addrOf.getType().dyn_cast<mlir::FunctionType>())
return !mustConvertCallOrFunc(funTy); return !mustConvertCallOrFunc(funTy);
return true; return true;
Show All 29 Lines

flang/lib/Optimizer/Transforms/AffineDemotion.cpp

Show All 17 Lines
#include "PassDetail.h" #include "PassDetail.h"
#include "flang/Optimizer/Dialect/FIRDialect.h" #include "flang/Optimizer/Dialect/FIRDialect.h"
#include "flang/Optimizer/Dialect/FIROps.h" #include "flang/Optimizer/Dialect/FIROps.h"
#include "flang/Optimizer/Dialect/FIRType.h" #include "flang/Optimizer/Dialect/FIRType.h"
#include "flang/Optimizer/Transforms/Passes.h" #include "flang/Optimizer/Transforms/Passes.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Affine/Utils.h" #include "mlir/Dialect/Affine/Utils.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BuiltinAttributes.h" #include "mlir/IR/BuiltinAttributes.h"
#include "mlir/IR/IntegerSet.h" #include "mlir/IR/IntegerSet.h"
#include "mlir/IR/Visitors.h" #include "mlir/IR/Visitors.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/ADT/DenseMap.h" #include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/Optional.h" #include "llvm/ADT/Optional.h"
#include "llvm/Support/CommandLine.h" #include "llvm/Support/CommandLine.h"
▲ Show 20 Lines • Show All 116 Lines • ▼ Show 20 Lines void runOnOperation() override {
target.addIllegalOp<memref::AllocOp>(); target.addIllegalOp<memref::AllocOp>();
target.addDynamicallyLegalOp<fir::ConvertOp>([](fir::ConvertOp op) { target.addDynamicallyLegalOp<fir::ConvertOp>([](fir::ConvertOp op) {
if (op.getRes().getType().isa<mlir::MemRefType>()) if (op.getRes().getType().isa<mlir::MemRefType>())
return false; return false;
return true; return true;
}); });
target.addLegalDialect<FIROpsDialect, mlir::scf::SCFDialect, target.addLegalDialect<FIROpsDialect, mlir::scf::SCFDialect,
mlir::arith::ArithmeticDialect, mlir::arith::ArithmeticDialect,
mlir::StandardOpsDialect>(); mlir::func::FuncDialect>();
if (mlir::failed(mlir::applyPartialConversion(function, target, if (mlir::failed(mlir::applyPartialConversion(function, target,
std::move(patterns)))) { std::move(patterns)))) {
mlir::emitError(mlir::UnknownLoc::get(context), mlir::emitError(mlir::UnknownLoc::get(context),
"error in converting affine dialect\n"); "error in converting affine dialect\n");
signalPassFailure(); signalPassFailure();
} }
} }
}; };
} // namespace } // namespace
std::unique_ptr<mlir::Pass> fir::createAffineDemotionPass() { std::unique_ptr<mlir::Pass> fir::createAffineDemotionPass() {
return std::make_unique<AffineDialectDemotion>(); return std::make_unique<AffineDialectDemotion>();
} }

flang/lib/Optimizer/Transforms/AffinePromotion.cpp

Show All 15 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "flang/Optimizer/Dialect/FIRDialect.h" #include "flang/Optimizer/Dialect/FIRDialect.h"
#include "flang/Optimizer/Dialect/FIROps.h" #include "flang/Optimizer/Dialect/FIROps.h"
#include "flang/Optimizer/Dialect/FIRType.h" #include "flang/Optimizer/Dialect/FIRType.h"
#include "flang/Optimizer/Transforms/Passes.h" #include "flang/Optimizer/Transforms/Passes.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BuiltinAttributes.h" #include "mlir/IR/BuiltinAttributes.h"
#include "mlir/IR/IntegerSet.h" #include "mlir/IR/IntegerSet.h"
#include "mlir/IR/Visitors.h" #include "mlir/IR/Visitors.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/ADT/DenseMap.h" #include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/Optional.h" #include "llvm/ADT/Optional.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
▲ Show 20 Lines • Show All 556 Lines • ▼ Show 20 Lines void runOnOperation() override {
markAllAnalysesPreserved(); markAllAnalysesPreserved();
auto functionAnalysis = AffineFunctionAnalysis(function); auto functionAnalysis = AffineFunctionAnalysis(function);
mlir::RewritePatternSet patterns(context); mlir::RewritePatternSet patterns(context);
patterns.insert<AffineIfConversion>(context, functionAnalysis); patterns.insert<AffineIfConversion>(context, functionAnalysis);
patterns.insert<AffineLoopConversion>(context, functionAnalysis); patterns.insert<AffineLoopConversion>(context, functionAnalysis);
mlir::ConversionTarget target = *context; mlir::ConversionTarget target = *context;
target.addLegalDialect< target.addLegalDialect<
mlir::AffineDialect, FIROpsDialect, mlir::scf::SCFDialect, mlir::AffineDialect, FIROpsDialect, mlir::scf::SCFDialect,
mlir::arith::ArithmeticDialect, mlir::StandardOpsDialect>(); mlir::arith::ArithmeticDialect, mlir::func::FuncDialect>();
target.addDynamicallyLegalOp<IfOp>([&functionAnalysis](fir::IfOp op) { target.addDynamicallyLegalOp<IfOp>([&functionAnalysis](fir::IfOp op) {
return !(functionAnalysis.getChildIfAnalysis(op).canPromoteToAffine()); return !(functionAnalysis.getChildIfAnalysis(op).canPromoteToAffine());
}); });
target.addDynamicallyLegalOp<DoLoopOp>([&functionAnalysis]( target.addDynamicallyLegalOp<DoLoopOp>([&functionAnalysis](
fir::DoLoopOp op) { fir::DoLoopOp op) {
return !(functionAnalysis.getChildLoopAnalysis(op).canPromoteToAffine()); return !(functionAnalysis.getChildLoopAnalysis(op).canPromoteToAffine());
}); });
Show All 18 Lines

flang/lib/Optimizer/Transforms/ArrayValueCopy.cpp

Show First 20 Lines • Show All 786 Lines • ▼ Show 20 Lines void runOnOperation() override {
// /copy-out refers the Fortran copy-in/copy-out semantics on statements. // /copy-out refers the Fortran copy-in/copy-out semantics on statements.
mlir::RewritePatternSet patterns1(context); mlir::RewritePatternSet patterns1(context);
patterns1.insert<ArrayFetchConversion>(context, useMap); patterns1.insert<ArrayFetchConversion>(context, useMap);
patterns1.insert<ArrayUpdateConversion>(context, analysis, useMap); patterns1.insert<ArrayUpdateConversion>(context, analysis, useMap);
patterns1.insert<ArrayModifyConversion>(context, analysis, useMap); patterns1.insert<ArrayModifyConversion>(context, analysis, useMap);
mlir::ConversionTarget target(*context); mlir::ConversionTarget target(*context);
target.addLegalDialect< target.addLegalDialect<
FIROpsDialect, mlir::scf::SCFDialect, mlir::arith::ArithmeticDialect, FIROpsDialect, mlir::scf::SCFDialect, mlir::arith::ArithmeticDialect,
mlir::cf::ControlFlowDialect, mlir::StandardOpsDialect>(); mlir::cf::ControlFlowDialect, mlir::func::FuncDialect>();
target.addIllegalOp<ArrayFetchOp, ArrayUpdateOp, ArrayModifyOp>(); target.addIllegalOp<ArrayFetchOp, ArrayUpdateOp, ArrayModifyOp>();
// Rewrite the array fetch and array update ops. // Rewrite the array fetch and array update ops.
if (mlir::failed( if (mlir::failed(
mlir::applyPartialConversion(func, target, std::move(patterns1)))) { mlir::applyPartialConversion(func, target, std::move(patterns1)))) {
mlir::emitError(mlir::UnknownLoc::get(context), mlir::emitError(mlir::UnknownLoc::get(context),
"failure in array-value-copy pass, phase 1"); "failure in array-value-copy pass, phase 1");
signalPassFailure(); signalPassFailure();
} }
Show All 18 Lines

flang/lib/Optimizer/Transforms/CharacterConversion.cpp

//===- CharacterConversion.cpp -- convert between character encodings -----===// //===- CharacterConversion.cpp -- convert between character encodings -----===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "flang/Optimizer/Dialect/FIRDialect.h" #include "flang/Optimizer/Dialect/FIRDialect.h"
#include "flang/Optimizer/Dialect/FIROps.h" #include "flang/Optimizer/Dialect/FIROps.h"
#include "flang/Optimizer/Dialect/FIRType.h" #include "flang/Optimizer/Dialect/FIRType.h"
#include "flang/Optimizer/Support/FIRContext.h" #include "flang/Optimizer/Support/FIRContext.h"
#include "flang/Optimizer/Support/KindMapping.h" #include "flang/Optimizer/Support/KindMapping.h"
#include "flang/Optimizer/Transforms/Passes.h" #include "flang/Optimizer/Transforms/Passes.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/Diagnostics.h" #include "mlir/IR/Diagnostics.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#define DEBUG_TYPE "flang-character-conversion" #define DEBUG_TYPE "flang-character-conversion"
namespace { namespace {
▲ Show 20 Lines • Show All 77 Lines • ▼ Show 20 Lines void runOnOperation() override {
if (clOpts.runtimeName.empty()) { if (clOpts.runtimeName.empty()) {
auto *context = &getContext(); auto *context = &getContext();
auto *func = getOperation(); auto *func = getOperation();
mlir::RewritePatternSet patterns(context); mlir::RewritePatternSet patterns(context);
patterns.insert<CharacterConvertConversion>(context); patterns.insert<CharacterConvertConversion>(context);
mlir::ConversionTarget target(*context); mlir::ConversionTarget target(*context);
target.addLegalDialect<mlir::AffineDialect, fir::FIROpsDialect, target.addLegalDialect<mlir::AffineDialect, fir::FIROpsDialect,
mlir::arith::ArithmeticDialect, mlir::arith::ArithmeticDialect,
mlir::StandardOpsDialect>(); mlir::func::FuncDialect>();
// apply the patterns // apply the patterns
target.addIllegalOp<fir::CharConvertOp>(); target.addIllegalOp<fir::CharConvertOp>();
if (mlir::failed(mlir::applyPartialConversion(func, target, if (mlir::failed(mlir::applyPartialConversion(func, target,
std::move(patterns)))) { std::move(patterns)))) {
mlir::emitError(mlir::UnknownLoc::get(context), mlir::emitError(mlir::UnknownLoc::get(context),
"error in rewriting character convert op"); "error in rewriting character convert op");
signalPassFailure(); signalPassFailure();
Show All 13 Lines

flang/lib/Optimizer/Transforms/MemRefDataFlowOpt.cpp

//===- MemRefDataFlowOpt.cpp - Memory DataFlow Optimization pass ----------===// //===- MemRefDataFlowOpt.cpp - Memory DataFlow Optimization pass ----------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "flang/Optimizer/Dialect/FIRDialect.h" #include "flang/Optimizer/Dialect/FIRDialect.h"
#include "flang/Optimizer/Dialect/FIROps.h" #include "flang/Optimizer/Dialect/FIROps.h"
#include "flang/Optimizer/Dialect/FIRType.h" #include "flang/Optimizer/Dialect/FIRType.h"
#include "flang/Optimizer/Transforms/Passes.h" #include "flang/Optimizer/Transforms/Passes.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/Dominance.h" #include "mlir/IR/Dominance.h"
#include "mlir/IR/Operation.h" #include "mlir/IR/Operation.h"
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
#include "llvm/ADT/Optional.h" #include "llvm/ADT/Optional.h"
#include "llvm/ADT/STLExtras.h" #include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/SmallVector.h" #include "llvm/ADT/SmallVector.h"
#define DEBUG_TYPE "fir-memref-dataflow-opt" #define DEBUG_TYPE "fir-memref-dataflow-opt"
▲ Show 20 Lines • Show All 108 Lines • Show Last 20 Lines

flang/lib/Optimizer/Transforms/MemoryAllocation.cpp

//===- MemoryAllocation.cpp -----------------------------------------------===// //===- MemoryAllocation.cpp -----------------------------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "flang/Optimizer/Dialect/FIRDialect.h" #include "flang/Optimizer/Dialect/FIRDialect.h"
#include "flang/Optimizer/Dialect/FIROps.h" #include "flang/Optimizer/Dialect/FIROps.h"
#include "flang/Optimizer/Dialect/FIRType.h" #include "flang/Optimizer/Dialect/FIRType.h"
#include "flang/Optimizer/Transforms/Passes.h" #include "flang/Optimizer/Transforms/Passes.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/Diagnostics.h" #include "mlir/IR/Diagnostics.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
#include "llvm/ADT/TypeSwitch.h" #include "llvm/ADT/TypeSwitch.h"
#define DEBUG_TYPE "flang-memory-allocation-opt" #define DEBUG_TYPE "flang-memory-allocation-opt"
Show All 13 Lines
}; };
class ReturnAnalysis { class ReturnAnalysis {
public: public:
ReturnAnalysis(mlir::Operation *op) { ReturnAnalysis(mlir::Operation *op) {
if (auto func = mlir::dyn_cast<mlir::FuncOp>(op)) if (auto func = mlir::dyn_cast<mlir::FuncOp>(op))
for (mlir::Block &block : func) for (mlir::Block &block : func)
for (mlir::Operation &i : block) for (mlir::Operation &i : block)
if (mlir::isa<mlir::ReturnOp>(i)) { if (mlir::isa<mlir::func::ReturnOp>(i)) {
returnMap[op].push_back(&i); returnMap[op].push_back(&i);
break; break;
} }
} }
llvm::SmallVector<mlir::Operation *> getReturns(mlir::Operation *func) const { llvm::SmallVector<mlir::Operation *> getReturns(mlir::Operation *func) const {
auto iter = returnMap.find(func); auto iter = returnMap.find(func);
if (iter != returnMap.end()) if (iter != returnMap.end())
▲ Show 20 Lines • Show All 131 Lines • ▼ Show 20 Lines void runOnOperation() override {
// If func is a declaration, skip it. // If func is a declaration, skip it.
if (func.empty()) if (func.empty())
return; return;
const auto &analysis = getAnalysis<ReturnAnalysis>(); const auto &analysis = getAnalysis<ReturnAnalysis>();
target.addLegalDialect<fir::FIROpsDialect, mlir::arith::ArithmeticDialect, target.addLegalDialect<fir::FIROpsDialect, mlir::arith::ArithmeticDialect,
mlir::StandardOpsDialect>(); mlir::func::FuncDialect>();
target.addDynamicallyLegalOp<fir::AllocaOp>([&](fir::AllocaOp alloca) { target.addDynamicallyLegalOp<fir::AllocaOp>([&](fir::AllocaOp alloca) {
return keepStackAllocation(alloca, &func.front(), options); return keepStackAllocation(alloca, &func.front(), options);
}); });
patterns.insert<AllocaOpConversion>(context, analysis.getReturns(func)); patterns.insert<AllocaOpConversion>(context, analysis.getReturns(func));
if (mlir::failed( if (mlir::failed(
mlir::applyPartialConversion(func, target, std::move(patterns)))) { mlir::applyPartialConversion(func, target, std::move(patterns)))) {
mlir::emitError(func.getLoc(), mlir::emitError(func.getLoc(),
Show All 18 Lines

flang/lib/Optimizer/Transforms/PassDetail.h

//===- PassDetail.h - Optimizer Transforms Pass class details ---*- C++ -*-===// //===- PassDetail.h - Optimizer Transforms Pass class details ---*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef FORTRAN_OPTMIZER_TRANSFORMS_PASSDETAIL_H #ifndef FORTRAN_OPTMIZER_TRANSFORMS_PASSDETAIL_H
#define FORTRAN_OPTMIZER_TRANSFORMS_PASSDETAIL_H #define FORTRAN_OPTMIZER_TRANSFORMS_PASSDETAIL_H
#include "flang/Optimizer/Dialect/FIRDialect.h" #include "flang/Optimizer/Dialect/FIRDialect.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/OpenACC/OpenACC.h" #include "mlir/Dialect/OpenACC/OpenACC.h"
#include "mlir/Dialect/OpenMP/OpenMPDialect.h" #include "mlir/Dialect/OpenMP/OpenMPDialect.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Pass/PassRegistry.h" #include "mlir/Pass/PassRegistry.h"
namespace fir { namespace fir {
#define GEN_PASS_CLASSES #define GEN_PASS_CLASSES
#include "flang/Optimizer/Transforms/Passes.h.inc" #include "flang/Optimizer/Transforms/Passes.h.inc"
} // namespace fir } // namespace fir
#endif // FORTRAN_OPTMIZER_TRANSFORMS_PASSDETAIL_H #endif // FORTRAN_OPTMIZER_TRANSFORMS_PASSDETAIL_H

flang/lib/Optimizer/Transforms/RewriteLoop.cpp

//===-- RewriteLoop.cpp ---------------------------------------------------===// //===-- RewriteLoop.cpp ---------------------------------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "flang/Optimizer/Dialect/FIRDialect.h" #include "flang/Optimizer/Dialect/FIRDialect.h"
#include "flang/Optimizer/Dialect/FIROps.h" #include "flang/Optimizer/Dialect/FIROps.h"
#include "flang/Optimizer/Transforms/Passes.h" #include "flang/Optimizer/Transforms/Passes.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h" #include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/Support/CommandLine.h" #include "llvm/Support/CommandLine.h"
using namespace fir; using namespace fir;
namespace { namespace {
▲ Show 20 Lines • Show All 275 Lines • ▼ Show 20 Lines
public: public:
void runOnOperation() override { void runOnOperation() override {
auto *context = &getContext(); auto *context = &getContext();
mlir::RewritePatternSet patterns(context); mlir::RewritePatternSet patterns(context);
patterns.insert<CfgLoopConv, CfgIfConv, CfgIterWhileConv>( patterns.insert<CfgLoopConv, CfgIfConv, CfgIterWhileConv>(
context, forceLoopToExecuteOnce); context, forceLoopToExecuteOnce);
mlir::ConversionTarget target(*context); mlir::ConversionTarget target(*context);
target.addLegalDialect<mlir::AffineDialect, mlir::cf::ControlFlowDialect, target.addLegalDialect<mlir::AffineDialect, mlir::cf::ControlFlowDialect,
FIROpsDialect, mlir::StandardOpsDialect>(); FIROpsDialect, mlir::func::FuncDialect>();
// apply the patterns // apply the patterns
target.addIllegalOp<ResultOp, DoLoopOp, IfOp, IterWhileOp>(); target.addIllegalOp<ResultOp, DoLoopOp, IfOp, IterWhileOp>();
target.markUnknownOpDynamicallyLegal([](Operation *) { return true; }); target.markUnknownOpDynamicallyLegal([](Operation *) { return true; });
if (mlir::failed(mlir::applyPartialConversion(getOperation(), target, if (mlir::failed(mlir::applyPartialConversion(getOperation(), target,
std::move(patterns)))) { std::move(patterns)))) {
mlir::emitError(mlir::UnknownLoc::get(context), mlir::emitError(mlir::UnknownLoc::get(context),
"error in converting to CFG\n"); "error in converting to CFG\n");
Show All 12 Lines

mlir/benchmark/python/common.py

"""Common utilities that are useful for all the benchmarks.""" """Common utilities that are useful for all the benchmarks."""
import numpy as np import numpy as np
import mlir.all_passes_registration import mlir.all_passes_registration
from mlir import ir from mlir import ir
from mlir.dialects import arith from mlir.dialects import arith
from mlir.dialects import builtin from mlir.dialects import builtin
from mlir.dialects import func
from mlir.dialects import memref from mlir.dialects import memref
from mlir.dialects import scf from mlir.dialects import scf
from mlir.dialects import std
from mlir.passmanager import PassManager from mlir.passmanager import PassManager
def setup_passes(mlir_module): def setup_passes(mlir_module):
"""Setup pass pipeline parameters for benchmark functions. """Setup pass pipeline parameters for benchmark functions.
""" """
opt = ( opt = (
"parallelization-strategy=0" "parallelization-strategy=0"
▲ Show 20 Lines • Show All 68 Lines • ▼ Show 20 Linesdef emit_benchmark_wrapped_main_func(func, timer_func):
with ir.InsertionPoint(wrapped_func.add_entry_block()): with ir.InsertionPoint(wrapped_func.add_entry_block()):
timer_buffer = wrapped_func.arguments[-1] timer_buffer = wrapped_func.arguments[-1]
zero = arith.ConstantOp.create_index(0) zero = arith.ConstantOp.create_index(0)
n_iterations = memref.DimOp(ir.IndexType.get(), timer_buffer, zero) n_iterations = memref.DimOp(ir.IndexType.get(), timer_buffer, zero)
one = arith.ConstantOp.create_index(1) one = arith.ConstantOp.create_index(1)
iter_args = list(wrapped_func.arguments[-num_results - 1:-1]) iter_args = list(wrapped_func.arguments[-num_results - 1:-1])
loop = scf.ForOp(zero, n_iterations, one, iter_args) loop = scf.ForOp(zero, n_iterations, one, iter_args)
with ir.InsertionPoint(loop.body): with ir.InsertionPoint(loop.body):
start = std.CallOp(timer_func, []) start = func.CallOp(timer_func, [])
call = std.CallOp( call = func.CallOp(
func, func,
wrapped_func.arguments[:-num_results - 1] + loop.inner_iter_args wrapped_func.arguments[:-num_results - 1] + loop.inner_iter_args
) )
end = std.CallOp(timer_func, []) end = func.CallOp(timer_func, [])
time_taken = arith.SubIOp(end, start) time_taken = arith.SubIOp(end, start)
memref.StoreOp(time_taken, timer_buffer, [loop.induction_variable]) memref.StoreOp(time_taken, timer_buffer, [loop.induction_variable])
scf.YieldOp(list(call.results)) scf.YieldOp(list(call.results))
std.ReturnOp(loop) func.ReturnOp(loop)
return wrapped_func return wrapped_func

mlir/docs/Bindings/Python.md

Show First 20 Lines • Show All 879 Lines • ▼ Show 20 Lines
directory with `__init__.py` file) as the entrypoint. directory with `__init__.py` file) as the entrypoint.
### Generating `_{DIALECT_NAMESPACE}_ops_gen.py` wrapper modules ### Generating `_{DIALECT_NAMESPACE}_ops_gen.py` wrapper modules
Each dialect with a mapping to python requires that an appropriate Each dialect with a mapping to python requires that an appropriate
`_{DIALECT_NAMESPACE}_ops_gen.py` wrapper module is created. This is done by `_{DIALECT_NAMESPACE}_ops_gen.py` wrapper module is created. This is done by
invoking `mlir-tblgen` on a python-bindings specific tablegen wrapper that invoking `mlir-tblgen` on a python-bindings specific tablegen wrapper that
includes the boilerplate and actual dialect specific `td` file. An example, for includes the boilerplate and actual dialect specific `td` file. An example, for
the `StandardOps` (which is assigned the namespace `std` as a special case): the `Func` (which is assigned the namespace `func` as a special case):
```tablegen ```tablegen
#ifndef PYTHON_BINDINGS_STANDARD_OPS #ifndef PYTHON_BINDINGS_FUNC_OPS
#define PYTHON_BINDINGS_STANDARD_OPS #define PYTHON_BINDINGS_FUNC_OPS
include "mlir/Bindings/Python/Attributes.td" include "mlir/Bindings/Python/Attributes.td"
include "mlir/Dialect/StandardOps/IR/Ops.td" include "mlir/Dialect/Func/IR/FuncOps.td"
#endif #endif // PYTHON_BINDINGS_FUNC_OPS
``` ```
In the main repository, building the wrapper is done via the CMake function In the main repository, building the wrapper is done via the CMake function
`declare_mlir_dialect_python_bindings`, which invokes: `declare_mlir_dialect_python_bindings`, which invokes:
``` ```
mlir-tblgen -gen-python-op-bindings -bind-dialect={DIALECT_NAMESPACE} \ mlir-tblgen -gen-python-op-bindings -bind-dialect={DIALECT_NAMESPACE} \
{PYTHON_BINDING_TD_FILE} {PYTHON_BINDING_TD_FILE}
▲ Show 20 Lines • Show All 268 Lines • Show Last 20 Lines

mlir/docs/Bufferization.md

Show First 20 Lines • Show All 163 Lines • ▼ Show 20 Linesstruct TensorBufferizePass : public TensorBufferizeBase<TensorBufferizePass> {
void runOnOperation() override { void runOnOperation() override {
auto *context = &getContext(); auto *context = &getContext();
BufferizeTypeConverter typeConverter; BufferizeTypeConverter typeConverter;
RewritePatternSet patterns(context); RewritePatternSet patterns(context);
ConversionTarget target(*context); ConversionTarget target(*context);
populateTensorBufferizePatterns(typeConverter, patterns); populateTensorBufferizePatterns(typeConverter, patterns);
target.addIllegalOp<tensor::CastOp, tensor::ExtractOp>(); target.addIllegalOp<tensor::CastOp, tensor::ExtractOp>();
target.addLegalDialect<StandardOpsDialect>(); target.addLegalDialect<func::FuncDialect>();
if (failed( if (failed(
applyPartialConversion(getOperation(), target, std::move(patterns)))) applyPartialConversion(getOperation(), target, std::move(patterns))))
signalPassFailure(); signalPassFailure();
} }
}; };
``` ```
▲ Show 20 Lines • Show All 99 Lines • Show Last 20 Lines

mlir/docs/DialectConversion.md

Show First 20 Lines • Show All 94 Lines • ▼ Show 20 Lines MyTarget(MLIRContext &ctx) : ConversionTarget(ctx) {
//-------------------------------------------------------------------------- //--------------------------------------------------------------------------
// Marking an operation as dynamically legal. // Marking an operation as dynamically legal.
/// Mark all operations within Affine dialect have dynamic legality /// Mark all operations within Affine dialect have dynamic legality
/// constraints. /// constraints.
addDynamicallyLegalDialect<AffineDialect>([](Operation *op) { ... }); addDynamicallyLegalDialect<AffineDialect>([](Operation *op) { ... });
/// Mark `std.return` as dynamically legal, but provide a specific legality /// Mark `func.return` as dynamically legal, but provide a specific legality
/// callback. /// callback.
addDynamicallyLegalOp<ReturnOp>([](ReturnOp op) { ... }); addDynamicallyLegalOp<func::ReturnOp>([](func::ReturnOp op) { ... });
/// Treat unknown operations, i.e. those without a legalization action /// Treat unknown operations, i.e. those without a legalization action
/// directly set, as dynamically legal. /// directly set, as dynamically legal.
markUnknownOpDynamicallyLegal([](Operation *op) { ... }); markUnknownOpDynamicallyLegal([](Operation *op) { ... });
//-------------------------------------------------------------------------- //--------------------------------------------------------------------------
// Marking an operation as illegal. // Marking an operation as illegal.
▲ Show 20 Lines • Show All 305 Lines • ▼ Show 20 Lines
output is formatted as a tree structure, mirroring the structure of the output is formatted as a tree structure, mirroring the structure of the
conversion process. This output contains all of the actions performed by the conversion process. This output contains all of the actions performed by the
rewriter, how generated operations get legalized, and why they fail. rewriter, how generated operations get legalized, and why they fail.
Example output is shown below: Example output is shown below:
``` ```
//===-------------------------------------------===// //===-------------------------------------------===//
Legalizing operation : 'std.return'(0x608000002e20) { Legalizing operation : 'func.return'(0x608000002e20) {
"std.return"() : () -> () "func.return"() : () -> ()
* Fold { * Fold {
} -> FAILURE : unable to fold } -> FAILURE : unable to fold
* Pattern : 'std.return -> ()' { * Pattern : 'func.return -> ()' {
** Insert : 'spv.Return'(0x6070000453e0) ** Insert : 'spv.Return'(0x6070000453e0)
** Replace : 'std.return'(0x608000002e20) ** Replace : 'func.return'(0x608000002e20)
//===-------------------------------------------===// //===-------------------------------------------===//
Legalizing operation : 'spv.Return'(0x6070000453e0) { Legalizing operation : 'spv.Return'(0x6070000453e0) {
"spv.Return"() : () -> () "spv.Return"() : () -> ()
} -> SUCCESS : operation marked legal by the target } -> SUCCESS : operation marked legal by the target
//===-------------------------------------------===// //===-------------------------------------------===//
} -> SUCCESS : pattern applied successfully } -> SUCCESS : pattern applied successfully
} -> SUCCESS } -> SUCCESS
//===-------------------------------------------===// //===-------------------------------------------===//
``` ```
This output is describing the legalization of an `std.return` operation. We This output is describing the legalization of an `func.return` operation. We
first try to legalize by folding the operation, but that is unsuccessful for first try to legalize by folding the operation, but that is unsuccessful for
`std.return`. From there, a pattern is applied that replaces the `std.return` `func.return`. From there, a pattern is applied that replaces the `func.return`
with a `spv.Return`. The newly generated `spv.Return` is then processed for with a `spv.Return`. The newly generated `spv.Return` is then processed for
legalization, but is found to already legal as per the target. legalization, but is found to already legal as per the target.

mlir/docs/Dialects/Affine.md

Show First 20 Lines • Show All 60 Lines • ▼ Show 20 Lines
The affine dialect imposes certain restrictions on dimension and symbolic The affine dialect imposes certain restrictions on dimension and symbolic
identifiers to enable powerful analysis and transformation. An SSA value's use identifiers to enable powerful analysis and transformation. An SSA value's use
can be bound to a symbolic identifier if that SSA value is either 1. a region can be bound to a symbolic identifier if that SSA value is either 1. a region
argument for an op with trait `AffineScope` (eg. `FuncOp`), 2. a value defined argument for an op with trait `AffineScope` (eg. `FuncOp`), 2. a value defined
at the top level of an `AffineScope` op (i.e., immediately enclosed by the at the top level of an `AffineScope` op (i.e., immediately enclosed by the
latter), 3. a value that dominates the `AffineScope` op enclosing the value's latter), 3. a value that dominates the `AffineScope` op enclosing the value's
use, 4. the result of a use, 4. the result of a
[`constant` operation](Standard.md/#stdconstant-constantop), 5. the result of an constant operation, 5. the result of an
[`affine.apply` operation](#affineapply-affineapplyop) that recursively takes as [`affine.apply` operation](#affineapply-affineapplyop) that recursively takes as
arguments any valid symbolic identifiers, or 6. the result of a arguments any valid symbolic identifiers, or 6. the result of a
[`dim` operation](MemRef.md/#memrefdim-mlirmemrefdimop) on either a memref that [`dim` operation](MemRef.md/#memrefdim-mlirmemrefdimop) on either a memref that
is an argument to a `AffineScope` op or a memref where the corresponding is an argument to a `AffineScope` op or a memref where the corresponding
dimension is either static or a dynamic one in turn bound to a valid symbol. dimension is either static or a dynamic one in turn bound to a valid symbol.
*Note:* if the use of an SSA value is not contained in any op with the *Note:* if the use of an SSA value is not contained in any op with the
`AffineScope` trait, only the rules 4-6 can be applied. `AffineScope` trait, only the rules 4-6 can be applied.
▲ Show 20 Lines • Show All 362 Lines • Show Last 20 Lines

mlir/docs/Dialects/Func.md

  • This file was added.
# 'func' Dialect
This dialect provides documentation for operations within the Func dialect.
This dialect contains operations surrounding high order function
abstractions, such as calls.
**Please post an RFC on the [forum](https://llvm.discourse.group/c/mlir/31)
before adding or changing any operation in this dialect.**
[TOC]
## Operations
[include "Dialects/FuncOps.md"]

mlir/docs/Dialects/Standard.md

  • This file was deleted.
# 'std' Dialect
This dialect provides documentation for operations within the Standard dialect.
Note: This dialect is a collection of operations for several different concepts,
and should be split into multiple more-focused dialects accordingly.
**Please post an RFC on the [forum](https://llvm.discourse.group/c/mlir/31)
before adding or changing any operation in this dialect.**
[TOC]
## Operations
[include "Dialects/StandardOps.md"]

mlir/docs/Dialects/Vector.md

Show First 20 Lines • Show All 93 Lines • ▼ Show 20 Lines
Vectors) are welcome future extensions. Vectors) are welcome future extensions.
### Virtual Vector Ops ### Virtual Vector Ops
Some existing Standard and Vector Dialect on `n-D` `vector` types comprise: Some existing Standard and Vector Dialect on `n-D` `vector` types comprise:
```mlir ```mlir
%2 = arith.addf %0, %1 : vector<3x7x8xf32> // -> vector<3x7x8xf32> %2 = %2 = arith.addf %0, %1 : vector<3x7x8xf32> // -> vector<3x7x8xf32> %2 =
arith.mulf %0, %1 : vector<3x7x8xf32> // -> vector<3x7x8xf32> %2 = std.splat arith.mulf %0, %1 : vector<3x7x8xf32> // -> vector<3x7x8xf32> %2 = vector.splat
%1 : vector<3x7x8xf32> // -> vector<3x7x8xf32> %1 : vector<3x7x8xf32> // -> vector<3x7x8xf32>
%1 = vector.extract %0[1]: vector<3x7x8xf32> // -> vector<7x8xf32> %1 = %1 = vector.extract %0[1]: vector<3x7x8xf32> // -> vector<7x8xf32> %1 =
vector.extract %0[1, 5]: vector<3x7x8xf32> // -> vector<8xf32> %2 = vector.extract %0[1, 5]: vector<3x7x8xf32> // -> vector<8xf32> %2 =
vector.outerproduct %0, %1: vector<4xf32>, vector<8xf32> // -> vector<4x8xf32> vector.outerproduct %0, %1: vector<4xf32>, vector<8xf32> // -> vector<4x8xf32>
%3 = vector.outerproduct %0, %1, %2: vector<4xf32>, vector<8xf32> // fma when %3 = vector.outerproduct %0, %1, %2: vector<4xf32>, vector<8xf32> // fma when
adding %2 %3 = vector.strided_slice %0 {offsets = [2, 2], sizes = [2, 2], adding %2 %3 = vector.strided_slice %0 {offsets = [2, 2], sizes = [2, 2],
strides = [1, 1]}: vector<4x8x16xf32> // Returns a slice of type strides = [1, 1]}: vector<4x8x16xf32> // Returns a slice of type
▲ Show 20 Lines • Show All 384 Lines • Show Last 20 Lines

mlir/docs/Dialects/emitc.md

Show All 20 Lines
argument `-declare-variables-at-top`. argument `-declare-variables-at-top`.
Besides operations part of the EmitC dialect, the Cpp targets supports Besides operations part of the EmitC dialect, the Cpp targets supports
translating the following operations: translating the following operations:
* 'cf' Dialect * 'cf' Dialect
* `cf.br` * `cf.br`
* `cf.cond_br` * `cf.cond_br`
* 'std' Dialect * 'func' Dialect
* `std.call` * `func.call`
* `std.constant` * `func.constant`
* `std.return` * `func.return`
* 'scf' Dialect * 'scf' Dialect
* `scf.for` * `scf.for`
* `scf.if` * `scf.if`
* `scf.yield` * `scf.yield`
* 'arith' Dialect * 'arith' Dialect
* `arith.constant` * `arith.constant`

mlir/docs/LangRef.md

Show First 20 Lines • Show All 250 Lines • ▼ Show 20 Lines
MLIR allows for multiple dialects, even those outside of the main tree, to MLIR allows for multiple dialects, even those outside of the main tree, to
co-exist together within one module. Dialects are produced and consumed by co-exist together within one module. Dialects are produced and consumed by
certain passes. MLIR provides a [framework](DialectConversion.md) to convert certain passes. MLIR provides a [framework](DialectConversion.md) to convert
between, and within, different dialects. between, and within, different dialects.
A few of the dialects supported by MLIR: A few of the dialects supported by MLIR:
* [Affine dialect](Dialects/Affine.md) * [Affine dialect](Dialects/Affine.md)
* [Func dialect](Dialects/Func.md)
* [GPU dialect](Dialects/GPU.md) * [GPU dialect](Dialects/GPU.md)
* [LLVM dialect](Dialects/LLVM.md) * [LLVM dialect](Dialects/LLVM.md)
* [SPIR-V dialect](Dialects/SPIR-V.md) * [SPIR-V dialect](Dialects/SPIR-V.md)
* [Standard dialect](Dialects/Standard.md)
* [Vector dialect](Dialects/Vector.md) * [Vector dialect](Dialects/Vector.md)
### Target specific operations ### Target specific operations
Dialects provide a modular way in which targets can expose target-specific Dialects provide a modular way in which targets can expose target-specific
operations directly through to MLIR. As an example, some targets go through operations directly through to MLIR. As an example, some targets go through
LLVM. LLVM has a rich set of intrinsics for certain target-independent LLVM. LLVM has a rich set of intrinsics for certain target-independent
operations (e.g. addition with overflow check) as well as providing access to operations (e.g. addition with overflow check) as well as providing access to
▲ Show 20 Lines • Show All 581 Lines • Show Last 20 Lines

mlir/docs/OpDefinitions.md

Show First 20 Lines • Show All 855 Lines • ▼ Show 20 Lines
* Literals, variables, custom directives, and type directives are the only * Literals, variables, custom directives, and type directives are the only
valid elements within the group. valid elements within the group.
- Any attribute variable may be used, but only optional attributes can be - Any attribute variable may be used, but only optional attributes can be
marked as the anchor. marked as the anchor.
- Only variadic or optional results and operand arguments and can be used. - Only variadic or optional results and operand arguments and can be used.
- All region variables can be used. When a non-variable length region is - All region variables can be used. When a non-variable length region is
used, if the group is not present the region is empty. used, if the group is not present the region is empty.
An example of an operation with an optional group is `std.return`, which has a An example of an operation with an optional group is `func.return`, which has a
variadic number of operands. variadic number of operands.
```tablegen ```tablegen
def ReturnOp : ... { def ReturnOp : ... {
let arguments = (ins Variadic<AnyType>:$operands); let arguments = (ins Variadic<AnyType>:$operands);
// We only print the operands and types if there are a non-zero number // We only print the operands and types if there are a non-zero number
// of operands. // of operands.
▲ Show 20 Lines • Show All 1,058 Lines • Show Last 20 Lines

mlir/docs/Passes.md

Show All 22 Lines
## `affine` Dialect Passes ## `affine` Dialect Passes
[include "AffinePasses.md"] [include "AffinePasses.md"]
## `arith` Dialect Passes ## `arith` Dialect Passes
[include "ArithmeticPasses.md"] [include "ArithmeticPasses.md"]
## `func` Dialect Passes
[include "FuncPasses.md"]
## `gpu` Dialect Passes ## `gpu` Dialect Passes
[include "GPUPasses.md"] [include "GPUPasses.md"]
## `linalg` Dialect Passes ## `linalg` Dialect Passes
[include "LinalgPasses.md"] [include "LinalgPasses.md"]
Show All 24 Lines
## `sparse_tensor` Dialect Passes ## `sparse_tensor` Dialect Passes
[include "SparseTensorPasses.md"] [include "SparseTensorPasses.md"]
## `spv` Dialect Passes ## `spv` Dialect Passes
[include "SPIRVPasses.md"] [include "SPIRVPasses.md"]
## `standard` Dialect Passes
[include "StandardPasses.md"]
## `tensor` Dialect Passes ## `tensor` Dialect Passes
[include "TensorPasses.md"] [include "TensorPasses.md"]
## `vector` Dialect Passes ## `vector` Dialect Passes
[include "VectorPasses.md"] [include "VectorPasses.md"]
## TOSA Dialect Passes ## TOSA Dialect Passes
[include "TosaPasses.md"] [include "TosaPasses.md"]

mlir/docs/SymbolsAndSymbolTables.md

Show All 27 Lines
A `Symbol` is a named operation that resides immediately within a region that A `Symbol` is a named operation that resides immediately within a region that
defines a [`SymbolTable`](#symbol-table). The name of a symbol *must* be unique defines a [`SymbolTable`](#symbol-table). The name of a symbol *must* be unique
within the parent `SymbolTable`. This name is semantically similarly to an SSA within the parent `SymbolTable`. This name is semantically similarly to an SSA
result value, and may be referred to by other operations to provide a symbolic result value, and may be referred to by other operations to provide a symbolic
link, or use, to the symbol. An example of a `Symbol` operation is link, or use, to the symbol. An example of a `Symbol` operation is
[`builtin.func`](Dialects/Builtin.md/#func-mlirfuncop). `builtin.func` defines a [`builtin.func`](Dialects/Builtin.md/#func-mlirfuncop). `builtin.func` defines a
symbol name, which is [referred to](#referencing-a-symbol) by operations like symbol name, which is [referred to](#referencing-a-symbol) by operations like
[`std.call`](Dialects/Standard.md/#stdcall-callop). [`func.call`](Dialects/Func.md/#funccall-callop).
### Defining or declaring a Symbol ### Defining or declaring a Symbol
A `Symbol` operation should use the `SymbolOpInterface` interface to provide the A `Symbol` operation should use the `SymbolOpInterface` interface to provide the
necessary verification and accessors; it also supports operations, such as necessary verification and accessors; it also supports operations, such as
`builtin.module`, that conditionally define a symbol. `Symbol`s must have the `builtin.module`, that conditionally define a symbol. `Symbol`s must have the
following properties: following properties:
▲ Show 20 Lines • Show All 183 Lines • Show Last 20 Lines

mlir/docs/TargetLLVMIR.md

Show First 20 Lines • Show All 342 Lines • ▼ Show 20 Lines
The default calling convention converts `memref`-typed function arguments to The default calling convention converts `memref`-typed function arguments to
LLVM dialect literal structs LLVM dialect literal structs
[defined above](TargetLLVMIR.md#ranked-memref-types) before unbundling them into [defined above](TargetLLVMIR.md#ranked-memref-types) before unbundling them into
individual scalar arguments. individual scalar arguments.
Examples: Examples:
This convention is implemented in the conversion of `std.func` and `std.call` to This convention is implemented in the conversion of `builtin.func` and `func.call` to
the LLVM dialect, with the former unpacking the descriptor into a set of the LLVM dialect, with the former unpacking the descriptor into a set of
individual values and the latter packing those values back into a descriptor so individual values and the latter packing those values back into a descriptor so
as to make it transparently usable by other operations. Conversions from other as to make it transparently usable by other operations. Conversions from other
dialects should take this convention into account. dialects should take this convention into account.
This specific convention is motivated by the necessity to specify alignment and This specific convention is motivated by the necessity to specify alignment and
aliasing attributes on the raw pointers underpinning the memref. aliasing attributes on the raw pointers underpinning the memref.
▲ Show 20 Lines • Show All 116 Lines • ▼ Show 20 Lines
some memory in which the ranked memref descriptor is stored. By convention, this some memory in which the ranked memref descriptor is stored. By convention, this
memory is allocated on stack and has the lifetime of the function. (*Note:* due memory is allocated on stack and has the lifetime of the function. (*Note:* due
to function-length lifetime, creation of multiple unranked memref descriptors, to function-length lifetime, creation of multiple unranked memref descriptors,
e.g., in a loop, may lead to stack overflows.) If an unranked descriptor has to e.g., in a loop, may lead to stack overflows.) If an unranked descriptor has to
be returned from a function, the ranked descriptor it points to is copied into be returned from a function, the ranked descriptor it points to is copied into
dynamically allocated memory, and the pointer in the unranked descriptor is dynamically allocated memory, and the pointer in the unranked descriptor is
updated accordingly. The allocation happens immediately before returning. It is updated accordingly. The allocation happens immediately before returning. It is
the responsibility of the caller to free the dynamically allocated memory. The the responsibility of the caller to free the dynamically allocated memory. The
default conversion of `std.call` and `std.call_indirect` copies the ranked default conversion of `func.call` and `func.call_indirect` copies the ranked
descriptor to newly allocated memory on the caller's stack. Thus, the convention descriptor to newly allocated memory on the caller's stack. Thus, the convention
of the ranked memref descriptor pointed to by an unranked memref descriptor of the ranked memref descriptor pointed to by an unranked memref descriptor
being stored on stack is respected. being stored on stack is respected.
#### Bare Pointer Calling Convention for Ranked MemRef #### Bare Pointer Calling Convention for Ranked MemRef
The "bare pointer" calling convention converts `memref`-typed function arguments The "bare pointer" calling convention converts `memref`-typed function arguments
to a *single* pointer to the aligned data. Note that this does *not* apply to to a *single* pointer to the aligned data. Note that this does *not* apply to
▲ Show 20 Lines • Show All 406 Lines • Show Last 20 Lines

mlir/docs/Tutorials/CreatingADialect.md

Show First 20 Lines • Show All 124 Lines • ▼ Show 20 Lines
mlir/lib/Conversion/XCommon, for example mlir/lib/Conversion/XCommon, for example
mlir/lib/Conversion/GPUCommon. mlir/lib/Conversion/GPUCommon.
## CMake best practices ## CMake best practices
Each conversion typically exists in a separate library, declared with Each conversion typically exists in a separate library, declared with
add_mlir_conversion_library(). Conversion libraries typically depend add_mlir_conversion_library(). Conversion libraries typically depend
on their source and target dialects, but may also depend on other on their source and target dialects, but may also depend on other
dialects (e.g. MLIRStandard). Typically this dependence is specified dialects (e.g. MLIRFunc). Typically this dependence is specified
using target_link_libraries() and the PUBLIC keyword. For instance: using target_link_libraries() and the PUBLIC keyword. For instance:
```cmake ```cmake
add_mlir_conversion_library(MLIRBarToFoo add_mlir_conversion_library(MLIRBarToFoo
BarToFoo.cpp BarToFoo.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/BarToFoo ${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/BarToFoo
Show All 24 Lines

mlir/docs/Tutorials/Toy/Ch-5.md

Show First 20 Lines • Show All 57 Lines • ▼ Show 20 Lines
```c++ ```c++
void ToyToAffineLoweringPass::runOnOperation() { void ToyToAffineLoweringPass::runOnOperation() {
// The first thing to define is the conversion target. This will define the // The first thing to define is the conversion target. This will define the
// final target for this lowering. // final target for this lowering.
mlir::ConversionTarget target(getContext()); mlir::ConversionTarget target(getContext());
// We define the specific operations, or dialects, that are legal targets for // We define the specific operations, or dialects, that are legal targets for
// this lowering. In our case, we are lowering to a combination of the // this lowering. In our case, we are lowering to a combination of the
// `Affine`, `Arithmetic`, `MemRef`, and `Standard` dialects. // `Affine`, `Arithmetic`, `Func`, and `MemRef` dialects.
target.addLegalDialect<AffineDialect, arith::ArithmeticDialect, target.addLegalDialect<AffineDialect, arith::ArithmeticDialect,
memref::MemRefDialect, StandardOpsDialect>(); func::FuncDialect, memref::MemRefDialect>();
// We also define the Toy dialect as Illegal so that the conversion will fail // We also define the Toy dialect as Illegal so that the conversion will fail
// if any of these operations are *not* converted. Given that we actually want // if any of these operations are *not* converted. Given that we actually want
// a partial lowering, we explicitly mark the Toy operations that don't want // a partial lowering, we explicitly mark the Toy operations that don't want
// to lower, `toy.print`, as *legal*. `toy.print` will still need its operands // to lower, `toy.print`, as *legal*. `toy.print` will still need its operands
// to be updated though (as we convert from TensorType to MemRefType), so we // to be updated though (as we convert from TensorType to MemRefType), so we
// only treat it as `legal` if its operands are legal. // only treat it as `legal` if its operands are legal.
target.addIllegalDialect<ToyDialect>(); target.addIllegalDialect<ToyDialect>();
▲ Show 20 Lines • Show All 275 Lines • Show Last 20 Lines

mlir/examples/standalone/standalone-opt/standalone-opt.cpp

Show All 22 Lines
#include "Standalone/StandaloneDialect.h" #include "Standalone/StandaloneDialect.h"
int main(int argc, char **argv) { int main(int argc, char **argv) {
mlir::registerAllPasses(); mlir::registerAllPasses();
// TODO: Register standalone passes here. // TODO: Register standalone passes here.
mlir::DialectRegistry registry; mlir::DialectRegistry registry;
registry.insert<mlir::standalone::StandaloneDialect, registry.insert<mlir::standalone::StandaloneDialect,
mlir::arith::ArithmeticDialect, mlir::StandardOpsDialect>(); mlir::arith::ArithmeticDialect, mlir::func::FuncDialect>();
// Add the following to include *all* MLIR Core dialects, or selectively // Add the following to include *all* MLIR Core dialects, or selectively
// include what you need like above. You only need to register dialects that // include what you need like above. You only need to register dialects that
// will be *parsed* by the tool, not the one generated // will be *parsed* by the tool, not the one generated
// registerAllDialects(registry); // registerAllDialects(registry);
return mlir::asMainReturnCode( return mlir::asMainReturnCode(
mlir::MlirOptMain(argc, argv, "Standalone optimizer driver\n", registry)); mlir::MlirOptMain(argc, argv, "Standalone optimizer driver\n", registry));
} }

mlir/examples/toy/Ch5/mlir/LowerToAffineLoops.cpp

Show All 11 Lines
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "toy/Dialect.h" #include "toy/Dialect.h"
#include "toy/Passes.h" #include "toy/Passes.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/ADT/Sequence.h" #include "llvm/ADT/Sequence.h"
using namespace mlir; using namespace mlir;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// ToyToAffine RewritePatterns // ToyToAffine RewritePatterns
▲ Show 20 Lines • Show All 194 Lines • ▼ Show 20 Linesstruct ReturnOpLowering : public OpRewritePattern<toy::ReturnOp> {
LogicalResult matchAndRewrite(toy::ReturnOp op, LogicalResult matchAndRewrite(toy::ReturnOp op,
PatternRewriter &rewriter) const final { PatternRewriter &rewriter) const final {
// During this lowering, we expect that all function calls have been // During this lowering, we expect that all function calls have been
// inlined. // inlined.
if (op.hasOperand()) if (op.hasOperand())
return failure(); return failure();
// We lower "toy.return" directly to "std.return". // We lower "toy.return" directly to "func.return".
rewriter.replaceOpWithNewOp<ReturnOp>(op); rewriter.replaceOpWithNewOp<func::ReturnOp>(op);
return success(); return success();
} }
}; };
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// ToyToAffine RewritePatterns: Transpose operations // ToyToAffine RewritePatterns: Transpose operations
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
Show All 32 Lines
/// This is a partial lowering to affine loops of the toy operations that are /// This is a partial lowering to affine loops of the toy operations that are
/// computationally intensive (like matmul for example...) while keeping the /// computationally intensive (like matmul for example...) while keeping the
/// rest of the code in the Toy dialect. /// rest of the code in the Toy dialect.
namespace { namespace {
struct ToyToAffineLoweringPass struct ToyToAffineLoweringPass
: public PassWrapper<ToyToAffineLoweringPass, OperationPass<FuncOp>> { : public PassWrapper<ToyToAffineLoweringPass, OperationPass<FuncOp>> {
void getDependentDialects(DialectRegistry &registry) const override { void getDependentDialects(DialectRegistry &registry) const override {
registry.insert<AffineDialect, memref::MemRefDialect, StandardOpsDialect>(); registry.insert<AffineDialect, func::FuncDialect, memref::MemRefDialect>();
} }
void runOnOperation() final; void runOnOperation() final;
}; };
} // namespace } // namespace
void ToyToAffineLoweringPass::runOnOperation() { void ToyToAffineLoweringPass::runOnOperation() {
FuncOp function = getOperation(); FuncOp function = getOperation();
Show All 9 Linesvoid ToyToAffineLoweringPass::runOnOperation() {
} }
// The first thing to define is the conversion target. This will define the // The first thing to define is the conversion target. This will define the
// final target for this lowering. // final target for this lowering.
ConversionTarget target(getContext()); ConversionTarget target(getContext());
// We define the specific operations, or dialects, that are legal targets for // We define the specific operations, or dialects, that are legal targets for
// this lowering. In our case, we are lowering to a combination of the // this lowering. In our case, we are lowering to a combination of the
// `Affine`, `Arithmetic`, `MemRef`, and `Standard` dialects. // `Affine`, `Arithmetic`, `Func`, and `MemRef` dialects.
target.addLegalDialect<AffineDialect, arith::ArithmeticDialect, target.addLegalDialect<AffineDialect, arith::ArithmeticDialect,
memref::MemRefDialect, StandardOpsDialect>(); func::FuncDialect, memref::MemRefDialect>();
// We also define the Toy dialect as Illegal so that the conversion will fail // We also define the Toy dialect as Illegal so that the conversion will fail
// if any of these operations are *not* converted. Given that we actually want // if any of these operations are *not* converted. Given that we actually want
// a partial lowering, we explicitly mark the Toy operations that don't want // a partial lowering, we explicitly mark the Toy operations that don't want
// to lower, `toy.print`, as `legal`. `toy.print` will still need its operands // to lower, `toy.print`, as `legal`. `toy.print` will still need its operands
// to be updated though (as we convert from TensorType to MemRefType), so we // to be updated though (as we convert from TensorType to MemRefType), so we
// only treat it as `legal` if its operands are legal. // only treat it as `legal` if its operands are legal.
target.addIllegalDialect<toy::ToyDialect>(); target.addIllegalDialect<toy::ToyDialect>();
Show All 25 Lines

mlir/examples/toy/Ch6/mlir/LowerToAffineLoops.cpp

Show All 11 Lines
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "toy/Dialect.h" #include "toy/Dialect.h"
#include "toy/Passes.h" #include "toy/Passes.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/ADT/Sequence.h" #include "llvm/ADT/Sequence.h"
using namespace mlir; using namespace mlir;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// ToyToAffine RewritePatterns // ToyToAffine RewritePatterns
▲ Show 20 Lines • Show All 194 Lines • ▼ Show 20 Linesstruct ReturnOpLowering : public OpRewritePattern<toy::ReturnOp> {
LogicalResult matchAndRewrite(toy::ReturnOp op, LogicalResult matchAndRewrite(toy::ReturnOp op,
PatternRewriter &rewriter) const final { PatternRewriter &rewriter) const final {
// During this lowering, we expect that all function calls have been // During this lowering, we expect that all function calls have been
// inlined. // inlined.
if (op.hasOperand()) if (op.hasOperand())
return failure(); return failure();
// We lower "toy.return" directly to "std.return". // We lower "toy.return" directly to "func.return".
rewriter.replaceOpWithNewOp<ReturnOp>(op); rewriter.replaceOpWithNewOp<func::ReturnOp>(op);
return success(); return success();
} }
}; };
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// ToyToAffine RewritePatterns: Transpose operations // ToyToAffine RewritePatterns: Transpose operations
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
Show All 32 Lines
/// This is a partial lowering to affine loops of the toy operations that are /// This is a partial lowering to affine loops of the toy operations that are
/// computationally intensive (like matmul for example...) while keeping the /// computationally intensive (like matmul for example...) while keeping the
/// rest of the code in the Toy dialect. /// rest of the code in the Toy dialect.
namespace { namespace {
struct ToyToAffineLoweringPass struct ToyToAffineLoweringPass
: public PassWrapper<ToyToAffineLoweringPass, OperationPass<FuncOp>> { : public PassWrapper<ToyToAffineLoweringPass, OperationPass<FuncOp>> {
void getDependentDialects(DialectRegistry &registry) const override { void getDependentDialects(DialectRegistry &registry) const override {
registry.insert<AffineDialect, memref::MemRefDialect, StandardOpsDialect>(); registry.insert<AffineDialect, func::FuncDialect, memref::MemRefDialect>();
} }
void runOnOperation() final; void runOnOperation() final;
}; };
} // namespace } // namespace
void ToyToAffineLoweringPass::runOnOperation() { void ToyToAffineLoweringPass::runOnOperation() {
auto function = getOperation(); auto function = getOperation();
Show All 9 Linesvoid ToyToAffineLoweringPass::runOnOperation() {
} }
// The first thing to define is the conversion target. This will define the // The first thing to define is the conversion target. This will define the
// final target for this lowering. // final target for this lowering.
ConversionTarget target(getContext()); ConversionTarget target(getContext());
// We define the specific operations, or dialects, that are legal targets for // We define the specific operations, or dialects, that are legal targets for
// this lowering. In our case, we are lowering to a combination of the // this lowering. In our case, we are lowering to a combination of the
// `Affine`, `Arithmetic`, `MemRef`, and `Standard` dialects. // `Affine`, `Arithmetic`, `Func`, and `MemRef` dialects.
target.addLegalDialect<AffineDialect, arith::ArithmeticDialect, target.addLegalDialect<AffineDialect, arith::ArithmeticDialect,
memref::MemRefDialect, StandardOpsDialect>(); func::FuncDialect, memref::MemRefDialect>();
// We also define the Toy dialect as Illegal so that the conversion will fail // We also define the Toy dialect as Illegal so that the conversion will fail
// if any of these operations are *not* converted. Given that we actually want // if any of these operations are *not* converted. Given that we actually want
// a partial lowering, we explicitly mark the Toy operations that don't want // a partial lowering, we explicitly mark the Toy operations that don't want
// to lower, `toy.print`, as `legal`. `toy.print` will still need its operands // to lower, `toy.print`, as `legal`. `toy.print` will still need its operands
// to be updated though (as we convert from TensorType to MemRefType), so we // to be updated though (as we convert from TensorType to MemRefType), so we
// only treat it as `legal` if its operands are legal. // only treat it as `legal` if its operands are legal.
target.addIllegalDialect<toy::ToyDialect>(); target.addIllegalDialect<toy::ToyDialect>();
Show All 25 Lines

mlir/examples/toy/Ch6/mlir/LowerToLLVM.cpp

Show All 29 Lines
#include "mlir/Conversion/LLVMCommon/ConversionTarget.h" #include "mlir/Conversion/LLVMCommon/ConversionTarget.h"
#include "mlir/Conversion/LLVMCommon/TypeConverter.h" #include "mlir/Conversion/LLVMCommon/TypeConverter.h"
#include "mlir/Conversion/MemRefToLLVM/MemRefToLLVM.h" #include "mlir/Conversion/MemRefToLLVM/MemRefToLLVM.h"
#include "mlir/Conversion/SCFToControlFlow/SCFToControlFlow.h" #include "mlir/Conversion/SCFToControlFlow/SCFToControlFlow.h"
#include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h" #include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h"
#include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVMPass.h" #include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVMPass.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/ADT/Sequence.h" #include "llvm/ADT/Sequence.h"
using namespace mlir; using namespace mlir;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// ToyToLLVM RewritePatterns // ToyToLLVM RewritePatterns
Show All 36 Lines for (unsigned i = 0, e = memRefShape.size(); i != e; ++i) {
rewriter.eraseOp(&nested); rewriter.eraseOp(&nested);
loopIvs.push_back(loop.getInductionVar()); loopIvs.push_back(loop.getInductionVar());
// Terminate the loop body. // Terminate the loop body.
rewriter.setInsertionPointToEnd(loop.getBody()); rewriter.setInsertionPointToEnd(loop.getBody());
// Insert a newline after each of the inner dimensions of the shape. // Insert a newline after each of the inner dimensions of the shape.
if (i != e - 1) if (i != e - 1)
rewriter.create<CallOp>(loc, printfRef, rewriter.getIntegerType(32), rewriter.create<func::CallOp>(loc, printfRef,
newLineCst); rewriter.getIntegerType(32), newLineCst);
rewriter.create<scf::YieldOp>(loc); rewriter.create<scf::YieldOp>(loc);
rewriter.setInsertionPointToStart(loop.getBody()); rewriter.setInsertionPointToStart(loop.getBody());
} }
// Generate a call to printf for the current element of the loop. // Generate a call to printf for the current element of the loop.
auto printOp = cast<toy::PrintOp>(op); auto printOp = cast<toy::PrintOp>(op);
auto elementLoad = auto elementLoad =
rewriter.create<memref::LoadOp>(loc, printOp.input(), loopIvs); rewriter.create<memref::LoadOp>(loc, printOp.input(), loopIvs);
rewriter.create<CallOp>(loc, printfRef, rewriter.getIntegerType(32), rewriter.create<func::CallOp>(
loc, printfRef, rewriter.getIntegerType(32),
ArrayRef<Value>({formatSpecifierCst, elementLoad})); ArrayRef<Value>({formatSpecifierCst, elementLoad}));
// Notify the rewriter that this operation has been removed. // Notify the rewriter that this operation has been removed.
rewriter.eraseOp(op); rewriter.eraseOp(op);
return success(); return success();
} }
private: private:
/// Return a symbol reference to the printf function, inserting it into the /// Return a symbol reference to the printf function, inserting it into the
▲ Show 20 Lines • Show All 113 Lines • Show Last 20 Lines

mlir/examples/toy/Ch7/mlir/LowerToAffineLoops.cpp

Show All 11 Lines
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "toy/Dialect.h" #include "toy/Dialect.h"
#include "toy/Passes.h" #include "toy/Passes.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/ADT/Sequence.h" #include "llvm/ADT/Sequence.h"
using namespace mlir; using namespace mlir;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// ToyToAffine RewritePatterns // ToyToAffine RewritePatterns
▲ Show 20 Lines • Show All 194 Lines • ▼ Show 20 Linesstruct ReturnOpLowering : public OpRewritePattern<toy::ReturnOp> {
LogicalResult matchAndRewrite(toy::ReturnOp op, LogicalResult matchAndRewrite(toy::ReturnOp op,
PatternRewriter &rewriter) const final { PatternRewriter &rewriter) const final {
// During this lowering, we expect that all function calls have been // During this lowering, we expect that all function calls have been
// inlined. // inlined.
if (op.hasOperand()) if (op.hasOperand())
return failure(); return failure();
// We lower "toy.return" directly to "std.return". // We lower "toy.return" directly to "func.return".
rewriter.replaceOpWithNewOp<ReturnOp>(op); rewriter.replaceOpWithNewOp<func::ReturnOp>(op);
return success(); return success();
} }
}; };
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// ToyToAffine RewritePatterns: Transpose operations // ToyToAffine RewritePatterns: Transpose operations
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
Show All 32 Lines
/// This is a partial lowering to affine loops of the toy operations that are /// This is a partial lowering to affine loops of the toy operations that are
/// computationally intensive (like matmul for example...) while keeping the /// computationally intensive (like matmul for example...) while keeping the
/// rest of the code in the Toy dialect. /// rest of the code in the Toy dialect.
namespace { namespace {
struct ToyToAffineLoweringPass struct ToyToAffineLoweringPass
: public PassWrapper<ToyToAffineLoweringPass, OperationPass<FuncOp>> { : public PassWrapper<ToyToAffineLoweringPass, OperationPass<FuncOp>> {
void getDependentDialects(DialectRegistry &registry) const override { void getDependentDialects(DialectRegistry &registry) const override {
registry.insert<AffineDialect, memref::MemRefDialect, StandardOpsDialect>(); registry.insert<AffineDialect, func::FuncDialect, memref::MemRefDialect>();
} }
void runOnOperation() final; void runOnOperation() final;
}; };
} // namespace } // namespace
void ToyToAffineLoweringPass::runOnOperation() { void ToyToAffineLoweringPass::runOnOperation() {
auto function = getOperation(); auto function = getOperation();
Show All 9 Linesvoid ToyToAffineLoweringPass::runOnOperation() {
} }
// The first thing to define is the conversion target. This will define the // The first thing to define is the conversion target. This will define the
// final target for this lowering. // final target for this lowering.
ConversionTarget target(getContext()); ConversionTarget target(getContext());
// We define the specific operations, or dialects, that are legal targets for // We define the specific operations, or dialects, that are legal targets for
// this lowering. In our case, we are lowering to a combination of the // this lowering. In our case, we are lowering to a combination of the
// `Affine`, `Arithmetic`, `MemRef`, and `Standard` dialects. // `Affine`, `Arithmetic`, `Func`, and `MemRef` dialects.
target.addLegalDialect<AffineDialect, arith::ArithmeticDialect, target.addLegalDialect<AffineDialect, arith::ArithmeticDialect,
memref::MemRefDialect, StandardOpsDialect>(); func::FuncDialect, memref::MemRefDialect>();
// We also define the Toy dialect as Illegal so that the conversion will fail // We also define the Toy dialect as Illegal so that the conversion will fail
// if any of these operations are *not* converted. Given that we actually want // if any of these operations are *not* converted. Given that we actually want
// a partial lowering, we explicitly mark the Toy operations that don't want // a partial lowering, we explicitly mark the Toy operations that don't want
// to lower, `toy.print`, as `legal`. `toy.print` will still need its operands // to lower, `toy.print`, as `legal`. `toy.print` will still need its operands
// to be updated though (as we convert from TensorType to MemRefType), so we // to be updated though (as we convert from TensorType to MemRefType), so we
// only treat it as `legal` if its operands are legal. // only treat it as `legal` if its operands are legal.
target.addIllegalDialect<toy::ToyDialect>(); target.addIllegalDialect<toy::ToyDialect>();
Show All 25 Lines

mlir/examples/toy/Ch7/mlir/LowerToLLVM.cpp

Show All 29 Lines
#include "mlir/Conversion/LLVMCommon/ConversionTarget.h" #include "mlir/Conversion/LLVMCommon/ConversionTarget.h"
#include "mlir/Conversion/LLVMCommon/TypeConverter.h" #include "mlir/Conversion/LLVMCommon/TypeConverter.h"
#include "mlir/Conversion/MemRefToLLVM/MemRefToLLVM.h" #include "mlir/Conversion/MemRefToLLVM/MemRefToLLVM.h"
#include "mlir/Conversion/SCFToControlFlow/SCFToControlFlow.h" #include "mlir/Conversion/SCFToControlFlow/SCFToControlFlow.h"
#include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h" #include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h"
#include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVMPass.h" #include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVMPass.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/ADT/Sequence.h" #include "llvm/ADT/Sequence.h"
using namespace mlir; using namespace mlir;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// ToyToLLVM RewritePatterns // ToyToLLVM RewritePatterns
Show All 36 Lines for (unsigned i = 0, e = memRefShape.size(); i != e; ++i) {
rewriter.eraseOp(&nested); rewriter.eraseOp(&nested);
loopIvs.push_back(loop.getInductionVar()); loopIvs.push_back(loop.getInductionVar());
// Terminate the loop body. // Terminate the loop body.
rewriter.setInsertionPointToEnd(loop.getBody()); rewriter.setInsertionPointToEnd(loop.getBody());
// Insert a newline after each of the inner dimensions of the shape. // Insert a newline after each of the inner dimensions of the shape.
if (i != e - 1) if (i != e - 1)
rewriter.create<CallOp>(loc, printfRef, rewriter.getIntegerType(32), rewriter.create<func::CallOp>(loc, printfRef,
newLineCst); rewriter.getIntegerType(32), newLineCst);
rewriter.create<scf::YieldOp>(loc); rewriter.create<scf::YieldOp>(loc);
rewriter.setInsertionPointToStart(loop.getBody()); rewriter.setInsertionPointToStart(loop.getBody());
} }
// Generate a call to printf for the current element of the loop. // Generate a call to printf for the current element of the loop.
auto printOp = cast<toy::PrintOp>(op); auto printOp = cast<toy::PrintOp>(op);
auto elementLoad = auto elementLoad =
rewriter.create<memref::LoadOp>(loc, printOp.input(), loopIvs); rewriter.create<memref::LoadOp>(loc, printOp.input(), loopIvs);
rewriter.create<CallOp>(loc, printfRef, rewriter.getIntegerType(32), rewriter.create<func::CallOp>(
loc, printfRef, rewriter.getIntegerType(32),
ArrayRef<Value>({formatSpecifierCst, elementLoad})); ArrayRef<Value>({formatSpecifierCst, elementLoad}));
// Notify the rewriter that this operation has been removed. // Notify the rewriter that this operation has been removed.
rewriter.eraseOp(op); rewriter.eraseOp(op);
return success(); return success();
} }
private: private:
/// Return a symbol reference to the printf function, inserting it into the /// Return a symbol reference to the printf function, inserting it into the
▲ Show 20 Lines • Show All 113 Lines • Show Last 20 Lines

mlir/include/mlir-c/Dialect/Func.h

  • This file was moved from mlir/include/mlir-c/Dialect/Standard.h.
//===-- mlir-c/Dialect/Standard.h - C API for Standard dialect ----*- C -*-===// //===-- mlir-c/Dialect/Func.h - C API for Func dialect ------------*- C -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM // Part of the LLVM Project, under the Apache License v2.0 with LLVM
// Exceptions. // Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This header declares the C interface for registering and accessing the // This header declares the C interface for registering and accessing the
// Standard dialect. A dialect should be registered with a context to make it // Func dialect. A dialect should be registered with a context to make it
// available to users of the context. These users must load the dialect // available to users of the context. These users must load the dialect
// before using any of its attributes, operations or types. Parser and pass // before using any of its attributes, operations or types. Parser and pass
// manager can load registered dialects automatically. // manager can load registered dialects automatically.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef MLIR_C_DIALECT_STANDARD_H #ifndef MLIR_C_DIALECT_FUNC_H
#define MLIR_C_DIALECT_STANDARD_H #define MLIR_C_DIALECT_FUNC_H
#include "mlir-c/Registration.h" #include "mlir-c/Registration.h"
#ifdef __cplusplus #ifdef __cplusplus
extern "C" { extern "C" {
#endif #endif
MLIR_DECLARE_CAPI_DIALECT_REGISTRATION(Standard, std); MLIR_DECLARE_CAPI_DIALECT_REGISTRATION(Func, func);
#ifdef __cplusplus #ifdef __cplusplus
} }
#endif #endif
#endif // MLIR_C_DIALECT_STANDARD_H #endif // MLIR_C_DIALECT_FUNC_H

mlir/include/mlir-c/Dialect/Standard.h

  • This file was moved to mlir/include/mlir-c/Dialect/Func.h.

mlir/include/mlir-c/Registration.h

Show All 14 Lines
#ifdef __cplusplus #ifdef __cplusplus
extern "C" { extern "C" {
#endif #endif
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Dialect registration declarations. // Dialect registration declarations.
// Registration entry-points for each dialect are declared using the common // Registration entry-points for each dialect are declared using the common
// MLIR_DECLARE_DIALECT_REGISTRATION_CAPI macro, which takes the dialect // MLIR_DECLARE_DIALECT_REGISTRATION_CAPI macro, which takes the dialect
// API name (i.e. "Standard", "Tensor", "Linalg") and namespace (i.e. "std", // API name (i.e. "Func", "Tensor", "Linalg") and namespace (i.e. "func",
// "tensor", "linalg"). The following declarations are produced: // "tensor", "linalg"). The following declarations are produced:
// //
// /// Gets the above hook methods in struct form for a dialect by namespace. // /// Gets the above hook methods in struct form for a dialect by namespace.
// /// This is intended to facilitate dynamic lookup and registration of // /// This is intended to facilitate dynamic lookup and registration of
// /// dialects via a plugin facility based on shared library symbol lookup. // /// dialects via a plugin facility based on shared library symbol lookup.
// const MlirDialectHandle *mlirGetDialectHandle__{NAMESPACE}__(); // const MlirDialectHandle *mlirGetDialectHandle__{NAMESPACE}__();
// //
// This is done via a common macro to facilitate future expansion to // This is done via a common macro to facilitate future expansion to
▲ Show 20 Lines • Show All 44 Lines • Show Last 20 Lines

mlir/include/mlir/Conversion/LLVMCommon/LoweringOptions.h

Show All 19 Lines
class DataLayout; class DataLayout;
class MLIRContext; class MLIRContext;
/// Value to pass as bitwidth for the index type when the converter is expected /// Value to pass as bitwidth for the index type when the converter is expected
/// to derive the bitwidth from the LLVM data layout. /// to derive the bitwidth from the LLVM data layout.
static constexpr unsigned kDeriveIndexBitwidthFromDataLayout = 0; static constexpr unsigned kDeriveIndexBitwidthFromDataLayout = 0;
/// Options to control the Standard dialect to LLVM lowering. The struct is used /// Options to control the LLVM lowering. The struct is used to share lowering
/// to share lowering options between passes, patterns, and type converter. /// options between passes, patterns, and type converter.
class LowerToLLVMOptions { class LowerToLLVMOptions {
public: public:
explicit LowerToLLVMOptions(MLIRContext *ctx); explicit LowerToLLVMOptions(MLIRContext *ctx);
LowerToLLVMOptions(MLIRContext *ctx, const DataLayout &dl); LowerToLLVMOptions(MLIRContext *ctx, const DataLayout &dl);
bool useBarePtrCallConv = false; bool useBarePtrCallConv = false;
bool emitCWrappers = false; bool emitCWrappers = false;
Show All 36 Lines

mlir/include/mlir/Conversion/LLVMCommon/TypeConverter.h

Show All 20 Lines
class DataLayoutAnalysis; class DataLayoutAnalysis;
class LowerToLLVMOptions; class LowerToLLVMOptions;
namespace LLVM { namespace LLVM {
class LLVMDialect; class LLVMDialect;
} // namespace LLVM } // namespace LLVM
/// Conversion from types in the Standard dialect to the LLVM IR dialect. /// Conversion from types to the LLVM IR dialect.
class LLVMTypeConverter : public TypeConverter { class LLVMTypeConverter : public TypeConverter {
/// Give structFuncArgTypeConverter access to memref-specific functions. /// Give structFuncArgTypeConverter access to memref-specific functions.
friend LogicalResult friend LogicalResult
structFuncArgTypeConverter(LLVMTypeConverter &converter, Type type, structFuncArgTypeConverter(LLVMTypeConverter &converter, Type type,
SmallVectorImpl<Type> &result); SmallVectorImpl<Type> &result);
public: public:
using TypeConverter::convertType; using TypeConverter::convertType;
▲ Show 20 Lines • Show All 190 Lines • Show Last 20 Lines

mlir/include/mlir/Conversion/LinalgToStandard/LinalgToStandard.h

Show All 14 Lines
namespace mlir { namespace mlir {
class ModuleOp; class ModuleOp;
template <typename T> template <typename T>
class OperationPass; class OperationPass;
namespace linalg { namespace linalg {
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Patterns to convert a LinalgOp to std.call @external library implementation. // Patterns to convert a LinalgOp to func.call @external library implementation.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// These patterns are exposed individually because they are expected to be // These patterns are exposed individually because they are expected to be
// typically used individually. // typically used individually.
// Create a new call to the type-canonicalized `LinalgOp::getLibraryCallName()` // Create a new call to the type-canonicalized `LinalgOp::getLibraryCallName()`
// function. The implementation of the function can be either in the same module // function. The implementation of the function can be either in the same module
// or in an externally linked library. // or in an externally linked library.
// This is a generic entry point for all LinalgOp, except for CopyOp, for which // This is a generic entry point for all LinalgOp, except for CopyOp, for which
Show All 21 Lines

mlir/include/mlir/Conversion/Passes.td

Show First 20 Lines • Show All 61 Lines • ▼ Show 20 Lines let description = [{
- Functions without a body are not modified. - Functions without a body are not modified.
- The semantics of the other functions is preserved. - The semantics of the other functions is preserved.
- Individual operations other than those mentioned above are not modified - Individual operations other than those mentioned above are not modified
if they do not depend on the loop iterator value or on the result of if they do not depend on the loop iterator value or on the result of
`affine.apply`. `affine.apply`.
}]; }];
let constructor = "mlir::createLowerAffinePass()"; let constructor = "mlir::createLowerAffinePass()";
let dependentDialects = [ let dependentDialects = [
"func::FuncDialect",
"memref::MemRefDialect", "memref::MemRefDialect",
"scf::SCFDialect", "scf::SCFDialect",
"StandardOpsDialect",
"vector::VectorDialect" "vector::VectorDialect"
]; ];
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// ArithmeticToLLVM // ArithmeticToLLVM
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 141 Lines • ▼ Show 20 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// GPUToNVVM // GPUToNVVM
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
def ConvertGpuOpsToNVVMOps : Pass<"convert-gpu-to-nvvm", "gpu::GPUModuleOp"> { def ConvertGpuOpsToNVVMOps : Pass<"convert-gpu-to-nvvm", "gpu::GPUModuleOp"> {
let summary = "Generate NVVM operations for gpu operations"; let summary = "Generate NVVM operations for gpu operations";
let constructor = "mlir::createLowerGpuOpsToNVVMOpsPass()"; let constructor = "mlir::createLowerGpuOpsToNVVMOpsPass()";
let dependentDialects = [ let dependentDialects = [
"func::FuncDialect",
"memref::MemRefDialect", "memref::MemRefDialect",
"NVVM::NVVMDialect", "NVVM::NVVMDialect",
"StandardOpsDialect",
]; ];
let options = [ let options = [
Option<"indexBitwidth", "index-bitwidth", "unsigned", Option<"indexBitwidth", "index-bitwidth", "unsigned",
/*default=kDeriveIndexBitwidthFromDataLayout*/"0", /*default=kDeriveIndexBitwidthFromDataLayout*/"0",
"Bitwidth of the index type, 0 to use size of machine word"> "Bitwidth of the index type, 0 to use size of machine word">
]; ];
} }
▲ Show 20 Lines • Show All 81 Lines • ▼ Show 20 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// LinalgToStandard // LinalgToStandard
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
def ConvertLinalgToStandard : Pass<"convert-linalg-to-std", "ModuleOp"> { def ConvertLinalgToStandard : Pass<"convert-linalg-to-std", "ModuleOp"> {
let summary = "Convert the operations from the linalg dialect into the " let summary = "Convert the operations from the linalg dialect into the "
"Standard dialect"; "Standard dialect";
let constructor = "mlir::createConvertLinalgToStandardPass()"; let constructor = "mlir::createConvertLinalgToStandardPass()";
let dependentDialects = ["memref::MemRefDialect", "StandardOpsDialect"]; let dependentDialects = ["func::FuncDialect", "memref::MemRefDialect"];
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// LinalgToSPIRV // LinalgToSPIRV
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
def ConvertLinalgToSPIRV : Pass<"convert-linalg-to-spirv", "ModuleOp"> { def ConvertLinalgToSPIRV : Pass<"convert-linalg-to-spirv", "ModuleOp"> {
let summary = "Convert Linalg dialect to SPIR-V dialect"; let summary = "Convert Linalg dialect to SPIR-V dialect";
Show All 12 Lines
def ConvertMathToLibm : Pass<"convert-math-to-libm", "ModuleOp"> { def ConvertMathToLibm : Pass<"convert-math-to-libm", "ModuleOp"> {
let summary = "Convert Math dialect to libm calls"; let summary = "Convert Math dialect to libm calls";
let description = [{ let description = [{
This pass converts supported Math ops to libm calls. This pass converts supported Math ops to libm calls.
}]; }];
let constructor = "mlir::createConvertMathToLibmPass()"; let constructor = "mlir::createConvertMathToLibmPass()";
let dependentDialects = [ let dependentDialects = [
"arith::ArithmeticDialect", "arith::ArithmeticDialect",
"StandardOpsDialect", "func::FuncDialect",
"vector::VectorDialect", "vector::VectorDialect",
]; ];
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// MathToLLVM // MathToLLVM
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 179 Lines • ▼ Show 20 Lines
// ShapeToStandard // ShapeToStandard
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
def ConvertShapeToStandard : Pass<"convert-shape-to-std", "ModuleOp"> { def ConvertShapeToStandard : Pass<"convert-shape-to-std", "ModuleOp"> {
let summary = "Convert operations from the shape dialect into the standard " let summary = "Convert operations from the shape dialect into the standard "
"dialect"; "dialect";
let constructor = "mlir::createConvertShapeToStandardPass()"; let constructor = "mlir::createConvertShapeToStandardPass()";
let dependentDialects = [ let dependentDialects = [
"StandardOpsDialect", "func::FuncDialect",
"scf::SCFDialect", "scf::SCFDialect",
]; ];
} }
def ConvertShapeConstraints: Pass<"convert-shape-constraints", "FuncOp"> { def ConvertShapeConstraints: Pass<"convert-shape-constraints", "FuncOp"> {
let summary = "Convert shape constraint operations to the standard dialect"; let summary = "Convert shape constraint operations to the standard dialect";
let description = [{ let description = [{
This pass eliminates shape constraints from the program, converting them to This pass eliminates shape constraints from the program, converting them to
▲ Show 20 Lines • Show All 135 Lines • ▼ Show 20 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// TosaToStandard // TosaToStandard
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
def TosaToStandard : Pass<"tosa-to-standard"> { def TosaToStandard : Pass<"tosa-to-standard"> {
let summary = "Lower TOSA to the Standard dialect"; let summary = "Lower TOSA to the Standard dialect";
let dependentDialects = [ let dependentDialects = [
"arith::ArithmeticDialect", "arith::ArithmeticDialect",
"StandardOpsDialect", "func::FuncDialect",
"tensor::TensorDialect", "tensor::TensorDialect",
]; ];
let description = [{ let description = [{
Pass that converts TOSA operations to the equivalent operations using the Pass that converts TOSA operations to the equivalent operations using the
operations in the Standard dialect. operations in the Standard dialect.
}]; }];
let constructor = "tosa::createTosaToStandard()"; let constructor = "tosa::createTosaToStandard()";
▲ Show 20 Lines • Show All 122 Lines • Show Last 20 Lines

mlir/include/mlir/Conversion/VectorToSCF/VectorToSCF.h

Show First 20 Lines • Show All 68 Lines • ▼ Show 20 Linesstruct VectorTransferToSCFOptions {
/// Triggers full unrolling (vs iterating with a loop) during transfer to scf. /// Triggers full unrolling (vs iterating with a loop) during transfer to scf.
bool unroll = false; bool unroll = false;
VectorTransferToSCFOptions &enableFullUnroll(bool u = true) { VectorTransferToSCFOptions &enableFullUnroll(bool u = true) {
unroll = u; unroll = u;
return *this; return *this;
} }
}; };
/// Collect a set of patterns to convert from the Vector dialect to SCF + std. /// Collect a set of patterns to convert from the Vector dialect to SCF + func.
void populateVectorToSCFConversionPatterns( void populateVectorToSCFConversionPatterns(
RewritePatternSet &patterns, RewritePatternSet &patterns,
const VectorTransferToSCFOptions &options = VectorTransferToSCFOptions()); const VectorTransferToSCFOptions &options = VectorTransferToSCFOptions());
/// Create a pass to convert a subset of vector ops to SCF. /// Create a pass to convert a subset of vector ops to SCF.
std::unique_ptr<Pass> createConvertVectorToSCFPass( std::unique_ptr<Pass> createConvertVectorToSCFPass(
const VectorTransferToSCFOptions &options = VectorTransferToSCFOptions()); const VectorTransferToSCFOptions &options = VectorTransferToSCFOptions());
} // namespace mlir } // namespace mlir
#endif // MLIR_CONVERSION_VECTORTOSCF_VECTORTOSCF_H_ #endif // MLIR_CONVERSION_VECTORTOSCF_VECTORTOSCF_H_

mlir/include/mlir/Dialect/Affine/Analysis/AffineAnalysis.h

Show All 10 Lines
// and other IR structures that in turn use these. // and other IR structures that in turn use these.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef MLIR_DIALECT_AFFINE_ANALYSIS_AFFINEANALYSIS_H #ifndef MLIR_DIALECT_AFFINE_ANALYSIS_AFFINEANALYSIS_H
#define MLIR_DIALECT_AFFINE_ANALYSIS_AFFINEANALYSIS_H #define MLIR_DIALECT_AFFINE_ANALYSIS_AFFINEANALYSIS_H
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/Value.h" #include "mlir/IR/Value.h"
#include "llvm/ADT/Optional.h" #include "llvm/ADT/Optional.h"
#include "llvm/ADT/SmallVector.h" #include "llvm/ADT/SmallVector.h"
namespace mlir { namespace mlir {
class AffineApplyOp; class AffineApplyOp;
class AffineForOp; class AffineForOp;
▲ Show 20 Lines • Show All 161 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Arithmetic/Transforms/Passes.td

Show All 25 Lines let options = [
Option<"alignment", "alignment", "unsigned", /*default=*/"0", Option<"alignment", "alignment", "unsigned", /*default=*/"0",
"Create global memrefs with a specified alignment">, "Create global memrefs with a specified alignment">,
]; ];
} }
def ArithmeticExpandOps : Pass<"arith-expand"> { def ArithmeticExpandOps : Pass<"arith-expand"> {
let summary = "Legalize Arithmetic ops to be convertible to LLVM."; let summary = "Legalize Arithmetic ops to be convertible to LLVM.";
let constructor = "mlir::arith::createArithmeticExpandOpsPass()"; let constructor = "mlir::arith::createArithmeticExpandOpsPass()";
let dependentDialects = ["StandardOpsDialect"]; let dependentDialects = ["func::FuncDialect"];
} }
#endif // MLIR_DIALECT_ARITHMETIC_TRANSFORMS_PASSES #endif // MLIR_DIALECT_ARITHMETIC_TRANSFORMS_PASSES

mlir/include/mlir/Dialect/ArmSVE/ArmSVEDialect.h

Show All 13 Lines
#define MLIR_DIALECT_ARMSVE_ARMSVEDIALECT_H #define MLIR_DIALECT_ARMSVE_ARMSVEDIALECT_H
#include "mlir/IR/BuiltinTypes.h" #include "mlir/IR/BuiltinTypes.h"
#include "mlir/IR/Dialect.h" #include "mlir/IR/Dialect.h"
#include "mlir/IR/OpDefinition.h" #include "mlir/IR/OpDefinition.h"
#include "mlir/Interfaces/SideEffectInterfaces.h" #include "mlir/Interfaces/SideEffectInterfaces.h"
#include "mlir/Dialect/ArmSVE/ArmSVEDialect.h.inc" #include "mlir/Dialect/ArmSVE/ArmSVEDialect.h.inc"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#define GET_OP_CLASSES #define GET_OP_CLASSES
#include "mlir/Dialect/ArmSVE/ArmSVE.h.inc" #include "mlir/Dialect/ArmSVE/ArmSVE.h.inc"
#endif // MLIR_DIALECT_ARMSVE_ARMSVEDIALECT_H #endif // MLIR_DIALECT_ARMSVE_ARMSVEDIALECT_H

mlir/include/mlir/Dialect/CMakeLists.txt

add_subdirectory(Affine) add_subdirectory(Affine)
add_subdirectory(Arithmetic) add_subdirectory(Arithmetic)
add_subdirectory(Async) add_subdirectory(Async)
add_subdirectory(ArmNeon) add_subdirectory(ArmNeon)
add_subdirectory(ArmSVE) add_subdirectory(ArmSVE)
add_subdirectory(AMX) add_subdirectory(AMX)
add_subdirectory(Bufferization) add_subdirectory(Bufferization)
add_subdirectory(Complex) add_subdirectory(Complex)
add_subdirectory(ControlFlow) add_subdirectory(ControlFlow)
add_subdirectory(DLTI) add_subdirectory(DLTI)
add_subdirectory(EmitC) add_subdirectory(EmitC)
add_subdirectory(Func)
add_subdirectory(GPU) add_subdirectory(GPU)
add_subdirectory(Math) add_subdirectory(Math)
add_subdirectory(Linalg) add_subdirectory(Linalg)
add_subdirectory(LLVMIR) add_subdirectory(LLVMIR)
add_subdirectory(MemRef) add_subdirectory(MemRef)
add_subdirectory(OpenACC) add_subdirectory(OpenACC)
add_subdirectory(OpenMP) add_subdirectory(OpenMP)
add_subdirectory(PDL) add_subdirectory(PDL)
add_subdirectory(PDLInterp) add_subdirectory(PDLInterp)
add_subdirectory(Quant) add_subdirectory(Quant)
add_subdirectory(SCF) add_subdirectory(SCF)
add_subdirectory(Shape) add_subdirectory(Shape)
add_subdirectory(SparseTensor) add_subdirectory(SparseTensor)
add_subdirectory(SPIRV) add_subdirectory(SPIRV)
add_subdirectory(StandardOps)
add_subdirectory(Tensor) add_subdirectory(Tensor)
add_subdirectory(Tosa) add_subdirectory(Tosa)
add_subdirectory(Vector) add_subdirectory(Vector)
add_subdirectory(X86Vector) add_subdirectory(X86Vector)

mlir/include/mlir/Dialect/ControlFlow/IR/ControlFlowOps.td

//===- ControlFlowOps.td - ControlFlow operations ----------*- tablegen -*-===// //===- ControlFlowOps.td - ControlFlow operations ----------*- tablegen -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file contains definitions for the operations within the ControlFlow // This file contains definitions for the operations within the ControlFlow
// dialect. // dialect.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef STANDARD_OPS #ifndef MLIR_DIALECTS_CONTROLFLOW_IR_CONTROLFLOWOPS_TD
#define STANDARD_OPS #define MLIR_DIALECTS_CONTROLFLOW_IR_CONTROLFLOWOPS_TD
include "mlir/IR/OpAsmInterface.td" include "mlir/IR/OpAsmInterface.td"
include "mlir/Interfaces/ControlFlowInterfaces.td" include "mlir/Interfaces/ControlFlowInterfaces.td"
include "mlir/Interfaces/SideEffectInterfaces.td" include "mlir/Interfaces/SideEffectInterfaces.td"
def ControlFlow_Dialect : Dialect { def ControlFlow_Dialect : Dialect {
let name = "cf"; let name = "cf";
let cppNamespace = "::mlir::cf"; let cppNamespace = "::mlir::cf";
▲ Show 20 Lines • Show All 281 Lines • ▼ Show 20 Lines MutableOperandRange getCaseOperandsMutable(unsigned index) {
return getCaseOperandsMutable()[index]; return getCaseOperandsMutable()[index];
} }
}]; }];
let hasCanonicalizer = 1; let hasCanonicalizer = 1;
let hasVerifier = 1; let hasVerifier = 1;
} }
#endif // STANDARD_OPS #endif // MLIR_DIALECTS_CONTROLFLOW_IR_CONTROLFLOWOPS_TD

mlir/include/mlir/Dialect/Func/CMakeLists.txt

  • This file was copied from mlir/include/mlir/Dialect/StandardOps/CMakeLists.txt.
add_subdirectory(IR) add_subdirectory(IR)
add_subdirectory(Transforms) add_subdirectory(Transforms)

mlir/include/mlir/Dialect/Func/IR/CMakeLists.txt

  • This file was added.
set(LLVM_TARGET_DEFINITIONS FuncOps.td)
mlir_tablegen(FuncOps.h.inc -gen-op-decls)
mlir_tablegen(FuncOps.cpp.inc -gen-op-defs)
mlir_tablegen(FuncOpsDialect.h.inc -gen-dialect-decls)
mlir_tablegen(FuncOpsDialect.cpp.inc -gen-dialect-defs)
add_public_tablegen_target(MLIRFuncOpsIncGen)
add_mlir_doc(FuncOps FuncOps Dialects/ -gen-op-doc)

mlir/include/mlir/Dialect/Func/IR/FuncOps.h

  • This file was moved from mlir/include/mlir/Dialect/StandardOps/IR/Ops.h.
//===- Ops.h - Standard MLIR Operations -------------------------*- C++ -*-===// //===- FuncOps.h - Func Dialect Operations ----------------------*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
//
// This file defines convenience types for working with standard operations
// in the MLIR operation set.
//
//===----------------------------------------------------------------------===//
#ifndef MLIR_DIALECT_STANDARDOPS_IR_OPS_H #ifndef MLIR_DIALECT_FUNC_IR_OPS_H
#define MLIR_DIALECT_STANDARDOPS_IR_OPS_H #define MLIR_DIALECT_FUNC_IR_OPS_H
#include "mlir/Dialect/ControlFlow/IR/ControlFlow.h" #include "mlir/Dialect/ControlFlow/IR/ControlFlow.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/BuiltinTypes.h" #include "mlir/IR/BuiltinTypes.h"
#include "mlir/IR/Dialect.h" #include "mlir/IR/Dialect.h"
#include "mlir/IR/OpImplementation.h" #include "mlir/IR/OpImplementation.h"
#include "mlir/Interfaces/CallInterfaces.h" #include "mlir/Interfaces/CallInterfaces.h"
#include "mlir/Interfaces/CastInterfaces.h"
#include "mlir/Interfaces/ControlFlowInterfaces.h" #include "mlir/Interfaces/ControlFlowInterfaces.h"
#include "mlir/Interfaces/InferTypeOpInterface.h" #include "mlir/Interfaces/InferTypeOpInterface.h"
#include "mlir/Interfaces/SideEffectInterfaces.h" #include "mlir/Interfaces/SideEffectInterfaces.h"
// Pull in all enum type definitions and utility function declarations.
#include "mlir/Dialect/StandardOps/IR/OpsEnums.h.inc"
namespace mlir { namespace mlir {
class AffineMap;
class Builder;
class FuncOp;
class OpBuilder;
class PatternRewriter; class PatternRewriter;
} // namespace mlir } // namespace mlir
#define GET_OP_CLASSES #define GET_OP_CLASSES
#include "mlir/Dialect/StandardOps/IR/Ops.h.inc" #include "mlir/Dialect/Func/IR/FuncOps.h.inc"
#include "mlir/Dialect/StandardOps/IR/OpsDialect.h.inc" #include "mlir/Dialect/Func/IR/FuncOpsDialect.h.inc"
#endif // MLIR_DIALECT_STANDARDOPS_IR_OPS_H #endif // MLIR_DIALECT_FUNC_IR_OPS_H

mlir/include/mlir/Dialect/Func/IR/FuncOps.td

  • This file was moved from mlir/include/mlir/Dialect/StandardOps/IR/Ops.td.
//===- Ops.td - Standard operation definitions -------------*- tablegen -*-===// //===- FuncOps.td - Func operation definitions -------------*- tablegen -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
//
// Defines some MLIR standard operations.
//
//===----------------------------------------------------------------------===//
#ifndef STANDARD_OPS #ifndef MLIR_DIALECT_FUNC_IR_FUNCOPS_TD
#define STANDARD_OPS #define MLIR_DIALECT_FUNC_IR_FUNCOPS_TD
include "mlir/IR/OpAsmInterface.td" include "mlir/IR/OpAsmInterface.td"
include "mlir/IR/SymbolInterfaces.td" include "mlir/IR/SymbolInterfaces.td"
include "mlir/Interfaces/CallInterfaces.td" include "mlir/Interfaces/CallInterfaces.td"
include "mlir/Interfaces/CastInterfaces.td"
include "mlir/Interfaces/ControlFlowInterfaces.td" include "mlir/Interfaces/ControlFlowInterfaces.td"
include "mlir/Interfaces/InferTypeOpInterface.td" include "mlir/Interfaces/InferTypeOpInterface.td"
include "mlir/Interfaces/SideEffectInterfaces.td" include "mlir/Interfaces/SideEffectInterfaces.td"
def StandardOps_Dialect : Dialect { def Func_Dialect : Dialect {
let name = "std"; let name = "func";
let cppNamespace = "::mlir"; let cppNamespace = "::mlir::func";
let dependentDialects = ["cf::ControlFlowDialect"]; let dependentDialects = ["cf::ControlFlowDialect"];
let hasConstantMaterializer = 1; let hasConstantMaterializer = 1;
let emitAccessorPrefix = kEmitAccessorPrefix_Prefixed; let emitAccessorPrefix = kEmitAccessorPrefix_Prefixed;
} }
// Base class for Standard dialect ops. // Base class for Func dialect ops.
class Std_Op<string mnemonic, list<Trait> traits = []> : class Func_Op<string mnemonic, list<Trait> traits = []> :
Op<StandardOps_Dialect, mnemonic, traits>; Op<Func_Dialect, mnemonic, traits>;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// CallOp // CallOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
def CallOp : Std_Op<"call", def CallOp : Func_Op<"call",
[CallOpInterface, MemRefsNormalizable, [CallOpInterface, MemRefsNormalizable,
DeclareOpInterfaceMethods<SymbolUserOpInterface>]> { DeclareOpInterfaceMethods<SymbolUserOpInterface>]> {
let summary = "call operation"; let summary = "call operation";
let description = [{ let description = [{
The `call` operation represents a direct call to a function that is within The `func.call` operation represents a direct call to a function that is
the same symbol scope as the call. The operands and result types of the within the same symbol scope as the call. The operands and result types of
call must match the specified function type. The callee is encoded as a the call must match the specified function type. The callee is encoded as a
symbol reference attribute named "callee". symbol reference attribute named "callee".
Example: Example:
```mlir ```mlir
%2 = call @my_add(%0, %1) : (f32, f32) -> f32 %2 = func.call @my_add(%0, %1) : (f32, f32) -> f32
``` ```
}]; }];
let arguments = (ins FlatSymbolRefAttr:$callee, Variadic<AnyType>:$operands); let arguments = (ins FlatSymbolRefAttr:$callee, Variadic<AnyType>:$operands);
let results = (outs Variadic<AnyType>); let results = (outs Variadic<AnyType>);
let builders = [ let builders = [
OpBuilder<(ins "FuncOp":$callee, CArg<"ValueRange", "{}">:$operands), [{ OpBuilder<(ins "FuncOp":$callee, CArg<"ValueRange", "{}">:$operands), [{
Show All 38 Lines let assemblyFormat = [{
$callee `(` $operands `)` attr-dict `:` functional-type($operands, results) $callee `(` $operands `)` attr-dict `:` functional-type($operands, results)
}]; }];
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// CallIndirectOp // CallIndirectOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
def CallIndirectOp : Std_Op<"call_indirect", [ def CallIndirectOp : Func_Op<"call_indirect", [
CallOpInterface, CallOpInterface,
TypesMatchWith<"callee input types match argument types", TypesMatchWith<"callee input types match argument types",
"callee", "callee_operands", "callee", "callee_operands",
"$_self.cast<FunctionType>().getInputs()">, "$_self.cast<FunctionType>().getInputs()">,
TypesMatchWith<"callee result types match result types", TypesMatchWith<"callee result types match result types",
"callee", "results", "callee", "results",
"$_self.cast<FunctionType>().getResults()"> "$_self.cast<FunctionType>().getResults()">
]> { ]> {
let summary = "indirect call operation"; let summary = "indirect call operation";
let description = [{ let description = [{
The `call_indirect` operation represents an indirect call to a value of The `func.call_indirect` operation represents an indirect call to a value
function type. Functions are first class types in MLIR, and may be passed as of function type. The operands and result types of the call must match the
arguments and merged together with block arguments. The operands and result specified function type.
types of the call must match the specified function type.
Function values can be created with the Function values can be created with the
[`constant` operation](#stdconstant-constantop). [`func.constant` operation](#funcconstant-constantop).
Example: Example:
```mlir ```mlir
%31 = call_indirect %15(%0, %1) %func = func.constant @my_func : (tensor<16xf32>, tensor<16xf32>) -> tensor<16xf32>
: (tensor<16xf32>, tensor<16xf32>) -> tensor<16xf32> %result = func.call_indirect %func(%0, %1) : (tensor<16xf32>, tensor<16xf32>) -> tensor<16xf32>
``` ```
}]; }];
let arguments = (ins FunctionType:$callee, let arguments = (ins FunctionType:$callee,
Variadic<AnyType>:$callee_operands); Variadic<AnyType>:$callee_operands);
let results = (outs Variadic<AnyType>:$results); let results = (outs Variadic<AnyType>:$results);
let builders = [ let builders = [
Show All 15 Lines let extraClassDeclaration = [{
operand_iterator arg_operand_begin() { return ++operand_begin(); } operand_iterator arg_operand_begin() { return ++operand_begin(); }
operand_iterator arg_operand_end() { return operand_end(); } operand_iterator arg_operand_end() { return operand_end(); }
/// Return the callee of this operation. /// Return the callee of this operation.
CallInterfaceCallable getCallableForCallee() { return getCallee(); } CallInterfaceCallable getCallableForCallee() { return getCallee(); }
}]; }];
let hasCanonicalizeMethod = 1; let hasCanonicalizeMethod = 1;
let assemblyFormat = [{
let assemblyFormat = $callee `(` $callee_operands `)` attr-dict `:` type($callee)
"$callee `(` $callee_operands `)` attr-dict `:` type($callee)"; }];
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// ConstantOp // ConstantOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
def ConstantOp : Std_Op<"constant", def ConstantOp : Func_Op<"constant",
[ConstantLike, NoSideEffect, [ConstantLike, NoSideEffect,
DeclareOpInterfaceMethods<OpAsmOpInterface, ["getAsmResultNames"]>]> { DeclareOpInterfaceMethods<OpAsmOpInterface, ["getAsmResultNames"]>]> {
let summary = "constant"; let summary = "constant";
let description = [{ let description = [{
Syntax: The `func.constant` operation produces an SSA value from a symbol reference
to a `func.func` operation
```
operation ::= ssa-id `=` `std.constant` attribute-value `:` type
```
The `constant` operation produces an SSA value from a symbol reference to a
`builtin.func` operation
Example: Example:
```mlir ```mlir
// Reference to function @myfn. // Reference to function @myfn.
%2 = constant @myfn : (tensor<16xf32>, f32) -> tensor<16xf32> %2 = func.constant @myfn : (tensor<16xf32>, f32) -> tensor<16xf32>
// Equivalent generic forms // Equivalent generic forms
%2 = "std.constant"() {value = @myfn} %2 = "func.constant"() { value = @myfn } : () -> ((tensor<16xf32>, f32) -> tensor<16xf32>)
: () -> ((tensor<16xf32>, f32) -> tensor<16xf32>)
``` ```
MLIR does not allow direct references to functions in SSA operands because MLIR does not allow direct references to functions in SSA operands because
the compiler is multithreaded, and disallowing SSA values to directly the compiler is multithreaded, and disallowing SSA values to directly
reference a function simplifies this reference a function simplifies this
([rationale](../Rationale/Rationale.md#multithreading-the-compiler)). ([rationale](../Rationale/Rationale.md#multithreading-the-compiler)).
}]; }];
Show All 10 Linesdef ConstantOp : Func_Op<"constant",
let hasFolder = 1; let hasFolder = 1;
let hasVerifier = 1; let hasVerifier = 1;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// ReturnOp // ReturnOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
def ReturnOp : Std_Op<"return", [NoSideEffect, HasParent<"FuncOp">, def ReturnOp : Func_Op<"return", [NoSideEffect, HasParent<"FuncOp">,
MemRefsNormalizable, ReturnLike, Terminator]> { MemRefsNormalizable, ReturnLike, Terminator]> {
let summary = "return operation"; let summary = "Function return operation";
let description = [{ let description = [{
The `return` operation represents a return operation within a function. The `func.return` operation represents a return operation within a function.
The operation takes variable number of operands and produces no results. The operation takes variable number of operands and produces no results.
The operand number and types must match the signature of the function The operand number and types must match the signature of the function
that contains the operation. that contains the operation.
Example: Example:
```mlir ```mlir
func @foo() : (i32, f8) { func.func @foo() : (i32, f8) {
... ...
return %0, %1 : i32, f8 return %0, %1 : i32, f8
} }
``` ```
}]; }];
let arguments = (ins Variadic<AnyType>:$operands); let arguments = (ins Variadic<AnyType>:$operands);
let builders = [ let builders = [OpBuilder<(ins), [{
OpBuilder<(ins), build($_builder, $_state, llvm::None);
[{ build($_builder, $_state, llvm::None); }]>]; }]>];
let assemblyFormat = "attr-dict ($operands^ `:` type($operands))?"; let assemblyFormat = "attr-dict ($operands^ `:` type($operands))?";
let hasVerifier = 1; let hasVerifier = 1;
} }
#endif // STANDARD_OPS #endif // MLIR_DIALECT_FUNC_IR_FUNCOPS_TD

mlir/include/mlir/Dialect/Func/Transforms/CMakeLists.txt

  • This file was added.
set(LLVM_TARGET_DEFINITIONS Passes.td)
mlir_tablegen(Passes.h.inc -gen-pass-decls -name Func)
add_public_tablegen_target(MLIRFuncTransformsIncGen)
add_mlir_doc(Passes FuncPasses ./ -gen-pass-doc)

mlir/include/mlir/Dialect/Func/Transforms/DecomposeCallGraphTypes.h

  • This file was moved from mlir/include/mlir/Dialect/StandardOps/Transforms/DecomposeCallGraphTypes.h.
Show All 9 Lines
// decomposing types for calls, returns, and function args. // decomposing types for calls, returns, and function args.
// //
// TODO: Make this handle dialect-defined functions, calls, and returns. // TODO: Make this handle dialect-defined functions, calls, and returns.
// Currently, the generic interfaces aren't sophisticated enough for the // Currently, the generic interfaces aren't sophisticated enough for the
// types of mutations that we are doing here. // types of mutations that we are doing here.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef MLIR_DIALECT_STANDARDOPS_TRANSFORMS_DECOMPOSECALLGRAPHTYPES_H #ifndef MLIR_DIALECT_FUNC_TRANSFORMS_DECOMPOSECALLGRAPHTYPES_H
#define MLIR_DIALECT_STANDARDOPS_TRANSFORMS_DECOMPOSECALLGRAPHTYPES_H #define MLIR_DIALECT_FUNC_TRANSFORMS_DECOMPOSECALLGRAPHTYPES_H
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
namespace mlir { namespace mlir {
/// This class provides a hook that expands one Value into multiple Value's, /// This class provides a hook that expands one Value into multiple Value's,
/// with a TypeConverter-inspired callback registration mechanism. /// with a TypeConverter-inspired callback registration mechanism.
/// ///
▲ Show 20 Lines • Show All 55 Lines • ▼ Show 20 Lines
/// decomposing call graph types with the given `ValueDecomposer`. /// decomposing call graph types with the given `ValueDecomposer`.
void populateDecomposeCallGraphTypesPatterns(MLIRContext *context, void populateDecomposeCallGraphTypesPatterns(MLIRContext *context,
TypeConverter &typeConverter, TypeConverter &typeConverter,
ValueDecomposer &decomposer, ValueDecomposer &decomposer,
RewritePatternSet &patterns); RewritePatternSet &patterns);
} // namespace mlir } // namespace mlir
#endif // MLIR_DIALECT_STANDARDOPS_TRANSFORMS_DECOMPOSECALLGRAPHTYPES_H #endif // MLIR_DIALECT_FUNC_TRANSFORMS_DECOMPOSECALLGRAPHTYPES_H

mlir/include/mlir/Dialect/Func/Transforms/FuncConversions.h

  • This file was moved from mlir/include/mlir/Dialect/StandardOps/Transforms/FuncConversions.h.
//===- FuncConversions.h - Patterns for converting std.funcs ----*- C++ -*-===// //===- FuncConversions.h - Patterns for converting func.func ----*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This files contains patterns for converting standard functions. // This files contains patterns for converting functions.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef MLIR_DIALECT_STANDARDOPS_TRANSFORMS_FUNCCONVERSIONS_H_ #ifndef MLIR_DIALECT_FUNC_TRANSFORMS_FUNCCONVERSIONS_H_
#define MLIR_DIALECT_STANDARDOPS_TRANSFORMS_FUNCCONVERSIONS_H_ #define MLIR_DIALECT_FUNC_TRANSFORMS_FUNCCONVERSIONS_H_
#include "mlir/Support/LLVM.h" #include "mlir/Support/LLVM.h"
#include "llvm/ADT/STLExtras.h" #include "llvm/ADT/STLExtras.h"
namespace mlir { namespace mlir {
// Forward declarations. // Forward declarations.
class BranchOpInterface; class BranchOpInterface;
▲ Show 20 Lines • Show All 42 Lines • ▼ Show 20 Lines
/// Return true if op is neither BranchOpInterface nor ReturnLike. /// Return true if op is neither BranchOpInterface nor ReturnLike.
/// ///
/// TODO Try to get rid of this function and invert the meaning of /// TODO Try to get rid of this function and invert the meaning of
/// `isLegalForBranchOpInterfaceTypeConversionPattern` and /// `isLegalForBranchOpInterfaceTypeConversionPattern` and
/// `isLegalForReturnOpTypeConversionPattern`. /// `isLegalForReturnOpTypeConversionPattern`.
bool isNotBranchOpInterfaceOrReturnLikeOp(Operation *op); bool isNotBranchOpInterfaceOrReturnLikeOp(Operation *op);
} // namespace mlir } // namespace mlir
#endif // MLIR_DIALECT_STANDARDOPS_TRANSFORMS_FUNCCONVERSIONS_H_ #endif // MLIR_DIALECT_FUNC_TRANSFORMS_FUNCCONVERSIONS_H_

mlir/include/mlir/Dialect/Func/Transforms/Passes.h

  • This file was moved from mlir/include/mlir/Dialect/StandardOps/Transforms/Passes.h.
//===- Passes.h - Pass Entrypoints ------------------------------*- C++ -*-===// //===- Passes.h - Pass Entrypoints ------------------------------*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This header file defines prototypes that expose pass constructors in the loop // This header file defines prototypes that expose pass constructors in the Func
// transformation library. // transformation library.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef MLIR_DIALECT_STANDARDOPS_TRANSFORMS_PASSES_H #ifndef MLIR_DIALECT_FUNC_TRANSFORMS_PASSES_H
#define MLIR_DIALECT_STANDARDOPS_TRANSFORMS_PASSES_H #define MLIR_DIALECT_FUNC_TRANSFORMS_PASSES_H
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
namespace mlir { namespace mlir {
namespace bufferization { namespace bufferization {
class BufferizeTypeConverter; class BufferizeTypeConverter;
} // namespace bufferization } // namespace bufferization
class RewritePatternSet; class RewritePatternSet;
namespace func {
/// Creates an instance of func bufferization pass. /// Creates an instance of func bufferization pass.
std::unique_ptr<Pass> createFuncBufferizePass(); std::unique_ptr<Pass> createFuncBufferizePass();
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Registration // Registration
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
/// Generate the code for registering passes. /// Generate the code for registering passes.
#define GEN_PASS_REGISTRATION #define GEN_PASS_REGISTRATION
#include "mlir/Dialect/StandardOps/Transforms/Passes.h.inc" #include "mlir/Dialect/Func/Transforms/Passes.h.inc"
} // namespace func
} // namespace mlir } // namespace mlir
#endif // MLIR_DIALECT_STANDARDOPS_TRANSFORMS_PASSES_H #endif // MLIR_DIALECT_FUNC_TRANSFORMS_PASSES_H

mlir/include/mlir/Dialect/Func/Transforms/Passes.td

  • This file was moved from mlir/include/mlir/Dialect/StandardOps/Transforms/Passes.td.
//===-- Passes.td - StandardOps pass definition file -------*- tablegen -*-===// //===-- Passes.td - Func pass definition file --------------*- tablegen -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef MLIR_DIALECT_STANDARD_TRANSFORMS_PASSES #ifndef MLIR_DIALECT_FUNC_TRANSFORMS_PASSES_TD
#define MLIR_DIALECT_STANDARD_TRANSFORMS_PASSES #define MLIR_DIALECT_FUNC_TRANSFORMS_PASSES_TD
include "mlir/Pass/PassBase.td" include "mlir/Pass/PassBase.td"
def FuncBufferize : Pass<"func-bufferize", "ModuleOp"> { def FuncBufferize : Pass<"func-bufferize", "ModuleOp"> {
let summary = "Bufferize func/call/return ops"; let summary = "Bufferize func/call/return ops";
let description = [{ let description = [{
A bufferize pass that bufferizes std.func and std.call ops. A bufferize pass that bufferizes builtin.func and func.call ops.
Because this pass updates std.func ops, it must be a module pass. It is Because this pass updates builtin.func ops, it must be a module pass. It is
useful to keep this pass separate from other bufferizations so that the useful to keep this pass separate from other bufferizations so that the
other ones can be run at function-level in parallel. other ones can be run at function-level in parallel.
This pass must be done atomically because it changes func op signatures, This pass must be done atomically because it changes func op signatures,
which requires atomically updating calls as well throughout the entire which requires atomically updating calls as well throughout the entire
module. module.
This pass also changes the type of block arguments, which requires that all This pass also changes the type of block arguments, which requires that all
successor arguments of predecessors be converted. This is achieved by successor arguments of predecessors be converted. This is achieved by
rewriting terminators based on the information provided by the rewriting terminators based on the information provided by the
`BranchOpInterface`. `BranchOpInterface`.
As this pass rewrites function operations, it also rewrites the As this pass rewrites function operations, it also rewrites the
corresponding return operations. Other return-like operations that corresponding return operations. Other return-like operations that
implement the `ReturnLike` trait are not rewritten in general, as they implement the `ReturnLike` trait are not rewritten in general, as they
require that the corresponding parent operation is also rewritten. require that the corresponding parent operation is also rewritten.
Finally, this pass fails for unknown terminators, as we cannot decide Finally, this pass fails for unknown terminators, as we cannot decide
whether they need rewriting. whether they need rewriting.
}]; }];
let constructor = "mlir::createFuncBufferizePass()"; let constructor = "mlir::func::createFuncBufferizePass()";
let dependentDialects = ["bufferization::BufferizationDialect", let dependentDialects = ["bufferization::BufferizationDialect",
"memref::MemRefDialect"]; "memref::MemRefDialect"];
} }
#endif // MLIR_DIALECT_STANDARD_TRANSFORMS_PASSES #endif // MLIR_DIALECT_FUNC_TRANSFORMS_PASSES_TD

mlir/include/mlir/Dialect/Linalg/Transforms/Transforms.h

Show First 20 Lines • Show All 538 Lines • ▼ Show 20 Lines
/// %c1 = arith.constant 1 : index /// %c1 = arith.constant 1 : index
/// %c0 = arith.constant 0 : index /// %c0 = arith.constant 0 : index
/// %c25 = arith.constant 25 : index /// %c25 = arith.constant 25 : index
/// %c10 = arith.constant 10 : index /// %c10 = arith.constant 10 : index
/// operand_dim_0 = dim %operand, 0 : memref<50x100xf32> /// operand_dim_0 = dim %operand, 0 : memref<50x100xf32>
/// operand_dim_1 = dim %operand, 1 : memref<50x100xf32> /// operand_dim_1 = dim %operand, 1 : memref<50x100xf32>
/// scf.for %k = %c0 to operand_dim_0 step %c10 { /// scf.for %k = %c0 to operand_dim_0 step %c10 {
/// scf.for %l = %c0 to operand_dim_1 step %c25 { /// scf.for %l = %c0 to operand_dim_1 step %c25 {
/// %4 = std.subview %operand[%k, %l][%c10, %c25][%c1, %c1] /// %4 = memref.subview %operand[%k, %l][%c10, %c25][%c1, %c1]
/// : memref<50x100xf32> to memref<?x?xf32, #strided> /// : memref<50x100xf32> to memref<?x?xf32, #strided>
/// %5 = std.subview %result[%k, %l][%c10, %c25][%c1, %c1] /// %5 = memref.subview %result[%k, %l][%c10, %c25][%c1, %c1]
/// : memref<50x100xf32> to memref<?x?xf32, #strided> /// : memref<50x100xf32> to memref<?x?xf32, #strided>
/// linalg.generic pointwise_2d_trait %4, %5 { /// linalg.generic pointwise_2d_trait %4, %5 {
/// ^bb0(%operand_in: f32, %result_in: f32): /// ^bb0(%operand_in: f32, %result_in: f32):
/// %i = linalg.index 0 : index /// %i = linalg.index 0 : index
/// %j = linalg.index 1 : index /// %j = linalg.index 1 : index
/// // Indices `k` and `l` are implicitly captured in the body. /// // Indices `k` and `l` are implicitly captured in the body.
/// %transformed_i = arith.addi %i, %k : index // index `i` is offset by /// %transformed_i = arith.addi %i, %k : index // index `i` is offset by
/// %k %transformed_j = arith.addi %j, %l : index // index `j` is offset /// %k %transformed_j = arith.addi %j, %l : index // index `j` is offset
▲ Show 20 Lines • Show All 935 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Linalg/Utils/Utils.h

//===- Utils.h - Utilities to support the Linalg dialect --------*- C++ -*-===// //===- Utils.h - Utilities to support the Linalg dialect --------*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef MLIR_DIALECT_LINALG_UTILS_UTILS_H #ifndef MLIR_DIALECT_LINALG_UTILS_UTILS_H
#define MLIR_DIALECT_LINALG_UTILS_UTILS_H #define MLIR_DIALECT_LINALG_UTILS_UTILS_H
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Linalg/Analysis/DependenceAnalysis.h" #include "mlir/Dialect/Linalg/Analysis/DependenceAnalysis.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "llvm/ADT/MapVector.h" #include "llvm/ADT/MapVector.h"
#include "llvm/ADT/SetVector.h" #include "llvm/ADT/SetVector.h"
namespace mlir { namespace mlir {
class AffineExpr; class AffineExpr;
class AffineForOp; class AffineForOp;
class AffineMap; class AffineMap;
class PatternRewriter; class PatternRewriter;
▲ Show 20 Lines • Show All 426 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Math/IR/MathOps.td

Show First 20 Lines • Show All 726 Lines • ▼ Show 20 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// TanhOp // TanhOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
def Math_TanhOp : Math_FloatUnaryOp<"tanh"> { def Math_TanhOp : Math_FloatUnaryOp<"tanh"> {
let summary = "hyperbolic tangent of the specified value"; let summary = "hyperbolic tangent of the specified value";
let description = [{ let description = [{
Syntax:
```
operation ::= ssa-id `=` `std.tanh` ssa-use `:` type
```
The `tanh` operation computes the hyperbolic tangent. It takes one operand The `tanh` operation computes the hyperbolic tangent. It takes one operand
and returns one result of the same type. This type may be a float scalar and returns one result of the same type. This type may be a float scalar
type, a vector whose element type is float, or a tensor of floats. It has type, a vector whose element type is float, or a tensor of floats. It has
no standard attributes. no standard attributes.
Example: Example:
```mlir ```mlir
Show All 13 Lines

mlir/include/mlir/Dialect/MemRef/IR/MemRefOps.td

Show First 20 Lines • Show All 896 Lines • ▼ Show 20 Lines let description = [{
output of load is a new value with the same type as the elements of the output of load is a new value with the same type as the elements of the
memref. The arity of indices is the rank of the memref (i.e., if the memref memref. The arity of indices is the rank of the memref (i.e., if the memref
loaded from is of rank 3, then 3 indices are required for the load following loaded from is of rank 3, then 3 indices are required for the load following
the memref identifier). the memref identifier).
In an `affine.if` or `affine.for` body, the indices of a load are restricted In an `affine.if` or `affine.for` body, the indices of a load are restricted
to SSA values bound to surrounding loop induction variables, to SSA values bound to surrounding loop induction variables,
[symbols](Affine.md/#dimensions-and-symbols), results of a [symbols](Affine.md/#dimensions-and-symbols), results of a
[`constant` operation](Standard.md/#stdconstant-constantop), or the result of an constant operations, or the result of an
`affine.apply` operation that can in turn take as arguments all of the `affine.apply` operation that can in turn take as arguments all of the
aforementioned SSA values or the recursively result of such an aforementioned SSA values or the recursively result of such an
`affine.apply` operation. `affine.apply` operation.
Example: Example:
```mlir ```mlir
%1 = affine.apply affine_map<(d0, d1) -> (3*d0)> (%i, %j) %1 = affine.apply affine_map<(d0, d1) -> (3*d0)> (%i, %j)
▲ Show 20 Lines • Show All 918 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/MemRef/Transforms/Passes.h

Show All 12 Lines
#ifndef MLIR_DIALECT_MEMREF_TRANSFORMS_PASSES_H #ifndef MLIR_DIALECT_MEMREF_TRANSFORMS_PASSES_H
#define MLIR_DIALECT_MEMREF_TRANSFORMS_PASSES_H #define MLIR_DIALECT_MEMREF_TRANSFORMS_PASSES_H
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
namespace mlir { namespace mlir {
class AffineDialect; class AffineDialect;
class StandardOpsDialect; namespace func {
class FuncDialect;
} // namespace func
namespace tensor { namespace tensor {
class TensorDialect; class TensorDialect;
} // namespace tensor } // namespace tensor
namespace vector { namespace vector {
class VectorDialect; class VectorDialect;
} // namespace vector } // namespace vector
namespace memref { namespace memref {
▲ Show 20 Lines • Show All 91 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/MemRef/Transforms/Passes.td

//===-- Passes.td - MemRef transformation definition file --*- tablegen -*-===// //===-- Passes.td - MemRef transformation definition file --*- tablegen -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef MLIR_DIALECT_MEMREF_TRANSFORMS_PASSES #ifndef MLIR_DIALECT_MEMREF_TRANSFORMS_PASSES
#define MLIR_DIALECT_MEMREF_TRANSFORMS_PASSES #define MLIR_DIALECT_MEMREF_TRANSFORMS_PASSES
include "mlir/Pass/PassBase.td" include "mlir/Pass/PassBase.td"
def ExpandOps : Pass<"memref-expand", "FuncOp"> { def ExpandOps : Pass<"memref-expand", "FuncOp"> {
let summary = "Legalize memref operations to be convertible to LLVM."; let summary = "Legalize memref operations to be convertible to LLVM.";
let constructor = "mlir::memref::createExpandOpsPass()"; let constructor = "mlir::memref::createExpandOpsPass()";
let dependentDialects = ["StandardOpsDialect"]; let dependentDialects = ["func::FuncDialect"];
} }
def FoldSubViewOps : Pass<"fold-memref-subview-ops"> { def FoldSubViewOps : Pass<"fold-memref-subview-ops"> {
let summary = "Fold memref.subview ops into consumer load/store ops"; let summary = "Fold memref.subview ops into consumer load/store ops";
let description = [{ let description = [{
The pass folds loading/storing from/to subview ops to loading/storing The pass folds loading/storing from/to subview ops to loading/storing
from/to the original memref. from/to the original memref.
}]; }];
Show All 14 Lines let description = [{
memref types while preserving the original behavior, users of those memref types while preserving the original behavior, users of those
memref types are also modified to incorporate the resulting layout map. memref types are also modified to incorporate the resulting layout map.
For instance, an [AffineLoadOp] For instance, an [AffineLoadOp]
(https://mlir.llvm.org/docs/Dialects/Affine/#affineload-affineloadop) (https://mlir.llvm.org/docs/Dialects/Affine/#affineload-affineloadop)
will be updated to compose the layout map with with the affine expression will be updated to compose the layout map with with the affine expression
contained in the op. Operations marked with the [MemRefsNormalizable] contained in the op. Operations marked with the [MemRefsNormalizable]
(https://mlir.llvm.org/docs/Traits/#memrefsnormalizable) trait are (https://mlir.llvm.org/docs/Traits/#memrefsnormalizable) trait are
expected to be normalizable. Supported operations include affine expected to be normalizable. Supported operations include affine
operations, memref.alloc, memref.dealloc, and std.return. operations, memref.alloc, memref.dealloc, and func.return.
Given an appropriate layout map specified in the code, this transformation Given an appropriate layout map specified in the code, this transformation
can express tiled or linearized access to multi-dimensional data can express tiled or linearized access to multi-dimensional data
structures, but will not modify memref types without an explicit layout structures, but will not modify memref types without an explicit layout
map. map.
Currently this pass is limited to only modify Currently this pass is limited to only modify
functions where all memref types can be normalized. If a function functions where all memref types can be normalized. If a function
▲ Show 20 Lines • Show All 122 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/PDL/IR/PDLOps.td

Show First 20 Lines • Show All 308 Lines • ▼ Show 20 Lines let description = [{
1) A single !pdl.range<value>: 1) A single !pdl.range<value>:
In this case, the single range is treated as all of the operands of the In this case, the single range is treated as all of the operands of the
operation. operation.
```mlir ```mlir
// Define an instance with single range of operands. // Define an instance with single range of operands.
%op = pdl.operation "std.return"(%allArgs : !pdl.range<value>) %op = pdl.operation "func.return"(%allArgs : !pdl.range<value>)
``` ```
2) A variadic number of either !pdl.value or !pdl.range<value>: 2) A variadic number of either !pdl.value or !pdl.range<value>:
In this case, the inputs are expected to correspond with the operand groups In this case, the inputs are expected to correspond with the operand groups
defined on the operation in ODS. defined on the operation in ODS.
```tablgen ```tablgen
▲ Show 20 Lines • Show All 371 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/StandardOps/CMakeLists.txt

  • This file was deleted after being copied to mlir/include/mlir/Dialect/Func/CMakeLists.txt, mlir/lib/Dialect/Func/CMakeLists.txt.
The contents of this file were not changed.

mlir/include/mlir/Dialect/StandardOps/IR/CMakeLists.txt

  • This file was deleted.
set(LLVM_TARGET_DEFINITIONS Ops.td)
mlir_tablegen(Ops.h.inc -gen-op-decls)
mlir_tablegen(Ops.cpp.inc -gen-op-defs)
mlir_tablegen(OpsDialect.h.inc -gen-dialect-decls)
mlir_tablegen(OpsDialect.cpp.inc -gen-dialect-defs)
mlir_tablegen(OpsEnums.h.inc -gen-enum-decls)
mlir_tablegen(OpsEnums.cpp.inc -gen-enum-defs)
add_public_tablegen_target(MLIRStandardOpsIncGen)
add_mlir_doc(Ops StandardOps Dialects/ -gen-op-doc)

mlir/include/mlir/Dialect/StandardOps/IR/Ops.h

  • This file was moved to mlir/include/mlir/Dialect/Func/IR/FuncOps.h.

mlir/include/mlir/Dialect/StandardOps/IR/Ops.td

  • This file was moved to mlir/include/mlir/Dialect/Func/IR/FuncOps.td.

mlir/include/mlir/Dialect/StandardOps/Transforms/CMakeLists.txt

  • This file was deleted.
set(LLVM_TARGET_DEFINITIONS Passes.td)
mlir_tablegen(Passes.h.inc -gen-pass-decls -name Standard)
add_public_tablegen_target(MLIRStandardTransformsIncGen)
add_mlir_doc(Passes StandardPasses ./ -gen-pass-doc)

mlir/include/mlir/Dialect/StandardOps/Transforms/DecomposeCallGraphTypes.h

  • This file was moved to mlir/include/mlir/Dialect/Func/Transforms/DecomposeCallGraphTypes.h.

mlir/include/mlir/Dialect/StandardOps/Transforms/FuncConversions.h

  • This file was moved to mlir/include/mlir/Dialect/Func/Transforms/FuncConversions.h.

mlir/include/mlir/Dialect/StandardOps/Transforms/Passes.h

  • This file was moved to mlir/include/mlir/Dialect/Func/Transforms/Passes.h.

mlir/include/mlir/Dialect/StandardOps/Transforms/Passes.td

  • This file was moved to mlir/include/mlir/Dialect/Func/Transforms/Passes.td.

mlir/include/mlir/Dialect/Tosa/Transforms/PassDetail.h

//===- PassDetail.h - TOSA Pass class details -------------------*- C++ -*-===// //===- PassDetail.h - TOSA Pass class details -------------------*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef MLIR_DIALECT_TOSA_TRANSFORMS_PASSDETAIL_H #ifndef MLIR_DIALECT_TOSA_TRANSFORMS_PASSDETAIL_H
#define MLIR_DIALECT_TOSA_TRANSFORMS_PASSDETAIL_H #define MLIR_DIALECT_TOSA_TRANSFORMS_PASSDETAIL_H
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/Dialect/Tosa/IR/TosaOps.h" #include "mlir/Dialect/Tosa/IR/TosaOps.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
namespace mlir { namespace mlir {
#define GEN_PASS_CLASSES #define GEN_PASS_CLASSES
#include "mlir/Dialect/Tosa/Transforms/Passes.h.inc" #include "mlir/Dialect/Tosa/Transforms/Passes.h.inc"
} // namespace mlir } // namespace mlir
#endif // MLIR_DIALECT_TOSA_TRANSFORMS_PASSDETAIL_H #endif // MLIR_DIALECT_TOSA_TRANSFORMS_PASSDETAIL_H

mlir/include/mlir/Dialect/Tosa/Transforms/Passes.td

Show All 18 Linesdef TosaInferShapes : Pass<"tosa-infer-shapes", "FuncOp"> {
let summary = "Propagate shapes across TOSA operations"; let summary = "Propagate shapes across TOSA operations";
let description = [{ let description = [{
Pass that uses operand types and propagates shapes to TOSA operations. Pass that uses operand types and propagates shapes to TOSA operations.
This includes legalizing rankless and dynamic shapes towards static. This includes legalizing rankless and dynamic shapes towards static.
}]; }];
let constructor = "createTosaInferShapesPass()"; let constructor = "createTosaInferShapesPass()";
let dependentDialects = [ let dependentDialects = [
"StandardOpsDialect", "func::FuncDialect",
"tensor::TensorDialect", "tensor::TensorDialect",
"tosa::TosaDialect", "tosa::TosaDialect",
]; ];
} }
def TosaMakeBroadcastable : Pass<"tosa-make-broadcastable", "FuncOp"> { def TosaMakeBroadcastable : Pass<"tosa-make-broadcastable", "FuncOp"> {
let summary = "TOSA rank Reshape to enable Broadcasting"; let summary = "TOSA rank Reshape to enable Broadcasting";
let description = [{ let description = [{
Show All 22 Lines

mlir/include/mlir/Dialect/Tosa/Utils/CoversionUtils.h

//===- ConversionUtils.h - Helper functions for tosa conversion -*- C++ -*-===// //===- ConversionUtils.h - Helper functions for tosa conversion -*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// Utility functions for TOSA lowering // Utility functions for TOSA lowering
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef DIALECT_TOSA_UTILS_COVERSION_UTILS_H_ #ifndef DIALECT_TOSA_UTILS_COVERSION_UTILS_H_
#define DIALECT_TOSA_UTILS_COVERSION_UTILS_H_ #define DIALECT_TOSA_UTILS_COVERSION_UTILS_H_
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/Dialect/Utils/StructuredOpsUtils.h" #include "mlir/Dialect/Utils/StructuredOpsUtils.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
namespace mlir { namespace mlir {
namespace tosa { namespace tosa {
// Creates a SmallVector of Stringrefs for N parallel loops // Creates a SmallVector of Stringrefs for N parallel loops
▲ Show 20 Lines • Show All 60 Lines • Show Last 20 Lines

mlir/include/mlir/InitAllDialects.h

Show All 20 Lines
#include "mlir/Dialect/ArmNeon/ArmNeonDialect.h" #include "mlir/Dialect/ArmNeon/ArmNeonDialect.h"
#include "mlir/Dialect/ArmSVE/ArmSVEDialect.h" #include "mlir/Dialect/ArmSVE/ArmSVEDialect.h"
#include "mlir/Dialect/Async/IR/Async.h" #include "mlir/Dialect/Async/IR/Async.h"
#include "mlir/Dialect/Bufferization/IR/Bufferization.h" #include "mlir/Dialect/Bufferization/IR/Bufferization.h"
#include "mlir/Dialect/Complex/IR/Complex.h" #include "mlir/Dialect/Complex/IR/Complex.h"
#include "mlir/Dialect/ControlFlow/IR/ControlFlow.h" #include "mlir/Dialect/ControlFlow/IR/ControlFlow.h"
#include "mlir/Dialect/DLTI/DLTI.h" #include "mlir/Dialect/DLTI/DLTI.h"
#include "mlir/Dialect/EmitC/IR/EmitC.h" #include "mlir/Dialect/EmitC/IR/EmitC.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/GPU/GPUDialect.h" #include "mlir/Dialect/GPU/GPUDialect.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/LLVMIR/NVVMDialect.h" #include "mlir/Dialect/LLVMIR/NVVMDialect.h"
#include "mlir/Dialect/LLVMIR/ROCDLDialect.h" #include "mlir/Dialect/LLVMIR/ROCDLDialect.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/Linalg/Transforms/BufferizableOpInterfaceImpl.h" #include "mlir/Dialect/Linalg/Transforms/BufferizableOpInterfaceImpl.h"
#include "mlir/Dialect/Math/IR/Math.h" #include "mlir/Dialect/Math/IR/Math.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/OpenACC/OpenACC.h" #include "mlir/Dialect/OpenACC/OpenACC.h"
#include "mlir/Dialect/OpenMP/OpenMPDialect.h" #include "mlir/Dialect/OpenMP/OpenMPDialect.h"
#include "mlir/Dialect/PDL/IR/PDL.h" #include "mlir/Dialect/PDL/IR/PDL.h"
#include "mlir/Dialect/PDLInterp/IR/PDLInterp.h" #include "mlir/Dialect/PDLInterp/IR/PDLInterp.h"
#include "mlir/Dialect/Quant/QuantOps.h" #include "mlir/Dialect/Quant/QuantOps.h"
#include "mlir/Dialect/SCF/BufferizableOpInterfaceImpl.h" #include "mlir/Dialect/SCF/BufferizableOpInterfaceImpl.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h" #include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h"
#include "mlir/Dialect/Shape/IR/Shape.h" #include "mlir/Dialect/Shape/IR/Shape.h"
#include "mlir/Dialect/SparseTensor/IR/SparseTensor.h" #include "mlir/Dialect/SparseTensor/IR/SparseTensor.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/Dialect/Tensor/IR/TensorInferTypeOpInterfaceImpl.h" #include "mlir/Dialect/Tensor/IR/TensorInferTypeOpInterfaceImpl.h"
#include "mlir/Dialect/Tensor/IR/TensorTilingInterfaceImpl.h" #include "mlir/Dialect/Tensor/IR/TensorTilingInterfaceImpl.h"
#include "mlir/Dialect/Tensor/Transforms/BufferizableOpInterfaceImpl.h" #include "mlir/Dialect/Tensor/Transforms/BufferizableOpInterfaceImpl.h"
#include "mlir/Dialect/Tosa/IR/TosaOps.h" #include "mlir/Dialect/Tosa/IR/TosaOps.h"
#include "mlir/Dialect/Vector/IR/VectorOps.h" #include "mlir/Dialect/Vector/IR/VectorOps.h"
#include "mlir/Dialect/Vector/Transforms/BufferizableOpInterfaceImpl.h" #include "mlir/Dialect/Vector/Transforms/BufferizableOpInterfaceImpl.h"
#include "mlir/Dialect/X86Vector/X86VectorDialect.h" #include "mlir/Dialect/X86Vector/X86VectorDialect.h"
Show All 10 Lines registry.insert<acc::OpenACCDialect,
amx::AMXDialect, amx::AMXDialect,
arm_neon::ArmNeonDialect, arm_neon::ArmNeonDialect,
async::AsyncDialect, async::AsyncDialect,
bufferization::BufferizationDialect, bufferization::BufferizationDialect,
cf::ControlFlowDialect, cf::ControlFlowDialect,
complex::ComplexDialect, complex::ComplexDialect,
DLTIDialect, DLTIDialect,
emitc::EmitCDialect, emitc::EmitCDialect,
func::FuncDialect,
gpu::GPUDialect, gpu::GPUDialect,
LLVM::LLVMDialect, LLVM::LLVMDialect,
linalg::LinalgDialect, linalg::LinalgDialect,
math::MathDialect, math::MathDialect,
memref::MemRefDialect, memref::MemRefDialect,
scf::SCFDialect, scf::SCFDialect,
omp::OpenMPDialect, omp::OpenMPDialect,
pdl::PDLDialect, pdl::PDLDialect,
pdl_interp::PDLInterpDialect, pdl_interp::PDLInterpDialect,
quant::QuantizationDialect, quant::QuantizationDialect,
spirv::SPIRVDialect, spirv::SPIRVDialect,
StandardOpsDialect,
arm_sve::ArmSVEDialect, arm_sve::ArmSVEDialect,
vector::VectorDialect, vector::VectorDialect,
NVVM::NVVMDialect, NVVM::NVVMDialect,
ROCDL::ROCDLDialect, ROCDL::ROCDLDialect,
shape::ShapeDialect, shape::ShapeDialect,
sparse_tensor::SparseTensorDialect, sparse_tensor::SparseTensorDialect,
tensor::TensorDialect, tensor::TensorDialect,
tosa::TosaDialect, tosa::TosaDialect,
Show All 21 Lines

mlir/include/mlir/InitAllPasses.h

Show All 13 Lines
#ifndef MLIR_INITALLPASSES_H_ #ifndef MLIR_INITALLPASSES_H_
#define MLIR_INITALLPASSES_H_ #define MLIR_INITALLPASSES_H_
#include "mlir/Conversion/Passes.h" #include "mlir/Conversion/Passes.h"
#include "mlir/Dialect/Affine/Passes.h" #include "mlir/Dialect/Affine/Passes.h"
#include "mlir/Dialect/Arithmetic/Transforms/Passes.h" #include "mlir/Dialect/Arithmetic/Transforms/Passes.h"
#include "mlir/Dialect/Async/Passes.h" #include "mlir/Dialect/Async/Passes.h"
#include "mlir/Dialect/Bufferization/Transforms/Passes.h" #include "mlir/Dialect/Bufferization/Transforms/Passes.h"
#include "mlir/Dialect/Func/Transforms/Passes.h"
#include "mlir/Dialect/GPU/Passes.h" #include "mlir/Dialect/GPU/Passes.h"
#include "mlir/Dialect/LLVMIR/Transforms/Passes.h" #include "mlir/Dialect/LLVMIR/Transforms/Passes.h"
#include "mlir/Dialect/Linalg/Passes.h" #include "mlir/Dialect/Linalg/Passes.h"
#include "mlir/Dialect/MemRef/Transforms/Passes.h" #include "mlir/Dialect/MemRef/Transforms/Passes.h"
#include "mlir/Dialect/Quant/Passes.h" #include "mlir/Dialect/Quant/Passes.h"
#include "mlir/Dialect/SCF/Passes.h" #include "mlir/Dialect/SCF/Passes.h"
#include "mlir/Dialect/SPIRV/Transforms/Passes.h" #include "mlir/Dialect/SPIRV/Transforms/Passes.h"
#include "mlir/Dialect/Shape/Transforms/Passes.h" #include "mlir/Dialect/Shape/Transforms/Passes.h"
#include "mlir/Dialect/SparseTensor/Pipelines/Passes.h" #include "mlir/Dialect/SparseTensor/Pipelines/Passes.h"
#include "mlir/Dialect/SparseTensor/Transforms/Passes.h" #include "mlir/Dialect/SparseTensor/Transforms/Passes.h"
#include "mlir/Dialect/StandardOps/Transforms/Passes.h"
#include "mlir/Dialect/Tensor/Transforms/Passes.h" #include "mlir/Dialect/Tensor/Transforms/Passes.h"
#include "mlir/Dialect/Tosa/Transforms/Passes.h" #include "mlir/Dialect/Tosa/Transforms/Passes.h"
#include "mlir/Dialect/Vector/Transforms/Passes.h" #include "mlir/Dialect/Vector/Transforms/Passes.h"
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
#include <cstdlib> #include <cstdlib>
namespace mlir { namespace mlir {
Show All 12 Linesinline void registerAllPasses() {
// Conversion passes // Conversion passes
registerConversionPasses(); registerConversionPasses();
// Dialect passes // Dialect passes
registerAffinePasses(); registerAffinePasses();
registerAsyncPasses(); registerAsyncPasses();
arith::registerArithmeticPasses(); arith::registerArithmeticPasses();
bufferization::registerBufferizationPasses(); bufferization::registerBufferizationPasses();
func::registerFuncPasses();
registerGPUPasses(); registerGPUPasses();
registerGpuSerializeToCubinPass(); registerGpuSerializeToCubinPass();
registerGpuSerializeToHsacoPass(); registerGpuSerializeToHsacoPass();
registerLinalgPasses(); registerLinalgPasses();
registerSparseTensorPasses(); registerSparseTensorPasses();
LLVM::registerLLVMPasses(); LLVM::registerLLVMPasses();
memref::registerMemRefPasses(); memref::registerMemRefPasses();
quant::registerQuantPasses(); quant::registerQuantPasses();
registerSCFPasses(); registerSCFPasses();
registerShapePasses(); registerShapePasses();
spirv::registerSPIRVPasses(); spirv::registerSPIRVPasses();
registerStandardPasses();
tensor::registerTensorPasses(); tensor::registerTensorPasses();
tosa::registerTosaOptPasses(); tosa::registerTosaOptPasses();
vector::registerVectorPasses(); vector::registerVectorPasses();
// Dialect pipelines // Dialect pipelines
sparse_tensor::registerSparseTensorPipelines(); sparse_tensor::registerSparseTensorPipelines();
} }
} // namespace mlir } // namespace mlir
#endif // MLIR_INITALLPASSES_H_ #endif // MLIR_INITALLPASSES_H_

mlir/lib/CAPI/Dialect/CMakeLists.txt

Show First 20 Lines • Show All 74 Lines • ▼ Show 20 Linesadd_mlir_upstream_c_api_library(MLIRCAPISparseTensor
PARTIAL_SOURCES_INTENDED PARTIAL_SOURCES_INTENDED
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRCAPIIR MLIRCAPIIR
MLIRSparseTensor MLIRSparseTensor
MLIRSparseTensorTransforms MLIRSparseTensorTransforms
) )
add_mlir_upstream_c_api_library(MLIRCAPIStandard add_mlir_upstream_c_api_library(MLIRCAPIFunc
Standard.cpp Func.cpp
PARTIAL_SOURCES_INTENDED PARTIAL_SOURCES_INTENDED
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRCAPIIR MLIRCAPIIR
MLIRStandard MLIRFunc
) )
add_mlir_upstream_c_api_library(MLIRCAPITensor add_mlir_upstream_c_api_library(MLIRCAPITensor
Tensor.cpp Tensor.cpp
PARTIAL_SOURCES_INTENDED PARTIAL_SOURCES_INTENDED
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRCAPIIR MLIRCAPIIR
Show All 20 Lines

mlir/lib/CAPI/Dialect/Func.cpp

  • This file was moved from mlir/lib/CAPI/Dialect/Standard.cpp.
//===- Standard.cpp - C Interface for Standard dialect --------------------===// //===- Func.cpp - C Interface for Func dialect ----------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir-c/Dialect/Standard.h" #include "mlir-c/Dialect/Func.h"
#include "mlir/CAPI/Registration.h" #include "mlir/CAPI/Registration.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
MLIR_DEFINE_CAPI_DIALECT_REGISTRATION(Standard, std, mlir::StandardOpsDialect) MLIR_DEFINE_CAPI_DIALECT_REGISTRATION(Func, func, mlir::func::FuncDialect)

mlir/lib/CAPI/Dialect/Standard.cpp

  • This file was moved to mlir/lib/CAPI/Dialect/Func.cpp.

mlir/lib/Conversion/AffineToStandard/AffineToStandard.cpp

Show All 10 Lines
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/AffineToStandard/AffineToStandard.h" #include "mlir/Conversion/AffineToStandard/AffineToStandard.h"
#include "../PassDetail.h" #include "../PassDetail.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Affine/Utils.h" #include "mlir/Dialect/Affine/Utils.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Vector/IR/VectorOps.h" #include "mlir/Dialect/Vector/IR/VectorOps.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/IntegerSet.h" #include "mlir/IR/IntegerSet.h"
#include "mlir/IR/MLIRContext.h" #include "mlir/IR/MLIRContext.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
▲ Show 20 Lines • Show All 518 Lines • ▼ Show 20 Lines
namespace { namespace {
class LowerAffinePass : public ConvertAffineToStandardBase<LowerAffinePass> { class LowerAffinePass : public ConvertAffineToStandardBase<LowerAffinePass> {
void runOnOperation() override { void runOnOperation() override {
RewritePatternSet patterns(&getContext()); RewritePatternSet patterns(&getContext());
populateAffineToStdConversionPatterns(patterns); populateAffineToStdConversionPatterns(patterns);
populateAffineToVectorConversionPatterns(patterns); populateAffineToVectorConversionPatterns(patterns);
ConversionTarget target(getContext()); ConversionTarget target(getContext());
target target.addLegalDialect<arith::ArithmeticDialect, func::FuncDialect,
.addLegalDialect<arith::ArithmeticDialect, memref::MemRefDialect, memref::MemRefDialect, scf::SCFDialect,
scf::SCFDialect, StandardOpsDialect, VectorDialect>(); VectorDialect>();
if (failed(applyPartialConversion(getOperation(), target, if (failed(applyPartialConversion(getOperation(), target,
std::move(patterns)))) std::move(patterns))))
signalPassFailure(); signalPassFailure();
} }
}; };
} // namespace } // namespace
/// Lowers If and For operations within a function into their lower level CFG /// Lowers If and For operations within a function into their lower level CFG
/// equivalent blocks. /// equivalent blocks.
std::unique_ptr<Pass> mlir::createLowerAffinePass() { std::unique_ptr<Pass> mlir::createLowerAffinePass() {
return std::make_unique<LowerAffinePass>(); return std::make_unique<LowerAffinePass>();
} }

mlir/lib/Conversion/AffineToStandard/CMakeLists.txt

add_mlir_conversion_library(MLIRAffineToStandard add_mlir_conversion_library(MLIRAffineToStandard
AffineToStandard.cpp AffineToStandard.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/AffineToStandard ${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/AffineToStandard
DEPENDS DEPENDS
MLIRConversionPassIncGen MLIRConversionPassIncGen
LINK_COMPONENTS LINK_COMPONENTS
Core Core
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRAffine MLIRAffine
MLIRAffineUtils MLIRAffineUtils
MLIRArithmetic MLIRArithmetic
MLIRFunc
MLIRIR MLIRIR
MLIRMemRef MLIRMemRef
MLIRSCF MLIRSCF
MLIRPass MLIRPass
MLIRStandard
MLIRTransforms MLIRTransforms
MLIRVector MLIRVector
) )

mlir/lib/Conversion/AsyncToLLVM/AsyncToLLVM.cpp

//===- AsyncToLLVM.cpp - Convert Async to LLVM dialect --------------------===// //===- AsyncToLLVM.cpp - Convert Async to LLVM dialect --------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/AsyncToLLVM/AsyncToLLVM.h" #include "mlir/Conversion/AsyncToLLVM/AsyncToLLVM.h"
#include "../PassDetail.h" #include "../PassDetail.h"
#include "mlir/Conversion/LLVMCommon/ConversionTarget.h" #include "mlir/Conversion/LLVMCommon/ConversionTarget.h"
#include "mlir/Conversion/LLVMCommon/TypeConverter.h" #include "mlir/Conversion/LLVMCommon/TypeConverter.h"
#include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h" #include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Async/IR/Async.h" #include "mlir/Dialect/Async/IR/Async.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Func/Transforms/FuncConversions.h"
#include "mlir/Dialect/LLVMIR/FunctionCallUtils.h" #include "mlir/Dialect/LLVMIR/FunctionCallUtils.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/StandardOps/Transforms/FuncConversions.h"
#include "mlir/IR/ImplicitLocOpBuilder.h" #include "mlir/IR/ImplicitLocOpBuilder.h"
#include "mlir/IR/TypeUtilities.h" #include "mlir/IR/TypeUtilities.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/ADT/TypeSwitch.h" #include "llvm/ADT/TypeSwitch.h"
#define DEBUG_TYPE "convert-async-to-llvm" #define DEBUG_TYPE "convert-async-to-llvm"
▲ Show 20 Lines • Show All 515 Lines • ▼ Show 20 Linespublic:
LogicalResult LogicalResult
matchAndRewrite(RuntimeCreateOp op, OpAdaptor adaptor, matchAndRewrite(RuntimeCreateOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override { ConversionPatternRewriter &rewriter) const override {
TypeConverter *converter = getTypeConverter(); TypeConverter *converter = getTypeConverter();
Type resultType = op->getResultTypes()[0]; Type resultType = op->getResultTypes()[0];
// Tokens creation maps to a simple function call. // Tokens creation maps to a simple function call.
if (resultType.isa<TokenType>()) { if (resultType.isa<TokenType>()) {
rewriter.replaceOpWithNewOp<CallOp>(op, kCreateToken, rewriter.replaceOpWithNewOp<func::CallOp>(
converter->convertType(resultType)); op, kCreateToken, converter->convertType(resultType));
return success(); return success();
} }
// To create a value we need to compute the storage requirement. // To create a value we need to compute the storage requirement.
if (auto value = resultType.dyn_cast<ValueType>()) { if (auto value = resultType.dyn_cast<ValueType>()) {
// Returns the size requirements for the async value storage. // Returns the size requirements for the async value storage.
auto sizeOf = [&](ValueType valueType) -> Value { auto sizeOf = [&](ValueType valueType) -> Value {
auto loc = op->getLoc(); auto loc = op->getLoc();
auto i64 = rewriter.getI64Type(); auto i64 = rewriter.getI64Type();
auto storedType = converter->convertType(valueType.getValueType()); auto storedType = converter->convertType(valueType.getValueType());
auto storagePtrType = LLVM::LLVMPointerType::get(storedType); auto storagePtrType = LLVM::LLVMPointerType::get(storedType);
// %Size = getelementptr %T* null, int 1 // %Size = getelementptr %T* null, int 1
// %SizeI = ptrtoint %T* %Size to i64 // %SizeI = ptrtoint %T* %Size to i64
auto nullPtr = rewriter.create<LLVM::NullOp>(loc, storagePtrType); auto nullPtr = rewriter.create<LLVM::NullOp>(loc, storagePtrType);
auto one = rewriter.create<LLVM::ConstantOp>( auto one = rewriter.create<LLVM::ConstantOp>(
loc, i64, rewriter.getI64IntegerAttr(1)); loc, i64, rewriter.getI64IntegerAttr(1));
auto gep = rewriter.create<LLVM::GEPOp>(loc, storagePtrType, nullPtr, auto gep = rewriter.create<LLVM::GEPOp>(loc, storagePtrType, nullPtr,
one.getResult()); one.getResult());
return rewriter.create<LLVM::PtrToIntOp>(loc, i64, gep); return rewriter.create<LLVM::PtrToIntOp>(loc, i64, gep);
}; };
rewriter.replaceOpWithNewOp<CallOp>(op, kCreateValue, resultType, rewriter.replaceOpWithNewOp<func::CallOp>(op, kCreateValue, resultType,
sizeOf(value)); sizeOf(value));
return success(); return success();
} }
return rewriter.notifyMatchFailure(op, "unsupported async type"); return rewriter.notifyMatchFailure(op, "unsupported async type");
} }
}; };
} // namespace } // namespace
Show All 9 Linespublic:
using OpConversionPattern::OpConversionPattern; using OpConversionPattern::OpConversionPattern;
LogicalResult LogicalResult
matchAndRewrite(RuntimeCreateGroupOp op, OpAdaptor adaptor, matchAndRewrite(RuntimeCreateGroupOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override { ConversionPatternRewriter &rewriter) const override {
TypeConverter *converter = getTypeConverter(); TypeConverter *converter = getTypeConverter();
Type resultType = op.getResult().getType(); Type resultType = op.getResult().getType();
rewriter.replaceOpWithNewOp<CallOp>(op, kCreateGroup, rewriter.replaceOpWithNewOp<func::CallOp>(
converter->convertType(resultType), op, kCreateGroup, converter->convertType(resultType),
adaptor.getOperands()); adaptor.getOperands());
return success(); return success();
} }
}; };
} // namespace } // namespace
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Convert async.runtime.set_available to the corresponding runtime API call. // Convert async.runtime.set_available to the corresponding runtime API call.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
namespace { namespace {
class RuntimeSetAvailableOpLowering class RuntimeSetAvailableOpLowering
: public OpConversionPattern<RuntimeSetAvailableOp> { : public OpConversionPattern<RuntimeSetAvailableOp> {
public: public:
using OpConversionPattern::OpConversionPattern; using OpConversionPattern::OpConversionPattern;
LogicalResult LogicalResult
matchAndRewrite(RuntimeSetAvailableOp op, OpAdaptor adaptor, matchAndRewrite(RuntimeSetAvailableOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override { ConversionPatternRewriter &rewriter) const override {
StringRef apiFuncName = StringRef apiFuncName =
TypeSwitch<Type, StringRef>(op.operand().getType()) TypeSwitch<Type, StringRef>(op.operand().getType())
.Case<TokenType>([](Type) { return kEmplaceToken; }) .Case<TokenType>([](Type) { return kEmplaceToken; })
.Case<ValueType>([](Type) { return kEmplaceValue; }); .Case<ValueType>([](Type) { return kEmplaceValue; });
rewriter.replaceOpWithNewOp<CallOp>(op, apiFuncName, TypeRange(), rewriter.replaceOpWithNewOp<func::CallOp>(op, apiFuncName, TypeRange(),
adaptor.getOperands()); adaptor.getOperands());
return success(); return success();
} }
}; };
} // namespace } // namespace
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Convert async.runtime.set_error to the corresponding runtime API call. // Convert async.runtime.set_error to the corresponding runtime API call.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
namespace { namespace {
class RuntimeSetErrorOpLowering class RuntimeSetErrorOpLowering
: public OpConversionPattern<RuntimeSetErrorOp> { : public OpConversionPattern<RuntimeSetErrorOp> {
public: public:
using OpConversionPattern::OpConversionPattern; using OpConversionPattern::OpConversionPattern;
LogicalResult LogicalResult
matchAndRewrite(RuntimeSetErrorOp op, OpAdaptor adaptor, matchAndRewrite(RuntimeSetErrorOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override { ConversionPatternRewriter &rewriter) const override {
StringRef apiFuncName = StringRef apiFuncName =
TypeSwitch<Type, StringRef>(op.operand().getType()) TypeSwitch<Type, StringRef>(op.operand().getType())
.Case<TokenType>([](Type) { return kSetTokenError; }) .Case<TokenType>([](Type) { return kSetTokenError; })
.Case<ValueType>([](Type) { return kSetValueError; }); .Case<ValueType>([](Type) { return kSetValueError; });
rewriter.replaceOpWithNewOp<CallOp>(op, apiFuncName, TypeRange(), rewriter.replaceOpWithNewOp<func::CallOp>(op, apiFuncName, TypeRange(),
adaptor.getOperands()); adaptor.getOperands());
return success(); return success();
} }
}; };
} // namespace } // namespace
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Convert async.runtime.is_error to the corresponding runtime API call. // Convert async.runtime.is_error to the corresponding runtime API call.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
namespace { namespace {
class RuntimeIsErrorOpLowering : public OpConversionPattern<RuntimeIsErrorOp> { class RuntimeIsErrorOpLowering : public OpConversionPattern<RuntimeIsErrorOp> {
public: public:
using OpConversionPattern::OpConversionPattern; using OpConversionPattern::OpConversionPattern;
LogicalResult LogicalResult
matchAndRewrite(RuntimeIsErrorOp op, OpAdaptor adaptor, matchAndRewrite(RuntimeIsErrorOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override { ConversionPatternRewriter &rewriter) const override {
StringRef apiFuncName = StringRef apiFuncName =
TypeSwitch<Type, StringRef>(op.operand().getType()) TypeSwitch<Type, StringRef>(op.operand().getType())
.Case<TokenType>([](Type) { return kIsTokenError; }) .Case<TokenType>([](Type) { return kIsTokenError; })
.Case<GroupType>([](Type) { return kIsGroupError; }) .Case<GroupType>([](Type) { return kIsGroupError; })
.Case<ValueType>([](Type) { return kIsValueError; }); .Case<ValueType>([](Type) { return kIsValueError; });
rewriter.replaceOpWithNewOp<CallOp>(op, apiFuncName, rewriter.getI1Type(), rewriter.replaceOpWithNewOp<func::CallOp>(
adaptor.getOperands()); op, apiFuncName, rewriter.getI1Type(), adaptor.getOperands());
return success(); return success();
} }
}; };
} // namespace } // namespace
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Convert async.runtime.await to the corresponding runtime API call. // Convert async.runtime.await to the corresponding runtime API call.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
namespace { namespace {
class RuntimeAwaitOpLowering : public OpConversionPattern<RuntimeAwaitOp> { class RuntimeAwaitOpLowering : public OpConversionPattern<RuntimeAwaitOp> {
public: public:
using OpConversionPattern::OpConversionPattern; using OpConversionPattern::OpConversionPattern;
LogicalResult LogicalResult
matchAndRewrite(RuntimeAwaitOp op, OpAdaptor adaptor, matchAndRewrite(RuntimeAwaitOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override { ConversionPatternRewriter &rewriter) const override {
StringRef apiFuncName = StringRef apiFuncName =
TypeSwitch<Type, StringRef>(op.operand().getType()) TypeSwitch<Type, StringRef>(op.operand().getType())
.Case<TokenType>([](Type) { return kAwaitToken; }) .Case<TokenType>([](Type) { return kAwaitToken; })
.Case<ValueType>([](Type) { return kAwaitValue; }) .Case<ValueType>([](Type) { return kAwaitValue; })
.Case<GroupType>([](Type) { return kAwaitGroup; }); .Case<GroupType>([](Type) { return kAwaitGroup; });
rewriter.create<CallOp>(op->getLoc(), apiFuncName, TypeRange(), rewriter.create<func::CallOp>(op->getLoc(), apiFuncName, TypeRange(),
adaptor.getOperands()); adaptor.getOperands());
rewriter.eraseOp(op); rewriter.eraseOp(op);
return success(); return success();
} }
}; };
} // namespace } // namespace
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
Show All 19 Lines matchAndRewrite(RuntimeAwaitAndResumeOp op, OpAdaptor adaptor,
Value handle = adaptor.handle(); Value handle = adaptor.handle();
// A pointer to coroutine resume intrinsic wrapper. // A pointer to coroutine resume intrinsic wrapper.
addResumeFunction(op->getParentOfType<ModuleOp>()); addResumeFunction(op->getParentOfType<ModuleOp>());
auto resumeFnTy = AsyncAPI::resumeFunctionType(op->getContext()); auto resumeFnTy = AsyncAPI::resumeFunctionType(op->getContext());
auto resumePtr = rewriter.create<LLVM::AddressOfOp>( auto resumePtr = rewriter.create<LLVM::AddressOfOp>(
op->getLoc(), LLVM::LLVMPointerType::get(resumeFnTy), kResume); op->getLoc(), LLVM::LLVMPointerType::get(resumeFnTy), kResume);
rewriter.create<CallOp>(op->getLoc(), apiFuncName, TypeRange(), rewriter.create<func::CallOp>(
op->getLoc(), apiFuncName, TypeRange(),
ValueRange({operand, handle, resumePtr.getRes()})); ValueRange({operand, handle, resumePtr.getRes()}));
rewriter.eraseOp(op); rewriter.eraseOp(op);
return success(); return success();
} }
}; };
} // namespace } // namespace
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
Show All 11 Lines matchAndRewrite(RuntimeResumeOp op, OpAdaptor adaptor,
// A pointer to coroutine resume intrinsic wrapper. // A pointer to coroutine resume intrinsic wrapper.
addResumeFunction(op->getParentOfType<ModuleOp>()); addResumeFunction(op->getParentOfType<ModuleOp>());
auto resumeFnTy = AsyncAPI::resumeFunctionType(op->getContext()); auto resumeFnTy = AsyncAPI::resumeFunctionType(op->getContext());
auto resumePtr = rewriter.create<LLVM::AddressOfOp>( auto resumePtr = rewriter.create<LLVM::AddressOfOp>(
op->getLoc(), LLVM::LLVMPointerType::get(resumeFnTy), kResume); op->getLoc(), LLVM::LLVMPointerType::get(resumeFnTy), kResume);
// Call async runtime API to execute a coroutine in the managed thread. // Call async runtime API to execute a coroutine in the managed thread.
auto coroHdl = adaptor.handle(); auto coroHdl = adaptor.handle();
rewriter.replaceOpWithNewOp<CallOp>( rewriter.replaceOpWithNewOp<func::CallOp>(
op, TypeRange(), kExecute, ValueRange({coroHdl, resumePtr.getRes()})); op, TypeRange(), kExecute, ValueRange({coroHdl, resumePtr.getRes()}));
return success(); return success();
} }
}; };
} // namespace } // namespace
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Convert async.runtime.store to the corresponding runtime API call. // Convert async.runtime.store to the corresponding runtime API call.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
namespace { namespace {
class RuntimeStoreOpLowering : public OpConversionPattern<RuntimeStoreOp> { class RuntimeStoreOpLowering : public OpConversionPattern<RuntimeStoreOp> {
public: public:
using OpConversionPattern::OpConversionPattern; using OpConversionPattern::OpConversionPattern;
LogicalResult LogicalResult
matchAndRewrite(RuntimeStoreOp op, OpAdaptor adaptor, matchAndRewrite(RuntimeStoreOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override { ConversionPatternRewriter &rewriter) const override {
Location loc = op->getLoc(); Location loc = op->getLoc();
// Get a pointer to the async value storage from the runtime. // Get a pointer to the async value storage from the runtime.
auto i8Ptr = AsyncAPI::opaquePointerType(rewriter.getContext()); auto i8Ptr = AsyncAPI::opaquePointerType(rewriter.getContext());
auto storage = adaptor.storage(); auto storage = adaptor.storage();
auto storagePtr = rewriter.create<CallOp>(loc, kGetValueStorage, auto storagePtr = rewriter.create<func::CallOp>(loc, kGetValueStorage,
TypeRange(i8Ptr), storage); TypeRange(i8Ptr), storage);
// Cast from i8* to the LLVM pointer type. // Cast from i8* to the LLVM pointer type.
auto valueType = op.value().getType(); auto valueType = op.value().getType();
auto llvmValueType = getTypeConverter()->convertType(valueType); auto llvmValueType = getTypeConverter()->convertType(valueType);
if (!llvmValueType) if (!llvmValueType)
return rewriter.notifyMatchFailure( return rewriter.notifyMatchFailure(
op, "failed to convert stored value type to LLVM type"); op, "failed to convert stored value type to LLVM type");
Show All 25 Linespublic:
LogicalResult LogicalResult
matchAndRewrite(RuntimeLoadOp op, OpAdaptor adaptor, matchAndRewrite(RuntimeLoadOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override { ConversionPatternRewriter &rewriter) const override {
Location loc = op->getLoc(); Location loc = op->getLoc();
// Get a pointer to the async value storage from the runtime. // Get a pointer to the async value storage from the runtime.
auto i8Ptr = AsyncAPI::opaquePointerType(rewriter.getContext()); auto i8Ptr = AsyncAPI::opaquePointerType(rewriter.getContext());
auto storage = adaptor.storage(); auto storage = adaptor.storage();
auto storagePtr = rewriter.create<CallOp>(loc, kGetValueStorage, auto storagePtr = rewriter.create<func::CallOp>(loc, kGetValueStorage,
TypeRange(i8Ptr), storage); TypeRange(i8Ptr), storage);
// Cast from i8* to the LLVM pointer type. // Cast from i8* to the LLVM pointer type.
auto valueType = op.result().getType(); auto valueType = op.result().getType();
auto llvmValueType = getTypeConverter()->convertType(valueType); auto llvmValueType = getTypeConverter()->convertType(valueType);
if (!llvmValueType) if (!llvmValueType)
return rewriter.notifyMatchFailure( return rewriter.notifyMatchFailure(
op, "failed to convert loaded value type to LLVM type"); op, "failed to convert loaded value type to LLVM type");
Show All 22 Linespublic:
LogicalResult LogicalResult
matchAndRewrite(RuntimeAddToGroupOp op, OpAdaptor adaptor, matchAndRewrite(RuntimeAddToGroupOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override { ConversionPatternRewriter &rewriter) const override {
// Currently we can only add tokens to the group. // Currently we can only add tokens to the group.
if (!op.operand().getType().isa<TokenType>()) if (!op.operand().getType().isa<TokenType>())
return rewriter.notifyMatchFailure(op, "only token type is supported"); return rewriter.notifyMatchFailure(op, "only token type is supported");
// Replace with a runtime API function call. // Replace with a runtime API function call.
rewriter.replaceOpWithNewOp<CallOp>( rewriter.replaceOpWithNewOp<func::CallOp>(
op, kAddTokenToGroup, rewriter.getI64Type(), adaptor.getOperands()); op, kAddTokenToGroup, rewriter.getI64Type(), adaptor.getOperands());
return success(); return success();
} }
}; };
} // namespace } // namespace
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Convert async.runtime.num_worker_threads to the corresponding runtime API // Convert async.runtime.num_worker_threads to the corresponding runtime API
// call. // call.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
namespace { namespace {
class RuntimeNumWorkerThreadsOpLowering class RuntimeNumWorkerThreadsOpLowering
: public OpConversionPattern<RuntimeNumWorkerThreadsOp> { : public OpConversionPattern<RuntimeNumWorkerThreadsOp> {
public: public:
using OpConversionPattern::OpConversionPattern; using OpConversionPattern::OpConversionPattern;
LogicalResult LogicalResult
matchAndRewrite(RuntimeNumWorkerThreadsOp op, OpAdaptor adaptor, matchAndRewrite(RuntimeNumWorkerThreadsOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override { ConversionPatternRewriter &rewriter) const override {
// Replace with a runtime API function call. // Replace with a runtime API function call.
rewriter.replaceOpWithNewOp<CallOp>(op, kGetNumWorkerThreads, rewriter.replaceOpWithNewOp<func::CallOp>(op, kGetNumWorkerThreads,
rewriter.getIndexType()); rewriter.getIndexType());
return success(); return success();
} }
}; };
} // namespace } // namespace
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Async reference counting ops lowering (`async.runtime.add_ref` and // Async reference counting ops lowering (`async.runtime.add_ref` and
Show All 12 Linespublic:
LogicalResult LogicalResult
matchAndRewrite(RefCountingOp op, typename RefCountingOp::Adaptor adaptor, matchAndRewrite(RefCountingOp op, typename RefCountingOp::Adaptor adaptor,
ConversionPatternRewriter &rewriter) const override { ConversionPatternRewriter &rewriter) const override {
auto count = rewriter.create<arith::ConstantOp>( auto count = rewriter.create<arith::ConstantOp>(
op->getLoc(), rewriter.getI64Type(), op->getLoc(), rewriter.getI64Type(),
rewriter.getI64IntegerAttr(op.count())); rewriter.getI64IntegerAttr(op.count()));
auto operand = adaptor.operand(); auto operand = adaptor.operand();
rewriter.replaceOpWithNewOp<CallOp>(op, TypeRange(), apiFunctionName, rewriter.replaceOpWithNewOp<func::CallOp>(op, TypeRange(), apiFunctionName,
ValueRange({operand, count})); ValueRange({operand, count}));
return success(); return success();
} }
private: private:
StringRef apiFunctionName; StringRef apiFunctionName;
}; };
Show All 11 Lines
}; };
} // namespace } // namespace
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Convert return operations that return async values from async regions. // Convert return operations that return async values from async regions.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
namespace { namespace {
class ReturnOpOpConversion : public OpConversionPattern<ReturnOp> { class ReturnOpOpConversion : public OpConversionPattern<func::ReturnOp> {
public: public:
using OpConversionPattern::OpConversionPattern; using OpConversionPattern::OpConversionPattern;
LogicalResult LogicalResult
matchAndRewrite(ReturnOp op, OpAdaptor adaptor, matchAndRewrite(func::ReturnOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override { ConversionPatternRewriter &rewriter) const override {
rewriter.replaceOpWithNewOp<ReturnOp>(op, adaptor.getOperands()); rewriter.replaceOpWithNewOp<func::ReturnOp>(op, adaptor.getOperands());
return success(); return success();
} }
}; };
} // namespace } // namespace
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
namespace { namespace {
▲ Show 20 Lines • Show All 47 Lines • ▼ Show 20 Linesvoid ConvertAsyncToLLVMPass::runOnOperation() {
// Lower async coroutine operations to LLVM coroutine intrinsics. // Lower async coroutine operations to LLVM coroutine intrinsics.
patterns patterns
.add<CoroIdOpConversion, CoroBeginOpConversion, CoroFreeOpConversion, .add<CoroIdOpConversion, CoroBeginOpConversion, CoroFreeOpConversion,
CoroEndOpConversion, CoroSaveOpConversion, CoroSuspendOpConversion>( CoroEndOpConversion, CoroSaveOpConversion, CoroSuspendOpConversion>(
converter, ctx); converter, ctx);
ConversionTarget target(*ctx); ConversionTarget target(*ctx);
target target.addLegalOp<arith::ConstantOp, func::ConstantOp,
.addLegalOp<arith::ConstantOp, ConstantOp, UnrealizedConversionCastOp>(); UnrealizedConversionCastOp>();
target.addLegalDialect<LLVM::LLVMDialect>(); target.addLegalDialect<LLVM::LLVMDialect>();
// All operations from Async dialect must be lowered to the runtime API and // All operations from Async dialect must be lowered to the runtime API and
// LLVM intrinsics calls. // LLVM intrinsics calls.
target.addIllegalDialect<AsyncDialect>(); target.addIllegalDialect<AsyncDialect>();
// Add dynamic legality constraints to apply conversions defined above. // Add dynamic legality constraints to apply conversions defined above.
target.addDynamicallyLegalOp<FuncOp>( target.addDynamicallyLegalOp<FuncOp>(
[&](FuncOp op) { return converter.isSignatureLegal(op.getType()); }); [&](FuncOp op) { return converter.isSignatureLegal(op.getType()); });
target.addDynamicallyLegalOp<ReturnOp>( target.addDynamicallyLegalOp<func::ReturnOp>([&](func::ReturnOp op) {
[&](ReturnOp op) { return converter.isLegal(op.getOperandTypes()); }); return converter.isLegal(op.getOperandTypes());
target.addDynamicallyLegalOp<CallOp>([&](CallOp op) { });
target.addDynamicallyLegalOp<func::CallOp>([&](func::CallOp op) {
return converter.isSignatureLegal(op.getCalleeType()); return converter.isSignatureLegal(op.getCalleeType());
}); });
if (failed(applyPartialConversion(module, target, std::move(patterns)))) if (failed(applyPartialConversion(module, target, std::move(patterns))))
signalPassFailure(); signalPassFailure();
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 71 Lines • Show Last 20 Lines

mlir/lib/Conversion/AsyncToLLVM/CMakeLists.txt

add_mlir_conversion_library(MLIRAsyncToLLVM add_mlir_conversion_library(MLIRAsyncToLLVM
AsyncToLLVM.cpp AsyncToLLVM.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/AsyncToLLVM ${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/AsyncToLLVM
DEPENDS DEPENDS
MLIRConversionPassIncGen MLIRConversionPassIncGen
LINK_COMPONENTS LINK_COMPONENTS
Core Core
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRArithmetic MLIRArithmetic
MLIRAsync MLIRAsync
MLIRFuncTransforms
MLIRLLVMCommonConversion MLIRLLVMCommonConversion
MLIRLLVMIR MLIRLLVMIR
MLIRStandardOpsTransforms
MLIRStandardToLLVM MLIRStandardToLLVM
MLIRTransforms MLIRTransforms
) )

mlir/lib/Conversion/ComplexToLLVM/CMakeLists.txt

add_mlir_conversion_library(MLIRComplexToLLVM add_mlir_conversion_library(MLIRComplexToLLVM
ComplexToLLVM.cpp ComplexToLLVM.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/ComplexToLLVM ${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/ComplexToLLVM
DEPENDS DEPENDS
MLIRConversionPassIncGen MLIRConversionPassIncGen
LINK_COMPONENTS LINK_COMPONENTS
Core Core
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRComplex MLIRComplex
MLIRFuncTransforms
MLIRLLVMCommonConversion MLIRLLVMCommonConversion
MLIRLLVMIR MLIRLLVMIR
MLIRStandardOpsTransforms
MLIRTransforms MLIRTransforms
) )

mlir/lib/Conversion/ComplexToLLVM/ComplexToLLVM.cpp

//===- ComplexToLLVM.cpp - conversion from Complex to LLVM dialect --------===// //===- ComplexToLLVM.cpp - conversion from Complex to LLVM dialect --------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/ComplexToLLVM/ComplexToLLVM.h" #include "mlir/Conversion/ComplexToLLVM/ComplexToLLVM.h"
#include "../PassDetail.h" #include "../PassDetail.h"
#include "mlir/Conversion/LLVMCommon/ConversionTarget.h" #include "mlir/Conversion/LLVMCommon/ConversionTarget.h"
#include "mlir/Conversion/LLVMCommon/Pattern.h" #include "mlir/Conversion/LLVMCommon/Pattern.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Complex/IR/Complex.h" #include "mlir/Dialect/Complex/IR/Complex.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
using namespace mlir; using namespace mlir;
using namespace mlir::LLVM; using namespace mlir::LLVM;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// ComplexStructBuilder implementation. // ComplexStructBuilder implementation.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 320 Lines • Show Last 20 Lines

mlir/lib/Conversion/ComplexToStandard/CMakeLists.txt

add_mlir_conversion_library(MLIRComplexToStandard add_mlir_conversion_library(MLIRComplexToStandard
ComplexToStandard.cpp ComplexToStandard.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/ComplexToStandard ${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/ComplexToStandard
DEPENDS DEPENDS
MLIRConversionPassIncGen MLIRConversionPassIncGen
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRArithmetic MLIRArithmetic
MLIRComplex MLIRComplex
MLIRFunc
MLIRIR MLIRIR
MLIRMath MLIRMath
MLIRStandard
MLIRTransforms MLIRTransforms
) )

mlir/lib/Conversion/ComplexToStandard/ComplexToStandard.cpp

//===- ComplexToStandard.cpp - conversion from Complex to Standard dialect ===// //===- ComplexToStandard.cpp - conversion from Complex to Standard dialect ===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/ComplexToStandard/ComplexToStandard.h" #include "mlir/Conversion/ComplexToStandard/ComplexToStandard.h"
#include <memory> #include <memory>
#include <type_traits> #include <type_traits>
#include "../PassDetail.h" #include "../PassDetail.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Complex/IR/Complex.h" #include "mlir/Dialect/Complex/IR/Complex.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Math/IR/Math.h" #include "mlir/Dialect/Math/IR/Math.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/ImplicitLocOpBuilder.h" #include "mlir/IR/ImplicitLocOpBuilder.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
using namespace mlir; using namespace mlir;
namespace { namespace {
struct AbsOpConversion : public OpConversionPattern<complex::AbsOp> { struct AbsOpConversion : public OpConversionPattern<complex::AbsOp> {
▲ Show 20 Lines • Show All 620 Lines • ▼ Show 20 Lines
void ConvertComplexToStandardPass::runOnOperation() { void ConvertComplexToStandardPass::runOnOperation() {
auto function = getOperation(); auto function = getOperation();
// Convert to the Standard dialect using the converter defined above. // Convert to the Standard dialect using the converter defined above.
RewritePatternSet patterns(&getContext()); RewritePatternSet patterns(&getContext());
populateComplexToStandardConversionPatterns(patterns); populateComplexToStandardConversionPatterns(patterns);
ConversionTarget target(getContext()); ConversionTarget target(getContext());
target.addLegalDialect<arith::ArithmeticDialect, StandardOpsDialect, target.addLegalDialect<arith::ArithmeticDialect, func::FuncDialect,
math::MathDialect>(); math::MathDialect>();
target.addLegalOp<complex::CreateOp, complex::ImOp, complex::ReOp>(); target.addLegalOp<complex::CreateOp, complex::ImOp, complex::ReOp>();
if (failed(applyPartialConversion(function, target, std::move(patterns)))) if (failed(applyPartialConversion(function, target, std::move(patterns))))
signalPassFailure(); signalPassFailure();
} }
} // namespace } // namespace
std::unique_ptr<OperationPass<FuncOp>> std::unique_ptr<OperationPass<FuncOp>>
mlir::createConvertComplexToStandardPass() { mlir::createConvertComplexToStandardPass() {
return std::make_unique<ConvertComplexToStandardPass>(); return std::make_unique<ConvertComplexToStandardPass>();
} }

mlir/lib/Conversion/ControlFlowToLLVM/ControlFlowToLLVM.cpp

Show All 23 Lines
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include <functional> #include <functional>
using namespace mlir; using namespace mlir;
#define PASS_NAME "convert-cf-to-llvm" #define PASS_NAME "convert-cf-to-llvm"
namespace { namespace {
/// Lower `std.assert`. The default lowering calls the `abort` function if the /// Lower `cf.assert`. The default lowering calls the `abort` function if the
/// assertion is violated and has no effect otherwise. The failure message is /// assertion is violated and has no effect otherwise. The failure message is
/// ignored by the default lowering but should be propagated by any custom /// ignored by the default lowering but should be propagated by any custom
/// lowering. /// lowering.
struct AssertOpLowering : public ConvertOpToLLVMPattern<cf::AssertOp> { struct AssertOpLowering : public ConvertOpToLLVMPattern<cf::AssertOp> {
using ConvertOpToLLVMPattern<cf::AssertOp>::ConvertOpToLLVMPattern; using ConvertOpToLLVMPattern<cf::AssertOp>::ConvertOpToLLVMPattern;
LogicalResult LogicalResult
matchAndRewrite(cf::AssertOp op, OpAdaptor adaptor, matchAndRewrite(cf::AssertOp op, OpAdaptor adaptor,
▲ Show 20 Lines • Show All 108 Lines • Show Last 20 Lines

mlir/lib/Conversion/GPUCommon/GPUOpsLowering.cpp

//===- GPUOpsLowering.cpp - GPU FuncOp / ReturnOp lowering ----------------===// //===- GPUOpsLowering.cpp - GPU FuncOp / ReturnOp lowering ----------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "GPUOpsLowering.h" #include "GPUOpsLowering.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "llvm/Support/FormatVariadic.h" #include "llvm/Support/FormatVariadic.h"
using namespace mlir; using namespace mlir;
LogicalResult LogicalResult
GPUFuncOpLowering::matchAndRewrite(gpu::GPUFuncOp gpuFuncOp, OpAdaptor adaptor, GPUFuncOpLowering::matchAndRewrite(gpu::GPUFuncOp gpuFuncOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const { ConversionPatternRewriter &rewriter) const {
▲ Show 20 Lines • Show All 324 Lines • Show Last 20 Lines

mlir/lib/Conversion/GPUCommon/IndexIntrinsicsOpLowering.h

Show All 10 Lines
#include "mlir/Conversion/LLVMCommon/Pattern.h" #include "mlir/Conversion/LLVMCommon/Pattern.h"
#include "mlir/Dialect/GPU/GPUDialect.h" #include "mlir/Dialect/GPU/GPUDialect.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "llvm/ADT/StringSwitch.h" #include "llvm/ADT/StringSwitch.h"
namespace mlir { namespace mlir {
// Rewriting that replaces Op with XOp, YOp, or ZOp depending on the dimension // Rewriting that replaces Op with XOp, YOp, or ZOp depending on the dimension
// that Op operates on. Op is assumed to return an `std.index` value and // that Op operates on. Op is assumed to return an `index` value and
// XOp, YOp and ZOp are assumed to return an `llvm.i32` value. Depending on // XOp, YOp and ZOp are assumed to return an `llvm.i32` value. Depending on
// `indexBitwidth`, sign-extend or truncate the resulting value to match the // `indexBitwidth`, sign-extend or truncate the resulting value to match the
// bitwidth expected by the consumers of the value. // bitwidth expected by the consumers of the value.
template <typename Op, typename XOp, typename YOp, typename ZOp> template <typename Op, typename XOp, typename YOp, typename ZOp>
struct GPUIndexIntrinsicOpLowering : public ConvertOpToLLVMPattern<Op> { struct GPUIndexIntrinsicOpLowering : public ConvertOpToLLVMPattern<Op> {
private: private:
unsigned indexBitwidth; unsigned indexBitwidth;
Show All 40 Lines

mlir/lib/Conversion/GPUCommon/OpToFuncCallLowering.h

//===- OpToFuncCallLowering.h - GPU ops lowering to custom calls *- C++ -*-===// //===- OpToFuncCallLowering.h - GPU ops lowering to custom calls *- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef MLIR_CONVERSION_GPUCOMMON_OPTOFUNCCALLLOWERING_H_ #ifndef MLIR_CONVERSION_GPUCOMMON_OPTOFUNCCALLLOWERING_H_
#define MLIR_CONVERSION_GPUCOMMON_OPTOFUNCCALLLOWERING_H_ #define MLIR_CONVERSION_GPUCOMMON_OPTOFUNCCALLLOWERING_H_
#include "mlir/Conversion/LLVMCommon/Pattern.h" #include "mlir/Conversion/LLVMCommon/Pattern.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/GPU/GPUDialect.h" #include "mlir/Dialect/GPU/GPUDialect.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
namespace mlir { namespace mlir {
/// Rewriting that replace SourceOp with a CallOp to `f32Func` or `f64Func` /// Rewriting that replace SourceOp with a CallOp to `f32Func` or `f64Func`
/// depending on the element type that Op operates upon. The function /// depending on the element type that Op operates upon. The function
/// declaration is added in case it was not added before. /// declaration is added in case it was not added before.
/// ///
/// If the input values are of f16 type, the value is first casted to f32, the /// If the input values are of f16 type, the value is first casted to f32, the
/// function called and then the result casted back. /// function called and then the result casted back.
/// ///
/// Example with NVVM: /// Example with NVVM:
/// %exp_f32 = std.exp %arg_f32 : f32 /// %exp_f32 = math.exp %arg_f32 : f32
/// ///
/// will be transformed into /// will be transformed into
/// llvm.call @__nv_expf(%arg_f32) : (f32) -> f32 /// llvm.call @__nv_expf(%arg_f32) : (f32) -> f32
template <typename SourceOp> template <typename SourceOp>
struct OpToFuncCallLowering : public ConvertOpToLLVMPattern<SourceOp> { struct OpToFuncCallLowering : public ConvertOpToLLVMPattern<SourceOp> {
public: public:
explicit OpToFuncCallLowering(LLVMTypeConverter &lowering, StringRef f32Func, explicit OpToFuncCallLowering(LLVMTypeConverter &lowering, StringRef f32Func,
StringRef f64Func) StringRef f64Func)
▲ Show 20 Lines • Show All 86 Lines • Show Last 20 Lines

mlir/lib/Conversion/GPUToVulkan/CMakeLists.txt

add_mlir_conversion_library(MLIRGPUToVulkanTransforms add_mlir_conversion_library(MLIRGPUToVulkanTransforms
ConvertLaunchFuncToVulkanCalls.cpp ConvertLaunchFuncToVulkanCalls.cpp
ConvertGPULaunchFuncToVulkanLaunchFunc.cpp ConvertGPULaunchFuncToVulkanLaunchFunc.cpp
DEPENDS DEPENDS
MLIRConversionPassIncGen MLIRConversionPassIncGen
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRFunc
MLIRGPUOps MLIRGPUOps
MLIRIR MLIRIR
MLIRLLVMIR MLIRLLVMIR
MLIRPass MLIRPass
MLIRSPIRV MLIRSPIRV
MLIRSPIRVSerialization MLIRSPIRVSerialization
MLIRStandard
MLIRSupport MLIRSupport
MLIRTransforms MLIRTransforms
MLIRTranslation MLIRTranslation
) )

mlir/lib/Conversion/GPUToVulkan/ConvertGPULaunchFuncToVulkanLaunchFunc.cpp

Show All 9 Lines
// launch function. Creates a SPIR-V binary shader from the `spirv::ModuleOp` // launch function. Creates a SPIR-V binary shader from the `spirv::ModuleOp`
// using `spirv::serialize` function, attaches binary data and entry point name // using `spirv::serialize` function, attaches binary data and entry point name
// as an attributes to vulkan launch call op. // as an attributes to vulkan launch call op.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "../PassDetail.h" #include "../PassDetail.h"
#include "mlir/Conversion/GPUToVulkan/ConvertGPUToVulkanPass.h" #include "mlir/Conversion/GPUToVulkan/ConvertGPUToVulkanPass.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/GPU/GPUDialect.h" #include "mlir/Dialect/GPU/GPUDialect.h"
#include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h" #include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h"
#include "mlir/Dialect/SPIRV/IR/SPIRVOps.h" #include "mlir/Dialect/SPIRV/IR/SPIRVOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/Attributes.h" #include "mlir/IR/Attributes.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/BuiltinTypes.h" #include "mlir/IR/BuiltinTypes.h"
#include "mlir/Target/SPIRV/Serialization.h" #include "mlir/Target/SPIRV/Serialization.h"
using namespace mlir; using namespace mlir;
▲ Show 20 Lines • Show All 135 Lines • ▼ Show 20 Linesvoid ConvertGpuLaunchFuncToVulkanLaunchFunc::convertGpuLaunchFunc(
SmallVector<Value, 8> vulkanLaunchOperands( SmallVector<Value, 8> vulkanLaunchOperands(
gpuLaunchOperands.begin(), gpuLaunchOperands.begin(),
gpuLaunchOperands.begin() + kVulkanLaunchNumConfigOperands); gpuLaunchOperands.begin() + kVulkanLaunchNumConfigOperands);
vulkanLaunchOperands.append(gpuLaunchOperands.begin() + vulkanLaunchOperands.append(gpuLaunchOperands.begin() +
gpu::LaunchOp::kNumConfigOperands, gpu::LaunchOp::kNumConfigOperands,
gpuLaunchOperands.end()); gpuLaunchOperands.end());
// Create vulkan launch call op. // Create vulkan launch call op.
auto vulkanLaunchCallOp = builder.create<CallOp>( auto vulkanLaunchCallOp = builder.create<func::CallOp>(
loc, TypeRange{}, SymbolRefAttr::get(builder.getContext(), kVulkanLaunch), loc, TypeRange{}, SymbolRefAttr::get(builder.getContext(), kVulkanLaunch),
vulkanLaunchOperands); vulkanLaunchOperands);
// Set SPIR-V binary shader data as an attribute. // Set SPIR-V binary shader data as an attribute.
vulkanLaunchCallOp->setAttr( vulkanLaunchCallOp->setAttr(
kSPIRVBlobAttrName, kSPIRVBlobAttrName,
builder.getStringAttr(StringRef(binary.data(), binary.size()))); builder.getStringAttr(StringRef(binary.data(), binary.size())));
Show All 11 Lines

mlir/lib/Conversion/LinalgToSPIRV/LinalgToSPIRV.cpp

//===- LinalgToSPIRV.cpp - Linalg to SPIR-V Patterns ----------------------===// //===- LinalgToSPIRV.cpp - Linalg to SPIR-V Patterns ----------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/LinalgToSPIRV/LinalgToSPIRV.h" #include "mlir/Conversion/LinalgToSPIRV/LinalgToSPIRV.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/Linalg/Utils/Utils.h" #include "mlir/Dialect/Linalg/Utils/Utils.h"
#include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h" #include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h"
#include "mlir/Dialect/SPIRV/IR/SPIRVOps.h" #include "mlir/Dialect/SPIRV/IR/SPIRVOps.h"
#include "mlir/Dialect/SPIRV/Transforms/SPIRVConversion.h" #include "mlir/Dialect/SPIRV/Transforms/SPIRVConversion.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Utils/StructuredOpsUtils.h" #include "mlir/Dialect/Utils/StructuredOpsUtils.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
using namespace mlir; using namespace mlir;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Utilities // Utilities
▲ Show 20 Lines • Show All 189 Lines • Show Last 20 Lines

mlir/lib/Conversion/LinalgToStandard/LinalgToStandard.cpp

//===- LinalgToStandard.cpp - conversion from Linalg to Standard dialect --===// //===- LinalgToStandard.cpp - conversion from Linalg to Standard dialect --===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/LinalgToStandard/LinalgToStandard.h" #include "mlir/Conversion/LinalgToStandard/LinalgToStandard.h"
#include "../PassDetail.h" #include "../PassDetail.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/Linalg/Transforms/Transforms.h" #include "mlir/Dialect/Linalg/Transforms/Transforms.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
using namespace mlir; using namespace mlir;
using namespace mlir::linalg; using namespace mlir::linalg;
/// Helper function to extract the operand types that are passed to the /// Helper function to extract the operand types that are passed to the
/// generated CallOp. MemRefTypes have their layout canonicalized since the /// generated CallOp. MemRefTypes have their layout canonicalized since the
/// information is not used in signature generation. /// information is not used in signature generation.
/// Note that static size information is not modified. /// Note that static size information is not modified.
▲ Show 20 Lines • Show All 71 Lines • ▼ Show 20 Lines
LogicalResult mlir::linalg::LinalgOpToLibraryCallRewrite::matchAndRewrite( LogicalResult mlir::linalg::LinalgOpToLibraryCallRewrite::matchAndRewrite(
LinalgOp op, PatternRewriter &rewriter) const { LinalgOp op, PatternRewriter &rewriter) const {
auto libraryCallName = getLibraryCallSymbolRef(op, rewriter); auto libraryCallName = getLibraryCallSymbolRef(op, rewriter);
if (!libraryCallName) if (!libraryCallName)
return failure(); return failure();
// TODO: Add support for more complex library call signatures that include // TODO: Add support for more complex library call signatures that include
// indices or captured values. // indices or captured values.
rewriter.replaceOpWithNewOp<mlir::CallOp>( rewriter.replaceOpWithNewOp<func::CallOp>(
op, libraryCallName.getValue(), TypeRange(), op, libraryCallName.getValue(), TypeRange(),
createTypeCanonicalizedMemRefOperands(rewriter, op->getLoc(), createTypeCanonicalizedMemRefOperands(rewriter, op->getLoc(),
op->getOperands())); op->getOperands()));
return success(); return success();
} }
/// Populate the given list with patterns that convert from Linalg to Standard. /// Populate the given list with patterns that convert from Linalg to Standard.
Show All 10 Linesstruct ConvertLinalgToStandardPass
void runOnOperation() override; void runOnOperation() override;
}; };
} // namespace } // namespace
void ConvertLinalgToStandardPass::runOnOperation() { void ConvertLinalgToStandardPass::runOnOperation() {
auto module = getOperation(); auto module = getOperation();
ConversionTarget target(getContext()); ConversionTarget target(getContext());
target.addLegalDialect<AffineDialect, arith::ArithmeticDialect, target.addLegalDialect<AffineDialect, arith::ArithmeticDialect,
memref::MemRefDialect, scf::SCFDialect, func::FuncDialect, memref::MemRefDialect,
StandardOpsDialect>(); scf::SCFDialect>();
target.addLegalOp<ModuleOp, FuncOp, ReturnOp>(); target.addLegalOp<ModuleOp, FuncOp, func::ReturnOp>();
RewritePatternSet patterns(&getContext()); RewritePatternSet patterns(&getContext());
populateLinalgToStandardConversionPatterns(patterns); populateLinalgToStandardConversionPatterns(patterns);
if (failed(applyFullConversion(module, target, std::move(patterns)))) if (failed(applyFullConversion(module, target, std::move(patterns))))
signalPassFailure(); signalPassFailure();
} }
std::unique_ptr<OperationPass<ModuleOp>> std::unique_ptr<OperationPass<ModuleOp>>
mlir::createConvertLinalgToStandardPass() { mlir::createConvertLinalgToStandardPass() {
return std::make_unique<ConvertLinalgToStandardPass>(); return std::make_unique<ConvertLinalgToStandardPass>();
} }

mlir/lib/Conversion/MathToLibm/CMakeLists.txt

add_mlir_conversion_library(MLIRMathToLibm add_mlir_conversion_library(MLIRMathToLibm
MathToLibm.cpp MathToLibm.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/MathToLibm ${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/MathToLibm
DEPENDS DEPENDS
MLIRConversionPassIncGen MLIRConversionPassIncGen
LINK_COMPONENTS LINK_COMPONENTS
Core Core
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRArithmetic MLIRArithmetic
MLIRDialectUtils MLIRDialectUtils
MLIRFuncTransforms
MLIRMath MLIRMath
MLIRStandardOpsTransforms
MLIRVector MLIRVector
MLIRVectorUtils MLIRVectorUtils
) )

mlir/lib/Conversion/MathToLibm/MathToLibm.cpp

//===-- MathToLibm.cpp - conversion from Math to libm calls ---------------===// //===-- MathToLibm.cpp - conversion from Math to libm calls ---------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/MathToLibm/MathToLibm.h" #include "mlir/Conversion/MathToLibm/MathToLibm.h"
#include "../PassDetail.h" #include "../PassDetail.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Math/IR/Math.h" #include "mlir/Dialect/Math/IR/Math.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Utils/IndexingUtils.h" #include "mlir/Dialect/Utils/IndexingUtils.h"
#include "mlir/Dialect/Vector/IR/VectorOps.h" #include "mlir/Dialect/Vector/IR/VectorOps.h"
#include "mlir/Dialect/Vector/Utils/VectorUtils.h" #include "mlir/Dialect/Vector/Utils/VectorUtils.h"
#include "mlir/IR/BuiltinDialect.h" #include "mlir/IR/BuiltinDialect.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
using namespace mlir; using namespace mlir;
▲ Show 20 Lines • Show All 79 Lines • ▼ Show 20 Lines if (!opFunc) {
auto opFunctionTy = FunctionType::get( auto opFunctionTy = FunctionType::get(
rewriter.getContext(), op->getOperandTypes(), op->getResultTypes()); rewriter.getContext(), op->getOperandTypes(), op->getResultTypes());
opFunc = opFunc =
rewriter.create<FuncOp>(rewriter.getUnknownLoc(), name, opFunctionTy); rewriter.create<FuncOp>(rewriter.getUnknownLoc(), name, opFunctionTy);
opFunc.setPrivate(); opFunc.setPrivate();
} }
assert(isa<FunctionOpInterface>(SymbolTable::lookupSymbolIn(module, name))); assert(isa<FunctionOpInterface>(SymbolTable::lookupSymbolIn(module, name)));
rewriter.replaceOpWithNewOp<CallOp>(op, name, op.getType(), rewriter.replaceOpWithNewOp<func::CallOp>(op, name, op.getType(),
op->getOperands()); op->getOperands());
return success(); return success();
} }
void mlir::populateMathToLibmConversionPatterns(RewritePatternSet &patterns, void mlir::populateMathToLibmConversionPatterns(RewritePatternSet &patterns,
PatternBenefit benefit) { PatternBenefit benefit) {
patterns.add<VecOpToScalarOp<math::Atan2Op>, VecOpToScalarOp<math::ExpM1Op>, patterns.add<VecOpToScalarOp<math::Atan2Op>, VecOpToScalarOp<math::ExpM1Op>,
VecOpToScalarOp<math::TanhOp>>(patterns.getContext(), benefit); VecOpToScalarOp<math::TanhOp>>(patterns.getContext(), benefit);
Show All 17 Lines
void ConvertMathToLibmPass::runOnOperation() { void ConvertMathToLibmPass::runOnOperation() {
auto module = getOperation(); auto module = getOperation();
RewritePatternSet patterns(&getContext()); RewritePatternSet patterns(&getContext());
populateMathToLibmConversionPatterns(patterns, /*benefit=*/1); populateMathToLibmConversionPatterns(patterns, /*benefit=*/1);
ConversionTarget target(getContext()); ConversionTarget target(getContext());
target.addLegalDialect<arith::ArithmeticDialect, BuiltinDialect, target.addLegalDialect<arith::ArithmeticDialect, BuiltinDialect,
StandardOpsDialect, vector::VectorDialect>(); func::FuncDialect, vector::VectorDialect>();
target.addIllegalDialect<math::MathDialect>(); target.addIllegalDialect<math::MathDialect>();
if (failed(applyPartialConversion(module, target, std::move(patterns)))) if (failed(applyPartialConversion(module, target, std::move(patterns))))
signalPassFailure(); signalPassFailure();
} }
std::unique_ptr<OperationPass<ModuleOp>> mlir::createConvertMathToLibmPass() { std::unique_ptr<OperationPass<ModuleOp>> mlir::createConvertMathToLibmPass() {
return std::make_unique<ConvertMathToLibmPass>(); return std::make_unique<ConvertMathToLibmPass>();
} }

mlir/lib/Conversion/MemRefToLLVM/MemRefToLLVM.cpp

Show First 20 Lines • Show All 1,747 Lines • ▼ Show 20 Lines matchAndRewrite(memref::ViewOp viewOp, OpAdaptor adaptor,
return success(); return success();
} }
}; };
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// AtomicRMWOpLowering // AtomicRMWOpLowering
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
/// Try to match the kind of a std.atomic_rmw to determine whether to use a /// Try to match the kind of a memref.atomic_rmw to determine whether to use a
/// lowering to llvm.atomicrmw or fallback to llvm.cmpxchg. /// lowering to llvm.atomicrmw or fallback to llvm.cmpxchg.
static Optional<LLVM::AtomicBinOp> static Optional<LLVM::AtomicBinOp>
matchSimpleAtomicOp(memref::AtomicRMWOp atomicOp) { matchSimpleAtomicOp(memref::AtomicRMWOp atomicOp) {
switch (atomicOp.kind()) { switch (atomicOp.kind()) {
case arith::AtomicRMWKind::addf: case arith::AtomicRMWKind::addf:
return LLVM::AtomicBinOp::fadd; return LLVM::AtomicBinOp::fadd;
case arith::AtomicRMWKind::addi: case arith::AtomicRMWKind::addi:
return LLVM::AtomicBinOp::add; return LLVM::AtomicBinOp::add;
▲ Show 20 Lines • Show All 108 Lines • Show Last 20 Lines

mlir/lib/Conversion/OpenACCToSCF/OpenACCToSCF.cpp

//===- OpenACCToSCF.cpp - OpenACC condition to SCF if conversion ----------===// //===- OpenACCToSCF.cpp - OpenACC condition to SCF if conversion ----------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "../PassDetail.h" #include "../PassDetail.h"
#include "mlir/Conversion/OpenACCToSCF/ConvertOpenACCToSCF.h" #include "mlir/Conversion/OpenACCToSCF/ConvertOpenACCToSCF.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/OpenACC/OpenACC.h" #include "mlir/Dialect/OpenACC/OpenACC.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
using namespace mlir; using namespace mlir;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Conversion patterns // Conversion patterns
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 69 Lines • Show Last 20 Lines

mlir/lib/Conversion/PassDetail.h

Show All 9 Lines
#define CONVERSION_PASSDETAIL_H_ #define CONVERSION_PASSDETAIL_H_
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Conversion/GPUToROCDL/Runtimes.h" #include "mlir/Conversion/GPUToROCDL/Runtimes.h"
namespace mlir { namespace mlir {
class AffineDialect; class AffineDialect;
class StandardOpsDialect;
// Forward declaration from Dialect.h // Forward declaration from Dialect.h
template <typename ConcreteDialect> template <typename ConcreteDialect>
void registerDialect(DialectRegistry &registry); void registerDialect(DialectRegistry &registry);
namespace acc { namespace acc {
class OpenACCDialect; class OpenACCDialect;
} // namespace acc } // namespace acc
Show All 10 Lines
class ComplexDialect; class ComplexDialect;
} // namespace complex } // namespace complex
namespace gpu { namespace gpu {
class GPUDialect; class GPUDialect;
class GPUModuleOp; class GPUModuleOp;
} // namespace gpu } // namespace gpu
namespace func {
class FuncDialect;
} // namespace func
namespace LLVM { namespace LLVM {
class LLVMDialect; class LLVMDialect;
} // namespace LLVM } // namespace LLVM
namespace NVVM { namespace NVVM {
class NVVMDialect; class NVVMDialect;
} // namespace NVVM } // namespace NVVM
▲ Show 20 Lines • Show All 50 Lines • Show Last 20 Lines

mlir/lib/Conversion/SCFToGPU/CMakeLists.txt

add_mlir_conversion_library(MLIRSCFToGPU add_mlir_conversion_library(MLIRSCFToGPU
SCFToGPU.cpp SCFToGPU.cpp
SCFToGPUPass.cpp SCFToGPUPass.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/SCFToGPU ${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/SCFToGPU
DEPENDS DEPENDS
MLIRConversionPassIncGen MLIRConversionPassIncGen
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRAffine MLIRAffine
MLIRAffineToStandard MLIRAffineToStandard
MLIRArithmetic MLIRArithmetic
MLIRComplex MLIRComplex
MLIRFunc
MLIRGPUTransforms MLIRGPUTransforms
MLIRIR MLIRIR
MLIRLinalg MLIRLinalg
MLIRMemRef MLIRMemRef
MLIRPass MLIRPass
MLIRStandard
MLIRSupport MLIRSupport
MLIRTransforms MLIRTransforms
) )

mlir/lib/Conversion/SCFToGPU/SCFToGPU.cpp

Show All 11 Lines
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/SCFToGPU/SCFToGPU.h" #include "mlir/Conversion/SCFToGPU/SCFToGPU.h"
#include "mlir/Conversion/AffineToStandard/AffineToStandard.h" #include "mlir/Conversion/AffineToStandard/AffineToStandard.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/GPU/GPUDialect.h" #include "mlir/Dialect/GPU/GPUDialect.h"
#include "mlir/Dialect/GPU/ParallelLoopMapper.h" #include "mlir/Dialect/GPU/ParallelLoopMapper.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
#include "mlir/Transforms/RegionUtils.h" #include "mlir/Transforms/RegionUtils.h"
#include "llvm/ADT/Sequence.h" #include "llvm/ADT/Sequence.h"
▲ Show 20 Lines • Show All 654 Lines • Show Last 20 Lines

mlir/lib/Conversion/SCFToGPU/SCFToGPUPass.cpp

//===- SCFToGPUPass.cpp - Convert a loop nest to a GPU kernel -----------===// //===- SCFToGPUPass.cpp - Convert a loop nest to a GPU kernel -----------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/SCFToGPU/SCFToGPUPass.h" #include "mlir/Conversion/SCFToGPU/SCFToGPUPass.h"
#include "../PassDetail.h" #include "../PassDetail.h"
#include "mlir/Conversion/SCFToGPU/SCFToGPU.h" #include "mlir/Conversion/SCFToGPU/SCFToGPU.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Complex/IR/Complex.h" #include "mlir/Dialect/Complex/IR/Complex.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/GPU/GPUDialect.h" #include "mlir/Dialect/GPU/GPUDialect.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/ADT/ArrayRef.h" #include "llvm/ADT/ArrayRef.h"
#include "llvm/Support/CommandLine.h" #include "llvm/Support/CommandLine.h"
using namespace mlir; using namespace mlir;
using namespace mlir::scf; using namespace mlir::scf;
▲ Show 20 Lines • Show All 50 Lines • Show Last 20 Lines

mlir/lib/Conversion/SCFToOpenMP/CMakeLists.txt

add_mlir_conversion_library(MLIRSCFToOpenMP add_mlir_conversion_library(MLIRSCFToOpenMP
SCFToOpenMP.cpp SCFToOpenMP.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/SCFToStandard ${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/SCFToStandard
DEPENDS DEPENDS
MLIRConversionPassIncGen MLIRConversionPassIncGen
LINK_COMPONENTS LINK_COMPONENTS
Core Core
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRAnalysis MLIRAnalysis
MLIRArithmetic MLIRArithmetic
MLIRFunc
MLIRLLVMIR MLIRLLVMIR
MLIROpenMP MLIROpenMP
MLIRSCF MLIRSCF
MLIRStandard
MLIRTransforms MLIRTransforms
) )

mlir/lib/Conversion/SCFToOpenMP/SCFToOpenMP.cpp

Show All 10 Lines
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/SCFToOpenMP/SCFToOpenMP.h" #include "mlir/Conversion/SCFToOpenMP/SCFToOpenMP.h"
#include "../PassDetail.h" #include "../PassDetail.h"
#include "mlir/Analysis/SliceAnalysis.h" #include "mlir/Analysis/SliceAnalysis.h"
#include "mlir/Dialect/Affine/Analysis/LoopAnalysis.h" #include "mlir/Dialect/Affine/Analysis/LoopAnalysis.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/OpenMP/OpenMPDialect.h" #include "mlir/Dialect/OpenMP/OpenMPDialect.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/ImplicitLocOpBuilder.h" #include "mlir/IR/ImplicitLocOpBuilder.h"
#include "mlir/IR/SymbolTable.h" #include "mlir/IR/SymbolTable.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
using namespace mlir; using namespace mlir;
/// Matches a block containing a "simple" reduction. The expected shape of the /// Matches a block containing a "simple" reduction. The expected shape of the
/// block is as follows. /// block is as follows.
▲ Show 20 Lines • Show All 432 Lines • Show Last 20 Lines

mlir/lib/Conversion/SCFToSPIRV/CMakeLists.txt

add_mlir_conversion_library(MLIRSCFToSPIRV add_mlir_conversion_library(MLIRSCFToSPIRV
SCFToSPIRV.cpp SCFToSPIRV.cpp
SCFToSPIRVPass.cpp SCFToSPIRVPass.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/SCFToSPIRV ${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/SCFToSPIRV
DEPENDS DEPENDS
MLIRConversionPassIncGen MLIRConversionPassIncGen
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRArithmeticToSPIRV MLIRArithmeticToSPIRV
MLIRMemRefToSPIRV MLIRMemRefToSPIRV
MLIRSPIRV MLIRSPIRV
MLIRSPIRVConversion MLIRSPIRVConversion
MLIRStandardToSPIRV MLIRStandardToSPIRV
MLIRFunc
MLIRIR MLIRIR
MLIRPass MLIRPass
MLIRSCF MLIRSCF
MLIRStandard
MLIRSupport MLIRSupport
MLIRTransforms MLIRTransforms
) )

mlir/lib/Conversion/SPIRVToLLVM/ConvertLaunchFuncToLLVMCalls.cpp

Show All 14 Lines
#include "mlir/Conversion/ArithmeticToLLVM/ArithmeticToLLVM.h" #include "mlir/Conversion/ArithmeticToLLVM/ArithmeticToLLVM.h"
#include "mlir/Conversion/LLVMCommon/LoweringOptions.h" #include "mlir/Conversion/LLVMCommon/LoweringOptions.h"
#include "mlir/Conversion/LLVMCommon/Pattern.h" #include "mlir/Conversion/LLVMCommon/Pattern.h"
#include "mlir/Conversion/LLVMCommon/TypeConverter.h" #include "mlir/Conversion/LLVMCommon/TypeConverter.h"
#include "mlir/Conversion/MemRefToLLVM/MemRefToLLVM.h" #include "mlir/Conversion/MemRefToLLVM/MemRefToLLVM.h"
#include "mlir/Conversion/SPIRVToLLVM/SPIRVToLLVM.h" #include "mlir/Conversion/SPIRVToLLVM/SPIRVToLLVM.h"
#include "mlir/Conversion/SPIRVToLLVM/SPIRVToLLVMPass.h" #include "mlir/Conversion/SPIRVToLLVM/SPIRVToLLVMPass.h"
#include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h" #include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/GPU/GPUDialect.h" #include "mlir/Dialect/GPU/GPUDialect.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/SPIRV/IR/SPIRVOps.h" #include "mlir/Dialect/SPIRV/IR/SPIRVOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/SymbolTable.h" #include "mlir/IR/SymbolTable.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/ADT/DenseMap.h" #include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/StringExtras.h" #include "llvm/ADT/StringExtras.h"
#include "llvm/Support/FormatVariadic.h" #include "llvm/Support/FormatVariadic.h"
▲ Show 20 Lines • Show All 283 Lines • Show Last 20 Lines

mlir/lib/Conversion/SPIRVToLLVM/SPIRVToLLVM.cpp

//===- SPIRVToLLVM.cpp - SPIR-V to LLVM Patterns --------------------------===// //===- SPIRVToLLVM.cpp - SPIR-V to LLVM Patterns --------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements patterns to convert SPIR-V dialect to LLVM dialect. // This file implements patterns to convert SPIR-V dialect to LLVM dialect.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/SPIRVToLLVM/SPIRVToLLVM.h" #include "mlir/Conversion/SPIRVToLLVM/SPIRVToLLVM.h"
#include "mlir/Conversion/LLVMCommon/Pattern.h" #include "mlir/Conversion/LLVMCommon/Pattern.h"
#include "mlir/Conversion/LLVMCommon/TypeConverter.h" #include "mlir/Conversion/LLVMCommon/TypeConverter.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h" #include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h"
#include "mlir/Dialect/SPIRV/IR/SPIRVOps.h" #include "mlir/Dialect/SPIRV/IR/SPIRVOps.h"
#include "mlir/Dialect/SPIRV/Utils/LayoutUtils.h" #include "mlir/Dialect/SPIRV/Utils/LayoutUtils.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Support/LogicalResult.h" #include "mlir/Support/LogicalResult.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#include "llvm/Support/FormatVariadic.h" #include "llvm/Support/FormatVariadic.h"
#define DEBUG_TYPE "spirv-to-llvm-pattern" #define DEBUG_TYPE "spirv-to-llvm-pattern"
▲ Show 20 Lines • Show All 1,583 Lines • Show Last 20 Lines

mlir/lib/Conversion/ShapeToStandard/ConvertShapeConstraints.cpp

//===- ConvertShapeConstraints.cpp - Conversion of shape constraints ------===// //===- ConvertShapeConstraints.cpp - Conversion of shape constraints ------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/ShapeToStandard/ShapeToStandard.h" #include "mlir/Conversion/ShapeToStandard/ShapeToStandard.h"
#include "../PassDetail.h" #include "../PassDetail.h"
#include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h" #include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/Shape/IR/Shape.h" #include "mlir/Dialect/Shape/IR/Shape.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Pass/PassRegistry.h" #include "mlir/Pass/PassRegistry.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
using namespace mlir; using namespace mlir;
namespace { namespace {
▲ Show 20 Lines • Show All 47 Lines • Show Last 20 Lines

mlir/lib/Conversion/ShapeToStandard/ShapeToStandard.cpp

//===- ShapeToStandard.cpp - conversion from Shape to Standard dialect ----===// //===- ShapeToStandard.cpp - conversion from Shape to Standard dialect ----===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/ShapeToStandard/ShapeToStandard.h" #include "mlir/Conversion/ShapeToStandard/ShapeToStandard.h"
#include "../PassDetail.h" #include "../PassDetail.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/Shape/IR/Shape.h" #include "mlir/Dialect/Shape/IR/Shape.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/ImplicitLocOpBuilder.h" #include "mlir/IR/ImplicitLocOpBuilder.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/ADT/STLExtras.h" #include "llvm/ADT/STLExtras.h"
using namespace mlir; using namespace mlir;
using namespace mlir::shape; using namespace mlir::shape;
▲ Show 20 Lines • Show All 639 Lines • ▼ Show 20 Linesclass ConvertShapeToStandardPass
void runOnOperation() override; void runOnOperation() override;
}; };
} // namespace } // namespace
void ConvertShapeToStandardPass::runOnOperation() { void ConvertShapeToStandardPass::runOnOperation() {
// Setup target legality. // Setup target legality.
MLIRContext &ctx = getContext(); MLIRContext &ctx = getContext();
ConversionTarget target(ctx); ConversionTarget target(ctx);
target.addLegalDialect<arith::ArithmeticDialect, StandardOpsDialect, target.addLegalDialect<arith::ArithmeticDialect, func::FuncDialect,
SCFDialect, tensor::TensorDialect>(); SCFDialect, tensor::TensorDialect>();
target.addLegalOp<CstrRequireOp, FuncOp, ModuleOp>(); target.addLegalOp<CstrRequireOp, FuncOp, ModuleOp>();
// Setup conversion patterns. // Setup conversion patterns.
RewritePatternSet patterns(&ctx); RewritePatternSet patterns(&ctx);
populateShapeToStandardConversionPatterns(patterns); populateShapeToStandardConversionPatterns(patterns);
// Apply conversion. // Apply conversion.
Show All 31 Lines

mlir/lib/Conversion/StandardToLLVM/CMakeLists.txt

Show All 10 Linesadd_mlir_conversion_library(MLIRStandardToLLVM
LINK_COMPONENTS LINK_COMPONENTS
Core Core
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRAnalysis MLIRAnalysis
MLIRArithmeticToLLVM MLIRArithmeticToLLVM
MLIRControlFlowToLLVM MLIRControlFlowToLLVM
MLIRDataLayoutInterfaces MLIRDataLayoutInterfaces
MLIRFunc
MLIRLLVMCommonConversion MLIRLLVMCommonConversion
MLIRLLVMIR MLIRLLVMIR
MLIRMath MLIRMath
MLIRStandard
MLIRTransforms MLIRTransforms
) )

mlir/lib/Conversion/StandardToLLVM/StandardToLLVM.cpp

Show All 14 Lines
#include "mlir/Analysis/DataLayoutAnalysis.h" #include "mlir/Analysis/DataLayoutAnalysis.h"
#include "mlir/Conversion/ArithmeticToLLVM/ArithmeticToLLVM.h" #include "mlir/Conversion/ArithmeticToLLVM/ArithmeticToLLVM.h"
#include "mlir/Conversion/ControlFlowToLLVM/ControlFlowToLLVM.h" #include "mlir/Conversion/ControlFlowToLLVM/ControlFlowToLLVM.h"
#include "mlir/Conversion/LLVMCommon/ConversionTarget.h" #include "mlir/Conversion/LLVMCommon/ConversionTarget.h"
#include "mlir/Conversion/LLVMCommon/Pattern.h" #include "mlir/Conversion/LLVMCommon/Pattern.h"
#include "mlir/Conversion/LLVMCommon/VectorPattern.h" #include "mlir/Conversion/LLVMCommon/VectorPattern.h"
#include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h" #include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h"
#include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVMPass.h" #include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVMPass.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/LLVMIR/FunctionCallUtils.h" #include "mlir/Dialect/LLVMIR/FunctionCallUtils.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Utils/StaticValueUtils.h" #include "mlir/Dialect/Utils/StaticValueUtils.h"
#include "mlir/IR/Attributes.h" #include "mlir/IR/Attributes.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/IR/TypeUtilities.h" #include "mlir/IR/TypeUtilities.h"
#include "mlir/Support/LogicalResult.h" #include "mlir/Support/LogicalResult.h"
Show All 16 Lines
/// `LLVMFuncOp::build`. If `filterArgAttrs` is set, also filter out argument /// `LLVMFuncOp::build`. If `filterArgAttrs` is set, also filter out argument
/// attributes. /// attributes.
static void filterFuncAttributes(ArrayRef<NamedAttribute> attrs, static void filterFuncAttributes(ArrayRef<NamedAttribute> attrs,
bool filterArgAttrs, bool filterArgAttrs,
SmallVectorImpl<NamedAttribute> &result) { SmallVectorImpl<NamedAttribute> &result) {
for (const auto &attr : attrs) { for (const auto &attr : attrs) {
if (attr.getName() == SymbolTable::getSymbolAttrName() || if (attr.getName() == SymbolTable::getSymbolAttrName() ||
attr.getName() == FunctionOpInterface::getTypeAttrName() || attr.getName() == FunctionOpInterface::getTypeAttrName() ||
attr.getName() == "std.varargs" || attr.getName() == "func.varargs" ||
(filterArgAttrs && (filterArgAttrs &&
attr.getName() == FunctionOpInterface::getArgDictAttrName())) attr.getName() == FunctionOpInterface::getArgDictAttrName()))
continue; continue;
result.push_back(attr); result.push_back(attr);
} }
} }
/// Creates an auxiliary function with pointer-to-memref-descriptor-struct /// Creates an auxiliary function with pointer-to-memref-descriptor-struct
▲ Show 20 Lines • Show All 156 Lines • ▼ Show 20 Linesprotected:
// Convert input FuncOp to LLVMFuncOp by using the LLVMTypeConverter provided // Convert input FuncOp to LLVMFuncOp by using the LLVMTypeConverter provided
// to this legalization pattern. // to this legalization pattern.
LLVM::LLVMFuncOp LLVM::LLVMFuncOp
convertFuncOpToLLVMFuncOp(FuncOp funcOp, convertFuncOpToLLVMFuncOp(FuncOp funcOp,
ConversionPatternRewriter &rewriter) const { ConversionPatternRewriter &rewriter) const {
// Convert the original function arguments. They are converted using the // Convert the original function arguments. They are converted using the
// LLVMTypeConverter provided to this legalization pattern. // LLVMTypeConverter provided to this legalization pattern.
auto varargsAttr = funcOp->getAttrOfType<BoolAttr>("std.varargs"); auto varargsAttr = funcOp->getAttrOfType<BoolAttr>("func.varargs");
TypeConverter::SignatureConversion result(funcOp.getNumArguments()); TypeConverter::SignatureConversion result(funcOp.getNumArguments());
auto llvmType = getTypeConverter()->convertFunctionSignature( auto llvmType = getTypeConverter()->convertFunctionSignature(
funcOp.getType(), varargsAttr && varargsAttr.getValue(), result); funcOp.getType(), varargsAttr && varargsAttr.getValue(), result);
if (!llvmType) if (!llvmType)
return nullptr; return nullptr;
// Propagate argument attributes to all converted arguments obtained after // Propagate argument attributes to all converted arguments obtained after
// converting a given original argument. // converting a given original argument.
▲ Show 20 Lines • Show All 143 Lines • ▼ Show 20 Lines for (auto it : llvm::zip(blockArgs, oldArgTypes)) {
rewriter.replaceOp(placeholder, {desc}); rewriter.replaceOp(placeholder, {desc});
} }
rewriter.eraseOp(funcOp); rewriter.eraseOp(funcOp);
return success(); return success();
} }
}; };
struct ConstantOpLowering : public ConvertOpToLLVMPattern<ConstantOp> { struct ConstantOpLowering : public ConvertOpToLLVMPattern<func::ConstantOp> {
using ConvertOpToLLVMPattern<ConstantOp>::ConvertOpToLLVMPattern; using ConvertOpToLLVMPattern<func::ConstantOp>::ConvertOpToLLVMPattern;
LogicalResult LogicalResult
matchAndRewrite(ConstantOp op, OpAdaptor adaptor, matchAndRewrite(func::ConstantOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override { ConversionPatternRewriter &rewriter) const override {
auto type = typeConverter->convertType(op.getResult().getType()); auto type = typeConverter->convertType(op.getResult().getType());
if (!type || !LLVM::isCompatibleType(type)) if (!type || !LLVM::isCompatibleType(type))
return rewriter.notifyMatchFailure(op, "failed to convert result type"); return rewriter.notifyMatchFailure(op, "failed to convert result type");
auto newOp = auto newOp =
rewriter.create<LLVM::AddressOfOp>(op.getLoc(), type, op.getValue()); rewriter.create<LLVM::AddressOfOp>(op.getLoc(), type, op.getValue());
for (const NamedAttribute &attr : op->getAttrs()) { for (const NamedAttribute &attr : op->getAttrs()) {
▲ Show 20 Lines • Show All 65 Lines • ▼ Show 20 Lines if (this->getTypeConverter()->getOptions().useBarePtrCallConv) {
return failure(); return failure();
} }
rewriter.replaceOp(callOp, results); rewriter.replaceOp(callOp, results);
return success(); return success();
} }
}; };
struct CallOpLowering : public CallOpInterfaceLowering<CallOp> { struct CallOpLowering : public CallOpInterfaceLowering<func::CallOp> {
using Super::Super; using Super::Super;
}; };
struct CallIndirectOpLowering : public CallOpInterfaceLowering<CallIndirectOp> { struct CallIndirectOpLowering
: public CallOpInterfaceLowering<func::CallIndirectOp> {
using Super::Super; using Super::Super;
}; };
struct UnrealizedConversionCastOpLowering struct UnrealizedConversionCastOpLowering
: public ConvertOpToLLVMPattern<UnrealizedConversionCastOp> { : public ConvertOpToLLVMPattern<UnrealizedConversionCastOp> {
using ConvertOpToLLVMPattern< using ConvertOpToLLVMPattern<
UnrealizedConversionCastOp>::ConvertOpToLLVMPattern; UnrealizedConversionCastOp>::ConvertOpToLLVMPattern;
Show All 20 Lines
}; };
// Special lowering pattern for `ReturnOps`. Unlike all other operations, // Special lowering pattern for `ReturnOps`. Unlike all other operations,
// `ReturnOp` interacts with the function signature and must have as many // `ReturnOp` interacts with the function signature and must have as many
// operands as the function has return values. Because in LLVM IR, functions // operands as the function has return values. Because in LLVM IR, functions
// can only return 0 or 1 value, we pack multiple values into a structure type. // can only return 0 or 1 value, we pack multiple values into a structure type.
// Emit `UndefOp` followed by `InsertValueOp`s to create such structure if // Emit `UndefOp` followed by `InsertValueOp`s to create such structure if
// necessary before returning it // necessary before returning it
struct ReturnOpLowering : public ConvertOpToLLVMPattern<ReturnOp> { struct ReturnOpLowering : public ConvertOpToLLVMPattern<func::ReturnOp> {
using ConvertOpToLLVMPattern<ReturnOp>::ConvertOpToLLVMPattern; using ConvertOpToLLVMPattern<func::ReturnOp>::ConvertOpToLLVMPattern;
LogicalResult LogicalResult
matchAndRewrite(ReturnOp op, OpAdaptor adaptor, matchAndRewrite(func::ReturnOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override { ConversionPatternRewriter &rewriter) const override {
Location loc = op.getLoc(); Location loc = op.getLoc();
unsigned numArguments = op.getNumOperands(); unsigned numArguments = op.getNumOperands();
SmallVector<Value, 4> updatedOperands; SmallVector<Value, 4> updatedOperands;
if (getTypeConverter()->getOptions().useBarePtrCallConv) { if (getTypeConverter()->getOptions().useBarePtrCallConv) {
// For the bare-ptr calling convention, extract the aligned pointer to // For the bare-ptr calling convention, extract the aligned pointer to
// be returned from the memref descriptor. // be returned from the memref descriptor.
▲ Show 20 Lines • Show All 146 Lines • Show Last 20 Lines

mlir/lib/Conversion/StandardToSPIRV/CMakeLists.txt

add_mlir_conversion_library(MLIRStandardToSPIRV add_mlir_conversion_library(MLIRStandardToSPIRV
StandardToSPIRV.cpp StandardToSPIRV.cpp
StandardToSPIRVPass.cpp StandardToSPIRVPass.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/SPIRV ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/SPIRV
${MLIR_MAIN_INCLUDE_DIR}/mlir/IR ${MLIR_MAIN_INCLUDE_DIR}/mlir/IR
DEPENDS DEPENDS
MLIRConversionPassIncGen MLIRConversionPassIncGen
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRArithmeticToSPIRV MLIRArithmeticToSPIRV
MLIRControlFlowToSPIRV MLIRControlFlowToSPIRV
MLIRFunc
MLIRIR MLIRIR
MLIRMathToSPIRV MLIRMathToSPIRV
MLIRMemRef MLIRMemRef
MLIRPass MLIRPass
MLIRSPIRV MLIRSPIRV
MLIRSPIRVConversion MLIRSPIRVConversion
MLIRSupport MLIRSupport
MLIRTransformUtils MLIRTransformUtils
MLIRStandard
MLIRTensor MLIRTensor
) )

mlir/lib/Conversion/StandardToSPIRV/StandardToSPIRV.cpp

//===- StandardToSPIRV.cpp - Standard to SPIR-V Patterns ------------------===// //===- StandardToSPIRV.cpp - Standard to SPIR-V Patterns ------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements patterns to convert standard dialect to SPIR-V dialect. // This file implements patterns to convert standard dialect to SPIR-V dialect.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "../SPIRVCommon/Pattern.h" #include "../SPIRVCommon/Pattern.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h" #include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h"
#include "mlir/Dialect/SPIRV/IR/SPIRVOps.h" #include "mlir/Dialect/SPIRV/IR/SPIRVOps.h"
#include "mlir/Dialect/SPIRV/Transforms/SPIRVConversion.h" #include "mlir/Dialect/SPIRV/Transforms/SPIRVConversion.h"
#include "mlir/Dialect/SPIRV/Utils/LayoutUtils.h" #include "mlir/Dialect/SPIRV/Utils/LayoutUtils.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/IR/AffineMap.h" #include "mlir/IR/AffineMap.h"
#include "mlir/Support/LogicalResult.h" #include "mlir/Support/LogicalResult.h"
#include "llvm/ADT/SetVector.h" #include "llvm/ADT/SetVector.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#define DEBUG_TYPE "std-to-spirv-pattern" #define DEBUG_TYPE "std-to-spirv-pattern"
using namespace mlir; using namespace mlir;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Operation conversion // Operation conversion
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Note that DRR cannot be used for the patterns in this file: we may need to // Note that DRR cannot be used for the patterns in this file: we may need to
// convert type along the way, which requires ConversionPattern. DRR generates // convert type along the way, which requires ConversionPattern. DRR generates
// normal RewritePattern. // normal RewritePattern.
namespace { namespace {
/// Converts std.return to spv.Return. /// Converts func.return to spv.Return.
class ReturnOpPattern final : public OpConversionPattern<ReturnOp> { class ReturnOpPattern final : public OpConversionPattern<func::ReturnOp> {
public: public:
using OpConversionPattern<ReturnOp>::OpConversionPattern; using OpConversionPattern<func::ReturnOp>::OpConversionPattern;
LogicalResult LogicalResult
matchAndRewrite(ReturnOp returnOp, OpAdaptor adaptor, matchAndRewrite(func::ReturnOp returnOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override; ConversionPatternRewriter &rewriter) const override;
}; };
/// Converts tensor.extract into loading using access chains from SPIR-V local /// Converts tensor.extract into loading using access chains from SPIR-V local
/// variables. /// variables.
class TensorExtractPattern final class TensorExtractPattern final
: public OpConversionPattern<tensor::ExtractOp> { : public OpConversionPattern<tensor::ExtractOp> {
public: public:
▲ Show 20 Lines • Show All 55 Lines • ▼ Show 20 Lines
} // namespace } // namespace
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// ReturnOp // ReturnOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
LogicalResult LogicalResult
ReturnOpPattern::matchAndRewrite(ReturnOp returnOp, OpAdaptor adaptor, ReturnOpPattern::matchAndRewrite(func::ReturnOp returnOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const { ConversionPatternRewriter &rewriter) const {
if (returnOp.getNumOperands() > 1) if (returnOp.getNumOperands() > 1)
return failure(); return failure();
if (returnOp.getNumOperands() == 1) { if (returnOp.getNumOperands() == 1) {
rewriter.replaceOpWithNewOp<spirv::ReturnValueOp>(returnOp, rewriter.replaceOpWithNewOp<spirv::ReturnValueOp>(returnOp,
adaptor.getOperands()[0]); adaptor.getOperands()[0]);
} else { } else {
Show All 34 Lines

mlir/lib/Conversion/TosaToLinalg/TosaToLinalg.cpp

//===- TosaToLinalg.cpp - Lowering Tosa to Linalg Dialect -----------------===// //===- TosaToLinalg.cpp - Lowering Tosa to Linalg Dialect -----------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// These rewriters lower from the Tosa to the Linalg dialect. // These rewriters lower from the Tosa to the Linalg dialect.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/TosaToLinalg/TosaToLinalg.h" #include "mlir/Conversion/TosaToLinalg/TosaToLinalg.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/Math/IR/Math.h" #include "mlir/Dialect/Math/IR/Math.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/Dialect/Tensor/Utils/Utils.h" #include "mlir/Dialect/Tensor/Utils/Utils.h"
#include "mlir/Dialect/Tosa/IR/TosaOps.h" #include "mlir/Dialect/Tosa/IR/TosaOps.h"
#include "mlir/Dialect/Tosa/Utils/CoversionUtils.h" #include "mlir/Dialect/Tosa/Utils/CoversionUtils.h"
#include "mlir/Dialect/Utils/ReshapeOpsUtils.h" #include "mlir/Dialect/Utils/ReshapeOpsUtils.h"
#include "mlir/IR/Matchers.h" #include "mlir/IR/Matchers.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
▲ Show 20 Lines • Show All 2,288 Lines • Show Last 20 Lines

mlir/lib/Conversion/TosaToLinalg/TosaToLinalgNamed.cpp

//===- TosaToLinalgNamed.cpp - Lowering Tosa to Linalg Named Ops ----------===// //===- TosaToLinalgNamed.cpp - Lowering Tosa to Linalg Named Ops ----------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// These rewriters lower from the Tosa to the Linalg named ops. // These rewriters lower from the Tosa to the Linalg named ops.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/TosaToLinalg/TosaToLinalg.h" #include "mlir/Conversion/TosaToLinalg/TosaToLinalg.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/Math/IR/Math.h" #include "mlir/Dialect/Math/IR/Math.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/Dialect/Tensor/Utils/Utils.h" #include "mlir/Dialect/Tensor/Utils/Utils.h"
#include "mlir/Dialect/Tosa/IR/TosaOps.h" #include "mlir/Dialect/Tosa/IR/TosaOps.h"
#include "mlir/Dialect/Tosa/Utils/CoversionUtils.h" #include "mlir/Dialect/Tosa/Utils/CoversionUtils.h"
#include "mlir/Dialect/Utils/ReshapeOpsUtils.h" #include "mlir/Dialect/Utils/ReshapeOpsUtils.h"
#include "mlir/IR/Matchers.h" #include "mlir/IR/Matchers.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
▲ Show 20 Lines • Show All 847 Lines • Show Last 20 Lines

mlir/lib/Conversion/TosaToLinalg/TosaToLinalgNamedPass.cpp

//===- TosaToLinalgPass.cpp - Lowering Tosa to Linalg Dialect -------------===// //===- TosaToLinalgPass.cpp - Lowering Tosa to Linalg Dialect -------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This transformation pass legalizes Tosa operations to the Linalg dialect. // This transformation pass legalizes Tosa operations to the Linalg dialect.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "../PassDetail.h" #include "../PassDetail.h"
#include "mlir/Conversion/TosaToLinalg/TosaToLinalg.h" #include "mlir/Conversion/TosaToLinalg/TosaToLinalg.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/Math/IR/Math.h" #include "mlir/Dialect/Math/IR/Math.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/Dialect/Tosa/IR/TosaOps.h" #include "mlir/Dialect/Tosa/IR/TosaOps.h"
#include "mlir/Dialect/Tosa/Transforms/PassDetail.h" #include "mlir/Dialect/Tosa/Transforms/PassDetail.h"
#include "mlir/Dialect/Tosa/Transforms/Passes.h" #include "mlir/Dialect/Tosa/Transforms/Passes.h"
#include "mlir/Dialect/Tosa/Utils/QuantUtils.h" #include "mlir/Dialect/Tosa/Utils/QuantUtils.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Pass/PassManager.h" #include "mlir/Pass/PassManager.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
using namespace mlir; using namespace mlir;
namespace { namespace {
struct TosaToLinalgNamed : public TosaToLinalgNamedBase<TosaToLinalgNamed> { struct TosaToLinalgNamed : public TosaToLinalgNamedBase<TosaToLinalgNamed> {
public: public:
void getDependentDialects(DialectRegistry &registry) const override { void getDependentDialects(DialectRegistry &registry) const override {
registry.insert<arith::ArithmeticDialect, linalg::LinalgDialect, registry.insert<arith::ArithmeticDialect, linalg::LinalgDialect,
math::MathDialect, StandardOpsDialect, math::MathDialect, func::FuncDialect, tensor::TensorDialect,
tensor::TensorDialect, scf::SCFDialect>(); scf::SCFDialect>();
} }
void runOnOperation() override { void runOnOperation() override {
RewritePatternSet patterns(&getContext()); RewritePatternSet patterns(&getContext());
ConversionTarget target(getContext()); ConversionTarget target(getContext());
target.addLegalDialect<linalg::LinalgDialect, StandardOpsDialect, target.addLegalDialect<linalg::LinalgDialect, func::FuncDialect,
tosa::TosaDialect, tensor::TensorDialect, tosa::TosaDialect, tensor::TensorDialect,
scf::SCFDialect>(); scf::SCFDialect>();
// Not every TOSA op can be legalized to linalg. // Not every TOSA op can be legalized to linalg.
target.addIllegalOp<tosa::Conv2DOp>(); target.addIllegalOp<tosa::Conv2DOp>();
target.addIllegalOp<tosa::DepthwiseConv2DOp>(); target.addIllegalOp<tosa::DepthwiseConv2DOp>();
target.addIllegalOp<tosa::MaxPool2dOp>(); target.addIllegalOp<tosa::MaxPool2dOp>();
target.addIllegalOp<tosa::AvgPool2dOp>(); target.addIllegalOp<tosa::AvgPool2dOp>();
Show All 16 Lines

mlir/lib/Conversion/TosaToLinalg/TosaToLinalgPass.cpp

//===- TosaToLinalgPass.cpp - Lowering Tosa to Linalg Dialect -------------===// //===- TosaToLinalgPass.cpp - Lowering Tosa to Linalg Dialect -------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This transformation pass legalizes Tosa operations to the Linalg dialect. // This transformation pass legalizes Tosa operations to the Linalg dialect.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "../PassDetail.h" #include "../PassDetail.h"
#include "mlir/Conversion/TosaToLinalg/TosaToLinalg.h" #include "mlir/Conversion/TosaToLinalg/TosaToLinalg.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/Math/IR/Math.h" #include "mlir/Dialect/Math/IR/Math.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/Dialect/Tosa/IR/TosaOps.h" #include "mlir/Dialect/Tosa/IR/TosaOps.h"
#include "mlir/Dialect/Tosa/Transforms/PassDetail.h" #include "mlir/Dialect/Tosa/Transforms/PassDetail.h"
#include "mlir/Dialect/Tosa/Transforms/Passes.h" #include "mlir/Dialect/Tosa/Transforms/Passes.h"
#include "mlir/Dialect/Tosa/Utils/QuantUtils.h" #include "mlir/Dialect/Tosa/Utils/QuantUtils.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Pass/PassManager.h" #include "mlir/Pass/PassManager.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
using namespace mlir; using namespace mlir;
namespace { namespace {
struct TosaToLinalg : public TosaToLinalgBase<TosaToLinalg> { struct TosaToLinalg : public TosaToLinalgBase<TosaToLinalg> {
public: public:
void getDependentDialects(DialectRegistry &registry) const override { void getDependentDialects(DialectRegistry &registry) const override {
registry.insert<arith::ArithmeticDialect, linalg::LinalgDialect, registry.insert<arith::ArithmeticDialect, linalg::LinalgDialect,
math::MathDialect, StandardOpsDialect, math::MathDialect, func::FuncDialect, tensor::TensorDialect,
tensor::TensorDialect, scf::SCFDialect>(); scf::SCFDialect>();
} }
void runOnOperation() override { void runOnOperation() override {
RewritePatternSet patterns(&getContext()); RewritePatternSet patterns(&getContext());
ConversionTarget target(getContext()); ConversionTarget target(getContext());
target.addLegalDialect<linalg::LinalgDialect, StandardOpsDialect, target.addLegalDialect<linalg::LinalgDialect, func::FuncDialect,
tensor::TensorDialect, scf::SCFDialect>(); tensor::TensorDialect, scf::SCFDialect>();
target.addIllegalDialect<tosa::TosaDialect>(); target.addIllegalDialect<tosa::TosaDialect>();
// Not every TOSA op can be legalized to linalg. // Not every TOSA op can be legalized to linalg.
target.addLegalOp<tosa::ApplyScaleOp>(); target.addLegalOp<tosa::ApplyScaleOp>();
target.addLegalOp<tosa::IfOp>(); target.addLegalOp<tosa::IfOp>();
target.addLegalOp<tosa::ConstOp>(); target.addLegalOp<tosa::ConstOp>();
target.addLegalOp<tosa::WhileOp>(); target.addLegalOp<tosa::WhileOp>();
Show All 27 Lines

mlir/lib/Conversion/TosaToSCF/CMakeLists.txt

add_mlir_conversion_library(MLIRTosaToSCF add_mlir_conversion_library(MLIRTosaToSCF
TosaToSCF.cpp TosaToSCF.cpp
TosaToSCFPass.cpp TosaToSCFPass.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Tosa ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Tosa
${MLIR_MAIN_INCLUDE_DIR}/mlir/IR ${MLIR_MAIN_INCLUDE_DIR}/mlir/IR
DEPENDS DEPENDS
MLIRConversionPassIncGen MLIRConversionPassIncGen
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRIR MLIRIR
MLIRFunc
MLIRSCF MLIRSCF
MLIRStandard
MLIRPass MLIRPass
MLIRTensor MLIRTensor
MLIRTosa MLIRTosa
MLIRTosaTransforms MLIRTosaTransforms
MLIRSupport MLIRSupport
) )

mlir/lib/Conversion/TosaToStandard/CMakeLists.txt

add_mlir_conversion_library(MLIRTosaToStandard add_mlir_conversion_library(MLIRTosaToStandard
TosaToStandard.cpp TosaToStandard.cpp
TosaToStandardPass.cpp TosaToStandardPass.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Tosa ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Tosa
${MLIR_MAIN_INCLUDE_DIR}/mlir/IR ${MLIR_MAIN_INCLUDE_DIR}/mlir/IR
DEPENDS DEPENDS
MLIRConversionPassIncGen MLIRConversionPassIncGen
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRArithmetic MLIRArithmetic
MLIRFunc
MLIRIR MLIRIR
MLIRStandard
MLIRPass MLIRPass
MLIRTosa MLIRTosa
MLIRTosaTransforms MLIRTosaTransforms
MLIRSupport MLIRSupport
) )

mlir/lib/Conversion/TosaToStandard/TosaToStandard.cpp

//===- TosaToStandard.cpp - Lowering Tosa to Standard Dialect -------------===// //===- TosaToStandard.cpp - Lowering Tosa to Standard Dialect -------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// These rewriters lower from the Tosa to the Standard dialect. // These rewriters lower from the Tosa to the Standard dialect.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/TosaToStandard/TosaToStandard.h" #include "mlir/Conversion/TosaToStandard/TosaToStandard.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/Dialect/Tosa/IR/TosaOps.h" #include "mlir/Dialect/Tosa/IR/TosaOps.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
using namespace mlir; using namespace mlir;
using namespace tosa; using namespace tosa;
Show All 40 Lines if (auto shapedTy = type.dyn_cast<ShapedType>()) {
Type eTy = shapedTy.getElementType(); Type eTy = shapedTy.getElementType();
APInt valueInt(eTy.getIntOrFloatBitWidth(), value); APInt valueInt(eTy.getIntOrFloatBitWidth(), value);
return DenseIntElementsAttr::get(shapedTy, valueInt); return DenseIntElementsAttr::get(shapedTy, valueInt);
} }
return rewriter.getIntegerAttr(type, value); return rewriter.getIntegerAttr(type, value);
} }
// This converts the TOSA ApplyScale operator to a set of StandardOps ops, // This converts the TOSA ApplyScale operator to a set of arithmetic ops,
// using 64-bit operations to perform the necessary multiply, bias, and shift. // using 64-bit operations to perform the necessary multiply, bias, and shift.
// Multiple types are used to use minimal bit width operations. // Multiple types are used to use minimal bit width operations.
class ApplyScaleOpConverter : public OpRewritePattern<tosa::ApplyScaleOp> { class ApplyScaleOpConverter : public OpRewritePattern<tosa::ApplyScaleOp> {
public: public:
using OpRewritePattern<tosa::ApplyScaleOp>::OpRewritePattern; using OpRewritePattern<tosa::ApplyScaleOp>::OpRewritePattern;
LogicalResult matchAndRewrite(tosa::ApplyScaleOp op, LogicalResult matchAndRewrite(tosa::ApplyScaleOp op,
PatternRewriter &rewriter) const final { PatternRewriter &rewriter) const final {
▲ Show 20 Lines • Show All 104 Lines • Show Last 20 Lines

mlir/lib/Conversion/TosaToStandard/TosaToStandardPass.cpp

//===- TosaToStandardPass.cpp - Lowering Tosa to Linalg Dialect -----------===// //===- TosaToStandardPass.cpp - Lowering Tosa to Linalg Dialect -----------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This transformation pass legalizes Tosa operations to the Standard dialect. // This transformation pass legalizes Tosa operations to the Standard dialect.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "../PassDetail.h" #include "../PassDetail.h"
#include "mlir/Conversion/TosaToStandard/TosaToStandard.h" #include "mlir/Conversion/TosaToStandard/TosaToStandard.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/Dialect/Tosa/IR/TosaOps.h" #include "mlir/Dialect/Tosa/IR/TosaOps.h"
#include "mlir/Dialect/Tosa/Transforms/PassDetail.h" #include "mlir/Dialect/Tosa/Transforms/PassDetail.h"
#include "mlir/Dialect/Tosa/Transforms/Passes.h" #include "mlir/Dialect/Tosa/Transforms/Passes.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Pass/PassManager.h" #include "mlir/Pass/PassManager.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
using namespace mlir; using namespace mlir;
using namespace tosa; using namespace tosa;
namespace { namespace {
struct TosaToStandard : public TosaToStandardBase<TosaToStandard> { struct TosaToStandard : public TosaToStandardBase<TosaToStandard> {
public: public:
void runOnOperation() override { void runOnOperation() override {
RewritePatternSet patterns(&getContext()); RewritePatternSet patterns(&getContext());
ConversionTarget target(getContext()); ConversionTarget target(getContext());
target.addIllegalOp<tosa::ConstOp>(); target.addIllegalOp<tosa::ConstOp>();
target.addIllegalOp<tosa::SliceOp>(); target.addIllegalOp<tosa::SliceOp>();
target.addIllegalOp<tosa::ApplyScaleOp>(); target.addIllegalOp<tosa::ApplyScaleOp>();
target.addLegalDialect<arith::ArithmeticDialect>(); target.addLegalDialect<arith::ArithmeticDialect>();
target.addLegalDialect<StandardOpsDialect>(); target.addLegalDialect<func::FuncDialect>();
target.addLegalDialect<tensor::TensorDialect>(); target.addLegalDialect<tensor::TensorDialect>();
mlir::tosa::populateTosaToStandardConversionPatterns(&patterns); mlir::tosa::populateTosaToStandardConversionPatterns(&patterns);
if (failed(applyPartialConversion(getOperation(), target, if (failed(applyPartialConversion(getOperation(), target,
std::move(patterns)))) std::move(patterns))))
signalPassFailure(); signalPassFailure();
} }
}; };
Show All 9 Lines

mlir/lib/Conversion/VectorToLLVM/ConvertVectorToLLVM.cpp

//===- VectorToLLVM.cpp - Conversion from Vector to the LLVM dialect ------===// //===- VectorToLLVM.cpp - Conversion from Vector to the LLVM dialect ------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/VectorToLLVM/ConvertVectorToLLVM.h" #include "mlir/Conversion/VectorToLLVM/ConvertVectorToLLVM.h"
#include "mlir/Conversion/LLVMCommon/VectorPattern.h" #include "mlir/Conversion/LLVMCommon/VectorPattern.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/LLVMIR/FunctionCallUtils.h" #include "mlir/Dialect/LLVMIR/FunctionCallUtils.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Vector/Transforms/VectorTransforms.h" #include "mlir/Dialect/Vector/Transforms/VectorTransforms.h"
#include "mlir/IR/BuiltinTypes.h" #include "mlir/IR/BuiltinTypes.h"
#include "mlir/Support/MathExtras.h" #include "mlir/Support/MathExtras.h"
#include "mlir/Target/LLVMIR/TypeToLLVM.h" #include "mlir/Target/LLVMIR/TypeToLLVM.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
using namespace mlir; using namespace mlir;
using namespace mlir::vector; using namespace mlir::vector;
▲ Show 20 Lines • Show All 1,167 Lines • Show Last 20 Lines

mlir/lib/Conversion/VectorToLLVM/ConvertVectorToLLVMPass.cpp

Show All 12 Lines
#include "mlir/Conversion/LLVMCommon/ConversionTarget.h" #include "mlir/Conversion/LLVMCommon/ConversionTarget.h"
#include "mlir/Conversion/LLVMCommon/TypeConverter.h" #include "mlir/Conversion/LLVMCommon/TypeConverter.h"
#include "mlir/Dialect/AMX/AMXDialect.h" #include "mlir/Dialect/AMX/AMXDialect.h"
#include "mlir/Dialect/AMX/Transforms.h" #include "mlir/Dialect/AMX/Transforms.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/ArmNeon/ArmNeonDialect.h" #include "mlir/Dialect/ArmNeon/ArmNeonDialect.h"
#include "mlir/Dialect/ArmSVE/ArmSVEDialect.h" #include "mlir/Dialect/ArmSVE/ArmSVEDialect.h"
#include "mlir/Dialect/ArmSVE/Transforms.h" #include "mlir/Dialect/ArmSVE/Transforms.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Vector/Transforms/VectorRewritePatterns.h" #include "mlir/Dialect/Vector/Transforms/VectorRewritePatterns.h"
#include "mlir/Dialect/X86Vector/Transforms.h" #include "mlir/Dialect/X86Vector/Transforms.h"
#include "mlir/Dialect/X86Vector/X86VectorDialect.h" #include "mlir/Dialect/X86Vector/X86VectorDialect.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
using namespace mlir; using namespace mlir;
using namespace mlir::vector; using namespace mlir::vector;
▲ Show 20 Lines • Show All 51 Lines • ▼ Show 20 Linesvoid LowerVectorToLLVMPass::runOnOperation() {
populateVectorToLLVMConversionPatterns(converter, patterns, populateVectorToLLVMConversionPatterns(converter, patterns,
reassociateFPReductions); reassociateFPReductions);
populateVectorToLLVMMatrixConversionPatterns(converter, patterns); populateVectorToLLVMMatrixConversionPatterns(converter, patterns);
// Architecture specific augmentations. // Architecture specific augmentations.
LLVMConversionTarget target(getContext()); LLVMConversionTarget target(getContext());
target.addLegalDialect<arith::ArithmeticDialect>(); target.addLegalDialect<arith::ArithmeticDialect>();
target.addLegalDialect<memref::MemRefDialect>(); target.addLegalDialect<memref::MemRefDialect>();
target.addLegalDialect<StandardOpsDialect>(); target.addLegalDialect<func::FuncDialect>();
target.addLegalOp<UnrealizedConversionCastOp>(); target.addLegalOp<UnrealizedConversionCastOp>();
if (armNeon) { if (armNeon) {
// TODO: we may or may not want to include in-dialect lowering to // TODO: we may or may not want to include in-dialect lowering to
// LLVM-compatible operations here. So far, all operations in the dialect // LLVM-compatible operations here. So far, all operations in the dialect
// can be translated to LLVM IR so there is no conversion necessary. // can be translated to LLVM IR so there is no conversion necessary.
target.addLegalDialect<arm_neon::ArmNeonDialect>(); target.addLegalDialect<arm_neon::ArmNeonDialect>();
} }
if (armSVE) { if (armSVE) {
Show All 21 Lines

mlir/lib/Conversion/VectorToROCDL/VectorToROCDL.cpp

Show All 12 Lines
#include "mlir/Conversion/VectorToROCDL/VectorToROCDL.h" #include "mlir/Conversion/VectorToROCDL/VectorToROCDL.h"
#include "../PassDetail.h" #include "../PassDetail.h"
#include "mlir/Conversion/LLVMCommon/ConversionTarget.h" #include "mlir/Conversion/LLVMCommon/ConversionTarget.h"
#include "mlir/Conversion/LLVMCommon/Pattern.h" #include "mlir/Conversion/LLVMCommon/Pattern.h"
#include "mlir/Conversion/MemRefToLLVM/MemRefToLLVM.h" #include "mlir/Conversion/MemRefToLLVM/MemRefToLLVM.h"
#include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h" #include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/GPU/GPUDialect.h" #include "mlir/Dialect/GPU/GPUDialect.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/LLVMIR/ROCDLDialect.h" #include "mlir/Dialect/LLVMIR/ROCDLDialect.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Vector/IR/VectorOps.h" #include "mlir/Dialect/Vector/IR/VectorOps.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
using namespace mlir; using namespace mlir;
using namespace mlir::vector; using namespace mlir::vector;
static LogicalResult replaceTransferOpWithMubuf( static LogicalResult replaceTransferOpWithMubuf(
▲ Show 20 Lines • Show All 150 Lines • Show Last 20 Lines

mlir/lib/Dialect/AMX/Transforms/LegalizeForLLVMExport.cpp

//===- LegalizeForLLVMExport.cpp - Prepare AMX for LLVM translation ----===// //===- LegalizeForLLVMExport.cpp - Prepare AMX for LLVM translation ----===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/AMX/Transforms.h" #include "mlir/Dialect/AMX/Transforms.h"
#include "mlir/Conversion/LLVMCommon/ConversionTarget.h" #include "mlir/Conversion/LLVMCommon/ConversionTarget.h"
#include "mlir/Conversion/LLVMCommon/Pattern.h" #include "mlir/Conversion/LLVMCommon/Pattern.h"
#include "mlir/Dialect/AMX/AMXDialect.h" #include "mlir/Dialect/AMX/AMXDialect.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
using namespace mlir; using namespace mlir;
using namespace mlir::amx; using namespace mlir::amx;
namespace { namespace {
▲ Show 20 Lines • Show All 204 Lines • Show Last 20 Lines

mlir/lib/Dialect/Affine/Analysis/AffineAnalysis.cpp

Show All 12 Lines
#include "mlir/Dialect/Affine/Analysis/AffineAnalysis.h" #include "mlir/Dialect/Affine/Analysis/AffineAnalysis.h"
#include "mlir/Analysis/SliceAnalysis.h" #include "mlir/Analysis/SliceAnalysis.h"
#include "mlir/Dialect/Affine/Analysis/LoopAnalysis.h" #include "mlir/Dialect/Affine/Analysis/LoopAnalysis.h"
#include "mlir/Dialect/Affine/Analysis/Utils.h" #include "mlir/Dialect/Affine/Analysis/Utils.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Affine/IR/AffineValueMap.h" #include "mlir/Dialect/Affine/IR/AffineValueMap.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/AffineExprVisitor.h" #include "mlir/IR/AffineExprVisitor.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/IntegerSet.h" #include "mlir/IR/IntegerSet.h"
#include "mlir/Support/MathExtras.h" #include "mlir/Support/MathExtras.h"
#include "llvm/ADT/DenseMap.h" #include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/TypeSwitch.h" #include "llvm/ADT/TypeSwitch.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#include "llvm/Support/raw_ostream.h" #include "llvm/Support/raw_ostream.h"
▲ Show 20 Lines • Show All 639 Lines • Show Last 20 Lines

mlir/lib/Dialect/Affine/Analysis/AffineStructures.cpp

Show All 11 Lines
#include "mlir/Dialect/Affine/Analysis/AffineStructures.h" #include "mlir/Dialect/Affine/Analysis/AffineStructures.h"
#include "mlir/Analysis/Presburger/LinearTransform.h" #include "mlir/Analysis/Presburger/LinearTransform.h"
#include "mlir/Analysis/Presburger/Simplex.h" #include "mlir/Analysis/Presburger/Simplex.h"
#include "mlir/Analysis/Presburger/Utils.h" #include "mlir/Analysis/Presburger/Utils.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Affine/IR/AffineValueMap.h" #include "mlir/Dialect/Affine/IR/AffineValueMap.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/AffineExprVisitor.h" #include "mlir/IR/AffineExprVisitor.h"
#include "mlir/IR/IntegerSet.h" #include "mlir/IR/IntegerSet.h"
#include "mlir/Support/LLVM.h" #include "mlir/Support/LLVM.h"
#include "mlir/Support/MathExtras.h" #include "mlir/Support/MathExtras.h"
#include "llvm/ADT/STLExtras.h" #include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/SmallPtrSet.h" #include "llvm/ADT/SmallPtrSet.h"
#include "llvm/ADT/SmallVector.h" #include "llvm/ADT/SmallVector.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
▲ Show 20 Lines • Show All 1,750 Lines • Show Last 20 Lines

mlir/lib/Dialect/Affine/Analysis/NestedMatcher.cpp

//===- NestedMatcher.cpp - NestedMatcher Impl ----------------------------===// //===- NestedMatcher.cpp - NestedMatcher Impl ----------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include <utility> #include <utility>
#include "mlir/Dialect/Affine/Analysis/NestedMatcher.h" #include "mlir/Dialect/Affine/Analysis/NestedMatcher.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "llvm/ADT/ArrayRef.h" #include "llvm/ADT/ArrayRef.h"
#include "llvm/ADT/STLExtras.h" #include "llvm/ADT/STLExtras.h"
#include "llvm/Support/Allocator.h" #include "llvm/Support/Allocator.h"
#include "llvm/Support/raw_ostream.h" #include "llvm/Support/raw_ostream.h"
using namespace mlir; using namespace mlir;
▲ Show 20 Lines • Show All 159 Lines • Show Last 20 Lines

mlir/lib/Dialect/Affine/Analysis/Utils.cpp

Show All 12 Lines
#include "mlir/Dialect/Affine/Analysis/Utils.h" #include "mlir/Dialect/Affine/Analysis/Utils.h"
#include "mlir/Analysis/Presburger/PresburgerSet.h" #include "mlir/Analysis/Presburger/PresburgerSet.h"
#include "mlir/Dialect/Affine/Analysis/AffineAnalysis.h" #include "mlir/Dialect/Affine/Analysis/AffineAnalysis.h"
#include "mlir/Dialect/Affine/Analysis/LoopAnalysis.h" #include "mlir/Dialect/Affine/Analysis/LoopAnalysis.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Affine/IR/AffineValueMap.h" #include "mlir/Dialect/Affine/IR/AffineValueMap.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/IntegerSet.h" #include "mlir/IR/IntegerSet.h"
#include "llvm/ADT/SmallPtrSet.h" #include "llvm/ADT/SmallPtrSet.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#include "llvm/Support/raw_ostream.h" #include "llvm/Support/raw_ostream.h"
#define DEBUG_TYPE "analysis-utils" #define DEBUG_TYPE "analysis-utils"
using namespace mlir; using namespace mlir;
▲ Show 20 Lines • Show All 1,348 Lines • Show Last 20 Lines

mlir/lib/Dialect/Affine/Transforms/AffineDataCopyGeneration.cpp

Show All 19 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Affine/Analysis/Utils.h" #include "mlir/Dialect/Affine/Analysis/Utils.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Affine/LoopUtils.h" #include "mlir/Dialect/Affine/LoopUtils.h"
#include "mlir/Dialect/Affine/Passes.h" #include "mlir/Dialect/Affine/Passes.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
#include "llvm/ADT/MapVector.h" #include "llvm/ADT/MapVector.h"
#include "llvm/Support/CommandLine.h" #include "llvm/Support/CommandLine.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#include <algorithm> #include <algorithm>
#define DEBUG_TYPE "affine-data-copy-generate" #define DEBUG_TYPE "affine-data-copy-generate"
▲ Show 20 Lines • Show All 198 Lines • Show Last 20 Lines

mlir/lib/Dialect/Affine/Transforms/AffineLoopInvariantCodeMotion.cpp

Show First 20 Lines • Show All 76 Lines • ▼ Show 20 Linesbool isOpLoopInvariant(Operation &op, Value indVar, ValueRange iterArgs,
} else if (auto forOp = dyn_cast<AffineForOp>(op)) { } else if (auto forOp = dyn_cast<AffineForOp>(op)) {
if (!areAllOpsInTheBlockListInvariant(forOp.getLoopBody(), indVar, iterArgs, if (!areAllOpsInTheBlockListInvariant(forOp.getLoopBody(), indVar, iterArgs,
opsWithUsers, opsToHoist)) { opsWithUsers, opsToHoist)) {
return false; return false;
} }
} else if (isa<AffineDmaStartOp, AffineDmaWaitOp>(op)) { } else if (isa<AffineDmaStartOp, AffineDmaWaitOp>(op)) {
// TODO: Support DMA ops. // TODO: Support DMA ops.
return false; return false;
} else if (!isa<arith::ConstantOp, ConstantOp>(op)) { } else if (!isa<arith::ConstantOp, func::ConstantOp>(op)) {
// Register op in the set of ops that have users. // Register op in the set of ops that have users.
opsWithUsers.insert(&op); opsWithUsers.insert(&op);
if (isa<AffineMapAccessInterface>(op)) { if (isa<AffineMapAccessInterface>(op)) {
Value memref = isa<AffineReadOpInterface>(op) Value memref = isa<AffineReadOpInterface>(op)
? cast<AffineReadOpInterface>(op).getMemRef() ? cast<AffineReadOpInterface>(op).getMemRef()
: cast<AffineWriteOpInterface>(op).getMemRef(); : cast<AffineWriteOpInterface>(op).getMemRef();
for (auto *user : memref.getUsers()) { for (auto *user : memref.getUsers()) {
// If this memref has a user that is a DMA, give up because these // If this memref has a user that is a DMA, give up because these
▲ Show 20 Lines • Show All 150 Lines • Show Last 20 Lines

mlir/lib/Dialect/Affine/Transforms/CMakeLists.txt

Show All 20 Linesadd_mlir_dialect_library(MLIRAffineTransforms
MLIRAffinePassIncGen MLIRAffinePassIncGen
MLIRLoopLikeInterfaceIncGen MLIRLoopLikeInterfaceIncGen
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRAffine MLIRAffine
MLIRAffineAnalysis MLIRAffineAnalysis
MLIRAffineUtils MLIRAffineUtils
MLIRArithmetic MLIRArithmetic
MLIRFunc
MLIRIR MLIRIR
MLIRMemRef MLIRMemRef
MLIRPass MLIRPass
MLIRSCFUtils MLIRSCFUtils
MLIRSideEffectInterfaces MLIRSideEffectInterfaces
MLIRStandard
MLIRTransformUtils MLIRTransformUtils
MLIRVector MLIRVector
MLIRVectorUtils MLIRVectorUtils
MLIRVectorToLLVM MLIRVectorToLLVM
) )

mlir/lib/Dialect/Arithmetic/Transforms/ExpandOps.cpp

//===- ExpandOps.cpp - Pass to legalize Arithmetic ops for LLVM lowering --===// //===- ExpandOps.cpp - Pass to legalize Arithmetic ops for LLVM lowering --===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Arithmetic/Transforms/Passes.h" #include "mlir/Dialect/Arithmetic/Transforms/Passes.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/TypeUtilities.h" #include "mlir/IR/TypeUtilities.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
using namespace mlir; using namespace mlir;
/// Create an integer or index constant. /// Create an integer or index constant.
static Value createConst(Location loc, Type type, int value, static Value createConst(Location loc, Type type, int value,
PatternRewriter &rewriter) { PatternRewriter &rewriter) {
▲ Show 20 Lines • Show All 171 Lines • ▼ Show 20 Lines
struct ArithmeticExpandOpsPass struct ArithmeticExpandOpsPass
: public ArithmeticExpandOpsBase<ArithmeticExpandOpsPass> { : public ArithmeticExpandOpsBase<ArithmeticExpandOpsPass> {
void runOnOperation() override { void runOnOperation() override {
RewritePatternSet patterns(&getContext()); RewritePatternSet patterns(&getContext());
ConversionTarget target(getContext()); ConversionTarget target(getContext());
arith::populateArithmeticExpandOpsPatterns(patterns); arith::populateArithmeticExpandOpsPatterns(patterns);
target.addLegalDialect<arith::ArithmeticDialect, StandardOpsDialect>(); target.addLegalDialect<arith::ArithmeticDialect, func::FuncDialect>();
// clang-format off // clang-format off
target.addIllegalOp< target.addIllegalOp<
arith::CeilDivSIOp, arith::CeilDivSIOp,
arith::CeilDivUIOp, arith::CeilDivUIOp,
arith::FloorDivSIOp, arith::FloorDivSIOp,
arith::MaxFOp, arith::MaxFOp,
arith::MaxSIOp, arith::MaxSIOp,
arith::MaxUIOp, arith::MaxUIOp,
Show All 33 Lines

mlir/lib/Dialect/Arithmetic/Transforms/PassDetail.h

//===- PassDetail.h - Arithmetic Pass details -------------------*- C++ -*-===// //===- PassDetail.h - Arithmetic Pass details -------------------*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef DIALECT_ARITHMETIC_TRANSFORMS_PASSDETAIL_H_ #ifndef DIALECT_ARITHMETIC_TRANSFORMS_PASSDETAIL_H_
#define DIALECT_ARITHMETIC_TRANSFORMS_PASSDETAIL_H_ #define DIALECT_ARITHMETIC_TRANSFORMS_PASSDETAIL_H_
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
namespace mlir { namespace mlir {
class StandardOpsDialect;
namespace bufferization { namespace bufferization {
class BufferizationDialect; class BufferizationDialect;
} // namespace bufferization } // namespace bufferization
namespace func {
class FuncDialect;
} // namespace func
namespace memref { namespace memref {
class MemRefDialect; class MemRefDialect;
} // namespace memref } // namespace memref
#define GEN_PASS_CLASSES #define GEN_PASS_CLASSES
#include "mlir/Dialect/Arithmetic/Transforms/Passes.h.inc" #include "mlir/Dialect/Arithmetic/Transforms/Passes.h.inc"
} // namespace mlir } // namespace mlir
#endif // DIALECT_ARITHMETIC_TRANSFORMS_PASSDETAIL_H_ #endif // DIALECT_ARITHMETIC_TRANSFORMS_PASSDETAIL_H_

mlir/lib/Dialect/ArmSVE/Transforms/CMakeLists.txt

add_mlir_dialect_library(MLIRArmSVETransforms add_mlir_dialect_library(MLIRArmSVETransforms
LegalizeForLLVMExport.cpp LegalizeForLLVMExport.cpp
DEPENDS DEPENDS
MLIRArmSVEConversionsIncGen MLIRArmSVEConversionsIncGen
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRArmSVE MLIRArmSVE
MLIRFunc
MLIRIR MLIRIR
MLIRLLVMCommonConversion MLIRLLVMCommonConversion
MLIRLLVMIR MLIRLLVMIR
MLIRStandard
) )

mlir/lib/Dialect/ArmSVE/Transforms/LegalizeForLLVMExport.cpp

Show First 20 Lines • Show All 65 Lines • ▼ Show 20 Lines OneToOneConvertToLLVMPattern<ScalableMaskedDivFOp,
ScalableMaskedDivFIntrOp>; ScalableMaskedDivFIntrOp>;
/// Populate the given list with patterns that convert from ArmSVE to LLVM. /// Populate the given list with patterns that convert from ArmSVE to LLVM.
void mlir::populateArmSVELegalizeForLLVMExportPatterns( void mlir::populateArmSVELegalizeForLLVMExportPatterns(
LLVMTypeConverter &converter, RewritePatternSet &patterns) { LLVMTypeConverter &converter, RewritePatternSet &patterns) {
// Populate conversion patterns // Populate conversion patterns
// clang-format off // clang-format off
patterns.add<ForwardOperands<CallOp>, patterns.add<ForwardOperands<func::CallOp>,
ForwardOperands<CallIndirectOp>, ForwardOperands<func::CallIndirectOp>,
ForwardOperands<ReturnOp>>(converter, ForwardOperands<func::ReturnOp>>(converter,
&converter.getContext()); &converter.getContext());
patterns.add<SdotOpLowering, patterns.add<SdotOpLowering,
SmmlaOpLowering, SmmlaOpLowering,
UdotOpLowering, UdotOpLowering,
UmmlaOpLowering, UmmlaOpLowering,
ScalableMaskedAddIOpLowering, ScalableMaskedAddIOpLowering,
ScalableMaskedAddFOpLowering, ScalableMaskedAddFOpLowering,
ScalableMaskedSubIOpLowering, ScalableMaskedSubIOpLowering,
Show All 40 Lines

mlir/lib/Dialect/Async/Transforms/AsyncParallelFor.cpp

Show All 11 Lines
#include <utility> #include <utility>
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Async/IR/Async.h" #include "mlir/Dialect/Async/IR/Async.h"
#include "mlir/Dialect/Async/Passes.h" #include "mlir/Dialect/Async/Passes.h"
#include "mlir/Dialect/Async/Transforms.h" #include "mlir/Dialect/Async/Transforms.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/ImplicitLocOpBuilder.h" #include "mlir/IR/ImplicitLocOpBuilder.h"
#include "mlir/IR/Matchers.h" #include "mlir/IR/Matchers.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Support/LLVM.h" #include "mlir/Support/LLVM.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
#include "mlir/Transforms/RegionUtils.h" #include "mlir/Transforms/RegionUtils.h"
▲ Show 20 Lines • Show All 396 Lines • ▼ Show 20 Lines return [&, loopIdx](OpBuilder &nestedBuilder, Location loc, Value iv,
for (auto &bodyOp : op.getLoopBody().getOps()) for (auto &bodyOp : op.getLoopBody().getOps())
b.clone(bodyOp, mapping); b.clone(bodyOp, mapping);
}; };
}; };
b.create<scf::ForOp>(blockFirstCoord[0], blockEndCoord[0], c1, ValueRange(), b.create<scf::ForOp>(blockFirstCoord[0], blockEndCoord[0], c1, ValueRange(),
workLoopBuilder(0)); workLoopBuilder(0));
b.create<ReturnOp>(ValueRange()); b.create<func::ReturnOp>(ValueRange());
return {op.getNumLoops(), func, std::move(computeFuncType.captures)}; return {op.getNumLoops(), func, std::move(computeFuncType.captures)};
} }
// Creates recursive async dispatch function for the given parallel compute // Creates recursive async dispatch function for the given parallel compute
// function. Dispatch function keeps splitting block range into halves until it // function. Dispatch function keeps splitting block range into halves until it
// reaches a single block, and then excecutes it inline. // reaches a single block, and then excecutes it inline.
// //
▲ Show 20 Lines • Show All 93 Lines • ▼ Show 20 Lines // for the seconds half of the original range and go to the next iteration.
auto executeBodyBuilder = [&](OpBuilder &executeBuilder, auto executeBodyBuilder = [&](OpBuilder &executeBuilder,
Location executeLoc, ValueRange executeArgs) { Location executeLoc, ValueRange executeArgs) {
// Update the original `blockStart` and `blockEnd` with new range. // Update the original `blockStart` and `blockEnd` with new range.
SmallVector<Value> operands{block->getArguments().begin(), SmallVector<Value> operands{block->getArguments().begin(),
block->getArguments().end()}; block->getArguments().end()};
operands[1] = midIndex; operands[1] = midIndex;
operands[2] = end; operands[2] = end;
executeBuilder.create<CallOp>(executeLoc, func.sym_name(), executeBuilder.create<func::CallOp>(executeLoc, func.sym_name(),
func.getCallableResults(), operands); func.getCallableResults(), operands);
executeBuilder.create<async::YieldOp>(executeLoc, ValueRange()); executeBuilder.create<async::YieldOp>(executeLoc, ValueRange());
}; };
// Create async.execute operation to dispatch half of the block range. // Create async.execute operation to dispatch half of the block range.
auto execute = b.create<ExecuteOp>(TypeRange(), ValueRange(), ValueRange(), auto execute = b.create<ExecuteOp>(TypeRange(), ValueRange(), ValueRange(),
executeBodyBuilder); executeBodyBuilder);
b.create<AddToGroupOp>(indexTy, execute.token(), group); b.create<AddToGroupOp>(indexTy, execute.token(), group);
b.create<scf::YieldOp>(ValueRange({start, midIndex})); b.create<scf::YieldOp>(ValueRange({start, midIndex}));
} }
// After dispatching async operations to process the tail of the block range // After dispatching async operations to process the tail of the block range
// call the parallel compute function for the first block of the range. // call the parallel compute function for the first block of the range.
b.setInsertionPointAfter(whileOp); b.setInsertionPointAfter(whileOp);
// Drop async dispatch specific arguments: async group, block start and end. // Drop async dispatch specific arguments: async group, block start and end.
auto forwardedInputs = block->getArguments().drop_front(3); auto forwardedInputs = block->getArguments().drop_front(3);
SmallVector<Value> computeFuncOperands = {blockStart}; SmallVector<Value> computeFuncOperands = {blockStart};
computeFuncOperands.append(forwardedInputs.begin(), forwardedInputs.end()); computeFuncOperands.append(forwardedInputs.begin(), forwardedInputs.end());
b.create<CallOp>(computeFunc.func.sym_name(), b.create<func::CallOp>(computeFunc.func.sym_name(),
computeFunc.func.getCallableResults(), computeFuncOperands); computeFunc.func.getCallableResults(),
b.create<ReturnOp>(ValueRange()); computeFuncOperands);
b.create<func::ReturnOp>(ValueRange());
return func; return func;
} }
// Launch async dispatch of the parallel compute function. // Launch async dispatch of the parallel compute function.
static void doAsyncDispatch(ImplicitLocOpBuilder &b, PatternRewriter &rewriter, static void doAsyncDispatch(ImplicitLocOpBuilder &b, PatternRewriter &rewriter,
ParallelComputeFunction &parallelComputeFunction, ParallelComputeFunction &parallelComputeFunction,
scf::ParallelOp op, Value blockSize, scf::ParallelOp op, Value blockSize,
Show All 27 Linesstatic void doAsyncDispatch(ImplicitLocOpBuilder &b, PatternRewriter &rewriter,
auto syncDispatch = [&](OpBuilder &nestedBuilder, Location loc) { auto syncDispatch = [&](OpBuilder &nestedBuilder, Location loc) {
ImplicitLocOpBuilder b(loc, nestedBuilder); ImplicitLocOpBuilder b(loc, nestedBuilder);
// Call parallel compute function for the single block. // Call parallel compute function for the single block.
SmallVector<Value> operands = {c0, blockSize}; SmallVector<Value> operands = {c0, blockSize};
appendBlockComputeOperands(operands); appendBlockComputeOperands(operands);
b.create<CallOp>(parallelComputeFunction.func.sym_name(), b.create<func::CallOp>(parallelComputeFunction.func.sym_name(),
parallelComputeFunction.func.getCallableResults(), parallelComputeFunction.func.getCallableResults(),
operands); operands);
b.create<scf::YieldOp>(); b.create<scf::YieldOp>();
}; };
auto asyncDispatch = [&](OpBuilder &nestedBuilder, Location loc) { auto asyncDispatch = [&](OpBuilder &nestedBuilder, Location loc) {
ImplicitLocOpBuilder b(loc, nestedBuilder); ImplicitLocOpBuilder b(loc, nestedBuilder);
// Create an async.group to wait on all async tokens from the concurrent // Create an async.group to wait on all async tokens from the concurrent
// execution of multiple parallel compute function. First block will be // execution of multiple parallel compute function. First block will be
// executed synchronously in the caller thread. // executed synchronously in the caller thread.
Value groupSize = b.create<arith::SubIOp>(blockCount, c1); Value groupSize = b.create<arith::SubIOp>(blockCount, c1);
Value group = b.create<CreateGroupOp>(GroupType::get(ctx), groupSize); Value group = b.create<CreateGroupOp>(GroupType::get(ctx), groupSize);
// Launch async dispatch function for [0, blockCount) range. // Launch async dispatch function for [0, blockCount) range.
SmallVector<Value> operands = {group, c0, blockCount, blockSize}; SmallVector<Value> operands = {group, c0, blockCount, blockSize};
appendBlockComputeOperands(operands); appendBlockComputeOperands(operands);
b.create<CallOp>(asyncDispatchFunction.sym_name(), b.create<func::CallOp>(asyncDispatchFunction.sym_name(),
asyncDispatchFunction.getCallableResults(), operands); asyncDispatchFunction.getCallableResults(),
operands);
// Wait for the completion of all parallel compute operations. // Wait for the completion of all parallel compute operations.
b.create<AwaitAllOp>(group); b.create<AwaitAllOp>(group);
b.create<scf::YieldOp>(); b.create<scf::YieldOp>();
}; };
// Dispatch either single block compute function, or launch async dispatch. // Dispatch either single block compute function, or launch async dispatch.
Show All 40 LinesdoSequentialDispatch(ImplicitLocOpBuilder &b, PatternRewriter &rewriter,
// Induction variable is the index of the block: [0, blockCount). // Induction variable is the index of the block: [0, blockCount).
LoopBodyBuilder loopBuilder = [&](OpBuilder &loopBuilder, Location loc, LoopBodyBuilder loopBuilder = [&](OpBuilder &loopBuilder, Location loc,
Value iv, ValueRange args) { Value iv, ValueRange args) {
ImplicitLocOpBuilder b(loc, loopBuilder); ImplicitLocOpBuilder b(loc, loopBuilder);
// Call parallel compute function inside the async.execute region. // Call parallel compute function inside the async.execute region.
auto executeBodyBuilder = [&](OpBuilder &executeBuilder, auto executeBodyBuilder = [&](OpBuilder &executeBuilder,
Location executeLoc, ValueRange executeArgs) { Location executeLoc, ValueRange executeArgs) {
executeBuilder.create<CallOp>(executeLoc, compute.sym_name(), executeBuilder.create<func::CallOp>(executeLoc, compute.sym_name(),
compute.getCallableResults(), compute.getCallableResults(),
computeFuncOperands(iv)); computeFuncOperands(iv));
executeBuilder.create<async::YieldOp>(executeLoc, ValueRange()); executeBuilder.create<async::YieldOp>(executeLoc, ValueRange());
}; };
// Create async.execute operation to launch parallel computate function. // Create async.execute operation to launch parallel computate function.
auto execute = b.create<ExecuteOp>(TypeRange(), ValueRange(), ValueRange(), auto execute = b.create<ExecuteOp>(TypeRange(), ValueRange(), ValueRange(),
executeBodyBuilder); executeBodyBuilder);
b.create<AddToGroupOp>(rewriter.getIndexType(), execute.token(), group); b.create<AddToGroupOp>(rewriter.getIndexType(), execute.token(), group);
b.create<scf::YieldOp>(); b.create<scf::YieldOp>();
}; };
// Iterate over all compute blocks and launch parallel compute operations. // Iterate over all compute blocks and launch parallel compute operations.
b.create<scf::ForOp>(c1, blockCount, c1, ValueRange(), loopBuilder); b.create<scf::ForOp>(c1, blockCount, c1, ValueRange(), loopBuilder);
// Call parallel compute function for the first block in the caller thread. // Call parallel compute function for the first block in the caller thread.
b.create<CallOp>(compute.sym_name(), compute.getCallableResults(), b.create<func::CallOp>(compute.sym_name(), compute.getCallableResults(),
computeFuncOperands(c0)); computeFuncOperands(c0));
// Wait for the completion of all async compute operations. // Wait for the completion of all async compute operations.
b.create<AwaitAllOp>(group); b.create<AwaitAllOp>(group);
} }
LogicalResult LogicalResult
AsyncParallelForRewrite::matchAndRewrite(scf::ParallelOp op, AsyncParallelForRewrite::matchAndRewrite(scf::ParallelOp op,
PatternRewriter &rewriter) const { PatternRewriter &rewriter) const {
▲ Show 20 Lines • Show All 238 Lines • Show Last 20 Lines

mlir/lib/Dialect/Async/Transforms/AsyncRuntimeRefCounting.cpp

Show All 10 Lines
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Analysis/Liveness.h" #include "mlir/Analysis/Liveness.h"
#include "mlir/Dialect/Async/IR/Async.h" #include "mlir/Dialect/Async/IR/Async.h"
#include "mlir/Dialect/Async/Passes.h" #include "mlir/Dialect/Async/Passes.h"
#include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h" #include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/ImplicitLocOpBuilder.h" #include "mlir/IR/ImplicitLocOpBuilder.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
#include "llvm/ADT/SmallSet.h" #include "llvm/ADT/SmallSet.h"
using namespace mlir; using namespace mlir;
using namespace mlir::async; using namespace mlir::async;
▲ Show 20 Lines • Show All 284 Lines • ▼ Show 20 Lines
} }
LogicalResult LogicalResult
AsyncRuntimeRefCountingPass::addAddRefBeforeFunctionCall(Value value) { AsyncRuntimeRefCountingPass::addAddRefBeforeFunctionCall(Value value) {
OpBuilder builder(value.getContext()); OpBuilder builder(value.getContext());
Location loc = value.getLoc(); Location loc = value.getLoc();
for (Operation *user : value.getUsers()) { for (Operation *user : value.getUsers()) {
if (!isa<CallOp>(user)) if (!isa<func::CallOp>(user))
continue; continue;
// Add a reference before the function call to pass the value at `+1` // Add a reference before the function call to pass the value at `+1`
// reference to the function entry block. // reference to the function entry block.
builder.setInsertionPoint(user); builder.setInsertionPoint(user);
builder.create<RuntimeAddRefOp>(loc, value, builder.getI64IntegerAttr(1)); builder.create<RuntimeAddRefOp>(loc, value, builder.getI64IntegerAttr(1));
} }
▲ Show 20 Lines • Show All 231 Lines • Show Last 20 Lines

mlir/lib/Dialect/Async/Transforms/AsyncRuntimeRefCountingOpt.cpp

//===- AsyncRuntimeRefCountingOpt.cpp - Async Ref Counting --------------===// //===- AsyncRuntimeRefCountingOpt.cpp - Async Ref Counting --------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// Optimize Async dialect reference counting operations. // Optimize Async dialect reference counting operations.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Async/IR/Async.h" #include "mlir/Dialect/Async/IR/Async.h"
#include "mlir/Dialect/Async/Passes.h" #include "mlir/Dialect/Async/Passes.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "llvm/ADT/SmallSet.h" #include "llvm/ADT/SmallSet.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
using namespace mlir; using namespace mlir;
using namespace mlir::async; using namespace mlir::async;
#define DEBUG_TYPE "async-ref-counting" #define DEBUG_TYPE "async-ref-counting"
▲ Show 20 Lines • Show All 109 Lines • ▼ Show 20 Lines for (RuntimeAddRefOp addRef : info.addRefs) {
// `user` operation lies after `addRef` ... // `user` operation lies after `addRef` ...
if (user == addRef || user->isBeforeInBlock(addRef)) if (user == addRef || user->isBeforeInBlock(addRef))
continue; continue;
// ... and before `dropRef`. // ... and before `dropRef`.
if (user == dropRef || dropRef->isBeforeInBlock(user)) if (user == dropRef || dropRef->isBeforeInBlock(user))
break; break;
// Find the first function call user of the reference counted value. // Find the first function call user of the reference counted value.
Operation *functionCall = dyn_cast<CallOp>(user); Operation *functionCall = dyn_cast<func::CallOp>(user);
if (functionCall && if (functionCall &&
(!firstFunctionCallUser || (!firstFunctionCallUser ||
functionCall->isBeforeInBlock(firstFunctionCallUser))) { functionCall->isBeforeInBlock(firstFunctionCallUser))) {
firstFunctionCallUser = functionCall; firstFunctionCallUser = functionCall;
continue; continue;
} }
// Find the last regular user of the reference counted value. // Find the last regular user of the reference counted value.
▲ Show 20 Lines • Show All 80 Lines • Show Last 20 Lines

mlir/lib/Dialect/Async/Transforms/AsyncToAsyncRuntime.cpp

Show All 11 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Conversion/SCFToControlFlow/SCFToControlFlow.h" #include "mlir/Conversion/SCFToControlFlow/SCFToControlFlow.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Async/IR/Async.h" #include "mlir/Dialect/Async/IR/Async.h"
#include "mlir/Dialect/Async/Passes.h" #include "mlir/Dialect/Async/Passes.h"
#include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h" #include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/ImplicitLocOpBuilder.h" #include "mlir/IR/ImplicitLocOpBuilder.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "mlir/Transforms/RegionUtils.h" #include "mlir/Transforms/RegionUtils.h"
#include "llvm/ADT/SetVector.h" #include "llvm/ADT/SetVector.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
▲ Show 20 Lines • Show All 140 Lines • ▼ Show 20 Linesstatic CoroMachinery setupCoroMachinery(FuncOp func) {
// Mark the end of a coroutine: async.coro.end // Mark the end of a coroutine: async.coro.end
builder.create<CoroEndOp>(coroHdlOp.handle()); builder.create<CoroEndOp>(coroHdlOp.handle());
// Return created `async.token` and `async.values` from the suspend block. // Return created `async.token` and `async.values` from the suspend block.
// This will be the return value of a coroutine ramp function. // This will be the return value of a coroutine ramp function.
SmallVector<Value, 4> ret{retToken}; SmallVector<Value, 4> ret{retToken};
ret.insert(ret.end(), retValues.begin(), retValues.end()); ret.insert(ret.end(), retValues.begin(), retValues.end());
builder.create<ReturnOp>(ret); builder.create<func::ReturnOp>(ret);
// `async.await` op lowering will create resume blocks for async // `async.await` op lowering will create resume blocks for async
// continuations, and will conditionally branch to cleanup or suspend blocks. // continuations, and will conditionally branch to cleanup or suspend blocks.
for (Block &block : func.body().getBlocks()) { for (Block &block : func.body().getBlocks()) {
if (&block == entryBlock || &block == cleanupBlock || if (&block == entryBlock || &block == cleanupBlock ||
&block == suspendBlock) &block == suspendBlock)
continue; continue;
▲ Show 20 Lines • Show All 135 Lines • ▼ Show 20 Lines builder.create<CoroSuspendOp>(coroSaveOp.state(), coro.suspend,
branch.getDest(), coro.cleanup); branch.getDest(), coro.cleanup);
branch.erase(); branch.erase();
} }
// Replace the original `async.execute` with a call to outlined function. // Replace the original `async.execute` with a call to outlined function.
{ {
ImplicitLocOpBuilder callBuilder(loc, execute); ImplicitLocOpBuilder callBuilder(loc, execute);
auto callOutlinedFunc = callBuilder.create<CallOp>( auto callOutlinedFunc = callBuilder.create<func::CallOp>(
func.getName(), execute.getResultTypes(), functionInputs.getArrayRef()); func.getName(), execute.getResultTypes(), functionInputs.getArrayRef());
execute.replaceAllUsesWith(callOutlinedFunc.getResults()); execute.replaceAllUsesWith(callOutlinedFunc.getResults());
execute.erase(); execute.erase();
} }
return {func, coro}; return {func, coro};
} }
▲ Show 20 Lines • Show All 217 Lines • ▼ Show 20 Lines matchAndRewrite(async::YieldOp op, OpAdaptor adaptor,
return success(); return success();
} }
private: private:
const llvm::DenseMap<FuncOp, CoroMachinery> &outlinedFunctions; const llvm::DenseMap<FuncOp, CoroMachinery> &outlinedFunctions;
}; };
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Convert std.assert operation to cf.cond_br into `set_error` block. // Convert cf.assert operation to cf.cond_br into `set_error` block.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
class AssertOpLowering : public OpConversionPattern<cf::AssertOp> { class AssertOpLowering : public OpConversionPattern<cf::AssertOp> {
public: public:
AssertOpLowering(MLIRContext *ctx, AssertOpLowering(MLIRContext *ctx,
llvm::DenseMap<FuncOp, CoroMachinery> &outlinedFunctions) llvm::DenseMap<FuncOp, CoroMachinery> &outlinedFunctions)
: OpConversionPattern<cf::AssertOp>(ctx), : OpConversionPattern<cf::AssertOp>(ctx),
outlinedFunctions(outlinedFunctions) {} outlinedFunctions(outlinedFunctions) {}
Show All 40 Linesstatic CoroMachinery rewriteFuncAsCoroutine(FuncOp func) {
SmallVector<Type> resultTypes; SmallVector<Type> resultTypes;
resultTypes.reserve(func.getCallableResults().size()); resultTypes.reserve(func.getCallableResults().size());
llvm::transform(func.getCallableResults(), std::back_inserter(resultTypes), llvm::transform(func.getCallableResults(), std::back_inserter(resultTypes),
[](Type type) { return ValueType::get(type); }); [](Type type) { return ValueType::get(type); });
func.setType(FunctionType::get(ctx, func.getType().getInputs(), resultTypes)); func.setType(FunctionType::get(ctx, func.getType().getInputs(), resultTypes));
func.insertResult(0, TokenType::get(ctx), {}); func.insertResult(0, TokenType::get(ctx), {});
for (Block &block : func.getBlocks()) { for (Block &block : func.getBlocks()) {
Operation *terminator = block.getTerminator(); Operation *terminator = block.getTerminator();
if (auto returnOp = dyn_cast<ReturnOp>(*terminator)) { if (auto returnOp = dyn_cast<func::ReturnOp>(*terminator)) {
ImplicitLocOpBuilder builder(loc, returnOp); ImplicitLocOpBuilder builder(loc, returnOp);
builder.create<YieldOp>(returnOp.getOperands()); builder.create<YieldOp>(returnOp.getOperands());
returnOp.erase(); returnOp.erase();
} }
} }
return setupCoroMachinery(func); return setupCoroMachinery(func);
} }
/// Rewrites a call into a function that has been rewritten as a coroutine. /// Rewrites a call into a function that has been rewritten as a coroutine.
/// ///
/// The invocation of this function is safe only when call ops are traversed in /// The invocation of this function is safe only when call ops are traversed in
/// reverse order of how they appear in a single block. See `funcsToCoroutines`. /// reverse order of how they appear in a single block. See `funcsToCoroutines`.
static void rewriteCallsiteForCoroutine(CallOp oldCall, FuncOp func) { static void rewriteCallsiteForCoroutine(func::CallOp oldCall, FuncOp func) {
auto loc = func.getLoc(); auto loc = func.getLoc();
ImplicitLocOpBuilder callBuilder(loc, oldCall); ImplicitLocOpBuilder callBuilder(loc, oldCall);
auto newCall = callBuilder.create<CallOp>( auto newCall = callBuilder.create<func::CallOp>(
func.getName(), func.getCallableResults(), oldCall.getArgOperands()); func.getName(), func.getCallableResults(), oldCall.getArgOperands());
// Await on the async token and all the value results and unwrap the latter. // Await on the async token and all the value results and unwrap the latter.
callBuilder.create<AwaitOp>(loc, newCall.getResults().front()); callBuilder.create<AwaitOp>(loc, newCall.getResults().front());
SmallVector<Value> unwrappedResults; SmallVector<Value> unwrappedResults;
unwrappedResults.reserve(newCall->getResults().size() - 1); unwrappedResults.reserve(newCall->getResults().size() - 1);
for (Value result : newCall.getResults().drop_front()) for (Value result : newCall.getResults().drop_front())
unwrappedResults.push_back( unwrappedResults.push_back(
▲ Show 20 Lines • Show All 65 Lines • ▼ Show 20 Lines while (!funcWorklist.empty()) {
llvm::sort(users, [](Operation *a, Operation *b) { llvm::sort(users, [](Operation *a, Operation *b) {
Block *blockA = a->getBlock(); Block *blockA = a->getBlock();
Block *blockB = b->getBlock(); Block *blockB = b->getBlock();
// Impose arbitrary order on blocks so that there is a well-defined order. // Impose arbitrary order on blocks so that there is a well-defined order.
return blockA > blockB || (blockA == blockB && !a->isBeforeInBlock(b)); return blockA > blockB || (blockA == blockB && !a->isBeforeInBlock(b));
}); });
// Rewrite the callsites to await on results of the newly created coroutine. // Rewrite the callsites to await on results of the newly created coroutine.
for (Operation *op : users) { for (Operation *op : users) {
if (CallOp call = dyn_cast<mlir::CallOp>(*op)) { if (func::CallOp call = dyn_cast<func::CallOp>(*op)) {
FuncOp caller = call->getParentOfType<FuncOp>(); FuncOp caller = call->getParentOfType<FuncOp>();
rewriteCallsiteForCoroutine(call, func); // Careful, erases the call op. rewriteCallsiteForCoroutine(call, func); // Careful, erases the call op.
addToWorklist(caller); addToWorklist(caller);
} else { } else {
op->emitError("Unexpected reference to func referenced by symbol"); op->emitError("Unexpected reference to func referenced by symbol");
return failure(); return failure();
} }
} }
▲ Show 20 Lines • Show All 61 Lines • ▼ Show 20 Lines runtimeTarget.addDynamicallyLegalDialect<scf::SCFDialect>([&](Operation *op) {
auto walkResult = op->walk([&](Operation *nested) { auto walkResult = op->walk([&](Operation *nested) {
bool isAsync = isa<async::AsyncDialect>(nested->getDialect()); bool isAsync = isa<async::AsyncDialect>(nested->getDialect());
return isAsync && isInCoroutine(nested) ? WalkResult::interrupt() return isAsync && isInCoroutine(nested) ? WalkResult::interrupt()
: WalkResult::advance(); : WalkResult::advance();
}); });
return !walkResult.wasInterrupted(); return !walkResult.wasInterrupted();
}); });
runtimeTarget.addLegalOp<cf::AssertOp, arith::XOrIOp, arith::ConstantOp, runtimeTarget.addLegalOp<cf::AssertOp, arith::XOrIOp, arith::ConstantOp,
ConstantOp, cf::BranchOp, cf::CondBranchOp>(); func::ConstantOp, cf::BranchOp, cf::CondBranchOp>();
// Assertions must be converted to runtime errors inside async functions. // Assertions must be converted to runtime errors inside async functions.
runtimeTarget.addDynamicallyLegalOp<cf::AssertOp>( runtimeTarget.addDynamicallyLegalOp<cf::AssertOp>(
[&](cf::AssertOp op) -> bool { [&](cf::AssertOp op) -> bool {
auto func = op->getParentOfType<FuncOp>(); auto func = op->getParentOfType<FuncOp>();
return outlinedFunctions.find(func) == outlinedFunctions.end(); return outlinedFunctions.find(func) == outlinedFunctions.end();
}); });
Show All 16 Lines

mlir/lib/Dialect/Async/Transforms/CMakeLists.txt

add_mlir_dialect_library(MLIRAsyncTransforms add_mlir_dialect_library(MLIRAsyncTransforms
AsyncParallelFor.cpp AsyncParallelFor.cpp
AsyncRuntimeRefCounting.cpp AsyncRuntimeRefCounting.cpp
AsyncRuntimeRefCountingOpt.cpp AsyncRuntimeRefCountingOpt.cpp
AsyncToAsyncRuntime.cpp AsyncToAsyncRuntime.cpp
PassDetail.cpp PassDetail.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Async ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Async
DEPENDS DEPENDS
MLIRAsyncPassIncGen MLIRAsyncPassIncGen
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRArithmetic MLIRArithmetic
MLIRAsync MLIRAsync
MLIRFunc
MLIRIR MLIRIR
MLIRPass MLIRPass
MLIRSCF MLIRSCF
MLIRSCFToControlFlow MLIRSCFToControlFlow
MLIRStandard
MLIRTransforms MLIRTransforms
MLIRTransformUtils MLIRTransformUtils
) )

mlir/lib/Dialect/Bufferization/Transforms/BufferResultsToOutParams.cpp

//===- BufferResultsToOutParams.cpp - Calling convention conversion -------===// //===- BufferResultsToOutParams.cpp - Calling convention conversion -------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Bufferization/Transforms/Passes.h" #include "mlir/Dialect/Bufferization/Transforms/Passes.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/Operation.h" #include "mlir/IR/Operation.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
using namespace mlir; using namespace mlir;
// Updates the func op and entry block. // Updates the func op and entry block.
// //
// Any args appended to the entry block are added to `appendedEntryArgs`. // Any args appended to the entry block are added to `appendedEntryArgs`.
Show All 36 Lines for (Type type : erasedResultTypes)
appendedEntryArgs.push_back(func.front().addArgument(type, loc)); appendedEntryArgs.push_back(func.front().addArgument(type, loc));
} }
// Updates all ReturnOps in the scope of the given FuncOp by either keeping them // Updates all ReturnOps in the scope of the given FuncOp by either keeping them
// as return values or copying the associated buffer contents into the given // as return values or copying the associated buffer contents into the given
// out-params. // out-params.
static void updateReturnOps(FuncOp func, static void updateReturnOps(FuncOp func,
ArrayRef<BlockArgument> appendedEntryArgs) { ArrayRef<BlockArgument> appendedEntryArgs) {
func.walk([&](ReturnOp op) { func.walk([&](func::ReturnOp op) {
SmallVector<Value, 6> copyIntoOutParams; SmallVector<Value, 6> copyIntoOutParams;
SmallVector<Value, 6> keepAsReturnOperands; SmallVector<Value, 6> keepAsReturnOperands;
for (Value operand : op.getOperands()) { for (Value operand : op.getOperands()) {
if (operand.getType().isa<BaseMemRefType>()) if (operand.getType().isa<BaseMemRefType>())
copyIntoOutParams.push_back(operand); copyIntoOutParams.push_back(operand);
else else
keepAsReturnOperands.push_back(operand); keepAsReturnOperands.push_back(operand);
} }
OpBuilder builder(op); OpBuilder builder(op);
for (auto t : llvm::zip(copyIntoOutParams, appendedEntryArgs)) for (auto t : llvm::zip(copyIntoOutParams, appendedEntryArgs))
builder.create<memref::CopyOp>(op.getLoc(), std::get<0>(t), builder.create<memref::CopyOp>(op.getLoc(), std::get<0>(t),
std::get<1>(t)); std::get<1>(t));
builder.create<ReturnOp>(op.getLoc(), keepAsReturnOperands); builder.create<func::ReturnOp>(op.getLoc(), keepAsReturnOperands);
op.erase(); op.erase();
}); });
} }
// Updates all CallOps in the scope of the given ModuleOp by allocating // Updates all CallOps in the scope of the given ModuleOp by allocating
// temporary buffers for newly introduced out params. // temporary buffers for newly introduced out params.
static LogicalResult updateCalls(ModuleOp module) { static LogicalResult updateCalls(ModuleOp module) {
bool didFail = false; bool didFail = false;
module.walk([&](CallOp op) { module.walk([&](func::CallOp op) {
SmallVector<Value, 6> replaceWithNewCallResults; SmallVector<Value, 6> replaceWithNewCallResults;
SmallVector<Value, 6> replaceWithOutParams; SmallVector<Value, 6> replaceWithOutParams;
for (OpResult result : op.getResults()) { for (OpResult result : op.getResults()) {
if (result.getType().isa<BaseMemRefType>()) if (result.getType().isa<BaseMemRefType>())
replaceWithOutParams.push_back(result); replaceWithOutParams.push_back(result);
else else
replaceWithNewCallResults.push_back(result); replaceWithNewCallResults.push_back(result);
} }
Show All 11 Lines for (Value memref : replaceWithOutParams) {
memref.replaceAllUsesWith(outParam); memref.replaceAllUsesWith(outParam);
outParams.push_back(outParam); outParams.push_back(outParam);
} }
auto newOperands = llvm::to_vector<6>(op.getOperands()); auto newOperands = llvm::to_vector<6>(op.getOperands());
newOperands.append(outParams.begin(), outParams.end()); newOperands.append(outParams.begin(), outParams.end());
auto newResultTypes = llvm::to_vector<6>(llvm::map_range( auto newResultTypes = llvm::to_vector<6>(llvm::map_range(
replaceWithNewCallResults, [](Value v) { return v.getType(); })); replaceWithNewCallResults, [](Value v) { return v.getType(); }));
auto newCall = builder.create<CallOp>(op.getLoc(), op.getCalleeAttr(), auto newCall = builder.create<func::CallOp>(op.getLoc(), op.getCalleeAttr(),
newResultTypes, newOperands); newResultTypes, newOperands);
for (auto t : llvm::zip(replaceWithNewCallResults, newCall.getResults())) for (auto t : llvm::zip(replaceWithNewCallResults, newCall.getResults()))
std::get<0>(t).replaceAllUsesWith(std::get<1>(t)); std::get<0>(t).replaceAllUsesWith(std::get<1>(t));
op.erase(); op.erase();
}); });
return failure(didFail); return failure(didFail);
} }
Show All 23 Lines

mlir/lib/Dialect/Bufferization/Transforms/Bufferize.cpp

//===- Bufferize.cpp - Bufferization utilities ----------------------------===// //===- Bufferize.cpp - Bufferization utilities ----------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Bufferization/IR/BufferizableOpInterface.h" #include "mlir/Dialect/Bufferization/IR/BufferizableOpInterface.h"
#include "mlir/Dialect/Bufferization/IR/Bufferization.h" #include "mlir/Dialect/Bufferization/IR/Bufferization.h"
#include "mlir/Dialect/Bufferization/Transforms/Bufferize.h" #include "mlir/Dialect/Bufferization/Transforms/Bufferize.h"
#include "mlir/Dialect/Bufferization/Transforms/OneShotAnalysis.h" #include "mlir/Dialect/Bufferization/Transforms/OneShotAnalysis.h"
#include "mlir/Dialect/Bufferization/Transforms/Passes.h" #include "mlir/Dialect/Bufferization/Transforms/Passes.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/Operation.h" #include "mlir/IR/Operation.h"
#include "mlir/Pass/PassManager.h" #include "mlir/Pass/PassManager.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
using namespace mlir; using namespace mlir;
using namespace mlir::bufferization; using namespace mlir::bufferization;
▲ Show 20 Lines • Show All 145 Lines • ▼ Show 20 Lines if (!options) {
opt.analysisFuzzerSeed = analysisFuzzerSeed; opt.analysisFuzzerSeed = analysisFuzzerSeed;
opt.createDeallocs = createDeallocs; opt.createDeallocs = createDeallocs;
opt.fullyDynamicLayoutMaps = fullyDynamicLayoutMaps; opt.fullyDynamicLayoutMaps = fullyDynamicLayoutMaps;
opt.printConflicts = printConflicts; opt.printConflicts = printConflicts;
opt.testAnalysisOnly = testAnalysisOnly; opt.testAnalysisOnly = testAnalysisOnly;
BufferizationOptions::OpFilterEntry::FilterFn filterFn = BufferizationOptions::OpFilterEntry::FilterFn filterFn =
[&](Operation *op) { [&](Operation *op) {
// Disallow non-std dialect ops. I.e., no ops related to function // Disallow non-func dialect ops. I.e., no ops related to function
// calls. // calls.
if (op->getDialect()->getNamespace() == if (isa<func::FuncDialect>(op->getDialect()))
StandardOpsDialect::getDialectNamespace())
return false; return false;
// Filter may be specified via options. // Filter may be specified via options.
if (this->dialectFilter.hasValue()) if (this->dialectFilter.hasValue())
return llvm::find(this->dialectFilter, return llvm::find(this->dialectFilter,
op->getDialect()->getNamespace()) != op->getDialect()->getNamespace()) !=
this->dialectFilter.end(); this->dialectFilter.end();
// No filter specified: All other ops are allowed. // No filter specified: All other ops are allowed.
return true; return true;
▲ Show 20 Lines • Show All 170 Lines • Show Last 20 Lines

mlir/lib/Dialect/Bufferization/Transforms/CMakeLists.txt

Show All 9 Linesadd_mlir_dialect_library(MLIRBufferizationTransforms
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Bufferization ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Bufferization
DEPENDS DEPENDS
MLIRBufferizationPassIncGen MLIRBufferizationPassIncGen
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRBufferization MLIRBufferization
MLIRControlFlowInterfaces MLIRControlFlowInterfaces
MLIRFunc
MLIRInferTypeOpInterface MLIRInferTypeOpInterface
MLIRIR MLIRIR
MLIRMemRef MLIRMemRef
MLIRPass MLIRPass
MLIRStandard
MLIRTransforms MLIRTransforms
) )

mlir/lib/Dialect/Bufferization/Transforms/PassDetail.h

//===- PassDetail.h - Bufferization Pass details ----------------*- C++ -*-===// //===- PassDetail.h - Bufferization Pass details ----------------*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef DIALECT_BUFFERIZATION_TRANSFORMS_PASSDETAIL_H_ #ifndef DIALECT_BUFFERIZATION_TRANSFORMS_PASSDETAIL_H_
#define DIALECT_BUFFERIZATION_TRANSFORMS_PASSDETAIL_H_ #define DIALECT_BUFFERIZATION_TRANSFORMS_PASSDETAIL_H_
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
namespace mlir { namespace mlir {
class StandardOpsDialect;
namespace bufferization { namespace bufferization {
class BufferizationDialect; class BufferizationDialect;
} // namespace bufferization } // namespace bufferization
namespace func {
class FuncDialect;
} // namespace func
namespace memref { namespace memref {
class MemRefDialect; class MemRefDialect;
} // namespace memref } // namespace memref
#define GEN_PASS_CLASSES #define GEN_PASS_CLASSES
#include "mlir/Dialect/Bufferization/Transforms/Passes.h.inc" #include "mlir/Dialect/Bufferization/Transforms/Passes.h.inc"
} // namespace mlir } // namespace mlir
#endif // DIALECT_BUFFERIZATION_TRANSFORMS_PASSDETAIL_H_ #endif // DIALECT_BUFFERIZATION_TRANSFORMS_PASSDETAIL_H_

mlir/lib/Dialect/CMakeLists.txt

add_subdirectory(Affine) add_subdirectory(Affine)
add_subdirectory(Arithmetic) add_subdirectory(Arithmetic)
add_subdirectory(ArmNeon) add_subdirectory(ArmNeon)
add_subdirectory(ArmSVE) add_subdirectory(ArmSVE)
add_subdirectory(Async) add_subdirectory(Async)
add_subdirectory(AMX) add_subdirectory(AMX)
add_subdirectory(Bufferization) add_subdirectory(Bufferization)
add_subdirectory(Complex) add_subdirectory(Complex)
add_subdirectory(ControlFlow) add_subdirectory(ControlFlow)
add_subdirectory(DLTI) add_subdirectory(DLTI)
add_subdirectory(EmitC) add_subdirectory(EmitC)
add_subdirectory(Func)
add_subdirectory(GPU) add_subdirectory(GPU)
add_subdirectory(Linalg) add_subdirectory(Linalg)
add_subdirectory(LLVMIR) add_subdirectory(LLVMIR)
add_subdirectory(Math) add_subdirectory(Math)
add_subdirectory(MemRef) add_subdirectory(MemRef)
add_subdirectory(OpenACC) add_subdirectory(OpenACC)
add_subdirectory(OpenMP) add_subdirectory(OpenMP)
add_subdirectory(PDL) add_subdirectory(PDL)
add_subdirectory(PDLInterp) add_subdirectory(PDLInterp)
add_subdirectory(Quant) add_subdirectory(Quant)
add_subdirectory(SCF) add_subdirectory(SCF)
add_subdirectory(Shape) add_subdirectory(Shape)
add_subdirectory(SparseTensor) add_subdirectory(SparseTensor)
add_subdirectory(SPIRV) add_subdirectory(SPIRV)
add_subdirectory(StandardOps)
add_subdirectory(Tensor) add_subdirectory(Tensor)
add_subdirectory(Tosa) add_subdirectory(Tosa)
add_subdirectory(Utils) add_subdirectory(Utils)
add_subdirectory(Vector) add_subdirectory(Vector)
add_subdirectory(X86Vector) add_subdirectory(X86Vector)
set(LLVM_OPTIONAL_SOURCES set(LLVM_OPTIONAL_SOURCES
Traits.cpp Traits.cpp
Show All 11 Lines

mlir/lib/Dialect/Func/CMakeLists.txt

  • This file was moved from mlir/include/mlir/Dialect/StandardOps/CMakeLists.txt.
add_subdirectory(IR) add_subdirectory(IR)
add_subdirectory(Transforms) add_subdirectory(Transforms)

mlir/lib/Dialect/Func/IR/CMakeLists.txt

  • This file was added.
add_mlir_dialect_library(MLIRFunc
FuncOps.cpp
ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Func
DEPENDS
MLIRFuncOpsIncGen
LINK_LIBS PUBLIC
MLIRCallInterfaces
MLIRControlFlow
MLIRControlFlowInterfaces
MLIRInferTypeOpInterface
MLIRIR
MLIRSideEffectInterfaces
)

mlir/lib/Dialect/Func/IR/FuncOps.cpp

  • This file was moved from mlir/lib/Dialect/StandardOps/IR/Ops.cpp.
//===- Ops.cpp - Standard MLIR Operations ---------------------------------===// //===- FuncOps.cpp - Func Dialect Operations ------------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/CommonFolders.h"
#include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h" #include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h"
#include "mlir/IR/AffineExpr.h"
#include "mlir/IR/AffineMap.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/BuiltinTypes.h" #include "mlir/IR/BuiltinTypes.h"
#include "mlir/IR/Matchers.h" #include "mlir/IR/Matchers.h"
#include "mlir/IR/OpImplementation.h" #include "mlir/IR/OpImplementation.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/IR/TypeUtilities.h" #include "mlir/IR/TypeUtilities.h"
#include "mlir/IR/Value.h" #include "mlir/IR/Value.h"
#include "mlir/Support/MathExtras.h" #include "mlir/Support/MathExtras.h"
#include "mlir/Transforms/InliningUtils.h" #include "mlir/Transforms/InliningUtils.h"
#include "llvm/ADT/APFloat.h" #include "llvm/ADT/APFloat.h"
#include "llvm/ADT/STLExtras.h" #include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/StringSwitch.h" #include "llvm/ADT/StringSwitch.h"
#include "llvm/Support/FormatVariadic.h" #include "llvm/Support/FormatVariadic.h"
#include "llvm/Support/raw_ostream.h" #include "llvm/Support/raw_ostream.h"
#include <numeric> #include <numeric>
#include "mlir/Dialect/StandardOps/IR/OpsDialect.cpp.inc" #include "mlir/Dialect/Func/IR/FuncOpsDialect.cpp.inc"
// Pull in all enum type definitions and utility function declarations.
#include "mlir/Dialect/StandardOps/IR/OpsEnums.cpp.inc"
using namespace mlir; using namespace mlir;
using namespace mlir::func;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// StandardOpsDialect Interfaces // FuncDialect Interfaces
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
namespace { namespace {
/// This class defines the interface for handling inlining with standard /// This class defines the interface for handling inlining with func operations.
/// operations. struct FuncInlinerInterface : public DialectInlinerInterface {
struct StdInlinerInterface : public DialectInlinerInterface {
using DialectInlinerInterface::DialectInlinerInterface; using DialectInlinerInterface::DialectInlinerInterface;
//===--------------------------------------------------------------------===// //===--------------------------------------------------------------------===//
// Analysis Hooks // Analysis Hooks
//===--------------------------------------------------------------------===// //===--------------------------------------------------------------------===//
/// All call operations within standard ops can be inlined. /// All call operations can be inlined.
bool isLegalToInline(Operation *call, Operation *callable, bool isLegalToInline(Operation *call, Operation *callable,
bool wouldBeCloned) const final { bool wouldBeCloned) const final {
return true; return true;
} }
/// All operations within standard ops can be inlined. /// All operations can be inlined.
bool isLegalToInline(Operation *, Region *, bool, bool isLegalToInline(Operation *, Region *, bool,
BlockAndValueMapping &) const final { BlockAndValueMapping &) const final {
return true; return true;
} }
//===--------------------------------------------------------------------===// //===--------------------------------------------------------------------===//
// Transformation Hooks // Transformation Hooks
//===--------------------------------------------------------------------===// //===--------------------------------------------------------------------===//
/// Handle the given inlined terminator by replacing it with a new operation /// Handle the given inlined terminator by replacing it with a new operation
/// as necessary. /// as necessary.
void handleTerminator(Operation *op, Block *newDest) const final { void handleTerminator(Operation *op, Block *newDest) const final {
// Only "std.return" needs to be handled here. // Only return needs to be handled here.
auto returnOp = dyn_cast<ReturnOp>(op); auto returnOp = dyn_cast<ReturnOp>(op);
if (!returnOp) if (!returnOp)
return; return;
// Replace the return with a branch to the dest. // Replace the return with a branch to the dest.
OpBuilder builder(op); OpBuilder builder(op);
builder.create<cf::BranchOp>(op->getLoc(), newDest, returnOp.getOperands()); builder.create<cf::BranchOp>(op->getLoc(), newDest, returnOp.getOperands());
op->erase(); op->erase();
} }
/// Handle the given inlined terminator by replacing it with a new operation /// Handle the given inlined terminator by replacing it with a new operation
/// as necessary. /// as necessary.
void handleTerminator(Operation *op, void handleTerminator(Operation *op,
ArrayRef<Value> valuesToRepl) const final { ArrayRef<Value> valuesToRepl) const final {
// Only "std.return" needs to be handled here. // Only return needs to be handled here.
auto returnOp = cast<ReturnOp>(op); auto returnOp = cast<ReturnOp>(op);
// Replace the values directly with the return operands. // Replace the values directly with the return operands.
assert(returnOp.getNumOperands() == valuesToRepl.size()); assert(returnOp.getNumOperands() == valuesToRepl.size());
for (const auto &it : llvm::enumerate(returnOp.getOperands())) for (const auto &it : llvm::enumerate(returnOp.getOperands()))
valuesToRepl[it.index()].replaceAllUsesWith(it.value()); valuesToRepl[it.index()].replaceAllUsesWith(it.value());
} }
}; };
} // namespace } // namespace
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// StandardOpsDialect // FuncDialect
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
void StandardOpsDialect::initialize() { void FuncDialect::initialize() {
addOperations< addOperations<
#define GET_OP_LIST #define GET_OP_LIST
#include "mlir/Dialect/StandardOps/IR/Ops.cpp.inc" #include "mlir/Dialect/Func/IR/FuncOps.cpp.inc"
>(); >();
addInterfaces<StdInlinerInterface>(); addInterfaces<FuncInlinerInterface>();
} }
/// Materialize a single constant operation from a given attribute value with /// Materialize a single constant operation from a given attribute value with
/// the desired resultant type. /// the desired resultant type.
Operation *StandardOpsDialect::materializeConstant(OpBuilder &builder, Operation *FuncDialect::materializeConstant(OpBuilder &builder, Attribute value,
Attribute value, Type type, Type type, Location loc) {
Location loc) {
if (arith::ConstantOp::isBuildableWith(value, type))
return builder.create<arith::ConstantOp>(loc, type, value);
if (ConstantOp::isBuildableWith(value, type)) if (ConstantOp::isBuildableWith(value, type))
return builder.create<ConstantOp>(loc, type, return builder.create<ConstantOp>(loc, type,
value.cast<FlatSymbolRefAttr>()); value.cast<FlatSymbolRefAttr>());
return nullptr; return nullptr;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// CallOp // CallOp
▲ Show 20 Lines • Show All 117 Lines • ▼ Show 20 LinesLogicalResult ReturnOp::verify() {
return success(); return success();
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// TableGen'd op method definitions // TableGen'd op method definitions
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#define GET_OP_CLASSES #define GET_OP_CLASSES
#include "mlir/Dialect/StandardOps/IR/Ops.cpp.inc" #include "mlir/Dialect/Func/IR/FuncOps.cpp.inc"

mlir/lib/Dialect/Func/Transforms/CMakeLists.txt

  • This file was moved from mlir/lib/Dialect/StandardOps/Transforms/CMakeLists.txt.
add_mlir_dialect_library(MLIRStandardOpsTransforms add_mlir_dialect_library(MLIRFuncTransforms
DecomposeCallGraphTypes.cpp DecomposeCallGraphTypes.cpp
FuncBufferize.cpp FuncBufferize.cpp
FuncConversions.cpp FuncConversions.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/StandardOps/Transforms ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Func/Transforms
DEPENDS DEPENDS
MLIRStandardTransformsIncGen MLIRFuncTransformsIncGen
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRAffine MLIRAffine
MLIRArithmeticTransforms MLIRArithmeticTransforms
MLIRBufferization MLIRBufferization
MLIRBufferizationTransforms MLIRBufferizationTransforms
MLIRFunc
MLIRIR MLIRIR
MLIRMemRef MLIRMemRef
MLIRPass MLIRPass
MLIRSCF MLIRSCF
MLIRStandard
MLIRTransforms MLIRTransforms
) )

mlir/lib/Dialect/Func/Transforms/DecomposeCallGraphTypes.cpp

  • This file was moved from mlir/lib/Dialect/StandardOps/Transforms/DecomposeCallGraphTypes.cpp.
//===- DecomposeCallGraphTypes.cpp - CG type decomposition ----------------===// //===- DecomposeCallGraphTypes.cpp - CG type decomposition ----------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/StandardOps/Transforms/DecomposeCallGraphTypes.h" #include "mlir/Dialect/Func/Transforms/DecomposeCallGraphTypes.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
using namespace mlir; using namespace mlir;
using namespace mlir::func;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// ValueDecomposer // ValueDecomposer
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
void ValueDecomposer::decomposeValue(OpBuilder &builder, Location loc, void ValueDecomposer::decomposeValue(OpBuilder &builder, Location loc,
Type type, Value value, Type type, Value value,
SmallVectorImpl<Value> &results) { SmallVectorImpl<Value> &results) {
▲ Show 20 Lines • Show All 175 Lines • Show Last 20 Lines

mlir/lib/Dialect/Func/Transforms/FuncBufferize.cpp

  • This file was moved from mlir/lib/Dialect/StandardOps/Transforms/FuncBufferize.cpp.
//===- Bufferize.cpp - Bufferization for std ops --------------------------===// //===- Bufferize.cpp - Bufferization for func ops -------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements bufferization of std.func's and std.call's. // This file implements bufferization of builtin.func's and func.call's.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Bufferization/IR/Bufferization.h" #include "mlir/Dialect/Bufferization/IR/Bufferization.h"
#include "mlir/Dialect/Bufferization/Transforms/Bufferize.h" #include "mlir/Dialect/Bufferization/Transforms/Bufferize.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/StandardOps/Transforms/FuncConversions.h" #include "mlir/Dialect/Func/Transforms/FuncConversions.h"
#include "mlir/Dialect/StandardOps/Transforms/Passes.h" #include "mlir/Dialect/Func/Transforms/Passes.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
using namespace mlir; using namespace mlir;
using namespace mlir::func;
namespace { namespace {
struct FuncBufferizePass : public FuncBufferizeBase<FuncBufferizePass> { struct FuncBufferizePass : public FuncBufferizeBase<FuncBufferizePass> {
using FuncBufferizeBase<FuncBufferizePass>::FuncBufferizeBase; using FuncBufferizeBase<FuncBufferizePass>::FuncBufferizeBase;
void runOnOperation() override { void runOnOperation() override {
auto module = getOperation(); auto module = getOperation();
auto *context = &getContext(); auto *context = &getContext();
Show All 24 Lines void runOnOperation() override {
}); });
if (failed(applyFullConversion(module, target, std::move(patterns)))) if (failed(applyFullConversion(module, target, std::move(patterns))))
signalPassFailure(); signalPassFailure();
} }
}; };
} // namespace } // namespace
std::unique_ptr<Pass> mlir::createFuncBufferizePass() { std::unique_ptr<Pass> mlir::func::createFuncBufferizePass() {
return std::make_unique<FuncBufferizePass>(); return std::make_unique<FuncBufferizePass>();
} }

mlir/lib/Dialect/Func/Transforms/FuncConversions.cpp

  • This file was moved from mlir/lib/Dialect/StandardOps/Transforms/FuncConversions.cpp.
//===- FuncConversions.cpp - Standard Function conversions ----------------===// //===- FuncConversions.cpp - Function conversions -------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/StandardOps/Transforms/FuncConversions.h" #include "mlir/Dialect/Func/Transforms/FuncConversions.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
using namespace mlir; using namespace mlir;
using namespace mlir::func;
namespace { namespace {
/// Converts the operand and result types of the Standard's CallOp, used /// Converts the operand and result types of the CallOp, used together with the
/// together with the FuncOpSignatureConversion. /// FuncOpSignatureConversion.
struct CallOpSignatureConversion : public OpConversionPattern<CallOp> { struct CallOpSignatureConversion : public OpConversionPattern<CallOp> {
using OpConversionPattern<CallOp>::OpConversionPattern; using OpConversionPattern<CallOp>::OpConversionPattern;
/// Hook for derived classes to implement combined matching and rewriting. /// Hook for derived classes to implement combined matching and rewriting.
LogicalResult LogicalResult
matchAndRewrite(CallOp callOp, OpAdaptor adaptor, matchAndRewrite(CallOp callOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override { ConversionPatternRewriter &rewriter) const override {
// Convert the original function results. // Convert the original function results.
▲ Show 20 Lines • Show All 144 Lines • Show Last 20 Lines

mlir/lib/Dialect/Func/Transforms/PassDetail.h

  • This file was moved from mlir/lib/Dialect/StandardOps/Transforms/PassDetail.h.
//===- PassDetail.h - GPU Pass class details --------------------*- C++ -*-===// //===- PassDetail.h - Func Pass class details -------------------*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef DIALECT_STANDARD_TRANSFORMS_PASSDETAIL_H_ #ifndef DIALECT_FUNC_TRANSFORMS_PASSDETAIL_H_
#define DIALECT_STANDARD_TRANSFORMS_PASSDETAIL_H_ #define DIALECT_FUNC_TRANSFORMS_PASSDETAIL_H_
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
namespace mlir { namespace mlir {
class AtomicRMWOp; class AtomicRMWOp;
namespace bufferization { namespace bufferization {
class BufferizationDialect; class BufferizationDialect;
} // namespace bufferization } // namespace bufferization
namespace memref { namespace memref {
class MemRefDialect; class MemRefDialect;
} // namespace memref } // namespace memref
#define GEN_PASS_CLASSES #define GEN_PASS_CLASSES
#include "mlir/Dialect/StandardOps/Transforms/Passes.h.inc" #include "mlir/Dialect/Func/Transforms/Passes.h.inc"
} // namespace mlir } // namespace mlir
#endif // DIALECT_STANDARD_TRANSFORMS_PASSDETAIL_H_ #endif // DIALECT_FUNC_TRANSFORMS_PASSDETAIL_H_

mlir/lib/Dialect/GPU/CMakeLists.txt

Show First 20 Lines • Show All 63 Lines • ▼ Show 20 Linesadd_mlir_dialect_library(MLIRGPUTransforms
MLIRParallelLoopMapperAttrGen MLIRParallelLoopMapperAttrGen
MLIRParallelLoopMapperEnumsGen MLIRParallelLoopMapperEnumsGen
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRAffineUtils MLIRAffineUtils
MLIRArithmetic MLIRArithmetic
MLIRAsync MLIRAsync
MLIRDataLayoutInterfaces MLIRDataLayoutInterfaces
MLIRFunc
MLIRGPUOps MLIRGPUOps
MLIRIR MLIRIR
MLIRLLVMIR MLIRLLVMIR
MLIRLLVMToLLVMIRTranslation MLIRLLVMToLLVMIRTranslation
MLIRMemRef MLIRMemRef
MLIRPass MLIRPass
MLIRSCF MLIRSCF
MLIRStandard
MLIRSupport MLIRSupport
MLIRTransformUtils MLIRTransformUtils
) )
if(MLIR_ENABLE_CUDA_RUNNER) if(MLIR_ENABLE_CUDA_RUNNER)
if(NOT MLIR_ENABLE_CUDA_CONVERSIONS) if(NOT MLIR_ENABLE_CUDA_CONVERSIONS)
message(SEND_ERROR message(SEND_ERROR
"Building mlir with cuda support requires the NVPTX backend") "Building mlir with cuda support requires the NVPTX backend")
▲ Show 20 Lines • Show All 54 Lines • Show Last 20 Lines

mlir/lib/Dialect/GPU/Transforms/KernelOutlining.cpp

//===- KernelOutlining.cpp - Implementation of GPU kernel outlining -------===// //===- KernelOutlining.cpp - Implementation of GPU kernel outlining -------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements the GPU dialect kernel outlining pass. // This file implements the GPU dialect kernel outlining pass.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h" #include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h"
#include "mlir/Dialect/DLTI/DLTI.h" #include "mlir/Dialect/DLTI/DLTI.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/GPU/GPUDialect.h" #include "mlir/Dialect/GPU/GPUDialect.h"
#include "mlir/Dialect/GPU/Passes.h" #include "mlir/Dialect/GPU/Passes.h"
#include "mlir/Dialect/GPU/Utils.h" #include "mlir/Dialect/GPU/Utils.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/SymbolTable.h" #include "mlir/IR/SymbolTable.h"
#include "mlir/Parser.h" #include "mlir/Parser.h"
#include "mlir/Support/LLVM.h" #include "mlir/Support/LLVM.h"
#include "mlir/Transforms/RegionUtils.h" #include "mlir/Transforms/RegionUtils.h"
using namespace mlir; using namespace mlir;
Show All 25 Linesstatic void injectGpuIndexOperations(Location loc, Region &launchFuncOpBody,
for (const auto &indexOp : enumerate(indexOps)) for (const auto &indexOp : enumerate(indexOps))
map.map(firstBlock.getArgument(indexOp.index()), indexOp.value()); map.map(firstBlock.getArgument(indexOp.index()), indexOp.value());
} }
/// Identifies operations that are beneficial to sink into kernels. These /// Identifies operations that are beneficial to sink into kernels. These
/// operations may not have side-effects, as otherwise sinking (and hence /// operations may not have side-effects, as otherwise sinking (and hence
/// duplicating them) is not legal. /// duplicating them) is not legal.
static bool isLikelyAnIndexComputation(Operation *op) { static bool isLikelyAnIndexComputation(Operation *op) {
return isa<arith::ConstantOp, ConstantOp, memref::DimOp, arith::SelectOp, return isa<arith::ConstantOp, func::ConstantOp, memref::DimOp,
arith::CmpIOp>(op); arith::SelectOp, arith::CmpIOp>(op);
} }
/// For a given operation `op`, computes whether it is beneficial to sink the /// For a given operation `op`, computes whether it is beneficial to sink the
/// operation into the kernel. An operation can be sunk if doing so does not /// operation into the kernel. An operation can be sunk if doing so does not
/// introduce new kernel arguments. Whether a value is already available in the /// introduce new kernel arguments. Whether a value is already available in the
/// kernel (and hence does not introduce new arguments) is checked by /// kernel (and hence does not introduce new arguments) is checked by
/// querying `existingDependencies` and `availableValues`. /// querying `existingDependencies` and `availableValues`.
/// If an operand is not yet available, we recursively check whether it can be /// If an operand is not yet available, we recursively check whether it can be
▲ Show 20 Lines • Show All 315 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/Analysis/CMakeLists.txt

add_mlir_dialect_library(MLIRLinalgAnalysis add_mlir_dialect_library(MLIRLinalgAnalysis
DependenceAnalysis.cpp DependenceAnalysis.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Linalg ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Linalg
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRAffineAnalysis MLIRAffineAnalysis
MLIRAnalysis MLIRAnalysis
MLIRFunc
MLIRIR MLIRIR
MLIRLinalg MLIRLinalg
MLIRMemRef MLIRMemRef
MLIRStandard
) )

mlir/lib/Dialect/Linalg/Analysis/DependenceAnalysis.cpp

//===- DependenceAnalysis.cpp - Dependence analysis on SSA views ----------===// //===- DependenceAnalysis.cpp - Dependence analysis on SSA views ----------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements view-based alias and dependence analyses. // This file implements view-based alias and dependence analyses.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Linalg/Analysis/DependenceAnalysis.h" #include "mlir/Dialect/Linalg/Analysis/DependenceAnalysis.h"
#include "mlir/Dialect/Bufferization/IR/Bufferization.h" #include "mlir/Dialect/Bufferization/IR/Bufferization.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "llvm/Support/CommandLine.h" #include "llvm/Support/CommandLine.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#define DEBUG_TYPE "linalg-dependence-analysis" #define DEBUG_TYPE "linalg-dependence-analysis"
using namespace mlir; using namespace mlir;
▲ Show 20 Lines • Show All 332 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/ComprehensiveBufferize/CMakeLists.txt

Show All 11 Lines
) )
add_mlir_dialect_library(MLIRModuleBufferization add_mlir_dialect_library(MLIRModuleBufferization
ModuleBufferization.cpp ModuleBufferization.cpp
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRBufferization MLIRBufferization
MLIRBufferizationTransforms MLIRBufferizationTransforms
MLIRFunc
MLIRFuncTransforms
MLIRIR MLIRIR
MLIRMemRef MLIRMemRef
MLIRStandard
MLIRStandardOpsTransforms
) )

mlir/lib/Dialect/Linalg/ComprehensiveBufferize/ModuleBufferization.cpp

Show First 20 Lines • Show All 70 Lines • ▼ Show 20 Lines
// as "not reading" and/or "not writing". // as "not reading" and/or "not writing".
#include "mlir/Dialect/Linalg/ComprehensiveBufferize/ModuleBufferization.h" #include "mlir/Dialect/Linalg/ComprehensiveBufferize/ModuleBufferization.h"
#include "mlir/Dialect/Bufferization/IR/BufferizableOpInterface.h" #include "mlir/Dialect/Bufferization/IR/BufferizableOpInterface.h"
#include "mlir/Dialect/Bufferization/IR/Bufferization.h" #include "mlir/Dialect/Bufferization/IR/Bufferization.h"
#include "mlir/Dialect/Bufferization/Transforms/Bufferize.h" #include "mlir/Dialect/Bufferization/Transforms/Bufferize.h"
#include "mlir/Dialect/Bufferization/Transforms/OneShotAnalysis.h" #include "mlir/Dialect/Bufferization/Transforms/OneShotAnalysis.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/Operation.h" #include "mlir/IR/Operation.h"
using namespace mlir; using namespace mlir;
using namespace linalg; using namespace linalg;
using namespace tensor; using namespace tensor;
using namespace comprehensive_bufferize; using namespace comprehensive_bufferize;
using namespace mlir::bufferization; using namespace mlir::bufferization;
Show All 28 Lines
}; };
} // namespace } // namespace
/// Get ModuleBufferizationState. /// Get ModuleBufferizationState.
static const ModuleBufferizationState & static const ModuleBufferizationState &
getModuleBufferizationState(const BufferizationState &state) { getModuleBufferizationState(const BufferizationState &state) {
Optional<const ModuleBufferizationState *> maybeState = Optional<const ModuleBufferizationState *> maybeState =
state.getDialectState<ModuleBufferizationState>( state.getDialectState<ModuleBufferizationState>(
StandardOpsDialect::getDialectNamespace()); func::FuncDialect::getDialectNamespace());
assert(maybeState.hasValue() && "ModuleBufferizationState does not exist"); assert(maybeState.hasValue() && "ModuleBufferizationState does not exist");
return **maybeState; return **maybeState;
} }
/// Get or create ModuleBufferizationState. /// Get or create ModuleBufferizationState.
static ModuleBufferizationState & static ModuleBufferizationState &
getModuleBufferizationState(BufferizationState &state) { getModuleBufferizationState(BufferizationState &state) {
return state.getOrCreateDialectState<ModuleBufferizationState>( return state.getOrCreateDialectState<ModuleBufferizationState>(
StandardOpsDialect::getDialectNamespace()); func::FuncDialect::getDialectNamespace());
} }
/// Return the state (phase) of analysis of the FuncOp. /// Return the state (phase) of analysis of the FuncOp.
static FuncOpAnalysisState static FuncOpAnalysisState
getFuncOpAnalysisState(const BufferizationState &state, FuncOp funcOp) { getFuncOpAnalysisState(const BufferizationState &state, FuncOp funcOp) {
const ModuleBufferizationState &moduleState = const ModuleBufferizationState &moduleState =
getModuleBufferizationState(state); getModuleBufferizationState(state);
auto it = moduleState.analyzedFuncOps.find(funcOp); auto it = moduleState.analyzedFuncOps.find(funcOp);
if (it == moduleState.analyzedFuncOps.end()) if (it == moduleState.analyzedFuncOps.end())
return FuncOpAnalysisState::NotAnalyzed; return FuncOpAnalysisState::NotAnalyzed;
return it->second; return it->second;
} }
/// Return the unique ReturnOp that terminates `funcOp`. /// Return the unique ReturnOp that terminates `funcOp`.
/// Return nullptr if there is no such unique ReturnOp. /// Return nullptr if there is no such unique ReturnOp.
static ReturnOp getAssumedUniqueReturnOp(FuncOp funcOp) { static func::ReturnOp getAssumedUniqueReturnOp(FuncOp funcOp) {
ReturnOp returnOp; func::ReturnOp returnOp;
for (Block &b : funcOp.body()) { for (Block &b : funcOp.body()) {
if (auto candidateOp = dyn_cast<ReturnOp>(b.getTerminator())) { if (auto candidateOp = dyn_cast<func::ReturnOp>(b.getTerminator())) {
if (returnOp) if (returnOp)
return nullptr; return nullptr;
returnOp = candidateOp; returnOp = candidateOp;
} }
} }
return returnOp; return returnOp;
} }
Show All 25 Lines
static LogicalResult static LogicalResult
equivalentFuncOpBBArgsAnalysis(Operation *op, BufferizationState &state, equivalentFuncOpBBArgsAnalysis(Operation *op, BufferizationState &state,
BufferizationAliasInfo &aliasInfo, BufferizationAliasInfo &aliasInfo,
SmallVector<Operation *> &newOps) { SmallVector<Operation *> &newOps) {
ModuleBufferizationState &moduleState = getModuleBufferizationState(state); ModuleBufferizationState &moduleState = getModuleBufferizationState(state);
// Support only single return-terminated block in the function. // Support only single return-terminated block in the function.
auto funcOp = cast<FuncOp>(op); auto funcOp = cast<FuncOp>(op);
ReturnOp returnOp = getAssumedUniqueReturnOp(funcOp); func::ReturnOp returnOp = getAssumedUniqueReturnOp(funcOp);
assert(returnOp && "expected func with single return op"); assert(returnOp && "expected func with single return op");
for (OpOperand &returnVal : returnOp->getOpOperands()) for (OpOperand &returnVal : returnOp->getOpOperands())
if (returnVal.get().getType().isa<RankedTensorType>()) if (returnVal.get().getType().isa<RankedTensorType>())
for (BlockArgument bbArg : funcOp.getArguments()) for (BlockArgument bbArg : funcOp.getArguments())
if (bbArg.getType().isa<RankedTensorType>()) if (bbArg.getType().isa<RankedTensorType>())
if (aliasInfo.areEquivalentBufferizedValues(returnVal.get(), bbArg)) { if (aliasInfo.areEquivalentBufferizedValues(returnVal.get(), bbArg)) {
moduleState moduleState
▲ Show 20 Lines • Show All 141 Lines • ▼ Show 20 Lines
} }
/// Gather equivalence info of CallOps. /// Gather equivalence info of CallOps.
/// Note: This only adds new equivalence info if `funcOp` was already analyzed. /// Note: This only adds new equivalence info if `funcOp` was already analyzed.
// TODO: This does not handle cyclic function call graphs etc. // TODO: This does not handle cyclic function call graphs etc.
static void equivalenceAnalysis(FuncOp funcOp, static void equivalenceAnalysis(FuncOp funcOp,
BufferizationAliasInfo &aliasInfo, BufferizationAliasInfo &aliasInfo,
ModuleBufferizationState &moduleState) { ModuleBufferizationState &moduleState) {
funcOp->walk([&](CallOp callOp) { funcOp->walk([&](func::CallOp callOp) {
FuncOp calledFunction = getCalledFunction(callOp); FuncOp calledFunction = getCalledFunction(callOp);
assert(calledFunction && "could not retrieved called FuncOp"); assert(calledFunction && "could not retrieved called FuncOp");
// No equivalence info available for the called function. // No equivalence info available for the called function.
if (!moduleState.equivalentFuncArgs.count(calledFunction)) if (!moduleState.equivalentFuncArgs.count(calledFunction))
return WalkResult::skip(); return WalkResult::skip();
for (auto it : moduleState.equivalentFuncArgs[calledFunction]) { for (auto it : moduleState.equivalentFuncArgs[calledFunction]) {
▲ Show 20 Lines • Show All 57 Lines • ▼ Show 20 Lines if (funcOp.getBody().empty()) {
FunctionType bufferizedFuncType = getBufferizedFunctionType( FunctionType bufferizedFuncType = getBufferizedFunctionType(
funcOp.getContext(), funcOp.getType().getInputs(), TypeRange{}, funcOp.getContext(), funcOp.getType().getInputs(), TypeRange{},
state.getOptions()); state.getOptions());
funcOp.setType(bufferizedFuncType); funcOp.setType(bufferizedFuncType);
return success(); return success();
} }
// Support only single return-terminated block in the function. // Support only single return-terminated block in the function.
ReturnOp returnOp = getAssumedUniqueReturnOp(funcOp); func::ReturnOp returnOp = getAssumedUniqueReturnOp(funcOp);
assert(returnOp && "expected func with single return op"); assert(returnOp && "expected func with single return op");
// 1. For each FuncOp result, keep track of which inplace argument it reuses. // 1. For each FuncOp result, keep track of which inplace argument it reuses.
SmallVector<Value> returnValues; SmallVector<Value> returnValues;
for (OpOperand &returnOperand : returnOp->getOpOperands()) { for (OpOperand &returnOperand : returnOp->getOpOperands()) {
Value returnVal = returnOperand.get(); Value returnVal = returnOperand.get();
// If not a renturn tensor type just forward it. // If not a renturn tensor type just forward it.
Show All 13 Linesstatic LogicalResult bufferizeFuncOpBoundary(FuncOp funcOp,
} }
// 2. Rewrite the terminator without the inPlace bufferizable values. // 2. Rewrite the terminator without the inPlace bufferizable values.
ValueRange retValues{returnValues}; ValueRange retValues{returnValues};
FunctionType bufferizedFuncType = getBufferizedFunctionType( FunctionType bufferizedFuncType = getBufferizedFunctionType(
funcOp.getContext(), funcOp.getType().getInputs(), retValues.getTypes(), funcOp.getContext(), funcOp.getType().getInputs(), retValues.getTypes(),
state.getOptions()); state.getOptions());
OpBuilder b(returnOp); OpBuilder b(returnOp);
b.create<ReturnOp>(returnOp.getLoc(), returnValues); b.create<func::ReturnOp>(returnOp.getLoc(), returnValues);
returnOp->erase(); returnOp->erase();
// 3. Rewrite the bbArgs. // 3. Rewrite the bbArgs.
// Iterate on the original `numArgs` and replace them in order. // Iterate on the original `numArgs` and replace them in order.
// This guarantees the argument order still matches after the rewrite. // This guarantees the argument order still matches after the rewrite.
Block &frontBlock = funcOp.body().front(); Block &frontBlock = funcOp.body().front();
unsigned numArgs = frontBlock.getNumArguments(); unsigned numArgs = frontBlock.getNumArguments();
for (unsigned idx = 0; idx < numArgs; ++idx) { for (unsigned idx = 0; idx < numArgs; ++idx) {
▲ Show 20 Lines • Show All 57 Lines • ▼ Show 20 LinesgetFuncOpsOrderedByCalls(ModuleOp moduleOp,
DenseMap<FuncOp, DenseSet<Operation *>> &callerMap) { DenseMap<FuncOp, DenseSet<Operation *>> &callerMap) {
// For each FuncOp, the set of functions called by it (i.e. the union of // For each FuncOp, the set of functions called by it (i.e. the union of
// symbols of all nested CallOpInterfaceOp). // symbols of all nested CallOpInterfaceOp).
DenseMap<FuncOp, DenseSet<FuncOp>> calledBy; DenseMap<FuncOp, DenseSet<FuncOp>> calledBy;
// For each FuncOp, the number of CallOpInterface it contains. // For each FuncOp, the number of CallOpInterface it contains.
DenseMap<FuncOp, unsigned> numberCallOpsContainedInFuncOp; DenseMap<FuncOp, unsigned> numberCallOpsContainedInFuncOp;
WalkResult res = moduleOp.walk([&](FuncOp funcOp) -> WalkResult { WalkResult res = moduleOp.walk([&](FuncOp funcOp) -> WalkResult {
if (!funcOp.body().empty()) { if (!funcOp.body().empty()) {
ReturnOp returnOp = getAssumedUniqueReturnOp(funcOp); func::ReturnOp returnOp = getAssumedUniqueReturnOp(funcOp);
if (!returnOp) if (!returnOp)
return funcOp->emitError() return funcOp->emitError()
<< "cannot bufferize a FuncOp with tensors and " << "cannot bufferize a FuncOp with tensors and "
"without a unique ReturnOp"; "without a unique ReturnOp";
} }
numberCallOpsContainedInFuncOp[funcOp] = 0; numberCallOpsContainedInFuncOp[funcOp] = 0;
return funcOp.walk([&](CallOpInterface callOp) -> WalkResult { return funcOp.walk([&](CallOpInterface callOp) -> WalkResult {
// Only support CallOp for now. // Only support CallOp for now.
if (!isa<CallOp>(callOp.getOperation())) if (!isa<func::CallOp>(callOp.getOperation()))
return callOp->emitError() << "expected a CallOp"; return callOp->emitError() << "expected a CallOp";
FuncOp calledFunction = getCalledFunction(callOp); FuncOp calledFunction = getCalledFunction(callOp);
assert(calledFunction && "could not retrieved called FuncOp"); assert(calledFunction && "could not retrieved called FuncOp");
auto it = callerMap.try_emplace(calledFunction, DenseSet<Operation *>{}); auto it = callerMap.try_emplace(calledFunction, DenseSet<Operation *>{});
it.first->getSecond().insert(callOp); it.first->getSecond().insert(callOp);
if (calledBy[calledFunction].count(funcOp) == 0) { if (calledBy[calledFunction].count(funcOp) == 0) {
calledBy[calledFunction].insert(funcOp); calledBy[calledFunction].insert(funcOp);
numberCallOpsContainedInFuncOp[funcOp]++; numberCallOpsContainedInFuncOp[funcOp]++;
▲ Show 20 Lines • Show All 135 Lines • ▼ Show 20 LinesgetEquivalentFuncArgIdx(FuncOp funcOp, const ModuleBufferizationState &state,
if (retValIt == funcOpIt->getSecond().end()) if (retValIt == funcOpIt->getSecond().end())
// Return value has no equivalent bbArg. // Return value has no equivalent bbArg.
return None; return None;
return retValIt->getSecond(); return retValIt->getSecond();
} }
struct CallOpInterface struct CallOpInterface
: public BufferizableOpInterface::ExternalModel<CallOpInterface, CallOp> { : public BufferizableOpInterface::ExternalModel<CallOpInterface,
func::CallOp> {
bool bufferizesToMemoryRead(Operation *op, OpOperand &opOperand, bool bufferizesToMemoryRead(Operation *op, OpOperand &opOperand,
const BufferizationState &state) const { const BufferizationState &state) const {
CallOp callOp = cast<CallOp>(op); func::CallOp callOp = cast<func::CallOp>(op);
FuncOp funcOp = getCalledFunction(callOp); FuncOp funcOp = getCalledFunction(callOp);
assert(funcOp && "expected CallOp to a FuncOp"); assert(funcOp && "expected CallOp to a FuncOp");
const ModuleBufferizationState &moduleState = const ModuleBufferizationState &moduleState =
getModuleBufferizationState(state); getModuleBufferizationState(state);
if (getFuncOpAnalysisState(state, funcOp) != FuncOpAnalysisState::Analyzed) if (getFuncOpAnalysisState(state, funcOp) != FuncOpAnalysisState::Analyzed)
// FuncOp not analyzed yet. Assume that OpOperand is read. // FuncOp not analyzed yet. Assume that OpOperand is read.
return true; return true;
return moduleState.readBbArgs.contains( return moduleState.readBbArgs.contains(
funcOp.getArgument(opOperand.getOperandNumber())); funcOp.getArgument(opOperand.getOperandNumber()));
} }
bool bufferizesToMemoryWrite(Operation *op, OpOperand &opOperand, bool bufferizesToMemoryWrite(Operation *op, OpOperand &opOperand,
const BufferizationState &state) const { const BufferizationState &state) const {
CallOp callOp = cast<CallOp>(op); func::CallOp callOp = cast<func::CallOp>(op);
FuncOp funcOp = getCalledFunction(callOp); FuncOp funcOp = getCalledFunction(callOp);
assert(funcOp && "expected CallOp to a FuncOp"); assert(funcOp && "expected CallOp to a FuncOp");
const ModuleBufferizationState &moduleState = const ModuleBufferizationState &moduleState =
getModuleBufferizationState(state); getModuleBufferizationState(state);
if (getFuncOpAnalysisState(state, funcOp) != FuncOpAnalysisState::Analyzed) if (getFuncOpAnalysisState(state, funcOp) != FuncOpAnalysisState::Analyzed)
// FuncOp not analyzed yet. Assume that OpOperand is written. // FuncOp not analyzed yet. Assume that OpOperand is written.
return true; return true;
return moduleState.writtenBbArgs.contains( return moduleState.writtenBbArgs.contains(
funcOp.getArgument(opOperand.getOperandNumber())); funcOp.getArgument(opOperand.getOperandNumber()));
} }
SmallVector<OpResult> SmallVector<OpResult>
getAliasingOpResult(Operation *op, OpOperand &opOperand, getAliasingOpResult(Operation *op, OpOperand &opOperand,
const BufferizationState &state) const { const BufferizationState &state) const {
CallOp callOp = cast<CallOp>(op); func::CallOp callOp = cast<func::CallOp>(op);
FuncOp funcOp = getCalledFunction(callOp); FuncOp funcOp = getCalledFunction(callOp);
assert(funcOp && "expected CallOp to a FuncOp"); assert(funcOp && "expected CallOp to a FuncOp");
const ModuleBufferizationState &moduleState = const ModuleBufferizationState &moduleState =
getModuleBufferizationState(state); getModuleBufferizationState(state);
SmallVector<OpResult> result; SmallVector<OpResult> result;
for (int64_t resultIdx = 0; resultIdx < callOp->getNumResults(); for (int64_t resultIdx = 0; resultIdx < callOp->getNumResults();
++resultIdx) ++resultIdx)
if (Optional<int64_t> maybeArgNumber = if (Optional<int64_t> maybeArgNumber =
getEquivalentFuncArgIdx(funcOp, moduleState, resultIdx)) getEquivalentFuncArgIdx(funcOp, moduleState, resultIdx))
if (*maybeArgNumber == opOperand.getOperandNumber()) if (*maybeArgNumber == opOperand.getOperandNumber())
result.push_back(callOp->getOpResult(resultIdx)); result.push_back(callOp->getOpResult(resultIdx));
return result; return result;
} }
SmallVector<OpOperand *> SmallVector<OpOperand *>
getAliasingOpOperand(Operation *op, OpResult opResult, getAliasingOpOperand(Operation *op, OpResult opResult,
const BufferizationState &state) const { const BufferizationState &state) const {
CallOp callOp = cast<CallOp>(op); func::CallOp callOp = cast<func::CallOp>(op);
FuncOp funcOp = getCalledFunction(callOp); FuncOp funcOp = getCalledFunction(callOp);
assert(funcOp && "expected CallOp to a FuncOp"); assert(funcOp && "expected CallOp to a FuncOp");
const ModuleBufferizationState &moduleState = const ModuleBufferizationState &moduleState =
getModuleBufferizationState(state); getModuleBufferizationState(state);
// TODO: We should be looking for aliasing block arguments here. The current // TODO: We should be looking for aliasing block arguments here. The current
// condition is actually stronger than neccesary. Once we check for aliasing // condition is actually stronger than neccesary. Once we check for aliasing
// block arguments, we may be multiple. // block arguments, we may be multiple.
Show All 11 Lines BufferRelation bufferRelation(Operation *op, OpResult opResult,
return BufferRelation::Equivalent; return BufferRelation::Equivalent;
} }
/// In a first approximation, all the function arguments of a FuncOp are /// In a first approximation, all the function arguments of a FuncOp are
/// marked inplaceable. For now, it is the responsibility of the `callOp` /// marked inplaceable. For now, it is the responsibility of the `callOp`
/// bufferization to allow FuncOp that are inplaceable to write inPlace. /// bufferization to allow FuncOp that are inplaceable to write inPlace.
LogicalResult bufferize(Operation *op, RewriterBase &rewriter, LogicalResult bufferize(Operation *op, RewriterBase &rewriter,
const BufferizationState &state) const { const BufferizationState &state) const {
CallOp callOp = cast<CallOp>(op); func::CallOp callOp = cast<func::CallOp>(op);
unsigned numResults = callOp.getNumResults(); unsigned numResults = callOp.getNumResults();
unsigned numOperands = callOp->getNumOperands(); unsigned numOperands = callOp->getNumOperands();
FuncOp funcOp = getCalledFunction(callOp); FuncOp funcOp = getCalledFunction(callOp);
assert(isa<CallOp>(callOp.getOperation()) && funcOp && assert(funcOp && "expected CallOp to a FuncOp");
"expected CallOp to a FuncOp");
const ModuleBufferizationState &moduleState = const ModuleBufferizationState &moduleState =
getModuleBufferizationState(state); getModuleBufferizationState(state);
// Result types of the bufferized CallOp. // Result types of the bufferized CallOp.
SmallVector<Type> resultTypes; SmallVector<Type> resultTypes;
// Replacement values for the existing CallOp. These are usually the results // Replacement values for the existing CallOp. These are usually the results
// of the bufferized CallOp, unless a tensor result folds onto an operand. // of the bufferized CallOp, unless a tensor result folds onto an operand.
SmallVector<Value> replacementValues(numResults, Value()); SmallVector<Value> replacementValues(numResults, Value());
▲ Show 20 Lines • Show All 89 Lines • ▼ Show 20 Lines for (OpOperand &opOperand : callOp->getOpOperands()) {
Value castBuffer = rewriter.create<memref::CastOp>(callOp.getLoc(), Value castBuffer = rewriter.create<memref::CastOp>(callOp.getLoc(),
memRefType, buffer); memRefType, buffer);
buffer = castBuffer; buffer = castBuffer;
} }
newOperands[idx] = buffer; newOperands[idx] = buffer;
} }
// 4. Create the new CallOp. // 4. Create the new CallOp.
Operation *newCallOp = rewriter.create<CallOp>( Operation *newCallOp = rewriter.create<func::CallOp>(
callOp.getLoc(), funcOp.sym_name(), resultTypes, newOperands); callOp.getLoc(), funcOp.sym_name(), resultTypes, newOperands);
newCallOp->setAttrs(callOp->getAttrs()); newCallOp->setAttrs(callOp->getAttrs());
// Get replacement values for non-tensor / non-equivalent results. // Get replacement values for non-tensor / non-equivalent results.
for (unsigned i = 0; i < replacementValues.size(); ++i) { for (unsigned i = 0; i < replacementValues.size(); ++i) {
if (replacementValues[i]) if (replacementValues[i])
continue; continue;
replacementValues[i] = newCallOp->getResult(*retValMapping[i]); replacementValues[i] = newCallOp->getResult(*retValMapping[i]);
} }
// 5. Replace the old op with the new op. // 5. Replace the old op with the new op.
replaceOpWithBufferizedValues(rewriter, callOp, replacementValues); replaceOpWithBufferizedValues(rewriter, callOp, replacementValues);
return success(); return success();
} }
}; };
struct ReturnOpInterface struct ReturnOpInterface
: public BufferizableOpInterface::ExternalModel<ReturnOpInterface, : public BufferizableOpInterface::ExternalModel<ReturnOpInterface,
ReturnOp> { func::ReturnOp> {
bool bufferizesToMemoryRead(Operation *op, OpOperand &opOperand, bool bufferizesToMemoryRead(Operation *op, OpOperand &opOperand,
const BufferizationState &state) const { const BufferizationState &state) const {
return true; return true;
} }
bool bufferizesToMemoryWrite(Operation *op, OpOperand &opOperand, bool bufferizesToMemoryWrite(Operation *op, OpOperand &opOperand,
const BufferizationState &state) const { const BufferizationState &state) const {
return false; return false;
} }
SmallVector<OpResult> SmallVector<OpResult>
getAliasingOpResult(Operation *op, OpOperand &opOperand, getAliasingOpResult(Operation *op, OpOperand &opOperand,
const BufferizationState &state) const { const BufferizationState &state) const {
return {}; return {};
} }
LogicalResult bufferize(Operation *op, RewriterBase &rewriter, LogicalResult bufferize(Operation *op, RewriterBase &rewriter,
const BufferizationState &state) const { const BufferizationState &state) const {
#ifndef NDEBUG #ifndef NDEBUG
auto returnOp = cast<ReturnOp>(op); auto returnOp = cast<func::ReturnOp>(op);
assert(isa<FuncOp>(returnOp->getParentOp()) && assert(isa<FuncOp>(returnOp->getParentOp()) &&
"only support FuncOp parent for ReturnOp"); "only support FuncOp parent for ReturnOp");
#endif // NDEBUG #endif // NDEBUG
return failure(); return failure();
} }
}; };
struct FuncOpInterface struct FuncOpInterface
Show All 26 Lines
} // namespace std_ext } // namespace std_ext
} // namespace comprehensive_bufferize } // namespace comprehensive_bufferize
} // namespace linalg } // namespace linalg
} // namespace mlir } // namespace mlir
void mlir::linalg::comprehensive_bufferize::std_ext:: void mlir::linalg::comprehensive_bufferize::std_ext::
registerModuleBufferizationExternalModels(DialectRegistry &registry) { registerModuleBufferizationExternalModels(DialectRegistry &registry) {
registry.addOpInterface<CallOp, std_ext::CallOpInterface>(); registry.addOpInterface<func::CallOp, std_ext::CallOpInterface>();
registry.addOpInterface<ReturnOp, std_ext::ReturnOpInterface>(); registry.addOpInterface<func::ReturnOp, std_ext::ReturnOpInterface>();
registry.addOpInterface<FuncOp, std_ext::FuncOpInterface>(); registry.addOpInterface<FuncOp, std_ext::FuncOpInterface>();
} }
/// Set the attribute that triggers inplace bufferization on a FuncOp argument /// Set the attribute that triggers inplace bufferization on a FuncOp argument
/// `bbArg`. /// `bbArg`.
static void setInPlaceFuncArgument(BlockArgument bbArg, bool inPlace) { static void setInPlaceFuncArgument(BlockArgument bbArg, bool inPlace) {
auto funcOp = cast<FuncOp>(bbArg.getOwner()->getParentOp()); auto funcOp = cast<FuncOp>(bbArg.getOwner()->getParentOp());
funcOp.setArgAttr(bbArg.getArgNumber(), funcOp.setArgAttr(bbArg.getArgNumber(),
▲ Show 20 Lines • Show All 95 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/Transforms/Bufferize.cpp

//===- Bufferize.cpp - Bufferization of linalg ops ------------------===// //===- Bufferize.cpp - Bufferization of linalg ops ------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Arithmetic/Utils/Utils.h" #include "mlir/Dialect/Arithmetic/Utils/Utils.h"
#include "mlir/Dialect/Bufferization/IR/Bufferization.h" #include "mlir/Dialect/Bufferization/IR/Bufferization.h"
#include "mlir/Dialect/Bufferization/Transforms/Bufferize.h" #include "mlir/Dialect/Bufferization/Transforms/Bufferize.h"
#include "mlir/Dialect/Func/Transforms/Passes.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/Linalg/Passes.h" #include "mlir/Dialect/Linalg/Passes.h"
#include "mlir/Dialect/Linalg/Transforms/Transforms.h" #include "mlir/Dialect/Linalg/Transforms/Transforms.h"
#include "mlir/Dialect/Linalg/Utils/Utils.h" #include "mlir/Dialect/Linalg/Utils/Utils.h"
#include "mlir/Dialect/Math/IR/Math.h" #include "mlir/Dialect/Math/IR/Math.h"
#include "mlir/Dialect/StandardOps/Transforms/Passes.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/Dialect/Vector/IR/VectorOps.h" #include "mlir/Dialect/Vector/IR/VectorOps.h"
#include "mlir/IR/BuiltinDialect.h" #include "mlir/IR/BuiltinDialect.h"
#include "mlir/IR/Operation.h" #include "mlir/IR/Operation.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
using namespace ::mlir; using namespace ::mlir;
using namespace ::mlir::linalg; using namespace ::mlir::linalg;
▲ Show 20 Lines • Show All 144 Lines • ▼ Show 20 Lines
/// Converts Linalg operations that work on tensor-type operands or results to /// Converts Linalg operations that work on tensor-type operands or results to
/// work on buffers. /// work on buffers.
struct LinalgBufferizePass : public LinalgBufferizeBase<LinalgBufferizePass> { struct LinalgBufferizePass : public LinalgBufferizeBase<LinalgBufferizePass> {
void runOnOperation() override { void runOnOperation() override {
MLIRContext &context = getContext(); MLIRContext &context = getContext();
ConversionTarget target(context); ConversionTarget target(context);
bufferization::BufferizeTypeConverter typeConverter; bufferization::BufferizeTypeConverter typeConverter;
// Mark all Standard operations legal. // Mark certain operations legal.
target.addLegalDialect<arith::ArithmeticDialect, AffineDialect, target.addLegalDialect<arith::ArithmeticDialect, AffineDialect,
memref::MemRefDialect, StandardOpsDialect, func::FuncDialect, memref::MemRefDialect,
tensor::TensorDialect>(); tensor::TensorDialect>();
target.addIllegalOp<InitTensorOp>(); target.addIllegalOp<InitTensorOp>();
// Mark all Linalg operations illegal as long as they work on tensors. // Mark all Linalg operations illegal as long as they work on tensors.
auto isLegalOperation = [&](Operation *op) { auto isLegalOperation = [&](Operation *op) {
return typeConverter.isLegal(op); return typeConverter.isLegal(op);
}; };
target.addDynamicallyLegalDialect<linalg::LinalgDialect>(isLegalOperation); target.addDynamicallyLegalDialect<linalg::LinalgDialect>(isLegalOperation);
Show All 26 Lines

mlir/lib/Dialect/Linalg/Transforms/CMakeLists.txt

Show All 34 Linesadd_mlir_dialect_library(MLIRLinalgTransforms
MLIRAffine MLIRAffine
MLIRAffineBufferizableOpInterfaceImpl MLIRAffineBufferizableOpInterfaceImpl
MLIRAffineUtils MLIRAffineUtils
MLIRAnalysis MLIRAnalysis
MLIRArithmetic MLIRArithmetic
MLIRArithmeticTransforms MLIRArithmeticTransforms
MLIRBufferization MLIRBufferization
MLIRComplex MLIRComplex
MLIRFunc
MLIRFuncTransforms
MLIRInferTypeOpInterface MLIRInferTypeOpInterface
MLIRIR MLIRIR
MLIRMemRef MLIRMemRef
MLIRLinalg MLIRLinalg
MLIRLinalgAnalysis MLIRLinalgAnalysis
MLIRLinalgUtils MLIRLinalgUtils
MLIRModuleBufferization MLIRModuleBufferization
MLIRSCF MLIRSCF
MLIRSCFTransforms MLIRSCFTransforms
MLIRSCFUtils MLIRSCFUtils
MLIRPass MLIRPass
MLIRSparseTensor MLIRSparseTensor
MLIRStandard
MLIRStandardOpsTransforms
MLIRStandardToLLVM MLIRStandardToLLVM
MLIRTensor MLIRTensor
MLIRTensorTilingInterfaceImpl MLIRTensorTilingInterfaceImpl
MLIRTensorTransforms MLIRTensorTransforms
MLIRTransforms MLIRTransforms
MLIRTransformUtils MLIRTransformUtils
MLIRVector MLIRVector
MLIRVectorTransforms MLIRVectorTransforms
MLIRVectorUtils MLIRVectorUtils
MLIRX86VectorTransforms MLIRX86VectorTransforms
MLIRVectorToSCF MLIRVectorToSCF
) )

mlir/lib/Dialect/Linalg/Transforms/ComprehensiveBufferizePass.cpp

Show First 20 Lines • Show All 43 Lines • ▼ Show 20 Linesstruct LinalgComprehensiveModuleBufferize
void runOnOperation() override; void runOnOperation() override;
void getDependentDialects(DialectRegistry &registry) const override { void getDependentDialects(DialectRegistry &registry) const override {
registry registry
.insert<bufferization::BufferizationDialect, linalg::LinalgDialect, .insert<bufferization::BufferizationDialect, linalg::LinalgDialect,
memref::MemRefDialect, tensor::TensorDialect, memref::MemRefDialect, tensor::TensorDialect,
vector::VectorDialect, scf::SCFDialect, vector::VectorDialect, scf::SCFDialect,
arith::ArithmeticDialect, StandardOpsDialect, AffineDialect>(); arith::ArithmeticDialect, func::FuncDialect, AffineDialect>();
affine_ext::registerBufferizableOpInterfaceExternalModels(registry); affine_ext::registerBufferizableOpInterfaceExternalModels(registry);
arith::registerBufferizableOpInterfaceExternalModels(registry); arith::registerBufferizableOpInterfaceExternalModels(registry);
linalg::registerBufferizableOpInterfaceExternalModels(registry); linalg::registerBufferizableOpInterfaceExternalModels(registry);
scf::registerBufferizableOpInterfaceExternalModels(registry); scf::registerBufferizableOpInterfaceExternalModels(registry);
std_ext::registerModuleBufferizationExternalModels(registry); std_ext::registerModuleBufferizationExternalModels(registry);
tensor::registerBufferizableOpInterfaceExternalModels(registry); tensor::registerBufferizableOpInterfaceExternalModels(registry);
vector::registerBufferizableOpInterfaceExternalModels(registry); vector::registerBufferizableOpInterfaceExternalModels(registry);
} }
▲ Show 20 Lines • Show All 73 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/Transforms/Detensorize.cpp

//===- Detensorize.cpp - Linalg transformations as patterns ----------===// //===- Detensorize.cpp - Linalg transformations as patterns ----------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h" #include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h"
#include "mlir/Dialect/Func/Transforms/FuncConversions.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/Linalg/Passes.h" #include "mlir/Dialect/Linalg/Passes.h"
#include "mlir/Dialect/StandardOps/Transforms/FuncConversions.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/IR/OpDefinition.h" #include "mlir/IR/OpDefinition.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
#include <iterator> #include <iterator>
#include <memory> #include <memory>
#include <utility> #include <utility>
▲ Show 20 Lines • Show All 545 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/Transforms/ElementwiseToLinalg.cpp

//===- ElementwiseToLinalg.cpp - conversion of elementwise to linalg ------===// //===- ElementwiseToLinalg.cpp - conversion of elementwise to linalg ------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Linalg/Passes.h" #include "mlir/Dialect/Linalg/Passes.h"
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Arithmetic/Utils/Utils.h" #include "mlir/Dialect/Arithmetic/Utils/Utils.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/Linalg/Transforms/Transforms.h" #include "mlir/Dialect/Linalg/Transforms/Transforms.h"
#include "mlir/Dialect/Linalg/Utils/Utils.h" #include "mlir/Dialect/Linalg/Utils/Utils.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
using namespace mlir; using namespace mlir;
static bool isElementwiseMappableOpOnRankedTensors(Operation *op) { static bool isElementwiseMappableOpOnRankedTensors(Operation *op) {
if (!OpTrait::hasElementwiseMappableTraits(op)) if (!OpTrait::hasElementwiseMappableTraits(op))
return false; return false;
▲ Show 20 Lines • Show All 124 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/Transforms/HoistPadding.cpp

//===- HoistPadding.cpp - Hoisting for tensor::PadOp ----------------------===// //===- HoistPadding.cpp - Hoisting for tensor::PadOp ----------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements functions concerned with hoisting padding operations. // This file implements functions concerned with hoisting padding operations.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Linalg/Transforms/HoistPadding.h" #include "mlir/Dialect/Linalg/Transforms/HoistPadding.h"
#include "mlir/Analysis/SliceAnalysis.h" #include "mlir/Analysis/SliceAnalysis.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/Linalg/Transforms/Transforms.h" #include "mlir/Dialect/Linalg/Transforms/Transforms.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/SCF/Utils/Utils.h" #include "mlir/Dialect/SCF/Utils/Utils.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/Dialect/Vector/IR/VectorOps.h" #include "mlir/Dialect/Vector/IR/VectorOps.h"
#include "mlir/Dialect/Vector/Utils/VectorUtils.h" #include "mlir/Dialect/Vector/Utils/VectorUtils.h"
#include "mlir/IR/AsmState.h" #include "mlir/IR/AsmState.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/Dominance.h" #include "mlir/IR/Dominance.h"
#include "llvm/ADT/StringRef.h" #include "llvm/ADT/StringRef.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
▲ Show 20 Lines • Show All 335 Lines • ▼ Show 20 Lines Value res = b.createOrFold<AffineApplyOp>(
cast<scf::ForOp>(forOp).getStep()}); cast<scf::ForOp>(forOp).getStep()});
dynamicTensorSizes.push_back(res); dynamicTensorSizes.push_back(res);
} }
return dynamicTensorSizes; return dynamicTensorSizes;
} }
static bool isDefinedOutsideOrConstant(scf::ForOp outer, Value v) { static bool isDefinedOutsideOrConstant(scf::ForOp outer, Value v) {
return outer.isDefinedOutsideOfLoop(v) || v.getDefiningOp<ConstantOp>(); return outer.isDefinedOutsideOfLoop(v) || v.getDefiningOp<func::ConstantOp>();
} }
/// Return the current iteration number in the loop (iv - lb).ceilDiv(step). /// Return the current iteration number in the loop (iv - lb).ceilDiv(step).
/// The returned Value is guaranteed not to depend on any loop comprised in /// The returned Value is guaranteed not to depend on any loop comprised in
/// [`outer`, `forOp`]. /// [`outer`, `forOp`].
/// Return null if such a loop-independent quantity cannot be computed. /// Return null if such a loop-independent quantity cannot be computed.
static Value buildLoopIterationCount(OpBuilder &b, scf::ForOp outer, static Value buildLoopIterationCount(OpBuilder &b, scf::ForOp outer,
scf::ForOp forOp) { scf::ForOp forOp) {
▲ Show 20 Lines • Show All 176 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/Transforms/Hoisting.cpp

Show All 9 Lines
// in the context of Linalg transformations. // in the context of Linalg transformations.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Linalg/Transforms/Hoisting.h" #include "mlir/Dialect/Linalg/Transforms/Hoisting.h"
#include "mlir/Analysis/SliceAnalysis.h" #include "mlir/Analysis/SliceAnalysis.h"
#include "mlir/Dialect/Affine/Analysis/AffineStructures.h" #include "mlir/Dialect/Affine/Analysis/AffineStructures.h"
#include "mlir/Dialect/Affine/IR/AffineValueMap.h" #include "mlir/Dialect/Affine/IR/AffineValueMap.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/Linalg/Transforms/Transforms.h" #include "mlir/Dialect/Linalg/Transforms/Transforms.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/SCF/Utils/Utils.h" #include "mlir/Dialect/SCF/Utils/Utils.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/Dialect/Vector/IR/VectorOps.h" #include "mlir/Dialect/Vector/IR/VectorOps.h"
#include "mlir/Dialect/Vector/Utils/VectorUtils.h" #include "mlir/Dialect/Vector/Utils/VectorUtils.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/Dominance.h" #include "mlir/IR/Dominance.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
#include "llvm/ADT/StringRef.h" #include "llvm/ADT/StringRef.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
▲ Show 20 Lines • Show All 481 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/Transforms/Vectorization.cpp

Show First 20 Lines • Show All 359 Lines • ▼ Show 20 Lines for (auto &customFunc : customVectorizationHooks) {
if (result.status == VectorizationStatus::Failure) if (result.status == VectorizationStatus::Failure)
continue; continue;
return result; return result;
} }
} }
// 2. Constant ops don't get vectorized but rather broadcasted at their users. // 2. Constant ops don't get vectorized but rather broadcasted at their users.
// Clone so that the constant is not confined to the linalgOp block . // Clone so that the constant is not confined to the linalgOp block .
if (isa<arith::ConstantOp, ConstantOp>(op)) if (isa<arith::ConstantOp, func::ConstantOp>(op))
return VectorizationResult{VectorizationStatus::NewOp, b.clone(*op)}; return VectorizationResult{VectorizationStatus::NewOp, b.clone(*op)};
// 3. Only ElementwiseMappable are allowed in the generic vectorization. // 3. Only ElementwiseMappable are allowed in the generic vectorization.
if (!OpTrait::hasElementwiseMappableTraits(op)) if (!OpTrait::hasElementwiseMappableTraits(op))
return VectorizationResult{VectorizationStatus::Failure, nullptr}; return VectorizationResult{VectorizationStatus::Failure, nullptr};
// 4 . Check if the operation is a reduction. // 4 . Check if the operation is a reduction.
SmallVector<std::pair<Value, Value>> reductionOperands; SmallVector<std::pair<Value, Value>> reductionOperands;
▲ Show 20 Lines • Show All 46 Lines • ▼ Show 20 Lines return VectorizationResult{
llvm::to_vector<4>(returnTypes))}; llvm::to_vector<4>(returnTypes))};
} }
/// Detect whether `r` has only ConstantOp, ElementwiseMappable and YieldOp. /// Detect whether `r` has only ConstantOp, ElementwiseMappable and YieldOp.
static bool hasOnlyScalarElementwiseOp(Region &r) { static bool hasOnlyScalarElementwiseOp(Region &r) {
if (!llvm::hasSingleElement(r)) if (!llvm::hasSingleElement(r))
return false; return false;
for (Operation &op : r.front()) { for (Operation &op : r.front()) {
if (!(isa<arith::ConstantOp, ConstantOp, linalg::YieldOp, linalg::IndexOp>( if (!(isa<arith::ConstantOp, func::ConstantOp, linalg::YieldOp,
op) || linalg::IndexOp>(op) ||
OpTrait::hasElementwiseMappableTraits(&op)) || OpTrait::hasElementwiseMappableTraits(&op)) ||
llvm::any_of(op.getResultTypes(), llvm::any_of(op.getResultTypes(),
[](Type type) { return !type.isIntOrIndexOrFloat(); })) [](Type type) { return !type.isIntOrIndexOrFloat(); }))
return false; return false;
} }
return true; return true;
} }
▲ Show 20 Lines • Show All 1,274 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/Utils/CMakeLists.txt

add_mlir_dialect_library(MLIRLinalgUtils add_mlir_dialect_library(MLIRLinalgUtils
Utils.cpp Utils.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Linalg ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Linalg
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRAffine MLIRAffine
MLIRAffineAnalysis MLIRAffineAnalysis
MLIRAffineUtils MLIRAffineUtils
MLIRArithmetic MLIRArithmetic
MLIRFunc
MLIRIR MLIRIR
MLIRLinalg MLIRLinalg
MLIRSCF MLIRSCF
MLIRPass MLIRPass
MLIRStandard
MLIRTensorUtils MLIRTensorUtils
MLIRTransformUtils MLIRTransformUtils
) )

mlir/lib/Dialect/Linalg/Utils/Utils.cpp

Show All 13 Lines
#include "mlir/Analysis/SliceAnalysis.h" #include "mlir/Analysis/SliceAnalysis.h"
#include "mlir/Dialect/Affine/Analysis/AffineStructures.h" #include "mlir/Dialect/Affine/Analysis/AffineStructures.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Affine/IR/AffineValueMap.h" #include "mlir/Dialect/Affine/IR/AffineValueMap.h"
#include "mlir/Dialect/Affine/LoopUtils.h" #include "mlir/Dialect/Affine/LoopUtils.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Arithmetic/Utils/Utils.h" #include "mlir/Dialect/Arithmetic/Utils/Utils.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/Dialect/Tensor/Utils/Utils.h" #include "mlir/Dialect/Tensor/Utils/Utils.h"
#include "mlir/Dialect/Utils/StaticValueUtils.h" #include "mlir/Dialect/Utils/StaticValueUtils.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/AffineExprVisitor.h" #include "mlir/IR/AffineExprVisitor.h"
#include "mlir/IR/AffineMap.h" #include "mlir/IR/AffineMap.h"
#include "mlir/IR/Matchers.h" #include "mlir/IR/Matchers.h"
#include "mlir/IR/OpImplementation.h" #include "mlir/IR/OpImplementation.h"
▲ Show 20 Lines • Show All 953 Lines • Show Last 20 Lines

mlir/lib/Dialect/Math/Transforms/CMakeLists.txt

add_mlir_dialect_library(MLIRMathTransforms add_mlir_dialect_library(MLIRMathTransforms
AlgebraicSimplification.cpp AlgebraicSimplification.cpp
ExpandTanh.cpp ExpandTanh.cpp
PolynomialApproximation.cpp PolynomialApproximation.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Math/Transforms ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Math/Transforms
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRArithmetic MLIRArithmetic
MLIRDialectUtils MLIRDialectUtils
MLIRFunc
MLIRIR MLIRIR
MLIRMath MLIRMath
MLIRPass MLIRPass
MLIRStandard
MLIRTransforms MLIRTransforms
MLIRX86Vector MLIRX86Vector
MLIRVector MLIRVector
MLIRVectorUtils MLIRVectorUtils
) )

mlir/lib/Dialect/Math/Transforms/ExpandTanh.cpp

//===- ExpandTanh.cpp - Code to perform expanding tanh op -----------------===// //===- ExpandTanh.cpp - Code to perform expanding tanh op -----------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements expansion of tanh op. // This file implements expansion of tanh op.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Math/IR/Math.h" #include "mlir/Dialect/Math/IR/Math.h"
#include "mlir/Dialect/Math/Transforms/Passes.h" #include "mlir/Dialect/Math/Transforms/Passes.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
using namespace mlir; using namespace mlir;
/// Expands tanh op into /// Expands tanh op into
/// 1) 1-exp^{-2x} / 1+exp^{-2x}, if x => 0 /// 1) 1-exp^{-2x} / 1+exp^{-2x}, if x => 0
/// 2) exp^{2x}-1 / exp^{2x}+1 , if x < 0 /// 2) exp^{2x}-1 / exp^{2x}+1 , if x < 0
Show All 35 Lines

mlir/lib/Dialect/MemRef/IR/CMakeLists.txt

add_mlir_dialect_library(MLIRMemRef add_mlir_dialect_library(MLIRMemRef
MemRefDialect.cpp MemRefDialect.cpp
MemRefOps.cpp MemRefOps.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${PROJECT_SOURCE_DIR}/inlude/mlir/Dialect/MemRefDialect ${PROJECT_SOURCE_DIR}/inlude/mlir/Dialect/MemRefDialect
DEPENDS DEPENDS
MLIRStandardOpsIncGen
MLIRMemRefOpsIncGen MLIRMemRefOpsIncGen
LINK_COMPONENTS LINK_COMPONENTS
Core Core
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRArithmetic MLIRArithmetic
MLIRArithmeticUtils MLIRArithmeticUtils
MLIRControlFlowInterfaces MLIRControlFlowInterfaces
MLIRDialect MLIRDialect
MLIRDialectUtils MLIRDialectUtils
MLIRInferTypeOpInterface MLIRInferTypeOpInterface
MLIRIR MLIRIR
MLIRMemRefUtils MLIRMemRefUtils
MLIRViewLikeInterface MLIRViewLikeInterface
) )

mlir/lib/Dialect/MemRef/Transforms/CMakeLists.txt

Show All 10 Linesadd_mlir_dialect_library(MLIRMemRefTransforms
DEPENDS DEPENDS
MLIRMemRefPassIncGen MLIRMemRefPassIncGen
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRAffine MLIRAffine
MLIRAffineUtils MLIRAffineUtils
MLIRArithmetic MLIRArithmetic
MLIRFunc
MLIRInferTypeOpInterface MLIRInferTypeOpInterface
MLIRLoopLikeInterface MLIRLoopLikeInterface
MLIRMemRef MLIRMemRef
MLIRPass MLIRPass
MLIRStandard
MLIRTensor MLIRTensor
MLIRTransforms MLIRTransforms
MLIRVector MLIRVector
) )

mlir/lib/Dialect/MemRef/Transforms/ExpandOps.cpp

Show All 10 Lines
// for Signed Integers. // for Signed Integers.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Arithmetic/Transforms/Passes.h" #include "mlir/Dialect/Arithmetic/Transforms/Passes.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/MemRef/Transforms/Passes.h" #include "mlir/Dialect/MemRef/Transforms/Passes.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/TypeUtilities.h" #include "mlir/IR/TypeUtilities.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
using namespace mlir; using namespace mlir;
namespace { namespace {
/// Converts `atomic_rmw` that cannot be lowered to a simple atomic op with /// Converts `atomic_rmw` that cannot be lowered to a simple atomic op with
▲ Show 20 Lines • Show All 94 Lines • ▼ Show 20 Linesstruct ExpandOpsPass : public ExpandOpsBase<ExpandOpsPass> {
void runOnOperation() override { void runOnOperation() override {
MLIRContext &ctx = getContext(); MLIRContext &ctx = getContext();
RewritePatternSet patterns(&ctx); RewritePatternSet patterns(&ctx);
memref::populateExpandOpsPatterns(patterns); memref::populateExpandOpsPatterns(patterns);
ConversionTarget target(ctx); ConversionTarget target(ctx);
target.addLegalDialect<arith::ArithmeticDialect, memref::MemRefDialect, target.addLegalDialect<arith::ArithmeticDialect, memref::MemRefDialect,
StandardOpsDialect>(); func::FuncDialect>();
target.addDynamicallyLegalOp<memref::AtomicRMWOp>( target.addDynamicallyLegalOp<memref::AtomicRMWOp>(
[](memref::AtomicRMWOp op) { [](memref::AtomicRMWOp op) {
return op.kind() != arith::AtomicRMWKind::maxf && return op.kind() != arith::AtomicRMWKind::maxf &&
op.kind() != arith::AtomicRMWKind::minf; op.kind() != arith::AtomicRMWKind::minf;
}); });
target.addDynamicallyLegalOp<memref::ReshapeOp>([](memref::ReshapeOp op) { target.addDynamicallyLegalOp<memref::ReshapeOp>([](memref::ReshapeOp op) {
return !op.shape().getType().cast<MemRefType>().hasStaticShape(); return !op.shape().getType().cast<MemRefType>().hasStaticShape();
}); });
Show All 16 Lines

mlir/lib/Dialect/MemRef/Transforms/NormalizeMemRefs.cpp

Show First 20 Lines • Show All 119 Lines • ▼ Show 20 Lines for (FuncOp &funcOp : normalizableFuncs) {
setCalleesAndCallersNonNormalizable(funcOp, moduleOp, setCalleesAndCallersNonNormalizable(funcOp, moduleOp,
normalizableFuncs); normalizableFuncs);
break; break;
} }
} }
} }
// Functions called by this function. // Functions called by this function.
funcOp.walk([&](CallOp callOp) { funcOp.walk([&](func::CallOp callOp) {
StringAttr callee = callOp.getCalleeAttr().getAttr(); StringAttr callee = callOp.getCalleeAttr().getAttr();
for (FuncOp &funcOp : normalizableFuncs) { for (FuncOp &funcOp : normalizableFuncs) {
// We compare FuncOp and callee's name. // We compare FuncOp and callee's name.
if (callee == funcOp.getNameAttr()) { if (callee == funcOp.getNameAttr()) {
setCalleesAndCallersNonNormalizable(funcOp, moduleOp, setCalleesAndCallersNonNormalizable(funcOp, moduleOp,
normalizableFuncs); normalizableFuncs);
break; break;
} }
Show All 19 Lines if (funcOp
if (!isMemRefNormalizable(oldMemRef.getUsers())) if (!isMemRefNormalizable(oldMemRef.getUsers()))
return WalkResult::interrupt(); return WalkResult::interrupt();
return WalkResult::advance(); return WalkResult::advance();
}) })
.wasInterrupted()) .wasInterrupted())
return false; return false;
if (funcOp if (funcOp
.walk([&](CallOp callOp) -> WalkResult { .walk([&](func::CallOp callOp) -> WalkResult {
for (unsigned resIndex : for (unsigned resIndex :
llvm::seq<unsigned>(0, callOp.getNumResults())) { llvm::seq<unsigned>(0, callOp.getNumResults())) {
Value oldMemRef = callOp.getResult(resIndex); Value oldMemRef = callOp.getResult(resIndex);
if (oldMemRef.getType().isa<MemRefType>()) if (oldMemRef.getType().isa<MemRefType>())
if (!isMemRefNormalizable(oldMemRef.getUsers())) if (!isMemRefNormalizable(oldMemRef.getUsers()))
return WalkResult::interrupt(); return WalkResult::interrupt();
} }
return WalkResult::advance(); return WalkResult::advance();
Show All 28 Linesvoid NormalizeMemRefs::updateFunctionSignature(FuncOp funcOp,
// 'normalizeFuncOpMemRefs()'. // 'normalizeFuncOpMemRefs()'.
if (!funcOp.isExternal()) { if (!funcOp.isExternal()) {
SmallVector<Type, 8> argTypes; SmallVector<Type, 8> argTypes;
for (const auto &argEn : llvm::enumerate(funcOp.getArguments())) for (const auto &argEn : llvm::enumerate(funcOp.getArguments()))
argTypes.push_back(argEn.value().getType()); argTypes.push_back(argEn.value().getType());
// Traverse ReturnOps to check if an update to the return type in the // Traverse ReturnOps to check if an update to the return type in the
// function signature is required. // function signature is required.
funcOp.walk([&](ReturnOp returnOp) { funcOp.walk([&](func::ReturnOp returnOp) {
for (const auto &operandEn : llvm::enumerate(returnOp.getOperands())) { for (const auto &operandEn : llvm::enumerate(returnOp.getOperands())) {
Type opType = operandEn.value().getType(); Type opType = operandEn.value().getType();
MemRefType memrefType = opType.dyn_cast<MemRefType>(); MemRefType memrefType = opType.dyn_cast<MemRefType>();
// If type is not memref or if the memref type is same as that in // If type is not memref or if the memref type is same as that in
// function's return signature then no update is required. // function's return signature then no update is required.
if (!memrefType || memrefType == resultTypes[operandEn.index()]) if (!memrefType || memrefType == resultTypes[operandEn.index()])
continue; continue;
// Update function's return type signature. // Update function's return type signature.
Show All 26 Linesvoid NormalizeMemRefs::updateFunctionSignature(FuncOp funcOp,
Optional<SymbolTable::UseRange> symbolUses = funcOp.getSymbolUses(moduleOp); Optional<SymbolTable::UseRange> symbolUses = funcOp.getSymbolUses(moduleOp);
for (SymbolTable::SymbolUse symbolUse : *symbolUses) { for (SymbolTable::SymbolUse symbolUse : *symbolUses) {
Operation *userOp = symbolUse.getUser(); Operation *userOp = symbolUse.getUser();
OpBuilder builder(userOp); OpBuilder builder(userOp);
// When `userOp` can not be casted to `CallOp`, it is skipped. This assumes // When `userOp` can not be casted to `CallOp`, it is skipped. This assumes
// that the non-CallOp has no memrefs to be replaced. // that the non-CallOp has no memrefs to be replaced.
// TODO: Handle cases where a non-CallOp symbol use of a function deals with // TODO: Handle cases where a non-CallOp symbol use of a function deals with
// memrefs. // memrefs.
auto callOp = dyn_cast<CallOp>(userOp); auto callOp = dyn_cast<func::CallOp>(userOp);
if (!callOp) if (!callOp)
continue; continue;
Operation *newCallOp = Operation *newCallOp =
builder.create<CallOp>(userOp->getLoc(), callOp.getCalleeAttr(), builder.create<func::CallOp>(userOp->getLoc(), callOp.getCalleeAttr(),
resultTypes, userOp->getOperands()); resultTypes, userOp->getOperands());
bool replacingMemRefUsesFailed = false; bool replacingMemRefUsesFailed = false;
bool returnTypeChanged = false; bool returnTypeChanged = false;
for (unsigned resIndex : llvm::seq<unsigned>(0, userOp->getNumResults())) { for (unsigned resIndex : llvm::seq<unsigned>(0, userOp->getNumResults())) {
OpResult oldResult = userOp->getResult(resIndex); OpResult oldResult = userOp->getResult(resIndex);
OpResult newResult = newCallOp->getResult(resIndex); OpResult newResult = newCallOp->getResult(resIndex);
// This condition ensures that if the result is not of type memref or if // This condition ensures that if the result is not of type memref or if
// the resulting memref was already having a trivial map layout then we // the resulting memref was already having a trivial map layout then we
// need not perform any use replacement here. // need not perform any use replacement here.
▲ Show 20 Lines • Show All 120 Lines • ▼ Show 20 Linesvoid NormalizeMemRefs::normalizeFuncOpMemRefs(FuncOp funcOp,
// Walk over normalizable operations to normalize memrefs of the operation // Walk over normalizable operations to normalize memrefs of the operation
// results. When `op` has memrefs with affine map in the operation results, // results. When `op` has memrefs with affine map in the operation results,
// new operation containin normalized memrefs is created. Then, the memrefs // new operation containin normalized memrefs is created. Then, the memrefs
// are replaced. `CallOp` is skipped here because it is handled in // are replaced. `CallOp` is skipped here because it is handled in
// `updateFunctionSignature()`. // `updateFunctionSignature()`.
funcOp.walk([&](Operation *op) { funcOp.walk([&](Operation *op) {
if (op->hasTrait<OpTrait::MemRefsNormalizable>() && if (op->hasTrait<OpTrait::MemRefsNormalizable>() &&
op->getNumResults() > 0 && !isa<CallOp>(op) && !funcOp.isExternal()) { op->getNumResults() > 0 && !isa<func::CallOp>(op) &&
!funcOp.isExternal()) {
// Create newOp containing normalized memref in the operation result. // Create newOp containing normalized memref in the operation result.
Operation *newOp = createOpResultsNormalized(funcOp, op); Operation *newOp = createOpResultsNormalized(funcOp, op);
// When all of the operation results have no memrefs or memrefs without // When all of the operation results have no memrefs or memrefs without
// affine map, `newOp` is the same with `op` and following process is // affine map, `newOp` is the same with `op` and following process is
// skipped. // skipped.
if (op != newOp) { if (op != newOp) {
bool replacingMemRefUsesFailed = false; bool replacingMemRefUsesFailed = false;
for (unsigned resIndex : llvm::seq<unsigned>(0, op->getNumResults())) { for (unsigned resIndex : llvm::seq<unsigned>(0, op->getNumResults())) {
▲ Show 20 Lines • Show All 119 Lines • Show Last 20 Lines

mlir/lib/Dialect/MemRef/Transforms/PassDetail.h

//===- PassDetail.h - MemRef Pass class details -----------------*- C++ -*-===// //===- PassDetail.h - MemRef Pass class details -----------------*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef DIALECT_MEMREF_TRANSFORMS_PASSDETAIL_H_ #ifndef DIALECT_MEMREF_TRANSFORMS_PASSDETAIL_H_
#define DIALECT_MEMREF_TRANSFORMS_PASSDETAIL_H_ #define DIALECT_MEMREF_TRANSFORMS_PASSDETAIL_H_
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
namespace mlir { namespace mlir {
class AffineDialect; class AffineDialect;
class StandardOpsDialect;
// Forward declaration from Dialect.h // Forward declaration from Dialect.h
template <typename ConcreteDialect> template <typename ConcreteDialect>
void registerDialect(DialectRegistry &registry); void registerDialect(DialectRegistry &registry);
namespace arith { namespace arith {
class ArithmeticDialect; class ArithmeticDialect;
} // namespace arith } // namespace arith
namespace func {
class FuncDialect;
} // namespace func
namespace memref { namespace memref {
class MemRefDialect; class MemRefDialect;
} // namespace memref } // namespace memref
namespace tensor { namespace tensor {
class TensorDialect; class TensorDialect;
} // namespace tensor } // namespace tensor
Show All 10 Lines

mlir/lib/Dialect/SCF/CMakeLists.txt

add_mlir_dialect_library(MLIRSCF add_mlir_dialect_library(MLIRSCF
SCF.cpp SCF.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/LoopOps ${MLIR_MAIN_INCLUDE_DIR}/mlir/LoopOps
DEPENDS DEPENDS
MLIRSCFOpsIncGen MLIRSCFOpsIncGen
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRArithmetic MLIRArithmetic
MLIRBufferization MLIRBufferization
MLIRFunc
MLIRIR MLIRIR
MLIRLoopLikeInterface MLIRLoopLikeInterface
MLIRSideEffectInterfaces MLIRSideEffectInterfaces
MLIRStandard
) )
add_subdirectory(Transforms) add_subdirectory(Transforms)
add_subdirectory(Utils) add_subdirectory(Utils)

mlir/lib/Dialect/SCF/SCF.cpp

//===- SCF.cpp - Structured Control Flow Operations -----------------------===// //===- SCF.cpp - Structured Control Flow Operations -----------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Bufferization/IR/Bufferization.h" #include "mlir/Dialect/Bufferization/IR/Bufferization.h"
#include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h" #include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/Matchers.h" #include "mlir/IR/Matchers.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Support/MathExtras.h" #include "mlir/Support/MathExtras.h"
#include "mlir/Transforms/InliningUtils.h" #include "mlir/Transforms/InliningUtils.h"
using namespace mlir; using namespace mlir;
using namespace mlir::scf; using namespace mlir::scf;
▲ Show 20 Lines • Show All 2,848 Lines • Show Last 20 Lines

mlir/lib/Dialect/SCF/Transforms/Bufferize.cpp

//===- Bufferize.cpp - scf bufferize pass ---------------------------------===// //===- Bufferize.cpp - scf bufferize pass ---------------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Bufferization/Transforms/Bufferize.h" #include "mlir/Dialect/Bufferization/Transforms/Bufferize.h"
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Bufferization/IR/Bufferization.h" #include "mlir/Dialect/Bufferization/IR/Bufferization.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/SCF/Passes.h" #include "mlir/Dialect/SCF/Passes.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/SCF/Transforms.h" #include "mlir/Dialect/SCF/Transforms.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
using namespace mlir; using namespace mlir;
using namespace mlir::scf; using namespace mlir::scf;
namespace { namespace {
struct SCFBufferizePass : public SCFBufferizeBase<SCFBufferizePass> { struct SCFBufferizePass : public SCFBufferizeBase<SCFBufferizePass> {
void runOnOperation() override { void runOnOperation() override {
Show All 19 Lines

mlir/lib/Dialect/SCF/Transforms/CMakeLists.txt

Show All 18 Linesadd_mlir_dialect_library(MLIRSCFTransforms
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRAffine MLIRAffine
MLIRAffineAnalysis MLIRAffineAnalysis
MLIRArithmetic MLIRArithmetic
MLIRBufferization MLIRBufferization
MLIRBufferizationTransforms MLIRBufferizationTransforms
MLIRDialectUtils MLIRDialectUtils
MLIRFunc
MLIRIR MLIRIR
MLIRMemRef MLIRMemRef
MLIRPass MLIRPass
MLIRSCF MLIRSCF
MLIRSCFUtils MLIRSCFUtils
MLIRStandard
MLIRSupport MLIRSupport
MLIRTensor MLIRTensor
MLIRTransforms MLIRTransforms
MLIRTransformUtils MLIRTransformUtils
) )

mlir/lib/Dialect/SCF/Transforms/ForToWhile.cpp

//===- ForToWhile.cpp - scf.for to scf.while loop conversion --------------===// //===- ForToWhile.cpp - scf.for to scf.while loop conversion --------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// Transforms SCF.ForOp's into SCF.WhileOp's. // Transforms SCF.ForOp's into SCF.WhileOp's.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/SCF/Passes.h" #include "mlir/Dialect/SCF/Passes.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/SCF/Transforms.h" #include "mlir/Dialect/SCF/Transforms.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
using namespace llvm; using namespace llvm;
using namespace mlir; using namespace mlir;
using scf::ForOp; using scf::ForOp;
using scf::WhileOp; using scf::WhileOp;
▲ Show 20 Lines • Show All 89 Lines • Show Last 20 Lines

mlir/lib/Dialect/SCF/Transforms/LoopPipelining.cpp

//===- LoopPipelining.cpp - Code to perform loop software pipelining-------===// //===- LoopPipelining.cpp - Code to perform loop software pipelining-------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements loop software pipelining // This file implements loop software pipelining
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/SCF/Transforms.h" #include "mlir/Dialect/SCF/Transforms.h"
#include "mlir/Dialect/SCF/Utils/Utils.h" #include "mlir/Dialect/SCF/Utils/Utils.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Support/MathExtras.h" #include "mlir/Support/MathExtras.h"
using namespace mlir; using namespace mlir;
using namespace mlir::scf; using namespace mlir::scf;
namespace { namespace {
▲ Show 20 Lines • Show All 455 Lines • Show Last 20 Lines

mlir/lib/Dialect/SCF/Transforms/LoopRangeFolding.cpp

//===- LoopRangeFolding.cpp - Code to perform loop range folding-----------===// //===- LoopRangeFolding.cpp - Code to perform loop range folding-----------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements loop range folding. // This file implements loop range folding.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/SCF/Passes.h" #include "mlir/Dialect/SCF/Passes.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/SCF/Transforms.h" #include "mlir/Dialect/SCF/Transforms.h"
#include "mlir/Dialect/SCF/Utils/Utils.h" #include "mlir/Dialect/SCF/Utils/Utils.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
using namespace mlir; using namespace mlir;
using namespace mlir::scf; using namespace mlir::scf;
namespace { namespace {
struct ForLoopRangeFolding struct ForLoopRangeFolding
: public SCFForLoopRangeFoldingBase<ForLoopRangeFolding> { : public SCFForLoopRangeFoldingBase<ForLoopRangeFolding> {
▲ Show 20 Lines • Show All 60 Lines • Show Last 20 Lines

mlir/lib/Dialect/SCF/Transforms/LoopSpecialization.cpp

Show All 9 Lines
// vectorization. // vectorization.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Affine/Analysis/AffineStructures.h" #include "mlir/Dialect/Affine/Analysis/AffineStructures.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/SCF/Passes.h" #include "mlir/Dialect/SCF/Passes.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/SCF/Transforms.h" #include "mlir/Dialect/SCF/Transforms.h"
#include "mlir/Dialect/SCF/Utils/AffineCanonicalizationUtils.h" #include "mlir/Dialect/SCF/Utils/AffineCanonicalizationUtils.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Utils/StaticValueUtils.h" #include "mlir/Dialect/Utils/StaticValueUtils.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
#include "llvm/ADT/DenseMap.h" #include "llvm/ADT/DenseMap.h"
using namespace mlir; using namespace mlir;
▲ Show 20 Lines • Show All 250 Lines • Show Last 20 Lines

mlir/lib/Dialect/SCF/Transforms/ParallelLoopFusion.cpp

//===- ParallelLoopFusion.cpp - Code to perform loop fusion ---------------===// //===- ParallelLoopFusion.cpp - Code to perform loop fusion ---------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements loop fusion on parallel loops. // This file implements loop fusion on parallel loops.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/SCF/Passes.h" #include "mlir/Dialect/SCF/Passes.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/SCF/Transforms.h" #include "mlir/Dialect/SCF/Transforms.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/OpDefinition.h" #include "mlir/IR/OpDefinition.h"
using namespace mlir; using namespace mlir;
using namespace mlir::scf; using namespace mlir::scf;
/// Verify there are no nested ParallelOps. /// Verify there are no nested ParallelOps.
▲ Show 20 Lines • Show All 152 Lines • Show Last 20 Lines

mlir/lib/Dialect/SCF/Transforms/ParallelLoopTiling.cpp

//===- ParallelLoopTiling.cpp - Tiles scf.parallel ---------------===// //===- ParallelLoopTiling.cpp - Tiles scf.parallel ---------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements loop tiling on parallel loops. // This file implements loop tiling on parallel loops.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/SCF/Passes.h" #include "mlir/Dialect/SCF/Passes.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/SCF/Transforms.h" #include "mlir/Dialect/SCF/Transforms.h"
#include "mlir/Dialect/SCF/Utils/Utils.h" #include "mlir/Dialect/SCF/Utils/Utils.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
using namespace mlir; using namespace mlir;
using namespace mlir::scf; using namespace mlir::scf;
/// Tile a parallel loop of the form /// Tile a parallel loop of the form
/// scf.parallel (%i0, %i1) = (%arg0, %arg1) to (%arg2, %arg3) /// scf.parallel (%i0, %i1) = (%arg0, %arg1) to (%arg2, %arg3)
/// step (%arg4, %arg5) /// step (%arg4, %arg5)
/// ///
▲ Show 20 Lines • Show All 186 Lines • Show Last 20 Lines

mlir/lib/Dialect/SCF/Transforms/StructuralTypeConversions.cpp

//===- StructuralTypeConversions.cpp - scf structural type conversions ----===// //===- StructuralTypeConversions.cpp - scf structural type conversions ----===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/SCF/Passes.h" #include "mlir/Dialect/SCF/Passes.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/SCF/Transforms.h" #include "mlir/Dialect/SCF/Transforms.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
using namespace mlir; using namespace mlir;
using namespace mlir::scf; using namespace mlir::scf;
namespace { namespace {
class ConvertForOpTypes : public OpConversionPattern<ForOp> { class ConvertForOpTypes : public OpConversionPattern<ForOp> {
public: public:
▲ Show 20 Lines • Show All 175 Lines • Show Last 20 Lines

mlir/lib/Dialect/SCF/Utils/CMakeLists.txt

add_mlir_dialect_library(MLIRSCFUtils add_mlir_dialect_library(MLIRSCFUtils
AffineCanonicalizationUtils.cpp AffineCanonicalizationUtils.cpp
Utils.cpp Utils.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/SCF ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/SCF
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRAffine MLIRAffine
MLIRAffineAnalysis MLIRAffineAnalysis
MLIRAnalysis MLIRAnalysis
MLIRArithmetic MLIRArithmetic
MLIRDialectUtils MLIRDialectUtils
MLIRFunc
MLIRIR MLIRIR
MLIRSCF MLIRSCF
MLIRStandard
MLIRSupport MLIRSupport
MLIRTransforms MLIRTransforms
) )

mlir/lib/Dialect/SCF/Utils/Utils.cpp

//===- Utils.cpp ---- Misc utilities for loop transformation ----------===// //===- Utils.cpp ---- Misc utilities for loop transformation ----------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements miscellaneous loop transformation routines. // This file implements miscellaneous loop transformation routines.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/SCF/Utils/Utils.h" #include "mlir/Dialect/SCF/Utils/Utils.h"
#include "mlir/Analysis/SliceAnalysis.h" #include "mlir/Analysis/SliceAnalysis.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Support/MathExtras.h" #include "mlir/Support/MathExtras.h"
#include "mlir/Transforms/RegionUtils.h" #include "mlir/Transforms/RegionUtils.h"
#include "llvm/ADT/STLExtras.h" #include "llvm/ADT/STLExtras.h"
#include "llvm/ADT/SetVector.h" #include "llvm/ADT/SetVector.h"
#include "llvm/ADT/SmallPtrSet.h" #include "llvm/ADT/SmallPtrSet.h"
▲ Show 20 Lines • Show All 116 Lines • ▼ Show 20 LinesFailureOr<FuncOp> mlir::outlineSingleBlockRegion(RewriterBase &rewriter,
{ {
OpBuilder::InsertionGuard g(rewriter); OpBuilder::InsertionGuard g(rewriter);
rewriter.setInsertionPointToEnd(outlinedFuncBody); rewriter.setInsertionPointToEnd(outlinedFuncBody);
rewriter.mergeBlocks( rewriter.mergeBlocks(
originalBlock, outlinedFuncBody, originalBlock, outlinedFuncBody,
outlinedFuncBlockArgs.take_front(numOriginalBlockArguments)); outlinedFuncBlockArgs.take_front(numOriginalBlockArguments));
// Explicitly set up a new ReturnOp terminator. // Explicitly set up a new ReturnOp terminator.
rewriter.setInsertionPointToEnd(outlinedFuncBody); rewriter.setInsertionPointToEnd(outlinedFuncBody);
rewriter.create<ReturnOp>(loc, originalTerminator->getResultTypes(), rewriter.create<func::ReturnOp>(loc, originalTerminator->getResultTypes(),
originalTerminator->getOperands()); originalTerminator->getOperands());
} }
// Reconstruct the block that was deleted and add a // Reconstruct the block that was deleted and add a
// terminator(call_results). // terminator(call_results).
Block *newBlock = rewriter.createBlock( Block *newBlock = rewriter.createBlock(
&region, region.begin(), &region, region.begin(),
TypeRange{outlinedFuncArgTypes}.take_front(numOriginalBlockArguments), TypeRange{outlinedFuncArgTypes}.take_front(numOriginalBlockArguments),
ArrayRef<Location>(outlinedFuncArgLocs) ArrayRef<Location>(outlinedFuncArgLocs)
.take_front(numOriginalBlockArguments)); .take_front(numOriginalBlockArguments));
{ {
OpBuilder::InsertionGuard g(rewriter); OpBuilder::InsertionGuard g(rewriter);
rewriter.setInsertionPointToEnd(newBlock); rewriter.setInsertionPointToEnd(newBlock);
SmallVector<Value> callValues; SmallVector<Value> callValues;
llvm::append_range(callValues, newBlock->getArguments()); llvm::append_range(callValues, newBlock->getArguments());
llvm::append_range(callValues, outlinedValues); llvm::append_range(callValues, outlinedValues);
Operation *call = rewriter.create<CallOp>(loc, outlinedFunc, callValues); Operation *call =
rewriter.create<func::CallOp>(loc, outlinedFunc, callValues);
// `originalTerminator` was moved to `outlinedFuncBody` and is still valid. // `originalTerminator` was moved to `outlinedFuncBody` and is still valid.
// Clone `originalTerminator` to take the callOp results then erase it from // Clone `originalTerminator` to take the callOp results then erase it from
// `outlinedFuncBody`. // `outlinedFuncBody`.
BlockAndValueMapping bvm; BlockAndValueMapping bvm;
bvm.map(originalTerminator->getOperands(), call->getResults()); bvm.map(originalTerminator->getOperands(), call->getResults());
rewriter.clone(*originalTerminator, bvm); rewriter.clone(*originalTerminator, bvm);
rewriter.eraseOp(originalTerminator); rewriter.eraseOp(originalTerminator);
▲ Show 20 Lines • Show All 747 Lines • Show Last 20 Lines

mlir/lib/Dialect/Shape/Transforms/ShapeToShapeLowering.cpp

//===- ShapeToShapeLowering.cpp - Prepare for lowering to Standard --------===// //===- ShapeToShapeLowering.cpp - Prepare for lowering to Standard --------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Shape/IR/Shape.h" #include "mlir/Dialect/Shape/IR/Shape.h"
#include "mlir/Dialect/Shape/Transforms/Passes.h" #include "mlir/Dialect/Shape/Transforms/Passes.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
using namespace mlir; using namespace mlir;
using namespace mlir::shape; using namespace mlir::shape;
Show All 39 Lines
void ShapeToShapeLowering::runOnOperation() { void ShapeToShapeLowering::runOnOperation() {
MLIRContext &ctx = getContext(); MLIRContext &ctx = getContext();
RewritePatternSet patterns(&ctx); RewritePatternSet patterns(&ctx);
populateShapeRewritePatterns(patterns); populateShapeRewritePatterns(patterns);
ConversionTarget target(getContext()); ConversionTarget target(getContext());
target.addLegalDialect<arith::ArithmeticDialect, ShapeDialect, target.addLegalDialect<arith::ArithmeticDialect, func::FuncDialect,
StandardOpsDialect>(); ShapeDialect>();
target.addIllegalOp<NumElementsOp>(); target.addIllegalOp<NumElementsOp>();
if (failed(mlir::applyPartialConversion(getOperation(), target, if (failed(mlir::applyPartialConversion(getOperation(), target,
std::move(patterns)))) std::move(patterns))))
signalPassFailure(); signalPassFailure();
} }
void mlir::populateShapeRewritePatterns(RewritePatternSet &patterns) { void mlir::populateShapeRewritePatterns(RewritePatternSet &patterns) {
patterns.add<NumElementsOpConverter>(patterns.getContext()); patterns.add<NumElementsOpConverter>(patterns.getContext());
} }
std::unique_ptr<Pass> mlir::createShapeToShapeLowering() { std::unique_ptr<Pass> mlir::createShapeToShapeLowering() {
return std::make_unique<ShapeToShapeLowering>(); return std::make_unique<ShapeToShapeLowering>();
} }

mlir/lib/Dialect/SparseTensor/Pipelines/CMakeLists.txt

add_mlir_dialect_library(MLIRSparseTensorPipelines add_mlir_dialect_library(MLIRSparseTensorPipelines
SparseTensorPipelines.cpp SparseTensorPipelines.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/SparseTensor ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/SparseTensor
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRArithmeticTransforms MLIRArithmeticTransforms
MLIRAffineToStandard MLIRAffineToStandard
MLIRBufferizationTransforms MLIRBufferizationTransforms
MLIRFuncTransforms
MLIRLinalgTransforms MLIRLinalgTransforms
MLIRMathToLLVM MLIRMathToLLVM
MLIRMemRefToLLVM MLIRMemRefToLLVM
MLIRPass MLIRPass
MLIRReconcileUnrealizedCasts MLIRReconcileUnrealizedCasts
MLIRSCFToControlFlow MLIRSCFToControlFlow
MLIRSparseTensor MLIRSparseTensor
MLIRSparseTensorTransforms MLIRSparseTensorTransforms
MLIRStandardOpsTransforms
MLIRTensorTransforms MLIRTensorTransforms
MLIRVectorToLLVM MLIRVectorToLLVM
MLIRVectorTransforms MLIRVectorTransforms
) )

mlir/lib/Dialect/SparseTensor/Pipelines/SparseTensorPipelines.cpp

//===- SparseTensorPipelines.cpp - Pipelines for sparse tensor code -------===// //===- SparseTensorPipelines.cpp - Pipelines for sparse tensor code -------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/SparseTensor/Pipelines/Passes.h" #include "mlir/Dialect/SparseTensor/Pipelines/Passes.h"
#include "mlir/Conversion/Passes.h" #include "mlir/Conversion/Passes.h"
#include "mlir/Dialect/Arithmetic/Transforms/Passes.h" #include "mlir/Dialect/Arithmetic/Transforms/Passes.h"
#include "mlir/Dialect/Bufferization/Transforms/Passes.h" #include "mlir/Dialect/Bufferization/Transforms/Passes.h"
#include "mlir/Dialect/Func/Transforms/Passes.h"
#include "mlir/Dialect/Linalg/Passes.h" #include "mlir/Dialect/Linalg/Passes.h"
#include "mlir/Dialect/SparseTensor/IR/SparseTensor.h" #include "mlir/Dialect/SparseTensor/IR/SparseTensor.h"
#include "mlir/Dialect/SparseTensor/Transforms/Passes.h" #include "mlir/Dialect/SparseTensor/Transforms/Passes.h"
#include "mlir/Dialect/StandardOps/Transforms/Passes.h"
#include "mlir/Dialect/Tensor/Transforms/Passes.h" #include "mlir/Dialect/Tensor/Transforms/Passes.h"
#include "mlir/Dialect/Vector/Transforms/Passes.h" #include "mlir/Dialect/Vector/Transforms/Passes.h"
#include "mlir/Pass/PassManager.h" #include "mlir/Pass/PassManager.h"
using namespace mlir; using namespace mlir;
using namespace mlir::sparse_tensor; using namespace mlir::sparse_tensor;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Pipeline implementation. // Pipeline implementation.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
void mlir::sparse_tensor::buildSparseCompiler( void mlir::sparse_tensor::buildSparseCompiler(
OpPassManager &pm, const SparseCompilerOptions &options) { OpPassManager &pm, const SparseCompilerOptions &options) {
// TODO(wrengr): ensure the original `pm` is for ModuleOp // TODO(wrengr): ensure the original `pm` is for ModuleOp
pm.addNestedPass<FuncOp>(createLinalgGeneralizationPass()); pm.addNestedPass<FuncOp>(createLinalgGeneralizationPass());
pm.addPass(createLinalgElementwiseOpFusionPass()); pm.addPass(createLinalgElementwiseOpFusionPass());
pm.addPass(createSparsificationPass(options.sparsificationOptions())); pm.addPass(createSparsificationPass(options.sparsificationOptions()));
pm.addPass(createSparseTensorConversionPass()); pm.addPass(createSparseTensorConversionPass());
pm.addNestedPass<FuncOp>(createLinalgBufferizePass()); pm.addNestedPass<FuncOp>(createLinalgBufferizePass());
pm.addNestedPass<FuncOp>(vector::createVectorBufferizePass()); pm.addNestedPass<FuncOp>(vector::createVectorBufferizePass());
pm.addNestedPass<FuncOp>(createConvertLinalgToLoopsPass()); pm.addNestedPass<FuncOp>(createConvertLinalgToLoopsPass());
pm.addNestedPass<FuncOp>(createConvertVectorToSCFPass()); pm.addNestedPass<FuncOp>(createConvertVectorToSCFPass());
pm.addNestedPass<FuncOp>(createConvertSCFToCFPass()); pm.addNestedPass<FuncOp>(createConvertSCFToCFPass());
pm.addPass(createFuncBufferizePass()); pm.addPass(func::createFuncBufferizePass());
pm.addPass(arith::createConstantBufferizePass()); pm.addPass(arith::createConstantBufferizePass());
pm.addNestedPass<FuncOp>(createTensorBufferizePass()); pm.addNestedPass<FuncOp>(createTensorBufferizePass());
pm.addNestedPass<FuncOp>( pm.addNestedPass<FuncOp>(
mlir::bufferization::createFinalizingBufferizePass()); mlir::bufferization::createFinalizingBufferizePass());
pm.addPass(createLowerAffinePass()); pm.addPass(createLowerAffinePass());
pm.addPass(createConvertVectorToLLVMPass(options.lowerVectorToLLVMOptions())); pm.addPass(createConvertVectorToLLVMPass(options.lowerVectorToLLVMOptions()));
pm.addPass(createMemRefToLLVMPass()); pm.addPass(createMemRefToLLVMPass());
pm.addNestedPass<FuncOp>(createConvertMathToLLVMPass()); pm.addNestedPass<FuncOp>(createConvertMathToLLVMPass());
Show All 16 Lines

mlir/lib/Dialect/SparseTensor/Transforms/CMakeLists.txt

add_mlir_dialect_library(MLIRSparseTensorTransforms add_mlir_dialect_library(MLIRSparseTensorTransforms
CodegenUtils.cpp CodegenUtils.cpp
Sparsification.cpp Sparsification.cpp
SparseTensorConversion.cpp SparseTensorConversion.cpp
SparseTensorPasses.cpp SparseTensorPasses.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/SparseTensor ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/SparseTensor
DEPENDS DEPENDS
MLIRSparseTensorPassIncGen MLIRSparseTensorPassIncGen
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRArithmetic MLIRArithmetic
MLIRBufferization MLIRBufferization
MLIRFunc
MLIRIR MLIRIR
MLIRLLVMIR MLIRLLVMIR
MLIRLinalg MLIRLinalg
MLIRLinalgTransforms MLIRLinalgTransforms
MLIRMemRef MLIRMemRef
MLIRPass MLIRPass
MLIRSCF MLIRSCF
MLIRSCFTransforms MLIRSCFTransforms
MLIRSCFUtils MLIRSCFUtils
MLIRStandard
MLIRSparseTensor MLIRSparseTensor
MLIRSparseTensorUtils MLIRSparseTensorUtils
MLIRTensor MLIRTensor
MLIRTransforms MLIRTransforms
MLIRVector MLIRVector
) )

mlir/lib/Dialect/SparseTensor/Transforms/SparseTensorConversion.cpp

Show All 11 Lines
// elaborate IR code that implements the primitives on the selected sparse // elaborate IR code that implements the primitives on the selected sparse
// tensor storage schemes. // tensor storage schemes.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "CodegenUtils.h" #include "CodegenUtils.h"
#include "mlir/Dialect/Bufferization/IR/Bufferization.h" #include "mlir/Dialect/Bufferization/IR/Bufferization.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/Linalg/Utils/Utils.h" #include "mlir/Dialect/Linalg/Utils/Utils.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/SparseTensor/IR/SparseTensor.h" #include "mlir/Dialect/SparseTensor/IR/SparseTensor.h"
#include "mlir/Dialect/SparseTensor/Transforms/Passes.h" #include "mlir/Dialect/SparseTensor/Transforms/Passes.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/ExecutionEngine/SparseTensorUtils.h" #include "mlir/ExecutionEngine/SparseTensorUtils.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
using namespace mlir; using namespace mlir;
using namespace mlir::sparse_tensor; using namespace mlir::sparse_tensor;
namespace { namespace {
Show All 31 Lines if (!func) {
func.setPrivate(); func.setPrivate();
if (static_cast<bool>(emitCInterface)) if (static_cast<bool>(emitCInterface))
func->setAttr("llvm.emit_c_interface", UnitAttr::get(context)); func->setAttr("llvm.emit_c_interface", UnitAttr::get(context));
} }
return result; return result;
} }
/// Creates a `CallOp` to the function reference returned by `getFunc()`. /// Creates a `CallOp` to the function reference returned by `getFunc()`.
static CallOp createFuncCall(OpBuilder &builder, Operation *op, StringRef name, static func::CallOp createFuncCall(OpBuilder &builder, Operation *op,
TypeRange resultType, ValueRange operands, StringRef name, TypeRange resultType,
ValueRange operands,
EmitCInterface emitCInterface) { EmitCInterface emitCInterface) {
auto fn = getFunc(op, name, resultType, operands, emitCInterface); auto fn = getFunc(op, name, resultType, operands, emitCInterface);
return builder.create<CallOp>(op->getLoc(), resultType, fn, operands); return builder.create<func::CallOp>(op->getLoc(), resultType, fn, operands);
} }
/// Replaces the `op` with a `CallOp` to the function reference returned /// Replaces the `op` with a `CallOp` to the function reference returned
/// by `getFunc()`. /// by `getFunc()`.
static CallOp replaceOpWithFuncCall(PatternRewriter &rewriter, Operation *op, static func::CallOp replaceOpWithFuncCall(PatternRewriter &rewriter,
StringRef name, TypeRange resultType, Operation *op, StringRef name,
TypeRange resultType,
ValueRange operands, ValueRange operands,
EmitCInterface emitCInterface) { EmitCInterface emitCInterface) {
auto fn = getFunc(op, name, resultType, operands, emitCInterface); auto fn = getFunc(op, name, resultType, operands, emitCInterface);
return rewriter.replaceOpWithNewOp<CallOp>(op, resultType, fn, operands); return rewriter.replaceOpWithNewOp<func::CallOp>(op, resultType, fn,
operands);
} }
/// Generates dimension size call. /// Generates dimension size call.
static Value genDimSizeCall(ConversionPatternRewriter &rewriter, Operation *op, static Value genDimSizeCall(ConversionPatternRewriter &rewriter, Operation *op,
SparseTensorEncodingAttr &enc, Value src, SparseTensorEncodingAttr &enc, Value src,
int64_t idx) { int64_t idx) {
// Permute the index according to an optional dimension ordering. // Permute the index according to an optional dimension ordering.
if (AffineMap p = enc.getDimOrdering()) if (AffineMap p = enc.getDimOrdering())
▲ Show 20 Lines • Show All 255 Lines • ▼ Show 20 Linesstatic void insertScalarIntoDenseTensor(ConversionPatternRewriter &rewriter,
rewriter.create<memref::StoreOp>(loc, elemV, tensor, ivs); rewriter.create<memref::StoreOp>(loc, elemV, tensor, ivs);
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Conversion rules. // Conversion rules.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
/// Sparse conversion rule for returns. /// Sparse conversion rule for returns.
class SparseReturnConverter : public OpConversionPattern<ReturnOp> { class SparseReturnConverter : public OpConversionPattern<func::ReturnOp> {
public: public:
using OpConversionPattern::OpConversionPattern; using OpConversionPattern::OpConversionPattern;
LogicalResult LogicalResult
matchAndRewrite(ReturnOp op, OpAdaptor adaptor, matchAndRewrite(func::ReturnOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override { ConversionPatternRewriter &rewriter) const override {
rewriter.replaceOpWithNewOp<ReturnOp>(op, adaptor.getOperands()); rewriter.replaceOpWithNewOp<func::ReturnOp>(op, adaptor.getOperands());
return success(); return success();
} }
}; };
/// Sparse conversion rule for dimension accesses. /// Sparse conversion rule for dimension accesses.
class SparseTensorToDimSizeConverter class SparseTensorToDimSizeConverter
: public OpConversionPattern<tensor::DimOp> { : public OpConversionPattern<tensor::DimOp> {
public: public:
▲ Show 20 Lines • Show All 462 Lines • Show Last 20 Lines

mlir/lib/Dialect/SparseTensor/Transforms/SparseTensorPasses.cpp

//===- SparseTensorPasses.cpp - Pass for autogen sparse tensor code -------===// //===- SparseTensorPasses.cpp - Pass for autogen sparse tensor code -------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Bufferization/IR/Bufferization.h" #include "mlir/Dialect/Bufferization/IR/Bufferization.h"
#include "mlir/Dialect/Func/Transforms/FuncConversions.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/Linalg/Transforms/Transforms.h" #include "mlir/Dialect/Linalg/Transforms/Transforms.h"
#include "mlir/Dialect/SparseTensor/IR/SparseTensor.h" #include "mlir/Dialect/SparseTensor/IR/SparseTensor.h"
#include "mlir/Dialect/SparseTensor/Transforms/Passes.h" #include "mlir/Dialect/SparseTensor/Transforms/Passes.h"
#include "mlir/Dialect/StandardOps/Transforms/FuncConversions.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
using namespace mlir; using namespace mlir;
using namespace mlir::sparse_tensor; using namespace mlir::sparse_tensor;
namespace { namespace {
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 56 Lines • ▼ Show 20 Lines void runOnOperation() override {
ConversionTarget target(*ctx); ConversionTarget target(*ctx);
// Everything in the sparse dialect must go! // Everything in the sparse dialect must go!
target.addIllegalDialect<SparseTensorDialect>(); target.addIllegalDialect<SparseTensorDialect>();
// All dynamic rules below accept new function, call, return, and tensor // All dynamic rules below accept new function, call, return, and tensor
// dim and cast operations as legal output of the rewriting provided that // dim and cast operations as legal output of the rewriting provided that
// all sparse tensor types have been fully rewritten. // all sparse tensor types have been fully rewritten.
target.addDynamicallyLegalOp<FuncOp>( target.addDynamicallyLegalOp<FuncOp>(
[&](FuncOp op) { return converter.isSignatureLegal(op.getType()); }); [&](FuncOp op) { return converter.isSignatureLegal(op.getType()); });
target.addDynamicallyLegalOp<CallOp>([&](CallOp op) { target.addDynamicallyLegalOp<func::CallOp>([&](func::CallOp op) {
return converter.isSignatureLegal(op.getCalleeType()); return converter.isSignatureLegal(op.getCalleeType());
}); });
target.addDynamicallyLegalOp<ReturnOp>( target.addDynamicallyLegalOp<func::ReturnOp>([&](func::ReturnOp op) {
[&](ReturnOp op) { return converter.isLegal(op.getOperandTypes()); }); return converter.isLegal(op.getOperandTypes());
});
target.addDynamicallyLegalOp<tensor::DimOp>([&](tensor::DimOp op) { target.addDynamicallyLegalOp<tensor::DimOp>([&](tensor::DimOp op) {
return converter.isLegal(op.getOperandTypes()); return converter.isLegal(op.getOperandTypes());
}); });
target.addDynamicallyLegalOp<tensor::CastOp>([&](tensor::CastOp op) { target.addDynamicallyLegalOp<tensor::CastOp>([&](tensor::CastOp op) {
return converter.isLegal(op.getOperand().getType()); return converter.isLegal(op.getOperand().getType());
}); });
// The following operations and dialects may be introduced by the // The following operations and dialects may be introduced by the
// rewriting rules, and are therefore marked as legal. // rewriting rules, and are therefore marked as legal.
▲ Show 20 Lines • Show All 58 Lines • Show Last 20 Lines

mlir/lib/Dialect/SparseTensor/Transforms/Sparsification.cpp

Show All 9 Lines
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "CodegenUtils.h" #include "CodegenUtils.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Bufferization/IR/BufferizableOpInterface.h" #include "mlir/Dialect/Bufferization/IR/BufferizableOpInterface.h"
#include "mlir/Dialect/Bufferization/IR/Bufferization.h" #include "mlir/Dialect/Bufferization/IR/Bufferization.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/Linalg/Utils/Utils.h" #include "mlir/Dialect/Linalg/Utils/Utils.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/SCF/Transforms.h" #include "mlir/Dialect/SCF/Transforms.h"
#include "mlir/Dialect/SparseTensor/IR/SparseTensor.h" #include "mlir/Dialect/SparseTensor/IR/SparseTensor.h"
#include "mlir/Dialect/SparseTensor/Transforms/Passes.h" #include "mlir/Dialect/SparseTensor/Transforms/Passes.h"
#include "mlir/Dialect/SparseTensor/Utils/Merger.h" #include "mlir/Dialect/SparseTensor/Utils/Merger.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Vector/IR/VectorOps.h" #include "mlir/Dialect/Vector/IR/VectorOps.h"
#include "mlir/IR/Matchers.h" #include "mlir/IR/Matchers.h"
#include "mlir/IR/TensorEncoding.h" #include "mlir/IR/TensorEncoding.h"
#include "llvm/ADT/SmallBitVector.h" #include "llvm/ADT/SmallBitVector.h"
using namespace mlir; using namespace mlir;
using namespace mlir::sparse_tensor; using namespace mlir::sparse_tensor;
▲ Show 20 Lines • Show All 1,653 Lines • Show Last 20 Lines

mlir/lib/Dialect/StandardOps/CMakeLists.txt

  • This file was deleted.
add_mlir_dialect_library(MLIRStandard
IR/Ops.cpp
ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/StandardOps
DEPENDS
MLIRStandardOpsIncGen
LINK_LIBS PUBLIC
MLIRArithmetic
MLIRCallInterfaces
MLIRCastInterfaces
MLIRControlFlow
MLIRControlFlowInterfaces
MLIRInferTypeOpInterface
MLIRIR
MLIRSideEffectInterfaces
MLIRVectorInterfaces
)
add_subdirectory(Transforms)

mlir/lib/Dialect/StandardOps/IR/Ops.cpp

  • This file was moved to mlir/lib/Dialect/Func/IR/FuncOps.cpp.

mlir/lib/Dialect/StandardOps/Transforms/CMakeLists.txt

  • This file was moved to mlir/lib/Dialect/Func/Transforms/CMakeLists.txt.

mlir/lib/Dialect/StandardOps/Transforms/DecomposeCallGraphTypes.cpp

  • This file was moved to mlir/lib/Dialect/Func/Transforms/DecomposeCallGraphTypes.cpp.

mlir/lib/Dialect/StandardOps/Transforms/FuncBufferize.cpp

  • This file was moved to mlir/lib/Dialect/Func/Transforms/FuncBufferize.cpp.

mlir/lib/Dialect/StandardOps/Transforms/FuncConversions.cpp

  • This file was moved to mlir/lib/Dialect/Func/Transforms/FuncConversions.cpp.

mlir/lib/Dialect/StandardOps/Transforms/PassDetail.h

  • This file was moved to mlir/lib/Dialect/Func/Transforms/PassDetail.h.

mlir/lib/Dialect/Tensor/Transforms/Bufferize.cpp

Show All 9 Lines
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Bufferization/Transforms/Bufferize.h" #include "mlir/Dialect/Bufferization/Transforms/Bufferize.h"
#include "PassDetail.h" #include "PassDetail.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Bufferization/IR/BufferizableOpInterface.h" #include "mlir/Dialect/Bufferization/IR/BufferizableOpInterface.h"
#include "mlir/Dialect/Bufferization/IR/Bufferization.h" #include "mlir/Dialect/Bufferization/IR/Bufferization.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/Dialect/Tensor/Transforms/BufferizableOpInterfaceImpl.h" #include "mlir/Dialect/Tensor/Transforms/BufferizableOpInterfaceImpl.h"
#include "mlir/Dialect/Tensor/Transforms/Passes.h" #include "mlir/Dialect/Tensor/Transforms/Passes.h"
#include "mlir/IR/ImplicitLocOpBuilder.h" #include "mlir/IR/ImplicitLocOpBuilder.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
using namespace mlir; using namespace mlir;
using namespace bufferization; using namespace bufferization;
Show All 23 Lines

mlir/lib/Dialect/Tosa/Transforms/CMakeLists.txt

add_mlir_dialect_library(MLIRTosaTransforms add_mlir_dialect_library(MLIRTosaTransforms
TosaDecomposeTransposeConv.cpp TosaDecomposeTransposeConv.cpp
TosaDecomposeConv2D.cpp TosaDecomposeConv2D.cpp
TosaDecomposeDepthwise.cpp TosaDecomposeDepthwise.cpp
TosaInferShapes.cpp TosaInferShapes.cpp
TosaMakeBroadcastable.cpp TosaMakeBroadcastable.cpp
TosaOptionalDecompositions.cpp TosaOptionalDecompositions.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Tosa/Transforms ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Tosa/Transforms
DEPENDS DEPENDS
MLIRTosaPassIncGen MLIRTosaPassIncGen
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRFunc
MLIRPass MLIRPass
MLIRTosa MLIRTosa
MLIRStandard
MLIRTransformUtils MLIRTransformUtils
) )

mlir/lib/Dialect/Tosa/Transforms/TosaInferShapes.cpp

//===- TosaInferShapes.cpp ------------------------------------------===// //===- TosaInferShapes.cpp ------------------------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// Propogate shapes forward along TOSA operations to resolve dynamic shape // Propogate shapes forward along TOSA operations to resolve dynamic shape
// operations. // operations.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/DataFlowAnalysis.h" #include "mlir/Analysis/DataFlowAnalysis.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/Dialect/Tosa/IR/TosaOps.h" #include "mlir/Dialect/Tosa/IR/TosaOps.h"
#include "mlir/Dialect/Tosa/Transforms/PassDetail.h" #include "mlir/Dialect/Tosa/Transforms/PassDetail.h"
#include "mlir/Dialect/Tosa/Transforms/Passes.h" #include "mlir/Dialect/Tosa/Transforms/Passes.h"
#include "mlir/Dialect/Tosa/Utils/ShapeUtils.h" #include "mlir/Dialect/Tosa/Utils/ShapeUtils.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
▲ Show 20 Lines • Show All 195 Lines • ▼ Show 20 Lines for (Operation &op : block) {
for (auto it : llvm::zip(op.getResults(), returnedShapes)) { for (auto it : llvm::zip(op.getResults(), returnedShapes)) {
Value result = std::get<0>(it); Value result = std::get<0>(it);
ShapedTypeComponents predictedShape = std::get<1>(it); ShapedTypeComponents predictedShape = std::get<1>(it);
// Check whether this use case is replaceable. We define an op as // Check whether this use case is replaceable. We define an op as
// being replaceable if it is used by a ReturnOp or a TosaOp. // being replaceable if it is used by a ReturnOp or a TosaOp.
bool replaceable = true; bool replaceable = true;
for (auto *user : result.getUsers()) { for (auto *user : result.getUsers()) {
if (isa<ReturnOp>(user)) if (isa<func::ReturnOp>(user))
continue; continue;
if (user->getDialect()->getNamespace() == if (user->getDialect()->getNamespace() ==
TosaDialect::getDialectNamespace()) TosaDialect::getDialectNamespace())
continue; continue;
replaceable = false; replaceable = false;
} }
▲ Show 20 Lines • Show All 46 Lines • ▼ Show 20 Lines void runOnOperation() override {
FuncOp func = getOperation(); FuncOp func = getOperation();
IRRewriter rewriter(func.getContext()); IRRewriter rewriter(func.getContext());
propagateShapesInRegion(func.body()); propagateShapesInRegion(func.body());
// Insert UnrealizedConversionCasts to guarantee ReturnOp agress with // Insert UnrealizedConversionCasts to guarantee ReturnOp agress with
// the FuncOp type. // the FuncOp type.
func.walk([&](ReturnOp op) { func.walk([&](func::ReturnOp op) {
FuncOp parent = dyn_cast<FuncOp>(op->getParentOp()); FuncOp parent = dyn_cast<FuncOp>(op->getParentOp());
if (!parent) if (!parent)
return; return;
rewriter.setInsertionPoint(op); rewriter.setInsertionPoint(op);
FunctionType funcTy = func.getType(); FunctionType funcTy = func.getType();
auto resultTys = funcTy.getResults(); auto resultTys = funcTy.getResults();
Show All 11 Lines func.walk([&](func::ReturnOp op) {
castedValues.push_back( castedValues.push_back(
rewriter.create<tensor::CastOp>(op.getLoc(), castTy, operand) rewriter.create<tensor::CastOp>(op.getLoc(), castTy, operand)
.getResult()); .getResult());
castAdded = true; castAdded = true;
} }
if (castAdded) { if (castAdded) {
rewriter.replaceOpWithNewOp<ReturnOp>(op, castedValues); rewriter.replaceOpWithNewOp<func::ReturnOp>(op, castedValues);
} }
}); });
} }
}; };
} // namespace } // namespace
std::unique_ptr<Pass> mlir::tosa::createTosaInferShapesPass() { std::unique_ptr<Pass> mlir::tosa::createTosaInferShapesPass() {
return std::make_unique<TosaInferShapes>(); return std::make_unique<TosaInferShapes>();
} }

mlir/lib/Dialect/Vector/Utils/CMakeLists.txt

add_mlir_dialect_library(MLIRVectorUtils add_mlir_dialect_library(MLIRVectorUtils
VectorUtils.cpp VectorUtils.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Vector/Utils ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Vector/Utils
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRAffine MLIRAffine
MLIRAffineAnalysis MLIRAffineAnalysis
MLIRArithmetic MLIRArithmetic
MLIRDialectUtils MLIRDialectUtils
MLIRFunc
MLIRIR MLIRIR
MLIRMemRef MLIRMemRef
MLIRStandard
MLIRTensor MLIRTensor
MLIRVector MLIRVector
MLIRVectorInterfaces MLIRVectorInterfaces
) )

mlir/lib/Dialect/Vector/Utils/VectorUtils.cpp

Show All 9 Lines
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Vector/Utils/VectorUtils.h" #include "mlir/Dialect/Vector/Utils/VectorUtils.h"
#include "mlir/Dialect/Affine/Analysis/LoopAnalysis.h" #include "mlir/Dialect/Affine/Analysis/LoopAnalysis.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/Dialect/Vector/IR/VectorOps.h" #include "mlir/Dialect/Vector/IR/VectorOps.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/IntegerSet.h" #include "mlir/IR/IntegerSet.h"
#include "mlir/IR/Operation.h" #include "mlir/IR/Operation.h"
#include "mlir/IR/TypeUtilities.h" #include "mlir/IR/TypeUtilities.h"
#include "mlir/Support/LLVM.h" #include "mlir/Support/LLVM.h"
#include "mlir/Support/MathExtras.h" #include "mlir/Support/MathExtras.h"
▲ Show 20 Lines • Show All 250 Lines • ▼ Show 20 Linesbool matcher::operatesOnSuperVectorsOf(Operation &op,
/// do not have to special case. Maybe a trait, or just a method, unclear atm. /// do not have to special case. Maybe a trait, or just a method, unclear atm.
bool mustDivide = false; bool mustDivide = false;
(void)mustDivide; (void)mustDivide;
VectorType superVectorType; VectorType superVectorType;
if (auto transfer = dyn_cast<VectorTransferOpInterface>(op)) { if (auto transfer = dyn_cast<VectorTransferOpInterface>(op)) {
superVectorType = transfer.getVectorType(); superVectorType = transfer.getVectorType();
mustDivide = true; mustDivide = true;
} else if (op.getNumResults() == 0) { } else if (op.getNumResults() == 0) {
if (!isa<ReturnOp>(op)) { if (!isa<func::ReturnOp>(op)) {
op.emitError("NYI: assuming only return operations can have 0 " op.emitError("NYI: assuming only return operations can have 0 "
" results at this point"); " results at this point");
} }
return false; return false;
} else if (op.getNumResults() == 1) { } else if (op.getNumResults() == 1) {
if (auto v = op.getResult(0).getType().dyn_cast<VectorType>()) { if (auto v = op.getResult(0).getType().dyn_cast<VectorType>()) {
superVectorType = v; superVectorType = v;
} else { } else {
Show All 25 Lines

mlir/lib/Dialect/X86Vector/Transforms/LegalizeForLLVMExport.cpp

//===- LegalizeForLLVMExport.cpp - Prepare X86Vector for LLVM translation -===// //===- LegalizeForLLVMExport.cpp - Prepare X86Vector for LLVM translation -===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/X86Vector/Transforms.h" #include "mlir/Dialect/X86Vector/Transforms.h"
#include "mlir/Conversion/LLVMCommon/ConversionTarget.h" #include "mlir/Conversion/LLVMCommon/ConversionTarget.h"
#include "mlir/Conversion/LLVMCommon/Pattern.h" #include "mlir/Conversion/LLVMCommon/Pattern.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/X86Vector/X86VectorDialect.h" #include "mlir/Dialect/X86Vector/X86VectorDialect.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
using namespace mlir; using namespace mlir;
using namespace mlir::x86vector; using namespace mlir::x86vector;
/// Extracts the "main" vector element type from the given X86Vector operation. /// Extracts the "main" vector element type from the given X86Vector operation.
▲ Show 20 Lines • Show All 155 Lines • Show Last 20 Lines

mlir/lib/ExecutionEngine/CMakeLists.txt

Show First 20 Lines • Show All 71 Lines • ▼ Show 20 Linesadd_mlir_library(MLIRJitRunner
LINK_COMPONENTS LINK_COMPONENTS
Core Core
OrcJIT OrcJIT
JITLink JITLink
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
${dialect_libs} ${dialect_libs}
MLIRExecutionEngine MLIRExecutionEngine
MLIRFunc
MLIRIR MLIRIR
MLIRParser MLIRParser
MLIRStandard
MLIRLLVMToLLVMIRTranslation MLIRLLVMToLLVMIRTranslation
MLIRTargetLLVMIRExport MLIRTargetLLVMIRExport
MLIRTransforms MLIRTransforms
MLIRStandardToLLVM MLIRStandardToLLVM
MLIRSupport MLIRSupport
) )
add_mlir_library(mlir_c_runner_utils add_mlir_library(mlir_c_runner_utils
▲ Show 20 Lines • Show All 133 Lines • Show Last 20 Lines

mlir/lib/IR/Operation.cpp

Show First 20 Lines • Show All 602 Lines • ▼ Show 20 Lines
/// Print an operation name, eliding the dialect prefix if necessary. /// Print an operation name, eliding the dialect prefix if necessary.
void OpState::printOpName(Operation *op, OpAsmPrinter &p, void OpState::printOpName(Operation *op, OpAsmPrinter &p,
StringRef defaultDialect) { StringRef defaultDialect) {
StringRef name = op->getName().getStringRef(); StringRef name = op->getName().getStringRef();
if (name.startswith((defaultDialect + ".").str())) if (name.startswith((defaultDialect + ".").str()))
name = name.drop_front(defaultDialect.size() + 1); name = name.drop_front(defaultDialect.size() + 1);
// TODO: remove this special case (and update test/IR/parser.mlir) // TODO: remove this special case (and update test/IR/parser.mlir)
else if ((defaultDialect.empty() || defaultDialect == "builtin") && else if ((defaultDialect.empty() || defaultDialect == "builtin") &&
name.startswith("std.")) name.startswith("func."))
name = name.drop_front(4); name = name.drop_front(5);
p.getStream() << name; p.getStream() << name;
} }
/// Emit an error about fatal conditions with this operation, reporting up to /// Emit an error about fatal conditions with this operation, reporting up to
/// any diagnostic handlers that may be listening. /// any diagnostic handlers that may be listening.
InFlightDiagnostic OpState::emitError(const Twine &message) { InFlightDiagnostic OpState::emitError(const Twine &message) {
return getOperation()->emitError(message); return getOperation()->emitError(message);
} }
▲ Show 20 Lines • Show All 583 Lines • Show Last 20 Lines

mlir/lib/Parser/Parser.cpp

Show First 20 Lines • Show All 1,666 Lines • ▼ Show 20 Lines if (StringRef(opName).contains('.')) {
dialect = getContext()->getLoadedDialect(dialectName); dialect = getContext()->getLoadedDialect(dialectName);
if (!dialect && (dialect = getContext()->getOrLoadDialect(dialectName))) if (!dialect && (dialect = getContext()->getOrLoadDialect(dialectName)))
opInfo = RegisteredOperationName::lookup(opName, getContext()); opInfo = RegisteredOperationName::lookup(opName, getContext());
} else { } else {
// If the operation name has no namespace prefix we lookup the current // If the operation name has no namespace prefix we lookup the current
// default dialect (set through OpAsmOpInterface). // default dialect (set through OpAsmOpInterface).
opInfo = RegisteredOperationName::lookup( opInfo = RegisteredOperationName::lookup(
Twine(defaultDialect + "." + opName).str(), getContext()); Twine(defaultDialect + "." + opName).str(), getContext());
if (!opInfo && getContext()->getOrLoadDialect("std")) { // FIXME: Remove this in favor of using default dialects.
opInfo = RegisteredOperationName::lookup(Twine("std." + opName).str(), if (!opInfo && getContext()->getOrLoadDialect("func")) {
opInfo = RegisteredOperationName::lookup(Twine("func." + opName).str(),
getContext()); getContext());
} }
if (opInfo) { if (opInfo) {
dialect = &opInfo->getDialect(); dialect = &opInfo->getDialect();
opName = opInfo->getStringRef().str(); opName = opInfo->getStringRef().str();
} else if (!defaultDialect.empty()) { } else if (!defaultDialect.empty()) {
dialect = getContext()->getOrLoadDialect(defaultDialect); dialect = getContext()->getOrLoadDialect(defaultDialect);
opName = (defaultDialect + "." + opName).str(); opName = (defaultDialect + "." + opName).str();
▲ Show 20 Lines • Show All 584 Lines • Show Last 20 Lines

mlir/lib/Target/Cpp/CMakeLists.txt

add_mlir_translation_library(MLIRTargetCpp add_mlir_translation_library(MLIRTargetCpp
TranslateRegistration.cpp TranslateRegistration.cpp
TranslateToCpp.cpp TranslateToCpp.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${EMITC_MAIN_INCLUDE_DIR}/emitc/Target/Cpp ${EMITC_MAIN_INCLUDE_DIR}/emitc/Target/Cpp
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRArithmetic MLIRArithmetic
MLIRControlFlow MLIRControlFlow
MLIREmitC MLIREmitC
MLIRFunc
MLIRIR MLIRIR
MLIRMath MLIRMath
MLIRSCF MLIRSCF
MLIRStandard
MLIRSupport MLIRSupport
MLIRTranslation MLIRTranslation
) )

mlir/lib/Target/Cpp/TranslateRegistration.cpp

//===- TranslateRegistration.cpp - Register translation -------------------===// //===- TranslateRegistration.cpp - Register translation -------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/ControlFlow/IR/ControlFlow.h" #include "mlir/Dialect/ControlFlow/IR/ControlFlow.h"
#include "mlir/Dialect/EmitC/IR/EmitC.h" #include "mlir/Dialect/EmitC/IR/EmitC.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Math/IR/Math.h" #include "mlir/Dialect/Math/IR/Math.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/Dialect.h" #include "mlir/IR/Dialect.h"
#include "mlir/Target/Cpp/CppEmitter.h" #include "mlir/Target/Cpp/CppEmitter.h"
#include "mlir/Translation.h" #include "mlir/Translation.h"
#include "llvm/Support/CommandLine.h" #include "llvm/Support/CommandLine.h"
using namespace mlir; using namespace mlir;
Show All 16 Lines TranslateFromMLIRRegistration reg(
module, output, module, output,
/*declareVariablesAtTop=*/declareVariablesAtTop); /*declareVariablesAtTop=*/declareVariablesAtTop);
}, },
[](DialectRegistry &registry) { [](DialectRegistry &registry) {
// clang-format off // clang-format off
registry.insert<arith::ArithmeticDialect, registry.insert<arith::ArithmeticDialect,
cf::ControlFlowDialect, cf::ControlFlowDialect,
emitc::EmitCDialect, emitc::EmitCDialect,
func::FuncDialect,
math::MathDialect, math::MathDialect,
StandardOpsDialect,
scf::SCFDialect>(); scf::SCFDialect>();
// clang-format on // clang-format on
}); });
} }
} // namespace mlir } // namespace mlir

mlir/lib/Target/Cpp/TranslateToCpp.cpp

//===- TranslateToCpp.cpp - Translating to C++ calls ----------------------===// //===- TranslateToCpp.cpp - Translating to C++ calls ----------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h" #include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h"
#include "mlir/Dialect/EmitC/IR/EmitC.h" #include "mlir/Dialect/EmitC/IR/EmitC.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/BuiltinTypes.h" #include "mlir/IR/BuiltinTypes.h"
#include "mlir/IR/Dialect.h" #include "mlir/IR/Dialect.h"
#include "mlir/IR/Operation.h" #include "mlir/IR/Operation.h"
#include "mlir/Support/IndentedOstream.h" #include "mlir/Support/IndentedOstream.h"
#include "mlir/Target/Cpp/CppEmitter.h" #include "mlir/Target/Cpp/CppEmitter.h"
#include "llvm/ADT/DenseMap.h" #include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/StringExtras.h" #include "llvm/ADT/StringExtras.h"
▲ Show 20 Lines • Show All 212 Lines • ▼ Show 20 Linesstatic LogicalResult printOperation(CppEmitter &emitter,
arith::ConstantOp constantOp) { arith::ConstantOp constantOp) {
Operation *operation = constantOp.getOperation(); Operation *operation = constantOp.getOperation();
Attribute value = constantOp.getValue(); Attribute value = constantOp.getValue();
return printConstantOp(emitter, operation, value); return printConstantOp(emitter, operation, value);
} }
static LogicalResult printOperation(CppEmitter &emitter, static LogicalResult printOperation(CppEmitter &emitter,
mlir::ConstantOp constantOp) { func::ConstantOp constantOp) {
Operation *operation = constantOp.getOperation(); Operation *operation = constantOp.getOperation();
Attribute value = constantOp.getValueAttr(); Attribute value = constantOp.getValueAttr();
return printConstantOp(emitter, operation, value); return printConstantOp(emitter, operation, value);
} }
static LogicalResult printOperation(CppEmitter &emitter, static LogicalResult printOperation(CppEmitter &emitter,
cf::BranchOp branchOp) { cf::BranchOp branchOp) {
▲ Show 20 Lines • Show All 56 Lines • ▼ Show 20 Lines if (!(emitter.hasBlockLabel(falseSuccessor))) {
return condBranchOp.emitOpError() return condBranchOp.emitOpError()
<< "unable to find label for successor block"; << "unable to find label for successor block";
} }
os << emitter.getOrCreateName(falseSuccessor) << ";\n"; os << emitter.getOrCreateName(falseSuccessor) << ";\n";
os.unindent() << "}"; os.unindent() << "}";
return success(); return success();
} }
static LogicalResult printOperation(CppEmitter &emitter, mlir::CallOp callOp) { static LogicalResult printOperation(CppEmitter &emitter, func::CallOp callOp) {
if (failed(emitter.emitAssignPrefix(*callOp.getOperation()))) if (failed(emitter.emitAssignPrefix(*callOp.getOperation())))
return failure(); return failure();
raw_ostream &os = emitter.ostream(); raw_ostream &os = emitter.ostream();
os << callOp.getCallee() << "("; os << callOp.getCallee() << "(";
if (failed(emitter.emitOperands(*callOp.getOperation()))) if (failed(emitter.emitOperands(*callOp.getOperation())))
return failure(); return failure();
os << ")"; os << ")";
▲ Show 20 Lines • Show All 217 Lines • ▼ Show 20 Lines if (failed(interleaveWithError(
return success(); return success();
}, },
[&]() { os << ";\n"; }))) [&]() { os << ";\n"; })))
return failure(); return failure();
return success(); return success();
} }
static LogicalResult printOperation(CppEmitter &emitter, ReturnOp returnOp) { static LogicalResult printOperation(CppEmitter &emitter,
func::ReturnOp returnOp) {
raw_ostream &os = emitter.ostream(); raw_ostream &os = emitter.ostream();
os << "return"; os << "return";
switch (returnOp.getNumOperands()) { switch (returnOp.getNumOperands()) {
case 0: case 0:
return success(); return success();
case 1: case 1:
os << " " << emitter.getOrCreateName(returnOp.getOperand(0)); os << " " << emitter.getOrCreateName(returnOp.getOperand(0));
return success(emitter.hasValueInScope(returnOp.getOperand(0))); return success(emitter.hasValueInScope(returnOp.getOperand(0)));
▲ Show 20 Lines • Show All 347 Lines • ▼ Show 20 LinesLogicalResult CppEmitter::emitLabel(Block &block) {
// label instead of using `getOStream`. // label instead of using `getOStream`.
os.getOStream() << getOrCreateName(block) << ":\n"; os.getOStream() << getOrCreateName(block) << ":\n";
return success(); return success();
} }
LogicalResult CppEmitter::emitOperation(Operation &op, bool trailingSemicolon) { LogicalResult CppEmitter::emitOperation(Operation &op, bool trailingSemicolon) {
LogicalResult status = LogicalResult status =
llvm::TypeSwitch<Operation *, LogicalResult>(&op) llvm::TypeSwitch<Operation *, LogicalResult>(&op)
// Builtin ops.
.Case<FuncOp, ModuleOp>(
[&](auto op) { return printOperation(*this, op); })
// CF ops.
.Case<cf::BranchOp, cf::CondBranchOp>(
[&](auto op) { return printOperation(*this, op); })
// EmitC ops. // EmitC ops.
.Case<emitc::ApplyOp, emitc::CallOp, emitc::ConstantOp, .Case<emitc::ApplyOp, emitc::CallOp, emitc::ConstantOp,
emitc::IncludeOp, emitc::VariableOp>( emitc::IncludeOp, emitc::VariableOp>(
[&](auto op) { return printOperation(*this, op); }) [&](auto op) { return printOperation(*this, op); })
// Func ops.
.Case<func::CallOp, func::ConstantOp, func::ReturnOp>(
[&](auto op) { return printOperation(*this, op); })
// SCF ops. // SCF ops.
.Case<scf::ForOp, scf::IfOp, scf::YieldOp>( .Case<scf::ForOp, scf::IfOp, scf::YieldOp>(
[&](auto op) { return printOperation(*this, op); }) [&](auto op) { return printOperation(*this, op); })
// Standard ops.
.Case<cf::BranchOp, mlir::CallOp, cf::CondBranchOp, mlir::ConstantOp,
FuncOp, ModuleOp, ReturnOp>(
[&](auto op) { return printOperation(*this, op); })
// Arithmetic ops. // Arithmetic ops.
.Case<arith::ConstantOp>( .Case<arith::ConstantOp>(
[&](auto op) { return printOperation(*this, op); }) [&](auto op) { return printOperation(*this, op); })
.Default([&](Operation *) { .Default([&](Operation *) {
return op.emitOpError("unable to find printer for op"); return op.emitOpError("unable to find printer for op");
}); });
if (failed(status)) if (failed(status))
▲ Show 20 Lines • Show All 91 Lines • Show Last 20 Lines

mlir/python/CMakeLists.txt

Show First 20 Lines • Show All 77 Lines • ▼ Show 20 Linesdeclare_mlir_dialect_python_bindings(
TD_FILE dialects/ControlFlowOps.td TD_FILE dialects/ControlFlowOps.td
SOURCES SOURCES
dialects/cf.py dialects/cf.py
DIALECT_NAME cf) DIALECT_NAME cf)
declare_mlir_dialect_python_bindings( declare_mlir_dialect_python_bindings(
ADD_TO_PARENT MLIRPythonSources.Dialects ADD_TO_PARENT MLIRPythonSources.Dialects
ROOT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/mlir" ROOT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/mlir"
TD_FILE dialects/Func.td
SOURCES
dialects/func.py
dialects/_func_ops_ext.py
DIALECT_NAME func)
declare_mlir_dialect_python_bindings(
ADD_TO_PARENT MLIRPythonSources.Dialects
ROOT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/mlir"
TD_FILE dialects/GPUOps.td TD_FILE dialects/GPUOps.td
SOURCES_GLOB dialects/gpu/*.py SOURCES_GLOB dialects/gpu/*.py
DIALECT_NAME gpu) DIALECT_NAME gpu)
declare_mlir_dialect_python_bindings( declare_mlir_dialect_python_bindings(
ADD_TO_PARENT MLIRPythonSources.Dialects ADD_TO_PARENT MLIRPythonSources.Dialects
ROOT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/mlir" ROOT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/mlir"
TD_FILE dialects/LinalgOps.td TD_FILE dialects/LinalgOps.td
▲ Show 20 Lines • Show All 68 Lines • ▼ Show 20 Linesdeclare_mlir_dialect_python_bindings(
ROOT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/mlir" ROOT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/mlir"
TD_FILE dialects/SparseTensorOps.td TD_FILE dialects/SparseTensorOps.td
SOURCES dialects/sparse_tensor.py SOURCES dialects/sparse_tensor.py
DIALECT_NAME sparse_tensor) DIALECT_NAME sparse_tensor)
declare_mlir_dialect_python_bindings( declare_mlir_dialect_python_bindings(
ADD_TO_PARENT MLIRPythonSources.Dialects ADD_TO_PARENT MLIRPythonSources.Dialects
ROOT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/mlir" ROOT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/mlir"
TD_FILE dialects/StandardOps.td
SOURCES
dialects/std.py
dialects/_std_ops_ext.py
DIALECT_NAME std)
declare_mlir_dialect_python_bindings(
ADD_TO_PARENT MLIRPythonSources.Dialects
ROOT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/mlir"
TD_FILE dialects/TensorOps.td TD_FILE dialects/TensorOps.td
SOURCES dialects/tensor.py SOURCES dialects/tensor.py
DIALECT_NAME tensor) DIALECT_NAME tensor)
declare_mlir_dialect_python_bindings( declare_mlir_dialect_python_bindings(
ADD_TO_PARENT MLIRPythonSources.Dialects ADD_TO_PARENT MLIRPythonSources.Dialects
ROOT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/mlir" ROOT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/mlir"
TD_FILE dialects/TosaOps.td TD_FILE dialects/TosaOps.td
Show All 40 Lines PRIVATE_LINK_LIBS
LLVMSupport LLVMSupport
EMBED_CAPI_LINK_LIBS EMBED_CAPI_LINK_LIBS
MLIRCAPIDebug MLIRCAPIDebug
MLIRCAPIIR MLIRCAPIIR
MLIRCAPIInterfaces MLIRCAPIInterfaces
MLIRCAPIRegistration # TODO: See about dis-aggregating MLIRCAPIRegistration # TODO: See about dis-aggregating
# Dialects # Dialects
MLIRCAPIStandard MLIRCAPIFunc
) )
declare_mlir_python_extension(MLIRPythonExtension.Dialects.Linalg.Pybind declare_mlir_python_extension(MLIRPythonExtension.Dialects.Linalg.Pybind
MODULE_NAME _mlirDialectsLinalg MODULE_NAME _mlirDialectsLinalg
ADD_TO_PARENT MLIRPythonSources.Dialects.linalg ADD_TO_PARENT MLIRPythonSources.Dialects.linalg
ROOT_DIR "${PYTHON_SOURCE_DIR}" ROOT_DIR "${PYTHON_SOURCE_DIR}"
SOURCES SOURCES
DialectLinalg.cpp DialectLinalg.cpp
▲ Show 20 Lines • Show All 229 Lines • Show Last 20 Lines

mlir/python/mlir/dialects/FuncOps.td

  • This file was moved from mlir/python/mlir/dialects/StandardOps.td.
//===-- StandardOps.td - Entry point for StandardOps bind --*- tablegen -*-===// //===-- FuncOps.td - Entry point for Func bind -------------*- tablegen -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This is the main file from which the Python bindings for the Standard // This is the main file from which the Python bindings for the Func dialect
// dialect are generated. // are generated.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef PYTHON_BINDINGS_STANDARD_OPS #ifndef PYTHON_BINDINGS_FUNC
#define PYTHON_BINDINGS_STANDARD_OPS #define PYTHON_BINDINGS_FUNC
include "mlir/Bindings/Python/Attributes.td" include "mlir/Bindings/Python/Attributes.td"
include "mlir/Dialect/StandardOps/IR/Ops.td" include "mlir/Dialect/Func/IR/FuncOps.td"
#endif #endif

mlir/python/mlir/dialects/StandardOps.td

  • This file was moved to mlir/python/mlir/dialects/FuncOps.td.

mlir/python/mlir/dialects/_builtin_ops_ext.py

Show First 20 Lines • Show All 156 Lines • ▼ Show 20 Lines def from_py_func(FuncOp,
The decorated function can further be called from Python and will insert The decorated function can further be called from Python and will insert
a `CallOp` at the then-current insertion point, returning either None ( a `CallOp` at the then-current insertion point, returning either None (
if no return values), a unary Value (for one result), or a list of Values). if no return values), a unary Value (for one result), or a list of Values).
This mechanism cannot be used to emit recursive calls (by construction). This mechanism cannot be used to emit recursive calls (by construction).
""" """
def decorator(f): def decorator(f):
from . import std from . import func
# Introspect the callable for optional features. # Introspect the callable for optional features.
sig = inspect.signature(f) sig = inspect.signature(f)
has_arg_func_op = False has_arg_func_op = False
for param in sig.parameters.values(): for param in sig.parameters.values():
if param.kind == param.VAR_KEYWORD: if param.kind == param.VAR_KEYWORD:
has_arg_func_op = True has_arg_func_op = True
if param.name == "func_op" and (param.kind if param.name == "func_op" and (param.kind
== param.POSITIONAL_OR_KEYWORD or == param.POSITIONAL_OR_KEYWORD or
Show All 29 Lines def decorator(f):
elif isinstance(return_values, OpView): elif isinstance(return_values, OpView):
# Returning a single operation is fine, coerce its results a list. # Returning a single operation is fine, coerce its results a list.
return_values = return_values.operation.results return_values = return_values.operation.results
elif isinstance(return_values, Operation): elif isinstance(return_values, Operation):
# Returning a single operation is fine, coerce its results a list. # Returning a single operation is fine, coerce its results a list.
return_values = return_values.results return_values = return_values.results
else: else:
return_values = list(return_values) return_values = list(return_values)
std.ReturnOp(return_values) func.ReturnOp(return_values)
# Recompute the function type. # Recompute the function type.
return_types = [v.type for v in return_values] return_types = [v.type for v in return_values]
function_type = FunctionType.get(inputs=inputs, results=return_types) function_type = FunctionType.get(inputs=inputs, results=return_types)
func_op.attributes["type"] = TypeAttr.get(function_type) func_op.attributes["type"] = TypeAttr.get(function_type)
def emit_call_op(*call_args): def emit_call_op(*call_args):
call_op = std.CallOp(return_types, FlatSymbolRefAttr.get(symbol_name), call_op = func.CallOp(return_types, FlatSymbolRefAttr.get(symbol_name),
call_args) call_args)
if return_types is None: if return_types is None:
return None return None
elif len(return_types) == 1: elif len(return_types) == 1:
return call_op.result return call_op.result
else: else:
return call_op.results return call_op.results
wrapped = emit_call_op wrapped = emit_call_op
wrapped.__name__ = f.__name__ wrapped.__name__ = f.__name__
wrapped.func_op = func_op wrapped.func_op = func_op
return wrapped return wrapped
return decorator return decorator

mlir/python/mlir/dialects/_func_ops_ext.py

  • This file was moved from mlir/python/mlir/dialects/_std_ops_ext.py.
Show All 40 Lines The constructor accepts three different forms:
2. A list of result types, followed by the name of the function to be 2. A list of result types, followed by the name of the function to be
called as string, following by a list of arguments. called as string, following by a list of arguments.
3. A list of result types, followed by the name of the function to be 3. A list of result types, followed by the name of the function to be
called as symbol reference attribute, followed by a list of arguments. called as symbol reference attribute, followed by a list of arguments.
For example For example
f = builtin.FuncOp("foo", ...) f = builtin.FuncOp("foo", ...)
std.CallOp(f, [args]) func.CallOp(f, [args])
std.CallOp([result_types], "foo", [args]) func.CallOp([result_types], "foo", [args])
In all cases, the location and insertion point may be specified as keyword In all cases, the location and insertion point may be specified as keyword
arguments if not provided by the surrounding context managers. arguments if not provided by the surrounding context managers.
""" """
# TODO: consider supporting constructor "overloads", e.g., through a custom # TODO: consider supporting constructor "overloads", e.g., through a custom
# or pybind-provided metaclass. # or pybind-provided metaclass.
if isinstance(calleeOrResults, FuncOp): if isinstance(calleeOrResults, FuncOp):
Show All 35 Lines

mlir/python/mlir/dialects/_std_ops_ext.py

  • This file was moved to mlir/python/mlir/dialects/_func_ops_ext.py.

mlir/python/mlir/dialects/func.py

  • This file was moved from mlir/python/mlir/dialects/std.py.
# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. # Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
# See https://llvm.org/LICENSE.txt for license information. # See https://llvm.org/LICENSE.txt for license information.
# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
from ._std_ops_gen import * from ._func_ops_gen import *

mlir/python/mlir/dialects/std.py

  • This file was moved to mlir/python/mlir/dialects/func.py.

mlir/test/CAPI/CMakeLists.txt

Show All 28 Lines LINK_LIBS PRIVATE
MLIRCAPIRegistration MLIRCAPIRegistration
) )
endif() endif()
_add_capi_test_executable(mlir-capi-ir-test _add_capi_test_executable(mlir-capi-ir-test
ir.c ir.c
LINK_LIBS PRIVATE LINK_LIBS PRIVATE
MLIRCAPIIR MLIRCAPIIR
MLIRCAPIStandard MLIRCAPIFunc
MLIRCAPIRegistration MLIRCAPIRegistration
) )
_add_capi_test_executable(mlir-capi-llvm-test _add_capi_test_executable(mlir-capi-llvm-test
llvm.c llvm.c
LINK_LIBS PRIVATE LINK_LIBS PRIVATE
MLIRCAPIIR MLIRCAPIIR
MLIRCAPILLVM MLIRCAPILLVM
Show All 34 Lines

mlir/test/CAPI/ir.c

Show All 10 Lines
*/ */
#include "mlir-c/IR.h" #include "mlir-c/IR.h"
#include "mlir-c/AffineExpr.h" #include "mlir-c/AffineExpr.h"
#include "mlir-c/AffineMap.h" #include "mlir-c/AffineMap.h"
#include "mlir-c/BuiltinAttributes.h" #include "mlir-c/BuiltinAttributes.h"
#include "mlir-c/BuiltinTypes.h" #include "mlir-c/BuiltinTypes.h"
#include "mlir-c/Diagnostics.h" #include "mlir-c/Diagnostics.h"
#include "mlir-c/Dialect/Standard.h" #include "mlir-c/Dialect/Func.h"
#include "mlir-c/IntegerSet.h" #include "mlir-c/IntegerSet.h"
#include "mlir-c/Registration.h" #include "mlir-c/Registration.h"
#include "mlir-c/Support.h" #include "mlir-c/Support.h"
#include <assert.h> #include <assert.h>
#include <inttypes.h> #include <inttypes.h>
#include <math.h> #include <math.h>
#include <stdio.h> #include <stdio.h>
▲ Show 20 Lines • Show All 128 Lines • ▼ Show 20 LinesMlirModule makeAndDumpAdd(MlirContext ctx, MlirLocation location) {
mlirOperationStateAddOperands(&loopState, 3, loopOperands); mlirOperationStateAddOperands(&loopState, 3, loopOperands);
mlirOperationStateAddOwnedRegions(&loopState, 1, &loopBodyRegion); mlirOperationStateAddOwnedRegions(&loopState, 1, &loopBodyRegion);
MlirOperation loop = mlirOperationCreate(&loopState); MlirOperation loop = mlirOperationCreate(&loopState);
mlirBlockAppendOwnedOperation(funcBody, loop); mlirBlockAppendOwnedOperation(funcBody, loop);
populateLoopBody(ctx, loopBody, location, funcBody); populateLoopBody(ctx, loopBody, location, funcBody);
MlirOperationState retState = mlirOperationStateGet( MlirOperationState retState = mlirOperationStateGet(
mlirStringRefCreateFromCString("std.return"), location); mlirStringRefCreateFromCString("func.return"), location);
MlirOperation ret = mlirOperationCreate(&retState); MlirOperation ret = mlirOperationCreate(&retState);
mlirBlockAppendOwnedOperation(funcBody, ret); mlirBlockAppendOwnedOperation(funcBody, ret);
MlirOperation module = mlirModuleGetOperation(moduleOp); MlirOperation module = mlirModuleGetOperation(moduleOp);
mlirOperationDump(module); mlirOperationDump(module);
// clang-format off // clang-format off
// CHECK: module { // CHECK: module {
// CHECK: func @add(%[[ARG0:.*]]: memref<?xf32>, %[[ARG1:.*]]: memref<?xf32>) { // CHECK: func @add(%[[ARG0:.*]]: memref<?xf32>, %[[ARG1:.*]]: memref<?xf32>) {
▲ Show 20 Lines • Show All 659 Lines • ▼ Show 20 Linesint printBuiltinAttributes(MlirContext ctx) {
MlirAttribute boolean = mlirBoolAttrGet(ctx, 1); MlirAttribute boolean = mlirBoolAttrGet(ctx, 1);
if (!mlirAttributeIsABool(boolean) || !mlirBoolAttrGetValue(boolean)) if (!mlirAttributeIsABool(boolean) || !mlirBoolAttrGetValue(boolean))
return 3; return 3;
mlirAttributeDump(boolean); mlirAttributeDump(boolean);
// CHECK: true // CHECK: true
const char data[] = "abcdefghijklmnopqestuvwxyz"; const char data[] = "abcdefghijklmnopqestuvwxyz";
MlirAttribute opaque = MlirAttribute opaque =
mlirOpaqueAttrGet(ctx, mlirStringRefCreateFromCString("std"), 3, data, mlirOpaqueAttrGet(ctx, mlirStringRefCreateFromCString("func"), 3, data,
mlirNoneTypeGet(ctx)); mlirNoneTypeGet(ctx));
if (!mlirAttributeIsAOpaque(opaque) || if (!mlirAttributeIsAOpaque(opaque) ||
!stringIsEqual("std", mlirOpaqueAttrGetDialectNamespace(opaque))) !stringIsEqual("func", mlirOpaqueAttrGetDialectNamespace(opaque)))
return 4; return 4;
MlirStringRef opaqueData = mlirOpaqueAttrGetData(opaque); MlirStringRef opaqueData = mlirOpaqueAttrGetData(opaque);
if (opaqueData.length != 3 || if (opaqueData.length != 3 ||
strncmp(data, opaqueData.data, opaqueData.length)) strncmp(data, opaqueData.data, opaqueData.length))
return 5; return 5;
mlirAttributeDump(opaque); mlirAttributeDump(opaque);
// CHECK: #std.abc // CHECK: #func.abc
MlirAttribute string = MlirAttribute string =
mlirStringAttrGet(ctx, mlirStringRefCreate(data + 3, 2)); mlirStringAttrGet(ctx, mlirStringRefCreate(data + 3, 2));
if (!mlirAttributeIsAString(string)) if (!mlirAttributeIsAString(string))
return 6; return 6;
MlirStringRef stringValue = mlirStringAttrGetValue(string); MlirStringRef stringValue = mlirStringAttrGetValue(string);
if (stringValue.length != 2 || if (stringValue.length != 2 ||
▲ Show 20 Lines • Show All 649 Lines • ▼ Show 20 Lines
} }
int registerOnlyStd() { int registerOnlyStd() {
MlirContext ctx = mlirContextCreate(); MlirContext ctx = mlirContextCreate();
// The built-in dialect is always loaded. // The built-in dialect is always loaded.
if (mlirContextGetNumLoadedDialects(ctx) != 1) if (mlirContextGetNumLoadedDialects(ctx) != 1)
return 1; return 1;
MlirDialectHandle stdHandle = mlirGetDialectHandle__std__(); MlirDialectHandle stdHandle = mlirGetDialectHandle__func__();
MlirDialect std = mlirContextGetOrLoadDialect( MlirDialect std = mlirContextGetOrLoadDialect(
ctx, mlirDialectHandleGetNamespace(stdHandle)); ctx, mlirDialectHandleGetNamespace(stdHandle));
if (!mlirDialectIsNull(std)) if (!mlirDialectIsNull(std))
return 2; return 2;
mlirDialectHandleRegisterDialect(stdHandle, ctx); mlirDialectHandleRegisterDialect(stdHandle, ctx);
Show All 15 Linesint registerOnlyStd() {
fprintf(stderr, "@registration\n"); fprintf(stderr, "@registration\n");
// CHECK-LABEL: @registration // CHECK-LABEL: @registration
// CHECK: cf.cond_br is_registered: 1 // CHECK: cf.cond_br is_registered: 1
fprintf(stderr, "cf.cond_br is_registered: %d\n", fprintf(stderr, "cf.cond_br is_registered: %d\n",
mlirContextIsRegisteredOperation( mlirContextIsRegisteredOperation(
ctx, mlirStringRefCreateFromCString("cf.cond_br"))); ctx, mlirStringRefCreateFromCString("cf.cond_br")));
// CHECK: std.not_existing_op is_registered: 0 // CHECK: func.not_existing_op is_registered: 0
fprintf(stderr, "std.not_existing_op is_registered: %d\n", fprintf(stderr, "func.not_existing_op is_registered: %d\n",
mlirContextIsRegisteredOperation( mlirContextIsRegisteredOperation(
ctx, mlirStringRefCreateFromCString("std.not_existing_op"))); ctx, mlirStringRefCreateFromCString("func.not_existing_op")));
// CHECK: not_existing_dialect.not_existing_op is_registered: 0 // CHECK: not_existing_dialect.not_existing_op is_registered: 0
fprintf(stderr, "not_existing_dialect.not_existing_op is_registered: %d\n", fprintf(stderr, "not_existing_dialect.not_existing_op is_registered: %d\n",
mlirContextIsRegisteredOperation( mlirContextIsRegisteredOperation(
ctx, mlirStringRefCreateFromCString( ctx, mlirStringRefCreateFromCString(
"not_existing_dialect.not_existing_op"))); "not_existing_dialect.not_existing_op")));
mlirContextDestroy(ctx); mlirContextDestroy(ctx);
▲ Show 20 Lines • Show All 108 Lines • ▼ Show 20 Lines
/// Tests clone APIs. /// Tests clone APIs.
int testClone() { int testClone() {
fprintf(stderr, "@testClone\n"); fprintf(stderr, "@testClone\n");
// CHECK-LABEL: @testClone // CHECK-LABEL: @testClone
MlirContext ctx = mlirContextCreate(); MlirContext ctx = mlirContextCreate();
mlirRegisterAllDialects(ctx); mlirRegisterAllDialects(ctx);
mlirContextGetOrLoadDialect(ctx, mlirStringRefCreateFromCString("std")); mlirContextGetOrLoadDialect(ctx, mlirStringRefCreateFromCString("func"));
MlirLocation loc = mlirLocationUnknownGet(ctx); MlirLocation loc = mlirLocationUnknownGet(ctx);
MlirType indexType = mlirIndexTypeGet(ctx); MlirType indexType = mlirIndexTypeGet(ctx);
MlirStringRef valueStringRef = mlirStringRefCreateFromCString("value"); MlirStringRef valueStringRef = mlirStringRefCreateFromCString("value");
MlirAttribute indexZeroLiteral = MlirAttribute indexZeroLiteral =
mlirAttributeParseGet(ctx, mlirStringRefCreateFromCString("0 : index")); mlirAttributeParseGet(ctx, mlirStringRefCreateFromCString("0 : index"));
MlirNamedAttribute indexZeroValueAttr = mlirNamedAttributeGet( MlirNamedAttribute indexZeroValueAttr = mlirNamedAttributeGet(
mlirIdentifierGet(ctx, valueStringRef), indexZeroLiteral); mlirIdentifierGet(ctx, valueStringRef), indexZeroLiteral);
▲ Show 20 Lines • Show All 232 Lines • ▼ Show 20 Linesint testDialectRegistry() {
fprintf(stderr, "@testDialectRegistry\n"); fprintf(stderr, "@testDialectRegistry\n");
MlirDialectRegistry registry = mlirDialectRegistryCreate(); MlirDialectRegistry registry = mlirDialectRegistryCreate();
if (mlirDialectRegistryIsNull(registry)) { if (mlirDialectRegistryIsNull(registry)) {
fprintf(stderr, "ERROR: Expected registry to be present\n"); fprintf(stderr, "ERROR: Expected registry to be present\n");
return 1; return 1;
} }
MlirDialectHandle stdHandle = mlirGetDialectHandle__std__(); MlirDialectHandle stdHandle = mlirGetDialectHandle__func__();
mlirDialectHandleInsertDialect(stdHandle, registry); mlirDialectHandleInsertDialect(stdHandle, registry);
MlirContext ctx = mlirContextCreate(); MlirContext ctx = mlirContextCreate();
if (mlirContextGetNumRegisteredDialects(ctx) != 0) { if (mlirContextGetNumRegisteredDialects(ctx) != 0) {
fprintf(stderr, fprintf(stderr,
"ERROR: Expected no dialects to be registered to new context\n"); "ERROR: Expected no dialects to be registered to new context\n");
} }
▲ Show 20 Lines • Show All 104 Lines • Show Last 20 Lines

mlir/test/CAPI/pass.c

Show All 37 Lines if (mlirModuleIsNull(module)) {
fprintf(stderr, "Unexpected failure parsing module.\n"); fprintf(stderr, "Unexpected failure parsing module.\n");
exit(EXIT_FAILURE); exit(EXIT_FAILURE);
} }
// Run the print-op-stats pass on the top-level module: // Run the print-op-stats pass on the top-level module:
// CHECK-LABEL: Operations encountered: // CHECK-LABEL: Operations encountered:
// CHECK: arith.addi , 1 // CHECK: arith.addi , 1
// CHECK: builtin.func , 1 // CHECK: builtin.func , 1
// CHECK: std.return , 1 // CHECK: func.return , 1
{ {
MlirPassManager pm = mlirPassManagerCreate(ctx); MlirPassManager pm = mlirPassManagerCreate(ctx);
MlirPass printOpStatPass = mlirCreateTransformsPrintOpStats(); MlirPass printOpStatPass = mlirCreateTransformsPrintOpStats();
mlirPassManagerAddOwnedPass(pm, printOpStatPass); mlirPassManagerAddOwnedPass(pm, printOpStatPass);
MlirLogicalResult success = mlirPassManagerRun(pm, module); MlirLogicalResult success = mlirPassManagerRun(pm, module);
if (mlirLogicalResultIsFailure(success)) { if (mlirLogicalResultIsFailure(success)) {
fprintf(stderr, "Unexpected failure running pass manager.\n"); fprintf(stderr, "Unexpected failure running pass manager.\n");
exit(EXIT_FAILURE); exit(EXIT_FAILURE);
Show All 25 Lines"}"));
// clang-format on // clang-format on
if (mlirModuleIsNull(module)) if (mlirModuleIsNull(module))
exit(1); exit(1);
// Run the print-op-stats pass on functions under the top-level module: // Run the print-op-stats pass on functions under the top-level module:
// CHECK-LABEL: Operations encountered: // CHECK-LABEL: Operations encountered:
// CHECK: arith.addi , 1 // CHECK: arith.addi , 1
// CHECK: builtin.func , 1 // CHECK: builtin.func , 1
// CHECK: std.return , 1 // CHECK: func.return , 1
{ {
MlirPassManager pm = mlirPassManagerCreate(ctx); MlirPassManager pm = mlirPassManagerCreate(ctx);
MlirOpPassManager nestedFuncPm = mlirPassManagerGetNestedUnder( MlirOpPassManager nestedFuncPm = mlirPassManagerGetNestedUnder(
pm, mlirStringRefCreateFromCString("builtin.func")); pm, mlirStringRefCreateFromCString("builtin.func"));
MlirPass printOpStatPass = mlirCreateTransformsPrintOpStats(); MlirPass printOpStatPass = mlirCreateTransformsPrintOpStats();
mlirOpPassManagerAddOwnedPass(nestedFuncPm, printOpStatPass); mlirOpPassManagerAddOwnedPass(nestedFuncPm, printOpStatPass);
MlirLogicalResult success = mlirPassManagerRun(pm, module); MlirLogicalResult success = mlirPassManagerRun(pm, module);
if (mlirLogicalResultIsFailure(success)) if (mlirLogicalResultIsFailure(success))
exit(2); exit(2);
mlirPassManagerDestroy(pm); mlirPassManagerDestroy(pm);
} }
// Run the print-op-stats pass on functions under the nested module: // Run the print-op-stats pass on functions under the nested module:
// CHECK-LABEL: Operations encountered: // CHECK-LABEL: Operations encountered:
// CHECK: arith.addf , 1 // CHECK: arith.addf , 1
// CHECK: builtin.func , 1 // CHECK: builtin.func , 1
// CHECK: std.return , 1 // CHECK: func.return , 1
{ {
MlirPassManager pm = mlirPassManagerCreate(ctx); MlirPassManager pm = mlirPassManagerCreate(ctx);
MlirOpPassManager nestedModulePm = mlirPassManagerGetNestedUnder( MlirOpPassManager nestedModulePm = mlirPassManagerGetNestedUnder(
pm, mlirStringRefCreateFromCString("builtin.module")); pm, mlirStringRefCreateFromCString("builtin.module"));
MlirOpPassManager nestedFuncPm = mlirOpPassManagerGetNestedUnder( MlirOpPassManager nestedFuncPm = mlirOpPassManagerGetNestedUnder(
nestedModulePm, mlirStringRefCreateFromCString("builtin.func")); nestedModulePm, mlirStringRefCreateFromCString("builtin.func"));
MlirPass printOpStatPass = mlirCreateTransformsPrintOpStats(); MlirPass printOpStatPass = mlirCreateTransformsPrintOpStats();
mlirOpPassManagerAddOwnedPass(nestedFuncPm, printOpStatPass); mlirOpPassManagerAddOwnedPass(nestedFuncPm, printOpStatPass);
▲ Show 20 Lines • Show All 90 Lines • Show Last 20 Lines

mlir/test/Conversion/ArithmeticToLLVM/arith-to-llvm.mlir

Show First 20 Lines • Show All 318 Lines • ▼ Show 20 Lines
// ----- // -----
// CHECK-LABEL: @index_vector // CHECK-LABEL: @index_vector
func @index_vector(%arg0: vector<4xindex>) { func @index_vector(%arg0: vector<4xindex>) {
// CHECK: %[[CST:.*]] = llvm.mlir.constant(dense<[0, 1, 2, 3]> : vector<4xindex>) : vector<4xi64> // CHECK: %[[CST:.*]] = llvm.mlir.constant(dense<[0, 1, 2, 3]> : vector<4xindex>) : vector<4xi64>
%0 = arith.constant dense<[0, 1, 2, 3]> : vector<4xindex> %0 = arith.constant dense<[0, 1, 2, 3]> : vector<4xindex>
// CHECK: %[[V:.*]] = llvm.add %{{.*}}, %[[CST]] : vector<4xi64> // CHECK: %[[V:.*]] = llvm.add %{{.*}}, %[[CST]] : vector<4xi64>
%1 = arith.addi %arg0, %0 : vector<4xindex> %1 = arith.addi %arg0, %0 : vector<4xindex>
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: @bitcast_1d // CHECK-LABEL: @bitcast_1d
func @bitcast_1d(%arg0: vector<2xf32>) { func @bitcast_1d(%arg0: vector<2xf32>) {
// CHECK: llvm.bitcast %{{.*}} : vector<2xf32> to vector<2xi32> // CHECK: llvm.bitcast %{{.*}} : vector<2xf32> to vector<2xi32>
arith.bitcast %arg0 : vector<2xf32> to vector<2xi32> arith.bitcast %arg0 : vector<2xf32> to vector<2xi32>
return return
} }
// ----- // -----
// CHECK-LABEL: func @cmpf_2dvector( // CHECK-LABEL: func @cmpf_2dvector(
func @cmpf_2dvector(%arg0 : vector<4x3xf32>, %arg1 : vector<4x3xf32>) { func @cmpf_2dvector(%arg0 : vector<4x3xf32>, %arg1 : vector<4x3xf32>) {
// CHECK: %[[ARG0:.*]] = builtin.unrealized_conversion_cast // CHECK: %[[ARG0:.*]] = builtin.unrealized_conversion_cast
// CHECK: %[[ARG1:.*]] = builtin.unrealized_conversion_cast // CHECK: %[[ARG1:.*]] = builtin.unrealized_conversion_cast
// CHECK: %[[EXTRACT1:.*]] = llvm.extractvalue %[[ARG0]][0] : !llvm.array<4 x vector<3xf32>> // CHECK: %[[EXTRACT1:.*]] = llvm.extractvalue %[[ARG0]][0] : !llvm.array<4 x vector<3xf32>>
// CHECK: %[[EXTRACT2:.*]] = llvm.extractvalue %[[ARG1]][0] : !llvm.array<4 x vector<3xf32>> // CHECK: %[[EXTRACT2:.*]] = llvm.extractvalue %[[ARG1]][0] : !llvm.array<4 x vector<3xf32>>
// CHECK: %[[CMP:.*]] = llvm.fcmp "olt" %[[EXTRACT1]], %[[EXTRACT2]] : vector<3xf32> // CHECK: %[[CMP:.*]] = llvm.fcmp "olt" %[[EXTRACT1]], %[[EXTRACT2]] : vector<3xf32>
// CHECK: %[[INSERT:.*]] = llvm.insertvalue %[[CMP]], %2[0] : !llvm.array<4 x vector<3xi1>> // CHECK: %[[INSERT:.*]] = llvm.insertvalue %[[CMP]], %2[0] : !llvm.array<4 x vector<3xi1>>
%0 = arith.cmpf olt, %arg0, %arg1 : vector<4x3xf32> %0 = arith.cmpf olt, %arg0, %arg1 : vector<4x3xf32>
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @cmpi_0dvector( // CHECK-LABEL: func @cmpi_0dvector(
func @cmpi_0dvector(%arg0 : vector<i32>, %arg1 : vector<i32>) { func @cmpi_0dvector(%arg0 : vector<i32>, %arg1 : vector<i32>) {
// CHECK: %[[ARG0:.*]] = builtin.unrealized_conversion_cast // CHECK: %[[ARG0:.*]] = builtin.unrealized_conversion_cast
// CHECK: %[[ARG1:.*]] = builtin.unrealized_conversion_cast // CHECK: %[[ARG1:.*]] = builtin.unrealized_conversion_cast
// CHECK: %[[CMP:.*]] = llvm.icmp "ult" %[[ARG0]], %[[ARG1]] : vector<1xi32> // CHECK: %[[CMP:.*]] = llvm.icmp "ult" %[[ARG0]], %[[ARG1]] : vector<1xi32>
%0 = arith.cmpi ult, %arg0, %arg1 : vector<i32> %0 = arith.cmpi ult, %arg0, %arg1 : vector<i32>
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @cmpi_2dvector( // CHECK-LABEL: func @cmpi_2dvector(
func @cmpi_2dvector(%arg0 : vector<4x3xi32>, %arg1 : vector<4x3xi32>) { func @cmpi_2dvector(%arg0 : vector<4x3xi32>, %arg1 : vector<4x3xi32>) {
// CHECK: %[[ARG0:.*]] = builtin.unrealized_conversion_cast // CHECK: %[[ARG0:.*]] = builtin.unrealized_conversion_cast
// CHECK: %[[ARG1:.*]] = builtin.unrealized_conversion_cast // CHECK: %[[ARG1:.*]] = builtin.unrealized_conversion_cast
// CHECK: %[[EXTRACT1:.*]] = llvm.extractvalue %[[ARG0]][0] : !llvm.array<4 x vector<3xi32>> // CHECK: %[[EXTRACT1:.*]] = llvm.extractvalue %[[ARG0]][0] : !llvm.array<4 x vector<3xi32>>
// CHECK: %[[EXTRACT2:.*]] = llvm.extractvalue %[[ARG1]][0] : !llvm.array<4 x vector<3xi32>> // CHECK: %[[EXTRACT2:.*]] = llvm.extractvalue %[[ARG1]][0] : !llvm.array<4 x vector<3xi32>>
// CHECK: %[[CMP:.*]] = llvm.icmp "ult" %[[EXTRACT1]], %[[EXTRACT2]] : vector<3xi32> // CHECK: %[[CMP:.*]] = llvm.icmp "ult" %[[EXTRACT1]], %[[EXTRACT2]] : vector<3xi32>
// CHECK: %[[INSERT:.*]] = llvm.insertvalue %[[CMP]], %2[0] : !llvm.array<4 x vector<3xi1>> // CHECK: %[[INSERT:.*]] = llvm.insertvalue %[[CMP]], %2[0] : !llvm.array<4 x vector<3xi1>>
%0 = arith.cmpi ult, %arg0, %arg1 : vector<4x3xi32> %0 = arith.cmpi ult, %arg0, %arg1 : vector<4x3xi32>
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: @select // CHECK-LABEL: @select
func @select(%arg0 : i1, %arg1 : i32, %arg2 : i32) -> i32 { func @select(%arg0 : i1, %arg1 : i32, %arg2 : i32) -> i32 {
// CHECK: = llvm.select %arg0, %arg1, %arg2 : i1, i32 // CHECK: = llvm.select %arg0, %arg1, %arg2 : i1, i32
%0 = arith.select %arg0, %arg1, %arg2 : i32 %0 = arith.select %arg0, %arg1, %arg2 : i32
return %0 : i32 return %0 : i32
} }

mlir/test/Conversion/ArithmeticToLLVM/convert-nd-vector-to-llvmir.mlir

Show First 20 Lines • Show All 202 Lines • ▼ Show 20 Linesfunc @select_2d(%arg0 : vector<4x3xi1>, %arg1 : vector<4x3xi32>, %arg2 : vector<4x3xi32>) {
// CHECK: %[[ARG1:.*]] = builtin.unrealized_conversion_cast %arg1 // CHECK: %[[ARG1:.*]] = builtin.unrealized_conversion_cast %arg1
// CHECK: %[[ARG2:.*]] = builtin.unrealized_conversion_cast %arg2 // CHECK: %[[ARG2:.*]] = builtin.unrealized_conversion_cast %arg2
// CHECK: %[[EXTRACT1:.*]] = llvm.extractvalue %[[ARG0]][0] : !llvm.array<4 x vector<3xi1>> // CHECK: %[[EXTRACT1:.*]] = llvm.extractvalue %[[ARG0]][0] : !llvm.array<4 x vector<3xi1>>
// CHECK: %[[EXTRACT2:.*]] = llvm.extractvalue %[[ARG1]][0] : !llvm.array<4 x vector<3xi32>> // CHECK: %[[EXTRACT2:.*]] = llvm.extractvalue %[[ARG1]][0] : !llvm.array<4 x vector<3xi32>>
// CHECK: %[[EXTRACT3:.*]] = llvm.extractvalue %[[ARG2]][0] : !llvm.array<4 x vector<3xi32>> // CHECK: %[[EXTRACT3:.*]] = llvm.extractvalue %[[ARG2]][0] : !llvm.array<4 x vector<3xi32>>
// CHECK: %[[SELECT:.*]] = llvm.select %[[EXTRACT1]], %[[EXTRACT2]], %[[EXTRACT3]] : vector<3xi1>, vector<3xi32> // CHECK: %[[SELECT:.*]] = llvm.select %[[EXTRACT1]], %[[EXTRACT2]], %[[EXTRACT3]] : vector<3xi1>, vector<3xi32>
// CHECK: %[[INSERT:.*]] = llvm.insertvalue %[[SELECT]], %{{.*}}[0] : !llvm.array<4 x vector<3xi32>> // CHECK: %[[INSERT:.*]] = llvm.insertvalue %[[SELECT]], %{{.*}}[0] : !llvm.array<4 x vector<3xi32>>
%0 = arith.select %arg0, %arg1, %arg2 : vector<4x3xi1>, vector<4x3xi32> %0 = arith.select %arg0, %arg1, %arg2 : vector<4x3xi1>, vector<4x3xi32>
std.return func.return
} }

mlir/test/Conversion/GPUToNVVM/gpu-to-nvvm.mlir

Show First 20 Lines • Show All 43 Lines • ▼ Show 20 Lines builtin.func @gpu_index_ops()
%gDimX = gpu.grid_dim x %gDimX = gpu.grid_dim x
// CHECK: = nvvm.read.ptx.sreg.nctaid.y : i32 // CHECK: = nvvm.read.ptx.sreg.nctaid.y : i32
// CHECK: = llvm.sext %{{.*}} : i32 to i64 // CHECK: = llvm.sext %{{.*}} : i32 to i64
%gDimY = gpu.grid_dim y %gDimY = gpu.grid_dim y
// CHECK: = nvvm.read.ptx.sreg.nctaid.z : i32 // CHECK: = nvvm.read.ptx.sreg.nctaid.z : i32
// CHECK: = llvm.sext %{{.*}} : i32 to i64 // CHECK: = llvm.sext %{{.*}} : i32 to i64
%gDimZ = gpu.grid_dim z %gDimZ = gpu.grid_dim z
std.return %tIdX, %tIdY, %tIdZ, %bDimX, %bDimY, %bDimZ, func.return %tIdX, %tIdY, %tIdZ, %bDimX, %bDimY, %bDimZ,
%bIdX, %bIdY, %bIdZ, %gDimX, %gDimY, %gDimZ %bIdX, %bIdY, %bIdZ, %gDimX, %gDimY, %gDimZ
: index, index, index, index, index, index, : index, index, index, index, index, index,
index, index, index, index, index, index index, index, index, index, index, index
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK-LABEL: func @gpu_index_comp // CHECK-LABEL: func @gpu_index_comp
// CHECK32-LABEL: func @gpu_index_comp // CHECK32-LABEL: func @gpu_index_comp
builtin.func @gpu_index_comp(%idx : index) -> index { builtin.func @gpu_index_comp(%idx : index) -> index {
// CHECK: = llvm.add %{{.*}}, %{{.*}} : i64 // CHECK: = llvm.add %{{.*}}, %{{.*}} : i64
// CHECK32: = llvm.add %{{.*}}, %{{.*}} : i32 // CHECK32: = llvm.add %{{.*}}, %{{.*}} : i32
%0 = arith.addi %idx, %idx : index %0 = arith.addi %idx, %idx : index
// CHECK: llvm.return %{{.*}} : i64 // CHECK: llvm.return %{{.*}} : i64
// CHECK32: llvm.return %{{.*}} : i32 // CHECK32: llvm.return %{{.*}} : i32
std.return %0 : index func.return %0 : index
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK-LABEL: func @gpu_all_reduce_op() // CHECK-LABEL: func @gpu_all_reduce_op()
gpu.func @gpu_all_reduce_op() { gpu.func @gpu_all_reduce_op() {
▲ Show 20 Lines • Show All 56 Lines • ▼ Show 20 Lines builtin.func @gpu_shuffle() -> (f32, f32, f32, f32) {
// CHECK: llvm.extractvalue %[[#SHFL]][0 : index] : !llvm.struct<(f32, i1)> // CHECK: llvm.extractvalue %[[#SHFL]][0 : index] : !llvm.struct<(f32, i1)>
// CHECK: llvm.extractvalue %[[#SHFL]][1 : index] : !llvm.struct<(f32, i1)> // CHECK: llvm.extractvalue %[[#SHFL]][1 : index] : !llvm.struct<(f32, i1)>
%shflu, %predu = gpu.shuffle up %arg0, %arg1, %arg2 : f32 %shflu, %predu = gpu.shuffle up %arg0, %arg1, %arg2 : f32
// CHECK: nvvm.shfl.sync down {{.*}} {return_value_and_is_valid} : f32 -> !llvm.struct<(f32, i1)> // CHECK: nvvm.shfl.sync down {{.*}} {return_value_and_is_valid} : f32 -> !llvm.struct<(f32, i1)>
%shfld, %predd = gpu.shuffle down %arg0, %arg1, %arg2 : f32 %shfld, %predd = gpu.shuffle down %arg0, %arg1, %arg2 : f32
// CHECK: nvvm.shfl.sync idx {{.*}} {return_value_and_is_valid} : f32 -> !llvm.struct<(f32, i1)> // CHECK: nvvm.shfl.sync idx {{.*}} {return_value_and_is_valid} : f32 -> !llvm.struct<(f32, i1)>
%shfli, %predi = gpu.shuffle idx %arg0, %arg1, %arg2 : f32 %shfli, %predi = gpu.shuffle idx %arg0, %arg1, %arg2 : f32
std.return %shfl, %shflu, %shfld, %shfli : f32, f32,f32, f32 func.return %shfl, %shflu, %shfld, %shfli : f32, f32,f32, f32
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK-LABEL: func @gpu_sync() // CHECK-LABEL: func @gpu_sync()
builtin.func @gpu_sync() { builtin.func @gpu_sync() {
// CHECK: nvvm.barrier0 // CHECK: nvvm.barrier0
gpu.barrier gpu.barrier
std.return func.return
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__nv_fabsf(f32) -> f32 // CHECK: llvm.func @__nv_fabsf(f32) -> f32
// CHECK: llvm.func @__nv_fabs(f64) -> f64 // CHECK: llvm.func @__nv_fabs(f64) -> f64
// CHECK-LABEL: func @gpu_fabs // CHECK-LABEL: func @gpu_fabs
builtin.func @gpu_fabs(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_fabs(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.abs %arg_f32 : f32 %result32 = math.abs %arg_f32 : f32
// CHECK: llvm.call @__nv_fabsf(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__nv_fabsf(%{{.*}}) : (f32) -> f32
%result64 = math.abs %arg_f64 : f64 %result64 = math.abs %arg_f64 : f64
// CHECK: llvm.call @__nv_fabs(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__nv_fabs(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__nv_ceilf(f32) -> f32 // CHECK: llvm.func @__nv_ceilf(f32) -> f32
// CHECK: llvm.func @__nv_ceil(f64) -> f64 // CHECK: llvm.func @__nv_ceil(f64) -> f64
// CHECK-LABEL: func @gpu_ceil // CHECK-LABEL: func @gpu_ceil
builtin.func @gpu_ceil(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_ceil(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.ceil %arg_f32 : f32 %result32 = math.ceil %arg_f32 : f32
// CHECK: llvm.call @__nv_ceilf(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__nv_ceilf(%{{.*}}) : (f32) -> f32
%result64 = math.ceil %arg_f64 : f64 %result64 = math.ceil %arg_f64 : f64
// CHECK: llvm.call @__nv_ceil(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__nv_ceil(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__nv_floorf(f32) -> f32 // CHECK: llvm.func @__nv_floorf(f32) -> f32
// CHECK: llvm.func @__nv_floor(f64) -> f64 // CHECK: llvm.func @__nv_floor(f64) -> f64
// CHECK-LABEL: func @gpu_floor // CHECK-LABEL: func @gpu_floor
builtin.func @gpu_floor(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_floor(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.floor %arg_f32 : f32 %result32 = math.floor %arg_f32 : f32
// CHECK: llvm.call @__nv_floorf(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__nv_floorf(%{{.*}}) : (f32) -> f32
%result64 = math.floor %arg_f64 : f64 %result64 = math.floor %arg_f64 : f64
// CHECK: llvm.call @__nv_floor(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__nv_floor(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__nv_cosf(f32) -> f32 // CHECK: llvm.func @__nv_cosf(f32) -> f32
// CHECK: llvm.func @__nv_cos(f64) -> f64 // CHECK: llvm.func @__nv_cos(f64) -> f64
// CHECK-LABEL: func @gpu_cos // CHECK-LABEL: func @gpu_cos
builtin.func @gpu_cos(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_cos(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.cos %arg_f32 : f32 %result32 = math.cos %arg_f32 : f32
// CHECK: llvm.call @__nv_cosf(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__nv_cosf(%{{.*}}) : (f32) -> f32
%result64 = math.cos %arg_f64 : f64 %result64 = math.cos %arg_f64 : f64
// CHECK: llvm.call @__nv_cos(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__nv_cos(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__nv_expf(f32) -> f32 // CHECK: llvm.func @__nv_expf(f32) -> f32
// CHECK: llvm.func @__nv_exp(f64) -> f64 // CHECK: llvm.func @__nv_exp(f64) -> f64
// CHECK-LABEL: func @gpu_exp // CHECK-LABEL: func @gpu_exp
builtin.func @gpu_exp(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_exp(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.exp %arg_f32 : f32 %result32 = math.exp %arg_f32 : f32
// CHECK: llvm.call @__nv_expf(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__nv_expf(%{{.*}}) : (f32) -> f32
%result64 = math.exp %arg_f64 : f64 %result64 = math.exp %arg_f64 : f64
// CHECK: llvm.call @__nv_exp(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__nv_exp(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__nv_exp2f(f32) -> f32 // CHECK: llvm.func @__nv_exp2f(f32) -> f32
// CHECK: llvm.func @__nv_exp2(f64) -> f64 // CHECK: llvm.func @__nv_exp2(f64) -> f64
// CHECK-LABEL: func @gpu_exp2 // CHECK-LABEL: func @gpu_exp2
builtin.func @gpu_exp2(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_exp2(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.exp2 %arg_f32 : f32 %result32 = math.exp2 %arg_f32 : f32
// CHECK: llvm.call @__nv_exp2f(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__nv_exp2f(%{{.*}}) : (f32) -> f32
%result64 = math.exp2 %arg_f64 : f64 %result64 = math.exp2 %arg_f64 : f64
// CHECK: llvm.call @__nv_exp2(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__nv_exp2(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__nv_logf(f32) -> f32 // CHECK: llvm.func @__nv_logf(f32) -> f32
// CHECK: llvm.func @__nv_log(f64) -> f64 // CHECK: llvm.func @__nv_log(f64) -> f64
// CHECK-LABEL: func @gpu_log // CHECK-LABEL: func @gpu_log
builtin.func @gpu_log(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_log(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.log %arg_f32 : f32 %result32 = math.log %arg_f32 : f32
// CHECK: llvm.call @__nv_logf(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__nv_logf(%{{.*}}) : (f32) -> f32
%result64 = math.log %arg_f64 : f64 %result64 = math.log %arg_f64 : f64
// CHECK: llvm.call @__nv_log(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__nv_log(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__nv_log10f(f32) -> f32 // CHECK: llvm.func @__nv_log10f(f32) -> f32
// CHECK: llvm.func @__nv_log10(f64) -> f64 // CHECK: llvm.func @__nv_log10(f64) -> f64
// CHECK-LABEL: func @gpu_log10 // CHECK-LABEL: func @gpu_log10
builtin.func @gpu_log10(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_log10(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.log10 %arg_f32 : f32 %result32 = math.log10 %arg_f32 : f32
// CHECK: llvm.call @__nv_log10f(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__nv_log10f(%{{.*}}) : (f32) -> f32
%result64 = math.log10 %arg_f64 : f64 %result64 = math.log10 %arg_f64 : f64
// CHECK: llvm.call @__nv_log10(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__nv_log10(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__nv_log1pf(f32) -> f32 // CHECK: llvm.func @__nv_log1pf(f32) -> f32
// CHECK: llvm.func @__nv_log1p(f64) -> f64 // CHECK: llvm.func @__nv_log1p(f64) -> f64
// CHECK-LABEL: func @gpu_log1p // CHECK-LABEL: func @gpu_log1p
builtin.func @gpu_log1p(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_log1p(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.log1p %arg_f32 : f32 %result32 = math.log1p %arg_f32 : f32
// CHECK: llvm.call @__nv_log1pf(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__nv_log1pf(%{{.*}}) : (f32) -> f32
%result64 = math.log1p %arg_f64 : f64 %result64 = math.log1p %arg_f64 : f64
// CHECK: llvm.call @__nv_log1p(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__nv_log1p(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__nv_log2f(f32) -> f32 // CHECK: llvm.func @__nv_log2f(f32) -> f32
// CHECK: llvm.func @__nv_log2(f64) -> f64 // CHECK: llvm.func @__nv_log2(f64) -> f64
// CHECK-LABEL: func @gpu_log2 // CHECK-LABEL: func @gpu_log2
builtin.func @gpu_log2(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_log2(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.log2 %arg_f32 : f32 %result32 = math.log2 %arg_f32 : f32
// CHECK: llvm.call @__nv_log2f(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__nv_log2f(%{{.*}}) : (f32) -> f32
%result64 = math.log2 %arg_f64 : f64 %result64 = math.log2 %arg_f64 : f64
// CHECK: llvm.call @__nv_log2(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__nv_log2(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__nv_sinf(f32) -> f32 // CHECK: llvm.func @__nv_sinf(f32) -> f32
// CHECK: llvm.func @__nv_sin(f64) -> f64 // CHECK: llvm.func @__nv_sin(f64) -> f64
// CHECK-LABEL: func @gpu_sin // CHECK-LABEL: func @gpu_sin
builtin.func @gpu_sin(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_sin(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.sin %arg_f32 : f32 %result32 = math.sin %arg_f32 : f32
// CHECK: llvm.call @__nv_sinf(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__nv_sinf(%{{.*}}) : (f32) -> f32
%result64 = math.sin %arg_f64 : f64 %result64 = math.sin %arg_f64 : f64
// CHECK: llvm.call @__nv_sin(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__nv_sin(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__nv_tanhf(f32) -> f32 // CHECK: llvm.func @__nv_tanhf(f32) -> f32
// CHECK: llvm.func @__nv_tanh(f64) -> f64 // CHECK: llvm.func @__nv_tanh(f64) -> f64
// CHECK-LABEL: func @gpu_tanh // CHECK-LABEL: func @gpu_tanh
builtin.func @gpu_tanh(%arg_f16 : f16, %arg_f32 : f32, %arg_f64 : f64) -> (f16, f32, f64) { builtin.func @gpu_tanh(%arg_f16 : f16, %arg_f32 : f32, %arg_f64 : f64) -> (f16, f32, f64) {
%result16 = math.tanh %arg_f16 : f16 %result16 = math.tanh %arg_f16 : f16
// CHECK: llvm.fpext %{{.*}} : f16 to f32 // CHECK: llvm.fpext %{{.*}} : f16 to f32
// CHECK-NEXT: llvm.call @__nv_tanhf(%{{.*}}) : (f32) -> f32 // CHECK-NEXT: llvm.call @__nv_tanhf(%{{.*}}) : (f32) -> f32
// CHECK-NEXT: llvm.fptrunc %{{.*}} : f32 to f16 // CHECK-NEXT: llvm.fptrunc %{{.*}} : f32 to f16
%result32 = math.tanh %arg_f32 : f32 %result32 = math.tanh %arg_f32 : f32
// CHECK: llvm.call @__nv_tanhf(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__nv_tanhf(%{{.*}}) : (f32) -> f32
%result64 = math.tanh %arg_f64 : f64 %result64 = math.tanh %arg_f64 : f64
// CHECK: llvm.call @__nv_tanh(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__nv_tanh(%{{.*}}) : (f64) -> f64
std.return %result16, %result32, %result64 : f16, f32, f64 func.return %result16, %result32, %result64 : f16, f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__nv_rsqrtf(f32) -> f32 // CHECK: llvm.func @__nv_rsqrtf(f32) -> f32
// CHECK: llvm.func @__nv_rsqrt(f64) -> f64 // CHECK: llvm.func @__nv_rsqrt(f64) -> f64
// CHECK-LABEL: func @gpu_rsqrt // CHECK-LABEL: func @gpu_rsqrt
builtin.func @gpu_rsqrt(%arg_f16 : f16, %arg_f32 : f32, %arg_f64 : f64) builtin.func @gpu_rsqrt(%arg_f16 : f16, %arg_f32 : f32, %arg_f64 : f64)
-> (f16, f32, f64) { -> (f16, f32, f64) {
%result16 = math.rsqrt %arg_f16 : f16 %result16 = math.rsqrt %arg_f16 : f16
// CHECK: llvm.fpext %{{.*}} : f16 to f32 // CHECK: llvm.fpext %{{.*}} : f16 to f32
// CHECK-NEXT: llvm.call @__nv_rsqrtf(%{{.*}}) : (f32) -> f32 // CHECK-NEXT: llvm.call @__nv_rsqrtf(%{{.*}}) : (f32) -> f32
// CHECK-NEXT: llvm.fptrunc %{{.*}} : f32 to f16 // CHECK-NEXT: llvm.fptrunc %{{.*}} : f32 to f16
%result32 = math.rsqrt %arg_f32 : f32 %result32 = math.rsqrt %arg_f32 : f32
// CHECK: llvm.call @__nv_rsqrtf(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__nv_rsqrtf(%{{.*}}) : (f32) -> f32
%result64 = math.rsqrt %arg_f64 : f64 %result64 = math.rsqrt %arg_f64 : f64
// CHECK: llvm.call @__nv_rsqrt(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__nv_rsqrt(%{{.*}}) : (f64) -> f64
std.return %result16, %result32, %result64 : f16, f32, f64 func.return %result16, %result32, %result64 : f16, f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__nv_sqrtf(f32) -> f32 // CHECK: llvm.func @__nv_sqrtf(f32) -> f32
// CHECK: llvm.func @__nv_sqrt(f64) -> f64 // CHECK: llvm.func @__nv_sqrt(f64) -> f64
// CHECK-LABEL: func @gpu_sqrt // CHECK-LABEL: func @gpu_sqrt
builtin.func @gpu_sqrt(%arg_f16 : f16, %arg_f32 : f32, %arg_f64 : f64) builtin.func @gpu_sqrt(%arg_f16 : f16, %arg_f32 : f32, %arg_f64 : f64)
-> (f16, f32, f64) { -> (f16, f32, f64) {
%result16 = math.sqrt %arg_f16 : f16 %result16 = math.sqrt %arg_f16 : f16
// CHECK: llvm.fpext %{{.*}} : f16 to f32 // CHECK: llvm.fpext %{{.*}} : f16 to f32
// CHECK-NEXT: llvm.call @__nv_sqrtf(%{{.*}}) : (f32) -> f32 // CHECK-NEXT: llvm.call @__nv_sqrtf(%{{.*}}) : (f32) -> f32
// CHECK-NEXT: llvm.fptrunc %{{.*}} : f32 to f16 // CHECK-NEXT: llvm.fptrunc %{{.*}} : f32 to f16
%result32 = math.sqrt %arg_f32 : f32 %result32 = math.sqrt %arg_f32 : f32
// CHECK: llvm.call @__nv_sqrtf(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__nv_sqrtf(%{{.*}}) : (f32) -> f32
%result64 = math.sqrt %arg_f64 : f64 %result64 = math.sqrt %arg_f64 : f64
// CHECK: llvm.call @__nv_sqrt(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__nv_sqrt(%{{.*}}) : (f64) -> f64
std.return %result16, %result32, %result64 : f16, f32, f64 func.return %result16, %result32, %result64 : f16, f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__nv_atanf(f32) -> f32 // CHECK: llvm.func @__nv_atanf(f32) -> f32
// CHECK: llvm.func @__nv_atan(f64) -> f64 // CHECK: llvm.func @__nv_atan(f64) -> f64
// CHECK-LABEL: func @gpu_atan // CHECK-LABEL: func @gpu_atan
builtin.func @gpu_atan(%arg_f16 : f16, %arg_f32 : f32, %arg_f64 : f64) builtin.func @gpu_atan(%arg_f16 : f16, %arg_f32 : f32, %arg_f64 : f64)
-> (f16, f32, f64) { -> (f16, f32, f64) {
%result16 = math.atan %arg_f16 : f16 %result16 = math.atan %arg_f16 : f16
// CHECK: llvm.fpext %{{.*}} : f16 to f32 // CHECK: llvm.fpext %{{.*}} : f16 to f32
// CHECK-NEXT: llvm.call @__nv_atanf(%{{.*}}) : (f32) -> f32 // CHECK-NEXT: llvm.call @__nv_atanf(%{{.*}}) : (f32) -> f32
// CHECK-NEXT: llvm.fptrunc %{{.*}} : f32 to f16 // CHECK-NEXT: llvm.fptrunc %{{.*}} : f32 to f16
%result32 = math.atan %arg_f32 : f32 %result32 = math.atan %arg_f32 : f32
// CHECK: llvm.call @__nv_atanf(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__nv_atanf(%{{.*}}) : (f32) -> f32
%result64 = math.atan %arg_f64 : f64 %result64 = math.atan %arg_f64 : f64
// CHECK: llvm.call @__nv_atan(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__nv_atan(%{{.*}}) : (f64) -> f64
std.return %result16, %result32, %result64 : f16, f32, f64 func.return %result16, %result32, %result64 : f16, f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__nv_atan2f(f32, f32) -> f32 // CHECK: llvm.func @__nv_atan2f(f32, f32) -> f32
// CHECK: llvm.func @__nv_atan2(f64, f64) -> f64 // CHECK: llvm.func @__nv_atan2(f64, f64) -> f64
// CHECK-LABEL: func @gpu_atan2 // CHECK-LABEL: func @gpu_atan2
builtin.func @gpu_atan2(%arg_f16 : f16, %arg_f32 : f32, %arg_f64 : f64) builtin.func @gpu_atan2(%arg_f16 : f16, %arg_f32 : f32, %arg_f64 : f64)
-> (f16, f32, f64) { -> (f16, f32, f64) {
%result16 = math.atan2 %arg_f16, %arg_f16 : f16 %result16 = math.atan2 %arg_f16, %arg_f16 : f16
// CHECK: llvm.fpext %{{.*}} : f16 to f32 // CHECK: llvm.fpext %{{.*}} : f16 to f32
// CHECK: llvm.fpext %{{.*}} : f16 to f32 // CHECK: llvm.fpext %{{.*}} : f16 to f32
// CHECK-NEXT: llvm.call @__nv_atan2f(%{{.*}}) : (f32, f32) -> f32 // CHECK-NEXT: llvm.call @__nv_atan2f(%{{.*}}) : (f32, f32) -> f32
// CHECK-NEXT: llvm.fptrunc %{{.*}} : f32 to f16 // CHECK-NEXT: llvm.fptrunc %{{.*}} : f32 to f16
%result32 = math.atan2 %arg_f32, %arg_f32 : f32 %result32 = math.atan2 %arg_f32, %arg_f32 : f32
// CHECK: llvm.call @__nv_atan2f(%{{.*}}) : (f32, f32) -> f32 // CHECK: llvm.call @__nv_atan2f(%{{.*}}) : (f32, f32) -> f32
%result64 = math.atan2 %arg_f64, %arg_f64 : f64 %result64 = math.atan2 %arg_f64, %arg_f64 : f64
// CHECK: llvm.call @__nv_atan2(%{{.*}}) : (f64, f64) -> f64 // CHECK: llvm.call @__nv_atan2(%{{.*}}) : (f64, f64) -> f64
std.return %result16, %result32, %result64 : f16, f32, f64 func.return %result16, %result32, %result64 : f16, f32, f64
} }
} }
// ----- // -----
// Test that we handled properly operation with SymbolTable other than module op // Test that we handled properly operation with SymbolTable other than module op
gpu.module @test_module { gpu.module @test_module {
"test.symbol_scope"() ({ "test.symbol_scope"() ({
// CHECK: test.symbol_scope // CHECK: test.symbol_scope
// CHECK: llvm.func @__nv_expf(f32) -> f32 // CHECK: llvm.func @__nv_expf(f32) -> f32
// CHECK: llvm.func @__nv_exp(f64) -> f64 // CHECK: llvm.func @__nv_exp(f64) -> f64
// CHECK-LABEL: func @gpu_exp // CHECK-LABEL: func @gpu_exp
builtin.func @gpu_exp(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_exp(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.exp %arg_f32 : f32 %result32 = math.exp %arg_f32 : f32
// CHECK: llvm.call @__nv_expf(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__nv_expf(%{{.*}}) : (f32) -> f32
%result64 = math.exp %arg_f64 : f64 %result64 = math.exp %arg_f64 : f64
// CHECK: llvm.call @__nv_exp(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__nv_exp(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
"test.finish" () : () -> () "test.finish" () : () -> ()
}) : () -> () }) : () -> ()
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__nv_expm1f(f32) -> f32 // CHECK: llvm.func @__nv_expm1f(f32) -> f32
// CHECK: llvm.func @__nv_expm1(f64) -> f64 // CHECK: llvm.func @__nv_expm1(f64) -> f64
// CHECK-LABEL: func @gpu_expm1 // CHECK-LABEL: func @gpu_expm1
builtin.func @gpu_expm1(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_expm1(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.expm1 %arg_f32 : f32 %result32 = math.expm1 %arg_f32 : f32
// CHECK: llvm.call @__nv_expm1f(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__nv_expm1f(%{{.*}}) : (f32) -> f32
%result64 = math.expm1 %arg_f64 : f64 %result64 = math.expm1 %arg_f64 : f64
// CHECK: llvm.call @__nv_expm1(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__nv_expm1(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__nv_powf(f32, f32) -> f32 // CHECK: llvm.func @__nv_powf(f32, f32) -> f32
// CHECK: llvm.func @__nv_pow(f64, f64) -> f64 // CHECK: llvm.func @__nv_pow(f64, f64) -> f64
// CHECK-LABEL: func @gpu_pow // CHECK-LABEL: func @gpu_pow
builtin.func @gpu_pow(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_pow(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.powf %arg_f32, %arg_f32 : f32 %result32 = math.powf %arg_f32, %arg_f32 : f32
// CHECK: llvm.call @__nv_powf(%{{.*}}, %{{.*}}) : (f32, f32) -> f32 // CHECK: llvm.call @__nv_powf(%{{.*}}, %{{.*}}) : (f32, f32) -> f32
%result64 = math.powf %arg_f64, %arg_f64 : f64 %result64 = math.powf %arg_f64, %arg_f64 : f64
// CHECK: llvm.call @__nv_pow(%{{.*}}, %{{.*}}) : (f64, f64) -> f64 // CHECK: llvm.call @__nv_pow(%{{.*}}, %{{.*}}) : (f64, f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK-LABEL: @kernel_func // CHECK-LABEL: @kernel_func
// CHECK: attributes // CHECK: attributes
Show All 39 Lines

mlir/test/Conversion/GPUToROCDL/gpu-to-rocdl.mlir

Show First 20 Lines • Show All 43 Lines • ▼ Show 20 Lines builtin.func @gpu_index_ops()
%gDimX = gpu.grid_dim x %gDimX = gpu.grid_dim x
// CHECK: rocdl.grid.dim.y : i32 // CHECK: rocdl.grid.dim.y : i32
// CHECK: = llvm.sext %{{.*}} : i32 to i64 // CHECK: = llvm.sext %{{.*}} : i32 to i64
%gDimY = gpu.grid_dim y %gDimY = gpu.grid_dim y
// CHECK: rocdl.grid.dim.z : i32 // CHECK: rocdl.grid.dim.z : i32
// CHECK: = llvm.sext %{{.*}} : i32 to i64 // CHECK: = llvm.sext %{{.*}} : i32 to i64
%gDimZ = gpu.grid_dim z %gDimZ = gpu.grid_dim z
std.return %tIdX, %tIdY, %tIdZ, %bDimX, %bDimY, %bDimZ, func.return %tIdX, %tIdY, %tIdZ, %bDimX, %bDimY, %bDimZ,
%bIdX, %bIdY, %bIdZ, %gDimX, %gDimY, %gDimZ %bIdX, %bIdY, %bIdZ, %gDimX, %gDimY, %gDimZ
: index, index, index, index, index, index, : index, index, index, index, index, index,
index, index, index, index, index, index index, index, index, index, index, index
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK-LABEL: func @gpu_index_comp // CHECK-LABEL: func @gpu_index_comp
// CHECK32-LABEL: func @gpu_index_comp // CHECK32-LABEL: func @gpu_index_comp
builtin.func @gpu_index_comp(%idx : index) -> index { builtin.func @gpu_index_comp(%idx : index) -> index {
// CHECK: = llvm.add %{{.*}}, %{{.*}} : i64 // CHECK: = llvm.add %{{.*}}, %{{.*}} : i64
// CHECK32: = llvm.add %{{.*}}, %{{.*}} : i32 // CHECK32: = llvm.add %{{.*}}, %{{.*}} : i32
%0 = arith.addi %idx, %idx : index %0 = arith.addi %idx, %idx : index
// CHECK: llvm.return %{{.*}} : i64 // CHECK: llvm.return %{{.*}} : i64
// CHECK32: llvm.return %{{.*}} : i32 // CHECK32: llvm.return %{{.*}} : i32
std.return %0 : index func.return %0 : index
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK-LABEL: func @gpu_sync() // CHECK-LABEL: func @gpu_sync()
builtin.func @gpu_sync() { builtin.func @gpu_sync() {
// CHECK: rocdl.barrier // CHECK: rocdl.barrier
gpu.barrier gpu.barrier
std.return func.return
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__ocml_fabs_f32(f32) -> f32 // CHECK: llvm.func @__ocml_fabs_f32(f32) -> f32
// CHECK: llvm.func @__ocml_fabs_f64(f64) -> f64 // CHECK: llvm.func @__ocml_fabs_f64(f64) -> f64
// CHECK-LABEL: func @gpu_fabs // CHECK-LABEL: func @gpu_fabs
builtin.func @gpu_fabs(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_fabs(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.abs %arg_f32 : f32 %result32 = math.abs %arg_f32 : f32
// CHECK: llvm.call @__ocml_fabs_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_fabs_f32(%{{.*}}) : (f32) -> f32
%result64 = math.abs %arg_f64 : f64 %result64 = math.abs %arg_f64 : f64
// CHECK: llvm.call @__ocml_fabs_f64(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__ocml_fabs_f64(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__ocml_ceil_f32(f32) -> f32 // CHECK: llvm.func @__ocml_ceil_f32(f32) -> f32
// CHECK: llvm.func @__ocml_ceil_f64(f64) -> f64 // CHECK: llvm.func @__ocml_ceil_f64(f64) -> f64
// CHECK-LABEL: func @gpu_ceil // CHECK-LABEL: func @gpu_ceil
builtin.func @gpu_ceil(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_ceil(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.ceil %arg_f32 : f32 %result32 = math.ceil %arg_f32 : f32
// CHECK: llvm.call @__ocml_ceil_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_ceil_f32(%{{.*}}) : (f32) -> f32
%result64 = math.ceil %arg_f64 : f64 %result64 = math.ceil %arg_f64 : f64
// CHECK: llvm.call @__ocml_ceil_f64(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__ocml_ceil_f64(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__ocml_floor_f32(f32) -> f32 // CHECK: llvm.func @__ocml_floor_f32(f32) -> f32
// CHECK: llvm.func @__ocml_floor_f64(f64) -> f64 // CHECK: llvm.func @__ocml_floor_f64(f64) -> f64
// CHECK-LABEL: func @gpu_floor // CHECK-LABEL: func @gpu_floor
builtin.func @gpu_floor(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_floor(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.floor %arg_f32 : f32 %result32 = math.floor %arg_f32 : f32
// CHECK: llvm.call @__ocml_floor_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_floor_f32(%{{.*}}) : (f32) -> f32
%result64 = math.floor %arg_f64 : f64 %result64 = math.floor %arg_f64 : f64
// CHECK: llvm.call @__ocml_floor_f64(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__ocml_floor_f64(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__ocml_cos_f32(f32) -> f32 // CHECK: llvm.func @__ocml_cos_f32(f32) -> f32
// CHECK: llvm.func @__ocml_cos_f64(f64) -> f64 // CHECK: llvm.func @__ocml_cos_f64(f64) -> f64
// CHECK-LABEL: func @gpu_cos // CHECK-LABEL: func @gpu_cos
builtin.func @gpu_cos(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_cos(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.cos %arg_f32 : f32 %result32 = math.cos %arg_f32 : f32
// CHECK: llvm.call @__ocml_cos_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_cos_f32(%{{.*}}) : (f32) -> f32
%result64 = math.cos %arg_f64 : f64 %result64 = math.cos %arg_f64 : f64
// CHECK: llvm.call @__ocml_cos_f64(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__ocml_cos_f64(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__ocml_exp_f32(f32) -> f32 // CHECK: llvm.func @__ocml_exp_f32(f32) -> f32
// CHECK: llvm.func @__ocml_exp_f64(f64) -> f64 // CHECK: llvm.func @__ocml_exp_f64(f64) -> f64
// CHECK-LABEL: func @gpu_exp // CHECK-LABEL: func @gpu_exp
builtin.func @gpu_exp(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_exp(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%exp_f32 = math.exp %arg_f32 : f32 %exp_f32 = math.exp %arg_f32 : f32
// CHECK: llvm.call @__ocml_exp_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_exp_f32(%{{.*}}) : (f32) -> f32
%result32 = math.exp %exp_f32 : f32 %result32 = math.exp %exp_f32 : f32
// CHECK: llvm.call @__ocml_exp_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_exp_f32(%{{.*}}) : (f32) -> f32
%result64 = math.exp %arg_f64 : f64 %result64 = math.exp %arg_f64 : f64
// CHECK: llvm.call @__ocml_exp_f64(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__ocml_exp_f64(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__ocml_exp2_f32(f32) -> f32 // CHECK: llvm.func @__ocml_exp2_f32(f32) -> f32
// CHECK: llvm.func @__ocml_exp2_f64(f64) -> f64 // CHECK: llvm.func @__ocml_exp2_f64(f64) -> f64
// CHECK-LABEL: func @gpu_exp2 // CHECK-LABEL: func @gpu_exp2
builtin.func @gpu_exp2(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_exp2(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%exp2_f32 = math.exp2 %arg_f32 : f32 %exp2_f32 = math.exp2 %arg_f32 : f32
// CHECK: llvm.call @__ocml_exp2_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_exp2_f32(%{{.*}}) : (f32) -> f32
%result32 = math.exp2 %exp2_f32 : f32 %result32 = math.exp2 %exp2_f32 : f32
// CHECK: llvm.call @__ocml_exp2_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_exp2_f32(%{{.*}}) : (f32) -> f32
%result64 = math.exp2 %arg_f64 : f64 %result64 = math.exp2 %arg_f64 : f64
// CHECK: llvm.call @__ocml_exp2_f64(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__ocml_exp2_f64(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
// Test that we handled properly operation with SymbolTable other than module op // Test that we handled properly operation with SymbolTable other than module op
gpu.module @test_module { gpu.module @test_module {
"test.symbol_scope"() ({ "test.symbol_scope"() ({
// CHECK: test.symbol_scope // CHECK: test.symbol_scope
// CHECK: llvm.func @__ocml_exp_f32(f32) -> f32 // CHECK: llvm.func @__ocml_exp_f32(f32) -> f32
// CHECK: llvm.func @__ocml_exp_f64(f64) -> f64 // CHECK: llvm.func @__ocml_exp_f64(f64) -> f64
// CHECK-LABEL: func @gpu_exp // CHECK-LABEL: func @gpu_exp
builtin.func @gpu_exp(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_exp(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%exp_f32 = math.exp %arg_f32 : f32 %exp_f32 = math.exp %arg_f32 : f32
// CHECK: llvm.call @__ocml_exp_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_exp_f32(%{{.*}}) : (f32) -> f32
%result32 = math.exp %exp_f32 : f32 %result32 = math.exp %exp_f32 : f32
// CHECK: llvm.call @__ocml_exp_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_exp_f32(%{{.*}}) : (f32) -> f32
%result64 = math.exp %arg_f64 : f64 %result64 = math.exp %arg_f64 : f64
// CHECK: llvm.call @__ocml_exp_f64(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__ocml_exp_f64(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
"test.finish" () : () -> () "test.finish" () : () -> ()
}) : () -> () }) : () -> ()
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__ocml_expm1_f32(f32) -> f32 // CHECK: llvm.func @__ocml_expm1_f32(f32) -> f32
// CHECK: llvm.func @__ocml_expm1_f64(f64) -> f64 // CHECK: llvm.func @__ocml_expm1_f64(f64) -> f64
// CHECK-LABEL: func @gpu_expm1 // CHECK-LABEL: func @gpu_expm1
builtin.func @gpu_expm1(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_expm1(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%expm1_f32 = math.expm1 %arg_f32 : f32 %expm1_f32 = math.expm1 %arg_f32 : f32
// CHECK: llvm.call @__ocml_expm1_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_expm1_f32(%{{.*}}) : (f32) -> f32
%result32 = math.expm1 %expm1_f32 : f32 %result32 = math.expm1 %expm1_f32 : f32
// CHECK: llvm.call @__ocml_expm1_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_expm1_f32(%{{.*}}) : (f32) -> f32
%result64 = math.expm1 %arg_f64 : f64 %result64 = math.expm1 %arg_f64 : f64
// CHECK: llvm.call @__ocml_expm1_f64(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__ocml_expm1_f64(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__ocml_log_f32(f32) -> f32 // CHECK: llvm.func @__ocml_log_f32(f32) -> f32
// CHECK: llvm.func @__ocml_log_f64(f64) -> f64 // CHECK: llvm.func @__ocml_log_f64(f64) -> f64
// CHECK-LABEL: func @gpu_log // CHECK-LABEL: func @gpu_log
builtin.func @gpu_log(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_log(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.log %arg_f32 : f32 %result32 = math.log %arg_f32 : f32
// CHECK: llvm.call @__ocml_log_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_log_f32(%{{.*}}) : (f32) -> f32
%result64 = math.log %arg_f64 : f64 %result64 = math.log %arg_f64 : f64
// CHECK: llvm.call @__ocml_log_f64(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__ocml_log_f64(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__ocml_log1p_f32(f32) -> f32 // CHECK: llvm.func @__ocml_log1p_f32(f32) -> f32
// CHECK: llvm.func @__ocml_log1p_f64(f64) -> f64 // CHECK: llvm.func @__ocml_log1p_f64(f64) -> f64
// CHECK-LABEL: func @gpu_log1p // CHECK-LABEL: func @gpu_log1p
builtin.func @gpu_log1p(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_log1p(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.log1p %arg_f32 : f32 %result32 = math.log1p %arg_f32 : f32
// CHECK: llvm.call @__ocml_log1p_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_log1p_f32(%{{.*}}) : (f32) -> f32
%result64 = math.log1p %arg_f64 : f64 %result64 = math.log1p %arg_f64 : f64
// CHECK: llvm.call @__ocml_log1p_f64(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__ocml_log1p_f64(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__ocml_log10_f32(f32) -> f32 // CHECK: llvm.func @__ocml_log10_f32(f32) -> f32
// CHECK: llvm.func @__ocml_log10_f64(f64) -> f64 // CHECK: llvm.func @__ocml_log10_f64(f64) -> f64
// CHECK-LABEL: func @gpu_log10 // CHECK-LABEL: func @gpu_log10
builtin.func @gpu_log10(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_log10(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.log10 %arg_f32 : f32 %result32 = math.log10 %arg_f32 : f32
// CHECK: llvm.call @__ocml_log10_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_log10_f32(%{{.*}}) : (f32) -> f32
%result64 = math.log10 %arg_f64 : f64 %result64 = math.log10 %arg_f64 : f64
// CHECK: llvm.call @__ocml_log10_f64(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__ocml_log10_f64(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__ocml_log2_f32(f32) -> f32 // CHECK: llvm.func @__ocml_log2_f32(f32) -> f32
// CHECK: llvm.func @__ocml_log2_f64(f64) -> f64 // CHECK: llvm.func @__ocml_log2_f64(f64) -> f64
// CHECK-LABEL: func @gpu_log2 // CHECK-LABEL: func @gpu_log2
builtin.func @gpu_log2(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_log2(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.log2 %arg_f32 : f32 %result32 = math.log2 %arg_f32 : f32
// CHECK: llvm.call @__ocml_log2_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_log2_f32(%{{.*}}) : (f32) -> f32
%result64 = math.log2 %arg_f64 : f64 %result64 = math.log2 %arg_f64 : f64
// CHECK: llvm.call @__ocml_log2_f64(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__ocml_log2_f64(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__ocml_rsqrt_f32(f32) -> f32 // CHECK: llvm.func @__ocml_rsqrt_f32(f32) -> f32
// CHECK: llvm.func @__ocml_rsqrt_f64(f64) -> f64 // CHECK: llvm.func @__ocml_rsqrt_f64(f64) -> f64
// CHECK-LABEL: func @gpu_rsqrt // CHECK-LABEL: func @gpu_rsqrt
builtin.func @gpu_rsqrt(%arg_f16 : f16, %arg_f32 : f32, %arg_f64 : f64) builtin.func @gpu_rsqrt(%arg_f16 : f16, %arg_f32 : f32, %arg_f64 : f64)
-> (f16, f32, f64) { -> (f16, f32, f64) {
%result16 = math.rsqrt %arg_f16 : f16 %result16 = math.rsqrt %arg_f16 : f16
// CHECK: llvm.fpext %{{.*}} : f16 to f32 // CHECK: llvm.fpext %{{.*}} : f16 to f32
// CHECK-NEXT: llvm.call @__ocml_rsqrt_f32(%{{.*}}) : (f32) -> f32 // CHECK-NEXT: llvm.call @__ocml_rsqrt_f32(%{{.*}}) : (f32) -> f32
// CHECK-NEXT: llvm.fptrunc %{{.*}} : f32 to f16 // CHECK-NEXT: llvm.fptrunc %{{.*}} : f32 to f16
%result32 = math.rsqrt %arg_f32 : f32 %result32 = math.rsqrt %arg_f32 : f32
// CHECK: llvm.call @__ocml_rsqrt_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_rsqrt_f32(%{{.*}}) : (f32) -> f32
%result64 = math.rsqrt %arg_f64 : f64 %result64 = math.rsqrt %arg_f64 : f64
// CHECK: llvm.call @__ocml_rsqrt_f64(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__ocml_rsqrt_f64(%{{.*}}) : (f64) -> f64
std.return %result16, %result32, %result64 : f16, f32, f64 func.return %result16, %result32, %result64 : f16, f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__ocml_sqrt_f32(f32) -> f32 // CHECK: llvm.func @__ocml_sqrt_f32(f32) -> f32
// CHECK: llvm.func @__ocml_sqrt_f64(f64) -> f64 // CHECK: llvm.func @__ocml_sqrt_f64(f64) -> f64
// CHECK-LABEL: func @gpu_sqrt // CHECK-LABEL: func @gpu_sqrt
builtin.func @gpu_sqrt(%arg_f16 : f16, %arg_f32 : f32, %arg_f64 : f64) builtin.func @gpu_sqrt(%arg_f16 : f16, %arg_f32 : f32, %arg_f64 : f64)
-> (f16, f32, f64) { -> (f16, f32, f64) {
%result16 = math.sqrt %arg_f16 : f16 %result16 = math.sqrt %arg_f16 : f16
// CHECK: llvm.fpext %{{.*}} : f16 to f32 // CHECK: llvm.fpext %{{.*}} : f16 to f32
// CHECK-NEXT: llvm.call @__ocml_sqrt_f32(%{{.*}}) : (f32) -> f32 // CHECK-NEXT: llvm.call @__ocml_sqrt_f32(%{{.*}}) : (f32) -> f32
// CHECK-NEXT: llvm.fptrunc %{{.*}} : f32 to f16 // CHECK-NEXT: llvm.fptrunc %{{.*}} : f32 to f16
%result32 = math.sqrt %arg_f32 : f32 %result32 = math.sqrt %arg_f32 : f32
// CHECK: llvm.call @__ocml_sqrt_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_sqrt_f32(%{{.*}}) : (f32) -> f32
%result64 = math.sqrt %arg_f64 : f64 %result64 = math.sqrt %arg_f64 : f64
// CHECK: llvm.call @__ocml_sqrt_f64(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__ocml_sqrt_f64(%{{.*}}) : (f64) -> f64
std.return %result16, %result32, %result64 : f16, f32, f64 func.return %result16, %result32, %result64 : f16, f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__ocml_tanh_f32(f32) -> f32 // CHECK: llvm.func @__ocml_tanh_f32(f32) -> f32
// CHECK: llvm.func @__ocml_tanh_f64(f64) -> f64 // CHECK: llvm.func @__ocml_tanh_f64(f64) -> f64
// CHECK-LABEL: func @gpu_tanh // CHECK-LABEL: func @gpu_tanh
builtin.func @gpu_tanh(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_tanh(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.tanh %arg_f32 : f32 %result32 = math.tanh %arg_f32 : f32
// CHECK: llvm.call @__ocml_tanh_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_tanh_f32(%{{.*}}) : (f32) -> f32
%result64 = math.tanh %arg_f64 : f64 %result64 = math.tanh %arg_f64 : f64
// CHECK: llvm.call @__ocml_tanh_f64(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__ocml_tanh_f64(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__ocml_atan_f32(f32) -> f32 // CHECK: llvm.func @__ocml_atan_f32(f32) -> f32
// CHECK: llvm.func @__ocml_atan_f64(f64) -> f64 // CHECK: llvm.func @__ocml_atan_f64(f64) -> f64
// CHECK-LABEL: func @gpu_atan // CHECK-LABEL: func @gpu_atan
builtin.func @gpu_atan(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_atan(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.atan %arg_f32 : f32 %result32 = math.atan %arg_f32 : f32
// CHECK: llvm.call @__ocml_atan_f32(%{{.*}}) : (f32) -> f32 // CHECK: llvm.call @__ocml_atan_f32(%{{.*}}) : (f32) -> f32
%result64 = math.atan %arg_f64 : f64 %result64 = math.atan %arg_f64 : f64
// CHECK: llvm.call @__ocml_atan_f64(%{{.*}}) : (f64) -> f64 // CHECK: llvm.call @__ocml_atan_f64(%{{.*}}) : (f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__ocml_atan2_f32(f32, f32) -> f32 // CHECK: llvm.func @__ocml_atan2_f32(f32, f32) -> f32
// CHECK: llvm.func @__ocml_atan2_f64(f64, f64) -> f64 // CHECK: llvm.func @__ocml_atan2_f64(f64, f64) -> f64
// CHECK-LABEL: func @gpu_atan2 // CHECK-LABEL: func @gpu_atan2
builtin.func @gpu_atan2(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_atan2(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.atan2 %arg_f32, %arg_f32 : f32 %result32 = math.atan2 %arg_f32, %arg_f32 : f32
// CHECK: llvm.call @__ocml_atan2_f32(%{{.*}}) : (f32, f32) -> f32 // CHECK: llvm.call @__ocml_atan2_f32(%{{.*}}) : (f32, f32) -> f32
%result64 = math.atan2 %arg_f64, %arg_f64 : f64 %result64 = math.atan2 %arg_f64, %arg_f64 : f64
// CHECK: llvm.call @__ocml_atan2_f64(%{{.*}}) : (f64, f64) -> f64 // CHECK: llvm.call @__ocml_atan2_f64(%{{.*}}) : (f64, f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK: llvm.func @__ocml_pow_f32(f32, f32) -> f32 // CHECK: llvm.func @__ocml_pow_f32(f32, f32) -> f32
// CHECK: llvm.func @__ocml_pow_f64(f64, f64) -> f64 // CHECK: llvm.func @__ocml_pow_f64(f64, f64) -> f64
// CHECK-LABEL: func @gpu_pow // CHECK-LABEL: func @gpu_pow
builtin.func @gpu_pow(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) { builtin.func @gpu_pow(%arg_f32 : f32, %arg_f64 : f64) -> (f32, f64) {
%result32 = math.powf %arg_f32, %arg_f32 : f32 %result32 = math.powf %arg_f32, %arg_f32 : f32
// CHECK: llvm.call @__ocml_pow_f32(%{{.*}}, %{{.*}}) : (f32, f32) -> f32 // CHECK: llvm.call @__ocml_pow_f32(%{{.*}}, %{{.*}}) : (f32, f32) -> f32
%result64 = math.powf %arg_f64, %arg_f64 : f64 %result64 = math.powf %arg_f64, %arg_f64 : f64
// CHECK: llvm.call @__ocml_pow_f64(%{{.*}}, %{{.*}}) : (f64, f64) -> f64 // CHECK: llvm.call @__ocml_pow_f64(%{{.*}}, %{{.*}}) : (f64, f64) -> f64
std.return %result32, %result64 : f32, f64 func.return %result32, %result64 : f32, f64
} }
} }
// ----- // -----
gpu.module @test_module { gpu.module @test_module {
// CHECK-LABEL: @kernel_func // CHECK-LABEL: @kernel_func
// CHECK: attributes // CHECK: attributes
// CHECK: gpu.kernel // CHECK: gpu.kernel
// CHECK: rocdl.kernel // CHECK: rocdl.kernel
gpu.func @kernel_func() kernel { gpu.func @kernel_func() kernel {
gpu.return gpu.return
} }
} }

mlir/test/Conversion/MathToLLVM/math-to-llvm.mlir

// RUN: mlir-opt %s -split-input-file -convert-math-to-llvm | FileCheck %s // RUN: mlir-opt %s -split-input-file -convert-math-to-llvm | FileCheck %s
// CHECK-LABEL: @ops // CHECK-LABEL: @ops
func @ops(%arg0: f32, %arg1: f32, %arg2: i32, %arg3: i32, %arg4: f64) { func @ops(%arg0: f32, %arg1: f32, %arg2: i32, %arg3: i32, %arg4: f64) {
// CHECK: = "llvm.intr.exp"(%{{.*}}) : (f32) -> f32 // CHECK: = "llvm.intr.exp"(%{{.*}}) : (f32) -> f32
%13 = math.exp %arg0 : f32 %13 = math.exp %arg0 : f32
// CHECK: = "llvm.intr.exp2"(%{{.*}}) : (f32) -> f32 // CHECK: = "llvm.intr.exp2"(%{{.*}}) : (f32) -> f32
%14 = math.exp2 %arg0 : f32 %14 = math.exp2 %arg0 : f32
// CHECK: = "llvm.intr.sqrt"(%{{.*}}) : (f32) -> f32 // CHECK: = "llvm.intr.sqrt"(%{{.*}}) : (f32) -> f32
%19 = math.sqrt %arg0 : f32 %19 = math.sqrt %arg0 : f32
// CHECK: = "llvm.intr.sqrt"(%{{.*}}) : (f64) -> f64 // CHECK: = "llvm.intr.sqrt"(%{{.*}}) : (f64) -> f64
%20 = math.sqrt %arg4 : f64 %20 = math.sqrt %arg4 : f64
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @log1p( // CHECK-LABEL: func @log1p(
// CHECK-SAME: f32 // CHECK-SAME: f32
func @log1p(%arg0 : f32) { func @log1p(%arg0 : f32) {
// CHECK: %[[ONE:.*]] = llvm.mlir.constant(1.000000e+00 : f32) : f32 // CHECK: %[[ONE:.*]] = llvm.mlir.constant(1.000000e+00 : f32) : f32
// CHECK: %[[ADD:.*]] = llvm.fadd %[[ONE]], %arg0 : f32 // CHECK: %[[ADD:.*]] = llvm.fadd %[[ONE]], %arg0 : f32
// CHECK: %[[LOG:.*]] = "llvm.intr.log"(%[[ADD]]) : (f32) -> f32 // CHECK: %[[LOG:.*]] = "llvm.intr.log"(%[[ADD]]) : (f32) -> f32
%0 = math.log1p %arg0 : f32 %0 = math.log1p %arg0 : f32
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @log1p_2dvector( // CHECK-LABEL: func @log1p_2dvector(
func @log1p_2dvector(%arg0 : vector<4x3xf32>) { func @log1p_2dvector(%arg0 : vector<4x3xf32>) {
// CHECK: %[[EXTRACT:.*]] = llvm.extractvalue %{{.*}}[0] : !llvm.array<4 x vector<3xf32>> // CHECK: %[[EXTRACT:.*]] = llvm.extractvalue %{{.*}}[0] : !llvm.array<4 x vector<3xf32>>
// CHECK: %[[ONE:.*]] = llvm.mlir.constant(dense<1.000000e+00> : vector<3xf32>) : vector<3xf32> // CHECK: %[[ONE:.*]] = llvm.mlir.constant(dense<1.000000e+00> : vector<3xf32>) : vector<3xf32>
// CHECK: %[[ADD:.*]] = llvm.fadd %[[ONE]], %[[EXTRACT]] : vector<3xf32> // CHECK: %[[ADD:.*]] = llvm.fadd %[[ONE]], %[[EXTRACT]] : vector<3xf32>
// CHECK: %[[LOG:.*]] = "llvm.intr.log"(%[[ADD]]) : (vector<3xf32>) -> vector<3xf32> // CHECK: %[[LOG:.*]] = "llvm.intr.log"(%[[ADD]]) : (vector<3xf32>) -> vector<3xf32>
// CHECK: %[[INSERT:.*]] = llvm.insertvalue %[[LOG]], %{{.*}}[0] : !llvm.array<4 x vector<3xf32>> // CHECK: %[[INSERT:.*]] = llvm.insertvalue %[[LOG]], %{{.*}}[0] : !llvm.array<4 x vector<3xf32>>
%0 = math.log1p %arg0 : vector<4x3xf32> %0 = math.log1p %arg0 : vector<4x3xf32>
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @expm1( // CHECK-LABEL: func @expm1(
// CHECK-SAME: f32 // CHECK-SAME: f32
func @expm1(%arg0 : f32) { func @expm1(%arg0 : f32) {
// CHECK: %[[ONE:.*]] = llvm.mlir.constant(1.000000e+00 : f32) : f32 // CHECK: %[[ONE:.*]] = llvm.mlir.constant(1.000000e+00 : f32) : f32
// CHECK: %[[EXP:.*]] = "llvm.intr.exp"(%arg0) : (f32) -> f32 // CHECK: %[[EXP:.*]] = "llvm.intr.exp"(%arg0) : (f32) -> f32
// CHECK: %[[SUB:.*]] = llvm.fsub %[[EXP]], %[[ONE]] : f32 // CHECK: %[[SUB:.*]] = llvm.fsub %[[EXP]], %[[ONE]] : f32
%0 = math.expm1 %arg0 : f32 %0 = math.expm1 %arg0 : f32
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @rsqrt( // CHECK-LABEL: func @rsqrt(
// CHECK-SAME: f32 // CHECK-SAME: f32
func @rsqrt(%arg0 : f32) { func @rsqrt(%arg0 : f32) {
// CHECK: %[[ONE:.*]] = llvm.mlir.constant(1.000000e+00 : f32) : f32 // CHECK: %[[ONE:.*]] = llvm.mlir.constant(1.000000e+00 : f32) : f32
// CHECK: %[[SQRT:.*]] = "llvm.intr.sqrt"(%arg0) : (f32) -> f32 // CHECK: %[[SQRT:.*]] = "llvm.intr.sqrt"(%arg0) : (f32) -> f32
// CHECK: %[[DIV:.*]] = llvm.fdiv %[[ONE]], %[[SQRT]] : f32 // CHECK: %[[DIV:.*]] = llvm.fdiv %[[ONE]], %[[SQRT]] : f32
%0 = math.rsqrt %arg0 : f32 %0 = math.rsqrt %arg0 : f32
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @sine( // CHECK-LABEL: func @sine(
// CHECK-SAME: f32 // CHECK-SAME: f32
func @sine(%arg0 : f32) { func @sine(%arg0 : f32) {
// CHECK: "llvm.intr.sin"(%arg0) : (f32) -> f32 // CHECK: "llvm.intr.sin"(%arg0) : (f32) -> f32
%0 = math.sin %arg0 : f32 %0 = math.sin %arg0 : f32
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @ctlz( // CHECK-LABEL: func @ctlz(
// CHECK-SAME: i32 // CHECK-SAME: i32
func @ctlz(%arg0 : i32) { func @ctlz(%arg0 : i32) {
// CHECK: %[[ZERO:.+]] = llvm.mlir.constant(false) : i1 // CHECK: %[[ZERO:.+]] = llvm.mlir.constant(false) : i1
// CHECK: "llvm.intr.ctlz"(%arg0, %[[ZERO]]) : (i32, i1) -> i32 // CHECK: "llvm.intr.ctlz"(%arg0, %[[ZERO]]) : (i32, i1) -> i32
%0 = math.ctlz %arg0 : i32 %0 = math.ctlz %arg0 : i32
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @cttz( // CHECK-LABEL: func @cttz(
// CHECK-SAME: i32 // CHECK-SAME: i32
func @cttz(%arg0 : i32) { func @cttz(%arg0 : i32) {
// CHECK: %[[ZERO:.+]] = llvm.mlir.constant(false) : i1 // CHECK: %[[ZERO:.+]] = llvm.mlir.constant(false) : i1
// CHECK: "llvm.intr.cttz"(%arg0, %[[ZERO]]) : (i32, i1) -> i32 // CHECK: "llvm.intr.cttz"(%arg0, %[[ZERO]]) : (i32, i1) -> i32
%0 = math.cttz %arg0 : i32 %0 = math.cttz %arg0 : i32
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @cttz_vec( // CHECK-LABEL: func @cttz_vec(
// CHECK-SAME: i32 // CHECK-SAME: i32
func @cttz_vec(%arg0 : vector<4xi32>) { func @cttz_vec(%arg0 : vector<4xi32>) {
// CHECK: %[[ZERO:.+]] = llvm.mlir.constant(false) : i1 // CHECK: %[[ZERO:.+]] = llvm.mlir.constant(false) : i1
// CHECK: "llvm.intr.cttz"(%arg0, %[[ZERO]]) : (vector<4xi32>, i1) -> vector<4xi32> // CHECK: "llvm.intr.cttz"(%arg0, %[[ZERO]]) : (vector<4xi32>, i1) -> vector<4xi32>
%0 = math.cttz %arg0 : vector<4xi32> %0 = math.cttz %arg0 : vector<4xi32>
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @ctpop( // CHECK-LABEL: func @ctpop(
// CHECK-SAME: i32 // CHECK-SAME: i32
func @ctpop(%arg0 : i32) { func @ctpop(%arg0 : i32) {
// CHECK: "llvm.intr.ctpop"(%arg0) : (i32) -> i32 // CHECK: "llvm.intr.ctpop"(%arg0) : (i32) -> i32
%0 = math.ctpop %arg0 : i32 %0 = math.ctpop %arg0 : i32
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @ctpop_vector( // CHECK-LABEL: func @ctpop_vector(
// CHECK-SAME: vector<3xi32> // CHECK-SAME: vector<3xi32>
func @ctpop_vector(%arg0 : vector<3xi32>) { func @ctpop_vector(%arg0 : vector<3xi32>) {
// CHECK: "llvm.intr.ctpop"(%arg0) : (vector<3xi32>) -> vector<3xi32> // CHECK: "llvm.intr.ctpop"(%arg0) : (vector<3xi32>) -> vector<3xi32>
%0 = math.ctpop %arg0 : vector<3xi32> %0 = math.ctpop %arg0 : vector<3xi32>
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @rsqrt_double( // CHECK-LABEL: func @rsqrt_double(
// CHECK-SAME: f64 // CHECK-SAME: f64
func @rsqrt_double(%arg0 : f64) { func @rsqrt_double(%arg0 : f64) {
// CHECK: %[[ONE:.*]] = llvm.mlir.constant(1.000000e+00 : f64) : f64 // CHECK: %[[ONE:.*]] = llvm.mlir.constant(1.000000e+00 : f64) : f64
// CHECK: %[[SQRT:.*]] = "llvm.intr.sqrt"(%arg0) : (f64) -> f64 // CHECK: %[[SQRT:.*]] = "llvm.intr.sqrt"(%arg0) : (f64) -> f64
// CHECK: %[[DIV:.*]] = llvm.fdiv %[[ONE]], %[[SQRT]] : f64 // CHECK: %[[DIV:.*]] = llvm.fdiv %[[ONE]], %[[SQRT]] : f64
%0 = math.rsqrt %arg0 : f64 %0 = math.rsqrt %arg0 : f64
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @rsqrt_vector( // CHECK-LABEL: func @rsqrt_vector(
// CHECK-SAME: vector<4xf32> // CHECK-SAME: vector<4xf32>
func @rsqrt_vector(%arg0 : vector<4xf32>) { func @rsqrt_vector(%arg0 : vector<4xf32>) {
// CHECK: %[[ONE:.*]] = llvm.mlir.constant(dense<1.000000e+00> : vector<4xf32>) : vector<4xf32> // CHECK: %[[ONE:.*]] = llvm.mlir.constant(dense<1.000000e+00> : vector<4xf32>) : vector<4xf32>
// CHECK: %[[SQRT:.*]] = "llvm.intr.sqrt"(%arg0) : (vector<4xf32>) -> vector<4xf32> // CHECK: %[[SQRT:.*]] = "llvm.intr.sqrt"(%arg0) : (vector<4xf32>) -> vector<4xf32>
// CHECK: %[[DIV:.*]] = llvm.fdiv %[[ONE]], %[[SQRT]] : vector<4xf32> // CHECK: %[[DIV:.*]] = llvm.fdiv %[[ONE]], %[[SQRT]] : vector<4xf32>
%0 = math.rsqrt %arg0 : vector<4xf32> %0 = math.rsqrt %arg0 : vector<4xf32>
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @rsqrt_multidim_vector( // CHECK-LABEL: func @rsqrt_multidim_vector(
func @rsqrt_multidim_vector(%arg0 : vector<4x3xf32>) { func @rsqrt_multidim_vector(%arg0 : vector<4x3xf32>) {
// CHECK: %[[EXTRACT:.*]] = llvm.extractvalue %{{.*}}[0] : !llvm.array<4 x vector<3xf32>> // CHECK: %[[EXTRACT:.*]] = llvm.extractvalue %{{.*}}[0] : !llvm.array<4 x vector<3xf32>>
// CHECK: %[[ONE:.*]] = llvm.mlir.constant(dense<1.000000e+00> : vector<3xf32>) : vector<3xf32> // CHECK: %[[ONE:.*]] = llvm.mlir.constant(dense<1.000000e+00> : vector<3xf32>) : vector<3xf32>
// CHECK: %[[SQRT:.*]] = "llvm.intr.sqrt"(%[[EXTRACT]]) : (vector<3xf32>) -> vector<3xf32> // CHECK: %[[SQRT:.*]] = "llvm.intr.sqrt"(%[[EXTRACT]]) : (vector<3xf32>) -> vector<3xf32>
// CHECK: %[[DIV:.*]] = llvm.fdiv %[[ONE]], %[[SQRT]] : vector<3xf32> // CHECK: %[[DIV:.*]] = llvm.fdiv %[[ONE]], %[[SQRT]] : vector<3xf32>
// CHECK: %[[INSERT:.*]] = llvm.insertvalue %[[DIV]], %{{.*}}[0] : !llvm.array<4 x vector<3xf32>> // CHECK: %[[INSERT:.*]] = llvm.insertvalue %[[DIV]], %{{.*}}[0] : !llvm.array<4 x vector<3xf32>>
%0 = math.rsqrt %arg0 : vector<4x3xf32> %0 = math.rsqrt %arg0 : vector<4x3xf32>
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @powf( // CHECK-LABEL: func @powf(
// CHECK-SAME: f64 // CHECK-SAME: f64
func @powf(%arg0 : f64) { func @powf(%arg0 : f64) {
// CHECK: %[[POWF:.*]] = "llvm.intr.pow"(%arg0, %arg0) : (f64, f64) -> f64 // CHECK: %[[POWF:.*]] = "llvm.intr.pow"(%arg0, %arg0) : (f64, f64) -> f64
%0 = math.powf %arg0, %arg0 : f64 %0 = math.powf %arg0, %arg0 : f64
std.return func.return
} }

mlir/test/Conversion/MemRefToLLVM/memref-to-llvm.mlir

Show First 20 Lines • Show All 565 Lines • ▼ Show 20 Lines
// ----- // -----
// CHECK-LABEL: func @address_space( // CHECK-LABEL: func @address_space(
func @address_space(%arg0 : memref<32xf32, affine_map<(d0) -> (d0)>, 7>) { func @address_space(%arg0 : memref<32xf32, affine_map<(d0) -> (d0)>, 7>) {
%0 = memref.alloc() : memref<32xf32, affine_map<(d0) -> (d0)>, 5> %0 = memref.alloc() : memref<32xf32, affine_map<(d0) -> (d0)>, 5>
%1 = arith.constant 7 : index %1 = arith.constant 7 : index
// CHECK: llvm.load %{{.*}} : !llvm.ptr<f32, 5> // CHECK: llvm.load %{{.*}} : !llvm.ptr<f32, 5>
%2 = memref.load %0[%1] : memref<32xf32, affine_map<(d0) -> (d0)>, 5> %2 = memref.load %0[%1] : memref<32xf32, affine_map<(d0) -> (d0)>, 5>
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @transpose // CHECK-LABEL: func @transpose
// CHECK: llvm.mlir.undef : !llvm.struct<(ptr<f32>, ptr<f32>, i64, array<3 x i64>, array<3 x i64>)> // CHECK: llvm.mlir.undef : !llvm.struct<(ptr<f32>, ptr<f32>, i64, array<3 x i64>, array<3 x i64>)>
// CHECK: llvm.insertvalue {{.*}}[0] : !llvm.struct<(ptr<f32>, ptr<f32>, i64, array<3 x i64>, array<3 x i64>)> // CHECK: llvm.insertvalue {{.*}}[0] : !llvm.struct<(ptr<f32>, ptr<f32>, i64, array<3 x i64>, array<3 x i64>)>
// CHECK: llvm.insertvalue {{.*}}[1] : !llvm.struct<(ptr<f32>, ptr<f32>, i64, array<3 x i64>, array<3 x i64>)> // CHECK: llvm.insertvalue {{.*}}[1] : !llvm.struct<(ptr<f32>, ptr<f32>, i64, array<3 x i64>, array<3 x i64>)>
▲ Show 20 Lines • Show All 435 Lines • Show Last 20 Lines

mlir/test/Conversion/ShapeToStandard/shape-to-standard.mlir

Show First 20 Lines • Show All 68 Lines • ▼ Show 20 Linesfunc @get_extent_shape_of(%arg : tensor<2x3xf32>, %idx : index) -> index {
// CHECK: return %[[RESULT]] : index // CHECK: return %[[RESULT]] : index
%shape = shape.shape_of %arg : tensor<2x3xf32> -> tensor<?xindex> %shape = shape.shape_of %arg : tensor<2x3xf32> -> tensor<?xindex>
%result = shape.get_extent %shape, %idx : tensor<?xindex>, index -> index %result = shape.get_extent %shape, %idx : tensor<?xindex>, index -> index
return %result : index return %result : index
} }
// ----- // -----
// Express `get_extent` as `std.tensor.extract`. // Express `get_extent` as `tensor.extract`.
// CHECK-LABEL: @get_extent_from_extent_tensor // CHECK-LABEL: @get_extent_from_extent_tensor
// CHECK-SAME: (%[[EXTENTS:.*]]: tensor<?xindex>, %[[IDX:.*]]: index) -> index // CHECK-SAME: (%[[EXTENTS:.*]]: tensor<?xindex>, %[[IDX:.*]]: index) -> index
func @get_extent_from_extent_tensor(%extents : tensor<?xindex>, %idx : index) func @get_extent_from_extent_tensor(%extents : tensor<?xindex>, %idx : index)
-> index { -> index {
// CHECK: %[[RESULT:.*]] = tensor.extract %[[EXTENTS]][%[[IDX]]] : tensor<?xindex> // CHECK: %[[RESULT:.*]] = tensor.extract %[[EXTENTS]][%[[IDX]]] : tensor<?xindex>
// CHECK: return %[[RESULT]] : index // CHECK: return %[[RESULT]] : index
%result = shape.get_extent %extents, %idx : tensor<?xindex>, index -> index %result = shape.get_extent %extents, %idx : tensor<?xindex>, index -> index
return %result : index return %result : index
▲ Show 20 Lines • Show All 540 Lines • Show Last 20 Lines

mlir/test/Conversion/StandardToLLVM/standard-to-llvm.mlir

Show First 20 Lines • Show All 448 Lines • ▼ Show 20 Lines
// ----- // -----
// CHECK-LABEL: func @ceilf( // CHECK-LABEL: func @ceilf(
// CHECK-SAME: f32 // CHECK-SAME: f32
func @ceilf(%arg0 : f32) { func @ceilf(%arg0 : f32) {
// CHECK: "llvm.intr.ceil"(%arg0) : (f32) -> f32 // CHECK: "llvm.intr.ceil"(%arg0) : (f32) -> f32
%0 = math.ceil %arg0 : f32 %0 = math.ceil %arg0 : f32
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @floorf( // CHECK-LABEL: func @floorf(
// CHECK-SAME: f32 // CHECK-SAME: f32
func @floorf(%arg0 : f32) { func @floorf(%arg0 : f32) {
// CHECK: "llvm.intr.floor"(%arg0) : (f32) -> f32 // CHECK: "llvm.intr.floor"(%arg0) : (f32) -> f32
%0 = math.floor %arg0 : f32 %0 = math.floor %arg0 : f32
std.return func.return
} }
// ----- // -----
// Lowers `cf.assert` to a function call to `abort` if the assertion is violated. // Lowers `cf.assert` to a function call to `abort` if the assertion is violated.
// CHECK: llvm.func @abort() // CHECK: llvm.func @abort()
// CHECK-LABEL: @assert_test_function // CHECK-LABEL: @assert_test_function
// CHECK-SAME: (%[[ARG:.*]]: i1) // CHECK-SAME: (%[[ARG:.*]]: i1)
Show All 26 Lines
// CHECK-LABEL: func @fmaf( // CHECK-LABEL: func @fmaf(
// CHECK-SAME: %[[ARG0:.*]]: f32 // CHECK-SAME: %[[ARG0:.*]]: f32
// CHECK-SAME: %[[ARG1:.*]]: vector<4xf32> // CHECK-SAME: %[[ARG1:.*]]: vector<4xf32>
func @fmaf(%arg0: f32, %arg1: vector<4xf32>) { func @fmaf(%arg0: f32, %arg1: vector<4xf32>) {
// CHECK: %[[S:.*]] = "llvm.intr.fma"(%[[ARG0]], %[[ARG0]], %[[ARG0]]) : (f32, f32, f32) -> f32 // CHECK: %[[S:.*]] = "llvm.intr.fma"(%[[ARG0]], %[[ARG0]], %[[ARG0]]) : (f32, f32, f32) -> f32
%0 = math.fma %arg0, %arg0, %arg0 : f32 %0 = math.fma %arg0, %arg0, %arg0 : f32
// CHECK: %[[V:.*]] = "llvm.intr.fma"(%[[ARG1]], %[[ARG1]], %[[ARG1]]) : (vector<4xf32>, vector<4xf32>, vector<4xf32>) -> vector<4xf32> // CHECK: %[[V:.*]] = "llvm.intr.fma"(%[[ARG1]], %[[ARG1]], %[[ARG1]]) : (vector<4xf32>, vector<4xf32>, vector<4xf32>) -> vector<4xf32>
%1 = math.fma %arg1, %arg1, %arg1 : vector<4xf32> %1 = math.fma %arg1, %arg1, %arg1 : vector<4xf32>
std.return func.return
} }
// ----- // -----
// CHECK-LABEL: func @switchi8( // CHECK-LABEL: func @switchi8(
func @switchi8(%arg0 : i8) -> i32 { func @switchi8(%arg0 : i8) -> i32 {
cf.switch %arg0 : i8, [ cf.switch %arg0 : i8, [
default: ^bb1, default: ^bb1,
42: ^bb1, 42: ^bb1,
43: ^bb3 43: ^bb3
] ]
^bb1: ^bb1:
%c_1 = arith.constant 1 : i32 %c_1 = arith.constant 1 : i32
std.return %c_1 : i32 func.return %c_1 : i32
^bb3: ^bb3:
%c_42 = arith.constant 42 : i32 %c_42 = arith.constant 42 : i32
std.return %c_42: i32 func.return %c_42: i32
} }
// CHECK: llvm.switch %arg0 : i8, ^bb1 [ // CHECK: llvm.switch %arg0 : i8, ^bb1 [
// CHECK-NEXT: 42: ^bb1, // CHECK-NEXT: 42: ^bb1,
// CHECK-NEXT: 43: ^bb2 // CHECK-NEXT: 43: ^bb2
// CHECK-NEXT: ] // CHECK-NEXT: ]
// CHECK: ^bb1: // 2 preds: ^bb0, ^bb0 // CHECK: ^bb1: // 2 preds: ^bb0, ^bb0
// CHECK-NEXT: %[[E0:.+]] = llvm.mlir.constant(1 : i32) : i32 // CHECK-NEXT: %[[E0:.+]] = llvm.mlir.constant(1 : i32) : i32
// CHECK-NEXT: llvm.return %[[E0]] : i32 // CHECK-NEXT: llvm.return %[[E0]] : i32
// CHECK: ^bb2: // pred: ^bb0 // CHECK: ^bb2: // pred: ^bb0
// CHECK-NEXT: %[[E1:.+]] = llvm.mlir.constant(42 : i32) : i32 // CHECK-NEXT: %[[E1:.+]] = llvm.mlir.constant(42 : i32) : i32
// CHECK-NEXT: llvm.return %[[E1]] : i32 // CHECK-NEXT: llvm.return %[[E1]] : i32
// CHECK-NEXT: } // CHECK-NEXT: }

mlir/test/Conversion/StandardToSPIRV/std-ops-to-spirv.mlir

Show First 20 Lines • Show All 258 Lines • ▼ Show 20 Linesfunc @shift_vector(%arg0 : vector<4xi32>, %arg1 : vector<4xi32>) {
return return
} }
} // end module } // end module
// ----- // -----
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// std.cmpf // arith.cmpf
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
module attributes { module attributes {
spv.target_env = #spv.target_env<#spv.vce<v1.0, [], []>, {}> spv.target_env = #spv.target_env<#spv.vce<v1.0, [], []>, {}>
} { } {
// CHECK-LABEL: @cmpf // CHECK-LABEL: @cmpf
func @cmpf(%arg0 : f32, %arg1 : f32) { func @cmpf(%arg0 : f32, %arg1 : f32) {
▲ Show 20 Lines • Show All 67 Lines • ▼ Show 20 Linesfunc @cmpf(%arg0 : f32, %arg1 : f32) {
return return
} }
} // end module } // end module
// ----- // -----
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// std.cmpi // arith.cmpi
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
module attributes { module attributes {
spv.target_env = #spv.target_env<#spv.vce<v1.0, [], []>, {}> spv.target_env = #spv.target_env<#spv.vce<v1.0, [], []>, {}>
} { } {
// CHECK-LABEL: @cmpi // CHECK-LABEL: @cmpi
func @cmpi(%arg0 : i32, %arg1 : i32) { func @cmpi(%arg0 : i32, %arg1 : i32) {
▲ Show 20 Lines • Show All 484 Lines • ▼ Show 20 Linesfunc @sitofp(%arg0 : i64) -> f64 {
return %0: f64 return %0: f64
} }
} // end module } // end module
// ----- // -----
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// std.return // func.return
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
module attributes { module attributes {
spv.target_env = #spv.target_env<#spv.vce<v1.0, [], []>, {}> spv.target_env = #spv.target_env<#spv.vce<v1.0, [], []>, {}>
} { } {
// CHECK-LABEL: spv.func @return_one_val // CHECK-LABEL: spv.func @return_one_val
// CHECK-SAME: (%[[ARG:.+]]: f32) // CHECK-SAME: (%[[ARG:.+]]: f32)
▲ Show 20 Lines • Show All 42 Lines • Show Last 20 Lines

mlir/test/Dialect/Arithmetic/ops.mlir

Show First 20 Lines • Show All 897 Lines • ▼ Show 20 Lines
// CHECK-LABEL: test_constant // CHECK-LABEL: test_constant
func @test_constant() -> () { func @test_constant() -> () {
// CHECK: %c42_i32 = arith.constant 42 : i32 // CHECK: %c42_i32 = arith.constant 42 : i32
%0 = "arith.constant"(){value = 42 : i32} : () -> i32 %0 = "arith.constant"(){value = 42 : i32} : () -> i32
// CHECK: %c42_i32_0 = arith.constant 42 : i32 // CHECK: %c42_i32_0 = arith.constant 42 : i32
%1 = arith.constant 42 : i32 %1 = arith.constant 42 : i32
// CHECK: %c43 = arith.constant {crazy = "std.foo"} 43 : index // CHECK: %c43 = arith.constant {crazy = "func.foo"} 43 : index
%2 = arith.constant {crazy = "std.foo"} 43: index %2 = arith.constant {crazy = "func.foo"} 43: index
// CHECK: %cst = arith.constant 4.300000e+01 : bf16 // CHECK: %cst = arith.constant 4.300000e+01 : bf16
%3 = arith.constant 43.0 : bf16 %3 = arith.constant 43.0 : bf16
// CHECK: %cst_1 = arith.constant dense<0> : vector<4xi32> // CHECK: %cst_1 = arith.constant dense<0> : vector<4xi32>
%4 = arith.constant dense<0> : vector<4 x i32> %4 = arith.constant dense<0> : vector<4 x i32>
// CHECK: %cst_2 = arith.constant dense<0> : tensor<42xi32> // CHECK: %cst_2 = arith.constant dense<0> : tensor<42xi32>
Show All 39 Lines

mlir/test/Dialect/Tensor/invalid.mlir

Show First 20 Lines • Show All 85 Lines • ▼ Show 20 Linesfunc @tensor.generate(%m : index, %n : index)
} : tensor<?x3x?xf32> } : tensor<?x3x?xf32>
return %tnsr : tensor<?x3x?xf32> return %tnsr : tensor<?x3x?xf32>
} }
// ----- // -----
func @tensor.generate(%m : index, %n : index) func @tensor.generate(%m : index, %n : index)
-> tensor<?x3x?xf32> { -> tensor<?x3x?xf32> {
// expected-error @+2 {{op expects regions to end with 'tensor.yield', found 'std.return'}} // expected-error @+2 {{op expects regions to end with 'tensor.yield', found 'func.return'}}
// expected-note @+1 {{in custom textual format, the absence of terminator implies 'tensor.yield'}} // expected-note @+1 {{in custom textual format, the absence of terminator implies 'tensor.yield'}}
%tnsr = tensor.generate %m, %n { %tnsr = tensor.generate %m, %n {
^bb0(%i : index, %j : index, %k : index): ^bb0(%i : index, %j : index, %k : index):
%elem = arith.constant 8.0 : f32 %elem = arith.constant 8.0 : f32
return %elem : f32 return %elem : f32
} : tensor<?x3x?xf32> } : tensor<?x3x?xf32>
return %tnsr : tensor<?x3x?xf32> return %tnsr : tensor<?x3x?xf32>
} }
▲ Show 20 Lines • Show All 270 Lines • ▼ Show 20 Lines
// ----- // -----
func @invalid_splat(%v : vector<8xf32>) { func @invalid_splat(%v : vector<8xf32>) {
// expected-error@+1 {{must be integer/index/float type}} // expected-error@+1 {{must be integer/index/float type}}
%w = tensor.splat %v : tensor<8xvector<8xf32>> %w = tensor.splat %v : tensor<8xvector<8xf32>>
return return
} }
No newline at end of file No newline at end of file

mlir/test/IR/core-ops.mlir

Show First 20 Lines • Show All 199 Lines • ▼ Show 20 Linesfunc @zero_dim_no_idx(%arg0 : memref<i32>, %arg1 : memref<i32>, %arg2 : memref<i32>) {
return return
// CHECK: %0 = memref.load %{{.*}}[] : memref<i32> // CHECK: %0 = memref.load %{{.*}}[] : memref<i32>
// CHECK: memref.store %{{.*}}, %{{.*}}[] : memref<i32> // CHECK: memref.store %{{.*}}, %{{.*}}[] : memref<i32>
} }
// CHECK-LABEL: func @return_op(%arg0: i32) -> i32 { // CHECK-LABEL: func @return_op(%arg0: i32) -> i32 {
func @return_op(%a : i32) -> i32 { func @return_op(%a : i32) -> i32 {
// CHECK: return %arg0 : i32 // CHECK: return %arg0 : i32
"std.return" (%a) : (i32)->() "func.return" (%a) : (i32)->()
} }
// CHECK-LABEL: func @calls(%arg0: i32) { // CHECK-LABEL: func @calls(%arg0: i32) {
func @calls(%arg0: i32) { func @calls(%arg0: i32) {
// CHECK: %0 = call @return_op(%arg0) : (i32) -> i32 // CHECK: %0 = call @return_op(%arg0) : (i32) -> i32
%x = call @return_op(%arg0) : (i32) -> i32 %x = call @return_op(%arg0) : (i32) -> i32
// CHECK: %1 = call @return_op(%0) : (i32) -> i32 // CHECK: %1 = call @return_op(%0) : (i32) -> i32
%y = call @return_op(%x) : (i32) -> i32 %y = call @return_op(%x) : (i32) -> i32
// CHECK: %2 = call @return_op(%0) : (i32) -> i32 // CHECK: %2 = call @return_op(%0) : (i32) -> i32
%z = "std.call"(%x) {callee = @return_op} : (i32) -> i32 %z = "func.call"(%x) {callee = @return_op} : (i32) -> i32
// CHECK: %f = constant @affine_apply : () -> () // CHECK: %f = constant @affine_apply : () -> ()
%f = constant @affine_apply : () -> () %f = constant @affine_apply : () -> ()
// CHECK: call_indirect %f() : () -> () // CHECK: call_indirect %f() : () -> ()
call_indirect %f() : () -> () call_indirect %f() : () -> ()
// CHECK: %f_0 = constant @return_op : (i32) -> i32 // CHECK: %f_0 = constant @return_op : (i32) -> i32
%f_0 = constant @return_op : (i32) -> i32 %f_0 = constant @return_op : (i32) -> i32
// CHECK: %3 = call_indirect %f_0(%arg0) : (i32) -> i32 // CHECK: %3 = call_indirect %f_0(%arg0) : (i32) -> i32
%2 = call_indirect %f_0(%arg0) : (i32) -> i32 %2 = call_indirect %f_0(%arg0) : (i32) -> i32
// CHECK: %4 = call_indirect %f_0(%arg0) : (i32) -> i32 // CHECK: %4 = call_indirect %f_0(%arg0) : (i32) -> i32
%3 = "std.call_indirect"(%f_0, %arg0) : ((i32) -> i32, i32) -> i32 %3 = "func.call_indirect"(%f_0, %arg0) : ((i32) -> i32, i32) -> i32
return return
} }
// CHECK-LABEL: func @memref_cast(%arg0 // CHECK-LABEL: func @memref_cast(%arg0
func @memref_cast(%arg0: memref<4xf32>, %arg1 : memref<?xf32>, %arg2 : memref<64x16x4xf32, offset: 0, strides: [64, 4, 1]>) { func @memref_cast(%arg0: memref<4xf32>, %arg1 : memref<?xf32>, %arg2 : memref<64x16x4xf32, offset: 0, strides: [64, 4, 1]>) {
// CHECK: %0 = memref.cast %arg0 : memref<4xf32> to memref<?xf32> // CHECK: %0 = memref.cast %arg0 : memref<4xf32> to memref<?xf32>
%0 = memref.cast %arg0 : memref<4xf32> to memref<?xf32> %0 = memref.cast %arg0 : memref<4xf32> to memref<?xf32>
▲ Show 20 Lines • Show All 78 Lines • Show Last 20 Lines

mlir/test/IR/generic-visitors-interrupt.mlir

Show First 20 Lines • Show All 102 Lines • ▼ Show 20 Linesfunc @main(%arg0: f32) -> f32 {
}) {skip_before_all = true} : () -> f32 }) {skip_before_all = true} : () -> f32
%v2 = arith.addf %v1, %arg0 : f32 %v2 = arith.addf %v1, %arg0 : f32
return %v2 : f32 return %v2 : f32
} }
// CHECK: step 0 op 'builtin.module' before all regions // CHECK: step 0 op 'builtin.module' before all regions
// CHECK: step 1 op 'builtin.func' before all regions // CHECK: step 1 op 'builtin.func' before all regions
// CHECK: step 2 op 'arith.addf' before all regions // CHECK: step 2 op 'arith.addf' before all regions
// CHECK: step 3 op 'std.return' before all regions // CHECK: step 3 op 'func.return' before all regions
// CHECK: step 4 op 'builtin.func' after all regions // CHECK: step 4 op 'builtin.func' after all regions
// CHECK: step 5 op 'builtin.module' after all regions // CHECK: step 5 op 'builtin.module' after all regions
// ----- // -----
// Walk is skipped after visiting all regions of "foo". // Walk is skipped after visiting all regions of "foo".
func @main(%arg0: f32) -> f32 { func @main(%arg0: f32) -> f32 {
%v1 = "foo"() ({ %v1 = "foo"() ({
"bar0"() : () -> () "bar0"() : () -> ()
}, { }, {
"bar1"() : () -> () "bar1"() : () -> ()
}) {skip_after_all = true} : () -> f32 }) {skip_after_all = true} : () -> f32
%v2 = arith.addf %v1, %arg0 : f32 %v2 = arith.addf %v1, %arg0 : f32
return %v2 : f32 return %v2 : f32
} }
// CHECK: step 0 op 'builtin.module' before all regions // CHECK: step 0 op 'builtin.module' before all regions
// CHECK: step 1 op 'builtin.func' before all regions // CHECK: step 1 op 'builtin.func' before all regions
// CHECK: step 2 op 'foo' before all regions // CHECK: step 2 op 'foo' before all regions
// CHECK: step 3 op 'bar0' before all regions // CHECK: step 3 op 'bar0' before all regions
// CHECK: step 4 op 'foo' before region #1 // CHECK: step 4 op 'foo' before region #1
// CHECK: step 5 op 'bar1' before all regions // CHECK: step 5 op 'bar1' before all regions
// CHECK: step 6 op 'arith.addf' before all regions // CHECK: step 6 op 'arith.addf' before all regions
// CHECK: step 7 op 'std.return' before all regions // CHECK: step 7 op 'func.return' before all regions
// CHECK: step 8 op 'builtin.func' after all regions // CHECK: step 8 op 'builtin.func' after all regions
// CHECK: step 9 op 'builtin.module' after all regions // CHECK: step 9 op 'builtin.module' after all regions
// ----- // -----
// Walk is skipped after visiting first region of "foo". // Walk is skipped after visiting first region of "foo".
func @main(%arg0: f32) -> f32 { func @main(%arg0: f32) -> f32 {
%v1 = "foo"() ({ %v1 = "foo"() ({
"bar0"() : () -> () "bar0"() : () -> ()
}, { }, {
"bar1"() : () -> () "bar1"() : () -> ()
}) {skip_after_region = 0} : () -> f32 }) {skip_after_region = 0} : () -> f32
%v2 = arith.addf %v1, %arg0 : f32 %v2 = arith.addf %v1, %arg0 : f32
return %v2 : f32 return %v2 : f32
} }
// CHECK: step 0 op 'builtin.module' before all regions // CHECK: step 0 op 'builtin.module' before all regions
// CHECK: step 1 op 'builtin.func' before all regions // CHECK: step 1 op 'builtin.func' before all regions
// CHECK: step 2 op 'foo' before all regions // CHECK: step 2 op 'foo' before all regions
// CHECK: step 3 op 'bar0' before all regions // CHECK: step 3 op 'bar0' before all regions
// CHECK: step 4 op 'arith.addf' before all regions // CHECK: step 4 op 'arith.addf' before all regions
// CHECK: step 5 op 'std.return' before all regions // CHECK: step 5 op 'func.return' before all regions
// CHECK: step 6 op 'builtin.func' after all regions // CHECK: step 6 op 'builtin.func' after all regions
// CHECK: step 7 op 'builtin.module' after all regions // CHECK: step 7 op 'builtin.module' after all regions

mlir/test/IR/generic-visitors.mlir

Show All 30 Lines
// CHECK: step 9 op 'scf.yield' before all regions // CHECK: step 9 op 'scf.yield' before all regions
// CHECK: step 10 op 'scf.if' before region #1 // CHECK: step 10 op 'scf.if' before region #1
// CHECK: step 11 op 'use2' before all regions // CHECK: step 11 op 'use2' before all regions
// CHECK: step 12 op 'scf.yield' before all regions // CHECK: step 12 op 'scf.yield' before all regions
// CHECK: step 13 op 'scf.if' after all regions // CHECK: step 13 op 'scf.if' after all regions
// CHECK: step 14 op 'use3' before all regions // CHECK: step 14 op 'use3' before all regions
// CHECK: step 15 op 'scf.yield' before all regions // CHECK: step 15 op 'scf.yield' before all regions
// CHECK: step 16 op 'scf.for' after all regions // CHECK: step 16 op 'scf.for' after all regions
// CHECK: step 17 op 'std.return' before all regions // CHECK: step 17 op 'func.return' before all regions
// CHECK: step 18 op 'builtin.func' after all regions // CHECK: step 18 op 'builtin.func' after all regions
// CHECK: step 19 op 'builtin.module' after all regions // CHECK: step 19 op 'builtin.module' after all regions
// ----- // -----
// Test the specific operation type visitor. // Test the specific operation type visitor.
func @correct_number_of_regions() { func @correct_number_of_regions() {
%c0 = arith.constant 0 : index %c0 = arith.constant 0 : index
Show All 16 Lines

mlir/test/IR/invalid-ops.mlir

Show All 31 Lines
^bb0: ^bb0:
%i = test.crazyThing() {value = 0} : () -> index // expected-error {{custom op 'test.crazyThing' is unknown}} %i = test.crazyThing() {value = 0} : () -> index // expected-error {{custom op 'test.crazyThing' is unknown}}
return return
} }
// ----- // -----
func @unknown_std_op() { func @unknown_std_op() {
// expected-error@+1 {{unregistered operation 'std.foo_bar_op' found in dialect ('std') that does not allow unknown operations}} // expected-error@+1 {{unregistered operation 'func.foo_bar_op' found in dialect ('func') that does not allow unknown operations}}
%0 = "std.foo_bar_op"() : () -> index %0 = "func.foo_bar_op"() : () -> index
return return
} }
// ----- // -----
func @calls(%arg0: i32) { func @calls(%arg0: i32) {
%x = call @calls() : () -> i32 // expected-error {{incorrect number of operands for callee}} %x = call @calls() : () -> i32 // expected-error {{incorrect number of operands for callee}}
return return
▲ Show 20 Lines • Show All 43 Lines • ▼ Show 20 Lines^bb0(%cond : tensor<12xi1>, %t : tensor<42xi32>, %f : tensor<42xi32>):
// expected-error@+1 {{all non-scalar operands/results must have the same shape and base type}} // expected-error@+1 {{all non-scalar operands/results must have the same shape and base type}}
%r = "arith.select"(%cond, %t, %f) : (tensor<12xi1>, tensor<42xi32>, tensor<42xi32>) -> tensor<42xi32> %r = "arith.select"(%cond, %t, %f) : (tensor<12xi1>, tensor<42xi32>, tensor<42xi32>) -> tensor<42xi32>
} }
// ----- // -----
func @return_not_in_function() { func @return_not_in_function() {
"foo.region"() ({ "foo.region"() ({
// expected-error@+1 {{'std.return' op expects parent op 'builtin.func'}} // expected-error@+1 {{'func.return' op expects parent op 'builtin.func'}}
return return
}): () -> () }): () -> ()
return return
} }
// ----- // -----
func @invalid_splat(%v : f32) { // expected-note {{prior use here}} func @invalid_splat(%v : f32) { // expected-note {{prior use here}}
vector.splat %v : vector<8xf64> vector.splat %v : vector<8xf64>
// expected-error@-1 {{expects different type than prior uses}} // expected-error@-1 {{expects different type than prior uses}}
return return
} }

mlir/test/IR/invalid.mlir

Show First 20 Lines • Show All 349 Lines • ▼ Show 20 Lines
func @malformed_type(%a : intt) { // expected-error {{expected non-function type}} func @malformed_type(%a : intt) { // expected-error {{expected non-function type}}
} }
// ----- // -----
func @resulterror() -> i32 { func @resulterror() -> i32 {
^bb42: ^bb42:
return // expected-error {{'std.return' op has 0 operands, but enclosing function (@resulterror) returns 1}} return // expected-error {{'func.return' op has 0 operands, but enclosing function (@resulterror) returns 1}}
} }
// ----- // -----
func @func_resulterror() -> i32 { func @func_resulterror() -> i32 {
return // expected-error {{'std.return' op has 0 operands, but enclosing function (@func_resulterror) returns 1}} return // expected-error {{'func.return' op has 0 operands, but enclosing function (@func_resulterror) returns 1}}
} }
// ----- // -----
func @argError() { func @argError() {
^bb1(%a: i64): // expected-note {{previously defined here}} ^bb1(%a: i64): // expected-note {{previously defined here}}
cf.br ^bb2 cf.br ^bb2
^bb2(%a: i64): // expected-error{{redefinition of SSA value '%a'}} ^bb2(%a: i64): // expected-error{{redefinition of SSA value '%a'}}
▲ Show 20 Lines • Show All 158 Lines • ▼ Show 20 Linesfunc @return_type_mismatch() -> i32 {
%0 = "foo"() : ()->f32 %0 = "foo"() : ()->f32
return %0 : f32 // expected-error {{type of return operand 0 ('f32') doesn't match function result type ('i32') in function @return_type_mismatch}} return %0 : f32 // expected-error {{type of return operand 0 ('f32') doesn't match function result type ('i32') in function @return_type_mismatch}}
} }
// ----- // -----
func @return_inside_loop() { func @return_inside_loop() {
affine.for %i = 1 to 100 { affine.for %i = 1 to 100 {
// expected-error@-1 {{op expects regions to end with 'affine.yield', found 'std.return'}} // expected-error@-1 {{op expects regions to end with 'affine.yield', found 'func.return'}}
// expected-note@-2 {{in custom textual format, the absence of terminator implies}} // expected-note@-2 {{in custom textual format, the absence of terminator implies}}
return return
} }
return return
} }
// ----- // -----
▲ Show 20 Lines • Show All 1,093 Lines • Show Last 20 Lines

mlir/test/IR/op-stats.mlir

Show All 25 Lines^bb0(%arg0: tensor<4xf32>, %arg1: tensor<4xf32>):
%25 = "xla.add"(%0, %arg1) : (tensor<4xf32>,tensor<4xf32>)-> tensor<4xf32> %25 = "xla.add"(%0, %arg1) : (tensor<4xf32>,tensor<4xf32>)-> tensor<4xf32>
%26 = "xla.add"(%0, %arg1) : (tensor<4xf32>,tensor<4xf32>)-> tensor<4xf32> %26 = "xla.add"(%0, %arg1) : (tensor<4xf32>,tensor<4xf32>)-> tensor<4xf32>
%30 = "long_op_name"(%0, %arg1) : (tensor<4xf32>,tensor<4xf32>)-> tensor<4xf32> %30 = "long_op_name"(%0, %arg1) : (tensor<4xf32>,tensor<4xf32>)-> tensor<4xf32>
return %1 : tensor<4xf32> return %1 : tensor<4xf32>
} }
// CHECK-LABEL: Operations encountered // CHECK-LABEL: Operations encountered
// CHECK: arith.addf , 6 // CHECK: arith.addf , 6
// CHECK: func.return , 1
// CHECK: long_op_name , 1 // CHECK: long_op_name , 1
// CHECK: std.return , 1
// CHECK: xla.add , 17 // CHECK: xla.add , 17

mlir/test/IR/parser.mlir

Show First 20 Lines • Show All 450 Lines • ▼ Show 20 Lines
^bb1(%x : i1, %y : i17): ^bb1(%x : i1, %y : i17):
// CHECK: cf.cond_br %{{.*}}, ^bb2(%{{.*}} : i17), ^bb3(%{{.*}}, %{{.*}} : i1, i17) // CHECK: cf.cond_br %{{.*}}, ^bb2(%{{.*}} : i17), ^bb3(%{{.*}}, %{{.*}} : i1, i17)
"cf.cond_br"(%x, %y, %x, %y) [^bb2, ^bb3] {operand_segment_sizes = dense<[1, 1, 2]>: vector<3xi32>} : (i1, i17, i1, i17) -> () "cf.cond_br"(%x, %y, %x, %y) [^bb2, ^bb3] {operand_segment_sizes = dense<[1, 1, 2]>: vector<3xi32>} : (i1, i17, i1, i17) -> ()
^bb2(%a : i17): ^bb2(%a : i17):
%true = arith.constant true %true = arith.constant true
// CHECK: return %{{.*}}, %{{.*}} : i1, i17 // CHECK: return %{{.*}}, %{{.*}} : i1, i17
"std.return"(%true, %a) : (i1, i17) -> () "func.return"(%true, %a) : (i1, i17) -> ()
^bb3(%b : i1, %c : i17): ^bb3(%b : i1, %c : i17):
// CHECK: return %{{.*}}, %{{.*}} : i1, i17 // CHECK: return %{{.*}}, %{{.*}} : i1, i17
"std.return"(%b, %c) : (i1, i17) -> () "func.return"(%b, %c) : (i1, i17) -> ()
} }
// CHECK-LABEL: func @condbr_simple // CHECK-LABEL: func @condbr_simple
func @condbr_simple() -> (i32) { func @condbr_simple() -> (i32) {
%cond = "foo"() : () -> i1 %cond = "foo"() : () -> i1
%a = "bar"() : () -> i32 %a = "bar"() : () -> i32
%b = "bar"() : () -> i64 %b = "bar"() : () -> i64
// CHECK: cf.cond_br %{{.*}}, ^bb1(%{{.*}} : i32), ^bb2(%{{.*}} : i64) // CHECK: cf.cond_br %{{.*}}, ^bb1(%{{.*}} : i32), ^bb2(%{{.*}} : i64)
▲ Show 20 Lines • Show All 799 Lines • ▼ Show 20 Lines test.default_dialect {
// CHECK-NOT: test.parse_integer_literal // CHECK-NOT: test.parse_integer_literal
parse_integer_literal : 5 parse_integer_literal : 5
// CHECK: parse_integer_literal : 6 // CHECK: parse_integer_literal : 6
test.parse_integer_literal : 6 test.parse_integer_literal : 6
// Verify that only an op prefix is stripped, not an attribute value for // Verify that only an op prefix is stripped, not an attribute value for
// example. // example.
// CHECK: "test.op_with_attr"() {test.attr = "test.value"} : () -> () // CHECK: "test.op_with_attr"() {test.attr = "test.value"} : () -> ()
"test.op_with_attr"() {test.attr = "test.value"} : () -> () "test.op_with_attr"() {test.attr = "test.value"} : () -> ()
// TODO: remove this after removing the special casing for std in the printer.
// Verify that operations in the standard dialect keep the `std.` prefix.
// CHECK: cf.assert
cf.assert %bool, "Assertion"
"test.terminator"() : ()->() "test.terminator"() : ()->()
} }
// The same operation outside of the region does not have an std. prefix. // The same operation outside of the region does not have an func. prefix.
// CHECK-NOT: std.assert
// CHECK: return // CHECK: return
std.return func.return
} }
// CHECK-LABEL: func @unreachable_dominance_violation_ok // CHECK-LABEL: func @unreachable_dominance_violation_ok
func @unreachable_dominance_violation_ok() -> i1 { func @unreachable_dominance_violation_ok() -> i1 {
// CHECK: [[VAL:%.*]] = arith.constant false // CHECK: [[VAL:%.*]] = arith.constant false
// CHECK: return [[VAL]] : i1 // CHECK: return [[VAL]] : i1
// CHECK: ^bb1: // no predecessors // CHECK: ^bb1: // no predecessors
// CHECK: [[VAL2:%.*]]:3 = "bar"([[VAL3:%.*]]) : (i64) -> (i1, i1, i1) // CHECK: [[VAL2:%.*]]:3 = "bar"([[VAL3:%.*]]) : (i64) -> (i1, i1, i1)
▲ Show 20 Lines • Show All 131 Lines • Show Last 20 Lines

mlir/test/IR/print-op-generic.mlir

// # RUN: mlir-opt %s | FileCheck %s // # RUN: mlir-opt %s | FileCheck %s
// # RUN: mlir-opt %s --mlir-print-op-generic | FileCheck %s --check-prefix=GENERIC // # RUN: mlir-opt %s --mlir-print-op-generic | FileCheck %s --check-prefix=GENERIC
// CHECK-LABEL: func @pretty_names // CHECK-LABEL: func @pretty_names
// CHECK-GENERIC: "func"() // CHECK-GENERIC: "func"()
func @pretty_names() { func @pretty_names() {
%x = test.string_attr_pretty_name %x = test.string_attr_pretty_name
// CHECK: %x = test.string_attr_pretty_name // CHECK: %x = test.string_attr_pretty_name
// GENERIC: %0 = "test.string_attr_pretty_name"() // GENERIC: %0 = "test.string_attr_pretty_name"()
return return
// CHECK: return // CHECK: return
// GENERIC: "std.return"() // GENERIC: "func.return"()
} }

mlir/test/IR/visitors.mlir

Show All 24 Lines
// CHECK: Visiting op 'builtin.module' // CHECK: Visiting op 'builtin.module'
// CHECK: Visiting op 'builtin.func' // CHECK: Visiting op 'builtin.func'
// CHECK: Visiting op 'scf.for' // CHECK: Visiting op 'scf.for'
// CHECK: Visiting op 'use0' // CHECK: Visiting op 'use0'
// CHECK: Visiting op 'scf.if' // CHECK: Visiting op 'scf.if'
// CHECK: Visiting op 'use1' // CHECK: Visiting op 'use1'
// CHECK: Visiting op 'use2' // CHECK: Visiting op 'use2'
// CHECK: Visiting op 'use3' // CHECK: Visiting op 'use3'
// CHECK: Visiting op 'std.return' // CHECK: Visiting op 'func.return'
// CHECK-LABEL: Block pre-order visits // CHECK-LABEL: Block pre-order visits
// CHECK: Visiting block ^bb0 from region 0 from operation 'builtin.module' // CHECK: Visiting block ^bb0 from region 0 from operation 'builtin.module'
// CHECK: Visiting block ^bb0 from region 0 from operation 'builtin.func' // CHECK: Visiting block ^bb0 from region 0 from operation 'builtin.func'
// CHECK: Visiting block ^bb0 from region 0 from operation 'scf.for' // CHECK: Visiting block ^bb0 from region 0 from operation 'scf.for'
// CHECK: Visiting block ^bb0 from region 0 from operation 'scf.if' // CHECK: Visiting block ^bb0 from region 0 from operation 'scf.if'
// CHECK: Visiting block ^bb0 from region 1 from operation 'scf.if' // CHECK: Visiting block ^bb0 from region 1 from operation 'scf.if'
// CHECK-LABEL: Region pre-order visits // CHECK-LABEL: Region pre-order visits
// CHECK: Visiting region 0 from operation 'builtin.module' // CHECK: Visiting region 0 from operation 'builtin.module'
// CHECK: Visiting region 0 from operation 'builtin.func' // CHECK: Visiting region 0 from operation 'builtin.func'
// CHECK: Visiting region 0 from operation 'scf.for' // CHECK: Visiting region 0 from operation 'scf.for'
// CHECK: Visiting region 0 from operation 'scf.if' // CHECK: Visiting region 0 from operation 'scf.if'
// CHECK: Visiting region 1 from operation 'scf.if' // CHECK: Visiting region 1 from operation 'scf.if'
// CHECK-LABEL: Op post-order visits // CHECK-LABEL: Op post-order visits
// CHECK: Visiting op 'use0' // CHECK: Visiting op 'use0'
// CHECK: Visiting op 'use1' // CHECK: Visiting op 'use1'
// CHECK: Visiting op 'use2' // CHECK: Visiting op 'use2'
// CHECK: Visiting op 'scf.if' // CHECK: Visiting op 'scf.if'
// CHECK: Visiting op 'use3' // CHECK: Visiting op 'use3'
// CHECK: Visiting op 'scf.for' // CHECK: Visiting op 'scf.for'
// CHECK: Visiting op 'std.return' // CHECK: Visiting op 'func.return'
// CHECK: Visiting op 'builtin.func' // CHECK: Visiting op 'builtin.func'
// CHECK: Visiting op 'builtin.module' // CHECK: Visiting op 'builtin.module'
// CHECK-LABEL: Block post-order visits // CHECK-LABEL: Block post-order visits
// CHECK: Visiting block ^bb0 from region 0 from operation 'scf.if' // CHECK: Visiting block ^bb0 from region 0 from operation 'scf.if'
// CHECK: Visiting block ^bb0 from region 1 from operation 'scf.if' // CHECK: Visiting block ^bb0 from region 1 from operation 'scf.if'
// CHECK: Visiting block ^bb0 from region 0 from operation 'scf.for' // CHECK: Visiting block ^bb0 from region 0 from operation 'scf.for'
// CHECK: Visiting block ^bb0 from region 0 from operation 'builtin.func' // CHECK: Visiting block ^bb0 from region 0 from operation 'builtin.func'
// CHECK: Visiting block ^bb0 from region 0 from operation 'builtin.module' // CHECK: Visiting block ^bb0 from region 0 from operation 'builtin.module'
// CHECK-LABEL: Region post-order visits // CHECK-LABEL: Region post-order visits
// CHECK: Visiting region 0 from operation 'scf.if' // CHECK: Visiting region 0 from operation 'scf.if'
// CHECK: Visiting region 1 from operation 'scf.if' // CHECK: Visiting region 1 from operation 'scf.if'
// CHECK: Visiting region 0 from operation 'scf.for' // CHECK: Visiting region 0 from operation 'scf.for'
// CHECK: Visiting region 0 from operation 'builtin.func' // CHECK: Visiting region 0 from operation 'builtin.func'
// CHECK: Visiting region 0 from operation 'builtin.module' // CHECK: Visiting region 0 from operation 'builtin.module'
// CHECK-LABEL: Op pre-order erasures // CHECK-LABEL: Op pre-order erasures
// CHECK: Erasing op 'scf.for' // CHECK: Erasing op 'scf.for'
// CHECK: Erasing op 'std.return' // CHECK: Erasing op 'func.return'
// CHECK-LABEL: Block pre-order erasures // CHECK-LABEL: Block pre-order erasures
// CHECK: Erasing block ^bb0 from region 0 from operation 'scf.for' // CHECK: Erasing block ^bb0 from region 0 from operation 'scf.for'
// CHECK-LABEL: Op post-order erasures (skip) // CHECK-LABEL: Op post-order erasures (skip)
// CHECK: Erasing op 'use0' // CHECK: Erasing op 'use0'
// CHECK: Erasing op 'use1' // CHECK: Erasing op 'use1'
// CHECK: Erasing op 'use2' // CHECK: Erasing op 'use2'
// CHECK: Erasing op 'scf.if' // CHECK: Erasing op 'scf.if'
// CHECK: Erasing op 'use3' // CHECK: Erasing op 'use3'
// CHECK: Erasing op 'scf.for' // CHECK: Erasing op 'scf.for'
// CHECK: Erasing op 'std.return' // CHECK: Erasing op 'func.return'
// CHECK-LABEL: Block post-order erasures (skip) // CHECK-LABEL: Block post-order erasures (skip)
// CHECK: Erasing block ^bb0 from region 0 from operation 'scf.if' // CHECK: Erasing block ^bb0 from region 0 from operation 'scf.if'
// CHECK: Erasing block ^bb0 from region 1 from operation 'scf.if' // CHECK: Erasing block ^bb0 from region 1 from operation 'scf.if'
// CHECK: Erasing block ^bb0 from region 0 from operation 'scf.for' // CHECK: Erasing block ^bb0 from region 0 from operation 'scf.for'
// CHECK-LABEL: Op post-order erasures (no skip) // CHECK-LABEL: Op post-order erasures (no skip)
// CHECK: Erasing op 'use0' // CHECK: Erasing op 'use0'
// CHECK: Erasing op 'use1' // CHECK: Erasing op 'use1'
// CHECK: Erasing op 'use2' // CHECK: Erasing op 'use2'
// CHECK: Erasing op 'scf.if' // CHECK: Erasing op 'scf.if'
// CHECK: Erasing op 'use3' // CHECK: Erasing op 'use3'
// CHECK: Erasing op 'scf.for' // CHECK: Erasing op 'scf.for'
// CHECK: Erasing op 'std.return' // CHECK: Erasing op 'func.return'
// CHECK: Erasing op 'builtin.func' // CHECK: Erasing op 'builtin.func'
// CHECK: Erasing op 'builtin.module' // CHECK: Erasing op 'builtin.module'
// CHECK-LABEL: Block post-order erasures (no skip) // CHECK-LABEL: Block post-order erasures (no skip)
// CHECK: Erasing block ^bb0 from region 0 from operation 'scf.if' // CHECK: Erasing block ^bb0 from region 0 from operation 'scf.if'
// CHECK: Erasing block ^bb0 from region 1 from operation 'scf.if' // CHECK: Erasing block ^bb0 from region 1 from operation 'scf.if'
// CHECK: Erasing block ^bb0 from region 0 from operation 'scf.for' // CHECK: Erasing block ^bb0 from region 0 from operation 'scf.for'
// CHECK: Erasing block ^bb0 from region 0 from operation 'builtin.func' // CHECK: Erasing block ^bb0 from region 0 from operation 'builtin.func'
Show All 19 Lines
// CHECK: Visiting op 'builtin.module' // CHECK: Visiting op 'builtin.module'
// CHECK: Visiting op 'builtin.func' // CHECK: Visiting op 'builtin.func'
// CHECK: Visiting op 'regionOp0' // CHECK: Visiting op 'regionOp0'
// CHECK: Visiting op 'op0' // CHECK: Visiting op 'op0'
// CHECK: Visiting op 'cf.br' // CHECK: Visiting op 'cf.br'
// CHECK: Visiting op 'op1' // CHECK: Visiting op 'op1'
// CHECK: Visiting op 'cf.br' // CHECK: Visiting op 'cf.br'
// CHECK: Visiting op 'op2' // CHECK: Visiting op 'op2'
// CHECK: Visiting op 'std.return' // CHECK: Visiting op 'func.return'
// CHECK-LABEL: Block pre-order visits // CHECK-LABEL: Block pre-order visits
// CHECK: Visiting block ^bb0 from region 0 from operation 'builtin.module' // CHECK: Visiting block ^bb0 from region 0 from operation 'builtin.module'
// CHECK: Visiting block ^bb0 from region 0 from operation 'builtin.func' // CHECK: Visiting block ^bb0 from region 0 from operation 'builtin.func'
// CHECK: Visiting block ^bb0 from region 0 from operation 'regionOp0' // CHECK: Visiting block ^bb0 from region 0 from operation 'regionOp0'
// CHECK: Visiting block ^bb1 from region 0 from operation 'regionOp0' // CHECK: Visiting block ^bb1 from region 0 from operation 'regionOp0'
// CHECK: Visiting block ^bb2 from region 0 from operation 'regionOp0' // CHECK: Visiting block ^bb2 from region 0 from operation 'regionOp0'
// CHECK-LABEL: Region pre-order visits // CHECK-LABEL: Region pre-order visits
// CHECK: Visiting region 0 from operation 'builtin.module' // CHECK: Visiting region 0 from operation 'builtin.module'
// CHECK: Visiting region 0 from operation 'builtin.func' // CHECK: Visiting region 0 from operation 'builtin.func'
// CHECK: Visiting region 0 from operation 'regionOp0' // CHECK: Visiting region 0 from operation 'regionOp0'
// CHECK-LABEL: Op post-order visits // CHECK-LABEL: Op post-order visits
// CHECK: Visiting op 'op0' // CHECK: Visiting op 'op0'
// CHECK: Visiting op 'cf.br' // CHECK: Visiting op 'cf.br'
// CHECK: Visiting op 'op1' // CHECK: Visiting op 'op1'
// CHECK: Visiting op 'cf.br' // CHECK: Visiting op 'cf.br'
// CHECK: Visiting op 'op2' // CHECK: Visiting op 'op2'
// CHECK: Visiting op 'regionOp0' // CHECK: Visiting op 'regionOp0'
// CHECK: Visiting op 'std.return' // CHECK: Visiting op 'func.return'
// CHECK: Visiting op 'builtin.func' // CHECK: Visiting op 'builtin.func'
// CHECK: Visiting op 'builtin.module' // CHECK: Visiting op 'builtin.module'
// CHECK-LABEL: Block post-order visits // CHECK-LABEL: Block post-order visits
// CHECK: Visiting block ^bb0 from region 0 from operation 'regionOp0' // CHECK: Visiting block ^bb0 from region 0 from operation 'regionOp0'
// CHECK: Visiting block ^bb1 from region 0 from operation 'regionOp0' // CHECK: Visiting block ^bb1 from region 0 from operation 'regionOp0'
// CHECK: Visiting block ^bb2 from region 0 from operation 'regionOp0' // CHECK: Visiting block ^bb2 from region 0 from operation 'regionOp0'
// CHECK: Visiting block ^bb0 from region 0 from operation 'builtin.func' // CHECK: Visiting block ^bb0 from region 0 from operation 'builtin.func'
// CHECK: Visiting block ^bb0 from region 0 from operation 'builtin.module' // CHECK: Visiting block ^bb0 from region 0 from operation 'builtin.module'
// CHECK-LABEL: Region post-order visits // CHECK-LABEL: Region post-order visits
// CHECK: Visiting region 0 from operation 'regionOp0' // CHECK: Visiting region 0 from operation 'regionOp0'
// CHECK: Visiting region 0 from operation 'builtin.func' // CHECK: Visiting region 0 from operation 'builtin.func'
// CHECK: Visiting region 0 from operation 'builtin.module' // CHECK: Visiting region 0 from operation 'builtin.module'
// CHECK-LABEL: Op pre-order erasures (skip) // CHECK-LABEL: Op pre-order erasures (skip)
// CHECK: Erasing op 'regionOp0' // CHECK: Erasing op 'regionOp0'
// CHECK: Erasing op 'std.return' // CHECK: Erasing op 'func.return'
// CHECK-LABEL: Block pre-order erasures (skip) // CHECK-LABEL: Block pre-order erasures (skip)
// CHECK: Erasing block ^bb0 from region 0 from operation 'regionOp0' // CHECK: Erasing block ^bb0 from region 0 from operation 'regionOp0'
// CHECK: Erasing block ^bb0 from region 0 from operation 'regionOp0' // CHECK: Erasing block ^bb0 from region 0 from operation 'regionOp0'
// CHECK: Erasing block ^bb0 from region 0 from operation 'regionOp0' // CHECK: Erasing block ^bb0 from region 0 from operation 'regionOp0'
// CHECK-LABEL: Op post-order erasures (skip) // CHECK-LABEL: Op post-order erasures (skip)
// CHECK: Erasing op 'op0' // CHECK: Erasing op 'op0'
// CHECK: Erasing op 'cf.br' // CHECK: Erasing op 'cf.br'
// CHECK: Erasing op 'op1' // CHECK: Erasing op 'op1'
// CHECK: Erasing op 'cf.br' // CHECK: Erasing op 'cf.br'
// CHECK: Erasing op 'op2' // CHECK: Erasing op 'op2'
// CHECK: Erasing op 'regionOp0' // CHECK: Erasing op 'regionOp0'
// CHECK: Erasing op 'std.return' // CHECK: Erasing op 'func.return'
// CHECK-LABEL: Block post-order erasures (skip) // CHECK-LABEL: Block post-order erasures (skip)
// CHECK: Erasing block ^bb0 from region 0 from operation 'regionOp0' // CHECK: Erasing block ^bb0 from region 0 from operation 'regionOp0'
// CHECK: Erasing block ^bb0 from region 0 from operation 'regionOp0' // CHECK: Erasing block ^bb0 from region 0 from operation 'regionOp0'
// CHECK: Erasing block ^bb0 from region 0 from operation 'regionOp0' // CHECK: Erasing block ^bb0 from region 0 from operation 'regionOp0'
// CHECK-LABEL: Op post-order erasures (no skip) // CHECK-LABEL: Op post-order erasures (no skip)
// CHECK: Erasing op 'op0' // CHECK: Erasing op 'op0'
// CHECK: Erasing op 'cf.br' // CHECK: Erasing op 'cf.br'
// CHECK: Erasing op 'op1' // CHECK: Erasing op 'op1'
// CHECK: Erasing op 'cf.br' // CHECK: Erasing op 'cf.br'
// CHECK: Erasing op 'op2' // CHECK: Erasing op 'op2'
// CHECK: Erasing op 'regionOp0' // CHECK: Erasing op 'regionOp0'
// CHECK: Erasing op 'std.return' // CHECK: Erasing op 'func.return'
// CHECK-LABEL: Block post-order erasures (no skip) // CHECK-LABEL: Block post-order erasures (no skip)
// CHECK: Erasing block ^bb0 from region 0 from operation 'regionOp0' // CHECK: Erasing block ^bb0 from region 0 from operation 'regionOp0'
// CHECK: Erasing block ^bb0 from region 0 from operation 'regionOp0' // CHECK: Erasing block ^bb0 from region 0 from operation 'regionOp0'
// CHECK: Erasing block ^bb0 from region 0 from operation 'regionOp0' // CHECK: Erasing block ^bb0 from region 0 from operation 'regionOp0'
// CHECK: Erasing block ^bb0 from region 0 from operation 'builtin.func' // CHECK: Erasing block ^bb0 from region 0 from operation 'builtin.func'
// CHECK: Erasing block ^bb0 from region 0 from operation 'builtin.module' // CHECK: Erasing block ^bb0 from region 0 from operation 'builtin.module'

mlir/test/Integration/Dialect/SparseTensor/python/test_stress.py

Show All 10 Lines
import numpy as np import numpy as np
from mlir import ir from mlir import ir
from mlir import runtime as rt from mlir import runtime as rt
from mlir.execution_engine import ExecutionEngine from mlir.execution_engine import ExecutionEngine
from mlir.dialects import builtin from mlir.dialects import builtin
from mlir.dialects import std from mlir.dialects import func
from mlir.dialects import sparse_tensor as st from mlir.dialects import sparse_tensor as st
_SCRIPT_PATH = os.path.dirname(os.path.abspath(__file__)) _SCRIPT_PATH = os.path.dirname(os.path.abspath(__file__))
sys.path.append(_SCRIPT_PATH) sys.path.append(_SCRIPT_PATH)
from tools import sparse_compiler from tools import sparse_compiler
# ===----------------------------------------------------------------------=== # # ===----------------------------------------------------------------------=== #
▲ Show 20 Lines • Show All 89 Lines • ▼ Show 20 Lines with ir.InsertionPoint(self._module.body):
self._assertEqualsRoundtripTp(arg0.type) self._assertEqualsRoundtripTp(arg0.type)
v = st.ConvertOp(types.pop(0), arg0) v = st.ConvertOp(types.pop(0), arg0)
for tp in types: for tp in types:
w = st.ConvertOp(tp, v) w = st.ConvertOp(tp, v)
# Release intermediate tensors before they fall out of scope. # Release intermediate tensors before they fall out of scope.
st.ReleaseOp(v.result) st.ReleaseOp(v.result)
v = w v = w
self._assertEqualsRoundtripTp(v.result.type) self._assertEqualsRoundtripTp(v.result.type)
std.ReturnOp(v) func.ReturnOp(v)
return self return self
def writeTo(self, filename): def writeTo(self, filename):
"""Write the ir.Module to the given file. If the file already exists, """Write the ir.Module to the given file. If the file already exists,
then raises an error. If the filename is None, then is a no-op.""" then raises an error. If the filename is None, then is a no-op."""
assert self._module is not None, \ assert self._module is not None, \
'StressTest: must call build() before writeTo()' 'StressTest: must call build() before writeTo()'
if filename is None: if filename is None:
▲ Show 20 Lines • Show All 110 Lines • Show Last 20 Lines

mlir/test/Integration/Dialect/SparseTensor/taco/tools/mlir_pytaco_utils.py

Show First 20 Lines • Show All 302 Lines • ▼ Show 20 Linesdef _get_output_sparse_tensor_kernel(
return f""" return f"""
!Ptr = type !llvm.ptr<i8> !Ptr = type !llvm.ptr<i8>
#enc = #sparse_tensor.encoding<{{ #enc = #sparse_tensor.encoding<{{
dimLevelType = [ {sparsity} ] dimLevelType = [ {sparsity} ]
}}> }}>
func @{_ENTRY_NAME}(%t: tensor<{shape}x{type}, #enc>, %filename: !Ptr) func @{_ENTRY_NAME}(%t: tensor<{shape}x{type}, #enc>, %filename: !Ptr)
attributes {{ llvm.emit_c_interface }} {{ attributes {{ llvm.emit_c_interface }} {{
sparse_tensor.out %t, %filename : tensor<{shape}x{type}, #enc>, !Ptr sparse_tensor.out %t, %filename : tensor<{shape}x{type}, #enc>, !Ptr
std.return func.return
}}""" }}"""
def output_sparse_tensor(tensor: ctypes.c_void_p, filename: str, def output_sparse_tensor(tensor: ctypes.c_void_p, filename: str,
sparsity: Sequence[sparse_tensor.DimLevelType], sparsity: Sequence[sparse_tensor.DimLevelType],
type: str) -> None: type: str) -> None:
"""Outputs an MLIR sparse tensor to the given file. """Outputs an MLIR sparse tensor to the given file.
Show All 27 Lines

mlir/test/Transforms/test-legalizer.mlir

// RUN: mlir-opt -allow-unregistered-dialect -split-input-file -test-legalize-patterns -verify-diagnostics %s | FileCheck %s // RUN: mlir-opt -allow-unregistered-dialect -split-input-file -test-legalize-patterns -verify-diagnostics %s | FileCheck %s
// CHECK-LABEL: verifyDirectPattern // CHECK-LABEL: verifyDirectPattern
func @verifyDirectPattern() -> i32 { func @verifyDirectPattern() -> i32 {
// CHECK-NEXT: "test.legal_op_a"() {status = "Success"} // CHECK-NEXT: "test.legal_op_a"() {status = "Success"}
%result = "test.illegal_op_a"() : () -> (i32) %result = "test.illegal_op_a"() : () -> (i32)
// expected-remark@+1 {{op 'std.return' is not legalizable}} // expected-remark@+1 {{op 'func.return' is not legalizable}}
return %result : i32 return %result : i32
} }
// CHECK-LABEL: verifyLargerBenefit // CHECK-LABEL: verifyLargerBenefit
func @verifyLargerBenefit() -> i32 { func @verifyLargerBenefit() -> i32 {
// CHECK-NEXT: "test.legal_op_a"() {status = "Success"} // CHECK-NEXT: "test.legal_op_a"() {status = "Success"}
%result = "test.illegal_op_c"() : () -> (i32) %result = "test.illegal_op_c"() : () -> (i32)
// expected-remark@+1 {{op 'std.return' is not legalizable}} // expected-remark@+1 {{op 'func.return' is not legalizable}}
return %result : i32 return %result : i32
} }
// CHECK-LABEL: func private @remap_input_1_to_0() // CHECK-LABEL: func private @remap_input_1_to_0()
func private @remap_input_1_to_0(i16) func private @remap_input_1_to_0(i16)
// CHECK-LABEL: func @remap_input_1_to_1(%arg0: f64) // CHECK-LABEL: func @remap_input_1_to_1(%arg0: f64)
func @remap_input_1_to_1(%arg0: i64) { func @remap_input_1_to_1(%arg0: i64) {
// CHECK-NEXT: "test.valid"{{.*}} : (f64) // CHECK-NEXT: "test.valid"{{.*}} : (f64)
"test.invalid"(%arg0) : (i64) -> () "test.invalid"(%arg0) : (i64) -> ()
} }
// CHECK-LABEL: func @remap_call_1_to_1(%arg0: f64) // CHECK-LABEL: func @remap_call_1_to_1(%arg0: f64)
func @remap_call_1_to_1(%arg0: i64) { func @remap_call_1_to_1(%arg0: i64) {
// CHECK-NEXT: call @remap_input_1_to_1(%arg0) : (f64) -> () // CHECK-NEXT: call @remap_input_1_to_1(%arg0) : (f64) -> ()
call @remap_input_1_to_1(%arg0) : (i64) -> () call @remap_input_1_to_1(%arg0) : (i64) -> ()
// expected-remark@+1 {{op 'std.return' is not legalizable}} // expected-remark@+1 {{op 'func.return' is not legalizable}}
return return
} }
// CHECK-LABEL: func @remap_input_1_to_N({{.*}}f16, {{.*}}f16) // CHECK-LABEL: func @remap_input_1_to_N({{.*}}f16, {{.*}}f16)
func @remap_input_1_to_N(%arg0: f32) -> f32 { func @remap_input_1_to_N(%arg0: f32) -> f32 {
// CHECK-NEXT: "test.return"{{.*}} : (f16, f16) -> () // CHECK-NEXT: "test.return"{{.*}} : (f16, f16) -> ()
"test.return"(%arg0) : (f32) -> () "test.return"(%arg0) : (f32) -> ()
} }
Show All 32 Linesfunc @no_remap_nested() {
// CHECK-NEXT: "foo.region" // CHECK-NEXT: "foo.region"
// expected-remark@+1 {{op 'foo.region' is not legalizable}} // expected-remark@+1 {{op 'foo.region' is not legalizable}}
"foo.region"() ({ "foo.region"() ({
// CHECK-NEXT: ^bb0(%{{.*}}: i64, %{{.*}}: i16, %{{.*}}: i64): // CHECK-NEXT: ^bb0(%{{.*}}: i64, %{{.*}}: i16, %{{.*}}: i64):
^bb0(%i0: i64, %unused: i16, %i1: i64): ^bb0(%i0: i64, %unused: i16, %i1: i64):
// CHECK-NEXT: "test.valid"{{.*}} : (i64, i64) // CHECK-NEXT: "test.valid"{{.*}} : (i64, i64)
"test.invalid"(%i0, %i1) : (i64, i64) -> () "test.invalid"(%i0, %i1) : (i64, i64) -> ()
}) : () -> () }) : () -> ()
// expected-remark@+1 {{op 'std.return' is not legalizable}} // expected-remark@+1 {{op 'func.return' is not legalizable}}
return return
} }
// CHECK-LABEL: func @remap_moved_region_args // CHECK-LABEL: func @remap_moved_region_args
func @remap_moved_region_args() { func @remap_moved_region_args() {
// CHECK-NEXT: return // CHECK-NEXT: return
// CHECK-NEXT: ^bb1(%{{.*}}: f64, %{{.*}}: f64, %{{.*}}: f16, %{{.*}}: f16): // CHECK-NEXT: ^bb1(%{{.*}}: f64, %{{.*}}: f64, %{{.*}}: f16, %{{.*}}: f16):
// CHECK-NEXT: "test.cast"{{.*}} : (f16, f16) -> f32 // CHECK-NEXT: "test.cast"{{.*}} : (f16, f16) -> f32
// CHECK-NEXT: "test.valid"{{.*}} : (f64, f64, f32) // CHECK-NEXT: "test.valid"{{.*}} : (f64, f64, f32)
"test.region"() ({ "test.region"() ({
^bb1(%i0: i64, %unused: i16, %i1: i64, %2: f32): ^bb1(%i0: i64, %unused: i16, %i1: i64, %2: f32):
"test.invalid"(%i0, %i1, %2) : (i64, i64, f32) -> () "test.invalid"(%i0, %i1, %2) : (i64, i64, f32) -> ()
}) : () -> () }) : () -> ()
// expected-remark@+1 {{op 'std.return' is not legalizable}} // expected-remark@+1 {{op 'func.return' is not legalizable}}
return return
} }
// CHECK-LABEL: func @remap_cloned_region_args // CHECK-LABEL: func @remap_cloned_region_args
func @remap_cloned_region_args() { func @remap_cloned_region_args() {
// CHECK-NEXT: return // CHECK-NEXT: return
// CHECK-NEXT: ^bb1(%{{.*}}: f64, %{{.*}}: f64, %{{.*}}: f16, %{{.*}}: f16): // CHECK-NEXT: ^bb1(%{{.*}}: f64, %{{.*}}: f64, %{{.*}}: f16, %{{.*}}: f16):
// CHECK-NEXT: "test.cast"{{.*}} : (f16, f16) -> f32 // CHECK-NEXT: "test.cast"{{.*}} : (f16, f16) -> f32
// CHECK-NEXT: "test.valid"{{.*}} : (f64, f64, f32) // CHECK-NEXT: "test.valid"{{.*}} : (f64, f64, f32)
"test.region"() ({ "test.region"() ({
^bb1(%i0: i64, %unused: i16, %i1: i64, %2: f32): ^bb1(%i0: i64, %unused: i16, %i1: i64, %2: f32):
"test.invalid"(%i0, %i1, %2) : (i64, i64, f32) -> () "test.invalid"(%i0, %i1, %2) : (i64, i64, f32) -> ()
}) {legalizer.should_clone} : () -> () }) {legalizer.should_clone} : () -> ()
// expected-remark@+1 {{op 'std.return' is not legalizable}} // expected-remark@+1 {{op 'func.return' is not legalizable}}
return return
} }
// CHECK-LABEL: func @remap_drop_region // CHECK-LABEL: func @remap_drop_region
func @remap_drop_region() { func @remap_drop_region() {
// CHECK-NEXT: return // CHECK-NEXT: return
// CHECK-NEXT: } // CHECK-NEXT: }
"test.drop_region_op"() ({ "test.drop_region_op"() ({
^bb1(%i0: i64, %unused: i16, %i1: i64, %2: f32): ^bb1(%i0: i64, %unused: i16, %i1: i64, %2: f32):
"test.invalid"(%i0, %i1, %2) : (i64, i64, f32) -> () "test.invalid"(%i0, %i1, %2) : (i64, i64, f32) -> ()
}) : () -> () }) : () -> ()
// expected-remark@+1 {{op 'std.return' is not legalizable}} // expected-remark@+1 {{op 'func.return' is not legalizable}}
return return
} }
// CHECK-LABEL: func @dropped_input_in_use // CHECK-LABEL: func @dropped_input_in_use
func @dropped_input_in_use(%arg: i16, %arg2: i64) { func @dropped_input_in_use(%arg: i16, %arg2: i64) {
// CHECK-NEXT: "test.cast"{{.*}} : () -> i16 // CHECK-NEXT: "test.cast"{{.*}} : () -> i16
// CHECK-NEXT: "work"{{.*}} : (i16) // CHECK-NEXT: "work"{{.*}} : (i16)
// expected-remark@+1 {{op 'work' is not legalizable}} // expected-remark@+1 {{op 'work' is not legalizable}}
"work"(%arg) : (i16) -> () "work"(%arg) : (i16) -> ()
} }
// CHECK-LABEL: func @up_to_date_replacement // CHECK-LABEL: func @up_to_date_replacement
func @up_to_date_replacement(%arg: i8) -> i8 { func @up_to_date_replacement(%arg: i8) -> i8 {
// CHECK-NEXT: return // CHECK-NEXT: return
%repl_1 = "test.rewrite"(%arg) : (i8) -> i8 %repl_1 = "test.rewrite"(%arg) : (i8) -> i8
%repl_2 = "test.rewrite"(%repl_1) : (i8) -> i8 %repl_2 = "test.rewrite"(%repl_1) : (i8) -> i8
// expected-remark@+1 {{op 'std.return' is not legalizable}} // expected-remark@+1 {{op 'func.return' is not legalizable}}
return %repl_2 : i8 return %repl_2 : i8
} }
// CHECK-LABEL: func @remove_foldable_op // CHECK-LABEL: func @remove_foldable_op
// CHECK-SAME: (%[[ARG_0:[a-z0-9]*]]: i32) // CHECK-SAME: (%[[ARG_0:[a-z0-9]*]]: i32)
func @remove_foldable_op(%arg0 : i32) -> (i32) { func @remove_foldable_op(%arg0 : i32) -> (i32) {
// CHECK-NEXT: return %[[ARG_0]] // CHECK-NEXT: return %[[ARG_0]]
%0 = "test.op_with_region_fold"(%arg0) ({ %0 = "test.op_with_region_fold"(%arg0) ({
"foo.op_with_region_terminator"() : () -> () "foo.op_with_region_terminator"() : () -> ()
}) : (i32) -> (i32) }) : (i32) -> (i32)
// expected-remark@+1 {{op 'std.return' is not legalizable}} // expected-remark@+1 {{op 'func.return' is not legalizable}}
return %0 : i32 return %0 : i32
} }
// CHECK-LABEL: @create_block // CHECK-LABEL: @create_block
func @create_block() { func @create_block() {
// Check that we created a block with arguments. // Check that we created a block with arguments.
// CHECK-NOT: test.create_block // CHECK-NOT: test.create_block
// CHECK: ^{{.*}}(%{{.*}}: i32, %{{.*}}: i32): // CHECK: ^{{.*}}(%{{.*}}: i32, %{{.*}}: i32):
"test.create_block"() : () -> () "test.create_block"() : () -> ()
// expected-remark@+1 {{op 'std.return' is not legalizable}} // expected-remark@+1 {{op 'func.return' is not legalizable}}
return return
} }
// CHECK-LABEL: @bounded_recursion // CHECK-LABEL: @bounded_recursion
func @bounded_recursion() { func @bounded_recursion() {
// CHECK: test.recursive_rewrite 0 // CHECK: test.recursive_rewrite 0
test.recursive_rewrite 3 test.recursive_rewrite 3
// expected-remark@+1 {{op 'std.return' is not legalizable}} // expected-remark@+1 {{op 'func.return' is not legalizable}}
return return
} }
// ----- // -----
// expected-remark@+1 {{applyPartialConversion failed}} // expected-remark@+1 {{applyPartialConversion failed}}
builtin.module { builtin.module {
▲ Show 20 Lines • Show All 45 Lines • ▼ Show 20 Lines
// CHECK-LABEL: @create_illegal_block // CHECK-LABEL: @create_illegal_block
func @create_illegal_block() { func @create_illegal_block() {
// Check that we can undo block creation, i.e. that the block was removed. // Check that we can undo block creation, i.e. that the block was removed.
// CHECK: test.create_illegal_block // CHECK: test.create_illegal_block
// CHECK-NOT: ^{{.*}}(%{{.*}}: i32, %{{.*}}: i32): // CHECK-NOT: ^{{.*}}(%{{.*}}: i32, %{{.*}}: i32):
// expected-remark@+1 {{op 'test.create_illegal_block' is not legalizable}} // expected-remark@+1 {{op 'test.create_illegal_block' is not legalizable}}
"test.create_illegal_block"() : () -> () "test.create_illegal_block"() : () -> ()
// expected-remark@+1 {{op 'std.return' is not legalizable}} // expected-remark@+1 {{op 'func.return' is not legalizable}}
return return
} }
// ----- // -----
// CHECK-LABEL: @undo_block_arg_replace // CHECK-LABEL: @undo_block_arg_replace
func @undo_block_arg_replace() { func @undo_block_arg_replace() {
// expected-remark@+1 {{op 'test.undo_block_arg_replace' is not legalizable}} // expected-remark@+1 {{op 'test.undo_block_arg_replace' is not legalizable}}
"test.undo_block_arg_replace"() ({ "test.undo_block_arg_replace"() ({
^bb0(%arg0: i32): ^bb0(%arg0: i32):
// CHECK: ^bb0(%[[ARG:.*]]: i32): // CHECK: ^bb0(%[[ARG:.*]]: i32):
// CHECK-NEXT: "test.return"(%[[ARG]]) : (i32) // CHECK-NEXT: "test.return"(%[[ARG]]) : (i32)
"test.return"(%arg0) : (i32) -> () "test.return"(%arg0) : (i32) -> ()
}) : () -> () }) : () -> ()
// expected-remark@+1 {{op 'std.return' is not legalizable}} // expected-remark@+1 {{op 'func.return' is not legalizable}}
return return
} }
// ----- // -----
// The op in this function is rewritten to itself (and thus remains illegal) by // The op in this function is rewritten to itself (and thus remains illegal) by
// a pattern that removes its second block after adding an operation into it. // a pattern that removes its second block after adding an operation into it.
// Check that we can undo block removal successfully. // Check that we can undo block removal successfully.
Show All 19 Lines
// with an attached region containing an invalid terminator. The terminator is // with an attached region containing an invalid terminator. The terminator is
// created before the parent op. The deletion should not crash when deleting // created before the parent op. The deletion should not crash when deleting
// created ops in the inverse order, i.e. deleting the parent op and then the // created ops in the inverse order, i.e. deleting the parent op and then the
// child op. // child op.
// CHECK-LABEL: @undo_child_created_before_parent // CHECK-LABEL: @undo_child_created_before_parent
func @undo_child_created_before_parent() { func @undo_child_created_before_parent() {
// expected-remark@+1 {{is not legalizable}} // expected-remark@+1 {{is not legalizable}}
"test.illegal_op_with_region_anchor"() : () -> () "test.illegal_op_with_region_anchor"() : () -> ()
// expected-remark@+1 {{op 'std.return' is not legalizable}} // expected-remark@+1 {{op 'func.return' is not legalizable}}
return return
} }
// ----- // -----
// Check that a conversion pattern on `test.blackhole` can mark the producer // Check that a conversion pattern on `test.blackhole` can mark the producer
// for deletion. // for deletion.
// CHECK-LABEL: @blackhole // CHECK-LABEL: @blackhole
func @blackhole() { func @blackhole() {
%input = "test.blackhole_producer"() : () -> (i32) %input = "test.blackhole_producer"() : () -> (i32)
"test.blackhole"(%input) : (i32) -> () "test.blackhole"(%input) : (i32) -> ()
// expected-remark@+1 {{op 'std.return' is not legalizable}} // expected-remark@+1 {{op 'func.return' is not legalizable}}
return return
} }
// ----- // -----
// expected-remark@+1 {{applyPartialConversion failed}} // expected-remark@+1 {{applyPartialConversion failed}}
builtin.module { builtin.module {
Show All 17 Lines

mlir/test/lib/Conversion/StandardToLLVM/TestConvertCallOp.cpp

//===- TestConvertCallOp.cpp - Test LLVM Conversion of Standard CallOp ----===// //===- TestConvertCallOp.cpp - Test LLVM Conversion of Standard CallOp ----===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "TestDialect.h" #include "TestDialect.h"
#include "TestTypes.h" #include "TestTypes.h"
#include "mlir/Conversion/LLVMCommon/Pattern.h" #include "mlir/Conversion/LLVMCommon/Pattern.h"
#include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h" #include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
using namespace mlir; using namespace mlir;
namespace { namespace {
class TestTypeProducerOpConverter class TestTypeProducerOpConverter
: public ConvertOpToLLVMPattern<test::TestTypeProducerOp> { : public ConvertOpToLLVMPattern<test::TestTypeProducerOp> {
Show All 12 Lines
class TestConvertCallOp class TestConvertCallOp
: public PassWrapper<TestConvertCallOp, OperationPass<ModuleOp>> { : public PassWrapper<TestConvertCallOp, OperationPass<ModuleOp>> {
public: public:
void getDependentDialects(DialectRegistry &registry) const final { void getDependentDialects(DialectRegistry &registry) const final {
registry.insert<LLVM::LLVMDialect>(); registry.insert<LLVM::LLVMDialect>();
} }
StringRef getArgument() const final { return "test-convert-call-op"; } StringRef getArgument() const final { return "test-convert-call-op"; }
StringRef getDescription() const final { StringRef getDescription() const final {
return "Tests conversion of `std.call` to `llvm.call` in " return "Tests conversion of `func.call` to `llvm.call` in "
"presence of custom types"; "presence of custom types";
} }
void runOnOperation() override { void runOnOperation() override {
ModuleOp m = getOperation(); ModuleOp m = getOperation();
// Populate type conversions. // Populate type conversions.
LLVMTypeConverter typeConverter(m.getContext()); LLVMTypeConverter typeConverter(m.getContext());
typeConverter.addConversion([&](test::TestType type) { typeConverter.addConversion([&](test::TestType type) {
return LLVM::LLVMPointerType::get(IntegerType::get(m.getContext(), 8)); return LLVM::LLVMPointerType::get(IntegerType::get(m.getContext(), 8));
}); });
typeConverter.addConversion([&](test::SimpleAType type) { typeConverter.addConversion([&](test::SimpleAType type) {
return IntegerType::get(type.getContext(), 42); return IntegerType::get(type.getContext(), 42);
}); });
// Populate patterns. // Populate patterns.
RewritePatternSet patterns(m.getContext()); RewritePatternSet patterns(m.getContext());
populateStdToLLVMConversionPatterns(typeConverter, patterns); populateStdToLLVMConversionPatterns(typeConverter, patterns);
patterns.add<TestTypeProducerOpConverter>(typeConverter); patterns.add<TestTypeProducerOpConverter>(typeConverter);
// Set target. // Set target.
ConversionTarget target(getContext()); ConversionTarget target(getContext());
target.addLegalDialect<LLVM::LLVMDialect>(); target.addLegalDialect<LLVM::LLVMDialect>();
target.addIllegalDialect<test::TestDialect>(); target.addIllegalDialect<test::TestDialect>();
target.addIllegalDialect<StandardOpsDialect>(); target.addIllegalDialect<func::FuncDialect>();
if (failed(applyPartialConversion(m, target, std::move(patterns)))) if (failed(applyPartialConversion(m, target, std::move(patterns))))
signalPassFailure(); signalPassFailure();
} }
}; };
} // namespace } // namespace
namespace mlir { namespace mlir {
namespace test { namespace test {
void registerConvertCallOpPass() { PassRegistration<TestConvertCallOp>(); } void registerConvertCallOpPass() { PassRegistration<TestConvertCallOp>(); }
} // namespace test } // namespace test
} // namespace mlir } // namespace mlir

mlir/test/lib/Dialect/Affine/TestLoopFusion.cpp

//===- TestLoopFusion.cpp - Test loop fusion ------------------------------===// //===- TestLoopFusion.cpp - Test loop fusion ------------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements a pass to test various loop fusion utility functions. // This file implements a pass to test various loop fusion utility functions.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Affine/Analysis/Utils.h" #include "mlir/Dialect/Affine/Analysis/Utils.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Affine/LoopFusionUtils.h" #include "mlir/Dialect/Affine/LoopFusionUtils.h"
#include "mlir/Dialect/Affine/LoopUtils.h" #include "mlir/Dialect/Affine/LoopUtils.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#define DEBUG_TYPE "test-loop-fusion" #define DEBUG_TYPE "test-loop-fusion"
using namespace mlir; using namespace mlir;
static llvm::cl::OptionCategory clOptionsCategory(DEBUG_TYPE " options"); static llvm::cl::OptionCategory clOptionsCategory(DEBUG_TYPE " options");
▲ Show 20 Lines • Show All 182 Lines • Show Last 20 Lines

mlir/test/lib/Dialect/CMakeLists.txt

add_subdirectory(Affine) add_subdirectory(Affine)
add_subdirectory(DLTI) add_subdirectory(DLTI)
add_subdirectory(Func)
add_subdirectory(GPU) add_subdirectory(GPU)
add_subdirectory(Linalg) add_subdirectory(Linalg)
add_subdirectory(Math) add_subdirectory(Math)
add_subdirectory(MemRef) add_subdirectory(MemRef)
add_subdirectory(SCF) add_subdirectory(SCF)
add_subdirectory(Shape) add_subdirectory(Shape)
add_subdirectory(SPIRV) add_subdirectory(SPIRV)
add_subdirectory(StandardOps)
add_subdirectory(Tensor) add_subdirectory(Tensor)
add_subdirectory(Test) add_subdirectory(Test)
add_subdirectory(Tosa) add_subdirectory(Tosa)
add_subdirectory(Vector) add_subdirectory(Vector)

mlir/test/lib/Dialect/Func/CMakeLists.txt

  • This file was moved from mlir/test/lib/Dialect/StandardOps/CMakeLists.txt.
# Exclude tests from libMLIR.so # Exclude tests from libMLIR.so
add_mlir_library(MLIRStandardOpsTestPasses add_mlir_library(MLIRFuncTestPasses
TestDecomposeCallGraphTypes.cpp TestDecomposeCallGraphTypes.cpp
EXCLUDE_FROM_LIBMLIR EXCLUDE_FROM_LIBMLIR
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRAffine MLIRAffine
MLIRPass MLIRPass
MLIRStandardOpsTransforms MLIRFuncTransforms
MLIRTestDialect MLIRTestDialect
MLIRTransformUtils MLIRTransformUtils
) )
target_include_directories(MLIRStandardOpsTestPasses target_include_directories(MLIRFuncTestPasses
PRIVATE PRIVATE
${CMAKE_CURRENT_SOURCE_DIR}/../Test ${CMAKE_CURRENT_SOURCE_DIR}/../Test
${CMAKE_CURRENT_BINARY_DIR}/../Test ${CMAKE_CURRENT_BINARY_DIR}/../Test
) )

mlir/test/lib/Dialect/Func/TestDecomposeCallGraphTypes.cpp

  • This file was moved from mlir/test/lib/Dialect/StandardOps/TestDecomposeCallGraphTypes.cpp.
//===- TestDecomposeCallGraphTypes.cpp - Test CG type decomposition -------===// //===- TestDecomposeCallGraphTypes.cpp - Test CG type decomposition -------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "TestDialect.h" #include "TestDialect.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/StandardOps/Transforms/DecomposeCallGraphTypes.h" #include "mlir/Dialect/Func/Transforms/DecomposeCallGraphTypes.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
using namespace mlir; using namespace mlir;
namespace { namespace {
/// A pass for testing call graph type decomposition. /// A pass for testing call graph type decomposition.
Show All 18 Lines void runOnOperation() override {
auto *context = &getContext(); auto *context = &getContext();
TypeConverter typeConverter; TypeConverter typeConverter;
ConversionTarget target(*context); ConversionTarget target(*context);
ValueDecomposer decomposer; ValueDecomposer decomposer;
RewritePatternSet patterns(context); RewritePatternSet patterns(context);
target.addLegalDialect<test::TestDialect>(); target.addLegalDialect<test::TestDialect>();
target.addDynamicallyLegalOp<ReturnOp>([&](ReturnOp op) { target.addDynamicallyLegalOp<func::ReturnOp>([&](func::ReturnOp op) {
return typeConverter.isLegal(op.getOperandTypes()); return typeConverter.isLegal(op.getOperandTypes());
}); });
target.addDynamicallyLegalOp<CallOp>( target.addDynamicallyLegalOp<func::CallOp>(
[&](CallOp op) { return typeConverter.isLegal(op); }); [&](func::CallOp op) { return typeConverter.isLegal(op); });
target.addDynamicallyLegalOp<FuncOp>([&](FuncOp op) { target.addDynamicallyLegalOp<FuncOp>([&](FuncOp op) {
return typeConverter.isSignatureLegal(op.getType()); return typeConverter.isSignatureLegal(op.getType());
}); });
typeConverter.addConversion([](Type type) { return type; }); typeConverter.addConversion([](Type type) { return type; });
typeConverter.addConversion( typeConverter.addConversion(
[](TupleType tupleType, SmallVectorImpl<Type> &types) { [](TupleType tupleType, SmallVectorImpl<Type> &types) {
tupleType.getFlattenedTypes(types); tupleType.getFlattenedTypes(types);
▲ Show 20 Lines • Show All 41 Lines • Show Last 20 Lines

mlir/test/lib/Dialect/GPU/CMakeLists.txt

# Exclude tests from libMLIR.so # Exclude tests from libMLIR.so
add_mlir_library(MLIRGPUTestPasses add_mlir_library(MLIRGPUTestPasses
TestConvertGPUKernelToCubin.cpp TestConvertGPUKernelToCubin.cpp
TestConvertGPUKernelToHsaco.cpp TestConvertGPUKernelToHsaco.cpp
TestGpuMemoryPromotion.cpp TestGpuMemoryPromotion.cpp
TestGpuParallelLoopMapping.cpp TestGpuParallelLoopMapping.cpp
TestGpuRewrite.cpp TestGpuRewrite.cpp
EXCLUDE_FROM_LIBMLIR EXCLUDE_FROM_LIBMLIR
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRAffine MLIRAffine
MLIRFunc
MLIRGPUOps MLIRGPUOps
MLIRGPUToGPURuntimeTransforms MLIRGPUToGPURuntimeTransforms
MLIRLLVMToLLVMIRTranslation MLIRLLVMToLLVMIRTranslation
MLIRNVVMIR MLIRNVVMIR
MLIRNVVMToLLVMIRTranslation MLIRNVVMToLLVMIRTranslation
MLIRPass MLIRPass
MLIRROCDLIR MLIRROCDLIR
MLIRROCDLToLLVMIRTranslation MLIRROCDLToLLVMIRTranslation
MLIRSCF MLIRSCF
MLIRStandard
MLIRTransformUtils MLIRTransformUtils
) )

mlir/test/lib/Dialect/GPU/TestGpuMemoryPromotion.cpp

//===- TestGPUMemoryPromotionPass.cpp - Test pass for GPU promotion -------===// //===- TestGPUMemoryPromotionPass.cpp - Test pass for GPU promotion -------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements the pass testing the utilities for moving data across // This file implements the pass testing the utilities for moving data across
// different levels of the GPU memory hierarchy. // different levels of the GPU memory hierarchy.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/GPU/GPUDialect.h" #include "mlir/Dialect/GPU/GPUDialect.h"
#include "mlir/Dialect/GPU/MemoryPromotion.h" #include "mlir/Dialect/GPU/MemoryPromotion.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h" #include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/Attributes.h" #include "mlir/IR/Attributes.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
using namespace mlir; using namespace mlir;
namespace { namespace {
/// Simple pass for testing the promotion to workgroup memory in GPU functions. /// Simple pass for testing the promotion to workgroup memory in GPU functions.
/// Promotes all arguments with "gpu.test_promote_workgroup" attribute. This /// Promotes all arguments with "gpu.test_promote_workgroup" attribute. This
/// does not check whether the promotion is legal (e.g., amount of memory used) /// does not check whether the promotion is legal (e.g., amount of memory used)
/// or beneficial (e.g., makes previously uncoalesced loads coalesced). /// or beneficial (e.g., makes previously uncoalesced loads coalesced).
class TestGpuMemoryPromotionPass class TestGpuMemoryPromotionPass
: public PassWrapper<TestGpuMemoryPromotionPass, : public PassWrapper<TestGpuMemoryPromotionPass,
OperationPass<gpu::GPUFuncOp>> { OperationPass<gpu::GPUFuncOp>> {
void getDependentDialects(DialectRegistry &registry) const override { void getDependentDialects(DialectRegistry &registry) const override {
registry.insert<AffineDialect, memref::MemRefDialect, StandardOpsDialect, registry.insert<AffineDialect, func::FuncDialect, memref::MemRefDialect,
scf::SCFDialect>(); scf::SCFDialect>();
} }
StringRef getArgument() const final { return "test-gpu-memory-promotion"; } StringRef getArgument() const final { return "test-gpu-memory-promotion"; }
StringRef getDescription() const final { StringRef getDescription() const final {
return "Promotes the annotated arguments of gpu.func to workgroup memory."; return "Promotes the annotated arguments of gpu.func to workgroup memory.";
} }
void runOnOperation() override { void runOnOperation() override {
Show All 14 Lines

mlir/test/lib/Dialect/GPU/TestGpuRewrite.cpp

//===- TestAllReduceLowering.cpp - Test gpu.all_reduce lowering -----------===// //===- TestAllReduceLowering.cpp - Test gpu.all_reduce lowering -----------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file contains test passes for lowering the gpu.all_reduce op. // This file contains test passes for lowering the gpu.all_reduce op.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/GPU/Passes.h" #include "mlir/Dialect/GPU/Passes.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
using namespace mlir; using namespace mlir;
namespace { namespace {
struct TestGpuRewritePass struct TestGpuRewritePass
: public PassWrapper<TestGpuRewritePass, OperationPass<ModuleOp>> { : public PassWrapper<TestGpuRewritePass, OperationPass<ModuleOp>> {
void getDependentDialects(DialectRegistry &registry) const override { void getDependentDialects(DialectRegistry &registry) const override {
registry.insert<arith::ArithmeticDialect, StandardOpsDialect, registry.insert<arith::ArithmeticDialect, func::FuncDialect,
memref::MemRefDialect>(); memref::MemRefDialect>();
} }
StringRef getArgument() const final { return "test-gpu-rewrite"; } StringRef getArgument() const final { return "test-gpu-rewrite"; }
StringRef getDescription() const final { StringRef getDescription() const final {
return "Applies all rewrite patterns within the GPU dialect."; return "Applies all rewrite patterns within the GPU dialect.";
} }
void runOnOperation() override { void runOnOperation() override {
RewritePatternSet patterns(&getContext()); RewritePatternSet patterns(&getContext());
Show All 11 Lines

mlir/test/lib/Dialect/Linalg/CMakeLists.txt

Show All 11 Linesadd_mlir_library(MLIRLinalgTestPasses
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRAffine MLIRAffine
MLIRAffineBufferizableOpInterfaceImpl MLIRAffineBufferizableOpInterfaceImpl
MLIRArithmetic MLIRArithmetic
MLIRArithmeticTransforms MLIRArithmeticTransforms
MLIRBufferization MLIRBufferization
MLIRBufferizationTransforms MLIRBufferizationTransforms
MLIRFunc
MLIRFuncTransforms
MLIRGPUTransforms MLIRGPUTransforms
MLIRLinalg MLIRLinalg
MLIRLinalgTransforms MLIRLinalgTransforms
MLIRLLVMToLLVMIRTranslation MLIRLLVMToLLVMIRTranslation
MLIRMemRef MLIRMemRef
MLIRPass MLIRPass
MLIRSCF MLIRSCF
MLIRSCFTransforms MLIRSCFTransforms
MLIRStandard
MLIRStandardOpsTransforms
MLIRTensor MLIRTensor
MLIRTensorTransforms MLIRTensorTransforms
MLIRTransformUtils MLIRTransformUtils
MLIRVector MLIRVector
MLIRVectorToSCF MLIRVectorToSCF
MLIRVectorTransforms MLIRVectorTransforms
) )

mlir/test/lib/Dialect/Linalg/TestLinalgCodegenStrategy.cpp

//===- TestLinalgCodegenStrategy.cpp - Test Linalg codegen strategy -------===// //===- TestLinalgCodegenStrategy.cpp - Test Linalg codegen strategy -------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements logic for testing the Linalg codegen strategy. // This file implements logic for testing the Linalg codegen strategy.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include <utility> #include <utility>
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/GPU/GPUDialect.h" #include "mlir/Dialect/GPU/GPUDialect.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/Linalg/Transforms/CodegenStrategy.h" #include "mlir/Dialect/Linalg/Transforms/CodegenStrategy.h"
#include "mlir/Dialect/Linalg/Utils/Utils.h" #include "mlir/Dialect/Linalg/Utils/Utils.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Vector/IR/VectorOps.h" #include "mlir/Dialect/Vector/IR/VectorOps.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "llvm/ADT/SetVector.h" #include "llvm/ADT/SetVector.h"
using namespace mlir; using namespace mlir;
using namespace mlir::linalg; using namespace mlir::linalg;
namespace { namespace {
struct TestLinalgCodegenStrategy struct TestLinalgCodegenStrategy
: public PassWrapper<TestLinalgCodegenStrategy, OperationPass<FuncOp>> { : public PassWrapper<TestLinalgCodegenStrategy, OperationPass<FuncOp>> {
StringRef getArgument() const final { return "test-linalg-codegen-strategy"; } StringRef getArgument() const final { return "test-linalg-codegen-strategy"; }
StringRef getDescription() const final { StringRef getDescription() const final {
return "Test Linalg Codegen Strategy."; return "Test Linalg Codegen Strategy.";
} }
TestLinalgCodegenStrategy() = default; TestLinalgCodegenStrategy() = default;
TestLinalgCodegenStrategy(const TestLinalgCodegenStrategy &pass) TestLinalgCodegenStrategy(const TestLinalgCodegenStrategy &pass)
: PassWrapper(pass) {} : PassWrapper(pass) {}
void getDependentDialects(DialectRegistry &registry) const override { void getDependentDialects(DialectRegistry &registry) const override {
// clang-format off // clang-format off
registry.insert<AffineDialect, registry.insert<AffineDialect,
func::FuncDialect,
gpu::GPUDialect, gpu::GPUDialect,
linalg::LinalgDialect, linalg::LinalgDialect,
memref::MemRefDialect, memref::MemRefDialect,
scf::SCFDialect, scf::SCFDialect,
StandardOpsDialect,
vector::VectorDialect>(); vector::VectorDialect>();
// clang-format on // clang-format on
} }
template <typename LinalgNamedOp> template <typename LinalgNamedOp>
void applyStrategyToNamedLinalgOp(); void applyStrategyToNamedLinalgOp();
void runOnOperation() override; void runOnOperation() override;
▲ Show 20 Lines • Show All 266 Lines • Show Last 20 Lines

mlir/test/lib/Dialect/Linalg/TestLinalgFusionTransforms.cpp

Show First 20 Lines • Show All 108 Lines • ▼ Show 20 Lines
} }
namespace { namespace {
template <LinalgTilingLoopType LoopType> template <LinalgTilingLoopType LoopType>
struct TestLinalgFusionTransforms struct TestLinalgFusionTransforms
: public PassWrapper<TestLinalgFusionTransforms<LoopType>, : public PassWrapper<TestLinalgFusionTransforms<LoopType>,
OperationPass<FuncOp>> { OperationPass<FuncOp>> {
void getDependentDialects(DialectRegistry &registry) const override { void getDependentDialects(DialectRegistry &registry) const override {
registry.insert<AffineDialect, linalg::LinalgDialect, memref::MemRefDialect, registry.insert<AffineDialect, func::FuncDialect, linalg::LinalgDialect,
scf::SCFDialect, StandardOpsDialect>(); memref::MemRefDialect, scf::SCFDialect>();
} }
TestLinalgFusionTransforms() = default; TestLinalgFusionTransforms() = default;
TestLinalgFusionTransforms(const TestLinalgFusionTransforms &pass) {} TestLinalgFusionTransforms(const TestLinalgFusionTransforms &pass) {}
void runOnOperation() override { void runOnOperation() override {
MLIRContext *context = &this->getContext(); MLIRContext *context = &this->getContext();
FuncOp funcOp = this->getOperation(); FuncOp funcOp = this->getOperation();
RewritePatternSet fusionPatterns(context); RewritePatternSet fusionPatterns(context);
▲ Show 20 Lines • Show All 207 Lines • Show Last 20 Lines

mlir/test/lib/Dialect/Linalg/TestLinalgTransforms.cpp

//===- TestLinalgTransforms.cpp - Test Linalg transformation patterns -----===// //===- TestLinalgTransforms.cpp - Test Linalg transformation patterns -----===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements logic for testing Linalg transformations. // This file implements logic for testing Linalg transformations.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/GPU/GPUDialect.h" #include "mlir/Dialect/GPU/GPUDialect.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/Linalg/Passes.h" #include "mlir/Dialect/Linalg/Passes.h"
#include "mlir/Dialect/Linalg/Transforms/HoistPadding.h" #include "mlir/Dialect/Linalg/Transforms/HoistPadding.h"
#include "mlir/Dialect/Linalg/Transforms/Hoisting.h" #include "mlir/Dialect/Linalg/Transforms/Hoisting.h"
#include "mlir/Dialect/Linalg/Transforms/Transforms.h" #include "mlir/Dialect/Linalg/Transforms/Transforms.h"
#include "mlir/Dialect/Linalg/Utils/Utils.h" #include "mlir/Dialect/Linalg/Utils/Utils.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Vector/IR/VectorOps.h" #include "mlir/Dialect/Vector/IR/VectorOps.h"
#include "mlir/Pass/PassManager.h" #include "mlir/Pass/PassManager.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
#include "llvm/ADT/SetVector.h" #include "llvm/ADT/SetVector.h"
#include "llvm/ADT/SmallVector.h" #include "llvm/ADT/SmallVector.h"
using namespace mlir; using namespace mlir;
using namespace mlir::linalg; using namespace mlir::linalg;
namespace { namespace {
struct TestLinalgTransforms struct TestLinalgTransforms
: public PassWrapper<TestLinalgTransforms, OperationPass<FuncOp>> { : public PassWrapper<TestLinalgTransforms, OperationPass<FuncOp>> {
TestLinalgTransforms() = default; TestLinalgTransforms() = default;
TestLinalgTransforms(const TestLinalgTransforms &pass) : PassWrapper(pass) {} TestLinalgTransforms(const TestLinalgTransforms &pass) : PassWrapper(pass) {}
void getDependentDialects(DialectRegistry &registry) const override { void getDependentDialects(DialectRegistry &registry) const override {
// clang-format off // clang-format off
registry.insert<AffineDialect, registry.insert<AffineDialect,
func::FuncDialect,
memref::MemRefDialect, memref::MemRefDialect,
scf::SCFDialect, scf::SCFDialect,
StandardOpsDialect,
linalg::LinalgDialect, linalg::LinalgDialect,
vector::VectorDialect, vector::VectorDialect,
gpu::GPUDialect>(); gpu::GPUDialect>();
// clang-format on // clang-format on
} }
StringRef getArgument() const final { StringRef getArgument() const final {
return "test-linalg-transform-patterns"; return "test-linalg-transform-patterns";
} }
▲ Show 20 Lines • Show All 707 Lines • Show Last 20 Lines

mlir/test/lib/Dialect/SCF/TestLoopUnrolling.cpp

//===-------- TestLoopUnrolling.cpp --- loop unrolling test pass ----------===// //===-------- TestLoopUnrolling.cpp --- loop unrolling test pass ----------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements a pass to unroll loops by a specified unroll factor. // This file implements a pass to unroll loops by a specified unroll factor.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/SCF/Utils/Utils.h" #include "mlir/Dialect/SCF/Utils/Utils.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
using namespace mlir; using namespace mlir;
namespace { namespace {
static unsigned getNestingDepth(Operation *op) { static unsigned getNestingDepth(Operation *op) {
Show All 19 Lines explicit TestLoopUnrollingPass(uint64_t unrollFactorParam,
unsigned loopDepthParam, unsigned loopDepthParam,
bool annotateLoopParam) { bool annotateLoopParam) {
unrollFactor = unrollFactorParam; unrollFactor = unrollFactorParam;
loopDepth = loopDepthParam; loopDepth = loopDepthParam;
annotateLoop = annotateLoopParam; annotateLoop = annotateLoopParam;
} }
void getDependentDialects(DialectRegistry &registry) const override { void getDependentDialects(DialectRegistry &registry) const override {
registry.insert<arith::ArithmeticDialect, StandardOpsDialect>(); registry.insert<arith::ArithmeticDialect, func::FuncDialect>();
} }
void runOnOperation() override { void runOnOperation() override {
FuncOp func = getOperation(); FuncOp func = getOperation();
SmallVector<scf::ForOp, 4> loops; SmallVector<scf::ForOp, 4> loops;
func.walk([&](scf::ForOp forOp) { func.walk([&](scf::ForOp forOp) {
if (getNestingDepth(forOp) == loopDepth) if (getNestingDepth(forOp) == loopDepth)
loops.push_back(forOp); loops.push_back(forOp);
Show All 30 Lines

mlir/test/lib/Dialect/SCF/TestSCFUtils.cpp

//===- TestSCFUtils.cpp --- Pass to test independent SCF dialect utils ----===// //===- TestSCFUtils.cpp --- Pass to test independent SCF dialect utils ----===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements a pass to test SCF dialect utils. // This file implements a pass to test SCF dialect utils.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/SCF/Transforms.h" #include "mlir/Dialect/SCF/Transforms.h"
#include "mlir/Dialect/SCF/Utils/Utils.h" #include "mlir/Dialect/SCF/Utils/Utils.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
#include "llvm/ADT/SetVector.h" #include "llvm/ADT/SetVector.h"
using namespace mlir; using namespace mlir;
▲ Show 20 Lines • Show All 115 Lines • ▼ Show 20 Lines case mlir::scf::PipeliningOption::PipelinerPart::Epilogue:
op->setAttr(kTestPipeliningAnnotationPart, b.getStringAttr("epilogue")); op->setAttr(kTestPipeliningAnnotationPart, b.getStringAttr("epilogue"));
break; break;
} }
op->setAttr(kTestPipeliningAnnotationIteration, op->setAttr(kTestPipeliningAnnotationIteration,
b.getI32IntegerAttr(iteration)); b.getI32IntegerAttr(iteration));
} }
void getDependentDialects(DialectRegistry &registry) const override { void getDependentDialects(DialectRegistry &registry) const override {
registry.insert<arith::ArithmeticDialect, StandardOpsDialect>(); registry.insert<arith::ArithmeticDialect, func::FuncDialect>();
} }
void runOnOperation() override { void runOnOperation() override {
RewritePatternSet patterns(&getContext()); RewritePatternSet patterns(&getContext());
mlir::scf::PipeliningOption options; mlir::scf::PipeliningOption options;
options.getScheduleFn = getSchedule; options.getScheduleFn = getSchedule;
if (annotatePipeline) if (annotatePipeline)
options.annotateFn = annotate; options.annotateFn = annotate;
Show All 20 Lines

mlir/test/lib/Dialect/StandardOps/CMakeLists.txt

  • This file was moved to mlir/test/lib/Dialect/Func/CMakeLists.txt.

mlir/test/lib/Dialect/StandardOps/TestDecomposeCallGraphTypes.cpp

  • This file was moved to mlir/test/lib/Dialect/Func/TestDecomposeCallGraphTypes.cpp.

mlir/test/lib/Dialect/Test/CMakeLists.txt

Show First 20 Lines • Show All 53 Lines • ▼ Show 20 Linesadd_mlir_library(MLIRTestDialect
MLIRTestOpsIncGen MLIRTestOpsIncGen
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRControlFlowInterfaces MLIRControlFlowInterfaces
MLIRDataLayoutInterfaces MLIRDataLayoutInterfaces
MLIRDerivedAttributeOpInterface MLIRDerivedAttributeOpInterface
MLIRDialect MLIRDialect
MLIRDLTI MLIRDLTI
MLIRFunc
MLIRFuncTransforms
MLIRIR MLIRIR
MLIRInferTypeOpInterface MLIRInferTypeOpInterface
MLIRLinalg MLIRLinalg
MLIRLinalgTransforms MLIRLinalgTransforms
MLIRLLVMIR MLIRLLVMIR
MLIRPass MLIRPass
MLIRReduce MLIRReduce
MLIRStandard
MLIRStandardOpsTransforms
MLIRTensor MLIRTensor
MLIRTransformUtils MLIRTransformUtils
MLIRTransforms MLIRTransforms
) )

mlir/test/lib/Dialect/Test/TestDialect.cpp

//===- TestDialect.cpp - MLIR Dialect for Testing -------------------------===// //===- TestDialect.cpp - MLIR Dialect for Testing -------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "TestDialect.h" #include "TestDialect.h"
#include "TestAttributes.h" #include "TestAttributes.h"
#include "TestInterfaces.h" #include "TestInterfaces.h"
#include "TestTypes.h" #include "TestTypes.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/DLTI/DLTI.h" #include "mlir/Dialect/DLTI/DLTI.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/DialectImplementation.h" #include "mlir/IR/DialectImplementation.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/IR/TypeUtilities.h" #include "mlir/IR/TypeUtilities.h"
#include "mlir/Reducer/ReductionPatternInterface.h" #include "mlir/Reducer/ReductionPatternInterface.h"
#include "mlir/Transforms/FoldUtils.h" #include "mlir/Transforms/FoldUtils.h"
#include "mlir/Transforms/InliningUtils.h" #include "mlir/Transforms/InliningUtils.h"
▲ Show 20 Lines • Show All 339 Lines • ▼ Show 20 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
namespace { namespace {
struct FoldToCallOpPattern : public OpRewritePattern<FoldToCallOp> { struct FoldToCallOpPattern : public OpRewritePattern<FoldToCallOp> {
using OpRewritePattern<FoldToCallOp>::OpRewritePattern; using OpRewritePattern<FoldToCallOp>::OpRewritePattern;
LogicalResult matchAndRewrite(FoldToCallOp op, LogicalResult matchAndRewrite(FoldToCallOp op,
PatternRewriter &rewriter) const override { PatternRewriter &rewriter) const override {
rewriter.replaceOpWithNewOp<CallOp>(op, TypeRange(), op.getCalleeAttr(), rewriter.replaceOpWithNewOp<func::CallOp>(op, TypeRange(),
ValueRange()); op.getCalleeAttr(), ValueRange());
return success(); return success();
} }
}; };
} // namespace } // namespace
void FoldToCallOp::getCanonicalizationPatterns(RewritePatternSet &results, void FoldToCallOp::getCanonicalizationPatterns(RewritePatternSet &results,
MLIRContext *context) { MLIRContext *context) {
results.add<FoldToCallOpPattern>(context); results.add<FoldToCallOpPattern>(context);
▲ Show 20 Lines • Show All 900 Lines • Show Last 20 Lines

mlir/test/lib/Dialect/Test/TestPatterns.cpp

//===- TestPatterns.cpp - Test dialect pattern driver ---------------------===// //===- TestPatterns.cpp - Test dialect pattern driver ---------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "TestDialect.h" #include "TestDialect.h"
#include "TestTypes.h" #include "TestTypes.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/StandardOps/Transforms/FuncConversions.h" #include "mlir/Dialect/Func/Transforms/FuncConversions.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/IR/Matchers.h" #include "mlir/IR/Matchers.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "mlir/Transforms/FoldUtils.h" #include "mlir/Transforms/FoldUtils.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
using namespace mlir; using namespace mlir;
▲ Show 20 Lines • Show All 615 Lines • ▼ Show 20 Lines StringRef getDescription() const final {
return "Run test dialect legalization patterns"; return "Run test dialect legalization patterns";
} }
/// The mode of conversion to use with the driver. /// The mode of conversion to use with the driver.
enum class ConversionMode { Analysis, Full, Partial }; enum class ConversionMode { Analysis, Full, Partial };
TestLegalizePatternDriver(ConversionMode mode) : mode(mode) {} TestLegalizePatternDriver(ConversionMode mode) : mode(mode) {}
void getDependentDialects(DialectRegistry &registry) const override { void getDependentDialects(DialectRegistry &registry) const override {
registry.insert<StandardOpsDialect>(); registry.insert<func::FuncDialect>();
} }
void runOnOperation() override { void runOnOperation() override {
TestTypeConverter converter; TestTypeConverter converter;
mlir::RewritePatternSet patterns(&getContext()); mlir::RewritePatternSet patterns(&getContext());
populateWithGenerated(patterns); populateWithGenerated(patterns);
patterns patterns
.add<TestRegionRewriteBlockMovement, TestRegionRewriteUndo, .add<TestRegionRewriteBlockMovement, TestRegionRewriteUndo,
Show All 20 Lines target.addDynamicallyLegalOp<TestReturnOp>([](TestReturnOp op) {
// Don't allow F32 operands. // Don't allow F32 operands.
return llvm::none_of(op.getOperandTypes(), return llvm::none_of(op.getOperandTypes(),
[](Type type) { return type.isF32(); }); [](Type type) { return type.isF32(); });
}); });
target.addDynamicallyLegalOp<FuncOp>([&](FuncOp op) { target.addDynamicallyLegalOp<FuncOp>([&](FuncOp op) {
return converter.isSignatureLegal(op.getType()) && return converter.isSignatureLegal(op.getType()) &&
converter.isLegal(&op.getBody()); converter.isLegal(&op.getBody());
}); });
target.addDynamicallyLegalOp<CallOp>( target.addDynamicallyLegalOp<func::CallOp>(
[&](CallOp op) { return converter.isLegal(op); }); [&](func::CallOp op) { return converter.isLegal(op); });
// TestCreateUnregisteredOp creates `arith.constant` operation, // TestCreateUnregisteredOp creates `arith.constant` operation,
// which was not added to target intentionally to test // which was not added to target intentionally to test
// correct error code from conversion driver. // correct error code from conversion driver.
target.addDynamicallyLegalOp<ILLegalOpG>([](ILLegalOpG) { return false; }); target.addDynamicallyLegalOp<ILLegalOpG>([](ILLegalOpG) { return false; });
// Expect the type_producer/type_consumer operations to only operate on f64. // Expect the type_producer/type_consumer operations to only operate on f64.
target.addDynamicallyLegalOp<TestTypeProducerOp>( target.addDynamicallyLegalOp<TestTypeProducerOp>(
▲ Show 20 Lines • Show All 687 Lines • Show Last 20 Lines

mlir/test/lib/Dialect/Tosa/TosaTestPasses.cpp

//===- TosaTestPasses.cpp -------------------------------------------------===// //===- TosaTestPasses.cpp -------------------------------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// Test passes to exercise TOSA helper functions. // Test passes to exercise TOSA helper functions.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Tensor/IR/Tensor.h" #include "mlir/Dialect/Tensor/IR/Tensor.h"
#include "mlir/Dialect/Tosa/IR/TosaOps.h" #include "mlir/Dialect/Tosa/IR/TosaOps.h"
#include "mlir/Dialect/Tosa/Transforms/PassDetail.h" #include "mlir/Dialect/Tosa/Transforms/PassDetail.h"
#include "mlir/Dialect/Tosa/Transforms/Passes.h" #include "mlir/Dialect/Tosa/Transforms/Passes.h"
#include "mlir/Dialect/Tosa/Utils/QuantUtils.h" #include "mlir/Dialect/Tosa/Utils/QuantUtils.h"
#include "mlir/IR/BuiltinTypes.h" #include "mlir/IR/BuiltinTypes.h"
#include "mlir/IR/Matchers.h" #include "mlir/IR/Matchers.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
▲ Show 20 Lines • Show All 185 Lines • Show Last 20 Lines

mlir/test/lib/Dialect/Vector/CMakeLists.txt

# Exclude tests from libMLIR.so # Exclude tests from libMLIR.so
add_mlir_library(MLIRVectorTestPasses add_mlir_library(MLIRVectorTestPasses
TestVectorTransforms.cpp TestVectorTransforms.cpp
EXCLUDE_FROM_LIBMLIR EXCLUDE_FROM_LIBMLIR
LINK_LIBS PUBLIC LINK_LIBS PUBLIC
MLIRAffine MLIRAffine
MLIRAnalysis MLIRAnalysis
MLIRFunc
MLIRLinalg MLIRLinalg
MLIRLinalgTransforms MLIRLinalgTransforms
MLIRLLVMIR MLIRLLVMIR
MLIRMemRef MLIRMemRef
MLIRPass MLIRPass
MLIRSCF MLIRSCF
MLIRStandard
MLIRTransformUtils MLIRTransformUtils
MLIRVector MLIRVector
MLIRVectorToSCF MLIRVectorToSCF
MLIRX86Vector MLIRX86Vector
) )

mlir/test/lib/Dialect/Vector/TestVectorTransforms.cpp

//===- TestVectorTransforms.cpp - Test Vector transforms and lowerings ----===// //===- TestVectorTransforms.cpp - Test Vector transforms and lowerings ----===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include <type_traits> #include <type_traits>
#include "mlir/Analysis/SliceAnalysis.h" #include "mlir/Analysis/SliceAnalysis.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/Linalg/Passes.h" #include "mlir/Dialect/Linalg/Passes.h"
#include "mlir/Dialect/Linalg/Transforms/Transforms.h" #include "mlir/Dialect/Linalg/Transforms/Transforms.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/SCF/SCF.h" #include "mlir/Dialect/SCF/SCF.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Vector/Transforms/VectorTransforms.h" #include "mlir/Dialect/Vector/Transforms/VectorTransforms.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Pass/PassManager.h" #include "mlir/Pass/PassManager.h"
#include "mlir/Transforms/GreedyPatternRewriteDriver.h" #include "mlir/Transforms/GreedyPatternRewriteDriver.h"
using namespace mlir; using namespace mlir;
using namespace mlir::linalg; using namespace mlir::linalg;
using namespace mlir::vector; using namespace mlir::vector;
▲ Show 20 Lines • Show All 358 Lines • ▼ Show 20 Lines void runOnOperation() override {
FuncOp func = getOperation(); FuncOp func = getOperation();
func.walk([&](arith::AddFOp op) { func.walk([&](arith::AddFOp op) {
// Check that the operation type can be broken down into a loop. // Check that the operation type can be broken down into a loop.
VectorType type = op.getType().dyn_cast<VectorType>(); VectorType type = op.getType().dyn_cast<VectorType>();
if (!type || type.getRank() != 1 || if (!type || type.getRank() != 1 ||
type.getNumElements() % multiplicity != 0) type.getNumElements() % multiplicity != 0)
return mlir::WalkResult::advance(); return mlir::WalkResult::advance();
auto filterAlloc = [](Operation *op) { auto filterAlloc = [](Operation *op) {
return !isa<arith::ConstantOp, memref::AllocOp, CallOp>(op); return !isa<arith::ConstantOp, memref::AllocOp, func::CallOp>(op);
}; };
auto dependentOps = getSlice(op, filterAlloc); auto dependentOps = getSlice(op, filterAlloc);
// Create a loop and move instructions from the Op slice into the loop. // Create a loop and move instructions from the Op slice into the loop.
OpBuilder builder(op); OpBuilder builder(op);
auto zero = builder.create<arith::ConstantIndexOp>(op.getLoc(), 0); auto zero = builder.create<arith::ConstantIndexOp>(op.getLoc(), 0);
auto one = builder.create<arith::ConstantIndexOp>(op.getLoc(), 1); auto one = builder.create<arith::ConstantIndexOp>(op.getLoc(), 1);
auto numIter = auto numIter =
builder.create<arith::ConstantIndexOp>(op.getLoc(), multiplicity); builder.create<arith::ConstantIndexOp>(op.getLoc(), multiplicity);
▲ Show 20 Lines • Show All 280 Lines • Show Last 20 Lines

mlir/test/lib/IR/TestMatchers.cpp

//===- TestMatchers.cpp - Pass to test matchers ---------------------------===// //===- TestMatchers.cpp - Pass to test matchers ---------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/Matchers.h" #include "mlir/IR/Matchers.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
using namespace mlir; using namespace mlir;
namespace { namespace {
/// This is a test pass for verifying matchers. /// This is a test pass for verifying matchers.
▲ Show 20 Lines • Show All 141 Lines • Show Last 20 Lines

mlir/test/lib/IR/TestOperationEquals.cpp

//===- TestOperationEquals.cpp - Passes to test OperationEquivalence ------===// //===- TestOperationEquals.cpp - Passes to test OperationEquivalence ------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
using namespace mlir; using namespace mlir;
namespace { namespace {
/// This pass illustrates the IR def-use chains through printing. /// This pass illustrates the IR def-use chains through printing.
struct TestOperationEqualPass struct TestOperationEqualPass
Show All 38 Lines

mlir/test/lib/IR/TestPrintDefUse.cpp

//===- TestPrintDefUse.cpp - Passes to illustrate the IR def-use chains ---===// //===- TestPrintDefUse.cpp - Passes to illustrate the IR def-use chains ---===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
using namespace mlir; using namespace mlir;
namespace { namespace {
/// This pass illustrates the IR def-use chains through printing. /// This pass illustrates the IR def-use chains through printing.
struct TestPrintDefUsePass struct TestPrintDefUsePass
▲ Show 20 Lines • Show All 53 Lines • Show Last 20 Lines

mlir/test/lib/IR/TestPrintNesting.cpp

//===- TestPrintNesting.cpp - Passes to illustrate the IR nesting ---------===// //===- TestPrintNesting.cpp - Passes to illustrate the IR nesting ---------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
using namespace mlir; using namespace mlir;
namespace { namespace {
/// This pass illustrates the IR nesting through printing. /// This pass illustrates the IR nesting through printing.
struct TestPrintNestingPass struct TestPrintNestingPass
▲ Show 20 Lines • Show All 80 Lines • Show Last 20 Lines

mlir/test/lib/IR/TestSlicing.cpp

//===- TestSlicing.cpp - Testing slice functionality ----------------------===// //===- TestSlicing.cpp - Testing slice functionality ----------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements a simple testing pass for slicing. // This file implements a simple testing pass for slicing.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/SliceAnalysis.h" #include "mlir/Analysis/SliceAnalysis.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/Linalg/IR/Linalg.h" #include "mlir/Dialect/Linalg/IR/Linalg.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Support/LLVM.h" #include "mlir/Support/LLVM.h"
using namespace mlir; using namespace mlir;
Show All 10 Linesstatic LogicalResult createBackwardSliceFunction(Operation *op,
BlockAndValueMapping mapper; BlockAndValueMapping mapper;
builder.setInsertionPointToEnd(clonedFuncOp.addEntryBlock()); builder.setInsertionPointToEnd(clonedFuncOp.addEntryBlock());
for (const auto &arg : enumerate(parentFuncOp.getArguments())) for (const auto &arg : enumerate(parentFuncOp.getArguments()))
mapper.map(arg.value(), clonedFuncOp.getArgument(arg.index())); mapper.map(arg.value(), clonedFuncOp.getArgument(arg.index()));
SetVector<Operation *> slice; SetVector<Operation *> slice;
getBackwardSlice(op, &slice); getBackwardSlice(op, &slice);
for (Operation *slicedOp : slice) for (Operation *slicedOp : slice)
builder.clone(*slicedOp, mapper); builder.clone(*slicedOp, mapper);
builder.create<ReturnOp>(loc); builder.create<func::ReturnOp>(loc);
return success(); return success();
} }
namespace { namespace {
/// Pass to test slice generated from slice analysis. /// Pass to test slice generated from slice analysis.
struct SliceAnalysisTestPass struct SliceAnalysisTestPass
: public PassWrapper<SliceAnalysisTestPass, OperationPass<ModuleOp>> { : public PassWrapper<SliceAnalysisTestPass, OperationPass<ModuleOp>> {
StringRef getArgument() const final { return "slice-analysis-test"; } StringRef getArgument() const final { return "slice-analysis-test"; }
Show All 33 Lines

mlir/test/lib/Transforms/TestInlining.cpp

//===- TestInlining.cpp - Pass to inline calls in the test dialect --------===// //===- TestInlining.cpp - Pass to inline calls in the test dialect --------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// TODO: This pass is only necessary because the main inlining pass // TODO: This pass is only necessary because the main inlining pass
// has no abstracted away the call+callee relationship. When the inlining // has no abstracted away the call+callee relationship. When the inlining
// interface has this support, this pass should be removed. // interface has this support, this pass should be removed.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "TestDialect.h" #include "TestDialect.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/InliningUtils.h" #include "mlir/Transforms/InliningUtils.h"
#include "llvm/ADT/StringSet.h" #include "llvm/ADT/StringSet.h"
using namespace mlir; using namespace mlir;
using namespace test; using namespace test;
namespace { namespace {
struct Inliner : public PassWrapper<Inliner, OperationPass<FuncOp>> { struct Inliner : public PassWrapper<Inliner, OperationPass<FuncOp>> {
StringRef getArgument() const final { return "test-inline"; } StringRef getArgument() const final { return "test-inline"; }
StringRef getDescription() const final { StringRef getDescription() const final {
return "Test inlining region calls"; return "Test inlining region calls";
} }
void runOnOperation() override { void runOnOperation() override {
auto function = getOperation(); auto function = getOperation();
// Collect each of the direct function calls within the module. // Collect each of the direct function calls within the module.
SmallVector<CallIndirectOp, 16> callers; SmallVector<func::CallIndirectOp, 16> callers;
function.walk([&](CallIndirectOp caller) { callers.push_back(caller); }); function.walk(
[&](func::CallIndirectOp caller) { callers.push_back(caller); });
// Build the inliner interface. // Build the inliner interface.
InlinerInterface interface(&getContext()); InlinerInterface interface(&getContext());
// Try to inline each of the call operations. // Try to inline each of the call operations.
for (auto caller : callers) { for (auto caller : callers) {
auto callee = dyn_cast_or_null<FunctionalRegionOp>( auto callee = dyn_cast_or_null<FunctionalRegionOp>(
caller.getCallee().getDefiningOp()); caller.getCallee().getDefiningOp());
Show All 26 Lines

mlir/test/mlir-opt/commandline.mlir

// RUN: mlir-opt --show-dialects | FileCheck %s // RUN: mlir-opt --show-dialects | FileCheck %s
// CHECK: Available Dialects: // CHECK: Available Dialects:
// CHECK-NEXT: acc // CHECK-NEXT: acc
// CHECK-NEXT: affine // CHECK-NEXT: affine
// CHECK-NEXT: amx // CHECK-NEXT: amx
// CHECK-NEXT: arith // CHECK-NEXT: arith
// CHECK-NEXT: arm_neon // CHECK-NEXT: arm_neon
// CHECK-NEXT: arm_sve // CHECK-NEXT: arm_sve
// CHECK-NEXT: async // CHECK-NEXT: async
// CHECK-NEXT: bufferization // CHECK-NEXT: bufferization
// CHECK-NEXT: builtin // CHECK-NEXT: builtin
// CHECK-NEXT: cf // CHECK-NEXT: cf
// CHECK-NEXT: complex // CHECK-NEXT: complex
// CHECK-NEXT: dlti // CHECK-NEXT: dlti
// CHECK-NEXT: emitc // CHECK-NEXT: emitc
// CHECK-NEXT: func
// CHECK-NEXT: gpu // CHECK-NEXT: gpu
// CHECK-NEXT: linalg // CHECK-NEXT: linalg
// CHECK-NEXT: llvm // CHECK-NEXT: llvm
// CHECK-NEXT: math // CHECK-NEXT: math
// CHECK-NEXT: memref // CHECK-NEXT: memref
// CHECK-NEXT: nvvm // CHECK-NEXT: nvvm
// CHECK-NEXT: omp // CHECK-NEXT: omp
// CHECK-NEXT: pdl // CHECK-NEXT: pdl
// CHECK-NEXT: pdl_interp // CHECK-NEXT: pdl_interp
// CHECK-NEXT: quant // CHECK-NEXT: quant
// CHECK-NEXT: rocdl // CHECK-NEXT: rocdl
// CHECK-NEXT: scf // CHECK-NEXT: scf
// CHECK-NEXT: shape // CHECK-NEXT: shape
// CHECK-NEXT: sparse_tensor // CHECK-NEXT: sparse_tensor
// CHECK-NEXT: spv // CHECK-NEXT: spv
// CHECK-NEXT: std
// CHECK-NEXT: tensor // CHECK-NEXT: tensor
// CHECK-NEXT: test // CHECK-NEXT: test
// CHECK-NEXT: tosa // CHECK-NEXT: tosa
// CHECK-NEXT: vector // CHECK-NEXT: vector
// CHECK-NEXT: x86vector // CHECK-NEXT: x86vector

mlir/test/python/dialects/builtin.py

# RUN: %PYTHON %s | FileCheck %s # RUN: %PYTHON %s | FileCheck %s
from mlir.ir import * from mlir.ir import *
import mlir.dialects.builtin as builtin import mlir.dialects.builtin as builtin
import mlir.dialects.std as std import mlir.dialects.func as func
def run(f): def run(f):
print("\nTEST:", f.__name__) print("\nTEST:", f.__name__)
f() f()
return f return f
▲ Show 20 Lines • Show All 73 Lines • ▼ Show 20 Lines with InsertionPoint(m.body):
def kwargs_func_op(a, b=None, **kwargs): def kwargs_func_op(a, b=None, **kwargs):
assert isinstance(kwargs["func_op"], builtin.FuncOp) assert isinstance(kwargs["func_op"], builtin.FuncOp)
return b return b
# CHECK-LABEL: func @explicit_results(%arg0: f32, %arg1: f64) -> f64 # CHECK-LABEL: func @explicit_results(%arg0: f32, %arg1: f64) -> f64
# CHECK: return %arg1 : f64 # CHECK: return %arg1 : f64
@builtin.FuncOp.from_py_func(f32, f64, results=[f64]) @builtin.FuncOp.from_py_func(f32, f64, results=[f64])
def explicit_results(a, b): def explicit_results(a, b):
std.ReturnOp([b]) func.ReturnOp([b])
print(m) print(m)
# CHECK-LABEL: TEST: testFromPyFuncErrors # CHECK-LABEL: TEST: testFromPyFuncErrors
@run @run
def testFromPyFuncErrors(): def testFromPyFuncErrors():
with Context() as ctx, Location.unknown() as loc: with Context() as ctx, Location.unknown() as loc:
Show All 37 Lines with InsertionPoint.at_block_begin(m.body):
try: try:
entry_block = func.entry_block entry_block = func.entry_block
except IndexError as e: except IndexError as e:
# CHECK: External function does not have a body # CHECK: External function does not have a body
print(e) print(e)
with InsertionPoint(func.add_entry_block()): with InsertionPoint(func.add_entry_block()):
std.ReturnOp([func.entry_block.arguments[0]]) func.ReturnOp([func.entry_block.arguments[0]])
pass pass
try: try:
func.add_entry_block() func.add_entry_block()
except IndexError as e: except IndexError as e:
# CHECK: The function already has an entry block! # CHECK: The function already has an entry block!
print(e) print(e)
# Try the callback builder and passing type as tuple. # Try the callback builder and passing type as tuple.
func = builtin.FuncOp(name="some_other_func", func = builtin.FuncOp(name="some_other_func",
type=([tensor_type, tensor_type], [tensor_type]), type=([tensor_type, tensor_type], [tensor_type]),
visibility="nested", visibility="nested",
body_builder=lambda func: std.ReturnOp( body_builder=lambda func: func.ReturnOp(
[func.entry_block.arguments[0]])) [func.entry_block.arguments[0]]))
# CHECK: module { # CHECK: module {
# CHECK: func nested @some_func(%arg0: tensor<2x3x4xf32>, %arg1: tensor<2x3x4xf32>) -> tensor<2x3x4xf32> { # CHECK: func nested @some_func(%arg0: tensor<2x3x4xf32>, %arg1: tensor<2x3x4xf32>) -> tensor<2x3x4xf32> {
# CHECK: return %arg0 : tensor<2x3x4xf32> # CHECK: return %arg0 : tensor<2x3x4xf32>
# CHECK: } # CHECK: }
# CHECK: func nested @some_other_func(%arg0: tensor<2x3x4xf32>, %arg1: tensor<2x3x4xf32>) -> tensor<2x3x4xf32> { # CHECK: func nested @some_other_func(%arg0: tensor<2x3x4xf32>, %arg1: tensor<2x3x4xf32>) -> tensor<2x3x4xf32> {
# CHECK: return %arg0 : tensor<2x3x4xf32> # CHECK: return %arg0 : tensor<2x3x4xf32>
# CHECK: } # CHECK: }
print(m) print(m)
# CHECK-LABEL: TEST: testFuncArgumentAccess # CHECK-LABEL: TEST: testFuncArgumentAccess
@run @run
def testFuncArgumentAccess(): def testFuncArgumentAccess():
with Context() as ctx, Location.unknown(): with Context() as ctx, Location.unknown():
ctx.allow_unregistered_dialects = True ctx.allow_unregistered_dialects = True
module = Module.create() module = Module.create()
f32 = F32Type.get() f32 = F32Type.get()
f64 = F64Type.get() f64 = F64Type.get()
with InsertionPoint(module.body): with InsertionPoint(module.body):
func = builtin.FuncOp("some_func", ([f32, f32], [f32, f32])) func = builtin.FuncOp("some_func", ([f32, f32], [f32, f32]))
with InsertionPoint(func.add_entry_block()): with InsertionPoint(func.add_entry_block()):
std.ReturnOp(func.arguments) func.ReturnOp(func.arguments)
func.arg_attrs = ArrayAttr.get([ func.arg_attrs = ArrayAttr.get([
DictAttr.get({ DictAttr.get({
"custom_dialect.foo": StringAttr.get("bar"), "custom_dialect.foo": StringAttr.get("bar"),
"custom_dialect.baz": UnitAttr.get() "custom_dialect.baz": UnitAttr.get()
}), }),
DictAttr.get({"custom_dialect.qux": ArrayAttr.get([])}) DictAttr.get({"custom_dialect.qux": ArrayAttr.get([])})
]) ])
func.result_attrs = ArrayAttr.get([ func.result_attrs = ArrayAttr.get([
DictAttr.get({"custom_dialect.res1": FloatAttr.get(f32, 42.0)}), DictAttr.get({"custom_dialect.res1": FloatAttr.get(f32, 42.0)}),
DictAttr.get({"custom_dialect.res2": FloatAttr.get(f64, 256.0)}) DictAttr.get({"custom_dialect.res2": FloatAttr.get(f64, 256.0)})
]) ])
other = builtin.FuncOp("other_func", ([f32, f32], [])) other = builtin.FuncOp("other_func", ([f32, f32], []))
with InsertionPoint(other.add_entry_block()): with InsertionPoint(other.add_entry_block()):
std.ReturnOp([]) func.ReturnOp([])
other.arg_attrs = [ other.arg_attrs = [
DictAttr.get({"custom_dialect.foo": StringAttr.get("qux")}), DictAttr.get({"custom_dialect.foo": StringAttr.get("qux")}),
DictAttr.get() DictAttr.get()
] ]
# CHECK: [{custom_dialect.baz, custom_dialect.foo = "bar"}, {custom_dialect.qux = []}] # CHECK: [{custom_dialect.baz, custom_dialect.foo = "bar"}, {custom_dialect.qux = []}]
print(func.arg_attrs) print(func.arg_attrs)
Show All 14 Lines

mlir/test/python/dialects/func.py

  • This file was moved from mlir/test/python/dialects/std.py.
# RUN: %PYTHON %s | FileCheck %s # RUN: %PYTHON %s | FileCheck %s
from mlir.ir import * from mlir.ir import *
from mlir.dialects import arith from mlir.dialects import arith
from mlir.dialects import builtin from mlir.dialects import builtin
from mlir.dialects import std from mlir.dialects import func
def constructAndPrintInModule(f): def constructAndPrintInModule(f):
print("\nTEST:", f.__name__) print("\nTEST:", f.__name__)
with Context(), Location.unknown(): with Context(), Location.unknown():
module = Module.create() module = Module.create()
with InsertionPoint(module.body): with InsertionPoint(module.body):
f() f()
▲ Show 20 Lines • Show All 65 Lines • ▼ Show 20 Linesdef testFunctionCalls():
foo = builtin.FuncOp("foo", ([], [])) foo = builtin.FuncOp("foo", ([], []))
foo.sym_visibility = StringAttr.get("private") foo.sym_visibility = StringAttr.get("private")
bar = builtin.FuncOp("bar", ([], [IndexType.get()])) bar = builtin.FuncOp("bar", ([], [IndexType.get()]))
bar.sym_visibility = StringAttr.get("private") bar.sym_visibility = StringAttr.get("private")
qux = builtin.FuncOp("qux", ([], [F32Type.get()])) qux = builtin.FuncOp("qux", ([], [F32Type.get()]))
qux.sym_visibility = StringAttr.get("private") qux.sym_visibility = StringAttr.get("private")
with InsertionPoint(builtin.FuncOp("caller", ([], [])).add_entry_block()): with InsertionPoint(builtin.FuncOp("caller", ([], [])).add_entry_block()):
std.CallOp(foo, []) func.CallOp(foo, [])
std.CallOp([IndexType.get()], "bar", []) func.CallOp([IndexType.get()], "bar", [])
std.CallOp([F32Type.get()], FlatSymbolRefAttr.get("qux"), []) func.CallOp([F32Type.get()], FlatSymbolRefAttr.get("qux"), [])
std.ReturnOp([]) func.ReturnOp([])
# CHECK: func private @foo() # CHECK: func private @foo()
# CHECK: func private @bar() -> index # CHECK: func private @bar() -> index
# CHECK: func private @qux() -> f32 # CHECK: func private @qux() -> f32
# CHECK: func @caller() { # CHECK: func @caller() {
# CHECK: call @foo() : () -> () # CHECK: call @foo() : () -> ()
# CHECK: %0 = call @bar() : () -> index # CHECK: %0 = call @bar() : () -> index
# CHECK: %1 = call @qux() : () -> f32 # CHECK: %1 = call @qux() : () -> f32
# CHECK: return # CHECK: return
# CHECK: } # CHECK: }

mlir/test/python/dialects/memref.py

# RUN: %PYTHON %s | FileCheck %s # RUN: %PYTHON %s | FileCheck %s
from mlir.ir import * from mlir.ir import *
import mlir.dialects.std as std import mlir.dialects.func as func
import mlir.dialects.memref as memref import mlir.dialects.memref as memref
def run(f): def run(f):
print("\nTEST:", f.__name__) print("\nTEST:", f.__name__)
f() f()
return f return f
▲ Show 20 Lines • Show All 45 Lines • ▼ Show 20 Lines
@run @run
def testCustomBuidlers(): def testCustomBuidlers():
with Context() as ctx, Location.unknown(ctx): with Context() as ctx, Location.unknown(ctx):
module = Module.parse(r""" module = Module.parse(r"""
func @f1(%arg0: memref<?x?xf32>, %arg1: index, %arg2: index) { func @f1(%arg0: memref<?x?xf32>, %arg1: index, %arg2: index) {
return return
} }
""") """)
func = module.body.operations[0] f = module.body.operations[0]
func_body = func.regions[0].blocks[0] func_body = f.regions[0].blocks[0]
with InsertionPoint.at_block_terminator(func_body): with InsertionPoint.at_block_terminator(func_body):
memref.LoadOp(func.arguments[0], func.arguments[1:]) memref.LoadOp(f.arguments[0], f.arguments[1:])
# CHECK: func @f1(%[[ARG0:.*]]: memref<?x?xf32>, %[[ARG1:.*]]: index, %[[ARG2:.*]]: index) # CHECK: func @f1(%[[ARG0:.*]]: memref<?x?xf32>, %[[ARG1:.*]]: index, %[[ARG2:.*]]: index)
# CHECK: memref.load %[[ARG0]][%[[ARG1]], %[[ARG2]]] # CHECK: memref.load %[[ARG0]][%[[ARG1]], %[[ARG2]]]
print(module) print(module)
assert module.operation.verify() assert module.operation.verify()

mlir/test/python/dialects/scf.py

# RUN: %PYTHON %s | FileCheck %s # RUN: %PYTHON %s | FileCheck %s
from mlir.ir import * from mlir.ir import *
from mlir.dialects import arith from mlir.dialects import arith
from mlir.dialects import func
from mlir.dialects import scf from mlir.dialects import scf
from mlir.dialects import std
from mlir.dialects import builtin from mlir.dialects import builtin
def constructAndPrintInModule(f): def constructAndPrintInModule(f):
print("\nTEST:", f.__name__) print("\nTEST:", f.__name__)
with Context(), Location.unknown(): with Context(), Location.unknown():
module = Module.create() module = Module.create()
with InsertionPoint(module.body): with InsertionPoint(module.body):
Show All 39 Lines
# CHECK: scf.yield %[[IV]] # CHECK: scf.yield %[[IV]]
@constructAndPrintInModule @constructAndPrintInModule
def testOpsAsArguments(): def testOpsAsArguments():
index_type = IndexType.get() index_type = IndexType.get()
callee = builtin.FuncOp( callee = builtin.FuncOp(
"callee", ([], [index_type, index_type]), visibility="private") "callee", ([], [index_type, index_type]), visibility="private")
func = builtin.FuncOp("ops_as_arguments", ([], [])) f = builtin.FuncOp("ops_as_arguments", ([], []))
with InsertionPoint(func.add_entry_block()): with InsertionPoint(func.add_entry_block()):
lb = arith.ConstantOp.create_index(0) lb = arith.ConstantOp.create_index(0)
ub = arith.ConstantOp.create_index(42) ub = arith.ConstantOp.create_index(42)
step = arith.ConstantOp.create_index(2) step = arith.ConstantOp.create_index(2)
iter_args = std.CallOp(callee, []) iter_args = func.CallOp(callee, [])
ftynseUnsubmitted
Done
ReplyInline Actions

There's a name conflict between the imported func module and the func variable defined at line 62.

ftynse: There's a name conflict between the imported `func` module and the `func` variable defined at…
loop = scf.ForOp(lb, ub, step, iter_args) loop = scf.ForOp(lb, ub, step, iter_args)
with InsertionPoint(loop.body): with InsertionPoint(loop.body):
scf.YieldOp(loop.inner_iter_args) scf.YieldOp(loop.inner_iter_args)
std.ReturnOp([]) func.ReturnOp([])
# CHECK-LABEL: TEST: testOpsAsArguments # CHECK-LABEL: TEST: testOpsAsArguments
# CHECK: func private @callee() -> (index, index) # CHECK: func private @callee() -> (index, index)
# CHECK: func @ops_as_arguments() { # CHECK: func @ops_as_arguments() {
# CHECK: %[[LB:.*]] = arith.constant 0 # CHECK: %[[LB:.*]] = arith.constant 0
# CHECK: %[[UB:.*]] = arith.constant 42 # CHECK: %[[UB:.*]] = arith.constant 42
# CHECK: %[[STEP:.*]] = arith.constant 2 # CHECK: %[[STEP:.*]] = arith.constant 2
# CHECK: %[[ARGS:.*]]:2 = call @callee() # CHECK: %[[ARGS:.*]]:2 = call @callee()
# CHECK: scf.for %arg0 = %c0 to %c42 step %c2 # CHECK: scf.for %arg0 = %c0 to %c42 step %c2
# CHECK: iter_args(%{{.*}} = %[[ARGS]]#0, %{{.*}} = %[[ARGS]]#1) # CHECK: iter_args(%{{.*}} = %[[ARGS]]#0, %{{.*}} = %[[ARGS]]#1)
# CHECK: scf.yield %{{.*}}, %{{.*}} # CHECK: scf.yield %{{.*}}, %{{.*}}
# CHECK: return # CHECK: return

mlir/test/python/dialects/std.py

  • This file was moved to mlir/test/python/dialects/func.py.

mlir/test/python/dialects/vector.py

# RUN: %PYTHON %s | FileCheck %s # RUN: %PYTHON %s | FileCheck %s
from mlir.ir import * from mlir.ir import *
import mlir.dialects.builtin as builtin import mlir.dialects.builtin as builtin
import mlir.dialects.std as std import mlir.dialects.func as func
import mlir.dialects.vector as vector import mlir.dialects.vector as vector
def run(f): def run(f):
print("\nTEST:", f.__name__) print("\nTEST:", f.__name__)
with Context(), Location.unknown(): with Context(), Location.unknown():
f() f()
return f return f
Show All 30 Lines func = builtin.FuncOp("transfer_read",
([memref_type, index_type, ([memref_type, index_type,
F32Type.get(), mask_type], [])) F32Type.get(), mask_type], []))
with InsertionPoint(func.add_entry_block()): with InsertionPoint(func.add_entry_block()):
A, zero, padding, mask = func.arguments A, zero, padding, mask = func.arguments
vector.TransferReadOp(vector_type, A, [zero, zero], identity_map_attr, vector.TransferReadOp(vector_type, A, [zero, zero], identity_map_attr,
padding, mask, None) padding, mask, None)
vector.TransferReadOp(vector_type, A, [zero, zero], identity_map_attr, vector.TransferReadOp(vector_type, A, [zero, zero], identity_map_attr,
padding, None, None) padding, None, None)
std.ReturnOp([]) func.ReturnOp([])
# CHECK: @transfer_read(%[[MEM:.*]]: memref<?x?xf32>, %[[IDX:.*]]: index, # CHECK: @transfer_read(%[[MEM:.*]]: memref<?x?xf32>, %[[IDX:.*]]: index,
# CHECK: %[[PAD:.*]]: f32, %[[MASK:.*]]: vector<2x3xi1>) # CHECK: %[[PAD:.*]]: f32, %[[MASK:.*]]: vector<2x3xi1>)
# CHECK: vector.transfer_read %[[MEM]][%[[IDX]], %[[IDX]]], %[[PAD]], %[[MASK]] # CHECK: vector.transfer_read %[[MEM]][%[[IDX]], %[[IDX]]], %[[PAD]], %[[MASK]]
# CHECK: vector.transfer_read %[[MEM]][%[[IDX]], %[[IDX]]], %[[PAD]] # CHECK: vector.transfer_read %[[MEM]][%[[IDX]], %[[IDX]]], %[[PAD]]
# CHECK-NOT: %[[MASK]] # CHECK-NOT: %[[MASK]]
print(module) print(module)

mlir/test/python/ir/blocks.py

# RUN: %PYTHON %s | FileCheck %s # RUN: %PYTHON %s | FileCheck %s
import gc import gc
import io import io
import itertools import itertools
from mlir.ir import * from mlir.ir import *
from mlir.dialects import builtin from mlir.dialects import builtin
from mlir.dialects import cf from mlir.dialects import cf
# Note: std dialect needed for terminators. from mlir.dialects import func
from mlir.dialects import std
def run(f): def run(f):
print("\nTEST:", f.__name__) print("\nTEST:", f.__name__)
f() f()
gc.collect() gc.collect()
assert Context._get_live_count() == 0 assert Context._get_live_count() == 0
return f return f
Show All 15 Lines with InsertionPoint(module.body):
[IntegerType.get_signless(32), [IntegerType.get_signless(32),
IntegerType.get_signless(16)], []) IntegerType.get_signless(16)], [])
f_op = builtin.FuncOp("test", f_type) f_op = builtin.FuncOp("test", f_type)
entry_block = f_op.add_entry_block() entry_block = f_op.add_entry_block()
i32_arg, i16_arg = entry_block.arguments i32_arg, i16_arg = entry_block.arguments
successor_block = entry_block.create_after(i32_arg.type) successor_block = entry_block.create_after(i32_arg.type)
with InsertionPoint(successor_block) as successor_ip: with InsertionPoint(successor_block) as successor_ip:
assert successor_ip.block == successor_block assert successor_ip.block == successor_block
std.ReturnOp([]) func.ReturnOp([])
middle_block = successor_block.create_before(i16_arg.type) middle_block = successor_block.create_before(i16_arg.type)
with InsertionPoint(entry_block) as entry_ip: with InsertionPoint(entry_block) as entry_ip:
assert entry_ip.block == entry_block assert entry_ip.block == entry_block
cf.BranchOp([i16_arg], dest=middle_block) cf.BranchOp([i16_arg], dest=middle_block)
with InsertionPoint(middle_block) as middle_ip: with InsertionPoint(middle_block) as middle_ip:
assert middle_ip.block == middle_block assert middle_ip.block == middle_block
Show All 10 Lines
def testFirstBlockCreation(): def testFirstBlockCreation():
with Context() as ctx, Location.unknown(): with Context() as ctx, Location.unknown():
module = Module.create() module = Module.create()
f32 = F32Type.get() f32 = F32Type.get()
with InsertionPoint(module.body): with InsertionPoint(module.body):
func = builtin.FuncOp("test", ([f32], [])) func = builtin.FuncOp("test", ([f32], []))
entry_block = Block.create_at_start(func.operation.regions[0], [f32]) entry_block = Block.create_at_start(func.operation.regions[0], [f32])
with InsertionPoint(entry_block): with InsertionPoint(entry_block):
std.ReturnOp([]) func.ReturnOp([])
print(module) print(module)
assert module.operation.verify() assert module.operation.verify()
assert func.body.blocks[0] == entry_block assert func.body.blocks[0] == entry_block

mlir/test/python/ir/dialects.py

Show All 10 Linesdef run(f):
assert Context._get_live_count() == 0 assert Context._get_live_count() == 0
return f return f
# CHECK-LABEL: TEST: testDialectDescriptor # CHECK-LABEL: TEST: testDialectDescriptor
@run @run
def testDialectDescriptor(): def testDialectDescriptor():
ctx = Context() ctx = Context()
d = ctx.get_dialect_descriptor("std") d = ctx.get_dialect_descriptor("func")
# CHECK: <DialectDescriptor std> # CHECK: <DialectDescriptor func>
print(d) print(d)
# CHECK: std # CHECK: func
print(d.namespace) print(d.namespace)
try: try:
_ = ctx.get_dialect_descriptor("not_existing") _ = ctx.get_dialect_descriptor("not_existing")
except ValueError: except ValueError:
pass pass
else: else:
assert False, "Expected exception" assert False, "Expected exception"
# CHECK-LABEL: TEST: testUserDialectClass # CHECK-LABEL: TEST: testUserDialectClass
@run @run
def testUserDialectClass(): def testUserDialectClass():
ctx = Context() ctx = Context()
# Access using attribute. # Access using attribute.
d = ctx.dialects.std d = ctx.dialects.func
# CHECK: <Dialect std (class mlir.dialects._std_ops_gen._Dialect)> # CHECK: <Dialect func (class mlir.dialects._func_ops_gen._Dialect)>
print(d) print(d)
try: try:
_ = ctx.dialects.not_existing _ = ctx.dialects.not_existing
except AttributeError: except AttributeError:
pass pass
else: else:
assert False, "Expected exception" assert False, "Expected exception"
# Access using index. # Access using index.
d = ctx.dialects["std"] d = ctx.dialects["func"]
# CHECK: <Dialect std (class mlir.dialects._std_ops_gen._Dialect)> # CHECK: <Dialect func (class mlir.dialects._func_ops_gen._Dialect)>
print(d) print(d)
try: try:
_ = ctx.dialects["not_existing"] _ = ctx.dialects["not_existing"]
except IndexError: except IndexError:
pass pass
else: else:
assert False, "Expected exception" assert False, "Expected exception"
# Using the 'd' alias. # Using the 'd' alias.
d = ctx.d["std"] d = ctx.d["func"]
# CHECK: <Dialect std (class mlir.dialects._std_ops_gen._Dialect)> # CHECK: <Dialect func (class mlir.dialects._func_ops_gen._Dialect)>
print(d) print(d)
# CHECK-LABEL: TEST: testCustomOpView # CHECK-LABEL: TEST: testCustomOpView
# This test uses the standard dialect AddFOp as an example of a user op. # This test uses the standard dialect AddFOp as an example of a user op.
# TODO: Op creation and access is still quite verbose: simplify this test as # TODO: Op creation and access is still quite verbose: simplify this test as
# additional capabilities come online. # additional capabilities come online.
@run @run
Show All 28 Lines
# CHECK-LABEL: TEST: testIsRegisteredOperation # CHECK-LABEL: TEST: testIsRegisteredOperation
@run @run
def testIsRegisteredOperation(): def testIsRegisteredOperation():
ctx = Context() ctx = Context()
# CHECK: cf.cond_br: True # CHECK: cf.cond_br: True
print(f"cf.cond_br: {ctx.is_registered_operation('cf.cond_br')}") print(f"cf.cond_br: {ctx.is_registered_operation('cf.cond_br')}")
# CHECK: std.not_existing: False # CHECK: func.not_existing: False
print(f"std.not_existing: {ctx.is_registered_operation('std.not_existing')}") print(f"func.not_existing: {ctx.is_registered_operation('func.not_existing')}")

mlir/test/python/ir/operation.py

Show First 20 Lines • Show All 122 Lines • ▼ Show 20 Lines
@run @run
def testBlockAndRegionOwners(): def testBlockAndRegionOwners():
ctx = Context() ctx = Context()
ctx.allow_unregistered_dialects = True ctx.allow_unregistered_dialects = True
module = Module.parse( module = Module.parse(
r""" r"""
builtin.module { builtin.module {
builtin.func @f() { builtin.func @f() {
std.return func.return
} }
} }
""", ctx) """, ctx)
assert module.operation.regions[0].owner == module.operation assert module.operation.regions[0].owner == module.operation
assert module.operation.regions[0].blocks[0].owner == module.operation assert module.operation.regions[0].blocks[0].owner == module.operation
func = module.body.operations[0] func = module.body.operations[0]
▲ Show 20 Lines • Show All 419 Lines • ▼ Show 20 Linesdef testOperationPrint():
# CHECK: return %arg0 : i32 # CHECK: return %arg0 : i32
module.operation.print(file=f, binary=True) module.operation.print(file=f, binary=True)
bytes_value = f.getvalue() bytes_value = f.getvalue()
print(bytes_value.__class__) print(bytes_value.__class__)
print(bytes_value) print(bytes_value)
# Test get_asm with options. # Test get_asm with options.
# CHECK: value = opaque<"elided_large_const", "0xDEADBEEF"> : tensor<4xi32> # CHECK: value = opaque<"elided_large_const", "0xDEADBEEF"> : tensor<4xi32>
# CHECK: "std.return"(%arg0) : (i32) -> () -:4:7 # CHECK: "func.return"(%arg0) : (i32) -> () -:4:7
module.operation.print( module.operation.print(
large_elements_limit=2, large_elements_limit=2,
enable_debug_info=True, enable_debug_info=True,
pretty_debug_info=True, pretty_debug_info=True,
print_generic_op_form=True, print_generic_op_form=True,
use_local_scope=True) use_local_scope=True)
# CHECK-LABEL: TEST: testKnownOpView # CHECK-LABEL: TEST: testKnownOpView
@run @run
def testKnownOpView(): def testKnownOpView():
with Context(), Location.unknown(): with Context(), Location.unknown():
Context.current.allow_unregistered_dialects = True Context.current.allow_unregistered_dialects = True
module = Module.parse(r""" module = Module.parse(r"""
%1 = "custom.f32"() : () -> f32 %1 = "custom.f32"() : () -> f32
%2 = "custom.f32"() : () -> f32 %2 = "custom.f32"() : () -> f32
%3 = arith.addf %1, %2 : f32 %3 = arith.addf %1, %2 : f32
""") """)
print(module) print(module)
# addf should map to a known OpView class in the std dialect. # addf should map to a known OpView class in the arithmetic dialect.
# We know the OpView for it defines an 'lhs' attribute. # We know the OpView for it defines an 'lhs' attribute.
addf = module.body.operations[2] addf = module.body.operations[2]
# CHECK: <mlir.dialects._arith_ops_gen._AddFOp object # CHECK: <mlir.dialects._arith_ops_gen._AddFOp object
print(repr(addf)) print(repr(addf))
# CHECK: "custom.f32"() # CHECK: "custom.f32"()
print(addf.lhs) print(addf.lhs)
# One of the custom ops should resolve to the default OpView. # One of the custom ops should resolve to the default OpView.
▲ Show 20 Lines • Show All 255 Lines • Show Last 20 Lines

mlir/test/python/pass_manager.py

Show First 20 Lines • Show All 86 Lines • ▼ Show 20 Lines
def testRunPipeline(): def testRunPipeline():
with Context(): with Context():
pm = PassManager.parse("print-op-stats") pm = PassManager.parse("print-op-stats")
module = Module.parse(r"""func @successfulParse() { return }""") module = Module.parse(r"""func @successfulParse() { return }""")
pm.run(module) pm.run(module)
# CHECK: Operations encountered: # CHECK: Operations encountered:
# CHECK: builtin.func , 1 # CHECK: builtin.func , 1
# CHECK: builtin.module , 1 # CHECK: builtin.module , 1
# CHECK: std.return , 1 # CHECK: func.return , 1
run(testRunPipeline) run(testRunPipeline)

mlir/tools/mlir-opt/CMakeLists.txt

Show All 9 Linesset(LLVM_LINK_COMPONENTS
AsmParser AsmParser
) )
if(MLIR_INCLUDE_TESTS) if(MLIR_INCLUDE_TESTS)
set(test_libs set(test_libs
MLIRTestStandardToLLVM MLIRTestStandardToLLVM
MLIRAffineTransformsTestPasses MLIRAffineTransformsTestPasses
MLIRDLTITestPasses MLIRDLTITestPasses
MLIRFuncTestPasses
MLIRGPUTestPasses MLIRGPUTestPasses
MLIRLinalgTestPasses MLIRLinalgTestPasses
MLIRMathTestPasses MLIRMathTestPasses
MLIRMemRefTestPasses MLIRMemRefTestPasses
MLIRSCFTestPasses MLIRSCFTestPasses
MLIRShapeTestPasses MLIRShapeTestPasses
MLIRSPIRVTestPasses MLIRSPIRVTestPasses
MLIRStandardOpsTestPasses
MLIRTensorTestPasses MLIRTensorTestPasses
MLIRTestAnalysis MLIRTestAnalysis
MLIRTestDialect MLIRTestDialect
MLIRTestIR MLIRTestIR
MLIRTestPass MLIRTestPass
MLIRTestReducer MLIRTestReducer
MLIRTestRewrite MLIRTestRewrite
MLIRTestTransforms MLIRTestTransforms
▲ Show 20 Lines • Show All 41 Lines • Show Last 20 Lines

mlir/tools/mlir-spirv-cpu-runner/CMakeLists.txt

Show All 12 Linesif (MLIR_ENABLE_SPIRV_CPU_RUNNER)
get_property(conversion_libs GLOBAL PROPERTY MLIR_CONVERSION_LIBS) get_property(conversion_libs GLOBAL PROPERTY MLIR_CONVERSION_LIBS)
target_link_libraries(mlir-spirv-cpu-runner PRIVATE target_link_libraries(mlir-spirv-cpu-runner PRIVATE
${conversion_libs} ${conversion_libs}
MLIRAnalysis MLIRAnalysis
MLIRArithmetic MLIRArithmetic
MLIRExecutionEngine MLIRExecutionEngine
MLIRFunc
MLIRGPUOps MLIRGPUOps
MLIRIR MLIRIR
MLIRJitRunner MLIRJitRunner
MLIRLLVMIR MLIRLLVMIR
MLIRLLVMToLLVMIRTranslation MLIRLLVMToLLVMIRTranslation
MLIRMemRef MLIRMemRef
MLIRParser MLIRParser
MLIRSPIRV MLIRSPIRV
MLIRStandard
MLIRTargetLLVMIRExport MLIRTargetLLVMIRExport
MLIRTransforms MLIRTransforms
MLIRTranslation MLIRTranslation
MLIRSupport MLIRSupport
) )
endif() endif()

mlir/tools/mlir-spirv-cpu-runner/mlir-spirv-cpu-runner.cpp

Show All 10 Lines
// JIT-compiling and executing. // JIT-compiling and executing.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/GPUToSPIRV/GPUToSPIRVPass.h" #include "mlir/Conversion/GPUToSPIRV/GPUToSPIRVPass.h"
#include "mlir/Conversion/SPIRVToLLVM/SPIRVToLLVMPass.h" #include "mlir/Conversion/SPIRVToLLVM/SPIRVToLLVMPass.h"
#include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVMPass.h" #include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVMPass.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/GPU/GPUDialect.h" #include "mlir/Dialect/GPU/GPUDialect.h"
#include "mlir/Dialect/GPU/Passes.h" #include "mlir/Dialect/GPU/Passes.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h" #include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h"
#include "mlir/Dialect/SPIRV/IR/SPIRVOps.h" #include "mlir/Dialect/SPIRV/IR/SPIRVOps.h"
#include "mlir/Dialect/SPIRV/Transforms/Passes.h" #include "mlir/Dialect/SPIRV/Transforms/Passes.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/ExecutionEngine/JitRunner.h" #include "mlir/ExecutionEngine/JitRunner.h"
#include "mlir/ExecutionEngine/OptUtils.h" #include "mlir/ExecutionEngine/OptUtils.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Pass/PassManager.h" #include "mlir/Pass/PassManager.h"
#include "mlir/Target/LLVMIR/Dialect/LLVMIR/LLVMToLLVMIRTranslation.h" #include "mlir/Target/LLVMIR/Dialect/LLVMIR/LLVMToLLVMIRTranslation.h"
#include "mlir/Target/LLVMIR/Export.h" #include "mlir/Target/LLVMIR/Export.h"
#include "llvm/IR/LLVMContext.h" #include "llvm/IR/LLVMContext.h"
▲ Show 20 Lines • Show All 59 Lines • ▼ Show 20 Linesint main(int argc, char **argv) {
mlir::JitRunnerConfig jitRunnerConfig; mlir::JitRunnerConfig jitRunnerConfig;
jitRunnerConfig.mlirTransformer = runMLIRPasses; jitRunnerConfig.mlirTransformer = runMLIRPasses;
jitRunnerConfig.llvmModuleBuilder = convertMLIRModule; jitRunnerConfig.llvmModuleBuilder = convertMLIRModule;
mlir::DialectRegistry registry; mlir::DialectRegistry registry;
registry.insert<mlir::arith::ArithmeticDialect, mlir::LLVM::LLVMDialect, registry.insert<mlir::arith::ArithmeticDialect, mlir::LLVM::LLVMDialect,
mlir::gpu::GPUDialect, mlir::spirv::SPIRVDialect, mlir::gpu::GPUDialect, mlir::spirv::SPIRVDialect,
mlir::StandardOpsDialect, mlir::memref::MemRefDialect>(); mlir::func::FuncDialect, mlir::memref::MemRefDialect>();
ftynseUnsubmitted
Done
ReplyInline Actions

This needs the ::func namespace.

ftynse: This needs the `::func` namespace.
mlir::registerLLVMDialectTranslation(registry); mlir::registerLLVMDialectTranslation(registry);
return mlir::JitRunnerMain(argc, argv, registry, jitRunnerConfig); return mlir::JitRunnerMain(argc, argv, registry, jitRunnerConfig);
} }

mlir/tools/mlir-vulkan-runner/CMakeLists.txt

Show First 20 Lines • Show All 52 Lines • ▼ Show 20 Linesif (MLIR_ENABLE_VULKAN_RUNNER)
) )
get_property(conversion_libs GLOBAL PROPERTY MLIR_CONVERSION_LIBS) get_property(conversion_libs GLOBAL PROPERTY MLIR_CONVERSION_LIBS)
set(LIBS set(LIBS
${conversion_libs} ${conversion_libs}
MLIRAnalysis MLIRAnalysis
MLIRArithmetic MLIRArithmetic
MLIRExecutionEngine MLIRExecutionEngine
MLIRFunc
MLIRGPUOps MLIRGPUOps
MLIRIR MLIRIR
MLIRJitRunner MLIRJitRunner
MLIRLLVMIR MLIRLLVMIR
MLIRLLVMCommonConversion MLIRLLVMCommonConversion
MLIRLLVMToLLVMIRTranslation MLIRLLVMToLLVMIRTranslation
MLIRMemRef MLIRMemRef
MLIRMemRefToLLVM MLIRMemRefToLLVM
MLIRParser MLIRParser
MLIRSPIRV MLIRSPIRV
MLIRSPIRVTransforms MLIRSPIRVTransforms
MLIRStandard
MLIRSupport MLIRSupport
MLIRTargetLLVMIRExport MLIRTargetLLVMIRExport
MLIRTransforms MLIRTransforms
MLIRTranslation MLIRTranslation
${Vulkan_LIBRARY} ${Vulkan_LIBRARY}
) )
# Manually expand the target library, since our MLIR libraries # Manually expand the target library, since our MLIR libraries
Show All 20 Lines

mlir/tools/mlir-vulkan-runner/mlir-vulkan-runner.cpp

Show All 14 Lines
#include "mlir/Conversion/GPUToSPIRV/GPUToSPIRVPass.h" #include "mlir/Conversion/GPUToSPIRV/GPUToSPIRVPass.h"
#include "mlir/Conversion/GPUToVulkan/ConvertGPUToVulkanPass.h" #include "mlir/Conversion/GPUToVulkan/ConvertGPUToVulkanPass.h"
#include "mlir/Conversion/LLVMCommon/LoweringOptions.h" #include "mlir/Conversion/LLVMCommon/LoweringOptions.h"
#include "mlir/Conversion/MemRefToLLVM/MemRefToLLVM.h" #include "mlir/Conversion/MemRefToLLVM/MemRefToLLVM.h"
#include "mlir/Conversion/ReconcileUnrealizedCasts/ReconcileUnrealizedCasts.h" #include "mlir/Conversion/ReconcileUnrealizedCasts/ReconcileUnrealizedCasts.h"
#include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVMPass.h" #include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVMPass.h"
#include "mlir/Conversion/StandardToSPIRV/StandardToSPIRVPass.h" #include "mlir/Conversion/StandardToSPIRV/StandardToSPIRVPass.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/Dialect/GPU/GPUDialect.h" #include "mlir/Dialect/GPU/GPUDialect.h"
#include "mlir/Dialect/GPU/Passes.h" #include "mlir/Dialect/GPU/Passes.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/MemRef/IR/MemRef.h" #include "mlir/Dialect/MemRef/IR/MemRef.h"
#include "mlir/Dialect/MemRef/Transforms/Passes.h" #include "mlir/Dialect/MemRef/Transforms/Passes.h"
#include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h" #include "mlir/Dialect/SPIRV/IR/SPIRVDialect.h"
#include "mlir/Dialect/SPIRV/IR/SPIRVOps.h" #include "mlir/Dialect/SPIRV/IR/SPIRVOps.h"
#include "mlir/Dialect/SPIRV/Transforms/Passes.h" #include "mlir/Dialect/SPIRV/Transforms/Passes.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/ExecutionEngine/JitRunner.h" #include "mlir/ExecutionEngine/JitRunner.h"
#include "mlir/ExecutionEngine/OptUtils.h" #include "mlir/ExecutionEngine/OptUtils.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Pass/PassManager.h" #include "mlir/Pass/PassManager.h"
#include "mlir/Target/LLVMIR/Dialect/LLVMIR/LLVMToLLVMIRTranslation.h" #include "mlir/Target/LLVMIR/Dialect/LLVMIR/LLVMToLLVMIRTranslation.h"
#include "mlir/Target/LLVMIR/Export.h" #include "mlir/Target/LLVMIR/Export.h"
#include "llvm/Support/InitLLVM.h" #include "llvm/Support/InitLLVM.h"
#include "llvm/Support/TargetSelect.h" #include "llvm/Support/TargetSelect.h"
Show All 30 Linesint main(int argc, char **argv) {
mlir::initializeLLVMPasses(); mlir::initializeLLVMPasses();
mlir::JitRunnerConfig jitRunnerConfig; mlir::JitRunnerConfig jitRunnerConfig;
jitRunnerConfig.mlirTransformer = runMLIRPasses; jitRunnerConfig.mlirTransformer = runMLIRPasses;
mlir::DialectRegistry registry; mlir::DialectRegistry registry;
registry.insert<mlir::arith::ArithmeticDialect, mlir::LLVM::LLVMDialect, registry.insert<mlir::arith::ArithmeticDialect, mlir::LLVM::LLVMDialect,
mlir::gpu::GPUDialect, mlir::spirv::SPIRVDialect, mlir::gpu::GPUDialect, mlir::spirv::SPIRVDialect,
mlir::StandardOpsDialect, mlir::memref::MemRefDialect>(); mlir::func::FuncDialect, mlir::memref::MemRefDialect>();
ftynseUnsubmitted
Done
ReplyInline Actions

This needs the ::func namespace.

ftynse: This needs the `::func` namespace.
mlir::registerLLVMDialectTranslation(registry); mlir::registerLLVMDialectTranslation(registry);
return mlir::JitRunnerMain(argc, argv, registry, jitRunnerConfig); return mlir::JitRunnerMain(argc, argv, registry, jitRunnerConfig);
} }

mlir/unittests/Conversion/PDLToPDLInterp/CMakeLists.txt

add_mlir_unittest(MLIRPDLToPDLInterpTests add_mlir_unittest(MLIRPDLToPDLInterpTests
RootOrderingTest.cpp RootOrderingTest.cpp
) )
target_link_libraries(MLIRPDLToPDLInterpTests target_link_libraries(MLIRPDLToPDLInterpTests
PRIVATE PRIVATE
MLIRStandard MLIRFunc
MLIRPDLToPDLInterp MLIRPDLToPDLInterp
) )

mlir/unittests/Interfaces/CMakeLists.txt

add_mlir_unittest(MLIRInterfacesTests add_mlir_unittest(MLIRInterfacesTests
ControlFlowInterfacesTest.cpp ControlFlowInterfacesTest.cpp
DataLayoutInterfacesTest.cpp DataLayoutInterfacesTest.cpp
InferTypeOpInterfaceTest.cpp InferTypeOpInterfaceTest.cpp
) )
target_link_libraries(MLIRInterfacesTests target_link_libraries(MLIRInterfacesTests
PRIVATE PRIVATE
MLIRControlFlowInterfaces MLIRControlFlowInterfaces
MLIRDataLayoutInterfaces MLIRDataLayoutInterfaces
MLIRDLTI MLIRDLTI
MLIRFunc
MLIRInferTypeOpInterface MLIRInferTypeOpInterface
MLIRParser MLIRParser
MLIRStandard
) )

mlir/unittests/Interfaces/InferTypeOpInterfaceTest.cpp

//===- InferTypeOpInterfaceTest.cpp - Unit Test for type interface --------===// //===- InferTypeOpInterfaceTest.cpp - Unit Test for type interface --------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Interfaces/InferTypeOpInterface.h" #include "mlir/Interfaces/InferTypeOpInterface.h"
#include "mlir/Dialect/Arithmetic/IR/Arithmetic.h" #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/Func/IR/FuncOps.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/BuiltinOps.h" #include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/Dialect.h" #include "mlir/IR/Dialect.h"
#include "mlir/IR/DialectImplementation.h" #include "mlir/IR/DialectImplementation.h"
#include "mlir/IR/ImplicitLocOpBuilder.h" #include "mlir/IR/ImplicitLocOpBuilder.h"
#include "mlir/IR/OpDefinition.h" #include "mlir/IR/OpDefinition.h"
#include "mlir/IR/OpImplementation.h" #include "mlir/IR/OpImplementation.h"
#include "mlir/Parser.h" #include "mlir/Parser.h"
#include <gtest/gtest.h> #include <gtest/gtest.h>
using namespace mlir; using namespace mlir;
class ValueShapeRangeTest : public testing::Test { class ValueShapeRangeTest : public testing::Test {
protected: protected:
void SetUp() override { void SetUp() override {
const char *ir = R"MLIR( const char *ir = R"MLIR(
func @map(%arg : tensor<1xi64>) { func @map(%arg : tensor<1xi64>) {
%0 = arith.constant dense<[10]> : tensor<1xi64> %0 = arith.constant dense<[10]> : tensor<1xi64>
%1 = arith.addi %arg, %0 : tensor<1xi64> %1 = arith.addi %arg, %0 : tensor<1xi64>
return return
} }
)MLIR"; )MLIR";
registry.insert<StandardOpsDialect, arith::ArithmeticDialect>(); registry.insert<func::FuncDialect, arith::ArithmeticDialect>();
ctx.appendDialectRegistry(registry); ctx.appendDialectRegistry(registry);
module = parseSourceString(ir, &ctx); module = parseSourceString(ir, &ctx);
mapFn = cast<FuncOp>(module->front()); mapFn = cast<FuncOp>(module->front());
} }
// Create ValueShapeRange on the arith.addi operation. // Create ValueShapeRange on the arith.addi operation.
ValueShapeRange addiRange() { ValueShapeRange addiRange() {
auto &fnBody = mapFn.body(); auto &fnBody = mapFn.body();
▲ Show 20 Lines • Show All 60 Lines • Show Last 20 Lines

mlir/utils/vim/syntax/mlir.vim

Show All 28 Lines
syn match mlirType /\<memref\ze\s*<.*>/ syn match mlirType /\<memref\ze\s*<.*>/
syn match mlirType /\<tensor\ze\s*<.*>/ syn match mlirType /\<tensor\ze\s*<.*>/
syn match mlirType /\<vector\ze\s*<.*>/ syn match mlirType /\<vector\ze\s*<.*>/
" vector types inside memref or tensor. " vector types inside memref or tensor.
syn match mlirType /x\s*\zsvector/ syn match mlirType /x\s*\zsvector/
" Operations. " Operations.
" Standard dialect ops.
" TODO: this list is not exhaustive. " TODO: this list is not exhaustive.
syn keyword mlirOps alloc alloca addf addi and call call_indirect cmpf cmpi syn keyword mlirOps alloc alloca addf addi and call call_indirect cmpf cmpi
syn keyword mlirOps constant dealloc divf dma_start dma_wait dim exp syn keyword mlirOps constant dealloc divf dma_start dma_wait dim exp
syn keyword mlirOps getTensor index_cast load log memref_cast syn keyword mlirOps getTensor index_cast load log memref_cast
syn keyword mlirOps memref_shape_cast mulf muli negf powf prefetch rsqrt sitofp syn keyword mlirOps memref_shape_cast mulf muli negf powf prefetch rsqrt sitofp
syn keyword mlirOps splat store select sqrt subf subi subview tanh syn keyword mlirOps splat store select sqrt subf subi subview tanh
syn keyword mlirOps view syn keyword mlirOps view
▲ Show 20 Lines • Show All 85 Lines • Show Last 20 Lines
mlir/lib/Dialect/SparseTensor/Transforms/Sparsification.cpp