This is an archive of the discontinued LLVM Phabricator instance.

Differential D138984

mlir: replace llvm::None with std::nullopt (NFC)
AbandonedPublic

Authored by artagnon on Nov 29 2022, 11:57 PM.

Details

Reviewers
kazu
antiagainst
rriddle
aartbik
jpienaar
ftynse
mravishankar
clementval
bondhugula
ThomasRaoux
nicolasvasilache
herhut
dcaballe
hanchung
Summary

This is part of an effort to migrate from llvm::Optional to
std::optional.

See also: https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

Signed-off-by: Ramkumar Ramachandra <r@artagnon.com>

Diff Detail

Event Timeline

artagnon created this revision.Nov 29 2022, 11:57 PM
Herald added a reviewer: antiagainst. · View Herald TranscriptNov 29 2022, 11:57 PM
Herald added a reviewer: rriddle. · View Herald Transcript
Herald added a reviewer: antiagainst. · View Herald Transcript
Herald added a reviewer: aartbik. · View Herald Transcript
Herald added a reviewer: jpienaar. · View Herald Transcript
Herald added a reviewer: ftynse. · View Herald Transcript
Herald added a reviewer: aartbik. · View Herald Transcript
Herald added a reviewer: mravishankar. · View Herald Transcript
Herald added a reviewer: clementval. · View Herald Transcript
Herald added a reviewer: bondhugula. · View Herald Transcript
Herald added a reviewer: ThomasRaoux. · View Herald Transcript
Herald added a project: Restricted Project. · View Herald Transcript
Herald added subscribers: jsetoain, hanchung, Moerafaat and 41 others. · View Herald Transcript
artagnon requested review of this revision.Nov 29 2022, 11:57 PM
Herald added a reviewer: nicolasvasilache. · View Herald TranscriptNov 29 2022, 11:57 PM
Herald added a reviewer: herhut. · View Herald Transcript
Herald added a reviewer: dcaballe. · View Herald Transcript
Herald added a project: Restricted Project. · View Herald Transcript
Herald added subscribers: • pcwang-thead, stephenneuendorffer, nicolasvasilache, jholewinski. · View Herald Transcript
artagnon added a child revision: D138934: mlir/tblgen: use std::optional in generation.Nov 30 2022, 12:09 AM
Harbormaster completed remote builds in B200188: Diff 478824.Nov 30 2022, 12:09 AM
artagnon mentioned this in D138934: mlir/tblgen: use std::optional in generation.Nov 30 2022, 12:09 AM
artagnon updated this revision to Diff 478847.Nov 30 2022, 1:25 AM

Use using llvm::None to fix compile-time errors.

Harbormaster completed remote builds in B200202: Diff 478847.Nov 30 2022, 1:55 AM
kazu added a comment.Nov 30 2022, 9:45 AM
In D138984#3959557, @artagnon wrote:

Use using llvm::None to fix compile-time errors.

Are you trying to replace llvm::None first in this patch and then address None (without llvm::) later?

mlir/include/mlir/Support/LLVM.h
139 ↗(On Diff #478847)

Do we need this? We try our best to use std:: explicitly:

https://llvm.org/docs/CodingStandards.html#do-not-use-using-namespace-std

artagnon updated this revision to Diff 479000.Nov 30 2022, 9:51 AM

Address review comment by @kazu, git clang-format.

artagnon marked an inline comment as done.Nov 30 2022, 9:53 AM

Unfortunately, there is no easy solution to replacing None without the llvm:: prefix. I edited it by hand in a few places.

mlir/include/mlir/Support/LLVM.h
139 ↗(On Diff #478847)

Thanks for catching this! It seems to be a stray edit.

artagnon marked an inline comment as done.Nov 30 2022, 10:21 AM
In D138984#3960969, @kazu wrote:
In D138984#3959557, @artagnon wrote:

Use using llvm::None to fix compile-time errors.

Are you trying to replace llvm::None first in this patch and then address None (without llvm::) later?

Yes, I know what happened: I did a replacement of llvm::None with std::nullopt in mlir/, and replaced the instance in using llvm::None by mistake. Then, I was puzzled about the compile failure: that was the reason for the stray edit. Anyway, the diff should be good now.

kazu added a comment.Nov 30 2022, 10:27 AM
In D138984#3960984, @artagnon wrote:

Unfortunately, there is no easy solution to replacing None without the llvm:: prefix. I edited it by hand in a few places.

Have you considered putting [[deprecated]] in front of inline constexpr std::nullopt_t None = std::nullopt; in llvm/include/llvm/ADT/None.h? Doing so will cause the compiler to warn you every time you use None or llvm::None. You can then extract warnings from your build log and use sed and such to automatically replace llvm::None and None with std::nullopt. If I were you, I would do this mechanical replacements in non-generated code first because if you touch tablegen or .td files, you may have to update corresponding tests that check the automatic generation. You might even choose to update comments in a separate patch. The idea is to break up the big migration into series of super safe steps that are easy to review.

Harbormaster completed remote builds in B200310: Diff 479000.Nov 30 2022, 10:29 AM
artagnon updated this revision to Diff 479018.Nov 30 2022, 10:40 AM

Address review comment by @kazu. Now, the patch is generated by the
following steps:

$ find . -exec gsed -i s/llvm::Optional/std::nullopt/ {} \;
$ git checkout -- include/MLIR/Support/LLVM.h # stray edit
$ git commit
$ git clang-format @~
$ git commit --amend

artagnon added a comment.Nov 30 2022, 10:45 AM

Thanks for the suggestion @kazu! I'll try to tackle that in a follow-up patch. For now, I've updated the diff to a programmatically-generated and safe-subset replacement of llvm::Optional to std::nullopt. We should have additional assurance from the CI. I understand your concerns about wanting super-safe steps, but D138934 involved a lot of manual work, and I'm not sure how we can fix that: will it be sufficient to trust the CI in that case?

kazu added a comment.Nov 30 2022, 11:03 AM
In D138984#3961180, @artagnon wrote:

Thanks for the suggestion @kazu! I'll try to tackle that in a follow-up patch. For now, I've updated the diff to a programmatically-generated and safe-subset replacement of llvm::Optional to std::nullopt. We should have additional assurance from the CI. I understand your concerns about wanting super-safe steps, but D138934 involved a lot of manual work, and I'm not sure how we can fix that: will it be sufficient to trust the CI in that case?

I am not sure how much we can rely on CI. For example, we may not be able to catch performance regressions of generated code. llvm/include/llvm/Support/Casting.h knows about Optional but not std::optional yet, so dyn_cast of an std::optional object quietly returns nullptr, which could in turn cause a missed optimization. There may be other pitfalls like this, so I am not sure if there is a better way than go through every single instance one by one.

mlir/include/mlir/Dialect/Affine/Analysis/AffineAnalysis.h
146

Did you mean to replace this one also?

mlir/unittests/Transforms/DialectConversion.cpp
18

Did you mean to replace these also?

Herald added a reviewer: hanchung. · View Herald TranscriptNov 30 2022, 11:03 AM
Harbormaster completed remote builds in B200324: Diff 479018.Nov 30 2022, 11:03 AM
artagnon updated this revision to Diff 479027.Nov 30 2022, 11:10 AM

Missed /g in gsed's replace command

artagnon added a comment.Nov 30 2022, 11:11 AM
In D138984#3961243, @kazu wrote:
In D138984#3961180, @artagnon wrote:

Thanks for the suggestion @kazu! I'll try to tackle that in a follow-up patch. For now, I've updated the diff to a programmatically-generated and safe-subset replacement of llvm::Optional to std::nullopt. We should have additional assurance from the CI. I understand your concerns about wanting super-safe steps, but D138934 involved a lot of manual work, and I'm not sure how we can fix that: will it be sufficient to trust the CI in that case?

I am not sure how much we can rely on CI. For example, we may not be able to catch performance regressions of generated code. llvm/include/llvm/Support/Casting.h knows about Optional but not std::optional yet, so dyn_cast of an std::optional object quietly returns nullptr, which could in turn cause a missed optimization. There may be other pitfalls like this, so I am not sure if there is a better way than go through every single instance one by one.

Interesting. Okay, let's try to go through the diff by hand, and find any issues: thanks a lot for volunteering to review this; it's definitely not easy.

Harbormaster completed remote builds in B200329: Diff 479027.Nov 30 2022, 11:39 AM
artagnon abandoned this revision.Dec 13 2022, 4:27 AM

Subsumed by 1a36588ec64 by @kazu.

artagnon removed a child revision: D138934: mlir/tblgen: use std::optional in generation.Dec 13 2022, 6:39 AM

Revision Contents

PathSize
mlir/
docs/
2 lines
8 lines
10 lines
examples/
toy/
Ch1/
include/
toy/
2 lines
Ch2/
include/
toy/
2 lines
2 lines
mlir/
2 lines
Ch3/
include/
toy/
2 lines
2 lines
mlir/
2 lines
Ch4/
include/
toy/
2 lines
2 lines
mlir/
2 lines
Ch5/
include/
toy/
2 lines
2 lines
mlir/
2 lines
Ch6/
include/
toy/
2 lines
2 lines
mlir/
2 lines
Ch7/
include/
toy/
2 lines
2 lines
mlir/
12 lines
include/
mlir/
AsmParser/
2 lines
CAPI/
2 lines
Conversion/
MathToLibm/
2 lines
Dialect/
Affine/
Analysis/
2 lines
4 lines
IR/
6 lines
Async/
IR/
2 lines
Bufferization/
IR/
8 lines
2 lines
Func/
IR/
2 lines
Transforms/
2 lines
LLVMIR/
2 lines
Linalg/
Analysis/
20 lines
Transforms/
2 lines
Utils/
4 lines
MLProgram/
IR/
4 lines
PDL/
IR/
12 lines
SCF/
IR/
8 lines
SPIRV/
IR/
4 lines
6 lines
50 lines
4 lines
Shape/
IR/
2 lines
SparseTensor/
Utils/
4 lines
Tensor/
IR/
2 lines
Tosa/
Utils/
2 lines
Transform/
IR/
4 lines
Utils/
6 lines
Vector/
IR/
2 lines
12 lines
Interfaces/
2 lines
Transforms/
3 lines
ExecutionEngine/
4 lines
IR/
2 lines
8 lines
2 lines
10 lines
4 lines
2 lines
2 lines
4 lines
4 lines
2 lines
4 lines
2 lines
2 lines
4 lines
10 lines
2 lines
4 lines
15 lines
2 lines
2 lines
2 lines
2 lines
6 lines
Interfaces/
6 lines
4 lines
16 lines
2 lines
Pass/
4 lines
2 lines
2 lines
Rewrite/
4 lines
Support/
4 lines
2 lines
TableGen/
4 lines
8 lines
Tools/
PDLL/
AST/
2 lines
8 lines
4 lines
mlir-translate/
4 lines
Transforms/
18 lines
8 lines
4 lines
lib/
Analysis/
AliasAnalysis/
4 lines
DataFlow/
3 lines
Presburger/
14 lines
2 lines
2 lines
2 lines
AsmParser/
12 lines
2 lines
28 lines
6 lines
14 lines
2 lines
Bindings/
Python/
2 lines
10 lines
2 lines
Conversion/
AsyncToLLVM/
2 lines
ControlFlowToLLVM/
2 lines
GPUToNVVM/
2 lines
GPUToSPIRV/
4 lines
LLVMCommon/
22 lines
LinalgToSPIRV/
16 lines
MemRefToLLVM/
13 lines
MemRefToSPIRV/
14 lines
NVGPUToNVVM/
6 lines
PDLToPDLInterp/
8 lines
4 lines
4 lines
SCFToGPU/
6 lines
SCFToSPIRV/
2 lines
SPIRVToLLVM/
10 lines
VectorToGPU/
8 lines
Dialect/
Affine/
Analysis/
2 lines
28 lines
IR/
14 lines
Transforms/
6 lines
2 lines
Arith/
IR/
2 lines
Async/
IR/
2 lines
Bufferization/
Transforms/
3 lines
2 lines
3 lines
2 lines
Func/
IR/
2 lines
GPU/
TransformOps/
32 lines
Transforms/
4 lines
4 lines
LLVMIR/
IR/
4 lines
4 lines
6 lines
Linalg/
TransformOps/
2 lines
Transforms/
4 lines
2 lines
14 lines
2 lines
4 lines
Utils/
12 lines
MemRef/
IR/
2 lines
6 lines
OpenACC/
IR/
6 lines
OpenMP/
IR/
4 lines
SCF/
IR/
11 lines
Transforms/
8 lines
SPIRV/
IR/
2 lines
4 lines
14 lines
10 lines
4 lines
Transforms/
2 lines
16 lines
13 lines
Shape/
IR/
6 lines
SparseTensor/
IR/
2 lines
Transforms/
6 lines
2 lines
4 lines
5 lines
Tensor/
IR/
2 lines
4 lines
Utils/
18 lines
4 lines
Vector/
IR/
2 lines
Transforms/
4 lines
14 lines
IR/
6 lines
6 lines
4 lines
14 lines
4 lines
2 lines
9 lines
Interfaces/
20 lines
Pass/
2 lines
2 lines
Support/
2 lines
2 lines
TableGen/
4 lines
2 lines
6 lines
8 lines
2 lines
4 lines
Target/
LLVMIR/
2 lines
Dialect/
LLVMIR/
2 lines
2 lines
SPIRV/
Deserialization/
2 lines
8 lines
Tools/
PDLL/
CodeGen/
4 lines
Parser/
13 lines
lsp-server-support/
10 lines
mlir-lsp-server/
10 lines
mlir-pdll-lsp-server/
16 lines
mlir-translate/
4 lines
tblgen-lsp-server/
10 lines
Transforms/
Utils/
12 lines
2 lines
test/
lib/
Dialect/
Func/
2 lines
Test/
2 lines
2 lines
6 lines
Vector/
8 lines
mlir-tblgen/
2 lines
4 lines
2 lines
tools/
mlir-linalg-ods-gen/
14 lines
mlir-tblgen/
2 lines
6 lines
14 lines
4 lines
12 lines
4 lines
unittests/
IR/
23 lines
Pass/
4 lines
2 lines
TableGen/
10 lines
Transforms/
14 lines

Diff 479000

mlir/docs/AttributesAndTypes.md

Show First 20 Lines • Show All 647 Lines • ▼ Show 20 Lines
`defaultValue`, a string of the C++ default value. If a value for the parameter `defaultValue`, a string of the C++ default value. If a value for the parameter
was not encountered during parsing, it is set to this default value. If a was not encountered during parsing, it is set to this default value. If a
parameter is equal to its default value, it is not printed. The `comparator` parameter is equal to its default value, it is not printed. The `comparator`
field of the parameter is used, but if one is not specified, the equality field of the parameter is used, but if one is not specified, the equality
operator is used. operator is used.
When using `OptionalParameter`, the default value is set to the C++ When using `OptionalParameter`, the default value is set to the C++
default-constructed value for the C++ storage type. For example, `Optional<int>` default-constructed value for the C++ storage type. For example, `Optional<int>`
will be set to `llvm::None` and `Attribute` will be set to `nullptr`. The will be set to `std::nullopt` and `Attribute` will be set to `nullptr`. The
presence of these parameters is tested by comparing them to their "null" values. presence of these parameters is tested by comparing them to their "null" values.
An optional group is a set of elements optionally printed based on the presence An optional group is a set of elements optionally printed based on the presence
of an anchor. Only optional parameters or directives that only capture optional of an anchor. Only optional parameters or directives that only capture optional
parameters can be used in optional groups. The group in which the anchor is parameters can be used in optional groups. The group in which the anchor is
placed is printed if it is present, otherwise the other one is printed. If a placed is printed if it is present, otherwise the other one is printed. If a
directive that captures more than one optional parameter is used as the anchor, directive that captures more than one optional parameter is used as the anchor,
the optional group is printed if any of the captured parameters is present. For the optional group is printed if any of the captured parameters is present. For
▲ Show 20 Lines • Show All 503 Lines • Show Last 20 Lines

mlir/docs/DialectConversion.md

Show First 20 Lines • Show All 295 Lines • ▼ Show 20 Lines
```c++ ```c++
class TypeConverter { class TypeConverter {
public: public:
/// Register a conversion function. A conversion function defines how a given /// Register a conversion function. A conversion function defines how a given
/// source type should be converted. A conversion function must be convertible /// source type should be converted. A conversion function must be convertible
/// to any of the following forms(where `T` is a class derived from `Type`: /// to any of the following forms(where `T` is a class derived from `Type`:
/// * Optional<Type>(T) /// * Optional<Type>(T)
/// - This form represents a 1-1 type conversion. It should return nullptr /// - This form represents a 1-1 type conversion. It should return nullptr
/// or `llvm::None` to signify failure. If `llvm::None` is returned, the /// or `std::nullopt` to signify failure. If `llvm::None` is returned, the
/// converter is allowed to try another conversion function to perform /// converter is allowed to try another conversion function to perform
/// the conversion. /// the conversion.
/// * Optional<LogicalResult>(T, SmallVectorImpl<Type> &) /// * Optional<LogicalResult>(T, SmallVectorImpl<Type> &)
/// - This form represents a 1-N type conversion. It should return /// - This form represents a 1-N type conversion. It should return
/// `failure` or `llvm::None` to signify a failed conversion. If the new /// `failure` or `std::nullopt` to signify a failed conversion. If the new
/// set of types is empty, the type is removed and any usages of the /// set of types is empty, the type is removed and any usages of the
/// existing value are expected to be removed during conversion. If /// existing value are expected to be removed during conversion. If
/// `llvm::None` is returned, the converter is allowed to try another /// `std::nullopt` is returned, the converter is allowed to try another
/// conversion function to perform the conversion. /// conversion function to perform the conversion.
/// * Optional<LogicalResult>(T, SmallVectorImpl<Type> &, ArrayRef<Type>) /// * Optional<LogicalResult>(T, SmallVectorImpl<Type> &, ArrayRef<Type>)
/// - This form represents a 1-N type conversion supporting recursive /// - This form represents a 1-N type conversion supporting recursive
/// types. The first two arguments and the return value are the same as /// types. The first two arguments and the return value are the same as
/// for the regular 1-N form. The third argument is contains is the /// for the regular 1-N form. The third argument is contains is the
/// "call stack" of the recursive conversion: it contains the list of /// "call stack" of the recursive conversion: it contains the list of
/// types currently being converted, with the current type being the /// types currently being converted, with the current type being the
/// last one. If it is present more than once in the list, the /// last one. If it is present more than once in the list, the
/// conversion concerns a recursive type. /// conversion concerns a recursive type.
/// Note: When attempting to convert a type, e.g. via 'convertType', the /// Note: When attempting to convert a type, e.g. via 'convertType', the
/// mostly recently added conversions will be invoked first. /// mostly recently added conversions will be invoked first.
template <typename FnT, template <typename FnT,
typename T = typename llvm::function_traits<FnT>::template arg_t<0>> typename T = typename llvm::function_traits<FnT>::template arg_t<0>>
void addConversion(FnT &&callback) { void addConversion(FnT &&callback) {
registerConversion(wrapCallback<T>(std::forward<FnT>(callback))); registerConversion(wrapCallback<T>(std::forward<FnT>(callback)));
} }
/// Register a materialization function, which must be convertible to the /// Register a materialization function, which must be convertible to the
/// following form: /// following form:
/// `Optional<Value> (OpBuilder &, T, ValueRange, Location)`, /// `Optional<Value> (OpBuilder &, T, ValueRange, Location)`,
/// where `T` is any subclass of `Type`. /// where `T` is any subclass of `Type`.
/// This function is responsible for creating an operation, using the /// This function is responsible for creating an operation, using the
/// OpBuilder and Location provided, that "converts" a range of values into a /// OpBuilder and Location provided, that "converts" a range of values into a
/// single value of the given type `T`. It must return a Value of the /// single value of the given type `T`. It must return a Value of the
/// converted type on success, an `llvm::None` if it failed but other /// converted type on success, an `std::nullopt` if it failed but other
/// materialization can be attempted, and `nullptr` on unrecoverable failure. /// materialization can be attempted, and `nullptr` on unrecoverable failure.
/// It will only be called for (sub)types of `T`. /// It will only be called for (sub)types of `T`.
/// ///
/// This method registers a materialization that will be called when /// This method registers a materialization that will be called when
/// converting an illegal block argument type, to a legal type. /// converting an illegal block argument type, to a legal type.
template <typename FnT, template <typename FnT,
typename T = typename llvm::function_traits<FnT>::template arg_t<1>> typename T = typename llvm::function_traits<FnT>::template arg_t<1>>
void addArgumentMaterialization(FnT &&callback) { void addArgumentMaterialization(FnT &&callback) {
▲ Show 20 Lines • Show All 107 Lines • Show Last 20 Lines

mlir/docs/OpDefinitions.md

Show First 20 Lines • Show All 1,385 Lines • ▼ Show 20 Linesllvm::StringRef ConvertToString(MyIntEnum val) {
} }
return ""; return "";
} }
llvm::Optional<MyIntEnum> ConvertToEnum(llvm::StringRef str) { llvm::Optional<MyIntEnum> ConvertToEnum(llvm::StringRef str) {
return llvm::StringSwitch<llvm::Optional<MyIntEnum>>(str) return llvm::StringSwitch<llvm::Optional<MyIntEnum>>(str)
.Case("Case15", MyIntEnum::Case15) .Case("Case15", MyIntEnum::Case15)
.Case("Case20", MyIntEnum::Case20) .Case("Case20", MyIntEnum::Case20)
.Default(llvm::None); .Default(std::nullopt);
} }
llvm::Optional<MyIntEnum> symbolizeMyIntEnum(uint32_t value) { llvm::Optional<MyIntEnum> symbolizeMyIntEnum(uint32_t value) {
switch (value) { switch (value) {
case 15: return MyIntEnum::Case15; case 15: return MyIntEnum::Case15;
case 20: return MyIntEnum::Case20; case 20: return MyIntEnum::Case20;
default: return llvm::None; default: return std::nullopt;
} }
} }
} // namespace Inner } // namespace Inner
} // namespace Outer } // namespace Outer
``` ```
Similarly for the following `BitEnumAttr` definition: Similarly for the following `BitEnumAttr` definition:
▲ Show 20 Lines • Show All 106 Lines • ▼ Show 20 Linesllvm::Optional<MyBitEnum> symbolizeMyBitEnum(llvm::StringRef str) {
uint32_t val = 0; uint32_t val = 0;
for (auto symbol : symbols) { for (auto symbol : symbols) {
auto bit = llvm::StringSwitch<llvm::Optional<uint32_t>>(symbol) auto bit = llvm::StringSwitch<llvm::Optional<uint32_t>>(symbol)
.Case("tagged", 1) .Case("tagged", 1)
.Case("Bit1", 2) .Case("Bit1", 2)
.Case("Bit2", 4) .Case("Bit2", 4)
.Case("Bit3", 8) .Case("Bit3", 8)
.Default(llvm::None); .Default(std::nullopt);
if (bit) { val |= *bit; } else { return llvm::None; } if (bit) { val |= *bit; } else { return std::nullopt; }
} }
return static_cast<MyBitEnum>(val); return static_cast<MyBitEnum>(val);
} }
llvm::Optional<MyBitEnum> symbolizeMyBitEnum(uint32_t value) { llvm::Optional<MyBitEnum> symbolizeMyBitEnum(uint32_t value) {
// Special case for all bits unset. // Special case for all bits unset.
if (value == 0) return MyBitEnum::None; if (value == 0) return MyBitEnum::None;
if (value & ~static_cast<uint32_t>(15u)) return llvm::None; if (value & ~static_cast<uint32_t>(15u)) return std::nullopt;
return static_cast<MyBitEnum>(value); return static_cast<MyBitEnum>(value);
} }
``` ```
## Debugging Tips ## Debugging Tips
### Run `mlir-tblgen` to see the generated content ### Run `mlir-tblgen` to see the generated content
▲ Show 20 Lines • Show All 112 Lines • Show Last 20 Lines

mlir/examples/toy/Ch1/include/toy/AST.h

Show First 20 Lines • Show All 131 Lines • ▼ Show 20 Lines
public: public:
ReturnExprAST(Location loc, llvm::Optional<std::unique_ptr<ExprAST>> expr) ReturnExprAST(Location loc, llvm::Optional<std::unique_ptr<ExprAST>> expr)
: ExprAST(Expr_Return, std::move(loc)), expr(std::move(expr)) {} : ExprAST(Expr_Return, std::move(loc)), expr(std::move(expr)) {}
llvm::Optional<ExprAST *> getExpr() { llvm::Optional<ExprAST *> getExpr() {
if (expr.has_value()) if (expr.has_value())
return expr->get(); return expr->get();
return llvm::None; return std::nullopt;
} }
/// LLVM style RTTI /// LLVM style RTTI
static bool classof(const ExprAST *c) { return c->getKind() == Expr_Return; } static bool classof(const ExprAST *c) { return c->getKind() == Expr_Return; }
}; };
/// Expression class for a binary operator. /// Expression class for a binary operator.
class BinaryExprAST : public ExprAST { class BinaryExprAST : public ExprAST {
▲ Show 20 Lines • Show All 97 Lines • Show Last 20 Lines

mlir/examples/toy/Ch2/include/toy/AST.h

Show First 20 Lines • Show All 131 Lines • ▼ Show 20 Lines
public: public:
ReturnExprAST(Location loc, llvm::Optional<std::unique_ptr<ExprAST>> expr) ReturnExprAST(Location loc, llvm::Optional<std::unique_ptr<ExprAST>> expr)
: ExprAST(Expr_Return, std::move(loc)), expr(std::move(expr)) {} : ExprAST(Expr_Return, std::move(loc)), expr(std::move(expr)) {}
llvm::Optional<ExprAST *> getExpr() { llvm::Optional<ExprAST *> getExpr() {
if (expr.has_value()) if (expr.has_value())
return expr->get(); return expr->get();
return llvm::None; return std::nullopt;
} }
/// LLVM style RTTI /// LLVM style RTTI
static bool classof(const ExprAST *c) { return c->getKind() == Expr_Return; } static bool classof(const ExprAST *c) { return c->getKind() == Expr_Return; }
}; };
/// Expression class for a binary operator. /// Expression class for a binary operator.
class BinaryExprAST : public ExprAST { class BinaryExprAST : public ExprAST {
▲ Show 20 Lines • Show All 97 Lines • Show Last 20 Lines

mlir/examples/toy/Ch2/include/toy/Ops.td

Show First 20 Lines • Show All 285 Lines • ▼ Show 20 Linesdef ReturnOp : Toy_Op<"return", [Pure, HasParent<"FuncOp">,
// value must match the return type of the enclosing function. // value must match the return type of the enclosing function.
let arguments = (ins Variadic<F64Tensor>:$input); let arguments = (ins Variadic<F64Tensor>:$input);
// The return operation only emits the input in the format if it is present. // The return operation only emits the input in the format if it is present.
let assemblyFormat = "($input^ `:` type($input))? attr-dict "; let assemblyFormat = "($input^ `:` type($input))? attr-dict ";
// Allow building a ReturnOp with no return operand. // Allow building a ReturnOp with no return operand.
let builders = [ let builders = [
OpBuilder<(ins), [{ build($_builder, $_state, llvm::None); }]> OpBuilder<(ins), [{ build($_builder, $_state, std::nullopt); }]>
]; ];
// Provide extra utility definitions on the c++ operation class definition. // Provide extra utility definitions on the c++ operation class definition.
let extraClassDeclaration = [{ let extraClassDeclaration = [{
bool hasOperand() { return getNumOperands() != 0; } bool hasOperand() { return getNumOperands() != 0; }
}]; }];
// Invoke a static verify method to verify this return operation. // Invoke a static verify method to verify this return operation.
Show All 27 Lines

mlir/examples/toy/Ch2/mlir/MLIRGen.cpp

Show First 20 Lines • Show All 105 Lines • ▼ Show 20 Linesprivate:
/// provided Toy AST prototype. /// provided Toy AST prototype.
mlir::toy::FuncOp mlirGen(PrototypeAST &proto) { mlir::toy::FuncOp mlirGen(PrototypeAST &proto) {
auto location = loc(proto.loc()); auto location = loc(proto.loc());
// This is a generic function, the return type will be inferred later. // This is a generic function, the return type will be inferred later.
// Arguments type are uniformly unranked tensors. // Arguments type are uniformly unranked tensors.
llvm::SmallVector<mlir::Type, 4> argTypes(proto.getArgs().size(), llvm::SmallVector<mlir::Type, 4> argTypes(proto.getArgs().size(),
getType(VarType{})); getType(VarType{}));
auto funcType = builder.getFunctionType(argTypes, llvm::None); auto funcType = builder.getFunctionType(argTypes, std::nullopt);
return builder.create<mlir::toy::FuncOp>(location, proto.getName(), return builder.create<mlir::toy::FuncOp>(location, proto.getName(),
funcType); funcType);
} }
/// Emit a new function and add it to the MLIR module. /// Emit a new function and add it to the MLIR module.
mlir::toy::FuncOp mlirGen(FunctionAST &funcAST) { mlir::toy::FuncOp mlirGen(FunctionAST &funcAST) {
// Create a scope in the symbol table to hold variable declarations. // Create a scope in the symbol table to hold variable declarations.
ScopedHashTableScope<llvm::StringRef, mlir::Value> varScope(symbolTable); ScopedHashTableScope<llvm::StringRef, mlir::Value> varScope(symbolTable);
▲ Show 20 Lines • Show All 325 Lines • Show Last 20 Lines

mlir/examples/toy/Ch3/include/toy/AST.h

Show First 20 Lines • Show All 131 Lines • ▼ Show 20 Lines
public: public:
ReturnExprAST(Location loc, llvm::Optional<std::unique_ptr<ExprAST>> expr) ReturnExprAST(Location loc, llvm::Optional<std::unique_ptr<ExprAST>> expr)
: ExprAST(Expr_Return, std::move(loc)), expr(std::move(expr)) {} : ExprAST(Expr_Return, std::move(loc)), expr(std::move(expr)) {}
llvm::Optional<ExprAST *> getExpr() { llvm::Optional<ExprAST *> getExpr() {
if (expr.has_value()) if (expr.has_value())
return expr->get(); return expr->get();
return llvm::None; return std::nullopt;
} }
/// LLVM style RTTI /// LLVM style RTTI
static bool classof(const ExprAST *c) { return c->getKind() == Expr_Return; } static bool classof(const ExprAST *c) { return c->getKind() == Expr_Return; }
}; };
/// Expression class for a binary operator. /// Expression class for a binary operator.
class BinaryExprAST : public ExprAST { class BinaryExprAST : public ExprAST {
▲ Show 20 Lines • Show All 97 Lines • Show Last 20 Lines

mlir/examples/toy/Ch3/include/toy/Ops.td

Show First 20 Lines • Show All 287 Lines • ▼ Show 20 Linesdef ReturnOp : Toy_Op<"return", [Pure, HasParent<"FuncOp">,
// value must match the return type of the enclosing function. // value must match the return type of the enclosing function.
let arguments = (ins Variadic<F64Tensor>:$input); let arguments = (ins Variadic<F64Tensor>:$input);
// The return operation only emits the input in the format if it is present. // The return operation only emits the input in the format if it is present.
let assemblyFormat = "($input^ `:` type($input))? attr-dict "; let assemblyFormat = "($input^ `:` type($input))? attr-dict ";
// Allow building a ReturnOp with no return operand. // Allow building a ReturnOp with no return operand.
let builders = [ let builders = [
OpBuilder<(ins), [{ build($_builder, $_state, llvm::None); }]> OpBuilder<(ins), [{ build($_builder, $_state, std::nullopt); }]>
]; ];
// Provide extra utility definitions on the c++ operation class definition. // Provide extra utility definitions on the c++ operation class definition.
let extraClassDeclaration = [{ let extraClassDeclaration = [{
bool hasOperand() { return getNumOperands() != 0; } bool hasOperand() { return getNumOperands() != 0; }
}]; }];
// Indicate that additional verification for this operation is necessary. // Indicate that additional verification for this operation is necessary.
Show All 30 Lines

mlir/examples/toy/Ch3/mlir/MLIRGen.cpp

Show First 20 Lines • Show All 105 Lines • ▼ Show 20 Linesprivate:
/// provided Toy AST prototype. /// provided Toy AST prototype.
mlir::toy::FuncOp mlirGen(PrototypeAST &proto) { mlir::toy::FuncOp mlirGen(PrototypeAST &proto) {
auto location = loc(proto.loc()); auto location = loc(proto.loc());
// This is a generic function, the return type will be inferred later. // This is a generic function, the return type will be inferred later.
// Arguments type are uniformly unranked tensors. // Arguments type are uniformly unranked tensors.
llvm::SmallVector<mlir::Type, 4> argTypes(proto.getArgs().size(), llvm::SmallVector<mlir::Type, 4> argTypes(proto.getArgs().size(),
getType(VarType{})); getType(VarType{}));
auto funcType = builder.getFunctionType(argTypes, llvm::None); auto funcType = builder.getFunctionType(argTypes, std::nullopt);
return builder.create<mlir::toy::FuncOp>(location, proto.getName(), return builder.create<mlir::toy::FuncOp>(location, proto.getName(),
funcType); funcType);
} }
/// Emit a new function and add it to the MLIR module. /// Emit a new function and add it to the MLIR module.
mlir::toy::FuncOp mlirGen(FunctionAST &funcAST) { mlir::toy::FuncOp mlirGen(FunctionAST &funcAST) {
// Create a scope in the symbol table to hold variable declarations. // Create a scope in the symbol table to hold variable declarations.
ScopedHashTableScope<llvm::StringRef, mlir::Value> varScope(symbolTable); ScopedHashTableScope<llvm::StringRef, mlir::Value> varScope(symbolTable);
▲ Show 20 Lines • Show All 325 Lines • Show Last 20 Lines

mlir/examples/toy/Ch4/include/toy/AST.h

Show First 20 Lines • Show All 131 Lines • ▼ Show 20 Lines
public: public:
ReturnExprAST(Location loc, llvm::Optional<std::unique_ptr<ExprAST>> expr) ReturnExprAST(Location loc, llvm::Optional<std::unique_ptr<ExprAST>> expr)
: ExprAST(Expr_Return, std::move(loc)), expr(std::move(expr)) {} : ExprAST(Expr_Return, std::move(loc)), expr(std::move(expr)) {}
llvm::Optional<ExprAST *> getExpr() { llvm::Optional<ExprAST *> getExpr() {
if (expr.has_value()) if (expr.has_value())
return expr->get(); return expr->get();
return llvm::None; return std::nullopt;
} }
/// LLVM style RTTI /// LLVM style RTTI
static bool classof(const ExprAST *c) { return c->getKind() == Expr_Return; } static bool classof(const ExprAST *c) { return c->getKind() == Expr_Return; }
}; };
/// Expression class for a binary operator. /// Expression class for a binary operator.
class BinaryExprAST : public ExprAST { class BinaryExprAST : public ExprAST {
▲ Show 20 Lines • Show All 97 Lines • Show Last 20 Lines

mlir/examples/toy/Ch4/include/toy/Ops.td

Show First 20 Lines • Show All 319 Lines • ▼ Show 20 Linesdef ReturnOp : Toy_Op<"return", [Pure, HasParent<"FuncOp">,
// value must match the return type of the enclosing function. // value must match the return type of the enclosing function.
let arguments = (ins Variadic<F64Tensor>:$input); let arguments = (ins Variadic<F64Tensor>:$input);
// The return operation only emits the input in the format if it is present. // The return operation only emits the input in the format if it is present.
let assemblyFormat = "($input^ `:` type($input))? attr-dict "; let assemblyFormat = "($input^ `:` type($input))? attr-dict ";
// Allow building a ReturnOp with no return operand. // Allow building a ReturnOp with no return operand.
let builders = [ let builders = [
OpBuilder<(ins), [{ build($_builder, $_state, llvm::None); }]> OpBuilder<(ins), [{ build($_builder, $_state, std::nullopt); }]>
]; ];
// Provide extra utility definitions on the c++ operation class definition. // Provide extra utility definitions on the c++ operation class definition.
let extraClassDeclaration = [{ let extraClassDeclaration = [{
bool hasOperand() { return getNumOperands() != 0; } bool hasOperand() { return getNumOperands() != 0; }
}]; }];
// Indicate that additional verification for this operation is necessary. // Indicate that additional verification for this operation is necessary.
Show All 31 Lines

mlir/examples/toy/Ch4/mlir/MLIRGen.cpp

Show First 20 Lines • Show All 105 Lines • ▼ Show 20 Linesprivate:
/// provided Toy AST prototype. /// provided Toy AST prototype.
mlir::toy::FuncOp mlirGen(PrototypeAST &proto) { mlir::toy::FuncOp mlirGen(PrototypeAST &proto) {
auto location = loc(proto.loc()); auto location = loc(proto.loc());
// This is a generic function, the return type will be inferred later. // This is a generic function, the return type will be inferred later.
// Arguments type are uniformly unranked tensors. // Arguments type are uniformly unranked tensors.
llvm::SmallVector<mlir::Type, 4> argTypes(proto.getArgs().size(), llvm::SmallVector<mlir::Type, 4> argTypes(proto.getArgs().size(),
getType(VarType{})); getType(VarType{}));
auto funcType = builder.getFunctionType(argTypes, llvm::None); auto funcType = builder.getFunctionType(argTypes, std::nullopt);
return builder.create<mlir::toy::FuncOp>(location, proto.getName(), return builder.create<mlir::toy::FuncOp>(location, proto.getName(),
funcType); funcType);
} }
/// Emit a new function and add it to the MLIR module. /// Emit a new function and add it to the MLIR module.
mlir::toy::FuncOp mlirGen(FunctionAST &funcAST) { mlir::toy::FuncOp mlirGen(FunctionAST &funcAST) {
// Create a scope in the symbol table to hold variable declarations. // Create a scope in the symbol table to hold variable declarations.
ScopedHashTableScope<llvm::StringRef, mlir::Value> varScope(symbolTable); ScopedHashTableScope<llvm::StringRef, mlir::Value> varScope(symbolTable);
▲ Show 20 Lines • Show All 329 Lines • Show Last 20 Lines

mlir/examples/toy/Ch5/include/toy/AST.h

Show First 20 Lines • Show All 131 Lines • ▼ Show 20 Lines
public: public:
ReturnExprAST(Location loc, llvm::Optional<std::unique_ptr<ExprAST>> expr) ReturnExprAST(Location loc, llvm::Optional<std::unique_ptr<ExprAST>> expr)
: ExprAST(Expr_Return, std::move(loc)), expr(std::move(expr)) {} : ExprAST(Expr_Return, std::move(loc)), expr(std::move(expr)) {}
llvm::Optional<ExprAST *> getExpr() { llvm::Optional<ExprAST *> getExpr() {
if (expr.has_value()) if (expr.has_value())
return expr->get(); return expr->get();
return llvm::None; return std::nullopt;
} }
/// LLVM style RTTI /// LLVM style RTTI
static bool classof(const ExprAST *c) { return c->getKind() == Expr_Return; } static bool classof(const ExprAST *c) { return c->getKind() == Expr_Return; }
}; };
/// Expression class for a binary operator. /// Expression class for a binary operator.
class BinaryExprAST : public ExprAST { class BinaryExprAST : public ExprAST {
▲ Show 20 Lines • Show All 97 Lines • Show Last 20 Lines

mlir/examples/toy/Ch5/include/toy/Ops.td

Show First 20 Lines • Show All 320 Lines • ▼ Show 20 Linesdef ReturnOp : Toy_Op<"return", [Pure, HasParent<"FuncOp">,
// value must match the return type of the enclosing function. // value must match the return type of the enclosing function.
let arguments = (ins Variadic<F64Tensor>:$input); let arguments = (ins Variadic<F64Tensor>:$input);
// The return operation only emits the input in the format if it is present. // The return operation only emits the input in the format if it is present.
let assemblyFormat = "($input^ `:` type($input))? attr-dict "; let assemblyFormat = "($input^ `:` type($input))? attr-dict ";
// Allow building a ReturnOp with no return operand. // Allow building a ReturnOp with no return operand.
let builders = [ let builders = [
OpBuilder<(ins), [{ build($_builder, $_state, llvm::None); }]> OpBuilder<(ins), [{ build($_builder, $_state, std::nullopt); }]>
]; ];
// Provide extra utility definitions on the c++ operation class definition. // Provide extra utility definitions on the c++ operation class definition.
let extraClassDeclaration = [{ let extraClassDeclaration = [{
bool hasOperand() { return getNumOperands() != 0; } bool hasOperand() { return getNumOperands() != 0; }
}]; }];
// Indicate that additional verification for this operation is necessary. // Indicate that additional verification for this operation is necessary.
Show All 31 Lines

mlir/examples/toy/Ch5/mlir/MLIRGen.cpp

Show First 20 Lines • Show All 105 Lines • ▼ Show 20 Linesprivate:
/// provided Toy AST prototype. /// provided Toy AST prototype.
mlir::toy::FuncOp mlirGen(PrototypeAST &proto) { mlir::toy::FuncOp mlirGen(PrototypeAST &proto) {
auto location = loc(proto.loc()); auto location = loc(proto.loc());
// This is a generic function, the return type will be inferred later. // This is a generic function, the return type will be inferred later.
// Arguments type are uniformly unranked tensors. // Arguments type are uniformly unranked tensors.
llvm::SmallVector<mlir::Type, 4> argTypes(proto.getArgs().size(), llvm::SmallVector<mlir::Type, 4> argTypes(proto.getArgs().size(),
getType(VarType{})); getType(VarType{}));
auto funcType = builder.getFunctionType(argTypes, llvm::None); auto funcType = builder.getFunctionType(argTypes, std::nullopt);
return builder.create<mlir::toy::FuncOp>(location, proto.getName(), return builder.create<mlir::toy::FuncOp>(location, proto.getName(),
funcType); funcType);
} }
/// Emit a new function and add it to the MLIR module. /// Emit a new function and add it to the MLIR module.
mlir::toy::FuncOp mlirGen(FunctionAST &funcAST) { mlir::toy::FuncOp mlirGen(FunctionAST &funcAST) {
// Create a scope in the symbol table to hold variable declarations. // Create a scope in the symbol table to hold variable declarations.
ScopedHashTableScope<llvm::StringRef, mlir::Value> varScope(symbolTable); ScopedHashTableScope<llvm::StringRef, mlir::Value> varScope(symbolTable);
▲ Show 20 Lines • Show All 329 Lines • Show Last 20 Lines

mlir/examples/toy/Ch6/include/toy/AST.h

Show First 20 Lines • Show All 131 Lines • ▼ Show 20 Lines
public: public:
ReturnExprAST(Location loc, llvm::Optional<std::unique_ptr<ExprAST>> expr) ReturnExprAST(Location loc, llvm::Optional<std::unique_ptr<ExprAST>> expr)
: ExprAST(Expr_Return, std::move(loc)), expr(std::move(expr)) {} : ExprAST(Expr_Return, std::move(loc)), expr(std::move(expr)) {}
llvm::Optional<ExprAST *> getExpr() { llvm::Optional<ExprAST *> getExpr() {
if (expr.has_value()) if (expr.has_value())
return expr->get(); return expr->get();
return llvm::None; return std::nullopt;
} }
/// LLVM style RTTI /// LLVM style RTTI
static bool classof(const ExprAST *c) { return c->getKind() == Expr_Return; } static bool classof(const ExprAST *c) { return c->getKind() == Expr_Return; }
}; };
/// Expression class for a binary operator. /// Expression class for a binary operator.
class BinaryExprAST : public ExprAST { class BinaryExprAST : public ExprAST {
▲ Show 20 Lines • Show All 97 Lines • Show Last 20 Lines

mlir/examples/toy/Ch6/include/toy/Ops.td

Show First 20 Lines • Show All 320 Lines • ▼ Show 20 Linesdef ReturnOp : Toy_Op<"return", [Pure, HasParent<"FuncOp">,
// value must match the return type of the enclosing function. // value must match the return type of the enclosing function.
let arguments = (ins Variadic<F64Tensor>:$input); let arguments = (ins Variadic<F64Tensor>:$input);
// The return operation only emits the input in the format if it is present. // The return operation only emits the input in the format if it is present.
let assemblyFormat = "($input^ `:` type($input))? attr-dict "; let assemblyFormat = "($input^ `:` type($input))? attr-dict ";
// Allow building a ReturnOp with no return operand. // Allow building a ReturnOp with no return operand.
let builders = [ let builders = [
OpBuilder<(ins), [{ build($_builder, $_state, llvm::None); }]> OpBuilder<(ins), [{ build($_builder, $_state, std::nullopt); }]>
]; ];
// Provide extra utility definitions on the c++ operation class definition. // Provide extra utility definitions on the c++ operation class definition.
let extraClassDeclaration = [{ let extraClassDeclaration = [{
bool hasOperand() { return getNumOperands() != 0; } bool hasOperand() { return getNumOperands() != 0; }
}]; }];
// Indicate that additional verification for this operation is necessary. // Indicate that additional verification for this operation is necessary.
Show All 31 Lines

mlir/examples/toy/Ch6/mlir/MLIRGen.cpp

Show First 20 Lines • Show All 105 Lines • ▼ Show 20 Linesprivate:
/// provided Toy AST prototype. /// provided Toy AST prototype.
mlir::toy::FuncOp mlirGen(PrototypeAST &proto) { mlir::toy::FuncOp mlirGen(PrototypeAST &proto) {
auto location = loc(proto.loc()); auto location = loc(proto.loc());
// This is a generic function, the return type will be inferred later. // This is a generic function, the return type will be inferred later.
// Arguments type are uniformly unranked tensors. // Arguments type are uniformly unranked tensors.
llvm::SmallVector<mlir::Type, 4> argTypes(proto.getArgs().size(), llvm::SmallVector<mlir::Type, 4> argTypes(proto.getArgs().size(),
getType(VarType{})); getType(VarType{}));
auto funcType = builder.getFunctionType(argTypes, llvm::None); auto funcType = builder.getFunctionType(argTypes, std::nullopt);
return builder.create<mlir::toy::FuncOp>(location, proto.getName(), return builder.create<mlir::toy::FuncOp>(location, proto.getName(),
funcType); funcType);
} }
/// Emit a new function and add it to the MLIR module. /// Emit a new function and add it to the MLIR module.
mlir::toy::FuncOp mlirGen(FunctionAST &funcAST) { mlir::toy::FuncOp mlirGen(FunctionAST &funcAST) {
// Create a scope in the symbol table to hold variable declarations. // Create a scope in the symbol table to hold variable declarations.
ScopedHashTableScope<llvm::StringRef, mlir::Value> varScope(symbolTable); ScopedHashTableScope<llvm::StringRef, mlir::Value> varScope(symbolTable);
▲ Show 20 Lines • Show All 329 Lines • Show Last 20 Lines

mlir/examples/toy/Ch7/include/toy/AST.h

Show First 20 Lines • Show All 151 Lines • ▼ Show 20 Lines
public: public:
ReturnExprAST(Location loc, llvm::Optional<std::unique_ptr<ExprAST>> expr) ReturnExprAST(Location loc, llvm::Optional<std::unique_ptr<ExprAST>> expr)
: ExprAST(Expr_Return, std::move(loc)), expr(std::move(expr)) {} : ExprAST(Expr_Return, std::move(loc)), expr(std::move(expr)) {}
llvm::Optional<ExprAST *> getExpr() { llvm::Optional<ExprAST *> getExpr() {
if (expr.has_value()) if (expr.has_value())
return expr->get(); return expr->get();
return llvm::None; return std::nullopt;
} }
/// LLVM style RTTI /// LLVM style RTTI
static bool classof(const ExprAST *c) { return c->getKind() == Expr_Return; } static bool classof(const ExprAST *c) { return c->getKind() == Expr_Return; }
}; };
/// Expression class for a binary operator. /// Expression class for a binary operator.
class BinaryExprAST : public ExprAST { class BinaryExprAST : public ExprAST {
▲ Show 20 Lines • Show All 144 Lines • Show Last 20 Lines

mlir/examples/toy/Ch7/include/toy/Ops.td

Show First 20 Lines • Show All 344 Lines • ▼ Show 20 Linesdef ReturnOp : Toy_Op<"return", [Pure, HasParent<"FuncOp">,
// value must match the return type of the enclosing function. // value must match the return type of the enclosing function.
let arguments = (ins Variadic<Toy_Type>:$input); let arguments = (ins Variadic<Toy_Type>:$input);
// The return operation only emits the input in the format if it is present. // The return operation only emits the input in the format if it is present.
let assemblyFormat = "($input^ `:` type($input))? attr-dict "; let assemblyFormat = "($input^ `:` type($input))? attr-dict ";
// Allow building a ReturnOp with no return operand. // Allow building a ReturnOp with no return operand.
let builders = [ let builders = [
OpBuilder<(ins), [{ build($_builder, $_state, llvm::None); }]> OpBuilder<(ins), [{ build($_builder, $_state, std::nullopt); }]>
]; ];
// Provide extra utility definitions on the c++ operation class definition. // Provide extra utility definitions on the c++ operation class definition.
let extraClassDeclaration = [{ let extraClassDeclaration = [{
bool hasOperand() { return getNumOperands() != 0; } bool hasOperand() { return getNumOperands() != 0; }
}]; }];
// Indicate that additional verification for this operation is necessary. // Indicate that additional verification for this operation is necessary.
▲ Show 20 Lines • Show All 88 Lines • Show Last 20 Lines

mlir/examples/toy/Ch7/mlir/MLIRGen.cpp

Show First 20 Lines • Show All 161 Lines • ▼ Show 20 Lines mlir::toy::FuncOp mlirGen(PrototypeAST &proto) {
llvm::SmallVector<mlir::Type, 4> argTypes; llvm::SmallVector<mlir::Type, 4> argTypes;
argTypes.reserve(proto.getArgs().size()); argTypes.reserve(proto.getArgs().size());
for (auto &arg : proto.getArgs()) { for (auto &arg : proto.getArgs()) {
mlir::Type type = getType(arg->getType(), arg->loc()); mlir::Type type = getType(arg->getType(), arg->loc());
if (!type) if (!type)
return nullptr; return nullptr;
argTypes.push_back(type); argTypes.push_back(type);
} }
auto funcType = builder.getFunctionType(argTypes, llvm::None); auto funcType = builder.getFunctionType(argTypes, std::nullopt);
return builder.create<mlir::toy::FuncOp>(location, proto.getName(), return builder.create<mlir::toy::FuncOp>(location, proto.getName(),
funcType); funcType);
} }
/// Emit a new function and add it to the MLIR module. /// Emit a new function and add it to the MLIR module.
mlir::toy::FuncOp mlirGen(FunctionAST &funcAST) { mlir::toy::FuncOp mlirGen(FunctionAST &funcAST) {
// Create a scope in the symbol table to hold variable declarations. // Create a scope in the symbol table to hold variable declarations.
SymbolTableScopeT varScope(symbolTable); SymbolTableScopeT varScope(symbolTable);
▲ Show 20 Lines • Show All 93 Lines • ▼ Show 20 Linesprivate:
/// Return the numeric member index of the given struct access expression. /// Return the numeric member index of the given struct access expression.
llvm::Optional<size_t> getMemberIndex(BinaryExprAST &accessOp) { llvm::Optional<size_t> getMemberIndex(BinaryExprAST &accessOp) {
assert(accessOp.getOp() == '.' && "expected access operation"); assert(accessOp.getOp() == '.' && "expected access operation");
// Lookup the struct node for the LHS. // Lookup the struct node for the LHS.
StructAST *structAST = getStructFor(accessOp.getLHS()); StructAST *structAST = getStructFor(accessOp.getLHS());
if (!structAST) if (!structAST)
return llvm::None; return std::nullopt;
// Get the name from the RHS. // Get the name from the RHS.
VariableExprAST *name = llvm::dyn_cast<VariableExprAST>(accessOp.getRHS()); VariableExprAST *name = llvm::dyn_cast<VariableExprAST>(accessOp.getRHS());
if (!name) if (!name)
return llvm::None; return std::nullopt;
auto structVars = structAST->getVariables(); auto structVars = structAST->getVariables();
const auto *it = llvm::find_if(structVars, [&](auto &var) { const auto *it = llvm::find_if(structVars, [&](auto &var) {
return var->getName() == name->getName(); return var->getName() == name->getName();
}); });
if (it == structVars.end()) if (it == structVars.end())
return llvm::None; return std::nullopt;
return it - structVars.begin(); return it - structVars.begin();
} }
/// Emit a binary operation /// Emit a binary operation
mlir::Value mlirGen(BinaryExprAST &binop) { mlir::Value mlirGen(BinaryExprAST &binop) {
// First emit the operations for each side of the operation before emitting // First emit the operations for each side of the operation before emitting
// the operation itself. For example if the expression is `a + foo(a)` // the operation itself. For example if the expression is `a + foo(a)`
// 1) First it will visiting the LHS, which will return a reference to the // 1) First it will visiting the LHS, which will return a reference to the
▲ Show 20 Lines • Show All 120 Lines • ▼ Show 20 Linesprivate:
std::pair<mlir::ArrayAttr, mlir::Type> std::pair<mlir::ArrayAttr, mlir::Type>
getConstantAttr(StructLiteralExprAST &lit) { getConstantAttr(StructLiteralExprAST &lit) {
std::vector<mlir::Attribute> attrElements; std::vector<mlir::Attribute> attrElements;
std::vector<mlir::Type> typeElements; std::vector<mlir::Type> typeElements;
for (auto &var : lit.getValues()) { for (auto &var : lit.getValues()) {
if (auto *number = llvm::dyn_cast<NumberExprAST>(var.get())) { if (auto *number = llvm::dyn_cast<NumberExprAST>(var.get())) {
attrElements.push_back(getConstantAttr(*number)); attrElements.push_back(getConstantAttr(*number));
typeElements.push_back(getType(llvm::None)); typeElements.push_back(getType(std::nullopt));
} else if (auto *lit = llvm::dyn_cast<LiteralExprAST>(var.get())) { } else if (auto *lit = llvm::dyn_cast<LiteralExprAST>(var.get())) {
attrElements.push_back(getConstantAttr(*lit)); attrElements.push_back(getConstantAttr(*lit));
typeElements.push_back(getType(llvm::None)); typeElements.push_back(getType(std::nullopt));
} else { } else {
auto *structLit = llvm::cast<StructLiteralExprAST>(var.get()); auto *structLit = llvm::cast<StructLiteralExprAST>(var.get());
auto attrTypePair = getConstantAttr(*structLit); auto attrTypePair = getConstantAttr(*structLit);
attrElements.push_back(attrTypePair.first); attrElements.push_back(attrTypePair.first);
typeElements.push_back(attrTypePair.second); typeElements.push_back(attrTypePair.second);
} }
} }
mlir::ArrayAttr dataAttr = builder.getArrayAttr(attrElements); mlir::ArrayAttr dataAttr = builder.getArrayAttr(attrElements);
▲ Show 20 Lines • Show All 236 Lines • Show Last 20 Lines

mlir/include/mlir/AsmParser/AsmParserState.h

Show First 20 Lines • Show All 137 Lines • ▼ Show 20 Linespublic:
void finalize(Operation *topLevelOp); void finalize(Operation *topLevelOp);
/// Start a definition for an operation with the given name. /// Start a definition for an operation with the given name.
void startOperationDefinition(const OperationName &opName); void startOperationDefinition(const OperationName &opName);
/// Finalize the most recently started operation definition. /// Finalize the most recently started operation definition.
void finalizeOperationDefinition( void finalizeOperationDefinition(
Operation *op, SMRange nameLoc, SMLoc endLoc, Operation *op, SMRange nameLoc, SMLoc endLoc,
ArrayRef<std::pair<unsigned, SMLoc>> resultGroups = llvm::None); ArrayRef<std::pair<unsigned, SMLoc>> resultGroups = std::nullopt);
/// Start a definition for a region nested under the current operation. /// Start a definition for a region nested under the current operation.
void startRegionDefinition(); void startRegionDefinition();
/// Finalize the most recently started region definition. /// Finalize the most recently started region definition.
void finalizeRegionDefinition(); void finalizeRegionDefinition();
/// Add a definition of the given entity. /// Add a definition of the given entity.
Show All 28 Lines

mlir/include/mlir/CAPI/Wrap.h

Show All 38 Lines
template <typename CppTy, typename CTy> template <typename CppTy, typename CTy>
static llvm::ArrayRef<CppTy> unwrapList(size_t size, CTy *first, static llvm::ArrayRef<CppTy> unwrapList(size_t size, CTy *first,
llvm::SmallVectorImpl<CppTy> &storage) { llvm::SmallVectorImpl<CppTy> &storage) {
static_assert( static_assert(
std::is_same<decltype(unwrap(std::declval<CTy>())), CppTy>::value, std::is_same<decltype(unwrap(std::declval<CTy>())), CppTy>::value,
"incompatible C and C++ types"); "incompatible C and C++ types");
if (size == 0) if (size == 0)
return llvm::None; return std::nullopt;
assert(storage.empty() && "expected to populate storage"); assert(storage.empty() && "expected to populate storage");
storage.reserve(size); storage.reserve(size);
for (size_t i = 0; i < size; ++i) for (size_t i = 0; i < size; ++i)
storage.push_back(unwrap(*(first + i))); storage.push_back(unwrap(*(first + i)));
return storage; return storage;
} }
#endif // MLIR_CAPI_WRAP_H #endif // MLIR_CAPI_WRAP_H

mlir/include/mlir/Conversion/MathToLibm/MathToLibm.h

Show All 15 Lines
#define GEN_PASS_DECL_CONVERTMATHTOLIBM #define GEN_PASS_DECL_CONVERTMATHTOLIBM
#include "mlir/Conversion/Passes.h.inc" #include "mlir/Conversion/Passes.h.inc"
/// Populate the given list with patterns that convert from Math to Libm calls. /// Populate the given list with patterns that convert from Math to Libm calls.
/// If log1pBenefit is present, use it instead of benefit for the Log1p op. /// If log1pBenefit is present, use it instead of benefit for the Log1p op.
void populateMathToLibmConversionPatterns( void populateMathToLibmConversionPatterns(
RewritePatternSet &patterns, PatternBenefit benefit, RewritePatternSet &patterns, PatternBenefit benefit,
llvm::Optional<PatternBenefit> log1pBenefit = llvm::None); llvm::Optional<PatternBenefit> log1pBenefit = std::nullopt);
/// Create a pass to convert Math operations to libm calls. /// Create a pass to convert Math operations to libm calls.
std::unique_ptr<OperationPass<ModuleOp>> createConvertMathToLibmPass(); std::unique_ptr<OperationPass<ModuleOp>> createConvertMathToLibmPass();
} // namespace mlir } // namespace mlir
#endif // MLIR_CONVERSION_MATHTOLIBM_MATHTOLIBM_H_ #endif // MLIR_CONVERSION_MATHTOLIBM_MATHTOLIBM_H_

mlir/include/mlir/Dialect/Affine/Analysis/AffineAnalysis.h

Show First 20 Lines • Show All 137 Lines • ▼ Show 20 Lines
// lb < ub. Note that ub/lb == None means unbounded. // lb < ub. Note that ub/lb == None means unbounded.
struct DependenceComponent { struct DependenceComponent {
// The AffineForOp Operation associated with this dependence component. // The AffineForOp Operation associated with this dependence component.
Operation *op = nullptr; Operation *op = nullptr;
// The lower bound of the dependence distance. // The lower bound of the dependence distance.
Optional<int64_t> lb; Optional<int64_t> lb;
// The upper bound of the dependence distance (inclusive). // The upper bound of the dependence distance (inclusive).
Optional<int64_t> ub; Optional<int64_t> ub;
DependenceComponent() : lb(llvm::None), ub(llvm::None) {} DependenceComponent() : lb(std::nullopt), ub(llvm::None) {}
kazuUnsubmitted
Not Done
ReplyInline Actions

Did you mean to replace this one also?

kazu: Did you mean to replace this one also?
}; };
/// Checks whether two accesses to the same memref access the same element. /// Checks whether two accesses to the same memref access the same element.
/// Each access is specified using the MemRefAccess structure, which contains /// Each access is specified using the MemRefAccess structure, which contains
/// the operation, indices and memref associated with the access. Returns /// the operation, indices and memref associated with the access. Returns
/// 'NoDependence' if it can be determined conclusively that the accesses do not /// 'NoDependence' if it can be determined conclusively that the accesses do not
/// access the same memref element. If 'allowRAR' is true, will consider /// access the same memref element. If 'allowRAR' is true, will consider
/// read-after-read dependences (typically used by applications trying to /// read-after-read dependences (typically used by applications trying to
▲ Show 20 Lines • Show All 41 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Affine/Analysis/Utils.h

Show First 20 Lines • Show All 104 Lines • ▼ Show 20 Linesstruct ComputationSliceState {
// Clears all bounds and operands in slice state. // Clears all bounds and operands in slice state.
void clearBounds(); void clearBounds();
/// Returns true if the computation slice is empty. /// Returns true if the computation slice is empty.
bool isEmpty() const { return ivs.empty(); } bool isEmpty() const { return ivs.empty(); }
/// Returns true if the computation slice encloses all the iterations of the /// Returns true if the computation slice encloses all the iterations of the
/// sliced loop nest. Returns false if it does not. Returns llvm::None if it /// sliced loop nest. Returns false if it does not. Returns std::nullopt if it
/// cannot determine if the slice is maximal or not. /// cannot determine if the slice is maximal or not.
// TODO: Cache 'isMaximal' so that we don't recompute it when the slice // TODO: Cache 'isMaximal' so that we don't recompute it when the slice
// information hasn't changed. // information hasn't changed.
Optional<bool> isMaximal() const; Optional<bool> isMaximal() const;
/// Checks the validity of the slice computed. This is done using the /// Checks the validity of the slice computed. This is done using the
/// following steps: /// following steps:
/// 1. Get the new domain of the slice that would be created if fusion /// 1. Get the new domain of the slice that would be created if fusion
Show All 12 Linesstruct ComputationSliceState {
void dump() const; void dump() const;
private: private:
/// Fast check to determine if the computation slice is maximal. Returns true /// Fast check to determine if the computation slice is maximal. Returns true
/// if each slice dimension maps to an existing dst dimension and both the src /// if each slice dimension maps to an existing dst dimension and both the src
/// and the dst loops for those dimensions have the same bounds. Returns false /// and the dst loops for those dimensions have the same bounds. Returns false
/// if both the src and the dst loops don't have the same bounds. Returns /// if both the src and the dst loops don't have the same bounds. Returns
/// llvm::None if none of the above can be proven. /// std::nullopt if none of the above can be proven.
Optional<bool> isSliceMaximalFastCheck() const; Optional<bool> isSliceMaximalFastCheck() const;
}; };
/// Computes the computation slice loop bounds for one loop nest as affine maps /// Computes the computation slice loop bounds for one loop nest as affine maps
/// of the other loop nest's IVs and symbols, using 'dependenceConstraints' /// of the other loop nest's IVs and symbols, using 'dependenceConstraints'
/// computed between 'depSourceAccess' and 'depSinkAccess'. /// computed between 'depSourceAccess' and 'depSinkAccess'.
/// If 'isBackwardSlice' is true, a backwards slice is computed in which the /// If 'isBackwardSlice' is true, a backwards slice is computed in which the
/// slice bounds of loop nest surrounding 'depSourceAccess' are computed in /// slice bounds of loop nest surrounding 'depSourceAccess' are computed in
▲ Show 20 Lines • Show All 238 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Affine/IR/AffineOps.td

Show First 20 Lines • Show All 218 Lines • ▼ Show 20 Linesdef AffineForOp : Affine_Op<"for",
}]; }];
let arguments = (ins Variadic<AnyType>); let arguments = (ins Variadic<AnyType>);
let results = (outs Variadic<AnyType>:$results); let results = (outs Variadic<AnyType>:$results);
let regions = (region SizedRegion<1>:$region); let regions = (region SizedRegion<1>:$region);
let skipDefaultBuilders = 1; let skipDefaultBuilders = 1;
let builders = [ let builders = [
OpBuilder<(ins "int64_t":$lowerBound, "int64_t":$upperBound, OpBuilder<(ins "int64_t":$lowerBound, "int64_t":$upperBound,
CArg<"int64_t", "1">:$step, CArg<"ValueRange", "llvm::None">:$iterArgs, CArg<"int64_t", "1">:$step, CArg<"ValueRange", "std::nullopt">:$iterArgs,
CArg<"function_ref<void(OpBuilder &, Location, Value, ValueRange)>", CArg<"function_ref<void(OpBuilder &, Location, Value, ValueRange)>",
"nullptr">:$bodyBuilder)>, "nullptr">:$bodyBuilder)>,
OpBuilder<(ins "ValueRange":$lbOperands, "AffineMap":$lbMap, OpBuilder<(ins "ValueRange":$lbOperands, "AffineMap":$lbMap,
"ValueRange":$ubOperands, "AffineMap":$ubMap, CArg<"int64_t", "1">:$step, "ValueRange":$ubOperands, "AffineMap":$ubMap, CArg<"int64_t", "1">:$step,
CArg<"ValueRange", "llvm::None">:$iterArgs, CArg<"ValueRange", "std::nullopt">:$iterArgs,
CArg<"function_ref<void(OpBuilder &, Location, Value, ValueRange)>", CArg<"function_ref<void(OpBuilder &, Location, Value, ValueRange)>",
"nullptr">:$bodyBuilder)> "nullptr">:$bodyBuilder)>
]; ];
let extraClassDeclaration = [{ let extraClassDeclaration = [{
/// Defining the function type we use for building the body of affine.for. /// Defining the function type we use for building the body of affine.for.
using BodyBuilderFn = using BodyBuilderFn =
function_ref<void(OpBuilder &, Location, Value, ValueRange)>; function_ref<void(OpBuilder &, Location, Value, ValueRange)>;
▲ Show 20 Lines • Show All 679 Lines • ▼ Show 20 Lines let description = [{
Otherwise, it has to be present in the syntax to indicate which values are Otherwise, it has to be present in the syntax to indicate which values are
yielded. yielded.
``` ```
}]; }];
let arguments = (ins Variadic<AnyType>:$operands); let arguments = (ins Variadic<AnyType>:$operands);
let builders = [ let builders = [
OpBuilder<(ins), [{ build($_builder, $_state, llvm::None); }]> OpBuilder<(ins), [{ build($_builder, $_state, std::nullopt); }]>
]; ];
let assemblyFormat = "attr-dict ($operands^ `:` type($operands))?"; let assemblyFormat = "attr-dict ($operands^ `:` type($operands))?";
let hasVerifier = 1; let hasVerifier = 1;
} }
def AffineVectorLoadOp : AffineLoadOpBase<"vector_load"> { def AffineVectorLoadOp : AffineLoadOpBase<"vector_load"> {
let summary = "affine vector load operation"; let summary = "affine vector load operation";
▲ Show 20 Lines • Show All 173 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Async/IR/AsyncOps.td

Show First 20 Lines • Show All 275 Lines • ▼ Show 20 Lines let description = [{
async.func @foo() : !async.token { async.func @foo() : !async.token {
return return
} }
``` ```
}]; }];
let arguments = (ins Variadic<AnyType>:$operands); let arguments = (ins Variadic<AnyType>:$operands);
let builders = [OpBuilder<(ins), [{build($_builder, $_state, llvm::None);}]>]; let builders = [OpBuilder<(ins), [{build($_builder, $_state, std::nullopt);}]>];
let assemblyFormat = "attr-dict ($operands^ `:` type($operands))?"; let assemblyFormat = "attr-dict ($operands^ `:` type($operands))?";
let hasVerifier = 1; let hasVerifier = 1;
} }
def Async_YieldOp : def Async_YieldOp :
Async_Op<"yield", [ Async_Op<"yield", [
HasParent<"ExecuteOp">, Pure, Terminator, HasParent<"ExecuteOp">, Pure, Terminator,
▲ Show 20 Lines • Show All 439 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Bufferization/IR/AllocationOpInterface.td

Show All 29 Linesdef AllocationOpInterface : OpInterface<"AllocationOpInterface"> {
let cppNamespace = "::mlir::bufferization"; let cppNamespace = "::mlir::bufferization";
let methods = [ let methods = [
StaticInterfaceMethod<[{ StaticInterfaceMethod<[{
Builds a deallocation operation using the provided builder and the Builds a deallocation operation using the provided builder and the
current allocation value (which refers to the current Op implementing current allocation value (which refers to the current Op implementing
this interface). The allocation value is a result of the current this interface). The allocation value is a result of the current
operation implementing this interface. If there is no compatible operation implementing this interface. If there is no compatible
deallocation operation, this method can return ::llvm::None. deallocation operation, this method can return ::std::nullopt.
}], }],
"::mlir::Optional<::mlir::Operation*>", "buildDealloc", "::mlir::Optional<::mlir::Operation*>", "buildDealloc",
(ins "::mlir::OpBuilder&":$builder, "::mlir::Value":$alloc), [{}], (ins "::mlir::OpBuilder&":$builder, "::mlir::Value":$alloc), [{}],
/*defaultImplementation=*/[{ return llvm::None; }] /*defaultImplementation=*/[{ return std::nullopt; }]
>, >,
StaticInterfaceMethod<[{ StaticInterfaceMethod<[{
Builds a clone operation using the provided builder and the current Builds a clone operation using the provided builder and the current
allocation value (which refers to the current Op implementing this allocation value (which refers to the current Op implementing this
interface). The allocation value is a result of the current operation interface). The allocation value is a result of the current operation
implementing this interface. If there is no compatible clone operation, implementing this interface. If there is no compatible clone operation,
this method can return ::llvm::None. this method can return ::std::nullopt.
}], }],
"::mlir::Optional<::mlir::Value>", "buildClone", "::mlir::Optional<::mlir::Value>", "buildClone",
(ins "::mlir::OpBuilder&":$builder, "::mlir::Value":$alloc), [{}], (ins "::mlir::OpBuilder&":$builder, "::mlir::Value":$alloc), [{}],
/*defaultImplementation=*/[{ return llvm::None; }] /*defaultImplementation=*/[{ return std::nullopt; }]
> >
]; ];
} }
#endif // ALLOCATION_OP_INTERFACE #endif // ALLOCATION_OP_INTERFACE

mlir/include/mlir/Dialect/Bufferization/IR/BufferizableOpInterface.h

Show First 20 Lines • Show All 224 Lines • ▼ Show 20 Linesstruct BufferizationOptions {
/// the boundaries. /// the boundaries.
bool allowUnknownOps = false; bool allowUnknownOps = false;
/// Specifies whether function boundaries (ops in the func dialect) should be /// Specifies whether function boundaries (ops in the func dialect) should be
/// bufferized or not. /// bufferized or not.
bool bufferizeFunctionBoundaries = false; bool bufferizeFunctionBoundaries = false;
/// The default memory space that should be used when it cannot be inferred /// The default memory space that should be used when it cannot be inferred
/// from the context. If case of llvm::None, bufferization fails when the /// from the context. If case of std::nullopt, bufferization fails when the
/// memory space cannot be inferred at any point. /// memory space cannot be inferred at any point.
Optional<Attribute> defaultMemorySpace = Attribute(); Optional<Attribute> defaultMemorySpace = Attribute();
/// Certain ops have aliasing OpOperand/OpResult invariants (e.g., scf.for). /// Certain ops have aliasing OpOperand/OpResult invariants (e.g., scf.for).
/// If this flag is set to `false`, those invariants are no longer enforced /// If this flag is set to `false`, those invariants are no longer enforced
/// with buffer copies. /// with buffer copies.
/// ///
/// Note: Deactivating this flag can lead to incorrect bufferization results /// Note: Deactivating this flag can lead to incorrect bufferization results
▲ Show 20 Lines • Show All 310 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Func/IR/FuncOps.td

Show First 20 Lines • Show All 343 Lines • ▼ Show 20 Lines func.func @foo() : (i32, f8) {
return %0, %1 : i32, f8 return %0, %1 : i32, f8
} }
``` ```
}]; }];
let arguments = (ins Variadic<AnyType>:$operands); let arguments = (ins Variadic<AnyType>:$operands);
let builders = [OpBuilder<(ins), [{ let builders = [OpBuilder<(ins), [{
build($_builder, $_state, llvm::None); build($_builder, $_state, std::nullopt);
}]>]; }]>];
let assemblyFormat = "attr-dict ($operands^ `:` type($operands))?"; let assemblyFormat = "attr-dict ($operands^ `:` type($operands))?";
let hasVerifier = 1; let hasVerifier = 1;
} }
#endif // MLIR_DIALECT_FUNC_IR_FUNCOPS_TD #endif // MLIR_DIALECT_FUNC_IR_FUNCOPS_TD

mlir/include/mlir/Dialect/Func/Transforms/DecomposeCallGraphTypes.h

Show First 20 Lines • Show All 66 Lines • ▼ Show 20 Linesprivate:
template <typename T, typename FnT> template <typename T, typename FnT>
DecomposeValueConversionCallFn DecomposeValueConversionCallFn
wrapDecomposeValueConversionCallback(FnT &&callback) { wrapDecomposeValueConversionCallback(FnT &&callback) {
return [callback = std::forward<FnT>(callback)]( return [callback = std::forward<FnT>(callback)](
OpBuilder &builder, Location loc, Type type, Value value, OpBuilder &builder, Location loc, Type type, Value value,
SmallVectorImpl<Value> &newValues) -> Optional<LogicalResult> { SmallVectorImpl<Value> &newValues) -> Optional<LogicalResult> {
if (T derivedType = type.dyn_cast<T>()) if (T derivedType = type.dyn_cast<T>())
return callback(builder, loc, derivedType, value, newValues); return callback(builder, loc, derivedType, value, newValues);
return llvm::None; return std::nullopt;
}; };
} }
SmallVector<DecomposeValueConversionCallFn, 2> decomposeValueConversions; SmallVector<DecomposeValueConversionCallFn, 2> decomposeValueConversions;
}; };
/// Populates the patterns needed to drive the conversion process for /// Populates the patterns needed to drive the conversion process for
/// decomposing call graph types with the given `ValueDecomposer`. /// decomposing call graph types with the given `ValueDecomposer`.
void populateDecomposeCallGraphTypesPatterns(MLIRContext *context, void populateDecomposeCallGraphTypesPatterns(MLIRContext *context,
TypeConverter &typeConverter, TypeConverter &typeConverter,
ValueDecomposer &decomposer, ValueDecomposer &decomposer,
RewritePatternSet &patterns); RewritePatternSet &patterns);
} // namespace mlir } // namespace mlir
#endif // MLIR_DIALECT_FUNC_TRANSFORMS_DECOMPOSECALLGRAPHTYPES_H #endif // MLIR_DIALECT_FUNC_TRANSFORMS_DECOMPOSECALLGRAPHTYPES_H

mlir/include/mlir/Dialect/LLVMIR/LLVMIntrinsicOps.td

Show First 20 Lines • Show All 267 Lines • ▼ Show 20 Lines let llvmBuilder = [{
llvm::LLVMContext &ctx = module->getContext(); llvm::LLVMContext &ctx = module->getContext();
llvm::Function *fn = llvm::Function *fn =
llvm::Intrinsic::getDeclaration(module, llvm::Intrinsic::}] llvm::Intrinsic::getDeclaration(module, llvm::Intrinsic::}]
# !subst(".", "_", name) # [{); # !subst(".", "_", name) # [{);
builder.CreateCall(fn, { builder.CreateCall(fn, {
llvm::MetadataAsValue::get(ctx, llvm::MetadataAsValue::get(ctx,
llvm::ValueAsMetadata::get(moduleTranslation.lookupValue(opInst.getOperand(0)))), llvm::ValueAsMetadata::get(moduleTranslation.lookupValue(opInst.getOperand(0)))),
llvm::MetadataAsValue::get(ctx, moduleTranslation.translateDebugInfo($varInfo)), llvm::MetadataAsValue::get(ctx, moduleTranslation.translateDebugInfo($varInfo)),
llvm::MetadataAsValue::get(ctx, llvm::DIExpression::get(ctx, llvm::None)), llvm::MetadataAsValue::get(ctx, llvm::DIExpression::get(ctx, std::nullopt)),
}); });
}]; }];
let mlirBuilder = [{ let mlirBuilder = [{
// Drop all debug intrinsics with a non-empty debug expression. // Drop all debug intrinsics with a non-empty debug expression.
// TODO: Stop dropping debug intrinsics once debug expressions are // TODO: Stop dropping debug intrinsics once debug expressions are
// convertible to MLIR. // convertible to MLIR.
auto *dbgIntr = cast<llvm::DbgVariableIntrinsic>(inst); auto *dbgIntr = cast<llvm::DbgVariableIntrinsic>(inst);
if (dbgIntr->getExpression()->getNumElements() == 0) if (dbgIntr->getExpression()->getNumElements() == 0)
▲ Show 20 Lines • Show All 592 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Linalg/Analysis/DependenceAnalysis.h

Show First 20 Lines • Show All 70 Lines • ▼ Show 20 Lines struct LinalgDependenceGraphElem {
} }
// Return the operand or the result Value represented by the `opView`. // Return the operand or the result Value represented by the `opView`.
static Value getValue(OpView opView) { static Value getValue(OpView opView) {
if (OpOperand *operand = opView.dyn_cast<OpOperand *>()) if (OpOperand *operand = opView.dyn_cast<OpOperand *>())
return operand->get(); return operand->get();
return opView.get<Value>(); return opView.get<Value>();
} }
// Return the indexing map of the operand/result in `opView` specified in // Return the indexing map of the operand/result in `opView` specified in
// the owning LinalgOp. If the owner is not a LinalgOp returns llvm::None. // the owning LinalgOp. If the owner is not a LinalgOp returns std::nullopt.
static Optional<AffineMap> getIndexingMap(OpView opView) { static Optional<AffineMap> getIndexingMap(OpView opView) {
auto owner = dyn_cast<LinalgOp>(getOwner(opView)); auto owner = dyn_cast<LinalgOp>(getOwner(opView));
if (!owner) if (!owner)
return llvm::None; return std::nullopt;
if (OpOperand *operand = opView.dyn_cast<OpOperand *>()) if (OpOperand *operand = opView.dyn_cast<OpOperand *>())
return owner.getMatchingIndexingMap(operand); return owner.getMatchingIndexingMap(operand);
return owner.getMatchingIndexingMap(owner.getDpsInitOperand( return owner.getMatchingIndexingMap(owner.getDpsInitOperand(
opView.get<Value>().cast<OpResult>().getResultNumber())); opView.get<Value>().cast<OpResult>().getResultNumber()));
} }
// Return the operand number if the `opView` is an OpOperand *. Otherwise // Return the operand number if the `opView` is an OpOperand *. Otherwise
// return llvm::None. // return std::nullopt.
static Optional<unsigned> getOperandNumber(OpView opView) { static Optional<unsigned> getOperandNumber(OpView opView) {
if (OpOperand *operand = opView.dyn_cast<OpOperand *>()) if (OpOperand *operand = opView.dyn_cast<OpOperand *>())
return operand->getOperandNumber(); return operand->getOperandNumber();
return llvm::None; return std::nullopt;
} }
// Return the result number if the `opView` is an OpResult. Otherwise return // Return the result number if the `opView` is an OpResult. Otherwise return
// llvm::None. // std::nullopt.
static Optional<unsigned> getResultNumber(OpView opView) { static Optional<unsigned> getResultNumber(OpView opView) {
if (OpResult result = opView.dyn_cast<Value>().cast<OpResult>()) if (OpResult result = opView.dyn_cast<Value>().cast<OpResult>())
return result.getResultNumber(); return result.getResultNumber();
return llvm::None; return std::nullopt;
} }
// Return the owner of the dependent OpView. // Return the owner of the dependent OpView.
Operation *getDependentOp() const { return getOwner(dependentOpView); } Operation *getDependentOp() const { return getOwner(dependentOpView); }
// Return the owner of the indexing OpView. // Return the owner of the indexing OpView.
Operation *getIndexingOp() const { return getOwner(indexingOpView); } Operation *getIndexingOp() const { return getOwner(indexingOpView); }
// Return the operand or result stored in the dependentOpView. // Return the operand or result stored in the dependentOpView.
Value getDependentValue() const { return getValue(dependentOpView); } Value getDependentValue() const { return getValue(dependentOpView); }
// Return the operand or result stored in the indexingOpView. // Return the operand or result stored in the indexingOpView.
Value getIndexingValue() const { return getValue(indexingOpView); } Value getIndexingValue() const { return getValue(indexingOpView); }
// If the dependent OpView is an operand, return operand number. Return // If the dependent OpView is an operand, return operand number. Return
// llvm::None otherwise. // std::nullopt otherwise.
Optional<unsigned> getDependentOpViewOperandNum() const { Optional<unsigned> getDependentOpViewOperandNum() const {
return getOperandNumber(dependentOpView); return getOperandNumber(dependentOpView);
} }
// If the indexing OpView is an operand, return operand number. Return // If the indexing OpView is an operand, return operand number. Return
// llvm::None otherwise. // std::nullopt otherwise.
Optional<unsigned> getIndexingOpViewOperandNum() const { Optional<unsigned> getIndexingOpViewOperandNum() const {
return getOperandNumber(indexingOpView); return getOperandNumber(indexingOpView);
} }
// If the dependent OpView is a result value, return the result // If the dependent OpView is a result value, return the result
// number. Return llvm::None otherwise. // number. Return std::nullopt otherwise.
Optional<unsigned> getDependentOpViewResultNum() const { Optional<unsigned> getDependentOpViewResultNum() const {
return getResultNumber(dependentOpView); return getResultNumber(dependentOpView);
} }
// If the dependent OpView is a result value, return the result // If the dependent OpView is a result value, return the result
// number. Return llvm::None otherwise. // number. Return std::nullopt otherwise.
Optional<unsigned> getIndexingOpViewResultNum() const { Optional<unsigned> getIndexingOpViewResultNum() const {
return getResultNumber(indexingOpView); return getResultNumber(indexingOpView);
} }
// Return the indexing map of the operand/result in the dependent OpView as // Return the indexing map of the operand/result in the dependent OpView as
// specified in the owner of the OpView. // specified in the owner of the OpView.
Optional<AffineMap> getDependentOpViewIndexingMap() const { Optional<AffineMap> getDependentOpViewIndexingMap() const {
return getIndexingMap(dependentOpView); return getIndexingMap(dependentOpView);
▲ Show 20 Lines • Show All 131 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Linalg/Transforms/Transforms.h

Show First 20 Lines • Show All 178 Lines • ▼ Show 20 Lines
/// and permute the loop nest according to `interchangeVector` /// and permute the loop nest according to `interchangeVector`
/// The permutation is expressed as a list of integers that specify /// The permutation is expressed as a list of integers that specify
/// the new ordering of the loop nest. The length of `interchangeVector` /// the new ordering of the loop nest. The length of `interchangeVector`
/// must be equal to the length of `tileSizes`. /// must be equal to the length of `tileSizes`.
/// An empty vector is interpreted as the identity permutation and the /// An empty vector is interpreted as the identity permutation and the
/// transformation returns early. /// transformation returns early.
/// ///
/// Return a struct containing the tiled loops in the specified order /// Return a struct containing the tiled loops in the specified order
/// and the cloned op if successful, llvm::None otherwise. /// and the cloned op if successful, std::nullopt otherwise.
/// ///
/// E.g. the permutation `(i,j,k) -> (j,k,i)` is expressed by /// E.g. the permutation `(i,j,k) -> (j,k,i)` is expressed by
/// `interchangeVector = [1,2,0]`. All values in `interchangeVector` must be /// `interchangeVector = [1,2,0]`. All values in `interchangeVector` must be
/// integers, in the range 0..`tileSizes.size()` without duplications /// integers, in the range 0..`tileSizes.size()` without duplications
/// (i.e. `[1,1,2]` is an invalid permutation). /// (i.e. `[1,1,2]` is an invalid permutation).
struct TiledLinalgOp { struct TiledLinalgOp {
LinalgOp op; LinalgOp op;
SmallVector<Operation *, 8> loops; SmallVector<Operation *, 8> loops;
▲ Show 20 Lines • Show All 886 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Linalg/Utils/Utils.h

Show First 20 Lines • Show All 132 Lines • ▼ Show 20 Lines
/// rank-reduced version. This is only done for the cases where the size is 1 /// rank-reduced version. This is only done for the cases where the size is 1
/// and offset is 0. Strictly speaking the offset 0 is not required in general, /// and offset is 0. Strictly speaking the offset 0 is not required in general,
/// but non-zero offsets are not handled by SPIR-V backend at this point (and /// but non-zero offsets are not handled by SPIR-V backend at this point (and
/// potentially cannot be handled). /// potentially cannot be handled).
Optional<SmallVector<ReassociationIndices>> Optional<SmallVector<ReassociationIndices>>
getReassociationMapForFoldingUnitDims(ArrayRef<OpFoldResult> mixedSizes); getReassociationMapForFoldingUnitDims(ArrayRef<OpFoldResult> mixedSizes);
/// Return the identity numeric value associated to the give op. Return /// Return the identity numeric value associated to the give op. Return
/// llvm::None if there is no known neutral element. /// std::nullopt if there is no known neutral element.
Optional<Attribute> getNeutralElement(Operation *op); Optional<Attribute> getNeutralElement(Operation *op);
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Fusion / Tiling utilities // Fusion / Tiling utilities
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
/// The type of loops to be generated during tiling. /// The type of loops to be generated during tiling.
enum class LinalgTilingLoopType { enum class LinalgTilingLoopType {
▲ Show 20 Lines • Show All 67 Lines • ▼ Show 20 Lines
/// assuming `linalgOp` is being fused into a loop nest. Calls /// assuming `linalgOp` is being fused into a loop nest. Calls
/// `computeSliceParameters` for every individual value. /// `computeSliceParameters` for every individual value.
/// ///
/// Note that a constant zero in `tileSizes` means no tiling at that implicit /// Note that a constant zero in `tileSizes` means no tiling at that implicit
/// loop. The number of non-zero values in `tileSizes` should be equal to the /// loop. The number of non-zero values in `tileSizes` should be equal to the
/// number of values in `ivs`. /// number of values in `ivs`.
/// ///
/// Some of the `valuesToTile` won't be affected by tiling. For these values, /// Some of the `valuesToTile` won't be affected by tiling. For these values,
/// llvm::None will be returned. /// std::nullopt will be returned.
SmallVector<Optional<SliceParameters>> SmallVector<Optional<SliceParameters>>
computeAllSliceParameters(OpBuilder &builder, Location loc, LinalgOp linalgOp, computeAllSliceParameters(OpBuilder &builder, Location loc, LinalgOp linalgOp,
ValueRange valuesToTile, ArrayRef<OpFoldResult> ivs, ValueRange valuesToTile, ArrayRef<OpFoldResult> ivs,
ArrayRef<OpFoldResult> tileSizes, ArrayRef<OpFoldResult> tileSizes,
ArrayRef<OpFoldResult> sizeBounds, ArrayRef<OpFoldResult> sizeBounds,
bool omitPartialTileCheck); bool omitPartialTileCheck);
/// Creates an extract_slice/subview op for a single `valueToTile` with /// Creates an extract_slice/subview op for a single `valueToTile` with
▲ Show 20 Lines • Show All 278 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/MLProgram/IR/MLProgramOps.td

Show First 20 Lines • Show All 451 Lines • ▼ Show 20 Lines let description = [{
The operation takes variable number of operands and produces no results. The operation takes variable number of operands and produces no results.
The operand number and types must match the signature of the function The operand number and types must match the signature of the function
that contains the operation. that contains the operation.
}]; }];
let arguments = (ins Variadic<AnyType>:$operands); let arguments = (ins Variadic<AnyType>:$operands);
let builders = [OpBuilder<(ins), [{ let builders = [OpBuilder<(ins), [{
build($_builder, $_state, llvm::None); build($_builder, $_state, std::nullopt);
}]>]; }]>];
let assemblyFormat = "attr-dict ($operands^ `:` type($operands))?"; let assemblyFormat = "attr-dict ($operands^ `:` type($operands))?";
let hasVerifier = 1; let hasVerifier = 1;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// ReturnOp // ReturnOp
Show All 9 Lines let description = [{
The operation takes variable number of operands and produces no results. The operation takes variable number of operands and produces no results.
The operand number and types must match the signature of the function The operand number and types must match the signature of the function
that contains the operation. that contains the operation.
}]; }];
let arguments = (ins Variadic<AnyType>:$operands); let arguments = (ins Variadic<AnyType>:$operands);
let builders = [OpBuilder<(ins), [{ let builders = [OpBuilder<(ins), [{
build($_builder, $_state, llvm::None); build($_builder, $_state, std::nullopt);
}]>]; }]>];
let assemblyFormat = "attr-dict ($operands^ `:` type($operands))?"; let assemblyFormat = "attr-dict ($operands^ `:` type($operands))?";
let hasVerifier = 1; let hasVerifier = 1;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// TokenOp // TokenOp
Show All 19 Lines

mlir/include/mlir/Dialect/PDL/IR/PDLOps.td

Show First 20 Lines • Show All 348 Lines • ▼ Show 20 Linesdef PDL_OperationOp : PDL_Op<"operation", [AttrSizedOperandSegments]> {
let results = (outs PDL_Operation:$op); let results = (outs PDL_Operation:$op);
let assemblyFormat = [{ let assemblyFormat = [{
($opName^)? (`(` $operandValues^ `:` type($operandValues) `)`)? ($opName^)? (`(` $operandValues^ `:` type($operandValues) `)`)?
custom<OperationOpAttributes>($attributeValues, $attributeValueNames) custom<OperationOpAttributes>($attributeValues, $attributeValueNames)
(`->` `(` $typeValues^ `:` type($typeValues) `)`)? attr-dict (`->` `(` $typeValues^ `:` type($typeValues) `)`)? attr-dict
}]; }];
let builders = [ let builders = [
OpBuilder<(ins CArg<"Optional<StringRef>", "llvm::None">:$name, OpBuilder<(ins CArg<"Optional<StringRef>", "std::nullopt">:$name,
CArg<"ValueRange", "llvm::None">:$operandValues, CArg<"ValueRange", "std::nullopt">:$operandValues,
CArg<"ArrayRef<StringRef>", "llvm::None">:$attrNames, CArg<"ArrayRef<StringRef>", "std::nullopt">:$attrNames,
CArg<"ValueRange", "llvm::None">:$attrValues, CArg<"ValueRange", "std::nullopt">:$attrValues,
CArg<"ValueRange", "llvm::None">:$resultTypes), [{ CArg<"ValueRange", "std::nullopt">:$resultTypes), [{
auto nameAttr = name ? $_builder.getStringAttr(*name) : StringAttr(); auto nameAttr = name ? $_builder.getStringAttr(*name) : StringAttr();
build($_builder, $_state, $_builder.getType<OperationType>(), nameAttr, build($_builder, $_state, $_builder.getType<OperationType>(), nameAttr,
operandValues, attrValues, $_builder.getStrArrayAttr(attrNames), operandValues, attrValues, $_builder.getStrArrayAttr(attrNames),
resultTypes); resultTypes);
}]>, }]>,
]; ];
let extraClassDeclaration = [{ let extraClassDeclaration = [{
/// Returns true if the operation type referenced supports result type /// Returns true if the operation type referenced supports result type
▲ Show 20 Lines • Show All 45 Lines • ▼ Show 20 Lines let arguments = (ins ConfinedAttr<I16Attr, [IntNonNegative]>:$benefit,
OptionalAttr<SymbolNameAttr>:$sym_name); OptionalAttr<SymbolNameAttr>:$sym_name);
let regions = (region SizedRegion<1>:$bodyRegion); let regions = (region SizedRegion<1>:$bodyRegion);
let assemblyFormat = [{ let assemblyFormat = [{
($sym_name^)? `:` `benefit` `(` $benefit `)` attr-dict-with-keyword $bodyRegion ($sym_name^)? `:` `benefit` `(` $benefit `)` attr-dict-with-keyword $bodyRegion
}]; }];
let builders = [ let builders = [
OpBuilder<(ins CArg<"Optional<uint16_t>", "1">:$benefit, OpBuilder<(ins CArg<"Optional<uint16_t>", "1">:$benefit,
CArg<"Optional<StringRef>", "llvm::None">:$name)>, CArg<"Optional<StringRef>", "std::nullopt">:$name)>,
]; ];
let extraClassDeclaration = [{ let extraClassDeclaration = [{
//===------------------------------------------------------------------===// //===------------------------------------------------------------------===//
// SymbolOpInterface Methods // SymbolOpInterface Methods
//===------------------------------------------------------------------===// //===------------------------------------------------------------------===//
/// A PatternOp may optionally define a symbol. /// A PatternOp may optionally define a symbol.
bool isOptionalSymbol() { return true; } bool isOptionalSymbol() { return true; }
▲ Show 20 Lines • Show All 284 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/SCF/IR/SCFOps.td

Show First 20 Lines • Show All 216 Lines • ▼ Show 20 Lines let arguments = (ins Index:$lowerBound,
Index:$step, Index:$step,
Variadic<AnyType>:$initArgs); Variadic<AnyType>:$initArgs);
let results = (outs Variadic<AnyType>:$results); let results = (outs Variadic<AnyType>:$results);
let regions = (region SizedRegion<1>:$region); let regions = (region SizedRegion<1>:$region);
let skipDefaultBuilders = 1; let skipDefaultBuilders = 1;
let builders = [ let builders = [
OpBuilder<(ins "Value":$lowerBound, "Value":$upperBound, "Value":$step, OpBuilder<(ins "Value":$lowerBound, "Value":$upperBound, "Value":$step,
CArg<"ValueRange", "llvm::None">:$iterArgs, CArg<"ValueRange", "std::nullopt">:$iterArgs,
CArg<"function_ref<void(OpBuilder &, Location, Value, ValueRange)>", CArg<"function_ref<void(OpBuilder &, Location, Value, ValueRange)>",
"nullptr">)> "nullptr">)>
]; ];
let extraClassDeclaration = [{ let extraClassDeclaration = [{
using BodyBuilderFn = using BodyBuilderFn =
function_ref<void(OpBuilder &, Location, Value, ValueRange)>; function_ref<void(OpBuilder &, Location, Value, ValueRange)>;
Show All 34 Lines let extraClassDeclaration = [{
bool hasIterOperands() { bool hasIterOperands() {
return getOperation()->getNumOperands() > getNumControlOperands(); return getOperation()->getNumOperands() > getNumControlOperands();
} }
/// Get Number of loop-carried values /// Get Number of loop-carried values
unsigned getNumIterOperands() { unsigned getNumIterOperands() {
return getOperation()->getNumOperands() - getNumControlOperands(); return getOperation()->getNumOperands() - getNumControlOperands();
} }
/// Get the iter arg number for an operand. If it isnt an iter arg /// Get the iter arg number for an operand. If it isnt an iter arg
/// operand return llvm::None. /// operand return std::nullopt.
Optional<unsigned> getIterArgNumberForOpOperand(OpOperand &opOperand) { Optional<unsigned> getIterArgNumberForOpOperand(OpOperand &opOperand) {
if (opOperand.getOwner() != getOperation()) if (opOperand.getOwner() != getOperation())
return llvm::None; return std::nullopt;
unsigned operandNumber = opOperand.getOperandNumber(); unsigned operandNumber = opOperand.getOperandNumber();
if (operandNumber < getNumControlOperands()) if (operandNumber < getNumControlOperands())
return llvm::None; return std::nullopt;
return operandNumber - getNumControlOperands(); return operandNumber - getNumControlOperands();
} }
/// Get the region iter arg that corresponds to an OpOperand. /// Get the region iter arg that corresponds to an OpOperand.
/// This helper prevents internal op implementation detail leakage to /// This helper prevents internal op implementation detail leakage to
/// clients by hiding the operand / block argument mapping. /// clients by hiding the operand / block argument mapping.
BlockArgument getRegionIterArgForOpOperand(OpOperand &opOperand) { BlockArgument getRegionIterArgForOpOperand(OpOperand &opOperand) {
assert(opOperand.getOperandNumber() >= getNumControlOperands() && assert(opOperand.getOperandNumber() >= getNumControlOperands() &&
▲ Show 20 Lines • Show All 819 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/SPIRV/IR/SPIRVAvailability.td

Show First 20 Lines • Show All 62 Lines • ▼ Show 20 Linesclass MinVersionBase<string name, EnumAttr scheme, I32EnumAttrCase min>
let queryFnRetType = "llvm::Optional<" # scheme.returnType # ">"; let queryFnRetType = "llvm::Optional<" # scheme.returnType # ">";
let queryFnName = "getMinVersion"; let queryFnName = "getMinVersion";
let mergeAction = "{ " let mergeAction = "{ "
"if ($overall) { " "if ($overall) { "
"$overall = static_cast<" # scheme.returnType # ">(" "$overall = static_cast<" # scheme.returnType # ">("
"std::max(*$overall, $instance)); " "std::max(*$overall, $instance)); "
"} else { $overall = $instance; }}"; "} else { $overall = $instance; }}";
let initializer = "::llvm::None"; let initializer = "::std::nullopt";
let instanceType = scheme.cppNamespace # "::" # scheme.enum.className; let instanceType = scheme.cppNamespace # "::" # scheme.enum.className;
let instance = scheme.cppNamespace # "::" # scheme.enum.className # "::" # let instance = scheme.cppNamespace # "::" # scheme.enum.className # "::" #
min.symbol; min.symbol;
} }
class MaxVersionBase<string name, EnumAttr scheme, I32EnumAttrCase max> class MaxVersionBase<string name, EnumAttr scheme, I32EnumAttrCase max>
: Availability { : Availability {
let interfaceName = name; let interfaceName = name;
let queryFnRetType = "llvm::Optional<" # scheme.returnType # ">"; let queryFnRetType = "llvm::Optional<" # scheme.returnType # ">";
let queryFnName = "getMaxVersion"; let queryFnName = "getMaxVersion";
let mergeAction = "{ " let mergeAction = "{ "
"if ($overall) { " "if ($overall) { "
"$overall = static_cast<" # scheme.returnType # ">(" "$overall = static_cast<" # scheme.returnType # ">("
"std::min(*$overall, $instance)); " "std::min(*$overall, $instance)); "
"} else { $overall = $instance; }}"; "} else { $overall = $instance; }}";
let initializer = "::llvm::None"; let initializer = "::std::nullopt";
let instanceType = scheme.cppNamespace # "::" # scheme.enum.className; let instanceType = scheme.cppNamespace # "::" # scheme.enum.className;
let instance = scheme.cppNamespace # "::" # scheme.enum.className # "::" # let instance = scheme.cppNamespace # "::" # scheme.enum.className # "::" #
max.symbol; max.symbol;
} }
#endif // MLIR_DIALECT_SPIRV_IR_AVAILABILITY #endif // MLIR_DIALECT_SPIRV_IR_AVAILABILITY

mlir/include/mlir/Dialect/SPIRV/IR/SPIRVStructureOps.td

Show First 20 Lines • Show All 484 Lines • ▼ Show 20 Lines let arguments = (ins
OptionalAttr<StrAttr>:$sym_name OptionalAttr<StrAttr>:$sym_name
); );
let results = (outs); let results = (outs);
let regions = (region AnyRegion); let regions = (region AnyRegion);
let builders = [ let builders = [
OpBuilder<(ins CArg<"Optional<StringRef>", "llvm::None">:$name)>, OpBuilder<(ins CArg<"Optional<StringRef>", "std::nullopt">:$name)>,
OpBuilder<(ins "spirv::AddressingModel":$addressing_model, OpBuilder<(ins "spirv::AddressingModel":$addressing_model,
"spirv::MemoryModel":$memory_model, "spirv::MemoryModel":$memory_model,
CArg<"Optional<spirv::VerCapExtAttr>", "llvm::None">:$vce_triple, CArg<"Optional<spirv::VerCapExtAttr>", "std::nullopt">:$vce_triple,
CArg<"Optional<StringRef>", "llvm::None">:$name)> CArg<"Optional<StringRef>", "std::nullopt">:$name)>
]; ];
// We need to ensure the block inside the region is properly terminated; // We need to ensure the block inside the region is properly terminated;
// the auto-generated builders do not guarantee that. // the auto-generated builders do not guarantee that.
let skipDefaultBuilders = 1; let skipDefaultBuilders = 1;
let hasOpcode = 0; let hasOpcode = 0;
▲ Show 20 Lines • Show All 297 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/SPIRV/IR/SPIRVTypes.h

Show First 20 Lines • Show All 50 Lines • ▼ Show 20 Linespublic:
/// ((Extension::A OR Extension::B) AND (Extension::C OR Extension::D)) /// ((Extension::A OR Extension::B) AND (Extension::C OR Extension::D))
/// convention. /// convention.
using ExtensionArrayRefVector = SmallVectorImpl<ArrayRef<Extension>>; using ExtensionArrayRefVector = SmallVectorImpl<ArrayRef<Extension>>;
/// Appends to `extensions` the extensions needed for this type to appear in /// Appends to `extensions` the extensions needed for this type to appear in
/// the given `storage` class. This method does not guarantee the uniqueness /// the given `storage` class. This method does not guarantee the uniqueness
/// of extensions; the same extension may be appended multiple times. /// of extensions; the same extension may be appended multiple times.
void getExtensions(ExtensionArrayRefVector &extensions, void getExtensions(ExtensionArrayRefVector &extensions,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
/// The capability requirements for each type are following the /// The capability requirements for each type are following the
/// ((Capability::A OR Extension::B) AND (Capability::C OR Capability::D)) /// ((Capability::A OR Extension::B) AND (Capability::C OR Capability::D))
/// convention. /// convention.
using CapabilityArrayRefVector = SmallVectorImpl<ArrayRef<Capability>>; using CapabilityArrayRefVector = SmallVectorImpl<ArrayRef<Capability>>;
/// Appends to `capabilities` the capabilities needed for this type to appear /// Appends to `capabilities` the capabilities needed for this type to appear
/// in the given `storage` class. This method does not guarantee the /// in the given `storage` class. This method does not guarantee the
/// uniqueness of capabilities; the same capability may be appended multiple /// uniqueness of capabilities; the same capability may be appended multiple
/// times. /// times.
void getCapabilities(CapabilityArrayRefVector &capabilities, void getCapabilities(CapabilityArrayRefVector &capabilities,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
/// Returns the size in bytes for each type. If no size can be calculated, /// Returns the size in bytes for each type. If no size can be calculated,
/// returns `llvm::None`. Note that if the type has explicit layout, it is /// returns `std::nullopt`. Note that if the type has explicit layout, it is
/// also taken into account in calculation. /// also taken into account in calculation.
Optional<int64_t> getSizeInBytes(); Optional<int64_t> getSizeInBytes();
}; };
// SPIR-V scalar type: bool type, integer type, floating point type. // SPIR-V scalar type: bool type, integer type, floating point type.
class ScalarType : public SPIRVType { class ScalarType : public SPIRVType {
public: public:
using SPIRVType::SPIRVType; using SPIRVType::SPIRVType;
static bool classof(Type type); static bool classof(Type type);
/// Returns true if the given integer type is valid for the SPIR-V dialect. /// Returns true if the given integer type is valid for the SPIR-V dialect.
static bool isValid(FloatType); static bool isValid(FloatType);
/// Returns true if the given float type is valid for the SPIR-V dialect. /// Returns true if the given float type is valid for the SPIR-V dialect.
static bool isValid(IntegerType); static bool isValid(IntegerType);
void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions, void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities, void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
Optional<int64_t> getSizeInBytes(); Optional<int64_t> getSizeInBytes();
}; };
// SPIR-V composite type: VectorType, SPIR-V ArrayType, or SPIR-V StructType. // SPIR-V composite type: VectorType, SPIR-V ArrayType, or SPIR-V StructType.
class CompositeType : public SPIRVType { class CompositeType : public SPIRVType {
public: public:
using SPIRVType::SPIRVType; using SPIRVType::SPIRVType;
Show All 9 Linespublic:
Type getElementType(unsigned) const; Type getElementType(unsigned) const;
/// Return true if the number of elements is known at compile time and is not /// Return true if the number of elements is known at compile time and is not
/// implementation dependent. /// implementation dependent.
bool hasCompileTimeKnownNumElements() const; bool hasCompileTimeKnownNumElements() const;
void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions, void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities, void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
Optional<int64_t> getSizeInBytes(); Optional<int64_t> getSizeInBytes();
}; };
// SPIR-V array type // SPIR-V array type
class ArrayType : public Type::TypeBase<ArrayType, CompositeType, class ArrayType : public Type::TypeBase<ArrayType, CompositeType,
detail::ArrayTypeStorage> { detail::ArrayTypeStorage> {
public: public:
Show All 9 Linespublic:
Type getElementType() const; Type getElementType() const;
/// Returns the array stride in bytes. 0 means no stride decorated on this /// Returns the array stride in bytes. 0 means no stride decorated on this
/// type. /// type.
unsigned getArrayStride() const; unsigned getArrayStride() const;
void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions, void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities, void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
/// Returns the array size in bytes. Since array type may have an explicit /// Returns the array size in bytes. Since array type may have an explicit
/// stride declaration (in bytes), we also include it in the calculation. /// stride declaration (in bytes), we also include it in the calculation.
Optional<int64_t> getSizeInBytes(); Optional<int64_t> getSizeInBytes();
}; };
// SPIR-V image type // SPIR-V image type
class ImageType class ImageType
Show All 24 Linespublic:
ImageDepthInfo getDepthInfo() const; ImageDepthInfo getDepthInfo() const;
ImageArrayedInfo getArrayedInfo() const; ImageArrayedInfo getArrayedInfo() const;
ImageSamplingInfo getSamplingInfo() const; ImageSamplingInfo getSamplingInfo() const;
ImageSamplerUseInfo getSamplerUseInfo() const; ImageSamplerUseInfo getSamplerUseInfo() const;
ImageFormat getImageFormat() const; ImageFormat getImageFormat() const;
// TODO: Add support for Access qualifier // TODO: Add support for Access qualifier
void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions, void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities, void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
}; };
// SPIR-V pointer type // SPIR-V pointer type
class PointerType : public Type::TypeBase<PointerType, SPIRVType, class PointerType : public Type::TypeBase<PointerType, SPIRVType,
detail::PointerTypeStorage> { detail::PointerTypeStorage> {
public: public:
using Base::Base; using Base::Base;
static PointerType get(Type pointeeType, StorageClass storageClass); static PointerType get(Type pointeeType, StorageClass storageClass);
Type getPointeeType() const; Type getPointeeType() const;
StorageClass getStorageClass() const; StorageClass getStorageClass() const;
void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions, void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities, void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
}; };
// SPIR-V run-time array type // SPIR-V run-time array type
class RuntimeArrayType class RuntimeArrayType
: public Type::TypeBase<RuntimeArrayType, SPIRVType, : public Type::TypeBase<RuntimeArrayType, SPIRVType,
detail::RuntimeArrayTypeStorage> { detail::RuntimeArrayTypeStorage> {
public: public:
using Base::Base; using Base::Base;
static RuntimeArrayType get(Type elementType); static RuntimeArrayType get(Type elementType);
/// Returns a runtime array type with the given stride in bytes. /// Returns a runtime array type with the given stride in bytes.
static RuntimeArrayType get(Type elementType, unsigned stride); static RuntimeArrayType get(Type elementType, unsigned stride);
Type getElementType() const; Type getElementType() const;
/// Returns the array stride in bytes. 0 means no stride decorated on this /// Returns the array stride in bytes. 0 means no stride decorated on this
/// type. /// type.
unsigned getArrayStride() const; unsigned getArrayStride() const;
void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions, void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities, void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
}; };
// SPIR-V sampled image type // SPIR-V sampled image type
class SampledImageType class SampledImageType
: public Type::TypeBase<SampledImageType, SPIRVType, : public Type::TypeBase<SampledImageType, SPIRVType,
detail::SampledImageTypeStorage> { detail::SampledImageTypeStorage> {
public: public:
using Base::Base; using Base::Base;
static SampledImageType get(Type imageType); static SampledImageType get(Type imageType);
static SampledImageType static SampledImageType
getChecked(function_ref<InFlightDiagnostic()> emitError, Type imageType); getChecked(function_ref<InFlightDiagnostic()> emitError, Type imageType);
static LogicalResult verify(function_ref<InFlightDiagnostic()> emitError, static LogicalResult verify(function_ref<InFlightDiagnostic()> emitError,
Type imageType); Type imageType);
Type getImageType() const; Type getImageType() const;
void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions, void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions,
Optional<spirv::StorageClass> storage = llvm::None); Optional<spirv::StorageClass> storage = std::nullopt);
void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities, void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities,
Optional<spirv::StorageClass> storage = llvm::None); Optional<spirv::StorageClass> storage = std::nullopt);
}; };
/// SPIR-V struct type. Two kinds of struct types are supported: /// SPIR-V struct type. Two kinds of struct types are supported:
/// - Literal: a literal struct type is uniqued by its fields (types + offset /// - Literal: a literal struct type is uniqued by its fields (types + offset
/// info + decoration info). /// info + decoration info).
/// - Identified: an indentified struct type is uniqued by its string identifier /// - Identified: an indentified struct type is uniqued by its string identifier
/// (name). This is useful in representing recursive structs. For example, the /// (name). This is useful in representing recursive structs. For example, the
/// following C struct: /// following C struct:
▲ Show 20 Lines • Show All 117 Lines • ▼ Show 20 Linespublic:
/// Sets the contents of an incomplete identified StructType. This method must /// Sets the contents of an incomplete identified StructType. This method must
/// be called only for identified StructTypes and it must be called only once /// be called only for identified StructTypes and it must be called only once
/// per instance. Otherwise, failure() is returned. /// per instance. Otherwise, failure() is returned.
LogicalResult LogicalResult
trySetBody(ArrayRef<Type> memberTypes, ArrayRef<OffsetInfo> offsetInfo = {}, trySetBody(ArrayRef<Type> memberTypes, ArrayRef<OffsetInfo> offsetInfo = {},
ArrayRef<MemberDecorationInfo> memberDecorations = {}); ArrayRef<MemberDecorationInfo> memberDecorations = {});
void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions, void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities, void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
}; };
llvm::hash_code llvm::hash_code
hash_value(const StructType::MemberDecorationInfo &memberDecorationInfo); hash_value(const StructType::MemberDecorationInfo &memberDecorationInfo);
// SPIR-V cooperative matrix type // SPIR-V cooperative matrix type
class CooperativeMatrixNVType class CooperativeMatrixNVType
: public Type::TypeBase<CooperativeMatrixNVType, CompositeType, : public Type::TypeBase<CooperativeMatrixNVType, CompositeType,
detail::CooperativeMatrixTypeStorage> { detail::CooperativeMatrixTypeStorage> {
public: public:
using Base::Base; using Base::Base;
static CooperativeMatrixNVType get(Type elementType, Scope scope, static CooperativeMatrixNVType get(Type elementType, Scope scope,
unsigned rows, unsigned columns); unsigned rows, unsigned columns);
Type getElementType() const; Type getElementType() const;
/// Return the scope of the cooperative matrix. /// Return the scope of the cooperative matrix.
Scope getScope() const; Scope getScope() const;
/// return the number of rows of the matrix. /// return the number of rows of the matrix.
unsigned getRows() const; unsigned getRows() const;
/// return the number of columns of the matrix. /// return the number of columns of the matrix.
unsigned getColumns() const; unsigned getColumns() const;
void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions, void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities, void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
}; };
// SPIR-V joint matrix type // SPIR-V joint matrix type
class JointMatrixINTELType class JointMatrixINTELType
: public Type::TypeBase<JointMatrixINTELType, CompositeType, : public Type::TypeBase<JointMatrixINTELType, CompositeType,
detail::JointMatrixTypeStorage> { detail::JointMatrixTypeStorage> {
public: public:
using Base::Base; using Base::Base;
static JointMatrixINTELType get(Type elementType, Scope scope, unsigned rows, static JointMatrixINTELType get(Type elementType, Scope scope, unsigned rows,
unsigned columns, MatrixLayout matrixLayout); unsigned columns, MatrixLayout matrixLayout);
Type getElementType() const; Type getElementType() const;
/// Return the scope of the joint matrix. /// Return the scope of the joint matrix.
Scope getScope() const; Scope getScope() const;
/// return the number of rows of the matrix. /// return the number of rows of the matrix.
unsigned getRows() const; unsigned getRows() const;
/// return the number of columns of the matrix. /// return the number of columns of the matrix.
unsigned getColumns() const; unsigned getColumns() const;
/// return the layout of the matrix /// return the layout of the matrix
MatrixLayout getMatrixLayout() const; MatrixLayout getMatrixLayout() const;
void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions, void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities, void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
}; };
// SPIR-V matrix type // SPIR-V matrix type
class MatrixType : public Type::TypeBase<MatrixType, CompositeType, class MatrixType : public Type::TypeBase<MatrixType, CompositeType,
detail::MatrixTypeStorage> { detail::MatrixTypeStorage> {
public: public:
using Base::Base; using Base::Base;
Show All 18 Linespublic:
/// Returns total number of elements (rows*columns). /// Returns total number of elements (rows*columns).
unsigned getNumElements() const; unsigned getNumElements() const;
/// Returns the elements' type (i.e, single element type). /// Returns the elements' type (i.e, single element type).
Type getElementType() const; Type getElementType() const;
void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions, void getExtensions(SPIRVType::ExtensionArrayRefVector &extensions,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities, void getCapabilities(SPIRVType::CapabilityArrayRefVector &capabilities,
Optional<StorageClass> storage = llvm::None); Optional<StorageClass> storage = std::nullopt);
}; };
} // namespace spirv } // namespace spirv
} // namespace mlir } // namespace mlir
#endif // MLIR_DIALECT_SPIRV_IR_SPIRVTYPES_H_ #endif // MLIR_DIALECT_SPIRV_IR_SPIRVTYPES_H_

mlir/include/mlir/Dialect/SPIRV/IR/TargetAndABI.h

Show All 28 Lines
public: public:
explicit TargetEnv(TargetEnvAttr targetAttr); explicit TargetEnv(TargetEnvAttr targetAttr);
Version getVersion() const; Version getVersion() const;
/// Returns true if the given capability is allowed. /// Returns true if the given capability is allowed.
bool allows(Capability) const; bool allows(Capability) const;
/// Returns the first allowed one if any of the given capabilities is allowed. /// Returns the first allowed one if any of the given capabilities is allowed.
/// Returns llvm::None otherwise. /// Returns std::nullopt otherwise.
Optional<Capability> allows(ArrayRef<Capability>) const; Optional<Capability> allows(ArrayRef<Capability>) const;
/// Returns true if the given extension is allowed. /// Returns true if the given extension is allowed.
bool allows(Extension) const; bool allows(Extension) const;
/// Returns the first allowed one if any of the given extensions is allowed. /// Returns the first allowed one if any of the given extensions is allowed.
/// Returns llvm::None otherwise. /// Returns std::nullopt otherwise.
Optional<Extension> allows(ArrayRef<Extension>) const; Optional<Extension> allows(ArrayRef<Extension>) const;
/// Returns the vendor ID. /// Returns the vendor ID.
Vendor getVendorID() const; Vendor getVendorID() const;
/// Returns the device type. /// Returns the device type.
DeviceType getDeviceType() const; DeviceType getDeviceType() const;
▲ Show 20 Lines • Show All 84 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Shape/IR/ShapeOps.td

Show First 20 Lines • Show All 746 Lines • ▼ Show 20 Linesdef Shape_YieldOp : Shape_Op<"yield",
Pure, Pure,
ReturnLike, ReturnLike,
Terminator]> { Terminator]> {
let summary = "Returns the value to parent op"; let summary = "Returns the value to parent op";
let arguments = (ins Variadic<AnyType>:$operands); let arguments = (ins Variadic<AnyType>:$operands);
let builders = [OpBuilder<(ins), let builders = [OpBuilder<(ins),
[{ build($_builder, $_state, llvm::None); }]> [{ build($_builder, $_state, std::nullopt); }]>
]; ];
let assemblyFormat = "attr-dict ($operands^ `:` type($operands))?"; let assemblyFormat = "attr-dict ($operands^ `:` type($operands))?";
let hasVerifier = 1; let hasVerifier = 1;
} }
// TODO: Add Ops: if_static, if_ranked // TODO: Add Ops: if_static, if_ranked
▲ Show 20 Lines • Show All 419 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/SparseTensor/Utils/Merger.h

Show First 20 Lines • Show All 168 Lines • ▼ Show 20 Linespublic:
/// by avoiding scanning the full stored index sparse (keeping the last /// by avoiding scanning the full stored index sparse (keeping the last
/// position in ordered list) or even apply binary search to find the index. /// position in ordered list) or even apply binary search to find the index.
Merger(unsigned t, unsigned l, unsigned fl) Merger(unsigned t, unsigned l, unsigned fl)
: outTensor(t - 1), syntheticTensor(t), numTensors(t + 1), : outTensor(t - 1), syntheticTensor(t), numTensors(t + 1),
numNativeLoops(l), numLoops(l + fl), hasSparseOut(false), numNativeLoops(l), numLoops(l + fl), hasSparseOut(false),
dimTypes(numTensors, dimTypes(numTensors,
std::vector<DimLevelType>(numLoops, DimLevelType::Undef)), std::vector<DimLevelType>(numLoops, DimLevelType::Undef)),
loopIdxToDim(numTensors, loopIdxToDim(numTensors,
std::vector<Optional<unsigned>>(numLoops, llvm::None)), std::vector<Optional<unsigned>>(numLoops, std::nullopt)),
dimToLoopIdx(numTensors, dimToLoopIdx(numTensors,
std::vector<Optional<unsigned>>(numLoops, llvm::None)) {} std::vector<Optional<unsigned>>(numLoops, std::nullopt)) {}
/// Adds a tensor expression. Returns its index. /// Adds a tensor expression. Returns its index.
unsigned addExp(Kind k, unsigned e0, unsigned e1 = -1u, Value v = Value(), unsigned addExp(Kind k, unsigned e0, unsigned e1 = -1u, Value v = Value(),
Operation *op = nullptr); Operation *op = nullptr);
unsigned addExp(Kind k, unsigned e, Value v, Operation *op = nullptr) { unsigned addExp(Kind k, unsigned e, Value v, Operation *op = nullptr) {
return addExp(k, e, -1u, v, op); return addExp(k, e, -1u, v, op);
} }
unsigned addExp(Kind k, Value v, Operation *op = nullptr) { unsigned addExp(Kind k, Value v, Operation *op = nullptr) {
▲ Show 20 Lines • Show All 208 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Tensor/IR/TensorOps.td

Show First 20 Lines • Show All 1,759 Lines • ▼ Show 20 Lines let assemblyFormat = [{
custom<DynamicIndexList>($inner_tiles, $static_inner_tiles) custom<DynamicIndexList>($inner_tiles, $static_inner_tiles)
`into` $dest attr-dict `:` type($source) `->` type($dest) `into` $dest attr-dict `:` type($source) `->` type($dest)
}]; }];
let builders = [ let builders = [
OpBuilder<(ins "Value":$source, "Value":$dest, OpBuilder<(ins "Value":$source, "Value":$dest,
"ArrayRef<int64_t>":$innerDimsPos, "ArrayRef<int64_t>":$innerDimsPos,
"ArrayRef<OpFoldResult>":$innerTiles, "ArrayRef<OpFoldResult>":$innerTiles,
CArg<"Optional<Value>", "llvm::None">:$paddingValue, CArg<"Optional<Value>", "std::nullopt">:$paddingValue,
CArg<"ArrayRef<int64_t>", "{}">:$outerDimsPerm)> CArg<"ArrayRef<int64_t>", "{}">:$outerDimsPerm)>
]; ];
let extraClassDeclaration = commonExtraClassDeclaration # [{ let extraClassDeclaration = commonExtraClassDeclaration # [{
// Method to get the `ShapedType` of the result based on the inner tiles, // Method to get the `ShapedType` of the result based on the inner tiles,
// position of the inner tiles (innerDimsPos) and interchange vector of // position of the inner tiles (innerDimsPos) and interchange vector of
// outer loops (outerDimsPerm). // outer loops (outerDimsPerm).
static ShapedType inferPackedType(ShapedType sourceType, static ShapedType inferPackedType(ShapedType sourceType,
▲ Show 20 Lines • Show All 75 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Tosa/Utils/CoversionUtils.h

Show First 20 Lines • Show All 62 Lines • ▼ Show 20 LinesOptional<SmallVector<Value>> checkHasDynamicBatchDims(PatternRewriter &rewriter,
if (dynTypes.empty()) if (dynTypes.empty())
return dynamicDims; return dynamicDims;
for (const ShapedType &dynTy : dynTypes) { for (const ShapedType &dynTy : dynTypes) {
if (llvm::any_of(dynTy.getShape().drop_front(), ShapedType::isDynamic)) { if (llvm::any_of(dynTy.getShape().drop_front(), ShapedType::isDynamic)) {
(void)rewriter.notifyMatchFailure( (void)rewriter.notifyMatchFailure(
op, "input can only be dynamic for batch size"); op, "input can only be dynamic for batch size");
return llvm::None; return std::nullopt;
} }
} }
dynamicDims.push_back( dynamicDims.push_back(
rewriter.create<tensor::DimOp>(op->getLoc(), params[0], 0)); rewriter.create<tensor::DimOp>(op->getLoc(), params[0], 0));
return dynamicDims; return dynamicDims;
} }
} // namespace tosa } // namespace tosa
} // namespace mlir } // namespace mlir
#endif // DIALECT_TOSA_UTILS_COVERSION_UTILS_H_ #endif // DIALECT_TOSA_UTILS_COVERSION_UTILS_H_

mlir/include/mlir/Dialect/Transform/IR/TransformInterfaces.h

Show First 20 Lines • Show All 143 Lines • ▼ Show 20 Lines#endif // LLVM_ENABLE_ABI_BREAKING_CHECKS
} }
template <typename T> template <typename T>
DiagnosedSilenceableFailure &&operator<<(T &&value) && { DiagnosedSilenceableFailure &&operator<<(T &&value) && {
return std::move(this->operator<<(std::forward<T>(value))); return std::move(this->operator<<(std::forward<T>(value)));
} }
/// Attaches a note to the last diagnostic. /// Attaches a note to the last diagnostic.
/// Expects this object to be a silenceable failure. /// Expects this object to be a silenceable failure.
Diagnostic &attachNote(Optional<Location> loc = llvm::None) { Diagnostic &attachNote(Optional<Location> loc = std::nullopt) {
assert(isSilenceableFailure() && assert(isSilenceableFailure() &&
"can only attach notes to silenceable failures"); "can only attach notes to silenceable failures");
return diagnostics.back().attachNote(loc); return diagnostics.back().attachNote(loc);
} }
private: private:
explicit DiagnosedSilenceableFailure(Diagnostic &&diagnostic) explicit DiagnosedSilenceableFailure(Diagnostic &&diagnostic)
: result(failure()) { : result(failure()) {
▲ Show 20 Lines • Show All 46 Lines • ▼ Show 20 Lines DiagnosedDefiniteFailure &operator<<(T &&value) & {
return *this; return *this;
} }
template <typename T> template <typename T>
DiagnosedDefiniteFailure &&operator<<(T &&value) && { DiagnosedDefiniteFailure &&operator<<(T &&value) && {
return std::move(this->operator<<(std::forward<T>(value))); return std::move(this->operator<<(std::forward<T>(value)));
} }
/// Attaches a note to the error. /// Attaches a note to the error.
Diagnostic &attachNote(Optional<Location> loc = llvm::None) { Diagnostic &attachNote(Optional<Location> loc = std::nullopt) {
return diag.attachNote(loc); return diag.attachNote(loc);
} }
/// Implicit conversion to DiagnosedSilenceableFailure in the definite failure /// Implicit conversion to DiagnosedSilenceableFailure in the definite failure
/// state. Reports the error. /// state. Reports the error.
operator DiagnosedSilenceableFailure() { operator DiagnosedSilenceableFailure() {
diag.report(); diag.report();
return DiagnosedSilenceableFailure::definiteFailure(); return DiagnosedSilenceableFailure::definiteFailure();
▲ Show 20 Lines • Show All 866 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Utils/ReshapeOpsUtils.h

Show First 20 Lines • Show All 58 Lines • ▼ Show 20 Lines
getReassociationIndicesAttribute(OpBuilder &b, getReassociationIndicesAttribute(OpBuilder &b,
ArrayRef<ReassociationIndices> reassociation); ArrayRef<ReassociationIndices> reassociation);
/// Convert Array<Array<AffineExpr>> to Array<Array<int64_t>>. /// Convert Array<Array<AffineExpr>> to Array<Array<int64_t>>.
SmallVector<ReassociationIndices, 2> convertReassociationMapsToIndices( SmallVector<ReassociationIndices, 2> convertReassociationMapsToIndices(
OpBuilder &b, ArrayRef<ReassociationExprs> reassociationExprs); OpBuilder &b, ArrayRef<ReassociationExprs> reassociationExprs);
/// Return the reassociations maps to use to reshape given the source type and /// Return the reassociations maps to use to reshape given the source type and
/// the target type when possible. Return llvm::None when this computation /// the target type when possible. Return std::nullopt when this computation
/// failed. /// failed.
Optional<SmallVector<ReassociationIndices>> Optional<SmallVector<ReassociationIndices>>
getReassociationIndicesForReshape(ShapedType sourceType, ShapedType targetType); getReassociationIndicesForReshape(ShapedType sourceType, ShapedType targetType);
/// Returns the reassociation maps to collapse `sourceShape` to `targetShape` if /// Returns the reassociation maps to collapse `sourceShape` to `targetShape` if
/// possible. /// possible.
Optional<SmallVector<ReassociationIndices>> Optional<SmallVector<ReassociationIndices>>
getReassociationIndicesForCollapse(ArrayRef<int64_t> sourceShape, getReassociationIndicesForCollapse(ArrayRef<int64_t> sourceShape,
▲ Show 20 Lines • Show All 279 Lines • ▼ Show 20 Lines for (auto item : llvm::zip(srcReassociation, resultReassociation)) {
auto srcSubShape = srcShape.slice(srcIndices.front(), srcIndices.size()); auto srcSubShape = srcShape.slice(srcIndices.front(), srcIndices.size());
auto resultSubShape = auto resultSubShape =
resultShape.slice(resultIndices.front(), resultIndices.size()); resultShape.slice(resultIndices.front(), resultIndices.size());
if (srcSubShape.size() == resultSubShape.size()) { if (srcSubShape.size() == resultSubShape.size()) {
if (srcSubShape == resultSubShape) if (srcSubShape == resultSubShape)
composedReassociation.push_back(srcIndices); composedReassociation.push_back(srcIndices);
else else
return llvm::None; return std::nullopt;
} }
// Find reassociation to collapse `srcSubShape` into `resultSubShape`. // Find reassociation to collapse `srcSubShape` into `resultSubShape`.
auto subShapeReassociation = auto subShapeReassociation =
getReassociationIndicesForCollapse(srcSubShape, resultSubShape); getReassociationIndicesForCollapse(srcSubShape, resultSubShape);
if (!subShapeReassociation) if (!subShapeReassociation)
return llvm::None; return std::nullopt;
// Remap the subshape indices back to the original srcShape. // Remap the subshape indices back to the original srcShape.
for (auto &subshape_indices : *subShapeReassociation) { for (auto &subshape_indices : *subShapeReassociation) {
ReassociationIndices shape_indices; ReassociationIndices shape_indices;
for (int64_t index : subshape_indices) for (int64_t index : subshape_indices)
shape_indices.push_back(srcIndices.front() + index); shape_indices.push_back(srcIndices.front() + index);
composedReassociation.push_back(shape_indices); composedReassociation.push_back(shape_indices);
} }
▲ Show 20 Lines • Show All 146 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Vector/IR/VectorOps.h

Show First 20 Lines • Show All 112 Lines • ▼ Show 20 Lines
/// Collect a set of transfer read/write lowering patterns. /// Collect a set of transfer read/write lowering patterns.
/// ///
/// These patterns lower transfer ops to simpler ops like `vector.load`, /// These patterns lower transfer ops to simpler ops like `vector.load`,
/// `vector.store` and `vector.broadcast`. Only transfers with a transfer rank /// `vector.store` and `vector.broadcast`. Only transfers with a transfer rank
/// of a most `maxTransferRank` are lowered. This is useful when combined with /// of a most `maxTransferRank` are lowered. This is useful when combined with
/// VectorToSCF, which reduces the rank of vector transfer ops. /// VectorToSCF, which reduces the rank of vector transfer ops.
void populateVectorTransferLoweringPatterns( void populateVectorTransferLoweringPatterns(
RewritePatternSet &patterns, RewritePatternSet &patterns,
llvm::Optional<unsigned> maxTransferRank = llvm::None, llvm::Optional<unsigned> maxTransferRank = std::nullopt,
PatternBenefit benefit = 1); PatternBenefit benefit = 1);
/// These patterns materialize masks for various vector ops such as transfers. /// These patterns materialize masks for various vector ops such as transfers.
void populateVectorMaskMaterializationPatterns(RewritePatternSet &patterns, void populateVectorMaskMaterializationPatterns(RewritePatternSet &patterns,
bool force32BitVectorIndices, bool force32BitVectorIndices,
PatternBenefit benefit = 1); PatternBenefit benefit = 1);
/// Collects patterns to progressively lower vector.broadcast ops on high-D /// Collects patterns to progressively lower vector.broadcast ops on high-D
▲ Show 20 Lines • Show All 79 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Vector/IR/VectorOps.td

Show First 20 Lines • Show All 531 Lines • ▼ Show 20 Lines Vector_Op<"extractelement", [Pure,
Results<(outs AnyType:$result)> { Results<(outs AnyType:$result)> {
let summary = "extractelement operation"; let summary = "extractelement operation";
let description = [{ let description = [{
Takes a 0-D or 1-D vector and a optional dynamic index position and Takes a 0-D or 1-D vector and a optional dynamic index position and
extracts the scalar at that position. extracts the scalar at that position.
Note that this instruction resembles vector.extract, but is restricted to Note that this instruction resembles vector.extract, but is restricted to
0-D and 1-D vectors and relaxed to dynamic indices. 0-D and 1-D vectors and relaxed to dynamic indices.
If the vector is 0-D, the position must be llvm::None. If the vector is 0-D, the position must be std::nullopt.
It is meant to be closer to LLVM's version: It is meant to be closer to LLVM's version:
https://llvm.org/docs/LangRef.html#extractelement-instruction https://llvm.org/docs/LangRef.html#extractelement-instruction
Example: Example:
```mlir ```mlir
▲ Show 20 Lines • Show All 797 Lines • ▼ Show 20 Lines OpBuilder<(ins "VectorType":$vectorType,
"ValueRange":$indices, "ValueRange":$indices,
"AffineMapAttr":$permutationMapAttr, "AffineMapAttr":$permutationMapAttr,
"ArrayAttr":$inBoundsAttr)>, "ArrayAttr":$inBoundsAttr)>,
/// 2. Builder that sets padding to zero and an empty mask (variant without attrs). /// 2. Builder that sets padding to zero and an empty mask (variant without attrs).
OpBuilder<(ins "VectorType":$vectorType, OpBuilder<(ins "VectorType":$vectorType,
"Value":$source, "Value":$source,
"ValueRange":$indices, "ValueRange":$indices,
"AffineMap":$permutationMap, "AffineMap":$permutationMap,
CArg<"Optional<ArrayRef<bool>>", "::llvm::None">:$inBounds)>, CArg<"Optional<ArrayRef<bool>>", "::std::nullopt">:$inBounds)>,
/// 3. Builder that sets permutation map to 'getMinorIdentityMap'. /// 3. Builder that sets permutation map to 'getMinorIdentityMap'.
OpBuilder<(ins "VectorType":$vectorType, OpBuilder<(ins "VectorType":$vectorType,
"Value":$source, "Value":$source,
"ValueRange":$indices, "ValueRange":$indices,
"Value":$padding, "Value":$padding,
CArg<"Optional<ArrayRef<bool>>", "::llvm::None">:$inBounds)>, CArg<"Optional<ArrayRef<bool>>", "::std::nullopt">:$inBounds)>,
/// 4. Builder that sets padding to zero and permutation map to /// 4. Builder that sets padding to zero and permutation map to
/// 'getMinorIdentityMap'. /// 'getMinorIdentityMap'.
OpBuilder<(ins "VectorType":$vectorType, OpBuilder<(ins "VectorType":$vectorType,
"Value":$source, "Value":$source,
"ValueRange":$indices, "ValueRange":$indices,
CArg<"Optional<ArrayRef<bool>>", "::llvm::None">:$inBounds)>, CArg<"Optional<ArrayRef<bool>>", "::std::nullopt">:$inBounds)>,
]; ];
let extraClassDeclaration = [{ let extraClassDeclaration = [{
// MaskableOpInterface methods. // MaskableOpInterface methods.
bool supportsPassthru() { return true; } bool supportsPassthru() { return true; }
}]; }];
let hasCanonicalizer = 1; let hasCanonicalizer = 1;
▲ Show 20 Lines • Show All 120 Lines • ▼ Show 20 Lines OpBuilder<(ins "Value":$vector,
"ValueRange":$indices, "ValueRange":$indices,
"AffineMapAttr":$permutationMapAttr, "AffineMapAttr":$permutationMapAttr,
"ArrayAttr":$inBoundsAttr)>, "ArrayAttr":$inBoundsAttr)>,
/// 3. Builder with type inference that sets an empty mask (variant without attrs). /// 3. Builder with type inference that sets an empty mask (variant without attrs).
OpBuilder<(ins "Value":$vector, OpBuilder<(ins "Value":$vector,
"Value":$dest, "Value":$dest,
"ValueRange":$indices, "ValueRange":$indices,
"AffineMap":$permutationMap, "AffineMap":$permutationMap,
CArg<"Optional<ArrayRef<bool>>", "::llvm::None">:$inBounds)>, CArg<"Optional<ArrayRef<bool>>", "::std::nullopt">:$inBounds)>,
/// 4. Builder with type inference that sets an empty mask and sets permutation /// 4. Builder with type inference that sets an empty mask and sets permutation
/// map to 'getMinorIdentityMap'. /// map to 'getMinorIdentityMap'.
OpBuilder<(ins "Value":$vector, OpBuilder<(ins "Value":$vector,
"Value":$dest, "Value":$dest,
"ValueRange":$indices, "ValueRange":$indices,
CArg<"Optional<ArrayRef<bool>>", "::llvm::None">:$inBounds)>, CArg<"Optional<ArrayRef<bool>>", "::std::nullopt">:$inBounds)>,
]; ];
let extraClassDeclaration = [{ let extraClassDeclaration = [{
/// This method is added to maintain uniformity with load/store /// This method is added to maintain uniformity with load/store
/// ops of other dialects. /// ops of other dialects.
Value getValue() { return getVector(); } Value getValue() { return getVector(); }
std::pair<int64_t, int64_t> getDpsInitsPositionRange() { std::pair<int64_t, int64_t> getDpsInitsPositionRange() {
▲ Show 20 Lines • Show All 1,299 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Vector/Interfaces/MaskableOpInterface.td

Show First 20 Lines • Show All 57 Lines • ▼ Show 20 Lines InterfaceMethod<
"the operation to be vectorized.", "the operation to be vectorized.",
/*retTy=*/"mlir::VectorType", /*retTy=*/"mlir::VectorType",
/*methodName=*/"getExpectedMaskType", /*methodName=*/"getExpectedMaskType",
/*args=*/(ins), /*args=*/(ins),
/*methodBody=*/"", /*methodBody=*/"",
/*defaultImplementation=*/[{ /*defaultImplementation=*/[{
// Default implementation is only aimed for operations that implement the // Default implementation is only aimed for operations that implement the
// `getVectorType()` method. // `getVectorType()` method.
return $_op.getVectorType().cloneWith(/*shape=*/llvm::None, return $_op.getVectorType().cloneWith(/*shape=*/std::nullopt,
IntegerType::get($_op.getContext(), /*width=*/1)); IntegerType::get($_op.getContext(), /*width=*/1));
}]>, }]>,
]; ];
} }
#endif // MLIR_DIALECT_VECTOR_INTERFACES_MASKABLEOPINTERFACE_TD #endif // MLIR_DIALECT_VECTOR_INTERFACES_MASKABLEOPINTERFACE_TD

mlir/include/mlir/Dialect/Vector/Transforms/VectorRewritePatterns.h

Show First 20 Lines • Show All 107 Lines • ▼ Show 20 Linesstruct UnrollVectorOptions {
UnrollVectorOptions &setFilterConstraint(FilterConstraintFnType constraint) { UnrollVectorOptions &setFilterConstraint(FilterConstraintFnType constraint) {
filterConstraint = std::move(constraint); filterConstraint = std::move(constraint);
return *this; return *this;
} }
using NativeShapeFnType = using NativeShapeFnType =
std::function<Optional<SmallVector<int64_t>>(Operation *op)>; std::function<Optional<SmallVector<int64_t>>(Operation *op)>;
/// Function that returns the shape of the vector to unroll to for a given /// Function that returns the shape of the vector to unroll to for a given
/// operation. The unrolling is aborted if the function returns `llvm::None`. /// operation. The unrolling is aborted if the function returns
/// `std::nullopt`.
NativeShapeFnType nativeShape = nullptr; NativeShapeFnType nativeShape = nullptr;
UnrollVectorOptions &setNativeShapeFn(NativeShapeFnType fn) { UnrollVectorOptions &setNativeShapeFn(NativeShapeFnType fn) {
nativeShape = std::move(fn); nativeShape = std::move(fn);
return *this; return *this;
} }
/// Set the native shape to use for unrolling. /// Set the native shape to use for unrolling.
UnrollVectorOptions &setNativeShape(ArrayRef<int64_t> shape) { UnrollVectorOptions &setNativeShape(ArrayRef<int64_t> shape) {
▲ Show 20 Lines • Show All 439 Lines • Show Last 20 Lines

mlir/include/mlir/ExecutionEngine/ExecutionEngine.h

Show First 20 Lines • Show All 61 Lines • ▼ Show 20 Lines llvm::function_ref<std::unique_ptr<llvm::Module>(Operation *,
llvmModuleBuilder = nullptr; llvmModuleBuilder = nullptr;
/// If `transformer` is provided, it will be called on the LLVM module during /// If `transformer` is provided, it will be called on the LLVM module during
/// JIT-compilation and can be used, e.g., for reporting or optimization. /// JIT-compilation and can be used, e.g., for reporting or optimization.
llvm::function_ref<llvm::Error(llvm::Module *)> transformer = {}; llvm::function_ref<llvm::Error(llvm::Module *)> transformer = {};
/// `jitCodeGenOptLevel`, when provided, is used as the optimization level for /// `jitCodeGenOptLevel`, when provided, is used as the optimization level for
/// target code generation. /// target code generation.
Optional<llvm::CodeGenOpt::Level> jitCodeGenOptLevel = llvm::None; Optional<llvm::CodeGenOpt::Level> jitCodeGenOptLevel = std::nullopt;
/// If `sharedLibPaths` are provided, the underlying JIT-compilation will /// If `sharedLibPaths` are provided, the underlying JIT-compilation will
/// open and link the shared libraries for symbol resolution. /// open and link the shared libraries for symbol resolution.
ArrayRef<StringRef> sharedLibPaths = {}; ArrayRef<StringRef> sharedLibPaths = {};
/// Specifies an existing `sectionMemoryMapper` to be associated with the /// Specifies an existing `sectionMemoryMapper` to be associated with the
/// compiled code. If none is provided, a default memory mapper that directly /// compiled code. If none is provided, a default memory mapper that directly
/// calls into the operating system is used. /// calls into the operating system is used.
Show All 39 Linespublic:
/// Looks up the original function with the given name and returns a /// Looks up the original function with the given name and returns a
/// pointer to it. This is not necesarily a packed function. Propagates /// pointer to it. This is not necesarily a packed function. Propagates
/// errors in case of failure. /// errors in case of failure.
llvm::Expected<void *> lookup(StringRef name) const; llvm::Expected<void *> lookup(StringRef name) const;
/// Invokes the function with the given name passing it the list of opaque /// Invokes the function with the given name passing it the list of opaque
/// pointers to the actual arguments. /// pointers to the actual arguments.
llvm::Error invokePacked(StringRef name, llvm::Error invokePacked(StringRef name,
MutableArrayRef<void *> args = llvm::None); MutableArrayRef<void *> args = std::nullopt);
/// Trait that defines how a given type is passed to the JIT code. This /// Trait that defines how a given type is passed to the JIT code. This
/// defaults to passing the address but can be specialized. /// defaults to passing the address but can be specialized.
template <typename T> template <typename T>
struct Argument { struct Argument {
static void pack(SmallVectorImpl<void *> &args, T &val) { static void pack(SmallVectorImpl<void *> &args, T &val) {
args.push_back(&val); args.push_back(&val);
} }
▲ Show 20 Lines • Show All 83 Lines • Show Last 20 Lines

mlir/include/mlir/IR/BlockSupport.h

Show First 20 Lines • Show All 100 Lines • ▼ Show 20 Lines
/// suitable for a more derived type (e.g. ArrayRef) or a template range /// suitable for a more derived type (e.g. ArrayRef) or a template range
/// parameter. /// parameter.
class BlockRange final class BlockRange final
: public llvm::detail::indexed_accessor_range_base< : public llvm::detail::indexed_accessor_range_base<
BlockRange, llvm::PointerUnion<BlockOperand *, Block *const *>, BlockRange, llvm::PointerUnion<BlockOperand *, Block *const *>,
Block *, Block *, Block *> { Block *, Block *, Block *> {
public: public:
using RangeBaseT::RangeBaseT; using RangeBaseT::RangeBaseT;
BlockRange(ArrayRef<Block *> blocks = llvm::None); BlockRange(ArrayRef<Block *> blocks = std::nullopt);
BlockRange(SuccessorRange successors); BlockRange(SuccessorRange successors);
template <typename Arg, typename = std::enable_if_t<std::is_constructible< template <typename Arg, typename = std::enable_if_t<std::is_constructible<
ArrayRef<Block *>, Arg>::value>> ArrayRef<Block *>, Arg>::value>>
BlockRange(Arg &&arg) BlockRange(Arg &&arg)
: BlockRange(ArrayRef<Block *>(std::forward<Arg>(arg))) {} : BlockRange(ArrayRef<Block *>(std::forward<Arg>(arg))) {}
BlockRange(std::initializer_list<Block *> blocks) BlockRange(std::initializer_list<Block *> blocks)
: BlockRange(ArrayRef<Block *>(blocks)) {} : BlockRange(ArrayRef<Block *>(blocks)) {}
▲ Show 20 Lines • Show All 143 Lines • Show Last 20 Lines

mlir/include/mlir/IR/Builders.h

Show First 20 Lines • Show All 402 Lines • ▼ Show 20 Linespublic:
// Block Creation // Block Creation
//===--------------------------------------------------------------------===// //===--------------------------------------------------------------------===//
/// Add new block with 'argTypes' arguments and set the insertion point to the /// Add new block with 'argTypes' arguments and set the insertion point to the
/// end of it. The block is inserted at the provided insertion point of /// end of it. The block is inserted at the provided insertion point of
/// 'parent'. `locs` contains the locations of the inserted arguments, and /// 'parent'. `locs` contains the locations of the inserted arguments, and
/// should match the size of `argTypes`. /// should match the size of `argTypes`.
Block *createBlock(Region *parent, Region::iterator insertPt = {}, Block *createBlock(Region *parent, Region::iterator insertPt = {},
TypeRange argTypes = llvm::None, TypeRange argTypes = std::nullopt,
ArrayRef<Location> locs = llvm::None); ArrayRef<Location> locs = std::nullopt);
/// Add new block with 'argTypes' arguments and set the insertion point to the /// Add new block with 'argTypes' arguments and set the insertion point to the
/// end of it. The block is placed before 'insertBefore'. `locs` contains the /// end of it. The block is placed before 'insertBefore'. `locs` contains the
/// locations of the inserted arguments, and should match the size of /// locations of the inserted arguments, and should match the size of
/// `argTypes`. /// `argTypes`.
Block *createBlock(Block *insertBefore, TypeRange argTypes = llvm::None, Block *createBlock(Block *insertBefore, TypeRange argTypes = std::nullopt,
ArrayRef<Location> locs = llvm::None); ArrayRef<Location> locs = std::nullopt);
//===--------------------------------------------------------------------===// //===--------------------------------------------------------------------===//
// Operation Creation // Operation Creation
//===--------------------------------------------------------------------===// //===--------------------------------------------------------------------===//
/// Insert the given operation at the current insertion point and return it. /// Insert the given operation at the current insertion point and return it.
Operation *insert(Operation *op); Operation *insert(Operation *op);
▲ Show 20 Lines • Show All 122 Lines • Show Last 20 Lines

mlir/include/mlir/IR/BuiltinAttributeInterfaces.h

Show First 20 Lines • Show All 306 Lines • ▼ Show 20 Linesauto ElementsAttr::value_begin() const -> DefaultValueCheckT<T, iterator<T>> {
llvm_unreachable("invalid `T` for ElementsAttr::getValues"); llvm_unreachable("invalid `T` for ElementsAttr::getValues");
} }
template <typename T> template <typename T>
auto ElementsAttr::try_value_begin() const auto ElementsAttr::try_value_begin() const
-> DefaultValueCheckT<T, Optional<iterator<T>>> { -> DefaultValueCheckT<T, Optional<iterator<T>>> {
FailureOr<detail::ElementsAttrIndexer> indexer = FailureOr<detail::ElementsAttrIndexer> indexer =
getValuesImpl(TypeID::get<T>()); getValuesImpl(TypeID::get<T>());
if (failed(indexer)) if (failed(indexer))
return llvm::None; return std::nullopt;
return iterator<T>(std::move(*indexer), 0); return iterator<T>(std::move(*indexer), 0);
} }
} // namespace mlir. } // namespace mlir.
#endif // MLIR_IR_BUILTINATTRIBUTEINTERFACES_H #endif // MLIR_IR_BUILTINATTRIBUTEINTERFACES_H

mlir/include/mlir/IR/BuiltinAttributeInterfaces.td

Show First 20 Lines • Show All 387 Lines • ▼ Show 20 Lines let extraClassDeclaration = [{
DerivedAttrValueIterator<T> value_end() const { DerivedAttrValueIterator<T> value_end() const {
return {value_end<Attribute>(), nullptr}; return {value_end<Attribute>(), nullptr};
} }
//===------------------------------------------------------------------===// //===------------------------------------------------------------------===//
// Failable Value Iteration // Failable Value Iteration
/// If this attribute supports iterating over element values of type `T`, /// If this attribute supports iterating over element values of type `T`,
/// return the iterable range. Otherwise, return llvm::None. /// return the iterable range. Otherwise, return std::nullopt.
template <typename T> template <typename T>
DefaultValueCheckT<T, Optional<iterator_range<T>>> tryGetValues() const { DefaultValueCheckT<T, Optional<iterator_range<T>>> tryGetValues() const {
if (Optional<iterator<T>> beginIt = try_value_begin<T>()) if (Optional<iterator<T>> beginIt = try_value_begin<T>())
return iterator_range<T>(getType(), *beginIt, value_end<T>()); return iterator_range<T>(getType(), *beginIt, value_end<T>());
return llvm::None; return std::nullopt;
} }
template <typename T> template <typename T>
DefaultValueCheckT<T, Optional<iterator<T>>> try_value_begin() const; DefaultValueCheckT<T, Optional<iterator<T>>> try_value_begin() const;
/// If this attribute supports iterating over element values of type `T`, /// If this attribute supports iterating over element values of type `T`,
/// return the iterable range. Otherwise, return llvm::None. /// return the iterable range. Otherwise, return std::nullopt.
template <typename T, typename = DerivedAttrValueCheckT<T>> template <typename T, typename = DerivedAttrValueCheckT<T>>
Optional<DerivedAttrValueIteratorRange<T>> tryGetValues() const { Optional<DerivedAttrValueIteratorRange<T>> tryGetValues() const {
auto values = tryGetValues<Attribute>(); auto values = tryGetValues<Attribute>();
if (!values) if (!values)
return llvm::None; return std::nullopt;
auto castFn = [](Attribute attr) { return attr.template cast<T>(); }; auto castFn = [](Attribute attr) { return attr.template cast<T>(); };
return DerivedAttrValueIteratorRange<T>( return DerivedAttrValueIteratorRange<T>(
getType(), getType(),
llvm::map_range(*values, static_cast<T (*)(Attribute)>(castFn)) llvm::map_range(*values, static_cast<T (*)(Attribute)>(castFn))
); );
} }
template <typename T, typename = DerivedAttrValueCheckT<T>> template <typename T, typename = DerivedAttrValueCheckT<T>>
Optional<DerivedAttrValueIterator<T>> try_value_begin() const { Optional<DerivedAttrValueIterator<T>> try_value_begin() const {
if (auto values = tryGetValues<T>()) if (auto values = tryGetValues<T>())
return values->begin(); return values->begin();
return llvm::None; return std::nullopt;
} }
}] # ElementsAttrInterfaceAccessors; }] # ElementsAttrInterfaceAccessors;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// MemRefLayoutAttrInterface // MemRefLayoutAttrInterface
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 64 Lines • Show Last 20 Lines

mlir/include/mlir/IR/BuiltinAttributes.td

Show First 20 Lines • Show All 528 Lines • ▼ Show 20 Lines let description = [{
```mlir ```mlir
{} {}
{attr_name = "string attribute"} {attr_name = "string attribute"}
{int_attr = 10, "string attr name" = "string attribute"} {int_attr = 10, "string attr name" = "string attribute"}
``` ```
}]; }];
let parameters = (ins ArrayRefParameter<"NamedAttribute", "">:$value); let parameters = (ins ArrayRefParameter<"NamedAttribute", "">:$value);
let builders = [ let builders = [
AttrBuilder<(ins CArg<"ArrayRef<NamedAttribute>", "llvm::None">:$value)> AttrBuilder<(ins CArg<"ArrayRef<NamedAttribute>", "std::nullopt">:$value)>
]; ];
let extraClassDeclaration = [{ let extraClassDeclaration = [{
using ValueType = ArrayRef<NamedAttribute>; using ValueType = ArrayRef<NamedAttribute>;
/// Construct a dictionary with an array of values that is known to already /// Construct a dictionary with an array of values that is known to already
/// be sorted by name and uniqued. /// be sorted by name and uniqued.
static DictionaryAttr getWithSorted(MLIRContext *context, static DictionaryAttr getWithSorted(MLIRContext *context,
ArrayRef<NamedAttribute> value); ArrayRef<NamedAttribute> value);
Show All 25 Lines let extraClassDeclaration = [{
/// Sorts the NamedAttributes in the array ordered by name as expected by /// Sorts the NamedAttributes in the array ordered by name as expected by
/// getWithSorted in place on an array and returns whether the values needed /// getWithSorted in place on an array and returns whether the values needed
/// to be sorted. /// to be sorted.
/// Requires: uniquely named attributes. /// Requires: uniquely named attributes.
static bool sortInPlace(SmallVectorImpl<NamedAttribute> &array); static bool sortInPlace(SmallVectorImpl<NamedAttribute> &array);
/// Returns an entry with a duplicate name in `array`, if it exists, else /// Returns an entry with a duplicate name in `array`, if it exists, else
/// returns llvm::None. If `isSorted` is true, the array is assumed to be /// returns std::nullopt. If `isSorted` is true, the array is assumed to be
/// sorted else it will be sorted in place before finding the duplicate entry. /// sorted else it will be sorted in place before finding the duplicate entry.
static Optional<NamedAttribute> static Optional<NamedAttribute>
findDuplicate(SmallVectorImpl<NamedAttribute> &array, bool isSorted); findDuplicate(SmallVectorImpl<NamedAttribute> &array, bool isSorted);
/// Return the specified attribute if present and is an instance of /// Return the specified attribute if present and is an instance of
/// `AttrClass`, null otherwise. /// `AttrClass`, null otherwise.
template<typename AttrClass, typename NameClass> template<typename AttrClass, typename NameClass>
AttrClass getAs(NameClass &&name) const { AttrClass getAs(NameClass &&name) const {
▲ Show 20 Lines • Show All 677 Lines • Show Last 20 Lines

mlir/include/mlir/IR/BuiltinOps.td

Show First 20 Lines • Show All 55 Lines • ▼ Show 20 Linesdef ModuleOp : Builtin_Op<"module", [
let arguments = (ins OptionalAttr<SymbolNameAttr>:$sym_name, let arguments = (ins OptionalAttr<SymbolNameAttr>:$sym_name,
OptionalAttr<StrAttr>:$sym_visibility); OptionalAttr<StrAttr>:$sym_visibility);
let regions = (region SizedRegion<1>:$bodyRegion); let regions = (region SizedRegion<1>:$bodyRegion);
let assemblyFormat = "($sym_name^)? attr-dict-with-keyword $bodyRegion"; let assemblyFormat = "($sym_name^)? attr-dict-with-keyword $bodyRegion";
let builders = [OpBuilder<(ins CArg<"Optional<StringRef>", "{}">:$name)>]; let builders = [OpBuilder<(ins CArg<"Optional<StringRef>", "{}">:$name)>];
let extraClassDeclaration = [{ let extraClassDeclaration = [{
/// Construct a module from the given location with an optional name. /// Construct a module from the given location with an optional name.
static ModuleOp create(Location loc, Optional<StringRef> name = llvm::None); static ModuleOp create(Location loc, Optional<StringRef> name = std::nullopt);
/// Return the name of this module if present. /// Return the name of this module if present.
Optional<StringRef> getName() { return getSymName(); } Optional<StringRef> getName() { return getSymName(); }
//===------------------------------------------------------------------===// //===------------------------------------------------------------------===//
// SymbolOpInterface Methods // SymbolOpInterface Methods
//===------------------------------------------------------------------===// //===------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 74 Lines • Show Last 20 Lines

mlir/include/mlir/IR/BuiltinTypeInterfaces.td

Show First 20 Lines • Show All 115 Lines • ▼ Show 20 Lines auto clone(::llvm::ArrayRef<int64_t> shape, Type elementType) {
return $_type.cloneWith(shape, elementType); return $_type.cloneWith(shape, elementType);
} }
/// Return a clone of this type with the given new shape. /// Return a clone of this type with the given new shape.
auto clone(::llvm::ArrayRef<int64_t> shape) { auto clone(::llvm::ArrayRef<int64_t> shape) {
return $_type.cloneWith(shape, $_type.getElementType()); return $_type.cloneWith(shape, $_type.getElementType());
} }
/// Return a clone of this type with the given new element type. /// Return a clone of this type with the given new element type.
auto clone(::mlir::Type elementType) { auto clone(::mlir::Type elementType) {
return $_type.cloneWith(/*shape=*/llvm::None, elementType); return $_type.cloneWith(/*shape=*/std::nullopt, elementType);
} }
/// If an element type is an integer or a float, return its width. Otherwise, /// If an element type is an integer or a float, return its width. Otherwise,
/// abort. /// abort.
unsigned getElementTypeBitWidth() const { unsigned getElementTypeBitWidth() const {
return $_type.getElementType().getIntOrFloatBitWidth(); return $_type.getElementType().getIntOrFloatBitWidth();
} }
▲ Show 20 Lines • Show All 57 Lines • Show Last 20 Lines

mlir/include/mlir/IR/BuiltinTypes.td

Show First 20 Lines • Show All 874 Lines • ▼ Show 20 Lines let extraClassDeclaration = [{
using ShapedType::Trait<UnrankedMemRefType>::getRank; using ShapedType::Trait<UnrankedMemRefType>::getRank;
using ShapedType::Trait<UnrankedMemRefType>::getNumElements; using ShapedType::Trait<UnrankedMemRefType>::getNumElements;
using ShapedType::Trait<UnrankedMemRefType>::isDynamicDim; using ShapedType::Trait<UnrankedMemRefType>::isDynamicDim;
using ShapedType::Trait<UnrankedMemRefType>::hasStaticShape; using ShapedType::Trait<UnrankedMemRefType>::hasStaticShape;
using ShapedType::Trait<UnrankedMemRefType>::getNumDynamicDims; using ShapedType::Trait<UnrankedMemRefType>::getNumDynamicDims;
using ShapedType::Trait<UnrankedMemRefType>::getDimSize; using ShapedType::Trait<UnrankedMemRefType>::getDimSize;
using ShapedType::Trait<UnrankedMemRefType>::getDynamicDimIndex; using ShapedType::Trait<UnrankedMemRefType>::getDynamicDimIndex;
ArrayRef<int64_t> getShape() const { return llvm::None; } ArrayRef<int64_t> getShape() const { return std::nullopt; }
/// [deprecated] Returns the memory space in old raw integer representation. /// [deprecated] Returns the memory space in old raw integer representation.
/// New `Attribute getMemorySpace()` method should be used instead. /// New `Attribute getMemorySpace()` method should be used instead.
unsigned getMemorySpaceAsInt() const; unsigned getMemorySpaceAsInt() const;
}]; }];
let skipDefaultBuilders = 1; let skipDefaultBuilders = 1;
let genVerifyDecl = 1; let genVerifyDecl = 1;
} }
Show All 36 Lines let extraClassDeclaration = [{
using ShapedType::Trait<UnrankedTensorType>::getRank; using ShapedType::Trait<UnrankedTensorType>::getRank;
using ShapedType::Trait<UnrankedTensorType>::getNumElements; using ShapedType::Trait<UnrankedTensorType>::getNumElements;
using ShapedType::Trait<UnrankedTensorType>::isDynamicDim; using ShapedType::Trait<UnrankedTensorType>::isDynamicDim;
using ShapedType::Trait<UnrankedTensorType>::hasStaticShape; using ShapedType::Trait<UnrankedTensorType>::hasStaticShape;
using ShapedType::Trait<UnrankedTensorType>::getNumDynamicDims; using ShapedType::Trait<UnrankedTensorType>::getNumDynamicDims;
using ShapedType::Trait<UnrankedTensorType>::getDimSize; using ShapedType::Trait<UnrankedTensorType>::getDimSize;
using ShapedType::Trait<UnrankedTensorType>::getDynamicDimIndex; using ShapedType::Trait<UnrankedTensorType>::getDynamicDimIndex;
ArrayRef<int64_t> getShape() const { return llvm::None; } ArrayRef<int64_t> getShape() const { return std::nullopt; }
}]; }];
let skipDefaultBuilders = 1; let skipDefaultBuilders = 1;
let genVerifyDecl = 1; let genVerifyDecl = 1;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// VectorType // VectorType
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 97 Lines • Show Last 20 Lines

mlir/include/mlir/IR/Diagnostics.h

Show First 20 Lines • Show All 238 Lines • ▼ Show 20 Linespublic:
void print(raw_ostream &os) const; void print(raw_ostream &os) const;
/// Converts the diagnostic to a string. /// Converts the diagnostic to a string.
std::string str() const; std::string str() const;
/// Attaches a note to this diagnostic. A new location may be optionally /// Attaches a note to this diagnostic. A new location may be optionally
/// provided, if not, then the location defaults to the one specified for this /// provided, if not, then the location defaults to the one specified for this
/// diagnostic. Notes may not be attached to other notes. /// diagnostic. Notes may not be attached to other notes.
Diagnostic &attachNote(Optional<Location> noteLoc = llvm::None); Diagnostic &attachNote(Optional<Location> noteLoc = std::nullopt);
using note_iterator = llvm::pointee_iterator<NoteVector::iterator>; using note_iterator = llvm::pointee_iterator<NoteVector::iterator>;
using const_note_iterator = using const_note_iterator =
llvm::pointee_iterator<NoteVector::const_iterator>; llvm::pointee_iterator<NoteVector::const_iterator>;
/// Returns the notes held by this diagnostic. /// Returns the notes held by this diagnostic.
iterator_range<note_iterator> getNotes() { iterator_range<note_iterator> getNotes() {
return llvm::make_pointee_range(notes); return llvm::make_pointee_range(notes);
▲ Show 20 Lines • Show All 81 Lines • ▼ Show 20 Lines InFlightDiagnostic &append(Args &&...args) & {
return *this; return *this;
} }
template <typename... Args> template <typename... Args>
InFlightDiagnostic &&append(Args &&...args) && { InFlightDiagnostic &&append(Args &&...args) && {
return std::move(append(std::forward<Args>(args)...)); return std::move(append(std::forward<Args>(args)...));
} }
/// Attaches a note to this diagnostic. /// Attaches a note to this diagnostic.
Diagnostic &attachNote(Optional<Location> noteLoc = llvm::None) { Diagnostic &attachNote(Optional<Location> noteLoc = std::nullopt) {
assert(isActive() && "diagnostic not active"); assert(isActive() && "diagnostic not active");
return impl->attachNote(noteLoc); return impl->attachNote(noteLoc);
} }
/// Reports the diagnostic to the engine. /// Reports the diagnostic to the engine.
void report(); void report();
/// Abandons this diagnostic so that it will no longer be reported. /// Abandons this diagnostic so that it will no longer be reported.
▲ Show 20 Lines • Show All 322 Lines • Show Last 20 Lines

mlir/include/mlir/IR/DialectImplementation.h

Show First 20 Lines • Show All 112 Lines • ▼ Show 20 Linesstruct FieldParser<
static FailureOr<Optional<IntT>> parse(AsmParser &parser) { static FailureOr<Optional<IntT>> parse(AsmParser &parser) {
IntT value; IntT value;
OptionalParseResult result = parser.parseOptionalInteger(value); OptionalParseResult result = parser.parseOptionalInteger(value);
if (result.has_value()) { if (result.has_value()) {
if (succeeded(*result)) if (succeeded(*result))
return {Optional<IntT>(value)}; return {Optional<IntT>(value)};
return failure(); return failure();
} }
return {llvm::None}; return {std::nullopt};
} }
}; };
/// Parse any container that supports back insertion as a list. /// Parse any container that supports back insertion as a list.
template <typename ContainerT> template <typename ContainerT>
struct FieldParser< struct FieldParser<
ContainerT, std::enable_if_t<std::is_member_function_pointer< ContainerT, std::enable_if_t<std::is_member_function_pointer<
decltype(&ContainerT::push_back)>::value, decltype(&ContainerT::push_back)>::value,
Show All 31 Lines

mlir/include/mlir/IR/FunctionInterfaces.h

Show First 20 Lines • Show All 56 Lines • ▼ Show 20 Lines
void setAllArgAttrDicts(Operation *op, ArrayRef<DictionaryAttr> attrs); void setAllArgAttrDicts(Operation *op, ArrayRef<DictionaryAttr> attrs);
void setAllArgAttrDicts(Operation *op, ArrayRef<Attribute> attrs); void setAllArgAttrDicts(Operation *op, ArrayRef<Attribute> attrs);
void setAllResultAttrDicts(Operation *op, ArrayRef<DictionaryAttr> attrs); void setAllResultAttrDicts(Operation *op, ArrayRef<DictionaryAttr> attrs);
void setAllResultAttrDicts(Operation *op, ArrayRef<Attribute> attrs); void setAllResultAttrDicts(Operation *op, ArrayRef<Attribute> attrs);
/// Return all of the attributes for the argument at 'index'. /// Return all of the attributes for the argument at 'index'.
inline ArrayRef<NamedAttribute> getArgAttrs(Operation *op, unsigned index) { inline ArrayRef<NamedAttribute> getArgAttrs(Operation *op, unsigned index) {
auto argDict = getArgAttrDict(op, index); auto argDict = getArgAttrDict(op, index);
return argDict ? argDict.getValue() : llvm::None; return argDict ? argDict.getValue() : std::nullopt;
} }
/// Return all of the attributes for the result at 'index'. /// Return all of the attributes for the result at 'index'.
inline ArrayRef<NamedAttribute> getResultAttrs(Operation *op, unsigned index) { inline ArrayRef<NamedAttribute> getResultAttrs(Operation *op, unsigned index) {
auto resultDict = getResultAttrDict(op, index); auto resultDict = getResultAttrDict(op, index);
return resultDict ? resultDict.getValue() : llvm::None; return resultDict ? resultDict.getValue() : std::nullopt;
} }
/// Insert the specified arguments and update the function type attribute. /// Insert the specified arguments and update the function type attribute.
void insertFunctionArguments(Operation *op, ArrayRef<unsigned> argIndices, void insertFunctionArguments(Operation *op, ArrayRef<unsigned> argIndices,
TypeRange argTypes, TypeRange argTypes,
ArrayRef<DictionaryAttr> argAttrs, ArrayRef<DictionaryAttr> argAttrs,
ArrayRef<Location> argLocs, ArrayRef<Location> argLocs,
unsigned originalNumArgs, Type newType); unsigned originalNumArgs, Type newType);
▲ Show 20 Lines • Show All 219 Lines • Show Last 20 Lines

mlir/include/mlir/IR/Matchers.h

Show First 20 Lines • Show All 71 Lines • ▼ Show 20 Linesstruct constant_op_binder {
constant_op_binder() : bind_value(nullptr) {} constant_op_binder() : bind_value(nullptr) {}
bool match(Operation *op) { bool match(Operation *op) {
if (!isConstantLike(op)) if (!isConstantLike(op))
return false; return false;
// Fold the constant to an attribute. // Fold the constant to an attribute.
SmallVector<OpFoldResult, 1> foldedOp; SmallVector<OpFoldResult, 1> foldedOp;
LogicalResult result = op->fold(/*operands=*/llvm::None, foldedOp); LogicalResult result = op->fold(/*operands=*/std::nullopt, foldedOp);
(void)result; (void)result;
assert(succeeded(result) && "expected ConstantLike op to be foldable"); assert(succeeded(result) && "expected ConstantLike op to be foldable");
if (auto attr = foldedOp.front().get<Attribute>().dyn_cast<AttrT>()) { if (auto attr = foldedOp.front().get<Attribute>().dyn_cast<AttrT>()) {
if (bind_value) if (bind_value)
*bind_value = attr; *bind_value = attr;
return true; return true;
} }
▲ Show 20 Lines • Show All 281 Lines • Show Last 20 Lines

mlir/include/mlir/IR/OpBase.td

Show First 20 Lines • Show All 1,029 Lines • ▼ Show 20 Lines
// Decorates an attribute as optional. The return type of the generated // Decorates an attribute as optional. The return type of the generated
// attribute accessor method will be Optional<>. // attribute accessor method will be Optional<>.
class OptionalAttr<Attr attr> : Attr<attr.predicate, attr.summary> { class OptionalAttr<Attr attr> : Attr<attr.predicate, attr.summary> {
// Rewrite the attribute to be optional. // Rewrite the attribute to be optional.
// Note: this has to be kept up to date with Attr above. // Note: this has to be kept up to date with Attr above.
let storageType = attr.storageType; let storageType = attr.storageType;
let returnType = "::llvm::Optional<" # attr.returnType #">"; let returnType = "::llvm::Optional<" # attr.returnType #">";
let convertFromStorage = "$_self ? " # returnType # "(" # let convertFromStorage = "$_self ? " # returnType # "(" #
attr.convertFromStorage # ") : (::llvm::None)"; attr.convertFromStorage # ") : (::std::nullopt)";
let valueType = attr.valueType; let valueType = attr.valueType;
let isOptional = 1; let isOptional = 1;
let baseAttr = attr; let baseAttr = attr;
} }
// Default-valued string-based attribute. Wraps the default value in escaped // Default-valued string-based attribute. Wraps the default value in escaped
// quotes. // quotes.
▲ Show 20 Lines • Show All 1,432 Lines • Show Last 20 Lines

mlir/include/mlir/IR/OpDefinition.h

Show All 35 Lines
/// shouldn't be attributed to "not present". /// shouldn't be attributed to "not present".
class OptionalParseResult { class OptionalParseResult {
public: public:
OptionalParseResult() = default; OptionalParseResult() = default;
OptionalParseResult(LogicalResult result) : impl(result) {} OptionalParseResult(LogicalResult result) : impl(result) {}
OptionalParseResult(ParseResult result) : impl(result) {} OptionalParseResult(ParseResult result) : impl(result) {}
OptionalParseResult(const InFlightDiagnostic &) OptionalParseResult(const InFlightDiagnostic &)
: OptionalParseResult(failure()) {} : OptionalParseResult(failure()) {}
OptionalParseResult(std::nullopt_t) : impl(llvm::None) {} OptionalParseResult(std::nullopt_t) : impl(std::nullopt) {}
/// Returns true if we contain a valid ParseResult value. /// Returns true if we contain a valid ParseResult value.
bool has_value() const { return impl.has_value(); } bool has_value() const { return impl.has_value(); }
/// Access the internal ParseResult value. /// Access the internal ParseResult value.
ParseResult value() const { return impl.value(); } ParseResult value() const { return impl.value(); }
ParseResult operator*() const { return value(); } ParseResult operator*() const { return value(); }
Show All 36 Linespublic:
/// Return the operation that this refers to. /// Return the operation that this refers to.
Operation *getOperation() { return state; } Operation *getOperation() { return state; }
/// Return the context this operation belongs to. /// Return the context this operation belongs to.
MLIRContext *getContext() { return getOperation()->getContext(); } MLIRContext *getContext() { return getOperation()->getContext(); }
/// Print the operation to the given stream. /// Print the operation to the given stream.
void print(raw_ostream &os, OpPrintingFlags flags = llvm::None) { void print(raw_ostream &os, OpPrintingFlags flags = std::nullopt) {
state->print(os, flags); state->print(os, flags);
} }
void print(raw_ostream &os, AsmState &asmState) { void print(raw_ostream &os, AsmState &asmState) {
state->print(os, asmState); state->print(os, asmState);
} }
/// Dump this operation. /// Dump this operation.
void dump() { state->dump(); } void dump() { state->dump(); }
▲ Show 20 Lines • Show All 1,877 Lines • Show Last 20 Lines

mlir/include/mlir/IR/OpImplementation.h

Show First 20 Lines • Show All 1,328 Lines • ▼ Show 20 Linespublic:
/// Parse different components, viz., use-info of operand(s), successor(s), /// Parse different components, viz., use-info of operand(s), successor(s),
/// region(s), attribute(s) and function-type, of the generic form of an /// region(s), attribute(s) and function-type, of the generic form of an
/// operation instance and populate the input operation-state 'result' with /// operation instance and populate the input operation-state 'result' with
/// those components. If any of the components is explicitly provided, then /// those components. If any of the components is explicitly provided, then
/// skip parsing that component. /// skip parsing that component.
virtual ParseResult parseGenericOperationAfterOpName( virtual ParseResult parseGenericOperationAfterOpName(
OperationState &result, OperationState &result,
Optional<ArrayRef<UnresolvedOperand>> parsedOperandType = llvm::None, Optional<ArrayRef<UnresolvedOperand>> parsedOperandType = std::nullopt,
Optional<ArrayRef<Block *>> parsedSuccessors = llvm::None, Optional<ArrayRef<Block *>> parsedSuccessors = std::nullopt,
Optional<MutableArrayRef<std::unique_ptr<Region>>> parsedRegions = Optional<MutableArrayRef<std::unique_ptr<Region>>> parsedRegions =
llvm::None, std::nullopt,
Optional<ArrayRef<NamedAttribute>> parsedAttributes = llvm::None, Optional<ArrayRef<NamedAttribute>> parsedAttributes = std::nullopt,
Optional<FunctionType> parsedFnType = llvm::None) = 0; Optional<FunctionType> parsedFnType = std::nullopt) = 0;
/// Parse a single SSA value operand name along with a result number if /// Parse a single SSA value operand name along with a result number if
/// `allowResultNumber` is true. /// `allowResultNumber` is true.
virtual ParseResult parseOperand(UnresolvedOperand &result, virtual ParseResult parseOperand(UnresolvedOperand &result,
bool allowResultNumber = true) = 0; bool allowResultNumber = true) = 0;
/// Parse a single operand if present. /// Parse a single operand if present.
virtual OptionalParseResult virtual OptionalParseResult
▲ Show 20 Lines • Show All 293 Lines • Show Last 20 Lines

mlir/include/mlir/IR/Operation.h

Show First 20 Lines • Show All 234 Lines • ▼ Show 20 Linespublic:
/// Given an operation 'other' that is within the same parent block, return /// Given an operation 'other' that is within the same parent block, return
/// whether the current operation is before 'other' in the operation list /// whether the current operation is before 'other' in the operation list
/// of the parent block. /// of the parent block.
/// Note: This function has an average complexity of O(1), but worst case may /// Note: This function has an average complexity of O(1), but worst case may
/// take O(N) where N is the number of operations within the parent block. /// take O(N) where N is the number of operations within the parent block.
bool isBeforeInBlock(Operation *other); bool isBeforeInBlock(Operation *other);
void print(raw_ostream &os, const OpPrintingFlags &flags = llvm::None); void print(raw_ostream &os, const OpPrintingFlags &flags = std::nullopt);
void print(raw_ostream &os, AsmState &state); void print(raw_ostream &os, AsmState &state);
void dump(); void dump();
//===--------------------------------------------------------------------===// //===--------------------------------------------------------------------===//
// Operands // Operands
//===--------------------------------------------------------------------===// //===--------------------------------------------------------------------===//
/// Replace the current operands of this operation with the ones provided in /// Replace the current operands of this operation with the ones provided in
▲ Show 20 Lines • Show All 608 Lines • Show Last 20 Lines

mlir/include/mlir/IR/OperationSupport.h

Show First 20 Lines • Show All 82 Lines • ▼ Show 20 Lines
protected: protected:
/// This class represents a type erased version of an operation. It contains /// This class represents a type erased version of an operation. It contains
/// all of the components necessary for opaquely interacting with an /// all of the components necessary for opaquely interacting with an
/// operation. If the operation is not registered, some of these components /// operation. If the operation is not registered, some of these components
/// may not be populated. /// may not be populated.
struct Impl { struct Impl {
Impl(StringAttr name) Impl(StringAttr name)
: name(name), dialect(nullptr), interfaceMap(llvm::None) {} : name(name), dialect(nullptr), interfaceMap(std::nullopt) {}
/// The name of the operation. /// The name of the operation.
StringAttr name; StringAttr name;
//===------------------------------------------------------------------===// //===------------------------------------------------------------------===//
// Registered Operation Info // Registered Operation Info
/// The following fields are only populated when the operation is /// The following fields are only populated when the operation is
▲ Show 20 Lines • Show All 449 Lines • ▼ Show 20 Linespublic:
/// Add an attribute with the specified name. /// Add an attribute with the specified name.
void push_back(NamedAttribute newAttribute); void push_back(NamedAttribute newAttribute);
/// Pop last element from list. /// Pop last element from list.
void pop_back() { attrs.pop_back(); } void pop_back() { attrs.pop_back(); }
/// Returns an entry with a duplicate name the list, if it exists, else /// Returns an entry with a duplicate name the list, if it exists, else
/// returns llvm::None. /// returns std::nullopt.
Optional<NamedAttribute> findDuplicate() const; Optional<NamedAttribute> findDuplicate() const;
/// Return a dictionary attribute for the underlying dictionary. This will /// Return a dictionary attribute for the underlying dictionary. This will
/// return an empty dictionary attribute if empty rather than null. /// return an empty dictionary attribute if empty rather than null.
DictionaryAttr getDictionary(MLIRContext *context) const; DictionaryAttr getDictionary(MLIRContext *context) const;
/// Return all of the attributes on this operation. /// Return all of the attributes on this operation.
ArrayRef<NamedAttribute> getAttrs() const; ArrayRef<NamedAttribute> getAttrs() const;
▲ Show 20 Lines • Show All 410 Lines • Show Last 20 Lines

mlir/include/mlir/IR/PatternMatch.h

Show First 20 Lines • Show All 86 Lines • ▼ Show 20 Linespublic:
/// operation instance with this pattern. /// operation instance with this pattern.
ArrayRef<OperationName> getGeneratedOps() const { return generatedOps; } ArrayRef<OperationName> getGeneratedOps() const { return generatedOps; }
/// Return the root node that this pattern matches. Patterns that can match /// Return the root node that this pattern matches. Patterns that can match
/// multiple root types return None. /// multiple root types return None.
Optional<OperationName> getRootKind() const { Optional<OperationName> getRootKind() const {
if (rootKind == RootKind::OperationName) if (rootKind == RootKind::OperationName)
return OperationName::getFromOpaquePointer(rootValue); return OperationName::getFromOpaquePointer(rootValue);
return llvm::None; return std::nullopt;
} }
/// Return the interface ID used to match the root operation of this pattern. /// Return the interface ID used to match the root operation of this pattern.
/// If the pattern does not use an interface ID for deciding the root match, /// If the pattern does not use an interface ID for deciding the root match,
/// this returns None. /// this returns None.
Optional<TypeID> getRootInterfaceID() const { Optional<TypeID> getRootInterfaceID() const {
if (rootKind == RootKind::InterfaceID) if (rootKind == RootKind::InterfaceID)
return TypeID::getFromOpaquePointer(rootValue); return TypeID::getFromOpaquePointer(rootValue);
return llvm::None; return std::nullopt;
} }
/// Return the trait ID used to match the root operation of this pattern. /// Return the trait ID used to match the root operation of this pattern.
/// If the pattern does not use a trait ID for deciding the root match, this /// If the pattern does not use a trait ID for deciding the root match, this
/// returns None. /// returns None.
Optional<TypeID> getRootTraitID() const { Optional<TypeID> getRootTraitID() const {
if (rootKind == RootKind::TraitID) if (rootKind == RootKind::TraitID)
return TypeID::getFromOpaquePointer(rootValue); return TypeID::getFromOpaquePointer(rootValue);
return llvm::None; return std::nullopt;
} }
/// Return the benefit (the inverse of "cost") of matching this pattern. The /// Return the benefit (the inverse of "cost") of matching this pattern. The
/// benefit of a Pattern is always static - rewrites that may have dynamic /// benefit of a Pattern is always static - rewrites that may have dynamic
/// benefit can be instantiated multiple times (different Pattern instances) /// benefit can be instantiated multiple times (different Pattern instances)
/// for each benefit that they may return, and be guarded by different match /// for each benefit that they may return, and be guarded by different match
/// condition predicates. /// condition predicates.
PatternBenefit getBenefit() const { return benefit; } PatternBenefit getBenefit() const { return benefit; }
▲ Show 20 Lines • Show All 338 Lines • ▼ Show 20 Linespublic:
/// This method erases all operations in a block. /// This method erases all operations in a block.
virtual void eraseBlock(Block *block); virtual void eraseBlock(Block *block);
/// Merge the operations of block 'source' into the end of block 'dest'. /// Merge the operations of block 'source' into the end of block 'dest'.
/// 'source's predecessors must either be empty or only contain 'dest`. /// 'source's predecessors must either be empty or only contain 'dest`.
/// 'argValues' is used to replace the block arguments of 'source' after /// 'argValues' is used to replace the block arguments of 'source' after
/// merging. /// merging.
virtual void mergeBlocks(Block *source, Block *dest, virtual void mergeBlocks(Block *source, Block *dest,
ValueRange argValues = llvm::None); ValueRange argValues = std::nullopt);
// Merge the operations of block 'source' before the operation 'op'. Source // Merge the operations of block 'source' before the operation 'op'. Source
// block should not have existing predecessors or successors. // block should not have existing predecessors or successors.
void mergeBlockBefore(Block *source, Operation *op, void mergeBlockBefore(Block *source, Operation *op,
ValueRange argValues = llvm::None); ValueRange argValues = std::nullopt);
/// Split the operations starting at "before" (inclusive) out of the given /// Split the operations starting at "before" (inclusive) out of the given
/// block into a new block, and return it. /// block into a new block, and return it.
virtual Block *splitBlock(Block *block, Block::iterator before); virtual Block *splitBlock(Block *block, Block::iterator before);
/// This method is used to notify the rewriter that an in-place operation /// This method is used to notify the rewriter that an in-place operation
/// modification is about to happen. A call to this function *must* be /// modification is about to happen. A call to this function *must* be
/// followed by a call to either `finalizeRootUpdate` or `cancelRootUpdate`. /// followed by a call to either `finalizeRootUpdate` or `cancelRootUpdate`.
▲ Show 20 Lines • Show All 1,100 Lines • ▼ Show 20 Linespublic:
/// the given arguments. Return a reference to `this` for chaining insertions. /// the given arguments. Return a reference to `this` for chaining insertions.
/// Note: ConstructorArg is necessary here to separate the two variadic lists. /// Note: ConstructorArg is necessary here to separate the two variadic lists.
template <typename... Ts, typename ConstructorArg, template <typename... Ts, typename ConstructorArg,
typename... ConstructorArgs, typename... ConstructorArgs,
typename = std::enable_if_t<sizeof...(Ts) != 0>> typename = std::enable_if_t<sizeof...(Ts) != 0>>
RewritePatternSet &add(ConstructorArg &&arg, ConstructorArgs &&...args) { RewritePatternSet &add(ConstructorArg &&arg, ConstructorArgs &&...args) {
// The following expands a call to emplace_back for each of the pattern // The following expands a call to emplace_back for each of the pattern
// types 'Ts'. // types 'Ts'.
(addImpl<Ts>(/*debugLabels=*/llvm::None, std::forward<ConstructorArg>(arg), (addImpl<Ts>(/*debugLabels=*/std::nullopt,
std::forward<ConstructorArg>(arg),
std::forward<ConstructorArgs>(args)...), std::forward<ConstructorArgs>(args)...),
...); ...);
return *this; return *this;
} }
/// An overload of the above `add` method that allows for attaching a set /// An overload of the above `add` method that allows for attaching a set
/// of debug labels to the attached patterns. This is useful for labeling /// of debug labels to the attached patterns. This is useful for labeling
/// groups of patterns that may be shared between multiple different /// groups of patterns that may be shared between multiple different
/// passes/users. /// passes/users.
▲ Show 20 Lines • Show All 62 Lines • ▼ Show 20 Linespublic:
/// the given arguments. Return a reference to `this` for chaining insertions. /// the given arguments. Return a reference to `this` for chaining insertions.
/// Note: ConstructorArg is necessary here to separate the two variadic lists. /// Note: ConstructorArg is necessary here to separate the two variadic lists.
template <typename... Ts, typename ConstructorArg, template <typename... Ts, typename ConstructorArg,
typename... ConstructorArgs, typename... ConstructorArgs,
typename = std::enable_if_t<sizeof...(Ts) != 0>> typename = std::enable_if_t<sizeof...(Ts) != 0>>
RewritePatternSet &insert(ConstructorArg &&arg, ConstructorArgs &&...args) { RewritePatternSet &insert(ConstructorArg &&arg, ConstructorArgs &&...args) {
// The following expands a call to emplace_back for each of the pattern // The following expands a call to emplace_back for each of the pattern
// types 'Ts'. // types 'Ts'.
(addImpl<Ts>(/*debugLabels=*/llvm::None, arg, args...), ...); (addImpl<Ts>(/*debugLabels=*/std::nullopt, arg, args...), ...);
return *this; return *this;
} }
/// Add an instance of each of the pattern types 'Ts'. Return a reference to /// Add an instance of each of the pattern types 'Ts'. Return a reference to
/// `this` for chaining insertions. /// `this` for chaining insertions.
template <typename... Ts> template <typename... Ts>
RewritePatternSet &insert() { RewritePatternSet &insert() {
(addImpl<Ts>(), ...); (addImpl<Ts>(), ...);
▲ Show 20 Lines • Show All 67 Lines • Show Last 20 Lines

mlir/include/mlir/IR/Region.h

Show First 20 Lines • Show All 332 Lines • ▼ Show 20 Linesclass RegionRange
/// list of regions, a list of region unique pointers, or a list of region /// list of regions, a list of region unique pointers, or a list of region
/// pointers. /// pointers.
using OwnerT = using OwnerT =
PointerUnion<Region *, const std::unique_ptr<Region> *, Region **>; PointerUnion<Region *, const std::unique_ptr<Region> *, Region **>;
public: public:
using RangeBaseT::RangeBaseT; using RangeBaseT::RangeBaseT;
RegionRange(MutableArrayRef<Region> regions = llvm::None); RegionRange(MutableArrayRef<Region> regions = std::nullopt);
template <typename Arg, typename = std::enable_if_t<std::is_constructible< template <typename Arg, typename = std::enable_if_t<std::is_constructible<
ArrayRef<std::unique_ptr<Region>>, Arg>::value>> ArrayRef<std::unique_ptr<Region>>, Arg>::value>>
RegionRange(Arg &&arg) RegionRange(Arg &&arg)
: RegionRange(ArrayRef<std::unique_ptr<Region>>(std::forward<Arg>(arg))) { : RegionRange(ArrayRef<std::unique_ptr<Region>>(std::forward<Arg>(arg))) {
} }
template <typename Arg> template <typename Arg>
RegionRange( RegionRange(
Show All 20 Lines

mlir/include/mlir/IR/SubElementInterfaces.h

Show First 20 Lines • Show All 65 Lines • ▼ Show 20 Linespublic:
/// Register a replacement function for mapping a given attribute or type. A /// Register a replacement function for mapping a given attribute or type. A
/// replacement function must be convertible to any of the following /// replacement function must be convertible to any of the following
/// forms(where `T` is a class derived from `Type` or `Attribute`, and `BaseT` /// forms(where `T` is a class derived from `Type` or `Attribute`, and `BaseT`
/// is either `Type` or `Attribute` respectively): /// is either `Type` or `Attribute` respectively):
/// ///
/// * Optional<BaseT>(T) /// * Optional<BaseT>(T)
/// - This either returns a valid Attribute/Type in the case of success, /// - This either returns a valid Attribute/Type in the case of success,
/// nullptr in the case of failure, or `llvm::None` to signify that /// nullptr in the case of failure, or `std::nullopt` to signify that
/// additional replacement functions may be applied (i.e. this function /// additional replacement functions may be applied (i.e. this function
/// doesn't handle that instance). /// doesn't handle that instance).
/// ///
/// * Optional<std::pair<BaseT, WalkResult>>(T) /// * Optional<std::pair<BaseT, WalkResult>>(T)
/// - Similar to the above, but also allows specifying a WalkResult to /// - Similar to the above, but also allows specifying a WalkResult to
/// control the replacement of sub elements of a given attribute or /// control the replacement of sub elements of a given attribute or
/// type. Returning a `skip` result, for example, will not recursively /// type. Returning a `skip` result, for example, will not recursively
/// process the resultant attribute or type value. /// process the resultant attribute or type value.
▲ Show 20 Lines • Show All 327 Lines • Show Last 20 Lines

mlir/include/mlir/IR/SymbolTable.h

Show First 20 Lines • Show All 295 Lines • ▼ Show 20 Linespublic:
/// 'symbolTableOp'. A reference to the provided symbol table collection is /// 'symbolTableOp'. A reference to the provided symbol table collection is
/// kept by the user map to ensure efficient lookups, thus the lifetime should /// kept by the user map to ensure efficient lookups, thus the lifetime should
/// extend beyond that of this map. /// extend beyond that of this map.
SymbolUserMap(SymbolTableCollection &symbolTable, Operation *symbolTableOp); SymbolUserMap(SymbolTableCollection &symbolTable, Operation *symbolTableOp);
/// Return the users of the provided symbol operation. /// Return the users of the provided symbol operation.
ArrayRef<Operation *> getUsers(Operation *symbol) const { ArrayRef<Operation *> getUsers(Operation *symbol) const {
auto it = symbolToUsers.find(symbol); auto it = symbolToUsers.find(symbol);
return it != symbolToUsers.end() ? it->second.getArrayRef() : llvm::None; return it != symbolToUsers.end() ? it->second.getArrayRef() : std::nullopt;
} }
/// Return true if the given symbol has no uses. /// Return true if the given symbol has no uses.
bool useEmpty(Operation *symbol) const { bool useEmpty(Operation *symbol) const {
return !symbolToUsers.count(symbol); return !symbolToUsers.count(symbol);
} }
/// Replace all of the uses of the given symbol with `newSymbolName`. /// Replace all of the uses of the given symbol with `newSymbolName`.
▲ Show 20 Lines • Show All 80 Lines • Show Last 20 Lines

mlir/include/mlir/IR/TypeRange.h

Show All 30 Lines
/// parameter. /// parameter.
class TypeRange : public llvm::detail::indexed_accessor_range_base< class TypeRange : public llvm::detail::indexed_accessor_range_base<
TypeRange, TypeRange,
llvm::PointerUnion<const Value *, const Type *, llvm::PointerUnion<const Value *, const Type *,
OpOperand *, detail::OpResultImpl *>, OpOperand *, detail::OpResultImpl *>,
Type, Type, Type> { Type, Type, Type> {
public: public:
using RangeBaseT::RangeBaseT; using RangeBaseT::RangeBaseT;
TypeRange(ArrayRef<Type> types = llvm::None); TypeRange(ArrayRef<Type> types = std::nullopt);
explicit TypeRange(OperandRange values); explicit TypeRange(OperandRange values);
explicit TypeRange(ResultRange values); explicit TypeRange(ResultRange values);
explicit TypeRange(ValueRange values); explicit TypeRange(ValueRange values);
template <typename ValueRangeT> template <typename ValueRangeT>
TypeRange(ValueTypeRange<ValueRangeT> values) TypeRange(ValueTypeRange<ValueRangeT> values)
: TypeRange(ValueRange(ValueRangeT(values.begin().getCurrent(), : TypeRange(ValueRange(ValueRangeT(values.begin().getCurrent(),
values.end().getCurrent()))) {} values.end().getCurrent()))) {}
template <typename Arg, typename = std::enable_if_t<std::is_constructible< template <typename Arg, typename = std::enable_if_t<std::is_constructible<
▲ Show 20 Lines • Show All 187 Lines • Show Last 20 Lines

mlir/include/mlir/IR/ValueRange.h

Show First 20 Lines • Show All 116 Lines • ▼ Show 20 Linespublic:
/// and the index within that attribute. The attribute should correspond to a /// and the index within that attribute. The attribute should correspond to a
/// dense i32 array attr. /// dense i32 array attr.
using OperandSegment = std::pair<unsigned, NamedAttribute>; using OperandSegment = std::pair<unsigned, NamedAttribute>;
/// Construct a new mutable range from the given operand, operand start index, /// Construct a new mutable range from the given operand, operand start index,
/// and range length. `operandSegments` is an optional set of operand segments /// and range length. `operandSegments` is an optional set of operand segments
/// to be updated when mutating the operand list. /// to be updated when mutating the operand list.
MutableOperandRange(Operation *owner, unsigned start, unsigned length, MutableOperandRange(Operation *owner, unsigned start, unsigned length,
ArrayRef<OperandSegment> operandSegments = llvm::None); ArrayRef<OperandSegment> operandSegments = std::nullopt);
MutableOperandRange(Operation *owner); MutableOperandRange(Operation *owner);
/// Slice this range into a sub range, with the additional operand segment. /// Slice this range into a sub range, with the additional operand segment.
MutableOperandRange MutableOperandRange
slice(unsigned subStart, unsigned subLen, slice(unsigned subStart, unsigned subLen,
Optional<OperandSegment> segment = llvm::None) const; Optional<OperandSegment> segment = std::nullopt) const;
/// Append the given values to the range. /// Append the given values to the range.
void append(ValueRange values); void append(ValueRange values);
/// Assign this range to the given values. /// Assign this range to the given values.
void assign(ValueRange values); void assign(ValueRange values);
/// Assign the range to the given value. /// Assign the range to the given value.
▲ Show 20 Lines • Show All 224 Lines • ▼ Show 20 Linespublic:
ValueRange(const std::initializer_list<Value> &values) ValueRange(const std::initializer_list<Value> &values)
: ValueRange(ArrayRef<Value>(values)) {} : ValueRange(ArrayRef<Value>(values)) {}
ValueRange(iterator_range<OperandRange::iterator> values) ValueRange(iterator_range<OperandRange::iterator> values)
: ValueRange(OperandRange(values)) {} : ValueRange(OperandRange(values)) {}
ValueRange(iterator_range<ResultRange::iterator> values) ValueRange(iterator_range<ResultRange::iterator> values)
: ValueRange(ResultRange(values)) {} : ValueRange(ResultRange(values)) {}
ValueRange(ArrayRef<BlockArgument> values) ValueRange(ArrayRef<BlockArgument> values)
: ValueRange(ArrayRef<Value>(values.data(), values.size())) {} : ValueRange(ArrayRef<Value>(values.data(), values.size())) {}
ValueRange(ArrayRef<Value> values = llvm::None); ValueRange(ArrayRef<Value> values = std::nullopt);
ValueRange(OperandRange values); ValueRange(OperandRange values);
ValueRange(ResultRange values); ValueRange(ResultRange values);
/// Returns the types of the values within this range. /// Returns the types of the values within this range.
using type_iterator = ValueTypeIterator<iterator>; using type_iterator = ValueTypeIterator<iterator>;
using type_range = ValueTypeRange<ValueRange>; using type_range = ValueTypeRange<ValueRange>;
type_range getTypes() const; type_range getTypes() const;
type_range getType() const; type_range getType() const;
Show All 14 Lines

mlir/include/mlir/Interfaces/ControlFlowInterfaces.h

Show First 20 Lines • Show All 196 Lines • ▼ Show 20 Linespublic:
/// Return the lower bound. /// Return the lower bound.
unsigned getLowerBound() const { return lower; } unsigned getLowerBound() const { return lower; }
/// Return the upper bound. /// Return the upper bound.
Optional<unsigned> getUpperBound() const { return upper; } Optional<unsigned> getUpperBound() const { return upper; }
/// Returns the unknown invocation bounds, i.e., there is no information on /// Returns the unknown invocation bounds, i.e., there is no information on
/// how many times a region may be invoked. /// how many times a region may be invoked.
static InvocationBounds getUnknown() { return {0, llvm::None}; } static InvocationBounds getUnknown() { return {0, std::nullopt}; }
private: private:
/// The minimum number of times the successor region will be invoked. /// The minimum number of times the successor region will be invoked.
unsigned lower; unsigned lower;
/// The maximum number of times the successor region will be invoked or `None` /// The maximum number of times the successor region will be invoked or `None`
/// if an upper bound is not known. /// if an upper bound is not known.
Optional<unsigned> upper; Optional<unsigned> upper;
}; };
Show All 18 Lines
/// Returns true if the given operation is either annotated with the /// Returns true if the given operation is either annotated with the
/// `ReturnLike` trait or implements the `RegionBranchTerminatorOpInterface`. /// `ReturnLike` trait or implements the `RegionBranchTerminatorOpInterface`.
bool isRegionReturnLike(Operation *operation); bool isRegionReturnLike(Operation *operation);
/// Returns the mutable operands that are passed to the region with the given /// Returns the mutable operands that are passed to the region with the given
/// `regionIndex`. If the operation does not implement the /// `regionIndex`. If the operation does not implement the
/// `RegionBranchTerminatorOpInterface` and is not marked as `ReturnLike`, the /// `RegionBranchTerminatorOpInterface` and is not marked as `ReturnLike`, the
/// result will be `llvm::None`. In all other cases, the resulting /// result will be `std::nullopt`. In all other cases, the resulting
/// `OperandRange` represents all operands that are passed to the specified /// `OperandRange` represents all operands that are passed to the specified
/// successor region. If `regionIndex` is `llvm::None`, all operands that are /// successor region. If `regionIndex` is `std::nullopt`, all operands that are
/// passed to the parent operation will be returned. /// passed to the parent operation will be returned.
Optional<MutableOperandRange> Optional<MutableOperandRange>
getMutableRegionBranchSuccessorOperands(Operation *operation, getMutableRegionBranchSuccessorOperands(Operation *operation,
Optional<unsigned> regionIndex); Optional<unsigned> regionIndex);
/// Returns the read only operands that are passed to the region with the given /// Returns the read only operands that are passed to the region with the given
/// `regionIndex`. See `getMutableRegionBranchSuccessorOperands` for more /// `regionIndex`. See `getMutableRegionBranchSuccessorOperands` for more
/// information. /// information.
Show All 37 Lines

mlir/include/mlir/Interfaces/ControlFlowInterfaces.td

Show First 20 Lines • Show All 73 Lines • ▼ Show 20 Lines InterfaceMethod<[{
(ins "unsigned":$operandIndex), [{ (ins "unsigned":$operandIndex), [{
::mlir::Operation *opaqueOp = $_op; ::mlir::Operation *opaqueOp = $_op;
for (unsigned i = 0, e = opaqueOp->getNumSuccessors(); i != e; ++i) { for (unsigned i = 0, e = opaqueOp->getNumSuccessors(); i != e; ++i) {
if (::llvm::Optional<::mlir::BlockArgument> arg = ::mlir::detail::getBranchSuccessorArgument( if (::llvm::Optional<::mlir::BlockArgument> arg = ::mlir::detail::getBranchSuccessorArgument(
$_op.getSuccessorOperands(i), operandIndex, $_op.getSuccessorOperands(i), operandIndex,
opaqueOp->getSuccessor(i))) opaqueOp->getSuccessor(i)))
return arg; return arg;
} }
return ::llvm::None; return ::std::nullopt;
}] }]
>, >,
InterfaceMethod<[{ InterfaceMethod<[{
Returns the successor that would be chosen with the given constant Returns the successor that would be chosen with the given constant
operands. Returns nullptr if a single successor could not be chosen. operands. Returns nullptr if a single successor could not be chosen.
}], }],
"::mlir::Block *", "getSuccessorForOperands", "::mlir::Block *", "getSuccessorForOperands",
(ins "::llvm::ArrayRef<::mlir::Attribute>":$operands), [{}], (ins "::llvm::ArrayRef<::mlir::Attribute>":$operands), [{}],
▲ Show 20 Lines • Show All 75 Lines • ▼ Show 20 Lines InterfaceMethod<[{
"::llvm::SmallVectorImpl<::mlir::RegionSuccessor> &":$regions) "::llvm::SmallVectorImpl<::mlir::RegionSuccessor> &":$regions)
>, >,
InterfaceMethod<[{ InterfaceMethod<[{
Populates `invocationBounds` with the minimum and maximum number of Populates `invocationBounds` with the minimum and maximum number of
times this operation will invoke the attached regions (assuming the times this operation will invoke the attached regions (assuming the
regions yield normally, i.e. do not abort or invoke an infinite loop). regions yield normally, i.e. do not abort or invoke an infinite loop).
The minimum number of invocations is at least 0. If the maximum number The minimum number of invocations is at least 0. If the maximum number
of invocations cannot be statically determined, then it will not have a of invocations cannot be statically determined, then it will not have a
value (i.e., it is set to `llvm::None`). value (i.e., it is set to `std::nullopt`).
`operands` is a set of optional attributes that either correspond to `operands` is a set of optional attributes that either correspond to
constant values for each operand of this operation or null if that constant values for each operand of this operation or null if that
operand is not a constant. operand is not a constant.
This method may be called speculatively on operations where the provided This method may be called speculatively on operations where the provided
operands are not necessarily the same as the operation's current operands are not necessarily the same as the operation's current
operands. This may occur in analyses that wish to determine "what would operands. This may occur in analyses that wish to determine "what would
▲ Show 20 Lines • Show All 94 Lines • Show Last 20 Lines

mlir/include/mlir/Interfaces/LoopLikeInterface.td

Show First 20 Lines • Show All 44 Lines • ▼ Show 20 Lines InterfaceMethod<[{
}], }],
"void", "moveOutOfLoop", "void", "moveOutOfLoop",
(ins "::mlir::Operation *":$op), [{}], [{ (ins "::mlir::Operation *":$op), [{}], [{
op->moveBefore($_op); op->moveBefore($_op);
}] }]
>, >,
InterfaceMethod<[{ InterfaceMethod<[{
If there is a single induction variable return it, otherwise return If there is a single induction variable return it, otherwise return
llvm::None. std::nullopt.
}], }],
/*retTy=*/"::llvm::Optional<::mlir::Value>", /*retTy=*/"::llvm::Optional<::mlir::Value>",
/*methodName=*/"getSingleInductionVar", /*methodName=*/"getSingleInductionVar",
/*args=*/(ins), /*args=*/(ins),
/*methodBody=*/"", /*methodBody=*/"",
/*defaultImplementation=*/[{ /*defaultImplementation=*/[{
return llvm::None; return std::nullopt;
}] }]
>, >,
InterfaceMethod<[{ InterfaceMethod<[{
Return the single lower bound value or attribute if it exists, otherwise Return the single lower bound value or attribute if it exists, otherwise
return llvm::None. return std::nullopt.
}], }],
/*retTy=*/"::llvm::Optional<::mlir::OpFoldResult>", /*retTy=*/"::llvm::Optional<::mlir::OpFoldResult>",
/*methodName=*/"getSingleLowerBound", /*methodName=*/"getSingleLowerBound",
/*args=*/(ins), /*args=*/(ins),
/*methodBody=*/"", /*methodBody=*/"",
/*defaultImplementation=*/[{ /*defaultImplementation=*/[{
return llvm::None; return std::nullopt;
}] }]
>, >,
InterfaceMethod<[{ InterfaceMethod<[{
Return the single step value or attribute if it exists, otherwise Return the single step value or attribute if it exists, otherwise
return llvm::None. return std::nullopt.
}], }],
/*retTy=*/"::llvm::Optional<::mlir::OpFoldResult>", /*retTy=*/"::llvm::Optional<::mlir::OpFoldResult>",
/*methodName=*/"getSingleStep", /*methodName=*/"getSingleStep",
/*args=*/(ins), /*args=*/(ins),
/*methodBody=*/"", /*methodBody=*/"",
/*defaultImplementation=*/[{ /*defaultImplementation=*/[{
return llvm::None; return std::nullopt;
}] }]
>, >,
InterfaceMethod<[{ InterfaceMethod<[{
Return the single upper bound value or attribute if it exists, otherwise Return the single upper bound value or attribute if it exists, otherwise
return llvm::None. return std::nullopt.
}], }],
/*retTy=*/"::llvm::Optional<::mlir::OpFoldResult>", /*retTy=*/"::llvm::Optional<::mlir::OpFoldResult>",
/*methodName=*/"getSingleUpperBound", /*methodName=*/"getSingleUpperBound",
/*args=*/(ins), /*args=*/(ins),
/*methodBody=*/"", /*methodBody=*/"",
/*defaultImplementation=*/[{ /*defaultImplementation=*/[{
return llvm::None; return std::nullopt;
}] }]
>, >,
]; ];
} }
#endif // MLIR_INTERFACES_LOOPLIKEINTERFACE #endif // MLIR_INTERFACES_LOOPLIKEINTERFACE

mlir/include/mlir/Interfaces/SideEffectInterfaceBase.td

Show First 20 Lines • Show All 109 Lines • ▼ Show 20 Lines let extraClassDeclaration = [{
getEffectOnValue(::mlir::Value value) { getEffectOnValue(::mlir::Value value) {
llvm::SmallVector<::mlir::SideEffects::EffectInstance< llvm::SmallVector<::mlir::SideEffects::EffectInstance<
}] # baseEffect # [{>, 4> effects; }] # baseEffect # [{>, 4> effects;
getEffects(effects); getEffects(effects);
auto it = llvm::find_if(effects, [&](auto &it) { auto it = llvm::find_if(effects, [&](auto &it) {
return isa<Effect>(it.getEffect()) && it.getValue() == value; return isa<Effect>(it.getEffect()) && it.getValue() == value;
}); });
if (it == effects.end()) if (it == effects.end())
return llvm::None; return std::nullopt;
return *it; return *it;
} }
/// Collect all of the effect instances that operate on the provided symbol /// Collect all of the effect instances that operate on the provided symbol
/// reference and place them in 'effects'. /// reference and place them in 'effects'.
void getEffectsOnSymbol(::mlir::SymbolRefAttr value, void getEffectsOnSymbol(::mlir::SymbolRefAttr value,
llvm::SmallVectorImpl<::mlir::SideEffects::EffectInstance< llvm::SmallVectorImpl<::mlir::SideEffects::EffectInstance<
}] # baseEffect # [{>> & effects) { }] # baseEffect # [{>> & effects) {
▲ Show 20 Lines • Show All 60 Lines • Show Last 20 Lines

mlir/include/mlir/Pass/AnalysisManager.h

Show First 20 Lines • Show All 165 Lines • ▼ Show 20 Lines getAnalysis(PassInstrumentor *pi, AnalysisManager &am) {
return getAnalysisImpl<AnalysisT, OpT>(pi, cast<OpT>(ir), am); return getAnalysisImpl<AnalysisT, OpT>(pi, cast<OpT>(ir), am);
} }
/// Get a cached analysis instance if one exists, otherwise return null. /// Get a cached analysis instance if one exists, otherwise return null.
template <typename AnalysisT> template <typename AnalysisT>
Optional<std::reference_wrapper<AnalysisT>> getCachedAnalysis() const { Optional<std::reference_wrapper<AnalysisT>> getCachedAnalysis() const {
auto res = analyses.find(TypeID::get<AnalysisT>()); auto res = analyses.find(TypeID::get<AnalysisT>());
if (res == analyses.end()) if (res == analyses.end())
return llvm::None; return std::nullopt;
return {static_cast<AnalysisModel<AnalysisT> &>(*res->second).analysis}; return {static_cast<AnalysisModel<AnalysisT> &>(*res->second).analysis};
} }
/// Returns the operation that this analysis map represents. /// Returns the operation that this analysis map represents.
Operation *getOperation() const { return ir; } Operation *getOperation() const { return ir; }
/// Clear any held analyses. /// Clear any held analyses.
void clear() { analyses.clear(); } void clear() { analyses.clear(); }
▲ Show 20 Lines • Show All 164 Lines • ▼ Show 20 Linespublic:
/// Query for a cached analysis of a child operation, or return null. /// Query for a cached analysis of a child operation, or return null.
template <typename AnalysisT> template <typename AnalysisT>
Optional<std::reference_wrapper<AnalysisT>> Optional<std::reference_wrapper<AnalysisT>>
getCachedChildAnalysis(Operation *op) const { getCachedChildAnalysis(Operation *op) const {
assert(op->getParentOp() == impl->getOperation()); assert(op->getParentOp() == impl->getOperation());
auto it = impl->childAnalyses.find(op); auto it = impl->childAnalyses.find(op);
if (it == impl->childAnalyses.end()) if (it == impl->childAnalyses.end())
return llvm::None; return std::nullopt;
return it->second->analyses.getCachedAnalysis<AnalysisT>(); return it->second->analyses.getCachedAnalysis<AnalysisT>();
} }
/// Get an analysis manager for the given operation, which must be a proper /// Get an analysis manager for the given operation, which must be a proper
/// descendant of the current operation represented by this analysis manager. /// descendant of the current operation represented by this analysis manager.
AnalysisManager nest(Operation *op); AnalysisManager nest(Operation *op);
/// Invalidate any non preserved analyses, /// Invalidate any non preserved analyses,
▲ Show 20 Lines • Show All 49 Lines • Show Last 20 Lines

mlir/include/mlir/Pass/Pass.h

Show First 20 Lines • Show All 153 Lines • ▼ Show 20 Linespublic:
/// Returns the thread sibling of this pass, or the pass itself it has no /// Returns the thread sibling of this pass, or the pass itself it has no
/// sibling. See `getThreadingSibling()` for details. /// sibling. See `getThreadingSibling()` for details.
const Pass *getThreadingSiblingOrThis() const { const Pass *getThreadingSiblingOrThis() const {
return threadingSibling ? threadingSibling : this; return threadingSibling ? threadingSibling : this;
} }
protected: protected:
explicit Pass(TypeID passID, Optional<StringRef> opName = llvm::None) explicit Pass(TypeID passID, Optional<StringRef> opName = std::nullopt)
: passID(passID), opName(opName) {} : passID(passID), opName(opName) {}
Pass(const Pass &other) : Pass(other.passID, other.opName) {} Pass(const Pass &other) : Pass(other.passID, other.opName) {}
/// Returns the current pass state. /// Returns the current pass state.
detail::PassExecutionState &getPassState() { detail::PassExecutionState &getPassState() {
assert(passState && "pass state was never initialized"); assert(passState && "pass state was never initialized");
return *passState; return *passState;
} }
▲ Show 20 Lines • Show All 292 Lines • Show Last 20 Lines

mlir/include/mlir/Pass/PassOptions.h

Show First 20 Lines • Show All 116 Lines • ▼ Show 20 Linesprivate:
struct GenericOptionParser : public llvm::cl::parser<DataType> { struct GenericOptionParser : public llvm::cl::parser<DataType> {
using llvm::cl::parser<DataType>::parser; using llvm::cl::parser<DataType>::parser;
/// Returns an argument name that maps to the specified value. /// Returns an argument name that maps to the specified value.
Optional<StringRef> findArgStrForValue(const DataType &value) { Optional<StringRef> findArgStrForValue(const DataType &value) {
for (auto &it : this->Values) for (auto &it : this->Values)
if (it.V.compare(value)) if (it.V.compare(value))
return it.Name; return it.Name;
return llvm::None; return std::nullopt;
} }
}; };
/// Utility methods for printing option values. /// Utility methods for printing option values.
template <typename DataT> template <typename DataT>
static void printValue(raw_ostream &os, GenericOptionParser<DataT> &parser, static void printValue(raw_ostream &os, GenericOptionParser<DataT> &parser,
const DataT &value) { const DataT &value) {
if (Optional<StringRef> argStr = parser.findArgStrForValue(value)) if (Optional<StringRef> argStr = parser.findArgStrForValue(value))
▲ Show 20 Lines • Show All 372 Lines • Show Last 20 Lines

mlir/include/mlir/Rewrite/FrozenRewritePatternSet.h

Show First 20 Lines • Show All 43 Lines • ▼ Show 20 Linespublic:
/// `enabledPatternLabels` is a set of labels used to filter out input /// `enabledPatternLabels` is a set of labels used to filter out input
/// patterns that do not have one of the labels in this set. Debug labels must /// patterns that do not have one of the labels in this set. Debug labels must
/// be set explicitly on patterns or when adding them with /// be set explicitly on patterns or when adding them with
/// `RewritePatternSet::addWithLabel`. Debug names may be empty, but patterns /// `RewritePatternSet::addWithLabel`. Debug names may be empty, but patterns
/// created with `RewritePattern::create` have their default debug name set to /// created with `RewritePattern::create` have their default debug name set to
/// their type name. /// their type name.
FrozenRewritePatternSet( FrozenRewritePatternSet(
RewritePatternSet &&patterns, RewritePatternSet &&patterns,
ArrayRef<std::string> disabledPatternLabels = llvm::None, ArrayRef<std::string> disabledPatternLabels = std::nullopt,
ArrayRef<std::string> enabledPatternLabels = llvm::None); ArrayRef<std::string> enabledPatternLabels = std::nullopt);
/// Return the op specific native patterns held by this list. /// Return the op specific native patterns held by this list.
const OpSpecificNativePatternListT &getOpSpecificNativePatterns() const { const OpSpecificNativePatternListT &getOpSpecificNativePatterns() const {
return impl->nativeOpSpecificPatternMap; return impl->nativeOpSpecificPatternMap;
} }
/// Return the "match any" native patterns held by this list. /// Return the "match any" native patterns held by this list.
iterator_range<llvm::pointee_iterator<NativePatternListT::const_iterator>> iterator_range<llvm::pointee_iterator<NativePatternListT::const_iterator>>
Show All 40 Lines

mlir/include/mlir/Support/StorageUniquer.h

Show First 20 Lines • Show All 91 Lines • ▼ Show 20 Linespublic:
/// derived types. /// derived types.
class StorageAllocator { class StorageAllocator {
public: public:
/// Copy the specified array of elements into memory managed by our bump /// Copy the specified array of elements into memory managed by our bump
/// pointer allocator. This assumes the elements are all PODs. /// pointer allocator. This assumes the elements are all PODs.
template <typename T> template <typename T>
ArrayRef<T> copyInto(ArrayRef<T> elements) { ArrayRef<T> copyInto(ArrayRef<T> elements) {
if (elements.empty()) if (elements.empty())
return llvm::None; return std::nullopt;
auto result = allocator.Allocate<T>(elements.size()); auto result = allocator.Allocate<T>(elements.size());
std::uninitialized_copy(elements.begin(), elements.end(), result); std::uninitialized_copy(elements.begin(), elements.end(), result);
return ArrayRef<T>(result, elements.size()); return ArrayRef<T>(result, elements.size());
} }
/// Copy the provided string into memory managed by our bump pointer /// Copy the provided string into memory managed by our bump pointer
/// allocator. /// allocator.
StringRef copyInto(StringRef str) { StringRef copyInto(StringRef str) {
▲ Show 20 Lines • Show All 64 Lines • ▼ Show 20 Lines auto ctorFn = [&](StorageAllocator &allocator) {
if (initFn) if (initFn)
initFn(storage); initFn(storage);
return storage; return storage;
}; };
registerSingletonImpl(id, ctorFn); registerSingletonImpl(id, ctorFn);
} }
template <typename Storage> template <typename Storage>
void registerSingletonStorageType(TypeID id) { void registerSingletonStorageType(TypeID id) {
registerSingletonStorageType<Storage>(id, llvm::None); registerSingletonStorageType<Storage>(id, std::nullopt);
} }
/// Utility override when the storage type represents the type id. /// Utility override when the storage type represents the type id.
template <typename Storage> template <typename Storage>
void registerSingletonStorageType(function_ref<void(Storage *)> initFn = {}) { void registerSingletonStorageType(function_ref<void(Storage *)> initFn = {}) {
registerSingletonStorageType<Storage>(TypeID::get<Storage>(), initFn); registerSingletonStorageType<Storage>(TypeID::get<Storage>(), initFn);
} }
/// Gets a uniqued instance of 'Storage'. 'id' is the type id used when /// Gets a uniqued instance of 'Storage'. 'id' is the type id used when
▲ Show 20 Lines • Show All 151 Lines • Show Last 20 Lines

mlir/include/mlir/Support/Timing.h

Show First 20 Lines • Show All 130 Lines • ▼ Show 20 Linesprotected:
// Allow `Timer` access to the protected callbacks. // Allow `Timer` access to the protected callbacks.
friend class Timer; friend class Timer;
//===--------------------------------------------------------------------===// //===--------------------------------------------------------------------===//
// Callbacks // Callbacks
// //
// See the corresponding functions in `Timer` for additional details. // See the corresponding functions in `Timer` for additional details.
/// Return the root timer. Implementations should return `llvm::None` if the /// Return the root timer. Implementations should return `std::nullopt` if the
/// collection of timing samples is disabled. This will cause the timers /// collection of timing samples is disabled. This will cause the timers
/// constructed from the manager to be tombstones which can be skipped /// constructed from the manager to be tombstones which can be skipped
/// quickly. /// quickly.
virtual Optional<void *> rootTimer() = 0; virtual Optional<void *> rootTimer() = 0;
/// Start the timer with the given handle. /// Start the timer with the given handle.
virtual void startTimer(void *handle) = 0; virtual void startTimer(void *handle) = 0;
▲ Show 20 Lines • Show All 276 Lines • Show Last 20 Lines

mlir/include/mlir/TableGen/Operator.h

Show First 20 Lines • Show All 253 Lines • ▼ Show 20 Linespublic:
void print(llvm::raw_ostream &os) const; void print(llvm::raw_ostream &os) const;
// Return whether all the result types are known. // Return whether all the result types are known.
bool allResultTypesKnown() const { return allResultsHaveKnownTypes; }; bool allResultTypesKnown() const { return allResultsHaveKnownTypes; };
// Pair representing either a index to an argument or a type constraint. Only // Pair representing either a index to an argument or a type constraint. Only
// one of these entries should have the non-default value. // one of these entries should have the non-default value.
struct ArgOrType { struct ArgOrType {
explicit ArgOrType(int index) : index(index), constraint(None) {} explicit ArgOrType(int index) : index(index), constraint(std::nullopt) {}
explicit ArgOrType(TypeConstraint constraint) explicit ArgOrType(TypeConstraint constraint)
: index(None), constraint(constraint) {} : index(std::nullopt), constraint(constraint) {}
bool isArg() const { bool isArg() const {
assert(constraint.has_value() ^ index.has_value()); assert(constraint.has_value() ^ index.has_value());
return index.has_value(); return index.has_value();
} }
bool isType() const { bool isType() const {
assert(constraint.has_value() ^ index.has_value()); assert(constraint.has_value() ^ index.has_value());
return constraint.has_value(); return constraint.has_value();
} }
▲ Show 20 Lines • Show All 109 Lines • Show Last 20 Lines

mlir/include/mlir/TableGen/Pattern.h

Show First 20 Lines • Show All 276 Lines • ▼ Show 20 Lines private:
// * Operand: op operand // * Operand: op operand
// * Result: op result // * Result: op result
// * Value: a value not attached to an op (e.g., from NativeCodeCall) // * Value: a value not attached to an op (e.g., from NativeCodeCall)
// * MultipleValues: a pack of values not attached to an op (e.g., from // * MultipleValues: a pack of values not attached to an op (e.g., from
// NativeCodeCall). This kind supports indexing. // NativeCodeCall). This kind supports indexing.
enum class Kind : uint8_t { Attr, Operand, Result, Value, MultipleValues }; enum class Kind : uint8_t { Attr, Operand, Result, Value, MultipleValues };
// Creates a SymbolInfo instance. `dagAndConstant` is only used for `Attr` // Creates a SymbolInfo instance. `dagAndConstant` is only used for `Attr`
// and `Operand` so should be llvm::None for `Result` and `Value` kind. // and `Operand` so should be std::nullopt for `Result` and `Value` kind.
SymbolInfo(const Operator *op, Kind kind, SymbolInfo(const Operator *op, Kind kind,
Optional<DagAndConstant> dagAndConstant); Optional<DagAndConstant> dagAndConstant);
// Static methods for creating SymbolInfo. // Static methods for creating SymbolInfo.
static SymbolInfo getAttr(const Operator *op, int index) { static SymbolInfo getAttr(const Operator *op, int index) {
return SymbolInfo(op, Kind::Attr, DagAndConstant(nullptr, index)); return SymbolInfo(op, Kind::Attr, DagAndConstant(nullptr, index));
} }
static SymbolInfo getAttr() { static SymbolInfo getAttr() {
return SymbolInfo(nullptr, Kind::Attr, llvm::None); return SymbolInfo(nullptr, Kind::Attr, std::nullopt);
} }
static SymbolInfo getOperand(DagNode node, const Operator *op, int index) { static SymbolInfo getOperand(DagNode node, const Operator *op, int index) {
return SymbolInfo(op, Kind::Operand, return SymbolInfo(op, Kind::Operand,
DagAndConstant(node.getAsOpaquePointer(), index)); DagAndConstant(node.getAsOpaquePointer(), index));
} }
static SymbolInfo getResult(const Operator *op) { static SymbolInfo getResult(const Operator *op) {
return SymbolInfo(op, Kind::Result, llvm::None); return SymbolInfo(op, Kind::Result, std::nullopt);
} }
static SymbolInfo getValue() { static SymbolInfo getValue() {
return SymbolInfo(nullptr, Kind::Value, llvm::None); return SymbolInfo(nullptr, Kind::Value, std::nullopt);
} }
static SymbolInfo getMultipleValues(int numValues) { static SymbolInfo getMultipleValues(int numValues) {
return SymbolInfo(nullptr, Kind::MultipleValues, return SymbolInfo(nullptr, Kind::MultipleValues,
DagAndConstant(nullptr, numValues)); DagAndConstant(nullptr, numValues));
} }
// Returns the number of static values this symbol corresponds to. // Returns the number of static values this symbol corresponds to.
// A static value is an operand/result declared in ODS. Normally a symbol // A static value is an operand/result declared in ODS. Normally a symbol
▲ Show 20 Lines • Show All 241 Lines • Show Last 20 Lines

mlir/include/mlir/Tools/PDLL/AST/Diagnostic.h

Show All 38 Linespublic:
/// Return the location of this diagnostic. /// Return the location of this diagnostic.
SMRange getLocation() const { return location; } SMRange getLocation() const { return location; }
/// Return the notes of this diagnostic. /// Return the notes of this diagnostic.
auto getNotes() const { return llvm::make_pointee_range(notes); } auto getNotes() const { return llvm::make_pointee_range(notes); }
/// Attach a note to this diagnostic. /// Attach a note to this diagnostic.
Diagnostic &attachNote(const Twine &msg, Diagnostic &attachNote(const Twine &msg,
Optional<SMRange> noteLoc = llvm::None) { Optional<SMRange> noteLoc = std::nullopt) {
assert(getSeverity() != Severity::DK_Note && assert(getSeverity() != Severity::DK_Note &&
"cannot attach a Note to a Note"); "cannot attach a Note to a Note");
notes.emplace_back( notes.emplace_back(
new Diagnostic(Severity::DK_Note, noteLoc.value_or(location), msg)); new Diagnostic(Severity::DK_Note, noteLoc.value_or(location), msg));
return *notes.back(); return *notes.back();
} }
/// Allow an inflight diagnostic to be converted to 'failure', otherwise /// Allow an inflight diagnostic to be converted to 'failure', otherwise
▲ Show 20 Lines • Show All 127 Lines • Show Last 20 Lines

mlir/include/mlir/Tools/PDLL/AST/Nodes.h

Show First 20 Lines • Show All 880 Lines • ▼ Show 20 Linespublic:
} }
/// Create a PDLL constraint with the given body. /// Create a PDLL constraint with the given body.
static UserConstraintDecl *createPDLL(Context &ctx, const Name &name, static UserConstraintDecl *createPDLL(Context &ctx, const Name &name,
ArrayRef<VariableDecl *> inputs, ArrayRef<VariableDecl *> inputs,
ArrayRef<VariableDecl *> results, ArrayRef<VariableDecl *> results,
const CompoundStmt *body, const CompoundStmt *body,
Type resultType) { Type resultType) {
return createImpl(ctx, name, inputs, /*nativeInputTypes=*/llvm::None, return createImpl(ctx, name, inputs, /*nativeInputTypes=*/std::nullopt,
results, /*codeBlock=*/llvm::None, body, resultType); results, /*codeBlock=*/std::nullopt, body, resultType);
} }
/// Return the name of the constraint. /// Return the name of the constraint.
const Name &getName() const { return *Decl::getName(); } const Name &getName() const { return *Decl::getName(); }
/// Return the input arguments of this constraint. /// Return the input arguments of this constraint.
MutableArrayRef<VariableDecl *> getInputs() { MutableArrayRef<VariableDecl *> getInputs() {
return {getTrailingObjects<VariableDecl *>(), numInputs}; return {getTrailingObjects<VariableDecl *>(), numInputs};
▲ Show 20 Lines • Show All 104 Lines • ▼ Show 20 Lines
class OpNameDecl : public Node::NodeBase<OpNameDecl, Decl> { class OpNameDecl : public Node::NodeBase<OpNameDecl, Decl> {
public: public:
static OpNameDecl *create(Context &ctx, const Name &name); static OpNameDecl *create(Context &ctx, const Name &name);
static OpNameDecl *create(Context &ctx, SMRange loc); static OpNameDecl *create(Context &ctx, SMRange loc);
/// Return the name of this operation, or none if the name is unknown. /// Return the name of this operation, or none if the name is unknown.
Optional<StringRef> getName() const { Optional<StringRef> getName() const {
const Name *name = Decl::getName(); const Name *name = Decl::getName();
return name ? Optional<StringRef>(name->getName()) : llvm::None; return name ? Optional<StringRef>(name->getName()) : std::nullopt;
} }
private: private:
explicit OpNameDecl(const Name &name) : Base(name.getLoc(), &name) {} explicit OpNameDecl(const Name &name) : Base(name.getLoc(), &name) {}
explicit OpNameDecl(SMRange loc) : Base(loc) {} explicit OpNameDecl(SMRange loc) : Base(loc) {}
}; };
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 68 Lines • ▼ Show 20 Linespublic:
} }
/// Create a PDLL rewrite with the given body. /// Create a PDLL rewrite with the given body.
static UserRewriteDecl *createPDLL(Context &ctx, const Name &name, static UserRewriteDecl *createPDLL(Context &ctx, const Name &name,
ArrayRef<VariableDecl *> inputs, ArrayRef<VariableDecl *> inputs,
ArrayRef<VariableDecl *> results, ArrayRef<VariableDecl *> results,
const CompoundStmt *body, const CompoundStmt *body,
Type resultType) { Type resultType) {
return createImpl(ctx, name, inputs, results, /*codeBlock=*/llvm::None, return createImpl(ctx, name, inputs, results, /*codeBlock=*/std::nullopt,
body, resultType); body, resultType);
} }
/// Return the name of the rewrite. /// Return the name of the rewrite.
const Name &getName() const { return *Decl::getName(); } const Name &getName() const { return *Decl::getName(); }
/// Return the input arguments of this rewrite. /// Return the input arguments of this rewrite.
MutableArrayRef<VariableDecl *> getInputs() { MutableArrayRef<VariableDecl *> getInputs() {
▲ Show 20 Lines • Show All 232 Lines • Show Last 20 Lines

mlir/include/mlir/Tools/PDLL/AST/Types.h

Show First 20 Lines • Show All 155 Lines • ▼ Show 20 Lines
/// This class represents a PDLL type that corresponds to an mlir::Operation. /// This class represents a PDLL type that corresponds to an mlir::Operation.
class OperationType : public Type::TypeBase<detail::OperationTypeStorage> { class OperationType : public Type::TypeBase<detail::OperationTypeStorage> {
public: public:
using Base::Base; using Base::Base;
/// Return an instance of the Operation type with an optional operation name. /// Return an instance of the Operation type with an optional operation name.
/// If no name is provided, this type may refer to any operation. /// If no name is provided, this type may refer to any operation.
static OperationType get(Context &context, static OperationType get(Context &context,
Optional<StringRef> name = llvm::None, Optional<StringRef> name = std::nullopt,
const ods::Operation *odsOp = nullptr); const ods::Operation *odsOp = nullptr);
/// Return the name of this operation type, or None if it doesn't have on. /// Return the name of this operation type, or None if it doesn't have on.
Optional<StringRef> getName() const; Optional<StringRef> getName() const;
/// Return the ODS operation that this type refers to, or nullptr if the ODS /// Return the ODS operation that this type refers to, or nullptr if the ODS
/// operation is unknown. /// operation is unknown.
const ods::Operation *getODSOperation() const; const ods::Operation *getODSOperation() const;
▲ Show 20 Lines • Show All 69 Lines • ▼ Show 20 Lines
class TupleType : public Type::TypeBase<detail::TupleTypeStorage> { class TupleType : public Type::TypeBase<detail::TupleTypeStorage> {
public: public:
using Base::Base; using Base::Base;
/// Return an instance of the Tuple type. /// Return an instance of the Tuple type.
static TupleType get(Context &context, ArrayRef<Type> elementTypes, static TupleType get(Context &context, ArrayRef<Type> elementTypes,
ArrayRef<StringRef> elementNames); ArrayRef<StringRef> elementNames);
static TupleType get(Context &context, static TupleType get(Context &context,
ArrayRef<Type> elementTypes = llvm::None); ArrayRef<Type> elementTypes = std::nullopt);
/// Return the element types of this tuple. /// Return the element types of this tuple.
ArrayRef<Type> getElementTypes() const; ArrayRef<Type> getElementTypes() const;
/// Return the element names of this tuple. /// Return the element names of this tuple.
ArrayRef<StringRef> getElementNames() const; ArrayRef<StringRef> getElementNames() const;
/// Return the number of elements within this tuple. /// Return the number of elements within this tuple.
▲ Show 20 Lines • Show All 68 Lines • Show Last 20 Lines

mlir/include/mlir/Tools/mlir-translate/Translation.h

Show First 20 Lines • Show All 94 Lines • ▼ Show 20 Lines
/// } /// }
/// } // namespace mlir /// } // namespace mlir
/// ///
/// \{ /// \{
struct TranslateToMLIRRegistration { struct TranslateToMLIRRegistration {
TranslateToMLIRRegistration( TranslateToMLIRRegistration(
llvm::StringRef name, llvm::StringRef description, llvm::StringRef name, llvm::StringRef description,
const TranslateSourceMgrToMLIRFunction &function, const TranslateSourceMgrToMLIRFunction &function,
Optional<llvm::Align> inputAlignment = llvm::None); Optional<llvm::Align> inputAlignment = std::nullopt);
TranslateToMLIRRegistration( TranslateToMLIRRegistration(
llvm::StringRef name, llvm::StringRef description, llvm::StringRef name, llvm::StringRef description,
const TranslateStringRefToMLIRFunction &function, const TranslateStringRefToMLIRFunction &function,
Optional<llvm::Align> inputAlignment = llvm::None); Optional<llvm::Align> inputAlignment = std::nullopt);
}; };
struct TranslateFromMLIRRegistration { struct TranslateFromMLIRRegistration {
TranslateFromMLIRRegistration( TranslateFromMLIRRegistration(
llvm::StringRef name, llvm::StringRef description, llvm::StringRef name, llvm::StringRef description,
const TranslateFromMLIRFunction &function, const TranslateFromMLIRFunction &function,
const std::function<void(DialectRegistry &)> &dialectRegistration = const std::function<void(DialectRegistry &)> &dialectRegistration =
[](DialectRegistry &) {}); [](DialectRegistry &) {});
Show All 38 Lines

mlir/include/mlir/Transforms/DialectConversion.h

Show First 20 Lines • Show All 85 Lines • ▼ Show 20 Lines private:
/// The set of new argument types. /// The set of new argument types.
SmallVector<Type, 4> argTypes; SmallVector<Type, 4> argTypes;
}; };
/// Register a conversion function. A conversion function must be convertible /// Register a conversion function. A conversion function must be convertible
/// to any of the following forms(where `T` is a class derived from `Type`: /// to any of the following forms(where `T` is a class derived from `Type`:
/// * Optional<Type>(T) /// * Optional<Type>(T)
/// - This form represents a 1-1 type conversion. It should return nullptr /// - This form represents a 1-1 type conversion. It should return nullptr
/// or `llvm::None` to signify failure. If `llvm::None` is returned, the /// or `std::nullopt` to signify failure. If `llvm::None` is returned,
/// converter is allowed to try another conversion function to perform /// the converter is allowed to try another conversion function to
/// the conversion. /// perform the conversion.
/// * Optional<LogicalResult>(T, SmallVectorImpl<Type> &) /// * Optional<LogicalResult>(T, SmallVectorImpl<Type> &)
/// - This form represents a 1-N type conversion. It should return /// - This form represents a 1-N type conversion. It should return
/// `failure` or `llvm::None` to signify a failed conversion. If the new /// `failure` or `std::nullopt` to signify a failed conversion. If the
/// set of types is empty, the type is removed and any usages of the /// new set of types is empty, the type is removed and any usages of the
/// existing value are expected to be removed during conversion. If /// existing value are expected to be removed during conversion. If
/// `llvm::None` is returned, the converter is allowed to try another /// `std::nullopt` is returned, the converter is allowed to try another
/// conversion function to perform the conversion. /// conversion function to perform the conversion.
/// * Optional<LogicalResult>(T, SmallVectorImpl<Type> &, ArrayRef<Type>) /// * Optional<LogicalResult>(T, SmallVectorImpl<Type> &, ArrayRef<Type>)
/// - This form represents a 1-N type conversion supporting recursive /// - This form represents a 1-N type conversion supporting recursive
/// types. The first two arguments and the return value are the same as /// types. The first two arguments and the return value are the same as
/// for the regular 1-N form. The third argument is contains is the /// for the regular 1-N form. The third argument is contains is the
/// "call stack" of the recursive conversion: it contains the list of /// "call stack" of the recursive conversion: it contains the list of
/// types currently being converted, with the current type being the /// types currently being converted, with the current type being the
/// last one. If it is present more than once in the list, the /// last one. If it is present more than once in the list, the
/// conversion concerns a recursive type. /// conversion concerns a recursive type.
/// Note: When attempting to convert a type, e.g. via 'convertType', the /// Note: When attempting to convert a type, e.g. via 'convertType', the
/// mostly recently added conversions will be invoked first. /// mostly recently added conversions will be invoked first.
template <typename FnT, typename T = typename llvm::function_traits< template <typename FnT, typename T = typename llvm::function_traits<
std::decay_t<FnT>>::template arg_t<0>> std::decay_t<FnT>>::template arg_t<0>>
void addConversion(FnT &&callback) { void addConversion(FnT &&callback) {
registerConversion(wrapCallback<T>(std::forward<FnT>(callback))); registerConversion(wrapCallback<T>(std::forward<FnT>(callback)));
} }
/// Register a materialization function, which must be convertible to the /// Register a materialization function, which must be convertible to the
/// following form: /// following form:
/// `Optional<Value>(OpBuilder &, T, ValueRange, Location)`, /// `Optional<Value>(OpBuilder &, T, ValueRange, Location)`,
/// where `T` is any subclass of `Type`. This function is responsible for /// where `T` is any subclass of `Type`. This function is responsible for
/// creating an operation, using the OpBuilder and Location provided, that /// creating an operation, using the OpBuilder and Location provided, that
/// "casts" a range of values into a single value of the given type `T`. It /// "casts" a range of values into a single value of the given type `T`. It
/// must return a Value of the converted type on success, an `llvm::None` if /// must return a Value of the converted type on success, an `std::nullopt` if
/// it failed but other materialization can be attempted, and `nullptr` on /// it failed but other materialization can be attempted, and `nullptr` on
/// unrecoverable failure. It will only be called for (sub)types of `T`. /// unrecoverable failure. It will only be called for (sub)types of `T`.
/// Materialization functions must be provided when a type conversion may /// Materialization functions must be provided when a type conversion may
/// persist after the conversion has finished. /// persist after the conversion has finished.
/// ///
/// This method registers a materialization that will be called when /// This method registers a materialization that will be called when
/// converting an illegal block argument type, to a legal type. /// converting an illegal block argument type, to a legal type.
template <typename FnT, typename T = typename llvm::function_traits< template <typename FnT, typename T = typename llvm::function_traits<
▲ Show 20 Lines • Show All 144 Lines • ▼ Show 20 Lines std::enable_if_t<
std::is_invocable_v<FnT, T, SmallVectorImpl<Type> &, ArrayRef<Type>>, std::is_invocable_v<FnT, T, SmallVectorImpl<Type> &, ArrayRef<Type>>,
ConversionCallbackFn> ConversionCallbackFn>
wrapCallback(FnT &&callback) { wrapCallback(FnT &&callback) {
return [callback = std::forward<FnT>(callback)]( return [callback = std::forward<FnT>(callback)](
Type type, SmallVectorImpl<Type> &results, Type type, SmallVectorImpl<Type> &results,
ArrayRef<Type> callStack) -> Optional<LogicalResult> { ArrayRef<Type> callStack) -> Optional<LogicalResult> {
T derivedType = type.dyn_cast<T>(); T derivedType = type.dyn_cast<T>();
if (!derivedType) if (!derivedType)
return llvm::None; return std::nullopt;
return callback(derivedType, results, callStack); return callback(derivedType, results, callStack);
}; };
} }
/// Register a type conversion. /// Register a type conversion.
void registerConversion(ConversionCallbackFn callback) { void registerConversion(ConversionCallbackFn callback) {
conversions.emplace_back(std::move(callback)); conversions.emplace_back(std::move(callback));
cachedDirectConversions.clear(); cachedDirectConversions.clear();
cachedMultiConversions.clear(); cachedMultiConversions.clear();
} }
/// Generate a wrapper for the given materialization callback. The callback /// Generate a wrapper for the given materialization callback. The callback
/// may take any subclass of `Type` and the wrapper will check for the target /// may take any subclass of `Type` and the wrapper will check for the target
/// type to be of the expected class before calling the callback. /// type to be of the expected class before calling the callback.
template <typename T, typename FnT> template <typename T, typename FnT>
MaterializationCallbackFn wrapMaterialization(FnT &&callback) { MaterializationCallbackFn wrapMaterialization(FnT &&callback) {
return [callback = std::forward<FnT>(callback)]( return [callback = std::forward<FnT>(callback)](
OpBuilder &builder, Type resultType, ValueRange inputs, OpBuilder &builder, Type resultType, ValueRange inputs,
Location loc) -> Optional<Value> { Location loc) -> Optional<Value> {
if (T derivedType = resultType.dyn_cast<T>()) if (T derivedType = resultType.dyn_cast<T>())
return callback(builder, derivedType, inputs, loc); return callback(builder, derivedType, inputs, loc);
return llvm::None; return std::nullopt;
}; };
} }
/// The set of registered conversion functions. /// The set of registered conversion functions.
SmallVector<ConversionCallbackFn, 4> conversions; SmallVector<ConversionCallbackFn, 4> conversions;
/// The list of registered materialization functions. /// The list of registered materialization functions.
SmallVector<MaterializationCallbackFn, 2> argumentMaterializations; SmallVector<MaterializationCallbackFn, 2> argumentMaterializations;
▲ Show 20 Lines • Show All 671 Lines • Show Last 20 Lines

mlir/include/mlir/Transforms/InliningUtils.h

Show First 20 Lines • Show All 202 Lines • ▼ Show 20 Lines
/// the terminators in the region. 'inlineLoc' is an optional Location that, if /// the terminators in the region. 'inlineLoc' is an optional Location that, if
/// provided, will be used to update the inlined operations' location /// provided, will be used to update the inlined operations' location
/// information. 'shouldCloneInlinedRegion' corresponds to whether the source /// information. 'shouldCloneInlinedRegion' corresponds to whether the source
/// region should be cloned into the 'inlinePoint' or spliced directly. /// region should be cloned into the 'inlinePoint' or spliced directly.
LogicalResult inlineRegion(InlinerInterface &interface, Region *src, LogicalResult inlineRegion(InlinerInterface &interface, Region *src,
Operation *inlinePoint, BlockAndValueMapping &mapper, Operation *inlinePoint, BlockAndValueMapping &mapper,
ValueRange resultsToReplace, ValueRange resultsToReplace,
TypeRange regionResultTypes, TypeRange regionResultTypes,
Optional<Location> inlineLoc = llvm::None, Optional<Location> inlineLoc = std::nullopt,
bool shouldCloneInlinedRegion = true); bool shouldCloneInlinedRegion = true);
LogicalResult inlineRegion(InlinerInterface &interface, Region *src, LogicalResult inlineRegion(InlinerInterface &interface, Region *src,
Block *inlineBlock, Block::iterator inlinePoint, Block *inlineBlock, Block::iterator inlinePoint,
BlockAndValueMapping &mapper, BlockAndValueMapping &mapper,
ValueRange resultsToReplace, ValueRange resultsToReplace,
TypeRange regionResultTypes, TypeRange regionResultTypes,
Optional<Location> inlineLoc = llvm::None, Optional<Location> inlineLoc = std::nullopt,
bool shouldCloneInlinedRegion = true); bool shouldCloneInlinedRegion = true);
/// This function is an overload of the above 'inlineRegion' that allows for /// This function is an overload of the above 'inlineRegion' that allows for
/// providing the set of operands ('inlinedOperands') that should be used /// providing the set of operands ('inlinedOperands') that should be used
/// in-favor of the region arguments when inlining. /// in-favor of the region arguments when inlining.
LogicalResult inlineRegion(InlinerInterface &interface, Region *src, LogicalResult inlineRegion(InlinerInterface &interface, Region *src,
Operation *inlinePoint, ValueRange inlinedOperands, Operation *inlinePoint, ValueRange inlinedOperands,
ValueRange resultsToReplace, ValueRange resultsToReplace,
Optional<Location> inlineLoc = llvm::None, Optional<Location> inlineLoc = std::nullopt,
bool shouldCloneInlinedRegion = true); bool shouldCloneInlinedRegion = true);
LogicalResult inlineRegion(InlinerInterface &interface, Region *src, LogicalResult inlineRegion(InlinerInterface &interface, Region *src,
Block *inlineBlock, Block::iterator inlinePoint, Block *inlineBlock, Block::iterator inlinePoint,
ValueRange inlinedOperands, ValueRange inlinedOperands,
ValueRange resultsToReplace, ValueRange resultsToReplace,
Optional<Location> inlineLoc = llvm::None, Optional<Location> inlineLoc = std::nullopt,
bool shouldCloneInlinedRegion = true); bool shouldCloneInlinedRegion = true);
/// This function inlines a given region, 'src', of a callable operation, /// This function inlines a given region, 'src', of a callable operation,
/// 'callable', into the location defined by the given call operation. This /// 'callable', into the location defined by the given call operation. This
/// function returns failure if inlining is not possible, success otherwise. On /// function returns failure if inlining is not possible, success otherwise. On
/// failure, no changes are made to the module. 'shouldCloneInlinedRegion' /// failure, no changes are made to the module. 'shouldCloneInlinedRegion'
/// corresponds to whether the source region should be cloned into the 'call' or /// corresponds to whether the source region should be cloned into the 'call' or
/// spliced directly. /// spliced directly.
LogicalResult inlineCall(InlinerInterface &interface, CallOpInterface call, LogicalResult inlineCall(InlinerInterface &interface, CallOpInterface call,
CallableOpInterface callable, Region *src, CallableOpInterface callable, Region *src,
bool shouldCloneInlinedRegion = true); bool shouldCloneInlinedRegion = true);
} // namespace mlir } // namespace mlir
#endif // MLIR_TRANSFORMS_INLININGUTILS_H #endif // MLIR_TRANSFORMS_INLININGUTILS_H

mlir/include/mlir/Transforms/Passes.h

Show First 20 Lines • Show All 49 Lines • ▼ Show 20 Lines
/// a debug label or debug name in this set. `enabledPatterns` is a set of /// a debug label or debug name in this set. `enabledPatterns` is a set of
/// labels used to filter out input patterns that do not have one of the labels /// labels used to filter out input patterns that do not have one of the labels
/// in this set. Debug labels must be set explicitly on patterns or when adding /// in this set. Debug labels must be set explicitly on patterns or when adding
/// them with `RewritePatternSet::addWithLabel`. Debug names may be empty, but /// them with `RewritePatternSet::addWithLabel`. Debug names may be empty, but
/// patterns created with `RewritePattern::create` have their default debug name /// patterns created with `RewritePattern::create` have their default debug name
/// set to their type name. /// set to their type name.
std::unique_ptr<Pass> std::unique_ptr<Pass>
createCanonicalizerPass(const GreedyRewriteConfig &config, createCanonicalizerPass(const GreedyRewriteConfig &config,
ArrayRef<std::string> disabledPatterns = llvm::None, ArrayRef<std::string> disabledPatterns = std::nullopt,
ArrayRef<std::string> enabledPatterns = llvm::None); ArrayRef<std::string> enabledPatterns = std::nullopt);
/// Creates a pass to perform control-flow sinking. /// Creates a pass to perform control-flow sinking.
std::unique_ptr<Pass> createControlFlowSinkPass(); std::unique_ptr<Pass> createControlFlowSinkPass();
/// Creates a pass to perform common sub expression elimination. /// Creates a pass to perform common sub expression elimination.
std::unique_ptr<Pass> createCSEPass(); std::unique_ptr<Pass> createCSEPass();
/// Creates a loop invariant code motion pass that hoists loop invariant /// Creates a loop invariant code motion pass that hoists loop invariant
▲ Show 20 Lines • Show All 60 Lines • Show Last 20 Lines

mlir/lib/Analysis/AliasAnalysis/LocalAliasAnalysis.cpp

Show First 20 Lines • Show All 64 Lines • ▼ Show 20 Lines for (RegionSuccessor &successor : successors) {
lastInputIndex = inputs.back().cast<OpResult>().getResultNumber(); lastInputIndex = inputs.back().cast<OpResult>().getResultNumber();
} }
if (firstInputIndex > inputIndex || lastInputIndex < inputIndex) { if (firstInputIndex > inputIndex || lastInputIndex < inputIndex) {
output.push_back(inputValue); output.push_back(inputValue);
break; break;
} }
return inputIndex - firstInputIndex; return inputIndex - firstInputIndex;
} }
return llvm::None; return std::nullopt;
}; };
// Check branches from the parent operation. // Check branches from the parent operation.
Optional<unsigned> regionIndex; Optional<unsigned> regionIndex;
if (region) { if (region) {
// Determine the actual region number from the passed region. // Determine the actual region number from the passed region.
regionIndex = region->getRegionNumber(); regionIndex = region->getRegionNumber();
} }
if (Optional<unsigned> operandIndex = if (Optional<unsigned> operandIndex =
getOperandIndexIfPred(/*predIndex=*/llvm::None)) { getOperandIndexIfPred(/*predIndex=*/std::nullopt)) {
collectUnderlyingAddressValues( collectUnderlyingAddressValues(
branch.getSuccessorEntryOperands(regionIndex)[*operandIndex], maxDepth, branch.getSuccessorEntryOperands(regionIndex)[*operandIndex], maxDepth,
visited, output); visited, output);
} }
// Check branches from each child region. // Check branches from each child region.
Operation *op = branch.getOperation(); Operation *op = branch.getOperation();
for (int i = 0, e = op->getNumRegions(); i != e; ++i) { for (int i = 0, e = op->getNumRegions(); i != e; ++i) {
if (Optional<unsigned> operandIndex = getOperandIndexIfPred(i)) { if (Optional<unsigned> operandIndex = getOperandIndexIfPred(i)) {
▲ Show 20 Lines • Show All 305 Lines • Show Last 20 Lines

mlir/lib/Analysis/DataFlow/SparseAnalysis.cpp

Show First 20 Lines • Show All 89 Lines • ▼ Show 20 Linesvoid AbstractSparseDataFlowAnalysis::visitOperation(Operation *op) {
for (Value result : op->getResults()) { for (Value result : op->getResults()) {
AbstractSparseLattice *resultLattice = getLatticeElement(result); AbstractSparseLattice *resultLattice = getLatticeElement(result);
resultLattices.push_back(resultLattice); resultLattices.push_back(resultLattice);
} }
// The results of a region branch operation are determined by control-flow. // The results of a region branch operation are determined by control-flow.
if (auto branch = dyn_cast<RegionBranchOpInterface>(op)) { if (auto branch = dyn_cast<RegionBranchOpInterface>(op)) {
return visitRegionSuccessors({branch}, branch, return visitRegionSuccessors({branch}, branch,
/*successorIndex=*/llvm::None, resultLattices); /*successorIndex=*/std::nullopt,
resultLattices);
} }
// The results of a call operation are determined by the callgraph. // The results of a call operation are determined by the callgraph.
if (auto call = dyn_cast<CallOpInterface>(op)) { if (auto call = dyn_cast<CallOpInterface>(op)) {
const auto *predecessors = getOrCreateFor<PredecessorState>(op, call); const auto *predecessors = getOrCreateFor<PredecessorState>(op, call);
// If not all return sites are known, then conservatively assume we can't // If not all return sites are known, then conservatively assume we can't
// reason about the data-flow. // reason about the data-flow.
if (!predecessors->allPredecessorsKnown()) if (!predecessors->allPredecessorsKnown())
▲ Show 20 Lines • Show All 176 Lines • Show Last 20 Lines

mlir/lib/Analysis/Presburger/IntegerRelation.cpp

Show First 20 Lines • Show All 1,399 Lines • ▼ Show 20 LinesOptional<MPInt> IntegerRelation::getConstantBoundOnDimSize(
int eqPos = findEqualityToConstant(*this, pos, /*symbolic=*/true); int eqPos = findEqualityToConstant(*this, pos, /*symbolic=*/true);
if (eqPos != -1) { if (eqPos != -1) {
auto eq = getEquality(eqPos); auto eq = getEquality(eqPos);
// If the equality involves a local var, punt for now. // If the equality involves a local var, punt for now.
// TODO: this can be handled in the future by using the explicit // TODO: this can be handled in the future by using the explicit
// representation of the local vars. // representation of the local vars.
if (!std::all_of(eq.begin() + getNumDimAndSymbolVars(), eq.end() - 1, if (!std::all_of(eq.begin() + getNumDimAndSymbolVars(), eq.end() - 1,
[](const MPInt &coeff) { return coeff == 0; })) [](const MPInt &coeff) { return coeff == 0; }))
return None; return std::nullopt;
// This variable can only take a single value. // This variable can only take a single value.
if (lb) { if (lb) {
// Set lb to that symbolic value. // Set lb to that symbolic value.
lb->resize(getNumSymbolVars() + 1); lb->resize(getNumSymbolVars() + 1);
if (ub) if (ub)
ub->resize(getNumSymbolVars() + 1); ub->resize(getNumSymbolVars() + 1);
for (unsigned c = 0, f = getNumSymbolVars() + 1; c < f; c++) { for (unsigned c = 0, f = getNumSymbolVars() + 1; c < f; c++) {
Show All 20 LinesOptional<MPInt> IntegerRelation::getConstantBoundOnDimSize(
// Check if the variable appears at all in any of the inequalities. // Check if the variable appears at all in any of the inequalities.
unsigned r, e; unsigned r, e;
for (r = 0, e = getNumInequalities(); r < e; r++) { for (r = 0, e = getNumInequalities(); r < e; r++) {
if (atIneq(r, pos) != 0) if (atIneq(r, pos) != 0)
break; break;
} }
if (r == e) if (r == e)
// If it doesn't, there isn't a bound on it. // If it doesn't, there isn't a bound on it.
return None; return std::nullopt;
// Positions of constraints that are lower/upper bounds on the variable. // Positions of constraints that are lower/upper bounds on the variable.
SmallVector<unsigned, 4> lbIndices, ubIndices; SmallVector<unsigned, 4> lbIndices, ubIndices;
// Gather all symbolic lower bounds and upper bounds of the variable, i.e., // Gather all symbolic lower bounds and upper bounds of the variable, i.e.,
// the bounds can only involve symbolic (and local) variables. Since the // the bounds can only involve symbolic (and local) variables. Since the
// canonical form c_1*x_1 + c_2*x_2 + ... + c_0 >= 0, a constraint is a lower // canonical form c_1*x_1 + c_2*x_2 + ... + c_0 >= 0, a constraint is a lower
// bound for x_i if c_i >= 1, and an upper bound if c_i <= -1. // bound for x_i if c_i >= 1, and an upper bound if c_i <= -1.
Show All 18 Lines for (auto lbPos : lbIndices) {
} }
if (j < getNumCols() - 1) if (j < getNumCols() - 1)
continue; continue;
MPInt diff = ceilDiv(atIneq(ubPos, getNumCols() - 1) + MPInt diff = ceilDiv(atIneq(ubPos, getNumCols() - 1) +
atIneq(lbPos, getNumCols() - 1) + 1, atIneq(lbPos, getNumCols() - 1) + 1,
atIneq(lbPos, pos)); atIneq(lbPos, pos));
// This bound is non-negative by definition. // This bound is non-negative by definition.
diff = std::max<MPInt>(diff, MPInt(0)); diff = std::max<MPInt>(diff, MPInt(0));
if (minDiff == None || diff < minDiff) { if (minDiff == std::nullopt || diff < minDiff) {
minDiff = diff; minDiff = diff;
minLbPosition = lbPos; minLbPosition = lbPos;
minUbPosition = ubPos; minUbPosition = ubPos;
} }
} }
} }
if (lb && minDiff) { if (lb && minDiff) {
// Set lb to the symbolic lower bound. // Set lb to the symbolic lower bound.
▲ Show 20 Lines • Show All 42 Lines • ▼ Show 20 LinesIntegerRelation::computeConstantLowerOrUpperBound(unsigned pos) {
// Check if the variable appears at all in any of the inequalities. // Check if the variable appears at all in any of the inequalities.
unsigned r, e; unsigned r, e;
for (r = 0, e = getNumInequalities(); r < e; r++) { for (r = 0, e = getNumInequalities(); r < e; r++) {
if (atIneq(r, 0) != 0) if (atIneq(r, 0) != 0)
break; break;
} }
if (r == e) if (r == e)
// If it doesn't, there isn't a bound on it. // If it doesn't, there isn't a bound on it.
return None; return std::nullopt;
Optional<MPInt> minOrMaxConst; Optional<MPInt> minOrMaxConst;
// Take the max across all const lower bounds (or min across all constant // Take the max across all const lower bounds (or min across all constant
// upper bounds). // upper bounds).
for (unsigned r = 0, e = getNumInequalities(); r < e; r++) { for (unsigned r = 0, e = getNumInequalities(); r < e; r++) {
if (isLower) { if (isLower) {
if (atIneq(r, 0) <= 0) if (atIneq(r, 0) <= 0)
Show All 10 Lines for (unsigned r = 0, e = getNumInequalities(); r < e; r++) {
if (c < getNumCols() - 1) if (c < getNumCols() - 1)
// Not a constant bound. // Not a constant bound.
continue; continue;
MPInt boundConst = MPInt boundConst =
isLower ? ceilDiv(-atIneq(r, getNumCols() - 1), atIneq(r, 0)) isLower ? ceilDiv(-atIneq(r, getNumCols() - 1), atIneq(r, 0))
: floorDiv(atIneq(r, getNumCols() - 1), -atIneq(r, 0)); : floorDiv(atIneq(r, getNumCols() - 1), -atIneq(r, 0));
if (isLower) { if (isLower) {
if (minOrMaxConst == None || boundConst > minOrMaxConst) if (minOrMaxConst == std::nullopt || boundConst > minOrMaxConst)
minOrMaxConst = boundConst; minOrMaxConst = boundConst;
} else { } else {
if (minOrMaxConst == None || boundConst < minOrMaxConst) if (minOrMaxConst == std::nullopt || boundConst < minOrMaxConst)
minOrMaxConst = boundConst; minOrMaxConst = boundConst;
} }
} }
return minOrMaxConst; return minOrMaxConst;
} }
Optional<MPInt> IntegerRelation::getConstantBound(BoundType type, Optional<MPInt> IntegerRelation::getConstantBound(BoundType type,
unsigned pos) const { unsigned pos) const {
if (type == BoundType::LB) if (type == BoundType::LB)
return IntegerRelation(*this) return IntegerRelation(*this)
.computeConstantLowerOrUpperBound</*isLower=*/true>(pos); .computeConstantLowerOrUpperBound</*isLower=*/true>(pos);
if (type == BoundType::UB) if (type == BoundType::UB)
return IntegerRelation(*this) return IntegerRelation(*this)
.computeConstantLowerOrUpperBound</*isLower=*/false>(pos); .computeConstantLowerOrUpperBound</*isLower=*/false>(pos);
assert(type == BoundType::EQ && "expected EQ"); assert(type == BoundType::EQ && "expected EQ");
Optional<MPInt> lb = Optional<MPInt> lb =
IntegerRelation(*this).computeConstantLowerOrUpperBound</*isLower=*/true>( IntegerRelation(*this).computeConstantLowerOrUpperBound</*isLower=*/true>(
pos); pos);
Optional<MPInt> ub = Optional<MPInt> ub =
IntegerRelation(*this) IntegerRelation(*this)
.computeConstantLowerOrUpperBound</*isLower=*/false>(pos); .computeConstantLowerOrUpperBound</*isLower=*/false>(pos);
return (lb && ub && *lb == *ub) ? Optional<MPInt>(*ub) : None; return (lb && ub && *lb == *ub) ? Optional<MPInt>(*ub) : std::nullopt;
} }
// A simple (naive and conservative) check for hyper-rectangularity. // A simple (naive and conservative) check for hyper-rectangularity.
bool IntegerRelation::isHyperRectangular(unsigned pos, unsigned num) const { bool IntegerRelation::isHyperRectangular(unsigned pos, unsigned num) const {
assert(pos < getNumCols() - 1); assert(pos < getNumCols() - 1);
// Check for two non-zero coefficients in the range [pos, pos + sum). // Check for two non-zero coefficients in the range [pos, pos + sum).
for (unsigned r = 0, e = getNumInequalities(); r < e; r++) { for (unsigned r = 0, e = getNumInequalities(); r < e; r++) {
unsigned sum = 0; unsigned sum = 0;
▲ Show 20 Lines • Show All 676 Lines • Show Last 20 Lines

mlir/lib/Analysis/Presburger/PWMAFunction.cpp

Show First 20 Lines • Show All 424 Lines • ▼ Show 20 Lines
Optional<SmallVector<MPInt, 8>> Optional<SmallVector<MPInt, 8>>
PWMAFunction::valueAt(ArrayRef<MPInt> point) const { PWMAFunction::valueAt(ArrayRef<MPInt> point) const {
assert(point.size() == getNumDomainVars() + getNumSymbolVars()); assert(point.size() == getNumDomainVars() + getNumSymbolVars());
for (const Piece &piece : pieces) for (const Piece &piece : pieces)
if (piece.domain.containsPoint(point)) if (piece.domain.containsPoint(point))
return piece.output.valueAt(point); return piece.output.valueAt(point);
return None; return std::nullopt;
} }

mlir/lib/Analysis/Presburger/PresburgerRelation.cpp

Show First 20 Lines • Show All 363 Lines • ▼ Show 20 Lines if (level > frames.size()) {
// Nothing to process; return. (we have no frame to pop.) // Nothing to process; return. (we have no frame to pop.)
level = frames.size(); level = frames.size();
continue; continue;
} }
unsigned simplexSnapshot = simplex.getSnapshot(); unsigned simplexSnapshot = simplex.getSnapshot();
IntegerRelation::CountsSnapshot bCounts = b.getCounts(); IntegerRelation::CountsSnapshot bCounts = b.getCounts();
frames.push_back(Frame{simplexSnapshot, bCounts, sI, ineqsToProcess, frames.push_back(Frame{simplexSnapshot, bCounts, sI, ineqsToProcess,
/*lastIneqProcessed=*/llvm::None}); /*lastIneqProcessed=*/std::nullopt});
// We have completed the initial setup for this level. // We have completed the initial setup for this level.
// Fallthrough to the main recursive part below. // Fallthrough to the main recursive part below.
} }
// For each inequality ineq, we first recurse with the part where ineq // For each inequality ineq, we first recurse with the part where ineq
// is not satisfied, and then add ineq to b and simplex because // is not satisfied, and then add ineq to b and simplex because
// ineq must be satisfied by all later parts. // ineq must be satisfied by all later parts.
if (level == frames.size()) { if (level == frames.size()) {
▲ Show 20 Lines • Show All 485 Lines • Show Last 20 Lines

mlir/lib/Analysis/Presburger/Utils.cpp

Show First 20 Lines • Show All 375 Lines • ▼ Show 20 LinesSmallVector<MPInt, 8> presburger::getComplementIneq(ArrayRef<MPInt> ineq) {
--coeffs.back(); --coeffs.back();
return coeffs; return coeffs;
} }
SmallVector<Optional<MPInt>, 4> SmallVector<Optional<MPInt>, 4>
DivisionRepr::divValuesAt(ArrayRef<MPInt> point) const { DivisionRepr::divValuesAt(ArrayRef<MPInt> point) const {
assert(point.size() == getNumNonDivs() && "Incorrect point size"); assert(point.size() == getNumNonDivs() && "Incorrect point size");
SmallVector<Optional<MPInt>, 4> divValues(getNumDivs(), None); SmallVector<Optional<MPInt>, 4> divValues(getNumDivs(), std::nullopt);
bool changed = true; bool changed = true;
while (changed) { while (changed) {
changed = false; changed = false;
for (unsigned i = 0, e = getNumDivs(); i < e; ++i) { for (unsigned i = 0, e = getNumDivs(); i < e; ++i) {
// If division value is found, continue; // If division value is found, continue;
if (divValues[i]) if (divValues[i])
continue; continue;
▲ Show 20 Lines • Show All 120 Lines • Show Last 20 Lines

mlir/lib/AsmParser/AttributeParser.cpp

Show First 20 Lines • Show All 351 Lines • ▼ Show 20 Lines
/// Construct an APint from a parsed value, a known attribute type and /// Construct an APint from a parsed value, a known attribute type and
/// sign. /// sign.
static Optional<APInt> buildAttributeAPInt(Type type, bool isNegative, static Optional<APInt> buildAttributeAPInt(Type type, bool isNegative,
StringRef spelling) { StringRef spelling) {
// Parse the integer value into an APInt that is big enough to hold the value. // Parse the integer value into an APInt that is big enough to hold the value.
APInt result; APInt result;
bool isHex = spelling.size() > 1 && spelling[1] == 'x'; bool isHex = spelling.size() > 1 && spelling[1] == 'x';
if (spelling.getAsInteger(isHex ? 0 : 10, result)) if (spelling.getAsInteger(isHex ? 0 : 10, result))
return llvm::None; return std::nullopt;
// Extend or truncate the bitwidth to the right size. // Extend or truncate the bitwidth to the right size.
unsigned width = type.isIndex() ? IndexType::kInternalStorageBitWidth unsigned width = type.isIndex() ? IndexType::kInternalStorageBitWidth
: type.getIntOrFloatBitWidth(); : type.getIntOrFloatBitWidth();
if (width > result.getBitWidth()) { if (width > result.getBitWidth()) {
result = result.zext(width); result = result.zext(width);
} else if (width < result.getBitWidth()) { } else if (width < result.getBitWidth()) {
// The parser can return an unnecessarily wide result with leading zeros. // The parser can return an unnecessarily wide result with leading zeros.
// This isn't a problem, but truncating off bits is bad. // This isn't a problem, but truncating off bits is bad.
if (result.countLeadingZeros() < result.getBitWidth() - width) if (result.countLeadingZeros() < result.getBitWidth() - width)
return llvm::None; return std::nullopt;
result = result.trunc(width); result = result.trunc(width);
} }
if (width == 0) { if (width == 0) {
// 0 bit integers cannot be negative and manipulation of their sign bit will // 0 bit integers cannot be negative and manipulation of their sign bit will
// assert, so short-cut validation here. // assert, so short-cut validation here.
if (isNegative) if (isNegative)
return llvm::None; return std::nullopt;
} else if (isNegative) { } else if (isNegative) {
// The value is negative, we have an overflow if the sign bit is not set // The value is negative, we have an overflow if the sign bit is not set
// in the negated apInt. // in the negated apInt.
result.negate(); result.negate();
if (!result.isSignBitSet()) if (!result.isSignBitSet())
return llvm::None; return std::nullopt;
} else if ((type.isSignedInteger() || type.isIndex()) && } else if ((type.isSignedInteger() || type.isIndex()) &&
result.isSignBitSet()) { result.isSignBitSet()) {
// The value is a positive signed integer or index, // The value is a positive signed integer or index,
// we have an overflow if the sign bit is set. // we have an overflow if the sign bit is set.
return llvm::None; return std::nullopt;
} }
return result; return result;
} }
/// Parse a decimal or a hexadecimal literal, which can be either an integer /// Parse a decimal or a hexadecimal literal, which can be either an integer
/// or a float attribute. /// or a float attribute.
Attribute Parser::parseDecOrHexAttr(Type type, bool isNegative) { Attribute Parser::parseDecOrHexAttr(Type type, bool isNegative) {
▲ Show 20 Lines • Show All 769 Lines • ▼ Show 20 LinesAttribute Parser::parseStridedLayoutAttr() {
// fit into int64_t limits. // fit into int64_t limits.
auto parseStrideOrOffset = [&]() -> Optional<int64_t> { auto parseStrideOrOffset = [&]() -> Optional<int64_t> {
if (consumeIf(Token::question)) if (consumeIf(Token::question))
return ShapedType::kDynamic; return ShapedType::kDynamic;
SMLoc loc = getToken().getLoc(); SMLoc loc = getToken().getLoc();
auto emitWrongTokenError = [&] { auto emitWrongTokenError = [&] {
emitError(loc, "expected a 64-bit signed integer or '?'"); emitError(loc, "expected a 64-bit signed integer or '?'");
return llvm::None; return std::nullopt;
}; };
bool negative = consumeIf(Token::minus); bool negative = consumeIf(Token::minus);
if (getToken().is(Token::integer)) { if (getToken().is(Token::integer)) {
Optional<uint64_t> value = getToken().getUInt64IntegerValue(); Optional<uint64_t> value = getToken().getUInt64IntegerValue();
if (!value || if (!value ||
*value > static_cast<uint64_t>(std::numeric_limits<int64_t>::max())) *value > static_cast<uint64_t>(std::numeric_limits<int64_t>::max()))
▲ Show 20 Lines • Show All 48 Lines • Show Last 20 Lines

mlir/lib/AsmParser/Parser.h

Show First 20 Lines • Show All 230 Lines • ▼ Show 20 Linespublic:
OptionalParseResult parseOptionalAttribute(StringAttr &attribute, Type type); OptionalParseResult parseOptionalAttribute(StringAttr &attribute, Type type);
/// Parse an optional attribute that is demarcated by a specific token. /// Parse an optional attribute that is demarcated by a specific token.
template <typename AttributeT> template <typename AttributeT>
OptionalParseResult parseOptionalAttributeWithToken(Token::Kind kind, OptionalParseResult parseOptionalAttributeWithToken(Token::Kind kind,
AttributeT &attr, AttributeT &attr,
Type type = {}) { Type type = {}) {
if (getToken().isNot(kind)) if (getToken().isNot(kind))
return llvm::None; return std::nullopt;
if (Attribute parsedAttr = parseAttribute(type)) { if (Attribute parsedAttr = parseAttribute(type)) {
attr = parsedAttr.cast<AttributeT>(); attr = parsedAttr.cast<AttributeT>();
return success(); return success();
} }
return failure(); return failure();
} }
▲ Show 20 Lines • Show All 97 Lines • Show Last 20 Lines

mlir/lib/AsmParser/Parser.cpp

Show First 20 Lines • Show All 244 Lines • ▼ Show 20 LinesOptionalParseResult Parser::parseOptionalInteger(APInt &result) {
} }
if (consumeIf(Token::kw_true)) { if (consumeIf(Token::kw_true)) {
result = true; result = true;
return success(); return success();
} }
Token curToken = getToken(); Token curToken = getToken();
if (curToken.isNot(Token::integer, Token::minus)) if (curToken.isNot(Token::integer, Token::minus))
return llvm::None; return std::nullopt;
bool negative = consumeIf(Token::minus); bool negative = consumeIf(Token::minus);
Token curTok = getToken(); Token curTok = getToken();
if (parseToken(Token::integer, "expected integer value")) if (parseToken(Token::integer, "expected integer value"))
return failure(); return failure();
StringRef spelling = curTok.getSpelling(); StringRef spelling = curTok.getSpelling();
bool isHex = spelling.size() > 1 && spelling[1] == 'x'; bool isHex = spelling.size() > 1 && spelling[1] == 'x';
▲ Show 20 Lines • Show All 266 Lines • ▼ Show 20 Linespublic:
/// Parse different components, viz., use-info of operand(s), successor(s), /// Parse different components, viz., use-info of operand(s), successor(s),
/// region(s), attribute(s) and function-type, of the generic form of an /// region(s), attribute(s) and function-type, of the generic form of an
/// operation instance and populate the input operation-state 'result' with /// operation instance and populate the input operation-state 'result' with
/// those components. If any of the components is explicitly provided, then /// those components. If any of the components is explicitly provided, then
/// skip parsing that component. /// skip parsing that component.
ParseResult parseGenericOperationAfterOpName( ParseResult parseGenericOperationAfterOpName(
OperationState &result, OperationState &result,
Optional<ArrayRef<UnresolvedOperand>> parsedOperandUseInfo = llvm::None, Optional<ArrayRef<UnresolvedOperand>> parsedOperandUseInfo = std::nullopt,
Optional<ArrayRef<Block *>> parsedSuccessors = llvm::None, Optional<ArrayRef<Block *>> parsedSuccessors = std::nullopt,
Optional<MutableArrayRef<std::unique_ptr<Region>>> parsedRegions = Optional<MutableArrayRef<std::unique_ptr<Region>>> parsedRegions =
llvm::None, std::nullopt,
Optional<ArrayRef<NamedAttribute>> parsedAttributes = llvm::None, Optional<ArrayRef<NamedAttribute>> parsedAttributes = std::nullopt,
Optional<FunctionType> parsedFnType = llvm::None); Optional<FunctionType> parsedFnType = std::nullopt);
/// Parse an operation instance that is in the generic form and insert it at /// Parse an operation instance that is in the generic form and insert it at
/// the provided insertion point. /// the provided insertion point.
Operation *parseGenericOperation(Block *insertBlock, Operation *parseGenericOperation(Block *insertBlock,
Block::iterator insertPt); Block::iterator insertPt);
/// This type is used to keep track of things that are either an Operation or /// This type is used to keep track of things that are either an Operation or
/// a BlockArgument. We cannot use Value for this, because not all Operations /// a BlockArgument. We cannot use Value for this, because not all Operations
▲ Show 20 Lines • Show All 518 Lines • ▼ Show 20 LinesValue OperationParser::createForwardRefPlaceholder(SMLoc loc, Type type) {
// something with a def/use chain. // something with a def/use chain.
// //
// We create these placeholders as having an empty name, which we know // We create these placeholders as having an empty name, which we know
// cannot be created through normal user input, allowing us to distinguish // cannot be created through normal user input, allowing us to distinguish
// them. // them.
auto name = OperationName("builtin.unrealized_conversion_cast", getContext()); auto name = OperationName("builtin.unrealized_conversion_cast", getContext());
auto *op = Operation::create( auto *op = Operation::create(
getEncodedSourceLocation(loc), name, type, /*operands=*/{}, getEncodedSourceLocation(loc), name, type, /*operands=*/{},
/*attributes=*/llvm::None, /*successors=*/{}, /*numRegions=*/0); /*attributes=*/std::nullopt, /*successors=*/{}, /*numRegions=*/0);
forwardRefPlaceholders[op->getResult(0)] = loc; forwardRefPlaceholders[op->getResult(0)] = loc;
return op->getResult(0); return op->getResult(0);
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Operation Parsing // Operation Parsing
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 434 Lines • ▼ Show 20 Linespublic:
} }
/// Parse a single operand if present. /// Parse a single operand if present.
OptionalParseResult OptionalParseResult
parseOptionalOperand(UnresolvedOperand &result, parseOptionalOperand(UnresolvedOperand &result,
bool allowResultNumber = true) override { bool allowResultNumber = true) override {
if (parser.getToken().isOrIsCodeCompletionFor(Token::percent_identifier)) if (parser.getToken().isOrIsCodeCompletionFor(Token::percent_identifier))
return parseOperand(result, allowResultNumber); return parseOperand(result, allowResultNumber);
return llvm::None; return std::nullopt;
} }
/// Parse zero or more SSA comma-separated operand references with a specified /// Parse zero or more SSA comma-separated operand references with a specified
/// surrounding delimiter, and an optional required operand count. /// surrounding delimiter, and an optional required operand count.
ParseResult parseOperandList(SmallVectorImpl<UnresolvedOperand> &result, ParseResult parseOperandList(SmallVectorImpl<UnresolvedOperand> &result,
Delimiter delimiter = Delimiter::None, Delimiter delimiter = Delimiter::None,
bool allowResultNumber = true, bool allowResultNumber = true,
int requiredOperandCount = -1) override { int requiredOperandCount = -1) override {
▲ Show 20 Lines • Show All 116 Lines • ▼ Show 20 Lines ParseResult parseArgument(Argument &result, bool allowType = false,
return success(); return success();
} }
/// Parse a single argument if present. /// Parse a single argument if present.
OptionalParseResult parseOptionalArgument(Argument &result, bool allowType, OptionalParseResult parseOptionalArgument(Argument &result, bool allowType,
bool allowAttrs) override { bool allowAttrs) override {
if (parser.getToken().is(Token::percent_identifier)) if (parser.getToken().is(Token::percent_identifier))
return parseArgument(result, allowType, allowAttrs); return parseArgument(result, allowType, allowAttrs);
return llvm::None; return std::nullopt;
} }
ParseResult parseArgumentList(SmallVectorImpl<Argument> &result, ParseResult parseArgumentList(SmallVectorImpl<Argument> &result,
Delimiter delimiter, bool allowType, Delimiter delimiter, bool allowType,
bool allowAttrs) override { bool allowAttrs) override {
// The no-delimiter case has some special handling for the empty case. // The no-delimiter case has some special handling for the empty case.
if (delimiter == Delimiter::None && if (delimiter == Delimiter::None &&
parser.getToken().isNot(Token::percent_identifier)) parser.getToken().isNot(Token::percent_identifier))
Show All 23 Lines ParseResult parseRegion(Region &region, ArrayRef<Argument> arguments,
return success(); return success();
} }
/// Parses a region if present. /// Parses a region if present.
OptionalParseResult parseOptionalRegion(Region &region, OptionalParseResult parseOptionalRegion(Region &region,
ArrayRef<Argument> arguments, ArrayRef<Argument> arguments,
bool enableNameShadowing) override { bool enableNameShadowing) override {
if (parser.getToken().isNot(Token::l_brace)) if (parser.getToken().isNot(Token::l_brace))
return llvm::None; return std::nullopt;
return parseRegion(region, arguments, enableNameShadowing); return parseRegion(region, arguments, enableNameShadowing);
} }
/// Parses a region if present. If the region is present, a new region is /// Parses a region if present. If the region is present, a new region is
/// allocated and placed in `region`. If no region is present, `region` /// allocated and placed in `region`. If no region is present, `region`
/// remains untouched. /// remains untouched.
OptionalParseResult OptionalParseResult
parseOptionalRegion(std::unique_ptr<Region> &region, parseOptionalRegion(std::unique_ptr<Region> &region,
ArrayRef<Argument> arguments, ArrayRef<Argument> arguments,
bool enableNameShadowing = false) override { bool enableNameShadowing = false) override {
if (parser.getToken().isNot(Token::l_brace)) if (parser.getToken().isNot(Token::l_brace))
return llvm::None; return std::nullopt;
std::unique_ptr<Region> newRegion = std::make_unique<Region>(); std::unique_ptr<Region> newRegion = std::make_unique<Region>();
if (parseRegion(*newRegion, arguments, enableNameShadowing)) if (parseRegion(*newRegion, arguments, enableNameShadowing))
return failure(); return failure();
region = std::move(newRegion); region = std::move(newRegion);
return success(); return success();
} }
//===--------------------------------------------------------------------===// //===--------------------------------------------------------------------===//
// Successor Parsing // Successor Parsing
//===--------------------------------------------------------------------===// //===--------------------------------------------------------------------===//
/// Parse a single operation successor. /// Parse a single operation successor.
ParseResult parseSuccessor(Block *&dest) override { ParseResult parseSuccessor(Block *&dest) override {
return parser.parseSuccessor(dest); return parser.parseSuccessor(dest);
} }
/// Parse an optional operation successor and its operand list. /// Parse an optional operation successor and its operand list.
OptionalParseResult parseOptionalSuccessor(Block *&dest) override { OptionalParseResult parseOptionalSuccessor(Block *&dest) override {
if (!parser.getToken().isOrIsCodeCompletionFor(Token::caret_identifier)) if (!parser.getToken().isOrIsCodeCompletionFor(Token::caret_identifier))
return llvm::None; return std::nullopt;
return parseSuccessor(dest); return parseSuccessor(dest);
} }
/// Parse a single operation successor and its operand list. /// Parse a single operation successor and its operand list.
ParseResult ParseResult
parseSuccessorAndUseList(Block *&dest, parseSuccessorAndUseList(Block *&dest,
SmallVectorImpl<Value> &operands) override { SmallVectorImpl<Value> &operands) override {
if (parseSuccessor(dest)) if (parseSuccessor(dest))
Show All 12 Linespublic:
//===--------------------------------------------------------------------===// //===--------------------------------------------------------------------===//
/// Parse a list of assignments of the form /// Parse a list of assignments of the form
/// (%x1 = %y1, %x2 = %y2, ...). /// (%x1 = %y1, %x2 = %y2, ...).
OptionalParseResult parseOptionalAssignmentList( OptionalParseResult parseOptionalAssignmentList(
SmallVectorImpl<Argument> &lhs, SmallVectorImpl<Argument> &lhs,
SmallVectorImpl<UnresolvedOperand> &rhs) override { SmallVectorImpl<UnresolvedOperand> &rhs) override {
if (failed(parseOptionalLParen())) if (failed(parseOptionalLParen()))
return llvm::None; return std::nullopt;
auto parseElt = [&]() -> ParseResult { auto parseElt = [&]() -> ParseResult {
if (parseArgument(lhs.emplace_back()) || parseEqual() || if (parseArgument(lhs.emplace_back()) || parseEqual() ||
parseOperand(rhs.emplace_back())) parseOperand(rhs.emplace_back()))
return failure(); return failure();
return success(); return success();
}; };
return parser.parseCommaSeparatedListUntil(Token::r_paren, parseElt); return parser.parseCommaSeparatedListUntil(Token::r_paren, parseElt);
▲ Show 20 Lines • Show All 615 Lines • ▼ Show 20 Linespublic:
} }
FailureOr<AsmResourceBlob> FailureOr<AsmResourceBlob>
parseAsBlob(BlobAllocatorFn allocator) const final { parseAsBlob(BlobAllocatorFn allocator) const final {
// Blob data within then textual format is represented as a hex string. // Blob data within then textual format is represented as a hex string.
// TODO: We could avoid an additional alloc+copy here if we pre-allocated // TODO: We could avoid an additional alloc+copy here if we pre-allocated
// the buffer to use during hex processing. // the buffer to use during hex processing.
Optional<std::string> blobData = Optional<std::string> blobData =
value.is(Token::string) ? value.getHexStringValue() : llvm::None; value.is(Token::string) ? value.getHexStringValue() : std::nullopt;
if (!blobData) if (!blobData)
return p.emitError(value.getLoc(), return p.emitError(value.getLoc(),
"expected hex string blob for key '" + key + "'"); "expected hex string blob for key '" + key + "'");
// Extract the alignment of the blob data, which gets stored at the // Extract the alignment of the blob data, which gets stored at the
// beginning of the string. // beginning of the string.
if (blobData->size() < sizeof(uint32_t)) { if (blobData->size() < sizeof(uint32_t)) {
return p.emitError(value.getLoc(), return p.emitError(value.getLoc(),
▲ Show 20 Lines • Show All 258 Lines • Show Last 20 Lines

mlir/lib/AsmParser/Token.h

Show First 20 Lines • Show All 85 Lines • ▼ Show 20 Lines#include "TokenKinds.def"
/// For a floatliteral token, return its value as a double. Returns None in /// For a floatliteral token, return its value as a double. Returns None in
/// the case of underflow or overflow. /// the case of underflow or overflow.
Optional<double> getFloatingPointValue() const; Optional<double> getFloatingPointValue() const;
/// For an inttype token, return its bitwidth. /// For an inttype token, return its bitwidth.
Optional<unsigned> getIntTypeBitwidth() const; Optional<unsigned> getIntTypeBitwidth() const;
/// For an inttype token, return its signedness semantics: llvm::None means no /// For an inttype token, return its signedness semantics: std::nullopt means
/// signedness semantics; true means signed integer type; false means unsigned /// no signedness semantics; true means signed integer type; false means
/// integer type. /// unsigned integer type.
Optional<bool> getIntTypeSignedness() const; Optional<bool> getIntTypeSignedness() const;
/// Given a hash_identifier token like #123, try to parse the number out of /// Given a hash_identifier token like #123, try to parse the number out of
/// the identifier, returning None if it is a named identifier like #x or /// the identifier, returning None if it is a named identifier like #x or
/// if the integer doesn't fit. /// if the integer doesn't fit.
Optional<unsigned> getHashIdentifierNumber() const; Optional<unsigned> getHashIdentifierNumber() const;
/// Given a token containing a string literal, return its value, including /// Given a token containing a string literal, return its value, including
Show All 34 Lines

mlir/lib/AsmParser/Token.cpp

Show All 24 Lines
/// For an integer token, return its value as an unsigned. If it doesn't fit, /// For an integer token, return its value as an unsigned. If it doesn't fit,
/// return None. /// return None.
Optional<unsigned> Token::getUnsignedIntegerValue() const { Optional<unsigned> Token::getUnsignedIntegerValue() const {
bool isHex = spelling.size() > 1 && spelling[1] == 'x'; bool isHex = spelling.size() > 1 && spelling[1] == 'x';
unsigned result = 0; unsigned result = 0;
if (spelling.getAsInteger(isHex ? 0 : 10, result)) if (spelling.getAsInteger(isHex ? 0 : 10, result))
return None; return std::nullopt;
return result; return result;
} }
/// For an integer token, return its value as a uint64_t. If it doesn't fit, /// For an integer token, return its value as a uint64_t. If it doesn't fit,
/// return None. /// return None.
Optional<uint64_t> Token::getUInt64IntegerValue(StringRef spelling) { Optional<uint64_t> Token::getUInt64IntegerValue(StringRef spelling) {
bool isHex = spelling.size() > 1 && spelling[1] == 'x'; bool isHex = spelling.size() > 1 && spelling[1] == 'x';
uint64_t result = 0; uint64_t result = 0;
if (spelling.getAsInteger(isHex ? 0 : 10, result)) if (spelling.getAsInteger(isHex ? 0 : 10, result))
return None; return std::nullopt;
return result; return result;
} }
/// For a floatliteral, return its value as a double. Return None if the value /// For a floatliteral, return its value as a double. Return None if the value
/// underflows or overflows. /// underflows or overflows.
Optional<double> Token::getFloatingPointValue() const { Optional<double> Token::getFloatingPointValue() const {
double result = 0; double result = 0;
if (spelling.getAsDouble(result)) if (spelling.getAsDouble(result))
return None; return std::nullopt;
return result; return result;
} }
/// For an inttype token, return its bitwidth. /// For an inttype token, return its bitwidth.
Optional<unsigned> Token::getIntTypeBitwidth() const { Optional<unsigned> Token::getIntTypeBitwidth() const {
assert(getKind() == inttype); assert(getKind() == inttype);
unsigned bitwidthStart = (spelling[0] == 'i' ? 1 : 2); unsigned bitwidthStart = (spelling[0] == 'i' ? 1 : 2);
unsigned result = 0; unsigned result = 0;
if (spelling.drop_front(bitwidthStart).getAsInteger(10, result)) if (spelling.drop_front(bitwidthStart).getAsInteger(10, result))
return None; return std::nullopt;
return result; return result;
} }
Optional<bool> Token::getIntTypeSignedness() const { Optional<bool> Token::getIntTypeSignedness() const {
assert(getKind() == inttype); assert(getKind() == inttype);
if (spelling[0] == 'i') if (spelling[0] == 'i')
return llvm::None; return std::nullopt;
if (spelling[0] == 's') if (spelling[0] == 's')
return true; return true;
assert(spelling[0] == 'u'); assert(spelling[0] == 'u');
return false; return false;
} }
/// Given a token containing a string literal, return its value, including /// Given a token containing a string literal, return its value, including
/// removing the quote characters and unescaping the contents of the string. The /// removing the quote characters and unescaping the contents of the string. The
▲ Show 20 Lines • Show All 54 Lines • ▼ Show 20 LinesOptional<std::string> Token::getHexStringValue() const {
StringRef bytes = getSpelling().drop_front().drop_back(); StringRef bytes = getSpelling().drop_front().drop_back();
// Try to extract the binary data from the hex string. We expect the hex // Try to extract the binary data from the hex string. We expect the hex
// string to start with `0x` and have an even number of hex nibbles (nibbles // string to start with `0x` and have an even number of hex nibbles (nibbles
// should come in pairs). // should come in pairs).
std::string hex; std::string hex;
if (!bytes.consume_front("0x") || (bytes.size() & 1) || if (!bytes.consume_front("0x") || (bytes.size() & 1) ||
!llvm::tryGetFromHex(bytes, hex)) !llvm::tryGetFromHex(bytes, hex))
return llvm::None; return std::nullopt;
return hex; return hex;
} }
/// Given a token containing a symbol reference, return the unescaped string /// Given a token containing a symbol reference, return the unescaped string
/// value. /// value.
std::string Token::getSymbolReference() const { std::string Token::getSymbolReference() const {
assert(is(Token::at_identifier) && "expected valid @-identifier"); assert(is(Token::at_identifier) && "expected valid @-identifier");
StringRef nameStr = getSpelling().drop_front(); StringRef nameStr = getSpelling().drop_front();
// Check to see if the reference is a string literal, or a bare identifier. // Check to see if the reference is a string literal, or a bare identifier.
if (nameStr.front() == '"') if (nameStr.front() == '"')
return getStringValue(); return getStringValue();
return std::string(nameStr); return std::string(nameStr);
} }
/// Given a hash_identifier token like #123, try to parse the number out of /// Given a hash_identifier token like #123, try to parse the number out of
/// the identifier, returning None if it is a named identifier like #x or /// the identifier, returning None if it is a named identifier like #x or
/// if the integer doesn't fit. /// if the integer doesn't fit.
Optional<unsigned> Token::getHashIdentifierNumber() const { Optional<unsigned> Token::getHashIdentifierNumber() const {
assert(getKind() == hash_identifier); assert(getKind() == hash_identifier);
unsigned result = 0; unsigned result = 0;
if (spelling.drop_front().getAsInteger(10, result)) if (spelling.drop_front().getAsInteger(10, result))
return None; return std::nullopt;
return result; return result;
} }
/// Given a punctuation or keyword token kind, return the spelling of the /// Given a punctuation or keyword token kind, return the spelling of the
/// token as a string. Warning: This will abort on markers, identifiers and /// token as a string. Warning: This will abort on markers, identifiers and
/// literal tokens since they have no fixed spelling. /// literal tokens since they have no fixed spelling.
StringRef Token::getTokenSpelling(Kind kind) { StringRef Token::getTokenSpelling(Kind kind) {
switch (kind) { switch (kind) {
▲ Show 20 Lines • Show All 42 Lines • Show Last 20 Lines

mlir/lib/AsmParser/TypeParser.cpp

Show All 38 LinesOptionalParseResult Parser::parseOptionalType(Type &type) {
case Token::kw_f80: case Token::kw_f80:
case Token::kw_f128: case Token::kw_f128:
case Token::kw_index: case Token::kw_index:
case Token::kw_none: case Token::kw_none:
case Token::exclamation_identifier: case Token::exclamation_identifier:
return failure(!(type = parseType())); return failure(!(type = parseType()));
default: default:
return llvm::None; return std::nullopt;
} }
} }
/// Parse an arbitrary type. /// Parse an arbitrary type.
/// ///
/// type ::= function-type /// type ::= function-type
/// | non-function-type /// | non-function-type
/// ///
▲ Show 20 Lines • Show All 532 Lines • Show Last 20 Lines

mlir/lib/Bindings/Python/IRCore.cpp

Show First 20 Lines • Show All 2,620 Lines • ▼ Show 20 Lines py::class_<PyOperationBase>(m, "_OperationBase", py::module_local())
mlirOperationGetLocation(operation.get())); mlirOperationGetLocation(operation.get()));
}, },
"Returns the source location the operation was defined or derived " "Returns the source location the operation was defined or derived "
"from.") "from.")
.def( .def(
"__str__", "__str__",
[](PyOperationBase &self) { [](PyOperationBase &self) {
return self.getAsm(/*binary=*/false, return self.getAsm(/*binary=*/false,
/*largeElementsLimit=*/llvm::None, /*largeElementsLimit=*/std::nullopt,
/*enableDebugInfo=*/false, /*enableDebugInfo=*/false,
/*prettyDebugInfo=*/false, /*prettyDebugInfo=*/false,
/*printGenericOpForm=*/false, /*printGenericOpForm=*/false,
/*useLocalScope=*/false, /*useLocalScope=*/false,
/*assumeVerified=*/false); /*assumeVerified=*/false);
}, },
"Returns the assembly form of the operation.") "Returns the assembly form of the operation.")
.def("print", &PyOperationBase::print, .def("print", &PyOperationBase::print,
▲ Show 20 Lines • Show All 589 Lines • Show Last 20 Lines

mlir/lib/Bindings/Python/IRModule.cpp

Show First 20 Lines • Show All 85 Lines • ▼ Show 20 Lines
llvm::Optional<py::object> llvm::Optional<py::object>
PyGlobals::lookupDialectClass(const std::string &dialectNamespace) { PyGlobals::lookupDialectClass(const std::string &dialectNamespace) {
loadDialectModule(dialectNamespace); loadDialectModule(dialectNamespace);
// Fast match against the class map first (common case). // Fast match against the class map first (common case).
const auto foundIt = dialectClassMap.find(dialectNamespace); const auto foundIt = dialectClassMap.find(dialectNamespace);
if (foundIt != dialectClassMap.end()) { if (foundIt != dialectClassMap.end()) {
if (foundIt->second.is_none()) if (foundIt->second.is_none())
return llvm::None; return std::nullopt;
assert(foundIt->second && "py::object is defined"); assert(foundIt->second && "py::object is defined");
return foundIt->second; return foundIt->second;
} }
// Not found and loading did not yield a registration. Negative cache. // Not found and loading did not yield a registration. Negative cache.
dialectClassMap[dialectNamespace] = py::none(); dialectClassMap[dialectNamespace] = py::none();
return llvm::None; return std::nullopt;
} }
llvm::Optional<pybind11::object> llvm::Optional<pybind11::object>
PyGlobals::lookupRawOpViewClass(llvm::StringRef operationName) { PyGlobals::lookupRawOpViewClass(llvm::StringRef operationName) {
{ {
auto foundIt = rawOpViewClassMapCache.find(operationName); auto foundIt = rawOpViewClassMapCache.find(operationName);
if (foundIt != rawOpViewClassMapCache.end()) { if (foundIt != rawOpViewClassMapCache.end()) {
if (foundIt->second.is_none()) if (foundIt->second.is_none())
return llvm::None; return std::nullopt;
assert(foundIt->second && "py::object is defined"); assert(foundIt->second && "py::object is defined");
return foundIt->second; return foundIt->second;
} }
} }
// Not found. Load the dialect namespace. // Not found. Load the dialect namespace.
auto split = operationName.split('.'); auto split = operationName.split('.');
llvm::StringRef dialectNamespace = split.first; llvm::StringRef dialectNamespace = split.first;
loadDialectModule(dialectNamespace); loadDialectModule(dialectNamespace);
// Attempt to find from the canonical map and cache. // Attempt to find from the canonical map and cache.
{ {
auto foundIt = rawOpViewClassMap.find(operationName); auto foundIt = rawOpViewClassMap.find(operationName);
if (foundIt != rawOpViewClassMap.end()) { if (foundIt != rawOpViewClassMap.end()) {
if (foundIt->second.is_none()) if (foundIt->second.is_none())
return llvm::None; return std::nullopt;
assert(foundIt->second && "py::object is defined"); assert(foundIt->second && "py::object is defined");
// Positive cache. // Positive cache.
rawOpViewClassMapCache[operationName] = foundIt->second; rawOpViewClassMapCache[operationName] = foundIt->second;
return foundIt->second; return foundIt->second;
} }
// Negative cache. // Negative cache.
rawOpViewClassMap[operationName] = py::none(); rawOpViewClassMap[operationName] = py::none();
return llvm::None; return std::nullopt;
} }
} }
void PyGlobals::clearImportCache() { void PyGlobals::clearImportCache() {
loadedDialectModulesCache.clear(); loadedDialectModulesCache.clear();
rawOpViewClassMapCache.clear(); rawOpViewClassMapCache.clear();
} }

mlir/lib/Bindings/Python/IRTypes.cpp

Show First 20 Lines • Show All 384 Lines • ▼ Show 20 Lines c.def_static(
py::arg("shape"), py::arg("element_type"), py::arg("shape"), py::arg("element_type"),
py::arg("encoding") = py::none(), py::arg("loc") = py::none(), py::arg("encoding") = py::none(), py::arg("loc") = py::none(),
"Create a ranked tensor type"); "Create a ranked tensor type");
c.def_property_readonly( c.def_property_readonly(
"encoding", "encoding",
[](PyRankedTensorType &self) -> llvm::Optional<PyAttribute> { [](PyRankedTensorType &self) -> llvm::Optional<PyAttribute> {
MlirAttribute encoding = mlirRankedTensorTypeGetEncoding(self.get()); MlirAttribute encoding = mlirRankedTensorTypeGetEncoding(self.get());
if (mlirAttributeIsNull(encoding)) if (mlirAttributeIsNull(encoding))
return llvm::None; return std::nullopt;
return PyAttribute(self.getContext(), encoding); return PyAttribute(self.getContext(), encoding);
}); });
} }
}; };
/// Unranked Tensor Type subclass - UnrankedTensorType. /// Unranked Tensor Type subclass - UnrankedTensorType.
class PyUnrankedTensorType class PyUnrankedTensorType
: public PyConcreteType<PyUnrankedTensorType, PyShapedType> { : public PyConcreteType<PyUnrankedTensorType, PyShapedType> {
▲ Show 20 Lines • Show All 280 Lines • Show Last 20 Lines

mlir/lib/Conversion/AsyncToLLVM/AsyncToLLVM.cpp

Show First 20 Lines • Show All 298 Lines • ▼ Show 20 Lines static Optional<Type> convertAsyncTypes(Type type) {
if (type.isa<TokenType, GroupType, ValueType>()) if (type.isa<TokenType, GroupType, ValueType>())
return AsyncAPI::opaquePointerType(type.getContext()); return AsyncAPI::opaquePointerType(type.getContext());
if (type.isa<CoroIdType, CoroStateType>()) if (type.isa<CoroIdType, CoroStateType>())
return AsyncAPI::tokenType(type.getContext()); return AsyncAPI::tokenType(type.getContext());
if (type.isa<CoroHandleType>()) if (type.isa<CoroHandleType>())
return AsyncAPI::opaquePointerType(type.getContext()); return AsyncAPI::opaquePointerType(type.getContext());
return llvm::None; return std::nullopt;
} }
}; };
} // namespace } // namespace
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Convert async.coro.id to @llvm.coro.id intrinsic. // Convert async.coro.id to @llvm.coro.id intrinsic.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 827 Lines • Show Last 20 Lines

mlir/lib/Conversion/ControlFlowToLLVM/ControlFlowToLLVM.cpp

Show First 20 Lines • Show All 60 Lines • ▼ Show 20 Lines matchAndRewrite(cf::AssertOp op, OpAdaptor adaptor,
// Split block at `assert` operation. // Split block at `assert` operation.
Block *opBlock = rewriter.getInsertionBlock(); Block *opBlock = rewriter.getInsertionBlock();
auto opPosition = rewriter.getInsertionPoint(); auto opPosition = rewriter.getInsertionPoint();
Block *continuationBlock = rewriter.splitBlock(opBlock, opPosition); Block *continuationBlock = rewriter.splitBlock(opBlock, opPosition);
// Generate IR to call `abort`. // Generate IR to call `abort`.
Block *failureBlock = rewriter.createBlock(opBlock->getParent()); Block *failureBlock = rewriter.createBlock(opBlock->getParent());
rewriter.create<LLVM::CallOp>(loc, abortFunc, llvm::None); rewriter.create<LLVM::CallOp>(loc, abortFunc, std::nullopt);
rewriter.create<LLVM::UnreachableOp>(loc); rewriter.create<LLVM::UnreachableOp>(loc);
// Generate assertion test. // Generate assertion test.
rewriter.setInsertionPointToEnd(opBlock); rewriter.setInsertionPointToEnd(opBlock);
rewriter.replaceOpWithNewOp<LLVM::CondBrOp>( rewriter.replaceOpWithNewOp<LLVM::CondBrOp>(
op, adaptor.getArg(), continuationBlock, failureBlock); op, adaptor.getArg(), continuationBlock, failureBlock);
return success(); return success();
▲ Show 20 Lines • Show All 152 Lines • Show Last 20 Lines

mlir/lib/Conversion/GPUToNVVM/LowerGpuOpsToNVVMOps.cpp

Show First 20 Lines • Show All 177 Lines • ▼ Show 20 Lines void runOnOperation() override {
// MemRef conversion for GPU to NVVM lowering. The GPU dialect uses memory // MemRef conversion for GPU to NVVM lowering. The GPU dialect uses memory
// space 5 for private memory attributions, but NVVM represents private // space 5 for private memory attributions, but NVVM represents private
// memory allocations as local `alloca`s in the default address space. This // memory allocations as local `alloca`s in the default address space. This
// converter drops the private memory space to support the use case above. // converter drops the private memory space to support the use case above.
LLVMTypeConverter converter(m.getContext(), options); LLVMTypeConverter converter(m.getContext(), options);
converter.addConversion([&](MemRefType type) -> Optional<Type> { converter.addConversion([&](MemRefType type) -> Optional<Type> {
if (type.getMemorySpaceAsInt() != if (type.getMemorySpaceAsInt() !=
gpu::GPUDialect::getPrivateAddressSpace()) gpu::GPUDialect::getPrivateAddressSpace())
return llvm::None; return std::nullopt;
return converter.convertType(MemRefType::Builder(type).setMemorySpace( return converter.convertType(MemRefType::Builder(type).setMemorySpace(
IntegerAttr::get(IntegerType::get(m.getContext(), 64), 0))); IntegerAttr::get(IntegerType::get(m.getContext(), 64), 0)));
}); });
// Lowering for MMAMatrixType. // Lowering for MMAMatrixType.
converter.addConversion([&](gpu::MMAMatrixType type) -> Type { converter.addConversion([&](gpu::MMAMatrixType type) -> Type {
return convertMMAToLLVMType(type); return convertMMAToLLVMType(type);
}); });
RewritePatternSet patterns(m.getContext()); RewritePatternSet patterns(m.getContext());
▲ Show 20 Lines • Show All 105 Lines • Show Last 20 Lines

mlir/lib/Conversion/GPUToSPIRV/GPUToSPIRV.cpp

Show First 20 Lines • Show All 218 Lines • ▼ Show 20 Lines for (const auto &argType :
if (!convertedType) if (!convertedType)
return nullptr; return nullptr;
signatureConverter.addInputs(argType.index(), convertedType); signatureConverter.addInputs(argType.index(), convertedType);
} }
} }
auto newFuncOp = rewriter.create<spirv::FuncOp>( auto newFuncOp = rewriter.create<spirv::FuncOp>(
funcOp.getLoc(), funcOp.getName(), funcOp.getLoc(), funcOp.getName(),
rewriter.getFunctionType(signatureConverter.getConvertedTypes(), rewriter.getFunctionType(signatureConverter.getConvertedTypes(),
llvm::None)); std::nullopt));
for (const auto &namedAttr : funcOp->getAttrs()) { for (const auto &namedAttr : funcOp->getAttrs()) {
if (namedAttr.getName() == FunctionOpInterface::getTypeAttrName() || if (namedAttr.getName() == FunctionOpInterface::getTypeAttrName() ||
namedAttr.getName() == SymbolTable::getSymbolAttrName()) namedAttr.getName() == SymbolTable::getSymbolAttrName())
continue; continue;
newFuncOp->setAttr(namedAttr.getName(), namedAttr.getValue()); newFuncOp->setAttr(namedAttr.getName(), namedAttr.getValue());
} }
rewriter.inlineRegionBefore(funcOp.getBody(), newFuncOp.getBody(), rewriter.inlineRegionBefore(funcOp.getBody(), newFuncOp.getBody(),
▲ Show 20 Lines • Show All 88 Lines • ▼ Show 20 LinesLogicalResult GPUModuleConversion::matchAndRewrite(
FailureOr<spirv::MemoryModel> memoryModel = spirv::getMemoryModel(targetEnv); FailureOr<spirv::MemoryModel> memoryModel = spirv::getMemoryModel(targetEnv);
if (failed(memoryModel)) if (failed(memoryModel))
return moduleOp.emitRemark("match failure: could not selected memory model " return moduleOp.emitRemark("match failure: could not selected memory model "
"based on 'spirv.target_env'"); "based on 'spirv.target_env'");
// Add a keyword to the module name to avoid symbolic conflict. // Add a keyword to the module name to avoid symbolic conflict.
std::string spvModuleName = (kSPIRVModule + moduleOp.getName()).str(); std::string spvModuleName = (kSPIRVModule + moduleOp.getName()).str();
auto spvModule = rewriter.create<spirv::ModuleOp>( auto spvModule = rewriter.create<spirv::ModuleOp>(
moduleOp.getLoc(), addressingModel, *memoryModel, llvm::None, moduleOp.getLoc(), addressingModel, *memoryModel, std::nullopt,
StringRef(spvModuleName)); StringRef(spvModuleName));
// Move the region from the module op into the SPIR-V module. // Move the region from the module op into the SPIR-V module.
Region &spvModuleRegion = spvModule.getRegion(); Region &spvModuleRegion = spvModule.getRegion();
rewriter.inlineRegionBefore(moduleOp.getBodyRegion(), spvModuleRegion, rewriter.inlineRegionBefore(moduleOp.getBodyRegion(), spvModuleRegion,
spvModuleRegion.begin()); spvModuleRegion.begin());
// The spirv.module build method adds a block. Remove that. // The spirv.module build method adds a block. Remove that.
rewriter.eraseBlock(&spvModuleRegion.back()); rewriter.eraseBlock(&spvModuleRegion.back());
▲ Show 20 Lines • Show All 112 Lines • Show Last 20 Lines

mlir/lib/Conversion/LLVMCommon/TypeConverter.cpp

Show All 37 Lines addConversion(
[&](UnrankedMemRefType type) { return convertUnrankedMemRefType(type); }); [&](UnrankedMemRefType type) { return convertUnrankedMemRefType(type); });
addConversion([&](VectorType type) { return convertVectorType(type); }); addConversion([&](VectorType type) { return convertVectorType(type); });
// LLVM-compatible types are legal, so add a pass-through conversion. Do this // LLVM-compatible types are legal, so add a pass-through conversion. Do this
// before the conversions below since conversions are attempted in reverse // before the conversions below since conversions are attempted in reverse
// order and those should take priority. // order and those should take priority.
addConversion([](Type type) { addConversion([](Type type) {
return LLVM::isCompatibleType(type) ? llvm::Optional<Type>(type) return LLVM::isCompatibleType(type) ? llvm::Optional<Type>(type)
: llvm::None; : std::nullopt;
}); });
// LLVM container types may (recursively) contain other types that must be // LLVM container types may (recursively) contain other types that must be
// converted even when the outer type is compatible. // converted even when the outer type is compatible.
addConversion([&](LLVM::LLVMPointerType type) -> llvm::Optional<Type> { addConversion([&](LLVM::LLVMPointerType type) -> llvm::Optional<Type> {
if (type.isOpaque()) if (type.isOpaque())
return type; return type;
if (auto pointee = convertType(type.getElementType())) if (auto pointee = convertType(type.getElementType()))
return LLVM::LLVMPointerType::get(pointee, type.getAddressSpace()); return LLVM::LLVMPointerType::get(pointee, type.getAddressSpace());
return llvm::None; return std::nullopt;
}); });
addConversion([&](LLVM::LLVMStructType type, SmallVectorImpl<Type> &results, addConversion([&](LLVM::LLVMStructType type, SmallVectorImpl<Type> &results,
ArrayRef<Type> callStack) -> llvm::Optional<LogicalResult> { ArrayRef<Type> callStack) -> llvm::Optional<LogicalResult> {
// Fastpath for types that won't be converted by this callback anyway. // Fastpath for types that won't be converted by this callback anyway.
if (LLVM::isCompatibleType(type)) { if (LLVM::isCompatibleType(type)) {
results.push_back(type); results.push_back(type);
return success(); return success();
} }
Show All 12 Lines if (type.isIdentified()) {
if (llvm::count(callStack, type) > 1) { if (llvm::count(callStack, type) > 1) {
results.push_back(convertedType); results.push_back(convertedType);
return success(); return success();
} }
SmallVector<Type> convertedElemTypes; SmallVector<Type> convertedElemTypes;
convertedElemTypes.reserve(type.getBody().size()); convertedElemTypes.reserve(type.getBody().size());
if (failed(convertTypes(type.getBody(), convertedElemTypes))) if (failed(convertTypes(type.getBody(), convertedElemTypes)))
return llvm::None; return std::nullopt;
if (failed(convertedType.setBody(convertedElemTypes, type.isPacked()))) if (failed(convertedType.setBody(convertedElemTypes, type.isPacked())))
return failure(); return failure();
results.push_back(convertedType); results.push_back(convertedType);
return success(); return success();
} }
SmallVector<Type> convertedSubtypes; SmallVector<Type> convertedSubtypes;
convertedSubtypes.reserve(type.getBody().size()); convertedSubtypes.reserve(type.getBody().size());
if (failed(convertTypes(type.getBody(), convertedSubtypes))) if (failed(convertTypes(type.getBody(), convertedSubtypes)))
return llvm::None; return std::nullopt;
results.push_back(LLVM::LLVMStructType::getLiteral( results.push_back(LLVM::LLVMStructType::getLiteral(
type.getContext(), convertedSubtypes, type.isPacked())); type.getContext(), convertedSubtypes, type.isPacked()));
return success(); return success();
}); });
addConversion([&](LLVM::LLVMArrayType type) -> llvm::Optional<Type> { addConversion([&](LLVM::LLVMArrayType type) -> llvm::Optional<Type> {
if (auto element = convertType(type.getElementType())) if (auto element = convertType(type.getElementType()))
return LLVM::LLVMArrayType::get(element, type.getNumElements()); return LLVM::LLVMArrayType::get(element, type.getNumElements());
return llvm::None; return std::nullopt;
}); });
addConversion([&](LLVM::LLVMFunctionType type) -> llvm::Optional<Type> { addConversion([&](LLVM::LLVMFunctionType type) -> llvm::Optional<Type> {
Type convertedResType = convertType(type.getReturnType()); Type convertedResType = convertType(type.getReturnType());
if (!convertedResType) if (!convertedResType)
return llvm::None; return std::nullopt;
SmallVector<Type> convertedArgTypes; SmallVector<Type> convertedArgTypes;
convertedArgTypes.reserve(type.getNumParams()); convertedArgTypes.reserve(type.getNumParams());
if (failed(convertTypes(type.getParams(), convertedArgTypes))) if (failed(convertTypes(type.getParams(), convertedArgTypes)))
return llvm::None; return std::nullopt;
return LLVM::LLVMFunctionType::get(convertedResType, convertedArgTypes, return LLVM::LLVMFunctionType::get(convertedResType, convertedArgTypes,
type.isVarArg()); type.isVarArg());
}); });
// Materialization for memrefs creates descriptor structs from individual // Materialization for memrefs creates descriptor structs from individual
// values constituting them, when descriptors are used, i.e. more than one // values constituting them, when descriptors are used, i.e. more than one
// value represents a memref. // value represents a memref.
addArgumentMaterialization( addArgumentMaterialization(
[&](OpBuilder &builder, UnrankedMemRefType resultType, ValueRange inputs, [&](OpBuilder &builder, UnrankedMemRefType resultType, ValueRange inputs,
Location loc) -> Optional<Value> { Location loc) -> Optional<Value> {
if (inputs.size() == 1) if (inputs.size() == 1)
return llvm::None; return std::nullopt;
return UnrankedMemRefDescriptor::pack(builder, loc, *this, resultType, return UnrankedMemRefDescriptor::pack(builder, loc, *this, resultType,
inputs); inputs);
}); });
addArgumentMaterialization([&](OpBuilder &builder, MemRefType resultType, addArgumentMaterialization([&](OpBuilder &builder, MemRefType resultType,
ValueRange inputs, ValueRange inputs,
Location loc) -> Optional<Value> { Location loc) -> Optional<Value> {
// TODO: bare ptr conversion could be handled here but we would need a way // TODO: bare ptr conversion could be handled here but we would need a way
// to distinguish between FuncOp and other regions. // to distinguish between FuncOp and other regions.
if (inputs.size() == 1) if (inputs.size() == 1)
return llvm::None; return std::nullopt;
return MemRefDescriptor::pack(builder, loc, *this, resultType, inputs); return MemRefDescriptor::pack(builder, loc, *this, resultType, inputs);
}); });
// Add generic source and target materializations to handle cases where // Add generic source and target materializations to handle cases where
// non-LLVM types persist after an LLVM conversion. // non-LLVM types persist after an LLVM conversion.
addSourceMaterialization([&](OpBuilder &builder, Type resultType, addSourceMaterialization([&](OpBuilder &builder, Type resultType,
ValueRange inputs, ValueRange inputs,
Location loc) -> Optional<Value> { Location loc) -> Optional<Value> {
if (inputs.size() != 1) if (inputs.size() != 1)
return llvm::None; return std::nullopt;
return builder.create<UnrealizedConversionCastOp>(loc, resultType, inputs) return builder.create<UnrealizedConversionCastOp>(loc, resultType, inputs)
.getResult(0); .getResult(0);
}); });
addTargetMaterialization([&](OpBuilder &builder, Type resultType, addTargetMaterialization([&](OpBuilder &builder, Type resultType,
ValueRange inputs, ValueRange inputs,
Location loc) -> Optional<Value> { Location loc) -> Optional<Value> {
if (inputs.size() != 1) if (inputs.size() != 1)
return llvm::None; return std::nullopt;
return builder.create<UnrealizedConversionCastOp>(loc, resultType, inputs) return builder.create<UnrealizedConversionCastOp>(loc, resultType, inputs)
.getResult(0); .getResult(0);
}); });
} }
/// Returns the MLIR context. /// Returns the MLIR context.
MLIRContext &LLVMTypeConverter::getContext() { MLIRContext &LLVMTypeConverter::getContext() {
▲ Show 20 Lines • Show All 404 Lines • Show Last 20 Lines

mlir/lib/Conversion/LinalgToSPIRV/LinalgToSPIRV.cpp

Show First 20 Lines • Show All 61 Lines • ▼ Show 20 Lines
Optional<linalg::RegionMatcher::BinaryOpKind> Optional<linalg::RegionMatcher::BinaryOpKind>
SingleWorkgroupReduction::matchAsPerformingReduction( SingleWorkgroupReduction::matchAsPerformingReduction(
linalg::GenericOp genericOp) { linalg::GenericOp genericOp) {
Operation *op = genericOp.getOperation(); Operation *op = genericOp.getOperation();
// Make sure the linalg.generic is working on memrefs. // Make sure the linalg.generic is working on memrefs.
if (!genericOp.hasBufferSemantics()) if (!genericOp.hasBufferSemantics())
return llvm::None; return std::nullopt;
// Make sure this is reduction with one input and one output. // Make sure this is reduction with one input and one output.
if (genericOp.getNumDpsInputs() != 1 || genericOp.getNumDpsInits() != 1) if (genericOp.getNumDpsInputs() != 1 || genericOp.getNumDpsInits() != 1)
return llvm::None; return std::nullopt;
auto originalInputType = op->getOperand(0).getType().cast<MemRefType>(); auto originalInputType = op->getOperand(0).getType().cast<MemRefType>();
auto originalOutputType = op->getOperand(1).getType().cast<MemRefType>(); auto originalOutputType = op->getOperand(1).getType().cast<MemRefType>();
// Make sure the original input has one dimension. // Make sure the original input has one dimension.
if (!originalInputType.hasStaticShape() || originalInputType.getRank() != 1) if (!originalInputType.hasStaticShape() || originalInputType.getRank() != 1)
return llvm::None; return std::nullopt;
// Make sure the original output has one element. // Make sure the original output has one element.
if (!originalOutputType.hasStaticShape() || if (!originalOutputType.hasStaticShape() ||
originalOutputType.getNumElements() != 1) originalOutputType.getNumElements() != 1)
return llvm::None; return std::nullopt;
if (!genericOp.hasSingleReductionLoop()) if (!genericOp.hasSingleReductionLoop())
return llvm::None; return std::nullopt;
auto indexingMaps = genericOp.getIndexingMapsArray(); auto indexingMaps = genericOp.getIndexingMapsArray();
if (indexingMaps.size() != 2) if (indexingMaps.size() != 2)
return llvm::None; return std::nullopt;
// TODO: create utility functions for these checks in Linalg // TODO: create utility functions for these checks in Linalg
// and use them. // and use them.
auto inputMap = indexingMaps[0]; auto inputMap = indexingMaps[0];
auto outputMap = indexingMaps[1]; auto outputMap = indexingMaps[1];
// The indexing map for the input should be `(i) -> (i)`. // The indexing map for the input should be `(i) -> (i)`.
if (inputMap != AffineMap::get(1, 0, getAffineDimExpr(0, op->getContext()))) if (inputMap != AffineMap::get(1, 0, getAffineDimExpr(0, op->getContext())))
return llvm::None; return std::nullopt;
// The indexing map for the input should be `(i) -> (0)`. // The indexing map for the input should be `(i) -> (0)`.
if (outputMap != if (outputMap !=
AffineMap::get(1, 0, getAffineConstantExpr(0, op->getContext()))) AffineMap::get(1, 0, getAffineConstantExpr(0, op->getContext())))
return llvm::None; return std::nullopt;
return linalg::RegionMatcher::matchAsScalarBinaryOp(genericOp); return linalg::RegionMatcher::matchAsScalarBinaryOp(genericOp);
} }
LogicalResult SingleWorkgroupReduction::matchAndRewrite( LogicalResult SingleWorkgroupReduction::matchAndRewrite(
linalg::GenericOp genericOp, OpAdaptor adaptor, linalg::GenericOp genericOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const { ConversionPatternRewriter &rewriter) const {
Operation *op = genericOp.getOperation(); Operation *op = genericOp.getOperation();
▲ Show 20 Lines • Show All 97 Lines • Show Last 20 Lines

mlir/lib/Conversion/MemRefToLLVM/MemRefToLLVM.cpp

Show First 20 Lines • Show All 445 Lines • ▼ Show 20 Lines if (Optional<int64_t> idx = dimOp.getConstantIndex())
return idx; return idx;
if (auto constantOp = dimOp.getIndex().getDefiningOp<LLVM::ConstantOp>()) if (auto constantOp = dimOp.getIndex().getDefiningOp<LLVM::ConstantOp>())
return constantOp.getValue() return constantOp.getValue()
.cast<IntegerAttr>() .cast<IntegerAttr>()
.getValue() .getValue()
.getSExtValue(); .getSExtValue();
return llvm::None; return std::nullopt;
} }
Value extractSizeOfRankedMemRef(Type operandType, memref::DimOp dimOp, Value extractSizeOfRankedMemRef(Type operandType, memref::DimOp dimOp,
OpAdaptor adaptor, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const { ConversionPatternRewriter &rewriter) const {
Location loc = dimOp.getLoc(); Location loc = dimOp.getLoc();
// Take advantage if index is constant. // Take advantage if index is constant.
▲ Show 20 Lines • Show All 810 Lines • ▼ Show 20 Lines convertSourceMemRefToDescriptor(ConversionPatternRewriter &rewriter,
// inner descriptor is allocated on stack. // inner descriptor is allocated on stack.
auto targetDesc = UnrankedMemRefDescriptor::undef( auto targetDesc = UnrankedMemRefDescriptor::undef(
rewriter, loc, typeConverter->convertType(targetType)); rewriter, loc, typeConverter->convertType(targetType));
targetDesc.setRank(rewriter, loc, resultRank); targetDesc.setRank(rewriter, loc, resultRank);
SmallVector<Value, 4> sizes; SmallVector<Value, 4> sizes;
UnrankedMemRefDescriptor::computeSizes(rewriter, loc, *getTypeConverter(), UnrankedMemRefDescriptor::computeSizes(rewriter, loc, *getTypeConverter(),
targetDesc, sizes); targetDesc, sizes);
Value underlyingDescPtr = rewriter.create<LLVM::AllocaOp>( Value underlyingDescPtr = rewriter.create<LLVM::AllocaOp>(
loc, getVoidPtrType(), sizes.front(), llvm::None); loc, getVoidPtrType(), sizes.front(), std::nullopt);
targetDesc.setMemRefDescPtr(rewriter, loc, underlyingDescPtr); targetDesc.setMemRefDescPtr(rewriter, loc, underlyingDescPtr);
// Extract pointers and offset from the source memref. // Extract pointers and offset from the source memref.
Value allocatedPtr, alignedPtr, offset; Value allocatedPtr, alignedPtr, offset;
extractPointersAndOffset(loc, rewriter, *getTypeConverter(), extractPointersAndOffset(loc, rewriter, *getTypeConverter(),
reshapeOp.getSource(), adaptor.getSource(), reshapeOp.getSource(), adaptor.getSource(),
&allocatedPtr, &alignedPtr, &offset); &allocatedPtr, &alignedPtr, &offset);
▲ Show 20 Lines • Show All 67 Lines • ▼ Show 20 Lines convertSourceMemRefToDescriptor(ConversionPatternRewriter &rewriter,
rewriter.create<LLVM::BrOp>(loc, ValueRange({decrement, nextStride}), rewriter.create<LLVM::BrOp>(loc, ValueRange({decrement, nextStride}),
condBlock); condBlock);
Block *remainder = Block *remainder =
rewriter.splitBlock(bodyBlock, rewriter.getInsertionPoint()); rewriter.splitBlock(bodyBlock, rewriter.getInsertionPoint());
// Hook up the cond exit to the remainder. // Hook up the cond exit to the remainder.
rewriter.setInsertionPointToEnd(condBlock); rewriter.setInsertionPointToEnd(condBlock);
rewriter.create<LLVM::CondBrOp>(loc, pred, bodyBlock, llvm::None, remainder, rewriter.create<LLVM::CondBrOp>(loc, pred, bodyBlock, std::nullopt,
llvm::None); remainder, std::nullopt);
// Reset position to beginning of new remainder block. // Reset position to beginning of new remainder block.
rewriter.setInsertionPointToStart(remainder); rewriter.setInsertionPointToStart(remainder);
*descriptor = targetDesc; *descriptor = targetDesc;
return success(); return success();
} }
}; };
▲ Show 20 Lines • Show All 219 Lines • ▼ Show 20 Lines if (!ShapedType::isDynamic(srcShape[ref.back()]) || ref.size() == 1) {
loc, LLVM::ICmpPredicate::ne, srcDesc.size(rewriter, loc, srcIndex), loc, LLVM::ICmpPredicate::ne, srcDesc.size(rewriter, loc, srcIndex),
one); one);
{ {
OpBuilder::InsertionGuard guard(rewriter); OpBuilder::InsertionGuard guard(rewriter);
nextEntryBlock = rewriter.createBlock( nextEntryBlock = rewriter.createBlock(
initBlock->getParent(), Region::iterator(continueBlock), {}); initBlock->getParent(), Region::iterator(continueBlock), {});
} }
rewriter.create<LLVM::CondBrOp>(loc, predNeOne, continueBlock, rewriter.create<LLVM::CondBrOp>(loc, predNeOne, continueBlock,
srcStride, nextEntryBlock, llvm::None); srcStride, nextEntryBlock,
std::nullopt);
curEntryBlock = nextEntryBlock; curEntryBlock = nextEntryBlock;
} }
} }
} }
} }
static void fillInDynamicStridesForMemDescriptor( static void fillInDynamicStridesForMemDescriptor(
ConversionPatternRewriter &b, Location loc, Operation *op, ConversionPatternRewriter &b, Location loc, Operation *op,
▲ Show 20 Lines • Show All 433 Lines • ▼ Show 20 Lines case arith::AtomicRMWKind::mins:
return LLVM::AtomicBinOp::min; return LLVM::AtomicBinOp::min;
case arith::AtomicRMWKind::minu: case arith::AtomicRMWKind::minu:
return LLVM::AtomicBinOp::umin; return LLVM::AtomicBinOp::umin;
case arith::AtomicRMWKind::ori: case arith::AtomicRMWKind::ori:
return LLVM::AtomicBinOp::_or; return LLVM::AtomicBinOp::_or;
case arith::AtomicRMWKind::andi: case arith::AtomicRMWKind::andi:
return LLVM::AtomicBinOp::_and; return LLVM::AtomicBinOp::_and;
default: default:
return llvm::None; return std::nullopt;
} }
llvm_unreachable("Invalid AtomicRMWKind"); llvm_unreachable("Invalid AtomicRMWKind");
} }
struct AtomicRMWOpLowering : public LoadStoreOpLowering<memref::AtomicRMWOp> { struct AtomicRMWOpLowering : public LoadStoreOpLowering<memref::AtomicRMWOp> {
using Base::Base; using Base::Base;
LogicalResult LogicalResult
▲ Show 20 Lines • Show All 157 Lines • Show Last 20 Lines

mlir/lib/Conversion/MemRefToSPIRV/MapMemRefStorageClassPass.cpp

Show First 20 Lines • Show All 60 Lines • ▼ Show 20 Linesspirv::mapMemorySpaceToVulkanStorageClass(Attribute memorySpaceAttr) {
// Handle null memory space attribute specially. // Handle null memory space attribute specially.
if (!memorySpaceAttr) if (!memorySpaceAttr)
return spirv::StorageClass::StorageBuffer; return spirv::StorageClass::StorageBuffer;
// Unknown dialect custom attributes are not supported by default. // Unknown dialect custom attributes are not supported by default.
// Downstream callers should plug in more specialized ones. // Downstream callers should plug in more specialized ones.
auto intAttr = memorySpaceAttr.dyn_cast<IntegerAttr>(); auto intAttr = memorySpaceAttr.dyn_cast<IntegerAttr>();
if (!intAttr) if (!intAttr)
return llvm::None; return std::nullopt;
unsigned memorySpace = intAttr.getInt(); unsigned memorySpace = intAttr.getInt();
#define STORAGE_SPACE_MAP_FN(storage, space) \ #define STORAGE_SPACE_MAP_FN(storage, space) \
case space: \ case space: \
return storage; return storage;
switch (memorySpace) { switch (memorySpace) {
VULKAN_STORAGE_SPACE_MAP_LIST(STORAGE_SPACE_MAP_FN) VULKAN_STORAGE_SPACE_MAP_LIST(STORAGE_SPACE_MAP_FN)
default: default:
break; break;
} }
return llvm::None; return std::nullopt;
#undef STORAGE_SPACE_MAP_FN #undef STORAGE_SPACE_MAP_FN
} }
Optional<unsigned> Optional<unsigned>
spirv::mapVulkanStorageClassToMemorySpace(spirv::StorageClass storageClass) { spirv::mapVulkanStorageClassToMemorySpace(spirv::StorageClass storageClass) {
#define STORAGE_SPACE_MAP_FN(storage, space) \ #define STORAGE_SPACE_MAP_FN(storage, space) \
case storage: \ case storage: \
return space; return space;
switch (storageClass) { switch (storageClass) {
VULKAN_STORAGE_SPACE_MAP_LIST(STORAGE_SPACE_MAP_FN) VULKAN_STORAGE_SPACE_MAP_LIST(STORAGE_SPACE_MAP_FN)
default: default:
break; break;
} }
return llvm::None; return std::nullopt;
#undef STORAGE_SPACE_MAP_FN #undef STORAGE_SPACE_MAP_FN
} }
#undef VULKAN_STORAGE_SPACE_MAP_LIST #undef VULKAN_STORAGE_SPACE_MAP_LIST
#define OPENCL_STORAGE_SPACE_MAP_LIST(MAP_FN) \ #define OPENCL_STORAGE_SPACE_MAP_LIST(MAP_FN) \
MAP_FN(spirv::StorageClass::CrossWorkgroup, 0) \ MAP_FN(spirv::StorageClass::CrossWorkgroup, 0) \
Show All 9 Linesspirv::mapMemorySpaceToOpenCLStorageClass(Attribute memorySpaceAttr) {
// Handle null memory space attribute specially. // Handle null memory space attribute specially.
if (!memorySpaceAttr) if (!memorySpaceAttr)
return spirv::StorageClass::CrossWorkgroup; return spirv::StorageClass::CrossWorkgroup;
// Unknown dialect custom attributes are not supported by default. // Unknown dialect custom attributes are not supported by default.
// Downstream callers should plug in more specialized ones. // Downstream callers should plug in more specialized ones.
auto intAttr = memorySpaceAttr.dyn_cast<IntegerAttr>(); auto intAttr = memorySpaceAttr.dyn_cast<IntegerAttr>();
if (!intAttr) if (!intAttr)
return llvm::None; return std::nullopt;
unsigned memorySpace = intAttr.getInt(); unsigned memorySpace = intAttr.getInt();
#define STORAGE_SPACE_MAP_FN(storage, space) \ #define STORAGE_SPACE_MAP_FN(storage, space) \
case space: \ case space: \
return storage; return storage;
switch (memorySpace) { switch (memorySpace) {
OPENCL_STORAGE_SPACE_MAP_LIST(STORAGE_SPACE_MAP_FN) OPENCL_STORAGE_SPACE_MAP_LIST(STORAGE_SPACE_MAP_FN)
default: default:
break; break;
} }
return llvm::None; return std::nullopt;
#undef STORAGE_SPACE_MAP_FN #undef STORAGE_SPACE_MAP_FN
} }
Optional<unsigned> Optional<unsigned>
spirv::mapOpenCLStorageClassToMemorySpace(spirv::StorageClass storageClass) { spirv::mapOpenCLStorageClassToMemorySpace(spirv::StorageClass storageClass) {
#define STORAGE_SPACE_MAP_FN(storage, space) \ #define STORAGE_SPACE_MAP_FN(storage, space) \
case storage: \ case storage: \
return space; return space;
switch (storageClass) { switch (storageClass) {
OPENCL_STORAGE_SPACE_MAP_LIST(STORAGE_SPACE_MAP_FN) OPENCL_STORAGE_SPACE_MAP_LIST(STORAGE_SPACE_MAP_FN)
default: default:
break; break;
} }
return llvm::None; return std::nullopt;
#undef STORAGE_SPACE_MAP_FN #undef STORAGE_SPACE_MAP_FN
} }
#undef OPENCL_STORAGE_SPACE_MAP_LIST #undef OPENCL_STORAGE_SPACE_MAP_LIST
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Type Converter // Type Converter
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
spirv::MemorySpaceToStorageClassConverter::MemorySpaceToStorageClassConverter( spirv::MemorySpaceToStorageClassConverter::MemorySpaceToStorageClassConverter(
const spirv::MemorySpaceToStorageClassMap &memorySpaceMap) const spirv::MemorySpaceToStorageClassMap &memorySpaceMap)
: memorySpaceMap(memorySpaceMap) { : memorySpaceMap(memorySpaceMap) {
// Pass through for all other types. // Pass through for all other types.
addConversion([](Type type) { return type; }); addConversion([](Type type) { return type; });
addConversion([this](BaseMemRefType memRefType) -> Optional<Type> { addConversion([this](BaseMemRefType memRefType) -> Optional<Type> {
Optional<spirv::StorageClass> storage = Optional<spirv::StorageClass> storage =
this->memorySpaceMap(memRefType.getMemorySpace()); this->memorySpaceMap(memRefType.getMemorySpace());
if (!storage) { if (!storage) {
LLVM_DEBUG(llvm::dbgs() LLVM_DEBUG(llvm::dbgs()
<< "cannot convert " << memRefType << "cannot convert " << memRefType
<< " due to being unable to find memory space in map\n"); << " due to being unable to find memory space in map\n");
return llvm::None; return std::nullopt;
} }
auto storageAttr = auto storageAttr =
spirv::StorageClassAttr::get(memRefType.getContext(), *storage); spirv::StorageClassAttr::get(memRefType.getContext(), *storage);
if (auto rankedType = memRefType.dyn_cast<MemRefType>()) { if (auto rankedType = memRefType.dyn_cast<MemRefType>()) {
return MemRefType::get(memRefType.getShape(), memRefType.getElementType(), return MemRefType::get(memRefType.getShape(), memRefType.getElementType(),
rankedType.getLayout(), storageAttr); rankedType.getLayout(), storageAttr);
} }
▲ Show 20 Lines • Show All 181 Lines • Show Last 20 Lines

mlir/lib/Conversion/NVGPUToNVVM/NVGPUToNVVM.cpp

Show First 20 Lines • Show All 299 Lines • ▼ Show 20 Lines if (failed(ptxTypeB))
return op->emitOpError("failed to deduce operand PTX types"); return op->emitOpError("failed to deduce operand PTX types");
Optional<NVVM::MMATypes> ptxTypeC = NVVM::MmaOp::inferOperandMMAType( Optional<NVVM::MMATypes> ptxTypeC = NVVM::MmaOp::inferOperandMMAType(
cType.getElementType(), /*isAccumulator=*/true); cType.getElementType(), /*isAccumulator=*/true);
if (!ptxTypeC) if (!ptxTypeC)
return op->emitError( return op->emitError(
"could not infer the PTX type for the accumulator/result"); "could not infer the PTX type for the accumulator/result");
// TODO: add an attribute to the op to customize this behavior. // TODO: add an attribute to the op to customize this behavior.
Optional<NVVM::MMAIntOverflow> overflow(llvm::None); Optional<NVVM::MMAIntOverflow> overflow(std::nullopt);
if (aType.getElementType().isa<IntegerType>()) if (aType.getElementType().isa<IntegerType>())
overflow = NVVM::MMAIntOverflow::satfinite; overflow = NVVM::MMAIntOverflow::satfinite;
SmallVector<Value> matA = SmallVector<Value> matA =
unpackOperandVector(rewriter, loc, adaptor.getMatrixA(), *ptxTypeA); unpackOperandVector(rewriter, loc, adaptor.getMatrixA(), *ptxTypeA);
SmallVector<Value> matB = SmallVector<Value> matB =
unpackOperandVector(rewriter, loc, adaptor.getMatrixB(), *ptxTypeB); unpackOperandVector(rewriter, loc, adaptor.getMatrixB(), *ptxTypeB);
SmallVector<Value> matC = SmallVector<Value> matC =
unpackOperandVector(rewriter, loc, adaptor.getMatrixC(), *ptxTypeC); unpackOperandVector(rewriter, loc, adaptor.getMatrixC(), *ptxTypeC);
Type desiredRetTy = typeConverter->convertType(op->getResultTypes()[0]); Type desiredRetTy = typeConverter->convertType(op->getResultTypes()[0]);
Type intrinsicResTy = inferIntrinsicResultType( Type intrinsicResTy = inferIntrinsicResultType(
typeConverter->convertType(op->getResultTypes()[0])); typeConverter->convertType(op->getResultTypes()[0]));
Value intrinsicResult = rewriter.create<NVVM::MmaOp>( Value intrinsicResult = rewriter.create<NVVM::MmaOp>(
op.getLoc(), intrinsicResTy, matA, matB, matC, op.getLoc(), intrinsicResTy, matA, matB, matC,
/*shape=*/gemmShape, /*shape=*/gemmShape,
/*b1Op=*/llvm::None, /*b1Op=*/std::nullopt,
/*intOverflow=*/overflow, /*intOverflow=*/overflow,
/*multiplicandPtxTypes=*/ /*multiplicandPtxTypes=*/
std::array<NVVM::MMATypes, 2>{*ptxTypeA, *ptxTypeB}, std::array<NVVM::MMATypes, 2>{*ptxTypeA, *ptxTypeB},
/*multiplicandLayouts=*/ /*multiplicandLayouts=*/
std::array<NVVM::MMALayout, 2>{NVVM::MMALayout::row, std::array<NVVM::MMALayout, 2>{NVVM::MMALayout::row,
NVVM::MMALayout::col}); NVVM::MMALayout::col});
rewriter.replaceOp(op, convertIntrinsicResult(op.getLoc(), intrinsicResTy, rewriter.replaceOp(op, convertIntrinsicResult(op.getLoc(), intrinsicResTy,
desiredRetTy, intrinsicResult, desiredRetTy, intrinsicResult,
▲ Show 20 Lines • Show All 178 Lines • ▼ Show 20 Lines if (!ptxTypeC)
"could not infer the PTX type for the accumulator/result"); "could not infer the PTX type for the accumulator/result");
// Same as `mma.sync`, F32 works only with TensorFloat32 (TF32). // Same as `mma.sync`, F32 works only with TensorFloat32 (TF32).
bool tf32Enabled = op->hasAttr(op.getTf32EnabledAttrName()); bool tf32Enabled = op->hasAttr(op.getTf32EnabledAttrName());
if (aType.getElementType().isF32() && !tf32Enabled) if (aType.getElementType().isF32() && !tf32Enabled)
return failure(); return failure();
// TODO: add an attribute to the op to customize this behavior. // TODO: add an attribute to the op to customize this behavior.
Optional<NVVM::MMAIntOverflow> overflow(llvm::None); Optional<NVVM::MMAIntOverflow> overflow(std::nullopt);
if (aType.getElementType().isa<IntegerType>()) if (aType.getElementType().isa<IntegerType>())
overflow = NVVM::MMAIntOverflow::satfinite; overflow = NVVM::MMAIntOverflow::satfinite;
SmallVector<Value> matA = SmallVector<Value> matA =
unpackOperandVector(rewriter, loc, adaptor.getMatrixA(), *ptxTypeA); unpackOperandVector(rewriter, loc, adaptor.getMatrixA(), *ptxTypeA);
SmallVector<Value> matB = SmallVector<Value> matB =
unpackOperandVector(rewriter, loc, adaptor.getMatrixB(), *ptxTypeB); unpackOperandVector(rewriter, loc, adaptor.getMatrixB(), *ptxTypeB);
SmallVector<Value> matC = SmallVector<Value> matC =
▲ Show 20 Lines • Show All 143 Lines • Show Last 20 Lines

mlir/lib/Conversion/PDLToPDLInterp/PDLToPDLInterp.cpp

Show First 20 Lines • Show All 608 Lines • ▼ Show 20 Lines if (configMap)
configMap->try_emplace(matchOp, configMap->lookup(pattern)); configMap->try_emplace(matchOp, configMap->lookup(pattern));
} }
SymbolRefAttr PatternLowering::generateRewriter( SymbolRefAttr PatternLowering::generateRewriter(
pdl::PatternOp pattern, SmallVectorImpl<Position *> &usedMatchValues) { pdl::PatternOp pattern, SmallVectorImpl<Position *> &usedMatchValues) {
builder.setInsertionPointToEnd(rewriterModule.getBody()); builder.setInsertionPointToEnd(rewriterModule.getBody());
auto rewriterFunc = builder.create<pdl_interp::FuncOp>( auto rewriterFunc = builder.create<pdl_interp::FuncOp>(
pattern.getLoc(), "pdl_generated_rewriter", pattern.getLoc(), "pdl_generated_rewriter",
builder.getFunctionType(llvm::None, llvm::None)); builder.getFunctionType(std::nullopt, llvm::None));
rewriterSymbolTable.insert(rewriterFunc); rewriterSymbolTable.insert(rewriterFunc);
// Generate the rewriter function body. // Generate the rewriter function body.
builder.setInsertionPointToEnd(&rewriterFunc.front()); builder.setInsertionPointToEnd(&rewriterFunc.front());
// Map an input operand of the pattern to a generated interpreter value. // Map an input operand of the pattern to a generated interpreter value.
DenseMap<Value, Value> rewriteValues; DenseMap<Value, Value> rewriteValues;
auto mapRewriteValue = [&](Value oldValue) { auto mapRewriteValue = [&](Value oldValue) {
▲ Show 20 Lines • Show All 50 Lines • ▼ Show 20 Lines for (Operation &rewriteOp : *rewriter.getBody()) {
this->generateRewriter(op, rewriteValues, mapRewriteValue); this->generateRewriter(op, rewriteValues, mapRewriteValue);
}); });
} }
} }
// Update the signature of the rewrite function. // Update the signature of the rewrite function.
rewriterFunc.setType(builder.getFunctionType( rewriterFunc.setType(builder.getFunctionType(
llvm::to_vector<8>(rewriterFunc.front().getArgumentTypes()), llvm::to_vector<8>(rewriterFunc.front().getArgumentTypes()),
/*results=*/llvm::None)); /*results=*/std::nullopt));
builder.create<pdl_interp::FinalizeOp>(rewriter.getLoc()); builder.create<pdl_interp::FinalizeOp>(rewriter.getLoc());
return SymbolRefAttr::get( return SymbolRefAttr::get(
builder.getContext(), builder.getContext(),
pdl_interp::PDLInterpDialect::getRewriterModuleName(), pdl_interp::PDLInterpDialect::getRewriterModuleName(),
SymbolRefAttr::get(rewriterFunc)); SymbolRefAttr::get(rewriterFunc));
} }
▲ Show 20 Lines • Show All 270 Lines • ▼ Show 20 Linesvoid PDLToPDLInterpPass::runOnOperation() {
ModuleOp module = getOperation(); ModuleOp module = getOperation();
// Create the main matcher function This function contains all of the match // Create the main matcher function This function contains all of the match
// related functionality from patterns in the module. // related functionality from patterns in the module.
OpBuilder builder = OpBuilder::atBlockBegin(module.getBody()); OpBuilder builder = OpBuilder::atBlockBegin(module.getBody());
auto matcherFunc = builder.create<pdl_interp::FuncOp>( auto matcherFunc = builder.create<pdl_interp::FuncOp>(
module.getLoc(), pdl_interp::PDLInterpDialect::getMatcherFunctionName(), module.getLoc(), pdl_interp::PDLInterpDialect::getMatcherFunctionName(),
builder.getFunctionType(builder.getType<pdl::OperationType>(), builder.getFunctionType(builder.getType<pdl::OperationType>(),
/*results=*/llvm::None), /*results=*/std::nullopt),
/*attrs=*/llvm::None); /*attrs=*/std::nullopt);
// Create a nested module to hold the functions invoked for rewriting the IR // Create a nested module to hold the functions invoked for rewriting the IR
// after a successful match. // after a successful match.
ModuleOp rewriterModule = builder.create<ModuleOp>( ModuleOp rewriterModule = builder.create<ModuleOp>(
module.getLoc(), pdl_interp::PDLInterpDialect::getRewriterModuleName()); module.getLoc(), pdl_interp::PDLInterpDialect::getRewriterModuleName());
// Generate the code for the patterns within the module. // Generate the code for the patterns within the module.
PatternLowering generator(matcherFunc, rewriterModule, configMap); PatternLowering generator(matcherFunc, rewriterModule, configMap);
Show All 20 Lines

mlir/lib/Conversion/PDLToPDLInterp/Predicate.h

Show First 20 Lines • Show All 594 Lines • ▼ Show 20 Linespublic:
} }
/// Returns a position for a group of operands of the given operation. /// Returns a position for a group of operands of the given operation.
Position *getOperandGroup(OperationPosition *p, Optional<unsigned> group, Position *getOperandGroup(OperationPosition *p, Optional<unsigned> group,
bool isVariadic) { bool isVariadic) {
return OperandGroupPosition::get(uniquer, p, group, isVariadic); return OperandGroupPosition::get(uniquer, p, group, isVariadic);
} }
Position *getAllOperands(OperationPosition *p) { Position *getAllOperands(OperationPosition *p) {
return getOperandGroup(p, /*group=*/llvm::None, /*isVariadic=*/true); return getOperandGroup(p, /*group=*/std::nullopt, /*isVariadic=*/true);
} }
/// Returns a result position for a result of the given operation. /// Returns a result position for a result of the given operation.
Position *getResult(OperationPosition *p, unsigned result) { Position *getResult(OperationPosition *p, unsigned result) {
return ResultPosition::get(uniquer, p, result); return ResultPosition::get(uniquer, p, result);
} }
/// Returns a position for a group of results of the given operation. /// Returns a position for a group of results of the given operation.
Position *getResultGroup(OperationPosition *p, Optional<unsigned> group, Position *getResultGroup(OperationPosition *p, Optional<unsigned> group,
bool isVariadic) { bool isVariadic) {
return ResultGroupPosition::get(uniquer, p, group, isVariadic); return ResultGroupPosition::get(uniquer, p, group, isVariadic);
} }
Position *getAllResults(OperationPosition *p) { Position *getAllResults(OperationPosition *p) {
return getResultGroup(p, /*group=*/llvm::None, /*isVariadic=*/true); return getResultGroup(p, /*group=*/std::nullopt, /*isVariadic=*/true);
} }
/// Returns a type position for the given entity. /// Returns a type position for the given entity.
Position *getType(Position *p) { return TypePosition::get(uniquer, p); } Position *getType(Position *p) { return TypePosition::get(uniquer, p); }
/// Returns a type position for the given type value. The value is stored /// Returns a type position for the given type value. The value is stored
/// as either a TypeAttr, or an ArrayAttr of TypeAttr. /// as either a TypeAttr, or an ArrayAttr of TypeAttr.
Position *getTypeLiteral(Attribute attr) { Position *getTypeLiteral(Attribute attr) {
▲ Show 20 Lines • Show All 94 Lines • Show Last 20 Lines

mlir/lib/Conversion/PDLToPDLInterp/PredicateTree.cpp

Show First 20 Lines • Show All 104 Lines • ▼ Show 20 Lines TypeSwitch<Operation *>(val.getDefiningOp())
(Position *)parentPos); (Position *)parentPos);
}); });
} }
static void getTreePredicates(std::vector<PositionalPredicate> &predList, static void getTreePredicates(std::vector<PositionalPredicate> &predList,
Value val, PredicateBuilder &builder, Value val, PredicateBuilder &builder,
DenseMap<Value, Position *> &inputs, DenseMap<Value, Position *> &inputs,
OperationPosition *pos, OperationPosition *pos,
Optional<unsigned> ignoreOperand = llvm::None) { Optional<unsigned> ignoreOperand = std::nullopt) {
assert(val.getType().isa<pdl::OperationType>() && "expected operation"); assert(val.getType().isa<pdl::OperationType>() && "expected operation");
pdl::OperationOp op = cast<pdl::OperationOp>(val.getDefiningOp()); pdl::OperationOp op = cast<pdl::OperationOp>(val.getDefiningOp());
OperationPosition *opPos = cast<OperationPosition>(pos); OperationPosition *opPos = cast<OperationPosition>(pos);
// Ensure getDefiningOp returns a non-null operation. // Ensure getDefiningOp returns a non-null operation.
if (!opPos->isRoot()) if (!opPos->isRoot())
predList.emplace_back(pos, builder.getIsNotNull()); predList.emplace_back(pos, builder.getIsNotNull());
▲ Show 20 Lines • Show All 331 Lines • ▼ Show 20 Lines while (!toVisit.empty()) {
// are expensive to join on. // are expensive to join on.
TypeSwitch<Operation *>(entry.value.getDefiningOp()) TypeSwitch<Operation *>(entry.value.getDefiningOp())
.Case<pdl::OperationOp>([&](auto operationOp) { .Case<pdl::OperationOp>([&](auto operationOp) {
OperandRange operands = operationOp.getOperandValues(); OperandRange operands = operationOp.getOperandValues();
// Special case when we pass all the operands in one range. // Special case when we pass all the operands in one range.
// For those, the index is empty. // For those, the index is empty.
if (operands.size() == 1 && if (operands.size() == 1 &&
operands[0].getType().isa<pdl::RangeType>()) { operands[0].getType().isa<pdl::RangeType>()) {
toVisit.emplace(operands[0], entry.value, llvm::None, toVisit.emplace(operands[0], entry.value, std::nullopt,
entry.depth + 1); entry.depth + 1);
return; return;
} }
// Default case: visit all the operands. // Default case: visit all the operands.
for (const auto &p : for (const auto &p :
llvm::enumerate(operationOp.getOperandValues())) llvm::enumerate(operationOp.getOperandValues()))
toVisit.emplace(p.value(), entry.value, p.index(), toVisit.emplace(p.value(), entry.value, p.index(),
▲ Show 20 Lines • Show All 534 Lines • Show Last 20 Lines

mlir/lib/Conversion/SCFToGPU/SCFToGPU.cpp

Show First 20 Lines • Show All 173 Lines • ▼ Show 20 Lines if (auto def = value.getDefiningOp<arith::ConstantIndexOp>())
return def.value() == 1; return def.value() == 1;
return false; return false;
} }
// Collect ranges, bounds, steps and induction variables in preparation for // Collect ranges, bounds, steps and induction variables in preparation for
// mapping a loop nest of depth "numLoops" rooted at "forOp" to a GPU kernel. // mapping a loop nest of depth "numLoops" rooted at "forOp" to a GPU kernel.
// This may fail if the IR for computing loop bounds cannot be constructed, for // This may fail if the IR for computing loop bounds cannot be constructed, for
// example if an affine loop uses semi-affine maps. Return the last loop to be // example if an affine loop uses semi-affine maps. Return the last loop to be
// mapped on success, llvm::None on failure. // mapped on success, std::nullopt on failure.
Optional<AffineForOp> Optional<AffineForOp>
AffineLoopToGpuConverter::collectBounds(AffineForOp forOp, unsigned numLoops) { AffineLoopToGpuConverter::collectBounds(AffineForOp forOp, unsigned numLoops) {
OpBuilder builder(forOp.getOperation()); OpBuilder builder(forOp.getOperation());
dims.reserve(numLoops); dims.reserve(numLoops);
lbs.reserve(numLoops); lbs.reserve(numLoops);
ivs.reserve(numLoops); ivs.reserve(numLoops);
steps.reserve(numLoops); steps.reserve(numLoops);
AffineForOp currentLoop = forOp; AffineForOp currentLoop = forOp;
for (unsigned i = 0; i < numLoops; ++i) { for (unsigned i = 0; i < numLoops; ++i) {
Value lowerBound = getOrEmitLowerBound(currentLoop, builder); Value lowerBound = getOrEmitLowerBound(currentLoop, builder);
Value upperBound = getOrEmitUpperBound(currentLoop, builder); Value upperBound = getOrEmitUpperBound(currentLoop, builder);
if (!lowerBound || !upperBound) { if (!lowerBound || !upperBound) {
return llvm::None; return std::nullopt;
} }
Value range = builder.create<arith::SubIOp>(currentLoop.getLoc(), Value range = builder.create<arith::SubIOp>(currentLoop.getLoc(),
upperBound, lowerBound); upperBound, lowerBound);
Value step = getOrCreateStep(currentLoop, builder); Value step = getOrCreateStep(currentLoop, builder);
if (!isConstantOne(step)) if (!isConstantOne(step))
range = builder.create<arith::DivSIOp>(currentLoop.getLoc(), range, step); range = builder.create<arith::DivSIOp>(currentLoop.getLoc(), range, step);
dims.push_back(range); dims.push_back(range);
Show All 39 Linesvoid AffineLoopToGpuConverter::createLaunch(AffineForOp rootForOp,
// Replace the loop terminator (loops contain only a single block) with the // Replace the loop terminator (loops contain only a single block) with the
// gpu terminator and move the operations from the loop body block to the gpu // gpu terminator and move the operations from the loop body block to the gpu
// launch body block. Do not move the entire block because of the difference // launch body block. Do not move the entire block because of the difference
// in block arguments. // in block arguments.
Operation &terminator = innermostForOp.getBody()->back(); Operation &terminator = innermostForOp.getBody()->back();
Location terminatorLoc = terminator.getLoc(); Location terminatorLoc = terminator.getLoc();
terminator.erase(); terminator.erase();
builder.setInsertionPointToEnd(innermostForOp.getBody()); builder.setInsertionPointToEnd(innermostForOp.getBody());
builder.create<gpu::TerminatorOp>(terminatorLoc, llvm::None); builder.create<gpu::TerminatorOp>(terminatorLoc, std::nullopt);
launchOp.getBody().front().getOperations().splice( launchOp.getBody().front().getOperations().splice(
launchOp.getBody().front().begin(), launchOp.getBody().front().begin(),
innermostForOp.getBody()->getOperations()); innermostForOp.getBody()->getOperations());
// Remap the loop iterators to use block/thread identifiers instead. Loops // Remap the loop iterators to use block/thread identifiers instead. Loops
// may iterate from LB with step S whereas GPU thread/block ids always iterate // may iterate from LB with step S whereas GPU thread/block ids always iterate
// from 0 to N with step 1. Therefore, loop induction variables are replaced // from 0 to N with step 1. Therefore, loop induction variables are replaced
// with (gpu-thread/block-id * S) + LB. // with (gpu-thread/block-id * S) + LB.
▲ Show 20 Lines • Show All 435 Lines • Show Last 20 Lines

mlir/lib/Conversion/SCFToSPIRV/SCFToSPIRV.cpp

Show First 20 Lines • Show All 406 Lines • ▼ Show 20 Lines for (const auto &it : llvm::enumerate(condArgs)) {
// Store the current iteration's result value. // Store the current iteration's result value.
rewriter.setInsertionPointToEnd(&beforeBlock); rewriter.setInsertionPointToEnd(&beforeBlock);
rewriter.create<spirv::StoreOp>(condLoc, alloc, res); rewriter.create<spirv::StoreOp>(condLoc, alloc, res);
} }
rewriter.setInsertionPointToEnd(&beforeBlock); rewriter.setInsertionPointToEnd(&beforeBlock);
rewriter.replaceOpWithNewOp<spirv::BranchConditionalOp>( rewriter.replaceOpWithNewOp<spirv::BranchConditionalOp>(
cond, conditionVal, &afterBlock, condArgs, &mergeBlock, llvm::None); cond, conditionVal, &afterBlock, condArgs, &mergeBlock, std::nullopt);
// Convert the scf.yield op to a branch back to the header block. // Convert the scf.yield op to a branch back to the header block.
rewriter.setInsertionPointToEnd(&afterBlock); rewriter.setInsertionPointToEnd(&afterBlock);
rewriter.replaceOpWithNewOp<spirv::BranchOp>(yield, &beforeBlock, yieldArgs); rewriter.replaceOpWithNewOp<spirv::BranchOp>(yield, &beforeBlock, yieldArgs);
rewriter.replaceOp(whileOp, resultValues); rewriter.replaceOp(whileOp, resultValues);
return success(); return success();
} }
Show All 12 Lines

mlir/lib/Conversion/SPIRVToLLVM/SPIRVToLLVM.cpp

Show First 20 Lines • Show All 182 Lines • ▼ Show 20 Lines
} }
/// Converts SPIR-V struct with a regular (according to `VulkanLayoutUtils`) /// Converts SPIR-V struct with a regular (according to `VulkanLayoutUtils`)
/// offset to LLVM struct. Otherwise, the conversion is not supported. /// offset to LLVM struct. Otherwise, the conversion is not supported.
static Optional<Type> static Optional<Type>
convertStructTypeWithOffset(spirv::StructType type, convertStructTypeWithOffset(spirv::StructType type,
LLVMTypeConverter &converter) { LLVMTypeConverter &converter) {
if (type != VulkanLayoutUtils::decorateType(type)) if (type != VulkanLayoutUtils::decorateType(type))
return llvm::None; return std::nullopt;
auto elementsVector = llvm::to_vector<8>( auto elementsVector = llvm::to_vector<8>(
llvm::map_range(type.getElementTypes(), [&](Type elementType) { llvm::map_range(type.getElementTypes(), [&](Type elementType) {
return converter.convertType(elementType); return converter.convertType(elementType);
})); }));
return LLVM::LLVMStructType::getLiteral(type.getContext(), elementsVector, return LLVM::LLVMStructType::getLiteral(type.getContext(), elementsVector,
/*isPacked=*/false); /*isPacked=*/false);
} }
▲ Show 20 Lines • Show All 48 Lines • ▼ Show 20 Lines
/// `VulkanLayoutUtils`) is also mapped to LLVM array. This has to be respected /// `VulkanLayoutUtils`) is also mapped to LLVM array. This has to be respected
/// when converting ops that manipulate array types. /// when converting ops that manipulate array types.
static Optional<Type> convertArrayType(spirv::ArrayType type, static Optional<Type> convertArrayType(spirv::ArrayType type,
TypeConverter &converter) { TypeConverter &converter) {
unsigned stride = type.getArrayStride(); unsigned stride = type.getArrayStride();
Type elementType = type.getElementType(); Type elementType = type.getElementType();
auto sizeInBytes = elementType.cast<spirv::SPIRVType>().getSizeInBytes(); auto sizeInBytes = elementType.cast<spirv::SPIRVType>().getSizeInBytes();
if (stride != 0 && (!sizeInBytes || *sizeInBytes != stride)) if (stride != 0 && (!sizeInBytes || *sizeInBytes != stride))
return llvm::None; return std::nullopt;
auto llvmElementType = converter.convertType(elementType); auto llvmElementType = converter.convertType(elementType);
unsigned numElements = type.getNumElements(); unsigned numElements = type.getNumElements();
return LLVM::LLVMArrayType::get(llvmElementType, numElements); return LLVM::LLVMArrayType::get(llvmElementType, numElements);
} }
/// Converts SPIR-V pointer type to LLVM pointer. Pointer's storage class is not /// Converts SPIR-V pointer type to LLVM pointer. Pointer's storage class is not
/// modelled at the moment. /// modelled at the moment.
static Type convertPointerType(spirv::PointerType type, static Type convertPointerType(spirv::PointerType type,
TypeConverter &converter) { TypeConverter &converter) {
auto pointeeType = converter.convertType(type.getPointeeType()); auto pointeeType = converter.convertType(type.getPointeeType());
return LLVM::LLVMPointerType::get(pointeeType); return LLVM::LLVMPointerType::get(pointeeType);
} }
/// Converts SPIR-V runtime array to LLVM array. Since LLVM allows indexing over /// Converts SPIR-V runtime array to LLVM array. Since LLVM allows indexing over
/// the bounds, the runtime array is converted to a 0-sized LLVM array. There is /// the bounds, the runtime array is converted to a 0-sized LLVM array. There is
/// no modelling of array stride at the moment. /// no modelling of array stride at the moment.
static Optional<Type> convertRuntimeArrayType(spirv::RuntimeArrayType type, static Optional<Type> convertRuntimeArrayType(spirv::RuntimeArrayType type,
TypeConverter &converter) { TypeConverter &converter) {
if (type.getArrayStride() != 0) if (type.getArrayStride() != 0)
return llvm::None; return std::nullopt;
auto elementType = converter.convertType(type.getElementType()); auto elementType = converter.convertType(type.getElementType());
return LLVM::LLVMArrayType::get(elementType, 0); return LLVM::LLVMArrayType::get(elementType, 0);
} }
/// Converts SPIR-V struct to LLVM struct. There is no support of structs with /// Converts SPIR-V struct to LLVM struct. There is no support of structs with
/// member decorations. Also, only natural offset is supported. /// member decorations. Also, only natural offset is supported.
static Optional<Type> convertStructType(spirv::StructType type, static Optional<Type> convertStructType(spirv::StructType type,
LLVMTypeConverter &converter) { LLVMTypeConverter &converter) {
SmallVector<spirv::StructType::MemberDecorationInfo, 4> memberDecorations; SmallVector<spirv::StructType::MemberDecorationInfo, 4> memberDecorations;
type.getMemberDecorations(memberDecorations); type.getMemberDecorations(memberDecorations);
if (!memberDecorations.empty()) if (!memberDecorations.empty())
return llvm::None; return std::nullopt;
if (type.hasOffset()) if (type.hasOffset())
return convertStructTypeWithOffset(type, converter); return convertStructTypeWithOffset(type, converter);
return convertStructTypePacked(type, converter); return convertStructTypePacked(type, converter);
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Operation conversion // Operation conversion
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 509 Lines • ▼ Show 20 Lines
public: public:
using SPIRVToLLVMConversion<spirv::FunctionCallOp>::SPIRVToLLVMConversion; using SPIRVToLLVMConversion<spirv::FunctionCallOp>::SPIRVToLLVMConversion;
LogicalResult LogicalResult
matchAndRewrite(spirv::FunctionCallOp callOp, OpAdaptor adaptor, matchAndRewrite(spirv::FunctionCallOp callOp, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override { ConversionPatternRewriter &rewriter) const override {
if (callOp.getNumResults() == 0) { if (callOp.getNumResults() == 0) {
rewriter.replaceOpWithNewOp<LLVM::CallOp>( rewriter.replaceOpWithNewOp<LLVM::CallOp>(
callOp, llvm::None, adaptor.getOperands(), callOp->getAttrs()); callOp, std::nullopt, adaptor.getOperands(), callOp->getAttrs());
return success(); return success();
} }
// Function returns a single result. // Function returns a single result.
auto dstType = typeConverter.convertType(callOp.getType(0)); auto dstType = typeConverter.convertType(callOp.getType(0));
rewriter.replaceOpWithNewOp<LLVM::CallOp>( rewriter.replaceOpWithNewOp<LLVM::CallOp>(
callOp, dstType, adaptor.getOperands(), callOp->getAttrs()); callOp, dstType, adaptor.getOperands(), callOp->getAttrs());
return success(); return success();
▲ Show 20 Lines • Show All 787 Lines • Show Last 20 Lines

mlir/lib/Conversion/VectorToGPU/VectorToGPU.cpp

Show First 20 Lines • Show All 113 Lines • ▼ Show 20 Lines if (!memrefType)
return false; return false;
// If the memref is 0 or 1D the horizontal stride is 0. // If the memref is 0 or 1D the horizontal stride is 0.
if (memrefType.getRank() < 2) if (memrefType.getRank() < 2)
return 0; return 0;
int64_t offset = 0; int64_t offset = 0;
SmallVector<int64_t, 2> strides; SmallVector<int64_t, 2> strides;
if (failed(getStridesAndOffset(memrefType, strides, offset)) || if (failed(getStridesAndOffset(memrefType, strides, offset)) ||
strides.back() != 1) strides.back() != 1)
return llvm::None; return std::nullopt;
int64_t stride = strides[strides.size() - 2]; int64_t stride = strides[strides.size() - 2];
if (stride == ShapedType::kDynamic) if (stride == ShapedType::kDynamic)
return llvm::None; return std::nullopt;
return stride; return stride;
} }
// Return true if the transfer op can be converted to a MMA matrix load. // Return true if the transfer op can be converted to a MMA matrix load.
static bool transferReadSupportsMMAMatrixType(vector::TransferReadOp readOp, static bool transferReadSupportsMMAMatrixType(vector::TransferReadOp readOp,
bool useNvGpu) { bool useNvGpu) {
if (readOp.getMask() || readOp.hasOutOfBoundsDim() || if (readOp.getMask() || readOp.hasOutOfBoundsDim() ||
readOp.getVectorType().getRank() != 2) readOp.getVectorType().getRank() != 2)
▲ Show 20 Lines • Show All 45 Lines • ▼ Show 20 Lines
/// Return true if this is a broadcast from scalar to a 2D vector. /// Return true if this is a broadcast from scalar to a 2D vector.
static bool broadcastSupportsMMAMatrixType(vector::BroadcastOp broadcastOp) { static bool broadcastSupportsMMAMatrixType(vector::BroadcastOp broadcastOp) {
return broadcastOp.getVectorType().getRank() == 2 && return broadcastOp.getVectorType().getRank() == 2 &&
broadcastOp.getSource().getType().isa<FloatType>(); broadcastOp.getSource().getType().isa<FloatType>();
} }
/// Return the MMA elementwise enum associated with `op` if it is supported. /// Return the MMA elementwise enum associated with `op` if it is supported.
/// Return `llvm::None` otherwise. /// Return `std::nullopt` otherwise.
static std::optional<gpu::MMAElementwiseOp> static std::optional<gpu::MMAElementwiseOp>
convertElementwiseOpToMMA(Operation *op) { convertElementwiseOpToMMA(Operation *op) {
if (isa<arith::AddFOp>(op)) if (isa<arith::AddFOp>(op))
return gpu::MMAElementwiseOp::ADDF; return gpu::MMAElementwiseOp::ADDF;
if (isa<arith::MulFOp>(op)) if (isa<arith::MulFOp>(op))
return gpu::MMAElementwiseOp::MULF; return gpu::MMAElementwiseOp::MULF;
if (isa<arith::MaxFOp>(op)) if (isa<arith::MaxFOp>(op))
return gpu::MMAElementwiseOp::MAXF; return gpu::MMAElementwiseOp::MAXF;
if (isa<arith::MinFOp>(op)) if (isa<arith::MinFOp>(op))
return gpu::MMAElementwiseOp::MINF; return gpu::MMAElementwiseOp::MINF;
if (isa<arith::DivFOp>(op)) if (isa<arith::DivFOp>(op))
return gpu::MMAElementwiseOp::DIVF; return gpu::MMAElementwiseOp::DIVF;
return llvm::None; return std::nullopt;
} }
/// Return true if the op is supported as elementwise op on MMAMatrix type. /// Return true if the op is supported as elementwise op on MMAMatrix type.
static bool elementwiseSupportsMMAMatrixType(Operation *op) { static bool elementwiseSupportsMMAMatrixType(Operation *op) {
return convertElementwiseOpToMMA(op).has_value(); return convertElementwiseOpToMMA(op).has_value();
} }
/// Returns true if the extract strided slice op is supported with `mma.sync` /// Returns true if the extract strided slice op is supported with `mma.sync`
▲ Show 20 Lines • Show All 835 Lines • Show Last 20 Lines

mlir/lib/Dialect/Affine/Analysis/AffineAnalysis.cpp

Show First 20 Lines • Show All 61 Lines • ▼ Show 20 Lines Optional<arith::AtomicRMWKind> maybeKind =
.Case([](arith::MaxFOp) { return arith::AtomicRMWKind::maxf; }) .Case([](arith::MaxFOp) { return arith::AtomicRMWKind::maxf; })
.Case([](arith::MinSIOp) { return arith::AtomicRMWKind::mins; }) .Case([](arith::MinSIOp) { return arith::AtomicRMWKind::mins; })
.Case([](arith::MaxSIOp) { return arith::AtomicRMWKind::maxs; }) .Case([](arith::MaxSIOp) { return arith::AtomicRMWKind::maxs; })
.Case([](arith::MinUIOp) { return arith::AtomicRMWKind::minu; }) .Case([](arith::MinUIOp) { return arith::AtomicRMWKind::minu; })
.Case([](arith::MaxUIOp) { return arith::AtomicRMWKind::maxu; }) .Case([](arith::MaxUIOp) { return arith::AtomicRMWKind::maxu; })
.Default([](Operation *) -> Optional<arith::AtomicRMWKind> { .Default([](Operation *) -> Optional<arith::AtomicRMWKind> {
// TODO: AtomicRMW supports other kinds of reductions this is // TODO: AtomicRMW supports other kinds of reductions this is
// currently not detecting, add those when the need arises. // currently not detecting, add those when the need arises.
return llvm::None; return std::nullopt;
}); });
if (!maybeKind) if (!maybeKind)
return nullptr; return nullptr;
kind = *maybeKind; kind = *maybeKind;
return reducedVal; return reducedVal;
} }
▲ Show 20 Lines • Show All 609 Lines • Show Last 20 Lines

mlir/lib/Dialect/Affine/Analysis/Utils.cpp

Show First 20 Lines • Show All 140 Lines • ▼ Show 20 Lines for (Value ubOp : ubOperands[en.index()])
llvm::errs() << "\t\t\t" << ubOp << "\n"; llvm::errs() << "\t\t\t" << ubOp << "\n";
} }
} }
/// Fast check to determine if the computation slice is maximal. Returns true if /// Fast check to determine if the computation slice is maximal. Returns true if
/// each slice dimension maps to an existing dst dimension and both the src /// each slice dimension maps to an existing dst dimension and both the src
/// and the dst loops for those dimensions have the same bounds. Returns false /// and the dst loops for those dimensions have the same bounds. Returns false
/// if both the src and the dst loops don't have the same bounds. Returns /// if both the src and the dst loops don't have the same bounds. Returns
/// llvm::None if none of the above can be proven. /// std::nullopt if none of the above can be proven.
Optional<bool> ComputationSliceState::isSliceMaximalFastCheck() const { Optional<bool> ComputationSliceState::isSliceMaximalFastCheck() const {
assert(lbs.size() == ubs.size() && !lbs.empty() && !ivs.empty() && assert(lbs.size() == ubs.size() && !lbs.empty() && !ivs.empty() &&
"Unexpected number of lbs, ubs and ivs in slice"); "Unexpected number of lbs, ubs and ivs in slice");
for (unsigned i = 0, end = lbs.size(); i < end; ++i) { for (unsigned i = 0, end = lbs.size(); i < end; ++i) {
AffineMap lbMap = lbs[i]; AffineMap lbMap = lbs[i];
AffineMap ubMap = ubs[i]; AffineMap ubMap = ubs[i];
// Check if this slice is just an equality along this dimension. // Check if this slice is just an equality along this dimension.
if (!lbMap || !ubMap || lbMap.getNumResults() != 1 || if (!lbMap || !ubMap || lbMap.getNumResults() != 1 ||
ubMap.getNumResults() != 1 || ubMap.getNumResults() != 1 ||
lbMap.getResult(0) + 1 != ubMap.getResult(0) || lbMap.getResult(0) + 1 != ubMap.getResult(0) ||
// The condition above will be true for maps describing a single // The condition above will be true for maps describing a single
// iteration (e.g., lbMap.getResult(0) = 0, ubMap.getResult(0) = 1). // iteration (e.g., lbMap.getResult(0) = 0, ubMap.getResult(0) = 1).
// Make sure we skip those cases by checking that the lb result is not // Make sure we skip those cases by checking that the lb result is not
// just a constant. // just a constant.
lbMap.getResult(0).isa<AffineConstantExpr>()) lbMap.getResult(0).isa<AffineConstantExpr>())
return llvm::None; return std::nullopt;
// Limited support: we expect the lb result to be just a loop dimension for // Limited support: we expect the lb result to be just a loop dimension for
// now. // now.
AffineDimExpr result = lbMap.getResult(0).dyn_cast<AffineDimExpr>(); AffineDimExpr result = lbMap.getResult(0).dyn_cast<AffineDimExpr>();
if (!result) if (!result)
return llvm::None; return std::nullopt;
// Retrieve dst loop bounds. // Retrieve dst loop bounds.
AffineForOp dstLoop = AffineForOp dstLoop =
getForInductionVarOwner(lbOperands[i][result.getPosition()]); getForInductionVarOwner(lbOperands[i][result.getPosition()]);
if (!dstLoop) if (!dstLoop)
return llvm::None; return std::nullopt;
AffineMap dstLbMap = dstLoop.getLowerBoundMap(); AffineMap dstLbMap = dstLoop.getLowerBoundMap();
AffineMap dstUbMap = dstLoop.getUpperBoundMap(); AffineMap dstUbMap = dstLoop.getUpperBoundMap();
// Retrieve src loop bounds. // Retrieve src loop bounds.
AffineForOp srcLoop = getForInductionVarOwner(ivs[i]); AffineForOp srcLoop = getForInductionVarOwner(ivs[i]);
assert(srcLoop && "Expected affine for"); assert(srcLoop && "Expected affine for");
AffineMap srcLbMap = srcLoop.getLowerBoundMap(); AffineMap srcLbMap = srcLoop.getLowerBoundMap();
AffineMap srcUbMap = srcLoop.getUpperBoundMap(); AffineMap srcUbMap = srcLoop.getUpperBoundMap();
// Limited support: we expect simple src and dst loops with a single // Limited support: we expect simple src and dst loops with a single
// constant component per bound for now. // constant component per bound for now.
if (srcLbMap.getNumResults() != 1 || srcUbMap.getNumResults() != 1 || if (srcLbMap.getNumResults() != 1 || srcUbMap.getNumResults() != 1 ||
dstLbMap.getNumResults() != 1 || dstUbMap.getNumResults() != 1) dstLbMap.getNumResults() != 1 || dstUbMap.getNumResults() != 1)
return llvm::None; return std::nullopt;
AffineExpr srcLbResult = srcLbMap.getResult(0); AffineExpr srcLbResult = srcLbMap.getResult(0);
AffineExpr dstLbResult = dstLbMap.getResult(0); AffineExpr dstLbResult = dstLbMap.getResult(0);
AffineExpr srcUbResult = srcUbMap.getResult(0); AffineExpr srcUbResult = srcUbMap.getResult(0);
AffineExpr dstUbResult = dstUbMap.getResult(0); AffineExpr dstUbResult = dstUbMap.getResult(0);
if (!srcLbResult.isa<AffineConstantExpr>() || if (!srcLbResult.isa<AffineConstantExpr>() ||
!srcUbResult.isa<AffineConstantExpr>() || !srcUbResult.isa<AffineConstantExpr>() ||
!dstLbResult.isa<AffineConstantExpr>() || !dstLbResult.isa<AffineConstantExpr>() ||
!dstUbResult.isa<AffineConstantExpr>()) !dstUbResult.isa<AffineConstantExpr>())
return llvm::None; return std::nullopt;
// Check if src and dst loop bounds are the same. If not, we can guarantee // Check if src and dst loop bounds are the same. If not, we can guarantee
// that the slice is not maximal. // that the slice is not maximal.
if (srcLbResult != dstLbResult || srcUbResult != dstUbResult || if (srcLbResult != dstLbResult || srcUbResult != dstUbResult ||
srcLoop.getStep() != dstLoop.getStep()) srcLoop.getStep() != dstLoop.getStep())
return false; return false;
} }
Show All 18 LinesOptional<bool> ComputationSliceState::isSliceValid() {
if (isValidFastCheck && *isValidFastCheck) if (isValidFastCheck && *isValidFastCheck)
return true; return true;
// Create constraints for the source loop nest using which slice is computed. // Create constraints for the source loop nest using which slice is computed.
FlatAffineValueConstraints srcConstraints; FlatAffineValueConstraints srcConstraints;
// TODO: Store the source's domain to avoid computation at each depth. // TODO: Store the source's domain to avoid computation at each depth.
if (failed(getSourceAsConstraints(srcConstraints))) { if (failed(getSourceAsConstraints(srcConstraints))) {
LLVM_DEBUG(llvm::dbgs() << "Unable to compute source's domain\n"); LLVM_DEBUG(llvm::dbgs() << "Unable to compute source's domain\n");
return llvm::None; return std::nullopt;
} }
// As the set difference utility currently cannot handle symbols in its // As the set difference utility currently cannot handle symbols in its
// operands, validity of the slice cannot be determined. // operands, validity of the slice cannot be determined.
if (srcConstraints.getNumSymbolVars() > 0) { if (srcConstraints.getNumSymbolVars() > 0) {
LLVM_DEBUG(llvm::dbgs() << "Cannot handle symbols in source domain\n"); LLVM_DEBUG(llvm::dbgs() << "Cannot handle symbols in source domain\n");
return llvm::None; return std::nullopt;
} }
// TODO: Handle local vars in the source domains while using the 'projectOut' // TODO: Handle local vars in the source domains while using the 'projectOut'
// utility below. Currently, aligning is not done assuming that there will be // utility below. Currently, aligning is not done assuming that there will be
// no local vars in the source domain. // no local vars in the source domain.
if (srcConstraints.getNumLocalVars() != 0) { if (srcConstraints.getNumLocalVars() != 0) {
LLVM_DEBUG(llvm::dbgs() << "Cannot handle locals in source domain\n"); LLVM_DEBUG(llvm::dbgs() << "Cannot handle locals in source domain\n");
return llvm::None; return std::nullopt;
} }
// Create constraints for the slice loop nest that would be created if the // Create constraints for the slice loop nest that would be created if the
// fusion succeeds. // fusion succeeds.
FlatAffineValueConstraints sliceConstraints; FlatAffineValueConstraints sliceConstraints;
if (failed(getAsConstraints(&sliceConstraints))) { if (failed(getAsConstraints(&sliceConstraints))) {
LLVM_DEBUG(llvm::dbgs() << "Unable to compute slice's domain\n"); LLVM_DEBUG(llvm::dbgs() << "Unable to compute slice's domain\n");
return llvm::None; return std::nullopt;
} }
// Projecting out every dimension other than the 'ivs' to express slice's // Projecting out every dimension other than the 'ivs' to express slice's
// domain completely in terms of source's IVs. // domain completely in terms of source's IVs.
sliceConstraints.projectOut(ivs.size(), sliceConstraints.projectOut(ivs.size(),
sliceConstraints.getNumVars() - ivs.size()); sliceConstraints.getNumVars() - ivs.size());
LLVM_DEBUG(llvm::dbgs() << "Domain of the source of the slice:\n"); LLVM_DEBUG(llvm::dbgs() << "Domain of the source of the slice:\n");
Show All 10 LinesOptional<bool> ComputationSliceState::isSliceValid() {
if (!diffSet.isIntegerEmpty()) { if (!diffSet.isIntegerEmpty()) {
LLVM_DEBUG(llvm::dbgs() << "Incorrect slice\n"); LLVM_DEBUG(llvm::dbgs() << "Incorrect slice\n");
return false; return false;
} }
return true; return true;
} }
/// Returns true if the computation slice encloses all the iterations of the /// Returns true if the computation slice encloses all the iterations of the
/// sliced loop nest. Returns false if it does not. Returns llvm::None if it /// sliced loop nest. Returns false if it does not. Returns std::nullopt if it
/// cannot determine if the slice is maximal or not. /// cannot determine if the slice is maximal or not.
Optional<bool> ComputationSliceState::isMaximal() const { Optional<bool> ComputationSliceState::isMaximal() const {
// Fast check to determine if the computation slice is maximal. If the result // Fast check to determine if the computation slice is maximal. If the result
// is inconclusive, we proceed with a more expensive analysis. // is inconclusive, we proceed with a more expensive analysis.
Optional<bool> isMaximalFastCheck = isSliceMaximalFastCheck(); Optional<bool> isMaximalFastCheck = isSliceMaximalFastCheck();
if (isMaximalFastCheck) if (isMaximalFastCheck)
return isMaximalFastCheck; return isMaximalFastCheck;
// Create constraints for the src loop nest being sliced. // Create constraints for the src loop nest being sliced.
FlatAffineValueConstraints srcConstraints; FlatAffineValueConstraints srcConstraints;
srcConstraints.reset(/*numDims=*/ivs.size(), /*numSymbols=*/0, srcConstraints.reset(/*numDims=*/ivs.size(), /*numSymbols=*/0,
/*numLocals=*/0, ivs); /*numLocals=*/0, ivs);
for (Value iv : ivs) { for (Value iv : ivs) {
AffineForOp loop = getForInductionVarOwner(iv); AffineForOp loop = getForInductionVarOwner(iv);
assert(loop && "Expected affine for"); assert(loop && "Expected affine for");
if (failed(srcConstraints.addAffineForOpDomain(loop))) if (failed(srcConstraints.addAffineForOpDomain(loop)))
return llvm::None; return std::nullopt;
} }
// Create constraints for the slice using the dst loop nest information. We // Create constraints for the slice using the dst loop nest information. We
// retrieve existing dst loops from the lbOperands. // retrieve existing dst loops from the lbOperands.
SmallVector<Value, 8> consumerIVs; SmallVector<Value, 8> consumerIVs;
for (Value lbOp : lbOperands[0]) for (Value lbOp : lbOperands[0])
if (getForInductionVarOwner(lbOp)) if (getForInductionVarOwner(lbOp))
consumerIVs.push_back(lbOp); consumerIVs.push_back(lbOp);
// Add empty IV Values for those new loops that are not equalities and, // Add empty IV Values for those new loops that are not equalities and,
// therefore, are not yet materialized in the IR. // therefore, are not yet materialized in the IR.
for (int i = consumerIVs.size(), end = ivs.size(); i < end; ++i) for (int i = consumerIVs.size(), end = ivs.size(); i < end; ++i)
consumerIVs.push_back(Value()); consumerIVs.push_back(Value());
FlatAffineValueConstraints sliceConstraints; FlatAffineValueConstraints sliceConstraints;
sliceConstraints.reset(/*numDims=*/consumerIVs.size(), /*numSymbols=*/0, sliceConstraints.reset(/*numDims=*/consumerIVs.size(), /*numSymbols=*/0,
/*numLocals=*/0, consumerIVs); /*numLocals=*/0, consumerIVs);
if (failed(sliceConstraints.addDomainFromSliceMaps(lbs, ubs, lbOperands[0]))) if (failed(sliceConstraints.addDomainFromSliceMaps(lbs, ubs, lbOperands[0])))
return llvm::None; return std::nullopt;
if (srcConstraints.getNumDimVars() != sliceConstraints.getNumDimVars()) if (srcConstraints.getNumDimVars() != sliceConstraints.getNumDimVars())
// Constraint dims are different. The integer set difference can't be // Constraint dims are different. The integer set difference can't be
// computed so we don't know if the slice is maximal. // computed so we don't know if the slice is maximal.
return llvm::None; return std::nullopt;
// Compute the difference between the src loop nest and the slice integer // Compute the difference between the src loop nest and the slice integer
// sets. // sets.
PresburgerSet srcSet(srcConstraints); PresburgerSet srcSet(srcConstraints);
PresburgerSet sliceSet(sliceConstraints); PresburgerSet sliceSet(sliceConstraints);
PresburgerSet diffSet = srcSet.subtract(sliceSet); PresburgerSet diffSet = srcSet.subtract(sliceSet);
return diffSet.isIntegerEmpty(); return diffSet.isIntegerEmpty();
} }
▲ Show 20 Lines • Show All 1,028 Lines • Show Last 20 Lines

mlir/lib/Dialect/Affine/IR/AffineOps.cpp

Show First 20 Lines • Show All 2,257 Lines • ▼ Show 20 Lines
Region &AffineForOp::getLoopBody() { return getRegion(); } Region &AffineForOp::getLoopBody() { return getRegion(); }
Optional<Value> AffineForOp::getSingleInductionVar() { Optional<Value> AffineForOp::getSingleInductionVar() {
return getInductionVar(); return getInductionVar();
} }
Optional<OpFoldResult> AffineForOp::getSingleLowerBound() { Optional<OpFoldResult> AffineForOp::getSingleLowerBound() {
if (!hasConstantLowerBound()) if (!hasConstantLowerBound())
return llvm::None; return std::nullopt;
OpBuilder b(getContext()); OpBuilder b(getContext());
return OpFoldResult(b.getI64IntegerAttr(getConstantLowerBound())); return OpFoldResult(b.getI64IntegerAttr(getConstantLowerBound()));
} }
Optional<OpFoldResult> AffineForOp::getSingleStep() { Optional<OpFoldResult> AffineForOp::getSingleStep() {
OpBuilder b(getContext()); OpBuilder b(getContext());
return OpFoldResult(b.getI64IntegerAttr(getStep())); return OpFoldResult(b.getI64IntegerAttr(getStep()));
} }
Optional<OpFoldResult> AffineForOp::getSingleUpperBound() { Optional<OpFoldResult> AffineForOp::getSingleUpperBound() {
if (!hasConstantUpperBound()) if (!hasConstantUpperBound())
return llvm::None; return std::nullopt;
OpBuilder b(getContext()); OpBuilder b(getContext());
return OpFoldResult(b.getI64IntegerAttr(getConstantUpperBound())); return OpFoldResult(b.getI64IntegerAttr(getConstantUpperBound()));
} }
Speculation::Speculatability AffineForOp::getSpeculatability() { Speculation::Speculatability AffineForOp::getSpeculatability() {
// `affine.for (I = Start; I < End; I += 1)` terminates for all values of // `affine.for (I = Start; I < End; I += 1)` terminates for all values of
// Start and End. // Start and End.
// //
▲ Show 20 Lines • Show All 73 Lines • ▼ Show 20 Linesstatic void buildAffineLoopNestImpl(
} }
} }
/// Creates an affine loop from the bounds known to be constants. /// Creates an affine loop from the bounds known to be constants.
static AffineForOp static AffineForOp
buildAffineLoopFromConstants(OpBuilder &builder, Location loc, int64_t lb, buildAffineLoopFromConstants(OpBuilder &builder, Location loc, int64_t lb,
int64_t ub, int64_t step, int64_t ub, int64_t step,
AffineForOp::BodyBuilderFn bodyBuilderFn) { AffineForOp::BodyBuilderFn bodyBuilderFn) {
return builder.create<AffineForOp>(loc, lb, ub, step, /*iterArgs=*/llvm::None, return builder.create<AffineForOp>(loc, lb, ub, step,
bodyBuilderFn); /*iterArgs=*/std::nullopt, bodyBuilderFn);
} }
/// Creates an affine loop from the bounds that may or may not be constants. /// Creates an affine loop from the bounds that may or may not be constants.
static AffineForOp static AffineForOp
buildAffineLoopFromValues(OpBuilder &builder, Location loc, Value lb, Value ub, buildAffineLoopFromValues(OpBuilder &builder, Location loc, Value lb, Value ub,
int64_t step, int64_t step,
AffineForOp::BodyBuilderFn bodyBuilderFn) { AffineForOp::BodyBuilderFn bodyBuilderFn) {
auto lbConst = lb.getDefiningOp<arith::ConstantIndexOp>(); auto lbConst = lb.getDefiningOp<arith::ConstantIndexOp>();
auto ubConst = ub.getDefiningOp<arith::ConstantIndexOp>(); auto ubConst = ub.getDefiningOp<arith::ConstantIndexOp>();
if (lbConst && ubConst) if (lbConst && ubConst)
return buildAffineLoopFromConstants(builder, loc, lbConst.value(), return buildAffineLoopFromConstants(builder, loc, lbConst.value(),
ubConst.value(), step, bodyBuilderFn); ubConst.value(), step, bodyBuilderFn);
return builder.create<AffineForOp>(loc, lb, builder.getDimIdentityMap(), ub, return builder.create<AffineForOp>(loc, lb, builder.getDimIdentityMap(), ub,
builder.getDimIdentityMap(), step, builder.getDimIdentityMap(), step,
/*iterArgs=*/llvm::None, bodyBuilderFn); /*iterArgs=*/std::nullopt, bodyBuilderFn);
} }
void mlir::buildAffineLoopNest( void mlir::buildAffineLoopNest(
OpBuilder &builder, Location loc, ArrayRef<int64_t> lbs, OpBuilder &builder, Location loc, ArrayRef<int64_t> lbs,
ArrayRef<int64_t> ubs, ArrayRef<int64_t> steps, ArrayRef<int64_t> ubs, ArrayRef<int64_t> steps,
function_ref<void(OpBuilder &, Location, ValueRange)> bodyBuilderFn) { function_ref<void(OpBuilder &, Location, ValueRange)> bodyBuilderFn) {
buildAffineLoopNestImpl(builder, loc, lbs, ubs, steps, bodyBuilderFn, buildAffineLoopNestImpl(builder, loc, lbs, ubs, steps, bodyBuilderFn,
buildAffineLoopFromConstants); buildAffineLoopFromConstants);
▲ Show 20 Lines • Show All 1,153 Lines • ▼ Show 20 Lines
} }
AffineValueMap AffineParallelOp::getUpperBoundsValueMap() { AffineValueMap AffineParallelOp::getUpperBoundsValueMap() {
return AffineValueMap(getUpperBoundsMap(), getUpperBoundsOperands()); return AffineValueMap(getUpperBoundsMap(), getUpperBoundsOperands());
} }
Optional<SmallVector<int64_t, 8>> AffineParallelOp::getConstantRanges() { Optional<SmallVector<int64_t, 8>> AffineParallelOp::getConstantRanges() {
if (hasMinMaxBounds()) if (hasMinMaxBounds())
return llvm::None; return std::nullopt;
// Try to convert all the ranges to constant expressions. // Try to convert all the ranges to constant expressions.
SmallVector<int64_t, 8> out; SmallVector<int64_t, 8> out;
AffineValueMap rangesValueMap; AffineValueMap rangesValueMap;
AffineValueMap::difference(getUpperBoundsValueMap(), getLowerBoundsValueMap(), AffineValueMap::difference(getUpperBoundsValueMap(), getLowerBoundsValueMap(),
&rangesValueMap); &rangesValueMap);
out.reserve(rangesValueMap.getNumResults()); out.reserve(rangesValueMap.getNumResults());
for (unsigned i = 0, e = rangesValueMap.getNumResults(); i < e; ++i) { for (unsigned i = 0, e = rangesValueMap.getNumResults(); i < e; ++i) {
auto expr = rangesValueMap.getResult(i); auto expr = rangesValueMap.getResult(i);
auto cst = expr.dyn_cast<AffineConstantExpr>(); auto cst = expr.dyn_cast<AffineConstantExpr>();
if (!cst) if (!cst)
return llvm::None; return std::nullopt;
out.push_back(cst.getValue()); out.push_back(cst.getValue());
} }
return out; return out;
} }
Block *AffineParallelOp::getBody() { return &getRegion().front(); } Block *AffineParallelOp::getBody() { return &getRegion().front(); }
OpBuilder AffineParallelOp::getBodyBuilder() { OpBuilder AffineParallelOp::getBodyBuilder() {
▲ Show 20 Lines • Show All 664 Lines • Show Last 20 Lines

mlir/lib/Dialect/Affine/Transforms/AffineDataCopyGeneration.cpp

Show First 20 Lines • Show All 135 Lines • ▼ Show 20 Linesvoid AffineDataCopyGeneration::runOnBlock(Block *block,
// Create [begin, end) ranges. // Create [begin, end) ranges.
auto it = curBegin; auto it = curBegin;
while (it != block->end()) { while (it != block->end()) {
AffineForOp forOp; AffineForOp forOp;
// If you hit a non-copy for loop, we will split there. // If you hit a non-copy for loop, we will split there.
if ((forOp = dyn_cast<AffineForOp>(&*it)) && copyNests.count(forOp) == 0) { if ((forOp = dyn_cast<AffineForOp>(&*it)) && copyNests.count(forOp) == 0) {
// Perform the copying up unti this 'for' op first. // Perform the copying up unti this 'for' op first.
(void)affineDataCopyGenerate(/*begin=*/curBegin, /*end=*/it, copyOptions, (void)affineDataCopyGenerate(/*begin=*/curBegin, /*end=*/it, copyOptions,
/*filterMemRef=*/llvm::None, copyNests); /*filterMemRef=*/std::nullopt, copyNests);
// Returns true if the footprint is known to exceed capacity. // Returns true if the footprint is known to exceed capacity.
auto exceedsCapacity = [&](AffineForOp forOp) { auto exceedsCapacity = [&](AffineForOp forOp) {
Optional<int64_t> footprint = Optional<int64_t> footprint =
getMemoryFootprintBytes(forOp, getMemoryFootprintBytes(forOp,
/*memorySpace=*/0); /*memorySpace=*/0);
return (footprint.has_value() && return (footprint.has_value() &&
static_cast<uint64_t>(footprint.value()) > static_cast<uint64_t>(footprint.value()) >
Show All 18 Lines if ((forOp = dyn_cast<AffineForOp>(&*it)) && copyNests.count(forOp) == 0) {
// that for the latter, the copies are placed just before this loop (for // that for the latter, the copies are placed just before this loop (for
// incoming copies) and right after (for outgoing ones). // incoming copies) and right after (for outgoing ones).
// Inner loop copies have their own scope - we don't thus update // Inner loop copies have their own scope - we don't thus update
// consumed capacity. The footprint check above guarantees this inner // consumed capacity. The footprint check above guarantees this inner
// loop's footprint fits. // loop's footprint fits.
(void)affineDataCopyGenerate(/*begin=*/it, /*end=*/std::next(it), (void)affineDataCopyGenerate(/*begin=*/it, /*end=*/std::next(it),
copyOptions, copyOptions,
/*filterMemRef=*/llvm::None, copyNests); /*filterMemRef=*/std::nullopt, copyNests);
} }
// Get to the next load or store op after 'forOp'. // Get to the next load or store op after 'forOp'.
curBegin = std::find_if(std::next(it), block->end(), [&](Operation &op) { curBegin = std::find_if(std::next(it), block->end(), [&](Operation &op) {
return isa<AffineLoadOp, AffineStoreOp, AffineForOp>(op) && return isa<AffineLoadOp, AffineStoreOp, AffineForOp>(op) &&
copyNests.count(&op) == 0; copyNests.count(&op) == 0;
}); });
it = curBegin; it = curBegin;
} else { } else {
assert(copyNests.count(&*it) == 0 && assert(copyNests.count(&*it) == 0 &&
"all copy nests generated should have been skipped above"); "all copy nests generated should have been skipped above");
// We simply include this op in the current range and continue for more. // We simply include this op in the current range and continue for more.
++it; ++it;
} }
} }
// Generate the copy for the final block range. // Generate the copy for the final block range.
if (curBegin != block->end()) { if (curBegin != block->end()) {
// Can't be a terminator because it would have been skipped above. // Can't be a terminator because it would have been skipped above.
assert(!curBegin->hasTrait<OpTrait::IsTerminator>() && assert(!curBegin->hasTrait<OpTrait::IsTerminator>() &&
"can't be a terminator"); "can't be a terminator");
// Exclude the affine.yield - hence, the std::prev. // Exclude the affine.yield - hence, the std::prev.
(void)affineDataCopyGenerate(/*begin=*/curBegin, (void)affineDataCopyGenerate(/*begin=*/curBegin,
/*end=*/std::prev(block->end()), copyOptions, /*end=*/std::prev(block->end()), copyOptions,
/*filterMemRef=*/llvm::None, copyNests); /*filterMemRef=*/std::nullopt, copyNests);
} }
} }
void AffineDataCopyGeneration::runOnOperation() { void AffineDataCopyGeneration::runOnOperation() {
func::FuncOp f = getOperation(); func::FuncOp f = getOperation();
OpBuilder topBuilder(f.getBody()); OpBuilder topBuilder(f.getBody());
zeroIndex = topBuilder.create<arith::ConstantIndexOp>(f.getLoc(), 0); zeroIndex = topBuilder.create<arith::ConstantIndexOp>(f.getLoc(), 0);
Show All 32 Lines

mlir/lib/Dialect/Affine/Transforms/SuperVectorize.cpp

Show First 20 Lines • Show All 593 Lines • ▼ Show 20 LinesmakePattern(const DenseSet<Operation *> &parallelLoops, int vectorRank,
case 2: case 2:
return For(isVectorizableLoopPtrFactory(parallelLoops, d0), return For(isVectorizableLoopPtrFactory(parallelLoops, d0),
For(isVectorizableLoopPtrFactory(parallelLoops, d1))); For(isVectorizableLoopPtrFactory(parallelLoops, d1)));
case 3: case 3:
return For(isVectorizableLoopPtrFactory(parallelLoops, d0), return For(isVectorizableLoopPtrFactory(parallelLoops, d0),
For(isVectorizableLoopPtrFactory(parallelLoops, d1), For(isVectorizableLoopPtrFactory(parallelLoops, d1),
For(isVectorizableLoopPtrFactory(parallelLoops, d2)))); For(isVectorizableLoopPtrFactory(parallelLoops, d2))));
default: { default: {
return llvm::None; return std::nullopt;
} }
} }
} }
static NestedPattern &vectorTransferPattern() { static NestedPattern &vectorTransferPattern() {
static auto pattern = matcher::Op([](Operation &op) { static auto pattern = matcher::Op([](Operation &op) {
return isa<vector::TransferReadOp, vector::TransferWriteOp>(op); return isa<vector::TransferReadOp, vector::TransferWriteOp>(op);
}); });
▲ Show 20 Lines • Show All 1,253 Lines • Show Last 20 Lines

mlir/lib/Dialect/Arith/IR/ArithOps.cpp

Show First 20 Lines • Show All 1,503 Lines • ▼ Show 20 Lines
static Optional<int64_t> getIntegerWidth(Type t) { static Optional<int64_t> getIntegerWidth(Type t) {
if (auto intType = t.dyn_cast<IntegerType>()) { if (auto intType = t.dyn_cast<IntegerType>()) {
return intType.getWidth(); return intType.getWidth();
} }
if (auto vectorIntType = t.dyn_cast<VectorType>()) { if (auto vectorIntType = t.dyn_cast<VectorType>()) {
return vectorIntType.getElementType().cast<IntegerType>().getWidth(); return vectorIntType.getElementType().cast<IntegerType>().getWidth();
} }
return llvm::None; return std::nullopt;
} }
OpFoldResult arith::CmpIOp::fold(ArrayRef<Attribute> operands) { OpFoldResult arith::CmpIOp::fold(ArrayRef<Attribute> operands) {
assert(operands.size() == 2 && "cmpi takes two operands"); assert(operands.size() == 2 && "cmpi takes two operands");
// cmpi(pred, x, x) // cmpi(pred, x, x)
if (getLhs() == getRhs()) { if (getLhs() == getRhs()) {
auto val = applyCmpPredicateToEqualOperands(getPredicate()); auto val = applyCmpPredicateToEqualOperands(getPredicate());
▲ Show 20 Lines • Show All 742 Lines • Show Last 20 Lines

mlir/lib/Dialect/Async/IR/Async.cpp

Show First 20 Lines • Show All 336 Lines • ▼ Show 20 Linesvoid FuncOp::build(OpBuilder &builder, OperationState &state, StringRef name,
state.attributes.append(attrs.begin(), attrs.end()); state.attributes.append(attrs.begin(), attrs.end());
state.addRegion(); state.addRegion();
if (argAttrs.empty()) if (argAttrs.empty())
return; return;
assert(type.getNumInputs() == argAttrs.size()); assert(type.getNumInputs() == argAttrs.size());
function_interface_impl::addArgAndResultAttrs(builder, state, argAttrs, function_interface_impl::addArgAndResultAttrs(builder, state, argAttrs,
/*resultAttrs=*/llvm::None); /*resultAttrs=*/std::nullopt);
} }
ParseResult FuncOp::parse(OpAsmParser &parser, OperationState &result) { ParseResult FuncOp::parse(OpAsmParser &parser, OperationState &result) {
auto buildFuncType = auto buildFuncType =
[](Builder &builder, ArrayRef<Type> argTypes, ArrayRef<Type> results, [](Builder &builder, ArrayRef<Type> argTypes, ArrayRef<Type> results,
function_interface_impl::VariadicFlag, function_interface_impl::VariadicFlag,
std::string &) { return builder.getFunctionType(argTypes, results); }; std::string &) { return builder.getFunctionType(argTypes, results); };
▲ Show 20 Lines • Show All 128 Lines • Show Last 20 Lines

mlir/lib/Dialect/Bufferization/Transforms/BufferDeallocation.cpp

Show First 20 Lines • Show All 365 Lines • ▼ Show 20 Lines if (failed(introduceClonesForRegionSuccessors(
return successorRegion.getSuccessor() == argRegion; return successorRegion.getSuccessor() == argRegion;
}))) })))
return failure(); return failure();
// Check whether the block argument belongs to an entry region of the // Check whether the block argument belongs to an entry region of the
// parent operation. In this case, we have to introduce an additional clone // parent operation. In this case, we have to introduce an additional clone
// for buffer that is passed to the argument. // for buffer that is passed to the argument.
SmallVector<RegionSuccessor, 2> successorRegions; SmallVector<RegionSuccessor, 2> successorRegions;
regionInterface.getSuccessorRegions(/*index=*/llvm::None, successorRegions); regionInterface.getSuccessorRegions(/*index=*/std::nullopt,
successorRegions);
auto *it = auto *it =
llvm::find_if(successorRegions, [&](RegionSuccessor &successorRegion) { llvm::find_if(successorRegions, [&](RegionSuccessor &successorRegion) {
return successorRegion.getSuccessor() == argRegion; return successorRegion.getSuccessor() == argRegion;
}); });
if (it == successorRegions.end()) if (it == successorRegions.end())
return success(); return success();
// Determine the actual operand to introduce a clone for and rewire the // Determine the actual operand to introduce a clone for and rewire the
▲ Show 20 Lines • Show All 331 Lines • Show Last 20 Lines

mlir/lib/Dialect/Bufferization/Transforms/BufferUtils.cpp

Show First 20 Lines • Show All 125 Lines • ▼ Show 20 Lines std::function<bool(Region *)> recurse = [&](Region *current) {
for (RegionSuccessor &regionEntry : successors) for (RegionSuccessor &regionEntry : successors)
if (recurse(regionEntry.getSuccessor())) if (recurse(regionEntry.getSuccessor()))
return true; return true;
return false; return false;
}; };
// Start with all entry regions and test whether they induce a loop. // Start with all entry regions and test whether they induce a loop.
SmallVector<RegionSuccessor, 2> successorRegions; SmallVector<RegionSuccessor, 2> successorRegions;
regionInterface.getSuccessorRegions(/*index=*/llvm::None, successorRegions); regionInterface.getSuccessorRegions(/*index=*/std::nullopt, successorRegions);
for (RegionSuccessor &regionEntry : successorRegions) { for (RegionSuccessor &regionEntry : successorRegions) {
if (recurse(regionEntry.getSuccessor())) if (recurse(regionEntry.getSuccessor()))
return true; return true;
visitedRegions.clear(); visitedRegions.clear();
} }
return false; return false;
} }
▲ Show 20 Lines • Show All 56 Lines • Show Last 20 Lines

mlir/lib/Dialect/Bufferization/Transforms/BufferViewFlowAnalysis.cpp

Show First 20 Lines • Show All 76 Lines • ▼ Show 20 Lines for (auto it = parentBlock->succ_begin(), e = parentBlock->succ_end();
successorOperands.getProducedOperandCount())); successorOperands.getProducedOperandCount()));
} }
}); });
// Query the RegionBranchOpInterface to find potential successor regions. // Query the RegionBranchOpInterface to find potential successor regions.
op->walk([&](RegionBranchOpInterface regionInterface) { op->walk([&](RegionBranchOpInterface regionInterface) {
// Extract all entry regions and wire all initial entry successor inputs. // Extract all entry regions and wire all initial entry successor inputs.
SmallVector<RegionSuccessor, 2> entrySuccessors; SmallVector<RegionSuccessor, 2> entrySuccessors;
regionInterface.getSuccessorRegions(/*index=*/llvm::None, entrySuccessors); regionInterface.getSuccessorRegions(/*index=*/std::nullopt,
entrySuccessors);
for (RegionSuccessor &entrySuccessor : entrySuccessors) { for (RegionSuccessor &entrySuccessor : entrySuccessors) {
// Wire the entry region's successor arguments with the initial // Wire the entry region's successor arguments with the initial
// successor inputs. // successor inputs.
assert(entrySuccessor.getSuccessor() && assert(entrySuccessor.getSuccessor() &&
"Invalid entry region without an attached successor region"); "Invalid entry region without an attached successor region");
registerDependencies( registerDependencies(
regionInterface.getSuccessorEntryOperands( regionInterface.getSuccessorEntryOperands(
entrySuccessor.getSuccessor()->getRegionNumber()), entrySuccessor.getSuccessor()->getRegionNumber()),
Show All 36 Lines

mlir/lib/Dialect/Bufferization/Transforms/TensorCopyInsertion.cpp

Show First 20 Lines • Show All 160 Lines • ▼ Show 20 Linesmlir::bufferization::insertTensorCopies(Operation *op,
return failure(result.wasInterrupted()); return failure(result.wasInterrupted());
} }
namespace { namespace {
struct TensorCopyInsertionPass struct TensorCopyInsertionPass
: public bufferization::impl::TensorCopyInsertionBase< : public bufferization::impl::TensorCopyInsertionBase<
TensorCopyInsertionPass> { TensorCopyInsertionPass> {
TensorCopyInsertionPass() : options(llvm::None) {} TensorCopyInsertionPass() : options(std::nullopt) {}
TensorCopyInsertionPass(const OneShotBufferizationOptions &options) TensorCopyInsertionPass(const OneShotBufferizationOptions &options)
: options(options) {} : options(options) {}
void getDependentDialects(DialectRegistry &registry) const override { void getDependentDialects(DialectRegistry &registry) const override {
registry.insert<bufferization::BufferizationDialect>(); registry.insert<bufferization::BufferizationDialect>();
} }
void runOnOperation() override { void runOnOperation() override {
Show All 28 Lines

mlir/lib/Dialect/Func/IR/FuncOps.cpp

Show First 20 Lines • Show All 247 Lines • ▼ Show 20 Lines state.addAttribute(FunctionOpInterface::getTypeAttrName(),
TypeAttr::get(type)); TypeAttr::get(type));
state.attributes.append(attrs.begin(), attrs.end()); state.attributes.append(attrs.begin(), attrs.end());
state.addRegion(); state.addRegion();
if (argAttrs.empty()) if (argAttrs.empty())
return; return;
assert(type.getNumInputs() == argAttrs.size()); assert(type.getNumInputs() == argAttrs.size());
function_interface_impl::addArgAndResultAttrs(builder, state, argAttrs, function_interface_impl::addArgAndResultAttrs(builder, state, argAttrs,
/*resultAttrs=*/llvm::None); /*resultAttrs=*/std::nullopt);
} }
ParseResult FuncOp::parse(OpAsmParser &parser, OperationState &result) { ParseResult FuncOp::parse(OpAsmParser &parser, OperationState &result) {
auto buildFuncType = auto buildFuncType =
[](Builder &builder, ArrayRef<Type> argTypes, ArrayRef<Type> results, [](Builder &builder, ArrayRef<Type> argTypes, ArrayRef<Type> results,
function_interface_impl::VariadicFlag, function_interface_impl::VariadicFlag,
std::string &) { return builder.getFunctionType(argTypes, results); }; std::string &) { return builder.getFunctionType(argTypes, results); };
▲ Show 20 Lines • Show All 107 Lines • Show Last 20 Lines

mlir/lib/Dialect/GPU/TransformOps/GPUTransformOps.cpp

Show First 20 Lines • Show All 97 Lines • ▼ Show 20 LinescheckGpuLimits(TransformOpInterface transformOp, Optional<int64_t> gridDimX,
return DiagnosedSilenceableFailure::success(); return DiagnosedSilenceableFailure::success();
} }
/// Creates an empty-body gpu::LaunchOp using the provided kernel settings and /// Creates an empty-body gpu::LaunchOp using the provided kernel settings and
/// put a terminator within. /// put a terminator within.
static DiagnosedSilenceableFailure static DiagnosedSilenceableFailure
createGpuLaunch(RewriterBase &rewriter, Location loc, createGpuLaunch(RewriterBase &rewriter, Location loc,
TransformOpInterface transformOp, LaunchOp &launchOp, TransformOpInterface transformOp, LaunchOp &launchOp,
Optional<int64_t> gridDimX = llvm::None, Optional<int64_t> gridDimX = std::nullopt,
Optional<int64_t> gridDimY = llvm::None, Optional<int64_t> gridDimY = std::nullopt,
Optional<int64_t> gridDimZ = llvm::None, Optional<int64_t> gridDimZ = std::nullopt,
Optional<int64_t> blockDimX = llvm::None, Optional<int64_t> blockDimX = std::nullopt,
Optional<int64_t> blockDimY = llvm::None, Optional<int64_t> blockDimY = std::nullopt,
Optional<int64_t> blockDimZ = llvm::None) { Optional<int64_t> blockDimZ = std::nullopt) {
DiagnosedSilenceableFailure diag = DiagnosedSilenceableFailure diag =
checkGpuLimits(transformOp, gridDimX, gridDimY, gridDimZ, blockDimX, checkGpuLimits(transformOp, gridDimX, gridDimY, gridDimZ, blockDimX,
blockDimY, blockDimZ); blockDimY, blockDimZ);
if (!diag.succeeded()) if (!diag.succeeded())
return diag; return diag;
auto createConst = [&](int dim) { auto createConst = [&](int dim) {
return rewriter.create<arith::ConstantIndexOp>(loc, dim); return rewriter.create<arith::ConstantIndexOp>(loc, dim);
Show All 12 LinescreateGpuLaunch(RewriterBase &rewriter, Location loc,
rewriter.create<TerminatorOp>(loc); rewriter.create<TerminatorOp>(loc);
return DiagnosedSilenceableFailure(success()); return DiagnosedSilenceableFailure(success());
} }
/// Alter kernel configuration of the given kernel. /// Alter kernel configuration of the given kernel.
static DiagnosedSilenceableFailure static DiagnosedSilenceableFailure
alterGpuLaunch(SimpleRewriter &rewriter, LaunchOp gpuLaunch, alterGpuLaunch(SimpleRewriter &rewriter, LaunchOp gpuLaunch,
TransformOpInterface transformOp, TransformOpInterface transformOp,
Optional<int64_t> gridDimX = llvm::None, Optional<int64_t> gridDimX = std::nullopt,
Optional<int64_t> gridDimY = llvm::None, Optional<int64_t> gridDimY = std::nullopt,
Optional<int64_t> gridDimZ = llvm::None, Optional<int64_t> gridDimZ = std::nullopt,
Optional<int64_t> blockDimX = llvm::None, Optional<int64_t> blockDimX = std::nullopt,
Optional<int64_t> blockDimY = llvm::None, Optional<int64_t> blockDimY = std::nullopt,
Optional<int64_t> blockDimZ = llvm::None) { Optional<int64_t> blockDimZ = std::nullopt) {
DiagnosedSilenceableFailure diag = DiagnosedSilenceableFailure diag =
checkGpuLimits(transformOp, gridDimX, gridDimY, gridDimZ, blockDimX, checkGpuLimits(transformOp, gridDimX, gridDimY, gridDimZ, blockDimX,
blockDimY, blockDimZ); blockDimY, blockDimZ);
if (!diag.succeeded()) if (!diag.succeeded())
return diag; return diag;
KernelDim3 currentBlockdim = gpuLaunch.getBlockSizeOperandValues(); KernelDim3 currentBlockdim = gpuLaunch.getBlockSizeOperandValues();
OpBuilder::InsertionGuard guard(rewriter); OpBuilder::InsertionGuard guard(rewriter);
▲ Show 20 Lines • Show All 387 Lines • ▼ Show 20 Lines if (!gpuLaunch) {
results.assign({target}); results.assign({target});
return emitSilenceableError() << "Given target is not gpu.launch"; return emitSilenceableError() << "Given target is not gpu.launch";
} }
SmallVector<int64_t> blockDim = extractFromI64ArrayAttr(getBlockDim()); SmallVector<int64_t> blockDim = extractFromI64ArrayAttr(getBlockDim());
blockDim.resize(/*size=*/3, /*value=*/1); blockDim.resize(/*size=*/3, /*value=*/1);
DiagnosedSilenceableFailure diag = DiagnosedSilenceableFailure diag =
checkGpuLimits(transformOp, llvm::None, llvm::None, llvm::None, checkGpuLimits(transformOp, std::nullopt, llvm::None, llvm::None,
blockDim[0], blockDim[1], blockDim[2]); blockDim[0], blockDim[1], blockDim[2]);
if (diag.isSilenceableFailure()) { if (diag.isSilenceableFailure()) {
results.assign({target}); results.assign({target});
diag.attachNote(getLoc()) << getBlockDimAttrName() << " is very large"; diag.attachNote(getLoc()) << getBlockDimAttrName() << " is very large";
return diag; return diag;
} }
MLIRContext *ctx = getContext(); MLIRContext *ctx = getContext();
SimpleRewriter rewriter(ctx); SimpleRewriter rewriter(ctx);
rewriter.setInsertionPoint(target); rewriter.setInsertionPoint(target);
SmallVector<DeviceMappingAttrInterface> threadMappingAttributes = { SmallVector<DeviceMappingAttrInterface> threadMappingAttributes = {
GPUThreadMappingAttr::get(ctx, Threads::DimX), GPUThreadMappingAttr::get(ctx, Threads::DimX),
GPUThreadMappingAttr::get(ctx, Threads::DimY), GPUThreadMappingAttr::get(ctx, Threads::DimY),
GPUThreadMappingAttr::get(ctx, Threads::DimZ)}; GPUThreadMappingAttr::get(ctx, Threads::DimZ)};
diag = mlir::transform::gpu::mapNestedForeachToThreadsImpl( diag = mlir::transform::gpu::mapNestedForeachToThreadsImpl(
rewriter, target, blockDim, getSyncAfterDistribute(), transformOp, rewriter, target, blockDim, getSyncAfterDistribute(), transformOp,
threadMappingAttributes); threadMappingAttributes);
if (diag.succeeded()) { if (diag.succeeded()) {
diag = diag = alterGpuLaunch(rewriter, gpuLaunch, transformOp, std::nullopt,
alterGpuLaunch(rewriter, gpuLaunch, transformOp, llvm::None, llvm::None, llvm::None, std::nullopt, blockDim[0], blockDim[1],
llvm::None, blockDim[0], blockDim[1], blockDim[2]); blockDim[2]);
} }
results.assign({gpuLaunch}); results.assign({gpuLaunch});
return diag; return diag;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Transform op registration // Transform op registration
Show All 28 Lines

mlir/lib/Dialect/GPU/Transforms/SerializeToBlob.cpp

Show All 35 Linesgpu::SerializeToBlobPass::SerializeToBlobPass(const SerializeToBlobPass &other)
: OperationPass<gpu::GPUModuleOp>(other) {} : OperationPass<gpu::GPUModuleOp>(other) {}
Optional<std::string> Optional<std::string>
gpu::SerializeToBlobPass::translateToISA(llvm::Module &llvmModule, gpu::SerializeToBlobPass::translateToISA(llvm::Module &llvmModule,
llvm::TargetMachine &targetMachine) { llvm::TargetMachine &targetMachine) {
llvmModule.setDataLayout(targetMachine.createDataLayout()); llvmModule.setDataLayout(targetMachine.createDataLayout());
if (failed(optimizeLlvm(llvmModule, targetMachine))) if (failed(optimizeLlvm(llvmModule, targetMachine)))
return llvm::None; return std::nullopt;
std::string targetISA; std::string targetISA;
llvm::raw_string_ostream stream(targetISA); llvm::raw_string_ostream stream(targetISA);
{ // Drop pstream after this to prevent the ISA from being stuck buffering { // Drop pstream after this to prevent the ISA from being stuck buffering
llvm::buffer_ostream pstream(stream); llvm::buffer_ostream pstream(stream);
llvm::legacy::PassManager codegenPasses; llvm::legacy::PassManager codegenPasses;
if (targetMachine.addPassesToEmitFile(codegenPasses, pstream, nullptr, if (targetMachine.addPassesToEmitFile(codegenPasses, pstream, nullptr,
llvm::CGFT_AssemblyFile)) llvm::CGFT_AssemblyFile))
return llvm::None; return std::nullopt;
codegenPasses.run(llvmModule); codegenPasses.run(llvmModule);
} }
return stream.str(); return stream.str();
} }
void gpu::SerializeToBlobPass::runOnOperation() { void gpu::SerializeToBlobPass::runOnOperation() {
// Lower the module to an LLVM IR module using a separate context to enable // Lower the module to an LLVM IR module using a separate context to enable
▲ Show 20 Lines • Show All 75 Lines • Show Last 20 Lines

mlir/lib/Dialect/GPU/Transforms/SerializeToHsaco.cpp

Show First 20 Lines • Show All 149 Lines • ▼ Show 20 LinesSerializeToHsacoPass::loadLibraries(SmallVectorImpl<char> &path,
SmallVectorImpl<StringRef> &libraries, SmallVectorImpl<StringRef> &libraries,
llvm::LLVMContext &context) { llvm::LLVMContext &context) {
SmallVector<std::unique_ptr<llvm::Module>, 3> ret; SmallVector<std::unique_ptr<llvm::Module>, 3> ret;
size_t dirLength = path.size(); size_t dirLength = path.size();
if (!llvm::sys::fs::is_directory(path)) { if (!llvm::sys::fs::is_directory(path)) {
getOperation().emitRemark() << "Bitcode path: " << path getOperation().emitRemark() << "Bitcode path: " << path
<< " does not exist or is not a directory\n"; << " does not exist or is not a directory\n";
return llvm::None; return std::nullopt;
} }
for (const StringRef file : libraries) { for (const StringRef file : libraries) {
llvm::SMDiagnostic error; llvm::SMDiagnostic error;
llvm::sys::path::append(path, file); llvm::sys::path::append(path, file);
llvm::StringRef pathRef(path.data(), path.size()); llvm::StringRef pathRef(path.data(), path.size());
std::unique_ptr<llvm::Module> library = std::unique_ptr<llvm::Module> library =
llvm::getLazyIRFileModule(pathRef, error, context); llvm::getLazyIRFileModule(pathRef, error, context);
path.truncate(dirLength); path.truncate(dirLength);
if (!library) { if (!library) {
getOperation().emitError() << "Failed to load library " << file getOperation().emitError() << "Failed to load library " << file
<< " from " << path << error.getMessage(); << " from " << path << error.getMessage();
return llvm::None; return std::nullopt;
} }
// Some ROCM builds don't strip this like they should // Some ROCM builds don't strip this like they should
if (auto *openclVersion = library->getNamedMetadata("opencl.ocl.version")) if (auto *openclVersion = library->getNamedMetadata("opencl.ocl.version"))
library->eraseNamedMetadata(openclVersion); library->eraseNamedMetadata(openclVersion);
// Stop spamming us with clang version numbers // Stop spamming us with clang version numbers
if (auto *ident = library->getNamedMetadata("llvm.ident")) if (auto *ident = library->getNamedMetadata("llvm.ident"))
library->eraseNamedMetadata(ident); library->eraseNamedMetadata(ident);
ret.push_back(std::move(library)); ret.push_back(std::move(library));
▲ Show 20 Lines • Show All 317 Lines • Show Last 20 Lines

mlir/lib/Dialect/LLVMIR/IR/LLVMDialect.cpp

Show First 20 Lines • Show All 770 Lines • ▼ Show 20 Lines
// Extract the pointee type from the LLVM pointer type wrapped in MLIR. Return // Extract the pointee type from the LLVM pointer type wrapped in MLIR. Return
// the resulting type if any, null type if opaque pointers are used, and None // the resulting type if any, null type if opaque pointers are used, and None
// if the given type is not the pointer type. // if the given type is not the pointer type.
static Optional<Type> getLoadStoreElementType(OpAsmParser &parser, Type type, static Optional<Type> getLoadStoreElementType(OpAsmParser &parser, Type type,
SMLoc trailingTypeLoc) { SMLoc trailingTypeLoc) {
auto llvmTy = type.dyn_cast<LLVM::LLVMPointerType>(); auto llvmTy = type.dyn_cast<LLVM::LLVMPointerType>();
if (!llvmTy) { if (!llvmTy) {
parser.emitError(trailingTypeLoc, "expected LLVM pointer type"); parser.emitError(trailingTypeLoc, "expected LLVM pointer type");
return llvm::None; return std::nullopt;
} }
return llvmTy.getElementType(); return llvmTy.getElementType();
} }
// <operation> ::= `llvm.load` `volatile` ssa-use attribute-dict? `:` type // <operation> ::= `llvm.load` `volatile` ssa-use attribute-dict? `:` type
// (`->` type)? // (`->` type)?
ParseResult LoadOp::parse(OpAsmParser &parser, OperationState &result) { ParseResult LoadOp::parse(OpAsmParser &parser, OperationState &result) {
OpAsmParser::UnresolvedOperand addr; OpAsmParser::UnresolvedOperand addr;
▲ Show 20 Lines • Show All 1,214 Lines • ▼ Show 20 Linesvoid LLVMFuncOp::build(OpBuilder &builder, OperationState &result,
if (dsoLocal) if (dsoLocal)
result.addAttribute("dso_local", builder.getUnitAttr()); result.addAttribute("dso_local", builder.getUnitAttr());
if (argAttrs.empty()) if (argAttrs.empty())
return; return;
assert(type.cast<LLVMFunctionType>().getNumParams() == argAttrs.size() && assert(type.cast<LLVMFunctionType>().getNumParams() == argAttrs.size() &&
"expected as many argument attribute lists as arguments"); "expected as many argument attribute lists as arguments");
function_interface_impl::addArgAndResultAttrs(builder, result, argAttrs, function_interface_impl::addArgAndResultAttrs(builder, result, argAttrs,
/*resultAttrs=*/llvm::None); /*resultAttrs=*/std::nullopt);
} }
// Builds an LLVM function type from the given lists of input and output types. // Builds an LLVM function type from the given lists of input and output types.
// Returns a null type if any of the types provided are non-LLVM types, or if // Returns a null type if any of the types provided are non-LLVM types, or if
// there is more than one output type. // there is more than one output type.
static Type static Type
buildLLVMFunctionType(OpAsmParser &parser, SMLoc loc, ArrayRef<Type> inputs, buildLLVMFunctionType(OpAsmParser &parser, SMLoc loc, ArrayRef<Type> inputs,
ArrayRef<Type> outputs, ArrayRef<Type> outputs,
▲ Show 20 Lines • Show All 858 Lines • Show Last 20 Lines

mlir/lib/Dialect/LLVMIR/IR/LLVMTypes.cpp

Show First 20 Lines • Show All 299 Lines • ▼ Show 20 LinesgetPointerDataLayoutEntry(DataLayoutEntryListRef params, LLVMPointerType type,
// If not found, and this is the pointer to the default memory space, assume // If not found, and this is the pointer to the default memory space, assume
// 64-bit pointers. // 64-bit pointers.
if (type.getAddressSpace() == 0) { if (type.getAddressSpace() == 0) {
return pos == PtrDLEntryPos::Size ? kDefaultPointerSizeBits return pos == PtrDLEntryPos::Size ? kDefaultPointerSizeBits
: kDefaultPointerAlignment; : kDefaultPointerAlignment;
} }
return llvm::None; return std::nullopt;
} }
unsigned unsigned
LLVMPointerType::getTypeSizeInBits(const DataLayout &dataLayout, LLVMPointerType::getTypeSizeInBits(const DataLayout &dataLayout,
DataLayoutEntryListRef params) const { DataLayoutEntryListRef params) const {
if (Optional<unsigned> size = if (Optional<unsigned> size =
getPointerDataLayoutEntry(params, *this, PtrDLEntryPos::Size)) getPointerDataLayoutEntry(params, *this, PtrDLEntryPos::Size))
return *size; return *size;
▲ Show 20 Lines • Show All 216 Lines • ▼ Show 20 Lines
static Optional<unsigned> static Optional<unsigned>
getStructDataLayoutEntry(DataLayoutEntryListRef params, LLVMStructType type, getStructDataLayoutEntry(DataLayoutEntryListRef params, LLVMStructType type,
StructDLEntryPos pos) { StructDLEntryPos pos) {
const auto *currentEntry = const auto *currentEntry =
llvm::find_if(params, [](DataLayoutEntryInterface entry) { llvm::find_if(params, [](DataLayoutEntryInterface entry) {
return entry.isTypeEntry(); return entry.isTypeEntry();
}); });
if (currentEntry == params.end()) if (currentEntry == params.end())
return llvm::None; return std::nullopt;
auto attr = currentEntry->getValue().cast<DenseIntElementsAttr>(); auto attr = currentEntry->getValue().cast<DenseIntElementsAttr>();
if (pos == StructDLEntryPos::Preferred && if (pos == StructDLEntryPos::Preferred &&
attr.size() <= static_cast<unsigned>(StructDLEntryPos::Preferred)) attr.size() <= static_cast<unsigned>(StructDLEntryPos::Preferred))
// If no preferred was specified, fall back to abi alignment // If no preferred was specified, fall back to abi alignment
pos = StructDLEntryPos::Abi; pos = StructDLEntryPos::Abi;
return attr.getValues<unsigned>()[static_cast<unsigned>(pos)]; return attr.getValues<unsigned>()[static_cast<unsigned>(pos)];
▲ Show 20 Lines • Show All 464 Lines • Show Last 20 Lines

mlir/lib/Dialect/LLVMIR/IR/NVVMDialect.cpp

Show First 20 Lines • Show All 86 Lines • ▼ Show 20 Lines if (operandElType.isF16() || operandElType == half2Type)
return NVVM::MMATypes::f16; return NVVM::MMATypes::f16;
if (operandElType.isF32() && isAccumulator) if (operandElType.isF32() && isAccumulator)
return NVVM::MMATypes::f32; return NVVM::MMATypes::f32;
if (operandElType.isF32() && !isAccumulator) if (operandElType.isF32() && !isAccumulator)
return NVVM::MMATypes::tf32; return NVVM::MMATypes::tf32;
if (operandElType.isa<IntegerType>()) { if (operandElType.isa<IntegerType>()) {
if (isAccumulator) if (isAccumulator)
return NVVM::MMATypes::s32; return NVVM::MMATypes::s32;
return llvm::None; return std::nullopt;
} }
if (auto structType = operandElType.dyn_cast<LLVM::LLVMStructType>()) { if (auto structType = operandElType.dyn_cast<LLVM::LLVMStructType>()) {
if (structType.getBody().empty()) if (structType.getBody().empty())
return llvm::None; return std::nullopt;
return inferOperandMMAType(structType.getBody()[0], isAccumulator); return inferOperandMMAType(structType.getBody()[0], isAccumulator);
} }
return llvm::None; return std::nullopt;
} }
static bool isInt4PtxType(MMATypes type) { static bool isInt4PtxType(MMATypes type) {
return (type == MMATypes::u4 || type == MMATypes::s4); return (type == MMATypes::u4 || type == MMATypes::s4);
} }
static bool isInt8PtxType(MMATypes type) { static bool isInt8PtxType(MMATypes type) {
return (type == MMATypes::u8 || type == MMATypes::s8); return (type == MMATypes::u8 || type == MMATypes::s8);
▲ Show 20 Lines • Show All 604 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/TransformOps/LinalgTransformOps.cpp

Show First 20 Lines • Show All 1,168 Lines • ▼ Show 20 Linestransform::TileReductionUsingForeachThreadOp::applyToOne(
SmallVector<OpFoldResult> numThreadResults; SmallVector<OpFoldResult> numThreadResults;
for (int64_t num : numThreads) { for (int64_t num : numThreads) {
numThreadResults.push_back(rewriter.getIndexAttr(num)); numThreadResults.push_back(rewriter.getIndexAttr(num));
} }
FailureOr<linalg::ForeachThreadReductionTilingResult> result = FailureOr<linalg::ForeachThreadReductionTilingResult> result =
linalg::tileReductionUsingForeachThread( linalg::tileReductionUsingForeachThread(
rewriter, cast<PartialReductionOpInterface>(target.getOperation()), rewriter, cast<PartialReductionOpInterface>(target.getOperation()),
numThreadResults, /*mapping=*/llvm::None); numThreadResults, /*mapping=*/std::nullopt);
if (failed(result)) { if (failed(result)) {
results.assign(3, nullptr); results.assign(3, nullptr);
Diagnostic diag(target->getLoc(), DiagnosticSeverity::Remark); Diagnostic diag(target->getLoc(), DiagnosticSeverity::Remark);
diag << "could not tile reduction in target."; diag << "could not tile reduction in target.";
return DiagnosedSilenceableFailure::silenceableFailure(std::move(diag)); return DiagnosedSilenceableFailure::silenceableFailure(std::move(diag));
} }
results.push_back(result->initialOp); results.push_back(result->initialOp);
▲ Show 20 Lines • Show All 652 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/Transforms/ConstantFold.cpp

Show First 20 Lines • Show All 285 Lines • ▼ Show 20 Lines for (Value yieldVal : yieldOp.getValues()) {
return nullptr; return nullptr;
if (yieldArg.getArgNumber() != 0) if (yieldArg.getArgNumber() != 0)
return nullptr; return nullptr;
} }
// No computation; just return the orginal value. // No computation; just return the orginal value.
return [](const APIntOrFloatArray &inputs) { return [](const APIntOrFloatArray &inputs) {
if (inputs.apFloats.empty()) if (inputs.apFloats.empty())
return APIntOrFloat{inputs.apInts.front(), llvm::None}; return APIntOrFloat{inputs.apInts.front(), std::nullopt};
return APIntOrFloat{llvm::None, inputs.apFloats.front()}; return APIntOrFloat{std::nullopt, inputs.apFloats.front()};
}; };
} }
ControlFusionFn controlFn; ControlFusionFn controlFn;
}; };
} // namespace } // namespace
void mlir::linalg::populateConstantFoldLinalgOperations( void mlir::linalg::populateConstantFoldLinalgOperations(
RewritePatternSet &patterns, const ControlFusionFn &controlFn) { RewritePatternSet &patterns, const ControlFusionFn &controlFn) {
MLIRContext *context = patterns.getContext(); MLIRContext *context = patterns.getContext();
patterns.insert<FoldConstantTranspose>(context, controlFn); patterns.insert<FoldConstantTranspose>(context, controlFn);
} }

mlir/lib/Dialect/Linalg/Transforms/DropUnitDims.cpp

Show First 20 Lines • Show All 380 Lines • ▼ Show 20 Lines auto isUnitExtent = [&](int64_t dim) -> bool {
return shape[dim] == 1 && exprs[dim] == zeroExpr; return shape[dim] == 1 && exprs[dim] == zeroExpr;
}; };
// Early return for memrefs with affine maps to represent that we will always // Early return for memrefs with affine maps to represent that we will always
// leave them unchanged. // leave them unchanged.
Type actualType = opOperand->get().getType(); Type actualType = opOperand->get().getType();
if (auto memref = actualType.dyn_cast<MemRefType>()) { if (auto memref = actualType.dyn_cast<MemRefType>()) {
if (!memref.getLayout().isIdentity()) if (!memref.getLayout().isIdentity())
return llvm::None; return std::nullopt;
} }
int64_t dim = 0; int64_t dim = 0;
// Fold dimensions that are unit-extent at the beginning of the tensor. // Fold dimensions that are unit-extent at the beginning of the tensor.
while (dim < origRank && isUnitExtent(dim)) while (dim < origRank && isUnitExtent(dim))
reassociations.push_back(getAffineDimExpr(dim++, context)); reassociations.push_back(getAffineDimExpr(dim++, context));
while (dim < origRank) { while (dim < origRank) {
reassociations.push_back(getAffineDimExpr(dim, context)); reassociations.push_back(getAffineDimExpr(dim, context));
▲ Show 20 Lines • Show All 296 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/Transforms/ElementwiseOpFusion.cpp

Show First 20 Lines • Show All 728 Lines • ▼ Show 20 Lines RankedTensorType collapsedType = isExpanding
: collapsingReshapeOp.getResultType(); : collapsingReshapeOp.getResultType();
ExpansionInfo expansionInfo; ExpansionInfo expansionInfo;
if (failed(expansionInfo.compute( if (failed(expansionInfo.compute(
genericOp, fusableOpOperand, genericOp, fusableOpOperand,
isExpanding ? expandingReshapeOp.getReassociationMaps() isExpanding ? expandingReshapeOp.getReassociationMaps()
: collapsingReshapeOp.getReassociationMaps(), : collapsingReshapeOp.getReassociationMaps(),
expandedType.getShape(), collapsedType.getShape(), rewriter))) expandedType.getShape(), collapsedType.getShape(), rewriter)))
return llvm::None; return std::nullopt;
if (failed(isGenericOpExpandable(genericOp, expansionInfo, rewriter))) if (failed(isGenericOpExpandable(genericOp, expansionInfo, rewriter)))
return llvm::None; return std::nullopt;
SmallVector<AffineMap, 4> expandedOpIndexingMaps = llvm::to_vector<4>( SmallVector<AffineMap, 4> expandedOpIndexingMaps = llvm::to_vector<4>(
llvm::map_range(genericOp.getIndexingMapsArray(), [&](AffineMap m) { llvm::map_range(genericOp.getIndexingMapsArray(), [&](AffineMap m) {
return getIndexingMapInExpandedOp(rewriter, m, expansionInfo); return getIndexingMapInExpandedOp(rewriter, m, expansionInfo);
})); }));
// Set insertion point to the generic op. // Set insertion point to the generic op.
OpBuilder::InsertionGuard g(rewriter); OpBuilder::InsertionGuard g(rewriter);
Show All 18 Lines if (auto opOperandType =
getReassociationForExpansion(indexingMap, expansionInfo); getReassociationForExpansion(indexingMap, expansionInfo);
if (failed(reshapeLikeShapesAreCompatible( if (failed(reshapeLikeShapesAreCompatible(
[&](const Twine &msg) { [&](const Twine &msg) {
return rewriter.notifyMatchFailure(genericOp, msg); return rewriter.notifyMatchFailure(genericOp, msg);
}, },
opOperandType.getShape(), expandedOperandType.getShape(), opOperandType.getShape(), expandedOperandType.getShape(),
reassociation, reassociation,
/*isExpandingReshape=*/true))) /*isExpandingReshape=*/true)))
return llvm::None; return std::nullopt;
expandedOpOperands.push_back(rewriter.create<tensor::ExpandShapeOp>( expandedOpOperands.push_back(rewriter.create<tensor::ExpandShapeOp>(
genericOp.getLoc(), expandedOperandType, opOperand->get(), genericOp.getLoc(), expandedOperandType, opOperand->get(),
reassociation)); reassociation));
continue; continue;
} }
} }
expandedOpOperands.push_back(opOperand->get()); expandedOpOperands.push_back(opOperand->get());
} }
Show All 10 Lines if (expandedOutputType != opOperand->get().getType()) {
getReassociationForExpansion(indexingMap, expansionInfo); getReassociationForExpansion(indexingMap, expansionInfo);
if (failed(reshapeLikeShapesAreCompatible( if (failed(reshapeLikeShapesAreCompatible(
[&](const Twine &msg) { [&](const Twine &msg) {
return rewriter.notifyMatchFailure(genericOp, msg); return rewriter.notifyMatchFailure(genericOp, msg);
}, },
opOperandType.getShape(), expandedOutputType.getShape(), opOperandType.getShape(), expandedOutputType.getShape(),
reassociation, reassociation,
/*isExpandingReshape=*/true))) /*isExpandingReshape=*/true)))
return llvm::None; return std::nullopt;
outputs.push_back(rewriter.create<tensor::ExpandShapeOp>( outputs.push_back(rewriter.create<tensor::ExpandShapeOp>(
genericOp.getLoc(), expandedOutputType, opOperand->get(), genericOp.getLoc(), expandedOutputType, opOperand->get(),
reassociation)); reassociation));
} else { } else {
outputs.push_back(opOperand->get()); outputs.push_back(opOperand->get());
} }
} }
▲ Show 20 Lines • Show All 144 Lines • ▼ Show 20 Lines
} // namespace } // namespace
//===---------------------------------------------------------------------===// //===---------------------------------------------------------------------===//
// Methods and patterns to fuse reshape with linalg.generic operations by // Methods and patterns to fuse reshape with linalg.generic operations by
// contraction of dimensions. // contraction of dimensions.
//===---------------------------------------------------------------------===// //===---------------------------------------------------------------------===//
/// For a given list of indices in the range of the `indexingMap` that are /// For a given list of indices in the range of the `indexingMap` that are
/// folded, return the indices of the corresponding domain. Return `llvm::None` /// folded, return the indices of the corresponding domain. Return
/// on failure. Ensures that all the elements of the returned reassociation are /// `std::nullopt` on failure. Ensures that all the elements of the returned
/// distinct. /// reassociation are distinct.
static ReassociationIndices static ReassociationIndices
getDomainReassociation(AffineMap indexingMap, getDomainReassociation(AffineMap indexingMap,
ReassociationIndicesRef rangeReassociation) { ReassociationIndicesRef rangeReassociation) {
assert(indexingMap.isProjectedPermutation() && assert(indexingMap.isProjectedPermutation() &&
"expected projected permutation"); "expected projected permutation");
ReassociationIndices domainReassociation = llvm::to_vector<4>( ReassociationIndices domainReassociation = llvm::to_vector<4>(
llvm::map_range(rangeReassociation, [&](int64_t pos) -> int64_t { llvm::map_range(rangeReassociation, [&](int64_t pos) -> int64_t {
▲ Show 20 Lines • Show All 901 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/Transforms/Tiling.cpp

Show First 20 Lines • Show All 461 Lines • ▼ Show 20 Lines scf::ForeachThreadOp foreachThreadOp = b.create<scf::ForeachThreadOp>(
ValueRange(materializedNonZeroNumThreads), mapping); ValueRange(materializedNonZeroNumThreads), mapping);
// 3. calculate the tile offsets and sizes. // 3. calculate the tile offsets and sizes.
b.setInsertionPointToStart(foreachThreadOp.getBody(0)); b.setInsertionPointToStart(foreachThreadOp.getBody(0));
SmallVector<OpFoldResult> tiledOffsets, tiledSizes; SmallVector<OpFoldResult> tiledOffsets, tiledSizes;
calculateTileOffsetsAndSizes( calculateTileOffsetsAndSizes(
b, loc, foreachThreadOp, numThreads, iterationDomain, b, loc, foreachThreadOp, numThreads, iterationDomain,
/*omitTileOffsetBoundsCheck =*/false, /*omitTileOffsetBoundsCheck =*/false,
/*nominalTileSizes=*/llvm::None, tiledOffsets, tiledSizes); /*nominalTileSizes=*/std::nullopt, tiledOffsets, tiledSizes);
// 4. Clone the tileable op and update its destination operands to use the // 4. Clone the tileable op and update its destination operands to use the
// output bbArgs of the ForeachThreadOp. // output bbArgs of the ForeachThreadOp.
ArrayRef<BlockArgument> destBbArgs = ArrayRef<BlockArgument> destBbArgs =
foreachThreadOp.getOutputBlockArguments(); foreachThreadOp.getOutputBlockArguments();
Operation *clonedOp = b.clone(*op.getOperation()); Operation *clonedOp = b.clone(*op.getOperation());
auto destinationStyleOp = cast<DestinationStyleOpInterface>(clonedOp); auto destinationStyleOp = cast<DestinationStyleOpInterface>(clonedOp);
for (OpOperand *outOperand : destinationStyleOp.getDpsInitOperands()) { for (OpOperand *outOperand : destinationStyleOp.getDpsInitOperands()) {
▲ Show 20 Lines • Show All 414 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/Transforms/Vectorization.cpp

Show First 20 Lines • Show All 117 Lines • ▼ Show 20 Linesstruct VectorizationResult {
Operation *newOp; Operation *newOp;
}; };
llvm::Optional<vector::CombiningKind> llvm::Optional<vector::CombiningKind>
mlir::linalg::getCombinerOpKind(Operation *combinerOp) { mlir::linalg::getCombinerOpKind(Operation *combinerOp) {
using ::mlir::vector::CombiningKind; using ::mlir::vector::CombiningKind;
if (!combinerOp) if (!combinerOp)
return llvm::None; return std::nullopt;
return llvm::TypeSwitch<Operation *, llvm::Optional<CombiningKind>>( return llvm::TypeSwitch<Operation *, llvm::Optional<CombiningKind>>(
combinerOp) combinerOp)
.Case<arith::AddIOp, arith::AddFOp>( .Case<arith::AddIOp, arith::AddFOp>(
[&](auto op) { return CombiningKind::ADD; }) [&](auto op) { return CombiningKind::ADD; })
.Case<arith::AndIOp>([&](auto op) { return CombiningKind::AND; }) .Case<arith::AndIOp>([&](auto op) { return CombiningKind::AND; })
.Case<arith::MaxSIOp>([&](auto op) { return CombiningKind::MAXSI; }) .Case<arith::MaxSIOp>([&](auto op) { return CombiningKind::MAXSI; })
.Case<arith::MaxFOp>([&](auto op) { return CombiningKind::MAXF; }) .Case<arith::MaxFOp>([&](auto op) { return CombiningKind::MAXF; })
.Case<arith::MinSIOp>([&](auto op) { return CombiningKind::MINSI; }) .Case<arith::MinSIOp>([&](auto op) { return CombiningKind::MINSI; })
.Case<arith::MinFOp>([&](auto op) { return CombiningKind::MINF; }) .Case<arith::MinFOp>([&](auto op) { return CombiningKind::MINF; })
.Case<arith::MulIOp, arith::MulFOp>( .Case<arith::MulIOp, arith::MulFOp>(
[&](auto op) { return CombiningKind::MUL; }) [&](auto op) { return CombiningKind::MUL; })
.Case<arith::OrIOp>([&](auto op) { return CombiningKind::OR; }) .Case<arith::OrIOp>([&](auto op) { return CombiningKind::OR; })
.Case<arith::XOrIOp>([&](auto op) { return CombiningKind::XOR; }) .Case<arith::XOrIOp>([&](auto op) { return CombiningKind::XOR; })
.Default([&](auto op) { return llvm::None; }); .Default([&](auto op) { return std::nullopt; });
} }
/// Check whether `outputOperand` is a reduction with a single combiner /// Check whether `outputOperand` is a reduction with a single combiner
/// operation. Return the combiner operation of the reduction. Return /// operation. Return the combiner operation of the reduction. Return
/// nullptr otherwise. Multiple reduction operations would impose an /// nullptr otherwise. Multiple reduction operations would impose an
/// ordering between reduction dimensions and is currently unsupported in /// ordering between reduction dimensions and is currently unsupported in
/// Linalg. This limitation is motivated by the fact that e.g. min(max(X)) != /// Linalg. This limitation is motivated by the fact that e.g. min(max(X)) !=
/// max(min(X)) /// max(min(X))
▲ Show 20 Lines • Show All 1,772 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/Utils/Utils.cpp

Show First 20 Lines • Show All 99 Lines • ▼ Show 20 Lines if (isTiled(map.getResult(r), tileSizes))
return true; return true;
return false; return false;
} }
Optional<RegionMatcher::BinaryOpKind> Optional<RegionMatcher::BinaryOpKind>
RegionMatcher::matchAsScalarBinaryOp(GenericOp op) { RegionMatcher::matchAsScalarBinaryOp(GenericOp op) {
auto &region = op.getRegion(); auto &region = op.getRegion();
if (!llvm::hasSingleElement(region)) if (!llvm::hasSingleElement(region))
return llvm::None; return std::nullopt;
Block &block = region.front(); Block &block = region.front();
if (block.getNumArguments() != 2 || if (block.getNumArguments() != 2 ||
!block.getArgument(0).getType().isSignlessIntOrFloat() || !block.getArgument(0).getType().isSignlessIntOrFloat() ||
!block.getArgument(1).getType().isSignlessIntOrFloat()) !block.getArgument(1).getType().isSignlessIntOrFloat())
return llvm::None; return std::nullopt;
auto &ops = block.getOperations(); auto &ops = block.getOperations();
if (!llvm::hasSingleElement(block.without_terminator())) if (!llvm::hasSingleElement(block.without_terminator()))
return llvm::None; return std::nullopt;
using mlir::matchers::m_Val; using mlir::matchers::m_Val;
auto a = m_Val(block.getArgument(0)); auto a = m_Val(block.getArgument(0));
auto b = m_Val(block.getArgument(1)); auto b = m_Val(block.getArgument(1));
auto addPattern = m_Op<linalg::YieldOp>(m_Op<arith::AddIOp>(a, b)); auto addPattern = m_Op<linalg::YieldOp>(m_Op<arith::AddIOp>(a, b));
if (addPattern.match(&ops.back())) if (addPattern.match(&ops.back()))
return BinaryOpKind::IAdd; return BinaryOpKind::IAdd;
return llvm::None; return std::nullopt;
} }
/// Explicit instantiation of loop nest generator for different loop types. /// Explicit instantiation of loop nest generator for different loop types.
template struct mlir::linalg::GenerateLoopNest<scf::ForOp>; template struct mlir::linalg::GenerateLoopNest<scf::ForOp>;
template struct mlir::linalg::GenerateLoopNest<scf::ParallelOp>; template struct mlir::linalg::GenerateLoopNest<scf::ParallelOp>;
template struct mlir::linalg::GenerateLoopNest<AffineForOp>; template struct mlir::linalg::GenerateLoopNest<AffineForOp>;
/// Given a list of subview ranges, extract individual values for lower, upper /// Given a list of subview ranges, extract individual values for lower, upper
▲ Show 20 Lines • Show All 823 Lines • ▼ Show 20 Lines for (auto [opOperand, val] :
// an extract/insert slice pair for all output tensors simplifies follow up // an extract/insert slice pair for all output tensors simplifies follow up
// transformations such as padding and bufferization since the // transformations such as padding and bufferization since the
// extract/insert slice pairs make the accessed iteration argument // extract/insert slice pairs make the accessed iteration argument
// subdomains explicit. // subdomains explicit.
Type operandType = opOperand.get().getType(); Type operandType = opOperand.get().getType();
if (!isTiled(map, tileSizes) && !(operandType.isa<RankedTensorType>() && if (!isTiled(map, tileSizes) && !(operandType.isa<RankedTensorType>() &&
linalgOp.isDpsInit(&opOperand))) { linalgOp.isDpsInit(&opOperand))) {
allSliceParams.push_back(llvm::None); allSliceParams.push_back(std::nullopt);
LLVM_DEBUG(llvm::dbgs() LLVM_DEBUG(llvm::dbgs()
<< ": not tiled: use shape: " << operandType << "\n"); << ": not tiled: use shape: " << operandType << "\n");
continue; continue;
} }
LLVM_DEBUG(llvm::dbgs() << ": tiled: figure out subshape...\n"); LLVM_DEBUG(llvm::dbgs() << ": tiled: figure out subshape...\n");
allSliceParams.push_back(computeSliceParameters( allSliceParams.push_back(computeSliceParameters(
builder, loc, shapedOp, tileSizes, map, lbs, sizeBounds, subShapeSizes, builder, loc, shapedOp, tileSizes, map, lbs, sizeBounds, subShapeSizes,
▲ Show 20 Lines • Show All 105 Lines • ▼ Show 20 LinesOptional<Attribute> getNeutralElement(Operation *op) {
if (isa<arith::AndIOp>(op)) if (isa<arith::AndIOp>(op))
return b.getIntegerAttr(resultType, -1); return b.getIntegerAttr(resultType, -1);
if (isa<arith::MaxSIOp>(op)) if (isa<arith::MaxSIOp>(op))
return b.getIntegerAttr(resultType, std::numeric_limits<int64_t>::min()); return b.getIntegerAttr(resultType, std::numeric_limits<int64_t>::min());
if (isa<arith::MinSIOp>(op)) if (isa<arith::MinSIOp>(op))
return b.getIntegerAttr(resultType, std::numeric_limits<int64_t>::max()); return b.getIntegerAttr(resultType, std::numeric_limits<int64_t>::max());
if (isa<arith::MulIOp>(op)) if (isa<arith::MulIOp>(op))
return b.getIntegerAttr(resultType, 1); return b.getIntegerAttr(resultType, 1);
return llvm::None; return std::nullopt;
} }
} // namespace linalg } // namespace linalg
} // namespace mlir } // namespace mlir

mlir/lib/Dialect/MemRef/IR/MemRefDialect.cpp

Show First 20 Lines • Show All 44 Lines • ▼ Show 20 Lines
/// Finds the unique dealloc operation (if one exists) for `allocValue`. /// Finds the unique dealloc operation (if one exists) for `allocValue`.
llvm::Optional<Operation *> mlir::memref::findDealloc(Value allocValue) { llvm::Optional<Operation *> mlir::memref::findDealloc(Value allocValue) {
Operation *dealloc = nullptr; Operation *dealloc = nullptr;
for (Operation *user : allocValue.getUsers()) { for (Operation *user : allocValue.getUsers()) {
if (!hasEffect<MemoryEffects::Free>(user, allocValue)) if (!hasEffect<MemoryEffects::Free>(user, allocValue))
continue; continue;
// If we found > 1 dealloc, return None. // If we found > 1 dealloc, return None.
if (dealloc) if (dealloc)
return llvm::None; return std::nullopt;
dealloc = user; dealloc = user;
} }
return dealloc; return dealloc;
} }

mlir/lib/Dialect/MemRef/IR/MemRefOps.cpp

Show First 20 Lines • Show All 980 Lines • ▼ Show 20 Lines if (static_cast<int64_t>(unusedDims.count()) + reducedType.getRank() ==
return unusedDims; return unusedDims;
SmallVector<int64_t> originalStrides, candidateStrides; SmallVector<int64_t> originalStrides, candidateStrides;
int64_t originalOffset, candidateOffset; int64_t originalOffset, candidateOffset;
if (failed( if (failed(
getStridesAndOffset(originalType, originalStrides, originalOffset)) || getStridesAndOffset(originalType, originalStrides, originalOffset)) ||
failed( failed(
getStridesAndOffset(reducedType, candidateStrides, candidateOffset))) getStridesAndOffset(reducedType, candidateStrides, candidateOffset)))
return llvm::None; return std::nullopt;
// For memrefs, a dimension is truly dropped if its corresponding stride is // For memrefs, a dimension is truly dropped if its corresponding stride is
// also dropped. This is particularly important when more than one of the dims // also dropped. This is particularly important when more than one of the dims
// is 1. Track the number of occurences of the strides in the original type // is 1. Track the number of occurences of the strides in the original type
// and the candidate type. For each unused dim that stride should not be // and the candidate type. For each unused dim that stride should not be
// present in the candidate type. Note that there could be multiple dimensions // present in the candidate type. Note that there could be multiple dimensions
// that have the same size. We dont need to exactly figure out which dim // that have the same size. We dont need to exactly figure out which dim
// corresponds to which stride, we just need to verify that the number of // corresponds to which stride, we just need to verify that the number of
Show All 18 Lines if (currUnaccountedStrides[originalStride] ==
// The stride for this is not dropped. Keep as is. // The stride for this is not dropped. Keep as is.
unusedDims.reset(dim); unusedDims.reset(dim);
continue; continue;
} }
if (currUnaccountedStrides[originalStride] < if (currUnaccountedStrides[originalStride] <
candidateStridesNumOccurences[originalStride]) { candidateStridesNumOccurences[originalStride]) {
// This should never happen. Cant have a stride in the reduced rank type // This should never happen. Cant have a stride in the reduced rank type
// that wasnt in the original one. // that wasnt in the original one.
return llvm::None; return std::nullopt;
} }
} }
if ((int64_t)unusedDims.count() + reducedType.getRank() != if ((int64_t)unusedDims.count() + reducedType.getRank() !=
originalType.getRank()) originalType.getRank())
return llvm::None; return std::nullopt;
return unusedDims; return unusedDims;
} }
llvm::SmallBitVector SubViewOp::getDroppedDims() { llvm::SmallBitVector SubViewOp::getDroppedDims() {
MemRefType sourceType = getSourceType(); MemRefType sourceType = getSourceType();
MemRefType resultType = getType(); MemRefType resultType = getType();
llvm::Optional<llvm::SmallBitVector> unusedDims = llvm::Optional<llvm::SmallBitVector> unusedDims =
computeMemRefRankReductionMask(sourceType, resultType, getMixedSizes()); computeMemRefRankReductionMask(sourceType, resultType, getMixedSizes());
▲ Show 20 Lines • Show All 2,327 Lines • Show Last 20 Lines

mlir/lib/Dialect/OpenACC/IR/OpenACC.cpp

Show First 20 Lines • Show All 128 Lines • ▼ Show 20 Lines
static OptionalParseResult parseOptionalOperandAndType(OpAsmParser &parser, static OptionalParseResult parseOptionalOperandAndType(OpAsmParser &parser,
StringRef keyword, StringRef keyword,
OperationState &result) { OperationState &result) {
OpAsmParser::UnresolvedOperand operand; OpAsmParser::UnresolvedOperand operand;
if (succeeded(parser.parseOptionalKeyword(keyword))) { if (succeeded(parser.parseOptionalKeyword(keyword))) {
return failure(parser.parseLParen() || return failure(parser.parseLParen() ||
parseOperandAndType(parser, result) || parser.parseRParen()); parseOperandAndType(parser, result) || parser.parseRParen());
} }
return llvm::None; return std::nullopt;
} }
/// Parse optional operand and its type wrapped in parenthesis. /// Parse optional operand and its type wrapped in parenthesis.
/// Example: /// Example:
/// `(` %vectorLength: i64 `)` /// `(` %vectorLength: i64 `)`
static OptionalParseResult parseOptionalOperandAndType(OpAsmParser &parser, static OptionalParseResult parseOptionalOperandAndType(OpAsmParser &parser,
OperationState &result) { OperationState &result) {
if (succeeded(parser.parseOptionalLParen())) { if (succeeded(parser.parseOptionalLParen())) {
return failure(parseOperandAndType(parser, result) || parser.parseRParen()); return failure(parseOperandAndType(parser, result) || parser.parseRParen());
} }
return llvm::None; return std::nullopt;
} }
/// Parse optional operand with its type prefixed with prefixKeyword `=`. /// Parse optional operand with its type prefixed with prefixKeyword `=`.
/// Example: /// Example:
/// num=%gangNum: i32 /// num=%gangNum: i32
static OptionalParseResult parserOptionalOperandAndTypeWithPrefix( static OptionalParseResult parserOptionalOperandAndTypeWithPrefix(
OpAsmParser &parser, OperationState &result, StringRef prefixKeyword) { OpAsmParser &parser, OperationState &result, StringRef prefixKeyword) {
if (succeeded(parser.parseOptionalKeyword(prefixKeyword))) { if (succeeded(parser.parseOptionalKeyword(prefixKeyword))) {
if (parser.parseEqual() || parseOperandAndType(parser, result)) if (parser.parseEqual() || parseOperandAndType(parser, result))
return failure(); return failure();
return success(); return success();
} }
return llvm::None; return std::nullopt;
} }
static bool isComputeOperation(Operation *op) { static bool isComputeOperation(Operation *op) {
return isa<acc::ParallelOp>(op) || isa<acc::LoopOp>(op); return isa<acc::ParallelOp>(op) || isa<acc::LoopOp>(op);
} }
namespace { namespace {
/// Pattern to remove operation without region that have constant false `ifCond` /// Pattern to remove operation without region that have constant false `ifCond`
▲ Show 20 Lines • Show All 737 Lines • Show Last 20 Lines

mlir/lib/Dialect/OpenMP/IR/OpenMPDialect.cpp

Show First 20 Lines • Show All 306 Lines • ▼ Show 20 Linesstatic ParseResult parseScheduleClause(
case ClauseScheduleKind::Static: case ClauseScheduleKind::Static:
case ClauseScheduleKind::Dynamic: case ClauseScheduleKind::Dynamic:
case ClauseScheduleKind::Guided: case ClauseScheduleKind::Guided:
if (succeeded(parser.parseOptionalEqual())) { if (succeeded(parser.parseOptionalEqual())) {
chunkSize = OpAsmParser::UnresolvedOperand{}; chunkSize = OpAsmParser::UnresolvedOperand{};
if (parser.parseOperand(*chunkSize) || parser.parseColonType(chunkType)) if (parser.parseOperand(*chunkSize) || parser.parseColonType(chunkType))
return failure(); return failure();
} else { } else {
chunkSize = llvm::None; chunkSize = std::nullopt;
} }
break; break;
case ClauseScheduleKind::Auto: case ClauseScheduleKind::Auto:
case ClauseScheduleKind::Runtime: case ClauseScheduleKind::Runtime:
chunkSize = llvm::None; chunkSize = std::nullopt;
} }
// If there is a comma, we have one or more modifiers.. // If there is a comma, we have one or more modifiers..
SmallVector<SmallString<12>> modifiers; SmallVector<SmallString<12>> modifiers;
while (succeeded(parser.parseOptionalComma())) { while (succeeded(parser.parseOptionalComma())) {
StringRef mod; StringRef mod;
if (parser.parseKeyword(&mod)) if (parser.parseKeyword(&mod))
return failure(); return failure();
▲ Show 20 Lines • Show All 818 Lines • Show Last 20 Lines

mlir/lib/Dialect/SCF/IR/SCF.cpp

Show First 20 Lines • Show All 577 Lines • ▼ Show 20 LinesLoopNest mlir::scf::buildLoopNest(
return res; return res;
} }
LoopNest mlir::scf::buildLoopNest( LoopNest mlir::scf::buildLoopNest(
OpBuilder &builder, Location loc, ValueRange lbs, ValueRange ubs, OpBuilder &builder, Location loc, ValueRange lbs, ValueRange ubs,
ValueRange steps, ValueRange steps,
function_ref<void(OpBuilder &, Location, ValueRange)> bodyBuilder) { function_ref<void(OpBuilder &, Location, ValueRange)> bodyBuilder) {
// Delegate to the main function by wrapping the body builder. // Delegate to the main function by wrapping the body builder.
return buildLoopNest(builder, loc, lbs, ubs, steps, llvm::None, return buildLoopNest(builder, loc, lbs, ubs, steps, std::nullopt,
[&bodyBuilder](OpBuilder &nestedBuilder, [&bodyBuilder](OpBuilder &nestedBuilder,
Location nestedLoc, ValueRange ivs, Location nestedLoc, ValueRange ivs,
ValueRange) -> ValueVector { ValueRange) -> ValueVector {
if (bodyBuilder) if (bodyBuilder)
bodyBuilder(nestedBuilder, nestedLoc, ivs); bodyBuilder(nestedBuilder, nestedLoc, ivs);
return {}; return {};
}); });
} }
▲ Show 20 Lines • Show All 119 Lines • ▼ Show 20 Lines LogicalResult matchAndRewrite(scf::ForOp forOp,
cloneFilteredTerminator(mergedYieldOp); cloneFilteredTerminator(mergedYieldOp);
rewriter.eraseOp(mergedYieldOp); rewriter.eraseOp(mergedYieldOp);
rewriter.replaceOp(forOp, newResultValues); rewriter.replaceOp(forOp, newResultValues);
return success(); return success();
} }
}; };
/// Util function that tries to compute a constant diff between u and l. /// Util function that tries to compute a constant diff between u and l.
/// Returns llvm::None when the difference between two AffineValueMap is /// Returns std::nullopt when the difference between two AffineValueMap is
/// dynamic. /// dynamic.
static Optional<int64_t> computeConstDiff(Value l, Value u) { static Optional<int64_t> computeConstDiff(Value l, Value u) {
IntegerAttr clb, cub; IntegerAttr clb, cub;
if (matchPattern(l, m_Constant(&clb)) && matchPattern(u, m_Constant(&cub))) { if (matchPattern(l, m_Constant(&clb)) && matchPattern(u, m_Constant(&cub))) {
llvm::APInt lbValue = clb.getValue(); llvm::APInt lbValue = clb.getValue();
llvm::APInt ubValue = cub.getValue(); llvm::APInt ubValue = cub.getValue();
return (ubValue - lbValue).getSExtValue(); return (ubValue - lbValue).getSExtValue();
} }
// Else a simple pattern match for x + c or c + x // Else a simple pattern match for x + c or c + x
llvm::APInt diff; llvm::APInt diff;
if (matchPattern( if (matchPattern(
u, m_Op<arith::AddIOp>(matchers::m_Val(l), m_ConstantInt(&diff))) || u, m_Op<arith::AddIOp>(matchers::m_Val(l), m_ConstantInt(&diff))) ||
matchPattern( matchPattern(
u, m_Op<arith::AddIOp>(m_ConstantInt(&diff), matchers::m_Val(l)))) u, m_Op<arith::AddIOp>(m_ConstantInt(&diff), matchers::m_Val(l))))
return diff.getSExtValue(); return diff.getSExtValue();
return llvm::None; return std::nullopt;
} }
/// Rewriting pattern that erases loops that are known not to iterate, replaces /// Rewriting pattern that erases loops that are known not to iterate, replaces
/// single-iteration loops with their bodies, and removes empty loops that /// single-iteration loops with their bodies, and removes empty loops that
/// iterate at least once and only return values defined outside of the loop. /// iterate at least once and only return values defined outside of the loop.
struct SimplifyTrivialLoops : public OpRewritePattern<ForOp> { struct SimplifyTrivialLoops : public OpRewritePattern<ForOp> {
using OpRewritePattern<ForOp>::OpRewritePattern; using OpRewritePattern<ForOp>::OpRewritePattern;
▲ Show 20 Lines • Show All 698 Lines • ▼ Show 20 Linesbool mlir::scf::insideMutuallyExclusiveBranches(Operation *a, Operation *b) {
} }
// Could not find a common IfOp among a's and b's ancestors. // Could not find a common IfOp among a's and b's ancestors.
return false; return false;
} }
void IfOp::build(OpBuilder &builder, OperationState &result, Value cond, void IfOp::build(OpBuilder &builder, OperationState &result, Value cond,
bool withElseRegion) { bool withElseRegion) {
build(builder, result, /*resultTypes=*/llvm::None, cond, withElseRegion); build(builder, result, /*resultTypes=*/std::nullopt, cond, withElseRegion);
} }
void IfOp::build(OpBuilder &builder, OperationState &result, void IfOp::build(OpBuilder &builder, OperationState &result,
TypeRange resultTypes, Value cond, bool withElseRegion) { TypeRange resultTypes, Value cond, bool withElseRegion) {
auto addTerminator = [&](OpBuilder &nested, Location loc) { auto addTerminator = [&](OpBuilder &nested, Location loc) {
if (resultTypes.empty()) if (resultTypes.empty())
IfOp::ensureTerminator(*nested.getInsertionBlock()->getParent(), nested, IfOp::ensureTerminator(*nested.getInsertionBlock()->getParent(), nested,
loc); loc);
▲ Show 20 Lines • Show All 1,127 Lines • ▼ Show 20 Lines LogicalResult matchAndRewrite(ParallelOp op,
auto newLowerBounds = auto newLowerBounds =
concatValues(op.getLowerBound(), innerOp.getLowerBound()); concatValues(op.getLowerBound(), innerOp.getLowerBound());
auto newUpperBounds = auto newUpperBounds =
concatValues(op.getUpperBound(), innerOp.getUpperBound()); concatValues(op.getUpperBound(), innerOp.getUpperBound());
auto newSteps = concatValues(op.getStep(), innerOp.getStep()); auto newSteps = concatValues(op.getStep(), innerOp.getStep());
rewriter.replaceOpWithNewOp<ParallelOp>(op, newLowerBounds, newUpperBounds, rewriter.replaceOpWithNewOp<ParallelOp>(op, newLowerBounds, newUpperBounds,
newSteps, llvm::None, bodyBuilder); newSteps, std::nullopt,
bodyBuilder);
return success(); return success();
} }
}; };
} // namespace } // namespace
void ParallelOp::getCanonicalizationPatterns(RewritePatternSet &results, void ParallelOp::getCanonicalizationPatterns(RewritePatternSet &results,
MLIRContext *context) { MLIRContext *context) {
▲ Show 20 Lines • Show All 967 Lines • Show Last 20 Lines

mlir/lib/Dialect/SCF/Transforms/StructuralTypeConversions.cpp

Show All 38 Lines
class Structural1ToNConversionPattern : public OpConversionPattern<SourceOp> { class Structural1ToNConversionPattern : public OpConversionPattern<SourceOp> {
public: public:
using OpConversionPattern<SourceOp>::typeConverter; using OpConversionPattern<SourceOp>::typeConverter;
using OpConversionPattern<SourceOp>::OpConversionPattern; using OpConversionPattern<SourceOp>::OpConversionPattern;
using OpAdaptor = typename OpConversionPattern<SourceOp>::OpAdaptor; using OpAdaptor = typename OpConversionPattern<SourceOp>::OpAdaptor;
// //
// Derived classes should provide the following method which performs the // Derived classes should provide the following method which performs the
// actual conversion. It should return llvm::None upon conversion failure and // actual conversion. It should return std::nullopt upon conversion failure
// return the converted operation upon success. // and return the converted operation upon success.
// //
// Optional<SourceOp> convertSourceOp(SourceOp op, OpAdaptor adaptor, // Optional<SourceOp> convertSourceOp(SourceOp op, OpAdaptor adaptor,
// ConversionPatternRewriter &rewriter, // ConversionPatternRewriter &rewriter,
// TypeRange dstTypes) const; // TypeRange dstTypes) const;
LogicalResult LogicalResult
matchAndRewrite(SourceOp op, OpAdaptor adaptor, matchAndRewrite(SourceOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const override { ConversionPatternRewriter &rewriter) const override {
▲ Show 20 Lines • Show All 65 Lines • ▼ Show 20 Lines Optional<ForOp> convertSourceOp(ForOp op, OpAdaptor adaptor,
// the dialect conversion framework thinks that we just inserted all the // the dialect conversion framework thinks that we just inserted all the
// cloned child ops. But what we want is to "take" the child regions and let // cloned child ops. But what we want is to "take" the child regions and let
// the dialect conversion framework continue recursively into ops inside // the dialect conversion framework continue recursively into ops inside
// those regions (which are already in its worklist; inlining them into the // those regions (which are already in its worklist; inlining them into the
// new op's regions doesn't remove the child ops from the worklist). // new op's regions doesn't remove the child ops from the worklist).
// convertRegionTypes already takes care of 1:N conversion. // convertRegionTypes already takes care of 1:N conversion.
if (failed(rewriter.convertRegionTypes(&op.getLoopBody(), *typeConverter))) if (failed(rewriter.convertRegionTypes(&op.getLoopBody(), *typeConverter)))
return llvm::None; return std::nullopt;
// Unpacked the iteration arguments. // Unpacked the iteration arguments.
SmallVector<Value> flatArgs; SmallVector<Value> flatArgs;
for (Value arg : adaptor.getInitArgs()) for (Value arg : adaptor.getInitArgs())
unpackUnrealizedConversionCast(arg, flatArgs); unpackUnrealizedConversionCast(arg, flatArgs);
// We can not do clone as the number of result types after conversion // We can not do clone as the number of result types after conversion
// might be different. // might be different.
▲ Show 20 Lines • Show All 57 Lines • ▼ Show 20 Lines Optional<WhileOp> convertSourceOp(WhileOp op, OpAdaptor adaptor,
SmallVector<Value> flatArgs; SmallVector<Value> flatArgs;
for (Value arg : adaptor.getOperands()) for (Value arg : adaptor.getOperands())
unpackUnrealizedConversionCast(arg, flatArgs); unpackUnrealizedConversionCast(arg, flatArgs);
auto newOp = rewriter.create<WhileOp>(op.getLoc(), dstTypes, flatArgs); auto newOp = rewriter.create<WhileOp>(op.getLoc(), dstTypes, flatArgs);
for (auto i : {0u, 1u}) { for (auto i : {0u, 1u}) {
if (failed(rewriter.convertRegionTypes(&op.getRegion(i), *typeConverter))) if (failed(rewriter.convertRegionTypes(&op.getRegion(i), *typeConverter)))
return llvm::None; return std::nullopt;
auto &dstRegion = newOp.getRegion(i); auto &dstRegion = newOp.getRegion(i);
rewriter.inlineRegionBefore(op.getRegion(i), dstRegion, dstRegion.end()); rewriter.inlineRegionBefore(op.getRegion(i), dstRegion, dstRegion.end());
} }
return newOp; return newOp;
} }
}; };
} // namespace } // namespace
▲ Show 20 Lines • Show All 56 Lines • Show Last 20 Lines

mlir/lib/Dialect/SPIRV/IR/SPIRVAttributes.cpp

Show First 20 Lines • Show All 150 Lines • ▼ Show 20 Lines
uint32_t spirv::InterfaceVarABIAttr::getDescriptorSet() { uint32_t spirv::InterfaceVarABIAttr::getDescriptorSet() {
return getImpl()->descriptorSet.cast<IntegerAttr>().getInt(); return getImpl()->descriptorSet.cast<IntegerAttr>().getInt();
} }
Optional<spirv::StorageClass> spirv::InterfaceVarABIAttr::getStorageClass() { Optional<spirv::StorageClass> spirv::InterfaceVarABIAttr::getStorageClass() {
if (getImpl()->storageClass) if (getImpl()->storageClass)
return static_cast<spirv::StorageClass>( return static_cast<spirv::StorageClass>(
getImpl()->storageClass.cast<IntegerAttr>().getValue().getZExtValue()); getImpl()->storageClass.cast<IntegerAttr>().getValue().getZExtValue());
return llvm::None; return std::nullopt;
} }
LogicalResult spirv::InterfaceVarABIAttr::verify( LogicalResult spirv::InterfaceVarABIAttr::verify(
function_ref<InFlightDiagnostic()> emitError, IntegerAttr descriptorSet, function_ref<InFlightDiagnostic()> emitError, IntegerAttr descriptorSet,
IntegerAttr binding, IntegerAttr storageClass) { IntegerAttr binding, IntegerAttr storageClass) {
if (!descriptorSet.getType().isSignlessInteger(32)) if (!descriptorSet.getType().isSignlessInteger(32))
return emitError() << "expected 32-bit integer for descriptor set"; return emitError() << "expected 32-bit integer for descriptor set";
▲ Show 20 Lines • Show All 524 Lines • Show Last 20 Lines

mlir/lib/Dialect/SPIRV/IR/SPIRVCanonicalization.cpp

Show All 24 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Common utility functions // Common utility functions
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
/// Returns the boolean value under the hood if the given `boolAttr` is a scalar /// Returns the boolean value under the hood if the given `boolAttr` is a scalar
/// or splat vector bool constant. /// or splat vector bool constant.
static Optional<bool> getScalarOrSplatBoolAttr(Attribute attr) { static Optional<bool> getScalarOrSplatBoolAttr(Attribute attr) {
if (!attr) if (!attr)
return llvm::None; return std::nullopt;
if (auto boolAttr = attr.dyn_cast<BoolAttr>()) if (auto boolAttr = attr.dyn_cast<BoolAttr>())
return boolAttr.getValue(); return boolAttr.getValue();
if (auto splatAttr = attr.dyn_cast<SplatElementsAttr>()) if (auto splatAttr = attr.dyn_cast<SplatElementsAttr>())
if (splatAttr.getElementType().isInteger(1)) if (splatAttr.getElementType().isInteger(1))
return splatAttr.getSplatValue<bool>(); return splatAttr.getSplatValue<bool>();
return llvm::None; return std::nullopt;
} }
// Extracts an element from the given `composite` by following the given // Extracts an element from the given `composite` by following the given
// `indices`. Returns a null Attribute if error happens. // `indices`. Returns a null Attribute if error happens.
static Attribute extractCompositeElement(Attribute composite, static Attribute extractCompositeElement(Attribute composite,
ArrayRef<unsigned> indices) { ArrayRef<unsigned> indices) {
// Check that given composite is a constant. // Check that given composite is a constant.
if (!composite) if (!composite)
▲ Show 20 Lines • Show All 400 Lines • Show Last 20 Lines

mlir/lib/Dialect/SPIRV/IR/SPIRVDialect.cpp

Show First 20 Lines • Show All 473 Lines • ▼ Show 20 Lines
// Specialize this function to parse each of the parameters that define an // Specialize this function to parse each of the parameters that define an
// ImageType. By default it assumes this is an enum type. // ImageType. By default it assumes this is an enum type.
template <typename ValTy> template <typename ValTy>
static Optional<ValTy> parseAndVerify(SPIRVDialect const &dialect, static Optional<ValTy> parseAndVerify(SPIRVDialect const &dialect,
DialectAsmParser &parser) { DialectAsmParser &parser) {
StringRef enumSpec; StringRef enumSpec;
SMLoc enumLoc = parser.getCurrentLocation(); SMLoc enumLoc = parser.getCurrentLocation();
if (parser.parseKeyword(&enumSpec)) { if (parser.parseKeyword(&enumSpec)) {
return llvm::None; return std::nullopt;
} }
auto val = spirv::symbolizeEnum<ValTy>(enumSpec); auto val = spirv::symbolizeEnum<ValTy>(enumSpec);
if (!val) if (!val)
parser.emitError(enumLoc, "unknown attribute: '") << enumSpec << "'"; parser.emitError(enumLoc, "unknown attribute: '") << enumSpec << "'";
return val; return val;
} }
template <> template <>
Optional<Type> parseAndVerify<Type>(SPIRVDialect const &dialect, Optional<Type> parseAndVerify<Type>(SPIRVDialect const &dialect,
DialectAsmParser &parser) { DialectAsmParser &parser) {
// TODO: Further verify that the element type can be sampled // TODO: Further verify that the element type can be sampled
auto ty = parseAndVerifyType(dialect, parser); auto ty = parseAndVerifyType(dialect, parser);
if (!ty) if (!ty)
return llvm::None; return std::nullopt;
return ty; return ty;
} }
template <typename IntTy> template <typename IntTy>
static Optional<IntTy> parseAndVerifyInteger(SPIRVDialect const &dialect, static Optional<IntTy> parseAndVerifyInteger(SPIRVDialect const &dialect,
DialectAsmParser &parser) { DialectAsmParser &parser) {
IntTy offsetVal = std::numeric_limits<IntTy>::max(); IntTy offsetVal = std::numeric_limits<IntTy>::max();
if (parser.parseInteger(offsetVal)) if (parser.parseInteger(offsetVal))
return llvm::None; return std::nullopt;
return offsetVal; return offsetVal;
} }
template <> template <>
Optional<unsigned> parseAndVerify<unsigned>(SPIRVDialect const &dialect, Optional<unsigned> parseAndVerify<unsigned>(SPIRVDialect const &dialect,
DialectAsmParser &parser) { DialectAsmParser &parser) {
return parseAndVerifyInteger<unsigned>(dialect, parser); return parseAndVerifyInteger<unsigned>(dialect, parser);
} }
namespace { namespace {
// Functor object to parse a comma separated list of specs. The function // Functor object to parse a comma separated list of specs. The function
// parseAndVerify does the actual parsing and verification of individual // parseAndVerify does the actual parsing and verification of individual
// elements. This is a functor since parsing the last element of the list // elements. This is a functor since parsing the last element of the list
// (termination condition) needs partial specialization. // (termination condition) needs partial specialization.
template <typename ParseType, typename... Args> template <typename ParseType, typename... Args>
struct ParseCommaSeparatedList { struct ParseCommaSeparatedList {
Optional<std::tuple<ParseType, Args...>> Optional<std::tuple<ParseType, Args...>>
operator()(SPIRVDialect const &dialect, DialectAsmParser &parser) const { operator()(SPIRVDialect const &dialect, DialectAsmParser &parser) const {
auto parseVal = parseAndVerify<ParseType>(dialect, parser); auto parseVal = parseAndVerify<ParseType>(dialect, parser);
if (!parseVal) if (!parseVal)
return llvm::None; return std::nullopt;
auto numArgs = std::tuple_size<std::tuple<Args...>>::value; auto numArgs = std::tuple_size<std::tuple<Args...>>::value;
if (numArgs != 0 && failed(parser.parseComma())) if (numArgs != 0 && failed(parser.parseComma()))
return llvm::None; return std::nullopt;
auto remainingValues = ParseCommaSeparatedList<Args...>{}(dialect, parser); auto remainingValues = ParseCommaSeparatedList<Args...>{}(dialect, parser);
if (!remainingValues) if (!remainingValues)
return llvm::None; return std::nullopt;
return std::tuple_cat(std::tuple<ParseType>(parseVal.value()), return std::tuple_cat(std::tuple<ParseType>(parseVal.value()),
remainingValues.value()); remainingValues.value());
} }
}; };
// Partial specialization of the function to parse a comma separated list of // Partial specialization of the function to parse a comma separated list of
// specs to parse the last element of the list. // specs to parse the last element of the list.
template <typename ParseType> template <typename ParseType>
struct ParseCommaSeparatedList<ParseType> { struct ParseCommaSeparatedList<ParseType> {
Optional<std::tuple<ParseType>> operator()(SPIRVDialect const &dialect, Optional<std::tuple<ParseType>> operator()(SPIRVDialect const &dialect,
DialectAsmParser &parser) const { DialectAsmParser &parser) const {
if (auto value = parseAndVerify<ParseType>(dialect, parser)) if (auto value = parseAndVerify<ParseType>(dialect, parser))
return std::tuple<ParseType>(*value); return std::tuple<ParseType>(*value);
return llvm::None; return std::nullopt;
} }
}; };
} // namespace } // namespace
// dim ::= `1D` | `2D` | `3D` | `Cube` | <and other SPIR-V Dim specifiers...> // dim ::= `1D` | `2D` | `3D` | `Cube` | <and other SPIR-V Dim specifiers...>
// //
// depth-info ::= `NoDepth` | `IsDepth` | `DepthUnknown` // depth-info ::= `NoDepth` | `IsDepth` | `DepthUnknown`
// //
▲ Show 20 Lines • Show All 438 Lines • Show Last 20 Lines

mlir/lib/Dialect/SPIRV/IR/SPIRVTypes.cpp

Show First 20 Lines • Show All 74 Lines • ▼ Show 20 Linesvoid ArrayType::getCapabilities(
Optional<StorageClass> storage) { Optional<StorageClass> storage) {
getElementType().cast<SPIRVType>().getCapabilities(capabilities, storage); getElementType().cast<SPIRVType>().getCapabilities(capabilities, storage);
} }
Optional<int64_t> ArrayType::getSizeInBytes() { Optional<int64_t> ArrayType::getSizeInBytes() {
auto elementType = getElementType().cast<SPIRVType>(); auto elementType = getElementType().cast<SPIRVType>();
Optional<int64_t> size = elementType.getSizeInBytes(); Optional<int64_t> size = elementType.getSizeInBytes();
if (!size) if (!size)
return llvm::None; return std::nullopt;
return (*size + getArrayStride()) * getNumElements(); return (*size + getArrayStride()) * getNumElements();
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// CompositeType // CompositeType
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
bool CompositeType::classof(Type type) { bool CompositeType::classof(Type type) {
▲ Show 20 Lines • Show All 97 Lines • ▼ Show 20 LinesOptional<int64_t> CompositeType::getSizeInBytes() {
if (auto arrayType = dyn_cast<ArrayType>()) if (auto arrayType = dyn_cast<ArrayType>())
return arrayType.getSizeInBytes(); return arrayType.getSizeInBytes();
if (auto structType = dyn_cast<StructType>()) if (auto structType = dyn_cast<StructType>())
return structType.getSizeInBytes(); return structType.getSizeInBytes();
if (auto vectorType = dyn_cast<VectorType>()) { if (auto vectorType = dyn_cast<VectorType>()) {
Optional<int64_t> elementSize = Optional<int64_t> elementSize =
vectorType.getElementType().cast<ScalarType>().getSizeInBytes(); vectorType.getElementType().cast<ScalarType>().getSizeInBytes();
if (!elementSize) if (!elementSize)
return llvm::None; return std::nullopt;
return *elementSize * vectorType.getNumElements(); return *elementSize * vectorType.getNumElements();
} }
return llvm::None; return std::nullopt;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// CooperativeMatrixType // CooperativeMatrixType
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
struct spirv::detail::CooperativeMatrixTypeStorage : public TypeStorage { struct spirv::detail::CooperativeMatrixTypeStorage : public TypeStorage {
using KeyTy = std::tuple<Type, Scope, unsigned, unsigned>; using KeyTy = std::tuple<Type, Scope, unsigned, unsigned>;
▲ Show 20 Lines • Show All 502 Lines • ▼ Show 20 LinesOptional<int64_t> ScalarType::getSizeInBytes() {
auto bitWidth = getIntOrFloatBitWidth(); auto bitWidth = getIntOrFloatBitWidth();
// According to the SPIR-V spec: // According to the SPIR-V spec:
// "There is no physical size or bit pattern defined for values with boolean // "There is no physical size or bit pattern defined for values with boolean
// type. If they are stored (in conjunction with OpVariable), they can only // type. If they are stored (in conjunction with OpVariable), they can only
// be used with logical addressing operations, not physical, and only with // be used with logical addressing operations, not physical, and only with
// non-externally visible shader Storage Classes: Workgroup, CrossWorkgroup, // non-externally visible shader Storage Classes: Workgroup, CrossWorkgroup,
// Private, Function, Input, and Output." // Private, Function, Input, and Output."
if (bitWidth == 1) if (bitWidth == 1)
return llvm::None; return std::nullopt;
return bitWidth / 8; return bitWidth / 8;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// SPIRVType // SPIRVType
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
bool SPIRVType::classof(Type type) { bool SPIRVType::classof(Type type) {
▲ Show 20 Lines • Show All 50 Lines • ▼ Show 20 Linesvoid SPIRVType::getCapabilities(
} }
} }
Optional<int64_t> SPIRVType::getSizeInBytes() { Optional<int64_t> SPIRVType::getSizeInBytes() {
if (auto scalarType = dyn_cast<ScalarType>()) if (auto scalarType = dyn_cast<ScalarType>())
return scalarType.getSizeInBytes(); return scalarType.getSizeInBytes();
if (auto compositeType = dyn_cast<CompositeType>()) if (auto compositeType = dyn_cast<CompositeType>())
return compositeType.getSizeInBytes(); return compositeType.getSizeInBytes();
return llvm::None; return std::nullopt;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// SampledImageType // SampledImageType
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
struct spirv::detail::SampledImageTypeStorage : public TypeStorage { struct spirv::detail::SampledImageTypeStorage : public TypeStorage {
using KeyTy = Type; using KeyTy = Type;
▲ Show 20 Lines • Show All 460 Lines • Show Last 20 Lines

mlir/lib/Dialect/SPIRV/IR/TargetAndABI.cpp

Show First 20 Lines • Show All 48 Lines • ▼ Show 20 Lines
Optional<spirv::Capability> Optional<spirv::Capability>
spirv::TargetEnv::allows(ArrayRef<spirv::Capability> caps) const { spirv::TargetEnv::allows(ArrayRef<spirv::Capability> caps) const {
const auto *chosen = llvm::find_if(caps, [this](spirv::Capability cap) { const auto *chosen = llvm::find_if(caps, [this](spirv::Capability cap) {
return givenCapabilities.count(cap); return givenCapabilities.count(cap);
}); });
if (chosen != caps.end()) if (chosen != caps.end())
return *chosen; return *chosen;
return llvm::None; return std::nullopt;
} }
bool spirv::TargetEnv::allows(spirv::Extension extension) const { bool spirv::TargetEnv::allows(spirv::Extension extension) const {
return givenExtensions.count(extension); return givenExtensions.count(extension);
} }
Optional<spirv::Extension> Optional<spirv::Extension>
spirv::TargetEnv::allows(ArrayRef<spirv::Extension> exts) const { spirv::TargetEnv::allows(ArrayRef<spirv::Extension> exts) const {
const auto *chosen = llvm::find_if(exts, [this](spirv::Extension ext) { const auto *chosen = llvm::find_if(exts, [this](spirv::Extension ext) {
return givenExtensions.count(ext); return givenExtensions.count(ext);
}); });
if (chosen != exts.end()) if (chosen != exts.end())
return *chosen; return *chosen;
return llvm::None; return std::nullopt;
} }
spirv::Vendor spirv::TargetEnv::getVendorID() const { spirv::Vendor spirv::TargetEnv::getVendorID() const {
return targetAttr.getVendorID(); return targetAttr.getVendorID();
} }
spirv::DeviceType spirv::TargetEnv::getDeviceType() const { spirv::DeviceType spirv::TargetEnv::getDeviceType() const {
return targetAttr.getDeviceType(); return targetAttr.getDeviceType();
▲ Show 20 Lines • Show All 161 Lines • Show Last 20 Lines

mlir/lib/Dialect/SPIRV/Transforms/LowerABIAttributesPass.cpp

Show First 20 Lines • Show All 229 Lines • ▼ Show 20 LinesLogicalResult ProcessInterfaceVarABI::matchAndRewrite(
} }
if (failed(rewriter.convertRegionTypes(&funcOp.getBody(), *getTypeConverter(), if (failed(rewriter.convertRegionTypes(&funcOp.getBody(), *getTypeConverter(),
&signatureConverter))) &signatureConverter)))
return failure(); return failure();
// Creates a new function with the update signature. // Creates a new function with the update signature.
rewriter.updateRootInPlace(funcOp, [&] { rewriter.updateRootInPlace(funcOp, [&] {
funcOp.setType(rewriter.getFunctionType( funcOp.setType(rewriter.getFunctionType(
signatureConverter.getConvertedTypes(), llvm::None)); signatureConverter.getConvertedTypes(), std::nullopt));
}); });
return success(); return success();
} }
void LowerABIAttributesPass::runOnOperation() { void LowerABIAttributesPass::runOnOperation() {
// Uses the signature conversion methodology of the dialect conversion // Uses the signature conversion methodology of the dialect conversion
// framework to implement the conversion. // framework to implement the conversion.
spirv::ModuleOp module = getOperation(); spirv::ModuleOp module = getOperation();
▲ Show 20 Lines • Show All 56 Lines • Show Last 20 Lines

mlir/lib/Dialect/SPIRV/Transforms/SPIRVConversion.cpp

Show First 20 Lines • Show All 134 Lines • ▼ Show 20 Lines if (type.isa<spirv::ScalarType>()) {
auto bitWidth = type.getIntOrFloatBitWidth(); auto bitWidth = type.getIntOrFloatBitWidth();
// According to the SPIR-V spec: // According to the SPIR-V spec:
// "There is no physical size or bit pattern defined for values with boolean // "There is no physical size or bit pattern defined for values with boolean
// type. If they are stored (in conjunction with OpVariable), they can only // type. If they are stored (in conjunction with OpVariable), they can only
// be used with logical addressing operations, not physical, and only with // be used with logical addressing operations, not physical, and only with
// non-externally visible shader Storage Classes: Workgroup, CrossWorkgroup, // non-externally visible shader Storage Classes: Workgroup, CrossWorkgroup,
// Private, Function, Input, and Output." // Private, Function, Input, and Output."
if (bitWidth == 1) if (bitWidth == 1)
return llvm::None; return std::nullopt;
return bitWidth / 8; return bitWidth / 8;
} }
if (auto vecType = type.dyn_cast<VectorType>()) { if (auto vecType = type.dyn_cast<VectorType>()) {
auto elementSize = getTypeNumBytes(options, vecType.getElementType()); auto elementSize = getTypeNumBytes(options, vecType.getElementType());
if (!elementSize) if (!elementSize)
return llvm::None; return std::nullopt;
return vecType.getNumElements() * *elementSize; return vecType.getNumElements() * *elementSize;
} }
if (auto memRefType = type.dyn_cast<MemRefType>()) { if (auto memRefType = type.dyn_cast<MemRefType>()) {
// TODO: Layout should also be controlled by the ABI attributes. For now // TODO: Layout should also be controlled by the ABI attributes. For now
// using the layout from MemRef. // using the layout from MemRef.
int64_t offset; int64_t offset;
SmallVector<int64_t, 4> strides; SmallVector<int64_t, 4> strides;
if (!memRefType.hasStaticShape() || if (!memRefType.hasStaticShape() ||
failed(getStridesAndOffset(memRefType, strides, offset))) failed(getStridesAndOffset(memRefType, strides, offset)))
return llvm::None; return std::nullopt;
// To get the size of the memref object in memory, the total size is the // To get the size of the memref object in memory, the total size is the
// max(stride * dimension-size) computed for all dimensions times the size // max(stride * dimension-size) computed for all dimensions times the size
// of the element. // of the element.
auto elementSize = getTypeNumBytes(options, memRefType.getElementType()); auto elementSize = getTypeNumBytes(options, memRefType.getElementType());
if (!elementSize) if (!elementSize)
return llvm::None; return std::nullopt;
if (memRefType.getRank() == 0) if (memRefType.getRank() == 0)
return elementSize; return elementSize;
auto dims = memRefType.getShape(); auto dims = memRefType.getShape();
if (llvm::is_contained(dims, ShapedType::kDynamic) || if (llvm::is_contained(dims, ShapedType::kDynamic) ||
ShapedType::isDynamic(offset) || ShapedType::isDynamic(offset) ||
llvm::is_contained(strides, ShapedType::kDynamic)) llvm::is_contained(strides, ShapedType::kDynamic))
return llvm::None; return std::nullopt;
int64_t memrefSize = -1; int64_t memrefSize = -1;
for (const auto &shape : enumerate(dims)) for (const auto &shape : enumerate(dims))
memrefSize = std::max(memrefSize, shape.value() * strides[shape.index()]); memrefSize = std::max(memrefSize, shape.value() * strides[shape.index()]);
return (offset + memrefSize) * *elementSize; return (offset + memrefSize) * *elementSize;
} }
if (auto tensorType = type.dyn_cast<TensorType>()) { if (auto tensorType = type.dyn_cast<TensorType>()) {
if (!tensorType.hasStaticShape()) if (!tensorType.hasStaticShape())
return llvm::None; return std::nullopt;
auto elementSize = getTypeNumBytes(options, tensorType.getElementType()); auto elementSize = getTypeNumBytes(options, tensorType.getElementType());
if (!elementSize) if (!elementSize)
return llvm::None; return std::nullopt;
int64_t size = *elementSize; int64_t size = *elementSize;
for (auto shape : tensorType.getShape()) for (auto shape : tensorType.getShape())
size *= shape; size *= shape;
return size; return size;
} }
// TODO: Add size computation for other types. // TODO: Add size computation for other types.
return llvm::None; return std::nullopt;
} }
/// Converts a scalar `type` to a suitable type under the given `targetEnv`. /// Converts a scalar `type` to a suitable type under the given `targetEnv`.
static Type convertScalarType(const spirv::TargetEnv &targetEnv, static Type convertScalarType(const spirv::TargetEnv &targetEnv,
const SPIRVConversionOptions &options, const SPIRVConversionOptions &options,
spirv::ScalarType type, spirv::ScalarType type,
Optional<spirv::StorageClass> storageClass = {}) { Optional<spirv::StorageClass> storageClass = {}) {
// Get extension and capability requirements for the given type. // Get extension and capability requirements for the given type.
▲ Show 20 Lines • Show All 705 Lines • Show Last 20 Lines

mlir/lib/Dialect/SPIRV/Transforms/UnifyAliasedResourcePass.cpp

Show First 20 Lines • Show All 76 Lines • ▼ Show 20 Lines auto rtArrayType =
structType.getElementType(0).dyn_cast<spirv::RuntimeArrayType>(); structType.getElementType(0).dyn_cast<spirv::RuntimeArrayType>();
if (!rtArrayType) if (!rtArrayType)
return {}; return {};
return rtArrayType.getElementType(); return rtArrayType.getElementType();
} }
/// Given a list of resource element `types`, returns the index of the canonical /// Given a list of resource element `types`, returns the index of the canonical
/// resource that all resources should be unified into. Returns llvm::None if /// resource that all resources should be unified into. Returns std::nullopt if
/// unable to unify. /// unable to unify.
static Optional<int> deduceCanonicalResource(ArrayRef<spirv::SPIRVType> types) { static Optional<int> deduceCanonicalResource(ArrayRef<spirv::SPIRVType> types) {
// scalarNumBits: contains all resources' scalar types' bit counts. // scalarNumBits: contains all resources' scalar types' bit counts.
// vectorNumBits: only contains resources whose element types are vectors. // vectorNumBits: only contains resources whose element types are vectors.
// vectorIndices: each vector's original index in `types`. // vectorIndices: each vector's original index in `types`.
SmallVector<int> scalarNumBits, vectorNumBits, vectorIndices; SmallVector<int> scalarNumBits, vectorNumBits, vectorIndices;
scalarNumBits.reserve(types.size()); scalarNumBits.reserve(types.size());
vectorNumBits.reserve(types.size()); vectorNumBits.reserve(types.size());
vectorIndices.reserve(types.size()); vectorIndices.reserve(types.size());
for (const auto &indexedTypes : llvm::enumerate(types)) { for (const auto &indexedTypes : llvm::enumerate(types)) {
spirv::SPIRVType type = indexedTypes.value(); spirv::SPIRVType type = indexedTypes.value();
assert(type.isScalarOrVector()); assert(type.isScalarOrVector());
if (auto vectorType = type.dyn_cast<VectorType>()) { if (auto vectorType = type.dyn_cast<VectorType>()) {
if (vectorType.getNumElements() % 2 != 0) if (vectorType.getNumElements() % 2 != 0)
return llvm::None; // Odd-sized vector has special layout requirements. return std::nullopt; // Odd-sized vector has special layout
// requirements.
Optional<int64_t> numBytes = type.getSizeInBytes(); Optional<int64_t> numBytes = type.getSizeInBytes();
if (!numBytes) if (!numBytes)
return llvm::None; return std::nullopt;
scalarNumBits.push_back( scalarNumBits.push_back(
vectorType.getElementType().getIntOrFloatBitWidth()); vectorType.getElementType().getIntOrFloatBitWidth());
vectorNumBits.push_back(*numBytes * 8); vectorNumBits.push_back(*numBytes * 8);
vectorIndices.push_back(indexedTypes.index()); vectorIndices.push_back(indexedTypes.index());
} else { } else {
scalarNumBits.push_back(type.getIntOrFloatBitWidth()); scalarNumBits.push_back(type.getIntOrFloatBitWidth());
} }
} }
if (!vectorNumBits.empty()) { if (!vectorNumBits.empty()) {
// Choose the *vector* with the smallest bitwidth as the canonical resource, // Choose the *vector* with the smallest bitwidth as the canonical resource,
// so that we can still keep vectorized load/store and avoid partial updates // so that we can still keep vectorized load/store and avoid partial updates
// to large vectors. // to large vectors.
auto *minVal = std::min_element(vectorNumBits.begin(), vectorNumBits.end()); auto *minVal = std::min_element(vectorNumBits.begin(), vectorNumBits.end());
// Make sure that the canonical resource's bitwidth is divisible by others. // Make sure that the canonical resource's bitwidth is divisible by others.
// With out this, we cannot properly adjust the index later. // With out this, we cannot properly adjust the index later.
if (llvm::any_of(vectorNumBits, if (llvm::any_of(vectorNumBits,
[&](int bits) { return bits % *minVal != 0; })) [&](int bits) { return bits % *minVal != 0; }))
return llvm::None; return std::nullopt;
// Require all scalar type bit counts to be a multiple of the chosen // Require all scalar type bit counts to be a multiple of the chosen
// vector's primitive type to avoid reading/writing subcomponents. // vector's primitive type to avoid reading/writing subcomponents.
int index = vectorIndices[std::distance(vectorNumBits.begin(), minVal)]; int index = vectorIndices[std::distance(vectorNumBits.begin(), minVal)];
int baseNumBits = scalarNumBits[index]; int baseNumBits = scalarNumBits[index];
if (llvm::any_of(scalarNumBits, if (llvm::any_of(scalarNumBits,
[&](int bits) { return bits % baseNumBits != 0; })) [&](int bits) { return bits % baseNumBits != 0; }))
return llvm::None; return std::nullopt;
return index; return index;
} }
// All element types are scalars. Then choose the smallest bitwidth as the // All element types are scalars. Then choose the smallest bitwidth as the
// cannonical resource to avoid subcomponent load/store. // cannonical resource to avoid subcomponent load/store.
auto *minVal = std::min_element(scalarNumBits.begin(), scalarNumBits.end()); auto *minVal = std::min_element(scalarNumBits.begin(), scalarNumBits.end());
if (llvm::any_of(scalarNumBits, if (llvm::any_of(scalarNumBits,
[minVal](int64_t bit) { return bit % *minVal != 0; })) [minVal](int64_t bit) { return bit % *minVal != 0; }))
return llvm::None; return std::nullopt;
return std::distance(scalarNumBits.begin(), minVal); return std::distance(scalarNumBits.begin(), minVal);
} }
static bool areSameBitwidthScalarType(Type a, Type b) { static bool areSameBitwidthScalarType(Type a, Type b) {
return a.isIntOrFloat() && b.isIntOrFloat() && return a.isIntOrFloat() && b.isIntOrFloat() &&
a.getIntOrFloatBitWidth() == b.getIntOrFloatBitWidth(); a.getIntOrFloatBitWidth() == b.getIntOrFloatBitWidth();
} }
▲ Show 20 Lines • Show All 446 Lines • Show Last 20 Lines

mlir/lib/Dialect/Shape/IR/Shape.cpp

Show First 20 Lines • Show All 1,067 Lines • ▼ Show 20 Lines
// DimOp // DimOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
Optional<int64_t> DimOp::getConstantIndex() { Optional<int64_t> DimOp::getConstantIndex() {
if (auto constSizeOp = getIndex().getDefiningOp<ConstSizeOp>()) if (auto constSizeOp = getIndex().getDefiningOp<ConstSizeOp>())
return constSizeOp.getValue().getLimitedValue(); return constSizeOp.getValue().getLimitedValue();
if (auto constantOp = getIndex().getDefiningOp<arith::ConstantOp>()) if (auto constantOp = getIndex().getDefiningOp<arith::ConstantOp>())
return constantOp.getValue().cast<IntegerAttr>().getInt(); return constantOp.getValue().cast<IntegerAttr>().getInt();
return llvm::None; return std::nullopt;
} }
OpFoldResult DimOp::fold(ArrayRef<Attribute> operands) { OpFoldResult DimOp::fold(ArrayRef<Attribute> operands) {
Type valType = getValue().getType(); Type valType = getValue().getType();
auto valShapedType = valType.dyn_cast<ShapedType>(); auto valShapedType = valType.dyn_cast<ShapedType>();
if (!valShapedType || !valShapedType.hasRank()) if (!valShapedType || !valShapedType.hasRank())
return nullptr; return nullptr;
Optional<int64_t> index = getConstantIndex(); Optional<int64_t> index = getConstantIndex();
▲ Show 20 Lines • Show All 211 Lines • ▼ Show 20 Lines state.addAttribute(FuncOp::getFunctionTypeAttrName(state.name),
TypeAttr::get(type)); TypeAttr::get(type));
state.attributes.append(attrs.begin(), attrs.end()); state.attributes.append(attrs.begin(), attrs.end());
state.addRegion(); state.addRegion();
if (argAttrs.empty()) if (argAttrs.empty())
return; return;
assert(type.getNumInputs() == argAttrs.size()); assert(type.getNumInputs() == argAttrs.size());
function_interface_impl::addArgAndResultAttrs(builder, state, argAttrs, function_interface_impl::addArgAndResultAttrs(builder, state, argAttrs,
/*resultAttrs=*/llvm::None); /*resultAttrs=*/std::nullopt);
} }
ParseResult FuncOp::parse(OpAsmParser &parser, OperationState &result) { ParseResult FuncOp::parse(OpAsmParser &parser, OperationState &result) {
auto buildFuncType = auto buildFuncType =
[](Builder &builder, ArrayRef<Type> argTypes, ArrayRef<Type> results, [](Builder &builder, ArrayRef<Type> argTypes, ArrayRef<Type> results,
function_interface_impl::VariadicFlag, function_interface_impl::VariadicFlag,
std::string &) { return builder.getFunctionType(argTypes, results); }; std::string &) { return builder.getFunctionType(argTypes, results); };
Show All 9 Lines
// GetExtentOp // GetExtentOp
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
Optional<int64_t> GetExtentOp::getConstantDim() { Optional<int64_t> GetExtentOp::getConstantDim() {
if (auto constSizeOp = getDim().getDefiningOp<ConstSizeOp>()) if (auto constSizeOp = getDim().getDefiningOp<ConstSizeOp>())
return constSizeOp.getValue().getLimitedValue(); return constSizeOp.getValue().getLimitedValue();
if (auto constantOp = getDim().getDefiningOp<arith::ConstantOp>()) if (auto constantOp = getDim().getDefiningOp<arith::ConstantOp>())
return constantOp.getValue().cast<IntegerAttr>().getInt(); return constantOp.getValue().cast<IntegerAttr>().getInt();
return llvm::None; return std::nullopt;
} }
OpFoldResult GetExtentOp::fold(ArrayRef<Attribute> operands) { OpFoldResult GetExtentOp::fold(ArrayRef<Attribute> operands) {
auto elements = operands[0].dyn_cast_or_null<DenseIntElementsAttr>(); auto elements = operands[0].dyn_cast_or_null<DenseIntElementsAttr>();
if (!elements) if (!elements)
return nullptr; return nullptr;
Optional<int64_t> dim = getConstantDim(); Optional<int64_t> dim = getConstantDim();
if (!dim.has_value()) if (!dim.has_value())
▲ Show 20 Lines • Show All 667 Lines • Show Last 20 Lines

mlir/lib/Dialect/SparseTensor/IR/SparseTensorDialect.cpp

Show First 20 Lines • Show All 587 Lines • ▼ Show 20 Lines if (ttp.getRank() != 1 + static_cast<int64_t>(getIndices().size()))
return emitOpError("incorrect number of indices"); return emitOpError("incorrect number of indices");
return success(); return success();
} }
void ForeachOp::build( void ForeachOp::build(
OpBuilder &builder, OperationState &result, Value tensor, OpBuilder &builder, OperationState &result, Value tensor,
function_ref<void(OpBuilder &, Location, ValueRange, Value, ValueRange)> function_ref<void(OpBuilder &, Location, ValueRange, Value, ValueRange)>
bodyBuilder) { bodyBuilder) {
build(builder, result, tensor, llvm::None, bodyBuilder); build(builder, result, tensor, std::nullopt, bodyBuilder);
} }
void ForeachOp::build( void ForeachOp::build(
OpBuilder &builder, OperationState &result, Value tensor, OpBuilder &builder, OperationState &result, Value tensor,
ValueRange initArgs, ValueRange initArgs,
function_ref<void(OpBuilder &, Location, ValueRange, Value, ValueRange)> function_ref<void(OpBuilder &, Location, ValueRange, Value, ValueRange)>
bodyBuilder) { bodyBuilder) {
build(builder, result, initArgs.getTypes(), tensor, initArgs); build(builder, result, initArgs.getTypes(), tensor, initArgs);
▲ Show 20 Lines • Show All 194 Lines • Show Last 20 Lines

mlir/lib/Dialect/SparseTensor/Transforms/SparseTensorCodegen.cpp

Show First 20 Lines • Show All 111 Lines • ▼ Show 20 Lines
/// Gets the dimension size for the given sparse tensor at the given /// Gets the dimension size for the given sparse tensor at the given
/// original dimension 'dim'. Returns None if no sparse encoding is /// original dimension 'dim'. Returns None if no sparse encoding is
/// attached to the given tensor type. /// attached to the given tensor type.
static Optional<Value> sizeFromTensorAtDim(OpBuilder &builder, Location loc, static Optional<Value> sizeFromTensorAtDim(OpBuilder &builder, Location loc,
RankedTensorType tensorTp, RankedTensorType tensorTp,
Value adaptedValue, unsigned dim) { Value adaptedValue, unsigned dim) {
auto enc = getSparseTensorEncoding(tensorTp); auto enc = getSparseTensorEncoding(tensorTp);
if (!enc) if (!enc)
return llvm::None; return std::nullopt;
// Access into static dimension can query original type directly. // Access into static dimension can query original type directly.
// Note that this is typically already done by DimOp's folding. // Note that this is typically already done by DimOp's folding.
auto shape = tensorTp.getShape(); auto shape = tensorTp.getShape();
if (!ShapedType::isDynamic(shape[dim])) if (!ShapedType::isDynamic(shape[dim]))
return constantIndex(builder, loc, shape[dim]); return constantIndex(builder, loc, shape[dim]);
// Any other query can consult the dimSizes array at field DimSizesIdx, // Any other query can consult the dimSizes array at field DimSizesIdx,
▲ Show 20 Lines • Show All 61 Lines • ▼ Show 20 Linesstatic unsigned getFieldIndex(Type type, unsigned ptrDim, unsigned idxDim) {
return field + 1; // return values field index return field + 1; // return values field index
} }
/// Maps a sparse tensor type to the appropriate compounded buffers. /// Maps a sparse tensor type to the appropriate compounded buffers.
static Optional<LogicalResult> static Optional<LogicalResult>
convertSparseTensorType(Type type, SmallVectorImpl<Type> &fields) { convertSparseTensorType(Type type, SmallVectorImpl<Type> &fields) {
auto enc = getSparseTensorEncoding(type); auto enc = getSparseTensorEncoding(type);
if (!enc) if (!enc)
return llvm::None; return std::nullopt;
// Construct the basic types. // Construct the basic types.
auto *context = type.getContext(); auto *context = type.getContext();
unsigned idxWidth = enc.getIndexBitWidth(); unsigned idxWidth = enc.getIndexBitWidth();
unsigned ptrWidth = enc.getPointerBitWidth(); unsigned ptrWidth = enc.getPointerBitWidth();
RankedTensorType rType = type.cast<RankedTensorType>(); RankedTensorType rType = type.cast<RankedTensorType>();
Type indexType = IndexType::get(context); Type indexType = IndexType::get(context);
Type idxType = idxWidth ? IntegerType::get(context, idxWidth) : indexType; Type idxType = idxWidth ? IntegerType::get(context, idxWidth) : indexType;
Type ptrType = ptrWidth ? IntegerType::get(context, ptrWidth) : indexType; Type ptrType = ptrWidth ? IntegerType::get(context, ptrWidth) : indexType;
▲ Show 20 Lines • Show All 808 Lines • ▼ Show 20 Linesmlir::SparseTensorTypeToBufferConverter::SparseTensorTypeToBufferConverter() {
addConversion(convertSparseTensorType); addConversion(convertSparseTensorType);
// Required by scf.for 1:N type conversion. // Required by scf.for 1:N type conversion.
addSourceMaterialization([](OpBuilder &builder, RankedTensorType tp, addSourceMaterialization([](OpBuilder &builder, RankedTensorType tp,
ValueRange inputs, ValueRange inputs,
Location loc) -> Optional<Value> { Location loc) -> Optional<Value> {
if (!getSparseTensorEncoding(tp)) if (!getSparseTensorEncoding(tp))
// Not a sparse tensor. // Not a sparse tensor.
return llvm::None; return std::nullopt;
// Sparse compiler knows how to cancel out these casts. // Sparse compiler knows how to cancel out these casts.
return genTuple(builder, loc, tp, inputs); return genTuple(builder, loc, tp, inputs);
}); });
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Public method for populating conversion rules. // Public method for populating conversion rules.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
Show All 17 Lines

mlir/lib/Dialect/SparseTensor/Transforms/SparseTensorConversion.cpp

Show All 36 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Helper methods. // Helper methods.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
/// Maps each sparse tensor type to an opaque pointer. /// Maps each sparse tensor type to an opaque pointer.
static Optional<Type> convertSparseTensorTypes(Type type) { static Optional<Type> convertSparseTensorTypes(Type type) {
if (getSparseTensorEncoding(type) != nullptr) if (getSparseTensorEncoding(type) != nullptr)
return LLVM::LLVMPointerType::get(IntegerType::get(type.getContext(), 8)); return LLVM::LLVMPointerType::get(IntegerType::get(type.getContext(), 8));
return llvm::None; return std::nullopt;
} }
/// Replaces the `op` with a `CallOp` to the function reference returned /// Replaces the `op` with a `CallOp` to the function reference returned
/// by `getFunc()`. /// by `getFunc()`.
static func::CallOp replaceOpWithFuncCall(RewriterBase &rewriter, Operation *op, static func::CallOp replaceOpWithFuncCall(RewriterBase &rewriter, Operation *op,
StringRef name, TypeRange resultType, StringRef name, TypeRange resultType,
ValueRange operands, ValueRange operands,
EmitCInterface emitCInterface) { EmitCInterface emitCInterface) {
▲ Show 20 Lines • Show All 1,330 Lines • Show Last 20 Lines

mlir/lib/Dialect/SparseTensor/Transforms/SparseTensorRewriting.cpp

Show First 20 Lines • Show All 676 Lines • ▼ Show 20 Lines LogicalResult sparse2DenseRewrite(ConvertOp op,
SmallVector<Value> sizes; SmallVector<Value> sizes;
sizesForTensor(rewriter, sizes, loc, srcTp, src); sizesForTensor(rewriter, sizes, loc, srcTp, src);
Value dst = allocDenseTensor(rewriter, loc, dstTp, sizes); Value dst = allocDenseTensor(rewriter, loc, dstTp, sizes);
Block *insertionBlock = rewriter.getInsertionBlock(); Block *insertionBlock = rewriter.getInsertionBlock();
bool noEscape = bufferization::allocationDoesNotEscape(op->getOpResult(0)); bool noEscape = bufferization::allocationDoesNotEscape(op->getOpResult(0));
rewriter.create<ForeachOp>(loc, src, llvm::None, rewriter.create<ForeachOp>(loc, src, std::nullopt,
[&](OpBuilder &builder, Location loc, [&](OpBuilder &builder, Location loc,
ValueRange args, Value v, ValueRange reduc) { ValueRange args, Value v, ValueRange reduc) {
builder.create<memref::StoreOp>(loc, v, dst, builder.create<memref::StoreOp>(loc, v, dst,
args); args);
builder.create<sparse_tensor::YieldOp>(loc); builder.create<sparse_tensor::YieldOp>(loc);
}); });
rewriter.replaceOpWithNewOp<bufferization::ToTensorOp>(op, dstTp, dst); rewriter.replaceOpWithNewOp<bufferization::ToTensorOp>(op, dstTp, dst);
▲ Show 20 Lines • Show All 354 Lines • ▼ Show 20 Lines LogicalResult matchAndRewrite(OutOp op,
Value indices = dimSizes; // Reuse the dimSizes buffer for indices. Value indices = dimSizes; // Reuse the dimSizes buffer for indices.
Type eltTp = srcTp.getElementType(); Type eltTp = srcTp.getElementType();
SmallString<18> outNextFuncName{"outSparseTensorWriterNext", SmallString<18> outNextFuncName{"outSparseTensorWriterNext",
primaryTypeFunctionSuffix(eltTp)}; primaryTypeFunctionSuffix(eltTp)};
Value value = genAllocaScalar(rewriter, loc, eltTp); Value value = genAllocaScalar(rewriter, loc, eltTp);
ModuleOp module = op->getParentOfType<ModuleOp>(); ModuleOp module = op->getParentOfType<ModuleOp>();
// For each element in the source tensor, output the element. // For each element in the source tensor, output the element.
rewriter.create<ForeachOp>( rewriter.create<ForeachOp>(
loc, src, llvm::None, loc, src, std::nullopt,
[&](OpBuilder &builder, Location loc, ValueRange args, Value v, [&](OpBuilder &builder, Location loc, ValueRange args, Value v,
ValueRange reduc) { ValueRange reduc) {
for (uint64_t i = 0; i < rank; i++) { for (uint64_t i = 0; i < rank; i++) {
rewriter.create<memref::StoreOp>(loc, args[i], indices, rewriter.create<memref::StoreOp>(loc, args[i], indices,
constantIndex(builder, loc, i)); constantIndex(builder, loc, i));
} }
rewriter.create<memref::StoreOp>(loc, v, value); rewriter.create<memref::StoreOp>(loc, v, value);
SmallVector<Value> operands{writer, rankValue, indices, value}; SmallVector<Value> operands{writer, rankValue, indices, value};
▲ Show 20 Lines • Show All 44 Lines • Show Last 20 Lines

mlir/lib/Dialect/SparseTensor/Transforms/Sparsification.cpp

Show First 20 Lines • Show All 502 Lines • ▼ Show 20 Lines for (unsigned d = 0, rank = map.getNumResults(); d < rank; d++) {
AffineExpr ta = map.getResult(toOrigDim(enc, d)); AffineExpr ta = map.getResult(toOrigDim(enc, d));
Optional<unsigned> tldx = merger.getLoopIdx(t.getOperandNumber(), d); Optional<unsigned> tldx = merger.getLoopIdx(t.getOperandNumber(), d);
// Filter loops should be constructed after all the dependent loops, // Filter loops should be constructed after all the dependent loops,
// i.e., d0 + d1 < filter_loop(d0 + d1) // i.e., d0 + d1 < filter_loop(d0 + d1)
if (tldx && merger.isFilterLoop(tldx.value())) { if (tldx && merger.isFilterLoop(tldx.value())) {
assert(!ta.isa<AffineDimExpr>() && assert(!ta.isa<AffineDimExpr>() &&
!isDenseDLT(getDimLevelType(enc, d))); !isDenseDLT(getDimLevelType(enc, d)));
addAffineOrderings(adjM, inDegree, ta, AffineExpr(), llvm::None, tldx); addAffineOrderings(adjM, inDegree, ta, AffineExpr(), std::nullopt,
tldx);
// Now that the ordering of affine expression is captured by filter // Now that the ordering of affine expression is captured by filter
// loop idx, we only need to ensure the affine ordering against filter // loop idx, we only need to ensure the affine ordering against filter
// loop. Thus, we reset the affine express to nil here to mark it as // loop. Thus, we reset the affine express to nil here to mark it as
// resolved. // resolved.
ta = AffineExpr(); ta = AffineExpr();
} }
// Skip tensor during cycle resolution, though order between filter loop // Skip tensor during cycle resolution, though order between filter loop
▲ Show 20 Lines • Show All 1,007 Lines • ▼ Show 20 Lines if (auto whileOp = dyn_cast<scf::WhileOp>(loop)) {
finalizeWhileOp(merger, codegen, rewriter, op, idx, needsUniv, finalizeWhileOp(merger, codegen, rewriter, op, idx, needsUniv,
merger.lat(li).bits, whileOp); merger.lat(li).bits, whileOp);
} else { } else {
needsUniv = false; needsUniv = false;
} }
genLoopBoundary(codegen, merger, [&](MutableArrayRef<Value> reduc) { genLoopBoundary(codegen, merger, [&](MutableArrayRef<Value> reduc) {
codegen.loopEmitter.exitCurrentLoop(rewriter, op.getLoc(), reduc); codegen.loopEmitter.exitCurrentLoop(rewriter, op.getLoc(), reduc);
return llvm::None; return std::nullopt;
}); });
return needsUniv; return needsUniv;
} }
/// Ends a loop sequence at given level. /// Ends a loop sequence at given level.
static void endLoopSeq(Merger &merger, CodeGen &codegen, OpBuilder &builder, static void endLoopSeq(Merger &merger, CodeGen &codegen, OpBuilder &builder,
linalg::GenericOp op, unsigned exp, unsigned at, linalg::GenericOp op, unsigned exp, unsigned at,
▲ Show 20 Lines • Show All 229 Lines • Show Last 20 Lines

mlir/lib/Dialect/Tensor/IR/TensorOps.cpp

Show First 20 Lines • Show All 2,324 Lines • ▼ Show 20 Lines LogicalResult matchAndRewrite(InsertOpTy insertSliceOp,
if (llvm::any_of(insertSliceOp.getOperands(), [](Value operand) { if (llvm::any_of(insertSliceOp.getOperands(), [](Value operand) {
return matchPattern(operand, matchConstantIndex()); return matchPattern(operand, matchConstantIndex());
})) }))
return failure(); return failure();
auto getSourceOfCastOp = [](Value v) -> Optional<Value> { auto getSourceOfCastOp = [](Value v) -> Optional<Value> {
auto castOp = v.getDefiningOp<tensor::CastOp>(); auto castOp = v.getDefiningOp<tensor::CastOp>();
if (!castOp || !canFoldIntoConsumerOp(castOp)) if (!castOp || !canFoldIntoConsumerOp(castOp))
return llvm::None; return std::nullopt;
return castOp.getSource(); return castOp.getSource();
}; };
Optional<Value> sourceCastSource = Optional<Value> sourceCastSource =
getSourceOfCastOp(insertSliceOp.getSource()); getSourceOfCastOp(insertSliceOp.getSource());
Optional<Value> destCastSource = getSourceOfCastOp(insertSliceOp.getDest()); Optional<Value> destCastSource = getSourceOfCastOp(insertSliceOp.getDest());
if (!sourceCastSource && !destCastSource) if (!sourceCastSource && !destCastSource)
return failure(); return failure();
▲ Show 20 Lines • Show All 1,084 Lines • Show Last 20 Lines

mlir/lib/Dialect/Traits.cpp

Show First 20 Lines • Show All 142 Lines • ▼ Show 20 LinesType OpTrait::util::getBroadcastedType(Type type1, Type type2,
// vector and the result should have unranked tensor type. // vector and the result should have unranked tensor type.
if (type1.isa<UnrankedTensorType>() || type2.isa<UnrankedTensorType>()) { if (type1.isa<UnrankedTensorType>() || type2.isa<UnrankedTensorType>()) {
if (type1.isa<VectorType>() || type2.isa<VectorType>()) if (type1.isa<VectorType>() || type2.isa<VectorType>())
return {}; return {};
return UnrankedTensorType::get(elementType); return UnrankedTensorType::get(elementType);
} }
// Returns the type kind if the given type is a vector or ranked tensor type. // Returns the type kind if the given type is a vector or ranked tensor type.
// Returns llvm::None otherwise. // Returns std::nullopt otherwise.
auto getCompositeTypeKind = [](Type type) -> Optional<TypeID> { auto getCompositeTypeKind = [](Type type) -> Optional<TypeID> {
if (type.isa<VectorType, RankedTensorType>()) if (type.isa<VectorType, RankedTensorType>())
return type.getTypeID(); return type.getTypeID();
return llvm::None; return std::nullopt;
}; };
// Make sure the composite type, if has, is consistent. // Make sure the composite type, if has, is consistent.
Optional<TypeID> compositeKind1 = getCompositeTypeKind(type1); Optional<TypeID> compositeKind1 = getCompositeTypeKind(type1);
Optional<TypeID> compositeKind2 = getCompositeTypeKind(type2); Optional<TypeID> compositeKind2 = getCompositeTypeKind(type2);
Optional<TypeID> resultCompositeKind; Optional<TypeID> resultCompositeKind;
if (compositeKind1 && compositeKind2) { if (compositeKind1 && compositeKind2) {
▲ Show 20 Lines • Show All 116 Lines • Show Last 20 Lines

mlir/lib/Dialect/Utils/ReshapeOpsUtils.cpp

Show All 18 Lines
mlir::getReassociationIndicesForReshape(ShapedType sourceType, mlir::getReassociationIndicesForReshape(ShapedType sourceType,
ShapedType targetType) { ShapedType targetType) {
if (sourceType.getRank() > targetType.getRank()) if (sourceType.getRank() > targetType.getRank())
return getReassociationIndicesForCollapse(sourceType.getShape(), return getReassociationIndicesForCollapse(sourceType.getShape(),
targetType.getShape()); targetType.getShape());
if (sourceType.getRank() < targetType.getRank()) if (sourceType.getRank() < targetType.getRank())
return getReassociationIndicesForCollapse(targetType.getShape(), return getReassociationIndicesForCollapse(targetType.getShape(),
sourceType.getShape()); sourceType.getShape());
return llvm::None; return std::nullopt;
} }
Optional<SmallVector<ReassociationIndices>> Optional<SmallVector<ReassociationIndices>>
mlir::getReassociationIndicesForCollapse(ArrayRef<int64_t> sourceShape, mlir::getReassociationIndicesForCollapse(ArrayRef<int64_t> sourceShape,
ArrayRef<int64_t> targetShape) { ArrayRef<int64_t> targetShape) {
if (sourceShape.size() <= targetShape.size()) if (sourceShape.size() <= targetShape.size())
return llvm::None; return std::nullopt;
unsigned sourceDim = 0; unsigned sourceDim = 0;
SmallVector<ReassociationIndices> reassociationMap; SmallVector<ReassociationIndices> reassociationMap;
reassociationMap.reserve(targetShape.size()); reassociationMap.reserve(targetShape.size());
ReassociationIndices currIndices; ReassociationIndices currIndices;
int64_t prodOfCollapsedDims = 1; int64_t prodOfCollapsedDims = 1;
while (sourceDim < sourceShape.size()) { while (sourceDim < sourceShape.size()) {
unsigned targetDim = reassociationMap.size(); unsigned targetDim = reassociationMap.size();
Show All 11 Lines while (sourceDim < sourceShape.size()) {
} }
// If the current expanded dimension is dynamic, then the collapsed // If the current expanded dimension is dynamic, then the collapsed
// dimensions should also be dynamic and product of all previous unprocessed // dimensions should also be dynamic and product of all previous unprocessed
// dimensions of the expanded shape should be 1. // dimensions of the expanded shape should be 1.
if (sourceShape[sourceDim] == ShapedType::kDynamic && if (sourceShape[sourceDim] == ShapedType::kDynamic &&
(currTargetShape != ShapedType::kDynamic || (currTargetShape != ShapedType::kDynamic ||
prodOfCollapsedDims != 1)) prodOfCollapsedDims != 1))
return llvm::None; return std::nullopt;
// If the collapsed dim is dynamic, the current expanded dim should also // If the collapsed dim is dynamic, the current expanded dim should also
// be dynamic. // be dynamic.
if (currTargetShape == ShapedType::kDynamic && if (currTargetShape == ShapedType::kDynamic &&
sourceShape[sourceDim] != ShapedType::kDynamic) sourceShape[sourceDim] != ShapedType::kDynamic)
return llvm::None; return std::nullopt;
// For static shapes, if the product of dimensions of the expanded shape // For static shapes, if the product of dimensions of the expanded shape
// should match the collapsed dimension shape. // should match the collapsed dimension shape.
if (prodOfCollapsedDims * sourceShape[sourceDim] != currTargetShape) if (prodOfCollapsedDims * sourceShape[sourceDim] != currTargetShape)
return llvm::None; return std::nullopt;
currIndices.push_back(sourceDim++); currIndices.push_back(sourceDim++);
reassociationMap.emplace_back(ReassociationIndices{}); reassociationMap.emplace_back(ReassociationIndices{});
std::swap(reassociationMap.back(), currIndices); std::swap(reassociationMap.back(), currIndices);
prodOfCollapsedDims = 1; prodOfCollapsedDims = 1;
} }
// All the dimensions in the target must have been processed. // All the dimensions in the target must have been processed.
if (reassociationMap.size() != targetShape.size()) if (reassociationMap.size() != targetShape.size())
return llvm::None; return std::nullopt;
// Process any remaining entries in the source shape. They all need to be // Process any remaining entries in the source shape. They all need to be
// 1 or dynamic. // 1 or dynamic.
for (; sourceDim < sourceShape.size(); sourceDim++) { for (; sourceDim < sourceShape.size(); sourceDim++) {
if (sourceShape[sourceDim] != ShapedType::kDynamic && if (sourceShape[sourceDim] != ShapedType::kDynamic &&
sourceShape[sourceDim] != 1) sourceShape[sourceDim] != 1)
return llvm::None; return std::nullopt;
// The map is empty when the target type is a scalar. // The map is empty when the target type is a scalar.
if (!reassociationMap.empty()) if (!reassociationMap.empty())
reassociationMap.back().push_back(sourceDim); reassociationMap.back().push_back(sourceDim);
} }
return reassociationMap; return reassociationMap;
} }
Optional<SmallVector<ReassociationIndices>> mlir::composeReassociationIndices( Optional<SmallVector<ReassociationIndices>> mlir::composeReassociationIndices(
ArrayRef<ReassociationIndices> producerReassociations, ArrayRef<ReassociationIndices> producerReassociations,
ArrayRef<ReassociationIndices> consumerReassociations, ArrayRef<ReassociationIndices> consumerReassociations,
MLIRContext *context) { MLIRContext *context) {
SmallVector<ReassociationIndices> composedIndices; SmallVector<ReassociationIndices> composedIndices;
// Make the producer the larger sized vector. If they are of same size, the // Make the producer the larger sized vector. If they are of same size, the
// resulting reshape is not a supported reshape op. // resulting reshape is not a supported reshape op.
if (producerReassociations.size() == consumerReassociations.size()) if (producerReassociations.size() == consumerReassociations.size())
return llvm::None; return std::nullopt;
if (producerReassociations.size() < consumerReassociations.size()) if (producerReassociations.size() < consumerReassociations.size())
std::swap(producerReassociations, consumerReassociations); std::swap(producerReassociations, consumerReassociations);
// Handle the corner case of the result being a rank 0 shaped type. Return an // Handle the corner case of the result being a rank 0 shaped type. Return an
// empty reassociation. // empty reassociation.
if (consumerReassociations.empty()) if (consumerReassociations.empty())
return composedIndices; return composedIndices;
size_t consumerDims = std::accumulate( size_t consumerDims = std::accumulate(
consumerReassociations.begin(), consumerReassociations.end(), 0, consumerReassociations.begin(), consumerReassociations.end(), 0,
[](size_t all, ReassociationIndicesRef indices) { [](size_t all, ReassociationIndicesRef indices) {
return all + indices.size(); return all + indices.size();
}); });
if (producerReassociations.size() != consumerDims) if (producerReassociations.size() != consumerDims)
return llvm::None; return std::nullopt;
for (ReassociationIndicesRef consumerIndices : consumerReassociations) { for (ReassociationIndicesRef consumerIndices : consumerReassociations) {
ReassociationIndices reassociations; ReassociationIndices reassociations;
for (int64_t consumerIndex : consumerIndices) { for (int64_t consumerIndex : consumerIndices) {
llvm::append_range(reassociations, producerReassociations[consumerIndex]); llvm::append_range(reassociations, producerReassociations[consumerIndex]);
} }
composedIndices.push_back(std::move(reassociations)); composedIndices.push_back(std::move(reassociations));
} }
▲ Show 20 Lines • Show All 323 Lines • Show Last 20 Lines

mlir/lib/Dialect/Utils/StaticValueUtils.cpp

Show First 20 Lines • Show All 93 Lines • ▼ Show 20 Lines
/// If ofr is a constant integer or an IntegerAttr, return the integer. /// If ofr is a constant integer or an IntegerAttr, return the integer.
Optional<int64_t> getConstantIntValue(OpFoldResult ofr) { Optional<int64_t> getConstantIntValue(OpFoldResult ofr) {
// Case 1: Check for Constant integer. // Case 1: Check for Constant integer.
if (auto val = ofr.dyn_cast<Value>()) { if (auto val = ofr.dyn_cast<Value>()) {
APSInt intVal; APSInt intVal;
if (matchPattern(val, m_ConstantInt(&intVal))) if (matchPattern(val, m_ConstantInt(&intVal)))
return intVal.getSExtValue(); return intVal.getSExtValue();
return llvm::None; return std::nullopt;
} }
// Case 2: Check for IntegerAttr. // Case 2: Check for IntegerAttr.
Attribute attr = ofr.dyn_cast<Attribute>(); Attribute attr = ofr.dyn_cast<Attribute>();
if (auto intAttr = attr.dyn_cast_or_null<IntegerAttr>()) if (auto intAttr = attr.dyn_cast_or_null<IntegerAttr>())
return intAttr.getValue().getSExtValue(); return intAttr.getValue().getSExtValue();
return llvm::None; return std::nullopt;
} }
/// Return true if `ofr` is constant integer equal to `value`. /// Return true if `ofr` is constant integer equal to `value`.
bool isConstantIntValue(OpFoldResult ofr, int64_t value) { bool isConstantIntValue(OpFoldResult ofr, int64_t value) {
auto val = getConstantIntValue(ofr); auto val = getConstantIntValue(ofr);
return val && *val == value; return val && *val == value;
} }
▲ Show 20 Lines • Show All 61 Lines • Show Last 20 Lines

mlir/lib/Dialect/Vector/IR/VectorOps.cpp

Show First 20 Lines • Show All 5,456 Lines • ▼ Show 20 Linesvoid WarpExecuteOnLane0Op::getSuccessorRegions(
// The warp region is always executed // The warp region is always executed
regions.push_back(RegionSuccessor(&getWarpRegion())); regions.push_back(RegionSuccessor(&getWarpRegion()));
} }
void WarpExecuteOnLane0Op::build(OpBuilder &builder, OperationState &result, void WarpExecuteOnLane0Op::build(OpBuilder &builder, OperationState &result,
TypeRange resultTypes, Value laneId, TypeRange resultTypes, Value laneId,
int64_t warpSize) { int64_t warpSize) {
build(builder, result, resultTypes, laneId, warpSize, build(builder, result, resultTypes, laneId, warpSize,
/*operands=*/llvm::None, /*argTypes=*/llvm::None); /*operands=*/std::nullopt, /*argTypes=*/llvm::None);
} }
void WarpExecuteOnLane0Op::build(OpBuilder &builder, OperationState &result, void WarpExecuteOnLane0Op::build(OpBuilder &builder, OperationState &result,
TypeRange resultTypes, Value laneId, TypeRange resultTypes, Value laneId,
int64_t warpSize, ValueRange args, int64_t warpSize, ValueRange args,
TypeRange blockArgTypes) { TypeRange blockArgTypes) {
result.addOperands(laneId); result.addOperands(laneId);
result.addAttribute(getAttributeNames()[0], result.addAttribute(getAttributeNames()[0],
▲ Show 20 Lines • Show All 134 Lines • Show Last 20 Lines

mlir/lib/Dialect/Vector/Transforms/VectorTransforms.cpp

Show First 20 Lines • Show All 184 Lines • ▼ Show 20 Linesstatic SmallVector<int64_t> getReductionIndex(AffineMap map,
for (unsigned i = 0, e = map.getNumResults(); i < e; i++) { for (unsigned i = 0, e = map.getNumResults(); i < e; i++) {
if (isReductionIterator(iteratorTypes[map.getDimPosition(i)])) if (isReductionIterator(iteratorTypes[map.getDimPosition(i)]))
dimsIdx.push_back(i); dimsIdx.push_back(i);
} }
return dimsIdx; return dimsIdx;
} }
/// Look for a given dimension in an affine map and return its position. Return /// Look for a given dimension in an affine map and return its position. Return
/// llvm::None if the dimension is not in the map results. /// std::nullopt if the dimension is not in the map results.
static llvm::Optional<unsigned> getDimPosition(AffineMap map, unsigned dim) { static llvm::Optional<unsigned> getDimPosition(AffineMap map, unsigned dim) {
for (unsigned i = 0, e = map.getNumResults(); i < e; i++) { for (unsigned i = 0, e = map.getNumResults(); i < e; i++) {
if (map.getDimPosition(i) == dim) if (map.getDimPosition(i) == dim)
return i; return i;
} }
return llvm::None; return std::nullopt;
} }
namespace { namespace {
/// ShapeCastOpFolder folds cancelling ShapeCastOps away. /// ShapeCastOpFolder folds cancelling ShapeCastOps away.
// //
// Example: // Example:
// //
▲ Show 20 Lines • Show All 2,836 Lines • Show Last 20 Lines

mlir/lib/Dialect/Vector/Transforms/VectorUnroll.cpp

Show First 20 Lines • Show All 126 Lines • ▼ Show 20 Lines
static Operation *cloneOpWithOperandsAndTypes(OpBuilder &builder, Location loc, static Operation *cloneOpWithOperandsAndTypes(OpBuilder &builder, Location loc,
Operation *op, Operation *op,
ArrayRef<Value> operands, ArrayRef<Value> operands,
ArrayRef<Type> resultTypes) { ArrayRef<Type> resultTypes) {
return builder.create(loc, op->getName().getIdentifier(), operands, return builder.create(loc, op->getName().getIdentifier(), operands,
resultTypes, op->getAttrs()); resultTypes, op->getAttrs());
} }
/// Return the target shape for unrolling for the given `op`. Return llvm::None /// Return the target shape for unrolling for the given `op`. Return
/// if the op shouldn't be or cannot be unrolled. /// std::nullopt if the op shouldn't be or cannot be unrolled.
static Optional<SmallVector<int64_t>> static Optional<SmallVector<int64_t>>
getTargetShape(const vector::UnrollVectorOptions &options, Operation *op) { getTargetShape(const vector::UnrollVectorOptions &options, Operation *op) {
if (options.filterConstraint && failed(options.filterConstraint(op))) if (options.filterConstraint && failed(options.filterConstraint(op)))
return llvm::None; return std::nullopt;
assert(options.nativeShape && assert(options.nativeShape &&
"vector unrolling expects the native shape or native" "vector unrolling expects the native shape or native"
"shape call back function to be set"); "shape call back function to be set");
auto unrollableVectorOp = dyn_cast<VectorUnrollOpInterface>(op); auto unrollableVectorOp = dyn_cast<VectorUnrollOpInterface>(op);
if (!unrollableVectorOp) if (!unrollableVectorOp)
return llvm::None; return std::nullopt;
auto maybeUnrollShape = unrollableVectorOp.getShapeForUnroll(); auto maybeUnrollShape = unrollableVectorOp.getShapeForUnroll();
if (!maybeUnrollShape) if (!maybeUnrollShape)
return llvm::None; return std::nullopt;
Optional<SmallVector<int64_t>> targetShape = options.nativeShape(op); Optional<SmallVector<int64_t>> targetShape = options.nativeShape(op);
if (!targetShape) if (!targetShape)
return llvm::None; return std::nullopt;
auto maybeShapeRatio = computeShapeRatio(*maybeUnrollShape, *targetShape); auto maybeShapeRatio = computeShapeRatio(*maybeUnrollShape, *targetShape);
if (!maybeShapeRatio || if (!maybeShapeRatio ||
llvm::all_of(*maybeShapeRatio, [](int64_t v) { return v == 1; })) llvm::all_of(*maybeShapeRatio, [](int64_t v) { return v == 1; }))
return llvm::None; return std::nullopt;
return targetShape; return targetShape;
} }
static SmallVector<int64_t> static SmallVector<int64_t>
getUnrollOrder(unsigned numLoops, Operation *op, getUnrollOrder(unsigned numLoops, Operation *op,
const vector::UnrollVectorOptions &options) { const vector::UnrollVectorOptions &options) {
SmallVector<int64_t> loopOrder = SmallVector<int64_t> loopOrder =
llvm::to_vector(llvm::seq<int64_t>(0, static_cast<int64_t>(numLoops))); llvm::to_vector(llvm::seq<int64_t>(0, static_cast<int64_t>(numLoops)));
▲ Show 20 Lines • Show All 493 Lines • Show Last 20 Lines

mlir/lib/IR/AffineMap.cpp

Show First 20 Lines • Show All 57 Lines • ▼ Show 20 Lines case AffineExprKind::CeilDiv:
return constantFoldBinExpr( return constantFoldBinExpr(
expr, [](int64_t lhs, int64_t rhs) { return ceilDiv(lhs, rhs); }); expr, [](int64_t lhs, int64_t rhs) { return ceilDiv(lhs, rhs); });
case AffineExprKind::Constant: case AffineExprKind::Constant:
return expr.cast<AffineConstantExpr>().getValue(); return expr.cast<AffineConstantExpr>().getValue();
case AffineExprKind::DimId: case AffineExprKind::DimId:
if (auto attr = operandConsts[expr.cast<AffineDimExpr>().getPosition()] if (auto attr = operandConsts[expr.cast<AffineDimExpr>().getPosition()]
.dyn_cast_or_null<IntegerAttr>()) .dyn_cast_or_null<IntegerAttr>())
return attr.getInt(); return attr.getInt();
return llvm::None; return std::nullopt;
case AffineExprKind::SymbolId: case AffineExprKind::SymbolId:
if (auto attr = operandConsts[numDims + if (auto attr = operandConsts[numDims +
expr.cast<AffineSymbolExpr>().getPosition()] expr.cast<AffineSymbolExpr>().getPosition()]
.dyn_cast_or_null<IntegerAttr>()) .dyn_cast_or_null<IntegerAttr>())
return attr.getInt(); return attr.getInt();
return llvm::None; return std::nullopt;
} }
llvm_unreachable("Unknown AffineExpr"); llvm_unreachable("Unknown AffineExpr");
} }
// TODO: Change these to operate on APInts too. // TODO: Change these to operate on APInts too.
Optional<int64_t> constantFoldBinExpr(AffineExpr expr, Optional<int64_t> constantFoldBinExpr(AffineExpr expr,
int64_t (*op)(int64_t, int64_t)) { int64_t (*op)(int64_t, int64_t)) {
auto binOpExpr = expr.cast<AffineBinaryOpExpr>(); auto binOpExpr = expr.cast<AffineBinaryOpExpr>();
if (auto lhs = constantFoldImpl(binOpExpr.getLHS())) if (auto lhs = constantFoldImpl(binOpExpr.getLHS()))
if (auto rhs = constantFoldImpl(binOpExpr.getRHS())) if (auto rhs = constantFoldImpl(binOpExpr.getRHS()))
return op(*lhs, *rhs); return op(*lhs, *rhs);
return llvm::None; return std::nullopt;
} }
// The number of dimension operands in AffineMap containing this expression. // The number of dimension operands in AffineMap containing this expression.
unsigned numDims; unsigned numDims;
// The constant valued operands used to evaluate this AffineExpr. // The constant valued operands used to evaluate this AffineExpr.
ArrayRef<Attribute> operandConsts; ArrayRef<Attribute> operandConsts;
}; };
▲ Show 20 Lines • Show All 690 Lines • Show Last 20 Lines

mlir/lib/IR/BuiltinAttributes.cpp

Show First 20 Lines • Show All 86 Lines • ▼ Show 20 Lines default:
if (!isSorted) if (!isSorted)
llvm::array_pod_sort(storage.begin(), storage.end()); llvm::array_pod_sort(storage.begin(), storage.end());
return !isSorted; return !isSorted;
} }
return false; return false;
} }
/// Returns an entry with a duplicate name from the given sorted array of named /// Returns an entry with a duplicate name from the given sorted array of named
/// attributes. Returns llvm::None if all elements have unique names. /// attributes. Returns std::nullopt if all elements have unique names.
static Optional<NamedAttribute> static Optional<NamedAttribute>
findDuplicateElement(ArrayRef<NamedAttribute> value) { findDuplicateElement(ArrayRef<NamedAttribute> value) {
const Optional<NamedAttribute> none{llvm::None}; const Optional<NamedAttribute> none{std::nullopt};
if (value.size() < 2) if (value.size() < 2)
return none; return none;
if (value.size() == 2) if (value.size() == 2)
return value[0].getName() == value[1].getName() ? value[0] : none; return value[0].getName() == value[1].getName() ? value[0] : none;
const auto *it = std::adjacent_find(value.begin(), value.end(), const auto *it = std::adjacent_find(value.begin(), value.end(),
[](NamedAttribute l, NamedAttribute r) { [](NamedAttribute l, NamedAttribute r) {
▲ Show 20 Lines • Show All 1,492 Lines • ▼ Show 20 Lines return DenseResourceElementsAttr::get(type, blobName, std::move(blob))
.template cast<DenseResourceElementsAttrBase<T>>(); .template cast<DenseResourceElementsAttrBase<T>>();
} }
template <typename T> template <typename T>
Optional<ArrayRef<T>> Optional<ArrayRef<T>>
DenseResourceElementsAttrBase<T>::tryGetAsArrayRef() const { DenseResourceElementsAttrBase<T>::tryGetAsArrayRef() const {
if (AsmResourceBlob *blob = this->getRawHandle().getBlob()) if (AsmResourceBlob *blob = this->getRawHandle().getBlob())
return blob->template getDataAs<T>(); return blob->template getDataAs<T>();
return llvm::None; return std::nullopt;
} }
template <typename T> template <typename T>
bool DenseResourceElementsAttrBase<T>::classof(Attribute attr) { bool DenseResourceElementsAttrBase<T>::classof(Attribute attr) {
auto resourceAttr = attr.dyn_cast<DenseResourceElementsAttr>(); auto resourceAttr = attr.dyn_cast<DenseResourceElementsAttr>();
return resourceAttr && DenseResourceAttrUtil<T>::checkElementType( return resourceAttr && DenseResourceAttrUtil<T>::checkElementType(
resourceAttr.getElementType()); resourceAttr.getElementType());
} }
▲ Show 20 Lines • Show All 184 Lines • Show Last 20 Lines

mlir/lib/IR/BuiltinTypes.cpp

Show First 20 Lines • Show All 399 Lines • ▼ Show 20 Lines if (reducedIdx < reducedRank &&
reducedIdx++; reducedIdx++;
continue; continue;
} }
unusedDims.insert(originalIdx); unusedDims.insert(originalIdx);
// If no match on `originalIdx`, the `originalShape` at this dimension // If no match on `originalIdx`, the `originalShape` at this dimension
// must be 1, otherwise we bail. // must be 1, otherwise we bail.
if (originalShape[originalIdx] != 1) if (originalShape[originalIdx] != 1)
return llvm::None; return std::nullopt;
} }
// The whole reducedShape must be scanned, otherwise we bail. // The whole reducedShape must be scanned, otherwise we bail.
if (reducedIdx != reducedRank) if (reducedIdx != reducedRank)
return llvm::None; return std::nullopt;
return unusedDims; return unusedDims;
} }
SliceVerificationResult SliceVerificationResult
mlir::isRankReducedType(ShapedType originalType, mlir::isRankReducedType(ShapedType originalType,
ShapedType candidateReducedType) { ShapedType candidateReducedType) {
if (originalType == candidateReducedType) if (originalType == candidateReducedType)
return SliceVerificationResult::Success; return SliceVerificationResult::Success;
▲ Show 20 Lines • Show All 504 Lines • Show Last 20 Lines

mlir/lib/IR/Diagnostics.cpp

Show First 20 Lines • Show All 395 Lines • ▼ Show 20 Linesstatic Optional<CallSiteLoc> getCallSiteLoc(Location loc) {
if (auto callLoc = dyn_cast<CallSiteLoc>(loc)) if (auto callLoc = dyn_cast<CallSiteLoc>(loc))
return callLoc; return callLoc;
if (auto fusedLoc = dyn_cast<FusedLoc>(loc)) { if (auto fusedLoc = dyn_cast<FusedLoc>(loc)) {
for (auto subLoc : cast<FusedLoc>(loc).getLocations()) { for (auto subLoc : cast<FusedLoc>(loc).getLocations()) {
if (auto callLoc = getCallSiteLoc(subLoc)) { if (auto callLoc = getCallSiteLoc(subLoc)) {
return callLoc; return callLoc;
} }
} }
return llvm::None; return std::nullopt;
} }
return llvm::None; return std::nullopt;
} }
/// Given a diagnostic kind, returns the LLVM DiagKind. /// Given a diagnostic kind, returns the LLVM DiagKind.
static llvm::SourceMgr::DiagKind getDiagKind(DiagnosticSeverity kind) { static llvm::SourceMgr::DiagKind getDiagKind(DiagnosticSeverity kind) {
switch (kind) { switch (kind) {
case DiagnosticSeverity::Note: case DiagnosticSeverity::Note:
return llvm::SourceMgr::DK_Note; return llvm::SourceMgr::DK_Note;
case DiagnosticSeverity::Warning: case DiagnosticSeverity::Warning:
▲ Show 20 Lines • Show All 110 Lines • ▼ Show 20 Lines if (unsigned id = impl->getSourceMgrBufferIDForFile(mgr, filename))
return mgr.getMemoryBuffer(id); return mgr.getMemoryBuffer(id);
return nullptr; return nullptr;
} }
Optional<Location> SourceMgrDiagnosticHandler::findLocToShow(Location loc) { Optional<Location> SourceMgrDiagnosticHandler::findLocToShow(Location loc) {
if (!shouldShowLocFn) if (!shouldShowLocFn)
return loc; return loc;
if (!shouldShowLocFn(loc)) if (!shouldShowLocFn(loc))
return llvm::None; return std::nullopt;
// Recurse into the child locations of some of location types. // Recurse into the child locations of some of location types.
return TypeSwitch<LocationAttr, Optional<Location>>(loc) return TypeSwitch<LocationAttr, Optional<Location>>(loc)
.Case([&](CallSiteLoc callLoc) -> Optional<Location> { .Case([&](CallSiteLoc callLoc) -> Optional<Location> {
// We recurse into the callee of a call site, as the caller will be // We recurse into the callee of a call site, as the caller will be
// emitted in a different note on the main diagnostic. // emitted in a different note on the main diagnostic.
return findLocToShow(callLoc.getCallee()); return findLocToShow(callLoc.getCallee());
}) })
.Case([&](FileLineColLoc) -> Optional<Location> { return loc; }) .Case([&](FileLineColLoc) -> Optional<Location> { return loc; })
.Case([&](FusedLoc fusedLoc) -> Optional<Location> { .Case([&](FusedLoc fusedLoc) -> Optional<Location> {
// Fused location is unique in that we try to find a sub-location to // Fused location is unique in that we try to find a sub-location to
// show, rather than the top-level location itself. // show, rather than the top-level location itself.
for (Location childLoc : fusedLoc.getLocations()) for (Location childLoc : fusedLoc.getLocations())
if (Optional<Location> showableLoc = findLocToShow(childLoc)) if (Optional<Location> showableLoc = findLocToShow(childLoc))
return showableLoc; return showableLoc;
return llvm::None; return std::nullopt;
}) })
.Case([&](NameLoc nameLoc) -> Optional<Location> { .Case([&](NameLoc nameLoc) -> Optional<Location> {
return findLocToShow(nameLoc.getChildLoc()); return findLocToShow(nameLoc.getChildLoc());
}) })
.Case([&](OpaqueLoc opaqueLoc) -> Optional<Location> { .Case([&](OpaqueLoc opaqueLoc) -> Optional<Location> {
// OpaqueLoc always falls back to a different source location. // OpaqueLoc always falls back to a different source location.
return findLocToShow(opaqueLoc.getFallbackLocation()); return findLocToShow(opaqueLoc.getFallbackLocation());
}) })
.Case([](UnknownLoc) -> Optional<Location> { .Case([](UnknownLoc) -> Optional<Location> {
// Prefer not to show unknown locations. // Prefer not to show unknown locations.
return llvm::None; return std::nullopt;
}); });
} }
/// Get a memory buffer for the given file, or the main file of the source /// Get a memory buffer for the given file, or the main file of the source
/// manager if one doesn't exist. This always returns non-null. /// manager if one doesn't exist. This always returns non-null.
SMLoc SourceMgrDiagnosticHandler::convertLocToSMLoc(FileLineColLoc loc) { SMLoc SourceMgrDiagnosticHandler::convertLocToSMLoc(FileLineColLoc loc) {
// The column and line may be zero to represent unknown column and/or unknown // The column and line may be zero to represent unknown column and/or unknown
/// line/column information. /// line/column information.
▲ Show 20 Lines • Show All 125 Lines • ▼ Show 20 Linesstatic StringRef getDiagKindStr(DiagnosticSeverity kind) {
llvm_unreachable("Unknown DiagnosticSeverity"); llvm_unreachable("Unknown DiagnosticSeverity");
} }
Optional<MutableArrayRef<ExpectedDiag>> Optional<MutableArrayRef<ExpectedDiag>>
SourceMgrDiagnosticVerifierHandlerImpl::getExpectedDiags(StringRef bufName) { SourceMgrDiagnosticVerifierHandlerImpl::getExpectedDiags(StringRef bufName) {
auto expectedDiags = expectedDiagsPerFile.find(bufName); auto expectedDiags = expectedDiagsPerFile.find(bufName);
if (expectedDiags != expectedDiagsPerFile.end()) if (expectedDiags != expectedDiagsPerFile.end())
return MutableArrayRef<ExpectedDiag>(expectedDiags->second); return MutableArrayRef<ExpectedDiag>(expectedDiags->second);
return llvm::None; return std::nullopt;
} }
MutableArrayRef<ExpectedDiag> MutableArrayRef<ExpectedDiag>
SourceMgrDiagnosticVerifierHandlerImpl::computeExpectedDiags( SourceMgrDiagnosticVerifierHandlerImpl::computeExpectedDiags(
raw_ostream &os, llvm::SourceMgr &mgr, const llvm::MemoryBuffer *buf) { raw_ostream &os, llvm::SourceMgr &mgr, const llvm::MemoryBuffer *buf) {
// If the buffer is invalid, return an empty list. // If the buffer is invalid, return an empty list.
if (!buf) if (!buf)
return llvm::None; return std::nullopt;
auto &expectedDiags = expectedDiagsPerFile[buf->getBufferIdentifier()]; auto &expectedDiags = expectedDiagsPerFile[buf->getBufferIdentifier()];
// The number of the last line that did not correlate to a designator. // The number of the last line that did not correlate to a designator.
unsigned lastNonDesignatorLine = 0; unsigned lastNonDesignatorLine = 0;
// The indices of designators that apply to the next non designator line. // The indices of designators that apply to the next non designator line.
SmallVector<unsigned, 1> designatorsForNextLine; SmallVector<unsigned, 1> designatorsForNextLine;
▲ Show 20 Lines • Show All 316 Lines • Show Last 20 Lines

mlir/lib/IR/ExtensibleDialect.cpp

Show First 20 Lines • Show All 462 Lines • ▼ Show 20 Linesbool ExtensibleDialect::classof(const Dialect *dialect) {
return const_cast<Dialect *>(dialect) return const_cast<Dialect *>(dialect)
->getRegisteredInterface<IsExtensibleDialect>(); ->getRegisteredInterface<IsExtensibleDialect>();
} }
OptionalParseResult ExtensibleDialect::parseOptionalDynamicType( OptionalParseResult ExtensibleDialect::parseOptionalDynamicType(
StringRef typeName, AsmParser &parser, Type &resultType) const { StringRef typeName, AsmParser &parser, Type &resultType) const {
DynamicTypeDefinition *typeDef = lookupTypeDefinition(typeName); DynamicTypeDefinition *typeDef = lookupTypeDefinition(typeName);
if (!typeDef) if (!typeDef)
return llvm::None; return std::nullopt;
DynamicType dynType; DynamicType dynType;
if (DynamicType::parse(parser, typeDef, dynType)) if (DynamicType::parse(parser, typeDef, dynType))
return failure(); return failure();
resultType = dynType; resultType = dynType;
return success(); return success();
} }
LogicalResult ExtensibleDialect::printIfDynamicType(Type type, LogicalResult ExtensibleDialect::printIfDynamicType(Type type,
AsmPrinter &printer) { AsmPrinter &printer) {
if (auto dynType = type.dyn_cast<DynamicType>()) { if (auto dynType = type.dyn_cast<DynamicType>()) {
dynType.print(printer); dynType.print(printer);
return success(); return success();
} }
return failure(); return failure();
} }
OptionalParseResult ExtensibleDialect::parseOptionalDynamicAttr( OptionalParseResult ExtensibleDialect::parseOptionalDynamicAttr(
StringRef attrName, AsmParser &parser, Attribute &resultAttr) const { StringRef attrName, AsmParser &parser, Attribute &resultAttr) const {
DynamicAttrDefinition *attrDef = lookupAttrDefinition(attrName); DynamicAttrDefinition *attrDef = lookupAttrDefinition(attrName);
if (!attrDef) if (!attrDef)
return llvm::None; return std::nullopt;
DynamicAttr dynAttr; DynamicAttr dynAttr;
if (DynamicAttr::parse(parser, attrDef, dynAttr)) if (DynamicAttr::parse(parser, attrDef, dynAttr))
return failure(); return failure();
resultAttr = dynAttr; resultAttr = dynAttr;
return success(); return success();
} }
▲ Show 20 Lines • Show All 89 Lines • Show Last 20 Lines

mlir/lib/IR/MLIRContext.cpp

Show First 20 Lines • Show All 747 Lines • ▼ Show 20 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
Optional<RegisteredOperationName> Optional<RegisteredOperationName>
RegisteredOperationName::lookup(StringRef name, MLIRContext *ctx) { RegisteredOperationName::lookup(StringRef name, MLIRContext *ctx) {
auto &impl = ctx->getImpl(); auto &impl = ctx->getImpl();
auto it = impl.registeredOperations.find(name); auto it = impl.registeredOperations.find(name);
if (it != impl.registeredOperations.end()) if (it != impl.registeredOperations.end())
return it->getValue(); return it->getValue();
return llvm::None; return std::nullopt;
} }
ParseResult ParseResult
RegisteredOperationName::parseAssembly(OpAsmParser &parser, RegisteredOperationName::parseAssembly(OpAsmParser &parser,
OperationState &result) const { OperationState &result) const {
return impl->parseAssemblyFn(parser, result); return impl->parseAssemblyFn(parser, result);
} }
▲ Show 20 Lines • Show All 332 Lines • Show Last 20 Lines

mlir/lib/IR/SymbolTable.cpp

Show First 20 Lines • Show All 549 Lines • ▼ Show 20 Lines
/// regions, invoking the provided callback for each. This does not traverse /// regions, invoking the provided callback for each. This does not traverse
/// into any nested symbol tables. /// into any nested symbol tables.
static Optional<WalkResult> static Optional<WalkResult>
walkSymbolUses(MutableArrayRef<Region> regions, walkSymbolUses(MutableArrayRef<Region> regions,
function_ref<WalkResult(SymbolTable::SymbolUse)> callback) { function_ref<WalkResult(SymbolTable::SymbolUse)> callback) {
return walkSymbolTable(regions, [&](Operation *op) -> Optional<WalkResult> { return walkSymbolTable(regions, [&](Operation *op) -> Optional<WalkResult> {
// Check that this isn't a potentially unknown symbol table. // Check that this isn't a potentially unknown symbol table.
if (isPotentiallyUnknownSymbolTable(op)) if (isPotentiallyUnknownSymbolTable(op))
return llvm::None; return std::nullopt;
return walkSymbolRefs(op, callback); return walkSymbolRefs(op, callback);
}); });
} }
/// Walk all of the uses, for any symbol, that are nested within the given /// Walk all of the uses, for any symbol, that are nested within the given
/// operation 'from', invoking the provided callback for each. This does not /// operation 'from', invoking the provided callback for each. This does not
/// traverse into any nested symbol tables. /// traverse into any nested symbol tables.
static Optional<WalkResult> static Optional<WalkResult>
walkSymbolUses(Operation *from, walkSymbolUses(Operation *from,
function_ref<WalkResult(SymbolTable::SymbolUse)> callback) { function_ref<WalkResult(SymbolTable::SymbolUse)> callback) {
// If this operation has regions, and it, as well as its dialect, isn't // If this operation has regions, and it, as well as its dialect, isn't
// registered then conservatively fail. The operation may define a // registered then conservatively fail. The operation may define a
// symbol table, so we can't opaquely know if we should traverse to find // symbol table, so we can't opaquely know if we should traverse to find
// nested uses. // nested uses.
if (isPotentiallyUnknownSymbolTable(from)) if (isPotentiallyUnknownSymbolTable(from))
return llvm::None; return std::nullopt;
// Walk the uses on this operation. // Walk the uses on this operation.
if (walkSymbolRefs(from, callback).wasInterrupted()) if (walkSymbolRefs(from, callback).wasInterrupted())
return WalkResult::interrupt(); return WalkResult::interrupt();
// Only recurse if this operation is not a symbol table. A symbol table // Only recurse if this operation is not a symbol table. A symbol table
// defines a new scope, so we can't walk the attributes from within the symbol // defines a new scope, so we can't walk the attributes from within the symbol
// table op. // table op.
▲ Show 20 Lines • Show All 152 Lines • ▼ Show 20 Lines
template <typename FromT> template <typename FromT>
static Optional<SymbolTable::UseRange> getSymbolUsesImpl(FromT from) { static Optional<SymbolTable::UseRange> getSymbolUsesImpl(FromT from) {
std::vector<SymbolTable::SymbolUse> uses; std::vector<SymbolTable::SymbolUse> uses;
auto walkFn = [&](SymbolTable::SymbolUse symbolUse) { auto walkFn = [&](SymbolTable::SymbolUse symbolUse) {
uses.push_back(symbolUse); uses.push_back(symbolUse);
return WalkResult::advance(); return WalkResult::advance();
}; };
auto result = walkSymbolUses(from, walkFn); auto result = walkSymbolUses(from, walkFn);
return result ? Optional<SymbolTable::UseRange>(std::move(uses)) : llvm::None; return result ? Optional<SymbolTable::UseRange>(std::move(uses))
: std::nullopt;
} }
/// Get an iterator range for all of the uses, for any symbol, that are nested /// Get an iterator range for all of the uses, for any symbol, that are nested
/// within the given operation 'from'. This does not traverse into any nested /// within the given operation 'from'. This does not traverse into any nested
/// symbol tables, and will also only return uses on 'from' if it does not /// symbol tables, and will also only return uses on 'from' if it does not
/// also define a symbol table. This is because we treat the region as the /// also define a symbol table. This is because we treat the region as the
/// boundary of the symbol table, and not the op itself. This function returns /// boundary of the symbol table, and not the op itself. This function returns
/// None if there are any unknown operations that may potentially be symbol /// None if there are any unknown operations that may potentially be symbol
Show All 13 Lines
static Optional<SymbolTable::UseRange> getSymbolUsesImpl(SymbolT symbol, static Optional<SymbolTable::UseRange> getSymbolUsesImpl(SymbolT symbol,
IRUnitT *limit) { IRUnitT *limit) {
std::vector<SymbolTable::SymbolUse> uses; std::vector<SymbolTable::SymbolUse> uses;
for (SymbolScope &scope : collectSymbolScopes(symbol, limit)) { for (SymbolScope &scope : collectSymbolScopes(symbol, limit)) {
if (!scope.walk([&](SymbolTable::SymbolUse symbolUse) { if (!scope.walk([&](SymbolTable::SymbolUse symbolUse) {
if (isReferencePrefixOf(scope.symbol, symbolUse.getSymbolRef())) if (isReferencePrefixOf(scope.symbol, symbolUse.getSymbolRef()))
uses.push_back(symbolUse); uses.push_back(symbolUse);
})) }))
return llvm::None; return std::nullopt;
} }
return SymbolTable::UseRange(std::move(uses)); return SymbolTable::UseRange(std::move(uses));
} }
/// Get all of the uses of the given symbol that are nested within the given /// Get all of the uses of the given symbol that are nested within the given
/// operation 'from', invoking the provided callback for each. This does not /// operation 'from', invoking the provided callback for each. This does not
/// traverse into any nested symbol tables. This function returns None if there /// traverse into any nested symbol tables. This function returns None if there
/// are any unknown operations that may potentially be symbol tables. /// are any unknown operations that may potentially be symbol tables.
▲ Show 20 Lines • Show All 264 Lines • Show Last 20 Lines

mlir/lib/Interfaces/ControlFlowInterfaces.cpp

Show All 36 Lines
/// successor if 'operandIndex' is within the range of 'operands', or None if /// successor if 'operandIndex' is within the range of 'operands', or None if
/// `operandIndex` isn't a successor operand index. /// `operandIndex` isn't a successor operand index.
Optional<BlockArgument> Optional<BlockArgument>
detail::getBranchSuccessorArgument(const SuccessorOperands &operands, detail::getBranchSuccessorArgument(const SuccessorOperands &operands,
unsigned operandIndex, Block *successor) { unsigned operandIndex, Block *successor) {
OperandRange forwardedOperands = operands.getForwardedOperands(); OperandRange forwardedOperands = operands.getForwardedOperands();
// Check that the operands are valid. // Check that the operands are valid.
if (forwardedOperands.empty()) if (forwardedOperands.empty())
return llvm::None; return std::nullopt;
// Check to ensure that this operand is within the range. // Check to ensure that this operand is within the range.
unsigned operandsStart = forwardedOperands.getBeginOperandIndex(); unsigned operandsStart = forwardedOperands.getBeginOperandIndex();
if (operandIndex < operandsStart || if (operandIndex < operandsStart ||
operandIndex >= (operandsStart + forwardedOperands.size())) operandIndex >= (operandsStart + forwardedOperands.size()))
return llvm::None; return std::nullopt;
// Index the successor. // Index the successor.
unsigned argIndex = unsigned argIndex =
operands.getProducedOperandCount() + operandIndex - operandsStart; operands.getProducedOperandCount() + operandIndex - operandsStart;
return successor->getArgument(argIndex); return successor->getArgument(argIndex);
} }
/// Verify that the given operands match those of the given successor block. /// Verify that the given operands match those of the given successor block.
Show All 20 Linesdetail::verifyBranchSuccessorOperands(Operation *op, unsigned succNo,
return success(); return success();
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// RegionBranchOpInterface // RegionBranchOpInterface
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
/// Verify that types match along all region control flow edges originating from /// Verify that types match along all region control flow edges originating from
/// `sourceNo` (region # if source is a region, llvm::None if source is parent /// `sourceNo` (region # if source is a region, std::nullopt if source is parent
/// op). `getInputsTypesForRegion` is a function that returns the types of the /// op). `getInputsTypesForRegion` is a function that returns the types of the
/// inputs that flow from `sourceIndex' to the given region, or llvm::None if /// inputs that flow from `sourceIndex' to the given region, or std::nullopt if
/// the exact type match verification is not necessary (e.g., if the Op verifies /// the exact type match verification is not necessary (e.g., if the Op verifies
/// the match itself). /// the match itself).
static LogicalResult static LogicalResult
verifyTypesAlongAllEdges(Operation *op, Optional<unsigned> sourceNo, verifyTypesAlongAllEdges(Operation *op, Optional<unsigned> sourceNo,
function_ref<Optional<TypeRange>(Optional<unsigned>)> function_ref<Optional<TypeRange>(Optional<unsigned>)>
getInputsTypesForRegion) { getInputsTypesForRegion) {
auto regionInterface = cast<RegionBranchOpInterface>(op); auto regionInterface = cast<RegionBranchOpInterface>(op);
▲ Show 20 Lines • Show All 52 Lines • ▼ Show 20 Lines
LogicalResult detail::verifyTypesAlongControlFlowEdges(Operation *op) { LogicalResult detail::verifyTypesAlongControlFlowEdges(Operation *op) {
auto regionInterface = cast<RegionBranchOpInterface>(op); auto regionInterface = cast<RegionBranchOpInterface>(op);
auto inputTypesFromParent = [&](Optional<unsigned> regionNo) -> TypeRange { auto inputTypesFromParent = [&](Optional<unsigned> regionNo) -> TypeRange {
return regionInterface.getSuccessorEntryOperands(regionNo).getTypes(); return regionInterface.getSuccessorEntryOperands(regionNo).getTypes();
}; };
// Verify types along control flow edges originating from the parent. // Verify types along control flow edges originating from the parent.
if (failed(verifyTypesAlongAllEdges(op, llvm::None, inputTypesFromParent))) if (failed(verifyTypesAlongAllEdges(op, std::nullopt, inputTypesFromParent)))
return failure(); return failure();
auto areTypesCompatible = [&](TypeRange lhs, TypeRange rhs) { auto areTypesCompatible = [&](TypeRange lhs, TypeRange rhs) {
if (lhs.size() != rhs.size()) if (lhs.size() != rhs.size())
return false; return false;
for (auto types : llvm::zip(lhs, rhs)) { for (auto types : llvm::zip(lhs, rhs)) {
if (!regionInterface.areTypesCompatible(std::get<0>(types), if (!regionInterface.areTypesCompatible(std::get<0>(types),
std::get<1>(types))) { std::get<1>(types))) {
Show All 33 Lines for (Block &block : region) {
<< " operands mismatch between return-like terminators"; << " operands mismatch between return-like terminators";
} }
auto inputTypesFromRegion = auto inputTypesFromRegion =
[&](Optional<unsigned> regionNo) -> Optional<TypeRange> { [&](Optional<unsigned> regionNo) -> Optional<TypeRange> {
// If there is no return-like terminator, the op itself should verify // If there is no return-like terminator, the op itself should verify
// type consistency. // type consistency.
if (!regionReturnOperands) if (!regionReturnOperands)
return llvm::None; return std::nullopt;
// All successors get the same set of operand types. // All successors get the same set of operand types.
return TypeRange(regionReturnOperands->getTypes()); return TypeRange(regionReturnOperands->getTypes());
}; };
if (failed(verifyTypesAlongAllEdges(op, regionNo, inputTypesFromRegion))) if (failed(verifyTypesAlongAllEdges(op, regionNo, inputTypesFromRegion)))
return failure(); return failure();
} }
▲ Show 20 Lines • Show All 140 Lines • ▼ Show 20 Lines
bool mlir::isRegionReturnLike(Operation *operation) { bool mlir::isRegionReturnLike(Operation *operation) {
return dyn_cast<RegionBranchTerminatorOpInterface>(operation) || return dyn_cast<RegionBranchTerminatorOpInterface>(operation) ||
operation->hasTrait<OpTrait::ReturnLike>(); operation->hasTrait<OpTrait::ReturnLike>();
} }
/// Returns the mutable operands that are passed to the region with the given /// Returns the mutable operands that are passed to the region with the given
/// `regionIndex`. If the operation does not implement the /// `regionIndex`. If the operation does not implement the
/// `RegionBranchTerminatorOpInterface` and is not marked as `ReturnLike`, the /// `RegionBranchTerminatorOpInterface` and is not marked as `ReturnLike`, the
/// result will be `llvm::None`. In all other cases, the resulting /// result will be `std::nullopt`. In all other cases, the resulting
/// `OperandRange` represents all operands that are passed to the specified /// `OperandRange` represents all operands that are passed to the specified
/// successor region. If `regionIndex` is `llvm::None`, all operands that are /// successor region. If `regionIndex` is `std::nullopt`, all operands that are
/// passed to the parent operation will be returned. /// passed to the parent operation will be returned.
Optional<MutableOperandRange> Optional<MutableOperandRange>
mlir::getMutableRegionBranchSuccessorOperands(Operation *operation, mlir::getMutableRegionBranchSuccessorOperands(Operation *operation,
Optional<unsigned> regionIndex) { Optional<unsigned> regionIndex) {
// Try to query a RegionBranchTerminatorOpInterface to determine // Try to query a RegionBranchTerminatorOpInterface to determine
// all successor operands that will be passed to the successor // all successor operands that will be passed to the successor
// input arguments. // input arguments.
if (auto regionTerminatorInterface = if (auto regionTerminatorInterface =
dyn_cast<RegionBranchTerminatorOpInterface>(operation)) dyn_cast<RegionBranchTerminatorOpInterface>(operation))
return regionTerminatorInterface.getMutableSuccessorOperands(regionIndex); return regionTerminatorInterface.getMutableSuccessorOperands(regionIndex);
// TODO: The ReturnLike trait should imply a default implementation of the // TODO: The ReturnLike trait should imply a default implementation of the
// RegionBranchTerminatorOpInterface. This would make this code significantly // RegionBranchTerminatorOpInterface. This would make this code significantly
// easier. Furthermore, this may even make this function obsolete. // easier. Furthermore, this may even make this function obsolete.
if (operation->hasTrait<OpTrait::ReturnLike>()) if (operation->hasTrait<OpTrait::ReturnLike>())
return MutableOperandRange(operation); return MutableOperandRange(operation);
return llvm::None; return std::nullopt;
} }
/// Returns the read only operands that are passed to the region with the given /// Returns the read only operands that are passed to the region with the given
/// `regionIndex`. See `getMutableRegionBranchSuccessorOperands` for more /// `regionIndex`. See `getMutableRegionBranchSuccessorOperands` for more
/// information. /// information.
Optional<OperandRange> Optional<OperandRange>
mlir::getRegionBranchSuccessorOperands(Operation *operation, mlir::getRegionBranchSuccessorOperands(Operation *operation,
Optional<unsigned> regionIndex) { Optional<unsigned> regionIndex) {
auto range = getMutableRegionBranchSuccessorOperands(operation, regionIndex); auto range = getMutableRegionBranchSuccessorOperands(operation, regionIndex);
return range ? Optional<OperandRange>(*range) : llvm::None; return range ? Optional<OperandRange>(*range) : std::nullopt;
} }

mlir/lib/Pass/Pass.cpp

Show First 20 Lines • Show All 710 Lines • ▼ Show 20 Linesvoid OpToOpPassAdaptor::runOnOperationAsyncImpl(bool verifyPasses) {
// Run a prepass over the operation to collect the nested operations to // Run a prepass over the operation to collect the nested operations to
// execute over. This ensures that an analysis manager exists for each // execute over. This ensures that an analysis manager exists for each
// operation, as well as providing a queue of operations to execute over. // operation, as well as providing a queue of operations to execute over.
std::vector<OpPMInfo> opInfos; std::vector<OpPMInfo> opInfos;
DenseMap<OperationName, Optional<unsigned>> knownOpPMIdx; DenseMap<OperationName, Optional<unsigned>> knownOpPMIdx;
for (auto &region : getOperation()->getRegions()) { for (auto &region : getOperation()->getRegions()) {
for (Operation &op : region.getOps()) { for (Operation &op : region.getOps()) {
// Get the pass manager index for this operation type. // Get the pass manager index for this operation type.
auto pmIdxIt = knownOpPMIdx.try_emplace(op.getName(), llvm::None); auto pmIdxIt = knownOpPMIdx.try_emplace(op.getName(), std::nullopt);
if (pmIdxIt.second) { if (pmIdxIt.second) {
if (auto *mgr = findPassManagerFor(mgrs, op.getName(), *context)) if (auto *mgr = findPassManagerFor(mgrs, op.getName(), *context))
pmIdxIt.first->second = std::distance(mgrs.begin(), mgr); pmIdxIt.first->second = std::distance(mgrs.begin(), mgr);
} }
// If this operation can be scheduled, add it to the list. // If this operation can be scheduled, add it to the list.
if (pmIdxIt.first->second) if (pmIdxIt.first->second)
opInfos.emplace_back(*pmIdxIt.first->second, &op, am.nest(&op)); opInfos.emplace_back(*pmIdxIt.first->second, &op, am.nest(&op));
▲ Show 20 Lines • Show All 272 Lines • Show Last 20 Lines

mlir/lib/Pass/PassStatistics.cpp

Show All 21 Lines
struct Statistic { struct Statistic {
const char *name, *desc; const char *name, *desc;
uint64_t value; uint64_t value;
}; };
} // namespace } // namespace
/// Utility to print a pass entry in the statistics output. /// Utility to print a pass entry in the statistics output.
static void printPassEntry(raw_ostream &os, unsigned indent, StringRef pass, static void printPassEntry(raw_ostream &os, unsigned indent, StringRef pass,
MutableArrayRef<Statistic> stats = llvm::None) { MutableArrayRef<Statistic> stats = std::nullopt) {
os.indent(indent) << pass << "\n"; os.indent(indent) << pass << "\n";
if (stats.empty()) if (stats.empty())
return; return;
// Make sure to sort the statistics by name. // Make sure to sort the statistics by name.
llvm::array_pod_sort( llvm::array_pod_sort(
stats.begin(), stats.end(), [](const auto *lhs, const auto *rhs) { stats.begin(), stats.end(), [](const auto *lhs, const auto *rhs) {
return StringRef{lhs->name}.compare(StringRef{rhs->name}); return StringRef{lhs->name}.compare(StringRef{rhs->name});
▲ Show 20 Lines • Show All 212 Lines • Show Last 20 Lines

mlir/lib/Support/FileUtilities.cpp

Show All 31 Lines if (std::error_code error = fileOrErr.getError()) {
return nullptr; return nullptr;
} }
return std::move(*fileOrErr); return std::move(*fileOrErr);
} }
std::unique_ptr<llvm::MemoryBuffer> std::unique_ptr<llvm::MemoryBuffer>
mlir::openInputFile(StringRef inputFilename, std::string *errorMessage) { mlir::openInputFile(StringRef inputFilename, std::string *errorMessage) {
return openInputFileImpl(inputFilename, errorMessage, return openInputFileImpl(inputFilename, errorMessage,
/*alignment=*/llvm::None); /*alignment=*/std::nullopt);
} }
std::unique_ptr<llvm::MemoryBuffer> std::unique_ptr<llvm::MemoryBuffer>
mlir::openInputFile(llvm::StringRef inputFilename, llvm::Align alignment, mlir::openInputFile(llvm::StringRef inputFilename, llvm::Align alignment,
std::string *errorMessage) { std::string *errorMessage) {
return openInputFileImpl(inputFilename, errorMessage, alignment); return openInputFileImpl(inputFilename, errorMessage, alignment);
} }
std::unique_ptr<llvm::ToolOutputFile> std::unique_ptr<llvm::ToolOutputFile>
Show All 13 Lines

mlir/lib/Support/Timing.cpp

Show First 20 Lines • Show All 505 Lines • ▼ Show 20 Lines
void DefaultTimingManager::dumpAsTree(raw_ostream &os) { void DefaultTimingManager::dumpAsTree(raw_ostream &os) {
impl->rootTimer->finalize(); impl->rootTimer->finalize();
impl->rootTimer->print(os, DisplayMode::Tree); impl->rootTimer->print(os, DisplayMode::Tree);
} }
Optional<void *> DefaultTimingManager::rootTimer() { Optional<void *> DefaultTimingManager::rootTimer() {
if (impl->enabled) if (impl->enabled)
return impl->rootTimer.get(); return impl->rootTimer.get();
return llvm::None; return std::nullopt;
} }
void DefaultTimingManager::startTimer(void *handle) { void DefaultTimingManager::startTimer(void *handle) {
static_cast<TimerImpl *>(handle)->start(); static_cast<TimerImpl *>(handle)->start();
} }
void DefaultTimingManager::stopTimer(void *handle) { void DefaultTimingManager::stopTimer(void *handle) {
static_cast<TimerImpl *>(handle)->stop(); static_cast<TimerImpl *>(handle)->stop();
▲ Show 20 Lines • Show All 44 Lines • Show Last 20 Lines

mlir/lib/TableGen/AttrOrTypeDef.cpp

Show All 16 Lines
using namespace mlir::tblgen; using namespace mlir::tblgen;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// AttrOrTypeBuilder // AttrOrTypeBuilder
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
Optional<StringRef> AttrOrTypeBuilder::getReturnType() const { Optional<StringRef> AttrOrTypeBuilder::getReturnType() const {
Optional<StringRef> type = def->getValueAsOptionalString("returnType"); Optional<StringRef> type = def->getValueAsOptionalString("returnType");
return type && !type->empty() ? type : llvm::None; return type && !type->empty() ? type : std::nullopt;
} }
bool AttrOrTypeBuilder::hasInferredContextParameter() const { bool AttrOrTypeBuilder::hasInferredContextParameter() const {
return def->getValueAsBit("hasInferredContextParam"); return def->getValueAsBit("hasInferredContextParam");
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// AttrOrTypeDef // AttrOrTypeDef
▲ Show 20 Lines • Show All 243 Lines • ▼ Show 20 Lines
} }
bool AttrOrTypeParameter::isOptional() const { bool AttrOrTypeParameter::isOptional() const {
return getDefaultValue().has_value(); return getDefaultValue().has_value();
} }
Optional<StringRef> AttrOrTypeParameter::getDefaultValue() const { Optional<StringRef> AttrOrTypeParameter::getDefaultValue() const {
Optional<StringRef> result = getDefValue<llvm::StringInit>("defaultValue"); Optional<StringRef> result = getDefValue<llvm::StringInit>("defaultValue");
return result && !result->empty() ? result : llvm::None; return result && !result->empty() ? result : std::nullopt;
} }
llvm::Init *AttrOrTypeParameter::getDef() const { return def->getArg(index); } llvm::Init *AttrOrTypeParameter::getDef() const { return def->getArg(index); }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// AttributeSelfTypeParameter // AttributeSelfTypeParameter
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
bool AttributeSelfTypeParameter::classof(const AttrOrTypeParameter *param) { bool AttributeSelfTypeParameter::classof(const AttrOrTypeParameter *param) {
llvm::Init *paramDef = param->getDef(); llvm::Init *paramDef = param->getDef();
if (auto *paramDefInit = dyn_cast<llvm::DefInit>(paramDef)) if (auto *paramDefInit = dyn_cast<llvm::DefInit>(paramDef))
return paramDefInit->getDef()->isSubClassOf("AttributeSelfTypeParameter"); return paramDefInit->getDef()->isSubClassOf("AttributeSelfTypeParameter");
return false; return false;
} }

mlir/lib/TableGen/Attribute.cpp

Show First 20 Lines • Show All 67 Lines • ▼ Show 20 LinesStringRef Attribute::getReturnType() const {
return getValueAsString(init); return getValueAsString(init);
} }
// Return the type constraint corresponding to the type of this attribute, or // Return the type constraint corresponding to the type of this attribute, or
// None if this is not a TypedAttr. // None if this is not a TypedAttr.
llvm::Optional<Type> Attribute::getValueType() const { llvm::Optional<Type> Attribute::getValueType() const {
if (auto *defInit = dyn_cast<llvm::DefInit>(def->getValueInit("valueType"))) if (auto *defInit = dyn_cast<llvm::DefInit>(def->getValueInit("valueType")))
return Type(defInit->getDef()); return Type(defInit->getDef());
return llvm::None; return std::nullopt;
} }
StringRef Attribute::getConvertFromStorageCall() const { StringRef Attribute::getConvertFromStorageCall() const {
const auto *init = def->getValueInit("convertFromStorage"); const auto *init = def->getValueInit("convertFromStorage");
return getValueAsString(init); return getValueAsString(init);
} }
bool Attribute::isConstBuildable() const { bool Attribute::isConstBuildable() const {
▲ Show 20 Lines • Show All 160 Lines • Show Last 20 Lines

mlir/lib/TableGen/Builder.cpp

Show All 30 LinesStringRef Builder::Parameter::getCppType() const {
} }
return record->getValueAsString("type"); return record->getValueAsString("type");
} }
/// Return an optional string containing the default value to use for this /// Return an optional string containing the default value to use for this
/// parameter. /// parameter.
Optional<StringRef> Builder::Parameter::getDefaultValue() const { Optional<StringRef> Builder::Parameter::getDefaultValue() const {
if (isa<llvm::StringInit>(def)) if (isa<llvm::StringInit>(def))
return llvm::None; return std::nullopt;
const llvm::Record *record = cast<llvm::DefInit>(def)->getDef(); const llvm::Record *record = cast<llvm::DefInit>(def)->getDef();
Optional<StringRef> value = record->getValueAsOptionalString("defaultValue"); Optional<StringRef> value = record->getValueAsOptionalString("defaultValue");
return value && !value->empty() ? value : llvm::None; return value && !value->empty() ? value : std::nullopt;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Builder // Builder
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
Builder::Builder(const llvm::Record *record, ArrayRef<SMLoc> loc) Builder::Builder(const llvm::Record *record, ArrayRef<SMLoc> loc)
: def(record) { : def(record) {
Show All 21 Lines for (unsigned i = 0, e = dag->getNumArgs(); i < e; ++i) {
} }
parameters.emplace_back(param); parameters.emplace_back(param);
} }
} }
/// Return an optional string containing the body of the builder. /// Return an optional string containing the body of the builder.
Optional<StringRef> Builder::getBody() const { Optional<StringRef> Builder::getBody() const {
Optional<StringRef> body = def->getValueAsOptionalString("body"); Optional<StringRef> body = def->getValueAsOptionalString("body");
return body && !body->empty() ? body : llvm::None; return body && !body->empty() ? body : std::nullopt;
} }

mlir/lib/TableGen/Constraint.cpp

Show First 20 Lines • Show All 84 Lines • ▼ Show 20 Lines
Optional<StringRef> Constraint::getBaseDefName() const { Optional<StringRef> Constraint::getBaseDefName() const {
// Functor used to check a base def in the case where the current def is // Functor used to check a base def in the case where the current def is
// anonymous. // anonymous.
auto checkBaseDefFn = [&](StringRef baseName) -> Optional<StringRef> { auto checkBaseDefFn = [&](StringRef baseName) -> Optional<StringRef> {
if (const auto *defValue = def->getValue(baseName)) { if (const auto *defValue = def->getValue(baseName)) {
if (const auto *defInit = dyn_cast<llvm::DefInit>(defValue->getValue())) if (const auto *defInit = dyn_cast<llvm::DefInit>(defValue->getValue()))
return Constraint(defInit->getDef(), kind).getDefName(); return Constraint(defInit->getDef(), kind).getDefName();
} }
return llvm::None; return std::nullopt;
}; };
switch (kind) { switch (kind) {
case CK_Attr: case CK_Attr:
if (def->isAnonymous()) if (def->isAnonymous())
return checkBaseDefFn("baseAttr"); return checkBaseDefFn("baseAttr");
return llvm::None; return std::nullopt;
case CK_Type: case CK_Type:
if (def->isAnonymous()) if (def->isAnonymous())
return checkBaseDefFn("baseType"); return checkBaseDefFn("baseType");
return llvm::None; return std::nullopt;
default: default:
return llvm::None; return std::nullopt;
} }
} }
AppliedConstraint::AppliedConstraint(Constraint &&constraint, AppliedConstraint::AppliedConstraint(Constraint &&constraint,
llvm::StringRef self, llvm::StringRef self,
std::vector<std::string> &&entities) std::vector<std::string> &&entities)
: constraint(constraint), self(std::string(self)), : constraint(constraint), self(std::string(self)),
entities(std::move(entities)) {} entities(std::move(entities)) {}
Show All 31 Lines

mlir/lib/TableGen/Interfaces.cpp

Show First 20 Lines • Show All 114 Lines • ▼ Show 20 Linesllvm::Optional<StringRef> Interface::getExtraSharedClassDeclaration() const {
auto value = def->getValueAsString("extraSharedClassDeclaration"); auto value = def->getValueAsString("extraSharedClassDeclaration");
return value.empty() ? llvm::Optional<StringRef>() : value; return value.empty() ? llvm::Optional<StringRef>() : value;
} }
// Return the body for this method if it has one. // Return the body for this method if it has one.
llvm::Optional<StringRef> Interface::getVerify() const { llvm::Optional<StringRef> Interface::getVerify() const {
// Only OpInterface supports the verify method. // Only OpInterface supports the verify method.
if (!isa<OpInterface>(this)) if (!isa<OpInterface>(this))
return llvm::None; return std::nullopt;
auto value = def->getValueAsString("verify"); auto value = def->getValueAsString("verify");
return value.empty() ? llvm::Optional<StringRef>() : value; return value.empty() ? llvm::Optional<StringRef>() : value;
} }
bool Interface::verifyWithRegions() const { bool Interface::verifyWithRegions() const {
return def->getValueAsBit("verifyWithRegions"); return def->getValueAsBit("verifyWithRegions");
} }
Show All 23 Lines

mlir/lib/TableGen/Type.cpp

Show First 20 Lines • Show All 43 Lines • ▼ Show 20 Lines
Optional<StringRef> TypeConstraint::getBuilderCall() const { Optional<StringRef> TypeConstraint::getBuilderCall() const {
const llvm::Record *baseType = def; const llvm::Record *baseType = def;
if (isVariableLength()) if (isVariableLength())
baseType = baseType->getValueAsDef("baseType"); baseType = baseType->getValueAsDef("baseType");
// Check to see if this type constraint has a builder call. // Check to see if this type constraint has a builder call.
const llvm::RecordVal *builderCall = baseType->getValue("builderCall"); const llvm::RecordVal *builderCall = baseType->getValue("builderCall");
if (!builderCall || !builderCall->getValue()) if (!builderCall || !builderCall->getValue())
return llvm::None; return std::nullopt;
return TypeSwitch<llvm::Init *, Optional<StringRef>>(builderCall->getValue()) return TypeSwitch<llvm::Init *, Optional<StringRef>>(builderCall->getValue())
.Case<llvm::StringInit>([&](auto *init) { .Case<llvm::StringInit>([&](auto *init) {
StringRef value = init->getValue(); StringRef value = init->getValue();
return value.empty() ? Optional<StringRef>() : value; return value.empty() ? Optional<StringRef>() : value;
}) })
.Default([](auto *) { return llvm::None; }); .Default([](auto *) { return std::nullopt; });
} }
// Return the C++ class name for this type (which may just be ::mlir::Type). // Return the C++ class name for this type (which may just be ::mlir::Type).
std::string TypeConstraint::getCPPClassName() const { std::string TypeConstraint::getCPPClassName() const {
StringRef className = def->getValueAsString("cppClassName"); StringRef className = def->getValueAsString("cppClassName");
// If the class name is already namespace resolved, use it. // If the class name is already namespace resolved, use it.
if (className.contains("::")) if (className.contains("::"))
Show All 15 Lines

mlir/lib/Target/LLVMIR/DebugTranslation.cpp

Show First 20 Lines • Show All 119 Lines • ▼ Show 20 Linesllvm::DIDerivedType *DebugTranslation::translateImpl(DIDerivedTypeAttr attr) {
auto getMDStringOrNull = [&](StringAttr attr) -> llvm::MDString * { auto getMDStringOrNull = [&](StringAttr attr) -> llvm::MDString * {
return attr ? llvm::MDString::get(llvmCtx, attr) : nullptr; return attr ? llvm::MDString::get(llvmCtx, attr) : nullptr;
}; };
return llvm::DIDerivedType::get( return llvm::DIDerivedType::get(
llvmCtx, attr.getTag(), getMDStringOrNull(attr.getName()), llvmCtx, attr.getTag(), getMDStringOrNull(attr.getName()),
/*File=*/nullptr, /*Line=*/0, /*File=*/nullptr, /*Line=*/0,
/*Scope=*/nullptr, translate(attr.getBaseType()), attr.getSizeInBits(), /*Scope=*/nullptr, translate(attr.getBaseType()), attr.getSizeInBits(),
attr.getAlignInBits(), attr.getOffsetInBits(), attr.getAlignInBits(), attr.getOffsetInBits(),
/*DWARFAddressSpace=*/llvm::None, /*Flags=*/llvm::DINode::FlagZero); /*DWARFAddressSpace=*/std::nullopt, /*Flags=*/llvm::DINode::FlagZero);
} }
llvm::DIFile *DebugTranslation::translateImpl(DIFileAttr attr) { llvm::DIFile *DebugTranslation::translateImpl(DIFileAttr attr) {
return llvm::DIFile::get(llvmCtx, attr.getName(), attr.getDirectory()); return llvm::DIFile::get(llvmCtx, attr.getName(), attr.getDirectory());
} }
llvm::DILexicalBlock *DebugTranslation::translateImpl(DILexicalBlockAttr attr) { llvm::DILexicalBlock *DebugTranslation::translateImpl(DILexicalBlockAttr attr) {
return llvm::DILexicalBlock::getDistinct(llvmCtx, translate(attr.getScope()), return llvm::DILexicalBlock::getDistinct(llvmCtx, translate(attr.getScope()),
▲ Show 20 Lines • Show All 200 Lines • Show Last 20 Lines

mlir/lib/Target/LLVMIR/Dialect/LLVMIR/LLVMToLLVMIRTranslation.cpp

Show First 20 Lines • Show All 207 Lines • ▼ Show 20 Linesstatic void setLoopMetadata(Operation &opInst, llvm::Instruction &llvmInst,
if (Attribute attr = opInst.getAttr(LLVMDialect::getLoopAttrName())) { if (Attribute attr = opInst.getAttr(LLVMDialect::getLoopAttrName())) {
llvm::Module *module = builder.GetInsertBlock()->getModule(); llvm::Module *module = builder.GetInsertBlock()->getModule();
llvm::MDNode *loopMD = moduleTranslation.lookupLoopOptionsMetadata(attr); llvm::MDNode *loopMD = moduleTranslation.lookupLoopOptionsMetadata(attr);
if (!loopMD) { if (!loopMD) {
llvm::LLVMContext &ctx = module->getContext(); llvm::LLVMContext &ctx = module->getContext();
SmallVector<llvm::Metadata *> loopOptions; SmallVector<llvm::Metadata *> loopOptions;
// Reserve operand 0 for loop id self reference. // Reserve operand 0 for loop id self reference.
auto dummy = llvm::MDNode::getTemporary(ctx, llvm::None); auto dummy = llvm::MDNode::getTemporary(ctx, std::nullopt);
loopOptions.push_back(dummy.get()); loopOptions.push_back(dummy.get());
auto loopAttr = attr.cast<DictionaryAttr>(); auto loopAttr = attr.cast<DictionaryAttr>();
auto parallelAccessGroup = auto parallelAccessGroup =
loopAttr.getNamed(LLVMDialect::getParallelAccessAttrName()); loopAttr.getNamed(LLVMDialect::getParallelAccessAttrName());
if (parallelAccessGroup) { if (parallelAccessGroup) {
SmallVector<llvm::Metadata *> parallelAccess; SmallVector<llvm::Metadata *> parallelAccess;
parallelAccess.push_back( parallelAccess.push_back(
▲ Show 20 Lines • Show All 357 Lines • Show Last 20 Lines

mlir/lib/Target/LLVMIR/ModuleTranslation.cpp

Show First 20 Lines • Show All 50 Lines • ▼ Show 20 Lines
#include "mlir/Dialect/LLVMIR/LLVMConversionEnumsToLLVM.inc" #include "mlir/Dialect/LLVMIR/LLVMConversionEnumsToLLVM.inc"
/// Translates the given data layout spec attribute to the LLVM IR data layout. /// Translates the given data layout spec attribute to the LLVM IR data layout.
/// Only integer, float, pointer and endianness entries are currently supported. /// Only integer, float, pointer and endianness entries are currently supported.
static FailureOr<llvm::DataLayout> static FailureOr<llvm::DataLayout>
translateDataLayout(DataLayoutSpecInterface attribute, translateDataLayout(DataLayoutSpecInterface attribute,
const DataLayout &dataLayout, const DataLayout &dataLayout,
Optional<Location> loc = llvm::None) { Optional<Location> loc = std::nullopt) {
if (!loc) if (!loc)
loc = UnknownLoc::get(attribute.getContext()); loc = UnknownLoc::get(attribute.getContext());
// Translate the endianness attribute. // Translate the endianness attribute.
std::string llvmDataLayout; std::string llvmDataLayout;
llvm::raw_string_ostream layoutStream(llvmDataLayout); llvm::raw_string_ostream layoutStream(llvmDataLayout);
for (DataLayoutEntryInterface entry : attribute.getEntries()) { for (DataLayoutEntryInterface entry : attribute.getEntries()) {
auto key = entry.getKey().dyn_cast<StringAttr>(); auto key = entry.getKey().dyn_cast<StringAttr>();
▲ Show 20 Lines • Show All 1,211 Lines • Show Last 20 Lines

mlir/lib/Target/SPIRV/Deserialization/Deserializer.h

Show First 20 Lines • Show All 415 Lines • ▼ Show 20 Linesprivate:
Value getValue(uint32_t id); Value getValue(uint32_t id);
/// Slices the first instruction out of `binary` and returns its opcode and /// Slices the first instruction out of `binary` and returns its opcode and
/// operands via `opcode` and `operands` respectively. Returns failure if /// operands via `opcode` and `operands` respectively. Returns failure if
/// there is no more remaining instructions (`expectedOpcode` will be used to /// there is no more remaining instructions (`expectedOpcode` will be used to
/// compose the error message) or the next instruction is malformed. /// compose the error message) or the next instruction is malformed.
LogicalResult LogicalResult
sliceInstruction(spirv::Opcode &opcode, ArrayRef<uint32_t> &operands, sliceInstruction(spirv::Opcode &opcode, ArrayRef<uint32_t> &operands,
Optional<spirv::Opcode> expectedOpcode = llvm::None); Optional<spirv::Opcode> expectedOpcode = std::nullopt);
/// Processes a SPIR-V instruction with the given `opcode` and `operands`. /// Processes a SPIR-V instruction with the given `opcode` and `operands`.
/// This method is the main entrance for handling SPIR-V instruction; it /// This method is the main entrance for handling SPIR-V instruction; it
/// checks the instruction opcode and dispatches to the corresponding handler. /// checks the instruction opcode and dispatches to the corresponding handler.
/// Processing of Some instructions (like OpEntryPoint and OpExecutionMode) /// Processing of Some instructions (like OpEntryPoint and OpExecutionMode)
/// might need to be deferred, since they contain forward references to <id>s /// might need to be deferred, since they contain forward references to <id>s
/// in the deserialized binary, but module in SPIR-V dialect expects these to /// in the deserialized binary, but module in SPIR-V dialect expects these to
/// be ssa-uses. /// be ssa-uses.
▲ Show 20 Lines • Show All 182 Lines • Show Last 20 Lines

mlir/lib/Target/SPIRV/Deserialization/Deserializer.cpp

Show First 20 Lines • Show All 484 Lines • ▼ Show 20 Linesspirv::Deserializer::processFunctionEnd(ArrayRef<uint32_t> operands) {
// Wire up block arguments from OpPhi instructions. // Wire up block arguments from OpPhi instructions.
// Put all structured control flow in spirv.mlir.selection/spirv.mlir.loop // Put all structured control flow in spirv.mlir.selection/spirv.mlir.loop
// ops. // ops.
if (failed(wireUpBlockArgument()) || failed(structurizeControlFlow())) { if (failed(wireUpBlockArgument()) || failed(structurizeControlFlow())) {
return failure(); return failure();
} }
curBlock = nullptr; curBlock = nullptr;
curFunction = llvm::None; curFunction = std::nullopt;
LLVM_DEBUG({ LLVM_DEBUG({
logger.unindent(); logger.unindent();
logger.startLine() logger.startLine()
<< "//===-------------------------------------------===//\n"; << "//===-------------------------------------------===//\n";
}); });
return success(); return success();
} }
Optional<std::pair<Attribute, Type>> Optional<std::pair<Attribute, Type>>
spirv::Deserializer::getConstant(uint32_t id) { spirv::Deserializer::getConstant(uint32_t id) {
auto constIt = constantMap.find(id); auto constIt = constantMap.find(id);
if (constIt == constantMap.end()) if (constIt == constantMap.end())
return llvm::None; return std::nullopt;
return constIt->getSecond(); return constIt->getSecond();
} }
Optional<spirv::SpecConstOperationMaterializationInfo> Optional<spirv::SpecConstOperationMaterializationInfo>
spirv::Deserializer::getSpecConstantOperation(uint32_t id) { spirv::Deserializer::getSpecConstantOperation(uint32_t id) {
auto constIt = specConstOperationMap.find(id); auto constIt = specConstOperationMap.find(id);
if (constIt == specConstOperationMap.end()) if (constIt == specConstOperationMap.end())
return llvm::None; return std::nullopt;
return constIt->getSecond(); return constIt->getSecond();
} }
std::string spirv::Deserializer::getFunctionSymbol(uint32_t id) { std::string spirv::Deserializer::getFunctionSymbol(uint32_t id) {
auto funcName = nameMap.lookup(id).str(); auto funcName = nameMap.lookup(id).str();
if (funcName.empty()) { if (funcName.empty()) {
funcName = "spirv_fn_" + std::to_string(id); funcName = "spirv_fn_" + std::to_string(id);
} }
▲ Show 20 Lines • Show All 1,574 Lines • ▼ Show 20 Linesspirv::Deserializer::processDebugLine(ArrayRef<uint32_t> operands) {
// following: the next end of block, the next OpLine instruction, or the next // following: the next end of block, the next OpLine instruction, or the next
// OpNoLine instruction." // OpNoLine instruction."
if (operands.size() != 3) if (operands.size() != 3)
return emitError(unknownLoc, "OpLine must have 3 operands"); return emitError(unknownLoc, "OpLine must have 3 operands");
debugLine = DebugLine{operands[0], operands[1], operands[2]}; debugLine = DebugLine{operands[0], operands[1], operands[2]};
return success(); return success();
} }
void spirv::Deserializer::clearDebugLine() { debugLine = llvm::None; } void spirv::Deserializer::clearDebugLine() { debugLine = std::nullopt; }
LogicalResult LogicalResult
spirv::Deserializer::processDebugString(ArrayRef<uint32_t> operands) { spirv::Deserializer::processDebugString(ArrayRef<uint32_t> operands) {
if (operands.size() < 2) if (operands.size() < 2)
return emitError(unknownLoc, "OpString needs at least 2 operands"); return emitError(unknownLoc, "OpString needs at least 2 operands");
if (!debugInfoMap.lookup(operands[0]).empty()) if (!debugInfoMap.lookup(operands[0]).empty())
return emitError(unknownLoc, return emitError(unknownLoc,
Show All 12 Lines

mlir/lib/Tools/PDLL/CodeGen/MLIRGen.cpp

Show First 20 Lines • Show All 340 Lines • ▼ Show 20 Lines if (type.isa<ast::AttributeType>())
return builder.create<pdl::AttributeOp>(loc, getTypeConstraint()); return builder.create<pdl::AttributeOp>(loc, getTypeConstraint());
if (ast::OperationType opType = type.dyn_cast<ast::OperationType>()) { if (ast::OperationType opType = type.dyn_cast<ast::OperationType>()) {
Value operands = builder.create<pdl::OperandsOp>( Value operands = builder.create<pdl::OperandsOp>(
loc, pdl::RangeType::get(builder.getType<pdl::ValueType>()), loc, pdl::RangeType::get(builder.getType<pdl::ValueType>()),
/*type=*/Value()); /*type=*/Value());
Value results = builder.create<pdl::TypesOp>( Value results = builder.create<pdl::TypesOp>(
loc, pdl::RangeType::get(builder.getType<pdl::TypeType>()), loc, pdl::RangeType::get(builder.getType<pdl::TypeType>()),
/*types=*/ArrayAttr()); /*types=*/ArrayAttr());
return builder.create<pdl::OperationOp>(loc, opType.getName(), operands, return builder.create<pdl::OperationOp>(
llvm::None, ValueRange(), results); loc, opType.getName(), operands, std::nullopt, ValueRange(), results);
} }
if (ast::RangeType rangeTy = type.dyn_cast<ast::RangeType>()) { if (ast::RangeType rangeTy = type.dyn_cast<ast::RangeType>()) {
ast::Type eleTy = rangeTy.getElementType(); ast::Type eleTy = rangeTy.getElementType();
if (eleTy.isa<ast::ValueType>()) if (eleTy.isa<ast::ValueType>())
return builder.create<pdl::OperandsOp>(loc, mlirType, return builder.create<pdl::OperandsOp>(loc, mlirType,
getTypeConstraint()); getTypeConstraint());
if (eleTy.isa<ast::TypeType>()) if (eleTy.isa<ast::TypeType>())
▲ Show 20 Lines • Show All 268 Lines • Show Last 20 Lines

mlir/lib/Tools/PDLL/Parser/Parser.cpp

Show First 20 Lines • Show All 975 Lines • ▼ Show 20 Lines auto *paramVar = ast::VariableDecl::create(
ctx, ast::Name::create(ctx, "self", loc), type, ctx, ast::Name::create(ctx, "self", loc), type,
/*initExpr=*/nullptr, ast::ConstraintRef(ConstraintT::create(ctx, loc))); /*initExpr=*/nullptr, ast::ConstraintRef(ConstraintT::create(ctx, loc)));
argScope->add(paramVar); argScope->add(paramVar);
popDeclScope(); popDeclScope();
// Build the native constraint. // Build the native constraint.
auto *constraintDecl = ast::UserConstraintDecl::createNative( auto *constraintDecl = ast::UserConstraintDecl::createNative(
ctx, ast::Name::create(ctx, name, loc), paramVar, ctx, ast::Name::create(ctx, name, loc), paramVar,
/*results=*/llvm::None, codeBlock, ast::TupleType::get(ctx), nativeType); /*results=*/std::nullopt, codeBlock, ast::TupleType::get(ctx),
nativeType);
constraintDecl->setDocComment(ctx, docString); constraintDecl->setDocComment(ctx, docString);
curDeclScope->add(constraintDecl); curDeclScope->add(constraintDecl);
return constraintDecl; return constraintDecl;
} }
template <typename ConstraintT> template <typename ConstraintT>
ast::Decl * ast::Decl *
Parser::createODSNativePDLLConstraintDecl(const tblgen::Constraint &constraint, Parser::createODSNativePDLLConstraintDecl(const tblgen::Constraint &constraint,
▲ Show 20 Lines • Show All 785 Lines • ▼ Show 20 LinesParser::parseConstraint(Optional<SMRange> &typeConstraint,
default: default:
break; break;
} }
return emitError(loc, "expected identifier constraint"); return emitError(loc, "expected identifier constraint");
} }
FailureOr<ast::ConstraintRef> Parser::parseArgOrResultConstraint() { FailureOr<ast::ConstraintRef> Parser::parseArgOrResultConstraint() {
Optional<SMRange> typeConstraint; Optional<SMRange> typeConstraint;
return parseConstraint(typeConstraint, /*existingConstraints=*/llvm::None, return parseConstraint(typeConstraint, /*existingConstraints=*/std::nullopt,
/*allowInlineTypeConstraints=*/false); /*allowInlineTypeConstraints=*/false);
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Exprs // Exprs
FailureOr<ast::Expr *> Parser::parseExpr() { FailureOr<ast::Expr *> Parser::parseExpr() {
if (curToken.is(Token::underscore)) if (curToken.is(Token::underscore))
▲ Show 20 Lines • Show All 1,059 Lines • ▼ Show 20 Lines
} }
LogicalResult LogicalResult
Parser::validateOperationOperands(SMRange loc, Optional<StringRef> name, Parser::validateOperationOperands(SMRange loc, Optional<StringRef> name,
const ods::Operation *odsOp, const ods::Operation *odsOp,
SmallVectorImpl<ast::Expr *> &operands) { SmallVectorImpl<ast::Expr *> &operands) {
return validateOperationOperandsOrResults( return validateOperationOperandsOrResults(
"operand", loc, odsOp ? odsOp->getLoc() : Optional<SMRange>(), name, "operand", loc, odsOp ? odsOp->getLoc() : Optional<SMRange>(), name,
operands, odsOp ? odsOp->getOperands() : llvm::None, valueTy, operands, odsOp ? odsOp->getOperands() : std::nullopt, valueTy,
valueRangeTy); valueRangeTy);
} }
LogicalResult LogicalResult
Parser::validateOperationResults(SMRange loc, Optional<StringRef> name, Parser::validateOperationResults(SMRange loc, Optional<StringRef> name,
const ods::Operation *odsOp, const ods::Operation *odsOp,
SmallVectorImpl<ast::Expr *> &results) { SmallVectorImpl<ast::Expr *> &results) {
return validateOperationOperandsOrResults( return validateOperationOperandsOrResults(
"result", loc, odsOp ? odsOp->getLoc() : Optional<SMRange>(), name, "result", loc, odsOp ? odsOp->getLoc() : Optional<SMRange>(), name,
results, odsOp ? odsOp->getResults() : llvm::None, typeTy, typeRangeTy); results, odsOp ? odsOp->getResults() : std::nullopt, typeTy, typeRangeTy);
} }
void Parser::checkOperationResultTypeInferrence(SMRange loc, StringRef opName, void Parser::checkOperationResultTypeInferrence(SMRange loc, StringRef opName,
const ods::Operation *odsOp) { const ods::Operation *odsOp) {
// If the operation might not have inferrence support, emit a warning to the // If the operation might not have inferrence support, emit a warning to the
// user. We don't emit an error because the interface might be added to the // user. We don't emit an error because the interface might be added to the
// operation at runtime. It's rare, but it could still happen. We emit a // operation at runtime. It's rare, but it could still happen. We emit a
// warning here instead. // warning here instead.
▲ Show 20 Lines • Show All 82 Lines • ▼ Show 20 Lines if (values.empty()) {
// If we only have one variadic value, just use an empty list. // If we only have one variadic value, just use an empty list.
if (numVariadic == 1) if (numVariadic == 1)
return success(); return success();
// Otherwise, create dummy values for each of the entries so that we // Otherwise, create dummy values for each of the entries so that we
// adhere to the ODS signature. // adhere to the ODS signature.
for (unsigned i = 0, e = odsValues.size(); i < e; ++i) { for (unsigned i = 0, e = odsValues.size(); i < e; ++i) {
values.push_back( values.push_back(ast::RangeExpr::create(
ast::RangeExpr::create(ctx, loc, /*elements=*/llvm::None, rangeTy)); ctx, loc, /*elements=*/std::nullopt, rangeTy));
} }
return success(); return success();
} }
// Verify that the number of values provided matches the number of value // Verify that the number of values provided matches the number of value
// groups ODS expects. // groups ODS expects.
if (odsValues.size() != values.size()) if (odsValues.size() != values.size())
return emitSizeMismatchError(); return emitSizeMismatchError();
▲ Show 20 Lines • Show All 200 Lines • Show Last 20 Lines

mlir/lib/Tools/lsp-server-support/SourceMgrUtils.cpp

Show First 20 Lines • Show All 65 Lines • ▼ Show 20 Lines
} }
Optional<std::string> lsp::extractSourceDocComment(llvm::SourceMgr &sourceMgr, Optional<std::string> lsp::extractSourceDocComment(llvm::SourceMgr &sourceMgr,
SMLoc loc) { SMLoc loc) {
// This is a heuristic, and isn't intended to cover every case, but should // This is a heuristic, and isn't intended to cover every case, but should
// cover the most common. We essentially look for a comment preceding the // cover the most common. We essentially look for a comment preceding the
// line, and if we find one, use that as the documentation. // line, and if we find one, use that as the documentation.
if (!loc.isValid()) if (!loc.isValid())
return llvm::None; return std::nullopt;
int bufferId = sourceMgr.FindBufferContainingLoc(loc); int bufferId = sourceMgr.FindBufferContainingLoc(loc);
if (bufferId == 0) if (bufferId == 0)
return llvm::None; return std::nullopt;
const char *bufferStart = const char *bufferStart =
sourceMgr.getMemoryBuffer(bufferId)->getBufferStart(); sourceMgr.getMemoryBuffer(bufferId)->getBufferStart();
StringRef buffer(bufferStart, loc.getPointer() - bufferStart); StringRef buffer(bufferStart, loc.getPointer() - bufferStart);
// Pop the last line from the buffer string. // Pop the last line from the buffer string.
auto popLastLine = [&]() -> Optional<StringRef> { auto popLastLine = [&]() -> Optional<StringRef> {
size_t newlineOffset = buffer.find_last_of("\n"); size_t newlineOffset = buffer.find_last_of("\n");
if (newlineOffset == StringRef::npos) if (newlineOffset == StringRef::npos)
return llvm::None; return std::nullopt;
StringRef lastLine = buffer.drop_front(newlineOffset).trim(); StringRef lastLine = buffer.drop_front(newlineOffset).trim();
buffer = buffer.take_front(newlineOffset); buffer = buffer.take_front(newlineOffset);
return lastLine; return lastLine;
}; };
// Try to pop the current line. // Try to pop the current line.
if (!popLastLine()) if (!popLastLine())
return llvm::None; return std::nullopt;
// Try to parse a comment string from the source file. // Try to parse a comment string from the source file.
SmallVector<StringRef> commentLines; SmallVector<StringRef> commentLines;
while (Optional<StringRef> line = popLastLine()) { while (Optional<StringRef> line = popLastLine()) {
// Check for a comment at the beginning of the line. // Check for a comment at the beginning of the line.
if (!line->startswith("//")) if (!line->startswith("//"))
break; break;
// Extract the document string from the comment. // Extract the document string from the comment.
commentLines.push_back(line->drop_while([](char c) { return c == '/'; })); commentLines.push_back(line->drop_while([](char c) { return c == '/'; }));
} }
if (commentLines.empty()) if (commentLines.empty())
return llvm::None; return std::nullopt;
return llvm::join(llvm::reverse(commentLines), "\n"); return llvm::join(llvm::reverse(commentLines), "\n");
} }
bool lsp::contains(SMRange range, SMLoc loc) { bool lsp::contains(SMRange range, SMLoc loc) {
return range.Start.getPointer() <= loc.getPointer() && return range.Start.getPointer() <= loc.getPointer() &&
loc.getPointer() < range.End.getPointer(); loc.getPointer() < range.End.getPointer();
} }
▲ Show 20 Lines • Show All 42 Lines • Show Last 20 Lines

mlir/lib/Tools/mlir-lsp-server/MLIRServer.cpp

Show All 26 Lines
static Optional<lsp::Location> getLocationFromLoc(StringRef uriScheme, static Optional<lsp::Location> getLocationFromLoc(StringRef uriScheme,
FileLineColLoc loc) { FileLineColLoc loc) {
llvm::Expected<lsp::URIForFile> sourceURI = llvm::Expected<lsp::URIForFile> sourceURI =
lsp::URIForFile::fromFile(loc.getFilename(), uriScheme); lsp::URIForFile::fromFile(loc.getFilename(), uriScheme);
if (!sourceURI) { if (!sourceURI) {
lsp::Logger::error("Failed to create URI for file `{0}`: {1}", lsp::Logger::error("Failed to create URI for file `{0}`: {1}",
loc.getFilename(), loc.getFilename(),
llvm::toString(sourceURI.takeError())); llvm::toString(sourceURI.takeError()));
return llvm::None; return std::nullopt;
} }
lsp::Position position; lsp::Position position;
position.line = loc.getLine() - 1; position.line = loc.getLine() - 1;
position.character = loc.getColumn() ? loc.getColumn() - 1 : 0; position.character = loc.getColumn() ? loc.getColumn() - 1 : 0;
return lsp::Location{*sourceURI, lsp::Range(position)}; return lsp::Location{*sourceURI, lsp::Range(position)};
} }
▲ Show 20 Lines • Show All 91 Lines • ▼ Show 20 Linesstatic Optional<unsigned> getResultNumberFromLoc(SMLoc loc) {
}; };
const char *curPtr = loc.getPointer(); const char *curPtr = loc.getPointer();
while (isIdentifierChar(*curPtr)) while (isIdentifierChar(*curPtr))
++curPtr; ++curPtr;
// Check to see if this location indexes into the result group, via `#`. If it // Check to see if this location indexes into the result group, via `#`. If it
// doesn't, we can't extract a sub result number. // doesn't, we can't extract a sub result number.
if (*curPtr != '#') if (*curPtr != '#')
return llvm::None; return std::nullopt;
// Compute the sub result number from the remaining portion of the string. // Compute the sub result number from the remaining portion of the string.
const char *numberStart = ++curPtr; const char *numberStart = ++curPtr;
while (llvm::isDigit(*curPtr)) while (llvm::isDigit(*curPtr))
++curPtr; ++curPtr;
StringRef numberStr(numberStart, curPtr - numberStart); StringRef numberStr(numberStart, curPtr - numberStart);
unsigned resultNumber = 0; unsigned resultNumber = 0;
return numberStr.consumeInteger(10, resultNumber) ? Optional<unsigned>() return numberStr.consumeInteger(10, resultNumber) ? Optional<unsigned>()
▲ Show 20 Lines • Show All 333 Lines • ▼ Show 20 Lines for (const AsmParserState::BlockDefinition &block : asmState.getBlockDefs()) {
for (const auto &arg : llvm::enumerate(block.arguments)) { for (const auto &arg : llvm::enumerate(block.arguments)) {
if (!isDefOrUse(arg.value(), posLoc, &hoverRange)) if (!isDefOrUse(arg.value(), posLoc, &hoverRange))
continue; continue;
return buildHoverForBlockArgument( return buildHoverForBlockArgument(
hoverRange, block.block->getArgument(arg.index()), block); hoverRange, block.block->getArgument(arg.index()), block);
} }
} }
return llvm::None; return std::nullopt;
} }
Optional<lsp::Hover> MLIRDocument::buildHoverForOperation( Optional<lsp::Hover> MLIRDocument::buildHoverForOperation(
SMRange hoverRange, const AsmParserState::OperationDefinition &op) { SMRange hoverRange, const AsmParserState::OperationDefinition &op) {
lsp::Hover hover(lsp::Range(sourceMgr, hoverRange)); lsp::Hover hover(lsp::Range(sourceMgr, hoverRange));
llvm::raw_string_ostream os(hover.contents.value); llvm::raw_string_ostream os(hover.contents.value);
// Add the operation name to the hover. // Add the operation name to the hover.
▲ Show 20 Lines • Show All 720 Lines • ▼ Show 20 Linesvoid lsp::MLIRServer::addOrUpdateDocument(
std::vector<Diagnostic> &diagnostics) { std::vector<Diagnostic> &diagnostics) {
impl->files[uri.file()] = std::make_unique<MLIRTextFile>( impl->files[uri.file()] = std::make_unique<MLIRTextFile>(
uri, contents, version, impl->registry, diagnostics); uri, contents, version, impl->registry, diagnostics);
} }
Optional<int64_t> lsp::MLIRServer::removeDocument(const URIForFile &uri) { Optional<int64_t> lsp::MLIRServer::removeDocument(const URIForFile &uri) {
auto it = impl->files.find(uri.file()); auto it = impl->files.find(uri.file());
if (it == impl->files.end()) if (it == impl->files.end())
return llvm::None; return std::nullopt;
int64_t version = it->second->getVersion(); int64_t version = it->second->getVersion();
impl->files.erase(it); impl->files.erase(it);
return version; return version;
} }
void lsp::MLIRServer::getLocationsOf(const URIForFile &uri, void lsp::MLIRServer::getLocationsOf(const URIForFile &uri,
const Position &defPos, const Position &defPos,
Show All 11 Lines if (fileIt != impl->files.end())
fileIt->second->findReferencesOf(uri, pos, references); fileIt->second->findReferencesOf(uri, pos, references);
} }
Optional<lsp::Hover> lsp::MLIRServer::findHover(const URIForFile &uri, Optional<lsp::Hover> lsp::MLIRServer::findHover(const URIForFile &uri,
const Position &hoverPos) { const Position &hoverPos) {
auto fileIt = impl->files.find(uri.file()); auto fileIt = impl->files.find(uri.file());
if (fileIt != impl->files.end()) if (fileIt != impl->files.end())
return fileIt->second->findHover(uri, hoverPos); return fileIt->second->findHover(uri, hoverPos);
return llvm::None; return std::nullopt;
} }
void lsp::MLIRServer::findDocumentSymbols( void lsp::MLIRServer::findDocumentSymbols(
const URIForFile &uri, std::vector<DocumentSymbol> &symbols) { const URIForFile &uri, std::vector<DocumentSymbol> &symbols) {
auto fileIt = impl->files.find(uri.file()); auto fileIt = impl->files.find(uri.file());
if (fileIt != impl->files.end()) if (fileIt != impl->files.end())
fileIt->second->findDocumentSymbols(symbols); fileIt->second->findDocumentSymbols(symbols);
} }
▲ Show 20 Lines • Show All 79 Lines • Show Last 20 Lines

mlir/lib/Tools/mlir-pdll-lsp-server/PDLLServer.cpp

Show First 20 Lines • Show All 73 Lines • ▼ Show 20 LinesgetLspDiagnoticFromDiag(llvm::SourceMgr &sourceMgr, const ast::Diagnostic &diag,
lspDiag.category = "Parse Error"; lspDiag.category = "Parse Error";
// Try to grab a file location for this diagnostic. // Try to grab a file location for this diagnostic.
lsp::Location loc = getLocationFromLoc(sourceMgr, diag.getLocation(), uri); lsp::Location loc = getLocationFromLoc(sourceMgr, diag.getLocation(), uri);
lspDiag.range = loc.range; lspDiag.range = loc.range;
// Skip diagnostics that weren't emitted within the main file. // Skip diagnostics that weren't emitted within the main file.
if (loc.uri != uri) if (loc.uri != uri)
return llvm::None; return std::nullopt;
// Convert the severity for the diagnostic. // Convert the severity for the diagnostic.
switch (diag.getSeverity()) { switch (diag.getSeverity()) {
case ast::Diagnostic::Severity::DK_Note: case ast::Diagnostic::Severity::DK_Note:
llvm_unreachable("expected notes to be handled separately"); llvm_unreachable("expected notes to be handled separately");
case ast::Diagnostic::Severity::DK_Warning: case ast::Diagnostic::Severity::DK_Warning:
lspDiag.severity = lsp::DiagnosticSeverity::Warning; lspDiag.severity = lsp::DiagnosticSeverity::Warning;
break; break;
▲ Show 20 Lines • Show All 364 Lines • ▼ Show 20 LinesOptional<lsp::Hover> PDLDocument::findHover(const lsp::URIForFile &uri,
for (const lsp::SourceMgrInclude &include : parsedIncludes) for (const lsp::SourceMgrInclude &include : parsedIncludes)
if (include.range.contains(hoverPos)) if (include.range.contains(hoverPos))
return include.buildHover(); return include.buildHover();
// Find the symbol at the given location. // Find the symbol at the given location.
SMRange hoverRange; SMRange hoverRange;
const PDLIndexSymbol *symbol = index.lookup(posLoc, &hoverRange); const PDLIndexSymbol *symbol = index.lookup(posLoc, &hoverRange);
if (!symbol) if (!symbol)
return llvm::None; return std::nullopt;
// Add hover for operation names. // Add hover for operation names.
if (const auto *op = symbol->definition.dyn_cast<const ods::Operation *>()) if (const auto *op = symbol->definition.dyn_cast<const ods::Operation *>())
return buildHoverForOpName(op, hoverRange); return buildHoverForOpName(op, hoverRange);
const auto *decl = symbol->definition.get<const ast::Decl *>(); const auto *decl = symbol->definition.get<const ast::Decl *>();
return findHover(decl, hoverRange); return findHover(decl, hoverRange);
} }
Show All 14 LinesOptional<lsp::Hover> PDLDocument::findHover(const ast::Decl *decl,
// Add hover for user constraints. // Add hover for user constraints.
if (const auto *cst = dyn_cast<ast::UserConstraintDecl>(decl)) if (const auto *cst = dyn_cast<ast::UserConstraintDecl>(decl))
return buildHoverForUserConstraintOrRewrite("Constraint", cst, hoverRange); return buildHoverForUserConstraintOrRewrite("Constraint", cst, hoverRange);
// Add hover for user rewrites. // Add hover for user rewrites.
if (const auto *rewrite = dyn_cast<ast::UserRewriteDecl>(decl)) if (const auto *rewrite = dyn_cast<ast::UserRewriteDecl>(decl))
return buildHoverForUserConstraintOrRewrite("Rewrite", rewrite, hoverRange); return buildHoverForUserConstraintOrRewrite("Rewrite", rewrite, hoverRange);
return llvm::None; return std::nullopt;
} }
lsp::Hover PDLDocument::buildHoverForOpName(const ods::Operation *op, lsp::Hover PDLDocument::buildHoverForOpName(const ods::Operation *op,
const SMRange &hoverRange) { const SMRange &hoverRange) {
lsp::Hover hover(lsp::Range(sourceMgr, hoverRange)); lsp::Hover hover(lsp::Range(sourceMgr, hoverRange));
{ {
llvm::raw_string_ostream hoverOS(hover.contents.value); llvm::raw_string_ostream hoverOS(hover.contents.value);
hoverOS << "**OpName**: `" << op->getName() << "`\n***\n" hoverOS << "**OpName**: `" << op->getName() << "`\n***\n"
▲ Show 20 Lines • Show All 529 Lines • ▼ Show 20 Linespublic:
} }
void void
codeCompleteOperationOperandsSignature(Optional<StringRef> opName, codeCompleteOperationOperandsSignature(Optional<StringRef> opName,
unsigned currentNumOperands) final { unsigned currentNumOperands) final {
const ods::Operation *odsOp = const ods::Operation *odsOp =
opName ? odsContext.lookupOperation(*opName) : nullptr; opName ? odsContext.lookupOperation(*opName) : nullptr;
codeCompleteOperationOperandOrResultSignature( codeCompleteOperationOperandOrResultSignature(
opName, odsOp, odsOp ? odsOp->getOperands() : llvm::None, opName, odsOp, odsOp ? odsOp->getOperands() : std::nullopt,
currentNumOperands, "operand", "Value"); currentNumOperands, "operand", "Value");
} }
void codeCompleteOperationResultsSignature(Optional<StringRef> opName, void codeCompleteOperationResultsSignature(Optional<StringRef> opName,
unsigned currentNumResults) final { unsigned currentNumResults) final {
const ods::Operation *odsOp = const ods::Operation *odsOp =
opName ? odsContext.lookupOperation(*opName) : nullptr; opName ? odsContext.lookupOperation(*opName) : nullptr;
codeCompleteOperationOperandOrResultSignature( codeCompleteOperationOperandOrResultSignature(
opName, odsOp, odsOp ? odsOp->getResults() : llvm::None, opName, odsOp, odsOp ? odsOp->getResults() : std::nullopt,
currentNumResults, "result", "Type"); currentNumResults, "result", "Type");
} }
void codeCompleteOperationOperandOrResultSignature( void codeCompleteOperationOperandOrResultSignature(
Optional<StringRef> opName, const ods::Operation *odsOp, Optional<StringRef> opName, const ods::Operation *odsOp,
ArrayRef<ods::OperandOrResult> values, unsigned currentValue, ArrayRef<ods::OperandOrResult> values, unsigned currentValue,
StringRef label, StringRef dataType) { StringRef label, StringRef dataType) {
signatureHelp.activeParameter = currentValue; signatureHelp.activeParameter = currentValue;
▲ Show 20 Lines • Show All 669 Lines • ▼ Show 20 Linesvoid lsp::PDLLServer::updateDocument(
// failed updated generally means we've fallen out of sync somewhere. // failed updated generally means we've fallen out of sync somewhere.
if (failed(it->second->update(uri, version, changes, diagnostics))) if (failed(it->second->update(uri, version, changes, diagnostics)))
impl->files.erase(it); impl->files.erase(it);
} }
Optional<int64_t> lsp::PDLLServer::removeDocument(const URIForFile &uri) { Optional<int64_t> lsp::PDLLServer::removeDocument(const URIForFile &uri) {
auto it = impl->files.find(uri.file()); auto it = impl->files.find(uri.file());
if (it == impl->files.end()) if (it == impl->files.end())
return llvm::None; return std::nullopt;
int64_t version = it->second->getVersion(); int64_t version = it->second->getVersion();
impl->files.erase(it); impl->files.erase(it);
return version; return version;
} }
void lsp::PDLLServer::getLocationsOf(const URIForFile &uri, void lsp::PDLLServer::getLocationsOf(const URIForFile &uri,
const Position &defPos, const Position &defPos,
Show All 18 Lines if (fileIt != impl->files.end())
return fileIt->second->getDocumentLinks(uri, documentLinks); return fileIt->second->getDocumentLinks(uri, documentLinks);
} }
Optional<lsp::Hover> lsp::PDLLServer::findHover(const URIForFile &uri, Optional<lsp::Hover> lsp::PDLLServer::findHover(const URIForFile &uri,
const Position &hoverPos) { const Position &hoverPos) {
auto fileIt = impl->files.find(uri.file()); auto fileIt = impl->files.find(uri.file());
if (fileIt != impl->files.end()) if (fileIt != impl->files.end())
return fileIt->second->findHover(uri, hoverPos); return fileIt->second->findHover(uri, hoverPos);
return llvm::None; return std::nullopt;
} }
void lsp::PDLLServer::findDocumentSymbols( void lsp::PDLLServer::findDocumentSymbols(
const URIForFile &uri, std::vector<DocumentSymbol> &symbols) { const URIForFile &uri, std::vector<DocumentSymbol> &symbols) {
auto fileIt = impl->files.find(uri.file()); auto fileIt = impl->files.find(uri.file());
if (fileIt != impl->files.end()) if (fileIt != impl->files.end())
fileIt->second->findDocumentSymbols(symbols); fileIt->second->findDocumentSymbols(symbols);
} }
Show All 29 Lines
} }
Optional<lsp::PDLLViewOutputResult> Optional<lsp::PDLLViewOutputResult>
lsp::PDLLServer::getPDLLViewOutput(const URIForFile &uri, lsp::PDLLServer::getPDLLViewOutput(const URIForFile &uri,
PDLLViewOutputKind kind) { PDLLViewOutputKind kind) {
auto fileIt = impl->files.find(uri.file()); auto fileIt = impl->files.find(uri.file());
if (fileIt != impl->files.end()) if (fileIt != impl->files.end())
return fileIt->second->getPDLLViewOutput(kind); return fileIt->second->getPDLLViewOutput(kind);
return llvm::None; return std::nullopt;
} }

mlir/lib/Tools/mlir-translate/Translation.cpp

Show First 20 Lines • Show All 56 Lines • ▼ Show 20 Lines llvm::report_fatal_error(
"Attempting to overwrite an existing <file-to-file> function"); "Attempting to overwrite an existing <file-to-file> function");
assert(function && assert(function &&
"Attempting to register an empty translate <file-to-file> function"); "Attempting to register an empty translate <file-to-file> function");
registry[name] = Translation(function, description, inputAlignment); registry[name] = Translation(function, description, inputAlignment);
} }
TranslateRegistration::TranslateRegistration( TranslateRegistration::TranslateRegistration(
StringRef name, StringRef description, const TranslateFunction &function) { StringRef name, StringRef description, const TranslateFunction &function) {
registerTranslation(name, description, /*inputAlignment=*/llvm::None, registerTranslation(name, description, /*inputAlignment=*/std::nullopt,
function); function);
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Translation to MLIR // Translation to MLIR
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Puts `function` into the to-MLIR translation registry unless there is already // Puts `function` into the to-MLIR translation registry unless there is already
Show All 37 Lines
// Translation from MLIR // Translation from MLIR
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
TranslateFromMLIRRegistration::TranslateFromMLIRRegistration( TranslateFromMLIRRegistration::TranslateFromMLIRRegistration(
StringRef name, StringRef description, StringRef name, StringRef description,
const TranslateFromMLIRFunction &function, const TranslateFromMLIRFunction &function,
const std::function<void(DialectRegistry &)> &dialectRegistration) { const std::function<void(DialectRegistry &)> &dialectRegistration) {
registerTranslation( registerTranslation(
name, description, /*inputAlignment=*/llvm::None, name, description, /*inputAlignment=*/std::nullopt,
[function, dialectRegistration](llvm::SourceMgr &sourceMgr, [function, dialectRegistration](llvm::SourceMgr &sourceMgr,
raw_ostream &output, raw_ostream &output,
MLIRContext *context) { MLIRContext *context) {
DialectRegistry registry; DialectRegistry registry;
dialectRegistration(registry); dialectRegistration(registry);
context->appendDialectRegistry(registry); context->appendDialectRegistry(registry);
bool implicitModule = bool implicitModule =
(!clOptions.isConstructed() || !clOptions->noImplicitModule); (!clOptions.isConstructed() || !clOptions->noImplicitModule);
Show All 28 Lines

mlir/lib/Tools/tblgen-lsp-server/TableGenServer.cpp

Show First 20 Lines • Show All 53 Lines • ▼ Show 20 Lines
} }
/// Convert the given TableGen diagnostic to the LSP form. /// Convert the given TableGen diagnostic to the LSP form.
static Optional<lsp::Diagnostic> static Optional<lsp::Diagnostic>
getLspDiagnoticFromDiag(const llvm::SMDiagnostic &diag, getLspDiagnoticFromDiag(const llvm::SMDiagnostic &diag,
const lsp::URIForFile &uri) { const lsp::URIForFile &uri) {
auto *sourceMgr = const_cast<llvm::SourceMgr *>(diag.getSourceMgr()); auto *sourceMgr = const_cast<llvm::SourceMgr *>(diag.getSourceMgr());
if (!sourceMgr || !diag.getLoc().isValid()) if (!sourceMgr || !diag.getLoc().isValid())
return llvm::None; return std::nullopt;
lsp::Diagnostic lspDiag; lsp::Diagnostic lspDiag;
lspDiag.source = "tablegen"; lspDiag.source = "tablegen";
lspDiag.category = "Parse Error"; lspDiag.category = "Parse Error";
// Try to grab a file location for this diagnostic. // Try to grab a file location for this diagnostic.
lsp::Location loc = getLocationFromLoc(*sourceMgr, diag.getLoc(), uri); lsp::Location loc = getLocationFromLoc(*sourceMgr, diag.getLoc(), uri);
lspDiag.range = loc.range; lspDiag.range = loc.range;
// Skip diagnostics that weren't emitted within the main file. // Skip diagnostics that weren't emitted within the main file.
if (loc.uri != uri) if (loc.uri != uri)
return llvm::None; return std::nullopt;
// Convert the severity for the diagnostic. // Convert the severity for the diagnostic.
switch (diag.getKind()) { switch (diag.getKind()) {
case llvm::SourceMgr::DK_Warning: case llvm::SourceMgr::DK_Warning:
lspDiag.severity = lsp::DiagnosticSeverity::Warning; lspDiag.severity = lsp::DiagnosticSeverity::Warning;
break; break;
case llvm::SourceMgr::DK_Error: case llvm::SourceMgr::DK_Error:
lspDiag.severity = lsp::DiagnosticSeverity::Error; lspDiag.severity = lsp::DiagnosticSeverity::Error;
▲ Show 20 Lines • Show All 443 Lines • ▼ Show 20 Lines for (const lsp::SourceMgrInclude &include : parsedIncludes)
if (include.range.contains(hoverPos)) if (include.range.contains(hoverPos))
return include.buildHover(); return include.buildHover();
// Find the symbol at the given location. // Find the symbol at the given location.
SMRange hoverRange; SMRange hoverRange;
SMLoc posLoc = hoverPos.getAsSMLoc(sourceMgr); SMLoc posLoc = hoverPos.getAsSMLoc(sourceMgr);
const TableGenIndexSymbol *symbol = index.lookup(posLoc, &hoverRange); const TableGenIndexSymbol *symbol = index.lookup(posLoc, &hoverRange);
if (!symbol) if (!symbol)
return llvm::None; return std::nullopt;
// Build hover for a Record. // Build hover for a Record.
if (auto *record = dyn_cast<TableGenRecordSymbol>(symbol)) if (auto *record = dyn_cast<TableGenRecordSymbol>(symbol))
return buildHoverForRecord(record->getValue(), hoverRange); return buildHoverForRecord(record->getValue(), hoverRange);
// Build hover for a RecordVal, which is either a template argument or a // Build hover for a RecordVal, which is either a template argument or a
// field. // field.
auto *recordVal = cast<TableGenRecordValSymbol>(symbol); auto *recordVal = cast<TableGenRecordValSymbol>(symbol);
▲ Show 20 Lines • Show All 145 Lines • ▼ Show 20 Linesvoid lsp::TableGenServer::updateDocument(
// failed updated generally means we've fallen out of sync somewhere. // failed updated generally means we've fallen out of sync somewhere.
if (failed(it->second->update(uri, version, changes, diagnostics))) if (failed(it->second->update(uri, version, changes, diagnostics)))
impl->files.erase(it); impl->files.erase(it);
} }
Optional<int64_t> lsp::TableGenServer::removeDocument(const URIForFile &uri) { Optional<int64_t> lsp::TableGenServer::removeDocument(const URIForFile &uri) {
auto it = impl->files.find(uri.file()); auto it = impl->files.find(uri.file());
if (it == impl->files.end()) if (it == impl->files.end())
return llvm::None; return std::nullopt;
int64_t version = it->second->getVersion(); int64_t version = it->second->getVersion();
impl->files.erase(it); impl->files.erase(it);
return version; return version;
} }
void lsp::TableGenServer::getLocationsOf(const URIForFile &uri, void lsp::TableGenServer::getLocationsOf(const URIForFile &uri,
const Position &defPos, const Position &defPos,
Show All 18 Lines if (fileIt != impl->files.end())
return fileIt->second->getDocumentLinks(uri, documentLinks); return fileIt->second->getDocumentLinks(uri, documentLinks);
} }
Optional<lsp::Hover> lsp::TableGenServer::findHover(const URIForFile &uri, Optional<lsp::Hover> lsp::TableGenServer::findHover(const URIForFile &uri,
const Position &hoverPos) { const Position &hoverPos) {
auto fileIt = impl->files.find(uri.file()); auto fileIt = impl->files.find(uri.file());
if (fileIt != impl->files.end()) if (fileIt != impl->files.end())
return fileIt->second->findHover(uri, hoverPos); return fileIt->second->findHover(uri, hoverPos);
return llvm::None; return std::nullopt;
} }

mlir/lib/Transforms/Utils/DialectConversion.cpp

Show First 20 Lines • Show All 1,565 Lines • ▼ Show 20 Lines impl->logger.startLine() << "** Replace Argument : '" << from
<< "'(" << from.getOwner()->getParentOp() << ")\n"; << "'(" << from.getOwner()->getParentOp() << ")\n";
}); });
impl->argReplacements.push_back(from); impl->argReplacements.push_back(from);
impl->mapping.map(impl->mapping.lookupOrDefault(from), to); impl->mapping.map(impl->mapping.lookupOrDefault(from), to);
} }
Value ConversionPatternRewriter::getRemappedValue(Value key) { Value ConversionPatternRewriter::getRemappedValue(Value key) {
SmallVector<Value> remappedValues; SmallVector<Value> remappedValues;
if (failed(impl->remapValues("value", /*inputLoc=*/llvm::None, *this, key, if (failed(impl->remapValues("value", /*inputLoc=*/std::nullopt, *this, key,
remappedValues))) remappedValues)))
return nullptr; return nullptr;
return remappedValues.front(); return remappedValues.front();
} }
LogicalResult LogicalResult
ConversionPatternRewriter::getRemappedValues(ValueRange keys, ConversionPatternRewriter::getRemappedValues(ValueRange keys,
SmallVectorImpl<Value> &results) { SmallVectorImpl<Value> &results) {
if (keys.empty()) if (keys.empty())
return success(); return success();
return impl->remapValues("value", /*inputLoc=*/llvm::None, *this, keys, return impl->remapValues("value", /*inputLoc=*/std::nullopt, *this, keys,
results); results);
} }
void ConversionPatternRewriter::notifyBlockCreated(Block *block) { void ConversionPatternRewriter::notifyBlockCreated(Block *block) {
impl->notifyCreatedBlock(block); impl->notifyCreatedBlock(block);
} }
Block *ConversionPatternRewriter::splitBlock(Block *block, Block *ConversionPatternRewriter::splitBlock(Block *block,
▲ Show 20 Lines • Show All 1,449 Lines • ▼ Show 20 Lines if (Optional<Value> result = fn(builder, resultType, inputs, loc))
return *result; return *result;
return nullptr; return nullptr;
} }
auto TypeConverter::convertBlockSignature(Block *block) auto TypeConverter::convertBlockSignature(Block *block)
-> Optional<SignatureConversion> { -> Optional<SignatureConversion> {
SignatureConversion conversion(block->getNumArguments()); SignatureConversion conversion(block->getNumArguments());
if (failed(convertSignatureArgs(block->getArgumentTypes(), conversion))) if (failed(convertSignatureArgs(block->getArgumentTypes(), conversion)))
return llvm::None; return std::nullopt;
return conversion; return conversion;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// FunctionOpInterfaceSignatureConversion // FunctionOpInterfaceSignatureConversion
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
static LogicalResult convertFuncOpTypes(FunctionOpInterface funcOp, static LogicalResult convertFuncOpTypes(FunctionOpInterface funcOp,
▲ Show 20 Lines • Show All 82 Lines • ▼ Show 20 Linesauto ConversionTarget::getOpAction(OperationName op) const
Optional<LegalizationInfo> info = getOpInfo(op); Optional<LegalizationInfo> info = getOpInfo(op);
return info ? info->action : Optional<LegalizationAction>(); return info ? info->action : Optional<LegalizationAction>();
} }
auto ConversionTarget::isLegal(Operation *op) const auto ConversionTarget::isLegal(Operation *op) const
-> Optional<LegalOpDetails> { -> Optional<LegalOpDetails> {
Optional<LegalizationInfo> info = getOpInfo(op->getName()); Optional<LegalizationInfo> info = getOpInfo(op->getName());
if (!info) if (!info)
return llvm::None; return std::nullopt;
// Returns true if this operation instance is known to be legal. // Returns true if this operation instance is known to be legal.
auto isOpLegal = [&] { auto isOpLegal = [&] {
// Handle dynamic legality either with the provided legality function. // Handle dynamic legality either with the provided legality function.
if (info->action == LegalizationAction::Dynamic) { if (info->action == LegalizationAction::Dynamic) {
Optional<bool> result = info->legalityFn(op); Optional<bool> result = info->legalityFn(op);
if (result) if (result)
return *result; return *result;
} }
// Otherwise, the operation is only legal if it was marked 'Legal'. // Otherwise, the operation is only legal if it was marked 'Legal'.
return info->action == LegalizationAction::Legal; return info->action == LegalizationAction::Legal;
}; };
if (!isOpLegal()) if (!isOpLegal())
return llvm::None; return std::nullopt;
// This operation is legal, compute any additional legality information. // This operation is legal, compute any additional legality information.
LegalOpDetails legalityDetails; LegalOpDetails legalityDetails;
if (info->isRecursivelyLegal) { if (info->isRecursivelyLegal) {
auto legalityFnIt = opRecursiveLegalityFns.find(op->getName()); auto legalityFnIt = opRecursiveLegalityFns.find(op->getName());
if (legalityFnIt != opRecursiveLegalityFns.end()) { if (legalityFnIt != opRecursiveLegalityFns.end()) {
legalityDetails.isRecursivelyLegal = legalityDetails.isRecursivelyLegal =
legalityFnIt->second(op).value_or(true); legalityFnIt->second(op).value_or(true);
▲ Show 20 Lines • Show All 90 Lines • ▼ Show 20 Lines if (dialectFn != dialectLegalityFns.end())
callback = dialectFn->second; callback = dialectFn->second;
return LegalizationInfo{dialectIt->second, /*isRecursivelyLegal=*/false, return LegalizationInfo{dialectIt->second, /*isRecursivelyLegal=*/false,
callback}; callback};
} }
// Otherwise, check if we mark unknown operations as dynamic. // Otherwise, check if we mark unknown operations as dynamic.
if (unknownLegalityFn) if (unknownLegalityFn)
return LegalizationInfo{LegalizationAction::Dynamic, return LegalizationInfo{LegalizationAction::Dynamic,
/*isRecursivelyLegal=*/false, unknownLegalityFn}; /*isRecursivelyLegal=*/false, unknownLegalityFn};
return llvm::None; return std::nullopt;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// PDL Configuration // PDL Configuration
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
void PDLConversionConfig::notifyRewriteBegin(PatternRewriter &rewriter) { void PDLConversionConfig::notifyRewriteBegin(PatternRewriter &rewriter) {
auto &rewriterImpl = auto &rewriterImpl =
▲ Show 20 Lines • Show All 124 Lines • Show Last 20 Lines

mlir/lib/Transforms/ViewOpGraph.cpp

Show First 20 Lines • Show All 63 Lines • ▼ Show 20 Lines
/// identifier and an optional identifier for the cluster (subgraph) that /// identifier and an optional identifier for the cluster (subgraph) that
/// contains the node. /// contains the node.
/// Note: In the DOT language, edges can be drawn only from nodes to nodes, but /// Note: In the DOT language, edges can be drawn only from nodes to nodes, but
/// not between clusters. However, edges can be clipped to the boundary of a /// not between clusters. However, edges can be clipped to the boundary of a
/// cluster with `lhead` and `ltail` attributes. Therefore, when creating a new /// cluster with `lhead` and `ltail` attributes. Therefore, when creating a new
/// cluster, an invisible "anchor" node is created. /// cluster, an invisible "anchor" node is created.
struct Node { struct Node {
public: public:
Node(int id = 0, Optional<int> clusterId = llvm::None) Node(int id = 0, Optional<int> clusterId = std::nullopt)
: id(id), clusterId(clusterId) {} : id(id), clusterId(clusterId) {}
int id; int id;
Optional<int> clusterId; Optional<int> clusterId;
}; };
/// This pass generates a Graphviz dataflow visualization of an MLIR operation. /// This pass generates a Graphviz dataflow visualization of an MLIR operation.
/// Note: See https://www.graphviz.org/doc/info/lang.html for more information /// Note: See https://www.graphviz.org/doc/info/lang.html for more information
▲ Show 20 Lines • Show All 268 Lines • Show Last 20 Lines

mlir/test/lib/Dialect/Func/TestDecomposeCallGraphTypes.cpp

Show First 20 Lines • Show All 69 Lines • ▼ Show 20 Lines decomposer.addDecomposeValueConversion([](OpBuilder &builder, Location loc,
} }
return success(); return success();
}); });
typeConverter.addArgumentMaterialization( typeConverter.addArgumentMaterialization(
[](OpBuilder &builder, TupleType resultType, ValueRange inputs, [](OpBuilder &builder, TupleType resultType, ValueRange inputs,
Location loc) -> Optional<Value> { Location loc) -> Optional<Value> {
if (inputs.size() == 1) if (inputs.size() == 1)
return llvm::None; return std::nullopt;
TupleType tuple = builder.getTupleType(inputs.getTypes()); TupleType tuple = builder.getTupleType(inputs.getTypes());
Value value = builder.create<test::MakeTupleOp>(loc, tuple, inputs); Value value = builder.create<test::MakeTupleOp>(loc, tuple, inputs);
return value; return value;
}); });
populateDecomposeCallGraphTypesPatterns(context, typeConverter, decomposer, populateDecomposeCallGraphTypesPatterns(context, typeConverter, decomposer,
patterns); patterns);
Show All 14 Lines

mlir/test/lib/Dialect/Test/TestAttributes.cpp

Show First 20 Lines • Show All 151 Lines • ▼ Show 20 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// TestExtern1DI64ElementsAttr // TestExtern1DI64ElementsAttr
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
ArrayRef<uint64_t> TestExtern1DI64ElementsAttr::getElements() const { ArrayRef<uint64_t> TestExtern1DI64ElementsAttr::getElements() const {
if (auto *blob = getHandle().getBlob()) if (auto *blob = getHandle().getBlob())
return blob->getDataAs<uint64_t>(); return blob->getDataAs<uint64_t>();
return llvm::None; return std::nullopt;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// TestCustomAnchorAttr // TestCustomAnchorAttr
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
static ParseResult parseTrueFalse(AsmParser &p, static ParseResult parseTrueFalse(AsmParser &p,
FailureOr<Optional<int>> &result) { FailureOr<Optional<int>> &result) {
▲ Show 20 Lines • Show All 101 Lines • Show Last 20 Lines

mlir/test/lib/Dialect/Test/TestDialect.cpp

Show First 20 Lines • Show All 88 Lines • ▼ Show 20 Lines Optional<StringRef> aliasName =
.Case("alias_test:dot_in_name", StringRef("test.alias")) .Case("alias_test:dot_in_name", StringRef("test.alias"))
.Case("alias_test:trailing_digit", StringRef("test_alias0")) .Case("alias_test:trailing_digit", StringRef("test_alias0"))
.Case("alias_test:prefixed_digit", StringRef("0_test_alias")) .Case("alias_test:prefixed_digit", StringRef("0_test_alias"))
.Case("alias_test:sanitize_conflict_a", .Case("alias_test:sanitize_conflict_a",
StringRef("test_alias_conflict0")) StringRef("test_alias_conflict0"))
.Case("alias_test:sanitize_conflict_b", .Case("alias_test:sanitize_conflict_b",
StringRef("test_alias_conflict0_")) StringRef("test_alias_conflict0_"))
.Case("alias_test:tensor_encoding", StringRef("test_encoding")) .Case("alias_test:tensor_encoding", StringRef("test_encoding"))
.Default(llvm::None); .Default(std::nullopt);
if (!aliasName) if (!aliasName)
return AliasResult::NoAlias; return AliasResult::NoAlias;
os << *aliasName; os << *aliasName;
return AliasResult::FinalAlias; return AliasResult::FinalAlias;
} }
AliasResult getAlias(Type type, raw_ostream &os) const final { AliasResult getAlias(Type type, raw_ostream &os) const final {
▲ Show 20 Lines • Show All 1,506 Lines • Show Last 20 Lines

mlir/test/lib/Dialect/Test/TestPatterns.cpp

Show First 20 Lines • Show All 305 Lines • ▼ Show 20 Linesstatic void invokeCreateWithInferredReturnType(Operation *op) {
// Use permutations of 2 args as operands. // Use permutations of 2 args as operands.
assert(fop.getNumArguments() >= 2); assert(fop.getNumArguments() >= 2);
for (int i = 0, e = fop.getNumArguments(); i < e; ++i) { for (int i = 0, e = fop.getNumArguments(); i < e; ++i) {
for (int j = 0; j < e; ++j) { for (int j = 0; j < e; ++j) {
std::array<Value, 2> values = {{fop.getArgument(i), fop.getArgument(j)}}; std::array<Value, 2> values = {{fop.getArgument(i), fop.getArgument(j)}};
SmallVector<Type, 2> inferredReturnTypes; SmallVector<Type, 2> inferredReturnTypes;
if (succeeded(OpTy::inferReturnTypes( if (succeeded(OpTy::inferReturnTypes(
context, llvm::None, values, op->getAttrDictionary(), context, std::nullopt, values, op->getAttrDictionary(),
op->getRegions(), inferredReturnTypes))) { op->getRegions(), inferredReturnTypes))) {
OperationState state(location, OpTy::getOperationName()); OperationState state(location, OpTy::getOperationName());
// TODO: Expand to regions. // TODO: Expand to regions.
OpTy::build(b, state, values, op->getAttrs()); OpTy::build(b, state, values, op->getAttrs());
(void)b.create(state); (void)b.create(state);
} }
} }
} }
▲ Show 20 Lines • Show All 240 Lines • ▼ Show 20 Lines
}; };
/// This pattern simply updates the operands of the given operation. /// This pattern simply updates the operands of the given operation.
struct TestPassthroughInvalidOp : public ConversionPattern { struct TestPassthroughInvalidOp : public ConversionPattern {
TestPassthroughInvalidOp(MLIRContext *ctx) TestPassthroughInvalidOp(MLIRContext *ctx)
: ConversionPattern("test.invalid", 1, ctx) {} : ConversionPattern("test.invalid", 1, ctx) {}
LogicalResult LogicalResult
matchAndRewrite(Operation *op, ArrayRef<Value> operands, matchAndRewrite(Operation *op, ArrayRef<Value> operands,
ConversionPatternRewriter &rewriter) const final { ConversionPatternRewriter &rewriter) const final {
rewriter.replaceOpWithNewOp<TestValidOp>(op, llvm::None, operands, rewriter.replaceOpWithNewOp<TestValidOp>(op, std::nullopt, operands,
llvm::None); std::nullopt);
return success(); return success();
} }
}; };
/// This pattern handles the case of a split return value. /// This pattern handles the case of a split return value.
struct TestSplitReturnType : public ConversionPattern { struct TestSplitReturnType : public ConversionPattern {
TestSplitReturnType(MLIRContext *ctx) TestSplitReturnType(MLIRContext *ctx)
: ConversionPattern("test.return", 1, ctx) {} : ConversionPattern("test.return", 1, ctx) {}
LogicalResult LogicalResult
▲ Show 20 Lines • Show All 1,025 Lines • Show Last 20 Lines

mlir/test/lib/Dialect/Vector/TestVectorTransforms.cpp

Show First 20 Lines • Show All 78 Lines • ▼ Show 20 Lines if (isa<vector::ContractionOp>(op))
return SmallVector<int64_t>(3, 2); return SmallVector<int64_t>(3, 2);
// For transfer ops, just propagate the shape coming from // For transfer ops, just propagate the shape coming from
// InsertStridedSlices/ExtractStridedSlices. // InsertStridedSlices/ExtractStridedSlices.
if (auto readOp = dyn_cast<vector::TransferReadOp>(op)) { if (auto readOp = dyn_cast<vector::TransferReadOp>(op)) {
VectorType dstVec; VectorType dstVec;
for (Operation *users : readOp->getUsers()) { for (Operation *users : readOp->getUsers()) {
auto extract = dyn_cast<ExtractStridedSliceOp>(users); auto extract = dyn_cast<ExtractStridedSliceOp>(users);
if (!extract) if (!extract)
return llvm::None; return std::nullopt;
auto vecType = extract.getResult().getType().cast<VectorType>(); auto vecType = extract.getResult().getType().cast<VectorType>();
if (dstVec && dstVec != vecType) if (dstVec && dstVec != vecType)
return llvm::None; return std::nullopt;
dstVec = vecType; dstVec = vecType;
} }
return SmallVector<int64_t>(dstVec.getShape().begin(), return SmallVector<int64_t>(dstVec.getShape().begin(),
dstVec.getShape().end()); dstVec.getShape().end());
} }
if (auto writeOp = dyn_cast<vector::TransferWriteOp>(op)) { if (auto writeOp = dyn_cast<vector::TransferWriteOp>(op)) {
auto insert = writeOp.getVector().getDefiningOp<InsertStridedSliceOp>(); auto insert = writeOp.getVector().getDefiningOp<InsertStridedSliceOp>();
if (!insert) if (!insert)
return llvm::None; return std::nullopt;
ArrayRef<int64_t> shape = insert.getSourceVectorType().getShape(); ArrayRef<int64_t> shape = insert.getSourceVectorType().getShape();
return SmallVector<int64_t>(shape.begin(), shape.end()); return SmallVector<int64_t>(shape.begin(), shape.end());
} }
return llvm::None; return std::nullopt;
} }
static LogicalResult filter(Operation *op) { static LogicalResult filter(Operation *op) {
return success(isa<arith::AddFOp, arith::SelectOp, arith::CmpFOp, return success(isa<arith::AddFOp, arith::SelectOp, arith::CmpFOp,
ContractionOp, TransferReadOp, TransferWriteOp>(op)); ContractionOp, TransferReadOp, TransferWriteOp>(op));
} }
}; };
▲ Show 20 Lines • Show All 750 Lines • Show Last 20 Lines

mlir/test/mlir-tblgen/attrdefs.td

Show All 27 Lines
// DEF-NEXT: value = ::test::CompoundAAttr::parse(parser, type); // DEF-NEXT: value = ::test::CompoundAAttr::parse(parser, type);
// DEF-NEXT: return ::mlir::success(!!value); // DEF-NEXT: return ::mlir::success(!!value);
// DEF-NEXT: }) // DEF-NEXT: })
// DEF-NEXT: .Case(::test::IndexAttr::getMnemonic() // DEF-NEXT: .Case(::test::IndexAttr::getMnemonic()
// DEF-NEXT: value = ::test::IndexAttr::parse(parser, type); // DEF-NEXT: value = ::test::IndexAttr::parse(parser, type);
// DEF-NEXT: return ::mlir::success(!!value); // DEF-NEXT: return ::mlir::success(!!value);
// DEF: .Default([&](llvm::StringRef keyword, // DEF: .Default([&](llvm::StringRef keyword,
// DEF-NEXT: *mnemonic = keyword; // DEF-NEXT: *mnemonic = keyword;
// DEF-NEXT: return llvm::None; // DEF-NEXT: return std::nullopt;
def Test_Dialect: Dialect { def Test_Dialect: Dialect {
// DECL-NOT: TestDialect // DECL-NOT: TestDialect
// DEF-NOT: TestDialect // DEF-NOT: TestDialect
let name = "TestDialect"; let name = "TestDialect";
let cppNamespace = "::test"; let cppNamespace = "::test";
} }
▲ Show 20 Lines • Show All 125 Lines • Show Last 20 Lines

mlir/test/mlir-tblgen/op-attribute.td

Show First 20 Lines • Show All 106 Lines • ▼ Show 20 Lines
// DEF: some-return-type AOp::getBAttr() { // DEF: some-return-type AOp::getBAttr() {
// DEF-NEXT: auto attr = getBAttrAttr(); // DEF-NEXT: auto attr = getBAttrAttr();
// DEF-NEXT: return attr.some-convert-from-storage(); // DEF-NEXT: return attr.some-convert-from-storage();
// DEF: some-attr-kind AOp::getCAttrAttr() // DEF: some-attr-kind AOp::getCAttrAttr()
// DEF-NEXT: ::mlir::impl::getAttrFromSortedRange((*this)->getAttrs().begin() + 1, (*this)->getAttrs().end() - 0, getCAttrAttrName()).dyn_cast_or_null<some-attr-kind>() // DEF-NEXT: ::mlir::impl::getAttrFromSortedRange((*this)->getAttrs().begin() + 1, (*this)->getAttrs().end() - 0, getCAttrAttrName()).dyn_cast_or_null<some-attr-kind>()
// DEF: ::llvm::Optional<some-return-type> AOp::getCAttr() { // DEF: ::llvm::Optional<some-return-type> AOp::getCAttr() {
// DEF-NEXT: auto attr = getCAttrAttr() // DEF-NEXT: auto attr = getCAttrAttr()
// DEF-NEXT: return attr ? ::llvm::Optional<some-return-type>(attr.some-convert-from-storage()) : (::llvm::None); // DEF-NEXT: return attr ? ::llvm::Optional<some-return-type>(attr.some-convert-from-storage()) : (::std::nullopt);
// DEF: some-attr-kind AOp::getDAttrAttr() // DEF: some-attr-kind AOp::getDAttrAttr()
// DEF-NEXT: ::mlir::impl::getAttrFromSortedRange((*this)->getAttrs().begin() + 1, (*this)->getAttrs().end() - 0, getDAttrAttrName()).dyn_cast_or_null<some-attr-kind>() // DEF-NEXT: ::mlir::impl::getAttrFromSortedRange((*this)->getAttrs().begin() + 1, (*this)->getAttrs().end() - 0, getDAttrAttrName()).dyn_cast_or_null<some-attr-kind>()
// DEF: some-return-type AOp::getDAttr() { // DEF: some-return-type AOp::getDAttr() {
// DEF-NEXT: auto attr = getDAttrAttr(); // DEF-NEXT: auto attr = getDAttrAttr();
// DEF-NEXT: if (!attr) // DEF-NEXT: if (!attr)
// DEF-NEXT: return some-const-builder-call(::mlir::Builder((*this)->getContext()), 4.2).some-convert-from-storage(); // DEF-NEXT: return some-const-builder-call(::mlir::Builder((*this)->getContext()), 4.2).some-convert-from-storage();
// DEF-NEXT: return attr.some-convert-from-storage(); // DEF-NEXT: return attr.some-convert-from-storage();
▲ Show 20 Lines • Show All 120 Lines • ▼ Show 20 Lines
// DEF-NEXT: if (!attr) // DEF-NEXT: if (!attr)
// DEF-NEXT: return some-const-builder-call(::mlir::Builder((*this)->getContext()), 4.2).some-convert-from-storage(); // DEF-NEXT: return some-const-builder-call(::mlir::Builder((*this)->getContext()), 4.2).some-convert-from-storage();
// DEF-NEXT: return attr.some-convert-from-storage(); // DEF-NEXT: return attr.some-convert-from-storage();
// DEF: some-attr-kind AgetOp::getCAttrAttr() // DEF: some-attr-kind AgetOp::getCAttrAttr()
// DEF-NEXT: return ::mlir::impl::getAttrFromSortedRange({{.*}}).dyn_cast_or_null<some-attr-kind>() // DEF-NEXT: return ::mlir::impl::getAttrFromSortedRange({{.*}}).dyn_cast_or_null<some-attr-kind>()
// DEF: ::llvm::Optional<some-return-type> AgetOp::getCAttr() { // DEF: ::llvm::Optional<some-return-type> AgetOp::getCAttr() {
// DEF-NEXT: auto attr = getCAttrAttr() // DEF-NEXT: auto attr = getCAttrAttr()
// DEF-NEXT: return attr ? ::llvm::Optional<some-return-type>(attr.some-convert-from-storage()) : (::llvm::None); // DEF-NEXT: return attr ? ::llvm::Optional<some-return-type>(attr.some-convert-from-storage()) : (::std::nullopt);
// Test setter methods // Test setter methods
// --- // ---
// DEF: void AgetOp::setAAttrAttr(some-attr-kind attr) { // DEF: void AgetOp::setAAttrAttr(some-attr-kind attr) {
// DEF-NEXT: (*this)->setAttr(getAAttrAttrName(), attr); // DEF-NEXT: (*this)->setAttr(getAAttrAttrName(), attr);
// DEF: void AgetOp::setBAttrAttr(some-attr-kind attr) { // DEF: void AgetOp::setBAttrAttr(some-attr-kind attr) {
// DEF-NEXT: (*this)->setAttr(getBAttrAttrName(), attr); // DEF-NEXT: (*this)->setAttr(getBAttrAttrName(), attr);
▲ Show 20 Lines • Show All 252 Lines • Show Last 20 Lines

mlir/test/mlir-tblgen/typedefs.td

Show All 27 Lines
// DEF-NEXT: value = ::test::CompoundAType::parse(parser); // DEF-NEXT: value = ::test::CompoundAType::parse(parser);
// DEF-NEXT: return ::mlir::success(!!value); // DEF-NEXT: return ::mlir::success(!!value);
// DEF-NEXT: }) // DEF-NEXT: })
// DEF-NEXT: .Case(::test::IndexType::getMnemonic() // DEF-NEXT: .Case(::test::IndexType::getMnemonic()
// DEF-NEXT: value = ::test::IndexType::parse(parser); // DEF-NEXT: value = ::test::IndexType::parse(parser);
// DEF-NEXT: return ::mlir::success(!!value); // DEF-NEXT: return ::mlir::success(!!value);
// DEF: .Default([&](llvm::StringRef keyword, // DEF: .Default([&](llvm::StringRef keyword,
// DEF-NEXT: *mnemonic = keyword; // DEF-NEXT: *mnemonic = keyword;
// DEF-NEXT: return llvm::None; // DEF-NEXT: return std::nullopt;
def Test_Dialect: Dialect { def Test_Dialect: Dialect {
// DECL-NOT: TestDialect // DECL-NOT: TestDialect
let name = "TestDialect"; let name = "TestDialect";
let cppNamespace = "::test"; let cppNamespace = "::test";
} }
class TestType<string name> : TypeDef<Test_Dialect, name> { } class TestType<string name> : TypeDef<Test_Dialect, name> { }
▲ Show 20 Lines • Show All 109 Lines • Show Last 20 Lines

mlir/tools/mlir-linalg-ods-gen/mlir-linalg-ods-yaml-gen.cpp

Show First 20 Lines • Show All 392 Lines • ▼ Show 20 Lines
} }
static Optional<int> static Optional<int>
findTensorDefArgIndex(StringRef name, SmallVectorImpl<LinalgOperandDef> &args) { findTensorDefArgIndex(StringRef name, SmallVectorImpl<LinalgOperandDef> &args) {
for (const auto &it : llvm::enumerate(args)) { for (const auto &it : llvm::enumerate(args)) {
if (it.value().name == name) if (it.value().name == name)
return it.index(); return it.index();
} }
return None; return std::nullopt;
} }
// Try to map the TypeVar to a predefined or an argument type. // Try to map the TypeVar to a predefined or an argument type.
static Optional<std::string> static Optional<std::string>
findTypeValue(StringRef typeVar, SmallVectorImpl<LinalgOperandDef> &args) { findTypeValue(StringRef typeVar, SmallVectorImpl<LinalgOperandDef> &args) {
// Handle all predefined types. // Handle all predefined types.
if (typeVar == "I32") if (typeVar == "I32")
return std::string("helper.getIntegerType(32)"); return std::string("helper.getIntegerType(32)");
Show All 10 Lines if (it.value().kind != LinalgOperandDefKind::InputTensor &&
it.value().kind != LinalgOperandDefKind::Scalar && it.value().kind != LinalgOperandDefKind::Scalar &&
it.value().kind != LinalgOperandDefKind::OutputTensor) it.value().kind != LinalgOperandDefKind::OutputTensor)
continue; continue;
if (*it.value().typeVar == typeVar) if (*it.value().typeVar == typeVar)
return llvm::formatv("block.getArgument({0}).getType()", it.index()) return llvm::formatv("block.getArgument({0}).getType()", it.index())
.str(); .str();
} }
return None; return std::nullopt;
} }
static ScalarAssign *findAssignment(StringRef name, static ScalarAssign *findAssignment(StringRef name,
std::vector<ScalarAssign> &assignments) { std::vector<ScalarAssign> &assignments) {
for (auto &assign : assignments) { for (auto &assign : assignments) {
if (assign.arg == name) if (assign.arg == name)
return &assign; return &assign;
} }
▲ Show 20 Lines • Show All 83 Lines • ▼ Show 20 Lines /*extraInterfaces=*/[{2}])> {
let regions = (region AnyRegion:$region); let regions = (region AnyRegion:$region);
let skipDefaultBuilders = 1; let skipDefaultBuilders = 1;
let builders = [ let builders = [
OpBuilder< OpBuilder<
(ins "ValueRange":$inputs, "ValueRange":$outputs, (ins "ValueRange":$inputs, "ValueRange":$outputs,
CArg<"ArrayRef<NamedAttribute>", "{{}">:$attributes), CArg<"ArrayRef<NamedAttribute>", "{{}">:$attributes),
[{{ [{{
buildStructuredOp($_builder, $_state, llvm::None, inputs, outputs, buildStructuredOp($_builder, $_state, std::nullopt, inputs, outputs,
attributes, {0}::getRegionBuilder()); attributes, {0}::getRegionBuilder());
}]>, }]>,
OpBuilder< OpBuilder<
(ins "TypeRange":$resultTensorTypes, "ValueRange":$inputs, (ins "TypeRange":$resultTensorTypes, "ValueRange":$inputs,
"ValueRange":$outputs, "ValueRange":$outputs,
CArg<"ArrayRef<NamedAttribute>", "{{}">:$attributes), CArg<"ArrayRef<NamedAttribute>", "{{}">:$attributes),
[{{ [{{
buildStructuredOp($_builder, $_state, resultTensorTypes, buildStructuredOp($_builder, $_state, resultTensorTypes,
▲ Show 20 Lines • Show All 522 Lines • ▼ Show 20 Lines for (LinalgOperandDef &arg : args) {
generateExpression = generateExpression =
[&](ScalarExpression &expression) -> Optional<std::string> { [&](ScalarExpression &expression) -> Optional<std::string> {
if (expression.arg) { if (expression.arg) {
// Argument reference. // Argument reference.
Optional<int> argIndex = findTensorDefArgIndex(*expression.arg, args); Optional<int> argIndex = findTensorDefArgIndex(*expression.arg, args);
if (!argIndex) { if (!argIndex) {
emitError(genContext.getLoc()) emitError(genContext.getLoc())
<< "scalar argument not defined on the op: " << *expression.arg; << "scalar argument not defined on the op: " << *expression.arg;
return None; return std::nullopt;
} }
return std::string( return std::string(
llvm::formatv("block.getArgument({0})", *argIndex)); llvm::formatv("block.getArgument({0})", *argIndex));
} }
if (expression.constant) { if (expression.constant) {
std::string cppIdent = llvm::formatv("value{0}", ++localCounter); std::string cppIdent = llvm::formatv("value{0}", ++localCounter);
stmts.push_back( stmts.push_back(
llvm::formatv(R"FMT(Value {0} = helper.constant("{1}");)FMT", llvm::formatv(R"FMT(Value {0} = helper.constant("{1}");)FMT",
Show All 37 Lines for (LinalgOperandDef &arg : args) {
assert(expression.scalarFn->typeVar.has_value()); assert(expression.scalarFn->typeVar.has_value());
Optional<std::string> typeCppValue = Optional<std::string> typeCppValue =
findTypeValue(expression.scalarFn->typeVar.value(), args); findTypeValue(expression.scalarFn->typeVar.value(), args);
if (!typeCppValue) { if (!typeCppValue) {
emitError(genContext.getLoc()) emitError(genContext.getLoc())
<< "type variable " << expression.scalarFn->typeVar.value() << "type variable " << expression.scalarFn->typeVar.value()
<< ", used in a type conversion, must map to a predefined or " << ", used in a type conversion, must map to a predefined or "
<< "an argument type but it does not"; << "an argument type but it does not";
return None; return std::nullopt;
} }
operandCppValues.push_back(typeCppValue.value()); operandCppValues.push_back(typeCppValue.value());
} }
// Collect the scalar operands. // Collect the scalar operands.
for (ScalarExpression &operand : expression.scalarFn->operands) { for (ScalarExpression &operand : expression.scalarFn->operands) {
auto operandCppValue = generateExpression(operand); auto operandCppValue = generateExpression(operand);
if (!operandCppValue) if (!operandCppValue)
return None; return std::nullopt;
operandCppValues.push_back(*operandCppValue); operandCppValues.push_back(*operandCppValue);
} }
// Call the function builder. // Call the function builder.
std::string cppIdent = llvm::formatv("value{0}", ++localCounter); std::string cppIdent = llvm::formatv("value{0}", ++localCounter);
stmts.push_back(llvm::formatv( stmts.push_back(llvm::formatv(
"Value {0} = helper.build{1}({2}, {3});", cppIdent, enumName, "Value {0} = helper.build{1}({2}, {3});", cppIdent, enumName,
funcType, interleaveToString(operandCppValues, ", "))); funcType, interleaveToString(operandCppValues, ", ")));
return cppIdent; return cppIdent;
} }
emitError(genContext.getLoc()) << "unknown ScalarExpression type"; emitError(genContext.getLoc()) << "unknown ScalarExpression type";
return None; return std::nullopt;
}; };
Optional<std::string> cppValue = generateExpression(assignment->value); Optional<std::string> cppValue = generateExpression(assignment->value);
if (!cppValue) if (!cppValue)
return failure(); return failure();
stmts.push_back(llvm::formatv("yields.push_back({0});", cppValue)); stmts.push_back(llvm::formatv("yields.push_back({0});", cppValue));
} }
if (generatedAssignmentCount != assignments.size()) if (generatedAssignmentCount != assignments.size())
▲ Show 20 Lines • Show All 119 Lines • Show Last 20 Lines

mlir/tools/mlir-tblgen/AttrOrTypeDefGen.cpp

Show First 20 Lines • Show All 809 Lines • ▼ Show 20 Lines for (auto &def : defs) {
// If the def has no parameters and no printer, just print the mnemonic. // If the def has no parameters and no printer, just print the mnemonic.
StringRef printDef = ""; StringRef printDef = "";
if (hasParserPrinterDecl) if (hasParserPrinterDecl)
printDef = "\nt.print(printer);"; printDef = "\nt.print(printer);";
printer.body() << llvm::formatv(printValue, defClass, printDef); printer.body() << llvm::formatv(printValue, defClass, printDef);
} }
parse.body() << " .Default([&](llvm::StringRef keyword, llvm::SMLoc) {\n" parse.body() << " .Default([&](llvm::StringRef keyword, llvm::SMLoc) {\n"
" *mnemonic = keyword;\n" " *mnemonic = keyword;\n"
" return llvm::None;\n" " return std::nullopt;\n"
" });"; " });";
printer.body() << " .Default([](auto) { return ::mlir::failure(); });"; printer.body() << " .Default([](auto) { return ::mlir::failure(); });";
raw_indented_ostream indentedOs(os); raw_indented_ostream indentedOs(os);
parse.writeDeclTo(indentedOs); parse.writeDeclTo(indentedOs);
printer.writeDeclTo(indentedOs); printer.writeDeclTo(indentedOs);
} }
▲ Show 20 Lines • Show All 104 Lines • Show Last 20 Lines

mlir/tools/mlir-tblgen/DialectGen.cpp

Show First 20 Lines • Show All 53 Lines • ▼ Show 20 LinesfilterForDialect(ArrayRef<llvm::Record *> records, Dialect &dialect) {
}; };
return {DialectFilterIterator(records.begin(), records.end(), filterFn), return {DialectFilterIterator(records.begin(), records.end(), filterFn),
DialectFilterIterator(records.end(), records.end(), filterFn)}; DialectFilterIterator(records.end(), records.end(), filterFn)};
} }
Optional<Dialect> tblgen::findDialectToGenerate(ArrayRef<Dialect> dialects) { Optional<Dialect> tblgen::findDialectToGenerate(ArrayRef<Dialect> dialects) {
if (dialects.empty()) { if (dialects.empty()) {
llvm::errs() << "no dialect was found\n"; llvm::errs() << "no dialect was found\n";
return llvm::None; return std::nullopt;
} }
// Select the dialect to gen for. // Select the dialect to gen for.
if (dialects.size() == 1 && selectedDialect.getNumOccurrences() == 0) if (dialects.size() == 1 && selectedDialect.getNumOccurrences() == 0)
return dialects.front(); return dialects.front();
if (selectedDialect.getNumOccurrences() == 0) { if (selectedDialect.getNumOccurrences() == 0) {
llvm::errs() << "when more than 1 dialect is present, one must be selected " llvm::errs() << "when more than 1 dialect is present, one must be selected "
"via '-dialect'\n"; "via '-dialect'\n";
return llvm::None; return std::nullopt;
} }
const auto *dialectIt = llvm::find_if(dialects, [](const Dialect &dialect) { const auto *dialectIt = llvm::find_if(dialects, [](const Dialect &dialect) {
return dialect.getName() == selectedDialect; return dialect.getName() == selectedDialect;
}); });
if (dialectIt == dialects.end()) { if (dialectIt == dialects.end()) {
llvm::errs() << "selected dialect with '-dialect' does not exist\n"; llvm::errs() << "selected dialect with '-dialect' does not exist\n";
return llvm::None; return std::nullopt;
} }
return *dialectIt; return *dialectIt;
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// GEN: Dialect declarations // GEN: Dialect declarations
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 244 Lines • Show Last 20 Lines

mlir/tools/mlir-tblgen/EnumsGen.cpp

Show First 20 Lines • Show All 220 Lines • ▼ Show 20 Linesstatic void emitMaxValueFn(const Record &enumDef, raw_ostream &os) {
} }
// Emit the function to return the max enum value // Emit the function to return the max enum value
os << formatv("inline constexpr unsigned {0}() {{\n", maxEnumValFnName); os << formatv("inline constexpr unsigned {0}() {{\n", maxEnumValFnName);
os << formatv(" return {0};\n", maxEnumVal); os << formatv(" return {0};\n", maxEnumVal);
os << "}\n\n"; os << "}\n\n";
} }
// Returns the EnumAttrCase whose value is zero if exists; returns llvm::None // Returns the EnumAttrCase whose value is zero if exists; returns std::nullopt
// otherwise. // otherwise.
static llvm::Optional<EnumAttrCase> static llvm::Optional<EnumAttrCase>
getAllBitsUnsetCase(llvm::ArrayRef<EnumAttrCase> cases) { getAllBitsUnsetCase(llvm::ArrayRef<EnumAttrCase> cases) {
for (auto attrCase : cases) { for (auto attrCase : cases) {
if (attrCase.getValue() == 0) if (attrCase.getValue() == 0)
return attrCase; return attrCase;
} }
return llvm::None; return std::nullopt;
} }
// Emits the following inline function for bit enums: // Emits the following inline function for bit enums:
// //
// inline constexpr <enum-type> operator|(<enum-type> a, <enum-type> b); // inline constexpr <enum-type> operator|(<enum-type> a, <enum-type> b);
// inline constexpr <enum-type> operator&(<enum-type> a, <enum-type> b); // inline constexpr <enum-type> operator&(<enum-type> a, <enum-type> b);
// inline constexpr <enum-type> operator^(<enum-type> a, <enum-type> b); // inline constexpr <enum-type> operator^(<enum-type> a, <enum-type> b);
// inline constexpr <enum-type> operator~(<enum-type> bits); // inline constexpr <enum-type> operator~(<enum-type> bits);
▲ Show 20 Lines • Show All 141 Lines • ▼ Show 20 Linesstatic void emitStrToSymFnForIntEnum(const Record &enumDef, raw_ostream &os) {
os << formatv(" return ::llvm::StringSwitch<::llvm::Optional<{0}>>(str)\n", os << formatv(" return ::llvm::StringSwitch<::llvm::Optional<{0}>>(str)\n",
enumName); enumName);
for (const auto &enumerant : enumerants) { for (const auto &enumerant : enumerants) {
auto symbol = enumerant.getSymbol(); auto symbol = enumerant.getSymbol();
auto str = enumerant.getStr(); auto str = enumerant.getStr();
os << formatv(" .Case(\"{1}\", {0}::{2})\n", enumName, str, os << formatv(" .Case(\"{1}\", {0}::{2})\n", enumName, str,
makeIdentifier(symbol)); makeIdentifier(symbol));
} }
os << " .Default(::llvm::None);\n"; os << " .Default(::std::nullopt);\n";
os << "}\n"; os << "}\n";
} }
static void emitStrToSymFnForBitEnum(const Record &enumDef, raw_ostream &os) { static void emitStrToSymFnForBitEnum(const Record &enumDef, raw_ostream &os) {
EnumAttr enumAttr(enumDef); EnumAttr enumAttr(enumDef);
StringRef enumName = enumAttr.getEnumClassName(); StringRef enumName = enumAttr.getEnumClassName();
std::string underlyingType = std::string(enumAttr.getUnderlyingType()); std::string underlyingType = std::string(enumAttr.getUnderlyingType());
StringRef strToSymFnName = enumAttr.getStringToSymbolFnName(); StringRef strToSymFnName = enumAttr.getStringToSymbolFnName();
Show All 24 Linesstatic void emitStrToSymFnForBitEnum(const Record &enumDef, raw_ostream &os) {
os << formatv(" auto bit = " os << formatv(" auto bit = "
"llvm::StringSwitch<::llvm::Optional<{0}>>(symbol.trim())\n", "llvm::StringSwitch<::llvm::Optional<{0}>>(symbol.trim())\n",
underlyingType); underlyingType);
for (const auto &enumerant : enumerants) { for (const auto &enumerant : enumerants) {
// Skip the special enumerant for None. // Skip the special enumerant for None.
if (auto val = enumerant.getValue()) if (auto val = enumerant.getValue())
os.indent(6) << formatv(".Case(\"{0}\", {1})\n", enumerant.getStr(), val); os.indent(6) << formatv(".Case(\"{0}\", {1})\n", enumerant.getStr(), val);
} }
os.indent(6) << ".Default(::llvm::None);\n"; os.indent(6) << ".Default(::std::nullopt);\n";
os << " if (bit) { val |= *bit; } else { return ::llvm::None; }\n"; os << " if (bit) { val |= *bit; } else { return ::std::nullopt; }\n";
os << " }\n"; os << " }\n";
os << formatv(" return static_cast<{0}>(val);\n", enumName); os << formatv(" return static_cast<{0}>(val);\n", enumName);
os << "}\n\n"; os << "}\n\n";
} }
static void emitUnderlyingToSymFnForIntEnum(const Record &enumDef, static void emitUnderlyingToSymFnForIntEnum(const Record &enumDef,
raw_ostream &os) { raw_ostream &os) {
Show All 16 Lines os << formatv("::llvm::Optional<{0}> {1}({2} value) {{\n", enumName,
: underlyingType) : underlyingType)
<< " switch (value) {\n"; << " switch (value) {\n";
for (const auto &enumerant : enumerants) { for (const auto &enumerant : enumerants) {
auto symbol = enumerant.getSymbol(); auto symbol = enumerant.getSymbol();
auto value = enumerant.getValue(); auto value = enumerant.getValue();
os << formatv(" case {0}: return {1}::{2};\n", value, enumName, os << formatv(" case {0}: return {1}::{2};\n", value, enumName,
makeIdentifier(symbol)); makeIdentifier(symbol));
} }
os << " default: return ::llvm::None;\n" os << " default: return ::std::nullopt;\n"
<< " }\n" << " }\n"
<< "}\n\n"; << "}\n\n";
} }
static void emitSpecializedAttrDef(const Record &enumDef, raw_ostream &os) { static void emitSpecializedAttrDef(const Record &enumDef, raw_ostream &os) {
EnumAttr enumAttr(enumDef); EnumAttr enumAttr(enumDef);
StringRef enumName = enumAttr.getEnumClassName(); StringRef enumName = enumAttr.getEnumClassName();
StringRef attrClassName = enumAttr.getSpecializedAttrClassName(); StringRef attrClassName = enumAttr.getSpecializedAttrClassName();
▲ Show 20 Lines • Show All 63 Lines • ▼ Show 20 Linesstatic void emitUnderlyingToSymFnForBitEnum(const Record &enumDef,
os << formatv("::llvm::Optional<{0}> {1}({2} value) {{\n", enumName, os << formatv("::llvm::Optional<{0}> {1}({2} value) {{\n", enumName,
underlyingToSymFnName, underlyingType); underlyingToSymFnName, underlyingType);
if (allBitsUnsetCase) { if (allBitsUnsetCase) {
os << " // Special case for all bits unset.\n"; os << " // Special case for all bits unset.\n";
os << formatv(" if (value == 0) return {0}::{1};\n\n", enumName, os << formatv(" if (value == 0) return {0}::{1};\n\n", enumName,
makeIdentifier(allBitsUnsetCase->getSymbol())); makeIdentifier(allBitsUnsetCase->getSymbol()));
} }
int64_t validBits = enumDef.getValueAsInt("validBits"); int64_t validBits = enumDef.getValueAsInt("validBits");
os << formatv(" if (value & ~static_cast<{0}>({1}u)) return llvm::None;\n", os << formatv(" if (value & ~static_cast<{0}>({1}u)) return std::nullopt;\n",
underlyingType, validBits); underlyingType, validBits);
os << formatv(" return static_cast<{0}>(value);\n", enumName); os << formatv(" return static_cast<{0}>(value);\n", enumName);
os << "}\n"; os << "}\n";
} }
static void emitEnumDecl(const Record &enumDef, raw_ostream &os) { static void emitEnumDecl(const Record &enumDef, raw_ostream &os) {
EnumAttr enumAttr(enumDef); EnumAttr enumAttr(enumDef);
StringRef enumName = enumAttr.getEnumClassName(); StringRef enumName = enumAttr.getEnumClassName();
▲ Show 20 Lines • Show All 147 Lines • Show Last 20 Lines

mlir/tools/mlir-tblgen/OpDefinitionsGen.cpp

Show First 20 Lines • Show All 348 Lines • ▼ Show 20 Lines for (const NamedAttribute &namedAttr : op.getAttributes()) {
attrMetadata.insert( attrMetadata.insert(
{namedAttr.name, AttributeMetadata{namedAttr.name, !isOptional, attr}}); {namedAttr.name, AttributeMetadata{namedAttr.name, !isOptional, attr}});
} }
// Include key attributes from several traits as implicitly registered. // Include key attributes from several traits as implicitly registered.
if (op.getTrait("::mlir::OpTrait::AttrSizedOperandSegments")) { if (op.getTrait("::mlir::OpTrait::AttrSizedOperandSegments")) {
attrMetadata.insert( attrMetadata.insert(
{operandSegmentAttrName, {operandSegmentAttrName,
AttributeMetadata{operandSegmentAttrName, /*isRequired=*/true, AttributeMetadata{operandSegmentAttrName, /*isRequired=*/true,
/*attr=*/llvm::None}}); /*attr=*/std::nullopt}});
} }
if (op.getTrait("::mlir::OpTrait::AttrSizedResultSegments")) { if (op.getTrait("::mlir::OpTrait::AttrSizedResultSegments")) {
attrMetadata.insert( attrMetadata.insert(
{resultSegmentAttrName, {resultSegmentAttrName,
AttributeMetadata{resultSegmentAttrName, /*isRequired=*/true, AttributeMetadata{resultSegmentAttrName, /*isRequired=*/true,
/*attr=*/llvm::None}}); /*attr=*/std::nullopt}});
} }
// Store the metadata in sorted order. // Store the metadata in sorted order.
SmallVector<AttributeMetadata> sortedAttrMetadata = SmallVector<AttributeMetadata> sortedAttrMetadata =
llvm::to_vector(llvm::make_second_range(attrMetadata.takeVector())); llvm::to_vector(llvm::make_second_range(attrMetadata.takeVector()));
llvm::sort(sortedAttrMetadata, llvm::sort(sortedAttrMetadata,
[](const AttributeMetadata &lhs, const AttributeMetadata &rhs) { [](const AttributeMetadata &lhs, const AttributeMetadata &rhs) {
return lhs.attrName < rhs.attrName; return lhs.attrName < rhs.attrName;
▲ Show 20 Lines • Show All 2,838 Lines • Show Last 20 Lines

mlir/tools/mlir-tblgen/OpFormatGen.cpp

Show First 20 Lines • Show All 55 Lines • ▼ Show 20 Lines
struct AttributeVariable struct AttributeVariable
: public OpVariableElement<NamedAttribute, VariableElement::Attribute> { : public OpVariableElement<NamedAttribute, VariableElement::Attribute> {
using Base::Base; using Base::Base;
/// Return the constant builder call for the type of this attribute, or None /// Return the constant builder call for the type of this attribute, or None
/// if it doesn't have one. /// if it doesn't have one.
llvm::Optional<StringRef> getTypeBuilder() const { llvm::Optional<StringRef> getTypeBuilder() const {
llvm::Optional<Type> attrType = var->attr.getValueType(); llvm::Optional<Type> attrType = var->attr.getValueType();
return attrType ? attrType->getBuilderCall() : llvm::None; return attrType ? attrType->getBuilderCall() : std::nullopt;
} }
/// Return if this attribute refers to a UnitAttr. /// Return if this attribute refers to a UnitAttr.
bool isUnitAttr() const { bool isUnitAttr() const {
return var->attr.getBaseAttr().getAttrDefName() == "UnitAttr"; return var->attr.getBaseAttr().getAttrDefName() == "UnitAttr";
} }
/// Indicate if this attribute is printed "qualified" (that is it is /// Indicate if this attribute is printed "qualified" (that is it is
▲ Show 20 Lines • Show All 2,394 Lines • ▼ Show 20 Lines for (FormatElement *element : elementRange) {
// Skip optionally parsed elements. // Skip optionally parsed elements.
if (element != optionalGuard && isOptionallyParsed(element)) if (element != optionalGuard && isOptionallyParsed(element))
continue; continue;
// We found a closing element that is valid. // We found a closing element that is valid.
return success(); return success();
} }
// Return None to indicate that we reached the end. // Return None to indicate that we reached the end.
return llvm::None; return std::nullopt;
} }
/// For the given elements, check whether any attributes are followed by a colon /// For the given elements, check whether any attributes are followed by a colon
/// literal, resulting in an ambiguous assembly format. Returns a non-null /// literal, resulting in an ambiguous assembly format. Returns a non-null
/// attribute if verification of said attribute reached the end of the range. /// attribute if verification of said attribute reached the end of the range.
/// Returns null if all attribute elements are verified. /// Returns null if all attribute elements are verified.
static FailureOr<FormatElement *> verifyAdjacentElements( static FailureOr<FormatElement *> verifyAdjacentElements(
function_ref<bool(FormatElement *)> isBase, function_ref<bool(FormatElement *)> isBase,
▲ Show 20 Lines • Show All 262 Lines • ▼ Show 20 Linesvoid OpFormatParser::handleAllTypesMatchConstraint(
for (unsigned i = 0, e = values.size(); i != e; ++i) { for (unsigned i = 0, e = values.size(); i != e; ++i) {
// Check to see if this value matches a resolved operand or result type. // Check to see if this value matches a resolved operand or result type.
ConstArgument arg = findSeenArg(values[i]); ConstArgument arg = findSeenArg(values[i]);
if (!arg) if (!arg)
continue; continue;
// Mark this value as the type resolver for the other variables. // Mark this value as the type resolver for the other variables.
for (unsigned j = 0; j != i; ++j) for (unsigned j = 0; j != i; ++j)
variableTyResolver[values[j]] = {arg, llvm::None}; variableTyResolver[values[j]] = {arg, std::nullopt};
for (unsigned j = i + 1; j != e; ++j) for (unsigned j = i + 1; j != e; ++j)
variableTyResolver[values[j]] = {arg, llvm::None}; variableTyResolver[values[j]] = {arg, std::nullopt};
} }
} }
void OpFormatParser::handleSameTypesConstraint( void OpFormatParser::handleSameTypesConstraint(
llvm::StringMap<TypeResolutionInstance> &variableTyResolver, llvm::StringMap<TypeResolutionInstance> &variableTyResolver,
bool includeResults) { bool includeResults) {
const NamedTypeConstraint *resolver = nullptr; const NamedTypeConstraint *resolver = nullptr;
int resolvedIt = -1; int resolvedIt = -1;
// Check to see if there is an operand or result to use for the resolution. // Check to see if there is an operand or result to use for the resolution.
if ((resolvedIt = seenOperandTypes.find_first()) != -1) if ((resolvedIt = seenOperandTypes.find_first()) != -1)
resolver = &op.getOperand(resolvedIt); resolver = &op.getOperand(resolvedIt);
else if (includeResults && (resolvedIt = seenResultTypes.find_first()) != -1) else if (includeResults && (resolvedIt = seenResultTypes.find_first()) != -1)
resolver = &op.getResult(resolvedIt); resolver = &op.getResult(resolvedIt);
else else
return; return;
// Set the resolvers for each operand and result. // Set the resolvers for each operand and result.
for (unsigned i = 0, e = op.getNumOperands(); i != e; ++i) for (unsigned i = 0, e = op.getNumOperands(); i != e; ++i)
if (!seenOperandTypes.test(i)) if (!seenOperandTypes.test(i))
variableTyResolver[op.getOperand(i).name] = {resolver, llvm::None}; variableTyResolver[op.getOperand(i).name] = {resolver, std::nullopt};
if (includeResults) { if (includeResults) {
for (unsigned i = 0, e = op.getNumResults(); i != e; ++i) for (unsigned i = 0, e = op.getNumResults(); i != e; ++i)
if (!seenResultTypes.test(i)) if (!seenResultTypes.test(i))
variableTyResolver[op.getResultName(i)] = {resolver, llvm::None}; variableTyResolver[op.getResultName(i)] = {resolver, std::nullopt};
} }
} }
void OpFormatParser::handleTypesMatchConstraint( void OpFormatParser::handleTypesMatchConstraint(
llvm::StringMap<TypeResolutionInstance> &variableTyResolver, llvm::StringMap<TypeResolutionInstance> &variableTyResolver,
const llvm::Record &def) { const llvm::Record &def) {
StringRef lhsName = def.getValueAsString("lhs"); StringRef lhsName = def.getValueAsString("lhs");
StringRef rhsName = def.getValueAsString("rhs"); StringRef rhsName = def.getValueAsString("rhs");
▲ Show 20 Lines • Show All 522 Lines • Show Last 20 Lines

mlir/tools/mlir-tblgen/SPIRVUtilsGen.cpp

Show First 20 Lines • Show All 356 Lines • ▼ Show 20 Lines for (const auto &caseSpecPair : classCasePair.getValue()) {
os << formatv(" case {0}::{1}: { {2} return {3}({4}); }\n", enumName, os << formatv(" case {0}::{1}: { {2} return {3}({4}); }\n", enumName,
enumerant.getSymbol(), avail.getMergeInstancePreparation(), enumerant.getSymbol(), avail.getMergeInstancePreparation(),
avail.getMergeInstanceType(), avail.getMergeInstance()); avail.getMergeInstanceType(), avail.getMergeInstance());
} }
// Only emit default if uncovered cases. // Only emit default if uncovered cases.
if (classCasePair.getValue().size() < enumAttr.getAllCases().size()) if (classCasePair.getValue().size() < enumAttr.getAllCases().size())
os << " default: break;\n"; os << " default: break;\n";
os << " }\n" os << " }\n"
<< " return llvm::None;\n" << " return std::nullopt;\n"
<< "}\n"; << "}\n";
} }
} }
static void emitAvailabilityQueryForBitEnum(const Record &enumDef, static void emitAvailabilityQueryForBitEnum(const Record &enumDef,
raw_ostream &os) { raw_ostream &os) {
EnumAttr enumAttr(enumDef); EnumAttr enumAttr(enumDef);
StringRef enumName = enumAttr.getEnumClassName(); StringRef enumName = enumAttr.getEnumClassName();
Show All 28 Lines for (const auto &caseSpecPair : classCasePair.getValue()) {
EnumAttrCase enumerant = caseSpecPair.first; EnumAttrCase enumerant = caseSpecPair.first;
Availability avail = caseSpecPair.second; Availability avail = caseSpecPair.second;
os << formatv(" case {0}::{1}: { {2} return {3}({4}); }\n", enumName, os << formatv(" case {0}::{1}: { {2} return {3}({4}); }\n", enumName,
enumerant.getSymbol(), avail.getMergeInstancePreparation(), enumerant.getSymbol(), avail.getMergeInstancePreparation(),
avail.getMergeInstanceType(), avail.getMergeInstance()); avail.getMergeInstanceType(), avail.getMergeInstance());
} }
os << " default: break;\n"; os << " default: break;\n";
os << " }\n" os << " }\n"
<< " return llvm::None;\n" << " return std::nullopt;\n"
<< "}\n"; << "}\n";
} }
} }
static void emitEnumDecl(const Record &enumDef, raw_ostream &os) { static void emitEnumDecl(const Record &enumDef, raw_ostream &os) {
EnumAttr enumAttr(enumDef); EnumAttr enumAttr(enumDef);
StringRef enumName = enumAttr.getEnumClassName(); StringRef enumName = enumAttr.getEnumClassName();
StringRef cppNamespace = enumAttr.getCppNamespace(); StringRef cppNamespace = enumAttr.getCppNamespace();
▲ Show 20 Lines • Show All 1,042 Lines • Show Last 20 Lines

mlir/unittests/IR/OperationSupportTest.cpp

Show All 12 Lines
#include "llvm/ADT/BitVector.h" #include "llvm/ADT/BitVector.h"
#include "llvm/Support/FormatVariadic.h" #include "llvm/Support/FormatVariadic.h"
#include "gtest/gtest.h" #include "gtest/gtest.h"
using namespace mlir; using namespace mlir;
using namespace mlir::detail; using namespace mlir::detail;
static Operation *createOp(MLIRContext *context, static Operation *createOp(MLIRContext *context,
ArrayRef<Value> operands = llvm::None, ArrayRef<Value> operands = std::nullopt,
ArrayRef<Type> resultTypes = llvm::None, ArrayRef<Type> resultTypes = std::nullopt,
unsigned int numRegions = 0) { unsigned int numRegions = 0) {
context->allowUnregisteredDialects(); context->allowUnregisteredDialects();
return Operation::create(UnknownLoc::get(context), return Operation::create(UnknownLoc::get(context),
OperationName("foo.bar", context), resultTypes, OperationName("foo.bar", context), resultTypes,
operands, llvm::None, llvm::None, numRegions); operands, std::nullopt, llvm::None, numRegions);
} }
namespace { namespace {
TEST(OperandStorageTest, NonResizable) { TEST(OperandStorageTest, NonResizable) {
MLIRContext context; MLIRContext context;
Builder builder(&context); Builder builder(&context);
Operation *useOp = Operation *useOp =
createOp(&context, /*operands=*/llvm::None, builder.getIntegerType(16)); createOp(&context, /*operands=*/std::nullopt, builder.getIntegerType(16));
Value operand = useOp->getResult(0); Value operand = useOp->getResult(0);
// Create a non-resizable operation with one operand. // Create a non-resizable operation with one operand.
Operation *user = createOp(&context, operand); Operation *user = createOp(&context, operand);
// The same number of operands is okay. // The same number of operands is okay.
user->setOperands(operand); user->setOperands(operand);
EXPECT_EQ(user->getNumOperands(), 1u); EXPECT_EQ(user->getNumOperands(), 1u);
// Removing is okay. // Removing is okay.
user->setOperands(llvm::None); user->setOperands(std::nullopt);
EXPECT_EQ(user->getNumOperands(), 0u); EXPECT_EQ(user->getNumOperands(), 0u);
// Destroy the operations. // Destroy the operations.
user->destroy(); user->destroy();
useOp->destroy(); useOp->destroy();
} }
TEST(OperandStorageTest, Resizable) { TEST(OperandStorageTest, Resizable) {
MLIRContext context; MLIRContext context;
Builder builder(&context); Builder builder(&context);
Operation *useOp = Operation *useOp =
createOp(&context, /*operands=*/llvm::None, builder.getIntegerType(16)); createOp(&context, /*operands=*/std::nullopt, builder.getIntegerType(16));
Value operand = useOp->getResult(0); Value operand = useOp->getResult(0);
// Create a resizable operation with one operand. // Create a resizable operation with one operand.
Operation *user = createOp(&context, operand); Operation *user = createOp(&context, operand);
// The same number of operands is okay. // The same number of operands is okay.
user->setOperands(operand); user->setOperands(operand);
EXPECT_EQ(user->getNumOperands(), 1u); EXPECT_EQ(user->getNumOperands(), 1u);
// Removing is okay. // Removing is okay.
user->setOperands(llvm::None); user->setOperands(std::nullopt);
EXPECT_EQ(user->getNumOperands(), 0u); EXPECT_EQ(user->getNumOperands(), 0u);
// Adding more operands is okay. // Adding more operands is okay.
user->setOperands({operand, operand, operand}); user->setOperands({operand, operand, operand});
EXPECT_EQ(user->getNumOperands(), 3u); EXPECT_EQ(user->getNumOperands(), 3u);
// Destroy the operations. // Destroy the operations.
user->destroy(); user->destroy();
useOp->destroy(); useOp->destroy();
} }
TEST(OperandStorageTest, RangeReplace) { TEST(OperandStorageTest, RangeReplace) {
MLIRContext context; MLIRContext context;
Builder builder(&context); Builder builder(&context);
Operation *useOp = Operation *useOp =
createOp(&context, /*operands=*/llvm::None, builder.getIntegerType(16)); createOp(&context, /*operands=*/std::nullopt, builder.getIntegerType(16));
Value operand = useOp->getResult(0); Value operand = useOp->getResult(0);
// Create a resizable operation with one operand. // Create a resizable operation with one operand.
Operation *user = createOp(&context, operand); Operation *user = createOp(&context, operand);
// Check setting with the same number of operands. // Check setting with the same number of operands.
user->setOperands(/*start=*/0, /*length=*/1, operand); user->setOperands(/*start=*/0, /*length=*/1, operand);
EXPECT_EQ(user->getNumOperands(), 1u); EXPECT_EQ(user->getNumOperands(), 1u);
Show All 19 LinesTEST(OperandStorageTest, RangeReplace) {
useOp->destroy(); useOp->destroy();
} }
TEST(OperandStorageTest, MutableRange) { TEST(OperandStorageTest, MutableRange) {
MLIRContext context; MLIRContext context;
Builder builder(&context); Builder builder(&context);
Operation *useOp = Operation *useOp =
createOp(&context, /*operands=*/llvm::None, builder.getIntegerType(16)); createOp(&context, /*operands=*/std::nullopt, builder.getIntegerType(16));
Value operand = useOp->getResult(0); Value operand = useOp->getResult(0);
// Create a resizable operation with one operand. // Create a resizable operation with one operand.
Operation *user = createOp(&context, operand); Operation *user = createOp(&context, operand);
// Check setting with the same number of operands. // Check setting with the same number of operands.
MutableOperandRange mutableOperands(user); MutableOperandRange mutableOperands(user);
mutableOperands.assign(operand); mutableOperands.assign(operand);
Show All 20 LinesTEST(OperandStorageTest, MutableRange) {
useOp->destroy(); useOp->destroy();
} }
TEST(OperandStorageTest, RangeErase) { TEST(OperandStorageTest, RangeErase) {
MLIRContext context; MLIRContext context;
Builder builder(&context); Builder builder(&context);
Type type = builder.getNoneType(); Type type = builder.getNoneType();
Operation *useOp = createOp(&context, /*operands=*/llvm::None, {type, type}); Operation *useOp =
createOp(&context, /*operands=*/std::nullopt, {type, type});
Value operand1 = useOp->getResult(0); Value operand1 = useOp->getResult(0);
Value operand2 = useOp->getResult(1); Value operand2 = useOp->getResult(1);
// Create an operation with operands to erase. // Create an operation with operands to erase.
Operation *user = Operation *user =
createOp(&context, {operand2, operand1, operand2, operand1}); createOp(&context, {operand2, operand1, operand2, operand1});
BitVector eraseIndices(user->getNumOperands()); BitVector eraseIndices(user->getNumOperands());
Show All 14 LinesTEST(OperandStorageTest, RangeErase) {
useOp->destroy(); useOp->destroy();
} }
TEST(OperationOrderTest, OrderIsAlwaysValid) { TEST(OperationOrderTest, OrderIsAlwaysValid) {
MLIRContext context; MLIRContext context;
Builder builder(&context); Builder builder(&context);
Operation *containerOp = Operation *containerOp =
createOp(&context, /*operands=*/llvm::None, /*resultTypes=*/llvm::None, createOp(&context, /*operands=*/std::nullopt, /*resultTypes=*/llvm::None,
/*numRegions=*/1); /*numRegions=*/1);
Region &region = containerOp->getRegion(0); Region &region = containerOp->getRegion(0);
Block *block = new Block(); Block *block = new Block();
region.push_back(block); region.push_back(block);
// Insert two operations, then iteratively add more operations in the middle // Insert two operations, then iteratively add more operations in the middle
// of them. Eventually we will insert more than kOrderStride operations and // of them. Eventually we will insert more than kOrderStride operations and
// the block order will need to be recomputed. // the block order will need to be recomputed.
▲ Show 20 Lines • Show All 92 Lines • Show Last 20 Lines

mlir/unittests/Pass/AnalysisManagerTest.cpp

Show First 20 Lines • Show All 59 Lines • ▼ Show 20 LinesTEST(AnalysisManagerTest, FineGrainFunctionAnalysisPreservation) {
MLIRContext context; MLIRContext context;
context.loadDialect<func::FuncDialect>(); context.loadDialect<func::FuncDialect>();
Builder builder(&context); Builder builder(&context);
// Create a function and a module. // Create a function and a module.
OwningOpRef<ModuleOp> module(ModuleOp::create(UnknownLoc::get(&context))); OwningOpRef<ModuleOp> module(ModuleOp::create(UnknownLoc::get(&context)));
func::FuncOp func1 = func::FuncOp func1 =
func::FuncOp::create(builder.getUnknownLoc(), "foo", func::FuncOp::create(builder.getUnknownLoc(), "foo",
builder.getFunctionType(llvm::None, llvm::None)); builder.getFunctionType(std::nullopt, llvm::None));
func1.setPrivate(); func1.setPrivate();
module->push_back(func1); module->push_back(func1);
// Test fine grain invalidation of the function analysis manager. // Test fine grain invalidation of the function analysis manager.
ModuleAnalysisManager mam(*module, /*passInstrumentor=*/nullptr); ModuleAnalysisManager mam(*module, /*passInstrumentor=*/nullptr);
AnalysisManager am = mam; AnalysisManager am = mam;
AnalysisManager fam = am.nest(func1); AnalysisManager fam = am.nest(func1);
Show All 14 LinesTEST(AnalysisManagerTest, FineGrainChildFunctionAnalysisPreservation) {
MLIRContext context; MLIRContext context;
context.loadDialect<func::FuncDialect>(); context.loadDialect<func::FuncDialect>();
Builder builder(&context); Builder builder(&context);
// Create a function and a module. // Create a function and a module.
OwningOpRef<ModuleOp> module(ModuleOp::create(UnknownLoc::get(&context))); OwningOpRef<ModuleOp> module(ModuleOp::create(UnknownLoc::get(&context)));
func::FuncOp func1 = func::FuncOp func1 =
func::FuncOp::create(builder.getUnknownLoc(), "foo", func::FuncOp::create(builder.getUnknownLoc(), "foo",
builder.getFunctionType(llvm::None, llvm::None)); builder.getFunctionType(std::nullopt, llvm::None));
func1.setPrivate(); func1.setPrivate();
module->push_back(func1); module->push_back(func1);
// Test fine grain invalidation of a function analysis from within a module // Test fine grain invalidation of a function analysis from within a module
// analysis manager. // analysis manager.
ModuleAnalysisManager mam(*module, /*passInstrumentor=*/nullptr); ModuleAnalysisManager mam(*module, /*passInstrumentor=*/nullptr);
AnalysisManager am = mam; AnalysisManager am = mam;
▲ Show 20 Lines • Show All 162 Lines • Show Last 20 Lines

mlir/unittests/Pass/PassManagerTest.cpp

Show First 20 Lines • Show All 56 Lines • ▼ Show 20 LinesTEST(PassManagerTest, OpSpecificAnalysis) {
context.loadDialect<func::FuncDialect>(); context.loadDialect<func::FuncDialect>();
Builder builder(&context); Builder builder(&context);
// Create a module with 2 functions. // Create a module with 2 functions.
OwningOpRef<ModuleOp> module(ModuleOp::create(UnknownLoc::get(&context))); OwningOpRef<ModuleOp> module(ModuleOp::create(UnknownLoc::get(&context)));
for (StringRef name : {"secret", "not_secret"}) { for (StringRef name : {"secret", "not_secret"}) {
auto func = auto func =
func::FuncOp::create(builder.getUnknownLoc(), name, func::FuncOp::create(builder.getUnknownLoc(), name,
builder.getFunctionType(llvm::None, llvm::None)); builder.getFunctionType(std::nullopt, llvm::None));
func.setPrivate(); func.setPrivate();
module->push_back(func); module->push_back(func);
} }
// Instantiate and run our pass. // Instantiate and run our pass.
PassManager pm(&context); PassManager pm(&context);
pm.addNestedPass<func::FuncOp>(std::make_unique<AnnotateFunctionPass>()); pm.addNestedPass<func::FuncOp>(std::make_unique<AnnotateFunctionPass>());
LogicalResult result = pm.run(module.get()); LogicalResult result = pm.run(module.get());
▲ Show 20 Lines • Show All 71 Lines • Show Last 20 Lines

mlir/unittests/TableGen/EnumsGenTest.cpp

Show First 20 Lines • Show All 50 Lines • ▼ Show 20 Lines
TEST(EnumsGenTest, GeneratedSymbolToStringFn) { TEST(EnumsGenTest, GeneratedSymbolToStringFn) {
EXPECT_EQ(ConvertToString(FooEnum::CaseA), "CaseA"); EXPECT_EQ(ConvertToString(FooEnum::CaseA), "CaseA");
EXPECT_EQ(ConvertToString(FooEnum::CaseB), "CaseB"); EXPECT_EQ(ConvertToString(FooEnum::CaseB), "CaseB");
} }
TEST(EnumsGenTest, GeneratedStringToSymbolFn) { TEST(EnumsGenTest, GeneratedStringToSymbolFn) {
EXPECT_EQ(llvm::Optional<FooEnum>(FooEnum::CaseA), ConvertToEnum("CaseA")); EXPECT_EQ(llvm::Optional<FooEnum>(FooEnum::CaseA), ConvertToEnum("CaseA"));
EXPECT_EQ(llvm::Optional<FooEnum>(FooEnum::CaseB), ConvertToEnum("CaseB")); EXPECT_EQ(llvm::Optional<FooEnum>(FooEnum::CaseB), ConvertToEnum("CaseB"));
EXPECT_EQ(llvm::None, ConvertToEnum("X")); EXPECT_EQ(std::nullopt, ConvertToEnum("X"));
} }
TEST(EnumsGenTest, GeneratedUnderlyingType) { TEST(EnumsGenTest, GeneratedUnderlyingType) {
bool v = std::is_same<uint32_t, std::underlying_type<I32Enum>::type>::value; bool v = std::is_same<uint32_t, std::underlying_type<I32Enum>::type>::value;
EXPECT_TRUE(v); EXPECT_TRUE(v);
} }
TEST(EnumsGenTest, GeneratedBitEnumDefinition) { TEST(EnumsGenTest, GeneratedBitEnumDefinition) {
Show All 21 Lines
TEST(EnumsGenTest, GeneratedStringToSymbolForBitEnum) { TEST(EnumsGenTest, GeneratedStringToSymbolForBitEnum) {
EXPECT_EQ(symbolizeBitEnumWithNone("None"), BitEnumWithNone::None); EXPECT_EQ(symbolizeBitEnumWithNone("None"), BitEnumWithNone::None);
EXPECT_EQ(symbolizeBitEnumWithNone("Bit0"), BitEnumWithNone::Bit0); EXPECT_EQ(symbolizeBitEnumWithNone("Bit0"), BitEnumWithNone::Bit0);
EXPECT_EQ(symbolizeBitEnumWithNone("Bit3"), BitEnumWithNone::Bit3); EXPECT_EQ(symbolizeBitEnumWithNone("Bit3"), BitEnumWithNone::Bit3);
EXPECT_EQ(symbolizeBitEnumWithNone("Bit3|Bit0"), EXPECT_EQ(symbolizeBitEnumWithNone("Bit3|Bit0"),
BitEnumWithNone::Bit3 | BitEnumWithNone::Bit0); BitEnumWithNone::Bit3 | BitEnumWithNone::Bit0);
EXPECT_EQ(symbolizeBitEnumWithNone("Bit2"), llvm::None); EXPECT_EQ(symbolizeBitEnumWithNone("Bit2"), std::nullopt);
EXPECT_EQ(symbolizeBitEnumWithNone("Bit3 | Bit4"), llvm::None); EXPECT_EQ(symbolizeBitEnumWithNone("Bit3 | Bit4"), std::nullopt);
EXPECT_EQ(symbolizeBitEnumWithoutNone("None"), llvm::None); EXPECT_EQ(symbolizeBitEnumWithoutNone("None"), std::nullopt);
} }
TEST(EnumsGenTest, GeneratedSymbolToStringFnForGroupedBitEnum) { TEST(EnumsGenTest, GeneratedSymbolToStringFnForGroupedBitEnum) {
EXPECT_EQ(stringifyBitEnumWithGroup(BitEnumWithGroup::Bit0), "Bit0"); EXPECT_EQ(stringifyBitEnumWithGroup(BitEnumWithGroup::Bit0), "Bit0");
EXPECT_EQ(stringifyBitEnumWithGroup(BitEnumWithGroup::Bit3), "Bit3"); EXPECT_EQ(stringifyBitEnumWithGroup(BitEnumWithGroup::Bit3), "Bit3");
EXPECT_EQ(stringifyBitEnumWithGroup(BitEnumWithGroup::Bits0To3), EXPECT_EQ(stringifyBitEnumWithGroup(BitEnumWithGroup::Bits0To3),
"Bit0|Bit1|Bit2|Bit3|Bits0To3"); "Bit0|Bit1|Bit2|Bit3|Bits0To3");
EXPECT_EQ(stringifyBitEnumWithGroup(BitEnumWithGroup::Bit4), "Bit4"); EXPECT_EQ(stringifyBitEnumWithGroup(BitEnumWithGroup::Bit4), "Bit4");
EXPECT_EQ(stringifyBitEnumWithGroup( EXPECT_EQ(stringifyBitEnumWithGroup(
BitEnumWithGroup::Bit0 | BitEnumWithGroup::Bit1 | BitEnumWithGroup::Bit0 | BitEnumWithGroup::Bit1 |
BitEnumWithGroup::Bit2 | BitEnumWithGroup::Bit4), BitEnumWithGroup::Bit2 | BitEnumWithGroup::Bit4),
"Bit0|Bit1|Bit2|Bit4"); "Bit0|Bit1|Bit2|Bit4");
} }
TEST(EnumsGenTest, GeneratedStringToSymbolForGroupedBitEnum) { TEST(EnumsGenTest, GeneratedStringToSymbolForGroupedBitEnum) {
EXPECT_EQ(symbolizeBitEnumWithGroup("Bit0"), BitEnumWithGroup::Bit0); EXPECT_EQ(symbolizeBitEnumWithGroup("Bit0"), BitEnumWithGroup::Bit0);
EXPECT_EQ(symbolizeBitEnumWithGroup("Bit3"), BitEnumWithGroup::Bit3); EXPECT_EQ(symbolizeBitEnumWithGroup("Bit3"), BitEnumWithGroup::Bit3);
EXPECT_EQ(symbolizeBitEnumWithGroup("Bit5"), llvm::None); EXPECT_EQ(symbolizeBitEnumWithGroup("Bit5"), std::nullopt);
EXPECT_EQ(symbolizeBitEnumWithGroup("Bit3|Bit0"), EXPECT_EQ(symbolizeBitEnumWithGroup("Bit3|Bit0"),
BitEnumWithGroup::Bit3 | BitEnumWithGroup::Bit0); BitEnumWithGroup::Bit3 | BitEnumWithGroup::Bit0);
} }
TEST(EnumsGenTest, GeneratedSymbolToStringFnForPrimaryGroupBitEnum) { TEST(EnumsGenTest, GeneratedSymbolToStringFnForPrimaryGroupBitEnum) {
EXPECT_EQ(stringifyBitEnumPrimaryGroup( EXPECT_EQ(stringifyBitEnumPrimaryGroup(
BitEnumPrimaryGroup::Bit0 | BitEnumPrimaryGroup::Bit1 | BitEnumPrimaryGroup::Bit0 | BitEnumPrimaryGroup::Bit1 |
BitEnumPrimaryGroup::Bit2 | BitEnumPrimaryGroup::Bit3), BitEnumPrimaryGroup::Bit2 | BitEnumPrimaryGroup::Bit3),
▲ Show 20 Lines • Show All 69 Lines • Show Last 20 Lines

mlir/unittests/Transforms/DialectConversion.cpp

//===- DialectConversion.cpp - Dialect conversion unit tests --------------===// //===- DialectConversion.cpp - Dialect conversion unit tests --------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "gtest/gtest.h" #include "gtest/gtest.h"
using namespace mlir; using namespace mlir;
static Operation *createOp(MLIRContext *context) { static Operation *createOp(MLIRContext *context) {
context->allowUnregisteredDialects(); context->allowUnregisteredDialects();
return Operation::create(UnknownLoc::get(context), return Operation::create(UnknownLoc::get(context),
OperationName("foo.bar", context), llvm::None, OperationName("foo.bar", context), std::nullopt,
llvm::None, llvm::None, llvm::None, 0); std::nullopt, llvm::None, llvm::None, 0);
kazuUnsubmitted
Not Done
ReplyInline Actions

Did you mean to replace these also?

kazu: Did you mean to replace these also?
} }
namespace { namespace {
struct DummyOp { struct DummyOp {
MLIR_DEFINE_EXPLICIT_INTERNAL_INLINE_TYPE_ID(DummyOp) MLIR_DEFINE_EXPLICIT_INTERNAL_INLINE_TYPE_ID(DummyOp)
static StringRef getOperationName() { return "foo.bar"; } static StringRef getOperationName() { return "foo.bar"; }
}; };
TEST(DialectConversionTest, DynamicallyLegalOpCallbackOrder) { TEST(DialectConversionTest, DynamicallyLegalOpCallbackOrder) {
MLIRContext context; MLIRContext context;
ConversionTarget target(context); ConversionTarget target(context);
int index = 0; int index = 0;
int callbackCalled1 = 0; int callbackCalled1 = 0;
target.addDynamicallyLegalOp<DummyOp>([&](Operation *) { target.addDynamicallyLegalOp<DummyOp>([&](Operation *) {
callbackCalled1 = ++index; callbackCalled1 = ++index;
return true; return true;
}); });
int callbackCalled2 = 0; int callbackCalled2 = 0;
target.addDynamicallyLegalOp<DummyOp>([&](Operation *) -> Optional<bool> { target.addDynamicallyLegalOp<DummyOp>([&](Operation *) -> Optional<bool> {
callbackCalled2 = ++index; callbackCalled2 = ++index;
return llvm::None; return std::nullopt;
}); });
auto *op = createOp(&context); auto *op = createOp(&context);
EXPECT_TRUE(target.isLegal(op)); EXPECT_TRUE(target.isLegal(op));
EXPECT_EQ(2, callbackCalled1); EXPECT_EQ(2, callbackCalled1);
EXPECT_EQ(1, callbackCalled2); EXPECT_EQ(1, callbackCalled2);
EXPECT_FALSE(target.isIllegal(op)); EXPECT_FALSE(target.isIllegal(op));
EXPECT_EQ(4, callbackCalled1); EXPECT_EQ(4, callbackCalled1);
EXPECT_EQ(3, callbackCalled2); EXPECT_EQ(3, callbackCalled2);
op->destroy(); op->destroy();
} }
TEST(DialectConversionTest, DynamicallyLegalOpCallbackSkip) { TEST(DialectConversionTest, DynamicallyLegalOpCallbackSkip) {
MLIRContext context; MLIRContext context;
ConversionTarget target(context); ConversionTarget target(context);
int index = 0; int index = 0;
int callbackCalled = 0; int callbackCalled = 0;
target.addDynamicallyLegalOp<DummyOp>([&](Operation *) -> Optional<bool> { target.addDynamicallyLegalOp<DummyOp>([&](Operation *) -> Optional<bool> {
callbackCalled = ++index; callbackCalled = ++index;
return llvm::None; return std::nullopt;
}); });
auto *op = createOp(&context); auto *op = createOp(&context);
EXPECT_FALSE(target.isLegal(op)); EXPECT_FALSE(target.isLegal(op));
EXPECT_EQ(1, callbackCalled); EXPECT_EQ(1, callbackCalled);
EXPECT_FALSE(target.isIllegal(op)); EXPECT_FALSE(target.isIllegal(op));
EXPECT_EQ(2, callbackCalled); EXPECT_EQ(2, callbackCalled);
op->destroy(); op->destroy();
} }
TEST(DialectConversionTest, DynamicallyLegalUnknownOpCallbackOrder) { TEST(DialectConversionTest, DynamicallyLegalUnknownOpCallbackOrder) {
MLIRContext context; MLIRContext context;
ConversionTarget target(context); ConversionTarget target(context);
int index = 0; int index = 0;
int callbackCalled1 = 0; int callbackCalled1 = 0;
target.markUnknownOpDynamicallyLegal([&](Operation *) { target.markUnknownOpDynamicallyLegal([&](Operation *) {
callbackCalled1 = ++index; callbackCalled1 = ++index;
return true; return true;
}); });
int callbackCalled2 = 0; int callbackCalled2 = 0;
target.markUnknownOpDynamicallyLegal([&](Operation *) -> Optional<bool> { target.markUnknownOpDynamicallyLegal([&](Operation *) -> Optional<bool> {
callbackCalled2 = ++index; callbackCalled2 = ++index;
return llvm::None; return std::nullopt;
}); });
auto *op = createOp(&context); auto *op = createOp(&context);
EXPECT_TRUE(target.isLegal(op)); EXPECT_TRUE(target.isLegal(op));
EXPECT_EQ(2, callbackCalled1); EXPECT_EQ(2, callbackCalled1);
EXPECT_EQ(1, callbackCalled2); EXPECT_EQ(1, callbackCalled2);
EXPECT_FALSE(target.isIllegal(op)); EXPECT_FALSE(target.isIllegal(op));
EXPECT_EQ(4, callbackCalled1); EXPECT_EQ(4, callbackCalled1);
EXPECT_EQ(3, callbackCalled2); EXPECT_EQ(3, callbackCalled2);
op->destroy(); op->destroy();
} }
TEST(DialectConversionTest, DynamicallyLegalReturnNone) { TEST(DialectConversionTest, DynamicallyLegalReturnNone) {
MLIRContext context; MLIRContext context;
ConversionTarget target(context); ConversionTarget target(context);
target.addDynamicallyLegalOp<DummyOp>( target.addDynamicallyLegalOp<DummyOp>(
[&](Operation *) -> Optional<bool> { return llvm::None; }); [&](Operation *) -> Optional<bool> { return std::nullopt; });
auto *op = createOp(&context); auto *op = createOp(&context);
EXPECT_FALSE(target.isLegal(op)); EXPECT_FALSE(target.isLegal(op));
EXPECT_FALSE(target.isIllegal(op)); EXPECT_FALSE(target.isIllegal(op));
EXPECT_TRUE(succeeded(applyPartialConversion(op, target, {}))); EXPECT_TRUE(succeeded(applyPartialConversion(op, target, {})));
EXPECT_TRUE(failed(applyFullConversion(op, target, {}))); EXPECT_TRUE(failed(applyFullConversion(op, target, {})));
op->destroy(); op->destroy();
} }
TEST(DialectConversionTest, DynamicallyLegalUnknownReturnNone) { TEST(DialectConversionTest, DynamicallyLegalUnknownReturnNone) {
MLIRContext context; MLIRContext context;
ConversionTarget target(context); ConversionTarget target(context);
target.markUnknownOpDynamicallyLegal( target.markUnknownOpDynamicallyLegal(
[&](Operation *) -> Optional<bool> { return llvm::None; }); [&](Operation *) -> Optional<bool> { return std::nullopt; });
auto *op = createOp(&context); auto *op = createOp(&context);
EXPECT_FALSE(target.isLegal(op)); EXPECT_FALSE(target.isLegal(op));
EXPECT_FALSE(target.isIllegal(op)); EXPECT_FALSE(target.isIllegal(op));
EXPECT_TRUE(succeeded(applyPartialConversion(op, target, {}))); EXPECT_TRUE(succeeded(applyPartialConversion(op, target, {})));
EXPECT_TRUE(failed(applyFullConversion(op, target, {}))); EXPECT_TRUE(failed(applyFullConversion(op, target, {})));
op->destroy(); op->destroy();
} }
} // namespace } // namespace