This is an archive of the discontinued LLVM Phabricator instance.

Differential D76161

NFC - Moved AffineOps IR / Transforms
ClosedPublic

Authored by rsuderman on Mar 13 2020, 1:55 PM.

Details

Reviewers
rriddle
ftynse
nicolasvasilache
herhut
bondhugula
mravishankar
Commits
rGe708471395b6: [mlir][NFC] Cleanup AffineOps directory structure
Summary

NFC - Moved AffineOps related transforms

Change AffineOps Dialect structure to better group both IR and Tranforms. This included extracting transforms directly related to AffineOps. Also move AffineOps to Affine.

Diff Detail

Event Timeline

rsuderman created this revision.Mar 13 2020, 1:55 PM
Herald added a reviewer: mravishankar. · View Herald TranscriptMar 13 2020, 1:55 PM
Herald added a reviewer: nicolasvasilache. · View Herald Transcript
Herald added a reviewer: herhut. · View Herald Transcript
Herald added a reviewer: aartbik. · View Herald Transcript
Herald added a project: Restricted Project. · View Herald Transcript
Herald added subscribers: llvm-commits, Joonsoo, liufengdb and 15 others. · View Herald Transcript
Harbormaster completed remote builds in B49184: Diff 250297.Mar 13 2020, 2:03 PM
nicolasvasilache added a subscriber: ftynse.Mar 13 2020, 2:53 PM

I think this has a lot of conflict potential with a revision by @ftynse, maybe check with him?

rsuderman updated this revision to Diff 250324.Mar 13 2020, 4:32 PM

Updated CMake build to pass.

Updating D76161: Moved AffineOps IR / Transforms

Change AffineOps Dialect structure to better group both IR and Tranforms. This
included extracting transforms direclty related to AffineOps

rsuderman updated this revision to Diff 250325.Mar 13 2020, 4:33 PM

Typo

Updating D76161: Moved AffineOps IR / Transforms

Change AffineOps Dialect structure to better group both IR and Tranforms. This
included extracting transforms direclty related to AffineOps

rsuderman added a comment.Mar 13 2020, 4:33 PM
In D76161#1922471, @nicolasvasilache wrote:

I think this has a lot of conflict potential with a revision by @ftynse, maybe check with him?

I'll make sure to loop him in on the review.

rsuderman retitled this revision from Moved AffineOps IR / Transforms Change AffineOps Dialect structure to better group both IR and Tranforms. This included extracting transforms direclty related to AffineOps to Moved AffineOps IR / TransformsChange AffineOps Dialect structure to better group both IR and Tranforms. Thisincluded extracting transforms direclty related to AffineOps.Mar 13 2020, 4:34 PM
rsuderman removed reviewers: mravishankar, nicolasvasilache, herhut, aartbik.
Herald added a reviewer: mravishankar. · View Herald TranscriptMar 13 2020, 4:34 PM
Herald added a reviewer: nicolasvasilache. · View Herald Transcript
Herald added a reviewer: herhut. · View Herald Transcript
rsuderman edited reviewers, added: rriddle, ftynse; removed: mravishankar, nicolasvasilache, herhut.Mar 13 2020, 4:34 PM
Herald added a reviewer: mravishankar. · View Herald TranscriptMar 13 2020, 4:34 PM
Herald added a reviewer: nicolasvasilache. · View Herald Transcript
Herald added a reviewer: herhut. · View Herald Transcript
mehdi_amini added inline comments.Mar 13 2020, 5:11 PM
mlir/test/lib/Transforms/TestVectorizationUtils.cpp
16

Since you had to change all of them, and since every downstream user will have to adjust, what about also renaming the mlir/Dialect/AffineOps into mlir/Dialect/Affine ?

Harbormaster failed remote builds in B49192: Diff 250325!Mar 13 2020, 5:17 PM
Harbormaster failed remote builds in B49191: Diff 250324!
rsuderman updated this revision to Diff 250338.Mar 13 2020, 6:21 PM

clang-format

Harbormaster failed remote builds in B49203: Diff 250338!Mar 13 2020, 6:54 PM
bondhugula requested changes to this revision.Mar 13 2020, 9:03 PM
bondhugula added a reviewer: bondhugula.
bondhugula added a subscriber: bondhugula.

Please see this discussion started by @ftynse
https://llvm.discourse.group/t/rfc-canonical-file-paths-to-dialects/621
If we are doing a move, we might as well get the directory names renamed consistently. AffineOps -> Affine

bondhugula requested changes to this revision.Mar 13 2020, 9:04 PM
This revision now requires changes to proceed.Mar 13 2020, 9:04 PM
nicolasvasilache added a comment.Mar 14 2020, 6:22 AM

@rsuderman @mehdi_amini @bondhugula the renaming AffineOps -> Affine is part of the conflicting changes @ftynse has in flight and was holding off on submitting because internal integration would be impacted.

rriddle added a comment.Mar 14 2020, 9:04 AM
In D76161#1922847, @nicolasvasilache wrote:

@rsuderman @mehdi_amini @bondhugula the renaming AffineOps -> Affine is part of the conflicting changes @ftynse has in flight and was holding off on submitting because internal integration would be impacted.

Its been over a week, that isn't great justification. I'm pretty sure he's just on vacation.

mlir/include/mlir/InitAllPasses.h
29

Duplicate?

rsuderman updated this revision to Diff 250600.Mar 16 2020, 11:02 AM
rsuderman marked an inline comment as done.

Move AffineOps to Affine

rsuderman marked an inline comment as done and an inline comment as not done.Mar 16 2020, 11:04 AM

Should be updated for comments.

mlir/include/mlir/InitAllPasses.h
29

I don't see a duplicate? AffineOps/Passes.h was newly added in this CL.

mlir/test/lib/Transforms/TestVectorizationUtils.cpp
16

Moved AffineOps to Affine.

rsuderman retitled this revision from Moved AffineOps IR / TransformsChange AffineOps Dialect structure to better group both IR and Tranforms. Thisincluded extracting transforms direclty related to AffineOps to NFC - Moved AffineOps IR / Transforms.Mar 16 2020, 11:21 AM
rsuderman edited the summary of this revision. (Show Details)
bmahjour removed a subscriber: bmahjour.Mar 16 2020, 11:52 AM
rriddle accepted this revision.Mar 16 2020, 11:52 AM
rriddle marked an inline comment as done.
rriddle added inline comments.
mlir/include/mlir/Dialect/Affine/EDSC/Intrinsics.h
7–8

Can you update each of these header guard comments now?

mlir/include/mlir/Dialect/Affine/Passes.h
10

AffineOps -> Affine

mlir/include/mlir/InitAllPasses.h
29

It was previously showing + #include "mlir/Dialect/FxpMathOps/Passes.h"

Harbormaster failed remote builds in B49337: Diff 250600!Mar 16 2020, 12:01 PM
rsuderman updated this revision to Diff 250614.Mar 16 2020, 12:24 PM

Update for rriddle@ comments and synced to head.

rsuderman marked 5 inline comments as done.Mar 16 2020, 12:26 PM

Addressed rriddle@ comments and synced to head.

Harbormaster failed remote builds in B49347: Diff 250614!Mar 16 2020, 1:08 PM
rsuderman updated this revision to Diff 250621.Mar 16 2020, 1:14 PM

git clang-format

Harbormaster failed remote builds in B49353: Diff 250621!Mar 16 2020, 2:13 PM
mravishankar resigned from this revision.Mar 17 2020, 12:46 PM
rsuderman updated this revision to Diff 250921.Mar 17 2020, 3:00 PM

Sync to head and reupdate.

Harbormaster completed remote builds in B49511: Diff 250921.Mar 17 2020, 3:40 PM
rriddle added a comment.Mar 18 2020, 8:33 PM

Can you sync to head?

rsuderman updated this revision to Diff 251492.Mar 19 2020, 3:43 PM

Sync to head

Harbormaster failed remote builds in B49807: Diff 251492!Mar 19 2020, 4:28 PM
bondhugula accepted this revision.Mar 19 2020, 6:41 PM
This revision is now accepted and ready to land.Mar 19 2020, 6:41 PM
rsuderman updated this revision to Diff 251715.Mar 20 2020, 12:03 PM

Synced to head.

Harbormaster failed remote builds in B49932: Diff 251715!Mar 20 2020, 1:01 PM
Closed by commit rGe708471395b6: [mlir][NFC] Cleanup AffineOps directory structure (authored by rsuderman, committed by rriddle). · Explain WhyMar 20 2020, 2:40 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents

PathSize
mlir/
docs/
2 lines
Tutorials/
Toy/
2 lines
examples/
toy/
Ch5/
mlir/
4 lines
Ch6/
mlir/
4 lines
2 lines
Ch7/
mlir/
4 lines
2 lines
include/
mlir/
Dialect/
Affine/
1 line
48 lines
Affine/
EDSC/
AffineOps/
EDSC/
8 lines
6 lines
Affine/
IR/
AffineOps/
8 lines
8 lines
6 lines
AffineOps/
EDSC/
2 lines
Linalg/
IR/
2 lines
2 lines
Transforms/
2 lines
Vector/
2 lines
VectorOps/
Vector/
9 lines
4 lines
2 lines
Transforms/
13 lines
lib/
Analysis/
4 lines
4 lines
4 lines
4 lines
2 lines
2 lines
4 lines
Conversion/
AffineToStandard/
2 lines
2 lines
LoopsToGPU/
2 lines
2 lines
4 lines
VectorToLoops/
2 lines
Dialect/
Affine/
21 lines
Transforms/
21 lines
Affine/
EDSC/
AffineOps/
EDSC/
2 lines
Affine/
EDSC/
AffineOps/
8 lines
IR/
AffineOps/
16 lines
4 lines
7 lines
AffineOps/
EDSC/
2 lines
Linalg/
EDSC/
2 lines
2 lines
Transforms/
2 lines
2 lines
2 lines
2 lines
Utils/
2 lines
2 lines
LoopOps/
Transforms/
2 lines
2 lines
2 lines
Vector/
2 lines
2 lines
2 lines
VectorOps/
Vector/
182 lines
8 lines
Dialect/
Affine/
Transforms/
Transforms/
4 lines
2 lines
5 lines
Transforms/
5 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
Utils/
2 lines
2 lines
2 lines
2 lines
2 lines
test/
Dialect/
Affine/
AffineOps/
AffineOps/
EDSC/
4 lines
4 lines
lib/
Dialect/
Affine/
14 lines
1 line
Dialect/
Affine/
Transforms/
2 lines
Transforms/
3 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
tools/
mlir-opt/
1 line

Diff 251772

mlir/docs/Interfaces.md

Show First 20 Lines • Show All 57 Lines • ▼ Show 20 Linesstruct AffineInlinerInterface : public DialectInlinerInterface {
/// Affine structures have specific inlining constraints. /// Affine structures have specific inlining constraints.
bool isLegalToInline(Region *dest, Region *src, bool isLegalToInline(Region *dest, Region *src,
BlockAndValueMapping &valueMapping) const final { BlockAndValueMapping &valueMapping) const final {
... ...
} }
}; };
/// Register the interface with the dialect. /// Register the interface with the dialect.
AffineOpsDialect::AffineOpsDialect(MLIRContext *context) ... { AffineDialect::AffineDialect(MLIRContext *context) ... {
addInterfaces<AffineInlinerInterface>(); addInterfaces<AffineInlinerInterface>();
} }
``` ```
Once registered, these interfaces can be opaquely queried from the dialect by Once registered, these interfaces can be opaquely queried from the dialect by
the transformation/analysis that wants to use them: the transformation/analysis that wants to use them:
```c++ ```c++
▲ Show 20 Lines • Show All 126 Lines • Show Last 20 Lines

mlir/docs/Tutorials/Toy/Ch-5.md

Show First 20 Lines • Show All 56 Lines • ▼ Show 20 Lines
void ToyToAffineLoweringPass::runOnFunction() { void ToyToAffineLoweringPass::runOnFunction() {
// The first thing to define is the conversion target. This will define the // The first thing to define is the conversion target. This will define the
// final target for this lowering. // final target for this lowering.
mlir::ConversionTarget target(getContext()); mlir::ConversionTarget target(getContext());
// We define the specific operations, or dialects, that are legal targets for // We define the specific operations, or dialects, that are legal targets for
// this lowering. In our case, we are lowering to a combination of the // this lowering. In our case, we are lowering to a combination of the
// `Affine` and `Standard` dialects. // `Affine` and `Standard` dialects.
target.addLegalDialect<mlir::AffineOpsDialect, mlir::StandardOpsDialect>(); target.addLegalDialect<mlir::AffineDialect, mlir::StandardOpsDialect>();
// We also define the Toy dialect as Illegal so that the conversion will fail // We also define the Toy dialect as Illegal so that the conversion will fail
// if any of these operations are *not* converted. Given that we actually want // if any of these operations are *not* converted. Given that we actually want
// a partial lowering, we explicitly mark the Toy operations that don't want // a partial lowering, we explicitly mark the Toy operations that don't want
// to lower, `toy.print`, as *legal*. // to lower, `toy.print`, as *legal*.
target.addIllegalDialect<ToyDialect>(); target.addIllegalDialect<ToyDialect>();
target.addLegalOp<PrintOp>(); target.addLegalOp<PrintOp>();
... ...
▲ Show 20 Lines • Show All 271 Lines • Show Last 20 Lines

mlir/examples/toy/Ch5/mlir/LowerToAffineLoops.cpp

Show All 9 Lines
// affine loops and standard operations. This lowering expects that all calls // affine loops and standard operations. This lowering expects that all calls
// have been inlined, and all shapes have been resolved. // have been inlined, and all shapes have been resolved.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "toy/Dialect.h" #include "toy/Dialect.h"
#include "toy/Passes.h" #include "toy/Passes.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/ADT/Sequence.h" #include "llvm/ADT/Sequence.h"
using namespace mlir; using namespace mlir;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 248 Lines • ▼ Show 20 Linesvoid ToyToAffineLoweringPass::runOnFunction() {
// The first thing to define is the conversion target. This will define the // The first thing to define is the conversion target. This will define the
// final target for this lowering. // final target for this lowering.
ConversionTarget target(getContext()); ConversionTarget target(getContext());
// We define the specific operations, or dialects, that are legal targets for // We define the specific operations, or dialects, that are legal targets for
// this lowering. In our case, we are lowering to a combination of the // this lowering. In our case, we are lowering to a combination of the
// `Affine` and `Standard` dialects. // `Affine` and `Standard` dialects.
target.addLegalDialect<AffineOpsDialect, StandardOpsDialect>(); target.addLegalDialect<AffineDialect, StandardOpsDialect>();
// We also define the Toy dialect as Illegal so that the conversion will fail // We also define the Toy dialect as Illegal so that the conversion will fail
// if any of these operations are *not* converted. Given that we actually want // if any of these operations are *not* converted. Given that we actually want
// a partial lowering, we explicitly mark the Toy operations that don't want // a partial lowering, we explicitly mark the Toy operations that don't want
// to lower, `toy.print`, as `legal`. // to lower, `toy.print`, as `legal`.
target.addIllegalDialect<toy::ToyDialect>(); target.addIllegalDialect<toy::ToyDialect>();
target.addLegalOp<toy::PrintOp>(); target.addLegalOp<toy::PrintOp>();
Show All 18 Lines

mlir/examples/toy/Ch6/mlir/LowerToAffineLoops.cpp

Show All 9 Lines
// affine loops and standard operations. This lowering expects that all calls // affine loops and standard operations. This lowering expects that all calls
// have been inlined, and all shapes have been resolved. // have been inlined, and all shapes have been resolved.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "toy/Dialect.h" #include "toy/Dialect.h"
#include "toy/Passes.h" #include "toy/Passes.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/ADT/Sequence.h" #include "llvm/ADT/Sequence.h"
using namespace mlir; using namespace mlir;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 248 Lines • ▼ Show 20 Linesvoid ToyToAffineLoweringPass::runOnFunction() {
// The first thing to define is the conversion target. This will define the // The first thing to define is the conversion target. This will define the
// final target for this lowering. // final target for this lowering.
ConversionTarget target(getContext()); ConversionTarget target(getContext());
// We define the specific operations, or dialects, that are legal targets for // We define the specific operations, or dialects, that are legal targets for
// this lowering. In our case, we are lowering to a combination of the // this lowering. In our case, we are lowering to a combination of the
// `Affine` and `Standard` dialects. // `Affine` and `Standard` dialects.
target.addLegalDialect<AffineOpsDialect, StandardOpsDialect>(); target.addLegalDialect<AffineDialect, StandardOpsDialect>();
// We also define the Toy dialect as Illegal so that the conversion will fail // We also define the Toy dialect as Illegal so that the conversion will fail
// if any of these operations are *not* converted. Given that we actually want // if any of these operations are *not* converted. Given that we actually want
// a partial lowering, we explicitly mark the Toy operations that don't want // a partial lowering, we explicitly mark the Toy operations that don't want
// to lower, `toy.print`, as `legal`. // to lower, `toy.print`, as `legal`.
target.addIllegalDialect<toy::ToyDialect>(); target.addIllegalDialect<toy::ToyDialect>();
target.addLegalOp<toy::PrintOp>(); target.addLegalOp<toy::PrintOp>();
Show All 18 Lines

mlir/examples/toy/Ch6/mlir/LowerToLLVM.cpp

Show All 13 Lines
#include "toy/Dialect.h" #include "toy/Dialect.h"
#include "toy/Passes.h" #include "toy/Passes.h"
#include "mlir/Conversion/AffineToStandard/AffineToStandard.h" #include "mlir/Conversion/AffineToStandard/AffineToStandard.h"
#include "mlir/Conversion/LoopToStandard/ConvertLoopToStandard.h" #include "mlir/Conversion/LoopToStandard/ConvertLoopToStandard.h"
#include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h" #include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h"
#include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVMPass.h" #include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVMPass.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/LoopOps/LoopOps.h" #include "mlir/Dialect/LoopOps/LoopOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/ADT/Sequence.h" #include "llvm/ADT/Sequence.h"
using namespace mlir; using namespace mlir;
▲ Show 20 Lines • Show All 173 Lines • Show Last 20 Lines

mlir/examples/toy/Ch7/mlir/LowerToAffineLoops.cpp

Show All 9 Lines
// affine loops and standard operations. This lowering expects that all calls // affine loops and standard operations. This lowering expects that all calls
// have been inlined, and all shapes have been resolved. // have been inlined, and all shapes have been resolved.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "toy/Dialect.h" #include "toy/Dialect.h"
#include "toy/Passes.h" #include "toy/Passes.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/ADT/Sequence.h" #include "llvm/ADT/Sequence.h"
using namespace mlir; using namespace mlir;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 248 Lines • ▼ Show 20 Linesvoid ToyToAffineLoweringPass::runOnFunction() {
// The first thing to define is the conversion target. This will define the // The first thing to define is the conversion target. This will define the
// final target for this lowering. // final target for this lowering.
ConversionTarget target(getContext()); ConversionTarget target(getContext());
// We define the specific operations, or dialects, that are legal targets for // We define the specific operations, or dialects, that are legal targets for
// this lowering. In our case, we are lowering to a combination of the // this lowering. In our case, we are lowering to a combination of the
// `Affine` and `Standard` dialects. // `Affine` and `Standard` dialects.
target.addLegalDialect<AffineOpsDialect, StandardOpsDialect>(); target.addLegalDialect<AffineDialect, StandardOpsDialect>();
// We also define the Toy dialect as Illegal so that the conversion will fail // We also define the Toy dialect as Illegal so that the conversion will fail
// if any of these operations are *not* converted. Given that we actually want // if any of these operations are *not* converted. Given that we actually want
// a partial lowering, we explicitly mark the Toy operations that don't want // a partial lowering, we explicitly mark the Toy operations that don't want
// to lower, `toy.print`, as `legal`. // to lower, `toy.print`, as `legal`.
target.addIllegalDialect<toy::ToyDialect>(); target.addIllegalDialect<toy::ToyDialect>();
target.addLegalOp<toy::PrintOp>(); target.addLegalOp<toy::PrintOp>();
Show All 18 Lines

mlir/examples/toy/Ch7/mlir/LowerToLLVM.cpp

Show All 13 Lines
#include "toy/Dialect.h" #include "toy/Dialect.h"
#include "toy/Passes.h" #include "toy/Passes.h"
#include "mlir/Conversion/AffineToStandard/AffineToStandard.h" #include "mlir/Conversion/AffineToStandard/AffineToStandard.h"
#include "mlir/Conversion/LoopToStandard/ConvertLoopToStandard.h" #include "mlir/Conversion/LoopToStandard/ConvertLoopToStandard.h"
#include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h" #include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVM.h"
#include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVMPass.h" #include "mlir/Conversion/StandardToLLVM/ConvertStandardToLLVMPass.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/LoopOps/LoopOps.h" #include "mlir/Dialect/LoopOps/LoopOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/ADT/Sequence.h" #include "llvm/ADT/Sequence.h"
using namespace mlir; using namespace mlir;
▲ Show 20 Lines • Show All 173 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Affine/CMakeLists.txt

  • This file was added.
add_subdirectory(IR)

mlir/include/mlir/Dialect/Affine/EDSC/Builders.h

  • This file was moved from mlir/include/mlir/Dialect/AffineOps/EDSC/Builders.h.
//===- Builders.h - MLIR Declarative Builder Classes ------------*- C++ -*-===// //===- Builders.h - MLIR Declarative Builder Classes ------------*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// Provides intuitive composable interfaces for building structured MLIR // Provides intuitive composable interfaces for building structured MLIR
// snippets in a declarative fashion. // snippets in a declarative fashion.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef MLIR_DIALECT_AFFINEOPS_EDSC_BUILDERS_H_ #ifndef MLIR_DIALECT_AFFINE_EDSC_BUILDERS_H_
#define MLIR_DIALECT_AFFINEOPS_EDSC_BUILDERS_H_ #define MLIR_DIALECT_AFFINE_EDSC_BUILDERS_H_
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/EDSC/Builders.h" #include "mlir/EDSC/Builders.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/Types.h" #include "mlir/IR/Types.h"
namespace mlir { namespace mlir {
namespace edsc { namespace edsc {
/// Constructs a new AffineForOp and captures the associated induction /// Constructs a new AffineForOp and captures the associated induction
▲ Show 20 Lines • Show All 172 Lines • ▼ Show 20 Lines
ValueHandle TemplatedIndexedValue<Load, Store>::operator>=(ValueHandle e) { ValueHandle TemplatedIndexedValue<Load, Store>::operator>=(ValueHandle e) {
using op::operator>=; using op::operator>=;
return static_cast<ValueHandle>(*this) >= e; return static_cast<ValueHandle>(*this) >= e;
} }
} // namespace edsc } // namespace edsc
} // namespace mlir } // namespace mlir
#endif // MLIR_DIALECT_AFFINEOPS_EDSC_BUILDERS_H_ #endif // MLIR_DIALECT_AFFINE_EDSC_BUILDERS_H_

mlir/include/mlir/Dialect/Affine/EDSC/Intrinsics.h

  • This file was moved from mlir/include/mlir/Dialect/AffineOps/EDSC/Intrinsics.h.
//===- Intrinsics.h - MLIR EDSC Intrinsics for AffineOps --------*- C++ -*-===// //===- Intrinsics.h - MLIR EDSC Intrinsics for AffineOps --------*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef MLIR_DIALECT_AFFINEOPS_EDSC_INTRINSICS_H_ #ifndef MLIR_DIALECT_AFFINE_EDSC_INTRINSICS_H_
rriddleUnsubmitted
Done
ReplyInline Actions

Can you update each of these header guard comments now?

rriddle: Can you update each of these header guard comments now?
#define MLIR_DIALECT_AFFINEOPS_EDSC_INTRINSICS_H_ #define MLIR_DIALECT_AFFINE_EDSC_INTRINSICS_H_
#include "mlir/Dialect/AffineOps/EDSC/Builders.h" #include "mlir/Dialect/Affine/EDSC/Builders.h"
#include "mlir/EDSC/Intrinsics.h" #include "mlir/EDSC/Intrinsics.h"
namespace mlir { namespace mlir {
namespace edsc { namespace edsc {
namespace intrinsics { namespace intrinsics {
using affine_apply = ValueBuilder<AffineApplyOp>; using affine_apply = ValueBuilder<AffineApplyOp>;
using affine_if = OperationBuilder<AffineIfOp>; using affine_if = OperationBuilder<AffineIfOp>;
Show All 13 Lines

mlir/include/mlir/Dialect/Affine/IR/AffineOps.h

  • This file was moved from mlir/include/mlir/Dialect/AffineOps/AffineOps.h.
//===- AffineOps.h - MLIR Affine Operations -------------------------------===// //===- AffineOps.h - MLIR Affine Operations -------------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file defines convenience types for working with Affine operations // This file defines convenience types for working with Affine operations
// in the MLIR operation set. // in the MLIR operation set.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef MLIR_DIALECT_AFFINEOPS_AFFINEOPS_H #ifndef MLIR_DIALECT_AFFINE_IR_AFFINEOPS_H
#define MLIR_DIALECT_AFFINEOPS_AFFINEOPS_H #define MLIR_DIALECT_AFFINE_IR_AFFINEOPS_H
#include "mlir/IR/AffineMap.h" #include "mlir/IR/AffineMap.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/Dialect.h" #include "mlir/IR/Dialect.h"
#include "mlir/IR/OpDefinition.h" #include "mlir/IR/OpDefinition.h"
#include "mlir/IR/StandardTypes.h" #include "mlir/IR/StandardTypes.h"
#include "mlir/Interfaces/LoopLikeInterface.h" #include "mlir/Interfaces/LoopLikeInterface.h"
#include "mlir/Interfaces/SideEffects.h" #include "mlir/Interfaces/SideEffects.h"
▲ Show 20 Lines • Show All 464 Lines • ▼ Show 20 Lines
/// `operands`, iteratively until no more of `operands` are the result of an /// `operands`, iteratively until no more of `operands` are the result of an
/// AffineApplyOp. When this function returns, `map` becomes the composed affine /// AffineApplyOp. When this function returns, `map` becomes the composed affine
/// map, and each Value in `operands` is guaranteed to be either a loop IV or a /// map, and each Value in `operands` is guaranteed to be either a loop IV or a
/// terminal symbol, i.e., a symbol defined at the top level or a block/function /// terminal symbol, i.e., a symbol defined at the top level or a block/function
/// argument. /// argument.
void fullyComposeAffineMapAndOperands(AffineMap *map, void fullyComposeAffineMapAndOperands(AffineMap *map,
SmallVectorImpl<Value> *operands); SmallVectorImpl<Value> *operands);
#include "mlir/Dialect/AffineOps/AffineOpsDialect.h.inc" #include "mlir/Dialect/Affine/IR/AffineOpsDialect.h.inc"
#define GET_OP_CLASSES #define GET_OP_CLASSES
#include "mlir/Dialect/AffineOps/AffineOps.h.inc" #include "mlir/Dialect/Affine/IR/AffineOps.h.inc"
/// Returns if the provided value is the induction variable of a AffineForOp. /// Returns if the provided value is the induction variable of a AffineForOp.
bool isForInductionVar(Value val); bool isForInductionVar(Value val);
/// Returns the loop parent of an induction variable. If the provided value is /// Returns the loop parent of an induction variable. If the provided value is
/// not an induction variable, then return nullptr. /// not an induction variable, then return nullptr.
AffineForOp getForInductionVarOwner(Value val); AffineForOp getForInductionVarOwner(Value val);
▲ Show 20 Lines • Show All 117 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Affine/IR/AffineOps.td

  • This file was moved from mlir/include/mlir/Dialect/AffineOps/AffineOps.td.
//===- AffineOps.td - Affine operation definitions ---------*- tablegen -*-===// //===- AffineOps.td - Affine operation definitions ---------*- tablegen -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// Defines MLIR affine operations. // Defines MLIR affine operations.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef AFFINE_OPS #ifndef AFFINE_OPS
#define AFFINE_OPS #define AFFINE_OPS
include "mlir/Dialect/AffineOps/AffineOpsBase.td" include "mlir/Dialect/Affine/IR/AffineOpsBase.td"
include "mlir/Interfaces/LoopLikeInterface.td" include "mlir/Interfaces/LoopLikeInterface.td"
include "mlir/Interfaces/SideEffects.td" include "mlir/Interfaces/SideEffects.td"
def AffineOps_Dialect : Dialect { def Affine_Dialect : Dialect {
let name = "affine"; let name = "affine";
let cppNamespace = ""; let cppNamespace = "";
let hasConstantMaterializer = 1; let hasConstantMaterializer = 1;
} }
// Base class for Affine dialect ops. // Base class for Affine dialect ops.
class Affine_Op<string mnemonic, list<OpTrait> traits = []> : class Affine_Op<string mnemonic, list<OpTrait> traits = []> :
Op<AffineOps_Dialect, mnemonic, traits> { Op<Affine_Dialect, mnemonic, traits> {
// For every affine op, there needs to be a: // For every affine op, there needs to be a:
// * void print(OpAsmPrinter &p, ${C++ class of Op} op) // * void print(OpAsmPrinter &p, ${C++ class of Op} op)
// * LogicalResult verify(${C++ class of Op} op) // * LogicalResult verify(${C++ class of Op} op)
// * ParseResult parse${C++ class of Op}(OpAsmParser &parser, // * ParseResult parse${C++ class of Op}(OpAsmParser &parser,
// OperationState &result) // OperationState &result)
// functions. // functions.
let printer = [{ return ::print(p, *this); }]; let printer = [{ return ::print(p, *this); }];
let verifier = [{ return ::verify(*this); }]; let verifier = [{ return ::verify(*this); }];
▲ Show 20 Lines • Show All 249 Lines • ▼ Show 20 Lines OpBuilder getElseBodyBuilder() {
return OpBuilder(&body, std::prev(body.end())); return OpBuilder(&body, std::prev(body.end()));
} }
}]; }];
let hasFolder = 1; let hasFolder = 1;
} }
class AffineMinMaxOpBase<string mnemonic, list<OpTrait> traits = []> : class AffineMinMaxOpBase<string mnemonic, list<OpTrait> traits = []> :
Op<AffineOps_Dialect, mnemonic, traits> { Op<Affine_Dialect, mnemonic, traits> {
let arguments = (ins AffineMapAttr:$map, Variadic<Index>:$operands); let arguments = (ins AffineMapAttr:$map, Variadic<Index>:$operands);
let results = (outs Index); let results = (outs Index);
let builders = [ let builders = [
OpBuilder<"Builder *builder, OperationState &result, AffineMap affineMap, " OpBuilder<"Builder *builder, OperationState &result, AffineMap affineMap, "
"ValueRange mapOperands", "ValueRange mapOperands",
[{ [{
build(builder, result, builder->getIndexType(), affineMap, mapOperands); build(builder, result, builder->getIndexType(), affineMap, mapOperands);
▲ Show 20 Lines • Show All 208 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Affine/IR/AffineOpsBase.td

  • This file was moved from mlir/include/mlir/Dialect/AffineOps/AffineOpsBase.td.
The contents of this file were not changed.

mlir/include/mlir/Dialect/Affine/IR/AffineValueMap.h

  • This file was moved from mlir/include/mlir/Dialect/AffineOps/AffineValueMap.h.
//===- AffineValueMap.h - MLIR Affine Value Map Class -----------*- C++ -*-===// //===- AffineValueMap.h - MLIR Affine Value Map Class -----------*- C++ -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// An AffineValueMap is an affine map plus its ML value operands and results for // An AffineValueMap is an affine map plus its ML value operands and results for
// analysis purposes. // analysis purposes.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef MLIR_DIALECT_AFFINEOPS_AFFINEVALUEMAP_H #ifndef MLIR_DIALECT_AFFINE_IR_AFFINEVALUEMAP_H
#define MLIR_DIALECT_AFFINEOPS_AFFINEVALUEMAP_H #define MLIR_DIALECT_AFFINE_IR_AFFINEVALUEMAP_H
#include "mlir/IR/AffineMap.h" #include "mlir/IR/AffineMap.h"
#include "mlir/IR/OperationSupport.h" #include "mlir/IR/OperationSupport.h"
#include "mlir/IR/Value.h" #include "mlir/IR/Value.h"
namespace mlir { namespace mlir {
/// An AffineValueMap is an affine map plus its ML value operands and /// An AffineValueMap is an affine map plus its ML value operands and
▲ Show 20 Lines • Show All 59 Lines • ▼ Show 20 Linesprivate:
/// The SSA operands binding to the dim's and symbols of 'map'. /// The SSA operands binding to the dim's and symbols of 'map'.
SmallVector<Value, 4> operands; SmallVector<Value, 4> operands;
/// The SSA results binding to the results of 'map'. /// The SSA results binding to the results of 'map'.
SmallVector<Value, 4> results; SmallVector<Value, 4> results;
}; };
} // namespace mlir } // namespace mlir
#endif // MLIR_DIALECT_AFFINEOPS_AFFINEVALUEMAP_H #endif // MLIR_DIALECT_AFFINE_IR_AFFINEVALUEMAP_H

mlir/include/mlir/Dialect/Affine/IR/CMakeLists.txt

  • This file was moved from mlir/include/mlir/Dialect/AffineOps/CMakeLists.txt.
The contents of this file were not changed.

mlir/include/mlir/Dialect/Affine/Passes.h

  • This file was added.
//===- Passes.h - Pass Entrypoints ------------------------------*- C++ -*-===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
//
// This header file defines a set of transforms specific for the AffineOps
// dialect.
rriddleUnsubmitted
Done
ReplyInline Actions

AffineOps -> Affine

rriddle: AffineOps -> Affine
//
//===----------------------------------------------------------------------===//
#ifndef MLIR_DIALECT_AFFINE_TRANSFORMS_PASSES_H
#define MLIR_DIALECT_AFFINE_TRANSFORMS_PASSES_H
#include "mlir/Support/LLVM.h"
#include <functional>
#include <limits>
namespace mlir {
class AffineForOp;
class FuncOp;
class ModuleOp;
class Pass;
template <typename T> class OpPassBase;
/// Creates a simplification pass for affine structures (maps and sets). In
/// addition, this pass also normalizes memrefs to have the trivial (identity)
/// layout map.
std::unique_ptr<OpPassBase<FuncOp>> createSimplifyAffineStructuresPass();
/// Creates a loop invariant code motion pass that hoists loop invariant
/// instructions out of affine loop.
std::unique_ptr<OpPassBase<FuncOp>> createAffineLoopInvariantCodeMotionPass();
/// Performs packing (or explicit copying) of accessed memref regions into
/// buffers in the specified faster memory space through either pointwise copies
/// or DMA operations.
std::unique_ptr<OpPassBase<FuncOp>> createAffineDataCopyGenerationPass(
unsigned slowMemorySpace, unsigned fastMemorySpace,
unsigned tagMemorySpace = 0, int minDmaTransferSize = 1024,
uint64_t fastMemCapacityBytes = std::numeric_limits<uint64_t>::max());
} // end namespace mlir
#endif // MLIR_DIALECT_AFFINE_RANSFORMS_PASSES_H

mlir/include/mlir/Dialect/AffineOps/AffineOps.h

  • This file was moved to mlir/include/mlir/Dialect/Affine/IR/AffineOps.h.

mlir/include/mlir/Dialect/AffineOps/AffineOps.td

  • This file was moved to mlir/include/mlir/Dialect/Affine/IR/AffineOps.td.

mlir/include/mlir/Dialect/AffineOps/AffineOpsBase.td

  • This file was moved to mlir/include/mlir/Dialect/Affine/IR/AffineOpsBase.td.

mlir/include/mlir/Dialect/AffineOps/AffineValueMap.h

  • This file was moved to mlir/include/mlir/Dialect/Affine/IR/AffineValueMap.h.

mlir/include/mlir/Dialect/AffineOps/CMakeLists.txt

  • This file was moved to mlir/include/mlir/Dialect/Affine/IR/CMakeLists.txt.

mlir/include/mlir/Dialect/AffineOps/EDSC/Builders.h

  • This file was moved to mlir/include/mlir/Dialect/Affine/EDSC/Builders.h.

mlir/include/mlir/Dialect/AffineOps/EDSC/Intrinsics.h

  • This file was moved to mlir/include/mlir/Dialect/Affine/EDSC/Intrinsics.h.

mlir/include/mlir/Dialect/CMakeLists.txt

add_subdirectory(AffineOps) add_subdirectory(Affine)
add_subdirectory(AVX512) add_subdirectory(AVX512)
add_subdirectory(FxpMathOps) add_subdirectory(FxpMathOps)
add_subdirectory(GPU) add_subdirectory(GPU)
add_subdirectory(Linalg) add_subdirectory(Linalg)
add_subdirectory(LLVMIR) add_subdirectory(LLVMIR)
add_subdirectory(LoopOps) add_subdirectory(LoopOps)
add_subdirectory(OpenMP) add_subdirectory(OpenMP)
add_subdirectory(Quant) add_subdirectory(Quant)
add_subdirectory(Shape) add_subdirectory(Shape)
add_subdirectory(SPIRV) add_subdirectory(SPIRV)
add_subdirectory(StandardOps) add_subdirectory(StandardOps)
add_subdirectory(Vector) add_subdirectory(Vector)

mlir/include/mlir/Dialect/Linalg/IR/LinalgOps.td

//===- LinalgOps.td - Linalg dialect ops -------------------*- tablegen -*-===// //===- LinalgOps.td - Linalg dialect ops -------------------*- tablegen -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This is the operation definition file for linear algebra operations. // This is the operation definition file for linear algebra operations.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef LINALG_OPS #ifndef LINALG_OPS
#define LINALG_OPS #define LINALG_OPS
include "mlir/Dialect/AffineOps/AffineOpsBase.td" include "mlir/Dialect/Affine/IR/AffineOpsBase.td"
include "mlir/Dialect/Linalg/IR/LinalgBase.td" include "mlir/Dialect/Linalg/IR/LinalgBase.td"
include "mlir/Interfaces/SideEffects.td" include "mlir/Interfaces/SideEffects.td"
// Base class for Linalg dialect ops that do not correspond to library calls. // Base class for Linalg dialect ops that do not correspond to library calls.
class Linalg_Op<string mnemonic, list<OpTrait> traits = []> : class Linalg_Op<string mnemonic, list<OpTrait> traits = []> :
Op<Linalg_Dialect, mnemonic, traits> { Op<Linalg_Dialect, mnemonic, traits> {
// For every linalg op, there needs to be a: // For every linalg op, there needs to be a:
// * void print(OpAsmPrinter &p, ${C++ class of Op} op) // * void print(OpAsmPrinter &p, ${C++ class of Op} op)
▲ Show 20 Lines • Show All 229 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Linalg/IR/LinalgStructuredOps.td

//===- LinalgStructuredOps.td - Linalg dialect library ops -*- tablegen -*-===// //===- LinalgStructuredOps.td - Linalg dialect library ops -*- tablegen -*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This is the operation definition file for structured operations on buffers // This is the operation definition file for structured operations on buffers
// that correspond to underlying library calls (e.g. BLAS). // that correspond to underlying library calls (e.g. BLAS).
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef LINALG_STRUCTURED_OPS #ifndef LINALG_STRUCTURED_OPS
#define LINALG_STRUCTURED_OPS #define LINALG_STRUCTURED_OPS
include "mlir/Dialect/AffineOps/AffineOpsBase.td" include "mlir/Dialect/Affine/IR/AffineOpsBase.td"
include "mlir/Dialect/Linalg/IR/LinalgBase.td" include "mlir/Dialect/Linalg/IR/LinalgBase.td"
include "mlir/Dialect/Linalg/IR/LinalgStructuredOpsInterface.td" include "mlir/Dialect/Linalg/IR/LinalgStructuredOpsInterface.td"
// The Linalg `NInputs` trait provides the API for ops that are known // The Linalg `NInputs` trait provides the API for ops that are known
// to have a specified number of inputs, all passed as operands. // to have a specified number of inputs, all passed as operands.
// See Linalg/LinalgTraits.h for implementation details an usage. // See Linalg/LinalgTraits.h for implementation details an usage.
class NInputs<int args_in> : class NInputs<int args_in> :
NativeOpTrait<"linalg::NInputs<" # !cast<string>(args_in) # ">::Impl"> {} NativeOpTrait<"linalg::NInputs<" # !cast<string>(args_in) # ">::Impl"> {}
▲ Show 20 Lines • Show All 669 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Linalg/Transforms/LinalgTransformPatterns.td

Show All 9 Lines
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef LINALG_TRANSFORMS #ifndef LINALG_TRANSFORMS
#define LINALG_TRANSFORMS #define LINALG_TRANSFORMS
include "mlir/Dialect/Linalg/IR/LinalgOps.td" include "mlir/Dialect/Linalg/IR/LinalgOps.td"
include "mlir/Dialect/Linalg/IR/LinalgStructuredOps.td" include "mlir/Dialect/Linalg/IR/LinalgStructuredOps.td"
include "mlir/Dialect/AffineOps/AffineOps.td" include "mlir/Dialect/Affine/IR/AffineOps.td"
def HasNoLinalgTransformMarker : CPred<[{ def HasNoLinalgTransformMarker : CPred<[{
!op.getAttrOfType<StringAttr>(LinalgTransforms::kLinalgTransformMarker) !op.getAttrOfType<StringAttr>(LinalgTransforms::kLinalgTransformMarker)
}]>; }]>;
class HasLinalgTransformMarker<string str> : CPred<[{ class HasLinalgTransformMarker<string str> : CPred<[{
op.getAttrOfType<StringAttr>( op.getAttrOfType<StringAttr>(
LinalgTransforms::kLinalgTransformMarker) && LinalgTransforms::kLinalgTransformMarker) &&
▲ Show 20 Lines • Show All 91 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/Vector/VectorOps.td

  • This file was copied to mlir/include/mlir/Dialect/VectorOps/VectorOps.td.
//===- VectorOps.td - Vector op definitions ---------------*- tablegen -*-====// //===- VectorOps.td - Vector op definitions ---------------*- tablegen -*-====//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// Defines MLIR vector operations. // Defines MLIR vector operations.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef VECTOR_OPS #ifndef VECTOR_OPS
#define VECTOR_OPS #define VECTOR_OPS
include "mlir/Dialect/AffineOps/AffineOpsBase.td" include "mlir/Dialect/Affine/IR/AffineOpsBase.td"
include "mlir/Interfaces/SideEffects.td" include "mlir/Interfaces/SideEffects.td"
def Vector_Dialect : Dialect { def Vector_Dialect : Dialect {
let name = "vector"; let name = "vector";
let cppNamespace = "vector"; let cppNamespace = "vector";
let hasConstantMaterializer = 1; let hasConstantMaterializer = 1;
} }
▲ Show 20 Lines • Show All 1,377 Lines • Show Last 20 Lines

mlir/include/mlir/Dialect/VectorOps/VectorOps.td

  • This file was copied from mlir/include/mlir/Dialect/Vector/VectorOps.td.
//===- VectorOps.td - Vector op definitions ---------------*- tablegen -*-====// //===- VectorOps.td - Vector op definitions ---------------*- tablegen -*-====//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// Defines MLIR vector operations. // Defines MLIR vector operations.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef VECTOR_OPS #ifndef VECTOR_OPS
#define VECTOR_OPS #define VECTOR_OPS
include "mlir/Dialect/AffineOps/AffineOpsBase.td" include "mlir/Dialect/Affine/IR/AffineOpsBase.td"
include "mlir/Interfaces/SideEffects.td" include "mlir/Interfaces/SideEffects.td"
def Vector_Dialect : Dialect {
def VectorOps_Dialect : Dialect {
let name = "vector"; let name = "vector";
let cppNamespace = "vector"; let cppNamespace = "vector";
let hasConstantMaterializer = 1; let hasConstantMaterializer = 1;
} }
// Base class for Vector dialect ops. // Base class for Vector dialect ops.
class Vector_Op<string mnemonic, list<OpTrait> traits = []> : class Vector_Op<string mnemonic, list<OpTrait> traits = []> :
Op<Vector_Dialect, mnemonic, traits> { Op<VectorOps_Dialect, mnemonic, traits> {
// For every vector op, there needs to be a: // For every vector op, there needs to be a:
// * void print(OpAsmPrinter &p, ${C++ class of Op} op) // * void print(OpAsmPrinter &p, ${C++ class of Op} op)
// * LogicalResult verify(${C++ class of Op} op) // * LogicalResult verify(${C++ class of Op} op)
// * ParseResult parse${C++ class of Op}(OpAsmParser &parser, // * ParseResult parse${C++ class of Op}(OpAsmParser &parser,
// OperationState &result) // OperationState &result)
// functions. // functions.
let printer = [{ return ::print(p, *this); }]; let printer = [{ return ::print(p, *this); }];
let verifier = [{ return ::verify(*this); }]; let verifier = [{ return ::verify(*this); }];
▲ Show 20 Lines • Show All 392 Lines • ▼ Show 20 Linesdef Vector_ExtractSlicesOp :
}]; }];
let assemblyFormat = [{ let assemblyFormat = [{
$vector `,` $sizes `,` $strides attr-dict `:` type($vector) `into` $vector `,` $sizes `,` $strides attr-dict `:` type($vector) `into`
type(results) type(results)
}]; }];
} }
def Vector_FMAOp : def Vector_FMAOp :
Op<Vector_Dialect, "fma", [NoSideEffect, Op<VectorOps_Dialect, "fma", [NoSideEffect,
AllTypesMatch<["lhs", "rhs", "acc", "result"]>]>, AllTypesMatch<["lhs", "rhs", "acc", "result"]>]>,
Arguments<(ins AnyVector:$lhs, AnyVector:$rhs, AnyVector:$acc)>, Arguments<(ins AnyVector:$lhs, AnyVector:$rhs, AnyVector:$acc)>,
Results<(outs AnyVector:$result)> { Results<(outs AnyVector:$result)> {
let summary = "vector fused multiply-add"; let summary = "vector fused multiply-add";
let description = [{ let description = [{
Multiply-add expressions operate on n-D vectors and compute a fused Multiply-add expressions operate on n-D vectors and compute a fused
pointwise multiply-and-accumulate: `$result = `$lhs * $rhs + $acc`. pointwise multiply-and-accumulate: `$result = `$lhs * $rhs + $acc`.
All operands and result have the same vector type. The semantics All operands and result have the same vector type. The semantics
▲ Show 20 Lines • Show All 957 Lines • Show Last 20 Lines

mlir/include/mlir/InitAllDialects.h

Show All 9 Lines
// passes to the system. // passes to the system.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#ifndef MLIR_INITALLDIALECTS_H_ #ifndef MLIR_INITALLDIALECTS_H_
#define MLIR_INITALLDIALECTS_H_ #define MLIR_INITALLDIALECTS_H_
#include "mlir/Dialect/AVX512/AVX512Dialect.h" #include "mlir/Dialect/AVX512/AVX512Dialect.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/FxpMathOps/FxpMathOps.h" #include "mlir/Dialect/FxpMathOps/FxpMathOps.h"
#include "mlir/Dialect/GPU/GPUDialect.h" #include "mlir/Dialect/GPU/GPUDialect.h"
#include "mlir/Dialect/LLVMIR/LLVMAVX512Dialect.h" #include "mlir/Dialect/LLVMIR/LLVMAVX512Dialect.h"
#include "mlir/Dialect/LLVMIR/LLVMDialect.h" #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
#include "mlir/Dialect/LLVMIR/NVVMDialect.h" #include "mlir/Dialect/LLVMIR/NVVMDialect.h"
#include "mlir/Dialect/LLVMIR/ROCDLDialect.h" #include "mlir/Dialect/LLVMIR/ROCDLDialect.h"
#include "mlir/Dialect/Linalg/IR/LinalgOps.h" #include "mlir/Dialect/Linalg/IR/LinalgOps.h"
#include "mlir/Dialect/LoopOps/LoopOps.h" #include "mlir/Dialect/LoopOps/LoopOps.h"
#include "mlir/Dialect/OpenMP/OpenMPDialect.h" #include "mlir/Dialect/OpenMP/OpenMPDialect.h"
#include "mlir/Dialect/Quant/QuantOps.h" #include "mlir/Dialect/Quant/QuantOps.h"
#include "mlir/Dialect/SDBM/SDBMDialect.h" #include "mlir/Dialect/SDBM/SDBMDialect.h"
#include "mlir/Dialect/SPIRV/SPIRVDialect.h" #include "mlir/Dialect/SPIRV/SPIRVDialect.h"
#include "mlir/Dialect/Shape/IR/Shape.h" #include "mlir/Dialect/Shape/IR/Shape.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Vector/VectorOps.h" #include "mlir/Dialect/Vector/VectorOps.h"
#include "mlir/IR/Dialect.h" #include "mlir/IR/Dialect.h"
namespace mlir { namespace mlir {
// This function should be called before creating any MLIRContext if one expect // This function should be called before creating any MLIRContext if one expect
// all the possible dialects to be made available to the context automatically. // all the possible dialects to be made available to the context automatically.
inline void registerAllDialects() { inline void registerAllDialects() {
static bool init_once = []() { static bool init_once = []() {
registerDialect<AffineOpsDialect>(); registerDialect<AffineDialect>();
registerDialect<avx512::AVX512Dialect>(); registerDialect<avx512::AVX512Dialect>();
registerDialect<fxpmath::FxpMathOpsDialect>(); registerDialect<fxpmath::FxpMathOpsDialect>();
registerDialect<gpu::GPUDialect>(); registerDialect<gpu::GPUDialect>();
registerDialect<LLVM::LLVMAVX512Dialect>(); registerDialect<LLVM::LLVMAVX512Dialect>();
registerDialect<LLVM::LLVMDialect>(); registerDialect<LLVM::LLVMDialect>();
registerDialect<linalg::LinalgDialect>(); registerDialect<linalg::LinalgDialect>();
registerDialect<loop::LoopOpsDialect>(); registerDialect<loop::LoopOpsDialect>();
registerDialect<omp::OpenMPDialect>(); registerDialect<omp::OpenMPDialect>();
Show All 15 Lines

mlir/include/mlir/InitAllPasses.h

Show All 19 Lines
#include "mlir/Conversion/GPUToNVVM/GPUToNVVMPass.h" #include "mlir/Conversion/GPUToNVVM/GPUToNVVMPass.h"
#include "mlir/Conversion/GPUToROCDL/GPUToROCDLPass.h" #include "mlir/Conversion/GPUToROCDL/GPUToROCDLPass.h"
#include "mlir/Conversion/GPUToSPIRV/ConvertGPUToSPIRVPass.h" #include "mlir/Conversion/GPUToSPIRV/ConvertGPUToSPIRVPass.h"
#include "mlir/Conversion/GPUToVulkan/ConvertGPUToVulkanPass.h" #include "mlir/Conversion/GPUToVulkan/ConvertGPUToVulkanPass.h"
#include "mlir/Conversion/LinalgToLLVM/LinalgToLLVM.h" #include "mlir/Conversion/LinalgToLLVM/LinalgToLLVM.h"
#include "mlir/Conversion/LinalgToSPIRV/LinalgToSPIRVPass.h" #include "mlir/Conversion/LinalgToSPIRV/LinalgToSPIRVPass.h"
#include "mlir/Conversion/LoopsToGPU/LoopsToGPUPass.h" #include "mlir/Conversion/LoopsToGPU/LoopsToGPUPass.h"
#include "mlir/Conversion/StandardToSPIRV/ConvertStandardToSPIRVPass.h" #include "mlir/Conversion/StandardToSPIRV/ConvertStandardToSPIRVPass.h"
#include "mlir/Dialect/Affine/Passes.h"
#include "mlir/Dialect/FxpMathOps/Passes.h"
rriddleUnsubmitted
Done
ReplyInline Actions

Duplicate?

rriddle: Duplicate?
rsudermanAuthorUnsubmitted
Done
ReplyInline Actions

I don't see a duplicate? AffineOps/Passes.h was newly added in this CL.

rsuderman: I don't see a duplicate? AffineOps/Passes.h was newly added in this CL.
rriddleUnsubmitted
Done
ReplyInline Actions

It was previously showing + #include "mlir/Dialect/FxpMathOps/Passes.h"

rriddle: It was previously showing `+ #include "mlir/Dialect/FxpMathOps/Passes.h"`
#include "mlir/Dialect/FxpMathOps/Passes.h" #include "mlir/Dialect/FxpMathOps/Passes.h"
#include "mlir/Dialect/GPU/Passes.h" #include "mlir/Dialect/GPU/Passes.h"
#include "mlir/Dialect/LLVMIR/Transforms/LegalizeForExport.h" #include "mlir/Dialect/LLVMIR/Transforms/LegalizeForExport.h"
#include "mlir/Dialect/Linalg/Passes.h" #include "mlir/Dialect/Linalg/Passes.h"
#include "mlir/Dialect/LoopOps/Passes.h" #include "mlir/Dialect/LoopOps/Passes.h"
#include "mlir/Dialect/Quant/Passes.h" #include "mlir/Dialect/Quant/Passes.h"
#include "mlir/Dialect/SPIRV/Passes.h" #include "mlir/Dialect/SPIRV/Passes.h"
#include "mlir/Quantizer/Transforms/Passes.h" #include "mlir/Quantizer/Transforms/Passes.h"
▲ Show 20 Lines • Show All 111 Lines • Show Last 20 Lines

mlir/include/mlir/Transforms/Passes.h

Show First 20 Lines • Show All 47 Lines • ▼ Show 20 Linesstd::unique_ptr<OpPassBase<FuncOp>> createLoopUnrollPass(
const std::function<unsigned(AffineForOp)> &getUnrollFactor = nullptr); const std::function<unsigned(AffineForOp)> &getUnrollFactor = nullptr);
/// Creates a loop unroll jam pass to unroll jam by the specified factor. A /// Creates a loop unroll jam pass to unroll jam by the specified factor. A
/// factor of -1 lets the pass use the default factor or the one on the command /// factor of -1 lets the pass use the default factor or the one on the command
/// line if provided. /// line if provided.
std::unique_ptr<OpPassBase<FuncOp>> std::unique_ptr<OpPassBase<FuncOp>>
createLoopUnrollAndJamPass(int unrollJamFactor = -1); createLoopUnrollAndJamPass(int unrollJamFactor = -1);
/// Creates a simplification pass for affine structures (maps and sets). In
/// addition, this pass also normalizes memrefs to have the trivial (identity)
/// layout map.
std::unique_ptr<OpPassBase<FuncOp>> createSimplifyAffineStructuresPass();
/// Creates a loop fusion pass which fuses loops. Buffers of size less than or /// Creates a loop fusion pass which fuses loops. Buffers of size less than or
/// equal to `localBufSizeThreshold` are promoted to memory space /// equal to `localBufSizeThreshold` are promoted to memory space
/// `fastMemorySpace'. /// `fastMemorySpace'.
std::unique_ptr<OpPassBase<FuncOp>> std::unique_ptr<OpPassBase<FuncOp>>
createLoopFusionPass(unsigned fastMemorySpace = 0, createLoopFusionPass(unsigned fastMemorySpace = 0,
uint64_t localBufSizeThreshold = 0, uint64_t localBufSizeThreshold = 0,
bool maximalFusion = false); bool maximalFusion = false);
Show All 17 Lines
/// Creates a pass to perform tiling on loop nests. /// Creates a pass to perform tiling on loop nests.
std::unique_ptr<OpPassBase<FuncOp>> std::unique_ptr<OpPassBase<FuncOp>>
createLoopTilingPass(uint64_t cacheSizeBytes); createLoopTilingPass(uint64_t cacheSizeBytes);
/// Creates a pass that transforms perfectly nested loops with independent /// Creates a pass that transforms perfectly nested loops with independent
/// bounds into a single loop. /// bounds into a single loop.
std::unique_ptr<OpPassBase<FuncOp>> createLoopCoalescingPass(); std::unique_ptr<OpPassBase<FuncOp>> createLoopCoalescingPass();
/// Performs packing (or explicit copying) of accessed memref regions into
/// buffers in the specified faster memory space through either pointwise copies
/// or DMA operations.
std::unique_ptr<OpPassBase<FuncOp>> createAffineDataCopyGenerationPass(
unsigned slowMemorySpace, unsigned fastMemorySpace,
unsigned tagMemorySpace = 0, int minDmaTransferSize = 1024,
uint64_t fastMemCapacityBytes = std::numeric_limits<uint64_t>::max());
/// Creates a pass to perform optimizations relying on memref dataflow such as /// Creates a pass to perform optimizations relying on memref dataflow such as
/// store to load forwarding, elimination of dead stores, and dead allocs. /// store to load forwarding, elimination of dead stores, and dead allocs.
std::unique_ptr<OpPassBase<FuncOp>> createMemRefDataFlowOptPass(); std::unique_ptr<OpPassBase<FuncOp>> createMemRefDataFlowOptPass();
/// Creates a pass to strip debug information from a function. /// Creates a pass to strip debug information from a function.
std::unique_ptr<Pass> createStripDebugInfoPass(); std::unique_ptr<Pass> createStripDebugInfoPass();
/// Creates a pass which prints the list of ops and the number of occurences in /// Creates a pass which prints the list of ops and the number of occurences in
Show All 13 Lines

mlir/lib/Analysis/AffineAnalysis.cpp

//===- AffineAnalysis.cpp - Affine structures analysis routines -----------===// //===- AffineAnalysis.cpp - Affine structures analysis routines -----------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements miscellaneous analysis routines for affine structures // This file implements miscellaneous analysis routines for affine structures
// (expressions, maps, sets), and other utilities relying on such analysis. // (expressions, maps, sets), and other utilities relying on such analysis.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/AffineAnalysis.h" #include "mlir/Analysis/AffineAnalysis.h"
#include "mlir/Analysis/AffineStructures.h" #include "mlir/Analysis/AffineStructures.h"
#include "mlir/Analysis/Utils.h" #include "mlir/Analysis/Utils.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/AffineOps/AffineValueMap.h" #include "mlir/Dialect/Affine/IR/AffineValueMap.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/AffineExprVisitor.h" #include "mlir/IR/AffineExprVisitor.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/Function.h" #include "mlir/IR/Function.h"
#include "mlir/IR/IntegerSet.h" #include "mlir/IR/IntegerSet.h"
#include "mlir/IR/Operation.h" #include "mlir/IR/Operation.h"
#include "mlir/Support/MathExtras.h" #include "mlir/Support/MathExtras.h"
#include "mlir/Support/STLExtras.h" #include "mlir/Support/STLExtras.h"
▲ Show 20 Lines • Show All 862 Lines • Show Last 20 Lines

mlir/lib/Analysis/AffineStructures.cpp

//===- AffineStructures.cpp - MLIR Affine Structures Class-----------------===// //===- AffineStructures.cpp - MLIR Affine Structures Class-----------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// Structures for affine/polyhedral analysis of affine dialect ops. // Structures for affine/polyhedral analysis of affine dialect ops.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/AffineStructures.h" #include "mlir/Analysis/AffineStructures.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/AffineOps/AffineValueMap.h" #include "mlir/Dialect/Affine/IR/AffineValueMap.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/AffineExprVisitor.h" #include "mlir/IR/AffineExprVisitor.h"
#include "mlir/IR/IntegerSet.h" #include "mlir/IR/IntegerSet.h"
#include "mlir/Support/LLVM.h" #include "mlir/Support/LLVM.h"
#include "mlir/Support/MathExtras.h" #include "mlir/Support/MathExtras.h"
#include "llvm/ADT/SmallPtrSet.h" #include "llvm/ADT/SmallPtrSet.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#include "llvm/Support/raw_ostream.h" #include "llvm/Support/raw_ostream.h"
▲ Show 20 Lines • Show All 2,675 Lines • Show Last 20 Lines

mlir/lib/Analysis/CMakeLists.txt

Show All 18 Linesadd_mlir_library(MLIRAnalysis
Verifier.cpp Verifier.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Analysis ${MLIR_MAIN_INCLUDE_DIR}/mlir/Analysis
) )
target_link_libraries(MLIRAnalysis target_link_libraries(MLIRAnalysis
PUBLIC PUBLIC
MLIRAffineOps MLIRAffine
MLIRCallInterfaces MLIRCallInterfaces
MLIRControlFlowInterfaces MLIRControlFlowInterfaces
MLIRInferTypeOpInterface MLIRInferTypeOpInterface
MLIRLoopOps MLIRLoopOps
) )
add_mlir_library(MLIRLoopAnalysis add_mlir_library(MLIRLoopAnalysis
AffineAnalysis.cpp AffineAnalysis.cpp
AffineStructures.cpp AffineStructures.cpp
LoopAnalysis.cpp LoopAnalysis.cpp
NestedMatcher.cpp NestedMatcher.cpp
Utils.cpp Utils.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Analysis ${MLIR_MAIN_INCLUDE_DIR}/mlir/Analysis
) )
target_link_libraries(MLIRLoopAnalysis target_link_libraries(MLIRLoopAnalysis
PUBLIC PUBLIC
MLIRAffineOps MLIRAffine
MLIRCallInterfaces MLIRCallInterfaces
MLIRControlFlowInterfaces MLIRControlFlowInterfaces
MLIRInferTypeOpInterface MLIRInferTypeOpInterface
MLIRLoopOps) MLIRLoopOps)

mlir/lib/Analysis/LoopAnalysis.cpp

Show All 9 Lines
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/LoopAnalysis.h" #include "mlir/Analysis/LoopAnalysis.h"
#include "mlir/Analysis/AffineAnalysis.h" #include "mlir/Analysis/AffineAnalysis.h"
#include "mlir/Analysis/AffineStructures.h" #include "mlir/Analysis/AffineStructures.h"
#include "mlir/Analysis/NestedMatcher.h" #include "mlir/Analysis/NestedMatcher.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/AffineOps/AffineValueMap.h" #include "mlir/Dialect/Affine/IR/AffineValueMap.h"
#include "mlir/Support/MathExtras.h" #include "mlir/Support/MathExtras.h"
#include "llvm/ADT/DenseSet.h" #include "llvm/ADT/DenseSet.h"
#include "llvm/ADT/SmallString.h" #include "llvm/ADT/SmallString.h"
#include <type_traits> #include <type_traits>
using namespace mlir; using namespace mlir;
▲ Show 20 Lines • Show All 360 Lines • Show Last 20 Lines

mlir/lib/Analysis/NestedMatcher.cpp

//===- NestedMatcher.cpp - NestedMatcher Impl ----------------------------===// //===- NestedMatcher.cpp - NestedMatcher Impl ----------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/NestedMatcher.h" #include "mlir/Analysis/NestedMatcher.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "llvm/ADT/ArrayRef.h" #include "llvm/ADT/ArrayRef.h"
#include "llvm/ADT/STLExtras.h" #include "llvm/ADT/STLExtras.h"
#include "llvm/Support/Allocator.h" #include "llvm/Support/Allocator.h"
#include "llvm/Support/raw_ostream.h" #include "llvm/Support/raw_ostream.h"
using namespace mlir; using namespace mlir;
▲ Show 20 Lines • Show All 134 Lines • Show Last 20 Lines

mlir/lib/Analysis/SliceAnalysis.cpp

//===- UseDefAnalysis.cpp - Analysis for Transitive UseDef chains ---------===// //===- UseDefAnalysis.cpp - Analysis for Transitive UseDef chains ---------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements Analysis functions specific to slicing in Function. // This file implements Analysis functions specific to slicing in Function.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/SliceAnalysis.h" #include "mlir/Analysis/SliceAnalysis.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/LoopOps/LoopOps.h" #include "mlir/Dialect/LoopOps/LoopOps.h"
#include "mlir/IR/Function.h" #include "mlir/IR/Function.h"
#include "mlir/IR/Operation.h" #include "mlir/IR/Operation.h"
#include "mlir/Support/Functional.h" #include "mlir/Support/Functional.h"
#include "mlir/Support/LLVM.h" #include "mlir/Support/LLVM.h"
#include "mlir/Support/STLExtras.h" #include "mlir/Support/STLExtras.h"
#include "llvm/ADT/SetVector.h" #include "llvm/ADT/SetVector.h"
▲ Show 20 Lines • Show All 191 Lines • Show Last 20 Lines

mlir/lib/Analysis/Utils.cpp

//===- Utils.cpp ---- Misc utilities for analysis -------------------------===// //===- Utils.cpp ---- Misc utilities for analysis -------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements miscellaneous analysis routines for non-loop IR // This file implements miscellaneous analysis routines for non-loop IR
// structures. // structures.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/Utils.h" #include "mlir/Analysis/Utils.h"
#include "mlir/Analysis/AffineAnalysis.h" #include "mlir/Analysis/AffineAnalysis.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/AffineOps/AffineValueMap.h" #include "mlir/Dialect/Affine/IR/AffineValueMap.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "llvm/ADT/SmallPtrSet.h" #include "llvm/ADT/SmallPtrSet.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#include "llvm/Support/raw_ostream.h" #include "llvm/Support/raw_ostream.h"
#define DEBUG_TYPE "analysis-utils" #define DEBUG_TYPE "analysis-utils"
using namespace mlir; using namespace mlir;
▲ Show 20 Lines • Show All 983 Lines • Show Last 20 Lines

mlir/lib/Conversion/AffineToStandard/AffineToStandard.cpp

//===- AffineToStandard.cpp - Lower affine constructs to primitives -------===// //===- AffineToStandard.cpp - Lower affine constructs to primitives -------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file lowers affine constructs (If and For statements, AffineApply // This file lowers affine constructs (If and For statements, AffineApply
// operations) within a function into their standard If and For equivalent ops. // operations) within a function into their standard If and For equivalent ops.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/AffineToStandard/AffineToStandard.h" #include "mlir/Conversion/AffineToStandard/AffineToStandard.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/LoopOps/LoopOps.h" #include "mlir/Dialect/LoopOps/LoopOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/AffineExprVisitor.h" #include "mlir/IR/AffineExprVisitor.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/IntegerSet.h" #include "mlir/IR/IntegerSet.h"
#include "mlir/IR/MLIRContext.h" #include "mlir/IR/MLIRContext.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
▲ Show 20 Lines • Show All 576 Lines • Show Last 20 Lines

mlir/lib/Conversion/AffineToStandard/CMakeLists.txt

add_mlir_conversion_library(MLIRAffineToStandard add_mlir_conversion_library(MLIRAffineToStandard
AffineToStandard.cpp AffineToStandard.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/AffineToStandard ${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/AffineToStandard
) )
target_link_libraries( target_link_libraries(
MLIRAffineToStandard MLIRAffineToStandard
PUBLIC PUBLIC
MLIRAffineOps MLIRAffine
MLIRLoopOps MLIRLoopOps
MLIRPass MLIRPass
MLIRStandardOps MLIRStandardOps
MLIRTransforms MLIRTransforms
MLIRIR MLIRIR
LLVMCore LLVMCore
LLVMSupport LLVMSupport
) )

mlir/lib/Conversion/LoopsToGPU/CMakeLists.txt

add_mlir_conversion_library(MLIRLoopsToGPU add_mlir_conversion_library(MLIRLoopsToGPU
LoopsToGPU.cpp LoopsToGPU.cpp
LoopsToGPUPass.cpp LoopsToGPUPass.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/LoopsToGPU ${MLIR_MAIN_INCLUDE_DIR}/mlir/Conversion/LoopsToGPU
) )
target_link_libraries(MLIRLoopsToGPU target_link_libraries(MLIRLoopsToGPU
PUBLIC PUBLIC
MLIRAffineOps MLIRAffine
MLIRAffineToStandard MLIRAffineToStandard
MLIRGPU MLIRGPU
MLIRIR MLIRIR
MLIRLinalgOps MLIRLinalgOps
MLIRPass MLIRPass
MLIRStandardOps MLIRStandardOps
MLIRSupport MLIRSupport
MLIRTransforms MLIRTransforms
LLVMSupport LLVMSupport
) )

mlir/lib/Conversion/LoopsToGPU/LoopsToGPU.cpp

Show All 9 Lines
// kernel. The caller is expected to guarantee that the conversion is correct // kernel. The caller is expected to guarantee that the conversion is correct
// or to further transform the kernel to ensure correctness. // or to further transform the kernel to ensure correctness.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/LoopsToGPU/LoopsToGPU.h" #include "mlir/Conversion/LoopsToGPU/LoopsToGPU.h"
#include "mlir/Conversion/AffineToStandard/AffineToStandard.h" #include "mlir/Conversion/AffineToStandard/AffineToStandard.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/GPU/GPUDialect.h" #include "mlir/Dialect/GPU/GPUDialect.h"
#include "mlir/Dialect/GPU/ParallelLoopMapper.h" #include "mlir/Dialect/GPU/ParallelLoopMapper.h"
#include "mlir/Dialect/LoopOps/LoopOps.h" #include "mlir/Dialect/LoopOps/LoopOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
▲ Show 20 Lines • Show All 795 Lines • Show Last 20 Lines

mlir/lib/Conversion/LoopsToGPU/LoopsToGPUPass.cpp

//===- LoopsToGPUPass.cpp - Convert a loop nest to a GPU kernel -----------===// //===- LoopsToGPUPass.cpp - Convert a loop nest to a GPU kernel -----------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Conversion/LoopsToGPU/LoopsToGPUPass.h" #include "mlir/Conversion/LoopsToGPU/LoopsToGPUPass.h"
#include "mlir/Conversion/LoopsToGPU/LoopsToGPU.h" #include "mlir/Conversion/LoopsToGPU/LoopsToGPU.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/GPU/GPUDialect.h" #include "mlir/Dialect/GPU/GPUDialect.h"
#include "mlir/Dialect/LoopOps/LoopOps.h" #include "mlir/Dialect/LoopOps/LoopOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/DialectConversion.h" #include "mlir/Transforms/DialectConversion.h"
#include "llvm/ADT/ArrayRef.h" #include "llvm/ADT/ArrayRef.h"
#include "llvm/Support/CommandLine.h" #include "llvm/Support/CommandLine.h"
▲ Show 20 Lines • Show All 98 Lines • ▼ Show 20 Lines
}; };
struct ParallelLoopToGpuPass : public OperationPass<ParallelLoopToGpuPass> { struct ParallelLoopToGpuPass : public OperationPass<ParallelLoopToGpuPass> {
void runOnOperation() override { void runOnOperation() override {
OwningRewritePatternList patterns; OwningRewritePatternList patterns;
populateParallelLoopToGPUPatterns(patterns, &getContext()); populateParallelLoopToGPUPatterns(patterns, &getContext());
ConversionTarget target(getContext()); ConversionTarget target(getContext());
target.addLegalDialect<StandardOpsDialect>(); target.addLegalDialect<StandardOpsDialect>();
target.addLegalDialect<AffineOpsDialect>(); target.addLegalDialect<AffineDialect>();
target.addLegalDialect<gpu::GPUDialect>(); target.addLegalDialect<gpu::GPUDialect>();
target.addLegalDialect<loop::LoopOpsDialect>(); target.addLegalDialect<loop::LoopOpsDialect>();
target.addIllegalOp<loop::ParallelOp>(); target.addIllegalOp<loop::ParallelOp>();
if (failed(applyPartialConversion(getOperation(), target, patterns))) if (failed(applyPartialConversion(getOperation(), target, patterns)))
signalPassFailure(); signalPassFailure();
} }
}; };
Show All 38 Lines

mlir/lib/Conversion/VectorToLoops/ConvertVectorToLoops.cpp

//===- VectorToLoops.cpp - Conversion from Vector to mix of Loops and Std -===// //===- VectorToLoops.cpp - Conversion from Vector to mix of Loops and Std -===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements target-dependent lowering of vector transfer operations. // This file implements target-dependent lowering of vector transfer operations.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include <type_traits> #include <type_traits>
#include "mlir/Conversion/VectorToLoops/ConvertVectorToLoops.h" #include "mlir/Conversion/VectorToLoops/ConvertVectorToLoops.h"
#include "mlir/Dialect/AffineOps/EDSC/Intrinsics.h" #include "mlir/Dialect/Affine/EDSC/Intrinsics.h"
#include "mlir/Dialect/LoopOps/EDSC/Builders.h" #include "mlir/Dialect/LoopOps/EDSC/Builders.h"
#include "mlir/Dialect/StandardOps/EDSC/Intrinsics.h" #include "mlir/Dialect/StandardOps/EDSC/Intrinsics.h"
#include "mlir/Dialect/Vector/VectorOps.h" #include "mlir/Dialect/Vector/VectorOps.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/AffineMap.h" #include "mlir/IR/AffineMap.h"
#include "mlir/IR/Attributes.h" #include "mlir/IR/Attributes.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/Location.h" #include "mlir/IR/Location.h"
▲ Show 20 Lines • Show All 328 Lines • Show Last 20 Lines

mlir/lib/Dialect/Affine/CMakeLists.txt

  • This file was added.
add_mlir_dialect_library(MLIRAffine
IR/AffineOps.cpp
IR/AffineValueMap.cpp
EDSC/Builders.cpp
ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Affine
DEPENDS
MLIRAffineOpsIncGen
)
target_link_libraries(MLIRAffine
PUBLIC
MLIREDSC
MLIRIR
MLIRLoopLikeInterface
MLIRSideEffects
MLIRStandardOps
)
add_subdirectory(Transforms)

mlir/lib/Dialect/Affine/EDSC/Builders.cpp

  • This file was moved from mlir/lib/Dialect/AffineOps/EDSC/Builders.cpp.
//===- Builders.cpp - MLIR Declarative Builder Classes --------------------===// //===- Builders.cpp - MLIR Declarative Builder Classes --------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/AffineOps/EDSC/Builders.h" #include "mlir/Dialect/Affine/EDSC/Builders.h"
#include "mlir/Dialect/StandardOps/EDSC/Builders.h" #include "mlir/Dialect/StandardOps/EDSC/Builders.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/AffineMap.h" #include "mlir/IR/AffineMap.h"
using namespace mlir; using namespace mlir;
using namespace mlir::edsc; using namespace mlir::edsc;
static Optional<ValueHandle> emitStaticFor(ArrayRef<ValueHandle> lbs, static Optional<ValueHandle> emitStaticFor(ArrayRef<ValueHandle> lbs,
▲ Show 20 Lines • Show All 271 Lines • Show Last 20 Lines

mlir/lib/Dialect/Affine/EDSC/CMakeLists.txt

  • This file was copied from mlir/lib/Dialect/AffineOps/CMakeLists.txt.
add_mlir_dialect_library(MLIRAffineOps add_mlir_dialect_library(MLIRAffine
AffineOps.cpp
AffineValueMap.cpp
EDSC/Builders.cpp EDSC/Builders.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/AffineOps ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Affine
DEPENDS DEPENDS
MLIRAffineOpsIncGen MLIRAffineOpsIncGen
) )
target_link_libraries(MLIRAffineOps target_link_libraries(MLIRAffine
PUBLIC PUBLIC
MLIREDSC MLIREDSC
MLIRIR MLIRIR
MLIRLoopLikeInterface MLIRLoopLikeInterface
MLIRSideEffects MLIRSideEffects
MLIRStandardOps MLIRStandardOps
) )

mlir/lib/Dialect/Affine/IR/AffineOps.cpp

  • This file was moved from mlir/lib/Dialect/AffineOps/AffineOps.cpp.
//===- AffineOps.cpp - MLIR Affine Operations -----------------------------===// //===- AffineOps.cpp - MLIR Affine Operations -----------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/AffineOps/AffineValueMap.h" #include "mlir/Dialect/Affine/IR/AffineValueMap.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/Function.h" #include "mlir/IR/Function.h"
#include "mlir/IR/IntegerSet.h" #include "mlir/IR/IntegerSet.h"
#include "mlir/IR/Matchers.h" #include "mlir/IR/Matchers.h"
#include "mlir/IR/OpImplementation.h" #include "mlir/IR/OpImplementation.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/Transforms/InliningUtils.h" #include "mlir/Transforms/InliningUtils.h"
#include "llvm/ADT/SetVector.h" #include "llvm/ADT/SetVector.h"
#include "llvm/ADT/SmallBitVector.h" #include "llvm/ADT/SmallBitVector.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
using namespace mlir; using namespace mlir;
using llvm::dbgs; using llvm::dbgs;
#define DEBUG_TYPE "affine-analysis" #define DEBUG_TYPE "affine-analysis"
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// AffineOpsDialect Interfaces // AffineDialect Interfaces
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
namespace { namespace {
/// This class defines the interface for handling inlining with affine /// This class defines the interface for handling inlining with affine
/// operations. /// operations.
struct AffineInlinerInterface : public DialectInlinerInterface { struct AffineInlinerInterface : public DialectInlinerInterface {
using DialectInlinerInterface::DialectInlinerInterface; using DialectInlinerInterface::DialectInlinerInterface;
Show All 22 Linesstruct AffineInlinerInterface : public DialectInlinerInterface {
} }
/// Affine regions should be analyzed recursively. /// Affine regions should be analyzed recursively.
bool shouldAnalyzeRecursively(Operation *op) const final { return true; } bool shouldAnalyzeRecursively(Operation *op) const final { return true; }
}; };
} // end anonymous namespace } // end anonymous namespace
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// AffineOpsDialect // AffineDialect
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
AffineOpsDialect::AffineOpsDialect(MLIRContext *context) AffineDialect::AffineDialect(MLIRContext *context)
: Dialect(getDialectNamespace(), context) { : Dialect(getDialectNamespace(), context) {
addOperations<AffineDmaStartOp, AffineDmaWaitOp, AffineLoadOp, AffineStoreOp, addOperations<AffineDmaStartOp, AffineDmaWaitOp, AffineLoadOp, AffineStoreOp,
#define GET_OP_LIST #define GET_OP_LIST
#include "mlir/Dialect/AffineOps/AffineOps.cpp.inc" #include "mlir/Dialect/Affine/IR/AffineOps.cpp.inc"
>(); >();
addInterfaces<AffineInlinerInterface>(); addInterfaces<AffineInlinerInterface>();
} }
/// Materialize a single constant operation from a given attribute value with /// Materialize a single constant operation from a given attribute value with
/// the desired resultant type. /// the desired resultant type.
Operation *AffineOpsDialect::materializeConstant(OpBuilder &builder, Operation *AffineDialect::materializeConstant(OpBuilder &builder,
Attribute value, Type type, Attribute value, Type type,
Location loc) { Location loc) {
return builder.create<ConstantOp>(loc, type, value); return builder.create<ConstantOp>(loc, type, value);
} }
/// A utility function to check if a given region is attached to a function. /// A utility function to check if a given region is attached to a function.
static bool isFunctionRegion(Region *region) { static bool isFunctionRegion(Region *region) {
return llvm::isa<FuncOp>(region->getParentOp()); return llvm::isa<FuncOp>(region->getParentOp());
▲ Show 20 Lines • Show All 2,274 Lines • ▼ Show 20 Linesstatic ParseResult parseAffineParallelOp(OpAsmParser &parser,
return success(); return success();
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// TableGen'd op method definitions // TableGen'd op method definitions
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#define GET_OP_CLASSES #define GET_OP_CLASSES
#include "mlir/Dialect/AffineOps/AffineOps.cpp.inc" #include "mlir/Dialect/Affine/IR/AffineOps.cpp.inc"

mlir/lib/Dialect/Affine/IR/AffineValueMap.cpp

  • This file was moved from mlir/lib/Dialect/AffineOps/AffineValueMap.cpp.
//===- AffineValueMap.cpp - MLIR Affine Value Map Class -------------------===// //===- AffineValueMap.cpp - MLIR Affine Value Map Class -------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/AffineOps/AffineValueMap.h" #include "mlir/Dialect/Affine/IR/AffineValueMap.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
using namespace mlir; using namespace mlir;
AffineValueMap::AffineValueMap(AffineMap map, ValueRange operands, AffineValueMap::AffineValueMap(AffineMap map, ValueRange operands,
ValueRange results) ValueRange results)
: map(map), operands(operands.begin(), operands.end()), : map(map), operands(operands.begin(), operands.end()),
results(results.begin(), results.end()) {} results(results.begin(), results.end()) {}
▲ Show 20 Lines • Show All 84 Lines • Show Last 20 Lines

mlir/lib/Dialect/Affine/IR/CMakeLists.txt

  • This file was moved from mlir/lib/Dialect/AffineOps/CMakeLists.txt.
add_mlir_dialect_library(MLIRAffineOps add_mlir_dialect_library(MLIRAffine
AffineOps.cpp AffineOps.cpp
AffineValueMap.cpp AffineValueMap.cpp
EDSC/Builders.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/AffineOps ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Affine
DEPENDS DEPENDS
MLIRAffineOpsIncGen MLIRAffineOpsIncGen
) )
target_link_libraries(MLIRAffineOps target_link_libraries(MLIRAffine
PUBLIC PUBLIC
MLIREDSC MLIREDSC
MLIRIR MLIRIR
MLIRLoopLikeInterface MLIRLoopLikeInterface
MLIRSideEffects MLIRSideEffects
MLIRStandardOps MLIRStandardOps
) )

mlir/lib/Dialect/Affine/Transforms/AffineDataCopyGeneration.cpp

  • This file was moved from mlir/lib/Transforms/AffineDataCopyGeneration.cpp.
Show All 14 Lines
// use of hardware prefetch streams, and reduced false sharing. It is also // use of hardware prefetch streams, and reduced false sharing. It is also
// necessary for hardware that explicitly managed levels in the memory // necessary for hardware that explicitly managed levels in the memory
// hierarchy, and where DMAs may have to be used. This optimization is often // hierarchy, and where DMAs may have to be used. This optimization is often
// performed on already tiled code. // performed on already tiled code.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/Utils.h" #include "mlir/Analysis/Utils.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Affine/Passes.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/LoopUtils.h" #include "mlir/Transforms/LoopUtils.h"
#include "mlir/Transforms/Passes.h"
#include "mlir/Transforms/Utils.h" #include "mlir/Transforms/Utils.h"
#include "llvm/ADT/MapVector.h" #include "llvm/ADT/MapVector.h"
#include "llvm/Support/CommandLine.h" #include "llvm/Support/CommandLine.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#include <algorithm> #include <algorithm>
#define DEBUG_TYPE "affine-data-copy-generate" #define DEBUG_TYPE "affine-data-copy-generate"
▲ Show 20 Lines • Show All 233 Lines • Show Last 20 Lines

mlir/lib/Dialect/Affine/Transforms/AffineLoopInvariantCodeMotion.cpp

  • This file was moved from mlir/lib/Transforms/AffineLoopInvariantCodeMotion.cpp.
Show All 9 Lines
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/AffineAnalysis.h" #include "mlir/Analysis/AffineAnalysis.h"
#include "mlir/Analysis/AffineStructures.h" #include "mlir/Analysis/AffineStructures.h"
#include "mlir/Analysis/LoopAnalysis.h" #include "mlir/Analysis/LoopAnalysis.h"
#include "mlir/Analysis/SliceAnalysis.h" #include "mlir/Analysis/SliceAnalysis.h"
#include "mlir/Analysis/Utils.h" #include "mlir/Analysis/Utils.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/AffineMap.h" #include "mlir/IR/AffineMap.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/LoopUtils.h" #include "mlir/Transforms/LoopUtils.h"
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
#include "mlir/Transforms/Utils.h" #include "mlir/Transforms/Utils.h"
#include "llvm/ADT/DenseMap.h" #include "llvm/ADT/DenseMap.h"
▲ Show 20 Lines • Show All 212 Lines • Show Last 20 Lines

mlir/lib/Dialect/Affine/Transforms/CMakeLists.txt

  • This file was added.
add_mlir_dialect_library(MLIRAffineTransforms
AffineDataCopyGeneration.cpp
AffineLoopInvariantCodeMotion.cpp
SimplifyAffineStructures.cpp
ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Affine
DEPENDS
MLIRAffineOpsIncGen
MLIRLoopLikeInterfaceIncGen
)
target_link_libraries(MLIRAffineTransforms
PUBLIC
MLIRAffine
MLIREDSC
MLIRIR
MLIRSideEffects
MLIRStandardOps
)

mlir/lib/Dialect/Affine/Transforms/SimplifyAffineStructures.cpp

  • This file was moved from mlir/lib/Transforms/SimplifyAffineStructures.cpp.
//===- SimplifyAffineStructures.cpp ---------------------------------------===// //===- SimplifyAffineStructures.cpp ---------------------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements a pass to simplify affine structures. // This file implements a pass to simplify affine structures.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/AffineStructures.h" #include "mlir/Analysis/AffineStructures.h"
#include "mlir/IR/IntegerSet.h" #include "mlir/IR/IntegerSet.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/Passes.h" #include "mlir/Dialect/Affine/IR/AffineValueMap.h"
#include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Affine/Passes.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Transforms/Utils.h" #include "mlir/Transforms/Utils.h"
#define DEBUG_TYPE "simplify-affine-structure" #define DEBUG_TYPE "simplify-affine-structure"
using namespace mlir; using namespace mlir;
namespace { namespace {
▲ Show 20 Lines • Show All 84 Lines • Show Last 20 Lines

mlir/lib/Dialect/AffineOps/AffineOps.cpp

  • This file was moved to mlir/lib/Dialect/Affine/IR/AffineOps.cpp.

mlir/lib/Dialect/AffineOps/AffineValueMap.cpp

  • This file was moved to mlir/lib/Dialect/Affine/IR/AffineValueMap.cpp.

mlir/lib/Dialect/AffineOps/CMakeLists.txt

  • This file was deleted after being copied to mlir/lib/Dialect/Affine/EDSC/CMakeLists.txt, mlir/lib/Dialect/Affine/IR/CMakeLists.txt.
The contents of this file were not changed.

mlir/lib/Dialect/AffineOps/EDSC/Builders.cpp

  • This file was moved to mlir/lib/Dialect/Affine/EDSC/Builders.cpp.

mlir/lib/Dialect/CMakeLists.txt

add_subdirectory(Affine)
add_subdirectory(AVX512) add_subdirectory(AVX512)
add_subdirectory(AffineOps)
add_subdirectory(FxpMathOps) add_subdirectory(FxpMathOps)
add_subdirectory(GPU) add_subdirectory(GPU)
add_subdirectory(Linalg) add_subdirectory(Linalg)
add_subdirectory(LLVMIR) add_subdirectory(LLVMIR)
add_subdirectory(LoopOps) add_subdirectory(LoopOps)
add_subdirectory(OpenMP) add_subdirectory(OpenMP)
add_subdirectory(Quant) add_subdirectory(Quant)
add_subdirectory(SDBM) add_subdirectory(SDBM)
Show All 19 Lines

mlir/lib/Dialect/Linalg/EDSC/Builders.cpp

//===- Builders.cpp - MLIR Declarative Linalg Builders --------------------===// //===- Builders.cpp - MLIR Declarative Linalg Builders --------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/Dialect/AffineOps/EDSC/Intrinsics.h" #include "mlir/Dialect/Affine/EDSC/Intrinsics.h"
#include "mlir/Dialect/Linalg/EDSC/Intrinsics.h" #include "mlir/Dialect/Linalg/EDSC/Intrinsics.h"
#include "mlir/Dialect/LoopOps/EDSC/Builders.h" #include "mlir/Dialect/LoopOps/EDSC/Builders.h"
#include "mlir/Dialect/StandardOps/EDSC/Intrinsics.h" #include "mlir/Dialect/StandardOps/EDSC/Intrinsics.h"
#include "mlir/Dialect/Utils/StructuredOpsUtils.h" #include "mlir/Dialect/Utils/StructuredOpsUtils.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/Support/Functional.h" #include "mlir/Support/Functional.h"
using namespace mlir; using namespace mlir;
▲ Show 20 Lines • Show All 386 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/EDSC/CMakeLists.txt

add_mlir_dialect_library(MLIRLinalgEDSC add_mlir_dialect_library(MLIRLinalgEDSC
Builders.cpp Builders.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Linalg ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Linalg
DEPENDS DEPENDS
intrinsics_gen intrinsics_gen
) )
target_link_libraries(MLIRLinalgEDSC target_link_libraries(MLIRLinalgEDSC
PUBLIC PUBLIC
MLIREDSC MLIREDSC
MLIRIR MLIRIR
MLIRAffineOps MLIRAffine
MLIRLinalgOps MLIRLinalgOps
MLIRLoopOps MLIRLoopOps
MLIRStandardOps MLIRStandardOps
LLVMSupport LLVMSupport
) )

mlir/lib/Dialect/Linalg/Transforms/CMakeLists.txt

add_mlir_dialect_library(MLIRLinalgTransforms add_mlir_dialect_library(MLIRLinalgTransforms
Fusion.cpp Fusion.cpp
LinalgTransforms.cpp LinalgTransforms.cpp
LinalgToLoops.cpp LinalgToLoops.cpp
Promotion.cpp Promotion.cpp
Tiling.cpp Tiling.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Linalg ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Linalg
DEPENDS DEPENDS
intrinsics_gen intrinsics_gen
MLIRLinalgTransformPatternsIncGen MLIRLinalgTransformPatternsIncGen
) )
target_link_libraries(MLIRLinalgTransforms target_link_libraries(MLIRLinalgTransforms
PUBLIC PUBLIC
MLIRAffineOps MLIRAffine
MLIRAnalysis MLIRAnalysis
MLIREDSC MLIREDSC
MLIRIR MLIRIR
MLIRLinalgAnalysis MLIRLinalgAnalysis
MLIRLinalgEDSC MLIRLinalgEDSC
MLIRLinalgOps MLIRLinalgOps
MLIRLinalgUtils MLIRLinalgUtils
MLIRLoopOps MLIRLoopOps
MLIRPass MLIRPass
MLIRStandardOps MLIRStandardOps
MLIRStandardToLLVM MLIRStandardToLLVM
MLIRTransformUtils MLIRTransformUtils
MLIRVector MLIRVector
) )

mlir/lib/Dialect/Linalg/Transforms/LinalgToLoops.cpp

//===- LinalgToLoops.cpp - conversion from Linalg library ops to loops-----===// //===- LinalgToLoops.cpp - conversion from Linalg library ops to loops-----===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/AffineOps/EDSC/Intrinsics.h" #include "mlir/Dialect/Affine/EDSC/Intrinsics.h"
#include "mlir/Dialect/Linalg/EDSC/Intrinsics.h" #include "mlir/Dialect/Linalg/EDSC/Intrinsics.h"
#include "mlir/Dialect/Linalg/IR/LinalgOps.h" #include "mlir/Dialect/Linalg/IR/LinalgOps.h"
#include "mlir/Dialect/Linalg/IR/LinalgTypes.h" #include "mlir/Dialect/Linalg/IR/LinalgTypes.h"
#include "mlir/Dialect/Linalg/Passes.h" #include "mlir/Dialect/Linalg/Passes.h"
#include "mlir/Dialect/Linalg/Transforms/LinalgTransforms.h" #include "mlir/Dialect/Linalg/Transforms/LinalgTransforms.h"
#include "mlir/Dialect/Linalg/Utils/Utils.h" #include "mlir/Dialect/Linalg/Utils/Utils.h"
#include "mlir/Dialect/LoopOps/EDSC/Builders.h" #include "mlir/Dialect/LoopOps/EDSC/Builders.h"
#include "mlir/Dialect/StandardOps/EDSC/Intrinsics.h" #include "mlir/Dialect/StandardOps/EDSC/Intrinsics.h"
▲ Show 20 Lines • Show All 697 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/Transforms/Promotion.cpp

//===- Promotion.cpp - Implementation of linalg Promotion -----------------===// //===- Promotion.cpp - Implementation of linalg Promotion -----------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements the linalg dialect Promotion pass. // This file implements the linalg dialect Promotion pass.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/AffineOps/EDSC/Intrinsics.h" #include "mlir/Dialect/Affine/EDSC/Intrinsics.h"
#include "mlir/Dialect/Linalg/EDSC/Intrinsics.h" #include "mlir/Dialect/Linalg/EDSC/Intrinsics.h"
#include "mlir/Dialect/Linalg/IR/LinalgOps.h" #include "mlir/Dialect/Linalg/IR/LinalgOps.h"
#include "mlir/Dialect/Linalg/IR/LinalgTypes.h" #include "mlir/Dialect/Linalg/IR/LinalgTypes.h"
#include "mlir/Dialect/Linalg/Passes.h" #include "mlir/Dialect/Linalg/Passes.h"
#include "mlir/Dialect/Linalg/Utils/Utils.h" #include "mlir/Dialect/Linalg/Utils/Utils.h"
#include "mlir/Dialect/LoopOps/LoopOps.h" #include "mlir/Dialect/LoopOps/LoopOps.h"
#include "mlir/Dialect/StandardOps/EDSC/Intrinsics.h" #include "mlir/Dialect/StandardOps/EDSC/Intrinsics.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
▲ Show 20 Lines • Show All 238 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/Transforms/Tiling.cpp

//===- Tiling.cpp - Implementation of linalg Tiling -----------------------===// //===- Tiling.cpp - Implementation of linalg Tiling -----------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements the linalg dialect Tiling pass. // This file implements the linalg dialect Tiling pass.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/AffineOps/EDSC/Intrinsics.h" #include "mlir/Dialect/Affine/EDSC/Intrinsics.h"
#include "mlir/Dialect/Linalg/EDSC/Intrinsics.h" #include "mlir/Dialect/Linalg/EDSC/Intrinsics.h"
#include "mlir/Dialect/Linalg/IR/LinalgTypes.h" #include "mlir/Dialect/Linalg/IR/LinalgTypes.h"
#include "mlir/Dialect/Linalg/Passes.h" #include "mlir/Dialect/Linalg/Passes.h"
#include "mlir/Dialect/Linalg/Utils/Utils.h" #include "mlir/Dialect/Linalg/Utils/Utils.h"
#include "mlir/Dialect/LoopOps/EDSC/Builders.h" #include "mlir/Dialect/LoopOps/EDSC/Builders.h"
#include "mlir/Dialect/StandardOps/EDSC/Intrinsics.h" #include "mlir/Dialect/StandardOps/EDSC/Intrinsics.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/AffineExprVisitor.h" #include "mlir/IR/AffineExprVisitor.h"
▲ Show 20 Lines • Show All 532 Lines • Show Last 20 Lines

mlir/lib/Dialect/Linalg/Utils/CMakeLists.txt

add_mlir_dialect_library(MLIRLinalgUtils add_mlir_dialect_library(MLIRLinalgUtils
Utils.cpp Utils.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Linalg ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Linalg
DEPENDS DEPENDS
intrinsics_gen intrinsics_gen
) )
target_link_libraries(MLIRLinalgUtils target_link_libraries(MLIRLinalgUtils
PUBLIC PUBLIC
MLIRAffineOps MLIRAffine
MLIREDSC MLIREDSC
MLIRIR MLIRIR
MLIRLinalgOps MLIRLinalgOps
MLIRLoopOps MLIRLoopOps
MLIRPass MLIRPass
MLIRStandardOps MLIRStandardOps
MLIRTransformUtils MLIRTransformUtils
) )

mlir/lib/Dialect/Linalg/Utils/Utils.cpp

//===- Utils.cpp - Utilities to support the Linalg dialect ----------------===// //===- Utils.cpp - Utilities to support the Linalg dialect ----------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements utilities for the Linalg dialect. // This file implements utilities for the Linalg dialect.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Linalg/Utils/Utils.h" #include "mlir/Dialect/Linalg/Utils/Utils.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/Linalg/IR/LinalgOps.h" #include "mlir/Dialect/Linalg/IR/LinalgOps.h"
#include "mlir/Dialect/Linalg/IR/LinalgTypes.h" #include "mlir/Dialect/Linalg/IR/LinalgTypes.h"
#include "mlir/Dialect/LoopOps/LoopOps.h" #include "mlir/Dialect/LoopOps/LoopOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/AffineMap.h" #include "mlir/IR/AffineMap.h"
#include "mlir/IR/Matchers.h" #include "mlir/IR/Matchers.h"
#include "mlir/IR/OpImplementation.h" #include "mlir/IR/OpImplementation.h"
▲ Show 20 Lines • Show All 82 Lines • Show Last 20 Lines

mlir/lib/Dialect/LoopOps/Transforms/CMakeLists.txt

add_mlir_dialect_library(MLIRLoopOpsTransforms add_mlir_dialect_library(MLIRLoopOpsTransforms
ParallelLoopFusion.cpp ParallelLoopFusion.cpp
ParallelLoopSpecialization.cpp ParallelLoopSpecialization.cpp
ParallelLoopTiling.cpp ParallelLoopTiling.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/LoopOps ${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/LoopOps
) )
target_link_libraries(MLIRLoopOpsTransforms target_link_libraries(MLIRLoopOpsTransforms
PUBLIC PUBLIC
MLIRAffineOps MLIRAffine
MLIRIR MLIRIR
MLIRPass MLIRPass
MLIRLoopOps MLIRLoopOps
MLIRStandardOps MLIRStandardOps
MLIRSupport MLIRSupport
LLVMSupport LLVMSupport
) )

mlir/lib/Dialect/LoopOps/Transforms/ParallelLoopSpecialization.cpp

//===- ParallelLoopSpecialization.cpp - loop.parallel specializeation -----===// //===- ParallelLoopSpecialization.cpp - loop.parallel specializeation -----===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// Specializes parallel loops for easier unrolling and vectorization. // Specializes parallel loops for easier unrolling and vectorization.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/LoopOps/LoopOps.h" #include "mlir/Dialect/LoopOps/LoopOps.h"
#include "mlir/Dialect/LoopOps/Passes.h" #include "mlir/Dialect/LoopOps/Passes.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
using namespace mlir; using namespace mlir;
▲ Show 20 Lines • Show All 55 Lines • Show Last 20 Lines

mlir/lib/Dialect/LoopOps/Transforms/ParallelLoopTiling.cpp

//===- ParallelLoopTiling.cpp - Tiles loop.parallel ---------------===// //===- ParallelLoopTiling.cpp - Tiles loop.parallel ---------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements loop tiling on parallel loops. // This file implements loop tiling on parallel loops.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/LoopOps/LoopOps.h" #include "mlir/Dialect/LoopOps/LoopOps.h"
#include "mlir/Dialect/LoopOps/Passes.h" #include "mlir/Dialect/LoopOps/Passes.h"
#include "mlir/Dialect/LoopOps/Transforms.h" #include "mlir/Dialect/LoopOps/Transforms.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/RegionUtils.h" #include "mlir/Transforms/RegionUtils.h"
#include "llvm/Support/CommandLine.h" #include "llvm/Support/CommandLine.h"
▲ Show 20 Lines • Show All 113 Lines • Show Last 20 Lines

mlir/lib/Dialect/Vector/CMakeLists.txt

Show All 10 Linesadd_mlir_dialect_library(MLIRVector
MLIRVectorOpsIncGen MLIRVectorOpsIncGen
MLIRVectorTransformPatternsIncGen MLIRVectorTransformPatternsIncGen
) )
target_link_libraries(MLIRVector target_link_libraries(MLIRVector
PUBLIC PUBLIC
MLIREDSC MLIREDSC
MLIRIR MLIRIR
MLIRStandardOps MLIRStandardOps
MLIRAffineOps MLIRAffine
MLIRLoopOps MLIRLoopOps
MLIRLoopAnalysis MLIRLoopAnalysis
MLIRSideEffects MLIRSideEffects
) )

mlir/lib/Dialect/Vector/VectorTransforms.cpp

  • This file was copied to mlir/lib/Dialect/VectorOps/VectorTransforms.cpp.
//===- VectorToLoops.cpp - Conversion within the Vector dialect -----------===// //===- VectorToLoops.cpp - Conversion within the Vector dialect -----------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements target-independent rewrites as 1->N patterns. // This file implements target-independent rewrites as 1->N patterns.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include <type_traits> #include <type_traits>
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Utils/StructuredOpsUtils.h" #include "mlir/Dialect/Utils/StructuredOpsUtils.h"
#include "mlir/Dialect/Vector/VectorOps.h" #include "mlir/Dialect/Vector/VectorOps.h"
#include "mlir/Dialect/Vector/VectorTransforms.h" #include "mlir/Dialect/Vector/VectorTransforms.h"
#include "mlir/Dialect/Vector/VectorUtils.h" #include "mlir/Dialect/Vector/VectorUtils.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/AffineMap.h" #include "mlir/IR/AffineMap.h"
#include "mlir/IR/Attributes.h" #include "mlir/IR/Attributes.h"
▲ Show 20 Lines • Show All 1,336 Lines • Show Last 20 Lines

mlir/lib/Dialect/Vector/VectorUtils.cpp

  • This file was copied to mlir/lib/Dialect/VectorOps/VectorUtils.cpp.
//===- VectorUtils.cpp - MLIR Utilities for VectorOps ------------------===// //===- VectorUtils.cpp - MLIR Utilities for VectorOps ------------------===//
// //
// Part of the MLIR Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the MLIR Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements utility methods for working with the Vector dialect. // This file implements utility methods for working with the Vector dialect.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Vector/VectorUtils.h" #include "mlir/Dialect/Vector/VectorUtils.h"
#include "mlir/Analysis/LoopAnalysis.h" #include "mlir/Analysis/LoopAnalysis.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Vector/VectorOps.h" #include "mlir/Dialect/Vector/VectorOps.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/IntegerSet.h" #include "mlir/IR/IntegerSet.h"
#include "mlir/IR/Operation.h" #include "mlir/IR/Operation.h"
#include "mlir/Support/Functional.h" #include "mlir/Support/Functional.h"
#include "mlir/Support/LLVM.h" #include "mlir/Support/LLVM.h"
#include "mlir/Support/MathExtras.h" #include "mlir/Support/MathExtras.h"
▲ Show 20 Lines • Show All 255 Lines • Show Last 20 Lines

mlir/lib/Dialect/VectorOps/VectorTransforms.cpp

  • This file was copied from mlir/lib/Dialect/Vector/VectorTransforms.cpp.
//===- VectorToLoops.cpp - Conversion within the Vector dialect -----------===// //===- VectorToLoops.cpp - Conversion within the Vector dialect -----------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements target-independent rewrites as 1->N patterns. // This file implements target-independent rewrites as 1->N patterns.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include <type_traits> #include <type_traits>
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Utils/StructuredOpsUtils.h" #include "mlir/Dialect/Utils/StructuredOpsUtils.h"
#include "mlir/Dialect/Vector/VectorOps.h" #include "mlir/Dialect/VectorOps/VectorOps.h"
#include "mlir/Dialect/Vector/VectorTransforms.h" #include "mlir/Dialect/VectorOps/VectorTransforms.h"
#include "mlir/Dialect/Vector/VectorUtils.h" #include "mlir/Dialect/VectorOps/VectorUtils.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/AffineMap.h" #include "mlir/IR/AffineMap.h"
#include "mlir/IR/Attributes.h" #include "mlir/IR/Attributes.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/Function.h" #include "mlir/IR/Function.h"
#include "mlir/IR/Location.h" #include "mlir/IR/Location.h"
#include "mlir/IR/Matchers.h" #include "mlir/IR/Matchers.h"
#include "mlir/IR/Module.h" #include "mlir/IR/Module.h"
#include "mlir/IR/OperationSupport.h" #include "mlir/IR/OperationSupport.h"
#include "mlir/IR/PatternMatch.h" #include "mlir/IR/PatternMatch.h"
#include "mlir/IR/Types.h" #include "mlir/IR/Types.h"
#include "mlir/Support/Functional.h" #include "mlir/Support/Functional.h"
#include "mlir/Support/STLExtras.h" #include "mlir/Support/STLExtras.h"
#include "llvm/Support/CommandLine.h" #include "llvm/Support/CommandLine.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#include "llvm/Support/raw_ostream.h" #include "llvm/Support/raw_ostream.h"
#define DEBUG_TYPE "vector-to-vector" #define DEBUG_TYPE "vector-to-vector"
using namespace mlir; using namespace mlir;
using llvm::dbgs; using llvm::dbgs;
using mlir::functional::zipMap; using mlir::functional::zipMap;
static llvm::cl::OptionCategory clOptionsCategory(DEBUG_TYPE " options");
static llvm::cl::opt<bool> lowerToLLVMMatrixIntrinsics(
"vector-lower-matrix-intrinsics",
llvm::cl::desc("Lower vector.contract to llvm.intr.matrix.multiply"),
llvm::cl::init(false), llvm::cl::cat(clOptionsCategory));
/// Given a shape with sizes greater than 0 along all dimensions, /// Given a shape with sizes greater than 0 along all dimensions,
/// returns the distance, in number of elements, between a slice in a dimension /// returns the distance, in number of elements, between a slice in a dimension
/// and the next slice in the same dimension. /// and the next slice in the same dimension.
/// e.g. shape[3, 4, 5] -> linearization_basis[20, 5, 1] /// e.g. shape[3, 4, 5] -> linearization_basis[20, 5, 1]
static SmallVector<int64_t, 8> computeStrides(ArrayRef<int64_t> shape) { static SmallVector<int64_t, 8> computeStrides(ArrayRef<int64_t> shape) {
if (shape.empty()) if (shape.empty())
return {}; return {};
SmallVector<int64_t, 8> tmp; SmallVector<int64_t, 8> tmp;
▲ Show 20 Lines • Show All 487 Lines • ▼ Show 20 Lines
namespace { namespace {
// Splits vector TransferReadOp into smaller TransferReadOps based on slicing // Splits vector TransferReadOp into smaller TransferReadOps based on slicing
// scheme of its unique ExtractSlicesOp user. // scheme of its unique ExtractSlicesOp user.
struct SplitTransferReadOp : public OpRewritePattern<vector::TransferReadOp> { struct SplitTransferReadOp : public OpRewritePattern<vector::TransferReadOp> {
using OpRewritePattern<vector::TransferReadOp>::OpRewritePattern; using OpRewritePattern<vector::TransferReadOp>::OpRewritePattern;
LogicalResult matchAndRewrite(vector::TransferReadOp xferReadOp, PatternMatchResult matchAndRewrite(vector::TransferReadOp xferReadOp,
PatternRewriter &rewriter) const override { PatternRewriter &rewriter) const override {
// TODO(andydavis, ntv) Support splitting TransferReadOp with non-identity // TODO(andydavis, ntv) Support splitting TransferReadOp with non-identity
// permutation maps. Repurpose code from MaterializeVectors transformation. // permutation maps. Repurpose code from MaterializeVectors transformation.
if (!isIdentitySuffix(xferReadOp.permutation_map())) if (!isIdentitySuffix(xferReadOp.permutation_map()))
return failure(); return matchFailure();
// Return unless the unique 'xferReadOp' user is an ExtractSlicesOp. // Return unless the unique 'xferReadOp' user is an ExtractSlicesOp.
Value xferReadResult = xferReadOp.getResult(); Value xferReadResult = xferReadOp.getResult();
auto extractSlicesOp = auto extractSlicesOp =
dyn_cast<vector::ExtractSlicesOp>(*xferReadResult.getUsers().begin()); dyn_cast<vector::ExtractSlicesOp>(*xferReadResult.getUsers().begin());
if (!xferReadResult.hasOneUse() || !extractSlicesOp) if (!xferReadResult.hasOneUse() || !extractSlicesOp)
return failure(); return matchFailure();
// Get 'sizes' and 'strides' parameters from ExtractSlicesOp user. // Get 'sizes' and 'strides' parameters from ExtractSlicesOp user.
auto sourceVectorType = extractSlicesOp.getSourceVectorType(); auto sourceVectorType = extractSlicesOp.getSourceVectorType();
auto resultTupleType = extractSlicesOp.getResultTupleType(); auto resultTupleType = extractSlicesOp.getResultTupleType();
SmallVector<int64_t, 4> sizes; SmallVector<int64_t, 4> sizes;
extractSlicesOp.getSizes(sizes); extractSlicesOp.getSizes(sizes);
SmallVector<int64_t, 4> strides; SmallVector<int64_t, 4> strides;
extractSlicesOp.getStrides(strides); extractSlicesOp.getStrides(strides);
Show All 20 Lines PatternMatchResult matchAndRewrite(vector::TransferReadOp xferReadOp,
// Create tuple of splice xfer read operations. // Create tuple of splice xfer read operations.
Value tupleOp = rewriter.create<vector::TupleOp>(loc, resultTupleType, Value tupleOp = rewriter.create<vector::TupleOp>(loc, resultTupleType,
vectorTupleValues); vectorTupleValues);
// Replace 'xferReadOp' with result 'insertSlicesResult'. // Replace 'xferReadOp' with result 'insertSlicesResult'.
rewriter.replaceOpWithNewOp<vector::InsertSlicesOp>( rewriter.replaceOpWithNewOp<vector::InsertSlicesOp>(
xferReadOp, sourceVectorType, tupleOp, extractSlicesOp.sizes(), xferReadOp, sourceVectorType, tupleOp, extractSlicesOp.sizes(),
extractSlicesOp.strides()); extractSlicesOp.strides());
return success(); return matchSuccess();
} }
}; };
// Splits vector TransferWriteOp into smaller TransferWriteOps for each source. // Splits vector TransferWriteOp into smaller TransferWriteOps for each source.
struct SplitTransferWriteOp : public OpRewritePattern<vector::TransferWriteOp> { struct SplitTransferWriteOp : public OpRewritePattern<vector::TransferWriteOp> {
using OpRewritePattern<vector::TransferWriteOp>::OpRewritePattern; using OpRewritePattern<vector::TransferWriteOp>::OpRewritePattern;
LogicalResult matchAndRewrite(vector::TransferWriteOp xferWriteOp, PatternMatchResult matchAndRewrite(vector::TransferWriteOp xferWriteOp,
PatternRewriter &rewriter) const override { PatternRewriter &rewriter) const override {
// TODO(andydavis, ntv) Support splitting TransferWriteOp with non-identity // TODO(andydavis, ntv) Support splitting TransferWriteOp with non-identity
// permutation maps. Repurpose code from MaterializeVectors transformation. // permutation maps. Repurpose code from MaterializeVectors transformation.
if (!isIdentitySuffix(xferWriteOp.permutation_map())) if (!isIdentitySuffix(xferWriteOp.permutation_map()))
return failure(); return matchFailure();
// Return unless the 'xferWriteOp' 'vector' operand is an 'InsertSlicesOp'. // Return unless the 'xferWriteOp' 'vector' operand is an 'InsertSlicesOp'.
auto *vectorDefOp = xferWriteOp.vector().getDefiningOp(); auto *vectorDefOp = xferWriteOp.vector().getDefiningOp();
auto insertSlicesOp = dyn_cast_or_null<vector::InsertSlicesOp>(vectorDefOp); auto insertSlicesOp = dyn_cast_or_null<vector::InsertSlicesOp>(vectorDefOp);
if (!insertSlicesOp) if (!insertSlicesOp)
return failure(); return matchFailure();
// Get TupleOp operand of 'insertSlicesOp'. // Get TupleOp operand of 'insertSlicesOp'.
auto tupleOp = dyn_cast_or_null<vector::TupleOp>( auto tupleOp = dyn_cast_or_null<vector::TupleOp>(
insertSlicesOp.vectors().getDefiningOp()); insertSlicesOp.vectors().getDefiningOp());
if (!tupleOp) if (!tupleOp)
return failure(); return matchFailure();
// Get 'sizes' and 'strides' parameters from InsertSlicesOp user. // Get 'sizes' and 'strides' parameters from InsertSlicesOp user.
auto sourceTupleType = insertSlicesOp.getSourceTupleType(); auto sourceTupleType = insertSlicesOp.getSourceTupleType();
auto resultVectorType = insertSlicesOp.getResultVectorType(); auto resultVectorType = insertSlicesOp.getResultVectorType();
SmallVector<int64_t, 4> sizes; SmallVector<int64_t, 4> sizes;
insertSlicesOp.getSizes(sizes); insertSlicesOp.getSizes(sizes);
SmallVector<int64_t, 4> strides; SmallVector<int64_t, 4> strides;
insertSlicesOp.getStrides(strides); insertSlicesOp.getStrides(strides);
Show All 10 Lines auto createSlice = [&](unsigned index, ArrayRef<Value> sliceIndices) {
xferWriteOp.permutation_map()); xferWriteOp.permutation_map());
}; };
generateTransferOpSlices(memrefElementType, resultVectorType, generateTransferOpSlices(memrefElementType, resultVectorType,
sourceTupleType, sizes, strides, indices, rewriter, sourceTupleType, sizes, strides, indices, rewriter,
createSlice); createSlice);
// Erase old 'xferWriteOp'. // Erase old 'xferWriteOp'.
rewriter.eraseOp(xferWriteOp); rewriter.eraseOp(xferWriteOp);
return success(); return matchSuccess();
} }
}; };
/// Decomposes ShapeCastOp on tuple-of-vectors to multiple ShapeCastOps, each /// Decomposes ShapeCastOp on tuple-of-vectors to multiple ShapeCastOps, each
/// on vector types. /// on vector types.
struct ShapeCastOpDecomposer : public OpRewritePattern<vector::ShapeCastOp> { struct ShapeCastOpDecomposer : public OpRewritePattern<vector::ShapeCastOp> {
using OpRewritePattern<vector::ShapeCastOp>::OpRewritePattern; using OpRewritePattern<vector::ShapeCastOp>::OpRewritePattern;
LogicalResult matchAndRewrite(vector::ShapeCastOp shapeCastOp, PatternMatchResult matchAndRewrite(vector::ShapeCastOp shapeCastOp,
PatternRewriter &rewriter) const override { PatternRewriter &rewriter) const override {
// Check if 'shapeCastOp' has tuple source/result type. // Check if 'shapeCastOp' has tuple source/result type.
auto sourceTupleType = auto sourceTupleType =
shapeCastOp.source().getType().dyn_cast_or_null<TupleType>(); shapeCastOp.source().getType().dyn_cast_or_null<TupleType>();
auto resultTupleType = auto resultTupleType =
shapeCastOp.result().getType().dyn_cast_or_null<TupleType>(); shapeCastOp.result().getType().dyn_cast_or_null<TupleType>();
if (!sourceTupleType || !resultTupleType) if (!sourceTupleType || !resultTupleType)
return failure(); return matchFailure();
assert(sourceTupleType.size() == resultTupleType.size()); assert(sourceTupleType.size() == resultTupleType.size());
// Create single-vector ShapeCastOp for each source tuple element. // Create single-vector ShapeCastOp for each source tuple element.
Location loc = shapeCastOp.getLoc(); Location loc = shapeCastOp.getLoc();
SmallVector<Value, 8> resultElements; SmallVector<Value, 8> resultElements;
resultElements.reserve(resultTupleType.size()); resultElements.reserve(resultTupleType.size());
for (unsigned i = 0, e = sourceTupleType.size(); i < e; ++i) { for (unsigned i = 0, e = sourceTupleType.size(); i < e; ++i) {
auto sourceElement = rewriter.create<vector::TupleGetOp>( auto sourceElement = rewriter.create<vector::TupleGetOp>(
loc, sourceTupleType.getType(i), shapeCastOp.source(), loc, sourceTupleType.getType(i), shapeCastOp.source(),
rewriter.getI64IntegerAttr(i)); rewriter.getI64IntegerAttr(i));
resultElements.push_back(rewriter.create<vector::ShapeCastOp>( resultElements.push_back(rewriter.create<vector::ShapeCastOp>(
loc, resultTupleType.getType(i), sourceElement)); loc, resultTupleType.getType(i), sourceElement));
} }
// Replace 'shapeCastOp' with tuple of 'resultElements'. // Replace 'shapeCastOp' with tuple of 'resultElements'.
rewriter.replaceOpWithNewOp<vector::TupleOp>(shapeCastOp, resultTupleType, rewriter.replaceOpWithNewOp<vector::TupleOp>(shapeCastOp, resultTupleType,
resultElements); resultElements);
return success(); return matchSuccess();
} }
}; };
/// ShapeCastOpFolder folds cancelling ShapeCastOps away. /// ShapeCastOpFolder folds cancelling ShapeCastOps away.
// //
// Example: // Example:
// //
// The following MLIR with cancelling ShapeCastOps: // The following MLIR with cancelling ShapeCastOps:
// //
// %0 = source : vector<5x4x2xf32> // %0 = source : vector<5x4x2xf32>
// %1 = shape_cast %0 : vector<5x4x2xf32> to vector<20x2xf32> // %1 = shape_cast %0 : vector<5x4x2xf32> to vector<20x2xf32>
// %2 = shape_cast %1 : vector<20x2xf32> to vector<5x4x2xf32> // %2 = shape_cast %1 : vector<20x2xf32> to vector<5x4x2xf32>
// %3 = user %2 : vector<5x4x2xf32> // %3 = user %2 : vector<5x4x2xf32>
// //
// Should canonicalize to the following: // Should canonicalize to the following:
// //
// %0 = source : vector<5x4x2xf32> // %0 = source : vector<5x4x2xf32>
// %1 = user %0 : vector<5x4x2xf32> // %1 = user %0 : vector<5x4x2xf32>
// //
struct ShapeCastOpFolder : public OpRewritePattern<vector::ShapeCastOp> { struct ShapeCastOpFolder : public OpRewritePattern<vector::ShapeCastOp> {
using OpRewritePattern<vector::ShapeCastOp>::OpRewritePattern; using OpRewritePattern<vector::ShapeCastOp>::OpRewritePattern;
LogicalResult matchAndRewrite(vector::ShapeCastOp shapeCastOp, PatternMatchResult matchAndRewrite(vector::ShapeCastOp shapeCastOp,
PatternRewriter &rewriter) const override { PatternRewriter &rewriter) const override {
// Check if 'shapeCastOp' has vector source/result type. // Check if 'shapeCastOp' has vector source/result type.
auto sourceVectorType = auto sourceVectorType =
shapeCastOp.source().getType().dyn_cast_or_null<VectorType>(); shapeCastOp.source().getType().dyn_cast_or_null<VectorType>();
auto resultVectorType = auto resultVectorType =
shapeCastOp.result().getType().dyn_cast_or_null<VectorType>(); shapeCastOp.result().getType().dyn_cast_or_null<VectorType>();
if (!sourceVectorType || !resultVectorType) if (!sourceVectorType || !resultVectorType)
return failure(); return matchFailure();
// Check if shape cast op source operand is also a shape cast op. // Check if shape cast op source operand is also a shape cast op.
auto sourceShapeCastOp = dyn_cast_or_null<vector::ShapeCastOp>( auto sourceShapeCastOp = dyn_cast_or_null<vector::ShapeCastOp>(
shapeCastOp.source().getDefiningOp()); shapeCastOp.source().getDefiningOp());
if (!sourceShapeCastOp) if (!sourceShapeCastOp)
return failure(); return matchFailure();
auto operandSourceVectorType = auto operandSourceVectorType =
sourceShapeCastOp.source().getType().cast<VectorType>(); sourceShapeCastOp.source().getType().cast<VectorType>();
auto operandResultVectorType = auto operandResultVectorType =
sourceShapeCastOp.result().getType().cast<VectorType>(); sourceShapeCastOp.result().getType().cast<VectorType>();
// Check if shape cast operations invert each other. // Check if shape cast operations invert each other.
if (operandSourceVectorType != resultVectorType || if (operandSourceVectorType != resultVectorType ||
operandResultVectorType != sourceVectorType) operandResultVectorType != sourceVectorType)
return failure(); return matchFailure();
rewriter.replaceOp(shapeCastOp, sourceShapeCastOp.source()); rewriter.replaceOp(shapeCastOp, sourceShapeCastOp.source());
return success(); return matchSuccess();
} }
}; };
// Patter rewrite which forward tuple elements to their users. // Patter rewrite which forward tuple elements to their users.
// User(TupleGetOp(ExtractSlicesOp(InsertSlicesOp(TupleOp(Producer))))) // User(TupleGetOp(ExtractSlicesOp(InsertSlicesOp(TupleOp(Producer)))))
// -> User(Producer) // -> User(Producer)
struct TupleGetFolderOp : public OpRewritePattern<vector::TupleGetOp> { struct TupleGetFolderOp : public OpRewritePattern<vector::TupleGetOp> {
using OpRewritePattern<vector::TupleGetOp>::OpRewritePattern; using OpRewritePattern<vector::TupleGetOp>::OpRewritePattern;
LogicalResult matchAndRewrite(vector::TupleGetOp tupleGetOp, PatternMatchResult matchAndRewrite(vector::TupleGetOp tupleGetOp,
PatternRewriter &rewriter) const override { PatternRewriter &rewriter) const override {
// Return if 'tupleGetOp.vectors' arg was not defined by ExtractSlicesOp. // Return if 'tupleGetOp.vectors' arg was not defined by ExtractSlicesOp.
auto extractSlicesOp = dyn_cast_or_null<vector::ExtractSlicesOp>( auto extractSlicesOp = dyn_cast_or_null<vector::ExtractSlicesOp>(
tupleGetOp.vectors().getDefiningOp()); tupleGetOp.vectors().getDefiningOp());
if (!extractSlicesOp) if (!extractSlicesOp)
return failure(); return matchFailure();
// Return if 'extractSlicesOp.vector' arg was not defined by InsertSlicesOp. // Return if 'extractSlicesOp.vector' arg was not defined by InsertSlicesOp.
auto insertSlicesOp = dyn_cast_or_null<vector::InsertSlicesOp>( auto insertSlicesOp = dyn_cast_or_null<vector::InsertSlicesOp>(
extractSlicesOp.vector().getDefiningOp()); extractSlicesOp.vector().getDefiningOp());
if (!insertSlicesOp) if (!insertSlicesOp)
return failure(); return matchFailure();
// Return if 'insertSlicesOp.vectors' arg was not defined by TupleOp. // Return if 'insertSlicesOp.vectors' arg was not defined by TupleOp.
auto tupleOp = dyn_cast_or_null<vector::TupleOp>( auto tupleOp = dyn_cast_or_null<vector::TupleOp>(
insertSlicesOp.vectors().getDefiningOp()); insertSlicesOp.vectors().getDefiningOp());
if (!tupleOp) if (!tupleOp)
return failure(); return matchFailure();
// Forward Value from 'tupleOp' at 'tupleGetOp.index'. // Forward Value from 'tupleOp' at 'tupleGetOp.index'.
Value tupleValue = tupleOp.getOperand(tupleGetOp.getIndex()); Value tupleValue = tupleOp.getOperand(tupleGetOp.getIndex());
rewriter.replaceOp(tupleGetOp, tupleValue); rewriter.replaceOp(tupleGetOp, tupleValue);
return success(); return matchSuccess();
} }
}; };
/// Progressive lowering of ExtractSlicesOp to tuple of StridedSliceOp. /// Progressive lowering of ExtractSlicesOp to tuple of StridedSliceOp.
/// One: /// One:
/// %x = vector.extract_slices %0 /// %x = vector.extract_slices %0
/// is replaced by: /// is replaced by:
/// %a = vector.strided_slice %0 /// %a = vector.strided_slice %0
/// %b = vector.strided_slice %0 /// %b = vector.strided_slice %0
/// .. /// ..
/// %x = vector.tuple %a, %b, .. /// %x = vector.tuple %a, %b, ..
class ExtractSlicesOpLowering class ExtractSlicesOpLowering
: public OpRewritePattern<vector::ExtractSlicesOp> { : public OpRewritePattern<vector::ExtractSlicesOp> {
public: public:
using OpRewritePattern<vector::ExtractSlicesOp>::OpRewritePattern; using OpRewritePattern<vector::ExtractSlicesOp>::OpRewritePattern;
LogicalResult matchAndRewrite(vector::ExtractSlicesOp op, PatternMatchResult matchAndRewrite(vector::ExtractSlicesOp op,
PatternRewriter &rewriter) const override { PatternRewriter &rewriter) const override {
auto loc = op.getLoc(); auto loc = op.getLoc();
VectorType vectorType = op.getSourceVectorType(); VectorType vectorType = op.getSourceVectorType();
auto shape = vectorType.getShape(); auto shape = vectorType.getShape();
SmallVector<int64_t, 4> sizes; SmallVector<int64_t, 4> sizes;
op.getSizes(sizes); op.getSizes(sizes);
SmallVector<int64_t, 4> strides; SmallVector<int64_t, 4> strides;
Show All 10 Lines for (int64_t i = 0; i < tupleSize; ++i) {
computeElementOffsetsFromVectorSliceOffsets(sizes, vectorOffsets); computeElementOffsetsFromVectorSliceOffsets(sizes, vectorOffsets);
auto sliceSizes = computeSliceSizes(shape, sizes, elementOffsets); auto sliceSizes = computeSliceSizes(shape, sizes, elementOffsets);
// Insert in tuple. // Insert in tuple.
tupleValues[i] = rewriter.create<vector::StridedSliceOp>( tupleValues[i] = rewriter.create<vector::StridedSliceOp>(
loc, op.vector(), elementOffsets, sliceSizes, strides); loc, op.vector(), elementOffsets, sliceSizes, strides);
} }
rewriter.replaceOpWithNewOp<vector::TupleOp>(op, tupleType, tupleValues); rewriter.replaceOpWithNewOp<vector::TupleOp>(op, tupleType, tupleValues);
return success(); return matchSuccess();
} }
}; };
/// Progressive lowering of InsertSlicesOp to series of InsertStridedSliceOp. /// Progressive lowering of InsertSlicesOp to series of InsertStridedSliceOp.
/// One: /// One:
/// %x = vector.insert_slices %0 /// %x = vector.insert_slices %0
/// is replaced by: /// is replaced by:
/// %r0 = vector.splat 0 /// %r0 = vector.splat 0
// %t1 = vector.tuple_get %0, 0 // %t1 = vector.tuple_get %0, 0
/// %r1 = vector.insert_strided_slice %r0, %t1 /// %r1 = vector.insert_strided_slice %r0, %t1
// %t2 = vector.tuple_get %0, 1 // %t2 = vector.tuple_get %0, 1
/// %r2 = vector.insert_strided_slice %r1, %t2 /// %r2 = vector.insert_strided_slice %r1, %t2
/// .. /// ..
/// %x = .. /// %x = ..
class InsertSlicesOpLowering : public OpRewritePattern<vector::InsertSlicesOp> { class InsertSlicesOpLowering : public OpRewritePattern<vector::InsertSlicesOp> {
public: public:
using OpRewritePattern<vector::InsertSlicesOp>::OpRewritePattern; using OpRewritePattern<vector::InsertSlicesOp>::OpRewritePattern;
LogicalResult matchAndRewrite(vector::InsertSlicesOp op, PatternMatchResult matchAndRewrite(vector::InsertSlicesOp op,
PatternRewriter &rewriter) const override { PatternRewriter &rewriter) const override {
auto loc = op.getLoc(); auto loc = op.getLoc();
VectorType vectorType = op.getResultVectorType(); VectorType vectorType = op.getResultVectorType();
auto shape = vectorType.getShape(); auto shape = vectorType.getShape();
SmallVector<int64_t, 4> sizes; SmallVector<int64_t, 4> sizes;
op.getSizes(sizes); op.getSizes(sizes);
SmallVector<int64_t, 4> strides; SmallVector<int64_t, 4> strides;
Show All 17 Lines for (int64_t i = 0; i < tupleSize; ++i) {
auto index = rewriter.getI64IntegerAttr(i); auto index = rewriter.getI64IntegerAttr(i);
auto tupleGet = rewriter.create<vector::TupleGetOp>( auto tupleGet = rewriter.create<vector::TupleGetOp>(
loc, tupleType.getType(i), op.getOperand(), index); loc, tupleType.getType(i), op.getOperand(), index);
result = rewriter.create<vector::InsertStridedSliceOp>( result = rewriter.create<vector::InsertStridedSliceOp>(
loc, tupleGet, result, elementOffsets, strides); loc, tupleGet, result, elementOffsets, strides);
} }
rewriter.replaceOp(op, result); rewriter.replaceOp(op, result);
return success(); return matchSuccess();
} }
}; };
/// Progressive lowering of OuterProductOp. /// Progressive lowering of OuterProductOp.
/// One: /// One:
/// %x = vector.outerproduct %lhs, %rhs, %acc /// %x = vector.outerproduct %lhs, %rhs, %acc
/// is replaced by: /// is replaced by:
/// %z = zero-result /// %z = zero-result
/// %0 = vector.extract %lhs[0] /// %0 = vector.extract %lhs[0]
/// %1 = vector.broadcast %0 /// %1 = vector.broadcast %0
/// %2 = vector.extract %acc[0] /// %2 = vector.extract %acc[0]
/// %3 = vector.fma %1, %arg1, %2 /// %3 = vector.fma %1, %arg1, %2
/// %4 = vector.insert %3, %z[0] /// %4 = vector.insert %3, %z[0]
/// .. /// ..
/// %x = vector.insert %.., %..[N-1] /// %x = vector.insert %.., %..[N-1]
/// ///
class OuterProductOpLowering : public OpRewritePattern<vector::OuterProductOp> { class OuterProductOpLowering : public OpRewritePattern<vector::OuterProductOp> {
public: public:
using OpRewritePattern<vector::OuterProductOp>::OpRewritePattern; using OpRewritePattern<vector::OuterProductOp>::OpRewritePattern;
LogicalResult matchAndRewrite(vector::OuterProductOp op, PatternMatchResult matchAndRewrite(vector::OuterProductOp op,
PatternRewriter &rewriter) const override { PatternRewriter &rewriter) const override {
auto loc = op.getLoc(); auto loc = op.getLoc();
VectorType rhsType = op.getOperandVectorTypeRHS(); VectorType rhsType = op.getOperandVectorTypeRHS();
VectorType resType = op.getVectorType(); VectorType resType = op.getVectorType();
Type eltType = resType.getElementType(); Type eltType = resType.getElementType();
Value acc = (op.acc().empty()) ? nullptr : op.acc()[0]; Value acc = (op.acc().empty()) ? nullptr : op.acc()[0];
Value zero = rewriter.create<ConstantOp>(loc, eltType, Value zero = rewriter.create<ConstantOp>(loc, eltType,
rewriter.getZeroAttr(eltType)); rewriter.getZeroAttr(eltType));
Value result = rewriter.create<SplatOp>(loc, resType, zero); Value result = rewriter.create<SplatOp>(loc, resType, zero);
for (int64_t d = 0, e = resType.getDimSize(0); d < e; ++d) { for (int64_t d = 0, e = resType.getDimSize(0); d < e; ++d) {
auto pos = rewriter.getI64ArrayAttr(d); auto pos = rewriter.getI64ArrayAttr(d);
Value x = rewriter.create<vector::ExtractOp>(loc, eltType, op.lhs(), pos); Value x = rewriter.create<vector::ExtractOp>(loc, eltType, op.lhs(), pos);
Value b = rewriter.create<vector::BroadcastOp>(loc, rhsType, x); Value b = rewriter.create<vector::BroadcastOp>(loc, rhsType, x);
Value m; Value m;
if (acc) { if (acc) {
Value z = rewriter.create<vector::ExtractOp>(loc, rhsType, acc, pos); Value z = rewriter.create<vector::ExtractOp>(loc, rhsType, acc, pos);
m = rewriter.create<vector::FMAOp>(loc, b, op.rhs(), z); m = rewriter.create<vector::FMAOp>(loc, b, op.rhs(), z);
} else { } else {
m = rewriter.create<MulFOp>(loc, b, op.rhs()); m = rewriter.create<MulFOp>(loc, b, op.rhs());
} }
result = rewriter.create<vector::InsertOp>(loc, resType, m, result, pos); result = rewriter.create<vector::InsertOp>(loc, resType, m, result, pos);
} }
rewriter.replaceOp(op, result); rewriter.replaceOp(op, result);
return success(); return matchSuccess();
} }
}; };
/// Progressive lowering of ContractionOp. /// Progressive lowering of ContractionOp.
/// One: /// One:
/// %x = vector.contract with at least one free/batch dimension /// %x = vector.contract with at least one free/batch dimension
/// is replaced by: /// is replaced by:
/// %a = vector.contract with one less free/batch dimension /// %a = vector.contract with one less free/batch dimension
/// %b = vector.contract with one less free/batch dimension /// %b = vector.contract with one less free/batch dimension
/// .. /// ..
/// %x = combine %a %b .. /// %x = combine %a %b ..
/// until a pure contraction is reached (no free/batch dimensions), /// until a pure contraction is reached (no free/batch dimensions),
/// which is replaced by a fma/reduction op. /// which is replaced by a fma/reduction op.
/// ///
/// TODO(ajcbik): break down into transpose/reshape/cast ops /// TODO(ajcbik): break down into transpose/reshape/cast ops
/// when they become available to avoid code dup /// when they become available to avoid code dup
/// TODO(ajcbik): investigate lowering order impact on performance /// TODO(ajcbik): investigate lowering order impact on performance
class ContractionOpLowering : public OpRewritePattern<vector::ContractionOp> { class ContractionOpLowering : public OpRewritePattern<vector::ContractionOp> {
public: public:
using OpRewritePattern<vector::ContractionOp>::OpRewritePattern; using OpRewritePattern<vector::ContractionOp>::OpRewritePattern;
ContractionOpLowering(vector::VectorTransformsOptions vectorTransformsOptions, PatternMatchResult matchAndRewrite(vector::ContractionOp op,
MLIRContext *context)
: OpRewritePattern<vector::ContractionOp>(context),
vectorTransformsOptions(vectorTransformsOptions) {}
LogicalResult matchAndRewrite(vector::ContractionOp op,
PatternRewriter &rewriter) const override { PatternRewriter &rewriter) const override {
// TODO(ajcbik): implement masks // TODO(ajcbik): implement masks
if (llvm::size(op.masks()) != 0) if (llvm::size(op.masks()) != 0)
return failure(); return matchFailure();
// TODO(ntv, ajcbik): implement benefits, cost models, separate this out in // TODO(ntv, ajcbik): implement benefits, cost models, separate this out in
// a new pattern. // a new pattern.
// TODO(ntv, fhahn): once row-major mode is available in LLVM's matrix // TODO(ntv, fhahn): once row-major mode is available in LLVM's matrix
// intrinsics, use that. // intrinsics, use that.
if (vectorTransformsOptions.lowerToLLVMMatrixIntrinsics && if (lowerToLLVMMatrixIntrinsics &&
isColumnMajorMatmul(op.indexing_maps())) { isColumnMajorMatmul(op.indexing_maps())) {
VectorType lhsType = op.getLhsType(); VectorType lhsType = op.getLhsType();
VectorType rhsType = op.getRhsType(); VectorType rhsType = op.getRhsType();
unsigned lhsRows = op.getLhsType().getShape()[0];
unsigned lhsColumns = op.getLhsType().getShape()[1];
unsigned rhsColumns = op.getRhsType().getShape()[1];
// In cases where matrices are degenerate, scalarization issues occur in
// the backend. Avoid all LLVM scalarization issues for now.
// For more details, see: https://bugs.llvm.org/show_bug.cgi?id=45227 and
// https://bugs.llvm.org/show_bug.cgi?id=45229
// TODO(ntv, fhahn): Relax once above bugs are fixed.
if (lhsRows != 1 && lhsColumns != 1 && rhsColumns != 1) {
Type flattenedLHSType = Type flattenedLHSType =
VectorType::get(lhsType.getNumElements(), lhsType.getElementType()); VectorType::get(lhsType.getNumElements(), lhsType.getElementType());
Type flattenedRHSType = Type flattenedRHSType =
VectorType::get(rhsType.getNumElements(), rhsType.getElementType()); VectorType::get(rhsType.getNumElements(), rhsType.getElementType());
auto lhs = rewriter.create<vector::ShapeCastOp>( auto lhs = rewriter.create<vector::ShapeCastOp>(
op.getLoc(), flattenedLHSType, op.lhs()); op.getLoc(), flattenedLHSType, op.lhs());
auto rhs = rewriter.create<vector::ShapeCastOp>( auto rhs = rewriter.create<vector::ShapeCastOp>(
op.getLoc(), flattenedRHSType, op.rhs()); op.getLoc(), flattenedRHSType, op.rhs());
unsigned lhsRows = op.getLhsType().getShape()[0];
unsigned lhsColumns = op.getLhsType().getShape()[1];
unsigned rhsColumns = op.getRhsType().getShape()[1];
Value mul = rewriter.create<vector::MatmulOp>( Value mul = rewriter.create<vector::MatmulOp>(
op.getLoc(), lhs, rhs, lhsRows, lhsColumns, rhsColumns); op.getLoc(), lhs, rhs, lhsRows, lhsColumns, rhsColumns);
mul = rewriter.create<vector::ShapeCastOp>(op.getLoc(), mul = rewriter.create<vector::ShapeCastOp>(op.getLoc(),
op.acc().getType(), mul); op.acc().getType(), mul);
Type elementType = op.getLhsType().getElementType(); Type elementType = op.getLhsType().getElementType();
assert(elementType.isIntOrFloat()); assert(elementType.isIntOrFloat());
if (elementType.isa<IntegerType>()) if (elementType.isa<IntegerType>())
rewriter.replaceOpWithNewOp<AddIOp>(op, op.acc(), mul); rewriter.replaceOpWithNewOp<AddIOp>(op, op.acc(), mul);
else else
rewriter.replaceOpWithNewOp<AddFOp>(op, op.acc(), mul); rewriter.replaceOpWithNewOp<AddFOp>(op, op.acc(), mul);
return success(); return matchSuccess();
}
} }
// Find first batch dimension in LHS/RHS, and lower when found. // Find first batch dimension in LHS/RHS, and lower when found.
std::vector<std::pair<int64_t, int64_t>> batchDimMap = op.getBatchDimMap(); std::vector<std::pair<int64_t, int64_t>> batchDimMap = op.getBatchDimMap();
if (!batchDimMap.empty()) { if (!batchDimMap.empty()) {
int64_t lhsIndex = batchDimMap[0].first; int64_t lhsIndex = batchDimMap[0].first;
int64_t rhsIndex = batchDimMap[0].second; int64_t rhsIndex = batchDimMap[0].second;
rewriter.replaceOp(op, lowerParallel(op, lhsIndex, rhsIndex, rewriter)); rewriter.replaceOp(op, lowerParallel(op, lhsIndex, rhsIndex, rewriter));
return success(); return matchSuccess();
} }
// Collect contracting dimensions. // Collect contracting dimensions.
std::vector<std::pair<int64_t, int64_t>> contractingDimMap = std::vector<std::pair<int64_t, int64_t>> contractingDimMap =
op.getContractingDimMap(); op.getContractingDimMap();
DenseSet<int64_t> lhsContractingDimSet; DenseSet<int64_t> lhsContractingDimSet;
DenseSet<int64_t> rhsContractingDimSet; DenseSet<int64_t> rhsContractingDimSet;
for (auto &dimPair : contractingDimMap) { for (auto &dimPair : contractingDimMap) {
lhsContractingDimSet.insert(dimPair.first); lhsContractingDimSet.insert(dimPair.first);
rhsContractingDimSet.insert(dimPair.second); rhsContractingDimSet.insert(dimPair.second);
} }
// Find first free dimension in LHS, and lower when found. // Find first free dimension in LHS, and lower when found.
VectorType lhsType = op.getLhsType(); VectorType lhsType = op.getLhsType();
for (int64_t lhsIndex = 0, e = lhsType.getRank(); lhsIndex < e; for (int64_t lhsIndex = 0, e = lhsType.getRank(); lhsIndex < e;
++lhsIndex) { ++lhsIndex) {
if (lhsContractingDimSet.count(lhsIndex) == 0) { if (lhsContractingDimSet.count(lhsIndex) == 0) {
rewriter.replaceOp( rewriter.replaceOp(
op, lowerParallel(op, lhsIndex, /*rhsIndex=*/-1, rewriter)); op, lowerParallel(op, lhsIndex, /*rhsIndex=*/-1, rewriter));
return success(); return matchSuccess();
} }
} }
// Find first free dimension in RHS, and lower when found. // Find first free dimension in RHS, and lower when found.
VectorType rhsType = op.getRhsType(); VectorType rhsType = op.getRhsType();
for (int64_t rhsIndex = 0, e = rhsType.getRank(); rhsIndex < e; for (int64_t rhsIndex = 0, e = rhsType.getRank(); rhsIndex < e;
++rhsIndex) { ++rhsIndex) {
if (rhsContractingDimSet.count(rhsIndex) == 0) { if (rhsContractingDimSet.count(rhsIndex) == 0) {
rewriter.replaceOp( rewriter.replaceOp(
op, lowerParallel(op, /*lhsIndex=*/-1, rhsIndex, rewriter)); op, lowerParallel(op, /*lhsIndex=*/-1, rhsIndex, rewriter));
return success(); return matchSuccess();
} }
} }
// Lower the first remaining reduction dimension. // Lower the first remaining reduction dimension.
if (!contractingDimMap.empty()) { if (!contractingDimMap.empty()) {
rewriter.replaceOp(op, lowerReduction(op, rewriter)); rewriter.replaceOp(op, lowerReduction(op, rewriter));
return success(); return matchSuccess();
} }
return failure(); return matchFailure();
} }
private: private:
// Lower one parallel dimension. // Lower one parallel dimension.
// TODO(ajcbik): consider reusing existing contract unrolling // TODO(ajcbik): consider reusing existing contract unrolling
Value lowerParallel(vector::ContractionOp op, int64_t lhsIndex, Value lowerParallel(vector::ContractionOp op, int64_t lhsIndex,
int64_t rhsIndex, PatternRewriter &rewriter) const { int64_t rhsIndex, PatternRewriter &rewriter) const {
VectorType lhsType = op.getLhsType(); VectorType lhsType = op.getLhsType();
▲ Show 20 Lines • Show All 216 Lines • ▼ Show 20 Lines for (int64_t d = 0, e = type.getDimSize(0); d < e; d++) {
Value ins = Value ins =
rewriter.create<vector::ExtractOp>(loc, insType, val, posAttr); rewriter.create<vector::ExtractOp>(loc, insType, val, posAttr);
Value sto = reshapeStore(loc, ins, ext, vType, index - 1, pos, rewriter); Value sto = reshapeStore(loc, ins, ext, vType, index - 1, pos, rewriter);
result = result =
rewriter.create<vector::InsertOp>(loc, type, sto, result, posAttr); rewriter.create<vector::InsertOp>(loc, type, sto, result, posAttr);
} }
return result; return result;
} }
vector::VectorTransformsOptions vectorTransformsOptions;
}; };
/// ShapeOp 2D -> 1D downcast serves the purpose of flattening 2-D to 1-D /// ShapeOp 2D -> 1D downcast serves the purpose of flattening 2-D to 1-D
/// vectors progressively on the way to target llvm.matrix intrinsics. /// vectors progressively on the way to target llvm.matrix intrinsics.
/// This iterates over the most major dimension of the 2-D vector and performs /// This iterates over the most major dimension of the 2-D vector and performs
/// rewrites into: /// rewrites into:
/// vector.extract from 2-D + vector.insert_strided_slice offset into 1-D /// vector.extract from 2-D + vector.insert_strided_slice offset into 1-D
class ShapeCastOp2DDownCastRewritePattern class ShapeCastOp2DDownCastRewritePattern
: public OpRewritePattern<vector::ShapeCastOp> { : public OpRewritePattern<vector::ShapeCastOp> {
public: public:
using OpRewritePattern<vector::ShapeCastOp>::OpRewritePattern; using OpRewritePattern<vector::ShapeCastOp>::OpRewritePattern;
LogicalResult matchAndRewrite(vector::ShapeCastOp op, PatternMatchResult matchAndRewrite(vector::ShapeCastOp op,
PatternRewriter &rewriter) const override { PatternRewriter &rewriter) const override {
auto sourceVectorType = op.getSourceVectorType(); auto sourceVectorType = op.getSourceVectorType();
auto resultVectorType = op.getResultVectorType(); auto resultVectorType = op.getResultVectorType();
if (sourceVectorType.getRank() != 2 || resultVectorType.getRank() != 1) if (sourceVectorType.getRank() != 2 || resultVectorType.getRank() != 1)
return failure(); return matchFailure();
auto loc = op.getLoc(); auto loc = op.getLoc();
auto elemType = sourceVectorType.getElementType(); auto elemType = sourceVectorType.getElementType();
Value zero = rewriter.create<ConstantOp>(loc, elemType, Value zero = rewriter.create<ConstantOp>(loc, elemType,
rewriter.getZeroAttr(elemType)); rewriter.getZeroAttr(elemType));
Value desc = rewriter.create<SplatOp>(loc, resultVectorType, zero); Value desc = rewriter.create<SplatOp>(loc, resultVectorType, zero);
unsigned mostMinorVectorSize = sourceVectorType.getShape()[1]; unsigned mostMinorVectorSize = sourceVectorType.getShape()[1];
for (int64_t i = 0, e = sourceVectorType.getShape().front(); i != e; ++i) { for (int64_t i = 0, e = sourceVectorType.getShape().front(); i != e; ++i) {
Value vec = rewriter.create<vector::ExtractOp>(loc, op.source(), i); Value vec = rewriter.create<vector::ExtractOp>(loc, op.source(), i);
desc = rewriter.create<vector::InsertStridedSliceOp>( desc = rewriter.create<vector::InsertStridedSliceOp>(
loc, vec, desc, loc, vec, desc,
/*offsets=*/i * mostMinorVectorSize, /*strides=*/1); /*offsets=*/i * mostMinorVectorSize, /*strides=*/1);
} }
rewriter.replaceOp(op, desc); rewriter.replaceOp(op, desc);
return success(); return matchSuccess();
} }
}; };
/// ShapeOp 1D -> 2D upcast serves the purpose of unflattening 2-D from 1-D /// ShapeOp 1D -> 2D upcast serves the purpose of unflattening 2-D from 1-D
/// vectors progressively on the way from targeting llvm.matrix intrinsics. /// vectors progressively on the way from targeting llvm.matrix intrinsics.
/// This iterates over the most major dimension of the 2-D vector and performs /// This iterates over the most major dimension of the 2-D vector and performs
/// rewrites into: /// rewrites into:
/// vector.strided_slice from 1-D + vector.insert into 2-D /// vector.strided_slice from 1-D + vector.insert into 2-D
class ShapeCastOp2DUpCastRewritePattern class ShapeCastOp2DUpCastRewritePattern
: public OpRewritePattern<vector::ShapeCastOp> { : public OpRewritePattern<vector::ShapeCastOp> {
public: public:
using OpRewritePattern<vector::ShapeCastOp>::OpRewritePattern; using OpRewritePattern<vector::ShapeCastOp>::OpRewritePattern;
LogicalResult matchAndRewrite(vector::ShapeCastOp op, PatternMatchResult matchAndRewrite(vector::ShapeCastOp op,
PatternRewriter &rewriter) const override { PatternRewriter &rewriter) const override {
auto sourceVectorType = op.getSourceVectorType(); auto sourceVectorType = op.getSourceVectorType();
auto resultVectorType = op.getResultVectorType(); auto resultVectorType = op.getResultVectorType();
if (sourceVectorType.getRank() != 1 || resultVectorType.getRank() != 2) if (sourceVectorType.getRank() != 1 || resultVectorType.getRank() != 2)
return failure(); return matchFailure();
auto loc = op.getLoc(); auto loc = op.getLoc();
auto elemType = sourceVectorType.getElementType(); auto elemType = sourceVectorType.getElementType();
Value zero = rewriter.create<ConstantOp>(loc, elemType, Value zero = rewriter.create<ConstantOp>(loc, elemType,
rewriter.getZeroAttr(elemType)); rewriter.getZeroAttr(elemType));
Value desc = rewriter.create<SplatOp>(loc, resultVectorType, zero); Value desc = rewriter.create<SplatOp>(loc, resultVectorType, zero);
unsigned mostMinorVectorSize = resultVectorType.getShape()[1]; unsigned mostMinorVectorSize = resultVectorType.getShape()[1];
for (int64_t i = 0, e = resultVectorType.getShape().front(); i != e; ++i) { for (int64_t i = 0, e = resultVectorType.getShape().front(); i != e; ++i) {
Value vec = rewriter.create<vector::StridedSliceOp>( Value vec = rewriter.create<vector::StridedSliceOp>(
loc, op.source(), /*offsets=*/i * mostMinorVectorSize, loc, op.source(), /*offsets=*/i * mostMinorVectorSize,
/*sizes=*/mostMinorVectorSize, /*sizes=*/mostMinorVectorSize,
/*strides=*/1); /*strides=*/1);
desc = rewriter.create<vector::InsertOp>(loc, vec, desc, i); desc = rewriter.create<vector::InsertOp>(loc, vec, desc, i);
} }
rewriter.replaceOp(op, desc); rewriter.replaceOp(op, desc);
return success(); return matchSuccess();
} }
}; };
} // namespace } // namespace
// TODO(andydavis) Add pattern to rewrite ExtractSlices(ConstantMaskOp). // TODO(andydavis) Add pattern to rewrite ExtractSlices(ConstantMaskOp).
// TODO(andydavis) Add this as DRR pattern. // TODO(andydavis) Add this as DRR pattern.
void mlir::vector::populateVectorToVectorTransformationPatterns( void mlir::vector::populateVectorToVectorTransformationPatterns(
OwningRewritePatternList &patterns, MLIRContext *context) { OwningRewritePatternList &patterns, MLIRContext *context) {
patterns.insert<ShapeCastOpDecomposer, ShapeCastOpFolder, SplitTransferReadOp, patterns.insert<ShapeCastOpDecomposer, ShapeCastOpFolder, SplitTransferReadOp,
SplitTransferWriteOp, TupleGetFolderOp>(context); SplitTransferWriteOp, TupleGetFolderOp>(context);
} }
void mlir::vector::populateVectorSlicesLoweringPatterns( void mlir::vector::populateVectorSlicesLoweringPatterns(
OwningRewritePatternList &patterns, MLIRContext *context) { OwningRewritePatternList &patterns, MLIRContext *context) {
patterns.insert<ExtractSlicesOpLowering, InsertSlicesOpLowering>(context); patterns.insert<ExtractSlicesOpLowering, InsertSlicesOpLowering>(context);
} }
void mlir::vector::populateVectorContractLoweringPatterns( void mlir::vector::populateVectorContractLoweringPatterns(
OwningRewritePatternList &patterns, MLIRContext *context, OwningRewritePatternList &patterns, MLIRContext *context) {
VectorTransformsOptions parameters) { patterns.insert<ContractionOpLowering, ShapeCastOp2DDownCastRewritePattern,
patterns.insert<ShapeCastOp2DDownCastRewritePattern,
ShapeCastOp2DUpCastRewritePattern, OuterProductOpLowering>( ShapeCastOp2DUpCastRewritePattern, OuterProductOpLowering>(
context); context);
patterns.insert<ContractionOpLowering>(parameters, context);
} }

mlir/lib/Dialect/VectorOps/VectorUtils.cpp

  • This file was copied from mlir/lib/Dialect/Vector/VectorUtils.cpp.
//===- VectorUtils.cpp - MLIR Utilities for VectorOps ------------------===// //===- VectorUtils.cpp - MLIR Utilities for VectorOps ------------------===//
// //
// Part of the MLIR Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the MLIR Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements utility methods for working with the Vector dialect. // This file implements utility methods for working with the VectorOps dialect.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/Vector/VectorUtils.h" #include "mlir/Dialect/VectorOps/VectorUtils.h"
#include "mlir/Analysis/LoopAnalysis.h" #include "mlir/Analysis/LoopAnalysis.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Vector/VectorOps.h" #include "mlir/Dialect/VectorOps/VectorOps.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/IntegerSet.h" #include "mlir/IR/IntegerSet.h"
#include "mlir/IR/Operation.h" #include "mlir/IR/Operation.h"
#include "mlir/Support/Functional.h" #include "mlir/Support/Functional.h"
#include "mlir/Support/LLVM.h" #include "mlir/Support/LLVM.h"
#include "mlir/Support/MathExtras.h" #include "mlir/Support/MathExtras.h"
#include "mlir/Support/STLExtras.h" #include "mlir/Support/STLExtras.h"
▲ Show 20 Lines • Show All 253 Lines • Show Last 20 Lines

mlir/lib/Transforms/AffineDataCopyGeneration.cpp

  • This file was moved to mlir/lib/Dialect/Affine/Transforms/AffineDataCopyGeneration.cpp.

mlir/lib/Transforms/AffineLoopInvariantCodeMotion.cpp

  • This file was moved to mlir/lib/Dialect/Affine/Transforms/AffineLoopInvariantCodeMotion.cpp.

mlir/lib/Transforms/CMakeLists.txt

add_subdirectory(Utils) add_subdirectory(Utils)
add_mlir_library(MLIRTransforms add_mlir_library(MLIRTransforms
AffineDataCopyGeneration.cpp
AffineLoopInvariantCodeMotion.cpp
Canonicalizer.cpp Canonicalizer.cpp
CSE.cpp CSE.cpp
DialectConversion.cpp DialectConversion.cpp
Inliner.cpp Inliner.cpp
LocationSnapshot.cpp LocationSnapshot.cpp
LoopCoalescing.cpp LoopCoalescing.cpp
LoopFusion.cpp LoopFusion.cpp
LoopInvariantCodeMotion.cpp LoopInvariantCodeMotion.cpp
LoopTiling.cpp LoopTiling.cpp
LoopUnrollAndJam.cpp LoopUnrollAndJam.cpp
LoopUnroll.cpp LoopUnroll.cpp
MemRefDataFlowOpt.cpp MemRefDataFlowOpt.cpp
OpStats.cpp OpStats.cpp
PipelineDataTransfer.cpp PipelineDataTransfer.cpp
SimplifyAffineStructures.cpp
StripDebugInfo.cpp StripDebugInfo.cpp
SymbolDCE.cpp SymbolDCE.cpp
Vectorize.cpp Vectorize.cpp
ViewOpGraph.cpp ViewOpGraph.cpp
ViewRegionGraph.cpp ViewRegionGraph.cpp
ADDITIONAL_HEADER_DIRS ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Transforms ${MLIR_MAIN_INCLUDE_DIR}/mlir/Transforms
DEPENDS DEPENDS
MLIRStandardOpsIncGen MLIRStandardOpsIncGen
) )
target_link_libraries(MLIRTransforms target_link_libraries(MLIRTransforms
PUBLIC PUBLIC
MLIRAffineOps MLIRAffine
MLIRAnalysis MLIRAnalysis
MLIRLoopLikeInterface MLIRLoopLikeInterface
MLIRLoopOps MLIRLoopOps
MLIRPass MLIRPass
MLIRTransformUtils MLIRTransformUtils
MLIRVector MLIRVector
) )

mlir/lib/Transforms/LoopFusion.cpp

//===- LoopFusion.cpp - Code to perform loop fusion -----------------------===// //===- LoopFusion.cpp - Code to perform loop fusion -----------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements loop fusion. // This file implements loop fusion.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/AffineAnalysis.h" #include "mlir/Analysis/AffineAnalysis.h"
#include "mlir/Analysis/AffineStructures.h" #include "mlir/Analysis/AffineStructures.h"
#include "mlir/Analysis/LoopAnalysis.h" #include "mlir/Analysis/LoopAnalysis.h"
#include "mlir/Analysis/Utils.h" #include "mlir/Analysis/Utils.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/AffineMap.h" #include "mlir/IR/AffineMap.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/LoopFusionUtils.h" #include "mlir/Transforms/LoopFusionUtils.h"
#include "mlir/Transforms/LoopUtils.h" #include "mlir/Transforms/LoopUtils.h"
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
#include "mlir/Transforms/Utils.h" #include "mlir/Transforms/Utils.h"
▲ Show 20 Lines • Show All 1,953 Lines • Show Last 20 Lines

mlir/lib/Transforms/LoopTiling.cpp

//===- LoopTiling.cpp --- Loop tiling pass ------------------------------*-===// //===- LoopTiling.cpp --- Loop tiling pass ------------------------------*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements a pass to tile loop nests. // This file implements a pass to tile loop nests.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/AffineAnalysis.h" #include "mlir/Analysis/AffineAnalysis.h"
#include "mlir/Analysis/AffineStructures.h" #include "mlir/Analysis/AffineStructures.h"
#include "mlir/Analysis/LoopAnalysis.h" #include "mlir/Analysis/LoopAnalysis.h"
#include "mlir/Analysis/Utils.h" #include "mlir/Analysis/Utils.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/LoopUtils.h" #include "mlir/Transforms/LoopUtils.h"
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
#include "mlir/Transforms/Utils.h" #include "mlir/Transforms/Utils.h"
#include "llvm/Support/CommandLine.h" #include "llvm/Support/CommandLine.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
using namespace mlir; using namespace mlir;
▲ Show 20 Lines • Show All 377 Lines • Show Last 20 Lines

mlir/lib/Transforms/LoopUnroll.cpp

//===- LoopUnroll.cpp - Code to perform loop unrolling --------------------===// //===- LoopUnroll.cpp - Code to perform loop unrolling --------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements loop unrolling. // This file implements loop unrolling.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
#include "mlir/Analysis/LoopAnalysis.h" #include "mlir/Analysis/LoopAnalysis.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/AffineMap.h" #include "mlir/IR/AffineMap.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/LoopUtils.h" #include "mlir/Transforms/LoopUtils.h"
#include "llvm/ADT/DenseMap.h" #include "llvm/ADT/DenseMap.h"
#include "llvm/Support/CommandLine.h" #include "llvm/Support/CommandLine.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
▲ Show 20 Lines • Show All 158 Lines • Show Last 20 Lines

mlir/lib/Transforms/LoopUnrollAndJam.cpp

Show All 29 Lines
// S6(i+1); // S6(i+1);
// //
// Note: 'if/else' blocks are not jammed. So, if there are loops inside if // Note: 'if/else' blocks are not jammed. So, if there are loops inside if
// op's, bodies of those loops will not be jammed. // op's, bodies of those loops will not be jammed.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
#include "mlir/Analysis/LoopAnalysis.h" #include "mlir/Analysis/LoopAnalysis.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/AffineMap.h" #include "mlir/IR/AffineMap.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/LoopUtils.h" #include "mlir/Transforms/LoopUtils.h"
#include "llvm/ADT/DenseMap.h" #include "llvm/ADT/DenseMap.h"
#include "llvm/Support/CommandLine.h" #include "llvm/Support/CommandLine.h"
▲ Show 20 Lines • Show All 189 Lines • Show Last 20 Lines

mlir/lib/Transforms/MemRefDataFlowOpt.cpp

Show All 10 Lines
// TODO(mlir-team): In the future, similar techniques could be used to eliminate // TODO(mlir-team): In the future, similar techniques could be used to eliminate
// dead memref store's and perform more complex forwarding when support for // dead memref store's and perform more complex forwarding when support for
// SSA scalars live out of 'affine.for'/'affine.if' statements is available. // SSA scalars live out of 'affine.for'/'affine.if' statements is available.
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/AffineAnalysis.h" #include "mlir/Analysis/AffineAnalysis.h"
#include "mlir/Analysis/Dominance.h" #include "mlir/Analysis/Dominance.h"
#include "mlir/Analysis/Utils.h" #include "mlir/Analysis/Utils.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
#include "llvm/ADT/SmallPtrSet.h" #include "llvm/ADT/SmallPtrSet.h"
#include <algorithm> #include <algorithm>
#define DEBUG_TYPE "memref-dataflow-opt" #define DEBUG_TYPE "memref-dataflow-opt"
▲ Show 20 Lines • Show All 200 Lines • Show Last 20 Lines

mlir/lib/Transforms/PipelineDataTransfer.cpp

Show All 9 Lines
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
#include "mlir/Analysis/AffineAnalysis.h" #include "mlir/Analysis/AffineAnalysis.h"
#include "mlir/Analysis/LoopAnalysis.h" #include "mlir/Analysis/LoopAnalysis.h"
#include "mlir/Analysis/Utils.h" #include "mlir/Analysis/Utils.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/LoopUtils.h" #include "mlir/Transforms/LoopUtils.h"
#include "mlir/Transforms/Utils.h" #include "mlir/Transforms/Utils.h"
#include "llvm/ADT/DenseMap.h" #include "llvm/ADT/DenseMap.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#define DEBUG_TYPE "affine-pipeline-data-transfer" #define DEBUG_TYPE "affine-pipeline-data-transfer"
▲ Show 20 Lines • Show All 347 Lines • Show Last 20 Lines

mlir/lib/Transforms/SimplifyAffineStructures.cpp

  • This file was moved to mlir/lib/Dialect/Affine/Transforms/SimplifyAffineStructures.cpp.

mlir/lib/Transforms/Utils/CMakeLists.txt

Show All 10 Linesadd_mlir_library(MLIRTransformUtils
${MLIR_MAIN_INCLUDE_DIR}/mlir/Transforms ${MLIR_MAIN_INCLUDE_DIR}/mlir/Transforms
DEPENDS DEPENDS
MLIRStandardOpsIncGen MLIRStandardOpsIncGen
) )
target_link_libraries(MLIRTransformUtils target_link_libraries(MLIRTransformUtils
PUBLIC PUBLIC
MLIRAffineOps MLIRAffine
MLIRAnalysis MLIRAnalysis
MLIRLoopAnalysis MLIRLoopAnalysis
MLIRLoopOps MLIRLoopOps
MLIRPass MLIRPass
MLIRStandardOps MLIRStandardOps
) )

mlir/lib/Transforms/Utils/LoopFusionUtils.cpp

Show All 10 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Transforms/LoopFusionUtils.h" #include "mlir/Transforms/LoopFusionUtils.h"
#include "mlir/Analysis/AffineAnalysis.h" #include "mlir/Analysis/AffineAnalysis.h"
#include "mlir/Analysis/AffineStructures.h" #include "mlir/Analysis/AffineStructures.h"
#include "mlir/Analysis/LoopAnalysis.h" #include "mlir/Analysis/LoopAnalysis.h"
#include "mlir/Analysis/Utils.h" #include "mlir/Analysis/Utils.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/AffineMap.h" #include "mlir/IR/AffineMap.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/Function.h" #include "mlir/IR/Function.h"
#include "mlir/IR/Operation.h" #include "mlir/IR/Operation.h"
#include "mlir/Transforms/LoopUtils.h" #include "mlir/Transforms/LoopUtils.h"
#include "llvm/ADT/DenseMap.h" #include "llvm/ADT/DenseMap.h"
▲ Show 20 Lines • Show All 481 Lines • Show Last 20 Lines

mlir/lib/Transforms/Utils/LoopUtils.cpp

Show All 10 Lines
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Transforms/LoopUtils.h" #include "mlir/Transforms/LoopUtils.h"
#include "mlir/Analysis/AffineAnalysis.h" #include "mlir/Analysis/AffineAnalysis.h"
#include "mlir/Analysis/LoopAnalysis.h" #include "mlir/Analysis/LoopAnalysis.h"
#include "mlir/Analysis/SliceAnalysis.h" #include "mlir/Analysis/SliceAnalysis.h"
#include "mlir/Analysis/Utils.h" #include "mlir/Analysis/Utils.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/LoopOps/LoopOps.h" #include "mlir/Dialect/LoopOps/LoopOps.h"
#include "mlir/IR/AffineMap.h" #include "mlir/IR/AffineMap.h"
#include "mlir/IR/BlockAndValueMapping.h" #include "mlir/IR/BlockAndValueMapping.h"
#include "mlir/IR/Function.h" #include "mlir/IR/Function.h"
#include "mlir/Transforms/RegionUtils.h" #include "mlir/Transforms/RegionUtils.h"
#include "mlir/Transforms/Utils.h" #include "mlir/Transforms/Utils.h"
#include "llvm/ADT/DenseMap.h" #include "llvm/ADT/DenseMap.h"
#include "llvm/ADT/MapVector.h" #include "llvm/ADT/MapVector.h"
▲ Show 20 Lines • Show All 1,815 Lines • Show Last 20 Lines

mlir/lib/Transforms/Utils/Utils.cpp

Show All 12 Lines
#include "mlir/Transforms/Utils.h" #include "mlir/Transforms/Utils.h"
#include "mlir/ADT/TypeSwitch.h" #include "mlir/ADT/TypeSwitch.h"
#include "mlir/Analysis/AffineAnalysis.h" #include "mlir/Analysis/AffineAnalysis.h"
#include "mlir/Analysis/AffineStructures.h" #include "mlir/Analysis/AffineStructures.h"
#include "mlir/Analysis/Dominance.h" #include "mlir/Analysis/Dominance.h"
#include "mlir/Analysis/Utils.h" #include "mlir/Analysis/Utils.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/Function.h" #include "mlir/IR/Function.h"
#include "mlir/IR/Module.h" #include "mlir/IR/Module.h"
#include "mlir/Support/MathExtras.h" #include "mlir/Support/MathExtras.h"
#include "llvm/ADT/DenseMap.h" #include "llvm/ADT/DenseMap.h"
using namespace mlir; using namespace mlir;
/// Return true if this operation dereferences one or more memref's. /// Return true if this operation dereferences one or more memref's.
▲ Show 20 Lines • Show All 441 Lines • Show Last 20 Lines

mlir/lib/Transforms/Vectorize.cpp

Show All 9 Lines
// a target-independent, n-D super-vector abstraction. // a target-independent, n-D super-vector abstraction.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/LoopAnalysis.h" #include "mlir/Analysis/LoopAnalysis.h"
#include "mlir/Analysis/NestedMatcher.h" #include "mlir/Analysis/NestedMatcher.h"
#include "mlir/Analysis/SliceAnalysis.h" #include "mlir/Analysis/SliceAnalysis.h"
#include "mlir/Analysis/Utils.h" #include "mlir/Analysis/Utils.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/Dialect/Vector/VectorOps.h" #include "mlir/Dialect/Vector/VectorOps.h"
#include "mlir/Dialect/Vector/VectorUtils.h" #include "mlir/Dialect/Vector/VectorUtils.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/Location.h" #include "mlir/IR/Location.h"
#include "mlir/IR/Types.h" #include "mlir/IR/Types.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
▲ Show 20 Lines • Show All 1,265 Lines • Show Last 20 Lines

mlir/test/Dialect/Affine/canonicalize.mlir

  • This file was moved from mlir/test/Dialect/AffineOps/canonicalize.mlir.
The contents of this file were not changed.

mlir/test/Dialect/Affine/dma.mlir

  • This file was moved from mlir/test/Dialect/AffineOps/dma.mlir.
The contents of this file were not changed.

mlir/test/Dialect/Affine/inlining.mlir

  • This file was moved from mlir/test/Dialect/AffineOps/inlining.mlir.
The contents of this file were not changed.

mlir/test/Dialect/Affine/invalid.mlir

  • This file was moved from mlir/test/Dialect/AffineOps/invalid.mlir.
The contents of this file were not changed.

mlir/test/Dialect/Affine/load-store-invalid.mlir

  • This file was moved from mlir/test/Dialect/AffineOps/load-store-invalid.mlir.
The contents of this file were not changed.

mlir/test/Dialect/Affine/load-store.mlir

  • This file was moved from mlir/test/Dialect/AffineOps/load-store.mlir.
The contents of this file were not changed.

mlir/test/Dialect/Affine/memref-stride-calculation.mlir

  • This file was moved from mlir/test/Dialect/AffineOps/memref-stride-calculation.mlir.
The contents of this file were not changed.

mlir/test/Dialect/Affine/ops.mlir

  • This file was moved from mlir/test/Dialect/AffineOps/ops.mlir.
The contents of this file were not changed.

mlir/test/Dialect/AffineOps/canonicalize.mlir

  • This file was moved to mlir/test/Dialect/Affine/canonicalize.mlir.

mlir/test/Dialect/AffineOps/dma.mlir

  • This file was moved to mlir/test/Dialect/Affine/dma.mlir.

mlir/test/Dialect/AffineOps/inlining.mlir

  • This file was moved to mlir/test/Dialect/Affine/inlining.mlir.

mlir/test/Dialect/AffineOps/invalid.mlir

  • This file was moved to mlir/test/Dialect/Affine/invalid.mlir.

mlir/test/Dialect/AffineOps/load-store-invalid.mlir

  • This file was moved to mlir/test/Dialect/Affine/load-store-invalid.mlir.

mlir/test/Dialect/AffineOps/load-store.mlir

  • This file was moved to mlir/test/Dialect/Affine/load-store.mlir.

mlir/test/Dialect/AffineOps/memref-stride-calculation.mlir

  • This file was moved to mlir/test/Dialect/Affine/memref-stride-calculation.mlir.

mlir/test/Dialect/AffineOps/ops.mlir

  • This file was moved to mlir/test/Dialect/Affine/ops.mlir.

mlir/test/EDSC/CMakeLists.txt

add_llvm_executable(mlir-edsc-builder-api-test add_llvm_executable(mlir-edsc-builder-api-test
builder-api-test.cpp builder-api-test.cpp
) )
llvm_update_compile_flags(mlir-edsc-builder-api-test) llvm_update_compile_flags(mlir-edsc-builder-api-test)
target_link_libraries(mlir-edsc-builder-api-test target_link_libraries(mlir-edsc-builder-api-test
PRIVATE PRIVATE
MLIRAffineOps MLIRAffine
MLIREDSC MLIREDSC
MLIRIR MLIRIR
MLIRLinalgEDSC MLIRLinalgEDSC
MLIRLinalgOps MLIRLinalgOps
MLIRLoopOps MLIRLoopOps
MLIRStandardOps MLIRStandardOps
MLIRTransforms MLIRTransforms
MLIRVector MLIRVector
LLVMCore LLVMCore
LLVMSupport LLVMSupport
) )
target_include_directories(mlir-edsc-builder-api-test PRIVATE ..) target_include_directories(mlir-edsc-builder-api-test PRIVATE ..)
whole_archive_link(mlir-edsc-builder-api-test whole_archive_link(mlir-edsc-builder-api-test
MLIRAffineOps MLIRAffine
MLIRLinalgOps MLIRLinalgOps
MLIRLoopOps MLIRLoopOps
MLIRStandardOps MLIRStandardOps
MLIRVector MLIRVector
MLIRTransforms MLIRTransforms
) )

mlir/test/EDSC/builder-api-test.cpp

//===- builder-api-test.cpp - Tests for Declarative Builder APIs ----------===// //===- builder-api-test.cpp - Tests for Declarative Builder APIs ----------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// RUN: mlir-edsc-builder-api-test | FileCheck %s -dump-input-on-failure // RUN: mlir-edsc-builder-api-test | FileCheck %s -dump-input-on-failure
#include "mlir/Dialect/AffineOps/EDSC/Intrinsics.h" #include "mlir/Dialect/Affine/EDSC/Intrinsics.h"
#include "mlir/Dialect/Linalg/EDSC/Intrinsics.h" #include "mlir/Dialect/Linalg/EDSC/Intrinsics.h"
#include "mlir/Dialect/LoopOps/EDSC/Builders.h" #include "mlir/Dialect/LoopOps/EDSC/Builders.h"
#include "mlir/Dialect/StandardOps/EDSC/Intrinsics.h" #include "mlir/Dialect/StandardOps/EDSC/Intrinsics.h"
#include "mlir/Dialect/Vector/EDSC/Intrinsics.h" #include "mlir/Dialect/Vector/EDSC/Intrinsics.h"
#include "mlir/EDSC/Builders.h" #include "mlir/EDSC/Builders.h"
#include "mlir/EDSC/Intrinsics.h" #include "mlir/EDSC/Intrinsics.h"
#include "mlir/IR/AffineExpr.h" #include "mlir/IR/AffineExpr.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
Show All 13 Lines
#include "llvm/Support/raw_ostream.h" #include "llvm/Support/raw_ostream.h"
using namespace mlir; using namespace mlir;
using namespace mlir::edsc; using namespace mlir::edsc;
using namespace mlir::edsc::intrinsics; using namespace mlir::edsc::intrinsics;
static MLIRContext &globalContext() { static MLIRContext &globalContext() {
static bool init_once = []() { static bool init_once = []() {
registerDialect<AffineOpsDialect>(); registerDialect<AffineDialect>();
registerDialect<linalg::LinalgDialect>(); registerDialect<linalg::LinalgDialect>();
registerDialect<loop::LoopOpsDialect>(); registerDialect<loop::LoopOpsDialect>();
registerDialect<StandardOpsDialect>(); registerDialect<StandardOpsDialect>();
registerDialect<vector::VectorDialect>(); registerDialect<vector::VectorDialect>();
return true; return true;
}(); }();
(void)init_once; (void)init_once;
static thread_local MLIRContext context; static thread_local MLIRContext context;
▲ Show 20 Lines • Show All 1,031 Lines • Show Last 20 Lines

mlir/test/lib/Dialect/Affine/CMakeLists.txt

  • This file was added.
add_llvm_library(MLIRAffineTransformsTestPasses
TestAffineDataCopy.cpp
ADDITIONAL_HEADER_DIRS
${MLIR_MAIN_INCLUDE_DIR}/mlir/Dialect/Affine
${MLIR_MAIN_INCLUDE_DIR}/mlir/IR
)
target_link_libraries(MLIRAffineTransformsTestPasses PRIVATE
MLIRIR
MLIRPass
MLIRAffineTransforms
MLIRSupport
)

mlir/test/lib/Dialect/Affine/TestAffineDataCopy.cpp

  • This file was moved from mlir/test/lib/Transforms/TestAffineDataCopy.cpp.
//===- TestAffineDataCopy.cpp - Test affine data copy utility -------------===// //===- TestAffineDataCopy.cpp - Test affine data copy utility -------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements a pass to test affine data copy utility functions and // This file implements a pass to test affine data copy utility functions and
// options. // options.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/Passes.h" #include "mlir/Analysis/Passes.h"
#include "mlir/Analysis/Utils.h" #include "mlir/Analysis/Utils.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/LoopUtils.h" #include "mlir/Transforms/LoopUtils.h"
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
#define PASS_NAME "test-affine-data-copy" #define PASS_NAME "test-affine-data-copy"
using namespace mlir; using namespace mlir;
▲ Show 20 Lines • Show All 72 Lines • Show Last 20 Lines

mlir/test/lib/Dialect/CMakeLists.txt

add_subdirectory(Affine)
add_subdirectory(SPIRV) add_subdirectory(SPIRV)

mlir/test/lib/Transforms/CMakeLists.txt

add_llvm_library(MLIRTestTransforms add_llvm_library(MLIRTestTransforms
TestAffineDataCopy.cpp
TestAllReduceLowering.cpp TestAllReduceLowering.cpp
TestCallGraph.cpp TestCallGraph.cpp
TestConstantFold.cpp TestConstantFold.cpp
TestLoopFusion.cpp TestLoopFusion.cpp
TestGpuMemoryPromotion.cpp TestGpuMemoryPromotion.cpp
TestGpuParallelLoopMapping.cpp TestGpuParallelLoopMapping.cpp
TestInlining.cpp TestInlining.cpp
TestLinalgTransforms.cpp TestLinalgTransforms.cpp
Show All 20 Lines
include_directories(${CMAKE_CURRENT_SOURCE_DIR}/../TestDialect) include_directories(${CMAKE_CURRENT_SOURCE_DIR}/../TestDialect)
include_directories(${CMAKE_CURRENT_BINARY_DIR}/../TestDialect) include_directories(${CMAKE_CURRENT_BINARY_DIR}/../TestDialect)
include_directories(${CMAKE_CURRENT_SOURCE_DIR}/../DeclarativeTransforms) include_directories(${CMAKE_CURRENT_SOURCE_DIR}/../DeclarativeTransforms)
include_directories(${CMAKE_CURRENT_BINARY_DIR}/../DeclarativeTransforms) include_directories(${CMAKE_CURRENT_BINARY_DIR}/../DeclarativeTransforms)
target_link_libraries(MLIRTestTransforms target_link_libraries(MLIRTestTransforms
PUBLIC PUBLIC
MLIRAffineOps MLIRAffine
MLIRAnalysis MLIRAnalysis
MLIREDSC MLIREDSC
MLIRGPU MLIRGPU
MLIRLinalgOps MLIRLinalgOps
MLIRLinalgTransforms MLIRLinalgTransforms
MLIRLoopOps MLIRLoopOps
MLIRGPU MLIRGPU
MLIRPass MLIRPass
MLIRStandardToStandard MLIRStandardToStandard
MLIRTestDialect MLIRTestDialect
MLIRTransformUtils MLIRTransformUtils
MLIRVectorToLoops MLIRVectorToLoops
MLIRVector MLIRVector
) )

mlir/test/lib/Transforms/TestAffineDataCopy.cpp

  • This file was moved to mlir/test/lib/Dialect/Affine/TestAffineDataCopy.cpp.

mlir/test/lib/Transforms/TestConstantFold.cpp

//===- TestConstantFold.cpp - Pass to test constant folding ---------------===// //===- TestConstantFold.cpp - Pass to test constant folding ---------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/Function.h" #include "mlir/IR/Function.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/FoldUtils.h" #include "mlir/Transforms/FoldUtils.h"
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
#include "mlir/Transforms/Utils.h" #include "mlir/Transforms/Utils.h"
▲ Show 20 Lines • Show All 55 Lines • Show Last 20 Lines

mlir/test/lib/Transforms/TestLoopFusion.cpp

//===- TestLoopFusion.cpp - Test loop fusion ------------------------------===// //===- TestLoopFusion.cpp - Test loop fusion ------------------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements a pass to test various loop fusion utility functions. // This file implements a pass to test various loop fusion utility functions.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/AffineAnalysis.h" #include "mlir/Analysis/AffineAnalysis.h"
#include "mlir/Analysis/AffineStructures.h" #include "mlir/Analysis/AffineStructures.h"
#include "mlir/Analysis/Passes.h" #include "mlir/Analysis/Passes.h"
#include "mlir/Analysis/Utils.h" #include "mlir/Analysis/Utils.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Transforms/LoopFusionUtils.h" #include "mlir/Transforms/LoopFusionUtils.h"
#include "mlir/Transforms/LoopUtils.h" #include "mlir/Transforms/LoopUtils.h"
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
#include "llvm/ADT/STLExtras.h" #include "llvm/ADT/STLExtras.h"
▲ Show 20 Lines • Show All 182 Lines • Show Last 20 Lines

mlir/test/lib/Transforms/TestMemRefBoundCheck.cpp

Show All 10 Lines
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/ADT/TypeSwitch.h" #include "mlir/ADT/TypeSwitch.h"
#include "mlir/Analysis/AffineAnalysis.h" #include "mlir/Analysis/AffineAnalysis.h"
#include "mlir/Analysis/AffineStructures.h" #include "mlir/Analysis/AffineStructures.h"
#include "mlir/Analysis/Passes.h" #include "mlir/Analysis/Passes.h"
#include "mlir/Analysis/Utils.h" #include "mlir/Analysis/Utils.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#define DEBUG_TYPE "memref-bound-check" #define DEBUG_TYPE "memref-bound-check"
using namespace mlir; using namespace mlir;
Show All 29 Lines

mlir/test/lib/Transforms/TestMemRefDependenceCheck.cpp

//===- TestMemRefDependenceCheck.cpp - Test dep analysis ------------------===// //===- TestMemRefDependenceCheck.cpp - Test dep analysis ------------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements a pass to run pair-wise memref access dependence checks. // This file implements a pass to run pair-wise memref access dependence checks.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/AffineAnalysis.h" #include "mlir/Analysis/AffineAnalysis.h"
#include "mlir/Analysis/AffineStructures.h" #include "mlir/Analysis/AffineStructures.h"
#include "mlir/Analysis/Passes.h" #include "mlir/Analysis/Passes.h"
#include "mlir/Analysis/Utils.h" #include "mlir/Analysis/Utils.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/Dialect/StandardOps/IR/Ops.h" #include "mlir/Dialect/StandardOps/IR/Ops.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#define DEBUG_TYPE "test-memref-dependence-check" #define DEBUG_TYPE "test-memref-dependence-check"
using namespace mlir; using namespace mlir;
▲ Show 20 Lines • Show All 100 Lines • Show Last 20 Lines

mlir/test/lib/Transforms/TestParallelismDetection.cpp

//===- ParallelismDetection.cpp - Parallelism Detection pass ------------*-===// //===- ParallelismDetection.cpp - Parallelism Detection pass ------------*-===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements a pass to detect parallel affine 'affine.for' ops. // This file implements a pass to detect parallel affine 'affine.for' ops.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/Passes.h" #include "mlir/Analysis/Passes.h"
#include "mlir/Analysis/Utils.h" #include "mlir/Analysis/Utils.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
using namespace mlir; using namespace mlir;
namespace { namespace {
struct TestParallelismDetection struct TestParallelismDetection
Show All 26 Lines

mlir/test/lib/Transforms/TestVectorizationUtils.cpp

//===- VectorizerTestPass.cpp - VectorizerTestPass Pass Impl --------------===// //===- VectorizerTestPass.cpp - VectorizerTestPass Pass Impl --------------===//
// //
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information. // See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file implements a simple testing pass for vectorization functionality. // This file implements a simple testing pass for vectorization functionality.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "mlir/Analysis/AffineAnalysis.h" #include "mlir/Analysis/AffineAnalysis.h"
#include "mlir/Analysis/NestedMatcher.h" #include "mlir/Analysis/NestedMatcher.h"
#include "mlir/Analysis/SliceAnalysis.h" #include "mlir/Analysis/SliceAnalysis.h"
#include "mlir/Dialect/AffineOps/AffineOps.h" #include "mlir/Dialect/Affine/IR/AffineOps.h"
mehdi_aminiUnsubmitted
Done
ReplyInline Actions

Since you had to change all of them, and since every downstream user will have to adjust, what about also renaming the mlir/Dialect/AffineOps into mlir/Dialect/Affine ?

mehdi_amini: Since you had to change all of them, and since every downstream user will have to adjust, what…
rsudermanAuthorUnsubmitted
Done
ReplyInline Actions

Moved AffineOps to Affine.

rsuderman: Moved AffineOps to Affine.
#include "mlir/Dialect/Vector/VectorUtils.h" #include "mlir/Dialect/Vector/VectorUtils.h"
#include "mlir/IR/Builders.h" #include "mlir/IR/Builders.h"
#include "mlir/IR/Diagnostics.h" #include "mlir/IR/Diagnostics.h"
#include "mlir/IR/StandardTypes.h" #include "mlir/IR/StandardTypes.h"
#include "mlir/Pass/Pass.h" #include "mlir/Pass/Pass.h"
#include "mlir/Support/Functional.h" #include "mlir/Support/Functional.h"
#include "mlir/Support/STLExtras.h" #include "mlir/Support/STLExtras.h"
#include "mlir/Transforms/Passes.h" #include "mlir/Transforms/Passes.h"
▲ Show 20 Lines • Show All 265 Lines • Show Last 20 Lines

mlir/tools/mlir-opt/CMakeLists.txt

set(LLVM_OPTIONAL_SOURCES set(LLVM_OPTIONAL_SOURCES
null.cpp null.cpp
) )
get_property(dialect_libs GLOBAL PROPERTY MLIR_DIALECT_LIBS) get_property(dialect_libs GLOBAL PROPERTY MLIR_DIALECT_LIBS)
get_property(conversion_libs GLOBAL PROPERTY MLIR_CONVERSION_LIBS) get_property(conversion_libs GLOBAL PROPERTY MLIR_CONVERSION_LIBS)
set(LIBS set(LIBS
${dialect_libs} ${dialect_libs}
${conversion_libs} ${conversion_libs}
MLIRLoopOpsTransforms MLIRLoopOpsTransforms
MLIRLoopAnalysis MLIRLoopAnalysis
MLIRAffineTransformsTestPasses
MLIRAnalysis MLIRAnalysis
MLIRDialect MLIRDialect
MLIREDSC MLIREDSC
MLIROptLib MLIROptLib
MLIRParser MLIRParser
MLIRPass MLIRPass
MLIRQuantizerFxpMathConfig MLIRQuantizerFxpMathConfig
MLIRQuantizerSupport MLIRQuantizerSupport
▲ Show 20 Lines • Show All 42 Lines • Show Last 20 Lines