[mlir:PDL] Add support for DialectConversion with pattern configurations

[rriddle]

Summary

Up until now PDL(L) has not supported dialect conversion because we had no
way of remapping values or integrating with type conversions. This commit
rectifies that by adding a new "pattern configuration" concept to PDL. This
essentially allows for attaching external configurations to patterns, which
can hook into pattern events (for now just the scope of a rewrite, but we
could also pass configs to native rewrites as well). This allows for injecting
the type converter into the conversion pattern rewriter.

[jpienaar]

The type converter scopes is a bit new, is this solely for this or where else would you see this be used?

mlir/lib/IR/PatternMatch.cpp

194

This is creative use of bookkeeping :-) Is this due to invalidation of Value hash maps ? (E.g.,. Why couldn't this just be an Analysis that gets populated)

mlir/lib/Transforms/Utils/DialectConversion.cpp

3277

This is a bit undesirable as this is at runtime. And this could be for a dynamically compiled pattern which gets invoked from a pass that can actually flag a failure. I'd rather have an error propagated.

[rriddle]

mlir/lib/IR/PatternMatch.cpp

194

That gives me an idea, let me rework this.

mlir/lib/Transforms/Utils/DialectConversion.cpp

3277

Let me rework this then. We'll have to build failable rewrites into PDL, which will require some more substantial changes. I'll start building that under this.

[rriddle]

In D133142#3775985, @jpienaar wrote:

The type converter scopes is a bit new, is this solely for this or where else would you see this be used?

Removed it, thanks. We used to need that, but I hadn't checked in a while.

[jpienaar]

Nice, thanks! Haven't tried it out yet though :)

mlir/include/mlir/IR/PatternMatch.h

881

s/and/or/ ?

1249

So these could both fail but still be run and success returned? What should be expectation ? (fail if one fails, fails if both fails, shortcircuit return or evaluate both)

mlir/include/mlir/Transforms/DialectConversion.pdll

2

pdll ?

mlir/lib/Transforms/Utils/DialectConversion.cpp

3256

This pulls PDL into DialectConversion proper, any cost if PDL is not used? Could these be in a helper PDL side?

mlir/test/lib/Transforms/TestDialectConversion.pdll

16

OOC when would one not want the result types converted if there is a type converter registered? (I believe dialect conversion side that's implicit too). Is this to avoid making dialect conversion/type converter too ingrained in PDL? (although as you've done it above it is a config only).

I could see this maybe differ for intermediate results. But can't think of good example. (not a blocker, explicit "default" is good)

[rriddle]

mlir/include/mlir/IR/PatternMatch.h

1249

Originally I wasn't expecting nested logicalresult/failable things within pair/tuple. Just switched this to properly check for failure though (just in case).

mlir/lib/Transforms/Utils/DialectConversion.cpp

3256

No cost if it's unused. The class is just defined in DialectConversion, it's attached on the PDL patterns directly so you only pay a cost if you're using a PDL pattern that wants dialect conversion stuff.

mlir/test/lib/Transforms/TestDialectConversion.pdll

16

Most of the cases you wouldn't want result type conversion is for intermediate results. I'm a bit hesitant on implicit behavior changes here because then the underlying behavior would likely become harder to understand. If we could come up with a good model for it I'd be okay, but right now IMO being explicit is a better starting point (and matches the C++ side).

mlir/include/mlir/Conversion/PDLToPDLInterp/PDLToPDLInterp.h

 //===- PDLToPDLInterp.h - PDL to PDL Interpreter conversion -----*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file provides a pass for PDL to PDL Interpreter dialect conversion.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef MLIR_CONVERSION_PDLTOPDLINTERP_PDLTOPDLINTERP_H
 #define MLIR_CONVERSION_PDLTOPDLINTERP_PDLTOPDLINTERP_H
 
-#include <memory>
+#include "mlir/Support/LLVM.h"
 
 namespace mlir {
 class ModuleOp;
+class Operation;
 template <typename OpT>
 class OperationPass;
+class PDLPatternConfigSet;
 
 #define GEN_PASS_DECL_CONVERTPDLTOPDLINTERP
 #include "mlir/Conversion/Passes.h.inc"
 
 /// Creates and returns a pass to convert PDL ops to PDL interpreter ops.
 std::unique_ptr<OperationPass<ModuleOp>> createPDLToPDLInterpPass();
 
+/// Creates and returns a pass to convert PDL ops to PDL interpreter ops.
+/// `configMap` holds a map of the configurations for each pattern being
+/// compiled.
+std::unique_ptr<OperationPass<ModuleOp>> createPDLToPDLInterpPass(
+    DenseMap<Operation *, PDLPatternConfigSet *> &configMap);
+
 } // namespace mlir
 
 #endif // MLIR_CONVERSION_PDLTOPDLINTERP_PDLTOPDLINTERP_H

mlir/include/mlir/IR/PatternMatch.h

 /// rewrite pattern on the current IR being matched, providing a way to keep
 /// track of any mutations made. This class should be used to perform all
 /// necessary IR mutations within a rewrite pattern, as the pattern driver may
 /// be tracking various state that would be invalidated when a mutation takes
 /// place.
 class PatternRewriter : public RewriterBase {
 public:
   using RewriterBase::RewriterBase;
+
+  /// A hook used to indicate if the pattern rewriter can recover from failure
+  /// during the rewrite stage of a pattern. For example, if the pattern
+  /// rewriter supports rollback, it may progress smoothly even if IR was
+  /// changed during the rewrite.
+  virtual bool canRecoverFromRewriteFailure() const { return false; }
 };
 
 //===----------------------------------------------------------------------===//
-// PDLPatternModule
+// PDL Patterns
 //===----------------------------------------------------------------------===//
 
 //===----------------------------------------------------------------------===//
 // PDLValue
 
 /// Storage type of byte-code interpreter values. These are passed to constraint
 /// functions as arguments.
 class PDLValue {
protected:
   SmallVector<ValueRange> valueRanges;
   /// Memory allocated to store ranges in the result list whose lifetime was
   /// generated in the native function.
   SmallVector<llvm::OwningArrayRef<Type>> allocatedTypeRanges;
   SmallVector<llvm::OwningArrayRef<Value>> allocatedValueRanges;
 };
 
 //===----------------------------------------------------------------------===//
+// PDLPatternConfig
+
+/// An individual configuration for a pattern, which can be accessed by native
+/// functions via the PDLPatternConfigSet. This allows for injecting additional
+/// configuration into PDL patterns that is specific to certain compilation
+/// flows.
+class PDLPatternConfig {
+public:
+  virtual ~PDLPatternConfig() = default;
+
+  /// Hooks that are invoked at the beginning and end of a rewrite of a matched
+  /// pattern. These can be used to setup any specific state necessary for the
+  /// rewrite.
+  virtual void notifyRewriteBegin(PatternRewriter &rewriter) {}
+  virtual void notifyRewriteEnd(PatternRewriter &rewriter) {}
+
+  /// Return the TypeID that represents this configuration.
+  TypeID getTypeID() const { return id; }
+
+protected:
+  PDLPatternConfig(TypeID id) : id(id) {}
+
+private:
+  TypeID id;
+};
+
+/// This class provides a base class for users implementing a type of pattern
+/// configuration.
+template <typename T>
+class PDLPatternConfigBase : public PDLPatternConfig {
+public:
+  /// Support LLVM style casting.
+  static bool classof(const PDLPatternConfig *config) {
+    return config->getTypeID() == getConfigID();
+  }
+
+  /// Return the type id used for this configuration.
+  static TypeID getConfigID() { return TypeID::get<T>(); }
+
+protected:
+  PDLPatternConfigBase() : PDLPatternConfig(getConfigID()) {}
+};
+
+/// This class contains a set of configurations for a specific pattern.
+/// Configurations are uniqued by TypeID, meaning that only one configuration of
+/// each type is allowed.
+class PDLPatternConfigSet {
+public:
+  PDLPatternConfigSet() = default;
+
+  /// Construct a set with the given configurations.
+  template <typename... ConfigsT>
+  PDLPatternConfigSet(ConfigsT &&...configs) {
+    (addConfig(std::forward<ConfigsT>(configs)), ...);
+  }
+
+  /// Get the configuration defined by the given type. Asserts that the
+  /// configuration of the provided type exists.
+  template <typename T>
+  const T &get() const {
+    const T *config = tryGet<T>();
+    assert(config && "configuration not found");
+    return *config;
+  }
+
+  /// Get the configuration defined by the given type, returns nullptr if the
+  /// configuration does not exist.
+  template <typename T>
+  const T *tryGet() const {
+    for (const auto &configIt : configs)
+      if (const T *config = dyn_cast<T>(configIt.get()))
+        return config;
+    return nullptr;
+  }
+
+  /// Notify the configurations within this set at the beginning and end of a
+  /// rewrite of a matched pattern.
+  void notifyRewriteBegin(PatternRewriter &rewriter) {
+    for (const auto &config : configs)
+      config->notifyRewriteBegin(rewriter);
+  }
+  void notifyRewriteEnd(PatternRewriter &rewriter) {
+    for (const auto &config : configs)
+      config->notifyRewriteEnd(rewriter);
+  }
+
+protected:
+  /// Add a configuration to the set.
+  template <typename T>
+  void addConfig(T &&config) {
+    assert(!tryGet<std::decay_t<T>>() && "configuration already exists");
+    configs.emplace_back(
+        std::make_unique<std::decay_t<T>>(std::forward<T>(config)));
+  }
+
+  /// The set of configurations for this pattern. This uses a vector instead of
+  /// a map with the expectation that the number of configurations per set is
+  /// small (<= 1).
+  SmallVector<std::unique_ptr<PDLPatternConfig>> configs;
+};
+
+//===----------------------------------------------------------------------===//
 // PDLPatternModule
 
 /// A generic PDL pattern constraint function. This function applies a
 /// constraint to a given set of opaque PDLValue entities. Returns success if
 /// the constraint successfully held, failure otherwise.
 using PDLConstraintFunction =
     std::function<LogicalResult(PatternRewriter &, ArrayRef<PDLValue>)>;
 /// A native PDL rewrite function. This function performs a rewrite on the
 /// given set of values. Any results from this rewrite that should be passed
 /// back to PDL should be added to the provided result list. This method is only
-/// invoked when the corresponding match was successful.
-using PDLRewriteFunction =
-    std::function<void(PatternRewriter &, PDLResultList &, ArrayRef<PDLValue>)>;
+/// invoked when the corresponding match was successful. Returns failure if an
+/// invariant of the rewrite was broken (certain rewriters may recover from
+/// partial pattern application).
+using PDLRewriteFunction = std::function<LogicalResult(
+    PatternRewriter &, PDLResultList &, ArrayRef<PDLValue>)>;
 
 namespace detail {
 namespace pdl_function_builder {
 /// A utility variable that always resolves to false. This is useful for static
 /// asserts that are always false, but only should fire in certain templated
 /// constructs. For example, if a templated function should never be called, the
 /// function could be defined as:
 ///

 };
 template <>
 struct ProcessPDLValue<ValueTypeRange<ResultRange>> {
   static void processAsResult(PatternRewriter &, PDLResultList &results,
                               ValueTypeRange<ResultRange> types) {
     results.push_back(types);
   }
 };
+template <unsigned N>
+struct ProcessPDLValue<SmallVector<Type, N>> {
+  static void processAsResult(PatternRewriter &, PDLResultList &results,
+                              SmallVector<Type, N> values) {
+    results.push_back(TypeRange(values));
+  }
+};
 
 //===----------------------------------------------------------------------===//
 // Value
 
 template <>
 struct ProcessPDLValue<Value> : public ProcessBuiltinPDLValue<Value> {};
 
 //===----------------------------------------------------------------------===//

 };
 template <>
 struct ProcessPDLValue<ResultRange> {
   static void processAsResult(PatternRewriter &, PDLResultList &results,
                               ResultRange values) {
     results.push_back(values);
   }
 };
+template <unsigned N>
+struct ProcessPDLValue<SmallVector<Value, N>> {
+  static void processAsResult(PatternRewriter &, PDLResultList &results,
+                              SmallVector<Value, N> values) {
+    results.push_back(ValueRange(values));
+  }
+};
 
 //===----------------------------------------------------------------------===//
 // PDL Function Builder: Argument Handling
 //===----------------------------------------------------------------------===//
 
 /// Validate the given PDLValues match the constraints defined by the argument
 /// types of the given function. In the case of failure, a match failure
 /// diagnostic is emitted.

 }
 
 //===----------------------------------------------------------------------===//
 // PDL Function Builder: Results Handling
 //===----------------------------------------------------------------------===//
 
 /// Store a single result within the result list.
 template <typename T>
-static void processResults(PatternRewriter &rewriter, PDLResultList &results,
-                           T &&value) {
+static LogicalResult processResults(PatternRewriter &rewriter,
+                                    PDLResultList &results, T &&value) {
   ProcessPDLValue<T>::processAsResult(rewriter, results,
                                       std::forward<T>(value));
+  return success();
 }
 
 /// Store a std::pair<> as individual results within the result list.
 template <typename T1, typename T2>
-static void processResults(PatternRewriter &rewriter, PDLResultList &results,
+static LogicalResult processResults(PatternRewriter &rewriter,
+                                    PDLResultList &results,
                                     std::pair<T1, T2> &&pair) {
-  processResults(rewriter, results, std::move(pair.first));
-  processResults(rewriter, results, std::move(pair.second));
+  (void)processResults(rewriter, results, std::move(pair.first));
+  (void)processResults(rewriter, results, std::move(pair.second));
+  return success();
 }
 
 /// Store a std::tuple<> as individual results within the result list.
 template <typename... Ts>
-static void processResults(PatternRewriter &rewriter, PDLResultList &results,
+static LogicalResult processResults(PatternRewriter &rewriter,
+                                    PDLResultList &results,
                                     std::tuple<Ts...> &&tuple) {
   auto applyFn = [&](auto &&...args) {
-    (processResults(rewriter, results, std::move(args)), ...);
+    ((void)processResults(rewriter, results, std::move(args)), ...);
   };
   std::apply(applyFn, std::move(tuple));
+  return success();
+}
+
+/// Handle LogicalResult propagation.
+inline LogicalResult processResults(PatternRewriter &rewriter,
+                                    PDLResultList &results,
+                                    LogicalResult &&result) {
+  return result;
+}
+template <typename T>
+static LogicalResult processResults(PatternRewriter &rewriter,
+                                    PDLResultList &results,
+                                    FailureOr<T> &&result) {
+  if (failed(result))
+    return failure();
+  return processResults(rewriter, results, std::move(*result));
 }
 
 //===----------------------------------------------------------------------===//
 // PDL Constraint Builder
 //===----------------------------------------------------------------------===//
 
 /// Process the arguments of a native constraint and invoke it.
 template <typename PDLFnT, std::size_t... I,

 //===----------------------------------------------------------------------===//
 // PDL Rewrite Builder
 //===----------------------------------------------------------------------===//
 
 /// Process the arguments of a native rewrite and invoke it.
 /// This overload handles the case of no return values.
 template <typename PDLFnT, std::size_t... I,
           typename FnTraitsT = llvm::function_traits<PDLFnT>>
-std::enable_if_t<std::is_same<typename FnTraitsT::result_t, void>::value>
+std::enable_if_t<std::is_same<typename FnTraitsT::result_t, void>::value,
+                 LogicalResult>
 processArgsAndInvokeRewrite(PDLFnT &fn, PatternRewriter &rewriter,
                             PDLResultList &, ArrayRef<PDLValue> values,
                             std::index_sequence<I...>) {
   fn(rewriter,
      (ProcessPDLValue<typename FnTraitsT::template arg_t<I + 1>>::processAsArg(
          values[I]))...);
+  return success();
 }
 /// This overload handles the case of return values, which need to be packaged
 /// into the result list.
 template <typename PDLFnT, std::size_t... I,
           typename FnTraitsT = llvm::function_traits<PDLFnT>>
-std::enable_if_t<!std::is_same<typename FnTraitsT::result_t, void>::value>
+std::enable_if_t<!std::is_same<typename FnTraitsT::result_t, void>::value,
+                 LogicalResult>
 processArgsAndInvokeRewrite(PDLFnT &fn, PatternRewriter &rewriter,
                             PDLResultList &results, ArrayRef<PDLValue> values,
                             std::index_sequence<I...>) {
-  processResults(
+  return processResults(
       rewriter, results,
       fn(rewriter, (ProcessPDLValue<typename FnTraitsT::template arg_t<I + 1>>::
                         processAsArg(values[I]))...));
 }
 
 /// Build a rewrite function from the given function `RewriteFnT`. This
 /// allows for enabling the user to define simpler, more direct rewrite
 /// functions without needing to handle the low-level PDL goop.

 buildRewriteFn(RewriteFnT &&rewriteFn) {
   return [rewriteFn = std::forward<RewriteFnT>(rewriteFn)](
              PatternRewriter &rewriter, PDLResultList &results,
              ArrayRef<PDLValue> values) {
     auto argIndices =
         std::make_index_sequence<llvm::function_traits<RewriteFnT>::num_args -
                                  1>();
     assertArgs<RewriteFnT>(rewriter, values, argIndices);
-    processArgsAndInvokeRewrite(rewriteFn, rewriter, results, values,
+    return processArgsAndInvokeRewrite(rewriteFn, rewriter, results, values,
                                        argIndices);
   };
 }
 
 } // namespace pdl_function_builder
 } // namespace detail
 
+//===----------------------------------------------------------------------===//
+// PDLPatternModule
+
 /// This class contains all of the necessary data for a set of PDL patterns, or
 /// pattern rewrites specified in the form of the PDL dialect. This PDL module
 /// contained by this pattern may contain any number of `pdl.pattern`
 /// operations.
 class PDLPatternModule {
 public:
   PDLPatternModule() = default;
 
-  /// Construct a PDL pattern with the given module.
-  PDLPatternModule(OwningOpRef<ModuleOp> pdlModule)
-      : pdlModule(std::move(pdlModule)) {}
+  /// Construct a PDL pattern with the given module and configurations.
+  PDLPatternModule(OwningOpRef<ModuleOp> module)
+      : pdlModule(std::move(module)) {}
+  template <typename... ConfigsT>
+  PDLPatternModule(OwningOpRef<ModuleOp> module, ConfigsT &&...patternConfigs)
+      : PDLPatternModule(std::move(module)) {
+    auto configSet = std::make_unique<PDLPatternConfigSet>(
+        std::forward<ConfigsT>(patternConfigs)...);
+    attachConfigToPatterns(*pdlModule, *configSet);
+    configs.emplace_back(std::move(configSet));
+  }
 
   /// Merge the state in `other` into this pattern module.
   void mergeIn(PDLPatternModule &&other);
 
   /// Return the internal PDL module of this pattern.
   ModuleOp getModule() { return pdlModule.get(); }
 
   //===--------------------------------------------------------------------===//
public:
   /// Return the set of the registered rewrite functions.
   const llvm::StringMap<PDLRewriteFunction> &getRewriteFunctions() const {
     return rewriteFunctions;
   }
   llvm::StringMap<PDLRewriteFunction> takeRewriteFunctions() {
     return rewriteFunctions;
   }
 
+  /// Return the set of the registered pattern configs.
+  SmallVector<std::unique_ptr<PDLPatternConfigSet>> takeConfigs() {
+    return std::move(configs);
+  }
+  DenseMap<Operation *, PDLPatternConfigSet *> takeConfigMap() {
+    return std::move(configMap);
+  }
+
   /// Clear out the patterns and functions within this module.
   void clear() {
     pdlModule = nullptr;
     constraintFunctions.clear();
     rewriteFunctions.clear();
   }
 
 private:
+  /// Attach the given pattern config set to the patterns defined within the
+  /// given module.
+  void attachConfigToPatterns(ModuleOp module, PDLPatternConfigSet &configSet);
+
   /// The module containing the `pdl.pattern` operations.
   OwningOpRef<ModuleOp> pdlModule;
 
+  /// The set of configuration sets referenced by patterns within `pdlModule`.
+  SmallVector<std::unique_ptr<PDLPatternConfigSet>> configs;
+  DenseMap<Operation *, PDLPatternConfigSet *> configMap;
+
   /// The external functions referenced from within the PDL module.
   llvm::StringMap<PDLConstraintFunction> constraintFunctions;
   llvm::StringMap<PDLRewriteFunction> rewriteFunctions;
 };
 
 //===----------------------------------------------------------------------===//
 // RewritePatternSet
 //===----------------------------------------------------------------------===//

mlir/include/mlir/Transforms/DialectConversion.h

public:
   /// remap failed, success otherwise.
   LogicalResult getRemappedValues(ValueRange keys,
                                   SmallVectorImpl<Value> &results);
 
   //===--------------------------------------------------------------------===//
   // PatternRewriter Hooks
   //===--------------------------------------------------------------------===//
 
+  /// Indicate that the conversion rewriter can recover from rewrite failure.
+  /// Recovery is supported via rollback, allowing for continued processing of
+  /// patterns even if a failure is encountered during the rewrite step.
+  bool canRecoverFromRewriteFailure() const override { return true; }
+
   /// PatternRewriter hook for replacing the results of an operation when the
   /// given functor returns true.
   void replaceOpWithIf(
       Operation *op, ValueRange newValues, bool *allUsesReplaced,
       llvm::unique_function<bool(OpOperand &) const> functor) override;
 
   /// PatternRewriter hook for replacing the results of an operation.
   void replaceOp(Operation *op, ValueRange newValues) override;
private:
   /// An optional legality callback for unknown operations.
   DynamicLegalityCallbackFn unknownLegalityFn;
 
   /// The current context this target applies to.
   MLIRContext &ctx;
 };
 
 //===----------------------------------------------------------------------===//
+// PDL Configuration
+//===----------------------------------------------------------------------===//
+
+/// A PDL configuration that is used to supported dialect conversion
+/// functionality.
+class PDLConversionConfig final
+    : public PDLPatternConfigBase<PDLConversionConfig> {
+public:
+  PDLConversionConfig(TypeConverter *converter) : converter(converter) {}
+  ~PDLConversionConfig() final = default;
+
+  /// Return the type converter used by this configuration, which may be nullptr
+  /// if no type conversions are expected.
+  TypeConverter *getTypeConverter() const { return converter; }
+
+  /// Hooks that are invoked at the beginning and end of a rewrite of a matched
+  /// pattern.
+  void notifyRewriteBegin(PatternRewriter &rewriter) final;
+  void notifyRewriteEnd(PatternRewriter &rewriter) final;
+
+private:
+  /// An optional type converter to use for the pattern.
+  TypeConverter *converter;
+};
+
+/// Register the dialect conversion PDL functions with the given pattern set.
+void registerConversionPDLFunctions(RewritePatternSet &patterns);
+
+//===----------------------------------------------------------------------===//
 // Op Conversion Entry Points
 //===----------------------------------------------------------------------===//
 
 /// Below we define several entry points for operation conversion. It is
 /// important to note that the patterns provided to the conversion framework may
 /// have additional constraints. See the `PatternRewriter Hooks` section of the
 /// ConversionPatternRewriter, to see what additional constraints are imposed on
 /// the use of the PatternRewriter.

mlir/include/mlir/Transforms/DialectConversion.pdll

+//===- DialectConversion.pdll - DialectConversion PDLL Support --*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file defines various utilities for interacting with dialect conversion
+// within PDLL.
+//
+//===----------------------------------------------------------------------===//
+
+/// This rewrite returns the converted value of `value`, whose type is defined 
+/// by the type converted specified in the `PDLConversionConfig` of the current
+/// pattern.
+Rewrite convertValue(value: Value) -> Value;
+
+/// This rewrite returns the converted values of `values`, whose type is defined 
+/// by the type converted specified in the `PDLConversionConfig` of the current
+/// pattern.
+Rewrite convertValues(values: ValueRange) -> ValueRange;
+
+/// This rewrite returns the converted type of `type` as defined by the type
+/// converted specified in the `PDLConversionConfig` of the current pattern.
+Rewrite convertType(type: Type) -> Type;
+
+/// This rewrite returns the converted types of `types` as defined by the type
+/// converted specified in the `PDLConversionConfig` of the current pattern.
+Rewrite convertTypes(types: TypeRange) -> TypeRange;

mlir/lib/Conversion/PDLToPDLInterp/PDLToPDLInterp.cpp

 // PatternLowering
 //===----------------------------------------------------------------------===//
 
 namespace {
 /// This class generators operations within the PDL Interpreter dialect from a
 /// given module containing PDL pattern operations.
 struct PatternLowering {
 public:
-  PatternLowering(pdl_interp::FuncOp matcherFunc, ModuleOp rewriterModule);
+  PatternLowering(pdl_interp::FuncOp matcherFunc, ModuleOp rewriterModule,
+                  DenseMap<Operation *, PDLPatternConfigSet *> *configMap);
 
   /// Generate code for matching and rewriting based on the pattern operations
   /// within the module.
   void lower(ModuleOp module);
 
 private:
   using ValueMap = llvm::ScopedHashTable<Position *, Value>;
   using ValueMapScope = llvm::ScopedHashTableScope<Position *, Value>;
private:
 
   /// A mapping between values defined in a pattern match, and the corresponding
   /// positional value.
   DenseMap<Value, Position *> valueToPosition;
 
   /// The set of operation values whose whose location will be used for newly
   /// generated operations.
   SetVector<Value> locOps;
+
+  /// A mapping between pattern operations and the corresponding configuration
+  /// set.
+  DenseMap<Operation *, PDLPatternConfigSet *> *configMap;
 };
 } // namespace
 
-PatternLowering::PatternLowering(pdl_interp::FuncOp matcherFunc,
-                                 ModuleOp rewriterModule)
+PatternLowering::PatternLowering(
+    pdl_interp::FuncOp matcherFunc, ModuleOp rewriterModule,
+    DenseMap<Operation *, PDLPatternConfigSet *> *configMap)
     : builder(matcherFunc.getContext()), matcherFunc(matcherFunc),
-      rewriterModule(rewriterModule), rewriterSymbolTable(rewriterModule) {}
+      rewriterModule(rewriterModule), rewriterSymbolTable(rewriterModule),
+      configMap(configMap) {}
 
 void PatternLowering::lower(ModuleOp module) {
   PredicateUniquer predicateUniquer;
   PredicateBuilder predicateBuilder(predicateUniquer, module.getContext());
 
   // Define top-level scope for the arguments to the matcher function.
   ValueMapScope topLevelValueScope(values);
 
void PatternLowering::generate(SuccessNode *successNode, Block *&currentBlock) {
 
   // Grab the root kind if present.
   StringAttr rootKindAttr;
   if (pdl::OperationOp rootOp = root.getDefiningOp<pdl::OperationOp>())
     if (Optional<StringRef> rootKind = rootOp.name())
       rootKindAttr = builder.getStringAttr(*rootKind);
 
   builder.setInsertionPointToEnd(currentBlock);
-  builder.create<pdl_interp::RecordMatchOp>(
+  auto matchOp = builder.create<pdl_interp::RecordMatchOp>(
       pattern.getLoc(), mappedMatchValues, locOps.getArrayRef(),
       rewriterFuncRef, rootKindAttr, generatedOpsAttr, pattern.benefitAttr(),
       failureBlockStack.back());
+
+  // Set the config of the lowered match to the parent pattern.
+  if (configMap)
+    configMap->try_emplace(matchOp, configMap->lookup(pattern));
 }
 
 SymbolRefAttr PatternLowering::generateRewriter(
     pdl::PatternOp pattern, SmallVectorImpl<Position *> &usedMatchValues) {
   builder.setInsertionPointToEnd(rewriterModule.getBody());
   auto rewriterFunc = builder.create<pdl_interp::FuncOp>(
       pattern.getLoc(), "pdl_generated_rewriter",
       builder.getFunctionType(llvm::None, llvm::None));

 
 //===----------------------------------------------------------------------===//
 // Conversion Pass
 //===----------------------------------------------------------------------===//
 
 namespace {
 struct PDLToPDLInterpPass
     : public impl::ConvertPDLToPDLInterpBase<PDLToPDLInterpPass> {
+  PDLToPDLInterpPass() = default;
+  PDLToPDLInterpPass(const PDLToPDLInterpPass &rhs) = default;
+  PDLToPDLInterpPass(DenseMap<Operation *, PDLPatternConfigSet *> &configMap)
+      : configMap(&configMap) {}
   void runOnOperation() final;
+
+  /// A map containing the configuration for each pattern.
+  DenseMap<Operation *, PDLPatternConfigSet *> *configMap = nullptr;
 };
 } // namespace
 
 /// Convert the given module containing PDL pattern operations into a PDL
 /// Interpreter operations.
 void PDLToPDLInterpPass::runOnOperation() {
   ModuleOp module = getOperation();
 
   // Create the main matcher function This function contains all of the match
   // related functionality from patterns in the module.
   OpBuilder builder = OpBuilder::atBlockBegin(module.getBody());
   auto matcherFunc = builder.create<pdl_interp::FuncOp>(
       module.getLoc(), pdl_interp::PDLInterpDialect::getMatcherFunctionName(),
       builder.getFunctionType(builder.getType<pdl::OperationType>(),
                               /*results=*/llvm::None),
       /*attrs=*/llvm::None);
 
   // Create a nested module to hold the functions invoked for rewriting the IR
   // after a successful match.
   ModuleOp rewriterModule = builder.create<ModuleOp>(
       module.getLoc(), pdl_interp::PDLInterpDialect::getRewriterModuleName());
 
   // Generate the code for the patterns within the module.
-  PatternLowering generator(matcherFunc, rewriterModule);
+  PatternLowering generator(matcherFunc, rewriterModule, configMap);
   generator.lower(module);
 
   // After generation, delete all of the pattern operations.
   for (pdl::PatternOp pattern :
-       llvm::make_early_inc_range(module.getOps<pdl::PatternOp>()))
+       llvm::make_early_inc_range(module.getOps<pdl::PatternOp>())) {
+    // Drop the now dead config mappings.
+    if (configMap)
+      configMap->erase(pattern);
+
     pattern.erase();
   }
+}
 
 std::unique_ptr<OperationPass<ModuleOp>> mlir::createPDLToPDLInterpPass() {
   return std::make_unique<PDLToPDLInterpPass>();
 }
+std::unique_ptr<OperationPass<ModuleOp>> mlir::createPDLToPDLInterpPass(
+    DenseMap<Operation *, PDLPatternConfigSet *> &configMap) {
+  return std::make_unique<PDLToPDLInterpPass>(configMap);
+}

mlir/lib/IR/PatternMatch.cpp

 // PDLPatternModule
 //===----------------------------------------------------------------------===//
 
 void PDLPatternModule::mergeIn(PDLPatternModule &&other) {
   // Ignore the other module if it has no patterns.
   if (!other.pdlModule)
     return;
 
-  // Steal the functions of the other module.
+  // Steal the functions and config of the other module.
   for (auto &it : other.constraintFunctions)
     registerConstraintFunction(it.first(), std::move(it.second));
   for (auto &it : other.rewriteFunctions)
     registerRewriteFunction(it.first(), std::move(it.second));
+  for (auto &it : other.configs)
+    configs.emplace_back(std::move(it));
+  for (auto &it : other.configMap)
+    configMap.insert(it);
 
   // Steal the other state if we have no patterns.
   if (!pdlModule) {
     pdlModule = std::move(other.pdlModule);
     return;
   }
 
   // Merge the pattern operations from the other module into this one.
   Block *block = pdlModule->getBody();
   block->getOperations().splice(block->end(),
                                 other.pdlModule->getBody()->getOperations());
 }
 
+void PDLPatternModule::attachConfigToPatterns(ModuleOp module,
+                                              PDLPatternConfigSet &configSet) {
+  // Attach the configuration to the symbols within the module. We only add
+  // to symbols to avoid hardcoding any specific operation names here (given
+  // that we don't depend on any PDL dialect). We can't use
+  // cast<SymbolOpInterface> here because patterns may be optional symbols.
+  module->walk([&](Operation *op) {
+    if (op->hasTrait<SymbolOpInterface::Trait>())
+      configMap[op] = &configSet;
+  });
+}
+
 //===----------------------------------------------------------------------===//
 // Function Registry
 
 void PDLPatternModule::registerConstraintFunction(
     StringRef name, PDLConstraintFunction constraintFn) {
   // TODO: Is it possible to diagnose when `name` is already registered to
   // a function that is not equivalent to `constraintFn`?
   // Allow existing mappings in the case multiple patterns depend on the same

mlir/lib/Rewrite/ByteCode.h

 //===----------------------------------------------------------------------===//
 // PDLByteCodePattern
 //===----------------------------------------------------------------------===//
 
 /// All of the data pertaining to a specific pattern within the bytecode.
 class PDLByteCodePattern : public Pattern {
 public:
   static PDLByteCodePattern create(pdl_interp::RecordMatchOp matchOp,
+                                   PDLPatternConfigSet *configSet,
                                    ByteCodeAddr rewriterAddr);
 
   /// Return the bytecode address of the rewriter for this pattern.
   ByteCodeAddr getRewriterAddr() const { return rewriterAddr; }
 
+  /// Return the configuration set for this pattern, or null if there is none.
+  PDLPatternConfigSet *getConfigSet() const { return configSet; }
+
 private:
   template <typename... Args>
-  PDLByteCodePattern(ByteCodeAddr rewriterAddr, Args &&...patternArgs)
-      : Pattern(std::forward<Args>(patternArgs)...),
-        rewriterAddr(rewriterAddr) {}
+  PDLByteCodePattern(ByteCodeAddr rewriterAddr, PDLPatternConfigSet *configSet,
+                     Args &&...patternArgs)
+      : Pattern(std::forward<Args>(patternArgs)...), rewriterAddr(rewriterAddr),
+        configSet(configSet) {}
 
   /// The address of the rewriter for this pattern.
   ByteCodeAddr rewriterAddr;
+
+  /// The optional config set for this pattern.
+  PDLPatternConfigSet *configSet;
 };
 
 //===----------------------------------------------------------------------===//
 // PDLByteCodeMutableState
 //===----------------------------------------------------------------------===//
 
 /// This class contains the mutable state of a bytecode instance. This allows
 /// for a bytecode instance to be cached and reused across various different
struct MatchResult {
     const PDLByteCodePattern *pattern;
     /// The current benefit of the pattern that was matched.
     PatternBenefit benefit;
   };
 
   /// Create a ByteCode instance from the given module containing operations in
   /// the PDL interpreter dialect.
   PDLByteCode(ModuleOp module,
+              SmallVector<std::unique_ptr<PDLPatternConfigSet>> configs,
+              const DenseMap<Operation *, PDLPatternConfigSet *> &configMap,
               llvm::StringMap<PDLConstraintFunction> constraintFns,
               llvm::StringMap<PDLRewriteFunction> rewriteFns);
 
   /// Return the patterns held by the bytecode.
   ArrayRef<PDLByteCodePattern> getPatterns() const { return patterns; }
 
   /// Initialize the given state such that it can be used to execute the current
   /// bytecode.
   void initializeMutableState(PDLByteCodeMutableState &state) const;
 
   /// Run the pattern matcher on the given root operation, collecting the
   /// matched patterns in `matches`.
   void match(Operation *op, PatternRewriter &rewriter,
              SmallVectorImpl<MatchResult> &matches,
              PDLByteCodeMutableState &state) const;
 
   /// Run the rewriter of the given pattern that was previously matched in
-  /// `match`.
-  void rewrite(PatternRewriter &rewriter, const MatchResult &match,
+  /// `match`. Returns if a failure was encountered during the rewrite.
+  LogicalResult rewrite(PatternRewriter &rewriter, const MatchResult &match,
                         PDLByteCodeMutableState &state) const;
 
 private:
   /// Execute the given byte code starting at the provided instruction `inst`.
   /// `matches` is an optional field provided when this function is executed in
   /// a matching context.
   void executeByteCode(const ByteCodeField *inst, PatternRewriter &rewriter,
                        PDLByteCodeMutableState &state,
                        SmallVectorImpl<MatchResult> *matches) const;
 
+  /// The set of pattern configs referenced within the bytecode.
+  SmallVector<std::unique_ptr<PDLPatternConfigSet>> configs;
+
   /// A vector containing pointers to uniqued data. The storage is intentionally
   /// opaque such that we can store a wide range of data types. The types of
   /// data stored here include:
   ///  * Attribute, OperationName, Type
   std::vector<const void *> uniquedData;
 
   /// A vector containing the generated bytecode for the matcher.
   SmallVector<ByteCodeField, 64> matcherByteCode;

mlir/lib/Rewrite/ByteCode.cpp

 using namespace mlir;
 using namespace mlir::detail;
 
 //===----------------------------------------------------------------------===//
 // PDLByteCodePattern
 //===----------------------------------------------------------------------===//
 
 PDLByteCodePattern PDLByteCodePattern::create(pdl_interp::RecordMatchOp matchOp,
+                                              PDLPatternConfigSet *configSet,
                                               ByteCodeAddr rewriterAddr) {
+  PatternBenefit benefit = matchOp.getBenefit();
+  MLIRContext *ctx = matchOp.getContext();
+
+  // Collect the set of generated operations.
   SmallVector<StringRef, 8> generatedOps;
   if (ArrayAttr generatedOpsAttr = matchOp.getGeneratedOpsAttr())
     generatedOps =
         llvm::to_vector<8>(generatedOpsAttr.getAsValueRange<StringAttr>());
 
-  PatternBenefit benefit = matchOp.getBenefit();
-  MLIRContext *ctx = matchOp.getContext();
-
   // Check to see if this is pattern matches a specific operation type.
   if (Optional<StringRef> rootKind = matchOp.getRootKind())
-    return PDLByteCodePattern(rewriterAddr, *rootKind, benefit, ctx,
-                              generatedOps);
-  return PDLByteCodePattern(rewriterAddr, MatchAnyOpTypeTag(), benefit, ctx,
+    return PDLByteCodePattern(rewriterAddr, configSet, *rootKind, benefit, ctx,
                               generatedOps);
+  return PDLByteCodePattern(rewriterAddr, configSet, MatchAnyOpTypeTag(),
+                            benefit, ctx, generatedOps);
 }
 
 //===----------------------------------------------------------------------===//
 // PDLByteCodeMutableState
 //===----------------------------------------------------------------------===//
 
 /// Set the new benefit for a bytecode pattern. The `patternIndex` corresponds
 /// to the position of the pattern within the range returned by
Generator(MLIRContext *ctx, std::vector<const void *> &uniquedData,
             SmallVectorImpl<ByteCodeField> &rewriterByteCode,
             SmallVectorImpl<PDLByteCodePattern> &patterns,
             ByteCodeField &maxValueMemoryIndex,
             ByteCodeField &maxOpRangeMemoryIndex,
             ByteCodeField &maxTypeRangeMemoryIndex,
             ByteCodeField &maxValueRangeMemoryIndex,
             ByteCodeField &maxLoopLevel,
             llvm::StringMap<PDLConstraintFunction> &constraintFns,
-            llvm::StringMap<PDLRewriteFunction> &rewriteFns)
+            llvm::StringMap<PDLRewriteFunction> &rewriteFns,
+            const DenseMap<Operation *, PDLPatternConfigSet *> &configMap)
       : ctx(ctx), uniquedData(uniquedData), matcherByteCode(matcherByteCode),
         rewriterByteCode(rewriterByteCode), patterns(patterns),
         maxValueMemoryIndex(maxValueMemoryIndex),
         maxOpRangeMemoryIndex(maxOpRangeMemoryIndex),
         maxTypeRangeMemoryIndex(maxTypeRangeMemoryIndex),
         maxValueRangeMemoryIndex(maxValueRangeMemoryIndex),
-        maxLoopLevel(maxLoopLevel) {
+        maxLoopLevel(maxLoopLevel), configMap(configMap) {
     for (const auto &it : llvm::enumerate(constraintFns))
       constraintToMemIndex.try_emplace(it.value().first(), it.index());
     for (const auto &it : llvm::enumerate(rewriteFns))
       externalRewriterToMemIndex.try_emplace(it.value().first(), it.index());
   }
 
   /// Generate the bytecode for the given PDL interpreter module.
   void generate(ModuleOp module);
private:
   SmallVectorImpl<ByteCodeField> &matcherByteCode;
   SmallVectorImpl<ByteCodeField> &rewriterByteCode;
   SmallVectorImpl<PDLByteCodePattern> &patterns;
   ByteCodeField &maxValueMemoryIndex;
   ByteCodeField &maxOpRangeMemoryIndex;
   ByteCodeField &maxTypeRangeMemoryIndex;
   ByteCodeField &maxValueRangeMemoryIndex;
   ByteCodeField &maxLoopLevel;
+
+  /// A map of pattern configurations.
+  const DenseMap<Operation *, PDLPatternConfigSet *> &configMap;
 };
 
 /// This class provides utilities for writing a bytecode stream.
 struct ByteCodeWriter {
   ByteCodeWriter(SmallVectorImpl<ByteCodeField> &bytecode, Generator &generator)
       : bytecode(bytecode), generator(generator) {}
 
   /// Append a field to the bytecode.
void Generator::generate(pdl_interp::GetValueTypeOp op,
   }
 }
 void Generator::generate(pdl_interp::IsNotNullOp op, ByteCodeWriter &writer) {
   writer.append(OpCode::IsNotNull, op.getValue(), op.getSuccessors());
 }
 void Generator::generate(pdl_interp::RecordMatchOp op, ByteCodeWriter &writer) {
   ByteCodeField patternIndex = patterns.size();
   patterns.emplace_back(PDLByteCodePattern::create(
-      op, rewriterToAddr[op.getRewriter().getLeafReference().getValue()]));
+      op, configMap.lookup(op),
+      rewriterToAddr[op.getRewriter().getLeafReference().getValue()]));
   writer.append(OpCode::RecordMatch, patternIndex,
                 SuccessorRange(op.getOperation()), op.getMatchedOps());
   writer.appendPDLValueList(op.getInputs());
 }
 void Generator::generate(pdl_interp::ReplaceOp op, ByteCodeWriter &writer) {
   writer.append(OpCode::ReplaceOp, op.getInputOp());
   writer.appendPDLValueList(op.getReplValues());
 }
void Generator::generate(pdl_interp::SwitchTypesOp op, ByteCodeWriter &writer) {
   writer.append(OpCode::SwitchTypes, op.getValue(), op.getCaseValuesAttr(),
                 op.getSuccessors());
 }
 
 //===----------------------------------------------------------------------===//
 // PDLByteCode
 //===----------------------------------------------------------------------===//
 
-PDLByteCode::PDLByteCode(ModuleOp module,
-                         llvm::StringMap<PDLConstraintFunction> constraintFns,
-                         llvm::StringMap<PDLRewriteFunction> rewriteFns) {
+PDLByteCode::PDLByteCode(
+    ModuleOp module, SmallVector<std::unique_ptr<PDLPatternConfigSet>> configs,
+    const DenseMap<Operation *, PDLPatternConfigSet *> &configMap,
+    llvm::StringMap<PDLRewriteFunction> rewriteFns)
+    : configs(std::move(configs)) {
   Generator generator(module.getContext(), uniquedData, matcherByteCode,
                       rewriterByteCode, patterns, maxValueMemoryIndex,
                       maxOpRangeCount, maxTypeRangeCount, maxValueRangeCount,
-                      maxLoopLevel, constraintFns, rewriteFns);
+                      maxLoopLevel, constraintFns, rewriteFns, configMap);
   generator.generate(module);
 
   // Initialize the external functions.
   for (auto &it : constraintFns)
     constraintFunctions.push_back(std::move(it.second));
   for (auto &it : rewriteFns)
     rewriteFunctions.push_back(std::move(it.second));
 }
ByteCodeExecutor(
         loopIndex(loopIndex), uniquedMemory(uniquedMemory), code(code),
         currentPatternBenefits(currentPatternBenefits), patterns(patterns),
         constraintFunctions(constraintFunctions),
         rewriteFunctions(rewriteFunctions) {}
 
   /// Start executing the code at the current bytecode index. `matches` is an
   /// optional field provided when this function is executed in a matching
   /// context.
-  void execute(PatternRewriter &rewriter,
-               SmallVectorImpl<PDLByteCode::MatchResult> *matches = nullptr,
-               Optional<Location> mainRewriteLoc = {});
+  LogicalResult
+  execute(PatternRewriter &rewriter,
 
 private:
   /// Internal implementation of executing each of the bytecode commands.
   void executeApplyConstraint(PatternRewriter &rewriter);
-  void executeApplyRewrite(PatternRewriter &rewriter);
+  LogicalResult executeApplyRewrite(PatternRewriter &rewriter);
   void executeAreEqual();
   void executeAreRangesEqual();
   void executeBranch();
   void executeCheckOperandCount();
   void executeCheckOperationName();
   void executeCheckResultCount();
   void executeCheckTypes();
   void executeContinue();
LLVM_DEBUG({
     llvm::dbgs() << "  * Arguments: ";
     llvm::interleaveComma(args, llvm::dbgs());
   });
 
   // Invoke the constraint and jump to the proper destination.
   selectJump(succeeded(constraintFn(rewriter, args)));
 }
 
-void ByteCodeExecutor::executeApplyRewrite(PatternRewriter &rewriter) {
+LogicalResult ByteCodeExecutor::executeApplyRewrite(PatternRewriter &rewriter) {
   LLVM_DEBUG(llvm::dbgs() << "Executing ApplyRewrite:\n");
   const PDLRewriteFunction &rewriteFn = rewriteFunctions[read()];
   SmallVector<PDLValue, 16> args;
   readList<PDLValue>(args);
 
   LLVM_DEBUG({
     llvm::dbgs() << "  * Arguments: ";
     llvm::interleaveComma(args, llvm::dbgs());
   });
 
   // Execute the rewrite function.
   ByteCodeField numResults = read();
   ByteCodeRewriteResultList results(numResults);
-  rewriteFn(rewriter, results, args);
+  LogicalResult rewriteResult = rewriteFn(rewriter, results, args);
 
   assert(results.getResults().size() == numResults &&
          "native PDL rewrite function returned unexpected number of results");
 
   // Store the results in the bytecode memory.
   for (PDLValue &result : results.getResults()) {
     LLVM_DEBUG(llvm::dbgs() << "  * Result: " << result << "\n");
 
#endif
     }
   }
 
   // Copy over any underlying storage allocated for result ranges.
   for (auto &it : results.getAllocatedTypeRanges())
     allocatedTypeRangeMemory.push_back(std::move(it));
   for (auto &it : results.getAllocatedValueRanges())
     allocatedValueRangeMemory.push_back(std::move(it));
+
+  // Process the result of the rewrite.
+  if (failed(rewriteResult)) {
+    LLVM_DEBUG(llvm::dbgs() << "  - Failed");
+    return failure();
+  }
+  return success();
 }
 
 void ByteCodeExecutor::executeAreEqual() {
   LLVM_DEBUG(llvm::dbgs() << "Executing AreEqual:\n");
   const void *lhs = read<const void *>();
   const void *rhs = read<const void *>();
 
   LLVM_DEBUG(llvm::dbgs() << "  * " << lhs << " == " << rhs << "\n");
if (!value) {
     LLVM_DEBUG(llvm::dbgs() << "Types: <NULL>\n");
     return selectJump(size_t(0));
   }
   handleSwitch(*value, cases, [](ArrayAttr caseValue, const TypeRange &value) {
     return value == caseValue.getAsValueRange<TypeAttr>();
   });
 }
 
-void ByteCodeExecutor::execute(
-    PatternRewriter &rewriter,
+LogicalResult
+ByteCodeExecutor::execute(PatternRewriter &rewriter,
                           SmallVectorImpl<PDLByteCode::MatchResult> *matches,
                           Optional<Location> mainRewriteLoc) {
   while (true) {
     // Print the location of the operation being executed.
     LLVM_DEBUG(llvm::dbgs() << readInline<Location>() << "\n");
 
     OpCode opCode = static_cast<OpCode>(read());
     switch (opCode) {
     case ApplyConstraint:
       executeApplyConstraint(rewriter);
       break;
     case ApplyRewrite:
-      executeApplyRewrite(rewriter);
+      if (failed(executeApplyRewrite(rewriter)))
+        return failure();
       break;
     case AreEqual:
       executeAreEqual();
       break;
     case AreRangesEqual:
       executeAreRangesEqual();
       break;
     case Branch:
case ExtractType:
       executeExtract<Type, TypeRange, PDLValue::Kind::Type>();
       break;
     case ExtractValue:
       executeExtract<Value, ValueRange, PDLValue::Kind::Value>();
       break;
     case Finalize:
       executeFinalize();
       LLVM_DEBUG(llvm::dbgs() << "\n");
-      return;
+      return success();
     case ForEach:
       executeForEach();
       break;
     case GetAttribute:
       executeGetAttribute();
       break;
     case GetAttributeType:
       executeGetAttributeType();
while (true) {
     case SwitchTypes:
       executeSwitchTypes();
       break;
     }
     LLVM_DEBUG(llvm::dbgs() << "\n");
   }
 }
 
-/// Run the pattern matcher on the given root operation, collecting the matched
-/// patterns in `matches`.
 void PDLByteCode::match(Operation *op, PatternRewriter &rewriter,
                         SmallVectorImpl<MatchResult> &matches,
                         PDLByteCodeMutableState &state) const {
   // The first memory slot is always the root operation.
   state.memory[0] = op;
 
   // The matcher function always starts at code address 0.
   ByteCodeExecutor executor(
       matcherByteCode.data(), state.memory, state.opRangeMemory,
       state.typeRangeMemory, state.allocatedTypeRangeMemory,
       state.valueRangeMemory, state.allocatedValueRangeMemory, state.loopIndex,
       uniquedData, matcherByteCode, state.currentPatternBenefits, patterns,
       constraintFunctions, rewriteFunctions);
-  executor.execute(rewriter, &matches);
+  LogicalResult executeResult = executor.execute(rewriter, &matches);
+  assert(succeeded(executeResult) && "unexpected matcher execution failure");
 
   // Order the found matches by benefit.
   std::stable_sort(matches.begin(), matches.end(),
                    [](const MatchResult &lhs, const MatchResult &rhs) {
                      return lhs.benefit > rhs.benefit;
                    });
 }
 
-/// Run the rewriter of the given pattern on the root operation `op`.
-void PDLByteCode::rewrite(PatternRewriter &rewriter, const MatchResult &match,
+LogicalResult PDLByteCode::rewrite(PatternRewriter &rewriter,
+                                   const MatchResult &match,
                                    PDLByteCodeMutableState &state) const {
+  auto *configSet = match.pattern->getConfigSet();
+  if (configSet)
+    configSet->notifyRewriteBegin(rewriter);
+
   // The arguments of the rewrite function are stored at the start of the
   // memory buffer.
   llvm::copy(match.values, state.memory.begin());
 
   ByteCodeExecutor executor(
       &rewriterByteCode[match.pattern->getRewriterAddr()], state.memory,
       state.opRangeMemory, state.typeRangeMemory,
       state.allocatedTypeRangeMemory, state.valueRangeMemory,
       state.allocatedValueRangeMemory, state.loopIndex, uniquedData,
       rewriterByteCode, state.currentPatternBenefits, patterns,
       constraintFunctions, rewriteFunctions);
+  LogicalResult result =
       executor.execute(rewriter, /*matches=*/nullptr, match.location);
+
+  if (configSet)
+    configSet->notifyRewriteEnd(rewriter);
+
+  // If the rewrite failed, check if the pattern rewriter can recover. If it
+  // can, we can signal to the pattern applicator to keep trying patterns. If it
+  // doesn't, we need to bail. Bailing here should be fine, given that we have
+  // no means to propagate such a failure to the user, and it also indicates a
+  // bug in the user code (i.e. failable rewrites should not be used with
+  // pattern rewriters that don't support it).
+  if (failed(result) && !rewriter.canRecoverFromRewriteFailure()) {
+    LLVM_DEBUG(llvm::dbgs() << " and rollback is not supported - aborting");
+    llvm::report_fatal_error(
+        "Native PDL Rewrite failed, but the pattern "
+        "rewriter doesn't support recovery. Failable pattern rewrites should "
+        "not be used with pattern rewriters that do not support them.");
+  }
+  return result;
 }

mlir/lib/Rewrite/FrozenRewritePatternSet.cpp

 #include "mlir/Conversion/PDLToPDLInterp/PDLToPDLInterp.h"
 #include "mlir/Dialect/PDL/IR/PDLOps.h"
 #include "mlir/Interfaces/SideEffectInterfaces.h"
 #include "mlir/Pass/Pass.h"
 #include "mlir/Pass/PassManager.h"
 
 using namespace mlir;
 
-static LogicalResult convertPDLToPDLInterp(ModuleOp pdlModule) {
+static LogicalResult
+convertPDLToPDLInterp(ModuleOp pdlModule,
+                      DenseMap<Operation *, PDLPatternConfigSet *> &configMap) {
   // Skip the conversion if the module doesn't contain pdl.
   if (llvm::empty(pdlModule.getOps<pdl::PatternOp>()))
     return success();
 
   // Simplify the provided PDL module. Note that we can't use the canonicalizer
   // here because it would create a cyclic dependency.
   auto simplifyFn = [](Operation *op) {
     // TODO: Add folding here if ever necessary.
     if (isOpTriviallyDead(op))
       op->erase();
   };
   pdlModule.getBody()->walk(simplifyFn);
 
   /// Lower the PDL pattern module to the interpreter dialect.
   PassManager pdlPipeline(pdlModule.getContext());
 #ifdef NDEBUG
   // We don't want to incur the hit of running the verifier when in release
   // mode.
   pdlPipeline.enableVerifier(false);
 #endif
-  pdlPipeline.addPass(createPDLToPDLInterpPass());
+  pdlPipeline.addPass(createPDLToPDLInterpPass(configMap));
   if (failed(pdlPipeline.run(pdlModule)))
     return failure();
 
   // Simplify again after running the lowering pipeline.
   pdlModule.getBody()->walk(simplifyFn);
   return success();
 }
 
for (std::unique_ptr<RewritePattern> &pat : patterns.getNativePatterns()) {
     impl->nativeAnyOpPatterns.push_back(std::move(pat));
   }
 
   // Generate the bytecode for the PDL patterns if any were provided.
   PDLPatternModule &pdlPatterns = patterns.getPDLPatterns();
   ModuleOp pdlModule = pdlPatterns.getModule();
   if (!pdlModule)
     return;
-  if (failed(convertPDLToPDLInterp(pdlModule)))
+  DenseMap<Operation *, PDLPatternConfigSet *> configMap =
+      pdlPatterns.takeConfigMap();
+  if (failed(convertPDLToPDLInterp(pdlModule, configMap)))
     llvm::report_fatal_error(
         "failed to lower PDL pattern module to the PDL Interpreter");
 
   // Generate the pdl bytecode.
   impl->pdlByteCode = std::make_unique<detail::PDLByteCode>(
-      pdlModule, pdlPatterns.takeConstraintFunctions(),
+      pdlModule, pdlPatterns.takeConfigs(), configMap,
+      pdlPatterns.takeConstraintFunctions(),
       pdlPatterns.takeRewriteFunctions());
 }
 
 FrozenRewritePatternSet::~FrozenRewritePatternSet() = default;

mlir/lib/Rewrite/PatternApplicator.cpp

do {
     // benefit, so if we match we can immediately rewrite. For PDL patterns, the
     // match has already been performed, we just need to rewrite.
     rewriter.setInsertionPoint(op);
 #ifndef NDEBUG
     // Operation `op` may be invalidated after applying the rewrite pattern.
     Operation *dumpRootOp = getDumpRootOp(op);
 #endif
     if (pdlMatch) {
-      bytecode->rewrite(rewriter, *pdlMatch, *mutableByteCodeState);
-      result = success(!onSuccess || succeeded(onSuccess(*bestPattern)));
+      result = bytecode->rewrite(rewriter, *pdlMatch, *mutableByteCodeState);
     } else {
-      const auto *pattern = static_cast<const RewritePattern *>(bestPattern);
+      LLVM_DEBUG(llvm::dbgs() << "Trying to match \""
+                              << bestPattern->getDebugName() << "\"\n");
 
-      LLVM_DEBUG(llvm::dbgs()
-                 << "Trying to match \"" << pattern->getDebugName() << "\"\n");
+      const auto *pattern = static_cast<const RewritePattern *>(bestPattern);
       result = pattern->matchAndRewrite(op, rewriter);
-      LLVM_DEBUG(llvm::dbgs() << "\"" << pattern->getDebugName() << "\" result "
-                              << succeeded(result) << "\n");
 
-      if (succeeded(result) && onSuccess && failed(onSuccess(*pattern)))
-        result = failure();
+      LLVM_DEBUG(llvm::dbgs() << "\"" << bestPattern->getDebugName()
+                              << "\" result " << succeeded(result) << "\n");
     }
+
+    // Process the result of the pattern application.
+    if (succeeded(result) && onSuccess && failed(onSuccess(*bestPattern)))
+      result = failure();
     if (succeeded(result)) {
       LLVM_DEBUG(logSucessfulPatternApplication(dumpRootOp));
       break;
     }
 
     // Perform any necessary cleanups.
     if (onFailure)
       onFailure(*bestPattern);
   } while (true);
 
   if (mutableByteCodeState)
     mutableByteCodeState->cleanupAfterMatchAndRewrite();
   return result;
 }

mlir/lib/Tools/PDLL/CodeGen/CPPGen.cpp

if (Optional<StringRef> patternName = pattern.sym_name()) {
       } while (!patternNames.insert(name));
     }
 
     generate(pattern, patternNames.back(), nativeFunctions);
   }
   os << "} // end namespace\n\n";
 
   // Emit function to add the generated matchers to the pattern list.
-  os << "static void LLVM_ATTRIBUTE_UNUSED populateGeneratedPDLLPatterns("
-        "::mlir::RewritePatternSet &patterns) {\n";
+  os << "template <typename... ConfigsT>\n"
+        "static void LLVM_ATTRIBUTE_UNUSED populateGeneratedPDLLPatterns("
+        "::mlir::RewritePatternSet &patterns, ConfigsT &&...configs) {\n";
   for (const auto &name : patternNames)
-    os << "  patterns.add<" << name << ">(patterns.getContext());\n";
+    os << "  patterns.add<" << name
+       << ">(patterns.getContext(), configs...);\n";
   os << "}\n";
 }
 
 void CodeGen::generate(pdl::PatternOp pattern, StringRef patternName,
                        StringSet<> &nativeFunctions) {
   const char *patternClassStartStr = R"(
 struct {0} : ::mlir::PDLPatternModule {{
-  {0}(::mlir::MLIRContext *context)
+  template <typename... ConfigsT>
+  {0}(::mlir::MLIRContext *context, ConfigsT &&...configs)
     : ::mlir::PDLPatternModule(::mlir::parseSourceString<::mlir::ModuleOp>(
 )";
   os << llvm::formatv(patternClassStartStr, patternName);
 
   os << "R\"mlir(";
   pattern->print(os, OpPrintingFlags().enableDebugInfo());
-  os << "\n    )mlir\", context)) {\n";
+  os << "\n    )mlir\", context), std::forward<ConfigsT>(configs)...) {\n";
 
   // Register any native functions used within the pattern.
   StringSet<> registeredNativeFunctions;
   auto checkRegisterNativeFn = [&](StringRef fnName, StringRef fnType) {
     if (!nativeFunctions.count(fnName) ||
         !registeredNativeFunctions.insert(fnName).second)
       return;
     os << "    register" << fnType << "Function(\"" << fnName << "\", "

mlir/lib/Transforms/Utils/DialectConversion.cpp

auto ConversionTarget::getOpInfo(OperationName op) const
   // Otherwise, check if we mark unknown operations as dynamic.
   if (unknownLegalityFn)
     return LegalizationInfo{LegalizationAction::Dynamic,
                             /*isRecursivelyLegal=*/false, unknownLegalityFn};
   return llvm::None;
 }
 
 //===----------------------------------------------------------------------===//
+// PDL Configuration
+//===----------------------------------------------------------------------===//
+
+void PDLConversionConfig::notifyRewriteBegin(PatternRewriter &rewriter) {
+  auto &rewriterImpl =
+      static_cast<ConversionPatternRewriter &>(rewriter).getImpl();
+  rewriterImpl.currentTypeConverter = getTypeConverter();
+}
+
+void PDLConversionConfig::notifyRewriteEnd(PatternRewriter &rewriter) {
+  auto &rewriterImpl =
+      static_cast<ConversionPatternRewriter &>(rewriter).getImpl();
+  rewriterImpl.currentTypeConverter = nullptr;
+}
+
+/// Remap the given value using the rewriter and the type converter in the
+/// provided config.
+static FailureOr<SmallVector<Value>>
+pdllConvertValues(ConversionPatternRewriter &rewriter, ValueRange values) {
+  SmallVector<Value> mappedValues;
+  if (failed(rewriter.getRemappedValues(values, mappedValues)))
+    return failure();
+  return std::move(mappedValues);
+}
+
+void mlir::registerConversionPDLFunctions(RewritePatternSet &patterns) {
+  patterns.getPDLPatterns().registerRewriteFunction(
+      "convertValue",
+      [](PatternRewriter &rewriter, Value value) -> FailureOr<Value> {
+        auto results = pdllConvertValues(
+            static_cast<ConversionPatternRewriter &>(rewriter), value);
+        if (failed(results))
+          return failure();
+        return results->front();
+      });
+  patterns.getPDLPatterns().registerRewriteFunction(
+      "convertValues", [](PatternRewriter &rewriter, ValueRange values) {
+        return pdllConvertValues(
+            static_cast<ConversionPatternRewriter &>(rewriter), values);
+      });
+  patterns.getPDLPatterns().registerRewriteFunction(
+      "convertType",
+      [](PatternRewriter &rewriter, Type type) -> FailureOr<Type> {
+        auto &rewriterImpl =
+            static_cast<ConversionPatternRewriter &>(rewriter).getImpl();
+        if (TypeConverter *converter = rewriterImpl.currentTypeConverter) {
+          if (Type newType = converter->convertType(type))
+            return newType;
+          return failure();
+        }
+        return type;
+      });
+  patterns.getPDLPatterns().registerRewriteFunction(
+      "convertTypes",
+      [](PatternRewriter &rewriter,
+         TypeRange types) -> FailureOr<SmallVector<Type>> {
+        auto &rewriterImpl =
+            static_cast<ConversionPatternRewriter &>(rewriter).getImpl();
+        TypeConverter *converter = rewriterImpl.currentTypeConverter;
+        if (!converter)
+          return SmallVector<Type>(types);
+
+        SmallVector<Type> remappedTypes;
+        if (failed(converter->convertTypes(types, remappedTypes)))
+          return failure();
+        return std::move(remappedTypes);
+      });
+}
+
+//===----------------------------------------------------------------------===//
 // Op Conversion Entry Points
 //===----------------------------------------------------------------------===//
 
 //===----------------------------------------------------------------------===//
 // Partial Conversion
 
 LogicalResult
 mlir::applyPartialConversion(ArrayRef<Operation *> ops,

mlir/test/Transforms/test-dialect-conversion-pdll.mlir

+// RUN: mlir-opt %s -test-dialect-conversion-pdll | FileCheck %s
+
+// CHECK-LABEL: @TestSingleConversion
+func.func @TestSingleConversion() {
+  // CHECK: %[[CAST:.*]] = "test.cast"() : () -> f64
+  // CHECK-NEXT: "test.return"(%[[CAST]]) : (f64) -> ()
+  %result = "test.cast"() : () -> (i64)
+  "test.return"(%result) : (i64) -> ()
+}
+
+// CHECK-LABEL: @TestLingeringConversion
+func.func @TestLingeringConversion() -> i64 {
+  // CHECK: %[[ORIG_CAST:.*]] = "test.cast"() : () -> f64
+  // CHECK: %[[MATERIALIZE_CAST:.*]] = builtin.unrealized_conversion_cast %[[ORIG_CAST]] : f64 to i64
+  // CHECK-NEXT: return %[[MATERIALIZE_CAST]] : i64
+  %result = "test.cast"() : () -> (i64)
+  return %result : i64
+}

mlir/test/lib/Transforms/CMakeLists.txt

+add_mlir_pdll_library(MLIRTestDialectConversionPDLLPatternsIncGen
+  TestDialectConversion.pdll
+  TestDialectConversionPDLLPatterns.h.inc
+
+  EXTRA_INCLUDES
+  ${CMAKE_CURRENT_SOURCE_DIR}/../Dialect/Test
+  ${CMAKE_CURRENT_BINARY_DIR}/../Dialect/Test
+  )
+
 # Exclude tests from libMLIR.so
 add_mlir_library(MLIRTestTransforms
   TestCommutativityUtils.cpp
   TestConstantFold.cpp
   TestControlFlowSink.cpp
+  TestDialectConversion.cpp
   TestInlining.cpp
   TestIntRangeInference.cpp
   TestTopologicalSort.cpp
 
   EXCLUDE_FROM_LIBMLIR
 
   ADDITIONAL_HEADER_DIRS
   ${MLIR_MAIN_INCLUDE_DIR}/mlir/Transforms
 
+  DEPENDS
+  MLIRTestDialectConversionPDLLPatternsIncGen
+
   LINK_LIBS PUBLIC
   MLIRAnalysis
+  MLIRFuncDialect
   MLIRInferIntRangeInterface
   MLIRTestDialect
   MLIRTransforms
   )
 
 target_include_directories(MLIRTestTransforms
   PRIVATE
   ${CMAKE_CURRENT_SOURCE_DIR}/../Dialect/Test
   ${CMAKE_CURRENT_BINARY_DIR}/../Dialect/Test
   )

mlir/test/lib/Transforms/TestDialectConversion.cpp

+//===- TestDialectConversion.cpp - Test DialectConversion functionality ---===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "TestDialect.h"
+#include "mlir/Dialect/Func/IR/FuncOps.h"
+#include "mlir/Dialect/PDL/IR/PDL.h"
+#include "mlir/Dialect/PDLInterp/IR/PDLInterp.h"
+#include "mlir/Parser/Parser.h"
+#include "mlir/Pass/Pass.h"
+#include "mlir/Pass/PassManager.h"
+#include "mlir/Transforms/DialectConversion.h"
+
+using namespace mlir;
+using namespace test;
+
+//===----------------------------------------------------------------------===//
+// Test PDLL Support
+//===----------------------------------------------------------------------===//
+
+#include "TestDialectConversionPDLLPatterns.h.inc"
+
+namespace {
+struct PDLLTypeConverter : public TypeConverter {
+  PDLLTypeConverter() {
+    addConversion(convertType);
+    addArgumentMaterialization(materializeCast);
+    addSourceMaterialization(materializeCast);
+  }
+
+  static LogicalResult convertType(Type t, SmallVectorImpl<Type> &results) {
+    // Convert I64 to F64.
+    if (t.isSignlessInteger(64)) {
+      results.push_back(FloatType::getF64(t.getContext()));
+      return success();
+    }
+
+    // Otherwise, convert the type directly.
+    results.push_back(t);
+    return success();
+  }
+  /// Hook for materializing a conversion.
+  static Optional<Value> materializeCast(OpBuilder &builder, Type resultType,
+                                         ValueRange inputs, Location loc) {
+    return builder.create<UnrealizedConversionCastOp>(loc, resultType, inputs)
+        .getResult(0);
+  }
+};
+
+struct TestDialectConversionPDLLPass
+    : public PassWrapper<TestDialectConversionPDLLPass, OperationPass<>> {
+  MLIR_DEFINE_EXPLICIT_INTERNAL_INLINE_TYPE_ID(TestDialectConversionPDLLPass)
+
+  StringRef getArgument() const final { return "test-dialect-conversion-pdll"; }
+  StringRef getDescription() const final {
+    return "Test DialectConversion PDLL functionality";
+  }
+  void getDependentDialects(DialectRegistry &registry) const override {
+    registry.insert<pdl::PDLDialect, pdl_interp::PDLInterpDialect>();
+  }
+  LogicalResult initialize(MLIRContext *ctx) override {
+    // Build the pattern set within the `initialize` to avoid recompiling PDL
+    // patterns during each `runOnOperation` invocation.
+    RewritePatternSet patternList(ctx);
+    registerConversionPDLFunctions(patternList);
+    populateGeneratedPDLLPatterns(patternList, PDLConversionConfig(&converter));
+    patterns = std::move(patternList);
+    return success();
+  }
+
+  void runOnOperation() final {
+    mlir::ConversionTarget target(getContext());
+    target.addLegalOp<ModuleOp, func::FuncOp, func::ReturnOp>();
+    target.addDynamicallyLegalDialect<TestDialect>(
+        [this](Operation *op) { return converter.isLegal(op); });
+
+    if (failed(mlir::applyFullConversion(getOperation(), target, patterns)))
+      signalPassFailure();
+  }
+
+  FrozenRewritePatternSet patterns;
+  PDLLTypeConverter converter;
+};
+} // namespace
+
+namespace mlir {
+namespace test {
+void registerTestDialectConversionPasses() {
+  PassRegistration<TestDialectConversionPDLLPass>();
+}
+} // namespace test
+} // namespace mlir

mlir/test/lib/Transforms/TestDialectConversion.pdll

+//===- TestPDLL.pdll - Test PDLL functionality ----------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "TestOps.td"
+#include "mlir/Transforms/DialectConversion.pdll"
+
+/// Change the result type of a producer.
+// FIXME: We shouldn't need to specify arguments for the result cast.
+Pattern => replace op<test.cast>(args: ValueRange) -> (results: TypeRange)
+  with op<test.cast>(args) -> (convertTypes(results));
+
+/// Pass through test.return conversion.
+Pattern => replace op<test.return>(args: ValueRange)
+  with op<test.return>(convertValues(args));

mlir/test/lib/Transforms/lit.local.cfg

+config.suffixes.remove('.pdll')

mlir/test/mlir-pdll/CodeGen/CPP/general.pdll

 // RUN: mlir-pdll %s -I %S -split-input-file -x cpp | FileCheck %s
 
 // Check that we generate a wrapper pattern for each PDL pattern. Also
 // add in a pattern awkwardly named the same as our generated patterns to
 // check that we handle overlap.
 
 // CHECK: struct GeneratedPDLLPattern0 : ::mlir::PDLPatternModule {
+// CHECK:  template <typename... ConfigsT>
 // CHECK:  : ::mlir::PDLPatternModule(::mlir::parseSourceString<::mlir::ModuleOp>(
 // CHECK:  R"mlir(
 // CHECK:    pdl.pattern
 // CHECK:      operation "test.op"
-// CHECK:  )mlir", context))
+// CHECK:  )mlir", context), std::forward<ConfigsT>(configs)...)
 
 // CHECK: struct NamedPattern : ::mlir::PDLPatternModule {
 // CHECK:  : ::mlir::PDLPatternModule(::mlir::parseSourceString<::mlir::ModuleOp>(
 // CHECK:  R"mlir(
 // CHECK:    pdl.pattern
 // CHECK:      operation "test.op2"
-// CHECK:  )mlir", context))
+// CHECK:  )mlir", context), std::forward<ConfigsT>(configs)...)
 
 // CHECK: struct GeneratedPDLLPattern1 : ::mlir::PDLPatternModule {
 
 // CHECK: struct GeneratedPDLLPattern2 : ::mlir::PDLPatternModule {
 // CHECK:  : ::mlir::PDLPatternModule(::mlir::parseSourceString<::mlir::ModuleOp>(
 // CHECK:  R"mlir(
 // CHECK:    pdl.pattern
 // CHECK:      operation "test.op3"
-// CHECK:  )mlir", context))
+// CHECK:  )mlir", context), std::forward<ConfigsT>(configs)...)
 
-// CHECK:      static void LLVM_ATTRIBUTE_UNUSED populateGeneratedPDLLPatterns(::mlir::RewritePatternSet &patterns) {
-// CHECK-NEXT:   patterns.add<GeneratedPDLLPattern0>(patterns.getContext());
-// CHECK-NEXT:   patterns.add<NamedPattern>(patterns.getContext());
-// CHECK-NEXT:   patterns.add<GeneratedPDLLPattern1>(patterns.getContext());
-// CHECK-NEXT:   patterns.add<GeneratedPDLLPattern2>(patterns.getContext());
+// CHECK:      static void LLVM_ATTRIBUTE_UNUSED populateGeneratedPDLLPatterns(::mlir::RewritePatternSet &patterns, ConfigsT &&...configs) {
+// CHECK-NEXT:   patterns.add<GeneratedPDLLPattern0>(patterns.getContext(), configs...);
+// CHECK-NEXT:   patterns.add<NamedPattern>(patterns.getContext(), configs...);
+// CHECK-NEXT:   patterns.add<GeneratedPDLLPattern1>(patterns.getContext(), configs...);
+// CHECK-NEXT:   patterns.add<GeneratedPDLLPattern2>(patterns.getContext(), configs...);
 // CHECK-NEXT: }
 
 Pattern => erase op<test.op>;
 Pattern NamedPattern => erase op<test.op2>;
 Pattern GeneratedPDLLPattern1 => erase op<>;
 Pattern => erase op<test.op3>;
 
 // -----

mlir/tools/mlir-opt/mlir-opt.cpp

 void registerTestConstantFold();
 void registerTestControlFlowSink();
 void registerTestGpuSerializeToCubinPass();
 void registerTestGpuSerializeToHsacoPass();
 void registerTestDataLayoutQuery();
 void registerTestDeadCodeAnalysisPass();
 void registerTestDecomposeCallGraphTypes();
 void registerTestDiagnosticsPass();
+void registerTestDialectConversionPasses();
 void registerTestDominancePass();
 void registerTestDynamicPipelinePass();
 void registerTestExpandMathPass();
 void registerTestFooAnalysisPass();
 void registerTestComposeSubView();
 void registerTestMultiBuffering();
 void registerTestIntRangeInference();
 void registerTestIRVisitorsPass();
void registerTestPasses() {
   mlir::test::registerSimpleParametricTilingPass();
   mlir::test::registerTestAffineLoopParametricTilingPass();
   mlir::test::registerTestAliasAnalysisPass();
   mlir::test::registerTestBuiltinAttributeInterfaces();
   mlir::test::registerTestCallGraphPass();
   mlir::test::registerTestConstantFold();
   mlir::test::registerTestControlFlowSink();
   mlir::test::registerTestDiagnosticsPass();
+  mlir::test::registerTestDialectConversionPasses();
 #if MLIR_CUDA_CONVERSIONS_ENABLED
   mlir::test::registerTestGpuSerializeToCubinPass();
 #endif
 #if MLIR_ROCM_CONVERSIONS_ENABLED
   mlir::test::registerTestGpuSerializeToHsacoPass();
 #endif
   mlir::test::registerTestDecomposeCallGraphTypes();
   mlir::test::registerTestDataLayoutQuery();