[mlir] Allow for attaching external resources to .mlir files

[rriddle]

Summary

This commit enables support for providing and processing external
resources within MLIR assembly formats. This is a mechanism with which
dialects, and external clients, may attach additional information when
printing IR without that information being encoded in the IR itself.
External resources are not uniqued within the MLIR context, are not
attached directly to any operation, and are solely intended to live and be
processed outside of the immediate IR. There are many potential uses of this
functionality, for example MLIR's pass crash reproducer could utilize this to
attach the pass resource executing when a crash occurs. Other types of
uses may be embedding large amounts of binary data, such as weights in ML
applications, that shouldn't be copied directly into the MLIR context, but
need to be kept adjacent to the IR.

External resources are encoded using a key-value pair nested within a
dictionary anchored by name either on a dialect, or an externally registered
entity. The key is an identifier used to disambiguate the data. The value
may be stored in various limited forms, but general encodings use a string
(human readable) or blob format (binary). Within the textual format, an
example may be of the form:

mlir
{-#
  // The `dialect_resources` section within the file-level metadata
  // dictionary is used to contain any dialect resource entries.
  dialect_resources: {
    // Here is a dictionary anchored on "foo_dialect", which is a dialect
    // namespace.
    foo_dialect: {
      // `some_dialect_resource` is a key to be interpreted by the dialect,
      // and used to initialize/configure/etc.
      some_dialect_resource: "Some important resource value"
    }
  },
  // The `external_resources` section within the file-level metadata
  // dictionary is used to contain any non-dialect resource entries.
  external_resources: {
    // Here is a dictionary anchored on "mlir_reproducer", which is an
    // external entity representing MLIR's crash reproducer functionality.
    mlir_reproducer: {
      // `pipeline` is an entry that holds a crash reproducer pipeline
      // resource.
      pipeline: "func.func(canonicalize,cse)"
    }
  }

[mehdi_amini]

What about using directly YAML for this section?

[Mogball]

Doing it ourselves would keep MLIR light. And the external config API would just be built on top of YAML.

[Mogball]

mlir/include/mlir/IR/AsmState.h

57

Just curious, but what else could be stored here other than "external config"?

69

Is there any guarantee that dialect names won't collide with any random config key?

[rriddle]

In D126446#3539427, @mehdi_amini wrote:

What about using directly YAML for this section?

I considered, but ultimately it felt like more complexity than the benefits it brought (especially for this draft). Having something
just for us felt much simpler and more expressive (we could use Attributes/Types for config values if we wanted). I could likely be
convinced otherwise though, using something like YAML would conceptually make supporting more complex config values
easier (nested structs and stuff) if we wanted to go that approach. There are tradeoffs though (e.g. error reporting, attribute/type values,
still need a generic interface to enable seamless bitcode integration, etc.).

mlir/include/mlir/IR/AsmState.h

57

The idea for the top-level dictionary was to provide a convenient place to put things in the future. For example, this would be a natural place to put the dialect_versions metadata in Mehdi's WIP versioning patch: https://reviews.llvm.org/D117761

69

Non-dialect config keys are always wrapped with <>, so there won't be collision between the two.

[stellaraccident]

In D126446#3540627, @rriddle wrote:

In D126446#3539427, @mehdi_amini wrote:

What about using directly YAML for this section?

I considered, but ultimately it felt like more complexity than the benefits it brought (especially for this draft). Having something
just for us felt much simpler and more expressive (we could use Attributes/Types for config values if we wanted). I could likely be
convinced otherwise though, using something like YAML would conceptually make supporting more complex config values
easier (nested structs and stuff) if we wanted to go that approach. There are tradeoffs though (e.g. error reporting, attribute/type values,
still need a generic interface to enable seamless bitcode integration, etc.).

If the AsmConfig* classes were a bit more POD/model-ish, then it would be pretty easy to wrap YAML around that, but the way the implementation is laid out would be a bit of a weird fit (and I think I see why you have it the way you do). In general, I agree with the tradeoffs: it will feel and act like a complete side-thing. From my point of view, how this grows towards bitcode is more salient to me, and binding the text format close to YAML will just make it weird (all my subjective opinion based on trying to do such things before).

[mehdi_amini]

mlir/test/IR/file-metadata-config.mlir

17 ↗

(On Diff #433016)

I do have some concerns with this example (to the extent that this will be an example in-tree).

There is a problem in that we can't load in the same context some piece of IR if there was already another piece of IR using "blob1" loaded (even if the IR is deleted since).

[rriddle]

mlir/test/IR/file-metadata-config.mlir

17 ↗

(On Diff #433016)

Oh yeah, thanks for pointing this out. I originally wrote this quickly as a simple example, but forgot to update it. Do you have any suggestions on what we want to do here to set a good example? Compare the data? Just compare hashes?

[jfurtek]

Have you considered using base64 instead of hex for binary data?

[tschuett]

https://reviews.llvm.org/D126254 will bring base64 to LLVM and LLDB.

[bzcheeseman]

In D126446#3548652, @jfurtek wrote:

Have you considered using base64 instead of hex for binary data?

I would argue against base64 personally - I don't know if the benefits outweigh the issue that it inflates all data by 33%. It does beg the question though, of 'should we allow folks to choose some other encoding for data in this config struct?' If this is the only use case - that some folks want b64 data, then maybe it's better to just wait for the bitcode format and b64 encode the whole file if you want it to be Javascript-safe?

[jfurtek]

The current implementation uses hexadecimal, which requires 16 bits (2 bytes) to encode 8 bits of binary data. base64 would be an improvement over the current implementation that uses hexadecimal, requiring 8 bits to encode 6 bits of binary data. (The other constraint here is that encoding needs to be handled by the parser, which in the original example, means it can't have a quotation mark.) IMO, base64 is:

• smaller than hex
• somewhat ubiquitous
• compatible with the existing parser (I think?)
• a reasonable default for situations where the size overhead is an acceptable tradeoff for having the MLIR file be text-based and self-contained.

For text-based MLIR file situations where the size increase is an issue, I could imagine the config data containing a string with the name of an external file, and an application-defined way to resolve the full path. That could be out-of-tree and/or dialect specific.

[bzcheeseman]

In D126446#3553562, @jfurtek wrote:

The current implementation uses hexadecimal, which requires 16 bits (2 bytes) to encode 8 bits of binary data. base64 would be an improvement over the current implementation that uses hexadecimal, requiring 8 bits to encode 6 bits of binary data. (The other constraint here is that encoding needs to be handled by the parser, which in the original example, means it can't have a quotation mark.)

Good point! I was thinking of just the raw binary data, forgot that hex is silly that way :)

[rriddle]

In D126446#3553562, @jfurtek wrote:

The current implementation uses hexadecimal, which requires 16 bits (2 bytes) to encode 8 bits of binary data. base64 would be an improvement over the current implementation that uses hexadecimal, requiring 8 bits to encode 6 bits of binary data. (The other constraint here is that encoding needs to be handled by the parser, which in the original example, means it can't have a quotation mark.) IMO, base64 is:

• smaller than hex
• somewhat ubiquitous
• compatible with the existing parser (I think?)
• a reasonable default for situations where the size overhead is an acceptable tradeoff for having the MLIR file be text-based and self-contained.

For text-based MLIR file situations where the size increase is an issue, I could imagine the config data containing a string with the name of an external file, and an application-defined way to resolve the full path. That could be out-of-tree and/or dialect specific.

Yeah, I'd gladly switch to base64. We've only been using hex because it has prior art/use in-tree.

[stellaraccident]

A few very minor nits, but otherwise lgtm.

mlir/include/mlir/IR/AsmState.h

45

s/used/use/

97

sp. dataAlignment

mlir/lib/Parser/Parser.cpp

307

Missing closing ascii art?

[mehdi_amini]

mlir/include/mlir/IR/OpImplementation.h

32

Why didn't you keep an instance of the dialect here? Seems like you have to pass it around in a few places.

62

I find the naming and setup confusing with an inner class "Base" which inherits from its enclosing class? What are you trying to convey with this?
Is this some common pattern?

mlir/lib/Parser/Parser.cpp

317

I'm not sure what "otherwise" refer to here? This is the first comment in this function

mlir/test/lib/Dialect/Test/TestDialect.h

54

Where does deallocation happens?

[rriddle]

mlir/include/mlir/IR/OpImplementation.h

32

No good reason, added a dialect field (which is likely cleaner/more general anyways).

62

We do this in a few other places as well. The intent is to provide a CRTP Base/Impl class that defines away all of the boiler plate for the given resource type. Users don't really interact with it, it's solely for simplifying the definition of concrete handle types. Do you have a suggestion on better naming here?

mlir/test/lib/Dialect/Test/TestDialect.h

54

The base AsmResourceBlob holds a deleter function which deallocates the data within the destructor.

[mehdi_amini]

mlir/include/mlir/IR/OpImplementation.h

62

Why is it an inner class?

mlir/test/lib/Dialect/Test/TestDialect.h

54

I was missing that you were inheriting the base constructor and that it is how we get the deleter...

[rriddle]

mlir/include/mlir/IR/OpImplementation.h

62

Just to make it more contained (i.e. not pollute the mlir namespace), I could just pull it out if you prefer.

[uweigand]

This commit has broken the mlir builder on s390x-ibm-linux: https://lab.llvm.org/buildbot/#/builders/199/builds/6417

The immediate error is this failing assertion:

mlir-opt: /home/uweigand/llvm/llvm-head/mlir/test/lib/Dialect/Test/TestDialect.cpp:191: {anonymous}::TestOpAsmInterface::parseResource(mlir::AsmParsedResourceEntry&) const::<lambda(unsigned int, unsigned int)>: Assertion `align == alignof(uint64_t) && "unexpected data alignment"' failed.

which occurs because the value of align is 0x08000000 instead of the expected 8. The alignment was taken from the binary blob, encoded in little-endian format, but decoded as uint32_t on the host platform, which in our case is big-endian.

Now this is easily fixed by using proper endian conversion when accessing the alignment, which fixes the assertion, but then the test cases still fail because they expect little-endian data in the generated blobs, while they will actually contain big-endian data when generated on a big-endian machine.

What is the intended design point here? Are those binary blobs intended to be portable across machine architectures? In this case, I think we'd have to define a particular binary format that needs to be used (e.g. the blobs are per definition always little-endian), and then the code would have to be updated to always respect that format. Or, instead, are those blobs intended to be machine-specific? In this case, at the least the tests need to be made machine specific as well.

Note that in general, there may of course be other issues apart from endianness when attempting to re-use binary blobs across machines, e.g. pointer and/or word size, or ABI struct alignment rules. To define an actually portable format, all of that would have to be addressed.

[rriddle]

In D126446#3639047, @uweigand wrote:

This commit has broken the mlir builder on s390x-ibm-linux: https://lab.llvm.org/buildbot/#/builders/199/builds/6417

The immediate error is this failing assertion:

mlir-opt: /home/uweigand/llvm/llvm-head/mlir/test/lib/Dialect/Test/TestDialect.cpp:191: {anonymous}::TestOpAsmInterface::parseResource(mlir::AsmParsedResourceEntry&) const::<lambda(unsigned int, unsigned int)>: Assertion `align == alignof(uint64_t) && "unexpected data alignment"' failed.

which occurs because the value of align is 0x08000000 instead of the expected 8. The alignment was taken from the binary blob, encoded in little-endian format, but decoded as uint32_t on the host platform, which in our case is big-endian.

Now this is easily fixed by using proper endian conversion when accessing the alignment, which fixes the assertion, but then the test cases still fail because they expect little-endian data in the generated blobs, while they will actually contain big-endian data when generated on a big-endian machine.

What is the intended design point here? Are those binary blobs intended to be portable across machine architectures? In this case, I think we'd have to define a particular binary format that needs to be used (e.g. the blobs are per definition always little-endian), and then the code would have to be updated to always respect that format. Or, instead, are those blobs intended to be machine-specific? In this case, at the least the tests need to be made machine specific as well.

Note that in general, there may of course be other issues apart from endianness when attempting to re-use binary blobs across machines, e.g. pointer and/or word size, or ABI struct alignment rules. To define an actually portable format, all of that would have to be addressed.

Thanks for bringing this up. I think for the sake of the textual format, we should just always use little endian (this matches what DenseElementsAttr stores as in hex form as well). For the binary format of MLIR, depending on what we want to do in the long term, we could craft something more intricate so that blobs are more optimal on non-little endian (so we wouldn't need to do conversions). I'm also happy with extending the API/serialized data for blobs here to cover any information necessary for endian conversions/etc.

[mehdi_amini]

Can we XFAIL the test on this platform in the meantime?

[mehdi_amini]

(bring the bot back to green should be a priority)

[uweigand]

We have to fix the alignment tag for real, otherwise there will be failing assertions all over the place. With that fix in, there's still a failing test case - that one could indeed be XFAILed for now.

I've submitted a patch to implement this here: https://reviews.llvm.org/D129483

mlir/include/mlir/IR/AsmState.h

-//===- AsmState.h - State class for AsmPrinter ------------------*- C++ -*-===//
+//===- AsmState.h - Assembly State Utilities --------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
-// This file defines the AsmState class.
+// This file defines various classes and utilites for interacting with the MLIR
+// assembly formats.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef MLIR_IR_ASMSTATE_H_
 #define MLIR_IR_ASMSTATE_H_
 
 #include "mlir/IR/OperationSupport.h"
 #include "mlir/Support/LLVM.h"
 
 #include <memory>
 
 namespace mlir {
+class AsmResourcePrinter;
 class Operation;
 
 namespace detail {
 class AsmStateImpl;
 } // namespace detail
 
+//===----------------------------------------------------------------------===//
+// Resources
+//===----------------------------------------------------------------------===//
+
+/// The following classes enable support for parsing and printing resources
+/// within MLIR assembly formats. Resources are a mechanism by which dialects,
+/// and external clients, may attach additional information when parsing or
+/// printing IR without that information being encoded in the IR itself.
+/// Resources are not uniqued within the MLIR context, are not attached directly
+/// to any operation, and are solely intended to live and be processed outside
+/// of the immediate IR.
+///
+/// Resources are encoded using a key-value pair nested within dictionaries
+/// anchored either on a dialect, or an externally registered entity.
+/// Dictionaries anchored on dialects use the dialect namespace directly, and
+/// dictionaries anchored on external entities use a provided unique identifier.
+/// The resource key is an identifier used to disambiguate the data. The
+/// resource value may be stored in various limited forms, but general encodings
+/// use a string (human readable) or blob format (binary). Within the textual
+/// format, an example may be of the form:
+///
+/// {-#
+///   // The `dialect_resources` section within the file-level metadata
+///   // dictionary is used to contain any dialect resource entries.
+///   dialect_resources: {
+///     // Here is a dictionary anchored on "foo_dialect", which is a dialect
+///     // namespace.
+///     foo_dialect: {
+///       // `some_dialect_resource` is a key to be interpreted by the dialect,
+///       // and used to initialize/configure/etc.
+///       some_dialect_resource: "Some important resource value"
+///     }
+///   },
+///   // The `external_resources` section within the file-level metadata
+///   // dictionary is used to contain any non-dialect resource entries.
+///   external_resources: {
+///     // Here is a dictionary anchored on "mlir_reproducer", which is an
+///     // external entity representing MLIR's crash reproducer functionality.
+///     mlir_reproducer: {
+///       // `pipeline` is an entry that holds a crash reproducer pipeline
+///       // resource.
+///       pipeline: "func.func(canonicalize,cse)"
+///     }
+///   }
+/// #-}
+///
+
+//===----------------------------------------------------------------------===//
+// Resource Entry
+
+/// This class is used to build resource entries for use by the printer. Each
+/// resource entry is represented using a key/value pair. The provided key must
+/// be unique within the current context, which allows for a client to provide
+/// resource entries without worrying about overlap with other clients.
+class AsmResourceBuilder {
+public:
+  virtual ~AsmResourceBuilder();
+
+  /// Build a resource entry represented by the given bool.
+  virtual void buildBool(StringRef key, bool data) = 0;
+
+  /// Build a resource entry represented by the given human-readable string
+  /// value.
+  virtual void buildString(StringRef key, StringRef data) = 0;
+
+  /// Build an resource entry represented by the given binary blob data.
+  virtual void buildBlob(StringRef key, ArrayRef<char> data,
+                         uint32_t dataAlignment) = 0;
+  /// Build an resource entry represented by the given binary blob data. This is
+  /// a useful overload if the data type is known. Note that this does not
+  /// support `char` element types to avoid accidentally not providing the
+  /// expected alignment of data in situations that treat blobs generically.
+  template <typename T>
+  std::enable_if_t<!std::is_same<T, char>::value> buildBlob(StringRef key,
+                                                            ArrayRef<T> data) {
+    buildBlob(
+        key, ArrayRef<char>((const char *)data.data(), data.size() * sizeof(T)),
+        alignof(T));
+  }
+};
+
+/// This class represents a processed binary blob of data. A resource blob is
+/// essentially a collection of data, potentially mutable, with an associated
+/// deleter function (used if the data needs to be destroyed).
+class AsmResourceBlob {
+public:
+  /// A deleter function that frees a blob given the data and allocation size.
+  using DeleterFn = llvm::unique_function<void(const void *data, size_t size)>;
+
+  AsmResourceBlob() = default;
+  AsmResourceBlob(ArrayRef<char> data, DeleterFn deleter, bool dataIsMutable)
+      : data(data), deleter(std::move(deleter)), dataIsMutable(dataIsMutable) {}
+  /// Utility constructor that initializes a blob with a non-char type T.
+  template <typename T, typename DelT>
+  AsmResourceBlob(ArrayRef<T> data, DelT &&deleteFn, bool dataIsMutable)
+      : data((const char *)data.data(), data.size() * sizeof(T)),
+        deleter([deleteFn = std::forward<DelT>(deleteFn)](const void *data,
+                                                          size_t size) {
+          return deleteFn((const T *)data, size);
+        }),
+        dataIsMutable(dataIsMutable) {}
+  AsmResourceBlob(AsmResourceBlob &&) = default;
+  AsmResourceBlob &operator=(AsmResourceBlob &&) = default;
+  AsmResourceBlob(const AsmResourceBlob &) = delete;
+  AsmResourceBlob &operator=(const AsmResourceBlob &) = delete;
+  ~AsmResourceBlob() {
+    if (deleter)
+      deleter(data.data(), data.size());
+  }
+
+  /// Return the raw underlying data of this blob.
+  ArrayRef<char> getData() const { return data; }
+
+  /// Return a mutable reference to the raw underlying data of this blob.
+  /// Asserts that the blob `isMutable`.
+  MutableArrayRef<char> getMutableData() {
+    assert(isMutable() &&
+           "cannot access mutable reference to non-mutable data");
+    return MutableArrayRef<char>(const_cast<char *>(data.data()), data.size());
+  }
+
+  /// Return if the data of this blob is mutable.
+  bool isMutable() const { return dataIsMutable; }
+
+  /// Return the deleter function of this blob.
+  DeleterFn &getDeleter() { return deleter; }
+  const DeleterFn &getDeleter() const { return deleter; }
+
+private:
+  /// The raw, properly aligned, blob data.
+  ArrayRef<char> data;
+
+  /// An optional deleter function used to deallocate the underlying data when
+  /// necessary.
+  DeleterFn deleter;
+
+  /// Whether the data is mutable.
+  bool dataIsMutable;
+};
+
+/// This class represents a single parsed resource entry.
+class AsmParsedResourceEntry {
+public:
+  virtual ~AsmParsedResourceEntry();
+
+  /// Return the key of the resource entry.
+  virtual StringRef getKey() const = 0;
+
+  /// Emit an error at the location of this entry.
+  virtual InFlightDiagnostic emitError() const = 0;
+
+  /// Parse the resource entry represented by a boolean. Returns failure if the
+  /// entry does not correspond to a bool.
+  virtual FailureOr<bool> parseAsBool() const = 0;
+
+  /// Parse the resource entry represented by a human-readable string. Returns
+  /// failure if the entry does not correspond to a string.
+  virtual FailureOr<std::string> parseAsString() const = 0;
+
+  /// The type of an allocator function used to allocate memory for a blob when
+  /// required. The function is provided a size and alignment, and should return
+  /// an aligned allocation buffer.
+  using BlobAllocatorFn =
+      function_ref<AsmResourceBlob(unsigned size, unsigned align)>;
+
+  /// Parse the resource entry represented by a binary blob. Returns failure if
+  /// the entry does not correspond to a blob. If the blob needed to be
+  /// allocated, the given allocator function is invoked.
+  virtual FailureOr<AsmResourceBlob>
+  parseAsBlob(BlobAllocatorFn allocator) const = 0;
+};
+
+//===----------------------------------------------------------------------===//
+// Resource Parser/Printer
+
+/// This class represents an instance of a resource parser. This class should be
+/// implemented by non-dialect clients that want to inject additional resources
+/// into MLIR assembly formats.
+class AsmResourceParser {
+public:
+  /// Create a new parser with the given identifying name. This name uniquely
+  /// identifies the entries of this parser, and differentiates them from other
+  /// contexts.
+  AsmResourceParser(StringRef name) : name(name.str()) {}
+  virtual ~AsmResourceParser();
+
+  /// Return the name of this parser.
+  StringRef getName() const { return name; }
+
+  /// Parse the given resource entry. Returns failure if the key/data were not
+  /// valid, or could otherwise not be processed correctly. Any necessary errors
+  /// should be emitted with the provided entry.
+  virtual LogicalResult parseResource(AsmParsedResourceEntry &entry) = 0;
+
+  /// Return a resource parser implemented via the given callable, whose form
+  /// should match that of `parseResource` above.
+  template <typename CallableT>
+  static std::unique_ptr<AsmResourceParser> fromCallable(StringRef name,
+                                                         CallableT &&parseFn) {
+    struct Processor : public AsmResourceParser {
+      Processor(StringRef name, CallableT &&parseFn)
+          : AsmResourceParser(name), parseFn(std::move(parseFn)) {}
+      LogicalResult parseResource(AsmParsedResourceEntry &entry) override {
+        return parseFn(entry);
+      }
+
+      std::decay_t<CallableT> parseFn;
+    };
+    return std::make_unique<Processor>(name, std::forward<CallableT>(parseFn));
+  }
+
+private:
+  std::string name;
+};
+
+/// This class represents an instance of a resource printer. This class should
+/// be implemented by non-dialect clients that want to inject additional
+/// resources into MLIR assembly formats.
+class AsmResourcePrinter {
+public:
+  /// Create a new printer with the given identifying name. This name uniquely
+  /// identifies the entries of this printer, and differentiates them from
+  /// other contexts.
+  AsmResourcePrinter(StringRef name) : name(name.str()) {}
+  virtual ~AsmResourcePrinter();
+
+  /// Return the name of this printer.
+  StringRef getName() const { return name; }
+
+  /// Build any resources to include during printing, utilizing the given
+  /// top-level root operation to help determine what information to include.
+  /// Provided data should be registered in the form of a key/data pair, to the
+  /// given builder.
+  virtual void buildResources(Operation *op,
+                              AsmResourceBuilder &builder) const = 0;
+
+  /// Return a resource printer implemented via the given callable, whose form
+  /// should match that of `buildResources` above.
+  template <typename CallableT>
+  static std::unique_ptr<AsmResourcePrinter> fromCallable(StringRef name,
+                                                          CallableT &&printFn) {
+    struct Printer : public AsmResourcePrinter {
+      Printer(StringRef name, CallableT &&printFn)
+          : AsmResourcePrinter(name), printFn(std::move(printFn)) {}
+      void buildResources(Operation *op,
+                          AsmResourceBuilder &builder) const override {
+        printFn(op, builder);
+      }
+
+      std::decay_t<CallableT> printFn;
+    };
+    return std::make_unique<Printer>(name, std::forward<CallableT>(printFn));
+  }
+
+private:
+  std::string name;
+};
+
+//===----------------------------------------------------------------------===//
+// ParserConfig
+//===----------------------------------------------------------------------===//
+
+/// This class represents a configuration for the MLIR assembly parser. It
+/// contains all of the necessary state to parse a textual MLIR source file.
+class ParserConfig {
+public:
+  ParserConfig(MLIRContext *context) : context(context) {
+    assert(context && "expected valid MLIR context");
+  }
+
+  /// Return the MLIRContext to be used when parsing.
+  MLIRContext *getContext() const { return context; }
+
+  /// Return the resource parser registered to the given name, or nullptr if no
+  /// parser with `name` is registered.
+  AsmResourceParser *getResourceParser(StringRef name) const {
+    auto it = resourceParsers.find(name);
+    return it == resourceParsers.end() ? nullptr : it->second.get();
+  }
+
+  /// Attach the given resource parser.
+  void attachResourceParser(std::unique_ptr<AsmResourceParser> parser) {
+    StringRef name = parser->getName();
+    auto it = resourceParsers.try_emplace(name, std::move(parser));
+    assert(it.second &&
+           "resource parser already registered with the given name");
+  }
+
+  /// Attach the given callable resource parser with the given name.
+  template <typename CallableT>
+  std::enable_if_t<std::is_convertible<
+      CallableT, function_ref<LogicalResult(AsmParsedResourceEntry &)>>::value>
+  attachResourceParser(StringRef name, CallableT &&parserFn) {
+    attachResourceParser(AsmResourceParser::fromCallable(
+        name, std::forward<CallableT>(parserFn)));
+  }
+
+private:
+  MLIRContext *context;
+  DenseMap<StringRef, std::unique_ptr<AsmResourceParser>> resourceParsers;
+};
+
+//===----------------------------------------------------------------------===//
+// AsmState
+//===----------------------------------------------------------------------===//
+
 /// This class provides management for the lifetime of the state used when
 /// printing the IR. It allows for alleviating the cost of recomputing the
 /// internal state of the asm printer.
 ///
 /// The IR should not be mutated in-between invocations using this state, and
 /// the IR being printed must not be an parent of the IR originally used to
 /// initialize this state. This means that if a child operation is provided, a
 /// parent operation cannot reuse this state.
public:
 
   /// Get the printer flags.
   const OpPrintingFlags &getPrinterFlags() const;
 
   /// Return an instance of the internal implementation. Returns nullptr if the
   /// state has not been initialized.
   detail::AsmStateImpl &getImpl() { return *impl; }
 
+  //===--------------------------------------------------------------------===//
+  // Resources
+  //===--------------------------------------------------------------------===//
+
+  /// Attach the given resource printer to the AsmState.
+  void attachResourcePrinter(std::unique_ptr<AsmResourcePrinter> printer);
+
+  /// Attach an resource printer, in the form of a callable, to the AsmState.
+  template <typename CallableT>
+  std::enable_if_t<std::is_convertible<
+      CallableT, function_ref<void(Operation *, AsmResourceBuilder &)>>::value>
+  attachResourcePrinter(StringRef name, CallableT &&printFn) {
+    attachResourcePrinter(AsmResourcePrinter::fromCallable(
+        name, std::forward<CallableT>(printFn)));
+  }
+
 private:
   AsmState() = delete;
 
   /// A pointer to allocated storage for the impl state.
   std::unique_ptr<detail::AsmStateImpl> impl;
 };
 
 //===----------------------------------------------------------------------===//

mlir/include/mlir/IR/OpAsmInterface.td

 //===- OpAsmInterface.td - Asm Interfaces for opse ---------*- tablegen -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
-// This file contains Interfaces for interacting with the AsmParser and
-// AsmPrinter.
+// This file contains interfaces and other utilities for interacting with the
+// AsmParser and AsmPrinter.
 //
 //===----------------------------------------------------------------------===//
 
-#ifndef MLIR_OPASMINTERFACE
-#define MLIR_OPASMINTERFACE
+#ifndef MLIR_IR_OPASMINTERFACE_TD
+#define MLIR_IR_OPASMINTERFACE_TD
 
+include "mlir/IR/AttrTypeBase.td"
 include "mlir/IR/OpBase.td"
 
-/// Interface for hooking into the OpAsmPrinter and OpAsmParser.
+//===----------------------------------------------------------------------===//
+// OpAsmOpInterface
+//===----------------------------------------------------------------------===//
+
 def OpAsmOpInterface : OpInterface<"OpAsmOpInterface"> {
   let description = [{
     This interface provides hooks to interact with the AsmPrinter and AsmParser
     classes.
   }];
   let cppNamespace = "::mlir";
 
   let methods = [
StaticInterfaceMethod<[{
       returned `spv`. The default implementation returns an empty string which
       is ignored.
       }],
       "::llvm::StringRef", "getDefaultDialect", (ins), "", "return \"\";"
     >,
   ];
 }
 
-#endif // MLIR_OPASMINTERFACE
+//===----------------------------------------------------------------------===//
+// ResourceHandleParameter
+//===----------------------------------------------------------------------===//
+
+/// This parameter represents a handle to a resource that is encoded into the
+/// "dialect_resources" section of the assembly format. This parameter expects a
+/// C++ `handleType` that derives from `AsmDialectResourceHandleBase` and
+/// implements a derived handle to the desired resource type. 
+class ResourceHandleParameter<string handleType, string desc = "">
+    : AttrOrTypeParameter<handleType, desc> {
+  let parser = "$_parser.parseResourceHandle<" # handleType # ">()";
+  let printer = "$_printer.printResourceHandle($_self)";
+}
+
+#endif // MLIR_IR_OPASMINTERFACE_TD

mlir/include/mlir/IR/OpImplementation.h

 
 #include "mlir/IR/BuiltinTypes.h"
 #include "mlir/IR/DialectInterface.h"
 #include "mlir/IR/OpDefinition.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/Support/SMLoc.h"
 
 namespace mlir {
-
+class AsmParsedResourceEntry;
+class AsmResourceBuilder;
 class Builder;
 
 //===----------------------------------------------------------------------===//
+// AsmDialectResourceHandle
+//===----------------------------------------------------------------------===//
+
+/// This class represents an opaque handle to a dialect resource entry.
+class AsmDialectResourceHandle {
+public:
+  AsmDialectResourceHandle() = default;
+  AsmDialectResourceHandle(void *resource, TypeID resourceID, Dialect *dialect)
+      : resource(resource), opaqueID(resourceID), dialect(dialect) {}
+  bool operator==(const AsmDialectResourceHandle &other) const {
+    return resource == other.resource;
+  }
+
+  /// Return an opaque pointer to the referenced resource.
+  void *getResource() const { return resource; }
+
+  /// Return the type ID of the resource.
+  TypeID getTypeID() const { return opaqueID; }
+
+  /// Return the dialect that owns the resource.
+  Dialect *getDialect() const { return dialect; }
+
+private:
+  /// The opaque handle to the dialect resource.
+  void *resource = nullptr;
+  /// The type of the resource referenced.
+  TypeID opaqueID;
+  /// The dialect owning the given resource.
+  Dialect *dialect;
+};
+
+/// This class represents a CRTP base class for dialect resource handles. It
+/// abstracts away various utilities necessary for defined derived resource
+/// handles.
+template <typename DerivedT, typename ResourceT, typename DialectT>
+class AsmDialectResourceHandleBase : public AsmDialectResourceHandle {
+public:
+  using Dialect = DialectT;
+
+  /// Construct a handle from a pointer to the resource. The given pointer
+  /// should be guaranteed to live beyond the life of this handle.
+  AsmDialectResourceHandleBase(ResourceT *resource, DialectT *dialect)
+      : AsmDialectResourceHandle(resource, TypeID::get<DerivedT>(), dialect) {}
+  AsmDialectResourceHandleBase(AsmDialectResourceHandle handle)
+      : AsmDialectResourceHandle(handle) {
+    assert(handle.getTypeID() == TypeID::get<DerivedT>());
+  }
+
+  /// Return the resource referenced by this handle.
+  ResourceT *getResource() {
+    return static_cast<ResourceT *>(AsmDialectResourceHandle::getResource());
+  }
+  const ResourceT *getResource() const {
+    return const_cast<AsmDialectResourceHandleBase *>(this)->getResource();
+  }
+
+  /// Return the dialect that owns the resource.
+  DialectT *getDialect() const {
+    return static_cast<DialectT *>(AsmDialectResourceHandle::getDialect());
+  }
+
+  /// Support llvm style casting.
+  static bool classof(const AsmDialectResourceHandle *handle) {
+    return handle->getTypeID() == TypeID::get<DerivedT>();
+  }
+};
+
+inline llvm::hash_code hash_value(const AsmDialectResourceHandle &param) {
+  return llvm::hash_value(param.getResource());
+}
+
+//===----------------------------------------------------------------------===//
 // AsmPrinter
 //===----------------------------------------------------------------------===//
 
 /// This base class exposes generic asm printer hooks, usable across the various
 /// derived printers.
 class AsmPrinter {
 public:
   /// This class contains the internal default implementation of the base
public:
   virtual void printKeywordOrString(StringRef keyword);
 
   /// Print the given string as a symbol reference, i.e. a form representable by
   /// a SymbolRefAttr. A symbol reference is represented as a string prefixed
   /// with '@'. The reference is surrounded with ""'s and escaped if it has any
   /// special or non-printable characters in it.
   virtual void printSymbolName(StringRef symbolRef);
 
+  /// Print a handle to the given dialect resource.
+  void printResourceHandle(const AsmDialectResourceHandle &resource);
+
   /// Print an optional arrow followed by a type list.
   template <typename TypeRange>
   void printOptionalArrowTypeList(TypeRange &&types) {
     if (types.begin() != types.end())
       printArrowTypeList(types);
   }
   template <typename TypeRange>
   void printArrowTypeList(TypeRange &&types) {
public:
 
   /// Parse an optional @-identifier and store it (without the '@' symbol) in a
   /// string attribute named 'attrName'.
   virtual ParseResult parseOptionalSymbolName(StringAttr &result,
                                               StringRef attrName,
                                               NamedAttrList &attrs) = 0;
 
   //===--------------------------------------------------------------------===//
+  // Resource Parsing
+  //===--------------------------------------------------------------------===//
+
+  /// Parse a handle to a resource within the assembly format.
+  template <typename ResourceT>
+  FailureOr<ResourceT> parseResourceHandle() {
+    SMLoc handleLoc = getCurrentLocation();
+    FailureOr<AsmDialectResourceHandle> handle = parseResourceHandle(
+        getContext()->getOrLoadDialect<typename ResourceT::Dialect>());
+    if (failed(handle))
+      return failure();
+    if (auto *result = dyn_cast<ResourceT>(&*handle))
+      return std::move(*result);
+    return emitError(handleLoc) << "provided resource handle differs from the "
+                                   "expected resource type";
+  }
+
+  //===--------------------------------------------------------------------===//
   // Type Parsing
   //===--------------------------------------------------------------------===//
 
   /// Parse a type.
   virtual ParseResult parseType(Type &result) = 0;
 
   /// Parse a custom type with the provided callback, unless the next
   /// token is `#`, in which case the generic parser is invoked.
virtual ParseResult parseDimensionList(SmallVectorImpl<int64_t> &dimensions,
                                          bool allowDynamic = true,
                                          bool withTrailingX = true) = 0;
 
   /// Parse an 'x' token in a dimension list, handling the case where the x is
   /// juxtaposed with an element type, as in "xf32", leaving the "f32" as the
   /// next token.
   virtual ParseResult parseXInDimensionList() = 0;
 
+protected:
+  /// Parse a handle to a resource within the assembly format for the given
+  /// dialect.
+  virtual FailureOr<AsmDialectResourceHandle>
+  parseResourceHandle(Dialect *dialect) = 0;
+
 private:
   AsmParser(const AsmParser &) = delete;
   void operator=(const AsmParser &) = delete;
 };
 
 //===----------------------------------------------------------------------===//
 // OpAsmParser
 //===----------------------------------------------------------------------===//

 
 /// A functor used to set the name of blocks in regions directly nested under
 /// an operation.
 using OpAsmSetBlockNameFn = function_ref<void(Block *, StringRef)>;
 
 class OpAsmDialectInterface
     : public DialectInterface::Base<OpAsmDialectInterface> {
 public:
+  OpAsmDialectInterface(Dialect *dialect) : Base(dialect) {}
+
+  //===------------------------------------------------------------------===//
+  // Aliases
+  //===------------------------------------------------------------------===//
+
   /// Holds the result of `getAlias` hook call.
   enum class AliasResult {
     /// The object (type or attribute) is not supported by the hook
     /// and an alias was not provided.
     NoAlias,
     /// An alias was provided, but it might be overriden by other hook.
     OverridableAlias,
     /// An alias was provided and it should be used
     /// (no other hooks will be checked).
     FinalAlias
   };
 
-  OpAsmDialectInterface(Dialect *dialect) : Base(dialect) {}
-
   /// Hooks for getting an alias identifier alias for a given symbol, that is
   /// not necessarily a part of this dialect. The identifier is used in place of
   /// the symbol when printing textual IR. These aliases must not contain `.` or
   /// end with a numeric digit([0-9]+).
   virtual AliasResult getAlias(Attribute attr, raw_ostream &os) const {
     return AliasResult::NoAlias;
   }
   virtual AliasResult getAlias(Type type, raw_ostream &os) const {
     return AliasResult::NoAlias;
   }
+
+  //===--------------------------------------------------------------------===//
+  // Resources
+  //===--------------------------------------------------------------------===//
+
+  /// Declare a resource with the given key, returning a handle to use for any
+  /// references of this resource key within the IR during parsing. The result
+  /// of `getResourceKey` on the returned handle is permitted to be different
+  /// than `key`.
+  virtual FailureOr<AsmDialectResourceHandle>
+  declareResource(StringRef key) const {
+    return failure();
+  }
+
+  /// Return a key to use for the given resource. This key should uniquely
+  /// identify this resource within the dialect.
+  virtual std::string
+  getResourceKey(const AsmDialectResourceHandle &handle) const {
+    llvm_unreachable(
+        "Dialect must implement `getResourceKey` when defining resources");
+  }
+
+  /// Hook for parsing resource entries. Returns failure if the entry was not
+  /// valid, or could otherwise not be processed correctly. Any necessary errors
+  /// can be emitted via the provided entry.
+  virtual LogicalResult parseResource(AsmParsedResourceEntry &entry) const;
+
+  /// Hook for building resources to use during printing. The given `op` may be
+  /// inspected to help determine what information to include.
+  /// `referencedResources` contains all of the resources detected when printing
+  /// 'op'.
+  virtual void
+  buildResources(Operation *op,
+                 const SetVector<AsmDialectResourceHandle> &referencedResources,
+                 AsmResourceBuilder &builder) const {}
 };
 } // namespace mlir
 
 //===--------------------------------------------------------------------===//
 // Operation OpAsm interface.
 //===--------------------------------------------------------------------===//
 
 /// The OpAsmOpInterface, see OpAsmInterface.td for more details.
 #include "mlir/IR/OpAsmInterface.h.inc"
 
+namespace llvm {
+template <>
+struct DenseMapInfo<mlir::AsmDialectResourceHandle> {
+  static inline mlir::AsmDialectResourceHandle getEmptyKey() {
+    return {DenseMapInfo<void *>::getEmptyKey(),
+            DenseMapInfo<mlir::TypeID>::getEmptyKey(), nullptr};
+  }
+  static inline mlir::AsmDialectResourceHandle getTombstoneKey() {
+    return {DenseMapInfo<void *>::getTombstoneKey(),
+            DenseMapInfo<mlir::TypeID>::getTombstoneKey(), nullptr};
+  }
+  static unsigned getHashValue(const mlir::AsmDialectResourceHandle &handle) {
+    return DenseMapInfo<void *>::getHashValue(handle.getResource());
+  }
+  static bool isEqual(const mlir::AsmDialectResourceHandle &lhs,
+                      const mlir::AsmDialectResourceHandle &rhs) {
+    return lhs.getResource() == rhs.getResource();
+  }
+};
+} // namespace llvm
+
 #endif

mlir/include/mlir/Parser/Parser.h

 //===- Parser.h - MLIR Parser Library Interface -----------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file is contains the interface to the MLIR parser library.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef MLIR_PARSER_PARSER_H
 #define MLIR_PARSER_PARSER_H
 
+#include "mlir/IR/AsmState.h"
 #include "mlir/IR/Builders.h"
 #include "mlir/IR/BuiltinOps.h"
 #include <cstddef>
 
 namespace llvm {
 class SourceMgr;
 class SMDiagnostic;
 class StringRef;

 /// returned. Otherwise, an error message is emitted through the error handler
 /// registered in the context, and failure is returned. If `sourceFileLoc` is
 /// non-null, it is populated with a file location representing the start of the
 /// source file that is being parsed. If `asmState` is non-null, it is populated
 /// with detailed information about the parsed IR (including exact locations for
 /// SSA uses and definitions). `asmState` should only be provided if this
 /// detailed information is desired.
 LogicalResult parseSourceFile(const llvm::SourceMgr &sourceMgr, Block *block,
-                              MLIRContext *context,
+                              const ParserConfig &config,
                               LocationAttr *sourceFileLoc = nullptr,
                               AsmParserState *asmState = nullptr);
 
 /// This parses the file specified by the indicated filename and appends parsed
 /// operations to the given block. If the block is non-empty, the operations are
 /// placed before the current terminator. If parsing is successful, success is
 /// returned. Otherwise, an error message is emitted through the error handler
 /// registered in the context, and failure is returned. If `sourceFileLoc` is
 /// non-null, it is populated with a file location representing the start of the
 /// source file that is being parsed.
 LogicalResult parseSourceFile(llvm::StringRef filename, Block *block,
-                              MLIRContext *context,
+                              const ParserConfig &config,
                               LocationAttr *sourceFileLoc = nullptr);
 
 /// This parses the file specified by the indicated filename using the provided
 /// SourceMgr and appends parsed operations to the given block. If the block is
 /// non-empty, the operations are placed before the current terminator. If
 /// parsing is successful, success is returned. Otherwise, an error message is
 /// emitted through the error handler registered in the context, and failure is
 /// returned. If `sourceFileLoc` is non-null, it is populated with a file
 /// location representing the start of the source file that is being parsed. If
 /// `asmState` is non-null, it is populated with detailed information about the
 /// parsed IR (including exact locations for SSA uses and definitions).
 /// `asmState` should only be provided if this detailed information is desired.
 LogicalResult parseSourceFile(llvm::StringRef filename,
                               llvm::SourceMgr &sourceMgr, Block *block,
-                              MLIRContext *context,
+                              const ParserConfig &config,
                               LocationAttr *sourceFileLoc = nullptr,
                               AsmParserState *asmState = nullptr);
 
 /// This parses the IR string and appends parsed operations to the given block.
 /// If the block is non-empty, the operations are placed before the current
 /// terminator. If parsing is successful, success is returned. Otherwise, an
 /// error message is emitted through the error handler registered in the
 /// context, and failure is returned. If `sourceFileLoc` is non-null, it is
 /// populated with a file location representing the start of the source file
 /// that is being parsed.
 LogicalResult parseSourceString(llvm::StringRef sourceStr, Block *block,
-                                MLIRContext *context,
+                                const ParserConfig &config,
                                 LocationAttr *sourceFileLoc = nullptr);
 
 namespace detail {
 /// The internal implementation of the templated `parseSourceFile` methods
 /// below, that simply forwards to the non-templated version.
 template <typename ContainerOpT, typename... ParserArgs>
-inline OwningOpRef<ContainerOpT> parseSourceFile(MLIRContext *ctx,
+inline OwningOpRef<ContainerOpT> parseSourceFile(const ParserConfig &config,
                                                  ParserArgs &&...args) {
   LocationAttr sourceFileLoc;
   Block block;
-  if (failed(parseSourceFile(std::forward<ParserArgs>(args)..., &block, ctx,
+  if (failed(parseSourceFile(std::forward<ParserArgs>(args)..., &block, config,
                              &sourceFileLoc)))
     return OwningOpRef<ContainerOpT>();
   return detail::constructContainerOpForParserIfNecessary<ContainerOpT>(
-      &block, ctx, sourceFileLoc);
+      &block, config.getContext(), sourceFileLoc);
 }
 } // namespace detail
 
 /// This parses the file specified by the indicated SourceMgr. If the source IR
 /// contained a single instance of `ContainerOpT`, it is returned. Otherwise, a
 /// new instance of `ContainerOpT` is constructed containing all of the parsed
 /// operations. If parsing was not successful, null is returned and an error
 /// message is emitted through the error handler registered in the context, and
 /// failure is returned. `ContainerOpT` is required to have a single region
 /// containing a single block, and must implement the
 /// `SingleBlockImplicitTerminator` trait.
 template <typename ContainerOpT>
 inline OwningOpRef<ContainerOpT>
-parseSourceFile(const llvm::SourceMgr &sourceMgr, MLIRContext *context) {
-  return detail::parseSourceFile<ContainerOpT>(context, sourceMgr);
+parseSourceFile(const llvm::SourceMgr &sourceMgr, const ParserConfig &config) {
+  return detail::parseSourceFile<ContainerOpT>(config, sourceMgr);
 }
 
 /// This parses the file specified by the indicated filename. If the source IR
 /// contained a single instance of `ContainerOpT`, it is returned. Otherwise, a
 /// new instance of `ContainerOpT` is constructed containing all of the parsed
 /// operations. If parsing was not successful, null is returned and an error
 /// message is emitted through the error handler registered in the context, and
 /// failure is returned. `ContainerOpT` is required to have a single region
 /// containing a single block, and must implement the
 /// `SingleBlockImplicitTerminator` trait.
 template <typename ContainerOpT>
 inline OwningOpRef<ContainerOpT> parseSourceFile(StringRef filename,
-                                                 MLIRContext *context) {
-  return detail::parseSourceFile<ContainerOpT>(context, filename);
+                                                 const ParserConfig &config) {
+  return detail::parseSourceFile<ContainerOpT>(config, filename);
 }
 
 /// This parses the file specified by the indicated filename using the provided
 /// SourceMgr. If the source IR contained a single instance of `ContainerOpT`,
 /// it is returned. Otherwise, a new instance of `ContainerOpT` is constructed
 /// containing all of the parsed operations. If parsing was not successful, null
 /// is returned and an error message is emitted through the error handler
 /// registered in the context, and failure is returned. `ContainerOpT` is
 /// required to have a single region containing a single block, and must
 /// implement the `SingleBlockImplicitTerminator` trait.
 template <typename ContainerOpT>
 inline OwningOpRef<ContainerOpT> parseSourceFile(llvm::StringRef filename,
                                                  llvm::SourceMgr &sourceMgr,
-                                                 MLIRContext *context) {
-  return detail::parseSourceFile<ContainerOpT>(context, filename, sourceMgr);
+                                                 const ParserConfig &config) {
+  return detail::parseSourceFile<ContainerOpT>(config, filename, sourceMgr);
 }
 
 /// This parses the provided string containing MLIR. If the source IR contained
 /// a single instance of `ContainerOpT`, it is returned. Otherwise, a new
 /// instance of `ContainerOpT` is constructed containing all of the parsed
 /// operations. If parsing was not successful, null is returned and an error
 /// message is emitted through the error handler registered in the context, and
 /// failure is returned. `ContainerOpT` is required to have a single region
 /// containing a single block, and must implement the
 /// `SingleBlockImplicitTerminator` trait.
 template <typename ContainerOpT>
 inline OwningOpRef<ContainerOpT> parseSourceString(llvm::StringRef sourceStr,
-                                                   MLIRContext *context) {
+                                                   const ParserConfig &config) {
   LocationAttr sourceFileLoc;
   Block block;
-  if (failed(parseSourceString(sourceStr, &block, context, &sourceFileLoc)))
+  if (failed(parseSourceString(sourceStr, &block, config, &sourceFileLoc)))
     return OwningOpRef<ContainerOpT>();
   return detail::constructContainerOpForParserIfNecessary<ContainerOpT>(
-      &block, context, sourceFileLoc);
+      &block, config.getContext(), sourceFileLoc);
 }
 
 /// This parses a single MLIR attribute to an MLIR context if it was valid.  If
 /// not, an error message is emitted through a new SourceMgrDiagnosticHandler
 /// constructed from a new SourceMgr with a single a MemoryBuffer wrapping
 /// `attrStr`. If the passed `attrStr` has additional tokens that were not part
 /// of the type, an error is emitted.
 // TODO: Improve diagnostic reporting.

mlir/lib/IR/AsmPrinter.cpp

 
 //===----------------------------------------------------------------------===//
 // Operation OpAsm interface.
 //===----------------------------------------------------------------------===//
 
 /// The OpAsmOpInterface, see OpAsmInterface.td for more details.
 #include "mlir/IR/OpAsmInterface.cpp.inc"
 
+LogicalResult
+OpAsmDialectInterface::parseResource(AsmParsedResourceEntry &entry) const {
+  return entry.emitError() << "unknown 'resource' key '" << entry.getKey()
+                           << "' for dialect '" << getDialect()->getNamespace()
+                           << "'";
+}
+
 //===----------------------------------------------------------------------===//
 // OpPrintingFlags
 //===----------------------------------------------------------------------===//
 
 namespace {
 /// This struct contains command line options that can be used to initialize
 /// various bits of the AsmPrinter. This uses a struct wrapper to avoid the need
 /// for global command line options.
if (!usedNames.count(name)) {
     }
   }
 
   usedNames.insert(name, char());
   return name;
 }
 
 //===----------------------------------------------------------------------===//
+// Resources
+//===----------------------------------------------------------------------===//
+
+AsmParsedResourceEntry::~AsmParsedResourceEntry() = default;
+AsmResourceBuilder::~AsmResourceBuilder() = default;
+AsmResourceParser::~AsmResourceParser() = default;
+AsmResourcePrinter::~AsmResourcePrinter() = default;
+
+//===----------------------------------------------------------------------===//
 // AsmState
 //===----------------------------------------------------------------------===//
 
 namespace mlir {
 namespace detail {
 class AsmStateImpl {
 public:
   explicit AsmStateImpl(Operation *op, const OpPrintingFlags &printerFlags,
                         AsmState::LocationMap *locationMap)
       : interfaces(op->getContext()), nameState(op, printerFlags),
         printerFlags(printerFlags), locationMap(locationMap) {}
 
   /// Initialize the alias state to enable the printing of aliases.
   void initializeAliases(Operation *op) {
     aliasState.initialize(op, printerFlags, interfaces);
   }
 
   /// Get the state used for aliases.
   AliasState &getAliasState() { return aliasState; }
 
   /// Get the state used for SSA names.
   SSANameState &getSSANameState() { return nameState; }
 
+  /// Return the dialects within the context that implement
+  /// OpAsmDialectInterface.
+  DialectInterfaceCollection<OpAsmDialectInterface> &getDialectInterfaces() {
+    return interfaces;
+  }
+
+  /// Return the non-dialect resource printers.
+  auto getResourcePrinters() {
+    return llvm::make_pointee_range(externalResourcePrinters);
+  }
+
   /// Get the printer flags.
   const OpPrintingFlags &getPrinterFlags() const { return printerFlags; }
 
   /// Register the location, line and column, within the buffer that the given
   /// operation was printed at.
   void registerOperationLocation(Operation *op, unsigned line, unsigned col) {
     if (locationMap)
       (*locationMap)[op] = std::make_pair(line, col);
   }
 
 private:
   /// Collection of OpAsm interfaces implemented in the context.
   DialectInterfaceCollection<OpAsmDialectInterface> interfaces;
 
+  /// A collection of non-dialect resource printers.
+  SmallVector<std::unique_ptr<AsmResourcePrinter>> externalResourcePrinters;
+
   /// The state used for attribute and type aliases.
   AliasState aliasState;
 
   /// The state used for SSA value names.
   SSANameState nameState;
 
   /// Flags that control op output.
   OpPrintingFlags printerFlags;
 
   /// An optional location map to be populated.
   AsmState::LocationMap *locationMap;
+
+  // Allow direct access to the impl fields.
+  friend AsmState;
 };
 } // namespace detail
 } // namespace mlir
 
 /// Verifies the operation and switches to generic op printing if verification
 /// fails. We need to do this because custom print functions may fail for
 /// invalid ops.
 static OpPrintingFlags verifyOpAndAdjustFlags(Operation *op,
AsmState::AsmState(Operation *op, const OpPrintingFlags &printerFlags,
     : impl(std::make_unique<AsmStateImpl>(
           op, verifyOpAndAdjustFlags(op, printerFlags), locationMap)) {}
 AsmState::~AsmState() = default;
 
 const OpPrintingFlags &AsmState::getPrinterFlags() const {
   return impl->getPrinterFlags();
 }
 
+void AsmState::attachResourcePrinter(
+    std::unique_ptr<AsmResourcePrinter> printer) {
+  impl->externalResourcePrinters.emplace_back(std::move(printer));
+}
+
 //===----------------------------------------------------------------------===//
 // AsmPrinter::Impl
 //===----------------------------------------------------------------------===//
 
 namespace mlir {
 class AsmPrinter::Impl {
 public:
   Impl(raw_ostream &os, OpPrintingFlags flags = llvm::None,
public:
   /// Print the alias for the given type, return failure if no alias could
   /// be printed.
   LogicalResult printAlias(Type type);
 
   /// Print the given location to the stream. If `allowAlias` is true, this
   /// allows for the internal location to use an attribute alias.
   void printLocation(LocationAttr loc, bool allowAlias = false);
 
+  /// Print a reference to the given resource that is owned by the given
+  /// dialect.
+  void printResourceHandle(const AsmDialectResourceHandle &resource) {
+    auto *interface = cast<OpAsmDialectInterface>(resource.getDialect());
+    os << interface->getResourceKey(resource);
+    dialectResources[resource.getDialect()].insert(resource);
+  }
+
   void printAffineMap(AffineMap map);
   void
   printAffineExpr(AffineExpr expr,
                   function_ref<void(unsigned, bool)> printValueName = nullptr);
   void printAffineConstraint(AffineExpr expr, bool isEq);
   void printIntegerSet(IntegerSet set);
 
 protected:
protected:
   /// A set of flags to control the printer's behavior.
   OpPrintingFlags printerFlags;
 
   /// An optional printer state for the module.
   AsmStateImpl *state;
 
   /// A tracker for the number of new lines emitted during printing.
   NewLineCounter newLine;
+
+  /// A set of dialect resources that were referenced during printing.
+  DenseMap<Dialect *, SetVector<AsmDialectResourceHandle>> dialectResources;
 };
 } // namespace mlir
 
 void AsmPrinter::Impl::printTrailingLocation(Location loc, bool allowAlias) {
   // Check to see if we are printing debug information.
   if (!printerFlags.shouldPrintDebugInfo())
     return;
 
void AsmPrinter::Impl::printDialectAttribute(Attribute attr) {
 
   // Ask the dialect to serialize the attribute to a string.
   std::string attrName;
   {
     llvm::raw_string_ostream attrNameStr(attrName);
     Impl subPrinter(attrNameStr, printerFlags, state);
     DialectAsmPrinter printer(subPrinter);
     dialect.printAttribute(attr, printer);
+
+    // FIXME: Delete this when we no longer require a nested printer.
+    for (auto &it : subPrinter.dialectResources)
+      for (const auto &resource : it.second)
+        dialectResources[it.first].insert(resource);
   }
   printDialectSymbol(os, "#", dialect.getNamespace(), attrName);
 }
 
 void AsmPrinter::Impl::printDialectType(Type type) {
   auto &dialect = type.getDialect();
 
   // Ask the dialect to serialize the type to a string.
   std::string typeName;
   {
     llvm::raw_string_ostream typeNameStr(typeName);
     Impl subPrinter(typeNameStr, printerFlags, state);
     DialectAsmPrinter printer(subPrinter);
     dialect.printType(type, printer);
+
+    // FIXME: Delete this when we no longer require a nested printer.
+    for (auto &it : subPrinter.dialectResources)
+      for (const auto &resource : it.second)
+        dialectResources[it.first].insert(resource);
   }
   printDialectSymbol(os, "!", dialect.getNamespace(), typeName);
 }
 
 void AsmPrinter::Impl::printEscapedString(StringRef str) {
   os << "\"";
   llvm::printEscapedString(str, os);
   os << "\"";
void AsmPrinter::printKeywordOrString(StringRef keyword) {
   ::printKeywordOrString(keyword, impl->getStream());
 }
 
 void AsmPrinter::printSymbolName(StringRef symbolRef) {
   assert(impl && "expected AsmPrinter::printSymbolName to be overriden");
   ::printSymbolReference(symbolRef, impl->getStream());
 }
 
+void AsmPrinter::printResourceHandle(const AsmDialectResourceHandle &resource) {
+  assert(impl && "expected AsmPrinter::printResourceHandle to be overriden");
+  impl->printResourceHandle(resource);
+}
+
 //===----------------------------------------------------------------------===//
 // Affine expressions and maps
 //===----------------------------------------------------------------------===//
 
 void AsmPrinter::Impl::printAffineExpr(
     AffineExpr expr, function_ref<void(unsigned, bool)> printValueName) {
   printAffineExprInternal(expr, BindingStrength::Weak, printValueName);
 }
public:
   /// Print the users of a value.
   void printValueUsers(Value value);
 
   /// Print either the ids of the result values or the id of the operation if
   /// the operation has no results.
   void printUserIDs(Operation *user, bool prefixComma = false);
 
 private:
+  /// This class represents a resource builder implementation for the MLIR
+  /// textual assembly format.
+  class ResourceBuilder : public AsmResourceBuilder {
+  public:
+    using ValueFn = function_ref<void(raw_ostream &)>;
+    using PrintFn = function_ref<void(StringRef, ValueFn)>;
+
+    ResourceBuilder(OperationPrinter &p, PrintFn printFn)
+        : p(p), printFn(printFn) {}
+    ~ResourceBuilder() override = default;
+
+    void buildBool(StringRef key, bool data) final {
+      printFn(key, [&](raw_ostream &os) { p.os << (data ? "true" : "false"); });
+    }
+
+    void buildString(StringRef key, StringRef data) final {
+      printFn(key, [&](raw_ostream &os) { p.printEscapedString(data); });
+    }
+
+    void buildBlob(StringRef key, ArrayRef<char> data,
+                   uint32_t dataAlignment) final {
+      printFn(key, [&](raw_ostream &os) {
+        // Store the blob in a hex string containing the alignment and the data.
+        os << "\"0x"
+           << llvm::toHex(StringRef(reinterpret_cast<char *>(&dataAlignment),
+                                    sizeof(dataAlignment)))
+           << llvm::toHex(StringRef(data.data(), data.size())) << "\"";
+      });
+    }
+
+  private:
+    OperationPrinter &p;
+    PrintFn printFn;
+  };
+
+  /// Print the metadata dictionary for the file, eliding it if it is empty.
+  void printFileMetadataDictionary(Operation *op);
+
+  /// Print the resource sections for the file metadata dictionary.
+  /// `checkAddMetadataDict` is used to indicate that metadata is going to be
+  /// added, and the file metadata dictionary should be started if it hasn't
+  /// yet.
+  void printResourceFileMetadata(function_ref<void()> checkAddMetadataDict,
+                                 Operation *op);
+
   // Contains the stack of default dialects to use when printing regions.
   // A new dialect is pushed to the stack before parsing regions nested under an
   // operation implementing `OpAsmOpInterface`, and popped when done. At the
   // top-level we start with "builtin" as the default, so that the top-level
   // `module` operation prints as-is.
   SmallVector<StringRef> defaultDialectStack{"builtin"};
 
   /// The number of spaces used for indenting nested operations.
void OperationPrinter::printTopLevelOperation(Operation *op) {
   state->getAliasState().printNonDeferredAliases(os, newLine);
 
   // Print the module.
   print(op);
   os << newLine;
 
   // Output the aliases at the top level that can be deferred.
   state->getAliasState().printDeferredAliases(os, newLine);
+
+  // Output any file level metadata.
+  printFileMetadataDictionary(op);
+}
+
+void OperationPrinter::printFileMetadataDictionary(Operation *op) {
+  bool sawMetadataEntry = false;
+  auto checkAddMetadataDict = [&] {
+    if (!std::exchange(sawMetadataEntry, true))
+      os << newLine << "{-#" << newLine;
+  };
+
+  // Add the various types of metadata.
+  printResourceFileMetadata(checkAddMetadataDict, op);
+
+  // If the file dictionary exists, close it.
+  if (sawMetadataEntry)
+    os << newLine << "#-}" << newLine;
+}
+
+void OperationPrinter::printResourceFileMetadata(
+    function_ref<void()> checkAddMetadataDict, Operation *op) {
+  // Functor used to add data entries to the file metadata dictionary.
+  bool hadResource = false;
+  auto processProvider = [&](StringRef dictName, StringRef name, auto &provider,
+                             auto &&...providerArgs) {
+    bool hadEntry = false;
+    auto printFn = [&](StringRef key, ResourceBuilder::ValueFn valueFn) {
+      checkAddMetadataDict();
+
+      // Emit the top-level resource entry if we haven't yet.
+      if (!std::exchange(hadResource, true))
+        os << "  " << dictName << "_resources: {" << newLine;
+      // Emit the parent resource entry if we haven't yet.
+      if (!std::exchange(hadEntry, true))
+        os << "    " << name << ": {" << newLine;
+      else
+        os << "," << newLine;
+
+      os << "      " << key << ": ";
+      valueFn(os);
+    };
+    ResourceBuilder entryBuilder(*this, printFn);
+    provider.buildResources(op, providerArgs..., entryBuilder);
+
+    if (hadEntry)
+      os << newLine << "    }";
+  };
+
+  // Print the `dialect_resources` section if we have any dialects with
+  // resources.
+  for (const OpAsmDialectInterface &interface : state->getDialectInterfaces()) {
+    StringRef name = interface.getDialect()->getNamespace();
+    auto it = dialectResources.find(interface.getDialect());
+    if (it != dialectResources.end())
+      processProvider("dialect", name, interface, it->second);
+    else
+      processProvider("dialect", name, interface,
+                      SetVector<AsmDialectResourceHandle>());
+  }
+  if (hadResource)
+    os << newLine << "  }";
+
+  // Print the `external_resources` section if we have any external clients with
+  // resources.
+  hadResource = false;
+  for (const auto &printer : state->getResourcePrinters())
+    processProvider("external", printer.getName(), printer);
+  if (hadResource)
+    os << newLine << "  }";
 }
 
 /// Print a block argument in the usual format of:
 ///   %ssaName : type {attr1=42} loc("here")
 /// where location printing is controlled by the standard internal option.
 /// You may pass omitType=true to not print a type, and pass an empty
 /// attribute list if you don't care for attributes.
 void OperationPrinter::printRegionArgument(BlockArgument arg,

mlir/lib/Parser/AffineParser.cpp

 IntegerSet mlir::parseIntegerSet(StringRef inputStr, MLIRContext *context,
                                  bool printDiagnosticInfo) {
   llvm::SourceMgr sourceMgr;
   auto memBuffer = llvm::MemoryBuffer::getMemBuffer(
       inputStr, /*BufferName=*/"<mlir_parser_buffer>",
       /*RequiresNullTerminator=*/false);
   sourceMgr.AddNewSourceBuffer(std::move(memBuffer), SMLoc());
   SymbolState symbolState;
-  ParserState state(sourceMgr, context, symbolState, /*asmState=*/nullptr);
+  ParserConfig config(context);
+  ParserState state(sourceMgr, config, symbolState, /*asmState=*/nullptr);
   Parser parser(state);
 
   raw_ostream &os = printDiagnosticInfo ? llvm::errs() : llvm::nulls();
   SourceMgrDiagnosticHandler handler(sourceMgr, context, os);
   IntegerSet set;
   if (parser.parseIntegerSetReference(set))
     return IntegerSet();
 
   Token endTok = parser.getToken();
   if (endTok.isNot(Token::eof)) {
     parser.emitError(endTok.getLoc(), "encountered unexpected token");
     return IntegerSet();
   }
 
   return set;
 }

mlir/lib/Parser/AsmParserImpl.h

if (!parser.getToken().is(Token::string))
       return failure();
 
     if (string)
       *string = parser.getToken().getStringValue();
     parser.consumeToken();
     return success();
   }
 
-  /// Returns true if the current token corresponds to a keyword.
-  bool isCurrentTokenAKeyword() const {
-    return parser.getToken().isAny(Token::bare_identifier, Token::inttype) ||
-           parser.getToken().isKeyword();
-  }
-
   /// Parse the given keyword if present.
   ParseResult parseOptionalKeyword(StringRef keyword) override {
     // Check that the current token has the same spelling.
-    if (!isCurrentTokenAKeyword() || parser.getTokenSpelling() != keyword)
+    if (!parser.isCurrentTokenAKeyword() ||
+        parser.getTokenSpelling() != keyword)
       return failure();
     parser.consumeToken();
     return success();
   }
 
   /// Parse a keyword, if present, into 'keyword'.
   ParseResult parseOptionalKeyword(StringRef *keyword) override {
     // Check that the current token is a keyword.
-    if (!isCurrentTokenAKeyword())
+    if (!parser.isCurrentTokenAKeyword())
       return failure();
 
     *keyword = parser.getTokenSpelling();
     parser.consumeToken();
     return success();
   }
 
   /// Parse a keyword if it is one of the 'allowedKeywords'.
   ParseResult
   parseOptionalKeyword(StringRef *keyword,
                        ArrayRef<StringRef> allowedKeywords) override {
     // Check that the current token is a keyword.
-    if (!isCurrentTokenAKeyword())
+    if (!parser.isCurrentTokenAKeyword())
       return failure();
 
     StringRef currentKeyword = parser.getTokenSpelling();
     if (llvm::is_contained(allowedKeywords, currentKeyword)) {
       *keyword = currentKeyword;
       parser.consumeToken();
       return success();
     }
ParseResult parseOptionalSymbolName(StringAttr &result, StringRef attrName,
     if (parser.getState().asmState) {
       parser.getState().asmState->addUses(SymbolRefAttr::get(result),
                                           atToken.getLocRange());
     }
     return success();
   }
 
   //===--------------------------------------------------------------------===//
+  // Resource Parsing
+  //===--------------------------------------------------------------------===//
+
+  /// Parse a handle to a resource within the assembly format.
+  FailureOr<AsmDialectResourceHandle>
+  parseResourceHandle(Dialect *dialect) override {
+    const auto *interface = dyn_cast_or_null<OpAsmDialectInterface>(dialect);
+    if (!interface) {
+      return parser.emitError() << "dialect '" << dialect->getNamespace()
+                                << "' does not expect resource handles";
+    }
+    StringRef resourceName;
+    return parser.parseResourceHandle(interface, resourceName);
+  }
+
+  //===--------------------------------------------------------------------===//
   // Type Parsing
   //===--------------------------------------------------------------------===//
 
   /// Parse a type.
   ParseResult parseType(Type &result) override {
     return failure(!(result = parser.parseType()));
   }
 

mlir/lib/Parser/DialectSymbolParser.cpp

 static T parseSymbol(StringRef inputStr, MLIRContext *context,
                      SymbolState &symbolState, ParserFn &&parserFn,
                      size_t *numRead = nullptr) {
   SourceMgr sourceMgr;
   auto memBuffer = MemoryBuffer::getMemBuffer(
       inputStr, /*BufferName=*/"<mlir_parser_buffer>",
       /*RequiresNullTerminator=*/false);
   sourceMgr.AddNewSourceBuffer(std::move(memBuffer), SMLoc());
-  ParserState state(sourceMgr, context, symbolState, /*asmState=*/nullptr);
+  ParserConfig config(context);
+  ParserState state(sourceMgr, config, symbolState, /*asmState=*/nullptr);
   Parser parser(state);
 
   Token startTok = parser.getToken();
   T symbol = parserFn(parser);
   if (!symbol)
     return T();
 
   // If 'numRead' is valid, then provide the number of bytes that were read.

 /// Parse an extended attribute.
 ///
 ///   extended-attribute ::= (dialect-attribute | attribute-alias)
 ///   dialect-attribute  ::= `#` dialect-namespace `<` `"` attr-data `"` `>`
 ///   dialect-attribute  ::= `#` alias-name pretty-dialect-sym-body?
 ///   attribute-alias    ::= `#` alias-name
 ///
 Attribute Parser::parseExtendedAttr(Type type) {
+  MLIRContext *ctx = getContext();
   Attribute attr = parseExtendedSymbol<Attribute>(
       *this, Token::hash_identifier, state.symbols.attributeAliasDefinitions,
       [&](StringRef dialectName, StringRef symbolData,
           SMLoc loc) -> Attribute {
         // Parse an optional trailing colon type.
         Type attrType = type;
         if (consumeIf(Token::colon) && !(attrType = parseType()))
           return Attribute();
 
         // If we found a registered dialect, then ask it to parse the attribute.
         if (Dialect *dialect =
                 builder.getContext()->getOrLoadDialect(dialectName)) {
           return parseSymbol<Attribute>(
-              symbolData, state.context, state.symbols, [&](Parser &parser) {
+              symbolData, ctx, state.symbols, [&](Parser &parser) {
                 CustomDialectAsmParser customParser(symbolData, parser);
                 return dialect->parseAttribute(customParser, attrType);
               });
         }
 
         // Otherwise, form a new opaque attribute.
         return OpaqueAttr::getChecked(
-            [&] { return emitError(loc); },
-            StringAttr::get(state.context, dialectName), symbolData,
-            attrType ? attrType : NoneType::get(state.context));
+            [&] { return emitError(loc); }, StringAttr::get(ctx, dialectName),
+            symbolData, attrType ? attrType : NoneType::get(ctx));
       });
 
   // Ensure that the attribute has the same type as requested.
   if (attr && type && attr.getType() != type) {
     emitError("attribute type different than expected: expected ")
         << type << ", but got " << attr.getType();
     return nullptr;
   }
   return attr;
 }
 
 /// Parse an extended type.
 ///
 ///   extended-type ::= (dialect-type | type-alias)
 ///   dialect-type  ::= `!` dialect-namespace `<` `"` type-data `"` `>`
 ///   dialect-type  ::= `!` alias-name pretty-dialect-attribute-body?
 ///   type-alias    ::= `!` alias-name
 ///
 Type Parser::parseExtendedType() {
+  MLIRContext *ctx = getContext();
   return parseExtendedSymbol<Type>(
       *this, Token::exclamation_identifier, state.symbols.typeAliasDefinitions,
-      [&](StringRef dialectName, StringRef symbolData,
-          SMLoc loc) -> Type {
+      [&](StringRef dialectName, StringRef symbolData, SMLoc loc) -> Type {
         // If we found a registered dialect, then ask it to parse the type.
-        auto *dialect = state.context->getOrLoadDialect(dialectName);
-
-        if (dialect) {
+        if (auto *dialect = ctx->getOrLoadDialect(dialectName)) {
           return parseSymbol<Type>(
-              symbolData, state.context, state.symbols, [&](Parser &parser) {
+              symbolData, ctx, state.symbols, [&](Parser &parser) {
                 CustomDialectAsmParser customParser(symbolData, parser);
                 return dialect->parseType(customParser);
               });
         }
 
         // Otherwise, form a new opaque type.
-        return OpaqueType::getChecked(
-            [&] { return emitError(loc); },
-            StringAttr::get(state.context, dialectName), symbolData);
+        return OpaqueType::getChecked([&] { return emitError(loc); },
+                                      StringAttr::get(ctx, dialectName),
+                                      symbolData);
       });
 }
 
 //===----------------------------------------------------------------------===//
 // mlir::parseAttribute/parseType
 //===----------------------------------------------------------------------===//
 
 /// Parses a symbol, of type 'T', and returns it if parsing was successful. If

mlir/lib/Parser/Lexer.cpp

case ',':
       return formToken(Token::comma, tokStart);
     case '.':
       return lexEllipsis(tokStart);
     case '(':
       return formToken(Token::l_paren, tokStart);
     case ')':
       return formToken(Token::r_paren, tokStart);
     case '{':
+      if (*curPtr == '-' && *(curPtr + 1) == '#') {
+        curPtr += 2;
+        return formToken(Token::file_metadata_begin, tokStart);
+      }
       return formToken(Token::l_brace, tokStart);
     case '}':
       return formToken(Token::r_brace, tokStart);
     case '[':
       return formToken(Token::l_square, tokStart);
     case ']':
       return formToken(Token::r_square, tokStart);
     case '<':
case '/':
         skipComment();
         continue;
       }
       return emitError(tokStart, "unexpected character");
 
     case '@':
       return lexAtIdentifier(tokStart);
 
-    case '!':
-      LLVM_FALLTHROUGH;
-    case '^':
-      LLVM_FALLTHROUGH;
     case '#':
+      if (*curPtr == '-' && *(curPtr + 1) == '}') {
+        curPtr += 2;
+        return formToken(Token::file_metadata_end, tokStart);
+      }
       LLVM_FALLTHROUGH;
+    case '!':
+    case '^':
     case '%':
       return lexPrefixedIdentifier(tokStart);
     case '"':
       return lexString(tokStart);
 
     case '0':
     case '1':
     case '2':

mlir/lib/Parser/Parser.h

 /// types. It is intended to be subclassed by specialized subparsers that
 /// include state.
 class Parser {
 public:
   using Delimiter = OpAsmParser::Delimiter;
 
   Builder builder;
 
-  Parser(ParserState &state) : builder(state.context), state(state) {}
+  Parser(ParserState &state)
+      : builder(state.config.getContext()), state(state) {}
 
   // Helper methods to get stuff from the parser-global state.
   ParserState &getState() const { return state; }
-  MLIRContext *getContext() const { return state.context; }
+  MLIRContext *getContext() const { return state.config.getContext(); }
   const llvm::SourceMgr &getSourceMgr() { return state.lex.getSourceMgr(); }
 
   /// Parse a comma-separated list of elements up until the specified end token.
   ParseResult
   parseCommaSeparatedListUntil(Token::Kind rightToken,
                                function_ref<ParseResult()> parseElement,
                                bool allowEmptyList = true);
 
public:
   OptionalParseResult parseOptionalInteger(APInt &result);
 
   /// Parse a floating point value from an integer literal token.
   ParseResult parseFloatFromIntegerLiteral(Optional<APFloat> &result,
                                            const Token &tok, bool isNegative,
                                            const llvm::fltSemantics &semantics,
                                            size_t typeSizeInBits);
 
+  /// Returns true if the current token corresponds to a keyword.
+  bool isCurrentTokenAKeyword() const {
+    return getToken().isAny(Token::bare_identifier, Token::inttype) ||
+           getToken().isKeyword();
+  }
+
+  /// Parse a keyword, if present, into 'keyword'.
+  ParseResult parseOptionalKeyword(StringRef *keyword);
+
+  //===--------------------------------------------------------------------===//
+  // Resource Parsing
+  //===--------------------------------------------------------------------===//
+
+  /// Parse a handle to a dialect resource within the assembly format.
+  FailureOr<AsmDialectResourceHandle>
+  parseResourceHandle(const OpAsmDialectInterface *dialect, StringRef &name);
+
   //===--------------------------------------------------------------------===//
   // Type Parsing
   //===--------------------------------------------------------------------===//
 
   /// Invoke the `getChecked` method of the given Attribute or Type class, using
   /// the provided location to emit errors in the case of failure. Note that
   /// unlike `OpBuilder::getType`, this method does not implicitly insert a
   /// context parameter.

mlir/lib/Parser/Parser.cpp

 //===- Parser.cpp - MLIR Parser Implementation ----------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements the parser for the MLIR textual form.
 //
 //===----------------------------------------------------------------------===//
 
 #include "Parser.h"
 #include "AsmParserImpl.h"
 #include "mlir/IR/AffineMap.h"
+#include "mlir/IR/AsmState.h"
 #include "mlir/IR/BuiltinOps.h"
 #include "mlir/IR/Dialect.h"
 #include "mlir/IR/Verifier.h"
 #include "mlir/Parser/AsmParserState.h"
 #include "mlir/Parser/Parser.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/ScopeExit.h"
 #include "llvm/ADT/StringSet.h"
ParseResult Parser::parseFloatFromIntegerLiteral(
   APInt apInt(typeSizeInBits, *value);
   if (apInt != *value)
     return emitError(loc, "hexadecimal float constant out of range for type");
   result = APFloat(semantics, apInt);
 
   return success();
 }
 
+ParseResult Parser::parseOptionalKeyword(StringRef *keyword) {
+  // Check that the current token is a keyword.
+  if (!isCurrentTokenAKeyword())
+    return failure();
+
+  *keyword = getTokenSpelling();
+  consumeToken();
+  return success();
+}
+
+//===----------------------------------------------------------------------===//
+// Resource Parsing
+
+FailureOr<AsmDialectResourceHandle>
+Parser::parseResourceHandle(const OpAsmDialectInterface *dialect,
+                            StringRef &name) {
+  assert(dialect && "expected valid dialect interface");
+  SMLoc nameLoc = getToken().getLoc();
+  if (failed(parseOptionalKeyword(&name)))
+    return emitError("expected identifier key for 'resource' entry");
+  auto &resources = getState().symbols.dialectResources;
+
+  // If this is the first time encountering this handle, ask the dialect to
+  // resolve a reference to this handle. This allows for us to remap the name of
+  // the handle if necessary.
+  std::pair<std::string, AsmDialectResourceHandle> &entry =
+      resources[dialect][name];
+  if (entry.first.empty()) {
+    FailureOr<AsmDialectResourceHandle> result = dialect->declareResource(name);
+    if (failed(result)) {
+      return emitError(nameLoc)
+             << "unknown 'resource' key '" << name << "' for dialect '"
+             << dialect->getDialect()->getNamespace() << "'";
+    }
+    entry.first = dialect->getResourceKey(*result);
+    entry.second = *result;
+  }
+
+  name = entry.first;
+  return entry.second;
+}
+
 //===----------------------------------------------------------------------===//
 // OperationParser
 //===----------------------------------------------------------------------===//
 
 namespace {
 /// This class provides support for parsing operations and regions of
 /// operations.
 class OperationParser : public Parser {

 public:
   explicit TopLevelOperationParser(ParserState &state) : Parser(state) {}
 
   /// Parse a set of operations into the end of the given Block.
   ParseResult parse(Block *topLevelBlock, Location parserLoc);
 
 private:
   /// Parse an attribute alias declaration.
+  ///
+  ///   attribute-alias-def ::= '#' alias-name `=` attribute-value
+  ///
   ParseResult parseAttributeAliasDef();
 
-  /// Parse an attribute alias declaration.
+  /// Parse a type alias declaration.
+  ///
+  ///   type-alias-def ::= '!' alias-name `=` type
+  ///
   ParseResult parseTypeAliasDef();
-};
-} // namespace
 
-/// Parses an attribute alias declaration.
+  /// Parse a top-level file metadata dictionary.
   ///
-///   attribute-alias-def ::= '#' alias-name `=` attribute-value
+  ///   file-metadata-dict ::= '{-#' file-metadata-entry* `#-}'
   ///
+  ParseResult parseFileMetadataDictionary();
+
+  /// Parse a resource metadata dictionary.
+  ParseResult parseResourceFileMetadata(
+      function_ref<ParseResult(StringRef, SMLoc)> parseBody);
+  ParseResult parseDialectResourceFileMetadata();
+  ParseResult parseExternalResourceFileMetadata();
+};
+
+/// This class represents an implementation of a resource entry for the MLIR
+/// textual format.
+class ParsedResourceEntry : public AsmParsedResourceEntry {
+public:
+  ParsedResourceEntry(StringRef key, SMLoc keyLoc, Token value, Parser &p)
+      : key(key), keyLoc(keyLoc), value(value), p(p) {}
+  ~ParsedResourceEntry() override = default;
+
+  StringRef getKey() const final { return key; }
+
+  InFlightDiagnostic emitError() const final { return p.emitError(keyLoc); }
+
+  FailureOr<bool> parseAsBool() const final {
+    if (value.is(Token::kw_true))
+      return true;
+    if (value.is(Token::kw_false))
+      return false;
+    return p.emitError(value.getLoc(),
+                       "expected 'true' or 'false' value for key '" + key +
+                           "'");
+  }
+
+  FailureOr<std::string> parseAsString() const final {
+    if (value.isNot(Token::string))
+      return p.emitError(value.getLoc(),
+                         "expected string value for key '" + key + "'");
+    return value.getStringValue();
+  }
+
+  FailureOr<AsmResourceBlob>
+  parseAsBlob(BlobAllocatorFn allocator) const final {
+    // Blob data within then textual format is represented as a hex string.
+    // TODO: We could avoid an additional alloc+copy here if we pre-allocated
+    // the buffer to use during hex processing.
+    Optional<std::string> blobData =
+        value.is(Token::string) ? value.getHexStringValue() : llvm::None;
+    if (!blobData)
+      return p.emitError(value.getLoc(),
+                         "expected hex string blob for key '" + key + "'");
+
+    // Extract the alignment of the blob data, which gets stored at the
+    // beginning of the string.
+    if (blobData->size() < sizeof(uint32_t)) {
+      return p.emitError(value.getLoc(),
+                         "expected hex string blob for key '" + key +
+                             "' to encode alignment in first 4 bytes");
+    }
+    uint32_t align = 0;
+    memcpy(&align, blobData->data(), sizeof(uint32_t));
+
+    // Get the data portion of the blob.
+    StringRef data = StringRef(*blobData).drop_front(sizeof(uint32_t));
+    if (data.empty())
+      return AsmResourceBlob();
+
+    // Allocate memory for the blob using the provided allocator and copy the
+    // data into it.
+    AsmResourceBlob blob = allocator(data.size(), align);
+    assert(llvm::isAddrAligned(llvm::Align(align), blob.getData().data()) &&
+           blob.isMutable() &&
+           "blob allocator did not return a properly aligned address");
+    memcpy(blob.getMutableData().data(), data.data(), data.size());
+    return blob;
+  }
+
+private:
+  StringRef key;
+  SMLoc keyLoc;
+  Token value;
+  Parser &p;
+};
+} // namespace
+
 ParseResult TopLevelOperationParser::parseAttributeAliasDef() {
   assert(getToken().is(Token::hash_identifier));
   StringRef aliasName = getTokenSpelling().drop_front();
 
   // Check for redefinitions.
   if (state.symbols.attributeAliasDefinitions.count(aliasName) > 0)
     return emitError("redefinition of attribute alias id '" + aliasName + "'");
 
ParseResult TopLevelOperationParser::parseAttributeAliasDef() {
   Attribute attr = parseAttribute();
   if (!attr)
     return failure();
 
   state.symbols.attributeAliasDefinitions[aliasName] = attr;
   return success();
 }
 
-/// Parse a type alias declaration.
-///
-///   type-alias-def ::= '!' alias-name `=` type
-///
 ParseResult TopLevelOperationParser::parseTypeAliasDef() {
   assert(getToken().is(Token::exclamation_identifier));
   StringRef aliasName = getTokenSpelling().drop_front();
 
   // Check for redefinitions.
   if (state.symbols.typeAliasDefinitions.count(aliasName) > 0)
     return emitError("redefinition of type alias id '" + aliasName + "'");
 
ParseResult TopLevelOperationParser::parseTypeAliasDef() {
   if (!aliasedType)
     return failure();
 
   // Register this alias with the parser state.
   state.symbols.typeAliasDefinitions.try_emplace(aliasName, aliasedType);
   return success();
 }
 
+ParseResult TopLevelOperationParser::parseFileMetadataDictionary() {
+  consumeToken(Token::file_metadata_begin);
+  return parseCommaSeparatedListUntil(
+      Token::file_metadata_end, [&]() -> ParseResult {
+        // Parse the key of the metadata dictionary.
+        SMLoc keyLoc = getToken().getLoc();
+        StringRef key;
+        if (failed(parseOptionalKeyword(&key)))
+          return emitError("expected identifier key in file "
+                           "metadata dictionary");
+        if (parseToken(Token::colon, "expected ':'"))
+          return failure();
+
+        // Process the metadata entry.
+        if (key == "dialect_resources")
+          return parseDialectResourceFileMetadata();
+        if (key == "external_resources")
+          return parseExternalResourceFileMetadata();
+        return emitError(keyLoc, "unknown key '" + key +
+                                     "' in file metadata dictionary");
+      });
+}
+
+ParseResult TopLevelOperationParser::parseResourceFileMetadata(
+    function_ref<ParseResult(StringRef, SMLoc)> parseBody) {
+  if (parseToken(Token::l_brace, "expected '{'"))
+    return failure();
+
+  return parseCommaSeparatedListUntil(Token::r_brace, [&]() -> ParseResult {
+    // Parse the top-level name entry.
+    SMLoc nameLoc = getToken().getLoc();
+    StringRef name;
+    if (failed(parseOptionalKeyword(&name)))
+      return emitError("expected identifier key for 'resource' entry");
+
+    if (parseToken(Token::colon, "expected ':'") ||
+        parseToken(Token::l_brace, "expected '{'"))
+      return failure();
+    return parseBody(name, nameLoc);
+  });
+}
+
+ParseResult TopLevelOperationParser::parseDialectResourceFileMetadata() {
+  return parseResourceFileMetadata([&](StringRef name,
+                                       SMLoc nameLoc) -> ParseResult {
+    // Lookup the dialect and check that it can handle a resource entry.
+    Dialect *dialect = getContext()->getOrLoadDialect(name);
+    if (!dialect)
+      return emitError(nameLoc, "dialect '" + name + "' is unknown");
+    const auto *handler = dyn_cast<OpAsmDialectInterface>(dialect);
+    if (!handler) {
+      return emitError() << "unexpected 'resource' section for dialect '"
+                         << dialect->getNamespace() << "'";
+    }
+
+    return parseCommaSeparatedListUntil(Token::r_brace, [&]() -> ParseResult {
+      // Parse the name of the resource entry.
+      SMLoc keyLoc = getToken().getLoc();
+      StringRef key;
+      if (failed(parseResourceHandle(handler, key)) ||
+          parseToken(Token::colon, "expected ':'"))
+        return failure();
+      Token valueTok = getToken();
+      consumeToken();
+
+      ParsedResourceEntry entry(key, keyLoc, valueTok, *this);
+      return handler->parseResource(entry);
+    });
+  });
+}
+
+ParseResult TopLevelOperationParser::parseExternalResourceFileMetadata() {
+  return parseResourceFileMetadata([&](StringRef name,
+                                       SMLoc nameLoc) -> ParseResult {
+    AsmResourceParser *handler = state.config.getResourceParser(name);
+
+    // TODO: Should we require handling external resources in some scenarios?
+    if (!handler) {
+      emitWarning(getEncodedSourceLocation(nameLoc))
+          << "ignoring unknown external resources for '" << name << "'";
+    }
+
+    return parseCommaSeparatedListUntil(Token::r_brace, [&]() -> ParseResult {
+      // Parse the name of the resource entry.
+      SMLoc keyLoc = getToken().getLoc();
+      StringRef key;
+      if (failed(parseOptionalKeyword(&key)))
+        return emitError(
+            "expected identifier key for 'external_resources' entry");
+      if (parseToken(Token::colon, "expected ':'"))
+        return failure();
+      Token valueTok = getToken();
+      consumeToken();
+
+      if (!handler)
+        return success();
+      ParsedResourceEntry entry(key, keyLoc, valueTok, *this);
+      return handler->parseResource(entry);
+    });
+  });
+}
+
 ParseResult TopLevelOperationParser::parse(Block *topLevelBlock,
                                            Location parserLoc) {
   // Create a top-level operation to contain the parsed state.
   OwningOpRef<ModuleOp> topLevelOp(ModuleOp::create(parserLoc));
   OperationParser opParser(state, topLevelOp.get());
   while (true) {
     switch (getToken().getKind()) {
     default:
case Token::hash_identifier:
         return failure();
       break;
 
     // Parse a type alias.
     case Token::exclamation_identifier:
       if (parseTypeAliasDef())
         return failure();
       break;
+
+      // Parse a file-level metadata dictionary.
+    case Token::file_metadata_begin:
+      if (parseFileMetadataDictionary())
+        return failure();
+      break;
     }
   }
 }
 
 //===----------------------------------------------------------------------===//
 
 LogicalResult mlir::parseSourceFile(const llvm::SourceMgr &sourceMgr,
-                                    Block *block, MLIRContext *context,
+                                    Block *block, const ParserConfig &config,
                                     LocationAttr *sourceFileLoc,
                                     AsmParserState *asmState) {
   const auto *sourceBuf = sourceMgr.getMemoryBuffer(sourceMgr.getMainFileID());
 
-  Location parserLoc = FileLineColLoc::get(
-      context, sourceBuf->getBufferIdentifier(), /*line=*/0, /*column=*/0);
+  Location parserLoc =
+      FileLineColLoc::get(config.getContext(), sourceBuf->getBufferIdentifier(),
+                          /*line=*/0, /*column=*/0);
   if (sourceFileLoc)
     *sourceFileLoc = parserLoc;
 
   SymbolState aliasState;
-  ParserState state(sourceMgr, context, aliasState, asmState);
+  ParserState state(sourceMgr, config, aliasState, asmState);
   return TopLevelOperationParser(state).parse(block, parserLoc);
 }
 
 LogicalResult mlir::parseSourceFile(llvm::StringRef filename, Block *block,
-                                    MLIRContext *context,
+                                    const ParserConfig &config,
                                     LocationAttr *sourceFileLoc) {
   llvm::SourceMgr sourceMgr;
-  return parseSourceFile(filename, sourceMgr, block, context, sourceFileLoc);
+  return parseSourceFile(filename, sourceMgr, block, config, sourceFileLoc);
 }
 
 LogicalResult mlir::parseSourceFile(llvm::StringRef filename,
                                     llvm::SourceMgr &sourceMgr, Block *block,
-                                    MLIRContext *context,
+                                    const ParserConfig &config,
                                     LocationAttr *sourceFileLoc,
                                     AsmParserState *asmState) {
   if (sourceMgr.getNumBuffers() != 0) {
     // TODO: Extend to support multiple buffers.
-    return emitError(mlir::UnknownLoc::get(context),
+    return emitError(mlir::UnknownLoc::get(config.getContext()),
                      "only main buffer parsed at the moment");
   }
   auto fileOrErr = llvm::MemoryBuffer::getFileOrSTDIN(filename);
   if (std::error_code error = fileOrErr.getError())
-    return emitError(mlir::UnknownLoc::get(context),
+    return emitError(mlir::UnknownLoc::get(config.getContext()),
                      "could not open input file " + filename);
 
   // Load the MLIR source file.
   sourceMgr.AddNewSourceBuffer(std::move(*fileOrErr), SMLoc());
-  return parseSourceFile(sourceMgr, block, context, sourceFileLoc, asmState);
+  return parseSourceFile(sourceMgr, block, config, sourceFileLoc, asmState);
 }
 
 LogicalResult mlir::parseSourceString(llvm::StringRef sourceStr, Block *block,
-                                      MLIRContext *context,
+                                      const ParserConfig &config,
                                       LocationAttr *sourceFileLoc) {
   auto memBuffer = MemoryBuffer::getMemBuffer(sourceStr);
   if (!memBuffer)
     return failure();
 
   SourceMgr sourceMgr;
   sourceMgr.AddNewSourceBuffer(std::move(memBuffer), SMLoc());
-  return parseSourceFile(sourceMgr, block, context, sourceFileLoc);
+  return parseSourceFile(sourceMgr, block, config, sourceFileLoc);
 }

mlir/lib/Parser/ParserState.h

 namespace detail {
 
 //===----------------------------------------------------------------------===//
 // SymbolState
 //===----------------------------------------------------------------------===//
 
 /// This class contains record of any parsed top-level symbols.
 struct SymbolState {
-  // A map from attribute alias identifier to Attribute.
+  /// A map from attribute alias identifier to Attribute.
   llvm::StringMap<Attribute> attributeAliasDefinitions;
 
-  // A map from type alias identifier to Type.
+  /// A map from type alias identifier to Type.
   llvm::StringMap<Type> typeAliasDefinitions;
 
+  /// A map of dialect resource keys to the resolved resource name and handle
+  /// to use during parsing.
+  DenseMap<const OpAsmDialectInterface *,
+           llvm::StringMap<std::pair<std::string, AsmDialectResourceHandle>>>
+      dialectResources;
+
   /// A set of locations into the main parser memory buffer for each of the
   /// active nested parsers. Given that some nested parsers, i.e. custom dialect
   /// parsers, operate on a temporary memory buffer, this provides an anchor
   /// point for emitting diagnostics.
   SmallVector<SMLoc, 1> nestedParserLocs;
 
   /// The top-level lexer that contains the original memory buffer provided by
   /// the user. This is used by nested parsers to get a properly encoded source
   /// location.
   Lexer *topLevelLexer = nullptr;
 };
 
 //===----------------------------------------------------------------------===//
 // ParserState
 //===----------------------------------------------------------------------===//
 
 /// This class refers to all of the state maintained globally by the parser,
 /// such as the current lexer position etc.
 struct ParserState {
-  ParserState(const llvm::SourceMgr &sourceMgr, MLIRContext *ctx,
+  ParserState(const llvm::SourceMgr &sourceMgr, const ParserConfig &config,
               SymbolState &symbols, AsmParserState *asmState)
-      : context(ctx), lex(sourceMgr, ctx), curToken(lex.lexToken()),
-        symbols(symbols), parserDepth(symbols.nestedParserLocs.size()),
-        asmState(asmState) {
+      : config(config), lex(sourceMgr, config.getContext()),
+        curToken(lex.lexToken()), symbols(symbols),
+        parserDepth(symbols.nestedParserLocs.size()), asmState(asmState) {
     // Set the top level lexer for the symbol state if one doesn't exist.
     if (!symbols.topLevelLexer)
       symbols.topLevelLexer = &lex;
   }
   ~ParserState() {
     // Reset the top level lexer if it refers the lexer in our state.
     if (symbols.topLevelLexer == &lex)
       symbols.topLevelLexer = nullptr;
   }
   ParserState(const ParserState &) = delete;
   void operator=(const ParserState &) = delete;
 
-  /// The context we're parsing into.
-  MLIRContext *const context;
+  /// The configuration used to setup the parser.
+  const ParserConfig &config;
 
   /// The lexer for the source file we're parsing.
   Lexer lex;
 
   /// This is the next token that hasn't been consumed yet.
   Token curToken;
 
   /// The current state for symbol parsing.

mlir/lib/Parser/Token.cpp

 /// Given a token containing a hex string literal, return its value or None if
 /// the token does not contain a valid hex string.
 Optional<std::string> Token::getHexStringValue() const {
   assert(getKind() == string);
 
   // Get the internal string data, without the quotes.
   StringRef bytes = getSpelling().drop_front().drop_back();
 
-  // Try to extract the binary data from the hex string.
+  // Try to extract the binary data from the hex string. We expect the hex
+  // string to start with `0x` and have an even number of hex nibbles (nibbles
+  // should come in pairs).
   std::string hex;
-  if (!bytes.consume_front("0x") || !llvm::tryGetFromHex(bytes, hex))
+  if (!bytes.consume_front("0x") || (bytes.size() & 1) ||
+      !llvm::tryGetFromHex(bytes, hex))
     return llvm::None;
   return hex;
 }
 
 /// Given a token containing a symbol reference, return the unescaped string
 /// value.
 std::string Token::getSymbolReference() const {
   assert(is(Token::at_identifier) && "expected valid @-identifier");

mlir/lib/Parser/TokenKinds.def

 TOK_PUNCTUATION(plus, "+")
 TOK_PUNCTUATION(question, "?")
 TOK_PUNCTUATION(r_brace, "}")
 TOK_PUNCTUATION(r_paren, ")")
 TOK_PUNCTUATION(r_square, "]")
 TOK_PUNCTUATION(star, "*")
 TOK_PUNCTUATION(vertical_bar, "|")
 
+TOK_PUNCTUATION(file_metadata_begin, "{-#")
+TOK_PUNCTUATION(file_metadata_end, "#-}")
+
 // Keywords.  These turn "foo" into Token::kw_foo enums.
 
 // NOTE: Please key these alphabetized to make it easier to find something in
 // this list and to cater to OCD.
 TOK_KEYWORD(affine_map)
 TOK_KEYWORD(affine_set)
 TOK_KEYWORD(attributes)
 TOK_KEYWORD(bf16)

mlir/lib/Parser/TypeParser.cpp

Type Parser::parseMemRefType() {
   auto parseElt = [&]() -> ParseResult {
     // Check for AffineMap as offset/strides.
     if (getToken().is(Token::kw_offset)) {
       int64_t offset;
       SmallVector<int64_t, 4> strides;
       if (failed(parseStridedLayout(offset, strides)))
         return failure();
       // Construct strided affine map.
-      AffineMap map =
-          makeStridedLinearLayoutMap(strides, offset, state.context);
+      AffineMap map = makeStridedLinearLayoutMap(strides, offset, getContext());
       layout = AffineMapAttr::get(map);
     } else {
       // Either it is MemRefLayoutAttrInterface or memory space attribute.
       Attribute attr = parseAttribute();
       if (!attr)
         return failure();
 
       if (attr.isa<MemRefLayoutAttrInterface>()) {

mlir/test/IR/elements-attr-interface.mlir

 // expected-error@below {{Test iterating `int32_t`: 10, 11, -12, 13, 14}}
 arith.constant [:i32 10, 11, -12, 13, 14]
 // expected-error@below {{Test iterating `int64_t`: 10, 11, -12, 13, 14}}
 arith.constant [:i64 10, 11, -12, 13, 14]
 // expected-error@below {{Test iterating `float`: 10.00, 11.00, -12.00, 13.00, 14.00}}
 arith.constant [:f32 10., 11., -12., 13., 14.]
 // expected-error@below {{Test iterating `double`: 10.00, 11.00, -12.00, 13.00, 14.00}}
 arith.constant [:f64 10., 11., -12., 13., 14.]
+
+// Check that we handle an external constant parsed from the config.
+// expected-error@below {{Test iterating `int64_t`: unable to iterate type}}
+// expected-error@below {{Test iterating `uint64_t`: 1, 2, 3}}
+// expected-error@below {{Test iterating `APInt`: unable to iterate type}}
+// expected-error@below {{Test iterating `IntegerAttr`: unable to iterate type}}
+arith.constant #test.e1di64_elements<blob1> : tensor<3xi64>
+
+{-#
+  dialect_resources: {
+    test: {
+      blob1: "0x08000000010000000000000002000000000000000300000000000000"
+    }
+  }
+#-}

mlir/test/IR/file-metadata-resources.mlir

+// RUN: mlir-opt %s -split-input-file | FileCheck %s
+
+// Check that we only preserve the blob that got referenced.
+// CHECK:      test: {
+// CHECK-NEXT:   blob1: "0x08000000010000000000000002000000000000000300000000000000"
+// CHECK-NEXT: }
+
+module attributes { test.blob_ref = #test.e1di64_elements<blob1> } {}
+
+{-#
+  dialect_resources: {
+    test: {
+      blob1: "0x08000000010000000000000002000000000000000300000000000000",
+      blob2: "0x08000000040000000000000005000000000000000600000000000000"
+    }
+  }
+#-}

mlir/test/IR/invalid-file-metadata.mlir

+// RUN: mlir-opt %s -split-input-file -verify-diagnostics
+
+// expected-error@+2 {{expected identifier key in file metadata dictionary}}
+{-#
+
+// -----
+
+// expected-error@+2 {{expected ':'}}
+{-#
+  key
+#-}
+
+// -----
+
+// expected-error@+2 {{unknown key 'some_key' in file metadata dictionary}}
+{-#
+  some_key: {}
+#-}
+
+// -----
+
+//===----------------------------------------------------------------------===//
+// `dialect_resources`
+//===----------------------------------------------------------------------===//
+
+// expected-error@+2 {{expected '{'}}
+{-#
+  dialect_resources: "value"
+#-}
+
+// -----
+
+// expected-error@+3 {{expected identifier key for 'resource' entry}}
+{-#
+  dialect_resources: {
+    10
+  }
+#-}
+
+// -----
+
+// expected-error@+3 {{expected ':'}}
+{-#
+  dialect_resources: {
+    entry "value"
+  }
+#-}
+
+// -----
+
+// expected-error@+3 {{dialect 'foobar' is unknown}}
+{-#
+  dialect_resources: {
+    foobar: {
+      entry: "foo"
+    }
+  }
+#-}
+
+// -----
+
+// expected-error@+4 {{unknown 'resource' key 'unknown_entry' for dialect 'builtin'}}
+{-#
+  dialect_resources: {
+    builtin: {
+      unknown_entry: "foo"
+    }
+  }
+#-}
+
+// -----
+
+// expected-error@+4 {{expected hex string blob for key 'invalid_blob'}}
+{-#
+  dialect_resources: {
+    test: {
+      invalid_blob: 10
+    }
+  }
+#-}
+
+// -----
+
+// expected-error@+4 {{expected hex string blob for key 'invalid_blob'}}
+{-#
+  dialect_resources: {
+    test: {
+      invalid_blob: ""
+    }
+  }
+#-}
+
+// -----
+
+// expected-error@+4 {{expected hex string blob for key 'invalid_blob' to encode alignment in first 4 bytes}}
+{-#
+  dialect_resources: {
+    test: {
+      invalid_blob: "0x"
+    }
+  }
+#-}
+
+// -----
+
+//===----------------------------------------------------------------------===//
+// `external_resources`
+//===----------------------------------------------------------------------===//
+
+// expected-error@+2 {{expected '{'}}
+{-#
+  external_resources: "value"
+#-}
+
+// -----
+
+// expected-error@+3 {{expected identifier key for 'resource' entry}}
+{-#
+  external_resources: {
+    10
+  }
+#-}
+
+// -----
+
+// expected-error@+3 {{expected ':'}}
+{-#
+  external_resources: {
+    entry "value"
+  }
+#-}
+
+// -----
+
+// expected-warning@+3 {{ignoring unknown external resources for 'foobar'}}
+{-#
+  external_resources: {
+    foobar: {
+      entry: "foo"
+    }
+  }
+#-}

mlir/test/lib/Dialect/Test/TestAttrDefs.td

 
 #ifndef TEST_ATTRDEFS
 #define TEST_ATTRDEFS
 
 // To get the test dialect definition.
 include "TestDialect.td"
 include "mlir/IR/AttrTypeBase.td"
 include "mlir/IR/BuiltinAttributeInterfaces.td"
+include "mlir/IR/OpAsmInterface.td"
 include "mlir/IR/SubElementInterfaces.td"
 
 // All of the attributes will extend this class.
 class Test_Attr<string name, list<Trait> traits = []>
     : AttrDef<Test_Dialect, name, traits>;
 
 def SimpleAttrA : Test_Attr<"SimpleA"> {
   let mnemonic = "smpla";

 def TestAttrSelfTypeParameterFormat
     : Test_Attr<"TestAttrSelfTypeParameterFormat"> {
   let parameters = (ins "int":$a, AttributeSelfTypeParameter<"">:$type);
 
   let mnemonic = "attr_self_type_format";
   let assemblyFormat = "`<` $a `>`";
 }
 
+// Test simple extern 1D vector using ElementsAttrInterface.
+def TestExtern1DI64ElementsAttr : Test_Attr<"TestExtern1DI64Elements", [
+    ElementsAttrInterface
+  ]> {
+  let mnemonic = "e1di64_elements";
+  let parameters = (ins
+    AttributeSelfTypeParameter<"", "::mlir::ShapedType">:$type,
+    ResourceHandleParameter<"TestExternalElementsDataHandle">:$handle
+  );
+  let extraClassDeclaration = [{
+    /// Return the elements referenced by this attribute.
+    llvm::ArrayRef<uint64_t> getElements() const;
+
+    /// The set of data types that can be iterated by this attribute.
+    using ContiguousIterableTypesT = std::tuple<uint64_t>;
+
+    /// Provide begin iterators for the various iterable types.
+    // * uint64_t
+    auto value_begin_impl(OverloadToken<uint64_t>) const {
+      return getElements().begin();
+    }
+  }];
+  let assemblyFormat = "`<` $handle `>`";
+}
+
 #endif // TEST_ATTRDEFS

mlir/test/lib/Dialect/Test/TestAttributes.h

 #include "mlir/IR/Attributes.h"
 #include "mlir/IR/Diagnostics.h"
 #include "mlir/IR/Dialect.h"
 #include "mlir/IR/DialectImplementation.h"
 
 #include "TestAttrInterfaces.h.inc"
 #include "TestOpEnums.h.inc"
 
+namespace test {
+struct TestExternalElementsDataHandle;
+} // namespace test
+
 #define GET_ATTRDEF_CLASSES
 #include "TestAttrDefs.h.inc"
 
 #endif // MLIR_TESTATTRIBUTES_H

mlir/test/lib/Dialect/Test/TestAttributes.cpp

case 2:
       third = it.second;
       break;
     }
   }
   return get(getContext(), first, second, third);
 }
 
 //===----------------------------------------------------------------------===//
+// TestExtern1DI64ElementsAttr
+//===----------------------------------------------------------------------===//
+
+ArrayRef<uint64_t> TestExtern1DI64ElementsAttr::getElements() const {
+  return getHandle().getData()->getData();
+}
+
+//===----------------------------------------------------------------------===//
 // Tablegen Generated Definitions
 //===----------------------------------------------------------------------===//
 
 #include "TestAttrInterfaces.cpp.inc"
 
 #define GET_ATTRDEF_CLASSES
 #include "TestAttrDefs.cpp.inc"
 

mlir/test/lib/Dialect/Test/TestDialect.h

 
 #include "TestAttributes.h"
 #include "TestInterfaces.h"
 #include "mlir/Dialect/DLTI/DLTI.h"
 #include "mlir/Dialect/DLTI/Traits.h"
 #include "mlir/Dialect/Func/IR/FuncOps.h"
 #include "mlir/Dialect/Linalg/IR/Linalg.h"
 #include "mlir/Dialect/Traits.h"
+#include "mlir/IR/AsmState.h"
 #include "mlir/IR/BuiltinOps.h"
 #include "mlir/IR/BuiltinTypes.h"
 #include "mlir/IR/Dialect.h"
 #include "mlir/IR/ExtensibleDialect.h"
 #include "mlir/IR/OpDefinition.h"
 #include "mlir/IR/OpImplementation.h"
 #include "mlir/IR/RegionKindInterface.h"
 #include "mlir/IR/SymbolTable.h"
 #include "mlir/Interfaces/CallInterfaces.h"
 #include "mlir/Interfaces/ControlFlowInterfaces.h"
 #include "mlir/Interfaces/CopyOpInterface.h"
 #include "mlir/Interfaces/DerivedAttributeOpInterface.h"
 #include "mlir/Interfaces/InferIntRangeInterface.h"
 #include "mlir/Interfaces/InferTypeOpInterface.h"
 #include "mlir/Interfaces/LoopLikeInterface.h"
 #include "mlir/Interfaces/SideEffectInterfaces.h"
 #include "mlir/Interfaces/ViewLikeInterface.h"
 
 namespace mlir {
 class DLTIDialect;
 class RewritePatternSet;
 } // namespace mlir
 
+namespace test {
+class TestDialect;
+
+//===----------------------------------------------------------------------===//
+// External Elements Data
+//===----------------------------------------------------------------------===//
+
+/// This class represents a single external elements instance. It keeps track of
+/// the data, and deallocates when destructed.
+class TestExternalElementsData : public mlir::AsmResourceBlob {
+public:
+  using mlir::AsmResourceBlob::AsmResourceBlob;
+  TestExternalElementsData(mlir::AsmResourceBlob &&blob)
+      : mlir::AsmResourceBlob(std::move(blob)) {}
+
+  /// Return the data of this external elements instance.
+  llvm::ArrayRef<uint64_t> getData() const;
+
+  /// Allocate a new external elements instance with the given number of
+  /// elements.
+  static TestExternalElementsData allocate(size_t numElements);
+};
+
+/// A handle used to reference external elements instances.
+struct TestExternalElementsDataHandle
+    : public mlir::AsmDialectResourceHandleBase<
+          TestExternalElementsDataHandle,
+          llvm::StringMapEntry<std::unique_ptr<TestExternalElementsData>>,
+          TestDialect> {
+  using AsmDialectResourceHandleBase::AsmDialectResourceHandleBase;
+
+  /// Return a key to use for this handle.
+  llvm::StringRef getKey() const { return getResource()->getKey(); }
+
+  /// Return the data referenced by this handle.
+  TestExternalElementsData *getData() const {
+    return getResource()->getValue().get();
+  }
+};
+
+/// This class acts as a manager for external elements data. It provides API
+/// for creating and accessing registered elements data.
+class TestExternalElementsDataManager {
+  using DataMap = llvm::StringMap<std::unique_ptr<TestExternalElementsData>>;
+
+public:
+  /// Return the data registered for the given name, or nullptr if no data is
+  /// registered.
+  const TestExternalElementsData *getData(llvm::StringRef name) const;
+
+  /// Register an entry with the provided name, which may be modified if another
+  /// entry was already inserted with that name. Returns the inserted entry.
+  std::pair<DataMap::iterator, bool> insert(llvm::StringRef name);
+
+  /// Set the data for the given entry, which is expected to exist.
+  void setData(llvm::StringRef name, TestExternalElementsData &&data);
+
+private:
+  llvm::StringMap<std::unique_ptr<TestExternalElementsData>> dataMap;
+};
+} // namespace test
+
+//===----------------------------------------------------------------------===//
+// TestDialect
+//===----------------------------------------------------------------------===//
+
 #include "TestOpInterfaces.h.inc"
 #include "TestOpsDialect.h.inc"
 
 #define GET_OP_CLASSES
 #include "TestOps.h.inc"
 
 namespace test {
 void registerTestDialect(::mlir::DialectRegistry &registry);
 void populateTestReductionPatterns(::mlir::RewritePatternSet &patterns);
 } // namespace test
 
 #endif // MLIR_TESTDIALECT_H

mlir/test/lib/Dialect/Test/TestDialect.cpp

 //===- TestDialect.cpp - MLIR Dialect for Testing -------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "TestDialect.h"
 #include "TestAttributes.h"
 #include "TestInterfaces.h"
 #include "TestTypes.h"
 #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
 #include "mlir/Dialect/DLTI/DLTI.h"
 #include "mlir/Dialect/Func/IR/FuncOps.h"
 #include "mlir/Dialect/Tensor/IR/Tensor.h"
+#include "mlir/IR/AsmState.h"
 #include "mlir/IR/BuiltinAttributes.h"
 #include "mlir/IR/BuiltinOps.h"
 #include "mlir/IR/Diagnostics.h"
 #include "mlir/IR/DialectImplementation.h"
 #include "mlir/IR/ExtensibleDialect.h"
 #include "mlir/IR/MLIRContext.h"
 #include "mlir/IR/OperationSupport.h"
 #include "mlir/IR/PatternMatch.h"

 using namespace mlir;
 using namespace test;
 
 void test::registerTestDialect(DialectRegistry &registry) {
   registry.insert<TestDialect>();
 }
 
 //===----------------------------------------------------------------------===//
+// External Elements Data
+//===----------------------------------------------------------------------===//
+
+ArrayRef<uint64_t> TestExternalElementsData::getData() const {
+  ArrayRef<char> data = AsmResourceBlob::getData();
+  return ArrayRef<uint64_t>((const uint64_t *)data.data(),
+                            data.size() / sizeof(uint64_t));
+}
+
+TestExternalElementsData
+TestExternalElementsData::allocate(size_t numElements) {
+  return TestExternalElementsData(
+      llvm::ArrayRef<uint64_t>(new uint64_t[numElements], numElements),
+      [](const uint64_t *data, size_t) { delete[] data; },
+      /*dataIsMutable=*/true);
+}
+
+const TestExternalElementsData *
+TestExternalElementsDataManager::getData(StringRef name) const {
+  auto it = dataMap.find(name);
+  return it != dataMap.end() ? &*it->second : nullptr;
+}
+
+std::pair<TestExternalElementsDataManager::DataMap::iterator, bool>
+TestExternalElementsDataManager::insert(StringRef name) {
+  auto it = dataMap.try_emplace(name, nullptr);
+  if (it.second)
+    return it;
+
+  llvm::SmallString<32> nameStorage(name);
+  nameStorage.push_back('_');
+  size_t nameCounter = 1;
+  do {
+    nameStorage += std::to_string(nameCounter++);
+    auto it = dataMap.try_emplace(nameStorage, nullptr);
+    if (it.second)
+      return it;
+    nameStorage.resize(name.size() + 1);
+  } while (true);
+}
+
+void TestExternalElementsDataManager::setData(StringRef name,
+                                              TestExternalElementsData &&data) {
+  auto it = dataMap.find(name);
+  assert(it != dataMap.end() && "data not registered");
+  it->second = std::make_unique<TestExternalElementsData>(std::move(data));
+}
+
+//===----------------------------------------------------------------------===//
 // TestDialect Interfaces
 //===----------------------------------------------------------------------===//
 
 namespace {
 
 /// Testing the correctness of some traits.
 static_assert(
     llvm::is_detected<OpTrait::has_implicit_terminator_t,
                       SingleBlockImplicitTerminatorOp>::value,
     "has_implicit_terminator_t does not match SingleBlockImplicitTerminatorOp");
 static_assert(OpTrait::hasSingleBlockImplicitTerminator<
                   SingleBlockImplicitTerminatorOp>::value,
               "hasSingleBlockImplicitTerminator does not match "
               "SingleBlockImplicitTerminatorOp");
 
 // Test support for interacting with the AsmPrinter.
 struct TestOpAsmInterface : public OpAsmDialectInterface {
   using OpAsmDialectInterface::OpAsmDialectInterface;
 
+  //===------------------------------------------------------------------===//
+  // Aliases
+  //===------------------------------------------------------------------===//
+
   AliasResult getAlias(Attribute attr, raw_ostream &os) const final {
     StringAttr strAttr = attr.dyn_cast<StringAttr>();
     if (!strAttr)
       return AliasResult::NoAlias;
 
     // Check the contents of the string attribute to see what the test alias
     // should be named.
     Optional<StringRef> aliasName =
if (auto intType = type.dyn_cast<TestIntegerType>()) {
               TestIntegerType::SignednessSemantics::Unsigned &&
           intType.getWidth() == 8) {
         os << "test_ui8";
         return AliasResult::FinalAlias;
       }
     }
     return AliasResult::NoAlias;
   }
+
+  //===------------------------------------------------------------------===//
+  // Resources
+  //===------------------------------------------------------------------===//
+
+  std::string
+  getResourceKey(const AsmDialectResourceHandle &handle) const override {
+    return cast<TestExternalElementsDataHandle>(handle).getKey().str();
+  }
+
+  FailureOr<AsmDialectResourceHandle>
+  declareResource(StringRef key) const final {
+    TestDialect *dialect = cast<TestDialect>(getDialect());
+    TestExternalElementsDataManager &mgr = dialect->getExternalDataManager();
+
+    // Resolve the reference by inserting a new entry into the manager.
+    auto it = mgr.insert(key).first;
+    return TestExternalElementsDataHandle(&*it, dialect);
+  }
+
+  LogicalResult parseResource(AsmParsedResourceEntry &entry) const final {
+    TestDialect *dialect = cast<TestDialect>(getDialect());
+    TestExternalElementsDataManager &mgr = dialect->getExternalDataManager();
+
+    // The resource entries are external constant data.
+    auto blobAllocFn = [](unsigned size, unsigned align) {
+      assert(align == alignof(uint64_t) && "unexpected data alignment");
+      return TestExternalElementsData::allocate(size / sizeof(uint64_t));
+    };
+    FailureOr<AsmResourceBlob> blob = entry.parseAsBlob(blobAllocFn);
+    if (failed(blob))
+      return failure();
+
+    mgr.setData(entry.getKey(), std::move(*blob));
+    return success();
+  }
+
+  void
+  buildResources(Operation *op,
+                 const SetVector<AsmDialectResourceHandle> &referencedResources,
+                 AsmResourceBuilder &provider) const final {
+    for (const AsmDialectResourceHandle &handle : referencedResources) {
+      const auto &testHandle = cast<TestExternalElementsDataHandle>(handle);
+      provider.buildBlob(testHandle.getKey(), testHandle.getData()->getData());
+    }
+  }
 };
 
 struct TestDialectFoldInterface : public DialectFoldInterface {
   using DialectFoldInterface::DialectFoldInterface;
 
   /// Registered hook to check if the given region, which is attached to an
   /// operation that is *not* isolated from above, should be used when
   /// materializing constants.

mlir/test/lib/Dialect/Test/TestDialect.td

::llvm::Optional<ParseOpHook>
       getParseOperationHook(::llvm::StringRef opName) const override;
     ::llvm::unique_function<void(::mlir::Operation *,
                                  ::mlir::OpAsmPrinter &printer)>
      getOperationPrinter(::mlir::Operation *op) const override;
 
      ::mlir::Type parseType(::mlir::DialectAsmParser &parser) const override;
      void printType(::mlir::Type type,
                     ::mlir::DialectAsmPrinter &printer) const override;
+
+    /// Returns the external elements data manager for this dialect.
+    TestExternalElementsDataManager &getExternalDataManager() {
+      return externalDataManager;
+    }
+
   private:
     // Storage for a custom fallback interface.
     void *fallbackEffectOpInterfaces;
 
     ::mlir::Type parseTestType(::mlir::AsmParser &parser,
                                ::llvm::SetVector<::mlir::Type> &stack) const;
     void printTestType(::mlir::Type type, ::mlir::AsmPrinter &printer,
                        ::llvm::SetVector<::mlir::Type> &stack) const;
+
+    /// An external data manager used to test external elements data.
+    TestExternalElementsDataManager externalDataManager;
   }];
 }
 
 #endif // TEST_DIALECT

mlir/unittests/CMakeLists.txt

 set_target_properties(MLIRUnitTests PROPERTIES FOLDER "MLIR Tests")
 
 function(add_mlir_unittest test_dirname)
   add_unittest(MLIRUnitTests ${test_dirname} ${ARGN})
 endfunction()
 
 add_subdirectory(Analysis)
 add_subdirectory(Conversion)
 add_subdirectory(Dialect)
 add_subdirectory(Interfaces)
 add_subdirectory(IR)
+add_subdirectory(Parser)
 add_subdirectory(Pass)
 add_subdirectory(Support)
 add_subdirectory(Rewrite)
 add_subdirectory(TableGen)
 add_subdirectory(Transforms)
 
 # The native target may not be enabled when cross compiling.
 if(TARGET ${LLVM_NATIVE_ARCH})
   add_subdirectory(ExecutionEngine)
 endif()

mlir/unittests/Parser/CMakeLists.txt

+add_mlir_unittest(MLIRParserTests
+  ResourceTest.cpp
+
+  DEPENDS
+  MLIRTestInterfaceIncGen
+)
+target_include_directories(MLIRParserTests PRIVATE "${MLIR_BINARY_DIR}/test/lib/Dialect/Test")
+
+target_link_libraries(MLIRParserTests PRIVATE
+  MLIRIR
+  MLIRParser
+  MLIRTestDialect
+)

mlir/unittests/Parser/ResourceTest.cpp

+//===- ResourceTest.cpp -----------------------------------------*- C++ -*-===//
+//
+// This file is licensed under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "../../test/lib/Dialect/Test/TestAttributes.h"
+#include "../../test/lib/Dialect/Test/TestDialect.h"
+#include "mlir/Parser/Parser.h"
+
+#include "gmock/gmock.h"
+
+using namespace mlir;
+
+namespace {
+TEST(MLIRParser, ResourceKeyConflict) {
+  std::string moduleStr = R"mlir(
+    "test.use1"() {attr = #test.e1di64_elements<blob1> : tensor<3xi64> } : () -> ()
+
+    {-#
+      dialect_resources: {
+        test: {
+          blob1: "0x08000000010000000000000002000000000000000300000000000000"
+        }
+      }
+    #-}
+  )mlir";
+  std::string moduleStr2 = R"mlir(
+    "test.use2"() {attr = #test.e1di64_elements<blob1> : tensor<3xi64> } : () -> ()
+
+    {-#
+      dialect_resources: {
+        test: {
+          blob1: "0x08000000040000000000000005000000000000000600000000000000"
+        }
+      }
+    #-}
+  )mlir";
+
+  MLIRContext context;
+  context.loadDialect<test::TestDialect>();
+
+  // Parse both modules into the same context so that we ensure the conflicting
+  // resources have been loaded.
+  OwningOpRef<ModuleOp> module1 =
+      parseSourceString<ModuleOp>(moduleStr, &context);
+  OwningOpRef<ModuleOp> module2 =
+      parseSourceString<ModuleOp>(moduleStr2, &context);
+  ASSERT_TRUE(module1 && module2);
+
+  // Merge the two modules so that we can test printing the remapped resources.
+  Block *block = module1->getBody();
+  block->getOperations().splice(block->end(),
+                                module2->getBody()->getOperations());
+
+  // Check that conflicting resources were remapped.
+  std::string outputStr;
+  {
+    llvm::raw_string_ostream os(outputStr);
+    module1->print(os);
+  }
+  StringRef output(outputStr);
+  EXPECT_TRUE(
+      output.contains("\"test.use1\"() {attr = #test.e1di64_elements<blob1>"));
+  EXPECT_TRUE(output.contains(
+      "blob1: \"0x08000000010000000000000002000000000000000300000000000000\""));
+  EXPECT_TRUE(output.contains(
+      "\"test.use2\"() {attr = #test.e1di64_elements<blob1_1>"));
+  EXPECT_TRUE(output.contains(
+      "blob1_1: "
+      "\"0x08000000040000000000000005000000000000000600000000000000\""));
+}
+} // namespace