Allow IndexType inside tensors.

[silvas]

Summary

It's common in many dialects to use tensors to themselves hold tensor shapes (for example, the shape is itself the result of some non-trivial calculation). Currently, such dialects have to use tensor<?xi64> or worse (like allowing either i32 or i64 tensors to represent shapes). tensor<?xindex> is the natural type to represent this, but is currently disallowed. This patch allows it.

[silvas]

Fix extraneous stuff in the diff

[mehdi_amini]

Are you aware of https://mlir.llvm.org/docs/Rationale/#index-type-disallowed-in-vectortensormemref-types ? (I'd expect this to be addressed in such patch)

[silvas]

Update Rationale.md

[silvas]

Address comments

[silvas]

update

[silvas]

Thanks @mehdi_amini, addressed.

[mehdi_amini]

Since we accept tensors of dialect specific type, I am not sure why we should forbid the index type, so LGTM.

[rriddle]

LGTM

[jpienaar]

Thanks!

[herhut]

Thanks for adding this. Looking forward to removing the index -> i64 -> index casting chains!

mlir/docs/Rationale.md

 
 For more context, block arguments were previously used in the Swift
 [SIL Intermediate Representation](https://github.com/apple/swift/blob/master/docs/SIL.rst),
 and described in
 [a talk on YouTube](https://www.youtube.com/watch?v=Ntj8ab-5cvE). The section of
 interest
 [starts here](https://www.google.com/url?q=https://youtu.be/Ntj8ab-5cvE?t%3D596&sa=D&ust=1529450150971000&usg=AFQjCNFQHEWL7m8q3eO-1DiKw9zqC2v24Q).
 
-### Index type disallowed in vector/tensor/memref types
+### Index type disallowed in vector/memref types
 
-Index types are not allowed as elements of `vector`, `tensor` or `memref` type.
-Index types are intended to be used for platform-specific "size" values and may
-appear in subscripts, sizes of aggregate types and affine expressions. They are
-also tightly coupled with `affine.apply` and affine.load/store operations;
-having `index` type is a necessary precondition of a value to be acceptable by
-these operations. While it may be useful to have `memref<?xindex>` to express
-indirect accesses, e.g. sparse matrix manipulations or lookup tables, it creates
-problems MLIR is not ready to address yet. MLIR needs to internally store
-constants of aggregate types and emit code operating on values of those types,
-which are subject to target-specific size and alignment constraints.  Since MLIR
-does not have a target description mechanism at the moment, it cannot reliably
-emit such code. Moreover, some platforms may not support vectors of type
-equivalent to `index`.
+Index types are not allowed as elements of `vector` and `memref` type. Index
+types are intended to be used for platform-specific "size" values and may appear
+in subscripts, sizes of aggregate types and affine expressions. They are also
+tightly coupled with `affine.apply` and affine.load/store operations; having
+`index` type is a necessary precondition of a value to be acceptable by these
+operations. While it may be useful to have `memref<?xindex>` to express indirect
+accesses, e.g. sparse matrix manipulations or lookup tables, it creates problems
+MLIR is not ready to address yet. MLIR needs to internally store constants of
+aggregate types and emit code operating on values of those types, which are
+subject to target-specific size and alignment constraints. Since MLIR does not
+have a target description mechanism at the moment, it cannot reliably emit such
+code. Moreover, some platforms may not support vectors of type equivalent to
+`index`.
 
 Indirect access use cases can be alternatively supported by providing and
 `index_cast` instruction that allows for conversion between `index` and
 fixed-width integer types, at the SSA value level. It has an additional benefit
 of supporting smaller integer types, e.g. `i8` or `i16`, for small indices
 instead of (presumably larger) `index` type.
 
+Index types are allowed as element types of `tensor` type. The `tensor` type
+specifically abstracts the target-specific aspects that intersect with the
+code-generation-related/lowering-related concerns explained above. In fact, the
+`tensor` type even allows dialect-specific types as element types.
+
 ### Bit width of a non-primitive types and `index` is undefined
 
 The bit width of a compound type is not defined by MLIR, it may be defined by a
 specific lowering pass. In MLIR, bit width is a property of certain primitive
 _type_, in particular integers and floats. It is equal to the number that
 appears in the type definition, e.g. the bit width of `i32` is `32`, so is the
 bit width of `f32`. The bit width is not _necessarily_ related to the amount of
 memory (in bytes) or the size of register (in bits) that is necessary to store

mlir/include/mlir/IR/StandardTypes.h

public:
 
   /// Return true if the specified element type is ok in a tensor.
   static bool isValidElementType(Type type) {
     // Note: Non standard/builtin types are allowed to exist within tensor
     // types. Dialects are expected to verify that tensor types have a valid
     // element type within that dialect.
     return type.isa<ComplexType>() || type.isa<FloatType>() ||
            type.isa<IntegerType>() || type.isa<OpaqueType>() ||
-           type.isa<VectorType>() ||
+           type.isa<VectorType>() || type.isa<IndexType>() ||
            (type.getKind() > Type::Kind::LAST_STANDARD_TYPE);
   }
 
   /// Methods for support type inquiry through isa, cast, and dyn_cast.
   static bool classof(Type type) {
     return type.getKind() == StandardTypes::RankedTensor ||
            type.getKind() == StandardTypes::UnrankedTensor;
   }

mlir/test/IR/invalid.mlir

 
 func @indexvector(vector<4 x index>) -> () // expected-error {{vector elements must be int or float type}}
 
 // -----
 
 func @indexmemref(memref<? x index>) -> () // expected-error {{invalid memref element type}}
 
 // -----
-
-func @indextensor(tensor<4 x index>) -> () // expected-error {{invalid tensor element type}}
-
-// -----
 // Test no map in memref type.
 func @memrefs(memref<2x4xi8, >) // expected-error {{expected list element}}
 
 // -----
 // Test non-existent map in memref type.
 func @memrefs(memref<2x4xi8, #map7>) // expected-error {{undefined symbol alias id 'map7'}}
 
 // -----