Relax the constraint for MemRef type to allow any non-builtin type as element type.
Quant element type for MemRef is the feature request. However, I think it may be inappropriate to explicitly only allow QuantType(it needs to add QuantType dependency to StandardType). So I just sync the constraint with TensorType.
Similar changes have been proposed repeatedly. I am yet to see an RFC (which we normally require for standard/built-in changes) that clarifies the semantics of such memrefs. In particular, I am interested in the allocation/deallocation of such memrefs and any operations that require knowing the size of the elemental type. For example, how does one allocate memref<42 x opaque>? In general, there should be a clear specification of what dialects should check for in order for the type to be "valid" in a memref.
I would prefer to see more discussion around such an interface first. This interface as proposed seems very simplistic, and not something that would be able to support LLVM for example.
Along the same lines as the above, I'm not sure bytes are a good metric to use when considering size given that bytes can vary between platforms. I would prefer to have an explicit proposal for this interface that has its own motivations other than for supporting non-builtin types in Memref.