The idea here of preventing invalid ops makes sense to me, but @herhut probably is more knowledgeable about this code for a detailed review.

Nice :) This looks pretty straightforward, but given how core this is, wait for another approver or two.

Yay!!

In D111620#3104504, @rriddle wrote:

In D111620#3104485, @benvanik wrote:

This change broke 1->N type conversion (as noted in a FIXME in here). This is really unfortunate and there doesn't seem to be any workaround now that the materialization is deferred and the casts are inserted in a way that doesn't allow for patterns to match them. Is this something that can be rolled back or fixed soon? 1->N type conversion during dialect conversion did work before this change so this is a regression.

The FIXME is more of further elaborating on the existing state, it was not some new problem added by this patch. Can you file a bug report with what your exact situation is?

This looks good to me. @rriddle @dexonsmith @dblaikie for thoughts.

In D112736#3094966, @ezhulenev wrote:

Basically add a pass something like X86vectorMathApproximationsPass?

1. I would just like Math dialect transforms to not depend on target-specific dialects. So concretely just RsqrtOpApproximation pattern would move to x86vector/Transform/, since it is the target specific one.
2. I agree, removing Polynomial would be good.

Can we move this RsqrtOp lowering to the x86vector dialect? It seems really out of place in the math dialect, which should remain target agnostic. (also, since this approximation is newton-raphson based instead of polynomial based, it seems out of place in PolynomialApproximation.cpp as well)

This doesn't seem to be the right pass to do target-specific lowerings. Can you add math.rsqrt and move the vector unrolling to a generic pass in the vector dialect? Then this pass wouldn't depend on target-details.

Glad to see this happening!

nice!

LGTM. Probably wait for another pair of eyes to LGTM it. @ezhulenev perhaps?

Thanks!!!

I think it's fine for const_shape to always be static and require a tensor.cast to erase the static shape to dynamic.

Thanks!

+1 to Jacques suggestion to listing some prior art here. In the project ideally or in the industry as well would be good.

In D110425#3033028, @ftynse wrote:

What would be a more correct phrasing for the semantics description in your opinion? We want the op to always result in a new value.

I don't really understand the argument about the operation having to be always folded away. This would effectively render impossible any clone or copy operation. Yet, there is a TensorCloneOp in IREE if I remember correctly and it isn't an always-folded noop. So there is clearly precedent and a use case for operations defining new values with same "content" as the existing values.

The additional attribute may indeed be interpreted as preventing some canonicalization wrt the semantics of the original op (or its version with the attribute unset). However, I argue that we just choose to define different canonical forms for the op depending on it having the attribute. Referring to the post our doc on canonicalization quotes in the opening paragraph - https://sunfishcode.github.io/blog/2018/10/22/Canonicalization.html - that describes how canonicalization and redundancy elimination get ambiguous: "<...> ultimately, in its purest form, canonicalization just focuses on removing unnecessary variation so that subsequent optimizations can be simpler", this change is exactly about removing, or rather shifting, variation to simplify subsequent optimization. There exist subsequent optimizations such as hoisting that are rendered significantly simpler by always having a fresh value that is always defined by pad_tensor.

@herhut - thoughts on keeping scalars (i1) as disallowed from Same*Shape traits?

Personally I think we should deprecate this behavior. In my mind, an i1 doesn't have a shape at all. So it doesn't make sense to say it has the same shape as anything else. Is there a specific use case for this behavior (IIRC this might have fallen out of an implementation detail of an earlier implementation of the trait).

Specifically, it is impossible for these two functions to have different semantics (as long as linalg.pad_tensor is marked NoSideEffect) -- both return a value that is identical to their argument, and it is always legal to fold @with_pad to @without_pad per tensor value semantics:

In D110425#3029615, @nicolasvasilache wrote:

In D110425#3028927, @silvas wrote:

Sorry I didn't see this earlier, but I don't think this direction makes sense. This op operates on tensors, which are value semantic, so terminology like "guaranteed to create a new tensor suitable for packing" simply don't make sense.

I fail to see what doesn't make sense? Packing is done on tensor semantics too: this can be also called "tensor tiling" by opposition to "op tiling".

Having an attribute that prevents folding from happening gives me bad memories of e.g. tf.Identity being marked as having side effects to prevent folding. I would rather not go down that route.

I do not see the relation between the 2.
Marking tf.Identity as having side effects is clearly a hack misusing side effects for other purposes: the op still has no side effects.
Here, the semantics of the op are : it is illegitimate to fold this op. There is no discussion about side effects or any other interface.

Sorry I didn't see this earlier, but I don't think this direction makes sense. This op operates on tensors, which are value semantic, so terminology like "guaranteed to create a new tensor suitable for packing" simply don't make sense.

address

did a pass. looks good to me. probably want to wait for a handful of LGTM's given the far-reaching consequences of this change. Ideally both River and Chris could weigh in.

Nice!

Awesome! Thanks!!!

Thanks Alex!

Thanks Alex!

Can you please revert this commit and the previous one I objected to? I think they are fundamentally going in the wrong direction.

Can you please add the integration test cases I suggested in the other patch? I'm still pretty sure this will miscompile. Fundamentally you are writing this as an "in-place" bufferization and that cannot be done with the current framework because it involves non-local reasoning.

It feels like this bufferization pattern would miscompile if we were to write integration tests analogous to these:

Beautiful :)

great work yi!

Thanks!

I don't think we need an integration test here. A regular IR test would be enough to exercise it? (just snapshot the IR from the itnegration test before linalg-bufferize)

Do you know why previous tests didn't need this? Should we add a new test?

Please add lowering to LLVM for this too (doesn't have to be this patch precisely, but a fast follow-on please).

Will let @sjarus review this as the detailed shape transfer function review is best done by someone deeply aware of the spec.

reword

LGTM. Wait for @springerm / @nicolasvasilache approval too.

Oh, sorry, I thought you meant which woudl use it. It would be nice if it was in a generic helper file. You can probably put it next to PadTensorOpTransformationPattern for consistency.

In D105293#2855869, @cathyzhyi wrote:

@silvas @nicolasvasilache @springerm Any suggestion on which pass to put this pattern in? There is a LinalgGeneralizationPass which currently only applies to patterns lowering to Linalg.generic.

In D105293#2854047, @nicolasvasilache wrote:

Note that I asked for a similar rewrite in https://reviews.llvm.org/D102804 but it was eventually not done.
I am wondering whether the pattern you add here should be more generally be the rewrite on tensors I mentioned and then just let existing bufferization kick in?

Can you add an integration test analogous to mlir/test/Integration/Dialect/Linalg/CPU/test-subtensor-insert.mlir ?

Nice cleanup! Thanks!

(before submitting, please wait for LGTM from at least one other person since this change covers a lot of stuff)

I would personally like to see this as a nice clean module pass, and then make targeted changes informed by concrete use cases (actual need for more parallelism, need to be applied locally within a particular kind of enclosing op, etc.). Really for more advanced use cases this is going to be accessed via a direct C++ API, so the pass structure we choose here should be clean and simple and have nice invariants that make it easy to reason about extracting a targeted C++ API for new use cases. For example, the C++ API could take a std::function<Value(ConstantOp op)> for creating memref values from constants, which for the module pass would use a GlobalCreator, but for other users would be implemented otherwise. I don't want to promote uses of ComprehensiveBufferize with this weird non-support for constants.

Can you just delete the function pass and have the module pass be the only registered pass?

Thanks you Matthias for doing this! I apologize for leaving this in the sorry state that it was left in.

@HassanElDesouky sorry, I'm not that familiar with this code.

This makes me so sad that the memref.dim infection is forcing this to go in memref/transforms :'(

Hi @dfki-jugr, can we please split memref.dim into tensor.dim (and std.rank into tensor.rank if you have time). I've now independently heard complaints about this from 5+ people across many different teams I interact with.