Could you please give a brief outline of the approach?

In other instructions, we seem to just duplicate the handling in findBaseDefiningValueOfVector and findBaseDefiningValue. Doing so for freeze would be more consistent with the existing code.

The optimizer can insert new atomic memcpy/memmove calls, e.g loop-idiom-recognize would do that. Currently, when loop-idiom-recognize inserts such a call it doesn't tag it with the element type, which makes it an incorrect transform for us.

Copying arrays of pointers in environments with moving garbage collector is different from copying of 'primitive' types of the same types.
Well, I understand that upstream may not take it as a 'justification'.

Do you mind if I put this change under an on by default cl::opt flag? This way we can temporarily turn it off downstream.

This change broke the ability to discard checks in some cases. Consider this example:

define i1 @test1(i32 %a, i32* %b_ptr) {
  %b = load i32, i32* %b_ptr, align 8, !range !{i32 0, i32 2147483646}
  %c1 = icmp slt i32 %a, %b
  br i1 %c1, label %exit, label %cont.1

I agree with Philip, widening transformation doesn't rely on deoptimizations.

Add a simple test case using -inline-cost-full cl::opt.

I also notice that the new memcmp intrinsic returns i1, which is different from libc memcmp. libc memcmp returns and int indicating which of the sides was greater. Looks like, the new intrinsic matches with bcmp semantic.

Since this is a change to LangRef, please, post the proposal to llvm-dev.

There is a target-specific hook to emit code for regular (non-atomic) memcpys: EmitTargetCodeForMemcpy. Maybe we should just implement a similar hook for element-atomic copy?

I think the name of the pass is misleading. What you are doing here is you provide an inlined lowering for a memory builtin. Your current implementation has a limitation such that it only inlines gc-leaf element atomic memcpys. I don't think the fact that it is gc-leaf is critical here. This transform can be extended to handle non-GC leaf operations. It can also be extended to handle non-atomic operations.

Looks good.

FYI, I tried to take this patch for performance verification, but it crashed in smoke testing. I haven't yet looked why.

I looked at the GC part and it looks good to me.

Can we split restructuring and generalization (possibly adding a test for the generalization part)? It's hard to assess correctness when you combine the two in one change.

Accepting this change to unblock the progress.

Address review comments.

Still digging through the main logic. In the meantime see some minor comments inline.

Added a note into the doc that a GC parseable copy operation is not required to take a safepoint.

Currently, if we have a loop with a safepoint poll it is not converted into a memcpy/memmove. This is because the safepoint has read semantic and prevents LoopIdiomRecognize from performing the transform. In theory we can have a transform which recognizes loops with safepoints and converts them to non-leaf memcpy/memmove. It will be up to this new transform to figure out the legality and interactions with the runtime requirements.

This was a draft. Abandoned in favor of D88861.

@mtrofin, I think it makes sense to have these tests available in all configurations. We have a bunch of printer passes which are not conditioned by NDEBUG.

Look good.

In D81026#2069624, @mtrofin wrote:

That sounds fine, my point is that having some run on benchmarks to quantify the benefit would help understand the value of this particular omission from the cost analysis (or at minimum show there's no regression, or understand where the regressions may be). It could shed important insights into what matters for the cost analysis, basically.

Can you please elaborate what is the issue you are solving? May be give an example?

Demonstrating an improvement accuracy due to PHI translation turned out to be a bit tricky.

Finished by @ebrevnov as D65276.

I reworked the patch using PHITransAddr. Now instead of tracking the backedge flag I keep track of the address to check and translate the address when needed.

Couple of nit comments inlined, LGTM once addressed.

In D74346#1873034, @davidxl wrote:

what is the use case of these options? Do hidden nodes create too much noise?

LGTM with the renaming addressed.

Ping?

• Comment cleanup was landed separately
• Fixed the issue mentioned in https://reviews.llvm.org/D68006#inline-616849
• Added more tests

Yeah, this limitation seems arbitrary and overly conservative.

Can we handle the case with inbounds offsets only first and extend with non-inbounds support in a follow up change? This way in the first change you'll not need to change IR interfaces at all.

LGTM. Maybe add a test when the WC has one use but the branch condition is used in multiple branches.