[SROA] Improve SROA to prevent generating redundant coalescing operations.

[mingmingl]

Summary

Before this change, coalesce instructions are difficult to clean up, and prevent the tail call generation (see test case test_struct_of_two_int in llvm/test/Transforms/SROA/alloca-struct.ll)

Change SROA pass so a structure type won't be scalarized to smaller allocations (i.e., remain unchanged) in the following conditions

1. If it doesn't have scalar access

or

2. If the scalar stores will essentially make the struct a constant (i.e., all individual fields are constant values).

This change adds additional analysis information about usages, and mark all slices of an alloca as unsplittable if all conditions are true:

1. The alloca doesn't have a) scalar load b) non constant scalar store c) uncovered access type (e.g., memcpy, memset, etc)
2. In each basic block, the constant stores cover all bytes (i.e., the allocated type could be regarded as a constant in that basic block)

The change preserves original slice-split logic if there are scalar access, or if analysis information isn't sufficient to prevent scalarization.

[davidxl]

I suggest simplify the title to : "Improve SROA to prevent generating redundant coalescing operations'.

[mingmingl]

In D114832#3162966, @davidxl wrote:

I suggest simplify the title to : "Improve SROA to prevent generating redundant coalescing operations'.

Done, thanks for the suggestion!

[nikic]

llvm/test/Transforms/SROA/alloca-struct.ll

82

Please use update_test_checks.py.

[lebedev.ri]

That's a lot of complexity. It sounds like we are missing some cleanup transforms in some other passes?
Can you please post a standalone example of the "bad" codegen (via a godbolt link) that you are trying to avoid?

[mingmingl]

In D114832#3163500, @lebedev.ri wrote:

That's a lot of complexity. It sounds like we are missing some cleanup transforms in some other passes?
Can you please post a standalone example of the "bad" codegen (via a godbolt link) that you are trying to avoid?

To compare the LLVM and GCC codegen, tail call is generated in GCC but not LLVM.

• (LLVM) https://godbolt.org/z/qTY6MfMGa
• (GCC) https://godbolt.org/z/srh5fG6nW

test_struct_of_two_int (from llvm/test/Transforms/SROA/alloca-struct.ll) shows the diff in IR level before and after this change, and run clang with this patch (clang -S -O3 file.cpp) would generate a tail call.

[mingmingl]

In D114832#3164711, @luna wrote:

In D114832#3163500, @lebedev.ri wrote:

That's a lot of complexity. It sounds like we are missing some cleanup transforms in some other passes?
Can you please post a standalone example of the "bad" codegen (via a godbolt link) that you are trying to avoid?

To compare the LLVM and GCC codegen, tail call is generated in GCC but not LLVM.

• (LLVM) https://godbolt.org/z/qTY6MfMGa
• (GCC) https://godbolt.org/z/srh5fG6nW

test_struct_of_two_int (from llvm/test/Transforms/SROA/alloca-struct.ll) shows the diff in IR level before and after this change, and run clang with this patch (clang -S -O3 file.cpp) would generate a tail call.

I dig a little bit on how GCC SROA works, and the findings are

1. this [1] writeup also mentions two RPs which avoid creating unnecessary or harmfully excessive statements when aggregates are used as a whole. The relevant PR are PR 40744 and PR 44423, mentioned in xi) of 2nd section.
2. The implementation (https://github.com/gcc-mirror/gcc/blob/16e2427f50c208dfe07d07f18009969502c25dc8/gcc/tree-sra.c#L2435-L2472) has analysis of scalar/whole usage

[1] https://gcc.gnu.org/wiki/summit2010?action=AttachFile&do=get&target=jambor.pdf

[mingmingl]

llvm/test/Transforms/SROA/alloca-struct.ll

82

Thanks for the suggestions!

Using update_test_checks.py might make the test case a little fragile. This is a preview of the result (https://gist.github.com/minglotus-6/a05ac0300d8ea8204ec9eed409951b25)

To improve the current test style, maybe I can enrich it a little bit but still scrub updater output that look strict?

[davidxl]

The key is an aggregate is written field by field (piecemeal), but not read as such (e.g, as a whole), so splitting messes up the IR. For this reason, the pattern should probably be relaxed more (i.e. not requiring constant values in the stores). Example:

struct RetVal {

int x;
int y;

};
typedef RetVal (*Parser)();
RetVal Parse(bool test, Parser p, int v) {

if (test) return {v, v};
return p();

}

[aeubanks]

I'm not familiar with the term "coalesce instructions", and it doesn't appear anywhere in SROA.cpp, could you explain that in the description?

llvm/test/Transforms/SROA/alloca-struct.ll

82

it's very hard to tell what changed without seeing every instruction in the function, so I'd also recommend update_test_checks.py. yes it's fragile, but that's kind of the point, to be able to see changes in IR

[mingmingl]

Use update_test_checks.py to update test/Transforms/SROA/alloca-struct.ll.

1. In this revision, the opt used is built with this patch.
2. Test case test_struct_of_int_char and test_one_field_has_runtime_value should remain unchanged.
3. Diff of test case test_struct_of_two_int is shown in https://gist.github.com/minglotus-6/a05ac0300d8ea8204ec9eed409951b25/revisions

[mingmingl]

Clean up CHECK that were manually added previously, but not needed when we use update_test_checks.py.

[mingmingl]

In D114832#3168066, @aeubanks wrote:

I'm not familiar with the term "coalesce instructions", and it doesn't appear anywhere in SROA.cpp, could you explain that in the description?

In this example (https://godbolt.org/z/fhWT48db7), coalesce instruction is %10 = or i64 %9, %8, !dbg !27 from the following basic block.

7:                                                ; preds = %2, %3
  %8 = phi i64 [ %6, %3 ], [ 0, %2 ]
  %9 = phi i64 [ %5, %3 ], [ 0, %2 ]
  %10 = or i64 %9, %8, !dbg !27

Another similar example is https://godbolt.org/z/sna4TPPTY, coalesce instructions are the following three instructions

%17 = or i64 %13, %12, !dbg !30
%18 = or i64 %17, %14, !dbg !30
%19 = or i64 %18, %15, !dbg !30

in the last basic block, which are used to or scalar values together.

[mingmingl]

llvm/test/Transforms/SROA/alloca-struct.ll

82

Done.

[aeubanks]

llvm/test/Transforms/SROA/alloca-struct.ll

82

can you first run update_test_checks.py on this test, submit that, then rebase so we can see the effects of this change on the IR?

[mingmingl]

llvm/test/Transforms/SROA/alloca-struct.ll

82

Sure, sent out https://reviews.llvm.org/D115006.

[mingmingl]

Update revision after git rebase

[mingmingl]

A gentle ping. Please let me know if there are any feedback, thanks!

[lebedev.ri]

Can the issues instead be viewed as missed transformations that should be implemented instead of trying to prevent creating those "bad" patterns?

[mingmingl]

In D114832#3176957, @lebedev.ri wrote:

Can the issues instead be viewed as missed transformations that should be implemented instead of trying to prevent creating those "bad" patterns?

For my understanding of 'missed transformations', would this transformation takes the IR after SROA pass (pasted in https://gist.github.com/minglotus-6/cae98ad58f7ae6cac8dd848e75c2bde9) as input,
doing something like instruction combine or clean ups, with the desired output the same as this patch did in the SROA pass?

I have also thought of the approach of a separate transformation in the beginning, but decided to implement this first (easier to prevent split in the first place than analyzing results after split and combine them together)

Now upon this suggestion, I think it twice about how to do it in another pass (maybe right after SROA); some thoughts:

1. The basic essence of the current patch is to analyze the use pattern of an allocation, and the use pattern information (offsets of fields) is still in the IR after SROA pass. From this perspective it might be possible to do it in a separate pass.
2. However, a pattern matcher (to analyze offsets after split) might be fragile, and won't continue to work if the specific instructions in IR changes (but the semantics of constant stores remains).

I'm wondering if there happens to be an example of existing passes that might be inspiring for a use case of joining instructions like this. Also would like to hear concerns of the current approach so maybe I can iterate from there.

[davidxl]

The bad code pattern generated is probably not general enough to be useful to introduce a cleanup pass or to enhance existing pass to do so -- it will probably just shift the complexity from one pass to another. Fixing this at the source (SROA) is reasonable (unlike other canonicalization pass, this code pattern from SROA actually makes IR much worse)

[lebedev.ri]

In D114832#3177534, @davidxl wrote:

The bad code pattern generated is probably not general enough to be useful to introduce a cleanup pass or to enhance existing pass to do so -- it will probably just shift the complexity from one pass to another. Fixing this at the source (SROA) is reasonable (unlike other canonicalization pass, this code pattern from SROA actually makes IR much worse)

Who defines what is/isn't reasonable? :)

As i see it, the input IR may or may not already have such bad patterns that are similar to the bad patterns you are trying to avoid producing here.
Trying to avoid producing it has compile time cost: https://llvm-compile-time-tracker.com/compare.php?from=0850655da69a700b7def4fe8d9a44d1c8d55877c&to=8a4304c8f4253fb1944e5e5988a24285a14181c4&stat=instructions
So, you've paid for avoiding producing more bad patterns, but you are still left with the ones that were already there.

Alternatively, instead of paying for trying not to introduce bad patterns, you could pay for trying to improve all of the patterns afterwards, regardless of their source.
Then, you still end up paying, but end up with not just the SROA patterns improved.

This is a very standard logic behind IR changes - if something doesn't understand the pattern, generally don't try to workaround it elsewhere, just fix the missing piece.

[davidxl]

In D114832#3178967, @lebedev.ri wrote:

In D114832#3177534, @davidxl wrote:

The bad code pattern generated is probably not general enough to be useful to introduce a cleanup pass or to enhance existing pass to do so -- it will probably just shift the complexity from one pass to another. Fixing this at the source (SROA) is reasonable (unlike other canonicalization pass, this code pattern from SROA actually makes IR much worse)

Who defines what is/isn't reasonable? :)

As i see it, the input IR may or may not already have such bad patterns that are similar to the bad patterns you are trying to avoid producing here.
Trying to avoid producing it has compile time cost: https://llvm-compile-time-tracker.com/compare.php?from=0850655da69a700b7def4fe8d9a44d1c8d55877c&to=8a4304c8f4253fb1944e5e5988a24285a14181c4&stat=instructions
So, you've paid for avoiding producing more bad patterns, but you are still left with the ones that were already there.

Alternatively, instead of paying for trying not to introduce bad patterns, you could pay for trying to improve all of the patterns afterwards, regardless of their source.
Then, you still end up paying, but end up with not just the SROA patterns improved.

This is a very standard logic behind IR changes - if something doesn't understand the pattern, generally don't try to workaround it elsewhere, just fix the missing piece.

I agree with most of what you said as a general principle, while analysis should be done case by case :)

The assumption is that the producer (SROA) is pretty much the only creator of the pattern (excluding manually written IR), thus cleaning it up downstream does not really help in sharing. The compile time cost is also shifted. Imagine another hypothetical scenario: if we there are N different unique bad patterns that can be handled by one single analysis pass in the source pass, we may need to have N different downstream pass changes to handle them resulting in more waste.

Having said that, suggestions on which downstream pass to handle this is useful. IIRC Mingming considered CodegenPrepare where some taildup can be done to handle tailcalls ...

[mingmingl]

In D114832#3179810, @davidxl wrote:

In D114832#3178967, @lebedev.ri wrote:

In D114832#3177534, @davidxl wrote:

The bad code pattern generated is probably not general enough to be useful to introduce a cleanup pass or to enhance existing pass to do so -- it will probably just shift the complexity from one pass to another. Fixing this at the source (SROA) is reasonable (unlike other canonicalization pass, this code pattern from SROA actually makes IR much worse)

Who defines what is/isn't reasonable? :)

As i see it, the input IR may or may not already have such bad patterns that are similar to the bad patterns you are trying to avoid producing here.
Trying to avoid producing it has compile time cost: https://llvm-compile-time-tracker.com/compare.php?from=0850655da69a700b7def4fe8d9a44d1c8d55877c&to=8a4304c8f4253fb1944e5e5988a24285a14181c4&stat=instructions
So, you've paid for avoiding producing more bad patterns, but you are still left with the ones that were already there.

Alternatively, instead of paying for trying not to introduce bad patterns, you could pay for trying to improve all of the patterns afterwards, regardless of their source.
Then, you still end up paying, but end up with not just the SROA patterns improved.

This is a very standard logic behind IR changes - if something doesn't understand the pattern, generally don't try to workaround it elsewhere, just fix the missing piece.

I agree with most of what you said as a general principle, while analysis should be done case by case :)

The assumption is that the producer (SROA) is pretty much the only creator of the pattern (excluding manually written IR), thus cleaning it up downstream does not really help in sharing. The compile time cost is also shifted. Imagine another hypothetical scenario: if we there are N different unique bad patterns that can be handled by one single analysis pass in the source pass, we may need to have N different downstream pass changes to handle them resulting in more waste.

Having said that, suggestions on which downstream pass to handle this is useful. IIRC Mingming considered CodegenPrepare where some taildup can be done to handle tailcalls ...

thanks for the discussions and insight everyone!

The compile time link is helpful. Is there a pointer or rule-of-thumb to evaluate cost vs benefit?

• Ask since the current analysis is inlined in an existing instruction walker. Adding a separate transformation (as opposed to choose a good existing one to piggyback on) would likely increase the cost (with the same effect).

It seems the other option to consider is have a separate pass to transform bad patterns, or reuse an existing pass like CodeGenPrepare.

It'd be helpful to know a pass that I should probably look more into, to achieve the original optimization goal here.

[mingmingl]

In D114832#3179890, @luna wrote:

In D114832#3179810, @davidxl wrote:

In D114832#3178967, @lebedev.ri wrote:

In D114832#3177534, @davidxl wrote:

The bad code pattern generated is probably not general enough to be useful to introduce a cleanup pass or to enhance existing pass to do so -- it will probably just shift the complexity from one pass to another. Fixing this at the source (SROA) is reasonable (unlike other canonicalization pass, this code pattern from SROA actually makes IR much worse)

Who defines what is/isn't reasonable? :)

As i see it, the input IR may or may not already have such bad patterns that are similar to the bad patterns you are trying to avoid producing here.
Trying to avoid producing it has compile time cost: https://llvm-compile-time-tracker.com/compare.php?from=0850655da69a700b7def4fe8d9a44d1c8d55877c&to=8a4304c8f4253fb1944e5e5988a24285a14181c4&stat=instructions
So, you've paid for avoiding producing more bad patterns, but you are still left with the ones that were already there.

Alternatively, instead of paying for trying not to introduce bad patterns, you could pay for trying to improve all of the patterns afterwards, regardless of their source.
Then, you still end up paying, but end up with not just the SROA patterns improved.

This is a very standard logic behind IR changes - if something doesn't understand the pattern, generally don't try to workaround it elsewhere, just fix the missing piece.

I agree with most of what you said as a general principle, while analysis should be done case by case :)

The assumption is that the producer (SROA) is pretty much the only creator of the pattern (excluding manually written IR), thus cleaning it up downstream does not really help in sharing. The compile time cost is also shifted. Imagine another hypothetical scenario: if we there are N different unique bad patterns that can be handled by one single analysis pass in the source pass, we may need to have N different downstream pass changes to handle them resulting in more waste.

Having said that, suggestions on which downstream pass to handle this is useful. IIRC Mingming considered CodegenPrepare where some taildup can be done to handle tailcalls ...

thanks for the discussions and insight everyone!

The compile time link is helpful. Is there a pointer or rule-of-thumb to evaluate cost vs benefit?

• Ask since the current analysis is inlined in an existing instruction walker. Adding a separate transformation (as opposed to choose a good existing one to piggyback on) would likely increase the cost (with the same effect).

It seems the other option to consider is have a separate pass to transform bad patterns, or reuse an existing pass like CodeGenPrepare.

It'd be helpful to know a pass that I should probably look more into, to achieve the original optimization goal here.

Driven by the compile-time feedback, another thing for me to consider, (probably after a preliminary convergence on the pass to add analysis info or do the transform), is to prune the existing solution so it's faster on the benchmark.

[mingmingl]

In D114832#3179919, @luna wrote:

In D114832#3179890, @luna wrote:

In D114832#3179810, @davidxl wrote:

In D114832#3178967, @lebedev.ri wrote:

In D114832#3177534, @davidxl wrote:

The bad code pattern generated is probably not general enough to be useful to introduce a cleanup pass or to enhance existing pass to do so -- it will probably just shift the complexity from one pass to another. Fixing this at the source (SROA) is reasonable (unlike other canonicalization pass, this code pattern from SROA actually makes IR much worse)

Who defines what is/isn't reasonable? :)

As i see it, the input IR may or may not already have such bad patterns that are similar to the bad patterns you are trying to avoid producing here.
Trying to avoid producing it has compile time cost: https://llvm-compile-time-tracker.com/compare.php?from=0850655da69a700b7def4fe8d9a44d1c8d55877c&to=8a4304c8f4253fb1944e5e5988a24285a14181c4&stat=instructions
So, you've paid for avoiding producing more bad patterns, but you are still left with the ones that were already there.

Alternatively, instead of paying for trying not to introduce bad patterns, you could pay for trying to improve all of the patterns afterwards, regardless of their source.
Then, you still end up paying, but end up with not just the SROA patterns improved.

This is a very standard logic behind IR changes - if something doesn't understand the pattern, generally don't try to workaround it elsewhere, just fix the missing piece.

I agree with most of what you said as a general principle, while analysis should be done case by case :)

The assumption is that the producer (SROA) is pretty much the only creator of the pattern (excluding manually written IR), thus cleaning it up downstream does not really help in sharing. The compile time cost is also shifted. Imagine another hypothetical scenario: if we there are N different unique bad patterns that can be handled by one single analysis pass in the source pass, we may need to have N different downstream pass changes to handle them resulting in more waste.

Having said that, suggestions on which downstream pass to handle this is useful. IIRC Mingming considered CodegenPrepare where some taildup can be done to handle tailcalls ...

thanks for the discussions and insight everyone!

The compile time link is helpful. Is there a pointer or rule-of-thumb to evaluate cost vs benefit?

• Ask since the current analysis is inlined in an existing instruction walker. Adding a separate transformation (as opposed to choose a good existing one to piggyback on) would likely increase the cost (with the same effect).

It seems the other option to consider is have a separate pass to transform bad patterns, or reuse an existing pass like CodeGenPrepare.

It'd be helpful to know a pass that I should probably look more into, to achieve the original optimization goal here.

Driven by the compile-time feedback, another thing for me to consider, (probably after a preliminary convergence on the pass to add analysis info or do the transform), is to prune the existing solution so it's faster on the benchmark.

I did a quick experiment in https://godbolt.org/z/EWWaMedsx, which basically transforms {a series of PHI, one OR} into {a series of OR, one PHI}. This approach is not implemented in the first place, because result after transformation (i.e. {a series of OR, one PHI}) might be more machine instructions, and more expensive.

Some more context before starting to implement:

1. For llvm/test/Transforms/SROA/alloca-struct.ll, original cpp and assembly code is as shown in https://godbolt.org/z/qTY6MfMGa (mentioned above)
2. IR before the first SROA pass is https://gist.github.com/minglotus-6/24bccb53d5e101cb20a7118f5e9f389f IR after SROA is like https://gist.github.com/minglotus-6/1e6b93d52687e7fbbb43942563134117
   • In particular, the alloca instruction is allocating a struct before SROA pass, and scalarized in SROA pass, and lead to the verbose instructions (lshr, trunc, etc).

Based on this, the basic essence is not to create scalarization in the first place of the usage is whole. The idea of skip scalarization also exists in GCC (e.g., The relevant PR are PR 40744 and PR 44423, mentioned in xi) of 2nd section of https://gcc.gnu.org/wiki/summit2010?action=AttachFile&do=get&target=jambor.pdf).

The IR of test12 of llvm/test/Transforms/SROA/basictest-opaque-ptrs.ll is also more succinct (I didn't look into the codegen result though)

[aeubanks]

after running it through -O2:

define i64 @test_struct_of_two_int(i1 zeroext %test, i64 ()* nocapture %p) local_unnamed_addr {
entry:
  br i1 %test, label %return, label %if.end

if.end:                                           ; preds = %entry
  %call = tail call i64 %p()
  %retval.sroa.3.0.extract.shift = and i64 %call, -4294967296
  %phi.cast = and i64 %call, 4294967295
  br label %return

return:                                           ; preds = %entry, %if.end
  %retval.sroa.3.0 = phi i64 [ %retval.sroa.3.0.extract.shift, %if.end ], [ 0, %entry ]
  %retval.sroa.0.0 = phi i64 [ %phi.cast, %if.end ], [ 0, %entry ]
  %retval.sroa.0.0.insert.insert = or i64 %retval.sroa.0.0, %retval.sroa.3.0
  ret i64 %retval.sroa.0.0.insert.insert
}

yeah I'm not sure if this is something that only happens with SROA, but it does look very weird

perhaps trying to implement something like

b:
 phi1 = phi [b1, v1] [b2, v2]
 phi2 = phi [b1, v3] [b2, v4]
 r = op phi1, phi2

into

b1:
 n1 = op v1, v2
b2:
 n2 = op v3, v4
b:
 r = phi [b1, n1] [b2, n2]

and maybe only if at least one of the ops simplifies away to prevent increasing the number of op instructions

when test_struct_of_two_int is properly fixed we should add a phase ordering test for it to make sure it fully optimizes correctly if we do go down this route

[Carrot]

Driven by the compile-time feedback, another thing for me to consider, (probably after a preliminary convergence on the pass to add analysis info or do the transform), is to prune the existing solution so it's faster on the benchmark.

Your current method is

1 collect all accesses to an alloca object, mark the access type
2 check the aggregated access types, if there is only <store constant> type, mark the object as unsplittable.

I think in step1, if the access type is not <store constant>, you can bail out early, then for most splittable objects the check can be finished quickly.

[mingmingl]

thanks for the feedback!

In D114832#3181314, @aeubanks wrote:

after running it through -O2:

define i64 @test_struct_of_two_int(i1 zeroext %test, i64 ()* nocapture %p) local_unnamed_addr {
entry:
  br i1 %test, label %return, label %if.end

if.end:                                           ; preds = %entry
  %call = tail call i64 %p()
  %retval.sroa.3.0.extract.shift = and i64 %call, -4294967296
  %phi.cast = and i64 %call, 4294967295
  br label %return

return:                                           ; preds = %entry, %if.end
  %retval.sroa.3.0 = phi i64 [ %retval.sroa.3.0.extract.shift, %if.end ], [ 0, %entry ]
  %retval.sroa.0.0 = phi i64 [ %phi.cast, %if.end ], [ 0, %entry ]
  %retval.sroa.0.0.insert.insert = or i64 %retval.sroa.0.0, %retval.sroa.3.0
  ret i64 %retval.sroa.0.0.insert.insert
}

yeah I'm not sure if this is something that only happens with SROA, but it does look very weird

perhaps trying to implement something like

b:
 phi1 = phi [b1, v1] [b2, v2]
 phi2 = phi [b1, v3] [b2, v4]
 r = op phi1, phi2

into

b1:
 n1 = op v1, v2
b2:
 n2 = op v3, v4
b:
 r = phi [b1, n1] [b2, n2]

and maybe only if at least one of the ops simplifies away to prevent increasing the number of op instructions

To confirm my understanding, we want the transformation above when one or more op could be simplified away because the result of op already exists as a variable, so op (along with the instructions that generates op parameters) could be eliminated away. Is that roughly the idea?

In particular for the test_struct_of_two_int example, op is or, and

%retval.sroa.3.0.extract.shift = and i64 %call, -4294967296
%phi.cast = and i64 %call, 4294967295

are used to generate or parameters. As a result, both could be optimized away.

when test_struct_of_two_int is properly fixed we should add a phase ordering test for it to make sure it fully optimizes correctly if we do go down this route

Also, it sounds the transformation could happen in a pass orthogonal with SROA (possibly after SROA, because PHI are inserted after SROA in this case).

So shall I prototype this transformation in a pass (wondering if inst combine is a good one), and insert the modified pass between SROA and the pass after SROA?

I guess I'm trying to get more instructions where this transformation should happen, around phase ordering.

[mingmingl]

In D114832#3181366, @Carrot wrote:

Driven by the compile-time feedback, another thing for me to consider, (probably after a preliminary convergence on the pass to add analysis info or do the transform), is to prune the existing solution so it's faster on the benchmark.

Your current method is

1 collect all accesses to an alloca object, mark the access type
2 check the aggregated access types, if there is only <store constant> type, mark the object as unsplittable.

I think in step1, if the access type is not <store constant>, you can bail out early, then for most splittable objects the check can be finished quickly.

whoops; I was typing so probably missed this suggestions just now. Bailing out early for most splittable cases sounds reasonable. I will do a quick experiment and report back the compile time(if possible). thanks!

In D114832#3181366, @Carrot wrote:

Driven by the compile-time feedback, another thing for me to consider, (probably after a preliminary convergence on the pass to add analysis info or do the transform), is to prune the existing solution so it's faster on the benchmark.

Your current method is

1 collect all accesses to an alloca object, mark the access type
2 check the aggregated access types, if there is only <store constant> type, mark the object as unsplittable.

I think in step1, if the access type is not <store constant>, you can bail out early, then for most splittable objects the check can be finished quickly.

[aeubanks]

In D114832#3181393, @luna wrote:

thanks for the feedback!

In D114832#3181314, @aeubanks wrote:

after running it through -O2:

define i64 @test_struct_of_two_int(i1 zeroext %test, i64 ()* nocapture %p) local_unnamed_addr {
entry:
  br i1 %test, label %return, label %if.end

if.end:                                           ; preds = %entry
  %call = tail call i64 %p()
  %retval.sroa.3.0.extract.shift = and i64 %call, -4294967296
  %phi.cast = and i64 %call, 4294967295
  br label %return

return:                                           ; preds = %entry, %if.end
  %retval.sroa.3.0 = phi i64 [ %retval.sroa.3.0.extract.shift, %if.end ], [ 0, %entry ]
  %retval.sroa.0.0 = phi i64 [ %phi.cast, %if.end ], [ 0, %entry ]
  %retval.sroa.0.0.insert.insert = or i64 %retval.sroa.0.0, %retval.sroa.3.0
  ret i64 %retval.sroa.0.0.insert.insert
}

yeah I'm not sure if this is something that only happens with SROA, but it does look very weird

perhaps trying to implement something like

b:
 phi1 = phi [b1, v1] [b2, v2]
 phi2 = phi [b1, v3] [b2, v4]
 r = op phi1, phi2

into

b1:
 n1 = op v1, v2
b2:
 n2 = op v3, v4
b:
 r = phi [b1, n1] [b2, n2]

and maybe only if at least one of the ops simplifies away to prevent increasing the number of op instructions

To confirm my understanding, we want the transformation above when one or more op could be simplified away because the result of op already exists as a variable, so op (along with the instructions that generates op parameters) could be eliminated away. Is that roughly the idea?

my thoughts were that we don't want to increase the number of instructions doing op in the code for code size reasons, so hopefully at least one of them can fold away. might have to run instsimplify (llvm::SimplifyInstruction) on the new op instruction and check that it actually gets simplified before proceeding
also moving the op into one of the incoming blocks is strictly better because now we won't do the op if we're coming from the other branch

In particular for the test_struct_of_two_int example, op is or, and

%retval.sroa.3.0.extract.shift = and i64 %call, -4294967296
%phi.cast = and i64 %call, 4294967295

are used to generate or parameters. As a result, both could be optimized away.

when test_struct_of_two_int is properly fixed we should add a phase ordering test for it to make sure it fully optimizes correctly if we do go down this route

Also, it sounds the transformation could happen in a pass orthogonal with SROA (possibly after SROA, because PHI are inserted after SROA in this case).

So shall I prototype this transformation in a pass (wondering if inst combine is a good one), and insert the modified pass between SROA and the pass after SROA?

I guess I'm trying to get more instructions where this transformation should happen, around phase ordering.

could first start with adding this to instcombine, instcombine runs a lot in the opt pipelines, this sort of transform isn't large/important enough to warrant its own pass

[mingmingl]

thanks everyone for the feedback!

I took the liberty to create another revision (https://reviews.llvm.org/D115914) and add all reviewers there.

Let me know if it's more idiomatic to just revise in this revision and I could make that happen.

llvm/include/llvm/Transforms/Scalar/SROA.h

private:
   bool presplitLoadsAndStores(AllocaInst &AI, sroa::AllocaSlices &AS);
   AllocaInst *rewritePartition(AllocaInst &AI, sroa::AllocaSlices &AS,
                                sroa::Partition &P);
   bool splitAlloca(AllocaInst &AI, sroa::AllocaSlices &AS);
   bool runOnAlloca(AllocaInst &AI);
   void clobberUse(Use &U);
   bool deleteDeadInstructions(SmallPtrSetImpl<AllocaInst *> &DeletedAllocas);
   bool promoteAllocas(Function &F);
+
+  bool scalarConstantStoresEquivalentToWholeUsage(
+      SmallVector<BasicBlock *> &BBPtrToFurtherAnalyze, sroa::AllocaSlices &AS,
+      uint64_t AllocaSize);
 };
 
 } // end namespace llvm
 
 #endif // LLVM_TRANSFORMS_SCALAR_SROA_H

llvm/lib/Transforms/Scalar/SROA.cpp

 STATISTIC(NumVectorized, "Number of vectorized aggregates");
 
 /// Hidden option to experiment with completely strict handling of inbounds
 /// GEPs.
 static cl::opt<bool> SROAStrictInbounds("sroa-strict-inbounds", cl::init(false),
                                         cl::Hidden);
 
 namespace {
+// The access pattern for an allocated type.
+// They are used as bit position to mask / unmask a bit map.
+//
+// Usage:
+//   The bit positions are supposed to be set by instruction walkers, and read
+// by alloca-instruction rewriters as necessary. For instance, function
+// `shouldSplitAllocaPerAccessBitMap` reads the masked positions to determine
+// whether to proceed with scalarization or not.
+//
+// Examples inline.
+//
+// Context:
+// struct Foo {int, int}
+// Foo foo_generator() {
+//   return {1, rand()};
+// }
+//
+// int arr[5];
+enum class AllocaAccessKind {
+  // The allocated type is stored as a whole, and value might be constant or
+  // dynamic.
+  //
+  // Foo f = foo_generator();
+  //
+  // memset(arr, 0, sizeof(int) * 5);
+  StoreAsAWhole = 0,
+  // The allocated type is written into with a constant through a scalar type.
+  //
+  // Foo f;
+  // f.x = 1;
+  //
+  // a[0] = 2;
+  ConstantStoreByScalar = 1,
+
+  // The structure type is stored via sub-fields, and the value is not contant.
+  //
+  // scalars in `f` is read.
+  // Foo f;
+  // f.x = rand();
+  //
+  // arr[2] = rand();
+  NonConstantStoreByScalar = 2,
+  // The allocated type is read as a whole.
+  //
+  // f is read as a whole.
+  // Foo f = {2, 3};
+  // Foo another = f;
+  //
+  // a is read as a whole.
+  // int b[5];
+  //
+  //
+  ReadAsAWhole = 3,
+  // The structure type is read as a scalar.
+  //
+  // scalars in `f` is read.
+
+  // Foo f;
+  // int tmp = f.y;
+
+  // int element = arr[1];
+  ReadByScalar = 4,
+  //
+  // The usage of alloca might be partial or full, but its instruction type is
+  // not covered by the access pattern analysis in AllocaSlices::SliceBuilder.
+  //
+  // Examples include:
+  // - memcpy
+  // - memmove
+  // - volatile operations
+  OtherUsage = 5,
+
+  // The number of different access types, used to define the length of bitmap.
+  // When more types are added above, AccessKindCount should be adjusted
+  // accordingly.
+  AccessKindCount = 6,
+};
+
+// A helper function to cast a enum value to an integer.
+inline uint16_t accessTypeToBitMapIndex(AllocaAccessKind ap) {
+  uint64_t ret = static_cast<uint16_t>(ap);
+  assert(ret < static_cast<uint16_t>(AllocaAccessKind::AccessKindCount));
+  return ret;
+}
+
+inline uint16_t accessTypeBitMapLength() {
+  return static_cast<uint16_t>(AllocaAccessKind::AccessKindCount);
+}
+
+// Returns true if accesss bit map indicates the alloca usage should be split;
+// return false otherwise.
+bool shouldSplitAllocaPerAccessBitMap(const APInt allocAccessBitMap) {
+  return // If there are scalar reads or non-constant stores, return true.
+      (allocAccessBitMap[accessTypeToBitMapIndex(
+           AllocaAccessKind::ReadByScalar)] ||
+       allocAccessBitMap[accessTypeToBitMapIndex(
+           AllocaAccessKind::NonConstantStoreByScalar)] ||
+       // If there are uncovered access patterns, returns true conservatively.
+       allocAccessBitMap[accessTypeToBitMapIndex(
+           AllocaAccessKind::OtherUsage)]);
+}
 
 /// A custom IRBuilder inserter which prefixes all names, but only in
 /// Assert builds.
 class IRBuilderPrefixedInserter final : public IRBuilderDefaultInserter {
   std::string Prefix;
 
   Twine getNameWithPrefix(const Twine &Name) const {
     return Name.isTriviallyEmpty() ? Name : Prefix + Name;
public:
   void insert(ArrayRef<Slice> NewSlices) {
     int OldSize = Slices.size();
     Slices.append(NewSlices.begin(), NewSlices.end());
     auto SliceI = Slices.begin() + OldSize;
     llvm::sort(SliceI, Slices.end());
     std::inplace_merge(Slices.begin(), SliceI, Slices.end());
   }
 
+  void insertAllocAccessMap(
+      const SmallDenseMap<BasicBlock *, APInt> &allocAccessMap) {
+    for (const auto &kv : allocAccessMap) {
+      BasicBlock *BBPtr = kv.first;
+      APInt incrementalAccess = kv.second;
+      auto iter = allocAccessBitMapPerBB.find(BBPtr);
+      if (iter != allocAccessBitMapPerBB.end()) {
+        iter->second |= incrementalAccess;
+      } else {
+        allocAccessBitMapPerBB.insert(std::make_pair(BBPtr, incrementalAccess));
+      }
+    }
+  }
+
   // Forward declare the iterator and range accessor for walking the
   // partitions.
   class partition_iterator;
   iterator_range<partition_iterator> partitions();
 
   /// Access the dead users for this alloca.
   ArrayRef<Instruction *> getDeadUsers() const { return DeadUsers; }
 

 #endif
 
 private:
   template <typename DerivedT, typename RetT = void> class BuilderBase;
   class SliceBuilder;
 
   friend class AllocaSlices::SliceBuilder;
 
+  // The size of this alloca instruction.
+  const uint64_t AllocaSize;
+
 #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
   /// Handle to alloca instruction to simplify method interfaces.
   AllocaInst &AI;
 #endif
 
   /// The instruction responsible for this alloca not having a known set
   /// of slices.
   ///
   /// When an instruction (potentially) escapes the pointer to the alloca, we
   /// store a pointer to that here and abort trying to form slices of the
   /// alloca. This will be null if the alloca slices are analyzed successfully.
   Instruction *PointerEscapingInstr;
 
-  /// The slices of the alloca.
-  ///
-  /// We store a vector of the slices formed by uses of the alloca here. This
-  /// vector is sorted by increasing begin offset, and then the unsplittable
-  /// slices before the splittable ones. See the Slice inner class for more
-  /// details.
-  SmallVector<Slice, 8> Slices;
-
   /// Instructions which will become dead if we rewrite the alloca.
   ///
   /// Note that these are not separated by slice. This is because we expect an
   /// alloca to be completely rewritten or not rewritten at all. If rewritten,
   /// all these instructions can simply be removed and replaced with undef as
   /// they come from outside of the allocated space.
   SmallVector<Instruction *, 8> DeadUsers;
 
   /// Uses which will become dead if can promote the alloca.
   SmallVector<Use *, 8> DeadUseIfPromotable;
 
   /// Operands which will become dead if we rewrite the alloca.
   ///
   /// These are operands that in their particular use can be replaced with
   /// undef when we rewrite the alloca. These show up in out-of-bounds inputs
   /// to PHI nodes and the like. They aren't entirely dead (there might be
   /// a GEP back into the bounds using it elsewhere) and nor is the PHI, but we
   /// want to swap this particular input for undef to simplify the use lists of
   /// the alloca.
   SmallVector<Use *, 8> DeadOperands;
+
+public:
+  /// The slices of the alloca.
+  ///
+  /// We store a vector of the slices formed by uses of the alloca here. This
+  /// vector is sorted by increasing begin offset, and then the unsplittable
+  /// slices before the splittable ones. See the Slice inner class for more
+  /// details.
+  SmallVector<Slice, 8> Slices;
+
+  /// NOTE: The following two fields are valid iff AllocaSlicesRewriter::insert
+  /// hasn't been called.
+  ///
+  /// A map from basic block pointer to the access pattern bit map.
+  SmallDenseMap<BasicBlock *, APInt> allocAccessBitMapPerBB;
+
+  /// Records the slice intervals per basic block, for all constant stores.
+  SmallDenseMap<BasicBlock *, SmallVector<std::pair<uint64_t, uint64_t>>>
+      constantStoreIntervalPerBB;
 };
 
 /// A partition of the slices.
 ///
 /// An ephemeral representation for a range of slices which can be viewed as
 /// a partition of the alloca. This range represents a span of the alloca's
 /// memory which cannot be split, and provides access to all of the slices
 /// overlapping some part of the partition.
class AllocaSlices::SliceBuilder : public PtrUseVisitor<SliceBuilder> {
 
   /// Set to de-duplicate dead instructions found in the use walk.
   SmallPtrSet<Instruction *, 4> VisitedDeadInsts;
 
 public:
   SliceBuilder(const DataLayout &DL, AllocaInst &AI, AllocaSlices &AS)
       : PtrUseVisitor<SliceBuilder>(DL),
         AllocSize(DL.getTypeAllocSize(AI.getAllocatedType()).getFixedSize()),
-        AS(AS) {}
+        AS(AS) {
+    Function *funcPtr = AI.getParent()->getParent();
+    for (Function::iterator FI = funcPtr->begin(), FE = funcPtr->end();
+         FI != FE; ++FI) {
+      BasicBlock *BBPtr = &*FI;
+      AS.allocAccessBitMapPerBB.insert(
+          std::make_pair(BBPtr, APInt(accessTypeBitMapLength(), 0)));
+    }
+  }
 
 private:
   void markAsDead(Instruction &I) {
     if (VisitedDeadInsts.insert(&I).second)
       AS.DeadUsers.push_back(&I);
   }
 
+  // Constructs Slice and appends it to the internal states of AllocaSlices.
   void insertUse(Instruction &I, const APInt &Offset, uint64_t Size,
                  bool IsSplittable = false) {
     // Completely skip uses which have a zero size or start either before or
     // past the end of the allocation.
     if (Size == 0 || Offset.uge(AllocSize)) {
       LLVM_DEBUG(dbgs() << "WARNING: Ignoring " << Size << " byte use @"
                         << Offset
                         << " which has zero size or starts outside of the "
if (SROAStrictInbounds && GEPI.isInBounds()) {
         if (GEPOffset.ugt(AllocSize))
           return markAsDead(GEPI);
       }
     }
 
     return Base::visitGetElementPtrInst(GEPI);
   }
 
+  void InsertOrUpdateBasicBlockAccessBitMap(BasicBlock *BBPtr,
+                                            const unsigned bitPos) {
+    auto iter = AS.allocAccessBitMapPerBB.find(BBPtr);
+    if (iter != AS.allocAccessBitMapPerBB.end()) {
+      iter->second.setBit(bitPos);
+    } else {
+      AS.allocAccessBitMapPerBB.insert(std::make_pair(
+          BBPtr, APInt::getOneBitSet(accessTypeBitMapLength(), bitPos)));
+    }
+  }
+
   void handleLoadOrStore(Type *Ty, Instruction &I, const APInt &Offset,
                          uint64_t Size, bool IsVolatile) {
+    assert((isa<LoadInst>(I) || isa<StoreInst>(I)) &&
+           "Expected load or store instruction");
     // We allow splitting of non-volatile loads and stores where the type is an
     // integer type. These may be used to implement 'memcpy' or other "transfer
     // of bits" patterns.
     bool IsSplittable =
         Ty->isIntegerTy() && !IsVolatile && DL.typeSizeEqualsStoreSize(Ty);
 
+    if (isa<LoadInst>(I)) {
+      InsertOrUpdateBasicBlockAccessBitMap(
+          I.getParent(),
+          IsVolatile
+              ? accessTypeToBitMapIndex(AllocaAccessKind::OtherUsage)
+              : ((Size == AllocSize)
+                     ? accessTypeToBitMapIndex(AllocaAccessKind::ReadAsAWhole)
+                     : accessTypeToBitMapIndex(
+                           AllocaAccessKind::ReadByScalar)));
+    } else {
+      assert(isa<StoreInst>(I) && "Expected a store instruction");
+      StoreInst *storeInstPtr = dyn_cast<StoreInst>(&I);
+      bool storeConstant = isa<Constant>(storeInstPtr->getValueOperand());
+
+      const int bitPos =
+          IsVolatile
+              ? (accessTypeToBitMapIndex(AllocaAccessKind::OtherUsage))
+              : ((Size == AllocSize)
+                     ? accessTypeToBitMapIndex(AllocaAccessKind::StoreAsAWhole)
+                     : (storeConstant
+                            ? accessTypeToBitMapIndex(
+                                  AllocaAccessKind::ConstantStoreByScalar)
+                            : accessTypeToBitMapIndex(
+                                  AllocaAccessKind::NonConstantStoreByScalar)));
+      if (storeConstant) {
+        AS.constantStoreIntervalPerBB[I.getParent()].push_back(std::make_pair(
+            Offset.getLimitedValue(), Offset.getLimitedValue() + Size));
+      }
+      InsertOrUpdateBasicBlockAccessBitMap(I.getParent(), bitPos);
+    }
+
     insertUse(I, Offset, Size, IsSplittable);
   }
 
   void visitLoadInst(LoadInst &LI) {
     assert((!LI.isSimple() || LI.getType()->isSingleValueType()) &&
            "All simple FCA loads should have been pre-split");
 
     if (!IsOffsetKnown)
void visitMemSetInst(MemSetInst &II) {
     if (!IsOffsetKnown)
       return PI.setAborted(&II);
 
     // Don't replace this with a store with a different address space.  TODO:
     // Use a store with the casted new alloca?
     if (II.isVolatile() && II.getDestAddressSpace() != DL.getAllocaAddrSpace())
       return PI.setAborted(&II);
 
-    insertUse(II, Offset, Length ? Length->getLimitedValue()
-                                 : AllocSize - Offset.getLimitedValue(),
+    InsertOrUpdateBasicBlockAccessBitMap(
+        II.getParent(),
+
+        (accessTypeToBitMapIndex(AllocaAccessKind::OtherUsage)));
+
+    insertUse(II, Offset,
+              Length ? Length->getLimitedValue()
               (bool)Length);
   }
 
   void visitMemTransferInst(MemTransferInst &II) {
     ConstantInt *Length = dyn_cast<ConstantInt>(II.getLength());
     if (Length && Length->getValue() == 0)
       // Zero-length mem transfer intrinsics can be ignored entirely.
       return markAsDead(II);
 
     // Because we can visit these intrinsics twice, also check to see if the
     // first time marked this instruction as dead. If so, skip it.
     if (VisitedDeadInsts.count(&II))
       return;
 
     if (!IsOffsetKnown)
       return PI.setAborted(&II);
 
+    const uint64_t RawOffset = Offset.getLimitedValue();
+    const uint64_t SliceSize =
+        Length ? Length->getLimitedValue() : AllocSize - RawOffset;
+
+    // Regard all `memcpy` and `memmove` as other usages.
+    // There is a separate pass to optimize `memcpy`.
+    InsertOrUpdateBasicBlockAccessBitMap(
+        II.getParent(), accessTypeToBitMapIndex(AllocaAccessKind::OtherUsage));
+
     // Don't replace this with a load/store with a different address space.
     // TODO: Use a store with the casted new alloca?
     if (II.isVolatile() &&
         (II.getDestAddressSpace() != DL.getAllocaAddrSpace() ||
          II.getSourceAddressSpace() != DL.getAllocaAddrSpace()))
       return PI.setAborted(&II);
 
     // This side of the transfer is completely out-of-bounds, and so we can
     // nuke the entire transfer. However, we also need to nuke the other side
     // if already added to our partitions.
     // FIXME: Yet another place we really should bypass this when
     // instrumenting for ASan.
     if (Offset.uge(AllocSize)) {
       SmallDenseMap<Instruction *, unsigned>::iterator MTPI =
           MemTransferSliceMap.find(&II);
       if (MTPI != MemTransferSliceMap.end())
         AS.Slices[MTPI->second].kill();
       return markAsDead(II);
     }
 
-    uint64_t RawOffset = Offset.getLimitedValue();
-    uint64_t Size = Length ? Length->getLimitedValue() : AllocSize - RawOffset;
-
     // Check for the special case where the same exact value is used for both
     // source and dest.
     if (*U == II.getRawDest() && *U == II.getRawSource()) {
       // For non-volatile transfers this is a no-op.
       if (!II.isVolatile())
         return markAsDead(II);
 
-      return insertUse(II, Offset, Size, /*IsSplittable=*/false);
+      return insertUse(II, Offset, SliceSize, /*IsSplittable=*/false);
     }
 
     // If we have seen both source and destination for a mem transfer, then
     // they both point to the same alloca.
     bool Inserted;
     SmallDenseMap<Instruction *, unsigned>::iterator MTPI;
     std::tie(MTPI, Inserted) =
         MemTransferSliceMap.insert(std::make_pair(&II, AS.Slices.size()));
if (!Inserted) {
       }
 
       // Otherwise we have an offset transfer within the same alloca. We can't
       // split those.
       PrevP.makeUnsplittable();
     }
 
     // Insert the use now that we've fixed up the splittable nature.
-    insertUse(II, Offset, Size, /*IsSplittable=*/Inserted && Length);
+    insertUse(II, Offset, SliceSize, /*IsSplittable=*/Inserted && Length);
 
     // Check that we ended up with a valid index in the map.
     assert(AS.Slices[PrevIdx].getUse()->getUser() == &II &&
            "Map index doesn't point back to a slice with this user.");
   }
 
   // Disable SRoA for any intrinsics except for lifetime invariants and
   // invariant group.
+  //
   // FIXME: What about debug intrinsics? This matches old behavior, but
   // doesn't make sense.
   void visitIntrinsicInst(IntrinsicInst &II) {
     if (II.isDroppable()) {
       AS.DeadUseIfPromotable.push_back(U);
       return;
     }
 
     if (!IsOffsetKnown)
       return PI.setAborted(&II);
 
+    // For lifetime invariants, record the access kind as `OtherUsage`.
     if (II.isLifetimeStartOrEnd()) {
       ConstantInt *Length = cast<ConstantInt>(II.getArgOperand(0));
       uint64_t Size = std::min(AllocSize - Offset.getLimitedValue(),
                                Length->getLimitedValue());
+      InsertOrUpdateBasicBlockAccessBitMap(
+          II.getParent(),
+          accessTypeToBitMapIndex(AllocaAccessKind::OtherUsage));
+
       insertUse(II, Offset, Size, true);
       return;
     }
 
+    // For invariant group, record the access kind as `OtherUsage`.
     if (II.isLaunderOrStripInvariantGroup()) {
+      InsertOrUpdateBasicBlockAccessBitMap(
+          II.getParent(),
+          accessTypeToBitMapIndex(AllocaAccessKind::OtherUsage));
       enqueueUsers(II);
       return;
     }
 
     Base::visitIntrinsicInst(II);
   }
 
   Instruction *hasUnsafePHIOrSelectUse(Instruction *Root, uint64_t &Size) {
void visitPHINodeOrSelectInst(Instruction &I) {
     // themselves which should be replaced with undef.
     // FIXME: This should instead be escaped in the event we're instrumenting
     // for address sanitization.
     if (Offset.uge(AllocSize)) {
       AS.DeadOperands.push_back(U);
       return;
     }
 
+    InsertOrUpdateBasicBlockAccessBitMap(
+        I.getParent(),
+        (Offset.getLimitedValue() == 0 && Size == AllocSize)
+            ? accessTypeToBitMapIndex(AllocaAccessKind::ReadAsAWhole)
+            : accessTypeToBitMapIndex(AllocaAccessKind::ReadByScalar));
+
     insertUse(I, Offset, Size);
   }
 
   void visitPHINode(PHINode &PN) { visitPHINodeOrSelectInst(PN); }
 
   void visitSelectInst(SelectInst &SI) { visitPHINodeOrSelectInst(SI); }
 
   /// Disable SROA entirely if there are unhandled users of the alloca.
   void visitInstruction(Instruction &I) { PI.setAborted(&I); }
 };
 
 AllocaSlices::AllocaSlices(const DataLayout &DL, AllocaInst &AI)
-    :
+    : AllocaSize(DL.getTypeAllocSize(AI.getAllocatedType()).getFixedSize()),
 #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
       AI(AI),
 #endif
       PointerEscapingInstr(nullptr) {
   SliceBuilder PB(DL, AI, *this);
   SliceBuilder::PtrInfo PtrI = PB.visitPtr(AI);
   if (PtrI.isEscaped() || PtrI.isAborted()) {
     // FIXME: We should sink the escape vs. abort info into the caller nicely,
     // possibly by just storing the PtrInfo in the AllocaSlices.
     PointerEscapingInstr = PtrI.getEscapingInst() ? PtrI.getEscapingInst()
                                                   : PtrI.getAbortingInst();
     assert(PointerEscapingInstr && "Did not track a bad instruction");
     return;
   }
 
   llvm::erase_if(Slices, [](const Slice &S) { return S.isDead(); });
 
   // Sort the uses. This arranges for the offsets to be in ascending order,
   // and the sizes to be in descending order.
   llvm::stable_sort(Slices);
+
+  // Sort the intervals of each basic block.
+  for (auto IB = constantStoreIntervalPerBB.begin(),
+            IE = constantStoreIntervalPerBB.end();
+       IB != IE; IB++) {
+    llvm::stable_sort(IB->second, [](const std::pair<uint64_t, uint64_t> &a,
+                                     const std::pair<uint64_t, uint64_t> &b) {
+      if (a.first < b.first)
+        return true;
+      if (a.first > b.first)
+        return false;
+      if (a.second > b.second)
+        return true;
+      return false;
+    });
+  }
 }
 
 #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
 
 void AllocaSlices::print(raw_ostream &OS, const_iterator I,
                          StringRef Indent) const {
   printSlice(OS, I, Indent);
   OS << "\n";
class AggLoadStoreRewriter : public InstVisitor<AggLoadStoreRewriter, bool> {
   /// Used to calculate offsets, and hence alignment, of subobjects.
   const DataLayout &DL;
 
 public:
   AggLoadStoreRewriter(const DataLayout &DL) : DL(DL) {}
 
   /// Rewrite loads and stores through a pointer and all pointers derived from
   /// it.
-  bool rewrite(Instruction &I) {
+  bool rewrite(AllocaInst &AI) {
+
     LLVM_DEBUG(dbgs() << "  Rewriting FCA loads and stores...\n");
-    enqueueUsers(I);
+
+    enqueueUsers(AI);
     bool Changed = false;
     while (!Queue.empty()) {
       U = Queue.pop_back_val();
       Changed |= visit(cast<Instruction>(U->getUser()));
     }
     return Changed;
   }
 
bool SROAPass::presplitLoadsAndStores(AllocaInst &AI, AllocaSlices &AS) {
 
   // Collect the new slices which we will merge into the alloca slices.
   SmallVector<Slice, 4> NewSlices;
 
   // Track any allocas we end up splitting loads and stores for so we iterate
   // on them.
   SmallPtrSet<AllocaInst *, 4> ResplitPromotableAllocas;
 
+  // Tracks the alloc access bit map for newly added instructions.
+  SmallDenseMap<BasicBlock *, APInt> incrementalAllocAccessBitMapPerBasicBlock;
+
   // At this point, we have collected all of the loads and stores we can
   // pre-split, and the specific splits needed for them. We actually do the
   // splitting in a specific order in order to handle when one of the loads in
   // the value operand to one of the stores.
   //
   // First, we rewrite all of the split loads, and just accumulate each split
   // load in a parallel structure. We also build the slices for them and append
   // them to the alloca slices.
for (;;) {
           /*IsVolatile*/ false, LI->getName());
       PLoad->copyMetadata(*LI, {LLVMContext::MD_mem_parallel_loop_access,
                                 LLVMContext::MD_access_group});
 
       // Append this load onto the list of split loads so we can find it later
       // to rewrite the stores.
       SplitLoads.push_back(PLoad);
 
+      const unsigned bitPos =
+          PartSize == LoadSize
+              ? accessTypeToBitMapIndex(AllocaAccessKind::ReadAsAWhole)
+              : accessTypeToBitMapIndex(AllocaAccessKind::ReadByScalar);
+
+      auto iter =
+          incrementalAllocAccessBitMapPerBasicBlock.find(LI->getParent());
+      if (iter != incrementalAllocAccessBitMapPerBasicBlock.end()) {
+        iter->second.setBit(bitPos);
+      } else {
+        incrementalAllocAccessBitMapPerBasicBlock.insert(std::make_pair(
+            LI->getParent(),
+            APInt::getOneBitSet(accessTypeBitMapLength(), bitPos)));
+      }
+
       // Now build a new slice for the alloca.
       NewSlices.push_back(
           Slice(BaseOffset + PartOffset, BaseOffset + PartOffset + PartSize,
                 &PLoad->getOperandUse(PLoad->getPointerOperandIndex()),
                 /*IsSplittable*/ false));
       LLVM_DEBUG(dbgs() << "    new slice [" << NewSlices.back().beginOffset()
                         << ", " << NewSlices.back().endOffset()
                         << "): " << *PLoad << "\n");
for (;;) {
           getAdjustedPtr(IRB, DL, StoreBasePtr,
                          APInt(DL.getIndexSizeInBits(AS), PartOffset),
                          StorePartPtrTy, StoreBasePtr->getName() + "."),
           getAdjustedAlignment(SI, PartOffset),
           /*IsVolatile*/ false);
       PStore->copyMetadata(*SI, {LLVMContext::MD_mem_parallel_loop_access,
                                  LLVMContext::MD_access_group});
 
+      const bool storeValIsConstant = isa<Constant>(PStore->getValueOperand());
+      const int bitPos =
+          (PartSize == StoreSize)
+              ? accessTypeToBitMapIndex(AllocaAccessKind::StoreAsAWhole)
+              : (storeValIsConstant
+                     ? accessTypeToBitMapIndex(
+                           AllocaAccessKind::ConstantStoreByScalar)
+                     : accessTypeToBitMapIndex(
+                           AllocaAccessKind::NonConstantStoreByScalar));
+
+      auto iter =
+          incrementalAllocAccessBitMapPerBasicBlock.find(SI->getParent());
+      if (iter != incrementalAllocAccessBitMapPerBasicBlock.end()) {
+        iter->second.setBit(bitPos);
+      } else {
+        incrementalAllocAccessBitMapPerBasicBlock.insert(std::make_pair(
+            SI->getParent(),
+            APInt::getOneBitSet(accessTypeBitMapLength(), bitPos)));
+      }
+
       // Now build a new slice for the alloca.
       NewSlices.push_back(
           Slice(BaseOffset + PartOffset, BaseOffset + PartOffset + PartSize,
                 &PStore->getOperandUse(PStore->getPointerOperandIndex()),
                 /*IsSplittable*/ false));
       LLVM_DEBUG(dbgs() << "    new slice [" << NewSlices.back().beginOffset()
                         << ", " << NewSlices.back().endOffset()
                         << "): " << *PStore << "\n");
bool SROAPass::presplitLoadsAndStores(AllocaInst &AI, AllocaSlices &AS) {
   }
 
   // Remove the killed slices that have ben pre-split.
   llvm::erase_if(AS, [](const Slice &S) { return S.isDead(); });
 
   // Insert our new slices. This will sort and merge them into the sorted
   // sequence.
   AS.insert(NewSlices);
+  // Insert the incremental alloca access bit map.
+  AS.insertAllocAccessMap(incrementalAllocAccessBitMapPerBasicBlock);
 
   LLVM_DEBUG(dbgs() << "  Pre-split slices:\n");
 #ifndef NDEBUG
   for (auto I = AS.begin(), E = AS.end(); I != E; ++I)
     LLVM_DEBUG(AS.print(dbgs(), I, "    "));
 #endif
 
   // Finally, don't try to promote any allocas that new require re-splitting.
if (Promotable) {
     // refinements exposed by splitting the current alloca. Don't iterate on an
     // alloca which didn't actually change and didn't get promoted.
     Worklist.insert(NewAI);
   }
 
   return NewAI;
 }
 
+// For each basic block in `BBPtrToFurtherAnalyze`, analyze the intervals of
+// store instructions which stores constant value, and returns true iff constant
+// stores cover the whole allocated address.
+//
+// FIXME: Cover the scenario when constant stores effectively covers all
+// slices but `AllocaSize` is larger due to padding (e.g. see
+// `test_struct_of_int_char` in test/Transforms/SROA/alloca-struct.ll).
+//
+// Caller guarantees that there are no scalar usage in the basic blocks to
+// analyze.
+bool SROAPass::scalarConstantStoresEquivalentToWholeUsage(
+    SmallVector<BasicBlock *> &BBPtrToFurtherAnalyze, AllocaSlices &AS,
+    uint64_t AllocaSize) {
+  bool wholeUsage = true;
+  while (!BBPtrToFurtherAnalyze.empty()) {
+    BasicBlock *BBPtr = BBPtrToFurtherAnalyze.pop_back_val();
+    const auto &constantStoreIntervals = AS.constantStoreIntervalPerBB[BBPtr];
+
+    assert((!constantStoreIntervals.empty()) &&
+           "A basic block to analyze "
+           "should at least have scalar constant stores");
+
+    auto iter = constantStoreIntervals.begin();
+    uint64_t MinOffset = iter->first, MaxOffset = iter->second;
+    assert((MinOffset <= MaxOffset) && "MinOffset should be smaller than or "
+                                       "equal to MaxOffset");
+    for (++iter; iter != constantStoreIntervals.end(); ++iter) {
+      assert((iter->first <= iter->second) &&
+             "MinOffset should be smaller than or equal to MaxOffset");
+      if (iter->first > MaxOffset) {
+        // There are gaps in the intervals.
+        wholeUsage = false;
+        break;
+      }
+
+      MinOffset = std::min(MinOffset, iter->first);
+      MaxOffset = std::max(MaxOffset, iter->second);
+    } // end for
+    if (MinOffset != 0 || MaxOffset != AllocaSize) {
+      wholeUsage = false;
+    }
+
+    if (!wholeUsage) {
+      break;
+    }
+  } // end while
+  return wholeUsage;
+}
+
 /// Walks the slices of an alloca and form partitions based on them,
 /// rewriting each of their uses.
 bool SROAPass::splitAlloca(AllocaInst &AI, AllocaSlices &AS) {
   if (AS.begin() == AS.end())
     return false;
 
   unsigned NumPartitions = 0;
   bool Changed = false;
   const DataLayout &DL = AI.getModule()->getDataLayout();
 
   // First try to pre-split loads and stores.
+  // Note, `presplitLoadsAndStores` may insert new instructions and
+  // corresponding Slices, as well as update `allocAccessBitMapPerBB` and
+  // `allocAccessBitMapPerBB` accordingly.
+  // When this function has any update (i.e., presplit is true), the access
+  // pattern analysis below won't happen.
+  // FIXME: Make access pattern analysis work when `presplit` is true.
   Changed |= presplitLoadsAndStores(AI, AS);
 
   // Now that we have identified any pre-splitting opportunities,
   // mark loads and stores unsplittable except for the following case.
   // We leave a slice splittable if all other slices are disjoint or fully
   // included in the slice, such as whole-alloca loads and stores.
   // If we fail to split these during pre-splitting, we want to force them
   // to be rewritten into a partition.
   bool IsSorted = true;
 
   uint64_t AllocaSize =
       DL.getTypeAllocSize(AI.getAllocatedType()).getFixedSize();
   const uint64_t MaxBitVectorSize = 1024;
   if (AllocaSize <= MaxBitVectorSize) {
-    // If a byte boundary is included in any load or store, a slice starting or
-    // ending at the boundary is not splittable.
+    bool allocaHasScalarUsage = false;
+    SmallVector<BasicBlock *> BBPtrToFurtherAnalyze;
+    for (const auto &kv : AS.allocAccessBitMapPerBB) {
+      BasicBlock *BBPtr = kv.first;
+      APInt access = kv.second;
+      if (shouldSplitAllocaPerAccessBitMap(access)) {
+        allocaHasScalarUsage = true;
+        break;
+      }
+      if (access[accessTypeToBitMapIndex(
+              AllocaAccessKind::ConstantStoreByScalar)]) {
+        BBPtrToFurtherAnalyze.push_back(BBPtr);
+      }
+    }
+
+    if ((!Changed) && (!allocaHasScalarUsage) &&
+        scalarConstantStoresEquivalentToWholeUsage(BBPtrToFurtherAnalyze, AS,
+                                                   AllocaSize)) {
+      // Scalar load has been checked and excluded, there must be a load as
+      // whole as well as corresponding whole slice.
+      //
+      // Make each slice unsplittable so they form one partition; this way
+      // the alloca instruction won't be split.
+      //
+      // Rather than return early, function will proceed so that
+      // `SROA::rewritePartition` will be invoked and the logic to evaluate
+      // evaluate mem2reg readiness is reused.
+      for (Slice &S : AS) {
+        S.makeUnsplittable();
+      }
+    } else {
+      // If a byte boundary is included in any load or store, a slice starting
+      // or ending at the boundary is not splittable.
       SmallBitVector SplittableOffset(AllocaSize + 1, true);
       for (Slice &S : AS)
         for (unsigned O = S.beginOffset() + 1;
              O < S.endOffset() && O < AllocaSize; O++)
           SplittableOffset.reset(O);
 
       for (Slice &S : AS) {
         if (!S.isSplittable())
           continue;
 
-      if ((S.beginOffset() > AllocaSize || SplittableOffset[S.beginOffset()]) &&
+        if ((S.beginOffset() > AllocaSize ||
+             SplittableOffset[S.beginOffset()]) &&
             (S.endOffset() > AllocaSize || SplittableOffset[S.endOffset()]))
           continue;
 
         if (isa<LoadInst>(S.getUse()->getUser()) ||
             isa<StoreInst>(S.getUse()->getUser())) {
           S.makeUnsplittable();
           IsSorted = false;
         }
       }
     }
-  else {
+  } else {
     // We only allow whole-alloca splittable loads and stores
     // for a large alloca to avoid creating too large BitVector.
     for (Slice &S : AS) {
       if (!S.isSplittable())
         continue;
 
       if (S.beginOffset() == 0 && S.endOffset() >= AllocaSize)
         continue;

 ///
 /// We also record the alloca instructions deleted here so that they aren't
 /// subsequently handed to mem2reg to promote.
 bool SROAPass::deleteDeadInstructions(
     SmallPtrSetImpl<AllocaInst *> &DeletedAllocas) {
   bool Changed = false;
   while (!DeadInsts.empty()) {
     Instruction *I = dyn_cast_or_null<Instruction>(DeadInsts.pop_back_val());
-    if (!I) continue; 
+    if (!I)
+      continue;
     LLVM_DEBUG(dbgs() << "Deleting dead instruction: " << *I << "\n");
 
     // If the instruction is an alloca, find the possible dbg.declare connected
     // to it, and remove it too. We must do this before calling RAUW or we will
     // not be able to find it.
     if (AllocaInst *AI = dyn_cast<AllocaInst>(I)) {
       DeletedAllocas.insert(AI);
       for (DbgVariableIntrinsic *OldDII : FindDbgAddrUses(AI))

llvm/test/Transforms/SROA/alloca-struct.ll

 }
 
 ; Test that the alloca of struct{int, int} will be scalarized by SROA.
 define i64 @test_struct_of_two_int(i1 zeroext %test, i64 ()* %p) {
 ; CHECK-LABEL: @test_struct_of_two_int(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br i1 [[TEST:%.*]], label [[IF_THEN:%.*]], label [[IF_END:%.*]]
 ; CHECK:       if.then:
+; CHECK-NEXT:    [[RETVAL_SROA_0_0_INSERT_MASK:%.*]] = and i64 undef, -4294967296
+; CHECK-NEXT:    [[RETVAL_SROA_0_4_INSERT_MASK:%.*]] = and i64 [[RETVAL_SROA_0_0_INSERT_MASK]], 4294967295
 ; CHECK-NEXT:    br label [[RETURN:%.*]]
 ; CHECK:       if.end:
 ; CHECK-NEXT:    [[CALL:%.*]] = call i64 [[P:%.*]]()
-; CHECK-NEXT:    [[RETVAL_SROA_0_0_EXTRACT_TRUNC:%.*]] = trunc i64 [[CALL]] to i32
-; CHECK-NEXT:    [[RETVAL_SROA_3_0_EXTRACT_SHIFT:%.*]] = lshr i64 [[CALL]], 32
-; CHECK-NEXT:    [[RETVAL_SROA_3_0_EXTRACT_TRUNC:%.*]] = trunc i64 [[RETVAL_SROA_3_0_EXTRACT_SHIFT]] to i32
 ; CHECK-NEXT:    br label [[RETURN]]
 ; CHECK:       return:
-; CHECK-NEXT:    [[RETVAL_SROA_3_0:%.*]] = phi i32 [ 0, [[IF_THEN]] ], [ [[RETVAL_SROA_3_0_EXTRACT_TRUNC]], [[IF_END]] ]
-; CHECK-NEXT:    [[RETVAL_SROA_0_0:%.*]] = phi i32 [ 0, [[IF_THEN]] ], [ [[RETVAL_SROA_0_0_EXTRACT_TRUNC]], [[IF_END]] ]
-; CHECK-NEXT:    [[RETVAL_SROA_3_0_INSERT_EXT:%.*]] = zext i32 [[RETVAL_SROA_3_0]] to i64
-; CHECK-NEXT:    [[RETVAL_SROA_3_0_INSERT_SHIFT:%.*]] = shl i64 [[RETVAL_SROA_3_0_INSERT_EXT]], 32
-; CHECK-NEXT:    [[RETVAL_SROA_3_0_INSERT_MASK:%.*]] = and i64 undef, 4294967295
-; CHECK-NEXT:    [[RETVAL_SROA_3_0_INSERT_INSERT:%.*]] = or i64 [[RETVAL_SROA_3_0_INSERT_MASK]], [[RETVAL_SROA_3_0_INSERT_SHIFT]]
-; CHECK-NEXT:    [[RETVAL_SROA_0_0_INSERT_EXT:%.*]] = zext i32 [[RETVAL_SROA_0_0]] to i64
-; CHECK-NEXT:    [[RETVAL_SROA_0_0_INSERT_MASK:%.*]] = and i64 [[RETVAL_SROA_3_0_INSERT_INSERT]], -4294967296
-; CHECK-NEXT:    [[RETVAL_SROA_0_0_INSERT_INSERT:%.*]] = or i64 [[RETVAL_SROA_0_0_INSERT_MASK]], [[RETVAL_SROA_0_0_INSERT_EXT]]
-; CHECK-NEXT:    ret i64 [[RETVAL_SROA_0_0_INSERT_INSERT]]
+; CHECK-NEXT:    [[RETVAL_SROA_0_0:%.*]] = phi i64 [ [[RETVAL_SROA_0_4_INSERT_MASK]], [[IF_THEN]] ], [ [[CALL]], [[IF_END]] ]
+; CHECK-NEXT:    ret i64 [[RETVAL_SROA_0_0]]
 ;
 
 entry:
   %retval = alloca %struct.RetValTwoInts, align 4
   br i1 %test, label %if.then, label %if.end
 
 if.then:                                          ; preds = %entry
   %x = getelementptr inbounds %struct.RetValTwoInts, %struct.RetValTwoInts* %retval, i32 0, i32 0

llvm/test/Transforms/SROA/basictest-opaque-ptrs.ll

L2:
   ret i64 %Z
 }
 
 ; Avoid crashing when load/storing at at different offsets.
 define i64 @test2_addrspacecast_gep_offset(i64 %X) {
 ; CHECK-LABEL: @test2_addrspacecast_gep_offset(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[A_SROA_0:%.*]] = alloca [10 x i8], align 1
-; CHECK-NEXT:    [[A_SROA_0_2_GEPB_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_0]], i16 2
-; CHECK-NEXT:    [[A_SROA_0_2_GEPB_SROA_CAST:%.*]] = addrspacecast ptr [[A_SROA_0_2_GEPB_SROA_IDX]] to ptr addrspace(1)
-; CHECK-NEXT:    store i64 [[X:%.*]], ptr addrspace(1) [[A_SROA_0_2_GEPB_SROA_CAST]], align 1
+; CHECK-NEXT:    [[A_SROA_0_2_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_0]], i16 2
+; CHECK-NEXT:    [[A_SROA_0_2_SROA_CAST:%.*]] = addrspacecast ptr [[A_SROA_0_2_SROA_IDX]] to ptr addrspace(1)
+; CHECK-NEXT:    store i64 [[X:%.*]], ptr addrspace(1) [[A_SROA_0_2_SROA_CAST]], align 1
 ; CHECK-NEXT:    br label [[L2:%.*]]
 ; CHECK:       L2:
 ; CHECK-NEXT:    [[A_SROA_0_0_A_SROA_0_30_Z:%.*]] = load i64, ptr [[A_SROA_0]], align 1
 ; CHECK-NEXT:    ret i64 [[A_SROA_0_0_A_SROA_0_30_Z]]
 ;
 
 entry:
   %A = alloca [256 x i8]

 define void @test3(i8* %dst, i8* align 8 %src) {
 ; CHECK-LABEL: @test3(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[A_SROA_0:%.*]] = alloca [42 x i8], align 1
 ; CHECK-NEXT:    [[A_SROA_3:%.*]] = alloca [99 x i8], align 1
 ; CHECK-NEXT:    [[A_SROA_32:%.*]] = alloca [16 x i8], align 1
 ; CHECK-NEXT:    [[A_SROA_15:%.*]] = alloca [42 x i8], align 1
 ; CHECK-NEXT:    [[A_SROA_16:%.*]] = alloca [7 x i8], align 1
-; CHECK-NEXT:    [[A_SROA_234:%.*]] = alloca [7 x i8], align 1
+; CHECK-NEXT:    [[A_SROA_235:%.*]] = alloca [7 x i8], align 1
 ; CHECK-NEXT:    [[A_SROA_31:%.*]] = alloca [85 x i8], align 1
 ; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_0]], ptr align 8 [[SRC:%.*]], i32 42, i1 false), !tbaa [[TBAA0:![0-9]+]]
 ; CHECK-NEXT:    [[A_SROA_2_0_SRC_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i64 42
 ; CHECK-NEXT:    [[A_SROA_2_0_COPYLOAD:%.*]] = load i8, ptr [[A_SROA_2_0_SRC_SROA_IDX]], align 2, !tbaa [[TBAA0]]
 ; CHECK-NEXT:    [[A_SROA_3_0_SRC_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i64 43
 ; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_3]], ptr align 1 [[A_SROA_3_0_SRC_SROA_IDX]], i32 99, i1 false), !tbaa [[TBAA0]]
 ; CHECK-NEXT:    [[A_SROA_32_0_SRC_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i64 142
 ; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_32]], ptr align 2 [[A_SROA_32_0_SRC_SROA_IDX]], i32 16, i1 false), !tbaa [[TBAA0]]
 ; CHECK-NEXT:    [[A_SROA_15_0_SRC_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i64 158
 ; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_15]], ptr align 2 [[A_SROA_15_0_SRC_SROA_IDX]], i32 42, i1 false), !tbaa [[TBAA0]]
 ; CHECK-NEXT:    [[A_SROA_16_0_SRC_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i64 200
 ; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_16]], ptr align 8 [[A_SROA_16_0_SRC_SROA_IDX]], i32 7, i1 false), !tbaa [[TBAA0]]
 ; CHECK-NEXT:    [[A_SROA_23_0_SRC_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i64 207
 ; CHECK-NEXT:    [[A_SROA_23_0_COPYLOAD:%.*]] = load i8, ptr [[A_SROA_23_0_SRC_SROA_IDX]], align 1, !tbaa [[TBAA0]]
-; CHECK-NEXT:    [[A_SROA_234_0_SRC_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i64 208
-; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_234]], ptr align 8 [[A_SROA_234_0_SRC_SROA_IDX]], i32 7, i1 false), !tbaa [[TBAA0]]
+; CHECK-NEXT:    [[A_SROA_235_0_SRC_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i64 208
+; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_235]], ptr align 8 [[A_SROA_235_0_SRC_SROA_IDX]], i32 7, i1 false), !tbaa [[TBAA0]]
 ; CHECK-NEXT:    [[A_SROA_31_0_SRC_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i64 215
 ; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_31]], ptr align 1 [[A_SROA_31_0_SRC_SROA_IDX]], i32 85, i1 false), !tbaa [[TBAA0]]
 ; CHECK-NEXT:    store i8 1, ptr [[A_SROA_32]], align 1, !tbaa [[TBAA3:![0-9]+]]
 ; CHECK-NEXT:    store i16 1, ptr [[A_SROA_32]], align 1, !tbaa [[TBAA5:![0-9]+]]
 ; CHECK-NEXT:    store i32 1, ptr [[A_SROA_32]], align 1, !tbaa [[TBAA7:![0-9]+]]
 ; CHECK-NEXT:    store i64 1, ptr [[A_SROA_32]], align 1, !tbaa [[TBAA9:![0-9]+]]
-; CHECK-NEXT:    [[A_SROA_32_1_OVERLAP_2_I64_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_32]], i64 1
-; CHECK-NEXT:    store i64 2, ptr [[A_SROA_32_1_OVERLAP_2_I64_SROA_IDX]], align 1, !tbaa [[TBAA11:![0-9]+]]
-; CHECK-NEXT:    [[A_SROA_32_2_OVERLAP_3_I64_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_32]], i64 2
-; CHECK-NEXT:    store i64 3, ptr [[A_SROA_32_2_OVERLAP_3_I64_SROA_IDX]], align 1, !tbaa [[TBAA13:![0-9]+]]
-; CHECK-NEXT:    [[A_SROA_32_3_OVERLAP_4_I64_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_32]], i64 3
-; CHECK-NEXT:    store i64 4, ptr [[A_SROA_32_3_OVERLAP_4_I64_SROA_IDX]], align 1, !tbaa [[TBAA15:![0-9]+]]
-; CHECK-NEXT:    [[A_SROA_32_4_OVERLAP_5_I64_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_32]], i64 4
-; CHECK-NEXT:    store i64 5, ptr [[A_SROA_32_4_OVERLAP_5_I64_SROA_IDX]], align 1, !tbaa [[TBAA17:![0-9]+]]
-; CHECK-NEXT:    [[A_SROA_32_5_OVERLAP_6_I64_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_32]], i64 5
-; CHECK-NEXT:    store i64 6, ptr [[A_SROA_32_5_OVERLAP_6_I64_SROA_IDX]], align 1, !tbaa [[TBAA19:![0-9]+]]
-; CHECK-NEXT:    [[A_SROA_32_6_OVERLAP_7_I64_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_32]], i64 6
-; CHECK-NEXT:    store i64 7, ptr [[A_SROA_32_6_OVERLAP_7_I64_SROA_IDX]], align 1, !tbaa [[TBAA21:![0-9]+]]
-; CHECK-NEXT:    [[A_SROA_32_7_OVERLAP_8_I64_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_32]], i64 7
-; CHECK-NEXT:    store i64 8, ptr [[A_SROA_32_7_OVERLAP_8_I64_SROA_IDX]], align 1, !tbaa [[TBAA23:![0-9]+]]
-; CHECK-NEXT:    [[A_SROA_32_8_OVERLAP_9_I64_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_32]], i64 8
-; CHECK-NEXT:    store i64 9, ptr [[A_SROA_32_8_OVERLAP_9_I64_SROA_IDX]], align 1, !tbaa [[TBAA25:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_32_1_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_32]], i64 1
+; CHECK-NEXT:    store i64 2, ptr [[A_SROA_32_1_SROA_IDX]], align 1, !tbaa [[TBAA11:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_32_2_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_32]], i64 2
+; CHECK-NEXT:    store i64 3, ptr [[A_SROA_32_2_SROA_IDX]], align 1, !tbaa [[TBAA13:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_32_3_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_32]], i64 3
+; CHECK-NEXT:    store i64 4, ptr [[A_SROA_32_3_SROA_IDX]], align 1, !tbaa [[TBAA15:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_32_4_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_32]], i64 4
+; CHECK-NEXT:    store i64 5, ptr [[A_SROA_32_4_SROA_IDX]], align 1, !tbaa [[TBAA17:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_32_5_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_32]], i64 5
+; CHECK-NEXT:    store i64 6, ptr [[A_SROA_32_5_SROA_IDX]], align 1, !tbaa [[TBAA19:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_32_6_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_32]], i64 6
+; CHECK-NEXT:    store i64 7, ptr [[A_SROA_32_6_SROA_IDX]], align 1, !tbaa [[TBAA21:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_32_7_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_32]], i64 7
+; CHECK-NEXT:    store i64 8, ptr [[A_SROA_32_7_SROA_IDX]], align 1, !tbaa [[TBAA23:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_32_8_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_32]], i64 8
+; CHECK-NEXT:    store i64 9, ptr [[A_SROA_32_8_SROA_IDX]], align 1, !tbaa [[TBAA25:![0-9]+]]
 ; CHECK-NEXT:    store i8 1, ptr [[A_SROA_16]], align 1, !tbaa [[TBAA27:![0-9]+]]
 ; CHECK-NEXT:    store i16 1, ptr [[A_SROA_16]], align 1, !tbaa [[TBAA29:![0-9]+]]
 ; CHECK-NEXT:    store i32 1, ptr [[A_SROA_16]], align 1, !tbaa [[TBAA31:![0-9]+]]
-; CHECK-NEXT:    [[A_SROA_16_1_OVERLAP2_1_1_I32_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_16]], i64 1
-; CHECK-NEXT:    store i32 2, ptr [[A_SROA_16_1_OVERLAP2_1_1_I32_SROA_IDX]], align 1, !tbaa [[TBAA33:![0-9]+]]
-; CHECK-NEXT:    [[A_SROA_16_2_OVERLAP2_1_2_I32_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_16]], i64 2
-; CHECK-NEXT:    store i32 3, ptr [[A_SROA_16_2_OVERLAP2_1_2_I32_SROA_IDX]], align 1, !tbaa [[TBAA35:![0-9]+]]
-; CHECK-NEXT:    [[A_SROA_16_3_OVERLAP2_1_3_I32_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_16]], i64 3
-; CHECK-NEXT:    store i32 4, ptr [[A_SROA_16_3_OVERLAP2_1_3_I32_SROA_IDX]], align 1, !tbaa [[TBAA37:![0-9]+]]
-; CHECK-NEXT:    store i32 1, ptr [[A_SROA_234]], align 1, !tbaa [[TBAA39:![0-9]+]]
-; CHECK-NEXT:    [[A_SROA_234_1_OVERLAP2_2_1_I8_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_234]], i64 1
-; CHECK-NEXT:    store i8 1, ptr [[A_SROA_234_1_OVERLAP2_2_1_I8_SROA_IDX]], align 1, !tbaa [[TBAA41:![0-9]+]]
-; CHECK-NEXT:    [[A_SROA_234_1_OVERLAP2_2_1_I16_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_234]], i64 1
-; CHECK-NEXT:    store i16 1, ptr [[A_SROA_234_1_OVERLAP2_2_1_I16_SROA_IDX]], align 1, !tbaa [[TBAA43:![0-9]+]]
-; CHECK-NEXT:    [[A_SROA_234_1_OVERLAP2_2_1_I32_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_234]], i64 1
-; CHECK-NEXT:    store i32 1, ptr [[A_SROA_234_1_OVERLAP2_2_1_I32_SROA_IDX]], align 1, !tbaa [[TBAA45:![0-9]+]]
-; CHECK-NEXT:    [[A_SROA_234_2_OVERLAP2_2_2_I32_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_234]], i64 2
-; CHECK-NEXT:    store i32 3, ptr [[A_SROA_234_2_OVERLAP2_2_2_I32_SROA_IDX]], align 1, !tbaa [[TBAA47:![0-9]+]]
-; CHECK-NEXT:    [[A_SROA_234_3_OVERLAP2_2_3_I32_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_234]], i64 3
-; CHECK-NEXT:    store i32 4, ptr [[A_SROA_234_3_OVERLAP2_2_3_I32_SROA_IDX]], align 1, !tbaa [[TBAA49:![0-9]+]]
-; CHECK-NEXT:    [[A_SROA_15_197_OVERLAP2_PREFIX_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_15]], i64 39
-; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_15_197_OVERLAP2_PREFIX_SROA_IDX]], ptr align 1 [[SRC]], i32 3, i1 false), !tbaa [[TBAA51:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_16_1_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_16]], i64 1
+; CHECK-NEXT:    store i32 2, ptr [[A_SROA_16_1_SROA_IDX]], align 1, !tbaa [[TBAA33:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_16_2_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_16]], i64 2
+; CHECK-NEXT:    store i32 3, ptr [[A_SROA_16_2_SROA_IDX]], align 1, !tbaa [[TBAA35:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_16_3_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_16]], i64 3
+; CHECK-NEXT:    store i32 4, ptr [[A_SROA_16_3_SROA_IDX]], align 1, !tbaa [[TBAA37:![0-9]+]]
+; CHECK-NEXT:    store i32 1, ptr [[A_SROA_235]], align 1, !tbaa [[TBAA39:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_235_1_SROA_IDX11:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_235]], i64 1
+; CHECK-NEXT:    store i8 1, ptr [[A_SROA_235_1_SROA_IDX11]], align 1, !tbaa [[TBAA41:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_235_1_SROA_IDX10:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_235]], i64 1
+; CHECK-NEXT:    store i16 1, ptr [[A_SROA_235_1_SROA_IDX10]], align 1, !tbaa [[TBAA43:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_235_1_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_235]], i64 1
+; CHECK-NEXT:    store i32 1, ptr [[A_SROA_235_1_SROA_IDX]], align 1, !tbaa [[TBAA45:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_235_2_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_235]], i64 2
+; CHECK-NEXT:    store i32 3, ptr [[A_SROA_235_2_SROA_IDX]], align 1, !tbaa [[TBAA47:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_235_3_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_235]], i64 3
+; CHECK-NEXT:    store i32 4, ptr [[A_SROA_235_3_SROA_IDX]], align 1, !tbaa [[TBAA49:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_15_197_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_15]], i64 39
+; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_15_197_SROA_IDX]], ptr align 1 [[SRC]], i32 3, i1 false), !tbaa [[TBAA51:![0-9]+]]
 ; CHECK-NEXT:    [[A_SROA_16_197_SRC_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i64 3
 ; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_16]], ptr align 1 [[A_SROA_16_197_SRC_SROA_IDX]], i32 5, i1 false), !tbaa [[TBAA51]]
-; CHECK-NEXT:    [[A_SROA_16_2_OVERLAP2_1_2_I8_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_16]], i64 2
-; CHECK-NEXT:    call void @llvm.memset.p0.i32(ptr align 1 [[A_SROA_16_2_OVERLAP2_1_2_I8_SROA_IDX]], i8 42, i32 5, i1 false), !tbaa [[TBAA53:![0-9]+]]
-; CHECK-NEXT:    call void @llvm.memset.p0.i32(ptr align 1 [[A_SROA_234]], i8 42, i32 2, i1 false), !tbaa [[TBAA53]]
-; CHECK-NEXT:    [[A_SROA_234_209_OVERLAP2_2_1_I8_SROA_IDX5:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_234]], i64 1
-; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_234_209_OVERLAP2_2_1_I8_SROA_IDX5]], ptr align 1 [[SRC]], i32 5, i1 false), !tbaa [[TBAA55:![0-9]+]]
-; CHECK-NEXT:    [[A_SROA_234_210_OVERLAP2_2_2_I8_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_234]], i64 2
-; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_234_210_OVERLAP2_2_2_I8_SROA_IDX]], ptr align 1 [[SRC]], i32 5, i1 false), !tbaa [[TBAA57:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_16_2_SROA_IDX12:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_16]], i64 2
+; CHECK-NEXT:    call void @llvm.memset.p0.i32(ptr align 1 [[A_SROA_16_2_SROA_IDX12]], i8 42, i32 5, i1 false), !tbaa [[TBAA53:![0-9]+]]
+; CHECK-NEXT:    call void @llvm.memset.p0.i32(ptr align 1 [[A_SROA_235]], i8 42, i32 2, i1 false), !tbaa [[TBAA53]]
+; CHECK-NEXT:    [[A_SROA_235_209_SROA_IDX8:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_235]], i64 1
+; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_235_209_SROA_IDX8]], ptr align 1 [[SRC]], i32 5, i1 false), !tbaa [[TBAA55:![0-9]+]]
+; CHECK-NEXT:    [[A_SROA_235_210_SROA_IDX9:%.*]] = getelementptr inbounds i8, ptr [[A_SROA_235]], i64 2
+; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_235_210_SROA_IDX9]], ptr align 1 [[SRC]], i32 5, i1 false), !tbaa [[TBAA57:![0-9]+]]
 ; CHECK-NEXT:    [[A_SROA_31_210_SRC_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i64 5
 ; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_31]], ptr align 1 [[A_SROA_31_210_SRC_SROA_IDX]], i32 3, i1 false), !tbaa [[TBAA57]]
 ; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[DST:%.*]], ptr align 1 [[A_SROA_0]], i32 42, i1 false), !tbaa [[TBAA59:![0-9]+]]
 ; CHECK-NEXT:    [[A_SROA_2_0_DST_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 42
 ; CHECK-NEXT:    store i8 0, ptr [[A_SROA_2_0_DST_SROA_IDX]], align 1, !tbaa [[TBAA59]]
 ; CHECK-NEXT:    [[A_SROA_3_0_DST_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 43
 ; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_3_0_DST_SROA_IDX]], ptr align 1 [[A_SROA_3]], i32 99, i1 false), !tbaa [[TBAA59]]
 ; CHECK-NEXT:    [[A_SROA_32_0_DST_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 142
 ; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_32_0_DST_SROA_IDX]], ptr align 1 [[A_SROA_32]], i32 16, i1 false), !tbaa [[TBAA59]]
 ; CHECK-NEXT:    [[A_SROA_15_0_DST_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 158
 ; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_15_0_DST_SROA_IDX]], ptr align 1 [[A_SROA_15]], i32 42, i1 false), !tbaa [[TBAA59]]
 ; CHECK-NEXT:    [[A_SROA_16_0_DST_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 200
 ; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_16_0_DST_SROA_IDX]], ptr align 1 [[A_SROA_16]], i32 7, i1 false), !tbaa [[TBAA59]]
 ; CHECK-NEXT:    [[A_SROA_23_0_DST_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 207
 ; CHECK-NEXT:    store i8 42, ptr [[A_SROA_23_0_DST_SROA_IDX]], align 1, !tbaa [[TBAA59]]
-; CHECK-NEXT:    [[A_SROA_234_0_DST_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 208
-; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_234_0_DST_SROA_IDX]], ptr align 1 [[A_SROA_234]], i32 7, i1 false), !tbaa [[TBAA59]]
+; CHECK-NEXT:    [[A_SROA_235_0_DST_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 208
+; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_235_0_DST_SROA_IDX]], ptr align 1 [[A_SROA_235]], i32 7, i1 false), !tbaa [[TBAA59]]
 ; CHECK-NEXT:    [[A_SROA_31_0_DST_SROA_IDX:%.*]] = getelementptr inbounds i8, ptr [[DST]], i64 215
 ; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr align 1 [[A_SROA_31_0_DST_SROA_IDX]], ptr align 1 [[A_SROA_31]], i32 85, i1 false), !tbaa [[TBAA59]]
 ; CHECK-NEXT:    ret void
 ;
 
 entry:
   %a = alloca [300 x i8]
 

 }
 
 define i8 @test12() {
 ; We fully promote these to the i24 load or store size, resulting in just masks
 ; and other operations that instcombine will fold, but no alloca.
 ;
 ; CHECK-LABEL: @test12(
 ; CHECK-NEXT:  entry:
-; CHECK-NEXT:    [[A_SROA_3_0_INSERT_EXT:%.*]] = zext i8 0 to i24
-; CHECK-NEXT:    [[A_SROA_3_0_INSERT_SHIFT:%.*]] = shl i24 [[A_SROA_3_0_INSERT_EXT]], 16
-; CHECK-NEXT:    [[A_SROA_3_0_INSERT_MASK:%.*]] = and i24 undef, 65535
-; CHECK-NEXT:    [[A_SROA_3_0_INSERT_INSERT:%.*]] = or i24 [[A_SROA_3_0_INSERT_MASK]], [[A_SROA_3_0_INSERT_SHIFT]]
-; CHECK-NEXT:    [[A_SROA_2_0_INSERT_EXT:%.*]] = zext i8 0 to i24
-; CHECK-NEXT:    [[A_SROA_2_0_INSERT_SHIFT:%.*]] = shl i24 [[A_SROA_2_0_INSERT_EXT]], 8
-; CHECK-NEXT:    [[A_SROA_2_0_INSERT_MASK:%.*]] = and i24 [[A_SROA_3_0_INSERT_INSERT]], -65281
-; CHECK-NEXT:    [[A_SROA_2_0_INSERT_INSERT:%.*]] = or i24 [[A_SROA_2_0_INSERT_MASK]], [[A_SROA_2_0_INSERT_SHIFT]]
-; CHECK-NEXT:    [[A_SROA_0_0_INSERT_EXT:%.*]] = zext i8 0 to i24
-; CHECK-NEXT:    [[A_SROA_0_0_INSERT_MASK:%.*]] = and i24 [[A_SROA_2_0_INSERT_INSERT]], -256
-; CHECK-NEXT:    [[A_SROA_0_0_INSERT_INSERT:%.*]] = or i24 [[A_SROA_0_0_INSERT_MASK]], [[A_SROA_0_0_INSERT_EXT]]
-; CHECK-NEXT:    [[B_SROA_0_0_EXTRACT_TRUNC:%.*]] = trunc i24 [[A_SROA_0_0_INSERT_INSERT]] to i8
-; CHECK-NEXT:    [[B_SROA_2_0_EXTRACT_SHIFT:%.*]] = lshr i24 [[A_SROA_0_0_INSERT_INSERT]], 8
+; CHECK-NEXT:    [[A_SROA_0_0_INSERT_MASK:%.*]] = and i24 undef, -256
+; CHECK-NEXT:    [[A_SROA_0_1_INSERT_MASK:%.*]] = and i24 [[A_SROA_0_0_INSERT_MASK]], -65281
+; CHECK-NEXT:    [[A_SROA_0_2_INSERT_MASK:%.*]] = and i24 [[A_SROA_0_1_INSERT_MASK]], 65535
+; CHECK-NEXT:    [[B_SROA_0_0_EXTRACT_TRUNC:%.*]] = trunc i24 [[A_SROA_0_2_INSERT_MASK]] to i8
+; CHECK-NEXT:    [[B_SROA_2_0_EXTRACT_SHIFT:%.*]] = lshr i24 [[A_SROA_0_2_INSERT_MASK]], 8
 ; CHECK-NEXT:    [[B_SROA_2_0_EXTRACT_TRUNC:%.*]] = trunc i24 [[B_SROA_2_0_EXTRACT_SHIFT]] to i8
-; CHECK-NEXT:    [[B_SROA_3_0_EXTRACT_SHIFT:%.*]] = lshr i24 [[A_SROA_0_0_INSERT_INSERT]], 16
+; CHECK-NEXT:    [[B_SROA_3_0_EXTRACT_SHIFT:%.*]] = lshr i24 [[A_SROA_0_2_INSERT_MASK]], 16
 ; CHECK-NEXT:    [[B_SROA_3_0_EXTRACT_TRUNC:%.*]] = trunc i24 [[B_SROA_3_0_EXTRACT_SHIFT]] to i8
 ; CHECK-NEXT:    [[BSUM0:%.*]] = add i8 [[B_SROA_0_0_EXTRACT_TRUNC]], [[B_SROA_2_0_EXTRACT_TRUNC]]
 ; CHECK-NEXT:    [[BSUM1:%.*]] = add i8 [[BSUM0]], [[B_SROA_3_0_EXTRACT_TRUNC]]
 ; CHECK-NEXT:    ret i8 [[BSUM1]]
 ;
 
 entry:
   %a = alloca [3 x i8]

 ; CHECK-NEXT:    i32 4, label [[BB4:%.*]]
 ; CHECK-NEXT:    i32 3, label [[BB3:%.*]]
 ; CHECK-NEXT:    i32 2, label [[BB2:%.*]]
 ; CHECK-NEXT:    i32 1, label [[BB1:%.*]]
 ; CHECK-NEXT:    ]
 ; CHECK:       bb4:
 ; CHECK-NEXT:    [[SRC_GEP3:%.*]] = getelementptr inbounds i8, ptr [[SRC:%.*]], i32 3
 ; CHECK-NEXT:    [[SRC_3:%.*]] = load i8, ptr [[SRC_GEP3]], align 1
-; CHECK-NEXT:    [[TMP_SROA_0_3_TMP_GEP3_SROA_IDX3:%.*]] = getelementptr inbounds i8, ptr [[TMP_SROA_0]], i64 3
-; CHECK-NEXT:    store i8 [[SRC_3]], ptr [[TMP_SROA_0_3_TMP_GEP3_SROA_IDX3]], align 1
+; CHECK-NEXT:    [[TMP_SROA_0_3_SROA_IDX3:%.*]] = getelementptr inbounds i8, ptr [[TMP_SROA_0]], i64 3
+; CHECK-NEXT:    store i8 [[SRC_3]], ptr [[TMP_SROA_0_3_SROA_IDX3]], align 1
 ; CHECK-NEXT:    br label [[BB3]]
 ; CHECK:       bb3:
 ; CHECK-NEXT:    [[SRC_GEP2:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i32 2
 ; CHECK-NEXT:    [[SRC_2:%.*]] = load i8, ptr [[SRC_GEP2]], align 1
-; CHECK-NEXT:    [[TMP_SROA_0_2_TMP_GEP2_SROA_IDX2:%.*]] = getelementptr inbounds i8, ptr [[TMP_SROA_0]], i64 2
-; CHECK-NEXT:    store i8 [[SRC_2]], ptr [[TMP_SROA_0_2_TMP_GEP2_SROA_IDX2]], align 2
+; CHECK-NEXT:    [[TMP_SROA_0_2_SROA_IDX2:%.*]] = getelementptr inbounds i8, ptr [[TMP_SROA_0]], i64 2
+; CHECK-NEXT:    store i8 [[SRC_2]], ptr [[TMP_SROA_0_2_SROA_IDX2]], align 2
 ; CHECK-NEXT:    br label [[BB2]]
 ; CHECK:       bb2:
 ; CHECK-NEXT:    [[SRC_GEP1:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i32 1
 ; CHECK-NEXT:    [[SRC_1:%.*]] = load i8, ptr [[SRC_GEP1]], align 1
-; CHECK-NEXT:    [[TMP_SROA_0_1_TMP_GEP1_SROA_IDX1:%.*]] = getelementptr inbounds i8, ptr [[TMP_SROA_0]], i64 1
-; CHECK-NEXT:    store i8 [[SRC_1]], ptr [[TMP_SROA_0_1_TMP_GEP1_SROA_IDX1]], align 1
+; CHECK-NEXT:    [[TMP_SROA_0_1_SROA_IDX1:%.*]] = getelementptr inbounds i8, ptr [[TMP_SROA_0]], i64 1
+; CHECK-NEXT:    store i8 [[SRC_1]], ptr [[TMP_SROA_0_1_SROA_IDX1]], align 1
 ; CHECK-NEXT:    br label [[BB1]]
 ; CHECK:       bb1:
 ; CHECK-NEXT:    [[SRC_GEP0:%.*]] = getelementptr inbounds i8, ptr [[SRC]], i32 0
 ; CHECK-NEXT:    [[SRC_0:%.*]] = load i8, ptr [[SRC_GEP0]], align 1
 ; CHECK-NEXT:    store i8 [[SRC_0]], ptr [[TMP_SROA_0]], align 4
 ; CHECK-NEXT:    br label [[END]]
 ; CHECK:       end:
 ; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr [[DATA:%.*]], ptr align 4 [[TMP_SROA_0]], i32 [[SIZE]], i1 false)

llvm/test/Transforms/SROA/basictest.ll

 }
 
 define i8 @test12() {
 ; We fully promote these to the i24 load or store size, resulting in just masks
 ; and other operations that instcombine will fold, but no alloca.
 ;
 ; CHECK-LABEL: @test12(
 ; CHECK-NEXT:  entry:
-; CHECK-NEXT:    [[A_SROA_3_0_INSERT_EXT:%.*]] = zext i8 0 to i24
-; CHECK-NEXT:    [[A_SROA_3_0_INSERT_SHIFT:%.*]] = shl i24 [[A_SROA_3_0_INSERT_EXT]], 16
-; CHECK-NEXT:    [[A_SROA_3_0_INSERT_MASK:%.*]] = and i24 undef, 65535
-; CHECK-NEXT:    [[A_SROA_3_0_INSERT_INSERT:%.*]] = or i24 [[A_SROA_3_0_INSERT_MASK]], [[A_SROA_3_0_INSERT_SHIFT]]
-; CHECK-NEXT:    [[A_SROA_2_0_INSERT_EXT:%.*]] = zext i8 0 to i24
-; CHECK-NEXT:    [[A_SROA_2_0_INSERT_SHIFT:%.*]] = shl i24 [[A_SROA_2_0_INSERT_EXT]], 8
-; CHECK-NEXT:    [[A_SROA_2_0_INSERT_MASK:%.*]] = and i24 [[A_SROA_3_0_INSERT_INSERT]], -65281
-; CHECK-NEXT:    [[A_SROA_2_0_INSERT_INSERT:%.*]] = or i24 [[A_SROA_2_0_INSERT_MASK]], [[A_SROA_2_0_INSERT_SHIFT]]
-; CHECK-NEXT:    [[A_SROA_0_0_INSERT_EXT:%.*]] = zext i8 0 to i24
-; CHECK-NEXT:    [[A_SROA_0_0_INSERT_MASK:%.*]] = and i24 [[A_SROA_2_0_INSERT_INSERT]], -256
-; CHECK-NEXT:    [[A_SROA_0_0_INSERT_INSERT:%.*]] = or i24 [[A_SROA_0_0_INSERT_MASK]], [[A_SROA_0_0_INSERT_EXT]]
-; CHECK-NEXT:    [[B_SROA_0_0_EXTRACT_TRUNC:%.*]] = trunc i24 [[A_SROA_0_0_INSERT_INSERT]] to i8
-; CHECK-NEXT:    [[B_SROA_2_0_EXTRACT_SHIFT:%.*]] = lshr i24 [[A_SROA_0_0_INSERT_INSERT]], 8
+; CHECK-NEXT:    [[A_SROA_0_0_INSERT_MASK:%.*]] = and i24 undef, -256
+; CHECK-NEXT:    [[A_SROA_0_1_INSERT_MASK:%.*]] = and i24 [[A_SROA_0_0_INSERT_MASK]], -65281
+; CHECK-NEXT:    [[A_SROA_0_2_INSERT_MASK:%.*]] = and i24 [[A_SROA_0_1_INSERT_MASK]], 65535
+; CHECK-NEXT:    [[B_SROA_0_0_EXTRACT_TRUNC:%.*]] = trunc i24 [[A_SROA_0_2_INSERT_MASK]] to i8
+; CHECK-NEXT:    [[B_SROA_2_0_EXTRACT_SHIFT:%.*]] = lshr i24 [[A_SROA_0_2_INSERT_MASK]], 8
 ; CHECK-NEXT:    [[B_SROA_2_0_EXTRACT_TRUNC:%.*]] = trunc i24 [[B_SROA_2_0_EXTRACT_SHIFT]] to i8
-; CHECK-NEXT:    [[B_SROA_3_0_EXTRACT_SHIFT:%.*]] = lshr i24 [[A_SROA_0_0_INSERT_INSERT]], 16
+; CHECK-NEXT:    [[B_SROA_3_0_EXTRACT_SHIFT:%.*]] = lshr i24 [[A_SROA_0_2_INSERT_MASK]], 16
 ; CHECK-NEXT:    [[B_SROA_3_0_EXTRACT_TRUNC:%.*]] = trunc i24 [[B_SROA_3_0_EXTRACT_SHIFT]] to i8
 ; CHECK-NEXT:    [[BSUM0:%.*]] = add i8 [[B_SROA_0_0_EXTRACT_TRUNC]], [[B_SROA_2_0_EXTRACT_TRUNC]]
 ; CHECK-NEXT:    [[BSUM1:%.*]] = add i8 [[BSUM0]], [[B_SROA_3_0_EXTRACT_TRUNC]]
 ; CHECK-NEXT:    ret i8 [[BSUM1]]
 ;
 
 entry:
   %a = alloca [3 x i8]

llvm/test/Transforms/SROA/big-endian.ll

; CHECK-NOT: alloca
   %a1ptr = getelementptr [3 x i8], [3 x i8]* %a, i64 0, i32 1
   store i8 0, i8* %a1ptr
   %a2ptr = getelementptr [3 x i8], [3 x i8]* %a, i64 0, i32 2
   store i8 0, i8* %a2ptr
   %aiptr = bitcast [3 x i8]* %a to i24*
   %ai = load i24, i24* %aiptr
 ; CHECK-NOT: store
 ; CHECK-NOT: load
-; CHECK:      %[[ext2:.*]] = zext i8 0 to i24
-; CHECK-NEXT: %[[mask2:.*]] = and i24 undef, -256
-; CHECK-NEXT: %[[insert2:.*]] = or i24 %[[mask2]], %[[ext2]]
-; CHECK-NEXT: %[[ext1:.*]] = zext i8 0 to i24
-; CHECK-NEXT: %[[shift1:.*]] = shl i24 %[[ext1]], 8
-; CHECK-NEXT: %[[mask1:.*]] = and i24 %[[insert2]], -65281
-; CHECK-NEXT: %[[insert1:.*]] = or i24 %[[mask1]], %[[shift1]]
-; CHECK-NEXT: %[[ext0:.*]] = zext i8 0 to i24
-; CHECK-NEXT: %[[shift0:.*]] = shl i24 %[[ext0]], 16
-; CHECK-NEXT: %[[mask0:.*]] = and i24 %[[insert1]], 65535
-; CHECK-NEXT: %[[insert0:.*]] = or i24 %[[mask0]], %[[shift0]]
+; CHECK:    %[[insert0:.*]] = and i24 undef, 65535
+; CHECK-NEXT:    %[[insert1:.*]] = and i24 %[[insert0]], -65281
+; CHECK-NEXT:    %[[insert2:.*]] = and i24 %[[insert1]], -256
+
+
 
   %biptr = bitcast [3 x i8]* %b to i24*
   store i24 %ai, i24* %biptr
   %b0ptr = getelementptr [3 x i8], [3 x i8]* %b, i64 0, i32 0
   %b0 = load i8, i8* %b0ptr
   %b1ptr = getelementptr [3 x i8], [3 x i8]* %b, i64 0, i32 1
   %b1 = load i8, i8* %b1ptr
   %b2ptr = getelementptr [3 x i8], [3 x i8]* %b, i64 0, i32 2
   %b2 = load i8, i8* %b2ptr
 ; CHECK-NOT: store
 ; CHECK-NOT: load
-; CHECK:      %[[shift0:.*]] = lshr i24 %[[insert0]], 16
+; CHECK:      %[[shift0:.*]] = lshr i24 %[[insert2]], 16
 ; CHECK-NEXT: %[[trunc0:.*]] = trunc i24 %[[shift0]] to i8
-; CHECK-NEXT: %[[shift1:.*]] = lshr i24 %[[insert0]], 8
+; CHECK-NEXT: %[[shift1:.*]] = lshr i24 %[[insert2]], 8
 ; CHECK-NEXT: %[[trunc1:.*]] = trunc i24 %[[shift1]] to i8
-; CHECK-NEXT: %[[trunc2:.*]] = trunc i24 %[[insert0]] to i8
+; CHECK-NEXT: %[[trunc2:.*]] = trunc i24 %[[insert2]] to i8
 
   %bsum0 = add i8 %b0, %b1
   %bsum1 = add i8 %bsum0, %b2
   ret i8 %bsum1
 ; CHECK:      %[[sum0:.*]] = add i8 %[[trunc0]], %[[trunc1]]
 ; CHECK-NEXT: %[[sum1:.*]] = add i8 %[[sum0]], %[[trunc2]]
 ; CHECK-NEXT: ret i8 %[[sum1]]
 }
 
 define i64 @test2() {
 ; Test for various mixed sizes of integer loads and stores all getting
 ; promoted.
 ;
 ; CHECK-LABEL: @test2(
+; CHECK-NEXT:  entry:
 
 entry:
   %a = alloca [7 x i8]
 ; CHECK-NOT: alloca
 
   %a0ptr = getelementptr [7 x i8], [7 x i8]* %a, i64 0, i32 0
   %a1ptr = getelementptr [7 x i8], [7 x i8]* %a, i64 0, i32 1
   %a2ptr = getelementptr [7 x i8], [7 x i8]* %a, i64 0, i32 2
   %a3ptr = getelementptr [7 x i8], [7 x i8]* %a, i64 0, i32 3
 
 ; CHECK-NOT: store
 ; CHECK-NOT: load
 
   %a0i16ptr = bitcast i8* %a0ptr to i16*
   store i16 1, i16* %a0i16ptr
 
+; set a[0] to 1
+; Constant value is 0xFF FFFF FFFF in hexadecimal.
+; CHECK-NEXT:    [[mask0:%.*]] = and i56 undef, 1099511627775
+; Constant value is 0x100 0000 0000 in hexadecimal.
+; CHECK-NEXT:    [[insert0:%.*]] = or i56 [[mask0]], 1099511627776
+
   store i8 1, i8* %a2ptr
 
+; set a[2] to 1
+; Constant value is 0xFF 0000 0001 in hexadecimal.
+; CHECK-NEXT:    [[mask1:%.*]] = and i56 [[insert0]], -1095216660481
+; Constant value is 0x1 0000 0000 in hexadecimal.
+; CHECK-NEXT:    [[insert2:%.*]] = or i56 [[mask1]], 4294967296
+
   %a3i24ptr = bitcast i8* %a3ptr to i24*
   store i24 1, i24* %a3i24ptr
 
+; mask to get a[3] to a[5]
+; Constant value is 0xFFFF FF01 in hexadecimal.
+; CHECK-NEXT:    [[mask3:%.*]] = and i56 [[insert2]], -4294967041
+; set a[3] to 1, a[4] to 0 and a[5] to 0
+; 256 is 0x100 in hexadecimal.
+; CHECK-NEXT:    [[insert3:%.*]] = or i56 [[mask3]], 256
+
   %a2i40ptr = bitcast i8* %a2ptr to i40*
   store i40 1, i40* %a2i40ptr
+; Constant is 0x100 0000 0000 in hexadecimal.
+; set a[6] to 1, and overwrite a[2] to a[5] as 0.
+; CHECK-NEXT:    [[mask2:%.*]] = and i56 [[insert3]], -1099511627776
+; CHECK-NEXT:    [[insert4:%.*]] = or i56 [[mask2]], 1
+
+; the alloca is rewritten into i56
 
-; the alloca is splitted into multiple slices
-; Here, i8 1 is for %a[6]
-; CHECK: %[[ext1:.*]] = zext i8 1 to i40
-; CHECK-NEXT: %[[mask1:.*]] = and i40 undef, -256
-; CHECK-NEXT: %[[insert1:.*]] = or i40 %[[mask1]], %[[ext1]]
-
-; Here, i24 0 is for %a[3] to %a[5]
-; CHECK-NEXT: %[[ext2:.*]] = zext i24 0 to i40
-; CHECK-NEXT: %[[shift2:.*]] = shl i40 %[[ext2]], 8
-; CHECK-NEXT: %[[mask2:.*]] = and i40 %[[insert1]], -4294967041
-; CHECK-NEXT: %[[insert2:.*]] = or i40 %[[mask2]], %[[shift2]]
-
-; Here, i8 0 is for %a[2]
-; CHECK-NEXT: %[[ext3:.*]] = zext i8 0 to i40
-; CHECK-NEXT: %[[shift3:.*]] = shl i40 %[[ext3]], 32
-; CHECK-NEXT: %[[mask3:.*]] = and i40 %[[insert2]], 4294967295
-; CHECK-NEXT: %[[insert3:.*]] = or i40 %[[mask3]], %[[shift3]]
-
-; CHECK-NEXT: %[[ext4:.*]] = zext i40 %[[insert3]] to i56
-; CHECK-NEXT: %[[mask4:.*]] = and i56 undef, -1099511627776
-; CHECK-NEXT: %[[insert4:.*]] = or i56 %[[mask4]], %[[ext4]]
 
-; CHECK-NOT: store
-; CHECK-NOT: load
 
   %aiptr = bitcast [7 x i8]* %a to i56*
   %ai = load i56, i56* %aiptr
   %ret = zext i56 %ai to i64
   ret i64 %ret
-; Here, i16 1 is for %a[0] to %a[1]
-; CHECK-NEXT: %[[ext5:.*]] = zext i16 1 to i56
-; CHECK-NEXT: %[[shift5:.*]] = shl i56 %[[ext5]], 40
-; CHECK-NEXT: %[[mask5:.*]] = and i56 %[[insert4]], 1099511627775
-; CHECK-NEXT: %[[insert5:.*]] = or i56 %[[mask5]], %[[shift5]]
-; CHECK-NEXT: %[[ret:.*]] = zext i56 %[[insert5]] to i64
-; CHECK-NEXT: ret i64 %[[ret]]
+; CHECK-NEXT:    [[RET:%.*]] = zext i56 [[insert4]] to i64
+; CHECK-NEXT:    ret i64 [[RET]]
 }
 
 define i64 @PR14132(i1 %flag) {
 ; CHECK-LABEL: @PR14132(
 ; Here we form a PHI-node by promoting the pointer alloca first, and then in
 ; order to promote the other two allocas, we speculate the load of the
 ; now-phi-node-pointer. In doing so we end up loading a 64-bit value from an i8
 ; alloca. While this is a bit dubious, we were asserting on trying to