[SROA] Maintain shadow/backing alloca when some slices are noncapturnig read-only calls to allow alloca partitioning/promotion

[lebedev.ri]

Summary

This is inspired by the original variant of D109749 by Graham Hunter,
but is a more general version.

Roughly, instead of promoting the alloca, we call it a shadow/backing alloca,
go through all it's slices, clone(!) instructions that operated on it,
but make them operate on the cloned alloca, and promote cloned alloca instead.

This keeps the shadow/backing alloca, and all the original instructions around,
which results in said shadow/backing alloca being a perfect mirror/representation
of the promoted alloca's content, so calls that take the alloca
as arguments (non-capturingly!) can be supported.

For now, we require that the calls also don't modify the alloca's content,
but that is only to simplify the initial implementation,
and that will be supported in a follow-up.

Overall, this leads to *smaller* codesize:
https://llvm-compile-time-tracker.com/compare.php?from=a8b4f5bbab62091835205f3d648902432a4a5b58&to=aeae054055b125b011c1122f82c86457e159436f&stat=size-total
and is roughly neutral compile-time wise:
https://llvm-compile-time-tracker.com/compare.php?from=a8b4f5bbab62091835205f3d648902432a4a5b58&to=aeae054055b125b011c1122f82c86457e159436f&stat=instructions

[jdoerfert]

This is really cool. There is a FIXME, are you expecting to address this? Other than that, reading through it seems sensible to me.

llvm/lib/Transforms/Scalar/CMakeLists.txt

101 ↗

(On Diff #385987)

leftover, I assume

[lebedev.ri]

Some assortment of small improvements.

In D113520#3119995, @jdoerfert wrote:

This is really cool.

Yep.

There is a FIXME, are you expecting to address this? Other than that, reading through it seems sensible to me.

I've added run line with -opaque-pointers, and everything appears to just work.
@nikic - can you confirm that this is fine?

llvm/lib/Transforms/Scalar/CMakeLists.txt

101 ↗

(On Diff #385987)

yep.

[lebedev.ri]

llvm/test/Transforms/SROA/non-capturing-call.ll

667 ↗

(On Diff #386102)

We fail to drop some dead instructions, which prevents deletion of the alloca itself.
Not yet sure how to deal with this.

[lebedev.ri]

In D113520#3120892, @lebedev.ri wrote:

Some assortment of small improvements.

In D113520#3119995, @jdoerfert wrote:

This is really cool.

Yep.

There is a FIXME, are you expecting to address this? Other than that, reading through it seems sensible to me.

I've added run line with -opaque-pointers, and everything appears to just work.
@nikic - can you confirm that this is fine?

Actually thinking about it, that is obviously right,
because what opaque pointers disallow is *extracting*
the pointee type out of the pointer type,
and here we only get/use the whole pointer type,
so it doesn't matter if it contains the pointee knowledge or not.

[lebedev.ri]

Ok, it turned out that we do need to delete no-longer-used instructions,
the added test would otherwise assert as not being promoteable.
Also, don't perform this fixup if there are no non-escaping uses of alloca.

There is at least one more problem, but i'm not sure what it is yet.

[lebedev.ri]

Introduce profitability heuristic so that the compiler doesn't take out
the whole machine while trying to build e.g. vanilla llvm test-suite :)

The current rule-of-thumb is:
For an escaped alloca to be worthy of promotion,
it must be used by loads/stores in this function (i.e., no point in promoting https://godbolt.org/z/c51Yavv9P)
and either have a small total size (32 bytes currently,
very much a guess), or most of the bytes of the alloca
(>=80%, also a guess) must be loaded/stored within this function (i.e. not much point in promoting https://godbolt.org/z/7KToqzb6z)

This results in these numbers:
https://llvm-compile-time-tracker.com/compare.php?from=8d35c054e31e5a2bee082f7a587a660eeb24bf99&to=c7226d770b1ffa1117f724dd25de7fa2881eed18&stat=instructions
... and i'm not sure what is going on with tramp3d-v4's NewPM-ReleaseThinLTO build,
i can't reproduce the horrible slowdown locally (do i need a production debug/assert-less build?).

This transform does not do anything crazy on that code,
the alloca's that happen to be promoted are very tiny (<=32b),
and we introduce only a single spill/reload per each,
so this isn't something i see as preventable with profitability checks.

[xbolva00]

tramp3d-v4 - codesize increased by 4% so more inlining(?), more IR, slower build.

[lebedev.ri]

In D113520#3129441, @lebedev.ri wrote:

Introduce profitability heuristic so that the compiler doesn't take out
the whole machine while trying to build e.g. vanilla llvm test-suite :)

Fixed the "occupancy" to only consider bytes that are both loaded and stored.
After all, we are mostly interested in allowing mem2reg,
aka, avoiding using stack instead of registers, but if not a single byte
is eligible, then we don't really care. Notably, the current check is *still*
too soft, as the last test shows.

This results in these numbers:
https://llvm-compile-time-tracker.com/compare.php?from=8d35c054e31e5a2bee082f7a587a660eeb24bf99&to=c7226d770b1ffa1117f724dd25de7fa2881eed18&stat=instructions
... and i'm not sure what is going on with tramp3d-v4's NewPM-ReleaseThinLTO build,
i can't reproduce the horrible slowdown locally (do i need a production debug/assert-less build?).

... and now we get https://llvm-compile-time-tracker.com/compare.php?from=2c91f48c48c42f9bd730d0791fcb19dbe0038d96&to=b4282b9b3cf6ebe17938a4041af21681b762b881&stat=instructions,
and the horrible tramp3d-v4 regression is gone :)

This is mostly the point where i'd want some feedback.

[lebedev.ri]

Ping.
I'm not really sure who is comfortable/okay with reviewing SROA changes nowadays, so the CC list is a bit too large.

[lebedev.ri]

Did some perf testing on RawSpeed, while the results will be heavily workload-depend,
here they seem to be mostly positively neutral with some improvements,
and no heavy losses.

rsbench.log114 KBDownload

[lebedev.ri]

ping

[reames]

Spent a decent amount of time looking at this, but haven't yet fully formed a recommendation. I'm going to summarize my findings to date in the hopes this is useful.

I think the basic idea here is valuable, but as implemented, I don't think this is a good idea. The current implementation can result in a serious pessimization in cases like the following:

a = alloca <4 x i8> ... // assume a is otherwise splittable
<init a to zero>
loop {
   if (very_rare) a[3] = 5;
   foo(a);
}

This transformation will have the effect of rewriting this as:

a = alloca <4 x i8> // all other uses of a split and ssa constructed
loop {
   *a =  very_rate ? <0,0,0,5> : <0,0,0,0> 
   foo(a);
}

The result is that we've made the store to a dramatically more expensive by moving it into the hotpath.

I think this is a fatal flaw with the approach. We could maybe patch around it with some profiling data, but in general, we don't want to be moving init logic.

However, this is where I get struck. I don't really have a suggestion on how to implement this better. Conceptually, what we want is something along the lines of D113289, but applied to SROA. That is, doing SSA formation to eliminate loads while leaving the stores in place for use by the calls which reads the allocas. However, there are two problems with that:

1. If we want to support non-readonly calls, we'd need to materialize a load after the call. This has the same "move to hot" problem as above. Maybe we can just restrict it to readonly arguments?
2. This really doesn't cleanly fit into the code structure of SROA. While SROA indirectly does SSA construction, it does so by forming another alloca, and then mem2regging that. We'd have to figure out something analogous to mem2reg which didn't remove the stores or the allocas.

Purely in terms of the code structure of the current patch, the integration of this is weird. The remat transform should be done over the AllocaSlices array after initially scanned, and should directly rebuild that data structure rather than relying on the code to handle duplicates in the array and double scanning. That doesn't address the algorithmic concern above though, so this ends up being mostly an aside.

This might be helpful to brainstorm offline. Let me know if you'd like to chat.

[lebedev.ri]

In D113520#3162304, @reames wrote:

Spent a decent amount of time looking at this, but haven't yet fully formed a recommendation. I'm going to summarize my findings to date in the hopes this is useful.

Thank you for taking a look!

I think the basic idea here is valuable, but as implemented, I don't think this is a good idea. The current implementation can result in a serious pessimization in cases like the following:

a = alloca <4 x i8> ... // assume a is otherwise splittable
<init a to zero>
loop {
   if (very_rare) a[3] = 5;
   foo(a);
}

This transformation will have the effect of rewriting this as:

a = alloca <4 x i8> // all other uses of a split and ssa constructed
loop {
   *a =  very_rate ? <0,0,0,5> : <0,0,0,0> 
   foo(a);
}

The result is that we've made the store to a dramatically more expensive by moving it into the hotpath.

I think this is a fatal flaw with the approach. We could maybe patch around it with some profiling data, but in general, we don't want to be moving init logic.

Indeed: https://godbolt.org/z/n64ozqnsT

However, this is where I get struck. I don't really have a suggestion on how to implement this better.

Conceptually, what we want is something along the lines of D113289, but applied to SROA.

Right. Not sure how much relevant potential is left after that change, to be noted.

That is, doing SSA formation to eliminate loads while leaving the stores in place for use by the calls which reads the allocas. However, there are two problems with that:

1. If we want to support non-readonly calls, we'd need to materialize a load after the call. This has the same "move to hot" problem as above.

True.

Maybe we can just restrict it to readonly arguments?

2. This really doesn't cleanly fit into the code structure of SROA. While SROA indirectly does SSA construction, it does so by forming another alloca, and then mem2regging that. We'd have to figure out something analogous to mem2reg which didn't remove the stores or the allocas.

Well, this one seems straight-forward - instead of spilling before the call, just go through all the slices,
and duplicate each store to store into our new alloca, this will essentially preserve all the original stores.

Purely in terms of the code structure of the current patch, the integration of this is weird. The remat transform should be done over the AllocaSlices array after initially scanned, and should directly rebuild that data structure rather than relying on the code to handle duplicates in the array and double scanning. That doesn't address the algorithmic concern above though, so this ends up being mostly an aside.

Note that i only scan the newly-added uses, i do not rescan everything, so i don't believe this particular concern is significant.

This might be helpful to brainstorm offline. Let me know if you'd like to chat.

[lebedev.ri]

I think the obvious solution is:

1. Keep all stores. (in terms of this patch, iterate over all slices and duplicate all store instructions to also store into the cloned alloca)
2. Keep all loads (in terms of this patch, iterate over all slices and before the load from the original alloca, load from the cloned alloca and store into the original alloca) UNLESS we can omit particular load because we can tell that there was no taint (may-write calls) on every path from the every previous store.

[reames]

@lebedev.ri Your suggested approach makes sense to me at a basic level. Forming the alloca is starting to seem more and more like a hack, but we can come back to implementing partial mem2reg as a follow on code improvement.

A couple of suggestions:

• Please leave taint tracking to later patch. Start with the requirement that the non-escaping call use must also be readonly in that argument.
• Instead of having the cloned alloca be the one kept, have it be the one which gets mem2reged. That is, duplicate all the uses (except the escaping call) and let the code handle it. The primary point here is just to keep the naming in the IR more obvious. :)

[lebedev.ri]

In D113520#3164812, @reames wrote:

@lebedev.ri Your suggested approach makes sense to me at a basic level. Forming the alloca is starting to seem more and more like a hack, but we can come back to implementing partial mem2reg as a follow on code improvement.

A couple of suggestions:

• Please leave taint tracking to later patch. Start with the requirement that the non-escaping call use must also be readonly in that argument.

That was my plan, yes.

• Instead of having the cloned alloca be the one kept, have it be the one which gets mem2reged. That is, duplicate all the uses (except the escaping call) and let the code handle it. The primary point here is just to keep the naming in the IR more obvious. :)

Yeah, i was wondering about that, but didn't do that yet.

[lebedev.ri]

Finally managed to subdue crippling fear of the potential complexity
of the alternative design, only to discover it to be rather simple :)

In D113520#3164812, @reames wrote:

@lebedev.ri Your suggested approach makes sense to me at a basic level. Forming the alloca is starting to seem more and more like a hack, but we can come back to implementing partial mem2reg as a follow on code improvement.

A couple of suggestions:

• Please leave taint tracking to later patch. Start with the requirement that the non-escaping call use must also be readonly in that argument.
• Instead of having the cloned alloca be the one kept, have it be the one which gets mem2reged. That is, duplicate all the uses (except the escaping call) and let the code handle it. The primary point here is just to keep the naming in the IR more obvious. :)

Done, PTAL :)

[lebedev.ri]

ping @reames
thanks

[djtodoro]

The idea behind this seems very interesting/valuable, thanks for this!

I am wondering if you are able to get some benchmark data (SPEC, or phoronix) for this? IIUC, the code-size will be increased, so is that size reasonable?

llvm/lib/Transforms/Scalar/SROA.cpp

3632

Should we add a TODO here for clarifying that there is a plan to support calls that do modify the alloca's content?

[lebedev.ri]

Thanks for taking a look!

In D113520#3356615, @djtodoro wrote:

The idea behind this seems very interesting/valuable, thanks for this!

I am wondering if you are able to get some benchmark data (SPEC, or phoronix) for this? IIUC, the code-size will be increased, so is that size reasonable?

I suppose the LICM change, that came after this was initially posted,
has rendered this being less crucial than it would be,
but important nonetheless.

Size-wise, this is actually a win i would say:
https://llvm-compile-time-tracker.com/compare.php?from=a8b4f5bbab62091835205f3d648902432a4a5b58&to=aeae054055b125b011c1122f82c86457e159436f&stat=size-total

[djtodoro]

LGTM for my side, but please wait a few days to see if someone has some additional comments (although it looks like all the comments are already addressed).

llvm/lib/Transforms/Scalar/SROA.cpp

278

I guess ; is unintentional here.

[lebedev.ri]

In D113520#3357128, @djtodoro wrote:

LGTM for my side, but please wait a few days to see if someone has some additional comments (although it looks like all the comments are already addressed).

Thank you!

I'm hoping @reames would find time to re-review this, but given that
we've settled on the approach, and it's rather straight-forward,
i don't know how long i should wait indeed.

[lebedev.ri]

@reames i'm intending to land this in 24 hours if there are no further comments.

[lebedev.ri]

Ok, i'm not yet sure how to deal with this, but somehow we end up trying to re-promote the alloca's here:

; Function Attrs: argmemonly nofree nounwind willreturn
declare void @llvm.memcpy.p0i8.p0i8.i64(i8* noalias nocapture writeonly, i8* noalias nocapture readonly, i64, i1 immarg) #0

define fastcc void @TraceLine(i64 %tmp) {
entry:
  %LDir = alloca i64, align 8
  %NewDir1 = alloca i64, align 8
  %LDir.cast = bitcast i64* %LDir to i8*
  %NewDir1.cast = bitcast i64* %NewDir1 to i8*
  store i64 %tmp, i64* %LDir

  call void @llvm.memcpy.p0i8.p0i8.i64(i8* %NewDir1.cast, i8* %LDir.cast, i64 8, i1 false)
  ;%reload = load i64, i64* %LDir
  ;store i64 %reload, i64* %NewDir1

  %call33 = call fastcc double @IntersectObjs(i64* %NewDir1)
  %call38 = call fastcc double @IntersectObjs(i64* %LDir)
  ret void
}

declare fastcc double @IntersectObjs(i64* nocapture readonly)

; uselistorder directives
uselistorder double (i64*)* @IntersectObjs, { 1, 0 }

attributes #0 = { argmemonly nofree nounwind willreturn }

llvm/lib/Transforms/Scalar/SROA.cpp

public:
 
   uint64_t beginOffset() const { return BeginOffset; }
   uint64_t endOffset() const { return EndOffset; }
 
   bool isSplittable() const { return UseAndIsSplittable.getInt(); }
   void makeUnsplittable() { UseAndIsSplittable.setInt(false); }
 
   Use *getUse() const { return UseAndIsSplittable.getPointer(); }
+  void setUse(Use *U) { UseAndIsSplittable.setPointer(U); }
 
   bool isDead() const { return getUse() == nullptr; }
   void kill() { UseAndIsSplittable.setPointer(nullptr); }
 
   /// Support for ordering ranges.
   ///
   /// This provides an ordering over ranges such that start offsets are
   /// always increasing, and within equal start offsets, the end offsets are

 /// This class represents the slices of an alloca which are formed by its
 /// various uses. If a pointer escapes, we can't fully build a representation
 /// for the slices used and we reflect that in this structure. The uses are
 /// stored, sorted by increasing beginning offset and with unsplittable slices
 /// starting at a particular offset before splittable slices.
 class llvm::sroa::AllocaSlices {
 public:
   /// Construct the slices of a particular alloca.
-  AllocaSlices(const DataLayout &DL, AllocaInst &AI);
+  AllocaSlices(const DataLayout &DL, AllocaInst &AI, bool &Changed);
 
   /// Test whether a pointer to the allocation escapes our analysis.
   ///
   /// If this is true, the slices are never fully built and should be
   /// ignored.
   bool isEscaped() const { return PointerEscapingInstr; }
 
   /// Support for iterating over the slices.
public:
   /// Access the dead users for this alloca.
   ArrayRef<Instruction *> getDeadUsers() const { return DeadUsers; }
 
   /// Access Uses that should be dropped if the alloca is promotable.
   ArrayRef<Use *> getDeadUsesIfPromotable() const {
     return DeadUseIfPromotable;
   }
 
+  void forgetTheDead() {
+    DeadUsers.clear();
+    DeadUseIfPromotable.clear();
+    DeadOperands.clear();
+  };
+
   /// Access the dead operands referring to this alloca.
   ///
   /// These are operands which have cannot actually be used to refer to the
   /// alloca as they are outside its range and the user doesn't correct for
   /// that. These mostly consist of PHI node inputs and the like which we just
   /// need to replace with undef.
   ArrayRef<Use *> getDeadOperands() const { return DeadOperands; }
 

 #endif
 
 private:
   template <typename DerivedT, typename RetT = void> class BuilderBase;
   class SliceBuilder;
 
   friend class AllocaSlices::SliceBuilder;
 
+  void formBackingAlloca(AllocaInst *AI, bool &Changed);
+
 #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
   /// Handle to alloca instruction to simplify method interfaces.
   AllocaInst &AI;
 #endif
 
+  /// Certain escaping uses of an alloca (non-capturing-ones)
+  /// do not prevent promotion, but force retention of the alloca.
+  /// This records if there are any such uses.
+  bool NeedsBackingAlloca = false;
+
+  /// Track if there are any `select`s/PHI's involving the alloca pointers.
+  bool HavePHINodesOrSelectInstrs = false;
+
   /// The instruction responsible for this alloca not having a known set
   /// of slices.
   ///
   /// When an instruction (potentially) escapes the pointer to the alloca, we
   /// store a pointer to that here and abort trying to form slices of the
   /// alloca. This will be null if the alloca slices are analyzed successfully.
   Instruction *PointerEscapingInstr;
 
void visitPHINodeOrSelectInst(Instruction &I) {
     // themselves which should be replaced with poison.
     // FIXME: This should instead be escaped in the event we're instrumenting
     // for address sanitization.
     if (Offset.uge(AllocSize)) {
       AS.DeadOperands.push_back(U);
       return;
     }
 
+    AS.HavePHINodesOrSelectInstrs = true;
+    if (AS.NeedsBackingAlloca && AS.HavePHINodesOrSelectInstrs)
+      return PI.setAborted(&I);
+
     insertUse(I, Offset, Size);
   }
 
   void visitPHINode(PHINode &PN) { visitPHINodeOrSelectInst(PN); }
 
   void visitSelectInst(SelectInst &SI) { visitPHINodeOrSelectInst(SI); }
 
+  void visitCallBase(CallBase &CB) {
+    if (!IsOffsetKnown || !CB.doesNotCapture(U->getOperandNo()))
+      return PI.setAborted(&CB);
+    // If we know that the callee does not retain the pointer,
+    // then it does not prevent SROA, although we have to workaround this.
+    // However, for now, only allow uses, that, at most, read from said memory.
+    if (!CB.onlyReadsMemory() && !CB.onlyReadsMemory(U->getOperandNo()))
+      return PI.setAborted(&CB);
+    AS.NeedsBackingAlloca = true;
+    if (AS.NeedsBackingAlloca && AS.HavePHINodesOrSelectInstrs)
+      return PI.setAborted(&CB);
+  }
+
   /// Disable SROA entirely if there are unhandled users of the alloca.
   void visitInstruction(Instruction &I) { PI.setAborted(&I); }
 };
 
-AllocaSlices::AllocaSlices(const DataLayout &DL, AllocaInst &AI)
+AllocaSlices::AllocaSlices(const DataLayout &DL, AllocaInst &AI, bool &Changed)
     :
 #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
       AI(AI),
 #endif
       PointerEscapingInstr(nullptr) {
   SliceBuilder PB(DL, AI, *this);
   SliceBuilder::PtrInfo PtrI = PB.visitPtr(AI);
   if (PtrI.isEscaped() || PtrI.isAborted()) {
     // FIXME: We should sink the escape vs. abort info into the caller nicely,
     // possibly by just storing the PtrInfo in the AllocaSlices.
     PointerEscapingInstr = PtrI.getEscapingInst() ? PtrI.getEscapingInst()
                                                   : PtrI.getAbortingInst();
     assert(PointerEscapingInstr && "Did not track a bad instruction");
     return;
   }
 
+  // We may have found that the pointer to the AI escapes, but isn't captured.
+  if (NeedsBackingAlloca)
+    formBackingAlloca(&AI, Changed);
+
   llvm::erase_if(Slices, [](const Slice &S) { return S.isDead(); });
 
   // Sort the uses. This arranges for the offsets to be in ascending order,
   // and the sizes to be in descending order.
   llvm::stable_sort(Slices);
 }
 
 #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
private:
   bool visitSelectInst(SelectInst &SI) {
     enqueueUsers(SI);
     return false;
   }
 };
 
 } // end anonymous namespace
 
+/// Apparently, we can promote the alloca, but some uses of the alloca
+/// are calls (that don't capture it's address), which require for the
+/// trace alloca to remain. To do so, we must form a new "backing" alloca,
+/// which will be kept as an up-to-date backup of the to-be-promoted-alloca's
+/// content, and used in it's place in these non-capturing calls.
+/// FIXME: support non-readonly non-capturing calls.
+void AllocaSlices::formBackingAlloca(AllocaInst *AllocaToPromote,
+                                     bool &Changed) {
+  assert(NeedsBackingAlloca &&
+         "Should not be called if there is no need to rewrite.");
+
+  // We are going to preserve all of the original instructions that were
+  // operating on the original alloca, so we must forget any instructions
+  // that were deemed as dead-to-be-deleted during normal promotion.
+  forgetTheDead();
+
+  Changed = true;
+
+  // Now, we want to retain all of the instructions operating on the original
+  // alloca, so to avoid much hassle, create a new alloca, and swap (RAUW) them.
+  AllocaInst *ShadowAlloca = cast<AllocaInst>(AllocaToPromote->clone());
+  ShadowAlloca->takeName(AllocaToPromote);
+  AllocaToPromote->setName(ShadowAlloca->getName() + ".prom");
+  ShadowAlloca->insertBefore(AllocaToPromote);
+  AllocaToPromote->replaceAllUsesWith(ShadowAlloca);
+
+  // Avoid recomputing the same pointer over and over again, cache it.
+  SmallDenseMap<std::pair<uint64_t, Type *>, Value *> RebasedPtrsCSE;
+
+  // Don't do anything fancy, just put new insts "right after" the alloca.
+  IRBuilderTy Builder(AllocaToPromote->getContext());
+  BasicBlock *AllocaToPromoteBB = AllocaToPromote->getParent();
+  Builder.SetInsertPoint(AllocaToPromoteBB,
+                         AllocaToPromoteBB->getFirstInsertionPt());
+
+  // Give a pointer `Offset` bytes into the `AllocaToPromote` with `PtrTy` type.
+  auto getRebasedPtr = [&RebasedPtrsCSE, &Builder, AllocaToPromote,
+                        DL = AllocaToPromote->getModule()->getDataLayout()](
+                           PointerType *PtrTy, const uint64_t Offset) {
+    // Look it up in a cache first.
+    auto It = RebasedPtrsCSE.find({Offset, PtrTy});
+    if (It != RebasedPtrsCSE.end())
+      return It->second;
+
+    // Otherwise, create a new pointer, and cache it for the future.
+    Value *NewPtr = getAdjustedPtr(
+        Builder, DL, AllocaToPromote,
+        APInt(DL.getIndexSizeInBits(PtrTy->getAddressSpace()), Offset), PtrTy,
+        "");
+    RebasedPtrsCSE[{Offset, PtrTy}] = NewPtr;
+
+    return NewPtr;
+  };
+
+  // Some instructions may have several uses of an alloca, and there's
+  // a separate slice for each use, so we must cache each instruction
+  // we clone, so that we only clone it once,
+  // not for each slice that references it.
+  SmallDenseMap<Instruction *, Instruction *> InstrCloneMap;
+
+  // Now, let's just deal with each slice. Roughly, we need to clone each
+  // instruction that is referenced by a slice (once per instruction!),
+  // and change the appropriate pointer from pointing at the shadow alloca
+  // into pointing into the alloca we are going to promote.
+  //
+  // NOTE: the original instruction is generally preserved,
+  // because we need to maintain the content parity between the two allocas!
+  for (Slice &S : Slices) {
+    // Just completely ignore dead slices.
+    if (S.isDead())
+      continue;
+
+    // Which instruction does this slice represent?
+    Use *OrigUse = S.getUse();
+    auto *OrigInstr = cast<Instruction>(OrigUse->getUser());
+
+    // Now, we need to make a clone of this instruction, but operating on
+    // the alloca-to-be-promoted instead.
+    Instruction *ClonedInstr;
+    // Only clone instruction once! See if we already did that for this instr.
+    auto It = InstrCloneMap.find(OrigInstr);
+    if (It != InstrCloneMap.end())
+      ClonedInstr = It->second;
+    else {
+      // This is the first time this instruction is seen.
+      // Clone it next to the original instruction, and cache it.
+      ClonedInstr = OrigInstr->clone();
+      ClonedInstr->insertBefore(OrigInstr);
+      InstrCloneMap.insert({OrigInstr, ClonedInstr});
+
+      // Also, if the instruction was returning anything, we do that instead.
+      if (!ClonedInstr->getType()->isVoidTy()) {
+        assert(isa<LoadInst>(OrigInstr) &&
+               "Not expecting to encounter here anything other than a `load`.");
+        ClonedInstr->setName(OrigInstr->getName() + ".prom");
+        OrigInstr->replaceAllUsesWith(ClonedInstr);
+      }
+
+      if (isa<LoadInst>(OrigInstr))
+        // We know that all the offending (non-capturing) calls do not modify
+        // the content of the shadow alloca, so we do not need to propagate
+        // the content of the shadow alloca to the alloca-to-be-promoted.
+        DeadUsers.push_back(OrigInstr);
+    }
+
+    // Final touch: the slice should refer to the
+    // use of the alloca-to-be-promoted, while it currently refers to
+    // use of the shadow alloca, so rectify that.
+    Value *NewPtr = getRebasedPtr(cast<PointerType>(OrigUse->get()->getType()),
+                                  S.beginOffset());
+    Use &ClonedUse = ClonedInstr->getOperandUse(OrigUse->getOperandNo());
+    ClonedUse.set(NewPtr);
+    S.setUse(&ClonedUse);
+  }
+}
+
 /// Strip aggregate type wrapping.
 ///
 /// This removes no-op aggregate types wrapping an underlying type. It will
 /// strip as many layers of types as it can without changing either the type
 /// size or the allocated size.
 static Type *stripAggregateTypeWrapping(const DataLayout &DL, Type *Ty) {
   if (Ty->isSingleValueType())
     return Ty;
bool SROAPass::runOnAlloca(AllocaInst &AI) {
 
   // First, split any FCA loads and stores touching this alloca to promote
   // better splitting and promotion opportunities.
   IRBuilderTy IRB(&AI);
   AggLoadStoreRewriter AggRewriter(DL, IRB);
   Changed |= AggRewriter.rewrite(AI);
 
   // Build the slices using a recursive instruction-visiting builder.
-  AllocaSlices AS(DL, AI);
+  AllocaSlices AS(DL, AI, Changed);
   LLVM_DEBUG(AS.print(dbgs()));
   if (AS.isEscaped())
     return Changed;
 
   // Delete all the dead users of this alloca before splitting and rewriting it.
   for (Instruction *DeadUser : AS.getDeadUsers()) {
     // Free up everything used by this instruction.
     for (Use &DeadOp : DeadUser->operands())

llvm/test/Transforms/SROA/non-capturing-call-readonly.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt < %s -passes=sroa -S | FileCheck %s --check-prefix=CHECK
 ; RUN: opt < %s -passes=sroa -opaque-pointers -S | FileCheck %s --check-prefix=CHECK-OPAQUE
 
 define i32 @alloca_used_in_call(i32* %data, i64 %n) {
 ; CHECK-LABEL: @alloca_used_in_call(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[RETVAL:%.*]] = alloca i32, align 4
 ; CHECK-NEXT:    store i32 0, i32* [[RETVAL]], align 4
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
-; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[RETVAL_PROM_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-NEXT:    [[LD:%.*]] = load i32, i32* [[ARRAYIDX]], align 4
-; CHECK-NEXT:    [[RDX:%.*]] = load i32, i32* [[RETVAL]], align 4
-; CHECK-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_PROM_0]], [[LD]]
 ; CHECK-NEXT:    store i32 [[RDX_INC]], i32* [[RETVAL]], align 4
 ; CHECK-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    [[I0:%.*]] = call i32 @user_of_alloca(i32* [[RETVAL]])
-; CHECK-NEXT:    [[I1:%.*]] = load i32, i32* [[RETVAL]], align 4
-; CHECK-NEXT:    ret i32 [[I1]]
+; CHECK-NEXT:    ret i32 [[RDX_INC]]
 ;
 ; CHECK-OPAQUE-LABEL: @alloca_used_in_call(
 ; CHECK-OPAQUE-NEXT:  entry:
 ; CHECK-OPAQUE-NEXT:    [[RETVAL:%.*]] = alloca i32, align 4
 ; CHECK-OPAQUE-NEXT:    store i32 0, ptr [[RETVAL]], align 4
 ; CHECK-OPAQUE-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-OPAQUE:       loop:
-; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[RETVAL_PROM_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-OPAQUE-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-OPAQUE-NEXT:    [[LD:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX:%.*]] = load i32, ptr [[RETVAL]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-OPAQUE-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_PROM_0]], [[LD]]
 ; CHECK-OPAQUE-NEXT:    store i32 [[RDX_INC]], ptr [[RETVAL]], align 4
 ; CHECK-OPAQUE-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-OPAQUE-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-OPAQUE-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK-OPAQUE:       exit:
 ; CHECK-OPAQUE-NEXT:    [[I0:%.*]] = call i32 @user_of_alloca(ptr [[RETVAL]])
-; CHECK-OPAQUE-NEXT:    [[I1:%.*]] = load i32, ptr [[RETVAL]], align 4
-; CHECK-OPAQUE-NEXT:    ret i32 [[I1]]
+; CHECK-OPAQUE-NEXT:    ret i32 [[RDX_INC]]
 ;
 entry:
   %retval = alloca i32, align 4
   store i32 0, i32* %retval, align 4
   br label %loop
 
 loop:
   %indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %loop ]

 
 define i32 @alloca_not_captured_and_readonly_as_per_operand_attr(i32* %data, i64 %n) {
 ; CHECK-LABEL: @alloca_not_captured_and_readonly_as_per_operand_attr(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[RETVAL:%.*]] = alloca i32, align 4
 ; CHECK-NEXT:    store i32 0, i32* [[RETVAL]], align 4
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
-; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[RETVAL_PROM_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-NEXT:    [[LD:%.*]] = load i32, i32* [[ARRAYIDX]], align 4
-; CHECK-NEXT:    [[RDX:%.*]] = load i32, i32* [[RETVAL]], align 4
-; CHECK-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_PROM_0]], [[LD]]
 ; CHECK-NEXT:    store i32 [[RDX_INC]], i32* [[RETVAL]], align 4
 ; CHECK-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    [[I0:%.*]] = call i32 @capture_of_alloca(i32* nocapture readonly [[RETVAL]])
-; CHECK-NEXT:    [[I1:%.*]] = load i32, i32* [[RETVAL]], align 4
-; CHECK-NEXT:    ret i32 [[I1]]
+; CHECK-NEXT:    ret i32 [[RDX_INC]]
 ;
 ; CHECK-OPAQUE-LABEL: @alloca_not_captured_and_readonly_as_per_operand_attr(
 ; CHECK-OPAQUE-NEXT:  entry:
 ; CHECK-OPAQUE-NEXT:    [[RETVAL:%.*]] = alloca i32, align 4
 ; CHECK-OPAQUE-NEXT:    store i32 0, ptr [[RETVAL]], align 4
 ; CHECK-OPAQUE-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-OPAQUE:       loop:
-; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[RETVAL_PROM_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-OPAQUE-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-OPAQUE-NEXT:    [[LD:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX:%.*]] = load i32, ptr [[RETVAL]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-OPAQUE-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_PROM_0]], [[LD]]
 ; CHECK-OPAQUE-NEXT:    store i32 [[RDX_INC]], ptr [[RETVAL]], align 4
 ; CHECK-OPAQUE-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-OPAQUE-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-OPAQUE-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK-OPAQUE:       exit:
 ; CHECK-OPAQUE-NEXT:    [[I0:%.*]] = call i32 @capture_of_alloca(ptr nocapture readonly [[RETVAL]])
-; CHECK-OPAQUE-NEXT:    [[I1:%.*]] = load i32, ptr [[RETVAL]], align 4
-; CHECK-OPAQUE-NEXT:    ret i32 [[I1]]
+; CHECK-OPAQUE-NEXT:    ret i32 [[RDX_INC]]
 ;
 entry:
   %retval = alloca i32, align 4
   store i32 0, i32* %retval, align 4
   br label %loop
 
 loop:
   %indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %loop ]

 
 define i32 @alloca_not_captured_as_per_operand_attr_and_readonly_as_per_callbase_attr(i32* %data, i64 %n) {
 ; CHECK-LABEL: @alloca_not_captured_as_per_operand_attr_and_readonly_as_per_callbase_attr(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[RETVAL:%.*]] = alloca i32, align 4
 ; CHECK-NEXT:    store i32 0, i32* [[RETVAL]], align 4
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
-; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[RETVAL_PROM_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-NEXT:    [[LD:%.*]] = load i32, i32* [[ARRAYIDX]], align 4
-; CHECK-NEXT:    [[RDX:%.*]] = load i32, i32* [[RETVAL]], align 4
-; CHECK-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_PROM_0]], [[LD]]
 ; CHECK-NEXT:    store i32 [[RDX_INC]], i32* [[RETVAL]], align 4
 ; CHECK-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    [[I0:%.*]] = call i32 @capture_of_alloca(i32* nocapture [[RETVAL]]) #[[ATTR2:[0-9]+]]
-; CHECK-NEXT:    [[I1:%.*]] = load i32, i32* [[RETVAL]], align 4
-; CHECK-NEXT:    ret i32 [[I1]]
+; CHECK-NEXT:    ret i32 [[RDX_INC]]
 ;
 ; CHECK-OPAQUE-LABEL: @alloca_not_captured_as_per_operand_attr_and_readonly_as_per_callbase_attr(
 ; CHECK-OPAQUE-NEXT:  entry:
 ; CHECK-OPAQUE-NEXT:    [[RETVAL:%.*]] = alloca i32, align 4
 ; CHECK-OPAQUE-NEXT:    store i32 0, ptr [[RETVAL]], align 4
 ; CHECK-OPAQUE-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-OPAQUE:       loop:
-; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[RETVAL_PROM_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-OPAQUE-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-OPAQUE-NEXT:    [[LD:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX:%.*]] = load i32, ptr [[RETVAL]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-OPAQUE-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_PROM_0]], [[LD]]
 ; CHECK-OPAQUE-NEXT:    store i32 [[RDX_INC]], ptr [[RETVAL]], align 4
 ; CHECK-OPAQUE-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-OPAQUE-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-OPAQUE-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK-OPAQUE:       exit:
 ; CHECK-OPAQUE-NEXT:    [[I0:%.*]] = call i32 @capture_of_alloca(ptr nocapture [[RETVAL]]) #[[ATTR2:[0-9]+]]
-; CHECK-OPAQUE-NEXT:    [[I1:%.*]] = load i32, ptr [[RETVAL]], align 4
-; CHECK-OPAQUE-NEXT:    ret i32 [[I1]]
+; CHECK-OPAQUE-NEXT:    ret i32 [[RDX_INC]]
 ;
 entry:
   %retval = alloca i32, align 4
   store i32 0, i32* %retval, align 4
   br label %loop
 
 loop:
   %indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %loop ]

 
 define i32 @alloca_with_gep_used_in_call(i32* %data, i64 %n) {
 ; CHECK-LABEL: @alloca_with_gep_used_in_call(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[RETVAL:%.*]] = alloca i32, align 4
 ; CHECK-NEXT:    store i32 0, i32* [[RETVAL]], align 4
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
-; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[RETVAL_PROM_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-NEXT:    [[LD:%.*]] = load i32, i32* [[ARRAYIDX]], align 4
-; CHECK-NEXT:    [[RDX:%.*]] = load i32, i32* [[RETVAL]], align 4
-; CHECK-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_PROM_0]], [[LD]]
 ; CHECK-NEXT:    store i32 [[RDX_INC]], i32* [[RETVAL]], align 4
 ; CHECK-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, i32* [[RETVAL]], i32 0
 ; CHECK-NEXT:    [[I0:%.*]] = call i32 @user_of_alloca(i32* [[GEP]])
-; CHECK-NEXT:    [[I1:%.*]] = load i32, i32* [[RETVAL]], align 4
-; CHECK-NEXT:    ret i32 [[I1]]
+; CHECK-NEXT:    ret i32 [[RDX_INC]]
 ;
 ; CHECK-OPAQUE-LABEL: @alloca_with_gep_used_in_call(
 ; CHECK-OPAQUE-NEXT:  entry:
 ; CHECK-OPAQUE-NEXT:    [[RETVAL:%.*]] = alloca i32, align 4
 ; CHECK-OPAQUE-NEXT:    store i32 0, ptr [[RETVAL]], align 4
 ; CHECK-OPAQUE-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-OPAQUE:       loop:
-; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[RETVAL_PROM_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-OPAQUE-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-OPAQUE-NEXT:    [[LD:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX:%.*]] = load i32, ptr [[RETVAL]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-OPAQUE-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_PROM_0]], [[LD]]
 ; CHECK-OPAQUE-NEXT:    store i32 [[RDX_INC]], ptr [[RETVAL]], align 4
 ; CHECK-OPAQUE-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-OPAQUE-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-OPAQUE-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK-OPAQUE:       exit:
 ; CHECK-OPAQUE-NEXT:    [[GEP:%.*]] = getelementptr i32, ptr [[RETVAL]], i32 0
 ; CHECK-OPAQUE-NEXT:    [[I0:%.*]] = call i32 @user_of_alloca(ptr [[GEP]])
-; CHECK-OPAQUE-NEXT:    [[I1:%.*]] = load i32, ptr [[RETVAL]], align 4
-; CHECK-OPAQUE-NEXT:    ret i32 [[I1]]
+; CHECK-OPAQUE-NEXT:    ret i32 [[RDX_INC]]
 ;
 entry:
   %retval = alloca i32, align 4
   store i32 0, i32* %retval, align 4
   br label %loop
 
 loop:
   %indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %loop ]

 
 define i32 @alloca_used_in_maybe_throwing_call(i32* %data, i64 %n) personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*)  {
 ; CHECK-LABEL: @alloca_used_in_maybe_throwing_call(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[RETVAL:%.*]] = alloca i32, align 4
 ; CHECK-NEXT:    store i32 0, i32* [[RETVAL]], align 4
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
-; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[RETVAL_PROM_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-NEXT:    [[LD:%.*]] = load i32, i32* [[ARRAYIDX]], align 4
-; CHECK-NEXT:    [[RDX:%.*]] = load i32, i32* [[RETVAL]], align 4
-; CHECK-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_PROM_0]], [[LD]]
 ; CHECK-NEXT:    store i32 [[RDX_INC]], i32* [[RETVAL]], align 4
 ; CHECK-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    [[I0:%.*]] = invoke i32 @user_of_alloca(i32* [[RETVAL]])
 ; CHECK-NEXT:    to label [[CONT:%.*]] unwind label [[UW:%.*]]
 ; CHECK:       cont:
 ; CHECK-NEXT:    br label [[END:%.*]]
 ; CHECK:       uw:
 ; CHECK-NEXT:    [[I1:%.*]] = landingpad { i8*, i32 }
 ; CHECK-NEXT:    catch i8* null
 ; CHECK-NEXT:    br label [[END]]
 ; CHECK:       end:
-; CHECK-NEXT:    [[I2:%.*]] = load i32, i32* [[RETVAL]], align 4
-; CHECK-NEXT:    ret i32 [[I2]]
+; CHECK-NEXT:    ret i32 [[RDX_INC]]
 ;
 ; CHECK-OPAQUE-LABEL: @alloca_used_in_maybe_throwing_call(
 ; CHECK-OPAQUE-NEXT:  entry:
 ; CHECK-OPAQUE-NEXT:    [[RETVAL:%.*]] = alloca i32, align 4
 ; CHECK-OPAQUE-NEXT:    store i32 0, ptr [[RETVAL]], align 4
 ; CHECK-OPAQUE-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-OPAQUE:       loop:
-; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[RETVAL_PROM_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-OPAQUE-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-OPAQUE-NEXT:    [[LD:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX:%.*]] = load i32, ptr [[RETVAL]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-OPAQUE-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_PROM_0]], [[LD]]
 ; CHECK-OPAQUE-NEXT:    store i32 [[RDX_INC]], ptr [[RETVAL]], align 4
 ; CHECK-OPAQUE-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-OPAQUE-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-OPAQUE-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK-OPAQUE:       exit:
 ; CHECK-OPAQUE-NEXT:    [[I0:%.*]] = invoke i32 @user_of_alloca(ptr [[RETVAL]])
 ; CHECK-OPAQUE-NEXT:    to label [[CONT:%.*]] unwind label [[UW:%.*]]
 ; CHECK-OPAQUE:       cont:
 ; CHECK-OPAQUE-NEXT:    br label [[END:%.*]]
 ; CHECK-OPAQUE:       uw:
 ; CHECK-OPAQUE-NEXT:    [[I1:%.*]] = landingpad { ptr, i32 }
 ; CHECK-OPAQUE-NEXT:    catch ptr null
 ; CHECK-OPAQUE-NEXT:    br label [[END]]
 ; CHECK-OPAQUE:       end:
-; CHECK-OPAQUE-NEXT:    [[I2:%.*]] = load i32, ptr [[RETVAL]], align 4
-; CHECK-OPAQUE-NEXT:    ret i32 [[I2]]
+; CHECK-OPAQUE-NEXT:    ret i32 [[RDX_INC]]
 ;
 entry:
   %retval = alloca i32, align 4
   store i32 0, i32* %retval, align 4
   br label %loop
 
 loop:
   %indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %loop ]

 
 define i32 @alloca_used_in_maybe_throwing_call_with_same_dests(i32* %data, i64 %n) personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*)  {
 ; CHECK-LABEL: @alloca_used_in_maybe_throwing_call_with_same_dests(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[RETVAL:%.*]] = alloca i32, align 4
 ; CHECK-NEXT:    store i32 0, i32* [[RETVAL]], align 4
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
-; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[RETVAL_PROM_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-NEXT:    [[LD:%.*]] = load i32, i32* [[ARRAYIDX]], align 4
-; CHECK-NEXT:    [[RDX:%.*]] = load i32, i32* [[RETVAL]], align 4
-; CHECK-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_PROM_0]], [[LD]]
 ; CHECK-NEXT:    store i32 [[RDX_INC]], i32* [[RETVAL]], align 4
 ; CHECK-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    [[I0:%.*]] = invoke i32 @user_of_alloca(i32* [[RETVAL]])
 ; CHECK-NEXT:    to label [[END:%.*]] unwind label [[UW:%.*]]
 ; CHECK:       uw:
 ; CHECK-NEXT:    [[I1:%.*]] = landingpad { i8*, i32 }
 ; CHECK-NEXT:    catch i8* null
 ; CHECK-NEXT:    br label [[END]]
 ; CHECK:       end:
-; CHECK-NEXT:    [[I2:%.*]] = load i32, i32* [[RETVAL]], align 4
-; CHECK-NEXT:    ret i32 [[I2]]
+; CHECK-NEXT:    ret i32 [[RDX_INC]]
 ;
 ; CHECK-OPAQUE-LABEL: @alloca_used_in_maybe_throwing_call_with_same_dests(
 ; CHECK-OPAQUE-NEXT:  entry:
 ; CHECK-OPAQUE-NEXT:    [[RETVAL:%.*]] = alloca i32, align 4
 ; CHECK-OPAQUE-NEXT:    store i32 0, ptr [[RETVAL]], align 4
 ; CHECK-OPAQUE-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-OPAQUE:       loop:
-; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[RETVAL_PROM_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-OPAQUE-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-OPAQUE-NEXT:    [[LD:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX:%.*]] = load i32, ptr [[RETVAL]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-OPAQUE-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_PROM_0]], [[LD]]
 ; CHECK-OPAQUE-NEXT:    store i32 [[RDX_INC]], ptr [[RETVAL]], align 4
 ; CHECK-OPAQUE-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-OPAQUE-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-OPAQUE-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK-OPAQUE:       exit:
 ; CHECK-OPAQUE-NEXT:    [[I0:%.*]] = invoke i32 @user_of_alloca(ptr [[RETVAL]])
 ; CHECK-OPAQUE-NEXT:    to label [[END:%.*]] unwind label [[UW:%.*]]
 ; CHECK-OPAQUE:       uw:
 ; CHECK-OPAQUE-NEXT:    [[I1:%.*]] = landingpad { ptr, i32 }
 ; CHECK-OPAQUE-NEXT:    catch ptr null
 ; CHECK-OPAQUE-NEXT:    br label [[END]]
 ; CHECK-OPAQUE:       end:
-; CHECK-OPAQUE-NEXT:    [[I2:%.*]] = load i32, ptr [[RETVAL]], align 4
-; CHECK-OPAQUE-NEXT:    ret i32 [[I2]]
+; CHECK-OPAQUE-NEXT:    ret i32 [[RDX_INC]]
 ;
 entry:
   %retval = alloca i32, align 4
   store i32 0, i32* %retval, align 4
   br label %loop
 
 loop:
   %indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %loop ]

 ; CHECK-NEXT:    [[RETVAL_FULL:%.*]] = alloca [2 x i32], align 4
 ; CHECK-NEXT:    [[DOTFCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 0
 ; CHECK-NEXT:    store i32 0, i32* [[DOTFCA_0_GEP]], align 4
 ; CHECK-NEXT:    [[DOTFCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 1
 ; CHECK-NEXT:    store i32 0, i32* [[DOTFCA_1_GEP]], align 4
 ; CHECK-NEXT:    [[RETVAL:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i64 0, i64 1
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
-; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[RETVAL_FULL_PROM_SROA_2_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-NEXT:    [[LD:%.*]] = load i32, i32* [[ARRAYIDX]], align 4
-; CHECK-NEXT:    [[RDX:%.*]] = load i32, i32* [[RETVAL]], align 4
-; CHECK-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_FULL_PROM_SROA_2_0]], [[LD]]
 ; CHECK-NEXT:    store i32 [[RDX_INC]], i32* [[RETVAL]], align 4
 ; CHECK-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    [[I0:%.*]] = call i32 @user_of_alloca(i32* [[RETVAL]])
-; CHECK-NEXT:    [[I1_FCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 0
-; CHECK-NEXT:    [[I1_FCA_0_LOAD:%.*]] = load i32, i32* [[I1_FCA_0_GEP]], align 4
-; CHECK-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 [[I1_FCA_0_LOAD]], 0
-; CHECK-NEXT:    [[I1_FCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 1
-; CHECK-NEXT:    [[I1_FCA_1_LOAD:%.*]] = load i32, i32* [[I1_FCA_1_GEP]], align 4
-; CHECK-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[I1_FCA_1_LOAD]], 1
+; CHECK-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 0, 0
+; CHECK-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[RDX_INC]], 1
 ; CHECK-NEXT:    ret [2 x i32] [[I1_FCA_1_INSERT]]
 ;
 ; CHECK-OPAQUE-LABEL: @part_of_alloca_used_in_call(
 ; CHECK-OPAQUE-NEXT:  entry:
 ; CHECK-OPAQUE-NEXT:    [[RETVAL_FULL:%.*]] = alloca [2 x i32], align 4
 ; CHECK-OPAQUE-NEXT:    [[DOTFCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 0
 ; CHECK-OPAQUE-NEXT:    store i32 0, ptr [[DOTFCA_0_GEP]], align 4
 ; CHECK-OPAQUE-NEXT:    [[DOTFCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 1
 ; CHECK-OPAQUE-NEXT:    store i32 0, ptr [[DOTFCA_1_GEP]], align 4
 ; CHECK-OPAQUE-NEXT:    [[RETVAL:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i64 0, i64 1
 ; CHECK-OPAQUE-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-OPAQUE:       loop:
-; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[RETVAL_FULL_PROM_SROA_2_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-OPAQUE-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-OPAQUE-NEXT:    [[LD:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX:%.*]] = load i32, ptr [[RETVAL]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-OPAQUE-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_FULL_PROM_SROA_2_0]], [[LD]]
 ; CHECK-OPAQUE-NEXT:    store i32 [[RDX_INC]], ptr [[RETVAL]], align 4
 ; CHECK-OPAQUE-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-OPAQUE-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-OPAQUE-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK-OPAQUE:       exit:
 ; CHECK-OPAQUE-NEXT:    [[I0:%.*]] = call i32 @user_of_alloca(ptr [[RETVAL]])
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 0
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_LOAD:%.*]] = load i32, ptr [[I1_FCA_0_GEP]], align 4
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 [[I1_FCA_0_LOAD]], 0
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 1
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_LOAD:%.*]] = load i32, ptr [[I1_FCA_1_GEP]], align 4
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[I1_FCA_1_LOAD]], 1
+; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 0, 0
+; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[RDX_INC]], 1
 ; CHECK-OPAQUE-NEXT:    ret [2 x i32] [[I1_FCA_1_INSERT]]
 ;
 entry:
   %retval.full = alloca [2 x i32], align 4
   store [2 x i32] zeroinitializer, [2 x i32]* %retval.full, align 4
   %retval = getelementptr inbounds [2 x i32], [2 x i32]* %retval.full, i64 0, i64 1
   br label %loop
 

 ; CHECK-NEXT:    [[DOTFCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 0
 ; CHECK-NEXT:    store i32 0, i32* [[DOTFCA_0_GEP]], align 4
 ; CHECK-NEXT:    [[DOTFCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 1
 ; CHECK-NEXT:    store i32 0, i32* [[DOTFCA_1_GEP]], align 4
 ; CHECK-NEXT:    [[RETVAL_BASE:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i64 0, i64 0
 ; CHECK-NEXT:    [[RETVAL:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i64 0, i64 1
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
-; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[RETVAL_FULL_PROM_SROA_2_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-NEXT:    [[LD:%.*]] = load i32, i32* [[ARRAYIDX]], align 4
-; CHECK-NEXT:    [[RDX:%.*]] = load i32, i32* [[RETVAL]], align 4
-; CHECK-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_FULL_PROM_SROA_2_0]], [[LD]]
 ; CHECK-NEXT:    store i32 [[RDX_INC]], i32* [[RETVAL]], align 4
 ; CHECK-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    [[I0:%.*]] = call i32 @user_of_alloca_with_multiple_args(i32* [[RETVAL]], i32* [[RETVAL_BASE]])
-; CHECK-NEXT:    [[I1_FCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 0
-; CHECK-NEXT:    [[I1_FCA_0_LOAD:%.*]] = load i32, i32* [[I1_FCA_0_GEP]], align 4
-; CHECK-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 [[I1_FCA_0_LOAD]], 0
-; CHECK-NEXT:    [[I1_FCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 1
-; CHECK-NEXT:    [[I1_FCA_1_LOAD:%.*]] = load i32, i32* [[I1_FCA_1_GEP]], align 4
-; CHECK-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[I1_FCA_1_LOAD]], 1
+; CHECK-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 0, 0
+; CHECK-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[RDX_INC]], 1
 ; CHECK-NEXT:    ret [2 x i32] [[I1_FCA_1_INSERT]]
 ;
 ; CHECK-OPAQUE-LABEL: @all_parts_of_alloca_used_in_call_with_multiple_args(
 ; CHECK-OPAQUE-NEXT:  entry:
 ; CHECK-OPAQUE-NEXT:    [[RETVAL_FULL:%.*]] = alloca [2 x i32], align 4
 ; CHECK-OPAQUE-NEXT:    [[DOTFCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 0
 ; CHECK-OPAQUE-NEXT:    store i32 0, ptr [[DOTFCA_0_GEP]], align 4
 ; CHECK-OPAQUE-NEXT:    [[DOTFCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 1
 ; CHECK-OPAQUE-NEXT:    store i32 0, ptr [[DOTFCA_1_GEP]], align 4
 ; CHECK-OPAQUE-NEXT:    [[RETVAL_BASE:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i64 0, i64 0
 ; CHECK-OPAQUE-NEXT:    [[RETVAL:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i64 0, i64 1
 ; CHECK-OPAQUE-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-OPAQUE:       loop:
-; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[RETVAL_FULL_PROM_SROA_2_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-OPAQUE-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-OPAQUE-NEXT:    [[LD:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX:%.*]] = load i32, ptr [[RETVAL]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-OPAQUE-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_FULL_PROM_SROA_2_0]], [[LD]]
 ; CHECK-OPAQUE-NEXT:    store i32 [[RDX_INC]], ptr [[RETVAL]], align 4
 ; CHECK-OPAQUE-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-OPAQUE-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-OPAQUE-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK-OPAQUE:       exit:
 ; CHECK-OPAQUE-NEXT:    [[I0:%.*]] = call i32 @user_of_alloca_with_multiple_args(ptr [[RETVAL]], ptr [[RETVAL_BASE]])
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 0
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_LOAD:%.*]] = load i32, ptr [[I1_FCA_0_GEP]], align 4
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 [[I1_FCA_0_LOAD]], 0
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 1
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_LOAD:%.*]] = load i32, ptr [[I1_FCA_1_GEP]], align 4
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[I1_FCA_1_LOAD]], 1
+; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 0, 0
+; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[RDX_INC]], 1
 ; CHECK-OPAQUE-NEXT:    ret [2 x i32] [[I1_FCA_1_INSERT]]
 ;
 entry:
   %retval.full = alloca [2 x i32], align 4
   store [2 x i32] zeroinitializer, [2 x i32]* %retval.full, align 4
   %retval.base = getelementptr inbounds [2 x i32], [2 x i32]* %retval.full, i64 0, i64 0
   %retval = getelementptr inbounds [2 x i32], [2 x i32]* %retval.full, i64 0, i64 1
   br label %loop

 ; CHECK-NEXT:    [[DOTFCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 0
 ; CHECK-NEXT:    store i32 0, i32* [[DOTFCA_0_GEP]], align 4
 ; CHECK-NEXT:    [[DOTFCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 1
 ; CHECK-NEXT:    store i32 0, i32* [[DOTFCA_1_GEP]], align 4
 ; CHECK-NEXT:    [[RETVAL_BASE:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i64 0, i64 0
 ; CHECK-NEXT:    [[RETVAL:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i64 0, i64 1
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
-; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[RETVAL_FULL_PROM_SROA_3_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-NEXT:    [[LD:%.*]] = load i32, i32* [[ARRAYIDX]], align 4
-; CHECK-NEXT:    [[RDX:%.*]] = load i32, i32* [[RETVAL]], align 4
-; CHECK-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_FULL_PROM_SROA_3_0]], [[LD]]
 ; CHECK-NEXT:    store i32 [[RDX_INC]], i32* [[RETVAL]], align 4
 ; CHECK-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    call void @llvm.memcpy.p0i32.p0i32.i32(i32* [[RETVAL_BASE]], i32* [[RETVAL]], i32 4, i1 false)
 ; CHECK-NEXT:    [[I0:%.*]] = call i32 @user_of_alloca_with_multiple_args(i32* [[RETVAL]], i32* [[RETVAL_BASE]])
-; CHECK-NEXT:    [[I1_FCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 0
-; CHECK-NEXT:    [[I1_FCA_0_LOAD:%.*]] = load i32, i32* [[I1_FCA_0_GEP]], align 4
-; CHECK-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 [[I1_FCA_0_LOAD]], 0
-; CHECK-NEXT:    [[I1_FCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 1
-; CHECK-NEXT:    [[I1_FCA_1_LOAD:%.*]] = load i32, i32* [[I1_FCA_1_GEP]], align 4
-; CHECK-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[I1_FCA_1_LOAD]], 1
+; CHECK-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 [[RDX_INC]], 0
+; CHECK-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[RDX_INC]], 1
 ; CHECK-NEXT:    ret [2 x i32] [[I1_FCA_1_INSERT]]
 ;
 ; CHECK-OPAQUE-LABEL: @all_parts_of_alloca_used_in_call_with_multiple_args_with_memcpy_before_call(
 ; CHECK-OPAQUE-NEXT:  entry:
 ; CHECK-OPAQUE-NEXT:    [[RETVAL_FULL:%.*]] = alloca [2 x i32], align 4
 ; CHECK-OPAQUE-NEXT:    [[DOTFCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 0
 ; CHECK-OPAQUE-NEXT:    store i32 0, ptr [[DOTFCA_0_GEP]], align 4
 ; CHECK-OPAQUE-NEXT:    [[DOTFCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 1
 ; CHECK-OPAQUE-NEXT:    store i32 0, ptr [[DOTFCA_1_GEP]], align 4
 ; CHECK-OPAQUE-NEXT:    [[RETVAL_BASE:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i64 0, i64 0
 ; CHECK-OPAQUE-NEXT:    [[RETVAL:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i64 0, i64 1
 ; CHECK-OPAQUE-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-OPAQUE:       loop:
-; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[RETVAL_FULL_PROM_SROA_3_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-OPAQUE-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-OPAQUE-NEXT:    [[LD:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX:%.*]] = load i32, ptr [[RETVAL]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-OPAQUE-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_FULL_PROM_SROA_3_0]], [[LD]]
 ; CHECK-OPAQUE-NEXT:    store i32 [[RDX_INC]], ptr [[RETVAL]], align 4
 ; CHECK-OPAQUE-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-OPAQUE-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-OPAQUE-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK-OPAQUE:       exit:
 ; CHECK-OPAQUE-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr [[RETVAL_BASE]], ptr [[RETVAL]], i32 4, i1 false)
 ; CHECK-OPAQUE-NEXT:    [[I0:%.*]] = call i32 @user_of_alloca_with_multiple_args(ptr [[RETVAL]], ptr [[RETVAL_BASE]])
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 0
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_LOAD:%.*]] = load i32, ptr [[I1_FCA_0_GEP]], align 4
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 [[I1_FCA_0_LOAD]], 0
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 1
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_LOAD:%.*]] = load i32, ptr [[I1_FCA_1_GEP]], align 4
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[I1_FCA_1_LOAD]], 1
+; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 [[RDX_INC]], 0
+; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[RDX_INC]], 1
 ; CHECK-OPAQUE-NEXT:    ret [2 x i32] [[I1_FCA_1_INSERT]]
 ;
 entry:
   %retval.full = alloca [2 x i32], align 4
   store [2 x i32] zeroinitializer, [2 x i32]* %retval.full, align 4
   %retval.base = getelementptr inbounds [2 x i32], [2 x i32]* %retval.full, i64 0, i64 0
   %retval = getelementptr inbounds [2 x i32], [2 x i32]* %retval.full, i64 0, i64 1
   br label %loop

 ; CHECK-NEXT:    [[DOTFCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 0
 ; CHECK-NEXT:    store i32 0, i32* [[DOTFCA_0_GEP]], align 4
 ; CHECK-NEXT:    [[DOTFCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 1
 ; CHECK-NEXT:    store i32 0, i32* [[DOTFCA_1_GEP]], align 4
 ; CHECK-NEXT:    [[RETVAL_BASE:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i64 0, i64 0
 ; CHECK-NEXT:    [[RETVAL:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i64 0, i64 1
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
-; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[RETVAL_FULL_PROM_SROA_3_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-NEXT:    [[LD:%.*]] = load i32, i32* [[ARRAYIDX]], align 4
-; CHECK-NEXT:    [[RDX:%.*]] = load i32, i32* [[RETVAL]], align 4
-; CHECK-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_FULL_PROM_SROA_3_0]], [[LD]]
 ; CHECK-NEXT:    store i32 [[RDX_INC]], i32* [[RETVAL]], align 4
 ; CHECK-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    [[I0:%.*]] = call i32 @user_of_alloca_with_multiple_args(i32* [[RETVAL]], i32* [[RETVAL_BASE]])
 ; CHECK-NEXT:    call void @llvm.memcpy.p0i32.p0i32.i32(i32* [[RETVAL_BASE]], i32* [[RETVAL]], i32 4, i1 false)
-; CHECK-NEXT:    [[I1_FCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 0
-; CHECK-NEXT:    [[I1_FCA_0_LOAD:%.*]] = load i32, i32* [[I1_FCA_0_GEP]], align 4
-; CHECK-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 [[I1_FCA_0_LOAD]], 0
-; CHECK-NEXT:    [[I1_FCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 1
-; CHECK-NEXT:    [[I1_FCA_1_LOAD:%.*]] = load i32, i32* [[I1_FCA_1_GEP]], align 4
-; CHECK-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[I1_FCA_1_LOAD]], 1
+; CHECK-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 [[RDX_INC]], 0
+; CHECK-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[RDX_INC]], 1
 ; CHECK-NEXT:    ret [2 x i32] [[I1_FCA_1_INSERT]]
 ;
 ; CHECK-OPAQUE-LABEL: @all_parts_of_alloca_used_in_call_with_multiple_args_with_memcpy_after_call(
 ; CHECK-OPAQUE-NEXT:  entry:
 ; CHECK-OPAQUE-NEXT:    [[RETVAL_FULL:%.*]] = alloca [2 x i32], align 4
 ; CHECK-OPAQUE-NEXT:    [[DOTFCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 0
 ; CHECK-OPAQUE-NEXT:    store i32 0, ptr [[DOTFCA_0_GEP]], align 4
 ; CHECK-OPAQUE-NEXT:    [[DOTFCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 1
 ; CHECK-OPAQUE-NEXT:    store i32 0, ptr [[DOTFCA_1_GEP]], align 4
 ; CHECK-OPAQUE-NEXT:    [[RETVAL_BASE:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i64 0, i64 0
 ; CHECK-OPAQUE-NEXT:    [[RETVAL:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i64 0, i64 1
 ; CHECK-OPAQUE-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-OPAQUE:       loop:
-; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[RETVAL_FULL_PROM_SROA_3_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-OPAQUE-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-OPAQUE-NEXT:    [[LD:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX:%.*]] = load i32, ptr [[RETVAL]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-OPAQUE-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_FULL_PROM_SROA_3_0]], [[LD]]
 ; CHECK-OPAQUE-NEXT:    store i32 [[RDX_INC]], ptr [[RETVAL]], align 4
 ; CHECK-OPAQUE-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-OPAQUE-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-OPAQUE-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK-OPAQUE:       exit:
 ; CHECK-OPAQUE-NEXT:    [[I0:%.*]] = call i32 @user_of_alloca_with_multiple_args(ptr [[RETVAL]], ptr [[RETVAL_BASE]])
 ; CHECK-OPAQUE-NEXT:    call void @llvm.memcpy.p0.p0.i32(ptr [[RETVAL_BASE]], ptr [[RETVAL]], i32 4, i1 false)
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 0
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_LOAD:%.*]] = load i32, ptr [[I1_FCA_0_GEP]], align 4
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 [[I1_FCA_0_LOAD]], 0
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 1
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_LOAD:%.*]] = load i32, ptr [[I1_FCA_1_GEP]], align 4
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[I1_FCA_1_LOAD]], 1
+; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 [[RDX_INC]], 0
+; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[RDX_INC]], 1
 ; CHECK-OPAQUE-NEXT:    ret [2 x i32] [[I1_FCA_1_INSERT]]
 ;
 entry:
   %retval.full = alloca [2 x i32], align 4
   store [2 x i32] zeroinitializer, [2 x i32]* %retval.full, align 4
   %retval.base = getelementptr inbounds [2 x i32], [2 x i32]* %retval.full, i64 0, i64 0
   %retval = getelementptr inbounds [2 x i32], [2 x i32]* %retval.full, i64 0, i64 1
   br label %loop

 ; CHECK-NEXT:    [[RETVAL_FULL:%.*]] = alloca [2 x i32], align 4
 ; CHECK-NEXT:    [[DOTFCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 0
 ; CHECK-NEXT:    store i32 0, i32* [[DOTFCA_0_GEP]], align 4
 ; CHECK-NEXT:    [[DOTFCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 1
 ; CHECK-NEXT:    store i32 0, i32* [[DOTFCA_1_GEP]], align 4
 ; CHECK-NEXT:    [[RETVAL:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i64 0, i64 1
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
-; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[RETVAL_FULL_PROM_SROA_2_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-NEXT:    [[LD:%.*]] = load i32, i32* [[ARRAYIDX]], align 4
-; CHECK-NEXT:    [[RDX:%.*]] = load i32, i32* [[RETVAL]], align 4
-; CHECK-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_FULL_PROM_SROA_2_0]], [[LD]]
 ; CHECK-NEXT:    store i32 [[RDX_INC]], i32* [[RETVAL]], align 4
 ; CHECK-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    [[I0:%.*]] = call i32 @user_of_alloca_with_multiple_args(i32* [[RETVAL]], i32* [[RETVAL]])
-; CHECK-NEXT:    [[I1_FCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 0
-; CHECK-NEXT:    [[I1_FCA_0_LOAD:%.*]] = load i32, i32* [[I1_FCA_0_GEP]], align 4
-; CHECK-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 [[I1_FCA_0_LOAD]], 0
-; CHECK-NEXT:    [[I1_FCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 1
-; CHECK-NEXT:    [[I1_FCA_1_LOAD:%.*]] = load i32, i32* [[I1_FCA_1_GEP]], align 4
-; CHECK-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[I1_FCA_1_LOAD]], 1
+; CHECK-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 0, 0
+; CHECK-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[RDX_INC]], 1
 ; CHECK-NEXT:    ret [2 x i32] [[I1_FCA_1_INSERT]]
 ;
 ; CHECK-OPAQUE-LABEL: @part_of_alloca_used_in_call_with_multiple_args(
 ; CHECK-OPAQUE-NEXT:  entry:
 ; CHECK-OPAQUE-NEXT:    [[RETVAL_FULL:%.*]] = alloca [2 x i32], align 4
 ; CHECK-OPAQUE-NEXT:    [[DOTFCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 0
 ; CHECK-OPAQUE-NEXT:    store i32 0, ptr [[DOTFCA_0_GEP]], align 4
 ; CHECK-OPAQUE-NEXT:    [[DOTFCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 1
 ; CHECK-OPAQUE-NEXT:    store i32 0, ptr [[DOTFCA_1_GEP]], align 4
 ; CHECK-OPAQUE-NEXT:    [[RETVAL:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i64 0, i64 1
 ; CHECK-OPAQUE-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-OPAQUE:       loop:
-; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[RETVAL_FULL_PROM_SROA_2_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-OPAQUE-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-OPAQUE-NEXT:    [[LD:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX:%.*]] = load i32, ptr [[RETVAL]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-OPAQUE-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_FULL_PROM_SROA_2_0]], [[LD]]
 ; CHECK-OPAQUE-NEXT:    store i32 [[RDX_INC]], ptr [[RETVAL]], align 4
 ; CHECK-OPAQUE-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-OPAQUE-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-OPAQUE-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK-OPAQUE:       exit:
 ; CHECK-OPAQUE-NEXT:    [[I0:%.*]] = call i32 @user_of_alloca_with_multiple_args(ptr [[RETVAL]], ptr [[RETVAL]])
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 0
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_LOAD:%.*]] = load i32, ptr [[I1_FCA_0_GEP]], align 4
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 [[I1_FCA_0_LOAD]], 0
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 1
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_LOAD:%.*]] = load i32, ptr [[I1_FCA_1_GEP]], align 4
-; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[I1_FCA_1_LOAD]], 1
+; CHECK-OPAQUE-NEXT:    [[I1_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 0, 0
+; CHECK-OPAQUE-NEXT:    [[I1_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I1_FCA_0_INSERT]], i32 [[RDX_INC]], 1
 ; CHECK-OPAQUE-NEXT:    ret [2 x i32] [[I1_FCA_1_INSERT]]
 ;
 entry:
   %retval.full = alloca [2 x i32], align 4
   store [2 x i32] zeroinitializer, [2 x i32]* %retval.full, align 4
   %retval = getelementptr inbounds [2 x i32], [2 x i32]* %retval.full, i64 0, i64 1
   br label %loop
 

 ; CHECK-NEXT:    store i32 0, i32* [[DOTFCA_0_GEP]], align 4
 ; CHECK-NEXT:    [[DOTFCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 1
 ; CHECK-NEXT:    store i32 0, i32* [[DOTFCA_1_GEP]], align 4
 ; CHECK-NEXT:    [[RETVAL_BASE:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i64 0, i64 0
 ; CHECK-NEXT:    [[RETVAL:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i64 0, i64 1
 ; CHECK-NEXT:    [[SOME_ANOTHER_ALLOCA:%.*]] = getelementptr inbounds [42 x i32], [42 x i32]* [[SOME_ANOTHER_ALLOCA_FULL]], i64 0, i64 0
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
-; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[RETVAL_FULL_PROM_SROA_2_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-NEXT:    [[LD:%.*]] = load i32, i32* [[ARRAYIDX]], align 4
-; CHECK-NEXT:    [[RDX:%.*]] = load i32, i32* [[RETVAL]], align 4
-; CHECK-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_FULL_PROM_SROA_2_0]], [[LD]]
 ; CHECK-NEXT:    store i32 [[RDX_INC]], i32* [[RETVAL]], align 4
 ; CHECK-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    [[I0:%.*]] = call i32 @user_of_alloca_with_multiple_args(i32* [[RETVAL]], i32* [[RETVAL_BASE]])
 ; CHECK-NEXT:    [[I1:%.*]] = call i32 @user_of_alloca_with_multiple_args(i32* [[RETVAL_BASE]], i32* [[RETVAL]])
 ; CHECK-NEXT:    [[I2:%.*]] = call i32 @capture_of_alloca(i32* [[SOME_ANOTHER_ALLOCA]])
-; CHECK-NEXT:    [[I3_FCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 0
-; CHECK-NEXT:    [[I3_FCA_0_LOAD:%.*]] = load i32, i32* [[I3_FCA_0_GEP]], align 4
-; CHECK-NEXT:    [[I3_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 [[I3_FCA_0_LOAD]], 0
-; CHECK-NEXT:    [[I3_FCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], [2 x i32]* [[RETVAL_FULL]], i32 0, i32 1
-; CHECK-NEXT:    [[I3_FCA_1_LOAD:%.*]] = load i32, i32* [[I3_FCA_1_GEP]], align 4
-; CHECK-NEXT:    [[I3_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I3_FCA_0_INSERT]], i32 [[I3_FCA_1_LOAD]], 1
+; CHECK-NEXT:    [[I3_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 0, 0
+; CHECK-NEXT:    [[I3_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I3_FCA_0_INSERT]], i32 [[RDX_INC]], 1
 ; CHECK-NEXT:    ret [2 x i32] [[I3_FCA_1_INSERT]]
 ;
 ; CHECK-OPAQUE-LABEL: @all_parts_of_alloca_used_in_calls_with_multiple_args(
 ; CHECK-OPAQUE-NEXT:  entry:
 ; CHECK-OPAQUE-NEXT:    [[RETVAL_FULL:%.*]] = alloca [2 x i32], align 4
 ; CHECK-OPAQUE-NEXT:    [[SOME_ANOTHER_ALLOCA_FULL:%.*]] = alloca [42 x i32], align 4
 ; CHECK-OPAQUE-NEXT:    [[DOTFCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 0
 ; CHECK-OPAQUE-NEXT:    store i32 0, ptr [[DOTFCA_0_GEP]], align 4
 ; CHECK-OPAQUE-NEXT:    [[DOTFCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 1
 ; CHECK-OPAQUE-NEXT:    store i32 0, ptr [[DOTFCA_1_GEP]], align 4
 ; CHECK-OPAQUE-NEXT:    [[RETVAL_BASE:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i64 0, i64 0
 ; CHECK-OPAQUE-NEXT:    [[RETVAL:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i64 0, i64 1
 ; CHECK-OPAQUE-NEXT:    [[SOME_ANOTHER_ALLOCA:%.*]] = getelementptr inbounds [42 x i32], ptr [[SOME_ANOTHER_ALLOCA_FULL]], i64 0, i64 0
 ; CHECK-OPAQUE-NEXT:    br label [[LOOP:%.*]]
 ; CHECK-OPAQUE:       loop:
-; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[RETVAL_FULL_PROM_SROA_2_0:%.*]] = phi i32 [ 0, [[ENTRY:%.*]] ], [ [[RDX_INC:%.*]], [[LOOP]] ]
+; CHECK-OPAQUE-NEXT:    [[INDVARS_IV:%.*]] = phi i64 [ 0, [[ENTRY]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-OPAQUE-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds i32, ptr [[DATA:%.*]], i64 [[INDVARS_IV]]
 ; CHECK-OPAQUE-NEXT:    [[LD:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX:%.*]] = load i32, ptr [[RETVAL]], align 4
-; CHECK-OPAQUE-NEXT:    [[RDX_INC:%.*]] = add nsw i32 [[RDX]], [[LD]]
+; CHECK-OPAQUE-NEXT:    [[RDX_INC]] = add nsw i32 [[RETVAL_FULL_PROM_SROA_2_0]], [[LD]]
 ; CHECK-OPAQUE-NEXT:    store i32 [[RDX_INC]], ptr [[RETVAL]], align 4
 ; CHECK-OPAQUE-NEXT:    [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
 ; CHECK-OPAQUE-NEXT:    [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[N:%.*]]
 ; CHECK-OPAQUE-NEXT:    br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
 ; CHECK-OPAQUE:       exit:
 ; CHECK-OPAQUE-NEXT:    [[I0:%.*]] = call i32 @user_of_alloca_with_multiple_args(ptr [[RETVAL]], ptr [[RETVAL_BASE]])
 ; CHECK-OPAQUE-NEXT:    [[I1:%.*]] = call i32 @user_of_alloca_with_multiple_args(ptr [[RETVAL_BASE]], ptr [[RETVAL]])
 ; CHECK-OPAQUE-NEXT:    [[I2:%.*]] = call i32 @capture_of_alloca(ptr [[SOME_ANOTHER_ALLOCA]])
-; CHECK-OPAQUE-NEXT:    [[I3_FCA_0_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 0
-; CHECK-OPAQUE-NEXT:    [[I3_FCA_0_LOAD:%.*]] = load i32, ptr [[I3_FCA_0_GEP]], align 4
-; CHECK-OPAQUE-NEXT:    [[I3_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 [[I3_FCA_0_LOAD]], 0
-; CHECK-OPAQUE-NEXT:    [[I3_FCA_1_GEP:%.*]] = getelementptr inbounds [2 x i32], ptr [[RETVAL_FULL]], i32 0, i32 1
-; CHECK-OPAQUE-NEXT:    [[I3_FCA_1_LOAD:%.*]] = load i32, ptr [[I3_FCA_1_GEP]], align 4
-; CHECK-OPAQUE-NEXT:    [[I3_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I3_FCA_0_INSERT]], i32 [[I3_FCA_1_LOAD]], 1
+; CHECK-OPAQUE-NEXT:    [[I3_FCA_0_INSERT:%.*]] = insertvalue [2 x i32] poison, i32 0, 0
+; CHECK-OPAQUE-NEXT:    [[I3_FCA_1_INSERT:%.*]] = insertvalue [2 x i32] [[I3_FCA_0_INSERT]], i32 [[RDX_INC]], 1
 ; CHECK-OPAQUE-NEXT:    ret [2 x i32] [[I3_FCA_1_INSERT]]
 ;
 entry:
   %retval.full = alloca [2 x i32], align 4
   %some.another.alloca.full = alloca [42 x i32], align 4
   store [2 x i32] zeroinitializer, [2 x i32]* %retval.full, align 4
   %retval.base = getelementptr inbounds [2 x i32], [2 x i32]* %retval.full, i64 0, i64 0
   %retval = getelementptr inbounds [2 x i32], [2 x i32]* %retval.full, i64 0, i64 1
exit:
   %i3 = load [2 x i32], [2 x i32]* %retval.full, align 4
   ret [2 x i32] %i3
 }
 
 define i32 @all_uses_of_alloca_are_calls(i32* %data, i64 %n) {
 ; CHECK-LABEL: @all_uses_of_alloca_are_calls(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[RETVAL:%.*]] = alloca i32, align 4
+; CHECK-NEXT:    [[RETVAL_PROM:%.*]] = alloca i32, align 4
 ; CHECK-NEXT:    [[TMP0:%.*]] = call i32 @user_of_alloca(i32* [[RETVAL]])
 ; CHECK-NEXT:    [[TMP1:%.*]] = call i32 @user_of_alloca(i32* [[RETVAL]])
 ; CHECK-NEXT:    ret i32 0
 ;
 ; CHECK-OPAQUE-LABEL: @all_uses_of_alloca_are_calls(
 ; CHECK-OPAQUE-NEXT:  entry:
 ; CHECK-OPAQUE-NEXT:    [[RETVAL:%.*]] = alloca i32, align 4
+; CHECK-OPAQUE-NEXT:    [[RETVAL_PROM:%.*]] = alloca i32, align 4
 ; CHECK-OPAQUE-NEXT:    [[TMP0:%.*]] = call i32 @user_of_alloca(ptr [[RETVAL]])
 ; CHECK-OPAQUE-NEXT:    [[TMP1:%.*]] = call i32 @user_of_alloca(ptr [[RETVAL]])
 ; CHECK-OPAQUE-NEXT:    ret i32 0
 ;
 entry:
   %retval = alloca i32, align 4
   call i32 @user_of_alloca(i32* %retval)
   call i32 @user_of_alloca(i32* %retval)

 ; CHECK-NEXT:    [[TMP0:%.*]] = bitcast i64* [[C]] to i8*
 ; CHECK-NEXT:    call void @llvm.lifetime.start.p0i8(i64 1, i8* [[TMP0]])
 ; CHECK-NEXT:    store i64 0, i64* [[C]], align 4
 ; CHECK-NEXT:    [[ARRAYDECAY:%.*]] = bitcast i64* [[C]] to i32*
 ; CHECK-NEXT:    br label [[IF_END:%.*]]
 ; CHECK:       if.end:
 ; CHECK-NEXT:    [[ADD_PTR:%.*]] = getelementptr inbounds i32, i32* [[ARRAYDECAY]], i64 1
 ; CHECK-NEXT:    [[TMP1:%.*]] = call i32 @user_of_alloca(i32* [[ADD_PTR]])
-; CHECK-NEXT:    [[LD:%.*]] = load i64, i64* [[C]], align 4
-; CHECK-NEXT:    ret i64 [[LD]]
+; CHECK-NEXT:    ret i64 0
 ;
 ; CHECK-OPAQUE-LABEL: @do_schedule_instrs_for_dce_after_fixups(
 ; CHECK-OPAQUE-NEXT:  entry:
 ; CHECK-OPAQUE-NEXT:    [[C:%.*]] = alloca i64, align 2
 ; CHECK-OPAQUE-NEXT:    [[TMP0:%.*]] = bitcast ptr [[C]] to ptr
 ; CHECK-OPAQUE-NEXT:    call void @llvm.lifetime.start.p0(i64 1, ptr [[TMP0]])
 ; CHECK-OPAQUE-NEXT:    store i64 0, ptr [[C]], align 4
 ; CHECK-OPAQUE-NEXT:    [[ARRAYDECAY:%.*]] = bitcast ptr [[C]] to ptr
 ; CHECK-OPAQUE-NEXT:    br label [[IF_END:%.*]]
 ; CHECK-OPAQUE:       if.end:
 ; CHECK-OPAQUE-NEXT:    [[ADD_PTR:%.*]] = getelementptr inbounds i32, ptr [[ARRAYDECAY]], i64 1
 ; CHECK-OPAQUE-NEXT:    [[TMP1:%.*]] = call i32 @user_of_alloca(ptr [[ADD_PTR]])
-; CHECK-OPAQUE-NEXT:    [[LD:%.*]] = load i64, ptr [[C]], align 4
-; CHECK-OPAQUE-NEXT:    ret i64 [[LD]]
+; CHECK-OPAQUE-NEXT:    ret i64 0
 ;
 entry:
   %c = alloca i64, align 2
   %0 = bitcast i64* %c to i8*
   call void @llvm.lifetime.start.p0i8(i64 1, i8* %0)
   store i64 0, i64* %c
   %arraydecay = bitcast i64* %c to i32*
   br label %if.end
entry:
   call void @byte_user_of_alloca(i8* %a)
   ret void
 }
 define i8 @dont_transform_load_only() {
 ; CHECK-LABEL: @dont_transform_load_only(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[A:%.*]] = alloca i8, align 1
 ; CHECK-NEXT:    call void @byte_user_of_alloca(i8* [[A]])
-; CHECK-NEXT:    [[R:%.*]] = load i8, i8* [[A]], align 1
-; CHECK-NEXT:    ret i8 [[R]]
+; CHECK-NEXT:    ret i8 undef
 ;
 ; CHECK-OPAQUE-LABEL: @dont_transform_load_only(
 ; CHECK-OPAQUE-NEXT:  entry:
 ; CHECK-OPAQUE-NEXT:    [[A:%.*]] = alloca i8, align 1
 ; CHECK-OPAQUE-NEXT:    call void @byte_user_of_alloca(ptr [[A]])
-; CHECK-OPAQUE-NEXT:    [[R:%.*]] = load i8, ptr [[A]], align 1
-; CHECK-OPAQUE-NEXT:    ret i8 [[R]]
+; CHECK-OPAQUE-NEXT:    ret i8 undef
 ;
 entry:
   %a = alloca i8
   call void @byte_user_of_alloca(i8* %a)
   %r = load i8, i8* %a
   ret i8 %r
 }
 define i8 @transform_load_and_store() {
 ; CHECK-LABEL: @transform_load_and_store(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[A:%.*]] = alloca i8, align 1
 ; CHECK-NEXT:    store i8 0, i8* [[A]], align 1
 ; CHECK-NEXT:    call void @byte_user_of_alloca(i8* [[A]])
-; CHECK-NEXT:    [[R:%.*]] = load i8, i8* [[A]], align 1
-; CHECK-NEXT:    ret i8 [[R]]
+; CHECK-NEXT:    ret i8 0
 ;
 ; CHECK-OPAQUE-LABEL: @transform_load_and_store(
 ; CHECK-OPAQUE-NEXT:  entry:
 ; CHECK-OPAQUE-NEXT:    [[A:%.*]] = alloca i8, align 1
 ; CHECK-OPAQUE-NEXT:    store i8 0, ptr [[A]], align 1
 ; CHECK-OPAQUE-NEXT:    call void @byte_user_of_alloca(ptr [[A]])
-; CHECK-OPAQUE-NEXT:    [[R:%.*]] = load i8, ptr [[A]], align 1
-; CHECK-OPAQUE-NEXT:    ret i8 [[R]]
+; CHECK-OPAQUE-NEXT:    ret i8 0
 ;
 entry:
   %a = alloca i8
   store i8 0, i8* %a
   call void @byte_user_of_alloca(i8* %a)
   %r = load i8, i8* %a
   ret i8 %r
 }