[VectorCombine] Switch to using a worklist.

[fhahn]

Summary

This patch updates VectorCombine to use a worklist to allow iterative
simplifications where a combine enables other combines.

Suggested in D100302.

The main use case at the moment is foldSingleElementStore and
scalarizeLoadExtract working together to improve scalarization.

At the moment, this directly uses the instcombine worklist, which
provides the required infrastructure. Some functions related to the
worklist still include "IC" in the debug output. I can adjust that, if
we decide to re-use the worklist.

Note that we now also do not run SimplifyInstructionsInBlock on the
whole function if there have been changes. This means we fail to
remove/simplify instructions not related to any of the vector combines.
IMO this is fine, as simplifying the whole function seems more like a
workaround for not tracking the changed instructions.

Compile-time impact looks neutral:
NewPM-O3: +0.02%
NewPM-ReleaseThinLTO: -0.00%
NewPM-ReleaseLTO-g: -0.02%

http://llvm-compile-time-tracker.com/compare.php?from=52832cd917af00e2b9c6a9d1476ba79754dcabff&to=e66520a4637290550a945d528e3e59573485dd40&stat=instructions

[lebedev.ri]

Nice!
I'm not sure it is a good idea to just reuse part of another pass like that.
Perhaps InstCombineWorklist should be extracted+promoted to being more reusable?

[fhahn]

In D110171#3012460, @lebedev.ri wrote:

Nice!
I'm not sure it is a good idea to just reuse part of another pass like that.
Perhaps InstCombineWorklist should be extracted+promoted to being more reusable?

Yeah, it should probably be moved. I put up D110181 to move it to Utils and rename it to InstructionWorklist.

[lebedev.ri]

Hmm, but this still doesn't do instcombine-style whole-function reprocessing ("iteration"), but only reprocesses the ones explicitly added to worklist.
This is intentional?

[fhahn]

In D110171#3012911, @lebedev.ri wrote:

Hmm, but this still doesn't do instcombine-style whole-function reprocessing ("iteration"), but only reprocesses the ones explicitly added to worklist.
This is intentional?

At the moment, the first combining round is done by iterating over the function without the worklist. Only if something gets simplified, the modified instructions are added. Is this what you are referring to? I did not add all instructions to the worklist up front intentionally.

The number of combines is quite limited at the moment and I expect there are a lot of functions where no combine triggers at all. In those cases, adding everything to the worklist first seems a bit of unnecessary overhead.

[lebedev.ri]

In D110171#3012936, @fhahn wrote:

In D110171#3012911, @lebedev.ri wrote:

Hmm, but this still doesn't do instcombine-style whole-function reprocessing ("iteration"), but only reprocesses the ones explicitly added to worklist.
This is intentional?

At the moment, the first combining round is done by iterating over the function without the worklist. Only if something gets simplified, the modified instructions are added. Is this what you are referring to? I did not add all instructions to the worklist up front intentionally.

The number of combines is quite limited at the moment and I expect there are a lot of functions where no combine triggers at all. In those cases, adding everything to the worklist first seems a bit of unnecessary overhead.

Instcombine roughly does the following:

1. add all instructions into worklist
2. pop from worklist until it's empty
3. if made any changes, goto 1.

This essentially does 1. and 2., but not 3.
This may either be an oversight, or an intentional improvement that guarantees that in this pass we don't repeat instcombine's mistake
of relying on 3., but consistently add everything we want reprocessed into worklist.
Thus, i'm asking.

[nikic]

This may either be an oversight, or an intentional improvement that guarantees that in this pass we don't repeat instcombine's mistake
of relying on 3., but consistently add everything we want reprocessed into worklist.

I'd strongly suggest not to repeat InstCombine's mistakes :) Just the worklist iteration should be enough...

llvm/include/llvm/Transforms/Utils/InstructionWorklist.h

35 ↗

(On Diff #373953)

Drop this initializer, as initialized in ctor?

llvm/lib/Transforms/Vectorize/VectorCombine.cpp

108

Just Worklist.pushValue(&Old). The Instruction check is exactly what pushValue() does relative to push().

948

Probably not necessary, as replaceValue() adds the old instruction to the worklist for DCE purposes?

1074

Do we actually need the deferred worklist for VectorCombine (does it make a difference for any test cases)? InstCombine has some pretty complicated worklist management, it might be preferable to forgo it if we don't need it. (No strong opinion though.)

[fhahn]

Remove DbgPrefix, use pushValue, avoid using deferred worklist part.

In D110171#3013003, @nikic wrote:

This may either be an oversight, or an intentional improvement that guarantees that in this pass we don't repeat instcombine's mistake
of relying on 3., but consistently add everything we want reprocessed into worklist.

Thanks for elaborating! I think for now it is not worth re-processing the whole function, because the combines are very limited in scope. It would probably be best to make sure the correct instructions are added to the worklist instead, e.g. note the updates around line 522 which ensure we can clean up now-dead extracts.

[fhahn]

llvm/include/llvm/Transforms/Utils/InstructionWorklist.h

35 ↗

(On Diff #373953)

The DbgPrefix is gone now

llvm/lib/Transforms/Vectorize/VectorCombine.cpp

108

done thanks!

948

I here is a store, so it won't be trivially dead

1074

Not really, I removed the deferred handling .

[lebedev.ri]

In D110171#3013095, @fhahn wrote:

Remove DbgPrefix, use pushValue, avoid using deferred worklist part.

In D110171#3013003, @nikic wrote:

This may either be an oversight, or an intentional improvement that guarantees that in this pass we don't repeat instcombine's mistake
of relying on 3., but consistently add everything we want reprocessed into worklist.

Thanks for elaborating! I think for now it is not worth re-processing the whole function, because the combines are very limited in scope. It would probably be best to make sure the correct instructions are added to the worklist instead, e.g. note the updates around line 522 which ensure we can clean up now-dead extracts.

Aha, just checking. Sounds good to me.
Seems fine, but probably wait for @spatel.

llvm/lib/Transforms/Vectorize/VectorCombine.cpp

106

We also likely want to revisit users of the value of which we've just reduced the use-count.
But honestly this is quite fragile, because generally speaking we want to revisit all it's uses transitively.
https://bugs.llvm.org/show_bug.cgi?id=47238

[spatel]

Do you know if the compile-time diff remained the same after the changes from the initial draft? I know it's only a small diff, but I'm actually surprised given how little VectorCombine does. Maybe there's more vector code out there than I realized. :)

llvm/lib/Transforms/Vectorize/VectorCombine.cpp

1057

Nit: I'd name this "FoldInst" or "CombineInst" because I equate "Simplify" with "InstSimplify" and so not creating new values.

llvm/test/Transforms/VectorCombine/AArch64/load-extract-insert-store-scalarization.ll

21

Why change this test?

[fhahn]

Address comments.

In D110171#3013272, @spatel wrote:

Do you know if the compile-time diff remained the same after the changes from the initial draft? I know it's only a small diff, but I'm actually surprised given how little VectorCombine does. Maybe there's more vector code out there than I realized. :)

I took a closer look and it seems like the majority of the diff was coming from some other subtle restructuring of the code. I updated the patch to be more in inline with the original code. It looks mostly neutral now:

NewPM-O3: +0.02%
NewPM-ReleaseThinLTO: -0.00%
NewPM-ReleaseLTO-g: -0.02%

http://llvm-compile-time-tracker.com/compare.php?from=52832cd917af00e2b9c6a9d1476ba79754dcabff&to=e66520a4637290550a945d528e3e59573485dd40&stat=instructions

[fhahn]

llvm/lib/Transforms/Vectorize/VectorCombine.cpp

106

Do you think it's worth doing this in the current patch? Not sure if it is possible to come up with a test case with the current limited set of combines.

1057

Changed to FoldInst.

llvm/test/Transforms/VectorCombine/AArch64/load-extract-insert-store-scalarization.ll

21

This was an accidental change that sneaked in. Removed now.

[spatel]

LGTM

[lebedev.ri]

llvm/lib/Transforms/Vectorize/VectorCombine.cpp

106

I don't know.
I'm simply saying that by not reiterating over the whole function afterwards,
we are very much likely missing transforms.

[fhahn]

llvm/lib/Transforms/Vectorize/VectorCombine.cpp

106

Oh I see. I plan to add verification that re-running VectorCombine does not change the function as a follow-up. That should help surfacing such issues.

[lebedev.ri]

llvm/lib/Transforms/Vectorize/VectorCombine.cpp

106

Yep, that would be good, thanks!

llvm/lib/Transforms/Vectorize/VectorCombine.cpp

 #include "llvm/IR/IRBuilder.h"
 #include "llvm/IR/PatternMatch.h"
 #include "llvm/InitializePasses.h"
 #include "llvm/Pass.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Transforms/Utils/Local.h"
 #include "llvm/Transforms/Vectorize.h"
 
+#define DEBUG_TYPE "vector-combine"
+#include "llvm/Transforms/Utils/InstructionWorklist.h"
+
 using namespace llvm;
 using namespace llvm::PatternMatch;
 
-#define DEBUG_TYPE "vector-combine"
 STATISTIC(NumVecLoad, "Number of vector loads formed");
 STATISTIC(NumVecCmp, "Number of vector compares formed");
 STATISTIC(NumVecBO, "Number of vector binops formed");
 STATISTIC(NumVecCmpBO, "Number of vector compare + binop formed");
 STATISTIC(NumShufOfBitcast, "Number of shuffles moved after bitcast");
 STATISTIC(NumScalarBO, "Number of scalar binops formed");
 STATISTIC(NumScalarCmp, "Number of scalar compares formed");
 

 
 private:
   Function &F;
   IRBuilder<> Builder;
   const TargetTransformInfo &TTI;
   const DominatorTree &DT;
   AAResults &AA;
   AssumptionCache &AC;
+  InstructionWorklist Worklist;
 
   bool vectorizeLoadInsert(Instruction &I);
   ExtractElementInst *getShuffleExtract(ExtractElementInst *Ext0,
                                         ExtractElementInst *Ext1,
                                         unsigned PreferredExtractIndex) const;
   bool isExtractExtractCheap(ExtractElementInst *Ext0, ExtractElementInst *Ext1,
                              unsigned Opcode,
                              ExtractElementInst *&ConvertToShuffle,
                              unsigned PreferredExtractIndex);
   void foldExtExtCmp(ExtractElementInst *Ext0, ExtractElementInst *Ext1,
                      Instruction &I);
   void foldExtExtBinop(ExtractElementInst *Ext0, ExtractElementInst *Ext1,
                        Instruction &I);
   bool foldExtractExtract(Instruction &I);
   bool foldBitcastShuf(Instruction &I);
   bool scalarizeBinopOrCmp(Instruction &I);
   bool foldExtractedCmps(Instruction &I);
   bool foldSingleElementStore(Instruction &I);
   bool scalarizeLoadExtract(Instruction &I);
-};
-} // namespace
 
-static void replaceValue(Value &Old, Value &New) {
+  void replaceValue(Value &Old, Value &New) {
     Old.replaceAllUsesWith(&New);
     New.takeName(&Old);
+    if (auto *NewI = dyn_cast<Instruction>(&New)) {
+      Worklist.pushUsersToWorkList(*NewI);
+      Worklist.pushValue(NewI);
+    }
+    Worklist.pushValue(&Old);
   }
 
+  void eraseInstruction(Instruction &I) {
+    for (Value *Op : I.operands())
+      Worklist.pushValue(Op);
+    Worklist.remove(&I);
+    I.eraseFromParent();
+  }
+};
+} // namespace
+
 bool VectorCombine::vectorizeLoadInsert(Instruction &I) {
   // Match insert into fixed vector of scalar value.
   // TODO: Handle non-zero insert index.
   auto *Ty = dyn_cast<FixedVectorType>(I.getType());
   Value *Scalar;
   if (!Ty || !match(&I, m_InsertElt(m_Undef(), m_Value(Scalar), m_ZeroInt())) ||
       !Scalar->hasOneUse())
     return false;
else
       Ext1 = NewExtract;
   }
 
   if (Pred != CmpInst::BAD_ICMP_PREDICATE)
     foldExtExtCmp(Ext0, Ext1, I);
   else
     foldExtExtBinop(Ext0, Ext1, I);
 
+  Worklist.push(Ext0);
+  Worklist.push(Ext1);
   return true;
 }
 
 /// If this is a bitcast of a shuffle, try to bitcast the source vector to the
 /// destination type followed by shuffle. This can enable further transforms by
 /// moving bitcasts or shuffles together.
 bool VectorCombine::foldBitcastShuf(Instruction &I) {
   Value *V;
Value *GEP = Builder.CreateInBoundsGEP(
         {ConstantInt::get(Idx->getType(), 0), Idx});
     StoreInst *NSI = Builder.CreateStore(NewElement, GEP);
     NSI->copyMetadata(*SI);
     Align ScalarOpAlignment = computeAlignmentAfterScalarization(
         std::max(SI->getAlign(), Load->getAlign()), NewElement->getType(), Idx,
         DL);
     NSI->setAlignment(ScalarOpAlignment);
     replaceValue(I, *NSI);
-    // Need erasing the store manually.
-    I.eraseFromParent();
+    eraseInstruction(I);
     return true;
   }
 
   return false;
 }
 
 /// Try to scalarize vector loads feeding extractelement instructions.
 bool VectorCombine::scalarizeLoadExtract(Instruction &I) {
   Value *Ptr;
-  Value *Idx;
-  if (!match(&I, m_ExtractElt(m_Load(m_Value(Ptr)), m_Value(Idx))))
+  if (!match(&I, m_Load(m_Value(Ptr))))
     return false;
 
-  auto *LI = cast<LoadInst>(I.getOperand(0));
+  auto *LI = cast<LoadInst>(&I);
   const DataLayout &DL = I.getModule()->getDataLayout();
   if (LI->isVolatile() || !DL.typeSizeEqualsStoreSize(LI->getType()))
     return false;
 
   auto *FixedVT = dyn_cast<FixedVectorType>(LI->getType());
   if (!FixedVT)
     return false;
 
bool VectorCombine::run() {
   if (DisableVectorCombine)
     return false;
 
   // Don't attempt vectorization if the target does not support vectors.
   if (!TTI.getNumberOfRegisters(TTI.getRegisterClassForType(/*Vector*/ true)))
     return false;
 
   bool MadeChange = false;
+  auto SimplifyInst = [this](Instruction &I) {
+    Builder.SetInsertPoint(&I);
+    return vectorizeLoadInsert(I) || foldExtractExtract(I) ||
+           foldBitcastShuf(I) || scalarizeBinopOrCmp(I) ||
+           foldExtractedCmps(I) || scalarizeLoadExtract(I) ||
+           foldSingleElementStore(I);
+  };
   for (BasicBlock &BB : F) {
     // Ignore unreachable basic blocks.
     if (!DT.isReachableFromEntry(&BB))
       continue;
-    // Use early increment range so that we can erase instructions in loop.
-    for (Instruction &I : make_early_inc_range(BB)) {
-      if (isa<DbgInfoIntrinsic>(I))
-        continue;
-      Builder.SetInsertPoint(&I);
-      MadeChange |= vectorizeLoadInsert(I);
-      MadeChange |= foldExtractExtract(I);
-      MadeChange |= foldBitcastShuf(I);
-      MadeChange |= scalarizeBinopOrCmp(I);
-      MadeChange |= foldExtractedCmps(I);
-      MadeChange |= scalarizeLoadExtract(I);
-      MadeChange |= foldSingleElementStore(I);
-    }
+    for (Instruction &I : make_early_inc_range(BB.instructionsWithoutDebug()))
+      MadeChange |= SimplifyInst(I);
+
+  while (!Worklist.isEmpty()) {
+    Instruction *I = Worklist.removeOne();
+    if (!I)
+      continue;
+
+    if (isInstructionTriviallyDead(I)) {
+      eraseInstruction(*I);
+      continue;
     }
 
-  // We're done with transforms, so remove dead instructions.
-  if (MadeChange)
-    for (BasicBlock &BB : F)
-      SimplifyInstructionsInBlock(&BB);
+    MadeChange |= SimplifyInst(*I);
+  }
 
   return MadeChange;
 }
 
 // Pass manager boilerplate below here.
 
 namespace {
 class VectorCombineLegacyPass : public FunctionPass {

llvm/test/Transforms/PhaseOrdering/AArch64/matrix-extract-insert.ll

 ; CHECK-NEXT:    [[TMP3:%.*]] = bitcast [225 x double]* [[A:%.*]] to <225 x double>*
 ; CHECK-NEXT:    [[TMP4:%.*]] = getelementptr inbounds <225 x double>, <225 x double>* [[TMP3]], i64 0, i64 [[TMP1]]
 ; CHECK-NEXT:    [[MATRIXEXT:%.*]] = load double, double* [[TMP4]], align 8
 ; CHECK-NEXT:    [[CONV2:%.*]] = zext i32 [[I:%.*]] to i64
 ; CHECK-NEXT:    [[TMP5:%.*]] = add nuw nsw i64 [[TMP0]], [[CONV2]]
 ; CHECK-NEXT:    [[TMP6:%.*]] = icmp ult i64 [[TMP5]], 225
 ; CHECK-NEXT:    tail call void @llvm.assume(i1 [[TMP6]])
 ; CHECK-NEXT:    [[TMP7:%.*]] = bitcast [225 x double]* [[B:%.*]] to <225 x double>*
-; CHECK-NEXT:    [[TMP8:%.*]] = load <225 x double>, <225 x double>* [[TMP7]], align 8
-; CHECK-NEXT:    [[MATRIXEXT4:%.*]] = extractelement <225 x double> [[TMP8]], i64 [[TMP5]]
+; CHECK-NEXT:    [[TMP8:%.*]] = getelementptr inbounds <225 x double>, <225 x double>* [[TMP7]], i64 0, i64 [[TMP5]]
+; CHECK-NEXT:    [[MATRIXEXT4:%.*]] = load double, double* [[TMP8]], align 8
 ; CHECK-NEXT:    [[MUL:%.*]] = fmul double [[MATRIXEXT]], [[MATRIXEXT4]]
-; CHECK-NEXT:    [[MATRIXEXT7:%.*]] = extractelement <225 x double> [[TMP8]], i64 [[TMP1]]
-; CHECK-NEXT:    [[SUB:%.*]] = fsub double [[MATRIXEXT7]], [[MUL]]
 ; CHECK-NEXT:    [[TMP9:%.*]] = getelementptr inbounds <225 x double>, <225 x double>* [[TMP7]], i64 0, i64 [[TMP1]]
-; CHECK-NEXT:    store double [[SUB]], double* [[TMP9]], align 8
+; CHECK-NEXT:    [[MATRIXEXT7:%.*]] = load double, double* [[TMP9]], align 8
+; CHECK-NEXT:    [[SUB:%.*]] = fsub double [[MATRIXEXT7]], [[MUL]]
+; CHECK-NEXT:    [[TMP10:%.*]] = getelementptr inbounds <225 x double>, <225 x double>* [[TMP7]], i64 0, i64 [[TMP1]]
+; CHECK-NEXT:    store double [[SUB]], double* [[TMP10]], align 8
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %i.addr = alloca i32, align 4
   %k.addr = alloca i32, align 4
   %j.addr = alloca i32, align 4
   %A.addr = alloca [225 x double]*, align 8
   %B.addr = alloca [225 x double]*, align 8

llvm/test/Transforms/VectorCombine/AArch64/load-extract-insert-store-scalarization.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -vector-combine -S %s | FileCheck %s
 
 target triple = "arm64-apple-darwin"
 
 define void @load_extract_insert_store_const_idx(<225 x double>* %A) {
 ; CHECK-LABEL: @load_extract_insert_store_const_idx(
 ; CHECK-NEXT:  entry:
-; CHECK-NEXT:    [[LV:%.*]] = load <225 x double>, <225 x double>* [[A:%.*]], align 8
-; CHECK-NEXT:    [[EXT_0:%.*]] = extractelement <225 x double> [[LV]], i64 0
+; CHECK-NEXT:    [[TMP0:%.*]] = getelementptr inbounds <225 x double>, <225 x double>* [[A:%.*]], i32 0, i64 1
+; CHECK-NEXT:    [[EXT_0:%.*]] = load double, double* [[TMP0]], align 8
 ; CHECK-NEXT:    [[MUL:%.*]] = fmul double 2.000000e+01, [[EXT_0]]
-; CHECK-NEXT:    [[EXT_1:%.*]] = extractelement <225 x double> [[LV]], i64 1
+; CHECK-NEXT:    [[TMP1:%.*]] = getelementptr inbounds <225 x double>, <225 x double>* [[A]], i32 0, i64 1
+; CHECK-NEXT:    [[EXT_1:%.*]] = load double, double* [[TMP1]], align 8
 ; CHECK-NEXT:    [[SUB:%.*]] = fsub double [[EXT_1]], [[MUL]]
-; CHECK-NEXT:    [[TMP0:%.*]] = getelementptr inbounds <225 x double>, <225 x double>* [[A]], i64 0, i64 1
-; CHECK-NEXT:    store double [[SUB]], double* [[TMP0]], align 8
+; CHECK-NEXT:    [[TMP2:%.*]] = getelementptr inbounds <225 x double>, <225 x double>* [[A]], i64 0, i64 1
+; CHECK-NEXT:    store double [[SUB]], double* [[TMP2]], align 8
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %lv = load <225 x double>, <225 x double>* %A, align 8
-  %ext.0 = extractelement <225 x double> %lv, i64 0
+  %ext.0 = extractelement <225 x double> %lv, i64 1
   %mul = fmul double 20.0, %ext.0
   %ext.1 = extractelement <225 x double> %lv, i64 1
   %sub = fsub double %ext.1, %mul
   %ins = insertelement <225 x double> %lv, double %sub, i64 1
   store <225 x double> %ins, <225 x double>* %A, align 8
   ret void
 }
 
 define void @load_extract_insert_store_var_idx_assume_valid(i64 %idx.1, i64 %idx.2, <225 x double>* %A) {
 ; CHECK-LABEL: @load_extract_insert_store_var_idx_assume_valid(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[CMP_1:%.*]] = icmp ult i64 [[IDX_1:%.*]], 225
 ; CHECK-NEXT:    call void @llvm.assume(i1 [[CMP_1]])
 ; CHECK-NEXT:    [[CMP_2:%.*]] = icmp ult i64 [[IDX_2:%.*]], 225
 ; CHECK-NEXT:    call void @llvm.assume(i1 [[CMP_2]])
-; CHECK-NEXT:    [[LV:%.*]] = load <225 x double>, <225 x double>* [[A:%.*]], align 8
-; CHECK-NEXT:    [[EXT_0:%.*]] = extractelement <225 x double> [[LV]], i64 [[IDX_1]]
+; CHECK-NEXT:    [[TMP0:%.*]] = getelementptr inbounds <225 x double>, <225 x double>* [[A:%.*]], i32 0, i64 [[IDX_1]]
+; CHECK-NEXT:    [[EXT_0:%.*]] = load double, double* [[TMP0]], align 8
 ; CHECK-NEXT:    [[MUL:%.*]] = fmul double 2.000000e+01, [[EXT_0]]
-; CHECK-NEXT:    [[EXT_1:%.*]] = extractelement <225 x double> [[LV]], i64 [[IDX_2]]
+; CHECK-NEXT:    [[TMP1:%.*]] = getelementptr inbounds <225 x double>, <225 x double>* [[A]], i32 0, i64 [[IDX_2]]
+; CHECK-NEXT:    [[EXT_1:%.*]] = load double, double* [[TMP1]], align 8
 ; CHECK-NEXT:    [[SUB:%.*]] = fsub double [[EXT_1]], [[MUL]]
-; CHECK-NEXT:    [[TMP0:%.*]] = getelementptr inbounds <225 x double>, <225 x double>* [[A]], i64 0, i64 [[IDX_1]]
-; CHECK-NEXT:    store double [[SUB]], double* [[TMP0]], align 8
+; CHECK-NEXT:    [[TMP2:%.*]] = getelementptr inbounds <225 x double>, <225 x double>* [[A]], i64 0, i64 [[IDX_1]]
+; CHECK-NEXT:    store double [[SUB]], double* [[TMP2]], align 8
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %cmp.1 = icmp ult i64 %idx.1, 225
   call void @llvm.assume(i1 %cmp.1)
   %cmp.2 = icmp ult i64 %idx.2, 225
   call void @llvm.assume(i1 %cmp.2)
 

 ; CHECK-LABEL: @load_extract_insert_store_var_idx_assume_valid_in_dominating_block(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[CMP_1:%.*]] = icmp ult i64 [[IDX_1:%.*]], 225
 ; CHECK-NEXT:    call void @llvm.assume(i1 [[CMP_1]])
 ; CHECK-NEXT:    [[CMP_2:%.*]] = icmp ult i64 [[IDX_2:%.*]], 225
 ; CHECK-NEXT:    call void @llvm.assume(i1 [[CMP_2]])
 ; CHECK-NEXT:    br i1 [[C_1:%.*]], label [[LOOP:%.*]], label [[EXIT:%.*]]
 ; CHECK:       loop:
-; CHECK-NEXT:    [[LV:%.*]] = load <225 x double>, <225 x double>* [[A:%.*]], align 8
-; CHECK-NEXT:    [[EXT_0:%.*]] = extractelement <225 x double> [[LV]], i64 [[IDX_1]]
+; CHECK-NEXT:    [[TMP0:%.*]] = getelementptr inbounds <225 x double>, <225 x double>* [[A:%.*]], i32 0, i64 [[IDX_1]]
+; CHECK-NEXT:    [[EXT_0:%.*]] = load double, double* [[TMP0]], align 8
 ; CHECK-NEXT:    [[MUL:%.*]] = fmul double 2.000000e+01, [[EXT_0]]
-; CHECK-NEXT:    [[EXT_1:%.*]] = extractelement <225 x double> [[LV]], i64 [[IDX_2]]
+; CHECK-NEXT:    [[TMP1:%.*]] = getelementptr inbounds <225 x double>, <225 x double>* [[A]], i32 0, i64 [[IDX_2]]
+; CHECK-NEXT:    [[EXT_1:%.*]] = load double, double* [[TMP1]], align 8
 ; CHECK-NEXT:    [[SUB:%.*]] = fsub double [[EXT_1]], [[MUL]]
-; CHECK-NEXT:    [[TMP0:%.*]] = getelementptr inbounds <225 x double>, <225 x double>* [[A]], i64 0, i64 [[IDX_1]]
-; CHECK-NEXT:    store double [[SUB]], double* [[TMP0]], align 8
+; CHECK-NEXT:    [[TMP2:%.*]] = getelementptr inbounds <225 x double>, <225 x double>* [[A]], i64 0, i64 [[IDX_1]]
+; CHECK-NEXT:    store double [[SUB]], double* [[TMP2]], align 8
 ; CHECK-NEXT:    [[C_2:%.*]] = call i1 @cond()
 ; CHECK-NEXT:    br i1 [[C_2]], label [[LOOP]], label [[EXIT]]
 ; CHECK:       exit:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %cmp.1 = icmp ult i64 %idx.1, 225
   call void @llvm.assume(i1 %cmp.1)

llvm/test/Transforms/VectorCombine/X86/extract-binop-inseltpoison.ll

;
   %a23 = fadd float %a2, %a3
   call void @use_f32(float %a23)
   %v3 = insertelement <4 x float> %b, float %a23, i32 3
   ret <4 x float> %v3
 }
 
 define <4 x float> @PR34724(<4 x float> %a, <4 x float> %b) {
 ; CHECK-LABEL: @PR34724(
-; CHECK-NEXT:    [[SHIFT:%.*]] = shufflevector <4 x float> [[A:%.*]], <4 x float> poison, <4 x i32> <i32 undef, i32 undef, i32 3, i32 undef>
+; CHECK-NEXT:    [[A0:%.*]] = extractelement <4 x float> [[A:%.*]], i32 0
+; CHECK-NEXT:    [[A1:%.*]] = extractelement <4 x float> [[A]], i32 1
+; CHECK-NEXT:    [[SHIFT:%.*]] = shufflevector <4 x float> [[A]], <4 x float> poison, <4 x i32> <i32 undef, i32 undef, i32 3, i32 undef>
 ; CHECK-NEXT:    [[TMP1:%.*]] = fadd <4 x float> [[A]], [[SHIFT]]
 ; CHECK-NEXT:    [[A23:%.*]] = extractelement <4 x float> [[TMP1]], i32 2
 ; CHECK-NEXT:    [[SHIFT1:%.*]] = shufflevector <4 x float> [[B:%.*]], <4 x float> poison, <4 x i32> <i32 1, i32 undef, i32 undef, i32 undef>
 ; CHECK-NEXT:    [[TMP2:%.*]] = fadd <4 x float> [[B]], [[SHIFT1]]
 ; CHECK-NEXT:    [[B01:%.*]] = extractelement <4 x float> [[TMP2]], i32 0
 ; CHECK-NEXT:    [[SHIFT2:%.*]] = shufflevector <4 x float> [[B]], <4 x float> poison, <4 x i32> <i32 undef, i32 undef, i32 undef, i32 2>
 ; CHECK-NEXT:    [[TMP3:%.*]] = fadd <4 x float> [[SHIFT2]], [[B]]
 ; CHECK-NEXT:    [[B23:%.*]] = extractelement <4 x float> [[TMP3]], i64 3

llvm/test/Transforms/VectorCombine/X86/extract-binop.ll

;
   %a23 = fadd float %a2, %a3
   call void @use_f32(float %a23)
   %v3 = insertelement <4 x float> %b, float %a23, i32 3
   ret <4 x float> %v3
 }
 
 define <4 x float> @PR34724(<4 x float> %a, <4 x float> %b) {
 ; CHECK-LABEL: @PR34724(
-; CHECK-NEXT:    [[SHIFT:%.*]] = shufflevector <4 x float> [[A:%.*]], <4 x float> poison, <4 x i32> <i32 undef, i32 undef, i32 3, i32 undef>
+; CHECK-NEXT:    [[A0:%.*]] = extractelement <4 x float> [[A:%.*]], i32 0
+; CHECK-NEXT:    [[A1:%.*]] = extractelement <4 x float> [[A]], i32 1
+; CHECK-NEXT:    [[SHIFT:%.*]] = shufflevector <4 x float> [[A]], <4 x float> poison, <4 x i32> <i32 undef, i32 undef, i32 3, i32 undef>
 ; CHECK-NEXT:    [[TMP1:%.*]] = fadd <4 x float> [[A]], [[SHIFT]]
 ; CHECK-NEXT:    [[A23:%.*]] = extractelement <4 x float> [[TMP1]], i32 2
 ; CHECK-NEXT:    [[SHIFT1:%.*]] = shufflevector <4 x float> [[B:%.*]], <4 x float> poison, <4 x i32> <i32 1, i32 undef, i32 undef, i32 undef>
 ; CHECK-NEXT:    [[TMP2:%.*]] = fadd <4 x float> [[B]], [[SHIFT1]]
 ; CHECK-NEXT:    [[B01:%.*]] = extractelement <4 x float> [[TMP2]], i32 0
 ; CHECK-NEXT:    [[SHIFT2:%.*]] = shufflevector <4 x float> [[B]], <4 x float> poison, <4 x i32> <i32 undef, i32 undef, i32 undef, i32 2>
 ; CHECK-NEXT:    [[TMP3:%.*]] = fadd <4 x float> [[SHIFT2]], [[B]]
 ; CHECK-NEXT:    [[B23:%.*]] = extractelement <4 x float> [[TMP3]], i64 3

llvm/test/Transforms/VectorCombine/load-insert-store.ll

entry:
   store <16 x i8> %vecins, <16 x i8>* %q
   ret void
 }
 
 define void @insert_store_ptr_strip(<16 x i8>* %q, i8 zeroext %s) {
 ; CHECK-LABEL: @insert_store_ptr_strip(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[ADDR0:%.*]] = bitcast <16 x i8>* [[Q:%.*]] to <2 x i64>*
-; CHECK-NEXT:    [[TMP0:%.*]] = getelementptr inbounds <16 x i8>, <16 x i8>* [[Q]], i32 0, i32 3
+; CHECK-NEXT:    [[ADDR1:%.*]] = getelementptr <2 x i64>, <2 x i64>* [[ADDR0]], i64 0
+; CHECK-NEXT:    [[ADDR2:%.*]] = bitcast <2 x i64>* [[ADDR1]] to <16 x i8>*
+; CHECK-NEXT:    [[TMP0:%.*]] = getelementptr inbounds <16 x i8>, <16 x i8>* [[ADDR2]], i32 0, i32 3
 ; CHECK-NEXT:    store i8 [[S:%.*]], i8* [[TMP0]], align 1
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %0 = load <16 x i8>, <16 x i8>* %q
   %vecins = insertelement <16 x i8> %0, i8 %s, i32 3
   %addr0 = bitcast <16 x i8>* %q to <2 x i64>*
   %addr1 = getelementptr <2 x i64>, <2 x i64>* %addr0, i64 0