[InstCombine] Fold extractelement + vector GEP with one use

[david-arm]

Summary

We sometimes see code like this:

Case 1:

%gep = getelementptr i32, i32* %a, <2 x i64> %splat
%ext = extractelement <2 x i32*> %gep, i32 0

or this:

Case 2:

%gep = getelementptr i32, <4 x i32*> %a, i64 1
%ext = extractelement <4 x i32*> %gep, i32 0

where there is only one use of the GEP. In such cases it makes
sense to fold the two together such that we create a scalar GEP:

Case 1:

%ext = extractelement <2 x i64> %splat, i32 0
%gep = getelementptr i32, i32* %a, i64 %ext

Case 2:

%ext = extractelement <2 x i32*> %a, i32 0
%gep = getelementptr i32, i32* %ext, i64 1

This may create further folding opportunities as a result, i.e.
the extract of a splat vector can be completely eliminated. Also,
even for the general case where the vector operand is not a splat
it seems beneficial to create a scalar GEP and extract the scalar
element from the operand. Therefore, in this patch I've assumed
that a scalar GEP is always preferrable to a vector GEP and have
added code to unconditionally fold the extract + GEP.

I haven't added folds for the case when we have both a vector of
pointers and a vector of indices, since this would require
generating an additional extractelement operation.

Tests have been added here:

Transforms/InstCombine/gep-vector-indices.ll

[sdesmalen]

llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp

434

Isn't the type of GEP guaranteed to be of type VectorType?

440

Does this need to be limited to a vector of indices that is also a splat?

485

unnecessary change.

[CarolineConcatto]

Hey @david-arm,

I had a looks, but not too sure I have more comments than the ones I've added.

llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp

450

Can you add a message when asserting?

llvm/test/Transforms/InstCombine/gep-vector-indices.ll

93

Should we add a negative test for this too:
// 3. We have a vector of pointers and a vector of indices.

[david-arm]

• Tried to improve the commit message to explain why I've chosed to unconditionall apply the fold for all cases, not just splats.
• Added extra negative test for GEPs with vector of ptrs + vector of indices.
• Changed code to assume GEP result is always a VectorType.

[david-arm]

llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp

440

Sorry for not explaining this better in the previous commit message. I've tried to make it clear that I'm applying the fold unconditionall on the basis that in general a scalar GEP is still preferrable to a vector one.

[spatel]

llvm/test/Transforms/InstCombine/vec_demanded_elts-inseltpoison.ll

502

Remove stale comment.

511

IIUC, this is 1 of the 2 basic patterns that we want to transform.

It would be better to use different index values in the test though, so we can verify that the indexes are being translated as expected (for example, make the gep index an argument variable?).

If that is a good suggestion, please commit the test change as a preliminary patch (no review needed), so we just see the functional diff from this patch here.

llvm/test/Transforms/InstCombine/vec_demanded_elts.ll

511

This test file is the pre-poison equivalent of the other one, so see earlier test comments.

[david-arm]

llvm/test/Transforms/InstCombine/vec_demanded_elts-inseltpoison.ll

511

Hi @spatel, thanks for taking a look at the patch! I'm just not entirely clear what to do here to be honest - are you suggesting just changing gep_vbase_w_s_idx to take a variable index and use that instead of 1? Or are you thinking of adding another similar test that takes a variable index?

[spatel]

llvm/test/Transforms/InstCombine/vec_demanded_elts-inseltpoison.ll

511

Yes, I'd just change the function to take a variable index.
That generalizes the test without losing anything AFAICT.

[david-arm]

• Rebase.
• Removed TODO comments from tests.

[fhahn]

llvm/test/Transforms/InstCombine/gep-vector-indices.ll

93

Do we have test cases where the GEP has other users? And where the extractelement has a variable index?

[david-arm]

• Added two more negative tests.

[sdesmalen]

llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp

434

This condition feels a bit too restrictive, because it would avoid:

%gep = getelementptr i32, i32** %ptr, i32 0, <2 x i32> <i32 4, i32 4>
extractelement <2 x i32*> %gep, i32 0

to not be simplified.

450

You can probably avoid spelling out the three possibilities and just have a loop that extracts the operand if it's type is a VectorType, as long as you make sure only one of the operands is of type VectorType.

llvm/test/Transforms/InstCombine/gep-vector-indices.ll

11–13

nit: For fixed-width, this pattern is unnecessary and you can write <2 x i64> <i64 4, i64 4> directly in the GEP.

[david-arm]

• Generalised the optimisation to more than 2 GEP operands with the requirement that only one operand should be a vector.
• Tidied up the tests.

[david-arm]

llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp

450

I've changed this to a loop, but I left in some comments as examples of different cases we might hit as I thought that might be useful to explain what's going on?

[david-arm]

• After private discussion with @sdesmalen I've updated the patch to count the vector GEP operands using a lambda function with llvm::count_if.

[sdesmalen]

Other than my two nits, the patch looks good to me!

llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp

462–464

nit: I think this assert is unnecessary, because we can assume the incoming IR was legal to begin with.

llvm/test/Transforms/InstCombine/gep-vector-indices.ll

11

nit: is entry: needed? (I don't see it used in the other tests you had to fix in this patch)

[sdesmalen]

LGTM, thanks @david-arm!

llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp

if (auto *I = dyn_cast<Instruction>(SrcVec)) {
     if (auto *IE = dyn_cast<InsertElementInst>(I)) {
       // Extracting the inserted element?
       if (IE->getOperand(2) == Index)
         return replaceInstUsesWith(EI, IE->getOperand(1));
       // If the inserted and extracted elements are constants, they must not
       // be the same value, extract from the pre-inserted value instead.
       if (isa<Constant>(IE->getOperand(2)) && IndexC)
         return replaceOperand(EI, 0, IE->getOperand(0));
+    } else if (auto *GEP = dyn_cast<GetElementPtrInst>(I)) {
+      if (IndexC && GEP->hasOneUse() && GEP->getNumOperands() == 2) {
+        uint64_t IdxVal = IndexC->getZExtValue();
+        auto *VecType = cast<VectorType>(GEP->getType());
+        ElementCount EC = VecType->getElementCount();
+        if (IdxVal < EC.getKnownMinValue()) {
+          // Find out why we have a vector result - there are three
+          // possibilities:
+          //  1. We have a scalar pointer and a vector of indices, or
+          //  2. We have a vector of pointers and a scalar index, or
+          //  3. We have a vector of pointers and a vector of indices.
+          // Here we only consider combining for the first two cases, since for
+          // the last case the optimization is less obviously a win due to
+          // needing two extractelements instead of one.
+          Type *IdxType = GEP->getOperand(1)->getType();
+          Type *NewElType = nullptr;
+          Value *NewPtr = nullptr, *NewIdx = nullptr;
+          if (VecType->getElementType() == GEP->getPointerOperandType()) {
+            assert(isa<VectorType>(IdxType) &&
+                   EC == cast<VectorType>(IdxType)->getElementCount() &&
+                   "Expected 2nd GEP operand to be a vector of indices with "
+                   "same element count as the result");
+            // Extract the vector index and create a new GEP with a scalar
+            // result.
+            NewElType = cast<PointerType>(GEP->getPointerOperandType())
+                            ->getElementType();
+            NewPtr = GEP->getPointerOperand();
+            NewIdx = Builder.CreateExtractElement(GEP->getOperand(1), IndexC);
+          } else if (VecType == GEP->getPointerOperandType() &&
+                     IdxType->isIntegerTy()) {
+            // Extract the pointer and create a new GEP with a scalar result.
+            NewElType =
+                cast<PointerType>(VecType->getElementType())->getElementType();
+            NewPtr =
+                Builder.CreateExtractElement(GEP->getPointerOperand(), IndexC);
+            NewIdx = GEP->getOperand(1);
+          }
+          if (NewPtr) {
+            GetElementPtrInst *NewGEP =
+                GetElementPtrInst::Create(NewElType, NewPtr, {NewIdx});
+            NewGEP->setIsInBounds(GEP->isInBounds());
+            return NewGEP;
+          }
+        }
+      }
+      return nullptr;
     } else if (auto *SVI = dyn_cast<ShuffleVectorInst>(I)) {
       // If this is extracting an element from a shufflevector, figure out where
       // it came from and extract from the appropriate input element instead.
       // Restrict the following transformation to fixed-length vector.
       if (isa<FixedVectorType>(SVI->getType()) && isa<ConstantInt>(Index)) {
         int SrcIdx =
             SVI->getMaskValue(cast<ConstantInt>(Index)->getZExtValue());
         Value *Src;
         unsigned LHSWidth = cast<FixedVectorType>(SVI->getOperand(0)->getType())
                                 ->getNumElements();
 
         if (SrcIdx < 0)
           return replaceInstUsesWith(EI, UndefValue::get(EI.getType()));
         if (SrcIdx < (int)LHSWidth)
           Src = SVI->getOperand(0);

llvm/test/Transforms/InstCombine/gep-vector-indices.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; RUN: opt -instcombine %s -S | FileCheck %s
+
+define i32* @vector_splat_indices_v2i64_ext0(i32* %a) {
+; CHECK-LABEL: @vector_splat_indices_v2i64_ext0(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[RES:%.*]] = getelementptr i32, i32* [[A:%.*]], i64 4
+; CHECK-NEXT:    ret i32* [[RES]]
+;
+entry:
+  %tmp = insertelement <2 x i64> poison, i64 4, i32 0
+  %splatof4 = shufflevector <2 x i64> %tmp, <2 x i64> poison, <2 x i32> zeroinitializer
+  %gep = getelementptr i32, i32* %a, <2 x i64> %splatof4
+  %res = extractelement <2 x i32*> %gep, i32 0
+  ret i32* %res
+}
+
+define i32* @vector_splat_indices_nxv2i64_ext0(i32* %a) {
+; CHECK-LABEL: @vector_splat_indices_nxv2i64_ext0(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[RES:%.*]] = getelementptr inbounds i32, i32* [[A:%.*]], i64 extractelement (<vscale x 2 x i64> shufflevector (<vscale x 2 x i64> insertelement (<vscale x 2 x i64> poison, i64 4, i32 0), <vscale x 2 x i64> poison, <vscale x 2 x i32> zeroinitializer), i32 0)
+; CHECK-NEXT:    ret i32* [[RES]]
+;
+entry:
+  %tmp = insertelement <vscale x 2 x i64> poison, i64 4, i32 0
+  %splatof4 = shufflevector <vscale x 2 x i64> %tmp, <vscale x 2 x i64> poison, <vscale x 2 x i32> zeroinitializer
+  %gep = getelementptr inbounds i32, i32* %a, <vscale x 2 x i64> %splatof4
+  %res = extractelement <vscale x 2 x i32*> %gep, i32 0
+  ret i32* %res
+}
+
+define i32* @vector_indices_v2i64_ext0(i32* %a, <2 x i64> %indices) {
+; CHECK-LABEL: @vector_indices_v2i64_ext0(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[TMP0:%.*]] = extractelement <2 x i64> [[INDICES:%.*]], i32 0
+; CHECK-NEXT:    [[RES:%.*]] = getelementptr i32, i32* [[A:%.*]], i64 [[TMP0]]
+; CHECK-NEXT:    ret i32* [[RES]]
+;
+entry:
+  %gep = getelementptr i32, i32* %a, <2 x i64> %indices
+  %res = extractelement <2 x i32*> %gep, i32 0
+  ret i32* %res
+}
+
+define i32* @vector_indices_nxv1i64_ext0(i32* %a, <vscale x 1 x i64> %indices) {
+; CHECK-LABEL: @vector_indices_nxv1i64_ext0(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[TMP0:%.*]] = extractelement <vscale x 1 x i64> [[INDICES:%.*]], i32 0
+; CHECK-NEXT:    [[RES:%.*]] = getelementptr i32, i32* [[A:%.*]], i64 [[TMP0]]
+; CHECK-NEXT:    ret i32* [[RES]]
+;
+entry:
+  %gep = getelementptr i32, i32* %a, <vscale x 1 x i64> %indices
+  %res = extractelement <vscale x 1 x i32*> %gep, i32 0
+  ret i32* %res
+}
+
+
+define i32* @vector_splat_ptrs_v2i64_ext0(i32* %a, i64 %index) {
+; CHECK-LABEL: @vector_splat_ptrs_v2i64_ext0(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[RES:%.*]] = getelementptr i32, i32* [[A:%.*]], i64 [[INDEX:%.*]]
+; CHECK-NEXT:    ret i32* [[RES]]
+;
+entry:
+  %tmp = insertelement <2 x i32*> poison, i32* %a, i32 0
+  %splatofa = shufflevector <2 x i32*> %tmp, <2 x i32*> poison, <2 x i32> zeroinitializer
+  %gep = getelementptr i32, <2 x i32*> %splatofa, i64 %index
+  %res = extractelement <2 x i32*> %gep, i32 0
+  ret i32* %res
+}
+
+
+define i32* @vector_splat_ptrs_nxv2i64_ext0(i32* %a, i64 %index) {
+; CHECK-LABEL: @vector_splat_ptrs_nxv2i64_ext0(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[TMP:%.*]] = insertelement <vscale x 2 x i32*> poison, i32* [[A:%.*]], i32 0
+; CHECK-NEXT:    [[SPLATOFA:%.*]] = shufflevector <vscale x 2 x i32*> [[TMP]], <vscale x 2 x i32*> poison, <vscale x 2 x i32> zeroinitializer
+; CHECK-NEXT:    [[TMP0:%.*]] = extractelement <vscale x 2 x i32*> [[SPLATOFA]], i32 0
+; CHECK-NEXT:    [[RES:%.*]] = getelementptr i32, i32* [[TMP0]], i64 [[INDEX:%.*]]
+; CHECK-NEXT:    ret i32* [[RES]]
+;
+entry:
+  %tmp = insertelement <vscale x 2 x i32*> poison, i32* %a, i32 0
+  %splatofa = shufflevector <vscale x 2 x i32*> %tmp, <vscale x 2 x i32*> poison, <vscale x 2 x i32> zeroinitializer
+  %gep = getelementptr i32, <vscale x 2 x i32*> %splatofa, i64 %index
+  %res = extractelement <vscale x 2 x i32*> %gep, i32 0
+  ret i32* %res
+}
+
+
+; Negative tests
+
+define i32* @vector_indices_nxv2i64_ext3(i32* %a, <vscale x 2 x i64> %indices) {
+; CHECK-LABEL: @vector_indices_nxv2i64_ext3(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, i32* [[A:%.*]], <vscale x 2 x i64> [[INDICES:%.*]]
+; CHECK-NEXT:    [[RES:%.*]] = extractelement <vscale x 2 x i32*> [[GEP]], i32 3
+; CHECK-NEXT:    ret i32* [[RES]]
+;
+entry:
+  %gep = getelementptr i32, i32* %a, <vscale x 2 x i64> %indices
+  %res = extractelement <vscale x 2 x i32*> %gep, i32 3
+  ret i32* %res
+}
+
+define i32* @vector_ptrs_and_indices_ext0(<vscale x 4 x i32*> %a, <vscale x 4 x i64> %indices) {
+; CHECK-LABEL: @vector_ptrs_and_indices_ext0(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, <vscale x 4 x i32*> [[A:%.*]], <vscale x 4 x i64> [[INDICES:%.*]]
+; CHECK-NEXT:    [[RES:%.*]] = extractelement <vscale x 4 x i32*> [[GEP]], i32 0
+; CHECK-NEXT:    ret i32* [[RES]]
+;
+entry:
+  %gep = getelementptr i32, <vscale x 4 x i32*> %a, <vscale x 4 x i64> %indices
+  %res = extractelement <vscale x 4 x i32*> %gep, i32 0
+  ret i32* %res
+}

llvm/test/Transforms/InstCombine/vec_demanded_elts-inseltpoison.ll

;
   %t19 = bitcast <4 x float> %t12 to <4 x i32>
   %t21 = extractelement <4 x i32> %t19, i32 0
   ret i32 %t21
 }
 
 define void @get_image() nounwind {
 ; CHECK-LABEL: @get_image(
 ; CHECK-NEXT:  entry:
-; CHECK-NEXT:    [[TMP0:%.*]] = call i32 @fgetc(i8* null) [[ATTR0:#.*]]
+; CHECK-NEXT:    [[TMP0:%.*]] = call i32 @fgetc(i8* null) #[[ATTR0:[0-9]+]]
 ; CHECK-NEXT:    br i1 false, label [[BB2:%.*]], label [[BB3:%.*]]
 ; CHECK:       bb2:
 ; CHECK-NEXT:    br label [[BB3]]
 ; CHECK:       bb3:
 ; CHECK-NEXT:    unreachable
 ;
 entry:
   %0 = call i32 @fgetc(i8* null) nounwind

 ; CHECK-NEXT:    [[R:%.*]] = shufflevector <3 x float> [[BO]], <3 x float> poison, <3 x i32> <i32 1, i32 undef, i32 2>
 ; CHECK-NEXT:    ret <3 x float> [[R]]
 ;
   %bo = frem ninf reassoc <3 x float> %x, <float 1.0, float 2.0, float 3.0>
   %r = shufflevector <3 x float> %bo, <3 x float> poison, <3 x i32> <i32 1, i32 undef, i32 2>
   ret <3 x float> %r
 }
 
 ;; TODO: getelementptr tests below show missing simplifications for
 ;; vector demanded elements on vector geps.
 
 define i32* @gep_vbase_w_s_idx(<2 x i32*> %base) {
 ; CHECK-LABEL: @gep_vbase_w_s_idx(
-; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, <2 x i32*> [[BASE:%.*]], i64 1
-; CHECK-NEXT:    [[EE:%.*]] = extractelement <2 x i32*> [[GEP]], i32 1
+; CHECK-NEXT:    [[TMP1:%.*]] = extractelement <2 x i32*> [[BASE:%.*]], i32 1
+; CHECK-NEXT:    [[EE:%.*]] = getelementptr i32, i32* [[TMP1]], i64 1
 ; CHECK-NEXT:    ret i32* [[EE]]
 ;
   %gep = getelementptr i32, <2 x i32*> %base, i64 1
   %ee = extractelement <2 x i32*> %gep, i32 1
   ret i32* %ee
 }
 
 define i32* @gep_splat_base_w_s_idx(i32* %base) {
 ; CHECK-LABEL: @gep_splat_base_w_s_idx(
-; CHECK-NEXT:    [[BASEVEC2:%.*]] = insertelement <2 x i32*> poison, i32* [[BASE:%.*]], i32 1
-; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, <2 x i32*> [[BASEVEC2]], i64 1
-; CHECK-NEXT:    [[EE:%.*]] = extractelement <2 x i32*> [[GEP]], i32 1
+; CHECK-NEXT:    [[EE:%.*]] = getelementptr i32, i32* [[BASE:%.*]], i64 1
 ; CHECK-NEXT:    ret i32* [[EE]]
 ;
   %basevec1 = insertelement <2 x i32*> poison, i32* %base, i32 0
   %basevec2 = shufflevector <2 x i32*> %basevec1, <2 x i32*> poison, <2 x i32> zeroinitializer
   %gep = getelementptr i32, <2 x i32*> %basevec2, i64 1
   %ee = extractelement <2 x i32*> %gep, i32 1
   ret i32* %ee
 }
;
   ret i32* %ee
 }
 
 
 @GLOBAL = internal global i32 zeroinitializer
 
 define i32* @gep_cvbase_w_s_idx(<2 x i32*> %base, i64 %raw_addr) {
 ; CHECK-LABEL: @gep_cvbase_w_s_idx(
-; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, <2 x i32*> <i32* poison, i32* @GLOBAL>, i64 [[RAW_ADDR:%.*]]
-; CHECK-NEXT:    [[EE:%.*]] = extractelement <2 x i32*> [[GEP]], i32 1
+; CHECK-NEXT:    [[EE:%.*]] = getelementptr i32, i32* @GLOBAL, i64 [[RAW_ADDR:%.*]]
 ; CHECK-NEXT:    ret i32* [[EE]]
 ;
   %gep = getelementptr i32, <2 x i32*> <i32* @GLOBAL, i32* @GLOBAL>, i64 %raw_addr
   %ee = extractelement <2 x i32*> %gep, i32 1
   ret i32* %ee
 }
 
 define i32* @gep_cvbase_w_cv_idx(<2 x i32*> %base, i64 %raw_addr) {
 ; CHECK-LABEL: @gep_cvbase_w_cv_idx(
 ; CHECK-NEXT:    ret i32* getelementptr inbounds (i32, i32* @GLOBAL, i64 1)
 ;
   %gep = getelementptr i32, <2 x i32*> <i32* @GLOBAL, i32* @GLOBAL>, <2 x i64> <i64 0, i64 1>
   %ee = extractelement <2 x i32*> %gep, i32 1
   ret i32* %ee
 }
 
 
 define i32* @gep_sbase_w_cv_idx(i32* %base) {
 ; CHECK-LABEL: @gep_sbase_w_cv_idx(
-; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, i32* [[BASE:%.*]], <2 x i64> <i64 poison, i64 1>
-; CHECK-NEXT:    [[EE:%.*]] = extractelement <2 x i32*> [[GEP]], i32 1
+; CHECK-NEXT:    [[EE:%.*]] = getelementptr i32, i32* [[BASE:%.*]], i64 1
 ; CHECK-NEXT:    ret i32* [[EE]]
 ;
   %gep = getelementptr i32, i32* %base, <2 x i64> <i64 0, i64 1>
   %ee = extractelement <2 x i32*> %gep, i32 1
   ret i32* %ee
 }
 
 define i32* @gep_sbase_w_splat_idx(i32* %base, i64 %idx) {
 ; CHECK-LABEL: @gep_sbase_w_splat_idx(
-; CHECK-NEXT:    [[IDXVEC2:%.*]] = insertelement <2 x i64> poison, i64 [[IDX:%.*]], i32 1
-; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, i32* [[BASE:%.*]], <2 x i64> [[IDXVEC2]]
-; CHECK-NEXT:    [[EE:%.*]] = extractelement <2 x i32*> [[GEP]], i32 1
+; CHECK-NEXT:    [[EE:%.*]] = getelementptr i32, i32* [[BASE:%.*]], i64 [[IDX:%.*]]
 ; CHECK-NEXT:    ret i32* [[EE]]
 ;
   %idxvec1 = insertelement <2 x i64> poison, i64 %idx, i32 0
   %idxvec2 = shufflevector <2 x i64> %idxvec1, <2 x i64> poison, <2 x i32> zeroinitializer
   %gep = getelementptr i32, i32* %base, <2 x i64> %idxvec2
   %ee = extractelement <2 x i32*> %gep, i32 1
   ret i32* %ee
 }

llvm/test/Transforms/InstCombine/vec_demanded_elts.ll

;
   %t19 = bitcast <4 x float> %t12 to <4 x i32>
   %t21 = extractelement <4 x i32> %t19, i32 0
   ret i32 %t21
 }
 
 define void @get_image() nounwind {
 ; CHECK-LABEL: @get_image(
 ; CHECK-NEXT:  entry:
-; CHECK-NEXT:    [[TMP0:%.*]] = call i32 @fgetc(i8* null) [[ATTR0:#.*]]
+; CHECK-NEXT:    [[TMP0:%.*]] = call i32 @fgetc(i8* null) #[[ATTR0:[0-9]+]]
 ; CHECK-NEXT:    br i1 false, label [[BB2:%.*]], label [[BB3:%.*]]
 ; CHECK:       bb2:
 ; CHECK-NEXT:    br label [[BB3]]
 ; CHECK:       bb3:
 ; CHECK-NEXT:    unreachable
 ;
 entry:
   %0 = call i32 @fgetc(i8* null) nounwind
;
   ret <3 x float> %r
 }
 
 ;; TODO: getelementptr tests below show missing simplifications for
 ;; vector demanded elements on vector geps.
 
 define i32* @gep_vbase_w_s_idx(<2 x i32*> %base) {
 ; CHECK-LABEL: @gep_vbase_w_s_idx(
-; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, <2 x i32*> [[BASE:%.*]], i64 1
-; CHECK-NEXT:    [[EE:%.*]] = extractelement <2 x i32*> [[GEP]], i32 1
+; CHECK-NEXT:    [[TMP1:%.*]] = extractelement <2 x i32*> [[BASE:%.*]], i32 1
+; CHECK-NEXT:    [[EE:%.*]] = getelementptr i32, i32* [[TMP1]], i64 1
 ; CHECK-NEXT:    ret i32* [[EE]]
 ;
   %gep = getelementptr i32, <2 x i32*> %base, i64 1
   %ee = extractelement <2 x i32*> %gep, i32 1
   ret i32* %ee
 }
 
 define i32* @gep_splat_base_w_s_idx(i32* %base) {
 ; CHECK-LABEL: @gep_splat_base_w_s_idx(
-; CHECK-NEXT:    [[BASEVEC2:%.*]] = insertelement <2 x i32*> undef, i32* [[BASE:%.*]], i32 1
-; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, <2 x i32*> [[BASEVEC2]], i64 1
-; CHECK-NEXT:    [[EE:%.*]] = extractelement <2 x i32*> [[GEP]], i32 1
+; CHECK-NEXT:    [[EE:%.*]] = getelementptr i32, i32* [[BASE:%.*]], i64 1
 ; CHECK-NEXT:    ret i32* [[EE]]
 ;
   %basevec1 = insertelement <2 x i32*> undef, i32* %base, i32 0
   %basevec2 = shufflevector <2 x i32*> %basevec1, <2 x i32*> undef, <2 x i32> zeroinitializer
   %gep = getelementptr i32, <2 x i32*> %basevec2, i64 1
   %ee = extractelement <2 x i32*> %gep, i32 1
   ret i32* %ee
 }
;
   ret i32* %ee
 }
 
 
 @GLOBAL = internal global i32 zeroinitializer
 
 define i32* @gep_cvbase_w_s_idx(<2 x i32*> %base, i64 %raw_addr) {
 ; CHECK-LABEL: @gep_cvbase_w_s_idx(
-; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, <2 x i32*> <i32* poison, i32* @GLOBAL>, i64 [[RAW_ADDR:%.*]]
-; CHECK-NEXT:    [[EE:%.*]] = extractelement <2 x i32*> [[GEP]], i32 1
+; CHECK-NEXT:    [[EE:%.*]] = getelementptr i32, i32* @GLOBAL, i64 [[RAW_ADDR:%.*]]
 ; CHECK-NEXT:    ret i32* [[EE]]
 ;
   %gep = getelementptr i32, <2 x i32*> <i32* @GLOBAL, i32* @GLOBAL>, i64 %raw_addr
   %ee = extractelement <2 x i32*> %gep, i32 1
   ret i32* %ee
 }
 
 define i32* @gep_cvbase_w_cv_idx(<2 x i32*> %base, i64 %raw_addr) {
 ; CHECK-LABEL: @gep_cvbase_w_cv_idx(
 ; CHECK-NEXT:    ret i32* getelementptr inbounds (i32, i32* @GLOBAL, i64 1)
 ;
   %gep = getelementptr i32, <2 x i32*> <i32* @GLOBAL, i32* @GLOBAL>, <2 x i64> <i64 0, i64 1>
   %ee = extractelement <2 x i32*> %gep, i32 1
   ret i32* %ee
 }
 
 
 define i32* @gep_sbase_w_cv_idx(i32* %base) {
 ; CHECK-LABEL: @gep_sbase_w_cv_idx(
-; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, i32* [[BASE:%.*]], <2 x i64> <i64 poison, i64 1>
-; CHECK-NEXT:    [[EE:%.*]] = extractelement <2 x i32*> [[GEP]], i32 1
+; CHECK-NEXT:    [[EE:%.*]] = getelementptr i32, i32* [[BASE:%.*]], i64 1
 ; CHECK-NEXT:    ret i32* [[EE]]
 ;
   %gep = getelementptr i32, i32* %base, <2 x i64> <i64 0, i64 1>
   %ee = extractelement <2 x i32*> %gep, i32 1
   ret i32* %ee
 }
 
 define i32* @gep_sbase_w_splat_idx(i32* %base, i64 %idx) {
 ; CHECK-LABEL: @gep_sbase_w_splat_idx(
-; CHECK-NEXT:    [[IDXVEC2:%.*]] = insertelement <2 x i64> undef, i64 [[IDX:%.*]], i32 1
-; CHECK-NEXT:    [[GEP:%.*]] = getelementptr i32, i32* [[BASE:%.*]], <2 x i64> [[IDXVEC2]]
-; CHECK-NEXT:    [[EE:%.*]] = extractelement <2 x i32*> [[GEP]], i32 1
+; CHECK-NEXT:    [[EE:%.*]] = getelementptr i32, i32* [[BASE:%.*]], i64 [[IDX:%.*]]
 ; CHECK-NEXT:    ret i32* [[EE]]
 ;
   %idxvec1 = insertelement <2 x i64> undef, i64 %idx, i32 0
   %idxvec2 = shufflevector <2 x i64> %idxvec1, <2 x i64> undef, <2 x i32> zeroinitializer
   %gep = getelementptr i32, i32* %base, <2 x i64> %idxvec2
   %ee = extractelement <2 x i32*> %gep, i32 1
   ret i32* %ee
 }