SeparateConstOffsetFromGEP: Don't use SCEV

[arsenm]

Summary

This was only using the SCEV expressions as a map key, which we can
do just as well with the value pointers.

[aeubanks]

llvm/test/Transforms/SeparateConstOffsetFromGEP/AArch64/split-gep.ll

1 ↗

(On Diff #529629)

-verify-scev is on in expensive builds so I don't think it's necessary to add it to many tests. maybe we should turn on the other two as well in expensive builds?

[arsenm]

llvm/test/Transforms/SeparateConstOffsetFromGEP/AArch64/split-gep.ll

1 ↗

(On Diff #529629)

Currently can't turn them on because there are a lot of failures: https://github.com/llvm/llvm-project/issues/63195

[nikic]

llvm/test/Transforms/SeparateConstOffsetFromGEP/AArch64/split-gep.ll

1 ↗

(On Diff #529629)

Please do not use any SCEV verification flags other than -verify-scev.

[arsenm]

Drop test changes and rely on -verify-scev in EXPENSIVE_CHECKS

[nikic]

I think this is okay, but I just took a look at what this pass actually does with SCEV and ... wat.

As far as I can tell, SCEV here can be replaced by using a std::pair<Value *, Value *> instead of a SCEV expressions in the DominatingAdd/DominatingSub maps. Might be better to do that and not request SCEV at all?

[arsenm]

In D152456#4410228, @nikic wrote:

Might be better to do that and not request SCEV at all?

Possibly, but should be a separate patch. I'll file an issue to stop using SCEV here

[arsenm]

In D152456#4440452, @arsenm wrote:

In D152456#4410228, @nikic wrote:

Might be better to do that and not request SCEV at all?

Possibly, but should be a separate patch. I'll file an issue to stop using SCEV here

Went to describe the issue in the ticket and ended up with the patch which seems to work

[arsenm]

Don't use SCEV at all

[nikic]

llvm/lib/Transforms/Scalar/SeparateConstOffsetFromGEP.cpp

1235

I believe the previous code would also detect cases where the add operands are swapped as the same. Could add something like

static std::pair<Value *, Value *> createNormalizedPair(Value *A, Value *B) {
  if (A < B)
    return {A, B};
  return {B, A};
}

and use it for the add case to preserve that behavior. Though the fact that it's apparently untested is not reassuring...

[arsenm]

Commute add keys and rebase on new tests for reuniteExts

[arsenm]

llvm/lib/Transforms/Scalar/SeparateConstOffsetFromGEP.cpp

1235

Most the code in the pass has sparse testing. I've rebased on a new test which showed this broke

1262

Wasn't sure about trying to preserve this behavior. Seems like it would only fire for constant cases, where you would expect the extension of constant to fold to a larger constant which wasn't pattern matched before

[nikic]

LGTM

Nice to see that this also adds vector support as a side effect.

llvm/lib/Transforms/Scalar/SeparateConstOffsetFromGEP.cpp

1262

This doesn't constant fold either due to use of SCEVUnknown. The reason for the getNegativeSCEV call here is to prevent the commutation behavior for subs. In your new implementation it's no longer needed.

[arsenm]

f2596b754c2f286cc4df2feff27b3e14a88d4f7b

llvm/lib/Transforms/Scalar/SeparateConstOffsetFromGEP.cpp

 
 #include "llvm/Transforms/Scalar/SeparateConstOffsetFromGEP.h"
 #include "llvm/ADT/APInt.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/DepthFirstIterator.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/Analysis/LoopInfo.h"
 #include "llvm/Analysis/MemoryBuiltins.h"
-#include "llvm/Analysis/ScalarEvolution.h"
 #include "llvm/Analysis/TargetLibraryInfo.h"
 #include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/Analysis/ValueTracking.h"
 #include "llvm/IR/BasicBlock.h"
 #include "llvm/IR/Constant.h"
 #include "llvm/IR/Constants.h"
 #include "llvm/IR/DataLayout.h"
 #include "llvm/IR/DerivedTypes.h"
public:
   SeparateConstOffsetFromGEPLegacyPass(bool LowerGEP = false)
       : FunctionPass(ID), LowerGEP(LowerGEP) {
     initializeSeparateConstOffsetFromGEPLegacyPassPass(
         *PassRegistry::getPassRegistry());
   }
 
   void getAnalysisUsage(AnalysisUsage &AU) const override {
     AU.addRequired<DominatorTreeWrapperPass>();
-    AU.addRequired<ScalarEvolutionWrapperPass>();
     AU.addRequired<TargetTransformInfoWrapperPass>();
     AU.addRequired<LoopInfoWrapperPass>();
     AU.setPreservesCFG();
     AU.addRequired<TargetLibraryInfoWrapperPass>();
   }
 
   bool runOnFunction(Function &F) override;
 
 private:
   bool LowerGEP;
 };
 
 /// A pass that tries to split every GEP in the function into a variadic
 /// base and a constant offset. It is a FunctionPass because searching for the
 /// constant offset may inspect other basic blocks.
 class SeparateConstOffsetFromGEP {
 public:
   SeparateConstOffsetFromGEP(
-      DominatorTree *DT, ScalarEvolution *SE, LoopInfo *LI,
-      TargetLibraryInfo *TLI,
+      DominatorTree *DT, LoopInfo *LI, TargetLibraryInfo *TLI,
       function_ref<TargetTransformInfo &(Function &)> GetTTI, bool LowerGEP)
-      : DT(DT), SE(SE), LI(LI), TLI(TLI), GetTTI(GetTTI), LowerGEP(LowerGEP) {}
+      : DT(DT), LI(LI), TLI(TLI), GetTTI(GetTTI), LowerGEP(LowerGEP) {}
 
   bool run(Function &F);
 
 private:
+  /// Track the operands of an add or sub.
+  using ExprKey = std::pair<Value *, Value *>;
+
+  /// Create a pair for use as a map key for a commutable operation.
+  static ExprKey createNormalizedCommutablePair(Value *A, Value *B) {
+    if (A < B)
+      return {A, B};
+    return {B, A};
+  }
+
   /// Tries to split the given GEP into a variadic base and a constant offset,
   /// and returns true if the splitting succeeds.
   bool splitGEP(GetElementPtrInst *GEP);
 
   /// Lower a GEP with multiple indices into multiple GEPs with a single index.
   /// Function splitGEP already split the original GEP into a variadic part and
   /// a constant offset (i.e., AccumulativeByteOffset). This function lowers the
   /// variadic part into a set of GEPs with a single index and applies
private:
   ///   => reunion                 &a[sext(i +nsw j)] + 5
   bool reuniteExts(Function &F);
 
   /// A helper that reunites sexts in an instruction.
   bool reuniteExts(Instruction *I);
 
   /// Find the closest dominator of <Dominatee> that is equivalent to <Key>.
   Instruction *findClosestMatchingDominator(
-      const SCEV *Key, Instruction *Dominatee,
-      DenseMap<const SCEV *, SmallVector<Instruction *, 2>> &DominatingExprs);
+      ExprKey Key, Instruction *Dominatee,
+      DenseMap<ExprKey, SmallVector<Instruction *, 2>> &DominatingExprs);
 
   /// Verify F is free of dead code.
   void verifyNoDeadCode(Function &F);
 
   bool hasMoreThanOneUseInLoop(Value *v, Loop *L);
 
   // Swap the index operand of two GEP.
   void swapGEPOperand(GetElementPtrInst *First, GetElementPtrInst *Second);
 
   // Check if it is safe to swap operand of two GEP.
   bool isLegalToSwapOperand(GetElementPtrInst *First, GetElementPtrInst *Second,
                             Loop *CurLoop);
 
   const DataLayout *DL = nullptr;
   DominatorTree *DT = nullptr;
-  ScalarEvolution *SE;
   LoopInfo *LI;
   TargetLibraryInfo *TLI;
   // Retrieved lazily since not always used.
   function_ref<TargetTransformInfo &(Function &)> GetTTI;
 
   /// Whether to lower a GEP with multiple indices into arithmetic operations or
   /// multiple GEPs with a single index.
   bool LowerGEP;
 
-  DenseMap<const SCEV *, SmallVector<Instruction *, 2>> DominatingAdds;
-  DenseMap<const SCEV *, SmallVector<Instruction *, 2>> DominatingSubs;
+  DenseMap<ExprKey, SmallVector<Instruction *, 2>> DominatingAdds;
+  DenseMap<ExprKey, SmallVector<Instruction *, 2>> DominatingSubs;
 };
 
 } // end anonymous namespace
 
 char SeparateConstOffsetFromGEPLegacyPass::ID = 0;
 
 INITIALIZE_PASS_BEGIN(
     SeparateConstOffsetFromGEPLegacyPass, "separate-const-offset-from-gep",
bool SeparateConstOffsetFromGEP::splitGEP(GetElementPtrInst *GEP) {
 
   return true;
 }
 
 bool SeparateConstOffsetFromGEPLegacyPass::runOnFunction(Function &F) {
   if (skipFunction(F))
     return false;
   auto *DT = &getAnalysis<DominatorTreeWrapperPass>().getDomTree();
-  auto *SE = &getAnalysis<ScalarEvolutionWrapperPass>().getSE();
   auto *LI = &getAnalysis<LoopInfoWrapperPass>().getLoopInfo();
   auto *TLI = &getAnalysis<TargetLibraryInfoWrapperPass>().getTLI(F);
   auto GetTTI = [this](Function &F) -> TargetTransformInfo & {
     return this->getAnalysis<TargetTransformInfoWrapperPass>().getTTI(F);
   };
-  SeparateConstOffsetFromGEP Impl(DT, SE, LI, TLI, GetTTI, LowerGEP);
+  SeparateConstOffsetFromGEP Impl(DT, LI, TLI, GetTTI, LowerGEP);
   return Impl.run(F);
 }
 
 bool SeparateConstOffsetFromGEP::run(Function &F) {
   if (DisableSeparateConstOffsetFromGEP)
     return false;
 
   DL = &F.getParent()->getDataLayout();
bool SeparateConstOffsetFromGEP::run(Function &F) {
 
   if (VerifyNoDeadCode)
     verifyNoDeadCode(F);
 
   return Changed;
 }
 
 Instruction *SeparateConstOffsetFromGEP::findClosestMatchingDominator(
-    const SCEV *Key, Instruction *Dominatee,
-    DenseMap<const SCEV *, SmallVector<Instruction *, 2>> &DominatingExprs) {
+    ExprKey Key, Instruction *Dominatee,
+    DenseMap<ExprKey, SmallVector<Instruction *, 2>> &DominatingExprs) {
   auto Pos = DominatingExprs.find(Key);
   if (Pos == DominatingExprs.end())
     return nullptr;
 
   auto &Candidates = Pos->second;
   // Because we process the basic blocks in pre-order of the dominator tree, a
   // candidate that doesn't dominate the current instruction won't dominate any
   // future instruction either. Therefore, we pop it out of the stack. This
   // optimization makes the algorithm O(n).
   while (!Candidates.empty()) {
     Instruction *Candidate = Candidates.back();
     if (DT->dominates(Candidate, Dominatee))
       return Candidate;
     Candidates.pop_back();
   }
   return nullptr;
 }
 
 bool SeparateConstOffsetFromGEP::reuniteExts(Instruction *I) {
-  if (!SE->isSCEVable(I->getType()))
+  if (!I->getType()->isIntOrIntVectorTy())
     return false;
 
   //   Dom: LHS+RHS
   //   I: sext(LHS)+sext(RHS)
   // If Dom can't sign overflow and Dom dominates I, optimize I to sext(Dom).
   // TODO: handle zext
   Value *LHS = nullptr, *RHS = nullptr;
   if (match(I, m_Add(m_SExt(m_Value(LHS)), m_SExt(m_Value(RHS))))) {
     if (LHS->getType() == RHS->getType()) {
-      const SCEV *Key =
-          SE->getAddExpr(SE->getUnknown(LHS), SE->getUnknown(RHS));
+      ExprKey Key = createNormalizedCommutablePair(LHS, RHS);
       if (auto *Dom = findClosestMatchingDominator(Key, I, DominatingAdds)) {
         Instruction *NewSExt = new SExtInst(Dom, I->getType(), "", I);
         NewSExt->takeName(I);
         I->replaceAllUsesWith(NewSExt);
         RecursivelyDeleteTriviallyDeadInstructions(I);
         return true;
       }
     }
   } else if (match(I, m_Sub(m_SExt(m_Value(LHS)), m_SExt(m_Value(RHS))))) {
     if (LHS->getType() == RHS->getType()) {
-      const SCEV *Key = SE->getAddExpr(
-          SE->getUnknown(LHS), SE->getNegativeSCEV(SE->getUnknown(RHS)));
-      if (auto *Dom = findClosestMatchingDominator(Key, I, DominatingSubs)) {
+      if (auto *Dom =
+              findClosestMatchingDominator({LHS, RHS}, I, DominatingSubs)) {
         Instruction *NewSExt = new SExtInst(Dom, I->getType(), "", I);
         NewSExt->takeName(I);
         I->replaceAllUsesWith(NewSExt);
         RecursivelyDeleteTriviallyDeadInstructions(I);
         return true;
       }
     }
   }
 
   // Add I to DominatingExprs if it's an add/sub that can't sign overflow.
   if (match(I, m_NSWAdd(m_Value(LHS), m_Value(RHS)))) {
     if (programUndefinedIfPoison(I)) {
-      const SCEV *Key =
-          SE->getAddExpr(SE->getUnknown(LHS), SE->getUnknown(RHS));
+      ExprKey Key = createNormalizedCommutablePair(LHS, RHS);
       DominatingAdds[Key].push_back(I);
     }
   } else if (match(I, m_NSWSub(m_Value(LHS), m_Value(RHS)))) {
-    if (programUndefinedIfPoison(I)) {
-      const SCEV *Key = SE->getAddExpr(
-          SE->getUnknown(LHS), SE->getNegativeSCEV(SE->getUnknown(RHS)));
-      DominatingSubs[Key].push_back(I);
-    }
+    if (programUndefinedIfPoison(I))
+      DominatingSubs[{LHS, RHS}].push_back(I);
   }
   return false;
 }
 
 bool SeparateConstOffsetFromGEP::reuniteExts(Function &F) {
   bool Changed = false;
   DominatingAdds.clear();
   DominatingSubs.clear();
void SeparateConstOffsetFromGEPPass::printPipeline(
   if (LowerGEP)
     OS << "lower-gep";
   OS << '>';
 }
 
 PreservedAnalyses
 SeparateConstOffsetFromGEPPass::run(Function &F, FunctionAnalysisManager &AM) {
   auto *DT = &AM.getResult<DominatorTreeAnalysis>(F);
-  auto *SE = &AM.getResult<ScalarEvolutionAnalysis>(F);
   auto *LI = &AM.getResult<LoopAnalysis>(F);
   auto *TLI = &AM.getResult<TargetLibraryAnalysis>(F);
   auto GetTTI = [&AM](Function &F) -> TargetTransformInfo & {
     return AM.getResult<TargetIRAnalysis>(F);
   };
-  SeparateConstOffsetFromGEP Impl(DT, SE, LI, TLI, GetTTI, LowerGEP);
+  SeparateConstOffsetFromGEP Impl(DT, LI, TLI, GetTTI, LowerGEP);
   if (!Impl.run(F))
     return PreservedAnalyses::all();
   PreservedAnalyses PA;
   PA.preserveSet<CFGAnalyses>();
   return PA;
 }

llvm/test/CodeGen/AMDGPU/llc-pipeline.ll

 ; GCN-O1-OPTS-NEXT:    Lower uses of LDS variables from non-kernel functions
 ; GCN-O1-OPTS-NEXT:    FunctionPass Manager
 ; GCN-O1-OPTS-NEXT:      Infer address spaces
 ; GCN-O1-OPTS-NEXT:      Expand Atomic instructions
 ; GCN-O1-OPTS-NEXT:      AMDGPU Promote Alloca
 ; GCN-O1-OPTS-NEXT:      Dominator Tree Construction
 ; GCN-O1-OPTS-NEXT:      SROA
 ; GCN-O1-OPTS-NEXT:      Natural Loop Information
-; GCN-O1-OPTS-NEXT:      Scalar Evolution Analysis
 ; GCN-O1-OPTS-NEXT:      Split GEPs to a variadic base and a constant offset for better CSE
 ; GCN-O1-OPTS-NEXT:      Scalar Evolution Analysis
 ; GCN-O1-OPTS-NEXT:      Straight line strength reduction
 ; GCN-O1-OPTS-NEXT:      Early CSE
 ; GCN-O1-OPTS-NEXT:      Scalar Evolution Analysis
 ; GCN-O1-OPTS-NEXT:      Nary reassociation
 ; GCN-O1-OPTS-NEXT:      Early CSE
 ; GCN-O1-OPTS-NEXT:      Cycle Info Analysis

 ; GCN-O2-NEXT:    Lower uses of LDS variables from non-kernel functions
 ; GCN-O2-NEXT:    FunctionPass Manager
 ; GCN-O2-NEXT:      Infer address spaces
 ; GCN-O2-NEXT:      Expand Atomic instructions
 ; GCN-O2-NEXT:      AMDGPU Promote Alloca
 ; GCN-O2-NEXT:      Dominator Tree Construction
 ; GCN-O2-NEXT:      SROA
 ; GCN-O2-NEXT:      Natural Loop Information
-; GCN-O2-NEXT:      Scalar Evolution Analysis
 ; GCN-O2-NEXT:      Split GEPs to a variadic base and a constant offset for better CSE
 ; GCN-O2-NEXT:      Scalar Evolution Analysis
 ; GCN-O2-NEXT:      Straight line strength reduction
 ; GCN-O2-NEXT:      Early CSE
 ; GCN-O2-NEXT:      Scalar Evolution Analysis
 ; GCN-O2-NEXT:      Nary reassociation
 ; GCN-O2-NEXT:      Early CSE
 ; GCN-O2-NEXT:      Cycle Info Analysis

 ; GCN-O3-NEXT:    Lower uses of LDS variables from non-kernel functions
 ; GCN-O3-NEXT:    FunctionPass Manager
 ; GCN-O3-NEXT:      Infer address spaces
 ; GCN-O3-NEXT:      Expand Atomic instructions
 ; GCN-O3-NEXT:      AMDGPU Promote Alloca
 ; GCN-O3-NEXT:      Dominator Tree Construction
 ; GCN-O3-NEXT:      SROA
 ; GCN-O3-NEXT:      Natural Loop Information
-; GCN-O3-NEXT:      Scalar Evolution Analysis
 ; GCN-O3-NEXT:      Split GEPs to a variadic base and a constant offset for better CSE
 ; GCN-O3-NEXT:      Scalar Evolution Analysis
 ; GCN-O3-NEXT:      Straight line strength reduction
 ; GCN-O3-NEXT:      Basic Alias Analysis (stateless AA impl)
 ; GCN-O3-NEXT:      Function Alias Analysis Results
 ; GCN-O3-NEXT:      Memory Dependence Analysis
 ; GCN-O3-NEXT:      Lazy Branch Probability Analysis
 ; GCN-O3-NEXT:      Lazy Block Frequency Analysis

llvm/test/CodeGen/PowerPC/O3-pipeline.ll

 ; CHECK-NEXT:       Expand large div/rem
 ; CHECK-NEXT:       Expand large fp convert
 ; CHECK-NEXT:       Convert i1 constants to i32/i64 if they are returned
 ; CHECK-NEXT:       Expand Atomic instructions
 ; CHECK-NEXT:     PPC Lower MASS Entries
 ; CHECK-NEXT:     FunctionPass Manager
 ; CHECK-NEXT:       Dominator Tree Construction
 ; CHECK-NEXT:       Natural Loop Information
-; CHECK-NEXT:       Scalar Evolution Analysis
 ; CHECK-NEXT:       Split GEPs to a variadic base and a constant offset for better CSE
 ; CHECK-NEXT:       Early CSE
 ; CHECK-NEXT:       Basic Alias Analysis (stateless AA impl)
 ; CHECK-NEXT:       Function Alias Analysis Results
 ; CHECK-NEXT:       Memory SSA
 ; CHECK-NEXT:       Canonicalize natural loops
 ; CHECK-NEXT:       LCSSA Verifier
 ; CHECK-NEXT:       Loop-Closed SSA Form Pass

llvm/test/Transforms/SeparateConstOffsetFromGEP/AMDGPU/reunite-exts.ll

 ; --------------------------------------------------------------------
 
 ; Should fold out the 64-bit add
 define <2 x i64> @add_sext__dominating_add_nsw_vector(<2 x i32> %arg0, <2 x i32> %arg1) {
 ; CHECK-LABEL: define <2 x i64> @add_sext__dominating_add_nsw_vector
 ; CHECK-SAME: (<2 x i32> [[ARG0:%.*]], <2 x i32> [[ARG1:%.*]]) #[[ATTR0]] {
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[ADD_NSW:%.*]] = add nsw <2 x i32> [[ARG0]], [[ARG1]]
-; CHECK-NEXT:    [[ARG0_SEXT:%.*]] = sext <2 x i32> [[ARG0]] to <2 x i64>
-; CHECK-NEXT:    [[ARG1_SEXT:%.*]] = sext <2 x i32> [[ARG1]] to <2 x i64>
-; CHECK-NEXT:    [[ADD_SEXT:%.*]] = add <2 x i64> [[ARG0_SEXT]], [[ARG1_SEXT]]
+; CHECK-NEXT:    [[ADD_SEXT:%.*]] = sext <2 x i32> [[ADD_NSW]] to <2 x i64>
 ; CHECK-NEXT:    call void @use.v2i32(<2 x i32> [[ADD_NSW]])
 ; CHECK-NEXT:    ret <2 x i64> [[ADD_SEXT]]
 ;
 entry:
   %add.nsw = add nsw <2 x i32> %arg0, %arg1
   %arg0.sext = sext <2 x i32> %arg0 to <2 x i64>
   %arg1.sext = sext <2 x i32> %arg1 to <2 x i64>
   %add.sext = add <2 x i64> %arg0.sext, %arg1.sext
   call void @use.v2i32(<2 x i32> %add.nsw)
   ret <2 x i64> %add.sext
 }
 
 ; Should fold out the 64-bit sub
 define <2 x i64> @sub_sext__dominating_sub_nsw_vector(<2 x i32> %arg0, <2 x i32> %arg1) {
 ; CHECK-LABEL: define <2 x i64> @sub_sext__dominating_sub_nsw_vector
 ; CHECK-SAME: (<2 x i32> [[ARG0:%.*]], <2 x i32> [[ARG1:%.*]]) #[[ATTR0]] {
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[SUB_NSW:%.*]] = sub nsw <2 x i32> [[ARG0]], [[ARG1]]
-; CHECK-NEXT:    [[ARG0_SEXT:%.*]] = sext <2 x i32> [[ARG0]] to <2 x i64>
-; CHECK-NEXT:    [[ARG1_SEXT:%.*]] = sext <2 x i32> [[ARG1]] to <2 x i64>
-; CHECK-NEXT:    [[SUB_SEXT:%.*]] = sub <2 x i64> [[ARG0_SEXT]], [[ARG1_SEXT]]
+; CHECK-NEXT:    [[SUB_SEXT:%.*]] = sext <2 x i32> [[SUB_NSW]] to <2 x i64>
 ; CHECK-NEXT:    call void @use.v2i32(<2 x i32> [[SUB_NSW]])
 ; CHECK-NEXT:    ret <2 x i64> [[SUB_SEXT]]
 ;
 entry:
   %sub.nsw = sub nsw <2 x i32> %arg0, %arg1
   %arg0.sext = sext <2 x i32> %arg0 to <2 x i64>
   %arg1.sext = sext <2 x i32> %arg1 to <2 x i64>
   %sub.sext = sub <2 x i64> %arg0.sext, %arg1.sext
   call void @use.v2i32(<2 x i32> %sub.nsw)
   ret <2 x i64> %sub.sext
 }
 
 ; Should fold out the 64-bit add. The 64-bit add has commuted operands
 ; compared to the original.
 define <2 x i64> @add_sext__dominating_add_nsw_commuted_vector(<2 x i32> %arg0, <2 x i32> %arg1) {
 ; CHECK-LABEL: define <2 x i64> @add_sext__dominating_add_nsw_commuted_vector
 ; CHECK-SAME: (<2 x i32> [[ARG0:%.*]], <2 x i32> [[ARG1:%.*]]) #[[ATTR0]] {
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[ADD_NSW:%.*]] = add nsw <2 x i32> [[ARG0]], [[ARG1]]
-; CHECK-NEXT:    [[ARG0_SEXT:%.*]] = sext <2 x i32> [[ARG0]] to <2 x i64>
-; CHECK-NEXT:    [[ARG1_SEXT:%.*]] = sext <2 x i32> [[ARG1]] to <2 x i64>
-; CHECK-NEXT:    [[ADD_SEXT:%.*]] = add <2 x i64> [[ARG1_SEXT]], [[ARG0_SEXT]]
+; CHECK-NEXT:    [[ADD_SEXT:%.*]] = sext <2 x i32> [[ADD_NSW]] to <2 x i64>
 ; CHECK-NEXT:    call void @use.v2i32(<2 x i32> [[ADD_NSW]])
 ; CHECK-NEXT:    ret <2 x i64> [[ADD_SEXT]]
 ;
 entry:
   %add.nsw = add nsw <2 x i32> %arg0, %arg1
   %arg0.sext = sext <2 x i32> %arg0 to <2 x i64>
   %arg1.sext = sext <2 x i32> %arg1 to <2 x i64>
   %add.sext = add <2 x i64> %arg1.sext, %arg0.sext