[TTI] Use users of GEP to guess access type in getGEPCost

[luke]

Summary

Currently getGEPCost uses the target type of the GEP as a heuristic for
the type that will be accessed, to pass onto isLegalAddressingMode.
Targets use this to work out if a GEP can then be folded into the
load/store instruction that uses the GEP.
For example, on RISC-V loads and stores can have an offset added to a
base register folded into a single instruction, so the following GEP is
free:

%p = getelementptr i32, ptr %base, i32 42 ; getInstructionCost = 0
%x = load i32, ptr %p ; getInstructionCost = 1
------------------------------------------------------------------------
lw t0, a0(42)

However vector loads and stores cannot have an offset folded into them,
so the following GEP is costed:

%p = getelementptr <2 x i32>, ptr %base, i32 42 ; getInstructionCost = 1
%x = load <2 x i32>, ptr %p ; getInstructionCost = 1
------------------------------------------------------------------------
addi a0, 42
vle32 v8, (a0)

The issue arises whenever there is a mismatch between the target type of
the GEP and the type that is actually accessed:

%p = getelementptr i32, ptr %base, i32 42 ; getInstructionCost = 0
%x = load <2 x i32>, ptr %p ; getInstructionCost = 1
------------------------------------------------------------------------
addi a0, 42
vle32 v8, (a0)

Even though this GEP will result in an add instruction, because TTI
thinks it's loading an i32, it will think it can be folded and not
charge for it.

The target type can become mismatched with the memory access during
transformations, noticeably during SLP where a scalar base pointer will
be reused to perform a vector load or store.

This patch adds an optional AccessType argument to getGEPCost which
allows the type of memory accessed by users to be passed in as a hint,
so that we can more accurately determine if the GEP can be folded into
its users.

If AccessType is not provided, getGEPCost falls back to the old
behaviour of using the PointeeType to guess the memory access type. This
can be revisited in a later patch.

Also for now, only GEPs with exactly one user use the access type hint.
Whilst we could look through all users and use all access types to
determine if we can fold the GEP, this patch avoids doing so to prevent
O(N) behaviour.

[luke]

llvm/include/llvm/Analysis/TargetTransformInfo.h

301

Not overly thrilled with introducing a struct just for this, nor with the constructors. Suggestions are very welcome here

llvm/lib/Analysis/TargetTransformInfo.cpp

224–255

I remember seeing this logic being duplicated in a few places across llvm, earlycse is the first one that comes to mind. It might be worthwhile addressing this in a follow up patch

llvm/test/Transforms/SLPVectorizer/RISCV/gep.ll

18

This is the main result from all the noise in the test cases.

[nikic]

You should be able to avoid the LICM changes by adjusting the code at https://github.com/llvm/llvm-project/blob/bfb7c99f3aeab09236adf1f684f7144f384c6dd7/llvm/lib/Transforms/Scalar/LICM.cpp#L1346-L1361 to only pass the users in the loop (and drop the separate user iteration that code does).

Something I'm concerned about here is that your default implementation is going to scan all users of the instruction, which means that the cost query is now O(n) instead of O(1). This is probably not acceptable for compile-time reasons. Can we get away with only inspecting a single user in that case and assume it is representative?

This change seems to cause some pretty large code size changes on x86 at least (https://llvm-compile-time-tracker.com/compare.php?from=b77265711b390a6905e8901694d75f4b3812c71a&to=113f1989fa7e469407942249b4086e3a28da2bf4&stat=size-text), so it seems like a pretty high impact change. I can't say whether those changes are good or bad though :)

[luke]

Only account for users in loop in LICM, fix inliner test

[luke]

In D149889#4319936, @nikic wrote:

You should be able to avoid the LICM changes by adjusting the code at https://github.com/llvm/llvm-project/blob/bfb7c99f3aeab09236adf1f684f7144f384c6dd7/llvm/lib/Transforms/Scalar/LICM.cpp#L1346-L1361 to only pass the users in the loop (and drop the separate user iteration that code does).

Thanks, just tried it and it seems to work.

Something I'm concerned about here is that your default implementation is going to scan all users of the instruction, which means that the cost query is now O(n) instead of O(1). This is probably not acceptable for compile-time reasons. Can we get away with only inspecting a single user in that case and assume it is representative?

Yeah I think that should still work for our SLP use case, sounds like a good compromise. Will try that and report back.

This change seems to cause some pretty large code size changes on x86 at least (https://llvm-compile-time-tracker.com/compare.php?from=b77265711b390a6905e8901694d75f4b3812c71a&to=113f1989fa7e469407942249b4086e3a28da2bf4&stat=size-text), so it seems like a pretty high impact change. I can't say whether those changes are good or bad though :)

Indeed, from a quick look it looks like it might be affecting the inliner?

[luke]

Only use the first user to avoid O(N) compile times.
Move the memory access type logic into Instruction.h in a dependent patch.

[luke]

Remove redundant empty check

[luke]

llvm/lib/Transforms/Scalar/LICM.cpp

1346–1354 ↗

(On Diff #522223)

@nikic I kept the O(N) behaviour here because it looks like we're already iterating through all the users below. But this area seems sensitive. Do we want O(1) behaviour in this block too?

[asb]

llvm/include/llvm/Analysis/TargetTransformInfo.h

303

Perhaps it's worth expanding the doc comment to explain how AccessTypes may be used? e.g. "Callers may pass in the access type of users of the GEP in order to allow a more accurate cost, for instance on targets where the legal addressing modes are different for different types."

Of course personal tastes on the amount of doc comments to provide can vary....

[luke]

llvm/include/llvm/Analysis/TargetTransformInfo.h

303

It definitely needs some documentation, thanks for pointing that out. The usage of AccessTypes can be quite subtle when it comes to users that aren't loads/stores

[luke]

Add documentation comments

[ABataev]

llvm/include/llvm/Analysis/TargetTransformInfoImpl.h

1009

Shall we use getArithmeticInstrCost(ADD) instead of TCC_Basic?

1070

Same question, cost of int add instead of basic?

[luke]

llvm/include/llvm/Analysis/TargetTransformInfoImpl.h

1070

I agree we should use getArithmeticInstrCost here, but I have some ideas about adding a new target hook to control this and was wondering if I could defer it to that patch?
On RISC-V if Scale != 1 then we will need an extra mul instruction, as opposed to aarch64 which can perform both an add and multiply in one. I think this would make sense to expose the address calculation cost as a hook since it's target dependent, and we already have all the information available here to pass onto TTI.

[ABataev]

llvm/include/llvm/Analysis/TargetTransformInfoImpl.h

1070

Add FIXME/TODOs here then

[luke]

Add TODO about modelling with arithmetic instr

[ABataev]

llvm/include/llvm/Analysis/TargetTransformInfoImpl.h

1009

return BaseGV ? TTI::TCC_Basic : TTI::TCC_Free;

[nikic]

llvm/include/llvm/Analysis/TargetTransformInfo.h

310

to determine

313

stray "which"?

316

ArrayRef should suffice?

llvm/include/llvm/Analysis/TargetTransformInfoImpl.h

1009

Why is the BaseGV case not free as well?

llvm/lib/Transforms/Scalar/LICM.cpp

1346–1354 ↗

(On Diff #522223)

It's okay, but you should drop the extra loop below. It's purpose is to essentially do the check you're doing here, just in a very crude way.

[luke]

Rebase and address review comments

[luke]

@nikic @ABataev thanks for the review, and sorry for the long delay here. I've rebased this and have run some benchmarks now that SLP is enabled by default for RISC-V.
I'm still seeing a reduction in the number of small unprofitable VFs on RISC-V with this patch, and it also seems to make a few more VF=8s profitable.
As a rough overview, here's the number vsetivlis on the benchmark before:

vsetivli : 467 total
1: 26
2: 247
3: 2
4: 169
5: 1
6: 0
7: 1
8: 17

And after:

vsetivli : 454 total
1: 24
2: 235
3: 2
4: 168
5: 1
6: 0
7: 1
8: 19

llvm/include/llvm/Analysis/TargetTransformInfoImpl.h

1009

This is just to match the behaviour of the previous if (Operands.empty()) check. I'll refactor this so that they share this logic

[reames]

A macro comment on the approach here. I think you're trying to do too much in a single change, and that makes it very hard to review.

I would advise doing the following:

• Remove the array of users API. Instead, pass a single optional access type.
• In TTIImpl, get the access type of the *sole user*. If there isn't a sole user, *do not pass one*.
• As a first step, use the access type in the legal addressing mode if available, and *otherwise the pointee type*. This is slightly wrong, but is wrong in the "it's very close to what we used to do" kinda of way which will reduce test deltas significantly. You can change this behavior in a follow up change if profitable.
• Add tests *via cost model* which demonstrate the change above. You should be able to (on RISCV) demonstrate the difference between access type of vector vs scalar.
• All of the other changes in this review - i.e. everything in lib/Transform - should be independent changes structured after the above.

[luke]

• Make AccessType just a single pointer
• Isolate changes to just cost model for now: will follow up with SLP and other users later
• Preserve original behaviour if accesstype isn't passed

[luke]

In D149889#4438606, @reames wrote:

A macro comment on the approach here. I think you're trying to do too much in a single change, and that makes it very hard to review.

I would advise doing the following:

• Remove the array of users API. Instead, pass a single optional access type.
• In TTIImpl, get the access type of the *sole user*. If there isn't a sole user, *do not pass one*.
• As a first step, use the access type in the legal addressing mode if available, and *otherwise the pointee type*. This is slightly wrong, but is wrong in the "it's very close to what we used to do" kinda of way which will reduce test deltas significantly. You can change this behavior in a follow up change if profitable.
• Add tests *via cost model* which demonstrate the change above. You should be able to (on RISCV) demonstrate the difference between access type of vector vs scalar.
• All of the other changes in this review - i.e. everything in lib/Transform - should be independent changes structured after the above.

Agreed, thanks for the pointers. I've split it up and the diff is much smaller now. Will follow up with other patches to do the lib/Transform changes.

[reames]

LGTM w/required changes.

In addition to the inline code comment, make sure you update your commit message. The current review description is out of sync with the code and you must fix that before landing.

llvm/lib/Target/X86/X86TargetTransformInfo.cpp

4956

Pass nullptr here so that you don't change the x86 behavior.

llvm/include/llvm/Analysis/TargetTransformInfo.h

 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_ANALYSIS_TARGETTRANSFORMINFO_H
 #define LLVM_ANALYSIS_TARGETTRANSFORMINFO_H
 
 #include "llvm/ADT/SmallBitVector.h"
 #include "llvm/IR/FMF.h"
 #include "llvm/IR/InstrTypes.h"
+#include "llvm/IR/IntrinsicInst.h"
 #include "llvm/IR/PassManager.h"
 #include "llvm/Pass.h"
 #include "llvm/Support/AtomicOrdering.h"
 #include "llvm/Support/BranchProbability.h"
 #include "llvm/Support/InstructionCost.h"
 #include <functional>
 #include <optional>
 #include <utility>
public:
   /// skipped by renaming the registers in the CPU, but they still are encoded
   /// and thus wouldn't be considered 'free' here.
   enum TargetCostConstants {
     TCC_Free = 0,     ///< Expected to fold away in lowering.
     TCC_Basic = 1,    ///< The cost of a typical 'add' instruction.
     TCC_Expensive = 4 ///< The cost of a 'div' instruction on x86.
   };
 
+  /// A representation of a user of a GEP operation.
+  ///
+  /// If AccessType is nullptr, then the user isn't a memory op and the GEP
+  /// can't be folded.
+  ///
+  /// If AccessType is a type, then it will be passed to isLegalAddressingMode
+  /// to determine if the GEP can possibly be folded.
+  struct GEPUser {
+    Type *AccessType;
+    /// Create a user which accesses memory of type \p Ty. Useful when
+    /// estimating the cost of a GEP when its users haven't been created yet.
+    GEPUser(Type *Ty);
+    /// Create a user from an already existing Value and determine the type of
+    /// memory it accesses, if any.
+    GEPUser(const Value *User);
+  };
+
   /// Estimate the cost of a GEP operation when lowered.
   InstructionCost
   getGEPCost(Type *PointeeType, const Value *Ptr,
-             ArrayRef<const Value *> Operands,
+             ArrayRef<const Value *> Operands, ArrayRef<GEPUser> Users,
              TargetCostKind CostKind = TCK_SizeAndLatency) const;
 
+  InstructionCost
+  getGEPCost(GetElementPtrInst *GEP,
+             TargetCostKind CostKind = TCK_SizeAndLatency) const {
+    SmallVector<GEPUser> Users(GEP->users());
+    SmallVector<const Value *> Ops(GEP->indices());
+    return getGEPCost(GEP->getSourceElementType(), GEP->getPointerOperand(),
+                      Ops, Users, CostKind);
+  }
+
   /// Describe known properties for a set of pointers.
   struct PointersChainInfo {
     /// All the GEPs in a set have same base address.
     unsigned IsSameBaseAddress : 1;
     /// These properties only valid if SameBaseAddress is set.
     /// True if distance between any two neigbouring pointers is same value.
     unsigned IsUniformStride : 1;
     /// True if distance between any two neigbouring pointers is a known value.

 };
 
 class TargetTransformInfo::Concept {
 public:
   virtual ~Concept() = 0;
   virtual const DataLayout &getDataLayout() const = 0;
   virtual InstructionCost getGEPCost(Type *PointeeType, const Value *Ptr,
                                      ArrayRef<const Value *> Operands,
+                                     ArrayRef<GEPUser> Users,
                                      TTI::TargetCostKind CostKind) = 0;
   virtual InstructionCost
   getPointersChainCost(ArrayRef<const Value *> Ptrs, const Value *Base,
                        const TTI::PointersChainInfo &Info,
                        TTI::TargetCostKind CostKind) = 0;
   virtual unsigned getInliningThresholdMultiplier() = 0;
   virtual unsigned adjustInliningThreshold(const CallBase *CB) = 0;
   virtual int getInlinerVectorBonusPercent() = 0;
public:
   ~Model() override = default;
 
   const DataLayout &getDataLayout() const override {
     return Impl.getDataLayout();
   }
 
   InstructionCost
   getGEPCost(Type *PointeeType, const Value *Ptr,
-             ArrayRef<const Value *> Operands,
+             ArrayRef<const Value *> Operands, ArrayRef<GEPUser> Users,
              TargetTransformInfo::TargetCostKind CostKind) override {
-    return Impl.getGEPCost(PointeeType, Ptr, Operands, CostKind);
+    return Impl.getGEPCost(PointeeType, Ptr, Operands, Users, CostKind);
   }
   InstructionCost getPointersChainCost(ArrayRef<const Value *> Ptrs,
                                        const Value *Base,
                                        const PointersChainInfo &Info,
                                        TargetCostKind CostKind) override {
     return Impl.getPointersChainCost(Ptrs, Base, Info, CostKind);
   }
   unsigned getInliningThresholdMultiplier() override {

llvm/include/llvm/Analysis/TargetTransformInfoImpl.h

public:
   // Provide value semantics. MSVC requires that we spell all of these out.
   TargetTransformInfoImplBase(const TargetTransformInfoImplBase &Arg) = default;
   TargetTransformInfoImplBase(TargetTransformInfoImplBase &&Arg) : DL(Arg.DL) {}
 
   const DataLayout &getDataLayout() const { return DL; }
 
   InstructionCost getGEPCost(Type *PointeeType, const Value *Ptr,
                              ArrayRef<const Value *> Operands,
+                             ArrayRef<TTI::GEPUser> Users,
                              TTI::TargetCostKind CostKind) const {
     // In the basic model, we just assume that all-constant GEPs will be folded
     // into their uses via addressing modes.
     for (const Value *Operand : Operands)
       if (!isa<Constant>(Operand))
         return TTI::TCC_Basic;
 
     return TTI::TCC_Free;

 protected:
   explicit TargetTransformInfoImplCRTPBase(const DataLayout &DL) : BaseT(DL) {}
 
 public:
   using BaseT::getGEPCost;
 
   InstructionCost getGEPCost(Type *PointeeType, const Value *Ptr,
                              ArrayRef<const Value *> Operands,
+                             ArrayRef<TTI::GEPUser> Users,
                              TTI::TargetCostKind CostKind) {
     assert(PointeeType && Ptr && "can't get GEPCost of nullptr");
     assert(cast<PointerType>(Ptr->getType()->getScalarType())
                ->isOpaqueOrPointeeTypeMatches(PointeeType) &&
            "explicit pointee type doesn't match operand's pointee type");
     auto *BaseGV = dyn_cast<GlobalValue>(Ptr->stripPointerCasts());
     bool HasBaseReg = (BaseGV == nullptr);
 
     auto PtrSizeBits = DL.getPointerTypeSizeInBits(Ptr->getType());
     APInt BaseOffset(PtrSizeBits, 0);
     int64_t Scale = 0;
 
     auto GTI = gep_type_begin(PointeeType, Operands);
     Type *TargetType = nullptr;
 
     // Handle the case where the GEP instruction has a single operand,
     // the basis, therefore TargetType is a nullptr.
     if (Operands.empty())
       return !BaseGV ? TTI::TCC_Free : TTI::TCC_Basic;
 
+    // If the GEP would have all zero indices (and it's not using a global
+    // value), then it's just a type cast and is free.
+    if (all_of(Operands, [](const Value *Op) {
+          if (auto *CI = dyn_cast<ConstantInt>(Op))
+            return CI->isZero();
+          return isa<ConstantAggregateZero>(Op);
+        }))
+      return !BaseGV ? TTI::TCC_Free : TTI::TCC_Basic;
+
     for (auto I = Operands.begin(); I != Operands.end(); ++I, ++GTI) {
       TargetType = GTI.getIndexedType();
       // We assume that the cost of Scalar GEP with constant index and the
       // cost of Vector GEP with splat constant index are the same.
       const ConstantInt *ConstIdx = dyn_cast<ConstantInt>(*I);
       if (!ConstIdx)
         if (auto Splat = getSplatValue(*I))
           ConstIdx = dyn_cast<ConstantInt>(Splat);
for (auto I = Operands.begin(); I != Operands.end(); ++I, ++GTI) {
           if (Scale != 0)
             // No addressing mode takes two scale registers.
             return TTI::TCC_Basic;
           Scale = ElementSize;
         }
       }
     }
 
-    if (static_cast<T *>(this)->isLegalAddressingMode(
-            TargetType, const_cast<GlobalValue *>(BaseGV),
-            BaseOffset.sextOrTrunc(64).getSExtValue(), HasBaseReg, Scale,
-            Ptr->getType()->getPointerAddressSpace()))
+    uint64_t BaseOffsetVal = BaseOffset.sextOrTrunc(64).getSExtValue();
+
+    // If all the users of this GEP are memory instructions where the GEP can be
+    // folded into them via a legal addressing mode, then it's free.
+    bool IsFoldable = all_of(Users, [&](auto &U) {
+      if (!U.AccessType)
+        return false;
+      return static_cast<T *>(this)->isLegalAddressingMode(
+          U.AccessType, const_cast<GlobalValue *>(BaseGV), BaseOffsetVal,
+          HasBaseReg, Scale, Ptr->getType()->getPointerAddressSpace());
+    });
+    if (IsFoldable)
       return TTI::TCC_Free;
     return TTI::TCC_Basic;
   }
 
   InstructionCost getPointersChainCost(ArrayRef<const Value *> Ptrs,
                                        const Value *Base,
                                        const TTI::PointersChainInfo &Info,
                                        TTI::TargetCostKind CostKind) {
     InstructionCost Cost = TTI::TCC_Free;
     // In the basic model we take into account GEP instructions only
     // (although here can come alloca instruction, a value, constants and/or
     // constant expressions, PHIs, bitcasts ... whatever allowed to be used as a
     // pointer). Typically, if Base is a not a GEP-instruction and all the
     // pointers are relative to the same base address, all the rest are
     // either GEP instructions, PHIs, bitcasts or constants. When we have same
     // base, we just calculate cost of each non-Base GEP as an ADD operation if
     // any their index is a non-const.
     // If no known dependecies between the pointers cost is calculated as a sum
     // of costs of GEP instructions.
     for (const Value *V : Ptrs) {
       const auto *GEP = dyn_cast<GetElementPtrInst>(V);
       if (!GEP)
         continue;
       if (Info.isSameBase() && V != Base) {
         if (GEP->hasAllConstantIndices())
           continue;
         Cost += static_cast<T *>(this)->getArithmeticInstrCost(
             Instruction::Add, GEP->getType(), CostKind,
             {TTI::OK_AnyValue, TTI::OP_None}, {TTI::OK_AnyValue, TTI::OP_None},
             std::nullopt);
       } else {
-        SmallVector<const Value *> Indices(GEP->indices());
+        SmallVector<TTI::GEPUser> Users(GEP->users());
+        SmallVector<const Value *> Ops(GEP->indices());
         Cost += static_cast<T *>(this)->getGEPCost(GEP->getSourceElementType(),
                                                    GEP->getPointerOperand(),
-                                                   Indices, CostKind);
+                                                   Ops, Users, CostKind);
       }
     }
     return Cost;
   }
 
   InstructionCost getInstructionCost(const User *U,
                                      ArrayRef<const Value *> Operands,
                                      TTI::TargetCostKind CostKind) {
InstructionCost getInstructionCost(const User *U,
     case Instruction::Freeze:
       return TTI::TCC_Free;
     case Instruction::Alloca:
       if (cast<AllocaInst>(U)->isStaticAlloca())
         return TTI::TCC_Free;
       break;
     case Instruction::GetElementPtr: {
       const auto *GEP = cast<GEPOperator>(U);
-      return TargetTTI->getGEPCost(GEP->getSourceElementType(),
-                                   GEP->getPointerOperand(),
-                                   Operands.drop_front(), CostKind);
+      SmallVector<TTI::GEPUser> Users(GEP->users());
+
+      // Note: don't use the base pointer + indices from GEP->operands(),
+      // because the operands passed to getInstructionCost may differ from those
+      // actually on the instruction.
+      const Value *Ptr = Operands.front();
+      ArrayRef<const Value *> Indices = Operands.drop_front(1);
+
+      return TargetTTI->getGEPCost(GEP->getSourceElementType(), Ptr, Indices,
+                                   Users, CostKind);
     }
     case Instruction::Add:
     case Instruction::FAdd:
     case Instruction::Sub:
     case Instruction::FSub:
     case Instruction::Mul:
     case Instruction::FMul:
     case Instruction::UDiv:

llvm/include/llvm/CodeGen/BasicTTIImpl.h

public:
 
   unsigned getRegUsageForType(Type *Ty) {
     EVT ETy = getTLI()->getValueType(DL, Ty);
     return getTLI()->getNumRegisters(Ty->getContext(), ETy);
   }
 
   InstructionCost getGEPCost(Type *PointeeType, const Value *Ptr,
                              ArrayRef<const Value *> Operands,
+                             ArrayRef<TTI::GEPUser> Users,
                              TTI::TargetCostKind CostKind) {
-    return BaseT::getGEPCost(PointeeType, Ptr, Operands, CostKind);
+    return BaseT::getGEPCost(PointeeType, Ptr, Operands, Users, CostKind);
   }
 
   unsigned getEstimatedNumberOfCaseClusters(const SwitchInst &SI,
                                             unsigned &JumpTableSize,
                                             ProfileSummaryInfo *PSI,
                                             BlockFrequencyInfo *BFI) {
     /// Try to find the estimated number of clusters. Note that the number of
     /// clusters identified in this function could be different from the actual

llvm/lib/Analysis/TargetTransformInfo.cpp

 TargetTransformInfo::adjustInliningThreshold(const CallBase *CB) const {
   return TTIImpl->adjustInliningThreshold(CB);
 }
 
 int TargetTransformInfo::getInlinerVectorBonusPercent() const {
   return TTIImpl->getInlinerVectorBonusPercent();
 }
 
-InstructionCost
-TargetTransformInfo::getGEPCost(Type *PointeeType, const Value *Ptr,
-                                ArrayRef<const Value *> Operands,
-                                TTI::TargetCostKind CostKind) const {
-  return TTIImpl->getGEPCost(PointeeType, Ptr, Operands, CostKind);
+TargetTransformInfo::GEPUser::GEPUser(Type *Ty) : AccessType(Ty) {}
+
+TargetTransformInfo::GEPUser::GEPUser(const Value *User) {
+  if (auto *L = dyn_cast<LoadInst>(User))
+    AccessType = L->getType();
+  else if (auto *S = dyn_cast<StoreInst>(User))
+    AccessType = S->getValueOperand()->getType();
+  else if (const IntrinsicInst *Intr = dyn_cast<IntrinsicInst>(User)) {
+    switch (Intr->getIntrinsicID()) {
+    case Intrinsic::masked_load:
+    case Intrinsic::masked_gather:
+    case Intrinsic::masked_expandload:
+    case Intrinsic::vp_load:
+    case Intrinsic::vp_gather:
+    case Intrinsic::experimental_vp_strided_load:
+      AccessType = Intr->getType();
+      break;
+    case Intrinsic::masked_store:
+    case Intrinsic::masked_scatter:
+    case Intrinsic::masked_compressstore:
+    case Intrinsic::vp_store:
+    case Intrinsic::vp_scatter:
+    case Intrinsic::experimental_vp_strided_store:
+      AccessType = Intr->getOperand(0)->getType();
+      break;
+    default:
+      AccessType = nullptr;
+      break;
+    }
+  } else
+    AccessType = nullptr;
+}
+
+InstructionCost TargetTransformInfo::getGEPCost(
+    Type *PointeeType, const Value *Ptr, ArrayRef<const Value *> Operands,
+    ArrayRef<GEPUser> Users, TTI::TargetCostKind CostKind) const {
+  return TTIImpl->getGEPCost(PointeeType, Ptr, Operands, Users, CostKind);
 }
 
 InstructionCost TargetTransformInfo::getPointersChainCost(
     ArrayRef<const Value *> Ptrs, const Value *Base,
     const TTI::PointersChainInfo &Info, TTI::TargetCostKind CostKind) const {
   assert((Base || !Info.isSameBase()) &&
          "If pointers have same base address it has to be provided.");
   return TTIImpl->getPointersChainCost(Ptrs, Base, Info, CostKind);

llvm/lib/Target/X86/X86TargetTransformInfo.cpp

 InstructionCost X86TTIImpl::getPointersChainCost(
     ArrayRef<const Value *> Ptrs, const Value *Base,
     const TTI::PointersChainInfo &Info, TTI::TargetCostKind CostKind) {
   if (Info.isSameBase() && Info.isKnownStride()) {
     // If all the pointers have known stride all the differences are translated
     // into constants. X86 memory addressing allows encoding it into
     // displacement. So we just need to take the base GEP cost.
     if (const auto *BaseGEP = dyn_cast<GetElementPtrInst>(Base)) {
+      SmallVector<TTI::GEPUser> Users(BaseGEP->users());
       SmallVector<const Value *> Indices(BaseGEP->indices());
       return getGEPCost(BaseGEP->getSourceElementType(),
-                        BaseGEP->getPointerOperand(), Indices, CostKind);
+                        BaseGEP->getPointerOperand(), Indices, Users, CostKind);
     }
     return TTI::TCC_Free;
   }
   return BaseT::getPointersChainCost(Ptrs, Base, Info, CostKind);
 }
 
 InstructionCost X86TTIImpl::getAddressComputationCost(Type *Ty,
                                                       ScalarEvolution *SE,

llvm/lib/Transforms/Scalar/NaryReassociate.cpp

if ((ResI = matchAndReassociateMinOrMax<umin_pred_ty>(I, OrigSCEV)) ||
         (ResI = matchAndReassociateMinOrMax<smax_pred_ty>(I, OrigSCEV)))
       return ResI;
 
   return nullptr;
 }
 
 static bool isGEPFoldable(GetElementPtrInst *GEP,
                           const TargetTransformInfo *TTI) {
-  SmallVector<const Value *, 4> Indices(GEP->indices());
-  return TTI->getGEPCost(GEP->getSourceElementType(), GEP->getPointerOperand(),
-                         Indices) == TargetTransformInfo::TCC_Free;
+  return TTI->getGEPCost(GEP) == TargetTransformInfo::TCC_Free;
 }
 
 Instruction *NaryReassociatePass::tryReassociateGEP(GetElementPtrInst *GEP) {
   // Not worth reassociating GEP if it is foldable.
   if (isGEPFoldable(GEP, TTI))
     return nullptr;
 
   gep_type_iterator GTI = gep_type_begin(*GEP);

llvm/lib/Transforms/Scalar/StraightLineStrengthReduce.cpp

return (Basis.Ins != C.Ins && // skip the same instruction
           DT->dominates(Basis.Ins->getParent(), C.Ins->getParent()) &&
           // They share the same base, stride, and candidate kind.
           Basis.Base == C.Base && Basis.Stride == C.Stride &&
           Basis.CandidateKind == C.CandidateKind);
 }
 
 static bool isGEPFoldable(GetElementPtrInst *GEP,
                           const TargetTransformInfo *TTI) {
-  SmallVector<const Value *, 4> Indices(GEP->indices());
-  return TTI->getGEPCost(GEP->getSourceElementType(), GEP->getPointerOperand(),
-                         Indices) == TargetTransformInfo::TCC_Free;
+  return TTI->getGEPCost(GEP) == TargetTransformInfo::TCC_Free;
 }
 
 // Returns whether (Base + Index * Stride) can be folded to an addressing mode.
 static bool isAddFoldable(const SCEV *Base, ConstantInt *Index, Value *Stride,
                           TargetTransformInfo *TTI) {
   // Index->getSExtValue() may crash if Index is wider than 64-bit.
   return Index->getBitWidth() <= 64 &&
          TTI->isLegalAddressingMode(Base->getType(), nullptr, 0, true,

llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp

if (isa<LoadInst, StoreInst>(VL0)) {
 
       if (PtrsRetainedInVecCode.size() == Ptrs.size()) {
         // If all pointers stay in vectorized code then we don't have
         // any savings on that.
         LLVM_DEBUG(dumpTreeCosts(E, 0, ScalarCost, ScalarCost,
                                  "Calculated GEPs cost for Tree"));
         return InstructionCost{TTI::TCC_Free};
       }
-      VecCost = TTI->getPointersChainCost(
-          PtrsRetainedInVecCode, BasePtr,
-          TTI::PointersChainInfo::getKnownNonUniformStrided(), CostKind);
     } else {
       // Case 1: Ptrs are the arguments of loads that we are going to transform
       // into masked gather load intrinsic.
       // All the scalar GEPs will be removed as a result of vectorization.
       // For any external uses of some lanes extract element instructions will
       // be generated (which cost is estimated separately).
       TTI::PointersChainInfo PtrsInfo =
           all_of(Ptrs,
                  [](const Value *V) {
                    auto *Ptr = dyn_cast<GetElementPtrInst>(V);
                    return Ptr && !Ptr->hasAllConstantIndices();
                  })
               ? TTI::PointersChainInfo::getNonUniformStrided()
               : TTI::PointersChainInfo::getKnownNonUniformStrided();
 
       ScalarCost = TTI->getPointersChainCost(Ptrs, BasePtr, PtrsInfo, CostKind);
-
-      // Remark: it not quite correct to use scalar GEP cost for a vector GEP,
-      // but it's not clear how to do that without having vector GEP arguments
-      // ready.
-      // Perhaps using just TTI::TCC_Free/TTI::TCC_Basic would be better option.
-      if (const auto *Base = dyn_cast<GetElementPtrInst>(BasePtr)) {
-        SmallVector<const Value *> Indices(Base->indices());
-        VecCost = TTI->getGEPCost(Base->getSourceElementType(),
-                                  Base->getPointerOperand(), Indices, CostKind);
-      }
+
+    if (auto *BaseGEP = dyn_cast<GEPOperator>(BasePtr)) {
+      SmallVector<const Value *> Indices(BaseGEP->indices());
+      VecCost = TTI->getGEPCost(BaseGEP->getSourceElementType(),
+                                BaseGEP->getPointerOperand(), Indices, {VecTy},
+                                CostKind);
     }
 
     LLVM_DEBUG(dumpTreeCosts(E, 0, VecCost, ScalarCost,
                              "Calculated GEPs cost for Tree"));
 
     return VecCost - ScalarCost;
   };
 

llvm/test/Analysis/CostModel/AMDGPU/gep.ll

 ; NOTE: Assertions have been autogenerated by utils/update_analyze_test_checks.py
 ; RUN: opt -passes="print<cost-model>" 2>&1 -disable-output -mtriple=amdgcn-unknown-amdhsa -mcpu=gfx1010 < %s | FileCheck -check-prefixes=ALL %s
 ; RUN: opt -passes="print<cost-model>" 2>&1 -disable-output -mtriple=amdgcn-unknown-amdhsa -mcpu=gfx90a < %s | FileCheck -check-prefixes=ALL %s
 ; RUN: opt -passes="print<cost-model>" 2>&1 -disable-output -mtriple=amdgcn-unknown-amdhsa -mcpu=gfx900 < %s | FileCheck -check-prefixes=ALL %s
 ; RUN: opt -passes="print<cost-model>" 2>&1 -disable-output -mtriple=amdgcn-unknown-amdhsa < %s | FileCheck -check-prefixes=ALL %s
 
 ; RUN: opt -passes="print<cost-model>" -cost-kind=code-size 2>&1 -disable-output -mtriple=amdgcn-unknown-amdhsa -mcpu=gfx1010 < %s | FileCheck -check-prefixes=ALL-SIZE %s
 ; RUN: opt -passes="print<cost-model>" -cost-kind=code-size 2>&1 -disable-output -mtriple=amdgcn-unknown-amdhsa -mcpu=gfx90a < %s | FileCheck -check-prefixes=ALL-SIZE %s
 ; RUN: opt -passes="print<cost-model>" -cost-kind=code-size 2>&1 -disable-output -mtriple=amdgcn-unknown-amdhsa -mcpu=gfx900 < %s | FileCheck -check-prefixes=ALL-SIZE %s
 ; RUN: opt -passes="print<cost-model>" -cost-kind=code-size 2>&1 -disable-output -mtriple=amdgcn-unknown-amdhsa < %s | FileCheck -check-prefixes=ALL-SIZE %s
 ; END.
 
 define void @test_geps() {
 ; ALL-LABEL: 'test_geps'
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %giant_gep0 = getelementptr inbounds i8, ptr undef, i64 9223372036854775807
+; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %giant_gep0, align 1
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %giant_gep1 = getelementptr inbounds i8, ptr undef, i128 295147905179352825855
+; ALL-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %giant_gep1, align 1
 ; ALL-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: ret void
 ;
 ; ALL-SIZE-LABEL: 'test_geps'
 ; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %giant_gep0 = getelementptr inbounds i8, ptr undef, i64 9223372036854775807
+; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %giant_gep0, align 1
 ; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %giant_gep1 = getelementptr inbounds i8, ptr undef, i128 295147905179352825855
+; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %giant_gep1, align 1
 ; ALL-SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 
 
 
   ; Check that we handle outlandishly large GEPs properly.  This is unlikely to
   ; be a valid pointer, but LLVM still generates GEPs like this sometimes in
   ; dead code.
   ; This GEP has index INT64_MAX, which is cost 1.
   %giant_gep0 = getelementptr inbounds i8, ptr undef, i64 9223372036854775807
+  store volatile i8 undef, ptr %giant_gep0
 
   ; This GEP index wraps around to -1, which is cost 0.
   %giant_gep1 = getelementptr inbounds i8, ptr undef, i128 295147905179352825855
+  store volatile i8 undef, ptr %giant_gep1
 
   ret void
 }

llvm/test/Analysis/CostModel/ARM/gep.ll

 ; NOTE: Assertions have been autogenerated by utils/update_analyze_test_checks.py
 ; RUN: opt -passes="print<cost-model>" 2>&1 -disable-output -mtriple=thumbv6m-none-eabi < %s | FileCheck %s --check-prefix=CHECK-V6M
 ; RUN: opt -passes="print<cost-model>" 2>&1 -disable-output -mtriple=thumbv7m-none-eabi -mcpu=cortex-m3 < %s | FileCheck %s --check-prefix=CHECK-V7M-NOFP
 ; RUN: opt -passes="print<cost-model>" 2>&1 -disable-output -mtriple=thumbv7m-none-eabi -mcpu=cortex-m4 < %s | FileCheck %s --check-prefix=CHECK-V7M-FP
 ; RUN: opt -passes="print<cost-model>" 2>&1 -disable-output -mtriple=thumbv8.1m.main-none-eabi -mattr=+mve < %s | FileCheck %s --check-prefix=CHECK-MVE
 ; RUN: opt -passes="print<cost-model>" 2>&1 -disable-output -mtriple=thumbv8.1m.main-none-eabi -mattr=+mve.fp < %s | FileCheck %s --check-prefix=CHECK-MVEFP
 ; RUN: opt -passes="print<cost-model>" 2>&1 -disable-output -mtriple=thumbv7-apple-ios6.0.0 -mcpu=swift < %s | FileCheck %s --check-prefix=CHECK-T32
 ; RUN: opt -passes="print<cost-model>" 2>&1 -disable-output -mtriple=arm-none-eabi -mcpu=cortex-a53 < %s | FileCheck %s --check-prefix=CHECK-A32
 
 target datalayout = "e-m:e-p:32:32-Fi8-i64:64-v128:64:128-a:0:32-n32-S64"
 
 define void @testi8(ptr %a, i32 %i) {
 ; CHECK-V6M-LABEL: 'testi8'
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i8, ptr %a, i32 1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a1, align 1
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am4 = getelementptr inbounds i8, ptr %a, i32 -1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am4, align 1
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i8, ptr %a, i32 31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a31, align 1
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a32 = getelementptr inbounds i8, ptr %a, i32 32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a32, align 1
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4095 = getelementptr inbounds i8, ptr %a, i32 4095
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a4095, align 1
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i8, ptr %a, i32 4096
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a4096, align 1
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am255 = getelementptr inbounds i8, ptr %a, i32 -255
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am255, align 1
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i8, ptr %a, i32 -256
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am256, align 1
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %ai = getelementptr inbounds i8, ptr %a, i32 %i
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %ai, align 1
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-V7M-NOFP-LABEL: 'testi8'
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i8, ptr %a, i32 1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a1, align 1
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i8, ptr %a, i32 -1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am4, align 1
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i8, ptr %a, i32 31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a31, align 1
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i8, ptr %a, i32 32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a32, align 1
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a4095 = getelementptr inbounds i8, ptr %a, i32 4095
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a4095, align 1
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i8, ptr %a, i32 4096
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a4096, align 1
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i8, ptr %a, i32 -255
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am255, align 1
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i8, ptr %a, i32 -256
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am256, align 1
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %ai = getelementptr inbounds i8, ptr %a, i32 %i
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %ai, align 1
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-V7M-FP-LABEL: 'testi8'
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i8, ptr %a, i32 1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a1, align 1
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i8, ptr %a, i32 -1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am4, align 1
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i8, ptr %a, i32 31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a31, align 1
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i8, ptr %a, i32 32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a32, align 1
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a4095 = getelementptr inbounds i8, ptr %a, i32 4095
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a4095, align 1
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i8, ptr %a, i32 4096
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a4096, align 1
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i8, ptr %a, i32 -255
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am255, align 1
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i8, ptr %a, i32 -256
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am256, align 1
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %ai = getelementptr inbounds i8, ptr %a, i32 %i
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %ai, align 1
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-MVE-LABEL: 'testi8'
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i8, ptr %a, i32 1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a1, align 1
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i8, ptr %a, i32 -1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am4, align 1
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i8, ptr %a, i32 31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a31, align 1
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i8, ptr %a, i32 32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a32, align 1
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a4095 = getelementptr inbounds i8, ptr %a, i32 4095
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a4095, align 1
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i8, ptr %a, i32 4096
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a4096, align 1
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i8, ptr %a, i32 -255
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am255, align 1
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i8, ptr %a, i32 -256
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am256, align 1
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %ai = getelementptr inbounds i8, ptr %a, i32 %i
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %ai, align 1
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-MVEFP-LABEL: 'testi8'
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i8, ptr %a, i32 1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a1, align 1
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i8, ptr %a, i32 -1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am4, align 1
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i8, ptr %a, i32 31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a31, align 1
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i8, ptr %a, i32 32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a32, align 1
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a4095 = getelementptr inbounds i8, ptr %a, i32 4095
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a4095, align 1
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i8, ptr %a, i32 4096
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a4096, align 1
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i8, ptr %a, i32 -255
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am255, align 1
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i8, ptr %a, i32 -256
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am256, align 1
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %ai = getelementptr inbounds i8, ptr %a, i32 %i
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %ai, align 1
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-T32-LABEL: 'testi8'
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i8, ptr %a, i32 1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a1, align 1
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i8, ptr %a, i32 -1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am4, align 1
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i8, ptr %a, i32 31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a31, align 1
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i8, ptr %a, i32 32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a32, align 1
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a4095 = getelementptr inbounds i8, ptr %a, i32 4095
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a4095, align 1
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i8, ptr %a, i32 4096
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a4096, align 1
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i8, ptr %a, i32 -255
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am255, align 1
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i8, ptr %a, i32 -256
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am256, align 1
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %ai = getelementptr inbounds i8, ptr %a, i32 %i
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %ai, align 1
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-A32-LABEL: 'testi8'
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i8, ptr %a, i32 1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a1, align 1
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i8, ptr %a, i32 -1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am4, align 1
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i8, ptr %a, i32 31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a31, align 1
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i8, ptr %a, i32 32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a32, align 1
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a4095 = getelementptr inbounds i8, ptr %a, i32 4095
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a4095, align 1
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i8, ptr %a, i32 4096
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a4096, align 1
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i8, ptr %a, i32 -255
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am255, align 1
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am256 = getelementptr inbounds i8, ptr %a, i32 -256
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %am256, align 1
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %ai = getelementptr inbounds i8, ptr %a, i32 %i
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %ai, align 1
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
   %a1 = getelementptr inbounds i8, ptr %a, i32 1
+  store volatile i8 undef, ptr %a1
   %am4 = getelementptr inbounds i8, ptr %a, i32 -1
+  store volatile i8 undef, ptr %am4
   %a31 = getelementptr inbounds i8, ptr %a, i32 31
+  store volatile i8 undef, ptr %a31
   %a32 = getelementptr inbounds i8, ptr %a, i32 32
+  store volatile i8 undef, ptr %a32
   %a4095 = getelementptr inbounds i8, ptr %a, i32 4095
+  store volatile i8 undef, ptr %a4095
   %a4096 = getelementptr inbounds i8, ptr %a, i32 4096
+  store volatile i8 undef, ptr %a4096
   %am255 = getelementptr inbounds i8, ptr %a, i32 -255
+  store volatile i8 undef, ptr %am255
   %am256 = getelementptr inbounds i8, ptr %a, i32 -256
+  store volatile i8 undef, ptr %am256
   %ai = getelementptr inbounds i8, ptr %a, i32 %i
+  store volatile i8 undef, ptr %ai
 
   ret void
 }
 
 define void @testi16(ptr %a, i32 %i) {
 ; CHECK-V6M-LABEL: 'testi16'
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i16, ptr %a, i32 1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a1, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am4 = getelementptr inbounds i16, ptr %a, i32 -1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am4, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i16, ptr %a, i32 31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a31, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a32 = getelementptr inbounds i16, ptr %a, i32 32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a32, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4095 = getelementptr inbounds i16, ptr %a, i32 2046
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a4095, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i16, ptr %a, i32 2048
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a4096, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am255 = getelementptr inbounds i16, ptr %a, i32 -127
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am255, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i16, ptr %a, i32 -128
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am256, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds i16, ptr %a, i32 %i
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %ai, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-V7M-NOFP-LABEL: 'testi16'
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i16, ptr %a, i32 1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a1, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i16, ptr %a, i32 -1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am4, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i16, ptr %a, i32 31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a31, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i16, ptr %a, i32 32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a32, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a4095 = getelementptr inbounds i16, ptr %a, i32 2046
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a4095, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i16, ptr %a, i32 2048
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a4096, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i16, ptr %a, i32 -127
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am255, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i16, ptr %a, i32 -128
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am256, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %ai = getelementptr inbounds i16, ptr %a, i32 %i
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %ai, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-V7M-FP-LABEL: 'testi16'
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i16, ptr %a, i32 1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a1, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i16, ptr %a, i32 -1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am4, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i16, ptr %a, i32 31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a31, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i16, ptr %a, i32 32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a32, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a4095 = getelementptr inbounds i16, ptr %a, i32 2046
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a4095, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i16, ptr %a, i32 2048
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a4096, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i16, ptr %a, i32 -127
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am255, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i16, ptr %a, i32 -128
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am256, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %ai = getelementptr inbounds i16, ptr %a, i32 %i
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %ai, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-MVE-LABEL: 'testi16'
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i16, ptr %a, i32 1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a1, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i16, ptr %a, i32 -1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am4, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i16, ptr %a, i32 31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a31, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i16, ptr %a, i32 32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a32, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a4095 = getelementptr inbounds i16, ptr %a, i32 2046
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a4095, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i16, ptr %a, i32 2048
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a4096, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i16, ptr %a, i32 -127
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am255, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i16, ptr %a, i32 -128
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am256, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %ai = getelementptr inbounds i16, ptr %a, i32 %i
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %ai, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-MVEFP-LABEL: 'testi16'
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i16, ptr %a, i32 1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a1, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i16, ptr %a, i32 -1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am4, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i16, ptr %a, i32 31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a31, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i16, ptr %a, i32 32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a32, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a4095 = getelementptr inbounds i16, ptr %a, i32 2046
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a4095, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i16, ptr %a, i32 2048
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a4096, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i16, ptr %a, i32 -127
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am255, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i16, ptr %a, i32 -128
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am256, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %ai = getelementptr inbounds i16, ptr %a, i32 %i
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %ai, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-T32-LABEL: 'testi16'
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i16, ptr %a, i32 1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a1, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i16, ptr %a, i32 -1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am4, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i16, ptr %a, i32 31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a31, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i16, ptr %a, i32 32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a32, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a4095 = getelementptr inbounds i16, ptr %a, i32 2046
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a4095, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i16, ptr %a, i32 2048
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a4096, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i16, ptr %a, i32 -127
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am255, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i16, ptr %a, i32 -128
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am256, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %ai = getelementptr inbounds i16, ptr %a, i32 %i
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %ai, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-A32-LABEL: 'testi16'
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i16, ptr %a, i32 1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a1, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i16, ptr %a, i32 -1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am4, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i16, ptr %a, i32 31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a31, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i16, ptr %a, i32 32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a32, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4095 = getelementptr inbounds i16, ptr %a, i32 2046
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a4095, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i16, ptr %a, i32 2048
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a4096, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i16, ptr %a, i32 -127
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am255, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i16, ptr %a, i32 -128
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %am256, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds i16, ptr %a, i32 %i
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %ai, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
   %a1 = getelementptr inbounds i16, ptr %a, i32 1
+  store volatile i16 undef, ptr %a1
   %am4 = getelementptr inbounds i16, ptr %a, i32 -1
+  store volatile i16 undef, ptr %am4
   %a31 = getelementptr inbounds i16, ptr %a, i32 31
+  store volatile i16 undef, ptr %a31
   %a32 = getelementptr inbounds i16, ptr %a, i32 32
+  store volatile i16 undef, ptr %a32
   %a4095 = getelementptr inbounds i16, ptr %a, i32 2046
+  store volatile i16 undef, ptr %a4095
   %a4096 = getelementptr inbounds i16, ptr %a, i32 2048
+  store volatile i16 undef, ptr %a4096
   %am255 = getelementptr inbounds i16, ptr %a, i32 -127
+  store volatile i16 undef, ptr %am255
   %am256 = getelementptr inbounds i16, ptr %a, i32 -128
+  store volatile i16 undef, ptr %am256
   %ai = getelementptr inbounds i16, ptr %a, i32 %i
+  store volatile i16 undef, ptr %ai
 
   ret void
 }
 
 define void @testi32(ptr %a, i32 %i) {
 ; CHECK-V6M-LABEL: 'testi32'
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i32, ptr %a, i32 1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am4 = getelementptr inbounds i32, ptr %a, i32 -1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am4, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i32, ptr %a, i32 31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a31, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a32 = getelementptr inbounds i32, ptr %a, i32 32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a32, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds i32, ptr %a, i32 1023
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1023, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds i32, ptr %a, i32 1024
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1024, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am255 = getelementptr inbounds i32, ptr %a, i32 -63
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am255, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i32, ptr %a, i32 -64
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am256, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds i32, ptr %a, i32 %i
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %ai, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-V7M-NOFP-LABEL: 'testi32'
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i32, ptr %a, i32 1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i32, ptr %a, i32 -1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am4, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i32, ptr %a, i32 31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a31, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i32, ptr %a, i32 32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a32, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1023 = getelementptr inbounds i32, ptr %a, i32 1023
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1023, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds i32, ptr %a, i32 1024
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1024, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i32, ptr %a, i32 -63
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am255, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i32, ptr %a, i32 -64
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am256, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %ai = getelementptr inbounds i32, ptr %a, i32 %i
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %ai, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-V7M-FP-LABEL: 'testi32'
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i32, ptr %a, i32 1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i32, ptr %a, i32 -1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am4, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i32, ptr %a, i32 31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a31, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i32, ptr %a, i32 32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a32, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1023 = getelementptr inbounds i32, ptr %a, i32 1023
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1023, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds i32, ptr %a, i32 1024
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1024, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i32, ptr %a, i32 -63
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am255, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i32, ptr %a, i32 -64
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am256, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %ai = getelementptr inbounds i32, ptr %a, i32 %i
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %ai, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-MVE-LABEL: 'testi32'
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i32, ptr %a, i32 1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i32, ptr %a, i32 -1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am4, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i32, ptr %a, i32 31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a31, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i32, ptr %a, i32 32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a32, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1023 = getelementptr inbounds i32, ptr %a, i32 1023
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1023, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds i32, ptr %a, i32 1024
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1024, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i32, ptr %a, i32 -63
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am255, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i32, ptr %a, i32 -64
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am256, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %ai = getelementptr inbounds i32, ptr %a, i32 %i
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %ai, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-MVEFP-LABEL: 'testi32'
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i32, ptr %a, i32 1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i32, ptr %a, i32 -1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am4, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i32, ptr %a, i32 31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a31, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i32, ptr %a, i32 32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a32, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1023 = getelementptr inbounds i32, ptr %a, i32 1023
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1023, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds i32, ptr %a, i32 1024
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1024, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i32, ptr %a, i32 -63
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am255, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i32, ptr %a, i32 -64
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am256, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %ai = getelementptr inbounds i32, ptr %a, i32 %i
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %ai, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-T32-LABEL: 'testi32'
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i32, ptr %a, i32 1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i32, ptr %a, i32 -1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am4, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i32, ptr %a, i32 31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a31, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i32, ptr %a, i32 32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a32, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1023 = getelementptr inbounds i32, ptr %a, i32 1023
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1023, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds i32, ptr %a, i32 1024
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1024, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i32, ptr %a, i32 -63
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am255, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i32, ptr %a, i32 -64
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am256, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %ai = getelementptr inbounds i32, ptr %a, i32 %i
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %ai, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-A32-LABEL: 'testi32'
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i32, ptr %a, i32 1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i32, ptr %a, i32 -1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am4, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i32, ptr %a, i32 31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a31, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i32, ptr %a, i32 32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a32, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1023 = getelementptr inbounds i32, ptr %a, i32 1023
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1023, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds i32, ptr %a, i32 1024
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1024, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i32, ptr %a, i32 -63
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am255, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am256 = getelementptr inbounds i32, ptr %a, i32 -64
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %am256, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %ai = getelementptr inbounds i32, ptr %a, i32 %i
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %ai, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
   %a1 = getelementptr inbounds i32, ptr %a, i32 1
+  store volatile i32 undef, ptr %a1
   %am4 = getelementptr inbounds i32, ptr %a, i32 -1
+  store volatile i32 undef, ptr %am4
   %a31 = getelementptr inbounds i32, ptr %a, i32 31
+  store volatile i32 undef, ptr %a31
   %a32 = getelementptr inbounds i32, ptr %a, i32 32
+  store volatile i32 undef, ptr %a32
   %a1023 = getelementptr inbounds i32, ptr %a, i32 1023
+  store volatile i32 undef, ptr %a1023
   %a1024 = getelementptr inbounds i32, ptr %a, i32 1024
+  store volatile i32 undef, ptr %a1024
   %am255 = getelementptr inbounds i32, ptr %a, i32 -63
+  store volatile i32 undef, ptr %am255
   %am256 = getelementptr inbounds i32, ptr %a, i32 -64
+  store volatile i32 undef, ptr %am256
   %ai = getelementptr inbounds i32, ptr %a, i32 %i
+  store volatile i32 undef, ptr %ai
 
   ret void
 }
 
 define void @testi64(ptr %a, i32 %i) {
 ; CHECK-V6M-LABEL: 'testi64'
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i64, ptr %a, i32 1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a1, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am4 = getelementptr inbounds i64, ptr %a, i32 -1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am4, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a15 = getelementptr inbounds i64, ptr %a, i32 15
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a15, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a16 = getelementptr inbounds i64, ptr %a, i32 16
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a16, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a31 = getelementptr inbounds i64, ptr %a, i32 31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a31, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a32 = getelementptr inbounds i64, ptr %a, i32 32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a32, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4095 = getelementptr inbounds i64, ptr %a, i32 1023
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a4095, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i64, ptr %a, i32 1024
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a4096, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am255 = getelementptr inbounds i64, ptr %a, i32 -63
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am255, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i64, ptr %a, i32 -64
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am256, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds i64, ptr %a, i32 %i
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %ai, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-V7M-NOFP-LABEL: 'testi64'
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i64, ptr %a, i32 1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a1, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i64, ptr %a, i32 -1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am4, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a15 = getelementptr inbounds i64, ptr %a, i32 15
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a15, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a16 = getelementptr inbounds i64, ptr %a, i32 16
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a16, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i64, ptr %a, i32 31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a31, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i64, ptr %a, i32 32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a32, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4095 = getelementptr inbounds i64, ptr %a, i32 1023
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a4095, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i64, ptr %a, i32 1024
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a4096, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i64, ptr %a, i32 -63
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am255, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am256 = getelementptr inbounds i64, ptr %a, i32 -64
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am256, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds i64, ptr %a, i32 %i
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %ai, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-V7M-FP-LABEL: 'testi64'
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i64, ptr %a, i32 1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a1, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i64, ptr %a, i32 -1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am4, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a15 = getelementptr inbounds i64, ptr %a, i32 15
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a15, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a16 = getelementptr inbounds i64, ptr %a, i32 16
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a16, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i64, ptr %a, i32 31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a31, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i64, ptr %a, i32 32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a32, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4095 = getelementptr inbounds i64, ptr %a, i32 1023
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a4095, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i64, ptr %a, i32 1024
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a4096, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i64, ptr %a, i32 -63
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am255, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am256 = getelementptr inbounds i64, ptr %a, i32 -64
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am256, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds i64, ptr %a, i32 %i
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %ai, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-MVE-LABEL: 'testi64'
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i64, ptr %a, i32 1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a1, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i64, ptr %a, i32 -1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am4, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a15 = getelementptr inbounds i64, ptr %a, i32 15
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a15, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a16 = getelementptr inbounds i64, ptr %a, i32 16
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a16, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i64, ptr %a, i32 31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a31, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i64, ptr %a, i32 32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a32, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4095 = getelementptr inbounds i64, ptr %a, i32 1023
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a4095, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i64, ptr %a, i32 1024
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a4096, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i64, ptr %a, i32 -63
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am255, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am256 = getelementptr inbounds i64, ptr %a, i32 -64
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am256, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds i64, ptr %a, i32 %i
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %ai, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-MVEFP-LABEL: 'testi64'
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i64, ptr %a, i32 1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a1, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i64, ptr %a, i32 -1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am4, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a15 = getelementptr inbounds i64, ptr %a, i32 15
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a15, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a16 = getelementptr inbounds i64, ptr %a, i32 16
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a16, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i64, ptr %a, i32 31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a31, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i64, ptr %a, i32 32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a32, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4095 = getelementptr inbounds i64, ptr %a, i32 1023
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a4095, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i64, ptr %a, i32 1024
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a4096, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i64, ptr %a, i32 -63
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am255, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am256 = getelementptr inbounds i64, ptr %a, i32 -64
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am256, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds i64, ptr %a, i32 %i
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %ai, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-T32-LABEL: 'testi64'
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i64, ptr %a, i32 1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a1, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am4 = getelementptr inbounds i64, ptr %a, i32 -1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am4, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a15 = getelementptr inbounds i64, ptr %a, i32 15
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a15, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a16 = getelementptr inbounds i64, ptr %a, i32 16
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a16, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a31 = getelementptr inbounds i64, ptr %a, i32 31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a31, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a32 = getelementptr inbounds i64, ptr %a, i32 32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a32, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4095 = getelementptr inbounds i64, ptr %a, i32 1023
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a4095, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i64, ptr %a, i32 1024
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a4096, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds i64, ptr %a, i32 -63
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am255, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am256 = getelementptr inbounds i64, ptr %a, i32 -64
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am256, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds i64, ptr %a, i32 %i
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %ai, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-A32-LABEL: 'testi64'
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1 = getelementptr inbounds i64, ptr %a, i32 1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a1, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am4 = getelementptr inbounds i64, ptr %a, i32 -1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am4, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a15 = getelementptr inbounds i64, ptr %a, i32 15
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a15, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a16 = getelementptr inbounds i64, ptr %a, i32 16
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a16, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a31 = getelementptr inbounds i64, ptr %a, i32 31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a31, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a32 = getelementptr inbounds i64, ptr %a, i32 32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a32, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4095 = getelementptr inbounds i64, ptr %a, i32 1023
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a4095, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4096 = getelementptr inbounds i64, ptr %a, i32 1024
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %a4096, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am255 = getelementptr inbounds i64, ptr %a, i32 -63
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am255, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds i64, ptr %a, i32 -64
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %am256, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds i64, ptr %a, i32 %i
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile i64 undef, ptr %ai, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
   %a1 = getelementptr inbounds i64, ptr %a, i32 1
+  store volatile i64 undef, ptr %a1
   %am4 = getelementptr inbounds i64, ptr %a, i32 -1
+  store volatile i64 undef, ptr %am4
   %a15 = getelementptr inbounds i64, ptr %a, i32 15
+  store volatile i64 undef, ptr %a15
   %a16 = getelementptr inbounds i64, ptr %a, i32 16
+  store volatile i64 undef, ptr %a16
   %a31 = getelementptr inbounds i64, ptr %a, i32 31
+  store volatile i64 undef, ptr %a31
   %a32 = getelementptr inbounds i64, ptr %a, i32 32
+  store volatile i64 undef, ptr %a32
   %a4095 = getelementptr inbounds i64, ptr %a, i32 1023
+  store volatile i64 undef, ptr %a4095
   %a4096 = getelementptr inbounds i64, ptr %a, i32 1024
+  store volatile i64 undef, ptr %a4096
   %am255 = getelementptr inbounds i64, ptr %a, i32 -63
+  store volatile i64 undef, ptr %am255
   %am256 = getelementptr inbounds i64, ptr %a, i32 -64
+  store volatile i64 undef, ptr %am256
   %ai = getelementptr inbounds i64, ptr %a, i32 %i
+  store volatile i64 undef, ptr %ai
 
   ret void
 }
 
 define void @testhalf(ptr %a, i32 %i) {
 ; CHECK-V6M-LABEL: 'testhalf'
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1 = getelementptr inbounds half, ptr %a, i32 1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am1 = getelementptr inbounds half, ptr %a, i32 -1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am1, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a255 = getelementptr inbounds half, ptr %a, i32 255
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a255, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a256 = getelementptr inbounds half, ptr %a, i32 256
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a256, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am255 = getelementptr inbounds half, ptr %a, i32 -255
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am255, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds half, ptr %a, i32 -256
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am256, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds half, ptr %a, i32 1023
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1023, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds half, ptr %a, i32 1024
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1024, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am63 = getelementptr inbounds half, ptr %a, i32 -63
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am63, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am64 = getelementptr inbounds half, ptr %a, i32 -64
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am64, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds half, ptr %a, i32 %i
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %ai, align 2
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-V7M-NOFP-LABEL: 'testhalf'
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds half, ptr %a, i32 1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am1 = getelementptr inbounds half, ptr %a, i32 -1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am1, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a255 = getelementptr inbounds half, ptr %a, i32 255
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a255, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a256 = getelementptr inbounds half, ptr %a, i32 256
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a256, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am255 = getelementptr inbounds half, ptr %a, i32 -255
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am255, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds half, ptr %a, i32 -256
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am256, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1023 = getelementptr inbounds half, ptr %a, i32 1023
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1023, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1024 = getelementptr inbounds half, ptr %a, i32 1024
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1024, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am63 = getelementptr inbounds half, ptr %a, i32 -63
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am63, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am64 = getelementptr inbounds half, ptr %a, i32 -64
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am64, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds half, ptr %a, i32 %i
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %ai, align 2
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-V7M-FP-LABEL: 'testhalf'
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1 = getelementptr inbounds half, ptr %a, i32 1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am1 = getelementptr inbounds half, ptr %a, i32 -1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am1, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a255 = getelementptr inbounds half, ptr %a, i32 255
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a255, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a256 = getelementptr inbounds half, ptr %a, i32 256
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a256, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am255 = getelementptr inbounds half, ptr %a, i32 -255
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am255, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am256 = getelementptr inbounds half, ptr %a, i32 -256
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am256, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds half, ptr %a, i32 1023
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1023, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds half, ptr %a, i32 1024
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1024, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am63 = getelementptr inbounds half, ptr %a, i32 -63
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am63, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am64 = getelementptr inbounds half, ptr %a, i32 -64
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am64, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds half, ptr %a, i32 %i
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %ai, align 2
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-MVE-LABEL: 'testhalf'
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds half, ptr %a, i32 1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am1 = getelementptr inbounds half, ptr %a, i32 -1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am1, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a255 = getelementptr inbounds half, ptr %a, i32 255
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a255, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a256 = getelementptr inbounds half, ptr %a, i32 256
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a256, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds half, ptr %a, i32 -255
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am255, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds half, ptr %a, i32 -256
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am256, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds half, ptr %a, i32 1023
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1023, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds half, ptr %a, i32 1024
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1024, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am63 = getelementptr inbounds half, ptr %a, i32 -63
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am63, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am64 = getelementptr inbounds half, ptr %a, i32 -64
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am64, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds half, ptr %a, i32 %i
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %ai, align 2
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-MVEFP-LABEL: 'testhalf'
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds half, ptr %a, i32 1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am1 = getelementptr inbounds half, ptr %a, i32 -1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am1, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a255 = getelementptr inbounds half, ptr %a, i32 255
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a255, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a256 = getelementptr inbounds half, ptr %a, i32 256
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a256, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds half, ptr %a, i32 -255
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am255, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds half, ptr %a, i32 -256
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am256, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds half, ptr %a, i32 1023
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1023, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds half, ptr %a, i32 1024
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1024, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am63 = getelementptr inbounds half, ptr %a, i32 -63
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am63, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am64 = getelementptr inbounds half, ptr %a, i32 -64
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am64, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds half, ptr %a, i32 %i
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %ai, align 2
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-T32-LABEL: 'testhalf'
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1 = getelementptr inbounds half, ptr %a, i32 1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am1 = getelementptr inbounds half, ptr %a, i32 -1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am1, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a255 = getelementptr inbounds half, ptr %a, i32 255
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a255, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a256 = getelementptr inbounds half, ptr %a, i32 256
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a256, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am255 = getelementptr inbounds half, ptr %a, i32 -255
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am255, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am256 = getelementptr inbounds half, ptr %a, i32 -256
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am256, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds half, ptr %a, i32 1023
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1023, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds half, ptr %a, i32 1024
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1024, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am63 = getelementptr inbounds half, ptr %a, i32 -63
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am63, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am64 = getelementptr inbounds half, ptr %a, i32 -64
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am64, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds half, ptr %a, i32 %i
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %ai, align 2
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-A32-LABEL: 'testhalf'
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1 = getelementptr inbounds half, ptr %a, i32 1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am1 = getelementptr inbounds half, ptr %a, i32 -1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am1, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a255 = getelementptr inbounds half, ptr %a, i32 255
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a255, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a256 = getelementptr inbounds half, ptr %a, i32 256
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a256, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am255 = getelementptr inbounds half, ptr %a, i32 -255
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am255, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds half, ptr %a, i32 -256
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am256, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds half, ptr %a, i32 1023
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1023, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds half, ptr %a, i32 1024
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %a1024, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am63 = getelementptr inbounds half, ptr %a, i32 -63
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am63, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am64 = getelementptr inbounds half, ptr %a, i32 -64
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %am64, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds half, ptr %a, i32 %i
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile half undef, ptr %ai, align 2
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
   %a1 = getelementptr inbounds half, ptr %a, i32 1
+  store volatile half undef, ptr %a1
   %am1 = getelementptr inbounds half, ptr %a, i32 -1
+  store volatile half undef, ptr %am1
   %a255 = getelementptr inbounds half, ptr %a, i32 255
+  store volatile half undef, ptr %a255
   %a256 = getelementptr inbounds half, ptr %a, i32 256
+  store volatile half undef, ptr %a256
   %am255 = getelementptr inbounds half, ptr %a, i32 -255
+  store volatile half undef, ptr %am255
   %am256 = getelementptr inbounds half, ptr %a, i32 -256
+  store volatile half undef, ptr %am256
   %a1023 = getelementptr inbounds half, ptr %a, i32 1023
+  store volatile half undef, ptr %a1023
   %a1024 = getelementptr inbounds half, ptr %a, i32 1024
+  store volatile half undef, ptr %a1024
   %am63 = getelementptr inbounds half, ptr %a, i32 -63
+  store volatile half undef, ptr %am63
   %am64 = getelementptr inbounds half, ptr %a, i32 -64
+  store volatile half undef, ptr %am64
   %ai = getelementptr inbounds half, ptr %a, i32 %i
+  store volatile half undef, ptr %ai
 
   ret void
 }
 
 define void @testfloat(ptr %a, i32 %i) {
 ; CHECK-V6M-LABEL: 'testfloat'
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds float, ptr %a, i32 1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am1 = getelementptr inbounds float, ptr %a, i32 -1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am1, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a255 = getelementptr inbounds float, ptr %a, i32 255
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a255, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a256 = getelementptr inbounds float, ptr %a, i32 256
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a256, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am255 = getelementptr inbounds float, ptr %a, i32 -255
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am255, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds float, ptr %a, i32 -256
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am256, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds float, ptr %a, i32 1023
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1023, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds float, ptr %a, i32 1024
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1024, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am63 = getelementptr inbounds float, ptr %a, i32 -63
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am63, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am64 = getelementptr inbounds float, ptr %a, i32 -64
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am64, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds float, ptr %a, i32 %i
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %ai, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-V7M-NOFP-LABEL: 'testfloat'
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds float, ptr %a, i32 1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am1 = getelementptr inbounds float, ptr %a, i32 -1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am1, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a255 = getelementptr inbounds float, ptr %a, i32 255
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a255, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a256 = getelementptr inbounds float, ptr %a, i32 256
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a256, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am255 = getelementptr inbounds float, ptr %a, i32 -255
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am255, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds float, ptr %a, i32 -256
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am256, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1023 = getelementptr inbounds float, ptr %a, i32 1023
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1023, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds float, ptr %a, i32 1024
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1024, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am63 = getelementptr inbounds float, ptr %a, i32 -63
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am63, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am64 = getelementptr inbounds float, ptr %a, i32 -64
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am64, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds float, ptr %a, i32 %i
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %ai, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-V7M-FP-LABEL: 'testfloat'
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds float, ptr %a, i32 1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am1 = getelementptr inbounds float, ptr %a, i32 -1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am1, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a255 = getelementptr inbounds float, ptr %a, i32 255
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a255, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a256 = getelementptr inbounds float, ptr %a, i32 256
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a256, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds float, ptr %a, i32 -255
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am255, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds float, ptr %a, i32 -256
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am256, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds float, ptr %a, i32 1023
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1023, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds float, ptr %a, i32 1024
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1024, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am63 = getelementptr inbounds float, ptr %a, i32 -63
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am63, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am64 = getelementptr inbounds float, ptr %a, i32 -64
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am64, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds float, ptr %a, i32 %i
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %ai, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-MVE-LABEL: 'testfloat'
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds float, ptr %a, i32 1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am1 = getelementptr inbounds float, ptr %a, i32 -1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am1, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a255 = getelementptr inbounds float, ptr %a, i32 255
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a255, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a256 = getelementptr inbounds float, ptr %a, i32 256
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a256, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am255 = getelementptr inbounds float, ptr %a, i32 -255
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am255, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds float, ptr %a, i32 -256
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am256, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1023 = getelementptr inbounds float, ptr %a, i32 1023
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1023, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds float, ptr %a, i32 1024
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1024, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am63 = getelementptr inbounds float, ptr %a, i32 -63
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am63, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am64 = getelementptr inbounds float, ptr %a, i32 -64
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am64, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds float, ptr %a, i32 %i
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %ai, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-MVEFP-LABEL: 'testfloat'
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds float, ptr %a, i32 1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am1 = getelementptr inbounds float, ptr %a, i32 -1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am1, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a255 = getelementptr inbounds float, ptr %a, i32 255
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a255, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a256 = getelementptr inbounds float, ptr %a, i32 256
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a256, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds float, ptr %a, i32 -255
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am255, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds float, ptr %a, i32 -256
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am256, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds float, ptr %a, i32 1023
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1023, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds float, ptr %a, i32 1024
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1024, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am63 = getelementptr inbounds float, ptr %a, i32 -63
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am63, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am64 = getelementptr inbounds float, ptr %a, i32 -64
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am64, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds float, ptr %a, i32 %i
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %ai, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-T32-LABEL: 'testfloat'
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds float, ptr %a, i32 1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am1 = getelementptr inbounds float, ptr %a, i32 -1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am1, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a255 = getelementptr inbounds float, ptr %a, i32 255
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a255, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a256 = getelementptr inbounds float, ptr %a, i32 256
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a256, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds float, ptr %a, i32 -255
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am255, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds float, ptr %a, i32 -256
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am256, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds float, ptr %a, i32 1023
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1023, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds float, ptr %a, i32 1024
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1024, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am63 = getelementptr inbounds float, ptr %a, i32 -63
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am63, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am64 = getelementptr inbounds float, ptr %a, i32 -64
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am64, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds float, ptr %a, i32 %i
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %ai, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-A32-LABEL: 'testfloat'
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds float, ptr %a, i32 1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am1 = getelementptr inbounds float, ptr %a, i32 -1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am1, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a255 = getelementptr inbounds float, ptr %a, i32 255
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a255, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a256 = getelementptr inbounds float, ptr %a, i32 256
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a256, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds float, ptr %a, i32 -255
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am255, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds float, ptr %a, i32 -256
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am256, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds float, ptr %a, i32 1023
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1023, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds float, ptr %a, i32 1024
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1024, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am63 = getelementptr inbounds float, ptr %a, i32 -63
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am63, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am64 = getelementptr inbounds float, ptr %a, i32 -64
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %am64, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds float, ptr %a, i32 %i
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %ai, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
   %a1 = getelementptr inbounds float, ptr %a, i32 1
+  store volatile float undef, ptr %a1
   %am1 = getelementptr inbounds float, ptr %a, i32 -1
+  store volatile float undef, ptr %am1
   %a255 = getelementptr inbounds float, ptr %a, i32 255
+  store volatile float undef, ptr %a255
   %a256 = getelementptr inbounds float, ptr %a, i32 256
+  store volatile float undef, ptr %a256
   %am255 = getelementptr inbounds float, ptr %a, i32 -255
+  store volatile float undef, ptr %am255
   %am256 = getelementptr inbounds float, ptr %a, i32 -256
+  store volatile float undef, ptr %am256
   %a1023 = getelementptr inbounds float, ptr %a, i32 1023
+  store volatile float undef, ptr %a1023
   %a1024 = getelementptr inbounds float, ptr %a, i32 1024
+  store volatile float undef, ptr %a1024
   %am63 = getelementptr inbounds float, ptr %a, i32 -63
+  store volatile float undef, ptr %am63
   %am64 = getelementptr inbounds float, ptr %a, i32 -64
+  store volatile float undef, ptr %am64
   %ai = getelementptr inbounds float, ptr %a, i32 %i
+  store volatile float undef, ptr %ai
 
   ret void
 }
 
 define void @testdouble(ptr %a, i32 %i) {
 ; CHECK-V6M-LABEL: 'testdouble'
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds double, ptr %a, i32 1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %a1, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am1 = getelementptr inbounds double, ptr %a, i32 -1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %am1, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a255 = getelementptr inbounds double, ptr %a, i32 127
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %a255, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a256 = getelementptr inbounds double, ptr %a, i32 128
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %a256, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am255 = getelementptr inbounds double, ptr %a, i32 -127
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %am255, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds double, ptr %a, i32 -128
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %am256, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds double, ptr %a, i32 511
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %a1023, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds double, ptr %a, i32 512
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %a1024, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am63 = getelementptr inbounds double, ptr %a, i32 -31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %am63, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am64 = getelementptr inbounds double, ptr %a, i32 -32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %am64, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds double, ptr %a, i32 %i
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %ai, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-V7M-NOFP-LABEL: 'testdouble'
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds double, ptr %a, i32 1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %a1, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am1 = getelementptr inbounds double, ptr %a, i32 -1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %am1, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a255 = getelementptr inbounds double, ptr %a, i32 127
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %a255, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a256 = getelementptr inbounds double, ptr %a, i32 128
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %a256, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds double, ptr %a, i32 -127
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %am255, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds double, ptr %a, i32 -128
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %am256, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds double, ptr %a, i32 511
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %a1023, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds double, ptr %a, i32 512
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %a1024, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am63 = getelementptr inbounds double, ptr %a, i32 -31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %am63, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am64 = getelementptr inbounds double, ptr %a, i32 -32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %am64, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds double, ptr %a, i32 %i
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile double undef, ptr %ai, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-V7M-FP-LABEL: 'testdouble'
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds double, ptr %a, i32 1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a1, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am1 = getelementptr inbounds double, ptr %a, i32 -1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am1, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a255 = getelementptr inbounds double, ptr %a, i32 127
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a255, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a256 = getelementptr inbounds double, ptr %a, i32 128
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a256, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds double, ptr %a, i32 -127
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am255, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds double, ptr %a, i32 -128
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am256, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds double, ptr %a, i32 511
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a1023, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds double, ptr %a, i32 512
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a1024, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am63 = getelementptr inbounds double, ptr %a, i32 -31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am63, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am64 = getelementptr inbounds double, ptr %a, i32 -32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am64, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds double, ptr %a, i32 %i
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %ai, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-MVE-LABEL: 'testdouble'
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds double, ptr %a, i32 1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a1, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am1 = getelementptr inbounds double, ptr %a, i32 -1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am1, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a255 = getelementptr inbounds double, ptr %a, i32 127
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a255, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a256 = getelementptr inbounds double, ptr %a, i32 128
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a256, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds double, ptr %a, i32 -127
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am255, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds double, ptr %a, i32 -128
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am256, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds double, ptr %a, i32 511
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a1023, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds double, ptr %a, i32 512
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a1024, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am63 = getelementptr inbounds double, ptr %a, i32 -31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am63, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am64 = getelementptr inbounds double, ptr %a, i32 -32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am64, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds double, ptr %a, i32 %i
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %ai, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-MVEFP-LABEL: 'testdouble'
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds double, ptr %a, i32 1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a1, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am1 = getelementptr inbounds double, ptr %a, i32 -1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am1, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a255 = getelementptr inbounds double, ptr %a, i32 127
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a255, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a256 = getelementptr inbounds double, ptr %a, i32 128
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a256, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds double, ptr %a, i32 -127
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am255, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds double, ptr %a, i32 -128
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am256, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds double, ptr %a, i32 511
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a1023, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds double, ptr %a, i32 512
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a1024, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am63 = getelementptr inbounds double, ptr %a, i32 -31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am63, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am64 = getelementptr inbounds double, ptr %a, i32 -32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am64, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds double, ptr %a, i32 %i
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %ai, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-T32-LABEL: 'testdouble'
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds double, ptr %a, i32 1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a1, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am1 = getelementptr inbounds double, ptr %a, i32 -1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am1, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a255 = getelementptr inbounds double, ptr %a, i32 127
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a255, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a256 = getelementptr inbounds double, ptr %a, i32 128
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a256, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds double, ptr %a, i32 -127
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am255, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds double, ptr %a, i32 -128
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am256, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds double, ptr %a, i32 511
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a1023, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds double, ptr %a, i32 512
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a1024, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am63 = getelementptr inbounds double, ptr %a, i32 -31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am63, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am64 = getelementptr inbounds double, ptr %a, i32 -32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am64, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds double, ptr %a, i32 %i
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %ai, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-A32-LABEL: 'testdouble'
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds double, ptr %a, i32 1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a1, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am1 = getelementptr inbounds double, ptr %a, i32 -1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am1, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a255 = getelementptr inbounds double, ptr %a, i32 127
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a255, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a256 = getelementptr inbounds double, ptr %a, i32 128
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a256, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am255 = getelementptr inbounds double, ptr %a, i32 -127
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am255, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %am256 = getelementptr inbounds double, ptr %a, i32 -128
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am256, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1023 = getelementptr inbounds double, ptr %a, i32 511
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a1023, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a1024 = getelementptr inbounds double, ptr %a, i32 512
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a1024, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am63 = getelementptr inbounds double, ptr %a, i32 -31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am63, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %am64 = getelementptr inbounds double, ptr %a, i32 -32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %am64, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds double, ptr %a, i32 %i
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %ai, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
   %a1 = getelementptr inbounds double, ptr %a, i32 1
+  store volatile double undef, ptr %a1
   %am1 = getelementptr inbounds double, ptr %a, i32 -1
+  store volatile double undef, ptr %am1
   %a255 = getelementptr inbounds double, ptr %a, i32 127
+  store volatile double undef, ptr %a255
   %a256 = getelementptr inbounds double, ptr %a, i32 128
+  store volatile double undef, ptr %a256
   %am255 = getelementptr inbounds double, ptr %a, i32 -127
+  store volatile double undef, ptr %am255
   %am256 = getelementptr inbounds double, ptr %a, i32 -128
+  store volatile double undef, ptr %am256
   %a1023 = getelementptr inbounds double, ptr %a, i32 511
+  store volatile double undef, ptr %a1023
   %a1024 = getelementptr inbounds double, ptr %a, i32 512
+  store volatile double undef, ptr %a1024
   %am63 = getelementptr inbounds double, ptr %a, i32 -31
+  store volatile double undef, ptr %am63
   %am64 = getelementptr inbounds double, ptr %a, i32 -32
+  store volatile double undef, ptr %am64
   %ai = getelementptr inbounds double, ptr %a, i32 %i
+  store volatile double undef, ptr %ai
 
   ret void
 }
 
 define void @testvecs(i32 %i) {
 ; CHECK-V6M-LABEL: 'testvecs'
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b7 = getelementptr inbounds <4 x i8>, ptr undef, i32 1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %b7, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b8 = getelementptr inbounds <4 x i16>, ptr undef, i32 1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %b8, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b9 = getelementptr inbounds <4 x i32>, ptr undef, i32 1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %b9, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b10 = getelementptr inbounds <4 x i64>, ptr undef, i32 1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %b10, align 32
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b11 = getelementptr inbounds <4 x half>, ptr undef, i32 1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %b11, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b12 = getelementptr inbounds <4 x float>, ptr undef, i32 1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %b12, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b13 = getelementptr inbounds <4 x double>, ptr undef, i32 1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %b13, align 32
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o7 = getelementptr inbounds <4 x i8>, ptr undef, i32 4
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %o7, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o8 = getelementptr inbounds <4 x i16>, ptr undef, i32 4
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %o8, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o9 = getelementptr inbounds <4 x i32>, ptr undef, i32 4
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %o9, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o10 = getelementptr inbounds <4 x i64>, ptr undef, i32 4
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %o10, align 32
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o11 = getelementptr inbounds <4 x half>, ptr undef, i32 4
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %o11, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o12 = getelementptr inbounds <4 x float>, ptr undef, i32 4
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %o12, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o13 = getelementptr inbounds <4 x double>, ptr undef, i32 4
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %o13, align 32
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %p7 = getelementptr inbounds <4 x i8>, ptr undef, i32 31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %p7, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p8 = getelementptr inbounds <4 x i16>, ptr undef, i32 31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %p8, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p9 = getelementptr inbounds <4 x i32>, ptr undef, i32 31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %p9, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p10 = getelementptr inbounds <4 x i64>, ptr undef, i32 31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %p10, align 32
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p11 = getelementptr inbounds <4 x half>, ptr undef, i32 31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %p11, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p12 = getelementptr inbounds <4 x float>, ptr undef, i32 31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %p12, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p13 = getelementptr inbounds <4 x double>, ptr undef, i32 31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %p13, align 32
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q7 = getelementptr inbounds <4 x i8>, ptr undef, i32 32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %q7, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q8 = getelementptr inbounds <4 x i16>, ptr undef, i32 32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %q8, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q9 = getelementptr inbounds <4 x i32>, ptr undef, i32 32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %q9, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q10 = getelementptr inbounds <4 x i64>, ptr undef, i32 32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %q10, align 32
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q11 = getelementptr inbounds <4 x half>, ptr undef, i32 32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %q11, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q12 = getelementptr inbounds <4 x float>, ptr undef, i32 32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %q12, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q13 = getelementptr inbounds <4 x double>, ptr undef, i32 32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %q13, align 32
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r7 = getelementptr inbounds <4 x i8>, ptr undef, i32 -31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %r7, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r8 = getelementptr inbounds <4 x i16>, ptr undef, i32 -31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %r8, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r9 = getelementptr inbounds <4 x i32>, ptr undef, i32 -31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %r9, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r10 = getelementptr inbounds <4 x i64>, ptr undef, i32 -31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %r10, align 32
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r11 = getelementptr inbounds <4 x half>, ptr undef, i32 -31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %r11, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r12 = getelementptr inbounds <4 x float>, ptr undef, i32 -31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %r12, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r13 = getelementptr inbounds <4 x double>, ptr undef, i32 -31
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %r13, align 32
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s7 = getelementptr inbounds <4 x i8>, ptr undef, i32 -32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %s7, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s8 = getelementptr inbounds <4 x i16>, ptr undef, i32 -32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %s8, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s9 = getelementptr inbounds <4 x i32>, ptr undef, i32 -32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %s9, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s10 = getelementptr inbounds <4 x i64>, ptr undef, i32 -32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %s10, align 32
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s11 = getelementptr inbounds <4 x half>, ptr undef, i32 -32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %s11, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s12 = getelementptr inbounds <4 x float>, ptr undef, i32 -32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %s12, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s13 = getelementptr inbounds <4 x double>, ptr undef, i32 -32
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %s13, align 32
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c7 = getelementptr inbounds <4 x i8>, ptr undef, i32 %i
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %c7, align 4
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c8 = getelementptr inbounds <4 x i16>, ptr undef, i32 %i
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %c8, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c9 = getelementptr inbounds <4 x i32>, ptr undef, i32 %i
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %c9, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c10 = getelementptr inbounds <4 x i64>, ptr undef, i32 %i
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %c10, align 32
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c11 = getelementptr inbounds <4 x half>, ptr undef, i32 %i
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %c11, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c12 = getelementptr inbounds <4 x float>, ptr undef, i32 %i
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %c12, align 8
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c13 = getelementptr inbounds <4 x double>, ptr undef, i32 %i
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %c13, align 32
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %d0 = getelementptr inbounds i8, ptr undef, i32 -1
+; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %d0, align 1
 ; CHECK-V6M-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-V7M-NOFP-LABEL: 'testvecs'
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b7 = getelementptr inbounds <4 x i8>, ptr undef, i32 1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %b7, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b8 = getelementptr inbounds <4 x i16>, ptr undef, i32 1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %b8, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b9 = getelementptr inbounds <4 x i32>, ptr undef, i32 1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %b9, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b10 = getelementptr inbounds <4 x i64>, ptr undef, i32 1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %b10, align 32
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b11 = getelementptr inbounds <4 x half>, ptr undef, i32 1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %b11, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b12 = getelementptr inbounds <4 x float>, ptr undef, i32 1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %b12, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b13 = getelementptr inbounds <4 x double>, ptr undef, i32 1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %b13, align 32
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o7 = getelementptr inbounds <4 x i8>, ptr undef, i32 4
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %o7, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o8 = getelementptr inbounds <4 x i16>, ptr undef, i32 4
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %o8, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o9 = getelementptr inbounds <4 x i32>, ptr undef, i32 4
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %o9, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o10 = getelementptr inbounds <4 x i64>, ptr undef, i32 4
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %o10, align 32
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o11 = getelementptr inbounds <4 x half>, ptr undef, i32 4
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %o11, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o12 = getelementptr inbounds <4 x float>, ptr undef, i32 4
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %o12, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o13 = getelementptr inbounds <4 x double>, ptr undef, i32 4
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %o13, align 32
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %p7 = getelementptr inbounds <4 x i8>, ptr undef, i32 31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %p7, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %p8 = getelementptr inbounds <4 x i16>, ptr undef, i32 31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %p8, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p9 = getelementptr inbounds <4 x i32>, ptr undef, i32 31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %p9, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p10 = getelementptr inbounds <4 x i64>, ptr undef, i32 31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %p10, align 32
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %p11 = getelementptr inbounds <4 x half>, ptr undef, i32 31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %p11, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p12 = getelementptr inbounds <4 x float>, ptr undef, i32 31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %p12, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p13 = getelementptr inbounds <4 x double>, ptr undef, i32 31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %p13, align 32
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %q7 = getelementptr inbounds <4 x i8>, ptr undef, i32 32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %q7, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %q8 = getelementptr inbounds <4 x i16>, ptr undef, i32 32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %q8, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q9 = getelementptr inbounds <4 x i32>, ptr undef, i32 32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %q9, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q10 = getelementptr inbounds <4 x i64>, ptr undef, i32 32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %q10, align 32
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %q11 = getelementptr inbounds <4 x half>, ptr undef, i32 32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %q11, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q12 = getelementptr inbounds <4 x float>, ptr undef, i32 32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %q12, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q13 = getelementptr inbounds <4 x double>, ptr undef, i32 32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %q13, align 32
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %r7 = getelementptr inbounds <4 x i8>, ptr undef, i32 -31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %r7, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %r8 = getelementptr inbounds <4 x i16>, ptr undef, i32 -31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %r8, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r9 = getelementptr inbounds <4 x i32>, ptr undef, i32 -31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %r9, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r10 = getelementptr inbounds <4 x i64>, ptr undef, i32 -31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %r10, align 32
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %r11 = getelementptr inbounds <4 x half>, ptr undef, i32 -31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %r11, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r12 = getelementptr inbounds <4 x float>, ptr undef, i32 -31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %r12, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r13 = getelementptr inbounds <4 x double>, ptr undef, i32 -31
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %r13, align 32
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %s7 = getelementptr inbounds <4 x i8>, ptr undef, i32 -32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %s7, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %s8 = getelementptr inbounds <4 x i16>, ptr undef, i32 -32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %s8, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s9 = getelementptr inbounds <4 x i32>, ptr undef, i32 -32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %s9, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s10 = getelementptr inbounds <4 x i64>, ptr undef, i32 -32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %s10, align 32
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %s11 = getelementptr inbounds <4 x half>, ptr undef, i32 -32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %s11, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s12 = getelementptr inbounds <4 x float>, ptr undef, i32 -32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %s12, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s13 = getelementptr inbounds <4 x double>, ptr undef, i32 -32
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %s13, align 32
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c7 = getelementptr inbounds <4 x i8>, ptr undef, i32 %i
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %c7, align 4
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c8 = getelementptr inbounds <4 x i16>, ptr undef, i32 %i
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %c8, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c9 = getelementptr inbounds <4 x i32>, ptr undef, i32 %i
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %c9, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c10 = getelementptr inbounds <4 x i64>, ptr undef, i32 %i
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %c10, align 32
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c11 = getelementptr inbounds <4 x half>, ptr undef, i32 %i
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %c11, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c12 = getelementptr inbounds <4 x float>, ptr undef, i32 %i
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %c12, align 8
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c13 = getelementptr inbounds <4 x double>, ptr undef, i32 %i
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %c13, align 32
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %d0 = getelementptr inbounds i8, ptr undef, i32 -1
+; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %d0, align 1
 ; CHECK-V7M-NOFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-V7M-FP-LABEL: 'testvecs'
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b7 = getelementptr inbounds <4 x i8>, ptr undef, i32 1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %b7, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b8 = getelementptr inbounds <4 x i16>, ptr undef, i32 1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %b8, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b9 = getelementptr inbounds <4 x i32>, ptr undef, i32 1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %b9, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b10 = getelementptr inbounds <4 x i64>, ptr undef, i32 1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %b10, align 32
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b11 = getelementptr inbounds <4 x half>, ptr undef, i32 1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %b11, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b12 = getelementptr inbounds <4 x float>, ptr undef, i32 1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %b12, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b13 = getelementptr inbounds <4 x double>, ptr undef, i32 1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %b13, align 32
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o7 = getelementptr inbounds <4 x i8>, ptr undef, i32 4
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %o7, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o8 = getelementptr inbounds <4 x i16>, ptr undef, i32 4
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %o8, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o9 = getelementptr inbounds <4 x i32>, ptr undef, i32 4
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %o9, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o10 = getelementptr inbounds <4 x i64>, ptr undef, i32 4
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %o10, align 32
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o11 = getelementptr inbounds <4 x half>, ptr undef, i32 4
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %o11, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o12 = getelementptr inbounds <4 x float>, ptr undef, i32 4
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %o12, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o13 = getelementptr inbounds <4 x double>, ptr undef, i32 4
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %o13, align 32
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %p7 = getelementptr inbounds <4 x i8>, ptr undef, i32 31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %p7, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %p8 = getelementptr inbounds <4 x i16>, ptr undef, i32 31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %p8, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p9 = getelementptr inbounds <4 x i32>, ptr undef, i32 31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %p9, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p10 = getelementptr inbounds <4 x i64>, ptr undef, i32 31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %p10, align 32
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %p11 = getelementptr inbounds <4 x half>, ptr undef, i32 31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %p11, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %p12 = getelementptr inbounds <4 x float>, ptr undef, i32 31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %p12, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %p13 = getelementptr inbounds <4 x double>, ptr undef, i32 31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %p13, align 32
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %q7 = getelementptr inbounds <4 x i8>, ptr undef, i32 32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %q7, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %q8 = getelementptr inbounds <4 x i16>, ptr undef, i32 32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %q8, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q9 = getelementptr inbounds <4 x i32>, ptr undef, i32 32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %q9, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q10 = getelementptr inbounds <4 x i64>, ptr undef, i32 32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %q10, align 32
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %q11 = getelementptr inbounds <4 x half>, ptr undef, i32 32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %q11, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %q12 = getelementptr inbounds <4 x float>, ptr undef, i32 32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %q12, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q13 = getelementptr inbounds <4 x double>, ptr undef, i32 32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %q13, align 32
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %r7 = getelementptr inbounds <4 x i8>, ptr undef, i32 -31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %r7, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %r8 = getelementptr inbounds <4 x i16>, ptr undef, i32 -31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %r8, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r9 = getelementptr inbounds <4 x i32>, ptr undef, i32 -31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %r9, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r10 = getelementptr inbounds <4 x i64>, ptr undef, i32 -31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %r10, align 32
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %r11 = getelementptr inbounds <4 x half>, ptr undef, i32 -31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %r11, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %r12 = getelementptr inbounds <4 x float>, ptr undef, i32 -31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %r12, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %r13 = getelementptr inbounds <4 x double>, ptr undef, i32 -31
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %r13, align 32
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %s7 = getelementptr inbounds <4 x i8>, ptr undef, i32 -32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %s7, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %s8 = getelementptr inbounds <4 x i16>, ptr undef, i32 -32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %s8, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s9 = getelementptr inbounds <4 x i32>, ptr undef, i32 -32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %s9, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s10 = getelementptr inbounds <4 x i64>, ptr undef, i32 -32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %s10, align 32
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %s11 = getelementptr inbounds <4 x half>, ptr undef, i32 -32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %s11, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %s12 = getelementptr inbounds <4 x float>, ptr undef, i32 -32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %s12, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s13 = getelementptr inbounds <4 x double>, ptr undef, i32 -32
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %s13, align 32
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c7 = getelementptr inbounds <4 x i8>, ptr undef, i32 %i
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i8> undef, ptr %c7, align 4
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c8 = getelementptr inbounds <4 x i16>, ptr undef, i32 %i
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i16> undef, ptr %c8, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c9 = getelementptr inbounds <4 x i32>, ptr undef, i32 %i
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i32> undef, ptr %c9, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c10 = getelementptr inbounds <4 x i64>, ptr undef, i32 %i
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x i64> undef, ptr %c10, align 32
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c11 = getelementptr inbounds <4 x half>, ptr undef, i32 %i
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %c11, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c12 = getelementptr inbounds <4 x float>, ptr undef, i32 %i
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x float> undef, ptr %c12, align 8
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c13 = getelementptr inbounds <4 x double>, ptr undef, i32 %i
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %c13, align 32
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %d0 = getelementptr inbounds i8, ptr undef, i32 -1
+; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %d0, align 1
 ; CHECK-V7M-FP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; CHECK-MVE-LABEL: 'testvecs'
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b7 = getelementptr inbounds <4 x i8>, ptr undef, i32 1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i8> undef, ptr %b7, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b8 = getelementptr inbounds <4 x i16>, ptr undef, i32 1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i16> undef, ptr %b8, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b9 = getelementptr inbounds <4 x i32>, ptr undef, i32 1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i32> undef, ptr %b9, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b10 = getelementptr inbounds <4 x i64>, ptr undef, i32 1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i64> undef, ptr %b10, align 32
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b11 = getelementptr inbounds <4 x half>, ptr undef, i32 1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: store volatile <4 x half> undef, ptr %b11, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b12 = getelementptr inbounds <4 x float>, ptr undef, i32 1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x float> undef, ptr %b12, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b13 = getelementptr inbounds <4 x double>, ptr undef, i32 1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %b13, align 32
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o7 = getelementptr inbounds <4 x i8>, ptr undef, i32 4
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i8> undef, ptr %o7, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o8 = getelementptr inbounds <4 x i16>, ptr undef, i32 4
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i16> undef, ptr %o8, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o9 = getelementptr inbounds <4 x i32>, ptr undef, i32 4
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i32> undef, ptr %o9, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o10 = getelementptr inbounds <4 x i64>, ptr undef, i32 4
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i64> undef, ptr %o10, align 32
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o11 = getelementptr inbounds <4 x half>, ptr undef, i32 4
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: store volatile <4 x half> undef, ptr %o11, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o12 = getelementptr inbounds <4 x float>, ptr undef, i32 4
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x float> undef, ptr %o12, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o13 = getelementptr inbounds <4 x double>, ptr undef, i32 4
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %o13, align 32
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %p7 = getelementptr inbounds <4 x i8>, ptr undef, i32 31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i8> undef, ptr %p7, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %p8 = getelementptr inbounds <4 x i16>, ptr undef, i32 31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i16> undef, ptr %p8, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %p9 = getelementptr inbounds <4 x i32>, ptr undef, i32 31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i32> undef, ptr %p9, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p10 = getelementptr inbounds <4 x i64>, ptr undef, i32 31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i64> undef, ptr %p10, align 32
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p11 = getelementptr inbounds <4 x half>, ptr undef, i32 31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: store volatile <4 x half> undef, ptr %p11, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p12 = getelementptr inbounds <4 x float>, ptr undef, i32 31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x float> undef, ptr %p12, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p13 = getelementptr inbounds <4 x double>, ptr undef, i32 31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %p13, align 32
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q7 = getelementptr inbounds <4 x i8>, ptr undef, i32 32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i8> undef, ptr %q7, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q8 = getelementptr inbounds <4 x i16>, ptr undef, i32 32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i16> undef, ptr %q8, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q9 = getelementptr inbounds <4 x i32>, ptr undef, i32 32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i32> undef, ptr %q9, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q10 = getelementptr inbounds <4 x i64>, ptr undef, i32 32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i64> undef, ptr %q10, align 32
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q11 = getelementptr inbounds <4 x half>, ptr undef, i32 32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: store volatile <4 x half> undef, ptr %q11, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q12 = getelementptr inbounds <4 x float>, ptr undef, i32 32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x float> undef, ptr %q12, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q13 = getelementptr inbounds <4 x double>, ptr undef, i32 32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %q13, align 32
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %r7 = getelementptr inbounds <4 x i8>, ptr undef, i32 -31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i8> undef, ptr %r7, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %r8 = getelementptr inbounds <4 x i16>, ptr undef, i32 -31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i16> undef, ptr %r8, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %r9 = getelementptr inbounds <4 x i32>, ptr undef, i32 -31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i32> undef, ptr %r9, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r10 = getelementptr inbounds <4 x i64>, ptr undef, i32 -31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i64> undef, ptr %r10, align 32
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r11 = getelementptr inbounds <4 x half>, ptr undef, i32 -31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: store volatile <4 x half> undef, ptr %r11, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r12 = getelementptr inbounds <4 x float>, ptr undef, i32 -31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x float> undef, ptr %r12, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r13 = getelementptr inbounds <4 x double>, ptr undef, i32 -31
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %r13, align 32
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s7 = getelementptr inbounds <4 x i8>, ptr undef, i32 -32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i8> undef, ptr %s7, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s8 = getelementptr inbounds <4 x i16>, ptr undef, i32 -32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i16> undef, ptr %s8, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s9 = getelementptr inbounds <4 x i32>, ptr undef, i32 -32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i32> undef, ptr %s9, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s10 = getelementptr inbounds <4 x i64>, ptr undef, i32 -32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i64> undef, ptr %s10, align 32
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s11 = getelementptr inbounds <4 x half>, ptr undef, i32 -32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: store volatile <4 x half> undef, ptr %s11, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s12 = getelementptr inbounds <4 x float>, ptr undef, i32 -32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x float> undef, ptr %s12, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s13 = getelementptr inbounds <4 x double>, ptr undef, i32 -32
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %s13, align 32
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c7 = getelementptr inbounds <4 x i8>, ptr undef, i32 %i
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i8> undef, ptr %c7, align 4
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c8 = getelementptr inbounds <4 x i16>, ptr undef, i32 %i
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i16> undef, ptr %c8, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c9 = getelementptr inbounds <4 x i32>, ptr undef, i32 %i
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i32> undef, ptr %c9, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c10 = getelementptr inbounds <4 x i64>, ptr undef, i32 %i
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i64> undef, ptr %c10, align 32
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c11 = getelementptr inbounds <4 x half>, ptr undef, i32 %i
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: store volatile <4 x half> undef, ptr %c11, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c12 = getelementptr inbounds <4 x float>, ptr undef, i32 %i
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x float> undef, ptr %c12, align 8
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c13 = getelementptr inbounds <4 x double>, ptr undef, i32 %i
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %c13, align 32
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %d0 = getelementptr inbounds i8, ptr undef, i32 -1
+; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %d0, align 1
 ; CHECK-MVE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-MVEFP-LABEL: 'testvecs'
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b7 = getelementptr inbounds <4 x i8>, ptr undef, i32 1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i8> undef, ptr %b7, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b8 = getelementptr inbounds <4 x i16>, ptr undef, i32 1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i16> undef, ptr %b8, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b9 = getelementptr inbounds <4 x i32>, ptr undef, i32 1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i32> undef, ptr %b9, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b10 = getelementptr inbounds <4 x i64>, ptr undef, i32 1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i64> undef, ptr %b10, align 32
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b11 = getelementptr inbounds <4 x half>, ptr undef, i32 1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: store volatile <4 x half> undef, ptr %b11, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b12 = getelementptr inbounds <4 x float>, ptr undef, i32 1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x float> undef, ptr %b12, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b13 = getelementptr inbounds <4 x double>, ptr undef, i32 1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %b13, align 32
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o7 = getelementptr inbounds <4 x i8>, ptr undef, i32 4
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i8> undef, ptr %o7, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o8 = getelementptr inbounds <4 x i16>, ptr undef, i32 4
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i16> undef, ptr %o8, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o9 = getelementptr inbounds <4 x i32>, ptr undef, i32 4
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i32> undef, ptr %o9, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o10 = getelementptr inbounds <4 x i64>, ptr undef, i32 4
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i64> undef, ptr %o10, align 32
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o11 = getelementptr inbounds <4 x half>, ptr undef, i32 4
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: store volatile <4 x half> undef, ptr %o11, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %o12 = getelementptr inbounds <4 x float>, ptr undef, i32 4
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x float> undef, ptr %o12, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o13 = getelementptr inbounds <4 x double>, ptr undef, i32 4
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %o13, align 32
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %p7 = getelementptr inbounds <4 x i8>, ptr undef, i32 31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i8> undef, ptr %p7, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %p8 = getelementptr inbounds <4 x i16>, ptr undef, i32 31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i16> undef, ptr %p8, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %p9 = getelementptr inbounds <4 x i32>, ptr undef, i32 31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i32> undef, ptr %p9, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p10 = getelementptr inbounds <4 x i64>, ptr undef, i32 31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i64> undef, ptr %p10, align 32
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %p11 = getelementptr inbounds <4 x half>, ptr undef, i32 31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: store volatile <4 x half> undef, ptr %p11, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %p12 = getelementptr inbounds <4 x float>, ptr undef, i32 31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x float> undef, ptr %p12, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p13 = getelementptr inbounds <4 x double>, ptr undef, i32 31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %p13, align 32
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q7 = getelementptr inbounds <4 x i8>, ptr undef, i32 32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i8> undef, ptr %q7, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q8 = getelementptr inbounds <4 x i16>, ptr undef, i32 32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i16> undef, ptr %q8, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q9 = getelementptr inbounds <4 x i32>, ptr undef, i32 32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i32> undef, ptr %q9, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q10 = getelementptr inbounds <4 x i64>, ptr undef, i32 32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i64> undef, ptr %q10, align 32
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q11 = getelementptr inbounds <4 x half>, ptr undef, i32 32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: store volatile <4 x half> undef, ptr %q11, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q12 = getelementptr inbounds <4 x float>, ptr undef, i32 32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x float> undef, ptr %q12, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q13 = getelementptr inbounds <4 x double>, ptr undef, i32 32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %q13, align 32
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %r7 = getelementptr inbounds <4 x i8>, ptr undef, i32 -31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i8> undef, ptr %r7, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %r8 = getelementptr inbounds <4 x i16>, ptr undef, i32 -31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i16> undef, ptr %r8, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %r9 = getelementptr inbounds <4 x i32>, ptr undef, i32 -31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i32> undef, ptr %r9, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r10 = getelementptr inbounds <4 x i64>, ptr undef, i32 -31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i64> undef, ptr %r10, align 32
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %r11 = getelementptr inbounds <4 x half>, ptr undef, i32 -31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: store volatile <4 x half> undef, ptr %r11, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %r12 = getelementptr inbounds <4 x float>, ptr undef, i32 -31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x float> undef, ptr %r12, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r13 = getelementptr inbounds <4 x double>, ptr undef, i32 -31
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %r13, align 32
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s7 = getelementptr inbounds <4 x i8>, ptr undef, i32 -32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i8> undef, ptr %s7, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s8 = getelementptr inbounds <4 x i16>, ptr undef, i32 -32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i16> undef, ptr %s8, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s9 = getelementptr inbounds <4 x i32>, ptr undef, i32 -32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i32> undef, ptr %s9, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s10 = getelementptr inbounds <4 x i64>, ptr undef, i32 -32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i64> undef, ptr %s10, align 32
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s11 = getelementptr inbounds <4 x half>, ptr undef, i32 -32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: store volatile <4 x half> undef, ptr %s11, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s12 = getelementptr inbounds <4 x float>, ptr undef, i32 -32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x float> undef, ptr %s12, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s13 = getelementptr inbounds <4 x double>, ptr undef, i32 -32
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %s13, align 32
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c7 = getelementptr inbounds <4 x i8>, ptr undef, i32 %i
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i8> undef, ptr %c7, align 4
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c8 = getelementptr inbounds <4 x i16>, ptr undef, i32 %i
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i16> undef, ptr %c8, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c9 = getelementptr inbounds <4 x i32>, ptr undef, i32 %i
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i32> undef, ptr %c9, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c10 = getelementptr inbounds <4 x i64>, ptr undef, i32 %i
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x i64> undef, ptr %c10, align 32
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c11 = getelementptr inbounds <4 x half>, ptr undef, i32 %i
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: store volatile <4 x half> undef, ptr %c11, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c12 = getelementptr inbounds <4 x float>, ptr undef, i32 %i
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x float> undef, ptr %c12, align 8
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c13 = getelementptr inbounds <4 x double>, ptr undef, i32 %i
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x double> undef, ptr %c13, align 32
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %d0 = getelementptr inbounds i8, ptr undef, i32 -1
+; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %d0, align 1
 ; CHECK-MVEFP-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-T32-LABEL: 'testvecs'
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b7 = getelementptr inbounds <4 x i8>, ptr undef, i32 1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 13 for instruction: store volatile <4 x i8> undef, ptr %b7, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b8 = getelementptr inbounds <4 x i16>, ptr undef, i32 1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i16> undef, ptr %b8, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b9 = getelementptr inbounds <4 x i32>, ptr undef, i32 1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i32> undef, ptr %b9, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b10 = getelementptr inbounds <4 x i64>, ptr undef, i32 1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i64> undef, ptr %b10, align 32
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b11 = getelementptr inbounds <4 x half>, ptr undef, i32 1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %b11, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b12 = getelementptr inbounds <4 x float>, ptr undef, i32 1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x float> undef, ptr %b12, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b13 = getelementptr inbounds <4 x double>, ptr undef, i32 1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %b13, align 32
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o7 = getelementptr inbounds <4 x i8>, ptr undef, i32 4
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 13 for instruction: store volatile <4 x i8> undef, ptr %o7, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o8 = getelementptr inbounds <4 x i16>, ptr undef, i32 4
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i16> undef, ptr %o8, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o9 = getelementptr inbounds <4 x i32>, ptr undef, i32 4
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i32> undef, ptr %o9, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o10 = getelementptr inbounds <4 x i64>, ptr undef, i32 4
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i64> undef, ptr %o10, align 32
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o11 = getelementptr inbounds <4 x half>, ptr undef, i32 4
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %o11, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o12 = getelementptr inbounds <4 x float>, ptr undef, i32 4
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x float> undef, ptr %o12, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o13 = getelementptr inbounds <4 x double>, ptr undef, i32 4
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %o13, align 32
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p7 = getelementptr inbounds <4 x i8>, ptr undef, i32 31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 13 for instruction: store volatile <4 x i8> undef, ptr %p7, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p8 = getelementptr inbounds <4 x i16>, ptr undef, i32 31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i16> undef, ptr %p8, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p9 = getelementptr inbounds <4 x i32>, ptr undef, i32 31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i32> undef, ptr %p9, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p10 = getelementptr inbounds <4 x i64>, ptr undef, i32 31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i64> undef, ptr %p10, align 32
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p11 = getelementptr inbounds <4 x half>, ptr undef, i32 31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %p11, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p12 = getelementptr inbounds <4 x float>, ptr undef, i32 31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x float> undef, ptr %p12, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p13 = getelementptr inbounds <4 x double>, ptr undef, i32 31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %p13, align 32
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q7 = getelementptr inbounds <4 x i8>, ptr undef, i32 32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 13 for instruction: store volatile <4 x i8> undef, ptr %q7, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q8 = getelementptr inbounds <4 x i16>, ptr undef, i32 32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i16> undef, ptr %q8, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q9 = getelementptr inbounds <4 x i32>, ptr undef, i32 32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i32> undef, ptr %q9, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q10 = getelementptr inbounds <4 x i64>, ptr undef, i32 32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i64> undef, ptr %q10, align 32
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q11 = getelementptr inbounds <4 x half>, ptr undef, i32 32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %q11, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q12 = getelementptr inbounds <4 x float>, ptr undef, i32 32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x float> undef, ptr %q12, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q13 = getelementptr inbounds <4 x double>, ptr undef, i32 32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %q13, align 32
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r7 = getelementptr inbounds <4 x i8>, ptr undef, i32 -31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 13 for instruction: store volatile <4 x i8> undef, ptr %r7, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r8 = getelementptr inbounds <4 x i16>, ptr undef, i32 -31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i16> undef, ptr %r8, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r9 = getelementptr inbounds <4 x i32>, ptr undef, i32 -31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i32> undef, ptr %r9, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r10 = getelementptr inbounds <4 x i64>, ptr undef, i32 -31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i64> undef, ptr %r10, align 32
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r11 = getelementptr inbounds <4 x half>, ptr undef, i32 -31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %r11, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r12 = getelementptr inbounds <4 x float>, ptr undef, i32 -31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x float> undef, ptr %r12, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r13 = getelementptr inbounds <4 x double>, ptr undef, i32 -31
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %r13, align 32
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s7 = getelementptr inbounds <4 x i8>, ptr undef, i32 -32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 13 for instruction: store volatile <4 x i8> undef, ptr %s7, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s8 = getelementptr inbounds <4 x i16>, ptr undef, i32 -32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i16> undef, ptr %s8, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s9 = getelementptr inbounds <4 x i32>, ptr undef, i32 -32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i32> undef, ptr %s9, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s10 = getelementptr inbounds <4 x i64>, ptr undef, i32 -32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i64> undef, ptr %s10, align 32
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s11 = getelementptr inbounds <4 x half>, ptr undef, i32 -32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %s11, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s12 = getelementptr inbounds <4 x float>, ptr undef, i32 -32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x float> undef, ptr %s12, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s13 = getelementptr inbounds <4 x double>, ptr undef, i32 -32
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %s13, align 32
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c7 = getelementptr inbounds <4 x i8>, ptr undef, i32 %i
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 13 for instruction: store volatile <4 x i8> undef, ptr %c7, align 4
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c8 = getelementptr inbounds <4 x i16>, ptr undef, i32 %i
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i16> undef, ptr %c8, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c9 = getelementptr inbounds <4 x i32>, ptr undef, i32 %i
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i32> undef, ptr %c9, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c10 = getelementptr inbounds <4 x i64>, ptr undef, i32 %i
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i64> undef, ptr %c10, align 32
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c11 = getelementptr inbounds <4 x half>, ptr undef, i32 %i
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %c11, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c12 = getelementptr inbounds <4 x float>, ptr undef, i32 %i
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x float> undef, ptr %c12, align 8
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c13 = getelementptr inbounds <4 x double>, ptr undef, i32 %i
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %c13, align 32
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %d0 = getelementptr inbounds i8, ptr undef, i32 -1
+; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %d0, align 1
 ; CHECK-T32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; CHECK-A32-LABEL: 'testvecs'
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b7 = getelementptr inbounds <4 x i8>, ptr undef, i32 1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 13 for instruction: store volatile <4 x i8> undef, ptr %b7, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b8 = getelementptr inbounds <4 x i16>, ptr undef, i32 1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i16> undef, ptr %b8, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b9 = getelementptr inbounds <4 x i32>, ptr undef, i32 1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i32> undef, ptr %b9, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b10 = getelementptr inbounds <4 x i64>, ptr undef, i32 1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i64> undef, ptr %b10, align 32
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b11 = getelementptr inbounds <4 x half>, ptr undef, i32 1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %b11, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b12 = getelementptr inbounds <4 x float>, ptr undef, i32 1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x float> undef, ptr %b12, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b13 = getelementptr inbounds <4 x double>, ptr undef, i32 1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %b13, align 32
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o7 = getelementptr inbounds <4 x i8>, ptr undef, i32 4
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 13 for instruction: store volatile <4 x i8> undef, ptr %o7, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o8 = getelementptr inbounds <4 x i16>, ptr undef, i32 4
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i16> undef, ptr %o8, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o9 = getelementptr inbounds <4 x i32>, ptr undef, i32 4
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i32> undef, ptr %o9, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o10 = getelementptr inbounds <4 x i64>, ptr undef, i32 4
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i64> undef, ptr %o10, align 32
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o11 = getelementptr inbounds <4 x half>, ptr undef, i32 4
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %o11, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o12 = getelementptr inbounds <4 x float>, ptr undef, i32 4
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x float> undef, ptr %o12, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %o13 = getelementptr inbounds <4 x double>, ptr undef, i32 4
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %o13, align 32
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p7 = getelementptr inbounds <4 x i8>, ptr undef, i32 31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 13 for instruction: store volatile <4 x i8> undef, ptr %p7, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p8 = getelementptr inbounds <4 x i16>, ptr undef, i32 31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i16> undef, ptr %p8, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p9 = getelementptr inbounds <4 x i32>, ptr undef, i32 31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i32> undef, ptr %p9, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p10 = getelementptr inbounds <4 x i64>, ptr undef, i32 31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i64> undef, ptr %p10, align 32
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p11 = getelementptr inbounds <4 x half>, ptr undef, i32 31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %p11, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p12 = getelementptr inbounds <4 x float>, ptr undef, i32 31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x float> undef, ptr %p12, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %p13 = getelementptr inbounds <4 x double>, ptr undef, i32 31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %p13, align 32
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q7 = getelementptr inbounds <4 x i8>, ptr undef, i32 32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 13 for instruction: store volatile <4 x i8> undef, ptr %q7, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q8 = getelementptr inbounds <4 x i16>, ptr undef, i32 32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i16> undef, ptr %q8, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q9 = getelementptr inbounds <4 x i32>, ptr undef, i32 32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i32> undef, ptr %q9, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q10 = getelementptr inbounds <4 x i64>, ptr undef, i32 32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i64> undef, ptr %q10, align 32
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q11 = getelementptr inbounds <4 x half>, ptr undef, i32 32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %q11, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q12 = getelementptr inbounds <4 x float>, ptr undef, i32 32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x float> undef, ptr %q12, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %q13 = getelementptr inbounds <4 x double>, ptr undef, i32 32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %q13, align 32
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r7 = getelementptr inbounds <4 x i8>, ptr undef, i32 -31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 13 for instruction: store volatile <4 x i8> undef, ptr %r7, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r8 = getelementptr inbounds <4 x i16>, ptr undef, i32 -31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i16> undef, ptr %r8, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r9 = getelementptr inbounds <4 x i32>, ptr undef, i32 -31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i32> undef, ptr %r9, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r10 = getelementptr inbounds <4 x i64>, ptr undef, i32 -31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i64> undef, ptr %r10, align 32
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r11 = getelementptr inbounds <4 x half>, ptr undef, i32 -31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %r11, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r12 = getelementptr inbounds <4 x float>, ptr undef, i32 -31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x float> undef, ptr %r12, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %r13 = getelementptr inbounds <4 x double>, ptr undef, i32 -31
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %r13, align 32
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s7 = getelementptr inbounds <4 x i8>, ptr undef, i32 -32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 13 for instruction: store volatile <4 x i8> undef, ptr %s7, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s8 = getelementptr inbounds <4 x i16>, ptr undef, i32 -32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i16> undef, ptr %s8, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s9 = getelementptr inbounds <4 x i32>, ptr undef, i32 -32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i32> undef, ptr %s9, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s10 = getelementptr inbounds <4 x i64>, ptr undef, i32 -32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i64> undef, ptr %s10, align 32
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s11 = getelementptr inbounds <4 x half>, ptr undef, i32 -32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %s11, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s12 = getelementptr inbounds <4 x float>, ptr undef, i32 -32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x float> undef, ptr %s12, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %s13 = getelementptr inbounds <4 x double>, ptr undef, i32 -32
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %s13, align 32
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c7 = getelementptr inbounds <4 x i8>, ptr undef, i32 %i
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 13 for instruction: store volatile <4 x i8> undef, ptr %c7, align 4
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c8 = getelementptr inbounds <4 x i16>, ptr undef, i32 %i
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i16> undef, ptr %c8, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c9 = getelementptr inbounds <4 x i32>, ptr undef, i32 %i
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i32> undef, ptr %c9, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c10 = getelementptr inbounds <4 x i64>, ptr undef, i32 %i
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i64> undef, ptr %c10, align 32
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c11 = getelementptr inbounds <4 x half>, ptr undef, i32 %i
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: store volatile <4 x half> undef, ptr %c11, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c12 = getelementptr inbounds <4 x float>, ptr undef, i32 %i
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x float> undef, ptr %c12, align 8
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c13 = getelementptr inbounds <4 x double>, ptr undef, i32 %i
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: store volatile <4 x double> undef, ptr %c13, align 32
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %d0 = getelementptr inbounds i8, ptr undef, i32 -1
+; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %d0, align 1
 ; CHECK-A32-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 
   %b7 = getelementptr inbounds <4 x i8>, ptr undef, i32 1
+  store volatile <4 x i8> undef, ptr %b7
   %b8 = getelementptr inbounds <4 x i16>, ptr undef, i32 1
+  store volatile <4 x i16> undef, ptr %b8
   %b9 = getelementptr inbounds <4 x i32>, ptr undef, i32 1
+  store volatile <4 x i32> undef, ptr %b9
   %b10 = getelementptr inbounds <4 x i64>, ptr undef, i32 1
+  store volatile <4 x i64> undef, ptr %b10
   %b11 = getelementptr inbounds <4 x half>, ptr undef, i32 1
+  store volatile <4 x half> undef, ptr %b11
   %b12 = getelementptr inbounds <4 x float>, ptr undef, i32 1
+  store volatile <4 x float> undef, ptr %b12
   %b13 = getelementptr inbounds <4 x double>, ptr undef, i32 1
+  store volatile <4 x double> undef, ptr %b13
 
   %o7 = getelementptr inbounds <4 x i8>, ptr undef, i32 4
+  store volatile <4 x i8> undef, ptr %o7
   %o8 = getelementptr inbounds <4 x i16>, ptr undef, i32 4
+  store volatile <4 x i16> undef, ptr %o8
   %o9 = getelementptr inbounds <4 x i32>, ptr undef, i32 4
+  store volatile <4 x i32> undef, ptr %o9
   %o10 = getelementptr inbounds <4 x i64>, ptr undef, i32 4
+  store volatile <4 x i64> undef, ptr %o10
   %o11 = getelementptr inbounds <4 x half>, ptr undef, i32 4
+  store volatile <4 x half> undef, ptr %o11
   %o12 = getelementptr inbounds <4 x float>, ptr undef, i32 4
+  store volatile <4 x float> undef, ptr %o12
   %o13 = getelementptr inbounds <4 x double>, ptr undef, i32 4
+  store volatile <4 x double> undef, ptr %o13
 
   %p7 = getelementptr inbounds <4 x i8>, ptr undef, i32 31
+  store volatile <4 x i8> undef, ptr %p7
   %p8 = getelementptr inbounds <4 x i16>, ptr undef, i32 31
+  store volatile <4 x i16> undef, ptr %p8
   %p9 = getelementptr inbounds <4 x i32>, ptr undef, i32 31
+  store volatile <4 x i32> undef, ptr %p9
   %p10 = getelementptr inbounds <4 x i64>, ptr undef, i32 31
+  store volatile <4 x i64> undef, ptr %p10
   %p11 = getelementptr inbounds <4 x half>, ptr undef, i32 31
+  store volatile <4 x half> undef, ptr %p11
   %p12 = getelementptr inbounds <4 x float>, ptr undef, i32 31
+  store volatile <4 x float> undef, ptr %p12
   %p13 = getelementptr inbounds <4 x double>, ptr undef, i32 31
+  store volatile <4 x double> undef, ptr %p13
 
   %q7 = getelementptr inbounds <4 x i8>, ptr undef, i32 32
+  store volatile <4 x i8> undef, ptr %q7
   %q8 = getelementptr inbounds <4 x i16>, ptr undef, i32 32
+  store volatile <4 x i16> undef, ptr %q8
   %q9 = getelementptr inbounds <4 x i32>, ptr undef, i32 32
+  store volatile <4 x i32> undef, ptr %q9
   %q10 = getelementptr inbounds <4 x i64>, ptr undef, i32 32
+  store volatile <4 x i64> undef, ptr %q10
   %q11 = getelementptr inbounds <4 x half>, ptr undef, i32 32
+  store volatile <4 x half> undef, ptr %q11
   %q12 = getelementptr inbounds <4 x float>, ptr undef, i32 32
+  store volatile <4 x float> undef, ptr %q12
   %q13 = getelementptr inbounds <4 x double>, ptr undef, i32 32
+  store volatile <4 x double> undef, ptr %q13
 
   %r7 = getelementptr inbounds <4 x i8>, ptr undef, i32 -31
+  store volatile <4 x i8> undef, ptr %r7
   %r8 = getelementptr inbounds <4 x i16>, ptr undef, i32 -31
+  store volatile <4 x i16> undef, ptr %r8
   %r9 = getelementptr inbounds <4 x i32>, ptr undef, i32 -31
+  store volatile <4 x i32> undef, ptr %r9
   %r10 = getelementptr inbounds <4 x i64>, ptr undef, i32 -31
+  store volatile <4 x i64> undef, ptr %r10
   %r11 = getelementptr inbounds <4 x half>, ptr undef, i32 -31
+  store volatile <4 x half> undef, ptr %r11
   %r12 = getelementptr inbounds <4 x float>, ptr undef, i32 -31
+  store volatile <4 x float> undef, ptr %r12
   %r13 = getelementptr inbounds <4 x double>, ptr undef, i32 -31
+  store volatile <4 x double> undef, ptr %r13
 
   %s7 = getelementptr inbounds <4 x i8>, ptr undef, i32 -32
+  store volatile <4 x i8> undef, ptr %s7
   %s8 = getelementptr inbounds <4 x i16>, ptr undef, i32 -32
+  store volatile <4 x i16> undef, ptr %s8
   %s9 = getelementptr inbounds <4 x i32>, ptr undef, i32 -32
+  store volatile <4 x i32> undef, ptr %s9
   %s10 = getelementptr inbounds <4 x i64>, ptr undef, i32 -32
+  store volatile <4 x i64> undef, ptr %s10
   %s11 = getelementptr inbounds <4 x half>, ptr undef, i32 -32
+  store volatile <4 x half> undef, ptr %s11
   %s12 = getelementptr inbounds <4 x float>, ptr undef, i32 -32
+  store volatile <4 x float> undef, ptr %s12
   %s13 = getelementptr inbounds <4 x double>, ptr undef, i32 -32
+  store volatile <4 x double> undef, ptr %s13
 
   %c7 = getelementptr inbounds <4 x i8>, ptr undef, i32 %i
+  store volatile <4 x i8> undef, ptr %c7
   %c8 = getelementptr inbounds <4 x i16>, ptr undef, i32 %i
+  store volatile <4 x i16> undef, ptr %c8
   %c9 = getelementptr inbounds <4 x i32>, ptr undef, i32 %i
+  store volatile <4 x i32> undef, ptr %c9
   %c10 = getelementptr inbounds <4 x i64>, ptr undef, i32 %i
+  store volatile <4 x i64> undef, ptr %c10
   %c11 = getelementptr inbounds <4 x half>, ptr undef, i32 %i
+  store volatile <4 x half> undef, ptr %c11
   %c12 = getelementptr inbounds <4 x float>, ptr undef, i32 %i
+  store volatile <4 x float> undef, ptr %c12
   %c13 = getelementptr inbounds <4 x double>, ptr undef, i32 %i
+  store volatile <4 x double> undef, ptr %c13
 
   %d0 = getelementptr inbounds i8, ptr undef, i32 -1
+  store volatile i8 undef, ptr %d0
 
   ret void
 }
 
 

llvm/test/Analysis/CostModel/ARM/mve-gather-scatter-cost.ll

;
   call void @llvm.masked.scatter.v4i8.v4p0(<4 x i8> undef, <4 x ptr> undef, i32 1, <4 x i1> undef)
   call void @llvm.masked.scatter.v2i8.v2p0(<2 x i8> undef, <2 x ptr> undef, i32 1, <2 x i1> undef)
 
   ret i32 0
 }
 
 define void @gep_v4i32(ptr %base, ptr %base16, ptr %base8, <4 x i32> %ind32, <4 x i16> %ind16, <4 x i1> %mask)  {
 ; CHECK-LABEL: 'gep_v4i32'
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gep1 = getelementptr i32, ptr %base, <4 x i32> %ind32
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep1 = getelementptr i32, ptr %base, <4 x i32> %ind32
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: %res1 = call <4 x i32> @llvm.masked.gather.v4i32.v4p0(<4 x ptr> %gep1, i32 4, <4 x i1> %mask, <4 x i32> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: call void @llvm.masked.scatter.v4i32.v4p0(<4 x i32> %res1, <4 x ptr> %gep1, i32 4, <4 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: %indzext = zext <4 x i16> %ind16 to <4 x i32>
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gep2 = getelementptr i32, ptr %base, <4 x i32> %indzext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep2 = getelementptr i32, ptr %base, <4 x i32> %indzext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: %res2 = call <4 x i32> @llvm.masked.gather.v4i32.v4p0(<4 x ptr> %gep2, i32 4, <4 x i1> %mask, <4 x i32> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: call void @llvm.masked.scatter.v4i32.v4p0(<4 x i32> %res2, <4 x ptr> %gep2, i32 4, <4 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: %indsext = sext <4 x i16> %ind16 to <4 x i32>
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gep3 = getelementptr i32, ptr %base, <4 x i32> %indsext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep3 = getelementptr i32, ptr %base, <4 x i32> %indsext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: %res3 = call <4 x i32> @llvm.masked.gather.v4i32.v4p0(<4 x ptr> %gep3, i32 4, <4 x i1> %mask, <4 x i32> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: call void @llvm.masked.scatter.v4i32.v4p0(<4 x i32> %res3, <4 x ptr> %gep3, i32 4, <4 x i1> %mask)
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gepu = getelementptr i32, ptr %base, <4 x i32> %ind32
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gepu = getelementptr i32, ptr %base, <4 x i32> %ind32
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 56 for instruction: %resu = call <4 x i32> @llvm.masked.gather.v4i32.v4p0(<4 x ptr> %gepu, i32 1, <4 x i1> %mask, <4 x i32> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 56 for instruction: call void @llvm.masked.scatter.v4i32.v4p0(<4 x i32> %resu, <4 x ptr> %gepu, i32 1, <4 x i1> %mask)
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gepos = getelementptr i8, ptr %base8, <4 x i32> %indzext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gepos = getelementptr i8, ptr %base8, <4 x i32> %indzext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %geposb = bitcast <4 x ptr> %gepos to <4 x ptr>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: %resos = call <4 x i32> @llvm.masked.gather.v4i32.v4p0(<4 x ptr> %geposb, i32 4, <4 x i1> %mask, <4 x i32> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: call void @llvm.masked.scatter.v4i32.v4p0(<4 x i32> %resos, <4 x ptr> %geposb, i32 4, <4 x i1> %mask)
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gepbs = getelementptr i16, ptr %base16, <4 x i32> %indzext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gepbs = getelementptr i16, ptr %base16, <4 x i32> %indzext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gepbsb = bitcast <4 x ptr> %gepbs to <4 x ptr>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: %resbs = call <4 x i32> @llvm.masked.gather.v4i32.v4p0(<4 x ptr> %gepbsb, i32 4, <4 x i1> %mask, <4 x i32> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: call void @llvm.masked.scatter.v4i32.v4p0(<4 x i32> %resbs, <4 x ptr> %gepbsb, i32 4, <4 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
   %gep1 = getelementptr i32, ptr %base, <4 x i32> %ind32
   %res1 = call <4 x i32> @llvm.masked.gather.v4i32.v4p0(<4 x ptr> %gep1, i32 4, <4 x i1> %mask, <4 x i32> undef)
   call void @llvm.masked.scatter.v4i32.v4p0(<4 x i32> %res1, <4 x ptr> %gep1, i32 4, <4 x i1> %mask)

 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: call void @llvm.masked.scatter.v4f32.v4p0(<4 x float> %res2, <4 x ptr> %gep2, i32 4, <4 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: %indsext = sext <4 x i16> %ind16 to <4 x i32>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep3 = getelementptr float, ptr %base, <4 x i32> %indsext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: %res3 = call <4 x float> @llvm.masked.gather.v4f32.v4p0(<4 x ptr> %gep3, i32 4, <4 x i1> %mask, <4 x float> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: call void @llvm.masked.scatter.v4f32.v4p0(<4 x float> %res3, <4 x ptr> %gep3, i32 4, <4 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gepu = getelementptr float, ptr %base, <4 x i32> %ind32
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 32 for instruction: %resu = call <4 x float> @llvm.masked.gather.v4f32.v4p0(<4 x ptr> %gepu, i32 1, <4 x i1> %mask, <4 x float> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 32 for instruction: call void @llvm.masked.scatter.v4f32.v4p0(<4 x float> %resu, <4 x ptr> %gepu, i32 1, <4 x i1> %mask)
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gepos = getelementptr i8, ptr %base8, <4 x i32> %indzext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gepos = getelementptr i8, ptr %base8, <4 x i32> %indzext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %geposb = bitcast <4 x ptr> %gepos to <4 x ptr>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: %resos = call <4 x float> @llvm.masked.gather.v4f32.v4p0(<4 x ptr> %geposb, i32 4, <4 x i1> %mask, <4 x float> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: call void @llvm.masked.scatter.v4f32.v4p0(<4 x float> %resos, <4 x ptr> %geposb, i32 4, <4 x i1> %mask)
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gepbs = getelementptr i16, ptr %base16, <4 x i32> %indzext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gepbs = getelementptr i16, ptr %base16, <4 x i32> %indzext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gepbsb = bitcast <4 x ptr> %gepbs to <4 x ptr>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: %resbs = call <4 x float> @llvm.masked.gather.v4f32.v4p0(<4 x ptr> %gepbsb, i32 4, <4 x i1> %mask, <4 x float> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: call void @llvm.masked.scatter.v4f32.v4p0(<4 x float> %resbs, <4 x ptr> %gepbsb, i32 4, <4 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
   %gep1 = getelementptr float, ptr %base, <4 x i32> %ind32
   %res1 = call <4 x float> @llvm.masked.gather.v4f32.v4p0(<4 x ptr> %gep1, i32 4, <4 x i1> %mask, <4 x float> undef)
   call void @llvm.masked.scatter.v4f32.v4p0(<4 x float> %res1, <4 x ptr> %gep1, i32 4, <4 x i1> %mask)
;
   %gepbsb = bitcast <4 x ptr> %gepbs to <4 x ptr>
   %resbs = call <4 x float> @llvm.masked.gather.v4f32.v4p0(<4 x ptr> %gepbsb, i32 4, <4 x i1> %mask, <4 x float> undef)
   call void @llvm.masked.scatter.v4f32.v4p0(<4 x float> %resbs, <4 x ptr> %gepbsb, i32 4, <4 x i1> %mask)
   ret void
 }
 
 define void @gep_v4i16(ptr %base, <4 x i32> %ind32, <4 x i16> %ind16, <4 x i1> %mask)  {
 ; CHECK-LABEL: 'gep_v4i16'
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gep1 = getelementptr i16, ptr %base, <4 x i32> %ind32
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep1 = getelementptr i16, ptr %base, <4 x i32> %ind32
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 56 for instruction: %res1 = call <4 x i16> @llvm.masked.gather.v4i16.v4p0(<4 x ptr> %gep1, i32 2, <4 x i1> %mask, <4 x i16> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 56 for instruction: call void @llvm.masked.scatter.v4i16.v4p0(<4 x i16> %res1, <4 x ptr> %gep1, i32 2, <4 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: %indzext = zext <4 x i16> %ind16 to <4 x i32>
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gep2 = getelementptr i16, ptr %base, <4 x i32> %indzext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep2 = getelementptr i16, ptr %base, <4 x i32> %indzext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 56 for instruction: %res2 = call <4 x i16> @llvm.masked.gather.v4i16.v4p0(<4 x ptr> %gep2, i32 2, <4 x i1> %mask, <4 x i16> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 56 for instruction: call void @llvm.masked.scatter.v4i16.v4p0(<4 x i16> %res2, <4 x ptr> %gep2, i32 2, <4 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: %indsext = sext <4 x i16> %ind16 to <4 x i32>
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gep3 = getelementptr i16, ptr %base, <4 x i32> %indsext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep3 = getelementptr i16, ptr %base, <4 x i32> %indsext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 56 for instruction: %res3 = call <4 x i16> @llvm.masked.gather.v4i16.v4p0(<4 x ptr> %gep3, i32 2, <4 x i1> %mask, <4 x i16> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 56 for instruction: call void @llvm.masked.scatter.v4i16.v4p0(<4 x i16> %res3, <4 x ptr> %gep3, i32 2, <4 x i1> %mask)
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gep5 = getelementptr i16, ptr %base, <4 x i16> %ind16
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep5 = getelementptr i16, ptr %base, <4 x i16> %ind16
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: %res5 = call <4 x i16> @llvm.masked.gather.v4i16.v4p0(<4 x ptr> %gep5, i32 2, <4 x i1> %mask, <4 x i16> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: %res5zext = zext <4 x i16> %res5 to <4 x i32>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %res5trunc = trunc <4 x i32> %res5zext to <4 x i16>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: call void @llvm.masked.scatter.v4i16.v4p0(<4 x i16> %res5trunc, <4 x ptr> %gep5, i32 4, <4 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: %res6 = call <4 x i16> @llvm.masked.gather.v4i16.v4p0(<4 x ptr> %gep5, i32 2, <4 x i1> %mask, <4 x i16> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: %res6sext = sext <4 x i16> %res6 to <4 x i32>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %res6trunc = trunc <4 x i32> %res6sext to <4 x i16>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: call void @llvm.masked.scatter.v4i16.v4p0(<4 x i16> %res6trunc, <4 x ptr> %gep5, i32 4, <4 x i1> %mask)
;
   %res6trunc = trunc <4 x i32> %res6sext to <4 x i16>
   call void @llvm.masked.scatter.v4i16.v4p0(<4 x i16> %res6trunc, <4 x ptr> %gep5, i32 4, <4 x i1> %mask)
 
   ret void
 }
 
 define void @gep_v4i8(ptr %base, <4 x i8> %ind8, <4 x i1> %mask)  {
 ; CHECK-LABEL: 'gep_v4i8'
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gep5 = getelementptr i8, ptr %base, <4 x i8> %ind8
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep5 = getelementptr i8, ptr %base, <4 x i8> %ind8
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: %res5 = call <4 x i8> @llvm.masked.gather.v4i8.v4p0(<4 x ptr> %gep5, i32 2, <4 x i1> %mask, <4 x i8> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: %res5zext = zext <4 x i8> %res5 to <4 x i32>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %res5trunc = trunc <4 x i32> %res5zext to <4 x i8>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: call void @llvm.masked.scatter.v4i8.v4p0(<4 x i8> %res5trunc, <4 x ptr> %gep5, i32 4, <4 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: %res6 = call <4 x i8> @llvm.masked.gather.v4i8.v4p0(<4 x ptr> %gep5, i32 2, <4 x i1> %mask, <4 x i8> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: %res6sext = sext <4 x i8> %res6 to <4 x i32>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %res6trunc = trunc <4 x i32> %res6sext to <4 x i8>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 8 for instruction: call void @llvm.masked.scatter.v4i8.v4p0(<4 x i8> %res6trunc, <4 x ptr> %gep5, i32 4, <4 x i1> %mask)
;
   %res6trunc = trunc <4 x i32> %res6sext to <4 x i8>
   call void @llvm.masked.scatter.v4i8.v4p0(<4 x i8> %res6trunc, <4 x ptr> %gep5, i32 4, <4 x i1> %mask)
 
   ret void
 }
 
 define void @gep_v8i16(ptr %base, ptr %base8, ptr %base32, <8 x i32> %ind32, <8 x i16> %ind16, <8 x i8> %ind8, <8 x i1> %mask)  {
 ; CHECK-LABEL: 'gep_v8i16'
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gep1 = getelementptr i16, ptr %base, <8 x i32> %ind32
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep1 = getelementptr i16, ptr %base, <8 x i32> %ind32
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 112 for instruction: %res1 = call <8 x i16> @llvm.masked.gather.v8i16.v8p0(<8 x ptr> %gep1, i32 2, <8 x i1> %mask, <8 x i16> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 112 for instruction: call void @llvm.masked.scatter.v8i16.v8p0(<8 x i16> %res1, <8 x ptr> %gep1, i32 2, <8 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: %indzext = zext <8 x i16> %ind16 to <8 x i32>
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gep2 = getelementptr i16, ptr %base, <8 x i32> %indzext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep2 = getelementptr i16, ptr %base, <8 x i32> %indzext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 16 for instruction: %res2 = call <8 x i16> @llvm.masked.gather.v8i16.v8p0(<8 x ptr> %gep2, i32 2, <8 x i1> %mask, <8 x i16> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 16 for instruction: call void @llvm.masked.scatter.v8i16.v8p0(<8 x i16> %res2, <8 x ptr> %gep2, i32 2, <8 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: %indsext = sext <8 x i16> %ind16 to <8 x i32>
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gep3 = getelementptr i16, ptr %base, <8 x i32> %indsext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep3 = getelementptr i16, ptr %base, <8 x i32> %indsext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 112 for instruction: %res3 = call <8 x i16> @llvm.masked.gather.v8i16.v8p0(<8 x ptr> %gep3, i32 2, <8 x i1> %mask, <8 x i16> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 112 for instruction: call void @llvm.masked.scatter.v8i16.v8p0(<8 x i16> %res3, <8 x ptr> %gep3, i32 2, <8 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 112 for instruction: %resu = call <8 x i16> @llvm.masked.gather.v8i16.v8p0(<8 x ptr> %gep2, i32 1, <8 x i1> %mask, <8 x i16> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 112 for instruction: call void @llvm.masked.scatter.v8i16.v8p0(<8 x i16> %resu, <8 x ptr> %gep2, i32 1, <8 x i1> %mask)
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gepos = getelementptr i8, ptr %base8, <8 x i32> %indzext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gepos = getelementptr i8, ptr %base8, <8 x i32> %indzext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %geposb = bitcast <8 x ptr> %gepos to <8 x ptr>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 16 for instruction: %resos = call <8 x i16> @llvm.masked.gather.v8i16.v8p0(<8 x ptr> %geposb, i32 2, <8 x i1> %mask, <8 x i16> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 16 for instruction: call void @llvm.masked.scatter.v8i16.v8p0(<8 x i16> %resos, <8 x ptr> %geposb, i32 2, <8 x i1> %mask)
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gepbs = getelementptr i32, ptr %base32, <8 x i32> %indzext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gepbs = getelementptr i32, ptr %base32, <8 x i32> %indzext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gepbsb = bitcast <8 x ptr> %gepbs to <8 x ptr>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 112 for instruction: %resbs = call <8 x i16> @llvm.masked.gather.v8i16.v8p0(<8 x ptr> %gepbsb, i32 2, <8 x i1> %mask, <8 x i16> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 112 for instruction: call void @llvm.masked.scatter.v8i16.v8p0(<8 x i16> %resbs, <8 x ptr> %gepbsb, i32 2, <8 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: %indzext4 = zext <8 x i16> %ind16 to <8 x i32>
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gep4 = getelementptr i16, ptr %base, <8 x i32> %indzext4
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep4 = getelementptr i16, ptr %base, <8 x i32> %indzext4
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 16 for instruction: %indtrunc = trunc <8 x i32> %ind32 to <8 x i16>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 16 for instruction: call void @llvm.masked.scatter.v8i16.v8p0(<8 x i16> %indtrunc, <8 x ptr> %gep4, i32 2, <8 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 16 for instruction: %res = call <8 x i16> @llvm.masked.gather.v8i16.v8p0(<8 x ptr> %gep4, i32 2, <8 x i1> %mask, <8 x i16> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: %ressext = sext <8 x i16> %res to <8 x i32>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 16 for instruction: %restrunc = trunc <8 x i32> %ressext to <8 x i16>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 16 for instruction: call void @llvm.masked.scatter.v8i16.v8p0(<8 x i16> %restrunc, <8 x ptr> %gep4, i32 4, <8 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;

 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 16 for instruction: %res2 = call <8 x half> @llvm.masked.gather.v8f16.v8p0(<8 x ptr> %gep2, i32 2, <8 x i1> %mask, <8 x half> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 16 for instruction: call void @llvm.masked.scatter.v8f16.v8p0(<8 x half> %res2, <8 x ptr> %gep2, i32 2, <8 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 10 for instruction: %indsext = sext <8 x i16> %ind16 to <8 x i32>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep3 = getelementptr half, ptr %base, <8 x i32> %indsext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 64 for instruction: %res3 = call <8 x half> @llvm.masked.gather.v8f16.v8p0(<8 x ptr> %gep3, i32 2, <8 x i1> %mask, <8 x half> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 64 for instruction: call void @llvm.masked.scatter.v8f16.v8p0(<8 x half> %res3, <8 x ptr> %gep3, i32 2, <8 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 64 for instruction: %resu = call <8 x half> @llvm.masked.gather.v8f16.v8p0(<8 x ptr> %gep2, i32 1, <8 x i1> %mask, <8 x half> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 64 for instruction: call void @llvm.masked.scatter.v8f16.v8p0(<8 x half> %resu, <8 x ptr> %gep2, i32 1, <8 x i1> %mask)
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gepos = getelementptr i8, ptr %base8, <8 x i32> %indzext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gepos = getelementptr i8, ptr %base8, <8 x i32> %indzext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %geposb = bitcast <8 x ptr> %gepos to <8 x ptr>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 16 for instruction: %resos = call <8 x half> @llvm.masked.gather.v8f16.v8p0(<8 x ptr> %geposb, i32 2, <8 x i1> %mask, <8 x half> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 16 for instruction: call void @llvm.masked.scatter.v8f16.v8p0(<8 x half> %resos, <8 x ptr> %geposb, i32 2, <8 x i1> %mask)
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gepbs = getelementptr i32, ptr %base32, <8 x i32> %indzext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gepbs = getelementptr i32, ptr %base32, <8 x i32> %indzext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gepbsb = bitcast <8 x ptr> %gepbs to <8 x ptr>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 64 for instruction: %resbs = call <8 x half> @llvm.masked.gather.v8f16.v8p0(<8 x ptr> %gepbsb, i32 2, <8 x i1> %mask, <8 x half> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 64 for instruction: call void @llvm.masked.scatter.v8f16.v8p0(<8 x half> %resbs, <8 x ptr> %gepbsb, i32 2, <8 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
   ; no offset ext
   %gep1 = getelementptr half, ptr %base, <8 x i32> %ind32
   %res1 = call <8 x half> @llvm.masked.gather.v8f16.v8p0(<8 x ptr> %gep1, i32 2, <8 x i1> %mask, <8 x half> undef)
;
   call void @llvm.masked.scatter.v8f16.v8p0(<8 x half> %resbs, <8 x ptr> %gepbsb, i32 2, <8 x i1> %mask)
 
   ret void
 }
 
 define void @gep_v8i8(ptr %base, <8 x i8> %ind8, <8 x i1> %mask)  {
 ; CHECK-LABEL: 'gep_v8i8'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 18 for instruction: %indzext = zext <8 x i8> %ind8 to <8 x i32>
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gep5 = getelementptr i8, ptr %base, <8 x i32> %indzext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep5 = getelementptr i8, ptr %base, <8 x i32> %indzext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 16 for instruction: %res5 = call <8 x i8> @llvm.masked.gather.v8i8.v8p0(<8 x ptr> %gep5, i32 2, <8 x i1> %mask, <8 x i8> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: %res5zext = zext <8 x i8> %res5 to <8 x i16>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %res5trunc = trunc <8 x i16> %res5zext to <8 x i8>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 16 for instruction: call void @llvm.masked.scatter.v8i8.v8p0(<8 x i8> %res5trunc, <8 x ptr> %gep5, i32 4, <8 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 16 for instruction: %res6 = call <8 x i8> @llvm.masked.gather.v8i8.v8p0(<8 x ptr> %gep5, i32 2, <8 x i1> %mask, <8 x i8> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: %res6sext = sext <8 x i8> %res6 to <8 x i16>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %res6trunc = trunc <8 x i16> %res6sext to <8 x i8>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 16 for instruction: call void @llvm.masked.scatter.v8i8.v8p0(<8 x i8> %res6trunc, <8 x ptr> %gep5, i32 4, <8 x i1> %mask)
;
   %res6trunc = trunc <8 x i16> %res6sext to <8 x i8>
   call void @llvm.masked.scatter.v8i8.v8p0(<8 x i8> %res6trunc, <8 x ptr> %gep5, i32 4, <8 x i1> %mask)
 
   ret void
 }
 
 define void @gep_v16i8(ptr %base, ptr %base16, <16 x i8> %ind8, <16 x i32> %ind32, <16 x i1> %mask)  {
 ; CHECK-LABEL: 'gep_v16i8'
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gep1 = getelementptr i8, ptr %base, <16 x i32> %ind32
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep1 = getelementptr i8, ptr %base, <16 x i32> %ind32
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 224 for instruction: %res1 = call <16 x i8> @llvm.masked.gather.v16i8.v16p0(<16 x ptr> %gep1, i32 1, <16 x i1> %mask, <16 x i8> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 224 for instruction: call void @llvm.masked.scatter.v16i8.v16p0(<16 x i8> %res1, <16 x ptr> %gep1, i32 2, <16 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 74 for instruction: %indzext = zext <16 x i8> %ind8 to <16 x i32>
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gep2 = getelementptr i8, ptr %base, <16 x i32> %indzext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep2 = getelementptr i8, ptr %base, <16 x i32> %indzext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 32 for instruction: %res2 = call <16 x i8> @llvm.masked.gather.v16i8.v16p0(<16 x ptr> %gep2, i32 2, <16 x i1> %mask, <16 x i8> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 32 for instruction: call void @llvm.masked.scatter.v16i8.v16p0(<16 x i8> %res2, <16 x ptr> %gep2, i32 2, <16 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 74 for instruction: %indsext = sext <16 x i8> %ind8 to <16 x i32>
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gep3 = getelementptr i8, ptr %base, <16 x i32> %indsext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep3 = getelementptr i8, ptr %base, <16 x i32> %indsext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 224 for instruction: %res3 = call <16 x i8> @llvm.masked.gather.v16i8.v16p0(<16 x ptr> %gep3, i32 2, <16 x i1> %mask, <16 x i8> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 224 for instruction: call void @llvm.masked.scatter.v16i8.v16p0(<16 x i8> %res3, <16 x ptr> %gep3, i32 2, <16 x i1> %mask)
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gepbs = getelementptr i16, ptr %base16, <16 x i32> %indzext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gepbs = getelementptr i16, ptr %base16, <16 x i32> %indzext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gepbsb = bitcast <16 x ptr> %gepbs to <16 x ptr>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 224 for instruction: %resbs = call <16 x i8> @llvm.masked.gather.v16i8.v16p0(<16 x ptr> %gepbsb, i32 2, <16 x i1> %mask, <16 x i8> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 224 for instruction: call void @llvm.masked.scatter.v16i8.v16p0(<16 x i8> %resbs, <16 x ptr> %gepbsb, i32 2, <16 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 74 for instruction: %indzext4 = zext <16 x i8> %ind8 to <16 x i32>
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gep4 = getelementptr i8, ptr %base, <16 x i32> %indzext
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gep4 = getelementptr i8, ptr %base, <16 x i32> %indzext
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 32 for instruction: %indtrunc = trunc <16 x i32> %ind32 to <16 x i8>
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 32 for instruction: call void @llvm.masked.scatter.v16i8.v16p0(<16 x i8> %indtrunc, <16 x ptr> %gep4, i32 2, <16 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
   ; no offset ext
   %gep1 = getelementptr i8, ptr %base, <16 x i32> %ind32
   %res1 = call <16 x i8> @llvm.masked.gather.v16i8.v16p0(<16 x ptr> %gep1, i32 1, <16 x i1> %mask, <16 x i8> undef)
   call void @llvm.masked.scatter.v16i8.v16p0(<16 x i8> %res1, <16 x ptr> %gep1, i32 2, <16 x i1> %mask)
;
   %indtrunc = trunc <16 x i32> %ind32 to <16 x i8>
   call void @llvm.masked.scatter.v16i8.v16p0(<16 x i8> %indtrunc, <16 x ptr> %gep4, i32 2, <16 x i1> %mask)
 
   ret void
 }
 
 define void @gep_v16i8p(<16 x ptr> %base, i32 %off, <16 x i1> %mask)  {
 ; CHECK-LABEL: 'gep_v16i8p'
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %gepbs = getelementptr i8, <16 x ptr> %base, i32 %off
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %gepbs = getelementptr i8, <16 x ptr> %base, i32 %off
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 224 for instruction: %resbs = call <16 x i8> @llvm.masked.gather.v16i8.v16p0(<16 x ptr> %gepbs, i32 2, <16 x i1> %mask, <16 x i8> undef)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 224 for instruction: call void @llvm.masked.scatter.v16i8.v16p0(<16 x i8> %resbs, <16 x ptr> %gepbs, i32 2, <16 x i1> %mask)
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
   %gepbs = getelementptr i8, <16 x ptr> %base, i32 %off
   %resbs = call <16 x i8> @llvm.masked.gather.v16i8.v16p0(<16 x ptr> %gepbs, i32 2, <16 x i1> %mask, <16 x i8> undef)
   call void @llvm.masked.scatter.v16i8.v16p0(<16 x i8> %resbs, <16 x ptr> %gepbs, i32 2, <16 x i1> %mask)
   ret void

llvm/test/Analysis/CostModel/RISCV/gep.ll

 ; NOTE: Assertions have been autogenerated by utils/update_analyze_test_checks.py
-; RUN: opt -mtriple=riscv32 -passes="print<cost-model>" 2>&1 -disable-output < %s \
-; RUN:   | FileCheck %s -check-prefix=RVI
-; RUN: opt -mtriple=riscv64 -passes="print<cost-model>" 2>&1 -disable-output < %s \
-; RUN:   | FileCheck %s -check-prefix=RVI
+; RUN: opt -mtriple=riscv32 -mattr=+v -passes="print<cost-model>" 2>&1 \
+; RUN:   -disable-output < %s | FileCheck %s -check-prefix=RVI
+; RUN: opt -mtriple=riscv64 -mattr=+v -passes="print<cost-model>" 2>&1 \
+; RUN:   -disable-output < %s | FileCheck %s -check-prefix=RVI
 
 define void @testi8(ptr %a, i32 %i) {
 ; RVI-LABEL: 'testi8'
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i8, ptr %a, i32 1
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a1, align 1
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a2 = getelementptr inbounds i8, ptr %a, i32 -1
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a2, align 1
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a3 = getelementptr inbounds i8, ptr %a, i32 2047
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a3, align 1
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4 = getelementptr inbounds i8, ptr %a, i32 2048
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a4, align 1
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a5 = getelementptr inbounds i8, ptr %a, i32 -2048
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a5, align 1
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a6 = getelementptr inbounds i8, ptr %a, i32 -2049
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %a6, align 1
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds i8, ptr %a, i32 %i
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %ai, align 1
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
   %a1 = getelementptr inbounds i8, ptr %a, i32 1
+  store volatile i8 undef, ptr %a1
   %a2 = getelementptr inbounds i8, ptr %a, i32 -1
+  store volatile i8 undef, ptr %a2
   %a3 = getelementptr inbounds i8, ptr %a, i32 2047
+  store volatile i8 undef, ptr %a3
   %a4 = getelementptr inbounds i8, ptr %a, i32 2048
+  store volatile i8 undef, ptr %a4
   %a5 = getelementptr inbounds i8, ptr %a, i32 -2048
+  store volatile i8 undef, ptr %a5
   %a6 = getelementptr inbounds i8, ptr %a, i32 -2049
+  store volatile i8 undef, ptr %a6
   %ai = getelementptr inbounds i8, ptr %a, i32 %i
+  store volatile i8 undef, ptr %ai
   ret void
 }
 
 define void @testi16(ptr %a, i32 %i) {
 ; RVI-LABEL: 'testi16'
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i16, ptr %a, i32 1
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a1, align 2
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a2 = getelementptr inbounds i16, ptr %a, i32 -1
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a2, align 2
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a3 = getelementptr inbounds i16, ptr %a, i32 1023
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a3, align 2
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4 = getelementptr inbounds i16, ptr %a, i32 1024
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a4, align 2
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a5 = getelementptr inbounds i16, ptr %a, i32 -1024
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a5, align 2
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a6 = getelementptr inbounds i16, ptr %a, i32 -1025
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %a6, align 2
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds i16, ptr %a, i32 %i
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i16 undef, ptr %ai, align 2
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
   %a1 = getelementptr inbounds i16, ptr %a, i32 1
+  store volatile i16 undef, ptr %a1
   %a2 = getelementptr inbounds i16, ptr %a, i32 -1
+  store volatile i16 undef, ptr %a2
   %a3 = getelementptr inbounds i16, ptr %a, i32 1023
+  store volatile i16 undef, ptr %a3
   %a4 = getelementptr inbounds i16, ptr %a, i32 1024
+  store volatile i16 undef, ptr %a4
   %a5 = getelementptr inbounds i16, ptr %a, i32 -1024
+  store volatile i16 undef, ptr %a5
   %a6 = getelementptr inbounds i16, ptr %a, i32 -1025
+  store volatile i16 undef, ptr %a6
   %ai = getelementptr inbounds i16, ptr %a, i32 %i
+  store volatile i16 undef, ptr %ai
   ret void
 }
 
 define void @testi32(ptr %a, i32 %i) {
 ; RVI-LABEL: 'testi32'
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i32, ptr %a, i32 1
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a1, align 4
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a2 = getelementptr inbounds i32, ptr %a, i32 -1
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a2, align 4
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a3 = getelementptr inbounds i32, ptr %a, i32 511
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a3, align 4
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4 = getelementptr inbounds i32, ptr %a, i32 512
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a4, align 4
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a5 = getelementptr inbounds i32, ptr %a, i32 -512
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a5, align 4
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a6 = getelementptr inbounds i32, ptr %a, i32 -513
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %a6, align 4
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds i32, ptr %a, i32 %i
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i32 undef, ptr %ai, align 4
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
   %a1 = getelementptr inbounds i32, ptr %a, i32 1
+  store volatile i32 undef, ptr %a1
   %a2 = getelementptr inbounds i32, ptr %a, i32 -1
+  store volatile i32 undef, ptr %a2
   %a3 = getelementptr inbounds i32, ptr %a, i32 511
+  store volatile i32 undef, ptr %a3
   %a4 = getelementptr inbounds i32, ptr %a, i32 512
+  store volatile i32 undef, ptr %a4
   %a5 = getelementptr inbounds i32, ptr %a, i32 -512
+  store volatile i32 undef, ptr %a5
   %a6 = getelementptr inbounds i32, ptr %a, i32 -513
+  store volatile i32 undef, ptr %a6
   %ai = getelementptr inbounds i32, ptr %a, i32 %i
+  store volatile i32 undef, ptr %ai
   ret void
 }
 
 define void @testi64(ptr %a, i32 %i) {
-; RVI-LABEL: 'testi64'
-; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds i64, ptr %a, i32 1
-; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a2 = getelementptr inbounds i64, ptr %a, i32 -1
-; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a3 = getelementptr inbounds i64, ptr %a, i32 255
-; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4 = getelementptr inbounds i64, ptr %a, i32 256
-; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a5 = getelementptr inbounds i64, ptr %a, i32 -256
-; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a6 = getelementptr inbounds i64, ptr %a, i32 -257
-; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds i64, ptr %a, i32 %i
-; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
-;
   %a1 = getelementptr inbounds i64, ptr %a, i32 1
+  store volatile i64 undef, ptr %a1
   %a2 = getelementptr inbounds i64, ptr %a, i32 -1
+  store volatile i64 undef, ptr %a2
   %a3 = getelementptr inbounds i64, ptr %a, i32 255
+  store volatile i64 undef, ptr %a3
   %a4 = getelementptr inbounds i64, ptr %a, i32 256
+  store volatile i64 undef, ptr %a4
   %a5 = getelementptr inbounds i64, ptr %a, i32 -256
+  store volatile i64 undef, ptr %a5
   %a6 = getelementptr inbounds i64, ptr %a, i32 -257
+  store volatile i64 undef, ptr %a6
   %ai = getelementptr inbounds i64, ptr %a, i32 %i
+  store volatile i64 undef, ptr %ai
   ret void
 }
 
 define void @testfloat(ptr %a, i32 %i) {
 ; RVI-LABEL: 'testfloat'
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds float, ptr %a, i32 1
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a1, align 4
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a2 = getelementptr inbounds float, ptr %a, i32 -1
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a2, align 4
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a3 = getelementptr inbounds float, ptr %a, i32 511
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a3, align 4
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4 = getelementptr inbounds float, ptr %a, i32 512
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a4, align 4
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a5 = getelementptr inbounds float, ptr %a, i32 -512
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a5, align 4
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a6 = getelementptr inbounds float, ptr %a, i32 -513
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %a6, align 4
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds float, ptr %a, i32 %i
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile float undef, ptr %ai, align 4
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
   %a1 = getelementptr inbounds float, ptr %a, i32 1
+  store volatile float undef, ptr %a1
   %a2 = getelementptr inbounds float, ptr %a, i32 -1
+  store volatile float undef, ptr %a2
   %a3 = getelementptr inbounds float, ptr %a, i32 511
+  store volatile float undef, ptr %a3
   %a4 = getelementptr inbounds float, ptr %a, i32 512
+  store volatile float undef, ptr %a4
   %a5 = getelementptr inbounds float, ptr %a, i32 -512
+  store volatile float undef, ptr %a5
   %a6 = getelementptr inbounds float, ptr %a, i32 -513
+  store volatile float undef, ptr %a6
   %ai = getelementptr inbounds float, ptr %a, i32 %i
+  store volatile float undef, ptr %ai
   ret void
 }
 
 define void @testdouble(ptr %a, i32 %i) {
 ; RVI-LABEL: 'testdouble'
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a1 = getelementptr inbounds double, ptr %a, i32 1
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a1, align 8
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a2 = getelementptr inbounds double, ptr %a, i32 -1
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a2, align 8
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a3 = getelementptr inbounds double, ptr %a, i32 255
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a3, align 8
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a4 = getelementptr inbounds double, ptr %a, i32 256
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a4, align 8
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %a5 = getelementptr inbounds double, ptr %a, i32 -256
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a5, align 8
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %a6 = getelementptr inbounds double, ptr %a, i32 -257
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %a6, align 8
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %ai = getelementptr inbounds double, ptr %a, i32 %i
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile double undef, ptr %ai, align 8
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
   %a1 = getelementptr inbounds double, ptr %a, i32 1
+  store volatile double undef, ptr %a1
   %a2 = getelementptr inbounds double, ptr %a, i32 -1
+  store volatile double undef, ptr %a2
   %a3 = getelementptr inbounds double, ptr %a, i32 255
+  store volatile double undef, ptr %a3
   %a4 = getelementptr inbounds double, ptr %a, i32 256
+  store volatile double undef, ptr %a4
   %a5 = getelementptr inbounds double, ptr %a, i32 -256
+  store volatile double undef, ptr %a5
   %a6 = getelementptr inbounds double, ptr %a, i32 -257
+  store volatile double undef, ptr %a6
   %ai = getelementptr inbounds double, ptr %a, i32 %i
+  store volatile double undef, ptr %ai
   ret void
 }
 
 define void @testvecs(i32 %i) {
 ; RVI-LABEL: 'testvecs'
-; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b0 = getelementptr inbounds <4 x i8>, ptr undef, i32 1
-; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b1 = getelementptr inbounds <4 x i16>, ptr undef, i32 1
-; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b2 = getelementptr inbounds <4 x i32>, ptr undef, i32 1
-; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b3 = getelementptr inbounds <4 x i64>, ptr undef, i32 1
-; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b4 = getelementptr inbounds <4 x float>, ptr undef, i32 1
-; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %b5 = getelementptr inbounds <4 x double>, ptr undef, i32 1
-; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %c1 = getelementptr inbounds <4 x i8>, ptr undef, i32 128
-; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %c2 = getelementptr inbounds <4 x i16>, ptr undef, i32 128
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b0 = getelementptr inbounds <4 x i8>, ptr undef, i32 1
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i8> undef, ptr %b0, align 4
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b1 = getelementptr inbounds <4 x i16>, ptr undef, i32 1
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i16> undef, ptr %b1, align 8
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b2 = getelementptr inbounds <4 x i32>, ptr undef, i32 1
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i32> undef, ptr %b2, align 16
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b3 = getelementptr inbounds <4 x i64>, ptr undef, i32 1
+; RVI-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i64> undef, ptr %b3, align 32
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b4 = getelementptr inbounds <4 x float>, ptr undef, i32 1
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x float> undef, ptr %b4, align 16
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %b5 = getelementptr inbounds <4 x double>, ptr undef, i32 1
+; RVI-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x double> undef, ptr %b5, align 32
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c1 = getelementptr inbounds <4 x i8>, ptr undef, i32 128
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i8> undef, ptr %c1, align 4
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c2 = getelementptr inbounds <4 x i16>, ptr undef, i32 128
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i16> undef, ptr %c2, align 8
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c3 = getelementptr inbounds <4 x i32>, ptr undef, i32 128
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x i32> undef, ptr %c3, align 16
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c4 = getelementptr inbounds <4 x i64>, ptr undef, i32 128
+; RVI-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x i64> undef, ptr %c4, align 32
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c5 = getelementptr inbounds <4 x float>, ptr undef, i32 128
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <4 x float> undef, ptr %c5, align 16
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %c6 = getelementptr inbounds <4 x double>, ptr undef, i32 128
+; RVI-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: store volatile <4 x double> undef, ptr %c6, align 32
 ; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 
   %b0 = getelementptr inbounds <4 x i8>, ptr undef, i32 1
+  store volatile <4 x i8> undef, ptr %b0
   %b1 = getelementptr inbounds <4 x i16>, ptr undef, i32 1
+  store volatile <4 x i16> undef, ptr %b1
   %b2 = getelementptr inbounds <4 x i32>, ptr undef, i32 1
+  store volatile <4 x i32> undef, ptr %b2
   %b3 = getelementptr inbounds <4 x i64>, ptr undef, i32 1
+  store volatile <4 x i64> undef, ptr %b3
   %b4 = getelementptr inbounds <4 x float>, ptr undef, i32 1
+  store volatile <4 x float> undef, ptr %b4
   %b5 = getelementptr inbounds <4 x double>, ptr undef, i32 1
+  store volatile <4 x double> undef, ptr %b5
 
   %c1 = getelementptr inbounds <4 x i8>, ptr undef, i32 128
+  store volatile <4 x i8> undef, ptr %c1
   %c2 = getelementptr inbounds <4 x i16>, ptr undef, i32 128
+  store volatile <4 x i16> undef, ptr %c2
   %c3 = getelementptr inbounds <4 x i32>, ptr undef, i32 128
+  store volatile <4 x i32> undef, ptr %c3
   %c4 = getelementptr inbounds <4 x i64>, ptr undef, i32 128
+  store volatile <4 x i64> undef, ptr %c4
   %c5 = getelementptr inbounds <4 x float>, ptr undef, i32 128
+  store volatile <4 x float> undef, ptr %c5
   %c6 = getelementptr inbounds <4 x double>, ptr undef, i32 128
+  store volatile <4 x double> undef, ptr %c6
 
   ret void
 }
+
+; Ensure that memory operations of a different type than the pointer source type
+; use the correct type to determine if folding is possible. These operations
+; are on vector types so there should be a cost for the GEP as the offset cannot
+; be folded into the instruction.
+define void @non_foldable_vector_uses(ptr %base, <2 x ptr> %base.vec) {
+; RVI-LABEL: 'non_foldable_vector_uses'
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %1 = getelementptr i8, ptr %base, i32 42
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x1 = load volatile <2 x i8>, ptr %1, align 2
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %2 = getelementptr i8, ptr %base, i32 42
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x2 = call <2 x i8> @llvm.masked.load.v2i8.p0(ptr %2, i32 1, <2 x i1> undef, <2 x i8> undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %3 = getelementptr i8, <2 x ptr> %base.vec, <2 x i32> <i32 42, i32 43>
+; RVI-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: %x3 = call <2 x i8> @llvm.masked.gather.v2i8.v2p0(<2 x ptr> %3, i32 1, <2 x i1> undef, <2 x i8> undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %4 = getelementptr i8, ptr %base, i32 42
+; RVI-NEXT:  Cost Model: Found an estimated cost of 5 for instruction: %x4 = call <2 x i8> @llvm.masked.expandload.v2i8(ptr %4, <2 x i1> undef, <2 x i8> undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %5 = getelementptr i8, ptr %base, i32 42
+; RVI-NEXT:  Cost Model: Found an estimated cost of 5 for instruction: %x5 = call <2 x i8> @llvm.vp.load.v2i8.p0(ptr %5, <2 x i1> undef, i32 undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %6 = getelementptr i8, ptr %base, i32 42
+; RVI-NEXT:  Cost Model: Found an estimated cost of 5 for instruction: %x6 = call <2 x i8> @llvm.experimental.vp.strided.load.v2i8.p0.i64(ptr %6, i64 undef, <2 x i1> undef, i32 undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %7 = getelementptr i8, ptr %base, i32 42
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <2 x i8> undef, ptr %7, align 2
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %8 = getelementptr i8, ptr %base, i32 42
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: call void @llvm.masked.store.v2i8.p0(<2 x i8> undef, ptr %8, i32 1, <2 x i1> undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %9 = getelementptr i8, <2 x ptr> %base.vec, <2 x i32> <i32 42, i32 43>
+; RVI-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: call void @llvm.masked.scatter.v2i8.v2p0(<2 x i8> undef, <2 x ptr> %9, i32 1, <2 x i1> undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %10 = getelementptr i8, ptr %base, i32 42
+; RVI-NEXT:  Cost Model: Found an estimated cost of 12 for instruction: call void @llvm.masked.compressstore.v2i8(<2 x i8> undef, ptr %10, <2 x i1> undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %11 = getelementptr i8, ptr %base, i32 42
+; RVI-NEXT:  Cost Model: Found an estimated cost of 12 for instruction: call void @llvm.vp.store.v2i8.p0(<2 x i8> undef, ptr %11, <2 x i1> undef, i32 undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %12 = getelementptr i8, ptr %base, i32 42
+; RVI-NEXT:  Cost Model: Found an estimated cost of 12 for instruction: call void @llvm.experimental.vp.strided.store.v2i8.p0.i64(<2 x i8> undef, ptr %12, i64 undef, <2 x i1> undef, i32 undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
+;
+  %1 = getelementptr i8, ptr %base, i32 42
+  %x1 = load volatile <2 x i8>, ptr %1
+
+  %2 = getelementptr i8, ptr %base, i32 42
+  %x2 = call <2 x i8> @llvm.masked.load.v2i8.p0(ptr %2, i32 1, <2 x i1> undef, <2 x i8> undef)
+
+  %3 = getelementptr i8, <2 x ptr> %base.vec, <2 x i32> <i32 42, i32 43>
+  %x3 = call <2 x i8> @llvm.masked.gather.v2i8.v2p0(<2 x ptr> %3, i32 1, <2 x i1> undef, <2 x i8> undef)
+
+  %4 = getelementptr i8, ptr %base, i32 42
+  %x4 = call <2 x i8> @llvm.masked.expandload.v2i8(ptr %4, <2 x i1> undef, <2 x i8> undef)
+
+  %5 = getelementptr i8, ptr %base, i32 42
+  %x5 = call <2 x i8> @llvm.vp.load.v2i8.p0(ptr %5, <2 x i1> undef, i32 undef)
+
+  %6 = getelementptr i8, ptr %base, i32 42
+  %x6 = call <2 x i8> @llvm.experimental.vp.strided.load.v2i8.i64(ptr %6, i64 undef, <2 x i1> undef, i32 undef)
+
+  %7 = getelementptr i8, ptr %base, i32 42
+  store volatile <2 x i8> undef, ptr %7
+
+  %8 = getelementptr i8, ptr %base, i32 42
+  call void @llvm.masked.store.v2i8.p0(<2 x i8> undef, ptr %8, i32 1, <2 x i1> undef)
+
+  %9 = getelementptr i8, <2 x ptr> %base.vec, <2 x i32> <i32 42, i32 43>
+  call void @llvm.masked.scatter.v2i8.v2p0(<2 x i8> undef, <2 x ptr> %9, i32 1, <2 x i1> undef)
+
+  %10 = getelementptr i8, ptr %base, i32 42
+  call void @llvm.masked.compressstore.v2i8(<2 x i8> undef, ptr %10, <2 x i1> undef)
+
+  %11 = getelementptr i8, ptr %base, i32 42
+  call void @llvm.vp.store.v2i8.p0(<2 x i8> undef, ptr %11, <2 x i1> undef, i32 undef)
+
+  %12 = getelementptr i8, ptr %base, i32 42
+  call void @llvm.experimental.vp.strided.store.v2i8.i64(<2 x i8> undef, ptr %12, i64 undef, <2 x i1> undef, i32 undef)
+
+  ret void
+}
+
+; Because there is no offset the GEP should be compltely foldable into the user.
+define void @foldable_vector_uses(ptr %base, <2 x ptr> %base.vec) {
+; RVI-LABEL: 'foldable_vector_uses'
+; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %1 = getelementptr i8, ptr %base, i32 0
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x1 = load volatile <2 x i8>, ptr %1, align 2
+; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %2 = getelementptr i8, ptr %base, i32 0
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x2 = call <2 x i8> @llvm.masked.load.v2i8.p0(ptr %2, i32 1, <2 x i1> undef, <2 x i8> undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %3 = getelementptr i8, <2 x ptr> %base.vec, <2 x i32> zeroinitializer
+; RVI-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: %x3 = call <2 x i8> @llvm.masked.gather.v2i8.v2p0(<2 x ptr> %3, i32 1, <2 x i1> undef, <2 x i8> undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %4 = getelementptr i8, ptr %base, i32 0
+; RVI-NEXT:  Cost Model: Found an estimated cost of 5 for instruction: %x4 = call <2 x i8> @llvm.masked.expandload.v2i8(ptr %4, <2 x i1> undef, <2 x i8> undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %5 = getelementptr i8, ptr %base, i32 0
+; RVI-NEXT:  Cost Model: Found an estimated cost of 5 for instruction: %x5 = call <2 x i8> @llvm.vp.load.v2i8.p0(ptr %5, <2 x i1> undef, i32 undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %6 = getelementptr i8, ptr %base, i32 0
+; RVI-NEXT:  Cost Model: Found an estimated cost of 5 for instruction: %x6 = call <2 x i8> @llvm.experimental.vp.strided.load.v2i8.p0.i64(ptr %6, i64 undef, <2 x i1> undef, i32 undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %7 = getelementptr i8, ptr %base, i32 0
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile <2 x i8> undef, ptr %7, align 2
+; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %8 = getelementptr i8, ptr %base, i32 0
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: call void @llvm.masked.store.v2i8.p0(<2 x i8> undef, ptr %8, i32 1, <2 x i1> undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %9 = getelementptr i8, <2 x ptr> %base.vec, <2 x i32> zeroinitializer
+; RVI-NEXT:  Cost Model: Found an estimated cost of 2 for instruction: call void @llvm.masked.scatter.v2i8.v2p0(<2 x i8> undef, <2 x ptr> %9, i32 1, <2 x i1> undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %10 = getelementptr i8, ptr %base, i32 0
+; RVI-NEXT:  Cost Model: Found an estimated cost of 12 for instruction: call void @llvm.masked.compressstore.v2i8(<2 x i8> undef, ptr %10, <2 x i1> undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %11 = getelementptr i8, ptr %base, i32 0
+; RVI-NEXT:  Cost Model: Found an estimated cost of 12 for instruction: call void @llvm.vp.store.v2i8.p0(<2 x i8> undef, ptr %11, <2 x i1> undef, i32 undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %12 = getelementptr i8, ptr %base, i32 0
+; RVI-NEXT:  Cost Model: Found an estimated cost of 12 for instruction: call void @llvm.experimental.vp.strided.store.v2i8.p0.i64(<2 x i8> undef, ptr %12, i64 undef, <2 x i1> undef, i32 undef)
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
+;
+  %1 = getelementptr i8, ptr %base, i32 0
+  %x1 = load volatile <2 x i8>, ptr %1
+
+  %2 = getelementptr i8, ptr %base, i32 0
+  %x2 = call <2 x i8> @llvm.masked.load.v2i8.p0(ptr %2, i32 1, <2 x i1> undef, <2 x i8> undef)
+
+  %3 = getelementptr i8, <2 x ptr> %base.vec, <2 x i32> <i32 0, i32 0>
+  %x3 = call <2 x i8> @llvm.masked.gather.v2i8.v2p0(<2 x ptr> %3, i32 1, <2 x i1> undef, <2 x i8> undef)
+
+  %4 = getelementptr i8, ptr %base, i32 0
+  %x4 = call <2 x i8> @llvm.masked.expandload.v2i8(ptr %4, <2 x i1> undef, <2 x i8> undef)
+
+  %5 = getelementptr i8, ptr %base, i32 0
+  %x5 = call <2 x i8> @llvm.vp.load.v2i8.p0(ptr %5, <2 x i1> undef, i32 undef)
+
+  %6 = getelementptr i8, ptr %base, i32 0
+  %x6 = call <2 x i8> @llvm.experimental.vp.strided.load.v2i8.i64(ptr %6, i64 undef, <2 x i1> undef, i32 undef)
+
+  %7 = getelementptr i8, ptr %base, i32 0
+  store volatile <2 x i8> undef, ptr %7
+
+  %8 = getelementptr i8, ptr %base, i32 0
+  call void @llvm.masked.store.v2i8.p0(<2 x i8> undef, ptr %8, i32 1, <2 x i1> undef)
+
+  %9 = getelementptr i8, <2 x ptr> %base.vec, <2 x i32> <i32 0, i32 0>
+  call void @llvm.masked.scatter.v2i8.v2p0(<2 x i8> undef, <2 x ptr> %9, i32 1, <2 x i1> undef)
+
+  %10 = getelementptr i8, ptr %base, i32 0
+  call void @llvm.masked.compressstore.v2i8(<2 x i8> undef, ptr %10, <2 x i1> undef)
+
+  %11 = getelementptr i8, ptr %base, i32 0
+  call void @llvm.vp.store.v2i8.p0(<2 x i8> undef, ptr %11, <2 x i1> undef, i32 undef)
+
+  %12 = getelementptr i8, ptr %base, i32 0
+  call void @llvm.experimental.vp.strided.store.v2i8.i64(<2 x i8> undef, ptr %12, i64 undef, <2 x i1> undef, i32 undef)
+
+  ret void
+}
+
+declare <2 x i8> @llvm.masked.load.v2i8.p0(ptr, i32, <2 x i1>, <2 x i8>)
+declare <2 x i8> @llvm.masked.gather.v2i8.v2p0(<2 x ptr>, i32, <2 x i1>, <2 x i8>)
+declare <2 x i8> @llvm.masked.expandload.v2i8(ptr, <2 x i1>, <2 x i8>)
+declare <2 x i8> @llvm.vp.load.v2i8.p0(ptr, <2 x i1>, i32)
+declare <2 x i8> @llvm.experimental.vp.strided.load.v2i8.i64(ptr, i64, <2 x i1>, i32)
+
+declare void @llvm.masked.store.v2i8.p0(<2 x i8>, ptr, i32, <2 x i1>)
+declare void @llvm.masked.scatter.v2i8.v2p0(<2 x i8>, <2 x ptr>, i32, <2 x i1>)
+declare void @llvm.masked.compressstore.v2i8(<2 x i8>, ptr, <2 x i1>)
+declare void @llvm.vp.store.v2i8.p0(<2 x i8>, ptr, <2 x i1>, i32)
+declare void @llvm.experimental.vp.strided.store.v2i8.i64(<2 x i8>, ptr, i64, <2 x i1>, i32)
+
+; Ensure that GEPs aren't folded if any of their uses aren't legal addressing
+; modes
+define <2 x i8> @fold_two_different_uses(ptr %base) {
+; RVI-LABEL: 'fold_two_different_uses'
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %1 = getelementptr i8, ptr %base, i32 42
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 2, ptr %1, align 1
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = load volatile <2 x i8>, ptr %1, align 2
+; RVI-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <2 x i8> %x
+;
+  %1 = getelementptr i8, ptr %base, i32 42
+  store volatile i8 2, ptr %1
+  %x = load volatile <2 x i8>, ptr %1
+  ret <2 x i8> %x
+}

llvm/test/Analysis/CostModel/X86/gep.ll

 ; NOTE: Assertions have been autogenerated by utils/update_analyze_test_checks.py
 ; RUN: opt < %s -mtriple=x86_64-apple-macosx10.8.0 -passes="print<cost-model>" 2>&1 -disable-output -cost-kind=throughput   -mcpu=corei7-avx | FileCheck %s --check-prefix=THRU
 ; RUN: opt < %s -mtriple=x86_64-apple-macosx10.8.0 -passes="print<cost-model>" 2>&1 -disable-output -cost-kind=latency      -mcpu=corei7-avx | FileCheck %s --check-prefix=LATE
 ; RUN: opt < %s -mtriple=x86_64-apple-macosx10.8.0 -passes="print<cost-model>" 2>&1 -disable-output -cost-kind=code-size    -mcpu=corei7-avx | FileCheck %s --check-prefix=SIZE
 ; RUN: opt < %s -mtriple=x86_64-apple-macosx10.8.0 -passes="print<cost-model>" 2>&1 -disable-output -cost-kind=size-latency -mcpu=corei7-avx | FileCheck %s --check-prefix=SIZE_LATE
 
 define void @test_geps() {
 ; THRU-LABEL: 'test_geps'
 ; THRU-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %giant_gep0 = getelementptr inbounds i8, ptr undef, i64 9223372036854775807
+; THRU-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %giant_gep0, align 1
 ; THRU-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %giant_gep1 = getelementptr inbounds i8, ptr undef, i128 295147905179352825855
+; THRU-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %giant_gep1, align 1
 ; THRU-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: ret void
 ;
 ; LATE-LABEL: 'test_geps'
 ; LATE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %giant_gep0 = getelementptr inbounds i8, ptr undef, i64 9223372036854775807
+; LATE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %giant_gep0, align 1
 ; LATE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %giant_gep1 = getelementptr inbounds i8, ptr undef, i128 295147905179352825855
+; LATE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %giant_gep1, align 1
 ; LATE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; SIZE-LABEL: 'test_geps'
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %giant_gep0 = getelementptr inbounds i8, ptr undef, i64 9223372036854775807
+; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %giant_gep0, align 1
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %giant_gep1 = getelementptr inbounds i8, ptr undef, i128 295147905179352825855
+; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %giant_gep1, align 1
 ; SIZE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 ; SIZE_LATE-LABEL: 'test_geps'
 ; SIZE_LATE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %giant_gep0 = getelementptr inbounds i8, ptr undef, i64 9223372036854775807
+; SIZE_LATE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %giant_gep0, align 1
 ; SIZE_LATE-NEXT:  Cost Model: Found an estimated cost of 0 for instruction: %giant_gep1 = getelementptr inbounds i8, ptr undef, i128 295147905179352825855
+; SIZE_LATE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: store volatile i8 undef, ptr %giant_gep1, align 1
 ; SIZE_LATE-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret void
 ;
 
   ; Cost of scalar geps should be zero.
   ; We expect it to be folded into the instruction addressing mode.
 
 
   ; Vector geps should also have zero cost.
 
   ; Check that we handle outlandishly large GEPs properly.  This is unlikely to
   ; be a valid pointer, but LLVM still generates GEPs like this sometimes in
   ; dead code.
   ; This GEP has index INT64_MAX, which is cost 1.
   %giant_gep0 = getelementptr inbounds i8, ptr undef, i64 9223372036854775807
+  store volatile i8 undef, ptr %giant_gep0
 
   ; This GEP index wraps around to -1, which is cost 0.
   %giant_gep1 = getelementptr inbounds i8, ptr undef, i128 295147905179352825855
+  store volatile i8 undef, ptr %giant_gep1
 
   ret void
 }

llvm/test/Transforms/Inline/AArch64/gep-cost.ll

define void @outer3(ptr %ptr, i32 %j) {
   call void @inner3(ptr %ptr, i32 0, i32 %j)
   ret void
 }
 
 ; The gep in inner1() is reg+reg, which is a legal addressing mode for AArch64.
 ; Thus, both the gep and ret can be simplified.
 ; CHECK: Analyzing call of inner1
 ; CHECK: NumInstructionsSimplified: 2
-; CHECK: NumInstructions: 2
+; CHECK: NumInstructions: 3
 define void @inner1(ptr %ptr, i32 %i) {
   %G = getelementptr inbounds [4 x i32], ptr %ptr, i32 0, i32 %i
+  store i32 undef, ptr %G
   ret void
 }
 
 ; The gep in inner2() is reg+imm+reg, which is not a legal addressing mode for
 ; AArch64.  Thus, only the ret can be simplified and not the gep.
 ; CHECK: Analyzing call of inner2
 ; CHECK: NumInstructionsSimplified: 1
-; CHECK: NumInstructions: 2
+; CHECK: NumInstructions: 3
 define void @inner2(ptr %ptr, i32 %i) {
   %G = getelementptr inbounds [4 x i32], ptr %ptr, i32 1, i32 %i
+  store i32 undef, ptr %G
   ret void
 }
 
 ; The gep in inner3() is reg+reg because %i is a known constant from the
 ; callsite. This case is a legal addressing mode for AArch64.  Thus, both the
 ; gep and ret can be simplified.
 ; CHECK: Analyzing call of inner3
 ; CHECK: NumInstructionsSimplified: 2
-; CHECK: NumInstructions: 2
+; CHECK: NumInstructions: 3
 define void @inner3(ptr %ptr, i32 %i, i32 %j) {
   %G = getelementptr inbounds [4 x i32], ptr %ptr, i32 %i, i32 %j
+  store i32 undef, ptr %G
   ret void
 }

llvm/test/Transforms/LICM/sink-foldable.ll

 ; CHECK-NEXT:    [[L1:%.*]] = load ptr, ptr [[ARRAYIDX1]], align 8
 ; CHECK-NEXT:    [[CMP4:%.*]] = icmp ugt ptr [[L1]], [[Q]]
 ; CHECK-NEXT:    [[I_ADD]] = add nsw i32 [[I0]], 2
 ; CHECK-NEXT:    br i1 [[CMP4]], label [[LOOPEXIT1:%.*]], label [[FOR_BODY]]
 ; CHECK:       loopexit0:
 ; CHECK-NEXT:    [[P1:%.*]] = phi ptr [ [[ARRAYIDX0]], [[FOR_BODY]] ]
 ; CHECK-NEXT:    br label [[RETURN]]
 ; CHECK:       loopexit1:
-; CHECK-NEXT:    [[ARRAYIDX0_LCSSA:%.*]] = phi ptr [ [[ARRAYIDX0]], [[IF_END]] ]
-; CHECK-NEXT:    [[ARRAYIDX1_LE:%.*]] = getelementptr inbounds ptr, ptr [[ARRAYIDX0_LCSSA]], i64 1
+; CHECK-NEXT:    [[P2:%.*]] = phi ptr [ [[ARRAYIDX1]], [[IF_END]] ]
 ; CHECK-NEXT:    br label [[RETURN]]
 ; CHECK:       return:
-; CHECK-NEXT:    [[RETVAL_0:%.*]] = phi ptr [ [[P1]], [[LOOPEXIT0]] ], [ [[ARRAYIDX1_LE]], [[LOOPEXIT1]] ], [ null, [[ENTRY:%.*]] ]
+; CHECK-NEXT:    [[RETVAL_0:%.*]] = phi ptr [ [[P1]], [[LOOPEXIT0]] ], [ [[P2]], [[LOOPEXIT1]] ], [ null, [[ENTRY:%.*]] ]
 ; CHECK-NEXT:    ret ptr [[RETVAL_0]]
 ;
 entry:
   %cmp0 = icmp slt i32 0, %j
   br i1 %cmp0, label %for.body.lr.ph, label %return
 
 for.body.lr.ph:
   br label %for.body

 ; CHECK-NEXT:    br i1 [[CMP2]], label [[LOOPEXIT2:%.*]], label [[FOR_COND]]
 ; CHECK:       loopexit0:
 ; CHECK-NEXT:    [[P0:%.*]] = phi ptr [ null, [[FOR_COND]] ]
 ; CHECK-NEXT:    br label [[RETURN:%.*]]
 ; CHECK:       loopexit1:
 ; CHECK-NEXT:    [[P1:%.*]] = phi ptr [ [[ADD_PTR]], [[FOR_BODY]] ]
 ; CHECK-NEXT:    br label [[RETURN]]
 ; CHECK:       loopexit2:
-; CHECK-NEXT:    [[IDX2_EXT_LCSSA:%.*]] = phi i64 [ [[IDX2_EXT]], [[IF_END]] ]
-; CHECK-NEXT:    [[ADD_PTR_LCSSA:%.*]] = phi ptr [ [[ADD_PTR]], [[IF_END]] ]
-; CHECK-NEXT:    [[ARRAYIDX2_LE:%.*]] = getelementptr inbounds ptr, ptr [[ADD_PTR_LCSSA]], i64 [[IDX2_EXT_LCSSA]]
+; CHECK-NEXT:    [[P2:%.*]] = phi ptr [ [[ARRAYIDX2]], [[IF_END]] ]
 ; CHECK-NEXT:    br label [[RETURN]]
 ; CHECK:       return:
-; CHECK-NEXT:    [[RETVAL_0:%.*]] = phi ptr [ [[P1]], [[LOOPEXIT1]] ], [ [[ARRAYIDX2_LE]], [[LOOPEXIT2]] ], [ [[P0]], [[LOOPEXIT0]] ]
+; CHECK-NEXT:    [[RETVAL_0:%.*]] = phi ptr [ [[P1]], [[LOOPEXIT1]] ], [ [[P2]], [[LOOPEXIT2]] ], [ [[P0]], [[LOOPEXIT0]] ]
 ; CHECK-NEXT:    ret ptr [[RETVAL_0]]
 ;
 entry:
   br label %for.body
 
 for.cond:
   %i.addr.0 = phi i32 [ %add, %if.end ]
   %P.addr.0 = phi ptr [ %add.ptr, %if.end ]

 ; CHECK-NEXT:    [[L1:%.*]] = load ptr, ptr [[ARRAYIDX1]], align 8
 ; CHECK-NEXT:    [[CMP4:%.*]] = icmp ugt ptr [[L1]], [[Q]]
 ; CHECK-NEXT:    [[I_ADD]] = add nsw i32 [[I0]], 2
 ; CHECK-NEXT:    br i1 [[CMP4]], label [[LOOPEXIT1:%.*]], label [[FOR_BODY]]
 ; CHECK:       loopexit0:
 ; CHECK-NEXT:    [[P1:%.*]] = phi ptr [ [[ARRAYIDX0]], [[FOR_BODY]] ]
 ; CHECK-NEXT:    br label [[RETURN]]
 ; CHECK:       loopexit1:
-; CHECK-NEXT:    [[TRUNC_LCSSA1:%.*]] = phi i32 [ [[TRUNC]], [[IF_END]] ]
-; CHECK-NEXT:    [[P_ADDR_LCSSA:%.*]] = phi ptr [ [[P_ADDR]], [[IF_END]] ]
 ; CHECK-NEXT:    [[TRUNC_LCSSA:%.*]] = phi i32 [ [[TRUNC]], [[IF_END]] ]
-; CHECK-NEXT:    [[ARRAYIDX1_LE:%.*]] = getelementptr inbounds ptr, ptr [[P_ADDR_LCSSA]], i32 [[TRUNC_LCSSA1]]
+; CHECK-NEXT:    [[P2:%.*]] = phi ptr [ [[ARRAYIDX1]], [[IF_END]] ]
 ; CHECK-NEXT:    call void @dummy(i32 [[TRUNC_LCSSA]])
 ; CHECK-NEXT:    br label [[RETURN]]
 ; CHECK:       return:
-; CHECK-NEXT:    [[RETVAL_0:%.*]] = phi ptr [ [[P1]], [[LOOPEXIT0]] ], [ [[ARRAYIDX1_LE]], [[LOOPEXIT1]] ], [ null, [[ENTRY:%.*]] ]
+; CHECK-NEXT:    [[RETVAL_0:%.*]] = phi ptr [ [[P1]], [[LOOPEXIT0]] ], [ [[P2]], [[LOOPEXIT1]] ], [ null, [[ENTRY:%.*]] ]
 ; CHECK-NEXT:    ret ptr [[RETVAL_0]]
 ;
 entry:
   %cmp0 = icmp slt i64 0, %j
   br i1 %cmp0, label %for.body.lr.ph, label %return
 
 for.body.lr.ph:
   br label %for.body

llvm/test/Transforms/LoopVectorize/RISCV/strided-accesses.ll

 ; NOSTRIDED-NEXT:    br i1 [[DONE]], label [[EXIT:%.*]], label [[LOOP]]
 ; NOSTRIDED:       exit:
 ; NOSTRIDED-NEXT:    ret void
 ;
 ; STRIDED-LABEL: @double_stride_ptr_iv(
 ; STRIDED-NEXT:  entry:
 ; STRIDED-NEXT:    [[TMP0:%.*]] = call i64 @llvm.vscale.i64()
 ; STRIDED-NEXT:    [[TMP1:%.*]] = mul i64 [[TMP0]], 4
-; STRIDED-NEXT:    [[TMP2:%.*]] = call i64 @llvm.umax.i64(i64 16, i64 [[TMP1]])
+; STRIDED-NEXT:    [[TMP2:%.*]] = call i64 @llvm.umax.i64(i64 24, i64 [[TMP1]])
 ; STRIDED-NEXT:    [[MIN_ITERS_CHECK:%.*]] = icmp ult i64 1024, [[TMP2]]
 ; STRIDED-NEXT:    br i1 [[MIN_ITERS_CHECK]], label [[SCALAR_PH:%.*]], label [[VECTOR_MEMCHECK:%.*]]
 ; STRIDED:       vector.memcheck:
 ; STRIDED-NEXT:    [[TMP3:%.*]] = mul i64 [[STRIDE:%.*]], 1023
 ; STRIDED-NEXT:    [[SCEVGEP:%.*]] = getelementptr i8, ptr [[P2:%.*]], i64 [[TMP3]]
 ; STRIDED-NEXT:    [[TMP4:%.*]] = icmp ult ptr [[P2]], [[SCEVGEP]]
 ; STRIDED-NEXT:    [[UMIN:%.*]] = select i1 [[TMP4]], ptr [[P2]], ptr [[SCEVGEP]]
 ; STRIDED-NEXT:    [[TMP5:%.*]] = icmp ugt ptr [[P2]], [[SCEVGEP]]

llvm/test/Transforms/LoopVectorize/X86/pointer-runtime-checks-unprofitable.ll

 ; RUN: opt -opaque-pointers=0 -passes="loop-vectorize" -mtriple=x86_64-unknown-linux -S -debug %s 2>&1 | FileCheck %s
 ; REQUIRES: asserts
 
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
 
 target triple = "x86_64-unknown-linux"
 
 declare double @llvm.pow.f64(double, double)
 
 ; Test case where the memory runtime checks and vector body is more expensive
 ; than running the scalar loop.
 define void @test(double* nocapture %A, double* nocapture %B, double* nocapture %C, double* nocapture %D, double* nocapture %E) {
 
 ; CHECK: Calculating cost of runtime checks:
-; CHECK-NEXT:  0  for   {{.+}} = getelementptr double, double* %A, i64 16
+; CHECK-NEXT:  1  for   {{.+}} = getelementptr double, double* %A, i64 16
 ; CHECK-NEXT:  0  for   {{.+}} = bitcast double*
-; CHECK-NEXT:  0  for   {{.+}} = getelementptr double, double* %B, i64 16
+; CHECK-NEXT:  1  for   {{.+}} = getelementptr double, double* %B, i64 16
 ; CHECK-NEXT:  0  for   {{.+}} = bitcast double*
-; CHECK-NEXT:  0  for   {{.+}} = getelementptr double, double* %E, i64 16
+; CHECK-NEXT:  1  for   {{.+}} = getelementptr double, double* %E, i64 16
 ; CHECK-NEXT:  0  for   {{.+}} = bitcast double*
-; CHECK-NEXT:  0  for   {{.+}} = getelementptr double, double* %C, i64 16
+; CHECK-NEXT:  1  for   {{.+}} = getelementptr double, double* %C, i64 16
 ; CHECK-NEXT:  0  for   {{.+}} = bitcast double*
-; CHECK-NEXT:  0  for   {{.+}} = getelementptr double, double* %D, i64 16
+; CHECK-NEXT:  1  for   {{.+}} = getelementptr double, double* %D, i64 16
 ; CHECK-NEXT:  0  for   {{.+}} = bitcast double*
 ; CHECK-NEXT:  1  for   {{.+}} = icmp ult i8*
 ; CHECK-NEXT:  1  for   {{.+}} = icmp ult i8*
 ; CHECK-NEXT:  1  for   {{.+}} = and i1
 ; CHECK-NEXT:  1  for   {{.+}} = icmp ult i8*
 ; CHECK-NEXT:  1  for   {{.+}} = icmp ult i8*
 ; CHECK-NEXT:  1  for   {{.+}} = and i1
 ; CHECK-NEXT:  1  for   {{.+}} = or i1

 ; CHECK-NEXT:  1  for   {{.+}} = icmp ult i8*
 ; CHECK-NEXT:  1  for   {{.+}} = icmp ult i8*
 ; CHECK-NEXT:  1  for   {{.+}} = and i1
 ; CHECK-NEXT:  1  for   {{.+}} = or i1
 ; CHECK-NEXT:  1  for   {{.+}} = icmp ult i8*
 ; CHECK-NEXT:  1  for   {{.+}} = icmp ult i8*
 ; CHECK-NEXT:  1  for   {{.+}} = and i1
 ; CHECK-NEXT:  1  for   {{.+}} = or i1
-; CHECK-NEXT: Total cost of runtime checks: 35
+; CHECK-NEXT: Total cost of runtime checks: 40
 
-; CHECK: LV: Vectorization is not beneficial: expected trip count < minimum profitable VF (16 < 24)
+; CHECK: LV: Vectorization is not beneficial: expected trip count < minimum profitable VF (16 < 28)
 ;
 ; CHECK-LABEL: @test(
 ; CHECK-NEXT: entry:
 ; CHECK-NEXT:  br label %for.body
 ; CHECK-NOT: vector.memcheck
 ; CHECK-NOT: vector.body
 ;
 entry:

llvm/test/Transforms/LoopVectorize/X86/pr23997.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -S -passes=loop-vectorize,dce,instcombine < %s | FileCheck %s
 
 target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128-ni:1"
 target triple = "x86_64-unknown-linux-gnu"
 
 ; Ensure that the 'inbounds' is preserved on the GEPs that feed the load and store in the loop.
 define void @foo(ptr addrspace(1) align 8 dereferenceable_or_null(16), ptr addrspace(1) align 8 dereferenceable_or_null(8), i64) #0 {
 ; CHECK-LABEL: @foo(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[PREHEADER:%.*]]
 ; CHECK:       preheader:
 ; CHECK-NEXT:    [[DOT10:%.*]] = getelementptr inbounds i8, ptr addrspace(1) [[TMP0:%.*]], i64 16
 ; CHECK-NEXT:    [[DOT12:%.*]] = getelementptr inbounds i8, ptr addrspace(1) [[TMP1:%.*]], i64 16
 ; CHECK-NEXT:    [[UMAX2:%.*]] = call i64 @llvm.umax.i64(i64 [[TMP2:%.*]], i64 1)
-; CHECK-NEXT:    [[MIN_ITERS_CHECK:%.*]] = icmp ult i64 [[UMAX2]], 16
+; CHECK-NEXT:    [[MIN_ITERS_CHECK:%.*]] = icmp ult i64 [[UMAX2]], 20
 ; CHECK-NEXT:    br i1 [[MIN_ITERS_CHECK]], label [[SCALAR_PH:%.*]], label [[VECTOR_MEMCHECK:%.*]]
 ; CHECK:       vector.memcheck:
 ; CHECK-NEXT:    [[UMAX:%.*]] = call i64 @llvm.umax.i64(i64 [[TMP2]], i64 1)
 ; CHECK-NEXT:    [[TMP3:%.*]] = shl i64 [[UMAX]], 3
 ; CHECK-NEXT:    [[TMP4:%.*]] = add i64 [[TMP3]], 16
 ; CHECK-NEXT:    [[SCEVGEP:%.*]] = getelementptr i8, ptr addrspace(1) [[TMP0]], i64 [[TMP4]]
 ; CHECK-NEXT:    [[SCEVGEP1:%.*]] = getelementptr i8, ptr addrspace(1) [[TMP1]], i64 [[TMP4]]
 ; CHECK-NEXT:    [[BOUND0:%.*]] = icmp ult ptr addrspace(1) [[DOT10]], [[SCEVGEP1]]
 ; CHECK-NEXT:    [[BOUND1:%.*]] = icmp ult ptr addrspace(1) [[DOT12]], [[SCEVGEP]]
 ; CHECK-NEXT:    [[FOUND_CONFLICT:%.*]] = and i1 [[BOUND0]], [[BOUND1]]
 ; CHECK-NEXT:    br i1 [[FOUND_CONFLICT]], label [[SCALAR_PH]], label [[VECTOR_PH:%.*]]
 ; CHECK:       vector.ph:
 ; CHECK-NEXT:    [[N_VEC:%.*]] = and i64 [[UMAX2]], -16
 ; CHECK-NEXT:    br label [[VECTOR_BODY:%.*]]
 ; CHECK:       vector.body:
 ; CHECK-NEXT:    [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
 ; CHECK-NEXT:    [[TMP5:%.*]] = getelementptr inbounds ptr addrspace(1), ptr addrspace(1) [[DOT12]], i64 [[INDEX]]
 ; CHECK-NEXT:    [[WIDE_LOAD:%.*]] = load <4 x ptr addrspace(1)>, ptr addrspace(1) [[TMP5]], align 8, !alias.scope !0
-; CHECK-NEXT:    [[TMP7:%.*]] = getelementptr inbounds ptr addrspace(1), ptr addrspace(1) [[TMP5]], i64 4
-; CHECK-NEXT:    [[WIDE_LOAD3:%.*]] = load <4 x ptr addrspace(1)>, ptr addrspace(1) [[TMP7]], align 8, !alias.scope !0
-; CHECK-NEXT:    [[TMP9:%.*]] = getelementptr inbounds ptr addrspace(1), ptr addrspace(1) [[TMP5]], i64 8
-; CHECK-NEXT:    [[WIDE_LOAD4:%.*]] = load <4 x ptr addrspace(1)>, ptr addrspace(1) [[TMP9]], align 8, !alias.scope !0
-; CHECK-NEXT:    [[TMP11:%.*]] = getelementptr inbounds ptr addrspace(1), ptr addrspace(1) [[TMP5]], i64 12
-; CHECK-NEXT:    [[WIDE_LOAD5:%.*]] = load <4 x ptr addrspace(1)>, ptr addrspace(1) [[TMP11]], align 8, !alias.scope !0
-; CHECK-NEXT:    [[TMP13:%.*]] = getelementptr inbounds ptr addrspace(1), ptr addrspace(1) [[DOT10]], i64 [[INDEX]]
-; CHECK-NEXT:    store <4 x ptr addrspace(1)> [[WIDE_LOAD]], ptr addrspace(1) [[TMP13]], align 8, !alias.scope !3, !noalias !0
-; CHECK-NEXT:    [[TMP15:%.*]] = getelementptr inbounds ptr addrspace(1), ptr addrspace(1) [[TMP13]], i64 4
-; CHECK-NEXT:    store <4 x ptr addrspace(1)> [[WIDE_LOAD3]], ptr addrspace(1) [[TMP15]], align 8, !alias.scope !3, !noalias !0
-; CHECK-NEXT:    [[TMP17:%.*]] = getelementptr inbounds ptr addrspace(1), ptr addrspace(1) [[TMP13]], i64 8
-; CHECK-NEXT:    store <4 x ptr addrspace(1)> [[WIDE_LOAD4]], ptr addrspace(1) [[TMP17]], align 8, !alias.scope !3, !noalias !0
-; CHECK-NEXT:    [[TMP19:%.*]] = getelementptr inbounds ptr addrspace(1), ptr addrspace(1) [[TMP13]], i64 12
-; CHECK-NEXT:    store <4 x ptr addrspace(1)> [[WIDE_LOAD5]], ptr addrspace(1) [[TMP19]], align 8, !alias.scope !3, !noalias !0
+; CHECK-NEXT:    [[TMP6:%.*]] = getelementptr inbounds ptr addrspace(1), ptr addrspace(1) [[TMP5]], i64 4
+; CHECK-NEXT:    [[WIDE_LOAD3:%.*]] = load <4 x ptr addrspace(1)>, ptr addrspace(1) [[TMP6]], align 8, !alias.scope !0
+; CHECK-NEXT:    [[TMP7:%.*]] = getelementptr inbounds ptr addrspace(1), ptr addrspace(1) [[TMP5]], i64 8
+; CHECK-NEXT:    [[WIDE_LOAD4:%.*]] = load <4 x ptr addrspace(1)>, ptr addrspace(1) [[TMP7]], align 8, !alias.scope !0
+; CHECK-NEXT:    [[TMP8:%.*]] = getelementptr inbounds ptr addrspace(1), ptr addrspace(1) [[TMP5]], i64 12
+; CHECK-NEXT:    [[WIDE_LOAD5:%.*]] = load <4 x ptr addrspace(1)>, ptr addrspace(1) [[TMP8]], align 8, !alias.scope !0
+; CHECK-NEXT:    [[TMP9:%.*]] = getelementptr inbounds ptr addrspace(1), ptr addrspace(1) [[DOT10]], i64 [[INDEX]]
+; CHECK-NEXT:    store <4 x ptr addrspace(1)> [[WIDE_LOAD]], ptr addrspace(1) [[TMP9]], align 8, !alias.scope !3, !noalias !0
+; CHECK-NEXT:    [[TMP10:%.*]] = getelementptr inbounds ptr addrspace(1), ptr addrspace(1) [[TMP9]], i64 4
+; CHECK-NEXT:    store <4 x ptr addrspace(1)> [[WIDE_LOAD3]], ptr addrspace(1) [[TMP10]], align 8, !alias.scope !3, !noalias !0
+; CHECK-NEXT:    [[TMP11:%.*]] = getelementptr inbounds ptr addrspace(1), ptr addrspace(1) [[TMP9]], i64 8
+; CHECK-NEXT:    store <4 x ptr addrspace(1)> [[WIDE_LOAD4]], ptr addrspace(1) [[TMP11]], align 8, !alias.scope !3, !noalias !0
+; CHECK-NEXT:    [[TMP12:%.*]] = getelementptr inbounds ptr addrspace(1), ptr addrspace(1) [[TMP9]], i64 12
+; CHECK-NEXT:    store <4 x ptr addrspace(1)> [[WIDE_LOAD5]], ptr addrspace(1) [[TMP12]], align 8, !alias.scope !3, !noalias !0
 ; CHECK-NEXT:    [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 16
-; CHECK-NEXT:    [[TMP21:%.*]] = icmp eq i64 [[INDEX_NEXT]], [[N_VEC]]
-; CHECK-NEXT:    br i1 [[TMP21]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP5:![0-9]+]]
+; CHECK-NEXT:    [[TMP13:%.*]] = icmp eq i64 [[INDEX_NEXT]], [[N_VEC]]
+; CHECK-NEXT:    br i1 [[TMP13]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP5:![0-9]+]]
 ; CHECK:       middle.block:
 ; CHECK-NEXT:    [[CMP_N:%.*]] = icmp eq i64 [[UMAX2]], [[N_VEC]]
 ; CHECK-NEXT:    br i1 [[CMP_N]], label [[LOOPEXIT:%.*]], label [[SCALAR_PH]]
 ; CHECK:       scalar.ph:
 ; CHECK-NEXT:    [[BC_RESUME_VAL:%.*]] = phi i64 [ [[N_VEC]], [[MIDDLE_BLOCK]] ], [ 0, [[PREHEADER]] ], [ 0, [[VECTOR_MEMCHECK]] ]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[INDVARS_IV3:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[INDVARS_IV_NEXT4:%.*]], [[LOOP]] ]
 ; CHECK-NEXT:    [[DOT18:%.*]] = getelementptr inbounds ptr addrspace(1), ptr addrspace(1) [[DOT12]], i64 [[INDVARS_IV3]]
 ; CHECK-NEXT:    [[V:%.*]] = load ptr addrspace(1), ptr addrspace(1) [[DOT18]], align 8
 ; CHECK-NEXT:    [[DOT20:%.*]] = getelementptr inbounds ptr addrspace(1), ptr addrspace(1) [[DOT10]], i64 [[INDVARS_IV3]]
 ; CHECK-NEXT:    store ptr addrspace(1) [[V]], ptr addrspace(1) [[DOT20]], align 8
 ; CHECK-NEXT:    [[INDVARS_IV_NEXT4]] = add nuw nsw i64 [[INDVARS_IV3]], 1
 ; CHECK-NEXT:    [[DOT21:%.*]] = icmp ult i64 [[INDVARS_IV_NEXT4]], [[TMP2]]
-; CHECK-NEXT:    br i1 [[DOT21]], label [[LOOP]], label [[LOOPEXIT]], !llvm.loop [[LOOP7:![0-9]+]]
+; CHECK-NEXT:    br i1 [[DOT21]], label [[LOOP]], label [[LOOPEXIT]], !llvm.loop [[LOOP8:![0-9]+]]
 ; CHECK:       loopexit:
 ; CHECK-NEXT:    ret void
 ;
 entry:
   br label %preheader
 
 preheader:                                       ; preds = %4
   %.10 = getelementptr inbounds i8, ptr addrspace(1) %0, i64 16

llvm/test/Transforms/LoopVectorize/X86/pr54634.ll

 ; CHECK-NEXT:    [[TMP5:%.*]] = load ptr addrspace(10), ptr [[TMP0:%.*]], align 8, !tbaa [[TBAA0:![0-9]+]]
 ; CHECK-NEXT:    [[TMP6:%.*]] = addrspacecast ptr addrspace(10) [[TMP4]] to ptr addrspace(11)
 ; CHECK-NEXT:    [[TMP7:%.*]] = load ptr addrspace(13), ptr addrspace(11) [[TMP6]], align 8, !tbaa [[TBAA5:![0-9]+]]
 ; CHECK-NEXT:    [[DOTELT:%.*]] = getelementptr inbounds { ptr addrspace(10), i64 }, ptr addrspace(10) [[TMP5]], i64 0, i32 0
 ; CHECK-NEXT:    [[DOTUNPACK:%.*]] = load ptr addrspace(10), ptr addrspace(10) [[DOTELT]], align 8, !tbaa [[TBAA8:![0-9]+]]
 ; CHECK-NEXT:    [[DOTELT1:%.*]] = getelementptr inbounds { ptr addrspace(10), i64 }, ptr addrspace(10) [[TMP5]], i64 0, i32 1
 ; CHECK-NEXT:    [[DOTUNPACK2:%.*]] = load i64, ptr addrspace(10) [[DOTELT1]], align 8, !tbaa [[TBAA8]]
 ; CHECK-NEXT:    [[TMP8:%.*]] = add nsw i64 [[TMP2]], 1
-; CHECK-NEXT:    [[MIN_ITERS_CHECK:%.*]] = icmp ult i64 [[TMP8]], 28
+; CHECK-NEXT:    [[MIN_ITERS_CHECK:%.*]] = icmp ult i64 [[TMP8]], 36
 ; CHECK-NEXT:    br i1 [[MIN_ITERS_CHECK]], label [[SCALAR_PH:%.*]], label [[VECTOR_SCEVCHECK:%.*]]
 ; CHECK:       vector.scevcheck:
 ; CHECK-NEXT:    [[MUL:%.*]] = call { i64, i1 } @llvm.umul.with.overflow.i64(i64 16, i64 [[TMP2]])
 ; CHECK-NEXT:    [[MUL_RESULT:%.*]] = extractvalue { i64, i1 } [[MUL]], 0
 ; CHECK-NEXT:    [[MUL_OVERFLOW:%.*]] = extractvalue { i64, i1 } [[MUL]], 1
 ; CHECK-NEXT:    [[TMP9:%.*]] = sub i64 0, [[MUL_RESULT]]
 ; CHECK-NEXT:    [[TMP10:%.*]] = getelementptr i8, ptr addrspace(13) [[TMP7]], i64 [[MUL_RESULT]]
 ; CHECK-NEXT:    [[TMP11:%.*]] = icmp ult ptr addrspace(13) [[TMP10]], [[TMP7]]

llvm/test/Transforms/SLPVectorizer/RISCV/gep.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt < %s -passes=slp-vectorizer -mtriple=riscv64 -mattr=+v \
 ; RUN: -riscv-v-slp-max-vf=0 -S | FileCheck %s
 
-; FIXME: This should not be vectorized
+; This should not be vectorized as the extra cost of the address computation
+; that can't be folded into the vle8/vse8 outweighs the scalar cost.
 
 define void @copy_with_offset_v2i8(ptr noalias %p, ptr noalias %q) {
 ; CHECK-LABEL: @copy_with_offset_v2i8(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[P1:%.*]] = getelementptr i8, ptr [[P:%.*]], i32 8
+; CHECK-NEXT:    [[X1:%.*]] = load i8, ptr [[P1]], align 1
 ; CHECK-NEXT:    [[Q1:%.*]] = getelementptr i8, ptr [[Q:%.*]], i32 16
-; CHECK-NEXT:    [[TMP0:%.*]] = load <2 x i8>, ptr [[P1]], align 1
-; CHECK-NEXT:    store <2 x i8> [[TMP0]], ptr [[Q1]], align 1
+; CHECK-NEXT:    store i8 [[X1]], ptr [[Q1]], align 1
+; CHECK-NEXT:    [[P2:%.*]] = getelementptr i8, ptr [[P]], i32 9
+; CHECK-NEXT:    [[X2:%.*]] = load i8, ptr [[P2]], align 1
+; CHECK-NEXT:    [[Q2:%.*]] = getelementptr i8, ptr [[Q]], i32 17
+; CHECK-NEXT:    store i8 [[X2]], ptr [[Q2]], align 1
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %p1 = getelementptr i8, ptr %p, i32 8
   %x1 = load i8, ptr %p1
   %q1 = getelementptr i8, ptr %q, i32 16
   store i8 %x1, ptr %q1
 
   %p2 = getelementptr i8, ptr %p, i32 9
   %x2 = load i8, ptr %p2
   %q2 = getelementptr i8, ptr %q, i32 17
   store i8 %x2, ptr %q2
 
   ret void
 }

llvm/test/Transforms/SLPVectorizer/X86/geps-non-pow-2.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -passes=slp-vectorizer -S -o - -mtriple=x86_64-unknown-linux -mcpu=haswell -slp-threshold=-3 < %s | FileCheck %s
 @e = dso_local local_unnamed_addr global i32 0, align 4
 @f = dso_local local_unnamed_addr global i32 0, align 4
 
 ; Function Attrs: nofree norecurse nounwind uwtable
 define dso_local i32 @g() local_unnamed_addr {
 ; CHECK-LABEL: @g(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[TMP0:%.*]] = load i32, ptr @e, align 4
 ; CHECK-NEXT:    [[TOBOOL_NOT19:%.*]] = icmp eq i32 [[TMP0]], 0
 ; CHECK-NEXT:    br i1 [[TOBOOL_NOT19]], label [[WHILE_END:%.*]], label [[WHILE_BODY:%.*]]
 ; CHECK:       while.body:
-; CHECK-NEXT:    [[C_022:%.*]] = phi ptr [ [[C_022_BE:%.*]], [[WHILE_BODY_BACKEDGE:%.*]] ], [ undef, [[ENTRY:%.*]] ]
-; CHECK-NEXT:    [[TMP1:%.*]] = phi <2 x ptr> [ [[TMP14:%.*]], [[WHILE_BODY_BACKEDGE]] ], [ undef, [[ENTRY]] ]
-; CHECK-NEXT:    [[INCDEC_PTR:%.*]] = getelementptr inbounds i32, ptr [[C_022]], i64 1
-; CHECK-NEXT:    [[TMP2:%.*]] = ptrtoint ptr [[C_022]] to i64
-; CHECK-NEXT:    [[TMP3:%.*]] = trunc i64 [[TMP2]] to i32
-; CHECK-NEXT:    [[TMP4:%.*]] = getelementptr i32, <2 x ptr> [[TMP1]], <2 x i64> <i64 1, i64 1>
-; CHECK-NEXT:    switch i32 [[TMP3]], label [[WHILE_BODY_BACKEDGE]] [
+; CHECK-NEXT:    [[A_020:%.*]] = phi ptr [ [[A_020_BE:%.*]], [[WHILE_BODY_BACKEDGE:%.*]] ], [ undef, [[ENTRY:%.*]] ]
+; CHECK-NEXT:    [[TMP1:%.*]] = phi <2 x ptr> [ [[TMP15:%.*]], [[WHILE_BODY_BACKEDGE]] ], [ undef, [[ENTRY]] ]
+; CHECK-NEXT:    [[TMP2:%.*]] = extractelement <2 x ptr> [[TMP1]], i32 0
+; CHECK-NEXT:    [[TMP3:%.*]] = ptrtoint ptr [[TMP2]] to i64
+; CHECK-NEXT:    [[TMP4:%.*]] = trunc i64 [[TMP3]] to i32
+; CHECK-NEXT:    [[INCDEC_PTR1:%.*]] = getelementptr inbounds i32, ptr [[A_020]], i64 1
+; CHECK-NEXT:    [[TMP5:%.*]] = getelementptr i32, <2 x ptr> [[TMP1]], <2 x i64> <i64 1, i64 1>
+; CHECK-NEXT:    switch i32 [[TMP4]], label [[WHILE_BODY_BACKEDGE]] [
 ; CHECK-NEXT:    i32 2, label [[SW_BB:%.*]]
 ; CHECK-NEXT:    i32 4, label [[SW_BB6:%.*]]
 ; CHECK-NEXT:    ]
 ; CHECK:       sw.bb:
-; CHECK-NEXT:    [[TMP5:%.*]] = extractelement <2 x ptr> [[TMP4]], i32 0
-; CHECK-NEXT:    [[TMP6:%.*]] = ptrtoint ptr [[TMP5]] to i64
-; CHECK-NEXT:    [[TMP7:%.*]] = trunc i64 [[TMP6]] to i32
-; CHECK-NEXT:    [[TMP8:%.*]] = getelementptr i32, <2 x ptr> [[TMP1]], <2 x i64> <i64 2, i64 2>
-; CHECK-NEXT:    [[TMP9:%.*]] = extractelement <2 x ptr> [[TMP4]], i32 1
-; CHECK-NEXT:    store i32 [[TMP7]], ptr [[TMP9]], align 4
-; CHECK-NEXT:    [[INCDEC_PTR5:%.*]] = getelementptr inbounds i32, ptr [[C_022]], i64 2
+; CHECK-NEXT:    [[TMP6:%.*]] = extractelement <2 x ptr> [[TMP5]], i32 1
+; CHECK-NEXT:    [[TMP7:%.*]] = ptrtoint ptr [[TMP6]] to i64
+; CHECK-NEXT:    [[TMP8:%.*]] = trunc i64 [[TMP7]] to i32
+; CHECK-NEXT:    [[INCDEC_PTR4:%.*]] = getelementptr inbounds i32, ptr [[A_020]], i64 2
+; CHECK-NEXT:    store i32 [[TMP8]], ptr [[INCDEC_PTR1]], align 4
+; CHECK-NEXT:    [[TMP9:%.*]] = getelementptr i32, <2 x ptr> [[TMP1]], <2 x i64> <i64 2, i64 2>
 ; CHECK-NEXT:    br label [[WHILE_BODY_BACKEDGE]]
 ; CHECK:       sw.bb6:
-; CHECK-NEXT:    [[INCDEC_PTR8:%.*]] = getelementptr inbounds i32, ptr [[C_022]], i64 2
-; CHECK-NEXT:    [[TMP10:%.*]] = ptrtoint ptr [[INCDEC_PTR]] to i64
-; CHECK-NEXT:    [[TMP11:%.*]] = trunc i64 [[TMP10]] to i32
-; CHECK-NEXT:    [[TMP12:%.*]] = getelementptr i32, <2 x ptr> [[TMP1]], <2 x i64> <i64 2, i64 2>
-; CHECK-NEXT:    [[TMP13:%.*]] = extractelement <2 x ptr> [[TMP4]], i32 0
-; CHECK-NEXT:    store i32 [[TMP11]], ptr [[TMP13]], align 4
+; CHECK-NEXT:    [[INCDEC_PTR7:%.*]] = getelementptr inbounds i32, ptr [[A_020]], i64 2
+; CHECK-NEXT:    [[TMP10:%.*]] = extractelement <2 x ptr> [[TMP5]], i32 0
+; CHECK-NEXT:    [[TMP11:%.*]] = ptrtoint ptr [[TMP10]] to i64
+; CHECK-NEXT:    [[TMP12:%.*]] = trunc i64 [[TMP11]] to i32
+; CHECK-NEXT:    [[TMP13:%.*]] = getelementptr i32, <2 x ptr> [[TMP1]], <2 x i64> <i64 2, i64 2>
+; CHECK-NEXT:    [[TMP14:%.*]] = extractelement <2 x ptr> [[TMP5]], i32 1
+; CHECK-NEXT:    store i32 [[TMP12]], ptr [[TMP14]], align 4
 ; CHECK-NEXT:    br label [[WHILE_BODY_BACKEDGE]]
 ; CHECK:       while.body.backedge:
-; CHECK-NEXT:    [[C_022_BE]] = phi ptr [ [[INCDEC_PTR]], [[WHILE_BODY]] ], [ [[INCDEC_PTR8]], [[SW_BB6]] ], [ [[INCDEC_PTR5]], [[SW_BB]] ]
-; CHECK-NEXT:    [[TMP14]] = phi <2 x ptr> [ [[TMP4]], [[WHILE_BODY]] ], [ [[TMP12]], [[SW_BB6]] ], [ [[TMP8]], [[SW_BB]] ]
+; CHECK-NEXT:    [[A_020_BE]] = phi ptr [ [[INCDEC_PTR1]], [[WHILE_BODY]] ], [ [[INCDEC_PTR7]], [[SW_BB6]] ], [ [[INCDEC_PTR4]], [[SW_BB]] ]
+; CHECK-NEXT:    [[TMP15]] = phi <2 x ptr> [ [[TMP5]], [[WHILE_BODY]] ], [ [[TMP13]], [[SW_BB6]] ], [ [[TMP9]], [[SW_BB]] ]
 ; CHECK-NEXT:    br label [[WHILE_BODY]]
 ; CHECK:       while.end:
 ; CHECK-NEXT:    ret i32 undef
 ;
 entry:
   %0 = load i32, ptr @e, align 4
   %tobool.not19 = icmp eq i32 %0, 0
   br i1 %tobool.not19, label %while.end, label %while.body