[RISCV] Cost constant materialization of vectors in phis

[luke]

Summary

If a phi has a constant operand it will need to be materialized at some
point, so account for this cost by reusing the logic used for store
immediates.
This should eventually be improved to also handle scalar constants.

[luke]

llvm/test/Analysis/CostModel/RISCV/rvv-phi-const.ll

9

The generated code:

f:                                      # @f
	.cfi_startproc
# %bb.0:
	vsetivli	zero, 2, e8, mf8, ta, ma
	vid.v	v8
	andi	a0, a0, 1
	vadd.vv	v8, v8, v8
	beqz	a0, .LBB0_2
# %bb.1:
	vrsub.vi	v8, v8, 1
	ret
.LBB0_2:                                # %b
	vadd.vi	v8, v8, -1
	ret

[ABataev]

llvm/test/Analysis/CostModel/RISCV/rvv-phi-const.ll

9

I have one concern here about loops. In the loop we need to account for the materialization cost only once, if the input constant is a live-in. So for such values better to consider it free, I assume.

[luke]

llvm/test/Analysis/CostModel/RISCV/rvv-phi-const.ll

9

That's a good point. The cost of constants in stores must also be currently affected by this too right?

[ABataev]

llvm/test/Analysis/CostModel/RISCV/rvv-phi-const.ll

9

Looks so. But it requires some significant rework of the cost model, I assume. For phi's you can try just to check if there is a loop in the graph (with the small enough limit), without actual analysis/use of the Loop objects.

[luke]

llvm/test/Analysis/CostModel/RISCV/rvv-phi-const.ll

9

Yes, and it it looks like arithmetic instructions are also affected, not just on RISC-V but on at least AArch64 too.
If this is a pervasive problem that already exists across the entire cost model then perhaps it would be best to stay consistent and include materialisation in getPHICost, but do some sort of "temporary" workaround in SLP whenever it's in a loop, as you suggest.

[luke]

Rebase

[ABataev]

llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp

1428

const OperandValueInfo &OpInfo

[luke]

Address comment and rebase

[luke]

After discussing offline with @reames, we're not sure if this is the most accurate way to model constant costs in phi nodes. Moving it off the review queue for the meantime

llvm/lib/Target/RISCV/RISCVTargetTransformInfo.h

InstructionCost getConstantPoolLoadCost(Type *Ty,
                                           TTI::TargetCostKind CostKind);
 public:
   explicit RISCVTTIImpl(const RISCVTargetMachine *TM, const Function &F)
       : BaseT(TM, F.getParent()->getDataLayout()), ST(TM->getSubtargetImpl(F)),
         TLI(ST->getTargetLowering()) {}
 
   /// Return the cost of materializing an immediate for a value operand of
   /// a store instruction.
-  InstructionCost getStoreImmCost(Type *VecTy, TTI::OperandValueInfo OpInfo,
+  InstructionCost getConstantMaterializationCost(Type *VecTy,
+                                                 TTI::OperandValueInfo OpInfo,
                                                  TTI::TargetCostKind CostKind);
 
   InstructionCost getIntImmCost(const APInt &Imm, Type *Ty,
                                 TTI::TargetCostKind CostKind);
   InstructionCost getIntImmCostInst(unsigned Opcode, unsigned Idx,
                                     const APInt &Imm, Type *Ty,
                                     TTI::TargetCostKind CostKind,
                                     Instruction *Inst = nullptr);
   InstructionCost getIntImmCostIntrin(Intrinsic::ID IID, unsigned Idx,
getMemoryOpCost(unsigned Opcode, Type *Src, MaybeAlign Alignment,
                   TTI::OperandValueInfo OpdInfo = {TTI::OK_AnyValue, TTI::OP_None},
                   const Instruction *I = nullptr);
 
   InstructionCost getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy,
                                      CmpInst::Predicate VecPred,
                                      TTI::TargetCostKind CostKind,
                                      const Instruction *I = nullptr);
 
+  InstructionCost getPHICost(Type *Ty, TTI::TargetCostKind CostKind,
+                             ArrayRef<TTI::OperandValueInfo> OpInfos,
+                             const Instruction *I = nullptr);
+
   using BaseT::getVectorInstrCost;
   InstructionCost getVectorInstrCost(unsigned Opcode, Type *Val,
                                      TTI::TargetCostKind CostKind,
                                      unsigned Index, Value *Op0, Value *Op1);
 
   InstructionCost getArithmeticInstrCost(
       unsigned Opcode, Type *Ty, TTI::TargetCostKind CostKind,
       TTI::OperandValueInfo Op1Info = {TTI::OK_AnyValue, TTI::OP_None},

llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp

InstructionCost RISCVTTIImpl::getExtendedReductionCost(
   if (ResTy->getScalarSizeInBits() != 2 * LT.second.getScalarSizeInBits())
     return BaseT::getExtendedReductionCost(Opcode, IsUnsigned, ResTy, ValTy,
                                            FMF, CostKind);
 
   return (LT.first - 1) +
          getArithmeticReductionCost(Opcode, ValTy, FMF, CostKind);
 }
 
-InstructionCost RISCVTTIImpl::getStoreImmCost(Type *Ty,
-                                              TTI::OperandValueInfo OpInfo,
-                                              TTI::TargetCostKind CostKind) {
+InstructionCost RISCVTTIImpl::getConstantMaterializationCost(
+    Type *Ty, TTI::OperandValueInfo OpInfo, TTI::TargetCostKind CostKind) {
   assert(OpInfo.isConstant() && "non constant operand?");
   if (!isa<VectorType>(Ty))
     // FIXME: We need to account for immediate materialization here, but doing
     // a decent job requires more knowledge about the immediate than we
     // currently have here.
     return 0;
 
   if (OpInfo.isUniform())
     // vmv.x.i, vmv.v.x, or vfmv.v.f
     // We ignore the cost of the scalar constant materialization to be consistent
     // with how we treat scalar constants themselves just above.
     return 1;
 
   return getConstantPoolLoadCost(Ty, CostKind);
 }
 
-
 InstructionCost RISCVTTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src,
                                               MaybeAlign Alignment,
                                               unsigned AddressSpace,
                                               TTI::TargetCostKind CostKind,
                                               TTI::OperandValueInfo OpInfo,
                                               const Instruction *I) {
   EVT VT = TLI->getValueType(DL, Src, true);
   // Type legalization can't handle structs
   if (VT == MVT::Other)
     return BaseT::getMemoryOpCost(Opcode, Src, Alignment, AddressSpace,
                                   CostKind, OpInfo, I);
 
   InstructionCost Cost = 0;
   if (Opcode == Instruction::Store && OpInfo.isConstant())
-    Cost += getStoreImmCost(Src, OpInfo, CostKind);
+    Cost += getConstantMaterializationCost(Src, OpInfo, CostKind);
   InstructionCost BaseCost =
     BaseT::getMemoryOpCost(Opcode, Src, Alignment, AddressSpace,
                            CostKind, OpInfo, I);
   // Assume memory ops cost scale with the number of vector registers
   // possible accessed by the instruction.  Note that BasicTTI already
   // handles the LT.first term for us.
   if (std::pair<InstructionCost, MVT> LT = getTypeLegalizationCost(Src);
       LT.second.isVector())
if ((Opcode == Instruction::ICmp || Opcode == Instruction::FCmp) &&
     }
   }
 
   // TODO: Add cost for scalar type.
 
   return BaseT::getCmpSelInstrCost(Opcode, ValTy, CondTy, VecPred, CostKind, I);
 }
 
+InstructionCost
+RISCVTTIImpl::getPHICost(Type *Ty, TTI::TargetCostKind CostKind,
+                         ArrayRef<TTI::OperandValueInfo> OpInfos,
+                         const Instruction *I) {
+  // TODO: Have getConstantMaterializationCost compute cost for scalars
+  if (CostKind == TTI::TCK_RecipThroughput && Ty->isVectorTy()) {
+    InstructionCost Cost = 0;
+    // The throughput cost is going to be the cost of one of the possible
+    // incoming values, so choose the most expensive
+    for (const TTI::OperandValueInfo &OpInfo : OpInfos)
+      if (OpInfo.isConstant())
+        Cost = std::max(Cost,
+                        getConstantMaterializationCost(Ty, OpInfo, CostKind));
+    return Cost;
+  }
+  return BaseT::getPHICost(Ty, CostKind, OpInfos, I);
+}
+
 InstructionCost RISCVTTIImpl::getVectorInstrCost(unsigned Opcode, Type *Val,
                                                  TTI::TargetCostKind CostKind,
                                                  unsigned Index, Value *Op0,
                                                  Value *Op1) {
   assert(Val->isVectorTy() && "This must be a vector type");
 
   if (Opcode != Instruction::ExtractElement &&
       Opcode != Instruction::InsertElement)

llvm/test/Analysis/CostModel/RISCV/rvv-phi-const.ll

 ; NOTE: Assertions have been autogenerated by utils/update_analyze_test_checks.py UTC_ARGS: --version 2
 ; RUN: opt < %s -mtriple=riscv64 -mattr=+v,+f,+d -disable-output -passes="print<cost-model>" 2>&1 | FileCheck %s
 
 define <2 x i8> @phi_v2i8_small(i1 %c) {
 ; CHECK-LABEL: 'phi_v2i8_small'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <2 x i8> [ <i8 1, i8 -1>, %a ], [ <i8 -1, i8 1>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 3 for instruction: %x = phi <2 x i8> [ <i8 1, i8 -1>, %a ], [ <i8 -1, i8 1>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <2 x i8> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
d:
   ret <4 x i8> %x
 }
 
 define <4 x i8> @phi_v4i8_small(i1 %c) {
 ; CHECK-LABEL: 'phi_v4i8_small'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x i8> [ <i8 1, i8 2, i8 3, i8 4>, %a ], [ <i8 2, i8 4, i8 6, i8 8>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 3 for instruction: %x = phi <4 x i8> [ <i8 1, i8 2, i8 3, i8 4>, %a ], [ <i8 2, i8 4, i8 6, i8 8>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x i8> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
   %x = phi <4 x i8> [<i8 1, i8 2, i8 3, i8 4>, %a], [<i8 2, i8 4, i8 6, i8 8>, %b]
   ret <4 x i8> %x
 }
 
 define <4 x i8> @phi_v4i8_large(i1 %c) {
 ; CHECK-LABEL: 'phi_v4i8_large'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x i8> [ <i8 -46, i8 -31, i8 15, i8 74>, %a ], [ <i8 8, i8 32, i8 -59, i8 12>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 3 for instruction: %x = phi <4 x i8> [ <i8 -46, i8 -31, i8 15, i8 74>, %a ], [ <i8 8, i8 32, i8 -59, i8 12>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x i8> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
   %x = phi <4 x i8> [<i8 1234, i8 4321, i8 9999, i8 2378>, %a], [<i8 7432, i8 -4832, i8 381381, i8 12>, %b]
   ret <4 x i8> %x
 }
 
 ; The cost should be conservative and assume it will need to materialize the more expensive constant
 define <4 x i8> @phi_v4i8_cheap_and_expensive(i1 %c) {
 ; CHECK-LABEL: 'phi_v4i8_cheap_and_expensive'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x i8> [ <i8 1, i8 1, i8 1, i8 1>, %a ], [ <i8 8, i8 32, i8 -59, i8 12>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 3 for instruction: %x = phi <4 x i8> [ <i8 1, i8 1, i8 1, i8 1>, %a ], [ <i8 8, i8 32, i8 -59, i8 12>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x i8> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
d:
   ret <4 x i16> %x
 }
 
 define <4 x i16> @phi_v4i16_small(i1 %c) {
 ; CHECK-LABEL: 'phi_v4i16_small'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x i16> [ <i16 1, i16 2, i16 3, i16 4>, %a ], [ <i16 2, i16 4, i16 6, i16 8>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 3 for instruction: %x = phi <4 x i16> [ <i16 1, i16 2, i16 3, i16 4>, %a ], [ <i16 2, i16 4, i16 6, i16 8>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x i16> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
   %x = phi <4 x i16> [<i16 1, i16 2, i16 3, i16 4>, %a], [<i16 2, i16 4, i16 6, i16 8>, %b]
   ret <4 x i16> %x
 }
 
 define <4 x i16> @phi_v4i16_large(i1 %c) {
 ; CHECK-LABEL: 'phi_v4i16_large'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x i16> [ <i16 1234, i16 4321, i16 9999, i16 2378>, %a ], [ <i16 7432, i16 -4832, i16 -11835, i16 12>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 3 for instruction: %x = phi <4 x i16> [ <i16 1234, i16 4321, i16 9999, i16 2378>, %a ], [ <i16 7432, i16 -4832, i16 -11835, i16 12>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x i16> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
   %x = phi <4 x i16> [<i16 1234, i16 4321, i16 9999, i16 2378>, %a], [<i16 7432, i16 -4832, i16 381381, i16 12>, %b]
   ret <4 x i16> %x
 }
 
 ; The cost should be conservative and assume it will need to materialize the more expensive constant
 define <4 x i16> @phi_v4i16_cheap_and_expensive(i1 %c) {
 ; CHECK-LABEL: 'phi_v4i16_cheap_and_expensive'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x i16> [ <i16 1, i16 1, i16 1, i16 1>, %a ], [ <i16 7432, i16 -4832, i16 -11835, i16 12>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 3 for instruction: %x = phi <4 x i16> [ <i16 1, i16 1, i16 1, i16 1>, %a ], [ <i16 7432, i16 -4832, i16 -11835, i16 12>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x i16> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
d:
   ret <4 x i32> %x
 }
 
 define <4 x i32> @phi_v4i32_small(i1 %c) {
 ; CHECK-LABEL: 'phi_v4i32_small'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x i32> [ <i32 1, i32 2, i32 3, i32 4>, %a ], [ <i32 2, i32 4, i32 6, i32 8>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 3 for instruction: %x = phi <4 x i32> [ <i32 1, i32 2, i32 3, i32 4>, %a ], [ <i32 2, i32 4, i32 6, i32 8>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x i32> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
   %x = phi <4 x i32> [<i32 1, i32 2, i32 3, i32 4>, %a], [<i32 2, i32 4, i32 6, i32 8>, %b]
   ret <4 x i32> %x
 }
 
 define <4 x i32> @phi_v4i32_large(i1 %c) {
 ; CHECK-LABEL: 'phi_v4i32_large'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x i32> [ <i32 1234, i32 4321, i32 9999, i32 2378>, %a ], [ <i32 7432, i32 -4832, i32 381381, i32 12>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 3 for instruction: %x = phi <4 x i32> [ <i32 1234, i32 4321, i32 9999, i32 2378>, %a ], [ <i32 7432, i32 -4832, i32 381381, i32 12>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x i32> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
   %x = phi <4 x i32> [<i32 1234, i32 4321, i32 9999, i32 2378>, %a], [<i32 7432, i32 -4832, i32 381381, i32 12>, %b]
   ret <4 x i32> %x
 }
 
 ; The cost should be conservative and assume it will need to materialize the more expensive constant
 define <4 x i32> @phi_v4i32_cheap_and_expensive(i1 %c) {
 ; CHECK-LABEL: 'phi_v4i32_cheap_and_expensive'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x i32> [ <i32 1, i32 1, i32 1, i32 1>, %a ], [ <i32 7432, i32 -4832, i32 381381, i32 12>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 3 for instruction: %x = phi <4 x i32> [ <i32 1, i32 1, i32 1, i32 1>, %a ], [ <i32 7432, i32 -4832, i32 381381, i32 12>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x i32> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
d:
   ret <4 x i64> %x
 }
 
 define <4 x i64> @phi_v4i64_small(i1 %c) {
 ; CHECK-LABEL: 'phi_v4i64_small'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x i64> [ <i64 1, i64 2, i64 3, i64 4>, %a ], [ <i64 2, i64 4, i64 6, i64 8>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: %x = phi <4 x i64> [ <i64 1, i64 2, i64 3, i64 4>, %a ], [ <i64 2, i64 4, i64 6, i64 8>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x i64> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
   %x = phi <4 x i64> [<i64 1, i64 2, i64 3, i64 4>, %a], [<i64 2, i64 4, i64 6, i64 8>, %b]
   ret <4 x i64> %x
 }
 
 define <4 x i64> @phi_v4i64_large(i1 %c) {
 ; CHECK-LABEL: 'phi_v4i64_large'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x i64> [ <i64 1234, i64 4321, i64 9999, i64 2378>, %a ], [ <i64 7432, i64 -4832, i64 381381, i64 12>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: %x = phi <4 x i64> [ <i64 1234, i64 4321, i64 9999, i64 2378>, %a ], [ <i64 7432, i64 -4832, i64 381381, i64 12>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x i64> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
   %x = phi <4 x i64> [<i64 1234, i64 4321, i64 9999, i64 2378>, %a], [<i64 7432, i64 -4832, i64 381381, i64 12>, %b]
   ret <4 x i64> %x
 }
 
 ; The cost should be conservative and assume it will need to materialize the more expensive constant
 define <4 x i64> @phi_v4i64_cheap_and_expensive(i1 %c) {
 ; CHECK-LABEL: 'phi_v4i64_cheap_and_expensive'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x i64> [ <i64 1, i64 1, i64 1, i64 1>, %a ], [ <i64 7432, i64 -4832, i64 381381, i64 12>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: %x = phi <4 x i64> [ <i64 1, i64 1, i64 1, i64 1>, %a ], [ <i64 7432, i64 -4832, i64 381381, i64 12>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x i64> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
   %x = phi <4 x i64> [<i64 1, i64 1, i64 1, i64 1>, %a], [<i64 7432, i64 -4832, i64 381381, i64 12>, %b]
   ret <4 x i64> %x
 }
 
 define <4 x half> @phi_v4f16_splat(i1 %c) {
 ; CHECK-LABEL: 'phi_v4f16_splat'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x half> [ <half 0xH3C00, half 0xH3C00, half 0xH3C00, half 0xH3C00>, %a ], [ <half 0xH4000, half 0xH4000, half 0xH4000, half 0xH4D00>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 6 for instruction: %x = phi <4 x half> [ <half 0xH3C00, half 0xH3C00, half 0xH3C00, half 0xH3C00>, %a ], [ <half 0xH4000, half 0xH4000, half 0xH4000, half 0xH4D00>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x half> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
   %x = phi <4 x half> [<half 1.0, half 1.0, half 1.0, half 1.0>, %a], [<half 2.0, half 2.0, half 2.0, half 20.>, %b]
   ret <4 x half> %x
 }
 
 define <4 x half> @phi_v4f16(i1 %c) {
 ; CHECK-LABEL: 'phi_v4f16'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x half> [ <half 0xH3C00, half 0xH4000, half 0xH4200, half 0xH4400>, %a ], [ <half 0xH4000, half 0xH4400, half 0xH4600, half 0xH4800>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 6 for instruction: %x = phi <4 x half> [ <half 0xH3C00, half 0xH4000, half 0xH4200, half 0xH4400>, %a ], [ <half 0xH4000, half 0xH4400, half 0xH4600, half 0xH4800>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x half> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
   %x = phi <4 x half> [<half 1.0, half 2.0, half 3.0, half 4.0>, %a], [<half 2.0, half 4.0, half 6.0, half 8.0>, %b]
   ret <4 x half> %x
 }
 
 ; The cost should be conservative and assume it will need to materialize the more expensive constant
 define <4 x half> @phi_v4f16_cheap_and_expensive(i1 %c) {
 ; CHECK-LABEL: 'phi_v4f16_cheap_and_expensive'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x half> [ <half 0xH3C00, half 0xH3C00, half 0xH3C00, half 0xH3C00>, %a ], [ <half 0xH6F42, half 0xHECB8, half 0xH5DF6, half 0xH4A40>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 6 for instruction: %x = phi <4 x half> [ <half 0xH3C00, half 0xH3C00, half 0xH3C00, half 0xH3C00>, %a ], [ <half 0xH6F42, half 0xHECB8, half 0xH5DF6, half 0xH4A40>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x half> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
   %x = phi <4 x half> [<half 1.0, half 1.0, half 1.0, half 1.0>, %a], [<half 7432.0, half -4832.0, half 381.5, half 12.5>, %b]
   ret <4 x half> %x
 }
 
 define <4 x float> @phi_v4f32_splat(i1 %c) {
 ; CHECK-LABEL: 'phi_v4f32_splat'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x float> [ <float 1.000000e+00, float 1.000000e+00, float 1.000000e+00, float 1.000000e+00>, %a ], [ <float 2.000000e+00, float 2.000000e+00, float 2.000000e+00, float 2.000000e+01>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 3 for instruction: %x = phi <4 x float> [ <float 1.000000e+00, float 1.000000e+00, float 1.000000e+00, float 1.000000e+00>, %a ], [ <float 2.000000e+00, float 2.000000e+00, float 2.000000e+00, float 2.000000e+01>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x float> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
   %x = phi <4 x float> [<float 1.0, float 1.0, float 1.0, float 1.0>, %a], [<float 2.0, float 2.0, float 2.0, float 20.>, %b]
   ret <4 x float> %x
 }
 
 define <4 x float> @phi_v4f32(i1 %c) {
 ; CHECK-LABEL: 'phi_v4f32'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x float> [ <float 1.000000e+00, float 2.000000e+00, float 3.000000e+00, float 4.000000e+00>, %a ], [ <float 2.000000e+00, float 4.000000e+00, float 6.000000e+00, float 8.000000e+00>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 3 for instruction: %x = phi <4 x float> [ <float 1.000000e+00, float 2.000000e+00, float 3.000000e+00, float 4.000000e+00>, %a ], [ <float 2.000000e+00, float 4.000000e+00, float 6.000000e+00, float 8.000000e+00>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x float> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
   %x = phi <4 x float> [<float 1.0, float 2.0, float 3.0, float 4.0>, %a], [<float 2.0, float 4.0, float 6.0, float 8.0>, %b]
   ret <4 x float> %x
 }
 
 ; The cost should be conservative and assume it will need to materialize the more expensive constant
 define <4 x float> @phi_v4f32_cheap_and_expensive(i1 %c) {
 ; CHECK-LABEL: 'phi_v4f32_cheap_and_expensive'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x float> [ <float 1.000000e+00, float 1.000000e+00, float 1.000000e+00, float 1.000000e+00>, %a ], [ <float 7.432000e+03, float -4.832000e+03, float 3.815000e+02, float 1.250000e+01>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 3 for instruction: %x = phi <4 x float> [ <float 1.000000e+00, float 1.000000e+00, float 1.000000e+00, float 1.000000e+00>, %a ], [ <float 7.432000e+03, float -4.832000e+03, float 3.815000e+02, float 1.250000e+01>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x float> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
   %x = phi <4 x float> [<float 1.0, float 1.0, float 1.0, float 1.0>, %a], [<float 7432.0, float -4832.0, float 381.5, float 12.5>, %b]
   ret <4 x float> %x
 }
 
 define <4 x double> @phi_v4f64_splat(i1 %c) {
 ; CHECK-LABEL: 'phi_v4f64_splat'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x double> [ <double 1.000000e+00, double 1.000000e+00, double 1.000000e+00, double 1.000000e+00>, %a ], [ <double 2.000000e+00, double 2.000000e+00, double 2.000000e+00, double 2.000000e+01>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: %x = phi <4 x double> [ <double 1.000000e+00, double 1.000000e+00, double 1.000000e+00, double 1.000000e+00>, %a ], [ <double 2.000000e+00, double 2.000000e+00, double 2.000000e+00, double 2.000000e+01>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x double> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
   %x = phi <4 x double> [<double 1.0, double 1.0, double 1.0, double 1.0>, %a], [<double 2.0, double 2.0, double 2.0, double 20.>, %b]
   ret <4 x double> %x
 }
 
 define <4 x double> @phi_v4f64(i1 %c) {
 ; CHECK-LABEL: 'phi_v4f64'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x double> [ <double 1.000000e+00, double 2.000000e+00, double 3.000000e+00, double 4.000000e+00>, %a ], [ <double 2.000000e+00, double 4.000000e+00, double 6.000000e+00, double 8.000000e+00>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: %x = phi <4 x double> [ <double 1.000000e+00, double 2.000000e+00, double 3.000000e+00, double 4.000000e+00>, %a ], [ <double 2.000000e+00, double 4.000000e+00, double 6.000000e+00, double 8.000000e+00>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x double> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
   %x = phi <4 x double> [<double 1.0, double 2.0, double 3.0, double 4.0>, %a], [<double 2.0, double 4.0, double 6.0, double 8.0>, %b]
   ret <4 x double> %x
 }
 
 ; The cost should be conservative and assume it will need to materialize the more expensive constant
 define <4 x double> @phi_v4f64_cheap_and_expensive(i1 %c) {
 ; CHECK-LABEL: 'phi_v4f64_cheap_and_expensive'
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br i1 %c, label %a, label %b
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: br label %d
-; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: %x = phi <4 x double> [ <double 1.000000e+00, double 1.000000e+00, double 1.000000e+00, double 1.000000e+00>, %a ], [ <double 7.432000e+03, double -4.832000e+03, double 3.815000e+02, double 1.250000e+01>, %b ]
+; CHECK-NEXT:  Cost Model: Found an estimated cost of 4 for instruction: %x = phi <4 x double> [ <double 1.000000e+00, double 1.000000e+00, double 1.000000e+00, double 1.000000e+00>, %a ], [ <double 7.432000e+03, double -4.832000e+03, double 3.815000e+02, double 1.250000e+01>, %b ]
 ; CHECK-NEXT:  Cost Model: Found an estimated cost of 1 for instruction: ret <4 x double> %x
 ;
   br i1 %c, label %a, label %b
 a:
   br label %d
 b:
   br label %d
 d:
   %x = phi <4 x double> [<double 1.0, double 1.0, double 1.0, double 1.0>, %a], [<double 7432.0, double -4832.0, double 381.5, double 12.5>, %b]
   ret <4 x double> %x
 }

llvm/test/Transforms/LoopVectorize/RISCV/strided-accesses.ll

 }
 
 
 define void @single_constant_stride_ptr_iv(ptr %p) {
 ; CHECK-LABEL: @single_constant_stride_ptr_iv(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br i1 false, label [[SCALAR_PH:%.*]], label [[VECTOR_PH:%.*]]
 ; CHECK:       vector.ph:
-; CHECK-NEXT:    [[IND_END:%.*]] = getelementptr i8, ptr [[P:%.*]], i64 8128
+; CHECK-NEXT:    [[IND_END:%.*]] = getelementptr i8, ptr [[P:%.*]], i64 8064
 ; CHECK-NEXT:    br label [[VECTOR_BODY:%.*]]
 ; CHECK:       vector.body:
 ; CHECK-NEXT:    [[POINTER_PHI:%.*]] = phi ptr [ [[P]], [[VECTOR_PH]] ], [ [[PTR_IND:%.*]], [[VECTOR_BODY]] ]
 ; CHECK-NEXT:    [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
 ; CHECK-NEXT:    [[TMP0:%.*]] = getelementptr i8, ptr [[POINTER_PHI]], <8 x i64> <i64 0, i64 8, i64 16, i64 24, i64 32, i64 40, i64 48, i64 56>
-; CHECK-NEXT:    [[TMP1:%.*]] = extractelement <8 x ptr> [[TMP0]], i32 0
-; CHECK-NEXT:    [[TMP2:%.*]] = getelementptr i32, ptr [[TMP1]], i32 0
-; CHECK-NEXT:    [[WIDE_VEC:%.*]] = load <16 x i32>, ptr [[TMP2]], align 4
+; CHECK-NEXT:    [[TMP1:%.*]] = getelementptr i8, ptr [[POINTER_PHI]], <8 x i64> <i64 64, i64 72, i64 80, i64 88, i64 96, i64 104, i64 112, i64 120>
+; CHECK-NEXT:    [[TMP2:%.*]] = extractelement <8 x ptr> [[TMP0]], i32 0
+; CHECK-NEXT:    [[TMP3:%.*]] = getelementptr i32, ptr [[TMP2]], i32 0
+; CHECK-NEXT:    [[TMP4:%.*]] = extractelement <8 x ptr> [[TMP1]], i32 0
+; CHECK-NEXT:    [[TMP5:%.*]] = getelementptr i32, ptr [[TMP4]], i32 0
+; CHECK-NEXT:    [[WIDE_VEC:%.*]] = load <16 x i32>, ptr [[TMP3]], align 4
+; CHECK-NEXT:    [[WIDE_VEC2:%.*]] = load <16 x i32>, ptr [[TMP5]], align 4
 ; CHECK-NEXT:    [[STRIDED_VEC:%.*]] = shufflevector <16 x i32> [[WIDE_VEC]], <16 x i32> poison, <8 x i32> <i32 0, i32 2, i32 4, i32 6, i32 8, i32 10, i32 12, i32 14>
-; CHECK-NEXT:    [[TMP3:%.*]] = add <8 x i32> [[STRIDED_VEC]], <i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1>
-; CHECK-NEXT:    call void @llvm.masked.scatter.v8i32.v8p0(<8 x i32> [[TMP3]], <8 x ptr> [[TMP0]], i32 4, <8 x i1> <i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true>)
-; CHECK-NEXT:    [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 8
-; CHECK-NEXT:    [[PTR_IND]] = getelementptr i8, ptr [[POINTER_PHI]], i64 64
-; CHECK-NEXT:    [[TMP4:%.*]] = icmp eq i64 [[INDEX_NEXT]], 1016
-; CHECK-NEXT:    br i1 [[TMP4]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP6:![0-9]+]]
+; CHECK-NEXT:    [[STRIDED_VEC3:%.*]] = shufflevector <16 x i32> [[WIDE_VEC2]], <16 x i32> poison, <8 x i32> <i32 0, i32 2, i32 4, i32 6, i32 8, i32 10, i32 12, i32 14>
+; CHECK-NEXT:    [[TMP6:%.*]] = add <8 x i32> [[STRIDED_VEC]], <i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1>
+; CHECK-NEXT:    [[TMP7:%.*]] = add <8 x i32> [[STRIDED_VEC3]], <i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1>
+; CHECK-NEXT:    call void @llvm.masked.scatter.v8i32.v8p0(<8 x i32> [[TMP6]], <8 x ptr> [[TMP0]], i32 4, <8 x i1> <i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true>)
+; CHECK-NEXT:    call void @llvm.masked.scatter.v8i32.v8p0(<8 x i32> [[TMP7]], <8 x ptr> [[TMP1]], i32 4, <8 x i1> <i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true>)
+; CHECK-NEXT:    [[INDEX_NEXT]] = add nuw i64 [[INDEX]], 16
+; CHECK-NEXT:    [[PTR_IND]] = getelementptr i8, ptr [[POINTER_PHI]], i64 128
+; CHECK-NEXT:    [[TMP8:%.*]] = icmp eq i64 [[INDEX_NEXT]], 1008
+; CHECK-NEXT:    br i1 [[TMP8]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop [[LOOP6:![0-9]+]]
 ; CHECK:       middle.block:
 ; CHECK-NEXT:    br label [[SCALAR_PH]]
 ; CHECK:       scalar.ph:
-; CHECK-NEXT:    [[BC_RESUME_VAL:%.*]] = phi i64 [ 1016, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]
+; CHECK-NEXT:    [[BC_RESUME_VAL:%.*]] = phi i64 [ 1008, [[MIDDLE_BLOCK]] ], [ 0, [[ENTRY:%.*]] ]
 ; CHECK-NEXT:    [[BC_RESUME_VAL1:%.*]] = phi ptr [ [[IND_END]], [[MIDDLE_BLOCK]] ], [ [[P]], [[ENTRY]] ]
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[I:%.*]] = phi i64 [ [[BC_RESUME_VAL]], [[SCALAR_PH]] ], [ [[NEXTI:%.*]], [[LOOP]] ]
 ; CHECK-NEXT:    [[PTR:%.*]] = phi ptr [ [[BC_RESUME_VAL1]], [[SCALAR_PH]] ], [ [[PTR_NEXT:%.*]], [[LOOP]] ]
 ; CHECK-NEXT:    [[X0:%.*]] = load i32, ptr [[PTR]], align 4
 ; CHECK-NEXT:    [[Y0:%.*]] = add i32 [[X0]], 1
 ; CHECK-NEXT:    store i32 [[Y0]], ptr [[PTR]], align 4