[LoadStoreVectorizer] Only upgrade align for alloca

[bjope]

Summary

In commit 2be0abb7fe72ed453 (D149893) the load store vectorized was
reimplemented. One thing that can happen with the new LSV is that
it can increase the align of alloca and global objects. However,
the code comments indicate that the intention only was to increase
alignment of alloca.
This patch is using stripInBoundsOffsets to analyse if the load/store
actually is accessing an alloca. And then we only try to change the
align if we find an alloca instruction. This way the actual code will
match better with code comments (and we won't see that alignment is
changed to 4 bytes, for global variables, just because the load
store vectorizer assumes that it is fine for align stack variables
at a four byte boundary).

[tra]

Can you think of a test that would directly demonstrate effect of the patch on the alignment?

I think the current test does it indirectly. We manage to vectorize loads/stores when we promote the alignment (though the test cases are inexplicably labeled UNALIGNED), and do it per-element otherwise.
At the very least, we should add a comment there explaining what the vectorization or lack of it indicates.

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/adjust-alloca-alignment.ll

291–299

Nit: I'd suggest keeping the common CHECK-LABEL, instead of replicating it for aligned/unaligned variants.

[bjope]

Add a test case that explicitly verify that the global variables alignment isn't updated.

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/adjust-alloca-alignment.ll

291–299

I only used the update script to re-generated the checks, and this is what I got.

If this test file shouldn't be based on automatically generated test checks, then I think you need to remove the first line saying

; NOTE: Assertions have been autogenerated by utils/update_test_checks.py

Otherwise people will end up updating this, e.g. when using the -u flag in utils/update_test_checks.py which afaik triggers on that line.

[bjope]

In D152386#4404088, @tra wrote:

Can you think of a test that would directly demonstrate effect of the patch on the alignment?

I added an explicit check it in a separate test file. Maybe it can be moved into the adjust-alloca-alignment.ll file, if using something else instead of --match-full-lines to ensure that there isn't anything else after the "align 1" (such as "align 16").

[bjope]

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/dont-adjust-globalobj-alignment.ll

7

I'll change this to [8 x i16] or similar. A bit ugly to use i32 here and then store using i16.

[bjope]

Update the array definition in the new test case.

[jlebar]

Looks reasonable to me. Thank you, and sorry for the breakage.

llvm/lib/Transforms/Vectorize/LoadStoreVectorizer.cpp

805

getOrEnforceKnownAlignment will merely stripPointerCasts, which is weaker than stripInBoundsOffsets. Presumably we should do the same here?

[bjope]

llvm/lib/Transforms/Vectorize/LoadStoreVectorizer.cpp

805

I agree that it makes sense to use the same kind of strip here.

When using stripPointerCasts I got some more diffs in the @load_alloca16_unknown_offset_align1_i32 test case, so then I changed this to use the stronger stripInBoundsOffsets to make the impact of my fixup smaller.

There is however already a FIXME for that specific test:

; FIXME: Although the offset is unknown here, we know it is a multiple
; of the element size, so should still be align 4

So I think that we will get back to a state when that FIXME makes sense (when the test show an opportunity to improve something since we do not vectorize the ALIGNED case any longer).

[bjope]

Addressed review feedback (now using stripPointerCasts instead of stripInBoundsOffsets). Only the @load_alloca16_unknown_offset_align1_i32 test case that was impacted.

[bjope]

llvm/lib/Transforms/Vectorize/LoadStoreVectorizer.cpp

805

Sorry, that FIXME was about @load_unknown_offset_align1_i32 (so not the same test case).

Anyway, I've changed to stripPointerCasts, and then @load_alloca16_unknown_offset_align1_i32 was impacted as it no longer vectorize the ALIGNED case. I don't know if that test case is a bit contrived anyway (why would it use align 1 on those i32 loads in the first place).

[arsenm]

llvm/lib/Transforms/Vectorize/LoadStoreVectorizer.cpp

805

Defer the strip pointer casts until after the cheaper getAllocAddrSpace checks?

[bjope]

llvm/lib/Transforms/Vectorize/LoadStoreVectorizer.cpp

805

Defer the strip pointer casts until after the cheaper getAllocAddrSpace checks?

Right, I can fix that!

Although I'm not sure if we really need to check the getAllocaAddrSpace now when we check that it is an alloca. But if doing that check first, then it at least is possible to avoid the strip pointer cast for targets with multiple address spaces.

llvm/lib/Transforms/Vectorize/LoadStoreVectorizer.cpp

for (auto It = CandidateChains.rbegin(), End = CandidateChains.rend();
       //
       // FIXME: We eagerly upgrade the alignment, regardless of whether TTI
       // tells us this is beneficial.  This feels a bit odd, but it matches
       // existing tests.  This isn't *so* bad, because at most we align to 4
       // bytes (current value of StackAdjustedAlignment).
       //
       // FIXME: We will upgrade the alignment of the alloca even if it turns out
       // we can't vectorize for some other reason.
+      Value *PtrOperand = getLoadStorePointerOperand(C[CBegin].Inst);
+      bool IsAllocaAccess = isa<AllocaInst>(PtrOperand->stripPointerCasts());
       Align Alignment = getLoadStoreAlignment(C[CBegin].Inst);
-      if (AS == DL.getAllocaAddrSpace() && Alignment.value() % SizeBytes != 0 &&
-          IsAllowedAndFast(Align(StackAdjustedAlignment))) {
+      Align PrefAlign = Align(StackAdjustedAlignment);
+      if (IsAllocaAccess && AS == DL.getAllocaAddrSpace() &&
+          Alignment.value() % SizeBytes != 0 && IsAllowedAndFast(PrefAlign)) {
         Align NewAlign = getOrEnforceKnownAlignment(
-            getLoadStorePointerOperand(C[CBegin].Inst),
-            Align(StackAdjustedAlignment), DL, C[CBegin].Inst, nullptr, &DT);
+            PtrOperand, PrefAlign, DL, C[CBegin].Inst, nullptr, &DT);
         if (NewAlign >= Alignment) {
           LLVM_DEBUG(dbgs()
                      << "LSV: splitByChain upgrading alloca alignment from "
                      << Alignment.value() << " to " << NewAlign.value()
                      << "\n");
           Alignment = NewAlign;
         }
       }

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/adjust-alloca-alignment.ll

 ; ALIGNED-NEXT:    [[VAL1:%.*]] = load i8, ptr addrspace(5) [[PTR1]], align 1
 ; ALIGNED-NEXT:    [[ADD:%.*]] = add i8 [[VAL0]], [[VAL1]]
 ; ALIGNED-NEXT:    store i8 [[ADD]], ptr addrspace(1) [[OUT:%.*]], align 1
 ; ALIGNED-NEXT:    ret void
 ;
 ; UNALIGNED-LABEL: @load_unknown_offset_align1_i8(
 ; UNALIGNED-NEXT:    [[ALLOCA:%.*]] = alloca [128 x i8], align 1, addrspace(5)
 ; UNALIGNED-NEXT:    [[PTR0:%.*]] = getelementptr inbounds [128 x i8], ptr addrspace(5) [[ALLOCA]], i32 0, i32 [[OFFSET:%.*]]
-; UNALIGNED-NEXT:    [[TMP2:%.*]] = load <2 x i8>, ptr addrspace(5) [[PTR0]], align 1
-; UNALIGNED-NEXT:    [[VAL01:%.*]] = extractelement <2 x i8> [[TMP2]], i32 0
-; UNALIGNED-NEXT:    [[VAL12:%.*]] = extractelement <2 x i8> [[TMP2]], i32 1
+; UNALIGNED-NEXT:    [[TMP1:%.*]] = load <2 x i8>, ptr addrspace(5) [[PTR0]], align 1
+; UNALIGNED-NEXT:    [[VAL01:%.*]] = extractelement <2 x i8> [[TMP1]], i32 0
+; UNALIGNED-NEXT:    [[VAL12:%.*]] = extractelement <2 x i8> [[TMP1]], i32 1
 ; UNALIGNED-NEXT:    [[ADD:%.*]] = add i8 [[VAL01]], [[VAL12]]
 ; UNALIGNED-NEXT:    store i8 [[ADD]], ptr addrspace(1) [[OUT:%.*]], align 1
 ; UNALIGNED-NEXT:    ret void
 ;
   %alloca = alloca [128 x i8], align 1, addrspace(5)
   %ptr0 = getelementptr inbounds [128 x i8], ptr addrspace(5) %alloca, i32 0, i32 %offset
   %val0 = load i8, ptr addrspace(5) %ptr0, align 1
   %ptr1 = getelementptr inbounds i8, ptr addrspace(5) %ptr0, i32 1

 ; ALIGNED-NEXT:    [[VAL1:%.*]] = load i16, ptr addrspace(5) [[PTR1]], align 1
 ; ALIGNED-NEXT:    [[ADD:%.*]] = add i16 [[VAL0]], [[VAL1]]
 ; ALIGNED-NEXT:    store i16 [[ADD]], ptr addrspace(1) [[OUT:%.*]], align 2
 ; ALIGNED-NEXT:    ret void
 ;
 ; UNALIGNED-LABEL: @load_unknown_offset_align1_i16(
 ; UNALIGNED-NEXT:    [[ALLOCA:%.*]] = alloca [128 x i16], align 1, addrspace(5)
 ; UNALIGNED-NEXT:    [[PTR0:%.*]] = getelementptr inbounds [128 x i16], ptr addrspace(5) [[ALLOCA]], i32 0, i32 [[OFFSET:%.*]]
-; UNALIGNED-NEXT:    [[TMP2:%.*]] = load <2 x i16>, ptr addrspace(5) [[PTR0]], align 1
-; UNALIGNED-NEXT:    [[VAL01:%.*]] = extractelement <2 x i16> [[TMP2]], i32 0
-; UNALIGNED-NEXT:    [[VAL12:%.*]] = extractelement <2 x i16> [[TMP2]], i32 1
+; UNALIGNED-NEXT:    [[TMP1:%.*]] = load <2 x i16>, ptr addrspace(5) [[PTR0]], align 1
+; UNALIGNED-NEXT:    [[VAL01:%.*]] = extractelement <2 x i16> [[TMP1]], i32 0
+; UNALIGNED-NEXT:    [[VAL12:%.*]] = extractelement <2 x i16> [[TMP1]], i32 1
 ; UNALIGNED-NEXT:    [[ADD:%.*]] = add i16 [[VAL01]], [[VAL12]]
 ; UNALIGNED-NEXT:    store i16 [[ADD]], ptr addrspace(1) [[OUT:%.*]], align 2
 ; UNALIGNED-NEXT:    ret void
 ;
   %alloca = alloca [128 x i16], align 1, addrspace(5)
   %ptr0 = getelementptr inbounds [128 x i16], ptr addrspace(5) %alloca, i32 0, i32 %offset
   %val0 = load i16, ptr addrspace(5) %ptr0, align 1
   %ptr1 = getelementptr inbounds i16, ptr addrspace(5) %ptr0, i32 1

 ; ALIGNED-NEXT:    [[VAL1:%.*]] = load i32, ptr addrspace(5) [[PTR1]], align 1
 ; ALIGNED-NEXT:    [[ADD:%.*]] = add i32 [[VAL0]], [[VAL1]]
 ; ALIGNED-NEXT:    store i32 [[ADD]], ptr addrspace(1) [[OUT:%.*]], align 4
 ; ALIGNED-NEXT:    ret void
 ;
 ; UNALIGNED-LABEL: @load_unknown_offset_align1_i32(
 ; UNALIGNED-NEXT:    [[ALLOCA:%.*]] = alloca [128 x i32], align 1, addrspace(5)
 ; UNALIGNED-NEXT:    [[PTR0:%.*]] = getelementptr inbounds [128 x i32], ptr addrspace(5) [[ALLOCA]], i32 0, i32 [[OFFSET:%.*]]
-; UNALIGNED-NEXT:    [[TMP2:%.*]] = load <2 x i32>, ptr addrspace(5) [[PTR0]], align 1
-; UNALIGNED-NEXT:    [[VAL01:%.*]] = extractelement <2 x i32> [[TMP2]], i32 0
-; UNALIGNED-NEXT:    [[VAL12:%.*]] = extractelement <2 x i32> [[TMP2]], i32 1
+; UNALIGNED-NEXT:    [[TMP1:%.*]] = load <2 x i32>, ptr addrspace(5) [[PTR0]], align 1
+; UNALIGNED-NEXT:    [[VAL01:%.*]] = extractelement <2 x i32> [[TMP1]], i32 0
+; UNALIGNED-NEXT:    [[VAL12:%.*]] = extractelement <2 x i32> [[TMP1]], i32 1
 ; UNALIGNED-NEXT:    [[ADD:%.*]] = add i32 [[VAL01]], [[VAL12]]
 ; UNALIGNED-NEXT:    store i32 [[ADD]], ptr addrspace(1) [[OUT:%.*]], align 4
 ; UNALIGNED-NEXT:    ret void
 ;
   %alloca = alloca [128 x i32], align 1, addrspace(5)
   %ptr0 = getelementptr inbounds [128 x i32], ptr addrspace(5) %alloca, i32 0, i32 %offset
   %val0 = load i32, ptr addrspace(5) %ptr0, align 1
   %ptr1 = getelementptr inbounds i32, ptr addrspace(5) %ptr0, i32 1
   %val1 = load i32, ptr addrspace(5) %ptr1, align 1
   %add = add i32 %val0, %val1
   store i32 %add, ptr addrspace(1) %out
   ret void
 }
 
 ; Make sure alloca alignment isn't decreased
 define amdgpu_kernel void @load_alloca16_unknown_offset_align1_i32(ptr addrspace(1) noalias %out, i32 %offset) #0 {
-; CHECK-LABEL: @load_alloca16_unknown_offset_align1_i32(
-; CHECK-NEXT:    [[ALLOCA:%.*]] = alloca [128 x i32], align 16, addrspace(5)
-; CHECK-NEXT:    [[PTR0:%.*]] = getelementptr inbounds [128 x i32], ptr addrspace(5) [[ALLOCA]], i32 0, i32 [[OFFSET:%.*]]
-; CHECK-NEXT:    [[TMP2:%.*]] = load <2 x i32>, ptr addrspace(5) [[PTR0]], align 4
-; CHECK-NEXT:    [[VAL01:%.*]] = extractelement <2 x i32> [[TMP2]], i32 0
-; CHECK-NEXT:    [[VAL12:%.*]] = extractelement <2 x i32> [[TMP2]], i32 1
-; CHECK-NEXT:    [[ADD:%.*]] = add i32 [[VAL01]], [[VAL12]]
-; CHECK-NEXT:    store i32 [[ADD]], ptr addrspace(1) [[OUT:%.*]], align 4
-; CHECK-NEXT:    ret void
+; ALIGNED-LABEL: @load_alloca16_unknown_offset_align1_i32(
+; ALIGNED-NEXT:    [[ALLOCA:%.*]] = alloca [128 x i32], align 16, addrspace(5)
+; ALIGNED-NEXT:    [[PTR0:%.*]] = getelementptr inbounds [128 x i32], ptr addrspace(5) [[ALLOCA]], i32 0, i32 [[OFFSET:%.*]]
+; ALIGNED-NEXT:    [[VAL0:%.*]] = load i32, ptr addrspace(5) [[PTR0]], align 1
+; ALIGNED-NEXT:    [[PTR1:%.*]] = getelementptr inbounds i32, ptr addrspace(5) [[PTR0]], i32 1
+; ALIGNED-NEXT:    [[VAL1:%.*]] = load i32, ptr addrspace(5) [[PTR1]], align 1
+; ALIGNED-NEXT:    [[ADD:%.*]] = add i32 [[VAL0]], [[VAL1]]
+; ALIGNED-NEXT:    store i32 [[ADD]], ptr addrspace(1) [[OUT:%.*]], align 4
+; ALIGNED-NEXT:    ret void
+;
+; UNALIGNED-LABEL: @load_alloca16_unknown_offset_align1_i32(
+; UNALIGNED-NEXT:    [[ALLOCA:%.*]] = alloca [128 x i32], align 16, addrspace(5)
+; UNALIGNED-NEXT:    [[PTR0:%.*]] = getelementptr inbounds [128 x i32], ptr addrspace(5) [[ALLOCA]], i32 0, i32 [[OFFSET:%.*]]
+; UNALIGNED-NEXT:    [[TMP1:%.*]] = load <2 x i32>, ptr addrspace(5) [[PTR0]], align 4
+; UNALIGNED-NEXT:    [[VAL01:%.*]] = extractelement <2 x i32> [[TMP1]], i32 0
+; UNALIGNED-NEXT:    [[VAL12:%.*]] = extractelement <2 x i32> [[TMP1]], i32 1
+; UNALIGNED-NEXT:    [[ADD:%.*]] = add i32 [[VAL01]], [[VAL12]]
+; UNALIGNED-NEXT:    store i32 [[ADD]], ptr addrspace(1) [[OUT:%.*]], align 4
+; UNALIGNED-NEXT:    ret void
 ;
   %alloca = alloca [128 x i32], align 16, addrspace(5)
   %ptr0 = getelementptr inbounds [128 x i32], ptr addrspace(5) %alloca, i32 0, i32 %offset
   %val0 = load i32, ptr addrspace(5) %ptr0, align 1
   %ptr1 = getelementptr inbounds i32, ptr addrspace(5) %ptr0, i32 1
   %val1 = load i32, ptr addrspace(5) %ptr1, align 1
   %add = add i32 %val0, %val1
   store i32 %add, ptr addrspace(1) %out
;
   store i8 23, ptr addrspace(5) %out.gep.2, align 1
   store i8 19, ptr addrspace(5) %out.gep.3, align 1
   ret void
 }
 
 define amdgpu_kernel void @merge_private_load_4_vector_elts_loads_v4i32() {
 ; CHECK-LABEL: @merge_private_load_4_vector_elts_loads_v4i32(
 ; CHECK-NEXT:    [[ALLOCA:%.*]] = alloca [8 x i32], align 4, addrspace(5)
-; CHECK-NEXT:    [[TMP2:%.*]] = load <4 x i32>, ptr addrspace(5) [[ALLOCA]], align 4
-; CHECK-NEXT:    [[LOAD01:%.*]] = extractelement <4 x i32> [[TMP2]], i32 0
-; CHECK-NEXT:    [[LOAD12:%.*]] = extractelement <4 x i32> [[TMP2]], i32 1
-; CHECK-NEXT:    [[LOAD23:%.*]] = extractelement <4 x i32> [[TMP2]], i32 2
-; CHECK-NEXT:    [[LOAD34:%.*]] = extractelement <4 x i32> [[TMP2]], i32 3
+; CHECK-NEXT:    [[TMP1:%.*]] = load <4 x i32>, ptr addrspace(5) [[ALLOCA]], align 4
+; CHECK-NEXT:    [[LOAD01:%.*]] = extractelement <4 x i32> [[TMP1]], i32 0
+; CHECK-NEXT:    [[LOAD12:%.*]] = extractelement <4 x i32> [[TMP1]], i32 1
+; CHECK-NEXT:    [[LOAD23:%.*]] = extractelement <4 x i32> [[TMP1]], i32 2
+; CHECK-NEXT:    [[LOAD34:%.*]] = extractelement <4 x i32> [[TMP1]], i32 3
 ; CHECK-NEXT:    ret void
 ;
   %alloca = alloca [8 x i32], align 1, addrspace(5)
   %out.gep.1 = getelementptr i32, ptr addrspace(5) %alloca, i32 1
   %out.gep.2 = getelementptr i32, ptr addrspace(5) %alloca, i32 2
   %out.gep.3 = getelementptr i32, ptr addrspace(5) %alloca, i32 3
 
   %load0 = load i32, ptr addrspace(5) %alloca, align 1
   %load1 = load i32, ptr addrspace(5) %out.gep.1, align 1
   %load2 = load i32, ptr addrspace(5) %out.gep.2, align 1
   %load3 = load i32, ptr addrspace(5) %out.gep.3, align 1
   ret void
 }
 
 define amdgpu_kernel void @merge_private_load_4_vector_elts_loads_v4i8() {
 ; CHECK-LABEL: @merge_private_load_4_vector_elts_loads_v4i8(
 ; CHECK-NEXT:    [[ALLOCA:%.*]] = alloca [8 x i8], align 4, addrspace(5)
-; CHECK-NEXT:    [[TMP2:%.*]] = load <4 x i8>, ptr addrspace(5) [[ALLOCA]], align 4
-; CHECK-NEXT:    [[LOAD01:%.*]] = extractelement <4 x i8> [[TMP2]], i32 0
-; CHECK-NEXT:    [[LOAD12:%.*]] = extractelement <4 x i8> [[TMP2]], i32 1
-; CHECK-NEXT:    [[LOAD23:%.*]] = extractelement <4 x i8> [[TMP2]], i32 2
-; CHECK-NEXT:    [[LOAD34:%.*]] = extractelement <4 x i8> [[TMP2]], i32 3
+; CHECK-NEXT:    [[TMP1:%.*]] = load <4 x i8>, ptr addrspace(5) [[ALLOCA]], align 4
+; CHECK-NEXT:    [[LOAD01:%.*]] = extractelement <4 x i8> [[TMP1]], i32 0
+; CHECK-NEXT:    [[LOAD12:%.*]] = extractelement <4 x i8> [[TMP1]], i32 1
+; CHECK-NEXT:    [[LOAD23:%.*]] = extractelement <4 x i8> [[TMP1]], i32 2
+; CHECK-NEXT:    [[LOAD34:%.*]] = extractelement <4 x i8> [[TMP1]], i32 3
 ; CHECK-NEXT:    ret void
 ;
   %alloca = alloca [8 x i8], align 1, addrspace(5)
   %out.gep.1 = getelementptr i8, ptr addrspace(5) %alloca, i8 1
   %out.gep.2 = getelementptr i8, ptr addrspace(5) %alloca, i8 2
   %out.gep.3 = getelementptr i8, ptr addrspace(5) %alloca, i8 3
 
   %load0 = load i8, ptr addrspace(5) %alloca, align 1
   %load1 = load i8, ptr addrspace(5) %out.gep.1, align 1
   %load2 = load i8, ptr addrspace(5) %out.gep.2, align 1
   %load3 = load i8, ptr addrspace(5) %out.gep.3, align 1
   ret void
 }
 
 ; Make sure we don't think the alignment will increase if the base address isn't an alloca
 define void @private_store_2xi16_align2_not_alloca(ptr addrspace(5) %p, ptr addrspace(5) %r) #0 {
-; CHECK-LABEL: @private_store_2xi16_align2_not_alloca(
-; ALIGNED-NEXT:  [[GEP_R:%.*]] = getelementptr i16, ptr addrspace(5) [[R:%.*]], i32 1
-; ALIGNED-NEXT:  store i16 1, ptr addrspace(5) [[R]], align 2
-; ALIGNED-NEXT:  store i16 2, ptr addrspace(5) [[GEP_R]], align 2
-; UNALIGNED-NEXT:store <2 x i16>
-; CHECK-NEXT:    ret void
+; ALIGNED-LABEL: @private_store_2xi16_align2_not_alloca(
+; ALIGNED-NEXT:    [[GEP_R:%.*]] = getelementptr i16, ptr addrspace(5) [[R:%.*]], i32 1
+; ALIGNED-NEXT:    store i16 1, ptr addrspace(5) [[R]], align 2
+; ALIGNED-NEXT:    store i16 2, ptr addrspace(5) [[GEP_R]], align 2
+; ALIGNED-NEXT:    ret void
+;
+; UNALIGNED-LABEL: @private_store_2xi16_align2_not_alloca(
+; UNALIGNED-NEXT:    store <2 x i16> <i16 1, i16 2>, ptr addrspace(5) [[R:%.*]], align 2
+; UNALIGNED-NEXT:    ret void
 ;
   %gep.r = getelementptr i16, ptr addrspace(5) %r, i32 1
   store i16 1, ptr addrspace(5) %r, align 2
   store i16 2, ptr addrspace(5) %gep.r, align 2
   ret void
 }
 
 define void @private_store_2xi16_align1_not_alloca(ptr addrspace(5) %p, ptr addrspace(5) %r) #0 {

 ;
   %gep.r = getelementptr i16, ptr addrspace(5) %r, i32 1
   store i16 1, ptr addrspace(5) %r, align 1
   store i16 2, ptr addrspace(5) %gep.r, align 1
   ret void
 }
 
 define i32 @private_load_2xi16_align2_not_alloca(ptr addrspace(5) %p) #0 {
-; CHECK-LABEL: @private_load_2xi16_align2_not_alloca(
-; ALIGNED-NEXT:  [[GEP_P:%.*]] = getelementptr i16, ptr addrspace(5) [[P:%.*]], i64 1
-; ALIGNED-NEXT:  [[P_0:%.*]] = load i16, ptr addrspace(5) [[P]], align 2
-; ALIGNED-NEXT:  [[P_1:%.*]] = load i16, ptr addrspace(5) [[GEP_P]], align 2
-; UNALIGNED-NEXT:load <2 x i16>
-; CHECK:         [[ZEXT_0:%.*]] = zext i16
-; CHECK-NEXT:    [[ZEXT_1:%.*]] = zext i16
-; CHECK-NEXT:    [[SHL_1:%.*]] = shl i32 [[ZEXT_1]], 16
-; CHECK-NEXT:    [[OR:%.*]] = or i32 [[ZEXT_0]], [[SHL_1]]
-; CHECK-NEXT:    ret i32 [[OR]]
+; ALIGNED-LABEL: @private_load_2xi16_align2_not_alloca(
+; ALIGNED-NEXT:    [[GEP_P:%.*]] = getelementptr i16, ptr addrspace(5) [[P:%.*]], i64 1
+; ALIGNED-NEXT:    [[P_0:%.*]] = load i16, ptr addrspace(5) [[P]], align 2
+; ALIGNED-NEXT:    [[P_1:%.*]] = load i16, ptr addrspace(5) [[GEP_P]], align 2
+; ALIGNED-NEXT:    [[ZEXT_0:%.*]] = zext i16 [[P_0]] to i32
+; ALIGNED-NEXT:    [[ZEXT_1:%.*]] = zext i16 [[P_1]] to i32
+; ALIGNED-NEXT:    [[SHL_1:%.*]] = shl i32 [[ZEXT_1]], 16
+; ALIGNED-NEXT:    [[OR:%.*]] = or i32 [[ZEXT_0]], [[SHL_1]]
+; ALIGNED-NEXT:    ret i32 [[OR]]
+;
+; UNALIGNED-LABEL: @private_load_2xi16_align2_not_alloca(
+; UNALIGNED-NEXT:    [[TMP1:%.*]] = load <2 x i16>, ptr addrspace(5) [[P:%.*]], align 2
+; UNALIGNED-NEXT:    [[P_01:%.*]] = extractelement <2 x i16> [[TMP1]], i32 0
+; UNALIGNED-NEXT:    [[P_12:%.*]] = extractelement <2 x i16> [[TMP1]], i32 1
+; UNALIGNED-NEXT:    [[ZEXT_0:%.*]] = zext i16 [[P_01]] to i32
+; UNALIGNED-NEXT:    [[ZEXT_1:%.*]] = zext i16 [[P_12]] to i32
+; UNALIGNED-NEXT:    [[SHL_1:%.*]] = shl i32 [[ZEXT_1]], 16
+; UNALIGNED-NEXT:    [[OR:%.*]] = or i32 [[ZEXT_0]], [[SHL_1]]
+; UNALIGNED-NEXT:    ret i32 [[OR]]
 ;
   %gep.p = getelementptr i16, ptr addrspace(5) %p, i64 1
   %p.0 = load i16, ptr addrspace(5) %p, align 2
   %p.1 = load i16, ptr addrspace(5) %gep.p, align 2
   %zext.0 = zext i16 %p.0 to i32
   %zext.1 = zext i16 %p.1 to i32
   %shl.1 = shl i32 %zext.1, 16
   %or = or i32 %zext.0, %shl.1
   ret i32 %or
 }
 
 define i32 @private_load_2xi16_align1_not_alloca(ptr addrspace(5) %p) #0 {
 ; ALIGNED-LABEL: @private_load_2xi16_align1_not_alloca(
 ; ALIGNED-NEXT:    [[GEP_P:%.*]] = getelementptr i16, ptr addrspace(5) [[P:%.*]], i64 1
 ; ALIGNED-NEXT:    [[P_0:%.*]] = load i16, ptr addrspace(5) [[P]], align 1
 ; ALIGNED-NEXT:    [[P_1:%.*]] = load i16, ptr addrspace(5) [[GEP_P]], align 1
 ; ALIGNED-NEXT:    [[ZEXT_0:%.*]] = zext i16 [[P_0]] to i32
 ; ALIGNED-NEXT:    [[ZEXT_1:%.*]] = zext i16 [[P_1]] to i32
 ; ALIGNED-NEXT:    [[SHL_1:%.*]] = shl i32 [[ZEXT_1]], 16
 ; ALIGNED-NEXT:    [[OR:%.*]] = or i32 [[ZEXT_0]], [[SHL_1]]
 ; ALIGNED-NEXT:    ret i32 [[OR]]
 ;
 ; UNALIGNED-LABEL: @private_load_2xi16_align1_not_alloca(
-; UNALIGNED-NEXT:    [[TMP2:%.*]] = load <2 x i16>, ptr addrspace(5) [[P:%.*]], align 1
-; UNALIGNED-NEXT:    [[P_01:%.*]] = extractelement <2 x i16> [[TMP2]], i32 0
-; UNALIGNED-NEXT:    [[P_12:%.*]] = extractelement <2 x i16> [[TMP2]], i32 1
+; UNALIGNED-NEXT:    [[TMP1:%.*]] = load <2 x i16>, ptr addrspace(5) [[P:%.*]], align 1
+; UNALIGNED-NEXT:    [[P_01:%.*]] = extractelement <2 x i16> [[TMP1]], i32 0
+; UNALIGNED-NEXT:    [[P_12:%.*]] = extractelement <2 x i16> [[TMP1]], i32 1
 ; UNALIGNED-NEXT:    [[ZEXT_0:%.*]] = zext i16 [[P_01]] to i32
 ; UNALIGNED-NEXT:    [[ZEXT_1:%.*]] = zext i16 [[P_12]] to i32
 ; UNALIGNED-NEXT:    [[SHL_1:%.*]] = shl i32 [[ZEXT_1]], 16
 ; UNALIGNED-NEXT:    [[OR:%.*]] = or i32 [[ZEXT_0]], [[SHL_1]]
 ; UNALIGNED-NEXT:    ret i32 [[OR]]
 ;
   %gep.p = getelementptr i16, ptr addrspace(5) %p, i64 1
   %p.0 = load i16, ptr addrspace(5) %p, align 1

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/dont-adjust-globalobj-alignment.ll

+; RUN: opt -S -passes=load-store-vectorizer --mcpu=hawaii -mattr=+unaligned-access-mode,+unaligned-scratch-access,+max-private-element-size-16 < %s | FileCheck --match-full-lines %s
+
+target triple = "amdgcn--"
+target datalayout = "e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-p7:160:256:256:32-p8:128:128-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5"
+
+@G = internal addrspace(5) global [8 x i16] undef, align 1
+
+; Verify that the alignment of the global remains at 1, even if we vectorize
+; the stores.
+;
+; CHECK: @G = internal addrspace(5) global [8 x i16] undef, align 1
+
+define void @private_store_2xi16_align2_not_alloca(ptr addrspace(5) %p, ptr addrspace(5) %r) {
+; CHECK: define void @private_store_2xi16_align2_not_alloca(ptr addrspace(5) [[P:%.*]], ptr addrspace(5) [[R:%.*]]) #0 {
+; CHECK-NEXT:    [[GEP0:%.*]] = getelementptr i16, ptr addrspace(5) @G, i32 0
+; CHECK-NEXT:    store <2 x i16> <i16 1, i16 2>, ptr addrspace(5) [[GEP0]], align 1
+; CHECK-NEXT:    ret void
+;
+  %gep0 = getelementptr i16, ptr addrspace(5) @G, i32 0
+  %gep1 = getelementptr i16, ptr addrspace(5) @G, i32 1
+  store i16 1, ptr addrspace(5) %gep0, align 1
+  store i16 2, ptr addrspace(5) %gep1, align 1
+  ret void
+}