[PowerPC] Select the D-Form load if we know its offset meets the requirement

[steven.zhang]

Summary

I happens to see this issue and this is a simple patch to fix the issue I saw.

The LD/STD likewise instruction are selected only when the alignment in the load/store >= 4 to deal with the case that the offset might not be known(i.e. relocations), which is done by this patch.

commit b09680b0f793254c7985dd42f3ef8ea56a388c10
Author: Hal Finkel <[e-mail]>
Date:   Mon Mar 18 23:00:58 2013 +0000

    Fix PPC unaligned 64-bit loads and stores

    PPC64 supports unaligned loads and stores of 64-bit values, but
    in order to use the r+i forms, the offset must be a multiple of 4.
    Unfortunately, this cannot always be determined by examining the
    immediate itself because it might be available only via a TOC entry.

    In order to get around this issue, we additionally predicate the
    selection of the r+i form on the alignment of the load or store
    (forcing it to be at least 4 in order to select the r+i form).

That means we have to select the X-Form load for %0 = load i64, i64* %arrayidx, align 2 In fact, we can still select the D-Form load if the offset is known. So, we only query the load/store alignment when we don't know if the offset is a multiple of 4.

Though we have the peephole to transform the X-Form load/store to D-Form load/store, and that's why we didn't see too much assembly changed. But it exposes more opportunities for us if we can select it as D-Form as early as possible as some optimization only works for D-Form load.

[nemanjai]

LGTM.
Most of this will be going away soon since we're refactoring how we select memory access instructions, but that's fine.

[jsji]

Although a refactor is coming, we should still avoid introducing more confusion if possible.
Please update the name and comments also. Thanks.

llvm/lib/Target/PowerPC/PPCInstrInfo.td

498

Comments also needs update.

504

The fix has changed the semantic of this PatFrag, so I think we should also rename the PatFrag to avoid confusion.

[steven.zhang]

llvm/lib/Target/PowerPC/PPCInstrInfo.td

498

I will update it.

504

The semantics now is quite the same as quadwOffsetLoad except that we are checking more on the ld/st alignment. So, how about the dwOffsetLoad/nonDwOffsetLoad ? The alignment is hidden inside the interface as it seems to be some workround to me for those TOC relocation offset.

[jsji]

llvm/lib/Target/PowerPC/PPCInstrInfo.td

504

How about DSFormLoad/DSFormStore, nonDSFormLoad/nonDSFormStore or XFormLoad/XFormStore?

[steven.zhang]

Address comments.

llvm/lib/Target/PowerPC/PPCInstrInfo.td

504

I will use the NonDSFormLoad/NonDSFormStore as non ds form doesn't mean it is x form.

llvm/lib/Target/PowerPC/PPCInstrInfo.td

 }], HI16>;
 
 def imm64ZExt32  : Operand<i64>, ImmLeaf<i64, [{
   // imm64ZExt32 predicate - True if the i64 immediate fits in a 32-bit
   // zero extended field.
   return isUInt<32>(Imm);
 }]>;
 
 // Some r+i load/store instructions (such as LD, STD, LDU, etc.) that require
 // restricted memrix (4-aligned) constants are alignment sensitive. If these
 // offsets are hidden behind TOC entries than the values of the lower-order
 // bits cannot be checked directly. As a result, we need to also incorporate
 // an alignment check into the relevant patterns.
 
 def aligned4load : PatFrag<(ops node:$ptr), (load node:$ptr), [{
-  return cast<LoadSDNode>(N)->getAlignment() >= 4;
+  return isOffsetMultipleOf(N, 4) || cast<LoadSDNode>(N)->getAlignment() >= 4;
 }]>;
 def aligned4store : PatFrag<(ops node:$val, node:$ptr),
                             (store node:$val, node:$ptr), [{
-  return cast<StoreSDNode>(N)->getAlignment() >= 4;
+  return isOffsetMultipleOf(N, 4) || cast<StoreSDNode>(N)->getAlignment() >= 4;
 }]>;
 def aligned4sextloadi32 : PatFrag<(ops node:$ptr), (sextloadi32 node:$ptr), [{
-  return cast<LoadSDNode>(N)->getAlignment() >= 4;
+  return isOffsetMultipleOf(N, 4) || cast<LoadSDNode>(N)->getAlignment() >= 4;
 }]>;
 def aligned4pre_store : PatFrag<
                           (ops node:$val, node:$base, node:$offset),
                           (pre_store node:$val, node:$base, node:$offset), [{
-  return cast<StoreSDNode>(N)->getAlignment() >= 4;
+  return isOffsetMultipleOf(N, 4) || cast<StoreSDNode>(N)->getAlignment() >= 4;
 }]>;
 
 def unaligned4load : PatFrag<(ops node:$ptr), (load node:$ptr), [{
-  return cast<LoadSDNode>(N)->getAlignment() < 4;
+  return cast<LoadSDNode>(N)->getAlignment() < 4 && !isOffsetMultipleOf(N, 4);
 }]>;
 def unaligned4store : PatFrag<(ops node:$val, node:$ptr),
                               (store node:$val, node:$ptr), [{
-  return cast<StoreSDNode>(N)->getAlignment() < 4;
+  return cast<StoreSDNode>(N)->getAlignment() < 4 && !isOffsetMultipleOf(N, 4);
 }]>;
 def unaligned4sextloadi32 : PatFrag<(ops node:$ptr), (sextloadi32 node:$ptr), [{
-  return cast<LoadSDNode>(N)->getAlignment() < 4;
+  return cast<LoadSDNode>(N)->getAlignment() < 4 && !isOffsetMultipleOf(N, 4);
 }]>;
 
 // This is a somewhat weaker condition than actually checking for 16-byte
 // alignment. It is simply checking that the displacement can be represented
 // as an immediate that is a multiple of 16 (i.e. the requirements for DQ-Form
 // instructions).
 def quadwOffsetLoad : PatFrag<(ops node:$ptr), (load node:$ptr), [{
   return isOffsetMultipleOf(N, 16);

llvm/test/CodeGen/PowerPC/ldst-align.ll

 ; NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
 ; RUN: llc -mcpu=pwr9 -mtriple=powerpc64le-unknown-linux-gnu < %s \
 ; RUN:   -stop-after=finalize-isel -verify-machineinstrs | FileCheck %s
 define i64 @load(i64* %p) {
   ; CHECK-LABEL: name: load
   ; CHECK: bb.0.entry:
   ; CHECK:   liveins: $x3
   ; CHECK:   [[COPY:%[0-9]+]]:g8rc_and_g8rc_nox0 = COPY $x3
-  ; CHECK:   [[ADDI8_:%[0-9]+]]:g8rc = nuw ADDI8 [[COPY]], 24
-  ; CHECK:   [[LDX:%[0-9]+]]:g8rc = LDX $zero8, killed [[ADDI8_]] :: (load 8 from %ir.arrayidx, align 2)
-  ; CHECK:   $x3 = COPY [[LDX]]
+  ; CHECK:   [[LD:%[0-9]+]]:g8rc = LD 24, [[COPY]] :: (load 8 from %ir.arrayidx, align 2)
+  ; CHECK:   $x3 = COPY [[LD]]
   ; CHECK:   BLR8 implicit $lr8, implicit $rm, implicit $x3
 entry:
   %arrayidx = getelementptr inbounds i64, i64* %p, i64 3
   %0 = load i64, i64* %arrayidx, align 2
   ret i64 %0
 }
 
 define void @store(i64* %p) {
   ; CHECK-LABEL: name: store
   ; CHECK: bb.0.entry:
   ; CHECK:   liveins: $x3
   ; CHECK:   [[COPY:%[0-9]+]]:g8rc_and_g8rc_nox0 = COPY $x3
-  ; CHECK:   [[ADDI8_:%[0-9]+]]:g8rc = nuw ADDI8 [[COPY]], 16
   ; CHECK:   [[LI8_:%[0-9]+]]:g8rc = LI8 9
-  ; CHECK:   STDX killed [[LI8_]], $zero8, killed [[ADDI8_]] :: (store 8 into %ir.arrayidx, align 1)
+  ; CHECK:   STD killed [[LI8_]], 16, [[COPY]] :: (store 8 into %ir.arrayidx, align 1)
   ; CHECK:   BLR8 implicit $lr8, implicit $rm
 entry:
   %arrayidx = getelementptr inbounds i64, i64* %p, i64 2
   store i64 9, i64* %arrayidx, align 1
   ret void
 }
 
 define void @store_aligned(i64* %p) {

llvm/test/CodeGen/PowerPC/memCmpUsedInZeroEqualityComparison.ll

; CHECK-NEXT:    blr
   %. = zext i1 %not.cmp to i32
   ret i32 %.
 }
 
 ; Check 16 bytes - requires 2 loads for each param (or use vectors?).
 define signext i32 @zeroEqualityTest01(i8* %x, i8* %y) {
 ; CHECK-LABEL: zeroEqualityTest01:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    ldx 5, 0, 3
-; CHECK-NEXT:    ldx 6, 0, 4
+; CHECK-NEXT:    ld 5, 0(3)
+; CHECK-NEXT:    ld 6, 0(4)
 ; CHECK-NEXT:    cmpld 5, 6
 ; CHECK-NEXT:    bne 0, .LBB1_2
 ; CHECK-NEXT:  # %bb.1: # %loadbb1
 ; CHECK-NEXT:    ld 3, 8(3)
 ; CHECK-NEXT:    ld 4, 8(4)
 ; CHECK-NEXT:    cmpld 3, 4
 ; CHECK-NEXT:    li 3, 0
 ; CHECK-NEXT:    beq 0, .LBB1_3
; CHECK-NEXT:    blr
   %not.tobool = icmp eq i32 %call, 0
   %cond = zext i1 %not.tobool to i32
   ret i32 %cond
 }
 
 define signext i32 @equalityFoldOneConstant(i8* %X) {
 ; CHECK-LABEL: equalityFoldOneConstant:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    ldx 4, 0, 3
+; CHECK-NEXT:    ld 4, 0(3)
 ; CHECK-NEXT:    li 5, 1
 ; CHECK-NEXT:    sldi 5, 5, 32
 ; CHECK-NEXT:    cmpld 4, 5
 ; CHECK-NEXT:    bne 0, .LBB6_2
 ; CHECK-NEXT:  # %bb.1: # %loadbb1
 ; CHECK-NEXT:    li 4, 3
 ; CHECK-NEXT:    ld 3, 8(3)
 ; CHECK-NEXT:    sldi 4, 4, 32

llvm/test/CodeGen/PowerPC/memcmp-mergeexpand.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -verify-machineinstrs -mcpu=pwr8 -mtriple=powerpc64le-unknown-gnu-linux  < %s | FileCheck %s -check-prefix=PPC64LE
 
 ; This tests interaction between MergeICmp and ExpandMemCmp.
 
 %"struct.std::pair" = type { i32, i32 }
 
 define zeroext i1 @opeq1(
 ; PPC64LE-LABEL: opeq1:
 ; PPC64LE:       # %bb.0: # %"entry+land.rhs.i"
-; PPC64LE-NEXT:    ldx 3, 0, 3
-; PPC64LE-NEXT:    ldx 4, 0, 4
+; PPC64LE-NEXT:    ld 3, 0(3)
+; PPC64LE-NEXT:    ld 4, 0(4)
 ; PPC64LE-NEXT:    xor 3, 3, 4
 ; PPC64LE-NEXT:    cntlzd 3, 3
 ; PPC64LE-NEXT:    rldicl 3, 3, 58, 63
 ; PPC64LE-NEXT:    blr
   %"struct.std::pair"* nocapture readonly dereferenceable(8) %a,
   %"struct.std::pair"* nocapture readonly dereferenceable(8) %b) local_unnamed_addr #0 {
 entry:
   %first.i = getelementptr inbounds %"struct.std::pair", %"struct.std::pair"* %a, i64 0, i32 0

llvm/test/CodeGen/PowerPC/pr45186.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -ppc-asm-full-reg-names -mtriple=powerpc64-- -mattr=+vsx \
 ; RUN:   -verify-machineinstrs < %s | FileCheck %s
 %struct.anon = type { i64, i64 }
 
 @d = local_unnamed_addr global %struct.anon zeroinitializer, align 8
 
 ; Function Attrs: norecurse nounwind readonly
 define i64 @e(i8* nocapture readonly %f) local_unnamed_addr #0 {
 ; CHECK-LABEL: e:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    ldx r3, 0, r3
+; CHECK-NEXT:    ld r3, 0(r3)
 ; CHECK-NEXT:    blr
 entry:
   %0 = load i8, i8* %f, align 1
   %conv = zext i8 %0 to i64
   %shl = shl nuw i64 %conv, 56
   %arrayidx1 = getelementptr inbounds i8, i8* %f, i64 1
   %1 = load i8, i8* %arrayidx1, align 1
   %conv2 = zext i8 %1 to i64

llvm/test/CodeGen/PowerPC/store-combine.ll

 ; p[3] = (m >> 24) & 0xFF;
 ; p[4] = (m >> 32) & 0xFF;
 ; p[5] = (m >> 40) & 0xFF;
 ; p[6] = (m >> 48) & 0xFF;
 ; p[7] = (m >> 56) & 0xFF;
 define void @store_i64_by_i8(i64 %m, i8* %p)  {
 ; CHECK-PPC64LE-LABEL: store_i64_by_i8:
 ; CHECK-PPC64LE:       # %bb.0: # %entry
-; CHECK-PPC64LE-NEXT:    stdx 3, 0, 4
+; CHECK-PPC64LE-NEXT:    std 3, 0(4)
 ; CHECK-PPC64LE-NEXT:    blr
 ;
 ; CHECK-PPC64-LABEL: store_i64_by_i8:
 ; CHECK-PPC64:       # %bb.0: # %entry
 ; CHECK-PPC64-NEXT:    stdbrx 3, 0, 4
 ; CHECK-PPC64-NEXT:    blr
 entry:
   %conv = trunc i64 %m to i8

 define void @store_i64_by_i8_bswap(i64 %m, i8* %p)  {
 ; CHECK-PPC64LE-LABEL: store_i64_by_i8_bswap:
 ; CHECK-PPC64LE:       # %bb.0: # %entry
 ; CHECK-PPC64LE-NEXT:    stdbrx 3, 0, 4
 ; CHECK-PPC64LE-NEXT:    blr
 ;
 ; CHECK-PPC64-LABEL: store_i64_by_i8_bswap:
 ; CHECK-PPC64:       # %bb.0: # %entry
-; CHECK-PPC64-NEXT:    stdx 3, 0, 4
+; CHECK-PPC64-NEXT:    std 3, 0(4)
 ; CHECK-PPC64-NEXT:    blr
 entry:
   %conv = trunc i64 %m to i8
   %arrayidx = getelementptr inbounds i8, i8* %p, i64 7
   store i8 %conv, i8* %arrayidx, align 1
   %0 = lshr i64 %m, 8
   %conv3 = trunc i64 %0 to i8
   %arrayidx4 = getelementptr inbounds i8, i8* %p, i64 6

 ; CHECK-PPC64LE-NEXT:    stdbrx 3, 0, 4
 ; CHECK-PPC64LE-NEXT:    blr
 ;
 ; CHECK-PPC64-LABEL: store_i64_by_i8_bswap_uses:
 ; CHECK-PPC64:       # %bb.0: # %entry
 ; CHECK-PPC64-NEXT:    slwi 5, 3, 3
 ; CHECK-PPC64-NEXT:    sub 3, 5, 3
 ; CHECK-PPC64-NEXT:    extsw 3, 3
-; CHECK-PPC64-NEXT:    stdx 3, 0, 4
+; CHECK-PPC64-NEXT:    std 3, 0(4)
 ; CHECK-PPC64-NEXT:    blr
 entry:
   %mul = mul nsw i32 %t, 7
   %conv = sext i32 %mul to i64
   %conv1 = trunc i32 %mul to i8
   %arrayidx = getelementptr inbounds i8, i8* %p, i64 7
   store i8 %conv1, i8* %arrayidx, align 1
   %0 = lshr i64 %conv, 8

llvm/test/CodeGen/PowerPC/unal4-std.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -verify-machineinstrs < %s -mcpu=pwr7 -mattr=-vsx| FileCheck %s
 ; RUN: llc -verify-machineinstrs < %s -mcpu=pwr7 -mattr=+vsx | FileCheck -check-prefix=CHECK-VSX %s
 target datalayout = "E-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-f128:128:128-v128:128:128-n32:64"
 target triple = "powerpc64-unknown-linux-gnu"
 
 define void @copy_to_conceal(<8 x i16>* %inp) #0 {
 ; CHECK-LABEL: copy_to_conceal:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    vxor 2, 2, 2
 ; CHECK-NEXT:    addi 4, 1, -16
 ; CHECK-NEXT:    stvx 2, 0, 4
 ; CHECK-NEXT:    ld 4, -8(1)
 ; CHECK-NEXT:    std 4, 8(3)
 ; CHECK-NEXT:    ld 4, -16(1)
-; CHECK-NEXT:    stdx 4, 0, 3
+; CHECK-NEXT:    std 4, 0(3)
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-VSX-LABEL: copy_to_conceal:
 ; CHECK-VSX:       # %bb.0: # %entry
 ; CHECK-VSX-NEXT:    xxlxor 0, 0, 0
 ; CHECK-VSX-NEXT:    stxvw4x 0, 0, 3
 ; CHECK-VSX-NEXT:    blr
 entry:

llvm/test/CodeGen/PowerPC/unaligned.ll

entry:
   ret void
 
 
 }
 
 define void @foo3(i64* %p, i64* %r) nounwind {
 ; CHECK-LABEL: foo3:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    ldx 3, 0, 3
-; CHECK-NEXT:    stdx 3, 0, 4
+; CHECK-NEXT:    ld 3, 0(3)
+; CHECK-NEXT:    std 3, 0(4)
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-VSX-LABEL: foo3:
 ; CHECK-VSX:       # %bb.0: # %entry
-; CHECK-VSX-NEXT:    ldx 3, 0, 3
-; CHECK-VSX-NEXT:    stdx 3, 0, 4
+; CHECK-VSX-NEXT:    ld 3, 0(3)
+; CHECK-VSX-NEXT:    std 3, 0(4)
 ; CHECK-VSX-NEXT:    blr
 entry:
   %v = load i64, i64* %p, align 1
   store i64 %v, i64* %r, align 1
   ret void
 
 
 }

 ; CHECK-NEXT:    lvx 2, 3, 5
 ; CHECK-NEXT:    lvx 4, 0, 3
 ; CHECK-NEXT:    addi 3, 1, -16
 ; CHECK-NEXT:    vperm 2, 4, 2, 3
 ; CHECK-NEXT:    stvx 2, 0, 3
 ; CHECK-NEXT:    ld 3, -8(1)
 ; CHECK-NEXT:    std 3, 8(4)
 ; CHECK-NEXT:    ld 3, -16(1)
-; CHECK-NEXT:    stdx 3, 0, 4
+; CHECK-NEXT:    std 3, 0(4)
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-VSX-LABEL: foo6:
 ; CHECK-VSX:       # %bb.0: # %entry
 ; CHECK-VSX-NEXT:    li 5, 15
 ; CHECK-VSX-NEXT:    lvsl 3, 0, 3
 ; CHECK-VSX-NEXT:    lvx 2, 3, 5
 ; CHECK-VSX-NEXT:    lvx 4, 0, 3