[X86][AVX] Fix wrong lowering of v4x64 shuffles into concat_vector plus extract_subvector nodes.

[andreadb]

Summary

This patch fixes a bug in the shuffle lowering logic implemented by function 'lowerV2X128VectorShuffle'.

There are few cases where function 'lowerV2X128VectorShuffle' wrongly expands a shuffle of two v4X64 vectors into a CONCAT_VECTORS of two EXTRACT_SUBVECTOR nodes.
The problematic expansion only occurs when the shuffle mask M has an 'undef' element at position 2, and M is considered equivalent to mask <0,1,4,5>.
In that case only, the algorithm propagates the wrong vector to one of the two new EXTRACT_SUBVECTOR nodes.

Example:

define <4 x double> @test(<4 x double> %A, <4 x double> %B) {
entry:
  %0 = shufflevector <4 x double> %A, <4 x double> %B, <4 x i32> <i32 undef, i32 1, i32 undef, i32 5>
  ret <4 x double> %0
}

Before this patch, llc (-mattr=+avx) generated:

vinsertf128 $1, %xmm0, %ymm0, %ymm0

With this patch, llc correctly generates:

vinsertf128 $1, %xmm1, %ymm0, %ymm0

This bug was originally spotted by Greg Bedwell.

Added test lower-vec-shuffle-bug.ll.

Please let me know if ok to submit.

Thanks,
Andrea

[qcolombet]

Hi Andrea,

Nice catch.

A small comment though.

Cheers,
-Quentin

lib/Target/X86/X86ISelLowering.cpp

9025 ↗

(On Diff #21737)

I think it would be clearer to do something like:
bool UseOnlyV1 = isShuffleEquivalent(V1, V2, Mask, {0, 1, 0, 1});
if (UseOnlyV1 || isShuffleEquivalent(V1, V2, Mask, {0, 1, 4, 5})) {
<...>
SDValue HiV = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, SubVT,

UseOnlyV1 ? V1 : V2, DAG.getIntPtrConstant(0));

<...>
}

[andreadb]

Hi Quentin,

lib/Target/X86/X86ISelLowering.cpp

9025 ↗

(On Diff #21737)

You are right. It would be a better fix in general. Thanks for the suggestion!
I am going to upload a new version of the patch soon.

[andreadb]

Hi Quentin,

Patch updated.
Please let me know if ok to submit.

Thanks again for your time!
-Andrea

[qcolombet]

LGTM.

Thanks Andrea!
-Quentin

[andreadb]

Thanks Quentin!
Committed revision 232179.

llvm/trunk/lib/Target/X86/X86ISelLowering.cpp

static SDValue lowerV2X128VectorShuffle(SDLoc DL, MVT VT, SDValue V1,
   if (SDValue Blend = lowerVectorShuffleAsBlend(DL, VT, V1, V2, Mask,
                                                 Subtarget, DAG))
     return Blend;
 
   MVT SubVT = MVT::getVectorVT(VT.getVectorElementType(),
                                VT.getVectorNumElements() / 2);
   // Check for patterns which can be matched with a single insert of a 128-bit
   // subvector.
-  if (isShuffleEquivalent(V1, V2, Mask, {0, 1, 0, 1}) ||
-      isShuffleEquivalent(V1, V2, Mask, {0, 1, 4, 5})) {
+  bool OnlyUsesV1 = isShuffleEquivalent(V1, V2, Mask, {0, 1, 0, 1});
+  if (OnlyUsesV1 || isShuffleEquivalent(V1, V2, Mask, {0, 1, 4, 5})) {
     SDValue LoV = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, SubVT, V1,
                               DAG.getIntPtrConstant(0));
     SDValue HiV = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, SubVT,
-                              Mask[2] < 4 ? V1 : V2, DAG.getIntPtrConstant(0));
+                              OnlyUsesV1 ? V1 : V2, DAG.getIntPtrConstant(0));
     return DAG.getNode(ISD::CONCAT_VECTORS, DL, VT, LoV, HiV);
   }
   if (isShuffleEquivalent(V1, V2, Mask, {0, 1, 6, 7})) {
     SDValue LoV = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, SubVT, V1,
                               DAG.getIntPtrConstant(0));
     SDValue HiV = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, SubVT, V2,
                               DAG.getIntPtrConstant(2));
     return DAG.getNode(ISD::CONCAT_VECTORS, DL, VT, LoV, HiV);

llvm/trunk/test/CodeGen/X86/lower-vec-shuffle-bug.ll

+; RUN: llc -mtriple=x86_64-unknown-unknown -mattr=+avx < %s | FileCheck %s
+
+define <4 x double> @test1(<4 x double> %A, <4 x double> %B) {
+; CHECK-LABEL: test1:
+; CHECK:       # BB#0:
+; CHECK-NEXT:    vinsertf128 $1, %xmm1, %ymm0, %ymm0
+; CHECK-NEXT:    retq
+entry:
+  %0 = shufflevector <4 x double> %A, <4 x double> %B, <4 x i32> <i32 undef, i32 1, i32 undef, i32 5>
+  ret <4 x double> %0
+}
+
+define <4 x double> @test2(<4 x double> %A, <4 x double> %B) {
+; CHECK-LABEL: test2:
+; CHECK:       # BB#0:
+; CHECK-NEXT:    vinsertf128 $1, %xmm0, %ymm0, %ymm0
+; CHECK-NEXT:    retq
+entry:
+  %0 = shufflevector <4 x double> %A, <4 x double> %B, <4 x i32> <i32 undef, i32 1, i32 undef, i32 1>
+  ret <4 x double> %0
+}
+
+define <4 x double> @test3(<4 x double> %A, <4 x double> %B) {
+; CHECK-LABEL: test3:
+; CHECK:       # BB#0:
+; CHECK-NEXT:    vinsertf128 $1, %xmm1, %ymm0, %ymm0
+; CHECK-NEXT:    retq
+entry:
+  %0 = shufflevector <4 x double> %A, <4 x double> %B, <4 x i32> <i32 0, i32 1, i32 undef, i32 5>
+  ret <4 x double> %0
+}
+
+define <4 x double> @test4(<4 x double> %A, <4 x double> %B) {
+; CHECK-LABEL: test4:
+; CHECK:       # BB#0:
+; CHECK-NEXT:    vinsertf128 $1, %xmm0, %ymm0, %ymm0
+; CHECK-NEXT:    retq
+entry:
+  %0 = shufflevector <4 x double> %A, <4 x double> %B, <4 x i32> <i32 0, i32 1, i32 undef, i32 1>
+  ret <4 x double> %0
+}