[ISEL] Canonicalize STEP_VECTOR to LHS if RHS is a splat.

[sdesmalen]

Summary

This helps recognise patterns where we're trying to match STEP_VECTOR
patterns to INDEX instructions that take a GPR for the Start/Step.

The reason for canonicalising this operation to the LHS is
because it will already be canonicalised to the LHS if the RHS
is a constant splat vector.

[sdesmalen]

This change is inspired by D117900 where it tries to fold some linear transform of step_vector into the gather/scatter addressing modes.

[RKSimon]

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp

5662

I expect we're going to need this for a lot of the DAG combines as well - is it worth adding a helper wrapper for isConstantIntBuildVectorOrConstantInt/isConstantFPBuildVectorOrConstantFP with SPLAT_VECTOR and STEP_VECTOR as well?

[sdesmalen]

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp

5662

I'd rather avoid creating another isConstant.... method, because we already have so many of them and despite the name, isConstantIntBuildVectorOrConstantInt already supports SPLAT_VECTOR (it would actually be good to clean that up a bit at some point).

For this patch, it may be worth moving this commutativity code as a whole to a separate function (e.g. swapCommutativeOperands), would that help?

[sdesmalen]

Moved some of the logic to a separate function (trySwapOpsCommutativeBinop).

[paulwalker-arm]

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp

5629

This comment is now out of date. I'd just remove the comment given you've moved the code into a function with CommutativeBinop in the name.

5643

What about changing this to binop(splat(x), not_splat(y)) -> binop(not_splat(y), splat(x))? That would solve your problem and be truly canonical.

[RKSimon]

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp

5627

Sorry for bikeshedding - but maybe canonicalizeCommutativeBinop is a better name?

[sdesmalen]

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp

5643

Unfortunately that has other side-effects, e.g. for:

getelementptr i8, i8* %base, <vscale x 2 x i32> %offsets

This would swap the splat(%base) and %offsets so that getUniformBase no longer recognises the %base as the scalar base pointer.

I wonder if this means the proposed canonicalisation is too specific, and therefore not worth it?

[paulwalker-arm]

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp

5643

Not sure I understand what you're saying here. Doesn't this just mean there's code that needs to be changed to check for the new canonical form?

[paulwalker-arm]

Just wanted to say that I'm not suggesting this patch implements the binop(splat(x), not_splat(y)) -> binop(not_splat(y), splat(x)) idea in one go. I just think that if we agree that it's ultimately the better canonical form then I'm happy to accept this patch as a step in the right direction to unblock your current work and we can fix the rest up later (well sooner rather than later :) )

[sdesmalen]

• Renamed trySwapOpsCommutativeBinop->canonicalizeCommutativeBinop.
• Removed stale comment.

[sdesmalen]

In D118459#3287061, @paulwalker-arm wrote:

Just wanted to say that I'm not suggesting this patch implements the binop(splat(x), not_splat(y)) -> binop(not_splat(y), splat(x)) idea in one go. I just think that if we agree that it's ultimately the better canonical form then I'm happy to accept this patch as a step in the right direction to unblock your current work and we can fix the rest up later (well sooner rather than later :) )

Okay that makes sense, thanks for clarifying!

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp

5627

That's indeed a much better name, thank you :)

llvm/include/llvm/CodeGen/SelectionDAG.h

SDValue getAddrSpaceCast(const SDLoc &dl, EVT VT, SDValue Ptr, unsigned SrcAS,
                            unsigned DestAS);
 
   /// Return a freeze using the SDLoc of the value operand.
   SDValue getFreeze(SDValue V);
 
   /// Return an AssertAlignSDNode.
   SDValue getAssertAlign(const SDLoc &DL, SDValue V, Align A);
 
+  /// Swap N1 and N2 if Opcode is a commutative binary opcode
+  /// and the canonical form expects the opposite order.
+  void trySwapOpsCommutativeBinop(unsigned Opcode, SDValue &N1,
+                                  SDValue &N2) const;
+
   /// Return the specified value casted to
   /// the target's desired shift amount type.
   SDValue getShiftAmountOperand(EVT LHSTy, SDValue Op);
 
   /// Expand the specified \c ISD::VAARG node as the Legalize pass would.
   SDValue expandVAArg(SDNode *Node);
 
   /// Expand the specified \c ISD::VACOPY node as the Legalize pass would.

llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp

 SDValue SelectionDAG::getNode(unsigned Opcode, const SDLoc &DL, EVT VT,
                               SDValue N1, SDValue N2) {
   SDNodeFlags Flags;
   if (Inserter)
     Flags = Inserter->getFlags();
   return getNode(Opcode, DL, VT, N1, N2, Flags);
 }
 
-SDValue SelectionDAG::getNode(unsigned Opcode, const SDLoc &DL, EVT VT,
-                              SDValue N1, SDValue N2, const SDNodeFlags Flags) {
-  assert(N1.getOpcode() != ISD::DELETED_NODE &&
-         N2.getOpcode() != ISD::DELETED_NODE &&
-         "Operand is DELETED_NODE!");
+void SelectionDAG::trySwapOpsCommutativeBinop(unsigned Opcode, SDValue &N1,
+                                              SDValue &N2) const {
   // Canonicalize constant to RHS if commutative.
-  if (TLI->isCommutativeBinOp(Opcode)) {
-    bool IsN1C = isConstantIntBuildVectorOrConstantInt(N1);
-    bool IsN2C = isConstantIntBuildVectorOrConstantInt(N2);
-    bool IsN1CFP = isConstantFPBuildVectorOrConstantFP(N1);
-    bool IsN2CFP = isConstantFPBuildVectorOrConstantFP(N2);
-    if ((IsN1C && !IsN2C) || (IsN1CFP && !IsN2CFP))
-      std::swap(N1, N2);
-  }
+  if (!TLI->isCommutativeBinOp(Opcode))
+    return;
+
+  // Canonicalize:
+  //   binop(const, nonconst) -> binop(nonconst, const)
+
+  // Canonicalize:
+  //  binop(splat(X), stepvector) -> binop(stepvector, splat(X))
+  //
+  // If X is a constant the splat will already have been moved to the RHS.
+  // By also doing this if X is not a constant, makes STEP_VECTOR easier
+  // to match in address logic.
+  else if (N1.getOpcode() == ISD::SPLAT_VECTOR &&
+           N2.getOpcode() == ISD::STEP_VECTOR)
+    std::swap(N1, N2);
 
+SDValue SelectionDAG::getNode(unsigned Opcode, const SDLoc &DL, EVT VT,
+                              SDValue N1, SDValue N2, const SDNodeFlags Flags) {
+  assert(N1.getOpcode() != ISD::DELETED_NODE &&
+         N2.getOpcode() != ISD::DELETED_NODE &&
+         "Operand is DELETED_NODE!");
+
+  trySwapOpsCommutativeBinop(Opcode, N1, N2);
+
   auto *N1C = dyn_cast<ConstantSDNode>(N1);
   auto *N2C = dyn_cast<ConstantSDNode>(N2);
 
   // Don't allow undefs in vector splats - we might be returning N2 when folding
   // to zero etc.
   ConstantSDNode *N2CV =
       isConstOrConstSplat(N2, /*AllowUndefs*/ false, /*AllowTruncation*/ true);
 
   switch (Opcode) {
   default: break;

llvm/test/CodeGen/AArch64/active_lane_mask.ll

; CHECK-NEXT:    ret
   ret <vscale x 2 x i1> %active.lane.mask
 }
 
 define <vscale x 16 x i1> @lane_mask_nxv16i1_i8(i8 %index, i8 %TC) {
 ; CHECK-LABEL: lane_mask_nxv16i1_i8:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    index z0.b, #0, #1
 ; CHECK-NEXT:    mov z1.b, w0
-; CHECK-NEXT:    uqadd z0.b, z1.b, z0.b
+; CHECK-NEXT:    uqadd z0.b, z0.b, z1.b
 ; CHECK-NEXT:    mov z1.b, w1
 ; CHECK-NEXT:    ptrue p0.b
 ; CHECK-NEXT:    cmphi p0.b, p0/z, z1.b, z0.b
 ; CHECK-NEXT:    ret
   %active.lane.mask = call <vscale x 16 x i1> @llvm.get.active.lane.mask.nxv16i1.i8(i8 %index, i8 %TC)
   ret <vscale x 16 x i1> %active.lane.mask
 }
 
 define <vscale x 8 x i1> @lane_mask_nxv8i1_i8(i8 %index, i8 %TC) {
 ; CHECK-LABEL: lane_mask_nxv8i1_i8:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    index z0.h, #0, #1
 ; CHECK-NEXT:    mov z1.h, w0
 ; CHECK-NEXT:    and z0.h, z0.h, #0xff
 ; CHECK-NEXT:    and z1.h, z1.h, #0xff
-; CHECK-NEXT:    add z0.h, z1.h, z0.h
+; CHECK-NEXT:    add z0.h, z0.h, z1.h
 ; CHECK-NEXT:    mov z1.h, w1
 ; CHECK-NEXT:    umin z0.h, z0.h, #255
 ; CHECK-NEXT:    and z1.h, z1.h, #0xff
 ; CHECK-NEXT:    and z0.h, z0.h, #0xff
 ; CHECK-NEXT:    ptrue p0.h
 ; CHECK-NEXT:    cmphi p0.h, p0/z, z1.h, z0.h
 ; CHECK-NEXT:    ret
   %active.lane.mask = call <vscale x 8 x i1> @llvm.get.active.lane.mask.nxv8i1.i8(i8 %index, i8 %TC)
   ret <vscale x 8 x i1> %active.lane.mask
 }
 
 define <vscale x 4 x i1> @lane_mask_nxv4i1_i8(i8 %index, i8 %TC) {
 ; CHECK-LABEL: lane_mask_nxv4i1_i8:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    index z0.s, #0, #1
 ; CHECK-NEXT:    mov z1.s, w0
 ; CHECK-NEXT:    and z0.s, z0.s, #0xff
 ; CHECK-NEXT:    and z1.s, z1.s, #0xff
-; CHECK-NEXT:    add z0.s, z1.s, z0.s
+; CHECK-NEXT:    add z0.s, z0.s, z1.s
 ; CHECK-NEXT:    mov z1.s, w1
 ; CHECK-NEXT:    umin z0.s, z0.s, #255
 ; CHECK-NEXT:    and z1.s, z1.s, #0xff
 ; CHECK-NEXT:    and z0.s, z0.s, #0xff
 ; CHECK-NEXT:    ptrue p0.s
 ; CHECK-NEXT:    cmphi p0.s, p0/z, z1.s, z0.s
 ; CHECK-NEXT:    ret
   %active.lane.mask = call <vscale x 4 x i1> @llvm.get.active.lane.mask.nxv4i1.i8(i8 %index, i8 %TC)
   ret <vscale x 4 x i1> %active.lane.mask
 }
 
 define <vscale x 2 x i1> @lane_mask_nxv2i1_i8(i8 %index, i8 %TC) {
 ; CHECK-LABEL: lane_mask_nxv2i1_i8:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    // kill: def $w0 killed $w0 def $x0
 ; CHECK-NEXT:    index z0.d, #0, #1
 ; CHECK-NEXT:    mov z1.d, x0
 ; CHECK-NEXT:    and z0.d, z0.d, #0xff
 ; CHECK-NEXT:    and z1.d, z1.d, #0xff
-; CHECK-NEXT:    add z0.d, z1.d, z0.d
+; CHECK-NEXT:    add z0.d, z0.d, z1.d
 ; CHECK-NEXT:    // kill: def $w1 killed $w1 def $x1
 ; CHECK-NEXT:    mov z2.d, x1
 ; CHECK-NEXT:    umin z0.d, z0.d, #255
 ; CHECK-NEXT:    and z2.d, z2.d, #0xff
 ; CHECK-NEXT:    and z0.d, z0.d, #0xff
 ; CHECK-NEXT:    ptrue p0.d
 ; CHECK-NEXT:    cmphi p0.d, p0/z, z2.d, z0.d
 ; CHECK-NEXT:    ret

 ; CHECK-NEXT:    .cfi_offset w29, -16
 ; CHECK-NEXT:    index z0.s, #0, #1
 ; CHECK-NEXT:    mov z3.s, w0
 ; CHECK-NEXT:    mov z1.d, z0.d
 ; CHECK-NEXT:    ptrue p0.s
 ; CHECK-NEXT:    mov z2.d, z0.d
 ; CHECK-NEXT:    mov z4.s, w1
 ; CHECK-NEXT:    incw z1.s
-; CHECK-NEXT:    uqadd z5.s, z3.s, z0.s
+; CHECK-NEXT:    uqadd z5.s, z0.s, z3.s
 ; CHECK-NEXT:    incw z2.s, all, mul #2
 ; CHECK-NEXT:    mov z6.d, z1.d
 ; CHECK-NEXT:    cmphi p1.s, p0/z, z4.s, z5.s
-; CHECK-NEXT:    uqadd z5.s, z3.s, z1.s
+; CHECK-NEXT:    uqadd z5.s, z1.s, z3.s
 ; CHECK-NEXT:    cmphi p2.s, p0/z, z4.s, z5.s
-; CHECK-NEXT:    uqadd z5.s, z3.s, z2.s
+; CHECK-NEXT:    uqadd z5.s, z2.s, z3.s
 ; CHECK-NEXT:    incw z6.s, all, mul #2
 ; CHECK-NEXT:    incw z0.s, all, mul #4
 ; CHECK-NEXT:    cmphi p3.s, p0/z, z4.s, z5.s
-; CHECK-NEXT:    uqadd z5.s, z3.s, z6.s
+; CHECK-NEXT:    uqadd z5.s, z6.s, z3.s
 ; CHECK-NEXT:    incw z1.s, all, mul #4
 ; CHECK-NEXT:    cmphi p4.s, p0/z, z4.s, z5.s
-; CHECK-NEXT:    uqadd z0.s, z3.s, z0.s
-; CHECK-NEXT:    uqadd z1.s, z3.s, z1.s
+; CHECK-NEXT:    uqadd z0.s, z0.s, z3.s
+; CHECK-NEXT:    uqadd z1.s, z1.s, z3.s
 ; CHECK-NEXT:    incw z2.s, all, mul #4
 ; CHECK-NEXT:    incw z6.s, all, mul #4
 ; CHECK-NEXT:    uzp1 p1.h, p1.h, p2.h
 ; CHECK-NEXT:    uzp1 p2.h, p3.h, p4.h
 ; CHECK-NEXT:    cmphi p3.s, p0/z, z4.s, z0.s
 ; CHECK-NEXT:    cmphi p4.s, p0/z, z4.s, z1.s
-; CHECK-NEXT:    uqadd z0.s, z3.s, z2.s
-; CHECK-NEXT:    uqadd z1.s, z3.s, z6.s
+; CHECK-NEXT:    uqadd z0.s, z2.s, z3.s
+; CHECK-NEXT:    uqadd z1.s, z6.s, z3.s
 ; CHECK-NEXT:    cmphi p5.s, p0/z, z4.s, z0.s
 ; CHECK-NEXT:    cmphi p0.s, p0/z, z4.s, z1.s
 ; CHECK-NEXT:    uzp1 p3.h, p3.h, p4.h
 ; CHECK-NEXT:    uzp1 p4.h, p5.h, p0.h
 ; CHECK-NEXT:    uzp1 p0.b, p1.b, p2.b
 ; CHECK-NEXT:    uzp1 p1.b, p3.b, p4.b
 ; CHECK-NEXT:    ldr p5, [sp, #6, mul vl] // 2-byte Folded Reload
 ; CHECK-NEXT:    ldr p4, [sp, #7, mul vl] // 2-byte Folded Reload

 ; CHECK-NEXT:    .cfi_offset w29, -16
 ; CHECK-NEXT:    index z0.d, #0, #1
 ; CHECK-NEXT:    mov z3.d, x0
 ; CHECK-NEXT:    mov z1.d, z0.d
 ; CHECK-NEXT:    ptrue p0.d
 ; CHECK-NEXT:    mov z2.d, z0.d
 ; CHECK-NEXT:    mov z4.d, x1
 ; CHECK-NEXT:    incd z1.d
-; CHECK-NEXT:    uqadd z5.d, z3.d, z0.d
-; CHECK-NEXT:    uqadd z6.d, z3.d, z1.d
+; CHECK-NEXT:    uqadd z5.d, z0.d, z3.d
+; CHECK-NEXT:    uqadd z6.d, z1.d, z3.d
 ; CHECK-NEXT:    cmphi p1.d, p0/z, z4.d, z5.d
 ; CHECK-NEXT:    mov z5.d, z1.d
 ; CHECK-NEXT:    incd z2.d, all, mul #2
 ; CHECK-NEXT:    cmphi p2.d, p0/z, z4.d, z6.d
-; CHECK-NEXT:    uqadd z6.d, z3.d, z2.d
+; CHECK-NEXT:    uqadd z6.d, z2.d, z3.d
 ; CHECK-NEXT:    mov z7.d, z0.d
 ; CHECK-NEXT:    incd z5.d, all, mul #2
 ; CHECK-NEXT:    uzp1 p1.s, p1.s, p2.s
 ; CHECK-NEXT:    cmphi p2.d, p0/z, z4.d, z6.d
-; CHECK-NEXT:    uqadd z6.d, z3.d, z5.d
+; CHECK-NEXT:    uqadd z6.d, z5.d, z3.d
 ; CHECK-NEXT:    mov z24.d, z1.d
 ; CHECK-NEXT:    incd z7.d, all, mul #4
 ; CHECK-NEXT:    cmphi p3.d, p0/z, z4.d, z6.d
-; CHECK-NEXT:    uqadd z6.d, z3.d, z7.d
+; CHECK-NEXT:    uqadd z6.d, z7.d, z3.d
 ; CHECK-NEXT:    mov z25.d, z2.d
 ; CHECK-NEXT:    incd z24.d, all, mul #4
 ; CHECK-NEXT:    mov z26.d, z5.d
 ; CHECK-NEXT:    cmphi p4.d, p0/z, z4.d, z6.d
-; CHECK-NEXT:    uqadd z6.d, z3.d, z24.d
+; CHECK-NEXT:    uqadd z6.d, z24.d, z3.d
 ; CHECK-NEXT:    incd z25.d, all, mul #4
 ; CHECK-NEXT:    cmphi p5.d, p0/z, z4.d, z6.d
-; CHECK-NEXT:    uqadd z6.d, z3.d, z25.d
+; CHECK-NEXT:    uqadd z6.d, z25.d, z3.d
 ; CHECK-NEXT:    incd z26.d, all, mul #4
 ; CHECK-NEXT:    cmphi p6.d, p0/z, z4.d, z6.d
-; CHECK-NEXT:    uqadd z6.d, z3.d, z26.d
+; CHECK-NEXT:    uqadd z6.d, z26.d, z3.d
 ; CHECK-NEXT:    uzp1 p2.s, p2.s, p3.s
 ; CHECK-NEXT:    cmphi p3.d, p0/z, z4.d, z6.d
 ; CHECK-NEXT:    incd z0.d, all, mul #8
 ; CHECK-NEXT:    incd z1.d, all, mul #8
 ; CHECK-NEXT:    uzp1 p4.s, p4.s, p5.s
 ; CHECK-NEXT:    uzp1 p3.s, p6.s, p3.s
-; CHECK-NEXT:    uqadd z0.d, z3.d, z0.d
-; CHECK-NEXT:    uqadd z1.d, z3.d, z1.d
+; CHECK-NEXT:    uqadd z0.d, z0.d, z3.d
+; CHECK-NEXT:    uqadd z1.d, z1.d, z3.d
 ; CHECK-NEXT:    incd z2.d, all, mul #8
 ; CHECK-NEXT:    incd z5.d, all, mul #8
 ; CHECK-NEXT:    uzp1 p1.h, p1.h, p2.h
 ; CHECK-NEXT:    uzp1 p2.h, p4.h, p3.h
 ; CHECK-NEXT:    cmphi p3.d, p0/z, z4.d, z0.d
 ; CHECK-NEXT:    cmphi p4.d, p0/z, z4.d, z1.d
-; CHECK-NEXT:    uqadd z0.d, z3.d, z2.d
-; CHECK-NEXT:    uqadd z1.d, z3.d, z5.d
+; CHECK-NEXT:    uqadd z0.d, z2.d, z3.d
+; CHECK-NEXT:    uqadd z1.d, z5.d, z3.d
 ; CHECK-NEXT:    incd z7.d, all, mul #8
 ; CHECK-NEXT:    incd z24.d, all, mul #8
 ; CHECK-NEXT:    cmphi p5.d, p0/z, z4.d, z0.d
 ; CHECK-NEXT:    cmphi p6.d, p0/z, z4.d, z1.d
-; CHECK-NEXT:    uqadd z0.d, z3.d, z7.d
-; CHECK-NEXT:    uqadd z1.d, z3.d, z24.d
+; CHECK-NEXT:    uqadd z0.d, z7.d, z3.d
+; CHECK-NEXT:    uqadd z1.d, z24.d, z3.d
 ; CHECK-NEXT:    incd z25.d, all, mul #8
 ; CHECK-NEXT:    incd z26.d, all, mul #8
 ; CHECK-NEXT:    uzp1 p3.s, p3.s, p4.s
 ; CHECK-NEXT:    uzp1 p4.s, p5.s, p6.s
 ; CHECK-NEXT:    cmphi p5.d, p0/z, z4.d, z0.d
 ; CHECK-NEXT:    cmphi p6.d, p0/z, z4.d, z1.d
-; CHECK-NEXT:    uqadd z0.d, z3.d, z25.d
-; CHECK-NEXT:    uqadd z1.d, z3.d, z26.d
+; CHECK-NEXT:    uqadd z0.d, z25.d, z3.d
+; CHECK-NEXT:    uqadd z1.d, z26.d, z3.d
 ; CHECK-NEXT:    cmphi p7.d, p0/z, z4.d, z0.d
 ; CHECK-NEXT:    cmphi p0.d, p0/z, z4.d, z1.d
 ; CHECK-NEXT:    uzp1 p5.s, p5.s, p6.s
 ; CHECK-NEXT:    uzp1 p0.s, p7.s, p0.s
 ; CHECK-NEXT:    uzp1 p3.h, p3.h, p4.h
 ; CHECK-NEXT:    uzp1 p4.h, p5.h, p0.h
 ; CHECK-NEXT:    uzp1 p0.b, p1.b, p2.b
 ; CHECK-NEXT:    uzp1 p1.b, p3.b, p4.b

 ; CHECK-LABEL: lane_mask_nxv32i1_i8:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    rdvl x8, #1
 ; CHECK-NEXT:    index z0.b, #0, #1
 ; CHECK-NEXT:    mov z1.b, w8
 ; CHECK-NEXT:    mov z2.b, w0
 ; CHECK-NEXT:    add z1.b, z0.b, z1.b
 ; CHECK-NEXT:    mov z3.b, w1
-; CHECK-NEXT:    uqadd z0.b, z2.b, z0.b
+; CHECK-NEXT:    uqadd z0.b, z0.b, z2.b
 ; CHECK-NEXT:    ptrue p1.b
-; CHECK-NEXT:    uqadd z1.b, z2.b, z1.b
+; CHECK-NEXT:    uqadd z1.b, z1.b, z2.b
 ; CHECK-NEXT:    cmphi p0.b, p1/z, z3.b, z0.b
 ; CHECK-NEXT:    cmphi p1.b, p1/z, z3.b, z1.b
 ; CHECK-NEXT:    ret
   %active.lane.mask = call <vscale x 32 x i1> @llvm.get.active.lane.mask.nxv32i1.i8(i8 %index, i8 %TC)
   ret <vscale x 32 x i1> %active.lane.mask
 }
 
 

llvm/test/CodeGen/AArch64/sve-stepvector.ll

entry:
   ret <vscale x 2 x i64> %3
 }
 
 define <vscale x 2 x i64> @multiple_use_stepvector_nxv2i64_1(i64 %data) {
 ; CHECK-LABEL: multiple_use_stepvector_nxv2i64_1:
 ; CHECK:       // %bb.0: // %entry
 ; CHECK-NEXT:    index z0.d, #0, #1
 ; CHECK-NEXT:    mov z1.d, x0
-; CHECK-NEXT:    add z1.d, z1.d, z0.d
+; CHECK-NEXT:    add z1.d, z0.d, z1.d
 ; CHECK-NEXT:    ptrue p0.d
 ; CHECK-NEXT:    mul z0.d, p0/m, z0.d, z1.d
 ; CHECK-NEXT:    ret
 entry:
   %0 = insertelement <vscale x 2 x i64> poison, i64 %data, i32 0
   %1 = shufflevector <vscale x 2 x i64> %0, <vscale x 2 x i64> poison, <vscale x 2 x i32> zeroinitializer
   %2 = call <vscale x 2 x i64> @llvm.experimental.stepvector.nxv2i64()
   %3 = add <vscale x 2 x i64> %1, %2
entry:
   %5 = shufflevector <vscale x 2 x i64> %4, <vscale x 2 x i64> poison, <vscale x 2 x i32> zeroinitializer
   %6 = add <vscale x 2 x i64> %3, %5
   ret <vscale x 2 x i64> %6
 }
 
 define <vscale x 2 x i64> @mul_add_stepvector_nxv2i64_commutative(i64 %x, i64 %y) {
 ; CHECK-LABEL: mul_add_stepvector_nxv2i64_commutative:
 ; CHECK:       // %bb.0: // %entry
-; CHECK-NEXT:    index z1.d, #0, #1
-; CHECK-NEXT:    mov z2.d, x1
-; CHECK-NEXT:    ptrue p0.d
-; CHECK-NEXT:    mov z0.d, x0
-; CHECK-NEXT:    mla z0.d, p0/m, z2.d, z1.d
+; CHECK-NEXT:    index z0.d, x0, x1
 ; CHECK-NEXT:    ret
 entry:
   %0 = insertelement <vscale x 2 x i64> poison, i64 %y, i32 0
   %1 = shufflevector <vscale x 2 x i64> %0, <vscale x 2 x i64> poison, <vscale x 2 x i32> zeroinitializer
   %2 = call <vscale x 2 x i64> @llvm.experimental.stepvector.nxv2i64()
   %3 = mul <vscale x 2 x i64> %1, %2
   %4 = insertelement <vscale x 2 x i64> poison, i64 %x, i32 0
   %5 = shufflevector <vscale x 2 x i64> %4, <vscale x 2 x i64> poison, <vscale x 2 x i32> zeroinitializer