[ARM] Match dual lane vmovs from insert_vector_elt

[dmgreen]

Summary

MVE has a dual lane vector move instruction, capable of moving two general purpose registers into lanes of a vector register. They look like one of:

vmov q0[2], q0[0], r2, r0
vmov q0[3], q0[1], r3, r1

They only accept these lane indices though (and only insert into an i32), either moving lanes 1 and 3, or 0 and 2.

This patch adds some tablegen patterns for them, selecting from vector inserts elements. Because the insert_elements are knows to be canonicalized to ascending order there are several patterns that we need to select. These lane indices are:

3 2 1 0    -> vmovqrr 31; vmovqrr 20
3 2 1      -> vmovqrr 31; vmov 2
3 1        -> vmovqrr 31
2 1 0      -> vmovqrr 20; vmov 1
2 0        -> vmovqrr 20

With the top one being the most common. All other potential patterns of lane indices will be matched by a combination of these and the individual vmov pattern already present. This does mean that we are selecting several machine instructions at once due to the need to re-arrange the inserts, but in this case there is at least nothing else that will attempt to match an insert_vector_elt node.

[dmgreen]

Now that I think about it more, perhaps the X86 code changes should be handled differently. There were three tests that changed, two of which (avx512-insert-extract.ll and haddsub-2.ll) are the same instructions in a different order. The other (avx512-mask-op.ll:test18) has a different series of kshiftl/kshiftr's with a single extra instruction.

[RKSimon]

Much of the ISD::BUILD_VECTOR creation code tries to use this and relies on sorted insertions chains - would it be better just to add a TLI virtual method that disables this at the target level for specific valuetypes?

[dmgreen]

Hmm. Yeah there might be some code that relies on this but isn't tested. It was added in https://github.com/llvm/llvm-project/commit/f99dd64f0afd42c5fc51a11dea94a21e7d63cf8e, but that didn't have any tests itself.

I could probably just do this entirely in tablegen. There are only actually 5 patterns when you get down to it, so long as you know the nodes are in-order.. It's not very pretty, but I'll look into doing it that way.

[dmgreen]

Now just using tablegen patterns.

[RKSimon]

LGTM (with one minor) but an ARM guru should really approve it.

llvm/lib/Target/ARM/ARMBaseInstrInfo.cpp

4794

Missing assert message.

[SjoerdMeijer]

In D92553#2442344, @RKSimon wrote:

LGTM (with one minor) but an ARM guru should really approve it.

This puts some pressure on things ;-) .... but this looks like nice codegen changes, so LGTM too.

llvm/lib/Target/ARM/ARMBaseInstrInfo.cpp

for (int i = 2, e = MI.getNumOperands(); i < e; ++i) {
         if (!(MI.getOpcode() == ARM::tPUSH && Reg == ARM::LR) &&
             !(MI.getOpcode() == ARM::tPOP_RET && Reg == ARM::PC)) {
           ErrInfo = "Unsupported register in Thumb1 push/pop";
           return false;
         }
       }
     }
   }
+  if (MI.getOpcode() == ARM::MVE_VMOV_q_rr) {
+    assert(MI.getOperand(4).isImm() && MI.getOperand(5).isImm());
+    if ((MI.getOperand(4).getImm() != 2 && MI.getOperand(4).getImm() != 3) ||
+        MI.getOperand(4).getImm() != MI.getOperand(5).getImm() + 2) {
+      ErrInfo = "Incorrect array index for MVE_VMOV_q_rr";
+      return false;
+    }
+  }
   return true;
 }
 
 // LoadStackGuard has so far only been implemented for MachO. Different code
 // sequence is needed for other targets.
 void ARMBaseInstrInfo::expandLoadStackGuardBase(MachineBasicBlock::iterator MI,
                                                 unsigned LoadImmOpc,
                                                 unsigned LoadOpc) const {

llvm/lib/Target/ARM/ARMInstrMVE.td

 }
 
 def MVE_VMOV_rr_q : MVE_VMOV_64bit<(outs rGPR:$Rt, rGPR:$Rt2), (ins MQPR:$Qd),
                                    0b0, "$Rt, $Rt2, $Qd$idx, $Qd$idx2", ""> {
   let DecoderMethod = "DecodeMVEVMOVQtoDReg";
   let AsmMatchConverter = "cvtMVEVMOVQtoDReg";
 }
 
+let Predicates = [HasMVEInt] in {
+  // Double lane moves. There are a number of patterns here. We know that the
+  // insertelt's will be in descending order by index, and need to match the 5
+  // patterns that might contain 2-0 or 3-1 pairs. These are:
+  // 3 2 1 0    -> vmovqrr 31; vmovqrr 20
+  // 3 2 1      -> vmovqrr 31; vmov 2
+  // 3 1        -> vmovqrr 31
+  // 2 1 0      -> vmovqrr 20; vmov 1
+  // 2 0        -> vmovqrr 20
+  // The other potential patterns will be handled by single lane inserts.
+  def : Pat<(insertelt (insertelt (insertelt (insertelt (v4i32 MQPR:$src1),
+                                                        rGPR:$srcA, (i32 0)),
+                                             rGPR:$srcB, (i32 1)),
+                                  rGPR:$srcC, (i32 2)),
+                       rGPR:$srcD, (i32 3)),
+            (MVE_VMOV_q_rr (MVE_VMOV_q_rr MQPR:$src1, rGPR:$srcC, rGPR:$srcA, (i32 2), (i32 0)),
+                           rGPR:$srcD, rGPR:$srcB, (i32 3), (i32 1))>;
+  def : Pat<(insertelt (insertelt (insertelt (v4i32 MQPR:$src1),
+                                             rGPR:$srcB, (i32 1)),
+                                  rGPR:$srcC, (i32 2)),
+                       rGPR:$srcD, (i32 3)),
+            (MVE_VMOV_q_rr (MVE_VMOV_to_lane_32 MQPR:$src1, rGPR:$srcC, (i32 2)),
+                           rGPR:$srcD, rGPR:$srcB, (i32 3), (i32 1))>;
+  def : Pat<(insertelt (insertelt (v4i32 MQPR:$src1), rGPR:$srcA, (i32 1)), rGPR:$srcB, (i32 3)),
+            (MVE_VMOV_q_rr MQPR:$src1, rGPR:$srcB, rGPR:$srcA, (i32 3), (i32 1))>;
+  def : Pat<(insertelt (insertelt (insertelt (v4i32 MQPR:$src1),
+                                             rGPR:$srcB, (i32 0)),
+                                  rGPR:$srcC, (i32 1)),
+                       rGPR:$srcD, (i32 2)),
+            (MVE_VMOV_q_rr (MVE_VMOV_to_lane_32 MQPR:$src1, rGPR:$srcC, (i32 1)),
+                           rGPR:$srcD, rGPR:$srcB, (i32 2), (i32 0))>;
+  def : Pat<(insertelt (insertelt (v4i32 MQPR:$src1), rGPR:$srcA, (i32 0)), rGPR:$srcB, (i32 2)),
+            (MVE_VMOV_q_rr MQPR:$src1, rGPR:$srcB, rGPR:$srcA, (i32 2), (i32 0))>;
+}
+
 // end of coproc mov
 
 // start of MVE interleaving load/store
 
 // Base class for the family of interleaving/deinterleaving
 // load/stores with names like VLD20.8 and VST43.32.
 class MVE_vldst24_base<bit writeback, bit fourregs, bits<2> stage, bits<2> size,
                        bit load, dag Oops, dag loadIops, dag wbIops,

llvm/test/CodeGen/Thumb2/active_lane_mask.ll

 }
 
 define <7 x i32> @v7i32(i32 %index, i32 %TC, <7 x i32> %V1, <7 x i32> %V2) {
 ; CHECK-LABEL: v7i32:
 ; CHECK:       @ %bb.0:
 ; CHECK-NEXT:    adr r3, .LCPI1_0
 ; CHECK-NEXT:    vdup.32 q1, r1
 ; CHECK-NEXT:    vldrw.u32 q0, [r3]
+; CHECK-NEXT:    ldr r3, [sp, #40]
 ; CHECK-NEXT:    vadd.i32 q2, q0, r1
 ; CHECK-NEXT:    vdup.32 q0, r2
 ; CHECK-NEXT:    vcmp.u32 hi, q1, q2
 ; CHECK-NEXT:    ldr r2, [sp, #32]
 ; CHECK-NEXT:    vpnot
 ; CHECK-NEXT:    vpst
 ; CHECK-NEXT:    vcmpt.u32 hi, q0, q2
-; CHECK-NEXT:    vmov.32 q2[0], r2
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r2
 ; CHECK-NEXT:    ldr r2, [sp, #36]
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    ldr r2, [sp, #40]
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    ldr r2, [sp, #44]
-; CHECK-NEXT:    vmov.32 q2[3], r2
+; CHECK-NEXT:    ldr r3, [sp, #44]
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r2
 ; CHECK-NEXT:    ldr r2, [sp]
-; CHECK-NEXT:    vmov.32 q3[0], r2
+; CHECK-NEXT:    ldr r3, [sp, #8]
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r2
 ; CHECK-NEXT:    ldr r2, [sp, #4]
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    ldr r2, [sp, #8]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    ldr r2, [sp, #12]
-; CHECK-NEXT:    vmov.32 q3[3], r2
+; CHECK-NEXT:    ldr r3, [sp, #12]
+; CHECK-NEXT:    vmov q3[3], q3[1], r3, r2
 ; CHECK-NEXT:    adr r2, .LCPI1_1
 ; CHECK-NEXT:    vpsel q2, q3, q2
 ; CHECK-NEXT:    vstrw.32 q2, [r0]
 ; CHECK-NEXT:    vldrw.u32 q2, [r2]
 ; CHECK-NEXT:    movw r2, #4095
 ; CHECK-NEXT:    vadd.i32 q2, q2, r1
 ; CHECK-NEXT:    vcmp.u32 hi, q1, q2
 ; CHECK-NEXT:    vmrs r1, p0
 ; CHECK-NEXT:    eors r1, r2
+; CHECK-NEXT:    ldr r2, [sp, #56]
 ; CHECK-NEXT:    vmsr p0, r1
-; CHECK-NEXT:    ldr r1, [sp, #48]
+; CHECK-NEXT:    ldr r1, [sp, #52]
 ; CHECK-NEXT:    vpst
 ; CHECK-NEXT:    vcmpt.u32 hi, q0, q2
-; CHECK-NEXT:    vmov.32 q0[0], r1
-; CHECK-NEXT:    ldr r1, [sp, #52]
 ; CHECK-NEXT:    vmov.32 q0[1], r1
-; CHECK-NEXT:    ldr r1, [sp, #56]
-; CHECK-NEXT:    vmov.32 q0[2], r1
-; CHECK-NEXT:    ldr r1, [sp, #16]
-; CHECK-NEXT:    vmov.32 q1[0], r1
+; CHECK-NEXT:    ldr r1, [sp, #48]
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r1
 ; CHECK-NEXT:    ldr r1, [sp, #20]
+; CHECK-NEXT:    ldr r2, [sp, #24]
 ; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    ldr r1, [sp, #24]
-; CHECK-NEXT:    vmov.32 q1[2], r1
+; CHECK-NEXT:    ldr r1, [sp, #16]
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, r1
 ; CHECK-NEXT:    vpsel q0, q1, q0
 ; CHECK-NEXT:    vmov r1, s2
 ; CHECK-NEXT:    vmov.f32 s2, s1
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    strd r3, r2, [r0, #16]
 ; CHECK-NEXT:    str r1, [r0, #24]
 ; CHECK-NEXT:    bx lr
; CHECK-NEXT:    .long 15 @ 0xf
   %active.lane.mask = call <16 x i1> @llvm.get.active.lane.mask.v16i1.i32(i32 %index, i32 %TC)
   %select = select <16 x i1> %active.lane.mask, <16 x i8> %V1, <16 x i8> %V2
   ret <16 x i8> %select
 }
 
 define void @test_width2(i32* nocapture readnone %x, i32* nocapture %y, i8 zeroext %m) {
 ; CHECK-LABEL: test_width2:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    push {r4, r5, r6, lr}
-; CHECK-NEXT:    vpush {d8, d9, d10, d11}
+; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, lr}
+; CHECK-NEXT:    sub sp, #4
+; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    sub sp, #8
 ; CHECK-NEXT:    cmp r2, #0
 ; CHECK-NEXT:    beq.w .LBB4_3
 ; CHECK-NEXT:  @ %bb.1: @ %for.body.preheader
 ; CHECK-NEXT:    adds r0, r2, #1
-; CHECK-NEXT:    movs r3, #1
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, r2
 ; CHECK-NEXT:    bic r0, r0, #1
-; CHECK-NEXT:    vmov.32 q2[0], r2
+; CHECK-NEXT:    adr r2, .LCPI4_0
 ; CHECK-NEXT:    subs r0, #2
+; CHECK-NEXT:    movs r3, #1
 ; CHECK-NEXT:    vmov.i64 q0, #0xffffffff
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    movs r6, #0
+; CHECK-NEXT:    vldrw.u32 q2, [r2]
 ; CHECK-NEXT:    add.w lr, r3, r0, lsr #1
-; CHECK-NEXT:    adr r3, .LCPI4_0
+; CHECK-NEXT:    mov.w r8, #0
 ; CHECK-NEXT:    dls lr, lr
-; CHECK-NEXT:    vldrw.u32 q1, [r3]
-; CHECK-NEXT:    vand q2, q2, q0
+; CHECK-NEXT:    vand q1, q1, q0
 ; CHECK-NEXT:  .LBB4_2: @ %vector.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    vmov.32 q3[0], r6
-; CHECK-NEXT:    vmov r5, s8
-; CHECK-NEXT:    vmov.32 q3[2], r6
-; CHECK-NEXT:    vmov r0, s9
+; CHECK-NEXT:    vmov q3[2], q3[0], r8, r8
+; CHECK-NEXT:    vmov r7, s4
 ; CHECK-NEXT:    vand q3, q3, q0
-; CHECK-NEXT:    adds r6, #2
+; CHECK-NEXT:    vmov r6, s5
 ; CHECK-NEXT:    vmov r3, s14
+; CHECK-NEXT:    add.w r8, r8, #2
+; CHECK-NEXT:    vmov r9, s12
 ; CHECK-NEXT:    vmov r2, s15
+; CHECK-NEXT:    vmov r0, s7
 ; CHECK-NEXT:    adds r3, #1
-; CHECK-NEXT:    adc r12, r2, #0
-; CHECK-NEXT:    vmov r2, s12
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.32 q3[2], r3
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r9
 ; CHECK-NEXT:    vand q3, q3, q0
-; CHECK-NEXT:    vmov r4, s12
-; CHECK-NEXT:    teq.w r4, r2
-; CHECK-NEXT:    cset r2, ne
-; CHECK-NEXT:    tst.w r2, #1
-; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q4[0], r2
-; CHECK-NEXT:    vmov.32 q4[1], r2
-; CHECK-NEXT:    vmov r2, s14
-; CHECK-NEXT:    eors r3, r2
-; CHECK-NEXT:    orrs.w r3, r3, r12
-; CHECK-NEXT:    cset r3, ne
-; CHECK-NEXT:    tst.w r3, #1
-; CHECK-NEXT:    csetm r3, ne
-; CHECK-NEXT:    subs r5, r4, r5
-; CHECK-NEXT:    vmov.32 q4[2], r3
-; CHECK-NEXT:    vmov r5, s10
-; CHECK-NEXT:    vmov.32 q4[3], r3
-; CHECK-NEXT:    vmov r3, s13
-; CHECK-NEXT:    veor q4, q4, q1
-; CHECK-NEXT:    sbcs.w r0, r3, r0
-; CHECK-NEXT:    vmov r3, s11
-; CHECK-NEXT:    mov.w r0, #0
+; CHECK-NEXT:    adc r12, r2, #0
+; CHECK-NEXT:    vmov r5, s12
+; CHECK-NEXT:    vmov r4, s13
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    subs r7, r5, r7
+; CHECK-NEXT:    vmov r7, s14
+; CHECK-NEXT:    sbcs r4, r6
+; CHECK-NEXT:    vmov r6, s15
+; CHECK-NEXT:    mov.w r4, #0
 ; CHECK-NEXT:    it lo
-; CHECK-NEXT:    movlo r0, #1
-; CHECK-NEXT:    cmp r0, #0
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    subs r2, r2, r5
-; CHECK-NEXT:    vmov.32 q5[0], r0
-; CHECK-NEXT:    vmov.32 q5[1], r0
-; CHECK-NEXT:    vmov r0, s15
-; CHECK-NEXT:    @ implicit-def: $q3
-; CHECK-NEXT:    sbcs r0, r3
+; CHECK-NEXT:    movlo r4, #1
+; CHECK-NEXT:    cmp r4, #0
+; CHECK-NEXT:    csetm r4, ne
+; CHECK-NEXT:    subs r2, r7, r2
+; CHECK-NEXT:    sbcs.w r0, r6, r0
 ; CHECK-NEXT:    mov.w r0, #0
 ; CHECK-NEXT:    it lo
 ; CHECK-NEXT:    movlo r0, #1
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q5[2], r0
-; CHECK-NEXT:    vmov.32 q5[3], r0
-; CHECK-NEXT:    vand q4, q4, q5
+; CHECK-NEXT:    vmov q3[2], q3[0], r0, r4
+; CHECK-NEXT:    vmov q3[3], q3[1], r0, r4
+; CHECK-NEXT:    eor.w r0, r7, r3
+; CHECK-NEXT:    orrs.w r0, r0, r12
+; CHECK-NEXT:    cset r0, ne
+; CHECK-NEXT:    tst.w r0, #1
+; CHECK-NEXT:    csetm r0, ne
+; CHECK-NEXT:    teq.w r5, r9
+; CHECK-NEXT:    cset r2, ne
+; CHECK-NEXT:    tst.w r2, #1
+; CHECK-NEXT:    csetm r2, ne
+; CHECK-NEXT:    vmov q4[2], q4[0], r0, r2
+; CHECK-NEXT:    vmov q4[3], q4[1], r0, r2
+; CHECK-NEXT:    veor q4, q4, q2
+; CHECK-NEXT:    vand q4, q4, q3
+; CHECK-NEXT:    @ implicit-def: $q3
 ; CHECK-NEXT:    vmov r2, s16
 ; CHECK-NEXT:    vmov r0, s18
 ; CHECK-NEXT:    and r2, r2, #1
 ; CHECK-NEXT:    orr.w r3, r2, r0, lsl #1
 ; CHECK-NEXT:    sub.w r2, r1, #8
 ; CHECK-NEXT:    lsls r0, r3, #31
 ; CHECK-NEXT:    itt ne
 ; CHECK-NEXT:    ldrne r0, [r2]

 ; CHECK-NEXT:    lsls r0, r0, #30
 ; CHECK-NEXT:    itt mi
 ; CHECK-NEXT:    vmovmi r0, s14
 ; CHECK-NEXT:    strmi r0, [r1, #4]
 ; CHECK-NEXT:    adds r1, #8
 ; CHECK-NEXT:    le lr, .LBB4_2
 ; CHECK-NEXT:  .LBB4_3: @ %for.cond.cleanup
 ; CHECK-NEXT:    add sp, #8
-; CHECK-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-NEXT:    pop {r4, r5, r6, pc}
+; CHECK-NEXT:    vpop {d8, d9}
+; CHECK-NEXT:    add sp, #4
+; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, pc}
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.4:
 ; CHECK-NEXT:  .LCPI4_0:
 ; CHECK-NEXT:    .long 1 @ 0x1
 ; CHECK-NEXT:    .long 0 @ 0x0
 ; CHECK-NEXT:    .long 1 @ 0x1
 ; CHECK-NEXT:    .long 0 @ 0x0
 entry:

llvm/test/CodeGen/Thumb2/mve-abs.ll

entry:
   ret <4 x i32> %2
 }
 
 define arm_aapcs_vfpcc <2 x i64> @abs_v2i64(<2 x i64> %s1) {
 ; CHECK-LABEL: abs_v2i64:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    vmov r1, s0
+; CHECK-NEXT:    vmov r3, s3
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    adds.w r1, r1, r0, asr #31
-; CHECK-NEXT:    adc.w r2, r0, r0, asr #31
-; CHECK-NEXT:    eor.w r2, r2, r0, asr #31
-; CHECK-NEXT:    eor.w r0, r1, r0, asr #31
-; CHECK-NEXT:    vmov.32 q1[0], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    vmov.32 q1[1], r2
-; CHECK-NEXT:    adds.w r1, r1, r0, asr #31
+; CHECK-NEXT:    adc.w r12, r0, r0, asr #31
 ; CHECK-NEXT:    eor.w r1, r1, r0, asr #31
-; CHECK-NEXT:    vmov.32 q1[2], r1
-; CHECK-NEXT:    adc.w r1, r0, r0, asr #31
-; CHECK-NEXT:    eor.w r0, r1, r0, asr #31
-; CHECK-NEXT:    vmov.32 q1[3], r0
-; CHECK-NEXT:    vmov q0, q1
+; CHECK-NEXT:    adds.w r2, r2, r3, asr #31
+; CHECK-NEXT:    eor.w r0, r12, r0, asr #31
+; CHECK-NEXT:    eor.w r2, r2, r3, asr #31
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r1
+; CHECK-NEXT:    adc.w r1, r3, r3, asr #31
+; CHECK-NEXT:    eor.w r1, r1, r3, asr #31
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
 ; CHECK-NEXT:    bx lr
 entry:
   %0 = icmp slt <2 x i64> %s1, zeroinitializer
   %1 = sub nsw <2 x i64> zeroinitializer, %s1
   %2 = select <2 x i1> %0, <2 x i64> %1, <2 x i64> %s1
   ret <2 x i64> %2
 }

llvm/test/CodeGen/Thumb2/mve-div-expand.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve,+fullfp16 -verify-machineinstrs %s -o - | FileCheck %s --check-prefix=CHECK
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp -verify-machineinstrs %s -o - | FileCheck %s --check-prefix=CHECK
 
 define arm_aapcs_vfpcc <4 x i32> @udiv_i32(<4 x i32> %in1, <4 x i32> %in2) {
 ; CHECK-LABEL: udiv_i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s4
 ; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    udiv r0, r1, r0
-; CHECK-NEXT:    vmov r1, s5
-; CHECK-NEXT:    vmov r2, s1
-; CHECK-NEXT:    vmov.32 q2[0], r0
+; CHECK-NEXT:    vmov r1, s6
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    udiv r1, r2, r1
-; CHECK-NEXT:    vmov r0, s6
-; CHECK-NEXT:    vmov.32 q2[1], r1
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    udiv r0, r1, r0
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov r0, s7
+; CHECK-NEXT:    vmov r2, s3
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
+; CHECK-NEXT:    vmov r0, s5
+; CHECK-NEXT:    vmov r1, s1
 ; CHECK-NEXT:    udiv r0, r1, r0
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    vmov r1, s7
+; CHECK-NEXT:    udiv r1, r2, r1
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vmov q0, q2
 ; CHECK-NEXT:    bx lr
 entry:
   %out = udiv <4 x i32> %in1, %in2
   ret <4 x i32> %out
 }
 
 define arm_aapcs_vfpcc <4 x i32> @sdiv_i32(<4 x i32> %in1, <4 x i32> %in2) {
 ; CHECK-LABEL: sdiv_i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s4
 ; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    sdiv r0, r1, r0
-; CHECK-NEXT:    vmov r1, s5
-; CHECK-NEXT:    vmov r2, s1
-; CHECK-NEXT:    vmov.32 q2[0], r0
+; CHECK-NEXT:    vmov r1, s6
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    sdiv r1, r2, r1
-; CHECK-NEXT:    vmov r0, s6
-; CHECK-NEXT:    vmov.32 q2[1], r1
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    sdiv r0, r1, r0
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov r0, s7
+; CHECK-NEXT:    vmov r2, s3
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
+; CHECK-NEXT:    vmov r0, s5
+; CHECK-NEXT:    vmov r1, s1
 ; CHECK-NEXT:    sdiv r0, r1, r0
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    vmov r1, s7
+; CHECK-NEXT:    sdiv r1, r2, r1
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vmov q0, q2
 ; CHECK-NEXT:    bx lr
 entry:
   %out = sdiv <4 x i32> %in1, %in2
   ret <4 x i32> %out
 }
 
 define arm_aapcs_vfpcc <4 x i32> @urem_i32(<4 x i32> %in1, <4 x i32> %in2) {
 ; CHECK-LABEL: urem_i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
 ; CHECK-NEXT:    vmov r0, s4
 ; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    udiv r2, r1, r0
 ; CHECK-NEXT:    mls r12, r2, r0, r1
-; CHECK-NEXT:    vmov r1, s5
-; CHECK-NEXT:    vmov r2, s1
+; CHECK-NEXT:    vmov r1, s6
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    udiv r3, r2, r1
 ; CHECK-NEXT:    mls lr, r3, r1, r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    vmov r2, s5
+; CHECK-NEXT:    vmov r3, s1
 ; CHECK-NEXT:    udiv r0, r3, r2
 ; CHECK-NEXT:    mls r0, r0, r2, r3
 ; CHECK-NEXT:    vmov r2, s7
 ; CHECK-NEXT:    vmov r3, s3
-; CHECK-NEXT:    vmov.32 q0[0], r12
+; CHECK-NEXT:    vmov q0[2], q0[0], lr, r12
 ; CHECK-NEXT:    udiv r1, r3, r2
-; CHECK-NEXT:    vmov.32 q0[1], lr
-; CHECK-NEXT:    vmov.32 q0[2], r0
 ; CHECK-NEXT:    mls r1, r1, r2, r3
-; CHECK-NEXT:    vmov.32 q0[3], r1
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
 ; CHECK-NEXT:    pop {r7, pc}
 entry:
   %out = urem <4 x i32> %in1, %in2
   ret <4 x i32> %out
 }
 
 define arm_aapcs_vfpcc <4 x i32> @srem_i32(<4 x i32> %in1, <4 x i32> %in2) {
 ; CHECK-LABEL: srem_i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
 ; CHECK-NEXT:    vmov r0, s4
 ; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    sdiv r2, r1, r0
 ; CHECK-NEXT:    mls r12, r2, r0, r1
-; CHECK-NEXT:    vmov r1, s5
-; CHECK-NEXT:    vmov r2, s1
+; CHECK-NEXT:    vmov r1, s6
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    sdiv r3, r2, r1
 ; CHECK-NEXT:    mls lr, r3, r1, r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    vmov r2, s5
+; CHECK-NEXT:    vmov r3, s1
 ; CHECK-NEXT:    sdiv r0, r3, r2
 ; CHECK-NEXT:    mls r0, r0, r2, r3
 ; CHECK-NEXT:    vmov r2, s7
 ; CHECK-NEXT:    vmov r3, s3
-; CHECK-NEXT:    vmov.32 q0[0], r12
+; CHECK-NEXT:    vmov q0[2], q0[0], lr, r12
 ; CHECK-NEXT:    sdiv r1, r3, r2
-; CHECK-NEXT:    vmov.32 q0[1], lr
-; CHECK-NEXT:    vmov.32 q0[2], r0
 ; CHECK-NEXT:    mls r1, r1, r2, r3
-; CHECK-NEXT:    vmov.32 q0[3], r1
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
 ; CHECK-NEXT:    pop {r7, pc}
 entry:
   %out = srem <4 x i32> %in1, %in2
   ret <4 x i32> %out
 }
 
 
 define arm_aapcs_vfpcc <8 x i16> @udiv_i16(<8 x i16> %in1, <8 x i16> %in2) {

 entry:
   %out = srem <16 x i8> %in1, %in2
   ret <16 x i8> %out
 }
 
 define arm_aapcs_vfpcc <2 x i64> @udiv_i64(<2 x i64> %in1, <2 x i64> %in2) {
 ; CHECK-LABEL: udiv_i64:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r7, lr}
-; CHECK-NEXT:    push {r7, lr}
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11}
 ; CHECK-NEXT:    vmov q4, q1
 ; CHECK-NEXT:    vmov q5, q0
 ; CHECK-NEXT:    vmov r0, s20
 ; CHECK-NEXT:    vmov r1, s21
 ; CHECK-NEXT:    vmov r2, s16
 ; CHECK-NEXT:    vmov r3, s17
 ; CHECK-NEXT:    bl __aeabi_uldivmod
-; CHECK-NEXT:    vmov r12, s22
-; CHECK-NEXT:    vmov lr, s23
+; CHECK-NEXT:    mov r4, r0
+; CHECK-NEXT:    mov r5, r1
+; CHECK-NEXT:    vmov r0, s22
+; CHECK-NEXT:    vmov r1, s23
 ; CHECK-NEXT:    vmov r2, s18
 ; CHECK-NEXT:    vmov r3, s19
-; CHECK-NEXT:    vmov.32 q4[0], r0
-; CHECK-NEXT:    vmov.32 q4[1], r1
-; CHECK-NEXT:    mov r0, r12
-; CHECK-NEXT:    mov r1, lr
 ; CHECK-NEXT:    bl __aeabi_uldivmod
-; CHECK-NEXT:    vmov.32 q4[2], r0
-; CHECK-NEXT:    vmov.32 q4[3], r1
-; CHECK-NEXT:    vmov q0, q4
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r4
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r5
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-NEXT:    pop {r7, pc}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %out = udiv <2 x i64> %in1, %in2
   ret <2 x i64> %out
 }
 
 define arm_aapcs_vfpcc <2 x i64> @sdiv_i64(<2 x i64> %in1, <2 x i64> %in2) {
 ; CHECK-LABEL: sdiv_i64:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r7, lr}
-; CHECK-NEXT:    push {r7, lr}
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11}
 ; CHECK-NEXT:    vmov q4, q1
 ; CHECK-NEXT:    vmov q5, q0
 ; CHECK-NEXT:    vmov r0, s20
 ; CHECK-NEXT:    vmov r1, s21
 ; CHECK-NEXT:    vmov r2, s16
 ; CHECK-NEXT:    vmov r3, s17
 ; CHECK-NEXT:    bl __aeabi_ldivmod
-; CHECK-NEXT:    vmov r12, s22
-; CHECK-NEXT:    vmov lr, s23
+; CHECK-NEXT:    mov r4, r0
+; CHECK-NEXT:    mov r5, r1
+; CHECK-NEXT:    vmov r0, s22
+; CHECK-NEXT:    vmov r1, s23
 ; CHECK-NEXT:    vmov r2, s18
 ; CHECK-NEXT:    vmov r3, s19
-; CHECK-NEXT:    vmov.32 q4[0], r0
-; CHECK-NEXT:    vmov.32 q4[1], r1
-; CHECK-NEXT:    mov r0, r12
-; CHECK-NEXT:    mov r1, lr
 ; CHECK-NEXT:    bl __aeabi_ldivmod
-; CHECK-NEXT:    vmov.32 q4[2], r0
-; CHECK-NEXT:    vmov.32 q4[3], r1
-; CHECK-NEXT:    vmov q0, q4
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r4
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r5
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-NEXT:    pop {r7, pc}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %out = sdiv <2 x i64> %in1, %in2
   ret <2 x i64> %out
 }
 
 define arm_aapcs_vfpcc <2 x i64> @urem_i64(<2 x i64> %in1, <2 x i64> %in2) {
 ; CHECK-LABEL: urem_i64:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r7, lr}
-; CHECK-NEXT:    push {r7, lr}
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11}
 ; CHECK-NEXT:    vmov q4, q1
 ; CHECK-NEXT:    vmov q5, q0
 ; CHECK-NEXT:    vmov r0, s20
 ; CHECK-NEXT:    vmov r1, s21
 ; CHECK-NEXT:    vmov r2, s16
 ; CHECK-NEXT:    vmov r3, s17
 ; CHECK-NEXT:    bl __aeabi_uldivmod
-; CHECK-NEXT:    vmov r12, s18
-; CHECK-NEXT:    vmov lr, s19
-; CHECK-NEXT:    vmov.32 q4[0], r2
+; CHECK-NEXT:    mov r4, r2
+; CHECK-NEXT:    mov r5, r3
 ; CHECK-NEXT:    vmov r0, s22
-; CHECK-NEXT:    vmov.32 q4[1], r3
 ; CHECK-NEXT:    vmov r1, s23
-; CHECK-NEXT:    mov r2, r12
-; CHECK-NEXT:    mov r3, lr
+; CHECK-NEXT:    vmov r2, s18
+; CHECK-NEXT:    vmov r3, s19
 ; CHECK-NEXT:    bl __aeabi_uldivmod
-; CHECK-NEXT:    vmov.32 q4[2], r2
-; CHECK-NEXT:    vmov.32 q4[3], r3
-; CHECK-NEXT:    vmov q0, q4
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r4
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r5
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-NEXT:    pop {r7, pc}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %out = urem <2 x i64> %in1, %in2
   ret <2 x i64> %out
 }
 
 define arm_aapcs_vfpcc <2 x i64> @srem_i64(<2 x i64> %in1, <2 x i64> %in2) {
 ; CHECK-LABEL: srem_i64:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r7, lr}
-; CHECK-NEXT:    push {r7, lr}
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11}
 ; CHECK-NEXT:    vmov q4, q1
 ; CHECK-NEXT:    vmov q5, q0
 ; CHECK-NEXT:    vmov r0, s20
 ; CHECK-NEXT:    vmov r1, s21
 ; CHECK-NEXT:    vmov r2, s16
 ; CHECK-NEXT:    vmov r3, s17
 ; CHECK-NEXT:    bl __aeabi_ldivmod
-; CHECK-NEXT:    vmov r12, s18
-; CHECK-NEXT:    vmov lr, s19
-; CHECK-NEXT:    vmov.32 q4[0], r2
+; CHECK-NEXT:    mov r4, r2
+; CHECK-NEXT:    mov r5, r3
 ; CHECK-NEXT:    vmov r0, s22
-; CHECK-NEXT:    vmov.32 q4[1], r3
 ; CHECK-NEXT:    vmov r1, s23
-; CHECK-NEXT:    mov r2, r12
-; CHECK-NEXT:    mov r3, lr
+; CHECK-NEXT:    vmov r2, s18
+; CHECK-NEXT:    vmov r3, s19
 ; CHECK-NEXT:    bl __aeabi_ldivmod
-; CHECK-NEXT:    vmov.32 q4[2], r2
-; CHECK-NEXT:    vmov.32 q4[3], r3
-; CHECK-NEXT:    vmov q0, q4
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r4
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r5
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-NEXT:    pop {r7, pc}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %out = srem <2 x i64> %in1, %in2
   ret <2 x i64> %out
 }
 
 
 
 

llvm/test/CodeGen/Thumb2/mve-gather-increment.ll

 ; CHECK-NEXT:    ldr r1, [sp] @ 4-byte Reload
 ; CHECK-NEXT:    vmov q2, q0
 ; CHECK-NEXT:    dls lr, r1
 ; CHECK-NEXT:    ldr r4, [sp, #4] @ 4-byte Reload
 ; CHECK-NEXT:  .LBB11_3: @ %vector.body
 ; CHECK-NEXT:    @ Parent Loop BB11_2 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
 ; CHECK-NEXT:    vmov.u16 r7, q2[4]
-; CHECK-NEXT:    vmov.u16 r5, q2[0]
-; CHECK-NEXT:    vmov.32 q4[0], r7
-; CHECK-NEXT:    vmov.u16 r7, q2[5]
-; CHECK-NEXT:    vmov.32 q4[1], r7
-; CHECK-NEXT:    vmov.u16 r7, q2[6]
-; CHECK-NEXT:    vmov.32 q4[2], r7
+; CHECK-NEXT:    vmov.u16 r3, q2[6]
+; CHECK-NEXT:    vmov q4[2], q4[0], r3, r7
+; CHECK-NEXT:    vmov.u16 r3, q2[5]
 ; CHECK-NEXT:    vmov.u16 r7, q2[7]
-; CHECK-NEXT:    vmov.32 q4[3], r7
-; CHECK-NEXT:    vmov.32 q3[0], r5
+; CHECK-NEXT:    vmov.u16 r5, q2[0]
+; CHECK-NEXT:    vmov q4[3], q4[1], r7, r3
+; CHECK-NEXT:    vmov.u16 r6, q2[2]
 ; CHECK-NEXT:    vmovlb.s16 q4, q4
-; CHECK-NEXT:    vmov.u16 r5, q2[1]
+; CHECK-NEXT:    vmov q3[2], q3[0], r6, r5
 ; CHECK-NEXT:    vshl.i32 q4, q4, #1
-; CHECK-NEXT:    vmov.32 q3[1], r5
+; CHECK-NEXT:    vmov.u16 r5, q2[1]
 ; CHECK-NEXT:    vadd.i32 q4, q4, r0
-; CHECK-NEXT:    vmov.u16 r5, q2[2]
-; CHECK-NEXT:    vmov r7, s16
-; CHECK-NEXT:    vmov.32 q3[2], r5
-; CHECK-NEXT:    vmov.u16 r5, q2[3]
-; CHECK-NEXT:    vmov r3, s17
-; CHECK-NEXT:    vmov.32 q3[3], r5
-; CHECK-NEXT:    vadd.i16 q2, q2, q1
+; CHECK-NEXT:    vmov.u16 r6, q2[3]
+; CHECK-NEXT:    vmov r3, s16
+; CHECK-NEXT:    vmov q3[3], q3[1], r6, r5
 ; CHECK-NEXT:    vmovlb.s16 q3, q3
+; CHECK-NEXT:    vmov r7, s17
 ; CHECK-NEXT:    vshl.i32 q3, q3, #1
+; CHECK-NEXT:    vadd.i16 q2, q2, q1
 ; CHECK-NEXT:    vadd.i32 q3, q3, r0
 ; CHECK-NEXT:    vmov r5, s15
 ; CHECK-NEXT:    vmov r6, s14
 ; CHECK-NEXT:    vmov r12, s13
-; CHECK-NEXT:    ldrh.w r11, [r7]
-; CHECK-NEXT:    vmov r7, s12
-; CHECK-NEXT:    ldrh r3, [r3]
+; CHECK-NEXT:    ldrh.w r11, [r3]
+; CHECK-NEXT:    vmov r3, s12
+; CHECK-NEXT:    ldrh r7, [r7]
 ; CHECK-NEXT:    ldrh.w r9, [r5]
 ; CHECK-NEXT:    vmov r5, s18
 ; CHECK-NEXT:    ldrh.w r10, [r6]
 ; CHECK-NEXT:    vmov r6, s19
 ; CHECK-NEXT:    ldrh.w r1, [r12]
-; CHECK-NEXT:    ldrh r7, [r7]
-; CHECK-NEXT:    vmov.16 q3[0], r7
+; CHECK-NEXT:    ldrh r3, [r3]
+; CHECK-NEXT:    vmov.16 q3[0], r3
 ; CHECK-NEXT:    vmov.16 q3[1], r1
 ; CHECK-NEXT:    vmov.16 q3[2], r10
-; CHECK-NEXT:    vmov.16 q3[3], r9
-; CHECK-NEXT:    vmov.16 q3[4], r11
 ; CHECK-NEXT:    ldrh r5, [r5]
-; CHECK-NEXT:    vmov.16 q3[5], r3
+; CHECK-NEXT:    vmov.16 q3[3], r9
 ; CHECK-NEXT:    ldrh r6, [r6]
+; CHECK-NEXT:    vmov.16 q3[4], r11
+; CHECK-NEXT:    vmov.16 q3[5], r7
 ; CHECK-NEXT:    vmov.16 q3[6], r5
 ; CHECK-NEXT:    vmov.16 q3[7], r6
 ; CHECK-NEXT:    vstrb.8 q3, [r4], #16
 ; CHECK-NEXT:    le lr, .LBB11_3
 ; CHECK-NEXT:  @ %bb.4: @ %middle.block
 ; CHECK-NEXT:    @ in Loop: Header=BB11_2 Depth=1
 ; CHECK-NEXT:    cmp r8, r2
 ; CHECK-NEXT:    bne .LBB11_2

 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    .pad #104
 ; CHECK-NEXT:    sub sp, #104
 ; CHECK-NEXT:    cmp r2, #1
 ; CHECK-NEXT:    str r1, [sp, #60] @ 4-byte Spill
 ; CHECK-NEXT:    blt.w .LBB12_5
 ; CHECK-NEXT:  @ %bb.1: @ %vector.ph.preheader
-; CHECK-NEXT:    bic r8, r2, #7
+; CHECK-NEXT:    bic r1, r2, #7
 ; CHECK-NEXT:    adr r6, .LCPI12_2
-; CHECK-NEXT:    sub.w r3, r8, #8
+; CHECK-NEXT:    sub.w r3, r1, #8
 ; CHECK-NEXT:    vldrw.u32 q0, [r6]
+; CHECK-NEXT:    str r1, [sp, #56] @ 4-byte Spill
 ; CHECK-NEXT:    movs r7, #1
-; CHECK-NEXT:    vmov.i16 q3, #0x18
 ; CHECK-NEXT:    add.w r1, r7, r3, lsr #3
 ; CHECK-NEXT:    adr r3, .LCPI12_0
 ; CHECK-NEXT:    vstrw.32 q0, [sp, #32] @ 16-byte Spill
 ; CHECK-NEXT:    vldrw.u32 q0, [r3]
 ; CHECK-NEXT:    adr r7, .LCPI12_1
-; CHECK-NEXT:    str r1, [sp, #56] @ 4-byte Spill
+; CHECK-NEXT:    vmov.i16 q3, #0x18
 ; CHECK-NEXT:    vstrw.32 q0, [sp, #16] @ 16-byte Spill
 ; CHECK-NEXT:    vldrw.u32 q0, [r7]
+; CHECK-NEXT:    str r1, [sp, #52] @ 4-byte Spill
 ; CHECK-NEXT:    vstrw.32 q3, [sp, #64] @ 16-byte Spill
 ; CHECK-NEXT:    vstrw.32 q0, [sp] @ 16-byte Spill
 ; CHECK-NEXT:  .LBB12_2: @ %vector.ph
 ; CHECK-NEXT:    @ =>This Loop Header: Depth=1
 ; CHECK-NEXT:    @ Child Loop BB12_3 Depth 2
-; CHECK-NEXT:    ldr r1, [sp, #56] @ 4-byte Reload
+; CHECK-NEXT:    ldr r1, [sp, #52] @ 4-byte Reload
 ; CHECK-NEXT:    dls lr, r1
 ; CHECK-NEXT:    ldr r4, [sp, #60] @ 4-byte Reload
 ; CHECK-NEXT:    vldrw.u32 q7, [sp, #16] @ 16-byte Reload
 ; CHECK-NEXT:    vldrw.u32 q5, [sp, #32] @ 16-byte Reload
 ; CHECK-NEXT:    vldrw.u32 q6, [sp] @ 16-byte Reload
 ; CHECK-NEXT:  .LBB12_3: @ %vector.body
 ; CHECK-NEXT:    @ Parent Loop BB12_2 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
 ; CHECK-NEXT:    vmov.u16 r3, q5[0]
-; CHECK-NEXT:    vmov.u16 r7, q7[4]
-; CHECK-NEXT:    vmov.32 q0[0], r3
+; CHECK-NEXT:    vmov.u16 r5, q5[2]
+; CHECK-NEXT:    vmov q0[2], q0[0], r5, r3
 ; CHECK-NEXT:    vmov.u16 r3, q5[1]
-; CHECK-NEXT:    vmov.32 q0[1], r3
-; CHECK-NEXT:    vmov.u16 r3, q5[2]
-; CHECK-NEXT:    vmov.32 q0[2], r3
-; CHECK-NEXT:    vmov.u16 r3, q5[3]
-; CHECK-NEXT:    vmov.32 q0[3], r3
-; CHECK-NEXT:    vmov.u16 r12, q6[0]
+; CHECK-NEXT:    vmov.u16 r5, q5[3]
+; CHECK-NEXT:    vmov.u16 r7, q7[4]
+; CHECK-NEXT:    vmov q0[3], q0[1], r5, r3
+; CHECK-NEXT:    vmov.u16 r5, q5[6]
 ; CHECK-NEXT:    vmovlb.s16 q0, q0
-; CHECK-NEXT:    vmov.32 q1[0], r12
+; CHECK-NEXT:    vmov.u16 r12, q7[6]
 ; CHECK-NEXT:    vshl.i32 q0, q0, #1
-; CHECK-NEXT:    vmov.u16 r1, q6[1]
+; CHECK-NEXT:    vmov.u16 r1, q7[7]
 ; CHECK-NEXT:    vadd.i32 q2, q0, r0
-; CHECK-NEXT:    vmov.32 q1[1], r1
 ; CHECK-NEXT:    vmov r3, s10
-; CHECK-NEXT:    vmov.u16 r1, q6[2]
-; CHECK-NEXT:    vmov.32 q1[2], r1
-; CHECK-NEXT:    vmov.u16 r1, q6[3]
-; CHECK-NEXT:    vmov.32 q1[3], r1
-; CHECK-NEXT:    vmov.u16 r1, q6[4]
-; CHECK-NEXT:    vmovlb.s16 q1, q1
 ; CHECK-NEXT:    vmov r6, s11
-; CHECK-NEXT:    vshl.i32 q1, q1, #1
-; CHECK-NEXT:    vadd.i32 q4, q1, r0
 ; CHECK-NEXT:    ldrh.w r9, [r3]
 ; CHECK-NEXT:    vmov.u16 r3, q5[4]
-; CHECK-NEXT:    vmov.32 q0[0], r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r5, r3
 ; CHECK-NEXT:    vmov.u16 r3, q5[5]
-; CHECK-NEXT:    vmov.32 q0[1], r3
-; CHECK-NEXT:    vmov.u16 r3, q5[6]
-; CHECK-NEXT:    vmov.32 q0[2], r3
-; CHECK-NEXT:    vmov.u16 r3, q5[7]
-; CHECK-NEXT:    vmov.32 q0[3], r3
+; CHECK-NEXT:    vmov.u16 r5, q5[7]
 ; CHECK-NEXT:    ldrh r6, [r6]
+; CHECK-NEXT:    vmov q0[3], q0[1], r5, r3
 ; CHECK-NEXT:    vmovlb.s16 q0, q0
 ; CHECK-NEXT:    vshl.i32 q0, q0, #1
 ; CHECK-NEXT:    vadd.i32 q0, q0, r0
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r5, s3
 ; CHECK-NEXT:    ldrh.w r10, [r3]
 ; CHECK-NEXT:    vmov r3, s1
 ; CHECK-NEXT:    ldrh r5, [r5]
 ; CHECK-NEXT:    ldrh.w r11, [r3]
 ; CHECK-NEXT:    vmov r3, s2
-; CHECK-NEXT:    vmov.32 q0[0], r7
+; CHECK-NEXT:    vmov q0[2], q0[0], r12, r7
 ; CHECK-NEXT:    vmov.u16 r7, q7[5]
-; CHECK-NEXT:    vmov.32 q0[1], r7
-; CHECK-NEXT:    vmov.u16 r7, q7[6]
-; CHECK-NEXT:    vmov.32 q0[2], r7
-; CHECK-NEXT:    vmov.u16 r7, q7[7]
-; CHECK-NEXT:    vmov.32 q0[3], r7
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r7
 ; CHECK-NEXT:    vmovlb.s16 q0, q0
 ; CHECK-NEXT:    vshl.i32 q0, q0, #1
 ; CHECK-NEXT:    vadd.i32 q0, q0, r0
-; CHECK-NEXT:    vmov r7, s2
+; CHECK-NEXT:    vmov r1, s2
 ; CHECK-NEXT:    vstrw.32 q0, [sp, #80] @ 16-byte Spill
-; CHECK-NEXT:    vmov.32 q0[0], r1
+; CHECK-NEXT:    ldrh.w r8, [r3]
+; CHECK-NEXT:    vmov.u16 r3, q6[2]
+; CHECK-NEXT:    ldrh r7, [r1]
+; CHECK-NEXT:    vmov.u16 r1, q6[0]
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r1
+; CHECK-NEXT:    vmov.u16 r1, q6[1]
+; CHECK-NEXT:    vmov.u16 r3, q6[3]
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r1
+; CHECK-NEXT:    vmov.u16 r1, q6[4]
+; CHECK-NEXT:    vmov.u16 r3, q6[6]
+; CHECK-NEXT:    vmovlb.s16 q1, q1
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r1
 ; CHECK-NEXT:    vmov.u16 r1, q6[5]
-; CHECK-NEXT:    vmov.32 q0[1], r1
-; CHECK-NEXT:    vmov.u16 r1, q6[6]
-; CHECK-NEXT:    vmov.32 q0[2], r1
-; CHECK-NEXT:    vmov.u16 r1, q6[7]
-; CHECK-NEXT:    vmov.32 q0[3], r1
+; CHECK-NEXT:    vmov.u16 r3, q6[7]
+; CHECK-NEXT:    vshl.i32 q1, q1, #1
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r1
 ; CHECK-NEXT:    vmov.u16 r1, q7[0]
-; CHECK-NEXT:    vmov.32 q3[0], r1
+; CHECK-NEXT:    vmov.u16 r3, q7[2]
+; CHECK-NEXT:    vadd.i32 q4, q1, r0
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r1
 ; CHECK-NEXT:    vmov.u16 r1, q7[1]
-; CHECK-NEXT:    vmov.32 q3[1], r1
-; CHECK-NEXT:    vmov.u16 r1, q7[2]
-; CHECK-NEXT:    vmov.32 q3[2], r1
-; CHECK-NEXT:    vmov.u16 r1, q7[3]
-; CHECK-NEXT:    vmov.32 q3[3], r1
-; CHECK-NEXT:    vmov r1, s8
+; CHECK-NEXT:    vmov.u16 r3, q7[3]
 ; CHECK-NEXT:    vmovlb.s16 q0, q0
-; CHECK-NEXT:    vmovlb.s16 q3, q3
+; CHECK-NEXT:    vmov q3[3], q3[1], r3, r1
+; CHECK-NEXT:    vmov r1, s8
 ; CHECK-NEXT:    vshl.i32 q0, q0, #1
-; CHECK-NEXT:    vshl.i32 q3, q3, #1
+; CHECK-NEXT:    vmovlb.s16 q3, q3
 ; CHECK-NEXT:    vadd.i32 q0, q0, r0
+; CHECK-NEXT:    vshl.i32 q3, q3, #1
 ; CHECK-NEXT:    vadd.i32 q3, q3, r0
-; CHECK-NEXT:    ldrh r3, [r3]
-; CHECK-NEXT:    ldrh r7, [r7]
 ; CHECK-NEXT:    ldrh r1, [r1]
 ; CHECK-NEXT:    vmov.16 q1[0], r1
 ; CHECK-NEXT:    vmov r1, s9
 ; CHECK-NEXT:    ldrh r1, [r1]
 ; CHECK-NEXT:    vmov.16 q1[1], r1
 ; CHECK-NEXT:    vmov r1, s16
 ; CHECK-NEXT:    vmov.16 q1[2], r9
 ; CHECK-NEXT:    vmov.16 q1[3], r6
 ; CHECK-NEXT:    vmov.16 q1[4], r10
 ; CHECK-NEXT:    vmov.16 q1[5], r11
-; CHECK-NEXT:    vmov.16 q1[6], r3
+; CHECK-NEXT:    vmov.16 q1[6], r8
 ; CHECK-NEXT:    vmov.16 q1[7], r5
 ; CHECK-NEXT:    ldrh r1, [r1]
 ; CHECK-NEXT:    vmov.16 q2[0], r1
 ; CHECK-NEXT:    vmov r1, s17
 ; CHECK-NEXT:    ldrh r1, [r1]
 ; CHECK-NEXT:    vmov.16 q2[1], r1
 ; CHECK-NEXT:    vmov r1, s18
 ; CHECK-NEXT:    ldrh r1, [r1]

 ; CHECK-NEXT:    ldrh r1, [r1]
 ; CHECK-NEXT:    vmov.16 q0[7], r1
 ; CHECK-NEXT:    vadd.i16 q0, q0, q2
 ; CHECK-NEXT:    vadd.i16 q0, q0, q1
 ; CHECK-NEXT:    vstrb.8 q0, [r4], #16
 ; CHECK-NEXT:    le lr, .LBB12_3
 ; CHECK-NEXT:  @ %bb.4: @ %middle.block
 ; CHECK-NEXT:    @ in Loop: Header=BB12_2 Depth=1
-; CHECK-NEXT:    cmp r8, r2
+; CHECK-NEXT:    ldr r1, [sp, #56] @ 4-byte Reload
+; CHECK-NEXT:    cmp r1, r2
 ; CHECK-NEXT:    bne.w .LBB12_2
 ; CHECK-NEXT:  .LBB12_5: @ %for.cond.cleanup
 ; CHECK-NEXT:    add sp, #104
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    add sp, #4
 ; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.6:

llvm/test/CodeGen/Thumb2/mve-gather-ind32-unscaled.ll

 
 ; VLDRW.u32 Qd, [P, 4]
 define arm_aapcs_vfpcc <4 x i32> @qi4(<4 x i32*> %p) {
 ; CHECK-LABEL: qi4:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.i32 q1, #0x10
 ; CHECK-NEXT:    vadd.i32 q0, q0, q1
 ; CHECK-NEXT:    vmov r0, s0
-; CHECK-NEXT:    vmov r3, s1
 ; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    vmov r2, s3
+; CHECK-NEXT:    vmov r2, s1
+; CHECK-NEXT:    vmov r3, s3
 ; CHECK-NEXT:    ldr r0, [r0]
-; CHECK-NEXT:    ldr r3, [r3]
-; CHECK-NEXT:    vmov.32 q0[0], r0
 ; CHECK-NEXT:    ldr r1, [r1]
-; CHECK-NEXT:    vmov.32 q0[1], r3
 ; CHECK-NEXT:    ldr r2, [r2]
-; CHECK-NEXT:    vmov.32 q0[2], r1
-; CHECK-NEXT:    vmov.32 q0[3], r2
+; CHECK-NEXT:    ldr r3, [r3]
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r2
 ; CHECK-NEXT:    bx lr
 entry:
   %g = getelementptr inbounds i32, <4 x i32*> %p, i32 4
   %gather = call <4 x i32> @llvm.masked.gather.v4i32.v4p0i32(<4 x i32*> %g, i32 1, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i32> undef)
   ret <4 x i32> %gather
 }
 
 declare <4 x i8>  @llvm.masked.gather.v4i8.v4p0i8(<4 x i8*>, i32, <4 x i1>, <4 x i8>)
 declare <4 x i16> @llvm.masked.gather.v4i16.v4p0i16(<4 x i16*>, i32, <4 x i1>, <4 x i16>)
 declare <4 x i32> @llvm.masked.gather.v4i32.v4p0i32(<4 x i32*>, i32, <4 x i1>, <4 x i32>)
 declare <4 x half> @llvm.masked.gather.v4f16.v4p0f16(<4 x half*>, i32, <4 x i1>, <4 x half>)
 declare <4 x float> @llvm.masked.gather.v4f32.v4p0f32(<4 x float*>, i32, <4 x i1>, <4 x float>)

llvm/test/CodeGen/Thumb2/mve-gather-ind8-unscaled.ll

 }
 
 define arm_aapcs_vfpcc <2 x i8> @unscaled_v2i8_i8(i8* %base, <2 x i8>* %offptr) {
 ; CHECK-LABEL: unscaled_v2i8_i8:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    ldrb r2, [r1]
 ; CHECK-NEXT:    vmov.i32 q0, #0xff
 ; CHECK-NEXT:    ldrb r1, [r1, #1]
-; CHECK-NEXT:    vmov.32 q1[0], r2
-; CHECK-NEXT:    vmov.32 q1[2], r1
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r2
 ; CHECK-NEXT:    vand q0, q1, q0
 ; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    ldrb r1, [r0, r1]
 ; CHECK-NEXT:    ldrb r0, [r0, r2]
-; CHECK-NEXT:    vmov.32 q0[0], r1
-; CHECK-NEXT:    vmov.32 q0[2], r0
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r1
 ; CHECK-NEXT:    bx lr
 entry:
   %offs = load <2 x i8>, <2 x i8>* %offptr, align 1
   %offs.zext = zext <2 x i8> %offs to <2 x i32>
   %ptrs = getelementptr inbounds i8, i8* %base, <2 x i32> %offs.zext
   %gather = call <2 x i8> @llvm.masked.gather.v2i8.v2p0i8(<2 x i8*> %ptrs, i32 1, <2 x i1> <i1 true, i1 true>, <2 x i8> undef)
   ret <2 x i8> %gather
 }

llvm/test/CodeGen/Thumb2/mve-gather-ptrs.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp -enable-arm-maskedldst %s -o - | FileCheck %s
 
 ; i32
 
 define arm_aapcs_vfpcc <2 x i32> @ptr_v2i32(<2 x i32*>* %offptr) {
 ; CHECK-LABEL: ptr_v2i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    ldrd r1, r0, [r0]
 ; CHECK-NEXT:    ldr r0, [r0]
 ; CHECK-NEXT:    ldr r1, [r1]
-; CHECK-NEXT:    vmov.32 q0[0], r1
-; CHECK-NEXT:    vmov.32 q0[2], r0
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r1
 ; CHECK-NEXT:    bx lr
 entry:
   %offs = load <2 x i32*>, <2 x i32*>* %offptr, align 4
   %gather = call <2 x i32> @llvm.masked.gather.v2i32.v2p0i32(<2 x i32*> %offs, i32 4, <2 x i1> <i1 true, i1 true>, <2 x i32> undef)
   ret <2 x i32> %gather
 }
 
 define arm_aapcs_vfpcc <4 x i32> @ptr_v4i32(<4 x i32*>* %offptr) {

 
 define arm_aapcs_vfpcc <8 x i32> @ptr_v8i32(<8 x i32*>* %offptr) {
 ; CHECK-LABEL: ptr_v8i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r7, lr}
 ; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    vldrw.u32 q0, [r0]
 ; CHECK-NEXT:    vldrw.u32 q1, [r0, #16]
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    vmov r2, s3
-; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    vmov r1, s0
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r5, s1
+; CHECK-NEXT:    vmov r3, s1
 ; CHECK-NEXT:    vmov r4, s7
+; CHECK-NEXT:    vmov r5, s3
 ; CHECK-NEXT:    ldr.w r12, [r1]
-; CHECK-NEXT:    vmov r1, s5
+; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    ldr.w lr, [r2]
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    ldr r3, [r3]
+; CHECK-NEXT:    vmov r2, s5
 ; CHECK-NEXT:    ldr r0, [r0]
-; CHECK-NEXT:    ldr r5, [r5]
-; CHECK-NEXT:    vmov.32 q0[0], r3
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    ldr r3, [r3]
+; CHECK-NEXT:    vmov q0[2], q0[0], lr, r12
 ; CHECK-NEXT:    ldr r4, [r4]
-; CHECK-NEXT:    vmov.32 q0[1], r5
-; CHECK-NEXT:    vmov.32 q0[2], r12
-; CHECK-NEXT:    vmov.32 q0[3], lr
+; CHECK-NEXT:    ldr r5, [r5]
+; CHECK-NEXT:    vmov q0[3], q0[1], r5, r3
 ; CHECK-NEXT:    ldr r1, [r1]
 ; CHECK-NEXT:    ldr r2, [r2]
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vmov.32 q1[3], r4
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r4, r2
 ; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %offs = load <8 x i32*>, <8 x i32*>* %offptr, align 4
   %gather = call <8 x i32> @llvm.masked.gather.v8i32.v8p0i32(<8 x i32*> %offs, i32 4, <8 x i1> <i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true>, <8 x i32> undef)
   ret <8 x i32> %gather
 }
 
 define arm_aapcs_vfpcc <16 x i32> @ptr_v16i32(<16 x i32*>* %offptr) {
 ; CHECK-LABEL: ptr_v16i32:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, r5, r6, lr}
-; CHECK-NEXT:    push {r4, r5, r6, lr}
+; CHECK-NEXT:    .save {r4, r5, r6, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r6, r7, lr}
 ; CHECK-NEXT:    vldrw.u32 q2, [r0, #32]
 ; CHECK-NEXT:    vldrw.u32 q0, [r0, #48]
 ; CHECK-NEXT:    vldrw.u32 q1, [r0]
 ; CHECK-NEXT:    vldrw.u32 q3, [r0, #16]
-; CHECK-NEXT:    vmov r1, s10
+; CHECK-NEXT:    vmov r1, s8
 ; CHECK-NEXT:    vmov r5, s4
+; CHECK-NEXT:    vmov r6, s6
 ; CHECK-NEXT:    vmov r2, s3
+; CHECK-NEXT:    vmov r7, s7
 ; CHECK-NEXT:    vmov r0, s12
-; CHECK-NEXT:    vmov r6, s7
-; CHECK-NEXT:    vmov r4, s11
+; CHECK-NEXT:    vmov r4, s10
 ; CHECK-NEXT:    ldr.w r12, [r1]
 ; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    ldr r5, [r5]
+; CHECK-NEXT:    ldr r6, [r6]
 ; CHECK-NEXT:    ldr r2, [r2]
+; CHECK-NEXT:    ldr r7, [r7]
 ; CHECK-NEXT:    ldr r0, [r0]
-; CHECK-NEXT:    ldr r6, [r6]
 ; CHECK-NEXT:    ldr r4, [r4]
 ; CHECK-NEXT:    ldr.w lr, [r1]
-; CHECK-NEXT:    vmov r1, s1
-; CHECK-NEXT:    ldr r3, [r1]
 ; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    vmov.32 q0[0], r5
-; CHECK-NEXT:    vmov r5, s5
+; CHECK-NEXT:    ldr r3, [r1]
+; CHECK-NEXT:    vmov r1, s1
+; CHECK-NEXT:    vmov q0[2], q0[0], r6, r5
+; CHECK-NEXT:    vmov r6, s5
+; CHECK-NEXT:    vmov r5, s9
 ; CHECK-NEXT:    ldr r1, [r1]
+; CHECK-NEXT:    ldr r6, [r6]
 ; CHECK-NEXT:    ldr r5, [r5]
-; CHECK-NEXT:    vmov.32 q0[1], r5
-; CHECK-NEXT:    vmov r5, s6
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r7, r6
+; CHECK-NEXT:    vmov r6, s14
+; CHECK-NEXT:    ldr r6, [r6]
+; CHECK-NEXT:    vmov q1[2], q1[0], r6, r0
 ; CHECK-NEXT:    vmov r0, s13
-; CHECK-NEXT:    ldr r5, [r5]
+; CHECK-NEXT:    vmov r6, s15
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, lr
+; CHECK-NEXT:    vmov q3[3], q3[1], r2, r1
 ; CHECK-NEXT:    ldr r0, [r0]
-; CHECK-NEXT:    vmov.32 q0[2], r5
-; CHECK-NEXT:    vmov r5, s8
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov r0, s14
-; CHECK-NEXT:    vmov.32 q0[3], r6
-; CHECK-NEXT:    ldr r5, [r5]
-; CHECK-NEXT:    ldr r0, [r0]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov r0, s15
-; CHECK-NEXT:    vmov.32 q3[0], lr
-; CHECK-NEXT:    vmov.32 q3[1], r3
-; CHECK-NEXT:    vmov.32 q3[2], r1
-; CHECK-NEXT:    vmov.32 q3[3], r2
-; CHECK-NEXT:    ldr r0, [r0]
-; CHECK-NEXT:    vmov.32 q1[3], r0
-; CHECK-NEXT:    vmov r0, s9
-; CHECK-NEXT:    vmov.32 q2[0], r5
+; CHECK-NEXT:    ldr r6, [r6]
+; CHECK-NEXT:    vmov q1[3], q1[1], r6, r0
+; CHECK-NEXT:    vmov r0, s11
+; CHECK-NEXT:    vmov q2[2], q2[0], r4, r12
 ; CHECK-NEXT:    ldr r0, [r0]
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov.32 q2[2], r12
-; CHECK-NEXT:    vmov.32 q2[3], r4
-; CHECK-NEXT:    pop {r4, r5, r6, pc}
+; CHECK-NEXT:    vmov q2[3], q2[1], r0, r5
+; CHECK-NEXT:    pop {r4, r5, r6, r7, pc}
 entry:
   %offs = load <16 x i32*>, <16 x i32*>* %offptr, align 4
   %gather = call <16 x i32> @llvm.masked.gather.v16i32.v16p0i32(<16 x i32*> %offs, i32 4, <16 x i1> <i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true>, <16 x i32> undef)
   ret <16 x i32> %gather
 }
 
 ; f32
 

 }
 
 define arm_aapcs_vfpcc <2 x i32> @ptr_v2i16_sext(<2 x i16*>* %offptr) {
 ; CHECK-LABEL: ptr_v2i16_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    ldrd r1, r0, [r0]
 ; CHECK-NEXT:    ldrsh.w r0, [r0]
 ; CHECK-NEXT:    ldrsh.w r1, [r1]
-; CHECK-NEXT:    asrs r2, r0, #31
-; CHECK-NEXT:    vmov.32 q0[0], r1
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r1
+; CHECK-NEXT:    asrs r0, r0, #31
 ; CHECK-NEXT:    asrs r1, r1, #31
-; CHECK-NEXT:    vmov.32 q0[1], r1
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    vmov.32 q0[3], r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r0, r1
 ; CHECK-NEXT:    bx lr
 entry:
   %offs = load <2 x i16*>, <2 x i16*>* %offptr, align 4
   %gather = call <2 x i16> @llvm.masked.gather.v2i16.v2p0i16(<2 x i16*> %offs, i32 2, <2 x i1> <i1 true, i1 true>, <2 x i16> undef)
   %ext = sext <2 x i16> %gather to <2 x i32>
   ret <2 x i32> %ext
 }
 
 define arm_aapcs_vfpcc <2 x i32> @ptr_v2i16_zext(<2 x i16*>* %offptr) {
 ; CHECK-LABEL: ptr_v2i16_zext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    ldrd r1, r0, [r0]
 ; CHECK-NEXT:    vmov.i64 q0, #0xffff
 ; CHECK-NEXT:    ldrh r0, [r0]
 ; CHECK-NEXT:    ldrh r1, [r1]
-; CHECK-NEXT:    vmov.32 q1[0], r1
-; CHECK-NEXT:    vmov.32 q1[2], r0
+; CHECK-NEXT:    vmov q1[2], q1[0], r0, r1
 ; CHECK-NEXT:    vand q0, q1, q0
 ; CHECK-NEXT:    bx lr
 entry:
   %offs = load <2 x i16*>, <2 x i16*>* %offptr, align 4
   %gather = call <2 x i16> @llvm.masked.gather.v2i16.v2p0i16(<2 x i16*> %offs, i32 2, <2 x i1> <i1 true, i1 true>, <2 x i16> undef)
   %ext = zext <2 x i16> %gather to <2 x i32>
   ret <2 x i32> %ext
 }
 
 define arm_aapcs_vfpcc <4 x i32> @ptr_v4i16_sext(<4 x i16*>* %offptr) {
 ; CHECK-LABEL: ptr_v4i16_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vldrw.u32 q0, [r0]
-; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    vmov r0, s1
-; CHECK-NEXT:    vmov r3, s2
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    ldrh r2, [r2]
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    vmov r1, s1
+; CHECK-NEXT:    vmov r3, s3
 ; CHECK-NEXT:    ldrh r0, [r0]
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    ldrh r3, [r3]
-; CHECK-NEXT:    vmov.32 q0[1], r0
+; CHECK-NEXT:    ldrh r2, [r2]
 ; CHECK-NEXT:    ldrh r1, [r1]
-; CHECK-NEXT:    vmov.32 q0[2], r3
-; CHECK-NEXT:    vmov.32 q0[3], r1
+; CHECK-NEXT:    ldrh r3, [r3]
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r1
 ; CHECK-NEXT:    vmovlb.s16 q0, q0
 ; CHECK-NEXT:    bx lr
 entry:
   %offs = load <4 x i16*>, <4 x i16*>* %offptr, align 4
   %gather = call <4 x i16> @llvm.masked.gather.v4i16.v4p0i16(<4 x i16*> %offs, i32 2, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i16> undef)
   %ext = sext <4 x i16> %gather to <4 x i32>
   ret <4 x i32> %ext
 }
 
 define arm_aapcs_vfpcc <4 x i32> @ptr_v4i16_zext(<4 x i16*>* %offptr) {
 ; CHECK-LABEL: ptr_v4i16_zext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vldrw.u32 q0, [r0]
-; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    vmov r0, s1
-; CHECK-NEXT:    vmov r3, s2
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    ldrh r2, [r2]
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    vmov r1, s1
+; CHECK-NEXT:    vmov r3, s3
 ; CHECK-NEXT:    ldrh r0, [r0]
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    ldrh r3, [r3]
-; CHECK-NEXT:    vmov.32 q0[1], r0
+; CHECK-NEXT:    ldrh r2, [r2]
 ; CHECK-NEXT:    ldrh r1, [r1]
-; CHECK-NEXT:    vmov.32 q0[2], r3
-; CHECK-NEXT:    vmov.32 q0[3], r1
+; CHECK-NEXT:    ldrh r3, [r3]
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r1
 ; CHECK-NEXT:    vmovlb.u16 q0, q0
 ; CHECK-NEXT:    bx lr
 entry:
   %offs = load <4 x i16*>, <4 x i16*>* %offptr, align 4
   %gather = call <4 x i16> @llvm.masked.gather.v4i16.v4p0i16(<4 x i16*> %offs, i32 2, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i16> undef)
   %ext = zext <4 x i16> %gather to <4 x i32>
   ret <4 x i32> %ext
 }
 
 define arm_aapcs_vfpcc <8 x i32> @ptr_v8i16_sext(<8 x i16*>* %offptr) {
 ; CHECK-LABEL: ptr_v8i16_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r7, lr}
 ; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    vldrw.u32 q0, [r0]
 ; CHECK-NEXT:    vldrw.u32 q1, [r0, #16]
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    vmov r2, s3
-; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    vmov r1, s0
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r5, s1
+; CHECK-NEXT:    vmov r3, s1
 ; CHECK-NEXT:    vmov r4, s7
+; CHECK-NEXT:    vmov r5, s3
 ; CHECK-NEXT:    ldrh.w r12, [r1]
-; CHECK-NEXT:    vmov r1, s5
+; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    ldrh.w lr, [r2]
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    ldrh r3, [r3]
+; CHECK-NEXT:    vmov r2, s5
 ; CHECK-NEXT:    ldrh r0, [r0]
-; CHECK-NEXT:    ldrh r5, [r5]
-; CHECK-NEXT:    vmov.32 q0[0], r3
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    ldrh r3, [r3]
+; CHECK-NEXT:    vmov q0[2], q0[0], lr, r12
 ; CHECK-NEXT:    ldrh r4, [r4]
-; CHECK-NEXT:    vmov.32 q0[1], r5
-; CHECK-NEXT:    vmov.32 q0[2], r12
-; CHECK-NEXT:    vmov.32 q0[3], lr
+; CHECK-NEXT:    ldrh r5, [r5]
+; CHECK-NEXT:    vmov q0[3], q0[1], r5, r3
 ; CHECK-NEXT:    vmovlb.s16 q0, q0
 ; CHECK-NEXT:    ldrh r1, [r1]
 ; CHECK-NEXT:    ldrh r2, [r2]
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vmov.32 q1[3], r4
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r4, r2
 ; CHECK-NEXT:    vmovlb.s16 q1, q1
 ; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %offs = load <8 x i16*>, <8 x i16*>* %offptr, align 4
   %gather = call <8 x i16> @llvm.masked.gather.v8i16.v8p0i16(<8 x i16*> %offs, i32 2, <8 x i1> <i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true>, <8 x i16> undef)
   %ext = sext <8 x i16> %gather to <8 x i32>
   ret <8 x i32> %ext
 }
 
 define arm_aapcs_vfpcc <8 x i32> @ptr_v8i16_zext(<8 x i16*>* %offptr) {
 ; CHECK-LABEL: ptr_v8i16_zext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r7, lr}
 ; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    vldrw.u32 q0, [r0]
 ; CHECK-NEXT:    vldrw.u32 q1, [r0, #16]
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    vmov r2, s3
-; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    vmov r1, s0
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r5, s1
+; CHECK-NEXT:    vmov r3, s1
 ; CHECK-NEXT:    vmov r4, s7
+; CHECK-NEXT:    vmov r5, s3
 ; CHECK-NEXT:    ldrh.w r12, [r1]
-; CHECK-NEXT:    vmov r1, s5
+; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    ldrh.w lr, [r2]
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    ldrh r3, [r3]
+; CHECK-NEXT:    vmov r2, s5
 ; CHECK-NEXT:    ldrh r0, [r0]
-; CHECK-NEXT:    ldrh r5, [r5]
-; CHECK-NEXT:    vmov.32 q0[0], r3
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    ldrh r3, [r3]
+; CHECK-NEXT:    vmov q0[2], q0[0], lr, r12
 ; CHECK-NEXT:    ldrh r4, [r4]
-; CHECK-NEXT:    vmov.32 q0[1], r5
-; CHECK-NEXT:    vmov.32 q0[2], r12
-; CHECK-NEXT:    vmov.32 q0[3], lr
+; CHECK-NEXT:    ldrh r5, [r5]
+; CHECK-NEXT:    vmov q0[3], q0[1], r5, r3
 ; CHECK-NEXT:    vmovlb.u16 q0, q0
 ; CHECK-NEXT:    ldrh r1, [r1]
 ; CHECK-NEXT:    ldrh r2, [r2]
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vmov.32 q1[3], r4
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r4, r2
 ; CHECK-NEXT:    vmovlb.u16 q1, q1
 ; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %offs = load <8 x i16*>, <8 x i16*>* %offptr, align 4
   %gather = call <8 x i16> @llvm.masked.gather.v8i16.v8p0i16(<8 x i16*> %offs, i32 2, <8 x i1> <i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true>, <8 x i16> undef)
   %ext = zext <8 x i16> %gather to <8 x i32>
   ret <8 x i32> %ext
 }
entry:
   ret <8 x i16> %ext
 }
 
 define arm_aapcs_vfpcc <4 x i32> @ptr_v4i8_sext32(<4 x i8*>* %offptr) {
 ; CHECK-LABEL: ptr_v4i8_sext32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vldrw.u32 q0, [r0]
 ; CHECK-NEXT:    vmov r0, s0
-; CHECK-NEXT:    vmov r3, s1
 ; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    vmov r2, s3
+; CHECK-NEXT:    vmov r2, s1
+; CHECK-NEXT:    vmov r3, s3
 ; CHECK-NEXT:    ldrb r0, [r0]
-; CHECK-NEXT:    ldrb r3, [r3]
-; CHECK-NEXT:    vmov.32 q0[0], r0
 ; CHECK-NEXT:    ldrb r1, [r1]
-; CHECK-NEXT:    vmov.32 q0[1], r3
 ; CHECK-NEXT:    ldrb r2, [r2]
-; CHECK-NEXT:    vmov.32 q0[2], r1
-; CHECK-NEXT:    vmov.32 q0[3], r2
+; CHECK-NEXT:    ldrb r3, [r3]
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r2
 ; CHECK-NEXT:    vmovlb.s8 q0, q0
 ; CHECK-NEXT:    vmovlb.s16 q0, q0
 ; CHECK-NEXT:    bx lr
 entry:
   %offs = load <4 x i8*>, <4 x i8*>* %offptr, align 4
   %gather = call <4 x i8> @llvm.masked.gather.v4i8.v4p0i8(<4 x i8*> %offs, i32 1, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i8> undef)
   %ext = sext <4 x i8> %gather to <4 x i32>
   ret <4 x i32> %ext
 }
 
 define arm_aapcs_vfpcc <4 x i32> @ptr_v4i8_zext32(<4 x i8*>* %offptr) {
 ; CHECK-LABEL: ptr_v4i8_zext32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vldrw.u32 q0, [r0]
 ; CHECK-NEXT:    vmov.i32 q1, #0xff
-; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    vmov r1, s1
-; CHECK-NEXT:    vmov r3, s2
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    ldrb r2, [r2]
+; CHECK-NEXT:    vmov r1, s0
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    vmov r0, s1
+; CHECK-NEXT:    vmov r3, s3
 ; CHECK-NEXT:    ldrb r1, [r1]
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    ldrb r3, [r3]
-; CHECK-NEXT:    vmov.32 q0[1], r1
+; CHECK-NEXT:    ldrb r2, [r2]
 ; CHECK-NEXT:    ldrb r0, [r0]
-; CHECK-NEXT:    vmov.32 q0[2], r3
-; CHECK-NEXT:    vmov.32 q0[3], r0
+; CHECK-NEXT:    ldrb r3, [r3]
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r1
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r0
 ; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    bx lr
 entry:
   %offs = load <4 x i8*>, <4 x i8*>* %offptr, align 4
   %gather = call <4 x i8> @llvm.masked.gather.v4i8.v4p0i8(<4 x i8*> %offs, i32 1, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i8> undef)
   %ext = zext <4 x i8> %gather to <4 x i32>
   ret <4 x i32> %ext
 }
 
 define arm_aapcs_vfpcc <8 x i32> @ptr_v8i8_sext32(<8 x i8*>* %offptr) {
 ; CHECK-LABEL: ptr_v8i8_sext32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r7, lr}
 ; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    vldrw.u32 q0, [r0]
 ; CHECK-NEXT:    vldrw.u32 q1, [r0, #16]
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    vmov r2, s3
-; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    vmov r1, s0
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r5, s1
+; CHECK-NEXT:    vmov r3, s1
 ; CHECK-NEXT:    vmov r4, s7
+; CHECK-NEXT:    vmov r5, s3
 ; CHECK-NEXT:    ldrb.w r12, [r1]
-; CHECK-NEXT:    vmov r1, s5
+; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    ldrb.w lr, [r2]
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    ldrb r3, [r3]
+; CHECK-NEXT:    vmov r2, s5
 ; CHECK-NEXT:    ldrb r0, [r0]
-; CHECK-NEXT:    ldrb r5, [r5]
-; CHECK-NEXT:    vmov.32 q0[0], r3
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    ldrb r3, [r3]
+; CHECK-NEXT:    vmov q0[2], q0[0], lr, r12
 ; CHECK-NEXT:    ldrb r4, [r4]
-; CHECK-NEXT:    vmov.32 q0[1], r5
-; CHECK-NEXT:    vmov.32 q0[2], r12
-; CHECK-NEXT:    vmov.32 q0[3], lr
+; CHECK-NEXT:    ldrb r5, [r5]
+; CHECK-NEXT:    vmov q0[3], q0[1], r5, r3
 ; CHECK-NEXT:    vmovlb.s8 q0, q0
 ; CHECK-NEXT:    vmovlb.s16 q0, q0
 ; CHECK-NEXT:    ldrb r1, [r1]
 ; CHECK-NEXT:    ldrb r2, [r2]
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vmov.32 q1[3], r4
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r4, r2
 ; CHECK-NEXT:    vmovlb.s8 q1, q1
 ; CHECK-NEXT:    vmovlb.s16 q1, q1
 ; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %offs = load <8 x i8*>, <8 x i8*>* %offptr, align 4
   %gather = call <8 x i8> @llvm.masked.gather.v8i8.v8p0i8(<8 x i8*> %offs, i32 1, <8 x i1> <i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true>, <8 x i8> undef)
   %ext = sext <8 x i8> %gather to <8 x i32>
   ret <8 x i32> %ext
 }
 
 define arm_aapcs_vfpcc <8 x i32> @ptr_v8i8_zext32(<8 x i8*>* %offptr) {
 ; CHECK-LABEL: ptr_v8i8_zext32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r7, lr}
 ; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    vldrw.u32 q0, [r0]
 ; CHECK-NEXT:    vldrw.u32 q1, [r0, #16]
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    vmov r2, s3
+; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r4, s0
-; CHECK-NEXT:    vmov r3, s5
-; CHECK-NEXT:    vmov r5, s1
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    vmov r3, s6
+; CHECK-NEXT:    vmov r4, s1
+; CHECK-NEXT:    vmov r5, s3
 ; CHECK-NEXT:    ldrb.w r12, [r1]
-; CHECK-NEXT:    vmov r1, s6
-; CHECK-NEXT:    ldrb.w lr, [r2]
-; CHECK-NEXT:    vmov r2, s7
-; CHECK-NEXT:    ldrb r0, [r0]
+; CHECK-NEXT:    vmov r1, s5
+; CHECK-NEXT:    ldrb.w lr, [r0]
+; CHECK-NEXT:    vmov r0, s7
+; CHECK-NEXT:    ldrb r2, [r2]
 ; CHECK-NEXT:    vmov.i32 q1, #0xff
-; CHECK-NEXT:    ldrb r4, [r4]
 ; CHECK-NEXT:    ldrb r3, [r3]
-; CHECK-NEXT:    vmov.32 q2[0], r0
+; CHECK-NEXT:    ldrb r4, [r4]
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r12
 ; CHECK-NEXT:    ldrb r5, [r5]
-; CHECK-NEXT:    vmov.32 q0[0], r4
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov.32 q0[1], r5
-; CHECK-NEXT:    vmov.32 q0[2], r12
-; CHECK-NEXT:    vmov.32 q0[3], lr
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, lr
+; CHECK-NEXT:    vmov q0[3], q0[1], r5, r4
 ; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    ldrb r1, [r1]
-; CHECK-NEXT:    ldrb r2, [r2]
-; CHECK-NEXT:    vmov.32 q2[2], r1
-; CHECK-NEXT:    vmov.32 q2[3], r2
+; CHECK-NEXT:    ldrb r0, [r0]
+; CHECK-NEXT:    vmov q2[3], q2[1], r0, r1
 ; CHECK-NEXT:    vand q1, q2, q1
 ; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %offs = load <8 x i8*>, <8 x i8*>* %offptr, align 4
   %gather = call <8 x i8> @llvm.masked.gather.v8i8.v8p0i8(<8 x i8*> %offs, i32 1, <8 x i1> <i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true>, <8 x i8> undef)
   %ext = zext <8 x i8> %gather to <8 x i32>
   ret <8 x i32> %ext
 }

 ; CHECK-LABEL: sext_unsigned_unscaled_i8_i8_toi64:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r7, lr}
 ; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    vldrb.u32 q0, [r1]
 ; CHECK-NEXT:    vldrb.u32 q1, [r1, #4]
 ; CHECK-NEXT:    vadd.i32 q0, q0, r0
 ; CHECK-NEXT:    vadd.i32 q1, q1, r0
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    vmov r3, s3
+; CHECK-NEXT:    vmov r2, s0
+; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r1, s5
-; CHECK-NEXT:    vmov r5, s1
+; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    vmov r4, s7
+; CHECK-NEXT:    vmov r5, s3
 ; CHECK-NEXT:    ldrb.w r12, [r2]
-; CHECK-NEXT:    vmov r2, s0
+; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    ldrb.w lr, [r3]
-; CHECK-NEXT:    vmov r3, s6
+; CHECK-NEXT:    vmov r3, s5
 ; CHECK-NEXT:    ldrb r0, [r0]
 ; CHECK-NEXT:    ldrb r1, [r1]
-; CHECK-NEXT:    ldrb r5, [r5]
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov q0[2], q0[0], lr, r12
 ; CHECK-NEXT:    ldrb r4, [r4]
-; CHECK-NEXT:    vmov.32 q1[1], r1
+; CHECK-NEXT:    ldrb r5, [r5]
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
 ; CHECK-NEXT:    ldrb r2, [r2]
 ; CHECK-NEXT:    ldrb r3, [r3]
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov.32 q0[1], r5
-; CHECK-NEXT:    vmov.32 q1[2], r3
-; CHECK-NEXT:    vmov.32 q0[2], r12
-; CHECK-NEXT:    vmov.32 q1[3], r4
-; CHECK-NEXT:    vmov.32 q0[3], lr
-; CHECK-NEXT:    vmovlb.s8 q1, q1
+; CHECK-NEXT:    vmov q0[3], q0[1], r5, r2
+; CHECK-NEXT:    vmov q1[3], q1[1], r4, r3
 ; CHECK-NEXT:    vmovlb.s8 q0, q0
-; CHECK-NEXT:    vmovlb.s16 q1, q1
+; CHECK-NEXT:    vmovlb.s8 q1, q1
 ; CHECK-NEXT:    vmovlb.s16 q0, q0
+; CHECK-NEXT:    vmovlb.s16 q1, q1
 ; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %offs = load <8 x i8>, <8 x i8>* %offptr, align 1
   %offs.zext = zext <8 x i8> %offs to <8 x i32>
   %ptrs = getelementptr inbounds i8, i8* %base, <8 x i32> %offs.zext
   %gather = call <8 x i8> @llvm.masked.gather.v8i8.v8p0i8(<8 x i8*> %ptrs, i32 1, <8 x i1> <i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true>, <8 x i8> undef)
   %gather.sext = sext <8 x i8> %gather to <8 x i32>
   ret <8 x i32> %gather.sext

llvm/test/CodeGen/Thumb2/mve-gather-scatter-opt.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp -enable-arm-maskedgatscat=false %s -o 2>/dev/null - | FileCheck --check-prefix NOGATSCAT %s
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=-mve %s -o 2>/dev/null - | FileCheck --check-prefix NOMVE %s
 
 define arm_aapcs_vfpcc <4 x i32> @unscaled_i32_i32_gather(i8* %base, <4 x i32>* %offptr) {
 ; NOGATSCAT-LABEL: unscaled_i32_i32_gather:
 ; NOGATSCAT:       @ %bb.0: @ %entry
 ; NOGATSCAT-NEXT:    vldrw.u32 q0, [r1]
 ; NOGATSCAT-NEXT:    vadd.i32 q0, q0, r0
 ; NOGATSCAT-NEXT:    vmov r0, s0
-; NOGATSCAT-NEXT:    vmov r3, s1
 ; NOGATSCAT-NEXT:    vmov r1, s2
-; NOGATSCAT-NEXT:    vmov r2, s3
+; NOGATSCAT-NEXT:    vmov r2, s1
+; NOGATSCAT-NEXT:    vmov r3, s3
 ; NOGATSCAT-NEXT:    ldr r0, [r0]
-; NOGATSCAT-NEXT:    ldr r3, [r3]
-; NOGATSCAT-NEXT:    vmov.32 q0[0], r0
 ; NOGATSCAT-NEXT:    ldr r1, [r1]
-; NOGATSCAT-NEXT:    vmov.32 q0[1], r3
 ; NOGATSCAT-NEXT:    ldr r2, [r2]
-; NOGATSCAT-NEXT:    vmov.32 q0[2], r1
-; NOGATSCAT-NEXT:    vmov.32 q0[3], r2
+; NOGATSCAT-NEXT:    ldr r3, [r3]
+; NOGATSCAT-NEXT:    vmov q0[2], q0[0], r1, r0
+; NOGATSCAT-NEXT:    vmov q0[3], q0[1], r3, r2
 ; NOGATSCAT-NEXT:    bx lr
 ;
 ; NOMVE-LABEL: unscaled_i32_i32_gather:
 ; NOMVE:       @ %bb.0: @ %entry
 ; NOMVE-NEXT:    .save {r4, lr}
 ; NOMVE-NEXT:    push {r4, lr}
 ; NOMVE-NEXT:    ldm.w r1, {r2, r3, lr}
 ; NOMVE-NEXT:    ldr r4, [r1, #12]

llvm/test/CodeGen/Thumb2/mve-masked-ldst.ll

 
 define void @foo_sext_v2i64_v2i32(<2 x i64> *%dest, <2 x i32> *%mask, <2 x i32> *%src) {
 ; CHECK-LE-LABEL: foo_sext_v2i64_v2i32:
 ; CHECK-LE:       @ %bb.0: @ %entry
 ; CHECK-LE-NEXT:    .save {r4, r5, r7, lr}
 ; CHECK-LE-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-LE-NEXT:    .pad #4
 ; CHECK-LE-NEXT:    sub sp, #4
-; CHECK-LE-NEXT:    ldrd lr, r12, [r1]
-; CHECK-LE-NEXT:    movs r1, #0
-; CHECK-LE-NEXT:    @ implicit-def: $q1
-; CHECK-LE-NEXT:    movs r4, #0
-; CHECK-LE-NEXT:    rsbs.w r3, lr, #0
-; CHECK-LE-NEXT:    vmov.32 q0[0], lr
-; CHECK-LE-NEXT:    sbcs.w r3, r1, lr, asr #31
-; CHECK-LE-NEXT:    mov.w lr, #0
-; CHECK-LE-NEXT:    it lt
-; CHECK-LE-NEXT:    movlt.w lr, #1
-; CHECK-LE-NEXT:    rsbs.w r3, r12, #0
-; CHECK-LE-NEXT:    sbcs.w r3, r1, r12, asr #31
+; CHECK-LE-NEXT:    ldrd lr, r5, [r1]
+; CHECK-LE-NEXT:    movs r3, #0
+; CHECK-LE-NEXT:    @ implicit-def: $q0
+; CHECK-LE-NEXT:    rsbs.w r1, lr, #0
+; CHECK-LE-NEXT:    vmov q1[2], q1[0], r5, lr
+; CHECK-LE-NEXT:    sbcs.w r1, r3, lr, asr #31
+; CHECK-LE-NEXT:    mov.w r1, #0
 ; CHECK-LE-NEXT:    it lt
 ; CHECK-LE-NEXT:    movlt r1, #1
-; CHECK-LE-NEXT:    cmp r1, #0
+; CHECK-LE-NEXT:    rsbs r4, r5, #0
+; CHECK-LE-NEXT:    sbcs.w r4, r3, r5, asr #31
+; CHECK-LE-NEXT:    it lt
+; CHECK-LE-NEXT:    movlt r3, #1
+; CHECK-LE-NEXT:    cmp r3, #0
 ; CHECK-LE-NEXT:    it ne
-; CHECK-LE-NEXT:    mvnne r1, #1
-; CHECK-LE-NEXT:    bfi r1, lr, #0, #1
-; CHECK-LE-NEXT:    vmov.32 q0[2], r12
-; CHECK-LE-NEXT:    and r3, r1, #3
-; CHECK-LE-NEXT:    lsls r1, r1, #31
+; CHECK-LE-NEXT:    mvnne r3, #1
+; CHECK-LE-NEXT:    bfi r3, r1, #0, #1
+; CHECK-LE-NEXT:    vmov r4, s4
+; CHECK-LE-NEXT:    and r12, r3, #3
+; CHECK-LE-NEXT:    lsls r1, r3, #31
 ; CHECK-LE-NEXT:    itt ne
 ; CHECK-LE-NEXT:    ldrne r1, [r2]
-; CHECK-LE-NEXT:    vmovne.32 q1[0], r1
-; CHECK-LE-NEXT:    lsls r1, r3, #30
+; CHECK-LE-NEXT:    vmovne.32 q0[0], r1
+; CHECK-LE-NEXT:    lsls.w r1, r12, #30
 ; CHECK-LE-NEXT:    itt mi
 ; CHECK-LE-NEXT:    ldrmi r1, [r2, #4]
-; CHECK-LE-NEXT:    vmovmi.32 q1[2], r1
-; CHECK-LE-NEXT:    vmov r2, s0
-; CHECK-LE-NEXT:    vmov r3, s4
-; CHECK-LE-NEXT:    vmov r1, s6
-; CHECK-LE-NEXT:    vmov.32 q1[0], r3
-; CHECK-LE-NEXT:    rsbs r5, r2, #0
-; CHECK-LE-NEXT:    sbcs.w r2, r4, r2, asr #31
-; CHECK-LE-NEXT:    vmov r2, s2
+; CHECK-LE-NEXT:    vmovmi.32 q0[2], r1
+; CHECK-LE-NEXT:    vmov r3, s2
+; CHECK-LE-NEXT:    movs r2, #0
+; CHECK-LE-NEXT:    vmov r1, s0
+; CHECK-LE-NEXT:    vmov q0[2], q0[0], r3, r1
+; CHECK-LE-NEXT:    rsbs r5, r4, #0
 ; CHECK-LE-NEXT:    asr.w lr, r3, #31
-; CHECK-LE-NEXT:    vmov.32 q1[1], lr
+; CHECK-LE-NEXT:    vmov r3, s6
 ; CHECK-LE-NEXT:    asr.w r12, r1, #31
-; CHECK-LE-NEXT:    vmov.32 q1[2], r1
+; CHECK-LE-NEXT:    sbcs.w r1, r2, r4, asr #31
 ; CHECK-LE-NEXT:    mov.w r1, #0
+; CHECK-LE-NEXT:    vmov q0[3], q0[1], lr, r12
 ; CHECK-LE-NEXT:    it lt
 ; CHECK-LE-NEXT:    movlt r1, #1
-; CHECK-LE-NEXT:    vmov.32 q1[3], r12
-; CHECK-LE-NEXT:    rsbs r3, r2, #0
-; CHECK-LE-NEXT:    sbcs.w r2, r4, r2, asr #31
+; CHECK-LE-NEXT:    rsbs r5, r3, #0
+; CHECK-LE-NEXT:    sbcs.w r3, r2, r3, asr #31
 ; CHECK-LE-NEXT:    it lt
-; CHECK-LE-NEXT:    movlt r4, #1
-; CHECK-LE-NEXT:    cmp r4, #0
+; CHECK-LE-NEXT:    movlt r2, #1
+; CHECK-LE-NEXT:    cmp r2, #0
 ; CHECK-LE-NEXT:    it ne
-; CHECK-LE-NEXT:    mvnne r4, #1
-; CHECK-LE-NEXT:    bfi r4, r1, #0, #1
-; CHECK-LE-NEXT:    and r1, r4, #3
-; CHECK-LE-NEXT:    lsls r2, r4, #31
+; CHECK-LE-NEXT:    mvnne r2, #1
+; CHECK-LE-NEXT:    bfi r2, r1, #0, #1
+; CHECK-LE-NEXT:    and r1, r2, #3
+; CHECK-LE-NEXT:    lsls r2, r2, #31
 ; CHECK-LE-NEXT:    it ne
-; CHECK-LE-NEXT:    vstrne d2, [r0]
+; CHECK-LE-NEXT:    vstrne d0, [r0]
 ; CHECK-LE-NEXT:    lsls r1, r1, #30
 ; CHECK-LE-NEXT:    it mi
-; CHECK-LE-NEXT:    vstrmi d3, [r0, #8]
+; CHECK-LE-NEXT:    vstrmi d1, [r0, #8]
 ; CHECK-LE-NEXT:    add sp, #4
 ; CHECK-LE-NEXT:    pop {r4, r5, r7, pc}
 ;
 ; CHECK-BE-LABEL: foo_sext_v2i64_v2i32:
 ; CHECK-BE:       @ %bb.0: @ %entry
 ; CHECK-BE-NEXT:    .save {r4, r5, r7, lr}
 ; CHECK-BE-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-BE-NEXT:    .pad #4
 ; CHECK-BE-NEXT:    sub sp, #4
 ; CHECK-BE-NEXT:    ldrd r12, lr, [r1]
 ; CHECK-BE-NEXT:    rsbs.w r1, lr, #0
 ; CHECK-BE-NEXT:    mov.w r3, #0
 ; CHECK-BE-NEXT:    sbcs.w r1, r3, lr, asr #31
-; CHECK-BE-NEXT:    vmov.32 q0[1], r12
-; CHECK-BE-NEXT:    @ implicit-def: $q2
-; CHECK-BE-NEXT:    vmov.32 q0[3], lr
+; CHECK-BE-NEXT:    vmov q0[3], q0[1], lr, r12
 ; CHECK-BE-NEXT:    mov.w lr, #0
 ; CHECK-BE-NEXT:    it lt
 ; CHECK-BE-NEXT:    movlt.w lr, #1
 ; CHECK-BE-NEXT:    rsbs.w r1, r12, #0
 ; CHECK-BE-NEXT:    sbcs.w r1, r3, r12, asr #31
 ; CHECK-BE-NEXT:    it lt
 ; CHECK-BE-NEXT:    movlt r3, #1
 ; CHECK-BE-NEXT:    cmp r3, #0
 ; CHECK-BE-NEXT:    it ne
 ; CHECK-BE-NEXT:    mvnne r3, #1
 ; CHECK-BE-NEXT:    bfi r3, lr, #0, #1
+; CHECK-BE-NEXT:    @ implicit-def: $q2
 ; CHECK-BE-NEXT:    and r1, r3, #3
 ; CHECK-BE-NEXT:    lsls r3, r3, #31
 ; CHECK-BE-NEXT:    beq .LBB5_2
 ; CHECK-BE-NEXT:  @ %bb.1: @ %cond.load
 ; CHECK-BE-NEXT:    ldr r3, [r2]
 ; CHECK-BE-NEXT:    vmov.32 q1[1], r3
 ; CHECK-BE-NEXT:    vrev64.32 q2, q1
 ; CHECK-BE-NEXT:  .LBB5_2: @ %else
 ; CHECK-BE-NEXT:    vrev64.32 q1, q0
 ; CHECK-BE-NEXT:    lsls r1, r1, #30
 ; CHECK-BE-NEXT:    bpl .LBB5_4
 ; CHECK-BE-NEXT:  @ %bb.3: @ %cond.load1
 ; CHECK-BE-NEXT:    ldr r1, [r2, #4]
 ; CHECK-BE-NEXT:    vrev64.32 q0, q2
 ; CHECK-BE-NEXT:    vmov.32 q0[3], r1
 ; CHECK-BE-NEXT:    vrev64.32 q2, q0
 ; CHECK-BE-NEXT:  .LBB5_4: @ %else2
 ; CHECK-BE-NEXT:    vrev64.32 q0, q2
 ; CHECK-BE-NEXT:    vrev64.32 q2, q1
 ; CHECK-BE-NEXT:    vmov r2, s11
 ; CHECK-BE-NEXT:    movs r4, #0
-; CHECK-BE-NEXT:    vmov r3, s1
-; CHECK-BE-NEXT:    vmov r1, s3
+; CHECK-BE-NEXT:    vmov r1, s1
+; CHECK-BE-NEXT:    vmov r3, s3
 ; CHECK-BE-NEXT:    rsbs r5, r2, #0
 ; CHECK-BE-NEXT:    sbcs.w r2, r4, r2, asr #31
 ; CHECK-BE-NEXT:    vmov r2, s9
-; CHECK-BE-NEXT:    asr.w lr, r3, #31
-; CHECK-BE-NEXT:    vmov.32 q1[0], lr
 ; CHECK-BE-NEXT:    asr.w r12, r1, #31
-; CHECK-BE-NEXT:    vmov.32 q1[1], r3
-; CHECK-BE-NEXT:    vmov.32 q1[2], r12
-; CHECK-BE-NEXT:    vmov.32 q1[3], r1
+; CHECK-BE-NEXT:    asr.w lr, r3, #31
+; CHECK-BE-NEXT:    vmov q1[2], q1[0], lr, r12
+; CHECK-BE-NEXT:    vmov q1[3], q1[1], r3, r1
 ; CHECK-BE-NEXT:    mov.w r1, #0
 ; CHECK-BE-NEXT:    it lt
 ; CHECK-BE-NEXT:    movlt r1, #1
 ; CHECK-BE-NEXT:    vrev64.32 q0, q1
 ; CHECK-BE-NEXT:    rsbs r3, r2, #0
 ; CHECK-BE-NEXT:    sbcs.w r2, r4, r2, asr #31
 ; CHECK-BE-NEXT:    it lt
 ; CHECK-BE-NEXT:    movlt r4, #1

 
 define void @foo_sext_v2i64_v2i32_unaligned(<2 x i64> *%dest, <2 x i32> *%mask, <2 x i32> *%src) {
 ; CHECK-LE-LABEL: foo_sext_v2i64_v2i32_unaligned:
 ; CHECK-LE:       @ %bb.0: @ %entry
 ; CHECK-LE-NEXT:    .save {r4, r5, r7, lr}
 ; CHECK-LE-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-LE-NEXT:    .pad #4
 ; CHECK-LE-NEXT:    sub sp, #4
-; CHECK-LE-NEXT:    ldrd lr, r12, [r1]
-; CHECK-LE-NEXT:    movs r1, #0
-; CHECK-LE-NEXT:    @ implicit-def: $q1
-; CHECK-LE-NEXT:    movs r4, #0
-; CHECK-LE-NEXT:    rsbs.w r3, lr, #0
-; CHECK-LE-NEXT:    vmov.32 q0[0], lr
-; CHECK-LE-NEXT:    sbcs.w r3, r1, lr, asr #31
-; CHECK-LE-NEXT:    mov.w lr, #0
-; CHECK-LE-NEXT:    it lt
-; CHECK-LE-NEXT:    movlt.w lr, #1
-; CHECK-LE-NEXT:    rsbs.w r3, r12, #0
-; CHECK-LE-NEXT:    sbcs.w r3, r1, r12, asr #31
+; CHECK-LE-NEXT:    ldrd lr, r5, [r1]
+; CHECK-LE-NEXT:    movs r3, #0
+; CHECK-LE-NEXT:    @ implicit-def: $q0
+; CHECK-LE-NEXT:    rsbs.w r1, lr, #0
+; CHECK-LE-NEXT:    vmov q1[2], q1[0], r5, lr
+; CHECK-LE-NEXT:    sbcs.w r1, r3, lr, asr #31
+; CHECK-LE-NEXT:    mov.w r1, #0
 ; CHECK-LE-NEXT:    it lt
 ; CHECK-LE-NEXT:    movlt r1, #1
-; CHECK-LE-NEXT:    cmp r1, #0
+; CHECK-LE-NEXT:    rsbs r4, r5, #0
+; CHECK-LE-NEXT:    sbcs.w r4, r3, r5, asr #31
+; CHECK-LE-NEXT:    it lt
+; CHECK-LE-NEXT:    movlt r3, #1
+; CHECK-LE-NEXT:    cmp r3, #0
 ; CHECK-LE-NEXT:    it ne
-; CHECK-LE-NEXT:    mvnne r1, #1
-; CHECK-LE-NEXT:    bfi r1, lr, #0, #1
-; CHECK-LE-NEXT:    vmov.32 q0[2], r12
-; CHECK-LE-NEXT:    and r3, r1, #3
-; CHECK-LE-NEXT:    lsls r1, r1, #31
+; CHECK-LE-NEXT:    mvnne r3, #1
+; CHECK-LE-NEXT:    bfi r3, r1, #0, #1
+; CHECK-LE-NEXT:    vmov r4, s4
+; CHECK-LE-NEXT:    and r12, r3, #3
+; CHECK-LE-NEXT:    lsls r1, r3, #31
 ; CHECK-LE-NEXT:    itt ne
 ; CHECK-LE-NEXT:    ldrne r1, [r2]
-; CHECK-LE-NEXT:    vmovne.32 q1[0], r1
-; CHECK-LE-NEXT:    lsls r1, r3, #30
+; CHECK-LE-NEXT:    vmovne.32 q0[0], r1
+; CHECK-LE-NEXT:    lsls.w r1, r12, #30
 ; CHECK-LE-NEXT:    itt mi
 ; CHECK-LE-NEXT:    ldrmi r1, [r2, #4]
-; CHECK-LE-NEXT:    vmovmi.32 q1[2], r1
-; CHECK-LE-NEXT:    vmov r2, s0
-; CHECK-LE-NEXT:    vmov r3, s4
-; CHECK-LE-NEXT:    vmov r1, s6
-; CHECK-LE-NEXT:    vmov.32 q1[0], r3
-; CHECK-LE-NEXT:    rsbs r5, r2, #0
-; CHECK-LE-NEXT:    sbcs.w r2, r4, r2, asr #31
-; CHECK-LE-NEXT:    vmov r2, s2
+; CHECK-LE-NEXT:    vmovmi.32 q0[2], r1
+; CHECK-LE-NEXT:    vmov r3, s2
+; CHECK-LE-NEXT:    movs r2, #0
+; CHECK-LE-NEXT:    vmov r1, s0
+; CHECK-LE-NEXT:    vmov q0[2], q0[0], r3, r1
+; CHECK-LE-NEXT:    rsbs r5, r4, #0
 ; CHECK-LE-NEXT:    asr.w lr, r3, #31
-; CHECK-LE-NEXT:    vmov.32 q1[1], lr
+; CHECK-LE-NEXT:    vmov r3, s6
 ; CHECK-LE-NEXT:    asr.w r12, r1, #31
-; CHECK-LE-NEXT:    vmov.32 q1[2], r1
+; CHECK-LE-NEXT:    sbcs.w r1, r2, r4, asr #31
 ; CHECK-LE-NEXT:    mov.w r1, #0
+; CHECK-LE-NEXT:    vmov q0[3], q0[1], lr, r12
 ; CHECK-LE-NEXT:    it lt
 ; CHECK-LE-NEXT:    movlt r1, #1
-; CHECK-LE-NEXT:    vmov.32 q1[3], r12
-; CHECK-LE-NEXT:    rsbs r3, r2, #0
-; CHECK-LE-NEXT:    sbcs.w r2, r4, r2, asr #31
+; CHECK-LE-NEXT:    rsbs r5, r3, #0
+; CHECK-LE-NEXT:    sbcs.w r3, r2, r3, asr #31
 ; CHECK-LE-NEXT:    it lt
-; CHECK-LE-NEXT:    movlt r4, #1
-; CHECK-LE-NEXT:    cmp r4, #0
+; CHECK-LE-NEXT:    movlt r2, #1
+; CHECK-LE-NEXT:    cmp r2, #0
 ; CHECK-LE-NEXT:    it ne
-; CHECK-LE-NEXT:    mvnne r4, #1
-; CHECK-LE-NEXT:    bfi r4, r1, #0, #1
-; CHECK-LE-NEXT:    and r1, r4, #3
-; CHECK-LE-NEXT:    lsls r2, r4, #31
+; CHECK-LE-NEXT:    mvnne r2, #1
+; CHECK-LE-NEXT:    bfi r2, r1, #0, #1
+; CHECK-LE-NEXT:    and r1, r2, #3
+; CHECK-LE-NEXT:    lsls r2, r2, #31
 ; CHECK-LE-NEXT:    itt ne
-; CHECK-LE-NEXT:    vmovne r2, r3, d2
+; CHECK-LE-NEXT:    vmovne r2, r3, d0
 ; CHECK-LE-NEXT:    strdne r2, r3, [r0]
 ; CHECK-LE-NEXT:    lsls r1, r1, #30
 ; CHECK-LE-NEXT:    itt mi
-; CHECK-LE-NEXT:    vmovmi r1, r2, d3
+; CHECK-LE-NEXT:    vmovmi r1, r2, d1
 ; CHECK-LE-NEXT:    strdmi r1, r2, [r0, #8]
 ; CHECK-LE-NEXT:    add sp, #4
 ; CHECK-LE-NEXT:    pop {r4, r5, r7, pc}
 ;
 ; CHECK-BE-LABEL: foo_sext_v2i64_v2i32_unaligned:
 ; CHECK-BE:       @ %bb.0: @ %entry
 ; CHECK-BE-NEXT:    .save {r4, r5, r7, lr}
 ; CHECK-BE-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-BE-NEXT:    .pad #4
 ; CHECK-BE-NEXT:    sub sp, #4
 ; CHECK-BE-NEXT:    ldrd r12, lr, [r1]
 ; CHECK-BE-NEXT:    rsbs.w r1, lr, #0
 ; CHECK-BE-NEXT:    mov.w r3, #0
 ; CHECK-BE-NEXT:    sbcs.w r1, r3, lr, asr #31
-; CHECK-BE-NEXT:    vmov.32 q0[1], r12
-; CHECK-BE-NEXT:    @ implicit-def: $q2
-; CHECK-BE-NEXT:    vmov.32 q0[3], lr
+; CHECK-BE-NEXT:    vmov q0[3], q0[1], lr, r12
 ; CHECK-BE-NEXT:    mov.w lr, #0
 ; CHECK-BE-NEXT:    it lt
 ; CHECK-BE-NEXT:    movlt.w lr, #1
 ; CHECK-BE-NEXT:    rsbs.w r1, r12, #0
 ; CHECK-BE-NEXT:    sbcs.w r1, r3, r12, asr #31
 ; CHECK-BE-NEXT:    it lt
 ; CHECK-BE-NEXT:    movlt r3, #1
 ; CHECK-BE-NEXT:    cmp r3, #0
 ; CHECK-BE-NEXT:    it ne
 ; CHECK-BE-NEXT:    mvnne r3, #1
 ; CHECK-BE-NEXT:    bfi r3, lr, #0, #1
+; CHECK-BE-NEXT:    @ implicit-def: $q2
 ; CHECK-BE-NEXT:    and r1, r3, #3
 ; CHECK-BE-NEXT:    lsls r3, r3, #31
 ; CHECK-BE-NEXT:    beq .LBB6_2
 ; CHECK-BE-NEXT:  @ %bb.1: @ %cond.load
 ; CHECK-BE-NEXT:    ldr r3, [r2]
 ; CHECK-BE-NEXT:    vmov.32 q1[1], r3
 ; CHECK-BE-NEXT:    vrev64.32 q2, q1
 ; CHECK-BE-NEXT:  .LBB6_2: @ %else
 ; CHECK-BE-NEXT:    vrev64.32 q1, q0
 ; CHECK-BE-NEXT:    lsls r1, r1, #30
 ; CHECK-BE-NEXT:    bpl .LBB6_4
 ; CHECK-BE-NEXT:  @ %bb.3: @ %cond.load1
 ; CHECK-BE-NEXT:    ldr r1, [r2, #4]
 ; CHECK-BE-NEXT:    vrev64.32 q0, q2
 ; CHECK-BE-NEXT:    vmov.32 q0[3], r1
 ; CHECK-BE-NEXT:    vrev64.32 q2, q0
 ; CHECK-BE-NEXT:  .LBB6_4: @ %else2
 ; CHECK-BE-NEXT:    vrev64.32 q0, q2
 ; CHECK-BE-NEXT:    vrev64.32 q2, q1
 ; CHECK-BE-NEXT:    vmov r2, s11
 ; CHECK-BE-NEXT:    movs r4, #0
-; CHECK-BE-NEXT:    vmov r3, s1
-; CHECK-BE-NEXT:    vmov r1, s3
+; CHECK-BE-NEXT:    vmov r1, s1
+; CHECK-BE-NEXT:    vmov r3, s3
 ; CHECK-BE-NEXT:    rsbs r5, r2, #0
 ; CHECK-BE-NEXT:    sbcs.w r2, r4, r2, asr #31
 ; CHECK-BE-NEXT:    vmov r2, s9
-; CHECK-BE-NEXT:    asr.w lr, r3, #31
-; CHECK-BE-NEXT:    vmov.32 q1[0], lr
 ; CHECK-BE-NEXT:    asr.w r12, r1, #31
-; CHECK-BE-NEXT:    vmov.32 q1[1], r3
-; CHECK-BE-NEXT:    vmov.32 q1[2], r12
-; CHECK-BE-NEXT:    vmov.32 q1[3], r1
+; CHECK-BE-NEXT:    asr.w lr, r3, #31
+; CHECK-BE-NEXT:    vmov q1[2], q1[0], lr, r12
+; CHECK-BE-NEXT:    vmov q1[3], q1[1], r3, r1
 ; CHECK-BE-NEXT:    mov.w r1, #0
 ; CHECK-BE-NEXT:    it lt
 ; CHECK-BE-NEXT:    movlt r1, #1
 ; CHECK-BE-NEXT:    vrev64.32 q0, q1
 ; CHECK-BE-NEXT:    rsbs r3, r2, #0
 ; CHECK-BE-NEXT:    sbcs.w r2, r4, r2, asr #31
 ; CHECK-BE-NEXT:    it lt
 ; CHECK-BE-NEXT:    movlt r4, #1
entry:
   %3 = sext <2 x i32> %2 to <2 x i64>
   call void @llvm.masked.store.v2i64.p0v2i64(<2 x i64> %3, <2 x i64>* %dest, i32 4, <2 x i1> %1)
   ret void
 }
 
 define void @foo_zext_v2i64_v2i32(<2 x i64> *%dest, <2 x i32> *%mask, <2 x i32> *%src) {
 ; CHECK-LE-LABEL: foo_zext_v2i64_v2i32:
 ; CHECK-LE:       @ %bb.0: @ %entry
-; CHECK-LE-NEXT:    .save {r7, lr}
-; CHECK-LE-NEXT:    push {r7, lr}
+; CHECK-LE-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-LE-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-LE-NEXT:    .pad #4
 ; CHECK-LE-NEXT:    sub sp, #4
-; CHECK-LE-NEXT:    ldrd lr, r12, [r1]
-; CHECK-LE-NEXT:    movs r1, #0
-; CHECK-LE-NEXT:    @ implicit-def: $q1
+; CHECK-LE-NEXT:    ldrd lr, r5, [r1]
+; CHECK-LE-NEXT:    movs r3, #0
+; CHECK-LE-NEXT:    @ implicit-def: $q0
 ; CHECK-LE-NEXT:    vmov.i64 q2, #0xffffffff
-; CHECK-LE-NEXT:    rsbs.w r3, lr, #0
-; CHECK-LE-NEXT:    vmov.32 q0[0], lr
-; CHECK-LE-NEXT:    sbcs.w r3, r1, lr, asr #31
-; CHECK-LE-NEXT:    mov.w lr, #0
-; CHECK-LE-NEXT:    it lt
-; CHECK-LE-NEXT:    movlt.w lr, #1
-; CHECK-LE-NEXT:    rsbs.w r3, r12, #0
-; CHECK-LE-NEXT:    sbcs.w r3, r1, r12, asr #31
+; CHECK-LE-NEXT:    rsbs.w r1, lr, #0
+; CHECK-LE-NEXT:    vmov q1[2], q1[0], r5, lr
+; CHECK-LE-NEXT:    sbcs.w r1, r3, lr, asr #31
+; CHECK-LE-NEXT:    mov.w r1, #0
 ; CHECK-LE-NEXT:    it lt
 ; CHECK-LE-NEXT:    movlt r1, #1
-; CHECK-LE-NEXT:    cmp r1, #0
+; CHECK-LE-NEXT:    rsbs r4, r5, #0
+; CHECK-LE-NEXT:    sbcs.w r4, r3, r5, asr #31
+; CHECK-LE-NEXT:    it lt
+; CHECK-LE-NEXT:    movlt r3, #1
+; CHECK-LE-NEXT:    cmp r3, #0
 ; CHECK-LE-NEXT:    it ne
-; CHECK-LE-NEXT:    mvnne r1, #1
-; CHECK-LE-NEXT:    bfi r1, lr, #0, #1
-; CHECK-LE-NEXT:    vmov.32 q0[2], r12
-; CHECK-LE-NEXT:    and r3, r1, #3
-; CHECK-LE-NEXT:    mov.w r12, #0
-; CHECK-LE-NEXT:    lsls r1, r1, #31
+; CHECK-LE-NEXT:    mvnne r3, #1
+; CHECK-LE-NEXT:    bfi r3, r1, #0, #1
+; CHECK-LE-NEXT:    and r12, r3, #3
+; CHECK-LE-NEXT:    lsls r1, r3, #31
 ; CHECK-LE-NEXT:    itt ne
 ; CHECK-LE-NEXT:    ldrne r1, [r2]
-; CHECK-LE-NEXT:    vmovne.32 q1[0], r1
-; CHECK-LE-NEXT:    lsls r1, r3, #30
+; CHECK-LE-NEXT:    vmovne.32 q0[0], r1
+; CHECK-LE-NEXT:    lsls.w r1, r12, #30
 ; CHECK-LE-NEXT:    itt mi
 ; CHECK-LE-NEXT:    ldrmi r1, [r2, #4]
-; CHECK-LE-NEXT:    vmovmi.32 q1[2], r1
-; CHECK-LE-NEXT:    vmov r1, s0
+; CHECK-LE-NEXT:    vmovmi.32 q0[2], r1
+; CHECK-LE-NEXT:    vmov r1, s4
 ; CHECK-LE-NEXT:    movs r2, #0
-; CHECK-LE-NEXT:    vand q1, q1, q2
+; CHECK-LE-NEXT:    vand q0, q0, q2
 ; CHECK-LE-NEXT:    rsbs r3, r1, #0
-; CHECK-LE-NEXT:    vmov r3, s2
+; CHECK-LE-NEXT:    vmov r3, s6
 ; CHECK-LE-NEXT:    sbcs.w r1, r2, r1, asr #31
+; CHECK-LE-NEXT:    mov.w r1, #0
 ; CHECK-LE-NEXT:    it lt
-; CHECK-LE-NEXT:    movlt.w r12, #1
-; CHECK-LE-NEXT:    rsbs r1, r3, #0
-; CHECK-LE-NEXT:    sbcs.w r1, r2, r3, asr #31
+; CHECK-LE-NEXT:    movlt r1, #1
+; CHECK-LE-NEXT:    rsbs r5, r3, #0
+; CHECK-LE-NEXT:    sbcs.w r3, r2, r3, asr #31
 ; CHECK-LE-NEXT:    it lt
 ; CHECK-LE-NEXT:    movlt r2, #1
 ; CHECK-LE-NEXT:    cmp r2, #0
 ; CHECK-LE-NEXT:    it ne
 ; CHECK-LE-NEXT:    mvnne r2, #1
-; CHECK-LE-NEXT:    bfi r2, r12, #0, #1
+; CHECK-LE-NEXT:    bfi r2, r1, #0, #1
 ; CHECK-LE-NEXT:    and r1, r2, #3
 ; CHECK-LE-NEXT:    lsls r2, r2, #31
 ; CHECK-LE-NEXT:    it ne
-; CHECK-LE-NEXT:    vstrne d2, [r0]
+; CHECK-LE-NEXT:    vstrne d0, [r0]
 ; CHECK-LE-NEXT:    lsls r1, r1, #30
 ; CHECK-LE-NEXT:    it mi
-; CHECK-LE-NEXT:    vstrmi d3, [r0, #8]
+; CHECK-LE-NEXT:    vstrmi d1, [r0, #8]
 ; CHECK-LE-NEXT:    add sp, #4
-; CHECK-LE-NEXT:    pop {r7, pc}
+; CHECK-LE-NEXT:    pop {r4, r5, r7, pc}
 ;
 ; CHECK-BE-LABEL: foo_zext_v2i64_v2i32:
 ; CHECK-BE:       @ %bb.0: @ %entry
 ; CHECK-BE-NEXT:    .save {r7, lr}
 ; CHECK-BE-NEXT:    push {r7, lr}
 ; CHECK-BE-NEXT:    .pad #4
 ; CHECK-BE-NEXT:    sub sp, #4
 ; CHECK-BE-NEXT:    ldrd r12, lr, [r1]
 ; CHECK-BE-NEXT:    rsbs.w r1, lr, #0
 ; CHECK-BE-NEXT:    mov.w r3, #0
 ; CHECK-BE-NEXT:    sbcs.w r1, r3, lr, asr #31
-; CHECK-BE-NEXT:    vmov.32 q0[1], r12
-; CHECK-BE-NEXT:    @ implicit-def: $q1
-; CHECK-BE-NEXT:    vmov.32 q0[3], lr
+; CHECK-BE-NEXT:    vmov q0[3], q0[1], lr, r12
 ; CHECK-BE-NEXT:    mov.w lr, #0
 ; CHECK-BE-NEXT:    it lt
 ; CHECK-BE-NEXT:    movlt.w lr, #1
 ; CHECK-BE-NEXT:    rsbs.w r1, r12, #0
 ; CHECK-BE-NEXT:    sbcs.w r1, r3, r12, asr #31
 ; CHECK-BE-NEXT:    it lt
 ; CHECK-BE-NEXT:    movlt r3, #1
 ; CHECK-BE-NEXT:    cmp r3, #0
 ; CHECK-BE-NEXT:    it ne
 ; CHECK-BE-NEXT:    mvnne r3, #1
 ; CHECK-BE-NEXT:    bfi r3, lr, #0, #1
+; CHECK-BE-NEXT:    @ implicit-def: $q1
 ; CHECK-BE-NEXT:    and r1, r3, #3
 ; CHECK-BE-NEXT:    lsls r3, r3, #31
 ; CHECK-BE-NEXT:    beq .LBB7_2
 ; CHECK-BE-NEXT:  @ %bb.1: @ %cond.load
 ; CHECK-BE-NEXT:    ldr r3, [r2]
 ; CHECK-BE-NEXT:    vmov.32 q2[1], r3
 ; CHECK-BE-NEXT:    vrev64.32 q1, q2
 ; CHECK-BE-NEXT:  .LBB7_2: @ %else
entry:
   %3 = zext <2 x i32> %2 to <2 x i64>
   call void @llvm.masked.store.v2i64.p0v2i64(<2 x i64> %3, <2 x i64>* %dest, i32 8, <2 x i1> %1)
   ret void
 }
 
 define void @foo_zext_v2i64_v2i32_unaligned(<2 x i64> *%dest, <2 x i32> *%mask, <2 x i32> *%src) {
 ; CHECK-LE-LABEL: foo_zext_v2i64_v2i32_unaligned:
 ; CHECK-LE:       @ %bb.0: @ %entry
-; CHECK-LE-NEXT:    .save {r7, lr}
-; CHECK-LE-NEXT:    push {r7, lr}
+; CHECK-LE-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-LE-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-LE-NEXT:    .pad #4
 ; CHECK-LE-NEXT:    sub sp, #4
-; CHECK-LE-NEXT:    ldrd lr, r12, [r1]
-; CHECK-LE-NEXT:    movs r1, #0
-; CHECK-LE-NEXT:    @ implicit-def: $q1
+; CHECK-LE-NEXT:    ldrd lr, r5, [r1]
+; CHECK-LE-NEXT:    movs r3, #0
+; CHECK-LE-NEXT:    @ implicit-def: $q0
 ; CHECK-LE-NEXT:    vmov.i64 q2, #0xffffffff
-; CHECK-LE-NEXT:    rsbs.w r3, lr, #0
-; CHECK-LE-NEXT:    vmov.32 q0[0], lr
-; CHECK-LE-NEXT:    sbcs.w r3, r1, lr, asr #31
-; CHECK-LE-NEXT:    mov.w lr, #0
-; CHECK-LE-NEXT:    it lt
-; CHECK-LE-NEXT:    movlt.w lr, #1
-; CHECK-LE-NEXT:    rsbs.w r3, r12, #0
-; CHECK-LE-NEXT:    sbcs.w r3, r1, r12, asr #31
+; CHECK-LE-NEXT:    rsbs.w r1, lr, #0
+; CHECK-LE-NEXT:    vmov q1[2], q1[0], r5, lr
+; CHECK-LE-NEXT:    sbcs.w r1, r3, lr, asr #31
+; CHECK-LE-NEXT:    mov.w r1, #0
 ; CHECK-LE-NEXT:    it lt
 ; CHECK-LE-NEXT:    movlt r1, #1
-; CHECK-LE-NEXT:    cmp r1, #0
+; CHECK-LE-NEXT:    rsbs r4, r5, #0
+; CHECK-LE-NEXT:    sbcs.w r4, r3, r5, asr #31
+; CHECK-LE-NEXT:    it lt
+; CHECK-LE-NEXT:    movlt r3, #1
+; CHECK-LE-NEXT:    cmp r3, #0
 ; CHECK-LE-NEXT:    it ne
-; CHECK-LE-NEXT:    mvnne r1, #1
-; CHECK-LE-NEXT:    bfi r1, lr, #0, #1
-; CHECK-LE-NEXT:    vmov.32 q0[2], r12
-; CHECK-LE-NEXT:    and r3, r1, #3
-; CHECK-LE-NEXT:    mov.w r12, #0
-; CHECK-LE-NEXT:    lsls r1, r1, #31
+; CHECK-LE-NEXT:    mvnne r3, #1
+; CHECK-LE-NEXT:    bfi r3, r1, #0, #1
+; CHECK-LE-NEXT:    and r12, r3, #3
+; CHECK-LE-NEXT:    lsls r1, r3, #31
 ; CHECK-LE-NEXT:    itt ne
 ; CHECK-LE-NEXT:    ldrne r1, [r2]
-; CHECK-LE-NEXT:    vmovne.32 q1[0], r1
-; CHECK-LE-NEXT:    lsls r1, r3, #30
+; CHECK-LE-NEXT:    vmovne.32 q0[0], r1
+; CHECK-LE-NEXT:    lsls.w r1, r12, #30
 ; CHECK-LE-NEXT:    itt mi
 ; CHECK-LE-NEXT:    ldrmi r1, [r2, #4]
-; CHECK-LE-NEXT:    vmovmi.32 q1[2], r1
-; CHECK-LE-NEXT:    vmov r1, s0
+; CHECK-LE-NEXT:    vmovmi.32 q0[2], r1
+; CHECK-LE-NEXT:    vmov r1, s4
 ; CHECK-LE-NEXT:    movs r2, #0
-; CHECK-LE-NEXT:    vand q1, q1, q2
+; CHECK-LE-NEXT:    vand q0, q0, q2
 ; CHECK-LE-NEXT:    rsbs r3, r1, #0
-; CHECK-LE-NEXT:    vmov r3, s2
+; CHECK-LE-NEXT:    vmov r3, s6
 ; CHECK-LE-NEXT:    sbcs.w r1, r2, r1, asr #31
+; CHECK-LE-NEXT:    mov.w r1, #0
 ; CHECK-LE-NEXT:    it lt
-; CHECK-LE-NEXT:    movlt.w r12, #1
-; CHECK-LE-NEXT:    rsbs r1, r3, #0
-; CHECK-LE-NEXT:    sbcs.w r1, r2, r3, asr #31
+; CHECK-LE-NEXT:    movlt r1, #1
+; CHECK-LE-NEXT:    rsbs r5, r3, #0
+; CHECK-LE-NEXT:    sbcs.w r3, r2, r3, asr #31
 ; CHECK-LE-NEXT:    it lt
 ; CHECK-LE-NEXT:    movlt r2, #1
 ; CHECK-LE-NEXT:    cmp r2, #0
 ; CHECK-LE-NEXT:    it ne
 ; CHECK-LE-NEXT:    mvnne r2, #1
-; CHECK-LE-NEXT:    bfi r2, r12, #0, #1
+; CHECK-LE-NEXT:    bfi r2, r1, #0, #1
 ; CHECK-LE-NEXT:    and r1, r2, #3
 ; CHECK-LE-NEXT:    lsls r2, r2, #31
 ; CHECK-LE-NEXT:    itt ne
-; CHECK-LE-NEXT:    vmovne r2, r3, d2
+; CHECK-LE-NEXT:    vmovne r2, r3, d0
 ; CHECK-LE-NEXT:    strdne r2, r3, [r0]
 ; CHECK-LE-NEXT:    lsls r1, r1, #30
 ; CHECK-LE-NEXT:    itt mi
-; CHECK-LE-NEXT:    vmovmi r1, r2, d3
+; CHECK-LE-NEXT:    vmovmi r1, r2, d1
 ; CHECK-LE-NEXT:    strdmi r1, r2, [r0, #8]
 ; CHECK-LE-NEXT:    add sp, #4
-; CHECK-LE-NEXT:    pop {r7, pc}
+; CHECK-LE-NEXT:    pop {r4, r5, r7, pc}
 ;
 ; CHECK-BE-LABEL: foo_zext_v2i64_v2i32_unaligned:
 ; CHECK-BE:       @ %bb.0: @ %entry
 ; CHECK-BE-NEXT:    .save {r7, lr}
 ; CHECK-BE-NEXT:    push {r7, lr}
 ; CHECK-BE-NEXT:    .pad #4
 ; CHECK-BE-NEXT:    sub sp, #4
 ; CHECK-BE-NEXT:    ldrd r12, lr, [r1]
 ; CHECK-BE-NEXT:    rsbs.w r1, lr, #0
 ; CHECK-BE-NEXT:    mov.w r3, #0
 ; CHECK-BE-NEXT:    sbcs.w r1, r3, lr, asr #31
-; CHECK-BE-NEXT:    vmov.32 q0[1], r12
-; CHECK-BE-NEXT:    @ implicit-def: $q1
-; CHECK-BE-NEXT:    vmov.32 q0[3], lr
+; CHECK-BE-NEXT:    vmov q0[3], q0[1], lr, r12
 ; CHECK-BE-NEXT:    mov.w lr, #0
 ; CHECK-BE-NEXT:    it lt
 ; CHECK-BE-NEXT:    movlt.w lr, #1
 ; CHECK-BE-NEXT:    rsbs.w r1, r12, #0
 ; CHECK-BE-NEXT:    sbcs.w r1, r3, r12, asr #31
 ; CHECK-BE-NEXT:    it lt
 ; CHECK-BE-NEXT:    movlt r3, #1
 ; CHECK-BE-NEXT:    cmp r3, #0
 ; CHECK-BE-NEXT:    it ne
 ; CHECK-BE-NEXT:    mvnne r3, #1
 ; CHECK-BE-NEXT:    bfi r3, lr, #0, #1
+; CHECK-BE-NEXT:    @ implicit-def: $q1
 ; CHECK-BE-NEXT:    and r1, r3, #3
 ; CHECK-BE-NEXT:    lsls r3, r3, #31
 ; CHECK-BE-NEXT:    beq .LBB8_2
 ; CHECK-BE-NEXT:  @ %bb.1: @ %cond.load
 ; CHECK-BE-NEXT:    ldr r3, [r2]
 ; CHECK-BE-NEXT:    vmov.32 q2[1], r3
 ; CHECK-BE-NEXT:    vrev64.32 q1, q2
 ; CHECK-BE-NEXT:  .LBB8_2: @ %else

llvm/test/CodeGen/Thumb2/mve-minmax.ll

entry:
   ret <4 x i32> %1
 }
 
 define arm_aapcs_vfpcc <2 x i64> @smin_v2i64(<2 x i64> %s1, <2 x i64> %s2) {
 ; CHECK-LABEL: smin_v2i64:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
-; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    vmov r2, s4
 ; CHECK-NEXT:    movs r0, #0
-; CHECK-NEXT:    vmov r3, s2
-; CHECK-NEXT:    vmov r12, s7
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    vmov lr, s1
-; CHECK-NEXT:    subs r2, r3, r2
 ; CHECK-NEXT:    vmov r3, s0
-; CHECK-NEXT:    vmov r2, s4
-; CHECK-NEXT:    sbcs.w r1, r1, r12
 ; CHECK-NEXT:    vmov r12, s5
+; CHECK-NEXT:    vmov r1, s1
+; CHECK-NEXT:    vmov lr, s3
+; CHECK-NEXT:    subs r2, r3, r2
+; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    sbcs.w r1, r1, r12
+; CHECK-NEXT:    vmov r12, s7
 ; CHECK-NEXT:    mov.w r1, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r1, #1
 ; CHECK-NEXT:    cmp r1, #0
 ; CHECK-NEXT:    csetm r1, ne
 ; CHECK-NEXT:    subs r2, r3, r2
 ; CHECK-NEXT:    sbcs.w r2, lr, r12
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r0, #1
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov.32 q2[2], r1
-; CHECK-NEXT:    vmov.32 q2[3], r1
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r1
+; CHECK-NEXT:    vmov q2[3], q2[1], r0, r1
 ; CHECK-NEXT:    vbic q1, q1, q2
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vorr q0, q0, q1
 ; CHECK-NEXT:    pop {r7, pc}
 entry:
   %0 = icmp slt <2 x i64> %s1, %s2
   %1 = select <2 x i1> %0, <2 x i64> %s1, <2 x i64> %s2
   ret <2 x i64> %1
entry:
   ret <4 x i32> %1
 }
 
 define arm_aapcs_vfpcc <2 x i64> @umin_v2i64(<2 x i64> %s1, <2 x i64> %s2) {
 ; CHECK-LABEL: umin_v2i64:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
-; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    vmov r2, s4
 ; CHECK-NEXT:    movs r0, #0
-; CHECK-NEXT:    vmov r3, s2
-; CHECK-NEXT:    vmov r12, s7
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    vmov lr, s1
-; CHECK-NEXT:    subs r2, r3, r2
 ; CHECK-NEXT:    vmov r3, s0
-; CHECK-NEXT:    vmov r2, s4
-; CHECK-NEXT:    sbcs.w r1, r1, r12
 ; CHECK-NEXT:    vmov r12, s5
+; CHECK-NEXT:    vmov r1, s1
+; CHECK-NEXT:    vmov lr, s3
+; CHECK-NEXT:    subs r2, r3, r2
+; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    sbcs.w r1, r1, r12
+; CHECK-NEXT:    vmov r12, s7
 ; CHECK-NEXT:    mov.w r1, #0
 ; CHECK-NEXT:    it lo
 ; CHECK-NEXT:    movlo r1, #1
 ; CHECK-NEXT:    cmp r1, #0
 ; CHECK-NEXT:    csetm r1, ne
 ; CHECK-NEXT:    subs r2, r3, r2
 ; CHECK-NEXT:    sbcs.w r2, lr, r12
 ; CHECK-NEXT:    it lo
 ; CHECK-NEXT:    movlo r0, #1
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov.32 q2[2], r1
-; CHECK-NEXT:    vmov.32 q2[3], r1
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r1
+; CHECK-NEXT:    vmov q2[3], q2[1], r0, r1
 ; CHECK-NEXT:    vbic q1, q1, q2
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vorr q0, q0, q1
 ; CHECK-NEXT:    pop {r7, pc}
 entry:
   %0 = icmp ult <2 x i64> %s1, %s2
   %1 = select <2 x i1> %0, <2 x i64> %s1, <2 x i64> %s2
   ret <2 x i64> %1
entry:
   ret <4 x i32> %1
 }
 
 define arm_aapcs_vfpcc <2 x i64> @smax_v2i64(<2 x i64> %s1, <2 x i64> %s2) {
 ; CHECK-LABEL: smax_v2i64:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
-; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    movs r0, #0
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    vmov r12, s3
-; CHECK-NEXT:    vmov r1, s7
-; CHECK-NEXT:    vmov lr, s5
-; CHECK-NEXT:    subs r2, r3, r2
 ; CHECK-NEXT:    vmov r3, s4
-; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    sbcs.w r1, r1, r12
 ; CHECK-NEXT:    vmov r12, s1
+; CHECK-NEXT:    vmov r1, s5
+; CHECK-NEXT:    vmov lr, s7
+; CHECK-NEXT:    subs r2, r3, r2
+; CHECK-NEXT:    vmov r3, s6
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    sbcs.w r1, r1, r12
+; CHECK-NEXT:    vmov r12, s3
 ; CHECK-NEXT:    mov.w r1, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r1, #1
 ; CHECK-NEXT:    cmp r1, #0
 ; CHECK-NEXT:    csetm r1, ne
 ; CHECK-NEXT:    subs r2, r3, r2
 ; CHECK-NEXT:    sbcs.w r2, lr, r12
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r0, #1
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov.32 q2[2], r1
-; CHECK-NEXT:    vmov.32 q2[3], r1
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r1
+; CHECK-NEXT:    vmov q2[3], q2[1], r0, r1
 ; CHECK-NEXT:    vbic q1, q1, q2
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vorr q0, q0, q1
 ; CHECK-NEXT:    pop {r7, pc}
 entry:
   %0 = icmp sgt <2 x i64> %s1, %s2
   %1 = select <2 x i1> %0, <2 x i64> %s1, <2 x i64> %s2
   ret <2 x i64> %1
entry:
   ret <4 x i32> %1
 }
 
 define arm_aapcs_vfpcc <2 x i64> @umax_v2i64(<2 x i64> %s1, <2 x i64> %s2) {
 ; CHECK-LABEL: umax_v2i64:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
-; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    movs r0, #0
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    vmov r12, s3
-; CHECK-NEXT:    vmov r1, s7
-; CHECK-NEXT:    vmov lr, s5
-; CHECK-NEXT:    subs r2, r3, r2
 ; CHECK-NEXT:    vmov r3, s4
-; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    sbcs.w r1, r1, r12
 ; CHECK-NEXT:    vmov r12, s1
+; CHECK-NEXT:    vmov r1, s5
+; CHECK-NEXT:    vmov lr, s7
+; CHECK-NEXT:    subs r2, r3, r2
+; CHECK-NEXT:    vmov r3, s6
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    sbcs.w r1, r1, r12
+; CHECK-NEXT:    vmov r12, s3
 ; CHECK-NEXT:    mov.w r1, #0
 ; CHECK-NEXT:    it lo
 ; CHECK-NEXT:    movlo r1, #1
 ; CHECK-NEXT:    cmp r1, #0
 ; CHECK-NEXT:    csetm r1, ne
 ; CHECK-NEXT:    subs r2, r3, r2
 ; CHECK-NEXT:    sbcs.w r2, lr, r12
 ; CHECK-NEXT:    it lo
 ; CHECK-NEXT:    movlo r0, #1
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov.32 q2[2], r1
-; CHECK-NEXT:    vmov.32 q2[3], r1
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r1
+; CHECK-NEXT:    vmov q2[3], q2[1], r0, r1
 ; CHECK-NEXT:    vbic q1, q1, q2
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vorr q0, q0, q1
 ; CHECK-NEXT:    pop {r7, pc}
 entry:
   %0 = icmp ugt <2 x i64> %s1, %s2
   %1 = select <2 x i1> %0, <2 x i64> %s1, <2 x i64> %s2
   ret <2 x i64> %1

 ; CHECK-LABEL: maxnm_float64_t:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, lr}
 ; CHECK-NEXT:    push {r4, lr}
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11}
 ; CHECK-NEXT:    vmov q4, q1
 ; CHECK-NEXT:    vmov q5, q0
-; CHECK-NEXT:    vmov r0, r1, d9
-; CHECK-NEXT:    vmov r2, r3, d11
+; CHECK-NEXT:    vmov r0, r1, d8
+; CHECK-NEXT:    vmov r2, r3, d10
 ; CHECK-NEXT:    bl __aeabi_dcmpgt
-; CHECK-NEXT:    vmov r12, r1, d8
+; CHECK-NEXT:    vmov r12, r1, d9
 ; CHECK-NEXT:    cmp r0, #0
-; CHECK-NEXT:    vmov r2, r3, d10
+; CHECK-NEXT:    vmov r2, r3, d11
 ; CHECK-NEXT:    it ne
 ; CHECK-NEXT:    movne r0, #1
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    csetm r4, ne
 ; CHECK-NEXT:    mov r0, r12
 ; CHECK-NEXT:    bl __aeabi_dcmpgt
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    it ne
 ; CHECK-NEXT:    movne r0, #1
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.32 q0[1], r0
-; CHECK-NEXT:    vmov.32 q0[2], r4
-; CHECK-NEXT:    vmov.32 q0[3], r4
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r4
+; CHECK-NEXT:    vmov q0[3], q0[1], r0, r4
 ; CHECK-NEXT:    vbic q1, q5, q0
 ; CHECK-NEXT:    vand q0, q4, q0
 ; CHECK-NEXT:    vorr q0, q0, q1
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11}
 ; CHECK-NEXT:    pop {r4, pc}
 entry:
   %cmp = fcmp fast ogt <2 x double> %src2, %src1
   %0 = select <2 x i1> %cmp, <2 x double> %src2, <2 x double> %src1
   ret <2 x double> %0
 }

llvm/test/CodeGen/Thumb2/mve-neg.ll

 entry:
   %0 = sub nsw <4 x i32> zeroinitializer, %s1
   ret <4 x i32> %0
 }
 
 define arm_aapcs_vfpcc <2 x i64> @neg_v2i64(<2 x i64> %s1) {
 ; CHECK-LABEL: neg_v2i64:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vmov r1, s2
+; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    mov.w r12, #0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    vmov r3, s1
+; CHECK-NEXT:    vmov r0, s1
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    vmov r3, s3
 ; CHECK-NEXT:    rsbs r1, r1, #0
 ; CHECK-NEXT:    sbc.w r0, r12, r0
 ; CHECK-NEXT:    rsbs r2, r2, #0
 ; CHECK-NEXT:    sbc.w r3, r12, r3
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov.32 q0[1], r3
-; CHECK-NEXT:    vmov.32 q0[2], r1
-; CHECK-NEXT:    vmov.32 q0[3], r0
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r1
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r0
 ; CHECK-NEXT:    bx lr
 entry:
   %0 = sub nsw <2 x i64> zeroinitializer, %s1
   ret <2 x i64> %0
 }
 

llvm/test/CodeGen/Thumb2/mve-phireg.ll

 @c = dso_local global i32 2, align 4
 @d = dso_local global i32 2, align 4
 
 define dso_local i32 @e() #0 {
 ; CHECK-LABEL: e:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, lr}
 ; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, lr}
-; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
+; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
+; CHECK-NEXT:    vpush {d8, d9, d10, d11}
 ; CHECK-NEXT:    .pad #392
 ; CHECK-NEXT:    sub sp, #392
-; CHECK-NEXT:    movw r9, :lower16:.L_MergedGlobals
+; CHECK-NEXT:    movw r10, :lower16:.L_MergedGlobals
 ; CHECK-NEXT:    vldr s0, .LCPI1_0
-; CHECK-NEXT:    movt r9, :upper16:.L_MergedGlobals
+; CHECK-NEXT:    movt r10, :upper16:.L_MergedGlobals
 ; CHECK-NEXT:    vldr s3, .LCPI1_1
-; CHECK-NEXT:    mov r5, r9
-; CHECK-NEXT:    mov r7, r9
-; CHECK-NEXT:    ldr r1, [r5, #8]!
-; CHECK-NEXT:    vmov r6, s3
-; CHECK-NEXT:    ldr r0, [r7, #4]!
-; CHECK-NEXT:    movw r4, :lower16:e
-; CHECK-NEXT:    vmov.32 q4[0], r5
-; CHECK-NEXT:    movt r4, :upper16:e
-; CHECK-NEXT:    vmov q1, q4
-; CHECK-NEXT:    vmov s1, r7
-; CHECK-NEXT:    vmov.32 q1[1], r6
-; CHECK-NEXT:    vmov.32 q5[0], r7
-; CHECK-NEXT:    vmov.32 q1[2], r5
-; CHECK-NEXT:    vmov s9, r4
-; CHECK-NEXT:    vmov.32 q1[3], r4
-; CHECK-NEXT:    vdup.32 q6, r7
-; CHECK-NEXT:    vstrw.32 q1, [sp, #76]
-; CHECK-NEXT:    vmov q1, q5
-; CHECK-NEXT:    vmov.32 q1[1], r7
+; CHECK-NEXT:    mov r6, r10
+; CHECK-NEXT:    mov r7, r10
+; CHECK-NEXT:    ldr r1, [r6, #4]!
+; CHECK-NEXT:    movw r5, :lower16:e
+; CHECK-NEXT:    ldr r0, [r7, #8]!
+; CHECK-NEXT:    vmov r4, s3
+; CHECK-NEXT:    vmov s1, r6
+; CHECK-NEXT:    movt r5, :upper16:e
+; CHECK-NEXT:    vmov q1[2], q1[0], r7, r7
+; CHECK-NEXT:    vmov s9, r5
+; CHECK-NEXT:    vdup.32 q4, r6
 ; CHECK-NEXT:    vmov.f32 s2, s1
 ; CHECK-NEXT:    vmov.f32 s8, s0
-; CHECK-NEXT:    vmov.32 q1[2], r6
-; CHECK-NEXT:    vmov q3, q6
-; CHECK-NEXT:    vmov q7, q6
+; CHECK-NEXT:    vmov q1[3], q1[1], r5, r4
+; CHECK-NEXT:    vmov q3, q4
+; CHECK-NEXT:    vmov q5, q4
 ; CHECK-NEXT:    vmov.f32 s10, s1
+; CHECK-NEXT:    vstrw.32 q1, [sp, #76]
+; CHECK-NEXT:    vmov q1[2], q1[0], r4, r6
 ; CHECK-NEXT:    mov.w r8, #4
-; CHECK-NEXT:    mov.w r10, #0
-; CHECK-NEXT:    vmov.32 q1[3], r4
-; CHECK-NEXT:    vmov.32 q3[0], r4
-; CHECK-NEXT:    vmov.32 q7[1], r4
-; CHECK-NEXT:    str r1, [r0]
+; CHECK-NEXT:    mov.w r9, #0
+; CHECK-NEXT:    vmov q1[3], q1[1], r5, r6
+; CHECK-NEXT:    vmov.32 q3[0], r5
+; CHECK-NEXT:    vmov.32 q5[1], r5
 ; CHECK-NEXT:    vmov.f32 s11, s3
+; CHECK-NEXT:    str r1, [sp, #24]
+; CHECK-NEXT:    str r1, [r0]
 ; CHECK-NEXT:    movs r1, #64
 ; CHECK-NEXT:    strh.w r8, [sp, #390]
-; CHECK-NEXT:    strd r0, r10, [sp, #24]
 ; CHECK-NEXT:    vstrw.32 q0, [sp, #44]
-; CHECK-NEXT:    str r0, [r0]
+; CHECK-NEXT:    str.w r9, [sp, #28]
 ; CHECK-NEXT:    vstrw.32 q2, [r0]
-; CHECK-NEXT:    vstrw.32 q7, [r0]
+; CHECK-NEXT:    str r0, [r0]
+; CHECK-NEXT:    vstrw.32 q5, [r0]
 ; CHECK-NEXT:    vstrw.32 q3, [r0]
 ; CHECK-NEXT:    vstrw.32 q1, [r0]
 ; CHECK-NEXT:    bl __aeabi_memclr4
-; CHECK-NEXT:    vmov.32 q5[1], r5
-; CHECK-NEXT:    vmov.32 q4[1], r4
-; CHECK-NEXT:    vmov.32 q5[2], r7
-; CHECK-NEXT:    vmov.32 q4[2], r7
-; CHECK-NEXT:    vmov.32 q5[3], r6
-; CHECK-NEXT:    vmov.32 q6[0], r10
-; CHECK-NEXT:    vmov.32 q4[3], r5
-; CHECK-NEXT:    str.w r10, [r9]
+; CHECK-NEXT:    vmov q0[2], q0[0], r6, r7
+; CHECK-NEXT:    vmov q1[2], q1[0], r6, r6
+; CHECK-NEXT:    vmov q0[3], q0[1], r7, r5
+; CHECK-NEXT:    vmov q1[3], q1[1], r4, r7
+; CHECK-NEXT:    vmov.32 q4[0], r9
+; CHECK-NEXT:    vstrw.32 q0, [r0]
+; CHECK-NEXT:    str.w r9, [r10]
 ; CHECK-NEXT:    vstrw.32 q4, [r0]
-; CHECK-NEXT:    vstrw.32 q6, [r0]
-; CHECK-NEXT:    vstrw.32 q5, [r0]
+; CHECK-NEXT:    vstrw.32 q1, [r0]
 ; CHECK-NEXT:    str.w r8, [sp, #308]
 ; CHECK-NEXT:  .LBB1_1: @ %for.cond
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    b .LBB1_1
 ; CHECK-NEXT:    .p2align 2
 ; CHECK-NEXT:  @ %bb.2:
 ; CHECK-NEXT:  .LCPI1_0:
 ; CHECK-NEXT:    .long 0x00000004 @ float 5.60519386E-45

llvm/test/CodeGen/Thumb2/mve-pred-and.ll

 
 
 define arm_aapcs_vfpcc <2 x i64> @cmpeqz_v2i1(<2 x i64> %a, <2 x i64> %b) {
 ; CHECK-LABEL: cmpeqz_v2i1:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vorr q2, q0, q1
 ; CHECK-NEXT:    vmov r0, s9
 ; CHECK-NEXT:    vmov r1, s8
+; CHECK-NEXT:    vmov r2, s10
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s10
+; CHECK-NEXT:    vmov r1, s11
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov r0, s11
-; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vbic q1, q1, q3
-; CHECK-NEXT:    vand q0, q0, q3
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
+; CHECK-NEXT:    vbic q1, q1, q2
+; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vorr q0, q0, q1
 ; CHECK-NEXT:    bx lr
 entry:
   %c1 = icmp eq <2 x i64> %a, zeroinitializer
   %c2 = icmp eq <2 x i64> %b, zeroinitializer
   %o = and <2 x i1> %c1, %c2
   %s = select <2 x i1> %o, <2 x i64> %a, <2 x i64> %b
   ret <2 x i64> %s
 }
 
 define arm_aapcs_vfpcc <2 x i64> @cmpeq_v2i1(<2 x i64> %a, <2 x i64> %b, <2 x i64> %c) {
 ; CHECK-LABEL: cmpeq_v2i1:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s9
 ; CHECK-NEXT:    vmov r1, s5
 ; CHECK-NEXT:    vmov r2, s4
+; CHECK-NEXT:    vmov r3, s6
 ; CHECK-NEXT:    eors r0, r1
 ; CHECK-NEXT:    vmov r1, s8
 ; CHECK-NEXT:    eors r1, r2
-; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    vmov r2, s7
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s7
+; CHECK-NEXT:    vmov r1, s11
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov r0, s11
-; CHECK-NEXT:    eors r0, r1
-; CHECK-NEXT:    vmov r1, s10
 ; CHECK-NEXT:    eors r1, r2
-; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s0
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    vmov r2, s10
+; CHECK-NEXT:    eors r2, r3
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vmov r0, s1
+; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    orrs r0, r1
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vand q2, q2, q3
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
+; CHECK-NEXT:    vand q2, q3, q2
 ; CHECK-NEXT:    vbic q1, q1, q2
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vorr q0, q0, q1
 ; CHECK-NEXT:    bx lr
 entry:
   %c1 = icmp eq <2 x i64> %a, zeroinitializer
   %c2 = icmp eq <2 x i64> %b, %c
   %o = and <2 x i1> %c1, %c2
   %s = select <2 x i1> %o, <2 x i64> %a, <2 x i64> %b
   ret <2 x i64> %s
 }
 
 define arm_aapcs_vfpcc <2 x i64> @cmpeqr_v2i1(<2 x i64> %a, <2 x i64> %b, i64 %c) {
 ; CHECK-LABEL: cmpeqr_v2i1:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r2, s5
 ; CHECK-NEXT:    vmov r3, s4
 ; CHECK-NEXT:    eors r2, r1
 ; CHECK-NEXT:    eors r3, r0
 ; CHECK-NEXT:    orrs r2, r3
+; CHECK-NEXT:    vmov r3, s7
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov r2, s7
-; CHECK-NEXT:    eors r1, r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    eors r0, r2
+; CHECK-NEXT:    eors r1, r3
+; CHECK-NEXT:    vmov r3, s6
+; CHECK-NEXT:    eors r0, r3
 ; CHECK-NEXT:    orrs r0, r1
 ; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r2
+; CHECK-NEXT:    vmov q2[3], q2[1], r0, r2
 ; CHECK-NEXT:    vmov r0, s1
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    orrs r0, r1
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
 ; CHECK-NEXT:    vand q2, q3, q2
 ; CHECK-NEXT:    vbic q1, q1, q2
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vorr q0, q0, q1
 ; CHECK-NEXT:    bx lr
 entry:
   %c1 = icmp eq <2 x i64> %a, zeroinitializer
   %i = insertelement <2 x i64> undef, i64 %c, i32 0
   %sp = shufflevector <2 x i64> %i, <2 x i64> undef, <2 x i32> zeroinitializer
   %c2 = icmp eq <2 x i64> %b, %sp
   %o = and <2 x i1> %c1, %c2
   %s = select <2 x i1> %o, <2 x i64> %a, <2 x i64> %b
   ret <2 x i64> %s
 }

llvm/test/CodeGen/Thumb2/mve-pred-bitcast.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve -verify-machineinstrs %s -o - | FileCheck %s --check-prefix=CHECK-LE
 ; RUN: llc -mtriple=thumbebv8.1m.main-none-none-eabi -mattr=+mve -verify-machineinstrs %s -o - | FileCheck %s --check-prefix=CHECK-BE
 
 define arm_aapcs_vfpcc <4 x i32> @bitcast_to_v4i1(i4 %b, <4 x i32> %a) {
 ; CHECK-LE-LABEL: bitcast_to_v4i1:
 ; CHECK-LE:       @ %bb.0: @ %entry
 ; CHECK-LE-NEXT:    .pad #4
 ; CHECK-LE-NEXT:    sub sp, #4
 ; CHECK-LE-NEXT:    and r0, r0, #15
 ; CHECK-LE-NEXT:    vmov.i8 q1, #0x0
 ; CHECK-LE-NEXT:    vmov.i8 q2, #0xff
 ; CHECK-LE-NEXT:    vmsr p0, r0
 ; CHECK-LE-NEXT:    vpsel q1, q2, q1
 ; CHECK-LE-NEXT:    vmov.u8 r0, q1[0]
-; CHECK-LE-NEXT:    vmov.32 q2[0], r0
+; CHECK-LE-NEXT:    vmov.u8 r1, q1[2]
+; CHECK-LE-NEXT:    vmov q2[2], q2[0], r1, r0
 ; CHECK-LE-NEXT:    vmov.u8 r0, q1[1]
-; CHECK-LE-NEXT:    vmov.32 q2[1], r0
-; CHECK-LE-NEXT:    vmov.u8 r0, q1[2]
-; CHECK-LE-NEXT:    vmov.32 q2[2], r0
-; CHECK-LE-NEXT:    vmov.u8 r0, q1[3]
-; CHECK-LE-NEXT:    vmov.32 q2[3], r0
+; CHECK-LE-NEXT:    vmov.u8 r1, q1[3]
 ; CHECK-LE-NEXT:    vmov.i32 q1, #0x0
+; CHECK-LE-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-LE-NEXT:    vcmp.i32 ne, q2, zr
 ; CHECK-LE-NEXT:    vpsel q0, q0, q1
 ; CHECK-LE-NEXT:    add sp, #4
 ; CHECK-LE-NEXT:    bx lr
 ;
 ; CHECK-BE-LABEL: bitcast_to_v4i1:
 ; CHECK-BE:       @ %bb.0: @ %entry
 ; CHECK-BE-NEXT:    .pad #4
 ; CHECK-BE-NEXT:    sub sp, #4
 ; CHECK-BE-NEXT:    and r0, r0, #15
 ; CHECK-BE-NEXT:    vmov.i8 q1, #0x0
 ; CHECK-BE-NEXT:    vmov.i8 q2, #0xff
 ; CHECK-BE-NEXT:    vmsr p0, r0
 ; CHECK-BE-NEXT:    vpsel q1, q2, q1
 ; CHECK-BE-NEXT:    vmov.u8 r0, q1[0]
-; CHECK-BE-NEXT:    vmov.32 q2[0], r0
+; CHECK-BE-NEXT:    vmov.u8 r1, q1[2]
+; CHECK-BE-NEXT:    vmov q2[2], q2[0], r1, r0
 ; CHECK-BE-NEXT:    vmov.u8 r0, q1[1]
-; CHECK-BE-NEXT:    vmov.32 q2[1], r0
-; CHECK-BE-NEXT:    vmov.u8 r0, q1[2]
-; CHECK-BE-NEXT:    vmov.32 q2[2], r0
-; CHECK-BE-NEXT:    vmov.u8 r0, q1[3]
-; CHECK-BE-NEXT:    vmov.32 q2[3], r0
+; CHECK-BE-NEXT:    vmov.u8 r1, q1[3]
 ; CHECK-BE-NEXT:    vrev64.32 q1, q0
-; CHECK-BE-NEXT:    vcmp.i32 ne, q2, zr
+; CHECK-BE-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-BE-NEXT:    vmov.i32 q0, #0x0
+; CHECK-BE-NEXT:    vcmp.i32 ne, q2, zr
 ; CHECK-BE-NEXT:    vpsel q1, q1, q0
 ; CHECK-BE-NEXT:    vrev64.32 q0, q1
 ; CHECK-BE-NEXT:    add sp, #4
 ; CHECK-BE-NEXT:    bx lr
 entry:
   %c = bitcast i4 %b to <4 x i1>
   %s = select <4 x i1> %c, <4 x i32> %a, <4 x i32> zeroinitializer
   ret <4 x i32> %s

 
 define arm_aapcs_vfpcc <2 x i64> @bitcast_to_v2i1(i2 %b, <2 x i64> %a) {
 ; CHECK-LE-LABEL: bitcast_to_v2i1:
 ; CHECK-LE:       @ %bb.0: @ %entry
 ; CHECK-LE-NEXT:    .pad #4
 ; CHECK-LE-NEXT:    sub sp, #4
 ; CHECK-LE-NEXT:    and r1, r0, #2
 ; CHECK-LE-NEXT:    and r0, r0, #1
-; CHECK-LE-NEXT:    rsbs r0, r0, #0
 ; CHECK-LE-NEXT:    movs r2, #0
-; CHECK-LE-NEXT:    vmov.32 q1[0], r0
+; CHECK-LE-NEXT:    rsbs r0, r0, #0
 ; CHECK-LE-NEXT:    sub.w r1, r2, r1, lsr #1
-; CHECK-LE-NEXT:    vmov.32 q1[1], r0
-; CHECK-LE-NEXT:    vmov.32 q1[2], r1
-; CHECK-LE-NEXT:    vmov.32 q1[3], r1
+; CHECK-LE-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-LE-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-LE-NEXT:    vand q0, q0, q1
 ; CHECK-LE-NEXT:    add sp, #4
 ; CHECK-LE-NEXT:    bx lr
 ;
 ; CHECK-BE-LABEL: bitcast_to_v2i1:
 ; CHECK-BE:       @ %bb.0: @ %entry
 ; CHECK-BE-NEXT:    .pad #4
 ; CHECK-BE-NEXT:    sub sp, #4
 ; CHECK-BE-NEXT:    and r1, r0, #2
-; CHECK-BE-NEXT:    movs r2, #0
 ; CHECK-BE-NEXT:    and r0, r0, #1
-; CHECK-BE-NEXT:    sub.w r1, r2, r1, lsr #1
+; CHECK-BE-NEXT:    movs r2, #0
 ; CHECK-BE-NEXT:    rsbs r0, r0, #0
-; CHECK-BE-NEXT:    vmov.32 q1[0], r1
-; CHECK-BE-NEXT:    vmov.32 q1[1], r1
-; CHECK-BE-NEXT:    vmov.32 q1[2], r0
-; CHECK-BE-NEXT:    vmov.32 q1[3], r0
+; CHECK-BE-NEXT:    sub.w r1, r2, r1, lsr #1
+; CHECK-BE-NEXT:    vmov q1[2], q1[0], r0, r1
+; CHECK-BE-NEXT:    vmov q1[3], q1[1], r0, r1
 ; CHECK-BE-NEXT:    vrev64.32 q2, q1
 ; CHECK-BE-NEXT:    vand q0, q0, q2
 ; CHECK-BE-NEXT:    add sp, #4
 ; CHECK-BE-NEXT:    bx lr
 entry:
   %c = bitcast i2 %b to <2 x i1>
   %s = select <2 x i1> %c, <2 x i64> %a, <2 x i64> zeroinitializer
   ret <2 x i64> %s

llvm/test/CodeGen/Thumb2/mve-pred-ext.ll

entry:
   %c = icmp sgt <16 x i8> %src, zeroinitializer
   %0 = sext <16 x i1> %c to <16 x i8>
   ret <16 x i8> %0
 }
 
 define arm_aapcs_vfpcc <2 x i64> @sext_v2i1_v2i64(<2 x i64> %src) {
 ; CHECK-LABEL: sext_v2i1_v2i64:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vmov r1, s2
+; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    movs r2, #0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    vmov r0, s1
+; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    rsbs r1, r1, #0
-; CHECK-NEXT:    vmov r1, s1
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    sbcs.w r0, r2, r0
 ; CHECK-NEXT:    mov.w r0, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r0, #1
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    csetm r0, ne
 ; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    sbcs.w r1, r2, r1
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r2, #1
 ; CHECK-NEXT:    cmp r2, #0
 ; CHECK-NEXT:    csetm r1, ne
-; CHECK-NEXT:    vmov.32 q0[0], r1
-; CHECK-NEXT:    vmov.32 q0[1], r1
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    vmov.32 q0[3], r0
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp sgt <2 x i64> %src, zeroinitializer
   %0 = sext <2 x i1> %c to <2 x i64>
   ret <2 x i64> %0
 }
 
 
entry:
   %c = icmp sgt <16 x i8> %src, zeroinitializer
   %0 = zext <16 x i1> %c to <16 x i8>
   ret <16 x i8> %0
 }
 
 define arm_aapcs_vfpcc <2 x i64> @zext_v2i1_v2i64(<2 x i64> %src) {
 ; CHECK-LABEL: zext_v2i1_v2i64:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    adr r1, .LCPI7_0
 ; CHECK-NEXT:    vldrw.u32 q1, [r1]
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    vmov r1, s1
+; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    movs r0, #0
 ; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov r2, s1
+; CHECK-NEXT:    vmov r2, s3
 ; CHECK-NEXT:    sbcs.w r1, r0, r1
 ; CHECK-NEXT:    mov.w r1, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r1, #1
 ; CHECK-NEXT:    cmp r1, #0
 ; CHECK-NEXT:    csetm r1, ne
 ; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    sbcs.w r2, r0, r2
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r0, #1
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.32 q0[2], r1
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r1
 ; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    bx lr
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.1:
 ; CHECK-NEXT:  .LCPI7_0:
 ; CHECK-NEXT:    .long 1 @ 0x1
 ; CHECK-NEXT:    .long 0 @ 0x0
 ; CHECK-NEXT:    .long 1 @ 0x1
entry:
   %0 = trunc <16 x i8> %src to <16 x i1>
   %1 = select <16 x i1> %0, <16 x i8> %src, <16 x i8> zeroinitializer
   ret <16 x i8> %1
 }
 
 define arm_aapcs_vfpcc <2 x i64> @trunc_v2i1_v2i64(<2 x i64> %src) {
 ; CHECK-LABEL: trunc_v2i1_v2i64:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vmov r1, s0
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    and r1, r1, #1
-; CHECK-NEXT:    rsbs r1, r1, #0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov r1, s2
 ; CHECK-NEXT:    and r0, r0, #1
-; CHECK-NEXT:    vmov.32 q1[0], r1
+; CHECK-NEXT:    and r1, r1, #1
 ; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    rsbs r1, r1, #0
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    bx lr
 entry:
   %0 = trunc <2 x i64> %src to <2 x i1>
   %1 = select <2 x i1> %0, <2 x i64> %src, <2 x i64> zeroinitializer
   ret <2 x i64> %1
 }
 

llvm/test/CodeGen/Thumb2/mve-pred-loadstore.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve -verify-machineinstrs %s -o - | FileCheck %s --check-prefix=CHECK-LE
 ; RUN: llc -mtriple=thumbebv8.1m.main-none-none-eabi -mattr=+mve -verify-machineinstrs %s -o - | FileCheck %s --check-prefix=CHECK-BE
 
 define arm_aapcs_vfpcc <4 x i32> @load_v4i1(<4 x i1> *%src, <4 x i32> %a) {
 ; CHECK-LE-LABEL: load_v4i1:
 ; CHECK-LE:       @ %bb.0: @ %entry
 ; CHECK-LE-NEXT:    ldrb r0, [r0]
 ; CHECK-LE-NEXT:    vmov.i8 q1, #0x0
 ; CHECK-LE-NEXT:    vmov.i8 q2, #0xff
 ; CHECK-LE-NEXT:    vmsr p0, r0
 ; CHECK-LE-NEXT:    vpsel q1, q2, q1
 ; CHECK-LE-NEXT:    vmov.u8 r0, q1[0]
-; CHECK-LE-NEXT:    vmov.32 q2[0], r0
+; CHECK-LE-NEXT:    vmov.u8 r1, q1[2]
+; CHECK-LE-NEXT:    vmov q2[2], q2[0], r1, r0
 ; CHECK-LE-NEXT:    vmov.u8 r0, q1[1]
-; CHECK-LE-NEXT:    vmov.32 q2[1], r0
-; CHECK-LE-NEXT:    vmov.u8 r0, q1[2]
-; CHECK-LE-NEXT:    vmov.32 q2[2], r0
-; CHECK-LE-NEXT:    vmov.u8 r0, q1[3]
-; CHECK-LE-NEXT:    vmov.32 q2[3], r0
+; CHECK-LE-NEXT:    vmov.u8 r1, q1[3]
 ; CHECK-LE-NEXT:    vmov.i32 q1, #0x0
+; CHECK-LE-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-LE-NEXT:    vcmp.i32 ne, q2, zr
 ; CHECK-LE-NEXT:    vpsel q0, q0, q1
 ; CHECK-LE-NEXT:    bx lr
 ;
 ; CHECK-BE-LABEL: load_v4i1:
 ; CHECK-BE:       @ %bb.0: @ %entry
 ; CHECK-BE-NEXT:    ldrb r0, [r0]
 ; CHECK-BE-NEXT:    vmov.i8 q1, #0x0
 ; CHECK-BE-NEXT:    vmov.i8 q2, #0xff
 ; CHECK-BE-NEXT:    vmsr p0, r0
 ; CHECK-BE-NEXT:    vpsel q1, q2, q1
 ; CHECK-BE-NEXT:    vmov.u8 r0, q1[0]
-; CHECK-BE-NEXT:    vmov.32 q2[0], r0
+; CHECK-BE-NEXT:    vmov.u8 r1, q1[2]
+; CHECK-BE-NEXT:    vmov q2[2], q2[0], r1, r0
 ; CHECK-BE-NEXT:    vmov.u8 r0, q1[1]
-; CHECK-BE-NEXT:    vmov.32 q2[1], r0
-; CHECK-BE-NEXT:    vmov.u8 r0, q1[2]
-; CHECK-BE-NEXT:    vmov.32 q2[2], r0
-; CHECK-BE-NEXT:    vmov.u8 r0, q1[3]
-; CHECK-BE-NEXT:    vmov.32 q2[3], r0
+; CHECK-BE-NEXT:    vmov.u8 r1, q1[3]
 ; CHECK-BE-NEXT:    vrev64.32 q1, q0
-; CHECK-BE-NEXT:    vcmp.i32 ne, q2, zr
+; CHECK-BE-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-BE-NEXT:    vmov.i32 q0, #0x0
+; CHECK-BE-NEXT:    vcmp.i32 ne, q2, zr
 ; CHECK-BE-NEXT:    vpsel q1, q1, q0
 ; CHECK-BE-NEXT:    vrev64.32 q0, q1
 ; CHECK-BE-NEXT:    bx lr
 entry:
   %c = load <4 x i1>, <4 x i1>* %src
   %s = select <4 x i1> %c, <4 x i32> %a, <4 x i32> zeroinitializer
   ret <4 x i32> %s
 }
entry:
   ret <16 x i8> %s
 }
 
 define arm_aapcs_vfpcc <2 x i64> @load_v2i1(<2 x i1> *%src, <2 x i64> %a) {
 ; CHECK-LE-LABEL: load_v2i1:
 ; CHECK-LE:       @ %bb.0: @ %entry
 ; CHECK-LE-NEXT:    ldrb r0, [r0]
 ; CHECK-LE-NEXT:    and r1, r0, #1
-; CHECK-LE-NEXT:    rsbs r1, r1, #0
 ; CHECK-LE-NEXT:    ubfx r0, r0, #1, #1
-; CHECK-LE-NEXT:    vmov.32 q1[0], r1
+; CHECK-LE-NEXT:    rsbs r1, r1, #0
 ; CHECK-LE-NEXT:    rsbs r0, r0, #0
-; CHECK-LE-NEXT:    vmov.32 q1[1], r1
-; CHECK-LE-NEXT:    vmov.32 q1[2], r0
-; CHECK-LE-NEXT:    vmov.32 q1[3], r0
+; CHECK-LE-NEXT:    vmov q1[2], q1[0], r0, r1
+; CHECK-LE-NEXT:    vmov q1[3], q1[1], r0, r1
 ; CHECK-LE-NEXT:    vand q0, q0, q1
 ; CHECK-LE-NEXT:    bx lr
 ;
 ; CHECK-BE-LABEL: load_v2i1:
 ; CHECK-BE:       @ %bb.0: @ %entry
 ; CHECK-BE-NEXT:    ldrb r0, [r0]
 ; CHECK-BE-NEXT:    ubfx r1, r0, #1, #1
 ; CHECK-BE-NEXT:    and r0, r0, #1
 ; CHECK-BE-NEXT:    rsbs r1, r1, #0
 ; CHECK-BE-NEXT:    rsbs r0, r0, #0
-; CHECK-BE-NEXT:    vmov.32 q1[0], r1
-; CHECK-BE-NEXT:    vmov.32 q1[1], r1
-; CHECK-BE-NEXT:    vmov.32 q1[2], r0
-; CHECK-BE-NEXT:    vmov.32 q1[3], r0
+; CHECK-BE-NEXT:    vmov q1[2], q1[0], r0, r1
+; CHECK-BE-NEXT:    vmov q1[3], q1[1], r0, r1
 ; CHECK-BE-NEXT:    vrev64.32 q2, q1
 ; CHECK-BE-NEXT:    vand q0, q0, q2
 ; CHECK-BE-NEXT:    bx lr
 entry:
   %c = load <2 x i1>, <2 x i1>* %src
   %s = select <2 x i1> %c, <2 x i64> %a, <2 x i64> zeroinitializer
   ret <2 x i64> %s
 }

llvm/test/CodeGen/Thumb2/mve-pred-not.ll

 }
 
 
 define arm_aapcs_vfpcc <2 x i64> @cmpeqz_v2i1(<2 x i64> %a, <2 x i64> %b) {
 ; CHECK-LABEL: cmpeqz_v2i1:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    vmov r1, s0
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s2
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vbic q0, q0, q2
 ; CHECK-NEXT:    vand q1, q1, q2
 ; CHECK-NEXT:    vorr q0, q1, q0
 ; CHECK-NEXT:    bx lr
 entry:
   %c1 = icmp eq <2 x i64> %a, zeroinitializer
   %o = xor <2 x i1> %c1, <i1 -1, i1 -1>
   %s = select <2 x i1> %o, <2 x i64> %a, <2 x i64> %b
   ret <2 x i64> %s
 }
 
 define arm_aapcs_vfpcc <2 x i64> @cmpeq_v2i1(<2 x i64> %a, <2 x i64> %b, <2 x i64> %c) {
 ; CHECK-LABEL: cmpeq_v2i1:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    vmov r1, s0
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    orrs r0, r1
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vbic q0, q0, q2
 ; CHECK-NEXT:    vand q1, q1, q2
 ; CHECK-NEXT:    vorr q0, q1, q0
 ; CHECK-NEXT:    bx lr
 entry:
   %c1 = icmp eq <2 x i64> %a, zeroinitializer
   %o = xor <2 x i1> %c1, <i1 -1, i1 -1>
   %s = select <2 x i1> %o, <2 x i64> %a, <2 x i64> %b

llvm/test/CodeGen/Thumb2/mve-pred-or.ll

 }
 
 
 define arm_aapcs_vfpcc <2 x i64> @cmpeqz_v2i1(<2 x i64> %a, <2 x i64> %b) {
 ; CHECK-LABEL: cmpeqz_v2i1:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s5
 ; CHECK-NEXT:    vmov r1, s4
+; CHECK-NEXT:    vmov r2, s6
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s6
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s7
-; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s0
+; CHECK-NEXT:    vmov r1, s7
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vmov r0, s1
+; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    orrs r0, r1
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
 ; CHECK-NEXT:    vorr q2, q3, q2
 ; CHECK-NEXT:    vbic q1, q1, q2
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vorr q0, q0, q1
 ; CHECK-NEXT:    bx lr
 entry:
   %c1 = icmp eq <2 x i64> %a, zeroinitializer
   %c2 = icmp eq <2 x i64> %b, zeroinitializer
   %o = or <2 x i1> %c1, %c2
   %s = select <2 x i1> %o, <2 x i64> %a, <2 x i64> %b
   ret <2 x i64> %s
 }
 
 define arm_aapcs_vfpcc <2 x i64> @cmpeq_v2i1(<2 x i64> %a, <2 x i64> %b, <2 x i64> %c) {
 ; CHECK-LABEL: cmpeq_v2i1:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s9
 ; CHECK-NEXT:    vmov r1, s5
 ; CHECK-NEXT:    vmov r2, s4
+; CHECK-NEXT:    vmov r3, s6
 ; CHECK-NEXT:    eors r0, r1
 ; CHECK-NEXT:    vmov r1, s8
 ; CHECK-NEXT:    eors r1, r2
-; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    vmov r2, s7
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s7
+; CHECK-NEXT:    vmov r1, s11
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov r0, s11
-; CHECK-NEXT:    eors r0, r1
-; CHECK-NEXT:    vmov r1, s10
 ; CHECK-NEXT:    eors r1, r2
-; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s0
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    vmov r2, s10
+; CHECK-NEXT:    eors r2, r3
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vmov r0, s1
+; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    orrs r0, r1
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vorr q2, q2, q3
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
+; CHECK-NEXT:    vorr q2, q3, q2
 ; CHECK-NEXT:    vbic q1, q1, q2
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vorr q0, q0, q1
 ; CHECK-NEXT:    bx lr
 entry:
   %c1 = icmp eq <2 x i64> %a, zeroinitializer
   %c2 = icmp eq <2 x i64> %b, %c
   %o = or <2 x i1> %c1, %c2
   %s = select <2 x i1> %o, <2 x i64> %a, <2 x i64> %b
   ret <2 x i64> %s
 }
 
 

llvm/test/CodeGen/Thumb2/mve-pred-shuffle.ll

 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov d1, r2, r3
 ; CHECK-NEXT:    vmov.i8 q1, #0xff
 ; CHECK-NEXT:    vmov d0, r0, r1
 ; CHECK-NEXT:    vcmp.i16 eq, q0, zr
 ; CHECK-NEXT:    vmov.i8 q0, #0x0
 ; CHECK-NEXT:    vpsel q0, q1, q0
 ; CHECK-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[2]
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[3]
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[3]
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    add r0, sp, #16
 ; CHECK-NEXT:    vldrw.u32 q0, [r0]
 ; CHECK-NEXT:    mov r0, sp
 ; CHECK-NEXT:    vcmp.i32 ne, q1, zr
 ; CHECK-NEXT:    vldrw.u32 q1, [r0]
 ; CHECK-NEXT:    vpsel q0, q1, q0
 ; CHECK-NEXT:    vmov r0, r1, d0
 ; CHECK-NEXT:    vmov r2, r3, d1

 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov d1, r2, r3
 ; CHECK-NEXT:    vmov.i8 q1, #0xff
 ; CHECK-NEXT:    vmov d0, r0, r1
 ; CHECK-NEXT:    vcmp.i16 eq, q0, zr
 ; CHECK-NEXT:    vmov.i8 q0, #0x0
 ; CHECK-NEXT:    vpsel q0, q1, q0
 ; CHECK-NEXT:    vmov.u16 r0, q0[4]
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[6]
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[5]
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[6]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[7]
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[7]
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    add r0, sp, #16
 ; CHECK-NEXT:    vldrw.u32 q0, [r0]
 ; CHECK-NEXT:    mov r0, sp
 ; CHECK-NEXT:    vcmp.i32 ne, q1, zr
 ; CHECK-NEXT:    vldrw.u32 q1, [r0]
 ; CHECK-NEXT:    vpsel q0, q1, q0
 ; CHECK-NEXT:    vmov r0, r1, d0
 ; CHECK-NEXT:    vmov r2, r3, d1

llvm/test/CodeGen/Thumb2/mve-pred-xor.ll

 }
 
 
 define arm_aapcs_vfpcc <2 x i64> @cmpeqz_v2i1(<2 x i64> %a, <2 x i64> %b) {
 ; CHECK-LABEL: cmpeqz_v2i1:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s5
 ; CHECK-NEXT:    vmov r1, s4
+; CHECK-NEXT:    vmov r2, s6
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s6
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s7
-; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s0
+; CHECK-NEXT:    vmov r1, s7
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vmov r0, s1
+; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    orrs r0, r1
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
 ; CHECK-NEXT:    veor q2, q3, q2
 ; CHECK-NEXT:    vbic q1, q1, q2
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vorr q0, q0, q1
 ; CHECK-NEXT:    bx lr
 entry:
   %c1 = icmp eq <2 x i64> %a, zeroinitializer
   %c2 = icmp eq <2 x i64> %b, zeroinitializer
   %o = xor <2 x i1> %c1, %c2
   %s = select <2 x i1> %o, <2 x i64> %a, <2 x i64> %b
   ret <2 x i64> %s
 }
 
 define arm_aapcs_vfpcc <2 x i64> @cmpeq_v2i1(<2 x i64> %a, <2 x i64> %b, <2 x i64> %c) {
 ; CHECK-LABEL: cmpeq_v2i1:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s9
 ; CHECK-NEXT:    vmov r1, s5
 ; CHECK-NEXT:    vmov r2, s4
+; CHECK-NEXT:    vmov r3, s6
 ; CHECK-NEXT:    eors r0, r1
 ; CHECK-NEXT:    vmov r1, s8
 ; CHECK-NEXT:    eors r1, r2
-; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    vmov r2, s7
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s7
+; CHECK-NEXT:    vmov r1, s11
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov r0, s11
-; CHECK-NEXT:    eors r0, r1
-; CHECK-NEXT:    vmov r1, s10
 ; CHECK-NEXT:    eors r1, r2
-; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s0
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    vmov r2, s10
+; CHECK-NEXT:    eors r2, r3
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vmov r0, s1
+; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    orrs r0, r1
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    veor q2, q2, q3
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
+; CHECK-NEXT:    veor q2, q3, q2
 ; CHECK-NEXT:    vbic q1, q1, q2
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vorr q0, q0, q1
 ; CHECK-NEXT:    bx lr
 entry:
   %c1 = icmp eq <2 x i64> %a, zeroinitializer
   %c2 = icmp eq <2 x i64> %b, %c
   %o = xor <2 x i1> %c1, %c2
   %s = select <2 x i1> %o, <2 x i64> %a, <2 x i64> %b
   ret <2 x i64> %s
 }
 
 

llvm/test/CodeGen/Thumb2/mve-satmul-loops.ll

 ; CHECK-NEXT:    cmp r3, #0
 ; CHECK-NEXT:    beq.w .LBB0_8
 ; CHECK-NEXT:  @ %bb.1: @ %entry
 ; CHECK-NEXT:    cmp r3, #1
 ; CHECK-NEXT:    bne .LBB0_3
 ; CHECK-NEXT:  @ %bb.2:
 ; CHECK-NEXT:    movs r7, #0
 ; CHECK-NEXT:    mov r12, r0
-; CHECK-NEXT:    mov r6, r1
+; CHECK-NEXT:    mov r10, r1
 ; CHECK-NEXT:    mov r11, r2
 ; CHECK-NEXT:    b .LBB0_6
 ; CHECK-NEXT:  .LBB0_3: @ %vector.ph
 ; CHECK-NEXT:    str r3, [sp, #4] @ 4-byte Spill
 ; CHECK-NEXT:    bic r3, r3, #1
 ; CHECK-NEXT:    subs r7, r3, #2
 ; CHECK-NEXT:    movs r6, #1
 ; CHECK-NEXT:    adr r4, .LCPI0_0
 ; CHECK-NEXT:    str r3, [sp] @ 4-byte Spill
 ; CHECK-NEXT:    add.w lr, r6, r7, lsr #1
 ; CHECK-NEXT:    add.w r11, r2, r3, lsl #2
-; CHECK-NEXT:    add.w r6, r1, r3, lsl #2
+; CHECK-NEXT:    add.w r10, r1, r3, lsl #2
 ; CHECK-NEXT:    add.w r12, r0, r3, lsl #2
 ; CHECK-NEXT:    dls lr, lr
 ; CHECK-NEXT:    vldrw.u32 q0, [r4]
-; CHECK-NEXT:    mvn r10, #-2147483648
 ; CHECK-NEXT:    vmvn.i32 q1, #0x80000000
 ; CHECK-NEXT:  .LBB0_4: @ %vector.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldrd r4, r8, [r0]
+; CHECK-NEXT:    ldrd r4, r5, [r0]
+; CHECK-NEXT:    mov.w r3, #-1
+; CHECK-NEXT:    ldrd r7, r8, [r1]
 ; CHECK-NEXT:    adds r0, #8
-; CHECK-NEXT:    ldrd r7, r5, [r1]
+; CHECK-NEXT:    smull r6, r5, r8, r5
 ; CHECK-NEXT:    adds r1, #8
-; CHECK-NEXT:    smull r8, r5, r5, r8
 ; CHECK-NEXT:    smull r4, r7, r7, r4
-; CHECK-NEXT:    asrl r8, r5, #31
+; CHECK-NEXT:    asrl r6, r5, #31
 ; CHECK-NEXT:    asrl r4, r7, #31
 ; CHECK-NEXT:    rsbs.w r9, r4, #-2147483648
-; CHECK-NEXT:    vmov.32 q4[0], r4
-; CHECK-NEXT:    mov.w r9, #-1
-; CHECK-NEXT:    sbcs.w r3, r9, r7
+; CHECK-NEXT:    vmov q4[2], q4[0], r6, r4
+; CHECK-NEXT:    sbcs r3, r7
+; CHECK-NEXT:    vmov q4[3], q4[1], r5, r7
 ; CHECK-NEXT:    mov.w r3, #0
-; CHECK-NEXT:    vmov.32 q4[1], r7
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r3, #1
 ; CHECK-NEXT:    cmp r3, #0
-; CHECK-NEXT:    csetm r3, ne
-; CHECK-NEXT:    vmov.32 q4[2], r8
-; CHECK-NEXT:    vmov.32 q2[0], r3
-; CHECK-NEXT:    vmov.32 q4[3], r5
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    rsbs.w r3, r8, #-2147483648
-; CHECK-NEXT:    sbcs.w r3, r9, r5
+; CHECK-NEXT:    csetm r9, ne
+; CHECK-NEXT:    rsbs.w r3, r6, #-2147483648
+; CHECK-NEXT:    mov.w r3, #-1
+; CHECK-NEXT:    mvn r6, #-2147483648
+; CHECK-NEXT:    sbcs r3, r5
 ; CHECK-NEXT:    mov.w r3, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r3, #1
 ; CHECK-NEXT:    cmp r3, #0
 ; CHECK-NEXT:    csetm r3, ne
-; CHECK-NEXT:    vmov.32 q2[2], r3
-; CHECK-NEXT:    vmov.32 q2[3], r3
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r9
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r9
 ; CHECK-NEXT:    vbic q3, q0, q2
 ; CHECK-NEXT:    vand q2, q4, q2
 ; CHECK-NEXT:    vorr q2, q2, q3
 ; CHECK-NEXT:    vmov r4, s8
 ; CHECK-NEXT:    vmov r3, s9
-; CHECK-NEXT:    subs.w r4, r4, r10
+; CHECK-NEXT:    vmov r5, s10
+; CHECK-NEXT:    subs r4, r4, r6
+; CHECK-NEXT:    vmov r4, s11
 ; CHECK-NEXT:    sbcs r3, r3, #0
-; CHECK-NEXT:    vmov r4, s10
 ; CHECK-NEXT:    mov.w r3, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r3, #1
 ; CHECK-NEXT:    cmp r3, #0
 ; CHECK-NEXT:    csetm r3, ne
-; CHECK-NEXT:    vmov.32 q3[0], r3
+; CHECK-NEXT:    subs r5, r5, r6
 ; CHECK-NEXT:    vmov.32 q3[1], r3
-; CHECK-NEXT:    vmov r3, s11
-; CHECK-NEXT:    subs.w r4, r4, r10
-; CHECK-NEXT:    sbcs r3, r3, #0
-; CHECK-NEXT:    mov.w r3, #0
+; CHECK-NEXT:    sbcs r4, r4, #0
+; CHECK-NEXT:    mov.w r4, #0
 ; CHECK-NEXT:    it lt
-; CHECK-NEXT:    movlt r3, #1
-; CHECK-NEXT:    cmp r3, #0
-; CHECK-NEXT:    csetm r3, ne
-; CHECK-NEXT:    vmov.32 q3[2], r3
+; CHECK-NEXT:    movlt r4, #1
+; CHECK-NEXT:    cmp r4, #0
+; CHECK-NEXT:    csetm r4, ne
+; CHECK-NEXT:    vmov q3[2], q3[0], r4, r3
 ; CHECK-NEXT:    vbic q4, q1, q3
 ; CHECK-NEXT:    vand q2, q2, q3
 ; CHECK-NEXT:    vorr q2, q2, q4
 ; CHECK-NEXT:    vmov r3, s10
 ; CHECK-NEXT:    vmov r4, s8
 ; CHECK-NEXT:    strd r4, r3, [r2]
 ; CHECK-NEXT:    adds r2, #8
 ; CHECK-NEXT:    le lr, .LBB0_4
 ; CHECK-NEXT:  @ %bb.5: @ %middle.block
 ; CHECK-NEXT:    ldrd r7, r3, [sp] @ 8-byte Folded Reload
 ; CHECK-NEXT:    cmp r7, r3
 ; CHECK-NEXT:    beq .LBB0_8
 ; CHECK-NEXT:  .LBB0_6: @ %for.body.preheader
 ; CHECK-NEXT:    sub.w lr, r3, r7
 ; CHECK-NEXT:    mov.w r0, #-1
 ; CHECK-NEXT:    dls lr, lr
 ; CHECK-NEXT:    mov.w r1, #-2147483648
 ; CHECK-NEXT:    mvn r2, #-2147483648
 ; CHECK-NEXT:  .LBB0_7: @ %for.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldr r3, [r12], #4
-; CHECK-NEXT:    ldr r4, [r6], #4
+; CHECK-NEXT:    ldr r4, [r10], #4
 ; CHECK-NEXT:    smull r4, r3, r4, r3
 ; CHECK-NEXT:    asrl r4, r3, #31
 ; CHECK-NEXT:    subs r5, r1, r4
 ; CHECK-NEXT:    sbcs.w r5, r0, r3
 ; CHECK-NEXT:    mov.w r5, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r5, #1
 ; CHECK-NEXT:    cmp r5, #0

 ; CHECK-LABEL: ssatmul_4_q31:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    .pad #4
 ; CHECK-NEXT:    sub sp, #4
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13}
-; CHECK-NEXT:    .pad #8
-; CHECK-NEXT:    sub sp, #8
+; CHECK-NEXT:    .pad #16
+; CHECK-NEXT:    sub sp, #16
 ; CHECK-NEXT:    cmp r3, #0
 ; CHECK-NEXT:    beq.w .LBB1_8
 ; CHECK-NEXT:  @ %bb.1: @ %for.body.preheader
-; CHECK-NEXT:    mov r9, r1
 ; CHECK-NEXT:    movs r7, #0
 ; CHECK-NEXT:    cmp r3, #3
 ; CHECK-NEXT:    bhi .LBB1_3
 ; CHECK-NEXT:  @ %bb.2:
 ; CHECK-NEXT:    mov r12, r0
-; CHECK-NEXT:    mov r1, r9
+; CHECK-NEXT:    mov r9, r1
 ; CHECK-NEXT:    mov r11, r2
 ; CHECK-NEXT:    b .LBB1_6
 ; CHECK-NEXT:  .LBB1_3: @ %vector.ph
-; CHECK-NEXT:    bic r7, r3, #3
-; CHECK-NEXT:    adr r4, .LCPI1_0
-; CHECK-NEXT:    subs r1, r7, #4
 ; CHECK-NEXT:    str r3, [sp, #4] @ 4-byte Spill
-; CHECK-NEXT:    movs r3, #1
+; CHECK-NEXT:    bic r3, r3, #3
+; CHECK-NEXT:    subs r7, r3, #4
+; CHECK-NEXT:    adr r4, .LCPI1_0
+; CHECK-NEXT:    movs r6, #1
 ; CHECK-NEXT:    vldrw.u32 q0, [r4]
-; CHECK-NEXT:    add.w lr, r3, r1, lsr #2
-; CHECK-NEXT:    str r7, [sp] @ 4-byte Spill
 ; CHECK-NEXT:    adr r4, .LCPI1_1
-; CHECK-NEXT:    add.w r11, r2, r7, lsl #2
-; CHECK-NEXT:    add.w r1, r9, r7, lsl #2
-; CHECK-NEXT:    add.w r12, r0, r7, lsl #2
+; CHECK-NEXT:    add.w lr, r6, r7, lsr #2
+; CHECK-NEXT:    str r3, [sp] @ 4-byte Spill
+; CHECK-NEXT:    add.w r11, r2, r3, lsl #2
+; CHECK-NEXT:    add.w r9, r1, r3, lsl #2
+; CHECK-NEXT:    add.w r12, r0, r3, lsl #2
 ; CHECK-NEXT:    dls lr, lr
 ; CHECK-NEXT:    vldrw.u32 q1, [r4]
-; CHECK-NEXT:    mov.w r10, #-1
 ; CHECK-NEXT:  .LBB1_4: @ %vector.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
+; CHECK-NEXT:    vldrw.u32 q3, [r1], #16
 ; CHECK-NEXT:    vldrw.u32 q2, [r0], #16
-; CHECK-NEXT:    vldrw.u32 q3, [r9], #16
+; CHECK-NEXT:    str r2, [sp, #12] @ 4-byte Spill
+; CHECK-NEXT:    mov.w r2, #-1
 ; CHECK-NEXT:    vmov.f32 s16, s10
+; CHECK-NEXT:    str r1, [sp, #8] @ 4-byte Spill
 ; CHECK-NEXT:    vmov.f32 s20, s14
 ; CHECK-NEXT:    vmov.f32 s18, s11
 ; CHECK-NEXT:    vmov.f32 s22, s15
 ; CHECK-NEXT:    vmullb.s32 q6, q5, q4
-; CHECK-NEXT:    vmov.f32 s10, s9
+; CHECK-NEXT:    vmov.f32 s14, s13
 ; CHECK-NEXT:    vmov r7, s25
 ; CHECK-NEXT:    vmov r4, s24
 ; CHECK-NEXT:    asrl r4, r7, #31
-; CHECK-NEXT:    vmov r8, s26
+; CHECK-NEXT:    vmov r10, s26
 ; CHECK-NEXT:    rsbs.w r5, r4, #-2147483648
-; CHECK-NEXT:    vmov.f32 s14, s13
-; CHECK-NEXT:    sbcs.w r5, r10, r7
+; CHECK-NEXT:    vmov.f32 s10, s9
+; CHECK-NEXT:    sbcs.w r5, r2, r7
 ; CHECK-NEXT:    mov.w r5, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r5, #1
 ; CHECK-NEXT:    cmp r5, #0
-; CHECK-NEXT:    csetm r5, ne
-; CHECK-NEXT:    vmov.32 q4[0], r5
-; CHECK-NEXT:    vmov.32 q4[1], r5
 ; CHECK-NEXT:    vmov r5, s27
-; CHECK-NEXT:    asrl r8, r5, #31
-; CHECK-NEXT:    vmov.32 q6[0], r4
-; CHECK-NEXT:    rsbs.w r6, r8, #-2147483648
-; CHECK-NEXT:    vmov.32 q6[1], r7
-; CHECK-NEXT:    sbcs.w r6, r10, r5
-; CHECK-NEXT:    vmov.32 q6[2], r8
-; CHECK-NEXT:    mov.w r6, #0
-; CHECK-NEXT:    vmov.32 q6[3], r5
-; CHECK-NEXT:    it lt
-; CHECK-NEXT:    movlt r6, #1
-; CHECK-NEXT:    cmp r6, #0
-; CHECK-NEXT:    csetm r6, ne
-; CHECK-NEXT:    mvn r8, #-2147483648
-; CHECK-NEXT:    vmov.32 q4[2], r6
-; CHECK-NEXT:    vmov.32 q4[3], r6
+; CHECK-NEXT:    csetm r8, ne
+; CHECK-NEXT:    asrl r10, r5, #31
 ; CHECK-NEXT:    vmov r6, s14
+; CHECK-NEXT:    rsbs.w r3, r10, #-2147483648
+; CHECK-NEXT:    vmov q6[2], q6[0], r10, r4
+; CHECK-NEXT:    sbcs.w r3, r2, r5
+; CHECK-NEXT:    vmov q6[3], q6[1], r5, r7
+; CHECK-NEXT:    mov.w r3, #0
+; CHECK-NEXT:    it lt
+; CHECK-NEXT:    movlt r3, #1
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    csetm r3, ne
+; CHECK-NEXT:    vmov q4[2], q4[0], r3, r8
+; CHECK-NEXT:    vmov q4[3], q4[1], r3, r8
+; CHECK-NEXT:    mvn r8, #-2147483648
 ; CHECK-NEXT:    vbic q5, q0, q4
 ; CHECK-NEXT:    vand q4, q6, q4
 ; CHECK-NEXT:    vorr q4, q4, q5
-; CHECK-NEXT:    vmov r5, s16
-; CHECK-NEXT:    vmov r4, s17
-; CHECK-NEXT:    subs.w r5, r5, r8
-; CHECK-NEXT:    sbcs r4, r4, #0
+; CHECK-NEXT:    vmov r4, s16
+; CHECK-NEXT:    vmov r3, s17
 ; CHECK-NEXT:    vmov r5, s18
-; CHECK-NEXT:    mov.w r4, #0
-; CHECK-NEXT:    it lt
-; CHECK-NEXT:    movlt r4, #1
-; CHECK-NEXT:    cmp r4, #0
-; CHECK-NEXT:    csetm r4, ne
-; CHECK-NEXT:    vmov.32 q5[0], r4
-; CHECK-NEXT:    vmov.32 q5[1], r4
+; CHECK-NEXT:    subs.w r4, r4, r8
 ; CHECK-NEXT:    vmov r4, s19
+; CHECK-NEXT:    sbcs r3, r3, #0
+; CHECK-NEXT:    mov.w r3, #0
+; CHECK-NEXT:    it lt
+; CHECK-NEXT:    movlt r3, #1
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    csetm r3, ne
 ; CHECK-NEXT:    subs.w r5, r5, r8
-; CHECK-NEXT:    vmov r5, s12
+; CHECK-NEXT:    vmov.32 q5[1], r3
+; CHECK-NEXT:    vmov r5, s10
 ; CHECK-NEXT:    sbcs r4, r4, #0
 ; CHECK-NEXT:    mov.w r4, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r4, #1
 ; CHECK-NEXT:    cmp r4, #0
 ; CHECK-NEXT:    csetm r4, ne
-; CHECK-NEXT:    vmov.32 q5[2], r4
-; CHECK-NEXT:    vmov r4, s8
+; CHECK-NEXT:    vmov q5[2], q5[0], r4, r3
+; CHECK-NEXT:    vmov r3, s8
+; CHECK-NEXT:    vmov r4, s12
+; CHECK-NEXT:    smull r6, r5, r6, r5
 ; CHECK-NEXT:    vbic q6, q1, q5
 ; CHECK-NEXT:    vand q4, q4, q5
 ; CHECK-NEXT:    vorr q4, q4, q6
-; CHECK-NEXT:    smull r4, r7, r5, r4
-; CHECK-NEXT:    asrl r4, r7, #31
-; CHECK-NEXT:    rsbs.w r5, r4, #-2147483648
-; CHECK-NEXT:    vmov.32 q3[0], r4
-; CHECK-NEXT:    sbcs.w r5, r10, r7
-; CHECK-NEXT:    vmov.32 q3[1], r7
-; CHECK-NEXT:    mov.w r5, #0
-; CHECK-NEXT:    it lt
-; CHECK-NEXT:    movlt r5, #1
-; CHECK-NEXT:    cmp r5, #0
-; CHECK-NEXT:    csetm r5, ne
-; CHECK-NEXT:    vmov.32 q5[0], r5
-; CHECK-NEXT:    vmov.32 q5[1], r5
-; CHECK-NEXT:    vmov r5, s10
-; CHECK-NEXT:    smull r6, r5, r6, r5
 ; CHECK-NEXT:    asrl r6, r5, #31
-; CHECK-NEXT:    rsbs.w r3, r6, #-2147483648
-; CHECK-NEXT:    vmov.32 q3[2], r6
-; CHECK-NEXT:    sbcs.w r3, r10, r5
-; CHECK-NEXT:    vmov.32 q3[3], r5
+; CHECK-NEXT:    smull r4, r7, r4, r3
+; CHECK-NEXT:    asrl r4, r7, #31
+; CHECK-NEXT:    rsbs.w r3, r4, #-2147483648
+; CHECK-NEXT:    vmov q5[2], q5[0], r6, r4
+; CHECK-NEXT:    sbcs.w r3, r2, r7
+; CHECK-NEXT:    vmov q5[3], q5[1], r5, r7
 ; CHECK-NEXT:    mov.w r3, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r3, #1
 ; CHECK-NEXT:    cmp r3, #0
 ; CHECK-NEXT:    csetm r3, ne
-; CHECK-NEXT:    vmov.32 q5[2], r3
-; CHECK-NEXT:    vmov.32 q5[3], r3
-; CHECK-NEXT:    vbic q2, q0, q5
-; CHECK-NEXT:    vand q3, q3, q5
-; CHECK-NEXT:    vorr q2, q3, q2
-; CHECK-NEXT:    vmov r4, s8
-; CHECK-NEXT:    vmov r3, s9
-; CHECK-NEXT:    subs.w r4, r4, r8
-; CHECK-NEXT:    sbcs r3, r3, #0
+; CHECK-NEXT:    rsbs.w r1, r6, #-2147483648
+; CHECK-NEXT:    sbcs.w r1, r2, r5
+; CHECK-NEXT:    mov.w r1, #0
+; CHECK-NEXT:    it lt
+; CHECK-NEXT:    movlt r1, #1
+; CHECK-NEXT:    cmp r1, #0
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r3
+; CHECK-NEXT:    ldrd r1, r2, [sp, #8] @ 8-byte Folded Reload
+; CHECK-NEXT:    vbic q3, q0, q2
+; CHECK-NEXT:    vand q2, q5, q2
+; CHECK-NEXT:    vorr q2, q2, q3
+; CHECK-NEXT:    vmov r3, s8
+; CHECK-NEXT:    vmov r4, s9
+; CHECK-NEXT:    subs.w r3, r3, r8
+; CHECK-NEXT:    sbcs r3, r4, #0
 ; CHECK-NEXT:    vmov r4, s10
 ; CHECK-NEXT:    mov.w r3, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r3, #1
 ; CHECK-NEXT:    cmp r3, #0
-; CHECK-NEXT:    csetm r3, ne
-; CHECK-NEXT:    vmov.32 q3[0], r3
-; CHECK-NEXT:    vmov.32 q3[1], r3
 ; CHECK-NEXT:    vmov r3, s11
+; CHECK-NEXT:    csetm r5, ne
+; CHECK-NEXT:    vmov.32 q3[1], r5
 ; CHECK-NEXT:    subs.w r4, r4, r8
 ; CHECK-NEXT:    sbcs r3, r3, #0
 ; CHECK-NEXT:    mov.w r3, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r3, #1
 ; CHECK-NEXT:    cmp r3, #0
 ; CHECK-NEXT:    csetm r3, ne
-; CHECK-NEXT:    vmov.32 q3[2], r3
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r5
 ; CHECK-NEXT:    vbic q5, q1, q3
 ; CHECK-NEXT:    vand q2, q2, q3
 ; CHECK-NEXT:    vorr q2, q2, q5
 ; CHECK-NEXT:    vmov.f32 s9, s10
 ; CHECK-NEXT:    vmov.f32 s10, s16
 ; CHECK-NEXT:    vmov.f32 s11, s18
 ; CHECK-NEXT:    vstrb.8 q2, [r2], #16
 ; CHECK-NEXT:    le lr, .LBB1_4
 ; CHECK-NEXT:  @ %bb.5: @ %middle.block
 ; CHECK-NEXT:    ldrd r7, r3, [sp] @ 8-byte Folded Reload
 ; CHECK-NEXT:    cmp r7, r3
 ; CHECK-NEXT:    beq .LBB1_8
 ; CHECK-NEXT:  .LBB1_6: @ %for.body.preheader21
 ; CHECK-NEXT:    sub.w lr, r3, r7
 ; CHECK-NEXT:    mov.w r0, #-1
 ; CHECK-NEXT:    dls lr, lr
 ; CHECK-NEXT:    mov.w r3, #-2147483648
 ; CHECK-NEXT:    mvn r2, #-2147483648
 ; CHECK-NEXT:  .LBB1_7: @ %for.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldr r4, [r12], #4
-; CHECK-NEXT:    ldr r5, [r1], #4
-; CHECK-NEXT:    smull r4, r5, r5, r4
-; CHECK-NEXT:    asrl r4, r5, #31
-; CHECK-NEXT:    subs r6, r3, r4
-; CHECK-NEXT:    sbcs.w r6, r0, r5
-; CHECK-NEXT:    mov.w r6, #0
+; CHECK-NEXT:    ldr r1, [r12], #4
+; CHECK-NEXT:    ldr r4, [r9], #4
+; CHECK-NEXT:    smull r4, r1, r4, r1
+; CHECK-NEXT:    asrl r4, r1, #31
+; CHECK-NEXT:    subs r5, r3, r4
+; CHECK-NEXT:    sbcs.w r5, r0, r1
+; CHECK-NEXT:    mov.w r5, #0
 ; CHECK-NEXT:    it lt
-; CHECK-NEXT:    movlt r6, #1
-; CHECK-NEXT:    cmp r6, #0
+; CHECK-NEXT:    movlt r5, #1
+; CHECK-NEXT:    cmp r5, #0
 ; CHECK-NEXT:    csel r4, r4, r3, ne
-; CHECK-NEXT:    csel r5, r5, r0, ne
-; CHECK-NEXT:    subs r6, r4, r2
-; CHECK-NEXT:    sbcs r5, r5, #0
-; CHECK-NEXT:    csel r4, r4, r2, lt
-; CHECK-NEXT:    str r4, [r11], #4
+; CHECK-NEXT:    csel r1, r1, r0, ne
+; CHECK-NEXT:    subs r5, r4, r2
+; CHECK-NEXT:    sbcs r1, r1, #0
+; CHECK-NEXT:    csel r1, r4, r2, lt
+; CHECK-NEXT:    str r1, [r11], #4
 ; CHECK-NEXT:    le lr, .LBB1_7
 ; CHECK-NEXT:  .LBB1_8: @ %for.cond.cleanup
-; CHECK-NEXT:    add sp, #8
+; CHECK-NEXT:    add sp, #16
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13}
 ; CHECK-NEXT:    add sp, #4
 ; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.9:
 ; CHECK-NEXT:  .LCPI1_0:
 ; CHECK-NEXT:    .long 2147483648 @ 0x80000000
 ; CHECK-NEXT:    .long 4294967295 @ 0xffffffff
for.body:                                         ; preds = %for.body.preheader21, %for.body
   %inc = add nuw i32 %i.012, 1
   %exitcond = icmp eq i32 %inc, %N
   br i1 %exitcond, label %for.cond.cleanup, label %for.body
 }
 
 define arm_aapcs_vfpcc void @ssatmul_4t_q31(i32* nocapture readonly %pSrcA, i32* nocapture readonly %pSrcB, i32* noalias nocapture %pDst, i32 %N) {
 ; CHECK-LABEL: ssatmul_4t_q31:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, lr}
-; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, lr}
-; CHECK-NEXT:    .pad #4
-; CHECK-NEXT:    sub sp, #4
+; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, lr}
+; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, lr}
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    .pad #16
 ; CHECK-NEXT:    sub sp, #16
 ; CHECK-NEXT:    cmp r3, #0
 ; CHECK-NEXT:    beq.w .LBB2_3
 ; CHECK-NEXT:  @ %bb.1: @ %vector.ph
 ; CHECK-NEXT:    adds r7, r3, #3
-; CHECK-NEXT:    adr r4, .LCPI2_1
-; CHECK-NEXT:    bic r7, r7, #3
 ; CHECK-NEXT:    movs r6, #1
+; CHECK-NEXT:    bic r7, r7, #3
+; CHECK-NEXT:    adr r4, .LCPI2_1
 ; CHECK-NEXT:    subs r7, #4
+; CHECK-NEXT:    adr r5, .LCPI2_2
 ; CHECK-NEXT:    vldrw.u32 q2, [r4]
-; CHECK-NEXT:    adr r4, .LCPI2_2
-; CHECK-NEXT:    mov.w r9, #0
+; CHECK-NEXT:    vldrw.u32 q3, [r5]
 ; CHECK-NEXT:    add.w lr, r6, r7, lsr #2
 ; CHECK-NEXT:    adr r6, .LCPI2_0
 ; CHECK-NEXT:    subs r7, r3, #1
 ; CHECK-NEXT:    dls lr, lr
 ; CHECK-NEXT:    vldrw.u32 q0, [r6]
-; CHECK-NEXT:    vldrw.u32 q3, [r4]
+; CHECK-NEXT:    mov.w r9, #0
 ; CHECK-NEXT:    vdup.32 q1, r7
 ; CHECK-NEXT:    mov.w r12, #-1
 ; CHECK-NEXT:    mvn r8, #-2147483648
 ; CHECK-NEXT:    vstrw.32 q0, [sp] @ 16-byte Spill
 ; CHECK-NEXT:  .LBB2_2: @ %vector.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vldrw.u32 q0, [sp] @ 16-byte Reload
 ; CHECK-NEXT:    vdup.32 q4, r9
 ; CHECK-NEXT:    add.w r9, r9, #4
 ; CHECK-NEXT:    vorr q4, q4, q0
 ; CHECK-NEXT:    vpt.u32 cs, q1, q4
 ; CHECK-NEXT:    vldrwt.u32 q4, [r0], #16
 ; CHECK-NEXT:    vmov.f32 s24, s18
 ; CHECK-NEXT:    vpst
 ; CHECK-NEXT:    vldrwt.u32 q5, [r1], #16
 ; CHECK-NEXT:    vmov.f32 s28, s22
 ; CHECK-NEXT:    vmov.f32 s26, s19
 ; CHECK-NEXT:    vmov.f32 s30, s23
 ; CHECK-NEXT:    vmullb.s32 q0, q7, q6
 ; CHECK-NEXT:    vmov.f32 s18, s17
 ; CHECK-NEXT:    vmov r5, s1
 ; CHECK-NEXT:    vmov r6, s0
 ; CHECK-NEXT:    asrl r6, r5, #31
-; CHECK-NEXT:    vmov r7, s3
-; CHECK-NEXT:    rsbs.w r4, r6, #-2147483648
-; CHECK-NEXT:    vmov.32 q7[0], r6
-; CHECK-NEXT:    sbcs.w r4, r12, r5
-; CHECK-NEXT:    vmov.32 q7[1], r5
-; CHECK-NEXT:    mov.w r4, #0
+; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    rsbs.w r7, r6, #-2147483648
 ; CHECK-NEXT:    vmov.f32 s22, s21
+; CHECK-NEXT:    sbcs.w r7, r12, r5
+; CHECK-NEXT:    mov.w r7, #0
 ; CHECK-NEXT:    it lt
-; CHECK-NEXT:    movlt r4, #1
-; CHECK-NEXT:    cmp r4, #0
-; CHECK-NEXT:    csetm r4, ne
-; CHECK-NEXT:    vmov.32 q6[0], r4
-; CHECK-NEXT:    vmov.32 q6[1], r4
-; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    movlt r7, #1
+; CHECK-NEXT:    cmp r7, #0
+; CHECK-NEXT:    vmov r7, s3
+; CHECK-NEXT:    csetm r10, ne
 ; CHECK-NEXT:    asrl r4, r7, #31
 ; CHECK-NEXT:    rsbs.w r3, r4, #-2147483648
-; CHECK-NEXT:    vmov.32 q7[2], r4
+; CHECK-NEXT:    vmov q7[2], q7[0], r4, r6
 ; CHECK-NEXT:    sbcs.w r3, r12, r7
-; CHECK-NEXT:    vmov.32 q7[3], r7
+; CHECK-NEXT:    vmov q7[3], q7[1], r7, r5
 ; CHECK-NEXT:    mov.w r3, #0
+; CHECK-NEXT:    vmov r7, s22
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r3, #1
 ; CHECK-NEXT:    cmp r3, #0
 ; CHECK-NEXT:    csetm r3, ne
-; CHECK-NEXT:    vmov.32 q6[2], r3
-; CHECK-NEXT:    vmov.32 q6[3], r3
-; CHECK-NEXT:    vbic q0, q2, q6
-; CHECK-NEXT:    vand q6, q7, q6
-; CHECK-NEXT:    vorr q6, q6, q0
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r10
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r10
+; CHECK-NEXT:    vbic q6, q2, q0
+; CHECK-NEXT:    vand q0, q7, q0
+; CHECK-NEXT:    vorr q6, q0, q6
 ; CHECK-NEXT:    vmov r4, s24
 ; CHECK-NEXT:    vmov r3, s25
+; CHECK-NEXT:    vmov r5, s26
 ; CHECK-NEXT:    subs.w r4, r4, r8
+; CHECK-NEXT:    vmov r4, s27
 ; CHECK-NEXT:    sbcs r3, r3, #0
-; CHECK-NEXT:    vmov r4, s26
 ; CHECK-NEXT:    mov.w r3, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r3, #1
 ; CHECK-NEXT:    cmp r3, #0
 ; CHECK-NEXT:    csetm r3, ne
-; CHECK-NEXT:    vmov.32 q0[0], r3
+; CHECK-NEXT:    subs.w r5, r5, r8
 ; CHECK-NEXT:    vmov.32 q0[1], r3
-; CHECK-NEXT:    vmov r3, s27
-; CHECK-NEXT:    subs.w r4, r4, r8
-; CHECK-NEXT:    vmov r4, s20
-; CHECK-NEXT:    sbcs r3, r3, #0
-; CHECK-NEXT:    mov.w r3, #0
+; CHECK-NEXT:    sbcs r4, r4, #0
+; CHECK-NEXT:    mov.w r4, #0
 ; CHECK-NEXT:    it lt
-; CHECK-NEXT:    movlt r3, #1
-; CHECK-NEXT:    cmp r3, #0
-; CHECK-NEXT:    csetm r3, ne
-; CHECK-NEXT:    vmov.32 q0[2], r3
+; CHECK-NEXT:    movlt r4, #1
+; CHECK-NEXT:    cmp r4, #0
+; CHECK-NEXT:    csetm r4, ne
+; CHECK-NEXT:    vmov q0[2], q0[0], r4, r3
 ; CHECK-NEXT:    vmov r3, s16
+; CHECK-NEXT:    vmov r4, s20
 ; CHECK-NEXT:    vbic q7, q3, q0
 ; CHECK-NEXT:    vand q0, q6, q0
 ; CHECK-NEXT:    vorr q6, q0, q7
 ; CHECK-NEXT:    smull r6, r5, r4, r3
-; CHECK-NEXT:    vmov r4, s22
+; CHECK-NEXT:    vmov r4, s18
 ; CHECK-NEXT:    asrl r6, r5, #31
 ; CHECK-NEXT:    rsbs.w r3, r6, #-2147483648
 ; CHECK-NEXT:    sbcs.w r3, r12, r5
 ; CHECK-NEXT:    mov.w r3, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r3, #1
 ; CHECK-NEXT:    cmp r3, #0
-; CHECK-NEXT:    csetm r3, ne
-; CHECK-NEXT:    vmov.32 q7[0], r3
-; CHECK-NEXT:    vmov.32 q7[1], r3
-; CHECK-NEXT:    vmov r3, s18
-; CHECK-NEXT:    vmov.32 q4[0], r6
-; CHECK-NEXT:    vmov.32 q4[1], r5
-; CHECK-NEXT:    smull r4, r7, r4, r3
+; CHECK-NEXT:    csetm r10, ne
+; CHECK-NEXT:    smull r4, r7, r7, r4
 ; CHECK-NEXT:    asrl r4, r7, #31
 ; CHECK-NEXT:    rsbs.w r3, r4, #-2147483648
-; CHECK-NEXT:    vmov.32 q4[2], r4
+; CHECK-NEXT:    vmov q5[2], q5[0], r4, r6
 ; CHECK-NEXT:    sbcs.w r3, r12, r7
-; CHECK-NEXT:    vmov.32 q4[3], r7
+; CHECK-NEXT:    vmov q5[3], q5[1], r7, r5
 ; CHECK-NEXT:    mov.w r3, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r3, #1
 ; CHECK-NEXT:    cmp r3, #0
 ; CHECK-NEXT:    csetm r3, ne
-; CHECK-NEXT:    vmov.32 q7[2], r3
-; CHECK-NEXT:    vmov.32 q7[3], r3
-; CHECK-NEXT:    vbic q0, q2, q7
-; CHECK-NEXT:    vand q4, q4, q7
-; CHECK-NEXT:    vorr q4, q4, q0
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r10
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r10
+; CHECK-NEXT:    vbic q4, q2, q0
+; CHECK-NEXT:    vand q0, q5, q0
+; CHECK-NEXT:    vorr q4, q0, q4
 ; CHECK-NEXT:    vmov r4, s16
 ; CHECK-NEXT:    vmov r3, s17
+; CHECK-NEXT:    vmov r5, s18
 ; CHECK-NEXT:    subs.w r4, r4, r8
+; CHECK-NEXT:    vmov r4, s19
 ; CHECK-NEXT:    sbcs r3, r3, #0
-; CHECK-NEXT:    vmov r4, s18
 ; CHECK-NEXT:    mov.w r3, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r3, #1
 ; CHECK-NEXT:    cmp r3, #0
 ; CHECK-NEXT:    csetm r3, ne
-; CHECK-NEXT:    vmov.32 q0[0], r3
+; CHECK-NEXT:    subs.w r5, r5, r8
 ; CHECK-NEXT:    vmov.32 q0[1], r3
-; CHECK-NEXT:    vmov r3, s19
-; CHECK-NEXT:    subs.w r4, r4, r8
-; CHECK-NEXT:    sbcs r3, r3, #0
-; CHECK-NEXT:    mov.w r3, #0
+; CHECK-NEXT:    sbcs r4, r4, #0
+; CHECK-NEXT:    mov.w r4, #0
 ; CHECK-NEXT:    it lt
-; CHECK-NEXT:    movlt r3, #1
-; CHECK-NEXT:    cmp r3, #0
-; CHECK-NEXT:    csetm r3, ne
-; CHECK-NEXT:    vmov.32 q0[2], r3
+; CHECK-NEXT:    movlt r4, #1
+; CHECK-NEXT:    cmp r4, #0
+; CHECK-NEXT:    csetm r4, ne
+; CHECK-NEXT:    vmov q0[2], q0[0], r4, r3
 ; CHECK-NEXT:    vbic q5, q3, q0
 ; CHECK-NEXT:    vand q0, q4, q0
 ; CHECK-NEXT:    vorr q0, q0, q5
 ; CHECK-NEXT:    vmov.f32 s1, s2
 ; CHECK-NEXT:    vmov.f32 s2, s24
 ; CHECK-NEXT:    vmov.f32 s3, s26
 ; CHECK-NEXT:    vpst
 ; CHECK-NEXT:    vstrwt.32 q0, [r2], #16
 ; CHECK-NEXT:    le lr, .LBB2_2
 ; CHECK-NEXT:  .LBB2_3: @ %for.cond.cleanup
 ; CHECK-NEXT:    add sp, #16
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    add sp, #4
-; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, pc}
+; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, pc}
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.4:
 ; CHECK-NEXT:  .LCPI2_0:
 ; CHECK-NEXT:    .long 0 @ 0x0
 ; CHECK-NEXT:    .long 1 @ 0x1
 ; CHECK-NEXT:    .long 2 @ 0x2
 ; CHECK-NEXT:    .long 3 @ 0x3
 ; CHECK-NEXT:  .LCPI2_1:

 ; CHECK-NEXT:    str r5, [sp] @ 4-byte Spill
 ; CHECK-NEXT:    add.w r8, r2, r5, lsl #2
 ; CHECK-NEXT:    add.w r11, r1, r5, lsl #2
 ; CHECK-NEXT:    add.w lr, r6, r7, lsr #1
 ; CHECK-NEXT:    add.w r12, r0, r5, lsl #2
 ; CHECK-NEXT:    dls lr, lr
 ; CHECK-NEXT:  .LBB3_4: @ %vector.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldrd r4, r9, [r0]
+; CHECK-NEXT:    ldrd r4, r7, [r0]
 ; CHECK-NEXT:    adds r0, #8
 ; CHECK-NEXT:    ldrd r5, r10, [r1]
 ; CHECK-NEXT:    adds r1, #8
 ; CHECK-NEXT:    umull r4, r5, r5, r4
 ; CHECK-NEXT:    lsrl r4, r5, #31
 ; CHECK-NEXT:    subs.w r6, r4, #-1
-; CHECK-NEXT:    vmov.32 q1[0], r4
+; CHECK-NEXT:    umull r6, r7, r10, r7
 ; CHECK-NEXT:    sbcs r5, r5, #0
 ; CHECK-NEXT:    mov.w r5, #0
 ; CHECK-NEXT:    it lo
 ; CHECK-NEXT:    movlo r5, #1
 ; CHECK-NEXT:    cmp r5, #0
-; CHECK-NEXT:    csetm r5, ne
-; CHECK-NEXT:    vmov.32 q0[0], r5
-; CHECK-NEXT:    vmov.32 q0[1], r5
-; CHECK-NEXT:    umull r6, r5, r10, r9
-; CHECK-NEXT:    lsrl r6, r5, #31
-; CHECK-NEXT:    subs.w r7, r6, #-1
-; CHECK-NEXT:    vmov.32 q1[2], r6
-; CHECK-NEXT:    sbcs r5, r5, #0
+; CHECK-NEXT:    lsrl r6, r7, #31
+; CHECK-NEXT:    csetm r9, ne
+; CHECK-NEXT:    subs.w r5, r6, #-1
+; CHECK-NEXT:    vmov.32 q0[1], r9
+; CHECK-NEXT:    sbcs r5, r7, #0
+; CHECK-NEXT:    vmov q1[2], q1[0], r6, r4
 ; CHECK-NEXT:    mov.w r5, #0
 ; CHECK-NEXT:    it lo
 ; CHECK-NEXT:    movlo r5, #1
 ; CHECK-NEXT:    cmp r5, #0
 ; CHECK-NEXT:    csetm r5, ne
-; CHECK-NEXT:    vmov.32 q0[2], r5
+; CHECK-NEXT:    vmov q0[2], q0[0], r5, r9
 ; CHECK-NEXT:    vand q1, q1, q0
 ; CHECK-NEXT:    vorn q0, q1, q0
 ; CHECK-NEXT:    vmov r4, s2
 ; CHECK-NEXT:    vmov r5, s0
 ; CHECK-NEXT:    strd r5, r4, [r2]
 ; CHECK-NEXT:    adds r2, #8
 ; CHECK-NEXT:    le lr, .LBB3_4
 ; CHECK-NEXT:  @ %bb.5: @ %middle.block
for.body:                                         ; preds = %for.body.preheader, %for.body
   %inc = add nuw i32 %i.012, 1
   %exitcond = icmp eq i32 %inc, %N
   br i1 %exitcond, label %for.cond.cleanup, label %for.body
 }
 
 define arm_aapcs_vfpcc void @usatmul_4_q31(i32* nocapture readonly %pSrcA, i32* nocapture readonly %pSrcB, i32* noalias nocapture %pDst, i32 %N) {
 ; CHECK-LABEL: usatmul_4_q31:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, lr}
-; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, lr}
+; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, r11, lr}
+; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, r11, lr}
+; CHECK-NEXT:    .pad #4
+; CHECK-NEXT:    sub sp, #4
 ; CHECK-NEXT:    .vsave {d8, d9}
 ; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    cmp r3, #0
 ; CHECK-NEXT:    beq.w .LBB4_8
 ; CHECK-NEXT:  @ %bb.1: @ %for.body.preheader
 ; CHECK-NEXT:    mov.w r8, #0
 ; CHECK-NEXT:    cmp r3, #3
 ; CHECK-NEXT:    bhi .LBB4_3

 ; CHECK-NEXT:    vmov.f32 s12, s10
 ; CHECK-NEXT:    vmov.f32 s6, s3
 ; CHECK-NEXT:    vmov.f32 s14, s11
 ; CHECK-NEXT:    vmullb.u32 q4, q3, q1
 ; CHECK-NEXT:    vmov.f32 s2, s1
 ; CHECK-NEXT:    vmov r5, s17
 ; CHECK-NEXT:    vmov r4, s16
 ; CHECK-NEXT:    lsrl r4, r5, #31
-; CHECK-NEXT:    vmov.f32 s10, s9
+; CHECK-NEXT:    vmov r7, s19
 ; CHECK-NEXT:    subs.w r6, r4, #-1
-; CHECK-NEXT:    vmov.32 q3[0], r4
+; CHECK-NEXT:    vmov.f32 s10, s9
 ; CHECK-NEXT:    sbcs r5, r5, #0
 ; CHECK-NEXT:    vmov r6, s18
 ; CHECK-NEXT:    mov.w r5, #0
+; CHECK-NEXT:    lsrl r6, r7, #31
 ; CHECK-NEXT:    it lo
 ; CHECK-NEXT:    movlo r5, #1
 ; CHECK-NEXT:    cmp r5, #0
-; CHECK-NEXT:    csetm r5, ne
-; CHECK-NEXT:    vmov.32 q1[0], r5
-; CHECK-NEXT:    vmov.32 q1[1], r5
-; CHECK-NEXT:    vmov r5, s19
-; CHECK-NEXT:    lsrl r6, r5, #31
-; CHECK-NEXT:    subs.w r7, r6, #-1
-; CHECK-NEXT:    vmov.32 q3[2], r6
-; CHECK-NEXT:    sbcs r5, r5, #0
+; CHECK-NEXT:    csetm r11, ne
+; CHECK-NEXT:    subs.w r5, r6, #-1
+; CHECK-NEXT:    sbcs r5, r7, #0
+; CHECK-NEXT:    vmov.32 q1[1], r11
 ; CHECK-NEXT:    mov.w r5, #0
+; CHECK-NEXT:    vmov q3[2], q3[0], r6, r4
 ; CHECK-NEXT:    it lo
 ; CHECK-NEXT:    movlo r5, #1
 ; CHECK-NEXT:    cmp r5, #0
 ; CHECK-NEXT:    csetm r5, ne
-; CHECK-NEXT:    vmov.32 q1[2], r5
+; CHECK-NEXT:    vmov q1[2], q1[0], r5, r11
 ; CHECK-NEXT:    vand q3, q3, q1
 ; CHECK-NEXT:    vorn q1, q3, q1
 ; CHECK-NEXT:    vmullb.u32 q3, q2, q0
 ; CHECK-NEXT:    vmov r5, s13
 ; CHECK-NEXT:    vmov r4, s12
 ; CHECK-NEXT:    lsrl r4, r5, #31
+; CHECK-NEXT:    vmov r7, s15
 ; CHECK-NEXT:    subs.w r6, r4, #-1
-; CHECK-NEXT:    vmov.32 q2[0], r4
 ; CHECK-NEXT:    sbcs r5, r5, #0
 ; CHECK-NEXT:    vmov r6, s14
 ; CHECK-NEXT:    mov.w r5, #0
+; CHECK-NEXT:    lsrl r6, r7, #31
 ; CHECK-NEXT:    it lo
 ; CHECK-NEXT:    movlo r5, #1
 ; CHECK-NEXT:    cmp r5, #0
-; CHECK-NEXT:    csetm r5, ne
-; CHECK-NEXT:    vmov.32 q0[0], r5
-; CHECK-NEXT:    vmov.32 q0[1], r5
-; CHECK-NEXT:    vmov r5, s15
-; CHECK-NEXT:    lsrl r6, r5, #31
-; CHECK-NEXT:    subs.w r7, r6, #-1
-; CHECK-NEXT:    vmov.32 q2[2], r6
-; CHECK-NEXT:    sbcs r5, r5, #0
+; CHECK-NEXT:    csetm r11, ne
+; CHECK-NEXT:    subs.w r5, r6, #-1
+; CHECK-NEXT:    sbcs r5, r7, #0
+; CHECK-NEXT:    vmov.32 q0[1], r11
 ; CHECK-NEXT:    mov.w r5, #0
+; CHECK-NEXT:    vmov q2[2], q2[0], r6, r4
 ; CHECK-NEXT:    it lo
 ; CHECK-NEXT:    movlo r5, #1
 ; CHECK-NEXT:    cmp r5, #0
 ; CHECK-NEXT:    csetm r5, ne
-; CHECK-NEXT:    vmov.32 q0[2], r5
+; CHECK-NEXT:    vmov q0[2], q0[0], r5, r11
 ; CHECK-NEXT:    vand q2, q2, q0
 ; CHECK-NEXT:    vorn q0, q2, q0
 ; CHECK-NEXT:    vmov.f32 s1, s2
 ; CHECK-NEXT:    vmov.f32 s2, s4
 ; CHECK-NEXT:    vmov.f32 s3, s6
 ; CHECK-NEXT:    vstrb.8 q0, [r2], #16
 ; CHECK-NEXT:    le lr, .LBB4_4
 ; CHECK-NEXT:  @ %bb.5: @ %middle.block

 ; CHECK-NEXT:    subs.w r2, r0, #-1
 ; CHECK-NEXT:    sbcs r1, r1, #0
 ; CHECK-NEXT:    it hs
 ; CHECK-NEXT:    movhs.w r0, #-1
 ; CHECK-NEXT:    str r0, [r10], #4
 ; CHECK-NEXT:    le lr, .LBB4_7
 ; CHECK-NEXT:  .LBB4_8: @ %for.cond.cleanup
 ; CHECK-NEXT:    vpop {d8, d9}
-; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, pc}
+; CHECK-NEXT:    add sp, #4
+; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 entry:
   %cmp8 = icmp eq i32 %N, 0
   br i1 %cmp8, label %for.cond.cleanup, label %for.body.preheader
 
 for.body.preheader:                               ; preds = %entry
   %min.iters.check = icmp ult i32 %N, 4
   br i1 %min.iters.check, label %for.body.preheader21, label %vector.ph
 

 
 for.cond.cleanup:                                 ; preds = %vector.body, %entry
   ret void
 }
 
 define arm_aapcs_vfpcc void @ssatmul_8t_q15(i16* nocapture readonly %pSrcA, i16* nocapture readonly %pSrcB, i16* noalias nocapture %pDst, i32 %N) {
 ; CHECK-LABEL: ssatmul_8t_q15:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, lr}
-; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    .pad #24
-; CHECK-NEXT:    sub sp, #24
+; CHECK-NEXT:    .pad #16
+; CHECK-NEXT:    sub sp, #16
 ; CHECK-NEXT:    cmp r3, #0
 ; CHECK-NEXT:    beq.w .LBB9_3
 ; CHECK-NEXT:  @ %bb.1: @ %vector.ph
 ; CHECK-NEXT:    add.w r12, r3, #7
 ; CHECK-NEXT:    adr r4, .LCPI9_0
 ; CHECK-NEXT:    bic r12, r12, #7
 ; CHECK-NEXT:    mov.w lr, #1
 ; CHECK-NEXT:    sub.w r12, r12, #8
 ; CHECK-NEXT:    vldrw.u32 q0, [r4]
 ; CHECK-NEXT:    adr r4, .LCPI9_1
-; CHECK-NEXT:    vmov.i8 q2, #0x0
+; CHECK-NEXT:    movs r5, #0
 ; CHECK-NEXT:    add.w lr, lr, r12, lsr #3
 ; CHECK-NEXT:    sub.w r12, r3, #1
 ; CHECK-NEXT:    dls lr, lr
 ; CHECK-NEXT:    vldrw.u32 q4, [r4]
-; CHECK-NEXT:    movs r3, #0
 ; CHECK-NEXT:    vdup.32 q1, r12
+; CHECK-NEXT:    vmov.i8 q2, #0x0
 ; CHECK-NEXT:    vmov.i8 q3, #0xff
 ; CHECK-NEXT:    vstrw.32 q0, [sp] @ 16-byte Spill
 ; CHECK-NEXT:  .LBB9_2: @ %vector.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    vldrw.u32 q5, [sp] @ 16-byte Reload
-; CHECK-NEXT:    vdup.32 q0, r3
-; CHECK-NEXT:    adds r3, #8
-; CHECK-NEXT:    vorr q5, q0, q5
-; CHECK-NEXT:    vorr q0, q0, q4
+; CHECK-NEXT:    vldrw.u32 q0, [sp] @ 16-byte Reload
+; CHECK-NEXT:    vdup.32 q6, r5
+; CHECK-NEXT:    adds r5, #8
+; CHECK-NEXT:    vorr q5, q6, q0
+; CHECK-NEXT:    vorr q6, q6, q4
 ; CHECK-NEXT:    vcmp.u32 cs, q1, q5
+; CHECK-NEXT:    vpsel q7, q3, q2
+; CHECK-NEXT:    vcmp.u32 cs, q1, q6
+; CHECK-NEXT:    vmov r4, s28
 ; CHECK-NEXT:    vpsel q6, q3, q2
-; CHECK-NEXT:    vcmp.u32 cs, q1, q0
-; CHECK-NEXT:    vmov r4, s24
-; CHECK-NEXT:    vpsel q0, q3, q2
 ; CHECK-NEXT:    vmov.16 q5[0], r4
-; CHECK-NEXT:    vmov r4, s25
+; CHECK-NEXT:    vmov r4, s29
 ; CHECK-NEXT:    vmov.16 q5[1], r4
-; CHECK-NEXT:    vmov r4, s26
+; CHECK-NEXT:    vmov r4, s30
 ; CHECK-NEXT:    vmov.16 q5[2], r4
-; CHECK-NEXT:    vmov r4, s27
+; CHECK-NEXT:    vmov r4, s31
 ; CHECK-NEXT:    vmov.16 q5[3], r4
-; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov r4, s24
 ; CHECK-NEXT:    vmov.16 q5[4], r4
-; CHECK-NEXT:    vmov r4, s1
+; CHECK-NEXT:    vmov r4, s25
 ; CHECK-NEXT:    vmov.16 q5[5], r4
-; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    vmov r4, s26
 ; CHECK-NEXT:    vmov.16 q5[6], r4
-; CHECK-NEXT:    vmov r4, s3
+; CHECK-NEXT:    vmov r4, s27
 ; CHECK-NEXT:    vmov.16 q5[7], r4
 ; CHECK-NEXT:    vpt.i16 ne, q5, zr
 ; CHECK-NEXT:    vldrht.u16 q6, [r0], #16
 ; CHECK-NEXT:    vmov.u16 r4, q6[0]
+; CHECK-NEXT:    vmov.u16 r3, q6[2]
+; CHECK-NEXT:    vmov q5[2], q5[0], r3, r4
+; CHECK-NEXT:    vmov.u16 r3, q6[1]
+; CHECK-NEXT:    vmov.u16 r4, q6[3]
 ; CHECK-NEXT:    vpst
 ; CHECK-NEXT:    vldrht.u16 q7, [r1], #16
-; CHECK-NEXT:    vmov.32 q5[0], r4
-; CHECK-NEXT:    vmov.u16 r4, q6[1]
-; CHECK-NEXT:    vmov.32 q5[1], r4
-; CHECK-NEXT:    vmov.u16 r4, q6[2]
-; CHECK-NEXT:    vmov.32 q5[2], r4
-; CHECK-NEXT:    vmov.u16 r4, q6[3]
-; CHECK-NEXT:    vmov.32 q5[3], r4
-; CHECK-NEXT:    vmov.u16 r4, q7[0]
-; CHECK-NEXT:    vmov.32 q0[0], r4
-; CHECK-NEXT:    vmov.u16 r4, q7[1]
-; CHECK-NEXT:    vmov.32 q0[1], r4
+; CHECK-NEXT:    vmov q5[3], q5[1], r4, r3
+; CHECK-NEXT:    vmov.u16 r3, q7[0]
 ; CHECK-NEXT:    vmov.u16 r4, q7[2]
-; CHECK-NEXT:    vmov.32 q0[2], r4
+; CHECK-NEXT:    vmov q0[2], q0[0], r4, r3
+; CHECK-NEXT:    vmov.u16 r3, q7[1]
 ; CHECK-NEXT:    vmov.u16 r4, q7[3]
-; CHECK-NEXT:    vmov.32 q0[3], r4
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r3
+; CHECK-NEXT:    vmov.u16 r4, q6[6]
 ; CHECK-NEXT:    vmullb.s16 q0, q0, q5
 ; CHECK-NEXT:    vqshrnb.s32 q0, q0, #15
 ; CHECK-NEXT:    vmovlb.s16 q0, q0
-; CHECK-NEXT:    vmov r4, s0
-; CHECK-NEXT:    vmov.16 q5[0], r4
-; CHECK-NEXT:    vmov r4, s1
-; CHECK-NEXT:    vmov.16 q5[1], r4
-; CHECK-NEXT:    vmov r4, s2
-; CHECK-NEXT:    vmov.16 q5[2], r4
-; CHECK-NEXT:    vmov r4, s3
-; CHECK-NEXT:    vmov.16 q5[3], r4
-; CHECK-NEXT:    vmov.u16 r4, q6[4]
-; CHECK-NEXT:    vmov.32 q0[0], r4
-; CHECK-NEXT:    vmov.u16 r4, q6[5]
-; CHECK-NEXT:    vmov.32 q0[1], r4
-; CHECK-NEXT:    vmov.u16 r4, q6[6]
-; CHECK-NEXT:    vmov.32 q0[2], r4
+; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    vmov.16 q5[0], r3
+; CHECK-NEXT:    vmov r3, s1
+; CHECK-NEXT:    vmov.16 q5[1], r3
+; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    vmov.16 q5[2], r3
+; CHECK-NEXT:    vmov r3, s3
+; CHECK-NEXT:    vmov.16 q5[3], r3
+; CHECK-NEXT:    vmov.u16 r3, q6[4]
+; CHECK-NEXT:    vmov q0[2], q0[0], r4, r3
+; CHECK-NEXT:    vmov.u16 r3, q6[5]
 ; CHECK-NEXT:    vmov.u16 r4, q6[7]
-; CHECK-NEXT:    vmov.32 q0[3], r4
-; CHECK-NEXT:    vmov.u16 r4, q7[4]
-; CHECK-NEXT:    vmov.32 q6[0], r4
-; CHECK-NEXT:    vmov.u16 r4, q7[5]
-; CHECK-NEXT:    vmov.32 q6[1], r4
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r3
+; CHECK-NEXT:    vmov.u16 r3, q7[4]
 ; CHECK-NEXT:    vmov.u16 r4, q7[6]
-; CHECK-NEXT:    vmov.32 q6[2], r4
+; CHECK-NEXT:    vmov q6[2], q6[0], r4, r3
+; CHECK-NEXT:    vmov.u16 r3, q7[5]
 ; CHECK-NEXT:    vmov.u16 r4, q7[7]
-; CHECK-NEXT:    vmov.32 q6[3], r4
+; CHECK-NEXT:    vmov q6[3], q6[1], r4, r3
 ; CHECK-NEXT:    vmullb.s16 q0, q6, q0
 ; CHECK-NEXT:    vqshrnb.s32 q0, q0, #15
 ; CHECK-NEXT:    vmovlb.s16 q0, q0
-; CHECK-NEXT:    vmov r4, s0
-; CHECK-NEXT:    vmov.16 q5[4], r4
-; CHECK-NEXT:    vmov r4, s1
-; CHECK-NEXT:    vmov.16 q5[5], r4
-; CHECK-NEXT:    vmov r4, s2
-; CHECK-NEXT:    vmov.16 q5[6], r4
-; CHECK-NEXT:    vmov r4, s3
-; CHECK-NEXT:    vmov.16 q5[7], r4
+; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    vmov.16 q5[4], r3
+; CHECK-NEXT:    vmov r3, s1
+; CHECK-NEXT:    vmov.16 q5[5], r3
+; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    vmov.16 q5[6], r3
+; CHECK-NEXT:    vmov r3, s3
+; CHECK-NEXT:    vmov.16 q5[7], r3
 ; CHECK-NEXT:    vpst
 ; CHECK-NEXT:    vstrht.16 q5, [r2], #16
 ; CHECK-NEXT:    le lr, .LBB9_2
 ; CHECK-NEXT:  .LBB9_3: @ %for.cond.cleanup
-; CHECK-NEXT:    add sp, #24
+; CHECK-NEXT:    add sp, #16
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    pop {r4, pc}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.4:
 ; CHECK-NEXT:  .LCPI9_0:
 ; CHECK-NEXT:    .long 0 @ 0x0
 ; CHECK-NEXT:    .long 1 @ 0x1
 ; CHECK-NEXT:    .long 2 @ 0x2
 ; CHECK-NEXT:    .long 3 @ 0x3
 ; CHECK-NEXT:  .LCPI9_1:

llvm/test/CodeGen/Thumb2/mve-saturating-arith.ll

 entry:
   %0 = call <4 x i32> @llvm.sadd.sat.v4i32(<4 x i32> %src1, <4 x i32> %src2)
   ret <4 x i32> %0
 }
 
 define arm_aapcs_vfpcc <2 x i64> @sadd_int64_t(<2 x i64> %src1, <2 x i64> %src2) {
 ; CHECK-LABEL: sadd_int64_t:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, lr}
-; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .save {r4, r5, r6, lr}
+; CHECK-NEXT:    push {r4, r5, r6, lr}
 ; CHECK-NEXT:    vmov r0, s5
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    vmov lr, s4
-; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    vmov r5, s6
+; CHECK-NEXT:    vmov r6, s2
 ; CHECK-NEXT:    cmp.w r0, #-1
 ; CHECK-NEXT:    cset r1, gt
 ; CHECK-NEXT:    cmp.w r2, #-1
 ; CHECK-NEXT:    cset r3, gt
 ; CHECK-NEXT:    cmp r3, r1
 ; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    cset r12, eq
 ; CHECK-NEXT:    adds.w r1, r1, lr
-; CHECK-NEXT:    adcs r2, r0
-; CHECK-NEXT:    cmp.w r2, #-1
-; CHECK-NEXT:    cset r0, gt
-; CHECK-NEXT:    cmp r3, r0
-; CHECK-NEXT:    cset r0, ne
-; CHECK-NEXT:    cmp r2, #0
-; CHECK-NEXT:    and.w r0, r0, r12
-; CHECK-NEXT:    mvn r12, #-2147483648
-; CHECK-NEXT:    and r3, r0, #1
-; CHECK-NEXT:    cset r0, mi
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    cinv r0, r12, eq
-; CHECK-NEXT:    cmp r3, #0
-; CHECK-NEXT:    it ne
-; CHECK-NEXT:    asrne r1, r2, #31
-; CHECK-NEXT:    csel r0, r0, r2, ne
-; CHECK-NEXT:    vmov.32 q2[0], r1
-; CHECK-NEXT:    vmov r2, s3
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s7
-; CHECK-NEXT:    cmp.w r0, #-1
-; CHECK-NEXT:    cset r1, gt
-; CHECK-NEXT:    cmp.w r2, #-1
-; CHECK-NEXT:    cset r3, gt
-; CHECK-NEXT:    cmp r3, r1
-; CHECK-NEXT:    vmov r1, s6
-; CHECK-NEXT:    cset lr, eq
-; CHECK-NEXT:    adds r1, r1, r4
 ; CHECK-NEXT:    adcs r0, r2
 ; CHECK-NEXT:    cmp.w r0, #-1
 ; CHECK-NEXT:    cset r2, gt
 ; CHECK-NEXT:    cmp r3, r2
+; CHECK-NEXT:    vmov r3, s7
 ; CHECK-NEXT:    cset r2, ne
-; CHECK-NEXT:    and.w r2, r2, lr
-; CHECK-NEXT:    ands r2, r2, #1
+; CHECK-NEXT:    and.w r2, r2, r12
+; CHECK-NEXT:    ands r12, r2, #1
+; CHECK-NEXT:    vmov r2, s3
 ; CHECK-NEXT:    it ne
 ; CHECK-NEXT:    asrne r1, r0, #31
+; CHECK-NEXT:    cmp.w r3, #-1
+; CHECK-NEXT:    cset lr, gt
+; CHECK-NEXT:    cmp.w r2, #-1
+; CHECK-NEXT:    cset r4, gt
+; CHECK-NEXT:    cmp r4, lr
+; CHECK-NEXT:    cset lr, eq
+; CHECK-NEXT:    adds r5, r5, r6
+; CHECK-NEXT:    adcs r2, r3
+; CHECK-NEXT:    mvn r6, #-2147483648
+; CHECK-NEXT:    cmp.w r2, #-1
+; CHECK-NEXT:    cset r3, gt
+; CHECK-NEXT:    cmp r4, r3
+; CHECK-NEXT:    cset r3, ne
+; CHECK-NEXT:    and.w r3, r3, lr
+; CHECK-NEXT:    ands r3, r3, #1
+; CHECK-NEXT:    it ne
+; CHECK-NEXT:    asrne r5, r2, #31
 ; CHECK-NEXT:    cmp r0, #0
-; CHECK-NEXT:    vmov.32 q2[2], r1
+; CHECK-NEXT:    vmov q0[2], q0[0], r5, r1
 ; CHECK-NEXT:    cset r1, mi
 ; CHECK-NEXT:    tst.w r1, #1
-; CHECK-NEXT:    cinv r1, r12, eq
-; CHECK-NEXT:    cmp r2, #0
+; CHECK-NEXT:    cinv r1, r6, eq
+; CHECK-NEXT:    cmp.w r12, #0
 ; CHECK-NEXT:    csel r0, r1, r0, ne
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vmov q0, q2
-; CHECK-NEXT:    pop {r4, pc}
+; CHECK-NEXT:    cmp r2, #0
+; CHECK-NEXT:    cset r1, mi
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    cinv r1, r6, eq
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    csel r1, r1, r2, ne
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
+; CHECK-NEXT:    pop {r4, r5, r6, pc}
 entry:
   %0 = call <2 x i64> @llvm.sadd.sat.v2i64(<2 x i64> %src1, <2 x i64> %src2)
   ret <2 x i64> %0
 }
 
 define arm_aapcs_vfpcc <16 x i8> @uadd_int8_t(<16 x i8> %src1, <16 x i8> %src2) {
 ; CHECK-LABEL: uadd_int8_t:
 ; CHECK:       @ %bb.0: @ %entry

 entry:
   %0 = call <4 x i32> @llvm.uadd.sat.v4i32(<4 x i32> %src1, <4 x i32> %src2)
   ret <4 x i32> %0
 }
 
 define arm_aapcs_vfpcc <2 x i64> @uadd_int64_t(<2 x i64> %src1, <2 x i64> %src2) {
 ; CHECK-LABEL: uadd_int64_t:
 ; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    vmov r2, s4
 ; CHECK-NEXT:    mov.w r12, #0
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r0, s5
 ; CHECK-NEXT:    vmov r1, s1
+; CHECK-NEXT:    vmov r4, s6
+; CHECK-NEXT:    vmov r5, s2
 ; CHECK-NEXT:    adds r2, r2, r3
-; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    vmov r3, s7
 ; CHECK-NEXT:    adcs r0, r1
-; CHECK-NEXT:    adcs r1, r12, #0
-; CHECK-NEXT:    itt ne
-; CHECK-NEXT:    movne.w r0, #-1
-; CHECK-NEXT:    movne.w r2, #-1
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s7
 ; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    adds r2, r2, r3
-; CHECK-NEXT:    adcs r0, r1
-; CHECK-NEXT:    adcs r1, r12, #0
+; CHECK-NEXT:    adcs lr, r12, #0
 ; CHECK-NEXT:    it ne
 ; CHECK-NEXT:    movne.w r2, #-1
-; CHECK-NEXT:    vmov.32 q2[2], r2
+; CHECK-NEXT:    adds r4, r4, r5
+; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    adcs r3, r12, #0
+; CHECK-NEXT:    it ne
+; CHECK-NEXT:    movne.w r4, #-1
+; CHECK-NEXT:    cmp.w lr, #0
+; CHECK-NEXT:    vmov q0[2], q0[0], r4, r2
 ; CHECK-NEXT:    it ne
 ; CHECK-NEXT:    movne.w r0, #-1
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vmov q0, q2
-; CHECK-NEXT:    bx lr
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    it ne
+; CHECK-NEXT:    movne.w r1, #-1
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %0 = call <2 x i64> @llvm.uadd.sat.v2i64(<2 x i64> %src1, <2 x i64> %src2)
   ret <2 x i64> %0
 }
 
 
 define arm_aapcs_vfpcc <16 x i8> @ssub_int8_t(<16 x i8> %src1, <16 x i8> %src2) {
 ; CHECK-LABEL: ssub_int8_t:

 entry:
   %0 = call <4 x i32> @llvm.ssub.sat.v4i32(<4 x i32> %src1, <4 x i32> %src2)
   ret <4 x i32> %0
 }
 
 define arm_aapcs_vfpcc <2 x i64> @ssub_int64_t(<2 x i64> %src1, <2 x i64> %src2) {
 ; CHECK-LABEL: ssub_int64_t:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, lr}
-; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .save {r4, r5, r6, lr}
+; CHECK-NEXT:    push {r4, r5, r6, lr}
 ; CHECK-NEXT:    vmov r0, s5
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    vmov lr, s4
-; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    vmov r5, s6
+; CHECK-NEXT:    vmov r6, s2
 ; CHECK-NEXT:    cmp.w r0, #-1
 ; CHECK-NEXT:    cset r1, gt
 ; CHECK-NEXT:    cmp.w r2, #-1
 ; CHECK-NEXT:    cset r3, gt
 ; CHECK-NEXT:    cmp r3, r1
 ; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    cset r12, ne
 ; CHECK-NEXT:    subs.w r1, r1, lr
-; CHECK-NEXT:    sbcs r2, r0
-; CHECK-NEXT:    cmp.w r2, #-1
-; CHECK-NEXT:    cset r0, gt
-; CHECK-NEXT:    cmp r3, r0
-; CHECK-NEXT:    cset r0, ne
-; CHECK-NEXT:    cmp r2, #0
-; CHECK-NEXT:    and.w r0, r0, r12
-; CHECK-NEXT:    mvn r12, #-2147483648
-; CHECK-NEXT:    and r3, r0, #1
-; CHECK-NEXT:    cset r0, mi
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    cinv r0, r12, eq
-; CHECK-NEXT:    cmp r3, #0
-; CHECK-NEXT:    it ne
-; CHECK-NEXT:    asrne r1, r2, #31
-; CHECK-NEXT:    csel r0, r0, r2, ne
-; CHECK-NEXT:    vmov.32 q2[0], r1
-; CHECK-NEXT:    vmov r2, s3
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s7
-; CHECK-NEXT:    cmp.w r0, #-1
-; CHECK-NEXT:    cset r1, gt
-; CHECK-NEXT:    cmp.w r2, #-1
-; CHECK-NEXT:    cset r3, gt
-; CHECK-NEXT:    cmp r3, r1
-; CHECK-NEXT:    vmov r1, s6
-; CHECK-NEXT:    cset lr, ne
-; CHECK-NEXT:    subs r1, r4, r1
 ; CHECK-NEXT:    sbc.w r0, r2, r0
 ; CHECK-NEXT:    cmp.w r0, #-1
 ; CHECK-NEXT:    cset r2, gt
 ; CHECK-NEXT:    cmp r3, r2
+; CHECK-NEXT:    vmov r3, s7
 ; CHECK-NEXT:    cset r2, ne
-; CHECK-NEXT:    and.w r2, r2, lr
-; CHECK-NEXT:    ands r2, r2, #1
+; CHECK-NEXT:    and.w r2, r2, r12
+; CHECK-NEXT:    ands r12, r2, #1
+; CHECK-NEXT:    vmov r2, s3
 ; CHECK-NEXT:    it ne
 ; CHECK-NEXT:    asrne r1, r0, #31
+; CHECK-NEXT:    cmp.w r3, #-1
+; CHECK-NEXT:    cset lr, gt
+; CHECK-NEXT:    cmp.w r2, #-1
+; CHECK-NEXT:    cset r4, gt
+; CHECK-NEXT:    cmp r4, lr
+; CHECK-NEXT:    cset lr, ne
+; CHECK-NEXT:    subs r5, r6, r5
+; CHECK-NEXT:    sbcs r2, r3
+; CHECK-NEXT:    mvn r6, #-2147483648
+; CHECK-NEXT:    cmp.w r2, #-1
+; CHECK-NEXT:    cset r3, gt
+; CHECK-NEXT:    cmp r4, r3
+; CHECK-NEXT:    cset r3, ne
+; CHECK-NEXT:    and.w r3, r3, lr
+; CHECK-NEXT:    ands r3, r3, #1
+; CHECK-NEXT:    it ne
+; CHECK-NEXT:    asrne r5, r2, #31
 ; CHECK-NEXT:    cmp r0, #0
-; CHECK-NEXT:    vmov.32 q2[2], r1
+; CHECK-NEXT:    vmov q0[2], q0[0], r5, r1
 ; CHECK-NEXT:    cset r1, mi
 ; CHECK-NEXT:    tst.w r1, #1
-; CHECK-NEXT:    cinv r1, r12, eq
-; CHECK-NEXT:    cmp r2, #0
+; CHECK-NEXT:    cinv r1, r6, eq
+; CHECK-NEXT:    cmp.w r12, #0
 ; CHECK-NEXT:    csel r0, r1, r0, ne
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vmov q0, q2
-; CHECK-NEXT:    pop {r4, pc}
+; CHECK-NEXT:    cmp r2, #0
+; CHECK-NEXT:    cset r1, mi
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    cinv r1, r6, eq
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    csel r1, r1, r2, ne
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
+; CHECK-NEXT:    pop {r4, r5, r6, pc}
 entry:
   %0 = call <2 x i64> @llvm.ssub.sat.v2i64(<2 x i64> %src1, <2 x i64> %src2)
   ret <2 x i64> %0
 }
 
 define arm_aapcs_vfpcc <16 x i8> @usub_int8_t(<16 x i8> %src1, <16 x i8> %src2) {
 ; CHECK-LABEL: usub_int8_t:
 ; CHECK:       @ %bb.0: @ %entry

 entry:
   %0 = call <4 x i32> @llvm.usub.sat.v4i32(<4 x i32> %src1, <4 x i32> %src2)
   ret <4 x i32> %0
 }
 
 define arm_aapcs_vfpcc <2 x i64> @usub_int64_t(<2 x i64> %src1, <2 x i64> %src2) {
 ; CHECK-LABEL: usub_int64_t:
 ; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    vmov r2, s4
 ; CHECK-NEXT:    mov.w r12, #0
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r0, s5
 ; CHECK-NEXT:    vmov r1, s1
+; CHECK-NEXT:    vmov r4, s6
+; CHECK-NEXT:    vmov r5, s2
 ; CHECK-NEXT:    subs r2, r3, r2
-; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    vmov r3, s7
 ; CHECK-NEXT:    sbcs.w r0, r1, r0
 ; CHECK-NEXT:    adc r1, r12, #0
-; CHECK-NEXT:    rsbs.w r1, r1, #1
-; CHECK-NEXT:    itt ne
-; CHECK-NEXT:    movne r0, #0
-; CHECK-NEXT:    movne r2, #0
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s7
+; CHECK-NEXT:    rsbs.w lr, r1, #1
 ; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    subs r2, r3, r2
-; CHECK-NEXT:    sbcs.w r0, r1, r0
-; CHECK-NEXT:    adc r1, r12, #0
-; CHECK-NEXT:    rsbs.w r1, r1, #1
 ; CHECK-NEXT:    it ne
 ; CHECK-NEXT:    movne r2, #0
-; CHECK-NEXT:    vmov.32 q2[2], r2
+; CHECK-NEXT:    subs r4, r5, r4
+; CHECK-NEXT:    sbcs r1, r3
+; CHECK-NEXT:    adc r3, r12, #0
+; CHECK-NEXT:    rsbs.w r3, r3, #1
+; CHECK-NEXT:    it ne
+; CHECK-NEXT:    movne r4, #0
+; CHECK-NEXT:    cmp.w lr, #0
+; CHECK-NEXT:    vmov q0[2], q0[0], r4, r2
 ; CHECK-NEXT:    it ne
 ; CHECK-NEXT:    movne r0, #0
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vmov q0, q2
-; CHECK-NEXT:    bx lr
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    it ne
+; CHECK-NEXT:    movne r1, #0
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %0 = call <2 x i64> @llvm.usub.sat.v2i64(<2 x i64> %src1, <2 x i64> %src2)
   ret <2 x i64> %0
 }
 
 
 declare <16 x i8> @llvm.sadd.sat.v16i8(<16 x i8> %src1, <16 x i8> %src2)
 declare <8 x i16> @llvm.sadd.sat.v8i16(<8 x i16> %src1, <8 x i16> %src2)

llvm/test/CodeGen/Thumb2/mve-scatter-ind8-unscaled.ll

 
 ; Expand
 define arm_aapcs_vfpcc void @unscaled_v2i8_i8(i8* %base, <2 x i8>* %offptr, <2 x i8> %input) {
 ; CHECK-LABEL: unscaled_v2i8_i8:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    ldrb r2, [r1]
 ; CHECK-NEXT:    vmov.i32 q1, #0xff
 ; CHECK-NEXT:    ldrb r1, [r1, #1]
-; CHECK-NEXT:    vmov.32 q2[0], r2
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r2
 ; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    vmov.32 q2[2], r1
 ; CHECK-NEXT:    vand q1, q2, q1
 ; CHECK-NEXT:    vmov r1, s4
 ; CHECK-NEXT:    strb r2, [r0, r1]
 ; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    strb r2, [r0, r1]
 ; CHECK-NEXT:    bx lr
 entry:

llvm/test/CodeGen/Thumb2/mve-sext.ll

entry:
   %shr = ashr exact <8 x i16> %shl, <i16 15, i16 15, i16 15, i16 15, i16 15, i16 15, i16 15, i16 15>
   ret <8 x i16> %shr
 }
 
 define arm_aapcs_vfpcc <2 x i64> @sext_v2i64_v2i64_v2i32(<2 x i64> %m) {
 ; CHECK-LABEL: sext_v2i64_v2i64_v2i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s0
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov r1, s2
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
 ; CHECK-NEXT:    asrs r0, r0, #31
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    asrs r0, r0, #31
-; CHECK-NEXT:    vmov.32 q1[3], r0
-; CHECK-NEXT:    vmov q0, q1
+; CHECK-NEXT:    asrs r1, r1, #31
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
 ; CHECK-NEXT:    bx lr
 entry:
   %shl = shl <2 x i64> %m, <i64 32, i64 32>
   %shr = ashr exact <2 x i64> %shl, <i64 32, i64 32>
   ret <2 x i64> %shr
 }
 
 define arm_aapcs_vfpcc <2 x i64> @sext_v2i64_v2i64_v2i35(<2 x i64> %m) {
 ; CHECK-LABEL: sext_v2i64_v2i64_v2i35:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s0
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov r1, s2
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
 ; CHECK-NEXT:    vmov r0, s1
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    sbfx r0, r0, #0, #3
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    sbfx r0, r0, #0, #3
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    sbfx r1, r1, #0, #3
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    vmov q0, q1
 ; CHECK-NEXT:    bx lr
 entry:
   %shl = shl <2 x i64> %m, <i64 29, i64 29>
   %shr = ashr exact <2 x i64> %shl, <i64 29, i64 29>
   ret <2 x i64> %shr
 }
 
entry:
   %0 = sext <16 x i8> %src to <16 x i16>
   ret <16 x i16> %0
 }
 
 define arm_aapcs_vfpcc <8 x i32> @sext_v8i16_v8i32(<8 x i16> %src) {
 ; CHECK-LABEL: sext_v8i16_v8i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[2]
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[3]
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[3]
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[4]
+; CHECK-NEXT:    vmov.u16 r1, q0[6]
 ; CHECK-NEXT:    vmovlb.s16 q2, q1
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[5]
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[6]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[7]
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[7]
 ; CHECK-NEXT:    vmov q0, q2
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    vmovlb.s16 q1, q1
 ; CHECK-NEXT:    bx lr
 entry:
   %0 = sext <8 x i16> %src to <8 x i32>
   ret <8 x i32> %0
 }
 
 define arm_aapcs_vfpcc <16 x i32> @sext_v16i8_v16i32(<16 x i8> %src) {
 ; CHECK-LABEL: sext_v16i8_v16i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .vsave {d8, d9}
 ; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    vmov.u8 r0, q0[0]
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov.u8 r1, q0[2]
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[1]
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[2]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[3]
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    vmov.u8 r1, q0[3]
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[4]
 ; CHECK-NEXT:    vmovlb.s8 q1, q1
+; CHECK-NEXT:    vmov.u8 r1, q0[6]
 ; CHECK-NEXT:    vmovlb.s16 q4, q1
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[5]
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[6]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[7]
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    vmov.u8 r1, q0[7]
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[8]
-; CHECK-NEXT:    vmov.32 q2[0], r0
+; CHECK-NEXT:    vmov.u8 r1, q0[10]
+; CHECK-NEXT:    vmovlb.s8 q1, q1
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[9]
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[10]
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[11]
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    vmov.u8 r1, q0[11]
+; CHECK-NEXT:    vmovlb.s16 q1, q1
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[12]
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[13]
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[14]
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[15]
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vmovlb.s8 q1, q1
+; CHECK-NEXT:    vmov.u8 r1, q0[14]
 ; CHECK-NEXT:    vmovlb.s8 q2, q2
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
+; CHECK-NEXT:    vmov.u8 r0, q0[13]
+; CHECK-NEXT:    vmov.u8 r1, q0[15]
+; CHECK-NEXT:    vmovlb.s16 q2, q2
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
 ; CHECK-NEXT:    vmovlb.s8 q0, q3
 ; CHECK-NEXT:    vmovlb.s16 q3, q0
-; CHECK-NEXT:    vmovlb.s16 q1, q1
-; CHECK-NEXT:    vmovlb.s16 q2, q2
 ; CHECK-NEXT:    vmov q0, q4
 ; CHECK-NEXT:    vpop {d8, d9}
 ; CHECK-NEXT:    bx lr
 entry:
   %0 = sext <16 x i8> %src to <16 x i32>
   ret <16 x i32> %0
 }
 
 define arm_aapcs_vfpcc <2 x i64> @sext_v2i32_v2i64(<2 x i32> %src) {
 ; CHECK-LABEL: sext_v2i32_v2i64:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s0
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov r1, s2
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
 ; CHECK-NEXT:    asrs r0, r0, #31
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    asrs r0, r0, #31
-; CHECK-NEXT:    vmov.32 q1[3], r0
-; CHECK-NEXT:    vmov q0, q1
+; CHECK-NEXT:    asrs r1, r1, #31
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
 ; CHECK-NEXT:    bx lr
 entry:
   %0 = sext <2 x i32> %src to <2 x i64>
   ret <2 x i64> %0
 }
 
 
 define arm_aapcs_vfpcc <8 x i16> @zext_v8i8_v8i16(<8 x i8> %src) {
entry:
   %0 = zext <16 x i8> %src to <16 x i16>
   ret <16 x i16> %0
 }
 
 define arm_aapcs_vfpcc <8 x i32> @zext_v8i16_v8i32(<8 x i16> %src) {
 ; CHECK-LABEL: zext_v8i16_v8i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[2]
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[3]
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[3]
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[4]
+; CHECK-NEXT:    vmov.u16 r1, q0[6]
 ; CHECK-NEXT:    vmovlb.u16 q2, q1
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[5]
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[6]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[7]
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[7]
 ; CHECK-NEXT:    vmov q0, q2
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    vmovlb.u16 q1, q1
 ; CHECK-NEXT:    bx lr
 entry:
   %0 = zext <8 x i16> %src to <8 x i32>
   ret <8 x i32> %0
 }
 
 define arm_aapcs_vfpcc <16 x i32> @zext_v16i8_v16i32(<16 x i8> %src) {
 ; CHECK-LABEL: zext_v16i8_v16i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11}
 ; CHECK-NEXT:    vmov.u8 r0, q0[0]
-; CHECK-NEXT:    vmov.i32 q3, #0xff
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov.u8 r1, q0[2]
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[1]
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[2]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[3]
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    vmov.u8 r1, q0[3]
+; CHECK-NEXT:    vmov.i32 q3, #0xff
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[4]
+; CHECK-NEXT:    vmov.u8 r1, q0[6]
 ; CHECK-NEXT:    vand q4, q1, q3
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[5]
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[6]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[7]
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    vmov.u8 r1, q0[7]
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[8]
-; CHECK-NEXT:    vmov.32 q2[0], r0
+; CHECK-NEXT:    vmov.u8 r1, q0[10]
+; CHECK-NEXT:    vand q1, q1, q3
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[9]
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[10]
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[11]
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    vmov.u8 r1, q0[11]
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[12]
-; CHECK-NEXT:    vmov.32 q5[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[13]
-; CHECK-NEXT:    vmov.32 q5[1], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[14]
-; CHECK-NEXT:    vmov.32 q5[2], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[15]
-; CHECK-NEXT:    vmov.32 q5[3], r0
-; CHECK-NEXT:    vand q1, q1, q3
+; CHECK-NEXT:    vmov.u8 r1, q0[14]
 ; CHECK-NEXT:    vand q2, q2, q3
-; CHECK-NEXT:    vand q3, q5, q3
+; CHECK-NEXT:    vmov q5[2], q5[0], r1, r0
+; CHECK-NEXT:    vmov.u8 r0, q0[13]
+; CHECK-NEXT:    vmov.u8 r1, q0[15]
 ; CHECK-NEXT:    vmov q0, q4
+; CHECK-NEXT:    vmov q5[3], q5[1], r1, r0
+; CHECK-NEXT:    vand q3, q5, q3
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11}
 ; CHECK-NEXT:    bx lr
 entry:
   %0 = zext <16 x i8> %src to <16 x i32>
   ret <16 x i32> %0
 }
 
 define arm_aapcs_vfpcc <2 x i64> @zext_v2i32_v2i64(<2 x i32> %src) {

llvm/test/CodeGen/Thumb2/mve-shifts.ll

 
 define arm_aapcs_vfpcc <2 x i64> @shl_qq_int64_t(<2 x i64> %src1, <2 x i64> %src2) {
 ; CHECK-LABEL: shl_qq_int64_t:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s4
 ; CHECK-NEXT:    vmov r1, s1
 ; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    lsll r2, r1, r0
-; CHECK-NEXT:    vmov r0, s6
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    vmov.32 q2[1], r1
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    lsll r2, r1, r0
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r1
-; CHECK-NEXT:    vmov q0, q2
+; CHECK-NEXT:    vmov r12, s6
+; CHECK-NEXT:    vmov r3, s3
+; CHECK-NEXT:    vmov r0, s2
+; CHECK-NEXT:    lsll r0, r3, r12
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r1
 ; CHECK-NEXT:    bx lr
 entry:
   %0 = shl <2 x i64> %src1, %src2
   ret <2 x i64> %0
 }
 
 
 define arm_aapcs_vfpcc <16 x i8> @shru_qq_int8_t(<16 x i8> %src1, <16 x i8> %src2) {

 entry:
   %0 = lshr <4 x i32> %src1, %src2
   ret <4 x i32> %0
 }
 
 define arm_aapcs_vfpcc <2 x i64> @shru_qq_int64_t(<2 x i64> %src1, <2 x i64> %src2) {
 ; CHECK-LABEL: shru_qq_int64_t:
 ; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r5, lr}
+; CHECK-NEXT:    push {r5, lr}
 ; CHECK-NEXT:    vmov r2, s4
-; CHECK-NEXT:    vmov r1, s1
+; CHECK-NEXT:    vmov r1, s6
+; CHECK-NEXT:    vmov r5, s1
 ; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov r3, s3
 ; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    lsll r0, r1, r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vmov.32 q2[1], r1
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    lsll r0, r1, r2
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r1
-; CHECK-NEXT:    vmov q0, q2
-; CHECK-NEXT:    bx lr
+; CHECK-NEXT:    lsll r0, r5, r2
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    rsbs r1, r1, #0
+; CHECK-NEXT:    lsll r2, r3, r1
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r5
+; CHECK-NEXT:    pop {r5, pc}
 entry:
   %0 = lshr <2 x i64> %src1, %src2
   ret <2 x i64> %0
 }
 
 
 define arm_aapcs_vfpcc <16 x i8> @shrs_qq_int8_t(<16 x i8> %src1, <16 x i8> %src2) {
 ; CHECK-LABEL: shrs_qq_int8_t:

 
 define arm_aapcs_vfpcc <2 x i64> @shrs_qq_int64_t(<2 x i64> %src1, <2 x i64> %src2) {
 ; CHECK-LABEL: shrs_qq_int64_t:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s4
 ; CHECK-NEXT:    vmov r1, s1
 ; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    asrl r2, r1, r0
-; CHECK-NEXT:    vmov r0, s6
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    vmov.32 q2[1], r1
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    asrl r2, r1, r0
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r1
-; CHECK-NEXT:    vmov q0, q2
+; CHECK-NEXT:    vmov r12, s6
+; CHECK-NEXT:    vmov r3, s3
+; CHECK-NEXT:    vmov r0, s2
+; CHECK-NEXT:    asrl r0, r3, r12
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r1
 ; CHECK-NEXT:    bx lr
 entry:
   %0 = ashr <2 x i64> %src1, %src2
   ret <2 x i64> %0
 }
 
 
 define arm_aapcs_vfpcc <16 x i8> @shl_qi_int8_t(<16 x i8> %src1) {
entry:
   ret <4 x i32> %0
 }
 
 define arm_aapcs_vfpcc <2 x i64> @shl_qi_int64_t(<2 x i64> %src1) {
 ; CHECK-LABEL: shl_qi_int64_t:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r1, s1
 ; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov r3, s3
 ; CHECK-NEXT:    lsll r0, r1, #4
-; CHECK-NEXT:    vmov.32 q1[0], r0
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    lsll r0, r1, #4
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.32 q1[3], r1
-; CHECK-NEXT:    vmov q0, q1
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    lsll r2, r3, #4
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r1
 ; CHECK-NEXT:    bx lr
 entry:
   %0 = shl <2 x i64> %src1, <i64 4, i64 4>
   ret <2 x i64> %0
 }
 
 
 define arm_aapcs_vfpcc <16 x i8> @shru_qi_int8_t(<16 x i8> %src1) {
entry:
   ret <4 x i32> %0
 }
 
 define arm_aapcs_vfpcc <2 x i64> @shru_qi_int64_t(<2 x i64> %src1) {
 ; CHECK-LABEL: shru_qi_int64_t:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r1, s1
 ; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov r3, s3
 ; CHECK-NEXT:    lsrl r0, r1, #4
-; CHECK-NEXT:    vmov.32 q1[0], r0
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    lsrl r0, r1, #4
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.32 q1[3], r1
-; CHECK-NEXT:    vmov q0, q1
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    lsrl r2, r3, #4
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r1
 ; CHECK-NEXT:    bx lr
 entry:
   %0 = lshr <2 x i64> %src1, <i64 4, i64 4>
   ret <2 x i64> %0
 }
 
 
 define arm_aapcs_vfpcc <16 x i8> @shrs_qi_int8_t(<16 x i8> %src1) {
entry:
   ret <4 x i32> %0
 }
 
 define arm_aapcs_vfpcc <2 x i64> @shrs_qi_int64_t(<2 x i64> %src1) {
 ; CHECK-LABEL: shrs_qi_int64_t:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r1, s1
 ; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov r3, s3
 ; CHECK-NEXT:    asrl r0, r1, #4
-; CHECK-NEXT:    vmov.32 q1[0], r0
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    asrl r0, r1, #4
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.32 q1[3], r1
-; CHECK-NEXT:    vmov q0, q1
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    asrl r2, r3, #4
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r1
 ; CHECK-NEXT:    bx lr
 entry:
   %0 = ashr <2 x i64> %src1, <i64 4, i64 4>
   ret <2 x i64> %0
 }
 
 
 define arm_aapcs_vfpcc <16 x i8> @shl_qr_int8_t(<16 x i8> %src1, i8 %src2) {
entry:
   %0 = shl <4 x i32> %src1, %s
   ret <4 x i32> %0
 }
 
 define arm_aapcs_vfpcc <2 x i64> @shl_qr_int64_t(<2 x i64> %src1, i64 %src2) {
 ; CHECK-LABEL: shl_qr_int64_t:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r1, s1
-; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    lsll r2, r1, r0
-; CHECK-NEXT:    vmov.32 q1[0], r2
+; CHECK-NEXT:    vmov r12, s0
+; CHECK-NEXT:    vmov r3, s3
+; CHECK-NEXT:    lsll r12, r1, r0
 ; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    lsll r2, r1, r0
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vmov.32 q1[3], r1
-; CHECK-NEXT:    vmov q0, q1
+; CHECK-NEXT:    lsll r2, r3, r0
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r12
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r1
 ; CHECK-NEXT:    bx lr
 entry:
   %i = insertelement <2 x i64> undef, i64 %src2, i32 0
   %s = shufflevector <2 x i64> %i, <2 x i64> undef, <2 x i32> zeroinitializer
   %0 = shl <2 x i64> %src1, %s
   ret <2 x i64> %0
 }
 
entry:
   %s = shufflevector <4 x i32> %i, <4 x i32> undef, <4 x i32> zeroinitializer
   %0 = lshr <4 x i32> %src1, %s
   ret <4 x i32> %0
 }
 
 define arm_aapcs_vfpcc <2 x i64> @shru_qr_int64_t(<2 x i64> %src1, i64 %src2) {
 ; CHECK-LABEL: shru_qr_int64_t:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    rsbs r0, r0, #0
+; CHECK-NEXT:    rsb.w r12, r0, #0
 ; CHECK-NEXT:    vmov r1, s1
 ; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    lsll r2, r1, r0
-; CHECK-NEXT:    vmov.32 q1[0], r2
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    lsll r2, r1, r0
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vmov.32 q1[3], r1
-; CHECK-NEXT:    vmov q0, q1
+; CHECK-NEXT:    vmov r3, s3
+; CHECK-NEXT:    lsll r2, r1, r12
+; CHECK-NEXT:    vmov r0, s2
+; CHECK-NEXT:    lsll r0, r3, r12
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r1
 ; CHECK-NEXT:    bx lr
 entry:
   %i = insertelement <2 x i64> undef, i64 %src2, i32 0
   %s = shufflevector <2 x i64> %i, <2 x i64> undef, <2 x i32> zeroinitializer
   %0 = lshr <2 x i64> %src1, %s
   ret <2 x i64> %0
 }
 
entry:
   %0 = ashr <4 x i32> %src1, %s
   ret <4 x i32> %0
 }
 
 define arm_aapcs_vfpcc <2 x i64> @shrs_qr_int64_t(<2 x i64> %src1, i64 %src2) {
 ; CHECK-LABEL: shrs_qr_int64_t:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r1, s1
-; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    asrl r2, r1, r0
-; CHECK-NEXT:    vmov.32 q1[0], r2
+; CHECK-NEXT:    vmov r12, s0
+; CHECK-NEXT:    vmov r3, s3
+; CHECK-NEXT:    asrl r12, r1, r0
 ; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    asrl r2, r1, r0
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vmov.32 q1[3], r1
-; CHECK-NEXT:    vmov q0, q1
+; CHECK-NEXT:    asrl r2, r3, r0
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r12
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r1
 ; CHECK-NEXT:    bx lr
 entry:
   %i = insertelement <2 x i64> undef, i64 %src2, i32 0
   %s = shufflevector <2 x i64> %i, <2 x i64> undef, <2 x i32> zeroinitializer
   %0 = ashr <2 x i64> %src1, %s
   ret <2 x i64> %0
 }
 

llvm/test/CodeGen/Thumb2/mve-simple-arith.ll

entry:
   ret <4 x i32> %0
 }
 
 define arm_aapcs_vfpcc <2 x i64> @add_int64_t(<2 x i64> %src1, <2 x i64> %src2) {
 ; CHECK-LABEL: add_int64_t:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    vmov r3, s2
-; CHECK-NEXT:    vmov r0, s7
-; CHECK-NEXT:    vmov r1, s3
+; CHECK-NEXT:    vmov r2, s4
+; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    vmov r0, s5
+; CHECK-NEXT:    vmov r1, s1
 ; CHECK-NEXT:    adds.w lr, r3, r2
-; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    vmov r3, s1
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    vmov r3, s3
 ; CHECK-NEXT:    adc.w r12, r1, r0
-; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r1, s5
+; CHECK-NEXT:    vmov r0, s6
+; CHECK-NEXT:    vmov r1, s7
 ; CHECK-NEXT:    adds r0, r0, r2
 ; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.32 q0[1], r1
-; CHECK-NEXT:    vmov.32 q0[2], lr
-; CHECK-NEXT:    vmov.32 q0[3], r12
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, lr
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r12
 ; CHECK-NEXT:    pop {r7, pc}
 entry:
   %0 = add nsw <2 x i64> %src1, %src2
   ret <2 x i64> %0
 }
 
 define arm_aapcs_vfpcc <4 x float> @add_float32_t(<4 x float> %src1, <4 x float> %src2) {
 ; CHECK-MVE-LABEL: add_float32_t:
entry:
   ret <4 x i32> %0
 }
 
 define arm_aapcs_vfpcc <2 x i64> @sub_int64_t(<2 x i64> %src1, <2 x i64> %src2) {
 ; CHECK-LABEL: sub_int64_t:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    vmov r1, s7
+; CHECK-NEXT:    vmov r2, s0
+; CHECK-NEXT:    vmov r3, s4
+; CHECK-NEXT:    vmov r0, s1
+; CHECK-NEXT:    vmov r1, s5
 ; CHECK-NEXT:    subs.w lr, r3, r2
-; CHECK-NEXT:    vmov r2, s4
-; CHECK-NEXT:    vmov r3, s5
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    vmov r3, s7
 ; CHECK-NEXT:    sbc.w r12, r1, r0
-; CHECK-NEXT:    vmov r0, s0
-; CHECK-NEXT:    vmov r1, s1
+; CHECK-NEXT:    vmov r0, s2
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    subs r0, r2, r0
 ; CHECK-NEXT:    sbc.w r1, r3, r1
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.32 q0[1], r1
-; CHECK-NEXT:    vmov.32 q0[2], lr
-; CHECK-NEXT:    vmov.32 q0[3], r12
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, lr
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r12
 ; CHECK-NEXT:    pop {r7, pc}
 entry:
   %0 = sub nsw <2 x i64> %src2, %src1
   ret <2 x i64> %0
 }
 
 define arm_aapcs_vfpcc <4 x float> @sub_float32_t(<4 x float> %src1, <4 x float> %src2) {
 ; CHECK-MVE-LABEL: sub_float32_t:

 ; CHECK-NEXT:    vmov r3, s6
 ; CHECK-NEXT:    vmov r1, s2
 ; CHECK-NEXT:    vmov r2, s7
 ; CHECK-NEXT:    umull r4, r5, r1, r3
 ; CHECK-NEXT:    mla r1, r1, r2, r5
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    mla r0, r2, r0, lr
 ; CHECK-NEXT:    vmov r2, s3
-; CHECK-NEXT:    vmov.32 q0[0], r12
-; CHECK-NEXT:    vmov.32 q0[1], r0
-; CHECK-NEXT:    vmov.32 q0[2], r4
+; CHECK-NEXT:    vmov q0[2], q0[0], r4, r12
 ; CHECK-NEXT:    mla r1, r2, r3, r1
-; CHECK-NEXT:    vmov.32 q0[3], r1
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
 ; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %0 = mul nsw <2 x i64> %src1, %src2
   ret <2 x i64> %0
 }
 
 define arm_aapcs_vfpcc <8 x half> @mul_float16_t(<8 x half> %src1, <8 x half> %src2) {
 ; CHECK-MVE-LABEL: mul_float16_t:

llvm/test/CodeGen/Thumb2/mve-soft-float-abi.ll

 ; CHECK-FP-LABEL: vector_add_i64:
 ; CHECK-FP:       @ %bb.0: @ %entry
 ; CHECK-FP-NEXT:    .save {r7, lr}
 ; CHECK-FP-NEXT:    push {r7, lr}
 ; CHECK-FP-NEXT:    vmov d1, r2, r3
 ; CHECK-FP-NEXT:    vmov d0, r0, r1
 ; CHECK-FP-NEXT:    add r0, sp, #8
 ; CHECK-FP-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-FP-NEXT:    vmov r1, s2
-; CHECK-FP-NEXT:    vmov r0, s3
-; CHECK-FP-NEXT:    vmov r3, s6
-; CHECK-FP-NEXT:    vmov r2, s7
+; CHECK-FP-NEXT:    vmov r1, s0
+; CHECK-FP-NEXT:    vmov r3, s4
+; CHECK-FP-NEXT:    vmov r0, s1
+; CHECK-FP-NEXT:    vmov r2, s5
 ; CHECK-FP-NEXT:    adds.w lr, r1, r3
-; CHECK-FP-NEXT:    vmov r3, s0
-; CHECK-FP-NEXT:    vmov r1, s4
+; CHECK-FP-NEXT:    vmov r3, s2
+; CHECK-FP-NEXT:    vmov r1, s6
 ; CHECK-FP-NEXT:    adc.w r12, r0, r2
-; CHECK-FP-NEXT:    vmov r2, s1
-; CHECK-FP-NEXT:    vmov r0, s5
+; CHECK-FP-NEXT:    vmov r2, s3
+; CHECK-FP-NEXT:    vmov r0, s7
 ; CHECK-FP-NEXT:    adds r1, r1, r3
-; CHECK-FP-NEXT:    vmov.32 q0[0], r1
+; CHECK-FP-NEXT:    vmov q0[2], q0[0], r1, lr
 ; CHECK-FP-NEXT:    adcs r0, r2
-; CHECK-FP-NEXT:    vmov.32 q0[1], r0
-; CHECK-FP-NEXT:    vmov.32 q0[2], lr
-; CHECK-FP-NEXT:    vmov.32 q0[3], r12
+; CHECK-FP-NEXT:    vmov q0[3], q0[1], r0, r12
 ; CHECK-FP-NEXT:    vmov r0, r1, d0
 ; CHECK-FP-NEXT:    vmov r2, r3, d1
 ; CHECK-FP-NEXT:    pop {r7, pc}
 entry:
   %sum = add <2 x i64> %lhs, %rhs
   ret <2 x i64> %sum
 }
 

llvm/test/CodeGen/Thumb2/mve-vabdus.ll

; CHECK-NEXT:    bx lr
   %result = trunc <16 x i16> %s to <16 x i8>
   ret <16 x i8> %result
 }
 
 define arm_aapcs_vfpcc <8 x i16> @vabd_s16(<8 x i16> %src1, <8 x i16> %src2) {
 ; CHECK-LABEL: vabd_s16:
 ; CHECK:       @ %bb.0:
 ; CHECK-NEXT:    vmov.u16 r0, q1[0]
-; CHECK-NEXT:    vmov.32 q2[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q1[2]
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q1[1]
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[2]
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[3]
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q1[3]
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[3]
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[2]
 ; CHECK-NEXT:    vmovlb.s16 q2, q2
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
+; CHECK-NEXT:    vmov.u16 r0, q0[1]
+; CHECK-NEXT:    vmov.u16 r1, q0[3]
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
+; CHECK-NEXT:    vmov.u16 r1, q1[6]
 ; CHECK-NEXT:    vmovlb.s16 q3, q3
 ; CHECK-NEXT:    vsub.i32 q2, q3, q2
 ; CHECK-NEXT:    vabs.s32 q3, q2
 ; CHECK-NEXT:    vmov r0, s12
 ; CHECK-NEXT:    vmov.16 q2[0], r0
 ; CHECK-NEXT:    vmov r0, s13
 ; CHECK-NEXT:    vmov.16 q2[1], r0
 ; CHECK-NEXT:    vmov r0, s14
 ; CHECK-NEXT:    vmov.16 q2[2], r0
 ; CHECK-NEXT:    vmov r0, s15
 ; CHECK-NEXT:    vmov.16 q2[3], r0
 ; CHECK-NEXT:    vmov.u16 r0, q1[4]
-; CHECK-NEXT:    vmov.32 q3[0], r0
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q1[5]
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[6]
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[7]
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q1[7]
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[4]
+; CHECK-NEXT:    vmov.u16 r1, q0[6]
 ; CHECK-NEXT:    vmovlb.s16 q1, q3
-; CHECK-NEXT:    vmov.32 q3[0], r0
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[5]
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[6]
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[7]
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[7]
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
 ; CHECK-NEXT:    vmovlb.s16 q0, q3
 ; CHECK-NEXT:    vsub.i32 q0, q0, q1
 ; CHECK-NEXT:    vabs.s32 q0, q0
 ; CHECK-NEXT:    vmov r0, s0
 ; CHECK-NEXT:    vmov.16 q2[4], r0
 ; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    vmov.16 q2[5], r0
 ; CHECK-NEXT:    vmov r0, s2
; CHECK-NEXT:    bx lr
   %s = select <8 x i1> %c, <8 x i32> %add1, <8 x i32> %add2
   %result = trunc <8 x i32> %s to <8 x i16>
   ret <8 x i16> %result
 }
 
 define arm_aapcs_vfpcc <4 x i32> @vabd_s32(<4 x i32> %src1, <4 x i32> %src2) {
 ; CHECK-LABEL: vabd_s32:
 ; CHECK:       @ %bb.0:
+; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
+; CHECK-NEXT:    vpush {d8, d9, d10, d11}
 ; CHECK-NEXT:    vmov.f32 s8, s0
 ; CHECK-NEXT:    vmov.f32 s12, s4
 ; CHECK-NEXT:    vmov.f32 s10, s1
+; CHECK-NEXT:    vmov r0, s8
 ; CHECK-NEXT:    vmov.f32 s14, s5
-; CHECK-NEXT:    vmov r3, s12
-; CHECK-NEXT:    vmov r0, s10
-; CHECK-NEXT:    vmov r2, s14
-; CHECK-NEXT:    vmov.f32 s12, s2
-; CHECK-NEXT:    vmov.f32 s14, s3
-; CHECK-NEXT:    vmov.f32 s0, s6
-; CHECK-NEXT:    vmov.f32 s2, s7
+; CHECK-NEXT:    vmov r2, s12
+; CHECK-NEXT:    vmov.f32 s16, s2
+; CHECK-NEXT:    vmov.f32 s20, s6
+; CHECK-NEXT:    vmov.f32 s18, s3
+; CHECK-NEXT:    vmov.f32 s22, s7
+; CHECK-NEXT:    vmov r3, s20
 ; CHECK-NEXT:    asrs r1, r0, #31
 ; CHECK-NEXT:    subs r0, r0, r2
 ; CHECK-NEXT:    sbc.w r1, r1, r2, asr #31
 ; CHECK-NEXT:    add.w r0, r0, r1, asr #31
 ; CHECK-NEXT:    eor.w r0, r0, r1, asr #31
-; CHECK-NEXT:    vmov r1, s8
+; CHECK-NEXT:    vmov r1, s16
 ; CHECK-NEXT:    asrs r2, r1, #31
 ; CHECK-NEXT:    subs r1, r1, r3
 ; CHECK-NEXT:    sbc.w r2, r2, r3, asr #31
+; CHECK-NEXT:    vmov r3, s22
 ; CHECK-NEXT:    add.w r1, r1, r2, asr #31
 ; CHECK-NEXT:    eor.w r1, r1, r2, asr #31
-; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    vmov.32 q2[0], r1
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s12
-; CHECK-NEXT:    asrs r1, r0, #31
-; CHECK-NEXT:    subs r0, r0, r2
-; CHECK-NEXT:    sbc.w r1, r1, r2, asr #31
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    add.w r0, r0, r1, asr #31
-; CHECK-NEXT:    eor.w r0, r0, r1, asr #31
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov r0, s14
+; CHECK-NEXT:    vmov r2, s14
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov r0, s10
 ; CHECK-NEXT:    asrs r1, r0, #31
 ; CHECK-NEXT:    subs r0, r0, r2
 ; CHECK-NEXT:    sbc.w r1, r1, r2, asr #31
 ; CHECK-NEXT:    add.w r0, r0, r1, asr #31
 ; CHECK-NEXT:    eor.w r0, r0, r1, asr #31
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vmov q0, q2
+; CHECK-NEXT:    vmov r1, s18
+; CHECK-NEXT:    asrs r2, r1, #31
+; CHECK-NEXT:    subs r1, r1, r3
+; CHECK-NEXT:    sbc.w r2, r2, r3, asr #31
+; CHECK-NEXT:    add.w r1, r1, r2, asr #31
+; CHECK-NEXT:    eor.w r1, r1, r2, asr #31
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
+; CHECK-NEXT:    vpop {d8, d9, d10, d11}
 ; CHECK-NEXT:    bx lr
   %sextsrc1 = sext <4 x i32> %src1 to <4 x i64>
   %sextsrc2 = sext <4 x i32> %src2 to <4 x i64>
   %add1 = sub <4 x i64> %sextsrc1, %sextsrc2
   %add2 = sub <4 x i64> zeroinitializer, %add1
   %c = icmp sge <4 x i64> %add1, zeroinitializer
   %s = select <4 x i1> %c, <4 x i64> %add1, <4 x i64> %add2
   %result = trunc <4 x i64> %s to <4 x i32>
; CHECK-NEXT:    bx lr
   %result = trunc <16 x i16> %s to <16 x i8>
   ret <16 x i8> %result
 }
 
 define arm_aapcs_vfpcc <8 x i16> @vabd_u16(<8 x i16> %src1, <8 x i16> %src2) {
 ; CHECK-LABEL: vabd_u16:
 ; CHECK:       @ %bb.0:
 ; CHECK-NEXT:    vmov.u16 r0, q1[0]
-; CHECK-NEXT:    vmov.32 q2[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q1[2]
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q1[1]
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[2]
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[3]
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q1[3]
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[3]
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[2]
 ; CHECK-NEXT:    vmovlb.u16 q2, q2
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
+; CHECK-NEXT:    vmov.u16 r0, q0[1]
+; CHECK-NEXT:    vmov.u16 r1, q0[3]
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
+; CHECK-NEXT:    vmov.u16 r1, q1[6]
 ; CHECK-NEXT:    vmovlb.u16 q3, q3
 ; CHECK-NEXT:    vsub.i32 q2, q3, q2
 ; CHECK-NEXT:    vabs.s32 q3, q2
 ; CHECK-NEXT:    vmov r0, s12
 ; CHECK-NEXT:    vmov.16 q2[0], r0
 ; CHECK-NEXT:    vmov r0, s13
 ; CHECK-NEXT:    vmov.16 q2[1], r0
 ; CHECK-NEXT:    vmov r0, s14
 ; CHECK-NEXT:    vmov.16 q2[2], r0
 ; CHECK-NEXT:    vmov r0, s15
 ; CHECK-NEXT:    vmov.16 q2[3], r0
 ; CHECK-NEXT:    vmov.u16 r0, q1[4]
-; CHECK-NEXT:    vmov.32 q3[0], r0
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q1[5]
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[6]
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[7]
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q1[7]
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[4]
+; CHECK-NEXT:    vmov.u16 r1, q0[6]
 ; CHECK-NEXT:    vmovlb.u16 q1, q3
-; CHECK-NEXT:    vmov.32 q3[0], r0
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[5]
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[6]
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[7]
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[7]
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
 ; CHECK-NEXT:    vmovlb.u16 q0, q3
 ; CHECK-NEXT:    vsub.i32 q0, q0, q1
 ; CHECK-NEXT:    vabs.s32 q0, q0
 ; CHECK-NEXT:    vmov r0, s0
 ; CHECK-NEXT:    vmov.16 q2[4], r0
 ; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    vmov.16 q2[5], r0
 ; CHECK-NEXT:    vmov r0, s2
; CHECK-NEXT:    bx lr
   %s = select <8 x i1> %c, <8 x i32> %add1, <8 x i32> %add2
   %result = trunc <8 x i32> %s to <8 x i16>
   ret <8 x i16> %result
 }
 
 define arm_aapcs_vfpcc <4 x i32> @vabd_u32(<4 x i32> %src1, <4 x i32> %src2) {
 ; CHECK-LABEL: vabd_u32:
 ; CHECK:       @ %bb.0:
-; CHECK-NEXT:    .vsave {d8, d9}
-; CHECK-NEXT:    vpush {d8, d9}
+; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
+; CHECK-NEXT:    vpush {d8, d9, d10, d11}
 ; CHECK-NEXT:    vmov.f32 s8, s4
-; CHECK-NEXT:    vmov.i64 q3, #0xffffffff
-; CHECK-NEXT:    vmov.f32 s16, s0
+; CHECK-NEXT:    vmov.i64 q4, #0xffffffff
+; CHECK-NEXT:    vmov.f32 s12, s0
 ; CHECK-NEXT:    vmov.f32 s10, s5
-; CHECK-NEXT:    vmov.f32 s18, s1
-; CHECK-NEXT:    vand q2, q2, q3
-; CHECK-NEXT:    vand q4, q4, q3
-; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    vmov r3, s18
-; CHECK-NEXT:    vmov r0, s11
-; CHECK-NEXT:    vmov r1, s19
+; CHECK-NEXT:    vmov.f32 s14, s1
+; CHECK-NEXT:    vand q2, q2, q4
+; CHECK-NEXT:    vand q3, q3, q4
+; CHECK-NEXT:    vmov r2, s8
+; CHECK-NEXT:    vmov r3, s12
+; CHECK-NEXT:    vmov r0, s9
+; CHECK-NEXT:    vmov r1, s13
+; CHECK-NEXT:    vmov.f32 s20, s6
+; CHECK-NEXT:    vmov.f32 s22, s7
+; CHECK-NEXT:    vand q1, q5, q4
+; CHECK-NEXT:    vmov.f32 s20, s2
+; CHECK-NEXT:    vmov.f32 s22, s3
+; CHECK-NEXT:    vand q4, q5, q4
 ; CHECK-NEXT:    subs r2, r3, r2
-; CHECK-NEXT:    vmov r3, s8
+; CHECK-NEXT:    vmov r3, s4
 ; CHECK-NEXT:    sbc.w r0, r1, r0
 ; CHECK-NEXT:    add.w r1, r2, r0, asr #31
 ; CHECK-NEXT:    vmov r2, s17
 ; CHECK-NEXT:    eor.w r12, r1, r0, asr #31
 ; CHECK-NEXT:    vmov r0, s16
-; CHECK-NEXT:    vmov r1, s9
-; CHECK-NEXT:    vmov.f32 s16, s6
-; CHECK-NEXT:    vmov.f32 s18, s7
-; CHECK-NEXT:    vand q1, q4, q3
-; CHECK-NEXT:    vmov.f32 s16, s2
-; CHECK-NEXT:    vmov.f32 s18, s3
-; CHECK-NEXT:    vand q0, q4, q3
+; CHECK-NEXT:    vmov r1, s5
 ; CHECK-NEXT:    subs r0, r0, r3
-; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    vmov r3, s14
 ; CHECK-NEXT:    sbc.w r1, r2, r1
-; CHECK-NEXT:    vmov r2, s4
+; CHECK-NEXT:    vmov r2, s10
 ; CHECK-NEXT:    add.w r0, r0, r1, asr #31
 ; CHECK-NEXT:    eor.w r0, r0, r1, asr #31
-; CHECK-NEXT:    vmov r1, s1
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov r0, s5
-; CHECK-NEXT:    vmov.32 q2[1], r12
-; CHECK-NEXT:    subs r2, r3, r2
-; CHECK-NEXT:    vmov r3, s2
-; CHECK-NEXT:    sbc.w r0, r1, r0
-; CHECK-NEXT:    add.w r1, r2, r0, asr #31
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    eor.w r0, r1, r0, asr #31
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov r0, s7
+; CHECK-NEXT:    vmov r1, s15
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r12
+; CHECK-NEXT:    vmov r0, s11
 ; CHECK-NEXT:    subs r2, r3, r2
+; CHECK-NEXT:    vmov r3, s6
 ; CHECK-NEXT:    sbc.w r0, r1, r0
 ; CHECK-NEXT:    add.w r1, r2, r0, asr #31
-; CHECK-NEXT:    eor.w r0, r1, r0, asr #31
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vmov q0, q2
-; CHECK-NEXT:    vpop {d8, d9}
+; CHECK-NEXT:    vmov r2, s19
+; CHECK-NEXT:    eor.w r12, r1, r0, asr #31
+; CHECK-NEXT:    vmov r0, s18
+; CHECK-NEXT:    vmov r1, s7
+; CHECK-NEXT:    subs r0, r0, r3
+; CHECK-NEXT:    sbc.w r1, r2, r1
+; CHECK-NEXT:    add.w r0, r0, r1, asr #31
+; CHECK-NEXT:    eor.w r0, r0, r1, asr #31
+; CHECK-NEXT:    vmov q0[3], q0[1], r0, r12
+; CHECK-NEXT:    vpop {d8, d9, d10, d11}
 ; CHECK-NEXT:    bx lr
   %zextsrc1 = zext <4 x i32> %src1 to <4 x i64>
   %zextsrc2 = zext <4 x i32> %src2 to <4 x i64>
   %add1 = sub <4 x i64> %zextsrc1, %zextsrc2
   %add2 = sub <4 x i64> zeroinitializer, %add1
   %c = icmp sge <4 x i64> %add1, zeroinitializer
   %s = select <4 x i1> %c, <4 x i64> %add1, <4 x i64> %add2
   %result = trunc <4 x i64> %s to <4 x i32>

 
 for.cond.cleanup:                                 ; preds = %vector.body
   ret void
 }
 
 define void @vabd_loop_s32(i32* nocapture readonly %x, i32* nocapture readonly %y, i32* noalias nocapture %z, i32 %n) {
 ; CHECK-LABEL: vabd_loop_s32:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, lr}
-; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, lr}
-; CHECK-NEXT:    .pad #4
-; CHECK-NEXT:    sub sp, #4
+; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, lr}
+; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, lr}
 ; CHECK-NEXT:    .vsave {d8, d9}
 ; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    mov.w lr, #256
 ; CHECK-NEXT:    mov.w r12, #1
 ; CHECK-NEXT:    dls lr, lr
 ; CHECK-NEXT:    vmov.i32 q0, #0x0
 ; CHECK-NEXT:  .LBB8_1: @ %vector.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1

 ; CHECK-NEXT:    vmov r5, s14
 ; CHECK-NEXT:    vmov.f32 s12, s6
 ; CHECK-NEXT:    vmov.f32 s14, s7
 ; CHECK-NEXT:    vmov.f32 s4, s10
 ; CHECK-NEXT:    vmov.f32 s6, s11
 ; CHECK-NEXT:    vmov r3, s4
 ; CHECK-NEXT:    subs.w r9, r5, r7
 ; CHECK-NEXT:    asr.w r6, r5, #31
-; CHECK-NEXT:    vmov r5, s6
 ; CHECK-NEXT:    sbc.w r6, r6, r7, asr #31
-; CHECK-NEXT:    vmov.32 q1[0], r8
-; CHECK-NEXT:    vmov.32 q1[1], r9
 ; CHECK-NEXT:    and.w r6, r12, r6, asr #31
 ; CHECK-NEXT:    rsbs r6, r6, #0
 ; CHECK-NEXT:    bfi r4, r6, #4, #4
 ; CHECK-NEXT:    vmov r6, s12
-; CHECK-NEXT:    asrs r7, r6, #31
-; CHECK-NEXT:    subs r6, r6, r3
+; CHECK-NEXT:    subs.w r10, r6, r3
+; CHECK-NEXT:    asr.w r7, r6, #31
 ; CHECK-NEXT:    sbc.w r3, r7, r3, asr #31
-; CHECK-NEXT:    vmov.32 q1[2], r6
+; CHECK-NEXT:    vmov r7, s14
+; CHECK-NEXT:    vmov r6, s6
 ; CHECK-NEXT:    asrs r3, r3, #31
-; CHECK-NEXT:    vmov.32 q2[0], r3
-; CHECK-NEXT:    vmov r3, s14
-; CHECK-NEXT:    asrs r7, r3, #31
-; CHECK-NEXT:    subs r3, r3, r5
-; CHECK-NEXT:    sbc.w r5, r7, r5, asr #31
-; CHECK-NEXT:    vmov.32 q1[3], r3
-; CHECK-NEXT:    asrs r7, r5, #31
-; CHECK-NEXT:    and.w r5, r12, r5, asr #31
-; CHECK-NEXT:    vmov.32 q2[2], r7
-; CHECK-NEXT:    rsbs r5, r5, #0
-; CHECK-NEXT:    vmov r7, s8
-; CHECK-NEXT:    and r7, r7, #1
-; CHECK-NEXT:    rsbs r7, r7, #0
-; CHECK-NEXT:    bfi r4, r7, #8, #4
-; CHECK-NEXT:    bfi r4, r5, #12, #4
+; CHECK-NEXT:    asrs r5, r7, #31
+; CHECK-NEXT:    subs r7, r7, r6
+; CHECK-NEXT:    sbc.w r5, r5, r6, asr #31
+; CHECK-NEXT:    asrs r6, r5, #31
+; CHECK-NEXT:    vmov q1[2], q1[0], r6, r3
+; CHECK-NEXT:    vmov r3, s4
+; CHECK-NEXT:    vmov q1[2], q1[0], r10, r8
+; CHECK-NEXT:    vmov q1[3], q1[1], r7, r9
+; CHECK-NEXT:    and r3, r3, #1
+; CHECK-NEXT:    rsbs r3, r3, #0
+; CHECK-NEXT:    bfi r4, r3, #8, #4
+; CHECK-NEXT:    and.w r3, r12, r5, asr #31
+; CHECK-NEXT:    rsbs r3, r3, #0
+; CHECK-NEXT:    bfi r4, r3, #12, #4
 ; CHECK-NEXT:    vmsr p0, r4
 ; CHECK-NEXT:    vpst
 ; CHECK-NEXT:    vsubt.i32 q1, q0, q1
 ; CHECK-NEXT:    vstrb.8 q1, [r2], #16
 ; CHECK-NEXT:    le lr, .LBB8_1
 ; CHECK-NEXT:  @ %bb.2: @ %for.cond.cleanup
 ; CHECK-NEXT:    vpop {d8, d9}
-; CHECK-NEXT:    add sp, #4
-; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, pc}
+; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, pc}
 entry:
   br label %vector.body
 
 vector.body:                                      ; preds = %vector.body, %entry
   %index = phi i32 [ 0, %entry ], [ %index.next, %vector.body ]
   %0 = getelementptr inbounds i32, i32* %x, i32 %index
   %1 = bitcast i32* %0 to <4 x i32>*
   %wide.load = load <4 x i32>, <4 x i32>* %1, align 4

 
 for.cond.cleanup:                                 ; preds = %vector.body
   ret void
 }
 
 define void @vabd_loop_u32(i32* nocapture readonly %x, i32* nocapture readonly %y, i32* noalias nocapture %z, i32 %n) {
 ; CHECK-LABEL: vabd_loop_u32:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, lr}
-; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, lr}
+; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, r11, lr}
+; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, r11, lr}
+; CHECK-NEXT:    .pad #4
+; CHECK-NEXT:    sub sp, #4
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11}
 ; CHECK-NEXT:    mov.w lr, #256
 ; CHECK-NEXT:    vmov.i64 q0, #0xffffffff
 ; CHECK-NEXT:    dls lr, lr
 ; CHECK-NEXT:    mov.w r12, #1
 ; CHECK-NEXT:    vmov.i32 q1, #0x0
 ; CHECK-NEXT:  .LBB11_1: @ %vector.body

 ; CHECK-NEXT:    sbc.w r3, r7, r6
 ; CHECK-NEXT:    vmov r7, s8
 ; CHECK-NEXT:    vmov r6, s13
 ; CHECK-NEXT:    and.w r3, r12, r3, asr #31
 ; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    bfi r4, r3, #4, #4
 ; CHECK-NEXT:    vmov r3, s9
 ; CHECK-NEXT:    subs.w r10, r5, r7
-; CHECK-NEXT:    vmov r7, s10
-; CHECK-NEXT:    vmov r5, s14
+; CHECK-NEXT:    vmov r5, s10
+; CHECK-NEXT:    vmov r7, s15
 ; CHECK-NEXT:    sbc.w r3, r6, r3
-; CHECK-NEXT:    vmov r6, s15
-; CHECK-NEXT:    asrs r3, r3, #31
-; CHECK-NEXT:    vmov.32 q4[0], r3
-; CHECK-NEXT:    vmov r3, s11
-; CHECK-NEXT:    vmov.32 q2[0], r8
-; CHECK-NEXT:    vmov.32 q2[1], r9
-; CHECK-NEXT:    vmov.32 q2[2], r10
-; CHECK-NEXT:    subs r5, r5, r7
-; CHECK-NEXT:    vmov.32 q2[3], r5
-; CHECK-NEXT:    sbc.w r3, r6, r3
-; CHECK-NEXT:    asrs r6, r3, #31
-; CHECK-NEXT:    and.w r3, r12, r3, asr #31
-; CHECK-NEXT:    vmov.32 q4[2], r6
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov r6, s16
+; CHECK-NEXT:    vmov r6, s11
+; CHECK-NEXT:    asr.w r11, r3, #31
+; CHECK-NEXT:    vmov r3, s14
+; CHECK-NEXT:    subs r3, r3, r5
+; CHECK-NEXT:    sbc.w r5, r7, r6
+; CHECK-NEXT:    asrs r6, r5, #31
+; CHECK-NEXT:    and.w r5, r12, r5, asr #31
+; CHECK-NEXT:    vmov q2[2], q2[0], r6, r11
+; CHECK-NEXT:    rsbs r5, r5, #0
+; CHECK-NEXT:    vmov r6, s8
+; CHECK-NEXT:    vmov q2[2], q2[0], r10, r8
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r9
 ; CHECK-NEXT:    and r6, r6, #1
 ; CHECK-NEXT:    rsbs r6, r6, #0
 ; CHECK-NEXT:    bfi r4, r6, #8, #4
-; CHECK-NEXT:    bfi r4, r3, #12, #4
+; CHECK-NEXT:    bfi r4, r5, #12, #4
 ; CHECK-NEXT:    vmsr p0, r4
 ; CHECK-NEXT:    vpst
 ; CHECK-NEXT:    vsubt.i32 q2, q1, q2
 ; CHECK-NEXT:    vstrb.8 q2, [r2], #16
 ; CHECK-NEXT:    le lr, .LBB11_1
 ; CHECK-NEXT:  @ %bb.2: @ %for.cond.cleanup
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, pc}
+; CHECK-NEXT:    add sp, #4
+; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 entry:
   br label %vector.body
 
 vector.body:                                      ; preds = %vector.body, %entry
   %index = phi i32 [ 0, %entry ], [ %index.next, %vector.body ]
   %0 = getelementptr inbounds i32, i32* %x, i32 %index
   %1 = bitcast i32* %0 to <4 x i32>*
   %wide.load = load <4 x i32>, <4 x i32>* %1, align 4

llvm/test/CodeGen/Thumb2/mve-vcmp.ll

entry:
   %s = select <16 x i1> %c, <16 x i8> %a, <16 x i8> %b
   ret <16 x i8> %s
 }
 
 
 define arm_aapcs_vfpcc <2 x i64> @vcmp_eq_v2i64(<2 x i64> %src, <2 x i64> %srcb, <2 x i64> %a, <2 x i64> %b) {
 ; CHECK-LABEL: vcmp_eq_v2i64:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .vsave {d8, d9}
-; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    vmov r0, s5
 ; CHECK-NEXT:    vmov r1, s1
 ; CHECK-NEXT:    vmov r2, s0
+; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    eors r0, r1
 ; CHECK-NEXT:    vmov r1, s4
 ; CHECK-NEXT:    eors r1, r2
-; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    vmov r2, s3
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s3
+; CHECK-NEXT:    vmov r1, s7
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q4[0], r0
-; CHECK-NEXT:    vmov.32 q4[1], r0
-; CHECK-NEXT:    vmov r0, s7
-; CHECK-NEXT:    eors r0, r1
-; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    eors r1, r2
-; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q4[2], r0
-; CHECK-NEXT:    vmov.32 q4[3], r0
-; CHECK-NEXT:    vbic q0, q3, q4
-; CHECK-NEXT:    vand q1, q2, q4
-; CHECK-NEXT:    vorr q0, q1, q0
-; CHECK-NEXT:    vpop {d8, d9}
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    eors r2, r3
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
+; CHECK-NEXT:    vbic q1, q3, q0
+; CHECK-NEXT:    vand q0, q2, q0
+; CHECK-NEXT:    vorr q0, q0, q1
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp eq <2 x i64> %src, %srcb
   %s = select <2 x i1> %c, <2 x i64> %a, <2 x i64> %b
   ret <2 x i64> %s
 }
 
 define arm_aapcs_vfpcc <2 x i32> @vcmp_eq_v2i32(<2 x i64> %src, <2 x i64> %srcb, <2 x i32> %a, <2 x i32> %b) {
 ; CHECK-LABEL: vcmp_eq_v2i32:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .vsave {d8, d9}
-; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    vmov r0, s5
 ; CHECK-NEXT:    vmov r1, s1
 ; CHECK-NEXT:    vmov r2, s0
+; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    eors r0, r1
 ; CHECK-NEXT:    vmov r1, s4
 ; CHECK-NEXT:    eors r1, r2
-; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    vmov r2, s3
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s3
+; CHECK-NEXT:    vmov r1, s7
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q4[0], r0
-; CHECK-NEXT:    vmov.32 q4[1], r0
-; CHECK-NEXT:    vmov r0, s7
-; CHECK-NEXT:    eors r0, r1
-; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    eors r1, r2
-; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q4[2], r0
-; CHECK-NEXT:    vmov.32 q4[3], r0
-; CHECK-NEXT:    vbic q0, q3, q4
-; CHECK-NEXT:    vand q1, q2, q4
-; CHECK-NEXT:    vorr q0, q1, q0
-; CHECK-NEXT:    vpop {d8, d9}
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    eors r2, r3
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
+; CHECK-NEXT:    vbic q1, q3, q0
+; CHECK-NEXT:    vand q0, q2, q0
+; CHECK-NEXT:    vorr q0, q0, q1
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp eq <2 x i64> %src, %srcb
   %s = select <2 x i1> %c, <2 x i32> %a, <2 x i32> %b
   ret <2 x i32> %s
 }
 
 define arm_aapcs_vfpcc <2 x i32> @vcmp_multi_v2i32(<2 x i64> %a, <2 x i32> %b, <2 x i32> %c) {
 ; CHECK-LABEL: vcmp_multi_v2i32:
 ; CHECK:       @ %bb.0:
-; CHECK-NEXT:    .save {r7, lr}
-; CHECK-NEXT:    push {r7, lr}
-; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-NEXT:    vpush {d8, d9, d10, d11}
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .vsave {d8, d9}
+; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    movs r3, #0
 ; CHECK-NEXT:    vmov r1, s0
-; CHECK-NEXT:    vmov r2, s8
-; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov r0, s3
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    orrs r0, r1
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vbic q0, q2, q3
-; CHECK-NEXT:    vmov lr, s0
-; CHECK-NEXT:    subs.w r1, lr, r2
-; CHECK-NEXT:    asr.w r12, lr, #31
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    vmov r2, s8
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
+; CHECK-NEXT:    vbic q0, q2, q0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    subs r1, r0, r2
+; CHECK-NEXT:    asr.w r12, r0, #31
 ; CHECK-NEXT:    sbcs.w r1, r12, r2, asr #31
 ; CHECK-NEXT:    mov.w r1, #0
-; CHECK-NEXT:    vmov r2, s10
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r1, #1
 ; CHECK-NEXT:    cmp r1, #0
-; CHECK-NEXT:    csetm r1, ne
-; CHECK-NEXT:    vmov.32 q3[0], r1
-; CHECK-NEXT:    vmov.32 q3[1], r1
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    subs r0, r1, r2
-; CHECK-NEXT:    asr.w r12, r1, #31
-; CHECK-NEXT:    sbcs.w r0, r12, r2, asr #31
+; CHECK-NEXT:    vmov r1, s10
+; CHECK-NEXT:    csetm lr, ne
+; CHECK-NEXT:    asr.w r12, r2, #31
+; CHECK-NEXT:    subs r4, r2, r1
+; CHECK-NEXT:    sbcs.w r1, r12, r1, asr #31
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r3, #1
 ; CHECK-NEXT:    cmp r3, #0
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    cmp.w lr, #0
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    cset r0, ne
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    cmp r1, #0
-; CHECK-NEXT:    vmov.32 q4[0], r0
-; CHECK-NEXT:    vmov.32 q4[1], r0
-; CHECK-NEXT:    cset r0, ne
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q4[2], r0
-; CHECK-NEXT:    vmov.32 q4[3], r0
-; CHECK-NEXT:    vmov r0, s4
+; CHECK-NEXT:    csetm r1, ne
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    cset r0, ne
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, lr
 ; CHECK-NEXT:    tst.w r0, #1
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, lr
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q5[0], r0
-; CHECK-NEXT:    vmov.32 q5[1], r0
-; CHECK-NEXT:    vmov r0, s6
+; CHECK-NEXT:    cmp r2, #0
+; CHECK-NEXT:    cset r1, ne
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q4[2], q4[0], r1, r0
+; CHECK-NEXT:    vmov q4[3], q4[1], r1, r0
+; CHECK-NEXT:    vmov r0, s4
+; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    cset r0, ne
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q5[2], r0
-; CHECK-NEXT:    vmov.32 q5[3], r0
-; CHECK-NEXT:    vand q1, q5, q4
+; CHECK-NEXT:    cmp r1, #0
+; CHECK-NEXT:    cset r1, ne
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
+; CHECK-NEXT:    vand q1, q1, q4
 ; CHECK-NEXT:    vand q1, q3, q1
 ; CHECK-NEXT:    vbic q0, q0, q1
 ; CHECK-NEXT:    vand q1, q2, q1
 ; CHECK-NEXT:    vorr q0, q1, q0
-; CHECK-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-NEXT:    pop {r7, pc}
+; CHECK-NEXT:    vpop {d8, d9}
+; CHECK-NEXT:    pop {r4, pc}
   %a4 = icmp eq <2 x i64> %a, zeroinitializer
   %a5 = select <2 x i1> %a4, <2 x i32> zeroinitializer, <2 x i32> %c
   %a6 = icmp ne <2 x i32> %b, zeroinitializer
   %a7 = icmp slt <2 x i32> %a5, %c
   %a8 = icmp ne <2 x i32> %a5, zeroinitializer
   %a9 = and <2 x i1> %a6, %a8
   %a10 = and <2 x i1> %a7, %a9
   %a11 = select <2 x i1> %a10, <2 x i32> %c, <2 x i32> %a5
   ret <2 x i32> %a11
 }

llvm/test/CodeGen/Thumb2/mve-vcmpr.ll

 define arm_aapcs_vfpcc <2 x i64> @vcmp_eq_v2i64(<2 x i64> %src, i64 %src2, <2 x i64> %a, <2 x i64> %b) {
 ; CHECK-LABEL: vcmp_eq_v2i64:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    eors r2, r1
 ; CHECK-NEXT:    eors r3, r0
 ; CHECK-NEXT:    orrs r2, r3
+; CHECK-NEXT:    vmov r3, s3
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov r2, s3
-; CHECK-NEXT:    eors r1, r2
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    eors r0, r2
+; CHECK-NEXT:    eors r1, r3
+; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    eors r0, r3
 ; CHECK-NEXT:    orrs r0, r1
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vbic q0, q2, q3
-; CHECK-NEXT:    vand q1, q1, q3
-; CHECK-NEXT:    vorr q0, q1, q0
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r0, r2
+; CHECK-NEXT:    vbic q2, q2, q0
+; CHECK-NEXT:    vand q0, q1, q0
+; CHECK-NEXT:    vorr q0, q0, q2
 ; CHECK-NEXT:    bx lr
 entry:
   %i = insertelement <2 x i64> undef, i64 %src2, i32 0
   %sp = shufflevector <2 x i64> %i, <2 x i64> undef, <2 x i32> zeroinitializer
   %c = icmp eq <2 x i64> %src, %sp
   %s = select <2 x i1> %c, <2 x i64> %a, <2 x i64> %b
   ret <2 x i64> %s
 }
 
 define arm_aapcs_vfpcc <2 x i32> @vcmp_eq_v2i32(<2 x i64> %src, i64 %src2, <2 x i32> %a, <2 x i32> %b) {
 ; CHECK-LABEL: vcmp_eq_v2i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    eors r2, r1
 ; CHECK-NEXT:    eors r3, r0
 ; CHECK-NEXT:    orrs r2, r3
+; CHECK-NEXT:    vmov r3, s3
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov r2, s3
-; CHECK-NEXT:    eors r1, r2
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    eors r0, r2
+; CHECK-NEXT:    eors r1, r3
+; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    eors r0, r3
 ; CHECK-NEXT:    orrs r0, r1
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vbic q0, q2, q3
-; CHECK-NEXT:    vand q1, q1, q3
-; CHECK-NEXT:    vorr q0, q1, q0
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r0, r2
+; CHECK-NEXT:    vbic q2, q2, q0
+; CHECK-NEXT:    vand q0, q1, q0
+; CHECK-NEXT:    vorr q0, q0, q2
 ; CHECK-NEXT:    bx lr
 entry:
   %i = insertelement <2 x i64> undef, i64 %src2, i32 0
   %sp = shufflevector <2 x i64> %i, <2 x i64> undef, <2 x i32> zeroinitializer
   %c = icmp eq <2 x i64> %src, %sp
   %s = select <2 x i1> %c, <2 x i32> %a, <2 x i32> %b
   ret <2 x i32> %s
 }
 
 define arm_aapcs_vfpcc <2 x i32> @vcmp_multi_v2i32(<2 x i64> %a, <2 x i32> %b, <2 x i32> %c) {
 ; CHECK-LABEL: vcmp_multi_v2i32:
 ; CHECK:       @ %bb.0:
-; CHECK-NEXT:    .save {r7, lr}
-; CHECK-NEXT:    push {r7, lr}
-; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-NEXT:    vpush {d8, d9, d10, d11}
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .vsave {d8, d9}
+; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    movs r3, #0
 ; CHECK-NEXT:    vmov r1, s0
-; CHECK-NEXT:    vmov r2, s8
-; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov r0, s3
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    orrs r0, r1
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vbic q0, q2, q3
-; CHECK-NEXT:    vmov lr, s0
-; CHECK-NEXT:    subs.w r1, lr, r2
-; CHECK-NEXT:    asr.w r12, lr, #31
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    vmov r2, s8
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
+; CHECK-NEXT:    vbic q0, q2, q0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    subs r1, r0, r2
+; CHECK-NEXT:    asr.w r12, r0, #31
 ; CHECK-NEXT:    sbcs.w r1, r12, r2, asr #31
 ; CHECK-NEXT:    mov.w r1, #0
-; CHECK-NEXT:    vmov r2, s10
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r1, #1
 ; CHECK-NEXT:    cmp r1, #0
-; CHECK-NEXT:    csetm r1, ne
-; CHECK-NEXT:    vmov.32 q3[0], r1
-; CHECK-NEXT:    vmov.32 q3[1], r1
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    subs r0, r1, r2
-; CHECK-NEXT:    asr.w r12, r1, #31
-; CHECK-NEXT:    sbcs.w r0, r12, r2, asr #31
+; CHECK-NEXT:    vmov r1, s10
+; CHECK-NEXT:    csetm lr, ne
+; CHECK-NEXT:    asr.w r12, r2, #31
+; CHECK-NEXT:    subs r4, r2, r1
+; CHECK-NEXT:    sbcs.w r1, r12, r1, asr #31
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r3, #1
 ; CHECK-NEXT:    cmp r3, #0
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    cmp.w lr, #0
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    cset r0, ne
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    cmp r1, #0
-; CHECK-NEXT:    vmov.32 q4[0], r0
-; CHECK-NEXT:    vmov.32 q4[1], r0
-; CHECK-NEXT:    cset r0, ne
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q4[2], r0
-; CHECK-NEXT:    vmov.32 q4[3], r0
-; CHECK-NEXT:    vmov r0, s4
+; CHECK-NEXT:    csetm r1, ne
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    cset r0, ne
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, lr
 ; CHECK-NEXT:    tst.w r0, #1
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, lr
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q5[0], r0
-; CHECK-NEXT:    vmov.32 q5[1], r0
-; CHECK-NEXT:    vmov r0, s6
+; CHECK-NEXT:    cmp r2, #0
+; CHECK-NEXT:    cset r1, ne
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q4[2], q4[0], r1, r0
+; CHECK-NEXT:    vmov q4[3], q4[1], r1, r0
+; CHECK-NEXT:    vmov r0, s4
+; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    cset r0, ne
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q5[2], r0
-; CHECK-NEXT:    vmov.32 q5[3], r0
-; CHECK-NEXT:    vand q1, q5, q4
+; CHECK-NEXT:    cmp r1, #0
+; CHECK-NEXT:    cset r1, ne
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
+; CHECK-NEXT:    vand q1, q1, q4
 ; CHECK-NEXT:    vand q1, q3, q1
 ; CHECK-NEXT:    vbic q0, q0, q1
 ; CHECK-NEXT:    vand q1, q2, q1
 ; CHECK-NEXT:    vorr q0, q1, q0
-; CHECK-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-NEXT:    pop {r7, pc}
+; CHECK-NEXT:    vpop {d8, d9}
+; CHECK-NEXT:    pop {r4, pc}
   %a4 = icmp eq <2 x i64> %a, zeroinitializer
   %a5 = select <2 x i1> %a4, <2 x i32> zeroinitializer, <2 x i32> %c
   %a6 = icmp ne <2 x i32> %b, zeroinitializer
   %a7 = icmp slt <2 x i32> %a5, %c
   %a8 = icmp ne <2 x i32> %a5, zeroinitializer
   %a9 = and <2 x i1> %a6, %a8
   %a10 = and <2 x i1> %a7, %a9
   %a11 = select <2 x i1> %a10, <2 x i32> %c, <2 x i32> %a5

 define arm_aapcs_vfpcc <2 x i64> @vcmp_r_eq_v2i64(<2 x i64> %src, i64 %src2, <2 x i64> %a, <2 x i64> %b) {
 ; CHECK-LABEL: vcmp_r_eq_v2i64:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    eors r2, r1
 ; CHECK-NEXT:    eors r3, r0
 ; CHECK-NEXT:    orrs r2, r3
+; CHECK-NEXT:    vmov r3, s3
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov r2, s3
-; CHECK-NEXT:    eors r1, r2
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    eors r0, r2
+; CHECK-NEXT:    eors r1, r3
+; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    eors r0, r3
 ; CHECK-NEXT:    orrs r0, r1
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vbic q0, q2, q3
-; CHECK-NEXT:    vand q1, q1, q3
-; CHECK-NEXT:    vorr q0, q1, q0
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r0, r2
+; CHECK-NEXT:    vbic q2, q2, q0
+; CHECK-NEXT:    vand q0, q1, q0
+; CHECK-NEXT:    vorr q0, q0, q2
 ; CHECK-NEXT:    bx lr
 entry:
   %i = insertelement <2 x i64> undef, i64 %src2, i32 0
   %sp = shufflevector <2 x i64> %i, <2 x i64> undef, <2 x i32> zeroinitializer
   %c = icmp eq <2 x i64> %sp, %src
   %s = select <2 x i1> %c, <2 x i64> %a, <2 x i64> %b
   ret <2 x i64> %s
 }
 
 define arm_aapcs_vfpcc <2 x i32> @vcmp_r_eq_v2i32(<2 x i64> %src, i64 %src2, <2 x i32> %a, <2 x i32> %b) {
 ; CHECK-LABEL: vcmp_r_eq_v2i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    eors r2, r1
 ; CHECK-NEXT:    eors r3, r0
 ; CHECK-NEXT:    orrs r2, r3
+; CHECK-NEXT:    vmov r3, s3
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov r2, s3
-; CHECK-NEXT:    eors r1, r2
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    eors r0, r2
+; CHECK-NEXT:    eors r1, r3
+; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    eors r0, r3
 ; CHECK-NEXT:    orrs r0, r1
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vbic q0, q2, q3
-; CHECK-NEXT:    vand q1, q1, q3
-; CHECK-NEXT:    vorr q0, q1, q0
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r0, r2
+; CHECK-NEXT:    vbic q2, q2, q0
+; CHECK-NEXT:    vand q0, q1, q0
+; CHECK-NEXT:    vorr q0, q0, q2
 ; CHECK-NEXT:    bx lr
 entry:
   %i = insertelement <2 x i64> undef, i64 %src2, i32 0
   %sp = shufflevector <2 x i64> %i, <2 x i64> undef, <2 x i32> zeroinitializer
   %c = icmp eq <2 x i64> %sp, %src
   %s = select <2 x i1> %c, <2 x i32> %a, <2 x i32> %b
   ret <2 x i32> %s
 }
 
 define arm_aapcs_vfpcc <2 x i32> @vcmp_r_multi_v2i32(<2 x i64> %a, <2 x i32> %b, <2 x i32> %c) {
 ; CHECK-LABEL: vcmp_r_multi_v2i32:
 ; CHECK:       @ %bb.0:
-; CHECK-NEXT:    .save {r7, lr}
-; CHECK-NEXT:    push {r7, lr}
-; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-NEXT:    vpush {d8, d9, d10, d11}
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .vsave {d8, d9}
+; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    movs r3, #0
 ; CHECK-NEXT:    vmov r1, s0
-; CHECK-NEXT:    vmov r2, s8
-; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov r0, s3
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    orrs r0, r1
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vbic q0, q2, q3
-; CHECK-NEXT:    vmov lr, s0
-; CHECK-NEXT:    subs.w r1, lr, r2
-; CHECK-NEXT:    asr.w r12, lr, #31
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    vmov r2, s8
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
+; CHECK-NEXT:    vbic q0, q2, q0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    subs r1, r0, r2
+; CHECK-NEXT:    asr.w r12, r0, #31
 ; CHECK-NEXT:    sbcs.w r1, r12, r2, asr #31
 ; CHECK-NEXT:    mov.w r1, #0
-; CHECK-NEXT:    vmov r2, s10
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r1, #1
 ; CHECK-NEXT:    cmp r1, #0
-; CHECK-NEXT:    csetm r1, ne
-; CHECK-NEXT:    vmov.32 q3[0], r1
-; CHECK-NEXT:    vmov.32 q3[1], r1
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    subs r0, r1, r2
-; CHECK-NEXT:    asr.w r12, r1, #31
-; CHECK-NEXT:    sbcs.w r0, r12, r2, asr #31
+; CHECK-NEXT:    vmov r1, s10
+; CHECK-NEXT:    csetm lr, ne
+; CHECK-NEXT:    asr.w r12, r2, #31
+; CHECK-NEXT:    subs r4, r2, r1
+; CHECK-NEXT:    sbcs.w r1, r12, r1, asr #31
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r3, #1
 ; CHECK-NEXT:    cmp r3, #0
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    cmp.w lr, #0
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    cset r0, ne
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    cmp r1, #0
-; CHECK-NEXT:    vmov.32 q4[0], r0
-; CHECK-NEXT:    vmov.32 q4[1], r0
-; CHECK-NEXT:    cset r0, ne
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q4[2], r0
-; CHECK-NEXT:    vmov.32 q4[3], r0
-; CHECK-NEXT:    vmov r0, s4
+; CHECK-NEXT:    csetm r1, ne
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    cset r0, ne
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, lr
 ; CHECK-NEXT:    tst.w r0, #1
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, lr
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q5[0], r0
-; CHECK-NEXT:    vmov.32 q5[1], r0
-; CHECK-NEXT:    vmov r0, s6
+; CHECK-NEXT:    cmp r2, #0
+; CHECK-NEXT:    cset r1, ne
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q4[2], q4[0], r1, r0
+; CHECK-NEXT:    vmov q4[3], q4[1], r1, r0
+; CHECK-NEXT:    vmov r0, s4
+; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    cset r0, ne
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q5[2], r0
-; CHECK-NEXT:    vmov.32 q5[3], r0
-; CHECK-NEXT:    vand q1, q5, q4
+; CHECK-NEXT:    cmp r1, #0
+; CHECK-NEXT:    cset r1, ne
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
+; CHECK-NEXT:    vand q1, q1, q4
 ; CHECK-NEXT:    vand q1, q3, q1
 ; CHECK-NEXT:    vbic q0, q0, q1
 ; CHECK-NEXT:    vand q1, q2, q1
 ; CHECK-NEXT:    vorr q0, q1, q0
-; CHECK-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-NEXT:    pop {r7, pc}
+; CHECK-NEXT:    vpop {d8, d9}
+; CHECK-NEXT:    pop {r4, pc}
   %a4 = icmp eq <2 x i64> %a, zeroinitializer
   %a5 = select <2 x i1> %a4, <2 x i32> zeroinitializer, <2 x i32> %c
   %a6 = icmp ne <2 x i32> %b, zeroinitializer
   %a7 = icmp slt <2 x i32> %a5, %c
   %a8 = icmp ne <2 x i32> %a5, zeroinitializer
   %a9 = and <2 x i1> %a6, %a8
   %a10 = and <2 x i1> %a7, %a9
   %a11 = select <2 x i1> %a10, <2 x i32> %c, <2 x i32> %a5
   ret <2 x i32> %a11
 }

llvm/test/CodeGen/Thumb2/mve-vcmpz.ll

 }
 
 
 define arm_aapcs_vfpcc <2 x i64> @vcmp_eqz_v2i64(<2 x i64> %src, <2 x i64> %a, <2 x i64> %b) {
 ; CHECK-LABEL: vcmp_eqz_v2i64:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    vmov r1, s0
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s2
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vbic q0, q2, q3
-; CHECK-NEXT:    vand q1, q1, q3
-; CHECK-NEXT:    vorr q0, q1, q0
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
+; CHECK-NEXT:    vbic q2, q2, q0
+; CHECK-NEXT:    vand q0, q1, q0
+; CHECK-NEXT:    vorr q0, q0, q2
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp eq <2 x i64> %src, zeroinitializer
   %s = select <2 x i1> %c, <2 x i64> %a, <2 x i64> %b
   ret <2 x i64> %s
 }
 
 define arm_aapcs_vfpcc <2 x i32> @vcmp_eqz_v2i32(<2 x i64> %src, <2 x i32> %a, <2 x i32> %b) {
 ; CHECK-LABEL: vcmp_eqz_v2i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    vmov r1, s0
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    orrs r0, r1
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vbic q0, q2, q3
-; CHECK-NEXT:    vand q1, q1, q3
-; CHECK-NEXT:    vorr q0, q1, q0
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
+; CHECK-NEXT:    vbic q2, q2, q0
+; CHECK-NEXT:    vand q0, q1, q0
+; CHECK-NEXT:    vorr q0, q0, q2
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp eq <2 x i64> %src, zeroinitializer
   %s = select <2 x i1> %c, <2 x i32> %a, <2 x i32> %b
   ret <2 x i32> %s
 }
 
 

 }
 
 
 define arm_aapcs_vfpcc <2 x i64> @vcmp_r_eqz_v2i64(<2 x i64> %src, <2 x i64> %a, <2 x i64> %b) {
 ; CHECK-LABEL: vcmp_r_eqz_v2i64:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    vmov r1, s0
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s2
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vbic q0, q2, q3
-; CHECK-NEXT:    vand q1, q1, q3
-; CHECK-NEXT:    vorr q0, q1, q0
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
+; CHECK-NEXT:    vbic q2, q2, q0
+; CHECK-NEXT:    vand q0, q1, q0
+; CHECK-NEXT:    vorr q0, q0, q2
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp eq <2 x i64> zeroinitializer, %src
   %s = select <2 x i1> %c, <2 x i64> %a, <2 x i64> %b
   ret <2 x i64> %s
 }
 
 define arm_aapcs_vfpcc <2 x i32> @vcmp_r_eqz_v2i32(<2 x i64> %src, <2 x i32> %a, <2 x i32> %b) {
 ; CHECK-LABEL: vcmp_r_eqz_v2i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    vmov r1, s0
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    orrs r0, r1
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vbic q0, q2, q3
-; CHECK-NEXT:    vand q1, q1, q3
-; CHECK-NEXT:    vorr q0, q1, q0
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
+; CHECK-NEXT:    vbic q2, q2, q0
+; CHECK-NEXT:    vand q0, q1, q0
+; CHECK-NEXT:    vorr q0, q0, q2
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp eq <2 x i64> %src, zeroinitializer
   %s = select <2 x i1> %c, <2 x i32> %a, <2 x i32> %b
   ret <2 x i32> %s
 }

llvm/test/CodeGen/Thumb2/mve-vcreate.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve -verify-machineinstrs %s -o - | FileCheck %s
 
 define arm_aapcs_vfpcc <4 x i32> @vcreate_i32(i32 %a, i32 %b, i32 %c, i32 %d) {
 ; CHECK-LABEL: vcreate_i32:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vmov.32 q0[0], r1
-; CHECK-NEXT:    vmov.32 q0[1], r0
-; CHECK-NEXT:    vmov.32 q0[2], r3
-; CHECK-NEXT:    vmov.32 q0[3], r2
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r1
+; CHECK-NEXT:    vmov q0[3], q0[1], r2, r0
 ; CHECK-NEXT:    bx lr
 entry:
   %conv = zext i32 %a to i64
   %shl = shl nuw i64 %conv, 32
   %conv1 = zext i32 %b to i64
   %or = or i64 %shl, %conv1
   %0 = insertelement <2 x i64> undef, i64 %or, i64 0
   %conv2 = zext i32 %c to i64
   %shl3 = shl nuw i64 %conv2, 32
   %conv4 = zext i32 %d to i64
   %or5 = or i64 %shl3, %conv4
   %1 = insertelement <2 x i64> %0, i64 %or5, i64 1
   %2 = bitcast <2 x i64> %1 to <4 x i32>
   ret <4 x i32> %2
 }
 
 define arm_aapcs_vfpcc <4 x i32> @insert_0123(i32 %a, i32 %b, i32 %c, i32 %d) {
 ; CHECK-LABEL: insert_0123:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.32 q0[1], r1
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vmov.32 q0[3], r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r1
 ; CHECK-NEXT:    bx lr
 entry:
   %v1 = insertelement <4 x i32> undef, i32 %a, i32 0
   %v2 = insertelement <4 x i32> %v1, i32 %b, i32 1
   %v3 = insertelement <4 x i32> %v2, i32 %c, i32 2
   %v4 = insertelement <4 x i32> %v3, i32 %d, i32 3
   ret <4 x i32> %v4
 }
 
 define arm_aapcs_vfpcc <4 x i32> @insert_3210(i32 %a, i32 %b, i32 %c, i32 %d) {
 ; CHECK-LABEL: insert_3210:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vmov.32 q0[0], r3
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.32 q0[2], r1
-; CHECK-NEXT:    vmov.32 q0[3], r0
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r3
+; CHECK-NEXT:    vmov q0[3], q0[1], r0, r2
 ; CHECK-NEXT:    bx lr
 entry:
   %v1 = insertelement <4 x i32> undef, i32 %a, i32 3
   %v2 = insertelement <4 x i32> %v1, i32 %b, i32 2
   %v3 = insertelement <4 x i32> %v2, i32 %c, i32 1
   %v4 = insertelement <4 x i32> %v3, i32 %d, i32 0
   ret <4 x i32> %v4
 }
 
 define arm_aapcs_vfpcc <4 x i32> @insert_0213(i32 %a, i32 %b, i32 %c, i32 %d) {
 ; CHECK-LABEL: insert_0213:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.32 q0[2], r1
-; CHECK-NEXT:    vmov.32 q0[3], r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r2
 ; CHECK-NEXT:    bx lr
 entry:
   %v1 = insertelement <4 x i32> undef, i32 %a, i32 0
   %v2 = insertelement <4 x i32> %v1, i32 %b, i32 2
   %v3 = insertelement <4 x i32> %v2, i32 %c, i32 1
   %v4 = insertelement <4 x i32> %v3, i32 %d, i32 3
   ret <4 x i32> %v4
 }
 
 define arm_aapcs_vfpcc <4 x i32> @insert_0220(i32 %a, i32 %b, i32 %c, i32 %d) {
 ; CHECK-LABEL: insert_0220:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vmov.32 q0[0], r3
-; CHECK-NEXT:    vmov.32 q0[2], r2
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r3
 ; CHECK-NEXT:    bx lr
 entry:
   %v1 = insertelement <4 x i32> undef, i32 %a, i32 0
   %v2 = insertelement <4 x i32> %v1, i32 %b, i32 2
   %v3 = insertelement <4 x i32> %v2, i32 %c, i32 2
   %v4 = insertelement <4 x i32> %v3, i32 %d, i32 0
   ret <4 x i32> %v4
 }
 
 define arm_aapcs_vfpcc <4 x i32> @insert_321(i32 %a, i32 %b, i32 %c, i32 %d) {
 ; CHECK-LABEL: insert_321:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vmov.32 q0[1], r2
 ; CHECK-NEXT:    vmov.32 q0[2], r1
-; CHECK-NEXT:    vmov.32 q0[3], r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r0, r2
 ; CHECK-NEXT:    bx lr
 entry:
   %v1 = insertelement <4 x i32> undef, i32 %a, i32 3
   %v2 = insertelement <4 x i32> %v1, i32 %b, i32 2
   %v3 = insertelement <4 x i32> %v2, i32 %c, i32 1
   ret <4 x i32> %v3
 }
 
 define arm_aapcs_vfpcc <4 x i32> @insert_310(i32 %a, i32 %b, i32 %c, i32 %d) {
 ; CHECK-LABEL: insert_310:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov.32 q0[1], r1
-; CHECK-NEXT:    vmov.32 q0[3], r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r0, r1
 ; CHECK-NEXT:    bx lr
 entry:
   %v1 = insertelement <4 x i32> undef, i32 %a, i32 3
   %v2 = insertelement <4 x i32> %v1, i32 %b, i32 1
   %v3 = insertelement <4 x i32> %v2, i32 %c, i32 0
   ret <4 x i32> %v3
 }
 
 define arm_aapcs_vfpcc <4 x i32> @insert_320(i32 %a, i32 %b, i32 %c, i32 %d) {
 ; CHECK-LABEL: insert_320:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov.32 q0[2], r1
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r2
 ; CHECK-NEXT:    vmov.32 q0[3], r0
 ; CHECK-NEXT:    bx lr
 entry:
   %v1 = insertelement <4 x i32> undef, i32 %a, i32 3
   %v2 = insertelement <4 x i32> %v1, i32 %b, i32 2
   %v3 = insertelement <4 x i32> %v2, i32 %c, i32 0
   ret <4 x i32> %v3
 }
 
 define arm_aapcs_vfpcc <4 x i32> @insert_31(i32 %a, i32 %b, i32 %c, i32 %d) {
 ; CHECK-LABEL: insert_31:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vmov.32 q0[1], r1
-; CHECK-NEXT:    vmov.32 q0[3], r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r0, r1
 ; CHECK-NEXT:    bx lr
 entry:
   %v1 = insertelement <4 x i32> undef, i32 %a, i32 3
   %v2 = insertelement <4 x i32> %v1, i32 %b, i32 1
   ret <4 x i32> %v2
 }
 
 define arm_aapcs_vfpcc <4 x i32> @insert_32(i32 %a, i32 %b, i32 %c, i32 %d) {

 entry:
   %v1 = insertelement <4 x i32> undef, i32 %a, i32 3
   ret <4 x i32> %v1
 }
 
 define arm_aapcs_vfpcc <4 x i32> @insert_210(i32 %a, i32 %b, i32 %c, i32 %d) {
 ; CHECK-LABEL: insert_210:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vmov.32 q0[0], r2
 ; CHECK-NEXT:    vmov.32 q0[1], r1
-; CHECK-NEXT:    vmov.32 q0[2], r0
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r2
 ; CHECK-NEXT:    bx lr
 entry:
   %v1 = insertelement <4 x i32> undef, i32 %a, i32 2
   %v2 = insertelement <4 x i32> %v1, i32 %b, i32 1
   %v3 = insertelement <4 x i32> %v2, i32 %c, i32 0
   ret <4 x i32> %v3
 }
 
 define arm_aapcs_vfpcc <4 x i32> @insert_20(i32 %a, i32 %b, i32 %c, i32 %d) {
 ; CHECK-LABEL: insert_20:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vmov.32 q0[0], r1
-; CHECK-NEXT:    vmov.32 q0[2], r0
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r1
 ; CHECK-NEXT:    bx lr
 entry:
   %v1 = insertelement <4 x i32> undef, i32 %a, i32 2
   %v2 = insertelement <4 x i32> %v1, i32 %b, i32 0
   ret <4 x i32> %v2
 }
 
 define arm_aapcs_vfpcc <4 x i32> @insert_21(i32 %a, i32 %b, i32 %c, i32 %d) {

 entry:
   %v1 = insertelement <4 x i32> undef, i32 %a, i32 0
   ret <4 x i32> %v1
 }
 
 define hidden <8 x i16> @create_i16(i16 zeroext %a, i16 zeroext %b, i16 zeroext %c, i16 zeroext %d, i16 zeroext %a2, i16 zeroext %b2, i16 zeroext %c2, i16 zeroext %d2) local_unnamed_addr #0 {
 ; CHECK-LABEL: create_i16:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-NEXT:    push {r4, r5, r7, lr}
+; CHECK-NEXT:    .save {r5, r6, r7, lr}
+; CHECK-NEXT:    push {r5, r6, r7, lr}
+; CHECK-NEXT:    movs r7, #0
 ; CHECK-NEXT:    movs r5, #0
+; CHECK-NEXT:    lsll r2, r7, #16
 ; CHECK-NEXT:    orr.w r0, r1, r0, lsl #16
+; CHECK-NEXT:    orr.w r12, r2, r3
+; CHECK-NEXT:    ldr r2, [sp, #24]
+; CHECK-NEXT:    ldr r3, [sp, #28]
+; CHECK-NEXT:    orrs r0, r7
 ; CHECK-NEXT:    lsll r2, r5, #16
-; CHECK-NEXT:    ldrd lr, r4, [sp, #16]
-; CHECK-NEXT:    orr.w r1, r2, r3
-; CHECK-NEXT:    ldr.w r12, [sp, #24]
-; CHECK-NEXT:    orrs r0, r5
-; CHECK-NEXT:    vmov.32 q0[0], r1
-; CHECK-NEXT:    movs r7, #0
-; CHECK-NEXT:    vmov.32 q0[1], r0
-; CHECK-NEXT:    ldr r0, [sp, #28]
-; CHECK-NEXT:    lsll r12, r7, #16
-; CHECK-NEXT:    orr.w r4, r4, lr, lsl #16
-; CHECK-NEXT:    orr.w r0, r0, r12
-; CHECK-NEXT:    orrs r7, r4
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    vmov.32 q0[3], r7
+; CHECK-NEXT:    orrs r2, r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r12
+; CHECK-NEXT:    ldrd r1, r2, [sp, #16]
+; CHECK-NEXT:    orr.w r1, r2, r1, lsl #16
+; CHECK-NEXT:    orrs r1, r5
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
 ; CHECK-NEXT:    vmov r0, r1, d0
 ; CHECK-NEXT:    vmov r2, r3, d1
-; CHECK-NEXT:    pop {r4, r5, r7, pc}
+; CHECK-NEXT:    pop {r5, r6, r7, pc}
 entry:
   %conv = zext i16 %a to i64
   %shl = shl nuw i64 %conv, 48
   %conv1 = zext i16 %b to i64
   %shl2 = shl nuw nsw i64 %conv1, 32
   %or = or i64 %shl2, %shl
   %conv3 = zext i16 %c to i64
   %shl4 = shl nuw nsw i64 %conv3, 16
entry:
   %v7 = insertelement <8 x i16> %v6, i16 %a6, i32 6
   %v8 = insertelement <8 x i16> %v7, i16 %a7, i32 7
   ret <8 x i16> %v8
 }
 
 define hidden <16 x i8> @create_i8(i8 zeroext %a1, i8 zeroext %b1, i8 zeroext %c1, i8 zeroext %d1, i8 zeroext %a2, i8 zeroext %b2, i8 zeroext %c2, i8 zeroext %d2, i8 zeroext %a3, i8 zeroext %b3, i8 zeroext %c3, i8 zeroext %d3, i8 zeroext %a4, i8 zeroext %b4, i8 zeroext %c4, i8 zeroext %d4) local_unnamed_addr #0 {
 ; CHECK-LABEL: create_i8:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, r5, r7, r9, r11, lr}
-; CHECK-NEXT:    push.w {r4, r5, r7, r9, r11, lr}
-; CHECK-NEXT:    ldr.w r12, [sp, #28]
+; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r11, lr}
+; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r11, lr}
+; CHECK-NEXT:    ldr r4, [sp, #36]
 ; CHECK-NEXT:    mov.w r11, #0
-; CHECK-NEXT:    ldr r4, [sp, #24]
-; CHECK-NEXT:    movs r5, #0
-; CHECK-NEXT:    lsll r12, r11, #16
-; CHECK-NEXT:    lsls r1, r1, #16
-; CHECK-NEXT:    lsll r4, r5, #24
-; CHECK-NEXT:    orr.w r0, r1, r0, lsl #22
-; CHECK-NEXT:    orr.w r12, r12, r4
-; CHECK-NEXT:    ldr r4, [sp, #32]
+; CHECK-NEXT:    ldr r6, [sp, #32]
 ; CHECK-NEXT:    movs r7, #0
-; CHECK-NEXT:    orr.w r0, r0, r2, lsl #8
-; CHECK-NEXT:    lsll r4, r7, #8
-; CHECK-NEXT:    add r0, r3
-; CHECK-NEXT:    orr.w r12, r12, r4
-; CHECK-NEXT:    ldr r4, [sp, #36]
-; CHECK-NEXT:    orrs r0, r5
-; CHECK-NEXT:    ldr r2, [sp, #56]
-; CHECK-NEXT:    orr.w r0, r0, r11
-; CHECK-NEXT:    orr.w r4, r4, r12
-; CHECK-NEXT:    vmov.32 q0[0], r4
-; CHECK-NEXT:    orrs r0, r7
-; CHECK-NEXT:    vmov.32 q0[1], r0
-; CHECK-NEXT:    ldr r0, [sp, #60]
-; CHECK-NEXT:    movs r1, #0
+; CHECK-NEXT:    lsll r4, r11, #16
+; CHECK-NEXT:    mov lr, r1
+; CHECK-NEXT:    lsll r6, r7, #24
+; CHECK-NEXT:    mov r12, r3
+; CHECK-NEXT:    orr.w r1, r6, r4
+; CHECK-NEXT:    ldr r4, [sp, #40]
 ; CHECK-NEXT:    movs r3, #0
-; CHECK-NEXT:    lsll r0, r1, #16
-; CHECK-NEXT:    lsll r2, r3, #24
-; CHECK-NEXT:    orrs r0, r2
-; CHECK-NEXT:    ldr r2, [sp, #64]
+; CHECK-NEXT:    ldr r6, [sp, #68]
+; CHECK-NEXT:    lsll r4, r3, #8
+; CHECK-NEXT:    movs r5, #0
+; CHECK-NEXT:    orrs r1, r4
+; CHECK-NEXT:    ldr r4, [sp, #44]
+; CHECK-NEXT:    lsll r6, r5, #16
 ; CHECK-NEXT:    mov.w r9, #0
-; CHECK-NEXT:    lsll r2, r9, #8
-; CHECK-NEXT:    orrs r0, r2
-; CHECK-NEXT:    ldr r2, [sp, #68]
-; CHECK-NEXT:    orrs r0, r2
-; CHECK-NEXT:    ldr r2, [sp, #40]
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    ldr r0, [sp, #44]
-; CHECK-NEXT:    lsls r0, r0, #16
-; CHECK-NEXT:    orr.w r0, r0, r2, lsl #22
-; CHECK-NEXT:    ldr r2, [sp, #48]
+; CHECK-NEXT:    orr.w r8, r1, r4
+; CHECK-NEXT:    ldr r4, [sp, #64]
+; CHECK-NEXT:    movs r1, #0
+; CHECK-NEXT:    lsll r4, r1, #24
+; CHECK-NEXT:    orrs r4, r6
+; CHECK-NEXT:    ldr r6, [sp, #72]
+; CHECK-NEXT:    lsll r6, r9, #8
+; CHECK-NEXT:    orrs r4, r6
+; CHECK-NEXT:    ldr r6, [sp, #76]
+; CHECK-NEXT:    orrs r4, r6
+; CHECK-NEXT:    lsl.w r6, lr, #16
+; CHECK-NEXT:    orr.w r0, r6, r0, lsl #22
+; CHECK-NEXT:    vmov q0[2], q0[0], r4, r8
 ; CHECK-NEXT:    orr.w r0, r0, r2, lsl #8
 ; CHECK-NEXT:    ldr r2, [sp, #52]
-; CHECK-NEXT:    add r0, r2
+; CHECK-NEXT:    add r0, r12
+; CHECK-NEXT:    orrs r0, r7
+; CHECK-NEXT:    orr.w r0, r0, r11
+; CHECK-NEXT:    lsls r2, r2, #16
 ; CHECK-NEXT:    orrs r0, r3
-; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    orr.w r0, r0, r9
-; CHECK-NEXT:    vmov.32 q0[3], r0
+; CHECK-NEXT:    ldr r3, [sp, #48]
+; CHECK-NEXT:    orr.w r2, r2, r3, lsl #22
+; CHECK-NEXT:    ldr r3, [sp, #56]
+; CHECK-NEXT:    orr.w r2, r2, r3, lsl #8
+; CHECK-NEXT:    ldr r3, [sp, #60]
+; CHECK-NEXT:    add r2, r3
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    orrs r1, r5
+; CHECK-NEXT:    orr.w r1, r1, r9
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
 ; CHECK-NEXT:    vmov r0, r1, d0
 ; CHECK-NEXT:    vmov r2, r3, d1
-; CHECK-NEXT:    pop.w {r4, r5, r7, r9, r11, pc}
+; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r11, pc}
 entry:
   %conv = zext i8 %a1 to i64
   %shl = shl nuw nsw i64 %conv, 54
   %conv1 = zext i8 %b1 to i64
   %shl2 = shl nuw nsw i64 %conv1, 48
   %or = or i64 %shl2, %shl
   %conv3 = zext i8 %c1 to i64
   %shl4 = shl nuw nsw i64 %conv3, 40

llvm/test/CodeGen/Thumb2/mve-vcvt.ll

entry:
   %out = uitofp <4 x i32> %src to <4 x float>
   ret <4 x float> %out
 }
 
 define arm_aapcs_vfpcc <4 x i32> @foo_int32_float(<4 x float> %src) {
 ; CHECK-MVE-LABEL: foo_int32_float:
 ; CHECK-MVE:       @ %bb.0: @ %entry
 ; CHECK-MVE-NEXT:    vcvt.s32.f32 s4, s0
-; CHECK-MVE-NEXT:    vcvt.s32.f32 s6, s1
-; CHECK-MVE-NEXT:    vcvt.s32.f32 s10, s2
-; CHECK-MVE-NEXT:    vcvt.s32.f32 s8, s3
+; CHECK-MVE-NEXT:    vcvt.s32.f32 s6, s2
+; CHECK-MVE-NEXT:    vcvt.s32.f32 s8, s1
+; CHECK-MVE-NEXT:    vcvt.s32.f32 s10, s3
 ; CHECK-MVE-NEXT:    vmov r0, s4
-; CHECK-MVE-NEXT:    vmov.32 q0[0], r0
-; CHECK-MVE-NEXT:    vmov r0, s6
-; CHECK-MVE-NEXT:    vmov.32 q0[1], r0
-; CHECK-MVE-NEXT:    vmov r0, s10
-; CHECK-MVE-NEXT:    vmov.32 q0[2], r0
+; CHECK-MVE-NEXT:    vmov r1, s6
+; CHECK-MVE-NEXT:    vmov q0[2], q0[0], r1, r0
 ; CHECK-MVE-NEXT:    vmov r0, s8
-; CHECK-MVE-NEXT:    vmov.32 q0[3], r0
+; CHECK-MVE-NEXT:    vmov r1, s10
+; CHECK-MVE-NEXT:    vmov q0[3], q0[1], r1, r0
 ; CHECK-MVE-NEXT:    bx lr
 ;
 ; CHECK-MVEFP-LABEL: foo_int32_float:
 ; CHECK-MVEFP:       @ %bb.0: @ %entry
 ; CHECK-MVEFP-NEXT:    vcvt.s32.f32 q0, q0
 ; CHECK-MVEFP-NEXT:    bx lr
 entry:
   %out = fptosi <4 x float> %src to <4 x i32>
   ret <4 x i32> %out
 }
 
 define arm_aapcs_vfpcc <4 x i32> @foo_uint32_float(<4 x float> %src) {
 ; CHECK-MVE-LABEL: foo_uint32_float:
 ; CHECK-MVE:       @ %bb.0: @ %entry
 ; CHECK-MVE-NEXT:    vcvt.u32.f32 s4, s0
-; CHECK-MVE-NEXT:    vcvt.u32.f32 s6, s1
-; CHECK-MVE-NEXT:    vcvt.u32.f32 s10, s2
-; CHECK-MVE-NEXT:    vcvt.u32.f32 s8, s3
+; CHECK-MVE-NEXT:    vcvt.u32.f32 s6, s2
+; CHECK-MVE-NEXT:    vcvt.u32.f32 s8, s1
+; CHECK-MVE-NEXT:    vcvt.u32.f32 s10, s3
 ; CHECK-MVE-NEXT:    vmov r0, s4
-; CHECK-MVE-NEXT:    vmov.32 q0[0], r0
-; CHECK-MVE-NEXT:    vmov r0, s6
-; CHECK-MVE-NEXT:    vmov.32 q0[1], r0
-; CHECK-MVE-NEXT:    vmov r0, s10
-; CHECK-MVE-NEXT:    vmov.32 q0[2], r0
+; CHECK-MVE-NEXT:    vmov r1, s6
+; CHECK-MVE-NEXT:    vmov q0[2], q0[0], r1, r0
 ; CHECK-MVE-NEXT:    vmov r0, s8
-; CHECK-MVE-NEXT:    vmov.32 q0[3], r0
+; CHECK-MVE-NEXT:    vmov r1, s10
+; CHECK-MVE-NEXT:    vmov q0[3], q0[1], r1, r0
 ; CHECK-MVE-NEXT:    bx lr
 ;
 ; CHECK-MVEFP-LABEL: foo_uint32_float:
 ; CHECK-MVEFP:       @ %bb.0: @ %entry
 ; CHECK-MVEFP-NEXT:    vcvt.u32.f32 q0, q0
 ; CHECK-MVEFP-NEXT:    bx lr
 entry:
   %out = fptoui <4 x float> %src to <4 x i32>

 entry:
   %out = uitofp <2 x i64> %src to <2 x double>
   ret <2 x double> %out
 }
 
 define arm_aapcs_vfpcc <2 x i64> @foo_int64_float(<2 x double> %src) {
 ; CHECK-LABEL: foo_int64_float:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r7, lr}
-; CHECK-NEXT:    push {r7, lr}
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    .vsave {d8, d9}
 ; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    vmov q4, q0
 ; CHECK-NEXT:    vmov r0, r1, d8
 ; CHECK-NEXT:    bl __aeabi_d2lz
-; CHECK-NEXT:    vmov r2, r3, d9
-; CHECK-NEXT:    vmov.32 q4[0], r0
-; CHECK-NEXT:    vmov.32 q4[1], r1
-; CHECK-NEXT:    mov r0, r2
-; CHECK-NEXT:    mov r1, r3
+; CHECK-NEXT:    mov r4, r0
+; CHECK-NEXT:    mov r5, r1
+; CHECK-NEXT:    vmov r0, r1, d9
 ; CHECK-NEXT:    bl __aeabi_d2lz
-; CHECK-NEXT:    vmov.32 q4[2], r0
-; CHECK-NEXT:    vmov.32 q4[3], r1
-; CHECK-NEXT:    vmov q0, q4
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r4
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r5
 ; CHECK-NEXT:    vpop {d8, d9}
-; CHECK-NEXT:    pop {r7, pc}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %out = fptosi <2 x double> %src to <2 x i64>
   ret <2 x i64> %out
 }
 
 define arm_aapcs_vfpcc <2 x i64> @foo_uint64_float(<2 x double> %src) {
 ; CHECK-LABEL: foo_uint64_float:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r7, lr}
-; CHECK-NEXT:    push {r7, lr}
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    .vsave {d8, d9}
 ; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    vmov q4, q0
 ; CHECK-NEXT:    vmov r0, r1, d8
 ; CHECK-NEXT:    bl __aeabi_d2ulz
-; CHECK-NEXT:    vmov r2, r3, d9
-; CHECK-NEXT:    vmov.32 q4[0], r0
-; CHECK-NEXT:    vmov.32 q4[1], r1
-; CHECK-NEXT:    mov r0, r2
-; CHECK-NEXT:    mov r1, r3
+; CHECK-NEXT:    mov r4, r0
+; CHECK-NEXT:    mov r5, r1
+; CHECK-NEXT:    vmov r0, r1, d9
 ; CHECK-NEXT:    bl __aeabi_d2ulz
-; CHECK-NEXT:    vmov.32 q4[2], r0
-; CHECK-NEXT:    vmov.32 q4[3], r1
-; CHECK-NEXT:    vmov q0, q4
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r4
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r5
 ; CHECK-NEXT:    vpop {d8, d9}
-; CHECK-NEXT:    pop {r7, pc}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %out = fptoui <2 x double> %src to <2 x i64>
   ret <2 x i64> %out
 }

llvm/test/CodeGen/Thumb2/mve-vdup.ll

entry:
   %0 = insertelement <16 x i8> undef, i8 %src, i32 0
   %out = shufflevector <16 x i8> %0, <16 x i8> undef, <16 x i32> zeroinitializer
   ret <16 x i8> %out
 }
 
 define arm_aapcs_vfpcc <2 x i64> @vdup_i64(i64 %src) {
 ; CHECK-LABEL: vdup_i64:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.32 q0[1], r1
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    vmov.32 q0[3], r1
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r1
 ; CHECK-NEXT:    bx lr
 entry:
   %0 = insertelement <2 x i64> undef, i64 %src, i32 0
   %out = shufflevector <2 x i64> %0, <2 x i64> undef, <2 x i32> zeroinitializer
   ret <2 x i64> %out
 }
 
 define arm_aapcs_vfpcc <4 x float> @vdup_f32_1(float %src) {

llvm/test/CodeGen/Thumb2/mve-vecreduce-add.ll

entry:
   %z = call i16 @llvm.vector.reduce.add.v8i16(<8 x i16> %x)
   ret i16 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v8i16_v8i64_zext(<8 x i16> %x) {
 ; CHECK-LABEL: add_v8i16_v8i64_zext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.u16 r0, q0[0]
+; CHECK-NEXT:    vmov.u16 r1, q0[1]
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
 ; CHECK-NEXT:    vmov.i64 q1, #0xffff
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-NEXT:    vmov.32 q2[2], r0
 ; CHECK-NEXT:    vand q2, q2, q1
+; CHECK-NEXT:    vmov.u16 r2, q0[3]
 ; CHECK-NEXT:    vmov r0, s10
 ; CHECK-NEXT:    vmov r1, s8
-; CHECK-NEXT:    vmov r2, s11
 ; CHECK-NEXT:    add r0, r1
 ; CHECK-NEXT:    vmov.u16 r1, q0[2]
-; CHECK-NEXT:    vmov.32 q3[0], r1
-; CHECK-NEXT:    vmov.u16 r1, q0[3]
-; CHECK-NEXT:    vmov.32 q3[2], r1
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r1
+; CHECK-NEXT:    vmov.u16 r2, q0[5]
 ; CHECK-NEXT:    vand q3, q3, q1
 ; CHECK-NEXT:    vmov r1, s12
 ; CHECK-NEXT:    add r0, r1
 ; CHECK-NEXT:    vmov r1, s14
 ; CHECK-NEXT:    add r0, r1
 ; CHECK-NEXT:    vmov.u16 r1, q0[4]
-; CHECK-NEXT:    vmov.32 q3[0], r1
-; CHECK-NEXT:    vmov.u16 r1, q0[5]
-; CHECK-NEXT:    vmov.32 q3[2], r1
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r1
+; CHECK-NEXT:    vmov r2, s11
 ; CHECK-NEXT:    vand q3, q3, q1
 ; CHECK-NEXT:    vmov r1, s12
 ; CHECK-NEXT:    vmov r3, s14
 ; CHECK-NEXT:    add r0, r1
 ; CHECK-NEXT:    vmov r1, s15
 ; CHECK-NEXT:    adds r0, r0, r3
+; CHECK-NEXT:    vmov.u16 r3, q0[7]
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov.u16 r2, q0[6]
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q0[7]
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vand q0, q2, q1
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov r2, s3
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    bx lr
 entry:
   %xx = zext <8 x i16> %x to <8 x i64>
   %z = call i64 @llvm.vector.reduce.add.v8i64(<8 x i64> %xx)
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v8i16_v8i64_sext(<8 x i16> %x) {
 ; CHECK-LABEL: add_v8i16_v8i64_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.u16 r0, q0[0]
+; CHECK-NEXT:    vmov.u16 r1, q0[1]
+; CHECK-NEXT:    sxth r1, r1
 ; CHECK-NEXT:    sxth r0, r0
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
 ; CHECK-NEXT:    asrs r0, r0, #31
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-NEXT:    sxth r0, r0
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    asrs r1, r0, #31
-; CHECK-NEXT:    vmov.32 q1[3], r1
+; CHECK-NEXT:    asrs r2, r1, #31
+; CHECK-NEXT:    vmov q1[3], q1[1], r2, r0
 ; CHECK-NEXT:    vmov r2, s6
 ; CHECK-NEXT:    vmov r3, s4
-; CHECK-NEXT:    vmov r1, s5
+; CHECK-NEXT:    vmov r0, s5
 ; CHECK-NEXT:    adds r2, r2, r3
-; CHECK-NEXT:    adc.w r12, r1, r0, asr #31
+; CHECK-NEXT:    vmov.u16 r3, q0[3]
+; CHECK-NEXT:    adc.w r12, r0, r1, asr #31
 ; CHECK-NEXT:    vmov.u16 r1, q0[2]
+; CHECK-NEXT:    sxth r3, r3
 ; CHECK-NEXT:    sxth r1, r1
-; CHECK-NEXT:    vmov.32 q1[0], r1
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r1
 ; CHECK-NEXT:    asrs r1, r1, #31
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov.u16 r1, q0[3]
-; CHECK-NEXT:    sxth r1, r1
-; CHECK-NEXT:    vmov.32 q1[2], r1
-; CHECK-NEXT:    asrs r3, r1, #31
-; CHECK-NEXT:    vmov.32 q1[3], r3
-; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r3, s5
-; CHECK-NEXT:    adds r0, r0, r2
-; CHECK-NEXT:    adc.w r2, r12, r3
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    adds.w r12, r0, r3
-; CHECK-NEXT:    adc.w r1, r2, r1, asr #31
+; CHECK-NEXT:    asrs r0, r3, #31
+; CHECK-NEXT:    vmov q1[3], q1[1], r0, r1
+; CHECK-NEXT:    vmov r1, s4
+; CHECK-NEXT:    vmov r0, s5
+; CHECK-NEXT:    adds r1, r1, r2
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    adc.w r0, r0, r12
+; CHECK-NEXT:    adds r1, r1, r2
 ; CHECK-NEXT:    vmov.u16 r2, q0[4]
+; CHECK-NEXT:    adc.w r12, r0, r3, asr #31
+; CHECK-NEXT:    vmov.u16 r3, q0[5]
+; CHECK-NEXT:    sxth r3, r3
 ; CHECK-NEXT:    sxth r2, r2
-; CHECK-NEXT:    vmov.32 q1[0], r2
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q1[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q0[5]
-; CHECK-NEXT:    sxth r2, r2
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    asrs r3, r2, #31
-; CHECK-NEXT:    vmov.32 q1[3], r3
-; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r3, s5
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    adds r0, r0, r3
-; CHECK-NEXT:    adc.w r1, r1, r2, asr #31
+; CHECK-NEXT:    asrs r0, r3, #31
+; CHECK-NEXT:    vmov q1[3], q1[1], r0, r2
+; CHECK-NEXT:    vmov r2, s4
+; CHECK-NEXT:    vmov r0, s5
+; CHECK-NEXT:    adds r1, r1, r2
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    adc.w r0, r0, r12
+; CHECK-NEXT:    adds r1, r1, r2
 ; CHECK-NEXT:    vmov.u16 r2, q0[6]
+; CHECK-NEXT:    adc.w r0, r0, r3, asr #31
 ; CHECK-NEXT:    sxth r2, r2
-; CHECK-NEXT:    adds r0, r0, r2
-; CHECK-NEXT:    adc.w r1, r1, r2, asr #31
-; CHECK-NEXT:    vmov.u16 r2, q0[7]
-; CHECK-NEXT:    sxth r2, r2
-; CHECK-NEXT:    adds r0, r0, r2
-; CHECK-NEXT:    adc.w r1, r1, r2, asr #31
+; CHECK-NEXT:    adds r1, r1, r2
+; CHECK-NEXT:    adc.w r2, r0, r2, asr #31
+; CHECK-NEXT:    vmov.u16 r0, q0[7]
+; CHECK-NEXT:    sxth r3, r0
+; CHECK-NEXT:    adds r0, r1, r3
+; CHECK-NEXT:    adc.w r1, r2, r3, asr #31
 ; CHECK-NEXT:    bx lr
 entry:
   %xx = sext <8 x i16> %x to <8 x i64>
   %z = call i64 @llvm.vector.reduce.add.v8i64(<8 x i64> %xx)
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i16_v2i64_zext(<2 x i16> %x) {
entry:
   %z = call i8 @llvm.vector.reduce.add.v16i8(<16 x i8> %x)
   ret i8 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v16i8_v16i64_zext(<16 x i8> %x) {
 ; CHECK-LABEL: add_v16i8_v16i64_zext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.u8 r0, q0[0]
+; CHECK-NEXT:    vmov.u8 r1, q0[1]
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
 ; CHECK-NEXT:    vmov.i64 q1, #0xff
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[1]
-; CHECK-NEXT:    vmov.32 q2[2], r0
 ; CHECK-NEXT:    vand q2, q2, q1
+; CHECK-NEXT:    vmov.u8 r2, q0[3]
 ; CHECK-NEXT:    vmov r0, s10
 ; CHECK-NEXT:    vmov r1, s8
-; CHECK-NEXT:    vmov r2, s11
 ; CHECK-NEXT:    add r0, r1
 ; CHECK-NEXT:    vmov.u8 r1, q0[2]
-; CHECK-NEXT:    vmov.32 q3[0], r1
-; CHECK-NEXT:    vmov.u8 r1, q0[3]
-; CHECK-NEXT:    vmov.32 q3[2], r1
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r1
+; CHECK-NEXT:    vmov.u8 r2, q0[5]
 ; CHECK-NEXT:    vand q3, q3, q1
 ; CHECK-NEXT:    vmov r1, s12
 ; CHECK-NEXT:    add r0, r1
 ; CHECK-NEXT:    vmov r1, s14
 ; CHECK-NEXT:    add r0, r1
 ; CHECK-NEXT:    vmov.u8 r1, q0[4]
-; CHECK-NEXT:    vmov.32 q3[0], r1
-; CHECK-NEXT:    vmov.u8 r1, q0[5]
-; CHECK-NEXT:    vmov.32 q3[2], r1
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r1
+; CHECK-NEXT:    vmov r2, s11
 ; CHECK-NEXT:    vand q3, q3, q1
 ; CHECK-NEXT:    vmov r1, s12
 ; CHECK-NEXT:    vmov r3, s14
 ; CHECK-NEXT:    add r0, r1
 ; CHECK-NEXT:    vmov r1, s15
 ; CHECK-NEXT:    adds r0, r0, r3
+; CHECK-NEXT:    vmov.u8 r3, q0[7]
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov.u8 r2, q0[6]
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[7]
-; CHECK-NEXT:    vmov.32 q2[2], r2
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r2
 ; CHECK-NEXT:    vand q2, q2, q1
 ; CHECK-NEXT:    vmov r3, s8
 ; CHECK-NEXT:    vmov r2, s9
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    vmov r3, s10
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov r2, s11
 ; CHECK-NEXT:    adds r0, r0, r3
+; CHECK-NEXT:    vmov.u8 r3, q0[9]
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov.u8 r2, q0[8]
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[9]
-; CHECK-NEXT:    vmov.32 q2[2], r2
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r2
 ; CHECK-NEXT:    vand q2, q2, q1
 ; CHECK-NEXT:    vmov r3, s8
 ; CHECK-NEXT:    vmov r2, s9
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    vmov r3, s10
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov r2, s11
 ; CHECK-NEXT:    adds r0, r0, r3
+; CHECK-NEXT:    vmov.u8 r3, q0[11]
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov.u8 r2, q0[10]
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[11]
-; CHECK-NEXT:    vmov.32 q2[2], r2
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r2
 ; CHECK-NEXT:    vand q2, q2, q1
 ; CHECK-NEXT:    vmov r3, s8
 ; CHECK-NEXT:    vmov r2, s9
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    vmov r3, s10
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov r2, s11
 ; CHECK-NEXT:    adds r0, r0, r3
+; CHECK-NEXT:    vmov.u8 r3, q0[13]
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov.u8 r2, q0[12]
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[13]
-; CHECK-NEXT:    vmov.32 q2[2], r2
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r2
 ; CHECK-NEXT:    vand q2, q2, q1
 ; CHECK-NEXT:    vmov r3, s8
 ; CHECK-NEXT:    vmov r2, s9
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    vmov r3, s10
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov r2, s11
 ; CHECK-NEXT:    adds r0, r0, r3
+; CHECK-NEXT:    vmov.u8 r3, q0[15]
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov.u8 r2, q0[14]
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[15]
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vand q0, q2, q1
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov r2, s3
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    bx lr
 entry:
   %xx = zext <16 x i8> %x to <16 x i64>
   %z = call i64 @llvm.vector.reduce.add.v16i64(<16 x i64> %xx)
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v16i8_v16i64_sext(<16 x i8> %x) {
 ; CHECK-LABEL: add_v16i8_v16i64_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.u8 r0, q0[0]
+; CHECK-NEXT:    vmov.u8 r1, q0[1]
+; CHECK-NEXT:    sxtb r1, r1
 ; CHECK-NEXT:    sxtb r0, r0
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
 ; CHECK-NEXT:    asrs r0, r0, #31
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[1]
-; CHECK-NEXT:    sxtb r0, r0
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    asrs r1, r0, #31
-; CHECK-NEXT:    vmov.32 q1[3], r1
+; CHECK-NEXT:    asrs r2, r1, #31
+; CHECK-NEXT:    vmov q1[3], q1[1], r2, r0
 ; CHECK-NEXT:    vmov r2, s6
 ; CHECK-NEXT:    vmov r3, s4
-; CHECK-NEXT:    vmov r1, s5
+; CHECK-NEXT:    vmov r0, s5
 ; CHECK-NEXT:    adds r2, r2, r3
-; CHECK-NEXT:    adc.w r12, r1, r0, asr #31
+; CHECK-NEXT:    vmov.u8 r3, q0[3]
+; CHECK-NEXT:    adc.w r12, r0, r1, asr #31
 ; CHECK-NEXT:    vmov.u8 r1, q0[2]
+; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    sxtb r1, r1
-; CHECK-NEXT:    vmov.32 q1[0], r1
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r1
 ; CHECK-NEXT:    asrs r1, r1, #31
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov.u8 r1, q0[3]
-; CHECK-NEXT:    sxtb r1, r1
-; CHECK-NEXT:    vmov.32 q1[2], r1
-; CHECK-NEXT:    asrs r3, r1, #31
-; CHECK-NEXT:    vmov.32 q1[3], r3
-; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r3, s5
-; CHECK-NEXT:    adds r0, r0, r2
-; CHECK-NEXT:    adc.w r2, r12, r3
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    adds.w r12, r0, r3
-; CHECK-NEXT:    adc.w r1, r2, r1, asr #31
+; CHECK-NEXT:    asrs r0, r3, #31
+; CHECK-NEXT:    vmov q1[3], q1[1], r0, r1
+; CHECK-NEXT:    vmov r1, s4
+; CHECK-NEXT:    vmov r0, s5
+; CHECK-NEXT:    adds r1, r1, r2
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    adc.w r0, r0, r12
+; CHECK-NEXT:    adds r1, r1, r2
 ; CHECK-NEXT:    vmov.u8 r2, q0[4]
+; CHECK-NEXT:    adc.w r12, r0, r3, asr #31
+; CHECK-NEXT:    vmov.u8 r3, q0[5]
+; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q1[0], r2
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q1[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[5]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    asrs r3, r2, #31
-; CHECK-NEXT:    vmov.32 q1[3], r3
-; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r3, s5
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    adds.w r12, r0, r3
-; CHECK-NEXT:    adc.w r1, r1, r2, asr #31
+; CHECK-NEXT:    asrs r0, r3, #31
+; CHECK-NEXT:    vmov q1[3], q1[1], r0, r2
+; CHECK-NEXT:    vmov r2, s4
+; CHECK-NEXT:    vmov r0, s5
+; CHECK-NEXT:    adds r1, r1, r2
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    adc.w r0, r0, r12
+; CHECK-NEXT:    adds r1, r1, r2
 ; CHECK-NEXT:    vmov.u8 r2, q0[6]
+; CHECK-NEXT:    adc.w r12, r0, r3, asr #31
+; CHECK-NEXT:    vmov.u8 r3, q0[7]
+; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q1[0], r2
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q1[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[7]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    asrs r3, r2, #31
-; CHECK-NEXT:    vmov.32 q1[3], r3
-; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r3, s5
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    adds.w r12, r0, r3
-; CHECK-NEXT:    adc.w r1, r1, r2, asr #31
+; CHECK-NEXT:    asrs r0, r3, #31
+; CHECK-NEXT:    vmov q1[3], q1[1], r0, r2
+; CHECK-NEXT:    vmov r2, s4
+; CHECK-NEXT:    vmov r0, s5
+; CHECK-NEXT:    adds r1, r1, r2
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    adc.w r0, r0, r12
+; CHECK-NEXT:    adds r1, r1, r2
 ; CHECK-NEXT:    vmov.u8 r2, q0[8]
+; CHECK-NEXT:    adc.w r12, r0, r3, asr #31
+; CHECK-NEXT:    vmov.u8 r3, q0[9]
+; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q1[0], r2
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q1[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[9]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    asrs r3, r2, #31
-; CHECK-NEXT:    vmov.32 q1[3], r3
-; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r3, s5
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    adds.w r12, r0, r3
-; CHECK-NEXT:    adc.w r1, r1, r2, asr #31
+; CHECK-NEXT:    asrs r0, r3, #31
+; CHECK-NEXT:    vmov q1[3], q1[1], r0, r2
+; CHECK-NEXT:    vmov r2, s4
+; CHECK-NEXT:    vmov r0, s5
+; CHECK-NEXT:    adds r1, r1, r2
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    adc.w r0, r0, r12
+; CHECK-NEXT:    adds r1, r1, r2
 ; CHECK-NEXT:    vmov.u8 r2, q0[10]
+; CHECK-NEXT:    adc.w r12, r0, r3, asr #31
+; CHECK-NEXT:    vmov.u8 r3, q0[11]
+; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q1[0], r2
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q1[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[11]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    asrs r3, r2, #31
-; CHECK-NEXT:    vmov.32 q1[3], r3
-; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r3, s5
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    adds.w r12, r0, r3
-; CHECK-NEXT:    adc.w r1, r1, r2, asr #31
+; CHECK-NEXT:    asrs r0, r3, #31
+; CHECK-NEXT:    vmov q1[3], q1[1], r0, r2
+; CHECK-NEXT:    vmov r2, s4
+; CHECK-NEXT:    vmov r0, s5
+; CHECK-NEXT:    adds r1, r1, r2
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    adc.w r0, r0, r12
+; CHECK-NEXT:    adds r1, r1, r2
 ; CHECK-NEXT:    vmov.u8 r2, q0[12]
+; CHECK-NEXT:    adc.w r12, r0, r3, asr #31
+; CHECK-NEXT:    vmov.u8 r3, q0[13]
+; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q1[0], r2
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q1[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[13]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    asrs r3, r2, #31
-; CHECK-NEXT:    vmov.32 q1[3], r3
-; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r3, s5
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    adds r0, r0, r3
-; CHECK-NEXT:    adc.w r1, r1, r2, asr #31
+; CHECK-NEXT:    asrs r0, r3, #31
+; CHECK-NEXT:    vmov q1[3], q1[1], r0, r2
+; CHECK-NEXT:    vmov r2, s4
+; CHECK-NEXT:    vmov r0, s5
+; CHECK-NEXT:    adds r1, r1, r2
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    adc.w r0, r0, r12
+; CHECK-NEXT:    adds r1, r1, r2
 ; CHECK-NEXT:    vmov.u8 r2, q0[14]
+; CHECK-NEXT:    adc.w r0, r0, r3, asr #31
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    adds r0, r0, r2
-; CHECK-NEXT:    adc.w r1, r1, r2, asr #31
-; CHECK-NEXT:    vmov.u8 r2, q0[15]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    adds r0, r0, r2
-; CHECK-NEXT:    adc.w r1, r1, r2, asr #31
+; CHECK-NEXT:    adds r1, r1, r2
+; CHECK-NEXT:    adc.w r2, r0, r2, asr #31
+; CHECK-NEXT:    vmov.u8 r0, q0[15]
+; CHECK-NEXT:    sxtb r3, r0
+; CHECK-NEXT:    adds r0, r1, r3
+; CHECK-NEXT:    adc.w r1, r2, r3, asr #31
 ; CHECK-NEXT:    bx lr
 entry:
   %xx = sext <16 x i8> %x to <16 x i64>
   %z = call i64 @llvm.vector.reduce.add.v16i64(<16 x i64> %xx)
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i8_v2i64_zext(<2 x i8> %x) {

 }
 
 define arm_aapcs_vfpcc i64 @add_v8i16_v8i64_acc_zext(<8 x i16> %x, i64 %a) {
 ; CHECK-LABEL: add_v8i16_v8i64_acc_zext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, lr}
 ; CHECK-NEXT:    push {r4, lr}
 ; CHECK-NEXT:    vmov.u16 r2, q0[0]
+; CHECK-NEXT:    vmov.u16 r3, q0[1]
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r2
 ; CHECK-NEXT:    vmov.i64 q1, #0xffff
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q0[1]
-; CHECK-NEXT:    vmov.32 q2[2], r2
 ; CHECK-NEXT:    vand q2, q2, q1
 ; CHECK-NEXT:    vmov r2, s10
 ; CHECK-NEXT:    vmov r3, s8
-; CHECK-NEXT:    add r2, r3
+; CHECK-NEXT:    add.w r12, r3, r2
 ; CHECK-NEXT:    vmov.u16 r3, q0[2]
-; CHECK-NEXT:    vmov.32 q3[0], r3
-; CHECK-NEXT:    vmov.u16 r3, q0[3]
-; CHECK-NEXT:    vmov.32 q3[2], r3
+; CHECK-NEXT:    vmov.u16 r2, q0[3]
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r3
 ; CHECK-NEXT:    vand q3, q3, q1
-; CHECK-NEXT:    vmov r3, s12
-; CHECK-NEXT:    add r2, r3
+; CHECK-NEXT:    vmov r2, s12
 ; CHECK-NEXT:    vmov r3, s14
-; CHECK-NEXT:    add r2, r3
+; CHECK-NEXT:    add r2, r12
+; CHECK-NEXT:    add.w r12, r2, r3
 ; CHECK-NEXT:    vmov.u16 r3, q0[4]
-; CHECK-NEXT:    vmov.32 q3[0], r3
-; CHECK-NEXT:    vmov.u16 r3, q0[5]
-; CHECK-NEXT:    vmov.32 q3[2], r3
+; CHECK-NEXT:    vmov.u16 r2, q0[5]
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r3
 ; CHECK-NEXT:    vand q3, q3, q1
-; CHECK-NEXT:    vmov r3, s12
-; CHECK-NEXT:    vmov lr, s15
-; CHECK-NEXT:    add.w r12, r2, r3
+; CHECK-NEXT:    vmov r2, s12
 ; CHECK-NEXT:    vmov r3, s14
+; CHECK-NEXT:    vmov lr, s15
+; CHECK-NEXT:    add r12, r2
 ; CHECK-NEXT:    vmov r2, s11
 ; CHECK-NEXT:    adds.w r4, r12, r3
+; CHECK-NEXT:    vmov.u16 r3, q0[7]
 ; CHECK-NEXT:    adc.w r12, r2, lr
 ; CHECK-NEXT:    vmov.u16 r2, q0[6]
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q0[7]
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vand q0, q2, q1
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r3, r3, r4
 ; CHECK-NEXT:    vmov r4, s3
 ; CHECK-NEXT:    adc.w r12, r12, r2
 ; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    adc.w r3, r12, r4
 ; CHECK-NEXT:    adds r0, r0, r2
 ; CHECK-NEXT:    adcs r1, r3
 ; CHECK-NEXT:    pop {r4, pc}
 entry:
   %xx = zext <8 x i16> %x to <8 x i64>
   %z = call i64 @llvm.vector.reduce.add.v8i64(<8 x i64> %xx)
   %r = add i64 %z, %a
   ret i64 %r
 }
 
 define arm_aapcs_vfpcc i64 @add_v8i16_v8i64_acc_sext(<8 x i16> %x, i64 %a) {
 ; CHECK-LABEL: add_v8i16_v8i64_acc_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, lr}
 ; CHECK-NEXT:    push {r4, lr}
 ; CHECK-NEXT:    vmov.u16 r2, q0[0]
+; CHECK-NEXT:    vmov.u16 r3, q0[1]
 ; CHECK-NEXT:    sxth r2, r2
-; CHECK-NEXT:    vmov.32 q1[0], r2
-; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q1[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q0[1]
-; CHECK-NEXT:    sxth r2, r2
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    asrs r3, r2, #31
-; CHECK-NEXT:    vmov.32 q1[3], r3
+; CHECK-NEXT:    sxth r3, r3
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
+; CHECK-NEXT:    asr.w r12, r2, #31
+; CHECK-NEXT:    asrs r2, r3, #31
+; CHECK-NEXT:    vmov q1[3], q1[1], r2, r12
 ; CHECK-NEXT:    vmov lr, s6
-; CHECK-NEXT:    vmov r3, s4
+; CHECK-NEXT:    vmov r2, s4
 ; CHECK-NEXT:    vmov r12, s5
-; CHECK-NEXT:    adds.w lr, lr, r3
-; CHECK-NEXT:    adc.w r12, r12, r2, asr #31
+; CHECK-NEXT:    adds.w lr, lr, r2
 ; CHECK-NEXT:    vmov.u16 r2, q0[2]
+; CHECK-NEXT:    adc.w r12, r12, r3, asr #31
+; CHECK-NEXT:    vmov.u16 r3, q0[3]
+; CHECK-NEXT:    sxth r3, r3
 ; CHECK-NEXT:    sxth r2, r2
-; CHECK-NEXT:    vmov.32 q1[0], r2
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q1[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q0[3]
+; CHECK-NEXT:    asrs r4, r3, #31
+; CHECK-NEXT:    vmov q1[3], q1[1], r4, r2
+; CHECK-NEXT:    vmov r4, s4
+; CHECK-NEXT:    vmov r2, s5
+; CHECK-NEXT:    adds.w r4, r4, lr
+; CHECK-NEXT:    adc.w r12, r12, r2
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    adds.w lr, r4, r2
+; CHECK-NEXT:    vmov.u16 r4, q0[4]
+; CHECK-NEXT:    vmov.u16 r2, q0[5]
+; CHECK-NEXT:    sxth r4, r4
 ; CHECK-NEXT:    sxth r2, r2
-; CHECK-NEXT:    vmov.32 q1[2], r2
+; CHECK-NEXT:    adc.w r12, r12, r3, asr #31
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, r4
+; CHECK-NEXT:    asrs r4, r4, #31
 ; CHECK-NEXT:    asrs r3, r2, #31
-; CHECK-NEXT:    vmov.32 q1[3], r3
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r4
 ; CHECK-NEXT:    vmov r4, s4
 ; CHECK-NEXT:    vmov r3, s5
 ; CHECK-NEXT:    adds.w r4, r4, lr
 ; CHECK-NEXT:    adc.w r12, r12, r3
 ; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    adds.w lr, r4, r3
-; CHECK-NEXT:    vmov.u16 r4, q0[4]
-; CHECK-NEXT:    sxth r4, r4
-; CHECK-NEXT:    adc.w r12, r12, r2, asr #31
-; CHECK-NEXT:    vmov.32 q1[0], r4
-; CHECK-NEXT:    asrs r4, r4, #31
-; CHECK-NEXT:    vmov.32 q1[1], r4
-; CHECK-NEXT:    vmov.u16 r4, q0[5]
-; CHECK-NEXT:    sxth r4, r4
-; CHECK-NEXT:    vmov.32 q1[2], r4
-; CHECK-NEXT:    asrs r2, r4, #31
-; CHECK-NEXT:    vmov.32 q1[3], r2
-; CHECK-NEXT:    vmov r3, s4
-; CHECK-NEXT:    vmov r2, s5
-; CHECK-NEXT:    adds.w r3, r3, lr
-; CHECK-NEXT:    adc.w r12, r12, r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    adds r2, r2, r3
-; CHECK-NEXT:    adc.w r3, r12, r4, asr #31
+; CHECK-NEXT:    adds r3, r3, r4
 ; CHECK-NEXT:    vmov.u16 r4, q0[6]
+; CHECK-NEXT:    adc.w r2, r12, r2, asr #31
 ; CHECK-NEXT:    sxth r4, r4
-; CHECK-NEXT:    adds r2, r2, r4
-; CHECK-NEXT:    adc.w r3, r3, r4, asr #31
+; CHECK-NEXT:    adds r3, r3, r4
+; CHECK-NEXT:    adc.w r2, r2, r4, asr #31
 ; CHECK-NEXT:    vmov.u16 r4, q0[7]
 ; CHECK-NEXT:    sxth r4, r4
-; CHECK-NEXT:    adds r2, r2, r4
-; CHECK-NEXT:    adc.w r3, r3, r4, asr #31
-; CHECK-NEXT:    adds r0, r0, r2
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    adds r3, r3, r4
+; CHECK-NEXT:    adc.w r2, r2, r4, asr #31
+; CHECK-NEXT:    adds r0, r0, r3
+; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    pop {r4, pc}
 entry:
   %xx = sext <8 x i16> %x to <8 x i64>
   %z = call i64 @llvm.vector.reduce.add.v8i64(<8 x i64> %xx)
   %r = add i64 %z, %a
   ret i64 %r
 }
 

 }
 
 define arm_aapcs_vfpcc i64 @add_v16i8_v16i64_acc_zext(<16 x i8> %x, i64 %a) {
 ; CHECK-LABEL: add_v16i8_v16i64_acc_zext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, lr}
 ; CHECK-NEXT:    push {r4, lr}
 ; CHECK-NEXT:    vmov.u8 r2, q0[0]
+; CHECK-NEXT:    vmov.u8 r3, q0[1]
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r2
 ; CHECK-NEXT:    vmov.i64 q1, #0xff
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[1]
-; CHECK-NEXT:    vmov.32 q2[2], r2
 ; CHECK-NEXT:    vand q2, q2, q1
 ; CHECK-NEXT:    vmov r2, s10
 ; CHECK-NEXT:    vmov r3, s8
-; CHECK-NEXT:    add r2, r3
+; CHECK-NEXT:    add.w r12, r3, r2
 ; CHECK-NEXT:    vmov.u8 r3, q0[2]
-; CHECK-NEXT:    vmov.32 q3[0], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[3]
-; CHECK-NEXT:    vmov.32 q3[2], r3
+; CHECK-NEXT:    vmov.u8 r2, q0[3]
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r3
 ; CHECK-NEXT:    vand q3, q3, q1
-; CHECK-NEXT:    vmov r3, s12
-; CHECK-NEXT:    add r2, r3
+; CHECK-NEXT:    vmov r2, s12
 ; CHECK-NEXT:    vmov r3, s14
-; CHECK-NEXT:    add r2, r3
+; CHECK-NEXT:    add r2, r12
+; CHECK-NEXT:    add.w r12, r2, r3
 ; CHECK-NEXT:    vmov.u8 r3, q0[4]
-; CHECK-NEXT:    vmov.32 q3[0], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[5]
-; CHECK-NEXT:    vmov.32 q3[2], r3
+; CHECK-NEXT:    vmov.u8 r2, q0[5]
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r3
 ; CHECK-NEXT:    vand q3, q3, q1
-; CHECK-NEXT:    vmov r3, s12
-; CHECK-NEXT:    vmov lr, s15
-; CHECK-NEXT:    add.w r12, r2, r3
+; CHECK-NEXT:    vmov r2, s12
 ; CHECK-NEXT:    vmov r3, s14
+; CHECK-NEXT:    vmov lr, s15
+; CHECK-NEXT:    add r12, r2
 ; CHECK-NEXT:    vmov r2, s11
 ; CHECK-NEXT:    adds.w r4, r12, r3
+; CHECK-NEXT:    vmov.u8 r3, q0[7]
 ; CHECK-NEXT:    adc.w r12, r2, lr
 ; CHECK-NEXT:    vmov.u8 r2, q0[6]
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[7]
-; CHECK-NEXT:    vmov.32 q2[2], r2
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r2
 ; CHECK-NEXT:    vand q2, q2, q1
 ; CHECK-NEXT:    vmov r3, s8
 ; CHECK-NEXT:    vmov r2, s9
 ; CHECK-NEXT:    adds r3, r3, r4
 ; CHECK-NEXT:    vmov r4, s11
 ; CHECK-NEXT:    adc.w r12, r12, r2
 ; CHECK-NEXT:    vmov r2, s10
 ; CHECK-NEXT:    adds.w lr, r3, r2
+; CHECK-NEXT:    vmov.u8 r2, q0[9]
 ; CHECK-NEXT:    adc.w r3, r12, r4
 ; CHECK-NEXT:    vmov.u8 r4, q0[8]
-; CHECK-NEXT:    vmov.32 q2[0], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[9]
-; CHECK-NEXT:    vmov.32 q2[2], r4
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r4
 ; CHECK-NEXT:    vand q2, q2, q1
-; CHECK-NEXT:    vmov r2, s8
-; CHECK-NEXT:    vmov r4, s9
-; CHECK-NEXT:    adds.w r12, lr, r2
+; CHECK-NEXT:    vmov r4, s8
+; CHECK-NEXT:    vmov r2, s9
+; CHECK-NEXT:    adds.w r4, r4, lr
+; CHECK-NEXT:    adc.w r12, r3, r2
 ; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    adcs r3, r4
-; CHECK-NEXT:    vmov r4, s11
-; CHECK-NEXT:    adds.w r12, r12, r2
-; CHECK-NEXT:    adcs r3, r4
+; CHECK-NEXT:    vmov r3, s11
+; CHECK-NEXT:    adds.w lr, r4, r2
 ; CHECK-NEXT:    vmov.u8 r4, q0[10]
-; CHECK-NEXT:    vmov.32 q2[0], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[11]
-; CHECK-NEXT:    vmov.32 q2[2], r4
+; CHECK-NEXT:    vmov.u8 r2, q0[11]
+; CHECK-NEXT:    adc.w r3, r3, r12
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r4
 ; CHECK-NEXT:    vand q2, q2, q1
-; CHECK-NEXT:    vmov r2, s8
-; CHECK-NEXT:    vmov r4, s9
-; CHECK-NEXT:    adds.w r12, r12, r2
+; CHECK-NEXT:    vmov r4, s8
+; CHECK-NEXT:    vmov r2, s9
+; CHECK-NEXT:    adds.w r4, r4, lr
+; CHECK-NEXT:    adc.w r12, r3, r2
 ; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    adcs r3, r4
-; CHECK-NEXT:    vmov r4, s11
-; CHECK-NEXT:    adds.w r12, r12, r2
-; CHECK-NEXT:    adcs r3, r4
+; CHECK-NEXT:    vmov r3, s11
+; CHECK-NEXT:    adds.w lr, r4, r2
 ; CHECK-NEXT:    vmov.u8 r4, q0[12]
-; CHECK-NEXT:    vmov.32 q2[0], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[13]
-; CHECK-NEXT:    vmov.32 q2[2], r4
+; CHECK-NEXT:    vmov.u8 r2, q0[13]
+; CHECK-NEXT:    adc.w r3, r3, r12
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r4
 ; CHECK-NEXT:    vand q2, q2, q1
-; CHECK-NEXT:    vmov r2, s8
-; CHECK-NEXT:    vmov r4, s9
-; CHECK-NEXT:    adds.w r12, r12, r2
+; CHECK-NEXT:    vmov r4, s8
+; CHECK-NEXT:    vmov r2, s9
+; CHECK-NEXT:    adds.w r4, r4, lr
+; CHECK-NEXT:    adc.w r12, r3, r2
 ; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    adcs r3, r4
-; CHECK-NEXT:    vmov r4, s11
-; CHECK-NEXT:    adds.w r12, r12, r2
-; CHECK-NEXT:    adcs r3, r4
+; CHECK-NEXT:    vmov r3, s11
+; CHECK-NEXT:    adds.w lr, r4, r2
 ; CHECK-NEXT:    vmov.u8 r4, q0[14]
-; CHECK-NEXT:    vmov.32 q2[0], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[15]
-; CHECK-NEXT:    vmov.32 q2[2], r4
-; CHECK-NEXT:    vand q0, q2, q1
-; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    vmov r4, s1
-; CHECK-NEXT:    adds.w r12, r12, r2
+; CHECK-NEXT:    vmov.u8 r2, q0[15]
+; CHECK-NEXT:    adc.w r3, r3, r12
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r4
+; CHECK-NEXT:    vand q0, q0, q1
+; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov r2, s1
+; CHECK-NEXT:    adds.w r4, r4, lr
+; CHECK-NEXT:    adc.w r12, r3, r2
 ; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    adcs r3, r4
-; CHECK-NEXT:    vmov r4, s3
-; CHECK-NEXT:    adds.w r2, r2, r12
-; CHECK-NEXT:    adcs r3, r4
+; CHECK-NEXT:    vmov r3, s3
+; CHECK-NEXT:    adds r2, r2, r4
+; CHECK-NEXT:    adc.w r3, r3, r12
 ; CHECK-NEXT:    adds r0, r0, r2
 ; CHECK-NEXT:    adcs r1, r3
 ; CHECK-NEXT:    pop {r4, pc}
 entry:
   %xx = zext <16 x i8> %x to <16 x i64>
   %z = call i64 @llvm.vector.reduce.add.v16i64(<16 x i64> %xx)
   %r = add i64 %z, %a
   ret i64 %r
 }
 
 define arm_aapcs_vfpcc i64 @add_v16i8_v16i64_acc_sext(<16 x i8> %x, i64 %a) {
 ; CHECK-LABEL: add_v16i8_v16i64_acc_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, lr}
 ; CHECK-NEXT:    push {r4, lr}
 ; CHECK-NEXT:    vmov.u8 r2, q0[0]
+; CHECK-NEXT:    vmov.u8 r3, q0[1]
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q1[0], r2
-; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q1[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[1]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    asrs r3, r2, #31
-; CHECK-NEXT:    vmov.32 q1[3], r3
+; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
+; CHECK-NEXT:    asr.w r12, r2, #31
+; CHECK-NEXT:    asrs r2, r3, #31
+; CHECK-NEXT:    vmov q1[3], q1[1], r2, r12
 ; CHECK-NEXT:    vmov lr, s6
-; CHECK-NEXT:    vmov r3, s4
+; CHECK-NEXT:    vmov r2, s4
 ; CHECK-NEXT:    vmov r12, s5
-; CHECK-NEXT:    adds.w lr, lr, r3
-; CHECK-NEXT:    adc.w r12, r12, r2, asr #31
+; CHECK-NEXT:    adds.w lr, lr, r2
 ; CHECK-NEXT:    vmov.u8 r2, q0[2]
+; CHECK-NEXT:    adc.w r12, r12, r3, asr #31
+; CHECK-NEXT:    vmov.u8 r3, q0[3]
+; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q1[0], r2
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q1[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[3]
+; CHECK-NEXT:    asrs r4, r3, #31
+; CHECK-NEXT:    vmov q1[3], q1[1], r4, r2
+; CHECK-NEXT:    vmov r4, s4
+; CHECK-NEXT:    vmov r2, s5
+; CHECK-NEXT:    adds.w r4, r4, lr
+; CHECK-NEXT:    adc.w r12, r12, r2
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    adds.w lr, r4, r2
+; CHECK-NEXT:    vmov.u8 r4, q0[4]
+; CHECK-NEXT:    vmov.u8 r2, q0[5]
+; CHECK-NEXT:    sxtb r4, r4
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q1[2], r2
+; CHECK-NEXT:    adc.w r12, r12, r3, asr #31
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, r4
+; CHECK-NEXT:    asrs r4, r4, #31
 ; CHECK-NEXT:    asrs r3, r2, #31
-; CHECK-NEXT:    vmov.32 q1[3], r3
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r4
 ; CHECK-NEXT:    vmov r4, s4
 ; CHECK-NEXT:    vmov r3, s5
 ; CHECK-NEXT:    adds.w r4, r4, lr
 ; CHECK-NEXT:    adc.w r12, r12, r3
 ; CHECK-NEXT:    vmov r3, s6
 ; CHECK-NEXT:    adds.w lr, r4, r3
-; CHECK-NEXT:    vmov.u8 r4, q0[4]
-; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    adc.w r12, r12, r2, asr #31
-; CHECK-NEXT:    vmov.32 q1[0], r4
-; CHECK-NEXT:    asrs r4, r4, #31
-; CHECK-NEXT:    vmov.32 q1[1], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[5]
-; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    vmov.32 q1[2], r4
-; CHECK-NEXT:    asrs r2, r4, #31
-; CHECK-NEXT:    vmov.32 q1[3], r2
-; CHECK-NEXT:    vmov r3, s4
-; CHECK-NEXT:    vmov r2, s5
-; CHECK-NEXT:    adds.w r3, r3, lr
-; CHECK-NEXT:    adc.w r12, r12, r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    adds.w lr, r3, r2
-; CHECK-NEXT:    adc.w r12, r12, r4, asr #31
 ; CHECK-NEXT:    vmov.u8 r4, q0[6]
+; CHECK-NEXT:    adc.w r12, r12, r2, asr #31
+; CHECK-NEXT:    vmov.u8 r2, q0[7]
+; CHECK-NEXT:    sxtb r2, r2
 ; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    vmov.32 q1[0], r4
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, r4
 ; CHECK-NEXT:    asrs r4, r4, #31
-; CHECK-NEXT:    vmov.32 q1[1], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[7]
-; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    vmov.32 q1[2], r4
-; CHECK-NEXT:    asrs r2, r4, #31
-; CHECK-NEXT:    vmov.32 q1[3], r2
-; CHECK-NEXT:    vmov r3, s4
-; CHECK-NEXT:    vmov r2, s5
-; CHECK-NEXT:    adds.w r3, r3, lr
-; CHECK-NEXT:    adc.w r12, r12, r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    adds.w lr, r3, r2
-; CHECK-NEXT:    adc.w r12, r12, r4, asr #31
+; CHECK-NEXT:    asrs r3, r2, #31
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r4
+; CHECK-NEXT:    vmov r4, s4
+; CHECK-NEXT:    vmov r3, s5
+; CHECK-NEXT:    adds.w r4, r4, lr
+; CHECK-NEXT:    adc.w r12, r12, r3
+; CHECK-NEXT:    vmov r3, s6
+; CHECK-NEXT:    adds.w lr, r4, r3
 ; CHECK-NEXT:    vmov.u8 r4, q0[8]
+; CHECK-NEXT:    adc.w r12, r12, r2, asr #31
+; CHECK-NEXT:    vmov.u8 r2, q0[9]
+; CHECK-NEXT:    sxtb r2, r2
 ; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    vmov.32 q1[0], r4
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, r4
 ; CHECK-NEXT:    asrs r4, r4, #31
-; CHECK-NEXT:    vmov.32 q1[1], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[9]
-; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    vmov.32 q1[2], r4
-; CHECK-NEXT:    asrs r2, r4, #31
-; CHECK-NEXT:    vmov.32 q1[3], r2
-; CHECK-NEXT:    vmov r3, s4
-; CHECK-NEXT:    vmov r2, s5
-; CHECK-NEXT:    adds.w r3, r3, lr
-; CHECK-NEXT:    adc.w r12, r12, r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    adds.w lr, r3, r2
-; CHECK-NEXT:    adc.w r12, r12, r4, asr #31
+; CHECK-NEXT:    asrs r3, r2, #31
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r4
+; CHECK-NEXT:    vmov r4, s4
+; CHECK-NEXT:    vmov r3, s5
+; CHECK-NEXT:    adds.w r4, r4, lr
+; CHECK-NEXT:    adc.w r12, r12, r3
+; CHECK-NEXT:    vmov r3, s6
+; CHECK-NEXT:    adds.w lr, r4, r3
 ; CHECK-NEXT:    vmov.u8 r4, q0[10]
+; CHECK-NEXT:    adc.w r12, r12, r2, asr #31
+; CHECK-NEXT:    vmov.u8 r2, q0[11]
+; CHECK-NEXT:    sxtb r2, r2
 ; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    vmov.32 q1[0], r4
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, r4
 ; CHECK-NEXT:    asrs r4, r4, #31
-; CHECK-NEXT:    vmov.32 q1[1], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[11]
-; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    vmov.32 q1[2], r4
-; CHECK-NEXT:    asrs r2, r4, #31
-; CHECK-NEXT:    vmov.32 q1[3], r2
-; CHECK-NEXT:    vmov r3, s4
-; CHECK-NEXT:    vmov r2, s5
-; CHECK-NEXT:    adds.w r3, r3, lr
-; CHECK-NEXT:    adc.w r12, r12, r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    adds.w lr, r3, r2
-; CHECK-NEXT:    adc.w r12, r12, r4, asr #31
+; CHECK-NEXT:    asrs r3, r2, #31
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r4
+; CHECK-NEXT:    vmov r4, s4
+; CHECK-NEXT:    vmov r3, s5
+; CHECK-NEXT:    adds.w r4, r4, lr
+; CHECK-NEXT:    adc.w r12, r12, r3
+; CHECK-NEXT:    vmov r3, s6
+; CHECK-NEXT:    adds.w lr, r4, r3
 ; CHECK-NEXT:    vmov.u8 r4, q0[12]
+; CHECK-NEXT:    adc.w r12, r12, r2, asr #31
+; CHECK-NEXT:    vmov.u8 r2, q0[13]
+; CHECK-NEXT:    sxtb r2, r2
 ; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    vmov.32 q1[0], r4
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, r4
 ; CHECK-NEXT:    asrs r4, r4, #31
-; CHECK-NEXT:    vmov.32 q1[1], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[13]
-; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    vmov.32 q1[2], r4
-; CHECK-NEXT:    asrs r2, r4, #31
-; CHECK-NEXT:    vmov.32 q1[3], r2
-; CHECK-NEXT:    vmov r3, s4
-; CHECK-NEXT:    vmov r2, s5
-; CHECK-NEXT:    adds.w r3, r3, lr
-; CHECK-NEXT:    adc.w r12, r12, r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    adds r2, r2, r3
-; CHECK-NEXT:    adc.w r3, r12, r4, asr #31
+; CHECK-NEXT:    asrs r3, r2, #31
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r4
+; CHECK-NEXT:    vmov r4, s4
+; CHECK-NEXT:    vmov r3, s5
+; CHECK-NEXT:    adds.w r4, r4, lr
+; CHECK-NEXT:    adc.w r12, r12, r3
+; CHECK-NEXT:    vmov r3, s6
+; CHECK-NEXT:    adds r3, r3, r4
 ; CHECK-NEXT:    vmov.u8 r4, q0[14]
+; CHECK-NEXT:    adc.w r2, r12, r2, asr #31
 ; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    adds r2, r2, r4
-; CHECK-NEXT:    adc.w r3, r3, r4, asr #31
+; CHECK-NEXT:    adds r3, r3, r4
+; CHECK-NEXT:    adc.w r2, r2, r4, asr #31
 ; CHECK-NEXT:    vmov.u8 r4, q0[15]
 ; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    adds r2, r2, r4
-; CHECK-NEXT:    adc.w r3, r3, r4, asr #31
-; CHECK-NEXT:    adds r0, r0, r2
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    adds r3, r3, r4
+; CHECK-NEXT:    adc.w r2, r2, r4, asr #31
+; CHECK-NEXT:    adds r0, r0, r3
+; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    pop {r4, pc}
 entry:
   %xx = sext <16 x i8> %x to <16 x i64>
   %z = call i64 @llvm.vector.reduce.add.v16i64(<16 x i64> %xx)
   %r = add i64 %z, %a
   ret i64 %r
 }
 

llvm/test/CodeGen/Thumb2/mve-vecreduce-addpred.ll

entry:
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i32_v2i64_zext(<2 x i32> %x, <2 x i32> %b) {
 ; CHECK-LABEL: add_v2i32_v2i64_zext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s4
 ; CHECK-NEXT:    vmov.i64 q2, #0xffffffff
+; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s6
-; CHECK-NEXT:    cmp r0, #0
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    cmp r1, #0
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp eq <2 x i32> %b, zeroinitializer
   %xx = zext <2 x i32> %x to <2 x i64>
   %s = select <2 x i1> %c, <2 x i64> %xx, <2 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> %s)
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i32_v2i64_sext(<2 x i32> %x, <2 x i32> %b) {
 ; CHECK-LABEL: add_v2i32_v2i64_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s0
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    asrs r0, r0, #31
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vmov.32 q2[2], r0
+; CHECK-NEXT:    vmov r1, s2
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
 ; CHECK-NEXT:    asrs r0, r0, #31
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    asrs r1, r1, #31
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
 ; CHECK-NEXT:    vmov r0, s4
+; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.32 q0[1], r0
-; CHECK-NEXT:    vmov r0, s6
-; CHECK-NEXT:    cmp r0, #0
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    vmov.32 q0[3], r0
-; CHECK-NEXT:    vand q0, q2, q0
+; CHECK-NEXT:    cmp r1, #0
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    bx lr
 entry:

 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .vsave {d8, d9}
 ; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    vmov.i8 q2, #0x0
 ; CHECK-NEXT:    vmov.i8 q3, #0xff
 ; CHECK-NEXT:    vcmp.i16 eq, q1, zr
 ; CHECK-NEXT:    vpsel q2, q3, q2
 ; CHECK-NEXT:    vmov.u16 r0, q2[0]
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q2[2]
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q2[1]
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[2]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[3]
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q2[3]
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    vcmp.i32 ne, q1, zr
 ; CHECK-NEXT:    vmov.i64 q1, #0xffff
 ; CHECK-NEXT:    vmrs r0, p0
 ; CHECK-NEXT:    and r1, r0, #1
+; CHECK-NEXT:    ubfx r2, r0, #4, #1
 ; CHECK-NEXT:    rsbs r1, r1, #0
-; CHECK-NEXT:    vmov.32 q3[0], r1
-; CHECK-NEXT:    vmov.32 q3[1], r1
-; CHECK-NEXT:    ubfx r1, r0, #4, #1
-; CHECK-NEXT:    rsbs r1, r1, #0
-; CHECK-NEXT:    vmov.32 q3[2], r1
-; CHECK-NEXT:    vmov.32 q3[3], r1
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r1
+; CHECK-NEXT:    vmov q3[3], q3[1], r2, r1
 ; CHECK-NEXT:    vmov.u16 r1, q0[0]
-; CHECK-NEXT:    vmov.32 q4[0], r1
-; CHECK-NEXT:    vmov.u16 r1, q0[1]
-; CHECK-NEXT:    vmov.32 q4[2], r1
+; CHECK-NEXT:    vmov.u16 r2, q0[1]
+; CHECK-NEXT:    vmov q4[2], q4[0], r2, r1
 ; CHECK-NEXT:    vand q4, q4, q1
 ; CHECK-NEXT:    vand q3, q4, q3
 ; CHECK-NEXT:    vmov r1, s15
 ; CHECK-NEXT:    vmov r2, s13
 ; CHECK-NEXT:    vmov r3, s12
 ; CHECK-NEXT:    orrs r1, r2
 ; CHECK-NEXT:    vmov r2, s14
 ; CHECK-NEXT:    add r2, r3
 ; CHECK-NEXT:    ubfx r3, r0, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    ubfx r0, r0, #12, #1
-; CHECK-NEXT:    vmov.32 q3[0], r3
+; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.32 q3[1], r3
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    vmov q3[2], q3[0], r0, r3
+; CHECK-NEXT:    vmov q3[3], q3[1], r0, r3
 ; CHECK-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-NEXT:    vmov.32 q4[0], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[3]
-; CHECK-NEXT:    vmov.32 q4[2], r0
+; CHECK-NEXT:    vmov.u16 r3, q0[3]
+; CHECK-NEXT:    vmov q4[2], q4[0], r3, r0
 ; CHECK-NEXT:    vand q4, q4, q1
 ; CHECK-NEXT:    vand q3, q4, q3
 ; CHECK-NEXT:    vmov r3, s12
 ; CHECK-NEXT:    vmov r0, s13
 ; CHECK-NEXT:    adds r2, r2, r3
-; CHECK-NEXT:    vmov r3, s15
-; CHECK-NEXT:    adcs r1, r0
-; CHECK-NEXT:    vmov r0, s14
-; CHECK-NEXT:    adds.w r12, r2, r0
-; CHECK-NEXT:    vmov.u16 r2, q2[4]
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[5]
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[6]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[7]
-; CHECK-NEXT:    vmov.32 q3[3], r2
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    vmov r3, s14
+; CHECK-NEXT:    adcs r0, r1
+; CHECK-NEXT:    vmov r1, s15
+; CHECK-NEXT:    adds r2, r2, r3
+; CHECK-NEXT:    vmov.u16 r3, q2[6]
+; CHECK-NEXT:    adc.w r12, r0, r1
+; CHECK-NEXT:    vmov.u16 r1, q2[4]
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r1
+; CHECK-NEXT:    vmov.u16 r1, q2[5]
+; CHECK-NEXT:    vmov.u16 r3, q2[7]
+; CHECK-NEXT:    vmov q3[3], q3[1], r3, r1
 ; CHECK-NEXT:    vcmp.i32 ne, q3, zr
-; CHECK-NEXT:    vmrs r2, p0
-; CHECK-NEXT:    and r3, r2, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q2[0], r3
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    ubfx r3, r2, #4, #1
+; CHECK-NEXT:    vmrs r1, p0
+; CHECK-NEXT:    and r3, r1, #1
+; CHECK-NEXT:    ubfx r0, r1, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q2[2], r3
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    vmov.u16 r3, q0[4]
-; CHECK-NEXT:    vmov.32 q3[0], r3
+; CHECK-NEXT:    rsbs r0, r0, #0
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r0, r3
+; CHECK-NEXT:    vmov.u16 r0, q0[4]
 ; CHECK-NEXT:    vmov.u16 r3, q0[5]
-; CHECK-NEXT:    vmov.32 q3[2], r3
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r0
 ; CHECK-NEXT:    vand q3, q3, q1
 ; CHECK-NEXT:    vand q2, q3, q2
-; CHECK-NEXT:    vmov r0, s8
-; CHECK-NEXT:    vmov r3, s9
-; CHECK-NEXT:    adds.w r12, r12, r0
-; CHECK-NEXT:    vmov r0, s10
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    vmov r3, s8
+; CHECK-NEXT:    vmov r0, s9
+; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    vmov r3, s11
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    ubfx r3, r2, #8, #1
+; CHECK-NEXT:    adc.w r12, r12, r0
+; CHECK-NEXT:    vmov r0, s10
+; CHECK-NEXT:    adds r0, r0, r2
+; CHECK-NEXT:    adc.w r2, r12, r3
+; CHECK-NEXT:    ubfx r3, r1, #8, #1
+; CHECK-NEXT:    ubfx r1, r1, #12, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q2[0], r3
-; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r2
-; CHECK-NEXT:    vmov.u16 r2, q0[6]
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q0[7]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vand q0, q3, q1
+; CHECK-NEXT:    rsbs r1, r1, #0
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r3
+; CHECK-NEXT:    vmov.u16 r1, q0[6]
+; CHECK-NEXT:    vmov.u16 r3, q0[7]
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r1
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vmov r3, s0
-; CHECK-NEXT:    vmov r2, s1
+; CHECK-NEXT:    vmov r1, s1
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov r2, s3
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vpop {d8, d9}
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp eq <8 x i16> %b, zeroinitializer
   %xx = zext <8 x i16> %x to <8 x i64>
   %s = select <8 x i1> %c, <8 x i64> %xx, <8 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v8i64(<8 x i64> %s)
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v8i16_v8i64_sext(<8 x i16> %x, <8 x i16> %b) {
 ; CHECK-LABEL: add_v8i16_v8i64_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.i8 q2, #0x0
 ; CHECK-NEXT:    vmov.i8 q3, #0xff
 ; CHECK-NEXT:    vcmp.i16 eq, q1, zr
 ; CHECK-NEXT:    vpsel q1, q3, q2
 ; CHECK-NEXT:    vmov.u16 r0, q1[0]
-; CHECK-NEXT:    vmov.32 q2[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q1[2]
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q1[1]
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[2]
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[3]
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q1[3]
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vcmp.i32 ne, q2, zr
 ; CHECK-NEXT:    vmrs r0, p0
 ; CHECK-NEXT:    and r1, r0, #1
+; CHECK-NEXT:    ubfx r2, r0, #4, #1
 ; CHECK-NEXT:    rsbs r1, r1, #0
-; CHECK-NEXT:    vmov.32 q2[0], r1
-; CHECK-NEXT:    vmov.32 q2[1], r1
-; CHECK-NEXT:    ubfx r1, r0, #4, #1
-; CHECK-NEXT:    rsbs r1, r1, #0
-; CHECK-NEXT:    vmov.32 q2[2], r1
-; CHECK-NEXT:    vmov.32 q2[3], r1
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r1
+; CHECK-NEXT:    vmov q2[3], q2[1], r2, r1
 ; CHECK-NEXT:    vmov.u16 r1, q0[0]
+; CHECK-NEXT:    vmov.u16 r2, q0[1]
 ; CHECK-NEXT:    sxth r1, r1
-; CHECK-NEXT:    vmov.32 q3[0], r1
-; CHECK-NEXT:    asrs r1, r1, #31
-; CHECK-NEXT:    vmov.32 q3[1], r1
-; CHECK-NEXT:    vmov.u16 r1, q0[1]
-; CHECK-NEXT:    sxth r1, r1
-; CHECK-NEXT:    vmov.32 q3[2], r1
+; CHECK-NEXT:    sxth r2, r2
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r1
 ; CHECK-NEXT:    asrs r1, r1, #31
-; CHECK-NEXT:    vmov.32 q3[3], r1
+; CHECK-NEXT:    asrs r2, r2, #31
+; CHECK-NEXT:    vmov q3[3], q3[1], r2, r1
 ; CHECK-NEXT:    vand q2, q3, q2
-; CHECK-NEXT:    vmov r1, s10
-; CHECK-NEXT:    vmov r2, s8
+; CHECK-NEXT:    vmov r3, s10
+; CHECK-NEXT:    vmov r1, s8
 ; CHECK-NEXT:    vmov r12, s11
-; CHECK-NEXT:    vmov r3, s9
-; CHECK-NEXT:    adds r1, r1, r2
-; CHECK-NEXT:    adc.w r2, r3, r12
+; CHECK-NEXT:    vmov r2, s9
+; CHECK-NEXT:    adds r1, r1, r3
 ; CHECK-NEXT:    ubfx r3, r0, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    ubfx r0, r0, #12, #1
-; CHECK-NEXT:    vmov.32 q2[0], r3
-; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    rsb.w r3, r3, #0
+; CHECK-NEXT:    rsb.w r0, r0, #0
+; CHECK-NEXT:    adc.w r2, r2, r12
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r0, r3
 ; CHECK-NEXT:    vmov.u16 r0, q0[2]
+; CHECK-NEXT:    vmov.u16 r3, q0[3]
 ; CHECK-NEXT:    sxth r0, r0
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    asrs r0, r0, #31
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[3]
-; CHECK-NEXT:    sxth r0, r0
-; CHECK-NEXT:    vmov.32 q3[2], r0
+; CHECK-NEXT:    sxth r3, r3
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r0
 ; CHECK-NEXT:    asrs r0, r0, #31
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    asrs r3, r3, #31
+; CHECK-NEXT:    vmov q3[3], q3[1], r3, r0
 ; CHECK-NEXT:    vand q2, q3, q2
 ; CHECK-NEXT:    vmov r3, s8
 ; CHECK-NEXT:    vmov r0, s9
 ; CHECK-NEXT:    adds r1, r1, r3
 ; CHECK-NEXT:    vmov r3, s11
 ; CHECK-NEXT:    adcs r2, r0
 ; CHECK-NEXT:    vmov r0, s10
 ; CHECK-NEXT:    adds.w r12, r1, r0
 ; CHECK-NEXT:    adc.w r1, r2, r3
 ; CHECK-NEXT:    vmov.u16 r2, q1[4]
-; CHECK-NEXT:    vmov.32 q2[0], r2
+; CHECK-NEXT:    vmov.u16 r3, q1[6]
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q1[5]
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[6]
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[7]
-; CHECK-NEXT:    vmov.32 q2[3], r2
+; CHECK-NEXT:    vmov.u16 r3, q1[7]
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q2, zr
 ; CHECK-NEXT:    vmrs r2, p0
 ; CHECK-NEXT:    and r3, r2, #1
+; CHECK-NEXT:    ubfx r0, r2, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q1[0], r3
-; CHECK-NEXT:    vmov.32 q1[1], r3
-; CHECK-NEXT:    ubfx r3, r2, #4, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q1[2], r3
-; CHECK-NEXT:    vmov.32 q1[3], r3
-; CHECK-NEXT:    vmov.u16 r3, q0[4]
-; CHECK-NEXT:    sxth r3, r3
-; CHECK-NEXT:    vmov.32 q2[0], r3
-; CHECK-NEXT:    asrs r3, r3, #31
-; CHECK-NEXT:    vmov.32 q2[1], r3
+; CHECK-NEXT:    rsbs r0, r0, #0
+; CHECK-NEXT:    vmov q1[2], q1[0], r0, r3
+; CHECK-NEXT:    vmov q1[3], q1[1], r0, r3
+; CHECK-NEXT:    vmov.u16 r0, q0[4]
 ; CHECK-NEXT:    vmov.u16 r3, q0[5]
+; CHECK-NEXT:    sxth r0, r0
 ; CHECK-NEXT:    sxth r3, r3
-; CHECK-NEXT:    vmov.32 q2[2], r3
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r0
+; CHECK-NEXT:    asrs r0, r0, #31
 ; CHECK-NEXT:    asrs r3, r3, #31
-; CHECK-NEXT:    vmov.32 q2[3], r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r0
 ; CHECK-NEXT:    vand q1, q2, q1
-; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r3, s5
-; CHECK-NEXT:    adds.w r12, r12, r0
+; CHECK-NEXT:    vmov r3, s4
+; CHECK-NEXT:    vmov r0, s5
+; CHECK-NEXT:    adds.w r3, r3, r12
+; CHECK-NEXT:    adc.w r12, r1, r0
 ; CHECK-NEXT:    vmov r0, s6
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vmov r3, s7
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    vmov r1, s7
+; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q1[0], r3
-; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q1[1], r3
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vmov.32 q1[3], r2
+; CHECK-NEXT:    rsb.w r3, r3, #0
+; CHECK-NEXT:    rsb.w r2, r2, #0
+; CHECK-NEXT:    adc.w r1, r1, r12
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, r3
+; CHECK-NEXT:    vmov q1[3], q1[1], r2, r3
 ; CHECK-NEXT:    vmov.u16 r2, q0[6]
+; CHECK-NEXT:    vmov.u16 r3, q0[7]
 ; CHECK-NEXT:    sxth r2, r2
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q0[7]
-; CHECK-NEXT:    sxth r2, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
+; CHECK-NEXT:    sxth r3, r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q2[3], r2
-; CHECK-NEXT:    vand q0, q2, q1
+; CHECK-NEXT:    asrs r3, r3, #31
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r2
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov r2, s3
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp eq <8 x i16> %b, zeroinitializer
   %xx = sext <8 x i16> %x to <8 x i64>
   %s = select <8 x i1> %c, <8 x i64> %xx, <8 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v8i64(<8 x i64> %s)
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i16_v2i64_zext(<2 x i16> %x, <2 x i16> %b) {
 ; CHECK-LABEL: add_v2i16_v2i64_zext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.i64 q2, #0xffff
 ; CHECK-NEXT:    vand q1, q1, q2
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vmov r0, s4
+; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov r0, s6
-; CHECK-NEXT:    cmp r0, #0
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vand q0, q0, q3
+; CHECK-NEXT:    cmp r1, #0
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    add r0, r1
 ; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    orrs r1, r2
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp eq <2 x i16> %b, zeroinitializer
   %xx = zext <2 x i16> %x to <2 x i64>
   %s = select <2 x i1> %c, <2 x i64> %xx, <2 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> %s)
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i16_v2i64_sext(<2 x i16> %x, <2 x i16> %b) {
 ; CHECK-LABEL: add_v2i16_v2i64_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.i32 q2, #0xffff
-; CHECK-NEXT:    vand q2, q1, q2
-; CHECK-NEXT:    vmov r0, s8
-; CHECK-NEXT:    cmp r0, #0
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q1[0], r0
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov r0, s10
+; CHECK-NEXT:    vand q1, q1, q2
+; CHECK-NEXT:    vmov r0, s4
+; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    cmp r1, #0
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov r1, s2
 ; CHECK-NEXT:    sxth r0, r0
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    asrs r0, r0, #31
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    sxth r0, r0
-; CHECK-NEXT:    vmov.32 q2[2], r0
+; CHECK-NEXT:    sxth r1, r1
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
 ; CHECK-NEXT:    asrs r0, r0, #31
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vand q0, q2, q1
+; CHECK-NEXT:    asrs r1, r1, #31
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    bx lr
 entry:

 ; CHECK-NEXT:    vmov.16 q1[5], r0
 ; CHECK-NEXT:    vmov.u8 r0, q4[6]
 ; CHECK-NEXT:    vmov.16 q1[6], r0
 ; CHECK-NEXT:    vmov.u8 r0, q4[7]
 ; CHECK-NEXT:    vmov.16 q1[7], r0
 ; CHECK-NEXT:    vcmp.i16 ne, q1, zr
 ; CHECK-NEXT:    vpsel q5, q3, q2
 ; CHECK-NEXT:    vmov.u16 r0, q5[0]
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q5[2]
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q5[1]
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q5[2]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q5[3]
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q5[3]
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    vcmp.i32 ne, q1, zr
 ; CHECK-NEXT:    vmov.i64 q1, #0xff
 ; CHECK-NEXT:    vmrs r0, p0
 ; CHECK-NEXT:    and r1, r0, #1
+; CHECK-NEXT:    ubfx r2, r0, #4, #1
 ; CHECK-NEXT:    rsbs r1, r1, #0
-; CHECK-NEXT:    vmov.32 q6[0], r1
-; CHECK-NEXT:    vmov.32 q6[1], r1
-; CHECK-NEXT:    ubfx r1, r0, #4, #1
-; CHECK-NEXT:    rsbs r1, r1, #0
-; CHECK-NEXT:    vmov.32 q6[2], r1
-; CHECK-NEXT:    vmov.32 q6[3], r1
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov q6[2], q6[0], r2, r1
+; CHECK-NEXT:    vmov q6[3], q6[1], r2, r1
 ; CHECK-NEXT:    vmov.u8 r1, q0[0]
-; CHECK-NEXT:    vmov.32 q7[0], r1
-; CHECK-NEXT:    vmov.u8 r1, q0[1]
-; CHECK-NEXT:    vmov.32 q7[2], r1
+; CHECK-NEXT:    vmov.u8 r2, q0[1]
+; CHECK-NEXT:    vmov q7[2], q7[0], r2, r1
 ; CHECK-NEXT:    vand q7, q7, q1
 ; CHECK-NEXT:    vand q6, q7, q6
 ; CHECK-NEXT:    vmov r1, s27
 ; CHECK-NEXT:    vmov r2, s25
 ; CHECK-NEXT:    vmov r3, s24
 ; CHECK-NEXT:    orrs r1, r2
 ; CHECK-NEXT:    vmov r2, s26
 ; CHECK-NEXT:    add r2, r3
 ; CHECK-NEXT:    ubfx r3, r0, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    ubfx r0, r0, #12, #1
-; CHECK-NEXT:    vmov.32 q6[0], r3
+; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.32 q6[1], r3
-; CHECK-NEXT:    vmov.32 q6[2], r0
-; CHECK-NEXT:    vmov.32 q6[3], r0
+; CHECK-NEXT:    vmov q6[2], q6[0], r0, r3
+; CHECK-NEXT:    vmov q6[3], q6[1], r0, r3
 ; CHECK-NEXT:    vmov.u8 r0, q0[2]
-; CHECK-NEXT:    vmov.32 q7[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[3]
-; CHECK-NEXT:    vmov.32 q7[2], r0
+; CHECK-NEXT:    vmov.u8 r3, q0[3]
+; CHECK-NEXT:    vmov q7[2], q7[0], r3, r0
 ; CHECK-NEXT:    vand q7, q7, q1
 ; CHECK-NEXT:    vand q6, q7, q6
 ; CHECK-NEXT:    vmov r3, s24
 ; CHECK-NEXT:    vmov r0, s25
 ; CHECK-NEXT:    adds r2, r2, r3
-; CHECK-NEXT:    vmov r3, s27
-; CHECK-NEXT:    adcs r1, r0
-; CHECK-NEXT:    vmov r0, s26
-; CHECK-NEXT:    adds.w r12, r2, r0
-; CHECK-NEXT:    vmov.u16 r2, q5[4]
-; CHECK-NEXT:    vmov.32 q6[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q5[5]
-; CHECK-NEXT:    vmov.32 q6[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q5[6]
-; CHECK-NEXT:    vmov.32 q6[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q5[7]
-; CHECK-NEXT:    vmov.32 q6[3], r2
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    vmov r3, s26
+; CHECK-NEXT:    adcs r0, r1
+; CHECK-NEXT:    vmov r1, s27
+; CHECK-NEXT:    adds r2, r2, r3
+; CHECK-NEXT:    vmov.u16 r3, q5[6]
+; CHECK-NEXT:    adc.w r12, r0, r1
+; CHECK-NEXT:    vmov.u16 r1, q5[4]
+; CHECK-NEXT:    vmov q6[2], q6[0], r3, r1
+; CHECK-NEXT:    vmov.u16 r1, q5[5]
+; CHECK-NEXT:    vmov.u16 r3, q5[7]
+; CHECK-NEXT:    vmov q6[3], q6[1], r3, r1
 ; CHECK-NEXT:    vcmp.i32 ne, q6, zr
-; CHECK-NEXT:    vmrs r2, p0
-; CHECK-NEXT:    and r3, r2, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q5[0], r3
-; CHECK-NEXT:    vmov.32 q5[1], r3
-; CHECK-NEXT:    ubfx r3, r2, #4, #1
+; CHECK-NEXT:    vmrs r1, p0
+; CHECK-NEXT:    and r3, r1, #1
+; CHECK-NEXT:    ubfx r0, r1, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q5[2], r3
-; CHECK-NEXT:    vmov.32 q5[3], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[4]
-; CHECK-NEXT:    vmov.32 q6[0], r3
+; CHECK-NEXT:    rsbs r0, r0, #0
+; CHECK-NEXT:    vmov q5[2], q5[0], r0, r3
+; CHECK-NEXT:    vmov q5[3], q5[1], r0, r3
+; CHECK-NEXT:    vmov.u8 r0, q0[4]
 ; CHECK-NEXT:    vmov.u8 r3, q0[5]
-; CHECK-NEXT:    vmov.32 q6[2], r3
+; CHECK-NEXT:    vmov q6[2], q6[0], r3, r0
 ; CHECK-NEXT:    vand q6, q6, q1
 ; CHECK-NEXT:    vand q5, q6, q5
-; CHECK-NEXT:    vmov r0, s20
-; CHECK-NEXT:    vmov r3, s21
-; CHECK-NEXT:    adds.w r12, r12, r0
-; CHECK-NEXT:    vmov r0, s22
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    vmov r3, s20
+; CHECK-NEXT:    vmov r0, s21
+; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    vmov r3, s23
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    ubfx r3, r2, #8, #1
+; CHECK-NEXT:    adc.w r12, r12, r0
+; CHECK-NEXT:    vmov r0, s22
+; CHECK-NEXT:    adds r0, r0, r2
+; CHECK-NEXT:    adc.w r2, r12, r3
+; CHECK-NEXT:    ubfx r3, r1, #8, #1
+; CHECK-NEXT:    ubfx r1, r1, #12, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q5[0], r3
-; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q5[1], r3
-; CHECK-NEXT:    vmov.32 q5[2], r2
-; CHECK-NEXT:    vmov.32 q5[3], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[6]
-; CHECK-NEXT:    vmov.32 q6[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[7]
-; CHECK-NEXT:    vmov.32 q6[2], r2
+; CHECK-NEXT:    rsbs r1, r1, #0
+; CHECK-NEXT:    vmov q5[2], q5[0], r1, r3
+; CHECK-NEXT:    vmov q5[3], q5[1], r1, r3
+; CHECK-NEXT:    vmov.u8 r1, q0[6]
+; CHECK-NEXT:    vmov.u8 r3, q0[7]
+; CHECK-NEXT:    vmov q6[2], q6[0], r3, r1
 ; CHECK-NEXT:    vand q6, q6, q1
 ; CHECK-NEXT:    vand q5, q6, q5
 ; CHECK-NEXT:    vmov r3, s20
-; CHECK-NEXT:    vmov r2, s21
+; CHECK-NEXT:    vmov r1, s21
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    vmov r3, s22
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov r2, s23
 ; CHECK-NEXT:    adds.w r12, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[8]
 ; CHECK-NEXT:    vmov.16 q5[0], r2

 ; CHECK-NEXT:    vmov.16 q5[5], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[14]
 ; CHECK-NEXT:    vmov.16 q5[6], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[15]
 ; CHECK-NEXT:    vmov.16 q5[7], r2
 ; CHECK-NEXT:    vcmp.i16 ne, q5, zr
 ; CHECK-NEXT:    vpsel q2, q3, q2
 ; CHECK-NEXT:    vmov.u16 r2, q2[0]
-; CHECK-NEXT:    vmov.32 q3[0], r2
+; CHECK-NEXT:    vmov.u16 r3, q2[2]
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q2[1]
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[2]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[3]
-; CHECK-NEXT:    vmov.32 q3[3], r2
+; CHECK-NEXT:    vmov.u16 r3, q2[3]
+; CHECK-NEXT:    vmov q3[3], q3[1], r3, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q3, zr
 ; CHECK-NEXT:    vmrs r2, p0
 ; CHECK-NEXT:    and r3, r2, #1
+; CHECK-NEXT:    ubfx r0, r2, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q3[0], r3
-; CHECK-NEXT:    vmov.32 q3[1], r3
-; CHECK-NEXT:    ubfx r3, r2, #4, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q3[2], r3
-; CHECK-NEXT:    vmov.32 q3[3], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[8]
-; CHECK-NEXT:    vmov.32 q4[0], r3
+; CHECK-NEXT:    rsbs r0, r0, #0
+; CHECK-NEXT:    vmov q3[2], q3[0], r0, r3
+; CHECK-NEXT:    vmov q3[3], q3[1], r0, r3
+; CHECK-NEXT:    vmov.u8 r0, q0[8]
 ; CHECK-NEXT:    vmov.u8 r3, q0[9]
-; CHECK-NEXT:    vmov.32 q4[2], r3
+; CHECK-NEXT:    vmov q4[2], q4[0], r3, r0
 ; CHECK-NEXT:    vand q4, q4, q1
 ; CHECK-NEXT:    vand q3, q4, q3
-; CHECK-NEXT:    vmov r0, s12
-; CHECK-NEXT:    vmov r3, s13
-; CHECK-NEXT:    adds.w r12, r12, r0
+; CHECK-NEXT:    vmov r3, s12
+; CHECK-NEXT:    vmov r0, s13
+; CHECK-NEXT:    adds.w r3, r3, r12
+; CHECK-NEXT:    adc.w r12, r1, r0
 ; CHECK-NEXT:    vmov r0, s14
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vmov r3, s15
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    vmov r1, s15
+; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q3[0], r3
-; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q3[1], r3
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.32 q3[3], r2
+; CHECK-NEXT:    rsb.w r3, r3, #0
+; CHECK-NEXT:    rsb.w r2, r2, #0
+; CHECK-NEXT:    adc.w r1, r1, r12
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r3
+; CHECK-NEXT:    vmov q3[3], q3[1], r2, r3
 ; CHECK-NEXT:    vmov.u8 r2, q0[10]
-; CHECK-NEXT:    vmov.32 q4[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[11]
-; CHECK-NEXT:    vmov.32 q4[2], r2
+; CHECK-NEXT:    vmov.u8 r3, q0[11]
+; CHECK-NEXT:    vmov q4[2], q4[0], r3, r2
 ; CHECK-NEXT:    vand q4, q4, q1
 ; CHECK-NEXT:    vand q3, q4, q3
 ; CHECK-NEXT:    vmov r3, s12
 ; CHECK-NEXT:    vmov r2, s13
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    vmov r3, s14
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov r2, s15
 ; CHECK-NEXT:    adds.w r12, r0, r3
+; CHECK-NEXT:    vmov.u16 r3, q2[6]
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov.u16 r2, q2[4]
-; CHECK-NEXT:    vmov.32 q3[0], r2
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q2[5]
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[6]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[7]
-; CHECK-NEXT:    vmov.32 q3[3], r2
+; CHECK-NEXT:    vmov.u16 r3, q2[7]
+; CHECK-NEXT:    vmov q3[3], q3[1], r3, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q3, zr
 ; CHECK-NEXT:    vmrs r2, p0
 ; CHECK-NEXT:    and r3, r2, #1
+; CHECK-NEXT:    ubfx r0, r2, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q2[0], r3
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    ubfx r3, r2, #4, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q2[2], r3
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[12]
-; CHECK-NEXT:    vmov.32 q3[0], r3
+; CHECK-NEXT:    rsbs r0, r0, #0
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r0, r3
+; CHECK-NEXT:    vmov.u8 r0, q0[12]
 ; CHECK-NEXT:    vmov.u8 r3, q0[13]
-; CHECK-NEXT:    vmov.32 q3[2], r3
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r0
 ; CHECK-NEXT:    vand q3, q3, q1
 ; CHECK-NEXT:    vand q2, q3, q2
-; CHECK-NEXT:    vmov r0, s8
-; CHECK-NEXT:    vmov r3, s9
-; CHECK-NEXT:    adds.w r12, r12, r0
+; CHECK-NEXT:    vmov r3, s8
+; CHECK-NEXT:    vmov r0, s9
+; CHECK-NEXT:    adds.w r3, r3, r12
+; CHECK-NEXT:    adc.w r12, r1, r0
 ; CHECK-NEXT:    vmov r0, s10
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vmov r3, s11
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    vmov r1, s11
+; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q2[0], r3
-; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r2
+; CHECK-NEXT:    rsb.w r3, r3, #0
+; CHECK-NEXT:    rsb.w r2, r2, #0
+; CHECK-NEXT:    adc.w r1, r1, r12
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r2, r3
 ; CHECK-NEXT:    vmov.u8 r2, q0[14]
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[15]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vand q0, q3, q1
+; CHECK-NEXT:    vmov.u8 r3, q0[15]
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov r2, s3
 ; CHECK-NEXT:    adds r0, r0, r3

 ; CHECK-NEXT:    vmov.16 q4[5], r0
 ; CHECK-NEXT:    vmov.u8 r0, q3[6]
 ; CHECK-NEXT:    vmov.16 q4[6], r0
 ; CHECK-NEXT:    vmov.u8 r0, q3[7]
 ; CHECK-NEXT:    vmov.16 q4[7], r0
 ; CHECK-NEXT:    vcmp.i16 ne, q4, zr
 ; CHECK-NEXT:    vpsel q4, q2, q1
 ; CHECK-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-NEXT:    vmov.32 q5[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q4[2]
+; CHECK-NEXT:    vmov q5[2], q5[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q4[1]
-; CHECK-NEXT:    vmov.32 q5[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-NEXT:    vmov.32 q5[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-NEXT:    vmov.32 q5[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q4[3]
+; CHECK-NEXT:    vmov q5[3], q5[1], r1, r0
 ; CHECK-NEXT:    vcmp.i32 ne, q5, zr
 ; CHECK-NEXT:    vmrs r0, p0
 ; CHECK-NEXT:    and r1, r0, #1
+; CHECK-NEXT:    ubfx r2, r0, #4, #1
 ; CHECK-NEXT:    rsbs r1, r1, #0
-; CHECK-NEXT:    vmov.32 q5[0], r1
-; CHECK-NEXT:    vmov.32 q5[1], r1
-; CHECK-NEXT:    ubfx r1, r0, #4, #1
-; CHECK-NEXT:    rsbs r1, r1, #0
-; CHECK-NEXT:    vmov.32 q5[2], r1
-; CHECK-NEXT:    vmov.32 q5[3], r1
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov q5[2], q5[0], r2, r1
+; CHECK-NEXT:    vmov q5[3], q5[1], r2, r1
 ; CHECK-NEXT:    vmov.u8 r1, q0[0]
+; CHECK-NEXT:    vmov.u8 r2, q0[1]
 ; CHECK-NEXT:    sxtb r1, r1
-; CHECK-NEXT:    vmov.32 q6[0], r1
-; CHECK-NEXT:    asrs r1, r1, #31
-; CHECK-NEXT:    vmov.32 q6[1], r1
-; CHECK-NEXT:    vmov.u8 r1, q0[1]
-; CHECK-NEXT:    sxtb r1, r1
-; CHECK-NEXT:    vmov.32 q6[2], r1
+; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    vmov q6[2], q6[0], r2, r1
 ; CHECK-NEXT:    asrs r1, r1, #31
-; CHECK-NEXT:    vmov.32 q6[3], r1
+; CHECK-NEXT:    asrs r2, r2, #31
+; CHECK-NEXT:    vmov q6[3], q6[1], r2, r1
 ; CHECK-NEXT:    vand q5, q6, q5
-; CHECK-NEXT:    vmov r1, s22
-; CHECK-NEXT:    vmov r2, s20
+; CHECK-NEXT:    vmov r3, s22
+; CHECK-NEXT:    vmov r1, s20
 ; CHECK-NEXT:    vmov r12, s23
-; CHECK-NEXT:    vmov r3, s21
-; CHECK-NEXT:    adds r1, r1, r2
-; CHECK-NEXT:    adc.w r2, r3, r12
+; CHECK-NEXT:    vmov r2, s21
+; CHECK-NEXT:    adds r1, r1, r3
 ; CHECK-NEXT:    ubfx r3, r0, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    ubfx r0, r0, #12, #1
-; CHECK-NEXT:    vmov.32 q5[0], r3
-; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.32 q5[1], r3
-; CHECK-NEXT:    vmov.32 q5[2], r0
-; CHECK-NEXT:    vmov.32 q5[3], r0
+; CHECK-NEXT:    rsb.w r3, r3, #0
+; CHECK-NEXT:    rsb.w r0, r0, #0
+; CHECK-NEXT:    adc.w r2, r2, r12
+; CHECK-NEXT:    vmov q5[2], q5[0], r0, r3
+; CHECK-NEXT:    vmov q5[3], q5[1], r0, r3
 ; CHECK-NEXT:    vmov.u8 r0, q0[2]
+; CHECK-NEXT:    vmov.u8 r3, q0[3]
 ; CHECK-NEXT:    sxtb r0, r0
-; CHECK-NEXT:    vmov.32 q6[0], r0
-; CHECK-NEXT:    asrs r0, r0, #31
-; CHECK-NEXT:    vmov.32 q6[1], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[3]
-; CHECK-NEXT:    sxtb r0, r0
-; CHECK-NEXT:    vmov.32 q6[2], r0
+; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    vmov q6[2], q6[0], r3, r0
 ; CHECK-NEXT:    asrs r0, r0, #31
-; CHECK-NEXT:    vmov.32 q6[3], r0
+; CHECK-NEXT:    asrs r3, r3, #31
+; CHECK-NEXT:    vmov q6[3], q6[1], r3, r0
 ; CHECK-NEXT:    vand q5, q6, q5
 ; CHECK-NEXT:    vmov r3, s20
 ; CHECK-NEXT:    vmov r0, s21
 ; CHECK-NEXT:    adds r1, r1, r3
 ; CHECK-NEXT:    vmov r3, s23
 ; CHECK-NEXT:    adcs r2, r0
 ; CHECK-NEXT:    vmov r0, s22
 ; CHECK-NEXT:    adds.w r12, r1, r0
 ; CHECK-NEXT:    adc.w r1, r2, r3
 ; CHECK-NEXT:    vmov.u16 r2, q4[4]
-; CHECK-NEXT:    vmov.32 q5[0], r2
+; CHECK-NEXT:    vmov.u16 r3, q4[6]
+; CHECK-NEXT:    vmov q5[2], q5[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q4[5]
-; CHECK-NEXT:    vmov.32 q5[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q4[6]
-; CHECK-NEXT:    vmov.32 q5[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q4[7]
-; CHECK-NEXT:    vmov.32 q5[3], r2
+; CHECK-NEXT:    vmov.u16 r3, q4[7]
+; CHECK-NEXT:    vmov q5[3], q5[1], r3, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q5, zr
 ; CHECK-NEXT:    vmrs r2, p0
 ; CHECK-NEXT:    and r3, r2, #1
+; CHECK-NEXT:    ubfx r0, r2, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q4[0], r3
-; CHECK-NEXT:    vmov.32 q4[1], r3
-; CHECK-NEXT:    ubfx r3, r2, #4, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q4[2], r3
-; CHECK-NEXT:    vmov.32 q4[3], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[4]
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    vmov.32 q5[0], r3
-; CHECK-NEXT:    asrs r3, r3, #31
-; CHECK-NEXT:    vmov.32 q5[1], r3
+; CHECK-NEXT:    rsbs r0, r0, #0
+; CHECK-NEXT:    vmov q4[2], q4[0], r0, r3
+; CHECK-NEXT:    vmov q4[3], q4[1], r0, r3
+; CHECK-NEXT:    vmov.u8 r0, q0[4]
 ; CHECK-NEXT:    vmov.u8 r3, q0[5]
+; CHECK-NEXT:    sxtb r0, r0
 ; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    vmov.32 q5[2], r3
+; CHECK-NEXT:    vmov q5[2], q5[0], r3, r0
+; CHECK-NEXT:    asrs r0, r0, #31
 ; CHECK-NEXT:    asrs r3, r3, #31
-; CHECK-NEXT:    vmov.32 q5[3], r3
+; CHECK-NEXT:    vmov q5[3], q5[1], r3, r0
 ; CHECK-NEXT:    vand q4, q5, q4
-; CHECK-NEXT:    vmov r0, s16
-; CHECK-NEXT:    vmov r3, s17
-; CHECK-NEXT:    adds.w r12, r12, r0
+; CHECK-NEXT:    vmov r3, s16
+; CHECK-NEXT:    vmov r0, s17
+; CHECK-NEXT:    adds.w r3, r3, r12
+; CHECK-NEXT:    adc.w r12, r1, r0
 ; CHECK-NEXT:    vmov r0, s18
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vmov r3, s19
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    vmov r1, s19
+; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q4[0], r3
-; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q4[1], r3
-; CHECK-NEXT:    vmov.32 q4[2], r2
-; CHECK-NEXT:    vmov.32 q4[3], r2
+; CHECK-NEXT:    rsb.w r3, r3, #0
+; CHECK-NEXT:    rsb.w r2, r2, #0
+; CHECK-NEXT:    adc.w r1, r1, r12
+; CHECK-NEXT:    vmov q4[2], q4[0], r2, r3
+; CHECK-NEXT:    vmov q4[3], q4[1], r2, r3
 ; CHECK-NEXT:    vmov.u8 r2, q0[6]
+; CHECK-NEXT:    vmov.u8 r3, q0[7]
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q5[0], r2
-; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q5[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[7]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q5[2], r2
+; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    vmov q5[2], q5[0], r3, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q5[3], r2
+; CHECK-NEXT:    asrs r3, r3, #31
+; CHECK-NEXT:    vmov q5[3], q5[1], r3, r2
 ; CHECK-NEXT:    vand q4, q5, q4
 ; CHECK-NEXT:    vmov r3, s16
 ; CHECK-NEXT:    vmov r2, s17
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    vmov r3, s18
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov r2, s19
 ; CHECK-NEXT:    adds.w r12, r0, r3

 ; CHECK-NEXT:    vmov.16 q4[5], r2
 ; CHECK-NEXT:    vmov.u8 r2, q3[14]
 ; CHECK-NEXT:    vmov.16 q4[6], r2
 ; CHECK-NEXT:    vmov.u8 r2, q3[15]
 ; CHECK-NEXT:    vmov.16 q4[7], r2
 ; CHECK-NEXT:    vcmp.i16 ne, q4, zr
 ; CHECK-NEXT:    vpsel q1, q2, q1
 ; CHECK-NEXT:    vmov.u16 r2, q1[0]
-; CHECK-NEXT:    vmov.32 q2[0], r2
+; CHECK-NEXT:    vmov.u16 r3, q1[2]
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q1[1]
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[2]
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[3]
-; CHECK-NEXT:    vmov.32 q2[3], r2
+; CHECK-NEXT:    vmov.u16 r3, q1[3]
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q2, zr
 ; CHECK-NEXT:    vmrs r2, p0
 ; CHECK-NEXT:    and r3, r2, #1
+; CHECK-NEXT:    ubfx r0, r2, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q2[0], r3
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    ubfx r3, r2, #4, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q2[2], r3
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[8]
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    vmov.32 q3[0], r3
-; CHECK-NEXT:    asrs r3, r3, #31
-; CHECK-NEXT:    vmov.32 q3[1], r3
+; CHECK-NEXT:    rsbs r0, r0, #0
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r0, r3
+; CHECK-NEXT:    vmov.u8 r0, q0[8]
 ; CHECK-NEXT:    vmov.u8 r3, q0[9]
+; CHECK-NEXT:    sxtb r0, r0
 ; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    vmov.32 q3[2], r3
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r0
+; CHECK-NEXT:    asrs r0, r0, #31
 ; CHECK-NEXT:    asrs r3, r3, #31
-; CHECK-NEXT:    vmov.32 q3[3], r3
+; CHECK-NEXT:    vmov q3[3], q3[1], r3, r0
 ; CHECK-NEXT:    vand q2, q3, q2
-; CHECK-NEXT:    vmov r0, s8
-; CHECK-NEXT:    vmov r3, s9
-; CHECK-NEXT:    adds.w r12, r12, r0
+; CHECK-NEXT:    vmov r3, s8
+; CHECK-NEXT:    vmov r0, s9
+; CHECK-NEXT:    adds.w r3, r3, r12
+; CHECK-NEXT:    adc.w r12, r1, r0
 ; CHECK-NEXT:    vmov r0, s10
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vmov r3, s11
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    vmov r1, s11
+; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q2[0], r3
-; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r2
+; CHECK-NEXT:    rsb.w r3, r3, #0
+; CHECK-NEXT:    rsb.w r2, r2, #0
+; CHECK-NEXT:    adc.w r1, r1, r12
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r2, r3
 ; CHECK-NEXT:    vmov.u8 r2, q0[10]
+; CHECK-NEXT:    vmov.u8 r3, q0[11]
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[11]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q3[2], r2
+; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q3[3], r2
+; CHECK-NEXT:    asrs r3, r3, #31
+; CHECK-NEXT:    vmov q3[3], q3[1], r3, r2
 ; CHECK-NEXT:    vand q2, q3, q2
 ; CHECK-NEXT:    vmov r3, s8
 ; CHECK-NEXT:    vmov r2, s9
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    vmov r3, s10
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov r2, s11
 ; CHECK-NEXT:    adds.w r12, r0, r3
+; CHECK-NEXT:    vmov.u16 r3, q1[6]
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov.u16 r2, q1[4]
-; CHECK-NEXT:    vmov.32 q2[0], r2
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q1[5]
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[6]
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[7]
-; CHECK-NEXT:    vmov.32 q2[3], r2
+; CHECK-NEXT:    vmov.u16 r3, q1[7]
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q2, zr
 ; CHECK-NEXT:    vmrs r2, p0
 ; CHECK-NEXT:    and r3, r2, #1
+; CHECK-NEXT:    ubfx r0, r2, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q1[0], r3
-; CHECK-NEXT:    vmov.32 q1[1], r3
-; CHECK-NEXT:    ubfx r3, r2, #4, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q1[2], r3
-; CHECK-NEXT:    vmov.32 q1[3], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[12]
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    vmov.32 q2[0], r3
-; CHECK-NEXT:    asrs r3, r3, #31
-; CHECK-NEXT:    vmov.32 q2[1], r3
+; CHECK-NEXT:    rsbs r0, r0, #0
+; CHECK-NEXT:    vmov q1[2], q1[0], r0, r3
+; CHECK-NEXT:    vmov q1[3], q1[1], r0, r3
+; CHECK-NEXT:    vmov.u8 r0, q0[12]
 ; CHECK-NEXT:    vmov.u8 r3, q0[13]
+; CHECK-NEXT:    sxtb r0, r0
 ; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    vmov.32 q2[2], r3
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r0
+; CHECK-NEXT:    asrs r0, r0, #31
 ; CHECK-NEXT:    asrs r3, r3, #31
-; CHECK-NEXT:    vmov.32 q2[3], r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r0
 ; CHECK-NEXT:    vand q1, q2, q1
-; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r3, s5
-; CHECK-NEXT:    adds.w r12, r12, r0
+; CHECK-NEXT:    vmov r3, s4
+; CHECK-NEXT:    vmov r0, s5
+; CHECK-NEXT:    adds.w r3, r3, r12
+; CHECK-NEXT:    adc.w r12, r1, r0
 ; CHECK-NEXT:    vmov r0, s6
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vmov r3, s7
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    vmov r1, s7
+; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q1[0], r3
-; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q1[1], r3
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vmov.32 q1[3], r2
+; CHECK-NEXT:    rsb.w r3, r3, #0
+; CHECK-NEXT:    rsb.w r2, r2, #0
+; CHECK-NEXT:    adc.w r1, r1, r12
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, r3
+; CHECK-NEXT:    vmov q1[3], q1[1], r2, r3
 ; CHECK-NEXT:    vmov.u8 r2, q0[14]
+; CHECK-NEXT:    vmov.u8 r3, q0[15]
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[15]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
+; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q2[3], r2
-; CHECK-NEXT:    vand q0, q2, q1
+; CHECK-NEXT:    asrs r3, r3, #31
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r2
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vmov r2, s3
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2

 
 define arm_aapcs_vfpcc i64 @add_v2i8_v2i64_zext(<2 x i8> %x, <2 x i8> %b) {
 ; CHECK-LABEL: add_v2i8_v2i64_zext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.i64 q2, #0xff
 ; CHECK-NEXT:    vand q1, q1, q2
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vmov r0, s4
+; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov r0, s6
-; CHECK-NEXT:    cmp r0, #0
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vand q0, q0, q3
+; CHECK-NEXT:    cmp r1, #0
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    add r0, r1
 ; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    orrs r1, r2
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp eq <2 x i8> %b, zeroinitializer
   %xx = zext <2 x i8> %x to <2 x i64>
   %s = select <2 x i1> %c, <2 x i64> %xx, <2 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> %s)
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i8_v2i64_sext(<2 x i8> %x, <2 x i8> %b) {
 ; CHECK-LABEL: add_v2i8_v2i64_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.i32 q2, #0xff
-; CHECK-NEXT:    vand q2, q1, q2
-; CHECK-NEXT:    vmov r0, s8
-; CHECK-NEXT:    cmp r0, #0
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q1[0], r0
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov r0, s10
+; CHECK-NEXT:    vand q1, q1, q2
+; CHECK-NEXT:    vmov r0, s4
+; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    cmp r1, #0
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov r1, s2
 ; CHECK-NEXT:    sxtb r0, r0
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    asrs r0, r0, #31
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    sxtb r0, r0
-; CHECK-NEXT:    vmov.32 q2[2], r0
+; CHECK-NEXT:    sxtb r1, r1
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
 ; CHECK-NEXT:    asrs r0, r0, #31
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vand q0, q2, q1
+; CHECK-NEXT:    asrs r1, r1, #31
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp eq <2 x i8> %b, zeroinitializer
   %xx = sext <2 x i8> %x to <2 x i64>
   %s = select <2 x i1> %c, <2 x i64> %xx, <2 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> %s)
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i64_v2i64(<2 x i64> %x, <2 x i64> %b) {
 ; CHECK-LABEL: add_v2i64_v2i64:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s5
 ; CHECK-NEXT:    vmov r1, s4
+; CHECK-NEXT:    vmov r2, s6
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s6
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s7
-; CHECK-NEXT:    orrs r0, r1
+; CHECK-NEXT:    vmov r1, s7
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    bx lr
 entry:

 
 define arm_aapcs_vfpcc i64 @add_v2i32_v2i64_acc_zext(<2 x i32> %x, <2 x i32> %b, i64 %a) {
 ; CHECK-LABEL: add_v2i32_v2i64_acc_zext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
 ; CHECK-NEXT:    vmov r2, s4
 ; CHECK-NEXT:    vmov.i64 q2, #0xffffffff
+; CHECK-NEXT:    vmov r3, s6
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    cmp r2, #0
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    cmp r2, #0
-; CHECK-NEXT:    cset r2, eq
-; CHECK-NEXT:    tst.w r2, #1
-; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r2
-; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    cset r3, eq
+; CHECK-NEXT:    tst.w r3, #1
+; CHECK-NEXT:    csetm r3, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r2
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r12, s3
 ; CHECK-NEXT:    vmov lr, s1
 ; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    adc.w r3, lr, r12
 ; CHECK-NEXT:    adds r0, r0, r2
 ; CHECK-NEXT:    adcs r1, r3
 ; CHECK-NEXT:    pop {r7, pc}
 entry:
   %c = icmp eq <2 x i32> %b, zeroinitializer
   %xx = zext <2 x i32> %x to <2 x i64>
   %s = select <2 x i1> %c, <2 x i64> %xx, <2 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> %s)
   %r = add i64 %z, %a
   ret i64 %r
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i32_v2i64_acc_sext(<2 x i32> %x, <2 x i32> %b, i64 %a) {
 ; CHECK-LABEL: add_v2i32_v2i64_acc_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
 ; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    vmov.32 q2[2], r2
+; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q2[3], r2
+; CHECK-NEXT:    asrs r3, r3, #31
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r2
 ; CHECK-NEXT:    vmov r2, s4
+; CHECK-NEXT:    vmov r3, s6
 ; CHECK-NEXT:    cmp r2, #0
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    cmp r2, #0
-; CHECK-NEXT:    cset r2, eq
-; CHECK-NEXT:    tst.w r2, #1
-; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vmov.32 q0[3], r2
-; CHECK-NEXT:    vand q0, q2, q0
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    cset r3, eq
+; CHECK-NEXT:    tst.w r3, #1
+; CHECK-NEXT:    csetm r3, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r2
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r12, s3
 ; CHECK-NEXT:    vmov lr, s1
 ; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    adc.w r3, lr, r12
 ; CHECK-NEXT:    adds r0, r0, r2
 ; CHECK-NEXT:    adcs r1, r3

 ; CHECK-NEXT:    push {r4, lr}
 ; CHECK-NEXT:    .vsave {d8, d9}
 ; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    vmov.i8 q2, #0x0
 ; CHECK-NEXT:    vmov.i8 q3, #0xff
 ; CHECK-NEXT:    vcmp.i16 eq, q1, zr
 ; CHECK-NEXT:    vpsel q2, q3, q2
 ; CHECK-NEXT:    vmov.u16 r2, q2[0]
-; CHECK-NEXT:    vmov.32 q1[0], r2
+; CHECK-NEXT:    vmov.u16 r3, q2[2]
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q2[1]
-; CHECK-NEXT:    vmov.32 q1[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[2]
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[3]
-; CHECK-NEXT:    vmov.32 q1[3], r2
+; CHECK-NEXT:    vmov.u16 r3, q2[3]
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q1, zr
 ; CHECK-NEXT:    vmov.i64 q1, #0xffff
 ; CHECK-NEXT:    vmrs r2, p0
 ; CHECK-NEXT:    and r3, r2, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q3[0], r3
-; CHECK-NEXT:    vmov.32 q3[1], r3
+; CHECK-NEXT:    rsb.w r12, r3, #0
 ; CHECK-NEXT:    ubfx r3, r2, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q3[2], r3
-; CHECK-NEXT:    vmov.32 q3[3], r3
-; CHECK-NEXT:    vmov.u16 r3, q0[0]
-; CHECK-NEXT:    vmov.32 q4[0], r3
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r12
+; CHECK-NEXT:    vmov q3[3], q3[1], r3, r12
+; CHECK-NEXT:    vmov.u16 r12, q0[0]
 ; CHECK-NEXT:    vmov.u16 r3, q0[1]
-; CHECK-NEXT:    vmov.32 q4[2], r3
+; CHECK-NEXT:    vmov q4[2], q4[0], r3, r12
 ; CHECK-NEXT:    vand q4, q4, q1
 ; CHECK-NEXT:    vand q3, q4, q3
 ; CHECK-NEXT:    vmov r12, s15
 ; CHECK-NEXT:    vmov r3, s13
 ; CHECK-NEXT:    vmov lr, s14
 ; CHECK-NEXT:    orr.w r12, r12, r3
 ; CHECK-NEXT:    vmov r3, s12
 ; CHECK-NEXT:    add lr, r3
 ; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q3[0], r3
+; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q3[1], r3
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.32 q3[3], r2
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r3
+; CHECK-NEXT:    vmov q3[3], q3[1], r2, r3
 ; CHECK-NEXT:    vmov.u16 r2, q0[2]
-; CHECK-NEXT:    vmov.32 q4[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q0[3]
-; CHECK-NEXT:    vmov.32 q4[2], r2
+; CHECK-NEXT:    vmov.u16 r3, q0[3]
+; CHECK-NEXT:    vmov q4[2], q4[0], r3, r2
 ; CHECK-NEXT:    vand q4, q4, q1
 ; CHECK-NEXT:    vand q3, q4, q3
 ; CHECK-NEXT:    vmov r3, s12
 ; CHECK-NEXT:    vmov r2, s13
-; CHECK-NEXT:    adds.w r4, lr, r3
+; CHECK-NEXT:    adds.w lr, lr, r3
 ; CHECK-NEXT:    vmov r3, s14
-; CHECK-NEXT:    adc.w lr, r12, r2
+; CHECK-NEXT:    adc.w r12, r12, r2
 ; CHECK-NEXT:    vmov r2, s15
-; CHECK-NEXT:    adds.w r12, r4, r3
-; CHECK-NEXT:    adc.w lr, lr, r2
+; CHECK-NEXT:    adds.w lr, lr, r3
+; CHECK-NEXT:    vmov.u16 r3, q2[6]
+; CHECK-NEXT:    adc.w r12, r12, r2
 ; CHECK-NEXT:    vmov.u16 r2, q2[4]
-; CHECK-NEXT:    vmov.32 q3[0], r2
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q2[5]
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[6]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[7]
-; CHECK-NEXT:    vmov.32 q3[3], r2
+; CHECK-NEXT:    vmov.u16 r3, q2[7]
+; CHECK-NEXT:    vmov q3[3], q3[1], r3, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q3, zr
 ; CHECK-NEXT:    vmrs r2, p0
-; CHECK-NEXT:    and r4, r2, #1
-; CHECK-NEXT:    rsbs r4, r4, #0
-; CHECK-NEXT:    vmov.32 q2[0], r4
-; CHECK-NEXT:    vmov.32 q2[1], r4
+; CHECK-NEXT:    and r3, r2, #1
 ; CHECK-NEXT:    ubfx r4, r2, #4, #1
+; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    rsbs r4, r4, #0
-; CHECK-NEXT:    vmov.32 q2[2], r4
-; CHECK-NEXT:    vmov.32 q2[3], r4
-; CHECK-NEXT:    vmov.u16 r4, q0[4]
-; CHECK-NEXT:    vmov.32 q3[0], r4
+; CHECK-NEXT:    vmov q2[2], q2[0], r4, r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r4, r3
+; CHECK-NEXT:    vmov.u16 r3, q0[4]
 ; CHECK-NEXT:    vmov.u16 r4, q0[5]
-; CHECK-NEXT:    vmov.32 q3[2], r4
+; CHECK-NEXT:    vmov q3[2], q3[0], r4, r3
 ; CHECK-NEXT:    vand q3, q3, q1
 ; CHECK-NEXT:    vand q2, q3, q2
-; CHECK-NEXT:    vmov r3, s8
-; CHECK-NEXT:    vmov r4, s9
-; CHECK-NEXT:    adds.w r12, r12, r3
-; CHECK-NEXT:    vmov r3, s11
-; CHECK-NEXT:    adc.w lr, lr, r4
+; CHECK-NEXT:    vmov r4, s8
+; CHECK-NEXT:    vmov r3, s9
+; CHECK-NEXT:    adds.w lr, lr, r4
 ; CHECK-NEXT:    vmov r4, s10
-; CHECK-NEXT:    adds.w r4, r4, r12
-; CHECK-NEXT:    adc.w r12, lr, r3
+; CHECK-NEXT:    adc.w r12, r12, r3
+; CHECK-NEXT:    vmov r3, s11
+; CHECK-NEXT:    adds.w r4, r4, lr
+; CHECK-NEXT:    adc.w r12, r12, r3
 ; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q2[0], r3
+; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r2
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r2, r3
 ; CHECK-NEXT:    vmov.u16 r2, q0[6]
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q0[7]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vand q0, q3, q1
+; CHECK-NEXT:    vmov.u16 r3, q0[7]
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r3, r3, r4
 ; CHECK-NEXT:    vmov r4, s3
 ; CHECK-NEXT:    adc.w r12, r12, r2
 ; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    adds r2, r2, r3

 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r7, lr}
 ; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    vmov.i8 q2, #0x0
 ; CHECK-NEXT:    vmov.i8 q3, #0xff
 ; CHECK-NEXT:    vcmp.i16 eq, q1, zr
 ; CHECK-NEXT:    vpsel q1, q3, q2
 ; CHECK-NEXT:    vmov.u16 r2, q1[0]
-; CHECK-NEXT:    vmov.32 q2[0], r2
+; CHECK-NEXT:    vmov.u16 r3, q1[2]
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q1[1]
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[2]
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[3]
-; CHECK-NEXT:    vmov.32 q2[3], r2
+; CHECK-NEXT:    vmov.u16 r3, q1[3]
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q2, zr
-; CHECK-NEXT:    vmrs r2, p0
-; CHECK-NEXT:    and r3, r2, #1
+; CHECK-NEXT:    vmrs r12, p0
+; CHECK-NEXT:    and r3, r12, #1
+; CHECK-NEXT:    ubfx r2, r12, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q2[0], r3
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    ubfx r3, r2, #4, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q2[2], r3
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    vmov.u16 r3, q0[0]
-; CHECK-NEXT:    sxth r3, r3
-; CHECK-NEXT:    vmov.32 q3[0], r3
-; CHECK-NEXT:    asrs r3, r3, #31
-; CHECK-NEXT:    vmov.32 q3[1], r3
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r2, r3
+; CHECK-NEXT:    vmov.u16 r2, q0[0]
 ; CHECK-NEXT:    vmov.u16 r3, q0[1]
+; CHECK-NEXT:    sxth r2, r2
 ; CHECK-NEXT:    sxth r3, r3
-; CHECK-NEXT:    vmov.32 q3[2], r3
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r2
+; CHECK-NEXT:    asrs r2, r2, #31
 ; CHECK-NEXT:    asrs r3, r3, #31
-; CHECK-NEXT:    vmov.32 q3[3], r3
+; CHECK-NEXT:    vmov q3[3], q3[1], r3, r2
 ; CHECK-NEXT:    vand q2, q3, q2
-; CHECK-NEXT:    vmov r3, s10
+; CHECK-NEXT:    vmov r2, s10
 ; CHECK-NEXT:    vmov r4, s8
-; CHECK-NEXT:    vmov r12, s11
-; CHECK-NEXT:    vmov r5, s9
-; CHECK-NEXT:    adds.w lr, r4, r3
-; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsb.w r3, r3, #0
-; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q2[0], r3
+; CHECK-NEXT:    vmov lr, s11
+; CHECK-NEXT:    vmov r3, s9
+; CHECK-NEXT:    adds r5, r4, r2
+; CHECK-NEXT:    ubfx r4, r12, #8, #1
+; CHECK-NEXT:    ubfx r2, r12, #12, #1
+; CHECK-NEXT:    rsb.w r4, r4, #0
 ; CHECK-NEXT:    rsb.w r2, r2, #0
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    adc.w r12, r12, r5
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r2
+; CHECK-NEXT:    adc.w r3, r3, lr
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r4
+; CHECK-NEXT:    vmov q2[3], q2[1], r2, r4
 ; CHECK-NEXT:    vmov.u16 r2, q0[2]
+; CHECK-NEXT:    vmov.u16 r4, q0[3]
 ; CHECK-NEXT:    sxth r2, r2
-; CHECK-NEXT:    vmov.32 q3[0], r2
+; CHECK-NEXT:    sxth r4, r4
+; CHECK-NEXT:    vmov q3[2], q3[0], r4, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q0[3]
-; CHECK-NEXT:    sxth r2, r2
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q3[3], r2
+; CHECK-NEXT:    asrs r4, r4, #31
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r2
 ; CHECK-NEXT:    vand q2, q3, q2
-; CHECK-NEXT:    vmov r3, s8
+; CHECK-NEXT:    vmov r4, s8
 ; CHECK-NEXT:    vmov r2, s9
-; CHECK-NEXT:    vmov r4, s10
-; CHECK-NEXT:    vmov r5, s11
-; CHECK-NEXT:    adds.w r3, r3, lr
-; CHECK-NEXT:    adc.w r2, r2, r12
-; CHECK-NEXT:    adds.w r12, r3, r4
-; CHECK-NEXT:    adc.w r3, r2, r5
-; CHECK-NEXT:    vmov.u16 r2, q1[4]
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[5]
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[6]
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[7]
-; CHECK-NEXT:    vmov.32 q2[3], r2
+; CHECK-NEXT:    adds r5, r5, r4
+; CHECK-NEXT:    vmov r4, s11
+; CHECK-NEXT:    adcs r3, r2
+; CHECK-NEXT:    vmov r2, s10
+; CHECK-NEXT:    adds.w r12, r5, r2
+; CHECK-NEXT:    vmov.u16 r5, q1[4]
+; CHECK-NEXT:    adcs r3, r4
+; CHECK-NEXT:    vmov.u16 r4, q1[6]
+; CHECK-NEXT:    vmov q2[2], q2[0], r4, r5
+; CHECK-NEXT:    vmov.u16 r5, q1[5]
+; CHECK-NEXT:    vmov.u16 r4, q1[7]
+; CHECK-NEXT:    vmov q2[3], q2[1], r4, r5
 ; CHECK-NEXT:    vcmp.i32 ne, q2, zr
-; CHECK-NEXT:    vmrs r2, p0
-; CHECK-NEXT:    and r5, r2, #1
-; CHECK-NEXT:    rsbs r5, r5, #0
-; CHECK-NEXT:    vmov.32 q1[0], r5
-; CHECK-NEXT:    vmov.32 q1[1], r5
-; CHECK-NEXT:    ubfx r5, r2, #4, #1
-; CHECK-NEXT:    rsbs r5, r5, #0
-; CHECK-NEXT:    vmov.32 q1[2], r5
-; CHECK-NEXT:    vmov.32 q1[3], r5
-; CHECK-NEXT:    vmov.u16 r5, q0[4]
-; CHECK-NEXT:    sxth r5, r5
-; CHECK-NEXT:    vmov.32 q2[0], r5
-; CHECK-NEXT:    asrs r5, r5, #31
-; CHECK-NEXT:    vmov.32 q2[1], r5
-; CHECK-NEXT:    vmov.u16 r5, q0[5]
-; CHECK-NEXT:    sxth r5, r5
-; CHECK-NEXT:    vmov.32 q2[2], r5
-; CHECK-NEXT:    asrs r5, r5, #31
-; CHECK-NEXT:    vmov.32 q2[3], r5
-; CHECK-NEXT:    vand q1, q2, q1
-; CHECK-NEXT:    vmov r4, s4
-; CHECK-NEXT:    vmov r5, s5
-; CHECK-NEXT:    adds.w r12, r12, r4
-; CHECK-NEXT:    vmov r4, s6
-; CHECK-NEXT:    adcs r5, r3
-; CHECK-NEXT:    vmov r3, s7
-; CHECK-NEXT:    adds.w r4, r4, r12
-; CHECK-NEXT:    adc.w r12, r5, r3
-; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q1[0], r3
+; CHECK-NEXT:    vmrs r5, p0
+; CHECK-NEXT:    and r4, r5, #1
+; CHECK-NEXT:    ubfx r2, r5, #4, #1
+; CHECK-NEXT:    rsbs r4, r4, #0
 ; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q1[1], r3
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vmov.32 q1[3], r2
-; CHECK-NEXT:    vmov.u16 r2, q0[6]
-; CHECK-NEXT:    sxth r2, r2
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q0[7]
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, r4
+; CHECK-NEXT:    vmov q1[3], q1[1], r2, r4
+; CHECK-NEXT:    vmov.u16 r2, q0[4]
+; CHECK-NEXT:    vmov.u16 r4, q0[5]
 ; CHECK-NEXT:    sxth r2, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
+; CHECK-NEXT:    sxth r4, r4
+; CHECK-NEXT:    vmov q2[2], q2[0], r4, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q2[3], r2
-; CHECK-NEXT:    vand q0, q2, q1
-; CHECK-NEXT:    vmov r3, s0
-; CHECK-NEXT:    vmov r2, s1
-; CHECK-NEXT:    vmov r5, s3
-; CHECK-NEXT:    adds r3, r3, r4
+; CHECK-NEXT:    asrs r4, r4, #31
+; CHECK-NEXT:    vmov q2[3], q2[1], r4, r2
+; CHECK-NEXT:    vand q1, q2, q1
+; CHECK-NEXT:    vmov r4, s4
+; CHECK-NEXT:    vmov r2, s5
+; CHECK-NEXT:    adds.w r4, r4, r12
+; CHECK-NEXT:    adc.w r12, r3, r2
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    vmov r3, s7
+; CHECK-NEXT:    adds r2, r2, r4
+; CHECK-NEXT:    ubfx r4, r5, #8, #1
+; CHECK-NEXT:    ubfx r5, r5, #12, #1
+; CHECK-NEXT:    rsb.w r4, r4, #0
+; CHECK-NEXT:    rsb.w r5, r5, #0
+; CHECK-NEXT:    adc.w r3, r3, r12
+; CHECK-NEXT:    vmov q1[2], q1[0], r5, r4
+; CHECK-NEXT:    vmov q1[3], q1[1], r5, r4
+; CHECK-NEXT:    vmov.u16 r5, q0[6]
+; CHECK-NEXT:    vmov.u16 r4, q0[7]
+; CHECK-NEXT:    sxth r5, r5
+; CHECK-NEXT:    sxth r4, r4
+; CHECK-NEXT:    vmov q0[2], q0[0], r4, r5
+; CHECK-NEXT:    asrs r5, r5, #31
+; CHECK-NEXT:    asrs r4, r4, #31
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r5
+; CHECK-NEXT:    vand q0, q0, q1
+; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov r5, s1
+; CHECK-NEXT:    adds r2, r2, r4
 ; CHECK-NEXT:    vmov r4, s2
-; CHECK-NEXT:    adc.w r2, r2, r12
-; CHECK-NEXT:    adds r3, r3, r4
-; CHECK-NEXT:    adcs r2, r5
-; CHECK-NEXT:    adds r0, r0, r3
-; CHECK-NEXT:    adcs r1, r2
+; CHECK-NEXT:    adcs r3, r5
+; CHECK-NEXT:    vmov r5, s3
+; CHECK-NEXT:    adds r2, r2, r4
+; CHECK-NEXT:    adcs r3, r5
+; CHECK-NEXT:    adds r0, r0, r2
+; CHECK-NEXT:    adcs r1, r3
 ; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %c = icmp eq <8 x i16> %b, zeroinitializer
   %xx = sext <8 x i16> %x to <8 x i64>
   %s = select <8 x i1> %c, <8 x i64> %xx, <8 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v8i64(<8 x i64> %s)
   %r = add i64 %z, %a
   ret i64 %r
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i16_v2i64_acc_zext(<2 x i16> %x, <2 x i16> %b, i64 %a) {
 ; CHECK-LABEL: add_v2i16_v2i64_acc_zext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.i64 q2, #0xffff
 ; CHECK-NEXT:    vand q1, q1, q2
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vmov r2, s4
+; CHECK-NEXT:    vmov r3, s6
 ; CHECK-NEXT:    cmp r2, #0
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    cmp r2, #0
-; CHECK-NEXT:    cset r2, eq
-; CHECK-NEXT:    tst.w r2, #1
-; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.32 q3[3], r2
-; CHECK-NEXT:    vand q0, q0, q3
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    cset r3, eq
+; CHECK-NEXT:    tst.w r3, #1
+; CHECK-NEXT:    csetm r3, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r2
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r2, s3
 ; CHECK-NEXT:    vmov r3, s1
 ; CHECK-NEXT:    orr.w r12, r3, r2
 ; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    add r2, r3
 ; CHECK-NEXT:    adds r0, r0, r2
 ; CHECK-NEXT:    adc.w r1, r1, r12
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp eq <2 x i16> %b, zeroinitializer
   %xx = zext <2 x i16> %x to <2 x i64>
   %s = select <2 x i1> %c, <2 x i64> %xx, <2 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> %s)
   %r = add i64 %z, %a
   ret i64 %r
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i16_v2i64_acc_sext(<2 x i16> %x, <2 x i16> %b, i64 %a) {
 ; CHECK-LABEL: add_v2i16_v2i64_acc_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
 ; CHECK-NEXT:    vmov.i32 q2, #0xffff
-; CHECK-NEXT:    vand q2, q1, q2
-; CHECK-NEXT:    vmov r2, s8
-; CHECK-NEXT:    cmp r2, #0
-; CHECK-NEXT:    cset r2, eq
-; CHECK-NEXT:    tst.w r2, #1
-; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q1[0], r2
-; CHECK-NEXT:    vmov.32 q1[1], r2
-; CHECK-NEXT:    vmov r2, s10
+; CHECK-NEXT:    vand q1, q1, q2
+; CHECK-NEXT:    vmov r2, s4
+; CHECK-NEXT:    vmov r3, s6
 ; CHECK-NEXT:    cmp r2, #0
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vmov.32 q1[3], r2
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    cset r3, eq
+; CHECK-NEXT:    tst.w r3, #1
+; CHECK-NEXT:    csetm r3, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r2
 ; CHECK-NEXT:    vmov r2, s0
+; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    sxth r2, r2
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    sxth r2, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
+; CHECK-NEXT:    sxth r3, r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q2[3], r2
-; CHECK-NEXT:    vand q0, q2, q1
+; CHECK-NEXT:    asrs r3, r3, #31
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r2
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r12, s3
 ; CHECK-NEXT:    vmov lr, s1
 ; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    adc.w r3, lr, r12
 ; CHECK-NEXT:    adds r0, r0, r2
 ; CHECK-NEXT:    adcs r1, r3
entry:
   %z = call i8 @llvm.vector.reduce.add.v16i8(<16 x i8> %s)
   %r = add i8 %z, %a
   ret i8 %r
 }
 
 define arm_aapcs_vfpcc i64 @add_v16i8_v16i64_acc_zext(<16 x i8> %x, <16 x i8> %b, i64 %a) {
 ; CHECK-LABEL: add_v16i8_v16i64_acc_zext:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, lr}
-; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vcmp.i8 eq, q1, zr
 ; CHECK-NEXT:    vmov.i8 q2, #0x0
 ; CHECK-NEXT:    vmov.i8 q3, #0xff
 ; CHECK-NEXT:    vpsel q4, q3, q2
 ; CHECK-NEXT:    vmov.u8 r2, q4[0]
 ; CHECK-NEXT:    vmov.16 q1[0], r2

 ; CHECK-NEXT:    vmov.16 q1[5], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[6]
 ; CHECK-NEXT:    vmov.16 q1[6], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[7]
 ; CHECK-NEXT:    vmov.16 q1[7], r2
 ; CHECK-NEXT:    vcmp.i16 ne, q1, zr
 ; CHECK-NEXT:    vpsel q5, q3, q2
 ; CHECK-NEXT:    vmov.u16 r2, q5[0]
-; CHECK-NEXT:    vmov.32 q1[0], r2
+; CHECK-NEXT:    vmov.u16 r3, q5[2]
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q5[1]
-; CHECK-NEXT:    vmov.32 q1[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q5[2]
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q5[3]
-; CHECK-NEXT:    vmov.32 q1[3], r2
+; CHECK-NEXT:    vmov.u16 r3, q5[3]
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q1, zr
 ; CHECK-NEXT:    vmov.i64 q1, #0xff
 ; CHECK-NEXT:    vmrs r2, p0
 ; CHECK-NEXT:    and r3, r2, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q6[0], r3
-; CHECK-NEXT:    vmov.32 q6[1], r3
+; CHECK-NEXT:    rsb.w r12, r3, #0
 ; CHECK-NEXT:    ubfx r3, r2, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q6[2], r3
-; CHECK-NEXT:    vmov.32 q6[3], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[0]
-; CHECK-NEXT:    vmov.32 q7[0], r3
+; CHECK-NEXT:    vmov q6[2], q6[0], r3, r12
+; CHECK-NEXT:    vmov q6[3], q6[1], r3, r12
+; CHECK-NEXT:    vmov.u8 r12, q0[0]
 ; CHECK-NEXT:    vmov.u8 r3, q0[1]
-; CHECK-NEXT:    vmov.32 q7[2], r3
+; CHECK-NEXT:    vmov q7[2], q7[0], r3, r12
 ; CHECK-NEXT:    vand q7, q7, q1
 ; CHECK-NEXT:    vand q6, q7, q6
 ; CHECK-NEXT:    vmov r12, s27
 ; CHECK-NEXT:    vmov r3, s25
 ; CHECK-NEXT:    vmov lr, s26
 ; CHECK-NEXT:    orr.w r12, r12, r3
 ; CHECK-NEXT:    vmov r3, s24
 ; CHECK-NEXT:    add lr, r3
 ; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q6[0], r3
+; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q6[1], r3
-; CHECK-NEXT:    vmov.32 q6[2], r2
-; CHECK-NEXT:    vmov.32 q6[3], r2
+; CHECK-NEXT:    vmov q6[2], q6[0], r2, r3
+; CHECK-NEXT:    vmov q6[3], q6[1], r2, r3
 ; CHECK-NEXT:    vmov.u8 r2, q0[2]
-; CHECK-NEXT:    vmov.32 q7[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[3]
-; CHECK-NEXT:    vmov.32 q7[2], r2
+; CHECK-NEXT:    vmov.u8 r3, q0[3]
+; CHECK-NEXT:    vmov q7[2], q7[0], r3, r2
 ; CHECK-NEXT:    vand q7, q7, q1
 ; CHECK-NEXT:    vand q6, q7, q6
 ; CHECK-NEXT:    vmov r3, s24
 ; CHECK-NEXT:    vmov r2, s25
-; CHECK-NEXT:    adds.w r4, lr, r3
+; CHECK-NEXT:    adds.w lr, lr, r3
 ; CHECK-NEXT:    vmov r3, s26
-; CHECK-NEXT:    adc.w lr, r12, r2
+; CHECK-NEXT:    adc.w r12, r12, r2
 ; CHECK-NEXT:    vmov r2, s27
-; CHECK-NEXT:    adds.w r12, r4, r3
-; CHECK-NEXT:    adc.w lr, lr, r2
+; CHECK-NEXT:    adds.w lr, lr, r3
+; CHECK-NEXT:    vmov.u16 r3, q5[6]
+; CHECK-NEXT:    adc.w r12, r12, r2
 ; CHECK-NEXT:    vmov.u16 r2, q5[4]
-; CHECK-NEXT:    vmov.32 q6[0], r2
+; CHECK-NEXT:    vmov q6[2], q6[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q5[5]
-; CHECK-NEXT:    vmov.32 q6[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q5[6]
-; CHECK-NEXT:    vmov.32 q6[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q5[7]
-; CHECK-NEXT:    vmov.32 q6[3], r2
+; CHECK-NEXT:    vmov.u16 r3, q5[7]
+; CHECK-NEXT:    vmov q6[3], q6[1], r3, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q6, zr
 ; CHECK-NEXT:    vmrs r2, p0
-; CHECK-NEXT:    and r4, r2, #1
-; CHECK-NEXT:    rsbs r4, r4, #0
-; CHECK-NEXT:    vmov.32 q5[0], r4
-; CHECK-NEXT:    vmov.32 q5[1], r4
+; CHECK-NEXT:    and r3, r2, #1
 ; CHECK-NEXT:    ubfx r4, r2, #4, #1
+; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    rsbs r4, r4, #0
-; CHECK-NEXT:    vmov.32 q5[2], r4
-; CHECK-NEXT:    vmov.32 q5[3], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[4]
-; CHECK-NEXT:    vmov.32 q6[0], r4
+; CHECK-NEXT:    vmov q5[2], q5[0], r4, r3
+; CHECK-NEXT:    vmov q5[3], q5[1], r4, r3
+; CHECK-NEXT:    vmov.u8 r3, q0[4]
 ; CHECK-NEXT:    vmov.u8 r4, q0[5]
-; CHECK-NEXT:    vmov.32 q6[2], r4
+; CHECK-NEXT:    vmov q6[2], q6[0], r4, r3
 ; CHECK-NEXT:    vand q6, q6, q1
 ; CHECK-NEXT:    vand q5, q6, q5
-; CHECK-NEXT:    vmov r3, s20
-; CHECK-NEXT:    vmov r4, s21
-; CHECK-NEXT:    adds.w r12, r12, r3
-; CHECK-NEXT:    vmov r3, s23
-; CHECK-NEXT:    adc.w lr, lr, r4
+; CHECK-NEXT:    vmov r4, s20
+; CHECK-NEXT:    vmov r3, s21
+; CHECK-NEXT:    adds.w lr, lr, r4
 ; CHECK-NEXT:    vmov r4, s22
-; CHECK-NEXT:    adds.w r4, r4, r12
-; CHECK-NEXT:    adc.w r12, lr, r3
+; CHECK-NEXT:    adc.w r12, r12, r3
+; CHECK-NEXT:    vmov r3, s23
+; CHECK-NEXT:    adds.w r4, r4, lr
+; CHECK-NEXT:    adc.w r12, r12, r3
 ; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q5[0], r3
+; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q5[1], r3
-; CHECK-NEXT:    vmov.32 q5[2], r2
-; CHECK-NEXT:    vmov.32 q5[3], r2
+; CHECK-NEXT:    vmov q5[2], q5[0], r2, r3
+; CHECK-NEXT:    vmov q5[3], q5[1], r2, r3
 ; CHECK-NEXT:    vmov.u8 r2, q0[6]
-; CHECK-NEXT:    vmov.32 q6[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[7]
-; CHECK-NEXT:    vmov.32 q6[2], r2
+; CHECK-NEXT:    vmov.u8 r3, q0[7]
+; CHECK-NEXT:    vmov q6[2], q6[0], r3, r2
 ; CHECK-NEXT:    vand q6, q6, q1
 ; CHECK-NEXT:    vand q5, q6, q5
 ; CHECK-NEXT:    vmov r3, s20
 ; CHECK-NEXT:    vmov r2, s21
-; CHECK-NEXT:    adds r3, r3, r4
-; CHECK-NEXT:    vmov r4, s23
-; CHECK-NEXT:    adc.w lr, r12, r2
-; CHECK-NEXT:    vmov r2, s22
-; CHECK-NEXT:    adds.w r12, r3, r2
+; CHECK-NEXT:    adds.w lr, r4, r3
+; CHECK-NEXT:    vmov r3, s22
+; CHECK-NEXT:    adc.w r4, r12, r2
+; CHECK-NEXT:    vmov r2, s23
+; CHECK-NEXT:    adds.w r12, lr, r3
+; CHECK-NEXT:    adc.w lr, r4, r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[8]
 ; CHECK-NEXT:    vmov.16 q5[0], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[9]
 ; CHECK-NEXT:    vmov.16 q5[1], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[10]
 ; CHECK-NEXT:    vmov.16 q5[2], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[11]
 ; CHECK-NEXT:    vmov.16 q5[3], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[12]
 ; CHECK-NEXT:    vmov.16 q5[4], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[13]
 ; CHECK-NEXT:    vmov.16 q5[5], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[14]
 ; CHECK-NEXT:    vmov.16 q5[6], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[15]
 ; CHECK-NEXT:    vmov.16 q5[7], r2
-; CHECK-NEXT:    adc.w lr, lr, r4
 ; CHECK-NEXT:    vcmp.i16 ne, q5, zr
 ; CHECK-NEXT:    vpsel q2, q3, q2
 ; CHECK-NEXT:    vmov.u16 r2, q2[0]
-; CHECK-NEXT:    vmov.32 q3[0], r2
+; CHECK-NEXT:    vmov.u16 r4, q2[2]
+; CHECK-NEXT:    vmov q3[2], q3[0], r4, r2
 ; CHECK-NEXT:    vmov.u16 r2, q2[1]
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[2]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[3]
-; CHECK-NEXT:    vmov.32 q3[3], r2
+; CHECK-NEXT:    vmov.u16 r4, q2[3]
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q3, zr
 ; CHECK-NEXT:    vmrs r2, p0
 ; CHECK-NEXT:    and r4, r2, #1
+; CHECK-NEXT:    ubfx r3, r2, #4, #1
 ; CHECK-NEXT:    rsbs r4, r4, #0
-; CHECK-NEXT:    vmov.32 q3[0], r4
-; CHECK-NEXT:    vmov.32 q3[1], r4
-; CHECK-NEXT:    ubfx r4, r2, #4, #1
-; CHECK-NEXT:    rsbs r4, r4, #0
-; CHECK-NEXT:    vmov.32 q3[2], r4
-; CHECK-NEXT:    vmov.32 q3[3], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[8]
-; CHECK-NEXT:    vmov.32 q4[0], r4
+; CHECK-NEXT:    rsbs r3, r3, #0
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r4
+; CHECK-NEXT:    vmov q3[3], q3[1], r3, r4
+; CHECK-NEXT:    vmov.u8 r3, q0[8]
 ; CHECK-NEXT:    vmov.u8 r4, q0[9]
-; CHECK-NEXT:    vmov.32 q4[2], r4
+; CHECK-NEXT:    vmov q4[2], q4[0], r4, r3
 ; CHECK-NEXT:    vand q4, q4, q1
 ; CHECK-NEXT:    vand q3, q4, q3
-; CHECK-NEXT:    vmov r3, s12
-; CHECK-NEXT:    vmov r4, s13
-; CHECK-NEXT:    adds.w r12, r12, r3
-; CHECK-NEXT:    vmov r3, s15
-; CHECK-NEXT:    adc.w lr, lr, r4
+; CHECK-NEXT:    vmov r4, s12
+; CHECK-NEXT:    vmov r3, s13
+; CHECK-NEXT:    adds.w r5, r12, r4
 ; CHECK-NEXT:    vmov r4, s14
-; CHECK-NEXT:    adds.w r4, r4, r12
 ; CHECK-NEXT:    adc.w r12, lr, r3
-; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
+; CHECK-NEXT:    vmov r3, s15
+; CHECK-NEXT:    adds r5, r5, r4
+; CHECK-NEXT:    ubfx r4, r2, #8, #1
 ; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q3[0], r3
-; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q3[1], r3
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.32 q3[3], r2
+; CHECK-NEXT:    rsb.w r4, r4, #0
+; CHECK-NEXT:    rsb.w r2, r2, #0
+; CHECK-NEXT:    adc.w r3, r3, r12
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r4
+; CHECK-NEXT:    vmov q3[3], q3[1], r2, r4
 ; CHECK-NEXT:    vmov.u8 r2, q0[10]
-; CHECK-NEXT:    vmov.32 q4[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[11]
-; CHECK-NEXT:    vmov.32 q4[2], r2
+; CHECK-NEXT:    vmov.u8 r4, q0[11]
+; CHECK-NEXT:    vmov q4[2], q4[0], r4, r2
 ; CHECK-NEXT:    vand q4, q4, q1
 ; CHECK-NEXT:    vand q3, q4, q3
-; CHECK-NEXT:    vmov r3, s12
+; CHECK-NEXT:    vmov r4, s12
 ; CHECK-NEXT:    vmov r2, s13
-; CHECK-NEXT:    adds r3, r3, r4
-; CHECK-NEXT:    vmov r4, s15
-; CHECK-NEXT:    adc.w lr, r12, r2
-; CHECK-NEXT:    vmov r2, s14
-; CHECK-NEXT:    adds.w r12, r3, r2
-; CHECK-NEXT:    vmov.u16 r2, q2[4]
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[5]
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[6]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[7]
-; CHECK-NEXT:    vmov.32 q3[3], r2
-; CHECK-NEXT:    adc.w lr, lr, r4
+; CHECK-NEXT:    adds r5, r5, r4
+; CHECK-NEXT:    vmov r4, s14
+; CHECK-NEXT:    adcs r2, r3
+; CHECK-NEXT:    vmov r3, s15
+; CHECK-NEXT:    adds r5, r5, r4
+; CHECK-NEXT:    vmov.u16 r4, q2[6]
+; CHECK-NEXT:    adc.w r12, r2, r3
+; CHECK-NEXT:    vmov.u16 r3, q2[4]
+; CHECK-NEXT:    vmov q3[2], q3[0], r4, r3
+; CHECK-NEXT:    vmov.u16 r3, q2[5]
+; CHECK-NEXT:    vmov.u16 r4, q2[7]
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r3
 ; CHECK-NEXT:    vcmp.i32 ne, q3, zr
-; CHECK-NEXT:    vmrs r2, p0
-; CHECK-NEXT:    and r4, r2, #1
+; CHECK-NEXT:    vmrs r3, p0
+; CHECK-NEXT:    and r4, r3, #1
+; CHECK-NEXT:    ubfx r2, r3, #4, #1
 ; CHECK-NEXT:    rsbs r4, r4, #0
-; CHECK-NEXT:    vmov.32 q2[0], r4
-; CHECK-NEXT:    vmov.32 q2[1], r4
-; CHECK-NEXT:    ubfx r4, r2, #4, #1
-; CHECK-NEXT:    rsbs r4, r4, #0
-; CHECK-NEXT:    vmov.32 q2[2], r4
-; CHECK-NEXT:    vmov.32 q2[3], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[12]
-; CHECK-NEXT:    vmov.32 q3[0], r4
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r4
+; CHECK-NEXT:    vmov q2[3], q2[1], r2, r4
+; CHECK-NEXT:    vmov.u8 r2, q0[12]
 ; CHECK-NEXT:    vmov.u8 r4, q0[13]
-; CHECK-NEXT:    vmov.32 q3[2], r4
+; CHECK-NEXT:    vmov q3[2], q3[0], r4, r2
 ; CHECK-NEXT:    vand q3, q3, q1
 ; CHECK-NEXT:    vand q2, q3, q2
-; CHECK-NEXT:    vmov r3, s8
-; CHECK-NEXT:    vmov r4, s9
-; CHECK-NEXT:    adds.w r12, r12, r3
-; CHECK-NEXT:    vmov r3, s11
-; CHECK-NEXT:    adc.w lr, lr, r4
-; CHECK-NEXT:    vmov r4, s10
-; CHECK-NEXT:    adds.w r4, r4, r12
-; CHECK-NEXT:    adc.w r12, lr, r3
-; CHECK-NEXT:    ubfx r3, r2, #8, #1
+; CHECK-NEXT:    vmov r4, s8
+; CHECK-NEXT:    vmov r2, s9
+; CHECK-NEXT:    adds r5, r5, r4
+; CHECK-NEXT:    vmov r4, s11
+; CHECK-NEXT:    adc.w r12, r12, r2
+; CHECK-NEXT:    vmov r2, s10
+; CHECK-NEXT:    adds r2, r2, r5
+; CHECK-NEXT:    adc.w r5, r12, r4
+; CHECK-NEXT:    ubfx r4, r3, #8, #1
+; CHECK-NEXT:    ubfx r3, r3, #12, #1
+; CHECK-NEXT:    rsbs r4, r4, #0
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q2[0], r3
-; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[14]
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[15]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vand q0, q3, q1
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r4
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r4
+; CHECK-NEXT:    vmov.u8 r3, q0[14]
+; CHECK-NEXT:    vmov.u8 r4, q0[15]
+; CHECK-NEXT:    vmov q0[2], q0[0], r4, r3
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vand q0, q0, q2
-; CHECK-NEXT:    vmov r3, s0
-; CHECK-NEXT:    vmov r2, s1
-; CHECK-NEXT:    adds r3, r3, r4
-; CHECK-NEXT:    vmov r4, s3
-; CHECK-NEXT:    adc.w r12, r12, r2
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    adds r2, r2, r3
-; CHECK-NEXT:    adc.w r3, r12, r4
+; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov r3, s1
+; CHECK-NEXT:    adds r2, r2, r4
+; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    adcs r3, r5
+; CHECK-NEXT:    vmov r5, s3
+; CHECK-NEXT:    adds r2, r2, r4
+; CHECK-NEXT:    adcs r3, r5
 ; CHECK-NEXT:    adds r0, r0, r2
 ; CHECK-NEXT:    adcs r1, r3
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    pop {r4, pc}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %c = icmp eq <16 x i8> %b, zeroinitializer
   %xx = zext <16 x i8> %x to <16 x i64>
   %s = select <16 x i1> %c, <16 x i64> %xx, <16 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v16i64(<16 x i64> %s)
   %r = add i64 %z, %a
   ret i64 %r
 }

 ; CHECK-NEXT:    vmov.16 q4[5], r2
 ; CHECK-NEXT:    vmov.u8 r2, q3[6]
 ; CHECK-NEXT:    vmov.16 q4[6], r2
 ; CHECK-NEXT:    vmov.u8 r2, q3[7]
 ; CHECK-NEXT:    vmov.16 q4[7], r2
 ; CHECK-NEXT:    vcmp.i16 ne, q4, zr
 ; CHECK-NEXT:    vpsel q4, q2, q1
 ; CHECK-NEXT:    vmov.u16 r2, q4[0]
-; CHECK-NEXT:    vmov.32 q5[0], r2
+; CHECK-NEXT:    vmov.u16 r3, q4[2]
+; CHECK-NEXT:    vmov q5[2], q5[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q4[1]
-; CHECK-NEXT:    vmov.32 q5[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q4[2]
-; CHECK-NEXT:    vmov.32 q5[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q4[3]
-; CHECK-NEXT:    vmov.32 q5[3], r2
+; CHECK-NEXT:    vmov.u16 r3, q4[3]
+; CHECK-NEXT:    vmov q5[3], q5[1], r3, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q5, zr
-; CHECK-NEXT:    vmrs r2, p0
-; CHECK-NEXT:    and r3, r2, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q5[0], r3
-; CHECK-NEXT:    vmov.32 q5[1], r3
-; CHECK-NEXT:    ubfx r3, r2, #4, #1
+; CHECK-NEXT:    vmrs r12, p0
+; CHECK-NEXT:    and r3, r12, #1
+; CHECK-NEXT:    ubfx r2, r12, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q5[2], r3
-; CHECK-NEXT:    vmov.32 q5[3], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[0]
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    vmov.32 q6[0], r3
-; CHECK-NEXT:    asrs r3, r3, #31
-; CHECK-NEXT:    vmov.32 q6[1], r3
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov q5[2], q5[0], r2, r3
+; CHECK-NEXT:    vmov q5[3], q5[1], r2, r3
+; CHECK-NEXT:    vmov.u8 r2, q0[0]
 ; CHECK-NEXT:    vmov.u8 r3, q0[1]
+; CHECK-NEXT:    sxtb r2, r2
 ; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    vmov.32 q6[2], r3
+; CHECK-NEXT:    vmov q6[2], q6[0], r3, r2
+; CHECK-NEXT:    asrs r2, r2, #31
 ; CHECK-NEXT:    asrs r3, r3, #31
-; CHECK-NEXT:    vmov.32 q6[3], r3
+; CHECK-NEXT:    vmov q6[3], q6[1], r3, r2
 ; CHECK-NEXT:    vand q5, q6, q5
-; CHECK-NEXT:    vmov r3, s22
+; CHECK-NEXT:    vmov r2, s22
 ; CHECK-NEXT:    vmov r4, s20
-; CHECK-NEXT:    vmov r12, s23
-; CHECK-NEXT:    vmov r5, s21
-; CHECK-NEXT:    adds.w lr, r4, r3
-; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsb.w r3, r3, #0
-; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q5[0], r3
+; CHECK-NEXT:    vmov lr, s23
+; CHECK-NEXT:    vmov r3, s21
+; CHECK-NEXT:    adds r5, r4, r2
+; CHECK-NEXT:    ubfx r4, r12, #8, #1
+; CHECK-NEXT:    ubfx r2, r12, #12, #1
+; CHECK-NEXT:    rsb.w r4, r4, #0
 ; CHECK-NEXT:    rsb.w r2, r2, #0
-; CHECK-NEXT:    vmov.32 q5[1], r3
-; CHECK-NEXT:    adc.w r12, r12, r5
-; CHECK-NEXT:    vmov.32 q5[2], r2
-; CHECK-NEXT:    vmov.32 q5[3], r2
+; CHECK-NEXT:    adc.w r3, r3, lr
+; CHECK-NEXT:    vmov q5[2], q5[0], r2, r4
+; CHECK-NEXT:    vmov q5[3], q5[1], r2, r4
 ; CHECK-NEXT:    vmov.u8 r2, q0[2]
+; CHECK-NEXT:    vmov.u8 r4, q0[3]
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q6[0], r2
-; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q6[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[3]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q6[2], r2
+; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    vmov q6[2], q6[0], r4, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q6[3], r2
+; CHECK-NEXT:    asrs r4, r4, #31
+; CHECK-NEXT:    vmov q6[3], q6[1], r4, r2
 ; CHECK-NEXT:    vand q5, q6, q5
-; CHECK-NEXT:    vmov r3, s20
+; CHECK-NEXT:    vmov r4, s20
 ; CHECK-NEXT:    vmov r2, s21
-; CHECK-NEXT:    vmov r4, s22
-; CHECK-NEXT:    vmov r5, s23
-; CHECK-NEXT:    adds.w r3, r3, lr
-; CHECK-NEXT:    adc.w r2, r2, r12
-; CHECK-NEXT:    adds.w r12, r3, r4
-; CHECK-NEXT:    adc.w r3, r2, r5
-; CHECK-NEXT:    vmov.u16 r2, q4[4]
-; CHECK-NEXT:    vmov.32 q5[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q4[5]
-; CHECK-NEXT:    vmov.32 q5[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q4[6]
-; CHECK-NEXT:    vmov.32 q5[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q4[7]
-; CHECK-NEXT:    vmov.32 q5[3], r2
+; CHECK-NEXT:    adds r5, r5, r4
+; CHECK-NEXT:    vmov r4, s23
+; CHECK-NEXT:    adcs r3, r2
+; CHECK-NEXT:    vmov r2, s22
+; CHECK-NEXT:    adds.w r12, r5, r2
+; CHECK-NEXT:    vmov.u16 r5, q4[4]
+; CHECK-NEXT:    adcs r3, r4
+; CHECK-NEXT:    vmov.u16 r4, q4[6]
+; CHECK-NEXT:    vmov q5[2], q5[0], r4, r5
+; CHECK-NEXT:    vmov.u16 r5, q4[5]
+; CHECK-NEXT:    vmov.u16 r4, q4[7]
+; CHECK-NEXT:    vmov q5[3], q5[1], r4, r5
 ; CHECK-NEXT:    vcmp.i32 ne, q5, zr
-; CHECK-NEXT:    vmrs r2, p0
-; CHECK-NEXT:    and r5, r2, #1
-; CHECK-NEXT:    rsbs r5, r5, #0
-; CHECK-NEXT:    vmov.32 q4[0], r5
-; CHECK-NEXT:    vmov.32 q4[1], r5
-; CHECK-NEXT:    ubfx r5, r2, #4, #1
-; CHECK-NEXT:    rsbs r5, r5, #0
-; CHECK-NEXT:    vmov.32 q4[2], r5
-; CHECK-NEXT:    vmov.32 q4[3], r5
-; CHECK-NEXT:    vmov.u8 r5, q0[4]
-; CHECK-NEXT:    sxtb r5, r5
-; CHECK-NEXT:    vmov.32 q5[0], r5
-; CHECK-NEXT:    asrs r5, r5, #31
-; CHECK-NEXT:    vmov.32 q5[1], r5
-; CHECK-NEXT:    vmov.u8 r5, q0[5]
+; CHECK-NEXT:    vmrs r5, p0
+; CHECK-NEXT:    and r4, r5, #1
+; CHECK-NEXT:    ubfx r2, r5, #4, #1
+; CHECK-NEXT:    rsbs r4, r4, #0
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov q4[2], q4[0], r2, r4
+; CHECK-NEXT:    vmov q4[3], q4[1], r2, r4
+; CHECK-NEXT:    vmov.u8 r2, q0[4]
+; CHECK-NEXT:    vmov.u8 r4, q0[5]
+; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    vmov q5[2], q5[0], r4, r2
+; CHECK-NEXT:    asrs r2, r2, #31
+; CHECK-NEXT:    asrs r4, r4, #31
+; CHECK-NEXT:    vmov q5[3], q5[1], r4, r2
+; CHECK-NEXT:    vand q4, q5, q4
+; CHECK-NEXT:    vmov r4, s16
+; CHECK-NEXT:    vmov r2, s17
+; CHECK-NEXT:    adds.w r4, r4, r12
+; CHECK-NEXT:    adc.w r12, r3, r2
+; CHECK-NEXT:    vmov r2, s18
+; CHECK-NEXT:    vmov r3, s19
+; CHECK-NEXT:    adds r2, r2, r4
+; CHECK-NEXT:    ubfx r4, r5, #8, #1
+; CHECK-NEXT:    ubfx r5, r5, #12, #1
+; CHECK-NEXT:    rsb.w r4, r4, #0
+; CHECK-NEXT:    rsb.w r5, r5, #0
+; CHECK-NEXT:    adc.w r3, r3, r12
+; CHECK-NEXT:    vmov q4[2], q4[0], r5, r4
+; CHECK-NEXT:    vmov q4[3], q4[1], r5, r4
+; CHECK-NEXT:    vmov.u8 r5, q0[6]
+; CHECK-NEXT:    vmov.u8 r4, q0[7]
 ; CHECK-NEXT:    sxtb r5, r5
-; CHECK-NEXT:    vmov.32 q5[2], r5
+; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    vmov q5[2], q5[0], r4, r5
 ; CHECK-NEXT:    asrs r5, r5, #31
-; CHECK-NEXT:    vmov.32 q5[3], r5
+; CHECK-NEXT:    asrs r4, r4, #31
+; CHECK-NEXT:    vmov q5[3], q5[1], r4, r5
 ; CHECK-NEXT:    vand q4, q5, q4
 ; CHECK-NEXT:    vmov r4, s16
 ; CHECK-NEXT:    vmov r5, s17
-; CHECK-NEXT:    adds.w r12, r12, r4
+; CHECK-NEXT:    adds r2, r2, r4
 ; CHECK-NEXT:    vmov r4, s18
-; CHECK-NEXT:    adcs r5, r3
-; CHECK-NEXT:    vmov r3, s19
-; CHECK-NEXT:    adds.w r4, r4, r12
-; CHECK-NEXT:    adc.w r12, r5, r3
-; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q4[0], r3
-; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q4[1], r3
-; CHECK-NEXT:    vmov.32 q4[2], r2
-; CHECK-NEXT:    vmov.32 q4[3], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[6]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q5[0], r2
-; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q5[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[7]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q5[2], r2
-; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q5[3], r2
-; CHECK-NEXT:    vand q4, q5, q4
-; CHECK-NEXT:    vmov r3, s16
-; CHECK-NEXT:    vmov r2, s17
+; CHECK-NEXT:    adcs r3, r5
 ; CHECK-NEXT:    vmov r5, s19
-; CHECK-NEXT:    adds r3, r3, r4
-; CHECK-NEXT:    vmov r4, s18
-; CHECK-NEXT:    adc.w r2, r2, r12
-; CHECK-NEXT:    adds.w r12, r3, r4
-; CHECK-NEXT:    adc.w r3, r2, r5
-; CHECK-NEXT:    vmov.u8 r2, q3[8]
-; CHECK-NEXT:    vmov.16 q4[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q3[9]
-; CHECK-NEXT:    vmov.16 q4[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q3[10]
-; CHECK-NEXT:    vmov.16 q4[2], r2
-; CHECK-NEXT:    vmov.u8 r2, q3[11]
-; CHECK-NEXT:    vmov.16 q4[3], r2
-; CHECK-NEXT:    vmov.u8 r2, q3[12]
-; CHECK-NEXT:    vmov.16 q4[4], r2
-; CHECK-NEXT:    vmov.u8 r2, q3[13]
-; CHECK-NEXT:    vmov.16 q4[5], r2
-; CHECK-NEXT:    vmov.u8 r2, q3[14]
-; CHECK-NEXT:    vmov.16 q4[6], r2
-; CHECK-NEXT:    vmov.u8 r2, q3[15]
-; CHECK-NEXT:    vmov.16 q4[7], r2
+; CHECK-NEXT:    adds.w r12, r2, r4
+; CHECK-NEXT:    adcs r3, r5
+; CHECK-NEXT:    vmov.u8 r5, q3[8]
+; CHECK-NEXT:    vmov.16 q4[0], r5
+; CHECK-NEXT:    vmov.u8 r5, q3[9]
+; CHECK-NEXT:    vmov.16 q4[1], r5
+; CHECK-NEXT:    vmov.u8 r5, q3[10]
+; CHECK-NEXT:    vmov.16 q4[2], r5
+; CHECK-NEXT:    vmov.u8 r5, q3[11]
+; CHECK-NEXT:    vmov.16 q4[3], r5
+; CHECK-NEXT:    vmov.u8 r5, q3[12]
+; CHECK-NEXT:    vmov.16 q4[4], r5
+; CHECK-NEXT:    vmov.u8 r5, q3[13]
+; CHECK-NEXT:    vmov.16 q4[5], r5
+; CHECK-NEXT:    vmov.u8 r5, q3[14]
+; CHECK-NEXT:    vmov.16 q4[6], r5
+; CHECK-NEXT:    vmov.u8 r5, q3[15]
+; CHECK-NEXT:    vmov.16 q4[7], r5
 ; CHECK-NEXT:    vcmp.i16 ne, q4, zr
 ; CHECK-NEXT:    vpsel q1, q2, q1
-; CHECK-NEXT:    vmov.u16 r2, q1[0]
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[1]
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[2]
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[3]
-; CHECK-NEXT:    vmov.32 q2[3], r2
+; CHECK-NEXT:    vmov.u16 r5, q1[0]
+; CHECK-NEXT:    vmov.u16 r4, q1[2]
+; CHECK-NEXT:    vmov q2[2], q2[0], r4, r5
+; CHECK-NEXT:    vmov.u16 r5, q1[1]
+; CHECK-NEXT:    vmov.u16 r4, q1[3]
+; CHECK-NEXT:    vmov q2[3], q2[1], r4, r5
 ; CHECK-NEXT:    vcmp.i32 ne, q2, zr
-; CHECK-NEXT:    vmrs r2, p0
-; CHECK-NEXT:    and r5, r2, #1
-; CHECK-NEXT:    rsbs r5, r5, #0
-; CHECK-NEXT:    vmov.32 q2[0], r5
-; CHECK-NEXT:    vmov.32 q2[1], r5
-; CHECK-NEXT:    ubfx r5, r2, #4, #1
-; CHECK-NEXT:    rsbs r5, r5, #0
-; CHECK-NEXT:    vmov.32 q2[2], r5
-; CHECK-NEXT:    vmov.32 q2[3], r5
-; CHECK-NEXT:    vmov.u8 r5, q0[8]
-; CHECK-NEXT:    sxtb r5, r5
-; CHECK-NEXT:    vmov.32 q3[0], r5
-; CHECK-NEXT:    asrs r5, r5, #31
-; CHECK-NEXT:    vmov.32 q3[1], r5
-; CHECK-NEXT:    vmov.u8 r5, q0[9]
+; CHECK-NEXT:    vmrs r5, p0
+; CHECK-NEXT:    and r4, r5, #1
+; CHECK-NEXT:    ubfx r2, r5, #4, #1
+; CHECK-NEXT:    rsbs r4, r4, #0
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r4
+; CHECK-NEXT:    vmov q2[3], q2[1], r2, r4
+; CHECK-NEXT:    vmov.u8 r2, q0[8]
+; CHECK-NEXT:    vmov.u8 r4, q0[9]
+; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    vmov q3[2], q3[0], r4, r2
+; CHECK-NEXT:    asrs r2, r2, #31
+; CHECK-NEXT:    asrs r4, r4, #31
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r2
+; CHECK-NEXT:    vand q2, q3, q2
+; CHECK-NEXT:    vmov r4, s8
+; CHECK-NEXT:    vmov r2, s9
+; CHECK-NEXT:    adds.w r4, r4, r12
+; CHECK-NEXT:    adc.w r12, r3, r2
+; CHECK-NEXT:    vmov r2, s10
+; CHECK-NEXT:    vmov r3, s11
+; CHECK-NEXT:    adds r2, r2, r4
+; CHECK-NEXT:    ubfx r4, r5, #8, #1
+; CHECK-NEXT:    ubfx r5, r5, #12, #1
+; CHECK-NEXT:    rsb.w r4, r4, #0
+; CHECK-NEXT:    rsb.w r5, r5, #0
+; CHECK-NEXT:    adc.w r3, r3, r12
+; CHECK-NEXT:    vmov q2[2], q2[0], r5, r4
+; CHECK-NEXT:    vmov q2[3], q2[1], r5, r4
+; CHECK-NEXT:    vmov.u8 r5, q0[10]
+; CHECK-NEXT:    vmov.u8 r4, q0[11]
 ; CHECK-NEXT:    sxtb r5, r5
-; CHECK-NEXT:    vmov.32 q3[2], r5
+; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    vmov q3[2], q3[0], r4, r5
 ; CHECK-NEXT:    asrs r5, r5, #31
-; CHECK-NEXT:    vmov.32 q3[3], r5
+; CHECK-NEXT:    asrs r4, r4, #31
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r5
 ; CHECK-NEXT:    vand q2, q3, q2
 ; CHECK-NEXT:    vmov r4, s8
 ; CHECK-NEXT:    vmov r5, s9
-; CHECK-NEXT:    adds.w r12, r12, r4
+; CHECK-NEXT:    adds r2, r2, r4
 ; CHECK-NEXT:    vmov r4, s10
-; CHECK-NEXT:    adcs r5, r3
-; CHECK-NEXT:    vmov r3, s11
-; CHECK-NEXT:    adds.w r4, r4, r12
-; CHECK-NEXT:    adc.w r12, r5, r3
-; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q2[0], r3
+; CHECK-NEXT:    adcs r3, r5
+; CHECK-NEXT:    vmov r5, s11
+; CHECK-NEXT:    adds.w r12, r2, r4
+; CHECK-NEXT:    vmov.u16 r4, q1[6]
+; CHECK-NEXT:    adcs r3, r5
+; CHECK-NEXT:    vmov.u16 r5, q1[4]
+; CHECK-NEXT:    vmov q2[2], q2[0], r4, r5
+; CHECK-NEXT:    vmov.u16 r5, q1[5]
+; CHECK-NEXT:    vmov.u16 r4, q1[7]
+; CHECK-NEXT:    vmov q2[3], q2[1], r4, r5
+; CHECK-NEXT:    vcmp.i32 ne, q2, zr
+; CHECK-NEXT:    vmrs r5, p0
+; CHECK-NEXT:    and r4, r5, #1
+; CHECK-NEXT:    ubfx r2, r5, #4, #1
+; CHECK-NEXT:    rsbs r4, r4, #0
 ; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[10]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[11]
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, r4
+; CHECK-NEXT:    vmov q1[3], q1[1], r2, r4
+; CHECK-NEXT:    vmov.u8 r2, q0[12]
+; CHECK-NEXT:    vmov.u8 r4, q0[13]
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q3[2], r2
+; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    vmov q2[2], q2[0], r4, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q3[3], r2
-; CHECK-NEXT:    vand q2, q3, q2
-; CHECK-NEXT:    vmov r3, s8
-; CHECK-NEXT:    vmov r2, s9
-; CHECK-NEXT:    vmov r5, s11
-; CHECK-NEXT:    adds r3, r3, r4
-; CHECK-NEXT:    vmov r4, s10
-; CHECK-NEXT:    adc.w r2, r2, r12
-; CHECK-NEXT:    adds.w r12, r3, r4
-; CHECK-NEXT:    adc.w r3, r2, r5
-; CHECK-NEXT:    vmov.u16 r2, q1[4]
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[5]
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[6]
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[7]
-; CHECK-NEXT:    vmov.32 q2[3], r2
-; CHECK-NEXT:    vcmp.i32 ne, q2, zr
-; CHECK-NEXT:    vmrs r2, p0
-; CHECK-NEXT:    and r5, r2, #1
-; CHECK-NEXT:    rsbs r5, r5, #0
-; CHECK-NEXT:    vmov.32 q1[0], r5
-; CHECK-NEXT:    vmov.32 q1[1], r5
-; CHECK-NEXT:    ubfx r5, r2, #4, #1
-; CHECK-NEXT:    rsbs r5, r5, #0
-; CHECK-NEXT:    vmov.32 q1[2], r5
-; CHECK-NEXT:    vmov.32 q1[3], r5
-; CHECK-NEXT:    vmov.u8 r5, q0[12]
-; CHECK-NEXT:    sxtb r5, r5
-; CHECK-NEXT:    vmov.32 q2[0], r5
-; CHECK-NEXT:    asrs r5, r5, #31
-; CHECK-NEXT:    vmov.32 q2[1], r5
-; CHECK-NEXT:    vmov.u8 r5, q0[13]
-; CHECK-NEXT:    sxtb r5, r5
-; CHECK-NEXT:    vmov.32 q2[2], r5
-; CHECK-NEXT:    asrs r5, r5, #31
-; CHECK-NEXT:    vmov.32 q2[3], r5
+; CHECK-NEXT:    asrs r4, r4, #31
+; CHECK-NEXT:    vmov q2[3], q2[1], r4, r2
 ; CHECK-NEXT:    vand q1, q2, q1
 ; CHECK-NEXT:    vmov r4, s4
-; CHECK-NEXT:    vmov r5, s5
-; CHECK-NEXT:    adds.w r12, r12, r4
-; CHECK-NEXT:    vmov r4, s6
-; CHECK-NEXT:    adcs r5, r3
-; CHECK-NEXT:    vmov r3, s7
+; CHECK-NEXT:    vmov r2, s5
 ; CHECK-NEXT:    adds.w r4, r4, r12
-; CHECK-NEXT:    adc.w r12, r5, r3
-; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q1[0], r3
-; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q1[1], r3
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vmov.32 q1[3], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[14]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[15]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q2[3], r2
-; CHECK-NEXT:    vand q0, q2, q1
-; CHECK-NEXT:    vmov r3, s0
-; CHECK-NEXT:    vmov r2, s1
-; CHECK-NEXT:    vmov r5, s3
-; CHECK-NEXT:    adds r3, r3, r4
+; CHECK-NEXT:    adc.w r12, r3, r2
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    vmov r3, s7
+; CHECK-NEXT:    adds r2, r2, r4
+; CHECK-NEXT:    ubfx r4, r5, #8, #1
+; CHECK-NEXT:    ubfx r5, r5, #12, #1
+; CHECK-NEXT:    rsb.w r4, r4, #0
+; CHECK-NEXT:    rsb.w r5, r5, #0
+; CHECK-NEXT:    adc.w r3, r3, r12
+; CHECK-NEXT:    vmov q1[2], q1[0], r5, r4
+; CHECK-NEXT:    vmov q1[3], q1[1], r5, r4
+; CHECK-NEXT:    vmov.u8 r5, q0[14]
+; CHECK-NEXT:    vmov.u8 r4, q0[15]
+; CHECK-NEXT:    sxtb r5, r5
+; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    vmov q0[2], q0[0], r4, r5
+; CHECK-NEXT:    asrs r5, r5, #31
+; CHECK-NEXT:    asrs r4, r4, #31
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r5
+; CHECK-NEXT:    vand q0, q0, q1
+; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov r5, s1
+; CHECK-NEXT:    adds r2, r2, r4
 ; CHECK-NEXT:    vmov r4, s2
-; CHECK-NEXT:    adc.w r2, r2, r12
-; CHECK-NEXT:    adds r3, r3, r4
-; CHECK-NEXT:    adcs r2, r5
-; CHECK-NEXT:    adds r0, r0, r3
-; CHECK-NEXT:    adcs r1, r2
+; CHECK-NEXT:    adcs r3, r5
+; CHECK-NEXT:    vmov r5, s3
+; CHECK-NEXT:    adds r2, r2, r4
+; CHECK-NEXT:    adcs r3, r5
+; CHECK-NEXT:    adds r0, r0, r2
+; CHECK-NEXT:    adcs r1, r3
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13}
 ; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %c = icmp eq <16 x i8> %b, zeroinitializer
   %xx = sext <16 x i8> %x to <16 x i64>
   %s = select <16 x i1> %c, <16 x i64> %xx, <16 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v16i64(<16 x i64> %s)
   %r = add i64 %z, %a
   ret i64 %r
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i8_v2i64_acc_zext(<2 x i8> %x, <2 x i8> %b, i64 %a) {
 ; CHECK-LABEL: add_v2i8_v2i64_acc_zext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.i64 q2, #0xff
 ; CHECK-NEXT:    vand q1, q1, q2
 ; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vmov r2, s4
+; CHECK-NEXT:    vmov r3, s6
 ; CHECK-NEXT:    cmp r2, #0
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    cmp r2, #0
-; CHECK-NEXT:    cset r2, eq
-; CHECK-NEXT:    tst.w r2, #1
-; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.32 q3[3], r2
-; CHECK-NEXT:    vand q0, q0, q3
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    cset r3, eq
+; CHECK-NEXT:    tst.w r3, #1
+; CHECK-NEXT:    csetm r3, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r2
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r2, s3
 ; CHECK-NEXT:    vmov r3, s1
 ; CHECK-NEXT:    orr.w r12, r3, r2
 ; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    add r2, r3
 ; CHECK-NEXT:    adds r0, r0, r2
 ; CHECK-NEXT:    adc.w r1, r1, r12
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp eq <2 x i8> %b, zeroinitializer
   %xx = zext <2 x i8> %x to <2 x i64>
   %s = select <2 x i1> %c, <2 x i64> %xx, <2 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> %s)
   %r = add i64 %z, %a
   ret i64 %r
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i8_v2i64_acc_sext(<2 x i8> %x, <2 x i8> %b, i64 %a) {
 ; CHECK-LABEL: add_v2i8_v2i64_acc_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
 ; CHECK-NEXT:    vmov.i32 q2, #0xff
-; CHECK-NEXT:    vand q2, q1, q2
-; CHECK-NEXT:    vmov r2, s8
-; CHECK-NEXT:    cmp r2, #0
-; CHECK-NEXT:    cset r2, eq
-; CHECK-NEXT:    tst.w r2, #1
-; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q1[0], r2
-; CHECK-NEXT:    vmov.32 q1[1], r2
-; CHECK-NEXT:    vmov r2, s10
+; CHECK-NEXT:    vand q1, q1, q2
+; CHECK-NEXT:    vmov r2, s4
+; CHECK-NEXT:    vmov r3, s6
 ; CHECK-NEXT:    cmp r2, #0
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vmov.32 q1[3], r2
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    cset r3, eq
+; CHECK-NEXT:    tst.w r3, #1
+; CHECK-NEXT:    csetm r3, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r2
 ; CHECK-NEXT:    vmov r2, s0
+; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
+; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
 ; CHECK-NEXT:    asrs r2, r2, #31
-; CHECK-NEXT:    vmov.32 q2[3], r2
-; CHECK-NEXT:    vand q0, q2, q1
+; CHECK-NEXT:    asrs r3, r3, #31
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r2
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r12, s3
 ; CHECK-NEXT:    vmov lr, s1
 ; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    adc.w r3, lr, r12
 ; CHECK-NEXT:    adds r0, r0, r2
 ; CHECK-NEXT:    adcs r1, r3

 
 define arm_aapcs_vfpcc i64 @add_v2i64_v2i64_acc(<2 x i64> %x, <2 x i64> %b, i64 %a) {
 ; CHECK-LABEL: add_v2i64_v2i64_acc:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
 ; CHECK-NEXT:    vmov r2, s5
 ; CHECK-NEXT:    vmov r3, s4
+; CHECK-NEXT:    vmov r12, s7
 ; CHECK-NEXT:    orrs r2, r3
 ; CHECK-NEXT:    vmov r3, s6
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov r2, s7
-; CHECK-NEXT:    orrs r2, r3
-; CHECK-NEXT:    cset r2, eq
-; CHECK-NEXT:    tst.w r2, #1
-; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r2
-; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    orrs.w r3, r3, r12
+; CHECK-NEXT:    cset r3, eq
+; CHECK-NEXT:    tst.w r3, #1
+; CHECK-NEXT:    csetm r3, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r2
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r12, s3
 ; CHECK-NEXT:    vmov lr, s1
 ; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    adc.w r3, lr, r12
 ; CHECK-NEXT:    adds r0, r0, r2
 ; CHECK-NEXT:    adcs r1, r3

llvm/test/CodeGen/Thumb2/mve-vecreduce-mla.ll

entry:
   %m = mul <16 x i8> %x, %y
   %z = call i8 @llvm.vector.reduce.add.v16i8(<16 x i8> %m)
   ret i8 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v16i8_v16i64_zext(<16 x i8> %x, <16 x i8> %y) {
 ; CHECK-LABEL: add_v16i8_v16i64_zext:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r7, lr}
-; CHECK-NEXT:    push {r7, lr}
-; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-NEXT:    vpush {d8, d9, d10, d11}
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .vsave {d8, d9}
+; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    vmov.u8 r0, q1[0]
+; CHECK-NEXT:    vmov.u8 r1, q1[1]
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
 ; CHECK-NEXT:    vmov.u8 r1, q0[0]
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[1]
-; CHECK-NEXT:    vmov.32 q4[0], r1
-; CHECK-NEXT:    vmov.u8 r1, q0[1]
-; CHECK-NEXT:    vmov.32 q3[2], r0
+; CHECK-NEXT:    vmov.u8 r2, q0[1]
 ; CHECK-NEXT:    vmov.i64 q2, #0xff
-; CHECK-NEXT:    vmov.32 q4[2], r1
+; CHECK-NEXT:    vmov q4[2], q4[0], r2, r1
 ; CHECK-NEXT:    vand q3, q3, q2
 ; CHECK-NEXT:    vand q4, q4, q2
 ; CHECK-NEXT:    vmov r0, s14
 ; CHECK-NEXT:    vmov r1, s18
 ; CHECK-NEXT:    vmov r2, s12
 ; CHECK-NEXT:    vmov r3, s16
 ; CHECK-NEXT:    umull r12, r1, r1, r0
-; CHECK-NEXT:    vmov.u8 r0, q0[2]
-; CHECK-NEXT:    vmov.32 q4[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[3]
-; CHECK-NEXT:    vmov.32 q4[2], r0
+; CHECK-NEXT:    vmov.u8 r0, q1[3]
 ; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vand q4, q4, q2
-; CHECK-NEXT:    vmov r0, s16
 ; CHECK-NEXT:    orr.w lr, r3, r1
 ; CHECK-NEXT:    vmov.u8 r3, q1[2]
-; CHECK-NEXT:    vmov.32 q3[0], r3
-; CHECK-NEXT:    vmov.u8 r3, q1[3]
-; CHECK-NEXT:    vmov.32 q3[2], r3
-; CHECK-NEXT:    add r2, r12
+; CHECK-NEXT:    vmov q3[2], q3[0], r0, r3
+; CHECK-NEXT:    vmov.u8 r3, q0[2]
+; CHECK-NEXT:    vmov.u8 r1, q0[3]
 ; CHECK-NEXT:    vand q3, q3, q2
-; CHECK-NEXT:    vmov r3, s12
-; CHECK-NEXT:    umull r0, r3, r0, r3
-; CHECK-NEXT:    vmov.32 q5[0], r0
-; CHECK-NEXT:    vmov r0, s14
-; CHECK-NEXT:    vmov.32 q5[1], r3
-; CHECK-NEXT:    vmov r3, s18
-; CHECK-NEXT:    umull r0, r3, r3, r0
-; CHECK-NEXT:    vmov.32 q5[2], r0
-; CHECK-NEXT:    vmov.32 q5[3], r3
-; CHECK-NEXT:    vmov r1, s20
-; CHECK-NEXT:    vmov r0, s21
+; CHECK-NEXT:    vmov q4[2], q4[0], r1, r3
+; CHECK-NEXT:    vmov r0, s12
+; CHECK-NEXT:    vand q4, q4, q2
+; CHECK-NEXT:    vmov r3, s14
+; CHECK-NEXT:    vmov r1, s16
+; CHECK-NEXT:    vmov r4, s18
+; CHECK-NEXT:    umull r0, r1, r1, r0
+; CHECK-NEXT:    umull r3, r4, r4, r3
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r0
+; CHECK-NEXT:    vmov.u8 r3, q1[5]
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r1
+; CHECK-NEXT:    add.w r1, r2, r12
+; CHECK-NEXT:    vmov r2, s12
+; CHECK-NEXT:    vmov r0, s13
+; CHECK-NEXT:    adds r1, r1, r2
+; CHECK-NEXT:    vmov r2, s14
+; CHECK-NEXT:    adc.w r0, r0, lr
 ; CHECK-NEXT:    adds r1, r1, r2
-; CHECK-NEXT:    adc.w r2, lr, r0
-; CHECK-NEXT:    vmov r0, s22
-; CHECK-NEXT:    adds.w r12, r1, r0
-; CHECK-NEXT:    adc.w r1, r2, r3
 ; CHECK-NEXT:    vmov.u8 r2, q1[4]
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r2
+; CHECK-NEXT:    adc.w r12, r0, r4
 ; CHECK-NEXT:    vmov.u8 r3, q0[4]
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[5]
-; CHECK-NEXT:    vmov.32 q4[0], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[5]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.32 q4[2], r3
+; CHECK-NEXT:    vmov.u8 r4, q0[5]
+; CHECK-NEXT:    vmov q4[2], q4[0], r4, r3
 ; CHECK-NEXT:    vand q3, q3, q2
 ; CHECK-NEXT:    vand q4, q4, q2
 ; CHECK-NEXT:    vmov r2, s12
 ; CHECK-NEXT:    vmov r3, s16
+; CHECK-NEXT:    vmov r4, s14
+; CHECK-NEXT:    vmov r0, s18
 ; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q5[0], r2
+; CHECK-NEXT:    umull r0, r4, r0, r4
+; CHECK-NEXT:    vmov q3[2], q3[0], r0, r2
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r3
+; CHECK-NEXT:    vmov.u8 r3, q1[7]
+; CHECK-NEXT:    vmov r2, s12
+; CHECK-NEXT:    vmov r0, s13
+; CHECK-NEXT:    adds r1, r1, r2
 ; CHECK-NEXT:    vmov r2, s14
-; CHECK-NEXT:    vmov.32 q5[1], r3
-; CHECK-NEXT:    vmov r3, s18
-; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q5[2], r2
-; CHECK-NEXT:    vmov.32 q5[3], r3
-; CHECK-NEXT:    vmov r0, s20
-; CHECK-NEXT:    vmov r2, s21
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r2
-; CHECK-NEXT:    vmov r2, s22
-; CHECK-NEXT:    adds.w r12, r0, r2
+; CHECK-NEXT:    adc.w r0, r0, r12
+; CHECK-NEXT:    adds r1, r1, r2
 ; CHECK-NEXT:    vmov.u8 r2, q1[6]
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r2
+; CHECK-NEXT:    adc.w r12, r0, r4
 ; CHECK-NEXT:    vmov.u8 r3, q0[6]
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[7]
-; CHECK-NEXT:    vmov.32 q4[0], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[7]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.32 q4[2], r3
+; CHECK-NEXT:    vmov.u8 r4, q0[7]
+; CHECK-NEXT:    vmov q4[2], q4[0], r4, r3
 ; CHECK-NEXT:    vand q3, q3, q2
 ; CHECK-NEXT:    vand q4, q4, q2
 ; CHECK-NEXT:    vmov r2, s12
 ; CHECK-NEXT:    vmov r3, s16
+; CHECK-NEXT:    vmov r4, s14
+; CHECK-NEXT:    vmov r0, s18
 ; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q5[0], r2
+; CHECK-NEXT:    umull r0, r4, r0, r4
+; CHECK-NEXT:    vmov q3[2], q3[0], r0, r2
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r3
+; CHECK-NEXT:    vmov.u8 r3, q1[9]
+; CHECK-NEXT:    vmov r2, s12
+; CHECK-NEXT:    vmov r0, s13
+; CHECK-NEXT:    adds r1, r1, r2
 ; CHECK-NEXT:    vmov r2, s14
-; CHECK-NEXT:    vmov.32 q5[1], r3
-; CHECK-NEXT:    vmov r3, s18
-; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q5[2], r2
-; CHECK-NEXT:    vmov.32 q5[3], r3
-; CHECK-NEXT:    vmov r0, s20
-; CHECK-NEXT:    vmov r2, s21
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r2
-; CHECK-NEXT:    vmov r2, s22
-; CHECK-NEXT:    adds.w r12, r0, r2
+; CHECK-NEXT:    adc.w r0, r0, r12
+; CHECK-NEXT:    adds r1, r1, r2
 ; CHECK-NEXT:    vmov.u8 r2, q1[8]
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r2
+; CHECK-NEXT:    adc.w r12, r0, r4
 ; CHECK-NEXT:    vmov.u8 r3, q0[8]
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[9]
-; CHECK-NEXT:    vmov.32 q4[0], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[9]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.32 q4[2], r3
+; CHECK-NEXT:    vmov.u8 r4, q0[9]
+; CHECK-NEXT:    vmov q4[2], q4[0], r4, r3
 ; CHECK-NEXT:    vand q3, q3, q2
 ; CHECK-NEXT:    vand q4, q4, q2
 ; CHECK-NEXT:    vmov r2, s12
 ; CHECK-NEXT:    vmov r3, s16
+; CHECK-NEXT:    vmov r4, s14
+; CHECK-NEXT:    vmov r0, s18
 ; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q5[0], r2
+; CHECK-NEXT:    umull r0, r4, r0, r4
+; CHECK-NEXT:    vmov q3[2], q3[0], r0, r2
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r3
+; CHECK-NEXT:    vmov.u8 r3, q1[11]
+; CHECK-NEXT:    vmov r2, s12
+; CHECK-NEXT:    vmov r0, s13
+; CHECK-NEXT:    adds r1, r1, r2
 ; CHECK-NEXT:    vmov r2, s14
-; CHECK-NEXT:    vmov.32 q5[1], r3
-; CHECK-NEXT:    vmov r3, s18
-; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q5[2], r2
-; CHECK-NEXT:    vmov.32 q5[3], r3
-; CHECK-NEXT:    vmov r0, s20
-; CHECK-NEXT:    vmov r2, s21
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r2
-; CHECK-NEXT:    vmov r2, s22
-; CHECK-NEXT:    adds.w r12, r0, r2
+; CHECK-NEXT:    adc.w r0, r0, r12
+; CHECK-NEXT:    adds r1, r1, r2
 ; CHECK-NEXT:    vmov.u8 r2, q1[10]
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r2
+; CHECK-NEXT:    adc.w r12, r0, r4
 ; CHECK-NEXT:    vmov.u8 r3, q0[10]
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[11]
-; CHECK-NEXT:    vmov.32 q4[0], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[11]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.32 q4[2], r3
+; CHECK-NEXT:    vmov.u8 r4, q0[11]
+; CHECK-NEXT:    vmov q4[2], q4[0], r4, r3
 ; CHECK-NEXT:    vand q3, q3, q2
 ; CHECK-NEXT:    vand q4, q4, q2
 ; CHECK-NEXT:    vmov r2, s12
 ; CHECK-NEXT:    vmov r3, s16
+; CHECK-NEXT:    vmov r4, s14
+; CHECK-NEXT:    vmov r0, s18
 ; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q5[0], r2
+; CHECK-NEXT:    umull r0, r4, r0, r4
+; CHECK-NEXT:    vmov q3[2], q3[0], r0, r2
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r3
+; CHECK-NEXT:    vmov.u8 r3, q1[13]
+; CHECK-NEXT:    vmov r2, s12
+; CHECK-NEXT:    vmov r0, s13
+; CHECK-NEXT:    adds r1, r1, r2
 ; CHECK-NEXT:    vmov r2, s14
-; CHECK-NEXT:    vmov.32 q5[1], r3
-; CHECK-NEXT:    vmov r3, s18
-; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q5[2], r2
-; CHECK-NEXT:    vmov.32 q5[3], r3
-; CHECK-NEXT:    vmov r0, s20
-; CHECK-NEXT:    vmov r2, s21
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r2
-; CHECK-NEXT:    vmov r2, s22
-; CHECK-NEXT:    adds.w r12, r0, r2
+; CHECK-NEXT:    adc.w r0, r0, r12
+; CHECK-NEXT:    adds r1, r1, r2
 ; CHECK-NEXT:    vmov.u8 r2, q1[12]
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r2
+; CHECK-NEXT:    adc.w r12, r0, r4
 ; CHECK-NEXT:    vmov.u8 r3, q0[12]
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[13]
-; CHECK-NEXT:    vmov.32 q4[0], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[13]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.32 q4[2], r3
+; CHECK-NEXT:    vmov.u8 r4, q0[13]
+; CHECK-NEXT:    vmov q4[2], q4[0], r4, r3
 ; CHECK-NEXT:    vand q3, q3, q2
 ; CHECK-NEXT:    vand q4, q4, q2
 ; CHECK-NEXT:    vmov r2, s12
 ; CHECK-NEXT:    vmov r3, s16
+; CHECK-NEXT:    vmov r4, s14
+; CHECK-NEXT:    vmov r0, s18
 ; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q5[0], r2
-; CHECK-NEXT:    vmov r2, s14
-; CHECK-NEXT:    vmov.32 q5[1], r3
-; CHECK-NEXT:    vmov r3, s18
-; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q5[2], r2
-; CHECK-NEXT:    vmov.32 q5[3], r3
-; CHECK-NEXT:    vmov r0, s20
-; CHECK-NEXT:    vmov r2, s21
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r2
-; CHECK-NEXT:    vmov r2, s22
-; CHECK-NEXT:    adds r0, r0, r2
+; CHECK-NEXT:    umull r0, r4, r0, r4
+; CHECK-NEXT:    vmov q3[2], q3[0], r0, r2
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r3
+; CHECK-NEXT:    vmov.u8 r3, q1[15]
+; CHECK-NEXT:    vmov r2, s12
+; CHECK-NEXT:    vmov r0, s13
+; CHECK-NEXT:    adds r1, r1, r2
+; CHECK-NEXT:    adc.w r2, r12, r0
+; CHECK-NEXT:    vmov r0, s14
+; CHECK-NEXT:    adds r0, r0, r1
+; CHECK-NEXT:    adc.w r1, r2, r4
 ; CHECK-NEXT:    vmov.u8 r2, q1[14]
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[15]
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vmov.32 q3[2], r2
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
 ; CHECK-NEXT:    vmov.u8 r3, q0[14]
-; CHECK-NEXT:    vand q1, q3, q2
-; CHECK-NEXT:    vmov.32 q3[0], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[15]
-; CHECK-NEXT:    vmov.32 q3[2], r3
+; CHECK-NEXT:    vmov.u8 r4, q0[15]
+; CHECK-NEXT:    vand q1, q1, q2
+; CHECK-NEXT:    vmov q0[2], q0[0], r4, r3
 ; CHECK-NEXT:    vmov r2, s4
-; CHECK-NEXT:    vand q0, q3, q2
+; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    umlal r0, r1, r3, r2
 ; CHECK-NEXT:    vmov r2, s6
 ; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    umlal r0, r1, r3, r2
-; CHECK-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-NEXT:    pop {r7, pc}
+; CHECK-NEXT:    vpop {d8, d9}
+; CHECK-NEXT:    pop {r4, pc}
 entry:
   %xx = zext <16 x i8> %x to <16 x i64>
   %yy = zext <16 x i8> %y to <16 x i64>
   %m = mul <16 x i64> %xx, %yy
   %z = call i64 @llvm.vector.reduce.add.v16i64(<16 x i64> %m)
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v16i8_v16i64_sext(<16 x i8> %x, <16 x i8> %y) {
 ; CHECK-LABEL: add_v16i8_v16i64_sext:
 ; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r7, lr}
+; CHECK-NEXT:    push {r7, lr}
 ; CHECK-NEXT:    vmov.u8 r0, q1[0]
 ; CHECK-NEXT:    vmov.u8 r1, q0[0]
+; CHECK-NEXT:    vmov.u8 r2, q1[1]
+; CHECK-NEXT:    vmov.u8 r3, q0[1]
 ; CHECK-NEXT:    sxtb r0, r0
 ; CHECK-NEXT:    sxtb r1, r1
+; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    smull r0, r1, r1, r0
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[1]
-; CHECK-NEXT:    vmov.32 q2[1], r1
-; CHECK-NEXT:    vmov.u8 r1, q0[1]
+; CHECK-NEXT:    smull r2, r3, r3, r2
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r0
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r1
+; CHECK-NEXT:    vmov r1, s10
+; CHECK-NEXT:    vmov r2, s8
+; CHECK-NEXT:    vmov r0, s9
+; CHECK-NEXT:    adds.w lr, r2, r1
+; CHECK-NEXT:    vmov.u8 r2, q1[2]
+; CHECK-NEXT:    adc.w r12, r0, r3
+; CHECK-NEXT:    vmov.u8 r3, q0[2]
+; CHECK-NEXT:    vmov.u8 r0, q1[3]
+; CHECK-NEXT:    vmov.u8 r1, q0[3]
+; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    sxtb r0, r0
 ; CHECK-NEXT:    sxtb r1, r1
+; CHECK-NEXT:    smull r2, r3, r3, r2
 ; CHECK-NEXT:    smull r0, r1, r1, r0
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r1
-; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    vmov r3, s8
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r2
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r3
+; CHECK-NEXT:    vmov r2, s8
 ; CHECK-NEXT:    vmov r0, s9
-; CHECK-NEXT:    adds r2, r2, r3
-; CHECK-NEXT:    vmov.u8 r3, q0[2]
+; CHECK-NEXT:    vmov r3, s10
+; CHECK-NEXT:    adds.w r2, r2, lr
+; CHECK-NEXT:    adc.w r0, r0, r12
+; CHECK-NEXT:    adds.w lr, r2, r3
+; CHECK-NEXT:    vmov.u8 r3, q0[4]
 ; CHECK-NEXT:    adc.w r12, r0, r1
-; CHECK-NEXT:    vmov.u8 r1, q1[2]
+; CHECK-NEXT:    vmov.u8 r1, q1[4]
+; CHECK-NEXT:    vmov.u8 r0, q1[5]
+; CHECK-NEXT:    vmov.u8 r2, q0[5]
 ; CHECK-NEXT:    sxtb r1, r1
 ; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r1, r3, r3, r1
-; CHECK-NEXT:    vmov.32 q2[0], r1
-; CHECK-NEXT:    vmov.u8 r1, q1[3]
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[3]
-; CHECK-NEXT:    sxtb r1, r1
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r1, r3, r3, r1
-; CHECK-NEXT:    vmov.32 q2[2], r1
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    vmov r0, s8
-; CHECK-NEXT:    vmov r1, s9
-; CHECK-NEXT:    adds r0, r0, r2
-; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    adc.w r1, r1, r12
-; CHECK-NEXT:    adds.w r12, r0, r2
-; CHECK-NEXT:    vmov.u8 r2, q1[4]
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vmov.u8 r3, q0[4]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[5]
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[5]
+; CHECK-NEXT:    sxtb r0, r0
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    vmov r0, s8
-; CHECK-NEXT:    vmov r2, s9
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r2
-; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    adds.w r12, r0, r2
-; CHECK-NEXT:    vmov.u8 r2, q1[6]
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    smull r1, r3, r3, r1
+; CHECK-NEXT:    smull r0, r2, r2, r0
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r1
+; CHECK-NEXT:    vmov q2[3], q2[1], r2, r3
+; CHECK-NEXT:    vmov r1, s8
+; CHECK-NEXT:    vmov r0, s9
+; CHECK-NEXT:    vmov r3, s10
+; CHECK-NEXT:    adds.w r1, r1, lr
+; CHECK-NEXT:    adc.w r0, r0, r12
+; CHECK-NEXT:    adds.w lr, r1, r3
 ; CHECK-NEXT:    vmov.u8 r3, q0[6]
+; CHECK-NEXT:    adc.w r12, r0, r2
+; CHECK-NEXT:    vmov.u8 r2, q1[6]
+; CHECK-NEXT:    vmov.u8 r0, q1[7]
+; CHECK-NEXT:    vmov.u8 r1, q0[7]
 ; CHECK-NEXT:    sxtb r2, r2
 ; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    sxtb r0, r0
+; CHECK-NEXT:    sxtb r1, r1
 ; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[7]
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[7]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    vmov r0, s8
-; CHECK-NEXT:    vmov r2, s9
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r2
-; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    adds.w r12, r0, r2
-; CHECK-NEXT:    vmov.u8 r2, q1[8]
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    smull r0, r1, r1, r0
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r2
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r3
+; CHECK-NEXT:    vmov r2, s8
+; CHECK-NEXT:    vmov r0, s9
+; CHECK-NEXT:    vmov r3, s10
+; CHECK-NEXT:    adds.w r2, r2, lr
+; CHECK-NEXT:    adc.w r0, r0, r12
+; CHECK-NEXT:    adds.w lr, r2, r3
 ; CHECK-NEXT:    vmov.u8 r3, q0[8]
-; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    adc.w r12, r0, r1
+; CHECK-NEXT:    vmov.u8 r1, q1[8]
+; CHECK-NEXT:    vmov.u8 r0, q1[9]
+; CHECK-NEXT:    vmov.u8 r2, q0[9]
+; CHECK-NEXT:    sxtb r1, r1
 ; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[9]
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[9]
+; CHECK-NEXT:    sxtb r0, r0
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    vmov r0, s8
-; CHECK-NEXT:    vmov r2, s9
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r2
-; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    adds.w r12, r0, r2
-; CHECK-NEXT:    vmov.u8 r2, q1[10]
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    smull r1, r3, r3, r1
+; CHECK-NEXT:    smull r0, r2, r2, r0
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r1
+; CHECK-NEXT:    vmov q2[3], q2[1], r2, r3
+; CHECK-NEXT:    vmov r1, s8
+; CHECK-NEXT:    vmov r0, s9
+; CHECK-NEXT:    vmov r3, s10
+; CHECK-NEXT:    adds.w r1, r1, lr
+; CHECK-NEXT:    adc.w r0, r0, r12
+; CHECK-NEXT:    adds.w lr, r1, r3
 ; CHECK-NEXT:    vmov.u8 r3, q0[10]
+; CHECK-NEXT:    adc.w r12, r0, r2
+; CHECK-NEXT:    vmov.u8 r2, q1[10]
+; CHECK-NEXT:    vmov.u8 r0, q1[11]
+; CHECK-NEXT:    vmov.u8 r1, q0[11]
 ; CHECK-NEXT:    sxtb r2, r2
 ; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    sxtb r0, r0
+; CHECK-NEXT:    sxtb r1, r1
 ; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[11]
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[11]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    vmov r0, s8
-; CHECK-NEXT:    vmov r2, s9
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r2
-; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    adds.w r12, r0, r2
-; CHECK-NEXT:    vmov.u8 r2, q1[12]
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    smull r0, r1, r1, r0
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r2
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r3
+; CHECK-NEXT:    vmov r2, s8
+; CHECK-NEXT:    vmov r0, s9
+; CHECK-NEXT:    vmov r3, s10
+; CHECK-NEXT:    adds.w r2, r2, lr
+; CHECK-NEXT:    adc.w r0, r0, r12
+; CHECK-NEXT:    adds.w lr, r2, r3
 ; CHECK-NEXT:    vmov.u8 r3, q0[12]
-; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    adc.w r12, r0, r1
+; CHECK-NEXT:    vmov.u8 r1, q1[12]
+; CHECK-NEXT:    vmov.u8 r0, q1[13]
+; CHECK-NEXT:    vmov.u8 r2, q0[13]
+; CHECK-NEXT:    sxtb r1, r1
 ; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[13]
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[13]
+; CHECK-NEXT:    sxtb r0, r0
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    vmov r0, s8
-; CHECK-NEXT:    vmov r2, s9
-; CHECK-NEXT:    adds.w r0, r0, r12
-; CHECK-NEXT:    adcs r1, r2
-; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    adds r0, r0, r2
+; CHECK-NEXT:    smull r1, r3, r3, r1
+; CHECK-NEXT:    smull r0, r2, r2, r0
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r1
+; CHECK-NEXT:    vmov q2[3], q2[1], r2, r3
+; CHECK-NEXT:    vmov r1, s8
+; CHECK-NEXT:    vmov r0, s9
+; CHECK-NEXT:    adds.w r1, r1, lr
+; CHECK-NEXT:    adc.w r3, r12, r0
+; CHECK-NEXT:    vmov r0, s10
+; CHECK-NEXT:    adds r0, r0, r1
+; CHECK-NEXT:    adc.w r1, r3, r2
 ; CHECK-NEXT:    vmov.u8 r2, q1[14]
-; CHECK-NEXT:    adcs r1, r3
 ; CHECK-NEXT:    vmov.u8 r3, q0[14]
 ; CHECK-NEXT:    sxtb r2, r2
 ; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    smlal r0, r1, r3, r2
 ; CHECK-NEXT:    vmov.u8 r2, q1[15]
 ; CHECK-NEXT:    vmov.u8 r3, q0[15]
 ; CHECK-NEXT:    sxtb r2, r2
 ; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    smlal r0, r1, r3, r2
-; CHECK-NEXT:    bx lr
+; CHECK-NEXT:    pop {r7, pc}
 entry:
   %xx = sext <16 x i8> %x to <16 x i64>
   %yy = sext <16 x i8> %y to <16 x i64>
   %m = mul <16 x i64> %xx, %yy
   %z = call i64 @llvm.vector.reduce.add.v16i64(<16 x i64> %m)
   ret i64 %z
 }
 
entry:
   %z = call i8 @llvm.vector.reduce.add.v16i8(<16 x i8> %m)
   %r = add i8 %z, %a
   ret i8 %r
 }
 
 define arm_aapcs_vfpcc i64 @add_v16i8_v16i64_acc_zext(<16 x i8> %x, <16 x i8> %y, i64 %a) {
 ; CHECK-LABEL: add_v16i8_v16i64_acc_zext:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-NEXT:    push {r4, r5, r7, lr}
-; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-NEXT:    vpush {d8, d9, d10, d11}
+; CHECK-NEXT:    .save {r4, r5, r6, lr}
+; CHECK-NEXT:    push {r4, r5, r6, lr}
+; CHECK-NEXT:    .vsave {d8, d9}
+; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    vmov.u8 r2, q1[0]
+; CHECK-NEXT:    vmov.u8 r3, q1[1]
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r2
 ; CHECK-NEXT:    vmov.u8 r3, q0[0]
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[1]
-; CHECK-NEXT:    vmov.32 q4[0], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[1]
-; CHECK-NEXT:    vmov.32 q3[2], r2
+; CHECK-NEXT:    vmov.u8 r2, q0[1]
 ; CHECK-NEXT:    vmov.i64 q2, #0xff
-; CHECK-NEXT:    vmov.32 q4[2], r3
+; CHECK-NEXT:    vmov q4[2], q4[0], r2, r3
 ; CHECK-NEXT:    vand q3, q3, q2
 ; CHECK-NEXT:    vand q4, q4, q2
-; CHECK-NEXT:    vmov r2, s14
-; CHECK-NEXT:    vmov r3, s18
-; CHECK-NEXT:    vmov.u8 r4, q0[2]
-; CHECK-NEXT:    umull r12, lr, r3, r2
+; CHECK-NEXT:    vmov r12, s14
+; CHECK-NEXT:    vmov r2, s18
+; CHECK-NEXT:    vmov.u8 r4, q1[3]
 ; CHECK-NEXT:    vmov r3, s16
+; CHECK-NEXT:    vmov.u8 r5, q0[3]
+; CHECK-NEXT:    umull r12, lr, r2, r12
 ; CHECK-NEXT:    vmov r2, s12
-; CHECK-NEXT:    vmov.32 q4[0], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[3]
-; CHECK-NEXT:    vmov.32 q4[2], r4
-; CHECK-NEXT:    vand q4, q4, q2
-; CHECK-NEXT:    vmov r4, s16
 ; CHECK-NEXT:    umull r2, r3, r3, r2
 ; CHECK-NEXT:    orr.w lr, lr, r3
 ; CHECK-NEXT:    vmov.u8 r3, q1[2]
-; CHECK-NEXT:    vmov.32 q3[0], r3
-; CHECK-NEXT:    vmov.u8 r3, q1[3]
-; CHECK-NEXT:    vmov.32 q3[2], r3
-; CHECK-NEXT:    add r2, r12
+; CHECK-NEXT:    vmov q3[2], q3[0], r4, r3
+; CHECK-NEXT:    vmov.u8 r4, q0[2]
+; CHECK-NEXT:    vmov q4[2], q4[0], r5, r4
 ; CHECK-NEXT:    vand q3, q3, q2
+; CHECK-NEXT:    vand q4, q4, q2
 ; CHECK-NEXT:    vmov r3, s12
+; CHECK-NEXT:    vmov r4, s16
+; CHECK-NEXT:    add r2, r12
+; CHECK-NEXT:    vmov r5, s14
+; CHECK-NEXT:    vmov r6, s18
 ; CHECK-NEXT:    umull r3, r4, r4, r3
-; CHECK-NEXT:    vmov.32 q5[0], r3
-; CHECK-NEXT:    vmov r3, s14
-; CHECK-NEXT:    vmov.32 q5[1], r4
-; CHECK-NEXT:    vmov r4, s18
-; CHECK-NEXT:    umull r3, r4, r4, r3
-; CHECK-NEXT:    vmov.32 q5[2], r3
-; CHECK-NEXT:    vmov.32 q5[3], r4
-; CHECK-NEXT:    vmov r3, s20
-; CHECK-NEXT:    vmov r5, s21
-; CHECK-NEXT:    adds r2, r2, r3
-; CHECK-NEXT:    adc.w r3, lr, r5
-; CHECK-NEXT:    vmov r5, s22
-; CHECK-NEXT:    adds.w r12, r2, r5
-; CHECK-NEXT:    vmov.u8 r5, q1[4]
-; CHECK-NEXT:    adcs r3, r4
-; CHECK-NEXT:    vmov.u8 r4, q0[4]
-; CHECK-NEXT:    vmov.32 q3[0], r5
-; CHECK-NEXT:    vmov.u8 r5, q1[5]
-; CHECK-NEXT:    vmov.32 q4[0], r4
+; CHECK-NEXT:    umull r5, r6, r6, r5
+; CHECK-NEXT:    vmov q3[2], q3[0], r5, r3
+; CHECK-NEXT:    vmov q3[3], q3[1], r6, r4
 ; CHECK-NEXT:    vmov.u8 r4, q0[5]
-; CHECK-NEXT:    vmov.32 q3[2], r5
-; CHECK-NEXT:    vmov.32 q4[2], r4
-; CHECK-NEXT:    vand q3, q3, q2
-; CHECK-NEXT:    vand q4, q4, q2
 ; CHECK-NEXT:    vmov r5, s12
-; CHECK-NEXT:    vmov r4, s16
-; CHECK-NEXT:    umull r5, r4, r4, r5
-; CHECK-NEXT:    vmov.32 q5[0], r5
+; CHECK-NEXT:    vmov r3, s13
+; CHECK-NEXT:    adds r2, r2, r5
 ; CHECK-NEXT:    vmov r5, s14
-; CHECK-NEXT:    vmov.32 q5[1], r4
-; CHECK-NEXT:    vmov r4, s18
-; CHECK-NEXT:    umull r5, r4, r4, r5
-; CHECK-NEXT:    vmov.32 q5[2], r5
-; CHECK-NEXT:    vmov.32 q5[3], r4
-; CHECK-NEXT:    vmov r2, s20
-; CHECK-NEXT:    vmov r5, s21
-; CHECK-NEXT:    adds.w r2, r2, r12
-; CHECK-NEXT:    adcs r3, r5
-; CHECK-NEXT:    vmov r5, s22
+; CHECK-NEXT:    adc.w r3, r3, lr
 ; CHECK-NEXT:    adds.w r12, r2, r5
-; CHECK-NEXT:    vmov.u8 r5, q1[6]
-; CHECK-NEXT:    adcs r3, r4
-; CHECK-NEXT:    vmov.u8 r4, q0[6]
-; CHECK-NEXT:    vmov.32 q3[0], r5
+; CHECK-NEXT:    vmov.u8 r5, q1[5]
+; CHECK-NEXT:    adcs r3, r6
+; CHECK-NEXT:    vmov.u8 r6, q1[4]
+; CHECK-NEXT:    vmov q3[2], q3[0], r5, r6
+; CHECK-NEXT:    vmov.u8 r5, q0[4]
+; CHECK-NEXT:    vmov q4[2], q4[0], r4, r5
+; CHECK-NEXT:    vand q3, q3, q2
+; CHECK-NEXT:    vand q4, q4, q2
+; CHECK-NEXT:    vmov r6, s12
+; CHECK-NEXT:    vmov r5, s16
+; CHECK-NEXT:    vmov r4, s14
+; CHECK-NEXT:    vmov r2, s18
+; CHECK-NEXT:    umull r6, r5, r5, r6
+; CHECK-NEXT:    umull r2, r4, r2, r4
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r6
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r5
 ; CHECK-NEXT:    vmov.u8 r5, q1[7]
-; CHECK-NEXT:    vmov.32 q4[0], r4
+; CHECK-NEXT:    vmov r6, s12
+; CHECK-NEXT:    vmov r2, s13
+; CHECK-NEXT:    adds.w r6, r6, r12
+; CHECK-NEXT:    adcs r2, r3
+; CHECK-NEXT:    vmov r3, s14
+; CHECK-NEXT:    adds r3, r3, r6
+; CHECK-NEXT:    vmov.u8 r6, q1[6]
+; CHECK-NEXT:    vmov q3[2], q3[0], r5, r6
+; CHECK-NEXT:    adc.w r12, r2, r4
+; CHECK-NEXT:    vmov.u8 r5, q0[6]
 ; CHECK-NEXT:    vmov.u8 r4, q0[7]
-; CHECK-NEXT:    vmov.32 q3[2], r5
-; CHECK-NEXT:    vmov.32 q4[2], r4
+; CHECK-NEXT:    vmov q4[2], q4[0], r4, r5
 ; CHECK-NEXT:    vand q3, q3, q2
 ; CHECK-NEXT:    vand q4, q4, q2
-; CHECK-NEXT:    vmov r5, s12
-; CHECK-NEXT:    vmov r4, s16
-; CHECK-NEXT:    umull r5, r4, r4, r5
-; CHECK-NEXT:    vmov.32 q5[0], r5
-; CHECK-NEXT:    vmov r5, s14
-; CHECK-NEXT:    vmov.32 q5[1], r4
-; CHECK-NEXT:    vmov r4, s18
-; CHECK-NEXT:    umull r5, r4, r4, r5
-; CHECK-NEXT:    vmov.32 q5[2], r5
-; CHECK-NEXT:    vmov.32 q5[3], r4
-; CHECK-NEXT:    vmov r2, s20
-; CHECK-NEXT:    vmov r5, s21
-; CHECK-NEXT:    adds.w r2, r2, r12
-; CHECK-NEXT:    adcs r3, r5
-; CHECK-NEXT:    vmov r5, s22
-; CHECK-NEXT:    adds.w r12, r2, r5
-; CHECK-NEXT:    vmov.u8 r5, q1[8]
-; CHECK-NEXT:    adcs r3, r4
-; CHECK-NEXT:    vmov.u8 r4, q0[8]
-; CHECK-NEXT:    vmov.32 q3[0], r5
+; CHECK-NEXT:    vmov r6, s12
+; CHECK-NEXT:    vmov r5, s16
+; CHECK-NEXT:    vmov r4, s14
+; CHECK-NEXT:    vmov r2, s18
+; CHECK-NEXT:    umull r6, r5, r5, r6
+; CHECK-NEXT:    umull r2, r4, r2, r4
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r6
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r5
 ; CHECK-NEXT:    vmov.u8 r5, q1[9]
-; CHECK-NEXT:    vmov.32 q4[0], r4
+; CHECK-NEXT:    vmov r6, s12
+; CHECK-NEXT:    vmov r2, s13
+; CHECK-NEXT:    adds r3, r3, r6
+; CHECK-NEXT:    vmov r6, s14
+; CHECK-NEXT:    adc.w r2, r2, r12
+; CHECK-NEXT:    adds r3, r3, r6
+; CHECK-NEXT:    vmov.u8 r6, q1[8]
+; CHECK-NEXT:    vmov q3[2], q3[0], r5, r6
+; CHECK-NEXT:    adc.w r12, r2, r4
+; CHECK-NEXT:    vmov.u8 r5, q0[8]
 ; CHECK-NEXT:    vmov.u8 r4, q0[9]
-; CHECK-NEXT:    vmov.32 q3[2], r5
-; CHECK-NEXT:    vmov.32 q4[2], r4
+; CHECK-NEXT:    vmov q4[2], q4[0], r4, r5
 ; CHECK-NEXT:    vand q3, q3, q2
 ; CHECK-NEXT:    vand q4, q4, q2
-; CHECK-NEXT:    vmov r5, s12
-; CHECK-NEXT:    vmov r4, s16
-; CHECK-NEXT:    umull r5, r4, r4, r5
-; CHECK-NEXT:    vmov.32 q5[0], r5
-; CHECK-NEXT:    vmov r5, s14
-; CHECK-NEXT:    vmov.32 q5[1], r4
-; CHECK-NEXT:    vmov r4, s18
-; CHECK-NEXT:    umull r5, r4, r4, r5
-; CHECK-NEXT:    vmov.32 q5[2], r5
-; CHECK-NEXT:    vmov.32 q5[3], r4
-; CHECK-NEXT:    vmov r2, s20
-; CHECK-NEXT:    vmov r5, s21
-; CHECK-NEXT:    adds.w r2, r2, r12
-; CHECK-NEXT:    adcs r3, r5
-; CHECK-NEXT:    vmov r5, s22
-; CHECK-NEXT:    adds.w r12, r2, r5
-; CHECK-NEXT:    vmov.u8 r5, q1[10]
-; CHECK-NEXT:    adcs r3, r4
-; CHECK-NEXT:    vmov.u8 r4, q0[10]
-; CHECK-NEXT:    vmov.32 q3[0], r5
+; CHECK-NEXT:    vmov r6, s12
+; CHECK-NEXT:    vmov r5, s16
+; CHECK-NEXT:    vmov r4, s14
+; CHECK-NEXT:    vmov r2, s18
+; CHECK-NEXT:    umull r6, r5, r5, r6
+; CHECK-NEXT:    umull r2, r4, r2, r4
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r6
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r5
 ; CHECK-NEXT:    vmov.u8 r5, q1[11]
-; CHECK-NEXT:    vmov.32 q4[0], r4
+; CHECK-NEXT:    vmov r6, s12
+; CHECK-NEXT:    vmov r2, s13
+; CHECK-NEXT:    adds r3, r3, r6
+; CHECK-NEXT:    vmov r6, s14
+; CHECK-NEXT:    adc.w r2, r2, r12
+; CHECK-NEXT:    adds r3, r3, r6
+; CHECK-NEXT:    vmov.u8 r6, q1[10]
+; CHECK-NEXT:    vmov q3[2], q3[0], r5, r6
+; CHECK-NEXT:    adc.w r12, r2, r4
+; CHECK-NEXT:    vmov.u8 r5, q0[10]
 ; CHECK-NEXT:    vmov.u8 r4, q0[11]
-; CHECK-NEXT:    vmov.32 q3[2], r5
-; CHECK-NEXT:    vmov.32 q4[2], r4
+; CHECK-NEXT:    vmov q4[2], q4[0], r4, r5
 ; CHECK-NEXT:    vand q3, q3, q2
 ; CHECK-NEXT:    vand q4, q4, q2
-; CHECK-NEXT:    vmov r5, s12
-; CHECK-NEXT:    vmov r4, s16
-; CHECK-NEXT:    umull r5, r4, r4, r5
-; CHECK-NEXT:    vmov.32 q5[0], r5
-; CHECK-NEXT:    vmov r5, s14
-; CHECK-NEXT:    vmov.32 q5[1], r4
-; CHECK-NEXT:    vmov r4, s18
-; CHECK-NEXT:    umull r5, r4, r4, r5
-; CHECK-NEXT:    vmov.32 q5[2], r5
-; CHECK-NEXT:    vmov.32 q5[3], r4
-; CHECK-NEXT:    vmov r2, s20
-; CHECK-NEXT:    vmov r5, s21
-; CHECK-NEXT:    adds.w r2, r2, r12
-; CHECK-NEXT:    adcs r3, r5
-; CHECK-NEXT:    vmov r5, s22
-; CHECK-NEXT:    adds.w r12, r2, r5
-; CHECK-NEXT:    vmov.u8 r5, q1[12]
-; CHECK-NEXT:    adcs r3, r4
-; CHECK-NEXT:    vmov.u8 r4, q0[12]
-; CHECK-NEXT:    vmov.32 q3[0], r5
+; CHECK-NEXT:    vmov r6, s12
+; CHECK-NEXT:    vmov r5, s16
+; CHECK-NEXT:    vmov r4, s14
+; CHECK-NEXT:    vmov r2, s18
+; CHECK-NEXT:    umull r6, r5, r5, r6
+; CHECK-NEXT:    umull r2, r4, r2, r4
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r6
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r5
 ; CHECK-NEXT:    vmov.u8 r5, q1[13]
-; CHECK-NEXT:    vmov.32 q4[0], r4
+; CHECK-NEXT:    vmov r6, s12
+; CHECK-NEXT:    vmov r2, s13
+; CHECK-NEXT:    adds r3, r3, r6
+; CHECK-NEXT:    vmov r6, s14
+; CHECK-NEXT:    adc.w r2, r2, r12
+; CHECK-NEXT:    adds r3, r3, r6
+; CHECK-NEXT:    vmov.u8 r6, q1[12]
+; CHECK-NEXT:    vmov q3[2], q3[0], r5, r6
+; CHECK-NEXT:    adc.w r12, r2, r4
+; CHECK-NEXT:    vmov.u8 r5, q0[12]
 ; CHECK-NEXT:    vmov.u8 r4, q0[13]
-; CHECK-NEXT:    vmov.32 q3[2], r5
-; CHECK-NEXT:    vmov.32 q4[2], r4
+; CHECK-NEXT:    vmov q4[2], q4[0], r4, r5
 ; CHECK-NEXT:    vand q3, q3, q2
 ; CHECK-NEXT:    vand q4, q4, q2
-; CHECK-NEXT:    vmov r5, s12
-; CHECK-NEXT:    vmov r4, s16
-; CHECK-NEXT:    umull r5, r4, r4, r5
-; CHECK-NEXT:    vmov.32 q5[0], r5
-; CHECK-NEXT:    vmov r5, s14
-; CHECK-NEXT:    vmov.32 q5[1], r4
-; CHECK-NEXT:    vmov r4, s18
-; CHECK-NEXT:    umull r5, r4, r4, r5
-; CHECK-NEXT:    vmov.32 q5[2], r5
-; CHECK-NEXT:    vmov.32 q5[3], r4
-; CHECK-NEXT:    vmov r2, s20
-; CHECK-NEXT:    vmov r5, s21
-; CHECK-NEXT:    adds.w r2, r2, r12
-; CHECK-NEXT:    adcs r3, r5
-; CHECK-NEXT:    vmov r5, s22
-; CHECK-NEXT:    adds r2, r2, r5
-; CHECK-NEXT:    vmov.u8 r5, q1[14]
-; CHECK-NEXT:    vmov.32 q3[0], r5
+; CHECK-NEXT:    vmov r6, s12
+; CHECK-NEXT:    vmov r5, s16
+; CHECK-NEXT:    vmov r4, s14
+; CHECK-NEXT:    vmov r2, s18
+; CHECK-NEXT:    umull r6, r5, r5, r6
+; CHECK-NEXT:    umull r2, r4, r2, r4
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r6
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r5
 ; CHECK-NEXT:    vmov.u8 r5, q1[15]
-; CHECK-NEXT:    adcs r3, r4
-; CHECK-NEXT:    vmov.32 q3[2], r5
-; CHECK-NEXT:    vmov.u8 r4, q0[14]
-; CHECK-NEXT:    vand q1, q3, q2
-; CHECK-NEXT:    vmov.32 q3[0], r4
+; CHECK-NEXT:    vmov r6, s12
+; CHECK-NEXT:    vmov r2, s13
+; CHECK-NEXT:    adds r3, r3, r6
+; CHECK-NEXT:    vmov r6, s14
+; CHECK-NEXT:    adc.w r2, r2, r12
+; CHECK-NEXT:    adds r3, r3, r6
+; CHECK-NEXT:    vmov.u8 r6, q1[14]
+; CHECK-NEXT:    vmov q1[2], q1[0], r5, r6
+; CHECK-NEXT:    adcs r2, r4
+; CHECK-NEXT:    vmov.u8 r5, q0[14]
 ; CHECK-NEXT:    vmov.u8 r4, q0[15]
-; CHECK-NEXT:    vmov.32 q3[2], r4
-; CHECK-NEXT:    vmov r5, s4
-; CHECK-NEXT:    vand q0, q3, q2
-; CHECK-NEXT:    vmov r4, s0
-; CHECK-NEXT:    umlal r2, r3, r4, r5
-; CHECK-NEXT:    vmov r5, s6
-; CHECK-NEXT:    vmov r4, s2
-; CHECK-NEXT:    umlal r2, r3, r4, r5
-; CHECK-NEXT:    adds r0, r0, r2
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-NEXT:    pop {r4, r5, r7, pc}
+; CHECK-NEXT:    vmov q0[2], q0[0], r4, r5
+; CHECK-NEXT:    vand q1, q1, q2
+; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    vmov r6, s4
+; CHECK-NEXT:    vmov r5, s0
+; CHECK-NEXT:    umlal r3, r2, r5, r6
+; CHECK-NEXT:    vmov r6, s6
+; CHECK-NEXT:    vmov r5, s2
+; CHECK-NEXT:    umlal r3, r2, r5, r6
+; CHECK-NEXT:    adds r0, r0, r3
+; CHECK-NEXT:    adcs r1, r2
+; CHECK-NEXT:    vpop {d8, d9}
+; CHECK-NEXT:    pop {r4, r5, r6, pc}
 entry:
   %xx = zext <16 x i8> %x to <16 x i64>
   %yy = zext <16 x i8> %y to <16 x i64>
   %m = mul <16 x i64> %xx, %yy
   %z = call i64 @llvm.vector.reduce.add.v16i64(<16 x i64> %m)
   %r = add i64 %z, %a
   ret i64 %r
 }
 
 define arm_aapcs_vfpcc i64 @add_v16i8_v16i64_acc_sext(<16 x i8> %x, <16 x i8> %y, i64 %a) {
 ; CHECK-LABEL: add_v16i8_v16i64_acc_sext:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, lr}
-; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    vmov.u8 r2, q1[0]
 ; CHECK-NEXT:    vmov.u8 r3, q0[0]
 ; CHECK-NEXT:    sxtb r2, r2
 ; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[1]
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[1]
-; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    smull lr, r12, r3, r2
+; CHECK-NEXT:    vmov.u8 r3, q1[1]
+; CHECK-NEXT:    vmov.u8 r2, q0[1]
 ; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r3
+; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    vmov.u8 r4, q1[3]
+; CHECK-NEXT:    smull r2, r3, r2, r3
+; CHECK-NEXT:    vmov.u8 r5, q0[3]
+; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    sxtb r5, r5
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, lr
+; CHECK-NEXT:    smull r4, r5, r5, r4
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r12
 ; CHECK-NEXT:    vmov lr, s10
 ; CHECK-NEXT:    vmov r2, s8
 ; CHECK-NEXT:    vmov r12, s9
 ; CHECK-NEXT:    adds.w lr, lr, r2
 ; CHECK-NEXT:    vmov.u8 r2, q1[2]
 ; CHECK-NEXT:    adc.w r12, r12, r3
 ; CHECK-NEXT:    vmov.u8 r3, q0[2]
 ; CHECK-NEXT:    sxtb r2, r2
 ; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[3]
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[3]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    vmov r4, s8
-; CHECK-NEXT:    vmov r2, s9
-; CHECK-NEXT:    adds.w r4, r4, lr
-; CHECK-NEXT:    adc.w r12, r12, r2
-; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    adds.w lr, r4, r2
-; CHECK-NEXT:    vmov.u8 r4, q1[4]
-; CHECK-NEXT:    vmov.u8 r2, q0[4]
-; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    adc.w r12, r12, r3
-; CHECK-NEXT:    smull r2, r4, r2, r4
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[5]
-; CHECK-NEXT:    vmov.32 q2[1], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[5]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    smull r2, r4, r4, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r4
+; CHECK-NEXT:    vmov q2[2], q2[0], r4, r2
+; CHECK-NEXT:    vmov q2[3], q2[1], r5, r3
 ; CHECK-NEXT:    vmov r3, s8
 ; CHECK-NEXT:    vmov r2, s9
+; CHECK-NEXT:    vmov r4, s10
 ; CHECK-NEXT:    adds.w r3, r3, lr
-; CHECK-NEXT:    adc.w r12, r12, r2
-; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    adds.w lr, r3, r2
-; CHECK-NEXT:    vmov.u8 r2, q0[6]
-; CHECK-NEXT:    adc.w r12, r12, r4
-; CHECK-NEXT:    vmov.u8 r4, q1[6]
+; CHECK-NEXT:    adc.w r2, r2, r12
+; CHECK-NEXT:    adds.w lr, r3, r4
+; CHECK-NEXT:    vmov.u8 r4, q0[4]
+; CHECK-NEXT:    adc.w r12, r2, r5
+; CHECK-NEXT:    vmov.u8 r5, q1[4]
+; CHECK-NEXT:    vmov.u8 r2, q1[5]
+; CHECK-NEXT:    vmov.u8 r3, q0[5]
+; CHECK-NEXT:    sxtb r5, r5
 ; CHECK-NEXT:    sxtb r4, r4
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    smull r2, r4, r2, r4
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[7]
-; CHECK-NEXT:    vmov.32 q2[1], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[7]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    smull r2, r4, r4, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r4
-; CHECK-NEXT:    vmov r3, s8
+; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    smull r5, r4, r4, r5
+; CHECK-NEXT:    smull r2, r3, r3, r2
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r5
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r4
+; CHECK-NEXT:    vmov r4, s8
 ; CHECK-NEXT:    vmov r2, s9
-; CHECK-NEXT:    adds.w r3, r3, lr
-; CHECK-NEXT:    adc.w r12, r12, r2
-; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    adds.w lr, r3, r2
-; CHECK-NEXT:    vmov.u8 r2, q0[8]
-; CHECK-NEXT:    adc.w r12, r12, r4
-; CHECK-NEXT:    vmov.u8 r4, q1[8]
+; CHECK-NEXT:    adds.w r5, lr, r4
+; CHECK-NEXT:    vmov r4, s10
+; CHECK-NEXT:    adc.w r2, r2, r12
+; CHECK-NEXT:    adds.w lr, r5, r4
+; CHECK-NEXT:    vmov.u8 r4, q0[6]
+; CHECK-NEXT:    adc.w r12, r2, r3
+; CHECK-NEXT:    vmov.u8 r3, q1[6]
+; CHECK-NEXT:    vmov.u8 r2, q1[7]
+; CHECK-NEXT:    vmov.u8 r5, q0[7]
+; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    sxtb r4, r4
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    smull r2, r4, r2, r4
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[9]
-; CHECK-NEXT:    vmov.32 q2[1], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[9]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    smull r2, r4, r4, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r4
+; CHECK-NEXT:    sxtb r5, r5
+; CHECK-NEXT:    smull r3, r4, r4, r3
+; CHECK-NEXT:    smull r2, r5, r5, r2
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r5, r4
 ; CHECK-NEXT:    vmov r3, s8
 ; CHECK-NEXT:    vmov r2, s9
+; CHECK-NEXT:    vmov r4, s10
 ; CHECK-NEXT:    adds.w r3, r3, lr
-; CHECK-NEXT:    adc.w r12, r12, r2
-; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    adds.w lr, r3, r2
-; CHECK-NEXT:    vmov.u8 r2, q0[10]
-; CHECK-NEXT:    adc.w r12, r12, r4
-; CHECK-NEXT:    vmov.u8 r4, q1[10]
+; CHECK-NEXT:    adc.w r2, r2, r12
+; CHECK-NEXT:    adds.w lr, r3, r4
+; CHECK-NEXT:    vmov.u8 r4, q0[8]
+; CHECK-NEXT:    adc.w r12, r2, r5
+; CHECK-NEXT:    vmov.u8 r5, q1[8]
+; CHECK-NEXT:    vmov.u8 r2, q1[9]
+; CHECK-NEXT:    vmov.u8 r3, q0[9]
+; CHECK-NEXT:    sxtb r5, r5
 ; CHECK-NEXT:    sxtb r4, r4
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    smull r2, r4, r2, r4
-; CHECK-NEXT:    vmov.32 q2[0], r2
+; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    smull r5, r4, r4, r5
+; CHECK-NEXT:    smull r2, r3, r3, r2
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r5
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r4
+; CHECK-NEXT:    vmov r4, s8
+; CHECK-NEXT:    vmov r2, s9
+; CHECK-NEXT:    adds.w r5, lr, r4
+; CHECK-NEXT:    vmov r4, s10
+; CHECK-NEXT:    adc.w r2, r2, r12
+; CHECK-NEXT:    adds.w lr, r5, r4
+; CHECK-NEXT:    vmov.u8 r4, q0[10]
+; CHECK-NEXT:    adc.w r12, r2, r3
+; CHECK-NEXT:    vmov.u8 r3, q1[10]
 ; CHECK-NEXT:    vmov.u8 r2, q1[11]
-; CHECK-NEXT:    vmov.32 q2[1], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[11]
-; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    vmov.u8 r5, q0[11]
+; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    smull r2, r4, r4, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r4
+; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    sxtb r5, r5
+; CHECK-NEXT:    smull r3, r4, r4, r3
+; CHECK-NEXT:    smull r2, r5, r5, r2
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r5, r4
 ; CHECK-NEXT:    vmov r3, s8
 ; CHECK-NEXT:    vmov r2, s9
+; CHECK-NEXT:    vmov r4, s10
 ; CHECK-NEXT:    adds.w r3, r3, lr
-; CHECK-NEXT:    adc.w r12, r12, r2
-; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    adds.w lr, r3, r2
-; CHECK-NEXT:    vmov.u8 r2, q0[12]
-; CHECK-NEXT:    adc.w r12, r12, r4
-; CHECK-NEXT:    vmov.u8 r4, q1[12]
-; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    smull r2, r4, r2, r4
-; CHECK-NEXT:    vmov.32 q2[0], r2
+; CHECK-NEXT:    adc.w r2, r2, r12
+; CHECK-NEXT:    adds.w lr, r3, r4
+; CHECK-NEXT:    vmov.u8 r4, q0[12]
+; CHECK-NEXT:    adc.w r12, r2, r5
+; CHECK-NEXT:    vmov.u8 r5, q1[12]
 ; CHECK-NEXT:    vmov.u8 r2, q1[13]
-; CHECK-NEXT:    vmov.32 q2[1], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[13]
-; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    vmov.u8 r3, q0[13]
+; CHECK-NEXT:    sxtb r5, r5
 ; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    smull r2, r4, r4, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r4
-; CHECK-NEXT:    vmov r3, s8
+; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    smull r5, r4, r4, r5
+; CHECK-NEXT:    smull r2, r3, r3, r2
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r5
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r4
+; CHECK-NEXT:    vmov r4, s8
 ; CHECK-NEXT:    vmov r2, s9
-; CHECK-NEXT:    adds.w r3, r3, lr
-; CHECK-NEXT:    adc.w r12, r12, r2
-; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    adds r2, r2, r3
-; CHECK-NEXT:    adc.w r3, r12, r4
-; CHECK-NEXT:    vmov.u8 r4, q1[14]
-; CHECK-NEXT:    sxtb.w r12, r4
+; CHECK-NEXT:    adds.w r5, lr, r4
+; CHECK-NEXT:    vmov r4, s10
+; CHECK-NEXT:    adc.w r2, r2, r12
+; CHECK-NEXT:    adds r5, r5, r4
 ; CHECK-NEXT:    vmov.u8 r4, q0[14]
+; CHECK-NEXT:    adcs r2, r3
+; CHECK-NEXT:    vmov.u8 r3, q1[14]
+; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    smlal r2, r3, r4, r12
-; CHECK-NEXT:    vmov.u8 r4, q1[15]
-; CHECK-NEXT:    sxtb.w r12, r4
+; CHECK-NEXT:    smlal r5, r2, r4, r3
+; CHECK-NEXT:    vmov.u8 r3, q1[15]
 ; CHECK-NEXT:    vmov.u8 r4, q0[15]
+; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    smlal r2, r3, r4, r12
-; CHECK-NEXT:    adds r0, r0, r2
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    pop {r4, pc}
+; CHECK-NEXT:    smlal r5, r2, r4, r3
+; CHECK-NEXT:    adds r0, r0, r5
+; CHECK-NEXT:    adcs r1, r2
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %xx = sext <16 x i8> %x to <16 x i64>
   %yy = sext <16 x i8> %y to <16 x i64>
   %m = mul <16 x i64> %xx, %yy
   %z = call i64 @llvm.vector.reduce.add.v16i64(<16 x i64> %m)
   %r = add i64 %z, %a
   ret i64 %r
 }

llvm/test/CodeGen/Thumb2/mve-vecreduce-mlapred.ll

entry:
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i32_v2i64_zext(<2 x i32> %x, <2 x i32> %y, <2 x i32> %b) {
 ; CHECK-LABEL: add_v2i32_v2i64_zext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s8
 ; CHECK-NEXT:    vmullb.u32 q3, q0, q1
+; CHECK-NEXT:    vmov r1, s10
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.32 q0[1], r0
-; CHECK-NEXT:    vmov r0, s10
-; CHECK-NEXT:    cmp r0, #0
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    vmov.32 q0[3], r0
+; CHECK-NEXT:    cmp r1, #0
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
 ; CHECK-NEXT:    vand q0, q3, q0
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp eq <2 x i32> %b, zeroinitializer
   %xx = zext <2 x i32> %x to <2 x i64>
   %yy = zext <2 x i32> %y to <2 x i64>
   %m = mul <2 x i64> %xx, %yy
   %s = select <2 x i1> %c, <2 x i64> %m, <2 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> %s)
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i32_v2i64_sext(<2 x i32> %x, <2 x i32> %y, <2 x i32> %b) {
 ; CHECK-LABEL: add_v2i32_v2i64_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r0, s8
 ; CHECK-NEXT:    vmullb.s32 q3, q0, q1
+; CHECK-NEXT:    vmov r1, s10
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.32 q0[1], r0
-; CHECK-NEXT:    vmov r0, s10
-; CHECK-NEXT:    cmp r0, #0
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    vmov.32 q0[3], r0
+; CHECK-NEXT:    cmp r1, #0
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
 ; CHECK-NEXT:    vand q0, q3, q0
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    bx lr
entry:
   %s = select <8 x i1> %c, <8 x i64> %ma, <8 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v8i64(<8 x i64> %s)
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i16_v2i64_zext(<2 x i16> %x, <2 x i16> %y, <2 x i16> %b) {
 ; CHECK-LABEL: add_v2i16_v2i64_zext:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .vsave {d8, d9}
-; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    vmov.i64 q3, #0xffff
 ; CHECK-NEXT:    vand q1, q1, q3
-; CHECK-NEXT:    vand q4, q0, q3
+; CHECK-NEXT:    vand q0, q0, q3
 ; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r1, s16
-; CHECK-NEXT:    umull r0, r1, r1, r0
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov r0, s6
-; CHECK-NEXT:    vmov.32 q0[1], r1
-; CHECK-NEXT:    vmov r1, s18
+; CHECK-NEXT:    vmov r1, s0
+; CHECK-NEXT:    vmov r2, s6
 ; CHECK-NEXT:    vand q1, q2, q3
+; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    umull r0, r1, r1, r0
-; CHECK-NEXT:    vmov.32 q0[2], r0
+; CHECK-NEXT:    umull r2, r3, r3, r2
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r0
 ; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov.32 q0[3], r1
-; CHECK-NEXT:    cmp r0, #0
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s6
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r1
+; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    cmp r1, #0
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
-; CHECK-NEXT:    vpop {d8, d9}
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp eq <2 x i16> %b, zeroinitializer
   %xx = zext <2 x i16> %x to <2 x i64>
   %yy = zext <2 x i16> %y to <2 x i64>
   %m = mul <2 x i64> %xx, %yy
   %s = select <2 x i1> %c, <2 x i64> %m, <2 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> %s)
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i16_v2i64_sext(<2 x i16> %x, <2 x i16> %y, <2 x i16> %b) {
 ; CHECK-LABEL: add_v2i16_v2i64_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.i32 q3, #0xffff
-; CHECK-NEXT:    vmov r1, s0
-; CHECK-NEXT:    vand q3, q2, q3
-; CHECK-NEXT:    vmov r0, s12
-; CHECK-NEXT:    sxth r1, r1
-; CHECK-NEXT:    cmp r0, #0
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s14
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    vand q2, q2, q3
+; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    vmov r0, s8
+; CHECK-NEXT:    vmov r1, s10
+; CHECK-NEXT:    sxth r2, r2
 ; CHECK-NEXT:    cmp r0, #0
+; CHECK-NEXT:    sxth r3, r3
 ; CHECK-NEXT:    cset r0, eq
+; CHECK-NEXT:    smull r2, r3, r3, r2
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    cmp r1, #0
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    sxth r0, r0
-; CHECK-NEXT:    smull r0, r1, r1, r0
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov r0, s6
-; CHECK-NEXT:    vmov.32 q3[1], r1
-; CHECK-NEXT:    vmov r1, s2
+; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    sxth r0, r0
 ; CHECK-NEXT:    sxth r1, r1
 ; CHECK-NEXT:    smull r0, r1, r1, r0
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r1
-; CHECK-NEXT:    vand q0, q3, q2
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r1
+; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    bx lr
 entry:
entry:
   %s = select <16 x i1> %c, <16 x i8> %m, <16 x i8> zeroinitializer
   %z = call i8 @llvm.vector.reduce.add.v16i8(<16 x i8> %s)
   ret i8 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v16i8_v16i64_zext(<16 x i8> %x, <16 x i8> %y, <16 x i8> %b) {
 ; CHECK-LABEL: add_v16i8_v16i64_zext:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r7, lr}
-; CHECK-NEXT:    push {r7, lr}
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    .pad #88
-; CHECK-NEXT:    sub sp, #88
-; CHECK-NEXT:    vmov q3, q1
-; CHECK-NEXT:    vstrw.32 q0, [sp, #64] @ 16-byte Spill
-; CHECK-NEXT:    vmov.i8 q0, #0x0
-; CHECK-NEXT:    vmov.i8 q1, #0xff
+; CHECK-NEXT:    .pad #40
+; CHECK-NEXT:    sub sp, #40
+; CHECK-NEXT:    vmov q4, q0
 ; CHECK-NEXT:    vcmp.i8 eq, q2, zr
-; CHECK-NEXT:    vstrw.32 q1, [sp, #16] @ 16-byte Spill
-; CHECK-NEXT:    vpsel q5, q1, q0
-; CHECK-NEXT:    vstrw.32 q0, [sp, #32] @ 16-byte Spill
+; CHECK-NEXT:    vmov.i8 q2, #0xff
+; CHECK-NEXT:    vmov.i8 q0, #0x0
+; CHECK-NEXT:    vpsel q5, q2, q0
+; CHECK-NEXT:    vmov q3, q2
 ; CHECK-NEXT:    vmov.u8 r0, q5[0]
-; CHECK-NEXT:    vmov.i64 q4, #0xff
+; CHECK-NEXT:    vstrw.32 q2, [sp] @ 16-byte Spill
 ; CHECK-NEXT:    vmov.16 q2[0], r0
 ; CHECK-NEXT:    vmov.u8 r0, q5[1]
 ; CHECK-NEXT:    vmov.16 q2[1], r0
 ; CHECK-NEXT:    vmov.u8 r0, q5[2]
 ; CHECK-NEXT:    vmov.16 q2[2], r0
 ; CHECK-NEXT:    vmov.u8 r0, q5[3]
 ; CHECK-NEXT:    vmov.16 q2[3], r0
 ; CHECK-NEXT:    vmov.u8 r0, q5[4]
 ; CHECK-NEXT:    vmov.16 q2[4], r0
 ; CHECK-NEXT:    vmov.u8 r0, q5[5]
 ; CHECK-NEXT:    vmov.16 q2[5], r0
 ; CHECK-NEXT:    vmov.u8 r0, q5[6]
 ; CHECK-NEXT:    vmov.16 q2[6], r0
 ; CHECK-NEXT:    vmov.u8 r0, q5[7]
 ; CHECK-NEXT:    vmov.16 q2[7], r0
+; CHECK-NEXT:    vstrw.32 q0, [sp, #16] @ 16-byte Spill
 ; CHECK-NEXT:    vcmp.i16 ne, q2, zr
-; CHECK-NEXT:    vpsel q6, q1, q0
+; CHECK-NEXT:    vmov.u8 r3, q4[1]
+; CHECK-NEXT:    vpsel q6, q3, q0
 ; CHECK-NEXT:    vmov.u16 r0, q6[0]
-; CHECK-NEXT:    vmov.32 q2[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q6[2]
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q6[1]
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q6[2]
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q6[3]
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q6[3]
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vcmp.i32 ne, q2, zr
+; CHECK-NEXT:    vmov.i64 q2, #0xff
 ; CHECK-NEXT:    vmrs r0, p0
-; CHECK-NEXT:    vldrw.u32 q1, [sp, #64] @ 16-byte Reload
-; CHECK-NEXT:    vmov.u8 r2, q1[0]
 ; CHECK-NEXT:    and r1, r0, #1
+; CHECK-NEXT:    ubfx r2, r0, #4, #1
 ; CHECK-NEXT:    rsbs r1, r1, #0
-; CHECK-NEXT:    vmov.32 q7[0], r1
-; CHECK-NEXT:    vmov.32 q7[1], r1
-; CHECK-NEXT:    ubfx r1, r0, #4, #1
-; CHECK-NEXT:    rsbs r1, r1, #0
-; CHECK-NEXT:    vmov.32 q7[2], r1
-; CHECK-NEXT:    vmov.32 q7[3], r1
-; CHECK-NEXT:    vmov.u8 r1, q3[0]
-; CHECK-NEXT:    vmov.32 q0[0], r1
-; CHECK-NEXT:    vmov.u8 r1, q3[1]
-; CHECK-NEXT:    vmov.32 q0[2], r1
-; CHECK-NEXT:    vand q2, q0, q4
-; CHECK-NEXT:    vmov.32 q0[0], r2
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov q7[2], q7[0], r2, r1
+; CHECK-NEXT:    vmov q7[3], q7[1], r2, r1
+; CHECK-NEXT:    vmov.u8 r1, q1[0]
 ; CHECK-NEXT:    vmov.u8 r2, q1[1]
-; CHECK-NEXT:    vmov r1, s8
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vand q1, q0, q4
-; CHECK-NEXT:    vmov r2, s4
-; CHECK-NEXT:    umull r1, r2, r2, r1
-; CHECK-NEXT:    vmov.32 q0[0], r1
-; CHECK-NEXT:    vmov r1, s10
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    vldrw.u32 q2, [sp, #64] @ 16-byte Reload
-; CHECK-NEXT:    umull r1, r2, r2, r1
-; CHECK-NEXT:    vmov.32 q0[2], r1
-; CHECK-NEXT:    vmov.32 q0[3], r2
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r1
+; CHECK-NEXT:    vmov.u8 r2, q4[0]
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r2
+; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    vand q3, q3, q2
+; CHECK-NEXT:    vmov r1, s0
+; CHECK-NEXT:    vmov r2, s12
+; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    umull r1, r12, r2, r1
+; CHECK-NEXT:    vmov r2, s14
+; CHECK-NEXT:    umull r2, r3, r2, r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r1
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r12
 ; CHECK-NEXT:    vand q0, q0, q7
 ; CHECK-NEXT:    vmov r1, s2
 ; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    vmov r12, s3
 ; CHECK-NEXT:    vmov r3, s1
-; CHECK-NEXT:    adds r1, r1, r2
-; CHECK-NEXT:    adc.w r2, r3, r12
+; CHECK-NEXT:    adds.w lr, r2, r1
+; CHECK-NEXT:    vmov.u8 r1, q4[3]
+; CHECK-NEXT:    adc.w r12, r12, r3
 ; CHECK-NEXT:    ubfx r3, r0, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    ubfx r0, r0, #12, #1
-; CHECK-NEXT:    vmov.32 q7[0], r3
+; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.32 q7[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q2[2]
-; CHECK-NEXT:    vmov.32 q7[2], r0
-; CHECK-NEXT:    vmov.32 q1[0], r3
-; CHECK-NEXT:    vmov.32 q7[3], r0
-; CHECK-NEXT:    vmov.u8 r0, q3[2]
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q3[3]
-; CHECK-NEXT:    vmov.u8 r3, q2[3]
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    vmov.32 q1[2], r3
-; CHECK-NEXT:    vand q0, q0, q4
-; CHECK-NEXT:    vand q1, q1, q4
+; CHECK-NEXT:    vmov q7[2], q7[0], r0, r3
+; CHECK-NEXT:    vmov q7[3], q7[1], r0, r3
+; CHECK-NEXT:    vmov.u8 r0, q1[2]
+; CHECK-NEXT:    vmov.u8 r3, q1[3]
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r0
+; CHECK-NEXT:    vmov.u8 r3, q4[2]
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r3
+; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    vand q3, q3, q2
 ; CHECK-NEXT:    vmov r0, s0
-; CHECK-NEXT:    vmov r3, s4
-; CHECK-NEXT:    umull r0, r3, r3, r0
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    umull r0, r3, r3, r0
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    vand q0, q2, q7
-; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    vmov r1, s12
+; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    vmov r2, s14
+; CHECK-NEXT:    umull r0, r1, r1, r0
+; CHECK-NEXT:    umull r2, r3, r2, r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r1
+; CHECK-NEXT:    vand q0, q0, q7
+; CHECK-NEXT:    vmov q7, q4
+; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    vmov r0, s1
-; CHECK-NEXT:    adds r1, r1, r3
 ; CHECK-NEXT:    vmov r3, s3
-; CHECK-NEXT:    adcs r2, r0
+; CHECK-NEXT:    adds.w r1, r1, lr
+; CHECK-NEXT:    adc.w r2, r12, r0
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    adds.w r12, r1, r0
-; CHECK-NEXT:    adc.w r1, r2, r3
+; CHECK-NEXT:    vmov.u8 r1, q4[5]
+; CHECK-NEXT:    adc.w lr, r2, r3
 ; CHECK-NEXT:    vmov.u16 r2, q6[4]
-; CHECK-NEXT:    vmov.32 q0[0], r2
+; CHECK-NEXT:    vmov.u16 r3, q6[6]
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q6[5]
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q6[6]
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q6[7]
-; CHECK-NEXT:    vmov.32 q0[3], r2
+; CHECK-NEXT:    vmov.u16 r3, q6[7]
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q0, zr
-; CHECK-NEXT:    vmrs lr, p0
-; CHECK-NEXT:    vstrw.32 q3, [sp, #48] @ 16-byte Spill
-; CHECK-NEXT:    vstrw.32 q4, [sp] @ 16-byte Spill
-; CHECK-NEXT:    and r3, lr, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q6[0], r3
-; CHECK-NEXT:    vmov.32 q6[1], r3
-; CHECK-NEXT:    ubfx r3, lr, #4, #1
+; CHECK-NEXT:    vmrs r2, p0
+; CHECK-NEXT:    and r3, r2, #1
+; CHECK-NEXT:    ubfx r0, r2, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q6[2], r3
-; CHECK-NEXT:    vmov.32 q6[3], r3
-; CHECK-NEXT:    vmov.u8 r3, q3[4]
-; CHECK-NEXT:    vmov.32 q0[0], r3
-; CHECK-NEXT:    vmov.u8 r3, q3[5]
-; CHECK-NEXT:    vldrw.u32 q3, [sp, #64] @ 16-byte Reload
-; CHECK-NEXT:    vmov.32 q0[2], r3
-; CHECK-NEXT:    vand q0, q0, q4
-; CHECK-NEXT:    vmov.u8 r0, q3[4]
-; CHECK-NEXT:    vmov r3, s0
-; CHECK-NEXT:    vmov.32 q1[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q3[5]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov q7, q3
-; CHECK-NEXT:    vand q1, q1, q4
-; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    umull r0, r3, r0, r3
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    vldrw.u32 q1, [sp, #48] @ 16-byte Reload
-; CHECK-NEXT:    umull r0, r3, r3, r0
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    vand q0, q2, q6
-; CHECK-NEXT:    vmov r3, s0
-; CHECK-NEXT:    vmov r0, s1
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    adds.w r3, r3, r12
-; CHECK-NEXT:    adcs r1, r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    adds r3, r3, r2
-; CHECK-NEXT:    vmov.u8 r2, q3[6]
-; CHECK-NEXT:    adcs r1, r0
-; CHECK-NEXT:    ubfx r0, lr, #8, #1
 ; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.32 q6[0], r0
-; CHECK-NEXT:    vmov.32 q6[1], r0
-; CHECK-NEXT:    ubfx r0, lr, #12, #1
-; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.32 q6[2], r0
-; CHECK-NEXT:    vmov.32 q6[3], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[6]
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[7]
-; CHECK-NEXT:    vmov.32 q1[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q3[7]
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vand q0, q0, q4
-; CHECK-NEXT:    vand q1, q1, q4
+; CHECK-NEXT:    vmov q6[2], q6[0], r0, r3
+; CHECK-NEXT:    vmov q6[3], q6[1], r0, r3
+; CHECK-NEXT:    vmov.u8 r0, q1[4]
+; CHECK-NEXT:    vmov.u8 r3, q1[5]
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r0
+; CHECK-NEXT:    vmov.u8 r3, q4[4]
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r3
+; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    vand q3, q3, q2
 ; CHECK-NEXT:    vmov r0, s0
-; CHECK-NEXT:    vmov r2, s4
-; CHECK-NEXT:    umull r0, r2, r2, r0
-; CHECK-NEXT:    vmov.32 q2[0], r0
+; CHECK-NEXT:    vmov r1, s12
+; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    vmov r4, s14
+; CHECK-NEXT:    umull r0, r1, r1, r0
+; CHECK-NEXT:    umull r3, r4, r4, r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r1
+; CHECK-NEXT:    vand q0, q0, q6
+; CHECK-NEXT:    vmov r1, s0
+; CHECK-NEXT:    vmov r0, s1
+; CHECK-NEXT:    vmov r4, s3
+; CHECK-NEXT:    adds.w r1, r1, r12
+; CHECK-NEXT:    adc.w r3, lr, r0
 ; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    vldrw.u32 q1, [sp, #16] @ 16-byte Reload
-; CHECK-NEXT:    umull r0, r2, r2, r0
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r2
-; CHECK-NEXT:    vand q0, q2, q6
+; CHECK-NEXT:    adds.w r12, r1, r0
+; CHECK-NEXT:    adc.w r1, r3, r4
+; CHECK-NEXT:    ubfx r3, r2, #8, #1
+; CHECK-NEXT:    ubfx r2, r2, #12, #1
+; CHECK-NEXT:    rsbs r3, r3, #0
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov.u8 r4, q4[7]
+; CHECK-NEXT:    vmov q6[2], q6[0], r2, r3
+; CHECK-NEXT:    vmov q6[3], q6[1], r2, r3
+; CHECK-NEXT:    vmov.u8 r2, q1[6]
+; CHECK-NEXT:    vmov.u8 r3, q1[7]
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
+; CHECK-NEXT:    vmov.u8 r3, q4[6]
+; CHECK-NEXT:    vmov q3[2], q3[0], r4, r3
+; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    vand q3, q3, q2
+; CHECK-NEXT:    vmov r2, s0
+; CHECK-NEXT:    vmov r3, s12
+; CHECK-NEXT:    vmov r0, s14
+; CHECK-NEXT:    vldrw.u32 q3, [sp] @ 16-byte Reload
+; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    umull r2, r3, r3, r2
+; CHECK-NEXT:    umull r0, r4, r0, r4
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r3
+; CHECK-NEXT:    vand q0, q0, q6
 ; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    vmov r0, s1
-; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    vmov r3, s3
+; CHECK-NEXT:    adds.w r2, r2, r12
 ; CHECK-NEXT:    adcs r1, r0
 ; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vldrw.u32 q0, [sp, #32] @ 16-byte Reload
+; CHECK-NEXT:    vldrw.u32 q0, [sp, #16] @ 16-byte Reload
 ; CHECK-NEXT:    adds.w r12, r2, r0
 ; CHECK-NEXT:    vmov.u8 r2, q5[8]
 ; CHECK-NEXT:    vmov.16 q6[0], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[9]
 ; CHECK-NEXT:    vmov.16 q6[1], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[10]
 ; CHECK-NEXT:    vmov.16 q6[2], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[11]
 ; CHECK-NEXT:    vmov.16 q6[3], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[12]
 ; CHECK-NEXT:    vmov.16 q6[4], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[13]
 ; CHECK-NEXT:    vmov.16 q6[5], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[14]
 ; CHECK-NEXT:    vmov.16 q6[6], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[15]
 ; CHECK-NEXT:    vmov.16 q6[7], r2
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    adc.w lr, r1, r3
 ; CHECK-NEXT:    vcmp.i16 ne, q6, zr
-; CHECK-NEXT:    vmov.u8 r0, q7[8]
-; CHECK-NEXT:    vpsel q3, q1, q0
-; CHECK-NEXT:    vmov.32 q1[0], r0
-; CHECK-NEXT:    vmov.u16 r2, q3[0]
 ; CHECK-NEXT:    vmov.u8 r0, q7[9]
-; CHECK-NEXT:    vmov.32 q0[0], r2
+; CHECK-NEXT:    vpsel q3, q3, q0
+; CHECK-NEXT:    vmov.u16 r2, q3[0]
+; CHECK-NEXT:    vmov.u16 r3, q3[2]
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q3[1]
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q3[2]
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q3[3]
-; CHECK-NEXT:    vmov.32 q0[3], r2
-; CHECK-NEXT:    vmov.32 q1[2], r0
+; CHECK-NEXT:    vmov.u16 r3, q3[3]
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q0, zr
-; CHECK-NEXT:    vmrs lr, p0
-; CHECK-NEXT:    vldrw.u32 q6, [sp, #48] @ 16-byte Reload
-; CHECK-NEXT:    vldrw.u32 q5, [sp] @ 16-byte Reload
-; CHECK-NEXT:    vand q1, q1, q5
-; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    and r3, lr, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q4[0], r3
-; CHECK-NEXT:    vmov.32 q4[1], r3
-; CHECK-NEXT:    ubfx r3, lr, #4, #1
+; CHECK-NEXT:    vmrs r2, p0
+; CHECK-NEXT:    and r3, r2, #1
+; CHECK-NEXT:    ubfx r4, r2, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q4[2], r3
-; CHECK-NEXT:    vmov.32 q4[3], r3
-; CHECK-NEXT:    vmov.u8 r3, q6[8]
-; CHECK-NEXT:    vmov.32 q0[0], r3
-; CHECK-NEXT:    vmov.u8 r3, q6[9]
-; CHECK-NEXT:    vmov.32 q0[2], r3
-; CHECK-NEXT:    vand q0, q0, q5
+; CHECK-NEXT:    rsbs r4, r4, #0
+; CHECK-NEXT:    vmov q4[2], q4[0], r4, r3
+; CHECK-NEXT:    vmov q4[3], q4[1], r4, r3
+; CHECK-NEXT:    vmov.u8 r3, q1[8]
+; CHECK-NEXT:    vmov.u8 r4, q1[9]
+; CHECK-NEXT:    vmov q0[2], q0[0], r4, r3
+; CHECK-NEXT:    vmov.u8 r4, q7[8]
+; CHECK-NEXT:    vmov q5[2], q5[0], r0, r4
+; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    vand q5, q5, q2
 ; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    vmov r0, s20
+; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    vmov r1, s22
 ; CHECK-NEXT:    umull r0, r3, r0, r3
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    umull r0, r3, r3, r0
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    vand q0, q2, q4
-; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    umull r1, r4, r1, r4
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r3
+; CHECK-NEXT:    vand q0, q0, q4
+; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    vmov r0, s1
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    adds.w r3, r3, r12
-; CHECK-NEXT:    adcs r1, r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    adds r3, r3, r2
-; CHECK-NEXT:    vmov.u8 r2, q7[10]
-; CHECK-NEXT:    vmov.32 q1[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q7[11]
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vand q1, q1, q5
-; CHECK-NEXT:    vmov r2, s4
-; CHECK-NEXT:    adcs r1, r0
-; CHECK-NEXT:    ubfx r0, lr, #8, #1
-; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.32 q4[0], r0
-; CHECK-NEXT:    vmov.32 q4[1], r0
-; CHECK-NEXT:    ubfx r0, lr, #12, #1
-; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.32 q4[2], r0
-; CHECK-NEXT:    vmov.32 q4[3], r0
-; CHECK-NEXT:    vmov.u8 r0, q6[10]
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q6[11]
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    vand q0, q0, q5
-; CHECK-NEXT:    vmov r0, s0
-; CHECK-NEXT:    umull r0, r2, r2, r0
-; CHECK-NEXT:    vmov.32 q2[0], r0
+; CHECK-NEXT:    vmov r4, s3
+; CHECK-NEXT:    adds.w r1, r1, r12
+; CHECK-NEXT:    adc.w r3, lr, r0
 ; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    umull r0, r2, r2, r0
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r2
-; CHECK-NEXT:    vand q0, q2, q4
+; CHECK-NEXT:    adds.w r12, r1, r0
+; CHECK-NEXT:    adc.w r1, r3, r4
+; CHECK-NEXT:    ubfx r3, r2, #8, #1
+; CHECK-NEXT:    ubfx r2, r2, #12, #1
+; CHECK-NEXT:    rsbs r3, r3, #0
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov.u8 r4, q7[11]
+; CHECK-NEXT:    vmov q4[2], q4[0], r2, r3
+; CHECK-NEXT:    vmov q4[3], q4[1], r2, r3
+; CHECK-NEXT:    vmov.u8 r2, q1[10]
+; CHECK-NEXT:    vmov.u8 r3, q1[11]
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
+; CHECK-NEXT:    vmov.u8 r3, q7[10]
+; CHECK-NEXT:    vmov q5[2], q5[0], r4, r3
+; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    vand q5, q5, q2
+; CHECK-NEXT:    vmov r2, s0
+; CHECK-NEXT:    vmov r3, s20
+; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    vmov r0, s22
+; CHECK-NEXT:    umull r2, r3, r3, r2
+; CHECK-NEXT:    umull r0, r4, r0, r4
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r3
+; CHECK-NEXT:    vand q0, q0, q4
 ; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    vmov r0, s1
-; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    vmov r3, s3
+; CHECK-NEXT:    adds.w r2, r2, r12
 ; CHECK-NEXT:    adcs r1, r0
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    adds.w r12, r2, r0
 ; CHECK-NEXT:    vmov.u16 r2, q3[4]
-; CHECK-NEXT:    vmov.32 q0[0], r2
+; CHECK-NEXT:    adc.w lr, r1, r3
+; CHECK-NEXT:    vmov.u16 r3, q3[6]
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q3[5]
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q3[6]
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q3[7]
-; CHECK-NEXT:    vmov.32 q0[3], r2
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vcmp.i32 ne, q0, zr
-; CHECK-NEXT:    vmov.u8 r0, q7[12]
-; CHECK-NEXT:    vmrs lr, p0
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov.u16 r3, q3[7]
 ; CHECK-NEXT:    vmov.u8 r0, q7[13]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vand q1, q1, q5
-; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    and r3, lr, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q3[0], r3
-; CHECK-NEXT:    vmov.32 q3[1], r3
-; CHECK-NEXT:    ubfx r3, lr, #4, #1
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r2
+; CHECK-NEXT:    vcmp.i32 ne, q0, zr
+; CHECK-NEXT:    vmrs r2, p0
+; CHECK-NEXT:    and r3, r2, #1
+; CHECK-NEXT:    ubfx r4, r2, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q3[2], r3
-; CHECK-NEXT:    vmov.32 q3[3], r3
-; CHECK-NEXT:    vmov.u8 r3, q6[12]
-; CHECK-NEXT:    vmov.32 q0[0], r3
-; CHECK-NEXT:    vmov.u8 r3, q6[13]
-; CHECK-NEXT:    vmov.32 q0[2], r3
-; CHECK-NEXT:    vand q0, q0, q5
+; CHECK-NEXT:    rsbs r4, r4, #0
+; CHECK-NEXT:    vmov q3[2], q3[0], r4, r3
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r3
+; CHECK-NEXT:    vmov.u8 r3, q1[12]
+; CHECK-NEXT:    vmov.u8 r4, q1[13]
+; CHECK-NEXT:    vmov q0[2], q0[0], r4, r3
+; CHECK-NEXT:    vmov.u8 r4, q7[12]
+; CHECK-NEXT:    vmov q4[2], q4[0], r0, r4
+; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    vand q4, q4, q2
 ; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    vmov r0, s16
+; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    vmov r1, s18
 ; CHECK-NEXT:    umull r0, r3, r0, r3
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    umull r0, r3, r3, r0
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    vand q0, q2, q3
-; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    umull r1, r4, r1, r4
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r3
+; CHECK-NEXT:    vand q0, q0, q3
+; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    vmov r0, s1
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    adds.w r3, r3, r12
-; CHECK-NEXT:    adcs r1, r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    adds r3, r3, r2
-; CHECK-NEXT:    vmov.u8 r2, q7[14]
-; CHECK-NEXT:    vmov.32 q1[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q7[15]
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vand q1, q1, q5
-; CHECK-NEXT:    vmov r2, s4
-; CHECK-NEXT:    adcs r1, r0
-; CHECK-NEXT:    ubfx r0, lr, #8, #1
-; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    ubfx r0, lr, #12, #1
-; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vmov.u8 r0, q6[14]
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q6[15]
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    vand q0, q0, q5
-; CHECK-NEXT:    vmov r0, s0
-; CHECK-NEXT:    umull r0, r2, r2, r0
-; CHECK-NEXT:    vmov.32 q2[0], r0
+; CHECK-NEXT:    vmov r4, s3
+; CHECK-NEXT:    adds.w r1, r1, r12
+; CHECK-NEXT:    adc.w r3, lr, r0
 ; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    umull r0, r2, r2, r0
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r2
-; CHECK-NEXT:    vand q0, q2, q3
+; CHECK-NEXT:    adds.w r12, r1, r0
+; CHECK-NEXT:    adc.w r1, r3, r4
+; CHECK-NEXT:    ubfx r3, r2, #8, #1
+; CHECK-NEXT:    ubfx r2, r2, #12, #1
+; CHECK-NEXT:    rsbs r3, r3, #0
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov.u8 r4, q7[15]
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r3
+; CHECK-NEXT:    vmov q3[3], q3[1], r2, r3
+; CHECK-NEXT:    vmov.u8 r2, q1[14]
+; CHECK-NEXT:    vmov.u8 r3, q1[15]
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
+; CHECK-NEXT:    vmov.u8 r3, q7[14]
+; CHECK-NEXT:    vmov q1[2], q1[0], r4, r3
+; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    vand q1, q1, q2
+; CHECK-NEXT:    vmov r2, s0
+; CHECK-NEXT:    vmov r3, s4
+; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    vmov r0, s6
+; CHECK-NEXT:    umull r2, r3, r3, r2
+; CHECK-NEXT:    umull r0, r4, r0, r4
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r3
+; CHECK-NEXT:    vand q0, q0, q3
 ; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    vmov r0, s1
-; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    vmov r3, s3
+; CHECK-NEXT:    adds.w r2, r2, r12
 ; CHECK-NEXT:    adcs r1, r0
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    adds r0, r0, r2
 ; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    add sp, #88
+; CHECK-NEXT:    add sp, #40
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    pop {r7, pc}
+; CHECK-NEXT:    pop {r4, pc}
 entry:
   %c = icmp eq <16 x i8> %b, zeroinitializer
   %xx = zext <16 x i8> %x to <16 x i64>
   %yy = zext <16 x i8> %y to <16 x i64>
   %m = mul <16 x i64> %xx, %yy
   %s = select <16 x i1> %c, <16 x i64> %m, <16 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v16i64(<16 x i64> %s)
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v16i8_v16i64_sext(<16 x i8> %x, <16 x i8> %y, <16 x i8> %b) {
 ; CHECK-LABEL: add_v16i8_v16i64_sext:
 ; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vcmp.i8 eq, q2, zr
 ; CHECK-NEXT:    vmov.i8 q2, #0x0
 ; CHECK-NEXT:    vmov.i8 q3, #0xff
-; CHECK-NEXT:    vmov.u8 r2, q0[0]
+; CHECK-NEXT:    vmov.u8 r3, q1[1]
 ; CHECK-NEXT:    vpsel q4, q3, q2
-; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    vmov.u8 r0, q4[0]
+; CHECK-NEXT:    vmov.u8 r4, q0[5]
 ; CHECK-NEXT:    vmov.16 q5[0], r0
 ; CHECK-NEXT:    vmov.u8 r0, q4[1]
 ; CHECK-NEXT:    vmov.16 q5[1], r0
 ; CHECK-NEXT:    vmov.u8 r0, q4[2]
 ; CHECK-NEXT:    vmov.16 q5[2], r0
 ; CHECK-NEXT:    vmov.u8 r0, q4[3]
 ; CHECK-NEXT:    vmov.16 q5[3], r0
 ; CHECK-NEXT:    vmov.u8 r0, q4[4]
 ; CHECK-NEXT:    vmov.16 q5[4], r0
 ; CHECK-NEXT:    vmov.u8 r0, q4[5]
 ; CHECK-NEXT:    vmov.16 q5[5], r0
 ; CHECK-NEXT:    vmov.u8 r0, q4[6]
 ; CHECK-NEXT:    vmov.16 q5[6], r0
 ; CHECK-NEXT:    vmov.u8 r0, q4[7]
 ; CHECK-NEXT:    vmov.16 q5[7], r0
+; CHECK-NEXT:    sxtb r4, r4
 ; CHECK-NEXT:    vcmp.i16 ne, q5, zr
 ; CHECK-NEXT:    vpsel q5, q3, q2
 ; CHECK-NEXT:    vmov.u16 r0, q5[0]
-; CHECK-NEXT:    vmov.32 q6[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q5[2]
+; CHECK-NEXT:    vmov q6[2], q6[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q5[1]
-; CHECK-NEXT:    vmov.32 q6[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q5[2]
-; CHECK-NEXT:    vmov.32 q6[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q5[3]
-; CHECK-NEXT:    vmov.32 q6[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q5[3]
+; CHECK-NEXT:    vmov q6[3], q6[1], r1, r0
 ; CHECK-NEXT:    vcmp.i32 ne, q6, zr
 ; CHECK-NEXT:    vmrs r0, p0
 ; CHECK-NEXT:    and r1, r0, #1
+; CHECK-NEXT:    ubfx r2, r0, #4, #1
 ; CHECK-NEXT:    rsbs r1, r1, #0
-; CHECK-NEXT:    vmov.32 q6[0], r1
-; CHECK-NEXT:    vmov.32 q6[1], r1
-; CHECK-NEXT:    ubfx r1, r0, #4, #1
-; CHECK-NEXT:    rsbs r1, r1, #0
-; CHECK-NEXT:    vmov.32 q6[2], r1
-; CHECK-NEXT:    vmov.32 q6[3], r1
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov q6[2], q6[0], r2, r1
+; CHECK-NEXT:    vmov q6[3], q6[1], r2, r1
 ; CHECK-NEXT:    vmov.u8 r1, q1[0]
+; CHECK-NEXT:    vmov.u8 r2, q0[0]
 ; CHECK-NEXT:    sxtb r1, r1
-; CHECK-NEXT:    smull r1, r2, r2, r1
-; CHECK-NEXT:    vmov.32 q7[0], r1
-; CHECK-NEXT:    vmov.u8 r1, q1[1]
-; CHECK-NEXT:    vmov.32 q7[1], r2
+; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    smull r1, r12, r2, r1
 ; CHECK-NEXT:    vmov.u8 r2, q0[1]
-; CHECK-NEXT:    sxtb r1, r1
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    smull r1, r2, r2, r1
-; CHECK-NEXT:    vmov.32 q7[2], r1
-; CHECK-NEXT:    vmov.32 q7[3], r2
+; CHECK-NEXT:    smull r2, r3, r2, r3
+; CHECK-NEXT:    vmov q7[2], q7[0], r2, r1
+; CHECK-NEXT:    vmov q7[3], q7[1], r3, r12
 ; CHECK-NEXT:    vand q6, q7, q6
-; CHECK-NEXT:    vmov r1, s26
-; CHECK-NEXT:    vmov r2, s24
+; CHECK-NEXT:    vmov r3, s26
+; CHECK-NEXT:    vmov r1, s24
 ; CHECK-NEXT:    vmov r12, s27
-; CHECK-NEXT:    vmov r3, s25
-; CHECK-NEXT:    adds r1, r1, r2
-; CHECK-NEXT:    adc.w r2, r3, r12
+; CHECK-NEXT:    vmov r2, s25
+; CHECK-NEXT:    adds.w lr, r1, r3
 ; CHECK-NEXT:    ubfx r3, r0, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
 ; CHECK-NEXT:    ubfx r0, r0, #12, #1
-; CHECK-NEXT:    vmov.32 q6[0], r3
-; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.32 q6[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[2]
-; CHECK-NEXT:    vmov.32 q6[2], r0
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    vmov.32 q6[3], r0
+; CHECK-NEXT:    rsb.w r3, r3, #0
+; CHECK-NEXT:    rsb.w r0, r0, #0
+; CHECK-NEXT:    vmov.u8 r1, q1[3]
+; CHECK-NEXT:    vmov q6[2], q6[0], r0, r3
+; CHECK-NEXT:    adc.w r12, r12, r2
+; CHECK-NEXT:    vmov q6[3], q6[1], r0, r3
 ; CHECK-NEXT:    vmov.u8 r0, q1[2]
-; CHECK-NEXT:    sxtb r0, r0
-; CHECK-NEXT:    smull r0, r3, r3, r0
-; CHECK-NEXT:    vmov.32 q7[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[3]
-; CHECK-NEXT:    vmov.32 q7[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[3]
+; CHECK-NEXT:    vmov.u8 r3, q0[2]
+; CHECK-NEXT:    vmov.u8 r2, q0[3]
 ; CHECK-NEXT:    sxtb r0, r0
 ; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    sxtb r1, r1
+; CHECK-NEXT:    sxtb r2, r2
 ; CHECK-NEXT:    smull r0, r3, r3, r0
-; CHECK-NEXT:    vmov.32 q7[2], r0
-; CHECK-NEXT:    vmov.32 q7[3], r3
+; CHECK-NEXT:    smull r1, r2, r2, r1
+; CHECK-NEXT:    vmov q7[2], q7[0], r1, r0
+; CHECK-NEXT:    vmov q7[3], q7[1], r2, r3
 ; CHECK-NEXT:    vand q6, q7, q6
-; CHECK-NEXT:    vmov r3, s24
+; CHECK-NEXT:    vmov r1, s24
 ; CHECK-NEXT:    vmov r0, s25
-; CHECK-NEXT:    adds r1, r1, r3
 ; CHECK-NEXT:    vmov r3, s27
-; CHECK-NEXT:    adcs r2, r0
+; CHECK-NEXT:    adds.w r1, r1, lr
+; CHECK-NEXT:    adc.w r2, r12, r0
 ; CHECK-NEXT:    vmov r0, s26
 ; CHECK-NEXT:    adds.w r12, r1, r0
-; CHECK-NEXT:    vmov.u8 r0, q0[4]
-; CHECK-NEXT:    adc.w r1, r2, r3
+; CHECK-NEXT:    vmov.u8 r1, q1[5]
+; CHECK-NEXT:    adc.w lr, r2, r3
 ; CHECK-NEXT:    vmov.u16 r2, q5[4]
-; CHECK-NEXT:    vmov.32 q6[0], r2
+; CHECK-NEXT:    vmov.u16 r3, q5[6]
+; CHECK-NEXT:    sxtb r1, r1
+; CHECK-NEXT:    vmov q6[2], q6[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q5[5]
-; CHECK-NEXT:    vmov.32 q6[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q5[6]
-; CHECK-NEXT:    vmov.32 q6[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q5[7]
-; CHECK-NEXT:    vmov.32 q6[3], r2
-; CHECK-NEXT:    sxtb r0, r0
+; CHECK-NEXT:    vmov.u16 r3, q5[7]
+; CHECK-NEXT:    smull r1, r4, r4, r1
+; CHECK-NEXT:    vmov q6[3], q6[1], r3, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q6, zr
 ; CHECK-NEXT:    vmrs r2, p0
 ; CHECK-NEXT:    and r3, r2, #1
+; CHECK-NEXT:    ubfx r0, r2, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q5[0], r3
-; CHECK-NEXT:    vmov.32 q5[1], r3
-; CHECK-NEXT:    ubfx r3, r2, #4, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q5[2], r3
-; CHECK-NEXT:    vmov.32 q5[3], r3
-; CHECK-NEXT:    vmov.u8 r3, q1[4]
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r0, r3, r0, r3
-; CHECK-NEXT:    vmov.32 q6[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[5]
-; CHECK-NEXT:    vmov.32 q6[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[5]
+; CHECK-NEXT:    rsbs r0, r0, #0
+; CHECK-NEXT:    vmov q5[2], q5[0], r0, r3
+; CHECK-NEXT:    vmov q5[3], q5[1], r0, r3
+; CHECK-NEXT:    vmov.u8 r0, q1[4]
+; CHECK-NEXT:    vmov.u8 r3, q0[4]
 ; CHECK-NEXT:    sxtb r0, r0
 ; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    smull r0, r3, r3, r0
-; CHECK-NEXT:    vmov.32 q6[2], r0
-; CHECK-NEXT:    vmov.32 q6[3], r3
+; CHECK-NEXT:    vmov q6[2], q6[0], r1, r0
+; CHECK-NEXT:    vmov q6[3], q6[1], r4, r3
 ; CHECK-NEXT:    vand q5, q6, q5
-; CHECK-NEXT:    vmov r3, s20
+; CHECK-NEXT:    vmov r1, s20
 ; CHECK-NEXT:    vmov r0, s21
-; CHECK-NEXT:    adds.w r3, r3, r12
-; CHECK-NEXT:    adc.w r12, r1, r0
-; CHECK-NEXT:    vmov r1, s22
-; CHECK-NEXT:    vmov r0, s23
-; CHECK-NEXT:    adds r3, r3, r1
-; CHECK-NEXT:    adc.w r1, r12, r0
-; CHECK-NEXT:    ubfx r0, r2, #8, #1
-; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.32 q5[0], r0
-; CHECK-NEXT:    vmov.32 q5[1], r0
-; CHECK-NEXT:    ubfx r0, r2, #12, #1
-; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.u8 r2, q0[6]
-; CHECK-NEXT:    vmov.32 q5[2], r0
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q5[3], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[6]
-; CHECK-NEXT:    sxtb r0, r0
-; CHECK-NEXT:    smull r0, r2, r2, r0
-; CHECK-NEXT:    vmov.32 q6[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[7]
-; CHECK-NEXT:    vmov.32 q6[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[7]
+; CHECK-NEXT:    vmov r4, s22
+; CHECK-NEXT:    vmov r3, s23
+; CHECK-NEXT:    adds.w r1, r1, r12
+; CHECK-NEXT:    adc.w r0, r0, lr
+; CHECK-NEXT:    adds r1, r1, r4
+; CHECK-NEXT:    vmov.u8 r4, q1[7]
+; CHECK-NEXT:    adc.w r12, r0, r3
+; CHECK-NEXT:    ubfx r3, r2, #8, #1
+; CHECK-NEXT:    ubfx r2, r2, #12, #1
+; CHECK-NEXT:    rsbs r3, r3, #0
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov.u8 r0, q0[7]
+; CHECK-NEXT:    vmov q5[2], q5[0], r2, r3
+; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    vmov q5[3], q5[1], r2, r3
+; CHECK-NEXT:    vmov.u8 r2, q1[6]
+; CHECK-NEXT:    vmov.u8 r3, q0[6]
 ; CHECK-NEXT:    sxtb r0, r0
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    smull r0, r2, r2, r0
-; CHECK-NEXT:    vmov.32 q6[2], r0
-; CHECK-NEXT:    vmov.32 q6[3], r2
+; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    smull r2, r3, r3, r2
+; CHECK-NEXT:    smull r0, r4, r0, r4
+; CHECK-NEXT:    vmov q6[2], q6[0], r0, r2
+; CHECK-NEXT:    vmov q6[3], q6[1], r4, r3
 ; CHECK-NEXT:    vand q5, q6, q5
 ; CHECK-NEXT:    vmov r2, s20
 ; CHECK-NEXT:    vmov r0, s21
-; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    vmov r3, s23
-; CHECK-NEXT:    adcs r1, r0
+; CHECK-NEXT:    adds r1, r1, r2
+; CHECK-NEXT:    adc.w r2, r12, r0
 ; CHECK-NEXT:    vmov r0, s22
-; CHECK-NEXT:    adds.w r12, r2, r0
+; CHECK-NEXT:    adds.w r12, r1, r0
+; CHECK-NEXT:    vmov.u8 r0, q1[9]
+; CHECK-NEXT:    adc.w lr, r2, r3
 ; CHECK-NEXT:    vmov.u8 r2, q4[8]
 ; CHECK-NEXT:    vmov.16 q5[0], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[9]
 ; CHECK-NEXT:    vmov.16 q5[1], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[10]
 ; CHECK-NEXT:    vmov.16 q5[2], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[11]
 ; CHECK-NEXT:    vmov.16 q5[3], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[12]
 ; CHECK-NEXT:    vmov.16 q5[4], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[13]
 ; CHECK-NEXT:    vmov.16 q5[5], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[14]
 ; CHECK-NEXT:    vmov.16 q5[6], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[15]
 ; CHECK-NEXT:    vmov.16 q5[7], r2
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    vmov.u8 r1, q0[9]
 ; CHECK-NEXT:    vcmp.i16 ne, q5, zr
-; CHECK-NEXT:    vmov.u8 r0, q0[8]
-; CHECK-NEXT:    vpsel q2, q3, q2
 ; CHECK-NEXT:    sxtb r0, r0
+; CHECK-NEXT:    vpsel q2, q3, q2
+; CHECK-NEXT:    sxtb r1, r1
 ; CHECK-NEXT:    vmov.u16 r2, q2[0]
-; CHECK-NEXT:    vmov.32 q3[0], r2
+; CHECK-NEXT:    vmov.u16 r3, q2[2]
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q2[1]
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[2]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[3]
-; CHECK-NEXT:    vmov.32 q3[3], r2
+; CHECK-NEXT:    vmov.u16 r3, q2[3]
+; CHECK-NEXT:    smull r0, r1, r1, r0
+; CHECK-NEXT:    vmov q3[3], q3[1], r3, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q3, zr
 ; CHECK-NEXT:    vmrs r2, p0
 ; CHECK-NEXT:    and r3, r2, #1
+; CHECK-NEXT:    ubfx r4, r2, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q3[0], r3
-; CHECK-NEXT:    vmov.32 q3[1], r3
-; CHECK-NEXT:    ubfx r3, r2, #4, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q3[2], r3
-; CHECK-NEXT:    vmov.32 q3[3], r3
+; CHECK-NEXT:    rsbs r4, r4, #0
+; CHECK-NEXT:    vmov q3[2], q3[0], r4, r3
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r3
 ; CHECK-NEXT:    vmov.u8 r3, q1[8]
+; CHECK-NEXT:    vmov.u8 r4, q0[8]
 ; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r0, r3, r0, r3
-; CHECK-NEXT:    vmov.32 q4[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[9]
-; CHECK-NEXT:    vmov.32 q4[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[9]
-; CHECK-NEXT:    sxtb r0, r0
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r0, r3, r3, r0
-; CHECK-NEXT:    vmov.32 q4[2], r0
-; CHECK-NEXT:    vmov.32 q4[3], r3
+; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    smull r3, r4, r4, r3
+; CHECK-NEXT:    vmov q4[2], q4[0], r0, r3
+; CHECK-NEXT:    vmov q4[3], q4[1], r1, r4
 ; CHECK-NEXT:    vand q3, q4, q3
-; CHECK-NEXT:    vmov r3, s12
+; CHECK-NEXT:    vmov r1, s12
 ; CHECK-NEXT:    vmov r0, s13
-; CHECK-NEXT:    adds.w r3, r3, r12
-; CHECK-NEXT:    adc.w r12, r1, r0
-; CHECK-NEXT:    vmov r1, s14
-; CHECK-NEXT:    vmov r0, s15
-; CHECK-NEXT:    adds r3, r3, r1
-; CHECK-NEXT:    adc.w r1, r12, r0
-; CHECK-NEXT:    ubfx r0, r2, #8, #1
-; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    ubfx r0, r2, #12, #1
-; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.u8 r2, q0[10]
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[10]
-; CHECK-NEXT:    sxtb r0, r0
-; CHECK-NEXT:    smull r0, r2, r2, r0
-; CHECK-NEXT:    vmov.32 q4[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[11]
-; CHECK-NEXT:    vmov.32 q4[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[11]
+; CHECK-NEXT:    vmov r4, s14
+; CHECK-NEXT:    vmov r3, s15
+; CHECK-NEXT:    adds.w r1, r1, r12
+; CHECK-NEXT:    adc.w r0, r0, lr
+; CHECK-NEXT:    adds r1, r1, r4
+; CHECK-NEXT:    vmov.u8 r4, q1[11]
+; CHECK-NEXT:    adc.w r12, r0, r3
+; CHECK-NEXT:    ubfx r3, r2, #8, #1
+; CHECK-NEXT:    ubfx r2, r2, #12, #1
+; CHECK-NEXT:    rsbs r3, r3, #0
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov.u8 r0, q0[11]
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r3
+; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    vmov q3[3], q3[1], r2, r3
+; CHECK-NEXT:    vmov.u8 r2, q1[10]
+; CHECK-NEXT:    vmov.u8 r3, q0[10]
 ; CHECK-NEXT:    sxtb r0, r0
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    smull r0, r2, r2, r0
-; CHECK-NEXT:    vmov.32 q4[2], r0
-; CHECK-NEXT:    vmov.32 q4[3], r2
+; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    smull r2, r3, r3, r2
+; CHECK-NEXT:    smull r0, r4, r0, r4
+; CHECK-NEXT:    vmov q4[2], q4[0], r0, r2
+; CHECK-NEXT:    vmov q4[3], q4[1], r4, r3
 ; CHECK-NEXT:    vand q3, q4, q3
 ; CHECK-NEXT:    vmov r2, s12
 ; CHECK-NEXT:    vmov r0, s13
-; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    vmov r3, s15
-; CHECK-NEXT:    adcs r1, r0
+; CHECK-NEXT:    adds r1, r1, r2
+; CHECK-NEXT:    adc.w r2, r12, r0
 ; CHECK-NEXT:    vmov r0, s14
-; CHECK-NEXT:    adds.w r12, r2, r0
+; CHECK-NEXT:    adds.w r12, r1, r0
+; CHECK-NEXT:    vmov.u8 r0, q1[13]
+; CHECK-NEXT:    adc.w lr, r2, r3
 ; CHECK-NEXT:    vmov.u16 r2, q2[4]
-; CHECK-NEXT:    vmov.32 q3[0], r2
+; CHECK-NEXT:    vmov.u16 r3, q2[6]
+; CHECK-NEXT:    vmov.u8 r1, q0[13]
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q2[5]
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[6]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[7]
-; CHECK-NEXT:    vmov.32 q3[3], r2
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    vmov.u16 r3, q2[7]
+; CHECK-NEXT:    sxtb r0, r0
+; CHECK-NEXT:    vmov q3[3], q3[1], r3, r2
+; CHECK-NEXT:    sxtb r1, r1
 ; CHECK-NEXT:    vcmp.i32 ne, q3, zr
-; CHECK-NEXT:    vmov.u8 r0, q0[12]
+; CHECK-NEXT:    smull r0, r1, r1, r0
 ; CHECK-NEXT:    vmrs r2, p0
-; CHECK-NEXT:    sxtb r0, r0
 ; CHECK-NEXT:    and r3, r2, #1
+; CHECK-NEXT:    ubfx r4, r2, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q2[0], r3
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    ubfx r3, r2, #4, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q2[2], r3
-; CHECK-NEXT:    vmov.32 q2[3], r3
+; CHECK-NEXT:    rsbs r4, r4, #0
+; CHECK-NEXT:    vmov q2[2], q2[0], r4, r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r4, r3
 ; CHECK-NEXT:    vmov.u8 r3, q1[12]
+; CHECK-NEXT:    vmov.u8 r4, q0[12]
 ; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r0, r3, r0, r3
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[13]
-; CHECK-NEXT:    vmov.32 q3[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[13]
-; CHECK-NEXT:    sxtb r0, r0
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r0, r3, r3, r0
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r3
+; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    smull r3, r4, r4, r3
+; CHECK-NEXT:    vmov q3[2], q3[0], r0, r3
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r4
 ; CHECK-NEXT:    vand q2, q3, q2
-; CHECK-NEXT:    vmov r3, s8
+; CHECK-NEXT:    vmov r1, s8
 ; CHECK-NEXT:    vmov r0, s9
-; CHECK-NEXT:    adds.w r3, r3, r12
-; CHECK-NEXT:    adc.w r12, r1, r0
-; CHECK-NEXT:    vmov r1, s10
-; CHECK-NEXT:    vmov r0, s11
-; CHECK-NEXT:    adds r3, r3, r1
-; CHECK-NEXT:    adc.w r1, r12, r0
-; CHECK-NEXT:    ubfx r0, r2, #8, #1
-; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    ubfx r0, r2, #12, #1
-; CHECK-NEXT:    rsbs r0, r0, #0
-; CHECK-NEXT:    vmov.u8 r2, q0[14]
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[14]
-; CHECK-NEXT:    sxtb r0, r0
-; CHECK-NEXT:    smull r0, r2, r2, r0
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[15]
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[15]
+; CHECK-NEXT:    vmov r4, s10
+; CHECK-NEXT:    vmov r3, s11
+; CHECK-NEXT:    adds.w r1, r1, r12
+; CHECK-NEXT:    adc.w r0, r0, lr
+; CHECK-NEXT:    adds r1, r1, r4
+; CHECK-NEXT:    vmov.u8 r4, q1[15]
+; CHECK-NEXT:    adc.w r12, r0, r3
+; CHECK-NEXT:    ubfx r3, r2, #8, #1
+; CHECK-NEXT:    ubfx r2, r2, #12, #1
+; CHECK-NEXT:    rsbs r3, r3, #0
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov.u8 r0, q0[15]
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r3
+; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    vmov q2[3], q2[1], r2, r3
+; CHECK-NEXT:    vmov.u8 r2, q1[14]
+; CHECK-NEXT:    vmov.u8 r3, q0[14]
 ; CHECK-NEXT:    sxtb r0, r0
 ; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    smull r0, r2, r2, r0
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r2
-; CHECK-NEXT:    vand q0, q3, q2
+; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    smull r2, r3, r3, r2
+; CHECK-NEXT:    smull r0, r4, r0, r4
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r3
+; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    vmov r0, s1
-; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    vmov r3, s3
-; CHECK-NEXT:    adcs r1, r0
+; CHECK-NEXT:    adds r1, r1, r2
+; CHECK-NEXT:    adc.w r2, r12, r0
 ; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    adds r0, r0, r2
-; CHECK-NEXT:    adcs r1, r3
+; CHECK-NEXT:    adds r0, r0, r1
+; CHECK-NEXT:    adc.w r1, r2, r3
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    bx lr
+; CHECK-NEXT:    pop {r4, pc}
 entry:
   %c = icmp eq <16 x i8> %b, zeroinitializer
   %xx = sext <16 x i8> %x to <16 x i64>
   %yy = sext <16 x i8> %y to <16 x i64>
   %m = mul <16 x i64> %xx, %yy
   %s = select <16 x i1> %c, <16 x i64> %m, <16 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v16i64(<16 x i64> %s)
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i8_v2i64_zext(<2 x i8> %x, <2 x i8> %y, <2 x i8> %b) {
 ; CHECK-LABEL: add_v2i8_v2i64_zext:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .vsave {d8, d9}
-; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    vmov.i64 q3, #0xff
 ; CHECK-NEXT:    vand q1, q1, q3
-; CHECK-NEXT:    vand q4, q0, q3
+; CHECK-NEXT:    vand q0, q0, q3
 ; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r1, s16
-; CHECK-NEXT:    umull r0, r1, r1, r0
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov r0, s6
-; CHECK-NEXT:    vmov.32 q0[1], r1
-; CHECK-NEXT:    vmov r1, s18
+; CHECK-NEXT:    vmov r1, s0
+; CHECK-NEXT:    vmov r2, s6
 ; CHECK-NEXT:    vand q1, q2, q3
+; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    umull r0, r1, r1, r0
-; CHECK-NEXT:    vmov.32 q0[2], r0
+; CHECK-NEXT:    umull r2, r3, r3, r2
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r0
 ; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov.32 q0[3], r1
-; CHECK-NEXT:    cmp r0, #0
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s6
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r1
+; CHECK-NEXT:    vmov r1, s6
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    cmp r1, #0
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
-; CHECK-NEXT:    vpop {d8, d9}
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp eq <2 x i8> %b, zeroinitializer
   %xx = zext <2 x i8> %x to <2 x i64>
   %yy = zext <2 x i8> %y to <2 x i64>
   %m = mul <2 x i64> %xx, %yy
   %s = select <2 x i1> %c, <2 x i64> %m, <2 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> %s)
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i8_v2i64_sext(<2 x i8> %x, <2 x i8> %y, <2 x i8> %b) {
 ; CHECK-LABEL: add_v2i8_v2i64_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.i32 q3, #0xff
-; CHECK-NEXT:    vmov r1, s0
-; CHECK-NEXT:    vand q3, q2, q3
-; CHECK-NEXT:    vmov r0, s12
-; CHECK-NEXT:    sxtb r1, r1
-; CHECK-NEXT:    cmp r0, #0
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s14
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    vand q2, q2, q3
+; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    vmov r0, s8
+; CHECK-NEXT:    vmov r1, s10
+; CHECK-NEXT:    sxtb r2, r2
 ; CHECK-NEXT:    cmp r0, #0
+; CHECK-NEXT:    sxtb r3, r3
 ; CHECK-NEXT:    cset r0, eq
+; CHECK-NEXT:    smull r2, r3, r3, r2
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    cmp r1, #0
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    sxtb r0, r0
-; CHECK-NEXT:    smull r0, r1, r1, r0
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov r0, s6
-; CHECK-NEXT:    vmov.32 q3[1], r1
-; CHECK-NEXT:    vmov r1, s2
+; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    sxtb r0, r0
 ; CHECK-NEXT:    sxtb r1, r1
 ; CHECK-NEXT:    smull r0, r1, r1, r0
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.32 q3[3], r1
-; CHECK-NEXT:    vand q0, q3, q2
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r1
+; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    bx lr
 entry:
   %c = icmp eq <2 x i8> %b, zeroinitializer
   %xx = sext <2 x i8> %x to <2 x i64>
   %yy = sext <2 x i8> %y to <2 x i64>
   %m = mul <2 x i64> %xx, %yy
   %s = select <2 x i1> %c, <2 x i64> %m, <2 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> %s)
   ret i64 %z
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i64_v2i64(<2 x i64> %x, <2 x i64> %y, <2 x i64> %b) {
 ; CHECK-LABEL: add_v2i64_v2i64:
 ; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    vmov r0, s4
 ; CHECK-NEXT:    vmov r1, s0
-; CHECK-NEXT:    vmov r2, s5
-; CHECK-NEXT:    umull r12, r3, r1, r0
-; CHECK-NEXT:    mla r1, r1, r2, r3
-; CHECK-NEXT:    vmov r2, s1
-; CHECK-NEXT:    vmov.32 q3[0], r12
-; CHECK-NEXT:    mla r0, r2, r0, r1
-; CHECK-NEXT:    vmov r1, s2
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov r0, s6
-; CHECK-NEXT:    umull r2, r3, r1, r0
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov r2, s7
-; CHECK-NEXT:    mla r1, r1, r2, r3
+; CHECK-NEXT:    vmov r3, s6
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    umull lr, r12, r1, r0
+; CHECK-NEXT:    umull r4, r5, r2, r3
+; CHECK-NEXT:    vmov q3[2], q3[0], r4, lr
+; CHECK-NEXT:    vmov r4, s5
+; CHECK-NEXT:    mla r1, r1, r4, r12
+; CHECK-NEXT:    vmov r4, s1
+; CHECK-NEXT:    mla r0, r4, r0, r1
+; CHECK-NEXT:    vmov r1, s7
+; CHECK-NEXT:    mla r1, r2, r1, r5
 ; CHECK-NEXT:    vmov r2, s3
-; CHECK-NEXT:    mla r0, r2, r0, r1
-; CHECK-NEXT:    vmov r1, s8
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    mla r1, r2, r3, r1
+; CHECK-NEXT:    vmov r2, s10
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
 ; CHECK-NEXT:    vmov r0, s9
+; CHECK-NEXT:    vmov r1, s8
 ; CHECK-NEXT:    orrs r0, r1
-; CHECK-NEXT:    vmov r1, s10
-; CHECK-NEXT:    cset r0, eq
-; CHECK-NEXT:    tst.w r0, #1
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.32 q0[1], r0
-; CHECK-NEXT:    vmov r0, s11
-; CHECK-NEXT:    orrs r0, r1
+; CHECK-NEXT:    vmov r1, s11
 ; CHECK-NEXT:    cset r0, eq
 ; CHECK-NEXT:    tst.w r0, #1
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    vmov.32 q0[3], r0
+; CHECK-NEXT:    orrs r1, r2
+; CHECK-NEXT:    cset r1, eq
+; CHECK-NEXT:    tst.w r1, #1
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
 ; CHECK-NEXT:    vand q0, q3, q0
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
-; CHECK-NEXT:    bx lr
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %c = icmp eq <2 x i64> %b, zeroinitializer
   %m = mul <2 x i64> %x, %y
   %s = select <2 x i1> %c, <2 x i64> %m, <2 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> %s)
   ret i64 %z
 }
 

 
 define arm_aapcs_vfpcc i64 @add_v2i32_v2i64_acc_zext(<2 x i32> %x, <2 x i32> %y, <2 x i32> %b, i64 %a) {
 ; CHECK-LABEL: add_v2i32_v2i64_acc_zext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
 ; CHECK-NEXT:    vmov r2, s8
 ; CHECK-NEXT:    vmullb.u32 q3, q0, q1
+; CHECK-NEXT:    vmov r3, s10
 ; CHECK-NEXT:    cmp r2, #0
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    cmp r2, #0
-; CHECK-NEXT:    cset r2, eq
-; CHECK-NEXT:    tst.w r2, #1
-; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vmov.32 q0[3], r2
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    cset r3, eq
+; CHECK-NEXT:    tst.w r3, #1
+; CHECK-NEXT:    csetm r3, ne
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r2
 ; CHECK-NEXT:    vand q0, q3, q0
 ; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r12, s3
 ; CHECK-NEXT:    vmov lr, s1
 ; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    adc.w r3, lr, r12
 ; CHECK-NEXT:    adds r0, r0, r2

 
 define arm_aapcs_vfpcc i64 @add_v2i32_v2i64_acc_sext(<2 x i32> %x, <2 x i32> %y, <2 x i32> %b, i64 %a) {
 ; CHECK-LABEL: add_v2i32_v2i64_acc_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
 ; CHECK-NEXT:    vmov r2, s8
 ; CHECK-NEXT:    vmullb.s32 q3, q0, q1
+; CHECK-NEXT:    vmov r3, s10
 ; CHECK-NEXT:    cmp r2, #0
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    cmp r2, #0
-; CHECK-NEXT:    cset r2, eq
-; CHECK-NEXT:    tst.w r2, #1
-; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vmov.32 q0[3], r2
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    cset r3, eq
+; CHECK-NEXT:    tst.w r3, #1
+; CHECK-NEXT:    csetm r3, ne
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r2
 ; CHECK-NEXT:    vand q0, q3, q0
 ; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r12, s3
 ; CHECK-NEXT:    vmov lr, s1
 ; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    adc.w r3, lr, r12
 ; CHECK-NEXT:    adds r0, r0, r2
entry:
   ret i64 %r
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i16_v2i64_acc_zext(<2 x i16> %x, <2 x i16> %y, <2 x i16> %b, i64 %a) {
 ; CHECK-LABEL: add_v2i16_v2i64_acc_zext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
-; CHECK-NEXT:    .vsave {d8, d9}
-; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    vmov.i64 q3, #0xffff
 ; CHECK-NEXT:    vand q1, q1, q3
-; CHECK-NEXT:    vand q4, q0, q3
+; CHECK-NEXT:    vand q0, q0, q3
 ; CHECK-NEXT:    vmov r2, s4
-; CHECK-NEXT:    vmov r3, s16
-; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    vmov.32 q0[1], r3
-; CHECK-NEXT:    vmov r3, s18
+; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    umull lr, r12, r3, r2
+; CHECK-NEXT:    vmov r3, s6
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vand q1, q2, q3
-; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q0[2], r2
+; CHECK-NEXT:    umull r2, r3, r2, r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, lr
 ; CHECK-NEXT:    vmov r2, s4
-; CHECK-NEXT:    vmov.32 q0[3], r3
-; CHECK-NEXT:    cmp r2, #0
-; CHECK-NEXT:    cset r2, eq
-; CHECK-NEXT:    tst.w r2, #1
-; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r12
+; CHECK-NEXT:    vmov r3, s6
 ; CHECK-NEXT:    cmp r2, #0
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r2
-; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    cset r3, eq
+; CHECK-NEXT:    tst.w r3, #1
+; CHECK-NEXT:    csetm r3, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r2
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r12, s3
 ; CHECK-NEXT:    vmov lr, s1
 ; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    adc.w r3, lr, r12
 ; CHECK-NEXT:    adds r0, r0, r2
 ; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vpop {d8, d9}
 ; CHECK-NEXT:    pop {r7, pc}
 entry:
   %c = icmp eq <2 x i16> %b, zeroinitializer
   %xx = zext <2 x i16> %x to <2 x i64>
   %yy = zext <2 x i16> %y to <2 x i64>
   %m = mul <2 x i64> %xx, %yy
   %s = select <2 x i1> %c, <2 x i64> %m, <2 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> %s)
   %r = add i64 %z, %a
   ret i64 %r
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i16_v2i64_acc_sext(<2 x i16> %x, <2 x i16> %y, <2 x i16> %b, i64 %a) {
 ; CHECK-LABEL: add_v2i16_v2i64_acc_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
 ; CHECK-NEXT:    vmov.i32 q3, #0xffff
-; CHECK-NEXT:    vmov r3, s0
-; CHECK-NEXT:    vand q3, q2, q3
-; CHECK-NEXT:    vmov r2, s12
-; CHECK-NEXT:    sxth r3, r3
-; CHECK-NEXT:    cmp r2, #0
-; CHECK-NEXT:    cset r2, eq
-; CHECK-NEXT:    tst.w r2, #1
-; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov r2, s14
+; CHECK-NEXT:    vand q2, q2, q3
+; CHECK-NEXT:    vmov r2, s8
+; CHECK-NEXT:    vmov r3, s10
 ; CHECK-NEXT:    cmp r2, #0
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r2
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    cset r3, eq
+; CHECK-NEXT:    tst.w r3, #1
+; CHECK-NEXT:    csetm r3, ne
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r2
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r2
 ; CHECK-NEXT:    vmov r2, s4
-; CHECK-NEXT:    sxth r2, r2
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    vmov.32 q3[1], r3
-; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    sxth r2, r2
 ; CHECK-NEXT:    sxth r3, r3
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.32 q3[3], r3
-; CHECK-NEXT:    vand q0, q3, q2
+; CHECK-NEXT:    smull lr, r12, r3, r2
+; CHECK-NEXT:    vmov r3, s6
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    sxth r3, r3
+; CHECK-NEXT:    sxth r2, r2
+; CHECK-NEXT:    smull r2, r3, r2, r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, lr
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r12
+; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r12, s3
 ; CHECK-NEXT:    vmov lr, s1
 ; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    adc.w r3, lr, r12
 ; CHECK-NEXT:    adds r0, r0, r2
 ; CHECK-NEXT:    adcs r1, r3

 
 define arm_aapcs_vfpcc i64 @add_v16i8_v16i64_acc_zext(<16 x i8> %x, <16 x i8> %y, <16 x i8> %b, i64 %a) {
 ; CHECK-LABEL: add_v16i8_v16i64_acc_zext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r6, lr}
 ; CHECK-NEXT:    push {r4, r5, r6, lr}
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    .pad #80
-; CHECK-NEXT:    sub sp, #80
-; CHECK-NEXT:    vmov q3, q1
-; CHECK-NEXT:    vstrw.32 q0, [sp, #64] @ 16-byte Spill
-; CHECK-NEXT:    vmov.i8 q0, #0x0
-; CHECK-NEXT:    vmov.i8 q1, #0xff
+; CHECK-NEXT:    .pad #32
+; CHECK-NEXT:    sub sp, #32
+; CHECK-NEXT:    vmov q4, q0
 ; CHECK-NEXT:    vcmp.i8 eq, q2, zr
-; CHECK-NEXT:    vstrw.32 q1, [sp, #16] @ 16-byte Spill
-; CHECK-NEXT:    vpsel q5, q1, q0
-; CHECK-NEXT:    vstrw.32 q0, [sp, #32] @ 16-byte Spill
+; CHECK-NEXT:    vmov.i8 q2, #0xff
+; CHECK-NEXT:    vmov.i8 q0, #0x0
+; CHECK-NEXT:    vpsel q5, q2, q0
+; CHECK-NEXT:    vmov q3, q2
 ; CHECK-NEXT:    vmov.u8 r2, q5[0]
-; CHECK-NEXT:    vmov.i64 q4, #0xff
+; CHECK-NEXT:    vstrw.32 q2, [sp] @ 16-byte Spill
 ; CHECK-NEXT:    vmov.16 q2[0], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[1]
 ; CHECK-NEXT:    vmov.16 q2[1], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[2]
 ; CHECK-NEXT:    vmov.16 q2[2], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[3]
 ; CHECK-NEXT:    vmov.16 q2[3], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[4]
 ; CHECK-NEXT:    vmov.16 q2[4], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[5]
 ; CHECK-NEXT:    vmov.16 q2[5], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[6]
 ; CHECK-NEXT:    vmov.16 q2[6], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[7]
 ; CHECK-NEXT:    vmov.16 q2[7], r2
+; CHECK-NEXT:    vstrw.32 q0, [sp, #16] @ 16-byte Spill
 ; CHECK-NEXT:    vcmp.i16 ne, q2, zr
-; CHECK-NEXT:    vpsel q6, q1, q0
+; CHECK-NEXT:    vpsel q6, q3, q0
 ; CHECK-NEXT:    vmov.u16 r2, q6[0]
-; CHECK-NEXT:    vmov.32 q2[0], r2
+; CHECK-NEXT:    vmov.u16 r3, q6[2]
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q6[1]
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q6[2]
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q6[3]
-; CHECK-NEXT:    vmov.32 q2[3], r2
+; CHECK-NEXT:    vmov.u16 r3, q6[3]
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q2, zr
+; CHECK-NEXT:    vmov.i64 q2, #0xff
 ; CHECK-NEXT:    vmrs lr, p0
-; CHECK-NEXT:    vldrw.u32 q1, [sp, #64] @ 16-byte Reload
 ; CHECK-NEXT:    and r3, lr, #1
+; CHECK-NEXT:    ubfx r2, lr, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q7[0], r3
-; CHECK-NEXT:    vmov.32 q7[1], r3
-; CHECK-NEXT:    ubfx r3, lr, #4, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q7[2], r3
-; CHECK-NEXT:    vmov.32 q7[3], r3
-; CHECK-NEXT:    vmov.u8 r3, q3[0]
-; CHECK-NEXT:    vmov.32 q0[0], r3
-; CHECK-NEXT:    vmov.u8 r3, q3[1]
-; CHECK-NEXT:    vmov.32 q0[2], r3
-; CHECK-NEXT:    vmov.u8 r3, q1[0]
-; CHECK-NEXT:    vand q2, q0, q4
-; CHECK-NEXT:    vmov.32 q0[0], r3
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov q7[2], q7[0], r2, r3
+; CHECK-NEXT:    vmov q7[3], q7[1], r2, r3
+; CHECK-NEXT:    vmov.u8 r2, q1[0]
 ; CHECK-NEXT:    vmov.u8 r3, q1[1]
-; CHECK-NEXT:    vmov r12, s8
-; CHECK-NEXT:    vmov.32 q0[2], r3
-; CHECK-NEXT:    vand q1, q0, q4
-; CHECK-NEXT:    vmov r3, s4
-; CHECK-NEXT:    umull r3, r2, r3, r12
-; CHECK-NEXT:    vmov.32 q0[0], r3
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    vldrw.u32 q2, [sp, #64] @ 16-byte Reload
-; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vmov.32 q0[3], r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
+; CHECK-NEXT:    vmov.u8 r3, q4[0]
+; CHECK-NEXT:    vmov.u8 r2, q4[1]
+; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r3
+; CHECK-NEXT:    vmov r12, s0
+; CHECK-NEXT:    vand q3, q3, q2
+; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    vmov r2, s12
+; CHECK-NEXT:    vmov r4, s14
+; CHECK-NEXT:    umull r2, r12, r2, r12
+; CHECK-NEXT:    umull r3, r4, r4, r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r12
 ; CHECK-NEXT:    vand q0, q0, q7
-; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    vmov r12, s3
-; CHECK-NEXT:    vmov r3, s1
-; CHECK-NEXT:    adds r4, r4, r2
+; CHECK-NEXT:    vmov r4, s1
+; CHECK-NEXT:    adds r6, r2, r3
 ; CHECK-NEXT:    ubfx r2, lr, #8, #1
 ; CHECK-NEXT:    rsb.w r2, r2, #0
-; CHECK-NEXT:    vmov.32 q7[0], r2
-; CHECK-NEXT:    adc.w r12, r12, r3
-; CHECK-NEXT:    vmov.32 q7[1], r2
-; CHECK-NEXT:    ubfx r2, lr, #12, #1
-; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.u8 r3, q2[2]
-; CHECK-NEXT:    vmov.32 q7[2], r2
-; CHECK-NEXT:    vmov.32 q1[0], r3
-; CHECK-NEXT:    vmov.32 q7[3], r2
-; CHECK-NEXT:    vmov.u8 r2, q3[2]
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q3[3]
-; CHECK-NEXT:    vmov.u8 r3, q2[3]
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vmov.32 q1[2], r3
-; CHECK-NEXT:    vand q0, q0, q4
-; CHECK-NEXT:    vand q1, q1, q4
+; CHECK-NEXT:    vmov.u8 r3, q4[3]
+; CHECK-NEXT:    adc.w r12, r12, r4
+; CHECK-NEXT:    ubfx r4, lr, #12, #1
+; CHECK-NEXT:    rsbs r4, r4, #0
+; CHECK-NEXT:    vmov q7[2], q7[0], r4, r2
+; CHECK-NEXT:    vmov q7[3], q7[1], r4, r2
+; CHECK-NEXT:    vmov.u8 r2, q1[2]
+; CHECK-NEXT:    vmov.u8 r4, q1[3]
+; CHECK-NEXT:    vmov q0[2], q0[0], r4, r2
+; CHECK-NEXT:    vmov.u8 r4, q4[2]
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r4
+; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    vand q3, q3, q2
 ; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    vmov r3, s4
-; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov r3, s6
+; CHECK-NEXT:    vmov r3, s12
+; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    vmov r5, s14
 ; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    vand q0, q2, q7
+; CHECK-NEXT:    umull r5, r4, r5, r4
+; CHECK-NEXT:    vmov q0[2], q0[0], r5, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r3
+; CHECK-NEXT:    vmov.u8 r4, q4[5]
+; CHECK-NEXT:    vand q0, q0, q7
+; CHECK-NEXT:    vmov q7, q4
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r2, s1
-; CHECK-NEXT:    adds r3, r3, r4
-; CHECK-NEXT:    vmov r4, s3
-; CHECK-NEXT:    adc.w lr, r12, r2
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    adds.w r12, r3, r2
+; CHECK-NEXT:    vmov r5, s2
+; CHECK-NEXT:    adds r3, r3, r6
+; CHECK-NEXT:    vmov r6, s3
+; CHECK-NEXT:    adc.w r2, r2, r12
+; CHECK-NEXT:    adds.w r12, r3, r5
+; CHECK-NEXT:    adc.w lr, r2, r6
 ; CHECK-NEXT:    vmov.u16 r2, q6[4]
-; CHECK-NEXT:    vmov.32 q0[0], r2
+; CHECK-NEXT:    vmov.u16 r6, q6[6]
+; CHECK-NEXT:    vmov q0[2], q0[0], r6, r2
 ; CHECK-NEXT:    vmov.u16 r2, q6[5]
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q6[6]
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q6[7]
-; CHECK-NEXT:    vmov.32 q0[3], r2
-; CHECK-NEXT:    adc.w lr, lr, r4
+; CHECK-NEXT:    vmov.u16 r6, q6[7]
+; CHECK-NEXT:    vmov q0[3], q0[1], r6, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q0, zr
-; CHECK-NEXT:    vmrs r6, p0
-; CHECK-NEXT:    vstrw.32 q3, [sp, #48] @ 16-byte Spill
-; CHECK-NEXT:    vstrw.32 q4, [sp] @ 16-byte Spill
-; CHECK-NEXT:    and r4, r6, #1
-; CHECK-NEXT:    rsbs r4, r4, #0
-; CHECK-NEXT:    vmov.32 q6[0], r4
-; CHECK-NEXT:    vmov.32 q6[1], r4
-; CHECK-NEXT:    ubfx r4, r6, #4, #1
-; CHECK-NEXT:    rsbs r4, r4, #0
-; CHECK-NEXT:    vmov.32 q6[2], r4
-; CHECK-NEXT:    vmov.32 q6[3], r4
-; CHECK-NEXT:    vmov.u8 r4, q3[4]
-; CHECK-NEXT:    vmov.32 q0[0], r4
-; CHECK-NEXT:    vmov.u8 r4, q3[5]
-; CHECK-NEXT:    vldrw.u32 q3, [sp, #64] @ 16-byte Reload
-; CHECK-NEXT:    vmov.32 q0[2], r4
-; CHECK-NEXT:    vand q0, q0, q4
-; CHECK-NEXT:    vmov.u8 r3, q3[4]
-; CHECK-NEXT:    vmov r4, s0
-; CHECK-NEXT:    vmov.32 q1[0], r3
-; CHECK-NEXT:    vmov.u8 r3, q3[5]
-; CHECK-NEXT:    vmov.32 q1[2], r3
-; CHECK-NEXT:    vmov q7, q3
-; CHECK-NEXT:    vand q1, q1, q4
-; CHECK-NEXT:    vmov r3, s4
+; CHECK-NEXT:    vmrs r2, p0
+; CHECK-NEXT:    and r6, r2, #1
+; CHECK-NEXT:    ubfx r5, r2, #4, #1
+; CHECK-NEXT:    rsbs r6, r6, #0
+; CHECK-NEXT:    rsbs r5, r5, #0
+; CHECK-NEXT:    vmov q6[2], q6[0], r5, r6
+; CHECK-NEXT:    vmov q6[3], q6[1], r5, r6
+; CHECK-NEXT:    vmov.u8 r6, q1[4]
+; CHECK-NEXT:    vmov.u8 r5, q1[5]
+; CHECK-NEXT:    vmov q0[2], q0[0], r5, r6
+; CHECK-NEXT:    vmov.u8 r5, q4[4]
+; CHECK-NEXT:    vmov q3[2], q3[0], r4, r5
+; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    vand q3, q3, q2
+; CHECK-NEXT:    vmov r6, s0
+; CHECK-NEXT:    vmov r5, s12
+; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    vmov r3, s14
+; CHECK-NEXT:    umull r6, r5, r5, r6
 ; CHECK-NEXT:    umull r3, r4, r3, r4
-; CHECK-NEXT:    vmov.32 q2[0], r3
-; CHECK-NEXT:    vmov r3, s2
-; CHECK-NEXT:    vmov.32 q2[1], r4
-; CHECK-NEXT:    vmov r4, s6
-; CHECK-NEXT:    vldrw.u32 q1, [sp, #48] @ 16-byte Reload
-; CHECK-NEXT:    umull r3, r4, r4, r3
-; CHECK-NEXT:    vmov.32 q2[2], r3
-; CHECK-NEXT:    vmov.32 q2[3], r4
-; CHECK-NEXT:    vand q0, q2, q6
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r6
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r5
+; CHECK-NEXT:    vand q0, q0, q6
 ; CHECK-NEXT:    vmov r4, s0
 ; CHECK-NEXT:    vmov r3, s1
-; CHECK-NEXT:    vmov r5, s2
-; CHECK-NEXT:    vmov r2, s3
-; CHECK-NEXT:    adds.w r4, r4, r12
-; CHECK-NEXT:    adc.w r3, r3, lr
-; CHECK-NEXT:    adds r4, r4, r5
-; CHECK-NEXT:    adc.w r12, r3, r2
-; CHECK-NEXT:    ubfx r2, r6, #8, #1
-; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.u8 r3, q3[6]
-; CHECK-NEXT:    vmov.32 q6[0], r2
-; CHECK-NEXT:    vmov.32 q6[1], r2
-; CHECK-NEXT:    ubfx r2, r6, #12, #1
-; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q6[2], r2
-; CHECK-NEXT:    vmov.32 q6[3], r2
+; CHECK-NEXT:    adds.w r6, r12, r4
+; CHECK-NEXT:    vmov r4, s3
+; CHECK-NEXT:    adc.w r5, lr, r3
+; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    adds r3, r3, r6
+; CHECK-NEXT:    ubfx r6, r2, #8, #1
+; CHECK-NEXT:    ubfx r2, r2, #12, #1
+; CHECK-NEXT:    rsb.w r6, r6, #0
+; CHECK-NEXT:    rsb.w r2, r2, #0
+; CHECK-NEXT:    adc.w r12, r5, r4
+; CHECK-NEXT:    vmov q6[2], q6[0], r2, r6
+; CHECK-NEXT:    vmov.u8 r5, q4[7]
+; CHECK-NEXT:    vmov q6[3], q6[1], r2, r6
 ; CHECK-NEXT:    vmov.u8 r2, q1[6]
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[7]
-; CHECK-NEXT:    vmov.32 q1[0], r3
-; CHECK-NEXT:    vmov.u8 r3, q3[7]
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vmov.32 q1[2], r3
-; CHECK-NEXT:    vand q0, q0, q4
-; CHECK-NEXT:    vand q1, q1, q4
+; CHECK-NEXT:    vmov.u8 r6, q1[7]
+; CHECK-NEXT:    vmov q0[2], q0[0], r6, r2
+; CHECK-NEXT:    vmov.u8 r6, q4[6]
+; CHECK-NEXT:    vmov q3[2], q3[0], r5, r6
+; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    vand q3, q3, q2
 ; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    vmov r3, s4
-; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    vldrw.u32 q1, [sp, #16] @ 16-byte Reload
-; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    vand q0, q2, q6
-; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    vmov r6, s12
+; CHECK-NEXT:    vmov r4, s14
+; CHECK-NEXT:    vldrw.u32 q3, [sp] @ 16-byte Reload
+; CHECK-NEXT:    vmov r5, s2
+; CHECK-NEXT:    umull r2, r6, r6, r2
+; CHECK-NEXT:    umull r5, r4, r4, r5
+; CHECK-NEXT:    vmov q0[2], q0[0], r5, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r6
+; CHECK-NEXT:    vmov.u8 r4, q7[9]
+; CHECK-NEXT:    vand q0, q0, q6
+; CHECK-NEXT:    vmov r6, s0
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    vmov r5, s2
+; CHECK-NEXT:    adds r3, r3, r6
 ; CHECK-NEXT:    vmov r6, s3
-; CHECK-NEXT:    vldrw.u32 q0, [sp, #32] @ 16-byte Reload
-; CHECK-NEXT:    adds r3, r3, r4
 ; CHECK-NEXT:    adc.w r2, r2, r12
+; CHECK-NEXT:    vldrw.u32 q0, [sp, #16] @ 16-byte Reload
 ; CHECK-NEXT:    adds.w r12, r3, r5
-; CHECK-NEXT:    vmov.u8 r5, q7[8]
-; CHECK-NEXT:    adc.w r3, r2, r6
+; CHECK-NEXT:    adc.w lr, r2, r6
 ; CHECK-NEXT:    vmov.u8 r2, q5[8]
 ; CHECK-NEXT:    vmov.16 q6[0], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[9]
 ; CHECK-NEXT:    vmov.16 q6[1], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[10]
 ; CHECK-NEXT:    vmov.16 q6[2], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[11]
 ; CHECK-NEXT:    vmov.16 q6[3], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[12]
 ; CHECK-NEXT:    vmov.16 q6[4], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[13]
 ; CHECK-NEXT:    vmov.16 q6[5], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[14]
 ; CHECK-NEXT:    vmov.16 q6[6], r2
 ; CHECK-NEXT:    vmov.u8 r2, q5[15]
 ; CHECK-NEXT:    vmov.16 q6[7], r2
 ; CHECK-NEXT:    vcmp.i16 ne, q6, zr
-; CHECK-NEXT:    vpsel q3, q1, q0
-; CHECK-NEXT:    vmov.32 q1[0], r5
+; CHECK-NEXT:    vpsel q3, q3, q0
 ; CHECK-NEXT:    vmov.u16 r2, q3[0]
-; CHECK-NEXT:    vmov.u8 r5, q7[9]
-; CHECK-NEXT:    vmov.32 q0[0], r2
+; CHECK-NEXT:    vmov.u16 r6, q3[2]
+; CHECK-NEXT:    vmov q0[2], q0[0], r6, r2
 ; CHECK-NEXT:    vmov.u16 r2, q3[1]
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q3[2]
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q3[3]
-; CHECK-NEXT:    vmov.32 q0[3], r2
-; CHECK-NEXT:    vmov.32 q1[2], r5
+; CHECK-NEXT:    vmov.u16 r6, q3[3]
+; CHECK-NEXT:    vmov q0[3], q0[1], r6, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q0, zr
 ; CHECK-NEXT:    vmrs r2, p0
-; CHECK-NEXT:    vldrw.u32 q6, [sp, #48] @ 16-byte Reload
-; CHECK-NEXT:    vldrw.u32 q5, [sp] @ 16-byte Reload
-; CHECK-NEXT:    vand q1, q1, q5
-; CHECK-NEXT:    vmov r5, s4
 ; CHECK-NEXT:    and r6, r2, #1
+; CHECK-NEXT:    ubfx r5, r2, #4, #1
 ; CHECK-NEXT:    rsbs r6, r6, #0
-; CHECK-NEXT:    vmov.32 q4[0], r6
-; CHECK-NEXT:    vmov.32 q4[1], r6
-; CHECK-NEXT:    ubfx r6, r2, #4, #1
-; CHECK-NEXT:    rsbs r6, r6, #0
-; CHECK-NEXT:    vmov.32 q4[2], r6
-; CHECK-NEXT:    vmov.32 q4[3], r6
-; CHECK-NEXT:    vmov.u8 r6, q6[8]
-; CHECK-NEXT:    vmov.32 q0[0], r6
-; CHECK-NEXT:    vmov.u8 r6, q6[9]
-; CHECK-NEXT:    vmov.32 q0[2], r6
-; CHECK-NEXT:    vand q0, q0, q5
+; CHECK-NEXT:    rsbs r5, r5, #0
+; CHECK-NEXT:    vmov q4[2], q4[0], r5, r6
+; CHECK-NEXT:    vmov q4[3], q4[1], r5, r6
+; CHECK-NEXT:    vmov.u8 r6, q1[8]
+; CHECK-NEXT:    vmov.u8 r5, q1[9]
+; CHECK-NEXT:    vmov q0[2], q0[0], r5, r6
+; CHECK-NEXT:    vmov.u8 r5, q7[8]
+; CHECK-NEXT:    vmov q5[2], q5[0], r4, r5
+; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    vand q5, q5, q2
 ; CHECK-NEXT:    vmov r6, s0
+; CHECK-NEXT:    vmov r5, s20
+; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    vmov r3, s22
 ; CHECK-NEXT:    umull r6, r5, r5, r6
-; CHECK-NEXT:    vmov.32 q2[0], r6
-; CHECK-NEXT:    vmov r6, s2
-; CHECK-NEXT:    vmov.32 q2[1], r5
-; CHECK-NEXT:    vmov r5, s6
-; CHECK-NEXT:    umull r6, r5, r5, r6
-; CHECK-NEXT:    vmov.32 q2[2], r6
-; CHECK-NEXT:    vmov.32 q2[3], r5
-; CHECK-NEXT:    vand q0, q2, q4
-; CHECK-NEXT:    vmov r5, s0
-; CHECK-NEXT:    vmov r6, s1
+; CHECK-NEXT:    umull r3, r4, r3, r4
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r6
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r5
+; CHECK-NEXT:    vand q0, q0, q4
+; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov r3, s1
+; CHECK-NEXT:    adds.w r6, r12, r4
 ; CHECK-NEXT:    vmov r4, s3
-; CHECK-NEXT:    adds.w r5, r5, r12
-; CHECK-NEXT:    adcs r6, r3
+; CHECK-NEXT:    adc.w r5, lr, r3
 ; CHECK-NEXT:    vmov r3, s2
-; CHECK-NEXT:    adds r3, r3, r5
-; CHECK-NEXT:    adc.w r12, r6, r4
+; CHECK-NEXT:    adds r3, r3, r6
 ; CHECK-NEXT:    ubfx r6, r2, #8, #1
-; CHECK-NEXT:    rsbs r6, r6, #0
 ; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q4[0], r6
-; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q4[1], r6
+; CHECK-NEXT:    rsb.w r6, r6, #0
+; CHECK-NEXT:    rsb.w r2, r2, #0
+; CHECK-NEXT:    adc.w r12, r5, r4
+; CHECK-NEXT:    vmov q4[2], q4[0], r2, r6
+; CHECK-NEXT:    vmov.u8 r5, q7[11]
+; CHECK-NEXT:    vmov q4[3], q4[1], r2, r6
+; CHECK-NEXT:    vmov.u8 r2, q1[10]
+; CHECK-NEXT:    vmov.u8 r6, q1[11]
+; CHECK-NEXT:    vmov q0[2], q0[0], r6, r2
 ; CHECK-NEXT:    vmov.u8 r6, q7[10]
-; CHECK-NEXT:    vmov.32 q4[2], r2
-; CHECK-NEXT:    vmov.32 q1[0], r6
-; CHECK-NEXT:    vmov.32 q4[3], r2
-; CHECK-NEXT:    vmov.u8 r2, q6[10]
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q6[11]
-; CHECK-NEXT:    vmov.u8 r6, q7[11]
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vmov.32 q1[2], r6
-; CHECK-NEXT:    vand q0, q0, q5
-; CHECK-NEXT:    vand q1, q1, q5
+; CHECK-NEXT:    vmov q5[2], q5[0], r5, r6
+; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    vand q5, q5, q2
 ; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    vmov r6, s4
-; CHECK-NEXT:    umull r2, r6, r6, r2
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    vmov.32 q2[1], r6
-; CHECK-NEXT:    vmov r6, s6
+; CHECK-NEXT:    vmov r6, s20
+; CHECK-NEXT:    vmov r5, s2
+; CHECK-NEXT:    vmov r4, s22
 ; CHECK-NEXT:    umull r2, r6, r6, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r6
-; CHECK-NEXT:    vand q0, q2, q4
+; CHECK-NEXT:    umull r5, r4, r4, r5
+; CHECK-NEXT:    vmov q0[2], q0[0], r5, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r6
+; CHECK-NEXT:    vmov.u8 r4, q7[13]
+; CHECK-NEXT:    vand q0, q0, q4
 ; CHECK-NEXT:    vmov r6, s0
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    vmov r5, s2
 ; CHECK-NEXT:    adds r3, r3, r6
 ; CHECK-NEXT:    vmov r6, s3
 ; CHECK-NEXT:    adc.w r2, r2, r12
 ; CHECK-NEXT:    adds.w r12, r3, r5
-; CHECK-NEXT:    vmov.u8 r5, q7[12]
-; CHECK-NEXT:    vmov.32 q1[0], r5
-; CHECK-NEXT:    vmov.u8 r5, q7[13]
-; CHECK-NEXT:    vmov.32 q1[2], r5
-; CHECK-NEXT:    vand q1, q1, q5
-; CHECK-NEXT:    vmov r5, s4
-; CHECK-NEXT:    adc.w r3, r2, r6
+; CHECK-NEXT:    adc.w lr, r2, r6
 ; CHECK-NEXT:    vmov.u16 r2, q3[4]
-; CHECK-NEXT:    vmov.32 q0[0], r2
+; CHECK-NEXT:    vmov.u16 r6, q3[6]
+; CHECK-NEXT:    vmov q0[2], q0[0], r6, r2
 ; CHECK-NEXT:    vmov.u16 r2, q3[5]
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q3[6]
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q3[7]
-; CHECK-NEXT:    vmov.32 q0[3], r2
+; CHECK-NEXT:    vmov.u16 r6, q3[7]
+; CHECK-NEXT:    vmov q0[3], q0[1], r6, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q0, zr
 ; CHECK-NEXT:    vmrs r2, p0
 ; CHECK-NEXT:    and r6, r2, #1
+; CHECK-NEXT:    ubfx r5, r2, #4, #1
 ; CHECK-NEXT:    rsbs r6, r6, #0
-; CHECK-NEXT:    vmov.32 q3[0], r6
-; CHECK-NEXT:    vmov.32 q3[1], r6
-; CHECK-NEXT:    ubfx r6, r2, #4, #1
-; CHECK-NEXT:    rsbs r6, r6, #0
-; CHECK-NEXT:    vmov.32 q3[2], r6
-; CHECK-NEXT:    vmov.32 q3[3], r6
-; CHECK-NEXT:    vmov.u8 r6, q6[12]
-; CHECK-NEXT:    vmov.32 q0[0], r6
-; CHECK-NEXT:    vmov.u8 r6, q6[13]
-; CHECK-NEXT:    vmov.32 q0[2], r6
-; CHECK-NEXT:    vand q0, q0, q5
+; CHECK-NEXT:    rsbs r5, r5, #0
+; CHECK-NEXT:    vmov q3[2], q3[0], r5, r6
+; CHECK-NEXT:    vmov q3[3], q3[1], r5, r6
+; CHECK-NEXT:    vmov.u8 r6, q1[12]
+; CHECK-NEXT:    vmov.u8 r5, q1[13]
+; CHECK-NEXT:    vmov q0[2], q0[0], r5, r6
+; CHECK-NEXT:    vmov.u8 r5, q7[12]
+; CHECK-NEXT:    vmov q4[2], q4[0], r4, r5
+; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    vand q4, q4, q2
 ; CHECK-NEXT:    vmov r6, s0
+; CHECK-NEXT:    vmov r5, s16
+; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    vmov r3, s18
 ; CHECK-NEXT:    umull r6, r5, r5, r6
-; CHECK-NEXT:    vmov.32 q2[0], r6
-; CHECK-NEXT:    vmov r6, s2
-; CHECK-NEXT:    vmov.32 q2[1], r5
-; CHECK-NEXT:    vmov r5, s6
-; CHECK-NEXT:    umull r6, r5, r5, r6
-; CHECK-NEXT:    vmov.32 q2[2], r6
-; CHECK-NEXT:    vmov.32 q2[3], r5
-; CHECK-NEXT:    vand q0, q2, q3
-; CHECK-NEXT:    vmov r5, s0
-; CHECK-NEXT:    vmov r6, s1
+; CHECK-NEXT:    umull r3, r4, r3, r4
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r6
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r5
+; CHECK-NEXT:    vand q0, q0, q3
+; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov r3, s1
+; CHECK-NEXT:    adds.w r6, r12, r4
 ; CHECK-NEXT:    vmov r4, s3
-; CHECK-NEXT:    adds.w r5, r5, r12
-; CHECK-NEXT:    adcs r6, r3
+; CHECK-NEXT:    adc.w r5, lr, r3
 ; CHECK-NEXT:    vmov r3, s2
-; CHECK-NEXT:    adds r3, r3, r5
-; CHECK-NEXT:    adc.w r12, r6, r4
+; CHECK-NEXT:    adds r3, r3, r6
 ; CHECK-NEXT:    ubfx r6, r2, #8, #1
-; CHECK-NEXT:    rsbs r6, r6, #0
 ; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q3[0], r6
-; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q3[1], r6
+; CHECK-NEXT:    rsb.w r6, r6, #0
+; CHECK-NEXT:    rsb.w r2, r2, #0
+; CHECK-NEXT:    adc.w r12, r5, r4
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r6
+; CHECK-NEXT:    vmov.u8 r5, q7[15]
+; CHECK-NEXT:    vmov q3[3], q3[1], r2, r6
+; CHECK-NEXT:    vmov.u8 r2, q1[14]
+; CHECK-NEXT:    vmov.u8 r6, q1[15]
+; CHECK-NEXT:    vmov q0[2], q0[0], r6, r2
 ; CHECK-NEXT:    vmov.u8 r6, q7[14]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.32 q1[0], r6
-; CHECK-NEXT:    vmov.32 q3[3], r2
-; CHECK-NEXT:    vmov.u8 r2, q6[14]
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q6[15]
-; CHECK-NEXT:    vmov.u8 r6, q7[15]
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vmov.32 q1[2], r6
-; CHECK-NEXT:    vand q0, q0, q5
-; CHECK-NEXT:    vand q1, q1, q5
+; CHECK-NEXT:    vmov q1[2], q1[0], r5, r6
+; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    vand q1, q1, q2
 ; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    vmov r6, s4
+; CHECK-NEXT:    vmov r5, s2
+; CHECK-NEXT:    vmov r4, s6
 ; CHECK-NEXT:    umull r2, r6, r6, r2
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    vmov.32 q2[1], r6
-; CHECK-NEXT:    vmov r6, s6
-; CHECK-NEXT:    umull r2, r6, r6, r2
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r6
-; CHECK-NEXT:    vand q0, q2, q3
+; CHECK-NEXT:    umull r5, r4, r4, r5
+; CHECK-NEXT:    vmov q0[2], q0[0], r5, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r6
+; CHECK-NEXT:    vand q0, q0, q3
 ; CHECK-NEXT:    vmov r6, s0
 ; CHECK-NEXT:    vmov r2, s1
 ; CHECK-NEXT:    vmov r5, s2
 ; CHECK-NEXT:    adds r3, r3, r6
 ; CHECK-NEXT:    vmov r6, s3
 ; CHECK-NEXT:    adc.w r2, r2, r12
 ; CHECK-NEXT:    adds r3, r3, r5
 ; CHECK-NEXT:    adcs r2, r6
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
-; CHECK-NEXT:    add sp, #80
+; CHECK-NEXT:    add sp, #32
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    pop {r4, r5, r6, pc}
 entry:
   %c = icmp eq <16 x i8> %b, zeroinitializer
   %xx = zext <16 x i8> %x to <16 x i64>
   %yy = zext <16 x i8> %y to <16 x i64>
   %m = mul <16 x i64> %xx, %yy
   %s = select <16 x i1> %c, <16 x i64> %m, <16 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v16i64(<16 x i64> %s)
   %r = add i64 %z, %a
   ret i64 %r
 }
 
 define arm_aapcs_vfpcc i64 @add_v16i8_v16i64_acc_sext(<16 x i8> %x, <16 x i8> %y, <16 x i8> %b, i64 %a) {
 ; CHECK-LABEL: add_v16i8_v16i64_acc_sext:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-NEXT:    push {r4, r5, r7, lr}
+; CHECK-NEXT:    .save {r4, r5, r6, lr}
+; CHECK-NEXT:    push {r4, r5, r6, lr}
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vcmp.i8 eq, q2, zr
 ; CHECK-NEXT:    vmov.i8 q2, #0x0
 ; CHECK-NEXT:    vmov.i8 q3, #0xff
+; CHECK-NEXT:    vmov.u8 r4, q0[1]
 ; CHECK-NEXT:    vpsel q4, q3, q2
+; CHECK-NEXT:    sxtb r4, r4
 ; CHECK-NEXT:    vmov.u8 r2, q4[0]
+; CHECK-NEXT:    vmov.u8 r5, q0[3]
 ; CHECK-NEXT:    vmov.16 q5[0], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[1]
 ; CHECK-NEXT:    vmov.16 q5[1], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[2]
 ; CHECK-NEXT:    vmov.16 q5[2], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[3]
 ; CHECK-NEXT:    vmov.16 q5[3], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[4]
 ; CHECK-NEXT:    vmov.16 q5[4], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[5]
 ; CHECK-NEXT:    vmov.16 q5[5], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[6]
 ; CHECK-NEXT:    vmov.16 q5[6], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[7]
 ; CHECK-NEXT:    vmov.16 q5[7], r2
+; CHECK-NEXT:    sxtb r5, r5
 ; CHECK-NEXT:    vcmp.i16 ne, q5, zr
 ; CHECK-NEXT:    vpsel q5, q3, q2
 ; CHECK-NEXT:    vmov.u16 r2, q5[0]
-; CHECK-NEXT:    vmov.32 q6[0], r2
+; CHECK-NEXT:    vmov.u16 r3, q5[2]
+; CHECK-NEXT:    vmov q6[2], q6[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q5[1]
-; CHECK-NEXT:    vmov.32 q6[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q5[2]
-; CHECK-NEXT:    vmov.32 q6[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q5[3]
-; CHECK-NEXT:    vmov.32 q6[3], r2
+; CHECK-NEXT:    vmov.u16 r3, q5[3]
+; CHECK-NEXT:    vmov q6[3], q6[1], r3, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q6, zr
-; CHECK-NEXT:    vmrs r2, p0
-; CHECK-NEXT:    and r3, r2, #1
+; CHECK-NEXT:    vmrs r12, p0
+; CHECK-NEXT:    and r3, r12, #1
+; CHECK-NEXT:    ubfx r2, r12, #4, #1
 ; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q6[0], r3
-; CHECK-NEXT:    vmov.32 q6[1], r3
-; CHECK-NEXT:    ubfx r3, r2, #4, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
-; CHECK-NEXT:    vmov.32 q6[2], r3
-; CHECK-NEXT:    vmov.32 q6[3], r3
-; CHECK-NEXT:    vmov.u8 r3, q1[0]
-; CHECK-NEXT:    sxtb.w r12, r3
+; CHECK-NEXT:    rsbs r2, r2, #0
+; CHECK-NEXT:    vmov q6[2], q6[0], r2, r3
+; CHECK-NEXT:    vmov q6[3], q6[1], r2, r3
+; CHECK-NEXT:    vmov.u8 r2, q1[0]
 ; CHECK-NEXT:    vmov.u8 r3, q0[0]
+; CHECK-NEXT:    sxtb r2, r2
 ; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r3, r12, r3, r12
-; CHECK-NEXT:    vmov.32 q7[0], r3
+; CHECK-NEXT:    smull r2, lr, r3, r2
 ; CHECK-NEXT:    vmov.u8 r3, q1[1]
-; CHECK-NEXT:    vmov.32 q7[1], r12
-; CHECK-NEXT:    sxtb.w r12, r3
-; CHECK-NEXT:    vmov.u8 r3, q0[1]
 ; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r3, r12, r3, r12
-; CHECK-NEXT:    vmov.32 q7[2], r3
-; CHECK-NEXT:    vmov.32 q7[3], r12
+; CHECK-NEXT:    smull r3, r4, r4, r3
+; CHECK-NEXT:    vmov q7[2], q7[0], r3, r2
+; CHECK-NEXT:    vmov q7[3], q7[1], r4, lr
 ; CHECK-NEXT:    vand q6, q7, q6
-; CHECK-NEXT:    vmov r3, s26
-; CHECK-NEXT:    vmov r4, s24
-; CHECK-NEXT:    vmov r12, s27
-; CHECK-NEXT:    vmov r5, s25
-; CHECK-NEXT:    adds.w lr, r4, r3
-; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsb.w r3, r3, #0
-; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q6[0], r3
+; CHECK-NEXT:    vmov r4, s26
+; CHECK-NEXT:    vmov r2, s24
+; CHECK-NEXT:    vmov lr, s27
+; CHECK-NEXT:    vmov r3, s25
+; CHECK-NEXT:    adds r6, r2, r4
+; CHECK-NEXT:    ubfx r4, r12, #8, #1
+; CHECK-NEXT:    ubfx r2, r12, #12, #1
+; CHECK-NEXT:    rsb.w r4, r4, #0
 ; CHECK-NEXT:    rsb.w r2, r2, #0
-; CHECK-NEXT:    vmov.32 q6[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[2]
-; CHECK-NEXT:    vmov.32 q6[2], r2
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    vmov.32 q6[3], r2
+; CHECK-NEXT:    adc.w lr, lr, r3
+; CHECK-NEXT:    vmov q6[2], q6[0], r2, r4
+; CHECK-NEXT:    vmov.u8 r3, q1[3]
+; CHECK-NEXT:    vmov q6[3], q6[1], r2, r4
 ; CHECK-NEXT:    vmov.u8 r2, q1[2]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    adc.w r12, r12, r5
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q7[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[3]
-; CHECK-NEXT:    vmov.32 q7[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[3]
-; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    vmov.u8 r4, q0[2]
 ; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q7[2], r2
-; CHECK-NEXT:    vmov.32 q7[3], r3
+; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    smull r2, r4, r4, r2
+; CHECK-NEXT:    smull r3, r5, r5, r3
+; CHECK-NEXT:    vmov q7[2], q7[0], r3, r2
+; CHECK-NEXT:    vmov q7[3], q7[1], r5, r4
+; CHECK-NEXT:    vmov.u8 r4, q1[5]
 ; CHECK-NEXT:    vand q6, q7, q6
+; CHECK-NEXT:    sxtb r4, r4
 ; CHECK-NEXT:    vmov r3, s24
 ; CHECK-NEXT:    vmov r2, s25
-; CHECK-NEXT:    vmov r4, s26
-; CHECK-NEXT:    vmov r5, s27
-; CHECK-NEXT:    adds.w r3, r3, lr
-; CHECK-NEXT:    adc.w r2, r2, r12
-; CHECK-NEXT:    adds.w r12, r3, r4
-; CHECK-NEXT:    vmov.u8 r4, q0[4]
-; CHECK-NEXT:    adc.w r3, r2, r5
+; CHECK-NEXT:    vmov r5, s26
+; CHECK-NEXT:    adds r3, r3, r6
+; CHECK-NEXT:    vmov r6, s27
+; CHECK-NEXT:    adc.w r2, r2, lr
+; CHECK-NEXT:    adds.w r12, r3, r5
+; CHECK-NEXT:    vmov.u8 r3, q0[5]
+; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    smull r3, r4, r3, r4
+; CHECK-NEXT:    adc.w lr, r2, r6
 ; CHECK-NEXT:    vmov.u16 r2, q5[4]
-; CHECK-NEXT:    vmov.32 q6[0], r2
+; CHECK-NEXT:    vmov.u16 r6, q5[6]
+; CHECK-NEXT:    vmov q6[2], q6[0], r6, r2
 ; CHECK-NEXT:    vmov.u16 r2, q5[5]
-; CHECK-NEXT:    vmov.32 q6[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q5[6]
-; CHECK-NEXT:    vmov.32 q6[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q5[7]
-; CHECK-NEXT:    vmov.32 q6[3], r2
-; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    vmov.u16 r6, q5[7]
+; CHECK-NEXT:    vmov q6[3], q6[1], r6, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q6, zr
 ; CHECK-NEXT:    vmrs r2, p0
-; CHECK-NEXT:    and r5, r2, #1
-; CHECK-NEXT:    rsbs r5, r5, #0
-; CHECK-NEXT:    vmov.32 q5[0], r5
-; CHECK-NEXT:    vmov.32 q5[1], r5
+; CHECK-NEXT:    and r6, r2, #1
 ; CHECK-NEXT:    ubfx r5, r2, #4, #1
+; CHECK-NEXT:    rsbs r6, r6, #0
 ; CHECK-NEXT:    rsbs r5, r5, #0
-; CHECK-NEXT:    vmov.32 q5[2], r5
-; CHECK-NEXT:    vmov.32 q5[3], r5
-; CHECK-NEXT:    vmov.u8 r5, q1[4]
-; CHECK-NEXT:    sxtb r5, r5
-; CHECK-NEXT:    smull r5, r4, r4, r5
-; CHECK-NEXT:    vmov.32 q6[0], r5
-; CHECK-NEXT:    vmov.u8 r5, q1[5]
-; CHECK-NEXT:    vmov.32 q6[1], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[5]
+; CHECK-NEXT:    vmov q5[2], q5[0], r5, r6
+; CHECK-NEXT:    vmov q5[3], q5[1], r5, r6
+; CHECK-NEXT:    vmov.u8 r6, q1[4]
+; CHECK-NEXT:    vmov.u8 r5, q0[4]
+; CHECK-NEXT:    sxtb r6, r6
 ; CHECK-NEXT:    sxtb r5, r5
-; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    smull r5, r4, r4, r5
-; CHECK-NEXT:    vmov.32 q6[2], r5
-; CHECK-NEXT:    vmov.32 q6[3], r4
+; CHECK-NEXT:    smull r6, r5, r5, r6
+; CHECK-NEXT:    vmov q6[2], q6[0], r3, r6
+; CHECK-NEXT:    vmov q6[3], q6[1], r4, r5
 ; CHECK-NEXT:    vand q5, q6, q5
 ; CHECK-NEXT:    vmov r4, s20
-; CHECK-NEXT:    vmov r5, s21
-; CHECK-NEXT:    adds.w r12, r12, r4
+; CHECK-NEXT:    vmov r3, s21
+; CHECK-NEXT:    vmov r5, s23
+; CHECK-NEXT:    adds.w r6, r12, r4
 ; CHECK-NEXT:    vmov r4, s22
-; CHECK-NEXT:    adcs r5, r3
-; CHECK-NEXT:    vmov r3, s23
-; CHECK-NEXT:    adds.w r4, r4, r12
-; CHECK-NEXT:    adc.w r12, r5, r3
-; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
+; CHECK-NEXT:    adc.w r3, r3, lr
+; CHECK-NEXT:    adds r6, r6, r4
+; CHECK-NEXT:    vmov.u8 r4, q1[7]
+; CHECK-NEXT:    adc.w r12, r3, r5
+; CHECK-NEXT:    ubfx r5, r2, #8, #1
 ; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q5[0], r3
+; CHECK-NEXT:    rsbs r5, r5, #0
 ; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q5[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[6]
-; CHECK-NEXT:    vmov.32 q5[2], r2
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    vmov.32 q5[3], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[6]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q6[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[7]
-; CHECK-NEXT:    vmov.32 q6[1], r3
 ; CHECK-NEXT:    vmov.u8 r3, q0[7]
-; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    vmov q5[2], q5[0], r2, r5
+; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    vmov q5[3], q5[1], r2, r5
+; CHECK-NEXT:    vmov.u8 r2, q1[6]
+; CHECK-NEXT:    vmov.u8 r5, q0[6]
 ; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q6[2], r2
-; CHECK-NEXT:    vmov.32 q6[3], r3
+; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    sxtb r5, r5
+; CHECK-NEXT:    smull r2, r5, r5, r2
+; CHECK-NEXT:    smull r3, r4, r3, r4
+; CHECK-NEXT:    vmov q6[2], q6[0], r3, r2
+; CHECK-NEXT:    vmov q6[3], q6[1], r4, r5
+; CHECK-NEXT:    vmov.u8 r4, q1[9]
 ; CHECK-NEXT:    vand q5, q6, q5
+; CHECK-NEXT:    sxtb r4, r4
 ; CHECK-NEXT:    vmov r3, s20
 ; CHECK-NEXT:    vmov r2, s21
-; CHECK-NEXT:    vmov r5, s23
-; CHECK-NEXT:    adds r3, r3, r4
-; CHECK-NEXT:    vmov r4, s22
+; CHECK-NEXT:    vmov r5, s22
+; CHECK-NEXT:    adds r3, r3, r6
+; CHECK-NEXT:    vmov r6, s23
 ; CHECK-NEXT:    adc.w r2, r2, r12
-; CHECK-NEXT:    adds.w r12, r3, r4
-; CHECK-NEXT:    vmov.u8 r4, q0[8]
-; CHECK-NEXT:    adc.w r3, r2, r5
+; CHECK-NEXT:    adds.w r12, r3, r5
+; CHECK-NEXT:    vmov.u8 r3, q0[9]
+; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    smull r3, r4, r3, r4
+; CHECK-NEXT:    adc.w lr, r2, r6
 ; CHECK-NEXT:    vmov.u8 r2, q4[8]
 ; CHECK-NEXT:    vmov.16 q5[0], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[9]
 ; CHECK-NEXT:    vmov.16 q5[1], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[10]
 ; CHECK-NEXT:    vmov.16 q5[2], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[11]
 ; CHECK-NEXT:    vmov.16 q5[3], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[12]
 ; CHECK-NEXT:    vmov.16 q5[4], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[13]
 ; CHECK-NEXT:    vmov.16 q5[5], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[14]
 ; CHECK-NEXT:    vmov.16 q5[6], r2
 ; CHECK-NEXT:    vmov.u8 r2, q4[15]
 ; CHECK-NEXT:    vmov.16 q5[7], r2
-; CHECK-NEXT:    sxtb r4, r4
 ; CHECK-NEXT:    vcmp.i16 ne, q5, zr
 ; CHECK-NEXT:    vpsel q2, q3, q2
 ; CHECK-NEXT:    vmov.u16 r2, q2[0]
-; CHECK-NEXT:    vmov.32 q3[0], r2
+; CHECK-NEXT:    vmov.u16 r6, q2[2]
+; CHECK-NEXT:    vmov q3[2], q3[0], r6, r2
 ; CHECK-NEXT:    vmov.u16 r2, q2[1]
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[2]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[3]
-; CHECK-NEXT:    vmov.32 q3[3], r2
+; CHECK-NEXT:    vmov.u16 r6, q2[3]
+; CHECK-NEXT:    vmov q3[3], q3[1], r6, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q3, zr
 ; CHECK-NEXT:    vmrs r2, p0
-; CHECK-NEXT:    and r5, r2, #1
-; CHECK-NEXT:    rsbs r5, r5, #0
-; CHECK-NEXT:    vmov.32 q3[0], r5
-; CHECK-NEXT:    vmov.32 q3[1], r5
+; CHECK-NEXT:    and r6, r2, #1
 ; CHECK-NEXT:    ubfx r5, r2, #4, #1
+; CHECK-NEXT:    rsbs r6, r6, #0
 ; CHECK-NEXT:    rsbs r5, r5, #0
-; CHECK-NEXT:    vmov.32 q3[2], r5
-; CHECK-NEXT:    vmov.32 q3[3], r5
-; CHECK-NEXT:    vmov.u8 r5, q1[8]
+; CHECK-NEXT:    vmov q3[2], q3[0], r5, r6
+; CHECK-NEXT:    vmov q3[3], q3[1], r5, r6
+; CHECK-NEXT:    vmov.u8 r6, q1[8]
+; CHECK-NEXT:    vmov.u8 r5, q0[8]
+; CHECK-NEXT:    sxtb r6, r6
 ; CHECK-NEXT:    sxtb r5, r5
-; CHECK-NEXT:    smull r5, r4, r4, r5
-; CHECK-NEXT:    vmov.32 q4[0], r5
-; CHECK-NEXT:    vmov.u8 r5, q1[9]
-; CHECK-NEXT:    vmov.32 q4[1], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[9]
-; CHECK-NEXT:    sxtb r5, r5
-; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    smull r5, r4, r4, r5
-; CHECK-NEXT:    vmov.32 q4[2], r5
-; CHECK-NEXT:    vmov.32 q4[3], r4
+; CHECK-NEXT:    smull r6, r5, r5, r6
+; CHECK-NEXT:    vmov q4[2], q4[0], r3, r6
+; CHECK-NEXT:    vmov q4[3], q4[1], r4, r5
 ; CHECK-NEXT:    vand q3, q4, q3
 ; CHECK-NEXT:    vmov r4, s12
-; CHECK-NEXT:    vmov r5, s13
-; CHECK-NEXT:    adds.w r12, r12, r4
+; CHECK-NEXT:    vmov r3, s13
+; CHECK-NEXT:    vmov r5, s15
+; CHECK-NEXT:    adds.w r6, r12, r4
 ; CHECK-NEXT:    vmov r4, s14
-; CHECK-NEXT:    adcs r5, r3
-; CHECK-NEXT:    vmov r3, s15
-; CHECK-NEXT:    adds.w r4, r4, r12
-; CHECK-NEXT:    adc.w r12, r5, r3
-; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
+; CHECK-NEXT:    adc.w r3, r3, lr
+; CHECK-NEXT:    adds r6, r6, r4
+; CHECK-NEXT:    vmov.u8 r4, q1[11]
+; CHECK-NEXT:    adc.w r12, r3, r5
+; CHECK-NEXT:    ubfx r5, r2, #8, #1
 ; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q3[0], r3
+; CHECK-NEXT:    rsbs r5, r5, #0
 ; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q3[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[10]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    vmov.32 q3[3], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[10]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q4[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[11]
-; CHECK-NEXT:    vmov.32 q4[1], r3
 ; CHECK-NEXT:    vmov.u8 r3, q0[11]
-; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r5
+; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    vmov q3[3], q3[1], r2, r5
+; CHECK-NEXT:    vmov.u8 r2, q1[10]
+; CHECK-NEXT:    vmov.u8 r5, q0[10]
 ; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q4[2], r2
-; CHECK-NEXT:    vmov.32 q4[3], r3
+; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    sxtb r5, r5
+; CHECK-NEXT:    smull r2, r5, r5, r2
+; CHECK-NEXT:    smull r3, r4, r3, r4
+; CHECK-NEXT:    vmov q4[2], q4[0], r3, r2
+; CHECK-NEXT:    vmov q4[3], q4[1], r4, r5
+; CHECK-NEXT:    vmov.u8 r4, q1[13]
 ; CHECK-NEXT:    vand q3, q4, q3
+; CHECK-NEXT:    sxtb r4, r4
 ; CHECK-NEXT:    vmov r3, s12
 ; CHECK-NEXT:    vmov r2, s13
-; CHECK-NEXT:    vmov r5, s15
-; CHECK-NEXT:    adds r3, r3, r4
-; CHECK-NEXT:    vmov r4, s14
+; CHECK-NEXT:    vmov r5, s14
+; CHECK-NEXT:    adds r3, r3, r6
+; CHECK-NEXT:    vmov r6, s15
 ; CHECK-NEXT:    adc.w r2, r2, r12
-; CHECK-NEXT:    adds.w r12, r3, r4
-; CHECK-NEXT:    vmov.u8 r4, q0[12]
-; CHECK-NEXT:    adc.w r3, r2, r5
+; CHECK-NEXT:    adds.w r12, r3, r5
+; CHECK-NEXT:    vmov.u8 r3, q0[13]
+; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    smull r3, r4, r3, r4
+; CHECK-NEXT:    adc.w lr, r2, r6
 ; CHECK-NEXT:    vmov.u16 r2, q2[4]
-; CHECK-NEXT:    vmov.32 q3[0], r2
+; CHECK-NEXT:    vmov.u16 r6, q2[6]
+; CHECK-NEXT:    vmov q3[2], q3[0], r6, r2
 ; CHECK-NEXT:    vmov.u16 r2, q2[5]
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[6]
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[7]
-; CHECK-NEXT:    vmov.32 q3[3], r2
-; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    vmov.u16 r6, q2[7]
+; CHECK-NEXT:    vmov q3[3], q3[1], r6, r2
 ; CHECK-NEXT:    vcmp.i32 ne, q3, zr
 ; CHECK-NEXT:    vmrs r2, p0
-; CHECK-NEXT:    and r5, r2, #1
-; CHECK-NEXT:    rsbs r5, r5, #0
-; CHECK-NEXT:    vmov.32 q2[0], r5
-; CHECK-NEXT:    vmov.32 q2[1], r5
+; CHECK-NEXT:    and r6, r2, #1
 ; CHECK-NEXT:    ubfx r5, r2, #4, #1
+; CHECK-NEXT:    rsbs r6, r6, #0
 ; CHECK-NEXT:    rsbs r5, r5, #0
-; CHECK-NEXT:    vmov.32 q2[2], r5
-; CHECK-NEXT:    vmov.32 q2[3], r5
-; CHECK-NEXT:    vmov.u8 r5, q1[12]
-; CHECK-NEXT:    sxtb r5, r5
-; CHECK-NEXT:    smull r5, r4, r4, r5
-; CHECK-NEXT:    vmov.32 q3[0], r5
-; CHECK-NEXT:    vmov.u8 r5, q1[13]
-; CHECK-NEXT:    vmov.32 q3[1], r4
-; CHECK-NEXT:    vmov.u8 r4, q0[13]
+; CHECK-NEXT:    vmov q2[2], q2[0], r5, r6
+; CHECK-NEXT:    vmov q2[3], q2[1], r5, r6
+; CHECK-NEXT:    vmov.u8 r6, q1[12]
+; CHECK-NEXT:    vmov.u8 r5, q0[12]
+; CHECK-NEXT:    sxtb r6, r6
 ; CHECK-NEXT:    sxtb r5, r5
-; CHECK-NEXT:    sxtb r4, r4
-; CHECK-NEXT:    smull r5, r4, r4, r5
-; CHECK-NEXT:    vmov.32 q3[2], r5
-; CHECK-NEXT:    vmov.32 q3[3], r4
+; CHECK-NEXT:    smull r6, r5, r5, r6
+; CHECK-NEXT:    vmov q3[2], q3[0], r3, r6
+; CHECK-NEXT:    vmov q3[3], q3[1], r4, r5
 ; CHECK-NEXT:    vand q2, q3, q2
 ; CHECK-NEXT:    vmov r4, s8
-; CHECK-NEXT:    vmov r5, s9
-; CHECK-NEXT:    adds.w r12, r12, r4
+; CHECK-NEXT:    vmov r3, s9
+; CHECK-NEXT:    vmov r5, s11
+; CHECK-NEXT:    adds.w r6, r12, r4
 ; CHECK-NEXT:    vmov r4, s10
-; CHECK-NEXT:    adcs r5, r3
-; CHECK-NEXT:    vmov r3, s11
-; CHECK-NEXT:    adds.w r4, r4, r12
-; CHECK-NEXT:    adc.w r12, r5, r3
-; CHECK-NEXT:    ubfx r3, r2, #8, #1
-; CHECK-NEXT:    rsbs r3, r3, #0
+; CHECK-NEXT:    adc.w r3, r3, lr
+; CHECK-NEXT:    adds r6, r6, r4
+; CHECK-NEXT:    vmov.u8 r4, q1[15]
+; CHECK-NEXT:    adc.w r12, r3, r5
+; CHECK-NEXT:    ubfx r5, r2, #8, #1
 ; CHECK-NEXT:    ubfx r2, r2, #12, #1
-; CHECK-NEXT:    vmov.32 q2[0], r3
+; CHECK-NEXT:    rsbs r5, r5, #0
 ; CHECK-NEXT:    rsbs r2, r2, #0
-; CHECK-NEXT:    vmov.32 q2[1], r3
-; CHECK-NEXT:    vmov.u8 r3, q0[14]
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    vmov.32 q2[3], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[14]
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov.u8 r2, q1[15]
-; CHECK-NEXT:    vmov.32 q3[1], r3
 ; CHECK-NEXT:    vmov.u8 r3, q0[15]
-; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r5
+; CHECK-NEXT:    sxtb r4, r4
+; CHECK-NEXT:    vmov q2[3], q2[1], r2, r5
+; CHECK-NEXT:    vmov.u8 r2, q1[14]
+; CHECK-NEXT:    vmov.u8 r5, q0[14]
 ; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.32 q3[3], r3
-; CHECK-NEXT:    vand q0, q3, q2
+; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    sxtb r5, r5
+; CHECK-NEXT:    smull r2, r5, r5, r2
+; CHECK-NEXT:    smull r3, r4, r3, r4
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r5
+; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r2, s1
-; CHECK-NEXT:    vmov r5, s3
-; CHECK-NEXT:    adds r3, r3, r4
-; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    vmov r5, s2
+; CHECK-NEXT:    adds r3, r3, r6
+; CHECK-NEXT:    vmov r6, s3
 ; CHECK-NEXT:    adc.w r2, r2, r12
-; CHECK-NEXT:    adds r3, r3, r4
-; CHECK-NEXT:    adcs r2, r5
+; CHECK-NEXT:    adds r3, r3, r5
+; CHECK-NEXT:    adcs r2, r6
 ; CHECK-NEXT:    adds r0, r0, r3
 ; CHECK-NEXT:    adcs r1, r2
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    pop {r4, r5, r7, pc}
+; CHECK-NEXT:    pop {r4, r5, r6, pc}
 entry:
   %c = icmp eq <16 x i8> %b, zeroinitializer
   %xx = sext <16 x i8> %x to <16 x i64>
   %yy = sext <16 x i8> %y to <16 x i64>
   %m = mul <16 x i64> %xx, %yy
   %s = select <16 x i1> %c, <16 x i64> %m, <16 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v16i64(<16 x i64> %s)
   %r = add i64 %z, %a
   ret i64 %r
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i8_v2i64_acc_zext(<2 x i8> %x, <2 x i8> %y, <2 x i8> %b, i64 %a) {
 ; CHECK-LABEL: add_v2i8_v2i64_acc_zext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
-; CHECK-NEXT:    .vsave {d8, d9}
-; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    vmov.i64 q3, #0xff
 ; CHECK-NEXT:    vand q1, q1, q3
-; CHECK-NEXT:    vand q4, q0, q3
+; CHECK-NEXT:    vand q0, q0, q3
 ; CHECK-NEXT:    vmov r2, s4
-; CHECK-NEXT:    vmov r3, s16
-; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    vmov.32 q0[1], r3
-; CHECK-NEXT:    vmov r3, s18
+; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    umull lr, r12, r3, r2
+; CHECK-NEXT:    vmov r3, s6
+; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vand q1, q2, q3
-; CHECK-NEXT:    umull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q0[2], r2
+; CHECK-NEXT:    umull r2, r3, r2, r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, lr
 ; CHECK-NEXT:    vmov r2, s4
-; CHECK-NEXT:    vmov.32 q0[3], r3
-; CHECK-NEXT:    cmp r2, #0
-; CHECK-NEXT:    cset r2, eq
-; CHECK-NEXT:    tst.w r2, #1
-; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r12
+; CHECK-NEXT:    vmov r3, s6
 ; CHECK-NEXT:    cmp r2, #0
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r2
-; CHECK-NEXT:    vand q0, q0, q2
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    cset r3, eq
+; CHECK-NEXT:    tst.w r3, #1
+; CHECK-NEXT:    csetm r3, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r2
+; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r12, s3
 ; CHECK-NEXT:    vmov lr, s1
 ; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    adc.w r3, lr, r12
 ; CHECK-NEXT:    adds r0, r0, r2
 ; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    vpop {d8, d9}
 ; CHECK-NEXT:    pop {r7, pc}
 entry:
   %c = icmp eq <2 x i8> %b, zeroinitializer
   %xx = zext <2 x i8> %x to <2 x i64>
   %yy = zext <2 x i8> %y to <2 x i64>
   %m = mul <2 x i64> %xx, %yy
   %s = select <2 x i1> %c, <2 x i64> %m, <2 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> %s)
   %r = add i64 %z, %a
   ret i64 %r
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i8_v2i64_acc_sext(<2 x i8> %x, <2 x i8> %y, <2 x i8> %b, i64 %a) {
 ; CHECK-LABEL: add_v2i8_v2i64_acc_sext:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
 ; CHECK-NEXT:    vmov.i32 q3, #0xff
-; CHECK-NEXT:    vmov r3, s0
-; CHECK-NEXT:    vand q3, q2, q3
-; CHECK-NEXT:    vmov r2, s12
-; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    cmp r2, #0
-; CHECK-NEXT:    cset r2, eq
-; CHECK-NEXT:    tst.w r2, #1
-; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov r2, s14
+; CHECK-NEXT:    vand q2, q2, q3
+; CHECK-NEXT:    vmov r2, s8
+; CHECK-NEXT:    vmov r3, s10
 ; CHECK-NEXT:    cmp r2, #0
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r2
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    cset r3, eq
+; CHECK-NEXT:    tst.w r3, #1
+; CHECK-NEXT:    csetm r3, ne
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r2
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r2
 ; CHECK-NEXT:    vmov r2, s4
-; CHECK-NEXT:    sxtb r2, r2
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    vmov.32 q3[1], r3
-; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    sxtb r2, r2
 ; CHECK-NEXT:    sxtb r3, r3
-; CHECK-NEXT:    smull r2, r3, r3, r2
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov.32 q3[3], r3
-; CHECK-NEXT:    vand q0, q3, q2
+; CHECK-NEXT:    smull lr, r12, r3, r2
+; CHECK-NEXT:    vmov r3, s6
+; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    sxtb r3, r3
+; CHECK-NEXT:    sxtb r2, r2
+; CHECK-NEXT:    smull r2, r3, r2, r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, lr
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r12
+; CHECK-NEXT:    vand q0, q0, q2
 ; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov r12, s3
 ; CHECK-NEXT:    vmov lr, s1
 ; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    adc.w r3, lr, r12
 ; CHECK-NEXT:    adds r0, r0, r2
 ; CHECK-NEXT:    adcs r1, r3
 ; CHECK-NEXT:    pop {r7, pc}
 entry:
   %c = icmp eq <2 x i8> %b, zeroinitializer
   %xx = sext <2 x i8> %x to <2 x i64>
   %yy = sext <2 x i8> %y to <2 x i64>
   %m = mul <2 x i64> %xx, %yy
   %s = select <2 x i1> %c, <2 x i64> %m, <2 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> %s)
   %r = add i64 %z, %a
   ret i64 %r
 }
 
 define arm_aapcs_vfpcc i64 @add_v2i64_v2i64_acc(<2 x i64> %x, <2 x i64> %y, <2 x i64> %b, i64 %a) {
 ; CHECK-LABEL: add_v2i64_v2i64_acc:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, lr}
-; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .save {r4, r5, r6, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r6, r7, lr}
 ; CHECK-NEXT:    vmov r2, s4
 ; CHECK-NEXT:    vmov r3, s0
-; CHECK-NEXT:    vmov r4, s5
+; CHECK-NEXT:    vmov r4, s6
+; CHECK-NEXT:    vmov r5, s2
 ; CHECK-NEXT:    umull r12, lr, r3, r2
-; CHECK-NEXT:    mla r3, r3, r4, lr
-; CHECK-NEXT:    vmov r4, s1
-; CHECK-NEXT:    vmov.32 q3[0], r12
-; CHECK-NEXT:    mla r2, r4, r2, r3
-; CHECK-NEXT:    vmov r3, s2
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    umull r4, r12, r3, r2
-; CHECK-NEXT:    vmov.32 q3[2], r4
-; CHECK-NEXT:    vmov r4, s7
-; CHECK-NEXT:    mla r3, r3, r4, r12
-; CHECK-NEXT:    vmov r4, s3
-; CHECK-NEXT:    mla r2, r4, r2, r3
-; CHECK-NEXT:    vmov r3, s8
-; CHECK-NEXT:    vmov.32 q3[3], r2
+; CHECK-NEXT:    umull r6, r7, r5, r4
+; CHECK-NEXT:    vmov q3[2], q3[0], r6, r12
+; CHECK-NEXT:    vmov r6, s5
+; CHECK-NEXT:    mla r3, r3, r6, lr
+; CHECK-NEXT:    vmov r6, s1
+; CHECK-NEXT:    mla r2, r6, r2, r3
+; CHECK-NEXT:    vmov r3, s7
+; CHECK-NEXT:    mla r3, r5, r3, r7
+; CHECK-NEXT:    vmov r7, s3
+; CHECK-NEXT:    mla r3, r7, r4, r3
+; CHECK-NEXT:    vmov r7, s10
+; CHECK-NEXT:    vmov q3[3], q3[1], r3, r2
 ; CHECK-NEXT:    vmov r2, s9
+; CHECK-NEXT:    vmov r3, s8
 ; CHECK-NEXT:    orrs r2, r3
-; CHECK-NEXT:    vmov r3, s10
-; CHECK-NEXT:    cset r2, eq
-; CHECK-NEXT:    tst.w r2, #1
-; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov r2, s11
-; CHECK-NEXT:    orrs r2, r3
+; CHECK-NEXT:    vmov r3, s11
 ; CHECK-NEXT:    cset r2, eq
 ; CHECK-NEXT:    tst.w r2, #1
 ; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vmov.32 q0[3], r2
+; CHECK-NEXT:    orrs r3, r7
+; CHECK-NEXT:    cset r3, eq
+; CHECK-NEXT:    tst.w r3, #1
+; CHECK-NEXT:    csetm r3, ne
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r2
 ; CHECK-NEXT:    vand q0, q3, q0
-; CHECK-NEXT:    vmov r4, s2
-; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    vmov r12, s3
+; CHECK-NEXT:    vmov r7, s2
+; CHECK-NEXT:    vmov r6, s0
+; CHECK-NEXT:    vmov r2, s3
 ; CHECK-NEXT:    vmov r3, s1
-; CHECK-NEXT:    adds r2, r2, r4
-; CHECK-NEXT:    adc.w r3, r3, r12
-; CHECK-NEXT:    adds r0, r0, r2
-; CHECK-NEXT:    adcs r1, r3
-; CHECK-NEXT:    pop {r4, pc}
+; CHECK-NEXT:    adds r7, r7, r6
+; CHECK-NEXT:    adcs r2, r3
+; CHECK-NEXT:    adds r0, r0, r7
+; CHECK-NEXT:    adcs r1, r2
+; CHECK-NEXT:    pop {r4, r5, r6, r7, pc}
 entry:
   %c = icmp eq <2 x i64> %b, zeroinitializer
   %m = mul <2 x i64> %x, %y
   %s = select <2 x i1> %c, <2 x i64> %m, <2 x i64> zeroinitializer
   %z = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> %s)
   %r = add i64 %z, %a
   ret i64 %r
 }

llvm/test/CodeGen/Thumb2/mve-vld2-post.ll

 ; CHECK-NEXT:    vldrw.u32 q2, [r0, #16]
 ; CHECK-NEXT:    vldrw.u32 q0, [r0], #32
 ; CHECK-NEXT:    vmov.f64 d2, d1
 ; CHECK-NEXT:    vmov.f32 s5, s3
 ; CHECK-NEXT:    vmov.f32 s6, s10
 ; CHECK-NEXT:    vmov.f32 s2, s8
 ; CHECK-NEXT:    vmov.f32 s7, s11
 ; CHECK-NEXT:    vmov.f32 s3, s9
-; CHECK-NEXT:    vmov r2, s6
-; CHECK-NEXT:    vmov r3, s2
-; CHECK-NEXT:    vmov r4, s4
-; CHECK-NEXT:    vmov r5, s0
-; CHECK-NEXT:    vmov r12, s7
-; CHECK-NEXT:    vmov lr, s3
+; CHECK-NEXT:    vmov r2, s4
+; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    vmov r12, s5
+; CHECK-NEXT:    vmov lr, s1
+; CHECK-NEXT:    vmov r4, s6
+; CHECK-NEXT:    vmov r5, s2
 ; CHECK-NEXT:    adds r6, r3, r2
-; CHECK-NEXT:    vmov r3, s5
-; CHECK-NEXT:    vmov r2, s1
+; CHECK-NEXT:    vmov r3, s7
+; CHECK-NEXT:    vmov r2, s3
 ; CHECK-NEXT:    adc.w r12, r12, lr
 ; CHECK-NEXT:    adds r5, r5, r4
-; CHECK-NEXT:    vmov.32 q0[0], r5
+; CHECK-NEXT:    vmov q0[2], q0[0], r5, r6
 ; CHECK-NEXT:    adcs r2, r3
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.32 q0[2], r6
-; CHECK-NEXT:    vmov.32 q0[3], r12
+; CHECK-NEXT:    vmov q0[3], q0[1], r2, r12
 ; CHECK-NEXT:    vstrw.32 q0, [r1]
 ; CHECK-NEXT:    pop {r4, r5, r6, pc}
 entry:
   %l1 = load <4 x i64>, <4 x i64>* %src, align 4
   %s1 = shufflevector <4 x i64> %l1, <4 x i64> undef, <2 x i32> <i32 0, i32 2>
   %s2 = shufflevector <4 x i64> %l1, <4 x i64> undef, <2 x i32> <i32 1, i32 3>
   %a = add <2 x i64> %s1, %s2
   store <2 x i64> %a, <2 x i64> *%dst

llvm/test/CodeGen/Thumb2/mve-vld2.ll

 ; CHECK-NEXT:    vldrw.u32 q0, [r0]
 ; CHECK-NEXT:    vldrw.u32 q2, [r0, #16]
 ; CHECK-NEXT:    vmov.f64 d2, d1
 ; CHECK-NEXT:    vmov.f32 s5, s3
 ; CHECK-NEXT:    vmov.f32 s6, s10
 ; CHECK-NEXT:    vmov.f32 s2, s8
 ; CHECK-NEXT:    vmov.f32 s3, s9
 ; CHECK-NEXT:    vmov.f32 s7, s11
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vmov r4, s0
-; CHECK-NEXT:    vmov r2, s3
-; CHECK-NEXT:    vmov r12, s7
+; CHECK-NEXT:    vmov r3, s4
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov r12, s5
+; CHECK-NEXT:    vmov r2, s1
+; CHECK-NEXT:    vmov r4, s2
 ; CHECK-NEXT:    adds.w lr, r0, r3
-; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r3, s5
+; CHECK-NEXT:    vmov r0, s6
+; CHECK-NEXT:    vmov r3, s7
 ; CHECK-NEXT:    adc.w r12, r12, r2
-; CHECK-NEXT:    vmov r2, s1
+; CHECK-NEXT:    vmov r2, s3
 ; CHECK-NEXT:    adds r0, r0, r4
-; CHECK-NEXT:    vmov.32 q0[0], r0
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, lr
 ; CHECK-NEXT:    adcs r2, r3
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.32 q0[2], lr
-; CHECK-NEXT:    vmov.32 q0[3], r12
+; CHECK-NEXT:    vmov q0[3], q0[1], r2, r12
 ; CHECK-NEXT:    vstrw.32 q0, [r1]
 ; CHECK-NEXT:    pop {r4, pc}
 entry:
   %l1 = load <4 x i64>, <4 x i64>* %src, align 4
   %s1 = shufflevector <4 x i64> %l1, <4 x i64> undef, <2 x i32> <i32 0, i32 2>
   %s2 = shufflevector <4 x i64> %l1, <4 x i64> undef, <2 x i32> <i32 1, i32 3>
   %a = add <2 x i64> %s1, %s2
   store <2 x i64> %a, <2 x i64> *%dst
   ret void
 }
 
 define void @vld2_v4i64(<8 x i64> *%src, <4 x i64> *%dst) {
 ; CHECK-LABEL: vld2_v4i64:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, lr}
-; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .save {r4, r5, r6, lr}
+; CHECK-NEXT:    push {r4, r5, r6, lr}
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11}
-; CHECK-NEXT:    vldrw.u32 q3, [r0, #32]
-; CHECK-NEXT:    vldrw.u32 q5, [r0, #48]
 ; CHECK-NEXT:    vldrw.u32 q0, [r0]
-; CHECK-NEXT:    vldrw.u32 q2, [r0, #16]
-; CHECK-NEXT:    vmov.f64 d8, d7
-; CHECK-NEXT:    vmov.f32 s17, s15
-; CHECK-NEXT:    vmov.f32 s18, s22
-; CHECK-NEXT:    vmov.f32 s14, s20
-; CHECK-NEXT:    vmov.f32 s15, s21
-; CHECK-NEXT:    vmov.f32 s19, s23
-; CHECK-NEXT:    vmov r3, s18
-; CHECK-NEXT:    vmov r0, s14
-; CHECK-NEXT:    vmov r4, s12
-; CHECK-NEXT:    vmov.f64 d2, d1
-; CHECK-NEXT:    vmov r12, s19
-; CHECK-NEXT:    vmov r2, s15
-; CHECK-NEXT:    vmov.f32 s5, s3
-; CHECK-NEXT:    vmov.f32 s6, s10
-; CHECK-NEXT:    vmov.f32 s2, s8
-; CHECK-NEXT:    vmov.f32 s7, s11
-; CHECK-NEXT:    vmov.f32 s3, s9
+; CHECK-NEXT:    vldrw.u32 q5, [r0, #16]
+; CHECK-NEXT:    vldrw.u32 q1, [r0, #32]
+; CHECK-NEXT:    vldrw.u32 q4, [r0, #48]
+; CHECK-NEXT:    vmov.f64 d4, d1
+; CHECK-NEXT:    vmov.f32 s9, s3
+; CHECK-NEXT:    vmov.f32 s10, s22
+; CHECK-NEXT:    vmov.f32 s2, s20
+; CHECK-NEXT:    vmov.f32 s11, s23
+; CHECK-NEXT:    vmov.f32 s3, s21
+; CHECK-NEXT:    vmov r3, s8
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov r12, s9
+; CHECK-NEXT:    vmov r2, s1
+; CHECK-NEXT:    vmov.f64 d6, d3
+; CHECK-NEXT:    vmov.f32 s13, s7
+; CHECK-NEXT:    vmov.f32 s14, s18
+; CHECK-NEXT:    vmov.f32 s6, s16
+; CHECK-NEXT:    vmov.f32 s7, s17
+; CHECK-NEXT:    vmov.f32 s15, s19
+; CHECK-NEXT:    vmov r4, s4
+; CHECK-NEXT:    vmov r5, s14
+; CHECK-NEXT:    vmov r6, s6
 ; CHECK-NEXT:    adds.w lr, r0, r3
-; CHECK-NEXT:    vmov r0, s16
-; CHECK-NEXT:    vmov r3, s17
+; CHECK-NEXT:    vmov r3, s12
+; CHECK-NEXT:    vmov r0, s5
 ; CHECK-NEXT:    adc.w r12, r12, r2
 ; CHECK-NEXT:    vmov r2, s13
-; CHECK-NEXT:    adds r0, r0, r4
-; CHECK-NEXT:    vmov r4, s2
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov r0, s7
-; CHECK-NEXT:    adcs r2, r3
-; CHECK-NEXT:    vmov r3, s6
-; CHECK-NEXT:    vmov.32 q3[1], r2
+; CHECK-NEXT:    adds r3, r3, r4
+; CHECK-NEXT:    vmov r4, s7
+; CHECK-NEXT:    adcs r0, r2
+; CHECK-NEXT:    vmov r2, s15
+; CHECK-NEXT:    adds r5, r5, r6
+; CHECK-NEXT:    vmov r6, s2
+; CHECK-NEXT:    vmov q3[2], q3[0], r5, r3
+; CHECK-NEXT:    adcs r2, r4
+; CHECK-NEXT:    vmov r4, s10
+; CHECK-NEXT:    vmov q3[3], q3[1], r2, r0
+; CHECK-NEXT:    vmov r0, s11
 ; CHECK-NEXT:    vmov r2, s3
-; CHECK-NEXT:    vmov.32 q3[2], lr
-; CHECK-NEXT:    vmov.32 q3[3], r12
 ; CHECK-NEXT:    vstrw.32 q3, [r1, #16]
-; CHECK-NEXT:    adds.w lr, r4, r3
-; CHECK-NEXT:    vmov r3, s0
-; CHECK-NEXT:    adc.w r12, r2, r0
-; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r2, s5
-; CHECK-NEXT:    vmov r4, s1
-; CHECK-NEXT:    adds r0, r0, r3
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    adcs r2, r4
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.32 q0[2], lr
-; CHECK-NEXT:    vmov.32 q0[3], r12
-; CHECK-NEXT:    vstrw.32 q0, [r1]
+; CHECK-NEXT:    adds r4, r4, r6
+; CHECK-NEXT:    vmov q1[2], q1[0], r4, lr
+; CHECK-NEXT:    adcs r0, r2
+; CHECK-NEXT:    vmov q1[3], q1[1], r0, r12
+; CHECK-NEXT:    vstrw.32 q1, [r1]
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-NEXT:    pop {r4, pc}
+; CHECK-NEXT:    pop {r4, r5, r6, pc}
 entry:
   %l1 = load <8 x i64>, <8 x i64>* %src, align 4
   %s1 = shufflevector <8 x i64> %l1, <8 x i64> undef, <4 x i32> <i32 0, i32 2, i32 4, i32 6>
   %s2 = shufflevector <8 x i64> %l1, <8 x i64> undef, <4 x i32> <i32 1, i32 3, i32 5, i32 7>
   %a = add <4 x i64> %s1, %s2
   store <4 x i64> %a, <4 x i64> *%dst
   ret void
 }

llvm/test/CodeGen/Thumb2/mve-vld3.ll

 }
 
 define void @vld3_v4i16(<12 x i16> *%src, <4 x i16> *%dst) {
 ; CHECK-LABEL: vld3_v4i16:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vldrw.u32 q0, [r0]
 ; CHECK-NEXT:    vldrh.u32 q3, [r0, #16]
 ; CHECK-NEXT:    vmov.u16 r2, q0[0]
-; CHECK-NEXT:    vmov r0, s14
-; CHECK-NEXT:    vmov.32 q1[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q0[3]
-; CHECK-NEXT:    vmov.32 q1[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q0[6]
-; CHECK-NEXT:    vmov.32 q1[2], r2
+; CHECK-NEXT:    vmov.u16 r3, q0[6]
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q0[1]
-; CHECK-NEXT:    vmov.32 q2[0], r2
+; CHECK-NEXT:    vmov.u16 r3, q0[7]
+; CHECK-NEXT:    vmov r0, s14
+; CHECK-NEXT:    vmov q2[2], q2[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q0[4]
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q0[7]
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vmov r0, s13
-; CHECK-NEXT:    vmov.32 q1[3], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-NEXT:    vadd.i32 q1, q1, q2
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[5]
-; CHECK-NEXT:    vmov.32 q2[1], r0
+; CHECK-NEXT:    vmov q2[3], q2[1], r0, r2
+; CHECK-NEXT:    vmov.u16 r0, q0[3]
+; CHECK-NEXT:    vmov r2, s13
+; CHECK-NEXT:    vmov q1[3], q1[1], r2, r0
 ; CHECK-NEXT:    vmov r0, s12
-; CHECK-NEXT:    vmov.32 q2[2], r0
+; CHECK-NEXT:    vmov.u16 r2, q0[2]
+; CHECK-NEXT:    vadd.i32 q1, q1, q2
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r2
 ; CHECK-NEXT:    vmov r0, s15
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    vmov.u16 r2, q0[5]
+; CHECK-NEXT:    vmov q2[3], q2[1], r0, r2
 ; CHECK-NEXT:    vadd.i32 q0, q1, q2
 ; CHECK-NEXT:    vstrh.32 q0, [r1]
 ; CHECK-NEXT:    bx lr
 entry:
   %l1 = load <12 x i16>, <12 x i16>* %src, align 4
   %s1 = shufflevector <12 x i16> %l1, <12 x i16> undef, <4 x i32> <i32 0, i32 3, i32 6, i32 9>
   %s2 = shufflevector <12 x i16> %l1, <12 x i16> undef, <4 x i32> <i32 1, i32 4, i32 7, i32 10>
   %s3 = shufflevector <12 x i16> %l1, <12 x i16> undef, <4 x i32> <i32 2, i32 5, i32 8, i32 11>
   %a1 = add <4 x i16> %s1, %s2
   %a = add <4 x i16> %a1, %s3
   store <4 x i16> %a, <4 x i16> *%dst
   ret void
 }
 
 define void @vld3_v8i16(<24 x i16> *%src, <8 x i16> *%dst) {
 ; CHECK-LABEL: vld3_v8i16:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13}
-; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13}
-; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    vldrw.u32 q3, [r0, #32]
-; CHECK-NEXT:    vldrw.u32 q2, [r0, #16]
-; CHECK-NEXT:    vmov.u16 r2, q1[2]
-; CHECK-NEXT:    vmov.u16 r0, q3[4]
+; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
+; CHECK-NEXT:    vpush {d8, d9, d10, d11}
+; CHECK-NEXT:    vldrw.u32 q0, [r0]
+; CHECK-NEXT:    vldrw.u32 q2, [r0, #32]
+; CHECK-NEXT:    vldrw.u32 q1, [r0, #16]
+; CHECK-NEXT:    vmov.u16 r2, q0[2]
+; CHECK-NEXT:    vmov.u16 r0, q2[4]
 ; CHECK-NEXT:    vmov.16 q4[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[5]
+; CHECK-NEXT:    vmov.u16 r2, q0[5]
 ; CHECK-NEXT:    vmov.16 q4[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[0]
+; CHECK-NEXT:    vmov.u16 r2, q1[0]
 ; CHECK-NEXT:    vmov.16 q5[6], r0
-; CHECK-NEXT:    vmov.u16 r0, q3[7]
+; CHECK-NEXT:    vmov.u16 r0, q2[7]
 ; CHECK-NEXT:    vmov.16 q5[7], r0
 ; CHECK-NEXT:    vmov.16 q4[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[3]
-; CHECK-NEXT:    vmov.f32 s22, s12
+; CHECK-NEXT:    vmov.u16 r2, q1[3]
+; CHECK-NEXT:    vmov.f32 s22, s8
 ; CHECK-NEXT:    vmov.16 q4[3], r2
-; CHECK-NEXT:    vmov q6, q5
-; CHECK-NEXT:    vmov.f32 s18, s11
+; CHECK-NEXT:    vmov q3, q5
+; CHECK-NEXT:    vmov.f32 s18, s7
+; CHECK-NEXT:    vmovnb.i32 q3, q4
 ; CHECK-NEXT:    vmov r2, s16
-; CHECK-NEXT:    vmovnb.i32 q6, q4
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov r2, s17
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov r0, s26
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    vmov r0, s23
-; CHECK-NEXT:    vmov.32 q0[3], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[0]
+; CHECK-NEXT:    vmov r0, s14
+; CHECK-NEXT:    vmov q3[2], q3[0], r0, r2
+; CHECK-NEXT:    vmov r0, s17
+; CHECK-NEXT:    vmov r2, s23
+; CHECK-NEXT:    vmov q3[3], q3[1], r2, r0
+; CHECK-NEXT:    vmov.u16 r0, q0[0]
 ; CHECK-NEXT:    vmov.16 q4[0], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[3]
+; CHECK-NEXT:    vmov.u16 r0, q0[3]
 ; CHECK-NEXT:    vmov.16 q4[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[6]
+; CHECK-NEXT:    vmov.u16 r0, q0[6]
 ; CHECK-NEXT:    vmov.16 q4[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[1]
+; CHECK-NEXT:    vmov.u16 r0, q1[1]
 ; CHECK-NEXT:    vmov.16 q4[3], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[4]
+; CHECK-NEXT:    vmov.u16 r0, q1[4]
 ; CHECK-NEXT:    vmov.16 q4[4], r0
-; CHECK-NEXT:    vmov.u16 r0, q3[2]
+; CHECK-NEXT:    vmov.u16 r0, q2[2]
 ; CHECK-NEXT:    vmov.16 q5[6], r0
-; CHECK-NEXT:    vmov.u16 r0, q3[5]
+; CHECK-NEXT:    vmov.u16 r0, q2[5]
 ; CHECK-NEXT:    vmov.16 q5[7], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[7]
+; CHECK-NEXT:    vmov.u16 r0, q1[7]
 ; CHECK-NEXT:    vmov.16 q4[5], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[1]
+; CHECK-NEXT:    vmov.u16 r0, q0[1]
 ; CHECK-NEXT:    vmov.f32 s19, s23
 ; CHECK-NEXT:    vmov.16 q5[0], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[4]
+; CHECK-NEXT:    vmov.u16 r0, q0[4]
 ; CHECK-NEXT:    vmov.16 q5[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[7]
+; CHECK-NEXT:    vmov.u16 r0, q0[7]
 ; CHECK-NEXT:    vmov.16 q5[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[2]
+; CHECK-NEXT:    vmov.u16 r0, q1[2]
 ; CHECK-NEXT:    vmov.16 q5[3], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[5]
+; CHECK-NEXT:    vmov.u16 r0, q1[5]
 ; CHECK-NEXT:    vmov.16 q5[4], r0
-; CHECK-NEXT:    vmov.u16 r0, q3[0]
-; CHECK-NEXT:    vmov.16 q1[5], r0
-; CHECK-NEXT:    vmov.u16 r0, q3[3]
-; CHECK-NEXT:    vmov.16 q1[6], r0
-; CHECK-NEXT:    vmov.u16 r0, q3[6]
-; CHECK-NEXT:    vmov.16 q1[7], r0
-; CHECK-NEXT:    vmov r0, s20
-; CHECK-NEXT:    vmov q2, q1
-; CHECK-NEXT:    vmov.32 q3[0], r0
+; CHECK-NEXT:    vmov.u16 r0, q2[0]
+; CHECK-NEXT:    vmov.16 q0[5], r0
+; CHECK-NEXT:    vmov.u16 r0, q2[3]
+; CHECK-NEXT:    vmov.16 q0[6], r0
+; CHECK-NEXT:    vmov.u16 r0, q2[6]
+; CHECK-NEXT:    vmov.16 q0[7], r0
+; CHECK-NEXT:    vmov r2, s20
+; CHECK-NEXT:    vmov q1, q0
+; CHECK-NEXT:    vmovnb.i32 q1, q5
+; CHECK-NEXT:    vmov r0, s6
+; CHECK-NEXT:    vmov q1[2], q1[0], r0, r2
 ; CHECK-NEXT:    vmov r0, s21
-; CHECK-NEXT:    vmovnb.i32 q2, q5
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov r0, s10
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov r0, s7
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vadd.i16 q1, q4, q3
-; CHECK-NEXT:    vadd.i16 q0, q1, q0
+; CHECK-NEXT:    vmov r2, s3
+; CHECK-NEXT:    vmov q1[3], q1[1], r2, r0
+; CHECK-NEXT:    vadd.i16 q0, q4, q1
+; CHECK-NEXT:    vadd.i16 q0, q0, q3
 ; CHECK-NEXT:    vstrw.32 q0, [r1]
-; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13}
+; CHECK-NEXT:    vpop {d8, d9, d10, d11}
 ; CHECK-NEXT:    bx lr
 entry:
   %l1 = load <24 x i16>, <24 x i16>* %src, align 4
   %s1 = shufflevector <24 x i16> %l1, <24 x i16> undef, <8 x i32> <i32 0, i32 3, i32 6, i32 9, i32 12, i32 15, i32 18, i32 21>
   %s2 = shufflevector <24 x i16> %l1, <24 x i16> undef, <8 x i32> <i32 1, i32 4, i32 7, i32 10, i32 13, i32 16, i32 19, i32 22>
   %s3 = shufflevector <24 x i16> %l1, <24 x i16> undef, <8 x i32> <i32 2, i32 5, i32 8, i32 11, i32 14, i32 17, i32 20, i32 23>
   %a1 = add <8 x i16> %s1, %s2
   %a = add <8 x i16> %a1, %s3
   store <8 x i16> %a, <8 x i16> *%dst
   ret void
 }
 
 define void @vld3_v16i16(<48 x i16> *%src, <16 x i16> *%dst) {
 ; CHECK-LABEL: vld3_v16i16:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    vldrw.u32 q1, [r0, #48]
-; CHECK-NEXT:    vldrw.u32 q2, [r0, #64]
-; CHECK-NEXT:    vldrw.u32 q3, [r0, #80]
-; CHECK-NEXT:    vmov.u16 r2, q1[2]
+; CHECK-NEXT:    vldrw.u32 q0, [r0, #48]
+; CHECK-NEXT:    vldrw.u32 q1, [r0, #64]
+; CHECK-NEXT:    vldrw.u32 q2, [r0, #80]
+; CHECK-NEXT:    vmov.u16 r2, q0[2]
 ; CHECK-NEXT:    vmov.16 q4[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[5]
+; CHECK-NEXT:    vmov.u16 r2, q0[5]
 ; CHECK-NEXT:    vmov.16 q4[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[0]
+; CHECK-NEXT:    vmov.u16 r2, q1[0]
 ; CHECK-NEXT:    vmov.16 q4[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[3]
+; CHECK-NEXT:    vmov.u16 r2, q1[3]
 ; CHECK-NEXT:    vmov.16 q4[3], r2
-; CHECK-NEXT:    vmov.f32 s18, s11
-; CHECK-NEXT:    vmov r2, s16
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov r2, s17
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q3[4]
+; CHECK-NEXT:    vmov.u16 r2, q2[4]
 ; CHECK-NEXT:    vmov.16 q5[6], r2
-; CHECK-NEXT:    vmov.u16 r2, q3[7]
+; CHECK-NEXT:    vmov.u16 r2, q2[7]
 ; CHECK-NEXT:    vmov.16 q5[7], r2
-; CHECK-NEXT:    vmov.f32 s22, s12
-; CHECK-NEXT:    vmov q6, q5
-; CHECK-NEXT:    vmovnb.i32 q6, q4
-; CHECK-NEXT:    vmov r2, s26
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vmov r2, s23
-; CHECK-NEXT:    vmov.32 q0[3], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[0]
+; CHECK-NEXT:    vmov.f32 s18, s7
+; CHECK-NEXT:    vmov.f32 s22, s8
+; CHECK-NEXT:    vmov q3, q5
+; CHECK-NEXT:    vmov r3, s16
+; CHECK-NEXT:    vmovnb.i32 q3, q4
+; CHECK-NEXT:    vmov r2, s14
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r3
+; CHECK-NEXT:    vmov r2, s17
+; CHECK-NEXT:    vmov r3, s23
+; CHECK-NEXT:    vmov q3[3], q3[1], r3, r2
+; CHECK-NEXT:    vmov.u16 r2, q0[0]
 ; CHECK-NEXT:    vmov.16 q4[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[3]
+; CHECK-NEXT:    vmov.u16 r2, q0[3]
 ; CHECK-NEXT:    vmov.16 q4[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[6]
+; CHECK-NEXT:    vmov.u16 r2, q0[6]
 ; CHECK-NEXT:    vmov.16 q4[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[1]
+; CHECK-NEXT:    vmov.u16 r2, q1[1]
 ; CHECK-NEXT:    vmov.16 q4[3], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[4]
+; CHECK-NEXT:    vmov.u16 r2, q1[4]
 ; CHECK-NEXT:    vmov.16 q4[4], r2
-; CHECK-NEXT:    vmov.u16 r2, q3[2]
+; CHECK-NEXT:    vmov.u16 r2, q2[2]
 ; CHECK-NEXT:    vmov.16 q5[6], r2
-; CHECK-NEXT:    vmov.u16 r2, q3[5]
+; CHECK-NEXT:    vmov.u16 r2, q2[5]
 ; CHECK-NEXT:    vmov.16 q5[7], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[7]
+; CHECK-NEXT:    vmov.u16 r2, q1[7]
 ; CHECK-NEXT:    vmov.16 q4[5], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[1]
+; CHECK-NEXT:    vmov.u16 r2, q0[1]
 ; CHECK-NEXT:    vmov.f32 s19, s23
 ; CHECK-NEXT:    vmov.16 q5[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[4]
+; CHECK-NEXT:    vmov.u16 r2, q0[4]
 ; CHECK-NEXT:    vmov.16 q5[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[7]
+; CHECK-NEXT:    vmov.u16 r2, q0[7]
 ; CHECK-NEXT:    vmov.16 q5[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[2]
+; CHECK-NEXT:    vmov.u16 r2, q1[2]
 ; CHECK-NEXT:    vmov.16 q5[3], r2
-; CHECK-NEXT:    vmov.u16 r2, q2[5]
+; CHECK-NEXT:    vmov.u16 r2, q1[5]
 ; CHECK-NEXT:    vmov.16 q5[4], r2
-; CHECK-NEXT:    vmov.u16 r2, q3[0]
-; CHECK-NEXT:    vmov.16 q1[5], r2
-; CHECK-NEXT:    vmov.u16 r2, q3[3]
-; CHECK-NEXT:    vmov.16 q1[6], r2
-; CHECK-NEXT:    vmov.u16 r2, q3[6]
-; CHECK-NEXT:    vmov.16 q1[7], r2
-; CHECK-NEXT:    vmov r2, s20
-; CHECK-NEXT:    vmov q2, q1
-; CHECK-NEXT:    vmov.32 q3[0], r2
+; CHECK-NEXT:    vmov.u16 r2, q2[0]
+; CHECK-NEXT:    vmov.16 q0[5], r2
+; CHECK-NEXT:    vmov.u16 r2, q2[3]
+; CHECK-NEXT:    vmov.16 q0[6], r2
+; CHECK-NEXT:    vmov.u16 r2, q2[6]
+; CHECK-NEXT:    vmov.16 q0[7], r2
+; CHECK-NEXT:    vmov r3, s20
+; CHECK-NEXT:    vmov q1, q0
+; CHECK-NEXT:    vldrw.u32 q2, [r0]
+; CHECK-NEXT:    vmovnb.i32 q1, q5
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, r3
 ; CHECK-NEXT:    vmov r2, s21
-; CHECK-NEXT:    vmovnb.i32 q2, q5
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov r2, s10
-; CHECK-NEXT:    vmov.32 q3[2], r2
-; CHECK-NEXT:    vmov r2, s7
-; CHECK-NEXT:    vmov.32 q3[3], r2
-; CHECK-NEXT:    vldrw.u32 q2, [r0, #16]
-; CHECK-NEXT:    vadd.i16 q1, q4, q3
-; CHECK-NEXT:    vldrw.u32 q3, [r0, #32]
-; CHECK-NEXT:    vadd.i16 q0, q1, q0
-; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    vmov.u16 r0, q3[4]
-; CHECK-NEXT:    vstrw.32 q0, [r1, #16]
-; CHECK-NEXT:    vmov.u16 r2, q1[2]
-; CHECK-NEXT:    vmov.16 q6[6], r0
-; CHECK-NEXT:    vmov.16 q5[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[5]
-; CHECK-NEXT:    vmov.16 q5[1], r2
+; CHECK-NEXT:    vmov r3, s3
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q2[0]
-; CHECK-NEXT:    vmov.u16 r0, q3[7]
-; CHECK-NEXT:    vmov.16 q5[2], r2
-; CHECK-NEXT:    vmov.16 q6[7], r0
+; CHECK-NEXT:    vadd.i16 q0, q4, q1
+; CHECK-NEXT:    vmov.16 q1[0], r2
+; CHECK-NEXT:    vadd.i16 q0, q0, q3
 ; CHECK-NEXT:    vmov.u16 r2, q2[3]
-; CHECK-NEXT:    vmov.16 q5[3], r2
-; CHECK-NEXT:    vmov.f32 s26, s12
-; CHECK-NEXT:    vmov.f32 s22, s11
-; CHECK-NEXT:    vmov q7, q6
-; CHECK-NEXT:    vmov r0, s20
-; CHECK-NEXT:    vmovnb.i32 q7, q5
-; CHECK-NEXT:    vmov.32 q4[0], r0
-; CHECK-NEXT:    vmov r0, s21
-; CHECK-NEXT:    vmov.32 q4[1], r0
-; CHECK-NEXT:    vmov r0, s30
-; CHECK-NEXT:    vmov.32 q4[2], r0
-; CHECK-NEXT:    vmov r0, s27
-; CHECK-NEXT:    vmov.32 q4[3], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[0]
-; CHECK-NEXT:    vmov.16 q5[0], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[3]
-; CHECK-NEXT:    vmov.16 q5[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[6]
-; CHECK-NEXT:    vmov.16 q5[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[1]
-; CHECK-NEXT:    vmov.16 q5[3], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[4]
-; CHECK-NEXT:    vmov.16 q5[4], r0
-; CHECK-NEXT:    vmov.u16 r0, q3[2]
-; CHECK-NEXT:    vmov.16 q6[6], r0
-; CHECK-NEXT:    vmov.u16 r0, q3[5]
-; CHECK-NEXT:    vmov.16 q6[7], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[7]
-; CHECK-NEXT:    vmov.16 q5[5], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[1]
-; CHECK-NEXT:    vmov.f32 s23, s27
+; CHECK-NEXT:    vldrw.u32 q3, [r0, #16]
+; CHECK-NEXT:    vldrw.u32 q4, [r0, #32]
+; CHECK-NEXT:    vmov.16 q1[1], r2
+; CHECK-NEXT:    vmov.u16 r2, q2[6]
+; CHECK-NEXT:    vmov.16 q1[2], r2
+; CHECK-NEXT:    vmov.u16 r2, q3[1]
+; CHECK-NEXT:    vmov.u16 r0, q4[2]
+; CHECK-NEXT:    vmov.16 q1[3], r2
+; CHECK-NEXT:    vmov.u16 r2, q3[4]
+; CHECK-NEXT:    vmov.16 q5[6], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[5]
+; CHECK-NEXT:    vmov.16 q1[4], r2
+; CHECK-NEXT:    vmov.16 q5[7], r0
+; CHECK-NEXT:    vmov.u16 r0, q3[7]
+; CHECK-NEXT:    vmov.16 q1[5], r0
+; CHECK-NEXT:    vmov.u16 r0, q2[2]
 ; CHECK-NEXT:    vmov.16 q6[0], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[4]
+; CHECK-NEXT:    vmov.u16 r0, q2[5]
 ; CHECK-NEXT:    vmov.16 q6[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[7]
+; CHECK-NEXT:    vmov.u16 r0, q3[0]
 ; CHECK-NEXT:    vmov.16 q6[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[2]
+; CHECK-NEXT:    vmov.u16 r0, q3[3]
 ; CHECK-NEXT:    vmov.16 q6[3], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[5]
-; CHECK-NEXT:    vmov.16 q6[4], r0
-; CHECK-NEXT:    vmov r0, s24
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[4]
+; CHECK-NEXT:    vmov.16 q7[6], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[7]
+; CHECK-NEXT:    vmov.16 q7[7], r0
+; CHECK-NEXT:    vmov.f32 s26, s15
+; CHECK-NEXT:    vmov.f32 s30, s16
+; CHECK-NEXT:    vstrw.32 q0, [r1, #16]
+; CHECK-NEXT:    vmov.f32 s7, s23
+; CHECK-NEXT:    vmov q5, q7
+; CHECK-NEXT:    vmovnb.i32 q5, q6
+; CHECK-NEXT:    vmov r2, s24
+; CHECK-NEXT:    vmov r0, s22
+; CHECK-NEXT:    vmov q5[2], q5[0], r0, r2
 ; CHECK-NEXT:    vmov r0, s25
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q3[0]
-; CHECK-NEXT:    vmov.16 q2[5], r0
-; CHECK-NEXT:    vmov.u16 r0, q3[3]
-; CHECK-NEXT:    vmov.16 q2[6], r0
-; CHECK-NEXT:    vmov.u16 r0, q3[6]
-; CHECK-NEXT:    vmov.16 q2[7], r0
-; CHECK-NEXT:    vmov q3, q2
-; CHECK-NEXT:    vmovnb.i32 q3, q6
-; CHECK-NEXT:    vmov r0, s14
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov r0, s11
-; CHECK-NEXT:    vmov.32 q1[3], r0
-; CHECK-NEXT:    vadd.i16 q1, q5, q1
-; CHECK-NEXT:    vadd.i16 q1, q1, q4
+; CHECK-NEXT:    vmov r2, s31
+; CHECK-NEXT:    vmov q5[3], q5[1], r2, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[0]
+; CHECK-NEXT:    vmov.16 q6[5], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[3]
+; CHECK-NEXT:    vmov.16 q6[6], r0
+; CHECK-NEXT:    vmov.u16 r0, q2[1]
+; CHECK-NEXT:    vmov.16 q7[0], r0
+; CHECK-NEXT:    vmov.u16 r0, q2[4]
+; CHECK-NEXT:    vmov.16 q7[1], r0
+; CHECK-NEXT:    vmov.u16 r0, q2[7]
+; CHECK-NEXT:    vmov.16 q7[2], r0
+; CHECK-NEXT:    vmov.u16 r0, q3[2]
+; CHECK-NEXT:    vmov.16 q7[3], r0
+; CHECK-NEXT:    vmov.u16 r0, q3[5]
+; CHECK-NEXT:    vmov.16 q7[4], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[6]
+; CHECK-NEXT:    vmov.16 q6[7], r0
+; CHECK-NEXT:    vmov r2, s28
+; CHECK-NEXT:    vmov q2, q6
+; CHECK-NEXT:    vmovnb.i32 q2, q7
+; CHECK-NEXT:    vmov r0, s10
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r2
+; CHECK-NEXT:    vmov r0, s29
+; CHECK-NEXT:    vmov r2, s27
+; CHECK-NEXT:    vmov q2[3], q2[1], r2, r0
+; CHECK-NEXT:    vadd.i16 q1, q1, q2
+; CHECK-NEXT:    vadd.i16 q1, q1, q5
 ; CHECK-NEXT:    vstrw.32 q1, [r1]
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    bx lr
 entry:
   %l1 = load <48 x i16>, <48 x i16>* %src, align 4
   %s1 = shufflevector <48 x i16> %l1, <48 x i16> undef, <16 x i32> <i32 0, i32 3, i32 6, i32 9, i32 12, i32 15, i32 18, i32 21, i32 24, i32 27, i32 30, i32 33, i32 36, i32 39, i32 42, i32 45>
   %s2 = shufflevector <48 x i16> %l1, <48 x i16> undef, <16 x i32> <i32 1, i32 4, i32 7, i32 10, i32 13, i32 16, i32 19, i32 22, i32 25, i32 28, i32 31, i32 34, i32 37, i32 40, i32 43, i32 46>
   %s3 = shufflevector <48 x i16> %l1, <48 x i16> undef, <16 x i32> <i32 2, i32 5, i32 8, i32 11, i32 14, i32 17, i32 20, i32 23, i32 26, i32 29, i32 32, i32 35, i32 38, i32 41, i32 44, i32 47>
entry:
   %a = add <2 x i8> %a1, %s3
   store <2 x i8> %a, <2 x i8> *%dst
   ret void
 }
 
 define void @vld3_v4i8(<12 x i8> *%src, <4 x i8> *%dst) {
 ; CHECK-LABEL: vld3_v4i8:
 ; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
 ; CHECK-NEXT:    .pad #8
 ; CHECK-NEXT:    sub sp, #8
-; CHECK-NEXT:    vldrb.u16 q2, [r0]
-; CHECK-NEXT:    ldr r3, [r0, #8]
-; CHECK-NEXT:    mov r2, sp
-; CHECK-NEXT:    str r3, [sp]
-; CHECK-NEXT:    vmov.u16 r0, q2[2]
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[5]
-; CHECK-NEXT:    vmov.32 q0[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[0]
-; CHECK-NEXT:    vmov.32 q1[0], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[3]
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[6]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[1]
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[4]
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[7]
-; CHECK-NEXT:    vldrb.u16 q2, [r2]
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[2]
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[1]
-; CHECK-NEXT:    vmov.32 q1[3], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[0]
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[3]
-; CHECK-NEXT:    vadd.i32 q1, q1, q3
-; CHECK-NEXT:    vmov.32 q0[3], r0
-; CHECK-NEXT:    vadd.i32 q0, q1, q0
+; CHECK-NEXT:    vldrb.u16 q0, [r0]
+; CHECK-NEXT:    ldr r0, [r0, #8]
+; CHECK-NEXT:    str r0, [sp]
+; CHECK-NEXT:    vmov.u16 r3, q0[0]
+; CHECK-NEXT:    vmov.u16 r4, q0[6]
+; CHECK-NEXT:    vmov q1[2], q1[0], r4, r3
+; CHECK-NEXT:    vmov.u16 r3, q0[1]
+; CHECK-NEXT:    vmov.u16 r4, q0[7]
+; CHECK-NEXT:    vmov.u16 r12, q0[5]
+; CHECK-NEXT:    vmov q2[2], q2[0], r4, r3
+; CHECK-NEXT:    mov r3, sp
+; CHECK-NEXT:    vmov.u16 lr, q0[2]
+; CHECK-NEXT:    vmov.u16 r2, q0[3]
+; CHECK-NEXT:    vmov.u16 r0, q0[4]
+; CHECK-NEXT:    vldrb.u16 q0, [r3]
+; CHECK-NEXT:    vmov.u16 r3, q0[2]
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r0
+; CHECK-NEXT:    vmov.u16 r0, q0[1]
+; CHECK-NEXT:    vmov q1[3], q1[1], r0, r2
+; CHECK-NEXT:    vmov.u16 r0, q0[0]
+; CHECK-NEXT:    vadd.i32 q1, q1, q2
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, lr
+; CHECK-NEXT:    vmov.u16 r0, q0[3]
+; CHECK-NEXT:    vmov q2[3], q2[1], r0, r12
+; CHECK-NEXT:    vadd.i32 q0, q1, q2
 ; CHECK-NEXT:    vstrb.32 q0, [r1]
 ; CHECK-NEXT:    add sp, #8
-; CHECK-NEXT:    bx lr
+; CHECK-NEXT:    pop {r4, pc}
 entry:
   %l1 = load <12 x i8>, <12 x i8>* %src, align 4
   %s1 = shufflevector <12 x i8> %l1, <12 x i8> undef, <4 x i32> <i32 0, i32 3, i32 6, i32 9>
   %s2 = shufflevector <12 x i8> %l1, <12 x i8> undef, <4 x i32> <i32 1, i32 4, i32 7, i32 10>
   %s3 = shufflevector <12 x i8> %l1, <12 x i8> undef, <4 x i32> <i32 2, i32 5, i32 8, i32 11>
   %a1 = add <4 x i8> %s1, %s2
   %a = add <4 x i8> %a1, %s3
   store <4 x i8> %a, <4 x i8> *%dst
entry:
   %a = add <8 x i8> %a1, %s3
   store <8 x i8> %a, <8 x i8> *%dst
   ret void
 }
 
 define void @vld3_v16i8(<48 x i8> *%src, <16 x i8> *%dst) {
 ; CHECK-LABEL: vld3_v16i8:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-NEXT:    vpush {d8, d9, d10, d11}
-; CHECK-NEXT:    vldrw.u32 q2, [r0]
+; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
+; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vldrw.u32 q0, [r0, #16]
+; CHECK-NEXT:    vldrw.u32 q2, [r0]
+; CHECK-NEXT:    vldrw.u32 q1, [r0, #32]
+; CHECK-NEXT:    vmov.u8 r2, q0[8]
+; CHECK-NEXT:    vmov.8 q4[8], r2
+; CHECK-NEXT:    vmov.u8 r2, q0[11]
+; CHECK-NEXT:    vmov.8 q4[9], r2
+; CHECK-NEXT:    vmov.u8 r2, q0[14]
+; CHECK-NEXT:    vmov.8 q4[10], r2
 ; CHECK-NEXT:    vmov.u8 r2, q2[0]
-; CHECK-NEXT:    vmov.8 q1[0], r2
+; CHECK-NEXT:    vmov.8 q3[0], r2
 ; CHECK-NEXT:    vmov.u8 r2, q2[3]
-; CHECK-NEXT:    vmov.8 q1[1], r2
+; CHECK-NEXT:    vmov.8 q3[1], r2
 ; CHECK-NEXT:    vmov.u8 r2, q2[6]
-; CHECK-NEXT:    vmov.8 q1[2], r2
+; CHECK-NEXT:    vmov.8 q3[2], r2
 ; CHECK-NEXT:    vmov.u8 r2, q2[9]
-; CHECK-NEXT:    vmov.8 q1[3], r2
+; CHECK-NEXT:    vmov.8 q3[3], r2
 ; CHECK-NEXT:    vmov.u8 r2, q2[12]
-; CHECK-NEXT:    vmov.8 q1[4], r2
+; CHECK-NEXT:    vmov.8 q3[4], r2
 ; CHECK-NEXT:    vmov.u8 r2, q2[15]
-; CHECK-NEXT:    vmov.8 q1[5], r2
+; CHECK-NEXT:    vmov.8 q3[5], r2
 ; CHECK-NEXT:    vmov.u8 r2, q0[2]
-; CHECK-NEXT:    vmov.8 q1[6], r2
+; CHECK-NEXT:    vmov.8 q3[6], r2
 ; CHECK-NEXT:    vmov.u8 r2, q0[5]
-; CHECK-NEXT:    vmov.8 q1[7], r2
-; CHECK-NEXT:    vmov r2, s4
-; CHECK-NEXT:    vmov.32 q3[0], r2
-; CHECK-NEXT:    vmov r2, s5
-; CHECK-NEXT:    vmov.32 q3[1], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[8]
-; CHECK-NEXT:    vldrw.u32 q1, [r0, #32]
-; CHECK-NEXT:    vmov.8 q4[8], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[11]
-; CHECK-NEXT:    vmov.8 q4[9], r2
-; CHECK-NEXT:    vmov.u8 r2, q0[14]
-; CHECK-NEXT:    vmov.8 q4[10], r2
 ; CHECK-NEXT:    vmov.u8 r0, q1[1]
+; CHECK-NEXT:    vmov.8 q3[7], r2
 ; CHECK-NEXT:    vmov.8 q4[11], r0
-; CHECK-NEXT:    vmov r0, s18
-; CHECK-NEXT:    vmov.32 q3[2], r0
+; CHECK-NEXT:    vmov r0, s12
+; CHECK-NEXT:    vmov r2, s18
+; CHECK-NEXT:    vmov q4[2], q4[0], r2, r0
 ; CHECK-NEXT:    vmov.u8 r0, q2[1]
 ; CHECK-NEXT:    vmov.8 q5[0], r0
 ; CHECK-NEXT:    vmov.u8 r0, q2[4]
 ; CHECK-NEXT:    vmov.8 q5[1], r0
 ; CHECK-NEXT:    vmov.u8 r0, q2[7]
 ; CHECK-NEXT:    vmov.8 q5[2], r0
 ; CHECK-NEXT:    vmov.u8 r0, q2[10]
 ; CHECK-NEXT:    vmov.8 q5[3], r0
 ; CHECK-NEXT:    vmov.u8 r0, q2[13]
+; CHECK-NEXT:    vmov.u8 r2, q0[9]
 ; CHECK-NEXT:    vmov.8 q5[4], r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[0]
+; CHECK-NEXT:    vmov.8 q6[8], r2
+; CHECK-NEXT:    vmov.u8 r2, q0[12]
 ; CHECK-NEXT:    vmov.8 q5[5], r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[3]
+; CHECK-NEXT:    vmov.8 q6[9], r2
+; CHECK-NEXT:    vmov.u8 r2, q0[15]
 ; CHECK-NEXT:    vmov.8 q5[6], r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[6]
+; CHECK-NEXT:    vmov.8 q6[10], r2
+; CHECK-NEXT:    vmov.u8 r2, q1[2]
 ; CHECK-NEXT:    vmov.8 q5[7], r0
+; CHECK-NEXT:    vmov.8 q6[11], r2
 ; CHECK-NEXT:    vmov r0, s20
-; CHECK-NEXT:    vmov.32 q4[0], r0
-; CHECK-NEXT:    vmov r0, s21
-; CHECK-NEXT:    vmov.32 q4[1], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[9]
-; CHECK-NEXT:    vmov.8 q5[8], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[12]
-; CHECK-NEXT:    vmov.8 q5[9], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[15]
-; CHECK-NEXT:    vmov.8 q5[10], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[2]
-; CHECK-NEXT:    vmov.8 q5[11], r0
-; CHECK-NEXT:    vmov r0, s22
-; CHECK-NEXT:    vmov.32 q4[2], r0
+; CHECK-NEXT:    vmov r2, s26
+; CHECK-NEXT:    vmov q6[2], q6[0], r2, r0
 ; CHECK-NEXT:    vmov.u8 r0, q1[5]
-; CHECK-NEXT:    vmov.8 q5[12], r0
+; CHECK-NEXT:    vmov.8 q7[12], r0
 ; CHECK-NEXT:    vmov.u8 r0, q1[8]
-; CHECK-NEXT:    vmov.8 q5[13], r0
+; CHECK-NEXT:    vmov.8 q7[13], r0
 ; CHECK-NEXT:    vmov.u8 r0, q1[11]
-; CHECK-NEXT:    vmov.8 q5[14], r0
+; CHECK-NEXT:    vmov.8 q7[14], r0
 ; CHECK-NEXT:    vmov.u8 r0, q1[14]
-; CHECK-NEXT:    vmov.8 q5[15], r0
-; CHECK-NEXT:    vmov r0, s23
-; CHECK-NEXT:    vmov.32 q4[3], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[4]
-; CHECK-NEXT:    vmov.8 q5[12], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[7]
-; CHECK-NEXT:    vmov.8 q5[13], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[10]
-; CHECK-NEXT:    vmov.8 q5[14], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[13]
-; CHECK-NEXT:    vmov.8 q5[15], r0
-; CHECK-NEXT:    vmov r0, s23
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    vmov.8 q7[15], r0
+; CHECK-NEXT:    vmov r0, s21
+; CHECK-NEXT:    vmov r2, s31
+; CHECK-NEXT:    vmov q6[3], q6[1], r2, r0
+; CHECK-NEXT:    vmov.u8 r2, q1[4]
+; CHECK-NEXT:    vmov r0, s13
+; CHECK-NEXT:    vmov.8 q3[12], r2
+; CHECK-NEXT:    vmov.u8 r2, q1[7]
+; CHECK-NEXT:    vmov.8 q3[13], r2
+; CHECK-NEXT:    vmov.u8 r2, q1[10]
+; CHECK-NEXT:    vmov.8 q3[14], r2
+; CHECK-NEXT:    vmov.u8 r2, q1[13]
+; CHECK-NEXT:    vmov.8 q3[15], r2
+; CHECK-NEXT:    vmov r2, s15
+; CHECK-NEXT:    vmov q4[3], q4[1], r2, r0
 ; CHECK-NEXT:    vmov.u8 r0, q2[2]
-; CHECK-NEXT:    vadd.i8 q3, q3, q4
+; CHECK-NEXT:    vadd.i8 q3, q4, q6
 ; CHECK-NEXT:    vmov.8 q4[0], r0
 ; CHECK-NEXT:    vmov.u8 r0, q2[5]
+; CHECK-NEXT:    vmov.u8 r2, q0[10]
 ; CHECK-NEXT:    vmov.8 q4[1], r0
 ; CHECK-NEXT:    vmov.u8 r0, q2[8]
 ; CHECK-NEXT:    vmov.8 q4[2], r0
 ; CHECK-NEXT:    vmov.u8 r0, q2[11]
 ; CHECK-NEXT:    vmov.8 q4[3], r0
 ; CHECK-NEXT:    vmov.u8 r0, q2[14]
 ; CHECK-NEXT:    vmov.8 q4[4], r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[1]
+; CHECK-NEXT:    vmov.8 q2[8], r2
+; CHECK-NEXT:    vmov.u8 r2, q0[13]
 ; CHECK-NEXT:    vmov.8 q4[5], r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[4]
+; CHECK-NEXT:    vmov.8 q2[9], r2
+; CHECK-NEXT:    vmov.u8 r2, q1[0]
 ; CHECK-NEXT:    vmov.8 q4[6], r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[7]
+; CHECK-NEXT:    vmov.8 q2[10], r2
+; CHECK-NEXT:    vmov.u8 r2, q1[3]
+; CHECK-NEXT:    vmov.8 q2[11], r2
 ; CHECK-NEXT:    vmov.8 q4[7], r0
 ; CHECK-NEXT:    vmov r0, s16
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    vmov r0, s17
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[10]
-; CHECK-NEXT:    vmov.8 q4[8], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[13]
-; CHECK-NEXT:    vmov.8 q4[9], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[0]
-; CHECK-NEXT:    vmov.8 q4[10], r0
-; CHECK-NEXT:    vmov.u8 r0, q1[3]
-; CHECK-NEXT:    vmov.8 q4[11], r0
-; CHECK-NEXT:    vmov r0, s18
-; CHECK-NEXT:    vmov.32 q2[2], r0
+; CHECK-NEXT:    vmov r2, s10
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r0
 ; CHECK-NEXT:    vmov.u8 r0, q1[6]
-; CHECK-NEXT:    vmov.8 q0[12], r0
+; CHECK-NEXT:    vmov.8 q2[12], r0
 ; CHECK-NEXT:    vmov.u8 r0, q1[9]
-; CHECK-NEXT:    vmov.8 q0[13], r0
+; CHECK-NEXT:    vmov.8 q2[13], r0
 ; CHECK-NEXT:    vmov.u8 r0, q1[12]
-; CHECK-NEXT:    vmov.8 q0[14], r0
+; CHECK-NEXT:    vmov.8 q2[14], r0
 ; CHECK-NEXT:    vmov.u8 r0, q1[15]
-; CHECK-NEXT:    vmov.8 q0[15], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vadd.i8 q0, q3, q2
+; CHECK-NEXT:    vmov.8 q2[15], r0
+; CHECK-NEXT:    vmov r0, s17
+; CHECK-NEXT:    vmov r2, s11
+; CHECK-NEXT:    vmov q0[3], q0[1], r2, r0
+; CHECK-NEXT:    vadd.i8 q0, q3, q0
 ; CHECK-NEXT:    vstrw.32 q0, [r1]
-; CHECK-NEXT:    vpop {d8, d9, d10, d11}
+; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    bx lr
 entry:
   %l1 = load <48 x i8>, <48 x i8>* %src, align 4
   %s1 = shufflevector <48 x i8> %l1, <48 x i8> undef, <16 x i32> <i32 0, i32 3, i32 6, i32 9, i32 12, i32 15, i32 18, i32 21, i32 24, i32 27, i32 30, i32 33, i32 36, i32 39, i32 42, i32 45>
   %s2 = shufflevector <48 x i8> %l1, <48 x i8> undef, <16 x i32> <i32 1, i32 4, i32 7, i32 10, i32 13, i32 16, i32 19, i32 22, i32 25, i32 28, i32 31, i32 34, i32 37, i32 40, i32 43, i32 46>
   %s3 = shufflevector <48 x i8> %l1, <48 x i8> undef, <16 x i32> <i32 2, i32 5, i32 8, i32 11, i32 14, i32 17, i32 20, i32 23, i32 26, i32 29, i32 32, i32 35, i32 38, i32 41, i32 44, i32 47>
   %a1 = add <16 x i8> %s1, %s2
   %a = add <16 x i8> %a1, %s3

 ; CHECK-NEXT:    vldrw.u32 q2, [r0, #16]
 ; CHECK-NEXT:    vldrw.u32 q4, [r0, #32]
 ; CHECK-NEXT:    vmov.f64 d6, d3
 ; CHECK-NEXT:    vmov.f32 s13, s7
 ; CHECK-NEXT:    vmov.f32 s14, s16
 ; CHECK-NEXT:    vmov.f32 s6, s10
 ; CHECK-NEXT:    vmov.f32 s7, s11
 ; CHECK-NEXT:    vmov.f32 s15, s17
-; CHECK-NEXT:    vmov r3, s14
-; CHECK-NEXT:    vmov r0, s6
+; CHECK-NEXT:    vmov r3, s12
+; CHECK-NEXT:    vmov r0, s4
 ; CHECK-NEXT:    vmov.f64 d0, d4
 ; CHECK-NEXT:    vmov.f32 s1, s9
 ; CHECK-NEXT:    vmov.f32 s2, s18
 ; CHECK-NEXT:    vmov.f32 s3, s19
-; CHECK-NEXT:    vmov r12, s15
-; CHECK-NEXT:    vmov r2, s7
-; CHECK-NEXT:    vmov r4, s4
+; CHECK-NEXT:    vmov r12, s13
+; CHECK-NEXT:    vmov r2, s5
+; CHECK-NEXT:    vmov r4, s6
 ; CHECK-NEXT:    adds.w lr, r0, r3
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vmov r3, s3
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov r3, s1
 ; CHECK-NEXT:    adc.w r2, r2, r12
 ; CHECK-NEXT:    adds.w lr, lr, r0
-; CHECK-NEXT:    vmov r0, s12
+; CHECK-NEXT:    vmov r0, s14
 ; CHECK-NEXT:    adc.w r12, r2, r3
-; CHECK-NEXT:    vmov r3, s13
-; CHECK-NEXT:    vmov r2, s5
+; CHECK-NEXT:    vmov r3, s15
+; CHECK-NEXT:    vmov r2, s7
 ; CHECK-NEXT:    adds r0, r0, r4
-; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov r4, s2
 ; CHECK-NEXT:    adcs r2, r3
-; CHECK-NEXT:    vmov r3, s1
+; CHECK-NEXT:    vmov r3, s3
 ; CHECK-NEXT:    adds r0, r0, r4
-; CHECK-NEXT:    vmov.32 q0[0], r0
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, lr
 ; CHECK-NEXT:    adcs r2, r3
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.32 q0[2], lr
-; CHECK-NEXT:    vmov.32 q0[3], r12
+; CHECK-NEXT:    vmov q0[3], q0[1], r2, r12
 ; CHECK-NEXT:    vstrw.32 q0, [r1]
 ; CHECK-NEXT:    vpop {d8, d9}
 ; CHECK-NEXT:    pop {r4, pc}
 entry:
   %l1 = load <6 x i64>, <6 x i64>* %src, align 4
   %s1 = shufflevector <6 x i64> %l1, <6 x i64> undef, <2 x i32> <i32 0, i32 3>
   %s2 = shufflevector <6 x i64> %l1, <6 x i64> undef, <2 x i32> <i32 1, i32 4>
   %s3 = shufflevector <6 x i64> %l1, <6 x i64> undef, <2 x i32> <i32 2, i32 5>
   %a1 = add <2 x i64> %s1, %s2
   %a = add <2 x i64> %a1, %s3
   store <2 x i64> %a, <2 x i64> *%dst
   ret void
 }
 
 define void @vld3_v4i64(<12 x i64> *%src, <4 x i64> *%dst) {
 ; CHECK-LABEL: vld3_v4i64:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, lr}
-; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, lr}
+; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, lr}
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    .pad #24
 ; CHECK-NEXT:    sub sp, #24
-; CHECK-NEXT:    vldrw.u32 q0, [r0, #16]
-; CHECK-NEXT:    vldrw.u32 q5, [r0, #48]
-; CHECK-NEXT:    vldrw.u32 q6, [r0, #64]
-; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    vmov.f64 d4, d0
-; CHECK-NEXT:    vstrw.32 q0, [sp] @ 16-byte Spill
-; CHECK-NEXT:    vldrw.u32 q4, [r0, #32]
-; CHECK-NEXT:    vmov.f32 s9, s1
-; CHECK-NEXT:    vldrw.u32 q0, [r0, #80]
-; CHECK-NEXT:    vmov.f64 d14, d11
-; CHECK-NEXT:    vmov.f32 s29, s23
-; CHECK-NEXT:    vmov.f32 s30, s0
-; CHECK-NEXT:    vmov.f32 s22, s26
-; CHECK-NEXT:    vmov.f32 s23, s27
-; CHECK-NEXT:    vmov.f32 s31, s1
-; CHECK-NEXT:    vmov r3, s30
-; CHECK-NEXT:    vmov r0, s22
-; CHECK-NEXT:    vmov.f64 d6, d3
-; CHECK-NEXT:    vmov.f32 s13, s7
-; CHECK-NEXT:    vmov.f32 s10, s18
-; CHECK-NEXT:    vmov.f32 s14, s16
-; CHECK-NEXT:    vmov.f32 s11, s19
-; CHECK-NEXT:    vmov.f32 s15, s17
-; CHECK-NEXT:    vmov.f64 d8, d12
-; CHECK-NEXT:    vmov.f32 s17, s25
-; CHECK-NEXT:    vmov.f32 s18, s2
-; CHECK-NEXT:    vmov.f32 s19, s3
-; CHECK-NEXT:    vmov r12, s31
-; CHECK-NEXT:    vmov r2, s23
+; CHECK-NEXT:    vldrw.u32 q2, [r0, #64]
+; CHECK-NEXT:    vldrw.u32 q1, [r0, #80]
+; CHECK-NEXT:    vldrw.u32 q5, [r0]
+; CHECK-NEXT:    vldrw.u32 q3, [r0, #16]
+; CHECK-NEXT:    vmov.f64 d0, d4
+; CHECK-NEXT:    vstrw.32 q2, [sp] @ 16-byte Spill
+; CHECK-NEXT:    vmov.f32 s1, s9
+; CHECK-NEXT:    vldrw.u32 q2, [r0, #48]
+; CHECK-NEXT:    vmov.f32 s2, s6
+; CHECK-NEXT:    vmov.f64 d8, d5
+; CHECK-NEXT:    vmov.f32 s17, s11
+; CHECK-NEXT:    vmov.f32 s18, s4
+; CHECK-NEXT:    vmov.f32 s19, s5
+; CHECK-NEXT:    vmov.f64 d12, d11
+; CHECK-NEXT:    vmov.f32 s3, s7
+; CHECK-NEXT:    vldrw.u32 q1, [r0, #32]
+; CHECK-NEXT:    vmov.f32 s25, s23
+; CHECK-NEXT:    vmov.f32 s26, s4
+; CHECK-NEXT:    vmov.f32 s22, s14
+; CHECK-NEXT:    vmov.f32 s27, s5
+; CHECK-NEXT:    vmov.f32 s23, s15
+; CHECK-NEXT:    vmov r3, s24
+; CHECK-NEXT:    vmov r0, s20
+; CHECK-NEXT:    vmov r12, s25
+; CHECK-NEXT:    vmov r2, s21
+; CHECK-NEXT:    vmov.f64 d14, d6
+; CHECK-NEXT:    vmov.f32 s29, s13
+; CHECK-NEXT:    vmov.f32 s30, s6
+; CHECK-NEXT:    vmov.f32 s31, s7
+; CHECK-NEXT:    vldrw.u32 q1, [sp] @ 16-byte Reload
+; CHECK-NEXT:    vmov r5, s18
+; CHECK-NEXT:    vmov.f32 s10, s6
+; CHECK-NEXT:    vmov.f32 s11, s7
+; CHECK-NEXT:    vmov r4, s8
+; CHECK-NEXT:    vmov r6, s10
+; CHECK-NEXT:    vmov r7, s26
 ; CHECK-NEXT:    adds.w lr, r0, r3
-; CHECK-NEXT:    vmov r0, s18
-; CHECK-NEXT:    vmov r4, s20
-; CHECK-NEXT:    vmov r3, s19
-; CHECK-NEXT:    adc.w r2, r2, r12
-; CHECK-NEXT:    adds.w lr, lr, r0
 ; CHECK-NEXT:    vmov r0, s28
-; CHECK-NEXT:    adc.w r12, r2, r3
-; CHECK-NEXT:    vmov r3, s29
-; CHECK-NEXT:    vmov r2, s21
-; CHECK-NEXT:    adds r0, r0, r4
-; CHECK-NEXT:    vmov r4, s16
-; CHECK-NEXT:    adcs r2, r3
+; CHECK-NEXT:    adc.w r3, r2, r12
+; CHECK-NEXT:    vmov r2, s29
+; CHECK-NEXT:    adds.w lr, lr, r0
+; CHECK-NEXT:    vmov r0, s16
+; CHECK-NEXT:    adc.w r12, r3, r2
 ; CHECK-NEXT:    vmov r3, s17
+; CHECK-NEXT:    vmov r2, s9
 ; CHECK-NEXT:    adds r0, r0, r4
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov r0, s15
+; CHECK-NEXT:    vmov r4, s0
 ; CHECK-NEXT:    adcs r2, r3
-; CHECK-NEXT:    vmov r3, s14
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.32 q0[2], lr
-; CHECK-NEXT:    vmov.32 q0[3], r12
-; CHECK-NEXT:    vstrw.32 q0, [r1, #16]
-; CHECK-NEXT:    vldrw.u32 q0, [sp] @ 16-byte Reload
-; CHECK-NEXT:    vmov.f32 s6, s2
-; CHECK-NEXT:    vmov.f32 s7, s3
-; CHECK-NEXT:    vmov r4, s6
-; CHECK-NEXT:    vmov r2, s7
-; CHECK-NEXT:    adds r3, r3, r4
-; CHECK-NEXT:    vmov r4, s10
-; CHECK-NEXT:    adcs r0, r2
-; CHECK-NEXT:    vmov r2, s11
-; CHECK-NEXT:    adds.w lr, r3, r4
-; CHECK-NEXT:    vmov r3, s4
-; CHECK-NEXT:    vmov r4, s5
-; CHECK-NEXT:    adc.w r12, r0, r2
-; CHECK-NEXT:    vmov r0, s12
-; CHECK-NEXT:    vmov r2, s13
-; CHECK-NEXT:    adds r0, r0, r3
-; CHECK-NEXT:    vmov r3, s9
-; CHECK-NEXT:    adcs r2, r4
-; CHECK-NEXT:    vmov r4, s8
+; CHECK-NEXT:    vmov r3, s1
 ; CHECK-NEXT:    adds r0, r0, r4
-; CHECK-NEXT:    adcs r2, r3
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.32 q0[2], lr
-; CHECK-NEXT:    vmov.32 q0[3], r12
+; CHECK-NEXT:    vmov r4, s11
+; CHECK-NEXT:    adc.w r8, r2, r3
+; CHECK-NEXT:    vmov r3, s19
+; CHECK-NEXT:    adds r5, r5, r6
+; CHECK-NEXT:    vmov r6, s2
+; CHECK-NEXT:    vmov r2, s22
+; CHECK-NEXT:    adcs r3, r4
+; CHECK-NEXT:    vmov r4, s3
+; CHECK-NEXT:    adds r5, r5, r6
+; CHECK-NEXT:    vmov r6, s23
+; CHECK-NEXT:    vmov q1[2], q1[0], r5, r0
+; CHECK-NEXT:    vmov r0, s31
+; CHECK-NEXT:    adcs r3, r4
+; CHECK-NEXT:    vmov r4, s27
+; CHECK-NEXT:    adds r2, r2, r7
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r8
+; CHECK-NEXT:    vstrw.32 q1, [r1, #16]
+; CHECK-NEXT:    adcs r4, r6
+; CHECK-NEXT:    vmov r6, s30
+; CHECK-NEXT:    adds r2, r2, r6
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, lr
+; CHECK-NEXT:    adcs r0, r4
+; CHECK-NEXT:    vmov q0[3], q0[1], r0, r12
 ; CHECK-NEXT:    vstrw.32 q0, [r1]
 ; CHECK-NEXT:    add sp, #24
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    pop {r4, pc}
+; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, pc}
 entry:
   %l1 = load <12 x i64>, <12 x i64>* %src, align 4
   %s1 = shufflevector <12 x i64> %l1, <12 x i64> undef, <4 x i32> <i32 0, i32 3, i32 6, i32 9>
   %s2 = shufflevector <12 x i64> %l1, <12 x i64> undef, <4 x i32> <i32 1, i32 4, i32 7, i32 10>
   %s3 = shufflevector <12 x i64> %l1, <12 x i64> undef, <4 x i32> <i32 2, i32 5, i32 8, i32 11>
   %a1 = add <4 x i64> %s1, %s2
   %a = add <4 x i64> %a1, %s3
   store <4 x i64> %a, <4 x i64> *%dst
entry:
   %a = fadd <4 x half> %a1, %s3
   store <4 x half> %a, <4 x half> *%dst
   ret void
 }
 
 define void @vld3_v8f16(<24 x half> *%src, <8 x half> *%dst) {
 ; CHECK-LABEL: vld3_v8f16:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, lr}
-; CHECK-NEXT:    push {r4, lr}
-; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13}
-; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13}
-; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    vldrw.u32 q2, [r0, #16]
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
+; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12}
+; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12}
+; CHECK-NEXT:    vldrw.u32 q2, [r0]
+; CHECK-NEXT:    vldrw.u32 q1, [r0, #16]
 ; CHECK-NEXT:    vldrw.u32 q4, [r0, #32]
-; CHECK-NEXT:    vmovx.f16 s0, s6
-; CHECK-NEXT:    vmov r3, s5
+; CHECK-NEXT:    vmovx.f16 s0, s10
+; CHECK-NEXT:    vmov r3, s9
 ; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    vmov.16 q0[0], r3
 ; CHECK-NEXT:    vmov.16 q0[1], r2
-; CHECK-NEXT:    vmov r2, s8
-; CHECK-NEXT:    vmovx.f16 s12, s9
+; CHECK-NEXT:    vmov r2, s4
+; CHECK-NEXT:    vmovx.f16 s12, s5
 ; CHECK-NEXT:    vmov.16 q0[2], r2
 ; CHECK-NEXT:    vmov r2, s12
 ; CHECK-NEXT:    vmovx.f16 s12, s19
 ; CHECK-NEXT:    vmov.16 q0[3], r2
 ; CHECK-NEXT:    vmov r3, s18
-; CHECK-NEXT:    vmov.f32 s2, s11
+; CHECK-NEXT:    vmov.f32 s2, s7
 ; CHECK-NEXT:    vmovx.f16 s20, s16
 ; CHECK-NEXT:    vmov r0, s12
 ; CHECK-NEXT:    vmov.16 q3[6], r3
 ; CHECK-NEXT:    vmov.16 q3[7], r0
 ; CHECK-NEXT:    vmov r0, s20
+; CHECK-NEXT:    vmovx.f16 s24, s11
+; CHECK-NEXT:    vmov r4, s16
+; CHECK-NEXT:    vmov r5, s8
 ; CHECK-NEXT:    vmov.f32 s14, s16
-; CHECK-NEXT:    vmovx.f16 s24, s8
-; CHECK-NEXT:    vmov r4, s0
 ; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vmov.16 q5[4], r2
-; CHECK-NEXT:    vmov r2, s17
 ; CHECK-NEXT:    vmov.16 q5[5], r0
 ; CHECK-NEXT:    vmov r0, s19
-; CHECK-NEXT:    vmov lr, s22
+; CHECK-NEXT:    vmov r12, s22
 ; CHECK-NEXT:    vmovx.f16 s20, s17
-; CHECK-NEXT:    vmov r3, s20
-; CHECK-NEXT:    vmov.16 q5[6], r3
+; CHECK-NEXT:    vmov r2, s20
+; CHECK-NEXT:    vmov.16 q5[6], r2
+; CHECK-NEXT:    vmov r2, s10
 ; CHECK-NEXT:    vmov.16 q5[7], r0
-; CHECK-NEXT:    vmov r0, s16
-; CHECK-NEXT:    vmov r12, s23
-; CHECK-NEXT:    vmovx.f16 s20, s10
-; CHECK-NEXT:    vmov r3, s20
-; CHECK-NEXT:    vmov.16 q5[4], r3
-; CHECK-NEXT:    vmov.16 q5[5], r0
-; CHECK-NEXT:    vmov r3, s22
-; CHECK-NEXT:    vmovx.f16 s20, s18
+; CHECK-NEXT:    vmov lr, s23
+; CHECK-NEXT:    vmovx.f16 s20, s8
 ; CHECK-NEXT:    vmov r0, s20
-; CHECK-NEXT:    vmov.16 q4[6], r2
-; CHECK-NEXT:    vmov.16 q4[7], r0
-; CHECK-NEXT:    vmovx.f16 s20, s5
-; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov r2, s20
 ; CHECK-NEXT:    vmov.16 q5[0], r0
+; CHECK-NEXT:    vmov r0, s24
 ; CHECK-NEXT:    vmov.16 q5[1], r2
-; CHECK-NEXT:    vmov r0, s7
 ; CHECK-NEXT:    vmov.16 q5[2], r0
-; CHECK-NEXT:    vmov r0, s24
+; CHECK-NEXT:    vmov r0, s5
 ; CHECK-NEXT:    vmov.16 q5[3], r0
-; CHECK-NEXT:    vmov r0, s10
-; CHECK-NEXT:    vmovx.f16 s24, s11
-; CHECK-NEXT:    vmov.16 q5[4], r0
-; CHECK-NEXT:    vmov r0, s24
-; CHECK-NEXT:    vmovx.f16 s24, s4
-; CHECK-NEXT:    vmov r2, s24
-; CHECK-NEXT:    vmov.16 q5[5], r0
-; CHECK-NEXT:    vmov r0, s6
-; CHECK-NEXT:    vmov.16 q6[0], r2
+; CHECK-NEXT:    vmov r0, s20
+; CHECK-NEXT:    vmovx.f16 s20, s6
+; CHECK-NEXT:    vmov r2, s20
+; CHECK-NEXT:    vmov r3, s21
+; CHECK-NEXT:    vmov.16 q5[4], r2
+; CHECK-NEXT:    vmov.16 q5[5], r4
+; CHECK-NEXT:    vmov r2, s22
+; CHECK-NEXT:    vmovx.f16 s20, s9
+; CHECK-NEXT:    vmov r4, s20
+; CHECK-NEXT:    vmov.16 q5[0], r5
+; CHECK-NEXT:    vmov.16 q5[1], r4
+; CHECK-NEXT:    vmov r4, s11
+; CHECK-NEXT:    vmovx.f16 s8, s4
+; CHECK-NEXT:    vmov.16 q5[2], r4
+; CHECK-NEXT:    vmov r4, s8
+; CHECK-NEXT:    vmovx.f16 s8, s18
+; CHECK-NEXT:    vmov.16 q5[3], r4
+; CHECK-NEXT:    vmov r4, s6
+; CHECK-NEXT:    vmov.16 q5[4], r4
+; CHECK-NEXT:    vmov r4, s17
 ; CHECK-NEXT:    vmovx.f16 s4, s7
-; CHECK-NEXT:    vmov.16 q6[1], r0
-; CHECK-NEXT:    vmov r0, s4
-; CHECK-NEXT:    vmov.32 q1[0], r4
-; CHECK-NEXT:    vmov.16 q6[2], r0
-; CHECK-NEXT:    vmov r0, s9
-; CHECK-NEXT:    vmov.16 q6[3], r0
-; CHECK-NEXT:    vmov r4, s1
-; CHECK-NEXT:    vmov r2, s24
-; CHECK-NEXT:    vmov.32 q1[1], r4
-; CHECK-NEXT:    vmov r0, s25
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov.32 q0[1], r0
-; CHECK-NEXT:    vmov.32 q1[2], lr
-; CHECK-NEXT:    vmov.32 q0[2], r3
+; CHECK-NEXT:    vmov r5, s8
+; CHECK-NEXT:    vmov.16 q2[6], r4
+; CHECK-NEXT:    vmov r4, s4
+; CHECK-NEXT:    vmov.16 q2[7], r5
+; CHECK-NEXT:    vmov.16 q5[5], r4
+; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov r5, s1
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r0
+; CHECK-NEXT:    vmov q1[2], q1[0], r12, r4
 ; CHECK-NEXT:    vmov r4, s15
-; CHECK-NEXT:    vmov.f32 s23, s19
-; CHECK-NEXT:    vmov.32 q0[3], r12
-; CHECK-NEXT:    vmov.32 q1[3], r4
+; CHECK-NEXT:    vmov.f32 s23, s11
+; CHECK-NEXT:    vmov q0[3], q0[1], lr, r3
+; CHECK-NEXT:    vmov q1[3], q1[1], r4, r5
 ; CHECK-NEXT:    vadd.f16 q0, q5, q0
 ; CHECK-NEXT:    vadd.f16 q0, q0, q1
 ; CHECK-NEXT:    vstrw.32 q0, [r1]
-; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13}
-; CHECK-NEXT:    pop {r4, pc}
+; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %l1 = load <24 x half>, <24 x half>* %src, align 4
   %s1 = shufflevector <24 x half> %l1, <24 x half> undef, <8 x i32> <i32 0, i32 3, i32 6, i32 9, i32 12, i32 15, i32 18, i32 21>
   %s2 = shufflevector <24 x half> %l1, <24 x half> undef, <8 x i32> <i32 1, i32 4, i32 7, i32 10, i32 13, i32 16, i32 19, i32 22>
   %s3 = shufflevector <24 x half> %l1, <24 x half> undef, <8 x i32> <i32 2, i32 5, i32 8, i32 11, i32 14, i32 17, i32 20, i32 23>
   %a1 = fadd <8 x half> %s1, %s2
   %a = fadd <8 x half> %a1, %s3
   store <8 x half> %a, <8 x half> *%dst
   ret void
 }
 
 define void @vld3_v16f16(<48 x half> *%src, <16 x half> *%dst) {
 ; CHECK-LABEL: vld3_v16f16:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-NEXT:    push {r4, r5, r7, lr}
-; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13}
-; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13}
-; CHECK-NEXT:    vldrw.u32 q4, [r0, #80]
-; CHECK-NEXT:    vldrw.u32 q2, [r0, #48]
-; CHECK-NEXT:    vldrw.u32 q3, [r0, #64]
-; CHECK-NEXT:    vmovx.f16 s0, s19
-; CHECK-NEXT:    vmovx.f16 s4, s16
-; CHECK-NEXT:    vmov r2, s18
-; CHECK-NEXT:    vmovx.f16 s20, s13
+; CHECK-NEXT:    .save {r4, r5, r6, lr}
+; CHECK-NEXT:    push {r4, r5, r6, lr}
+; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
+; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
+; CHECK-NEXT:    vldrw.u32 q1, [r0, #80]
+; CHECK-NEXT:    vldrw.u32 q3, [r0, #48]
+; CHECK-NEXT:    vldrw.u32 q4, [r0, #64]
+; CHECK-NEXT:    vmovx.f16 s0, s7
+; CHECK-NEXT:    vmovx.f16 s8, s4
+; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    vmovx.f16 s20, s17
 ; CHECK-NEXT:    vmov r3, s0
 ; CHECK-NEXT:    vmov.16 q0[6], r2
-; CHECK-NEXT:    vmov r12, s4
-; CHECK-NEXT:    vmovx.f16 s4, s10
-; CHECK-NEXT:    vmov r2, s9
+; CHECK-NEXT:    vmov r12, s8
+; CHECK-NEXT:    vmovx.f16 s8, s14
+; CHECK-NEXT:    vmov r2, s13
 ; CHECK-NEXT:    vmov.16 q0[7], r3
-; CHECK-NEXT:    vmov r3, s4
-; CHECK-NEXT:    vmov.16 q1[0], r2
-; CHECK-NEXT:    vmov.16 q1[1], r3
-; CHECK-NEXT:    vmov r2, s12
-; CHECK-NEXT:    vmov.16 q1[2], r2
+; CHECK-NEXT:    vmov r3, s8
+; CHECK-NEXT:    vmov.16 q2[0], r2
+; CHECK-NEXT:    vmov.16 q2[1], r3
+; CHECK-NEXT:    vmov r2, s16
+; CHECK-NEXT:    vmov.16 q2[2], r2
 ; CHECK-NEXT:    vmov r2, s20
-; CHECK-NEXT:    vmov.16 q1[3], r2
-; CHECK-NEXT:    vmov r3, s16
-; CHECK-NEXT:    vmov.f32 s6, s15
-; CHECK-NEXT:    vmovx.f16 s24, s12
-; CHECK-NEXT:    vmov.f32 s2, s16
-; CHECK-NEXT:    vmovx.f16 s16, s18
-; CHECK-NEXT:    vmov r4, s16
-; CHECK-NEXT:    vmov r5, s4
-; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    vmov.16 q2[3], r2
+; CHECK-NEXT:    vmov r3, s5
+; CHECK-NEXT:    vmov.f32 s10, s19
+; CHECK-NEXT:    vmovx.f16 s24, s13
+; CHECK-NEXT:    vmovx.f16 s28, s16
+; CHECK-NEXT:    vmov.f32 s2, s4
+; CHECK-NEXT:    vmov r4, s7
+; CHECK-NEXT:    vmov r2, s10
 ; CHECK-NEXT:    vmov.16 q5[4], r2
 ; CHECK-NEXT:    vmov.16 q5[5], r12
-; CHECK-NEXT:    vmov lr, s22
-; CHECK-NEXT:    vmovx.f16 s20, s14
-; CHECK-NEXT:    vmov r2, s20
-; CHECK-NEXT:    vmov.16 q5[4], r2
-; CHECK-NEXT:    vmov r2, s19
-; CHECK-NEXT:    vmov.16 q5[5], r3
 ; CHECK-NEXT:    vmov r12, s22
-; CHECK-NEXT:    vmovx.f16 s20, s17
-; CHECK-NEXT:    vmov r3, s20
+; CHECK-NEXT:    vmovx.f16 s20, s6
+; CHECK-NEXT:    vmov r2, s20
 ; CHECK-NEXT:    vmov.16 q5[6], r3
-; CHECK-NEXT:    vmov r3, s17
+; CHECK-NEXT:    vmov r3, s12
 ; CHECK-NEXT:    vmov.16 q5[7], r2
-; CHECK-NEXT:    vmov.16 q4[6], r3
-; CHECK-NEXT:    vmov r2, s23
-; CHECK-NEXT:    vmov.16 q4[7], r4
-; CHECK-NEXT:    vmovx.f16 s20, s9
-; CHECK-NEXT:    vmov r4, s8
-; CHECK-NEXT:    vmov r3, s20
-; CHECK-NEXT:    vmov.16 q5[0], r4
-; CHECK-NEXT:    vmov.16 q5[1], r3
-; CHECK-NEXT:    vmov r3, s11
-; CHECK-NEXT:    vmov.16 q5[2], r3
-; CHECK-NEXT:    vmov r3, s24
-; CHECK-NEXT:    vmov.16 q5[3], r3
-; CHECK-NEXT:    vmov r3, s14
-; CHECK-NEXT:    vmovx.f16 s24, s15
-; CHECK-NEXT:    vmov.16 q5[4], r3
-; CHECK-NEXT:    vmov r3, s24
-; CHECK-NEXT:    vmovx.f16 s24, s8
-; CHECK-NEXT:    vmov.16 q5[5], r3
-; CHECK-NEXT:    vmov r3, s24
-; CHECK-NEXT:    vmov r4, s10
+; CHECK-NEXT:    vmov r2, s24
 ; CHECK-NEXT:    vmov.16 q6[0], r3
-; CHECK-NEXT:    vmovx.f16 s8, s11
-; CHECK-NEXT:    vmov.16 q6[1], r4
-; CHECK-NEXT:    vmov r3, s8
-; CHECK-NEXT:    vmov.32 q2[0], r5
-; CHECK-NEXT:    vmov.16 q6[2], r3
-; CHECK-NEXT:    vmov r3, s13
-; CHECK-NEXT:    vmov.16 q6[3], r3
-; CHECK-NEXT:    vmov r5, s5
-; CHECK-NEXT:    vmov r3, s24
-; CHECK-NEXT:    vmov.32 q2[1], r5
-; CHECK-NEXT:    vmov r5, s3
-; CHECK-NEXT:    vmov.32 q0[0], r3
-; CHECK-NEXT:    vmov r4, s25
-; CHECK-NEXT:    vmov.32 q2[2], lr
-; CHECK-NEXT:    vmov.32 q0[1], r4
-; CHECK-NEXT:    vmov.f32 s23, s19
-; CHECK-NEXT:    vmov.32 q0[2], r12
-; CHECK-NEXT:    vldrw.u32 q4, [r0, #32]
-; CHECK-NEXT:    vmov.32 q0[3], r2
-; CHECK-NEXT:    vmov.32 q2[3], r5
-; CHECK-NEXT:    vadd.f16 q0, q5, q0
-; CHECK-NEXT:    vldrw.u32 q1, [r0, #16]
-; CHECK-NEXT:    vadd.f16 q0, q0, q2
-; CHECK-NEXT:    vldrw.u32 q2, [r0]
+; CHECK-NEXT:    vmov.16 q6[1], r2
+; CHECK-NEXT:    vmov r2, s15
+; CHECK-NEXT:    vmov.16 q6[2], r2
+; CHECK-NEXT:    vmov r2, s28
+; CHECK-NEXT:    vmov.16 q6[3], r2
+; CHECK-NEXT:    vmov r2, s18
+; CHECK-NEXT:    vmovx.f16 s28, s19
+; CHECK-NEXT:    vmov.16 q6[4], r2
+; CHECK-NEXT:    vmov r2, s28
+; CHECK-NEXT:    vmovx.f16 s28, s18
+; CHECK-NEXT:    vmov.16 q6[5], r2
+; CHECK-NEXT:    vmov r2, s28
+; CHECK-NEXT:    vmov r3, s4
+; CHECK-NEXT:    vmov.16 q7[4], r2
+; CHECK-NEXT:    vmov.16 q7[5], r3
+; CHECK-NEXT:    vmov r2, s14
+; CHECK-NEXT:    vmov lr, s30
+; CHECK-NEXT:    vmovx.f16 s28, s12
+; CHECK-NEXT:    vmov r3, s28
+; CHECK-NEXT:    vmovx.f16 s12, s15
+; CHECK-NEXT:    vmov.16 q7[0], r3
+; CHECK-NEXT:    vmovx.f16 s4, s5
+; CHECK-NEXT:    vmov.16 q7[1], r2
+; CHECK-NEXT:    vmov r2, s12
+; CHECK-NEXT:    vmov r5, s4
+; CHECK-NEXT:    vmov.16 q7[2], r2
+; CHECK-NEXT:    vmov r2, s17
+; CHECK-NEXT:    vmov.16 q1[6], r5
+; CHECK-NEXT:    vmov.16 q7[3], r2
+; CHECK-NEXT:    vmov.16 q1[7], r4
+; CHECK-NEXT:    vmov r2, s28
+; CHECK-NEXT:    vldrw.u32 q3, [r0]
+; CHECK-NEXT:    vmov r3, s3
+; CHECK-NEXT:    vmov q0[2], q0[0], lr, r2
+; CHECK-NEXT:    vmov r6, s29
+; CHECK-NEXT:    vmov r4, s7
+; CHECK-NEXT:    vmov r5, s8
+; CHECK-NEXT:    vmov q0[3], q0[1], r4, r6
+; CHECK-NEXT:    vmov q1[2], q1[0], r12, r5
+; CHECK-NEXT:    vmov r5, s9
+; CHECK-NEXT:    vmov.f32 s27, s23
+; CHECK-NEXT:    vmov q1[3], q1[1], r3, r5
+; CHECK-NEXT:    vadd.f16 q0, q6, q0
+; CHECK-NEXT:    vldrw.u32 q2, [r0, #16]
+; CHECK-NEXT:    vadd.f16 q0, q0, q1
+; CHECK-NEXT:    vldrw.u32 q1, [r0, #32]
 ; CHECK-NEXT:    vstrw.32 q0, [r1, #16]
-; CHECK-NEXT:    vmovx.f16 s0, s19
-; CHECK-NEXT:    vmov r0, s18
-; CHECK-NEXT:    vmovx.f16 s12, s16
+; CHECK-NEXT:    vmovx.f16 s20, s9
+; CHECK-NEXT:    vmovx.f16 s0, s7
+; CHECK-NEXT:    vmov r0, s6
 ; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    vmov.16 q0[6], r0
+; CHECK-NEXT:    vmovx.f16 s16, s4
 ; CHECK-NEXT:    vmov.16 q0[7], r2
-; CHECK-NEXT:    vmov r0, s12
-; CHECK-NEXT:    vmovx.f16 s12, s10
-; CHECK-NEXT:    vmov r2, s9
-; CHECK-NEXT:    vmov r3, s12
-; CHECK-NEXT:    vmov.16 q3[0], r2
-; CHECK-NEXT:    vmov.16 q3[1], r3
-; CHECK-NEXT:    vmov r2, s4
-; CHECK-NEXT:    vmovx.f16 s20, s5
-; CHECK-NEXT:    vmov.16 q3[2], r2
-; CHECK-NEXT:    vmov r2, s20
-; CHECK-NEXT:    vmovx.f16 s24, s4
-; CHECK-NEXT:    vmov.16 q3[3], r2
+; CHECK-NEXT:    vmov r0, s16
+; CHECK-NEXT:    vmovx.f16 s16, s14
+; CHECK-NEXT:    vmov r2, s13
+; CHECK-NEXT:    vmovx.f16 s24, s13
 ; CHECK-NEXT:    vmov r3, s16
-; CHECK-NEXT:    vmov.f32 s14, s7
-; CHECK-NEXT:    vmov.f32 s2, s16
-; CHECK-NEXT:    vmovx.f16 s16, s18
-; CHECK-NEXT:    vmov r4, s16
-; CHECK-NEXT:    vmov r2, s14
+; CHECK-NEXT:    vmov.16 q4[0], r2
+; CHECK-NEXT:    vmov.16 q4[1], r3
+; CHECK-NEXT:    vmov r2, s8
+; CHECK-NEXT:    vmov.16 q4[2], r2
+; CHECK-NEXT:    vmov r2, s20
+; CHECK-NEXT:    vmov.16 q4[3], r2
+; CHECK-NEXT:    vmovx.f16 s28, s8
+; CHECK-NEXT:    vmov.f32 s18, s11
+; CHECK-NEXT:    vmov.f32 s2, s4
+; CHECK-NEXT:    vmov r5, s7
+; CHECK-NEXT:    vmov r2, s18
 ; CHECK-NEXT:    vmov.16 q5[4], r2
+; CHECK-NEXT:    vmov r2, s5
 ; CHECK-NEXT:    vmov.16 q5[5], r0
-; CHECK-NEXT:    vmov r2, s22
+; CHECK-NEXT:    vmov r0, s22
 ; CHECK-NEXT:    vmovx.f16 s20, s6
-; CHECK-NEXT:    vmov r0, s20
-; CHECK-NEXT:    vmov.16 q5[4], r0
-; CHECK-NEXT:    vmov r0, s12
-; CHECK-NEXT:    vmov.16 q5[5], r3
-; CHECK-NEXT:    vmov r3, s19
-; CHECK-NEXT:    vmov r12, s22
-; CHECK-NEXT:    vmovx.f16 s20, s17
-; CHECK-NEXT:    vmov r5, s20
-; CHECK-NEXT:    vmov.16 q5[6], r5
-; CHECK-NEXT:    vmov r5, s17
+; CHECK-NEXT:    vmov r3, s20
+; CHECK-NEXT:    vmov.16 q5[6], r2
 ; CHECK-NEXT:    vmov.16 q5[7], r3
-; CHECK-NEXT:    vmov.16 q4[6], r5
-; CHECK-NEXT:    vmov r3, s23
-; CHECK-NEXT:    vmov.16 q4[7], r4
-; CHECK-NEXT:    vmovx.f16 s20, s9
-; CHECK-NEXT:    vmov r4, s8
-; CHECK-NEXT:    vmov r5, s20
-; CHECK-NEXT:    vmov.16 q5[0], r4
-; CHECK-NEXT:    vmov.16 q5[1], r5
-; CHECK-NEXT:    vmov r5, s11
-; CHECK-NEXT:    vmov.16 q5[2], r5
-; CHECK-NEXT:    vmov r5, s24
-; CHECK-NEXT:    vmov.16 q5[3], r5
-; CHECK-NEXT:    vmov r5, s6
-; CHECK-NEXT:    vmovx.f16 s24, s7
-; CHECK-NEXT:    vmov.16 q5[4], r5
-; CHECK-NEXT:    vmov r5, s24
-; CHECK-NEXT:    vmovx.f16 s24, s8
-; CHECK-NEXT:    vmov.16 q5[5], r5
-; CHECK-NEXT:    vmov r5, s24
-; CHECK-NEXT:    vmov r4, s10
-; CHECK-NEXT:    vmov.16 q6[0], r5
-; CHECK-NEXT:    vmovx.f16 s8, s11
-; CHECK-NEXT:    vmov.16 q6[1], r4
-; CHECK-NEXT:    vmov r5, s8
-; CHECK-NEXT:    vmov.16 q6[2], r5
-; CHECK-NEXT:    vmov r5, s5
-; CHECK-NEXT:    vmov.16 q6[3], r5
-; CHECK-NEXT:    vmov.32 q1[0], r0
-; CHECK-NEXT:    vmov r0, s13
-; CHECK-NEXT:    vmov r5, s24
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    vmov.32 q0[0], r5
-; CHECK-NEXT:    vmov r4, s25
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vmov.32 q0[1], r4
-; CHECK-NEXT:    vmov.f32 s23, s19
-; CHECK-NEXT:    vmov.32 q0[2], r12
-; CHECK-NEXT:    vmov.32 q1[3], r0
-; CHECK-NEXT:    vmov.32 q0[3], r3
-; CHECK-NEXT:    vadd.f16 q0, q5, q0
+; CHECK-NEXT:    vmov r3, s12
+; CHECK-NEXT:    vmov r2, s24
+; CHECK-NEXT:    vmov.16 q6[0], r3
+; CHECK-NEXT:    vmov.16 q6[1], r2
+; CHECK-NEXT:    vmov r2, s15
+; CHECK-NEXT:    vmov.16 q6[2], r2
+; CHECK-NEXT:    vmov r2, s28
+; CHECK-NEXT:    vmov.16 q6[3], r2
+; CHECK-NEXT:    vmov r2, s10
+; CHECK-NEXT:    vmovx.f16 s28, s11
+; CHECK-NEXT:    vmov.16 q6[4], r2
+; CHECK-NEXT:    vmov r2, s28
+; CHECK-NEXT:    vmovx.f16 s28, s10
+; CHECK-NEXT:    vmov.16 q6[5], r2
+; CHECK-NEXT:    vmov r2, s28
+; CHECK-NEXT:    vmov r3, s4
+; CHECK-NEXT:    vmov.16 q7[4], r2
+; CHECK-NEXT:    vmov.16 q7[5], r3
+; CHECK-NEXT:    vmov r3, s14
+; CHECK-NEXT:    vmov r2, s30
+; CHECK-NEXT:    vmovx.f16 s28, s12
+; CHECK-NEXT:    vmov r6, s28
+; CHECK-NEXT:    vmovx.f16 s12, s15
+; CHECK-NEXT:    vmov.16 q7[0], r6
+; CHECK-NEXT:    vmovx.f16 s4, s5
+; CHECK-NEXT:    vmov.16 q7[1], r3
+; CHECK-NEXT:    vmov r3, s12
+; CHECK-NEXT:    vmov r4, s4
+; CHECK-NEXT:    vmov.16 q7[2], r3
+; CHECK-NEXT:    vmov r3, s9
+; CHECK-NEXT:    vmov.16 q1[6], r4
+; CHECK-NEXT:    vmov.16 q7[3], r3
+; CHECK-NEXT:    vmov.16 q1[7], r5
+; CHECK-NEXT:    vmov r3, s28
+; CHECK-NEXT:    vmov r4, s16
+; CHECK-NEXT:    vmov r5, s7
+; CHECK-NEXT:    vmov q1[2], q1[0], r0, r4
+; CHECK-NEXT:    vmov r4, s3
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, r3
+; CHECK-NEXT:    vmov r6, s29
+; CHECK-NEXT:    vmov r0, s17
+; CHECK-NEXT:    vmov q0[3], q0[1], r5, r6
+; CHECK-NEXT:    vmov.f32 s27, s23
+; CHECK-NEXT:    vmov q1[3], q1[1], r4, r0
+; CHECK-NEXT:    vadd.f16 q0, q6, q0
 ; CHECK-NEXT:    vadd.f16 q0, q0, q1
 ; CHECK-NEXT:    vstrw.32 q0, [r1]
-; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13}
-; CHECK-NEXT:    pop {r4, r5, r7, pc}
+; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
+; CHECK-NEXT:    pop {r4, r5, r6, pc}
 entry:
   %l1 = load <48 x half>, <48 x half>* %src, align 4
   %s1 = shufflevector <48 x half> %l1, <48 x half> undef, <16 x i32> <i32 0, i32 3, i32 6, i32 9, i32 12, i32 15, i32 18, i32 21, i32 24, i32 27, i32 30, i32 33, i32 36, i32 39, i32 42, i32 45>
   %s2 = shufflevector <48 x half> %l1, <48 x half> undef, <16 x i32> <i32 1, i32 4, i32 7, i32 10, i32 13, i32 16, i32 19, i32 22, i32 25, i32 28, i32 31, i32 34, i32 37, i32 40, i32 43, i32 46>
   %s3 = shufflevector <48 x half> %l1, <48 x half> undef, <16 x i32> <i32 2, i32 5, i32 8, i32 11, i32 14, i32 17, i32 20, i32 23, i32 26, i32 29, i32 32, i32 35, i32 38, i32 41, i32 44, i32 47>
   %a1 = fadd <16 x half> %s1, %s2
   %a = fadd <16 x half> %a1, %s3
   store <16 x half> %a, <16 x half> *%dst

llvm/test/CodeGen/Thumb2/mve-vld4-post.ll

 ; CHECK-NEXT:    vldrw.u32 q5, [r0, #-16]
 ; CHECK-NEXT:    vmov.f64 d2, d1
 ; CHECK-NEXT:    vmov.f64 d8, d7
 ; CHECK-NEXT:    vmov.f32 s17, s15
 ; CHECK-NEXT:    vmov.f32 s18, s22
 ; CHECK-NEXT:    vmov.f32 s14, s20
 ; CHECK-NEXT:    vmov.f32 s19, s23
 ; CHECK-NEXT:    vmov.f32 s15, s21
-; CHECK-NEXT:    vmov r2, s18
-; CHECK-NEXT:    vmov r3, s14
+; CHECK-NEXT:    vmov r2, s16
+; CHECK-NEXT:    vmov r3, s12
 ; CHECK-NEXT:    vmov.f32 s5, s3
 ; CHECK-NEXT:    vmov.f32 s6, s10
 ; CHECK-NEXT:    vmov.f32 s2, s8
 ; CHECK-NEXT:    vmov.f32 s3, s9
 ; CHECK-NEXT:    vmov.f32 s7, s11
-; CHECK-NEXT:    vmov r12, s19
-; CHECK-NEXT:    vmov lr, s15
-; CHECK-NEXT:    vmov r4, s6
-; CHECK-NEXT:    vmov r5, s2
-; CHECK-NEXT:    vmov r7, s0
+; CHECK-NEXT:    vmov r12, s17
+; CHECK-NEXT:    vmov lr, s13
+; CHECK-NEXT:    vmov r4, s4
+; CHECK-NEXT:    vmov r5, s0
+; CHECK-NEXT:    vmov r7, s2
 ; CHECK-NEXT:    adds r6, r3, r2
-; CHECK-NEXT:    vmov r2, s7
-; CHECK-NEXT:    vmov r3, s3
+; CHECK-NEXT:    vmov r2, s5
+; CHECK-NEXT:    vmov r3, s1
 ; CHECK-NEXT:    adc.w r12, r12, lr
 ; CHECK-NEXT:    adds r5, r5, r4
-; CHECK-NEXT:    vmov r4, s16
+; CHECK-NEXT:    vmov r4, s18
 ; CHECK-NEXT:    adcs r2, r3
 ; CHECK-NEXT:    adds.w lr, r5, r6
 ; CHECK-NEXT:    adc.w r12, r12, r2
-; CHECK-NEXT:    vmov r2, s12
-; CHECK-NEXT:    vmov r6, s17
-; CHECK-NEXT:    vmov r5, s13
-; CHECK-NEXT:    vmov r3, s4
+; CHECK-NEXT:    vmov r2, s14
+; CHECK-NEXT:    vmov r6, s19
+; CHECK-NEXT:    vmov r5, s15
+; CHECK-NEXT:    vmov r3, s6
 ; CHECK-NEXT:    adds r2, r2, r4
-; CHECK-NEXT:    vmov r4, s1
+; CHECK-NEXT:    vmov r4, s3
 ; CHECK-NEXT:    adcs r6, r5
-; CHECK-NEXT:    vmov r5, s5
+; CHECK-NEXT:    vmov r5, s7
 ; CHECK-NEXT:    adds r3, r3, r7
 ; CHECK-NEXT:    adcs r4, r5
 ; CHECK-NEXT:    adds r2, r2, r3
 ; CHECK-NEXT:    adc.w r3, r4, r6
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov.32 q0[1], r3
-; CHECK-NEXT:    vmov.32 q0[2], lr
-; CHECK-NEXT:    vmov.32 q0[3], r12
+; CHECK-NEXT:    vmov q0[2], q0[0], r2, lr
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r12
 ; CHECK-NEXT:    vstrw.32 q0, [r1]
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11}
 ; CHECK-NEXT:    add sp, #4
 ; CHECK-NEXT:    pop {r4, r5, r6, r7, pc}
 entry:
   %l1 = load <8 x i64>, <8 x i64>* %src, align 4
   %s1 = shufflevector <8 x i64> %l1, <8 x i64> undef, <2 x i32> <i32 0, i32 4>
   %s2 = shufflevector <8 x i64> %l1, <8 x i64> undef, <2 x i32> <i32 1, i32 5>

llvm/test/CodeGen/Thumb2/mve-vld4.ll

entry:
   ret void
 }
 
 define void @vld4_v4i16(<16 x i16> *%src, <4 x i16> *%dst) {
 ; CHECK-LABEL: vld4_v4i16:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .vsave {d8, d9}
 ; CHECK-NEXT:    vpush {d8, d9}
-; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    vldrw.u32 q0, [r0, #16]
-; CHECK-NEXT:    vmov.u16 r2, q1[3]
-; CHECK-NEXT:    vmov.u16 r0, q0[3]
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[7]
-; CHECK-NEXT:    vmov.32 q2[1], r2
-; CHECK-NEXT:    vmov.32 q2[2], r0
+; CHECK-NEXT:    vldrw.u32 q0, [r0]
+; CHECK-NEXT:    vldrw.u32 q1, [r0, #16]
+; CHECK-NEXT:    vmov.u16 r2, q0[3]
+; CHECK-NEXT:    vmov.u16 r0, q1[3]
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r2
 ; CHECK-NEXT:    vmov.u16 r0, q0[7]
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[2]
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[6]
-; CHECK-NEXT:    vmov.32 q3[1], r0
+; CHECK-NEXT:    vmov.u16 r2, q1[7]
+; CHECK-NEXT:    vmov q2[3], q2[1], r2, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-NEXT:    vmov.32 q3[2], r0
+; CHECK-NEXT:    vmov.u16 r2, q1[2]
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[6]
-; CHECK-NEXT:    vmov.32 q3[3], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[0]
-; CHECK-NEXT:    vadd.i32 q2, q3, q2
-; CHECK-NEXT:    vmov.32 q3[0], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[4]
-; CHECK-NEXT:    vmov.32 q3[1], r0
+; CHECK-NEXT:    vmov.u16 r2, q1[6]
+; CHECK-NEXT:    vmov q3[3], q3[1], r2, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[1]
-; CHECK-NEXT:    vmov.32 q4[0], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[5]
-; CHECK-NEXT:    vmov.32 q4[1], r0
+; CHECK-NEXT:    vmov.u16 r2, q1[0]
+; CHECK-NEXT:    vadd.i32 q2, q3, q2
+; CHECK-NEXT:    vmov q3[2], q3[0], r2, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-NEXT:    vmov.32 q4[2], r0
+; CHECK-NEXT:    vmov.u16 r2, q1[1]
+; CHECK-NEXT:    vmov q4[2], q4[0], r2, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[5]
-; CHECK-NEXT:    vmov.32 q4[3], r0
+; CHECK-NEXT:    vmov.u16 r2, q1[5]
+; CHECK-NEXT:    vmov q4[3], q4[1], r2, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[4]
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    vmov.u16 r2, q1[4]
+; CHECK-NEXT:    vmov q3[3], q3[1], r2, r0
 ; CHECK-NEXT:    vadd.i32 q0, q3, q4
 ; CHECK-NEXT:    vadd.i32 q0, q0, q2
 ; CHECK-NEXT:    vstrh.32 q0, [r1]
 ; CHECK-NEXT:    vpop {d8, d9}
 ; CHECK-NEXT:    bx lr
 entry:
   %l1 = load <16 x i16>, <16 x i16>* %src, align 4
   %s1 = shufflevector <16 x i16> %l1, <16 x i16> undef, <4 x i32> <i32 0, i32 4, i32 8, i32 12>
entry:
   ret void
 }
 
 define void @vld4_v4i8(<16 x i8> *%src, <4 x i8> *%dst) {
 ; CHECK-LABEL: vld4_v4i8:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vldrw.u32 q0, [r0]
 ; CHECK-NEXT:    vmov.u8 r0, q0[2]
-; CHECK-NEXT:    vrev32.8 q2, q0
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov.u8 r2, q0[10]
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, r0
 ; CHECK-NEXT:    vmov.u8 r0, q0[6]
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[10]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.u8 r0, q0[14]
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    vmov.u8 r2, q0[14]
+; CHECK-NEXT:    vrev32.8 q2, q0
+; CHECK-NEXT:    vmov q1[3], q1[1], r2, r0
 ; CHECK-NEXT:    vadd.i32 q1, q1, q2
 ; CHECK-NEXT:    vrev16.8 q2, q0
 ; CHECK-NEXT:    vadd.i32 q0, q0, q2
 ; CHECK-NEXT:    vadd.i32 q0, q0, q1
 ; CHECK-NEXT:    vstrb.32 q0, [r1]
 ; CHECK-NEXT:    bx lr
 entry:
   %l1 = load <16 x i8>, <16 x i8>* %src, align 4

 ; CHECK-NEXT:    vldrw.u32 q0, [r0]
 ; CHECK-NEXT:    vldrw.u32 q2, [r0, #32]
 ; CHECK-NEXT:    vmov.f64 d8, d7
 ; CHECK-NEXT:    vmov.f32 s17, s15
 ; CHECK-NEXT:    vmov.f32 s18, s22
 ; CHECK-NEXT:    vmov.f32 s14, s20
 ; CHECK-NEXT:    vmov.f32 s19, s23
 ; CHECK-NEXT:    vmov.f32 s15, s21
-; CHECK-NEXT:    vmov r3, s18
-; CHECK-NEXT:    vmov r0, s14
+; CHECK-NEXT:    vmov r3, s16
+; CHECK-NEXT:    vmov r0, s12
+; CHECK-NEXT:    vmov r12, s17
+; CHECK-NEXT:    vmov r2, s13
 ; CHECK-NEXT:    vmov.f64 d2, d1
-; CHECK-NEXT:    vmov r12, s19
-; CHECK-NEXT:    vmov r2, s15
 ; CHECK-NEXT:    vmov.f32 s5, s3
 ; CHECK-NEXT:    vmov.f32 s6, s10
 ; CHECK-NEXT:    vmov.f32 s2, s8
 ; CHECK-NEXT:    vmov.f32 s3, s9
 ; CHECK-NEXT:    vmov.f32 s7, s11
-; CHECK-NEXT:    vmov r4, s2
-; CHECK-NEXT:    vmov r5, s4
-; CHECK-NEXT:    vmov r6, s0
+; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov r5, s6
+; CHECK-NEXT:    vmov r6, s2
 ; CHECK-NEXT:    adds.w lr, r0, r3
-; CHECK-NEXT:    vmov r3, s7
-; CHECK-NEXT:    vmov r0, s3
+; CHECK-NEXT:    vmov r3, s5
+; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    adc.w r12, r12, r2
-; CHECK-NEXT:    vmov r2, s6
+; CHECK-NEXT:    vmov r2, s4
 ; CHECK-NEXT:    adds r2, r2, r4
-; CHECK-NEXT:    vmov r4, s13
+; CHECK-NEXT:    vmov r4, s15
 ; CHECK-NEXT:    adcs r0, r3
 ; CHECK-NEXT:    adds.w lr, lr, r2
 ; CHECK-NEXT:    adc.w r12, r12, r0
-; CHECK-NEXT:    vmov r0, s16
-; CHECK-NEXT:    vmov r2, s12
-; CHECK-NEXT:    vmov r3, s17
+; CHECK-NEXT:    vmov r0, s18
+; CHECK-NEXT:    vmov r2, s14
+; CHECK-NEXT:    vmov r3, s19
 ; CHECK-NEXT:    adds r0, r0, r2
 ; CHECK-NEXT:    adc.w r2, r4, r3
-; CHECK-NEXT:    vmov r3, s5
-; CHECK-NEXT:    vmov r4, s1
+; CHECK-NEXT:    vmov r3, s7
+; CHECK-NEXT:    vmov r4, s3
 ; CHECK-NEXT:    adds r5, r5, r6
 ; CHECK-NEXT:    adcs r3, r4
 ; CHECK-NEXT:    adds r0, r0, r5
 ; CHECK-NEXT:    adcs r2, r3
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.32 q0[2], lr
-; CHECK-NEXT:    vmov.32 q0[3], r12
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, lr
+; CHECK-NEXT:    vmov q0[3], q0[1], r2, r12
 ; CHECK-NEXT:    vstrw.32 q0, [r1]
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11}
 ; CHECK-NEXT:    pop {r4, r5, r6, pc}
 entry:
   %l1 = load <8 x i64>, <8 x i64>* %src, align 4
   %s1 = shufflevector <8 x i64> %l1, <8 x i64> undef, <2 x i32> <i32 0, i32 4>
   %s2 = shufflevector <8 x i64> %l1, <8 x i64> undef, <2 x i32> <i32 1, i32 5>
   %s3 = shufflevector <8 x i64> %l1, <8 x i64> undef, <2 x i32> <i32 2, i32 6>
   %s4 = shufflevector <8 x i64> %l1, <8 x i64> undef, <2 x i32> <i32 3, i32 7>
   %a1 = add <2 x i64> %s1, %s2
   %a2 = add <2 x i64> %s3, %s4
   %a3 = add <2 x i64> %a1, %a2
   store <2 x i64> %a3, <2 x i64> *%dst
   ret void
 }
 
 define void @vld4_v4i64(<16 x i64> *%src, <4 x i64> *%dst) {
 ; CHECK-LABEL: vld4_v4i64:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, lr}
-; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, lr}
+; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, lr}
+; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, lr}
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    .pad #72
-; CHECK-NEXT:    sub sp, #72
+; CHECK-NEXT:    .pad #64
+; CHECK-NEXT:    sub sp, #64
 ; CHECK-NEXT:    vldrw.u32 q1, [r0, #64]
 ; CHECK-NEXT:    vldrw.u32 q0, [r0, #96]
-; CHECK-NEXT:    vldrw.u32 q5, [r0, #80]
-; CHECK-NEXT:    vldrw.u32 q7, [r0, #16]
-; CHECK-NEXT:    vmov.f64 d8, d3
-; CHECK-NEXT:    vstrw.32 q0, [sp, #32] @ 16-byte Spill
-; CHECK-NEXT:    vstrw.32 q1, [sp, #48] @ 16-byte Spill
-; CHECK-NEXT:    vmov.f32 s17, s7
-; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    vmov.f32 s18, s2
-; CHECK-NEXT:    vmov.f32 s19, s3
-; CHECK-NEXT:    vldrw.u32 q0, [r0, #112]
-; CHECK-NEXT:    vmov.f64 d12, d11
+; CHECK-NEXT:    vldrw.u32 q4, [r0, #80]
+; CHECK-NEXT:    vldrw.u32 q6, [r0, #16]
+; CHECK-NEXT:    vmov.f64 d4, d3
 ; CHECK-NEXT:    vstrw.32 q0, [sp, #16] @ 16-byte Spill
-; CHECK-NEXT:    vmov.f32 s25, s23
-; CHECK-NEXT:    vmov.f32 s26, s2
-; CHECK-NEXT:    vmov.f64 d6, d3
-; CHECK-NEXT:    vmov.f32 s27, s3
-; CHECK-NEXT:    vldrw.u32 q0, [r0, #32]
-; CHECK-NEXT:    vmov.f32 s13, s7
-; CHECK-NEXT:    vmov.f32 s14, s2
+; CHECK-NEXT:    vldrw.u32 q3, [r0]
+; CHECK-NEXT:    vstrw.32 q1, [sp, #32] @ 16-byte Spill
+; CHECK-NEXT:    vmov.f32 s9, s7
+; CHECK-NEXT:    vldrw.u32 q1, [r0, #32]
+; CHECK-NEXT:    vmov.f32 s10, s2
+; CHECK-NEXT:    vmov.f32 s11, s3
+; CHECK-NEXT:    vldrw.u32 q0, [r0, #112]
+; CHECK-NEXT:    vmov.f64 d14, d9
+; CHECK-NEXT:    vstrw.32 q2, [sp, #48] @ 16-byte Spill
 ; CHECK-NEXT:    vstrw.32 q0, [sp] @ 16-byte Spill
-; CHECK-NEXT:    vmov.f64 d4, d15
-; CHECK-NEXT:    vmov.f32 s15, s3
+; CHECK-NEXT:    vmov.f32 s29, s19
+; CHECK-NEXT:    vmov.f32 s30, s2
+; CHECK-NEXT:    vmov.f64 d4, d13
+; CHECK-NEXT:    vmov.f32 s31, s3
 ; CHECK-NEXT:    vldrw.u32 q0, [r0, #48]
-; CHECK-NEXT:    vmov.f32 s9, s31
+; CHECK-NEXT:    vmov.f32 s9, s27
 ; CHECK-NEXT:    vmov.f32 s10, s2
-; CHECK-NEXT:    vmov.f32 s30, s0
+; CHECK-NEXT:    vmov.f32 s26, s0
 ; CHECK-NEXT:    vmov.f32 s11, s3
-; CHECK-NEXT:    vmov.f32 s31, s1
+; CHECK-NEXT:    vmov.f32 s27, s1
 ; CHECK-NEXT:    vldrw.u32 q0, [sp] @ 16-byte Reload
-; CHECK-NEXT:    vmov r3, s10
-; CHECK-NEXT:    vmov r0, s30
-; CHECK-NEXT:    vmov.f32 s6, s0
-; CHECK-NEXT:    vmov.f32 s7, s1
-; CHECK-NEXT:    vldrw.u32 q0, [sp, #16] @ 16-byte Reload
-; CHECK-NEXT:    vmov r4, s6
-; CHECK-NEXT:    vstrw.32 q1, [sp] @ 16-byte Spill
-; CHECK-NEXT:    vmov r12, s11
-; CHECK-NEXT:    vmov r2, s31
-; CHECK-NEXT:    vmov.f32 s22, s0
-; CHECK-NEXT:    vmov.f32 s23, s1
+; CHECK-NEXT:    vmov r3, s8
+; CHECK-NEXT:    vmov r0, s24
+; CHECK-NEXT:    vmov r12, s9
+; CHECK-NEXT:    vmov r2, s25
+; CHECK-NEXT:    vmov.f64 d10, d7
+; CHECK-NEXT:    vmov.f32 s21, s15
+; CHECK-NEXT:    vmov.f32 s22, s6
+; CHECK-NEXT:    vmov.f32 s14, s4
+; CHECK-NEXT:    vmov.f32 s15, s5
+; CHECK-NEXT:    vmov.f32 s23, s7
+; CHECK-NEXT:    vldrw.u32 q1, [sp, #16] @ 16-byte Reload
+; CHECK-NEXT:    vmov r4, s12
+; CHECK-NEXT:    vmov.f32 s18, s0
+; CHECK-NEXT:    vmov.f32 s19, s1
 ; CHECK-NEXT:    vldrw.u32 q0, [sp, #48] @ 16-byte Reload
-; CHECK-NEXT:    vmov r5, s18
-; CHECK-NEXT:    vmov r7, s16
 ; CHECK-NEXT:    adds.w lr, r0, r3
-; CHECK-NEXT:    vmov r3, s14
-; CHECK-NEXT:    vmov r0, s7
-; CHECK-NEXT:    vldrw.u32 q1, [sp, #32] @ 16-byte Reload
-; CHECK-NEXT:    vmov.f32 s2, s4
-; CHECK-NEXT:    vmov.f32 s3, s5
-; CHECK-NEXT:    vldrw.u32 q1, [sp] @ 16-byte Reload
+; CHECK-NEXT:    vmov r3, s20
+; CHECK-NEXT:    vmov r0, s13
 ; CHECK-NEXT:    adc.w r12, r12, r2
-; CHECK-NEXT:    vmov r2, s15
-; CHECK-NEXT:    vmov r6, s2
+; CHECK-NEXT:    vmov r2, s21
 ; CHECK-NEXT:    adds r3, r3, r4
-; CHECK-NEXT:    vmov r4, s23
+; CHECK-NEXT:    vmov r4, s30
 ; CHECK-NEXT:    adcs r0, r2
 ; CHECK-NEXT:    adds.w lr, lr, r3
+; CHECK-NEXT:    vmov r2, s18
 ; CHECK-NEXT:    adc.w r12, r12, r0
-; CHECK-NEXT:    vmov r0, s26
-; CHECK-NEXT:    vmov r2, s22
-; CHECK-NEXT:    vmov r3, s27
-; CHECK-NEXT:    adds r0, r0, r2
-; CHECK-NEXT:    adc.w r2, r4, r3
+; CHECK-NEXT:    vmov r0, s31
 ; CHECK-NEXT:    vmov r3, s19
+; CHECK-NEXT:    adds r2, r2, r4
+; CHECK-NEXT:    adcs r3, r0
+; CHECK-NEXT:    vmov r0, s3
+; CHECK-NEXT:    vldrw.u32 q0, [sp, #32] @ 16-byte Reload
+; CHECK-NEXT:    vmov.f32 s2, s4
+; CHECK-NEXT:    vmov.f32 s3, s5
+; CHECK-NEXT:    vldrw.u32 q1, [sp, #48] @ 16-byte Reload
+; CHECK-NEXT:    vmov r6, s2
+; CHECK-NEXT:    vmov r5, s6
+; CHECK-NEXT:    vmov r7, s4
 ; CHECK-NEXT:    vmov r4, s3
 ; CHECK-NEXT:    adds r5, r5, r6
-; CHECK-NEXT:    vmov r6, s20
-; CHECK-NEXT:    adcs r3, r4
-; CHECK-NEXT:    adds r0, r0, r5
-; CHECK-NEXT:    vmov r5, s24
-; CHECK-NEXT:    adc.w r8, r3, r2
-; CHECK-NEXT:    vmov r2, s25
-; CHECK-NEXT:    vmov r4, s21
+; CHECK-NEXT:    vmov r6, s16
+; CHECK-NEXT:    adcs r4, r0
+; CHECK-NEXT:    adds.w r9, r5, r2
+; CHECK-NEXT:    vmov r5, s28
+; CHECK-NEXT:    adc.w r8, r4, r3
+; CHECK-NEXT:    vmov r2, s29
+; CHECK-NEXT:    vmov r4, s17
 ; CHECK-NEXT:    vmov r3, s0
+; CHECK-NEXT:    vmov r0, s10
 ; CHECK-NEXT:    adds r5, r5, r6
 ; CHECK-NEXT:    vmov r6, s1
 ; CHECK-NEXT:    adcs r2, r4
-; CHECK-NEXT:    vmov r4, s17
+; CHECK-NEXT:    vmov r4, s5
 ; CHECK-NEXT:    adds r3, r3, r7
-; CHECK-NEXT:    vmov r7, s28
+; CHECK-NEXT:    vmov r7, s14
 ; CHECK-NEXT:    adcs r4, r6
 ; CHECK-NEXT:    adds r3, r3, r5
-; CHECK-NEXT:    vmov r6, s8
-; CHECK-NEXT:    adcs r2, r4
-; CHECK-NEXT:    vmov r4, s9
-; CHECK-NEXT:    vmov.32 q0[0], r3
-; CHECK-NEXT:    vmov r5, s29
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    vmov r0, s12
-; CHECK-NEXT:    vmov r3, s13
-; CHECK-NEXT:    vmov.32 q0[3], r8
-; CHECK-NEXT:    vmov r2, s5
-; CHECK-NEXT:    vstrw.32 q0, [r1, #16]
+; CHECK-NEXT:    vmov r6, s22
+; CHECK-NEXT:    adc.w r10, r4, r2
+; CHECK-NEXT:    vmov r4, s23
+; CHECK-NEXT:    vmov q1[2], q1[0], r9, r3
+; CHECK-NEXT:    vmov r5, s15
+; CHECK-NEXT:    vmov q1[3], q1[1], r8, r10
+; CHECK-NEXT:    vmov r2, s26
+; CHECK-NEXT:    vstrw.32 q1, [r1, #16]
 ; CHECK-NEXT:    adds r6, r6, r7
+; CHECK-NEXT:    vmov r7, s27
 ; CHECK-NEXT:    adcs r4, r5
-; CHECK-NEXT:    vmov r5, s4
-; CHECK-NEXT:    adds r0, r0, r5
-; CHECK-NEXT:    adcs r2, r3
+; CHECK-NEXT:    vmov r5, s11
+; CHECK-NEXT:    adds r0, r0, r2
+; CHECK-NEXT:    adc.w r2, r7, r5
 ; CHECK-NEXT:    adds r0, r0, r6
-; CHECK-NEXT:    adcs r2, r4
-; CHECK-NEXT:    vmov.32 q0[0], r0
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.32 q0[2], lr
-; CHECK-NEXT:    vmov.32 q0[3], r12
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, lr
+; CHECK-NEXT:    adc.w r0, r4, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r0, r12
 ; CHECK-NEXT:    vstrw.32 q0, [r1]
-; CHECK-NEXT:    add sp, #72
+; CHECK-NEXT:    add sp, #64
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, pc}
+; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, pc}
 entry:
   %l1 = load <16 x i64>, <16 x i64>* %src, align 4
   %s1 = shufflevector <16 x i64> %l1, <16 x i64> undef, <4 x i32> <i32 0, i32 4, i32 8, i32 12>
   %s2 = shufflevector <16 x i64> %l1, <16 x i64> undef, <4 x i32> <i32 1, i32 5, i32 9, i32 13>
   %s3 = shufflevector <16 x i64> %l1, <16 x i64> undef, <4 x i32> <i32 2, i32 6, i32 10, i32 14>
   %s4 = shufflevector <16 x i64> %l1, <16 x i64> undef, <4 x i32> <i32 3, i32 7, i32 11, i32 15>
   %a1 = add <4 x i64> %s1, %s2
   %a2 = add <4 x i64> %s3, %s4

llvm/test/CodeGen/Thumb2/mve-vmulh.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve -verify-machineinstrs %s -o - | FileCheck %s --check-prefix=CHECK
 
 define arm_aapcs_vfpcc <2 x i32> @vmulhs_v2i32(<2 x i32> %s0, <2 x i32> %s1) {
 ; CHECK-LABEL: vmulhs_v2i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmullb.s32 q2, q0, q1
 ; CHECK-NEXT:    vmov r0, s9
-; CHECK-NEXT:    vmov.32 q0[0], r0
+; CHECK-NEXT:    vmov r1, s11
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
 ; CHECK-NEXT:    asrs r0, r0, #31
-; CHECK-NEXT:    vmov.32 q0[1], r0
-; CHECK-NEXT:    vmov r0, s11
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    asrs r0, r0, #31
-; CHECK-NEXT:    vmov.32 q0[3], r0
+; CHECK-NEXT:    asrs r1, r1, #31
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
 ; CHECK-NEXT:    bx lr
 entry:
   %s0s = sext <2 x i32> %s0 to <2 x i64>
   %s1s = sext <2 x i32> %s1 to <2 x i64>
   %m = mul <2 x i64> %s0s, %s1s
   %s = ashr <2 x i64> %m, <i64 32, i64 32>
   %s2 = trunc <2 x i64> %s to <2 x i32>
   ret <2 x i32> %s2
entry:
   %s = lshr <2 x i64> %m, <i64 32, i64 32>
   %s2 = trunc <2 x i64> %s to <2 x i32>
   ret <2 x i32> %s2
 }
 
 define arm_aapcs_vfpcc <4 x i32> @vmulhs_v4i32(<4 x i32> %s0, <4 x i32> %s1) {
 ; CHECK-LABEL: vmulhs_v4i32:
 ; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
+; CHECK-NEXT:    vpush {d8, d9, d10, d11}
 ; CHECK-NEXT:    vmov.f32 s8, s4
 ; CHECK-NEXT:    vmov.f32 s12, s0
-; CHECK-NEXT:    vmov.f32 s14, s1
 ; CHECK-NEXT:    vmov.f32 s10, s5
-; CHECK-NEXT:    vmov r2, s12
-; CHECK-NEXT:    vmov r1, s14
-; CHECK-NEXT:    vmov r0, s10
-; CHECK-NEXT:    vmov.f32 s12, s6
-; CHECK-NEXT:    vmov.f32 s14, s7
+; CHECK-NEXT:    vmov.f32 s14, s1
+; CHECK-NEXT:    vmov r0, s8
+; CHECK-NEXT:    vmov r1, s12
+; CHECK-NEXT:    vmov.f32 s16, s6
+; CHECK-NEXT:    vmov.f32 s18, s7
 ; CHECK-NEXT:    vmov.f32 s4, s2
 ; CHECK-NEXT:    vmov.f32 s6, s3
-; CHECK-NEXT:    vmullb.s32 q0, q1, q3
+; CHECK-NEXT:    vmullb.s32 q5, q1, q4
 ; CHECK-NEXT:    smmul r0, r1, r0
-; CHECK-NEXT:    vmov r1, s8
-; CHECK-NEXT:    smmul r1, r2, r1
-; CHECK-NEXT:    vmov.32 q2[0], r1
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov r0, s1
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    vmov.32 q2[3], r0
-; CHECK-NEXT:    vmov q0, q2
+; CHECK-NEXT:    vmov r1, s21
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
+; CHECK-NEXT:    vmov r0, s10
+; CHECK-NEXT:    vmov r1, s14
+; CHECK-NEXT:    smmul r0, r1, r0
+; CHECK-NEXT:    vmov r1, s23
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
+; CHECK-NEXT:    vpop {d8, d9, d10, d11}
 ; CHECK-NEXT:    bx lr
 entry:
   %s0s = sext <4 x i32> %s0 to <4 x i64>
   %s1s = sext <4 x i32> %s1 to <4 x i64>
   %m = mul <4 x i64> %s0s, %s1s
   %s = ashr <4 x i64> %m, <i64 32, i64 32, i64 32, i64 32>
   %s2 = trunc <4 x i64> %s to <4 x i32>
   ret <4 x i32> %s2
entry:
   %s2 = trunc <4 x i32> %s to <4 x i16>
   ret <4 x i16> %s2
 }
 
 define arm_aapcs_vfpcc <8 x i16> @vmulhs_v8i16(<8 x i16> %s0, <8 x i16> %s1) {
 ; CHECK-LABEL: vmulhs_v8i16:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.u16 r0, q1[0]
-; CHECK-NEXT:    vmov.32 q2[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q1[2]
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q1[1]
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[2]
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[3]
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q1[3]
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-NEXT:    vmov.32 q3[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[2]
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[3]
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[3]
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
+; CHECK-NEXT:    vmov.u16 r1, q1[6]
 ; CHECK-NEXT:    vmullb.s16 q2, q3, q2
 ; CHECK-NEXT:    vshr.s32 q3, q2, #16
 ; CHECK-NEXT:    vmov r0, s12
 ; CHECK-NEXT:    vmov.16 q2[0], r0
 ; CHECK-NEXT:    vmov r0, s13
 ; CHECK-NEXT:    vmov.16 q2[1], r0
 ; CHECK-NEXT:    vmov r0, s14
 ; CHECK-NEXT:    vmov.16 q2[2], r0
 ; CHECK-NEXT:    vmov r0, s15
 ; CHECK-NEXT:    vmov.16 q2[3], r0
 ; CHECK-NEXT:    vmov.u16 r0, q1[4]
-; CHECK-NEXT:    vmov.32 q3[0], r0
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q1[5]
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[6]
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[7]
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q1[7]
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[4]
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[6]
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[5]
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[6]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[7]
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[7]
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    vmullb.s16 q0, q1, q3
 ; CHECK-NEXT:    vshr.s32 q0, q0, #16
 ; CHECK-NEXT:    vmov r0, s0
 ; CHECK-NEXT:    vmov.16 q2[4], r0
 ; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    vmov.16 q2[5], r0
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    vmov.16 q2[6], r0
entry:
   %s2 = trunc <8 x i32> %s to <8 x i16>
   ret <8 x i16> %s2
 }
 
 define arm_aapcs_vfpcc <8 x i16> @vmulhu_v8i16(<8 x i16> %s0, <8 x i16> %s1) {
 ; CHECK-LABEL: vmulhu_v8i16:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.u16 r0, q1[0]
-; CHECK-NEXT:    vmov.32 q2[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q1[2]
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q1[1]
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[2]
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[3]
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q1[3]
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-NEXT:    vmov.32 q3[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[2]
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[3]
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[3]
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
+; CHECK-NEXT:    vmov.u16 r1, q1[6]
 ; CHECK-NEXT:    vmullb.u16 q2, q3, q2
 ; CHECK-NEXT:    vshr.u32 q3, q2, #16
 ; CHECK-NEXT:    vmov r0, s12
 ; CHECK-NEXT:    vmov.16 q2[0], r0
 ; CHECK-NEXT:    vmov r0, s13
 ; CHECK-NEXT:    vmov.16 q2[1], r0
 ; CHECK-NEXT:    vmov r0, s14
 ; CHECK-NEXT:    vmov.16 q2[2], r0
 ; CHECK-NEXT:    vmov r0, s15
 ; CHECK-NEXT:    vmov.16 q2[3], r0
 ; CHECK-NEXT:    vmov.u16 r0, q1[4]
-; CHECK-NEXT:    vmov.32 q3[0], r0
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q1[5]
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[6]
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[7]
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q1[7]
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[4]
-; CHECK-NEXT:    vmov.32 q1[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[6]
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q0[5]
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[6]
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q0[7]
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[7]
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    vmullb.u16 q0, q1, q3
 ; CHECK-NEXT:    vshr.u32 q0, q0, #16
 ; CHECK-NEXT:    vmov r0, s0
 ; CHECK-NEXT:    vmov.16 q2[4], r0
 ; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    vmov.16 q2[5], r0
 ; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    vmov.16 q2[6], r0

llvm/test/CodeGen/Thumb2/mve-vmull-loop.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve -verify-machineinstrs %s -o - | FileCheck %s --check-prefix=CHECK
 
 define arm_aapcs_vfpcc void @test32(i32* noalias nocapture readonly %x, i32* noalias nocapture readonly %y, i32* nocapture %z, i32 %n) {
 ; CHECK-LABEL: test32:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r5, lr}
-; CHECK-NEXT:    push {r5, lr}
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    cmp r3, #1
-; CHECK-NEXT:    it lt
-; CHECK-NEXT:    poplt {r5, pc}
+; CHECK-NEXT:    blt .LBB0_2
 ; CHECK-NEXT:  .LBB0_1: @ %vector.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vldrw.u32 q1, [r0], #16
 ; CHECK-NEXT:    vldrw.u32 q2, [r1], #16
 ; CHECK-NEXT:    subs r3, #4
-; CHECK-NEXT:    vmullt.s32 q3, q2, q1
-; CHECK-NEXT:    vmov r5, s13
-; CHECK-NEXT:    vmov r12, s12
-; CHECK-NEXT:    lsrl r12, r5, #31
-; CHECK-NEXT:    vmov.32 q0[0], r12
-; CHECK-NEXT:    vmov r12, s14
-; CHECK-NEXT:    vmov.32 q0[1], r5
-; CHECK-NEXT:    vmov r5, s15
-; CHECK-NEXT:    lsrl r12, r5, #31
+; CHECK-NEXT:    vmullt.s32 q0, q2, q1
 ; CHECK-NEXT:    vmullb.s32 q3, q2, q1
-; CHECK-NEXT:    vmov.32 q0[2], r12
+; CHECK-NEXT:    vmov r5, s1
+; CHECK-NEXT:    vmov r12, s0
+; CHECK-NEXT:    vmov r7, s3
+; CHECK-NEXT:    lsrl r12, r5, #31
+; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    lsrl r4, r7, #31
+; CHECK-NEXT:    vmov q0[2], q0[0], r4, r12
 ; CHECK-NEXT:    vmov r12, s12
-; CHECK-NEXT:    vmov.32 q0[3], r5
+; CHECK-NEXT:    vmov q0[3], q0[1], r7, r5
 ; CHECK-NEXT:    vmov r5, s13
+; CHECK-NEXT:    vmov r7, s15
 ; CHECK-NEXT:    lsrl r12, r5, #31
-; CHECK-NEXT:    vmov.32 q1[0], r12
-; CHECK-NEXT:    vmov r12, s14
-; CHECK-NEXT:    vmov.32 q1[1], r5
-; CHECK-NEXT:    vmov r5, s15
-; CHECK-NEXT:    lsrl r12, r5, #31
-; CHECK-NEXT:    vmov.32 q1[2], r12
-; CHECK-NEXT:    vmov.32 q1[3], r5
+; CHECK-NEXT:    vmov r4, s14
+; CHECK-NEXT:    lsrl r4, r7, #31
+; CHECK-NEXT:    vmov q1[2], q1[0], r4, r12
+; CHECK-NEXT:    vmov q1[3], q1[1], r7, r5
 ; CHECK-NEXT:    vmov.f32 s8, s6
 ; CHECK-NEXT:    vmov.f32 s9, s7
 ; CHECK-NEXT:    vmov.f32 s6, s0
 ; CHECK-NEXT:    vmov.f32 s7, s1
 ; CHECK-NEXT:    vmov.f32 s10, s2
 ; CHECK-NEXT:    vmov.f32 s5, s6
 ; CHECK-NEXT:    vmov.f32 s11, s3
 ; CHECK-NEXT:    vmov.f32 s6, s8
 ; CHECK-NEXT:    vmov.f32 s7, s10
 ; CHECK-NEXT:    vstrb.8 q1, [r2], #16
 ; CHECK-NEXT:    bne .LBB0_1
-; CHECK-NEXT:  @ %bb.2: @ %for.cond.cleanup
-; CHECK-NEXT:    pop {r5, pc}
+; CHECK-NEXT:  .LBB0_2: @ %for.cond.cleanup
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %0 = and i32 %n, 3
   %cmp = icmp eq i32 %0, 0
   %cmp113 = icmp sgt i32 %n, 0
   br i1 %cmp113, label %vector.body, label %for.cond.cleanup
 
 vector.body:                                      ; preds = %vector.body, %entry
   %index = phi i32 [ %index.next, %vector.body ], [ 0, %entry ]

llvm/test/CodeGen/Thumb2/mve-vqdmulh.ll

 
 define arm_aapcs_vfpcc <8 x i16> @vqdmulh_i16_c(<8 x i16> %s0, <8 x i16> %s1) {
 ; CHECK-LABEL: vqdmulh_i16_c:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .vsave {d8, d9}
 ; CHECK-NEXT:    vpush {d8, d9}
 ; CHECK-NEXT:    vmov q2, q0
 ; CHECK-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-NEXT:    vmov.32 q0[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q0[2]
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q2[1]
-; CHECK-NEXT:    vmov.32 q0[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[2]
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[3]
-; CHECK-NEXT:    vmov.32 q0[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q2[3]
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q1[0]
-; CHECK-NEXT:    vmov.32 q3[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q1[2]
+; CHECK-NEXT:    vmov q3[2], q3[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q1[1]
-; CHECK-NEXT:    vmov.32 q3[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[2]
-; CHECK-NEXT:    vmov.32 q3[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[3]
-; CHECK-NEXT:    vmov.32 q3[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q1[3]
+; CHECK-NEXT:    vmov q3[3], q3[1], r1, r0
+; CHECK-NEXT:    vmov.u16 r1, q2[6]
 ; CHECK-NEXT:    vmullb.s16 q0, q3, q0
 ; CHECK-NEXT:    vmov.i32 q3, #0x7fff
 ; CHECK-NEXT:    vshl.i32 q0, q0, #10
 ; CHECK-NEXT:    vshr.s32 q0, q0, #10
 ; CHECK-NEXT:    vshr.s32 q0, q0, #15
 ; CHECK-NEXT:    vmin.s32 q4, q0, q3
 ; CHECK-NEXT:    vmov r0, s16
 ; CHECK-NEXT:    vmov.16 q0[0], r0
 ; CHECK-NEXT:    vmov r0, s17
 ; CHECK-NEXT:    vmov.16 q0[1], r0
 ; CHECK-NEXT:    vmov r0, s18
 ; CHECK-NEXT:    vmov.16 q0[2], r0
 ; CHECK-NEXT:    vmov r0, s19
 ; CHECK-NEXT:    vmov.16 q0[3], r0
 ; CHECK-NEXT:    vmov.u16 r0, q2[4]
-; CHECK-NEXT:    vmov.32 q4[0], r0
+; CHECK-NEXT:    vmov q4[2], q4[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q2[5]
-; CHECK-NEXT:    vmov.32 q4[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[6]
-; CHECK-NEXT:    vmov.32 q4[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q2[7]
-; CHECK-NEXT:    vmov.32 q4[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q2[7]
+; CHECK-NEXT:    vmov q4[3], q4[1], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q1[4]
-; CHECK-NEXT:    vmov.32 q2[0], r0
+; CHECK-NEXT:    vmov.u16 r1, q1[6]
+; CHECK-NEXT:    vmov q2[2], q2[0], r1, r0
 ; CHECK-NEXT:    vmov.u16 r0, q1[5]
-; CHECK-NEXT:    vmov.32 q2[1], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[6]
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    vmov.u16 r0, q1[7]
-; CHECK-NEXT:    vmov.32 q2[3], r0
+; CHECK-NEXT:    vmov.u16 r1, q1[7]
+; CHECK-NEXT:    vmov q2[3], q2[1], r1, r0
 ; CHECK-NEXT:    vmullb.s16 q1, q2, q4
 ; CHECK-NEXT:    vshl.i32 q1, q1, #10
 ; CHECK-NEXT:    vshr.s32 q1, q1, #10
 ; CHECK-NEXT:    vshr.s32 q1, q1, #15
 ; CHECK-NEXT:    vmin.s32 q1, q1, q3
 ; CHECK-NEXT:    vmov r0, s4
 ; CHECK-NEXT:    vmov.16 q0[4], r0
 ; CHECK-NEXT:    vmov r0, s5

llvm/test/CodeGen/Thumb2/mve-vqmovn.ll

entry:
   %s2 = select <16 x i1> %c2, <16 x i8> %s0, <16 x i8> <i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15>
   ret <16 x i8> %s2
 }
 
 define arm_aapcs_vfpcc <2 x i64> @vqmovni64_smaxmin(<2 x i64> %s0) {
 ; CHECK-LABEL: vqmovni64_smaxmin:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    mvn r3, #-2147483648
+; CHECK-NEXT:    mvn r12, #-2147483648
 ; CHECK-NEXT:    vmov r1, s1
 ; CHECK-NEXT:    movs r0, #0
-; CHECK-NEXT:    subs r2, r2, r3
+; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    subs.w r2, r2, r12
+; CHECK-NEXT:    vmov r2, s3
 ; CHECK-NEXT:    sbcs r1, r1, #0
-; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    mov.w r1, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r1, #1
 ; CHECK-NEXT:    cmp r1, #0
 ; CHECK-NEXT:    csetm r1, ne
-; CHECK-NEXT:    vmov.32 q1[0], r1
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    subs r2, r2, r3
-; CHECK-NEXT:    mov.w r3, #-1
-; CHECK-NEXT:    sbcs r1, r1, #0
-; CHECK-NEXT:    mov.w r1, #0
+; CHECK-NEXT:    subs.w r3, r3, r12
+; CHECK-NEXT:    mov.w r12, #-1
+; CHECK-NEXT:    sbcs r2, r2, #0
+; CHECK-NEXT:    mov.w r2, #0
 ; CHECK-NEXT:    it lt
-; CHECK-NEXT:    movlt r1, #1
-; CHECK-NEXT:    cmp r1, #0
-; CHECK-NEXT:    csetm r1, ne
-; CHECK-NEXT:    vmov.32 q1[2], r1
-; CHECK-NEXT:    vmov.32 q1[3], r1
+; CHECK-NEXT:    movlt r2, #1
+; CHECK-NEXT:    cmp r2, #0
+; CHECK-NEXT:    csetm r2, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, r1
+; CHECK-NEXT:    vmov q1[3], q1[1], r2, r1
 ; CHECK-NEXT:    adr r1, .LCPI12_0
 ; CHECK-NEXT:    vldrw.u32 q2, [r1]
 ; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vbic q2, q2, q1
 ; CHECK-NEXT:    vorr q0, q0, q2
 ; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    vmov r1, s1
+; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    rsbs.w r2, r2, #-2147483648
-; CHECK-NEXT:    sbcs.w r1, r3, r1
-; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    vmov r2, s3
+; CHECK-NEXT:    sbcs.w r1, r12, r1
 ; CHECK-NEXT:    mov.w r1, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r1, #1
 ; CHECK-NEXT:    cmp r1, #0
 ; CHECK-NEXT:    csetm r1, ne
-; CHECK-NEXT:    vmov.32 q1[0], r1
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    rsbs.w r2, r2, #-2147483648
-; CHECK-NEXT:    sbcs.w r1, r3, r1
+; CHECK-NEXT:    rsbs.w r3, r3, #-2147483648
+; CHECK-NEXT:    sbcs.w r2, r12, r2
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r0, #1
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    vmov q1[2], q1[0], r0, r1
+; CHECK-NEXT:    vmov q1[3], q1[1], r0, r1
 ; CHECK-NEXT:    adr r0, .LCPI12_1
 ; CHECK-NEXT:    vldrw.u32 q2, [r0]
 ; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vbic q2, q2, q1
 ; CHECK-NEXT:    vorr q0, q0, q2
 ; CHECK-NEXT:    bx lr
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.1:
entry:
   %s2 = select <2 x i1> %c2, <2 x i64> %s1, <2 x i64> <i64 -2147483648, i64 -2147483648>
   ret <2 x i64> %s2
 }
 
 define arm_aapcs_vfpcc <2 x i64> @vqmovni64_sminmax(<2 x i64> %s0) {
 ; CHECK-LABEL: vqmovni64_sminmax:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    mov.w r3, #-1
+; CHECK-NEXT:    mov.w r12, #-1
 ; CHECK-NEXT:    vmov r1, s1
 ; CHECK-NEXT:    movs r0, #0
+; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    rsbs.w r2, r2, #-2147483648
-; CHECK-NEXT:    sbcs.w r1, r3, r1
-; CHECK-NEXT:    vmov r2, s2
+; CHECK-NEXT:    vmov r2, s3
+; CHECK-NEXT:    sbcs.w r1, r12, r1
 ; CHECK-NEXT:    mov.w r1, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r1, #1
 ; CHECK-NEXT:    cmp r1, #0
 ; CHECK-NEXT:    csetm r1, ne
-; CHECK-NEXT:    vmov.32 q1[0], r1
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    rsbs.w r2, r2, #-2147483648
-; CHECK-NEXT:    sbcs.w r1, r3, r1
-; CHECK-NEXT:    mvn r3, #-2147483648
-; CHECK-NEXT:    mov.w r1, #0
+; CHECK-NEXT:    rsbs.w r3, r3, #-2147483648
+; CHECK-NEXT:    sbcs.w r2, r12, r2
+; CHECK-NEXT:    mvn r12, #-2147483648
+; CHECK-NEXT:    mov.w r2, #0
 ; CHECK-NEXT:    it lt
-; CHECK-NEXT:    movlt r1, #1
-; CHECK-NEXT:    cmp r1, #0
-; CHECK-NEXT:    csetm r1, ne
-; CHECK-NEXT:    vmov.32 q1[2], r1
-; CHECK-NEXT:    vmov.32 q1[3], r1
+; CHECK-NEXT:    movlt r2, #1
+; CHECK-NEXT:    cmp r2, #0
+; CHECK-NEXT:    csetm r2, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, r1
+; CHECK-NEXT:    vmov q1[3], q1[1], r2, r1
 ; CHECK-NEXT:    adr r1, .LCPI13_0
 ; CHECK-NEXT:    vldrw.u32 q2, [r1]
 ; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vbic q2, q2, q1
 ; CHECK-NEXT:    vorr q0, q0, q2
 ; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    vmov r1, s1
-; CHECK-NEXT:    subs r2, r2, r3
+; CHECK-NEXT:    vmov r3, s2
+; CHECK-NEXT:    subs.w r2, r2, r12
+; CHECK-NEXT:    vmov r2, s3
 ; CHECK-NEXT:    sbcs r1, r1, #0
-; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    mov.w r1, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r1, #1
 ; CHECK-NEXT:    cmp r1, #0
 ; CHECK-NEXT:    csetm r1, ne
-; CHECK-NEXT:    vmov.32 q1[0], r1
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov r1, s3
-; CHECK-NEXT:    subs r2, r2, r3
-; CHECK-NEXT:    sbcs r1, r1, #0
+; CHECK-NEXT:    subs.w r3, r3, r12
+; CHECK-NEXT:    sbcs r2, r2, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r0, #1
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    vmov q1[2], q1[0], r0, r1
+; CHECK-NEXT:    vmov q1[3], q1[1], r0, r1
 ; CHECK-NEXT:    adr r0, .LCPI13_1
 ; CHECK-NEXT:    vldrw.u32 q2, [r0]
 ; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vbic q2, q2, q1
 ; CHECK-NEXT:    vorr q0, q0, q2
 ; CHECK-NEXT:    bx lr
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.1:

 
 define arm_aapcs_vfpcc <2 x i64> @vqmovni64_umaxmin(<2 x i64> %s0) {
 ; CHECK-LABEL: vqmovni64_umaxmin:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    movs r2, #0
 ; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    vmov.i64 q2, #0xffffffff
+; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    subs.w r1, r1, #-1
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    sbcs r0, r0, #0
-; CHECK-NEXT:    vmov r1, s2
 ; CHECK-NEXT:    mov.w r0, #0
 ; CHECK-NEXT:    it lo
 ; CHECK-NEXT:    movlo r0, #1
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q1[0], r0
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    subs.w r1, r1, #-1
-; CHECK-NEXT:    sbcs r0, r0, #0
+; CHECK-NEXT:    subs.w r3, r3, #-1
+; CHECK-NEXT:    sbcs r1, r1, #0
 ; CHECK-NEXT:    it lo
 ; CHECK-NEXT:    movlo r2, #1
 ; CHECK-NEXT:    cmp r2, #0
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    vbic q2, q2, q1
 ; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vorr q0, q0, q2
 ; CHECK-NEXT:    bx lr
 entry:
   %c1 = icmp ult <2 x i64> %s0, <i64 4294967295, i64 4294967295>
   %s1 = select <2 x i1> %c1, <2 x i64> %s0, <2 x i64> <i64 4294967295, i64 4294967295>
   ret <2 x i64> %s1
 }
 
 define arm_aapcs_vfpcc <2 x i64> @vqmovni64_uminmax(<2 x i64> %s0) {
 ; CHECK-LABEL: vqmovni64_uminmax:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    movs r2, #0
 ; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    vmov.i64 q2, #0xffffffff
+; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    subs.w r1, r1, #-1
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    sbcs r0, r0, #0
-; CHECK-NEXT:    vmov r1, s2
 ; CHECK-NEXT:    mov.w r0, #0
 ; CHECK-NEXT:    it lo
 ; CHECK-NEXT:    movlo r0, #1
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q1[0], r0
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    subs.w r1, r1, #-1
-; CHECK-NEXT:    sbcs r0, r0, #0
+; CHECK-NEXT:    subs.w r3, r3, #-1
+; CHECK-NEXT:    sbcs r1, r1, #0
 ; CHECK-NEXT:    it lo
 ; CHECK-NEXT:    movlo r2, #1
 ; CHECK-NEXT:    cmp r2, #0
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    vbic q2, q2, q1
 ; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vorr q0, q0, q2
 ; CHECK-NEXT:    bx lr
 entry:
   %c2 = icmp ult <2 x i64> %s0, <i64 4294967295, i64 4294967295>
   %s2 = select <2 x i1> %c2, <2 x i64> %s0, <2 x i64> <i64 4294967295, i64 4294967295>
   ret <2 x i64> %s2
 }

llvm/test/CodeGen/Thumb2/mve-vqshrn.ll

entry:
   %c2 = icmp ult <16 x i8> %s0, <i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15>
   %s2 = select <16 x i1> %c2, <16 x i8> %s0, <16 x i8> <i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15, i8 15>
   ret <16 x i8> %s2
 }
 
 define arm_aapcs_vfpcc <2 x i64> @vqshrni64_smaxmin(<2 x i64> %so) {
 ; CHECK-LABEL: vqshrni64_smaxmin:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r5, lr}
-; CHECK-NEXT:    push {r5, lr}
-; CHECK-NEXT:    vmov r5, s1
-; CHECK-NEXT:    mvn lr, #-2147483648
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
+; CHECK-NEXT:    vmov r1, s1
+; CHECK-NEXT:    mvn r12, #-2147483648
 ; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    mov.w r12, #0
-; CHECK-NEXT:    asrl r2, r5, #3
+; CHECK-NEXT:    mov.w lr, #0
+; CHECK-NEXT:    asrl r2, r1, #3
+; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    subs.w r3, r2, r12
+; CHECK-NEXT:    sbcs r3, r1, #0
+; CHECK-NEXT:    mov.w r3, #0
+; CHECK-NEXT:    it lt
+; CHECK-NEXT:    movlt r3, #1
+; CHECK-NEXT:    cmp r3, #0
 ; CHECK-NEXT:    vmov r3, s3
-; CHECK-NEXT:    subs.w r0, r2, lr
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    sbcs r0, r5, #0
-; CHECK-NEXT:    vmov.32 q2[1], r5
+; CHECK-NEXT:    csetm r5, ne
+; CHECK-NEXT:    asrl r4, r3, #3
+; CHECK-NEXT:    subs.w r0, r4, r12
+; CHECK-NEXT:    vmov q2[2], q2[0], r4, r2
+; CHECK-NEXT:    sbcs r0, r3, #0
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r1
 ; CHECK-NEXT:    mov.w r0, #0
 ; CHECK-NEXT:    mov.w r2, #-1
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r0, #1
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q1[0], r0
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    asrl r0, r3, #3
-; CHECK-NEXT:    subs.w r1, r0, lr
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    sbcs r1, r3, #0
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    mov.w r1, #0
-; CHECK-NEXT:    it lt
-; CHECK-NEXT:    movlt r1, #1
-; CHECK-NEXT:    cmp r1, #0
-; CHECK-NEXT:    csetm r1, ne
-; CHECK-NEXT:    vmov.32 q1[2], r1
-; CHECK-NEXT:    vmov.32 q1[3], r1
-; CHECK-NEXT:    adr r1, .LCPI12_0
-; CHECK-NEXT:    vldrw.u32 q0, [r1]
-; CHECK-NEXT:    vbic q0, q0, q1
-; CHECK-NEXT:    vand q1, q2, q1
-; CHECK-NEXT:    vorr q0, q1, q0
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r5
+; CHECK-NEXT:    vmov q0[3], q0[1], r0, r5
+; CHECK-NEXT:    adr r0, .LCPI12_0
+; CHECK-NEXT:    vldrw.u32 q1, [r0]
+; CHECK-NEXT:    vbic q1, q1, q0
+; CHECK-NEXT:    vand q0, q2, q0
+; CHECK-NEXT:    vorr q0, q0, q1
 ; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    vmov r0, s1
+; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    rsbs.w r1, r1, #-2147483648
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    sbcs.w r0, r2, r0
-; CHECK-NEXT:    vmov r1, s2
 ; CHECK-NEXT:    mov.w r0, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r0, #1
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q1[0], r0
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    rsbs.w r1, r1, #-2147483648
-; CHECK-NEXT:    sbcs.w r0, r2, r0
+; CHECK-NEXT:    rsbs.w r3, r3, #-2147483648
+; CHECK-NEXT:    sbcs.w r1, r2, r1
 ; CHECK-NEXT:    it lt
-; CHECK-NEXT:    movlt.w r12, #1
-; CHECK-NEXT:    cmp.w r12, #0
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    movlt.w lr, #1
+; CHECK-NEXT:    cmp.w lr, #0
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    adr r0, .LCPI12_1
 ; CHECK-NEXT:    vldrw.u32 q2, [r0]
 ; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vbic q2, q2, q1
 ; CHECK-NEXT:    vorr q0, q0, q2
-; CHECK-NEXT:    pop {r5, pc}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.1:
 ; CHECK-NEXT:  .LCPI12_0:
 ; CHECK-NEXT:    .long 2147483647 @ 0x7fffffff
 ; CHECK-NEXT:    .long 0 @ 0x0
 ; CHECK-NEXT:    .long 2147483647 @ 0x7fffffff
 ; CHECK-NEXT:    .long 0 @ 0x0
 ; CHECK-NEXT:  .LCPI12_1:
 ; CHECK-NEXT:    .long 2147483648 @ 0x80000000
 ; CHECK-NEXT:    .long 4294967295 @ 0xffffffff
 ; CHECK-NEXT:    .long 2147483648 @ 0x80000000
 ; CHECK-NEXT:    .long 4294967295 @ 0xffffffff
 entry:
   %s0 = ashr <2 x i64> %so, <i64 3, i64 3>
   %c1 = icmp slt <2 x i64> %s0, <i64 2147483647, i64 2147483647>
   %s1 = select <2 x i1> %c1, <2 x i64> %s0, <2 x i64> <i64 2147483647, i64 2147483647>
   %c2 = icmp sgt <2 x i64> %s1, <i64 -2147483648, i64 -2147483648>
   %s2 = select <2 x i1> %c2, <2 x i64> %s1, <2 x i64> <i64 -2147483648, i64 -2147483648>
   ret <2 x i64> %s2
 }
 
 define arm_aapcs_vfpcc <2 x i64> @vqshrni64_sminmax(<2 x i64> %so) {
 ; CHECK-LABEL: vqshrni64_sminmax:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, lr}
-; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    vmov r1, s1
-; CHECK-NEXT:    mov.w lr, #-1
+; CHECK-NEXT:    mov.w r12, #-1
 ; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    mov.w r12, #0
+; CHECK-NEXT:    mov.w lr, #0
 ; CHECK-NEXT:    asrl r2, r1, #3
-; CHECK-NEXT:    vmov r3, s3
-; CHECK-NEXT:    rsbs.w r0, r2, #-2147483648
-; CHECK-NEXT:    vmov.32 q2[0], r2
-; CHECK-NEXT:    sbcs.w r0, lr, r1
-; CHECK-NEXT:    vmov.32 q2[1], r1
-; CHECK-NEXT:    mov.w r0, #0
-; CHECK-NEXT:    mvn r2, #-2147483648
+; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    rsbs.w r3, r2, #-2147483648
+; CHECK-NEXT:    sbcs.w r3, r12, r1
+; CHECK-NEXT:    mov.w r3, #0
 ; CHECK-NEXT:    it lt
-; CHECK-NEXT:    movlt r0, #1
-; CHECK-NEXT:    cmp r0, #0
+; CHECK-NEXT:    movlt r3, #1
+; CHECK-NEXT:    cmp r3, #0
+; CHECK-NEXT:    vmov r3, s3
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q1[0], r0
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    asrl r0, r3, #3
-; CHECK-NEXT:    rsbs.w r4, r0, #-2147483648
-; CHECK-NEXT:    vmov.32 q2[2], r0
-; CHECK-NEXT:    sbcs.w r4, lr, r3
-; CHECK-NEXT:    vmov.32 q2[3], r3
-; CHECK-NEXT:    mov.w r4, #0
+; CHECK-NEXT:    asrl r4, r3, #3
+; CHECK-NEXT:    rsbs.w r5, r4, #-2147483648
+; CHECK-NEXT:    vmov q2[2], q2[0], r4, r2
+; CHECK-NEXT:    sbcs.w r5, r12, r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r1
+; CHECK-NEXT:    mov.w r5, #0
+; CHECK-NEXT:    mvn r2, #-2147483648
 ; CHECK-NEXT:    it lt
-; CHECK-NEXT:    movlt r4, #1
-; CHECK-NEXT:    cmp r4, #0
-; CHECK-NEXT:    csetm r4, ne
-; CHECK-NEXT:    vmov.32 q1[2], r4
-; CHECK-NEXT:    vmov.32 q1[3], r4
-; CHECK-NEXT:    adr r4, .LCPI13_0
-; CHECK-NEXT:    vldrw.u32 q0, [r4]
-; CHECK-NEXT:    vbic q0, q0, q1
-; CHECK-NEXT:    vand q1, q2, q1
-; CHECK-NEXT:    vorr q0, q1, q0
+; CHECK-NEXT:    movlt r5, #1
+; CHECK-NEXT:    cmp r5, #0
+; CHECK-NEXT:    csetm r5, ne
+; CHECK-NEXT:    vmov q0[2], q0[0], r5, r0
+; CHECK-NEXT:    vmov q0[3], q0[1], r5, r0
+; CHECK-NEXT:    adr r0, .LCPI13_0
+; CHECK-NEXT:    vldrw.u32 q1, [r0]
+; CHECK-NEXT:    vbic q1, q1, q0
+; CHECK-NEXT:    vand q0, q2, q0
+; CHECK-NEXT:    vorr q0, q0, q1
 ; CHECK-NEXT:    vmov r1, s0
 ; CHECK-NEXT:    vmov r0, s1
+; CHECK-NEXT:    vmov r3, s2
 ; CHECK-NEXT:    subs r1, r1, r2
+; CHECK-NEXT:    vmov r1, s3
 ; CHECK-NEXT:    sbcs r0, r0, #0
-; CHECK-NEXT:    vmov r1, s2
 ; CHECK-NEXT:    mov.w r0, #0
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    movlt r0, #1
 ; CHECK-NEXT:    cmp r0, #0
 ; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q1[0], r0
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov r0, s3
-; CHECK-NEXT:    subs r1, r1, r2
-; CHECK-NEXT:    sbcs r0, r0, #0
+; CHECK-NEXT:    subs r2, r3, r2
+; CHECK-NEXT:    sbcs r1, r1, #0
 ; CHECK-NEXT:    it lt
-; CHECK-NEXT:    movlt.w r12, #1
-; CHECK-NEXT:    cmp.w r12, #0
-; CHECK-NEXT:    csetm r0, ne
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.32 q1[3], r0
+; CHECK-NEXT:    movlt.w lr, #1
+; CHECK-NEXT:    cmp.w lr, #0
+; CHECK-NEXT:    csetm r1, ne
+; CHECK-NEXT:    vmov q1[2], q1[0], r1, r0
+; CHECK-NEXT:    vmov q1[3], q1[1], r1, r0
 ; CHECK-NEXT:    adr r0, .LCPI13_1
 ; CHECK-NEXT:    vldrw.u32 q2, [r0]
 ; CHECK-NEXT:    vand q0, q0, q1
 ; CHECK-NEXT:    vbic q2, q2, q1
 ; CHECK-NEXT:    vorr q0, q0, q2
-; CHECK-NEXT:    pop {r4, pc}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.1:
 ; CHECK-NEXT:  .LCPI13_0:
 ; CHECK-NEXT:    .long 2147483648 @ 0x80000000
 ; CHECK-NEXT:    .long 4294967295 @ 0xffffffff
 ; CHECK-NEXT:    .long 2147483648 @ 0x80000000
 ; CHECK-NEXT:    .long 4294967295 @ 0xffffffff
 ; CHECK-NEXT:  .LCPI13_1:
 ; CHECK-NEXT:    .long 2147483647 @ 0x7fffffff
 ; CHECK-NEXT:    .long 0 @ 0x0
 ; CHECK-NEXT:    .long 2147483647 @ 0x7fffffff
 ; CHECK-NEXT:    .long 0 @ 0x0
 entry:
   %s0 = ashr <2 x i64> %so, <i64 3, i64 3>
   %c1 = icmp sgt <2 x i64> %s0, <i64 -2147483648, i64 -2147483648>
   %s1 = select <2 x i1> %c1, <2 x i64> %s0, <2 x i64> <i64 -2147483648, i64 -2147483648>
   %c2 = icmp slt <2 x i64> %s1, <i64 2147483647, i64 2147483647>
   %s2 = select <2 x i1> %c2, <2 x i64> %s1, <2 x i64> <i64 2147483647, i64 2147483647>
   ret <2 x i64> %s2
 }
 
 define arm_aapcs_vfpcc <2 x i64> @vqshrni64_umaxmin(<2 x i64> %so) {
 ; CHECK-LABEL: vqshrni64_umaxmin:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r5, r6, r7, lr}
-; CHECK-NEXT:    push {r5, r6, r7, lr}
-; CHECK-NEXT:    vmov r7, s1
-; CHECK-NEXT:    movs r1, #0
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
+; CHECK-NEXT:    vmov r5, s1
+; CHECK-NEXT:    movs r2, #0
 ; CHECK-NEXT:    vmov r0, s0
-; CHECK-NEXT:    lsrl r0, r7, #3
+; CHECK-NEXT:    vmov.i64 q1, #0xffffffff
+; CHECK-NEXT:    lsrl r0, r5, #3
+; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    subs.w r3, r0, #-1
+; CHECK-NEXT:    sbcs r3, r5, #0
+; CHECK-NEXT:    mov.w r3, #0
+; CHECK-NEXT:    it lo
+; CHECK-NEXT:    movlo r3, #1
+; CHECK-NEXT:    cmp r3, #0
 ; CHECK-NEXT:    vmov r3, s3
-; CHECK-NEXT:    subs.w r2, r0, #-1
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    sbcs r2, r7, #0
-; CHECK-NEXT:    vmov.32 q2[1], r7
-; CHECK-NEXT:    mov.w r2, #0
+; CHECK-NEXT:    csetm r12, ne
+; CHECK-NEXT:    lsrl r4, r3, #3
+; CHECK-NEXT:    subs.w r1, r4, #-1
+; CHECK-NEXT:    vmov q2[2], q2[0], r4, r0
+; CHECK-NEXT:    sbcs r1, r3, #0
 ; CHECK-NEXT:    it lo
 ; CHECK-NEXT:    movlo r2, #1
 ; CHECK-NEXT:    cmp r2, #0
-; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q1[0], r2
-; CHECK-NEXT:    vmov.32 q1[1], r2
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    lsrl r2, r3, #3
-; CHECK-NEXT:    vmov.i64 q0, #0xffffffff
-; CHECK-NEXT:    subs.w r5, r2, #-1
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    sbcs r5, r3, #0
-; CHECK-NEXT:    it lo
-; CHECK-NEXT:    movlo r1, #1
-; CHECK-NEXT:    cmp r1, #0
-; CHECK-NEXT:    vmov.32 q2[3], r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r5
 ; CHECK-NEXT:    csetm r1, ne
-; CHECK-NEXT:    vmov.32 q1[2], r1
-; CHECK-NEXT:    vmov.32 q1[3], r1
-; CHECK-NEXT:    vbic q0, q0, q1
-; CHECK-NEXT:    vand q1, q2, q1
-; CHECK-NEXT:    vorr q0, q1, q0
-; CHECK-NEXT:    pop {r5, r6, r7, pc}
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r12
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r12
+; CHECK-NEXT:    vbic q1, q1, q0
+; CHECK-NEXT:    vand q0, q2, q0
+; CHECK-NEXT:    vorr q0, q0, q1
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %s0 = lshr <2 x i64> %so, <i64 3, i64 3>
   %c1 = icmp ult <2 x i64> %s0, <i64 4294967295, i64 4294967295>
   %s1 = select <2 x i1> %c1, <2 x i64> %s0, <2 x i64> <i64 4294967295, i64 4294967295>
   ret <2 x i64> %s1
 }
 
 define arm_aapcs_vfpcc <2 x i64> @vqshrni64_uminmax(<2 x i64> %so) {
 ; CHECK-LABEL: vqshrni64_uminmax:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r5, r6, r7, lr}
-; CHECK-NEXT:    push {r5, r6, r7, lr}
-; CHECK-NEXT:    vmov r7, s1
-; CHECK-NEXT:    movs r1, #0
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
+; CHECK-NEXT:    vmov r5, s1
+; CHECK-NEXT:    movs r2, #0
 ; CHECK-NEXT:    vmov r0, s0
-; CHECK-NEXT:    lsrl r0, r7, #3
+; CHECK-NEXT:    vmov.i64 q1, #0xffffffff
+; CHECK-NEXT:    lsrl r0, r5, #3
+; CHECK-NEXT:    vmov r4, s2
+; CHECK-NEXT:    subs.w r3, r0, #-1
+; CHECK-NEXT:    sbcs r3, r5, #0
+; CHECK-NEXT:    mov.w r3, #0
+; CHECK-NEXT:    it lo
+; CHECK-NEXT:    movlo r3, #1
+; CHECK-NEXT:    cmp r3, #0
 ; CHECK-NEXT:    vmov r3, s3
-; CHECK-NEXT:    subs.w r2, r0, #-1
-; CHECK-NEXT:    vmov.32 q2[0], r0
-; CHECK-NEXT:    sbcs r2, r7, #0
-; CHECK-NEXT:    vmov.32 q2[1], r7
-; CHECK-NEXT:    mov.w r2, #0
+; CHECK-NEXT:    csetm r12, ne
+; CHECK-NEXT:    lsrl r4, r3, #3
+; CHECK-NEXT:    subs.w r1, r4, #-1
+; CHECK-NEXT:    vmov q2[2], q2[0], r4, r0
+; CHECK-NEXT:    sbcs r1, r3, #0
 ; CHECK-NEXT:    it lo
 ; CHECK-NEXT:    movlo r2, #1
 ; CHECK-NEXT:    cmp r2, #0
-; CHECK-NEXT:    csetm r2, ne
-; CHECK-NEXT:    vmov.32 q1[0], r2
-; CHECK-NEXT:    vmov.32 q1[1], r2
-; CHECK-NEXT:    vmov r2, s2
-; CHECK-NEXT:    lsrl r2, r3, #3
-; CHECK-NEXT:    vmov.i64 q0, #0xffffffff
-; CHECK-NEXT:    subs.w r5, r2, #-1
-; CHECK-NEXT:    vmov.32 q2[2], r2
-; CHECK-NEXT:    sbcs r5, r3, #0
-; CHECK-NEXT:    it lo
-; CHECK-NEXT:    movlo r1, #1
-; CHECK-NEXT:    cmp r1, #0
-; CHECK-NEXT:    vmov.32 q2[3], r3
+; CHECK-NEXT:    vmov q2[3], q2[1], r3, r5
 ; CHECK-NEXT:    csetm r1, ne
-; CHECK-NEXT:    vmov.32 q1[2], r1
-; CHECK-NEXT:    vmov.32 q1[3], r1
-; CHECK-NEXT:    vbic q0, q0, q1
-; CHECK-NEXT:    vand q1, q2, q1
-; CHECK-NEXT:    vorr q0, q1, q0
-; CHECK-NEXT:    pop {r5, r6, r7, pc}
+; CHECK-NEXT:    vmov q0[2], q0[0], r1, r12
+; CHECK-NEXT:    vmov q0[3], q0[1], r1, r12
+; CHECK-NEXT:    vbic q1, q1, q0
+; CHECK-NEXT:    vand q0, q2, q0
+; CHECK-NEXT:    vorr q0, q0, q1
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %s0 = lshr <2 x i64> %so, <i64 3, i64 3>
   %c2 = icmp ult <2 x i64> %s0, <i64 4294967295, i64 4294967295>
   %s2 = select <2 x i1> %c2, <2 x i64> %s0, <2 x i64> <i64 4294967295, i64 4294967295>
   ret <2 x i64> %s2
 }

llvm/test/CodeGen/Thumb2/mve-vst2.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp,+fp64 -verify-machineinstrs %s -o - | FileCheck %s
 
 ; i32
 
 define void @vst2_v2i32(<2 x i32> *%src, <4 x i32> *%dst) {
 ; CHECK-LABEL: vst2_v2i32:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    ldrd r2, r12, [r0]
-; CHECK-NEXT:    ldrd r3, r0, [r0, #8]
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov.32 q1[0], r3
-; CHECK-NEXT:    vmov.32 q0[2], r12
-; CHECK-NEXT:    vmov.f64 d4, d1
-; CHECK-NEXT:    vmov.32 q1[2], r0
-; CHECK-NEXT:    vmov.f32 s9, s3
-; CHECK-NEXT:    vmov.f32 s2, s4
-; CHECK-NEXT:    vmov.f32 s3, s5
-; CHECK-NEXT:    vmov.f32 s10, s6
-; CHECK-NEXT:    vmov.f32 s1, s2
-; CHECK-NEXT:    vmov.f32 s11, s7
+; CHECK-NEXT:    ldrd r12, r3, [r0]
+; CHECK-NEXT:    ldrd r2, r0, [r0, #8]
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r12
+; CHECK-NEXT:    vmov.f64 d2, d1
+; CHECK-NEXT:    vmov q2[2], q2[0], r0, r2
+; CHECK-NEXT:    vmov.f32 s5, s3
 ; CHECK-NEXT:    vmov.f32 s2, s8
-; CHECK-NEXT:    vmov.f32 s3, s10
+; CHECK-NEXT:    vmov.f32 s3, s9
+; CHECK-NEXT:    vmov.f32 s6, s10
+; CHECK-NEXT:    vmov.f32 s1, s2
+; CHECK-NEXT:    vmov.f32 s7, s11
+; CHECK-NEXT:    vmov.f32 s2, s4
+; CHECK-NEXT:    vmov.f32 s3, s6
 ; CHECK-NEXT:    vstrw.32 q0, [r1]
 ; CHECK-NEXT:    bx lr
 entry:
   %s1 = getelementptr <2 x i32>, <2 x i32>* %src, i32 0
   %l1 = load <2 x i32>, <2 x i32>* %s1, align 4
   %s2 = getelementptr <2 x i32>, <2 x i32>* %src, i32 1
   %l2 = load <2 x i32>, <2 x i32>* %s2, align 4
   %s = shufflevector <2 x i32> %l1, <2 x i32> %l2, <4 x i32> <i32 0, i32 2, i32 1, i32 3>
entry:
   ret void
 }
 
 ; i16
 
 define void @vst2_v2i16(<2 x i16> *%src, <4 x i16> *%dst) {
 ; CHECK-LABEL: vst2_v2i16:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    ldrh r3, [r0]
-; CHECK-NEXT:    ldrh r2, [r0, #4]
-; CHECK-NEXT:    vmov.32 q0[0], r3
-; CHECK-NEXT:    ldrh.w r12, [r0, #6]
-; CHECK-NEXT:    ldrh r0, [r0, #2]
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    vmov.32 q0[3], r12
+; CHECK-NEXT:    ldrh r2, [r0]
+; CHECK-NEXT:    ldrh r3, [r0, #2]
+; CHECK-NEXT:    ldrh.w r12, [r0, #4]
+; CHECK-NEXT:    ldrh r0, [r0, #6]
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r0, r12
 ; CHECK-NEXT:    vstrh.32 q0, [r1]
 ; CHECK-NEXT:    bx lr
 entry:
   %s1 = getelementptr <2 x i16>, <2 x i16>* %src, i32 0
   %l1 = load <2 x i16>, <2 x i16>* %s1, align 4
   %s2 = getelementptr <2 x i16>, <2 x i16>* %src, i32 1
   %l2 = load <2 x i16>, <2 x i16>* %s2, align 4
   %s = shufflevector <2 x i16> %l1, <2 x i16> %l2, <4 x i32> <i32 0, i32 2, i32 1, i32 3>

 }
 
 ; i8
 
 define void @vst2_v2i8(<2 x i8> *%src, <4 x i8> *%dst) {
 ; CHECK-LABEL: vst2_v2i8:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    ldrb r2, [r0]
-; CHECK-NEXT:    ldrb r3, [r0, #2]
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    ldrb.w r12, [r0, #1]
-; CHECK-NEXT:    vmov.32 q0[1], r3
+; CHECK-NEXT:    ldrb r3, [r0, #1]
+; CHECK-NEXT:    ldrb.w r12, [r0, #2]
 ; CHECK-NEXT:    ldrb r0, [r0, #3]
-; CHECK-NEXT:    vmov.32 q0[2], r12
-; CHECK-NEXT:    vmov.32 q0[3], r0
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
+; CHECK-NEXT:    vmov q0[3], q0[1], r0, r12
 ; CHECK-NEXT:    vstrb.32 q0, [r1]
 ; CHECK-NEXT:    bx lr
 entry:
   %s1 = getelementptr <2 x i8>, <2 x i8>* %src, i32 0
   %l1 = load <2 x i8>, <2 x i8>* %s1, align 4
   %s2 = getelementptr <2 x i8>, <2 x i8>* %src, i32 1
   %l2 = load <2 x i8>, <2 x i8>* %s2, align 4
   %s = shufflevector <2 x i8> %l1, <2 x i8> %l2, <4 x i32> <i32 0, i32 2, i32 1, i32 3>

llvm/test/CodeGen/Thumb2/mve-vst3.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp,+fp64 -verify-machineinstrs %s -o - | FileCheck %s
 
 ; i32
 
 define void @vst3_v2i32(<2 x i32> *%src, <6 x i32> *%dst) {
 ; CHECK-LABEL: vst3_v2i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, lr}
 ; CHECK-NEXT:    push {r4, lr}
-; CHECK-NEXT:    ldm.w r0, {r2, r3, r12, lr}
+; CHECK-NEXT:    ldrd lr, r12, [r0]
+; CHECK-NEXT:    ldrd r3, r2, [r0, #8]
 ; CHECK-NEXT:    ldrd r4, r0, [r0, #16]
-; CHECK-NEXT:    vmov.32 q1[0], r2
-; CHECK-NEXT:    vmov.32 q1[1], r3
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, lr
 ; CHECK-NEXT:    vmov.32 q0[0], r4
-; CHECK-NEXT:    vmov.32 q1[2], r12
+; CHECK-NEXT:    vmov q1[3], q1[1], r2, r12
 ; CHECK-NEXT:    vmov.32 q0[1], r0
-; CHECK-NEXT:    vmov.32 q1[3], lr
 ; CHECK-NEXT:    vmov.f32 s8, s7
 ; CHECK-NEXT:    vmov.f32 s10, s1
 ; CHECK-NEXT:    vmov r2, s8
 ; CHECK-NEXT:    vmov r0, s10
 ; CHECK-NEXT:    vmov.f64 d4, d2
 ; CHECK-NEXT:    vmov.f32 s9, s6
 ; CHECK-NEXT:    vmov.f32 s10, s0
 ; CHECK-NEXT:    vmov.f32 s11, s5

 ; i16
 
 define void @vst3_v2i16(<2 x i16> *%src, <6 x i16> *%dst) {
 ; CHECK-LABEL: vst3_v2i16:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, lr}
 ; CHECK-NEXT:    push {r4, lr}
 ; CHECK-NEXT:    ldrh r2, [r0, #6]
-; CHECK-NEXT:    ldrh r3, [r0, #4]
+; CHECK-NEXT:    ldrh.w lr, [r0, #4]
 ; CHECK-NEXT:    ldrh.w r12, [r0, #8]
 ; CHECK-NEXT:    vmov.16 q0[4], r2
-; CHECK-NEXT:    ldrh.w lr, [r0, #2]
-; CHECK-NEXT:    vmov.32 q1[0], r3
+; CHECK-NEXT:    ldrh r3, [r0, #2]
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, lr
 ; CHECK-NEXT:    ldrh r4, [r0]
-; CHECK-NEXT:    vmov.32 q1[2], r2
 ; CHECK-NEXT:    ldrh r0, [r0, #10]
 ; CHECK-NEXT:    vmov.16 q0[5], r0
 ; CHECK-NEXT:    vmov r0, s2
-; CHECK-NEXT:    vmov.32 q0[0], r4
-; CHECK-NEXT:    vmov.32 q0[2], lr
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r4
 ; CHECK-NEXT:    vmov.f32 s1, s4
 ; CHECK-NEXT:    vdup.32 q1, r12
 ; CHECK-NEXT:    vmov.f32 s3, s2
 ; CHECK-NEXT:    vmov.f32 s2, s6
 ; CHECK-NEXT:    vstrh.32 q0, [r1]
 ; CHECK-NEXT:    str r0, [r1, #8]
 ; CHECK-NEXT:    pop {r4, pc}
 entry:

 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r6, lr}
 ; CHECK-NEXT:    push {r4, r5, r6, lr}
 ; CHECK-NEXT:    .pad #16
 ; CHECK-NEXT:    sub sp, #16
 ; CHECK-NEXT:    ldrb r2, [r0]
 ; CHECK-NEXT:    movs r6, #0
 ; CHECK-NEXT:    ldrb r3, [r0, #1]
-; CHECK-NEXT:    vmov.32 q0[0], r2
 ; CHECK-NEXT:    ldrb.w r12, [r0, #2]
-; CHECK-NEXT:    vmov.32 q0[2], r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
 ; CHECK-NEXT:    ldrb.w lr, [r0, #3]
 ; CHECK-NEXT:    vmov r4, s0
 ; CHECK-NEXT:    ldrb r5, [r0, #5]
 ; CHECK-NEXT:    vmov.16 q0[0], r4
 ; CHECK-NEXT:    ldrb r0, [r0, #4]
 ; CHECK-NEXT:    vmov.16 q0[1], r12
 ; CHECK-NEXT:    mov r2, sp
 ; CHECK-NEXT:    vmov.16 q0[2], r0
entry:
   %s = shufflevector <4 x half> %t1, <4 x half> %t2, <6 x i32> <i32 0, i32 2, i32 4, i32 1, i32 3, i32 5>
   store <6 x half> %s, <6 x half> *%dst
   ret void
 }
 
 define void @vst3_v4f16(<4 x half> *%src, <12 x half> *%dst) {
 ; CHECK-LABEL: vst3_v4f16:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r7, lr}
-; CHECK-NEXT:    push {r7, lr}
-; CHECK-NEXT:    ldm.w r0, {r2, r3, r12, lr}
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov.32 q0[1], r3
-; CHECK-NEXT:    vmov.32 q0[2], r12
-; CHECK-NEXT:    vmov.32 q0[3], lr
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    ldrd lr, r12, [r0]
+; CHECK-NEXT:    ldrd r3, r2, [r0, #8]
+; CHECK-NEXT:    ldrd r4, r0, [r0, #16]
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, lr
+; CHECK-NEXT:    vmov q0[3], q0[1], r2, r12
+; CHECK-NEXT:    vmov.32 q1[0], r4
 ; CHECK-NEXT:    vmov r3, s0
-; CHECK-NEXT:    vmovx.f16 s12, s0
+; CHECK-NEXT:    vmov.32 q1[1], r0
 ; CHECK-NEXT:    vmov r2, s2
 ; CHECK-NEXT:    vmov.16 q2[0], r3
 ; CHECK-NEXT:    vmov.16 q2[1], r2
-; CHECK-NEXT:    ldrd r2, r0, [r0, #16]
-; CHECK-NEXT:    vmovx.f16 s0, s3
-; CHECK-NEXT:    vmov.32 q1[0], r2
-; CHECK-NEXT:    vmov.32 q1[1], r0
 ; CHECK-NEXT:    vmov r0, s4
+; CHECK-NEXT:    vmovx.f16 s12, s0
 ; CHECK-NEXT:    vmov.16 q2[2], r0
 ; CHECK-NEXT:    vmov r0, s12
 ; CHECK-NEXT:    vmovx.f16 s12, s2
 ; CHECK-NEXT:    vmov.16 q2[3], r0
 ; CHECK-NEXT:    vmov r0, s12
 ; CHECK-NEXT:    vmovx.f16 s12, s4
 ; CHECK-NEXT:    vmov.16 q2[4], r0
 ; CHECK-NEXT:    vmov r0, s12
+; CHECK-NEXT:    vmovx.f16 s0, s3
 ; CHECK-NEXT:    vmov.16 q2[5], r0
 ; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    vmov.16 q2[6], r0
 ; CHECK-NEXT:    vmov r0, s3
 ; CHECK-NEXT:    vmov.16 q2[7], r0
 ; CHECK-NEXT:    vmov r2, s5
 ; CHECK-NEXT:    vstrw.32 q2, [r1]
 ; CHECK-NEXT:    vmovx.f16 s8, s1
 ; CHECK-NEXT:    vmov r0, s8
 ; CHECK-NEXT:    vmov.16 q2[0], r2
 ; CHECK-NEXT:    vmov.16 q2[1], r0
 ; CHECK-NEXT:    vmov r0, s0
 ; CHECK-NEXT:    vmovx.f16 s0, s5
 ; CHECK-NEXT:    vmov.16 q2[2], r0
 ; CHECK-NEXT:    vmov r0, s0
 ; CHECK-NEXT:    vmov.16 q2[3], r0
 ; CHECK-NEXT:    vmov r0, s9
 ; CHECK-NEXT:    vmov r2, s8
 ; CHECK-NEXT:    strd r2, r0, [r1, #16]
-; CHECK-NEXT:    pop {r7, pc}
+; CHECK-NEXT:    pop {r4, pc}
 entry:
   %s1 = getelementptr <4 x half>, <4 x half>* %src, i32 0
   %l1 = load <4 x half>, <4 x half>* %s1, align 4
   %s2 = getelementptr <4 x half>, <4 x half>* %src, i32 1
   %l2 = load <4 x half>, <4 x half>* %s2, align 4
   %s3 = getelementptr <4 x half>, <4 x half>* %src, i32 2
   %l3 = load <4 x half>, <4 x half>* %s3, align 4
   %t1 = shufflevector <4 x half> %l1, <4 x half> %l2, <8 x i32> <i32 0, i32 1, i32 2, i32 3, i32 4, i32 5, i32 6, i32 7>

llvm/test/CodeGen/Thumb2/mve-vst4.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp,+fp64 -mve-max-interleave-factor=4 -verify-machineinstrs %s -o - | FileCheck %s
 
 ; i32
 
 define void @vst4_v2i32(<2 x i32> *%src, <8 x i32> *%dst) {
 ; CHECK-LABEL: vst4_v2i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, lr}
 ; CHECK-NEXT:    push {r4, lr}
-; CHECK-NEXT:    ldm.w r0, {r2, r3, r12, lr}
+; CHECK-NEXT:    ldrd lr, r12, [r0]
+; CHECK-NEXT:    ldrd r3, r2, [r0, #8]
 ; CHECK-NEXT:    ldrd r4, r0, [r0, #16]
-; CHECK-NEXT:    vmov.32 q1[0], r4
-; CHECK-NEXT:    vmov.32 q1[2], r0
+; CHECK-NEXT:    vmov q1[2], q1[0], r0, r4
 ; CHECK-NEXT:    vmov.f64 d0, d2
 ; CHECK-NEXT:    vmov.f32 s1, s6
 ; CHECK-NEXT:    vmov.f32 s2, s4
 ; CHECK-NEXT:    vmov.f32 s3, s6
-; CHECK-NEXT:    vmov.32 q1[0], r2
-; CHECK-NEXT:    vmov.32 q1[1], r3
-; CHECK-NEXT:    vmov.32 q1[2], r12
-; CHECK-NEXT:    vmov.32 q1[3], lr
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, lr
+; CHECK-NEXT:    vmov q1[3], q1[1], r2, r12
 ; CHECK-NEXT:    vmov.f64 d4, d2
 ; CHECK-NEXT:    vmov.f32 s9, s6
 ; CHECK-NEXT:    vmov.f32 s10, s0
 ; CHECK-NEXT:    vmov.f32 s11, s2
 ; CHECK-NEXT:    vstrw.32 q2, [r1]
 ; CHECK-NEXT:    vmov.f32 s8, s5
 ; CHECK-NEXT:    vmov.f32 s9, s7
 ; CHECK-NEXT:    vmov.f32 s10, s1

 
 ; i16
 
 define void @vst4_v2i16(<2 x i16> *%src, <8 x i16> *%dst) {
 ; CHECK-LABEL: vst4_v2i16:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, lr}
 ; CHECK-NEXT:    push {r4, lr}
-; CHECK-NEXT:    ldrh r4, [r0]
-; CHECK-NEXT:    ldrh.w lr, [r0, #4]
+; CHECK-NEXT:    ldrh r2, [r0]
+; CHECK-NEXT:    ldrh.w r12, [r0, #4]
 ; CHECK-NEXT:    ldrh r3, [r0, #8]
-; CHECK-NEXT:    vmov.32 q0[0], r4
-; CHECK-NEXT:    ldrh.w r12, [r0, #6]
-; CHECK-NEXT:    ldrh r2, [r0, #10]
+; CHECK-NEXT:    ldrh.w lr, [r0, #6]
+; CHECK-NEXT:    ldrh r4, [r0, #10]
 ; CHECK-NEXT:    ldrh r0, [r0, #2]
-; CHECK-NEXT:    vmov.32 q0[2], r0
-; CHECK-NEXT:    vmov r4, s0
-; CHECK-NEXT:    vmov.16 q0[0], r4
-; CHECK-NEXT:    vmov.16 q0[1], lr
+; CHECK-NEXT:    vmov q0[2], q0[0], r0, r2
+; CHECK-NEXT:    vmov r2, s0
+; CHECK-NEXT:    vmov.16 q0[0], r2
+; CHECK-NEXT:    vmov.16 q0[1], r12
 ; CHECK-NEXT:    vmov.16 q0[2], r3
 ; CHECK-NEXT:    vmov.16 q0[3], r3
 ; CHECK-NEXT:    vmov.16 q0[4], r0
-; CHECK-NEXT:    vmov.16 q0[5], r12
-; CHECK-NEXT:    vmov.16 q0[6], r2
-; CHECK-NEXT:    vmov.16 q0[7], r2
+; CHECK-NEXT:    vmov.16 q0[5], lr
+; CHECK-NEXT:    vmov.16 q0[6], r4
+; CHECK-NEXT:    vmov.16 q0[7], r4
 ; CHECK-NEXT:    vstrw.32 q0, [r1]
 ; CHECK-NEXT:    pop {r4, pc}
 entry:
   %s1 = getelementptr <2 x i16>, <2 x i16>* %src, i32 0
   %l1 = load <2 x i16>, <2 x i16>* %s1, align 4
   %s2 = getelementptr <2 x i16>, <2 x i16>* %src, i32 1
   %l2 = load <2 x i16>, <2 x i16>* %s2, align 4
   %s3 = getelementptr <2 x i16>, <2 x i16>* %src, i32 2

 
 define void @vst4_v2i8(<2 x i8> *%src, <8 x i8> *%dst) {
 ; CHECK-LABEL: vst4_v2i8:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, lr}
 ; CHECK-NEXT:    push {r4, lr}
 ; CHECK-NEXT:    ldrb r2, [r0]
 ; CHECK-NEXT:    ldrb r3, [r0, #1]
-; CHECK-NEXT:    vmov.32 q0[0], r2
 ; CHECK-NEXT:    ldrb.w r12, [r0, #2]
-; CHECK-NEXT:    vmov.32 q0[2], r3
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
 ; CHECK-NEXT:    ldrb.w lr, [r0, #3]
 ; CHECK-NEXT:    vmov r2, s0
 ; CHECK-NEXT:    ldrb r4, [r0, #5]
 ; CHECK-NEXT:    vmov.16 q0[0], r2
 ; CHECK-NEXT:    ldrb r0, [r0, #4]
 ; CHECK-NEXT:    vmov.16 q0[1], r12
 ; CHECK-NEXT:    vmov.16 q0[2], r0
 ; CHECK-NEXT:    vmov.16 q0[3], r0
entry:
   %s = shufflevector <4 x half> %t1, <4 x half> %t2, <8 x i32> <i32 0, i32 2, i32 4, i32 6, i32 1, i32 3, i32 5, i32 7>
   store <8 x half> %s, <8 x half> *%dst
   ret void
 }
 
 define void @vst4_v4f16(<4 x half> *%src, <16 x half> *%dst) {
 ; CHECK-LABEL: vst4_v4f16:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r7, lr}
-; CHECK-NEXT:    push {r7, lr}
-; CHECK-NEXT:    ldm.w r0, {r2, r3, r12, lr}
-; CHECK-NEXT:    vmov.32 q0[0], r2
-; CHECK-NEXT:    vmov.32 q0[1], r3
-; CHECK-NEXT:    vmov.32 q0[2], r12
-; CHECK-NEXT:    vmov.32 q0[3], lr
-; CHECK-NEXT:    vmov r3, s1
-; CHECK-NEXT:    vmovx.f16 s12, s1
-; CHECK-NEXT:    vmov r2, s3
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    ldrd lr, r12, [r0]
+; CHECK-NEXT:    ldrd r3, r2, [r0, #8]
+; CHECK-NEXT:    ldrd r4, r0, [r0, #16]
+; CHECK-NEXT:    vmov q1[2], q1[0], r3, lr
+; CHECK-NEXT:    vmov q1[3], q1[1], r2, r12
+; CHECK-NEXT:    vmov q0[2], q0[0], r4, r4
+; CHECK-NEXT:    vmov r3, s5
+; CHECK-NEXT:    vmov q0[3], q0[1], r0, r0
+; CHECK-NEXT:    vmov r2, s7
 ; CHECK-NEXT:    vmov.16 q2[0], r3
 ; CHECK-NEXT:    vmov.16 q2[1], r2
-; CHECK-NEXT:    ldrd r2, r0, [r0, #16]
-; CHECK-NEXT:    vmov.32 q1[0], r2
-; CHECK-NEXT:    vmov.32 q1[1], r0
-; CHECK-NEXT:    vmov.32 q1[2], r2
-; CHECK-NEXT:    vmov r2, s0
-; CHECK-NEXT:    vmov.32 q1[3], r0
-; CHECK-NEXT:    vmov r0, s5
+; CHECK-NEXT:    vmov r0, s1
 ; CHECK-NEXT:    vmov.16 q2[2], r0
-; CHECK-NEXT:    vmov r0, s7
+; CHECK-NEXT:    vmov r0, s3
+; CHECK-NEXT:    vmovx.f16 s12, s5
 ; CHECK-NEXT:    vmov.16 q2[3], r0
 ; CHECK-NEXT:    vmov r0, s12
-; CHECK-NEXT:    vmovx.f16 s12, s3
+; CHECK-NEXT:    vmovx.f16 s12, s7
 ; CHECK-NEXT:    vmov.16 q2[4], r0
 ; CHECK-NEXT:    vmov r0, s12
-; CHECK-NEXT:    vmovx.f16 s12, s5
+; CHECK-NEXT:    vmovx.f16 s12, s1
 ; CHECK-NEXT:    vmov.16 q2[5], r0
 ; CHECK-NEXT:    vmov r0, s12
-; CHECK-NEXT:    vmovx.f16 s12, s7
+; CHECK-NEXT:    vmovx.f16 s12, s3
 ; CHECK-NEXT:    vmov.16 q2[6], r0
 ; CHECK-NEXT:    vmov r0, s12
-; CHECK-NEXT:    vmovx.f16 s12, s0
 ; CHECK-NEXT:    vmov.16 q2[7], r0
-; CHECK-NEXT:    vmov r0, s2
+; CHECK-NEXT:    vmov r2, s4
 ; CHECK-NEXT:    vstrw.32 q2, [r1, #16]
+; CHECK-NEXT:    vmov r0, s6
 ; CHECK-NEXT:    vmov.16 q2[0], r2
+; CHECK-NEXT:    vmovx.f16 s12, s4
 ; CHECK-NEXT:    vmov.16 q2[1], r0
-; CHECK-NEXT:    vmov r0, s4
+; CHECK-NEXT:    vmov r0, s0
 ; CHECK-NEXT:    vmov.16 q2[2], r0
-; CHECK-NEXT:    vmov r0, s6
+; CHECK-NEXT:    vmov r0, s2
 ; CHECK-NEXT:    vmov.16 q2[3], r0
 ; CHECK-NEXT:    vmov r0, s12
-; CHECK-NEXT:    vmovx.f16 s0, s2
+; CHECK-NEXT:    vmovx.f16 s4, s6
 ; CHECK-NEXT:    vmov.16 q2[4], r0
-; CHECK-NEXT:    vmov r0, s0
-; CHECK-NEXT:    vmovx.f16 s0, s4
+; CHECK-NEXT:    vmov r0, s4
+; CHECK-NEXT:    vmovx.f16 s4, s0
 ; CHECK-NEXT:    vmov.16 q2[5], r0
-; CHECK-NEXT:    vmov r0, s0
-; CHECK-NEXT:    vmovx.f16 s0, s6
+; CHECK-NEXT:    vmov r0, s4
+; CHECK-NEXT:    vmovx.f16 s0, s2
 ; CHECK-NEXT:    vmov.16 q2[6], r0
 ; CHECK-NEXT:    vmov r0, s0
 ; CHECK-NEXT:    vmov.16 q2[7], r0
 ; CHECK-NEXT:    vstrw.32 q2, [r1]
-; CHECK-NEXT:    pop {r7, pc}
+; CHECK-NEXT:    pop {r4, pc}
 entry:
   %s1 = getelementptr <4 x half>, <4 x half>* %src, i32 0
   %l1 = load <4 x half>, <4 x half>* %s1, align 4
   %s2 = getelementptr <4 x half>, <4 x half>* %src, i32 1
   %l2 = load <4 x half>, <4 x half>* %s2, align 4
   %s3 = getelementptr <4 x half>, <4 x half>* %src, i32 2
   %l3 = load <4 x half>, <4 x half>* %s3, align 4
   %s4 = getelementptr <4 x half>, <4 x half>* %src, i32 3

llvm/test/CodeGen/Thumb2/mve-widen-narrow.ll

 }
 
 
 define void @foo_int32_int8_both(<16 x i32>* %dest, <16 x i8>* readonly %src, i32 %n) {
 ; CHECK-LABEL: foo_int32_int8_both:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vldrb.s16 q1, [r1, #8]
 ; CHECK-NEXT:    vmov.u16 r2, q1[4]
-; CHECK-NEXT:    vmov.32 q0[0], r2
+; CHECK-NEXT:    vmov.u16 r3, q1[6]
+; CHECK-NEXT:    vmov q0[2], q0[0], r3, r2
 ; CHECK-NEXT:    vmov.u16 r2, q1[5]
-; CHECK-NEXT:    vmov.32 q0[1], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[6]
-; CHECK-NEXT:    vmov.32 q0[2], r2
-; CHECK-NEXT:    vmov.u16 r2, q1[7]
-; CHECK-NEXT:    vmov.32 q0[3], r2
+; CHECK-NEXT:    vmov.u16 r3, q1[7]
+; CHECK-NEXT:    vmov q0[3], q0[1], r3, r2
+; CHECK-NEXT:    vmov.u16 r2, q1[2]
 ; CHECK-NEXT:    vmovlb.u16 q2, q0
 ; CHECK-NEXT:    vldrb.s16 q0, [r1]
 ; CHECK-NEXT:    vmov.u16 r1, q1[0]
 ; CHECK-NEXT:    vstrw.32 q2, [r0, #48]
-; CHECK-NEXT:    vmov.32 q2[0], r1
+; CHECK-NEXT:    vmov q2[2], q2[0], r2, r1
 ; CHECK-NEXT:    vmov.u16 r1, q1[1]
-; CHECK-NEXT:    vmov.32 q2[1], r1
-; CHECK-NEXT:    vmov.u16 r1, q1[2]
-; CHECK-NEXT:    vmov.32 q2[2], r1
-; CHECK-NEXT:    vmov.u16 r1, q1[3]
-; CHECK-NEXT:    vmov.32 q2[3], r1
+; CHECK-NEXT:    vmov.u16 r2, q1[3]
+; CHECK-NEXT:    vmov q2[3], q2[1], r2, r1
 ; CHECK-NEXT:    vmov.u16 r1, q0[4]
 ; CHECK-NEXT:    vmovlb.u16 q1, q2
+; CHECK-NEXT:    vmov.u16 r2, q0[6]
 ; CHECK-NEXT:    vstrw.32 q1, [r0, #32]
-; CHECK-NEXT:    vmov.32 q1[0], r1
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, r1
 ; CHECK-NEXT:    vmov.u16 r1, q0[5]
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov.u16 r1, q0[6]
-; CHECK-NEXT:    vmov.32 q1[2], r1
-; CHECK-NEXT:    vmov.u16 r1, q0[7]
-; CHECK-NEXT:    vmov.32 q1[3], r1
+; CHECK-NEXT:    vmov.u16 r2, q0[7]
+; CHECK-NEXT:    vmov q1[3], q1[1], r2, r1
 ; CHECK-NEXT:    vmov.u16 r1, q0[0]
 ; CHECK-NEXT:    vmovlb.u16 q1, q1
+; CHECK-NEXT:    vmov.u16 r2, q0[2]
 ; CHECK-NEXT:    vstrw.32 q1, [r0, #16]
-; CHECK-NEXT:    vmov.32 q1[0], r1
+; CHECK-NEXT:    vmov q1[2], q1[0], r2, r1
 ; CHECK-NEXT:    vmov.u16 r1, q0[1]
-; CHECK-NEXT:    vmov.32 q1[1], r1
-; CHECK-NEXT:    vmov.u16 r1, q0[2]
-; CHECK-NEXT:    vmov.32 q1[2], r1
-; CHECK-NEXT:    vmov.u16 r1, q0[3]
-; CHECK-NEXT:    vmov.32 q1[3], r1
+; CHECK-NEXT:    vmov.u16 r2, q0[3]
+; CHECK-NEXT:    vmov q1[3], q1[1], r2, r1
 ; CHECK-NEXT:    vmovlb.u16 q0, q1
 ; CHECK-NEXT:    vstrw.32 q0, [r0]
 ; CHECK-NEXT:    bx lr
 entry:
   %wide.load = load <16 x i8>, <16 x i8>* %src, align 1
   %0 = sext <16 x i8> %wide.load to <16 x i16>
   %1 = zext <16 x i16> %0 to <16 x i32>
   store <16 x i32> %1, <16 x i32>* %dest, align 4