[ARM] MVE: allow soft-float ABI to pass vector types.

[simon_tatham]

Summary

Passing a vector type over the soft-float ABI involves it being split
into four GPRs, so the first thing that has to happen at the start of
the function is to recombine those into a vector register. The ABI
types all vectors as v2f64, so we need to support BUILD_VECTOR for
that type, which I do in this patch by allowing it to be expanded in
terms of INSERT_VECTOR_ELT, and writing an ISel pattern for that in
turn. Similarly, I provide a rule for EXTRACT_VECTOR_ELT so that a
returned vector can be marshalled back into GPRs.

While I'm here, I've also legalized ISD::UNDEF for all vector types,
because I noticed it was being expanded into a BUILD_VECTOR with
explicit zero inputs, which seems like a waste of effort compared to
the optimal handling of 'just do nothing'.

[dmgreen]

llvm/lib/Target/ARM/ARMISelLowering.cpp

275

What exactly does making UNDEF legal do? Is there a universal pattern to just ignore it? Why is that not the default?

302

Also if we make vector float and longs undef legal, should we be doing the same for integer vectors?

6756

My understanding is that this is (at least part of) the fix for some of the inefficient codegen we see in many of the mve float expansion tests. I would expect a lot of tests to need updating because of it. You can hopefully just run the update script on all the mve-* files.

llvm/test/CodeGen/Thumb2/mve-soft-float-abi.ll

3

I think this should be thumbv8.1m.main-none-eabi.

5

Can you add some float and half tests too. They should be the same (in terms of calling convention), but are easy enough to add tests for.

[simon_tatham]

llvm/lib/Target/ARM/ARMISelLowering.cpp

275

The observable effect of having UNDEF not be legal is this kind of thing you can see in the existing expected results for mve-fmath.ll:

sqrt_float32_t:
@ %bb.0: @ %entry
  vsqrt.f32 s4, s0
  movs r0, #0
  vsqrt.f32 s6, s1
  vsqrt.f32 s8, s2
  vsqrt.f32 s10, s3
  vdup.32 q0, r0
  vmov.f32 s0, s4
  vmov.f32 s1, s6
  vmov.f32 s2, s8
  vmov.f32 s3, s10
  bx lr

in which the movs r0,#0 and vdup.32 q0,r0 pair are filling q0 with all zeroes, which is pointless because the next four instructions overwrite every part of q0 in any case. That happens because an ISD::UNDEF for that vector type is not regarded as legal, and the fallback lowering turns it into 'make a vector of zeroes'.

I have to suppose that UNDEF is legal by default and does the obvious thing – it's just that when we use the blunt instrument of setAllExpand, that sets it to Expand along with everything else, and we have to remember to put it back again.

302

I think that's not necessary, because we didn't make it illegal in the first place by calling setAllExpand.

[simon_tatham]

Revised patch is intended to apply after D63938 rather than before.

Moved the re-enablement of ISD::UNDEF into setAllExpand itself
(after the other patch started centralising things in there), because
that way it will reliably be turned back on for any type where we
turned it off in the first place.

Also fixed the bogus triple, added some soft-float-abi tests for other
types, and updated a load of tests using update_llc_test_checks.

In order to get the new tests for handling vectors of floats to pass,
I also had to legalize INSERT_VECTOR_ELT with the type set to the
vector's element type, rather than the type of the vector as a
whole, because that seems to be how it gets queried in some situations
(particularly, if the scalar operand is of a floating type that needs
promotion, like f16).

[dmgreen]

Nice one. LGTM

llvm/lib/Target/ARM/ARMISelLowering.cpp

275

OK that makes sense. It should be legal anyway!

llvm/test/CodeGen/Thumb2/mve-soft-float-abi.ll

59

Are these auto-generated? It doesn't show the expanded form because it doesn't have a unique check-prefix? I thought this usually gave an error.

[simon_tatham]

llvm/test/CodeGen/Thumb2/mve-soft-float-abi.ll

59

I don't know about that – I just reran update_llc_test_checks on this file and it left it alone without complaint.

I deliberately left the expected results for the software-FP expanded form out of the test because it's absolutely enormous, and even more than most of these tests, full of arbitrary decisions about what order to do things in. I feel as if you'd be forever regenerating it on every tiny perturbation of this area of the code, and every time you did, it would be impossible to manually confirm the correctness of the new version without making mistakes...

llvm/lib/Target/ARM/ARMISelLowering.h

private:
     SDValue LowerFRAMEADDR(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerShiftRightParts(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerShiftLeftParts(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerFLT_ROUNDS_(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerConstantFP(SDValue Op, SelectionDAG &DAG,
                             const ARMSubtarget *ST) const;
     SDValue LowerBUILD_VECTOR(SDValue Op, SelectionDAG &DAG,
                               const ARMSubtarget *ST) const;
+    SDValue LowerINSERT_VECTOR_ELT(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerFSINCOS(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerDivRem(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerDIV_Windows(SDValue Op, SelectionDAG &DAG, bool Signed) const;
     void ExpandDIV_Windows(SDValue Op, SelectionDAG &DAG, bool Signed,
                            SmallVectorImpl<SDValue> &Results) const;
     SDValue LowerWindowsDIVLibCall(SDValue Op, SelectionDAG &DAG, bool Signed,
                                    SDValue &Chain) const;
     SDValue LowerREM(SDNode *N, SelectionDAG &DAG) const;

llvm/lib/Target/ARM/ARMISelLowering.cpp

for (unsigned Opc = 0; Opc < ISD::BUILTIN_OP_END; ++Opc)
     setOperationAction(Opc, VT, Expand);
 
   // We support these really simple operations even on types where all
   // the actual arithmetic has to be broken down into simpler
   // operations or turned into library calls.
   setOperationAction(ISD::BITCAST, VT, Legal);
   setOperationAction(ISD::LOAD, VT, Legal);
   setOperationAction(ISD::STORE, VT, Legal);
+  setOperationAction(ISD::UNDEF, VT, Legal);
 }
 
 void ARMTargetLowering::addAllExtLoads(const MVT From, const MVT To,
                                        LegalizeAction Action) {
   setLoadExtAction(ISD::EXTLOAD,  From, To, Action);
   setLoadExtAction(ISD::ZEXTLOAD, From, To, Action);
   setLoadExtAction(ISD::SEXTLOAD, From, To, Action);
 }
void ARMTargetLowering::addMVEVectorTypes(bool HasMVEFP) {
   for (auto VT : FloatTypes) {
     addRegisterClass(VT, &ARM::QPRRegClass);
     if (!HasMVEFP)
       setAllExpand(VT);
 
     // These are legal or custom whether we have MVE.fp or not
     setOperationAction(ISD::VECTOR_SHUFFLE, VT, Custom);
     setOperationAction(ISD::INSERT_VECTOR_ELT, VT, Custom);
+    setOperationAction(ISD::INSERT_VECTOR_ELT, VT.getVectorElementType(), Custom);
     setOperationAction(ISD::EXTRACT_VECTOR_ELT, VT, Custom);
     setOperationAction(ISD::BUILD_VECTOR, VT, Custom);
+    setOperationAction(ISD::BUILD_VECTOR, VT.getVectorElementType(), Custom);
     setOperationAction(ISD::SCALAR_TO_VECTOR, VT, Legal);
 
     if (HasMVEFP) {
       // No native support for these.
       setOperationAction(ISD::FDIV, VT, Expand);
       setOperationAction(ISD::FREM, VT, Expand);
       setOperationAction(ISD::FSQRT, VT, Expand);
       setOperationAction(ISD::FSIN, VT, Expand);
       setOperationAction(ISD::FCOS, VT, Expand);
       setOperationAction(ISD::FPOW, VT, Expand);
       setOperationAction(ISD::FLOG, VT, Expand);
       setOperationAction(ISD::FLOG2, VT, Expand);
       setOperationAction(ISD::FLOG10, VT, Expand);
       setOperationAction(ISD::FEXP, VT, Expand);
       setOperationAction(ISD::FEXP2, VT, Expand);
     }
   }
 
   // We 'support' these types up to bitcast/load/store level, regardless of
   // MVE integer-only / float support. Only doing FP data processing on the FP
   // vector types is inhibited at integer-only level.
   const MVT LongTypes[] = { MVT::v2i64, MVT::v2f64 };
   for (auto VT : LongTypes) {
     addRegisterClass(VT, &ARM::QPRRegClass);
     setAllExpand(VT);
+    setOperationAction(ISD::INSERT_VECTOR_ELT, VT, Custom);
+    setOperationAction(ISD::EXTRACT_VECTOR_ELT, VT, Custom);
+    setOperationAction(ISD::BUILD_VECTOR, VT, Custom);
   }
 
   // It is legal to extload from v4i8 to v4i16 or v4i32.
   addAllExtLoads(MVT::v8i16, MVT::v8i8, Legal);
   addAllExtLoads(MVT::v4i32, MVT::v4i16, Legal);
   addAllExtLoads(MVT::v4i32, MVT::v4i8, Legal);
 
   // Some truncating stores are legal too.
   setTruncStoreAction(MVT::v4i32, MVT::v4i16, Legal);
if (Upper.getOpcode() == ISD::BUILD_VECTOR)
       Upper = LowerBUILD_VECTOR(Upper, DAG, ST);
     if (Lower && Upper)
       return DAG.getNode(ISD::CONCAT_VECTORS, dl, VT, Lower, Upper);
   }
 
   // Vectors with 32- or 64-bit elements can be built by directly assigning
   // the subregisters.  Lower it to an ARMISD::BUILD_VECTOR so the operands
   // will be legalized.
-  if (ST->hasNEON() && EltSize >= 32) {
+  if (EltSize >= 32) {
     // Do the expansion with floating-point types, since that is what the VFP
     // registers are defined to use, and since i64 is not legal.
     EVT EltVT = EVT::getFloatingPointVT(EltSize);
     EVT VecVT = EVT::getVectorVT(*DAG.getContext(), EltVT, NumElts);
     SmallVector<SDValue, 8> Ops;
     for (unsigned i = 0; i < NumElts; ++i)
       Ops.push_back(DAG.getNode(ISD::BITCAST, dl, EltVT, Op.getOperand(i)));
     SDValue Val = DAG.getNode(ARMISD::BUILD_VECTOR, dl, VecVT, Ops);
static SDValue LowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG,
 
   if (ST->hasNEON() && VT == MVT::v8i8)
     if (SDValue NewOp = LowerVECTOR_SHUFFLEv8i8(Op, ShuffleMask, DAG))
       return NewOp;
 
   return SDValue();
 }
 
-static SDValue LowerINSERT_VECTOR_ELT(SDValue Op, SelectionDAG &DAG) {
+SDValue ARMTargetLowering::
+LowerINSERT_VECTOR_ELT(SDValue Op, SelectionDAG &DAG) const {
   // INSERT_VECTOR_ELT is legal only for immediate indexes.
   SDValue Lane = Op.getOperand(2);
   if (!isa<ConstantSDNode>(Lane))
     return SDValue();
 
+  SDValue Elt = Op.getOperand(1);
+  EVT EltVT = Elt.getValueType();
+  if (getTypeAction(*DAG.getContext(), EltVT) ==
+      TargetLowering::TypePromoteFloat) {
+    // INSERT_VECTOR_ELT doesn't want f16 operands promoting to f32,
+    // but the type system will try to do that if we don't intervene.
+    // Reinterpret any such vector-element insertion as one with the
+    // corresponding integer types.
+
+    SDLoc dl(Op);
+
+    EVT IEltVT = MVT::getIntegerVT(EltVT.getScalarSizeInBits());
+    assert(getTypeAction(*DAG.getContext(), IEltVT) !=
+           TargetLowering::TypePromoteFloat);
+
+    SDValue VecIn = Op.getOperand(0);
+    EVT VecVT = VecIn.getValueType();
+    EVT IVecVT = EVT::getVectorVT(*DAG.getContext(), IEltVT,
+                                  VecVT.getVectorNumElements());
+
+    SDValue IElt = DAG.getNode(ISD::BITCAST, dl, IEltVT, Elt);
+    SDValue IVecIn = DAG.getNode(ISD::BITCAST, dl, IVecVT, VecIn);
+    SDValue IVecOut = DAG.getNode(ISD::INSERT_VECTOR_ELT, dl, IVecVT,
+                                  IVecIn, IElt, Lane);
+    return DAG.getNode(ISD::BITCAST, dl, VecVT, IVecOut);
+  }
+
   return Op;
 }
 
 static SDValue LowerEXTRACT_VECTOR_ELT(SDValue Op, SelectionDAG &DAG) {
   // EXTRACT_VECTOR_ELT is legal only for immediate indexes.
   SDValue Lane = Op.getOperand(1);
   if (!isa<ConstantSDNode>(Lane))
     return SDValue();

llvm/lib/Target/ARM/ARMInstrMVE.td

 def MVE_VMOV_from_lane_s16 : MVE_VMOV_lane_16<"s16", 0b0, MVE_VMOV_from_lane>;
 def MVE_VMOV_from_lane_u16 : MVE_VMOV_lane_16<"u16", 0b1, MVE_VMOV_from_lane>;
 def MVE_VMOV_to_lane_16    : MVE_VMOV_lane_16< "16", 0b0, MVE_VMOV_to_lane>;
 def MVE_VMOV_from_lane_s8  : MVE_VMOV_lane_8 < "s8", 0b0, MVE_VMOV_from_lane>;
 def MVE_VMOV_from_lane_u8  : MVE_VMOV_lane_8 < "u8", 0b1, MVE_VMOV_from_lane>;
 def MVE_VMOV_to_lane_8     : MVE_VMOV_lane_8 <  "8", 0b0, MVE_VMOV_to_lane>;
 
 let Predicates = [HasMVEInt] in {
+  def : Pat<(extractelt (v2f64 MQPR:$src), imm:$lane),
+            (f64 (EXTRACT_SUBREG MQPR:$src, (DSubReg_f64_reg imm:$lane)))>;
+  def : Pat<(insertelt (v2f64 MQPR:$src1), DPR:$src2, imm:$lane),
+            (INSERT_SUBREG (v2f64 (COPY_TO_REGCLASS MQPR:$src1, MQPR)), DPR:$src2, (DSubReg_f64_reg imm:$lane))>;
+
   def : Pat<(extractelt (v4i32 MQPR:$src), imm:$lane),
             (COPY_TO_REGCLASS
               (i32 (EXTRACT_SUBREG MQPR:$src, (SSubReg_f32_reg imm:$lane))), rGPR)>;
   def : Pat<(insertelt (v4i32 MQPR:$src1), rGPR:$src2, imm:$lane),
             (MVE_VMOV_to_lane_32 MQPR:$src1, rGPR:$src2, imm:$lane)>;
 
   def : Pat<(vector_insert (v16i8 MQPR:$src1), rGPR:$src2, imm:$lane),
             (MVE_VMOV_to_lane_8  MQPR:$src1, rGPR:$src2, imm:$lane)>;

llvm/test/CodeGen/Thumb2/mve-div-expand.ll

 ; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, pc}
 entry:
   %out = srem <16 x i8> %in1, %in2
   ret <16 x i8> %out
 }
 
 
 define arm_aapcs_vfpcc <4 x float> @fdiv_f32(<4 x float> %in1, <4 x float> %in2) {
-; CHECK-MVE-LABEL: fdiv_f32:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    vdiv.f32 s8, s0, s4
-; CHECK-MVE-NEXT:    movs r0, #0
-; CHECK-MVE-NEXT:    vdiv.f32 s10, s1, s5
-; CHECK-MVE-NEXT:    vdiv.f32 s12, s2, s6
-; CHECK-MVE-NEXT:    vdiv.f32 s4, s3, s7
-; CHECK-MVE-NEXT:    vdup.32 q0, r0
-; CHECK-MVE-NEXT:    vmov.f32 s0, s8
-; CHECK-MVE-NEXT:    vmov.f32 s1, s10
-; CHECK-MVE-NEXT:    vmov.f32 s2, s12
-; CHECK-MVE-NEXT:    vmov.f32 s3, s4
-; CHECK-MVE-NEXT:    bx lr
-;
-; CHECK-MVEFP-LABEL: fdiv_f32:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    vmov q2, q0
-; CHECK-MVEFP-NEXT:    vdiv.f32 s0, s8, s4
-; CHECK-MVEFP-NEXT:    vdiv.f32 s1, s9, s5
-; CHECK-MVEFP-NEXT:    vdiv.f32 s2, s10, s6
-; CHECK-MVEFP-NEXT:    vdiv.f32 s3, s11, s7
-; CHECK-MVEFP-NEXT:    bx lr
+; CHECK-LABEL: fdiv_f32:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    vdiv.f32 s11, s3, s7
+; CHECK-NEXT:    vdiv.f32 s10, s2, s6
+; CHECK-NEXT:    vdiv.f32 s9, s1, s5
+; CHECK-NEXT:    vdiv.f32 s8, s0, s4
+; CHECK-NEXT:    vmov q0, q2
+; CHECK-NEXT:    bx lr
 entry:
   %out = fdiv <4 x float> %in1, %in2
   ret <4 x float> %out
 }
 
 define arm_aapcs_vfpcc <4 x float> @frem_f32(<4 x float> %in1, <4 x float> %in2) {
-; CHECK-MVE-LABEL: frem_f32:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    .save {r4, r5, r6, lr}
-; CHECK-MVE-NEXT:    push {r4, r5, r6, lr}
-; CHECK-MVE-NEXT:    .vsave {d8, d9, d10}
-; CHECK-MVE-NEXT:    vpush {d8, d9, d10}
-; CHECK-MVE-NEXT:    .pad #32
-; CHECK-MVE-NEXT:    sub sp, #32
-; CHECK-MVE-NEXT:    vstr s3, [sp, #24]
-; CHECK-MVE-NEXT:    ldr r4, [sp, #24]
-; CHECK-MVE-NEXT:    vstr s1, [sp, #8]
-; CHECK-MVE-NEXT:    vstr s5, [sp, #12]
-; CHECK-MVE-NEXT:    vstr s0, [sp]
-; CHECK-MVE-NEXT:    vstr s4, [sp, #4]
-; CHECK-MVE-NEXT:    vstr s7, [sp, #28]
-; CHECK-MVE-NEXT:    vstr s2, [sp, #16]
-; CHECK-MVE-NEXT:    vstr s6, [sp, #20]
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp, #8]
-; CHECK-MVE-NEXT:    bl fmodf
-; CHECK-MVE-NEXT:    mov r5, r0
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp]
-; CHECK-MVE-NEXT:    bl fmodf
-; CHECK-MVE-NEXT:    ldrd r2, r1, [sp, #16]
-; CHECK-MVE-NEXT:    vmov s16, r0
-; CHECK-MVE-NEXT:    ldr r6, [sp, #28]
-; CHECK-MVE-NEXT:    vmov s18, r5
-; CHECK-MVE-NEXT:    mov r0, r2
-; CHECK-MVE-NEXT:    bl fmodf
-; CHECK-MVE-NEXT:    vmov s20, r0
-; CHECK-MVE-NEXT:    mov r0, r4
-; CHECK-MVE-NEXT:    mov r1, r6
-; CHECK-MVE-NEXT:    bl fmodf
-; CHECK-MVE-NEXT:    movs r1, #0
-; CHECK-MVE-NEXT:    vmov s4, r0
-; CHECK-MVE-NEXT:    vdup.32 q0, r1
-; CHECK-MVE-NEXT:    vmov.f32 s0, s16
-; CHECK-MVE-NEXT:    vmov.f32 s1, s18
-; CHECK-MVE-NEXT:    vmov.f32 s2, s20
-; CHECK-MVE-NEXT:    vmov.f32 s3, s4
-; CHECK-MVE-NEXT:    add sp, #32
-; CHECK-MVE-NEXT:    vpop {d8, d9, d10}
-; CHECK-MVE-NEXT:    pop {r4, r5, r6, pc}
-;
-; CHECK-MVEFP-LABEL: frem_f32:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    .save {r4, r5, r6, lr}
-; CHECK-MVEFP-NEXT:    push {r4, r5, r6, lr}
-; CHECK-MVEFP-NEXT:    .vsave {d8, d9}
-; CHECK-MVEFP-NEXT:    vpush {d8, d9}
-; CHECK-MVEFP-NEXT:    .pad #32
-; CHECK-MVEFP-NEXT:    sub sp, #32
-; CHECK-MVEFP-NEXT:    vstr s3, [sp, #24]
-; CHECK-MVEFP-NEXT:    ldr r4, [sp, #24]
-; CHECK-MVEFP-NEXT:    vstr s1, [sp, #8]
-; CHECK-MVEFP-NEXT:    vstr s5, [sp, #12]
-; CHECK-MVEFP-NEXT:    vstr s0, [sp]
-; CHECK-MVEFP-NEXT:    vstr s4, [sp, #4]
-; CHECK-MVEFP-NEXT:    vstr s7, [sp, #28]
-; CHECK-MVEFP-NEXT:    vstr s2, [sp, #16]
-; CHECK-MVEFP-NEXT:    vstr s6, [sp, #20]
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp, #8]
-; CHECK-MVEFP-NEXT:    bl fmodf
-; CHECK-MVEFP-NEXT:    mov r5, r0
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp]
-; CHECK-MVEFP-NEXT:    bl fmodf
-; CHECK-MVEFP-NEXT:    ldrd r2, r1, [sp, #16]
-; CHECK-MVEFP-NEXT:    vmov s16, r0
-; CHECK-MVEFP-NEXT:    ldr r6, [sp, #28]
-; CHECK-MVEFP-NEXT:    vmov s17, r5
-; CHECK-MVEFP-NEXT:    mov r0, r2
-; CHECK-MVEFP-NEXT:    bl fmodf
-; CHECK-MVEFP-NEXT:    vmov s18, r0
-; CHECK-MVEFP-NEXT:    mov r0, r4
-; CHECK-MVEFP-NEXT:    mov r1, r6
-; CHECK-MVEFP-NEXT:    bl fmodf
-; CHECK-MVEFP-NEXT:    vmov s19, r0
-; CHECK-MVEFP-NEXT:    vmov q0, q4
-; CHECK-MVEFP-NEXT:    add sp, #32
-; CHECK-MVEFP-NEXT:    vpop {d8, d9}
-; CHECK-MVEFP-NEXT:    pop {r4, r5, r6, pc}
+; CHECK-LABEL: frem_f32:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, r5, r6, lr}
+; CHECK-NEXT:    push {r4, r5, r6, lr}
+; CHECK-NEXT:    .vsave {d8, d9}
+; CHECK-NEXT:    vpush {d8, d9}
+; CHECK-NEXT:    .pad #32
+; CHECK-NEXT:    sub sp, #32
+; CHECK-NEXT:    vstr s0, [sp]
+; CHECK-NEXT:    ldr r4, [sp]
+; CHECK-NEXT:    vstr s2, [sp, #16]
+; CHECK-NEXT:    vstr s6, [sp, #20]
+; CHECK-NEXT:    vstr s3, [sp, #24]
+; CHECK-NEXT:    vstr s7, [sp, #28]
+; CHECK-NEXT:    vstr s4, [sp, #4]
+; CHECK-NEXT:    vstr s1, [sp, #8]
+; CHECK-NEXT:    vstr s5, [sp, #12]
+; CHECK-NEXT:    ldrd r0, r1, [sp, #16]
+; CHECK-NEXT:    bl fmodf
+; CHECK-NEXT:    mov r5, r0
+; CHECK-NEXT:    ldrd r0, r1, [sp, #24]
+; CHECK-NEXT:    bl fmodf
+; CHECK-NEXT:    ldr r1, [sp, #12]
+; CHECK-NEXT:    vmov s19, r0
+; CHECK-NEXT:    ldrd r6, r2, [sp, #4]
+; CHECK-NEXT:    vmov s18, r5
+; CHECK-NEXT:    mov r0, r2
+; CHECK-NEXT:    bl fmodf
+; CHECK-NEXT:    vmov s17, r0
+; CHECK-NEXT:    mov r0, r4
+; CHECK-NEXT:    mov r1, r6
+; CHECK-NEXT:    bl fmodf
+; CHECK-NEXT:    vmov s16, r0
+; CHECK-NEXT:    vmov q0, q4
+; CHECK-NEXT:    add sp, #32
+; CHECK-NEXT:    vpop {d8, d9}
+; CHECK-NEXT:    pop {r4, r5, r6, pc}
 entry:
   %out = frem <4 x float> %in1, %in2
   ret <4 x float> %out
 }
 
 
 define arm_aapcs_vfpcc <8 x half> @fdiv_f16(<8 x half> %in1, <8 x half> %in2) {
-; CHECK-MVE-LABEL: fdiv_f16:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    vmov.u16 r0, q1[0]
-; CHECK-MVE-NEXT:    vmov.u16 r1, q0[0]
-; CHECK-MVE-NEXT:    vmov s10, r1
-; CHECK-MVE-NEXT:    vmov.u16 r1, q1[1]
-; CHECK-MVE-NEXT:    vmov s8, r0
-; CHECK-MVE-NEXT:    movs r2, #0
-; CHECK-MVE-NEXT:    vdiv.f16 s8, s10, s8
-; CHECK-MVE-NEXT:    vmov r0, s8
-; CHECK-MVE-NEXT:    vmov s8, r1
-; CHECK-MVE-NEXT:    vmov.u16 r1, q0[1]
-; CHECK-MVE-NEXT:    vmov s10, r1
-; CHECK-MVE-NEXT:    vdiv.f16 s8, s10, s8
-; CHECK-MVE-NEXT:    vmov r1, s8
-; CHECK-MVE-NEXT:    vdup.16 q2, r2
-; CHECK-MVE-NEXT:    vmov.16 q2[0], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q1[2]
-; CHECK-MVE-NEXT:    vmov s12, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-MVE-NEXT:    vmov s14, r0
-; CHECK-MVE-NEXT:    vmov.16 q2[1], r1
-; CHECK-MVE-NEXT:    vdiv.f16 s12, s14, s12
-; CHECK-MVE-NEXT:    vmov r0, s12
-; CHECK-MVE-NEXT:    vmov.16 q2[2], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q1[3]
-; CHECK-MVE-NEXT:    vmov s12, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[3]
-; CHECK-MVE-NEXT:    vmov s14, r0
-; CHECK-MVE-NEXT:    vdiv.f16 s12, s14, s12
-; CHECK-MVE-NEXT:    vmov r0, s12
-; CHECK-MVE-NEXT:    vmov.16 q2[3], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q1[4]
-; CHECK-MVE-NEXT:    vmov s12, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[4]
-; CHECK-MVE-NEXT:    vmov s14, r0
-; CHECK-MVE-NEXT:    vdiv.f16 s12, s14, s12
-; CHECK-MVE-NEXT:    vmov r0, s12
-; CHECK-MVE-NEXT:    vmov.16 q2[4], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q1[5]
-; CHECK-MVE-NEXT:    vmov s12, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[5]
-; CHECK-MVE-NEXT:    vmov s14, r0
-; CHECK-MVE-NEXT:    vdiv.f16 s12, s14, s12
-; CHECK-MVE-NEXT:    vmov r0, s12
-; CHECK-MVE-NEXT:    vmov.16 q2[5], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q1[6]
-; CHECK-MVE-NEXT:    vmov s12, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[6]
-; CHECK-MVE-NEXT:    vmov s14, r0
-; CHECK-MVE-NEXT:    vdiv.f16 s12, s14, s12
-; CHECK-MVE-NEXT:    vmov r0, s12
-; CHECK-MVE-NEXT:    vmov.16 q2[6], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q1[7]
-; CHECK-MVE-NEXT:    vmov s4, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[7]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vdiv.f16 s0, s0, s4
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q2[7], r0
-; CHECK-MVE-NEXT:    vmov q0, q2
-; CHECK-MVE-NEXT:    bx lr
-;
-; CHECK-MVEFP-LABEL: fdiv_f16:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q1[0]
-; CHECK-MVEFP-NEXT:    vmov.u16 r1, q1[1]
-; CHECK-MVEFP-NEXT:    vmov s8, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-MVEFP-NEXT:    vmov s10, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r2, q0[1]
-; CHECK-MVEFP-NEXT:    vdiv.f16 s8, s10, s8
-; CHECK-MVEFP-NEXT:    vmov s10, r2
-; CHECK-MVEFP-NEXT:    vmov r0, s8
-; CHECK-MVEFP-NEXT:    vmov s8, r1
-; CHECK-MVEFP-NEXT:    vdiv.f16 s8, s10, s8
-; CHECK-MVEFP-NEXT:    vmov r1, s8
-; CHECK-MVEFP-NEXT:    vmov.16 q2[0], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q1[2]
-; CHECK-MVEFP-NEXT:    vmov.16 q2[1], r1
-; CHECK-MVEFP-NEXT:    vmov s12, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-MVEFP-NEXT:    vmov s14, r0
-; CHECK-MVEFP-NEXT:    vdiv.f16 s12, s14, s12
-; CHECK-MVEFP-NEXT:    vmov r0, s12
-; CHECK-MVEFP-NEXT:    vmov.16 q2[2], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q1[3]
-; CHECK-MVEFP-NEXT:    vmov s12, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[3]
-; CHECK-MVEFP-NEXT:    vmov s14, r0
-; CHECK-MVEFP-NEXT:    vdiv.f16 s12, s14, s12
-; CHECK-MVEFP-NEXT:    vmov r0, s12
-; CHECK-MVEFP-NEXT:    vmov.16 q2[3], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q1[4]
-; CHECK-MVEFP-NEXT:    vmov s12, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[4]
-; CHECK-MVEFP-NEXT:    vmov s14, r0
-; CHECK-MVEFP-NEXT:    vdiv.f16 s12, s14, s12
-; CHECK-MVEFP-NEXT:    vmov r0, s12
-; CHECK-MVEFP-NEXT:    vmov.16 q2[4], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q1[5]
-; CHECK-MVEFP-NEXT:    vmov s12, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[5]
-; CHECK-MVEFP-NEXT:    vmov s14, r0
-; CHECK-MVEFP-NEXT:    vdiv.f16 s12, s14, s12
-; CHECK-MVEFP-NEXT:    vmov r0, s12
-; CHECK-MVEFP-NEXT:    vmov.16 q2[5], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q1[6]
-; CHECK-MVEFP-NEXT:    vmov s12, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[6]
-; CHECK-MVEFP-NEXT:    vmov s14, r0
-; CHECK-MVEFP-NEXT:    vdiv.f16 s12, s14, s12
-; CHECK-MVEFP-NEXT:    vmov r0, s12
-; CHECK-MVEFP-NEXT:    vmov.16 q2[6], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q1[7]
-; CHECK-MVEFP-NEXT:    vmov s4, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[7]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vdiv.f16 s0, s0, s4
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q2[7], r0
-; CHECK-MVEFP-NEXT:    vmov q0, q2
-; CHECK-MVEFP-NEXT:    bx lr
+; CHECK-LABEL: fdiv_f16:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    vmov.u16 r0, q1[0]
+; CHECK-NEXT:    vmov.u16 r1, q1[1]
+; CHECK-NEXT:    vmov s8, r0
+; CHECK-NEXT:    vmov.u16 r0, q0[0]
+; CHECK-NEXT:    vmov s10, r0
+; CHECK-NEXT:    vmov.u16 r2, q0[1]
+; CHECK-NEXT:    vdiv.f16 s8, s10, s8
+; CHECK-NEXT:    vmov s10, r2
+; CHECK-NEXT:    vmov r0, s8
+; CHECK-NEXT:    vmov s8, r1
+; CHECK-NEXT:    vdiv.f16 s8, s10, s8
+; CHECK-NEXT:    vmov r1, s8
+; CHECK-NEXT:    vmov.16 q2[0], r0
+; CHECK-NEXT:    vmov.u16 r0, q1[2]
+; CHECK-NEXT:    vmov.16 q2[1], r1
+; CHECK-NEXT:    vmov s12, r0
+; CHECK-NEXT:    vmov.u16 r0, q0[2]
+; CHECK-NEXT:    vmov s14, r0
+; CHECK-NEXT:    vdiv.f16 s12, s14, s12
+; CHECK-NEXT:    vmov r0, s12
+; CHECK-NEXT:    vmov.16 q2[2], r0
+; CHECK-NEXT:    vmov.u16 r0, q1[3]
+; CHECK-NEXT:    vmov s12, r0
+; CHECK-NEXT:    vmov.u16 r0, q0[3]
+; CHECK-NEXT:    vmov s14, r0
+; CHECK-NEXT:    vdiv.f16 s12, s14, s12
+; CHECK-NEXT:    vmov r0, s12
+; CHECK-NEXT:    vmov.16 q2[3], r0
+; CHECK-NEXT:    vmov.u16 r0, q1[4]
+; CHECK-NEXT:    vmov s12, r0
+; CHECK-NEXT:    vmov.u16 r0, q0[4]
+; CHECK-NEXT:    vmov s14, r0
+; CHECK-NEXT:    vdiv.f16 s12, s14, s12
+; CHECK-NEXT:    vmov r0, s12
+; CHECK-NEXT:    vmov.16 q2[4], r0
+; CHECK-NEXT:    vmov.u16 r0, q1[5]
+; CHECK-NEXT:    vmov s12, r0
+; CHECK-NEXT:    vmov.u16 r0, q0[5]
+; CHECK-NEXT:    vmov s14, r0
+; CHECK-NEXT:    vdiv.f16 s12, s14, s12
+; CHECK-NEXT:    vmov r0, s12
+; CHECK-NEXT:    vmov.16 q2[5], r0
+; CHECK-NEXT:    vmov.u16 r0, q1[6]
+; CHECK-NEXT:    vmov s12, r0
+; CHECK-NEXT:    vmov.u16 r0, q0[6]
+; CHECK-NEXT:    vmov s14, r0
+; CHECK-NEXT:    vdiv.f16 s12, s14, s12
+; CHECK-NEXT:    vmov r0, s12
+; CHECK-NEXT:    vmov.16 q2[6], r0
+; CHECK-NEXT:    vmov.u16 r0, q1[7]
+; CHECK-NEXT:    vmov s4, r0
+; CHECK-NEXT:    vmov.u16 r0, q0[7]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vdiv.f16 s0, s0, s4
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q2[7], r0
+; CHECK-NEXT:    vmov q0, q2
+; CHECK-NEXT:    bx lr
 entry:
   %out = fdiv <8 x half> %in1, %in2
   ret <8 x half> %out
 }
 
 define arm_aapcs_vfpcc <8 x half> @frem_f16(<8 x half> %in1, <8 x half> %in2) {
-; CHECK-MVE-LABEL: frem_f16:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    .save {r4, lr}
-; CHECK-MVE-NEXT:    push {r4, lr}
-; CHECK-MVE-NEXT:    .vsave {d8, d9, d10, d11, d12, d13}
-; CHECK-MVE-NEXT:    vpush {d8, d9, d10, d11, d12, d13}
-; CHECK-MVE-NEXT:    .pad #64
-; CHECK-MVE-NEXT:    sub sp, #64
-; CHECK-MVE-NEXT:    vmov q5, q1
-; CHECK-MVE-NEXT:    vmov q4, q0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q1[0]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-MVE-NEXT:    vmov s2, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVE-NEXT:    vstr s2, [sp, #56]
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #60]
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp, #56]
-; CHECK-MVE-NEXT:    bl fmodf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q5[1]
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r4, s0
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[1]
-; CHECK-MVE-NEXT:    vmov s2, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVE-NEXT:    vstr s2, [sp, #48]
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #52]
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp, #48]
-; CHECK-MVE-NEXT:    bl fmodf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    movs r1, #0
-; CHECK-MVE-NEXT:    vdup.16 q6, r1
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q6[0], r4
-; CHECK-MVE-NEXT:    vmov.16 q6[1], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q5[2]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-MVE-NEXT:    vmov s2, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVE-NEXT:    vstr s2, [sp, #40]
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #44]
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp, #40]
-; CHECK-MVE-NEXT:    bl fmodf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q6[2], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q5[3]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-MVE-NEXT:    vmov s2, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVE-NEXT:    vstr s2, [sp, #32]
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #36]
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp, #32]
-; CHECK-MVE-NEXT:    bl fmodf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q6[3], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q5[4]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[4]
-; CHECK-MVE-NEXT:    vmov s2, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVE-NEXT:    vstr s2, [sp, #24]
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #28]
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp, #24]
-; CHECK-MVE-NEXT:    bl fmodf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q6[4], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q5[5]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[5]
-; CHECK-MVE-NEXT:    vmov s2, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVE-NEXT:    vstr s2, [sp, #16]
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #20]
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp, #16]
-; CHECK-MVE-NEXT:    bl fmodf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q6[5], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q5[6]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[6]
-; CHECK-MVE-NEXT:    vmov s2, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVE-NEXT:    vstr s2, [sp, #8]
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #12]
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp, #8]
-; CHECK-MVE-NEXT:    bl fmodf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q6[6], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q5[7]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[7]
-; CHECK-MVE-NEXT:    vmov s2, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVE-NEXT:    vstr s2, [sp]
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #4]
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp]
-; CHECK-MVE-NEXT:    bl fmodf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q6[7], r0
-; CHECK-MVE-NEXT:    vmov q0, q6
-; CHECK-MVE-NEXT:    add sp, #64
-; CHECK-MVE-NEXT:    vpop {d8, d9, d10, d11, d12, d13}
-; CHECK-MVE-NEXT:    pop {r4, pc}
-;
-; CHECK-MVEFP-LABEL: frem_f16:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    .save {r4, lr}
-; CHECK-MVEFP-NEXT:    push {r4, lr}
-; CHECK-MVEFP-NEXT:    .vsave {d8, d9, d10, d11, d12, d13}
-; CHECK-MVEFP-NEXT:    vpush {d8, d9, d10, d11, d12, d13}
-; CHECK-MVEFP-NEXT:    .pad #64
-; CHECK-MVEFP-NEXT:    sub sp, #64
-; CHECK-MVEFP-NEXT:    vmov q5, q1
-; CHECK-MVEFP-NEXT:    vmov q4, q0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q1[0]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-MVEFP-NEXT:    vmov s2, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVEFP-NEXT:    vstr s2, [sp, #56]
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #60]
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp, #56]
-; CHECK-MVEFP-NEXT:    bl fmodf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q5[1]
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r4, s0
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[1]
-; CHECK-MVEFP-NEXT:    vmov s2, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVEFP-NEXT:    vstr s2, [sp, #48]
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #52]
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp, #48]
-; CHECK-MVEFP-NEXT:    bl fmodf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.16 q6[0], r4
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q6[1], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q5[2]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-MVEFP-NEXT:    vmov s2, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVEFP-NEXT:    vstr s2, [sp, #40]
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #44]
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp, #40]
-; CHECK-MVEFP-NEXT:    bl fmodf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q6[2], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q5[3]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-MVEFP-NEXT:    vmov s2, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVEFP-NEXT:    vstr s2, [sp, #32]
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #36]
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp, #32]
-; CHECK-MVEFP-NEXT:    bl fmodf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q6[3], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q5[4]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[4]
-; CHECK-MVEFP-NEXT:    vmov s2, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVEFP-NEXT:    vstr s2, [sp, #24]
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #28]
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp, #24]
-; CHECK-MVEFP-NEXT:    bl fmodf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q6[4], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q5[5]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[5]
-; CHECK-MVEFP-NEXT:    vmov s2, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVEFP-NEXT:    vstr s2, [sp, #16]
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #20]
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp, #16]
-; CHECK-MVEFP-NEXT:    bl fmodf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q6[5], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q5[6]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[6]
-; CHECK-MVEFP-NEXT:    vmov s2, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVEFP-NEXT:    vstr s2, [sp, #8]
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #12]
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp, #8]
-; CHECK-MVEFP-NEXT:    bl fmodf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q6[6], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q5[7]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[7]
-; CHECK-MVEFP-NEXT:    vmov s2, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVEFP-NEXT:    vstr s2, [sp]
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #4]
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp]
-; CHECK-MVEFP-NEXT:    bl fmodf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q6[7], r0
-; CHECK-MVEFP-NEXT:    vmov q0, q6
-; CHECK-MVEFP-NEXT:    add sp, #64
-; CHECK-MVEFP-NEXT:    vpop {d8, d9, d10, d11, d12, d13}
-; CHECK-MVEFP-NEXT:    pop {r4, pc}
+; CHECK-LABEL: frem_f16:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13}
+; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13}
+; CHECK-NEXT:    .pad #64
+; CHECK-NEXT:    sub sp, #64
+; CHECK-NEXT:    vmov q5, q1
+; CHECK-NEXT:    vmov q4, q0
+; CHECK-NEXT:    vmov.u16 r0, q1[0]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[0]
+; CHECK-NEXT:    vmov s2, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s2, s2
+; CHECK-NEXT:    vstr s2, [sp, #56]
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #60]
+; CHECK-NEXT:    ldrd r0, r1, [sp, #56]
+; CHECK-NEXT:    bl fmodf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q5[1]
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[1]
+; CHECK-NEXT:    vmov s2, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s2, s2
+; CHECK-NEXT:    vstr s2, [sp, #48]
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #52]
+; CHECK-NEXT:    ldrd r0, r1, [sp, #48]
+; CHECK-NEXT:    bl fmodf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.16 q6[0], r4
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q6[1], r0
+; CHECK-NEXT:    vmov.u16 r0, q5[2]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[2]
+; CHECK-NEXT:    vmov s2, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s2, s2
+; CHECK-NEXT:    vstr s2, [sp, #40]
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #44]
+; CHECK-NEXT:    ldrd r0, r1, [sp, #40]
+; CHECK-NEXT:    bl fmodf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q6[2], r0
+; CHECK-NEXT:    vmov.u16 r0, q5[3]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[3]
+; CHECK-NEXT:    vmov s2, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s2, s2
+; CHECK-NEXT:    vstr s2, [sp, #32]
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #36]
+; CHECK-NEXT:    ldrd r0, r1, [sp, #32]
+; CHECK-NEXT:    bl fmodf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q6[3], r0
+; CHECK-NEXT:    vmov.u16 r0, q5[4]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[4]
+; CHECK-NEXT:    vmov s2, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s2, s2
+; CHECK-NEXT:    vstr s2, [sp, #24]
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #28]
+; CHECK-NEXT:    ldrd r0, r1, [sp, #24]
+; CHECK-NEXT:    bl fmodf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q6[4], r0
+; CHECK-NEXT:    vmov.u16 r0, q5[5]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[5]
+; CHECK-NEXT:    vmov s2, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s2, s2
+; CHECK-NEXT:    vstr s2, [sp, #16]
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #20]
+; CHECK-NEXT:    ldrd r0, r1, [sp, #16]
+; CHECK-NEXT:    bl fmodf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q6[5], r0
+; CHECK-NEXT:    vmov.u16 r0, q5[6]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[6]
+; CHECK-NEXT:    vmov s2, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s2, s2
+; CHECK-NEXT:    vstr s2, [sp, #8]
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #12]
+; CHECK-NEXT:    ldrd r0, r1, [sp, #8]
+; CHECK-NEXT:    bl fmodf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q6[6], r0
+; CHECK-NEXT:    vmov.u16 r0, q5[7]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[7]
+; CHECK-NEXT:    vmov s2, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s2, s2
+; CHECK-NEXT:    vstr s2, [sp]
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #4]
+; CHECK-NEXT:    ldrd r0, r1, [sp]
+; CHECK-NEXT:    bl fmodf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q6[7], r0
+; CHECK-NEXT:    vmov q0, q6
+; CHECK-NEXT:    add sp, #64
+; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13}
+; CHECK-NEXT:    pop {r4, pc}
 entry:
   %out = frem <8 x half> %in1, %in2
   ret <8 x half> %out
 }

llvm/test/CodeGen/Thumb2/mve-fmath.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-arm-none-eabi -mattr=+mve,+fullfp16 -verify-machineinstrs %s -o - | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-MVE
 ; RUN: llc -mtriple=thumbv8.1m.main-arm-none-eabi -mattr=+mve.fp -verify-machineinstrs %s -o - | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-MVEFP
 
 define arm_aapcs_vfpcc <4 x float> @sqrt_float32_t(<4 x float> %src) {
-; CHECK-MVE-LABEL: sqrt_float32_t:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    vsqrt.f32 s4, s0
-; CHECK-MVE-NEXT:    movs r0, #0
-; CHECK-MVE-NEXT:    vsqrt.f32 s6, s1
-; CHECK-MVE-NEXT:    vsqrt.f32 s8, s2
-; CHECK-MVE-NEXT:    vsqrt.f32 s10, s3
-; CHECK-MVE-NEXT:    vdup.32 q0, r0
-; CHECK-MVE-NEXT:    vmov.f32 s0, s4
-; CHECK-MVE-NEXT:    vmov.f32 s1, s6
-; CHECK-MVE-NEXT:    vmov.f32 s2, s8
-; CHECK-MVE-NEXT:    vmov.f32 s3, s10
-; CHECK-MVE-NEXT:    bx lr
-;
-; CHECK-MVEFP-LABEL: sqrt_float32_t:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    vsqrt.f32 s4, s0
-; CHECK-MVEFP-NEXT:    vsqrt.f32 s5, s1
-; CHECK-MVEFP-NEXT:    vsqrt.f32 s6, s2
-; CHECK-MVEFP-NEXT:    vsqrt.f32 s7, s3
-; CHECK-MVEFP-NEXT:    vmov q0, q1
-; CHECK-MVEFP-NEXT:    bx lr
+; CHECK-LABEL: sqrt_float32_t:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    vsqrt.f32 s7, s3
+; CHECK-NEXT:    vsqrt.f32 s6, s2
+; CHECK-NEXT:    vsqrt.f32 s5, s1
+; CHECK-NEXT:    vsqrt.f32 s4, s0
+; CHECK-NEXT:    vmov q0, q1
+; CHECK-NEXT:    bx lr
 entry:
   %0 = call fast <4 x float> @llvm.sqrt.v4f32(<4 x float> %src)
   ret <4 x float> %0
 }
 
 define arm_aapcs_vfpcc <8 x half> @sqrt_float16_t(<8 x half> %src) {
-; CHECK-MVE-LABEL: sqrt_float16_t:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-MVE-NEXT:    vmov.u16 r1, q0[1]
-; CHECK-MVE-NEXT:    vmov s4, r0
-; CHECK-MVE-NEXT:    movs r2, #0
-; CHECK-MVE-NEXT:    vsqrt.f16 s4, s4
-; CHECK-MVE-NEXT:    vmov r0, s4
-; CHECK-MVE-NEXT:    vmov s4, r1
-; CHECK-MVE-NEXT:    vsqrt.f16 s4, s4
-; CHECK-MVE-NEXT:    vmov r1, s4
-; CHECK-MVE-NEXT:    vdup.16 q1, r2
-; CHECK-MVE-NEXT:    vmov.16 q1[0], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-MVE-NEXT:    vmov s8, r0
-; CHECK-MVE-NEXT:    vmov.16 q1[1], r1
-; CHECK-MVE-NEXT:    vsqrt.f16 s8, s8
-; CHECK-MVE-NEXT:    vmov r0, s8
-; CHECK-MVE-NEXT:    vmov.16 q1[2], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[3]
-; CHECK-MVE-NEXT:    vmov s8, r0
-; CHECK-MVE-NEXT:    vsqrt.f16 s8, s8
-; CHECK-MVE-NEXT:    vmov r0, s8
-; CHECK-MVE-NEXT:    vmov.16 q1[3], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[4]
-; CHECK-MVE-NEXT:    vmov s8, r0
-; CHECK-MVE-NEXT:    vsqrt.f16 s8, s8
-; CHECK-MVE-NEXT:    vmov r0, s8
-; CHECK-MVE-NEXT:    vmov.16 q1[4], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[5]
-; CHECK-MVE-NEXT:    vmov s8, r0
-; CHECK-MVE-NEXT:    vsqrt.f16 s8, s8
-; CHECK-MVE-NEXT:    vmov r0, s8
-; CHECK-MVE-NEXT:    vmov.16 q1[5], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[6]
-; CHECK-MVE-NEXT:    vmov s8, r0
-; CHECK-MVE-NEXT:    vsqrt.f16 s8, s8
-; CHECK-MVE-NEXT:    vmov r0, s8
-; CHECK-MVE-NEXT:    vmov.16 q1[6], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[7]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vsqrt.f16 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q1[7], r0
-; CHECK-MVE-NEXT:    vmov q0, q1
-; CHECK-MVE-NEXT:    bx lr
-;
-; CHECK-MVEFP-LABEL: sqrt_float16_t:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-MVEFP-NEXT:    vmov.u16 r1, q0[1]
-; CHECK-MVEFP-NEXT:    vmov s4, r0
-; CHECK-MVEFP-NEXT:    vsqrt.f16 s4, s4
-; CHECK-MVEFP-NEXT:    vmov r0, s4
-; CHECK-MVEFP-NEXT:    vmov s4, r1
-; CHECK-MVEFP-NEXT:    vsqrt.f16 s4, s4
-; CHECK-MVEFP-NEXT:    vmov r1, s4
-; CHECK-MVEFP-NEXT:    vmov.16 q1[0], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-MVEFP-NEXT:    vmov.16 q1[1], r1
-; CHECK-MVEFP-NEXT:    vmov s8, r0
-; CHECK-MVEFP-NEXT:    vsqrt.f16 s8, s8
-; CHECK-MVEFP-NEXT:    vmov r0, s8
-; CHECK-MVEFP-NEXT:    vmov.16 q1[2], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[3]
-; CHECK-MVEFP-NEXT:    vmov s8, r0
-; CHECK-MVEFP-NEXT:    vsqrt.f16 s8, s8
-; CHECK-MVEFP-NEXT:    vmov r0, s8
-; CHECK-MVEFP-NEXT:    vmov.16 q1[3], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[4]
-; CHECK-MVEFP-NEXT:    vmov s8, r0
-; CHECK-MVEFP-NEXT:    vsqrt.f16 s8, s8
-; CHECK-MVEFP-NEXT:    vmov r0, s8
-; CHECK-MVEFP-NEXT:    vmov.16 q1[4], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[5]
-; CHECK-MVEFP-NEXT:    vmov s8, r0
-; CHECK-MVEFP-NEXT:    vsqrt.f16 s8, s8
-; CHECK-MVEFP-NEXT:    vmov r0, s8
-; CHECK-MVEFP-NEXT:    vmov.16 q1[5], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[6]
-; CHECK-MVEFP-NEXT:    vmov s8, r0
-; CHECK-MVEFP-NEXT:    vsqrt.f16 s8, s8
-; CHECK-MVEFP-NEXT:    vmov r0, s8
-; CHECK-MVEFP-NEXT:    vmov.16 q1[6], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[7]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vsqrt.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q1[7], r0
-; CHECK-MVEFP-NEXT:    vmov q0, q1
-; CHECK-MVEFP-NEXT:    bx lr
+; CHECK-LABEL: sqrt_float16_t:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    vmov.u16 r0, q0[0]
+; CHECK-NEXT:    vmov.u16 r1, q0[1]
+; CHECK-NEXT:    vmov s4, r0
+; CHECK-NEXT:    vsqrt.f16 s4, s4
+; CHECK-NEXT:    vmov r0, s4
+; CHECK-NEXT:    vmov s4, r1
+; CHECK-NEXT:    vsqrt.f16 s4, s4
+; CHECK-NEXT:    vmov r1, s4
+; CHECK-NEXT:    vmov.16 q1[0], r0
+; CHECK-NEXT:    vmov.u16 r0, q0[2]
+; CHECK-NEXT:    vmov.16 q1[1], r1
+; CHECK-NEXT:    vmov s8, r0
+; CHECK-NEXT:    vsqrt.f16 s8, s8
+; CHECK-NEXT:    vmov r0, s8
+; CHECK-NEXT:    vmov.16 q1[2], r0
+; CHECK-NEXT:    vmov.u16 r0, q0[3]
+; CHECK-NEXT:    vmov s8, r0
+; CHECK-NEXT:    vsqrt.f16 s8, s8
+; CHECK-NEXT:    vmov r0, s8
+; CHECK-NEXT:    vmov.16 q1[3], r0
+; CHECK-NEXT:    vmov.u16 r0, q0[4]
+; CHECK-NEXT:    vmov s8, r0
+; CHECK-NEXT:    vsqrt.f16 s8, s8
+; CHECK-NEXT:    vmov r0, s8
+; CHECK-NEXT:    vmov.16 q1[4], r0
+; CHECK-NEXT:    vmov.u16 r0, q0[5]
+; CHECK-NEXT:    vmov s8, r0
+; CHECK-NEXT:    vsqrt.f16 s8, s8
+; CHECK-NEXT:    vmov r0, s8
+; CHECK-NEXT:    vmov.16 q1[5], r0
+; CHECK-NEXT:    vmov.u16 r0, q0[6]
+; CHECK-NEXT:    vmov s8, r0
+; CHECK-NEXT:    vsqrt.f16 s8, s8
+; CHECK-NEXT:    vmov r0, s8
+; CHECK-NEXT:    vmov.16 q1[6], r0
+; CHECK-NEXT:    vmov.u16 r0, q0[7]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vsqrt.f16 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q1[7], r0
+; CHECK-NEXT:    vmov q0, q1
+; CHECK-NEXT:    bx lr
 entry:
   %0 = call fast <8 x half> @llvm.sqrt.v8f16(<8 x half> %src)
   ret <8 x half> %0
 }
 
 define arm_aapcs_vfpcc <4 x float> @cos_float32_t(<4 x float> %src) {
-; CHECK-MVE-LABEL: cos_float32_t:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-MVE-NEXT:    push {r4, r5, r7, lr}
-; CHECK-MVE-NEXT:    .vsave {d8, d9, d10}
-; CHECK-MVE-NEXT:    vpush {d8, d9, d10}
-; CHECK-MVE-NEXT:    .pad #16
-; CHECK-MVE-NEXT:    sub sp, #16
-; CHECK-MVE-NEXT:    vstmia sp, {s0, s1, s2, s3}
-; CHECK-MVE-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVE-NEXT:    bl cosf
-; CHECK-MVE-NEXT:    mov r4, r0
-; CHECK-MVE-NEXT:    ldr r0, [sp]
-; CHECK-MVE-NEXT:    bl cosf
-; CHECK-MVE-NEXT:    ldrd r1, r5, [sp, #8]
-; CHECK-MVE-NEXT:    vmov s16, r0
-; CHECK-MVE-NEXT:    vmov s18, r4
-; CHECK-MVE-NEXT:    mov r0, r1
-; CHECK-MVE-NEXT:    bl cosf
-; CHECK-MVE-NEXT:    vmov s20, r0
-; CHECK-MVE-NEXT:    mov r0, r5
-; CHECK-MVE-NEXT:    bl cosf
-; CHECK-MVE-NEXT:    movs r1, #0
-; CHECK-MVE-NEXT:    vmov s4, r0
-; CHECK-MVE-NEXT:    vdup.32 q0, r1
-; CHECK-MVE-NEXT:    vmov.f32 s0, s16
-; CHECK-MVE-NEXT:    vmov.f32 s1, s18
-; CHECK-MVE-NEXT:    vmov.f32 s2, s20
-; CHECK-MVE-NEXT:    vmov.f32 s3, s4
-; CHECK-MVE-NEXT:    add sp, #16
-; CHECK-MVE-NEXT:    vpop {d8, d9, d10}
-; CHECK-MVE-NEXT:    pop {r4, r5, r7, pc}
-;
-; CHECK-MVEFP-LABEL: cos_float32_t:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-MVEFP-NEXT:    push {r4, r5, r7, lr}
-; CHECK-MVEFP-NEXT:    .vsave {d8, d9}
-; CHECK-MVEFP-NEXT:    vpush {d8, d9}
-; CHECK-MVEFP-NEXT:    .pad #16
-; CHECK-MVEFP-NEXT:    sub sp, #16
-; CHECK-MVEFP-NEXT:    vstmia sp, {s0, s1, s2, s3}
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVEFP-NEXT:    bl cosf
-; CHECK-MVEFP-NEXT:    mov r4, r0
-; CHECK-MVEFP-NEXT:    ldr r0, [sp]
-; CHECK-MVEFP-NEXT:    bl cosf
-; CHECK-MVEFP-NEXT:    ldrd r1, r5, [sp, #8]
-; CHECK-MVEFP-NEXT:    vmov s16, r0
-; CHECK-MVEFP-NEXT:    vmov s17, r4
-; CHECK-MVEFP-NEXT:    mov r0, r1
-; CHECK-MVEFP-NEXT:    bl cosf
-; CHECK-MVEFP-NEXT:    vmov s18, r0
-; CHECK-MVEFP-NEXT:    mov r0, r5
-; CHECK-MVEFP-NEXT:    bl cosf
-; CHECK-MVEFP-NEXT:    vmov s19, r0
-; CHECK-MVEFP-NEXT:    vmov q0, q4
-; CHECK-MVEFP-NEXT:    add sp, #16
-; CHECK-MVEFP-NEXT:    vpop {d8, d9}
-; CHECK-MVEFP-NEXT:    pop {r4, r5, r7, pc}
+; CHECK-LABEL: cos_float32_t:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
+; CHECK-NEXT:    .vsave {d8, d9}
+; CHECK-NEXT:    vpush {d8, d9}
+; CHECK-NEXT:    .pad #16
+; CHECK-NEXT:    sub sp, #16
+; CHECK-NEXT:    vstmia sp, {s0, s1, s2, s3}
+; CHECK-NEXT:    ldr r0, [sp, #8]
+; CHECK-NEXT:    bl cosf
+; CHECK-NEXT:    mov r4, r0
+; CHECK-NEXT:    ldr r0, [sp, #12]
+; CHECK-NEXT:    bl cosf
+; CHECK-NEXT:    ldrd r5, r1, [sp]
+; CHECK-NEXT:    vmov s19, r0
+; CHECK-NEXT:    mov r0, r1
+; CHECK-NEXT:    vmov s18, r4
+; CHECK-NEXT:    bl cosf
+; CHECK-NEXT:    vmov s17, r0
+; CHECK-NEXT:    mov r0, r5
+; CHECK-NEXT:    bl cosf
+; CHECK-NEXT:    vmov s16, r0
+; CHECK-NEXT:    vmov q0, q4
+; CHECK-NEXT:    add sp, #16
+; CHECK-NEXT:    vpop {d8, d9}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %0 = call fast <4 x float> @llvm.cos.v4f32(<4 x float> %src)
   ret <4 x float> %0
 }
 
 define arm_aapcs_vfpcc <8 x half> @cos_float16_t(<8 x half> %src) {
-; CHECK-MVE-LABEL: cos_float16_t:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    .save {r4, lr}
-; CHECK-MVE-NEXT:    push {r4, lr}
-; CHECK-MVE-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    vpush {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    .pad #32
-; CHECK-MVE-NEXT:    sub sp, #32
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-MVE-NEXT:    vmov q4, q0
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #24]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #24]
-; CHECK-MVE-NEXT:    bl cosf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r4, s0
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #28]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #28]
-; CHECK-MVE-NEXT:    bl cosf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    movs r1, #0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vdup.16 q5, r1
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[0], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.16 q5[1], r4
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #20]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #20]
-; CHECK-MVE-NEXT:    bl cosf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[2], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #16]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #16]
-; CHECK-MVE-NEXT:    bl cosf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[3], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[4]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #12]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #12]
-; CHECK-MVE-NEXT:    bl cosf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[4], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[5]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #8]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #8]
-; CHECK-MVE-NEXT:    bl cosf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[5], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[6]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #4]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVE-NEXT:    bl cosf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[6], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[7]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp]
-; CHECK-MVE-NEXT:    ldr r0, [sp]
-; CHECK-MVE-NEXT:    bl cosf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[7], r0
-; CHECK-MVE-NEXT:    vmov q0, q5
-; CHECK-MVE-NEXT:    add sp, #32
-; CHECK-MVE-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    pop {r4, pc}
-;
-; CHECK-MVEFP-LABEL: cos_float16_t:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    .save {r4, lr}
-; CHECK-MVEFP-NEXT:    push {r4, lr}
-; CHECK-MVEFP-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    vpush {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    .pad #32
-; CHECK-MVEFP-NEXT:    sub sp, #32
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-MVEFP-NEXT:    vmov q4, q0
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #24]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #24]
-; CHECK-MVEFP-NEXT:    bl cosf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r4, s0
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #28]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #28]
-; CHECK-MVEFP-NEXT:    bl cosf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[0], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[1], r4
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #20]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #20]
-; CHECK-MVEFP-NEXT:    bl cosf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[2], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #16]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #16]
-; CHECK-MVEFP-NEXT:    bl cosf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[3], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[4]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #12]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #12]
-; CHECK-MVEFP-NEXT:    bl cosf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[4], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[5]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #8]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #8]
-; CHECK-MVEFP-NEXT:    bl cosf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[5], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[6]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #4]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVEFP-NEXT:    bl cosf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[6], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[7]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp]
-; CHECK-MVEFP-NEXT:    bl cosf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[7], r0
-; CHECK-MVEFP-NEXT:    vmov q0, q5
-; CHECK-MVEFP-NEXT:    add sp, #32
-; CHECK-MVEFP-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    pop {r4, pc}
+; CHECK-LABEL: cos_float16_t:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
+; CHECK-NEXT:    vpush {d8, d9, d10, d11}
+; CHECK-NEXT:    .pad #32
+; CHECK-NEXT:    sub sp, #32
+; CHECK-NEXT:    vmov.u16 r0, q0[1]
+; CHECK-NEXT:    vmov q4, q0
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #24]
+; CHECK-NEXT:    ldr r0, [sp, #24]
+; CHECK-NEXT:    bl cosf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[0]
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #28]
+; CHECK-NEXT:    ldr r0, [sp, #28]
+; CHECK-NEXT:    bl cosf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[0], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[2]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.16 q5[1], r4
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #20]
+; CHECK-NEXT:    ldr r0, [sp, #20]
+; CHECK-NEXT:    bl cosf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[2], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[3]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #16]
+; CHECK-NEXT:    ldr r0, [sp, #16]
+; CHECK-NEXT:    bl cosf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[3], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[4]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #12]
+; CHECK-NEXT:    ldr r0, [sp, #12]
+; CHECK-NEXT:    bl cosf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[4], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[5]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #8]
+; CHECK-NEXT:    ldr r0, [sp, #8]
+; CHECK-NEXT:    bl cosf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[5], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[6]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #4]
+; CHECK-NEXT:    ldr r0, [sp, #4]
+; CHECK-NEXT:    bl cosf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[6], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[7]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp]
+; CHECK-NEXT:    ldr r0, [sp]
+; CHECK-NEXT:    bl cosf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[7], r0
+; CHECK-NEXT:    vmov q0, q5
+; CHECK-NEXT:    add sp, #32
+; CHECK-NEXT:    vpop {d8, d9, d10, d11}
+; CHECK-NEXT:    pop {r4, pc}
 entry:
   %0 = call fast <8 x half> @llvm.cos.v8f16(<8 x half> %src)
   ret <8 x half> %0
 }
 
 define arm_aapcs_vfpcc <4 x float> @sin_float32_t(<4 x float> %src) {
-; CHECK-MVE-LABEL: sin_float32_t:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-MVE-NEXT:    push {r4, r5, r7, lr}
-; CHECK-MVE-NEXT:    .vsave {d8, d9, d10}
-; CHECK-MVE-NEXT:    vpush {d8, d9, d10}
-; CHECK-MVE-NEXT:    .pad #16
-; CHECK-MVE-NEXT:    sub sp, #16
-; CHECK-MVE-NEXT:    vstmia sp, {s0, s1, s2, s3}
-; CHECK-MVE-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVE-NEXT:    bl sinf
-; CHECK-MVE-NEXT:    mov r4, r0
-; CHECK-MVE-NEXT:    ldr r0, [sp]
-; CHECK-MVE-NEXT:    bl sinf
-; CHECK-MVE-NEXT:    ldrd r1, r5, [sp, #8]
-; CHECK-MVE-NEXT:    vmov s16, r0
-; CHECK-MVE-NEXT:    vmov s18, r4
-; CHECK-MVE-NEXT:    mov r0, r1
-; CHECK-MVE-NEXT:    bl sinf
-; CHECK-MVE-NEXT:    vmov s20, r0
-; CHECK-MVE-NEXT:    mov r0, r5
-; CHECK-MVE-NEXT:    bl sinf
-; CHECK-MVE-NEXT:    movs r1, #0
-; CHECK-MVE-NEXT:    vmov s4, r0
-; CHECK-MVE-NEXT:    vdup.32 q0, r1
-; CHECK-MVE-NEXT:    vmov.f32 s0, s16
-; CHECK-MVE-NEXT:    vmov.f32 s1, s18
-; CHECK-MVE-NEXT:    vmov.f32 s2, s20
-; CHECK-MVE-NEXT:    vmov.f32 s3, s4
-; CHECK-MVE-NEXT:    add sp, #16
-; CHECK-MVE-NEXT:    vpop {d8, d9, d10}
-; CHECK-MVE-NEXT:    pop {r4, r5, r7, pc}
-;
-; CHECK-MVEFP-LABEL: sin_float32_t:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-MVEFP-NEXT:    push {r4, r5, r7, lr}
-; CHECK-MVEFP-NEXT:    .vsave {d8, d9}
-; CHECK-MVEFP-NEXT:    vpush {d8, d9}
-; CHECK-MVEFP-NEXT:    .pad #16
-; CHECK-MVEFP-NEXT:    sub sp, #16
-; CHECK-MVEFP-NEXT:    vstmia sp, {s0, s1, s2, s3}
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVEFP-NEXT:    bl sinf
-; CHECK-MVEFP-NEXT:    mov r4, r0
-; CHECK-MVEFP-NEXT:    ldr r0, [sp]
-; CHECK-MVEFP-NEXT:    bl sinf
-; CHECK-MVEFP-NEXT:    ldrd r1, r5, [sp, #8]
-; CHECK-MVEFP-NEXT:    vmov s16, r0
-; CHECK-MVEFP-NEXT:    vmov s17, r4
-; CHECK-MVEFP-NEXT:    mov r0, r1
-; CHECK-MVEFP-NEXT:    bl sinf
-; CHECK-MVEFP-NEXT:    vmov s18, r0
-; CHECK-MVEFP-NEXT:    mov r0, r5
-; CHECK-MVEFP-NEXT:    bl sinf
-; CHECK-MVEFP-NEXT:    vmov s19, r0
-; CHECK-MVEFP-NEXT:    vmov q0, q4
-; CHECK-MVEFP-NEXT:    add sp, #16
-; CHECK-MVEFP-NEXT:    vpop {d8, d9}
-; CHECK-MVEFP-NEXT:    pop {r4, r5, r7, pc}
+; CHECK-LABEL: sin_float32_t:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
+; CHECK-NEXT:    .vsave {d8, d9}
+; CHECK-NEXT:    vpush {d8, d9}
+; CHECK-NEXT:    .pad #16
+; CHECK-NEXT:    sub sp, #16
+; CHECK-NEXT:    vstmia sp, {s0, s1, s2, s3}
+; CHECK-NEXT:    ldr r0, [sp, #8]
+; CHECK-NEXT:    bl sinf
+; CHECK-NEXT:    mov r4, r0
+; CHECK-NEXT:    ldr r0, [sp, #12]
+; CHECK-NEXT:    bl sinf
+; CHECK-NEXT:    ldrd r5, r1, [sp]
+; CHECK-NEXT:    vmov s19, r0
+; CHECK-NEXT:    mov r0, r1
+; CHECK-NEXT:    vmov s18, r4
+; CHECK-NEXT:    bl sinf
+; CHECK-NEXT:    vmov s17, r0
+; CHECK-NEXT:    mov r0, r5
+; CHECK-NEXT:    bl sinf
+; CHECK-NEXT:    vmov s16, r0
+; CHECK-NEXT:    vmov q0, q4
+; CHECK-NEXT:    add sp, #16
+; CHECK-NEXT:    vpop {d8, d9}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %0 = call fast <4 x float> @llvm.sin.v4f32(<4 x float> %src)
   ret <4 x float> %0
 }
 
 define arm_aapcs_vfpcc <8 x half> @sin_float16_t(<8 x half> %src) {
-; CHECK-MVE-LABEL: sin_float16_t:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    .save {r4, lr}
-; CHECK-MVE-NEXT:    push {r4, lr}
-; CHECK-MVE-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    vpush {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    .pad #32
-; CHECK-MVE-NEXT:    sub sp, #32
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-MVE-NEXT:    vmov q4, q0
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #24]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #24]
-; CHECK-MVE-NEXT:    bl sinf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r4, s0
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #28]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #28]
-; CHECK-MVE-NEXT:    bl sinf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    movs r1, #0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vdup.16 q5, r1
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[0], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.16 q5[1], r4
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #20]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #20]
-; CHECK-MVE-NEXT:    bl sinf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[2], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #16]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #16]
-; CHECK-MVE-NEXT:    bl sinf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[3], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[4]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #12]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #12]
-; CHECK-MVE-NEXT:    bl sinf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[4], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[5]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #8]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #8]
-; CHECK-MVE-NEXT:    bl sinf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[5], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[6]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #4]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVE-NEXT:    bl sinf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[6], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[7]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp]
-; CHECK-MVE-NEXT:    ldr r0, [sp]
-; CHECK-MVE-NEXT:    bl sinf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[7], r0
-; CHECK-MVE-NEXT:    vmov q0, q5
-; CHECK-MVE-NEXT:    add sp, #32
-; CHECK-MVE-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    pop {r4, pc}
-;
-; CHECK-MVEFP-LABEL: sin_float16_t:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    .save {r4, lr}
-; CHECK-MVEFP-NEXT:    push {r4, lr}
-; CHECK-MVEFP-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    vpush {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    .pad #32
-; CHECK-MVEFP-NEXT:    sub sp, #32
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-MVEFP-NEXT:    vmov q4, q0
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #24]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #24]
-; CHECK-MVEFP-NEXT:    bl sinf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r4, s0
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #28]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #28]
-; CHECK-MVEFP-NEXT:    bl sinf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[0], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[1], r4
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #20]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #20]
-; CHECK-MVEFP-NEXT:    bl sinf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[2], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #16]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #16]
-; CHECK-MVEFP-NEXT:    bl sinf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[3], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[4]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #12]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #12]
-; CHECK-MVEFP-NEXT:    bl sinf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[4], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[5]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #8]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #8]
-; CHECK-MVEFP-NEXT:    bl sinf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[5], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[6]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #4]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVEFP-NEXT:    bl sinf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[6], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[7]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp]
-; CHECK-MVEFP-NEXT:    bl sinf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[7], r0
-; CHECK-MVEFP-NEXT:    vmov q0, q5
-; CHECK-MVEFP-NEXT:    add sp, #32
-; CHECK-MVEFP-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    pop {r4, pc}
+; CHECK-LABEL: sin_float16_t:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
+; CHECK-NEXT:    vpush {d8, d9, d10, d11}
+; CHECK-NEXT:    .pad #32
+; CHECK-NEXT:    sub sp, #32
+; CHECK-NEXT:    vmov.u16 r0, q0[1]
+; CHECK-NEXT:    vmov q4, q0
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #24]
+; CHECK-NEXT:    ldr r0, [sp, #24]
+; CHECK-NEXT:    bl sinf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[0]
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #28]
+; CHECK-NEXT:    ldr r0, [sp, #28]
+; CHECK-NEXT:    bl sinf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[0], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[2]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.16 q5[1], r4
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #20]
+; CHECK-NEXT:    ldr r0, [sp, #20]
+; CHECK-NEXT:    bl sinf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[2], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[3]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #16]
+; CHECK-NEXT:    ldr r0, [sp, #16]
+; CHECK-NEXT:    bl sinf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[3], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[4]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #12]
+; CHECK-NEXT:    ldr r0, [sp, #12]
+; CHECK-NEXT:    bl sinf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[4], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[5]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #8]
+; CHECK-NEXT:    ldr r0, [sp, #8]
+; CHECK-NEXT:    bl sinf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[5], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[6]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #4]
+; CHECK-NEXT:    ldr r0, [sp, #4]
+; CHECK-NEXT:    bl sinf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[6], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[7]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp]
+; CHECK-NEXT:    ldr r0, [sp]
+; CHECK-NEXT:    bl sinf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[7], r0
+; CHECK-NEXT:    vmov q0, q5
+; CHECK-NEXT:    add sp, #32
+; CHECK-NEXT:    vpop {d8, d9, d10, d11}
+; CHECK-NEXT:    pop {r4, pc}
 entry:
   %0 = call fast <8 x half> @llvm.sin.v8f16(<8 x half> %src)
   ret <8 x half> %0
 }
 
 define arm_aapcs_vfpcc <4 x float> @exp_float32_t(<4 x float> %src) {
-; CHECK-MVE-LABEL: exp_float32_t:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-MVE-NEXT:    push {r4, r5, r7, lr}
-; CHECK-MVE-NEXT:    .vsave {d8, d9, d10}
-; CHECK-MVE-NEXT:    vpush {d8, d9, d10}
-; CHECK-MVE-NEXT:    .pad #16
-; CHECK-MVE-NEXT:    sub sp, #16
-; CHECK-MVE-NEXT:    vstmia sp, {s0, s1, s2, s3}
-; CHECK-MVE-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVE-NEXT:    bl expf
-; CHECK-MVE-NEXT:    mov r4, r0
-; CHECK-MVE-NEXT:    ldr r0, [sp]
-; CHECK-MVE-NEXT:    bl expf
-; CHECK-MVE-NEXT:    ldrd r1, r5, [sp, #8]
-; CHECK-MVE-NEXT:    vmov s16, r0
-; CHECK-MVE-NEXT:    vmov s18, r4
-; CHECK-MVE-NEXT:    mov r0, r1
-; CHECK-MVE-NEXT:    bl expf
-; CHECK-MVE-NEXT:    vmov s20, r0
-; CHECK-MVE-NEXT:    mov r0, r5
-; CHECK-MVE-NEXT:    bl expf
-; CHECK-MVE-NEXT:    movs r1, #0
-; CHECK-MVE-NEXT:    vmov s4, r0
-; CHECK-MVE-NEXT:    vdup.32 q0, r1
-; CHECK-MVE-NEXT:    vmov.f32 s0, s16
-; CHECK-MVE-NEXT:    vmov.f32 s1, s18
-; CHECK-MVE-NEXT:    vmov.f32 s2, s20
-; CHECK-MVE-NEXT:    vmov.f32 s3, s4
-; CHECK-MVE-NEXT:    add sp, #16
-; CHECK-MVE-NEXT:    vpop {d8, d9, d10}
-; CHECK-MVE-NEXT:    pop {r4, r5, r7, pc}
-;
-; CHECK-MVEFP-LABEL: exp_float32_t:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-MVEFP-NEXT:    push {r4, r5, r7, lr}
-; CHECK-MVEFP-NEXT:    .vsave {d8, d9}
-; CHECK-MVEFP-NEXT:    vpush {d8, d9}
-; CHECK-MVEFP-NEXT:    .pad #16
-; CHECK-MVEFP-NEXT:    sub sp, #16
-; CHECK-MVEFP-NEXT:    vstmia sp, {s0, s1, s2, s3}
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVEFP-NEXT:    bl expf
-; CHECK-MVEFP-NEXT:    mov r4, r0
-; CHECK-MVEFP-NEXT:    ldr r0, [sp]
-; CHECK-MVEFP-NEXT:    bl expf
-; CHECK-MVEFP-NEXT:    ldrd r1, r5, [sp, #8]
-; CHECK-MVEFP-NEXT:    vmov s16, r0
-; CHECK-MVEFP-NEXT:    vmov s17, r4
-; CHECK-MVEFP-NEXT:    mov r0, r1
-; CHECK-MVEFP-NEXT:    bl expf
-; CHECK-MVEFP-NEXT:    vmov s18, r0
-; CHECK-MVEFP-NEXT:    mov r0, r5
-; CHECK-MVEFP-NEXT:    bl expf
-; CHECK-MVEFP-NEXT:    vmov s19, r0
-; CHECK-MVEFP-NEXT:    vmov q0, q4
-; CHECK-MVEFP-NEXT:    add sp, #16
-; CHECK-MVEFP-NEXT:    vpop {d8, d9}
-; CHECK-MVEFP-NEXT:    pop {r4, r5, r7, pc}
+; CHECK-LABEL: exp_float32_t:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
+; CHECK-NEXT:    .vsave {d8, d9}
+; CHECK-NEXT:    vpush {d8, d9}
+; CHECK-NEXT:    .pad #16
+; CHECK-NEXT:    sub sp, #16
+; CHECK-NEXT:    vstmia sp, {s0, s1, s2, s3}
+; CHECK-NEXT:    ldr r0, [sp, #8]
+; CHECK-NEXT:    bl expf
+; CHECK-NEXT:    mov r4, r0
+; CHECK-NEXT:    ldr r0, [sp, #12]
+; CHECK-NEXT:    bl expf
+; CHECK-NEXT:    ldrd r5, r1, [sp]
+; CHECK-NEXT:    vmov s19, r0
+; CHECK-NEXT:    mov r0, r1
+; CHECK-NEXT:    vmov s18, r4
+; CHECK-NEXT:    bl expf
+; CHECK-NEXT:    vmov s17, r0
+; CHECK-NEXT:    mov r0, r5
+; CHECK-NEXT:    bl expf
+; CHECK-NEXT:    vmov s16, r0
+; CHECK-NEXT:    vmov q0, q4
+; CHECK-NEXT:    add sp, #16
+; CHECK-NEXT:    vpop {d8, d9}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %0 = call fast <4 x float> @llvm.exp.v4f32(<4 x float> %src)
   ret <4 x float> %0
 }
 
 define arm_aapcs_vfpcc <8 x half> @exp_float16_t(<8 x half> %src) {
-; CHECK-MVE-LABEL: exp_float16_t:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    .save {r4, lr}
-; CHECK-MVE-NEXT:    push {r4, lr}
-; CHECK-MVE-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    vpush {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    .pad #32
-; CHECK-MVE-NEXT:    sub sp, #32
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-MVE-NEXT:    vmov q4, q0
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #24]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #24]
-; CHECK-MVE-NEXT:    bl expf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r4, s0
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #28]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #28]
-; CHECK-MVE-NEXT:    bl expf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    movs r1, #0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vdup.16 q5, r1
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[0], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.16 q5[1], r4
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #20]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #20]
-; CHECK-MVE-NEXT:    bl expf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[2], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #16]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #16]
-; CHECK-MVE-NEXT:    bl expf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[3], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[4]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #12]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #12]
-; CHECK-MVE-NEXT:    bl expf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[4], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[5]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #8]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #8]
-; CHECK-MVE-NEXT:    bl expf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[5], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[6]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #4]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVE-NEXT:    bl expf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[6], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[7]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp]
-; CHECK-MVE-NEXT:    ldr r0, [sp]
-; CHECK-MVE-NEXT:    bl expf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[7], r0
-; CHECK-MVE-NEXT:    vmov q0, q5
-; CHECK-MVE-NEXT:    add sp, #32
-; CHECK-MVE-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    pop {r4, pc}
-;
-; CHECK-MVEFP-LABEL: exp_float16_t:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    .save {r4, lr}
-; CHECK-MVEFP-NEXT:    push {r4, lr}
-; CHECK-MVEFP-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    vpush {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    .pad #32
-; CHECK-MVEFP-NEXT:    sub sp, #32
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-MVEFP-NEXT:    vmov q4, q0
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #24]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #24]
-; CHECK-MVEFP-NEXT:    bl expf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r4, s0
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #28]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #28]
-; CHECK-MVEFP-NEXT:    bl expf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[0], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[1], r4
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #20]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #20]
-; CHECK-MVEFP-NEXT:    bl expf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[2], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #16]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #16]
-; CHECK-MVEFP-NEXT:    bl expf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[3], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[4]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #12]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #12]
-; CHECK-MVEFP-NEXT:    bl expf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[4], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[5]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #8]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #8]
-; CHECK-MVEFP-NEXT:    bl expf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[5], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[6]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #4]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVEFP-NEXT:    bl expf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[6], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[7]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp]
-; CHECK-MVEFP-NEXT:    bl expf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[7], r0
-; CHECK-MVEFP-NEXT:    vmov q0, q5
-; CHECK-MVEFP-NEXT:    add sp, #32
-; CHECK-MVEFP-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    pop {r4, pc}
+; CHECK-LABEL: exp_float16_t:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
+; CHECK-NEXT:    vpush {d8, d9, d10, d11}
+; CHECK-NEXT:    .pad #32
+; CHECK-NEXT:    sub sp, #32
+; CHECK-NEXT:    vmov.u16 r0, q0[1]
+; CHECK-NEXT:    vmov q4, q0
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #24]
+; CHECK-NEXT:    ldr r0, [sp, #24]
+; CHECK-NEXT:    bl expf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[0]
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #28]
+; CHECK-NEXT:    ldr r0, [sp, #28]
+; CHECK-NEXT:    bl expf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[0], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[2]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.16 q5[1], r4
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #20]
+; CHECK-NEXT:    ldr r0, [sp, #20]
+; CHECK-NEXT:    bl expf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[2], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[3]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #16]
+; CHECK-NEXT:    ldr r0, [sp, #16]
+; CHECK-NEXT:    bl expf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[3], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[4]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #12]
+; CHECK-NEXT:    ldr r0, [sp, #12]
+; CHECK-NEXT:    bl expf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[4], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[5]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #8]
+; CHECK-NEXT:    ldr r0, [sp, #8]
+; CHECK-NEXT:    bl expf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[5], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[6]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #4]
+; CHECK-NEXT:    ldr r0, [sp, #4]
+; CHECK-NEXT:    bl expf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[6], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[7]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp]
+; CHECK-NEXT:    ldr r0, [sp]
+; CHECK-NEXT:    bl expf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[7], r0
+; CHECK-NEXT:    vmov q0, q5
+; CHECK-NEXT:    add sp, #32
+; CHECK-NEXT:    vpop {d8, d9, d10, d11}
+; CHECK-NEXT:    pop {r4, pc}
 entry:
   %0 = call fast <8 x half> @llvm.exp.v8f16(<8 x half> %src)
   ret <8 x half> %0
 }
 
 define arm_aapcs_vfpcc <4 x float> @exp2_float32_t(<4 x float> %src) {
-; CHECK-MVE-LABEL: exp2_float32_t:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-MVE-NEXT:    push {r4, r5, r7, lr}
-; CHECK-MVE-NEXT:    .vsave {d8, d9, d10}
-; CHECK-MVE-NEXT:    vpush {d8, d9, d10}
-; CHECK-MVE-NEXT:    .pad #16
-; CHECK-MVE-NEXT:    sub sp, #16
-; CHECK-MVE-NEXT:    vstmia sp, {s0, s1, s2, s3}
-; CHECK-MVE-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVE-NEXT:    bl exp2f
-; CHECK-MVE-NEXT:    mov r4, r0
-; CHECK-MVE-NEXT:    ldr r0, [sp]
-; CHECK-MVE-NEXT:    bl exp2f
-; CHECK-MVE-NEXT:    ldrd r1, r5, [sp, #8]
-; CHECK-MVE-NEXT:    vmov s16, r0
-; CHECK-MVE-NEXT:    vmov s18, r4
-; CHECK-MVE-NEXT:    mov r0, r1
-; CHECK-MVE-NEXT:    bl exp2f
-; CHECK-MVE-NEXT:    vmov s20, r0
-; CHECK-MVE-NEXT:    mov r0, r5
-; CHECK-MVE-NEXT:    bl exp2f
-; CHECK-MVE-NEXT:    movs r1, #0
-; CHECK-MVE-NEXT:    vmov s4, r0
-; CHECK-MVE-NEXT:    vdup.32 q0, r1
-; CHECK-MVE-NEXT:    vmov.f32 s0, s16
-; CHECK-MVE-NEXT:    vmov.f32 s1, s18
-; CHECK-MVE-NEXT:    vmov.f32 s2, s20
-; CHECK-MVE-NEXT:    vmov.f32 s3, s4
-; CHECK-MVE-NEXT:    add sp, #16
-; CHECK-MVE-NEXT:    vpop {d8, d9, d10}
-; CHECK-MVE-NEXT:    pop {r4, r5, r7, pc}
-;
-; CHECK-MVEFP-LABEL: exp2_float32_t:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-MVEFP-NEXT:    push {r4, r5, r7, lr}
-; CHECK-MVEFP-NEXT:    .vsave {d8, d9}
-; CHECK-MVEFP-NEXT:    vpush {d8, d9}
-; CHECK-MVEFP-NEXT:    .pad #16
-; CHECK-MVEFP-NEXT:    sub sp, #16
-; CHECK-MVEFP-NEXT:    vstmia sp, {s0, s1, s2, s3}
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVEFP-NEXT:    bl exp2f
-; CHECK-MVEFP-NEXT:    mov r4, r0
-; CHECK-MVEFP-NEXT:    ldr r0, [sp]
-; CHECK-MVEFP-NEXT:    bl exp2f
-; CHECK-MVEFP-NEXT:    ldrd r1, r5, [sp, #8]
-; CHECK-MVEFP-NEXT:    vmov s16, r0
-; CHECK-MVEFP-NEXT:    vmov s17, r4
-; CHECK-MVEFP-NEXT:    mov r0, r1
-; CHECK-MVEFP-NEXT:    bl exp2f
-; CHECK-MVEFP-NEXT:    vmov s18, r0
-; CHECK-MVEFP-NEXT:    mov r0, r5
-; CHECK-MVEFP-NEXT:    bl exp2f
-; CHECK-MVEFP-NEXT:    vmov s19, r0
-; CHECK-MVEFP-NEXT:    vmov q0, q4
-; CHECK-MVEFP-NEXT:    add sp, #16
-; CHECK-MVEFP-NEXT:    vpop {d8, d9}
-; CHECK-MVEFP-NEXT:    pop {r4, r5, r7, pc}
+; CHECK-LABEL: exp2_float32_t:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
+; CHECK-NEXT:    .vsave {d8, d9}
+; CHECK-NEXT:    vpush {d8, d9}
+; CHECK-NEXT:    .pad #16
+; CHECK-NEXT:    sub sp, #16
+; CHECK-NEXT:    vstmia sp, {s0, s1, s2, s3}
+; CHECK-NEXT:    ldr r0, [sp, #8]
+; CHECK-NEXT:    bl exp2f
+; CHECK-NEXT:    mov r4, r0
+; CHECK-NEXT:    ldr r0, [sp, #12]
+; CHECK-NEXT:    bl exp2f
+; CHECK-NEXT:    ldrd r5, r1, [sp]
+; CHECK-NEXT:    vmov s19, r0
+; CHECK-NEXT:    mov r0, r1
+; CHECK-NEXT:    vmov s18, r4
+; CHECK-NEXT:    bl exp2f
+; CHECK-NEXT:    vmov s17, r0
+; CHECK-NEXT:    mov r0, r5
+; CHECK-NEXT:    bl exp2f
+; CHECK-NEXT:    vmov s16, r0
+; CHECK-NEXT:    vmov q0, q4
+; CHECK-NEXT:    add sp, #16
+; CHECK-NEXT:    vpop {d8, d9}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %0 = call fast <4 x float> @llvm.exp2.v4f32(<4 x float> %src)
   ret <4 x float> %0
 }
 
 define arm_aapcs_vfpcc <8 x half> @exp2_float16_t(<8 x half> %src) {
-; CHECK-MVE-LABEL: exp2_float16_t:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    .save {r4, lr}
-; CHECK-MVE-NEXT:    push {r4, lr}
-; CHECK-MVE-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    vpush {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    .pad #32
-; CHECK-MVE-NEXT:    sub sp, #32
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-MVE-NEXT:    vmov q4, q0
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #24]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #24]
-; CHECK-MVE-NEXT:    bl exp2f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r4, s0
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #28]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #28]
-; CHECK-MVE-NEXT:    bl exp2f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    movs r1, #0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vdup.16 q5, r1
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[0], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.16 q5[1], r4
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #20]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #20]
-; CHECK-MVE-NEXT:    bl exp2f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[2], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #16]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #16]
-; CHECK-MVE-NEXT:    bl exp2f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[3], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[4]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #12]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #12]
-; CHECK-MVE-NEXT:    bl exp2f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[4], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[5]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #8]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #8]
-; CHECK-MVE-NEXT:    bl exp2f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[5], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[6]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #4]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVE-NEXT:    bl exp2f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[6], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[7]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp]
-; CHECK-MVE-NEXT:    ldr r0, [sp]
-; CHECK-MVE-NEXT:    bl exp2f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[7], r0
-; CHECK-MVE-NEXT:    vmov q0, q5
-; CHECK-MVE-NEXT:    add sp, #32
-; CHECK-MVE-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    pop {r4, pc}
-;
-; CHECK-MVEFP-LABEL: exp2_float16_t:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    .save {r4, lr}
-; CHECK-MVEFP-NEXT:    push {r4, lr}
-; CHECK-MVEFP-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    vpush {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    .pad #32
-; CHECK-MVEFP-NEXT:    sub sp, #32
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-MVEFP-NEXT:    vmov q4, q0
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #24]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #24]
-; CHECK-MVEFP-NEXT:    bl exp2f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r4, s0
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #28]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #28]
-; CHECK-MVEFP-NEXT:    bl exp2f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[0], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[1], r4
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #20]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #20]
-; CHECK-MVEFP-NEXT:    bl exp2f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[2], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #16]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #16]
-; CHECK-MVEFP-NEXT:    bl exp2f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[3], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[4]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #12]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #12]
-; CHECK-MVEFP-NEXT:    bl exp2f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[4], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[5]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #8]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #8]
-; CHECK-MVEFP-NEXT:    bl exp2f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[5], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[6]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #4]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVEFP-NEXT:    bl exp2f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[6], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[7]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp]
-; CHECK-MVEFP-NEXT:    bl exp2f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[7], r0
-; CHECK-MVEFP-NEXT:    vmov q0, q5
-; CHECK-MVEFP-NEXT:    add sp, #32
-; CHECK-MVEFP-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    pop {r4, pc}
+; CHECK-LABEL: exp2_float16_t:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
+; CHECK-NEXT:    vpush {d8, d9, d10, d11}
+; CHECK-NEXT:    .pad #32
+; CHECK-NEXT:    sub sp, #32
+; CHECK-NEXT:    vmov.u16 r0, q0[1]
+; CHECK-NEXT:    vmov q4, q0
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #24]
+; CHECK-NEXT:    ldr r0, [sp, #24]
+; CHECK-NEXT:    bl exp2f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[0]
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #28]
+; CHECK-NEXT:    ldr r0, [sp, #28]
+; CHECK-NEXT:    bl exp2f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[0], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[2]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.16 q5[1], r4
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #20]
+; CHECK-NEXT:    ldr r0, [sp, #20]
+; CHECK-NEXT:    bl exp2f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[2], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[3]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #16]
+; CHECK-NEXT:    ldr r0, [sp, #16]
+; CHECK-NEXT:    bl exp2f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[3], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[4]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #12]
+; CHECK-NEXT:    ldr r0, [sp, #12]
+; CHECK-NEXT:    bl exp2f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[4], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[5]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #8]
+; CHECK-NEXT:    ldr r0, [sp, #8]
+; CHECK-NEXT:    bl exp2f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[5], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[6]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #4]
+; CHECK-NEXT:    ldr r0, [sp, #4]
+; CHECK-NEXT:    bl exp2f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[6], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[7]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp]
+; CHECK-NEXT:    ldr r0, [sp]
+; CHECK-NEXT:    bl exp2f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[7], r0
+; CHECK-NEXT:    vmov q0, q5
+; CHECK-NEXT:    add sp, #32
+; CHECK-NEXT:    vpop {d8, d9, d10, d11}
+; CHECK-NEXT:    pop {r4, pc}
 entry:
   %0 = call fast <8 x half> @llvm.exp2.v8f16(<8 x half> %src)
   ret <8 x half> %0
 }
 
 define arm_aapcs_vfpcc <4 x float> @log_float32_t(<4 x float> %src) {
-; CHECK-MVE-LABEL: log_float32_t:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-MVE-NEXT:    push {r4, r5, r7, lr}
-; CHECK-MVE-NEXT:    .vsave {d8, d9, d10}
-; CHECK-MVE-NEXT:    vpush {d8, d9, d10}
-; CHECK-MVE-NEXT:    .pad #16
-; CHECK-MVE-NEXT:    sub sp, #16
-; CHECK-MVE-NEXT:    vstmia sp, {s0, s1, s2, s3}
-; CHECK-MVE-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVE-NEXT:    bl logf
-; CHECK-MVE-NEXT:    mov r4, r0
-; CHECK-MVE-NEXT:    ldr r0, [sp]
-; CHECK-MVE-NEXT:    bl logf
-; CHECK-MVE-NEXT:    ldrd r1, r5, [sp, #8]
-; CHECK-MVE-NEXT:    vmov s16, r0
-; CHECK-MVE-NEXT:    vmov s18, r4
-; CHECK-MVE-NEXT:    mov r0, r1
-; CHECK-MVE-NEXT:    bl logf
-; CHECK-MVE-NEXT:    vmov s20, r0
-; CHECK-MVE-NEXT:    mov r0, r5
-; CHECK-MVE-NEXT:    bl logf
-; CHECK-MVE-NEXT:    movs r1, #0
-; CHECK-MVE-NEXT:    vmov s4, r0
-; CHECK-MVE-NEXT:    vdup.32 q0, r1
-; CHECK-MVE-NEXT:    vmov.f32 s0, s16
-; CHECK-MVE-NEXT:    vmov.f32 s1, s18
-; CHECK-MVE-NEXT:    vmov.f32 s2, s20
-; CHECK-MVE-NEXT:    vmov.f32 s3, s4
-; CHECK-MVE-NEXT:    add sp, #16
-; CHECK-MVE-NEXT:    vpop {d8, d9, d10}
-; CHECK-MVE-NEXT:    pop {r4, r5, r7, pc}
-;
-; CHECK-MVEFP-LABEL: log_float32_t:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-MVEFP-NEXT:    push {r4, r5, r7, lr}
-; CHECK-MVEFP-NEXT:    .vsave {d8, d9}
-; CHECK-MVEFP-NEXT:    vpush {d8, d9}
-; CHECK-MVEFP-NEXT:    .pad #16
-; CHECK-MVEFP-NEXT:    sub sp, #16
-; CHECK-MVEFP-NEXT:    vstmia sp, {s0, s1, s2, s3}
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVEFP-NEXT:    bl logf
-; CHECK-MVEFP-NEXT:    mov r4, r0
-; CHECK-MVEFP-NEXT:    ldr r0, [sp]
-; CHECK-MVEFP-NEXT:    bl logf
-; CHECK-MVEFP-NEXT:    ldrd r1, r5, [sp, #8]
-; CHECK-MVEFP-NEXT:    vmov s16, r0
-; CHECK-MVEFP-NEXT:    vmov s17, r4
-; CHECK-MVEFP-NEXT:    mov r0, r1
-; CHECK-MVEFP-NEXT:    bl logf
-; CHECK-MVEFP-NEXT:    vmov s18, r0
-; CHECK-MVEFP-NEXT:    mov r0, r5
-; CHECK-MVEFP-NEXT:    bl logf
-; CHECK-MVEFP-NEXT:    vmov s19, r0
-; CHECK-MVEFP-NEXT:    vmov q0, q4
-; CHECK-MVEFP-NEXT:    add sp, #16
-; CHECK-MVEFP-NEXT:    vpop {d8, d9}
-; CHECK-MVEFP-NEXT:    pop {r4, r5, r7, pc}
+; CHECK-LABEL: log_float32_t:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
+; CHECK-NEXT:    .vsave {d8, d9}
+; CHECK-NEXT:    vpush {d8, d9}
+; CHECK-NEXT:    .pad #16
+; CHECK-NEXT:    sub sp, #16
+; CHECK-NEXT:    vstmia sp, {s0, s1, s2, s3}
+; CHECK-NEXT:    ldr r0, [sp, #8]
+; CHECK-NEXT:    bl logf
+; CHECK-NEXT:    mov r4, r0
+; CHECK-NEXT:    ldr r0, [sp, #12]
+; CHECK-NEXT:    bl logf
+; CHECK-NEXT:    ldrd r5, r1, [sp]
+; CHECK-NEXT:    vmov s19, r0
+; CHECK-NEXT:    mov r0, r1
+; CHECK-NEXT:    vmov s18, r4
+; CHECK-NEXT:    bl logf
+; CHECK-NEXT:    vmov s17, r0
+; CHECK-NEXT:    mov r0, r5
+; CHECK-NEXT:    bl logf
+; CHECK-NEXT:    vmov s16, r0
+; CHECK-NEXT:    vmov q0, q4
+; CHECK-NEXT:    add sp, #16
+; CHECK-NEXT:    vpop {d8, d9}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %0 = call fast <4 x float> @llvm.log.v4f32(<4 x float> %src)
   ret <4 x float> %0
 }
 
 define arm_aapcs_vfpcc <8 x half> @log_float16_t(<8 x half> %src) {
-; CHECK-MVE-LABEL: log_float16_t:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    .save {r4, lr}
-; CHECK-MVE-NEXT:    push {r4, lr}
-; CHECK-MVE-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    vpush {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    .pad #32
-; CHECK-MVE-NEXT:    sub sp, #32
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-MVE-NEXT:    vmov q4, q0
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #24]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #24]
-; CHECK-MVE-NEXT:    bl logf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r4, s0
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #28]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #28]
-; CHECK-MVE-NEXT:    bl logf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    movs r1, #0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vdup.16 q5, r1
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[0], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.16 q5[1], r4
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #20]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #20]
-; CHECK-MVE-NEXT:    bl logf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[2], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #16]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #16]
-; CHECK-MVE-NEXT:    bl logf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[3], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[4]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #12]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #12]
-; CHECK-MVE-NEXT:    bl logf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[4], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[5]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #8]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #8]
-; CHECK-MVE-NEXT:    bl logf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[5], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[6]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #4]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVE-NEXT:    bl logf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[6], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[7]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp]
-; CHECK-MVE-NEXT:    ldr r0, [sp]
-; CHECK-MVE-NEXT:    bl logf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[7], r0
-; CHECK-MVE-NEXT:    vmov q0, q5
-; CHECK-MVE-NEXT:    add sp, #32
-; CHECK-MVE-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    pop {r4, pc}
-;
-; CHECK-MVEFP-LABEL: log_float16_t:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    .save {r4, lr}
-; CHECK-MVEFP-NEXT:    push {r4, lr}
-; CHECK-MVEFP-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    vpush {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    .pad #32
-; CHECK-MVEFP-NEXT:    sub sp, #32
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-MVEFP-NEXT:    vmov q4, q0
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #24]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #24]
-; CHECK-MVEFP-NEXT:    bl logf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r4, s0
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #28]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #28]
-; CHECK-MVEFP-NEXT:    bl logf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[0], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[1], r4
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #20]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #20]
-; CHECK-MVEFP-NEXT:    bl logf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[2], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #16]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #16]
-; CHECK-MVEFP-NEXT:    bl logf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[3], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[4]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #12]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #12]
-; CHECK-MVEFP-NEXT:    bl logf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[4], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[5]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #8]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #8]
-; CHECK-MVEFP-NEXT:    bl logf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[5], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[6]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #4]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVEFP-NEXT:    bl logf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[6], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[7]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp]
-; CHECK-MVEFP-NEXT:    bl logf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[7], r0
-; CHECK-MVEFP-NEXT:    vmov q0, q5
-; CHECK-MVEFP-NEXT:    add sp, #32
-; CHECK-MVEFP-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    pop {r4, pc}
+; CHECK-LABEL: log_float16_t:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
+; CHECK-NEXT:    vpush {d8, d9, d10, d11}
+; CHECK-NEXT:    .pad #32
+; CHECK-NEXT:    sub sp, #32
+; CHECK-NEXT:    vmov.u16 r0, q0[1]
+; CHECK-NEXT:    vmov q4, q0
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #24]
+; CHECK-NEXT:    ldr r0, [sp, #24]
+; CHECK-NEXT:    bl logf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[0]
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #28]
+; CHECK-NEXT:    ldr r0, [sp, #28]
+; CHECK-NEXT:    bl logf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[0], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[2]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.16 q5[1], r4
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #20]
+; CHECK-NEXT:    ldr r0, [sp, #20]
+; CHECK-NEXT:    bl logf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[2], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[3]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #16]
+; CHECK-NEXT:    ldr r0, [sp, #16]
+; CHECK-NEXT:    bl logf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[3], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[4]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #12]
+; CHECK-NEXT:    ldr r0, [sp, #12]
+; CHECK-NEXT:    bl logf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[4], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[5]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #8]
+; CHECK-NEXT:    ldr r0, [sp, #8]
+; CHECK-NEXT:    bl logf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[5], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[6]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #4]
+; CHECK-NEXT:    ldr r0, [sp, #4]
+; CHECK-NEXT:    bl logf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[6], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[7]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp]
+; CHECK-NEXT:    ldr r0, [sp]
+; CHECK-NEXT:    bl logf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[7], r0
+; CHECK-NEXT:    vmov q0, q5
+; CHECK-NEXT:    add sp, #32
+; CHECK-NEXT:    vpop {d8, d9, d10, d11}
+; CHECK-NEXT:    pop {r4, pc}
 entry:
   %0 = call fast <8 x half> @llvm.log.v8f16(<8 x half> %src)
   ret <8 x half> %0
 }
 
 define arm_aapcs_vfpcc <4 x float> @log2_float32_t(<4 x float> %src) {
-; CHECK-MVE-LABEL: log2_float32_t:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-MVE-NEXT:    push {r4, r5, r7, lr}
-; CHECK-MVE-NEXT:    .vsave {d8, d9, d10}
-; CHECK-MVE-NEXT:    vpush {d8, d9, d10}
-; CHECK-MVE-NEXT:    .pad #16
-; CHECK-MVE-NEXT:    sub sp, #16
-; CHECK-MVE-NEXT:    vstmia sp, {s0, s1, s2, s3}
-; CHECK-MVE-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVE-NEXT:    bl log2f
-; CHECK-MVE-NEXT:    mov r4, r0
-; CHECK-MVE-NEXT:    ldr r0, [sp]
-; CHECK-MVE-NEXT:    bl log2f
-; CHECK-MVE-NEXT:    ldrd r1, r5, [sp, #8]
-; CHECK-MVE-NEXT:    vmov s16, r0
-; CHECK-MVE-NEXT:    vmov s18, r4
-; CHECK-MVE-NEXT:    mov r0, r1
-; CHECK-MVE-NEXT:    bl log2f
-; CHECK-MVE-NEXT:    vmov s20, r0
-; CHECK-MVE-NEXT:    mov r0, r5
-; CHECK-MVE-NEXT:    bl log2f
-; CHECK-MVE-NEXT:    movs r1, #0
-; CHECK-MVE-NEXT:    vmov s4, r0
-; CHECK-MVE-NEXT:    vdup.32 q0, r1
-; CHECK-MVE-NEXT:    vmov.f32 s0, s16
-; CHECK-MVE-NEXT:    vmov.f32 s1, s18
-; CHECK-MVE-NEXT:    vmov.f32 s2, s20
-; CHECK-MVE-NEXT:    vmov.f32 s3, s4
-; CHECK-MVE-NEXT:    add sp, #16
-; CHECK-MVE-NEXT:    vpop {d8, d9, d10}
-; CHECK-MVE-NEXT:    pop {r4, r5, r7, pc}
-;
-; CHECK-MVEFP-LABEL: log2_float32_t:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-MVEFP-NEXT:    push {r4, r5, r7, lr}
-; CHECK-MVEFP-NEXT:    .vsave {d8, d9}
-; CHECK-MVEFP-NEXT:    vpush {d8, d9}
-; CHECK-MVEFP-NEXT:    .pad #16
-; CHECK-MVEFP-NEXT:    sub sp, #16
-; CHECK-MVEFP-NEXT:    vstmia sp, {s0, s1, s2, s3}
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVEFP-NEXT:    bl log2f
-; CHECK-MVEFP-NEXT:    mov r4, r0
-; CHECK-MVEFP-NEXT:    ldr r0, [sp]
-; CHECK-MVEFP-NEXT:    bl log2f
-; CHECK-MVEFP-NEXT:    ldrd r1, r5, [sp, #8]
-; CHECK-MVEFP-NEXT:    vmov s16, r0
-; CHECK-MVEFP-NEXT:    vmov s17, r4
-; CHECK-MVEFP-NEXT:    mov r0, r1
-; CHECK-MVEFP-NEXT:    bl log2f
-; CHECK-MVEFP-NEXT:    vmov s18, r0
-; CHECK-MVEFP-NEXT:    mov r0, r5
-; CHECK-MVEFP-NEXT:    bl log2f
-; CHECK-MVEFP-NEXT:    vmov s19, r0
-; CHECK-MVEFP-NEXT:    vmov q0, q4
-; CHECK-MVEFP-NEXT:    add sp, #16
-; CHECK-MVEFP-NEXT:    vpop {d8, d9}
-; CHECK-MVEFP-NEXT:    pop {r4, r5, r7, pc}
+; CHECK-LABEL: log2_float32_t:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
+; CHECK-NEXT:    .vsave {d8, d9}
+; CHECK-NEXT:    vpush {d8, d9}
+; CHECK-NEXT:    .pad #16
+; CHECK-NEXT:    sub sp, #16
+; CHECK-NEXT:    vstmia sp, {s0, s1, s2, s3}
+; CHECK-NEXT:    ldr r0, [sp, #8]
+; CHECK-NEXT:    bl log2f
+; CHECK-NEXT:    mov r4, r0
+; CHECK-NEXT:    ldr r0, [sp, #12]
+; CHECK-NEXT:    bl log2f
+; CHECK-NEXT:    ldrd r5, r1, [sp]
+; CHECK-NEXT:    vmov s19, r0
+; CHECK-NEXT:    mov r0, r1
+; CHECK-NEXT:    vmov s18, r4
+; CHECK-NEXT:    bl log2f
+; CHECK-NEXT:    vmov s17, r0
+; CHECK-NEXT:    mov r0, r5
+; CHECK-NEXT:    bl log2f
+; CHECK-NEXT:    vmov s16, r0
+; CHECK-NEXT:    vmov q0, q4
+; CHECK-NEXT:    add sp, #16
+; CHECK-NEXT:    vpop {d8, d9}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %0 = call fast <4 x float> @llvm.log2.v4f32(<4 x float> %src)
   ret <4 x float> %0
 }
 
 define arm_aapcs_vfpcc <8 x half> @log2_float16_t(<8 x half> %src) {
-; CHECK-MVE-LABEL: log2_float16_t:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    .save {r4, lr}
-; CHECK-MVE-NEXT:    push {r4, lr}
-; CHECK-MVE-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    vpush {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    .pad #32
-; CHECK-MVE-NEXT:    sub sp, #32
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-MVE-NEXT:    vmov q4, q0
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #24]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #24]
-; CHECK-MVE-NEXT:    bl log2f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r4, s0
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #28]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #28]
-; CHECK-MVE-NEXT:    bl log2f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    movs r1, #0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vdup.16 q5, r1
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[0], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.16 q5[1], r4
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #20]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #20]
-; CHECK-MVE-NEXT:    bl log2f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[2], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #16]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #16]
-; CHECK-MVE-NEXT:    bl log2f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[3], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[4]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #12]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #12]
-; CHECK-MVE-NEXT:    bl log2f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[4], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[5]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #8]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #8]
-; CHECK-MVE-NEXT:    bl log2f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[5], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[6]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #4]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVE-NEXT:    bl log2f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[6], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[7]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp]
-; CHECK-MVE-NEXT:    ldr r0, [sp]
-; CHECK-MVE-NEXT:    bl log2f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[7], r0
-; CHECK-MVE-NEXT:    vmov q0, q5
-; CHECK-MVE-NEXT:    add sp, #32
-; CHECK-MVE-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    pop {r4, pc}
-;
-; CHECK-MVEFP-LABEL: log2_float16_t:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    .save {r4, lr}
-; CHECK-MVEFP-NEXT:    push {r4, lr}
-; CHECK-MVEFP-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    vpush {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    .pad #32
-; CHECK-MVEFP-NEXT:    sub sp, #32
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-MVEFP-NEXT:    vmov q4, q0
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #24]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #24]
-; CHECK-MVEFP-NEXT:    bl log2f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r4, s0
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #28]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #28]
-; CHECK-MVEFP-NEXT:    bl log2f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[0], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[1], r4
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #20]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #20]
-; CHECK-MVEFP-NEXT:    bl log2f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[2], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #16]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #16]
-; CHECK-MVEFP-NEXT:    bl log2f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[3], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[4]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #12]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #12]
-; CHECK-MVEFP-NEXT:    bl log2f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[4], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[5]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #8]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #8]
-; CHECK-MVEFP-NEXT:    bl log2f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[5], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[6]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #4]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVEFP-NEXT:    bl log2f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[6], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[7]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp]
-; CHECK-MVEFP-NEXT:    bl log2f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[7], r0
-; CHECK-MVEFP-NEXT:    vmov q0, q5
-; CHECK-MVEFP-NEXT:    add sp, #32
-; CHECK-MVEFP-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    pop {r4, pc}
+; CHECK-LABEL: log2_float16_t:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
+; CHECK-NEXT:    vpush {d8, d9, d10, d11}
+; CHECK-NEXT:    .pad #32
+; CHECK-NEXT:    sub sp, #32
+; CHECK-NEXT:    vmov.u16 r0, q0[1]
+; CHECK-NEXT:    vmov q4, q0
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #24]
+; CHECK-NEXT:    ldr r0, [sp, #24]
+; CHECK-NEXT:    bl log2f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[0]
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #28]
+; CHECK-NEXT:    ldr r0, [sp, #28]
+; CHECK-NEXT:    bl log2f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[0], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[2]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.16 q5[1], r4
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #20]
+; CHECK-NEXT:    ldr r0, [sp, #20]
+; CHECK-NEXT:    bl log2f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[2], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[3]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #16]
+; CHECK-NEXT:    ldr r0, [sp, #16]
+; CHECK-NEXT:    bl log2f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[3], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[4]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #12]
+; CHECK-NEXT:    ldr r0, [sp, #12]
+; CHECK-NEXT:    bl log2f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[4], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[5]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #8]
+; CHECK-NEXT:    ldr r0, [sp, #8]
+; CHECK-NEXT:    bl log2f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[5], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[6]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #4]
+; CHECK-NEXT:    ldr r0, [sp, #4]
+; CHECK-NEXT:    bl log2f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[6], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[7]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp]
+; CHECK-NEXT:    ldr r0, [sp]
+; CHECK-NEXT:    bl log2f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[7], r0
+; CHECK-NEXT:    vmov q0, q5
+; CHECK-NEXT:    add sp, #32
+; CHECK-NEXT:    vpop {d8, d9, d10, d11}
+; CHECK-NEXT:    pop {r4, pc}
 entry:
   %0 = call fast <8 x half> @llvm.log2.v8f16(<8 x half> %src)
   ret <8 x half> %0
 }
 
 define arm_aapcs_vfpcc <4 x float> @log10_float32_t(<4 x float> %src) {
-; CHECK-MVE-LABEL: log10_float32_t:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-MVE-NEXT:    push {r4, r5, r7, lr}
-; CHECK-MVE-NEXT:    .vsave {d8, d9, d10}
-; CHECK-MVE-NEXT:    vpush {d8, d9, d10}
-; CHECK-MVE-NEXT:    .pad #16
-; CHECK-MVE-NEXT:    sub sp, #16
-; CHECK-MVE-NEXT:    vstmia sp, {s0, s1, s2, s3}
-; CHECK-MVE-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVE-NEXT:    bl log10f
-; CHECK-MVE-NEXT:    mov r4, r0
-; CHECK-MVE-NEXT:    ldr r0, [sp]
-; CHECK-MVE-NEXT:    bl log10f
-; CHECK-MVE-NEXT:    ldrd r1, r5, [sp, #8]
-; CHECK-MVE-NEXT:    vmov s16, r0
-; CHECK-MVE-NEXT:    vmov s18, r4
-; CHECK-MVE-NEXT:    mov r0, r1
-; CHECK-MVE-NEXT:    bl log10f
-; CHECK-MVE-NEXT:    vmov s20, r0
-; CHECK-MVE-NEXT:    mov r0, r5
-; CHECK-MVE-NEXT:    bl log10f
-; CHECK-MVE-NEXT:    movs r1, #0
-; CHECK-MVE-NEXT:    vmov s4, r0
-; CHECK-MVE-NEXT:    vdup.32 q0, r1
-; CHECK-MVE-NEXT:    vmov.f32 s0, s16
-; CHECK-MVE-NEXT:    vmov.f32 s1, s18
-; CHECK-MVE-NEXT:    vmov.f32 s2, s20
-; CHECK-MVE-NEXT:    vmov.f32 s3, s4
-; CHECK-MVE-NEXT:    add sp, #16
-; CHECK-MVE-NEXT:    vpop {d8, d9, d10}
-; CHECK-MVE-NEXT:    pop {r4, r5, r7, pc}
-;
-; CHECK-MVEFP-LABEL: log10_float32_t:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-MVEFP-NEXT:    push {r4, r5, r7, lr}
-; CHECK-MVEFP-NEXT:    .vsave {d8, d9}
-; CHECK-MVEFP-NEXT:    vpush {d8, d9}
-; CHECK-MVEFP-NEXT:    .pad #16
-; CHECK-MVEFP-NEXT:    sub sp, #16
-; CHECK-MVEFP-NEXT:    vstmia sp, {s0, s1, s2, s3}
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVEFP-NEXT:    bl log10f
-; CHECK-MVEFP-NEXT:    mov r4, r0
-; CHECK-MVEFP-NEXT:    ldr r0, [sp]
-; CHECK-MVEFP-NEXT:    bl log10f
-; CHECK-MVEFP-NEXT:    ldrd r1, r5, [sp, #8]
-; CHECK-MVEFP-NEXT:    vmov s16, r0
-; CHECK-MVEFP-NEXT:    vmov s17, r4
-; CHECK-MVEFP-NEXT:    mov r0, r1
-; CHECK-MVEFP-NEXT:    bl log10f
-; CHECK-MVEFP-NEXT:    vmov s18, r0
-; CHECK-MVEFP-NEXT:    mov r0, r5
-; CHECK-MVEFP-NEXT:    bl log10f
-; CHECK-MVEFP-NEXT:    vmov s19, r0
-; CHECK-MVEFP-NEXT:    vmov q0, q4
-; CHECK-MVEFP-NEXT:    add sp, #16
-; CHECK-MVEFP-NEXT:    vpop {d8, d9}
-; CHECK-MVEFP-NEXT:    pop {r4, r5, r7, pc}
+; CHECK-LABEL: log10_float32_t:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, r5, r7, lr}
+; CHECK-NEXT:    push {r4, r5, r7, lr}
+; CHECK-NEXT:    .vsave {d8, d9}
+; CHECK-NEXT:    vpush {d8, d9}
+; CHECK-NEXT:    .pad #16
+; CHECK-NEXT:    sub sp, #16
+; CHECK-NEXT:    vstmia sp, {s0, s1, s2, s3}
+; CHECK-NEXT:    ldr r0, [sp, #8]
+; CHECK-NEXT:    bl log10f
+; CHECK-NEXT:    mov r4, r0
+; CHECK-NEXT:    ldr r0, [sp, #12]
+; CHECK-NEXT:    bl log10f
+; CHECK-NEXT:    ldrd r5, r1, [sp]
+; CHECK-NEXT:    vmov s19, r0
+; CHECK-NEXT:    mov r0, r1
+; CHECK-NEXT:    vmov s18, r4
+; CHECK-NEXT:    bl log10f
+; CHECK-NEXT:    vmov s17, r0
+; CHECK-NEXT:    mov r0, r5
+; CHECK-NEXT:    bl log10f
+; CHECK-NEXT:    vmov s16, r0
+; CHECK-NEXT:    vmov q0, q4
+; CHECK-NEXT:    add sp, #16
+; CHECK-NEXT:    vpop {d8, d9}
+; CHECK-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %0 = call fast <4 x float> @llvm.log10.v4f32(<4 x float> %src)
   ret <4 x float> %0
 }
 
 define arm_aapcs_vfpcc <8 x half> @log10_float16_t(<8 x half> %src) {
-; CHECK-MVE-LABEL: log10_float16_t:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    .save {r4, lr}
-; CHECK-MVE-NEXT:    push {r4, lr}
-; CHECK-MVE-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    vpush {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    .pad #32
-; CHECK-MVE-NEXT:    sub sp, #32
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-MVE-NEXT:    vmov q4, q0
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #24]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #24]
-; CHECK-MVE-NEXT:    bl log10f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r4, s0
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #28]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #28]
-; CHECK-MVE-NEXT:    bl log10f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    movs r1, #0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vdup.16 q5, r1
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[0], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.16 q5[1], r4
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #20]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #20]
-; CHECK-MVE-NEXT:    bl log10f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[2], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #16]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #16]
-; CHECK-MVE-NEXT:    bl log10f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[3], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[4]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #12]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #12]
-; CHECK-MVE-NEXT:    bl log10f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[4], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[5]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #8]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #8]
-; CHECK-MVE-NEXT:    bl log10f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[5], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[6]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #4]
-; CHECK-MVE-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVE-NEXT:    bl log10f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[6], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[7]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp]
-; CHECK-MVE-NEXT:    ldr r0, [sp]
-; CHECK-MVE-NEXT:    bl log10f
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q5[7], r0
-; CHECK-MVE-NEXT:    vmov q0, q5
-; CHECK-MVE-NEXT:    add sp, #32
-; CHECK-MVE-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-MVE-NEXT:    pop {r4, pc}
-;
-; CHECK-MVEFP-LABEL: log10_float16_t:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    .save {r4, lr}
-; CHECK-MVEFP-NEXT:    push {r4, lr}
-; CHECK-MVEFP-NEXT:    .vsave {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    vpush {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    .pad #32
-; CHECK-MVEFP-NEXT:    sub sp, #32
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-MVEFP-NEXT:    vmov q4, q0
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #24]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #24]
-; CHECK-MVEFP-NEXT:    bl log10f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r4, s0
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #28]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #28]
-; CHECK-MVEFP-NEXT:    bl log10f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[0], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[1], r4
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #20]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #20]
-; CHECK-MVEFP-NEXT:    bl log10f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[2], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #16]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #16]
-; CHECK-MVEFP-NEXT:    bl log10f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[3], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[4]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #12]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #12]
-; CHECK-MVEFP-NEXT:    bl log10f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[4], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[5]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #8]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #8]
-; CHECK-MVEFP-NEXT:    bl log10f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[5], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[6]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #4]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp, #4]
-; CHECK-MVEFP-NEXT:    bl log10f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[6], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[7]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp]
-; CHECK-MVEFP-NEXT:    ldr r0, [sp]
-; CHECK-MVEFP-NEXT:    bl log10f
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q5[7], r0
-; CHECK-MVEFP-NEXT:    vmov q0, q5
-; CHECK-MVEFP-NEXT:    add sp, #32
-; CHECK-MVEFP-NEXT:    vpop {d8, d9, d10, d11}
-; CHECK-MVEFP-NEXT:    pop {r4, pc}
+; CHECK-LABEL: log10_float16_t:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
+; CHECK-NEXT:    vpush {d8, d9, d10, d11}
+; CHECK-NEXT:    .pad #32
+; CHECK-NEXT:    sub sp, #32
+; CHECK-NEXT:    vmov.u16 r0, q0[1]
+; CHECK-NEXT:    vmov q4, q0
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #24]
+; CHECK-NEXT:    ldr r0, [sp, #24]
+; CHECK-NEXT:    bl log10f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[0]
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #28]
+; CHECK-NEXT:    ldr r0, [sp, #28]
+; CHECK-NEXT:    bl log10f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[0], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[2]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.16 q5[1], r4
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #20]
+; CHECK-NEXT:    ldr r0, [sp, #20]
+; CHECK-NEXT:    bl log10f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[2], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[3]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #16]
+; CHECK-NEXT:    ldr r0, [sp, #16]
+; CHECK-NEXT:    bl log10f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[3], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[4]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #12]
+; CHECK-NEXT:    ldr r0, [sp, #12]
+; CHECK-NEXT:    bl log10f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[4], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[5]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #8]
+; CHECK-NEXT:    ldr r0, [sp, #8]
+; CHECK-NEXT:    bl log10f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[5], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[6]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #4]
+; CHECK-NEXT:    ldr r0, [sp, #4]
+; CHECK-NEXT:    bl log10f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[6], r0
+; CHECK-NEXT:    vmov.u16 r0, q4[7]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp]
+; CHECK-NEXT:    ldr r0, [sp]
+; CHECK-NEXT:    bl log10f
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q5[7], r0
+; CHECK-NEXT:    vmov q0, q5
+; CHECK-NEXT:    add sp, #32
+; CHECK-NEXT:    vpop {d8, d9, d10, d11}
+; CHECK-NEXT:    pop {r4, pc}
 entry:
   %0 = call fast <8 x half> @llvm.log10.v8f16(<8 x half> %src)
   ret <8 x half> %0
 }
 
 define arm_aapcs_vfpcc <4 x float> @pow_float32_t(<4 x float> %src1, <4 x float> %src2) {
-; CHECK-MVE-LABEL: pow_float32_t:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    .save {r4, r5, r6, lr}
-; CHECK-MVE-NEXT:    push {r4, r5, r6, lr}
-; CHECK-MVE-NEXT:    .vsave {d8, d9, d10}
-; CHECK-MVE-NEXT:    vpush {d8, d9, d10}
-; CHECK-MVE-NEXT:    .pad #32
-; CHECK-MVE-NEXT:    sub sp, #32
-; CHECK-MVE-NEXT:    vstr s3, [sp, #24]
-; CHECK-MVE-NEXT:    ldr r4, [sp, #24]
-; CHECK-MVE-NEXT:    vstr s1, [sp, #8]
-; CHECK-MVE-NEXT:    vstr s5, [sp, #12]
-; CHECK-MVE-NEXT:    vstr s0, [sp]
-; CHECK-MVE-NEXT:    vstr s4, [sp, #4]
-; CHECK-MVE-NEXT:    vstr s7, [sp, #28]
-; CHECK-MVE-NEXT:    vstr s2, [sp, #16]
-; CHECK-MVE-NEXT:    vstr s6, [sp, #20]
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp, #8]
-; CHECK-MVE-NEXT:    bl powf
-; CHECK-MVE-NEXT:    mov r5, r0
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp]
-; CHECK-MVE-NEXT:    bl powf
-; CHECK-MVE-NEXT:    ldrd r2, r1, [sp, #16]
-; CHECK-MVE-NEXT:    vmov s16, r0
-; CHECK-MVE-NEXT:    ldr r6, [sp, #28]
-; CHECK-MVE-NEXT:    vmov s18, r5
-; CHECK-MVE-NEXT:    mov r0, r2
-; CHECK-MVE-NEXT:    bl powf
-; CHECK-MVE-NEXT:    vmov s20, r0
-; CHECK-MVE-NEXT:    mov r0, r4
-; CHECK-MVE-NEXT:    mov r1, r6
-; CHECK-MVE-NEXT:    bl powf
-; CHECK-MVE-NEXT:    movs r1, #0
-; CHECK-MVE-NEXT:    vmov s4, r0
-; CHECK-MVE-NEXT:    vdup.32 q0, r1
-; CHECK-MVE-NEXT:    vmov.f32 s0, s16
-; CHECK-MVE-NEXT:    vmov.f32 s1, s18
-; CHECK-MVE-NEXT:    vmov.f32 s2, s20
-; CHECK-MVE-NEXT:    vmov.f32 s3, s4
-; CHECK-MVE-NEXT:    add sp, #32
-; CHECK-MVE-NEXT:    vpop {d8, d9, d10}
-; CHECK-MVE-NEXT:    pop {r4, r5, r6, pc}
-;
-; CHECK-MVEFP-LABEL: pow_float32_t:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    .save {r4, r5, r6, lr}
-; CHECK-MVEFP-NEXT:    push {r4, r5, r6, lr}
-; CHECK-MVEFP-NEXT:    .vsave {d8, d9}
-; CHECK-MVEFP-NEXT:    vpush {d8, d9}
-; CHECK-MVEFP-NEXT:    .pad #32
-; CHECK-MVEFP-NEXT:    sub sp, #32
-; CHECK-MVEFP-NEXT:    vstr s3, [sp, #24]
-; CHECK-MVEFP-NEXT:    ldr r4, [sp, #24]
-; CHECK-MVEFP-NEXT:    vstr s1, [sp, #8]
-; CHECK-MVEFP-NEXT:    vstr s5, [sp, #12]
-; CHECK-MVEFP-NEXT:    vstr s0, [sp]
-; CHECK-MVEFP-NEXT:    vstr s4, [sp, #4]
-; CHECK-MVEFP-NEXT:    vstr s7, [sp, #28]
-; CHECK-MVEFP-NEXT:    vstr s2, [sp, #16]
-; CHECK-MVEFP-NEXT:    vstr s6, [sp, #20]
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp, #8]
-; CHECK-MVEFP-NEXT:    bl powf
-; CHECK-MVEFP-NEXT:    mov r5, r0
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp]
-; CHECK-MVEFP-NEXT:    bl powf
-; CHECK-MVEFP-NEXT:    ldrd r2, r1, [sp, #16]
-; CHECK-MVEFP-NEXT:    vmov s16, r0
-; CHECK-MVEFP-NEXT:    ldr r6, [sp, #28]
-; CHECK-MVEFP-NEXT:    vmov s17, r5
-; CHECK-MVEFP-NEXT:    mov r0, r2
-; CHECK-MVEFP-NEXT:    bl powf
-; CHECK-MVEFP-NEXT:    vmov s18, r0
-; CHECK-MVEFP-NEXT:    mov r0, r4
-; CHECK-MVEFP-NEXT:    mov r1, r6
-; CHECK-MVEFP-NEXT:    bl powf
-; CHECK-MVEFP-NEXT:    vmov s19, r0
-; CHECK-MVEFP-NEXT:    vmov q0, q4
-; CHECK-MVEFP-NEXT:    add sp, #32
-; CHECK-MVEFP-NEXT:    vpop {d8, d9}
-; CHECK-MVEFP-NEXT:    pop {r4, r5, r6, pc}
+; CHECK-LABEL: pow_float32_t:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, r5, r6, lr}
+; CHECK-NEXT:    push {r4, r5, r6, lr}
+; CHECK-NEXT:    .vsave {d8, d9}
+; CHECK-NEXT:    vpush {d8, d9}
+; CHECK-NEXT:    .pad #32
+; CHECK-NEXT:    sub sp, #32
+; CHECK-NEXT:    vstr s0, [sp]
+; CHECK-NEXT:    ldr r4, [sp]
+; CHECK-NEXT:    vstr s2, [sp, #16]
+; CHECK-NEXT:    vstr s6, [sp, #20]
+; CHECK-NEXT:    vstr s3, [sp, #24]
+; CHECK-NEXT:    vstr s7, [sp, #28]
+; CHECK-NEXT:    vstr s4, [sp, #4]
+; CHECK-NEXT:    vstr s1, [sp, #8]
+; CHECK-NEXT:    vstr s5, [sp, #12]
+; CHECK-NEXT:    ldrd r0, r1, [sp, #16]
+; CHECK-NEXT:    bl powf
+; CHECK-NEXT:    mov r5, r0
+; CHECK-NEXT:    ldrd r0, r1, [sp, #24]
+; CHECK-NEXT:    bl powf
+; CHECK-NEXT:    ldr r1, [sp, #12]
+; CHECK-NEXT:    vmov s19, r0
+; CHECK-NEXT:    ldrd r6, r2, [sp, #4]
+; CHECK-NEXT:    vmov s18, r5
+; CHECK-NEXT:    mov r0, r2
+; CHECK-NEXT:    bl powf
+; CHECK-NEXT:    vmov s17, r0
+; CHECK-NEXT:    mov r0, r4
+; CHECK-NEXT:    mov r1, r6
+; CHECK-NEXT:    bl powf
+; CHECK-NEXT:    vmov s16, r0
+; CHECK-NEXT:    vmov q0, q4
+; CHECK-NEXT:    add sp, #32
+; CHECK-NEXT:    vpop {d8, d9}
+; CHECK-NEXT:    pop {r4, r5, r6, pc}
 entry:
   %0 = call fast <4 x float> @llvm.pow.v4f32(<4 x float> %src1, <4 x float> %src2)
   ret <4 x float> %0
 }
 
 define arm_aapcs_vfpcc <8 x half> @pow_float16_t(<8 x half> %src1, <8 x half> %src2) {
-; CHECK-MVE-LABEL: pow_float16_t:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    .save {r4, lr}
-; CHECK-MVE-NEXT:    push {r4, lr}
-; CHECK-MVE-NEXT:    .vsave {d8, d9, d10, d11, d12, d13}
-; CHECK-MVE-NEXT:    vpush {d8, d9, d10, d11, d12, d13}
-; CHECK-MVE-NEXT:    .pad #64
-; CHECK-MVE-NEXT:    sub sp, #64
-; CHECK-MVE-NEXT:    vmov q5, q1
-; CHECK-MVE-NEXT:    vmov q4, q0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q1[0]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-MVE-NEXT:    vmov s2, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVE-NEXT:    vstr s2, [sp, #56]
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #60]
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp, #56]
-; CHECK-MVE-NEXT:    bl powf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q5[1]
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r4, s0
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[1]
-; CHECK-MVE-NEXT:    vmov s2, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVE-NEXT:    vstr s2, [sp, #48]
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #52]
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp, #48]
-; CHECK-MVE-NEXT:    bl powf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    movs r1, #0
-; CHECK-MVE-NEXT:    vdup.16 q6, r1
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q6[0], r4
-; CHECK-MVE-NEXT:    vmov.16 q6[1], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q5[2]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-MVE-NEXT:    vmov s2, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVE-NEXT:    vstr s2, [sp, #40]
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #44]
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp, #40]
-; CHECK-MVE-NEXT:    bl powf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q6[2], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q5[3]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-MVE-NEXT:    vmov s2, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVE-NEXT:    vstr s2, [sp, #32]
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #36]
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp, #32]
-; CHECK-MVE-NEXT:    bl powf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q6[3], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q5[4]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[4]
-; CHECK-MVE-NEXT:    vmov s2, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVE-NEXT:    vstr s2, [sp, #24]
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #28]
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp, #24]
-; CHECK-MVE-NEXT:    bl powf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q6[4], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q5[5]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[5]
-; CHECK-MVE-NEXT:    vmov s2, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVE-NEXT:    vstr s2, [sp, #16]
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #20]
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp, #16]
-; CHECK-MVE-NEXT:    bl powf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q6[5], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q5[6]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[6]
-; CHECK-MVE-NEXT:    vmov s2, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVE-NEXT:    vstr s2, [sp, #8]
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #12]
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp, #8]
-; CHECK-MVE-NEXT:    bl powf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q6[6], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q5[7]
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q4[7]
-; CHECK-MVE-NEXT:    vmov s2, r0
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVE-NEXT:    vstr s2, [sp]
-; CHECK-MVE-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVE-NEXT:    vstr s0, [sp, #4]
-; CHECK-MVE-NEXT:    ldrd r0, r1, [sp]
-; CHECK-MVE-NEXT:    bl powf
-; CHECK-MVE-NEXT:    vmov s0, r0
-; CHECK-MVE-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVE-NEXT:    vmov r0, s0
-; CHECK-MVE-NEXT:    vmov.16 q6[7], r0
-; CHECK-MVE-NEXT:    vmov q0, q6
-; CHECK-MVE-NEXT:    add sp, #64
-; CHECK-MVE-NEXT:    vpop {d8, d9, d10, d11, d12, d13}
-; CHECK-MVE-NEXT:    pop {r4, pc}
-;
-; CHECK-MVEFP-LABEL: pow_float16_t:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    .save {r4, lr}
-; CHECK-MVEFP-NEXT:    push {r4, lr}
-; CHECK-MVEFP-NEXT:    .vsave {d8, d9, d10, d11, d12, d13}
-; CHECK-MVEFP-NEXT:    vpush {d8, d9, d10, d11, d12, d13}
-; CHECK-MVEFP-NEXT:    .pad #64
-; CHECK-MVEFP-NEXT:    sub sp, #64
-; CHECK-MVEFP-NEXT:    vmov q5, q1
-; CHECK-MVEFP-NEXT:    vmov q4, q0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q1[0]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[0]
-; CHECK-MVEFP-NEXT:    vmov s2, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVEFP-NEXT:    vstr s2, [sp, #56]
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #60]
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp, #56]
-; CHECK-MVEFP-NEXT:    bl powf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q5[1]
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r4, s0
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[1]
-; CHECK-MVEFP-NEXT:    vmov s2, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVEFP-NEXT:    vstr s2, [sp, #48]
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #52]
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp, #48]
-; CHECK-MVEFP-NEXT:    bl powf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.16 q6[0], r4
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q6[1], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q5[2]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[2]
-; CHECK-MVEFP-NEXT:    vmov s2, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVEFP-NEXT:    vstr s2, [sp, #40]
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #44]
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp, #40]
-; CHECK-MVEFP-NEXT:    bl powf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q6[2], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q5[3]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[3]
-; CHECK-MVEFP-NEXT:    vmov s2, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVEFP-NEXT:    vstr s2, [sp, #32]
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #36]
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp, #32]
-; CHECK-MVEFP-NEXT:    bl powf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q6[3], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q5[4]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[4]
-; CHECK-MVEFP-NEXT:    vmov s2, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVEFP-NEXT:    vstr s2, [sp, #24]
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #28]
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp, #24]
-; CHECK-MVEFP-NEXT:    bl powf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q6[4], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q5[5]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[5]
-; CHECK-MVEFP-NEXT:    vmov s2, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVEFP-NEXT:    vstr s2, [sp, #16]
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #20]
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp, #16]
-; CHECK-MVEFP-NEXT:    bl powf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q6[5], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q5[6]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[6]
-; CHECK-MVEFP-NEXT:    vmov s2, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVEFP-NEXT:    vstr s2, [sp, #8]
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #12]
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp, #8]
-; CHECK-MVEFP-NEXT:    bl powf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q6[6], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q5[7]
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q4[7]
-; CHECK-MVEFP-NEXT:    vmov s2, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s2, s2
-; CHECK-MVEFP-NEXT:    vstr s2, [sp]
-; CHECK-MVEFP-NEXT:    vcvtb.f32.f16 s0, s0
-; CHECK-MVEFP-NEXT:    vstr s0, [sp, #4]
-; CHECK-MVEFP-NEXT:    ldrd r0, r1, [sp]
-; CHECK-MVEFP-NEXT:    bl powf
-; CHECK-MVEFP-NEXT:    vmov s0, r0
-; CHECK-MVEFP-NEXT:    vcvtb.f16.f32 s0, s0
-; CHECK-MVEFP-NEXT:    vmov r0, s0
-; CHECK-MVEFP-NEXT:    vmov.16 q6[7], r0
-; CHECK-MVEFP-NEXT:    vmov q0, q6
-; CHECK-MVEFP-NEXT:    add sp, #64
-; CHECK-MVEFP-NEXT:    vpop {d8, d9, d10, d11, d12, d13}
-; CHECK-MVEFP-NEXT:    pop {r4, pc}
+; CHECK-LABEL: pow_float16_t:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
+; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13}
+; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13}
+; CHECK-NEXT:    .pad #64
+; CHECK-NEXT:    sub sp, #64
+; CHECK-NEXT:    vmov q5, q1
+; CHECK-NEXT:    vmov q4, q0
+; CHECK-NEXT:    vmov.u16 r0, q1[0]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[0]
+; CHECK-NEXT:    vmov s2, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s2, s2
+; CHECK-NEXT:    vstr s2, [sp, #56]
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #60]
+; CHECK-NEXT:    ldrd r0, r1, [sp, #56]
+; CHECK-NEXT:    bl powf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q5[1]
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r4, s0
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[1]
+; CHECK-NEXT:    vmov s2, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s2, s2
+; CHECK-NEXT:    vstr s2, [sp, #48]
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #52]
+; CHECK-NEXT:    ldrd r0, r1, [sp, #48]
+; CHECK-NEXT:    bl powf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.16 q6[0], r4
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q6[1], r0
+; CHECK-NEXT:    vmov.u16 r0, q5[2]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[2]
+; CHECK-NEXT:    vmov s2, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s2, s2
+; CHECK-NEXT:    vstr s2, [sp, #40]
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #44]
+; CHECK-NEXT:    ldrd r0, r1, [sp, #40]
+; CHECK-NEXT:    bl powf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q6[2], r0
+; CHECK-NEXT:    vmov.u16 r0, q5[3]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[3]
+; CHECK-NEXT:    vmov s2, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s2, s2
+; CHECK-NEXT:    vstr s2, [sp, #32]
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #36]
+; CHECK-NEXT:    ldrd r0, r1, [sp, #32]
+; CHECK-NEXT:    bl powf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q6[3], r0
+; CHECK-NEXT:    vmov.u16 r0, q5[4]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[4]
+; CHECK-NEXT:    vmov s2, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s2, s2
+; CHECK-NEXT:    vstr s2, [sp, #24]
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #28]
+; CHECK-NEXT:    ldrd r0, r1, [sp, #24]
+; CHECK-NEXT:    bl powf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q6[4], r0
+; CHECK-NEXT:    vmov.u16 r0, q5[5]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[5]
+; CHECK-NEXT:    vmov s2, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s2, s2
+; CHECK-NEXT:    vstr s2, [sp, #16]
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #20]
+; CHECK-NEXT:    ldrd r0, r1, [sp, #16]
+; CHECK-NEXT:    bl powf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q6[5], r0
+; CHECK-NEXT:    vmov.u16 r0, q5[6]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[6]
+; CHECK-NEXT:    vmov s2, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s2, s2
+; CHECK-NEXT:    vstr s2, [sp, #8]
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #12]
+; CHECK-NEXT:    ldrd r0, r1, [sp, #8]
+; CHECK-NEXT:    bl powf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q6[6], r0
+; CHECK-NEXT:    vmov.u16 r0, q5[7]
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vmov.u16 r0, q4[7]
+; CHECK-NEXT:    vmov s2, r0
+; CHECK-NEXT:    vcvtb.f32.f16 s2, s2
+; CHECK-NEXT:    vstr s2, [sp]
+; CHECK-NEXT:    vcvtb.f32.f16 s0, s0
+; CHECK-NEXT:    vstr s0, [sp, #4]
+; CHECK-NEXT:    ldrd r0, r1, [sp]
+; CHECK-NEXT:    bl powf
+; CHECK-NEXT:    vmov s0, r0
+; CHECK-NEXT:    vcvtb.f16.f32 s0, s0
+; CHECK-NEXT:    vmov r0, s0
+; CHECK-NEXT:    vmov.16 q6[7], r0
+; CHECK-NEXT:    vmov q0, q6
+; CHECK-NEXT:    add sp, #64
+; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13}
+; CHECK-NEXT:    pop {r4, pc}
 entry:
   %0 = call fast <8 x half> @llvm.pow.v8f16(<8 x half> %src1, <8 x half> %src2)
   ret <8 x half> %0
 }
 
 declare <4 x float> @llvm.sqrt.v4f32(<4 x float>)
 declare <4 x float> @llvm.cos.v4f32(<4 x float>)
 declare <4 x float> @llvm.sin.v4f32(<4 x float>)

llvm/test/CodeGen/Thumb2/mve-fp-negabs.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-arm-none-eabi -mattr=+mve,+fullfp16 -verify-machineinstrs %s -o - | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-MVE
 ; RUN: llc -mtriple=thumbv8.1m.main-arm-none-eabi -mattr=+mve.fp -verify-machineinstrs %s -o - | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-MVEFP
 
 define arm_aapcs_vfpcc <8 x half> @fneg_float16_t(<8 x half> %src) {
 ; CHECK-MVE-LABEL: fneg_float16_t:
 ; CHECK-MVE:       @ %bb.0: @ %entry
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q0[0]
 ; CHECK-MVE-NEXT:    vmov.u16 r1, q0[1]
 ; CHECK-MVE-NEXT:    vmov s4, r0
-; CHECK-MVE-NEXT:    movs r2, #0
 ; CHECK-MVE-NEXT:    vneg.f16 s4, s4
 ; CHECK-MVE-NEXT:    vmov r0, s4
 ; CHECK-MVE-NEXT:    vmov s4, r1
 ; CHECK-MVE-NEXT:    vneg.f16 s4, s4
 ; CHECK-MVE-NEXT:    vmov r1, s4
-; CHECK-MVE-NEXT:    vdup.16 q1, r2
 ; CHECK-MVE-NEXT:    vmov.16 q1[0], r0
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-MVE-NEXT:    vmov s8, r0
 ; CHECK-MVE-NEXT:    vmov.16 q1[1], r1
+; CHECK-MVE-NEXT:    vmov s8, r0
 ; CHECK-MVE-NEXT:    vneg.f16 s8, s8
 ; CHECK-MVE-NEXT:    vmov r0, s8
 ; CHECK-MVE-NEXT:    vmov.16 q1[2], r0
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q0[3]
 ; CHECK-MVE-NEXT:    vmov s8, r0
 ; CHECK-MVE-NEXT:    vneg.f16 s8, s8
 ; CHECK-MVE-NEXT:    vmov r0, s8
 ; CHECK-MVE-NEXT:    vmov.16 q1[3], r0

 entry:
   %0 = fsub nnan ninf nsz <8 x half> <half 0.0e0, half 0.0e0, half 0.0e0, half 0.0e0, half 0.0e0, half 0.0e0, half 0.0e0, half 0.0e0>, %src
   ret <8 x half> %0
 }
 
 define arm_aapcs_vfpcc <4 x float> @fneg_float32_t(<4 x float> %src) {
 ; CHECK-MVE-LABEL: fneg_float32_t:
 ; CHECK-MVE:       @ %bb.0: @ %entry
+; CHECK-MVE-NEXT:    vneg.f32 s7, s3
+; CHECK-MVE-NEXT:    vneg.f32 s6, s2
+; CHECK-MVE-NEXT:    vneg.f32 s5, s1
 ; CHECK-MVE-NEXT:    vneg.f32 s4, s0
-; CHECK-MVE-NEXT:    movs r0, #0
-; CHECK-MVE-NEXT:    vneg.f32 s6, s1
-; CHECK-MVE-NEXT:    vneg.f32 s8, s2
-; CHECK-MVE-NEXT:    vneg.f32 s10, s3
-; CHECK-MVE-NEXT:    vdup.32 q0, r0
-; CHECK-MVE-NEXT:    vmov.f32 s0, s4
-; CHECK-MVE-NEXT:    vmov.f32 s1, s6
-; CHECK-MVE-NEXT:    vmov.f32 s2, s8
-; CHECK-MVE-NEXT:    vmov.f32 s3, s10
+; CHECK-MVE-NEXT:    vmov q0, q1
 ; CHECK-MVE-NEXT:    bx lr
 ;
 ; CHECK-MVEFP-LABEL: fneg_float32_t:
 ; CHECK-MVEFP:       @ %bb.0: @ %entry
 ; CHECK-MVEFP-NEXT:    vneg.f32 q0, q0
 ; CHECK-MVEFP-NEXT:    bx lr
 entry:
   %0 = fsub nnan ninf nsz <4 x float> <float 0.0e0, float 0.0e0, float 0.0e0, float 0.0e0>, %src
   ret <4 x float> %0
 }
 
 define arm_aapcs_vfpcc <8 x half> @fabs_float16_t(<8 x half> %src) {
 ; CHECK-MVE-LABEL: fabs_float16_t:
 ; CHECK-MVE:       @ %bb.0: @ %entry
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q0[0]
 ; CHECK-MVE-NEXT:    vmov.u16 r1, q0[1]
 ; CHECK-MVE-NEXT:    vmov s4, r0
-; CHECK-MVE-NEXT:    movs r2, #0
 ; CHECK-MVE-NEXT:    vabs.f16 s4, s4
 ; CHECK-MVE-NEXT:    vmov r0, s4
 ; CHECK-MVE-NEXT:    vmov s4, r1
 ; CHECK-MVE-NEXT:    vabs.f16 s4, s4
 ; CHECK-MVE-NEXT:    vmov r1, s4
-; CHECK-MVE-NEXT:    vdup.16 q1, r2
 ; CHECK-MVE-NEXT:    vmov.16 q1[0], r0
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-MVE-NEXT:    vmov s8, r0
 ; CHECK-MVE-NEXT:    vmov.16 q1[1], r1
+; CHECK-MVE-NEXT:    vmov s8, r0
 ; CHECK-MVE-NEXT:    vabs.f16 s8, s8
 ; CHECK-MVE-NEXT:    vmov r0, s8
 ; CHECK-MVE-NEXT:    vmov.16 q1[2], r0
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q0[3]
 ; CHECK-MVE-NEXT:    vmov s8, r0
 ; CHECK-MVE-NEXT:    vabs.f16 s8, s8
 ; CHECK-MVE-NEXT:    vmov r0, s8
 ; CHECK-MVE-NEXT:    vmov.16 q1[3], r0

 entry:
   %0 = call nnan ninf nsz <8 x half> @llvm.fabs.v8f16(<8 x half> %src)
   ret <8 x half> %0
 }
 
 define arm_aapcs_vfpcc <4 x float> @fabs_float32_t(<4 x float> %src) {
 ; CHECK-MVE-LABEL: fabs_float32_t:
 ; CHECK-MVE:       @ %bb.0: @ %entry
+; CHECK-MVE-NEXT:    vabs.f32 s7, s3
+; CHECK-MVE-NEXT:    vabs.f32 s6, s2
+; CHECK-MVE-NEXT:    vabs.f32 s5, s1
 ; CHECK-MVE-NEXT:    vabs.f32 s4, s0
-; CHECK-MVE-NEXT:    movs r0, #0
-; CHECK-MVE-NEXT:    vabs.f32 s6, s1
-; CHECK-MVE-NEXT:    vabs.f32 s8, s2
-; CHECK-MVE-NEXT:    vabs.f32 s10, s3
-; CHECK-MVE-NEXT:    vdup.32 q0, r0
-; CHECK-MVE-NEXT:    vmov.f32 s0, s4
-; CHECK-MVE-NEXT:    vmov.f32 s1, s6
-; CHECK-MVE-NEXT:    vmov.f32 s2, s8
-; CHECK-MVE-NEXT:    vmov.f32 s3, s10
+; CHECK-MVE-NEXT:    vmov q0, q1
 ; CHECK-MVE-NEXT:    bx lr
 ;
 ; CHECK-MVEFP-LABEL: fabs_float32_t:
 ; CHECK-MVEFP:       @ %bb.0: @ %entry
 ; CHECK-MVEFP-NEXT:    vabs.f32 q0, q0
 ; CHECK-MVEFP-NEXT:    bx lr
 entry:
   %0 = call nnan ninf nsz <4 x float> @llvm.fabs.v4f32(<4 x float> %src)
   ret <4 x float> %0
 }
 
 declare <4 x float> @llvm.fabs.v4f32(<4 x float>)
 declare <8 x half> @llvm.fabs.v8f16(<8 x half>)
 

llvm/test/CodeGen/Thumb2/mve-shuffle.ll

 ; CHECK-NEXT:    vrev64.32 q0, q0
 ; CHECK-NEXT:    bx lr
 entry:
   %out = shufflevector <4 x float> %src, <4 x float> undef, <4 x i32> <i32 1, i32 0, i32 3, i32 2>
   ret <4 x float> %out
 }
 
 define arm_aapcs_vfpcc <8 x half> @shuffle1_f16(<8 x half> %src) {
-; CHECK-MVE-LABEL: shuffle1_f16:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    movs r2, #0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[7]
-; CHECK-MVE-NEXT:    vdup.16 q1, r2
-; CHECK-MVE-NEXT:    vmov.u16 r1, q0[6]
-; CHECK-MVE-NEXT:    vmov.16 q1[0], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[5]
-; CHECK-MVE-NEXT:    vmov.16 q1[1], r1
-; CHECK-MVE-NEXT:    vmov.16 q1[2], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[4]
-; CHECK-MVE-NEXT:    vmov.16 q1[3], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[3]
-; CHECK-MVE-NEXT:    vmov.16 q1[4], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-MVE-NEXT:    vmov.16 q1[5], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-MVE-NEXT:    vmov.16 q1[6], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-MVE-NEXT:    vmov.16 q1[7], r0
-; CHECK-MVE-NEXT:    vmov q0, q1
-; CHECK-MVE-NEXT:    bx lr
-;
-; CHECK-MVEFP-LABEL: shuffle1_f16:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[7]
-; CHECK-MVEFP-NEXT:    vmov.u16 r1, q0[6]
-; CHECK-MVEFP-NEXT:    vmov.16 q1[0], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[5]
-; CHECK-MVEFP-NEXT:    vmov.16 q1[1], r1
-; CHECK-MVEFP-NEXT:    vmov.16 q1[2], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[4]
-; CHECK-MVEFP-NEXT:    vmov.16 q1[3], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[3]
-; CHECK-MVEFP-NEXT:    vmov.16 q1[4], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-MVEFP-NEXT:    vmov.16 q1[5], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-MVEFP-NEXT:    vmov.16 q1[6], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-MVEFP-NEXT:    vmov.16 q1[7], r0
-; CHECK-MVEFP-NEXT:    vmov q0, q1
-; CHECK-MVEFP-NEXT:    bx lr
+; CHECK-LABEL: shuffle1_f16:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    vmov.u16 r0, q0[7]
+; CHECK-NEXT:    vmov.u16 r1, q0[6]
+; CHECK-NEXT:    vmov.16 q1[0], r0
+; CHECK-NEXT:    vmov.u16 r0, q0[5]
+; CHECK-NEXT:    vmov.16 q1[1], r1
+; CHECK-NEXT:    vmov.16 q1[2], r0
+; CHECK-NEXT:    vmov.u16 r0, q0[4]
+; CHECK-NEXT:    vmov.16 q1[3], r0
+; CHECK-NEXT:    vmov.u16 r0, q0[3]
+; CHECK-NEXT:    vmov.16 q1[4], r0
+; CHECK-NEXT:    vmov.u16 r0, q0[2]
+; CHECK-NEXT:    vmov.16 q1[5], r0
+; CHECK-NEXT:    vmov.u16 r0, q0[1]
+; CHECK-NEXT:    vmov.16 q1[6], r0
+; CHECK-NEXT:    vmov.u16 r0, q0[0]
+; CHECK-NEXT:    vmov.16 q1[7], r0
+; CHECK-NEXT:    vmov q0, q1
+; CHECK-NEXT:    bx lr
 entry:
   %out = shufflevector <8 x half> %src, <8 x half> undef, <8 x i32> <i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
   ret <8 x half> %out
 }
 
 define arm_aapcs_vfpcc <8 x half> @shuffle2_f16(<8 x half> %src) {
 ; CHECK-LABEL: shuffle2_f16:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    bx lr
 entry:
   %out = shufflevector <8 x half> %src, <8 x half> undef, <8 x i32> <i32 0, i32 1, i32 2, i32 3, i32 4, i32 5, i32 6, i32 7>
   ret <8 x half> %out
 }
 
 define arm_aapcs_vfpcc <8 x half> @shuffle3_f16(<8 x half> %src) {
-; CHECK-MVE-LABEL: shuffle3_f16:
-; CHECK-MVE:       @ %bb.0: @ %entry
-; CHECK-MVE-NEXT:    movs r2, #0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[4]
-; CHECK-MVE-NEXT:    vdup.16 q1, r2
-; CHECK-MVE-NEXT:    vmov.u16 r1, q0[5]
-; CHECK-MVE-NEXT:    vmov.16 q1[0], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[7]
-; CHECK-MVE-NEXT:    vmov.16 q1[1], r1
-; CHECK-MVE-NEXT:    vmov.16 q1[2], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[6]
-; CHECK-MVE-NEXT:    vmov.16 q1[3], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[3]
-; CHECK-MVE-NEXT:    vmov.16 q1[4], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-MVE-NEXT:    vmov.16 q1[5], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-MVE-NEXT:    vmov.16 q1[6], r0
-; CHECK-MVE-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-MVE-NEXT:    vmov.16 q1[7], r0
-; CHECK-MVE-NEXT:    vmov q0, q1
-; CHECK-MVE-NEXT:    bx lr
-;
-; CHECK-MVEFP-LABEL: shuffle3_f16:
-; CHECK-MVEFP:       @ %bb.0: @ %entry
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[4]
-; CHECK-MVEFP-NEXT:    vmov.u16 r1, q0[5]
-; CHECK-MVEFP-NEXT:    vmov.16 q1[0], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[7]
-; CHECK-MVEFP-NEXT:    vmov.16 q1[1], r1
-; CHECK-MVEFP-NEXT:    vmov.16 q1[2], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[6]
-; CHECK-MVEFP-NEXT:    vmov.16 q1[3], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[3]
-; CHECK-MVEFP-NEXT:    vmov.16 q1[4], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[1]
-; CHECK-MVEFP-NEXT:    vmov.16 q1[5], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[2]
-; CHECK-MVEFP-NEXT:    vmov.16 q1[6], r0
-; CHECK-MVEFP-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-MVEFP-NEXT:    vmov.16 q1[7], r0
-; CHECK-MVEFP-NEXT:    vmov q0, q1
-; CHECK-MVEFP-NEXT:    bx lr
+; CHECK-LABEL: shuffle3_f16:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    vmov.u16 r0, q0[4]
+; CHECK-NEXT:    vmov.u16 r1, q0[5]
+; CHECK-NEXT:    vmov.16 q1[0], r0
+; CHECK-NEXT:    vmov.u16 r0, q0[7]
+; CHECK-NEXT:    vmov.16 q1[1], r1
+; CHECK-NEXT:    vmov.16 q1[2], r0
+; CHECK-NEXT:    vmov.u16 r0, q0[6]
+; CHECK-NEXT:    vmov.16 q1[3], r0
+; CHECK-NEXT:    vmov.u16 r0, q0[3]
+; CHECK-NEXT:    vmov.16 q1[4], r0
+; CHECK-NEXT:    vmov.u16 r0, q0[1]
+; CHECK-NEXT:    vmov.16 q1[5], r0
+; CHECK-NEXT:    vmov.u16 r0, q0[2]
+; CHECK-NEXT:    vmov.16 q1[6], r0
+; CHECK-NEXT:    vmov.u16 r0, q0[0]
+; CHECK-NEXT:    vmov.16 q1[7], r0
+; CHECK-NEXT:    vmov q0, q1
+; CHECK-NEXT:    bx lr
 entry:
   %out = shufflevector <8 x half> %src, <8 x half> undef, <8 x i32> <i32 4, i32 5, i32 7, i32 6, i32 3, i32 1, i32 2, i32 0>
   ret <8 x half> %out
 }
 
 define arm_aapcs_vfpcc <8 x half> @shuffle5_f16(<8 x half> %src) {
 ; CHECK-LABEL: shuffle5_f16:
 ; CHECK:       @ %bb.0: @ %entry

llvm/test/CodeGen/Thumb2/mve-simple-arith.ll

 entry:
   %0 = add nsw <4 x i32> %src1, %src2
   ret <4 x i32> %0
 }
 
 define arm_aapcs_vfpcc <4 x float> @add_float32_t(<4 x float> %src1, <4 x float> %src2) {
 ; CHECK-MVE-LABEL: add_float32_t:
 ; CHECK-MVE:       @ %bb.0: @ %entry
+; CHECK-MVE-NEXT:    vadd.f32 s11, s7, s3
+; CHECK-MVE-NEXT:    vadd.f32 s10, s6, s2
+; CHECK-MVE-NEXT:    vadd.f32 s9, s5, s1
 ; CHECK-MVE-NEXT:    vadd.f32 s8, s4, s0
-; CHECK-MVE-NEXT:    movs r0, #0
-; CHECK-MVE-NEXT:    vadd.f32 s10, s5, s1
-; CHECK-MVE-NEXT:    vadd.f32 s12, s6, s2
-; CHECK-MVE-NEXT:    vadd.f32 s4, s7, s3
-; CHECK-MVE-NEXT:    vdup.32 q0, r0
-; CHECK-MVE-NEXT:    vmov.f32 s0, s8
-; CHECK-MVE-NEXT:    vmov.f32 s1, s10
-; CHECK-MVE-NEXT:    vmov.f32 s2, s12
-; CHECK-MVE-NEXT:    vmov.f32 s3, s4
+; CHECK-MVE-NEXT:    vmov q0, q2
 ; CHECK-MVE-NEXT:    bx lr
 ;
 ; CHECK-MVEFP-LABEL: add_float32_t:
 ; CHECK-MVEFP:       @ %bb.0: @ %entry
 ; CHECK-MVEFP-NEXT:    vadd.f32 q0, q1, q0
 ; CHECK-MVEFP-NEXT:    bx lr
 entry:
   %0 = fadd nnan ninf nsz <4 x float> %src2, %src1
   ret <4 x float> %0
 }
 
 define arm_aapcs_vfpcc <8 x half> @add_float16_t(<8 x half> %src1, <8 x half> %src2) {
 ; CHECK-MVE-LABEL: add_float16_t:
 ; CHECK-MVE:       @ %bb.0: @ %entry
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-MVE-NEXT:    vmov.u16 r1, q1[0]
-; CHECK-MVE-NEXT:    vmov s10, r1
 ; CHECK-MVE-NEXT:    vmov.u16 r1, q0[1]
 ; CHECK-MVE-NEXT:    vmov s8, r0
-; CHECK-MVE-NEXT:    movs r2, #0
+; CHECK-MVE-NEXT:    vmov.u16 r0, q1[0]
+; CHECK-MVE-NEXT:    vmov s10, r0
+; CHECK-MVE-NEXT:    vmov.u16 r2, q1[1]
 ; CHECK-MVE-NEXT:    vadd.f16 s8, s10, s8
+; CHECK-MVE-NEXT:    vmov s10, r2
 ; CHECK-MVE-NEXT:    vmov r0, s8
 ; CHECK-MVE-NEXT:    vmov s8, r1
-; CHECK-MVE-NEXT:    vmov.u16 r1, q1[1]
-; CHECK-MVE-NEXT:    vmov s10, r1
 ; CHECK-MVE-NEXT:    vadd.f16 s8, s10, s8
 ; CHECK-MVE-NEXT:    vmov r1, s8
-; CHECK-MVE-NEXT:    vdup.16 q2, r2
 ; CHECK-MVE-NEXT:    vmov.16 q2[0], r0
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q0[2]
+; CHECK-MVE-NEXT:    vmov.16 q2[1], r1
 ; CHECK-MVE-NEXT:    vmov s12, r0
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q1[2]
 ; CHECK-MVE-NEXT:    vmov s14, r0
-; CHECK-MVE-NEXT:    vmov.16 q2[1], r1
 ; CHECK-MVE-NEXT:    vadd.f16 s12, s14, s12
 ; CHECK-MVE-NEXT:    vmov r0, s12
 ; CHECK-MVE-NEXT:    vmov.16 q2[2], r0
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q0[3]
 ; CHECK-MVE-NEXT:    vmov s12, r0
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q1[3]
 ; CHECK-MVE-NEXT:    vmov s14, r0
 ; CHECK-MVE-NEXT:    vadd.f16 s12, s14, s12

 entry:
   %0 = sub nsw <4 x i32> %src2, %src1
   ret <4 x i32> %0
 }
 
 define arm_aapcs_vfpcc <4 x float> @sub_float32_t(<4 x float> %src1, <4 x float> %src2) {
 ; CHECK-MVE-LABEL: sub_float32_t:
 ; CHECK-MVE:       @ %bb.0: @ %entry
+; CHECK-MVE-NEXT:    vsub.f32 s11, s7, s3
+; CHECK-MVE-NEXT:    vsub.f32 s10, s6, s2
+; CHECK-MVE-NEXT:    vsub.f32 s9, s5, s1
 ; CHECK-MVE-NEXT:    vsub.f32 s8, s4, s0
-; CHECK-MVE-NEXT:    movs r0, #0
-; CHECK-MVE-NEXT:    vsub.f32 s10, s5, s1
-; CHECK-MVE-NEXT:    vsub.f32 s12, s6, s2
-; CHECK-MVE-NEXT:    vsub.f32 s4, s7, s3
-; CHECK-MVE-NEXT:    vdup.32 q0, r0
-; CHECK-MVE-NEXT:    vmov.f32 s0, s8
-; CHECK-MVE-NEXT:    vmov.f32 s1, s10
-; CHECK-MVE-NEXT:    vmov.f32 s2, s12
-; CHECK-MVE-NEXT:    vmov.f32 s3, s4
+; CHECK-MVE-NEXT:    vmov q0, q2
 ; CHECK-MVE-NEXT:    bx lr
 ;
 ; CHECK-MVEFP-LABEL: sub_float32_t:
 ; CHECK-MVEFP:       @ %bb.0: @ %entry
 ; CHECK-MVEFP-NEXT:    vsub.f32 q0, q1, q0
 ; CHECK-MVEFP-NEXT:    bx lr
 entry:
   %0 = fsub nnan ninf nsz <4 x float> %src2, %src1
   ret <4 x float> %0
 }
 
 define arm_aapcs_vfpcc <8 x half> @sub_float16_t(<8 x half> %src1, <8 x half> %src2) {
 ; CHECK-MVE-LABEL: sub_float16_t:
 ; CHECK-MVE:       @ %bb.0: @ %entry
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-MVE-NEXT:    vmov.u16 r1, q1[0]
-; CHECK-MVE-NEXT:    vmov s10, r1
 ; CHECK-MVE-NEXT:    vmov.u16 r1, q0[1]
 ; CHECK-MVE-NEXT:    vmov s8, r0
-; CHECK-MVE-NEXT:    movs r2, #0
+; CHECK-MVE-NEXT:    vmov.u16 r0, q1[0]
+; CHECK-MVE-NEXT:    vmov s10, r0
+; CHECK-MVE-NEXT:    vmov.u16 r2, q1[1]
 ; CHECK-MVE-NEXT:    vsub.f16 s8, s10, s8
+; CHECK-MVE-NEXT:    vmov s10, r2
 ; CHECK-MVE-NEXT:    vmov r0, s8
 ; CHECK-MVE-NEXT:    vmov s8, r1
-; CHECK-MVE-NEXT:    vmov.u16 r1, q1[1]
-; CHECK-MVE-NEXT:    vmov s10, r1
 ; CHECK-MVE-NEXT:    vsub.f16 s8, s10, s8
 ; CHECK-MVE-NEXT:    vmov r1, s8
-; CHECK-MVE-NEXT:    vdup.16 q2, r2
 ; CHECK-MVE-NEXT:    vmov.16 q2[0], r0
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q0[2]
+; CHECK-MVE-NEXT:    vmov.16 q2[1], r1
 ; CHECK-MVE-NEXT:    vmov s12, r0
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q1[2]
 ; CHECK-MVE-NEXT:    vmov s14, r0
-; CHECK-MVE-NEXT:    vmov.16 q2[1], r1
 ; CHECK-MVE-NEXT:    vsub.f16 s12, s14, s12
 ; CHECK-MVE-NEXT:    vmov r0, s12
 ; CHECK-MVE-NEXT:    vmov.16 q2[2], r0
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q0[3]
 ; CHECK-MVE-NEXT:    vmov s12, r0
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q1[3]
 ; CHECK-MVE-NEXT:    vmov s14, r0
 ; CHECK-MVE-NEXT:    vsub.f16 s12, s14, s12
entry:
   %0 = mul nsw <4 x i32> %src1, %src2
   ret <4 x i32> %0
 }
 
 define arm_aapcs_vfpcc <8 x half> @mul_float16_t(<8 x half> %src1, <8 x half> %src2) {
 ; CHECK-MVE-LABEL: mul_float16_t:
 ; CHECK-MVE:       @ %bb.0: @ %entry
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q0[0]
-; CHECK-MVE-NEXT:    vmov.u16 r1, q1[0]
-; CHECK-MVE-NEXT:    vmov s10, r1
 ; CHECK-MVE-NEXT:    vmov.u16 r1, q0[1]
 ; CHECK-MVE-NEXT:    vmov s8, r0
-; CHECK-MVE-NEXT:    movs r2, #0
+; CHECK-MVE-NEXT:    vmov.u16 r0, q1[0]
+; CHECK-MVE-NEXT:    vmov s10, r0
+; CHECK-MVE-NEXT:    vmov.u16 r2, q1[1]
 ; CHECK-MVE-NEXT:    vmul.f16 s8, s10, s8
+; CHECK-MVE-NEXT:    vmov s10, r2
 ; CHECK-MVE-NEXT:    vmov r0, s8
 ; CHECK-MVE-NEXT:    vmov s8, r1
-; CHECK-MVE-NEXT:    vmov.u16 r1, q1[1]
-; CHECK-MVE-NEXT:    vmov s10, r1
 ; CHECK-MVE-NEXT:    vmul.f16 s8, s10, s8
 ; CHECK-MVE-NEXT:    vmov r1, s8
-; CHECK-MVE-NEXT:    vdup.16 q2, r2
 ; CHECK-MVE-NEXT:    vmov.16 q2[0], r0
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q0[2]
+; CHECK-MVE-NEXT:    vmov.16 q2[1], r1
 ; CHECK-MVE-NEXT:    vmov s12, r0
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q1[2]
 ; CHECK-MVE-NEXT:    vmov s14, r0
-; CHECK-MVE-NEXT:    vmov.16 q2[1], r1
 ; CHECK-MVE-NEXT:    vmul.f16 s12, s14, s12
 ; CHECK-MVE-NEXT:    vmov r0, s12
 ; CHECK-MVE-NEXT:    vmov.16 q2[2], r0
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q0[3]
 ; CHECK-MVE-NEXT:    vmov s12, r0
 ; CHECK-MVE-NEXT:    vmov.u16 r0, q1[3]
 ; CHECK-MVE-NEXT:    vmov s14, r0
 ; CHECK-MVE-NEXT:    vmul.f16 s12, s14, s12

 entry:
   %0 = fmul nnan ninf nsz <8 x half> %src2, %src1
   ret <8 x half> %0
 }
 
 define arm_aapcs_vfpcc <4 x float> @mul_float32_t(<4 x float> %src1, <4 x float> %src2) {
 ; CHECK-MVE-LABEL: mul_float32_t:
 ; CHECK-MVE:       @ %bb.0: @ %entry
+; CHECK-MVE-NEXT:    vmul.f32 s11, s7, s3
+; CHECK-MVE-NEXT:    vmul.f32 s10, s6, s2
+; CHECK-MVE-NEXT:    vmul.f32 s9, s5, s1
 ; CHECK-MVE-NEXT:    vmul.f32 s8, s4, s0
-; CHECK-MVE-NEXT:    movs r0, #0
-; CHECK-MVE-NEXT:    vmul.f32 s10, s5, s1
-; CHECK-MVE-NEXT:    vmul.f32 s12, s6, s2
-; CHECK-MVE-NEXT:    vmul.f32 s4, s7, s3
-; CHECK-MVE-NEXT:    vdup.32 q0, r0
-; CHECK-MVE-NEXT:    vmov.f32 s0, s8
-; CHECK-MVE-NEXT:    vmov.f32 s1, s10
-; CHECK-MVE-NEXT:    vmov.f32 s2, s12
-; CHECK-MVE-NEXT:    vmov.f32 s3, s4
+; CHECK-MVE-NEXT:    vmov q0, q2
 ; CHECK-MVE-NEXT:    bx lr
 ;
 ; CHECK-MVEFP-LABEL: mul_float32_t:
 ; CHECK-MVEFP:       @ %bb.0: @ %entry
 ; CHECK-MVEFP-NEXT:    vmul.f32 q0, q1, q0
 ; CHECK-MVEFP-NEXT:    bx lr
 entry:
   %0 = fmul nnan ninf nsz <4 x float> %src2, %src1
   ret <4 x float> %0
 }

llvm/test/CodeGen/Thumb2/mve-soft-float-abi.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=thumbv8.1m.main-none-eabi -mattr=+mve -o - %s | FileCheck %s
+; RUN: llc -mtriple=thumbv8.1m.main-none-eabi -mattr=+mve.fp -o - %s | FileCheck --check-prefix=CHECK --check-prefix=CHECK-FP %s
+
+define <16 x i8> @vector_add_i8(<16 x i8> %lhs, <16 x i8> %rhs) {
+; CHECK-LABEL: vector_add_i8:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    vmov d1, r2, r3
+; CHECK-NEXT:    vmov d0, r0, r1
+; CHECK-NEXT:    mov r0, sp
+; CHECK-NEXT:    vldrw.u32 q1, [r0]
+; CHECK-NEXT:    vadd.i8 q0, q0, q1
+; CHECK-NEXT:    vmov r0, r1, d0
+; CHECK-NEXT:    vmov r2, r3, d1
+; CHECK-NEXT:    bx lr
+entry:
+  %sum = add <16 x i8> %lhs, %rhs
+  ret <16 x i8> %sum
+}
+
+define <8 x i16> @vector_add_i16(<8 x i16> %lhs, <8 x i16> %rhs) {
+; CHECK-LABEL: vector_add_i16:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    vmov d1, r2, r3
+; CHECK-NEXT:    vmov d0, r0, r1
+; CHECK-NEXT:    mov r0, sp
+; CHECK-NEXT:    vldrw.u32 q1, [r0]
+; CHECK-NEXT:    vadd.i16 q0, q0, q1
+; CHECK-NEXT:    vmov r0, r1, d0
+; CHECK-NEXT:    vmov r2, r3, d1
+; CHECK-NEXT:    bx lr
+entry:
+  %sum = add <8 x i16> %lhs, %rhs
+  ret <8 x i16> %sum
+}
+
+define <4 x i32> @vector_add_i32(<4 x i32> %lhs, <4 x i32> %rhs) {
+; CHECK-LABEL: vector_add_i32:
+; CHECK:       @ %bb.0: @ %entry
+; CHECK-NEXT:    vmov d1, r2, r3
+; CHECK-NEXT:    vmov d0, r0, r1
+; CHECK-NEXT:    mov r0, sp
+; CHECK-NEXT:    vldrw.u32 q1, [r0]
+; CHECK-NEXT:    vadd.i32 q0, q0, q1
+; CHECK-NEXT:    vmov r0, r1, d0
+; CHECK-NEXT:    vmov r2, r3, d1
+; CHECK-NEXT:    bx lr
+entry:
+  %sum = add <4 x i32> %lhs, %rhs
+  ret <4 x i32> %sum
+}
+
+define <8 x half> @vector_add_f16(<8 x half> %lhs, <8 x half> %rhs) {
+; CHECK-FP-LABEL: vector_add_f16:
+; CHECK-FP:       @ %bb.0: @ %entry
+; CHECK-FP-NEXT:    vmov d1, r2, r3
+; CHECK-FP-NEXT:    vmov d0, r0, r1
+; CHECK-FP-NEXT:    mov r0, sp
+; CHECK-FP-NEXT:    vldrw.u32 q1, [r0]
+; CHECK-FP-NEXT:    vadd.f16 q0, q0, q1
+; CHECK-FP-NEXT:    vmov r0, r1, d0
+; CHECK-FP-NEXT:    vmov r2, r3, d1
+; CHECK-FP-NEXT:    bx lr
+entry:
+  %sum = fadd <8 x half> %lhs, %rhs
+  ret <8 x half> %sum
+}
+
+define <4 x float> @vector_add_f32(<4 x float> %lhs, <4 x float> %rhs) {
+; CHECK-FP-LABEL: vector_add_f32:
+; CHECK-FP:       @ %bb.0: @ %entry
+; CHECK-FP-NEXT:    vmov d1, r2, r3
+; CHECK-FP-NEXT:    vmov d0, r0, r1
+; CHECK-FP-NEXT:    mov r0, sp
+; CHECK-FP-NEXT:    vldrw.u32 q1, [r0]
+; CHECK-FP-NEXT:    vadd.f32 q0, q0, q1
+; CHECK-FP-NEXT:    vmov r0, r1, d0
+; CHECK-FP-NEXT:    vmov r2, r3, d1
+; CHECK-FP-NEXT:    bx lr
+entry:
+  %sum = fadd <4 x float> %lhs, %rhs
+  ret <4 x float> %sum
+}