[AArch64] Do not promote f16 when subtarget HasFullFP16

[SjoerdMeijer]

Summary

ARMv8.2a adds FP16 support , i.e. f16 is not only a storage-only type, but it also supports performing data processing on 16-bit floating-point quantities. All
all the necessary groundwork of adding the ARMv8.2-A FP16 (scalar) instructions was done D15014. To take advantage of this, we do not want to promote f16 to f32 when the subtarget supports FullFP16, which will allow instruction selection of these FP16 instructions.

[samparker]

Hi Sjoerd,

These changes look fine to me, but don't we need to test how the backend handles immediate values in these cases?

cheers,
sam

[SjoerdMeijer]

Probably the subject/description of this patch is a bit confusing/misleading. I should have mentioned in the description that all the groundwork of adding the ARMv8.2-A FP16 (scalar) instructions was done D15014, which added all the instructions, and also e.g. an fpimm16 immediate type, and the required regression tests for these instructions. This patch is just a tweak to avoid promotions for FP16 instructions that support f16 operands.

[olista01]

I think we could still do with tests for how half-precision constants are code-generated from IR (using FMOV or constant pools). I don't see any existing tests for that.

There are also a few classes of instruction where we still promote to 32-bit float, but it looks like we have the instructions to do them directly in half-precision. If they turn out to be more complicated that changing a setOperationAction call, then it would make sense to do them as separate patches though.

test/CodeGen/AArch64/f16-instructions.ll

517

Could this be selected as "fcvtzs w0, h0"?

940

Could this be done without the FCVTs, by changing the constant?

[SjoerdMeijer]

Hi Sam, Oliver, thanks for checking and reviewing. Your gut feelings were right: some more testing showed that target hook isFPImmLegal was not allowing f16. So I've modified that function, and added more tests.

[samparker]

test/CodeGen/AArch64/f16-imm.ll

3

A pedantic native speaker may point out that's its 'illegal'... ;)

22

I think it would be a good idea to add the tests that cover the edge cases too, so that the minimal and maximum values are shown to be accepted correctly.

[SjoerdMeijer]

I forgot to address some of Oliver's earlier comments:

• I have added fixes for the conversion functions, and modified the tests,
• I will address the codegen for copysign in a follow up patch, because it it requires a bit of custom lowering.

About the newly added immediate tests:

• I have added (even) more tests. Edge cases is a bit meaningless for 8-bit fp immediates, because there are many (the ARMARM shows a table with accepted immediates), but this now covers minimum/maximum values.
• I have fixed the spell error, just only for those pedantic natives. ;-)

[olista01]

LGTM.

lib/Target/AArch64/AArch64ISelLowering.cpp

AArch64TargetLowering::AArch64TargetLowering(const TargetMachine &TM,
   // Compute derived properties from the register classes
   computeRegisterProperties(Subtarget->getRegisterInfo());
 
   // Provide all sorts of operation actions
   setOperationAction(ISD::GlobalAddress, MVT::i64, Custom);
   setOperationAction(ISD::GlobalTLSAddress, MVT::i64, Custom);
   setOperationAction(ISD::SETCC, MVT::i32, Custom);
   setOperationAction(ISD::SETCC, MVT::i64, Custom);
+  setOperationAction(ISD::SETCC, MVT::f16, Custom);
   setOperationAction(ISD::SETCC, MVT::f32, Custom);
   setOperationAction(ISD::SETCC, MVT::f64, Custom);
   setOperationAction(ISD::BITREVERSE, MVT::i32, Legal);
   setOperationAction(ISD::BITREVERSE, MVT::i64, Legal);
   setOperationAction(ISD::BRCOND, MVT::Other, Expand);
   setOperationAction(ISD::BR_CC, MVT::i32, Custom);
   setOperationAction(ISD::BR_CC, MVT::i64, Custom);
+  setOperationAction(ISD::BR_CC, MVT::f16, Custom);
   setOperationAction(ISD::BR_CC, MVT::f32, Custom);
   setOperationAction(ISD::BR_CC, MVT::f64, Custom);
   setOperationAction(ISD::SELECT, MVT::i32, Custom);
   setOperationAction(ISD::SELECT, MVT::i64, Custom);
+  setOperationAction(ISD::SELECT, MVT::f16, Custom);
   setOperationAction(ISD::SELECT, MVT::f32, Custom);
   setOperationAction(ISD::SELECT, MVT::f64, Custom);
   setOperationAction(ISD::SELECT_CC, MVT::i32, Custom);
   setOperationAction(ISD::SELECT_CC, MVT::i64, Custom);
+  setOperationAction(ISD::SELECT_CC, MVT::f16, Custom);
   setOperationAction(ISD::SELECT_CC, MVT::f32, Custom);
   setOperationAction(ISD::SELECT_CC, MVT::f64, Custom);
   setOperationAction(ISD::BR_JT, MVT::Other, Expand);
   setOperationAction(ISD::JumpTable, MVT::i64, Custom);
 
   setOperationAction(ISD::SHL_PARTS, MVT::i64, Custom);
   setOperationAction(ISD::SRA_PARTS, MVT::i64, Custom);
   setOperationAction(ISD::SRL_PARTS, MVT::i64, Custom);
AArch64TargetLowering::AArch64TargetLowering(const TargetMachine &TM,
   setOperationAction(ISD::FSIN, MVT::f64, Expand);
   setOperationAction(ISD::FCOS, MVT::f32, Expand);
   setOperationAction(ISD::FCOS, MVT::f64, Expand);
   setOperationAction(ISD::FPOW, MVT::f32, Expand);
   setOperationAction(ISD::FPOW, MVT::f64, Expand);
   setOperationAction(ISD::FCOPYSIGN, MVT::f64, Custom);
   setOperationAction(ISD::FCOPYSIGN, MVT::f32, Custom);
 
-  // f16 is a storage-only type, always promote it to f32.
+  setOperationAction(ISD::FREM,        MVT::f16,  Promote);
+  setOperationAction(ISD::FPOW,        MVT::f16,  Promote);
+  setOperationAction(ISD::FPOWI,       MVT::f16,  Promote);
+  setOperationAction(ISD::FCOS,        MVT::f16,  Promote);
+  setOperationAction(ISD::FSIN,        MVT::f16,  Promote);
+  setOperationAction(ISD::FSINCOS,     MVT::f16,  Promote);
+  setOperationAction(ISD::FEXP,        MVT::f16,  Promote);
+  setOperationAction(ISD::FEXP2,       MVT::f16,  Promote);
+  setOperationAction(ISD::FLOG,        MVT::f16,  Promote);
+  setOperationAction(ISD::FLOG2,       MVT::f16,  Promote);
+  setOperationAction(ISD::FLOG10,      MVT::f16,  Promote);
+  setOperationAction(ISD::FCOPYSIGN,   MVT::f16,  Promote);
+
+  if (!Subtarget->hasFullFP16()) {
+    setOperationAction(ISD::SELECT,      MVT::f16,  Promote);
+    setOperationAction(ISD::SELECT_CC,   MVT::f16,  Promote);
     setOperationAction(ISD::SETCC,       MVT::f16,  Promote);
     setOperationAction(ISD::BR_CC,       MVT::f16,  Promote);
-  setOperationAction(ISD::SELECT_CC,   MVT::f16,  Promote);
-  setOperationAction(ISD::SELECT,      MVT::f16,  Promote);
     setOperationAction(ISD::FADD,        MVT::f16,  Promote);
     setOperationAction(ISD::FSUB,        MVT::f16,  Promote);
     setOperationAction(ISD::FMUL,        MVT::f16,  Promote);
     setOperationAction(ISD::FDIV,        MVT::f16,  Promote);
     setOperationAction(ISD::FREM,        MVT::f16,  Promote);
     setOperationAction(ISD::FMA,         MVT::f16,  Promote);
     setOperationAction(ISD::FNEG,        MVT::f16,  Promote);
     setOperationAction(ISD::FABS,        MVT::f16,  Promote);
     setOperationAction(ISD::FCEIL,       MVT::f16,  Promote);
-  setOperationAction(ISD::FCOPYSIGN,   MVT::f16,  Promote);
+    setOperationAction(ISD::FSQRT,       MVT::f16,  Promote);
     setOperationAction(ISD::FCOS,        MVT::f16,  Promote);
     setOperationAction(ISD::FFLOOR,      MVT::f16,  Promote);
     setOperationAction(ISD::FNEARBYINT,  MVT::f16,  Promote);
-  setOperationAction(ISD::FPOW,        MVT::f16,  Promote);
-  setOperationAction(ISD::FPOWI,       MVT::f16,  Promote);
     setOperationAction(ISD::FRINT,       MVT::f16,  Promote);
-  setOperationAction(ISD::FSIN,        MVT::f16,  Promote);
-  setOperationAction(ISD::FSINCOS,     MVT::f16,  Promote);
-  setOperationAction(ISD::FSQRT,       MVT::f16,  Promote);
-  setOperationAction(ISD::FEXP,        MVT::f16,  Promote);
-  setOperationAction(ISD::FEXP2,       MVT::f16,  Promote);
-  setOperationAction(ISD::FLOG,        MVT::f16,  Promote);
-  setOperationAction(ISD::FLOG2,       MVT::f16,  Promote);
-  setOperationAction(ISD::FLOG10,      MVT::f16,  Promote);
     setOperationAction(ISD::FROUND,      MVT::f16,  Promote);
     setOperationAction(ISD::FTRUNC,      MVT::f16,  Promote);
     setOperationAction(ISD::FMINNUM,     MVT::f16,  Promote);
     setOperationAction(ISD::FMAXNUM,     MVT::f16,  Promote);
     setOperationAction(ISD::FMINNAN,     MVT::f16,  Promote);
     setOperationAction(ISD::FMAXNAN,     MVT::f16,  Promote);
+  }
 
   // v4f16 is also a storage-only type, so promote it to v4f32 when that is
   // known to be safe.
   setOperationAction(ISD::FADD, MVT::v4f16, Promote);
   setOperationAction(ISD::FSUB, MVT::v4f16, Promote);
   setOperationAction(ISD::FMUL, MVT::v4f16, Promote);
   setOperationAction(ISD::FDIV, MVT::v4f16, Promote);
   setOperationAction(ISD::FP_EXTEND, MVT::v4f16, Promote);
for (MVT Ty : {MVT::f32, MVT::f64}) {
     setOperationAction(ISD::FTRUNC, Ty, Legal);
     setOperationAction(ISD::FROUND, Ty, Legal);
     setOperationAction(ISD::FMINNUM, Ty, Legal);
     setOperationAction(ISD::FMAXNUM, Ty, Legal);
     setOperationAction(ISD::FMINNAN, Ty, Legal);
     setOperationAction(ISD::FMAXNAN, Ty, Legal);
   }
 
+  if (Subtarget->hasFullFP16()) {
+    setOperationAction(ISD::FNEARBYINT, MVT::f16, Legal);
+    setOperationAction(ISD::FFLOOR,  MVT::f16, Legal);
+    setOperationAction(ISD::FCEIL,   MVT::f16, Legal);
+    setOperationAction(ISD::FRINT,   MVT::f16, Legal);
+    setOperationAction(ISD::FTRUNC,  MVT::f16, Legal);
+    setOperationAction(ISD::FROUND,  MVT::f16, Legal);
+    setOperationAction(ISD::FMINNUM, MVT::f16, Legal);
+    setOperationAction(ISD::FMAXNUM, MVT::f16, Legal);
+    setOperationAction(ISD::FMINNAN, MVT::f16, Legal);
+    setOperationAction(ISD::FMAXNAN, MVT::f16, Legal);
+  }
+
   setOperationAction(ISD::PREFETCH, MVT::Other, Custom);
 
   setOperationAction(ISD::ATOMIC_CMP_SWAP, MVT::i128, Custom);
 
   // Lower READCYCLECOUNTER using an mrs from PMCCNTR_EL0.
   // This requires the Performance Monitors extension.
   if (Subtarget->hasPerfMon())
     setOperationAction(ISD::READCYCLECOUNTER, MVT::i64, Legal);
void AArch64TargetLowering::addTypeForNEON(MVT VT, MVT PromotedBitwiseVT) {
   if (!VT.isFloatingPoint())
     setOperationAction(ISD::ABS, VT, Legal);
 
   // [SU][MIN|MAX] are available for all NEON types apart from i64.
   if (!VT.isFloatingPoint() && VT != MVT::v2i64 && VT != MVT::v1i64)
     for (unsigned Opcode : {ISD::SMIN, ISD::SMAX, ISD::UMIN, ISD::UMAX})
       setOperationAction(Opcode, VT, Legal);
 
-  // F[MIN|MAX][NUM|NAN] are available for all FP NEON types (not f16 though!).
-  if (VT.isFloatingPoint() && VT.getVectorElementType() != MVT::f16)
+  // F[MIN|MAX][NUM|NAN] are available for all FP NEON types.
+  if (VT.isFloatingPoint() &&
+      (VT.getVectorElementType() != MVT::f16 || Subtarget->hasFullFP16()))
     for (unsigned Opcode : {ISD::FMINNAN, ISD::FMAXNAN,
                             ISD::FMINNUM, ISD::FMAXNUM})
       setOperationAction(Opcode, VT, Legal);
 
   if (Subtarget->isLittleEndian()) {
     for (unsigned im = (unsigned)ISD::PRE_INC;
          im != (unsigned)ISD::LAST_INDEXED_MODE; ++im) {
       setIndexedLoadAction(im, VT, Legal);

 }
 
 static bool isLegalArithImmed(uint64_t C) {
   // Matches AArch64DAGToDAGISel::SelectArithImmed().
   return (C >> 12 == 0) || ((C & 0xFFFULL) == 0 && C >> 24 == 0);
 }
 
 static SDValue emitComparison(SDValue LHS, SDValue RHS, ISD::CondCode CC,
-                              const SDLoc &dl, SelectionDAG &DAG) {
+                              const SDLoc &dl, SelectionDAG &DAG,
+                              bool FullFP16 = false) {
   EVT VT = LHS.getValueType();
 
   if (VT.isFloatingPoint()) {
     assert(VT != MVT::f128);
-    if (VT == MVT::f16) {
+    if (VT == MVT::f16 && !FullFP16) {
       LHS = DAG.getNode(ISD::FP_EXTEND, dl, MVT::f32, LHS);
       RHS = DAG.getNode(ISD::FP_EXTEND, dl, MVT::f32, RHS);
       VT = MVT::f32;
     }
     return DAG.getNode(AArch64ISD::FCMP, dl, VT, LHS, RHS);
   }
 
   // The CMP instruction is just an alias for SUBS, and representing it as

 /// by conditional compare sequences.
 /// @{
 
 /// Create a conditional comparison; Use CCMP, CCMN or FCCMP as appropriate.
 static SDValue emitConditionalComparison(SDValue LHS, SDValue RHS,
                                          ISD::CondCode CC, SDValue CCOp,
                                          AArch64CC::CondCode Predicate,
                                          AArch64CC::CondCode OutCC,
-                                         const SDLoc &DL, SelectionDAG &DAG) {
+                                         const SDLoc &DL, SelectionDAG &DAG,
+                                         bool FullFP16 = false) {
   unsigned Opcode = 0;
   if (LHS.getValueType().isFloatingPoint()) {
     assert(LHS.getValueType() != MVT::f128);
-    if (LHS.getValueType() == MVT::f16) {
+    if (LHS.getValueType() == MVT::f16 && !FullFP16) {
       LHS = DAG.getNode(ISD::FP_EXTEND, DL, MVT::f32, LHS);
       RHS = DAG.getNode(ISD::FP_EXTEND, DL, MVT::f32, RHS);
     }
     Opcode = AArch64ISD::FCCMP;
   } else if (RHS.getOpcode() == ISD::SUB) {
     SDValue SubOp0 = RHS.getOperand(0);
     if (isNullConstant(SubOp0) && (CC == ISD::SETEQ || CC == ISD::SETNE)) {
       // See emitComparison() on why we can only do this for SETEQ and SETNE.

 /// and sets @p OutCC to the flags that should be tested or returns SDValue() if
 /// transformation was not possible.
 /// On recursive invocations @p PushNegate may be set to true to have negation
 /// effects pushed to the tree leafs; @p Predicate is an NZCV flag predicate
 /// for the comparisons in the current subtree; @p Depth limits the search
 /// depth to avoid stack overflow.
 static SDValue emitConjunctionDisjunctionTreeRec(SelectionDAG &DAG, SDValue Val,
     AArch64CC::CondCode &OutCC, bool Negate, SDValue CCOp,
-    AArch64CC::CondCode Predicate) {
+    AArch64CC::CondCode Predicate, bool FullFP16 = false) {
   // We're at a tree leaf, produce a conditional comparison operation.
   unsigned Opcode = Val->getOpcode();
   if (Opcode == ISD::SETCC) {
     SDValue LHS = Val->getOperand(0);
     SDValue RHS = Val->getOperand(1);
     ISD::CondCode CC = cast<CondCodeSDNode>(Val->getOperand(2))->get();
     bool isInteger = LHS.getValueType().isInteger();
     if (Negate)
if (isInteger) {
       // Some floating point conditions can't be tested with a single condition
       // code. Construct an additional comparison in this case.
       if (ExtraCC != AArch64CC::AL) {
         SDValue ExtraCmp;
         if (!CCOp.getNode())
           ExtraCmp = emitComparison(LHS, RHS, CC, DL, DAG);
         else
           ExtraCmp = emitConditionalComparison(LHS, RHS, CC, CCOp, Predicate,
-                                               ExtraCC, DL, DAG);
+                                               ExtraCC, DL, DAG,
+                                               FullFP16);
         CCOp = ExtraCmp;
         Predicate = ExtraCC;
       }
     }
 
     // Produce a normal comparison if we are first in the chain
     if (!CCOp)
       return emitComparison(LHS, RHS, CC, DL, DAG);
     // Otherwise produce a ccmp.
     return emitConditionalComparison(LHS, RHS, CC, CCOp, Predicate, OutCC, DL,
-                                     DAG);
+                                     DAG, FullFP16);
   }
   assert((Opcode == ISD::AND || (Opcode == ISD::OR && Val->hasOneUse())) &&
          "Valid conjunction/disjunction tree");
 
   // Check if both sides can be transformed.
   SDValue LHS = Val->getOperand(0);
   SDValue RHS = Val->getOperand(1);
 
if (NeedsNegOutL)
       std::swap(LHS, RHS);
   }
 
   // Emit RHS. If we want to negate the tree we only need to push a negate
   // through if we are already in a PushNegate case, otherwise we can negate
   // the "flags to test" afterwards.
   AArch64CC::CondCode RHSCC;
   SDValue CmpR = emitConjunctionDisjunctionTreeRec(DAG, RHS, RHSCC, Negate,
-                                                   CCOp, Predicate);
+                                                   CCOp, Predicate, FullFP16);
   if (NegateOpsAndResult && !Negate)
     RHSCC = AArch64CC::getInvertedCondCode(RHSCC);
   // Emit LHS. We may need to negate it.
   SDValue CmpL = emitConjunctionDisjunctionTreeRec(DAG, LHS, OutCC,
                                                    NegateOpsAndResult, CmpR,
-                                                   RHSCC);
+                                                   RHSCC, FullFP16);
   // If we transformed an OR to and AND then we have to negate the result
   // (or absorb the Negate parameter).
   if (NegateOpsAndResult && !Negate)
     OutCC = AArch64CC::getInvertedCondCode(OutCC);
   return CmpL;
 }
 
 /// Emit conjunction or disjunction tree with the CMP/FCMP followed by a chain
 /// of CCMP/CFCMP ops. See @ref AArch64CCMP.
 /// \see emitConjunctionDisjunctionTreeRec().
 static SDValue emitConjunctionDisjunctionTree(SelectionDAG &DAG, SDValue Val,
-                                              AArch64CC::CondCode &OutCC) {
+                                              AArch64CC::CondCode &OutCC,
+                                              bool FullFP16 = false) {
   bool CanNegate;
   if (!isConjunctionDisjunctionTree(Val, CanNegate))
     return SDValue();
 
   return emitConjunctionDisjunctionTreeRec(DAG, Val, OutCC, false, SDValue(),
-                                           AArch64CC::AL);
+                                           AArch64CC::AL, FullFP16);
 }
 
 /// @}
 
 static SDValue getAArch64Cmp(SDValue LHS, SDValue RHS, ISD::CondCode CC,
                              SDValue &AArch64cc, SelectionDAG &DAG,
-                             const SDLoc &dl) {
+                             const SDLoc &dl, bool FullFP16 = false) {
   if (ConstantSDNode *RHSC = dyn_cast<ConstantSDNode>(RHS.getNode())) {
     EVT VT = RHS.getValueType();
     uint64_t C = RHSC->getZExtValue();
     if (!isLegalArithImmed(C)) {
       // Constant does not fit, try adjusting it by one?
       switch (CC) {
       default:
         break;
if ((RHSC->getZExtValue() >> 16 == 0) && isa<LoadSDNode>(LHS) &&
         LHS.getNode()->hasNUsesOfValue(1, 0)) {
       int16_t ValueofRHS = cast<ConstantSDNode>(RHS)->getZExtValue();
       if (ValueofRHS < 0 && isLegalArithImmed(-ValueofRHS)) {
         SDValue SExt =
             DAG.getNode(ISD::SIGN_EXTEND_INREG, dl, LHS.getValueType(), LHS,
                         DAG.getValueType(MVT::i16));
         Cmp = emitComparison(SExt, DAG.getConstant(ValueofRHS, dl,
                                                    RHS.getValueType()),
-                             CC, dl, DAG);
+                             CC, dl, DAG, FullFP16);
         AArch64CC = changeIntCCToAArch64CC(CC);
       }
     }
 
     if (!Cmp && (RHSC->isNullValue() || RHSC->isOne())) {
-      if ((Cmp = emitConjunctionDisjunctionTree(DAG, LHS, AArch64CC))) {
+      if ((Cmp = emitConjunctionDisjunctionTree(DAG, LHS, AArch64CC, FullFP16))) {
         if ((CC == ISD::SETNE) ^ RHSC->isNullValue())
           AArch64CC = AArch64CC::getInvertedCondCode(AArch64CC);
       }
     }
   }
 
   if (!Cmp) {
-    Cmp = emitComparison(LHS, RHS, CC, dl, DAG);
+    Cmp = emitComparison(LHS, RHS, CC, dl, DAG, FullFP16);
     AArch64CC = changeIntCCToAArch64CC(CC);
   }
   AArch64cc = DAG.getConstant(AArch64CC, dl, MVT_CC);
   return Cmp;
 }
 
 static std::pair<SDValue, SDValue>
 getAArch64XALUOOp(AArch64CC::CondCode &CC, SDValue Op, SelectionDAG &DAG) {
static SDValue LowerVectorFP_TO_INT(SDValue Op, SelectionDAG &DAG) {
   return Op;
 }
 
 SDValue AArch64TargetLowering::LowerFP_TO_INT(SDValue Op,
                                               SelectionDAG &DAG) const {
   if (Op.getOperand(0).getValueType().isVector())
     return LowerVectorFP_TO_INT(Op, DAG);
 
-  // f16 conversions are promoted to f32.
-  if (Op.getOperand(0).getValueType() == MVT::f16) {
+  // f16 conversions are promoted to f32 when full fp16 is not supported.
+  if (Op.getOperand(0).getValueType() == MVT::f16 &&
+      !Subtarget->hasFullFP16()) {
     SDLoc dl(Op);
     return DAG.getNode(
         Op.getOpcode(), dl, Op.getValueType(),
         DAG.getNode(ISD::FP_EXTEND, dl, MVT::f32, Op.getOperand(0)));
   }
 
   if (Op.getOperand(0).getValueType() != MVT::f128) {
     // It's legal except when f128 is involved
static SDValue LowerVectorINT_TO_FP(SDValue Op, SelectionDAG &DAG) {
   return Op;
 }
 
 SDValue AArch64TargetLowering::LowerINT_TO_FP(SDValue Op,
                                             SelectionDAG &DAG) const {
   if (Op.getValueType().isVector())
     return LowerVectorINT_TO_FP(Op, DAG);
 
-  // f16 conversions are promoted to f32.
-  if (Op.getValueType() == MVT::f16) {
+  // f16 conversions are promoted to f32 when full fp16 is not supported.
+  if (Op.getValueType() == MVT::f16 &&
+      !Subtarget->hasFullFP16()) {
     SDLoc dl(Op);
     return DAG.getNode(
         ISD::FP_ROUND, dl, MVT::f16,
         DAG.getNode(Op.getOpcode(), dl, MVT::f32, Op.getOperand(0)),
         DAG.getIntPtrConstant(0, dl));
   }
 
   // i128 conversions are libcalls.
if (RHSC && RHSC->getSExtValue() == -1 && CC == ISD::SETGT &&
       // (a.k.a. TST) and the test in the test bit and branch instruction
       // becomes redundant.  This would also increase register pressure.
       uint64_t Mask = LHS.getValueSizeInBits() - 1;
       return DAG.getNode(AArch64ISD::TBZ, dl, MVT::Other, Chain, LHS,
                          DAG.getConstant(Mask, dl, MVT::i64), Dest);
     }
 
     SDValue CCVal;
-    SDValue Cmp = getAArch64Cmp(LHS, RHS, CC, CCVal, DAG, dl);
+    SDValue Cmp = getAArch64Cmp(LHS, RHS, CC, CCVal, DAG, dl,
+                                Subtarget->hasFullFP16());
     return DAG.getNode(AArch64ISD::BRCOND, dl, MVT::Other, Chain, Dest, CCVal,
                        Cmp);
   }
 
-  assert(LHS.getValueType() == MVT::f32 || LHS.getValueType() == MVT::f64);
+  assert(LHS.getValueType() == MVT::f16 || LHS.getValueType() == MVT::f32 ||
+         LHS.getValueType() == MVT::f64);
 
   // Unfortunately, the mapping of LLVM FP CC's onto AArch64 CC's isn't totally
   // clean.  Some of them require two branches to implement.
-  SDValue Cmp = emitComparison(LHS, RHS, CC, dl, DAG);
+  SDValue Cmp = emitComparison(LHS, RHS, CC, dl, DAG, Subtarget->hasFullFP16());
   AArch64CC::CondCode CC1, CC2;
   changeFPCCToAArch64CC(CC, CC1, CC2);
   SDValue CC1Val = DAG.getConstant(CC1, dl, MVT::i32);
   SDValue BR1 =
       DAG.getNode(AArch64ISD::BRCOND, dl, MVT::Other, Chain, Dest, CC1Val, Cmp);
   if (CC2 != AArch64CC::AL) {
     SDValue CC2Val = DAG.getConstant(CC2, dl, MVT::i32);
     return DAG.getNode(AArch64ISD::BRCOND, dl, MVT::Other, BR1, Dest, CC2Val,
if (!RHS.getNode()) {
              "Unexpected setcc expansion!");
       return LHS;
     }
   }
 
   if (LHS.getValueType().isInteger()) {
     SDValue CCVal;
     SDValue Cmp =
-        getAArch64Cmp(LHS, RHS, ISD::getSetCCInverse(CC, true), CCVal, DAG, dl);
+        getAArch64Cmp(LHS, RHS, ISD::getSetCCInverse(CC, true), CCVal, DAG, dl,
+                      Subtarget->hasFullFP16());
 
     // Note that we inverted the condition above, so we reverse the order of
     // the true and false operands here.  This will allow the setcc to be
     // matched to a single CSINC instruction.
     return DAG.getNode(AArch64ISD::CSEL, dl, VT, FVal, TVal, CCVal, Cmp);
   }
 
   // Now we know we're dealing with FP values.
-  assert(LHS.getValueType() == MVT::f32 || LHS.getValueType() == MVT::f64);
+  assert(LHS.getValueType() == MVT::f16 || LHS.getValueType() == MVT::f32 ||
+         LHS.getValueType() == MVT::f64);
 
   // If that fails, we'll need to perform an FCMP + CSEL sequence.  Go ahead
   // and do the comparison.
-  SDValue Cmp = emitComparison(LHS, RHS, CC, dl, DAG);
+  SDValue Cmp = emitComparison(LHS, RHS, CC, dl, DAG, Subtarget->hasFullFP16());
 
   AArch64CC::CondCode CC1, CC2;
   changeFPCCToAArch64CC(CC, CC1, CC2);
   if (CC2 == AArch64CC::AL) {
     changeFPCCToAArch64CC(ISD::getSetCCInverse(CC, false), CC1, CC2);
     SDValue CC1Val = DAG.getConstant(CC1, dl, MVT::i32);
 
     // Note that we inverted the condition above, so we reverse the order of
if (LHS.getValueType() == MVT::f128) {
     // against zero to select between true and false values.
     if (!RHS.getNode()) {
       RHS = DAG.getConstant(0, dl, LHS.getValueType());
       CC = ISD::SETNE;
     }
   }
 
   // Also handle f16, for which we need to do a f32 comparison.
-  if (LHS.getValueType() == MVT::f16) {
+  if (LHS.getValueType() == MVT::f16 && !Subtarget->hasFullFP16()) {
     LHS = DAG.getNode(ISD::FP_EXTEND, dl, MVT::f32, LHS);
     RHS = DAG.getNode(ISD::FP_EXTEND, dl, MVT::f32, RHS);
   }
 
   // Next, handle integers.
   if (LHS.getValueType().isInteger()) {
     assert((LHS.getValueType() == RHS.getValueType()) &&
            (LHS.getValueType() == MVT::i32 || LHS.getValueType() == MVT::i64));
if (Opcode == AArch64ISD::CSEL && RHSVal && !RHSVal->isOne() &&
       if (CTVal == RHSVal && AArch64CC == AArch64CC::EQ) {
         Opcode = AArch64ISD::CSINV;
         TVal = LHS;
         FVal = DAG.getConstant(0, dl, FVal.getValueType());
       }
     }
 
     SDValue CCVal;
-    SDValue Cmp = getAArch64Cmp(LHS, RHS, CC, CCVal, DAG, dl);
+    SDValue Cmp = getAArch64Cmp(LHS, RHS, CC, CCVal, DAG, dl,
+                                Subtarget->hasFullFP16());
 
     EVT VT = TVal.getValueType();
     return DAG.getNode(Opcode, dl, VT, TVal, FVal, CCVal, Cmp);
   }
 
   // Now we know we're dealing with FP values.
-  assert(LHS.getValueType() == MVT::f32 || LHS.getValueType() == MVT::f64);
+  assert(LHS.getValueType() == MVT::f16 || LHS.getValueType() == MVT::f32 ||
+         LHS.getValueType() == MVT::f64);
   assert(LHS.getValueType() == RHS.getValueType());
   EVT VT = TVal.getValueType();
-  SDValue Cmp = emitComparison(LHS, RHS, CC, dl, DAG);
+  SDValue Cmp = emitComparison(LHS, RHS, CC, dl, DAG, Subtarget->hasFullFP16());
 
   // Unfortunately, the mapping of LLVM FP CC's onto AArch64 CC's isn't totally
   // clean.  Some of them require two CSELs to implement.
   AArch64CC::CondCode CC1, CC2;
   changeFPCCToAArch64CC(CC, CC1, CC2);
 
   if (DAG.getTarget().Options.UnsafeFPMath) {
     // Transform "a == 0.0 ? 0.0 : x" to "a == 0.0 ? a : x" and
bool AArch64TargetLowering::isFPImmLegal(const APFloat &Imm, EVT VT) const {
   // FIXME: We should be able to handle f128 as well with a clever lowering.
   if (Imm.isPosZero() && (VT == MVT::f64 || VT == MVT::f32))
     return true;
 
   if (VT == MVT::f64)
     return AArch64_AM::getFP64Imm(Imm) != -1;
   else if (VT == MVT::f32)
     return AArch64_AM::getFP32Imm(Imm) != -1;
+  else if (VT == MVT::f16 && Subtarget->hasFullFP16())
+    return AArch64_AM::getFP16Imm(Imm) != -1;
   return false;
 }
 
 //===----------------------------------------------------------------------===//
 //                          AArch64 Optimization Hooks
 //===----------------------------------------------------------------------===//
 
 static SDValue getEstimate(const AArch64Subtarget *ST, unsigned Opcode,

test/CodeGen/AArch64/f16-imm.ll

+; RUN: llc < %s -mtriple=aarch64-none-eabi -mattr=+fullfp16 | FileCheck %s --check-prefix=CHECK --check-prefix=CHECK-ILLEGAL
+; RUN: llc < %s -mtriple=aarch64-none-eabi -mattr=-fullfp16 | FileCheck %s --check-prefix=CHECK-NOFP16 --check-prefix=CHECK-ILLEGAL
+
+define half @Const0() {
+entry:
+  ret half 0xH0000
+}
+; CHECK-ILLEGAL:        .[[LBL0:LCPI0_[0-9]]]:
+; CHECK-ILLEGAL-NEXT:   .hword  0 // half 0
+; CHECK-ILLEGAL-LABEL:  Const0:
+; CHECK-ILLEGAL:        adrp x[[NUM:[0-9]+]], .[[LBL0]]
+; CHECK-ILLEGAL-NEXT:   ldr h0, [x[[NUM]], :lo12:.[[LBL0]]]
+
+define half @Const1() {
+entry:
+  ret half 0xH3C00
+}
+; CHECK-DAG-LABEL: Const1:
+; CHECK-DAG-NEXT:   fmov h0, #1.00000000
+; CHECK-DAG-NEXT:   ret
+
+; CHECK-NOFP16:        .[[LBL1:LCPI1_[0-9]]]:
+; CHECK-NOFP16-NEXT:   .hword  15360 // half 1
+; CHECK-NOFP16-LABEL:  Const1:
+; CHECK-NOFP16:        adrp x[[NUM:[0-9]+]], .[[LBL1]]
+; CHECK-NOFP16-NEXT:   ldr h0, [x[[NUM]], :lo12:.[[LBL1]]]
+
+define half @Const2() {
+entry:
+  ret half 0xH3000
+}
+; CHECK-DAG-LABEL: Const2:
+; CHECK-DAG-NEXT:   fmov h0, #0.12500000
+; CHECK-DAG-NEXT:   ret
+
+; CHECK-NOFP16:        .[[LBL2:LCPI2_[0-9]]]:
+; CHECK-NOFP16-NEXT:   .hword  12288 // half 0.125
+; CHECK-NOFP16-LABEL:  Const2:
+; CHECK-NOFP16:        adrp x[[NUM:[0-9]+]], .[[LBL2]]
+; CHECK-NOFP16-NEXT:   ldr h0, [x[[NUM]], :lo12:.[[LBL2]]]
+
+define half @Const3() {
+entry:
+  ret half 0xH4F80
+}
+; CHECK-DAG-LABEL: Const3:
+; CHECK-DAG-NEXT:   fmov h0, #30.00000000
+; CHECK-DAG-NEXT:   ret
+
+; CHECK-NOFP16:        .[[LBL3:LCPI3_[0-9]]]:
+; CHECK-NOFP16-NEXT:   .hword  20352 // half 30
+; CHECK-NOFP16-LABEL:  Const3:
+; CHECK-NOFP16:        adrp x[[NUM:[0-9]+]], .[[LBL3]]
+; CHECK-NOFP16-NEXT:   ldr h0, [x[[NUM]], :lo12:.[[LBL3]]]
+
+
+define half @Const4() {
+entry:
+  ret half 0xH4FC0
+}
+; CHECK-DAG-LABEL: Const4:
+; CHECK-DAG-NEXT:  fmov h0, #31.00000000
+; CHECK-DAG-NEXT:  ret
+
+; CHECK-NOFP16:        .[[LBL4:LCPI4_[0-9]]]:
+; CHECK-NOFP16-NEXT:   .hword  20416                    // half 31
+; CHECK-NOFP16-LABEL:  Const4:
+; CHECK-NOFP16:        adrp x[[NUM:[0-9]+]], .[[LBL4]]
+; CHECK-NOFP16-NEXT:   ldr h0, [x[[NUM]], :lo12:.[[LBL4]]]
+
+define half @Const5() {
+entry:
+  ret half 0xH2FF0
+}
+; CHECK-ILLEGAL:        .[[LBL5:LCPI5_[0-9]]]:
+; CHECK-ILLEGAL-NEXT:   .hword  12272                   // half 0.12402
+; CHECK-ILLEGAL-LABEL:  Const5:
+; CHECK-ILLEGAL:        adrp x[[NUM:[0-9]+]], .[[LBL5]]
+; CHECK-ILLEGAL-NEXT:   ldr h0, [x[[NUM]], :lo12:.[[LBL5]]]
+
+define half @Const6() {
+entry:
+  ret half 0xH4FC1
+}
+; CHECK-ILLEGAL:        .[[LBL6:LCPI6_[0-9]]]:
+; CHECK-ILLEGAL-NEXT:   .hword  20417                   // half 31.016
+; CHECK-ILLEGAL-LABEL:  Const6:
+; CHECK-ILLEGAL:        adrp x[[NUM:[0-9]+]], .[[LBL6]]
+; CHECK-ILLEGAL-NEXT:   ldr h0, [x[[NUM]], :lo12:.[[LBL6]]]
+
+
+define half @Const7() {
+entry:
+  ret half 0xH5000
+}
+; CHECK-ILLEGAL:        .[[LBL7:LCPI7_[0-9]]]:
+; CHECK-ILLEGAL-NEXT:   .hword  20480                   // half 32
+; CHECK-ILLEGAL-LABEL:  Const7:
+; CHECK-ILLEGAL:        adrp x[[NUM:[0-9]+]], .[[LBL7]]
+; CHECK-ILLEGAL-NEXT:   ldr h0, [x[[NUM]], :lo12:.[[LBL7]]]
+
+

test/CodeGen/AArch64/f16-instructions.ll

-; RUN: llc < %s -mtriple aarch64-unknown-unknown -aarch64-neon-syntax=apple -asm-verbose=false -disable-post-ra -disable-fp-elim | FileCheck %s
+; RUN: llc < %s -mtriple aarch64-unknown-unknown -aarch64-neon-syntax=apple -asm-verbose=false -disable-post-ra -disable-fp-elim | FileCheck %s --check-prefix=CHECK-CVT --check-prefix=CHECK-COMMON
+; RUN: llc < %s -mtriple aarch64-unknown-unknown -mattr=+fullfp16 -aarch64-neon-syntax=apple -asm-verbose=false -disable-post-ra -disable-fp-elim | FileCheck %s --check-prefix=CHECK-COMMON --check-prefix=CHECK-FP16
 
 target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128"
 
-; CHECK-LABEL: test_fadd:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fadd s0, s0, s1
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fadd:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fadd s0, s0, s1
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fadd:
+; CHECK-FP16-NEXT:  fadd h0, h0, h1
+; CHECK-FP16-NEXT:  ret
+
 define half @test_fadd(half %a, half %b) #0 {
   %r = fadd half %a, %b
   ret half %r
 }
 
-; CHECK-LABEL: test_fsub:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fsub s0, s0, s1
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fsub:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fsub s0, s0, s1
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fsub:
+; CHECK-FP16-NEXT: fsub h0, h0, h1
+; CHECK-FP16-NEXT: ret
+
 define half @test_fsub(half %a, half %b) #0 {
   %r = fsub half %a, %b
   ret half %r
 }
 
-; CHECK-LABEL: test_fmul:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fmul s0, s0, s1
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fmul:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fmul s0, s0, s1
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fmul:
+; CHECK-FP16-NEXT: fmul h0, h0, h1
+; CHECK-FP16-NEXT: ret
+
 define half @test_fmul(half %a, half %b) #0 {
   %r = fmul half %a, %b
   ret half %r
 }
 
-; CHECK-LABEL: test_fdiv:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fdiv s0, s0, s1
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fdiv:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fdiv s0, s0, s1
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fdiv:
+; CHECK-FP16-NEXT: fdiv h0, h0, h1
+; CHECK-FP16-NEXT: ret
+
 define half @test_fdiv(half %a, half %b) #0 {
   %r = fdiv half %a, %b
   ret half %r
 }
 
-; CHECK-LABEL: test_frem:
-; CHECK-NEXT: stp x29, x30, [sp, #-16]!
-; CHECK-NEXT: mov  x29, sp
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: bl {{_?}}fmodf
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ldp x29, x30, [sp], #16
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_frem:
+; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
+; CHECK-COMMON-NEXT: mov  x29, sp
+; CHECK-COMMON-NEXT: fcvt s0, h0
+; CHECK-COMMON-NEXT: fcvt s1, h1
+; CHECK-COMMON-NEXT: bl {{_?}}fmodf
+; CHECK-COMMON-NEXT: fcvt h0, s0
+; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
+; CHECK-COMMON-NEXT: ret
 define half @test_frem(half %a, half %b) #0 {
   %r = frem half %a, %b
   ret half %r
 }
 
-; CHECK-LABEL: test_store:
-; CHECK-NEXT: str  h0, [x0]
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_store:
+; CHECK-COMMON-NEXT: str  h0, [x0]
+; CHECK-COMMON-NEXT: ret
 define void @test_store(half %a, half* %b) #0 {
   store half %a, half* %b
   ret void
 }
 
-; CHECK-LABEL: test_load:
-; CHECK-NEXT: ldr  h0, [x0]
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_load:
+; CHECK-COMMON-NEXT: ldr  h0, [x0]
+; CHECK-COMMON-NEXT: ret
 define half @test_load(half* %a) #0 {
   %r = load half, half* %a
   ret half %r
 }
 
-
 declare half @test_callee(half %a, half %b) #0
 
-; CHECK-LABEL: test_call:
-; CHECK-NEXT: stp x29, x30, [sp, #-16]!
-; CHECK-NEXT: mov  x29, sp
-; CHECK-NEXT: bl {{_?}}test_callee
-; CHECK-NEXT: ldp x29, x30, [sp], #16
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_call:
+; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
+; CHECK-COMMON-NEXT: mov  x29, sp
+; CHECK-COMMON-NEXT: bl {{_?}}test_callee
+; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
+; CHECK-COMMON-NEXT: ret
 define half @test_call(half %a, half %b) #0 {
   %r = call half @test_callee(half %a, half %b)
   ret half %r
 }
 
-; CHECK-LABEL: test_call_flipped:
-; CHECK-NEXT: stp x29, x30, [sp, #-16]!
-; CHECK-NEXT: mov  x29, sp
-; CHECK-NEXT: mov.16b  v2, v0
-; CHECK-NEXT: mov.16b  v0, v1
-; CHECK-NEXT: mov.16b  v1, v2
-; CHECK-NEXT: bl {{_?}}test_callee
-; CHECK-NEXT: ldp x29, x30, [sp], #16
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_call_flipped:
+; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
+; CHECK-COMMON-NEXT: mov  x29, sp
+; CHECK-COMMON-NEXT: mov.16b  v2, v0
+; CHECK-COMMON-NEXT: mov.16b  v0, v1
+; CHECK-COMMON-NEXT: mov.16b  v1, v2
+; CHECK-COMMON-NEXT: bl {{_?}}test_callee
+; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
+; CHECK-COMMON-NEXT: ret
 define half @test_call_flipped(half %a, half %b) #0 {
   %r = call half @test_callee(half %b, half %a)
   ret half %r
 }
 
-; CHECK-LABEL: test_tailcall_flipped:
-; CHECK-NEXT: mov.16b  v2, v0
-; CHECK-NEXT: mov.16b  v0, v1
-; CHECK-NEXT: mov.16b  v1, v2
-; CHECK-NEXT: b {{_?}}test_callee
+; CHECK-COMMON-LABEL: test_tailcall_flipped:
+; CHECK-COMMON-NEXT: mov.16b  v2, v0
+; CHECK-COMMON-NEXT: mov.16b  v0, v1
+; CHECK-COMMON-NEXT: mov.16b  v1, v2
+; CHECK-COMMON-NEXT: b {{_?}}test_callee
 define half @test_tailcall_flipped(half %a, half %b) #0 {
   %r = tail call half @test_callee(half %b, half %a)
   ret half %r
 }
 
-; CHECK-LABEL: test_select:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: cmp  w0, #0
-; CHECK-NEXT: fcsel s0, s0, s1, ne
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_select:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: cmp  w0, #0
+; CHECK-CVT-NEXT: fcsel s0, s0, s1, ne
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_select:
+; CHECK-FP16-NEXT: cmp w0, #0
+; CHECK-FP16-NEXT: fcsel h0, h0, h1, ne
+; CHECK-FP16-NEXT: ret
+
 define half @test_select(half %a, half %b, i1 zeroext %c) #0 {
   %r = select i1 %c, half %a, half %b
   ret half %r
 }
 
-; CHECK-LABEL: test_select_cc:
-; CHECK-DAG: fcvt s3, h3
-; CHECK-DAG: fcvt s2, h2
-; CHECK-DAG: fcvt s1, h1
-; CHECK-DAG: fcvt s0, h0
-; CHECK-DAG: fcmp s2, s3
-; CHECK-DAG: cset [[CC:w[0-9]+]], ne
-; CHECK-DAG: cmp [[CC]], #0
-; CHECK-NEXT: fcsel s0, s0, s1, ne
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_select_cc:
+; CHECK-CVT-DAG: fcvt s3, h3
+; CHECK-CVT-DAG: fcvt s2, h2
+; CHECK-CVT-DAG: fcvt s1, h1
+; CHECK-CVT-DAG: fcvt s0, h0
+; CHECK-CVT-DAG: fcmp s2, s3
+; CHECK-CVT-DAG: cset [[CC:w[0-9]+]], ne
+; CHECK-CVT-DAG: cmp [[CC]], #0
+; CHECK-CVT-NEXT: fcsel s0, s0, s1, ne
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_select_cc:
+; CHECK-FP16-NEXT: fcmp h2, h3
+; CHECK-FP16-NEXT: fcsel h0, h0, h1, ne
+; CHECK-FP16-NEXT: ret
+
 define half @test_select_cc(half %a, half %b, half %c, half %d) #0 {
   %cc = fcmp une half %c, %d
   %r = select i1 %cc, half %a, half %b
   ret half %r
 }
 
-; CHECK-LABEL: test_select_cc_f32_f16:
-; CHECK-DAG:   fcvt s2, h2
-; CHECK-DAG:   fcvt s3, h3
-; CHECK-NEXT:  fcmp s2, s3
-; CHECK-NEXT:  fcsel s0, s0, s1, ne
-; CHECK-NEXT:  ret
+; CHECK-CVT-LABEL: test_select_cc_f32_f16:
+; CHECK-CVT-DAG:   fcvt s2, h2
+; CHECK-CVT-DAG:   fcvt s3, h3
+; CHECK-CVT-NEXT:  fcmp s2, s3
+; CHECK-CVT-NEXT:  fcsel s0, s0, s1, ne
+; CHECK-CVT-NEXT:  ret
+
+; CHECK-FP16-LABEL: test_select_cc_f32_f16:
+; CHECK-FP16-NEXT: fcmp h2, h3
+; CHECK-FP16-NEXT: fcsel  s0, s0, s1, ne
+; CHECK-FP16-NEXT: ret
+
 define float @test_select_cc_f32_f16(float %a, float %b, half %c, half %d) #0 {
   %cc = fcmp une half %c, %d
   %r = select i1 %cc, float %a, float %b
   ret float %r
 }
 
-; CHECK-LABEL: test_select_cc_f16_f32:
-; CHECK-DAG:  fcvt s0, h0
-; CHECK-DAG:  fcvt s1, h1
-; CHECK-DAG:  fcmp s2, s3
-; CHECK-DAG:  cset w8, ne
-; CHECK-NEXT: cmp w8, #0
-; CHECK-NEXT: fcsel s0, s0, s1, ne
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_select_cc_f16_f32:
+; CHECK-CVT-DAG:  fcvt s0, h0
+; CHECK-CVT-DAG:  fcvt s1, h1
+; CHECK-CVT-DAG:  fcmp s2, s3
+; CHECK-CVT-DAG:  cset w8, ne
+; CHECK-CVT-NEXT: cmp w8, #0
+; CHECK-CVT-NEXT: fcsel s0, s0, s1, ne
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_select_cc_f16_f32:
+; CHECK-FP16-NEXT: fcmp s2, s3
+; CHECK-FP16-NEXT: fcsel h0, h0, h1, ne
+; CHECK-FP16-NEXT: ret
+
 define half @test_select_cc_f16_f32(half %a, half %b, float %c, float %d) #0 {
   %cc = fcmp une float %c, %d
   %r = select i1 %cc, half %a, half %b
   ret half %r
 }
 
-; CHECK-LABEL: test_fcmp_une:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcmp s0, s1
-; CHECK-NEXT: cset  w0, ne
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fcmp_une:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcmp s0, s1
+; CHECK-CVT-NEXT: cset  w0, ne
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fcmp_une:
+; CHECK-FP16-NEXT: fcmp h0, h1
+; CHECK-FP16-NEXT: cset w0, ne
+; CHECK-FP16-NEXT: ret
+
 define i1 @test_fcmp_une(half %a, half %b) #0 {
   %r = fcmp une half %a, %b
   ret i1 %r
 }
 
-; CHECK-LABEL: test_fcmp_ueq:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcmp s0, s1
-; CHECK-NEXT: cset [[TRUE:w[0-9]+]], eq
-; CHECK-NEXT: csinc w0, [[TRUE]], wzr, vc
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fcmp_ueq:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcmp s0, s1
+; CHECK-CVT-NEXT: cset [[TRUE:w[0-9]+]], eq
+; CHECK-CVT-NEXT: csinc w0, [[TRUE]], wzr, vc
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fcmp_ueq:
+; CHECK-FP16-NEXT: fcmp h0, h1
+; CHECK-FP16-NEXT: cset [[TRUE:w[0-9]+]], eq
+; CHECK-FP16-NEXT: csinc w0, [[TRUE]], wzr, vc
+; CHECK-FP16-NEXT: ret
+
 define i1 @test_fcmp_ueq(half %a, half %b) #0 {
   %r = fcmp ueq half %a, %b
   ret i1 %r
 }
 
-; CHECK-LABEL: test_fcmp_ugt:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcmp s0, s1
-; CHECK-NEXT: cset  w0, hi
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fcmp_ugt:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcmp s0, s1
+; CHECK-CVT-NEXT: cset  w0, hi
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fcmp_ugt:
+; CHECK-FP16-NEXT: fcmp h0, h1
+; CHECK-FP16-NEXT: cset  w0, hi
+; CHECK-FP16-NEXT: ret
+
 define i1 @test_fcmp_ugt(half %a, half %b) #0 {
   %r = fcmp ugt half %a, %b
   ret i1 %r
 }
 
-; CHECK-LABEL: test_fcmp_uge:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcmp s0, s1
-; CHECK-NEXT: cset  w0, pl
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fcmp_uge:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcmp s0, s1
+; CHECK-CVT-NEXT: cset  w0, pl
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fcmp_uge:
+; CHECK-FP16-NEXT: fcmp h0, h1
+; CHECK-FP16-NEXT: cset  w0, pl
+; CHECK-FP16-NEXT: ret
+
 define i1 @test_fcmp_uge(half %a, half %b) #0 {
   %r = fcmp uge half %a, %b
   ret i1 %r
 }
 
-; CHECK-LABEL: test_fcmp_ult:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcmp s0, s1
-; CHECK-NEXT: cset  w0, lt
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fcmp_ult:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcmp s0, s1
+; CHECK-CVT-NEXT: cset  w0, lt
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fcmp_ult:
+; CHECK-FP16-NEXT: fcmp h0, h1
+; CHECK-FP16-NEXT: cset  w0, lt
+; CHECK-FP16-NEXT: ret
+
 define i1 @test_fcmp_ult(half %a, half %b) #0 {
   %r = fcmp ult half %a, %b
   ret i1 %r
 }
 
-; CHECK-LABEL: test_fcmp_ule:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcmp s0, s1
-; CHECK-NEXT: cset  w0, le
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fcmp_ule:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcmp s0, s1
+; CHECK-CVT-NEXT: cset  w0, le
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fcmp_ule:
+; CHECK-FP16-NEXT: fcmp h0, h1
+; CHECK-FP16-NEXT: cset  w0, le
+; CHECK-FP16-NEXT: ret
+
 define i1 @test_fcmp_ule(half %a, half %b) #0 {
   %r = fcmp ule half %a, %b
   ret i1 %r
 }
 
+; CHECK-CVT-LABEL: test_fcmp_uno:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcmp s0, s1
+; CHECK-CVT-NEXT: cset  w0, vs
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fcmp_uno:
+; CHECK-FP16-NEXT: fcmp h0, h1
+; CHECK-FP16-NEXT: cset  w0, vs
+; CHECK-FP16-NEXT: ret
 
-; CHECK-LABEL: test_fcmp_uno:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcmp s0, s1
-; CHECK-NEXT: cset  w0, vs
-; CHECK-NEXT: ret
 define i1 @test_fcmp_uno(half %a, half %b) #0 {
   %r = fcmp uno half %a, %b
   ret i1 %r
 }
 
-; CHECK-LABEL: test_fcmp_one:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcmp s0, s1
-; CHECK-NEXT: cset [[TRUE:w[0-9]+]], mi
-; CHECK-NEXT: csinc w0, [[TRUE]], wzr, le
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fcmp_one:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcmp s0, s1
+; CHECK-CVT-NEXT: cset [[TRUE:w[0-9]+]], mi
+; CHECK-CVT-NEXT: csinc w0, [[TRUE]], wzr, le
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fcmp_one:
+; CHECK-FP16-NEXT: fcmp h0, h1
+; CHECK-FP16-NEXT: cset [[TRUE:w[0-9]+]], mi
+; CHECK-FP16-NEXT: csinc w0, [[TRUE]], wzr, le
+; CHECK-FP16-NEXT: ret
+
 define i1 @test_fcmp_one(half %a, half %b) #0 {
   %r = fcmp one half %a, %b
   ret i1 %r
 }
 
-; CHECK-LABEL: test_fcmp_oeq:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcmp s0, s1
-; CHECK-NEXT: cset  w0, eq
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fcmp_oeq:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcmp s0, s1
+; CHECK-CVT-NEXT: cset  w0, eq
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fcmp_oeq:
+; CHECK-FP16-NEXT: fcmp h0, h1
+; CHECK-FP16-NEXT: cset  w0, eq
+; CHECK-FP16-NEXT: ret
+
 define i1 @test_fcmp_oeq(half %a, half %b) #0 {
   %r = fcmp oeq half %a, %b
   ret i1 %r
 }
 
-; CHECK-LABEL: test_fcmp_ogt:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcmp s0, s1
-; CHECK-NEXT: cset  w0, gt
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fcmp_ogt:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcmp s0, s1
+; CHECK-CVT-NEXT: cset  w0, gt
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fcmp_ogt:
+; CHECK-FP16-NEXT: fcmp h0, h1
+; CHECK-FP16-NEXT: cset  w0, gt
+; CHECK-FP16-NEXT: ret
+
 define i1 @test_fcmp_ogt(half %a, half %b) #0 {
   %r = fcmp ogt half %a, %b
   ret i1 %r
 }
 
-; CHECK-LABEL: test_fcmp_oge:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcmp s0, s1
-; CHECK-NEXT: cset  w0, ge
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fcmp_oge:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcmp s0, s1
+; CHECK-CVT-NEXT: cset  w0, ge
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fcmp_oge:
+; CHECK-FP16-NEXT: fcmp h0, h1
+; CHECK-FP16-NEXT: cset  w0, ge
+; CHECK-FP16-NEXT: ret
+
 define i1 @test_fcmp_oge(half %a, half %b) #0 {
   %r = fcmp oge half %a, %b
   ret i1 %r
 }
 
-; CHECK-LABEL: test_fcmp_olt:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcmp s0, s1
-; CHECK-NEXT: cset  w0, mi
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fcmp_olt:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcmp s0, s1
+; CHECK-CVT-NEXT: cset  w0, mi
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fcmp_olt:
+; CHECK-FP16-NEXT: fcmp h0, h1
+; CHECK-FP16-NEXT: cset  w0, mi
+; CHECK-FP16-NEXT: ret
+
 define i1 @test_fcmp_olt(half %a, half %b) #0 {
   %r = fcmp olt half %a, %b
   ret i1 %r
 }
 
-; CHECK-LABEL: test_fcmp_ole:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcmp s0, s1
-; CHECK-NEXT: cset  w0, ls
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fcmp_ole:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcmp s0, s1
+; CHECK-CVT-NEXT: cset  w0, ls
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fcmp_ole:
+; CHECK-FP16-NEXT: fcmp h0, h1
+; CHECK-FP16-NEXT: cset  w0, ls
+; CHECK-FP16-NEXT: ret
+
 define i1 @test_fcmp_ole(half %a, half %b) #0 {
   %r = fcmp ole half %a, %b
   ret i1 %r
 }
 
-; CHECK-LABEL: test_fcmp_ord:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcmp s0, s1
-; CHECK-NEXT: cset  w0, vc
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fcmp_ord:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcmp s0, s1
+; CHECK-CVT-NEXT: cset  w0, vc
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fcmp_ord:
+; CHECK-FP16-NEXT: fcmp h0, h1
+; CHECK-FP16-NEXT: cset  w0, vc
+; CHECK-FP16-NEXT: ret
+
 define i1 @test_fcmp_ord(half %a, half %b) #0 {
   %r = fcmp ord half %a, %b
   ret i1 %r
 }
 
-; CHECK-LABEL: test_br_cc:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcmp s0, s1
-; CHECK-NEXT: b.mi [[BRCC_ELSE:.?LBB[0-9_]+]]
-; CHECK-NEXT: str  wzr, [x0]
-; CHECK-NEXT: ret
-; CHECK-NEXT: [[BRCC_ELSE]]:
-; CHECK-NEXT: str  wzr, [x1]
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_br_cc:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcmp s0, s1
+; CHECK-CVT-NEXT: b.mi [[BRCC_ELSE:.?LBB[0-9_]+]]
+; CHECK-CVT-NEXT: str  wzr, [x0]
+; CHECK-CVT-NEXT: ret
+; CHECK-CVT-NEXT: [[BRCC_ELSE]]:
+; CHECK-CVT-NEXT: str  wzr, [x1]
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_br_cc:
+; CHECK-FP16-NEXT: fcmp h0, h1
+; CHECK-FP16-NEXT: b.mi [[BRCC_ELSE:.?LBB[0-9_]+]]
+; CHECK-FP16-NEXT: str  wzr, [x0]
+; CHECK-FP16-NEXT: ret
+; CHECK-FP16-NEXT: [[BRCC_ELSE]]:
+; CHECK-FP16-NEXT: str  wzr, [x1]
+; CHECK-FP16-NEXT: ret
+
 define void @test_br_cc(half %a, half %b, i32* %p1, i32* %p2) #0 {
   %c = fcmp uge half %a, %b
   br i1 %c, label %then, label %else
 then:
   store i32 0, i32* %p1
   ret void
 else:
   store i32 0, i32* %p2
   ret void
 }
 
-; CHECK-LABEL: test_phi:
-; CHECK: mov  x[[PTR:[0-9]+]], x0
-; CHECK: ldr  h[[AB:[0-9]+]], [x[[PTR]]]
-; CHECK: [[LOOP:LBB[0-9_]+]]:
-; CHECK: mov.16b  v[[R:[0-9]+]], v[[AB]]
-; CHECK: ldr  h[[AB]], [x[[PTR]]]
-; CHECK: mov  x0, x[[PTR]]
-; CHECK: bl {{_?}}test_dummy
-; CHECK: mov.16b  v0, v[[R]]
-; CHECK: ret
+; CHECK-COMMON-LABEL: test_phi:
+; CHECK-COMMON: mov  x[[PTR:[0-9]+]], x0
+; CHECK-COMMON: ldr  h[[AB:[0-9]+]], [x[[PTR]]]
+; CHECK-COMMON: [[LOOP:LBB[0-9_]+]]:
+; CHECK-COMMON: mov.16b  v[[R:[0-9]+]], v[[AB]]
+; CHECK-COMMON: ldr  h[[AB]], [x[[PTR]]]
+; CHECK-COMMON: mov  x0, x[[PTR]]
+; CHECK-COMMON: bl {{_?}}test_dummy
+; CHECK-COMMON: mov.16b  v0, v[[R]]
+; CHECK-COMMON: ret
 define half @test_phi(half* %p1) #0 {
 entry:
   %a = load half, half* %p1
   br label %loop
 loop:
   %r = phi half [%a, %entry], [%b, %loop]
   %b = load half, half* %p1
   %c = call i1 @test_dummy(half* %p1)
   br i1 %c, label %loop, label %return
 return:
   ret half %r
 }
+
 declare i1 @test_dummy(half* %p1) #0
 
-; CHECK-LABEL: test_fptosi_i32:
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcvtzs w0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fptosi_i32:
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcvtzs w0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fptosi_i32:
+; CHECK-FP16-NEXT: fcvtzs w0, h0
+; CHECK-FP16-NEXT: ret
+
 define i32 @test_fptosi_i32(half %a) #0 {
   %r = fptosi half %a to i32
   ret i32 %r
 }
 
-; CHECK-LABEL: test_fptosi_i64:
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcvtzs x0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fptosi_i64:
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcvtzs x0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fptosi_i64:
+; CHECK-FP16-NEXT: fcvtzs x0, h0
+; CHECK-FP16-NEXT: ret
+
 define i64 @test_fptosi_i64(half %a) #0 {
   %r = fptosi half %a to i64
   ret i64 %r
 }
 
-; CHECK-LABEL: test_fptoui_i32:
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcvtzu w0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fptoui_i32:
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcvtzu w0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fptoui_i32:
+; CHECK-FP16-NEXT: fcvtzu w0, h0
+; CHECK-FP16-NEXT: ret
+
 define i32 @test_fptoui_i32(half %a) #0 {
   %r = fptoui half %a to i32
   ret i32 %r
 }
 
-; CHECK-LABEL: test_fptoui_i64:
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcvtzu x0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fptoui_i64:
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcvtzu x0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fptoui_i64:
+; CHECK-FP16-NEXT: fcvtzu x0, h0
+; CHECK-FP16-NEXT: ret
+
 define i64 @test_fptoui_i64(half %a) #0 {
   %r = fptoui half %a to i64
   ret i64 %r
 }
 
-; CHECK-LABEL: test_uitofp_i32:
-; CHECK-NEXT: ucvtf s0, w0
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_uitofp_i32:
+; CHECK-CVT-NEXT: ucvtf s0, w0
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_uitofp_i32:
+; CHECK-FP16-NEXT: ucvtf h0, w0
+; CHECK-FP16-NEXT: ret
+
 define half @test_uitofp_i32(i32 %a) #0 {
   %r = uitofp i32 %a to half
   ret half %r
 }
 
-; CHECK-LABEL: test_uitofp_i64:
-; CHECK-NEXT: ucvtf s0, x0
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_uitofp_i64:
+; CHECK-CVT-NEXT: ucvtf s0, x0
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_uitofp_i64:
+; CHECK-FP16-NEXT: ucvtf h0, x0
+; CHECK-FP16-NEXT: ret
+
 define half @test_uitofp_i64(i64 %a) #0 {
   %r = uitofp i64 %a to half
   ret half %r
 }
 
-; CHECK-LABEL: test_sitofp_i32:
-; CHECK-NEXT: scvtf s0, w0
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_sitofp_i32:
+; CHECK-CVT-NEXT: scvtf s0, w0
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_sitofp_i32:
+; CHECK-FP16-NEXT: scvtf h0, w0
+; CHECK-FP16-NEXT: ret
+
 define half @test_sitofp_i32(i32 %a) #0 {
   %r = sitofp i32 %a to half
   ret half %r
 }
 
-; CHECK-LABEL: test_sitofp_i64:
-; CHECK-NEXT: scvtf s0, x0
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_sitofp_i64:
+; CHECK-CVT-NEXT: scvtf s0, x0
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_sitofp_i64:
+; CHECK-FP16-NEXT: scvtf h0, x0
+; CHECK-FP16-NEXT: ret
 define half @test_sitofp_i64(i64 %a) #0 {
   %r = sitofp i64 %a to half
   ret half %r
 }
 
-; CHECK-LABEL: test_uitofp_i32_fadd:
-; CHECK-NEXT: ucvtf s1, w0
-; CHECK-NEXT: fcvt h1, s1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fadd s0, s0, s1
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_uitofp_i32_fadd:
+; CHECK-CVT-NEXT: ucvtf s1, w0
+; CHECK-CVT-NEXT: fcvt h1, s1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fadd s0, s0, s1
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_uitofp_i32_fadd:
+; CHECK-FP16-NEXT: ucvtf h1, w0
+; CHECK-FP16-NEXT: fadd h0, h0, h1
+; CHECK-FP16-NEXT: ret
+
 define half @test_uitofp_i32_fadd(i32 %a, half %b) #0 {
   %c = uitofp i32 %a to half
   %r = fadd half %b, %c
   ret half %r
 }
 
-; CHECK-LABEL: test_sitofp_i32_fadd:
-; CHECK-NEXT: scvtf s1, w0
-; CHECK-NEXT: fcvt h1, s1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fadd s0, s0, s1
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_sitofp_i32_fadd:
+; CHECK-CVT-NEXT: scvtf s1, w0
+; CHECK-CVT-NEXT: fcvt h1, s1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fadd s0, s0, s1
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_sitofp_i32_fadd:
+; CHECK-FP16-NEXT: scvtf h1, w0
+; CHECK-FP16-NEXT: fadd h0, h0, h1
+; CHECK-FP16-NEXT: ret
+
 define half @test_sitofp_i32_fadd(i32 %a, half %b) #0 {
   %c = sitofp i32 %a to half
   %r = fadd half %b, %c
   ret half %r
 }
 
-; CHECK-LABEL: test_fptrunc_float:
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_fptrunc_float:
+; CHECK-COMMON-NEXT: fcvt h0, s0
+; CHECK-COMMON-NEXT: ret
 
 define half @test_fptrunc_float(float %a) #0 {
   %r = fptrunc float %a to half
   ret half %r
 }
 
-; CHECK-LABEL: test_fptrunc_double:
-; CHECK-NEXT: fcvt h0, d0
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_fptrunc_double:
+; CHECK-COMMON-NEXT: fcvt h0, d0
+; CHECK-COMMON-NEXT: ret
 define half @test_fptrunc_double(double %a) #0 {
   %r = fptrunc double %a to half
   ret half %r
 }
 
-; CHECK-LABEL: test_fpext_float:
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_fpext_float:
+; CHECK-COMMON-NEXT: fcvt s0, h0
+; CHECK-COMMON-NEXT: ret
 define float @test_fpext_float(half %a) #0 {
   %r = fpext half %a to float
   ret float %r
 }
 
-; CHECK-LABEL: test_fpext_double:
-; CHECK-NEXT: fcvt d0, h0
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_fpext_double:
+; CHECK-COMMON-NEXT: fcvt d0, h0
+; CHECK-COMMON-NEXT: ret
 define double @test_fpext_double(half %a) #0 {
   %r = fpext half %a to double
   ret double %r
 }
 
 
-; CHECK-LABEL: test_bitcast_halftoi16:
-; CHECK-NEXT: fmov w0, s0
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_bitcast_halftoi16:
+; CHECK-COMMON-NEXT: fmov w0, s0
+; CHECK-COMMON-NEXT: ret
 define i16 @test_bitcast_halftoi16(half %a) #0 {
   %r = bitcast half %a to i16
   ret i16 %r
 }
 
-; CHECK-LABEL: test_bitcast_i16tohalf:
-; CHECK-NEXT: fmov s0, w0
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_bitcast_i16tohalf:
+; CHECK-COMMON-NEXT: fmov s0, w0
+; CHECK-COMMON-NEXT: ret
 define half @test_bitcast_i16tohalf(i16 %a) #0 {
   %r = bitcast i16 %a to half
   ret half %r
 }
 
 
 declare half @llvm.sqrt.f16(half %a) #0
 declare half @llvm.powi.f16(half %a, i32 %b) #0

 declare half @llvm.floor.f16(half %a) #0
 declare half @llvm.ceil.f16(half %a) #0
 declare half @llvm.trunc.f16(half %a) #0
 declare half @llvm.rint.f16(half %a) #0
 declare half @llvm.nearbyint.f16(half %a) #0
 declare half @llvm.round.f16(half %a) #0
 declare half @llvm.fmuladd.f16(half %a, half %b, half %c) #0
 
-; CHECK-LABEL: test_sqrt:
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fsqrt s0, s0
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_sqrt:
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fsqrt s0, s0
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_sqrt:
+; CHECK-FP16-NEXT: fsqrt h0, h0
+; CHECK-FP16-NEXT: ret
+
 define half @test_sqrt(half %a) #0 {
   %r = call half @llvm.sqrt.f16(half %a)
   ret half %r
 }
 
-; CHECK-LABEL: test_powi:
-; CHECK-NEXT: stp x29, x30, [sp, #-16]!
-; CHECK-NEXT: mov  x29, sp
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: bl {{_?}}__powisf2
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ldp x29, x30, [sp], #16
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_powi:
+; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
+; CHECK-COMMON-NEXT: mov  x29, sp
+; CHECK-COMMON-NEXT: fcvt s0, h0
+; CHECK-COMMON-NEXT: bl {{_?}}__powisf2
+; CHECK-COMMON-NEXT: fcvt h0, s0
+; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
+; CHECK-COMMON-NEXT: ret
 define half @test_powi(half %a, i32 %b) #0 {
   %r = call half @llvm.powi.f16(half %a, i32 %b)
   ret half %r
 }
 
-; CHECK-LABEL: test_sin:
-; CHECK-NEXT: stp x29, x30, [sp, #-16]!
-; CHECK-NEXT: mov  x29, sp
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: bl {{_?}}sinf
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ldp x29, x30, [sp], #16
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_sin:
+; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
+; CHECK-COMMON-NEXT: mov  x29, sp
+; CHECK-COMMON-NEXT: fcvt s0, h0
+; CHECK-COMMON-NEXT: bl {{_?}}sinf
+; CHECK-COMMON-NEXT: fcvt h0, s0
+; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
+; CHECK-COMMON-NEXT: ret
 define half @test_sin(half %a) #0 {
   %r = call half @llvm.sin.f16(half %a)
   ret half %r
 }
 
-; CHECK-LABEL: test_cos:
-; CHECK-NEXT: stp x29, x30, [sp, #-16]!
-; CHECK-NEXT: mov  x29, sp
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: bl {{_?}}cosf
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ldp x29, x30, [sp], #16
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_cos:
+; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
+; CHECK-COMMON-NEXT: mov  x29, sp
+; CHECK-COMMON-NEXT: fcvt s0, h0
+; CHECK-COMMON-NEXT: bl {{_?}}cosf
+; CHECK-COMMON-NEXT: fcvt h0, s0
+; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
+; CHECK-COMMON-NEXT: ret
 define half @test_cos(half %a) #0 {
   %r = call half @llvm.cos.f16(half %a)
   ret half %r
 }
 
-; CHECK-LABEL: test_pow:
-; CHECK-NEXT: stp x29, x30, [sp, #-16]!
-; CHECK-NEXT: mov  x29, sp
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: bl {{_?}}powf
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ldp x29, x30, [sp], #16
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_pow:
+; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
+; CHECK-COMMON-NEXT: mov  x29, sp
+; CHECK-COMMON-NEXT: fcvt s0, h0
+; CHECK-COMMON-NEXT: fcvt s1, h1
+; CHECK-COMMON-NEXT: bl {{_?}}powf
+; CHECK-COMMON-NEXT: fcvt h0, s0
+; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
+; CHECK-COMMON-NEXT: ret
 define half @test_pow(half %a, half %b) #0 {
   %r = call half @llvm.pow.f16(half %a, half %b)
   ret half %r
 }
 
-; CHECK-LABEL: test_exp:
-; CHECK-NEXT: stp x29, x30, [sp, #-16]!
-; CHECK-NEXT: mov  x29, sp
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: bl {{_?}}expf
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ldp x29, x30, [sp], #16
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_exp:
+; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
+; CHECK-COMMON-NEXT: mov  x29, sp
+; CHECK-COMMON-NEXT: fcvt s0, h0
+; CHECK-COMMON-NEXT: bl {{_?}}expf
+; CHECK-COMMON-NEXT: fcvt h0, s0
+; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
+; CHECK-COMMON-NEXT: ret
 define half @test_exp(half %a) #0 {
   %r = call half @llvm.exp.f16(half %a)
   ret half %r
 }
 
-; CHECK-LABEL: test_exp2:
-; CHECK-NEXT: stp x29, x30, [sp, #-16]!
-; CHECK-NEXT: mov  x29, sp
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: bl {{_?}}exp2f
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ldp x29, x30, [sp], #16
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_exp2:
+; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
+; CHECK-COMMON-NEXT: mov  x29, sp
+; CHECK-COMMON-NEXT: fcvt s0, h0
+; CHECK-COMMON-NEXT: bl {{_?}}exp2f
+; CHECK-COMMON-NEXT: fcvt h0, s0
+; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
+; CHECK-COMMON-NEXT: ret
 define half @test_exp2(half %a) #0 {
   %r = call half @llvm.exp2.f16(half %a)
   ret half %r
 }
 
-; CHECK-LABEL: test_log:
-; CHECK-NEXT: stp x29, x30, [sp, #-16]!
-; CHECK-NEXT: mov  x29, sp
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: bl {{_?}}logf
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ldp x29, x30, [sp], #16
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_log:
+; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
+; CHECK-COMMON-NEXT: mov  x29, sp
+; CHECK-COMMON-NEXT: fcvt s0, h0
+; CHECK-COMMON-NEXT: bl {{_?}}logf
+; CHECK-COMMON-NEXT: fcvt h0, s0
+; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
+; CHECK-COMMON-NEXT: ret
 define half @test_log(half %a) #0 {
   %r = call half @llvm.log.f16(half %a)
   ret half %r
 }
 
-; CHECK-LABEL: test_log10:
-; CHECK-NEXT: stp x29, x30, [sp, #-16]!
-; CHECK-NEXT: mov  x29, sp
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: bl {{_?}}log10f
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ldp x29, x30, [sp], #16
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_log10:
+; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
+; CHECK-COMMON-NEXT: mov  x29, sp
+; CHECK-COMMON-NEXT: fcvt s0, h0
+; CHECK-COMMON-NEXT: bl {{_?}}log10f
+; CHECK-COMMON-NEXT: fcvt h0, s0
+; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
+; CHECK-COMMON-NEXT: ret
 define half @test_log10(half %a) #0 {
   %r = call half @llvm.log10.f16(half %a)
   ret half %r
 }
 
-; CHECK-LABEL: test_log2:
-; CHECK-NEXT: stp x29, x30, [sp, #-16]!
-; CHECK-NEXT: mov  x29, sp
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: bl {{_?}}log2f
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ldp x29, x30, [sp], #16
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_log2:
+; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
+; CHECK-COMMON-NEXT: mov  x29, sp
+; CHECK-COMMON-NEXT: fcvt s0, h0
+; CHECK-COMMON-NEXT: bl {{_?}}log2f
+; CHECK-COMMON-NEXT: fcvt h0, s0
+; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
+; CHECK-COMMON-NEXT: ret
 define half @test_log2(half %a) #0 {
   %r = call half @llvm.log2.f16(half %a)
   ret half %r
 }
 
-; CHECK-LABEL: test_fma:
-; CHECK-NEXT: fcvt s2, h2
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fmadd s0, s0, s1, s2
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fma:
+; CHECK-CVT-NEXT: fcvt s2, h2
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fmadd s0, s0, s1, s2
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fma:
+; CHECK-FP16-NEXT: fmadd h0, h0, h1, h2
+; CHECK-FP16-NEXT: ret
+
 define half @test_fma(half %a, half %b, half %c) #0 {
   %r = call half @llvm.fma.f16(half %a, half %b, half %c)
   ret half %r
 }
 
-; CHECK-LABEL: test_fabs:
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fabs s0, s0
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fabs:
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fabs s0, s0
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fabs:
+; CHECK-FP16-NEXT: fabs h0, h0
+; CHECK-FP16-NEXT: ret
+
 define half @test_fabs(half %a) #0 {
   %r = call half @llvm.fabs.f16(half %a)
   ret half %r
 }
 
-; CHECK-LABEL: test_minnum:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fminnm s0, s0, s1
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_minnum:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fminnm s0, s0, s1
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_minnum:
+; CHECK-FP16-NEXT: fminnm h0, h0, h1
+; CHECK-FP16-NEXT: ret
+
 define half @test_minnum(half %a, half %b) #0 {
   %r = call half @llvm.minnum.f16(half %a, half %b)
   ret half %r
 }
 
-; CHECK-LABEL: test_maxnum:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fmaxnm s0, s0, s1
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_maxnum:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fmaxnm s0, s0, s1
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_maxnum:
+; CHECK-FP16-NEXT: fmaxnm h0, h0, h1
+; CHECK-FP16-NEXT: ret
+
 define half @test_maxnum(half %a, half %b) #0 {
   %r = call half @llvm.maxnum.f16(half %a, half %b)
   ret half %r
 }
 
-; CHECK-LABEL: test_copysign:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: movi.4s v2, #128, lsl #24
-; CHECK-NEXT: bit.16b v0, v1, v2
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_copysign:
+; CHECK-COMMON-NEXT: fcvt s1, h1
+; CHECK-COMMON-NEXT: fcvt s0, h0
+; CHECK-COMMON-NEXT: movi.4s v2, #128, lsl #24
+; CHECK-COMMON-NEXT: bit.16b v0, v1, v2
+; CHECK-COMMON-NEXT: fcvt h0, s0
+; CHECK-COMMON-NEXT: ret
 define half @test_copysign(half %a, half %b) #0 {
   %r = call half @llvm.copysign.f16(half %a, half %b)
   ret half %r
 }
 
-; CHECK-LABEL: test_copysign_f32:
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: movi.4s v2, #128, lsl #24
-; CHECK-NEXT: bit.16b v0, v1, v2
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_copysign_f32:
+; CHECK-COMMON-NEXT: fcvt s0, h0
+; CHECK-COMMON-NEXT: movi.4s v2, #128, lsl #24
+; CHECK-COMMON-NEXT: bit.16b v0, v1, v2
+; CHECK-COMMON-NEXT: fcvt h0, s0
+; CHECK-COMMON-NEXT: ret
 define half @test_copysign_f32(half %a, float %b) #0 {
   %tb = fptrunc float %b to half
   %r = call half @llvm.copysign.f16(half %a, half %tb)
   ret half %r
 }
 
-; CHECK-LABEL: test_copysign_f64:
-; CHECK-NEXT: fcvt s1, d1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: movi.4s v2, #128, lsl #24
-; CHECK-NEXT: bit.16b v0, v1, v2
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_copysign_f64:
+; CHECK-COMMON-NEXT: fcvt s1, d1
+; CHECK-COMMON-NEXT: fcvt s0, h0
+; CHECK-COMMON-NEXT: movi.4s v2, #128, lsl #24
+; CHECK-COMMON-NEXT: bit.16b v0, v1, v2
+; CHECK-COMMON-NEXT: fcvt h0, s0
+; CHECK-COMMON-NEXT: ret
 define half @test_copysign_f64(half %a, double %b) #0 {
   %tb = fptrunc double %b to half
   %r = call half @llvm.copysign.f16(half %a, half %tb)
   ret half %r
 }
 
 ; Check that the FP promotion will use a truncating FP_ROUND, so we can fold
 ; away the (fpext (fp_round <result>)) here.
 
-; CHECK-LABEL: test_copysign_extended:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: movi.4s v2, #128, lsl #24
-; CHECK-NEXT: bit.16b v0, v1, v2
-; CHECK-NEXT: ret
+; CHECK-COMMON-LABEL: test_copysign_extended:
+; CHECK-COMMON-NEXT: fcvt s1, h1
+; CHECK-COMMON-NEXT: fcvt s0, h0
+; CHECK-COMMON-NEXT: movi.4s v2, #128, lsl #24
+; CHECK-COMMON-NEXT: bit.16b v0, v1, v2
+; CHECK-COMMON-NEXT: ret
 define float @test_copysign_extended(half %a, half %b) #0 {
   %r = call half @llvm.copysign.f16(half %a, half %b)
   %xr = fpext half %r to float
   ret float %xr
 }
 
-; CHECK-LABEL: test_floor:
-; CHECK-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0
-; CHECK-NEXT: frintm [[INT32:s[0-9]+]], [[FLOAT32]]
-; CHECK-NEXT: fcvt h0, [[INT32]]
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_floor:
+; CHECK-CVT-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0
+; CHECK-CVT-NEXT: frintm [[INT32:s[0-9]+]], [[FLOAT32]]
+; CHECK-CVT-NEXT: fcvt h0, [[INT32]]
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_floor:
+; CHECK-FP16-NEXT: frintm h0, h0
+; CHECK-FP16-NEXT: ret
+
 define half @test_floor(half %a) #0 {
   %r = call half @llvm.floor.f16(half %a)
   ret half %r
 }
 
-; CHECK-LABEL: test_ceil:
-; CHECK-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0
-; CHECK-NEXT: frintp [[INT32:s[0-9]+]], [[FLOAT32]]
-; CHECK-NEXT: fcvt h0, [[INT32]]
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_ceil:
+; CHECK-CVT-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0
+; CHECK-CVT-NEXT: frintp [[INT32:s[0-9]+]], [[FLOAT32]]
+; CHECK-CVT-NEXT: fcvt h0, [[INT32]]
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_ceil:
+; CHECK-FP16-NEXT: frintp h0, h0
+; CHECK-FP16-NEXT: ret
+
 define half @test_ceil(half %a) #0 {
   %r = call half @llvm.ceil.f16(half %a)
   ret half %r
 }
 
-; CHECK-LABEL: test_trunc:
-; CHECK-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0
-; CHECK-NEXT: frintz [[INT32:s[0-9]+]], [[FLOAT32]]
-; CHECK-NEXT: fcvt h0, [[INT32]]
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_trunc:
+; CHECK-CVT-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0
+; CHECK-CVT-NEXT: frintz [[INT32:s[0-9]+]], [[FLOAT32]]
+; CHECK-CVT-NEXT: fcvt h0, [[INT32]]
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_trunc:
+; CHECK-FP16-NEXT: frintz h0, h0
+; CHECK-FP16-NEXT: ret
+
 define half @test_trunc(half %a) #0 {
   %r = call half @llvm.trunc.f16(half %a)
   ret half %r
 }
 
-; CHECK-LABEL: test_rint:
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: frintx s0, s0
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_rint:
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: frintx s0, s0
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_rint:
+; CHECK-FP16-NEXT: frintx h0, h0
+; CHECK-FP16-NEXT: ret
+
 define half @test_rint(half %a) #0 {
   %r = call half @llvm.rint.f16(half %a)
   ret half %r
 }
 
-; CHECK-LABEL: test_nearbyint:
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: frinti s0, s0
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_nearbyint:
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: frinti s0, s0
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_nearbyint:
+; CHECK-FP16-NEXT: frinti h0, h0
+; CHECK-FP16-NEXT: ret
+
 define half @test_nearbyint(half %a) #0 {
   %r = call half @llvm.nearbyint.f16(half %a)
   ret half %r
 }
 
-; CHECK-LABEL: test_round:
-; CHECK-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0
-; CHECK-NEXT: frinta [[INT32:s[0-9]+]], [[FLOAT32]]
-; CHECK-NEXT: fcvt h0, [[INT32]]
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_round:
+; CHECK-CVT-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0
+; CHECK-CVT-NEXT: frinta [[INT32:s[0-9]+]], [[FLOAT32]]
+; CHECK-CVT-NEXT: fcvt h0, [[INT32]]
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_round:
+; CHECK-FP16-NEXT: frinta h0, h0
+; CHECK-FP16-NEXT: ret
+
 define half @test_round(half %a) #0 {
   %r = call half @llvm.round.f16(half %a)
   ret half %r
 }
 
-; CHECK-LABEL: test_fmuladd:
-; CHECK-NEXT: fcvt s1, h1
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fmul s0, s0, s1
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: fcvt s0, h0
-; CHECK-NEXT: fcvt s1, h2
-; CHECK-NEXT: fadd s0, s0, s1
-; CHECK-NEXT: fcvt h0, s0
-; CHECK-NEXT: ret
+; CHECK-CVT-LABEL: test_fmuladd:
+; CHECK-CVT-NEXT: fcvt s1, h1
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fmul s0, s0, s1
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: fcvt s0, h0
+; CHECK-CVT-NEXT: fcvt s1, h2
+; CHECK-CVT-NEXT: fadd s0, s0, s1
+; CHECK-CVT-NEXT: fcvt h0, s0
+; CHECK-CVT-NEXT: ret
+
+; CHECK-FP16-LABEL: test_fmuladd:
+; CHECK-FP16-NEXT: fmul h0, h0, h1
+; CHECK-FP16-NEXT: fadd h0, h0, h2
+; CHECK-FP16-NEXT: ret
+
 define half @test_fmuladd(half %a, half %b, half %c) #0 {
   %r = call half @llvm.fmuladd.f16(half %a, half %b, half %c)
   ret half %r
 }
 
 attributes #0 = { nounwind }