Support __fp16 vectors

[ahatanak]

Summary

Currently, clang miscompiles operations on __fp16 vectors.

For example, when the following code is compiled:

typedef __fp16 half4 __attribute__ ((vector_size (8)));
half4 hv0, hv1, hv2;

void test() {
  hv0 = hv1 + hv2;
}

clang generates the following IR on ARM64:

%1 = load <4 x half>, <4 x half>* @hv1, align 8
%2 = load <4 x half>, <4 x half>* @hv2, align 8
%3 = fadd <4 x half> %1, %2
store <4 x half> %3, <4 x half>* @hv0, align 8

This isn't correct since fp16 values in C or C++ expressions have to be promoted to float if fp16 is not a natively supported type (see gcc's documentation).

https://gcc.gnu.org/onlinedocs/gcc/Half-Precision.html

The IR is incorrect on X86 too. The addition is done on <4xi16>vectors:

%1 = load <4 x i16>, <4 x i16>* @hv1, align 8
%2 = load <4 x i16>, <4 x i16>* @hv2, align 8
%3 = add <4 x i16> %2, %1
store <4 x i16> %3, <4 x i16>* @hv0, align 8

This patch makes the changes needed in Sema and IRGen to generate the correct IR on targets that set HalfArgsAndReturns to true but don't support fp16 natively (ARM and ARM64). It inserts implicit casts to promote fp16 vector operands to float vectors and truncate the result back to a __fp16 vector.

I plan to fix X86 and other targets that don't set HalfArgsAndReturns to true in another patch.

[ahatanak]

ping

[ahatanak]

Rebase.

[bruno]

Hi Akira,

This is nice, thanks for doing it!

include/clang/Sema/Sema.h

9270 ↗

(On Diff #100354)

Can you change CompAssign to IsCompAssign so that we match the pattern used in other methods?

9279 ↗

(On Diff #100354)

Same here.

lib/CodeGen/CGExprScalar.cpp

1017–1051

Please also add a comment explaining what's going on here, like we see for other snippets of logic above.

It also sounds like this is more generic than it should (which can have unexpected side effects due to the lack of testcases covering vector with other element sizes). I suggest you either (a) add testcases for other sizes or (b) make the condition more restrictive to be sure you're only changing the logic you intend to (i.e., half and i16).

After these changes, if it makes sense, can you refactor the logic under this condition into its own function? Seems like this function is too big already.

lib/Sema/SemaExpr.cpp

11589

Assuming we're able to handle other vector types here, is it in ConvertHalfVec really necessary? It seems odd to me that we need to special case it in every case below.

test/Sema/fp16vec-sema.c

48

Can you add a FIXME here?

[ahatanak]

Address review comments.

[ahatanak]

lib/CodeGen/CGExprScalar.cpp

1017–1051

I'm surprised that we don't have many code-gen tests for vectors, but I can add more test cases that are the same as the tests in fp16vec-ops.c except that the element types are different.

Just to be clear, the only change I'm making here is to handle cases in which the source and destination have different sizes. That shouldn't happen when for example an i8 vector is converted to an i32 because if it did happen, that would have previously caused an assertion when calling CreateBitCast (bitcast requires the sizes of the destination and source be the same).

Also, I removed the lambda and instead used Type::getPrimitiveSizeInBit to compute the vector size.

If this looks OK, I'll try refactoring this function.

lib/Sema/SemaExpr.cpp

11589

ConvertHalfVec is needed to distinguish operations that involve vectors of half from those that don't. If the operands are vectors of half, convertHalfVecBinOp is called to promote the operands and truncate the result.

[ahatanak]

Rebase.

[bruno]

lib/CodeGen/CGExprScalar.cpp

1042

What happens if the SrcElementTy is float and DstElementTy isn't? Seems like it will hit the assertion below.

lib/Sema/SemaExpr.cpp

7594

Since it seems that you're always doing the same conversion (the only variable input here is the number of elements), can you update the comment to mention the exact conversion?

8143

What about Convert vector E to a vector with the same number of elements but different element type?

11496

which are of a half vector type -> should be there an assertion below to make sure?

11509

Can you add a comment explaining that a) this conversion from float -> int and b) it's needed in case the original binop is a comparison?

11740

What about of half vector and truncating the result to of half vector to float and truncating the result back to half

11771–11775

Please add a comment here explaining that this path only happens when it's a compound assignment.

12190

This same logic is used elsewhere in the patch, perhaps factor it out into a static function?

[ahatanak]

Reopening this review. I accidentally closed this review when I committed r313717 (the patch that adds support for 'noescape').

[ahatanak]

Upload the rebased patch again.

[ahatanak]

Address review comments.

lib/CodeGen/CGExprScalar.cpp

1042

That's not supposed to happen since there are only three cases we have to consider here:

1. truncation from an int vector to a short vector
2. promotion from a half vector to a float vector
3. truncation from a float vector to a half vector

I've cleaned up the assertions and moved them up to make the code clearer.

lib/Sema/SemaExpr.cpp

7594

It turns out this code isn't needed at all. I accidentally left the code in that I had in my local branch. The conversions between half vectors and float vectors take place after the operands are checked, so both the LHS and the RHS are still half vectors at this point.

Sorry for causing confusion.

11740

I also changed the comment in CreateBuiltinUnaryOp.

12190

I factored it out to function 'needsConversionOfHalfVec'.

[bruno]

Thanks Akira.

lib/CodeGen/CGExprScalar.cpp

assert(DstType->castAs<ExtVectorType>()->getElementType().getTypePtr() ==
                SrcType.getTypePtr() &&
            "Splatted expr doesn't match with vector element type?");
 
     // Splat the element across to all elements
     unsigned NumElements = DstTy->getVectorNumElements();
     return Builder.CreateVectorSplat(NumElements, Src, "splat");
   }
 
+  if (isa<llvm::VectorType>(SrcTy) || isa<llvm::VectorType>(DstTy)) {
     // Allow bitcast from vector to integer/fp of the same size.
-  if (isa<llvm::VectorType>(SrcTy) ||
-      isa<llvm::VectorType>(DstTy))
+    unsigned SrcSize = SrcTy->getPrimitiveSizeInBits();
+    unsigned DstSize = DstTy->getPrimitiveSizeInBits();
+    if (SrcSize == DstSize)
       return Builder.CreateBitCast(Src, DstTy, "conv");
 
+    // Conversions between vectors of different sizes are not allowed except
+    // when vectors of half are involved. Operations on storage-only half
+    // vectors require promoting half vector operands to float vectors and
+    // truncating the result, which is either an int or float vector, to a
+    // short or half vector.
+
+    // Source and destination are both expected to be vectors.
+    llvm::Type *SrcElementTy = SrcTy->getVectorElementType();
+    llvm::Type *DstElementTy = DstTy->getVectorElementType();
+
+    assert(((SrcElementTy->isIntegerTy() &&
+             DstElementTy->isIntegerTy()) ||
+            (SrcElementTy->isFloatingPointTy() &&
+             DstElementTy->isFloatingPointTy())) &&
+           "unexpected conversion between a floating-point vector and an "
+           "integer vector");
+
+    // Truncate an i32 vector to an i16 vector.
+    if (SrcElementTy->isIntegerTy())
+      return Builder.CreateIntCast(Src, DstTy, false, "conv");
+
+    // Truncate a float vector to a half vector.
+    if (SrcSize > DstSize)
+      return Builder.CreateFPTrunc(Src, DstTy, "conv");
+
+    // Promote a half vector to a float vector.
+    return Builder.CreateFPExt(Src, DstTy, "conv");
+  }
+
   // Finally, we have the arithmetic types: real int/float.
   Value *Res = nullptr;
   llvm::Type *ResTy = DstTy;
 
   // An overflowing conversion has undefined behavior if either the source type
   // or the destination type is a floating-point type.
   if (CGF.SanOpts.has(SanitizerKind::FloatCastOverflow) &&
       (OrigSrcType->isFloatingType() || DstType->isFloatingType()))

lib/Sema/SemaExpr.cpp

Sema::CheckAssignmentConstraints(SourceLocation Loc,
   // usually happen on valid code.
   OpaqueValueExpr RHSExpr(Loc, RHSType, VK_RValue);
   ExprResult RHSPtr = &RHSExpr;
   CastKind K = CK_Invalid;
 
   return CheckAssignmentConstraints(LHSType, RHSPtr, K, /*ConvertRHS=*/false);
 }
 
+/// This helper function returns true if QT is a vector type that has element
+/// type ElementType.
+static bool isVector(QualType QT, QualType ElementType) {
+  if (const VectorType *VT = QT->getAs<VectorType>())
+    return VT->getElementType() == ElementType;
+  return false;
+}
+
 /// CheckAssignmentConstraints (C99 6.5.16) - This routine currently
 /// has code to accommodate several GCC extensions when type checking
 /// pointers. Here are some objectionable examples that GCC considers warnings:
 ///
 ///  int a, *pint;
 ///  short *pshort;
 ///  struct foo *pfoo;
 ///
if (RHSType->isArithmeticType()) {
       return Compatible;
     }
   }
 
   // Conversions to or from vector type.
   if (LHSType->isVectorType() || RHSType->isVectorType()) {
     if (LHSType->isVectorType() && RHSType->isVectorType()) {
       // Allow assignments of an AltiVec vector type to an equivalent GCC
       // vector type and vice versa
       if (Context.areCompatibleVectorTypes(LHSType, RHSType)) {
         Kind = CK_BitCast;
         return Compatible;
       }
 
       // If we are allowing lax vector conversions, and LHS and RHS are both
       // vectors, the total size only needs to be the same. This is a bitcast;
       // no bits are changed but the result type is different.
static bool tryVectorConvertAndSplat(Sema &S, ExprResult *scalar,
   if (scalar) {
     if (scalarCast != CK_Invalid)
       *scalar = S.ImpCastExprToType(scalar->get(), vectorEltTy, scalarCast);
     *scalar = S.ImpCastExprToType(scalar->get(), vectorTy, CK_VectorSplat);
   }
   return false;
 }
 
+/// Convert vector E to a vector with the same number of elements but different
+/// element type.
+static ExprResult convertVector(Expr *E, QualType ElementType, Sema &S) {
+  const auto *VecTy = E->getType()->getAs<VectorType>();
+  assert(VecTy && "Expression E must be a vector");
+  QualType NewVecTy = S.Context.getVectorType(ElementType,
+                                              VecTy->getNumElements(),
+                                              VecTy->getVectorKind());
+
+  // Look through the implicit cast. Return the subexpression if its type is
+  // NewVecTy.
+  if (auto *ICE = dyn_cast<ImplicitCastExpr>(E))
+    if (ICE->getSubExpr()->getType() == NewVecTy)
+      return ICE->getSubExpr();
+
+  auto Cast = ElementType->isIntegerType() ? CK_IntegralCast : CK_FloatingCast;
+  return S.ImpCastExprToType(E, NewVecTy, Cast);
+}
+
 /// Test if a (constant) integer Int can be casted to another integer type
 /// IntTy without losing precision.
 static bool canConvertIntToOtherIntTy(Sema &S, ExprResult *Int,
                                       QualType OtherIntTy) {
   QualType IntTy = Int->get()->getType().getUnqualifiedType();
 
   // Reject cases where the value of the Int is unknown as that would
   // possibly cause truncation, but accept cases where the scalar can be
if (auto *DRE = dyn_cast<DeclRefExpr>(E))
     return DRE->getDecl();
   if (auto *ME = dyn_cast<MemberExpr>(E))
     return ME->getMemberDecl();
   if (auto *IRE = dyn_cast<ObjCIvarRefExpr>(E))
     return IRE->getDecl();
   return nullptr;
 }
 
+// This helper function promotes a binary operator's operands (which are of a
+// half vector type) to a vector of floats and then truncates the result to
+// a vector of either half or short.
+static ExprResult convertHalfVecBinOp(Sema &S, ExprResult LHS, ExprResult RHS,
+                                      BinaryOperatorKind Opc, QualType ResultTy,
+                                      ExprValueKind VK, ExprObjectKind OK,
+                                      bool IsCompAssign, SourceLocation OpLoc,
+                                      FPOptions FPFeatures) {
+  auto &Context = S.getASTContext();
+  assert((isVector(ResultTy, Context.HalfTy) ||
+          isVector(ResultTy, Context.ShortTy)) &&
+         "Result must be a vector of half or short");
+  assert(isVector(LHS.get()->getType(), Context.HalfTy) &&
+         isVector(RHS.get()->getType(), Context.HalfTy) &&
+         "both operands expected to be a half vector");
+
+  RHS = convertVector(RHS.get(), Context.FloatTy, S);
+  QualType BinOpResTy = RHS.get()->getType();
+
+  // If Opc is a comparison, ResultType is a vector of shorts. In that case,
+  // change BinOpResTy to a vector of ints.
+  if (isVector(ResultTy, Context.ShortTy))
+    BinOpResTy = S.GetSignedVectorType(BinOpResTy);
+
+  if (IsCompAssign)
+    return new (Context) CompoundAssignOperator(
+        LHS.get(), RHS.get(), Opc, ResultTy, VK, OK, BinOpResTy, BinOpResTy,
+        OpLoc, FPFeatures);
+
+  LHS = convertVector(LHS.get(), Context.FloatTy, S);
+  auto *BO = new (Context) BinaryOperator(LHS.get(), RHS.get(), Opc, BinOpResTy,
+                                          VK, OK, OpLoc, FPFeatures);
+  return convertVector(BO, ResultTy->getAs<VectorType>()->getElementType(), S);
+}
+
 static std::pair<ExprResult, ExprResult>
 CorrectDelayedTyposInBinOp(Sema &S, BinaryOperatorKind Opc, Expr *LHSExpr,
                            Expr *RHSExpr) {
   ExprResult LHS = LHSExpr, RHS = RHSExpr;
   if (!S.getLangOpts().CPlusPlus) {
     // C cannot handle TypoExpr nodes on either side of a binop because it
     // doesn't handle dependent types properly, so make sure any TypoExprs have
     // been dealt with before checking the operands.
     LHS = S.CorrectDelayedTyposInExpr(LHS);
     RHS = S.CorrectDelayedTyposInExpr(RHS, [Opc, LHS](Expr *E) {
       if (Opc != BO_Assign)
         return ExprResult(E);
       // Avoid correcting the RHS to the same Expr as the LHS.
       Decl *D = getDeclFromExpr(E);
       return (D && D == getDeclFromExpr(LHS.get())) ? ExprError() : E;
     });
   }
   return std::make_pair(LHS, RHS);
 }
 
+/// Returns true if conversion between vectors of halfs and vectors of floats
+/// is needed.
+static bool needsConversionOfHalfVec(bool OpRequiresConversion, ASTContext &Ctx,
+                                     QualType SrcType) {
+  return OpRequiresConversion && !Ctx.getLangOpts().NativeHalfType &&
+         Ctx.getLangOpts().HalfArgsAndReturns && isVector(SrcType, Ctx.HalfTy);
+}
+
 /// CreateBuiltinBinOp - Creates a new built-in binary operation with
 /// operator @p Opc at location @c TokLoc. This routine only supports
 /// built-in operations; ActOnBinOp handles overloaded operators.
 ExprResult Sema::CreateBuiltinBinOp(SourceLocation OpLoc,
                                     BinaryOperatorKind Opc,
                                     Expr *LHSExpr, Expr *RHSExpr) {
   if (getLangOpts().CPlusPlus11 && isa<InitListExpr>(RHSExpr)) {
     // The syntax only allows initializer lists on the RHS of assignment,
ExprResult Sema::CreateBuiltinBinOp(SourceLocation OpLoc,
 
   ExprResult LHS = LHSExpr, RHS = RHSExpr;
   QualType ResultTy;     // Result type of the binary operator.
   // The following two variables are used for compound assignment operators
   QualType CompLHSTy;    // Type of LHS after promotions for computation
   QualType CompResultTy; // Type of computation result
   ExprValueKind VK = VK_RValue;
   ExprObjectKind OK = OK_Ordinary;
+  bool ConvertHalfVec = false;
 
   std::tie(LHS, RHS) = CorrectDelayedTyposInBinOp(*this, Opc, LHSExpr, RHSExpr);
   if (!LHS.isUsable() || !RHS.isUsable())
     return ExprError();
 
   if (getLangOpts().OpenCL) {
     QualType LHSTy = LHSExpr->getType();
     QualType RHSTy = RHSExpr->getType();
case BO_Assign:
     break;
   case BO_PtrMemD:
   case BO_PtrMemI:
     ResultTy = CheckPointerToMemberOperands(LHS, RHS, VK, OpLoc,
                                             Opc == BO_PtrMemI);
     break;
   case BO_Mul:
   case BO_Div:
+    ConvertHalfVec = true;
     ResultTy = CheckMultiplyDivideOperands(LHS, RHS, OpLoc, false,
                                            Opc == BO_Div);
     break;
   case BO_Rem:
     ResultTy = CheckRemainderOperands(LHS, RHS, OpLoc);
     break;
   case BO_Add:
+    ConvertHalfVec = true;
     ResultTy = CheckAdditionOperands(LHS, RHS, OpLoc, Opc);
     break;
   case BO_Sub:
+    ConvertHalfVec = true;
     ResultTy = CheckSubtractionOperands(LHS, RHS, OpLoc);
     break;
   case BO_Shl:
   case BO_Shr:
     ResultTy = CheckShiftOperands(LHS, RHS, OpLoc, Opc);
     break;
   case BO_LE:
   case BO_LT:
   case BO_GE:
   case BO_GT:
+    ConvertHalfVec = true;
     ResultTy = CheckCompareOperands(LHS, RHS, OpLoc, Opc, true);
     break;
   case BO_EQ:
   case BO_NE:
+    ConvertHalfVec = true;
     ResultTy = CheckCompareOperands(LHS, RHS, OpLoc, Opc, false);
     break;
   case BO_And:
     checkObjCPointerIntrospection(*this, LHS, RHS, OpLoc);
     LLVM_FALLTHROUGH;
   case BO_Xor:
   case BO_Or:
     ResultTy = CheckBitwiseOperands(LHS, RHS, OpLoc, Opc);
     break;
   case BO_LAnd:
   case BO_LOr:
+    ConvertHalfVec = true;
     ResultTy = CheckLogicalOperands(LHS, RHS, OpLoc, Opc);
     break;
   case BO_MulAssign:
   case BO_DivAssign:
+    ConvertHalfVec = true;
     CompResultTy = CheckMultiplyDivideOperands(LHS, RHS, OpLoc, true,
                                                Opc == BO_DivAssign);
     CompLHSTy = CompResultTy;
     if (!CompResultTy.isNull() && !LHS.isInvalid() && !RHS.isInvalid())
       ResultTy = CheckAssignmentOperands(LHS.get(), RHS, OpLoc, CompResultTy);
     break;
   case BO_RemAssign:
     CompResultTy = CheckRemainderOperands(LHS, RHS, OpLoc, true);
     CompLHSTy = CompResultTy;
     if (!CompResultTy.isNull() && !LHS.isInvalid() && !RHS.isInvalid())
       ResultTy = CheckAssignmentOperands(LHS.get(), RHS, OpLoc, CompResultTy);
     break;
   case BO_AddAssign:
+    ConvertHalfVec = true;
     CompResultTy = CheckAdditionOperands(LHS, RHS, OpLoc, Opc, &CompLHSTy);
     if (!CompResultTy.isNull() && !LHS.isInvalid() && !RHS.isInvalid())
       ResultTy = CheckAssignmentOperands(LHS.get(), RHS, OpLoc, CompResultTy);
     break;
   case BO_SubAssign:
+    ConvertHalfVec = true;
     CompResultTy = CheckSubtractionOperands(LHS, RHS, OpLoc, &CompLHSTy);
     if (!CompResultTy.isNull() && !LHS.isInvalid() && !RHS.isInvalid())
       ResultTy = CheckAssignmentOperands(LHS.get(), RHS, OpLoc, CompResultTy);
     break;
   case BO_ShlAssign:
   case BO_ShrAssign:
     CompResultTy = CheckShiftOperands(LHS, RHS, OpLoc, Opc, true);
     CompLHSTy = CompResultTy;
if (getLangOpts().CPlusPlus && !RHS.isInvalid()) {
       VK = RHS.get()->getValueKind();
       OK = RHS.get()->getObjectKind();
     }
     break;
   }
   if (ResultTy.isNull() || LHS.isInvalid() || RHS.isInvalid())
     return ExprError();
 
+  // Some of the binary operations require promoting operands of half vector to
+  // float vectors and truncating the result back to half vector. For now, we do
+  // this only when HalfArgsAndReturn is set (that is, when the target is arm or
+  // arm64).
+  assert(isVector(RHS.get()->getType(), Context.HalfTy) ==
+         isVector(LHS.get()->getType(), Context.HalfTy) &&
+         "both sides are half vectors or neither sides are");
+  ConvertHalfVec = needsConversionOfHalfVec(ConvertHalfVec, Context,
+                                            LHS.get()->getType());
+
   // Check for array bounds violations for both sides of the BinaryOperator
   CheckArrayAccess(LHS.get());
   CheckArrayAccess(RHS.get());
 
   if (const ObjCIsaExpr *OISA = dyn_cast<ObjCIsaExpr>(LHS.get()->IgnoreParenCasts())) {
     NamedDecl *ObjectSetClass = LookupSingleName(TUScope,
                                                  &Context.Idents.get("object_setClass"),
                                                  SourceLocation(), LookupOrdinaryName);
     if (ObjectSetClass && isa<ObjCIsaExpr>(LHS.get())) {
       SourceLocation RHSLocEnd = getLocForEndOfToken(RHS.get()->getLocEnd());
       Diag(LHS.get()->getExprLoc(), diag::warn_objc_isa_assign) <<
       FixItHint::CreateInsertion(LHS.get()->getLocStart(), "object_setClass(") <<
       FixItHint::CreateReplacement(SourceRange(OISA->getOpLoc(), OpLoc), ",") <<
       FixItHint::CreateInsertion(RHSLocEnd, ")");
     }
     else
       Diag(LHS.get()->getExprLoc(), diag::warn_objc_isa_assign);
   }
   else if (const ObjCIvarRefExpr *OIRE =
            dyn_cast<ObjCIvarRefExpr>(LHS.get()->IgnoreParenCasts()))
     DiagnoseDirectIsaAccess(*this, OIRE, OpLoc, RHS.get());
   
-  if (CompResultTy.isNull())
+  // Opc is not a compound assignment if CompResultTy is null.
+  if (CompResultTy.isNull()) {
+    if (ConvertHalfVec)
+      return convertHalfVecBinOp(*this, LHS, RHS, Opc, ResultTy, VK, OK, false,
+                                 OpLoc, FPFeatures);
     return new (Context) BinaryOperator(LHS.get(), RHS.get(), Opc, ResultTy, VK,
                                         OK, OpLoc, FPFeatures);
+  }
+
+  // Handle compound assignments.
   if (getLangOpts().CPlusPlus && LHS.get()->getObjectKind() !=
       OK_ObjCProperty) {
     VK = VK_LValue;
     OK = LHS.get()->getObjectKind();
   }
+
+  if (ConvertHalfVec)
+    return convertHalfVecBinOp(*this, LHS, RHS, Opc, ResultTy, VK, OK, true,
+                               OpLoc, FPFeatures);
+
   return new (Context) CompoundAssignOperator(
       LHS.get(), RHS.get(), Opc, ResultTy, VK, OK, CompLHSTy, CompResultTy,
       OpLoc, FPFeatures);
 }
 
 /// DiagnoseBitwisePrecedence - Emit a warning when bitwise and comparison
 /// operators are mixed in a way that suggests that the programmer forgot that
 /// comparison operators have higher precedence. The most typical example of

 
 ExprResult Sema::CreateBuiltinUnaryOp(SourceLocation OpLoc,
                                       UnaryOperatorKind Opc,
                                       Expr *InputExpr) {
   ExprResult Input = InputExpr;
   ExprValueKind VK = VK_RValue;
   ExprObjectKind OK = OK_Ordinary;
   QualType resultType;
+  bool ConvertHalfVec = false;
   if (getLangOpts().OpenCL) {
     QualType Ty = InputExpr->getType();
     // The only legal unary operation for atomics is '&'.
     if ((Opc != UO_AddrOf && Ty->isAtomicType()) ||
     // OpenCL special types - image, sampler, pipe, and blocks are to be used
     // only with a builtin functions and therefore should be disallowed here.
         (Ty->isImageType() || Ty->isSamplerT() || Ty->isPipeType()
         || Ty->isBlockPointerType())) {
case UO_Deref: {
     if (Input.isInvalid()) return ExprError();
     resultType = CheckIndirectionOperand(*this, Input.get(), VK, OpLoc);
     break;
   }
   case UO_Plus:
   case UO_Minus:
     Input = UsualUnaryConversions(Input.get());
     if (Input.isInvalid()) return ExprError();
+    // Unary plus and minus require promoting an operand of half vector to a
+    // float vector and truncating the result back to a half vector. For now, we
+    // do this only when HalfArgsAndReturns is set (that is, when the target is
+    // arm or arm64).
+    ConvertHalfVec =
+        needsConversionOfHalfVec(true, Context, Input.get()->getType());
+
+    // If the operand is a half vector, promote it to a float vector.
+    if (ConvertHalfVec)
+      Input = convertVector(Input.get(), Context.FloatTy, *this);
     resultType = Input.get()->getType();
     if (resultType->isDependentType())
       break;
     if (resultType->isArithmeticType()) // C99 6.5.3.3p1
       break;
     else if (resultType->isVectorType() &&
              // The z vector extensions don't allow + or - with bool vectors.
              (!Context.getLangOpts().ZVector ||
ExprResult Sema::CreateBuiltinUnaryOp(SourceLocation OpLoc,
 
   // Check for array bounds violations in the operand of the UnaryOperator,
   // except for the '*' and '&' operators that have to be handled specially
   // by CheckArrayAccess (as there are special cases like &array[arraysize]
   // that are explicitly defined as valid by the standard).
   if (Opc != UO_AddrOf && Opc != UO_Deref)
     CheckArrayAccess(Input.get());
 
-  return new (Context)
+  auto *UO = new (Context)
       UnaryOperator(Input.get(), Opc, resultType, VK, OK, OpLoc);
+  // Convert the result back to a half vector.
+  if (ConvertHalfVec)
+    return convertVector(UO, Context.HalfTy, *this);
+  return UO;
 }
 
 /// \brief Determine whether the given expression is a qualified member
 /// access expression, of a form that could be turned into a pointer to member
 /// with the address-of operator.
 static bool isQualifiedMemberAccess(Expr *E) {
   if (DeclRefExpr *DRE = dyn_cast<DeclRefExpr>(E)) {
     if (!DRE->getQualifier())

test/CodeGen/fp16vec-ops.c

+// REQUIRES: arm-registered-target
+// RUN: %clang_cc1 -triple arm64-apple-ios9 -emit-llvm -o - -fallow-half-arguments-and-returns %s | FileCheck %s --check-prefix=CHECK
+// RUN: %clang_cc1 -triple armv7-apple-ios9 -emit-llvm -o - -fallow-half-arguments-and-returns %s | FileCheck %s --check-prefix=CHECK
+
+typedef __fp16 half4 __attribute__ ((vector_size (8)));
+typedef short short4 __attribute__ ((vector_size (8)));
+
+half4 hv0, hv1;
+short4 sv0;
+
+// CHECK-LABEL: testFP16Vec0
+// CHECK: %[[V0:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV:.*]] = fpext <4 x half> %[[V0]] to <4 x float>
+// CHECK: %[[V1:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV1:.*]] = fpext <4 x half> %[[V1]] to <4 x float>
+// CHECK: %[[ADD:.*]] = fadd <4 x float> %[[CONV]], %[[CONV1]]
+// CHECK: %[[CONV2:.*]] = fptrunc <4 x float> %[[ADD]] to <4 x half>
+// CHECK: store <4 x half> %[[CONV2]], <4 x half>* @hv0, align 8
+// CHECK: %[[V2:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV3:.*]] = fpext <4 x half> %[[V2]] to <4 x float>
+// CHECK: %[[V3:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV4:.*]] = fpext <4 x half> %[[V3]] to <4 x float>
+// CHECK: %[[SUB:.*]] = fsub <4 x float> %[[CONV3]], %[[CONV4]]
+// CHECK: %[[CONV5:.*]] = fptrunc <4 x float> %[[SUB]] to <4 x half>
+// CHECK: store <4 x half> %[[CONV5]], <4 x half>* @hv0, align 8
+// CHECK: %[[V4:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV6:.*]] = fpext <4 x half> %[[V4]] to <4 x float>
+// CHECK: %[[V5:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV7:.*]] = fpext <4 x half> %[[V5]] to <4 x float>
+// CHECK: %[[MUL:.*]] = fmul <4 x float> %[[CONV6]], %[[CONV7]]
+// CHECK: %[[CONV8:.*]] = fptrunc <4 x float> %[[MUL]] to <4 x half>
+// CHECK: store <4 x half> %[[CONV8]], <4 x half>* @hv0, align 8
+// CHECK: %[[V6:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV9:.*]] = fpext <4 x half> %[[V6]] to <4 x float>
+// CHECK: %[[V7:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV10:.*]] = fpext <4 x half> %[[V7]] to <4 x float>
+// CHECK: %[[DIV:.*]] = fdiv <4 x float> %[[CONV9]], %[[CONV10]]
+// CHECK: %[[CONV11:.*]] = fptrunc <4 x float> %[[DIV]] to <4 x half>
+// CHECK: store <4 x half> %[[CONV11]], <4 x half>* @hv0, align 8
+
+void testFP16Vec0() {
+  hv0 = hv0 + hv1;
+  hv0 = hv0 - hv1;
+  hv0 = hv0 * hv1;
+  hv0 = hv0 / hv1;
+}
+
+// CHECK-LABEL: testFP16Vec1
+// CHECK: %[[V0:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV:.*]] = fpext <4 x half> %[[V0]] to <4 x float>
+// CHECK: %[[V1:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV1:.*]] = fpext <4 x half> %[[V1]] to <4 x float>
+// CHECK: %[[ADD:.*]] = fadd <4 x float> %[[CONV1]], %[[CONV]]
+// CHECK: %[[CONV2:.*]] = fptrunc <4 x float> %[[ADD]] to <4 x half>
+// CHECK: store <4 x half> %[[CONV2]], <4 x half>* @hv0, align 8
+// CHECK: %[[V2:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV3:.*]] = fpext <4 x half> %[[V2]] to <4 x float>
+// CHECK: %[[V3:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV4:.*]] = fpext <4 x half> %[[V3]] to <4 x float>
+// CHECK: %[[SUB:.*]] = fsub <4 x float> %[[CONV4]], %[[CONV3]]
+// CHECK: %[[CONV5:.*]] = fptrunc <4 x float> %[[SUB]] to <4 x half>
+// CHECK: store <4 x half> %[[CONV5]], <4 x half>* @hv0, align 8
+// CHECK: %[[V4:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV6:.*]] = fpext <4 x half> %[[V4]] to <4 x float>
+// CHECK: %[[V5:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV7:.*]] = fpext <4 x half> %[[V5]] to <4 x float>
+// CHECK: %[[MUL:.*]] = fmul <4 x float> %[[CONV7]], %[[CONV6]]
+// CHECK: %[[CONV8:.*]] = fptrunc <4 x float> %[[MUL]] to <4 x half>
+// CHECK: store <4 x half> %[[CONV8]], <4 x half>* @hv0, align 8
+// CHECK: %[[V6:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV9:.*]] = fpext <4 x half> %[[V6]] to <4 x float>
+// CHECK: %[[V7:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV10:.*]] = fpext <4 x half> %[[V7]] to <4 x float>
+// CHECK: %[[DIV:.*]] = fdiv <4 x float> %[[CONV10]], %[[CONV9]]
+// CHECK: %[[CONV11:.*]] = fptrunc <4 x float> %[[DIV]] to <4 x half>
+// CHECK: store <4 x half> %[[CONV11]], <4 x half>* @hv0, align 8
+
+void testFP16Vec1() {
+  hv0 += hv1;
+  hv0 -= hv1;
+  hv0 *= hv1;
+  hv0 /= hv1;
+}
+
+// CHECK-LABEL: testFP16Vec2
+// CHECK: %[[CADDR:.*]] = alloca i32, align 4
+// CHECK: store i32 %[[C:.*]], i32* %[[CADDR]], align 4
+// CHECK: %[[V0:.*]] = load i32, i32* %[[CADDR]], align 4
+// CHECK: %[[TOBOOL:.*]] = icmp ne i32 %[[V0]], 0
+// CHECK: br i1 %[[TOBOOL]], label %{{.*}}, label %{{.*}}
+//
+// CHECK: %[[V1:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: br label %{{.*}}
+//
+// CHECK: %[[V2:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: br label %{{.*}}
+//
+// CHECK: %[[COND:.*]] = phi <4 x half> [ %[[V1]], %{{.*}} ], [ %[[V2]], %{{.*}} ]
+// CHECK: store <4 x half> %[[COND]], <4 x half>* @hv0, align 8
+
+void testFP16Vec2(int c) {
+  hv0 = c ? hv0 : hv1;
+}
+
+// CHECK-LABEL: testFP16Vec3
+// CHECK: %[[V0:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV:.*]] = fpext <4 x half> %[[V0]] to <4 x float>
+// CHECK: %[[V1:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV1:.*]] = fpext <4 x half> %[[V1]] to <4 x float>
+// CHECK: %[[CMP:.*]] = fcmp oeq <4 x float> %[[CONV]], %[[CONV1]]
+// CHECK: %[[SEXT:.*]] = sext <4 x i1> %[[CMP]] to <4 x i32>
+// CHECK: %[[CONV2:.*]] = trunc <4 x i32> %[[SEXT]] to <4 x i16>
+// CHECK: store <4 x i16> %[[CONV2]], <4 x i16>* @sv0, align 8
+// CHECK: %[[V2:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV3:.*]] = fpext <4 x half> %[[V2]] to <4 x float>
+// CHECK: %[[V3:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV4:.*]] = fpext <4 x half> %[[V3]] to <4 x float>
+// CHECK: %[[CMP5:.*]] = fcmp une <4 x float> %[[CONV3]], %[[CONV4]]
+// CHECK: %[[SEXT6:.*]] = sext <4 x i1> %[[CMP5]] to <4 x i32>
+// CHECK: %[[CONV7:.*]] = trunc <4 x i32> %[[SEXT6]] to <4 x i16>
+// CHECK: store <4 x i16> %[[CONV7]], <4 x i16>* @sv0, align 8
+// CHECK: %[[V4:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV8:.*]] = fpext <4 x half> %[[V4]] to <4 x float>
+// CHECK: %[[V5:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV9:.*]] = fpext <4 x half> %[[V5]] to <4 x float>
+// CHECK: %[[CMP10:.*]] = fcmp olt <4 x float> %[[CONV8]], %[[CONV9]]
+// CHECK: %[[SEXT11:.*]] = sext <4 x i1> %[[CMP10]] to <4 x i32>
+// CHECK: %[[CONV12:.*]] = trunc <4 x i32> %[[SEXT11]] to <4 x i16>
+// CHECK: store <4 x i16> %[[CONV12]], <4 x i16>* @sv0, align 8
+// CHECK: %[[V6:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV13:.*]] = fpext <4 x half> %[[V6]] to <4 x float>
+// CHECK: %[[V7:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV14:.*]] = fpext <4 x half> %[[V7]] to <4 x float>
+// CHECK: %[[CMP15:.*]] = fcmp ogt <4 x float> %[[CONV13]], %[[CONV14]]
+// CHECK: %[[SEXT16:.*]] = sext <4 x i1> %[[CMP15]] to <4 x i32>
+// CHECK: %[[CONV17:.*]] = trunc <4 x i32> %[[SEXT16]] to <4 x i16>
+// CHECK: store <4 x i16> %[[CONV17]], <4 x i16>* @sv0, align 8
+// CHECK: %[[V8:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV18:.*]] = fpext <4 x half> %[[V8]] to <4 x float>
+// CHECK: %[[V9:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV19:.*]] = fpext <4 x half> %[[V9]] to <4 x float>
+// CHECK: %[[CMP20:.*]] = fcmp ole <4 x float> %[[CONV18]], %[[CONV19]]
+// CHECK: %[[SEXT21:.*]] = sext <4 x i1> %[[CMP20]] to <4 x i32>
+// CHECK: %[[CONV22:.*]] = trunc <4 x i32> %[[SEXT21]] to <4 x i16>
+// CHECK: store <4 x i16> %[[CONV22]], <4 x i16>* @sv0, align 8
+// CHECK: %[[V10:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV23:.*]] = fpext <4 x half> %[[V10]] to <4 x float>
+// CHECK: %[[V11:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV24:.*]] = fpext <4 x half> %[[V11]] to <4 x float>
+// CHECK: %[[CMP25:.*]] = fcmp oge <4 x float> %[[CONV23]], %[[CONV24]]
+// CHECK: %[[SEXT26:.*]] = sext <4 x i1> %[[CMP25]] to <4 x i32>
+// CHECK: %[[CONV27:.*]] = trunc <4 x i32> %[[SEXT26]] to <4 x i16>
+// CHECK: store <4 x i16> %[[CONV27]], <4 x i16>* @sv0, align 8
+
+void testFP16Vec3() {
+  sv0 = hv0 == hv1;
+  sv0 = hv0 != hv1;
+  sv0 = hv0 < hv1;
+  sv0 = hv0 > hv1;
+  sv0 = hv0 <= hv1;
+  sv0 = hv0 >= hv1;
+}

test/Sema/fp16vec-sema.c

+// RUN: %clang_cc1 -fsyntax-only -verify %s
+
+typedef __fp16 half4 __attribute__ ((vector_size (8)));
+typedef float float4 __attribute__ ((vector_size (16)));
+typedef short short4 __attribute__ ((vector_size (8)));
+typedef int int4 __attribute__ ((vector_size (16)));
+
+half4 hv0, hv1;
+float4 fv0, fv1;
+short4 sv0;
+int4 iv0;
+
+void testFP16Vec(int c) {
+  hv0 = hv0 + hv1;
+  hv0 = hv0 - hv1;
+  hv0 = hv0 * hv1;
+  hv0 = hv0 / hv1;
+  hv0 = c ? hv0 : hv1;
+  hv0 += hv1;
+  hv0 -= hv1;
+  hv0 *= hv1;
+  hv0 /= hv1;
+  sv0 = hv0 == hv1;
+  sv0 = hv0 != hv1;
+  sv0 = hv0 < hv1;
+  sv0 = hv0 > hv1;
+  sv0 = hv0 <= hv1;
+  sv0 = hv0 >= hv1;
+  sv0 = hv0 || hv1; // expected-error{{logical expression with vector types 'half4' (vector of 4 '__fp16' values) and 'half4' is only supported in C++}}
+  sv0 = hv0 && hv1; // expected-error{{logical expression with vector types 'half4' (vector of 4 '__fp16' values) and 'half4' is only supported in C++}}
+
+  // Implicit conversion between half vectors and float vectors are not allowed.
+  hv0 = fv0; // expected-error{{assigning to}}
+  fv0 = hv0; // expected-error{{assigning to}}
+  hv0 = (half4)fv0; // expected-error{{invalid conversion between}}
+  fv0 = (float4)hv0; // expected-error{{invalid conversion between}}
+  hv0 = fv0 + fv1; // expected-error{{assigning to}}
+  fv0 = hv0 + hv1; // expected-error{{assigning to}}
+  hv0 = hv0 + fv1; // expected-error{{cannot convert between vector}}
+  hv0 = c ? hv0 : fv1; // expected-error{{cannot convert between vector}}
+  sv0 = hv0 == fv1; // expected-error{{cannot convert between vector}}
+  sv0 = hv0 < fv1; // expected-error{{cannot convert between vector}}
+  sv0 = hv0 || fv1; // expected-error{{cannot convert between vector}} expected-error{{invalid operands to binary expression}}
+  iv0 = hv0 == hv1; // expected-error{{assigning to}}
+
+  // FIXME: clang currently disallows using these operators on vectors, which is
+  // allowed by gcc.
+  sv0 = !hv0; // expected-error{{invalid argument type}}
+  hv0++; // expected-error{{cannot increment value of type}}
+  ++hv0; // expected-error{{cannot increment value of type}}
+}