Support __fp16 vectors

[ahatanak]

Summary

Currently, clang miscompiles operations on __fp16 vectors.

For example, when the following code is compiled:

typedef __fp16 half4 __attribute__ ((vector_size (8)));
half4 hv0, hv1, hv2;

void test() {
  hv0 = hv1 + hv2;
}

clang generates the following IR on ARM64:

%1 = load <4 x half>, <4 x half>* @hv1, align 8
%2 = load <4 x half>, <4 x half>* @hv2, align 8
%3 = fadd <4 x half> %1, %2
store <4 x half> %3, <4 x half>* @hv0, align 8

This isn't correct since fp16 values in C or C++ expressions have to be promoted to float if fp16 is not a natively supported type (see gcc's documentation).

https://gcc.gnu.org/onlinedocs/gcc/Half-Precision.html

The IR is incorrect on X86 too. The addition is done on <4xi16>vectors:

%1 = load <4 x i16>, <4 x i16>* @hv1, align 8
%2 = load <4 x i16>, <4 x i16>* @hv2, align 8
%3 = add <4 x i16> %2, %1
store <4 x i16> %3, <4 x i16>* @hv0, align 8

This patch makes the changes needed in Sema and IRGen to generate the correct IR on targets that set HalfArgsAndReturns to true but don't support fp16 natively (ARM and ARM64). It inserts implicit casts to promote fp16 vector operands to float vectors and truncate the result back to a __fp16 vector.

I plan to fix X86 and other targets that don't set HalfArgsAndReturns to true in another patch.

[ahatanak]

ping

[ahatanak]

Rebase.

[bruno]

Hi Akira,

This is nice, thanks for doing it!

include/clang/Sema/Sema.h

9185

Can you change CompAssign to IsCompAssign so that we match the pattern used in other methods?

9194

Same here.

lib/CodeGen/CGExprScalar.cpp

931–960

Please also add a comment explaining what's going on here, like we see for other snippets of logic above.

It also sounds like this is more generic than it should (which can have unexpected side effects due to the lack of testcases covering vector with other element sizes). I suggest you either (a) add testcases for other sizes or (b) make the condition more restrictive to be sure you're only changing the logic you intend to (i.e., half and i16).

After these changes, if it makes sense, can you refactor the logic under this condition into its own function? Seems like this function is too big already.

lib/Sema/SemaExpr.cpp

11200

Assuming we're able to handle other vector types here, is it in ConvertHalfVec really necessary? It seems odd to me that we need to special case it in every case below.

test/Sema/fp16vec-sema.c

48

Can you add a FIXME here?

[ahatanak]

Address review comments.

[ahatanak]

lib/CodeGen/CGExprScalar.cpp

931–960

I'm surprised that we don't have many code-gen tests for vectors, but I can add more test cases that are the same as the tests in fp16vec-ops.c except that the element types are different.

Just to be clear, the only change I'm making here is to handle cases in which the source and destination have different sizes. That shouldn't happen when for example an i8 vector is converted to an i32 because if it did happen, that would have previously caused an assertion when calling CreateBitCast (bitcast requires the sizes of the destination and source be the same).

Also, I removed the lambda and instead used Type::getPrimitiveSizeInBit to compute the vector size.

If this looks OK, I'll try refactoring this function.

lib/Sema/SemaExpr.cpp

11200

ConvertHalfVec is needed to distinguish operations that involve vectors of half from those that don't. If the operands are vectors of half, convertHalfVecBinOp is called to promote the operands and truncate the result.

[ahatanak]

Rebase.

[bruno]

lib/CodeGen/CGExprScalar.cpp

956

What happens if the SrcElementTy is float and DstElementTy isn't? Seems like it will hit the assertion below.

lib/Sema/SemaExpr.cpp

7579

Since it seems that you're always doing the same conversion (the only variable input here is the number of elements), can you update the comment to mention the exact conversion?

8097

What about Convert vector E to a vector with the same number of elements but different element type?

11142

which are of a half vector type -> should be there an assertion below to make sure?

11155

Can you add a comment explaining that a) this conversion from float -> int and b) it's needed in case the original binop is a comparison?

11361

What about of half vector and truncating the result to of half vector to float and truncating the result back to half

11389–11392

Please add a comment here explaining that this path only happens when it's a compound assignment.

11777

This same logic is used elsewhere in the patch, perhaps factor it out into a static function?

[ahatanak]

Reopening this review. I accidentally closed this review when I committed r313717 (the patch that adds support for 'noescape').

[ahatanak]

Upload the rebased patch again.

[ahatanak]

Address review comments.

lib/CodeGen/CGExprScalar.cpp

956

That's not supposed to happen since there are only three cases we have to consider here:

1. truncation from an int vector to a short vector
2. promotion from a half vector to a float vector
3. truncation from a float vector to a half vector

I've cleaned up the assertions and moved them up to make the code clearer.

lib/Sema/SemaExpr.cpp

7579

It turns out this code isn't needed at all. I accidentally left the code in that I had in my local branch. The conversions between half vectors and float vectors take place after the operands are checked, so both the LHS and the RHS are still half vectors at this point.

Sorry for causing confusion.

11361

I also changed the comment in CreateBuiltinUnaryOp.

11777

I factored it out to function 'needsConversionOfHalfVec'.

[bruno]

Thanks Akira.

include/clang/Sema/Sema.h

public:
   void DiagnoseAssignmentEnum(QualType DstType, QualType SrcType,
                               Expr *SrcExpr);
 
   /// CheckAssignmentConstraints - Perform type checking for assignment,
   /// argument passing, variable initialization, and function return values.
   /// C99 6.5.16.
   AssignConvertType CheckAssignmentConstraints(SourceLocation Loc,
                                                QualType LHSType,
-                                               QualType RHSType);
+                                               QualType RHSType,
+                                               bool CompAssign = false);
 
   /// Check assignment constraints and optionally prepare for a conversion of
   /// the RHS to the LHS type. The conversion is prepared for if ConvertRHS
   /// is true.
   AssignConvertType CheckAssignmentConstraints(QualType LHSType,
                                                ExprResult &RHS,
                                                CastKind &Kind,
-                                               bool ConvertRHS = true);
+                                               bool ConvertRHS = true,
+                                               bool CompAssign = false);
 
   /// Check assignment constraints for an assignment of RHS to LHSType.
   ///
   /// \param LHSType The destination type for the assignment.
   /// \param RHS The source expression for the assignment.
   /// \param Diagnose If \c true, diagnostics may be produced when checking
   ///        for assignability. If a diagnostic is produced, \p RHS will be
   ///        set to ExprError(). Note that this function may still return

lib/CodeGen/CGExprScalar.cpp

assert(DstType->castAs<ExtVectorType>()->getElementType().getTypePtr() ==
                SrcType.getTypePtr() &&
            "Splatted expr doesn't match with vector element type?");
 
     // Splat the element across to all elements
     unsigned NumElements = DstTy->getVectorNumElements();
     return Builder.CreateVectorSplat(NumElements, Src, "splat");
   }
 
+  if (isa<llvm::VectorType>(SrcTy) || isa<llvm::VectorType>(DstTy)) {
+    auto GetSize = [](const llvm::Type *Ty) {
+      unsigned Size = Ty->getScalarSizeInBits();
+      if (const auto *VT = dyn_cast<llvm::VectorType>(Ty))
+        Size *= VT->getNumElements();
+      return Size;
+    };
+
     // Allow bitcast from vector to integer/fp of the same size.
-  if (isa<llvm::VectorType>(SrcTy) ||
-      isa<llvm::VectorType>(DstTy))
+    unsigned SrcSize = GetSize(SrcTy), DstSize = GetSize(DstTy);
+    if (SrcSize == DstSize)
       return Builder.CreateBitCast(Src, DstTy, "conv");
 
+    // Conversions between vectors of different sizes are not allowed except
+    // when vectors of half are involved.
+    assert(SrcTy->isVectorTy() && DstTy->isVectorTy() &&
+           "Both source and destination must be vectors");
+
+    if (SrcTy->isIntOrIntVectorTy()) {
+      assert(DstTy->isIntOrIntVectorTy() &&
+             "DstTy must be an integer vector too");
+      return Builder.CreateIntCast(Src, DstTy, false, "conv");
+    }
+
+    assert(SrcTy->isFPOrFPVectorTy() && DstTy->isFPOrFPVectorTy() &&
+           "Source and destination must be floating point vectors");
+    if (SrcSize > DstSize)
+      return Builder.CreateFPTrunc(Src, DstTy, "conv");
+    return Builder.CreateFPExt(Src, DstTy, "conv");
+  }
+
   // Finally, we have the arithmetic types: real int/float.
   Value *Res = nullptr;
   llvm::Type *ResTy = DstTy;
 
   // An overflowing conversion has undefined behavior if either the source type
   // or the destination type is a floating-point type.
   if (CGF.SanOpts.has(SanitizerKind::FloatCastOverflow) &&
       (OrigSrcType->isFloatingType() || DstType->isFloatingType()))

lib/Sema/SemaExpr.cpp

if (S.Context.typesAreCompatible(LHSType, RHSType))
     return Sema::Compatible;
   if (LHSType->isObjCQualifiedIdType() || RHSType->isObjCQualifiedIdType())
     return Sema::IncompatibleObjCQualifiedId;
   return Sema::IncompatiblePointer;
 }
 
 Sema::AssignConvertType
 Sema::CheckAssignmentConstraints(SourceLocation Loc,
-                                 QualType LHSType, QualType RHSType) {
+                                 QualType LHSType, QualType RHSType,
+                                 bool CompAssign) {
   // Fake up an opaque expression.  We don't actually care about what
   // cast operations are required, so if CheckAssignmentConstraints
   // adds casts to this they'll be wasted, but fortunately that doesn't
   // usually happen on valid code.
   OpaqueValueExpr RHSExpr(Loc, RHSType, VK_RValue);
   ExprResult RHSPtr = &RHSExpr;
   CastKind K = CK_Invalid;
 
-  return CheckAssignmentConstraints(LHSType, RHSPtr, K, /*ConvertRHS=*/false);
+  return CheckAssignmentConstraints(LHSType, RHSPtr, K, /*ConvertRHS=*/false,
+                                    CompAssign);
+}
+
+/// This helper function returns true if QT is a vector type that has element
+/// type ElementType.
+static bool isVector(QualType QT, QualType ElementType) {
+  if (const VectorType *VT = QT->getAs<VectorType>())
+    return VT->getElementType() == ElementType;
+  return false;
 }
 
 /// CheckAssignmentConstraints (C99 6.5.16) - This routine currently
 /// has code to accommodate several GCC extensions when type checking
 /// pointers. Here are some objectionable examples that GCC considers warnings:
 ///
 ///  int a, *pint;
 ///  short *pshort;
 ///  struct foo *pfoo;
 ///
 ///  pint = pshort; // warning: assignment from incompatible pointer type
 ///  a = pint; // warning: assignment makes integer from pointer without a cast
 ///  pint = a; // warning: assignment makes pointer from integer without a cast
 ///  pint = pfoo; // warning: assignment from incompatible pointer type
 ///
 /// As a result, the code for dealing with pointers is more complex than the
 /// C99 spec dictates.
 ///
 /// Sets 'Kind' for any result kind except Incompatible.
 Sema::AssignConvertType
 Sema::CheckAssignmentConstraints(QualType LHSType, ExprResult &RHS,
-                                 CastKind &Kind, bool ConvertRHS) {
+                                 CastKind &Kind, bool ConvertRHS,
+                                 bool CompAssign) {
   QualType RHSType = RHS.get()->getType();
   QualType OrigLHSType = LHSType;
 
   // Get canonical types.  We're not formatting these types, just comparing
   // them.
   LHSType = Context.getCanonicalType(LHSType).getUnqualifiedType();
   RHSType = Context.getCanonicalType(RHSType).getUnqualifiedType();
 
if (RHSType->isArithmeticType()) {
       Kind = CK_VectorSplat;
       return Compatible;
     }
   }
 
   // Conversions to or from vector type.
   if (LHSType->isVectorType() || RHSType->isVectorType()) {
     if (LHSType->isVectorType() && RHSType->isVectorType()) {
+      // If this is a compound assignment, allow converting the RHS to the type
+      // of the LHS.
+      if (CompAssign && isVector(LHSType, Context.HalfTy)) {
+        assert(isVector(RHSType, Context.FloatTy) &&
+               "RHS is not a vector of floats");
+        assert(LHSType->getAs<VectorType>()->getNumElements() ==
+               RHSType->getAs<VectorType>()->getNumElements() &&
+               "LHS and RHS have different number of elements");
+        Kind = CK_FloatingCast;
+        return Compatible;
+      }
+
       // Allow assignments of an AltiVec vector type to an equivalent GCC
       // vector type and vice versa
       if (Context.areCompatibleVectorTypes(LHSType, RHSType)) {
         Kind = CK_BitCast;
         return Compatible;
       }
 
       // If we are allowing lax vector conversions, and LHS and RHS are both
static bool tryVectorConvertAndSplat(Sema &S, ExprResult *scalar,
   if (scalar) {
     if (scalarCast != CK_Invalid)
       *scalar = S.ImpCastExprToType(scalar->get(), vectorEltTy, scalarCast);
     *scalar = S.ImpCastExprToType(scalar->get(), vectorTy, CK_VectorSplat);
   }
   return false;
 }
 
+/// Convert E, which is a vector, to a vector that has a different element
+/// type.
+static ExprResult convertVector(Expr *E, QualType ElementType, Sema &S) {
+  const auto *VecTy = E->getType()->getAs<VectorType>();
+  assert(VecTy && "Expression E must be a vector");
+  QualType NewVecTy = S.Context.getVectorType(ElementType,
+                                              VecTy->getNumElements(),
+                                              VecTy->getVectorKind());
+
+  // Look through the implicit cast. Return the subexpression if its type is
+  // NewVecTy.
+  if (auto *ICE = dyn_cast<ImplicitCastExpr>(E))
+    if (ICE->getSubExpr()->getType() == NewVecTy)
+      return ICE->getSubExpr();
+
+  auto Cast = ElementType->isIntegerType() ? CK_IntegralCast : CK_FloatingCast;
+  return S.ImpCastExprToType(E, NewVecTy, Cast);
+}
+
 QualType Sema::CheckVectorOperands(ExprResult &LHS, ExprResult &RHS,
                                    SourceLocation Loc, bool IsCompAssign,
                                    bool AllowBothBool,
                                    bool AllowBoolConversions) {
   if (!IsCompAssign) {
     LHS = DefaultFunctionArrayLvalueConversion(LHS.get());
     if (LHS.isInvalid())
       return QualType();
if (ConvTy == Compatible) {
           getCurFunction()->markSafeWeakUse(RHS.get());
 
       } else if (getLangOpts().ObjCAutoRefCount || getLangOpts().ObjCWeak) {
         checkUnsafeExprAssigns(Loc, LHSExpr, RHS.get());
       }
     }
   } else {
     // Compound assignment "x += y"
-    ConvTy = CheckAssignmentConstraints(Loc, LHSType, RHSType);
+    ConvTy = CheckAssignmentConstraints(Loc, LHSType, RHSType,
+                                        /*CompAssign*/true);
   }
 
   if (DiagnoseAssignmentResult(ConvTy, Loc, LHSType, RHSType,
                                RHS.get(), AA_Assigning))
     return QualType();
 
   CheckForNullPointerDereference(*this, LHSExpr);
 
if (auto *DRE = dyn_cast<DeclRefExpr>(E))
     return DRE->getDecl();
   if (auto *ME = dyn_cast<MemberExpr>(E))
     return ME->getMemberDecl();
   if (auto *IRE = dyn_cast<ObjCIvarRefExpr>(E))
     return IRE->getDecl();
   return nullptr;
 }
 
+// This helper function promotes a binary operator's operands (which are of a
+// half vector type) to a vector of floats and then truncates the result to
+// a vector of either half or short.
+static ExprResult convertHalfVecBinOp(Sema &S, ExprResult LHS, ExprResult RHS,
+                                      BinaryOperatorKind Opc, QualType ResultTy,
+                                      ExprValueKind VK, ExprObjectKind OK,
+                                      bool IsCompAssign, SourceLocation OpLoc,
+                                      FPOptions FPFeatures) {
+  auto &Context = S.getASTContext();
+  assert((isVector(ResultTy, Context.HalfTy) ||
+          isVector(ResultTy, Context.ShortTy)) &&
+         "Result must be a vector of half or short");
+  RHS = convertVector(RHS.get(), Context.FloatTy, S);
+  QualType BinOpResTy = RHS.get()->getType();
+  if (isVector(ResultTy, Context.ShortTy))
+    BinOpResTy = S.GetSignedVectorType(BinOpResTy);
+
+  if (IsCompAssign)
+    return new (Context) CompoundAssignOperator(
+        LHS.get(), RHS.get(), Opc, ResultTy, VK, OK, BinOpResTy, BinOpResTy,
+        OpLoc, FPFeatures);
+
+  LHS = convertVector(LHS.get(), Context.FloatTy, S);
+  auto *BO = new (Context) BinaryOperator(LHS.get(), RHS.get(), Opc, BinOpResTy,
+                                          VK, OK, OpLoc, FPFeatures);
+  return convertVector(BO, ResultTy->getAs<VectorType>()->getElementType(), S);
+}
+
 /// CreateBuiltinBinOp - Creates a new built-in binary operation with
 /// operator @p Opc at location @c TokLoc. This routine only supports
 /// built-in operations; ActOnBinOp handles overloaded operators.
 ExprResult Sema::CreateBuiltinBinOp(SourceLocation OpLoc,
                                     BinaryOperatorKind Opc,
                                     Expr *LHSExpr, Expr *RHSExpr) {
   if (getLangOpts().CPlusPlus11 && isa<InitListExpr>(RHSExpr)) {
     // The syntax only allows initializer lists on the RHS of assignment,
ExprResult Sema::CreateBuiltinBinOp(SourceLocation OpLoc,
 
   ExprResult LHS = LHSExpr, RHS = RHSExpr;
   QualType ResultTy;     // Result type of the binary operator.
   // The following two variables are used for compound assignment operators
   QualType CompLHSTy;    // Type of LHS after promotions for computation
   QualType CompResultTy; // Type of computation result
   ExprValueKind VK = VK_RValue;
   ExprObjectKind OK = OK_Ordinary;
+  bool ConvertHalfVec = false;
 
   if (!getLangOpts().CPlusPlus) {
     // C cannot handle TypoExpr nodes on either side of a binop because it
     // doesn't handle dependent types properly, so make sure any TypoExprs have
     // been dealt with before checking the operands.
     LHS = CorrectDelayedTyposInExpr(LHSExpr);
     RHS = CorrectDelayedTyposInExpr(RHSExpr, [Opc, LHS](Expr *E) {
       if (Opc != BO_Assign)
case BO_Assign:
     break;
   case BO_PtrMemD:
   case BO_PtrMemI:
     ResultTy = CheckPointerToMemberOperands(LHS, RHS, VK, OpLoc,
                                             Opc == BO_PtrMemI);
     break;
   case BO_Mul:
   case BO_Div:
+    ConvertHalfVec = true;
     ResultTy = CheckMultiplyDivideOperands(LHS, RHS, OpLoc, false,
                                            Opc == BO_Div);
     break;
   case BO_Rem:
     ResultTy = CheckRemainderOperands(LHS, RHS, OpLoc);
     break;
   case BO_Add:
+    ConvertHalfVec = true;
     ResultTy = CheckAdditionOperands(LHS, RHS, OpLoc, Opc);
     break;
   case BO_Sub:
+    ConvertHalfVec = true;
     ResultTy = CheckSubtractionOperands(LHS, RHS, OpLoc);
     break;
   case BO_Shl:
   case BO_Shr:
     ResultTy = CheckShiftOperands(LHS, RHS, OpLoc, Opc);
     break;
   case BO_LE:
   case BO_LT:
   case BO_GE:
   case BO_GT:
+    ConvertHalfVec = true;
     ResultTy = CheckCompareOperands(LHS, RHS, OpLoc, Opc, true);
     break;
   case BO_EQ:
   case BO_NE:
+    ConvertHalfVec = true;
     ResultTy = CheckCompareOperands(LHS, RHS, OpLoc, Opc, false);
     break;
   case BO_And:
     checkObjCPointerIntrospection(*this, LHS, RHS, OpLoc);
   case BO_Xor:
   case BO_Or:
     ResultTy = CheckBitwiseOperands(LHS, RHS, OpLoc, Opc);
     break;
   case BO_LAnd:
   case BO_LOr:
+    ConvertHalfVec = true;
     ResultTy = CheckLogicalOperands(LHS, RHS, OpLoc, Opc);
     break;
   case BO_MulAssign:
   case BO_DivAssign:
+    ConvertHalfVec = true;
     CompResultTy = CheckMultiplyDivideOperands(LHS, RHS, OpLoc, true,
                                                Opc == BO_DivAssign);
     CompLHSTy = CompResultTy;
     if (!CompResultTy.isNull() && !LHS.isInvalid() && !RHS.isInvalid())
       ResultTy = CheckAssignmentOperands(LHS.get(), RHS, OpLoc, CompResultTy);
     break;
   case BO_RemAssign:
     CompResultTy = CheckRemainderOperands(LHS, RHS, OpLoc, true);
     CompLHSTy = CompResultTy;
     if (!CompResultTy.isNull() && !LHS.isInvalid() && !RHS.isInvalid())
       ResultTy = CheckAssignmentOperands(LHS.get(), RHS, OpLoc, CompResultTy);
     break;
   case BO_AddAssign:
+    ConvertHalfVec = true;
     CompResultTy = CheckAdditionOperands(LHS, RHS, OpLoc, Opc, &CompLHSTy);
     if (!CompResultTy.isNull() && !LHS.isInvalid() && !RHS.isInvalid())
       ResultTy = CheckAssignmentOperands(LHS.get(), RHS, OpLoc, CompResultTy);
     break;
   case BO_SubAssign:
+    ConvertHalfVec = true;
     CompResultTy = CheckSubtractionOperands(LHS, RHS, OpLoc, &CompLHSTy);
     if (!CompResultTy.isNull() && !LHS.isInvalid() && !RHS.isInvalid())
       ResultTy = CheckAssignmentOperands(LHS.get(), RHS, OpLoc, CompResultTy);
     break;
   case BO_ShlAssign:
   case BO_ShrAssign:
     CompResultTy = CheckShiftOperands(LHS, RHS, OpLoc, Opc, true);
     CompLHSTy = CompResultTy;
if (getLangOpts().CPlusPlus && !RHS.isInvalid()) {
       VK = RHS.get()->getValueKind();
       OK = RHS.get()->getObjectKind();
     }
     break;
   }
   if (ResultTy.isNull() || LHS.isInvalid() || RHS.isInvalid())
     return ExprError();
 
+  bool LHSHalfVec = isVector(LHS.get()->getType(), Context.HalfTy);
+  assert(isVector(RHS.get()->getType(), Context.HalfTy) == LHSHalfVec &&
+         "both sides are half vectors or neither sides are");
+  ConvertHalfVec = ConvertHalfVec && LHSHalfVec &&
+                   !Context.getLangOpts().NativeHalfType &&
+                   Context.getLangOpts().HalfArgsAndReturns;
+
   // Check for array bounds violations for both sides of the BinaryOperator
   CheckArrayAccess(LHS.get());
   CheckArrayAccess(RHS.get());
 
   if (const ObjCIsaExpr *OISA = dyn_cast<ObjCIsaExpr>(LHS.get()->IgnoreParenCasts())) {
     NamedDecl *ObjectSetClass = LookupSingleName(TUScope,
                                                  &Context.Idents.get("object_setClass"),
                                                  SourceLocation(), LookupOrdinaryName);
     if (ObjectSetClass && isa<ObjCIsaExpr>(LHS.get())) {
       SourceLocation RHSLocEnd = getLocForEndOfToken(RHS.get()->getLocEnd());
       Diag(LHS.get()->getExprLoc(), diag::warn_objc_isa_assign) <<
       FixItHint::CreateInsertion(LHS.get()->getLocStart(), "object_setClass(") <<
       FixItHint::CreateReplacement(SourceRange(OISA->getOpLoc(), OpLoc), ",") <<
       FixItHint::CreateInsertion(RHSLocEnd, ")");
     }
     else
       Diag(LHS.get()->getExprLoc(), diag::warn_objc_isa_assign);
   }
   else if (const ObjCIvarRefExpr *OIRE =
            dyn_cast<ObjCIvarRefExpr>(LHS.get()->IgnoreParenCasts()))
     DiagnoseDirectIsaAccess(*this, OIRE, OpLoc, RHS.get());
   
-  if (CompResultTy.isNull())
+  if (CompResultTy.isNull()) {
+    if (ConvertHalfVec)
+      return convertHalfVecBinOp(*this, LHS, RHS, Opc, ResultTy, VK, OK, false,
+                                 OpLoc, FPFeatures);
     return new (Context) BinaryOperator(LHS.get(), RHS.get(), Opc, ResultTy, VK,
                                         OK, OpLoc, FPFeatures);
+  }
+
   if (getLangOpts().CPlusPlus && LHS.get()->getObjectKind() !=
       OK_ObjCProperty) {
     VK = VK_LValue;
     OK = LHS.get()->getObjectKind();
   }
+
+  if (ConvertHalfVec)
+    return convertHalfVecBinOp(*this, LHS, RHS, Opc, ResultTy, VK, OK, true,
+                               OpLoc, FPFeatures);
+
   return new (Context) CompoundAssignOperator(
       LHS.get(), RHS.get(), Opc, ResultTy, VK, OK, CompLHSTy, CompResultTy,
       OpLoc, FPFeatures);
 }
 
 /// DiagnoseBitwisePrecedence - Emit a warning when bitwise and comparison
 /// operators are mixed in a way that suggests that the programmer forgot that
 /// comparison operators have higher precedence. The most typical example of

 
 ExprResult Sema::CreateBuiltinUnaryOp(SourceLocation OpLoc,
                                       UnaryOperatorKind Opc,
                                       Expr *InputExpr) {
   ExprResult Input = InputExpr;
   ExprValueKind VK = VK_RValue;
   ExprObjectKind OK = OK_Ordinary;
   QualType resultType;
+  bool ConvertHalfVec = false;
   if (getLangOpts().OpenCL) {
     QualType Ty = InputExpr->getType();
     // The only legal unary operation for atomics is '&'.
     if ((Opc != UO_AddrOf && Ty->isAtomicType()) ||
     // OpenCL special types - image, sampler, pipe, and blocks are to be used
     // only with a builtin functions and therefore should be disallowed here.
         (Ty->isImageType() || Ty->isSamplerT() || Ty->isPipeType()
         || Ty->isBlockPointerType())) {
case UO_Deref: {
     if (Input.isInvalid()) return ExprError();
     resultType = CheckIndirectionOperand(*this, Input.get(), VK, OpLoc);
     break;
   }
   case UO_Plus:
   case UO_Minus:
     Input = UsualUnaryConversions(Input.get());
     if (Input.isInvalid()) return ExprError();
+    ConvertHalfVec =
+        !Context.getLangOpts().NativeHalfType &&
+        Context.getLangOpts().HalfArgsAndReturns &&
+        isVector(Input.get()->getType(), Context.HalfTy);
+
+    // If the operand is a half vector, promote it to a float vector.
+    if (ConvertHalfVec)
+      Input = convertVector(Input.get(), Context.FloatTy, *this);
     resultType = Input.get()->getType();
     if (resultType->isDependentType())
       break;
     if (resultType->isArithmeticType()) // C99 6.5.3.3p1
       break;
     else if (resultType->isVectorType() &&
              // The z vector extensions don't allow + or - with bool vectors.
              (!Context.getLangOpts().ZVector ||
ExprResult Sema::CreateBuiltinUnaryOp(SourceLocation OpLoc,
 
   // Check for array bounds violations in the operand of the UnaryOperator,
   // except for the '*' and '&' operators that have to be handled specially
   // by CheckArrayAccess (as there are special cases like &array[arraysize]
   // that are explicitly defined as valid by the standard).
   if (Opc != UO_AddrOf && Opc != UO_Deref)
     CheckArrayAccess(Input.get());
 
-  return new (Context)
+  auto *UO = new (Context)
       UnaryOperator(Input.get(), Opc, resultType, VK, OK, OpLoc);
+  // Convert the result back to a half vector.
+  if (ConvertHalfVec)
+    return convertVector(UO, Context.HalfTy, *this);
+  return UO;
 }
 
 /// \brief Determine whether the given expression is a qualified member
 /// access expression, of a form that could be turned into a pointer to member
 /// with the address-of operator.
 static bool isQualifiedMemberAccess(Expr *E) {
   if (DeclRefExpr *DRE = dyn_cast<DeclRefExpr>(E)) {
     if (!DRE->getQualifier())

test/CodeGen/fp16vec-ops.c

+// REQUIRES: arm-registered-target
+// RUN: %clang_cc1 -triple arm64-apple-ios9 -emit-llvm -o - -fallow-half-arguments-and-returns %s | FileCheck %s --check-prefix=CHECK
+// RUN: %clang_cc1 -triple armv7-apple-ios9 -emit-llvm -o - -fallow-half-arguments-and-returns %s | FileCheck %s --check-prefix=CHECK
+
+typedef __fp16 half4 __attribute__ ((vector_size (8)));
+typedef short short4 __attribute__ ((vector_size (8)));
+
+half4 hv0, hv1;
+short4 sv0;
+
+// CHECK-LABEL: testFP16Vec0
+// CHECK: %[[V0:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV:.*]] = fpext <4 x half> %[[V0]] to <4 x float>
+// CHECK: %[[V1:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV1:.*]] = fpext <4 x half> %[[V1]] to <4 x float>
+// CHECK: %[[ADD:.*]] = fadd <4 x float> %[[CONV]], %[[CONV1]]
+// CHECK: %[[CONV2:.*]] = fptrunc <4 x float> %[[ADD]] to <4 x half>
+// CHECK: store <4 x half> %[[CONV2]], <4 x half>* @hv0, align 8
+// CHECK: %[[V2:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV3:.*]] = fpext <4 x half> %[[V2]] to <4 x float>
+// CHECK: %[[V3:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV4:.*]] = fpext <4 x half> %[[V3]] to <4 x float>
+// CHECK: %[[SUB:.*]] = fsub <4 x float> %[[CONV3]], %[[CONV4]]
+// CHECK: %[[CONV5:.*]] = fptrunc <4 x float> %[[SUB]] to <4 x half>
+// CHECK: store <4 x half> %[[CONV5]], <4 x half>* @hv0, align 8
+// CHECK: %[[V4:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV6:.*]] = fpext <4 x half> %[[V4]] to <4 x float>
+// CHECK: %[[V5:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV7:.*]] = fpext <4 x half> %[[V5]] to <4 x float>
+// CHECK: %[[MUL:.*]] = fmul <4 x float> %[[CONV6]], %[[CONV7]]
+// CHECK: %[[CONV8:.*]] = fptrunc <4 x float> %[[MUL]] to <4 x half>
+// CHECK: store <4 x half> %[[CONV8]], <4 x half>* @hv0, align 8
+// CHECK: %[[V6:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV9:.*]] = fpext <4 x half> %[[V6]] to <4 x float>
+// CHECK: %[[V7:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV10:.*]] = fpext <4 x half> %[[V7]] to <4 x float>
+// CHECK: %[[DIV:.*]] = fdiv <4 x float> %[[CONV9]], %[[CONV10]]
+// CHECK: %[[CONV11:.*]] = fptrunc <4 x float> %[[DIV]] to <4 x half>
+// CHECK: store <4 x half> %[[CONV11]], <4 x half>* @hv0, align 8
+
+void testFP16Vec0() {
+  hv0 = hv0 + hv1;
+  hv0 = hv0 - hv1;
+  hv0 = hv0 * hv1;
+  hv0 = hv0 / hv1;
+}
+
+// CHECK-LABEL: testFP16Vec1
+// CHECK: %[[V0:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV:.*]] = fpext <4 x half> %[[V0]] to <4 x float>
+// CHECK: %[[V1:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV1:.*]] = fpext <4 x half> %[[V1]] to <4 x float>
+// CHECK: %[[ADD:.*]] = fadd <4 x float> %[[CONV1]], %[[CONV]]
+// CHECK: %[[CONV2:.*]] = fptrunc <4 x float> %[[ADD]] to <4 x half>
+// CHECK: store <4 x half> %[[CONV2]], <4 x half>* @hv0, align 8
+// CHECK: %[[V2:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV3:.*]] = fpext <4 x half> %[[V2]] to <4 x float>
+// CHECK: %[[V3:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV4:.*]] = fpext <4 x half> %[[V3]] to <4 x float>
+// CHECK: %[[SUB:.*]] = fsub <4 x float> %[[CONV4]], %[[CONV3]]
+// CHECK: %[[CONV5:.*]] = fptrunc <4 x float> %[[SUB]] to <4 x half>
+// CHECK: store <4 x half> %[[CONV5]], <4 x half>* @hv0, align 8
+// CHECK: %[[V4:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV6:.*]] = fpext <4 x half> %[[V4]] to <4 x float>
+// CHECK: %[[V5:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV7:.*]] = fpext <4 x half> %[[V5]] to <4 x float>
+// CHECK: %[[MUL:.*]] = fmul <4 x float> %[[CONV7]], %[[CONV6]]
+// CHECK: %[[CONV8:.*]] = fptrunc <4 x float> %[[MUL]] to <4 x half>
+// CHECK: store <4 x half> %[[CONV8]], <4 x half>* @hv0, align 8
+// CHECK: %[[V6:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV9:.*]] = fpext <4 x half> %[[V6]] to <4 x float>
+// CHECK: %[[V7:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV10:.*]] = fpext <4 x half> %[[V7]] to <4 x float>
+// CHECK: %[[DIV:.*]] = fdiv <4 x float> %[[CONV10]], %[[CONV9]]
+// CHECK: %[[CONV11:.*]] = fptrunc <4 x float> %[[DIV]] to <4 x half>
+// CHECK: store <4 x half> %[[CONV11]], <4 x half>* @hv0, align 8
+
+void testFP16Vec1() {
+  hv0 += hv1;
+  hv0 -= hv1;
+  hv0 *= hv1;
+  hv0 /= hv1;
+}
+
+// CHECK-LABEL: testFP16Vec2
+// CHECK: %[[CADDR:.*]] = alloca i32, align 4
+// CHECK: store i32 %[[C:.*]], i32* %[[CADDR]], align 4
+// CHECK: %[[V0:.*]] = load i32, i32* %[[CADDR]], align 4
+// CHECK: %[[TOBOOL:.*]] = icmp ne i32 %[[V0]], 0
+// CHECK: br i1 %[[TOBOOL]], label %{{.*}}, label %{{.*}}
+//
+// CHECK: %[[V1:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: br label %{{.*}}
+//
+// CHECK: %[[V2:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: br label %{{.*}}
+//
+// CHECK: %[[COND:.*]] = phi <4 x half> [ %[[V1]], %{{.*}} ], [ %[[V2]], %{{.*}} ]
+// CHECK: store <4 x half> %[[COND]], <4 x half>* @hv0, align 8
+
+void testFP16Vec2(int c) {
+  hv0 = c ? hv0 : hv1;
+}
+
+// CHECK-LABEL: testFP16Vec3
+// CHECK: %[[V0:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV:.*]] = fpext <4 x half> %[[V0]] to <4 x float>
+// CHECK: %[[V1:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV1:.*]] = fpext <4 x half> %[[V1]] to <4 x float>
+// CHECK: %[[CMP:.*]] = fcmp oeq <4 x float> %[[CONV]], %[[CONV1]]
+// CHECK: %[[SEXT:.*]] = sext <4 x i1> %[[CMP]] to <4 x i32>
+// CHECK: %[[CONV2:.*]] = trunc <4 x i32> %[[SEXT]] to <4 x i16>
+// CHECK: store <4 x i16> %[[CONV2]], <4 x i16>* @sv0, align 8
+// CHECK: %[[V2:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV3:.*]] = fpext <4 x half> %[[V2]] to <4 x float>
+// CHECK: %[[V3:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV4:.*]] = fpext <4 x half> %[[V3]] to <4 x float>
+// CHECK: %[[CMP5:.*]] = fcmp une <4 x float> %[[CONV3]], %[[CONV4]]
+// CHECK: %[[SEXT6:.*]] = sext <4 x i1> %[[CMP5]] to <4 x i32>
+// CHECK: %[[CONV7:.*]] = trunc <4 x i32> %[[SEXT6]] to <4 x i16>
+// CHECK: store <4 x i16> %[[CONV7]], <4 x i16>* @sv0, align 8
+// CHECK: %[[V4:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV8:.*]] = fpext <4 x half> %[[V4]] to <4 x float>
+// CHECK: %[[V5:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV9:.*]] = fpext <4 x half> %[[V5]] to <4 x float>
+// CHECK: %[[CMP10:.*]] = fcmp olt <4 x float> %[[CONV8]], %[[CONV9]]
+// CHECK: %[[SEXT11:.*]] = sext <4 x i1> %[[CMP10]] to <4 x i32>
+// CHECK: %[[CONV12:.*]] = trunc <4 x i32> %[[SEXT11]] to <4 x i16>
+// CHECK: store <4 x i16> %[[CONV12]], <4 x i16>* @sv0, align 8
+// CHECK: %[[V6:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV13:.*]] = fpext <4 x half> %[[V6]] to <4 x float>
+// CHECK: %[[V7:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV14:.*]] = fpext <4 x half> %[[V7]] to <4 x float>
+// CHECK: %[[CMP15:.*]] = fcmp ogt <4 x float> %[[CONV13]], %[[CONV14]]
+// CHECK: %[[SEXT16:.*]] = sext <4 x i1> %[[CMP15]] to <4 x i32>
+// CHECK: %[[CONV17:.*]] = trunc <4 x i32> %[[SEXT16]] to <4 x i16>
+// CHECK: store <4 x i16> %[[CONV17]], <4 x i16>* @sv0, align 8
+// CHECK: %[[V8:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV18:.*]] = fpext <4 x half> %[[V8]] to <4 x float>
+// CHECK: %[[V9:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV19:.*]] = fpext <4 x half> %[[V9]] to <4 x float>
+// CHECK: %[[CMP20:.*]] = fcmp ole <4 x float> %[[CONV18]], %[[CONV19]]
+// CHECK: %[[SEXT21:.*]] = sext <4 x i1> %[[CMP20]] to <4 x i32>
+// CHECK: %[[CONV22:.*]] = trunc <4 x i32> %[[SEXT21]] to <4 x i16>
+// CHECK: store <4 x i16> %[[CONV22]], <4 x i16>* @sv0, align 8
+// CHECK: %[[V10:.*]] = load <4 x half>, <4 x half>* @hv0, align 8
+// CHECK: %[[CONV23:.*]] = fpext <4 x half> %[[V10]] to <4 x float>
+// CHECK: %[[V11:.*]] = load <4 x half>, <4 x half>* @hv1, align 8
+// CHECK: %[[CONV24:.*]] = fpext <4 x half> %[[V11]] to <4 x float>
+// CHECK: %[[CMP25:.*]] = fcmp oge <4 x float> %[[CONV23]], %[[CONV24]]
+// CHECK: %[[SEXT26:.*]] = sext <4 x i1> %[[CMP25]] to <4 x i32>
+// CHECK: %[[CONV27:.*]] = trunc <4 x i32> %[[SEXT26]] to <4 x i16>
+// CHECK: store <4 x i16> %[[CONV27]], <4 x i16>* @sv0, align 8
+
+void testFP16Vec3() {
+  sv0 = hv0 == hv1;
+  sv0 = hv0 != hv1;
+  sv0 = hv0 < hv1;
+  sv0 = hv0 > hv1;
+  sv0 = hv0 <= hv1;
+  sv0 = hv0 >= hv1;
+}

test/Sema/fp16vec-sema.c

+// RUN: %clang_cc1 -fsyntax-only -verify %s
+
+typedef __fp16 half4 __attribute__ ((vector_size (8)));
+typedef float float4 __attribute__ ((vector_size (16)));
+typedef short short4 __attribute__ ((vector_size (8)));
+typedef int int4 __attribute__ ((vector_size (16)));
+
+half4 hv0, hv1;
+float4 fv0, fv1;
+short4 sv0;
+int4 iv0;
+
+void testFP16Vec(int c) {
+  hv0 = hv0 + hv1;
+  hv0 = hv0 - hv1;
+  hv0 = hv0 * hv1;
+  hv0 = hv0 / hv1;
+  hv0 = c ? hv0 : hv1;
+  hv0 += hv1;
+  hv0 -= hv1;
+  hv0 *= hv1;
+  hv0 /= hv1;
+  sv0 = hv0 == hv1;
+  sv0 = hv0 != hv1;
+  sv0 = hv0 < hv1;
+  sv0 = hv0 > hv1;
+  sv0 = hv0 <= hv1;
+  sv0 = hv0 >= hv1;
+  sv0 = hv0 || hv1; // FIXME: this should be an error in C.
+  sv0 = hv0 && hv1; // FIXME: this should be an error in C.
+
+  // Implicit conversion between half vectors and float vectors are not allowed.
+  hv0 = fv0; // expected-error{{assigning to}}
+  fv0 = hv0; // expected-error{{assigning to}}
+  hv0 = (half4)fv0; // expected-error{{invalid conversion between}}
+  fv0 = (float4)hv0; // expected-error{{invalid conversion between}}
+  hv0 = fv0 + fv1; // expected-error{{assigning to}}
+  fv0 = hv0 + hv1; // expected-error{{assigning to}}
+  hv0 = hv0 + fv1; // expected-error{{cannot convert between vector}}
+  hv0 = c ? hv0 : fv1; // expected-error{{cannot convert between vector}}
+  sv0 = hv0 == fv1; // expected-error{{cannot convert between vector}}
+  sv0 = hv0 < fv1; // expected-error{{cannot convert between vector}}
+  sv0 = hv0 || fv1; // expected-error{{cannot convert between vector}} expected-error{{invalid operands to binary expression}}
+  iv0 = hv0 == hv1; // expected-error{{assigning to}}
+
+  // clang currently disallows using these operators on vectors, which is
+  // allowed by gcc.
+  sv0 = !hv0; // expected-error{{invalid argument type}}
+  hv0++; // expected-error{{cannot increment value of type}}
+  ++hv0; // expected-error{{cannot increment value of type}}
+}