[Clang] Extend emitUnaryBuiltin to avoid duplicate logic.

[junaire]

Summary

This patch extends emitUnaryBuiltin so that we can better emitting IR when
implement builtins specified in D111529.

Also contains some NFC, applying it to existing code.

[fhahn]

Can you also update the existing places that could use it?

[junaire]

Update the existing place that can use emitUnaryBuiltin.

[junaire]

Sorry, It seems that the base branch is wrong, reupdate it.

[fhahn]

In D116161#3206442, @junaire wrote:

Update the existing place that can use emitUnaryBuiltin.

I meant just update the existing uses *without* adding floor, roundeven, trunc, so this change should be NFC (non-functional change)

[junaire]

In D116161#3206447, @fhahn wrote:

In D116161#3206442, @junaire wrote:

Update the existing place that can use emitUnaryBuiltin.

I meant just update the existing uses *without* adding floor, roundeven, trunc, so this change should be NFC (non-functional change)

I'm sorry about it, I'm just too careless. :-(

[junaire]

Update the existing place that can use emitUnaryBuiltin.

[junaire]

Update the existing place that can use emitUnaryBuiltin.

[junaire]

Fix wrong usage.

[fhahn]

clang/lib/CodeGen/CGBuiltin.cpp

3137–3138

Should also be used here?

3202

Should also be used here?

3221

Should also be used here?

[junaire]

Should also be used here?

I don't know why but these will cause tests to fail.
Maybe it is caused by my local cache? I'm gonna make a clean build to test it again.
BTW, thanks for taking look at this. Merry Christmas! ;D

[junaire]

I confirmed that we can use emitUnaryBuiltin in the cases you pointed out. Please see the logs below:

$ path/to/llvm-project/build/bin/clang -cc1 -internal-isystem /path/to/llvm-project/build/lib/clang/14.0.0/include -nostdsysteminc -triple x86_64-apple-darwin /path/to/llvm-project/clang/test/CodeGen/builtins-elementwise-math.c -emit-llvm -disable-llvm-passes -o - | /path/to/llvm-project/build/bin/FileCheck /path/to/llvm-project/clang/test/CodeGen/builtins-elementwise-math.c
/path/to/llvm-project/clang/test/CodeGen/builtins-elementwise-math.c:16:17: error: CHECK-NEXT: expected string not found in input
 // CHECK-NEXT: call float @llvm.fabs.f32(float [[F1]])
                ^
<stdin>:35:43: note: scanning from here
 %0 = load float, float* %f1.addr, align 4
                                          ^
<stdin>:35:43: note: with "F1" equal to "%0"
 %0 = load float, float* %f1.addr, align 4
                                          ^
<stdin>:37:13: note: possible intended match here
 %elt.abs = call float @llvm.fabs.f32(float %1)
            ^

Input file: <stdin>
Check file: /path/to/llvm-project/clang/test/CodeGen/builtins-elementwise-math.c

-dump-input=help explains the following input dump.

Input was:
<<<<<<
          30:  store <8 x i16> %vi1, <8 x i16>* %vi1.addr, align 16 
          31:  store <8 x i16> %vi2, <8 x i16>* %vi2.addr, align 16 
          32:  store i64 %i1, i64* %i1.addr, align 8 
          33:  store i64 %i2, i64* %i2.addr, align 8 
          34:  store i16 %si, i16* %si.addr, align 2 
          35:  %0 = load float, float* %f1.addr, align 4 
next:16'0                                               X error: no match found
next:16'1                                                 with "F1" equal to "%0"
          36:  %1 = load float, float* %f1.addr, align 4 
next:16'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          37:  %elt.abs = call float @llvm.fabs.f32(float %1) 
next:16'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
next:16'2                 ?                                    possible intended match
          38:  store float %elt.abs, float* %f2.addr, align 4 
next:16'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          39:  %2 = load double, double* %d1.addr, align 8 
next:16'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          40:  %3 = load double, double* %d1.addr, align 8 
next:16'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          41:  %elt.abs1 = call double @llvm.fabs.f64(double %3) 
next:16'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          42:  store double %elt.abs1, double* %d2.addr, align 8 
next:16'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>>

$ /path/to/llvm-project/build/bin/clang -cc1 -internal-isystem /path/to/llvm-project/build/lib/clang/14.0.0/include -nostdsysteminc -triple x86_64-apple-darwin ~/dev/cpp-projects/llvm-project/clang/test/CodeGen/builtins-reduction-math.c -emit-llvm -disable-llvm-passes -o - | ~/dev/cpp-projects/llvm-project/build/bin/FileCheck ~/dev/cpp-projects/llvm-project/clang/test/CodeGen/builtins-reduction-math.c
/path/to/llvm-project/clang/test/CodeGen/builtins-reduction-math.c:12:17: error: CHECK-NEXT: expected string not found in input
 // CHECK-NEXT: call float @llvm.vector.reduce.fmax.v4f32(<4 x float> [[VF1]])
                ^
<stdin>:23:57: note: scanning from here
 %0 = load <4 x float>, <4 x float>* %vf1.addr, align 16
                                                        ^
<stdin>:23:57: note: with "VF1" equal to "%0"
 %0 = load <4 x float>, <4 x float>* %vf1.addr, align 16
                                                        ^
<stdin>:25:13: note: possible intended match here
 %rdx.min = call float @llvm.vector.reduce.fmax.v4f32(<4 x float> %1)
            ^
/path/to/llvm-project/clang/test/CodeGen/builtins-reduction-math.c:38:17: error: CHECK-NEXT: expected string not found in input
 // CHECK-NEXT: call float @llvm.vector.reduce.fmin.v4f32(<4 x float> [[VF1]])
                ^
<stdin>:74:57: note: scanning from here
 %0 = load <4 x float>, <4 x float>* %vf1.addr, align 16
                                                        ^
<stdin>:74:57: note: with "VF1" equal to "%0"
 %0 = load <4 x float>, <4 x float>* %vf1.addr, align 16
                                                        ^
<stdin>:76:13: note: possible intended match here
 %rdx.min = call float @llvm.vector.reduce.fmin.v4f32(<4 x float> %1)
            ^

Input file: <stdin>
Check file: /path/to/llvm-project/clang/test/CodeGen/builtins-reduction-math.c

-dump-input=help explains the following input dump.

Input was:
<<<<<<
          18:  %cvi1 = alloca <8 x i16>, align 16 
          19:  %r5 = alloca i64, align 8 
          20:  store <4 x float> %vf1, <4 x float>* %vf1.addr, align 16 
          21:  store <8 x i16> %vi1, <8 x i16>* %vi1.addr, align 16 
          22:  store <4 x i32> %vu1, <4 x i32>* %vu1.addr, align 16 
          23:  %0 = load <4 x float>, <4 x float>* %vf1.addr, align 16 
next:12'0                                                             X error: no match found
next:12'1                                                               with "VF1" equal to "%0"
          24:  %1 = load <4 x float>, <4 x float>* %vf1.addr, align 16 
next:12'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          25:  %rdx.min = call float @llvm.vector.reduce.fmax.v4f32(<4 x float> %1) 
next:12'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
next:12'2                 ?                                                          possible intended match
          26:  store float %rdx.min, float* %r1, align 4 
next:12'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          27:  %2 = load <8 x i16>, <8 x i16>* %vi1.addr, align 16 
next:12'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          28:  %3 = load <8 x i16>, <8 x i16>* %vi1.addr, align 16 
next:12'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          29:  %rdx.min1 = call i16 @llvm.vector.reduce.smax.v8i16(<8 x i16> %3) 
next:12'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          30:  store i16 %rdx.min1, i16* %r2, align 2 
next:12'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          69:  %cvi1 = alloca <8 x i16>, align 16 
          70:  %r5 = alloca i64, align 8 
          71:  store <4 x float> %vf1, <4 x float>* %vf1.addr, align 16 
          72:  store <8 x i16> %vi1, <8 x i16>* %vi1.addr, align 16 
          73:  store <4 x i32> %vu1, <4 x i32>* %vu1.addr, align 16 
          74:  %0 = load <4 x float>, <4 x float>* %vf1.addr, align 16 
next:38'0                                                             X error: no match found
next:38'1                                                               with "VF1" equal to "%0"
          75:  %1 = load <4 x float>, <4 x float>* %vf1.addr, align 16 
next:38'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          76:  %rdx.min = call float @llvm.vector.reduce.fmin.v4f32(<4 x float> %1) 
next:38'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
next:38'2                 ?                                                          possible intended match
          77:  store float %rdx.min, float* %r1, align 4 
next:38'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          78:  %2 = load <8 x i16>, <8 x i16>* %vi1.addr, align 16 
next:38'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          79:  %3 = load <8 x i16>, <8 x i16>* %vi1.addr, align 16 
next:38'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          80:  %rdx.min1 = call i16 @llvm.vector.reduce.smin.v8i16(<8 x i16> %3) 
next:38'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          81:  store i16 %rdx.min1, i16* %r2, align 2 
next:38'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>>

Do you know why these will happen? @fhahn

[fhahn]

In D116161#3209286, @junaire wrote:

35:  %0 = load float, float* %f1.addr, align 4 
36:  %1 = load float, float* %f1.addr, align 4 
37:  %elt.abs = call float @llvm.fabs.f32(float %1)

It looks like the argument expression is evaluated twice. Did you remove the Value *Op0 = EmitScalarExpr(E->getArg(0)); calls?

[junaire]

In D116161#3209292, @fhahn wrote:

In D116161#3209286, @junaire wrote:

35:  %0 = load float, float* %f1.addr, align 4 
36:  %1 = load float, float* %f1.addr, align 4 
37:  %elt.abs = call float @llvm.fabs.f32(float %1)

It looks like the argument expression is evaluated twice. Did you remove the Value *Op0 = EmitScalarExpr(E->getArg(0)); calls?

Well, for example:
For __builtin_elementwise_abs, we have code like:

case Builtin::BI__builtin_elementwise_abs: {
  Value *Op0 = EmitScalarExpr(E->getArg(0));
  Value *Result;
  if (Op0->getType()->isIntOrIntVectorTy())
    Result = Builder.CreateBinaryIntrinsic(
        llvm::Intrinsic::abs, Op0, Builder.getFalse(), nullptr, "elt.abs");
  else
    Result = Builder.CreateUnaryIntrinsic(llvm::Intrinsic::fabs, Op0, nullptr,
                                          "elt.abs");

  return RValue::get(Result);
}

If we use emitUnaryBuiltin, we are supposed to do something like:

Result = emitUnaryBuiltin(*this, E, llvm::Intrinsic::fabs, "elt.abs");

We need to pass CallExpr* E to the function to meet its interface, and it calls CGF.EmitScalarExpr(E->getArg(0)); inside. Maybe this is what you said about the argument expression being evaluated twice? I think it is unavoidable if we don't change the function's interface.

Maybe we can have something like:

static Value *emitUnaryBuiltin(CodeGenFunction &CGF, Value* Op0,
                               unsigned IntrinsicID, llvm::StringRef Name) {
  return CGF.Builder.CreateUnaryIntrinsic(IntrinsicID, Op0, nullptr, Name);
}

Then for __builtin_elementwise_abs we can have:

Result = emitUnaryBuiltin(*this, Op0, llvm::Intrinsic::fabs, "elt.abs");

and for __builtin_elementwise_ceil have:

return RValue::get(
    emitUnaryBuiltin(*this, EmitScalarExpr(E->getArg(0)), llvm::Intrinsic::ceil, "elt.ceil"));

WDYT? Well, franking speaking I think this one-line function is ugly but I can't come up with a more elegant solution, I would appreciate it if you can offer some suggestions. :)

[junaire]

In order to use emitUnaryBuiltin in other cases, I changed the function interface.
This allows us to use it in all Builder.CreateUnaryIntrinsic() cases, but will make
the function body very small.

[fhahn]

In D116161#3211178, @junaire wrote:

In order to use emitUnaryBuiltin in other cases, I changed the function interface.
This allows us to use it in all Builder.CreateUnaryIntrinsic() cases, but will make
the function body very small.

I think we should extend & use the existing emitUnaryBuiltin. I think in most cases where it cannot be used straight away you should be able to slightly rewrite the existing code to not rely on llvm::Type. Then there should be no need to call EmitScalarExpr early (an example is in the inline comments)

clang/lib/CodeGen/CGBuiltin.cpp

535

I think we should extend this emitUnaryBuiltin function, rather than having a second one.

e.g.

 static Value *emitUnaryBuiltin(CodeGenFunction &CGF,
                                const CallExpr *E,
-                               unsigned IntrinsicID) {
+                               unsigned IntrinsicID, llvm::StringRef Name = "") {
   llvm::Value *Src0 = CGF.EmitScalarExpr(E->getArg(0));

   Function *F = CGF.CGM.getIntrinsic(IntrinsicID, Src0->getType());
-  return CGF.Builder.CreateCall(F, Src0);
+  return CGF.Builder.CreateCall(F, Src0, Name);
+}

3201–3204

IICU the only reason to call EmitScalarExpr early is the use in GetIntrinsicID, but it could solely rely on QualType:

-    auto GetIntrinsicID = [](QualType QT, llvm::Type *IrTy) {
-      if (IrTy->isIntOrIntVectorTy()) {
-        if (auto *VecTy = QT->getAs<VectorType>())
-          QT = VecTy->getElementType();
-        if (QT->isSignedIntegerType())
-          return llvm::Intrinsic::vector_reduce_smax;
-        else
-          return llvm::Intrinsic::vector_reduce_umax;
-      }
+    auto GetIntrinsicID = [](QualType QT) {
+      if (auto *VecTy = QT->getAs<VectorType>())
+        QT = VecTy->getElementType();
+      if (QT->isSignedIntegerType())
+        return llvm::Intrinsic::vector_reduce_smax;
+      if (QT->isUnsignedIntegerType())
+        return llvm::Intrinsic::vector_reduce_umax;
+      assert(QT->isFloatingType() && "must have a float here");
       return llvm::Intrinsic::vector_reduce_fmax;
     };

[junaire]

Extend emitUnaryBuiltin instead of adding a new overload, also apply it to __builtin_elementwise_abs.

[junaire]

Refactor code a little bit and fix wrong names.

[fhahn]

LGTM, thanks!

[fhahn]

@junaire please let me know if you want me to land this on your behalf.

[junaire]

In D116161#3219080, @fhahn wrote:

@junaire please let me know if you want me to land this on your behalf.

Yeah, thanks a lot!

You can use:
Jun Zhang
[e-mail]

[fhahn]

In D116161#3219149, @junaire wrote:

In D116161#3219080, @fhahn wrote:

@junaire please let me know if you want me to land this on your behalf.

Yeah, thanks a lot!

You can use:
Jun Zhang
[e-mail]

Committed! After D115429, I'd recommend you obtain commit access https://llvm.org/docs/DeveloperPolicy.html#obtaining-commit-access

[junaire]

In D116161#3219343, @fhahn wrote:

In D116161#3219149, @junaire wrote:

In D116161#3219080, @fhahn wrote:

@junaire please let me know if you want me to land this on your behalf.

Yeah, thanks a lot!

You can use:
Jun Zhang
[e-mail]

Committed! After D115429, I'd recommend you obtain commit access https://llvm.org/docs/DeveloperPolicy.html#obtaining-commit-access

That's so great! Thank you for your continued patient guidance and the review work, I really appreciate!

clang/include/clang/Basic/Builtins.def

 BUILTIN(__builtin_alloca, "v*z"   , "Fn")
 BUILTIN(__builtin_alloca_with_align, "v*zIz", "Fn")
 BUILTIN(__builtin_call_with_static_chain, "v.", "nt")
 
 BUILTIN(__builtin_elementwise_abs, "v.", "nct")
 BUILTIN(__builtin_elementwise_max, "v.", "nct")
 BUILTIN(__builtin_elementwise_min, "v.", "nct")
 BUILTIN(__builtin_elementwise_ceil, "v.", "nct")
+BUILTIN(__builtin_elementwise_floor, "v.", "nct")
+BUILTIN(__builtin_elementwise_roundeven, "v.", "nct")
+BUILTIN(__builtin_elementwise_trunc, "v.", "nct")
 BUILTIN(__builtin_reduce_max, "v.", "nct")
 BUILTIN(__builtin_reduce_min, "v.", "nct")
 
 BUILTIN(__builtin_matrix_transpose, "v.", "nFt")
 BUILTIN(__builtin_matrix_column_major_load, "v.", "nFt")
 BUILTIN(__builtin_matrix_column_major_store, "v.", "nFt")
 
 // "Overloaded" Atomic operator builtins.  These are overloaded to support data

clang/lib/CodeGen/CGBuiltin.cpp

static Value *emitCallMaybeConstrainedFPBuiltin(CodeGenFunction &CGF,
   if (CGF.Builder.getIsFPConstrained())
     return CGF.Builder.CreateConstrainedFPCall(F, Args);
   else
     return CGF.Builder.CreateCall(F, Args);
 }
 
 // Emit a simple mangled intrinsic that has 1 argument and a return type
 // matching the argument type.
 static Value *emitUnaryBuiltin(CodeGenFunction &CGF,
                                const CallExpr *E,
                                unsigned IntrinsicID) {
   llvm::Value *Src0 = CGF.EmitScalarExpr(E->getArg(0));
 
   Function *F = CGF.CGM.getIntrinsic(IntrinsicID, Src0->getType());
   return CGF.Builder.CreateCall(F, Src0);
 }
 
+static Value *emitUnaryBuiltin(CodeGenFunction &CGF, const CallExpr *E,
+                               unsigned IntrinsicID, llvm::StringRef Name) {
+  llvm::Value *Src0 = CGF.EmitScalarExpr(E->getArg(0));
+  return CGF.Builder.CreateUnaryIntrinsic(IntrinsicID, Src0, nullptr, Name);
+}
+
 // Emit an intrinsic that has 2 operands of the same type as its result.
 static Value *emitBinaryBuiltin(CodeGenFunction &CGF,
                                 const CallExpr *E,
                                 unsigned IntrinsicID) {
   llvm::Value *Src0 = CGF.EmitScalarExpr(E->getArg(0));
   llvm::Value *Src1 = CGF.EmitScalarExpr(E->getArg(1));
 
   Function *F = CGF.CGM.getIntrinsic(IntrinsicID, Src0->getType());
RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
   }
 
   case Builtin::BI__builtin_elementwise_abs: {
     Value *Op0 = EmitScalarExpr(E->getArg(0));
     Value *Result;
     if (Op0->getType()->isIntOrIntVectorTy())
       Result = Builder.CreateBinaryIntrinsic(
           llvm::Intrinsic::abs, Op0, Builder.getFalse(), nullptr, "elt.abs");
     else
-      Result = Builder.CreateUnaryIntrinsic(llvm::Intrinsic::fabs, Op0, nullptr,
-                                            "elt.abs");
-    return RValue::get(Result);
-  }
+      Result = emitUnaryBuiltin(*this, E, llvm::Intrinsic::fabs, "elt.abs");
 
-  case Builtin::BI__builtin_elementwise_ceil: {
-    Value *Op0 = EmitScalarExpr(E->getArg(0));
-    Value *Result = Builder.CreateUnaryIntrinsic(llvm::Intrinsic::ceil, Op0,
-                                                 nullptr, "elt.ceil");
     return RValue::get(Result);
   }
 
+  case Builtin::BI__builtin_elementwise_ceil:
+    return RValue::get(
+        emitUnaryBuiltin(*this, E, llvm::Intrinsic::ceil, "elt.ceil"));
+  case Builtin::BI__builtin_elementwise_floor:
+    return RValue::get(
+        emitUnaryBuiltin(*this, E, llvm::Intrinsic::floor, "elt.floor"));
+  case Builtin::BI__builtin_elementwise_roundeven:
+    return RValue::get(emitUnaryBuiltin(*this, E, llvm::Intrinsic::roundeven,
+                                        "elt.roundeven"));
+  case Builtin::BI__builtin_elementwise_trunc:
+    return RValue::get(
+        emitUnaryBuiltin(*this, E, llvm::Intrinsic::trunc, "elt.trunc"));
+
   case Builtin::BI__builtin_elementwise_max: {
     Value *Op0 = EmitScalarExpr(E->getArg(0));
     Value *Op1 = EmitScalarExpr(E->getArg(1));
     Value *Result;
     if (Op0->getType()->isIntOrIntVectorTy()) {
       QualType Ty = E->getArg(0)->getType();
       if (auto *VecTy = Ty->getAs<VectorType>())
         Ty = VecTy->getElementType();
auto GetIntrinsicID = [](QualType QT, llvm::Type *IrTy) {
           QT = VecTy->getElementType();
         if (QT->isSignedIntegerType())
           return llvm::Intrinsic::vector_reduce_smax;
         else
           return llvm::Intrinsic::vector_reduce_umax;
       }
       return llvm::Intrinsic::vector_reduce_fmax;
     };
     Value *Op0 = EmitScalarExpr(E->getArg(0));
     Value *Result = Builder.CreateUnaryIntrinsic(
         GetIntrinsicID(E->getArg(0)->getType(), Op0->getType()), Op0, nullptr,
         "rdx.min");
     return RValue::get(Result);
   }
 
   case Builtin::BI__builtin_reduce_min: {
     auto GetIntrinsicID = [](QualType QT, llvm::Type *IrTy) {
       if (IrTy->isIntOrIntVectorTy()) {
         if (auto *VecTy = QT->getAs<VectorType>())
           QT = VecTy->getElementType();
         if (QT->isSignedIntegerType())
           return llvm::Intrinsic::vector_reduce_smin;
         else
           return llvm::Intrinsic::vector_reduce_umin;
       }
       return llvm::Intrinsic::vector_reduce_fmin;
     };
     Value *Op0 = EmitScalarExpr(E->getArg(0));
     Value *Result = Builder.CreateUnaryIntrinsic(
         GetIntrinsicID(E->getArg(0)->getType(), Op0->getType()), Op0, nullptr,
         "rdx.min");
     return RValue::get(Result);
   }
 
   case Builtin::BI__builtin_matrix_transpose: {
     const auto *MatrixTy = E->getArg(0)->getType()->getAs<ConstantMatrixType>();
     Value *MatValue = EmitScalarExpr(E->getArg(0));

clang/lib/Sema/SemaChecking.cpp

if (EltTy->isUnsignedIntegerType()) {
       Diag(TheCall->getArg(0)->getBeginLoc(),
            diag::err_builtin_invalid_arg_type)
           << 1 << /* signed integer or float ty*/ 3 << ArgTy;
       return ExprError();
     }
     break;
   }
 
-  // __builtin_elementwise_ceil restricts the element type to floating point
+  // These builtins restrict the element type to floating point
   // types only.
-  case Builtin::BI__builtin_elementwise_ceil: {
+  case Builtin::BI__builtin_elementwise_ceil:
+  case Builtin::BI__builtin_elementwise_floor:
+  case Builtin::BI__builtin_elementwise_roundeven:
+  case Builtin::BI__builtin_elementwise_trunc: {
     if (PrepareBuiltinElementwiseMathOneArgCall(TheCall))
       return ExprError();
 
     QualType ArgTy = TheCall->getArg(0)->getType();
     QualType EltTy = ArgTy;
 
     if (auto *VecTy = EltTy->getAs<VectorType>())
       EltTy = VecTy->getElementType();

clang/test/CodeGen/builtins-elementwise-math.c

void test_builtin_elementwise_ceil(float f1, float f2, double d1, double d2,
   // CHECK:      [[D1:%.+]] = load double, double* %d1.addr, align 8
   // CHECK-NEXT: call double @llvm.ceil.f64(double [[D1]])
   d2 = __builtin_elementwise_ceil(d1);
 
   // CHECK:      [[VF1:%.+]] = load <4 x float>, <4 x float>* %vf1.addr, align 16
   // CHECK-NEXT: call <4 x float> @llvm.ceil.v4f32(<4 x float> [[VF1]])
   vf2 = __builtin_elementwise_ceil(vf1);
 }
+
+void test_builtin_elementwise_floor(float f1, float f2, double d1, double d2,
+                                    float4 vf1, float4 vf2) {
+  // CHECK-LABEL: define void @test_builtin_elementwise_floor(
+  // CHECK:      [[F1:%.+]] = load float, float* %f1.addr, align 4
+  // CHECK-NEXT:  call float @llvm.floor.f32(float [[F1]])
+  f2 = __builtin_elementwise_floor(f1);
+
+  // CHECK:      [[D1:%.+]] = load double, double* %d1.addr, align 8
+  // CHECK-NEXT: call double @llvm.floor.f64(double [[D1]])
+  d2 = __builtin_elementwise_floor(d1);
+
+  // CHECK:      [[VF1:%.+]] = load <4 x float>, <4 x float>* %vf1.addr, align 16
+  // CHECK-NEXT: call <4 x float> @llvm.floor.v4f32(<4 x float> [[VF1]])
+  vf2 = __builtin_elementwise_floor(vf1);
+}
+
+void test_builtin_elementwise_roundeven(float f1, float f2, double d1, double d2,
+                                        float4 vf1, float4 vf2) {
+  // CHECK-LABEL: define void @test_builtin_elementwise_roundeven(
+  // CHECK:      [[F1:%.+]] = load float, float* %f1.addr, align 4
+  // CHECK-NEXT:  call float @llvm.roundeven.f32(float [[F1]])
+  f2 = __builtin_elementwise_roundeven(f1);
+
+  // CHECK:      [[D1:%.+]] = load double, double* %d1.addr, align 8
+  // CHECK-NEXT: call double @llvm.roundeven.f64(double [[D1]])
+  d2 = __builtin_elementwise_roundeven(d1);
+
+  // CHECK:      [[VF1:%.+]] = load <4 x float>, <4 x float>* %vf1.addr, align 16
+  // CHECK-NEXT: call <4 x float> @llvm.roundeven.v4f32(<4 x float> [[VF1]])
+  vf2 = __builtin_elementwise_roundeven(vf1);
+}
+
+void test_builtin_elementwise_trunc(float f1, float f2, double d1, double d2,
+                                    float4 vf1, float4 vf2) {
+  // CHECK-LABEL: define void @test_builtin_elementwise_trunc(
+  // CHECK:      [[F1:%.+]] = load float, float* %f1.addr, align 4
+  // CHECK-NEXT:  call float @llvm.trunc.f32(float [[F1]])
+  f2 = __builtin_elementwise_trunc(f1);
+
+  // CHECK:      [[D1:%.+]] = load double, double* %d1.addr, align 8
+  // CHECK-NEXT: call double @llvm.trunc.f64(double [[D1]])
+  d2 = __builtin_elementwise_trunc(d1);
+
+  // CHECK:      [[VF1:%.+]] = load <4 x float>, <4 x float>* %vf1.addr, align 16
+  // CHECK-NEXT: call <4 x float> @llvm.trunc.v4f32(<4 x float> [[VF1]])
+  vf2 = __builtin_elementwise_trunc(vf1);
+}

clang/test/Sema/builtins-elementwise-math.c

void test_builtin_elementwise_ceil(int i, float f, double d, float4 v, int3 iv, unsigned u, unsigned4 uv) {
   // expected-error@-1 {{too many arguments to function call, expected 1, have 2}}
 
   u = __builtin_elementwise_ceil(u);
   // expected-error@-1 {{1st argument must be a floating point type (was 'unsigned int')}}
 
   uv = __builtin_elementwise_ceil(uv);
   // expected-error@-1 {{1st argument must be a floating point type (was 'unsigned4' (vector of 4 'unsigned int' values))}}
 }
+
+void test_builtin_elementwise_floor(int i, float f, double d, float4 v, int3 iv, unsigned u, unsigned4 uv) {
+
+  struct Foo s = __builtin_elementwise_floor(f);
+  // expected-error@-1 {{initializing 'struct Foo' with an expression of incompatible type 'float'}}
+
+  i = __builtin_elementwise_floor();
+  // expected-error@-1 {{too few arguments to function call, expected 1, have 0}}
+
+  i = __builtin_elementwise_floor(i);
+  // expected-error@-1 {{1st argument must be a floating point type (was 'int')}}
+
+  i = __builtin_elementwise_floor(f, f);
+  // expected-error@-1 {{too many arguments to function call, expected 1, have 2}}
+
+  u = __builtin_elementwise_floor(u);
+  // expected-error@-1 {{1st argument must be a floating point type (was 'unsigned int')}}
+
+  uv = __builtin_elementwise_floor(uv);
+  // expected-error@-1 {{1st argument must be a floating point type (was 'unsigned4' (vector of 4 'unsigned int' values))}}
+}
+
+void test_builtin_elementwise_roundeven(int i, float f, double d, float4 v, int3 iv, unsigned u, unsigned4 uv) {
+
+  struct Foo s = __builtin_elementwise_roundeven(f);
+  // expected-error@-1 {{initializing 'struct Foo' with an expression of incompatible type 'float'}}
+
+  i = __builtin_elementwise_roundeven();
+  // expected-error@-1 {{too few arguments to function call, expected 1, have 0}}
+
+  i = __builtin_elementwise_roundeven(i);
+  // expected-error@-1 {{1st argument must be a floating point type (was 'int')}}
+
+  i = __builtin_elementwise_roundeven(f, f);
+  // expected-error@-1 {{too many arguments to function call, expected 1, have 2}}
+
+  u = __builtin_elementwise_roundeven(u);
+  // expected-error@-1 {{1st argument must be a floating point type (was 'unsigned int')}}
+
+  uv = __builtin_elementwise_roundeven(uv);
+  // expected-error@-1 {{1st argument must be a floating point type (was 'unsigned4' (vector of 4 'unsigned int' values))}}
+}
+
+void test_builtin_elementwise_trunc(int i, float f, double d, float4 v, int3 iv, unsigned u, unsigned4 uv) {
+
+  struct Foo s = __builtin_elementwise_trunc(f);
+  // expected-error@-1 {{initializing 'struct Foo' with an expression of incompatible type 'float'}}
+
+  i = __builtin_elementwise_trunc();
+  // expected-error@-1 {{too few arguments to function call, expected 1, have 0}}
+
+  i = __builtin_elementwise_trunc(i);
+  // expected-error@-1 {{1st argument must be a floating point type (was 'int')}}
+
+  i = __builtin_elementwise_trunc(f, f);
+  // expected-error@-1 {{too many arguments to function call, expected 1, have 2}}
+
+  u = __builtin_elementwise_trunc(u);
+  // expected-error@-1 {{1st argument must be a floating point type (was 'unsigned int')}}
+
+  uv = __builtin_elementwise_trunc(uv);
+  // expected-error@-1 {{1st argument must be a floating point type (was 'unsigned4' (vector of 4 'unsigned int' values))}}
+}