[Clang] Implement the rest of __builtin_elementwise_* functions.

[junaire]

Summary

The patch implement the rest of __builtin_elementwise_* functions
specified in D111529, including:

• __builtin_elementwise_floor
• __builtin_elementwise_roundeven
• __builtin_elementwise_trunc

Signed-off-by: Jun <[e-mail]>

[junaire]

Ping.

[aaron.ballman]

clang/include/clang/Basic/Builtins.def

651

The RFC does not mention this builtin. It specifies __builtin_elementwise_rint and __builtin_elementwise_round. Is there a reason why this uses roundeven instead and should there be other rounding options provided at the same time?

clang/lib/CodeGen/CGBuiltin.cpp

3139–3155

It's starting to seem like we should have a helper function that takes the intrinsic to create and the name to give the operation; WDYT?

[junaire]

The RFC does not mention this builtin. It specifies __builtin_elementwise_rint and __builtin_elementwise_round. Is there a reason why this uses roundeven instead and should there be other rounding options provided at the same time?

Well, I think there may be a misunderstanding. I referenced: https://clang.llvm.org/docs/LanguageExtensions.html#vector-builtins
There is this builtin:

T __builtin_elementwise_roundeven(T x)

But I just found in the RFC(https://lists.llvm.org/pipermail/cfe-dev/2021-September/068999.html), we have:

• __builtin_elementwise_rint
• __builtin_elementwise_round

It looks confusing, I would like to review the RFC again.

[junaire]

Add a helper function to resue code. I'm not good at naming functions,
so I just oveloaded emitUnaryBuiltin.

[junaire]

Format the patch.

[junaire]

Hi, @aaron.ballman I'm sorry for not updating the patch in time because I'm preparing for my school final exam :-(
One thing I want to mention is that __builtin_elementwise_roundeven is actually been added in the RFC during the code review. You can find it in D111529.

"Prevailing rounding mode" is not super-useful, other than as a spelling for round-to-nearest-ties-to-even (IEEE 754 default rounding). Outside of a FENV_ACCESS ON context, there's not even really a notion of "prevailing rounding mode" to appeal to. I assume the intent is for this to lower to e.g. x86 ROUND* with the dynamic rounding-mode immediate.

I would recommend adding __builtin_elementwise_roundeven(T x) instead, which would statically bind IEEE default rounding (following TS 18661-1 naming) without having to appeal to prevailing rounding mode, and can still lower to ROUND* on x86 outside of FENV_ACCESS ON contexts, which is the norm for vector code (and FRINTN unconditionally on armv8). I think we can punt on rint/nearbyint for now, and add them in the future if there's a need.

[fhahn]

In D115429#3201713, @junaire wrote:

Hi, @aaron.ballman I'm sorry for not updating the patch in time because I'm preparing for my school final exam :-(
One thing I want to mention is that __builtin_elementwise_roundeven is actually been added in the RFC during the code review. You can find it in D111529.

"Prevailing rounding mode" is not super-useful, other than as a spelling for round-to-nearest-ties-to-even (IEEE 754 default rounding). Outside of a FENV_ACCESS ON context, there's not even really a notion of "prevailing rounding mode" to appeal to. I assume the intent is for this to lower to e.g. x86 ROUND* with the dynamic rounding-mode immediate.

I would recommend adding __builtin_elementwise_roundeven(T x) instead, which would statically bind IEEE default rounding (following TS 18661-1 naming) without having to appeal to prevailing rounding mode, and can still lower to ROUND* on x86 outside of FENV_ACCESS ON contexts, which is the norm for vector code (and FRINTN unconditionally on armv8). I think we can punt on rint/nearbyint for now, and add them in the future if there's a need.

Yes, this was a change from the RFC after some feedback to the patch.

clang/lib/CodeGen/CGBuiltin.cpp

3139–3155

Sounds good! Might be good to split the emitUnaryBuiltin changes off into a separate change?

[junaire]

Sounds good! Might be good to split the emitUnaryBuiltin changes off into a separate change?

Thanks for your suggestion! I just sent another patch: D116161

[fhahn]

This now needs a rebase after landing 5c57e6aa5777, then it should be good to go.

[junaire]

rebase to main.

[junaire]

In D115429#3219338, @fhahn wrote:

This now needs a rebase after landing 5c57e6aa5777, then it should be good to go.

Well, I just found that you seem to have committed under the wrong name...
My name is Jun Zhang, not Jun Zhan...
But I think it's fine. :)

[fhahn]

In D115429#3219395, @junaire wrote:

In D115429#3219338, @fhahn wrote:

This now needs a rebase after landing 5c57e6aa5777, then it should be good to go.

Well, I just found that you seem to have committed under the wrong name...
My name is Jun Zhang, not Jun Zhan...
But I think it's fine. :)

I am sorry for the typo! I reverted & recommitted the patch with the hopefully correct name now :)

[fhahn]

LGTM, thanks!

[junaire]

In D115429#3219563, @fhahn wrote:

LGTM, thanks!

May I ask you to help me to land this patch? I plan to request commit access after this one. Thanks a lot! ;D

clang/include/clang/Basic/Builtins.def

 BUILTIN(__builtin_alloca, "v*z"   , "Fn")
 BUILTIN(__builtin_alloca_with_align, "v*zIz", "Fn")
 BUILTIN(__builtin_call_with_static_chain, "v.", "nt")
 
 BUILTIN(__builtin_elementwise_abs, "v.", "nct")
 BUILTIN(__builtin_elementwise_max, "v.", "nct")
 BUILTIN(__builtin_elementwise_min, "v.", "nct")
 BUILTIN(__builtin_elementwise_ceil, "v.", "nct")
+BUILTIN(__builtin_elementwise_floor, "v.", "nct")
+BUILTIN(__builtin_elementwise_roundeven, "v.", "nct")
+BUILTIN(__builtin_elementwise_trunc, "v.", "nct")
 BUILTIN(__builtin_reduce_max, "v.", "nct")
 BUILTIN(__builtin_reduce_min, "v.", "nct")
 BUILTIN(__builtin_reduce_xor, "v.", "nct")
 
 BUILTIN(__builtin_matrix_transpose, "v.", "nFt")
 BUILTIN(__builtin_matrix_column_major_load, "v.", "nFt")
 BUILTIN(__builtin_matrix_column_major_store, "v.", "nFt")
 

clang/lib/CodeGen/CGBuiltin.cpp

case Builtin::BI__builtin_elementwise_abs: {
     if (auto *VecTy = QT->getAs<VectorType>())
       QT = VecTy->getElementType();
     if (QT->isIntegerType())
       Result = Builder.CreateBinaryIntrinsic(
           llvm::Intrinsic::abs, EmitScalarExpr(E->getArg(0)),
           Builder.getFalse(), nullptr, "elt.abs");
     else
       Result = emitUnaryBuiltin(*this, E, llvm::Intrinsic::fabs, "elt.abs");
 
     return RValue::get(Result);
   }
 
   case Builtin::BI__builtin_elementwise_ceil:
     return RValue::get(
         emitUnaryBuiltin(*this, E, llvm::Intrinsic::ceil, "elt.ceil"));
+  case Builtin::BI__builtin_elementwise_floor:
+    return RValue::get(
+        emitUnaryBuiltin(*this, E, llvm::Intrinsic::floor, "elt.floor"));
+  case Builtin::BI__builtin_elementwise_roundeven:
+    return RValue::get(emitUnaryBuiltin(*this, E, llvm::Intrinsic::roundeven,
+                                        "elt.roundeven"));
+  case Builtin::BI__builtin_elementwise_trunc:
+    return RValue::get(
+        emitUnaryBuiltin(*this, E, llvm::Intrinsic::trunc, "elt.trunc"));
 
   case Builtin::BI__builtin_elementwise_max: {
     Value *Op0 = EmitScalarExpr(E->getArg(0));
     Value *Op1 = EmitScalarExpr(E->getArg(1));
     Value *Result;
     if (Op0->getType()->isIntOrIntVectorTy()) {
       QualType Ty = E->getArg(0)->getType();
       if (auto *VecTy = Ty->getAs<VectorType>())
         Ty = VecTy->getElementType();

clang/lib/Sema/SemaChecking.cpp

if (EltTy->isUnsignedIntegerType()) {
       Diag(TheCall->getArg(0)->getBeginLoc(),
            diag::err_builtin_invalid_arg_type)
           << 1 << /* signed integer or float ty*/ 3 << ArgTy;
       return ExprError();
     }
     break;
   }
 
-  // __builtin_elementwise_ceil restricts the element type to floating point
+  // These builtins restrict the element type to floating point
   // types only.
-  case Builtin::BI__builtin_elementwise_ceil: {
+  case Builtin::BI__builtin_elementwise_ceil:
+  case Builtin::BI__builtin_elementwise_floor:
+  case Builtin::BI__builtin_elementwise_roundeven:
+  case Builtin::BI__builtin_elementwise_trunc: {
     if (PrepareBuiltinElementwiseMathOneArgCall(TheCall))
       return ExprError();
 
     QualType ArgTy = TheCall->getArg(0)->getType();
     QualType EltTy = ArgTy;
 
     if (auto *VecTy = EltTy->getAs<VectorType>())
       EltTy = VecTy->getElementType();

clang/test/CodeGen/builtins-elementwise-math.c

void test_builtin_elementwise_ceil(float f1, float f2, double d1, double d2,
   // CHECK:      [[D1:%.+]] = load double, double* %d1.addr, align 8
   // CHECK-NEXT: call double @llvm.ceil.f64(double [[D1]])
   d2 = __builtin_elementwise_ceil(d1);
 
   // CHECK:      [[VF1:%.+]] = load <4 x float>, <4 x float>* %vf1.addr, align 16
   // CHECK-NEXT: call <4 x float> @llvm.ceil.v4f32(<4 x float> [[VF1]])
   vf2 = __builtin_elementwise_ceil(vf1);
 }
+
+void test_builtin_elementwise_floor(float f1, float f2, double d1, double d2,
+                                    float4 vf1, float4 vf2) {
+  // CHECK-LABEL: define void @test_builtin_elementwise_floor(
+  // CHECK:      [[F1:%.+]] = load float, float* %f1.addr, align 4
+  // CHECK-NEXT:  call float @llvm.floor.f32(float [[F1]])
+  f2 = __builtin_elementwise_floor(f1);
+
+  // CHECK:      [[D1:%.+]] = load double, double* %d1.addr, align 8
+  // CHECK-NEXT: call double @llvm.floor.f64(double [[D1]])
+  d2 = __builtin_elementwise_floor(d1);
+
+  // CHECK:      [[VF1:%.+]] = load <4 x float>, <4 x float>* %vf1.addr, align 16
+  // CHECK-NEXT: call <4 x float> @llvm.floor.v4f32(<4 x float> [[VF1]])
+  vf2 = __builtin_elementwise_floor(vf1);
+}
+
+void test_builtin_elementwise_roundeven(float f1, float f2, double d1, double d2,
+                                        float4 vf1, float4 vf2) {
+  // CHECK-LABEL: define void @test_builtin_elementwise_roundeven(
+  // CHECK:      [[F1:%.+]] = load float, float* %f1.addr, align 4
+  // CHECK-NEXT:  call float @llvm.roundeven.f32(float [[F1]])
+  f2 = __builtin_elementwise_roundeven(f1);
+
+  // CHECK:      [[D1:%.+]] = load double, double* %d1.addr, align 8
+  // CHECK-NEXT: call double @llvm.roundeven.f64(double [[D1]])
+  d2 = __builtin_elementwise_roundeven(d1);
+
+  // CHECK:      [[VF1:%.+]] = load <4 x float>, <4 x float>* %vf1.addr, align 16
+  // CHECK-NEXT: call <4 x float> @llvm.roundeven.v4f32(<4 x float> [[VF1]])
+  vf2 = __builtin_elementwise_roundeven(vf1);
+}
+
+void test_builtin_elementwise_trunc(float f1, float f2, double d1, double d2,
+                                    float4 vf1, float4 vf2) {
+  // CHECK-LABEL: define void @test_builtin_elementwise_trunc(
+  // CHECK:      [[F1:%.+]] = load float, float* %f1.addr, align 4
+  // CHECK-NEXT:  call float @llvm.trunc.f32(float [[F1]])
+  f2 = __builtin_elementwise_trunc(f1);
+
+  // CHECK:      [[D1:%.+]] = load double, double* %d1.addr, align 8
+  // CHECK-NEXT: call double @llvm.trunc.f64(double [[D1]])
+  d2 = __builtin_elementwise_trunc(d1);
+
+  // CHECK:      [[VF1:%.+]] = load <4 x float>, <4 x float>* %vf1.addr, align 16
+  // CHECK-NEXT: call <4 x float> @llvm.trunc.v4f32(<4 x float> [[VF1]])
+  vf2 = __builtin_elementwise_trunc(vf1);
+}

clang/test/Sema/builtins-elementwise-math.c

void test_builtin_elementwise_ceil(int i, float f, double d, float4 v, int3 iv, unsigned u, unsigned4 uv) {
   // expected-error@-1 {{too many arguments to function call, expected 1, have 2}}
 
   u = __builtin_elementwise_ceil(u);
   // expected-error@-1 {{1st argument must be a floating point type (was 'unsigned int')}}
 
   uv = __builtin_elementwise_ceil(uv);
   // expected-error@-1 {{1st argument must be a floating point type (was 'unsigned4' (vector of 4 'unsigned int' values))}}
 }
+
+void test_builtin_elementwise_floor(int i, float f, double d, float4 v, int3 iv, unsigned u, unsigned4 uv) {
+
+  struct Foo s = __builtin_elementwise_floor(f);
+  // expected-error@-1 {{initializing 'struct Foo' with an expression of incompatible type 'float'}}
+
+  i = __builtin_elementwise_floor();
+  // expected-error@-1 {{too few arguments to function call, expected 1, have 0}}
+
+  i = __builtin_elementwise_floor(i);
+  // expected-error@-1 {{1st argument must be a floating point type (was 'int')}}
+
+  i = __builtin_elementwise_floor(f, f);
+  // expected-error@-1 {{too many arguments to function call, expected 1, have 2}}
+
+  u = __builtin_elementwise_floor(u);
+  // expected-error@-1 {{1st argument must be a floating point type (was 'unsigned int')}}
+
+  uv = __builtin_elementwise_floor(uv);
+  // expected-error@-1 {{1st argument must be a floating point type (was 'unsigned4' (vector of 4 'unsigned int' values))}}
+}
+
+void test_builtin_elementwise_roundeven(int i, float f, double d, float4 v, int3 iv, unsigned u, unsigned4 uv) {
+
+  struct Foo s = __builtin_elementwise_roundeven(f);
+  // expected-error@-1 {{initializing 'struct Foo' with an expression of incompatible type 'float'}}
+
+  i = __builtin_elementwise_roundeven();
+  // expected-error@-1 {{too few arguments to function call, expected 1, have 0}}
+
+  i = __builtin_elementwise_roundeven(i);
+  // expected-error@-1 {{1st argument must be a floating point type (was 'int')}}
+
+  i = __builtin_elementwise_roundeven(f, f);
+  // expected-error@-1 {{too many arguments to function call, expected 1, have 2}}
+
+  u = __builtin_elementwise_roundeven(u);
+  // expected-error@-1 {{1st argument must be a floating point type (was 'unsigned int')}}
+
+  uv = __builtin_elementwise_roundeven(uv);
+  // expected-error@-1 {{1st argument must be a floating point type (was 'unsigned4' (vector of 4 'unsigned int' values))}}
+}
+
+void test_builtin_elementwise_trunc(int i, float f, double d, float4 v, int3 iv, unsigned u, unsigned4 uv) {
+
+  struct Foo s = __builtin_elementwise_trunc(f);
+  // expected-error@-1 {{initializing 'struct Foo' with an expression of incompatible type 'float'}}
+
+  i = __builtin_elementwise_trunc();
+  // expected-error@-1 {{too few arguments to function call, expected 1, have 0}}
+
+  i = __builtin_elementwise_trunc(i);
+  // expected-error@-1 {{1st argument must be a floating point type (was 'int')}}
+
+  i = __builtin_elementwise_trunc(f, f);
+  // expected-error@-1 {{too many arguments to function call, expected 1, have 2}}
+
+  u = __builtin_elementwise_trunc(u);
+  // expected-error@-1 {{1st argument must be a floating point type (was 'unsigned int')}}
+
+  uv = __builtin_elementwise_trunc(uv);
+  // expected-error@-1 {{1st argument must be a floating point type (was 'unsigned4' (vector of 4 'unsigned int' values))}}
+}