[ConstantFolding] Fix crash when folding vector llvm.is.fpclass

[nikic]

LG

[mstorsjo]

I've bisected a failed assert to this commit. Repro:

$ cat repro.c 
int a;
float *b;
float c;
void d() {
  float *e = d;
  for (int f = 0; f < a; f++) {
    c = b[f];
    e[f] = 10.f * c;
    if (__builtin_isnormal(e[f]))
      e[f] = 351.f;
  }
}
$ clang -target aarch64-linux-gnu -c repro.c -O2
clang: ../lib/IR/Instructions.cpp:652: void llvm::CallInst::init(llvm::FunctionType*, llvm::Value*, llvm::ArrayRef<llvm::Value*>, llvm::ArrayRef<llvm::OperandBundleDefT<llvm::Value*> >, const llvm::Twine&): Assertion `(i >= FTy->getNumParams() || FTy->getParamType(i) == Args[i]->getType()) && "Calling a function with a bad signature!"' failed.

[foad]

In D148803#4286333, @mstorsjo wrote:

I've bisected a failed assert to this commit. Repro:

$ cat repro.c 
int a;
float *b;
float c;
void d() {
  float *e = d;
  for (int f = 0; f < a; f++) {
    c = b[f];
    e[f] = 10.f * c;
    if (__builtin_isnormal(e[f]))
      e[f] = 351.f;
  }
}
$ clang -target aarch64-linux-gnu -c repro.c -O2
clang: ../lib/IR/Instructions.cpp:652: void llvm::CallInst::init(llvm::FunctionType*, llvm::Value*, llvm::ArrayRef<llvm::Value*>, llvm::ArrayRef<llvm::OperandBundleDefT<llvm::Value*> >, const llvm::Twine&): Assertion `(i >= FTy->getNumParams() || FTy->getParamType(i) == Args[i]->getType()) && "Calling a function with a bad signature!"' failed.

Thanks! Here's an IR repro:

; RUN: opt < %s -passes=loop-vectorize -S | FileCheck %s

target triple = "aarch64-unknown-linux-gnu"

define void @d() {
bb:
  br label %bb2

bb1:
  ret void

bb2:
  %i = phi i64 [ 0, %bb ], [ %i7, %bb2 ]
  %i3 = load float, ptr null, align 4
  %i4 = getelementptr float, ptr @d, i64 %i
  %i5 = tail call i1 @llvm.is.fpclass.f32(float 0.0, i32 0)
  %i6 = select i1 %i5, float 0.0, float 0.0
  store float %i6, ptr %i4, align 4
  %i7 = add i64 %i, 1
  %i8 = icmp eq i64 %i7, 0
  br i1 %i8, label %bb1, label %bb2
}

declare i1 @llvm.is.fpclass.f32(float, i32 immarg)

The problem is that VPWidenCallRecipe::execute assumes that vector intrinsics are overloaded on their result type. llvm.is.fpclass is overloaded on the type of its first argument instead. I'm not sure what the best fix is. @fhahn?

[foad]

Suggested fix: D148905

[hokein]

Thanks for the quick fix, @foad. I encountered some internal failures caused by this patch as well. While the fix is waiting for review, can we temporarily revert this patch until the fix is ready?

[foad]

In D148803#4286685, @hokein wrote:

Thanks for the quick fix, @foad. I encountered some internal failures caused by this patch as well. While the fix is waiting for review, can we temporarily revert this patch until the fix is ready?

Done. I'll fold this patch into D148905.

[hokein]

In D148803#4286929, @foad wrote:

In D148803#4286685, @hokein wrote:

Thanks for the quick fix, @foad. I encountered some internal failures caused by this patch as well. While the fix is waiting for review, can we temporarily revert this patch until the fix is ready?

Done. I'll fold this patch into D148905.

Thanks!

llvm/lib/Analysis/VectorUtils.cpp

bool llvm::isTriviallyVectorizable(Intrinsic::ID ID) {
   case Intrinsic::trunc:
   case Intrinsic::rint:
   case Intrinsic::nearbyint:
   case Intrinsic::round:
   case Intrinsic::roundeven:
   case Intrinsic::pow:
   case Intrinsic::fma:
   case Intrinsic::fmuladd:
+  case Intrinsic::is_fpclass:
   case Intrinsic::powi:
   case Intrinsic::canonicalize:
   case Intrinsic::fptosi_sat:
   case Intrinsic::fptoui_sat:
     return true;
   default:
     return false;
   }
 }
 
 /// Identifies if the vector form of the intrinsic has a scalar operand.
 bool llvm::isVectorIntrinsicWithScalarOpAtArg(Intrinsic::ID ID,
                                               unsigned ScalarOpdIdx) {
   switch (ID) {
   case Intrinsic::abs:
   case Intrinsic::ctlz:
   case Intrinsic::cttz:
+  case Intrinsic::is_fpclass:
   case Intrinsic::powi:
     return (ScalarOpdIdx == 1);
   case Intrinsic::smul_fix:
   case Intrinsic::smul_fix_sat:
   case Intrinsic::umul_fix:
   case Intrinsic::umul_fix_sat:
     return (ScalarOpdIdx == 2);
   default:

llvm/test/Transforms/InstSimplify/is_fpclass.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 2
+; RUN: opt < %s -S -passes=instsimplify | FileCheck %s
+
+define <2 x i1> @f() {
+; CHECK-LABEL: define <2 x i1> @f() {
+; CHECK-NEXT:    ret <2 x i1> zeroinitializer
+;
+  %i = call <2 x i1> @llvm.is.fpclass.v2f16(<2 x half> <half 0xH7C00, half 0xH7C00>, i32 3)
+  ret <2 x i1> %i
+}
+
+declare <2 x i1> @llvm.is.fpclass.v2f16(<2 x half>, i32 immarg)