ConstantFolding: Constant fold some canonicalizes

[arsenm]

Summary

+/-0 is obviously foldable. Other non-special, non-subnormal
values are also probably OK. For denormal values, check
the calling function's denormal mode. For now, don't fold
denormals to the input for IEEE mode because as far as I know
the langref is still pretending LLVM's float isn't IEEE.

Also folds undef to 0, although NaN may make more sense. Skips
folding nans and infinities, although it should be OK to fold those
in a future change.

[spatel]

I don't have any real-world experience with canonicalize, but this seems fine. What happens with poison? (add a test for that)

[arsenm]

Rebase on a bunch more baseline tests. Handle infinities, poison->poison instead of 0. Skip the exotic FP types since I don't understand them

[spatel]

LGTM - see inline for code comment updates.

llvm/include/llvm/IR/Type.h

183

This comment ("six") hasn't aged well - I'd just delete it.

llvm/lib/Analysis/ConstantFolding.cpp

1948

Delete the "I sure hope" - if we're coding it this way, it's the law. If there's fallout, it'll change. :)

[arsenm]

841a0edd030d4cce2e6b764996d3372773d0089b

[jcranmer-intel]

llvm/include/llvm/IR/Type.h

167–170

I'm not sure I like the name "IEEELike", but, admittedly, I'm struggling to keep up with anything better (as bfloat isn't an IEEE type at all, and x86_fp80 is an IEEE extended precision format).

The documentation comment should clarify that it's referring to APFloat::isIEEE(). I think "does not have unnormal values" can be clarified as "does not have non-IEEE values, such as x86_fp80's unnormal values".

llvm/lib/Analysis/ConstantFolding.cpp

1948

If I'm reading the source code of APFloat, and of glibc, correctly, then APFloat considers {0.0, *} to be isZero() for ppc_fp128, which shouldn't be considered a canonical value.

You should be fine for x86_fp80.

1966

Denormal in IEEE denormal mode should be canonical, right? Or are you using this mode to assume "it's dynamic, so we don't know if it's canonical or not" (in which case, add a comment to this effect)?

[arsenm]

llvm/lib/Analysis/ConstantFolding.cpp

1966

Yes. This isn't a strictfp intrinsic, so it's not considering the dynamic mode at all.

I think this should fold the denormal, but if anything is going to break, it would be this. I was mostly worried about things like x87 unnormal values, but now it's just not touching those. I'll probably post a follow up patch to fold here in the future

[arsenm]

llvm/lib/Analysis/ConstantFolding.cpp

1948

https://reviews.llvm.org/D138331

llvm/include/llvm/IR/Type.h

public:
   bool isX86_FP80Ty() const { return getTypeID() == X86_FP80TyID; }
 
   /// Return true if this is 'fp128'.
   bool isFP128Ty() const { return getTypeID() == FP128TyID; }
 
   /// Return true if this is powerpc long double.
   bool isPPC_FP128Ty() const { return getTypeID() == PPC_FP128TyID; }
 
+  /// Return true if this is a well-behaved IEEE-like type, which has a IEEE
+  /// compatible layout as defined by isIEEE(), and does not have unnormal
+  /// values
+  bool isIEEELikeFPTy() const {
+    switch (getTypeID()) {
+    case DoubleTyID:
+    case FloatTyID:
+    case HalfTyID:
+    case BFloatTyID:
+    case FP128TyID:
+      return true;
+    default:
+      return false;
+    }
+  }
+
   /// Return true if this is one of the six floating-point types
   bool isFloatingPointTy() const {
-    return getTypeID() == HalfTyID || getTypeID() == BFloatTyID ||
-           getTypeID() == FloatTyID || getTypeID() == DoubleTyID ||
-           getTypeID() == X86_FP80TyID || getTypeID() == FP128TyID ||
+    return isIEEELikeFPTy() || getTypeID() == X86_FP80TyID ||
            getTypeID() == PPC_FP128TyID;
   }
 
   const fltSemantics &getFltSemantics() const;
 
   /// Return true if this is X86 MMX.
   bool isX86_MMXTy() const { return getTypeID() == X86_MMXTyID; }
 

llvm/lib/Analysis/ConstantFolding.cpp

bool llvm::canConstantFoldCallTo(const CallBase *Call, const Function *F) {
   // environment, they can be folded in any case.
   case Intrinsic::ceil:
   case Intrinsic::floor:
   case Intrinsic::round:
   case Intrinsic::roundeven:
   case Intrinsic::trunc:
   case Intrinsic::nearbyint:
   case Intrinsic::rint:
+  case Intrinsic::canonicalize:
   // Constrained intrinsics can be folded if FP environment is known
   // to compiler.
   case Intrinsic::experimental_constrained_fma:
   case Intrinsic::experimental_constrained_fmuladd:
   case Intrinsic::experimental_constrained_fadd:
   case Intrinsic::experimental_constrained_fsub:
   case Intrinsic::experimental_constrained_fmul:
   case Intrinsic::experimental_constrained_fdiv:
if (!ORM || *ORM == RoundingMode::Dynamic)
     // Even if the rounding mode is unknown, try evaluating the operation.
     // If it does not raise inexact exception, rounding was not applied,
     // so the result is exact and does not depend on rounding mode. Whether
     // other FP exceptions are raised, it does not depend on rounding mode.
     return RoundingMode::NearestTiesToEven;
   return *ORM;
 }
 
+/// Try to constant fold llvm.canonicalize for the given caller and value.
+static Constant *constantFoldCanonicalize(const Type *Ty, const CallBase *CI,
+                                          const APFloat &Src) {
+  // I sure hope zero, positive and negative, is always OK to fold.
+  if (Src.isZero())
+    return ConstantFP::get(CI->getContext(), Src);
+
+  if (!Ty->isIEEELikeFPTy())
+    return nullptr;
+
+  // Zero is always canonical and the sign must be preserved.
+  //
+  // Denorms and nans may have special encodings, but it should be OK to fold a
+  // totally average number.
+  if (Src.isNormal() || Src.isInfinity())
+    return ConstantFP::get(CI->getContext(), Src);
+
+  if (Src.isDenormal()) {
+    DenormalMode DenormMode =
+        CI->getFunction()->getDenormalMode(Src.getSemantics());
+    if (DenormMode == DenormalMode::getIEEE())
+      return nullptr;
+
+    bool IsPositive = !Src.isNegative() ||
+                      DenormMode.Input == DenormalMode::PositiveZero ||
+                      DenormMode.Output == DenormalMode::PositiveZero;
+    return ConstantFP::get(CI->getContext(),
+                           APFloat::getZero(Src.getSemantics(), !IsPositive));
+  }
+
+  // TODO: Handle special values?
+  return nullptr;
+}
+
 static Constant *ConstantFoldScalarCall1(StringRef Name,
                                          Intrinsic::ID IntrinsicID,
                                          Type *Ty,
                                          ArrayRef<Constant *> Operands,
                                          const TargetLibraryInfo *TLI,
                                          const CallBase *Call) {
   assert(Operands.size() == 1 && "Wrong number of operands.");
 
   if (IntrinsicID == Intrinsic::is_constant) {
     // We know we have a "Constant" argument. But we want to only
     // return true for manifest constants, not those that depend on
     // constants with unknowable values, e.g. GlobalValue or BlockAddress.
     if (Operands[0]->isManifestConstant())
       return ConstantInt::getTrue(Ty->getContext());
     return nullptr;
   }
+
+  if (isa<PoisonValue>(Operands[0])) {
+    // TODO: All of these operations should probably propagate poison.
+    if (IntrinsicID == Intrinsic::canonicalize)
+      return PoisonValue::get(Ty);
+  }
+
   if (isa<UndefValue>(Operands[0])) {
     // cosine(arg) is between -1 and 1. cosine(invalid arg) is NaN.
     // ctpop() is between 0 and bitwidth, pick 0 for undef.
     // fptoui.sat and fptosi.sat can always fold to zero (for a zero input).
     if (IntrinsicID == Intrinsic::cos ||
         IntrinsicID == Intrinsic::ctpop ||
         IntrinsicID == Intrinsic::fptoui_sat ||
-        IntrinsicID == Intrinsic::fptosi_sat)
+        IntrinsicID == Intrinsic::fptosi_sat ||
+        IntrinsicID == Intrinsic::canonicalize)
       return Constant::getNullValue(Ty);
     if (IntrinsicID == Intrinsic::bswap ||
         IntrinsicID == Intrinsic::bitreverse ||
         IntrinsicID == Intrinsic::launder_invariant_group ||
         IntrinsicID == Intrinsic::strip_invariant_group)
       return Operands[0];
   }
 
if (IntrinsicID == Intrinsic::fptoui_sat ||
       // convertToInteger() already has the desired saturation semantics.
       APSInt Int(Ty->getIntegerBitWidth(),
                  IntrinsicID == Intrinsic::fptoui_sat);
       bool IsExact;
       U.convertToInteger(Int, APFloat::rmTowardZero, &IsExact);
       return ConstantInt::get(Ty, Int);
     }
 
+    if (IntrinsicID == Intrinsic::canonicalize)
+      return constantFoldCanonicalize(Ty, Call, U);
+
     if (!Ty->isHalfTy() && !Ty->isFloatTy() && !Ty->isDoubleTy())
       return nullptr;
 
     // Use internal versions of these intrinsics.
 
     if (IntrinsicID == Intrinsic::nearbyint || IntrinsicID == Intrinsic::rint) {
       U.roundToIntegral(APFloat::rmNearestTiesToEven);
       return ConstantFP::get(Ty->getContext(), U);

llvm/test/Transforms/InstSimplify/canonicalize.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -S -passes=instsimplify %s | FileCheck %s
 
 define float @canonicalize_zero() {
 ; CHECK-LABEL: @canonicalize_zero(
-; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float 0.000000e+00)
-; CHECK-NEXT:    ret float [[RET]]
+; CHECK-NEXT:    ret float 0.000000e+00
 ;
   %ret = call float @llvm.canonicalize.f32(float 0.0)
   ret float %ret
 }
 
 define float @canonicalize_negzero() {
 ; CHECK-LABEL: @canonicalize_negzero(
-; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float -0.000000e+00)
-; CHECK-NEXT:    ret float [[RET]]
+; CHECK-NEXT:    ret float -0.000000e+00
 ;
   %ret = call float @llvm.canonicalize.f32(float -0.0)
   ret float %ret
 }
 
 define <2 x float> @canonicalize_zero_vector() {
 ; CHECK-LABEL: @canonicalize_zero_vector(
-; CHECK-NEXT:    [[RET:%.*]] = call <2 x float> @llvm.canonicalize.v2f32(<2 x float> zeroinitializer)
-; CHECK-NEXT:    ret <2 x float> [[RET]]
+; CHECK-NEXT:    ret <2 x float> zeroinitializer
 ;
   %ret = call <2 x float> @llvm.canonicalize.v2f32(<2 x float> zeroinitializer)
   ret <2 x float> %ret
 }
 
 define <2 x float> @canonicalize_negzero_vector() {
 ; CHECK-LABEL: @canonicalize_negzero_vector(
-; CHECK-NEXT:    [[RET:%.*]] = call <2 x float> @llvm.canonicalize.v2f32(<2 x float> <float -0.000000e+00, float -0.000000e+00>)
-; CHECK-NEXT:    ret <2 x float> [[RET]]
+; CHECK-NEXT:    ret <2 x float> <float -0.000000e+00, float -0.000000e+00>
 ;
   %ret = call <2 x float> @llvm.canonicalize.v2f32(<2 x float> <float -0.0, float -0.0>)
   ret <2 x float> %ret
 }
 
 define <2 x float> @canonicalize_negzero_vector_partialundef() {
 ; CHECK-LABEL: @canonicalize_negzero_vector_partialundef(
-; CHECK-NEXT:    [[RET:%.*]] = call <2 x float> @llvm.canonicalize.v2f32(<2 x float> <float -0.000000e+00, float undef>)
-; CHECK-NEXT:    ret <2 x float> [[RET]]
+; CHECK-NEXT:    ret <2 x float> <float -0.000000e+00, float 0.000000e+00>
 ;
   %ret = call <2 x float> @llvm.canonicalize.v2f32(<2 x float> <float -0.0, float undef>)
   ret <2 x float> %ret
 }
 
 define float @canonicalize_undef() {
 ; CHECK-LABEL: @canonicalize_undef(
-; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float undef)
-; CHECK-NEXT:    ret float [[RET]]
+; CHECK-NEXT:    ret float 0.000000e+00
 ;
   %ret = call float @llvm.canonicalize.f32(float undef)
   ret float %ret
 }
 
 define <2 x float> @canonicalize_undef_vector() {
 ; CHECK-LABEL: @canonicalize_undef_vector(
-; CHECK-NEXT:    [[RET:%.*]] = call <2 x float> @llvm.canonicalize.v2f32(<2 x float> undef)
-; CHECK-NEXT:    ret <2 x float> [[RET]]
+; CHECK-NEXT:    ret <2 x float> zeroinitializer
 ;
   %ret = call <2 x float> @llvm.canonicalize.v2f32(<2 x float> undef)
   ret <2 x float> %ret
 }
 
 define float @canonicalize_poison() {
 ; CHECK-LABEL: @canonicalize_poison(
-; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float poison)
-; CHECK-NEXT:    ret float [[RET]]
+; CHECK-NEXT:    ret float poison
 ;
   %ret = call float @llvm.canonicalize.f32(float poison)
   ret float %ret
 }
 
 define <2 x float> @canonicalize_poison_vector() {
 ; CHECK-LABEL: @canonicalize_poison_vector(
-; CHECK-NEXT:    [[RET:%.*]] = call <2 x float> @llvm.canonicalize.v2f32(<2 x float> poison)
-; CHECK-NEXT:    ret <2 x float> [[RET]]
+; CHECK-NEXT:    ret <2 x float> poison
 ;
   %ret = call <2 x float> @llvm.canonicalize.v2f32(<2 x float> poison)
   ret <2 x float> %ret
 }
 
 define float @canonicalize_denorm() {
 ; CHECK-LABEL: @canonicalize_denorm(
 ; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float 0x380FFFFFC0000000)
 ; CHECK-NEXT:    ret float [[RET]]
 ;
   %ret = call float @llvm.canonicalize.f32(float bitcast (i32 8388607 to float))
   ret float %ret
 }
 
 define float @canonicalize_pos_denorm_preserve_sign_output() "denormal-fp-math"="preserve-sign,ieee" {
 ; CHECK-LABEL: @canonicalize_pos_denorm_preserve_sign_output(
-; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float 0x380FFFFFC0000000)
-; CHECK-NEXT:    ret float [[RET]]
+; CHECK-NEXT:    ret float 0.000000e+00
 ;
   %ret = call float @llvm.canonicalize.f32(float bitcast (i32 8388607 to float))
   ret float %ret
 }
 
 define float @canonicalize_pos_denorm_preserve_sign_input() "denormal-fp-math"="ieee,preserve-sign" {
 ; CHECK-LABEL: @canonicalize_pos_denorm_preserve_sign_input(
-; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float 0x380FFFFFC0000000)
-; CHECK-NEXT:    ret float [[RET]]
+; CHECK-NEXT:    ret float 0.000000e+00
 ;
   %ret = call float @llvm.canonicalize.f32(float bitcast (i32 8388607 to float))
   ret float %ret
 }
 
 define float @canonicalize_neg_denorm_preserve_sign_output() "denormal-fp-math"="preserve-sign,ieee" {
 ; CHECK-LABEL: @canonicalize_neg_denorm_preserve_sign_output(
-; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float 0xB80FFFFFC0000000)
-; CHECK-NEXT:    ret float [[RET]]
+; CHECK-NEXT:    ret float -0.000000e+00
 ;
   %ret = call float @llvm.canonicalize.f32(float bitcast (i32 -2139095041 to float))
   ret float %ret
 }
 
 define float @canonicalize_neg_denorm_preserve_sign_input() "denormal-fp-math"="ieee,preserve-sign" {
 ; CHECK-LABEL: @canonicalize_neg_denorm_preserve_sign_input(
-; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float 0xB80FFFFFC0000000)
-; CHECK-NEXT:    ret float [[RET]]
+; CHECK-NEXT:    ret float -0.000000e+00
 ;
   %ret = call float @llvm.canonicalize.f32(float bitcast (i32 -2139095041 to float))
   ret float %ret
 }
 
 define float @canonicalize_pos_denorm_positive_zero_output() "denormal-fp-math"="positive-zero,ieee" {
 ; CHECK-LABEL: @canonicalize_pos_denorm_positive_zero_output(
-; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float 0x380FFFFFC0000000)
-; CHECK-NEXT:    ret float [[RET]]
+; CHECK-NEXT:    ret float 0.000000e+00
 ;
   %ret = call float @llvm.canonicalize.f32(float bitcast (i32 8388607 to float))
   ret float %ret
 }
 
 define float @canonicalize_pos_denorm_positive_zero_input() "denormal-fp-math"="ieee,positive-zero" {
 ; CHECK-LABEL: @canonicalize_pos_denorm_positive_zero_input(
-; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float 0x380FFFFFC0000000)
-; CHECK-NEXT:    ret float [[RET]]
+; CHECK-NEXT:    ret float 0.000000e+00
 ;
   %ret = call float @llvm.canonicalize.f32(float bitcast (i32 8388607 to float))
   ret float %ret
 }
 
 define float @canonicalize_neg_denorm_positive_zero_output() "denormal-fp-math"="positive-zero,ieee" {
 ; CHECK-LABEL: @canonicalize_neg_denorm_positive_zero_output(
-; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float 0xB80FFFFFC0000000)
-; CHECK-NEXT:    ret float [[RET]]
+; CHECK-NEXT:    ret float 0.000000e+00
 ;
   %ret = call float @llvm.canonicalize.f32(float bitcast (i32 -2139095041 to float))
   ret float %ret
 }
 
 define float @canonicalize_neg_denorm_positive_zero_input() "denormal-fp-math"="ieee,positive-zero" {
 ; CHECK-LABEL: @canonicalize_neg_denorm_positive_zero_input(
-; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float 0xB80FFFFFC0000000)
-; CHECK-NEXT:    ret float [[RET]]
+; CHECK-NEXT:    ret float 0.000000e+00
 ;
   %ret = call float @llvm.canonicalize.f32(float bitcast (i32 -2139095041 to float))
   ret float %ret
 }
 
 define float @canonicalize_inf() {
 ; CHECK-LABEL: @canonicalize_inf(
-; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float 0x7FF0000000000000)
-; CHECK-NEXT:    ret float [[RET]]
+; CHECK-NEXT:    ret float 0x7FF0000000000000
 ;
   %ret = call float @llvm.canonicalize.f32(float 0x7FF0000000000000)
   ret float %ret
 }
 
 define float @canonicalize_neg_inf() {
 ; CHECK-LABEL: @canonicalize_neg_inf(
-; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float 0xFFF0000000000000)
-; CHECK-NEXT:    ret float [[RET]]
+; CHECK-NEXT:    ret float 0xFFF0000000000000
 ;
   %ret = call float @llvm.canonicalize.f32(float 0xFFF0000000000000)
   ret float %ret
 }
 
 define float @canonicalize_qnan() {
 ; CHECK-LABEL: @canonicalize_qnan(
 ; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float 0x7FF8000000000000)

 ; CHECK-NEXT:    ret float [[RET]]
 ;
   %ret = call float @llvm.canonicalize.f32(float bitcast (i32 2139095041 to float))
   ret float %ret
 }
 
 define float @canonicalize_pos_normal() {
 ; CHECK-LABEL: @canonicalize_pos_normal(
-; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float 4.000000e+00)
-; CHECK-NEXT:    ret float [[RET]]
+; CHECK-NEXT:    ret float 4.000000e+00
 ;
   %ret = call float @llvm.canonicalize.f32(float 4.0)
   ret float %ret
 }
 
 define float @canonicalize_neg_normal() {
 ; CHECK-LABEL: @canonicalize_neg_normal(
-; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float -4.000000e+00)
-; CHECK-NEXT:    ret float [[RET]]
+; CHECK-NEXT:    ret float -4.000000e+00
 ;
   %ret = call float @llvm.canonicalize.f32(float -4.0)
   ret float %ret
 }
 
 define <2 x float> @canonicalize_pos_denorm_preserve_sign_output_mixed_vector() "denormal-fp-math"="preserve-sign,ieee" {
 ; CHECK-LABEL: @canonicalize_pos_denorm_preserve_sign_output_mixed_vector(
-; CHECK-NEXT:    [[RET:%.*]] = call <2 x float> @llvm.canonicalize.v2f32(<2 x float> <float 0x380FFFFFC0000000, float 0xB80FFFFFC0000000>)
-; CHECK-NEXT:    ret <2 x float> [[RET]]
+; CHECK-NEXT:    ret <2 x float> <float 0.000000e+00, float -0.000000e+00>
 ;
   %ret = call <2 x float> @llvm.canonicalize.v2f32(<2 x float> <float bitcast (i32 8388607 to float), float bitcast (i32 -2139095041 to float)>)
   ret <2 x float> %ret
 }
 
 define <2 x float> @canonicalize_pos_denorm_preserve_sign_input_mixed_vector() "denormal-fp-math"="ieee,preserve-sign" {
 ; CHECK-LABEL: @canonicalize_pos_denorm_preserve_sign_input_mixed_vector(
-; CHECK-NEXT:    [[RET:%.*]] = call <2 x float> @llvm.canonicalize.v2f32(<2 x float> <float 0xB80FFFFFC0000000, float 0x380FFFFFC0000000>)
-; CHECK-NEXT:    ret <2 x float> [[RET]]
+; CHECK-NEXT:    ret <2 x float> <float -0.000000e+00, float 0.000000e+00>
 ;
   %ret = call <2 x float> @llvm.canonicalize.v2f32(<2 x float> <float bitcast (i32 -2139095041 to float), float bitcast (i32 8388607 to float)>)
   ret <2 x float> %ret
 }
 
 define <2 x float> @canonicalize_pos_denorm_positive_zero_output_mixed_vector() "denormal-fp-math"="positive-zero,ieee" {
 ; CHECK-LABEL: @canonicalize_pos_denorm_positive_zero_output_mixed_vector(
-; CHECK-NEXT:    [[RET:%.*]] = call <2 x float> @llvm.canonicalize.v2f32(<2 x float> <float 0x380FFFFFC0000000, float 0xB80FFFFFC0000000>)
-; CHECK-NEXT:    ret <2 x float> [[RET]]
+; CHECK-NEXT:    ret <2 x float> zeroinitializer
 ;
   %ret = call <2 x float> @llvm.canonicalize.v2f32(<2 x float> <float bitcast (i32 8388607 to float), float bitcast (i32 -2139095041 to float)>)
   ret <2 x float> %ret
 }
 
 define <2 x float> @canonicalize_pos_denorm_positive_zero_input_mixed_vector() "denormal-fp-math"="ieee,positive-zero" {
 ; CHECK-LABEL: @canonicalize_pos_denorm_positive_zero_input_mixed_vector(
-; CHECK-NEXT:    [[RET:%.*]] = call <2 x float> @llvm.canonicalize.v2f32(<2 x float> <float 0xB80FFFFFC0000000, float 0x380FFFFFC0000000>)
-; CHECK-NEXT:    ret <2 x float> [[RET]]
+; CHECK-NEXT:    ret <2 x float> zeroinitializer
 ;
   %ret = call <2 x float> @llvm.canonicalize.v2f32(<2 x float> <float bitcast (i32 -2139095041 to float), float bitcast (i32 8388607 to float)>)
   ret <2 x float> %ret
 }
 
 define float @canonicalize_neg_denorm_preserve_sign_output_positive_zero_input() "denormal-fp-math"="preserve-sign,positive-zero" {
 ; CHECK-LABEL: @canonicalize_neg_denorm_preserve_sign_output_positive_zero_input(
-; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float 0xB80FFFFFC0000000)
-; CHECK-NEXT:    ret float [[RET]]
+; CHECK-NEXT:    ret float 0.000000e+00
 ;
   %ret = call float @llvm.canonicalize.f32(float bitcast (i32 -2139095041 to float))
   ret float %ret
 }
 
 define float @canonicalize_neg_denorm_positive_zero_output_preserve_sign_input() "denormal-fp-math"="positive-zero,preserve-sign" {
 ; CHECK-LABEL: @canonicalize_neg_denorm_positive_zero_output_preserve_sign_input(
-; CHECK-NEXT:    [[RET:%.*]] = call float @llvm.canonicalize.f32(float 0xB80FFFFFC0000000)
-; CHECK-NEXT:    ret float [[RET]]
+; CHECK-NEXT:    ret float 0.000000e+00
 ;
   %ret = call float @llvm.canonicalize.f32(float bitcast (i32 -2139095041 to float))
   ret float %ret
 }
 
 ; --------------------------------------------------------------------
 ; f64 handling
 ; --------------------------------------------------------------------
 
 define double @canonicalize_zero_f64() {
 ; CHECK-LABEL: @canonicalize_zero_f64(
-; CHECK-NEXT:    [[RET:%.*]] = call double @llvm.canonicalize.f64(double 0.000000e+00)
-; CHECK-NEXT:    ret double [[RET]]
+; CHECK-NEXT:    ret double 0.000000e+00
 ;
   %ret = call double @llvm.canonicalize.f64(double 0.0)
   ret double %ret
 }
 
 define double @canonicalize_negzero_f64() {
 ; CHECK-LABEL: @canonicalize_negzero_f64(
-; CHECK-NEXT:    [[RET:%.*]] = call double @llvm.canonicalize.f64(double -0.000000e+00)
-; CHECK-NEXT:    ret double [[RET]]
+; CHECK-NEXT:    ret double -0.000000e+00
 ;
   %ret = call double @llvm.canonicalize.f64(double -0.0)
   ret double %ret
 }
 
 define double @canonicalize_1.0_f64() {
 ; CHECK-LABEL: @canonicalize_1.0_f64(
-; CHECK-NEXT:    [[RET:%.*]] = call double @llvm.canonicalize.f64(double 1.000000e+00)
-; CHECK-NEXT:    ret double [[RET]]
+; CHECK-NEXT:    ret double 1.000000e+00
 ;
   %ret = call double @llvm.canonicalize.f64(double 1.0)
   ret double %ret
 }
 
 define double @canonicalize_0x00000000000001_f64() {
 ; CHECK-LABEL: @canonicalize_0x00000000000001_f64(
 ; CHECK-NEXT:    [[RET:%.*]] = call double @llvm.canonicalize.f64(double 4.940660e-324)
 ; CHECK-NEXT:    ret double [[RET]]
 ;
   %ret = call double @llvm.canonicalize.f64(double 0x00000000000001)
   ret double %ret
 }
 
 define double @canonicalize_inf_f64() {
 ; CHECK-LABEL: @canonicalize_inf_f64(
-; CHECK-NEXT:    [[RET:%.*]] = call double @llvm.canonicalize.f64(double 0x7FF0000000000000)
-; CHECK-NEXT:    ret double [[RET]]
+; CHECK-NEXT:    ret double 0x7FF0000000000000
 ;
   %ret = call double @llvm.canonicalize.f64(double 0x7FF0000000000000)
   ret double %ret
 }
 
 define double @canonicalize_ninf_f64() {
 ; CHECK-LABEL: @canonicalize_ninf_f64(
-; CHECK-NEXT:    [[RET:%.*]] = call double @llvm.canonicalize.f64(double 0xFFF0000000000000)
-; CHECK-NEXT:    ret double [[RET]]
+; CHECK-NEXT:    ret double 0xFFF0000000000000
 ;
   %ret = call double @llvm.canonicalize.f64(double 0xFFF0000000000000)
   ret double %ret
 }
 
 ; --------------------------------------------------------------------
 ; f16 handling
 ; --------------------------------------------------------------------
 
 define half @canonicalize_zero_f16() {
 ; CHECK-LABEL: @canonicalize_zero_f16(
-; CHECK-NEXT:    [[RET:%.*]] = call half @llvm.canonicalize.f16(half 0xH0000)
-; CHECK-NEXT:    ret half [[RET]]
+; CHECK-NEXT:    ret half 0xH0000
 ;
   %ret = call half @llvm.canonicalize.f16(half 0.0)
   ret half %ret
 }
 
 define half @canonicalize_1.0_f16() {
 ; CHECK-LABEL: @canonicalize_1.0_f16(
-; CHECK-NEXT:    [[RET:%.*]] = call half @llvm.canonicalize.f16(half 0xH3C00)
-; CHECK-NEXT:    ret half [[RET]]
+; CHECK-NEXT:    ret half 0xH3C00
 ;
   %ret = call half @llvm.canonicalize.f16(half 1.0)
   ret half %ret
 }
 
 define half @canonicalize_0x0001_f16() {
 ; CHECK-LABEL: @canonicalize_0x0001_f16(
 ; CHECK-NEXT:    [[RET:%.*]] = call half @llvm.canonicalize.f16(half 0xH0001)
 ; CHECK-NEXT:    ret half [[RET]]
 ;
   %ret = call half @llvm.canonicalize.f16(half 0xH0001)
   ret half %ret
 }
 
 define half @canonicalize_inf_f16() {
 ; CHECK-LABEL: @canonicalize_inf_f16(
-; CHECK-NEXT:    [[RET:%.*]] = call half @llvm.canonicalize.f16(half 0xH7C00)
-; CHECK-NEXT:    ret half [[RET]]
+; CHECK-NEXT:    ret half 0xH7C00
 ;
   %ret = call half @llvm.canonicalize.f16(half 0xH7C00)
   ret half %ret
 }
 
 define half @canonicalize_neg_inf_f16() {
 ; CHECK-LABEL: @canonicalize_neg_inf_f16(
-; CHECK-NEXT:    [[RET:%.*]] = call half @llvm.canonicalize.f16(half 0xHFC00)
-; CHECK-NEXT:    ret half [[RET]]
+; CHECK-NEXT:    ret half 0xHFC00
 ;
   %ret = call half @llvm.canonicalize.f16(half 0xHFC00)
   ret half %ret
 }
 
 ; --------------------------------------------------------------------
 ; fp128 handling
 ; --------------------------------------------------------------------
 
 define fp128 @canonicalize_zero_fp128() {
 ; CHECK-LABEL: @canonicalize_zero_fp128(
-; CHECK-NEXT:    [[RET:%.*]] = call fp128 @llvm.canonicalize.f128(fp128 0xL00000000000000000000000000000000)
-; CHECK-NEXT:    ret fp128 [[RET]]
+; CHECK-NEXT:    ret fp128 0xL00000000000000000000000000000000
 ;
   %ret = call fp128 @llvm.canonicalize.fp128(fp128 0xL00000000000000000000000000000000)
   ret fp128 %ret
 }
 
 define fp128 @canonicalize_1.0_fp128() {
 ; CHECK-LABEL: @canonicalize_1.0_fp128(
-; CHECK-NEXT:    [[RET:%.*]] = call fp128 @llvm.canonicalize.f128(fp128 0xL00000000000000003FFF000000000000)
-; CHECK-NEXT:    ret fp128 [[RET]]
+; CHECK-NEXT:    ret fp128 0xL00000000000000003FFF000000000000
 ;
   %ret = call fp128 @llvm.canonicalize.fp128(fp128 0xL00000000000000003FFF000000000000)
   ret fp128 %ret
 }
 
 define fp128 @canonicalize_0x00000000000000000000000000000001_fp128() {
 ; CHECK-LABEL: @canonicalize_0x00000000000000000000000000000001_fp128(
 ; CHECK-NEXT:    [[RET:%.*]] = call fp128 @llvm.canonicalize.f128(fp128 0xL00000000000000000000000000000001)
 ; CHECK-NEXT:    ret fp128 [[RET]]
 ;
   %ret = call fp128 @llvm.canonicalize.fp128(fp128 0xL00000000000000000000000000000001)
   ret fp128 %ret
 }
 
 define fp128 @canonicalize_inf_fp128() {
 ; CHECK-LABEL: @canonicalize_inf_fp128(
-; CHECK-NEXT:    [[RET:%.*]] = call fp128 @llvm.canonicalize.f128(fp128 0xL00000000000000007FFF000000000000)
-; CHECK-NEXT:    ret fp128 [[RET]]
+; CHECK-NEXT:    ret fp128 0xL00000000000000007FFF000000000000
 ;
   %ret = call fp128 @llvm.canonicalize.fp128(fp128 0xL00000000000000007FFF000000000000)
   ret fp128 %ret
 }
 
 define fp128 @canonicalize_neg_inf_fp128() {
 ; CHECK-LABEL: @canonicalize_neg_inf_fp128(
-; CHECK-NEXT:    [[RET:%.*]] = call fp128 @llvm.canonicalize.f128(fp128 0xL0000000000000000FFFF000000000000)
-; CHECK-NEXT:    ret fp128 [[RET]]
+; CHECK-NEXT:    ret fp128 0xL0000000000000000FFFF000000000000
 ;
   %ret = call fp128 @llvm.canonicalize.fp128(fp128 0xL0000000000000000FFFF000000000000)
   ret fp128 %ret
 }
 
 define fp128 @canonicalize_nan_fp128() {
 ; CHECK-LABEL: @canonicalize_nan_fp128(
 ; CHECK-NEXT:    [[RET:%.*]] = call fp128 @llvm.canonicalize.f128(fp128 0xL00000000000000007FFF800000000000)
 ; CHECK-NEXT:    ret fp128 [[RET]]
 ;
   %ret = call fp128 @llvm.canonicalize.fp128(fp128 0xL00000000000000007FFF800000000000)
   ret fp128 %ret
 }
 
 ; --------------------------------------------------------------------
 ; bfloat handling
 ; --------------------------------------------------------------------
 
 define bfloat @canonicalize_zero_bf16() {
 ; CHECK-LABEL: @canonicalize_zero_bf16(
-; CHECK-NEXT:    [[RET:%.*]] = call bfloat @llvm.canonicalize.bf16(bfloat 0xR0000)
-; CHECK-NEXT:    ret bfloat [[RET]]
+; CHECK-NEXT:    ret bfloat 0xR0000
 ;
   %ret = call bfloat @llvm.canonicalize.bf16(bfloat 0.0)
   ret bfloat %ret
 }
 
 define bfloat @canonicalize_1.0_bf16() {
 ; CHECK-LABEL: @canonicalize_1.0_bf16(
-; CHECK-NEXT:    [[RET:%.*]] = call bfloat @llvm.canonicalize.bf16(bfloat 0xR3F80)
-; CHECK-NEXT:    ret bfloat [[RET]]
+; CHECK-NEXT:    ret bfloat 0xR3F80
 ;
   %ret = call bfloat @llvm.canonicalize.bf16(bfloat 1.0)
   ret bfloat %ret
 }
 
 define bfloat @canonicalize_0x0001_bf16() {
 ; CHECK-LABEL: @canonicalize_0x0001_bf16(
 ; CHECK-NEXT:    [[RET:%.*]] = call bfloat @llvm.canonicalize.bf16(bfloat 0xR0001)
 ; CHECK-NEXT:    ret bfloat [[RET]]
 ;
   %ret = call bfloat @llvm.canonicalize.bf16(bfloat 0xR0001)
   ret bfloat %ret
 }
 
 define bfloat @canonicalize_inf_bf16() {
 ; CHECK-LABEL: @canonicalize_inf_bf16(
-; CHECK-NEXT:    [[RET:%.*]] = call bfloat @llvm.canonicalize.bf16(bfloat 0xR7F80)
-; CHECK-NEXT:    ret bfloat [[RET]]
+; CHECK-NEXT:    ret bfloat 0xR7F80
 ;
   %ret = call bfloat @llvm.canonicalize.bf16(bfloat 0xR7F80)
   ret bfloat %ret
 }
 
 define bfloat @canonicalize_neg_inf_bf16() {
 ; CHECK-LABEL: @canonicalize_neg_inf_bf16(
-; CHECK-NEXT:    [[RET:%.*]] = call bfloat @llvm.canonicalize.bf16(bfloat 0xRFF80)
-; CHECK-NEXT:    ret bfloat [[RET]]
+; CHECK-NEXT:    ret bfloat 0xRFF80
 ;
   %ret = call bfloat @llvm.canonicalize.bf16(bfloat 0xRFF80)
   ret bfloat %ret
 }
 
 define bfloat @canonicalize_nan_bf16() {
 ; CHECK-LABEL: @canonicalize_nan_bf16(
 ; CHECK-NEXT:    [[RET:%.*]] = call bfloat @llvm.canonicalize.bf16(bfloat 0xR7FC0)
 ; CHECK-NEXT:    ret bfloat [[RET]]
 ;
   %ret = call bfloat @llvm.canonicalize.bf16(bfloat 0xR7FC0)
   ret bfloat %ret
 }
 
 define bfloat @canonicalize_0xff_bf16() {
 ; CHECK-LABEL: @canonicalize_0xff_bf16(
-; CHECK-NEXT:    [[RET:%.*]] = call bfloat @llvm.canonicalize.bf16(bfloat 0xR00FF)
-; CHECK-NEXT:    ret bfloat [[RET]]
+; CHECK-NEXT:    ret bfloat 0xR00FF
 ;
   %ret = call bfloat @llvm.canonicalize.bf16(bfloat 0xR00FF)
   ret bfloat %ret
 }
 
 ; --------------------------------------------------------------------
 ; f80 handling
 ; --------------------------------------------------------------------
 
 define x86_fp80 @canonicalize_poison_f80() {
 ; CHECK-LABEL: @canonicalize_poison_f80(
-; CHECK-NEXT:    [[RET:%.*]] = call x86_fp80 @llvm.canonicalize.f80(x86_fp80 poison)
-; CHECK-NEXT:    ret x86_fp80 [[RET]]
+; CHECK-NEXT:    ret x86_fp80 poison
 ;
   %ret = call x86_fp80 @llvm.canonicalize.f80(x86_fp80 poison)
   ret x86_fp80 %ret
 }
 
 define x86_fp80 @canonicalize_undef_f80() {
 ; CHECK-LABEL: @canonicalize_undef_f80(
-; CHECK-NEXT:    [[RET:%.*]] = call x86_fp80 @llvm.canonicalize.f80(x86_fp80 undef)
-; CHECK-NEXT:    ret x86_fp80 [[RET]]
+; CHECK-NEXT:    ret x86_fp80 0xK00000000000000000000
 ;
   %ret = call x86_fp80 @llvm.canonicalize.f80(x86_fp80 undef)
   ret x86_fp80 %ret
 }
 
 define x86_fp80 @canonicalize_zero_f80() {
 ; CHECK-LABEL: @canonicalize_zero_f80(
-; CHECK-NEXT:    [[RET:%.*]] = call x86_fp80 @llvm.canonicalize.f80(x86_fp80 0xK00000000000000000000)
-; CHECK-NEXT:    ret x86_fp80 [[RET]]
+; CHECK-NEXT:    ret x86_fp80 0xK00000000000000000000
 ;
   %ret = call x86_fp80 @llvm.canonicalize.f80(x86_fp80 0xK00000000000000000000)
   ret x86_fp80 %ret
 }
 
 define x86_fp80 @canonicalize_negzero_f80() {
 ; CHECK-LABEL: @canonicalize_negzero_f80(
-; CHECK-NEXT:    [[RET:%.*]] = call x86_fp80 @llvm.canonicalize.f80(x86_fp80 0xK80000000000000000000)
-; CHECK-NEXT:    ret x86_fp80 [[RET]]
+; CHECK-NEXT:    ret x86_fp80 0xK80000000000000000000
 ;
   %ret = call x86_fp80 @llvm.canonicalize.f80(x86_fp80 0xK80000000000000000000)
   ret x86_fp80 %ret
 }
 
 define x86_fp80 @canonicalize_inf_f80() {
 ; CHECK-LABEL: @canonicalize_inf_f80(
 ; CHECK-NEXT:    [[RET:%.*]] = call x86_fp80 @llvm.canonicalize.f80(x86_fp80 0xK7FFF8000000000000000)

 }
 
 ; --------------------------------------------------------------------
 ; ppc_fp128 handling
 ; --------------------------------------------------------------------
 
 define ppc_fp128 @canonicalize_poison_ppcf128() {
 ; CHECK-LABEL: @canonicalize_poison_ppcf128(
-; CHECK-NEXT:    [[RET:%.*]] = call ppc_fp128 @llvm.canonicalize.ppcf128(ppc_fp128 poison)
-; CHECK-NEXT:    ret ppc_fp128 [[RET]]
+; CHECK-NEXT:    ret ppc_fp128 poison
 ;
   %ret = call ppc_fp128 @llvm.canonicalize.ppcf128(ppc_fp128 poison)
   ret ppc_fp128 %ret
 }
 
 define ppc_fp128 @canonicalize_undef_ppcf128() {
 ; CHECK-LABEL: @canonicalize_undef_ppcf128(
-; CHECK-NEXT:    [[RET:%.*]] = call ppc_fp128 @llvm.canonicalize.ppcf128(ppc_fp128 undef)
-; CHECK-NEXT:    ret ppc_fp128 [[RET]]
+; CHECK-NEXT:    ret ppc_fp128 0xM00000000000000000000000000000000
 ;
   %ret = call ppc_fp128 @llvm.canonicalize.ppcf128(ppc_fp128 undef)
   ret ppc_fp128 %ret
 }
 
 define ppc_fp128 @canonicalize_zero_ppcf128() {
 ; CHECK-LABEL: @canonicalize_zero_ppcf128(
-; CHECK-NEXT:    [[RET:%.*]] = call ppc_fp128 @llvm.canonicalize.ppcf128(ppc_fp128 0xM00000000000000000000000000000000)
-; CHECK-NEXT:    ret ppc_fp128 [[RET]]
+; CHECK-NEXT:    ret ppc_fp128 0xM00000000000000000000000000000000
 ;
   %ret = call ppc_fp128 @llvm.canonicalize.ppcf128(ppc_fp128 0xM00000000000000000000000000000000)
   ret ppc_fp128 %ret
 }
 
 define ppc_fp128 @canonicalize_negzero_ppcf128() {
 ; CHECK-LABEL: @canonicalize_negzero_ppcf128(
-; CHECK-NEXT:    [[RET:%.*]] = call ppc_fp128 @llvm.canonicalize.ppcf128(ppc_fp128 0xM80000000000000000000000000000000)
-; CHECK-NEXT:    ret ppc_fp128 [[RET]]
+; CHECK-NEXT:    ret ppc_fp128 0xM80000000000000000000000000000000
 ;
   %ret = call ppc_fp128 @llvm.canonicalize.ppcf128(ppc_fp128 0xM80000000000000000000000000000000)
   ret ppc_fp128 %ret
 }
 
 define ppc_fp128 @canonicalize_inf_ppcf128() {
 ; CHECK-LABEL: @canonicalize_inf_ppcf128(
 ; CHECK-NEXT:    [[RET:%.*]] = call ppc_fp128 @llvm.canonicalize.ppcf128(ppc_fp128 0xM7FF00000000000000000000000000000)