[ConstantFolding] Fold constrained compare intrinsics

[sepavloff]

Summary

The change implements constant folding of ‘llvm.experimental.constrained.fcmp’
and ‘llvm.experimental.constrained.fcmps’ intrinsics.

[bjope]

Sorry, I don't know much about fcmp.

[sepavloff]

Add check lines to the test

[kpn]

llvm/lib/Analysis/ConstantFolding.cpp

1854

This still worries me. Are _analysis_ passes allowed to change the input IR? What if the caller decides to not do a fold after calling this function to see if a fold is possible? And why is the ebIgnore case different from the other two?

Constant folding won't add this ReadNone attribute when the constant folding code doesn't have an Instruction to alter. I don't know what to do about that.

llvm/lib/IR/Instructions.cpp

4134

Besides moving this code to a new function, are there any changes to it? It's hard to tell.

llvm/test/Transforms/InstSimplify/constfold-constrained.ll

418

There are no "maytrap" tests here. For the sake of future readers it would probably be useful to show that "maytrap" was taken into account and is working correctly.

[sepavloff]

Added tests with "maytrap"

[nikic]

llvm/lib/Analysis/ConstantFolding.cpp

1854

Analysis passes are indeed not allowed to change the IR. And this isn't a harmless change either if it gets left behind -- e.g. it invalidates MemorySSA.

[sepavloff]

llvm/lib/Analysis/ConstantFolding.cpp

1854

Analysis passes are indeed not allowed to change the IR.

Constant folding is not an analysis, it changes IR. I don't know why this file is in Analysis directory.

What if the caller decides to not do a fold after calling this function to see if a fold is possible?

Nothing bad would happen. Only side effect would be removed, but we know for sure that it is absent, we just evaluated the operation. The function call would not be removed if its result is used.

And why is the ebIgnore case different from the other two?

If exception behavior is ebIgnore, such calls will get attribute SDNodeFlags::NoFPExcept in DAG and such instructions do not have side effects. But setting ReadNone for instructions with ebIgnore allows removal of these instructions earlier, at IR level, which could have positive effect.

And this isn't a harmless change either if it gets left behind -- e.g. it invalidates MemorySSA.

Comparison of two floating point numbers do not use memory access. But it can change bits in the floating point state register (only Invalid bit can be set). This change is emulated as memory access so that instruction be ordered correctly. This is why constrained intrinsics declared with attribute IntrInaccessibleMemOnly. As no actual memory access occurs, it is harmless to set ReadNone in this case.

llvm/lib/IR/Instructions.cpp

4134

No, this is only a code moving.

llvm/test/Transforms/InstSimplify/constfold-constrained.ll

418

Added tests with "maytrap".

[sepavloff]

Set ReadNone for all intrinsics if they can be evaluated without raising signals

[nikic]

llvm/lib/Analysis/ConstantFolding.cpp

1854

Constant folding is not an analysis, it changes IR. I don't know why this file is in Analysis directory.

The constant folding analysis does not change IR. Users of the constant folding analysis change IR based on the analysis result.

Comparison of two floating point numbers do not use memory access. But it can change bits in the floating point state register (only Invalid bit can be set). This change is emulated as memory access so that instruction be ordered correctly. This is why constrained intrinsics declared with attribute IntrInaccessibleMemOnly. As no actual memory access occurs, it is harmless to set ReadNone in this case.

My point here was that adding a readnone attribute invalidates MemorySSA, because it means that the instruction should no longer have a MemoryAccess -- it would result in a verification failure. Just calling ConstFold/InstSimplify should never have this kind of effect.

Unless @spatel or @lebedev.ri tell me I'm wrong here, I believe this should be considered a blocker for further work in this area.

[lebedev.ri]

Unless @spatel or @lebedev.ri tell me I'm wrong here, I believe this should be considered a blocker for further work in this area.

[sepavloff]

Move attribute modification to InstSimplify pass

[sepavloff]

Any feedback?

[craig.topper]

Should the change that effects fadd, fma, etc. tests be moved to a different patch that is a pre-requisite of the compare change?

llvm/lib/Analysis/ConstantFolding.cpp

2367

Does ConstrainedFPCmpIntrinsic have any method for determining fcmp vs fcmps? If not should it?

llvm/lib/IR/Instructions.cpp

4133

This feels a little like it shouldn't be part of FCmpInst, but I don't have a concrete suggestion of where to put it instead. ConstantFold.cpp feels like the right home, but unfortunatley the header for that isn't visible to ConstantFolding.cpp.

[sepavloff]

Addressed reviewer's notes

[sepavloff]

In D110322#3159399, @craig.topper wrote:

Should the change that effects fadd, fma, etc. tests be moved to a different patch that is a pre-requisite of the compare change?

Yes, it is a good idea to move this debatable change into a separate patch, it is here: D114766. This patch does not depend on it.

llvm/lib/Analysis/ConstantFolding.cpp

2367

Probably, but it does not help here. The interface is designed to evaluate constant value without construction of a node. ConstrainedFPCmpIntrinsic can be extracted from Call but checking intrinsic ID seems simpler.

llvm/lib/IR/Instructions.cpp

4133

I moved ConstantFold to include directory in dependency patch and put evaluatePredicate there.

[nikic]

llvm/lib/Analysis/ConstantFolding.cpp

2367

FYI recently ICmpInst::compare() was added for the corresponding operation on ICmpInsts (in https://github.com/llvm/llvm-project/commit/25043c8276644e684f8d14cd4cadaa87a7e99b0e), so it might make sense to have the same method on FCmpInst. (No strong opinion on placement though.)

[craig.topper]

llvm/lib/Analysis/ConstantFolding.cpp

2365

auto *FCmp I believe LLVM coding standards prefer to keep the * around with auto so things that are pointers are obvious.

2367

The ConstrainedFPCmpIntrinsic is already being extracted from the call to get the predicate. So I thought it would make sense to get the signaling state from it as well and not pass the intrinsic ID at all.

[sepavloff]

Changed arguments of evaluateCompare and rebased

[sepavloff]

llvm/lib/Analysis/ConstantFolding.cpp

2367

Changed signature of evaluateCompare and fixed interface of ConstrainedFPCmpIntrinsic accordingly.

[sepavloff]

Is there anything I can do to make this patch acceptable?

[kpn]

My guess:

Removing the addition of the readnone attribute? It's a correctness issue. We don't have a solution to the performance issue in generated code, but correctness beats performance every time.

[sepavloff]

In D110322#3195092, @kpn wrote:

My guess:

Removing the addition of the readnone attribute? It's a correctness issue. We don't have a solution to the performance issue in generated code, but correctness beats performance every time.

This patch now does not touch this attribute, this code is already removed.

[kpn]

llvm/lib/Analysis/ConstantFolding.cpp

1854

Here. These two lines still need to be removed. Along with the comment, of course.

[sepavloff]

llvm/lib/Analysis/ConstantFolding.cpp

1854

Done in the separate patch: https://reviews.llvm.org/D115870

[spatel]

llvm/lib/IR/ConstantFold.cpp

2028

This was suggested in an earlier comment - can we add this next to:
https://github.com/llvm/llvm-project/blob/80b1d472d6f7e10403e37cc74bfda278372ecb97/llvm/lib/IR/Instructions.cpp#L4135

(and adjust the names/parameters for consistency)

That can be a preliminary/NFC commit if I'm seeing it correctly.

[sepavloff]

llvm/lib/IR/ConstantFold.cpp

2028

Previously such method existed but I was recommended to remove it: https://reviews.llvm.org/D110322?id=388475#3159399.

Otherwise there is no problem to restore that implementation.

[spatel]

I have no objections to this patch now. I'd pre-commit the baseline tests though, and let's see if there's consensus on where to house the helper function.

llvm/lib/IR/ConstantFold.cpp

2028

Ah, sorry I didn't see that earlier comment/revision.
@craig.topper - do you still prefer the current structure?

[craig.topper]

llvm/lib/IR/ConstantFold.cpp

2028

I think the integer change happened while this patch was in review. I'm fine with restoring the previous implementation to make this consistent with integer.

[sepavloff]

Put back method of FCmpInst for constant comparison evaluation.

[spatel]

Please pre-commit the baseline tests and FCmpInst::compare(), so we're left with only real logic changes and tests that show the change in behavior.

[sepavloff]

FCmpInst::compare is moved to separate patch

[sepavloff]

Rebased patch

[sepavloff]

Any feedback?

[kpn]

LGTM

[sepavloff]

Thanks!

[bgraur]

Hi,

This commit causes clang to crash when compiling the attached reproducer with the following compilation command:

clang -cc1 \
  -emit-obj \
  -target-feature +sse4.2 \
  -frounding-math \
  -O1 \
  -std=gnu++17 \
  -fsized-deallocation \
  -o /tmp/repro.o -x c++ repro.cc

Please note that the crash only reproduces something like 60% of cases (just re-run a few times).
Please revert.

repro.cc633 BDownload

[sepavloff]

In D110322#3346560, @bgraur wrote:

Hi,

This commit causes clang to crash when compiling the attached reproducer with the following compilation command:

clang -cc1 \
  -emit-obj \
  -target-feature +sse4.2 \
  -frounding-math \
  -O1 \
  -std=gnu++17 \
  -fsized-deallocation \
  -o /tmp/repro.o -x c++ repro.cc

Please note that the crash only reproduces something like 60% of cases (just re-run a few times).
Please revert.

repro.cc633 BDownload

Thank you for the report. There was an issue with treatment of vector constants. Commit https://reviews.llvm.org/rG6982c38cb120 fixes it.

llvm/include/llvm/IR/Instructions.h

public:
   /// changed to retain the same result if the predicate is order dependent
   /// (e.g. ult).
   /// Swap operands and adjust predicate.
   void swapOperands() {
     setPredicate(getSwappedPredicate());
     Op<0>().swap(Op<1>());
   }
 
+  /// Evaluate the given predicate for the constant values.
+  static bool evaluate(Predicate Pred, const APFloat &Op1, const APFloat &Op2);
+
   /// Methods for support type inquiry through isa, cast, and dyn_cast:
   static bool classof(const Instruction *I) {
     return I->getOpcode() == Instruction::FCmp;
   }
   static bool classof(const Value *V) {
     return isa<Instruction>(V) && classof(cast<Instruction>(V));
   }
 };

llvm/lib/Analysis/ConstantFolding.cpp

bool llvm::canConstantFoldCallTo(const CallBase *Call, const Function *F) {
   case Intrinsic::experimental_constrained_frem:
   case Intrinsic::experimental_constrained_ceil:
   case Intrinsic::experimental_constrained_floor:
   case Intrinsic::experimental_constrained_round:
   case Intrinsic::experimental_constrained_roundeven:
   case Intrinsic::experimental_constrained_trunc:
   case Intrinsic::experimental_constrained_nearbyint:
   case Intrinsic::experimental_constrained_rint:
+  case Intrinsic::experimental_constrained_fcmp:
+  case Intrinsic::experimental_constrained_fcmps:
     return true;
   default:
     return false;
   case Intrinsic::not_intrinsic: break;
   }
 
   if (!F->hasName() || Call->isStrictFP())
     return false;
static bool getConstIntOrUndef(Value *Op, const APInt *&C) {
   return false;
 }
 
 /// Checks if the given intrinsic call, which evaluates to constant, is allowed
 /// to be folded.
 ///
 /// \param CI Constrained intrinsic call.
 /// \param St Exception flags raised during constant evaluation.
-static bool mayFoldConstrained(ConstrainedFPIntrinsic *CI,
+static bool mayFoldConstrained(const ConstrainedFPIntrinsic *CI,
                                APFloat::opStatus St) {
   Optional<RoundingMode> ORM = CI->getRoundingMode();
   Optional<fp::ExceptionBehavior> EB = CI->getExceptionBehavior();
 
   // If the operation does not change exception status flags, it is safe
   // to fold.
   if (St == APFloat::opStatus::opOK) {
     // When FP exceptions are not ignored, intrinsic call will not be
     // eliminated, because it is considered as having side effect. But we
     // know that its evaluation does not raise exceptions, so side effect
     // is absent. To allow removing the call, mark it as not accessing memory.
     if (EB && *EB != fp::ExceptionBehavior::ebIgnore)
-      CI->addFnAttr(Attribute::ReadNone);
+      const_cast<ConstrainedFPIntrinsic *>(CI)->addFnAttr(Attribute::ReadNone);
     return true;
   }
 
   // If evaluation raised FP exception, the result can depend on rounding
   // mode. If the latter is unknown, folding is not possible.
-  if (!ORM || *ORM == RoundingMode::Dynamic)
+  if (ORM && *ORM == RoundingMode::Dynamic)
     return false;
 
   // If FP exceptions are ignored, fold the call, even if such exception is
   // raised.
   if (!EB || *EB != fp::ExceptionBehavior::ebStrict)
     return true;
 
   // Leave the calculation for runtime so that exception flags be correctly set
case Intrinsic::x86_sse2_cvttsd2si64:
                                            /*IsSigned*/true);
       break;
     }
   }
 
   return nullptr;
 }
 
+static Constant *evaluateCompare(Intrinsic::ID IntrinsicID,
+                                 const CallBase *Call, const APFloat &Op1,
+                                 const APFloat &Op2) {
+  APFloat::opStatus St = APFloat::opOK;
+  auto FCmp = cast<ConstrainedFPCmpIntrinsic>(Call);
+  FCmpInst::Predicate Cond = FCmp->getPredicate();
+  if (IntrinsicID == Intrinsic::experimental_constrained_fcmp) {
+    if (Op1.isSignaling() || Op2.isSignaling())
+      St = APFloat::opInvalidOp;
+  } else {
+    assert(IntrinsicID == Intrinsic::experimental_constrained_fcmps);
+    if (Op1.isNaN() || Op2.isNaN())
+      St = APFloat::opInvalidOp;
+  }
+  bool Result = FCmpInst::evaluate(Cond, Op1, Op2);
+  if (mayFoldConstrained(FCmp, St))
+    return ConstantInt::get(Call->getType(), Result);
+  return nullptr;
+}
+
 static Constant *ConstantFoldScalarCall2(StringRef Name,
                                          Intrinsic::ID IntrinsicID,
                                          Type *Ty,
                                          ArrayRef<Constant *> Operands,
                                          const TargetLibraryInfo *TLI,
                                          const CallBase *Call) {
   assert(Operands.size() == 2 && "Wrong number of operands.");
 
case Intrinsic::minimum:
         return Operands[1];
       if (IsOp1Undef)
         return Operands[0];
       break;
     }
   }
 
   if (const auto *Op1 = dyn_cast<ConstantFP>(Operands[0])) {
-    if (!Ty->isFloatingPointTy())
-      return nullptr;
     APFloat Op1V = Op1->getValueAPF();
 
     if (const auto *Op2 = dyn_cast<ConstantFP>(Operands[1])) {
       if (Op2->getType() != Op1->getType())
         return nullptr;
       APFloat Op2V = Op2->getValueAPF();
 
       if (const auto *ConstrIntr = dyn_cast<ConstrainedFPIntrinsic>(Call)) {
if (const auto *Op2 = dyn_cast<ConstantFP>(Operands[1])) {
           St = Res.multiply(Op2V, RM);
           break;
         case Intrinsic::experimental_constrained_fdiv:
           St = Res.divide(Op2V, RM);
           break;
         case Intrinsic::experimental_constrained_frem:
           St = Res.mod(Op2V);
           break;
+        case Intrinsic::experimental_constrained_fcmp:
+        case Intrinsic::experimental_constrained_fcmps:
+          return evaluateCompare(IntrinsicID, Call, Op1V, Op2V);
         }
-        if (mayFoldConstrained(const_cast<ConstrainedFPIntrinsic *>(ConstrIntr),
-                               St))
+        if (mayFoldConstrained(ConstrIntr, St))
           return ConstantFP::get(Ty->getContext(), Res);
         return nullptr;
       }
 
       switch (IntrinsicID) {
       default:
         break;
       case Intrinsic::copysign:
if (const auto *Op2 = dyn_cast<ConstantFP>(Operands[1])) {
           switch (IntrinsicID) {
           default:
             return nullptr;
           case Intrinsic::experimental_constrained_fma:
           case Intrinsic::experimental_constrained_fmuladd:
             St = Res.fusedMultiplyAdd(C2, C3, RM);
             break;
           }
-          if (mayFoldConstrained(
-                  const_cast<ConstrainedFPIntrinsic *>(ConstrIntr), St))
+          if (mayFoldConstrained(ConstrIntr, St))
             return ConstantFP::get(Ty->getContext(), Res);
           return nullptr;
         }
 
         switch (IntrinsicID) {
         default: break;
         case Intrinsic::amdgcn_fma_legacy: {
           // The legacy behaviour is that multiplying +/- 0.0 by anything, even

llvm/lib/IR/ConstantFold.cpp

if (isa<ConstantInt>(C1) && isa<ConstantInt>(C2)) {
     case ICmpInst::ICMP_ULT: return ConstantInt::get(ResultTy, V1.ult(V2));
     case ICmpInst::ICMP_UGT: return ConstantInt::get(ResultTy, V1.ugt(V2));
     case ICmpInst::ICMP_ULE: return ConstantInt::get(ResultTy, V1.ule(V2));
     case ICmpInst::ICMP_UGE: return ConstantInt::get(ResultTy, V1.uge(V2));
     }
   } else if (isa<ConstantFP>(C1) && isa<ConstantFP>(C2)) {
     const APFloat &C1V = cast<ConstantFP>(C1)->getValueAPF();
     const APFloat &C2V = cast<ConstantFP>(C2)->getValueAPF();
-    APFloat::cmpResult R = C1V.compare(C2V);
-    switch (pred) {
-    default: llvm_unreachable("Invalid FCmp Predicate");
-    case FCmpInst::FCMP_FALSE: return Constant::getNullValue(ResultTy);
-    case FCmpInst::FCMP_TRUE:  return Constant::getAllOnesValue(ResultTy);
-    case FCmpInst::FCMP_UNO:
-      return ConstantInt::get(ResultTy, R==APFloat::cmpUnordered);
-    case FCmpInst::FCMP_ORD:
-      return ConstantInt::get(ResultTy, R!=APFloat::cmpUnordered);
-    case FCmpInst::FCMP_UEQ:
-      return ConstantInt::get(ResultTy, R==APFloat::cmpUnordered ||
-                                        R==APFloat::cmpEqual);
-    case FCmpInst::FCMP_OEQ:
-      return ConstantInt::get(ResultTy, R==APFloat::cmpEqual);
-    case FCmpInst::FCMP_UNE:
-      return ConstantInt::get(ResultTy, R!=APFloat::cmpEqual);
-    case FCmpInst::FCMP_ONE:
-      return ConstantInt::get(ResultTy, R==APFloat::cmpLessThan ||
-                                        R==APFloat::cmpGreaterThan);
-    case FCmpInst::FCMP_ULT:
-      return ConstantInt::get(ResultTy, R==APFloat::cmpUnordered ||
-                                        R==APFloat::cmpLessThan);
-    case FCmpInst::FCMP_OLT:
-      return ConstantInt::get(ResultTy, R==APFloat::cmpLessThan);
-    case FCmpInst::FCMP_UGT:
-      return ConstantInt::get(ResultTy, R==APFloat::cmpUnordered ||
-                                        R==APFloat::cmpGreaterThan);
-    case FCmpInst::FCMP_OGT:
-      return ConstantInt::get(ResultTy, R==APFloat::cmpGreaterThan);
-    case FCmpInst::FCMP_ULE:
-      return ConstantInt::get(ResultTy, R!=APFloat::cmpGreaterThan);
-    case FCmpInst::FCMP_OLE:
-      return ConstantInt::get(ResultTy, R==APFloat::cmpLessThan ||
-                                        R==APFloat::cmpEqual);
-    case FCmpInst::FCMP_UGE:
-      return ConstantInt::get(ResultTy, R!=APFloat::cmpLessThan);
-    case FCmpInst::FCMP_OGE:
-      return ConstantInt::get(ResultTy, R==APFloat::cmpGreaterThan ||
-                                        R==APFloat::cmpEqual);
-    }
+    CmpInst::Predicate Predicate = CmpInst::Predicate(pred);
+    return ConstantInt::get(ResultTy, FCmpInst::evaluate(Predicate, C1V, C2V));
   } else if (auto *C1VTy = dyn_cast<VectorType>(C1->getType())) {
 
     // Fast path for splatted constants.
     if (Constant *C1Splat = C1->getSplatValue())
       if (Constant *C2Splat = C2->getSplatValue())
         return ConstantVector::getSplat(
             C1VTy->getElementCount(),
             ConstantExpr::getCompare(pred, C1Splat, C2Splat));
if ((!isa<ConstantExpr>(C1) && isa<ConstantExpr>(C2)) ||
       pred = ICmpInst::getSwappedPredicate((ICmpInst::Predicate)pred);
       return ConstantExpr::getICmp(pred, C2, C1);
     }
   }
   return nullptr;
 }
 
 /// Test whether the given sequence of *normalized* indices is "inbounds".
 template<typename IndexTy>
 static bool isInBoundsIndices(ArrayRef<IndexTy> Idxs) {
   // No indices means nothing that could be out of bounds.
   if (Idxs.empty()) return true;
 
   // If the first index is zero, it's in bounds.
   if (cast<Constant>(Idxs[0])->isNullValue()) return true;
 
   // If the first index is one and all the rest are zero, it's in bounds,

llvm/lib/IR/Instructions.cpp

bool CmpInst::isImpliedTrueByMatchingCmp(Predicate Pred1, Predicate Pred2) {
   }
   return false;
 }
 
 bool CmpInst::isImpliedFalseByMatchingCmp(Predicate Pred1, Predicate Pred2) {
   return isImpliedTrueByMatchingCmp(Pred1, getInversePredicate(Pred2));
 }
 
+bool FCmpInst::evaluate(Predicate Pred, const APFloat &Op1,
+                        const APFloat &Op2) {
+  APFloat::cmpResult R = Op1.compare(Op2);
+  switch (Pred) {
+  default:
+    llvm_unreachable("Invalid FCmp Predicate");
+  case FCmpInst::FCMP_FALSE:
+    return false;
+  case FCmpInst::FCMP_TRUE:
+    return true;
+  case FCmpInst::FCMP_UNO:
+    return R == APFloat::cmpUnordered;
+  case FCmpInst::FCMP_ORD:
+    return R != APFloat::cmpUnordered;
+  case FCmpInst::FCMP_UEQ:
+    return R == APFloat::cmpUnordered || R == APFloat::cmpEqual;
+  case FCmpInst::FCMP_OEQ:
+    return R == APFloat::cmpEqual;
+  case FCmpInst::FCMP_UNE:
+    return R != APFloat::cmpEqual;
+  case FCmpInst::FCMP_ONE:
+    return R == APFloat::cmpLessThan || R == APFloat::cmpGreaterThan;
+  case FCmpInst::FCMP_ULT:
+    return R == APFloat::cmpUnordered || R == APFloat::cmpLessThan;
+  case FCmpInst::FCMP_OLT:
+    return R == APFloat::cmpLessThan;
+  case FCmpInst::FCMP_UGT:
+    return R == APFloat::cmpUnordered || R == APFloat::cmpGreaterThan;
+  case FCmpInst::FCMP_OGT:
+    return R == APFloat::cmpGreaterThan;
+  case FCmpInst::FCMP_ULE:
+    return R != APFloat::cmpGreaterThan;
+  case FCmpInst::FCMP_OLE:
+    return R == APFloat::cmpLessThan || R == APFloat::cmpEqual;
+  case FCmpInst::FCMP_UGE:
+    return R != APFloat::cmpLessThan;
+  case FCmpInst::FCMP_OGE:
+    return R == APFloat::cmpGreaterThan || R == APFloat::cmpEqual;
+  }
+}
+
 //===----------------------------------------------------------------------===//
 //                        SwitchInst Implementation
 //===----------------------------------------------------------------------===//
 
 void SwitchInst::init(Value *Value, BasicBlock *Default, unsigned NumReserved) {
   assert(Value && Default && NumReserved);
   ReservedSpace = NumReserved;
   setNumHungOffUseOperands(2);

llvm/test/Transforms/InstSimplify/constfold-constrained.ll

 ; CHECK-NEXT:    ret double 5.000000e+00
 ;
 entry:
   %result = call double @llvm.experimental.constrained.fmuladd.f64(double 1.0, double 2.0, double 3.0, metadata !"round.dynamic", metadata !"fpexcept.ignore") #0
   ret double %result
 }
 
 
+; When exceptions are ignored, comparison of constants can be folded, even for (signaling) NaNs.
+define i1 @cmp_eq_01() #0 {
+entry:
+  %result = call i1 @llvm.experimental.constrained.fcmp.f64(double 1.0, double 2.0, metadata !"oeq", metadata !"fpexcept.ignore") #0
+  ret i1 %result
+}
+
+define i1 @cmp_eq_02() #0 {
+entry:
+  %result = call i1 @llvm.experimental.constrained.fcmp.f64(double 2.0, double 2.0, metadata !"oeq", metadata !"fpexcept.ignore") #0
+  ret i1 %result
+}
+
+define i1 @cmp_eq_03() #0 {
+entry:
+  %result = call i1 @llvm.experimental.constrained.fcmp.f64(double 2.0, double 0x7ff8000000000000, metadata !"oeq", metadata !"fpexcept.ignore") #0
+  ret i1 %result
+}
+
+define i1 @cmp_eq_04() #0 {
+entry:
+  %result = call i1 @llvm.experimental.constrained.fcmp.f64(double 2.0, double 0x7ff4000000000000, metadata !"oeq", metadata !"fpexcept.ignore") #0
+  ret i1 %result
+}
+
+define i1 @cmp_eq_05() #0 {
+entry:
+  %result = call i1 @llvm.experimental.constrained.fcmps.f64(double 2.0, double 0x7ff8000000000000, metadata !"oeq", metadata !"fpexcept.ignore") #0
+  ret i1 %result
+}
+
+define i1 @cmp_eq_06() #0 {
+entry:
+  %result = call i1 @llvm.experimental.constrained.fcmps.f64(double 2.0, double 0x7ff4000000000000, metadata !"oeq", metadata !"fpexcept.ignore") #0
+  ret i1 %result
+}
+
+; Compare with SNAN is NOT folded if the exception behavior mode is not 'ignore'.
+define i1 @cmp_eq_nan_01() #0 {
+entry:
+  %result = call i1 @llvm.experimental.constrained.fcmp.f64(double 0x7ff4000000000000, double 1.0, metadata !"oeq", metadata !"fpexcept.strict") #0
+  ret i1 %result
+}
+
+define i1 @cmp_eq_nan_02() #0 {
+entry:
+  %result = call i1 @llvm.experimental.constrained.fcmps.f64(double 0x7ff4000000000000, double 1.0, metadata !"oeq", metadata !"fpexcept.strict") #0
+  ret i1 %result
+}
+
+; Compare with QNAN is folded for fcmp but is NOT folded for fcmps if the exception behavior mode is not 'ignore'.
+define i1 @cmp_eq_nan_03() #0 {
+entry:
+  %result = call i1 @llvm.experimental.constrained.fcmp.f64(double 0x7ff8000000000000, double 1.0, metadata !"oeq", metadata !"fpexcept.strict") #0
+  ret i1 %result
+}
+
+define i1 @cmp_eq_nan_04() #0 {
+entry:
+  %result = call i1 @llvm.experimental.constrained.fcmps.f64(double 0x7ff8000000000000, double 1.0, metadata !"oeq", metadata !"fpexcept.strict") #0
+  ret i1 %result
+}
+
+
 attributes #0 = { strictfp }
 
 declare double @llvm.experimental.constrained.nearbyint.f64(double, metadata, metadata)
 declare double @llvm.experimental.constrained.floor.f64(double, metadata)
 declare double @llvm.experimental.constrained.ceil.f64(double, metadata)
 declare double @llvm.experimental.constrained.trunc.f64(double, metadata)
 declare double @llvm.experimental.constrained.round.f64(double, metadata)
 declare double @llvm.experimental.constrained.rint.f64(double, metadata, metadata)
 declare double @llvm.experimental.constrained.fadd.f64(double, double, metadata, metadata)
 declare float @llvm.experimental.constrained.fadd.f32(float, float, metadata, metadata)
 declare half @llvm.experimental.constrained.fadd.f16(half, half, metadata, metadata)
 declare bfloat @llvm.experimental.constrained.fadd.bf16(bfloat, bfloat, metadata, metadata)
 declare double @llvm.experimental.constrained.fsub.f64(double, double, metadata, metadata)
 declare double @llvm.experimental.constrained.fmul.f64(double, double, metadata, metadata)
 declare double @llvm.experimental.constrained.fdiv.f64(double, double, metadata, metadata)
 declare double @llvm.experimental.constrained.frem.f64(double, double, metadata, metadata)
 declare double @llvm.experimental.constrained.fma.f64(double, double, double, metadata, metadata)
 declare double @llvm.experimental.constrained.fmuladd.f64(double, double, double, metadata, metadata)
+declare i1 @llvm.experimental.constrained.fcmp.f64(double, double, metadata, metadata)
+declare i1 @llvm.experimental.constrained.fcmps.f64(double, double, metadata, metadata)