[SelectionDAGBuilder] Pass fast math flags to most of VP SDNodes.

[fakepaper56]

Summary

The patch does not pass math flags to float VPCmpIntrinsics because LLParser
could not identify float VPCmpIntrinsics as FPMathOperators.

[fakepaper56]

Rename class form [SelectionDAGBuilder] to [VP].

[craig.topper]

I would split this patch. One for vp.fcmp and one for the other operations. The LLParser issue is concerning and may delay the vp.fcmp support. That shouldn't block the other nodes.

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

7604

If we hit this line on an FP operation with fast math flags, would we add fast math flags to an ISD::ZERO_EXTEND?

Might be safer to pass the flags to the DAG.getNode call in the default case.

llvm/test/CodeGen/RISCV/rvv/fast-math-flags-vp.ll

4

declartion -> declaration

[fakepaper56]

The update removes the part of vp.fcmp support and using argument of DAG.getNode to pass flags.

[fakepaper56]

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

7604

Thank your reminder. I fix the bug in the latest update.

[craig.topper]

llvm/test/CodeGen/RISCV/pass-fast-math-flags-sdnode.ll

1 ↗

(On Diff #429529)

You need REQUIRES: asserts if you use -debug-only. The -debug-only option isn't available on builds without asserts.

[craig.topper]

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

7616

Why can't we do?

SDValue Result;
SDNodeFlags SDFlags;
if (auto *FPMO = dyn_cast<FPMathOperator>(&VPIntrin))
  SDFlags.copyFMF(*FPMO);
Result = DAG.getNode(Opcode, DL, VTs, OpValues, SDFlags);

[fakepaper56]

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

7616

getNode with SDVTList does not use SDNodeFlags directly.

SDValue SelectionDAG::getNode(unsigned Opcode, const SDLoc &DL, SDVTList VTList,
                              ArrayRef<SDValue> Ops, const SDNodeFlags Flags) {
  if (VTList.NumVTs == 1) 
    return getNode(Opcode, DL, VTList.VTs[0], Ops);

[craig.topper]

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

7616

Can you submit a patch to fix that? It’s a bug.

[fakepaper56]

Add REQUIRES: asserts in pass-fast-math-flags-sdnode.ll.

[fakepaper56]

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

7616

Yeah, I am willing to do that.

[fakepaper56]

After fixing getNode, pass SDNodeFlags to getNode of SDVTList version.

[craig.topper]

llvm/test/CodeGen/RISCV/pass-fast-math-flags-sdnode.ll

1 ↗

(On Diff #429712)

I didn't think about this before, but is it possible to use -stop-after=finalize-isel and inspect the MIR output instead? That wouldn't require an asserts build.

[fakepaper56]

llvm/test/CodeGen/RISCV/pass-fast-math-flags-sdnode.ll

1 ↗

(On Diff #429712)

There is no optimization of VP using fast math flags and I don't know how to make MIR print fast-math flags. The output of llc -mtriple=riscv64 -mattr=+v fa. -stop-after=finalize-isel :
%7:vrnov0 = nofpexcept PseudoVFMUL_VV_M1_MASK %8, %0, %1, $v0, killed %6, 6 /* e64 */, 1, implicit $frm

1 ↗

(On Diff #429712)

Could we get MIR

[craig.topper]

LGTM

llvm/test/CodeGen/RISCV/pass-fast-math-flags-sdnode.ll

1 ↗

(On Diff #429712)

I think we're dropping the fast math flags when we go from ISD::VP* to RISCVISD::*_VL. So my suggestion won't work without a lot more work.

We can use the debug output and REQUIRES: asserts

llvm/include/llvm/IR/Operator.h

 #define LLVM_IR_OPERATOR_H
 
 #include "llvm/ADT/MapVector.h"
 #include "llvm/ADT/None.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/IR/Constants.h"
 #include "llvm/IR/FMF.h"
 #include "llvm/IR/Instruction.h"
+#include "llvm/IR/IntrinsicInst.h"
 #include "llvm/IR/Type.h"
 #include "llvm/IR/Value.h"
 #include "llvm/Support/Casting.h"
 #include <cstddef>
 
 namespace llvm {
 
 /// This is a utility class that provides an abstraction for the common
static bool classof(const Value *V) {
     //        This would make things consistent with Select/PHI (FP value type
     //        determines whether they are math ops and, therefore, capable of
     //        having fast-math-flags).
     case Instruction::FCmp:
       return true;
     case Instruction::PHI:
     case Instruction::Select:
     case Instruction::Call: {
+      if (auto *VPCmp = dyn_cast<VPCmpIntrinsic>(V))
+        return VPCmp->getOperand(0)->getType()->isFPOrFPVectorTy();
+
       Type *Ty = V->getType();
       while (ArrayType *ArrTy = dyn_cast<ArrayType>(Ty))
         Ty = ArrTy->getElementType();
       return Ty->isFPOrFPVectorTy();
     }
     default:
       return false;
     }

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

 }
 
 void SelectionDAGBuilder::visitVPCmp(const VPCmpIntrinsic &VPIntrin) {
   const TargetLowering &TLI = DAG.getTargetLoweringInfo();
   SDLoc DL = getCurSDLoc();
 
   ISD::CondCode Condition;
   CmpInst::Predicate CondCode = VPIntrin.getPredicate();
-  bool IsFP = VPIntrin.getOperand(0)->getType()->isFPOrFPVectorTy();
-  if (IsFP) {
-    // FIXME: Regular fcmps are FPMathOperators which may have fast-math (nnan)
-    // flags, but calls that don't return floating-point types can't be
-    // FPMathOperators, like vp.fcmp. This affects constrained fcmp too.
+  if (auto *FPMO = dyn_cast<FPMathOperator>(&VPIntrin)) {
     Condition = getFCmpCondCode(CondCode);
-    if (TM.Options.NoNaNsFPMath)
+    if (FPMO->hasNoNaNs() || TM.Options.NoNaNsFPMath)
       Condition = getFCmpCodeWithoutNaN(Condition);
   } else {
     Condition = getICmpCondCode(CondCode);
   }
 
   SDValue Op1 = getValue(VPIntrin.getOperand(0));
   SDValue Op2 = getValue(VPIntrin.getOperand(1));
   // #2 is the condition code

 
 void SelectionDAGBuilder::visitVectorPredicationIntrinsic(
     const VPIntrinsic &VPIntrin) {
   SDLoc DL = getCurSDLoc();
   unsigned Opcode = getISDForVPIntrinsic(VPIntrin);
 
   auto IID = VPIntrin.getIntrinsicID();
 
+  SDNodeFlags SDFlags;
+  if (auto *FPMO = dyn_cast<FPMathOperator>(&VPIntrin))
+    SDFlags.copyFMF(*FPMO);
+  SelectionDAG::FlagInserter FlagsInserter(DAG, SDFlags);
+
   if (const auto *CmpI = dyn_cast<VPCmpIntrinsic>(&VPIntrin))
     return visitVPCmp(*CmpI);
 
   SmallVector<EVT, 4> ValueVTs;
   const TargetLowering &TLI = DAG.getTargetLoweringInfo();
   ComputeValueVTs(TLI, DAG.getDataLayout(), VPIntrin.getType(), ValueVTs);
   SDVTList VTs = DAG.getVTList(ValueVTs);
 
   auto EVLParamPos = VPIntrinsic::getVectorLengthParamPos(IID);
 
   MVT EVLParamVT = TLI.getVPExplicitVectorLengthTy();
   assert(EVLParamVT.isScalarInteger() && EVLParamVT.bitsGE(MVT::i32) &&
          "Unexpected target EVL type");
 
   // Request operands.
   SmallVector<SDValue, 7> OpValues;
   for (unsigned I = 0; I < VPIntrin.arg_size(); ++I) {
     auto Op = getValue(VPIntrin.getArgOperand(I));
     if (I == EVLParamPos)
       Op = DAG.getNode(ISD::ZERO_EXTEND, DL, EVLParamVT, Op);
     OpValues.push_back(Op);
   }
 
   switch (Opcode) {
   default: {
     SDValue Result = DAG.getNode(Opcode, DL, VTs, OpValues);
     setValue(&VPIntrin, Result);
     break;
   }
   case ISD::VP_LOAD:
   case ISD::VP_GATHER:
     visitVPLoadGather(VPIntrin, ValueVTs[0], OpValues,
                       Opcode == ISD::VP_GATHER);
     break;
   case ISD::EXPERIMENTAL_VP_STRIDED_LOAD:
     visitVPStridedLoad(VPIntrin, ValueVTs[0], OpValues);
     break;
   case ISD::VP_STORE:
   case ISD::VP_SCATTER:
     visitVPStoreScatter(VPIntrin, OpValues, Opcode == ISD::VP_SCATTER);

llvm/test/CodeGen/RISCV/rvv/fast-math-flags-vp.ll

+; RUN: llc -mtriple=riscv64 -mattr=+v -target-abi=lp64d < %s
+
+; Note: If the declartion moved after its use, there is still an error:
+; 'fast-math-flags specified for call without floating-point scalar or vector return type'.
+declare <vscale x 8 x i1> @llvm.vp.fcmp.nxv8f64(<vscale x 8 x double>, <vscale x 8 x double>, metadata, <vscale x 8 x i1>, i32)
+
+define <vscale x 8 x i1> @fcmp_uno_vf_swap_nxv8f64(<vscale x 8 x double> %va, double %b, <vscale x 8 x i1> %m, i32 zeroext %evl) {
+  %elt.head = insertelement <vscale x 8 x double> poison, double %b, i32 0
+  %vb = shufflevector <vscale x 8 x double> %elt.head, <vscale x 8 x double> poison, <vscale x 8 x i32> zeroinitializer
+  %v = call fast <vscale x 8 x i1> @llvm.vp.fcmp.nxv8f64(<vscale x 8 x double> %vb, <vscale x 8 x double> %va, metadata !"uno", <vscale x 8 x i1> %m, i32 %evl)
+  ret <vscale x 8 x i1> %v
+}