[PowerPC] Require NSZ flag for c-a*b to FNMSUB

[qiucf]

Summary

On PowerPC, FNMSUB (including VSX and non-VSX version) means -(a*b-c). But the backend now generates these instructions regardless whether nsz flag exists or not.

If a*b-c==0, such transformation changes sign of zero. This patch fixes this issue.

[qiucf]

llvm/test/CodeGen/PowerPC/fma-negate.ll

291

This should be a regression. And the reason is SplitVectorRes did not pass SDNode flags to legalized ones. If that's truly a bug, I will cook a patch for it.

[spatel]

llvm/test/CodeGen/PowerPC/fma-negate.ll

291

This should not be a regression, right? If we dropped the flags during legalization, that is a bug.

[qiucf]

llvm/test/CodeGen/PowerPC/fma-negate.ll

291

I posted the fix at D76832 :)

[spatel]

llvm/test/CodeGen/PowerPC/fma-negate.ll

291

Thanks! Please rebase this patch after that the other one is pushed.
PPC experts can judge whether we want to proceed with this one or wait for possible improvements in getNegatibleCost().
We have known bugs with that logic (D76638, D76439), but I'm not sure yet what the best solution is. @qshanz may have a better idea.

[jsji]

@steven.zhang Can you please *Disable* or *Delete* the deprecated account @qshanz to avoid confusion. Thanks.

[qiucf]

Rebase to reflect changes after D76832

[amyk]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

1485

Make the return statement inline with the rest of the return statements.

15658

This could be a silly question but I'm curious and wanting to learn the transformation you implemented. When I read the comment "choose the cheaper one to negate," it looks like in the first condition that N1 is the cheaper one but N0 is being negated. Could you elaborate on this a little more?

16012

Maybe good to add a comment to this function to briefly describe the transformation you are trying to do.

16024

Did you mean, we don't need to care about fnmadd?

16025

s/transform/transformation

[steven.zhang]

In D76585#1943627, @jsji wrote:

@steven.zhang Can you please *Disable* or *Delete* the deprecated account @qshanz to avoid confusion. Thanks.

I have updated the profile. Thank you for reminder.

[qiucf]

Address comments

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

1485

Thanks. clang-format always changes this line...

15658

Yes. It's a little bit confusing since NegatibleCost defines Expensive=0; Neutral=1; Cheaper=2. And code in DAG combiner also follows such convention.

16024

I think at least for now we don't really need fnmadd since dag combiner already handles it ((fneg (fma ...))). But if we drop code for fnmsub here, we would get worse code sequence.

[qiucf]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

15658

This should be fixed after D77993.

[qiucf]

Rebase after the getNegatibleCost refactoring patch: D77319

[steven.zhang]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

15767

You should check the value of NegN2 instead of the cost though their semantics might be the same.

15783

Please refer to how we did for FMA. i.e. Your implementation has problems when A=neutral, B=neutral, and C=cheaper

[qiucf]

Address comments from Steven

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

15783

Thanks, I was not aware of (A=neutral && B=neutral && C=cheaper) => cheaper.

[steven.zhang]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

15780

As we already know that, N0Cost <= N1Cost, so, what we need to do is as:
Cost = std::min(N0Cost, N2Cost);
I will update the logic in DAGCombine there as it has similar issue.

[qiucf]

Simplify a cost calculation

[RKSimon]

This will need refactoring after D77319

[steven.zhang]

@spatel @RKSimon Do you know that why we didn't have the generic node for FNMSUB/FMSUB as I see the duplicate logic between X86 and PowerPC ?

[spatel]

In D76585#2009259, @steven.zhang wrote:

@spatel @RKSimon Do you know that why we didn't have the generic node for FNMSUB/FMSUB as I see the duplicate logic between X86 and PowerPC ?

I can't say for sure, but my guess is that target-specific logic could be added without incurring the overhead of general discussion/approval:
rG3cb3b0045c62bc6cc

There's also a question of deciding what name corresponds to each math operation. The x86 instructions don't match PPC. For example:
[PPC] fnmsub = -(A * B - C)
[x86] vfnmsub132sd = -(A * B) - C

It would be nice to have common nodes/optimizations if we can find enough code to share. If we do that, we also need to see if other targets can benefit (AArch64 has fnmsub machine IR).

[steven.zhang]

In D76585#2009908, @spatel wrote:

In D76585#2009259, @steven.zhang wrote:

@spatel @RKSimon Do you know that why we didn't have the generic node for FNMSUB/FMSUB as I see the duplicate logic between X86 and PowerPC ?

I can't say for sure, but my guess is that target-specific logic could be added without incurring the overhead of general discussion/approval:
rG3cb3b0045c62bc6cc

There's also a question of deciding what name corresponds to each math operation. The x86 instructions don't match PPC. For example:
[PPC] fnmsub = -(A * B - C)
[x86] vfnmsub132sd = -(A * B) - C

It would be nice to have common nodes/optimizations if we can find enough code to share. If we do that, we also need to see if other targets can benefit (AArch64 has fnmsub machine IR).

Thank you for the information. Maybe,we can add fmsub(A*B-C), fnmsub(-A*B-C), fnmadd(-A*B+C). But yes, we need to see if other targets can benefit from this.

[qiucf]

Fix test failure after several td/case changes

[steven.zhang]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

16242

Can we move all these logic into getNegatedExpression so that more opportunity will be found？

[qiucf]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

16242

Removing main part of these logic into getNegatedExpression makes code cleaner. But FNMSUB nodes will never try negate unless we combine it?

[steven.zhang]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

16242

ok. I think, we still need to combine it as FNMSUB as what you did now to reduce FNEG. But we miss to do the negation between FMA and FNMSUB. But it could be a follow up patch.

16252

Do we need this transformation ? I didn't see the benefit of doing this as it didn't reduces the FNEG.

[qiucf]

Address Steven's comments. Fixed a crash due to type legalizing.

[steven.zhang]

LGTM now overall with some minor comments. As you are adding the pattern for VNMSUBFP, I would suggest you do it in another patch.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

15818

Remove the InvertedOpc but use FMA directly as we know that it is FNMSUB now. And move the invertFMAOpcode as lambda of the getNegatedExpression.

15833

It will be great if you can add more wording on how it changes the sign of zeroes here.

[qiucf]

Move vnmsubfp exploitation to another patch

[steven.zhang]

LGTM now as far as you update the comments in getNegatedExpression about why we are changing the sign of zero. But please hold on several days in case others have more comments,

llvm/lib/Target/PowerPC/PPCISelLowering.h

enum NodeType : unsigned {
     /// are used, where n is log2(sizeof(element) * 8). See sld/slw, etc.
     /// for exact behaviors.
     ///
     /// For vector types, only the last n bits are used. See vsld.
     SRL,
     SRA,
     SHL,
 
+    /// FNMSUB - Negated multiply-subtract instruction.
+    FNMSUB,
+
     /// EXTSWSLI = The PPC extswsli instruction, which does an extend-sign
     /// word and shift left immediate.
     EXTSWSLI,
 
     /// The combination of sra[wd]i and addze used to implemented signed
     /// integer division by a power of 2. The first operand is the dividend,
     /// and the second is the constant shift amount (representing the
     /// divisor).
public:
     }
 
     bool preferIncOfAddToSubOfNot(EVT VT) const override;
 
     bool convertSetCCLogicToBitwiseLogic(EVT VT) const override {
       return VT.isScalarInteger();
     }
 
+    SDValue getNegatedExpression(SDValue Op, SelectionDAG &DAG, bool LegalOps,
+                                 bool OptForSize, NegatibleCost &Cost,
+                                 unsigned Depth = 0) const override;
+
     /// getSetCCResultType - Return the ISD::SETCC ValueType
     EVT getSetCCResultType(const DataLayout &DL, LLVMContext &Context,
                            EVT VT) const override;
 
     /// Return true if target always beneficiates from combining into FMA for a
     /// given value type. This must typically return false on targets where FMA
     /// takes more cycles to execute than FADD.
     bool enableAggressiveFMAFusion(EVT VT) const override;
private:
     SDValue DAGCombineTruncBoolExt(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineStoreFPToInt(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineFPToIntToFP(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineSHL(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineSRA(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineSRL(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineMUL(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineADD(SDNode *N, DAGCombinerInfo &DCI) const;
+    SDValue combineFMALike(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineTRUNCATE(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineSetCC(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineABS(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineVSelect(SDNode *N, DAGCombinerInfo &DCI) const;
     SDValue combineVReverseMemOP(ShuffleVectorSDNode *SVN, LSBaseSDNode *LSBase,
                                  DAGCombinerInfo &DCI) const;
 
     /// ConvertSETCCToSubtract - looks at SETCC that compares ints. It replaces

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

   setStackPointerRegisterToSaveRestore(isPPC64 ? PPC::X1 : PPC::R1);
 
   // We have target-specific dag combine patterns for the following nodes:
   setTargetDAGCombine(ISD::ADD);
   setTargetDAGCombine(ISD::SHL);
   setTargetDAGCombine(ISD::SRA);
   setTargetDAGCombine(ISD::SRL);
   setTargetDAGCombine(ISD::MUL);
+  setTargetDAGCombine(ISD::FMA);
   setTargetDAGCombine(ISD::SINT_TO_FP);
   setTargetDAGCombine(ISD::BUILD_VECTOR);
   if (Subtarget.hasFPCVT())
     setTargetDAGCombine(ISD::UINT_TO_FP);
   setTargetDAGCombine(ISD::LOAD);
   setTargetDAGCombine(ISD::STORE);
   setTargetDAGCombine(ISD::BR_CC);
   if (Subtarget.useCRBits())

     return "PPCISD::ANDI_rec_1_GT_BIT";
   case PPCISD::VCMP:            return "PPCISD::VCMP";
   case PPCISD::VCMPo:           return "PPCISD::VCMPo";
   case PPCISD::LBRX:            return "PPCISD::LBRX";
   case PPCISD::STBRX:           return "PPCISD::STBRX";
   case PPCISD::LFIWAX:          return "PPCISD::LFIWAX";
   case PPCISD::LFIWZX:          return "PPCISD::LFIWZX";
   case PPCISD::LXSIZX:          return "PPCISD::LXSIZX";
   case PPCISD::STXSIX:          return "PPCISD::STXSIX";
   case PPCISD::VEXTS:           return "PPCISD::VEXTS";
   case PPCISD::LXVD2X:          return "PPCISD::LXVD2X";
   case PPCISD::STXVD2X:         return "PPCISD::STXVD2X";
   case PPCISD::LOAD_VEC_BE:     return "PPCISD::LOAD_VEC_BE";
   case PPCISD::STORE_VEC_BE:    return "PPCISD::STORE_VEC_BE";
   case PPCISD::ST_VSR_SCAL_INT:
                                 return "PPCISD::ST_VSR_SCAL_INT";
   case PPCISD::COND_BRANCH:     return "PPCISD::COND_BRANCH";

   case PPCISD::BUILD_FP128:     return "PPCISD::BUILD_FP128";
   case PPCISD::BUILD_SPE64:     return "PPCISD::BUILD_SPE64";
   case PPCISD::EXTRACT_SPE:     return "PPCISD::EXTRACT_SPE";
   case PPCISD::EXTSWSLI:        return "PPCISD::EXTSWSLI";
   case PPCISD::LD_VSX_LH:       return "PPCISD::LD_VSX_LH";
   case PPCISD::FP_EXTEND_HALF:  return "PPCISD::FP_EXTEND_HALF";
   case PPCISD::MAT_PCREL_ADDR:  return "PPCISD::MAT_PCREL_ADDR";
   case PPCISD::LD_SPLAT:        return "PPCISD::LD_SPLAT";
+  case PPCISD::FNMSUB:          return "PPCISD::FNMSUB";
   }
   return nullptr;
 }
 
 EVT PPCTargetLowering::getSetCCResultType(const DataLayout &DL, LLVMContext &C,
                                           EVT VT) const {
   if (!VT.isVector())
     return Subtarget.useCRBits() ? MVT::i1 : MVT::i32;

   case ISD::SHL:
     return combineSHL(N, DCI);
   case ISD::SRA:
     return combineSRA(N, DCI);
   case ISD::SRL:
     return combineSRL(N, DCI);
   case ISD::MUL:
     return combineMUL(N, DCI);
+  case ISD::FMA:
+  case PPCISD::FNMSUB:
+    return combineFMALike(N, DCI);
   case PPCISD::SHL:
     if (isNullConstant(N->getOperand(0))) // 0 << V -> 0.
         return N->getOperand(0);
     break;
   case PPCISD::SRL:
     if (isNullConstant(N->getOperand(0))) // 0 >>u V -> 0.
         return N->getOperand(0);
     break;

 }
 
 bool PPCTargetLowering::allowsMisalignedMemoryAccesses(EVT VT,
                                                        unsigned,
                                                        unsigned,
                                                        MachineMemOperand::Flags,
                                                        bool *Fast) const {
   if (DisablePPCUnaligned)
     return false;
 
   // PowerPC supports unaligned memory access for simple non-vector types.
   // Although accessing unaligned addresses is not as efficient as accessing
   // aligned addresses, it is generally more efficient than manual expansion,
   // and generally only traps for software emulation when crossing page
   // boundaries.
 
   if (!VT.isSimple())

 }
 
 bool
 PPCTargetLowering::shouldExpandBuildVectorWithShuffles(
                      EVT VT , unsigned DefinedValues) const {
   if (VT == MVT::v2i64)
     return Subtarget.hasDirectMove(); // Don't need stack ops with direct moves
 
   if (Subtarget.hasVSX() || Subtarget.hasQPX())
     return true;
 
   return TargetLowering::shouldExpandBuildVectorWithShuffles(VT, DefinedValues);
 }
 
 Sched::Preference PPCTargetLowering::getSchedulingPreference(SDNode *N) const {
   if (DisableILPPref || Subtarget.enableMachineScheduler())
     return TargetLowering::getSchedulingPreference(N);
 
   return Sched::ILP;
 }
 
 // Create a fast isel object.
 FastISel *
 PPCTargetLowering::createFastISel(FunctionLoweringInfo &FuncInfo,
                                   const TargetLibraryInfo *LibInfo) const {
   return PPC::createFastISel(FuncInfo, LibInfo);
 }
 
+// 'Inverted' means the FMA opcode after negating one multiplicand.
+// For example, (fma -a b c) = (fnmsub a b c)
+static unsigned invertFMAOpcode(unsigned Opc) {
+  switch (Opc) {
+  default:
+    llvm_unreachable("Invalid FMA opcode for PowerPC!");
+  case ISD::FMA:
+    return PPCISD::FNMSUB;
+  case PPCISD::FNMSUB:
+    return ISD::FMA;
+  }
+}
+
+SDValue PPCTargetLowering::getNegatedExpression(SDValue Op, SelectionDAG &DAG,
+                                                bool LegalOps, bool OptForSize,
+                                                NegatibleCost &Cost,
+                                                unsigned Depth) const {
+  if (Depth > SelectionDAG::MaxRecursionDepth)
+    return SDValue();
+
+  unsigned Opc = Op.getOpcode();
+  EVT VT = Op.getValueType();
+  SDNodeFlags Flags = Op.getNode()->getFlags();
+
+  switch (Opc) {
+  case PPCISD::FNMSUB:
+    // TODO: QPX subtarget is deprecated. No transformation here.
+    if (!Op.hasOneUse() || !isTypeLegal(VT) || Subtarget.hasQPX())
+      break;
+
+    const TargetOptions &Options = getTargetMachine().Options;
+    SDValue N0 = Op.getOperand(0);
+    SDValue N1 = Op.getOperand(1);
+    SDValue N2 = Op.getOperand(2);
+    SDLoc Loc(Op);
+
+    // (fneg (fnmsub a b c)) => (fnmsub (fneg a) b (fneg c))
+    // (fneg (fnmsub a b c)) => (fnmsub a (fneg b) (fneg c))
+    NegatibleCost N2Cost = NegatibleCost::Expensive;
+    SDValue NegN2 =
+        getNegatedExpression(N2, DAG, LegalOps, OptForSize, N2Cost, Depth + 1);
+
+    if (!NegN2)
+      return SDValue();
+
+    // Transformations may change sign of zeroes.
+    if (Flags.hasNoSignedZeros() || Options.NoSignedZerosFPMath) {
+      // Try and choose the cheaper one to negate.
+      NegatibleCost N0Cost = NegatibleCost::Expensive;
+      SDValue NegN0 = getNegatedExpression(N0, DAG, LegalOps, OptForSize,
+                                           N0Cost, Depth + 1);
+
+      NegatibleCost N1Cost = NegatibleCost::Expensive;
+      SDValue NegN1 = getNegatedExpression(N1, DAG, LegalOps, OptForSize,
+                                           N1Cost, Depth + 1);
+
+      if (NegN0 && N0Cost <= N1Cost) {
+        Cost = std::min(N0Cost, N2Cost);
+        return DAG.getNode(Opc, Loc, VT, NegN0, N1, NegN2, Flags);
+      } else if (NegN1) {
+        Cost = std::min(N1Cost, N2Cost);
+        return DAG.getNode(Opc, Loc, VT, N0, NegN1, NegN2, Flags);
+      }
+    }
+
+    // (fneg (fnmsub a b c)) => (fma a b (fneg c))
+    if (isOperationLegal(ISD::FMA, VT)) {
+      Cost = N2Cost;
+      return DAG.getNode(ISD::FMA, Loc, VT, N0, N1, NegN2, Flags);
+    }
+
+    break;
+  }
+
+  return TargetLowering::getNegatedExpression(Op, DAG, LegalOps, OptForSize,
+                                              Cost, Depth);
+}
+
 // Override to enable LOAD_STACK_GUARD lowering on Linux.
 bool PPCTargetLowering::useLoadStackGuardNode() const {
   if (!Subtarget.isTargetLinux())
     return TargetLowering::useLoadStackGuardNode();
   return true;
 }
 
 // Override to disable global variable loading on Linux.

 
   bool LHSHasPattern = isZextOfCompareWithConstant(LHS);
   bool RHSHasPattern = isZextOfCompareWithConstant(RHS);
 
   // If there is a pattern, canonicalize a zext operand to the RHS.
   if (LHSHasPattern && !RHSHasPattern)
     std::swap(LHS, RHS);
   else if (!LHSHasPattern && !RHSHasPattern)
     return SDValue();
 
   SDLoc DL(N);
   SDVTList VTs = DAG.getVTList(MVT::i64, MVT::Glue);
   SDValue Cmp = RHS.getOperand(0);
   SDValue Z = Cmp.getOperand(0);
   auto *Constant = dyn_cast<ConstantSDNode>(Cmp.getOperand(1));
 
   assert(Constant && "Constant Should not be a null pointer.");
   int64_t NegConstant = 0 - Constant->getSExtValue();
 
   switch(cast<CondCodeSDNode>(Cmp.getOperand(2))->get()) {
   default: break;
   case ISD::SETNE: {
     //                                 when C == 0
     //                             --> addze X, (addic Z, -1).carry
     //                            /
     // add X, (zext(setne Z, C))--
     //                            \    when -32768 <= -C <= 32767 && C != 0
     //                             --> addze X, (addic (addi Z, -C), -1).carry
     SDValue Add = DAG.getNode(ISD::ADD, DL, MVT::i64, Z,
                               DAG.getConstant(NegConstant, DL, MVT::i64));

 
     if (!IsProfitable(IsNeg, false, VT))
       return SDValue();
 
     SDValue Op0 = N->getOperand(0);
     SDValue Op1 =
         DAG.getNode(ISD::SHL, DL, VT, N->getOperand(0),
                     DAG.getConstant((MulAmtAbs + 1).logBase2(), DL, VT));
 
     if (!IsNeg)
       return DAG.getNode(ISD::SUB, DL, VT, Op1, Op0);
     else
       return DAG.getNode(ISD::SUB, DL, VT, Op0, Op1);
 
   } else {
     return SDValue();
   }
 }
 
+// Combine fma-like op (like fnmsub) with fnegs to appropriate op. Do this
+// in combiner since we need to check SD flags and other subtarget features.
+SDValue PPCTargetLowering::combineFMALike(SDNode *N,
+                                          DAGCombinerInfo &DCI) const {
+  SDValue N0 = N->getOperand(0);
+  SDValue N1 = N->getOperand(1);
+  SDValue N2 = N->getOperand(2);
+  SDNodeFlags Flags = N->getFlags();
+  EVT VT = N->getValueType(0);
+  SelectionDAG &DAG = DCI.DAG;
+  const TargetOptions &Options = getTargetMachine().Options;
+  unsigned Opc = N->getOpcode();
+  bool CodeSize = DAG.getMachineFunction().getFunction().hasOptSize();
+  bool LegalOps = !DCI.isBeforeLegalizeOps();
+  SDLoc Loc(N);
+
+  // TODO: QPX subtarget is deprecated. No transformation here.
+  if (Subtarget.hasQPX() || !isOperationLegal(ISD::FMA, VT) ||
+      (VT.isVector() && !Subtarget.hasVSX()))
+    return SDValue();
+
+  // Allowing transformation to FNMSUB may change sign of zeroes when ab-c=0
+  // since (fnmsub a b c)=-0 while c-ab=+0.
+  if (!Flags.hasNoSignedZeros() && !Options.NoSignedZerosFPMath)
+    return SDValue();
+
+  // (fma (fneg a) b c) => (fnmsub a b c)
+  // (fnmsub (fneg a) b c) => (fma a b c)
+  if (SDValue NegN0 = getCheaperNegatedExpression(N0, DAG, LegalOps, CodeSize))
+    return DAG.getNode(invertFMAOpcode(Opc), Loc, VT, NegN0, N1, N2, Flags);
+
+  // (fma a (fneg b) c) => (fnmsub a b c)
+  // (fnmsub a (fneg b) c) => (fma a b c)
+  if (SDValue NegN1 = getCheaperNegatedExpression(N1, DAG, LegalOps, CodeSize))
+    return DAG.getNode(invertFMAOpcode(Opc), Loc, VT, N0, NegN1, N2, Flags);
+
+  return SDValue();
+}
+
 bool PPCTargetLowering::mayBeEmittedAsTailCall(const CallInst *CI) const {
   // Only duplicate to increase tail-calls for the 64bit SysV ABIs.
   if (!Subtarget.is64BitELFABI())
     return false;
 
   // If not a tail call then no need to proceed.
   if (!CI->isTailCall())
     return false;

llvm/lib/Target/PowerPC/PPCInstrInfo.td

 def PPCcmpb     : SDNode<"PPCISD::CMPB", SDTIntBinOp, []>;
 
 // These nodes represent the 32-bit PPC shifts that operate on 6-bit shift
 // amounts.  These nodes are generated by the multi-precision shift code.
 def PPCsrl        : SDNode<"PPCISD::SRL"       , SDTIntShiftOp>;
 def PPCsra        : SDNode<"PPCISD::SRA"       , SDTIntShiftOp>;
 def PPCshl        : SDNode<"PPCISD::SHL"       , SDTIntShiftOp>;
 
+def PPCfnmsub     : SDNode<"PPCISD::FNMSUB"    , SDTFPTernaryOp>;
+
 def PPCextswsli : SDNode<"PPCISD::EXTSWSLI" , SDT_PPCextswsli>;
 
 // Move 2 i64 values into a VSX register
 def PPCbuild_fp128: SDNode<"PPCISD::BUILD_FP128",
                            SDTypeProfile<1, 2,
                              [SDTCisFP<0>, SDTCisSameSizeAs<1,2>,
                               SDTCisSameAs<1,2>]>,
                            []>;

 // The rule for seq_cst is duplicated to work with both 64 bits and 32 bits
 // versions of Power.
 def : Pat<(atomic_fence (i64 7), (timm)), (SYNC 0)>, Requires<[HasSYNC]>;
 def : Pat<(atomic_fence (i32 7), (timm)), (SYNC 0)>, Requires<[HasSYNC]>;
 def : Pat<(atomic_fence (timm), (timm)), (SYNC 1)>, Requires<[HasSYNC]>;
 def : Pat<(atomic_fence (timm), (timm)), (MSYNC)>, Requires<[HasOnlyMSYNC]>;
 
 let Predicates = [HasFPU] in {
-// Additional FNMSUB patterns: -a*c + b == -(a*c - b)
-def : Pat<(fma (fneg f64:$A), f64:$C, f64:$B),
-          (FNMSUB $A, $C, $B)>;
-def : Pat<(fma f64:$A, (fneg f64:$C), f64:$B),
-          (FNMSUB $A, $C, $B)>;
-def : Pat<(fma (fneg f32:$A), f32:$C, f32:$B),
-          (FNMSUBS $A, $C, $B)>;
-def : Pat<(fma f32:$A, (fneg f32:$C), f32:$B),
-          (FNMSUBS $A, $C, $B)>;
+// Additional fnmsub patterns for custom node
+def : Pat<(PPCfnmsub f64:$A, f64:$B, f64:$C),
+          (FNMSUB $A, $B, $C)>;
+def : Pat<(PPCfnmsub f32:$A, f32:$B, f32:$C),
+          (FNMSUBS $A, $B, $C)>;
+def : Pat<(fneg (PPCfnmsub f64:$A, f64:$B, f64:$C)),
+          (FMSUB $A, $B, $C)>;
+def : Pat<(fneg (PPCfnmsub f32:$A, f32:$B, f32:$C)),
+          (FMSUBS $A, $B, $C)>;
+def : Pat<(PPCfnmsub f64:$A, f64:$B, (fneg f64:$C)),
+          (FNMADD $A, $B, $C)>;
+def : Pat<(PPCfnmsub f32:$A, f32:$B, (fneg f32:$C)),
+          (FNMADDS $A, $B, $C)>;
 
 // FCOPYSIGN's operand types need not agree.
 def : Pat<(fcopysign f64:$frB, f32:$frA),
           (FCPSGND (COPY_TO_REGCLASS $frA, F8RC), $frB)>;
 def : Pat<(fcopysign f32:$frB, f64:$frA),
           (FCPSGNS (COPY_TO_REGCLASS $frA, F4RC), $frB)>;
 }
 

llvm/lib/Target/PowerPC/PPCInstrVSX.td

 // Valid for any VSX subtarget, regardless of endianness.
 let Predicates = [HasVSX] in {
 def : Pat<(v4i32 (vnot_ppc v4i32:$A)),
           (v4i32 (XXLNOR $A, $A))>;
 def : Pat<(v4i32 (or (and (vnot_ppc v4i32:$C), v4i32:$A),
                      (and v4i32:$B, v4i32:$C))),
           (v4i32 (XXSEL $A, $B, $C))>;
 
-// Additional fnmsub patterns: -a*b + c == -(a*b - c)
-def : Pat<(fma (fneg f64:$A), f64:$B, f64:$C),
-          (XSNMSUBADP $C, $A, $B)>;
-def : Pat<(fma f64:$A, (fneg f64:$B), f64:$C),
+// Additional fnmsub pattern for PPC specific ISD opcode
+def : Pat<(PPCfnmsub f64:$A, f64:$B, f64:$C),
           (XSNMSUBADP $C, $A, $B)>;
+def : Pat<(fneg (PPCfnmsub f64:$A, f64:$B, f64:$C)),
+          (XSMSUBADP $C, $A, $B)>;
+def : Pat<(PPCfnmsub f64:$A, f64:$B, (fneg f64:$C)),
+          (XSNMADDADP $C, $A, $B)>;
 
-def : Pat<(fma (fneg v2f64:$A), v2f64:$B, v2f64:$C),
-          (XVNMSUBADP $C, $A, $B)>;
-def : Pat<(fma v2f64:$A, (fneg v2f64:$B), v2f64:$C),
+def : Pat<(PPCfnmsub v2f64:$A, v2f64:$B, v2f64:$C),
           (XVNMSUBADP $C, $A, $B)>;
+def : Pat<(fneg (PPCfnmsub v2f64:$A, v2f64:$B, v2f64:$C)),
+          (XVMSUBADP $C, $A, $B)>;
+def : Pat<(PPCfnmsub v2f64:$A, v2f64:$B, (fneg v2f64:$C)),
+          (XVNMADDADP $C, $A, $B)>;
 
-def : Pat<(fma (fneg v4f32:$A), v4f32:$B, v4f32:$C),
-          (XVNMSUBASP $C, $A, $B)>;
-def : Pat<(fma v4f32:$A, (fneg v4f32:$B), v4f32:$C),
+def : Pat<(PPCfnmsub v4f32:$A, v4f32:$B, v4f32:$C),
           (XVNMSUBASP $C, $A, $B)>;
+def : Pat<(fneg (PPCfnmsub v4f32:$A, v4f32:$B, v4f32:$C)),
+          (XVMSUBASP $C, $A, $B)>;
+def : Pat<(PPCfnmsub v4f32:$A, v4f32:$B, (fneg v4f32:$C)),
+          (XVNMADDASP $C, $A, $B)>;
 
 def : Pat<(v2f64 (bitconvert v4f32:$A)),
           (COPY_TO_REGCLASS $A, VSRC)>;
 def : Pat<(v2f64 (bitconvert v4i32:$A)),
           (COPY_TO_REGCLASS $A, VSRC)>;
 def : Pat<(v2f64 (bitconvert v8i16:$A)),
           (COPY_TO_REGCLASS $A, VSRC)>;
 def : Pat<(v2f64 (bitconvert v16i8:$A)),

           (SELECT_VSSRC (CRORC  $lhs, $rhs), $tval, $fval)>;
 def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETGT)),
           (SELECT_VSSRC (CRANDC $rhs, $lhs), $tval, $fval)>;
 def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETUGT)),
           (SELECT_VSSRC (CRANDC $lhs, $rhs), $tval, $fval)>;
 def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETNE)),
           (SELECT_VSSRC (CRXOR $lhs, $rhs), $tval, $fval)>;
 
-// Additional xsnmsubasp patterns: -a*b + c == -(a*b - c)
-def : Pat<(fma (fneg f32:$A), f32:$B, f32:$C),
-          (XSNMSUBASP $C, $A, $B)>;
-def : Pat<(fma f32:$A, (fneg f32:$B), f32:$C),
+// Additional fnmsub pattern for PPC specific ISD opcode
+def : Pat<(PPCfnmsub f32:$A, f32:$B, f32:$C),
           (XSNMSUBASP $C, $A, $B)>;
+def : Pat<(fneg (PPCfnmsub f32:$A, f32:$B, f32:$C)),
+          (XSMSUBASP $C, $A, $B)>;
+def : Pat<(PPCfnmsub f32:$A, f32:$B, (fneg f32:$C)),
+          (XSNMADDASP $C, $A, $B)>;
 
 // f32 neg
 // Although XSNEGDP is available in P7, we want to select it starting from P8,
 // so that FNMSUBS can be selected for fneg-fmsub pattern on P7. (VSX version,
 // XSNMSUBASP, is available since P8)
 def : Pat<(f32 (fneg f32:$S)),
           (f32 (COPY_TO_REGCLASS (XSNEGDP
                (COPY_TO_REGCLASS $S, VSFRC)), VSSRC))>;

             (COPY_TO_REGCLASS
               (MTVSRD (RLDIMI AnyExts.A, AnyExts.B, 32, 0)), VSRC), 0)>;
 def : Pat<(v4i32 (build_vector i32:$A, i32:$A, i32:$A, i32:$A)),
           (XXSPLTW (COPY_TO_REGCLASS (MTVSRWZ $A), VSRC), 1)>;
 }
 
 // Any Power9 VSX subtarget.
 let Predicates = [HasVSX, HasP9Vector] in {
-// Additional fnmsub patterns: -a*b + c == -(a*b - c)
-def : Pat<(fma (fneg f128:$A), f128:$B, f128:$C), (XSNMSUBQP $C, $A, $B)>;
-def : Pat<(fma f128:$A, (fneg f128:$B), f128:$C), (XSNMSUBQP $C, $A, $B)>;
+// Additional fnmsub pattern for PPC specific ISD opcode
+def : Pat<(PPCfnmsub f128:$A, f128:$B, f128:$C),
+          (XSNMSUBQP $C, $A, $B)>;
+def : Pat<(fneg (PPCfnmsub f128:$A, f128:$B, f128:$C)),
+          (XSMSUBQP $C, $A, $B)>;
+def : Pat<(PPCfnmsub f128:$A, f128:$B, (fneg f128:$C)),
+          (XSNMADDQP $C, $A, $B)>;
 
 def : Pat<(f128 (sint_to_fp i64:$src)),
           (f128 (XSCVSDQP (COPY_TO_REGCLASS $src, VFRC)))>;
 def : Pat<(f128 (sint_to_fp (i64 (PPCmfvsr f64:$src)))),
           (f128 (XSCVSDQP $src))>;
 def : Pat<(f128 (sint_to_fp (i32 (PPCmfvsr f64:$src)))),
           (f128 (XSCVSDQP (VEXTSW2Ds $src)))>;
 def : Pat<(f128 (uint_to_fp i64:$src)),

llvm/test/CodeGen/PowerPC/combine-fneg.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=powerpc64le-unknown-unknown | FileCheck %s
 
 ; Infinite loop identified in D62963.
 define <4 x double> @fneg_fdiv_splat(double %a0, <4 x double> %a1) {
 ; CHECK-LABEL: fneg_fdiv_splat:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    addis 3, 2, .LCPI0_0@toc@ha
 ; CHECK-NEXT:    # kill: def $f1 killed $f1 def $vsl1
 ; CHECK-NEXT:    xxspltd 0, 1, 0
 ; CHECK-NEXT:    addi 3, 3, .LCPI0_0@toc@l
 ; CHECK-NEXT:    lxvd2x 1, 0, 3
 ; CHECK-NEXT:    xvredp 2, 0
 ; CHECK-NEXT:    xxswapd 1, 1
 ; CHECK-NEXT:    xxlor 3, 1, 1
-; CHECK-NEXT:    xvmaddadp 3, 0, 2
-; CHECK-NEXT:    xvnmsubadp 2, 2, 3
-; CHECK-NEXT:    xvmaddadp 1, 0, 2
-; CHECK-NEXT:    xvmsubadp 2, 2, 1
+; CHECK-NEXT:    xvnmsubadp 3, 0, 2
+; CHECK-NEXT:    xvmaddadp 2, 2, 3
+; CHECK-NEXT:    xvnmsubadp 1, 0, 2
+; CHECK-NEXT:    xvnmaddadp 2, 2, 1
 ; CHECK-NEXT:    xvmuldp 34, 34, 2
 ; CHECK-NEXT:    xvmuldp 35, 35, 2
 ; CHECK-NEXT:    blr
 entry:
   %splat.splatinsert = insertelement <4 x double> undef, double %a0, i32 0
   %splat.splat = shufflevector <4 x double> %splat.splatinsert, <4 x double> undef, <4 x i32> zeroinitializer
   %div = fdiv reassoc nsz arcp <4 x double> %a1, %splat.splat
   %sub = fsub reassoc nsz <4 x double> <double 0.000000e+00, double 0.000000e+00, double 0.000000e+00, double 0.000000e+00>, %div
   ret <4 x double> %sub
 }

llvm/test/CodeGen/PowerPC/f128-fma.ll

 define void @qpFmsub(fp128* nocapture readonly %a,
                       fp128* nocapture readonly %b,
                       fp128* nocapture readonly %c, fp128* nocapture %res) {
 entry:
   %0 = load fp128, fp128* %a, align 16
   %1 = load fp128, fp128* %b, align 16
   %2 = load fp128, fp128* %c, align 16
   %mul = fmul contract fp128 %1, %2
-  %sub = fsub contract fp128 %0, %mul
+  %sub = fsub contract nsz fp128 %0, %mul
   store fp128 %sub, fp128* %res, align 16
   ret void
 ; CHECK-LABEL: qpFmsub
 ; CHECK-NOT: bl __multf3
 ; CHECK-DAG: lxv v[[REG3:[0-9]+]], 0(r3)
 ; CHECK-DAG: lxv v[[REG4:[0-9]+]], 0(r4)
 ; CHECK-DAG: lxv v[[REG5:[0-9]+]], 0(r5)
 ; CHECK: xsnmsubqp v[[REG3]], v[[REG4]], v[[REG5]]

llvm/test/CodeGen/PowerPC/fdiv.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -verify-machineinstrs < %s -mtriple=powerpc64le-unknown-linux-gnu -mcpu=pwr8 | FileCheck %s
 
 define dso_local float @foo(float %0, float %1) local_unnamed_addr {
 ; CHECK-LABEL: foo:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    xsresp 3, 2
 ; CHECK-NEXT:    xsmulsp 0, 1, 3
 ; CHECK-NEXT:    xsnmsubasp 1, 2, 0
 ; CHECK-NEXT:    xsmaddasp 0, 3, 1
 ; CHECK-NEXT:    fmr 1, 0
 ; CHECK-NEXT:    blr
-  %3 = fdiv reassoc arcp float %0, %1
+  %3 = fdiv reassoc arcp nsz float %0, %1
   ret float %3
 }

llvm/test/CodeGen/PowerPC/fma-assoc.ll

; CHECK-VSX-NEXT:    blr
   %I = fpext float %H to double   ; <double> [#uses=1]
   %J = fsub double %I, %E         ; <double> [#uses=1]
   ret double %J
 }
 
 define double @test_FMSUB_ASSOC_EXT3(float %A, float %B, double %C,
 ; CHECK-LABEL: test_FMSUB_ASSOC_EXT3:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    fnmsub 0, 1, 2, 5
-; CHECK-NEXT:    fnmsub 1, 3, 4, 0
+; CHECK-NEXT:    fneg 0, 1
+; CHECK-NEXT:    fmadd 0, 0, 2, 5
+; CHECK-NEXT:    fneg 1, 3
+; CHECK-NEXT:    fmadd 1, 1, 4, 0
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-VSX-LABEL: test_FMSUB_ASSOC_EXT3:
 ; CHECK-VSX:       # %bb.0:
-; CHECK-VSX-NEXT:    xsnmsubmdp 1, 2, 5
-; CHECK-VSX-NEXT:    xsnmsubadp 1, 3, 4
+; CHECK-VSX-NEXT:    xsnegdp 1, 1
+; CHECK-VSX-NEXT:    xsnegdp 0, 3
+; CHECK-VSX-NEXT:    xsmaddmdp 1, 2, 5
+; CHECK-VSX-NEXT:    xsmaddadp 1, 0, 4
 ; CHECK-VSX-NEXT:    blr
                                  double %D, double %E) {
   %F = fmul float %A, %B          ; <float> [#uses=1]
   %G = fpext float %F to double   ; <double> [#uses=1]
   %H = fmul double %C, %D         ; <double> [#uses=1]
   %I = fadd double %H, %G         ; <double> [#uses=1]
   %J = fsub double %E, %I         ; <double> [#uses=1]
   ret double %J
 }
 
 define double @test_FMSUB_ASSOC_EXT4(float %A, float %B, float %C,
 ; CHECK-LABEL: test_FMSUB_ASSOC_EXT4:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    fnmsub 0, 3, 4, 5
-; CHECK-NEXT:    fnmsub 1, 1, 2, 0
+; CHECK-NEXT:    fneg 0, 3
+; CHECK-NEXT:    fmadd 0, 0, 4, 5
+; CHECK-NEXT:    fneg 1, 1
+; CHECK-NEXT:    fmadd 1, 1, 2, 0
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-VSX-LABEL: test_FMSUB_ASSOC_EXT4:
 ; CHECK-VSX:       # %bb.0:
-; CHECK-VSX-NEXT:    xsnmsubmdp 3, 4, 5
-; CHECK-VSX-NEXT:    xsnmsubadp 3, 1, 2
-; CHECK-VSX-NEXT:    fmr 1, 3
+; CHECK-VSX-NEXT:    xsnegdp 0, 3
+; CHECK-VSX-NEXT:    xsnegdp 1, 1
+; CHECK-VSX-NEXT:    xsmaddmdp 0, 4, 5
+; CHECK-VSX-NEXT:    xsmaddadp 0, 1, 2
+; CHECK-VSX-NEXT:    fmr 1, 0
 ; CHECK-VSX-NEXT:    blr
                                  float %D, double %E) {
   %F = fmul float %A, %B          ; <float> [#uses=1]
   %G = fmul float %C, %D          ; <float> [#uses=1]
   %H = fadd float %F, %G          ; <float> [#uses=1]
   %I = fpext float %H to double   ; <double> [#uses=1]
   %J = fsub double %E, %I         ; <double> [#uses=1]
   ret double %J
; CHECK-VSX-NEXT:    blr
   %I = fpext float %H to double   ; <double> [#uses=1]
   %J = fsub reassoc double %I, %E         ; <double> [#uses=1]
   ret double %J
 }
 
 define double @test_reassoc_FMSUB_ASSOC_EXT3(float %A, float %B, double %C,
 ; CHECK-LABEL: test_reassoc_FMSUB_ASSOC_EXT3:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    fnmsub 0, 1, 2, 5
-; CHECK-NEXT:    fnmsub 1, 3, 4, 0
+; CHECK-NEXT:    fneg 0, 1
+; CHECK-NEXT:    fmadd 0, 0, 2, 5
+; CHECK-NEXT:    fneg 1, 3
+; CHECK-NEXT:    fmadd 1, 1, 4, 0
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-VSX-LABEL: test_reassoc_FMSUB_ASSOC_EXT3:
 ; CHECK-VSX:       # %bb.0:
-; CHECK-VSX-NEXT:    xsnmsubmdp 1, 2, 5
-; CHECK-VSX-NEXT:    xsnmsubadp 1, 3, 4
+; CHECK-VSX-NEXT:    xsnegdp 1, 1
+; CHECK-VSX-NEXT:    xsnegdp 0, 3
+; CHECK-VSX-NEXT:    xsmaddmdp 1, 2, 5
+; CHECK-VSX-NEXT:    xsmaddadp 1, 0, 4
 ; CHECK-VSX-NEXT:    blr
                                  double %D, double %E) {
   %F = fmul reassoc float %A, %B          ; <float> [#uses=1]
   %G = fpext float %F to double   ; <double> [#uses=1]
   %H = fmul reassoc double %C, %D         ; <double> [#uses=1]
   %I = fadd reassoc double %H, %G         ; <double> [#uses=1]
   %J = fsub reassoc double %E, %I         ; <double> [#uses=1]
   ret double %J
 }
 
+; fnmsub/xsnmsubadp may affect the sign of zero, we need nsz flag
+; to ensure generating them
+define double @test_fast_FMSUB_ASSOC_EXT3(float %A, float %B, double %C,
+; CHECK-LABEL: test_fast_FMSUB_ASSOC_EXT3:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    fnmsub 0, 1, 2, 5
+; CHECK-NEXT:    fnmsub 1, 3, 4, 0
+; CHECK-NEXT:    blr
+;
+; CHECK-VSX-LABEL: test_fast_FMSUB_ASSOC_EXT3:
+; CHECK-VSX:       # %bb.0:
+; CHECK-VSX-NEXT:    xsnmsubmdp 1, 2, 5
+; CHECK-VSX-NEXT:    xsnmsubadp 1, 3, 4
+; CHECK-VSX-NEXT:    blr
+                                             double %D, double %E) {
+  %F = fmul reassoc float %A, %B
+  %G = fpext float %F to double
+  %H = fmul reassoc double %C, %D
+  %I = fadd reassoc nsz double %H, %G
+  %J = fsub reassoc nsz double %E, %I
+  ret double %J
+}
+
 define double @test_reassoc_FMSUB_ASSOC_EXT4(float %A, float %B, float %C,
 ; CHECK-LABEL: test_reassoc_FMSUB_ASSOC_EXT4:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    fnmsub 0, 3, 4, 5
-; CHECK-NEXT:    fnmsub 1, 1, 2, 0
+; CHECK-NEXT:    fneg 0, 3
+; CHECK-NEXT:    fmadd 0, 0, 4, 5
+; CHECK-NEXT:    fneg 1, 1
+; CHECK-NEXT:    fmadd 1, 1, 2, 0
 ; CHECK-NEXT:    blr
 ;
 ; CHECK-VSX-LABEL: test_reassoc_FMSUB_ASSOC_EXT4:
 ; CHECK-VSX:       # %bb.0:
-; CHECK-VSX-NEXT:    xsnmsubmdp 3, 4, 5
-; CHECK-VSX-NEXT:    xsnmsubadp 3, 1, 2
-; CHECK-VSX-NEXT:    fmr 1, 3
+; CHECK-VSX-NEXT:    xsnegdp 0, 3
+; CHECK-VSX-NEXT:    xsnegdp 1, 1
+; CHECK-VSX-NEXT:    xsmaddmdp 0, 4, 5
+; CHECK-VSX-NEXT:    xsmaddadp 0, 1, 2
+; CHECK-VSX-NEXT:    fmr 1, 0
 ; CHECK-VSX-NEXT:    blr
                                  float %D, double %E) {
   %F = fmul reassoc float %A, %B          ; <float> [#uses=1]
   %G = fmul reassoc float %C, %D          ; <float> [#uses=1]
   %H = fadd reassoc float %F, %G          ; <float> [#uses=1]
   %I = fpext float %H to double   ; <double> [#uses=1]
   %J = fsub reassoc double %E, %I         ; <double> [#uses=1]
   ret double %J
 }
+
+define double @test_fast_FMSUB_ASSOC_EXT4(float %A, float %B, float %C,
+; CHECK-LABEL: test_fast_FMSUB_ASSOC_EXT4:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    fnmsub 0, 3, 4, 5
+; CHECK-NEXT:    fnmsub 1, 1, 2, 0
+; CHECK-NEXT:    blr
+;
+; CHECK-VSX-LABEL: test_fast_FMSUB_ASSOC_EXT4:
+; CHECK-VSX:       # %bb.0:
+; CHECK-VSX-NEXT:    xsnmsubmdp 3, 4, 5
+; CHECK-VSX-NEXT:    xsnmsubadp 3, 1, 2
+; CHECK-VSX-NEXT:    fmr 1, 3
+; CHECK-VSX-NEXT:    blr
+                                          float %D, double %E) {
+  %F = fmul reassoc float %A, %B
+  %G = fmul reassoc float %C, %D
+  %H = fadd reassoc nsz float %F, %G
+  %I = fpext float %H to double
+  %J = fsub reassoc nsz double %E, %I
+  ret double %J
+}

llvm/test/CodeGen/PowerPC/fma-combine.ll

 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    addis 3, 2, .LCPI4_0@toc@ha
 ; CHECK-NEXT:    addis 4, 2, .LCPI4_1@toc@ha
 ; CHECK-NEXT:    lfs 0, .LCPI4_0@toc@l(3)
 ; CHECK-NEXT:    lfsx 2, 0, 3
 ; CHECK-NEXT:    addis 3, 2, .LCPI4_2@toc@ha
 ; CHECK-NEXT:    lfs 3, .LCPI4_1@toc@l(4)
 ; CHECK-NEXT:    lfs 1, .LCPI4_2@toc@l(3)
+; CHECK-NEXT:    fmr 4, 3
 ; CHECK-NEXT:    xsmaddasp 3, 2, 0
+; CHECK-NEXT:    xsnmaddasp 4, 2, 0
 ; CHECK-NEXT:    xsmaddasp 1, 2, 3
-; CHECK-NEXT:    xsnmsubasp 1, 3, 2
+; CHECK-NEXT:    xsmaddasp 1, 4, 2
 ; CHECK-NEXT:    blr
   %tmp = load float, float* undef, align 4
   %tmp2 = load float, float* undef, align 4
   %tmp3 = fmul reassoc float %tmp, 0x3FE372D780000000
   %tmp4 = fadd reassoc float %tmp3, 1.000000e+00
   %tmp5 = fmul reassoc float %tmp2, %tmp4
   %tmp6 = load float, float* undef, align 4
   %tmp7 = load float, float* undef, align 4

llvm/test/CodeGen/PowerPC/fma-ext.ll

 }
 
 define double @test_FMSUB_EXT2(float %A, float %B, double %C) {
     %D = fmul float %A, %B          ; <float> [#uses=1]
     %E = fpext float %D to double   ; <double> [#uses=1]
     %F = fsub double %C, %E         ; <double> [#uses=1]
     ret double %F
 ; CHECK-LABEL: test_FMSUB_EXT2:
-; CHECK: fnmsub
+; CHECK: fneg
+; CHECK-NEXT: fmadd
 ; CHECK-NEXT: blr
                                 
 ; CHECK-VSX-LABEL: test_FMSUB_EXT2:
+; CHECK-VSX: xsnegdp
+; CHECK-VSX-NEXT: xsmaddmdp
+; CHECK-VSX-NEXT: blr
+}
+
+; need nsz flag to generate fnmsub since it may affect sign of zero
+define double @test_FMSUB_EXT2_NSZ(float %A, float %B, double %C) {
+    %D = fmul nsz float %A, %B      ; <float> [#uses=1]
+    %E = fpext float %D to double   ; <double> [#uses=1]
+    %F = fsub nsz double %C, %E     ; <double> [#uses=1]
+    ret double %F
+; CHECK-LABEL: test_FMSUB_EXT2_NSZ:
+; CHECK: fnmsub
+; CHECK-NEXT: blr
+
+; CHECK-VSX-LABEL: test_FMSUB_EXT2_NSZ:
 ; CHECK-VSX: xsnmsubmdp
 ; CHECK-VSX-NEXT: blr
 }
 
 define double @test_FMSUB_EXT3(float %A, float %B, double %C) {
     %D = fmul float %A, %B          ; <float> [#uses=1]
     %E = fsub float -0.000000e+00, %D ;    <float> [#uses=1]
     %F = fpext float %E to double   ; <double> [#uses=1]

llvm/test/CodeGen/PowerPC/fma-negate.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple powerpc64le -verify-machineinstrs \
 ; RUN:   | FileCheck -check-prefix=VSX %s
 ; RUN: llc < %s -mtriple powerpc64le -verify-machineinstrs -mattr=-vsx \
 ; RUN:   | FileCheck -check-prefix=NO-VSX %s
 
 define double @test_mul_sub_f64(double %a, double %b, double %c) {
 ; VSX-LABEL: test_mul_sub_f64:
 ; VSX:       # %bb.0: # %entry
-; VSX-NEXT:    xsnmsubadp 1, 2, 3
+; VSX-NEXT:    xsnegdp 0, 2
+; VSX-NEXT:    xsmaddadp 1, 0, 3
 ; VSX-NEXT:    blr
 ;
 ; NO-VSX-LABEL: test_mul_sub_f64:
 ; NO-VSX:       # %bb.0: # %entry
-; NO-VSX-NEXT:    fnmsub 1, 2, 3, 1
+; NO-VSX-NEXT:    fneg 0, 2
+; NO-VSX-NEXT:    fmadd 1, 0, 3, 1
 ; NO-VSX-NEXT:    blr
 entry:
   %0 = fmul contract reassoc double %b, %c
   %1 = fsub contract reassoc double %a, %0
   ret double %1
 }
 
 define double @test_2mul_sub_f64(double %a, double %b, double %c, double %d) {
entry:
   %1 = fmul contract reassoc double %c, %d
   %2 = fsub contract reassoc double %0, %1
   ret double %2
 }
 
 define double @test_neg_fma_f64(double %a, double %b, double %c) {
 ; VSX-LABEL: test_neg_fma_f64:
 ; VSX:       # %bb.0: # %entry
-; VSX-NEXT:    xsnmsubadp 3, 1, 2
+; VSX-NEXT:    xsnegdp 0, 1
+; VSX-NEXT:    xsmaddadp 3, 0, 2
 ; VSX-NEXT:    fmr 1, 3
 ; VSX-NEXT:    blr
 ;
 ; NO-VSX-LABEL: test_neg_fma_f64:
 ; NO-VSX:       # %bb.0: # %entry
-; NO-VSX-NEXT:    fnmsub 1, 1, 2, 3
+; NO-VSX-NEXT:    fneg 0, 1
+; NO-VSX-NEXT:    fmadd 1, 0, 2, 3
 ; NO-VSX-NEXT:    blr
 entry:
   %0 = fsub contract reassoc double -0.0, %a
   %1 = call contract reassoc double @llvm.fma.f64(double %0, double %b,
                                                   double %c)
   ret double %1
 }
 
 define float @test_mul_sub_f32(float %a, float %b, float %c) {
 ; VSX-LABEL: test_mul_sub_f32:
 ; VSX:       # %bb.0: # %entry
-; VSX-NEXT:    xsnmsubasp 1, 2, 3
+; VSX-NEXT:    xsnegdp 0, 2
+; VSX-NEXT:    xsmaddasp 1, 0, 3
 ; VSX-NEXT:    blr
 ;
 ; NO-VSX-LABEL: test_mul_sub_f32:
 ; NO-VSX:       # %bb.0: # %entry
-; NO-VSX-NEXT:    fnmsubs 1, 2, 3, 1
+; NO-VSX-NEXT:    fneg 0, 2
+; NO-VSX-NEXT:    fmadds 1, 0, 3, 1
 ; NO-VSX-NEXT:    blr
 entry:
   %0 = fmul contract reassoc float %b, %c
   %1 = fsub contract reassoc float %a, %0
   ret float %1
 }
 
 define float @test_2mul_sub_f32(float %a, float %b, float %c, float %d) {
entry:
   %1 = fmul contract reassoc float %c, %d
   %2 = fsub contract reassoc float %0, %1
   ret float %2
 }
 
 define float @test_neg_fma_f32(float %a, float %b, float %c) {
 ; VSX-LABEL: test_neg_fma_f32:
 ; VSX:       # %bb.0: # %entry
-; VSX-NEXT:    xsnmsubasp 3, 1, 2
+; VSX-NEXT:    xsnegdp 0, 1
+; VSX-NEXT:    xsmaddasp 3, 0, 2
 ; VSX-NEXT:    fmr 1, 3
 ; VSX-NEXT:    blr
 ;
 ; NO-VSX-LABEL: test_neg_fma_f32:
 ; NO-VSX:       # %bb.0: # %entry
-; NO-VSX-NEXT:    fnmsubs 1, 1, 2, 3
+; NO-VSX-NEXT:    fneg 0, 1
+; NO-VSX-NEXT:    fmadds 1, 0, 2, 3
 ; NO-VSX-NEXT:    blr
 entry:
   %0 = fsub contract reassoc float -0.0, %a
   %1 = call contract reassoc float @llvm.fma.f32(float %0, float %b, float %c)
   ret float %1
 }
 
 define <2 x double> @test_neg_fma_v2f64(<2 x double> %a, <2 x double> %b,
 ; VSX-LABEL: test_neg_fma_v2f64:
 ; VSX:       # %bb.0: # %entry
-; VSX-NEXT:    xvnmsubadp 36, 34, 35
+; VSX-NEXT:    xvnegdp 0, 34
+; VSX-NEXT:    xvmaddadp 36, 0, 35
 ; VSX-NEXT:    vmr 2, 4
 ; VSX-NEXT:    blr
 ;
 ; NO-VSX-LABEL: test_neg_fma_v2f64:
 ; NO-VSX:       # %bb.0: # %entry
-; NO-VSX-NEXT:    fnmsub 1, 1, 3, 5
-; NO-VSX-NEXT:    fnmsub 2, 2, 4, 6
+; NO-VSX-NEXT:    fneg 0, 2
+; NO-VSX-NEXT:    fneg 1, 1
+; NO-VSX-NEXT:    fmadd 1, 1, 3, 5
+; NO-VSX-NEXT:    fmadd 2, 0, 4, 6
 ; NO-VSX-NEXT:    blr
                                         <2 x double> %c) {
 entry:
   %0 = fsub contract reassoc <2 x double> <double -0.0, double -0.0>, %a
   %1 = call contract reassoc <2 x double> @llvm.fma.v2f64(<2 x double> %0,
                                                           <2 x double> %b,
                                                           <2 x double> %c)
   ret <2 x double> %1
 }
 
 define <4 x float> @test_neg_fma_v4f32(<4 x float> %a, <4 x float> %b,
 ; VSX-LABEL: test_neg_fma_v4f32:
 ; VSX:       # %bb.0: # %entry
-; VSX-NEXT:    xvnmsubasp 36, 34, 35
+; VSX-NEXT:    xvnegsp 0, 34
+; VSX-NEXT:    xvmaddasp 36, 0, 35
 ; VSX-NEXT:    vmr 2, 4
 ; VSX-NEXT:    blr
 ;
 ; NO-VSX-LABEL: test_neg_fma_v4f32:
 ; NO-VSX:       # %bb.0: # %entry
 ; NO-VSX-NEXT:    vspltisb 5, -1
 ; NO-VSX-NEXT:    vslw 5, 5, 5
 ; NO-VSX-NEXT:    vsubfp 2, 5, 2

 ; VSX-NEXT:    xsnmsubadp 1, 2, 3
 ; VSX-NEXT:    blr
 ;
 ; NO-VSX-LABEL: test_fast_mul_sub_f64:
 ; NO-VSX:       # %bb.0: # %entry
 ; NO-VSX-NEXT:    fnmsub 1, 2, 3, 1
 ; NO-VSX-NEXT:    blr
 entry:
-  %0 = fmul reassoc double %b, %c
-  %1 = fsub reassoc double %a, %0
+  %0 = fmul reassoc nsz double %b, %c
+  %1 = fsub reassoc nsz double %a, %0
   ret double %1
 }
 
 define double @test_fast_2mul_sub_f64(double %a, double %b, double %c,
 ; VSX-LABEL: test_fast_2mul_sub_f64:
 ; VSX:       # %bb.0: # %entry
 ; VSX-NEXT:    xsmuldp 0, 3, 4
 ; VSX-NEXT:    xsmsubadp 0, 1, 2

 ; VSX-NEXT:    blr
 ;
 ; NO-VSX-LABEL: test_fast_neg_fma_f64:
 ; NO-VSX:       # %bb.0: # %entry
 ; NO-VSX-NEXT:    fnmsub 1, 1, 2, 3
 ; NO-VSX-NEXT:    blr
 entry:
   %0 = fsub reassoc double -0.0, %a
-  %1 = call reassoc double @llvm.fma.f64(double %0, double %b, double %c)
+  %1 = call reassoc nsz double @llvm.fma.f64(double %0, double %b, double %c)
   ret double %1
 }
 
 define float @test_fast_mul_sub_f32(float %a, float %b, float %c) {
 ; VSX-LABEL: test_fast_mul_sub_f32:
 ; VSX:       # %bb.0: # %entry
 ; VSX-NEXT:    xsnmsubasp 1, 2, 3
 ; VSX-NEXT:    blr
 ;
 ; NO-VSX-LABEL: test_fast_mul_sub_f32:
 ; NO-VSX:       # %bb.0: # %entry
 ; NO-VSX-NEXT:    fnmsubs 1, 2, 3, 1
 ; NO-VSX-NEXT:    blr
 entry:
   %0 = fmul reassoc float %b, %c
-  %1 = fsub reassoc float %a, %0
+  %1 = fsub reassoc nsz float %a, %0
   ret float %1
 }
 
 define float @test_fast_2mul_sub_f32(float %a, float %b, float %c, float %d) {
 ; VSX-LABEL: test_fast_2mul_sub_f32:
 ; VSX:       # %bb.0: # %entry
 ; VSX-NEXT:    xsmulsp 0, 3, 4
 ; VSX-NEXT:    xsmsubasp 0, 1, 2
 ; VSX-NEXT:    fmr 1, 0
 ; VSX-NEXT:    blr
 ;
 ; NO-VSX-LABEL: test_fast_2mul_sub_f32:
 ; NO-VSX:       # %bb.0: # %entry
 ; NO-VSX-NEXT:    fmuls 0, 3, 4
 ; NO-VSX-NEXT:    fmsubs 1, 1, 2, 0
 ; NO-VSX-NEXT:    blr
 entry:
   %0 = fmul reassoc float %a, %b
   %1 = fmul reassoc float %c, %d
-  %2 = fsub reassoc float %0, %1
+  %2 = fsub reassoc nsz float %0, %1
   ret float %2
 }
 
 define float @test_fast_neg_fma_f32(float %a, float %b, float %c) {
 ; VSX-LABEL: test_fast_neg_fma_f32:
 ; VSX:       # %bb.0: # %entry
 ; VSX-NEXT:    xsnmsubasp 3, 1, 2
 ; VSX-NEXT:    fmr 1, 3
 ; VSX-NEXT:    blr
 ;
 ; NO-VSX-LABEL: test_fast_neg_fma_f32:
 ; NO-VSX:       # %bb.0: # %entry
 ; NO-VSX-NEXT:    fnmsubs 1, 1, 2, 3
 ; NO-VSX-NEXT:    blr
 entry:
   %0 = fsub reassoc float -0.0, %a
-  %1 = call reassoc float @llvm.fma.f32(float %0, float %b, float %c)
+  %1 = call reassoc nsz float @llvm.fma.f32(float %0, float %b, float %c)
   ret float %1
 }
 
 define <2 x double> @test_fast_neg_fma_v2f64(<2 x double> %a, <2 x double> %b,
 ; VSX-LABEL: test_fast_neg_fma_v2f64:
 ; VSX:       # %bb.0: # %entry
 ; VSX-NEXT:    xvnmsubadp 36, 34, 35
 ; VSX-NEXT:    vmr 2, 4
 ; VSX-NEXT:    blr
 ;
 ; NO-VSX-LABEL: test_fast_neg_fma_v2f64:
 ; NO-VSX:       # %bb.0: # %entry
 ; NO-VSX-NEXT:    fnmsub 1, 1, 3, 5
 ; NO-VSX-NEXT:    fnmsub 2, 2, 4, 6
 ; NO-VSX-NEXT:    blr
                                              <2 x double> %c) {
 entry:
   %0 = fsub reassoc <2 x double> <double -0.0, double -0.0>, %a
-  %1 = call reassoc <2 x double> @llvm.fma.v2f64(<2 x double> %0, <2 x double> %b,
+  %1 = call reassoc nsz <2 x double> @llvm.fma.v2f64(<2 x double> %0, <2 x double> %b,
                                               <2 x double> %c)
   ret <2 x double> %1
 }
 
 define <4 x float> @test_fast_neg_fma_v4f32(<4 x float> %a, <4 x float> %b,
 ; VSX-LABEL: test_fast_neg_fma_v4f32:
 ; VSX:       # %bb.0: # %entry
 ; VSX-NEXT:    xvnmsubasp 36, 34, 35
 ; VSX-NEXT:    vmr 2, 4
 ; VSX-NEXT:    blr
 ;
 ; NO-VSX-LABEL: test_fast_neg_fma_v4f32:
 ; NO-VSX:       # %bb.0: # %entry
 ; NO-VSX-NEXT:    vspltisb 5, -1
 ; NO-VSX-NEXT:    vslw 5, 5, 5
 ; NO-VSX-NEXT:    vsubfp 2, 5, 2
 ; NO-VSX-NEXT:    vmaddfp 2, 2, 3, 4
 ; NO-VSX-NEXT:    blr
                                             <4 x float> %c) {
 entry:
   %0 = fsub reassoc <4 x float> <float -0.0, float -0.0, float -0.0,
                                           float -0.0>, %a
-  %1 = call reassoc <4 x float> @llvm.fma.v4f32(<4 x float> %0, <4 x float> %b,
-                                                         <4 x float> %c)
+  %1 = call reassoc nsz <4 x float> @llvm.fma.v4f32(<4 x float> %0, <4 x float> %b,
   ret <4 x float> %1
 }
 
 declare float  @llvm.fma.f32(float  %a, float  %b, float  %c)
 declare double @llvm.fma.f64(double %a, double %b, double %c)
 declare <4 x float> @llvm.fma.v4f32(<4 x float> %a, <4 x float> %b,
                                                     <4 x float> %c)
 declare <2 x double> @llvm.fma.v2f64(<2 x double> %a, <2 x double> %b,
                                                       <2 x double> %c)

llvm/test/CodeGen/PowerPC/fma-precision.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -verify-machineinstrs -mcpu=pwr9 -mtriple=powerpc64le-linux-gnu | FileCheck %s
 
 ; Verify that the fold of a*b-c*d respect the uses of a*b
 define double @fsub1(double %a, double %b, double %c, double %d)  {
 ; CHECK-LABEL: fsub1:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    xsmuldp 0, 2, 1
 ; CHECK-NEXT:    fmr 1, 0
 ; CHECK-NEXT:    xsnmsubadp 1, 4, 3
 ; CHECK-NEXT:    xsmuldp 1, 0, 1
 ; CHECK-NEXT:    blr
 entry:
   %mul = fmul reassoc double %b, %a
   %mul1 = fmul reassoc double %d, %c
-  %sub = fsub reassoc double %mul, %mul1
+  %sub = fsub reassoc nsz double %mul, %mul1
   %mul3 = fmul reassoc double %mul, %sub
   ret double %mul3
 }
 
 ; Verify that the fold of a*b-c*d respect the uses of c*d
 define double @fsub2(double %a, double %b, double %c, double %d)  {
 ; CHECK-LABEL: fsub2:
 ; CHECK:       # %bb.0: # %entry

 ; CHECK-NEXT:    xsnmsubadp 1, 3, 4
 ; CHECK-NEXT:    stfd 0, 0(9)
 ; CHECK-NEXT:    blr
   %ab = fmul reassoc double %a, %b
   %cd = fmul reassoc double %c, %d
   store double %ab, double* %p1 ; extra use of %ab
   store double %ab, double* %p2 ; another extra use of %ab
   store double %cd, double* %p3 ; extra use of %cd
-  %r = fsub reassoc double %ab, %cd
+  %r = fsub reassoc nsz double %ab, %cd
   ret double %r
 }
 
 define double @fma_multi_uses2(double %a, double %b, double %c, double %d, double* %p1, double* %p2, double* %p3) {
 ; CHECK-LABEL: fma_multi_uses2:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    xsmuldp 5, 1, 2
 ; CHECK-NEXT:    xsmuldp 0, 3, 4

 ; CHECK-NEXT:    xsadddp 1, 0, 1
 ; CHECK-NEXT:    blr
   %ab = fmul reassoc double %a, %b
   %cd = fmul reassoc double %c, %d
   %fg = fmul reassoc double %f, %g
   store double %ab, double* %p1 ; extra use of %ab
   store double %ab, double* %p2 ; another extra use of %ab
   store double %fg, double* %p3 ; extra use of %fg
-  %q = fsub reassoc double %fg, %cd ; The uses of %cd reduce to 1 after %r is folded. 2 uses of %fg, fold %cd, remove def of %cd
-  %r = fsub reassoc double %ab, %cd ; Fold %r before %q. 3 uses of %ab, 2 uses of %cd, fold %cd
+  %q = fsub reassoc nsz double %fg, %cd ; The uses of %cd reduce to 1 after %r is folded. 2 uses of %fg, fold %cd, remove def of %cd
+  %r = fsub reassoc nsz double %ab, %cd ; Fold %r before %q. 3 uses of %ab, 2 uses of %cd, fold %cd
   %add = fadd reassoc double %r, %q
   ret double %add
 }

llvm/test/CodeGen/PowerPC/fma.ll

 ; CHECK-VSX-NEXT: blr
 }
 
 define double @test_FNMSUB1(double %A, double %B, double %C) {
   %D = fmul double %A, %B   ; <double> [#uses=1]
   %E = fsub double %C, %D   ; <double> [#uses=1]
   ret double %E
 ; CHECK-LABEL: test_FNMSUB1:
-; CHECK: fnmsub
+; CHECK: fneg
+; CHECK-NEXT: fmadd
 ; CHECK-NEXT: blr
 
 ; CHECK-VSX-LABEL: test_FNMSUB1:
+; CHECK-VSX: xsnegdp
+; CHECK-VSX-NEXT: xsmaddmdp
+}
+
+; need nsz flag to generate fnmsub since it may affect sign of zero
+define double @test_FNMSUB1_NSZ(double %A, double %B, double %C) {
+  %D = fmul nsz double %A, %B    ; <double> [#uses=1]
+  %E = fsub nsz double %C, %D    ; <double> [#uses=1]
+  ret double %E
+; CHECK-LABEL: test_FNMSUB1_NSZ:
+; CHECK: fnmsub
+; CHECK-NEXT: blr
+
+; CHECK-VSX-LABEL: test_FNMSUB1_NSZ:
 ; CHECK-VSX: xsnmsubmdp
 }
 
 define double @test_FNMSUB2(double %A, double %B, double %C) {
   %D = fmul double %A, %B   ; <double> [#uses=1]
   %E = fsub double %D, %C   ; <double> [#uses=1]
   %F = fsub double -0.000000e+00, %E    ; <double> [#uses=1]
   ret double %F

llvm/test/CodeGen/PowerPC/recipest.ll

 ; CHECK-P9-NEXT:    xsmulsp 0, 0, 1
 ; CHECK-P9-NEXT:    xsresp 1, 2
 ; CHECK-P9-NEXT:    xsmulsp 0, 0, 4
 ; CHECK-P9-NEXT:    xsmulsp 4, 0, 1
 ; CHECK-P9-NEXT:    xsnmsubasp 0, 2, 4
 ; CHECK-P9-NEXT:    xsmaddasp 4, 1, 0
 ; CHECK-P9-NEXT:    xsmulsp 1, 3, 4
 ; CHECK-P9-NEXT:    blr
-  %x = call reassoc arcp float @llvm.sqrt.f32(float %a)
-  %y = fmul reassoc float %x, %b
-  %z = fdiv reassoc arcp float %c, %y
+  %x = call reassoc arcp nsz float @llvm.sqrt.f32(float %a)
+  %y = fmul reassoc nsz float %x, %b
+  %z = fdiv reassoc arcp nsz float %c, %y
   ret float %z
 }
 
 define float @rsqrt_fmul_safe(float %a, float %b, float %c) {
 ; CHECK-P7-LABEL: rsqrt_fmul_safe:
 ; CHECK-P7:       # %bb.0:
 ; CHECK-P7-NEXT:    fsqrts 0, 1
 ; CHECK-P7-NEXT:    fmuls 0, 0, 2

 ; CHECK-P9-LABEL: goo2_fmf:
 ; CHECK-P9:       # %bb.0:
 ; CHECK-P9-NEXT:    xsresp 3, 2
 ; CHECK-P9-NEXT:    xsmulsp 0, 1, 3
 ; CHECK-P9-NEXT:    xsnmsubasp 1, 2, 0
 ; CHECK-P9-NEXT:    xsmaddasp 0, 3, 1
 ; CHECK-P9-NEXT:    fmr 1, 0
 ; CHECK-P9-NEXT:    blr
-  %r = fdiv reassoc arcp float %a, %b
+  %r = fdiv reassoc arcp nsz float %a, %b
   ret float %r
 }
 
 define float @goo2_safe(float %a, float %b) nounwind {
 ; CHECK-P7-LABEL: goo2_safe:
 ; CHECK-P7:       # %bb.0:
 ; CHECK-P7-NEXT:    fdivs 1, 1, 2
 ; CHECK-P7-NEXT:    blr

 ; CHECK-P9-LABEL: hoo2_fmf:
 ; CHECK-P9:       # %bb.0:
 ; CHECK-P9-NEXT:    xvresp 1, 35
 ; CHECK-P9-NEXT:    xvmulsp 0, 34, 1
 ; CHECK-P9-NEXT:    xvnmsubasp 34, 35, 0
 ; CHECK-P9-NEXT:    xvmaddasp 0, 1, 34
 ; CHECK-P9-NEXT:    xxlor 34, 0, 0
 ; CHECK-P9-NEXT:    blr
-  %r = fdiv reassoc arcp <4 x float> %a, %b
+  %r = fdiv reassoc arcp nsz <4 x float> %a, %b
   ret <4 x float> %r
 }
 
 define <4 x float> @hoo2_safe(<4 x float> %a, <4 x float> %b) nounwind {
 ; CHECK-P7-LABEL: hoo2_safe:
 ; CHECK-P7:       # %bb.0:
 ; CHECK-P7-NEXT:    addi 3, 1, -32
 ; CHECK-P7-NEXT:    addi 4, 1, -48

llvm/test/CodeGen/PowerPC/repeated-fp-divisors.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-- < %s | FileCheck %s
 
 define <4 x float> @repeated_fp_divisor(float %a, <4 x float> %b) {
 ; CHECK-LABEL: repeated_fp_divisor:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    xscvdpspn 0, 1
 ; CHECK-NEXT:    addis 3, 2, .LCPI0_0@toc@ha
 ; CHECK-NEXT:    addi 3, 3, .LCPI0_0@toc@l
 ; CHECK-NEXT:    lvx 3, 0, 3
 ; CHECK-NEXT:    addis 3, 2, .LCPI0_1@toc@ha
 ; CHECK-NEXT:    addi 3, 3, .LCPI0_1@toc@l
 ; CHECK-NEXT:    lvx 4, 0, 3
 ; CHECK-NEXT:    xxspltw 0, 0, 0
 ; CHECK-NEXT:    xvresp 1, 0
-; CHECK-NEXT:    xvmaddasp 35, 0, 1
+; CHECK-NEXT:    xvnmsubasp 35, 0, 1
 ; CHECK-NEXT:    xvmulsp 0, 34, 36
-; CHECK-NEXT:    xvnmsubasp 1, 1, 35
+; CHECK-NEXT:    xvmaddasp 1, 1, 35
 ; CHECK-NEXT:    xvmulsp 34, 0, 1
 ; CHECK-NEXT:    blr
   %ins = insertelement <4 x float> undef, float %a, i32 0
   %splat = shufflevector <4 x float> %ins, <4 x float> undef, <4 x i32> zeroinitializer
   %t1 = fmul reassoc <4 x float> %b, <float 1.000000e+00, float 1.000000e+00, float 1.000000e+00, float 0x3FF028F5C0000000>
   %mul = fdiv reassoc arcp nsz <4 x float> %t1, %splat
   ret <4 x float> %mul
 }