[PowerPC] Try to fold sqrt/sdiv test results with the branch.

[Esme]

Summary

The patch tries to fold sqrt/sdiv test node, i.g FTSQRT, XVTDIVDP, and the branch, i.e br_cc if they satisfy these patterns:

(br_cc seteq, (truncateToi1 SWTestOp), 0) -> (BCC PRED_NU, SWTestOp)
(br_cc seteq, (and Node_test, 2), 0) -> (BCC PRED_NE, PPCNode_test)
(br_cc seteq, (and Node_test, 4), 0) -> (BCC PRED_LE, PPCNode_test)
(br_cc seteq, (and Node_test, 8), 0) -> (BCC PRED_GE, PPCNode_test)
(br_cc setne, (truncateToi1 SWTestOp), 0) -> (BCC PRED_UN, SWTestOp) 
(br_cc setne, (and Node_test, 2), 0) -> (BCC PRED_EQ, PPCNode_test)
(br_cc setne, (and Node_test, 4), 0) -> (BCC PRED_GT, PPCNode_test)
(br_cc setne, (and Node_test, 8), 0) -> (BCC PRED_LT, PPCNode_test)

[steven.zhang]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

4384

Do you have any problems to implement this folding with .td ? It doesn't make sense to me to get the opcode and then do the folding, as that is done during instr match(.td).

[Esme]

Leave the ISel of intrinsics to td.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

4384

These pattens have to be matched before the selection of the single br_cc, which is implemented in PPCISelDAGToDAG.cpp, therefore these folding can't be implemented with .td. You're right, that it'll be better to leave the selection of intrinsics to td.

[nemanjai]

It seems like this patch depends on another patch. Please don't forget to add the dependencies to the Phabricator reviews.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

4384

If the only possible use for these test nodes are to and the result with 1/2/4, you might as well check the whole thing in this function.
Otherwise, I think it's fine as-is.

4390

Seems odd that a function called isTestSqrtOpcode() returns true for a test for SW divide.

4407

Please add an assert that N is the right type of node (i.e. BR_CC).

4409

It is quite hard to make sense of such huge condition and it aids readability to break it up and name the operands (i.e. CC, CmpLHS, CmpRHS, TgtBlock).

Then you can check the following in order and make it obvious:

• the RHS is zero
• the CC is EQ/NE
• the LHS is (and (testSqrt), imm)

[Esme]

Addressed nemanjai's comments. :)

[Esme]

In D94054#2479070, @nemanjai wrote:

It seems like this patch depends on another patch. Please don't forget to add the dependencies to the Phabricator reviews.

An NFC patch, this patch depends on, only committed test cases.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

4384

If the only possible use for these test nodes are to and the result with 1/2/4, you might as well check the whole thing in this function.
Otherwise, I think it's fine as-is.

Another pattern (truncateToi1 SWTestOp) is added now, well, I can't find more possible patterns.

[steven.zhang]

LGTM. Please hold on for days to see if @nemanjai has more comments.

[nemanjai]

Thanks for addressing my comments and I'm sorry I didn't get a chance to re-review prior to commit. Looks good now.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

StringRef getPassName() const override {
       return "PowerPC DAG->DAG Pattern Instruction Selection";
     }
 
 // Include the pieces autogenerated from the target description.
 #include "PPCGenDAGISel.inc"
 
 private:
     bool trySETCC(SDNode *N);
+    bool tryFoldSqrtTestBr(SDNode *N);
     bool tryAsSingleRLDICL(SDNode *N);
     bool tryAsSingleRLDICR(SDNode *N);
     bool tryAsSingleRLWINM(SDNode *N);
     bool tryAsSingleRLWINM8(SDNode *N);
     bool tryAsSingleRLWIMI(SDNode *N);
     bool tryAsPairOfRLDICL(SDNode *N);
     bool tryAsSingleRLDIMI(SDNode *N);
 
static bool mayUseP9Setb(SDNode *N, const ISD::CondCode &CC, SelectionDAG *DAG,
   }
 
   LLVM_DEBUG(dbgs() << "Found a node that can be lowered to a SETB: ");
   LLVM_DEBUG(N->dump());
 
   return true;
 }
 
+static unsigned getTestSqrtOpcode(SDValue N) {
+  if (N.getOpcode() == PPCISD::FTSQRT) {
+    if (N.getOperand(0).getValueType() == MVT::i32)
+      return PPC::FTSQRT;
+    if (N.getOperand(0).getValueType() == MVT::f64)
+      return PPC::XSTSQRTDP;
+  }
+  if (!isa<ConstantSDNode>(N.getOperand(0)))
+    return 0;
+  switch (N.getConstantOperandVal(0)) {
+  case Intrinsic::ppc_vsx_xvtdivdp:
+    assert(N.getOperand(1).getValueType() == MVT::v2f64 &&
+           "Invalid input type for XVTDIVDP.");
+    return PPC::XVTDIVDP;
+  case Intrinsic::ppc_vsx_xvtdivsp:
+    assert(N.getOperand(1).getValueType() == MVT::v4f32 &&
+           "Invalid input type for XVTDIVSP.");
+    return PPC::XVTDIVSP;
+  case Intrinsic::ppc_vsx_xvtsqrtdp:
+    assert(N.getOperand(1).getValueType() == MVT::v2f64 &&
+           "Invalid input type for XVTSQRTDP.");
+    return PPC::XVTSQRTDP;
+  case Intrinsic::ppc_vsx_xvtsqrtsp:
+    assert(N.getOperand(1).getValueType() == MVT::v4f32 &&
+           "Invalid input type for XVTSQRTSP.");
+    return PPC::XVTSQRTSP;
+  }
+  return 0;
+}
+
+bool PPCDAGToDAGISel::tryFoldSqrtTestBr(SDNode *N) {
+  // We are looking for following patterns.
+  // (br_cc seteq, (and Node_test, 2), 0) -> (BCC PRED_NE, PPCNode_test)
+  // (br_cc seteq, (and Node_test, 4), 0) -> (BCC PRED_LE, PPCNode_test)
+  // (br_cc seteq, (and Node_test, 8), 0) -> (BCC PRED_GE, PPCNode_test)
+  // (br_cc setne, (and Node_test, 2), 0) -> (BCC PRED_EQ, PPCNode_test)
+  // (br_cc setne, (and Node_test, 4), 0) -> (BCC PRED_GT, PPCNode_test)
+  // (br_cc setne, (and Node_test, 8), 0) -> (BCC PRED_LT, PPCNode_test)
+  SDValue CmpLHS = N->getOperand(2);
+  ISD::CondCode CC = cast<CondCodeSDNode>(N->getOperand(1))->get();
+  if (!isa<ConstantSDNode>(N->getOperand(3)) ||
+      N->getConstantOperandVal(3) != 0 ||
+      (CC != ISD::SETEQ && CC != ISD::SETNE) ||
+      CmpLHS.getOpcode() != ISD::AND ||
+      !isa<ConstantSDNode>(CmpLHS.getOperand(1)))
+    return false;
+  unsigned TestOpcode = getTestSqrtOpcode(CmpLHS.getOperand(0));
+  if (!TestOpcode)
+    return false;
+
+  unsigned PCC;
+  bool IsCCNE = CC == ISD::SETNE;
+  switch (cast<ConstantSDNode>(CmpLHS.getOperand(1))->getZExtValue()) {
+  case 2:
+    PCC = IsCCNE ? PPC::PRED_EQ : PPC::PRED_NE;
+    break;
+  case 4:
+    PCC = IsCCNE ? PPC::PRED_GT : PPC::PRED_LE;
+    break;
+  case 8:
+    PCC = IsCCNE ? PPC::PRED_LT : PPC::PRED_GE;
+    break;
+  default:
+    return false;
+  }
+
+  SDLoc dl(N);
+  SDNode *TestNode = CmpLHS.getOperand(0).getNode();
+  if (TestNode->getNumOperands() == 3)
+    TestNode = CurDAG->getMachineNode(TestOpcode, dl, TestNode->getValueType(0),
+                                      TestNode->getOperand(1),
+                                      TestNode->getOperand(2));
+  else
+    TestNode = CurDAG->getMachineNode(TestOpcode, dl, TestNode->getValueType(0),
+                                      TestNode->getOperand(1));
+  SDValue Ops[] = {getI32Imm(PCC, dl), SDValue(TestNode, 0), N->getOperand(4),
+                   N->getOperand(0)};
+  CurDAG->SelectNodeTo(N, PPC::BCC, MVT::Other, Ops);
+  return true;
+}
+
 bool PPCDAGToDAGISel::tryAsSingleRLWINM(SDNode *N) {
   assert(N->getOpcode() == ISD::AND && "ISD::AND SDNode expected");
   unsigned Imm;
   if (!isInt32Immediate(N->getOperand(1), Imm))
     return false;
 
   SDLoc dl(N);
   SDValue Val = N->getOperand(0);
case PPCISD::COND_BRANCH: {
 
     SDValue Pred = getI32Imm(PCC, dl);
     SDValue Ops[] = { Pred, N->getOperand(2), N->getOperand(3),
       N->getOperand(0), N->getOperand(4) };
     CurDAG->SelectNodeTo(N, PPC::BCC, MVT::Other, Ops);
     return;
   }
   case ISD::BR_CC: {
+    if (tryFoldSqrtTestBr(N))
+      return;
     ISD::CondCode CC = cast<CondCodeSDNode>(N->getOperand(1))->get();
     unsigned PCC =
         getPredicateForSetCC(CC, N->getOperand(2).getValueType(), Subtarget);
 
     if (N->getOperand(2).getValueType() == MVT::i1) {
       unsigned Opc;
       bool Swap;
       switch (PCC) {

llvm/test/CodeGen/PowerPC/branch_test.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -verify-machineinstrs -mcpu=pwr9 -mtriple=powerpc64le-unknown-unknown \
 ; RUN:   -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names < %s | FileCheck %s
 
 @val = external local_unnamed_addr global i32, align 4
 declare i32 @llvm.ppc.vsx.xvtsqrtdp(<2 x double>)
 
 define dso_local signext i32 @xvtsqrtdp_and_2_eq(<2 x double> %input) {
 ; CHECK-LABEL: xvtsqrtdp_and_2_eq:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    xvtsqrtdp cr0, v2
-; CHECK-NEXT:    mfocrf r3, 128
-; CHECK-NEXT:    srwi r3, r3, 28
-; CHECK-NEXT:    andi. r3, r3, 2
-; CHECK-NEXT:    beq cr0, .LBB0_2
+; CHECK-NEXT:    bne cr0, .LBB0_2
 ; CHECK-NEXT:  # %bb.1: # %if.then
 ; CHECK-NEXT:    addis r3, r2, .LC0@toc@ha
 ; CHECK-NEXT:    li r4, 100
 ; CHECK-NEXT:    ld r3, .LC0@toc@l(r3)
 ; CHECK-NEXT:    stw r4, 0(r3)
 ; CHECK-NEXT:  .LBB0_2: # %if.end
 ; CHECK-NEXT:    li r3, 1
 ; CHECK-NEXT:    blr

 if.end:                                           ; preds = %if.then, %entry
   ret i32 1
 }
 
 define dso_local signext i32 @xvtsqrtdp_and_4_eq(<2 x double> %input) {
 ; CHECK-LABEL: xvtsqrtdp_and_4_eq:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    xvtsqrtdp cr0, v2
-; CHECK-NEXT:    mfocrf r3, 128
-; CHECK-NEXT:    srwi r3, r3, 28
-; CHECK-NEXT:    andi. r3, r3, 4
-; CHECK-NEXT:    beq cr0, .LBB1_2
+; CHECK-NEXT:    ble cr0, .LBB1_2
 ; CHECK-NEXT:  # %bb.1: # %if.then
 ; CHECK-NEXT:    addis r3, r2, .LC0@toc@ha
 ; CHECK-NEXT:    li r4, 100
 ; CHECK-NEXT:    ld r3, .LC0@toc@l(r3)
 ; CHECK-NEXT:    stw r4, 0(r3)
 ; CHECK-NEXT:  .LBB1_2: # %if.end
 ; CHECK-NEXT:    li r3, 1
 ; CHECK-NEXT:    blr

 if.end:                                           ; preds = %if.then, %entry
   ret i32 1
 }
 
 define dso_local signext i32 @xvtsqrtdp_and_8_eq(<2 x double> %input) {
 ; CHECK-LABEL: xvtsqrtdp_and_8_eq:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    xvtsqrtdp cr0, v2
-; CHECK-NEXT:    mfocrf r3, 128
-; CHECK-NEXT:    srwi r3, r3, 28
-; CHECK-NEXT:    andi. r3, r3, 8
-; CHECK-NEXT:    beq cr0, .LBB2_2
+; CHECK-NEXT:    bge cr0, .LBB2_2
 ; CHECK-NEXT:  # %bb.1: # %if.then
 ; CHECK-NEXT:    addis r3, r2, .LC0@toc@ha
 ; CHECK-NEXT:    li r4, 100
 ; CHECK-NEXT:    ld r3, .LC0@toc@l(r3)
 ; CHECK-NEXT:    stw r4, 0(r3)
 ; CHECK-NEXT:  .LBB2_2: # %if.end
 ; CHECK-NEXT:    li r3, 1
 ; CHECK-NEXT:    blr

 if.end:                                           ; preds = %if.then, %entry
   ret i32 1
 }
 
 define dso_local signext i32 @xvtsqrtdp_and_2_ne(<2 x double> %input) {
 ; CHECK-LABEL: xvtsqrtdp_and_2_ne:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    xvtsqrtdp cr0, v2
-; CHECK-NEXT:    mfocrf r3, 128
-; CHECK-NEXT:    srwi r3, r3, 28
-; CHECK-NEXT:    andi. r3, r3, 2
-; CHECK-NEXT:    bne cr0, .LBB3_2
+; CHECK-NEXT:    beq cr0, .LBB3_2
 ; CHECK-NEXT:  # %bb.1: # %if.then
 ; CHECK-NEXT:    addis r3, r2, .LC0@toc@ha
 ; CHECK-NEXT:    li r4, 100
 ; CHECK-NEXT:    ld r3, .LC0@toc@l(r3)
 ; CHECK-NEXT:    stw r4, 0(r3)
 ; CHECK-NEXT:  .LBB3_2: # %if.end
 ; CHECK-NEXT:    li r3, 1
 ; CHECK-NEXT:    blr

 if.end:                                           ; preds = %if.then, %entry
   ret i32 1
 }
 
 define dso_local signext i32 @xvtsqrtdp_and_4_ne(<2 x double> %input) {
 ; CHECK-LABEL: xvtsqrtdp_and_4_ne:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    xvtsqrtdp cr0, v2
-; CHECK-NEXT:    mfocrf r3, 128
-; CHECK-NEXT:    srwi r3, r3, 28
-; CHECK-NEXT:    andi. r3, r3, 4
-; CHECK-NEXT:    bne cr0, .LBB4_2
+; CHECK-NEXT:    bgt cr0, .LBB4_2
 ; CHECK-NEXT:  # %bb.1: # %if.then
 ; CHECK-NEXT:    addis r3, r2, .LC0@toc@ha
 ; CHECK-NEXT:    li r4, 100
 ; CHECK-NEXT:    ld r3, .LC0@toc@l(r3)
 ; CHECK-NEXT:    stw r4, 0(r3)
 ; CHECK-NEXT:  .LBB4_2: # %if.end
 ; CHECK-NEXT:    li r3, 1
 ; CHECK-NEXT:    blr

 if.end:                                           ; preds = %if.then, %entry
   ret i32 1
 }
 
 define dso_local signext i32 @xvtsqrtdp_and_8_ne(<2 x double> %input) {
 ; CHECK-LABEL: xvtsqrtdp_and_8_ne:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    xvtsqrtdp cr0, v2
-; CHECK-NEXT:    mfocrf r3, 128
-; CHECK-NEXT:    srwi r3, r3, 28
-; CHECK-NEXT:    andi. r3, r3, 8
-; CHECK-NEXT:    bne cr0, .LBB5_2
+; CHECK-NEXT:    blt cr0, .LBB5_2
 ; CHECK-NEXT:  # %bb.1: # %if.then
 ; CHECK-NEXT:    addis r3, r2, .LC0@toc@ha
 ; CHECK-NEXT:    li r4, 100
 ; CHECK-NEXT:    ld r3, .LC0@toc@l(r3)
 ; CHECK-NEXT:    stw r4, 0(r3)
 ; CHECK-NEXT:  .LBB5_2: # %if.end
 ; CHECK-NEXT:    li r3, 1
 ; CHECK-NEXT:    blr