[PowerPC] Try to fold sqrt/sdiv test results with the branch.

[Esme]

Summary

The patch tries to fold sqrt/sdiv test node, i.g FTSQRT, XVTDIVDP, and the branch, i.e br_cc if they satisfy these patterns:

(br_cc seteq, (truncateToi1 SWTestOp), 0) -> (BCC PRED_NU, SWTestOp)
(br_cc seteq, (and Node_test, 2), 0) -> (BCC PRED_NE, PPCNode_test)
(br_cc seteq, (and Node_test, 4), 0) -> (BCC PRED_LE, PPCNode_test)
(br_cc seteq, (and Node_test, 8), 0) -> (BCC PRED_GE, PPCNode_test)
(br_cc setne, (truncateToi1 SWTestOp), 0) -> (BCC PRED_UN, SWTestOp) 
(br_cc setne, (and Node_test, 2), 0) -> (BCC PRED_EQ, PPCNode_test)
(br_cc setne, (and Node_test, 4), 0) -> (BCC PRED_GT, PPCNode_test)
(br_cc setne, (and Node_test, 8), 0) -> (BCC PRED_LT, PPCNode_test)

[steven.zhang]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

4382

Do you have any problems to implement this folding with .td ? It doesn't make sense to me to get the opcode and then do the folding, as that is done during instr match(.td).

[Esme]

Leave the ISel of intrinsics to td.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

4382

These pattens have to be matched before the selection of the single br_cc, which is implemented in PPCISelDAGToDAG.cpp, therefore these folding can't be implemented with .td. You're right, that it'll be better to leave the selection of intrinsics to td.

[nemanjai]

It seems like this patch depends on another patch. Please don't forget to add the dependencies to the Phabricator reviews.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

4382

If the only possible use for these test nodes are to and the result with 1/2/4, you might as well check the whole thing in this function.
Otherwise, I think it's fine as-is.

4388

Seems odd that a function called isTestSqrtOpcode() returns true for a test for SW divide.

4405

Please add an assert that N is the right type of node (i.e. BR_CC).

4407

It is quite hard to make sense of such huge condition and it aids readability to break it up and name the operands (i.e. CC, CmpLHS, CmpRHS, TgtBlock).

Then you can check the following in order and make it obvious:

• the RHS is zero
• the CC is EQ/NE
• the LHS is (and (testSqrt), imm)

[Esme]

Addressed nemanjai's comments. :)

[Esme]

In D94054#2479070, @nemanjai wrote:

It seems like this patch depends on another patch. Please don't forget to add the dependencies to the Phabricator reviews.

An NFC patch, this patch depends on, only committed test cases.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

4382

If the only possible use for these test nodes are to and the result with 1/2/4, you might as well check the whole thing in this function.
Otherwise, I think it's fine as-is.

Another pattern (truncateToi1 SWTestOp) is added now, well, I can't find more possible patterns.

[steven.zhang]

LGTM. Please hold on for days to see if @nemanjai has more comments.

[nemanjai]

Thanks for addressing my comments and I'm sorry I didn't get a chance to re-review prior to commit. Looks good now.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

StringRef getPassName() const override {
       return "PowerPC DAG->DAG Pattern Instruction Selection";
     }
 
 // Include the pieces autogenerated from the target description.
 #include "PPCGenDAGISel.inc"
 
 private:
     bool trySETCC(SDNode *N);
+    bool tryFoldSWTestBRCC(SDNode *N);
     bool tryAsSingleRLDICL(SDNode *N);
     bool tryAsSingleRLDICR(SDNode *N);
     bool tryAsSingleRLWINM(SDNode *N);
     bool tryAsSingleRLWINM8(SDNode *N);
     bool tryAsSingleRLWIMI(SDNode *N);
     bool tryAsPairOfRLDICL(SDNode *N);
     bool tryAsSingleRLDIMI(SDNode *N);
 
static bool mayUseP9Setb(SDNode *N, const ISD::CondCode &CC, SelectionDAG *DAG,
   }
 
   LLVM_DEBUG(dbgs() << "Found a node that can be lowered to a SETB: ");
   LLVM_DEBUG(N->dump());
 
   return true;
 }
 
+// Return true if it's a software square-root/divide operand.
+static bool isSWTestOp(SDValue N) {
+  if (N.getOpcode() == PPCISD::FTSQRT)
+    return true;
+  if (N.getNumOperands() < 1 || !isa<ConstantSDNode>(N.getOperand(0)))
+    return false;
+  switch (N.getConstantOperandVal(0)) {
+  case Intrinsic::ppc_vsx_xvtdivdp:
+  case Intrinsic::ppc_vsx_xvtdivsp:
+  case Intrinsic::ppc_vsx_xvtsqrtdp:
+  case Intrinsic::ppc_vsx_xvtsqrtsp:
+    return true;
+  }
+  return false;
+}
+
+bool PPCDAGToDAGISel::tryFoldSWTestBRCC(SDNode *N) {
+  assert(N->getOpcode() == ISD::BR_CC && "ISD::BR_CC is expected.");
+  // We are looking for following patterns, where `truncate to i1` actually has
+  // the same semantic with `and 1`.
+  // (br_cc seteq, (truncateToi1 SWTestOp), 0) -> (BCC PRED_NU, SWTestOp)
+  // (br_cc seteq, (and SWTestOp, 2), 0) -> (BCC PRED_NE, SWTestOp)
+  // (br_cc seteq, (and SWTestOp, 4), 0) -> (BCC PRED_LE, SWTestOp)
+  // (br_cc seteq, (and SWTestOp, 8), 0) -> (BCC PRED_GE, SWTestOp)
+  // (br_cc setne, (truncateToi1 SWTestOp), 0) -> (BCC PRED_UN, SWTestOp)
+  // (br_cc setne, (and SWTestOp, 2), 0) -> (BCC PRED_EQ, SWTestOp)
+  // (br_cc setne, (and SWTestOp, 4), 0) -> (BCC PRED_GT, SWTestOp)
+  // (br_cc setne, (and SWTestOp, 8), 0) -> (BCC PRED_LT, SWTestOp)
+  ISD::CondCode CC = cast<CondCodeSDNode>(N->getOperand(1))->get();
+  if (CC != ISD::SETEQ && CC != ISD::SETNE)
+    return false;
+
+  SDValue CmpRHS = N->getOperand(3);
+  if (!isa<ConstantSDNode>(CmpRHS) ||
+      cast<ConstantSDNode>(CmpRHS)->getSExtValue() != 0)
+    return false;
+
+  SDValue CmpLHS = N->getOperand(2);
+  if (CmpLHS.getNumOperands() < 1 || !isSWTestOp(CmpLHS.getOperand(0)))
+    return false;
+
+  unsigned PCC = 0;
+  bool IsCCNE = CC == ISD::SETNE;
+  if (CmpLHS.getOpcode() == ISD::AND &&
+      isa<ConstantSDNode>(CmpLHS.getOperand(1)))
+    switch (CmpLHS.getConstantOperandVal(1)) {
+    case 1:
+      PCC = IsCCNE ? PPC::PRED_UN : PPC::PRED_NU;
+      break;
+    case 2:
+      PCC = IsCCNE ? PPC::PRED_EQ : PPC::PRED_NE;
+      break;
+    case 4:
+      PCC = IsCCNE ? PPC::PRED_GT : PPC::PRED_LE;
+      break;
+    case 8:
+      PCC = IsCCNE ? PPC::PRED_LT : PPC::PRED_GE;
+      break;
+    default:
+      return false;
+    }
+  else if (CmpLHS.getOpcode() == ISD::TRUNCATE &&
+           CmpLHS.getValueType() == MVT::i1)
+    PCC = IsCCNE ? PPC::PRED_UN : PPC::PRED_NU;
+
+  if (PCC) {
+    SDLoc dl(N);
+    SDValue Ops[] = {getI32Imm(PCC, dl), CmpLHS.getOperand(0), N->getOperand(4),
+                     N->getOperand(0)};
+    CurDAG->SelectNodeTo(N, PPC::BCC, MVT::Other, Ops);
+    return true;
+  }
+  return false;
+}
+
 bool PPCDAGToDAGISel::tryAsSingleRLWINM(SDNode *N) {
   assert(N->getOpcode() == ISD::AND && "ISD::AND SDNode expected");
   unsigned Imm;
   if (!isInt32Immediate(N->getOperand(1), Imm))
     return false;
 
   SDLoc dl(N);
   SDValue Val = N->getOperand(0);
case PPCISD::COND_BRANCH: {
 
     SDValue Pred = getI32Imm(PCC, dl);
     SDValue Ops[] = { Pred, N->getOperand(2), N->getOperand(3),
       N->getOperand(0), N->getOperand(4) };
     CurDAG->SelectNodeTo(N, PPC::BCC, MVT::Other, Ops);
     return;
   }
   case ISD::BR_CC: {
+    if (tryFoldSWTestBRCC(N))
+      return;
     ISD::CondCode CC = cast<CondCodeSDNode>(N->getOperand(1))->get();
     unsigned PCC =
         getPredicateForSetCC(CC, N->getOperand(2).getValueType(), Subtarget);
 
     if (N->getOperand(2).getValueType() == MVT::i1) {
       unsigned Opc;
       bool Swap;
       switch (PCC) {

llvm/test/CodeGen/PowerPC/fold_swtest_br.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -verify-machineinstrs -mcpu=pwr9 -mtriple=powerpc64le-unknown-unknown \
+; RUN:   -ppc-vsr-nums-as-vr -ppc-asm-full-reg-names < %s | FileCheck %s
+
+@val = external local_unnamed_addr global i32, align 4
+declare i32 @llvm.ppc.vsx.xvtsqrtdp(<2 x double>)
+
+define dso_local signext i32 @xvtsqrtdp_and_1_eq(<2 x double> %input) {
+; CHECK-LABEL: xvtsqrtdp_and_1_eq:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    xvtsqrtdp cr0, v2
+; CHECK-NEXT:    bnu cr0, .LBB0_2
+; CHECK-NEXT:  # %bb.1: # %if.then
+; CHECK-NEXT:    addis r3, r2, .LC0@toc@ha
+; CHECK-NEXT:    li r4, 100
+; CHECK-NEXT:    ld r3, .LC0@toc@l(r3)
+; CHECK-NEXT:    stw r4, 0(r3)
+; CHECK-NEXT:  .LBB0_2: # %if.end
+; CHECK-NEXT:    li r3, 1
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call i32 @llvm.ppc.vsx.xvtsqrtdp(<2 x double> %input)
+  %1 = and i32 %0, 1
+  %cmp.not = icmp eq i32 %1, 0
+  br i1 %cmp.not, label %if.end, label %if.then
+
+if.then:                                          ; preds = %entry
+  store i32 100, i32* @val, align 4
+  br label %if.end
+
+if.end:                                           ; preds = %if.then, %entry
+  ret i32 1
+}
+
+define dso_local signext i32 @xvtsqrtdp_and_2_eq(<2 x double> %input) {
+; CHECK-LABEL: xvtsqrtdp_and_2_eq:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    xvtsqrtdp cr0, v2
+; CHECK-NEXT:    bne cr0, .LBB1_2
+; CHECK-NEXT:  # %bb.1: # %if.then
+; CHECK-NEXT:    addis r3, r2, .LC0@toc@ha
+; CHECK-NEXT:    li r4, 100
+; CHECK-NEXT:    ld r3, .LC0@toc@l(r3)
+; CHECK-NEXT:    stw r4, 0(r3)
+; CHECK-NEXT:  .LBB1_2: # %if.end
+; CHECK-NEXT:    li r3, 1
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call i32 @llvm.ppc.vsx.xvtsqrtdp(<2 x double> %input)
+  %1 = and i32 %0, 2
+  %cmp.not = icmp eq i32 %1, 0
+  br i1 %cmp.not, label %if.end, label %if.then
+
+if.then:                                          ; preds = %entry
+  store i32 100, i32* @val, align 4
+  br label %if.end
+
+if.end:                                           ; preds = %if.then, %entry
+  ret i32 1
+}
+
+define dso_local signext i32 @xvtsqrtdp_and_4_eq(<2 x double> %input) {
+; CHECK-LABEL: xvtsqrtdp_and_4_eq:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    xvtsqrtdp cr0, v2
+; CHECK-NEXT:    ble cr0, .LBB2_2
+; CHECK-NEXT:  # %bb.1: # %if.then
+; CHECK-NEXT:    addis r3, r2, .LC0@toc@ha
+; CHECK-NEXT:    li r4, 100
+; CHECK-NEXT:    ld r3, .LC0@toc@l(r3)
+; CHECK-NEXT:    stw r4, 0(r3)
+; CHECK-NEXT:  .LBB2_2: # %if.end
+; CHECK-NEXT:    li r3, 1
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call i32 @llvm.ppc.vsx.xvtsqrtdp(<2 x double> %input)
+  %1 = and i32 %0, 4
+  %cmp.not = icmp eq i32 %1, 0
+  br i1 %cmp.not, label %if.end, label %if.then
+
+if.then:                                          ; preds = %entry
+  store i32 100, i32* @val, align 4
+  br label %if.end
+
+if.end:                                           ; preds = %if.then, %entry
+  ret i32 1
+}
+
+define dso_local signext i32 @xvtsqrtdp_and_8_eq(<2 x double> %input) {
+; CHECK-LABEL: xvtsqrtdp_and_8_eq:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    xvtsqrtdp cr0, v2
+; CHECK-NEXT:    bge cr0, .LBB3_2
+; CHECK-NEXT:  # %bb.1: # %if.then
+; CHECK-NEXT:    addis r3, r2, .LC0@toc@ha
+; CHECK-NEXT:    li r4, 100
+; CHECK-NEXT:    ld r3, .LC0@toc@l(r3)
+; CHECK-NEXT:    stw r4, 0(r3)
+; CHECK-NEXT:  .LBB3_2: # %if.end
+; CHECK-NEXT:    li r3, 1
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call i32 @llvm.ppc.vsx.xvtsqrtdp(<2 x double> %input)
+  %1 = and i32 %0, 8
+  %cmp.not = icmp eq i32 %1, 0
+  br i1 %cmp.not, label %if.end, label %if.then
+
+if.then:                                          ; preds = %entry
+  store i32 100, i32* @val, align 4
+  br label %if.end
+
+if.end:                                           ; preds = %if.then, %entry
+  ret i32 1
+}
+
+define dso_local signext i32 @xvtsqrtdp_and_1_ne(<2 x double> %input) {
+; CHECK-LABEL: xvtsqrtdp_and_1_ne:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    xvtsqrtdp cr0, v2
+; CHECK-NEXT:    bun cr0, .LBB4_2
+; CHECK-NEXT:  # %bb.1: # %if.then
+; CHECK-NEXT:    addis r3, r2, .LC0@toc@ha
+; CHECK-NEXT:    li r4, 100
+; CHECK-NEXT:    ld r3, .LC0@toc@l(r3)
+; CHECK-NEXT:    stw r4, 0(r3)
+; CHECK-NEXT:  .LBB4_2: # %if.end
+; CHECK-NEXT:    li r3, 1
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call i32 @llvm.ppc.vsx.xvtsqrtdp(<2 x double> %input)
+  %1 = and i32 %0, 1
+  %cmp.not = icmp ne i32 %1, 0
+  br i1 %cmp.not, label %if.end, label %if.then
+
+if.then:                                          ; preds = %entry
+  store i32 100, i32* @val, align 4
+  br label %if.end
+
+if.end:                                           ; preds = %if.then, %entry
+  ret i32 1
+}
+
+define dso_local signext i32 @xvtsqrtdp_and_2_ne(<2 x double> %input) {
+; CHECK-LABEL: xvtsqrtdp_and_2_ne:
+; CHECK:       # %bb.0: # %if.end
+; CHECK-NEXT:    li r3, 1
+; CHECK-NEXT:    blr
+if.end:                                           ; preds = %if.then, %entry
+  ret i32 1
+}
+
+define dso_local signext i32 @xvtsqrtdp_and_4_ne(<2 x double> %input) {
+; CHECK-LABEL: xvtsqrtdp_and_4_ne:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    xvtsqrtdp cr0, v2
+; CHECK-NEXT:    bgt cr0, .LBB6_2
+; CHECK-NEXT:  # %bb.1: # %if.then
+; CHECK-NEXT:    addis r3, r2, .LC0@toc@ha
+; CHECK-NEXT:    li r4, 100
+; CHECK-NEXT:    ld r3, .LC0@toc@l(r3)
+; CHECK-NEXT:    stw r4, 0(r3)
+; CHECK-NEXT:  .LBB6_2: # %if.end
+; CHECK-NEXT:    li r3, 1
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call i32 @llvm.ppc.vsx.xvtsqrtdp(<2 x double> %input)
+  %1 = and i32 %0, 4
+  %cmp.not = icmp ne i32 %1, 0
+  br i1 %cmp.not, label %if.end, label %if.then
+
+if.then:                                          ; preds = %entry
+  store i32 100, i32* @val, align 4
+  br label %if.end
+
+if.end:                                           ; preds = %if.then, %entry
+  ret i32 1
+}
+
+define dso_local signext i32 @xvtsqrtdp_and_8_ne(<2 x double> %input) {
+; CHECK-LABEL: xvtsqrtdp_and_8_ne:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    xvtsqrtdp cr0, v2
+; CHECK-NEXT:    blt cr0, .LBB7_2
+; CHECK-NEXT:  # %bb.1: # %if.then
+; CHECK-NEXT:    addis r3, r2, .LC0@toc@ha
+; CHECK-NEXT:    li r4, 100
+; CHECK-NEXT:    ld r3, .LC0@toc@l(r3)
+; CHECK-NEXT:    stw r4, 0(r3)
+; CHECK-NEXT:  .LBB7_2: # %if.end
+; CHECK-NEXT:    li r3, 1
+; CHECK-NEXT:    blr
+entry:
+  %0 = tail call i32 @llvm.ppc.vsx.xvtsqrtdp(<2 x double> %input)
+  %1 = and i32 %0, 8
+  %cmp.not = icmp ne i32 %1, 0
+  br i1 %cmp.not, label %if.end, label %if.then
+
+if.then:                                          ; preds = %entry
+  store i32 100, i32* @val, align 4
+  br label %if.end
+
+if.end:                                           ; preds = %if.then, %entry
+  ret i32 1
+}