[DAGCombiner] Allow target to control combine with illegal types.

[alonkom]

Summary

This patch is meant to deal with an issue we encountered in an out of tree target. We have a lot of pre type legalization target specific combines in which we would like to take advantage of existing target independent combines.
For example we would like catch some patterns that include the abs node previously combined by the DAGCombiner.
Relying on the target independent abs combines, would save us copying the logic into the target itself.
The issue arises when we are dealing with illegal types in which case the target independent combines will not fire since they are guarded by TLI.isOperationLegalOrCustom which makes a check on the legality of the type.
We would like to allow the target to choose to combine even though the type is illegal.
This patch replaces the calls of isOperationLegalOrCustom just for the abs case to demonstrate but it may be relevant in other cases as well.

[craig.topper]

Do you do custom type legalization for ABS in your target? There's no support for legalizing it in the target type independent legalizer.

[craig.topper]

There is this patch that was trying to make ABS more available D50239

[RKSimon]

D49837 is a generic patch to handle SPF_ABS/SPF_NABS in SelectionDAGBuilder::visitSelect - @ikulagin are you still looking at this?

[ikulagin]

In D56270#1351287, @RKSimon wrote:

D49837 is a generic patch to handle SPF_ABS/SPF_NABS in SelectionDAGBuilder::visitSelect - @ikulagin are you still looking at this?

I am waiting for feedback and I want to coordinate the activity.

[delena]

ABS is only an example. We'd like to enable/disable the common transformation for other nodes as well.
Is it reasonable to add CombineLevel to the interface to make it more general?

[alonkom]

In D56270#1351232, @craig.topper wrote:

Do you do custom type legalization for ABS in your target? There's no support for legalizing it in the target type independent legalizer.

We do. We rely on the DAGCombiner to combine abs for all types, and deal with type legalization in our target.

[craig.topper]

How do you get your target to be able to type legalize an abs of any type? The interface for enabling custom legalization requires calling setOperation with a specific type does it not?

[alonkom]

In D56270#1352596, @craig.topper wrote:

How do you get your target to be able to type legalize an abs of any type? The interface for enabling custom legalization requires calling setOperation with a specific type does it not?

What I meant was that we use the combined abs only in very specific scenarios and with specific types (some are illegal), and combine it to some other target specific node.
Actually, we don't use custom type legalization for the abs, so we will get an error if the abs has an illegal type.

include/llvm/CodeGen/TargetLowering.h

LegalizeAction getStrictFPOperationAction(unsigned Op, EVT VT) const {
     // We don't currently handle Custom or Promote for strict FP pseudo-ops.
     // For now, we just expand for those cases.
     if (Action != Legal)
       Action = Expand;
 
     return Action;
   }
 
+  /// Return true if target should combine \p Op with \p VT.
+  virtual bool isOperationProfitableToCombine(unsigned Op, EVT VT) const {
+    return isOperationLegalOrCustom(Op, VT);
+  }
+
   /// Return true if the specified operation is legal on this target or can be
   /// made legal with custom lowering. This is used to help guide high-level
   /// lowering decisions.
   bool isOperationLegalOrCustom(unsigned Op, EVT VT) const {
     return (VT == MVT::Other || isTypeLegal(VT)) &&
       (getOperationAction(Op, VT) == Legal ||
        getOperationAction(Op, VT) == Custom);
   }

lib/CodeGen/SelectionDAG/DAGCombiner.cpp

SDValue DAGCombiner::visitSUB(SDNode *N) {
 
   if (SDValue V = foldAddSubBoolOfMaskedVal(N, DAG))
     return V;
 
   if (SDValue V = foldAddSubOfSignBit(N, DAG))
     return V;
 
   // fold Y = sra (X, size(X)-1); sub (xor (X, Y), Y) -> (abs X)
-  if (TLI.isOperationLegalOrCustom(ISD::ABS, VT)) {
+  if (TLI.isOperationProfitableToCombine(ISD::ABS, VT)) {
     if (N0.getOpcode() == ISD::XOR && N1.getOpcode() == ISD::SRA) {
       SDValue X0 = N0.getOperand(0), X1 = N0.getOperand(1);
       SDValue S0 = N1.getOperand(0);
       if ((X0 == S0 && X1 == N1) || (X0 == N1 && X1 == S0)) {
         unsigned OpSizeInBits = VT.getScalarSizeInBits();
         if (ConstantSDNode *C = isConstOrConstSplat(N1.getOperand(1)))
           if (C->getAPIntValue() == (OpSizeInBits - 1))
             return DAG.getNode(ISD::ABS, SDLoc(N), VT, S0);
if (XorC && ShiftC) {
           SDValue Not = DAG.getNOT(DL, N0.getOperand(0), VT);
           return DAG.getNode(N0Opcode, DL, VT, Not, N0.getOperand(1));
         }
       }
     }
   }
 
   // fold Y = sra (X, size(X)-1); xor (add (X, Y), Y) -> (abs X)
-  if (TLI.isOperationLegalOrCustom(ISD::ABS, VT)) {
+  if (TLI.isOperationProfitableToCombine(ISD::ABS, VT)) {
     SDValue A = N0Opcode == ISD::ADD ? N0 : N1;
     SDValue S = N0Opcode == ISD::SRA ? N0 : N1;
     if (A.getOpcode() == ISD::ADD && S.getOpcode() == ISD::SRA) {
       SDValue A0 = A.getOperand(0), A1 = A.getOperand(1);
       SDValue S0 = S.getOperand(0);
       if ((A0 == S && A1 == S0) || (A1 == S && A0 == S0)) {
         unsigned OpSizeInBits = VT.getScalarSizeInBits();
         if (ConstantSDNode *C = isConstOrConstSplat(S.getOperand(1)))
if (((RHSIsAllZeros && (CC == ISD::SETGT || CC == ISD::SETGE)) ||
         N1 == LHS && N2.getOpcode() == ISD::SUB && N1 == N2.getOperand(1))
       isAbs = ISD::isBuildVectorAllZeros(N2.getOperand(0).getNode());
     else if ((RHSIsAllZeros && (CC == ISD::SETLT || CC == ISD::SETLE)) &&
              N2 == LHS && N1.getOpcode() == ISD::SUB && N2 == N1.getOperand(1))
       isAbs = ISD::isBuildVectorAllZeros(N1.getOperand(0).getNode());
 
     if (isAbs) {
       EVT VT = LHS.getValueType();
-      if (TLI.isOperationLegalOrCustom(ISD::ABS, VT))
+      if (TLI.isOperationProfitableToCombine(ISD::ABS, VT))
         return DAG.getNode(ISD::ABS, DL, VT, LHS);
 
       SDValue Shift = DAG.getNode(
           ISD::SRA, DL, VT, LHS,
           DAG.getConstant(VT.getScalarSizeInBits() - 1, DL, VT));
       SDValue Add = DAG.getNode(ISD::ADD, DL, VT, LHS, Shift);
       AddToWorklist(Shift.getNode());
       AddToWorklist(Add.getNode());