[SelectionDAG][GISel] Make LegalizeDAG lower FNEG using integer ops.

[efriedma]

Summary

Previously, if a floating-point type was legal, but FNEG wasn't legal, we would use FSUB. Instead, we should use integer ops, to preserve the semantics. (Alternatively, there's a compiler-rt call we could use, but there isn't much reason to use that.)

It turns out we actually are still using this obscure SelectionDAG codepath in a few cases: on some targets, we have "legal" floating-point types that don't actually support any floating-point operations. In particular, ARM and AArch64 are using this path.

For GlobalISel, all targets without a native FNEG used this codepath.

The implementation for SelectionDAG is pretty simple because we can reuse the infrastructure from FCOPYSIGN. For GlobalISel, it's really easy because we can just blindly emit G_XOR and let legalization of the XOR take care of the rest.

See also 9a3dc3e, the corresponding change to type legalization.

Also includes a "bonus" change to STRICT_FSUB legalization, so we can lower an FSUB_STRICT to a float libcall.

Includes the changes to both LegalizeDAG and GlobalISel so we don't have inconsistent results in the future.

Fixes https://bugs.llvm.org/show_bug.cgi?id=46792 .

[arsenm]

GlobalISel has the same problem

[RKSimon]

llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp

3240

Is it worth moving this to a ExpandFNEG helper like the other ops?

[efriedma]

Refactor. Fix a regression that came up when I rebased.

[arsenm]

This also needs to be fixed for globalisel to avoid this bug appearing or not depending on other code in the function hitting the fallback

llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp

1574

The explanation is too vague and not quite correct. It's specifically signaling nans (and also possibly denormal flushing). fsub is a canonicalizing operation, but fneg is not

[efriedma]

Do you think the globalisel change has to be done in the same patch? I guess I can try to look into it.

llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp

1574

There's also a problem for non-signalling nans: the sign of the result is unspecified. Maybe common implementations flip the sign? Not sure.

[arsenm]

In D84287#2208438, @efriedma wrote:

Do you think the globalisel change has to be done in the same patch? I guess I can try to look into it.

Yes, but it would also make sense for it to be a separate patch. I just think it’s important that they’re at least consistent in their bugs

[efriedma]

Updated with corresponding GlobalISel changes.

[efriedma]

Ping

[RKSimon]

@arsenm Any more thoughts?

llvm/lib/Target/ARM/ARMLegalizerInfo.cpp

91

should this be a separate fix?

llvm/test/CodeGen/X86/GlobalISel/legalize-fneg.mir

2

Worth adding a f80 case as well?

[efriedma]

llvm/lib/Target/ARM/ARMLegalizerInfo.cpp

91

The tests in this patch exercise this codepath. I guess it could also be tested independently; I'll look at adding a few more tests.

llvm/test/CodeGen/X86/GlobalISel/legalize-fneg.mir

2

f80 currently isn't supported by GlobalISel.

[RKSimon]

llvm/test/CodeGen/X86/GlobalISel/legalize-fneg.mir

2

ok - I was mainly interested in seeing what happened with a non-legal integer type - not a big issue though

[arsenm]

llvm/test/CodeGen/X86/GlobalISel/legalize-fneg.mir

2

It doesn't work end to end, but it should work here if you use an 80 bit value

[efriedma]

llvm/test/CodeGen/X86/GlobalISel/legalize-fneg.mir

2

I wrote a test, and got "LLVM ERROR: unable to legalize instruction: %1:_(s80) = G_XOR %0:_, %2:_ (in function: test_fneg_f80)".

llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp

case TargetOpcode::G_UMULO: {
     return Legalized;
   }
   case TargetOpcode::G_FNEG: {
     // TODO: Handle vector types once we are able to
     // represent them.
     if (Ty.isVector())
       return UnableToLegalize;
     Register Res = MI.getOperand(0).getReg();
-    LLVMContext &Ctx = MIRBuilder.getMF().getFunction().getContext();
-    Type *ZeroTy = getFloatTypeForLLT(Ctx, Ty);
-    if (!ZeroTy)
-      return UnableToLegalize;
-    ConstantFP &ZeroForNegation =
-        *cast<ConstantFP>(ConstantFP::getZeroValueForNegation(ZeroTy));
-    auto Zero = MIRBuilder.buildFConstant(Ty, ZeroForNegation);
+    auto SignMask =
+        MIRBuilder.buildConstant(Ty, APInt::getSignMask(Ty.getSizeInBits()));
     Register SubByReg = MI.getOperand(1).getReg();
-    Register ZeroReg = Zero.getReg(0);
-    MIRBuilder.buildFSub(Res, ZeroReg, SubByReg, MI.getFlags());
+    MIRBuilder.buildXor(Res, SubByReg, SignMask);
     MI.eraseFromParent();
     return Legalized;
   }
   case TargetOpcode::G_FSUB: {
     // Lower (G_FSUB LHS, RHS) to (G_FADD LHS, (G_FNEG RHS)).
     // First, check if G_FNEG is marked as Lower. If so, we may
     // end up with an infinite loop as G_FSUB is used to legalize G_FNEG.
     if (LI.getAction({G_FNEG, {Ty}}).Action == Lower)

llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp

private:
   void ExpandDYNAMIC_STACKALLOC(SDNode *Node,
                                 SmallVectorImpl<SDValue> &Results);
   void getSignAsIntValue(FloatSignAsInt &State, const SDLoc &DL,
                          SDValue Value) const;
   SDValue modifySignAsInt(const FloatSignAsInt &State, const SDLoc &DL,
                           SDValue NewIntValue) const;
   SDValue ExpandFCOPYSIGN(SDNode *Node) const;
   SDValue ExpandFABS(SDNode *Node) const;
+  SDValue ExpandFNEG(SDNode *Node) const;
   SDValue ExpandLegalINT_TO_FP(SDNode *Node, SDValue &Chain);
   void PromoteLegalINT_TO_FP(SDNode *N, const SDLoc &dl,
                              SmallVectorImpl<SDValue> &Results);
   void PromoteLegalFP_TO_INT(SDNode *N, const SDLoc &dl,
                              SmallVectorImpl<SDValue> &Results);
 
   SDValue ExpandBITREVERSE(SDValue Op, const SDLoc &dl);
   SDValue ExpandBSWAP(SDValue Op, const SDLoc &dl);
if (SignBit.getValueSizeInBits() > ClearedSign.getValueSizeInBits()) {
     SignBit = DAG.getNode(ISD::TRUNCATE, DL, MagVT, SignBit);
   }
 
   // Store the part with the modified sign and convert back to float.
   SDValue CopiedSign = DAG.getNode(ISD::OR, DL, MagVT, ClearedSign, SignBit);
   return modifySignAsInt(MagAsInt, DL, CopiedSign);
 }
 
+SDValue SelectionDAGLegalize::ExpandFNEG(SDNode *Node) const {
+  // Get the sign bit as an integer.
+  SDLoc DL(Node);
+  FloatSignAsInt SignAsInt;
+  getSignAsIntValue(SignAsInt, DL, Node->getOperand(0));
+  EVT IntVT = SignAsInt.IntValue.getValueType();
+
+  // Flip the sign.
+  SDValue SignMask = DAG.getConstant(SignAsInt.SignMask, DL, IntVT);
+  SDValue SignFlip = DAG.getNode(ISD::XOR, DL, IntVT, SignAsInt.IntValue,
+                                 SignMask);
+
+  // Convert back to float.
+  return modifySignAsInt(SignAsInt, DL, SignFlip);
+}
+
 SDValue SelectionDAGLegalize::ExpandFABS(SDNode *Node) const {
   SDLoc DL(Node);
   SDValue Value = Node->getOperand(0);
 
   // Transform FABS(x) => FCOPYSIGN(x, 0.0) if FCOPYSIGN is legal.
   EVT FloatVT = Value.getValueType();
   if (TLI.isOperationLegalOrCustom(ISD::FCOPYSIGN, FloatVT)) {
     SDValue Zero = DAG.getConstantFP(0.0, DL, FloatVT);
bool SelectionDAGLegalize::ExpandNode(SDNode *Node) {
   case ISD::GET_DYNAMIC_AREA_OFFSET:
     Results.push_back(DAG.getConstant(0, dl, Node->getValueType(0)));
     Results.push_back(Results[0].getValue(0));
     break;
   case ISD::FCOPYSIGN:
     Results.push_back(ExpandFCOPYSIGN(Node));
     break;
   case ISD::FNEG:
-    // Expand Y = FNEG(X) ->  Y = SUB -0.0, X
-    Tmp1 = DAG.getConstantFP(-0.0, dl, Node->getValueType(0));
-    // TODO: If FNEG has fast-math-flags, propagate them to the FSUB.
-    Tmp1 = DAG.getNode(ISD::FSUB, dl, Node->getValueType(0), Tmp1,
-                       Node->getOperand(0));
-    Results.push_back(Tmp1);
+    Results.push_back(ExpandFNEG(Node));
     break;
   case ISD::FABS:
     Results.push_back(ExpandFABS(Node));
     break;
   case ISD::SMIN:
   case ISD::SMAX:
   case ISD::UMIN:
   case ISD::UMAX: {
     // Expand Y = MAX(A, B) -> Y = (A > B) ? A : B
     ISD::CondCode Pred;
if (!TLI.isStrictFPEnabled() && Results.empty() && Node->isStrictFPOpcode()) {
     switch (Node->getOpcode()) {
     default:
       if (TLI.getStrictFPOperationAction(Node->getOpcode(),
                                          Node->getValueType(0))
           == TargetLowering::Legal)
         return true;
       break;
     case ISD::STRICT_FSUB: {
-      if (TLI.getStrictFPOperationAction(Node->getOpcode(),
+      if (TLI.getStrictFPOperationAction(ISD::STRICT_FSUB,
                                          Node->getValueType(0))
           == TargetLowering::Legal)
         return true;
+      if (TLI.getStrictFPOperationAction(ISD::STRICT_FADD,
+                                         Node->getValueType(0))
+          != TargetLowering::Legal)
+        break;
 
       EVT VT = Node->getValueType(0);
       const SDNodeFlags Flags = Node->getFlags();
       SDValue Neg = DAG.getNode(ISD::FNEG, dl, VT, Node->getOperand(2), Flags);
       SDValue Fadd = DAG.getNode(ISD::STRICT_FADD, dl, Node->getVTList(),
                                  {Node->getOperand(0), Node->getOperand(1), Neg},
                          Flags);
 
       Results.push_back(Fadd);
       Results.push_back(Fadd.getValue(1));
       break;

llvm/lib/Target/ARM/ARMLegalizerInfo.cpp

ARMLegalizerInfo::ARMLegalizerInfo(const ARMSubtarget &ST) {
 
   getActionDefinitionsBuilder({G_SEXT, G_ZEXT, G_ANYEXT})
       .legalForCartesianProduct({s8, s16, s32}, {s1, s8, s16});
 
   getActionDefinitionsBuilder(G_SEXT_INREG).lower();
 
   getActionDefinitionsBuilder({G_MUL, G_AND, G_OR, G_XOR})
       .legalFor({s32})
-      .minScalar(0, s32);
+      .clampScalar(0, s32, s32);
 
   if (ST.hasNEON())
     getActionDefinitionsBuilder({G_ADD, G_SUB})
         .legalFor({s32, s64})
         .minScalar(0, s32);
   else
     getActionDefinitionsBuilder({G_ADD, G_SUB})
         .legalFor({s32})

llvm/test/CodeGen/AArch64/arm64-fp128.ll

 ; CHECK: bl __extenddftf2
 ; CHECK: str q0, [{{x[0-9]+}}, :lo12:lhs]
 
   ret void
 ; CHECK: ret
 }
 
 define fp128 @test_neg(fp128 %in) {
-; CHECK: [[$MINUS0:.LCPI[0-9]+_0]]:
-; Make sure the weird hex constant below *is* -0.0
-; CHECK-NEXT: fp128 -0
-
 ; CHECK-LABEL: test_neg:
 
-  ; Could in principle be optimized to fneg which we can't select, this makes
-  ; sure that doesn't happen.
+;; We convert this to fneg, and target-independent code expands it with
+;; integer operations.
   %ret = fsub fp128 0xL00000000000000008000000000000000, %in
-; CHECK: mov v1.16b, v0.16b
-; CHECK: ldr q0, [{{x[0-9]+}}, :lo12:[[$MINUS0]]]
-; CHECK: bl __subtf3
-
   ret fp128 %ret
-; CHECK: ret
+
+; CHECK:      str q0, [sp, #-16]!
+; CHECK-NEXT: ldrb w8, [sp, #15]
+; CHECK-NEXT: eor w8, w8, #0x80
+; CHECK-NEXT: strb w8, [sp, #15]
+; CHECK-NEXT: ldr q0, [sp], #16
+; CHECK-NEXT: ret
 }

llvm/test/CodeGen/ARM/GlobalISel/arm-legalize-fp.mir

registers:
   - { id: 1, class: _ }
 body:             |
   bb.0:
     liveins: $r0
 
     ; CHECK-DAG: [[X:%[0-9]+]]:_(s32) = COPY $r0
     %0(s32) = COPY $r0
     ; HARD: [[R:%[0-9]+]]:_(s32) = G_FNEG [[X]]
-    ; SOFT-NOT: G_FNEG
-    ; SOFT-DAG: [[ZERO:%[0-9]+]]:_(s32) = G_CONSTANT i32 -2147483648
-    ; SOFT: ADJCALLSTACKDOWN
-    ; SOFT-DAG: $r0 = COPY [[ZERO]]
-    ; SOFT-DAG: $r1 = COPY [[X]]
-    ; SOFT-AEABI: BL{{.*}} &__aeabi_fsub, {{.*}}, implicit $r0, implicit $r1, implicit-def $r0
-    ; SOFT-DEFAULT: BL{{.*}} &__subsf3, {{.*}}, implicit $r0, implicit $r1, implicit-def $r0
-    ; SOFT: [[R:%[0-9]+]]:_(s32) = COPY $r0
-    ; SOFT: ADJCALLSTACKUP
-    ; SOFT-NOT: G_FNEG
+    ; SOFT: [[ZERO:%[0-9]+]]:_(s32) = G_CONSTANT i32 -2147483648
+    ; SOFT: [[R:%[0-9]+]]:_(s32) = G_XOR [[X]], [[ZERO]]
     %1(s32) = G_FNEG %0
     ; CHECK: $r0 = COPY [[R]]
     $r0 = COPY %1(s32)
     BX_RET 14, $noreg, implicit $r0
 ...
 ---
 name:            test_fneg_double
 # CHECK-LABEL: name: test_fneg_double
bb.0:
 
     ; CHECK-DAG: [[X0:%[0-9]+]]:_(s32) = COPY $r0
     ; CHECK-DAG: [[X1:%[0-9]+]]:_(s32) = COPY $r1
     %0(s32) = COPY $r0
     %1(s32) = COPY $r1
     ; HARD-DAG: [[X:%[0-9]+]]:_(s64) = G_MERGE_VALUES [[X0]]
     %2(s64) = G_MERGE_VALUES %0(s32), %1(s32)
     ; HARD: [[R:%[0-9]+]]:_(s64) = G_FNEG [[X]]
-    ; SOFT-NOT: G_FNEG
-    ; SOFT-DAG: [[NEGATIVE_ZERO:%[0-9]+]]:_(s32) = G_CONSTANT i32 -2147483648
-    ; SOFT-DAG: [[POSITIVE_ZERO:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
-    ; SOFT: ADJCALLSTACKDOWN
-    ; SOFT-DAG: $r{{[0-1]}} = COPY [[NEGATIVE_ZERO]]
-    ; SOFT-DAG: $r{{[0-1]}} = COPY [[POSITIVE_ZERO]]
-    ; SOFT-DAG: $r{{[2-3]}} = COPY [[X0]]
-    ; SOFT-DAG: $r{{[2-3]}} = COPY [[X1]]
-    ; SOFT-AEABI: BL{{.*}} &__aeabi_dsub, {{.*}}, implicit $r0, implicit $r1, implicit $r2, implicit $r3, implicit-def $r0, implicit-def $r1
-    ; SOFT-DEFAULT: BL{{.*}} &__subdf3, {{.*}}, implicit $r0, implicit $r1, implicit $r2, implicit $r3, implicit-def $r0, implicit-def $r1
-    ; SOFT: ADJCALLSTACKUP
-    ; SOFT-NOT: G_FNEG
+    ; HARD: G_UNMERGE_VALUES [[R]](s64)
+    ; SOFT: [[POSITIVE_ZERO:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; SOFT: [[NEGATIVE_ZERO:%[0-9]+]]:_(s32) = G_CONSTANT i32 -2147483648
+    ; SOFT: [[LOWR:%[0-9]+]]:_(s32) = G_XOR [[X0]], [[POSITIVE_ZERO]]
+    ; SOFT: [[HIGHR:%[0-9]+]]:_(s32) = G_XOR [[X1]], [[NEGATIVE_ZERO]]
+    ; SOFT: $r0 = COPY [[LOWR]]
+    ; SOFT: $r1 = COPY [[HIGHR]]
     %3(s64) = G_FNEG %2
-    ; HARD-DAG: G_UNMERGE_VALUES [[R]](s64)
     %4(s32),%5(s32) = G_UNMERGE_VALUES %3(s64)
     $r0 = COPY %4(s32)
     $r1 = COPY %5(s32)
     BX_RET 14, $noreg, implicit $r0, implicit $r1
 ...
 ---
 name:            test_fpext_float_to_double
 # CHECK-LABEL: name: test_fpext_float_to_double

llvm/test/CodeGen/Thumb2/mve-fp-negabs.ll

 entry:
   %0 = fsub nnan ninf nsz <4 x float> <float 0.0e0, float 0.0e0, float 0.0e0, float 0.0e0>, %src
   ret <4 x float> %0
 }
 
 define arm_aapcs_vfpcc <2 x double> @fneg_float64_t(<2 x double> %src) {
 ; CHECK-LABEL: fneg_float64_t:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, r5, r7, lr}
-; CHECK-NEXT:    push {r4, r5, r7, lr}
-; CHECK-NEXT:    .vsave {d8, d9}
-; CHECK-NEXT:    vpush {d8, d9}
-; CHECK-NEXT:    vmov q4, q0
-; CHECK-NEXT:    vldr d0, .LCPI2_0
-; CHECK-NEXT:    vmov r2, r3, d9
-; CHECK-NEXT:    vmov r4, r5, d0
-; CHECK-NEXT:    mov r0, r4
-; CHECK-NEXT:    mov r1, r5
-; CHECK-NEXT:    bl __aeabi_dsub
-; CHECK-NEXT:    vmov r2, r3, d8
-; CHECK-NEXT:    vmov d9, r0, r1
-; CHECK-NEXT:    mov r0, r4
-; CHECK-NEXT:    mov r1, r5
-; CHECK-NEXT:    bl __aeabi_dsub
-; CHECK-NEXT:    vmov d8, r0, r1
-; CHECK-NEXT:    vmov q0, q4
-; CHECK-NEXT:    vpop {d8, d9}
-; CHECK-NEXT:    pop {r4, r5, r7, pc}
-; CHECK-NEXT:    .p2align 3
-; CHECK-NEXT:  @ %bb.1:
-; CHECK-NEXT:  .LCPI2_0:
-; CHECK-NEXT:    .long 0 @ double -0
-; CHECK-NEXT:    .long 2147483648
+; CHECK-NEXT:    .pad #16
+; CHECK-NEXT:    sub sp, #16
+; CHECK-NEXT:    vstr d1, [sp]
+; CHECK-NEXT:    ldrb.w r0, [sp, #7]
+; CHECK-NEXT:    vstr d0, [sp, #8]
+; CHECK-NEXT:    ldrb.w r1, [sp, #15]
+; CHECK-NEXT:    eor r0, r0, #128
+; CHECK-NEXT:    strb.w r0, [sp, #7]
+; CHECK-NEXT:    vldr d1, [sp]
+; CHECK-NEXT:    eor r0, r1, #128
+; CHECK-NEXT:    strb.w r0, [sp, #15]
+; CHECK-NEXT:    vldr d0, [sp, #8]
+; CHECK-NEXT:    add sp, #16
+; CHECK-NEXT:    bx lr
 entry:
   %0 = fsub nnan ninf nsz <2 x double> <double 0.0e0, double 0.0e0>, %src
   ret <2 x double> %0
 }
 
 define arm_aapcs_vfpcc <8 x half> @fabs_float16_t(<8 x half> %src) {
 ; CHECK-MVE-LABEL: fabs_float16_t:
 ; CHECK-MVE:       @ %bb.0: @ %entry

llvm/test/CodeGen/X86/GlobalISel/legalize-fneg.mir

 # NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
 # RUN: llc -O0 -run-pass=legalizer %s -o - | FileCheck %s
 
 --- |
   target triple = "x86_64--"
   define void @test_fneg_f32() {
   entry:
     ret void
   }
   define void @test_fneg_f64() {
   entry:
     ret void
   }
 ...
 ---
 name:            test_fneg_f32
 registers:
   - { id: 0, class: _ }
   - { id: 1, class: _ }
 body:             |
   bb.1:
     liveins:
     ; CHECK-LABEL: name: test_fneg_f32
     ; CHECK: [[DEF:%[0-9]+]]:_(s32) = IMPLICIT_DEF
-    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_FCONSTANT float -0.000000e+00
-    ; CHECK: [[FSUB:%[0-9]+]]:_(s32) = G_FSUB [[C]], [[DEF]]
-    ; CHECK: $edi = COPY [[FSUB]](s32)
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 -2147483648
+    ; CHECK: [[XOR:%[0-9]+]]:_(s32) = G_XOR [[DEF]], [[C]]
+    ; CHECK: $edi = COPY [[XOR]](s32)
     %0(s32) = IMPLICIT_DEF
     %1(s32) = G_FNEG %0
     $edi = COPY %1
 ...
 ---
 name:            test_fneg_f64
 registers:
   - { id: 0, class: _ }
   - { id: 1, class: _ }
 body:             |
   bb.1:
     liveins:
     ; CHECK-LABEL: name: test_fneg_f64
     ; CHECK: [[DEF:%[0-9]+]]:_(s64) = G_IMPLICIT_DEF
-    ; CHECK: [[C:%[0-9]+]]:_(s64) = G_FCONSTANT double -0.000000e+00
-    ; CHECK: [[FSUB:%[0-9]+]]:_(s64) = G_FSUB [[C]], [[DEF]]
-    ; CHECK: $rdi = COPY [[FSUB]](s64)
+    ; CHECK: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 -9223372036854775808
+    ; CHECK: [[XOR:%[0-9]+]]:_(s64) = G_XOR [[DEF]], [[C]]
+    ; CHECK: $rdi = COPY [[XOR]](s64)
     %0(s64) = G_IMPLICIT_DEF
     %1(s64) = G_FNEG %0
     $rdi = COPY %1
 ...