[AArch64][X86] Don't assume __powidf2 is available on Windows.

[efriedma]

Summary

We had some code for this for 32-bit ARM, but this doesn't really need to be in target-specific code; generalize it.

(I think this started showing up recently because we added an optimization that converts pow to powi.)

[compnerd]

I do believe that we need it for all the targets (x86, x86_64, armv7, aarch64).

lib/Target/AArch64/AArch64ISelLowering.cpp

529 ↗

(On Diff #225141)

Is there a reason to have the check do both? Is the latter not sufficient? The reason that we need to fall back to pow for powi is because MSVCRT does not support it, it has not much to do with the fact that it is Windows. That is - I can see this applying equally well to say ReactOS, which I suppose would be treated as Windows, but isn't really Windows.

lib/Target/ARM/ARMISelLowering.cpp

1163 ↗

(On Diff #225141)

Similar

[rnk]

I tried to get clang to make llvm.powi from C, but it was challenging. Setting -fno-math-errno was what ended up mattering, and even then, I couldn't go from llvm.pow.f64 to powi.

Then, I used powi directly from LLVM IR, and I couldn't get it to generate a libcall for x86. It just does repeated multiplication. So, I guess we don't need to do anything for x86?

lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp

581–582 ↗

(On Diff #225141)

Can this be made into an assert or non-fatal error by DAG->getContext().emitError("..."), and then returning the input SDNode unmodified?

[efriedma]

I do believe that we need it for all the targets (x86, x86_64, armv7, aarch64).

Okay.

Then, I used powi directly from LLVM IR, and I couldn't get it to generate a libcall for x86. It just does repeated multiplication.

We generally don't generate the libcall when the power is constant, only when it's a variable.

To make clang form powi from pow, I think you need -ffast-math. Try something like the following:

double pow(double,double);
double f(double x, int y) { return pow(x,y); }

[efriedma]

Removed redundant check. Added support for x86.

[compnerd]

I do like @rnk's suggestion for the emission of the error, but Im not really tied to it - if it triggers, the TLI for the target is not setup properly or is missing the symbol, in which case, it needs to be addressed anyways. Beyond that, assuming that @rnk is okay with this, LGTM. Thanks for fixing this.

lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp

581–582 ↗

(On Diff #225141)

I would rather have this be a report_fatal_error or the DAG->getContext().emitError + return SDNode(). Please don't make this into an assertion, I know too many people who tend to use the -Asserts variant of the toolchain.

[efriedma]

lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp

581–582 ↗

(On Diff #225141)

I think this is impossible to hit at the moment; there's no such thing as soft-float Windows. I just didn't want to make it an llvm_unreachable in case someone adds a new target where it might be relevant.

Or I could implement it without a testcase, I guess; it's not hard.

[compnerd]

lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp

581–582 ↗

(On Diff #225141)

@efriedma yeah, I think that both of us are in agreement, I just meant that leave this as is, or use the reporting error as @rnk suggested, but don't make it an assert is all.

[efriedma]

Updated. I'm a little uncomfortable with the untested codepath, but I guess it isn't any worse than report_fatal_error, which always kills the process.

[rnk]

lgtm

In D69013#1734847, @efriedma wrote:

Updated. I'm a little uncomfortable with the untested codepath, but I guess it isn't any worse than report_fatal_error, which always kills the process.

These days I'm trying to take a hard look at report_fatal_error and ask why it's not either unreachable, assert, or error.

llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp

void SelectionDAGLegalize::ConvertNodeToLibcall(SDNode *Node) {
   case ISD::STRICT_FROUND:
     Results.push_back(ExpandFPLibCall(Node, RTLIB::ROUND_F32,
                                       RTLIB::ROUND_F64,
                                       RTLIB::ROUND_F80,
                                       RTLIB::ROUND_F128,
                                       RTLIB::ROUND_PPCF128));
     break;
   case ISD::FPOWI:
-  case ISD::STRICT_FPOWI:
+  case ISD::STRICT_FPOWI: {
+    RTLIB::Libcall LC;
+    switch (Node->getSimpleValueType(0).SimpleTy) {
+    default: llvm_unreachable("Unexpected request for libcall!");
+    case MVT::f32: LC = RTLIB::POWI_F32; break;
+    case MVT::f64: LC = RTLIB::POWI_F64; break;
+    case MVT::f80: LC = RTLIB::POWI_F80; break;
+    case MVT::f128: LC = RTLIB::POWI_F128; break;
+    case MVT::ppcf128: LC = RTLIB::POWI_PPCF128; break;
+    }
+    if (!TLI.getLibcallName(LC)) {
+      // Some targets don't have a powi libcall; use pow instead.
+      SDValue Exponent = DAG.getNode(ISD::SINT_TO_FP, SDLoc(Node),
+                                     Node->getValueType(0),
+                                     Node->getOperand(1));
+      Results.push_back(DAG.getNode(ISD::FPOW, SDLoc(Node),
+                                    Node->getValueType(0), Node->getOperand(0),
+                                    Exponent));
+      break;
+    }
     Results.push_back(ExpandFPLibCall(Node, RTLIB::POWI_F32, RTLIB::POWI_F64,
                                       RTLIB::POWI_F80, RTLIB::POWI_F128,
                                       RTLIB::POWI_PPCF128));
     break;
+  }
   case ISD::FPOW:
   case ISD::STRICT_FPOW:
     if (CanUseFiniteLibCall && DAG.getLibInfo().has(LibFunc_pow_finite))
       Results.push_back(ExpandFPLibCall(Node, RTLIB::POW_FINITE_F32,
                                         RTLIB::POW_FINITE_F64,
                                         RTLIB::POW_FINITE_F80,
                                         RTLIB::POW_FINITE_F128,
                                         RTLIB::POW_FINITE_PPCF128));

llvm/lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp

return TLI.makeLibCall(DAG, GetFPLibCall(N->getValueType(0),
                                            RTLIB::POW_F128,
                                            RTLIB::POW_PPCF128),
                          NVT, Ops, CallOptions, SDLoc(N)).first;
 }
 
 SDValue DAGTypeLegalizer::SoftenFloatRes_FPOWI(SDNode *N) {
   assert(N->getOperand(1).getValueType() == MVT::i32 &&
          "Unsupported power type!");
+  RTLIB::Libcall LC = GetFPLibCall(N->getValueType(0),
+                                   RTLIB::POWI_F32,
+                                   RTLIB::POWI_F64,
+                                   RTLIB::POWI_F80,
+                                   RTLIB::POWI_F128,
+                                   RTLIB::POWI_PPCF128);
+  if (!TLI.getLibcallName(LC)) {
+    // Some targets don't have a powi libcall; use pow instead.
+    // FIXME: Implement this if some target needs it.
+    DAG.getContext()->emitError("Don't know how to soften fpowi to fpow");
+    return DAG.getUNDEF(N->getValueType(0));
+  }
+
   EVT NVT = TLI.getTypeToTransformTo(*DAG.getContext(), N->getValueType(0));
   SDValue Ops[2] = { GetSoftenedFloat(N->getOperand(0)), N->getOperand(1) };
   TargetLowering::MakeLibCallOptions CallOptions;
   EVT OpsVT[2] = { N->getOperand(0).getValueType(),
                    N->getOperand(1).getValueType() };
   CallOptions.setTypeListBeforeSoften(OpsVT, N->getValueType(0), true);
-  return TLI.makeLibCall(DAG, GetFPLibCall(N->getValueType(0),
-                                           RTLIB::POWI_F32,
-                                           RTLIB::POWI_F64,
-                                           RTLIB::POWI_F80,
-                                           RTLIB::POWI_F128,
-                                           RTLIB::POWI_PPCF128),
-                         NVT, Ops, CallOptions, SDLoc(N)).first;
+  return TLI.makeLibCall(DAG, LC, NVT, Ops, CallOptions, SDLoc(N)).first;
 }
 
 SDValue DAGTypeLegalizer::SoftenFloatRes_FREM(SDNode *N) {
   EVT NVT = TLI.getTypeToTransformTo(*DAG.getContext(), N->getValueType(0));
   SDValue Ops[2] = { GetSoftenedFloat(N->getOperand(0)),
                      GetSoftenedFloat(N->getOperand(1)) };
   TargetLowering::MakeLibCallOptions CallOptions;
   EVT OpsVT[2] = { N->getOperand(0).getValueType(),

llvm/lib/Target/AArch64/AArch64ISelLowering.cpp

if (getLibcallName(RTLIB::SINCOS_STRET_F32) != nullptr &&
     // Issue __sincos_stret if available.
     setOperationAction(ISD::FSINCOS, MVT::f64, Custom);
     setOperationAction(ISD::FSINCOS, MVT::f32, Custom);
   } else {
     setOperationAction(ISD::FSINCOS, MVT::f64, Expand);
     setOperationAction(ISD::FSINCOS, MVT::f32, Expand);
   }
 
+  if (Subtarget->getTargetTriple().isOSMSVCRT()) {
+    // MSVCRT doesn't have powi; fall back to pow
+    setLibcallName(RTLIB::POWI_F32, nullptr);
+    setLibcallName(RTLIB::POWI_F64, nullptr);
+  }
+
   // Make floating-point constants legal for the large code model, so they don't
   // become loads from the constant pool.
   if (Subtarget->isTargetMachO() && TM.getCodeModel() == CodeModel::Large) {
     setOperationAction(ISD::ConstantFP, MVT::f32, Legal);
     setOperationAction(ISD::ConstantFP, MVT::f64, Legal);
   }
 
   // AArch64 does not have floating-point extending loads, i1 sign-extending

llvm/lib/Target/ARM/ARMISelLowering.cpp

if (Subtarget->isTargetAEABI() || Subtarget->isTargetAndroid() ||
     setOperationAction(ISD::UDIVREM, MVT::i32, Custom);
     setOperationAction(ISD::SDIVREM, MVT::i64, Custom);
     setOperationAction(ISD::UDIVREM, MVT::i64, Custom);
   } else {
     setOperationAction(ISD::SDIVREM, MVT::i32, Expand);
     setOperationAction(ISD::UDIVREM, MVT::i32, Expand);
   }
 
-  if (Subtarget->isTargetWindows() && Subtarget->getTargetTriple().isOSMSVCRT())
-    for (auto &VT : {MVT::f32, MVT::f64})
-      setOperationAction(ISD::FPOWI, VT, Custom);
+  if (Subtarget->getTargetTriple().isOSMSVCRT()) {
+    // MSVCRT doesn't have powi; fall back to pow
+    setLibcallName(RTLIB::POWI_F32, nullptr);
+    setLibcallName(RTLIB::POWI_F64, nullptr);
+  }
 
   setOperationAction(ISD::GlobalAddress, MVT::i32,   Custom);
   setOperationAction(ISD::ConstantPool,  MVT::i32,   Custom);
   setOperationAction(ISD::GlobalTLSAddress, MVT::i32, Custom);
   setOperationAction(ISD::BlockAddress, MVT::i32, Custom);
 
   setOperationAction(ISD::TRAP, MVT::Other, Legal);
   setOperationAction(ISD::DEBUGTRAP, MVT::Other, Legal);
Results.push_back(
       DAG.getTargetExtractSubreg(isBigEndian ? ARM::gsub_1 : ARM::gsub_0,
                                  SDLoc(N), MVT::i32, SDValue(CmpSwap, 0)));
   Results.push_back(
       DAG.getTargetExtractSubreg(isBigEndian ? ARM::gsub_0 : ARM::gsub_1,
                                  SDLoc(N), MVT::i32, SDValue(CmpSwap, 0)));
   Results.push_back(SDValue(CmpSwap, 2));
 }
 
-static SDValue LowerFPOWI(SDValue Op, const ARMSubtarget &Subtarget,
-                          SelectionDAG &DAG) {
-  const auto &TLI = DAG.getTargetLoweringInfo();
-
-  assert(Subtarget.getTargetTriple().isOSMSVCRT() &&
-         "Custom lowering is MSVCRT specific!");
-
-  SDLoc dl(Op);
-  SDValue Val = Op.getOperand(0);
-  MVT Ty = Val->getSimpleValueType(0);
-  SDValue Exponent = DAG.getNode(ISD::SINT_TO_FP, dl, Ty, Op.getOperand(1));
-  SDValue Callee = DAG.getExternalSymbol(Ty == MVT::f32 ? "powf" : "pow",
-                                         TLI.getPointerTy(DAG.getDataLayout()));
-
-  TargetLowering::ArgListTy Args;
-  TargetLowering::ArgListEntry Entry;
-
-  Entry.Node = Val;
-  Entry.Ty = Val.getValueType().getTypeForEVT(*DAG.getContext());
-  Entry.IsZExt = true;
-  Args.push_back(Entry);
-
-  Entry.Node = Exponent;
-  Entry.Ty = Exponent.getValueType().getTypeForEVT(*DAG.getContext());
-  Entry.IsZExt = true;
-  Args.push_back(Entry);
-
-  Type *LCRTy = Val.getValueType().getTypeForEVT(*DAG.getContext());
-
-  // In the in-chain to the call is the entry node  If we are emitting a
-  // tailcall, the chain will be mutated if the node has a non-entry input
-  // chain.
-  SDValue InChain = DAG.getEntryNode();
-  SDValue TCChain = InChain;
-
-  const Function &F = DAG.getMachineFunction().getFunction();
-  bool IsTC = TLI.isInTailCallPosition(DAG, Op.getNode(), TCChain) &&
-              F.getReturnType() == LCRTy;
-  if (IsTC)
-    InChain = TCChain;
-
-  TargetLowering::CallLoweringInfo CLI(DAG);
-  CLI.setDebugLoc(dl)
-      .setChain(InChain)
-      .setCallee(CallingConv::ARM_AAPCS_VFP, LCRTy, Callee, std::move(Args))
-      .setTailCall(IsTC);
-  std::pair<SDValue, SDValue> CI = TLI.LowerCallTo(CLI);
-
-  // Return the chain (the DAG root) if it is a tail call
-  return !CI.second.getNode() ? DAG.getRoot() : CI.first;
-}
-
 SDValue ARMTargetLowering::LowerOperation(SDValue Op, SelectionDAG &DAG) const {
   LLVM_DEBUG(dbgs() << "Lowering node: "; Op.dump());
   switch (Op.getOpcode()) {
   default: llvm_unreachable("Don't know how to custom lower this!");
   case ISD::WRITE_REGISTER: return LowerWRITE_REGISTER(Op, DAG);
   case ISD::ConstantPool: return LowerConstantPool(Op, DAG);
   case ISD::BlockAddress:  return LowerBlockAddress(Op, DAG);
   case ISD::GlobalAddress: return LowerGlobalAddress(Op, DAG);
SDValue ARMTargetLowering::LowerOperation(SDValue Op, SelectionDAG &DAG) const {
   case ISD::SDIVREM:
   case ISD::UDIVREM:       return LowerDivRem(Op, DAG);
   case ISD::DYNAMIC_STACKALLOC:
     if (Subtarget->isTargetWindows())
       return LowerDYNAMIC_STACKALLOC(Op, DAG);
     llvm_unreachable("Don't know how to custom lower this!");
   case ISD::FP_ROUND: return LowerFP_ROUND(Op, DAG);
   case ISD::FP_EXTEND: return LowerFP_EXTEND(Op, DAG);
-  case ISD::FPOWI: return LowerFPOWI(Op, *Subtarget, DAG);
   case ARMISD::WIN__DBZCHK: return SDValue();
   }
 }
 
 static void ReplaceLongIntrinsic(SDNode *N, SmallVectorImpl<SDValue> &Results,
                                  SelectionDAG &DAG) {
   unsigned IntNo = cast<ConstantSDNode>(N->getOperand(0))->getZExtValue();
   unsigned Opc = 0;

llvm/lib/Target/X86/X86ISelLowering.cpp

if (Subtarget.isTargetWindowsMSVC() ||
     setLibcallName(RTLIB::MUL_I64, "_allmul");
     setLibcallCallingConv(RTLIB::SDIV_I64, CallingConv::X86_StdCall);
     setLibcallCallingConv(RTLIB::UDIV_I64, CallingConv::X86_StdCall);
     setLibcallCallingConv(RTLIB::SREM_I64, CallingConv::X86_StdCall);
     setLibcallCallingConv(RTLIB::UREM_I64, CallingConv::X86_StdCall);
     setLibcallCallingConv(RTLIB::MUL_I64, CallingConv::X86_StdCall);
   }
 
+  if (Subtarget.getTargetTriple().isOSMSVCRT()) {
+    // MSVCRT doesn't have powi; fall back to pow
+    setLibcallName(RTLIB::POWI_F32, nullptr);
+    setLibcallName(RTLIB::POWI_F64, nullptr);
+  }
+
   if (Subtarget.isTargetDarwin()) {
     // Darwin should use _setjmp/_longjmp instead of setjmp/longjmp.
     setUseUnderscoreSetJmp(false);
     setUseUnderscoreLongJmp(false);
   } else if (Subtarget.isTargetWindowsGNU()) {
     // MS runtime is weird: it exports _setjmp, but longjmp!
     setUseUnderscoreSetJmp(true);
     setUseUnderscoreLongJmp(false);

llvm/test/CodeGen/AArch64/powi-windows.ll

+; RUN: llc -mtriple aarch64-windows < %s | FileCheck %s
+
+declare double @llvm.powi.f64(double, i32)
+declare float @llvm.powi.f32(float, i32)
+
+define double @d(double %d, i32 %i) {
+entry:
+  %0 = tail call double @llvm.powi.f64(double %d, i32 %i)
+  ret double %0
+}
+
+; CHECK-LABEL: d:
+; CHECK: scvtf d1, w0
+; CHECK-NEXT: b pow
+
+define float @f(float %f, i32 %i) {
+entry:
+  %0 = tail call float @llvm.powi.f32(float %f, i32 %i)
+  ret float %0
+}
+
+; CHECK-LABEL: f:
+; CHECK: scvtf s1, w0
+; CHECK-NEXT: b powf
+
+define float @g(double %d, i32 %i) {
+entry:
+  %0 = tail call double @llvm.powi.f64(double %d, i32 %i)
+  %conv = fptrunc double %0 to float
+  ret float %conv
+}
+
+; CHECK-LABEL: g:
+; CHECK: scvtf d1, w0
+; CHECK-NEXT: bl pow
+
+define double @h(float %f, i32 %i) {
+entry:
+  %0 = tail call float @llvm.powi.f32(float %f, i32 %i)
+  %conv = fpext float %0 to double
+  ret double %conv
+}
+
+; CHECK-LABEL: h:
+; CHECK: scvtf s1, w0
+; CHECK-NEXT: bl powf

llvm/test/CodeGen/X86/powi-windows.ll

+; RUN: llc -mtriple x86_64-windows < %s | FileCheck %s
+
+declare double @llvm.powi.f64(double, i32)
+declare float @llvm.powi.f32(float, i32)
+
+define double @d(double %d, i32 %i) {
+entry:
+  %0 = tail call double @llvm.powi.f64(double %d, i32 %i)
+  ret double %0
+}
+
+; CHECK-LABEL: d:
+; CHECK: cvtsi2sd %edx, %xmm1
+; CHECK-NEXT: jmp pow
+
+define float @f(float %f, i32 %i) {
+entry:
+  %0 = tail call float @llvm.powi.f32(float %f, i32 %i)
+  ret float %0
+}
+
+; CHECK-LABEL: f:
+; CHECK: cvtsi2ss %edx, %xmm1
+; CHECK-NEXT: jmp powf
+
+define float @g(double %d, i32 %i) {
+entry:
+  %0 = tail call double @llvm.powi.f64(double %d, i32 %i)
+  %conv = fptrunc double %0 to float
+  ret float %conv
+}
+
+; CHECK-LABEL: g:
+; CHECK: cvtsi2sd %edx, %xmm1
+; CHECK-NEXT: callq pow
+
+define double @h(float %f, i32 %i) {
+entry:
+  %0 = tail call float @llvm.powi.f32(float %f, i32 %i)
+  %conv = fpext float %0 to double
+  ret double %conv
+}
+
+; CHECK-LABEL: h:
+; CHECK: cvtsi2ss %edx, %xmm1
+; CHECK-NEXT: callq powf