[X86][AVX512DQ] Use packed instructions for scalar FP<->i64 conversions on 32-bit targets (PR31630)

[craig.topper]

Summary

As i64 types are not legal on 32-bit targets, insert these into a suitable zero vector and use the packed vXi64<->FP conversion instructions instead.

[craig.topper]

Why a DAG combine and not Custom type legalization?

lib/Target/X86/X86ISelLowering.cpp

37030

Why bother with the AVX512 check here if we're just going to check DQI inside?

37057

Do we need the zeros or could this be scalar_to_vector?

[craig.topper]

Commandeering from Simon to finish this up.

[craig.topper]

Move everything into type legalization lowering.

[delena]

lib/Target/X86/X86ISelLowering.cpp

68

This case is the same for SINT and UINT. May be put them in a function?

75

VecVT = MVT::getVectorVT(VT, NumElts);

79

Why do you need to insert into zero vector? Can you insert to undef?

test/CodeGen/X86/scalar-fp-to-i64.ll

83

Can the memory operand be folded here?
VCVTTPD2UQQ ymm1 {k1}{z},ymm2/m256/m64bcst

[craig.topper]

lib/Target/X86/X86ISelLowering.cpp

79

I think so. I asked the same question before I commandeered it. It's probably no worse than the widening with undef we do for v2f32 legalization.

test/CodeGen/X86/scalar-fp-to-i64.ll

83

We'd have to detect the load and the possibilty of folding it during this lowering code. Or we'd have to use undef for the upper elts and add a DAG combine to turn insert into undef into a broadcast if its foldable.

[craig.topper]

Create helper function to share between SINT_TO_FP and UINT_TO_FP. Use scalar_to_vector instead of inserting into a zero vector.

[delena]

In D43441#1098588, @craig.topper wrote:

Or we'd have to use undef for the upper elts and add a DAG combine to turn insert into undef into a broadcast if its foldable.

I think it is simpler than load folding in td. You can open a bugzilla ticket for possible optimization.

[RKSimon]

lib/Target/X86/X86ISelLowering.cpp

79

In the original patch I was just trying to be very sure there wasn't anything in the other source elements that could cause fp exceptions/overflow flags etc.

[DavidKreitzer]

Could you please add full context to the patch, Craig?

lib/Target/X86/X86ISelLowering.cpp

79

I think the only possible side effect from the other source element being undef is raising "inexact". Do we care?

16048

I suspect we also want to use this vector sequence for unsigned i64 conventions on 64-bit.

[craig.topper]

Full context

[DavidKreitzer]

Thanks, Craig.

lib/Target/X86/X86ISelLowering.cpp

79

Forget what I wrote. I was thinking of the INT-->FP case. For the FP-->INT case, why wouldn't we need to worry about raising spurious exceptions?

Also, I'm probably missing something, but it looks like this code is expected to kick in for both 32-bit and 64-bit and both signed and unsigned FP-->i64. Is that intentional? For the 64-bit signed case, I would think we would prefer CVTTSx2SI.

[craig.topper]

Use zeros for other elements for fp->int to avoid spurious invalid exceptions. Since these sorts of scalar conversions are common there's a chance someone could notice the spurious exceptions and complain. We seem to have generated the same code in the test cases either way. Since we ultimatley had to load two i32s into an xmm register. We need use a movd/movss that 0 the upper bits anyway.

Also addeded an assert for !Subtarget.is64Bit() to the fp->int conversion. That code is in ReplaceNodeResults is a hook for the type legalizer to legalize the result type so we should only get there when i64 isn't legal.

[DavidKreitzer]

Looks good, Craig. And thanks for the explanation on the 64-bit cases. I had forgotten about the VCVTTSx2USI & VCVTUSI2Sx instructions, which we appear to be generating already.

[RKSimon]

LGTM - thanks Craig!

lib/Target/X86/X86ISelLowering.cpp

 #include <algorithm>
 #include <bitset>
 #include <cctype>
 #include <numeric>
 using namespace llvm;
 
 #define DEBUG_TYPE "x86-isel"
 
 STATISTIC(NumTailCalls, "Number of tail calls");
 
 static cl::opt<bool> ExperimentalVectorWideningLegalization(
     "x86-experimental-vector-widening-legalization", cl::init(false),
     cl::desc("Enable an experimental vector type legalization through widening "
              "rather than promotion."),
     cl::Hidden);
 
 static cl::opt<int> ExperimentalPrefLoopAlignment(
     "x86-experimental-pref-loop-alignment", cl::init(4),
     cl::desc("Sets the preferable loop alignment for experiments "
              "(the last x86-experimental-pref-loop-alignment bits"
              " of the loop header PC will be 0)."),
     cl::Hidden);
 
 static cl::opt<bool> MulConstantOptimization(
     "mul-constant-optimization", cl::init(true),
     cl::desc("Replace 'mul x, Const' with more effective instructions like "
              "SHIFT, LEA, etc."),
     cl::Hidden);

   setTargetDAGCombine(ISD::ZERO_EXTEND);
   setTargetDAGCombine(ISD::ANY_EXTEND);
   setTargetDAGCombine(ISD::SIGN_EXTEND);
   setTargetDAGCombine(ISD::SIGN_EXTEND_INREG);
   setTargetDAGCombine(ISD::SIGN_EXTEND_VECTOR_INREG);
   setTargetDAGCombine(ISD::ZERO_EXTEND_VECTOR_INREG);
   setTargetDAGCombine(ISD::SINT_TO_FP);
   setTargetDAGCombine(ISD::UINT_TO_FP);
+  setTargetDAGCombine(ISD::FP_TO_SINT);
+  setTargetDAGCombine(ISD::FP_TO_UINT);
   setTargetDAGCombine(ISD::SETCC);
   setTargetDAGCombine(ISD::MUL);
   setTargetDAGCombine(ISD::XOR);
   setTargetDAGCombine(ISD::MSCATTER);
   setTargetDAGCombine(ISD::MGATHER);
 
   computeRegisterProperties(Subtarget.getRegisterInfo());
 
static SDValue lowerUINT_TO_FP_v2i32(SDValue Op, SelectionDAG &DAG,
 
   // Legalize to v4i32 type.
   N0 = DAG.getNode(ISD::CONCAT_VECTORS, DL, MVT::v4i32, N0,
                    DAG.getUNDEF(MVT::v2i32));
 
   if (Subtarget.hasAVX512())
     return DAG.getNode(X86ISD::CVTUI2P, DL, MVT::v2f64, N0);
 
   // Same implementation as VectorLegalizer::ExpandUINT_TO_FLOAT,
   // but using v2i32 to v2f64 with X86ISD::CVTSI2P.
   SDValue HalfWord = DAG.getConstant(16, DL, MVT::v4i32);
   SDValue HalfWordMask = DAG.getConstant(0x0000FFFF, DL, MVT::v4i32);
 
   // Two to the power of half-word-size.
   SDValue TWOHW = DAG.getConstantFP(1 << 16, DL, MVT::v2f64);
 
   // Clear upper part of LO, lower HI.
SDValue NewAnd = DAG.getNode(ISD::AND, DL, IntVT,
                                  N->getOperand(0)->getOperand(0), MaskConst);
     SDValue Res = DAG.getBitcast(VT, NewAnd);
     return Res;
   }
 
   return SDValue();
 }
 
+static SDValue combineIntToFP_AVX512(SDNode *N, SelectionDAG &DAG,
+                                     const X86Subtarget &Subtarget) {
+  SDValue Op0 = N->getOperand(0);
+  EVT VT = N->getValueType(0);
+  EVT InVT = Op0.getValueType();
+
+  // AVX512DQ 32-bit targets don't have i64 scalar float/double conversions,
+  // so perform it entirely as a vector.
+  if (!Subtarget.hasDQI() || InVT != MVT::i64 ||
+      (VT != MVT::f32 && VT != MVT::f64) ||
+      DAG.getTargetLoweringInfo().isTypeLegal(InVT))
+    return SDValue();
+
+  SDLoc DL(N);
+  MVT VecInVT = Subtarget.hasVLX() ? MVT::v4i64 : MVT::v8i64;
+  EVT VecVT =
+      EVT::getVectorVT(*DAG.getContext(), VT, VecInVT.getVectorNumElements());
+
+  SDValue ZeroIdx = DAG.getIntPtrConstant(0, DL);
+  SDValue InVec =
+      DAG.getNode(ISD::INSERT_VECTOR_ELT, DL, VecInVT,
+                  getZeroVector(VecInVT, Subtarget, DAG, DL), Op0, ZeroIdx);
+  SDValue CvtVec = DAG.getNode(N->getOpcode(), SDLoc(N), VecVT, InVec);
+  return DAG.getNode(ISD::EXTRACT_VECTOR_ELT, DL, VT, CvtVec, ZeroIdx);
+}
+
 static SDValue combineUIntToFP(SDNode *N, SelectionDAG &DAG,
                                const X86Subtarget &Subtarget) {
   SDValue Op0 = N->getOperand(0);
   EVT VT = N->getValueType(0);
   EVT InVT = Op0.getValueType();
 
   // UINT_TO_FP(vXi1) -> SINT_TO_FP(SEXT(vXi1 to vXi32))
   // UINT_TO_FP(vXi8) -> SINT_TO_FP(ZEXT(vXi8 to vXi32))
static SDValue combineUIntToFP(SDNode *N, SelectionDAG &DAG,
   }
 
   // Since UINT_TO_FP is legal (it's marked custom), dag combiner won't
   // optimize it to a SINT_TO_FP when the sign bit is known zero. Perform
   // the optimization here.
   if (DAG.SignBitIsZero(Op0))
     return DAG.getNode(ISD::SINT_TO_FP, SDLoc(N), VT, Op0);
 
+  // Attempt AVX512-only combines.
+  if (Subtarget.hasAVX512())
+    if (SDValue Cvt = combineIntToFP_AVX512(N, DAG, Subtarget))
+      return Cvt;
+
   return SDValue();
 }
 
 static SDValue combineSIntToFP(SDNode *N, SelectionDAG &DAG,
                                const X86Subtarget &Subtarget) {
   // First try to optimize away the conversion entirely when it's
   // conditionally from a constant. Vectors only.
   if (SDValue Res = combineVectorCompareAndMaskUnaryOp(N, DAG))
if (!Ld->isVolatile() && !VT.isVector() &&
         ISD::isNON_EXTLoad(Op0.getNode()) && Op0.hasOneUse() &&
         !Subtarget.is64Bit() && LdVT == MVT::i64) {
       SDValue FILDChain = Subtarget.getTargetLowering()->BuildFILD(
           SDValue(N, 0), LdVT, Ld->getChain(), Op0, DAG);
       DAG.ReplaceAllUsesOfValueWith(Op0.getValue(1), FILDChain.getValue(1));
       return FILDChain;
     }
   }
+
+  // Attempt AVX512-only combines.
+  if (Subtarget.hasAVX512())
+    if (SDValue Cvt = combineIntToFP_AVX512(N, DAG, Subtarget))
+      return Cvt;
+
   return SDValue();
 }
 
+static SDValue combineFPToInt(SDNode *N, SelectionDAG &DAG,
+                              const X86Subtarget &Subtarget) {
+  SDValue Op0 = N->getOperand(0);
+  EVT VT = N->getValueType(0);
+  EVT InVT = Op0.getValueType();
+
+  // AVX512DQ 32-bit targets don't have float/double scalar i64 conversions,
+  // so perform it entirely as a vector.
+  if (!Subtarget.hasDQI() || VT != MVT::i64 ||
+      (InVT != MVT::f32 && InVT != MVT::f64) ||
+      DAG.getTargetLoweringInfo().isTypeLegal(VT))
+    return SDValue();
+
+  SDLoc DL(N);
+  MVT VecVT = Subtarget.hasVLX() ? MVT::v4i64 : MVT::v8i64;
+  MVT VecInVT =
+      MVT::getVectorVT(InVT.getSimpleVT(), VecVT.getVectorNumElements());
+
+  SDValue ZeroIdx = DAG.getIntPtrConstant(0, DL);
+  SDValue InVec =
+      DAG.getNode(ISD::INSERT_VECTOR_ELT, DL, VecInVT,
+                  getZeroVector(VecInVT, Subtarget, DAG, DL), Op0, ZeroIdx);
+  SDValue CvtVec = DAG.getNode(N->getOpcode(), SDLoc(N), VecVT, InVec);
+  return DAG.getNode(ISD::EXTRACT_VECTOR_ELT, DL, VT, CvtVec, ZeroIdx);
+}
+
 static SDValue combineSBB(SDNode *N, SelectionDAG &DAG) {
   if (SDValue Flags = combineCarryThroughADD(N->getOperand(2))) {
     MVT VT = N->getSimpleValueType(0);
     SDVTList VTs = DAG.getVTList(VT, MVT::i32);
     return DAG.getNode(X86ISD::SBB, SDLoc(N), VTs,
                        N->getOperand(0), N->getOperand(1),
                        Flags);
   }
SDValue X86TargetLowering::PerformDAGCombine(SDNode *N,
   case ISD::OR:             return combineOr(N, DAG, DCI, Subtarget);
   case ISD::XOR:            return combineXor(N, DAG, DCI, Subtarget);
   case ISD::LOAD:           return combineLoad(N, DAG, DCI, Subtarget);
   case ISD::MLOAD:          return combineMaskedLoad(N, DAG, DCI, Subtarget);
   case ISD::STORE:          return combineStore(N, DAG, Subtarget);
   case ISD::MSTORE:         return combineMaskedStore(N, DAG, Subtarget);
   case ISD::SINT_TO_FP:     return combineSIntToFP(N, DAG, Subtarget);
   case ISD::UINT_TO_FP:     return combineUIntToFP(N, DAG, Subtarget);
+  case ISD::FP_TO_SINT:
+  case ISD::FP_TO_UINT:     return combineFPToInt(N, DAG, Subtarget);
   case ISD::FADD:
   case ISD::FSUB:           return combineFaddFsub(N, DAG, Subtarget);
   case ISD::FNEG:           return combineFneg(N, DAG, Subtarget);
   case ISD::TRUNCATE:       return combineTruncate(N, DAG, Subtarget);
   case X86ISD::ANDNP:       return combineAndnp(N, DAG, DCI, Subtarget);
   case X86ISD::FAND:        return combineFAnd(N, DAG, Subtarget);
   case X86ISD::FANDN:       return combineFAndn(N, DAG, Subtarget);
   case X86ISD::FXOR:

test/CodeGen/X86/avx512-regcall-NoMask.ll

; LINUXOSX64-NEXT:    retq
   %x8 = fadd <32 x float> %c2, %x7
   ret <32 x float> %x8
 }
 
 ; Test regcall when passing/retrieving mixed types
 define x86_regcallcc i32 @test_argRetMixTypes(double, float, i8 signext, i32, i64, i16 signext, i32*) #0 {
 ; X32-LABEL: test_argRetMixTypes:
 ; X32:       # %bb.0:
-; X32-NEXT:    pushl %ebp
-; X32-NEXT:    movl %esp, %ebp
-; X32-NEXT:    andl $-8, %esp
-; X32-NEXT:    subl $16, %esp
-; X32-NEXT:    vmovd %edx, %xmm2
-; X32-NEXT:    vpinsrd $1, %edi, %xmm2, %xmm2
-; X32-NEXT:    movl 8(%ebp), %edx
+; X32-NEXT:    pushl %ebx
+; X32-NEXT:    movl {{[0-9]+}}(%esp), %ebx
 ; X32-NEXT:    vcvtss2sd %xmm1, %xmm1, %xmm1
 ; X32-NEXT:    vaddsd %xmm0, %xmm1, %xmm0
-; X32-NEXT:    vcvtsi2sdl %eax, %xmm3, %xmm1
+; X32-NEXT:    vcvtsi2sdl %eax, %xmm2, %xmm1
 ; X32-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
-; X32-NEXT:    vcvtsi2sdl %ecx, %xmm3, %xmm1
+; X32-NEXT:    vcvtsi2sdl %ecx, %xmm2, %xmm1
+; X32-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
+; X32-NEXT:    vmovd %edi, %xmm1
+; X32-NEXT:    vmovd %edx, %xmm2
+; X32-NEXT:    vpunpckldq {{.*#+}} xmm1 = xmm2[0],xmm1[0],xmm2[1],xmm1[1]
+; X32-NEXT:    vmovq {{.*#+}} xmm1 = xmm1[0],zero
+; X32-NEXT:    vmovdqa %xmm1, %xmm1
+; X32-NEXT:    vcvtqq2pd %ymm1, %ymm1
 ; X32-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
-; X32-NEXT:    vmovq %xmm2, {{[0-9]+}}(%esp)
-; X32-NEXT:    fildll {{[0-9]+}}(%esp)
-; X32-NEXT:    fstpl (%esp)
-; X32-NEXT:    vaddsd (%esp), %xmm0, %xmm0
 ; X32-NEXT:    vcvtsi2sdl %esi, %xmm3, %xmm1
 ; X32-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
-; X32-NEXT:    vcvtsi2sdl (%edx), %xmm3, %xmm1
+; X32-NEXT:    vcvtsi2sdl (%ebx), %xmm3, %xmm1
 ; X32-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
 ; X32-NEXT:    vcvttsd2si %xmm0, %eax
-; X32-NEXT:    movl %ebp, %esp
-; X32-NEXT:    popl %ebp
+; X32-NEXT:    popl %ebx
+; X32-NEXT:    vzeroupper
 ; X32-NEXT:    retl
 ;
 ; WIN64-LABEL: test_argRetMixTypes:
 ; WIN64:       # %bb.0:
 ; WIN64-NEXT:    vcvtss2sd %xmm1, %xmm1, %xmm1
 ; WIN64-NEXT:    vaddsd %xmm0, %xmm1, %xmm0
 ; WIN64-NEXT:    vcvtsi2sdl %eax, %xmm2, %xmm1
 ; WIN64-NEXT:    vaddsd %xmm1, %xmm0, %xmm0

test/CodeGen/X86/scalar-fp-to-i64.ll

 ; Otherwise the sequence will involve an FP subtract (fsub, subss or subsd),
 ; and a truncating conversion (cvtts[ds]2si, fisttp, or fnstcw+fist).  When
 ; both a subtract and fnstcw are needed, they can occur in either order.
 ;
 ; The interesting subtargets are AVX512F (vcvtts[ds]2usi), SSE3 (fisttp),
 ; SSE2 (cvtts[ds]2si) and vanilla X87 (fnstcw+fist, 32-bit only).
 
 define i64 @f_to_u64(float %a) nounwind {
-; AVX512_32_WIN-LABEL: f_to_u64:
-; AVX512_32_WIN:       # %bb.0:
-; AVX512_32_WIN-NEXT:    pushl %ebp
-; AVX512_32_WIN-NEXT:    movl %esp, %ebp
-; AVX512_32_WIN-NEXT:    andl $-8, %esp
-; AVX512_32_WIN-NEXT:    subl $16, %esp
-; AVX512_32_WIN-NEXT:    vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
-; AVX512_32_WIN-NEXT:    vmovss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; AVX512_32_WIN-NEXT:    vcmpltss %xmm1, %xmm0, %k1
-; AVX512_32_WIN-NEXT:    vsubss %xmm1, %xmm0, %xmm2
-; AVX512_32_WIN-NEXT:    vmovss %xmm0, %xmm0, %xmm2 {%k1}
-; AVX512_32_WIN-NEXT:    vmovss %xmm2, {{[0-9]+}}(%esp)
-; AVX512_32_WIN-NEXT:    flds {{[0-9]+}}(%esp)
-; AVX512_32_WIN-NEXT:    fisttpll (%esp)
-; AVX512_32_WIN-NEXT:    xorl %edx, %edx
-; AVX512_32_WIN-NEXT:    vucomiss %xmm0, %xmm1
-; AVX512_32_WIN-NEXT:    setbe %dl
-; AVX512_32_WIN-NEXT:    shll $31, %edx
-; AVX512_32_WIN-NEXT:    xorl {{[0-9]+}}(%esp), %edx
-; AVX512_32_WIN-NEXT:    movl (%esp), %eax
-; AVX512_32_WIN-NEXT:    movl %ebp, %esp
-; AVX512_32_WIN-NEXT:    popl %ebp
-; AVX512_32_WIN-NEXT:    retl
-;
-; AVX512_32_LIN-LABEL: f_to_u64:
-; AVX512_32_LIN:       # %bb.0:
-; AVX512_32_LIN-NEXT:    subl $20, %esp
-; AVX512_32_LIN-NEXT:    vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
-; AVX512_32_LIN-NEXT:    vmovss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; AVX512_32_LIN-NEXT:    vcmpltss %xmm1, %xmm0, %k1
-; AVX512_32_LIN-NEXT:    vsubss %xmm1, %xmm0, %xmm2
-; AVX512_32_LIN-NEXT:    vmovss %xmm0, %xmm0, %xmm2 {%k1}
-; AVX512_32_LIN-NEXT:    vmovss %xmm2, {{[0-9]+}}(%esp)
-; AVX512_32_LIN-NEXT:    flds {{[0-9]+}}(%esp)
-; AVX512_32_LIN-NEXT:    fisttpll (%esp)
-; AVX512_32_LIN-NEXT:    xorl %edx, %edx
-; AVX512_32_LIN-NEXT:    vucomiss %xmm0, %xmm1
-; AVX512_32_LIN-NEXT:    setbe %dl
-; AVX512_32_LIN-NEXT:    shll $31, %edx
-; AVX512_32_LIN-NEXT:    xorl {{[0-9]+}}(%esp), %edx
-; AVX512_32_LIN-NEXT:    movl (%esp), %eax
-; AVX512_32_LIN-NEXT:    addl $20, %esp
-; AVX512_32_LIN-NEXT:    retl
+; AVX512DQ_32_WIN-LABEL: f_to_u64:
+; AVX512DQ_32_WIN:       # %bb.0:
+; AVX512DQ_32_WIN-NEXT:    vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; AVX512DQ_32_WIN-NEXT:    vcvttps2uqq %ymm0, %zmm0
+; AVX512DQ_32_WIN-NEXT:    vmovd %xmm0, %eax
+; AVX512DQ_32_WIN-NEXT:    vpextrd $1, %xmm0, %edx
+; AVX512DQ_32_WIN-NEXT:    vzeroupper
+; AVX512DQ_32_WIN-NEXT:    retl
+;
+; AVX512DQ_32_LIN-LABEL: f_to_u64:
+; AVX512DQ_32_LIN:       # %bb.0:
+; AVX512DQ_32_LIN-NEXT:    vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; AVX512DQ_32_LIN-NEXT:    vcvttps2uqq %ymm0, %zmm0
+; AVX512DQ_32_LIN-NEXT:    vmovd %xmm0, %eax
+; AVX512DQ_32_LIN-NEXT:    vpextrd $1, %xmm0, %edx
+; AVX512DQ_32_LIN-NEXT:    vzeroupper
+; AVX512DQ_32_LIN-NEXT:    retl
 ;
 ; AVX512_64-LABEL: f_to_u64:
 ; AVX512_64:       # %bb.0:
 ; AVX512_64-NEXT:    vcvttss2usi %xmm0, %rax
 ; AVX512_64-NEXT:    retq
 ;
+; AVX512F_32_WIN-LABEL: f_to_u64:
+; AVX512F_32_WIN:       # %bb.0:
+; AVX512F_32_WIN-NEXT:    pushl %ebp
+; AVX512F_32_WIN-NEXT:    movl %esp, %ebp
+; AVX512F_32_WIN-NEXT:    andl $-8, %esp
+; AVX512F_32_WIN-NEXT:    subl $16, %esp
+; AVX512F_32_WIN-NEXT:    vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; AVX512F_32_WIN-NEXT:    vmovss {{.*#+}} xmm1 = mem[0],zero,zero,zero
+; AVX512F_32_WIN-NEXT:    vcmpltss %xmm1, %xmm0, %k1
+; AVX512F_32_WIN-NEXT:    vsubss %xmm1, %xmm0, %xmm2
+; AVX512F_32_WIN-NEXT:    vmovss %xmm0, %xmm0, %xmm2 {%k1}
+; AVX512F_32_WIN-NEXT:    vmovss %xmm2, {{[0-9]+}}(%esp)
+; AVX512F_32_WIN-NEXT:    flds {{[0-9]+}}(%esp)
+; AVX512F_32_WIN-NEXT:    fisttpll (%esp)
+; AVX512F_32_WIN-NEXT:    xorl %edx, %edx
+; AVX512F_32_WIN-NEXT:    vucomiss %xmm0, %xmm1
+; AVX512F_32_WIN-NEXT:    setbe %dl
+; AVX512F_32_WIN-NEXT:    shll $31, %edx
+; AVX512F_32_WIN-NEXT:    xorl {{[0-9]+}}(%esp), %edx
+; AVX512F_32_WIN-NEXT:    movl (%esp), %eax
+; AVX512F_32_WIN-NEXT:    movl %ebp, %esp
+; AVX512F_32_WIN-NEXT:    popl %ebp
+; AVX512F_32_WIN-NEXT:    retl
+;
+; AVX512F_32_LIN-LABEL: f_to_u64:
+; AVX512F_32_LIN:       # %bb.0:
+; AVX512F_32_LIN-NEXT:    subl $20, %esp
+; AVX512F_32_LIN-NEXT:    vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; AVX512F_32_LIN-NEXT:    vmovss {{.*#+}} xmm1 = mem[0],zero,zero,zero
+; AVX512F_32_LIN-NEXT:    vcmpltss %xmm1, %xmm0, %k1
+; AVX512F_32_LIN-NEXT:    vsubss %xmm1, %xmm0, %xmm2
+; AVX512F_32_LIN-NEXT:    vmovss %xmm0, %xmm0, %xmm2 {%k1}
+; AVX512F_32_LIN-NEXT:    vmovss %xmm2, {{[0-9]+}}(%esp)
+; AVX512F_32_LIN-NEXT:    flds {{[0-9]+}}(%esp)
+; AVX512F_32_LIN-NEXT:    fisttpll (%esp)
+; AVX512F_32_LIN-NEXT:    xorl %edx, %edx
+; AVX512F_32_LIN-NEXT:    vucomiss %xmm0, %xmm1
+; AVX512F_32_LIN-NEXT:    setbe %dl
+; AVX512F_32_LIN-NEXT:    shll $31, %edx
+; AVX512F_32_LIN-NEXT:    xorl {{[0-9]+}}(%esp), %edx
+; AVX512F_32_LIN-NEXT:    movl (%esp), %eax
+; AVX512F_32_LIN-NEXT:    addl $20, %esp
+; AVX512F_32_LIN-NEXT:    retl
+;
 ; SSE3_32_WIN-LABEL: f_to_u64:
 ; SSE3_32_WIN:       # %bb.0:
 ; SSE3_32_WIN-NEXT:    pushl %ebp
 ; SSE3_32_WIN-NEXT:    movl %esp, %ebp
 ; SSE3_32_WIN-NEXT:    andl $-8, %esp
 ; SSE3_32_WIN-NEXT:    subl $16, %esp
 ; SSE3_32_WIN-NEXT:    movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
 ; SSE3_32_WIN-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero

 ; X87_LIN-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X87_LIN-NEXT:    addl $20, %esp
 ; X87_LIN-NEXT:    retl
   %r = fptoui float %a to i64
   ret i64 %r
 }
 
 define i64 @f_to_s64(float %a) nounwind {
-; AVX512_32_WIN-LABEL: f_to_s64:
-; AVX512_32_WIN:       # %bb.0:
-; AVX512_32_WIN-NEXT:    pushl %ebp
-; AVX512_32_WIN-NEXT:    movl %esp, %ebp
-; AVX512_32_WIN-NEXT:    andl $-8, %esp
-; AVX512_32_WIN-NEXT:    subl $16, %esp
-; AVX512_32_WIN-NEXT:    vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
-; AVX512_32_WIN-NEXT:    vmovss %xmm0, {{[0-9]+}}(%esp)
-; AVX512_32_WIN-NEXT:    flds {{[0-9]+}}(%esp)
-; AVX512_32_WIN-NEXT:    fisttpll (%esp)
-; AVX512_32_WIN-NEXT:    movl (%esp), %eax
-; AVX512_32_WIN-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; AVX512_32_WIN-NEXT:    movl %ebp, %esp
-; AVX512_32_WIN-NEXT:    popl %ebp
-; AVX512_32_WIN-NEXT:    retl
-;
-; AVX512_32_LIN-LABEL: f_to_s64:
-; AVX512_32_LIN:       # %bb.0:
-; AVX512_32_LIN-NEXT:    subl $20, %esp
-; AVX512_32_LIN-NEXT:    vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
-; AVX512_32_LIN-NEXT:    vmovss %xmm0, {{[0-9]+}}(%esp)
-; AVX512_32_LIN-NEXT:    flds {{[0-9]+}}(%esp)
-; AVX512_32_LIN-NEXT:    fisttpll (%esp)
-; AVX512_32_LIN-NEXT:    movl (%esp), %eax
-; AVX512_32_LIN-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; AVX512_32_LIN-NEXT:    addl $20, %esp
-; AVX512_32_LIN-NEXT:    retl
+; AVX512DQ_32_WIN-LABEL: f_to_s64:
+; AVX512DQ_32_WIN:       # %bb.0:
+; AVX512DQ_32_WIN-NEXT:    vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; AVX512DQ_32_WIN-NEXT:    vcvttps2qq %ymm0, %zmm0
+; AVX512DQ_32_WIN-NEXT:    vmovd %xmm0, %eax
+; AVX512DQ_32_WIN-NEXT:    vpextrd $1, %xmm0, %edx
+; AVX512DQ_32_WIN-NEXT:    vzeroupper
+; AVX512DQ_32_WIN-NEXT:    retl
+;
+; AVX512DQ_32_LIN-LABEL: f_to_s64:
+; AVX512DQ_32_LIN:       # %bb.0:
+; AVX512DQ_32_LIN-NEXT:    vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; AVX512DQ_32_LIN-NEXT:    vcvttps2qq %ymm0, %zmm0
+; AVX512DQ_32_LIN-NEXT:    vmovd %xmm0, %eax
+; AVX512DQ_32_LIN-NEXT:    vpextrd $1, %xmm0, %edx
+; AVX512DQ_32_LIN-NEXT:    vzeroupper
+; AVX512DQ_32_LIN-NEXT:    retl
 ;
 ; AVX512_64-LABEL: f_to_s64:
 ; AVX512_64:       # %bb.0:
 ; AVX512_64-NEXT:    vcvttss2si %xmm0, %rax
 ; AVX512_64-NEXT:    retq
 ;
+; AVX512F_32_WIN-LABEL: f_to_s64:
+; AVX512F_32_WIN:       # %bb.0:
+; AVX512F_32_WIN-NEXT:    pushl %ebp
+; AVX512F_32_WIN-NEXT:    movl %esp, %ebp
+; AVX512F_32_WIN-NEXT:    andl $-8, %esp
+; AVX512F_32_WIN-NEXT:    subl $16, %esp
+; AVX512F_32_WIN-NEXT:    vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; AVX512F_32_WIN-NEXT:    vmovss %xmm0, {{[0-9]+}}(%esp)
+; AVX512F_32_WIN-NEXT:    flds {{[0-9]+}}(%esp)
+; AVX512F_32_WIN-NEXT:    fisttpll (%esp)
+; AVX512F_32_WIN-NEXT:    movl (%esp), %eax
+; AVX512F_32_WIN-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; AVX512F_32_WIN-NEXT:    movl %ebp, %esp
+; AVX512F_32_WIN-NEXT:    popl %ebp
+; AVX512F_32_WIN-NEXT:    retl
+;
+; AVX512F_32_LIN-LABEL: f_to_s64:
+; AVX512F_32_LIN:       # %bb.0:
+; AVX512F_32_LIN-NEXT:    subl $20, %esp
+; AVX512F_32_LIN-NEXT:    vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; AVX512F_32_LIN-NEXT:    vmovss %xmm0, {{[0-9]+}}(%esp)
+; AVX512F_32_LIN-NEXT:    flds {{[0-9]+}}(%esp)
+; AVX512F_32_LIN-NEXT:    fisttpll (%esp)
+; AVX512F_32_LIN-NEXT:    movl (%esp), %eax
+; AVX512F_32_LIN-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; AVX512F_32_LIN-NEXT:    addl $20, %esp
+; AVX512F_32_LIN-NEXT:    retl
+;
 ; SSE3_32_WIN-LABEL: f_to_s64:
 ; SSE3_32_WIN:       # %bb.0:
 ; SSE3_32_WIN-NEXT:    pushl %ebp
 ; SSE3_32_WIN-NEXT:    movl %esp, %ebp
 ; SSE3_32_WIN-NEXT:    andl $-8, %esp
 ; SSE3_32_WIN-NEXT:    subl $16, %esp
 ; SSE3_32_WIN-NEXT:    movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
 ; SSE3_32_WIN-NEXT:    movss %xmm0, {{[0-9]+}}(%esp)

 ; X87_LIN-NEXT:    movl {{[0-9]+}}(%esp), %edx
 ; X87_LIN-NEXT:    addl $20, %esp
 ; X87_LIN-NEXT:    retl
   %r = fptosi float %a to i64
   ret i64 %r
 }
 
 define i64 @d_to_u64(double %a) nounwind {
-; AVX512_32_WIN-LABEL: d_to_u64:
-; AVX512_32_WIN:       # %bb.0:
-; AVX512_32_WIN-NEXT:    pushl %ebp
-; AVX512_32_WIN-NEXT:    movl %esp, %ebp
-; AVX512_32_WIN-NEXT:    andl $-8, %esp
-; AVX512_32_WIN-NEXT:    subl $16, %esp
-; AVX512_32_WIN-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
-; AVX512_32_WIN-NEXT:    vmovsd {{.*#+}} xmm1 = mem[0],zero
-; AVX512_32_WIN-NEXT:    vcmpltsd %xmm1, %xmm0, %k1
-; AVX512_32_WIN-NEXT:    vsubsd %xmm1, %xmm0, %xmm2
-; AVX512_32_WIN-NEXT:    vmovsd %xmm0, %xmm0, %xmm2 {%k1}
-; AVX512_32_WIN-NEXT:    vmovsd %xmm2, {{[0-9]+}}(%esp)
-; AVX512_32_WIN-NEXT:    fldl {{[0-9]+}}(%esp)
-; AVX512_32_WIN-NEXT:    fisttpll (%esp)
-; AVX512_32_WIN-NEXT:    xorl %edx, %edx
-; AVX512_32_WIN-NEXT:    vucomisd %xmm0, %xmm1
-; AVX512_32_WIN-NEXT:    setbe %dl
-; AVX512_32_WIN-NEXT:    shll $31, %edx
-; AVX512_32_WIN-NEXT:    xorl {{[0-9]+}}(%esp), %edx
-; AVX512_32_WIN-NEXT:    movl (%esp), %eax
-; AVX512_32_WIN-NEXT:    movl %ebp, %esp
-; AVX512_32_WIN-NEXT:    popl %ebp
-; AVX512_32_WIN-NEXT:    retl
-;
-; AVX512_32_LIN-LABEL: d_to_u64:
-; AVX512_32_LIN:       # %bb.0:
-; AVX512_32_LIN-NEXT:    subl $20, %esp
-; AVX512_32_LIN-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
-; AVX512_32_LIN-NEXT:    vmovsd {{.*#+}} xmm1 = mem[0],zero
-; AVX512_32_LIN-NEXT:    vcmpltsd %xmm1, %xmm0, %k1
-; AVX512_32_LIN-NEXT:    vsubsd %xmm1, %xmm0, %xmm2
-; AVX512_32_LIN-NEXT:    vmovsd %xmm0, %xmm0, %xmm2 {%k1}
-; AVX512_32_LIN-NEXT:    vmovsd %xmm2, {{[0-9]+}}(%esp)
-; AVX512_32_LIN-NEXT:    fldl {{[0-9]+}}(%esp)
-; AVX512_32_LIN-NEXT:    fisttpll (%esp)
-; AVX512_32_LIN-NEXT:    xorl %edx, %edx
-; AVX512_32_LIN-NEXT:    vucomisd %xmm0, %xmm1
-; AVX512_32_LIN-NEXT:    setbe %dl
-; AVX512_32_LIN-NEXT:    shll $31, %edx
-; AVX512_32_LIN-NEXT:    xorl {{[0-9]+}}(%esp), %edx
-; AVX512_32_LIN-NEXT:    movl (%esp), %eax
-; AVX512_32_LIN-NEXT:    addl $20, %esp
-; AVX512_32_LIN-NEXT:    retl
+; AVX512DQ_32_WIN-LABEL: d_to_u64:
+; AVX512DQ_32_WIN:       # %bb.0:
+; AVX512DQ_32_WIN-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
+; AVX512DQ_32_WIN-NEXT:    vcvttpd2uqq %zmm0, %zmm0
+; AVX512DQ_32_WIN-NEXT:    vmovd %xmm0, %eax
+; AVX512DQ_32_WIN-NEXT:    vpextrd $1, %xmm0, %edx
+; AVX512DQ_32_WIN-NEXT:    vzeroupper
+; AVX512DQ_32_WIN-NEXT:    retl
+;
+; AVX512DQ_32_LIN-LABEL: d_to_u64:
+; AVX512DQ_32_LIN:       # %bb.0:
+; AVX512DQ_32_LIN-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
+; AVX512DQ_32_LIN-NEXT:    vcvttpd2uqq %zmm0, %zmm0
+; AVX512DQ_32_LIN-NEXT:    vmovd %xmm0, %eax
+; AVX512DQ_32_LIN-NEXT:    vpextrd $1, %xmm0, %edx
+; AVX512DQ_32_LIN-NEXT:    vzeroupper
+; AVX512DQ_32_LIN-NEXT:    retl
 ;
 ; AVX512_64-LABEL: d_to_u64:
 ; AVX512_64:       # %bb.0:
 ; AVX512_64-NEXT:    vcvttsd2usi %xmm0, %rax
 ; AVX512_64-NEXT:    retq
 ;
+; AVX512F_32_WIN-LABEL: d_to_u64:
+; AVX512F_32_WIN:       # %bb.0:
+; AVX512F_32_WIN-NEXT:    pushl %ebp
+; AVX512F_32_WIN-NEXT:    movl %esp, %ebp
+; AVX512F_32_WIN-NEXT:    andl $-8, %esp
+; AVX512F_32_WIN-NEXT:    subl $16, %esp
+; AVX512F_32_WIN-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
+; AVX512F_32_WIN-NEXT:    vmovsd {{.*#+}} xmm1 = mem[0],zero
+; AVX512F_32_WIN-NEXT:    vcmpltsd %xmm1, %xmm0, %k1
+; AVX512F_32_WIN-NEXT:    vsubsd %xmm1, %xmm0, %xmm2
+; AVX512F_32_WIN-NEXT:    vmovsd %xmm0, %xmm0, %xmm2 {%k1}
+; AVX512F_32_WIN-NEXT:    vmovsd %xmm2, {{[0-9]+}}(%esp)
+; AVX512F_32_WIN-NEXT:    fldl {{[0-9]+}}(%esp)
+; AVX512F_32_WIN-NEXT:    fisttpll (%esp)
+; AVX512F_32_WIN-NEXT:    xorl %edx, %edx
+; AVX512F_32_WIN-NEXT:    vucomisd %xmm0, %xmm1
+; AVX512F_32_WIN-NEXT:    setbe %dl
+; AVX512F_32_WIN-NEXT:    shll $31, %edx
+; AVX512F_32_WIN-NEXT:    xorl {{[0-9]+}}(%esp), %edx
+; AVX512F_32_WIN-NEXT:    movl (%esp), %eax
+; AVX512F_32_WIN-NEXT:    movl %ebp, %esp
+; AVX512F_32_WIN-NEXT:    popl %ebp
+; AVX512F_32_WIN-NEXT:    retl
+;
+; AVX512F_32_LIN-LABEL: d_to_u64:
+; AVX512F_32_LIN:       # %bb.0:
+; AVX512F_32_LIN-NEXT:    subl $20, %esp
+; AVX512F_32_LIN-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
+; AVX512F_32_LIN-NEXT:    vmovsd {{.*#+}} xmm1 = mem[0],zero
+; AVX512F_32_LIN-NEXT:    vcmpltsd %xmm1, %xmm0, %k1
+; AVX512F_32_LIN-NEXT:    vsubsd %xmm1, %xmm0, %xmm2
+; AVX512F_32_LIN-NEXT:    vmovsd %xmm0, %xmm0, %xmm2 {%k1}
+; AVX512F_32_LIN-NEXT:    vmovsd %xmm2, {{[0-9]+}}(%esp)
+; AVX512F_32_LIN-NEXT:    fldl {{[0-9]+}}(%esp)
+; AVX512F_32_LIN-NEXT:    fisttpll (%esp)
+; AVX512F_32_LIN-NEXT:    xorl %edx, %edx
+; AVX512F_32_LIN-NEXT:    vucomisd %xmm0, %xmm1
+; AVX512F_32_LIN-NEXT:    setbe %dl
+; AVX512F_32_LIN-NEXT:    shll $31, %edx
+; AVX512F_32_LIN-NEXT:    xorl {{[0-9]+}}(%esp), %edx
+; AVX512F_32_LIN-NEXT:    movl (%esp), %eax
+; AVX512F_32_LIN-NEXT:    addl $20, %esp
+; AVX512F_32_LIN-NEXT:    retl
+;
 ; SSE3_32_WIN-LABEL: d_to_u64:
 ; SSE3_32_WIN:       # %bb.0:
 ; SSE3_32_WIN-NEXT:    pushl %ebp
 ; SSE3_32_WIN-NEXT:    movl %esp, %ebp
 ; SSE3_32_WIN-NEXT:    andl $-8, %esp
 ; SSE3_32_WIN-NEXT:    subl $16, %esp
 ; SSE3_32_WIN-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
 ; SSE3_32_WIN-NEXT:    movsd {{.*#+}} xmm1 = mem[0],zero

 ; X87_LIN-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X87_LIN-NEXT:    addl $20, %esp
 ; X87_LIN-NEXT:    retl
   %r = fptoui double %a to i64
   ret i64 %r
 }
 
 define i64 @d_to_s64(double %a) nounwind {
-; AVX512_32_WIN-LABEL: d_to_s64:
-; AVX512_32_WIN:       # %bb.0:
-; AVX512_32_WIN-NEXT:    pushl %ebp
-; AVX512_32_WIN-NEXT:    movl %esp, %ebp
-; AVX512_32_WIN-NEXT:    andl $-8, %esp
-; AVX512_32_WIN-NEXT:    subl $16, %esp
-; AVX512_32_WIN-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
-; AVX512_32_WIN-NEXT:    vmovsd %xmm0, {{[0-9]+}}(%esp)
-; AVX512_32_WIN-NEXT:    fldl {{[0-9]+}}(%esp)
-; AVX512_32_WIN-NEXT:    fisttpll (%esp)
-; AVX512_32_WIN-NEXT:    movl (%esp), %eax
-; AVX512_32_WIN-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; AVX512_32_WIN-NEXT:    movl %ebp, %esp
-; AVX512_32_WIN-NEXT:    popl %ebp
-; AVX512_32_WIN-NEXT:    retl
-;
-; AVX512_32_LIN-LABEL: d_to_s64:
-; AVX512_32_LIN:       # %bb.0:
-; AVX512_32_LIN-NEXT:    subl $20, %esp
-; AVX512_32_LIN-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
-; AVX512_32_LIN-NEXT:    vmovsd %xmm0, {{[0-9]+}}(%esp)
-; AVX512_32_LIN-NEXT:    fldl {{[0-9]+}}(%esp)
-; AVX512_32_LIN-NEXT:    fisttpll (%esp)
-; AVX512_32_LIN-NEXT:    movl (%esp), %eax
-; AVX512_32_LIN-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; AVX512_32_LIN-NEXT:    addl $20, %esp
-; AVX512_32_LIN-NEXT:    retl
+; AVX512DQ_32_WIN-LABEL: d_to_s64:
+; AVX512DQ_32_WIN:       # %bb.0:
+; AVX512DQ_32_WIN-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
+; AVX512DQ_32_WIN-NEXT:    vcvttpd2qq %zmm0, %zmm0
+; AVX512DQ_32_WIN-NEXT:    vmovd %xmm0, %eax
+; AVX512DQ_32_WIN-NEXT:    vpextrd $1, %xmm0, %edx
+; AVX512DQ_32_WIN-NEXT:    vzeroupper
+; AVX512DQ_32_WIN-NEXT:    retl
+;
+; AVX512DQ_32_LIN-LABEL: d_to_s64:
+; AVX512DQ_32_LIN:       # %bb.0:
+; AVX512DQ_32_LIN-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
+; AVX512DQ_32_LIN-NEXT:    vcvttpd2qq %zmm0, %zmm0
+; AVX512DQ_32_LIN-NEXT:    vmovd %xmm0, %eax
+; AVX512DQ_32_LIN-NEXT:    vpextrd $1, %xmm0, %edx
+; AVX512DQ_32_LIN-NEXT:    vzeroupper
+; AVX512DQ_32_LIN-NEXT:    retl
 ;
 ; AVX512_64-LABEL: d_to_s64:
 ; AVX512_64:       # %bb.0:
 ; AVX512_64-NEXT:    vcvttsd2si %xmm0, %rax
 ; AVX512_64-NEXT:    retq
 ;
+; AVX512F_32_WIN-LABEL: d_to_s64:
+; AVX512F_32_WIN:       # %bb.0:
+; AVX512F_32_WIN-NEXT:    pushl %ebp
+; AVX512F_32_WIN-NEXT:    movl %esp, %ebp
+; AVX512F_32_WIN-NEXT:    andl $-8, %esp
+; AVX512F_32_WIN-NEXT:    subl $16, %esp
+; AVX512F_32_WIN-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
+; AVX512F_32_WIN-NEXT:    vmovsd %xmm0, {{[0-9]+}}(%esp)
+; AVX512F_32_WIN-NEXT:    fldl {{[0-9]+}}(%esp)
+; AVX512F_32_WIN-NEXT:    fisttpll (%esp)
+; AVX512F_32_WIN-NEXT:    movl (%esp), %eax
+; AVX512F_32_WIN-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; AVX512F_32_WIN-NEXT:    movl %ebp, %esp
+; AVX512F_32_WIN-NEXT:    popl %ebp
+; AVX512F_32_WIN-NEXT:    retl
+;
+; AVX512F_32_LIN-LABEL: d_to_s64:
+; AVX512F_32_LIN:       # %bb.0:
+; AVX512F_32_LIN-NEXT:    subl $20, %esp
+; AVX512F_32_LIN-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
+; AVX512F_32_LIN-NEXT:    vmovsd %xmm0, {{[0-9]+}}(%esp)
+; AVX512F_32_LIN-NEXT:    fldl {{[0-9]+}}(%esp)
+; AVX512F_32_LIN-NEXT:    fisttpll (%esp)
+; AVX512F_32_LIN-NEXT:    movl (%esp), %eax
+; AVX512F_32_LIN-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; AVX512F_32_LIN-NEXT:    addl $20, %esp
+; AVX512F_32_LIN-NEXT:    retl
+;
 ; SSE3_32_WIN-LABEL: d_to_s64:
 ; SSE3_32_WIN:       # %bb.0:
 ; SSE3_32_WIN-NEXT:    pushl %ebp
 ; SSE3_32_WIN-NEXT:    movl %esp, %ebp
 ; SSE3_32_WIN-NEXT:    andl $-8, %esp
 ; SSE3_32_WIN-NEXT:    subl $16, %esp
 ; SSE3_32_WIN-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
 ; SSE3_32_WIN-NEXT:    movsd %xmm0, {{[0-9]+}}(%esp)

test/CodeGen/X86/scalar-int-to-fp.ll

 ; CHECK64-NEXT:    movl %edi, -{{[0-9]+}}(%rsp)
 ; CHECK64-NEXT:    fildl -{{[0-9]+}}(%rsp)
 ; CHECK64-NEXT:    retq
   %r = sitofp i32 %a to x86_fp80
   ret x86_fp80 %r
 }
 
 define float @u64_to_f(i64 %a) nounwind {
-; AVX512_32-LABEL: u64_to_f:
-; AVX512_32:       # %bb.0:
-; AVX512_32-NEXT:    pushl %ebp
-; AVX512_32-NEXT:    movl %esp, %ebp
-; AVX512_32-NEXT:    andl $-8, %esp
-; AVX512_32-NEXT:    subl $16, %esp
-; AVX512_32-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
-; AVX512_32-NEXT:    vmovlps %xmm0, {{[0-9]+}}(%esp)
-; AVX512_32-NEXT:    xorl %eax, %eax
-; AVX512_32-NEXT:    cmpl $0, 12(%ebp)
-; AVX512_32-NEXT:    setns %al
-; AVX512_32-NEXT:    fildll {{[0-9]+}}(%esp)
-; AVX512_32-NEXT:    fadds {{\.LCPI.*}}(,%eax,4)
-; AVX512_32-NEXT:    fstps {{[0-9]+}}(%esp)
-; AVX512_32-NEXT:    vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
-; AVX512_32-NEXT:    vmovss %xmm0, (%esp)
-; AVX512_32-NEXT:    flds (%esp)
-; AVX512_32-NEXT:    movl %ebp, %esp
-; AVX512_32-NEXT:    popl %ebp
-; AVX512_32-NEXT:    retl
+; AVX512DQ_32-LABEL: u64_to_f:
+; AVX512DQ_32:       # %bb.0:
+; AVX512DQ_32-NEXT:    pushl %eax
+; AVX512DQ_32-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
+; AVX512DQ_32-NEXT:    vcvtuqq2ps %zmm0, %ymm0
+; AVX512DQ_32-NEXT:    vmovss %xmm0, (%esp)
+; AVX512DQ_32-NEXT:    flds (%esp)
+; AVX512DQ_32-NEXT:    popl %eax
+; AVX512DQ_32-NEXT:    vzeroupper
+; AVX512DQ_32-NEXT:    retl
 ;
 ; AVX512_64-LABEL: u64_to_f:
 ; AVX512_64:       # %bb.0:
 ; AVX512_64-NEXT:    vcvtusi2ssq %rdi, %xmm0, %xmm0
 ; AVX512_64-NEXT:    retq
 ;
+; AVX512F_32-LABEL: u64_to_f:
+; AVX512F_32:       # %bb.0:
+; AVX512F_32-NEXT:    pushl %ebp
+; AVX512F_32-NEXT:    movl %esp, %ebp
+; AVX512F_32-NEXT:    andl $-8, %esp
+; AVX512F_32-NEXT:    subl $16, %esp
+; AVX512F_32-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
+; AVX512F_32-NEXT:    vmovlps %xmm0, {{[0-9]+}}(%esp)
+; AVX512F_32-NEXT:    xorl %eax, %eax
+; AVX512F_32-NEXT:    cmpl $0, 12(%ebp)
+; AVX512F_32-NEXT:    setns %al
+; AVX512F_32-NEXT:    fildll {{[0-9]+}}(%esp)
+; AVX512F_32-NEXT:    fadds {{\.LCPI.*}}(,%eax,4)
+; AVX512F_32-NEXT:    fstps {{[0-9]+}}(%esp)
+; AVX512F_32-NEXT:    vmovss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; AVX512F_32-NEXT:    vmovss %xmm0, (%esp)
+; AVX512F_32-NEXT:    flds (%esp)
+; AVX512F_32-NEXT:    movl %ebp, %esp
+; AVX512F_32-NEXT:    popl %ebp
+; AVX512F_32-NEXT:    retl
+;
 ; SSE2_32-LABEL: u64_to_f:
 ; SSE2_32:       # %bb.0:
 ; SSE2_32-NEXT:    pushl %ebp
 ; SSE2_32-NEXT:    movl %esp, %ebp
 ; SSE2_32-NEXT:    andl $-8, %esp
 ; SSE2_32-NEXT:    subl $16, %esp
 ; SSE2_32-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
 ; SSE2_32-NEXT:    movlps %xmm0, {{[0-9]+}}(%esp)

 ; X87-NEXT:    movl %ebp, %esp
 ; X87-NEXT:    popl %ebp
 ; X87-NEXT:    retl
   %r = uitofp i64 %a to float
   ret float %r
 }
 
 define float @s64_to_f(i64 %a) nounwind {
-; AVX512_32-LABEL: s64_to_f:
-; AVX512_32:       # %bb.0:
-; AVX512_32-NEXT:    pushl %eax
-; AVX512_32-NEXT:    fildll {{[0-9]+}}(%esp)
-; AVX512_32-NEXT:    fstps (%esp)
-; AVX512_32-NEXT:    flds (%esp)
-; AVX512_32-NEXT:    popl %eax
-; AVX512_32-NEXT:    retl
+; AVX512DQ_32-LABEL: s64_to_f:
+; AVX512DQ_32:       # %bb.0:
+; AVX512DQ_32-NEXT:    pushl %eax
+; AVX512DQ_32-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
+; AVX512DQ_32-NEXT:    vcvtqq2ps %zmm0, %ymm0
+; AVX512DQ_32-NEXT:    vmovss %xmm0, (%esp)
+; AVX512DQ_32-NEXT:    flds (%esp)
+; AVX512DQ_32-NEXT:    popl %eax
+; AVX512DQ_32-NEXT:    vzeroupper
+; AVX512DQ_32-NEXT:    retl
 ;
 ; AVX512_64-LABEL: s64_to_f:
 ; AVX512_64:       # %bb.0:
 ; AVX512_64-NEXT:    vcvtsi2ssq %rdi, %xmm0, %xmm0
 ; AVX512_64-NEXT:    retq
 ;
+; AVX512F_32-LABEL: s64_to_f:
+; AVX512F_32:       # %bb.0:
+; AVX512F_32-NEXT:    pushl %eax
+; AVX512F_32-NEXT:    fildll {{[0-9]+}}(%esp)
+; AVX512F_32-NEXT:    fstps (%esp)
+; AVX512F_32-NEXT:    flds (%esp)
+; AVX512F_32-NEXT:    popl %eax
+; AVX512F_32-NEXT:    retl
+;
 ; SSE2_32-LABEL: s64_to_f:
 ; SSE2_32:       # %bb.0:
 ; SSE2_32-NEXT:    pushl %eax
 ; SSE2_32-NEXT:    fildll {{[0-9]+}}(%esp)
 ; SSE2_32-NEXT:    fstps (%esp)
 ; SSE2_32-NEXT:    flds (%esp)
 ; SSE2_32-NEXT:    popl %eax
 ; SSE2_32-NEXT:    retl
 ;
 ; SSE2_64-LABEL: s64_to_f:
 ; SSE2_64:       # %bb.0:
 ; SSE2_64-NEXT:    cvtsi2ssq %rdi, %xmm0
 ; SSE2_64-NEXT:    retq
 ;
 ; X87-LABEL: s64_to_f:
 ; X87:       # %bb.0:
 ; X87-NEXT:    fildll {{[0-9]+}}(%esp)
 ; X87-NEXT:    retl
   %r = sitofp i64 %a to float
   ret float %r
 }
 
 define float @s64_to_f_2(i64 %a) nounwind {
-; AVX512_32-LABEL: s64_to_f_2:
-; AVX512_32:       # %bb.0:
-; AVX512_32-NEXT:    pushl %ebp
-; AVX512_32-NEXT:    movl %esp, %ebp
-; AVX512_32-NEXT:    andl $-8, %esp
-; AVX512_32-NEXT:    subl $16, %esp
-; AVX512_32-NEXT:    movl 8(%ebp), %eax
-; AVX512_32-NEXT:    movl 12(%ebp), %ecx
-; AVX512_32-NEXT:    addl $5, %eax
-; AVX512_32-NEXT:    adcl $0, %ecx
-; AVX512_32-NEXT:    vmovd %eax, %xmm0
-; AVX512_32-NEXT:    vpinsrd $1, %ecx, %xmm0, %xmm0
-; AVX512_32-NEXT:    vmovq %xmm0, {{[0-9]+}}(%esp)
-; AVX512_32-NEXT:    fildll {{[0-9]+}}(%esp)
-; AVX512_32-NEXT:    fstps {{[0-9]+}}(%esp)
-; AVX512_32-NEXT:    flds {{[0-9]+}}(%esp)
-; AVX512_32-NEXT:    movl %ebp, %esp
-; AVX512_32-NEXT:    popl %ebp
-; AVX512_32-NEXT:    retl
+; AVX512DQ_32-LABEL: s64_to_f_2:
+; AVX512DQ_32:       # %bb.0:
+; AVX512DQ_32-NEXT:    pushl %eax
+; AVX512DQ_32-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; AVX512DQ_32-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; AVX512DQ_32-NEXT:    addl $5, %eax
+; AVX512DQ_32-NEXT:    adcl $0, %ecx
+; AVX512DQ_32-NEXT:    vmovd %ecx, %xmm0
+; AVX512DQ_32-NEXT:    vmovd %eax, %xmm1
+; AVX512DQ_32-NEXT:    vpunpckldq {{.*#+}} xmm0 = xmm1[0],xmm0[0],xmm1[1],xmm0[1]
+; AVX512DQ_32-NEXT:    vmovq {{.*#+}} xmm0 = xmm0[0],zero
+; AVX512DQ_32-NEXT:    vmovdqa %xmm0, %xmm0
+; AVX512DQ_32-NEXT:    vcvtqq2ps %zmm0, %ymm0
+; AVX512DQ_32-NEXT:    vmovss %xmm0, (%esp)
+; AVX512DQ_32-NEXT:    flds (%esp)
+; AVX512DQ_32-NEXT:    popl %eax
+; AVX512DQ_32-NEXT:    vzeroupper
+; AVX512DQ_32-NEXT:    retl
 ;
 ; AVX512_64-LABEL: s64_to_f_2:
 ; AVX512_64:       # %bb.0:
 ; AVX512_64-NEXT:    addq $5, %rdi
 ; AVX512_64-NEXT:    vcvtsi2ssq %rdi, %xmm0, %xmm0
 ; AVX512_64-NEXT:    retq
 ;
+; AVX512F_32-LABEL: s64_to_f_2:
+; AVX512F_32:       # %bb.0:
+; AVX512F_32-NEXT:    pushl %ebp
+; AVX512F_32-NEXT:    movl %esp, %ebp
+; AVX512F_32-NEXT:    andl $-8, %esp
+; AVX512F_32-NEXT:    subl $16, %esp
+; AVX512F_32-NEXT:    movl 8(%ebp), %eax
+; AVX512F_32-NEXT:    movl 12(%ebp), %ecx
+; AVX512F_32-NEXT:    addl $5, %eax
+; AVX512F_32-NEXT:    adcl $0, %ecx
+; AVX512F_32-NEXT:    vmovd %eax, %xmm0
+; AVX512F_32-NEXT:    vpinsrd $1, %ecx, %xmm0, %xmm0
+; AVX512F_32-NEXT:    vmovq %xmm0, {{[0-9]+}}(%esp)
+; AVX512F_32-NEXT:    fildll {{[0-9]+}}(%esp)
+; AVX512F_32-NEXT:    fstps {{[0-9]+}}(%esp)
+; AVX512F_32-NEXT:    flds {{[0-9]+}}(%esp)
+; AVX512F_32-NEXT:    movl %ebp, %esp
+; AVX512F_32-NEXT:    popl %ebp
+; AVX512F_32-NEXT:    retl
+;
 ; SSE2_32-LABEL: s64_to_f_2:
 ; SSE2_32:       # %bb.0:
 ; SSE2_32-NEXT:    pushl %ebp
 ; SSE2_32-NEXT:    movl %esp, %ebp
 ; SSE2_32-NEXT:    andl $-8, %esp
 ; SSE2_32-NEXT:    subl $16, %esp
 ; SSE2_32-NEXT:    movl 8(%ebp), %eax
 ; SSE2_32-NEXT:    movl 12(%ebp), %ecx

 ; X87-NEXT:    popl %ebp
 ; X87-NEXT:    retl
   %a1 = add i64 %a, 5
   %r = sitofp i64 %a1 to float
   ret float %r
 }
 
 define double @u64_to_d(i64 %a) nounwind {
-; AVX512_32-LABEL: u64_to_d:
-; AVX512_32:       # %bb.0:
-; AVX512_32-NEXT:    pushl %ebp
-; AVX512_32-NEXT:    movl %esp, %ebp
-; AVX512_32-NEXT:    andl $-8, %esp
-; AVX512_32-NEXT:    subl $8, %esp
-; AVX512_32-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
-; AVX512_32-NEXT:    vunpcklps {{.*#+}} xmm0 = xmm0[0],mem[0],xmm0[1],mem[1]
-; AVX512_32-NEXT:    vsubpd {{\.LCPI.*}}, %xmm0, %xmm0
-; AVX512_32-NEXT:    vhaddpd %xmm0, %xmm0, %xmm0
-; AVX512_32-NEXT:    vmovlpd %xmm0, (%esp)
-; AVX512_32-NEXT:    fldl (%esp)
-; AVX512_32-NEXT:    movl %ebp, %esp
-; AVX512_32-NEXT:    popl %ebp
-; AVX512_32-NEXT:    retl
+; AVX512DQ_32-LABEL: u64_to_d:
+; AVX512DQ_32:       # %bb.0:
+; AVX512DQ_32-NEXT:    pushl %ebp
+; AVX512DQ_32-NEXT:    movl %esp, %ebp
+; AVX512DQ_32-NEXT:    andl $-8, %esp
+; AVX512DQ_32-NEXT:    subl $8, %esp
+; AVX512DQ_32-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
+; AVX512DQ_32-NEXT:    vcvtuqq2pd %zmm0, %zmm0
+; AVX512DQ_32-NEXT:    vmovlps %xmm0, (%esp)
+; AVX512DQ_32-NEXT:    fldl (%esp)
+; AVX512DQ_32-NEXT:    movl %ebp, %esp
+; AVX512DQ_32-NEXT:    popl %ebp
+; AVX512DQ_32-NEXT:    vzeroupper
+; AVX512DQ_32-NEXT:    retl
 ;
 ; AVX512_64-LABEL: u64_to_d:
 ; AVX512_64:       # %bb.0:
 ; AVX512_64-NEXT:    vcvtusi2sdq %rdi, %xmm0, %xmm0
 ; AVX512_64-NEXT:    retq
 ;
+; AVX512F_32-LABEL: u64_to_d:
+; AVX512F_32:       # %bb.0:
+; AVX512F_32-NEXT:    pushl %ebp
+; AVX512F_32-NEXT:    movl %esp, %ebp
+; AVX512F_32-NEXT:    andl $-8, %esp
+; AVX512F_32-NEXT:    subl $8, %esp
+; AVX512F_32-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
+; AVX512F_32-NEXT:    vunpcklps {{.*#+}} xmm0 = xmm0[0],mem[0],xmm0[1],mem[1]
+; AVX512F_32-NEXT:    vsubpd {{\.LCPI.*}}, %xmm0, %xmm0
+; AVX512F_32-NEXT:    vhaddpd %xmm0, %xmm0, %xmm0
+; AVX512F_32-NEXT:    vmovlpd %xmm0, (%esp)
+; AVX512F_32-NEXT:    fldl (%esp)
+; AVX512F_32-NEXT:    movl %ebp, %esp
+; AVX512F_32-NEXT:    popl %ebp
+; AVX512F_32-NEXT:    retl
+;
 ; SSE2_32-LABEL: u64_to_d:
 ; SSE2_32:       # %bb.0:
 ; SSE2_32-NEXT:    pushl %ebp
 ; SSE2_32-NEXT:    movl %esp, %ebp
 ; SSE2_32-NEXT:    andl $-8, %esp
 ; SSE2_32-NEXT:    subl $8, %esp
 ; SSE2_32-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
 ; SSE2_32-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],mem[0],xmm0[1],mem[1]

 ; X87-NEXT:    movl %ebp, %esp
 ; X87-NEXT:    popl %ebp
 ; X87-NEXT:    retl
   %r = uitofp i64 %a to double
   ret double %r
 }
 
 define double @s64_to_d(i64 %a) nounwind {
-; AVX512_32-LABEL: s64_to_d:
-; AVX512_32:       # %bb.0:
-; AVX512_32-NEXT:    pushl %ebp
-; AVX512_32-NEXT:    movl %esp, %ebp
-; AVX512_32-NEXT:    andl $-8, %esp
-; AVX512_32-NEXT:    subl $8, %esp
-; AVX512_32-NEXT:    fildll 8(%ebp)
-; AVX512_32-NEXT:    fstpl (%esp)
-; AVX512_32-NEXT:    fldl (%esp)
-; AVX512_32-NEXT:    movl %ebp, %esp
-; AVX512_32-NEXT:    popl %ebp
-; AVX512_32-NEXT:    retl
+; AVX512DQ_32-LABEL: s64_to_d:
+; AVX512DQ_32:       # %bb.0:
+; AVX512DQ_32-NEXT:    pushl %ebp
+; AVX512DQ_32-NEXT:    movl %esp, %ebp
+; AVX512DQ_32-NEXT:    andl $-8, %esp
+; AVX512DQ_32-NEXT:    subl $8, %esp
+; AVX512DQ_32-NEXT:    vmovsd {{.*#+}} xmm0 = mem[0],zero
+; AVX512DQ_32-NEXT:    vcvtqq2pd %zmm0, %zmm0
+; AVX512DQ_32-NEXT:    vmovlps %xmm0, (%esp)
+; AVX512DQ_32-NEXT:    fldl (%esp)
+; AVX512DQ_32-NEXT:    movl %ebp, %esp
+; AVX512DQ_32-NEXT:    popl %ebp
+; AVX512DQ_32-NEXT:    vzeroupper
+; AVX512DQ_32-NEXT:    retl
 ;
 ; AVX512_64-LABEL: s64_to_d:
 ; AVX512_64:       # %bb.0:
 ; AVX512_64-NEXT:    vcvtsi2sdq %rdi, %xmm0, %xmm0
 ; AVX512_64-NEXT:    retq
 ;
+; AVX512F_32-LABEL: s64_to_d:
+; AVX512F_32:       # %bb.0:
+; AVX512F_32-NEXT:    pushl %ebp
+; AVX512F_32-NEXT:    movl %esp, %ebp
+; AVX512F_32-NEXT:    andl $-8, %esp
+; AVX512F_32-NEXT:    subl $8, %esp
+; AVX512F_32-NEXT:    fildll 8(%ebp)
+; AVX512F_32-NEXT:    fstpl (%esp)
+; AVX512F_32-NEXT:    fldl (%esp)
+; AVX512F_32-NEXT:    movl %ebp, %esp
+; AVX512F_32-NEXT:    popl %ebp
+; AVX512F_32-NEXT:    retl
+;
 ; SSE2_32-LABEL: s64_to_d:
 ; SSE2_32:       # %bb.0:
 ; SSE2_32-NEXT:    pushl %ebp
 ; SSE2_32-NEXT:    movl %esp, %ebp
 ; SSE2_32-NEXT:    andl $-8, %esp
 ; SSE2_32-NEXT:    subl $8, %esp
 ; SSE2_32-NEXT:    fildll 8(%ebp)
 ; SSE2_32-NEXT:    fstpl (%esp)

 ; X87:       # %bb.0:
 ; X87-NEXT:    fildll {{[0-9]+}}(%esp)
 ; X87-NEXT:    retl
   %r = sitofp i64 %a to double
   ret double %r
 }
 
 define double @s64_to_d_2(i64 %a) nounwind {
-; AVX512_32-LABEL: s64_to_d_2:
-; AVX512_32:       # %bb.0:
-; AVX512_32-NEXT:    pushl %ebp
-; AVX512_32-NEXT:    movl %esp, %ebp
-; AVX512_32-NEXT:    andl $-8, %esp
-; AVX512_32-NEXT:    subl $16, %esp
-; AVX512_32-NEXT:    movl 8(%ebp), %eax
-; AVX512_32-NEXT:    movl 12(%ebp), %ecx
-; AVX512_32-NEXT:    addl $5, %eax
-; AVX512_32-NEXT:    adcl $0, %ecx
-; AVX512_32-NEXT:    vmovd %eax, %xmm0
-; AVX512_32-NEXT:    vpinsrd $1, %ecx, %xmm0, %xmm0
-; AVX512_32-NEXT:    vmovq %xmm0, {{[0-9]+}}(%esp)
-; AVX512_32-NEXT:    fildll {{[0-9]+}}(%esp)
-; AVX512_32-NEXT:    fstpl (%esp)
-; AVX512_32-NEXT:    fldl (%esp)
-; AVX512_32-NEXT:    movl %ebp, %esp
-; AVX512_32-NEXT:    popl %ebp
-; AVX512_32-NEXT:    retl
+; AVX512DQ_32-LABEL: s64_to_d_2:
+; AVX512DQ_32:       # %bb.0:
+; AVX512DQ_32-NEXT:    pushl %ebp
+; AVX512DQ_32-NEXT:    movl %esp, %ebp
+; AVX512DQ_32-NEXT:    andl $-8, %esp
+; AVX512DQ_32-NEXT:    subl $8, %esp
+; AVX512DQ_32-NEXT:    movl 8(%ebp), %eax
+; AVX512DQ_32-NEXT:    movl 12(%ebp), %ecx
+; AVX512DQ_32-NEXT:    addl $5, %eax
+; AVX512DQ_32-NEXT:    adcl $0, %ecx
+; AVX512DQ_32-NEXT:    vmovd %ecx, %xmm0
+; AVX512DQ_32-NEXT:    vmovd %eax, %xmm1
+; AVX512DQ_32-NEXT:    vpunpckldq {{.*#+}} xmm0 = xmm1[0],xmm0[0],xmm1[1],xmm0[1]
+; AVX512DQ_32-NEXT:    vmovq {{.*#+}} xmm0 = xmm0[0],zero
+; AVX512DQ_32-NEXT:    vmovdqa %xmm0, %xmm0
+; AVX512DQ_32-NEXT:    vcvtqq2pd %zmm0, %zmm0
+; AVX512DQ_32-NEXT:    vmovlps %xmm0, (%esp)
+; AVX512DQ_32-NEXT:    fldl (%esp)
+; AVX512DQ_32-NEXT:    movl %ebp, %esp
+; AVX512DQ_32-NEXT:    popl %ebp
+; AVX512DQ_32-NEXT:    vzeroupper
+; AVX512DQ_32-NEXT:    retl
 ;
 ; AVX512_64-LABEL: s64_to_d_2:
 ; AVX512_64:       # %bb.0:
 ; AVX512_64-NEXT:    addq $5, %rdi
 ; AVX512_64-NEXT:    vcvtsi2sdq %rdi, %xmm0, %xmm0
 ; AVX512_64-NEXT:    retq
 ;
+; AVX512F_32-LABEL: s64_to_d_2:
+; AVX512F_32:       # %bb.0:
+; AVX512F_32-NEXT:    pushl %ebp
+; AVX512F_32-NEXT:    movl %esp, %ebp
+; AVX512F_32-NEXT:    andl $-8, %esp
+; AVX512F_32-NEXT:    subl $16, %esp
+; AVX512F_32-NEXT:    movl 8(%ebp), %eax
+; AVX512F_32-NEXT:    movl 12(%ebp), %ecx
+; AVX512F_32-NEXT:    addl $5, %eax
+; AVX512F_32-NEXT:    adcl $0, %ecx
+; AVX512F_32-NEXT:    vmovd %eax, %xmm0
+; AVX512F_32-NEXT:    vpinsrd $1, %ecx, %xmm0, %xmm0
+; AVX512F_32-NEXT:    vmovq %xmm0, {{[0-9]+}}(%esp)
+; AVX512F_32-NEXT:    fildll {{[0-9]+}}(%esp)
+; AVX512F_32-NEXT:    fstpl (%esp)
+; AVX512F_32-NEXT:    fldl (%esp)
+; AVX512F_32-NEXT:    movl %ebp, %esp
+; AVX512F_32-NEXT:    popl %ebp
+; AVX512F_32-NEXT:    retl
+;
 ; SSE2_32-LABEL: s64_to_d_2:
 ; SSE2_32:       # %bb.0:
 ; SSE2_32-NEXT:    pushl %ebp
 ; SSE2_32-NEXT:    movl %esp, %ebp
 ; SSE2_32-NEXT:    andl $-8, %esp
 ; SSE2_32-NEXT:    subl $16, %esp
 ; SSE2_32-NEXT:    movl 8(%ebp), %eax
 ; SSE2_32-NEXT:    movl 12(%ebp), %ecx