[x86] fix mappings of cvttp2si x86 intrinsics to x86-specific nodes and isel patterns (PR37551)

[craig.topper]

Summary

The tests in:
https://bugs.llvm.org/show_bug.cgi?id=37751
...show miscompiles because we wrongly mapped and folded x86-specific intrinsics into generic DAG nodes.

This patch corrects the mappings in X86IntrinsicsInfo.h and adds isel matching corresponding to the new patterns. The complete tests for the failure cases should be in avx-cvttp2si.ll and sse-cvttp2si.ll (let me know if you see any missing possibilities/targets). This could be 3 independent reviews/commits if that's preferred.

We mentioned reordering the existing defs in PR37751, but I've postponed that potential change because we might prefer the way we're handling the codegen for AVX512 as shown here. In particular, we're seeing improvements in avx512vl-intrinsics-upgrade.ll because we're able to fold the 2 conversion calls together and then use a simpler move+mask. This is because I didn't exclude AVX512 from the new tablegen patterns, but the existing defs have 'NoVLX' predicates. I don't know AVX512, so I'm not sure if the existing defs are ideal or not.

[craig.topper]

For avx512f I've thought for a little that we may want to add one use checks to the masked patterns for avx512 for the longer latency/low throughput instructions.

[spatel]

test/CodeGen/X86/avx-cvttp2si.ll

19–20 ↗

(On Diff #150655)

Missing load folding for AVX512?

[craig.topper]

test/CodeGen/X86/avx-cvttp2si.ll

19–20 ↗

(On Diff #150655)

Looks like we're missing isel folding patterns with loads for both avx and avx512. But avx is being fixed by peephole. AVX512 isn't because the load folding tables are incomplete.

[RKSimon]

test/CodeGen/X86/sse-cvttp2si.ll

7 ↗

(On Diff #150655)

I don't think they are affected, but please can you add the cvttss2si(64)/cvttsd2si(64) test cases as well to this and the avx file?

[spatel]

Patch updated:
Rebased after adding SSE scalar intrinsics with rL334404. There are no extra AVX1/2 intrinsics of this sort AFAICT, but let me know if I've missed something. As expected, this patch doesn't change any of the new scalar tests.

[RKSimon]

@craig.topper What do you want to do for the avx512 equivalents in X86IntrinsicsInfo.h

e.g.

X86_INTRINSIC_DATA(avx512_mask_cvttpd2dq_128, INTR_TYPE_1OP_MASK, X86ISD::CVTTP2SI, 0),
X86_INTRINSIC_DATA(avx512_mask_cvttpd2dq_512, INTR_TYPE_1OP_MASK, ISD::FP_TO_SINT, X86ISD::CVTTP2SI_RND),
X86_INTRINSIC_DATA(avx512_mask_cvttpd2qq_128, INTR_TYPE_1OP_MASK, ISD::FP_TO_SINT, 0),
X86_INTRINSIC_DATA(avx512_mask_cvttpd2qq_256, INTR_TYPE_1OP_MASK, ISD::FP_TO_SINT, 0),
X86_INTRINSIC_DATA(avx512_mask_cvttpd2qq_512, INTR_TYPE_1OP_MASK, ISD::FP_TO_SINT, X86ISD::CVTTP2SI_RND),

[craig.topper]

I think I've fixed the missing load folding in r334460.

For avx512 I think we should also update those table entries to use X86ISD::CVTTP2SI

[spatel]

Patch updated:
Rebased after load folding fixes.

[spatel]

In D47993#1129179, @craig.topper wrote:

I think I've fixed the missing load folding in r334460.

For avx512 I think we should also update those table entries to use X86ISD::CVTTP2SI

Is it correct that those updates would be independent of the current patch? If so, I'd like to move ahead with this part as-is.

I don't know anything about AVX512, so I don't want to jeopardize these fixes (assuming they're good) with more diffs.

Alternatively, if someone wants to commandeer this and add the AVX512 part, I'll gladly go hack on other stuff. :)

[craig.topper]

I'll fix avx512 including fp_to_uint usages which I assume have the same bug.

[craig.topper]

-Use X86cvttp2si and X86cvttp2usi in the instruction patterns. Add fp_to_sint/fp_to_uint as Pat patterns.
-Add NoVLX to the new patterns, this means we go back to duplicating the operation when we have masked and unmasked version in the same block. But that's a general problem and not something we should tackle here.
-Updated for new test file that tests all the avx512 specific intrinsics that are subject to the same bug.

[craig.topper]

I'm considering a follow up to replace fp_to_sint/fp_to_uint with X86ISD nodes in PreprocessISelDAG to cut down the number of patterns(and add the missing masked patterns) while still giving DAG combine plenty of freedom.

[spatel]

test/CodeGen/X86/avx512-cvttp2i.ll

7–22 ↗

(On Diff #151045)

I'm going to need new glasses after this review.

There are 16 intrinsics here, but there are 17 code diffs in the intrinsics header.
Are we missing tests for:
declare <8 x i32> @llvm.x86.avx512.mask.cvttpd2dq.512(<8 x double>, <8 x i32>, i8, i32)

[RKSimon]

test/CodeGen/X86/mmx-cvt.ll

159 ↗

(On Diff #151045)

You can fix this regression by updating the fp_to_sint pattern at the bottom of X86InstrMMX.td to take X86cvttp2si

[craig.topper]

Add additional MMX pattern. Update for the 17th avx512 intrinsic

[spatel]

LGTM.

Might want to get a 2nd opinion on 512 though since I'm not as familiar with that.

llvm/trunk/lib/Target/X86/X86InstrAVX512.td

defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v4i32x_info, v4f32x_info, OpNode,
                                sched.XMM>, EVEX_V128;
     defm Z256 : avx512_vcvt_fp<opc, OpcodeStr, v8i32x_info, v8f32x_info, OpNode,
                                sched.YMM>, EVEX_V256;
   }
 }
 
 // Convert Double to Signed/Unsigned Doubleword with truncation
 multiclass avx512_cvttpd2dq<bits<8> opc, string OpcodeStr, SDNode OpNode,
-                            SDNode OpNode128, SDNode OpNodeRnd,
-                            X86SchedWriteWidths sched> {
+                            SDNode OpNodeRnd, X86SchedWriteWidths sched> {
   let Predicates = [HasAVX512] in {
     defm Z : avx512_vcvt_fp<opc, OpcodeStr, v8i32x_info, v8f64_info, OpNode,
                             sched.ZMM>,
              avx512_vcvt_fp_sae<opc, OpcodeStr, v8i32x_info, v8f64_info,
                                 OpNodeRnd, sched.ZMM>, EVEX_V512;
   }
   let Predicates = [HasVLX] in {
     // we need "x"/"y" suffixes in order to distinguish between 128 and 256
     // memory forms of these instructions in Asm Parser. They have the same
     // dest type - 'v4i32x_info'. We also specify the broadcast string explicitly
     // due to the same reason.
     defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v4i32x_info, v2f64x_info,
-                               OpNode128, sched.XMM, "{1to2}", "{x}">, EVEX_V128;
+                               OpNode, sched.XMM, "{1to2}", "{x}">, EVEX_V128;
     defm Z256 : avx512_vcvt_fp<opc, OpcodeStr, v4i32x_info, v4f64x_info, OpNode,
                                sched.YMM, "{1to4}", "{y}">, EVEX_V256;
 
     def : InstAlias<OpcodeStr##"x\t{$src, $dst|$dst, $src}",
                     (!cast<Instruction>(NAME # "Z128rr") VR128X:$dst, VR128X:$src), 0>;
     def : InstAlias<OpcodeStr##"x\t{$src, $dst|$dst, $src}",
                     (!cast<Instruction>(NAME # "Z128rm") VR128X:$dst, i128mem:$src), 0, "intel">;
     def : InstAlias<OpcodeStr##"y\t{$src, $dst|$dst, $src}",
defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v2i64x_info, v4f32x_info, OpNode,
                                sched.XMM, "{1to2}", "", f64mem>, EVEX_V128;
     defm Z256 : avx512_vcvt_fp<opc, OpcodeStr, v4i64x_info, v4f32x_info, OpNode,
                                sched.YMM>, EVEX_V256;
   }
 }
 
 // Convert Float to Signed/Unsigned Quardword with truncation
 multiclass avx512_cvttps2qq<bits<8> opc, string OpcodeStr, SDNode OpNode,
-                            SDNode OpNode128, SDNode OpNodeRnd,
-                            X86SchedWriteWidths sched> {
+                            SDNode OpNodeRnd, X86SchedWriteWidths sched> {
   let Predicates = [HasDQI] in {
     defm Z : avx512_vcvt_fp<opc, OpcodeStr, v8i64_info, v8f32x_info, OpNode, sched.ZMM>,
              avx512_vcvt_fp_sae<opc, OpcodeStr, v8i64_info, v8f32x_info,
                                 OpNodeRnd, sched.ZMM>, EVEX_V512;
   }
   let Predicates = [HasDQI, HasVLX] in {
     // Explicitly specified broadcast string, since we take only 2 elements
     // from v4f32x_info source
-    defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v2i64x_info, v4f32x_info, OpNode128,
+    defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v2i64x_info, v4f32x_info, OpNode,
                                sched.XMM, "{1to2}", "", f64mem>, EVEX_V128;
     defm Z256 : avx512_vcvt_fp<opc, OpcodeStr, v4i64x_info, v4f32x_info, OpNode,
                                sched.YMM>, EVEX_V256;
   }
 }
 
 // Convert Signed/Unsigned Quardword to Float
 multiclass avx512_cvtqq2ps<bits<8> opc, string OpcodeStr, SDNode OpNode,

 
 defm VCVTDQ2PD : avx512_cvtdq2pd<0xE6, "vcvtdq2pd", sint_to_fp, X86VSintToFP,
                                  SchedWriteCvtDQ2PD>, XS, EVEX_CD8<32, CD8VH>;
 
 defm VCVTDQ2PS : avx512_cvtdq2ps<0x5B, "vcvtdq2ps", sint_to_fp,
                                 X86VSintToFpRnd, SchedWriteCvtDQ2PS>,
                                 PS, EVEX_CD8<32, CD8VF>;
 
-defm VCVTTPS2DQ : avx512_cvttps2dq<0x5B, "vcvttps2dq", fp_to_sint,
+defm VCVTTPS2DQ : avx512_cvttps2dq<0x5B, "vcvttps2dq", X86cvttp2si,
                                 X86cvttp2siRnd, SchedWriteCvtPS2DQ>,
                                 XS, EVEX_CD8<32, CD8VF>;
 
-defm VCVTTPD2DQ : avx512_cvttpd2dq<0xE6, "vcvttpd2dq", fp_to_sint, X86cvttp2si,
+defm VCVTTPD2DQ : avx512_cvttpd2dq<0xE6, "vcvttpd2dq", X86cvttp2si,
                                  X86cvttp2siRnd, SchedWriteCvtPD2DQ>,
                                  PD, VEX_W, EVEX_CD8<64, CD8VF>;
 
-defm VCVTTPS2UDQ : avx512_cvttps2dq<0x78, "vcvttps2udq", fp_to_uint,
+defm VCVTTPS2UDQ : avx512_cvttps2dq<0x78, "vcvttps2udq", X86cvttp2ui,
                                  X86cvttp2uiRnd, SchedWriteCvtPS2DQ>, PS,
                                  EVEX_CD8<32, CD8VF>;
 
-defm VCVTTPD2UDQ : avx512_cvttpd2dq<0x78, "vcvttpd2udq", fp_to_uint,
-                                 X86cvttp2ui, X86cvttp2uiRnd, SchedWriteCvtPD2DQ>,
+defm VCVTTPD2UDQ : avx512_cvttpd2dq<0x78, "vcvttpd2udq", X86cvttp2ui,
+                                 X86cvttp2uiRnd, SchedWriteCvtPD2DQ>,
                                  PS, VEX_W, EVEX_CD8<64, CD8VF>;
 
 defm VCVTUDQ2PD : avx512_cvtdq2pd<0x7A, "vcvtudq2pd", uint_to_fp,
                                   X86VUintToFP, SchedWriteCvtDQ2PD>, XS,
                                   EVEX_CD8<32, CD8VH>;
 
 defm VCVTUDQ2PS : avx512_cvtdq2ps<0x7A, "vcvtudq2ps", uint_to_fp,
                                  X86VUintToFpRnd, SchedWriteCvtDQ2PS>, XD,

 defm VCVTPD2UQQ : avx512_cvtpd2qq<0x79, "vcvtpd2uqq", X86cvtp2UInt,
                                  X86cvtp2UIntRnd, SchedWriteCvtPD2DQ>, VEX_W,
                                  PD, EVEX_CD8<64, CD8VF>;
 
 defm VCVTPS2UQQ : avx512_cvtps2qq<0x79, "vcvtps2uqq", X86cvtp2UInt,
                                  X86cvtp2UIntRnd, SchedWriteCvtPS2DQ>, PD,
                                  EVEX_CD8<32, CD8VH>;
 
-defm VCVTTPD2QQ : avx512_cvttpd2qq<0x7A, "vcvttpd2qq", fp_to_sint,
+defm VCVTTPD2QQ : avx512_cvttpd2qq<0x7A, "vcvttpd2qq", X86cvttp2si,
                                  X86cvttp2siRnd, SchedWriteCvtPD2DQ>, VEX_W,
                                  PD, EVEX_CD8<64, CD8VF>;
 
-defm VCVTTPS2QQ : avx512_cvttps2qq<0x7A, "vcvttps2qq", fp_to_sint, X86cvttp2si,
+defm VCVTTPS2QQ : avx512_cvttps2qq<0x7A, "vcvttps2qq", X86cvttp2si,
                                  X86cvttp2siRnd, SchedWriteCvtPS2DQ>, PD,
                                  EVEX_CD8<32, CD8VH>;
 
-defm VCVTTPD2UQQ : avx512_cvttpd2qq<0x78, "vcvttpd2uqq", fp_to_uint,
+defm VCVTTPD2UQQ : avx512_cvttpd2qq<0x78, "vcvttpd2uqq", X86cvttp2ui,
                                  X86cvttp2uiRnd, SchedWriteCvtPD2DQ>, VEX_W,
                                  PD, EVEX_CD8<64, CD8VF>;
 
-defm VCVTTPS2UQQ : avx512_cvttps2qq<0x78, "vcvttps2uqq", fp_to_uint, X86cvttp2ui,
+defm VCVTTPS2UQQ : avx512_cvttps2qq<0x78, "vcvttps2uqq", X86cvttp2ui,
                                  X86cvttp2uiRnd, SchedWriteCvtPS2DQ>, PD,
                                  EVEX_CD8<32, CD8VH>;
 
 defm VCVTQQ2PD : avx512_cvtqq2pd<0xE6, "vcvtqq2pd", sint_to_fp,
                             X86VSintToFpRnd, SchedWriteCvtDQ2PD>, VEX_W, XS,
                             EVEX_CD8<64, CD8VF>;
 
 defm VCVTUQQ2PD : avx512_cvtqq2pd<0x7A, "vcvtuqq2pd", uint_to_fp,
                             X86VUintToFpRnd, SchedWriteCvtDQ2PD>, VEX_W, XS,
                             EVEX_CD8<64, CD8VF>;
 
 defm VCVTQQ2PS : avx512_cvtqq2ps<0x5B, "vcvtqq2ps", sint_to_fp, X86VSintToFP,
                             X86VSintToFpRnd, SchedWriteCvtDQ2PS>, VEX_W, PS,
                             EVEX_CD8<64, CD8VF>;
 
 defm VCVTUQQ2PS : avx512_cvtqq2ps<0x7A, "vcvtuqq2ps", uint_to_fp, X86VUintToFP,
                             X86VUintToFpRnd, SchedWriteCvtDQ2PS>, VEX_W, XD,
                             EVEX_CD8<64, CD8VF>;
 
+let Predicates = [HasAVX512] in  {
+  def : Pat<(v16i32 (fp_to_sint (v16f32 VR512:$src))),
+            (VCVTTPS2DQZrr VR512:$src)>;
+  def : Pat<(v16i32 (fp_to_sint (loadv16f32 addr:$src))),
+            (VCVTTPS2DQZrm addr:$src)>;
+
+  def : Pat<(v16i32 (fp_to_uint (v16f32 VR512:$src))),
+            (VCVTTPS2UDQZrr VR512:$src)>;
+  def : Pat<(v16i32 (fp_to_uint (loadv16f32 addr:$src))),
+            (VCVTTPS2UDQZrm addr:$src)>;
+
+  def : Pat<(v8i32 (fp_to_sint (v8f64 VR512:$src))),
+            (VCVTTPD2DQZrr VR512:$src)>;
+  def : Pat<(v8i32 (fp_to_sint (loadv8f64 addr:$src))),
+            (VCVTTPD2DQZrm addr:$src)>;
+
+  def : Pat<(v8i32 (fp_to_uint (v8f64 VR512:$src))),
+            (VCVTTPD2UDQZrr VR512:$src)>;
+  def : Pat<(v8i32 (fp_to_uint (loadv8f64 addr:$src))),
+            (VCVTTPD2UDQZrm addr:$src)>;
+}
+
+let Predicates = [HasVLX] in {
+  def : Pat<(v4i32 (fp_to_sint (v4f32 VR128X:$src))),
+            (VCVTTPS2DQZ128rr VR128X:$src)>;
+  def : Pat<(v4i32 (fp_to_sint (loadv4f32 addr:$src))),
+            (VCVTTPS2DQZ128rm addr:$src)>;
+
+  def : Pat<(v4i32 (fp_to_uint (v4f32 VR128X:$src))),
+            (VCVTTPS2UDQZ128rr VR128X:$src)>;
+  def : Pat<(v4i32 (fp_to_uint (loadv4f32 addr:$src))),
+            (VCVTTPS2UDQZ128rm addr:$src)>;
+
+  def : Pat<(v8i32 (fp_to_sint (v8f32 VR256X:$src))),
+            (VCVTTPS2DQZ256rr VR256X:$src)>;
+  def : Pat<(v8i32 (fp_to_sint (loadv8f32 addr:$src))),
+            (VCVTTPS2DQZ256rm addr:$src)>;
+
+  def : Pat<(v8i32 (fp_to_uint (v8f32 VR256X:$src))),
+            (VCVTTPS2UDQZ256rr VR256X:$src)>;
+  def : Pat<(v8i32 (fp_to_uint (loadv8f32 addr:$src))),
+            (VCVTTPS2UDQZ256rm addr:$src)>;
+
+  def : Pat<(v4i32 (fp_to_sint (v4f64 VR256X:$src))),
+            (VCVTTPD2DQZ256rr VR256X:$src)>;
+  def : Pat<(v4i32 (fp_to_sint (loadv4f64 addr:$src))),
+            (VCVTTPD2DQZ256rm addr:$src)>;
+
+  def : Pat<(v4i32 (fp_to_uint (v4f64 VR256X:$src))),
+            (VCVTTPD2UDQZ256rr VR256X:$src)>;
+  def : Pat<(v4i32 (fp_to_uint (loadv4f64 addr:$src))),
+            (VCVTTPD2UDQZ256rm addr:$src)>;
+}
+
+let Predicates = [HasDQI] in {
+  def : Pat<(v8i64 (fp_to_sint (v8f32 VR256X:$src))),
+            (VCVTTPS2QQZrr VR256X:$src)>;
+  def : Pat<(v8i64 (fp_to_sint (loadv8f32 addr:$src))),
+            (VCVTTPS2QQZrm addr:$src)>;
+
+  def : Pat<(v8i64 (fp_to_uint (v8f32 VR256X:$src))),
+            (VCVTTPS2UQQZrr VR256X:$src)>;
+  def : Pat<(v8i64 (fp_to_uint (loadv8f32 addr:$src))),
+            (VCVTTPS2UQQZrm addr:$src)>;
+
+  def : Pat<(v8i64 (fp_to_sint (v8f64 VR512:$src))),
+            (VCVTTPD2QQZrr VR512:$src)>;
+  def : Pat<(v8i64 (fp_to_sint (loadv8f64 addr:$src))),
+            (VCVTTPD2QQZrm addr:$src)>;
+
+  def : Pat<(v8i64 (fp_to_uint (v8f64 VR512:$src))),
+            (VCVTTPD2UQQZrr VR512:$src)>;
+  def : Pat<(v8i64 (fp_to_uint (loadv8f64 addr:$src))),
+            (VCVTTPD2UQQZrm addr:$src)>;
+}
+
+let Predicates = [HasDQI, HasVLX] in {
+  def : Pat<(v4i64 (fp_to_sint (v4f32 VR128X:$src))),
+            (VCVTTPS2QQZ256rr VR128X:$src)>;
+  def : Pat<(v4i64 (fp_to_sint (loadv4f32 addr:$src))),
+            (VCVTTPS2QQZ256rm addr:$src)>;
+
+  def : Pat<(v4i64 (fp_to_uint (v4f32 VR128X:$src))),
+            (VCVTTPS2UQQZ256rr VR128X:$src)>;
+  def : Pat<(v4i64 (fp_to_uint (loadv4f32 addr:$src))),
+            (VCVTTPS2UQQZ256rm addr:$src)>;
+
+  def : Pat<(v2i64 (fp_to_sint (v2f64 VR128X:$src))),
+            (VCVTTPD2QQZ128rr VR128X:$src)>;
+  def : Pat<(v2i64 (fp_to_sint (loadv2f64 addr:$src))),
+            (VCVTTPD2QQZ128rm addr:$src)>;
+
+  def : Pat<(v2i64 (fp_to_uint (v2f64 VR128X:$src))),
+            (VCVTTPD2UQQZ128rr VR128X:$src)>;
+  def : Pat<(v2i64 (fp_to_uint (loadv2f64 addr:$src))),
+            (VCVTTPD2UQQZ128rm addr:$src)>;
+
+  def : Pat<(v4i64 (fp_to_sint (v4f64 VR256X:$src))),
+            (VCVTTPD2QQZ256rr VR256X:$src)>;
+  def : Pat<(v4i64 (fp_to_sint (loadv4f64 addr:$src))),
+            (VCVTTPD2QQZ256rm addr:$src)>;
+
+  def : Pat<(v4i64 (fp_to_uint (v4f64 VR256X:$src))),
+            (VCVTTPD2UQQZ256rr VR256X:$src)>;
+  def : Pat<(v4i64 (fp_to_uint (loadv4f64 addr:$src))),
+            (VCVTTPD2UQQZ256rm addr:$src)>;
+}
+
 let Predicates = [HasAVX512, NoVLX] in {
 def : Pat<(v8i32 (fp_to_uint (v8f32 VR256X:$src1))),
           (EXTRACT_SUBREG (v16i32 (VCVTTPS2UDQZrr
            (v16f32 (INSERT_SUBREG (IMPLICIT_DEF),
                                   VR256X:$src1, sub_ymm)))), sub_ymm)>;
 
 def : Pat<(v4i32 (fp_to_uint (v4f32 VR128X:$src1))),
           (EXTRACT_SUBREG (v16i32 (VCVTTPS2UDQZrr

llvm/trunk/lib/Target/X86/X86InstrMMX.td

 def : Pat<(f64 (bitconvert (x86mmx VR64:$src))),
           (MMX_MOVQ2FR64rr VR64:$src)>;
 def : Pat<(x86mmx (bitconvert (f64 FR64:$src))),
           (MMX_MOVFR642Qrr FR64:$src)>;
 def : Pat<(x86mmx (MMX_X86movdq2q
                    (bc_v2i64 (v4i32 (X86cvtp2Int (v4f32 VR128:$src)))))),
           (MMX_CVTPS2PIirr VR128:$src)>;
 def : Pat<(x86mmx (MMX_X86movdq2q
+                   (bc_v2i64 (v4i32 (X86cvttp2si (v4f32 VR128:$src)))))),
+          (MMX_CVTTPS2PIirr VR128:$src)>;
+def : Pat<(x86mmx (MMX_X86movdq2q
                    (bc_v2i64 (v4i32 (fp_to_sint (v4f32 VR128:$src)))))),
           (MMX_CVTTPS2PIirr VR128:$src)>;
 def : Pat<(x86mmx (MMX_X86movdq2q
                    (bc_v2i64 (v4i32 (X86cvtp2Int (v2f64 VR128:$src)))))),
           (MMX_CVTPD2PIirr VR128:$src)>;
 def : Pat<(x86mmx (MMX_X86movdq2q
                    (bc_v2i64 (v4i32 (X86cvttp2si (v2f64 VR128:$src)))))),
           (MMX_CVTTPD2PIirr VR128:$src)>;
 }

llvm/trunk/lib/Target/X86/X86InstrSSE.td

def CVTPD2DQrr  : SDI<0xE6, MRMSrcReg, (outs VR128:$dst), (ins VR128:$src),
                       Sched<[WriteCvtPD2I]>;
 
 // Convert with truncation packed single/double fp to doubleword
 // SSE2 packed instructions with XS prefix
 let Predicates = [HasAVX, NoVLX] in {
 def VCVTTPS2DQrr : VS2SI<0x5B, MRMSrcReg, (outs VR128:$dst), (ins VR128:$src),
                          "cvttps2dq\t{$src, $dst|$dst, $src}",
                          [(set VR128:$dst,
-                           (v4i32 (fp_to_sint (v4f32 VR128:$src))))]>,
+                           (v4i32 (X86cvttp2si (v4f32 VR128:$src))))]>,
                          VEX, Sched<[WriteCvtPS2I]>, VEX_WIG;
 def VCVTTPS2DQrm : VS2SI<0x5B, MRMSrcMem, (outs VR128:$dst), (ins f128mem:$src),
                          "cvttps2dq\t{$src, $dst|$dst, $src}",
                          [(set VR128:$dst,
-                           (v4i32 (fp_to_sint (loadv4f32 addr:$src))))]>,
+                           (v4i32 (X86cvttp2si (loadv4f32 addr:$src))))]>,
                          VEX, Sched<[WriteCvtPS2ILd]>, VEX_WIG;
 def VCVTTPS2DQYrr : VS2SI<0x5B, MRMSrcReg, (outs VR256:$dst), (ins VR256:$src),
                           "cvttps2dq\t{$src, $dst|$dst, $src}",
                           [(set VR256:$dst,
-                            (v8i32 (fp_to_sint (v8f32 VR256:$src))))]>,
+                            (v8i32 (X86cvttp2si (v8f32 VR256:$src))))]>,
                           VEX, VEX_L, Sched<[WriteCvtPS2IY]>, VEX_WIG;
 def VCVTTPS2DQYrm : VS2SI<0x5B, MRMSrcMem, (outs VR256:$dst), (ins f256mem:$src),
                           "cvttps2dq\t{$src, $dst|$dst, $src}",
                           [(set VR256:$dst,
-                            (v8i32 (fp_to_sint (loadv8f32 addr:$src))))]>,
+                            (v8i32 (X86cvttp2si (loadv8f32 addr:$src))))]>,
                           VEX, VEX_L,
                           Sched<[WriteCvtPS2IYLd]>, VEX_WIG;
 }
 
+let Predicates = [HasAVX, NoVLX] in {
+  def : Pat<(v4i32 (fp_to_sint (v4f32 VR128:$src))),
+            (VCVTTPS2DQrr VR128:$src)>;
+  def : Pat<(v4i32 (fp_to_sint (loadv4f32 addr:$src))),
+            (VCVTTPS2DQrm addr:$src)>;
+  def : Pat<(v8i32 (fp_to_sint (v8f32 VR256:$src))),
+            (VCVTTPS2DQYrr VR256:$src)>;
+  def : Pat<(v8i32 (fp_to_sint (loadv8f32 addr:$src))),
+            (VCVTTPS2DQYrm addr:$src)>;
+}
+
 def CVTTPS2DQrr : S2SI<0x5B, MRMSrcReg, (outs VR128:$dst), (ins VR128:$src),
                        "cvttps2dq\t{$src, $dst|$dst, $src}",
                        [(set VR128:$dst,
-                         (v4i32 (fp_to_sint (v4f32 VR128:$src))))]>,
+                         (v4i32 (X86cvttp2si (v4f32 VR128:$src))))]>,
                        Sched<[WriteCvtPS2I]>;
 def CVTTPS2DQrm : S2SI<0x5B, MRMSrcMem, (outs VR128:$dst), (ins f128mem:$src),
                        "cvttps2dq\t{$src, $dst|$dst, $src}",
                        [(set VR128:$dst,
-                         (v4i32 (fp_to_sint (memopv4f32 addr:$src))))]>,
+                         (v4i32 (X86cvttp2si (memopv4f32 addr:$src))))]>,
                        Sched<[WriteCvtPS2ILd]>;
 
+let Predicates = [UseSSE2] in {
+  def : Pat<(v4i32 (fp_to_sint (v4f32 VR128:$src))),
+            (CVTTPS2DQrr VR128:$src)>;
+  def : Pat<(v4i32 (fp_to_sint (memopv4f32 addr:$src))),
+            (CVTTPS2DQrm addr:$src)>;
+}
+
 let Predicates = [HasAVX, NoVLX] in
 def VCVTTPD2DQrr : VPDI<0xE6, MRMSrcReg, (outs VR128:$dst), (ins VR128:$src),
                         "cvttpd2dq\t{$src, $dst|$dst, $src}",
                         [(set VR128:$dst,
                           (v4i32 (X86cvttp2si (v2f64 VR128:$src))))]>,
                         VEX, Sched<[WriteCvtPD2I]>, VEX_WIG;
 
 // The assembler can recognize rr 256-bit instructions by seeing a ymm
 // register, but the same isn't true when using memory operands instead.
 // Provide other assembly rr and rm forms to address this explicitly.
 
 // XMM only
 def : InstAlias<"vcvttpd2dqx\t{$src, $dst|$dst, $src}",
                 (VCVTTPD2DQrr VR128:$dst, VR128:$src), 0>;
+
 let Predicates = [HasAVX, NoVLX] in
 def VCVTTPD2DQrm : VPDI<0xE6, MRMSrcMem, (outs VR128:$dst), (ins f128mem:$src),
                         "cvttpd2dq{x}\t{$src, $dst|$dst, $src}",
                         [(set VR128:$dst,
                           (v4i32 (X86cvttp2si (loadv2f64 addr:$src))))]>,
                         VEX, Sched<[WriteCvtPD2ILd]>, VEX_WIG;
 def : InstAlias<"vcvttpd2dqx\t{$src, $dst|$dst, $src}",
                 (VCVTTPD2DQrm VR128:$dst, f128mem:$src), 0, "intel">;
 
 // YMM only
 let Predicates = [HasAVX, NoVLX] in {
 def VCVTTPD2DQYrr : VPDI<0xE6, MRMSrcReg, (outs VR128:$dst), (ins VR256:$src),
                          "cvttpd2dq\t{$src, $dst|$dst, $src}",
                          [(set VR128:$dst,
-                           (v4i32 (fp_to_sint (v4f64 VR256:$src))))]>,
+                           (v4i32 (X86cvttp2si (v4f64 VR256:$src))))]>,
                          VEX, VEX_L, Sched<[WriteCvtPD2IY]>, VEX_WIG;
 def VCVTTPD2DQYrm : VPDI<0xE6, MRMSrcMem, (outs VR128:$dst), (ins f256mem:$src),
                          "cvttpd2dq{y}\t{$src, $dst|$dst, $src}",
                          [(set VR128:$dst,
-                           (v4i32 (fp_to_sint (loadv4f64 addr:$src))))]>,
+                           (v4i32 (X86cvttp2si (loadv4f64 addr:$src))))]>,
                          VEX, VEX_L, Sched<[WriteCvtPD2IYLd]>, VEX_WIG;
 }
 def : InstAlias<"vcvttpd2dqy\t{$src, $dst|$dst, $src}",
                 (VCVTTPD2DQYrr VR128:$dst, VR256:$src), 0>;
 def : InstAlias<"vcvttpd2dqy\t{$src, $dst|$dst, $src}",
                 (VCVTTPD2DQYrm VR128:$dst, f256mem:$src), 0, "intel">;
 
 let Predicates = [HasAVX, NoVLX] in {
+  def : Pat<(v4i32 (fp_to_sint (v4f64 VR256:$src))),
+            (VCVTTPD2DQYrr VR256:$src)>;
+  def : Pat<(v4i32 (fp_to_sint (loadv4f64 addr:$src))),
+            (VCVTTPD2DQYrm addr:$src)>;
+}
+
+let Predicates = [HasAVX, NoVLX] in {
   let AddedComplexity = 15 in {
     def : Pat<(X86vzmovl (v2i64 (bitconvert
                                  (v4i32 (X86cvtp2Int (v2f64 VR128:$src)))))),
               (VCVTPD2DQrr VR128:$src)>;
     def : Pat<(X86vzmovl (v2i64 (bitconvert
                                  (v4i32 (X86cvtp2Int (loadv2f64 addr:$src)))))),
               (VCVTPD2DQrm addr:$src)>;
     def : Pat<(X86vzmovl (v2i64 (bitconvert

llvm/trunk/lib/Target/X86/X86IntrinsicsInfo.h

 static const IntrinsicData  IntrinsicsWithoutChain[] = {
   X86_INTRINSIC_DATA(avx_addsub_pd_256, INTR_TYPE_2OP, X86ISD::ADDSUB, 0),
   X86_INTRINSIC_DATA(avx_addsub_ps_256, INTR_TYPE_2OP, X86ISD::ADDSUB, 0),
   X86_INTRINSIC_DATA(avx_cmp_pd_256,    INTR_TYPE_3OP, X86ISD::CMPP, 0),
   X86_INTRINSIC_DATA(avx_cmp_ps_256,    INTR_TYPE_3OP, X86ISD::CMPP, 0),
   X86_INTRINSIC_DATA(avx_cvt_pd2_ps_256,CVTPD2PS, ISD::FP_ROUND, 0),
   X86_INTRINSIC_DATA(avx_cvt_pd2dq_256, INTR_TYPE_1OP, X86ISD::CVTP2SI, 0),
   X86_INTRINSIC_DATA(avx_cvt_ps2dq_256, INTR_TYPE_1OP, X86ISD::CVTP2SI, 0),
-  X86_INTRINSIC_DATA(avx_cvtt_pd2dq_256,INTR_TYPE_1OP, ISD::FP_TO_SINT, 0),
-  X86_INTRINSIC_DATA(avx_cvtt_ps2dq_256,INTR_TYPE_1OP, ISD::FP_TO_SINT, 0),
+  X86_INTRINSIC_DATA(avx_cvtt_pd2dq_256,INTR_TYPE_1OP, X86ISD::CVTTP2SI, 0),
+  X86_INTRINSIC_DATA(avx_cvtt_ps2dq_256,INTR_TYPE_1OP, X86ISD::CVTTP2SI, 0),
   X86_INTRINSIC_DATA(avx_hadd_pd_256,   INTR_TYPE_2OP, X86ISD::FHADD, 0),
   X86_INTRINSIC_DATA(avx_hadd_ps_256,   INTR_TYPE_2OP, X86ISD::FHADD, 0),
   X86_INTRINSIC_DATA(avx_hsub_pd_256,   INTR_TYPE_2OP, X86ISD::FHSUB, 0),
   X86_INTRINSIC_DATA(avx_hsub_ps_256,   INTR_TYPE_2OP, X86ISD::FHSUB, 0),
   X86_INTRINSIC_DATA(avx_max_pd_256,    INTR_TYPE_2OP, X86ISD::FMAX, 0),
   X86_INTRINSIC_DATA(avx_max_ps_256,    INTR_TYPE_2OP, X86ISD::FMAX, 0),
   X86_INTRINSIC_DATA(avx_min_pd_256,    INTR_TYPE_2OP, X86ISD::FMIN, 0),
   X86_INTRINSIC_DATA(avx_min_ps_256,    INTR_TYPE_2OP, X86ISD::FMIN, 0),
X86_INTRINSIC_DATA(avx512_mask_cvtqq2ps_512, INTR_TYPE_1OP_MASK,
                      ISD::SINT_TO_FP, X86ISD::SINT_TO_FP_RND),
   X86_INTRINSIC_DATA(avx512_mask_cvtsd2ss_round, INTR_TYPE_SCALAR_MASK_RM,
                      X86ISD::VFPROUNDS_RND, 0),
   X86_INTRINSIC_DATA(avx512_mask_cvtss2sd_round, INTR_TYPE_SCALAR_MASK_RM,
                      X86ISD::VFPEXTS_RND, 0),
   X86_INTRINSIC_DATA(avx512_mask_cvttpd2dq_128, INTR_TYPE_1OP_MASK,
                      X86ISD::CVTTP2SI, 0),
   X86_INTRINSIC_DATA(avx512_mask_cvttpd2dq_512, INTR_TYPE_1OP_MASK,
-                     ISD::FP_TO_SINT, X86ISD::CVTTP2SI_RND),
+                     X86ISD::CVTTP2SI, X86ISD::CVTTP2SI_RND),
   X86_INTRINSIC_DATA(avx512_mask_cvttpd2qq_128, INTR_TYPE_1OP_MASK,
-                     ISD::FP_TO_SINT, 0),
+                     X86ISD::CVTTP2SI, 0),
   X86_INTRINSIC_DATA(avx512_mask_cvttpd2qq_256, INTR_TYPE_1OP_MASK,
-                     ISD::FP_TO_SINT, 0),
+                     X86ISD::CVTTP2SI, 0),
   X86_INTRINSIC_DATA(avx512_mask_cvttpd2qq_512, INTR_TYPE_1OP_MASK,
-                     ISD::FP_TO_SINT, X86ISD::CVTTP2SI_RND),
+                     X86ISD::CVTTP2SI, X86ISD::CVTTP2SI_RND),
   X86_INTRINSIC_DATA(avx512_mask_cvttpd2udq_128, INTR_TYPE_1OP_MASK,
                      X86ISD::CVTTP2UI, 0),
   X86_INTRINSIC_DATA(avx512_mask_cvttpd2udq_256, INTR_TYPE_1OP_MASK,
-                     ISD::FP_TO_UINT, 0),
+                     X86ISD::CVTTP2UI, 0),
   X86_INTRINSIC_DATA(avx512_mask_cvttpd2udq_512, INTR_TYPE_1OP_MASK,
-                     ISD::FP_TO_UINT, X86ISD::CVTTP2UI_RND),
+                     X86ISD::CVTTP2UI, X86ISD::CVTTP2UI_RND),
   X86_INTRINSIC_DATA(avx512_mask_cvttpd2uqq_128, INTR_TYPE_1OP_MASK,
-                     ISD::FP_TO_UINT, 0),
+                     X86ISD::CVTTP2UI, 0),
   X86_INTRINSIC_DATA(avx512_mask_cvttpd2uqq_256, INTR_TYPE_1OP_MASK,
-                     ISD::FP_TO_UINT, 0),
+                     X86ISD::CVTTP2UI, 0),
   X86_INTRINSIC_DATA(avx512_mask_cvttpd2uqq_512, INTR_TYPE_1OP_MASK,
-                     ISD::FP_TO_UINT, X86ISD::CVTTP2UI_RND),
+                     X86ISD::CVTTP2UI, X86ISD::CVTTP2UI_RND),
   X86_INTRINSIC_DATA(avx512_mask_cvttps2dq_512, INTR_TYPE_1OP_MASK,
-                     ISD::FP_TO_SINT, X86ISD::CVTTP2SI_RND),
+                     X86ISD::CVTTP2SI, X86ISD::CVTTP2SI_RND),
   X86_INTRINSIC_DATA(avx512_mask_cvttps2qq_128, INTR_TYPE_1OP_MASK,
                      X86ISD::CVTTP2SI, 0),
   X86_INTRINSIC_DATA(avx512_mask_cvttps2qq_256, INTR_TYPE_1OP_MASK,
-                     ISD::FP_TO_SINT, 0),
+                     X86ISD::CVTTP2SI, 0),
   X86_INTRINSIC_DATA(avx512_mask_cvttps2qq_512, INTR_TYPE_1OP_MASK,
-                     ISD::FP_TO_SINT, X86ISD::CVTTP2SI_RND),
+                     X86ISD::CVTTP2SI, X86ISD::CVTTP2SI_RND),
   X86_INTRINSIC_DATA(avx512_mask_cvttps2udq_128, INTR_TYPE_1OP_MASK,
-                     ISD::FP_TO_UINT, 0),
+                     X86ISD::CVTTP2UI, 0),
   X86_INTRINSIC_DATA(avx512_mask_cvttps2udq_256, INTR_TYPE_1OP_MASK,
-                     ISD::FP_TO_UINT, 0),
+                     X86ISD::CVTTP2UI, 0),
   X86_INTRINSIC_DATA(avx512_mask_cvttps2udq_512, INTR_TYPE_1OP_MASK,
-                     ISD::FP_TO_UINT, X86ISD::CVTTP2UI_RND),
+                     X86ISD::CVTTP2UI, X86ISD::CVTTP2UI_RND),
   X86_INTRINSIC_DATA(avx512_mask_cvttps2uqq_128, INTR_TYPE_1OP_MASK,
                      X86ISD::CVTTP2UI, 0),
   X86_INTRINSIC_DATA(avx512_mask_cvttps2uqq_256, INTR_TYPE_1OP_MASK,
-                     ISD::FP_TO_UINT, 0),
+                     X86ISD::CVTTP2UI, 0),
   X86_INTRINSIC_DATA(avx512_mask_cvttps2uqq_512, INTR_TYPE_1OP_MASK,
-                     ISD::FP_TO_UINT, X86ISD::CVTTP2UI_RND),
+                     X86ISD::CVTTP2UI, X86ISD::CVTTP2UI_RND),
   X86_INTRINSIC_DATA(avx512_mask_cvtudq2ps_512, INTR_TYPE_1OP_MASK,
                      ISD::UINT_TO_FP, X86ISD::UINT_TO_FP_RND),
   X86_INTRINSIC_DATA(avx512_mask_cvtuqq2pd_512, INTR_TYPE_1OP_MASK,
                      ISD::UINT_TO_FP, X86ISD::UINT_TO_FP_RND),
   X86_INTRINSIC_DATA(avx512_mask_cvtuqq2ps_128, INTR_TYPE_1OP_MASK,
                      X86ISD::CVTUI2P, 0),
   X86_INTRINSIC_DATA(avx512_mask_cvtuqq2ps_256, INTR_TYPE_1OP_MASK,
                      ISD::UINT_TO_FP, 0),

   X86_INTRINSIC_DATA(sse2_comigt_sd,    COMI, X86ISD::COMI, ISD::SETGT),
   X86_INTRINSIC_DATA(sse2_comile_sd,    COMI, X86ISD::COMI, ISD::SETLE),
   X86_INTRINSIC_DATA(sse2_comilt_sd,    COMI, X86ISD::COMI, ISD::SETLT),
   X86_INTRINSIC_DATA(sse2_comineq_sd,   COMI, X86ISD::COMI, ISD::SETNE),
   X86_INTRINSIC_DATA(sse2_cvtpd2dq,     INTR_TYPE_1OP, X86ISD::CVTP2SI, 0),
   X86_INTRINSIC_DATA(sse2_cvtpd2ps,     INTR_TYPE_1OP, X86ISD::VFPROUND, 0),
   X86_INTRINSIC_DATA(sse2_cvtps2dq,     INTR_TYPE_1OP, X86ISD::CVTP2SI, 0),
   X86_INTRINSIC_DATA(sse2_cvttpd2dq,    INTR_TYPE_1OP, X86ISD::CVTTP2SI, 0),
-  X86_INTRINSIC_DATA(sse2_cvttps2dq,    INTR_TYPE_1OP, ISD::FP_TO_SINT, 0),
+  X86_INTRINSIC_DATA(sse2_cvttps2dq,    INTR_TYPE_1OP, X86ISD::CVTTP2SI, 0),
   X86_INTRINSIC_DATA(sse2_max_pd,       INTR_TYPE_2OP, X86ISD::FMAX, 0),
   X86_INTRINSIC_DATA(sse2_max_sd,       INTR_TYPE_2OP, X86ISD::FMAXS, 0),
   X86_INTRINSIC_DATA(sse2_min_pd,       INTR_TYPE_2OP, X86ISD::FMIN, 0),
   X86_INTRINSIC_DATA(sse2_min_sd,       INTR_TYPE_2OP, X86ISD::FMINS, 0),
   X86_INTRINSIC_DATA(sse2_movmsk_pd,    INTR_TYPE_1OP, X86ISD::MOVMSK, 0),
   X86_INTRINSIC_DATA(sse2_packssdw_128, INTR_TYPE_2OP, X86ISD::PACKSS, 0),
   X86_INTRINSIC_DATA(sse2_packsswb_128, INTR_TYPE_2OP, X86ISD::PACKSS, 0),
   X86_INTRINSIC_DATA(sse2_packuswb_128, INTR_TYPE_2OP, X86ISD::PACKUS, 0),

llvm/trunk/test/CodeGen/X86/avx-cvttp2si.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-- -mattr=avx              | FileCheck %s --check-prefixes=AVX,AVX1
 ; RUN: llc < %s -mtriple=x86_64-- -mattr=avx512f,avx512vl | FileCheck %s --check-prefixes=AVX,AVX512
 
 ; PR37751 - https://bugs.llvm.org/show_bug.cgi?id=37751
 ; We can't combine into 'round' instructions because the behavior is different for out-of-range values.
 
 declare <8 x i32> @llvm.x86.avx.cvtt.ps2dq.256(<8 x float>)
 declare <4 x i32> @llvm.x86.avx.cvtt.pd2dq.256(<4 x double>)
 
 define <8 x float> @float_to_int_to_float_mem_v8f32(<8 x float>* %p) {
 ; AVX-LABEL: float_to_int_to_float_mem_v8f32:
 ; AVX:       # %bb.0:
-; AVX-NEXT:    vroundps $11, (%rdi), %ymm0
+; AVX-NEXT:    vcvttps2dq (%rdi), %ymm0
+; AVX-NEXT:    vcvtdq2ps %ymm0, %ymm0
 ; AVX-NEXT:    retq
   %x = load <8 x float>, <8 x float>* %p, align 16
   %fptosi = tail call <8 x i32> @llvm.x86.avx.cvtt.ps2dq.256(<8 x float> %x)
   %sitofp = sitofp <8 x i32> %fptosi to <8 x float>
   ret <8 x float> %sitofp
 }
 
 define <8 x float> @float_to_int_to_float_reg_v8f32(<8 x float> %x) {
 ; AVX-LABEL: float_to_int_to_float_reg_v8f32:
 ; AVX:       # %bb.0:
-; AVX-NEXT:    vroundps $11, %ymm0, %ymm0
+; AVX-NEXT:    vcvttps2dq %ymm0, %ymm0
+; AVX-NEXT:    vcvtdq2ps %ymm0, %ymm0
 ; AVX-NEXT:    retq
   %fptosi = tail call <8 x i32> @llvm.x86.avx.cvtt.ps2dq.256(<8 x float> %x)
   %sitofp = sitofp <8 x i32> %fptosi to <8 x float>
   ret <8 x float> %sitofp
 }
 
 define <4 x double> @float_to_int_to_float_mem_v4f64(<4 x double>* %p) {
 ; AVX-LABEL: float_to_int_to_float_mem_v4f64:
 ; AVX:       # %bb.0:
-; AVX-NEXT:    vroundpd $11, (%rdi), %ymm0
+; AVX-NEXT:    vcvttpd2dqy (%rdi), %xmm0
+; AVX-NEXT:    vcvtdq2pd %xmm0, %ymm0
 ; AVX-NEXT:    retq
   %x = load <4 x double>, <4 x double>* %p, align 16
   %fptosi = tail call <4 x i32> @llvm.x86.avx.cvtt.pd2dq.256(<4 x double> %x)
   %sitofp = sitofp <4 x i32> %fptosi to <4 x double>
   ret <4 x double> %sitofp
 }
 
 define <4 x double> @float_to_int_to_float_reg_v4f64(<4 x double> %x) {
 ; AVX-LABEL: float_to_int_to_float_reg_v4f64:
 ; AVX:       # %bb.0:
-; AVX-NEXT:    vroundpd $11, %ymm0, %ymm0
+; AVX-NEXT:    vcvttpd2dq %ymm0, %xmm0
+; AVX-NEXT:    vcvtdq2pd %xmm0, %ymm0
 ; AVX-NEXT:    retq
   %fptosi = tail call <4 x i32> @llvm.x86.avx.cvtt.pd2dq.256(<4 x double> %x)
   %sitofp = sitofp <4 x i32> %fptosi to <4 x double>
   ret <4 x double> %sitofp
 }
 

llvm/trunk/test/CodeGen/X86/avx512-cvttp2i.ll

 declare <8 x i64> @llvm.x86.avx512.mask.cvttpd2qq.512(<8 x double>, <8 x i64>, i8, i32)
 declare <2 x i64> @llvm.x86.avx512.mask.cvttpd2uqq.128(<2 x double>, <2 x i64>, i8)
 declare <4 x i64> @llvm.x86.avx512.mask.cvttpd2uqq.256(<4 x double>, <4 x i64>, i8)
 declare <8 x i64> @llvm.x86.avx512.mask.cvttpd2uqq.512(<8 x double>, <8 x i64>, i8, i32)
 
 define <16 x float> @float_to_sint_to_float_mem_v16f32(<16 x float>* %p) {
 ; CHECK-LABEL: float_to_sint_to_float_mem_v16f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vrndscaleps $11, (%rdi), %zmm0
+; CHECK-NEXT:    vcvttps2dq (%rdi), %zmm0
+; CHECK-NEXT:    vcvtdq2ps %zmm0, %zmm0
 ; CHECK-NEXT:    retq
   %x = load <16 x float>, <16 x float>* %p
   %fptosi = tail call <16 x i32> @llvm.x86.avx512.mask.cvttps2dq.512(<16 x float> %x, <16 x i32> undef, i16 -1, i32 4)
   %sitofp = sitofp <16 x i32> %fptosi to <16 x float>
   ret <16 x float> %sitofp
 }
 
 define <16 x float> @float_to_sint_to_float_reg_v16f32(<16 x float> %x) {
 ; CHECK-LABEL: float_to_sint_to_float_reg_v16f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vrndscaleps $11, %zmm0, %zmm0
+; CHECK-NEXT:    vcvttps2dq %zmm0, %zmm0
+; CHECK-NEXT:    vcvtdq2ps %zmm0, %zmm0
 ; CHECK-NEXT:    retq
   %fptosi = tail call <16 x i32> @llvm.x86.avx512.mask.cvttps2dq.512(<16 x float> %x, <16 x i32> undef, i16 -1, i32 4)
   %sitofp = sitofp <16 x i32> %fptosi to <16 x float>
   ret <16 x float> %sitofp
 }
 
 define <16 x float> @float_to_uint_to_float_mem_v16f32(<16 x float>* %p) {
 ; CHECK-LABEL: float_to_uint_to_float_mem_v16f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vrndscaleps $11, (%rdi), %zmm0
+; CHECK-NEXT:    vcvttps2udq (%rdi), %zmm0
+; CHECK-NEXT:    vcvtudq2ps %zmm0, %zmm0
 ; CHECK-NEXT:    retq
   %x = load <16 x float>, <16 x float>* %p
   %fptoui = tail call <16 x i32> @llvm.x86.avx512.mask.cvttps2udq.512(<16 x float> %x, <16 x i32> undef, i16 -1, i32 4)
   %uitofp = uitofp <16 x i32> %fptoui to <16 x float>
   ret <16 x float> %uitofp
 }
 
 define <16 x float> @float_to_uint_to_float_reg_v16f32(<16 x float> %x) {
 ; CHECK-LABEL: float_to_uint_to_float_reg_v16f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vrndscaleps $11, %zmm0, %zmm0
+; CHECK-NEXT:    vcvttps2udq %zmm0, %zmm0
+; CHECK-NEXT:    vcvtudq2ps %zmm0, %zmm0
 ; CHECK-NEXT:    retq
   %fptoui = tail call <16 x i32> @llvm.x86.avx512.mask.cvttps2udq.512(<16 x float> %x, <16 x i32> undef, i16 -1, i32 4)
   %uitofp = uitofp <16 x i32> %fptoui to <16 x float>
   ret <16 x float> %uitofp
 }
 
 define <4 x float> @float_to_uint_to_float_mem_v4f32(<4 x float>* %p) {
 ; CHECK-LABEL: float_to_uint_to_float_mem_v4f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundps $11, (%rdi), %xmm0
+; CHECK-NEXT:    vcvttps2udq (%rdi), %xmm0
+; CHECK-NEXT:    vcvtudq2ps %xmm0, %xmm0
 ; CHECK-NEXT:    retq
   %x = load <4 x float>, <4 x float>* %p
   %fptoui = tail call <4 x i32> @llvm.x86.avx512.mask.cvttps2udq.128(<4 x float> %x, <4 x i32> undef, i8 -1)
   %uitofp = uitofp <4 x i32> %fptoui to <4 x float>
   ret <4 x float> %uitofp
 }
 
 define <4 x float> @float_to_uint_to_float_reg_v4f32(<4 x float> %x) {
 ; CHECK-LABEL: float_to_uint_to_float_reg_v4f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundps $11, %xmm0, %xmm0
+; CHECK-NEXT:    vcvttps2udq %xmm0, %xmm0
+; CHECK-NEXT:    vcvtudq2ps %xmm0, %xmm0
 ; CHECK-NEXT:    retq
   %fptoui = tail call <4 x i32> @llvm.x86.avx512.mask.cvttps2udq.128(<4 x float> %x, <4 x i32> undef, i8 -1)
   %uitofp = uitofp <4 x i32> %fptoui to <4 x float>
   ret <4 x float> %uitofp
 }
 
 define <8 x float> @float_to_uint_to_float_mem_v8f32(<8 x float>* %p) {
 ; CHECK-LABEL: float_to_uint_to_float_mem_v8f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundps $11, (%rdi), %ymm0
+; CHECK-NEXT:    vcvttps2udq (%rdi), %ymm0
+; CHECK-NEXT:    vcvtudq2ps %ymm0, %ymm0
 ; CHECK-NEXT:    retq
   %x = load <8 x float>, <8 x float>* %p
   %fptoui = tail call <8 x i32> @llvm.x86.avx512.mask.cvttps2udq.256(<8 x float> %x, <8 x i32> undef, i8 -1)
   %uitofp = uitofp <8 x i32> %fptoui to <8 x float>
   ret <8 x float> %uitofp
 }
 
 define <8 x float> @float_to_uint_to_float_reg_v8f32(<8 x float> %x) {
 ; CHECK-LABEL: float_to_uint_to_float_reg_v8f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundps $11, %ymm0, %ymm0
+; CHECK-NEXT:    vcvttps2udq %ymm0, %ymm0
+; CHECK-NEXT:    vcvtudq2ps %ymm0, %ymm0
 ; CHECK-NEXT:    retq
   %fptoui = tail call <8 x i32> @llvm.x86.avx512.mask.cvttps2udq.256(<8 x float> %x, <8 x i32> undef, i8 -1)
   %uitofp = uitofp <8 x i32> %fptoui to <8 x float>
   ret <8 x float> %uitofp
 }
 
 define <4 x double> @double_to_uint_to_double_mem_v4f64(<4 x double>* %p) {
 ; CHECK-LABEL: double_to_uint_to_double_mem_v4f64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundpd $11, (%rdi), %ymm0
+; CHECK-NEXT:    vcvttpd2udqy (%rdi), %xmm0
+; CHECK-NEXT:    vcvtudq2pd %xmm0, %ymm0
 ; CHECK-NEXT:    retq
   %x = load <4 x double>, <4 x double>* %p
   %fptoui = tail call <4 x i32> @llvm.x86.avx512.mask.cvttpd2udq.256(<4 x double> %x, <4 x i32> undef, i8 -1)
   %uitofp = uitofp <4 x i32> %fptoui to <4 x double>
   ret <4 x double> %uitofp
 }
 
 define <4 x double> @double_to_uint_to_double_reg_v4f64(<4 x double> %x) {
 ; CHECK-LABEL: double_to_uint_to_double_reg_v4f64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundpd $11, %ymm0, %ymm0
+; CHECK-NEXT:    vcvttpd2udq %ymm0, %xmm0
+; CHECK-NEXT:    vcvtudq2pd %xmm0, %ymm0
 ; CHECK-NEXT:    retq
   %fptoui = tail call <4 x i32> @llvm.x86.avx512.mask.cvttpd2udq.256(<4 x double> %x, <4 x i32> undef, i8 -1)
   %uitofp = uitofp <4 x i32> %fptoui to <4 x double>
   ret <4 x double> %uitofp
 }
 
 define <8 x double> @double_to_sint_to_double_mem_v8f64(<8 x double>* %p) {
 ; CHECK-LABEL: double_to_sint_to_double_mem_v8f64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vrndscalepd $11, (%rdi), %zmm0
+; CHECK-NEXT:    vcvttpd2dq (%rdi), %ymm0
+; CHECK-NEXT:    vcvtdq2pd %ymm0, %zmm0
 ; CHECK-NEXT:    retq
   %x = load <8 x double>, <8 x double>* %p
   %fptosi = tail call <8 x i32> @llvm.x86.avx512.mask.cvttpd2dq.512(<8 x double> %x, <8 x i32> undef, i8 -1, i32 4)
   %sitofp = sitofp <8 x i32> %fptosi to <8 x double>
   ret <8 x double> %sitofp
 }
 
 define <8 x double> @double_to_sint_to_double_reg_v8f64(<8 x double> %x) {
 ; CHECK-LABEL: double_to_sint_to_double_reg_v8f64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vrndscalepd $11, %zmm0, %zmm0
+; CHECK-NEXT:    vcvttpd2dq %zmm0, %ymm0
+; CHECK-NEXT:    vcvtdq2pd %ymm0, %zmm0
 ; CHECK-NEXT:    retq
   %fptosi = tail call <8 x i32> @llvm.x86.avx512.mask.cvttpd2dq.512(<8 x double> %x, <8 x i32> undef, i8 -1, i32 4)
   %sitofp = sitofp <8 x i32> %fptosi to <8 x double>
   ret <8 x double> %sitofp
 }
 
 define <8 x double> @double_to_uint_to_double_mem_v8f64(<8 x double>* %p) {
 ; CHECK-LABEL: double_to_uint_to_double_mem_v8f64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vrndscalepd $11, (%rdi), %zmm0
+; CHECK-NEXT:    vcvttpd2udq (%rdi), %ymm0
+; CHECK-NEXT:    vcvtudq2pd %ymm0, %zmm0
 ; CHECK-NEXT:    retq
   %x = load <8 x double>, <8 x double>* %p
   %fptoui = tail call <8 x i32> @llvm.x86.avx512.mask.cvttpd2udq.512(<8 x double> %x, <8 x i32> undef, i8 -1, i32 4)
   %uitofp = uitofp <8 x i32> %fptoui to <8 x double>
   ret <8 x double> %uitofp
 }
 
 define <8 x double> @double_to_uint_to_double_reg_v8f64(<8 x double> %x) {
 ; CHECK-LABEL: double_to_uint_to_double_reg_v8f64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vrndscalepd $11, %zmm0, %zmm0
+; CHECK-NEXT:    vcvttpd2udq %zmm0, %ymm0
+; CHECK-NEXT:    vcvtudq2pd %ymm0, %zmm0
 ; CHECK-NEXT:    retq
   %fptoui = tail call <8 x i32> @llvm.x86.avx512.mask.cvttpd2udq.512(<8 x double> %x, <8 x i32> undef, i8 -1, i32 4)
   %uitofp = uitofp <8 x i32> %fptoui to <8 x double>
   ret <8 x double> %uitofp
 }
 
 define <4 x float> @float_to_sint64_to_float_mem_v4f32(<4 x float>* %p) {
 ; CHECK-LABEL: float_to_sint64_to_float_mem_v4f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundps $11, (%rdi), %xmm0
+; CHECK-NEXT:    vcvttps2qq (%rdi), %ymm0
+; CHECK-NEXT:    vcvtqq2ps %ymm0, %xmm0
+; CHECK-NEXT:    vzeroupper
 ; CHECK-NEXT:    retq
   %x = load <4 x float>, <4 x float>* %p
   %fptosi = tail call <4 x i64> @llvm.x86.avx512.mask.cvttps2qq.256(<4 x float> %x, <4 x i64> undef, i8 -1)
   %sitofp = sitofp <4 x i64> %fptosi to <4 x float>
   ret <4 x float> %sitofp
 }
 
 define <4 x float> @float_to_sint64_to_float_reg_v4f32(<4 x float> %x) {
 ; CHECK-LABEL: float_to_sint64_to_float_reg_v4f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundps $11, %xmm0, %xmm0
+; CHECK-NEXT:    vcvttps2qq %xmm0, %ymm0
+; CHECK-NEXT:    vcvtqq2ps %ymm0, %xmm0
+; CHECK-NEXT:    vzeroupper
 ; CHECK-NEXT:    retq
   %fptosi = tail call <4 x i64> @llvm.x86.avx512.mask.cvttps2qq.256(<4 x float> %x, <4 x i64> undef, i8 -1)
   %sitofp = sitofp <4 x i64> %fptosi to <4 x float>
   ret <4 x float> %sitofp
 }
 
 define <4 x float> @float_to_uint64_to_float_mem_v4f32(<4 x float>* %p) {
 ; CHECK-LABEL: float_to_uint64_to_float_mem_v4f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundps $11, (%rdi), %xmm0
+; CHECK-NEXT:    vcvttps2uqq (%rdi), %ymm0
+; CHECK-NEXT:    vcvtuqq2ps %ymm0, %xmm0
+; CHECK-NEXT:    vzeroupper
 ; CHECK-NEXT:    retq
   %x = load <4 x float>, <4 x float>* %p
   %fptoui = tail call <4 x i64> @llvm.x86.avx512.mask.cvttps2uqq.256(<4 x float> %x, <4 x i64> undef, i8 -1)
   %uitofp = uitofp <4 x i64> %fptoui to <4 x float>
   ret <4 x float> %uitofp
 }
 
 define <4 x float> @float_to_uint64_to_float_reg_v4f32(<4 x float> %x) {
 ; CHECK-LABEL: float_to_uint64_to_float_reg_v4f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundps $11, %xmm0, %xmm0
+; CHECK-NEXT:    vcvttps2uqq %xmm0, %ymm0
+; CHECK-NEXT:    vcvtuqq2ps %ymm0, %xmm0
+; CHECK-NEXT:    vzeroupper
 ; CHECK-NEXT:    retq
   %fptoui = tail call <4 x i64> @llvm.x86.avx512.mask.cvttps2uqq.256(<4 x float> %x, <4 x i64> undef, i8 -1)
   %uitofp = uitofp <4 x i64> %fptoui to <4 x float>
   ret <4 x float> %uitofp
 }
 
 define <8 x float> @float_to_sint64_to_float_mem_v8f32(<8 x float>* %p) {
 ; CHECK-LABEL: float_to_sint64_to_float_mem_v8f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundps $11, (%rdi), %ymm0
+; CHECK-NEXT:    vcvttps2qq (%rdi), %zmm0
+; CHECK-NEXT:    vcvtqq2ps %zmm0, %ymm0
 ; CHECK-NEXT:    retq
   %x = load <8 x float>, <8 x float>* %p
   %fptosi = tail call <8 x i64> @llvm.x86.avx512.mask.cvttps2qq.512(<8 x float> %x, <8 x i64> undef, i8 -1, i32 4)
   %sitofp = sitofp <8 x i64> %fptosi to <8 x float>
   ret <8 x float> %sitofp
 }
 
 define <8 x float> @float_to_sint64_to_float_reg_v8f32(<8 x float> %x) {
 ; CHECK-LABEL: float_to_sint64_to_float_reg_v8f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundps $11, %ymm0, %ymm0
+; CHECK-NEXT:    vcvttps2qq %ymm0, %zmm0
+; CHECK-NEXT:    vcvtqq2ps %zmm0, %ymm0
 ; CHECK-NEXT:    retq
   %fptosi = tail call <8 x i64> @llvm.x86.avx512.mask.cvttps2qq.512(<8 x float> %x, <8 x i64> undef, i8 -1, i32 4)
   %sitofp = sitofp <8 x i64> %fptosi to <8 x float>
   ret <8 x float> %sitofp
 }
 
 define <8 x float> @float_to_uint64_to_float_mem_v8f32(<8 x float>* %p) {
 ; CHECK-LABEL: float_to_uint64_to_float_mem_v8f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundps $11, (%rdi), %ymm0
+; CHECK-NEXT:    vcvttps2uqq (%rdi), %zmm0
+; CHECK-NEXT:    vcvtuqq2ps %zmm0, %ymm0
 ; CHECK-NEXT:    retq
   %x = load <8 x float>, <8 x float>* %p
   %fptoui = tail call <8 x i64> @llvm.x86.avx512.mask.cvttps2uqq.512(<8 x float> %x, <8 x i64> undef, i8 -1, i32 4)
   %uitofp = uitofp <8 x i64> %fptoui to <8 x float>
   ret <8 x float> %uitofp
 }
 
 define <8 x float> @float_to_uint64_to_float_reg_v8f32(<8 x float> %x) {
 ; CHECK-LABEL: float_to_uint64_to_float_reg_v8f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundps $11, %ymm0, %ymm0
+; CHECK-NEXT:    vcvttps2uqq %ymm0, %zmm0
+; CHECK-NEXT:    vcvtuqq2ps %zmm0, %ymm0
 ; CHECK-NEXT:    retq
   %fptoui = tail call <8 x i64> @llvm.x86.avx512.mask.cvttps2uqq.512(<8 x float> %x, <8 x i64> undef, i8 -1, i32 4)
   %uitofp = uitofp <8 x i64> %fptoui to <8 x float>
   ret <8 x float> %uitofp
 }
 
 define <2 x double> @double_to_sint64_to_double_mem_v2f64(<2 x double>* %p) {
 ; CHECK-LABEL: double_to_sint64_to_double_mem_v2f64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundpd $11, (%rdi), %xmm0
+; CHECK-NEXT:    vcvttpd2qq (%rdi), %xmm0
+; CHECK-NEXT:    vcvtqq2pd %xmm0, %xmm0
 ; CHECK-NEXT:    retq
   %x = load <2 x double>, <2 x double>* %p
   %fptosi = tail call <2 x i64> @llvm.x86.avx512.mask.cvttpd2qq.128(<2 x double> %x, <2 x i64> undef, i8 -1)
   %sitofp = sitofp <2 x i64> %fptosi to <2 x double>
   ret <2 x double> %sitofp
 }
 
 define <2 x double> @double_to_sint64_to_double_reg_v2f64(<2 x double> %x) {
 ; CHECK-LABEL: double_to_sint64_to_double_reg_v2f64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundpd $11, %xmm0, %xmm0
+; CHECK-NEXT:    vcvttpd2qq %xmm0, %xmm0
+; CHECK-NEXT:    vcvtqq2pd %xmm0, %xmm0
 ; CHECK-NEXT:    retq
   %fptosi = tail call <2 x i64> @llvm.x86.avx512.mask.cvttpd2qq.128(<2 x double> %x, <2 x i64> undef, i8 -1)
   %sitofp = sitofp <2 x i64> %fptosi to <2 x double>
   ret <2 x double> %sitofp
 }
 
 define <2 x double> @double_to_uint64_to_double_mem_v2f64(<2 x double>* %p) {
 ; CHECK-LABEL: double_to_uint64_to_double_mem_v2f64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundpd $11, (%rdi), %xmm0
+; CHECK-NEXT:    vcvttpd2uqq (%rdi), %xmm0
+; CHECK-NEXT:    vcvtuqq2pd %xmm0, %xmm0
 ; CHECK-NEXT:    retq
   %x = load <2 x double>, <2 x double>* %p
   %fptoui = tail call <2 x i64> @llvm.x86.avx512.mask.cvttpd2uqq.128(<2 x double> %x, <2 x i64> undef, i8 -1)
   %uitofp = uitofp <2 x i64> %fptoui to <2 x double>
   ret <2 x double> %uitofp
 }
 
 define <2 x double> @double_to_uint64_to_double_reg_v2f64(<2 x double> %x) {
 ; CHECK-LABEL: double_to_uint64_to_double_reg_v2f64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundpd $11, %xmm0, %xmm0
+; CHECK-NEXT:    vcvttpd2uqq %xmm0, %xmm0
+; CHECK-NEXT:    vcvtuqq2pd %xmm0, %xmm0
 ; CHECK-NEXT:    retq
   %fptoui = tail call <2 x i64> @llvm.x86.avx512.mask.cvttpd2uqq.128(<2 x double> %x, <2 x i64> undef, i8 -1)
   %uitofp = uitofp <2 x i64> %fptoui to <2 x double>
   ret <2 x double> %uitofp
 }
 
 define <4 x double> @double_to_sint64_to_double_mem_v4f64(<4 x double>* %p) {
 ; CHECK-LABEL: double_to_sint64_to_double_mem_v4f64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundpd $11, (%rdi), %ymm0
+; CHECK-NEXT:    vcvttpd2qq (%rdi), %ymm0
+; CHECK-NEXT:    vcvtqq2pd %ymm0, %ymm0
 ; CHECK-NEXT:    retq
   %x = load <4 x double>, <4 x double>* %p
   %fptosi = tail call <4 x i64> @llvm.x86.avx512.mask.cvttpd2qq.256(<4 x double> %x, <4 x i64> undef, i8 -1)
   %sitofp = sitofp <4 x i64> %fptosi to <4 x double>
   ret <4 x double> %sitofp
 }
 
 define <4 x double> @double_to_sint64_to_double_reg_v4f64(<4 x double> %x) {
 ; CHECK-LABEL: double_to_sint64_to_double_reg_v4f64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundpd $11, %ymm0, %ymm0
+; CHECK-NEXT:    vcvttpd2qq %ymm0, %ymm0
+; CHECK-NEXT:    vcvtqq2pd %ymm0, %ymm0
 ; CHECK-NEXT:    retq
   %fptosi = tail call <4 x i64> @llvm.x86.avx512.mask.cvttpd2qq.256(<4 x double> %x, <4 x i64> undef, i8 -1)
   %sitofp = sitofp <4 x i64> %fptosi to <4 x double>
   ret <4 x double> %sitofp
 }
 
 define <4 x double> @double_to_uint64_to_double_mem_v4f64(<4 x double>* %p) {
 ; CHECK-LABEL: double_to_uint64_to_double_mem_v4f64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundpd $11, (%rdi), %ymm0
+; CHECK-NEXT:    vcvttpd2uqq (%rdi), %ymm0
+; CHECK-NEXT:    vcvtuqq2pd %ymm0, %ymm0
 ; CHECK-NEXT:    retq
   %x = load <4 x double>, <4 x double>* %p
   %fptoui = tail call <4 x i64> @llvm.x86.avx512.mask.cvttpd2uqq.256(<4 x double> %x, <4 x i64> undef, i8 -1)
   %uitofp = uitofp <4 x i64> %fptoui to <4 x double>
   ret <4 x double> %uitofp
 }
 
 define <4 x double> @double_to_uint64_to_double_reg_v4f64(<4 x double> %x) {
 ; CHECK-LABEL: double_to_uint64_to_double_reg_v4f64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vroundpd $11, %ymm0, %ymm0
+; CHECK-NEXT:    vcvttpd2uqq %ymm0, %ymm0
+; CHECK-NEXT:    vcvtuqq2pd %ymm0, %ymm0
 ; CHECK-NEXT:    retq
   %fptoui = tail call <4 x i64> @llvm.x86.avx512.mask.cvttpd2uqq.256(<4 x double> %x, <4 x i64> undef, i8 -1)
   %uitofp = uitofp <4 x i64> %fptoui to <4 x double>
   ret <4 x double> %uitofp
 }
 
 define <8 x double> @double_to_sint64_to_double_mem_v8f64(<8 x double>* %p) {
 ; CHECK-LABEL: double_to_sint64_to_double_mem_v8f64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vrndscalepd $11, (%rdi), %zmm0
+; CHECK-NEXT:    vcvttpd2qq (%rdi), %zmm0
+; CHECK-NEXT:    vcvtqq2pd %zmm0, %zmm0
 ; CHECK-NEXT:    retq
   %x = load <8 x double>, <8 x double>* %p
   %fptosi = tail call <8 x i64> @llvm.x86.avx512.mask.cvttpd2qq.512(<8 x double> %x, <8 x i64> undef, i8 -1, i32 4)
   %sitofp = sitofp <8 x i64> %fptosi to <8 x double>
   ret <8 x double> %sitofp
 }
 
 define <8 x double> @double_to_sint64_to_double_reg_v8f64(<8 x double> %x) {
 ; CHECK-LABEL: double_to_sint64_to_double_reg_v8f64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vrndscalepd $11, %zmm0, %zmm0
+; CHECK-NEXT:    vcvttpd2qq %zmm0, %zmm0
+; CHECK-NEXT:    vcvtqq2pd %zmm0, %zmm0
 ; CHECK-NEXT:    retq
   %fptosi = tail call <8 x i64> @llvm.x86.avx512.mask.cvttpd2qq.512(<8 x double> %x, <8 x i64> undef, i8 -1, i32 4)
   %sitofp = sitofp <8 x i64> %fptosi to <8 x double>
   ret <8 x double> %sitofp
 }
 
 define <8 x double> @double_to_uint64_to_double_mem_v8f64(<8 x double>* %p) {
 ; CHECK-LABEL: double_to_uint64_to_double_mem_v8f64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vrndscalepd $11, (%rdi), %zmm0
+; CHECK-NEXT:    vcvttpd2uqq (%rdi), %zmm0
+; CHECK-NEXT:    vcvtuqq2pd %zmm0, %zmm0
 ; CHECK-NEXT:    retq
   %x = load <8 x double>, <8 x double>* %p
   %fptoui = tail call <8 x i64> @llvm.x86.avx512.mask.cvttpd2uqq.512(<8 x double> %x, <8 x i64> undef, i8 -1, i32 4)
   %uitofp = uitofp <8 x i64> %fptoui to <8 x double>
   ret <8 x double> %uitofp
 }
 
 define <8 x double> @double_to_uint64_to_double_reg_v8f64(<8 x double> %x) {
 ; CHECK-LABEL: double_to_uint64_to_double_reg_v8f64:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vrndscalepd $11, %zmm0, %zmm0
+; CHECK-NEXT:    vcvttpd2uqq %zmm0, %zmm0
+; CHECK-NEXT:    vcvtuqq2pd %zmm0, %zmm0
 ; CHECK-NEXT:    retq
   %fptoui = tail call <8 x i64> @llvm.x86.avx512.mask.cvttpd2uqq.512(<8 x double> %x, <8 x i64> undef, i8 -1, i32 4)
   %uitofp = uitofp <8 x i64> %fptoui to <8 x double>
   ret <8 x double> %uitofp
 }

llvm/trunk/test/CodeGen/X86/sse-cvttp2si.ll

; AVX-NEXT:    retq
   %fptosi = tail call i64 @llvm.x86.sse2.cvttsd2si64(<2 x double> %x)
   %sitofp = sitofp i64 %fptosi to double
   ret double %sitofp
 }
 
 define <4 x float> @float_to_int_to_float_mem_v4f32(<4 x float>* %p) {
 ; SSE-LABEL: float_to_int_to_float_mem_v4f32:
 ; SSE:       # %bb.0:
-; SSE-NEXT:    roundps $11, (%rdi), %xmm0
+; SSE-NEXT:    cvttps2dq (%rdi), %xmm0
+; SSE-NEXT:    cvtdq2ps %xmm0, %xmm0
 ; SSE-NEXT:    retq
 ;
 ; AVX-LABEL: float_to_int_to_float_mem_v4f32:
 ; AVX:       # %bb.0:
-; AVX-NEXT:    vroundps $11, (%rdi), %xmm0
+; AVX-NEXT:    vcvttps2dq (%rdi), %xmm0
+; AVX-NEXT:    vcvtdq2ps %xmm0, %xmm0
 ; AVX-NEXT:    retq
   %x = load <4 x float>, <4 x float>* %p, align 16
   %fptosi = tail call <4 x i32> @llvm.x86.sse2.cvttps2dq(<4 x float> %x)
   %sitofp = sitofp <4 x i32> %fptosi to <4 x float>
   ret <4 x float> %sitofp
 }
 
 define <4 x float> @float_to_int_to_float_reg_v4f32(<4 x float> %x) {
 ; SSE-LABEL: float_to_int_to_float_reg_v4f32:
 ; SSE:       # %bb.0:
-; SSE-NEXT:    roundps $11, %xmm0, %xmm0
+; SSE-NEXT:    cvttps2dq %xmm0, %xmm0
+; SSE-NEXT:    cvtdq2ps %xmm0, %xmm0
 ; SSE-NEXT:    retq
 ;
 ; AVX-LABEL: float_to_int_to_float_reg_v4f32:
 ; AVX:       # %bb.0:
-; AVX-NEXT:    vroundps $11, %xmm0, %xmm0
+; AVX-NEXT:    vcvttps2dq %xmm0, %xmm0
+; AVX-NEXT:    vcvtdq2ps %xmm0, %xmm0
 ; AVX-NEXT:    retq
   %fptosi = tail call <4 x i32> @llvm.x86.sse2.cvttps2dq(<4 x float> %x)
   %sitofp = sitofp <4 x i32> %fptosi to <4 x float>
   ret <4 x float> %sitofp
 }
 
 define <2 x double> @float_to_int_to_float_mem_v2f64(<2 x double>* %p) {
 ; SSE-LABEL: float_to_int_to_float_mem_v2f64: