[X86] Add SchedRW for PMULLD

[craig.topper]

Summary

It seems many CPUs don't implement this instruction as well as the other vector multiplies. Often using a multi uop flow. Silvermont in particular has a 7 uop flow with 11 cycle throughput. Sandy Bridge implements it as a single uop with 5 cycle latency and 1 cycle throughput. But Haswell and later use 2 uops with 10 cycle latency and 2 cycle throughput.

This patch adds a new X86SchedWritePair we can use to tag this instruction separately. I've provided correct information for Silvermont, Btver2, and Sandy Bridge. I've removed the InstRWs for SandyBridge. I've left Haswell/Broadwell/Skylake InstRWs in place because I wasn't sure how to account for the different load latency between 128 and 256 bits. I also left Znver1 InstRWs in place because the existing values don't match Agner's spreadsheet.

I also left a FIXME in the SandyBridge model because it being used for the "generic" model is too optimistic for the 256/512-bit versions since those are multiple uops on all known CPUs.

[courbet]

lib/Target/X86/X86SchedSkylakeClient.td

163

llvm-exegesis measures 2xP01 here.

[RKSimon]

You will need to check the llvm-mca tests as well - some of those will have changed (sorry no update script - its manual!)

[craig.topper]

Fix the ports for SKL. Remove the register form override from most of the schedulers. I left the memory form overrides in place. Update the llvm-mca tests.

[courbet]

lib/Target/X86/X86SchedSkylakeServer.td

163

I don't have a skylake server to test that, but I'm surprised that this is different from SKL. Is this a typo ?

[andreadb]

The btver2 change looks good to me. Thanks!

[RKSimon]

lib/Target/X86/X86SchedBroadwell.td

166

Remove BWWriteResGroup148 ((V?)PMULLDrm) overload and just leave BWWriteResGroup151 (VPMULLDYrm) ?

Same for others.

[craig.topper]

lib/Target/X86/X86SchedSkylakeServer.td

163

SKX adds an extra FMA unit and vector multiplier in port 5 for AVX512. 512-bit operations combine the 256-bit port0 and 1 units. So an extra unit was added to maintain 2 ports for 512-bit.

I’m not sure the port 5 unit can be used for 128 and 256 bit, but the scheduler model thinks so. The scheduler model definitely doesn’t model 512 bit correctly, but that’s a larger problem than I want to fix here.

[craig.topper]

Remove 128-bit memory instructions from most of the models. Didn't touch Skylake Server because there are many things about it that I don't understand.

[GGanesh]

lib/Target/X86/X86ScheduleZnver1.td

212

This needs a fix definitely. I will do it!

[RKSimon]

LGTM and then @GGanesh can fix the Zn model afterward.

lib/Target/X86/X86InstrAVX512.td

defm VPADDS : avx512_binop_rm_vl_bw<0xEC, 0xED, "vpadds", X86adds,
                                     SSE_INTALU_ITINS_P, HasBWI, 1>;
 defm VPSUBS : avx512_binop_rm_vl_bw<0xE8, 0xE9, "vpsubs", X86subs,
                                     SSE_INTALU_ITINS_P, HasBWI, 0>;
 defm VPADDUS : avx512_binop_rm_vl_bw<0xDC, 0xDD, "vpaddus", X86addus,
                                      SSE_INTALU_ITINS_P, HasBWI, 1>;
 defm VPSUBUS : avx512_binop_rm_vl_bw<0xD8, 0xD9, "vpsubus", X86subus,
                                      SSE_INTALU_ITINS_P, HasBWI, 0>;
 defm VPMULLD : avx512_binop_rm_vl_d<0x40, "vpmulld", mul,
-                                    SSE_INTMUL_ITINS_P, HasAVX512, 1>, T8PD;
+                                    SSE_PMULLD_ITINS, HasAVX512, 1>, T8PD;
 defm VPMULLW : avx512_binop_rm_vl_w<0xD5, "vpmullw", mul,
                                     SSE_INTMUL_ITINS_P, HasBWI, 1>;
 defm VPMULLQ : avx512_binop_rm_vl_q<0x40, "vpmullq", mul,
                                     SSE_INTMUL_ITINS_P, HasDQI, 1>, T8PD;
 defm VPMULHW : avx512_binop_rm_vl_w<0xE5, "vpmulhw", mulhs, SSE_INTMUL_ITINS_P,
                                     HasBWI, 1>;
 defm VPMULHUW : avx512_binop_rm_vl_w<0xE4, "vpmulhuw", mulhu, SSE_INTMUL_ITINS_P,
                                      HasBWI, 1>;

lib/Target/X86/X86InstrSSE.td

def SSE_INSERT_ITINS : OpndItins<
   IIC_SSE_INSERTPS_RR, IIC_SSE_INSERTPS_RM
 >;
 
 let Sched = WriteMPSAD in
 def SSE_MPSADBW_ITINS : OpndItins<
   IIC_SSE_MPSADBW_RR, IIC_SSE_MPSADBW_RM
 >;
 
-let Sched = WriteVecIMul in
+let Sched = WritePMULLD in
 def SSE_PMULLD_ITINS : OpndItins<
   IIC_SSE_PMULLD_RR, IIC_SSE_PMULLD_RM
 >;
 
 // Definitions for backward compatibility.
 // The instructions mapped on these definitions uses a different itinerary
 // than the actual scheduling model.
 let Sched = WriteShuffle in

lib/Target/X86/X86SchedBroadwell.td

 // Vector integer operations.
 def  : WriteRes<WriteVecLoad,        [BWPort23]> { let Latency = 5; }
 def  : WriteRes<WriteVecStore,       [BWPort237, BWPort4]>;
 def  : WriteRes<WriteVecMove,        [BWPort015]>;
 
 defm : BWWriteResPair<WriteVecALU,   [BWPort15],  1>; // Vector integer ALU op, no logicals.
 defm : BWWriteResPair<WriteVecShift, [BWPort0],  1>; // Vector integer shifts.
 defm : BWWriteResPair<WriteVecIMul,  [BWPort0],   5>; // Vector integer multiply.
+defm : BWWriteResPair<WritePMULLD,   [BWPort0], 10, [2], 2, 5>; // PMULLD
 defm : BWWriteResPair<WriteShuffle,  [BWPort5],  1>; // Vector shuffles.
 defm : BWWriteResPair<WriteBlend,  [BWPort15],  1>; // Vector blends.
 defm : BWWriteResPair<WriteVarBlend,  [BWPort5], 2, [2]>; // Vector variable blends.
 defm : BWWriteResPair<WriteMPSAD,  [BWPort0, BWPort5], 6, [1, 2]>; // Vector MPSAD.
 
 // Vector bitwise operations.
 // These are often used on both floating point and integer vectors.
 defm : BWWriteResPair<WriteVecLogic, [BWPort015], 1>; // Vector and/or/xor.

 def BWWriteResGroup113 : SchedWriteRes<[BWPort1,BWPort6,BWPort23,BWPort0156]> {
   let Latency = 9;
   let NumMicroOps = 5;
   let ResourceCycles = [1,2,1,1];
 }
 def: InstRW<[BWWriteResGroup113], (instregex "LAR(16|32|64)rm",
                                              "LSL(16|32|64)rm")>;
 
-def BWWriteResGroup114 : SchedWriteRes<[BWPort0]> {
-  let Latency = 10;
-  let NumMicroOps = 2;
-  let ResourceCycles = [2];
-}
-def: InstRW<[BWWriteResGroup114], (instregex "(V?)PMULLD(Y?)rr")>;
-
 def BWWriteResGroup115 : SchedWriteRes<[BWPort0,BWPort23]> {
   let Latency = 10;
   let NumMicroOps = 2;
   let ResourceCycles = [1,1];
 }
 def: InstRW<[BWWriteResGroup115], (instregex "MMX_PMADDUBSWrm",
                                              "MMX_PMADDWDirm",
                                              "MMX_PMULHRSWrm",
def BWWriteResGroup147 : SchedWriteRes<[BWPort0]> {
   let Latency = 15;
   let NumMicroOps = 1;
   let ResourceCycles = [1];
 }
 def: InstRW<[BWWriteResGroup147], (instregex "DIVR_FPrST0",
                                              "DIVR_FST0r",
                                              "DIVR_FrST0")>;
 
-def BWWriteResGroup148 : SchedWriteRes<[BWPort0,BWPort23]> {
-  let Latency = 15;
-  let NumMicroOps = 3;
-  let ResourceCycles = [2,1];
-}
-def: InstRW<[BWWriteResGroup148], (instregex "(V?)PMULLDrm")>;
-
 def BWWriteResGroup149 : SchedWriteRes<[BWPort1,BWPort23,BWPort237,BWPort06,BWPort15,BWPort0156]> {
   let Latency = 15;
   let NumMicroOps = 10;
   let ResourceCycles = [1,1,1,4,1,2];
 }
 def: InstRW<[BWWriteResGroup149], (instregex "RCL(8|16|32|64)mCL")>;
 
 def BWWriteResGroup150 : SchedWriteRes<[BWPort0,BWPort23]> {

lib/Target/X86/X86SchedHaswell.td

 def  : WriteRes<WriteVecStore,       [HWPort237, HWPort4]>;
 def  : WriteRes<WriteVecLoad,        [HWPort23]> { let Latency = 5; }
 def  : WriteRes<WriteVecMove,        [HWPort015]>;
 
 defm : HWWriteResPair<WriteVecShift, [HWPort0],  1>;
 defm : HWWriteResPair<WriteVecLogic, [HWPort015], 1>;
 defm : HWWriteResPair<WriteVecALU,   [HWPort15],  1>;
 defm : HWWriteResPair<WriteVecIMul,  [HWPort0],   5>;
+defm : HWWriteResPair<WritePMULLD,   [HWPort0], 10, [2], 2, 6>;
 defm : HWWriteResPair<WriteShuffle,  [HWPort5],  1>;
 defm : HWWriteResPair<WriteBlend,  [HWPort15],  1>;
 defm : HWWriteResPair<WriteShuffle256,  [HWPort5],  3>;
 defm : HWWriteResPair<WriteVarBlend,  [HWPort5], 2, [2]>;
 defm : HWWriteResPair<WriteVarVecShift,  [HWPort0, HWPort5], 2, [2, 1]>;
 defm : HWWriteResPair<WriteMPSAD,  [HWPort0, HWPort5], 6, [1, 2]>;
 
 // String instructions.

 
 def HWWriteResGroup117 : SchedWriteRes<[HWPort0,HWPort1,HWPort5,HWPort23]> {
   let Latency = 15;
   let NumMicroOps = 4;
   let ResourceCycles = [1,1,1,1];
 }
 def: InstRW<[HWWriteResGroup117], (instregex "(V?)DPPDrmi")>;
 
-def HWWriteResGroup118 : SchedWriteRes<[HWPort0]> {
-  let Latency = 10;
-  let NumMicroOps = 2;
-  let ResourceCycles = [2];
-}
-def: InstRW<[HWWriteResGroup118], (instregex "(V?)PMULLD(Y?)rr")>;
-
-def HWWriteResGroup119 : SchedWriteRes<[HWPort0,HWPort23]> {
-  let Latency = 16;
-  let NumMicroOps = 3;
-  let ResourceCycles = [2,1];
-}
-def: InstRW<[HWWriteResGroup119], (instregex "(V?)PMULLDrm")>;
-
 def HWWriteResGroup119_1 : SchedWriteRes<[HWPort0,HWPort23]> {
   let Latency = 17;
   let NumMicroOps = 3;
   let ResourceCycles = [2,1];
 }
 def: InstRW<[HWWriteResGroup119_1], (instregex "VPMULLDYrm")>;
 
 def HWWriteResGroup120 : SchedWriteRes<[HWPort1,HWPort23,HWPort237,HWPort06,HWPort15,HWPort0156]> {

lib/Target/X86/X86SchedSandyBridge.td

 def  : WriteRes<WriteVecStore,       [SBPort23, SBPort4]>;
 def  : WriteRes<WriteVecLoad,        [SBPort23]> { let Latency = 6; }
 def  : WriteRes<WriteVecMove,        [SBPort05]>;
 
 defm : SBWriteResPair<WriteVecShift, [SBPort5], 1>;
 defm : SBWriteResPair<WriteVecLogic, [SBPort5], 1>;
 defm : SBWriteResPair<WriteVecALU,   [SBPort1], 3>;
 defm : SBWriteResPair<WriteVecIMul,  [SBPort0], 5>;
+defm : SBWriteResPair<WritePMULLD,   [SBPort0], 5, [1], 1, 6>; // TODO this is probably wrong for 256/512-bit for the "generic" model
 defm : SBWriteResPair<WriteShuffle,  [SBPort5], 1>;
 defm : SBWriteResPair<WriteBlend,   [SBPort15], 1>;
 defm : SBWriteResPair<WriteVarBlend, [SBPort1, SBPort5], 2>;
 defm : SBWriteResPair<WriteMPSAD, [SBPort0, SBPort15], 5, [1,2], 3>;
 
 ////////////////////////////////////////////////////////////////////////////////
 // Horizontal add/sub  instructions.
 ////////////////////////////////////////////////////////////////////////////////
def: InstRW<[SBWriteResGroup20], (instregex "MMX_PMADDUBSWrr",
                                             "MMX_PMULUDQirr",
                                             "MMX_PSADBWirr",
                                             "(V?)PMADDUBSWrr",
                                             "(V?)PMADDWDrr",
                                             "(V?)PMULDQrr",
                                             "(V?)PMULHRSWrr",
                                             "(V?)PMULHUWrr",
                                             "(V?)PMULHWrr",
-                                            "(V?)PMULLDrr",
                                             "(V?)PMULLWrr",
                                             "(V?)PMULUDQrr",
                                             "(V?)PSADBWrr")>;
 
 def SBWriteResGroup21 : SchedWriteRes<[SBPort1]> {
   let Latency = 3;
   let NumMicroOps = 1;
   let ResourceCycles = [1];
def SBWriteResGroup89 : SchedWriteRes<[SBPort0,SBPort23]> {
   let ResourceCycles = [1,1];
 }
 def: InstRW<[SBWriteResGroup89], (instregex "(V?)PMADDUBSWrm",
                                             "(V?)PMADDWDrm",
                                             "(V?)PMULDQrm",
                                             "(V?)PMULHRSWrm",
                                             "(V?)PMULHUWrm",
                                             "(V?)PMULHWrm",
-                                            "(V?)PMULLDrm",
                                             "(V?)PMULLWrm",
                                             "(V?)PMULUDQrm",
                                             "(V?)PSADBWrm")>;
 
 def SBWriteResGroup89_2 : SchedWriteRes<[SBPort0,SBPort23]> {
   let Latency = 10;
   let NumMicroOps = 2;
   let ResourceCycles = [1,1];

lib/Target/X86/X86SchedSkylakeClient.td

 // Vector integer operations.
 def  : WriteRes<WriteVecLoad,         [SKLPort23]> { let Latency = 6; }
 def  : WriteRes<WriteVecStore,        [SKLPort237, SKLPort4]>;
 def  : WriteRes<WriteVecMove,         [SKLPort015]>;
 
 defm : SKLWriteResPair<WriteVecALU,   [SKLPort15],  1>; // Vector integer ALU op, no logicals.
 defm : SKLWriteResPair<WriteVecShift, [SKLPort0],  1>; // Vector integer shifts.
 defm : SKLWriteResPair<WriteVecIMul,  [SKLPort0],   5>; // Vector integer multiply.
+defm : SKLWriteResPair<WritePMULLD,   [SKLPort01], 10, [2], 2, 6>;
 defm : SKLWriteResPair<WriteShuffle,  [SKLPort5],  1>; // Vector shuffles.
 defm : SKLWriteResPair<WriteBlend,  [SKLPort15],  1>; // Vector blends.
 defm : SKLWriteResPair<WriteVarBlend,  [SKLPort5], 2, [2]>; // Vector variable blends.
 defm : SKLWriteResPair<WriteMPSAD,  [SKLPort0, SKLPort5], 6, [1, 2]>; // Vector MPSAD.
 
 // Vector bitwise operations.
 // These are often used on both floating point and integer vectors.
 defm : SKLWriteResPair<WriteVecLogic, [SKLPort015], 1>; // Vector and/or/xor.
def SKLWriteResGroup105 : SchedWriteRes<[SKLPort01]> {
   let NumMicroOps = 2;
   let ResourceCycles = [2];
 }
 def: InstRW<[SKLWriteResGroup105], (instregex "(V?)ROUNDPD(Y?)r",
                                               "(V?)ROUNDPS(Y?)r",
                                               "(V?)ROUNDSDr",
                                               "(V?)ROUNDSSr")>;
 
-def SKLWriteResGroup105_2 : SchedWriteRes<[SKLPort01]> {
-  let Latency = 10;
-  let NumMicroOps = 2;
-  let ResourceCycles = [2];
-}
-def: InstRW<[SKLWriteResGroup105_2], (instregex "(V?)PMULLD(Y?)rr")>;
-
 def SKLWriteResGroup106 : SchedWriteRes<[SKLPort0,SKLPort23]> {
   let Latency = 8;
   let NumMicroOps = 2;
   let ResourceCycles = [1,1];
 }
 def: InstRW<[SKLWriteResGroup106], (instregex "VTESTPDrm",
                                               "VTESTPSrm")>;
 
def SKLWriteResGroup168 : SchedWriteRes<[SKLPort23,SKLPort01]> {
   let NumMicroOps = 3;
   let ResourceCycles = [1,2];
 }
 def: InstRW<[SKLWriteResGroup168], (instregex "(V?)ROUNDPDm")>;
 def: InstRW<[SKLWriteResGroup168], (instregex "(V?)ROUNDPSm")>;
 def: InstRW<[SKLWriteResGroup168], (instregex "(V?)ROUNDSDm")>;
 def: InstRW<[SKLWriteResGroup168], (instregex "(V?)ROUNDSSm")>;
 
-def SKLWriteResGroup168_2 : SchedWriteRes<[SKLPort23,SKLPort01]> {
-  let Latency = 16;
-  let NumMicroOps = 3;
-  let ResourceCycles = [1,2];
-}
-def: InstRW<[SKLWriteResGroup168_2], (instregex "(V?)PMULLDrm")>;
-
 def SKLWriteResGroup169 : SchedWriteRes<[SKLPort0,SKLPort5,SKLPort23]> {
   let Latency = 14;
   let NumMicroOps = 3;
   let ResourceCycles = [1,1,1];
 }
 def: InstRW<[SKLWriteResGroup169], (instregex "MUL_FI16m",
                                               "MUL_FI32m")>;
 

lib/Target/X86/X86SchedSkylakeServer.td

 // Vector integer operations.
 def  : WriteRes<WriteVecLoad,         [SKXPort23]> { let Latency = 5; }
 def  : WriteRes<WriteVecStore,        [SKXPort237, SKXPort4]>;
 def  : WriteRes<WriteVecMove,         [SKXPort015]>;
 
 defm : SKXWriteResPair<WriteVecALU,   [SKXPort15],  1>; // Vector integer ALU op, no logicals.
 defm : SKXWriteResPair<WriteVecShift, [SKXPort0],  1>; // Vector integer shifts.
 defm : SKXWriteResPair<WriteVecIMul,  [SKXPort0],   5>; // Vector integer multiply.
+defm : SKXWriteResPair<WritePMULLD,   [SKXPort015], 10, [2], 2, 6>; // Vector integer multiply.
 defm : SKXWriteResPair<WriteShuffle,  [SKXPort5],  1>; // Vector shuffles.
 defm : SKXWriteResPair<WriteBlend,  [SKXPort15],  1>; // Vector blends.
 defm : SKXWriteResPair<WriteVarBlend,  [SKXPort5], 2, [2]>; // Vector variable blends.
 defm : SKXWriteResPair<WriteMPSAD,  [SKXPort0, SKXPort5], 6, [1, 2]>; // Vector MPSAD.
 
 // Vector bitwise operations.
 // These are often used on both floating point and integer vectors.
 defm : SKXWriteResPair<WriteVecLogic, [SKXPort015], 1>; // Vector and/or/xor.

lib/Target/X86/X86Schedule.td

 
 // Vector integer operations.
 def  WriteVecLoad  : SchedWrite;
 def  WriteVecStore : SchedWrite;
 def  WriteVecMove  : SchedWrite;
 defm WriteVecALU   : X86SchedWritePair; // Vector integer ALU op, no logicals.
 defm WriteVecShift : X86SchedWritePair; // Vector integer shifts.
 defm WriteVecIMul  : X86SchedWritePair; // Vector integer multiply.
+defm WritePMULLD : X86SchedWritePair; // PMULLD
 defm WriteShuffle  : X86SchedWritePair; // Vector shuffles.
 defm WriteBlend  : X86SchedWritePair; // Vector blends.
 defm WriteVarBlend  : X86SchedWritePair; // Vector variable blends.
 defm WriteMPSAD : X86SchedWritePair; // Vector MPSAD.
 
 // Vector bitwise operations.
 // These are often used on both floating point and integer vectors.
 defm WriteVecLogic : X86SchedWritePair; // Vector and/or/xor.

lib/Target/X86/X86ScheduleBtVer2.td

 
 def  : WriteRes<WriteVecLoad,      [JLAGU, JFPU01, JVALU]> { let Latency = 5; }
 def  : WriteRes<WriteVecStore,       [JSAGU, JFPU1, JSTC]>;
 def  : WriteRes<WriteVecMove,             [JFPU01, JVALU]>;
 
 defm : JWriteResFpuPair<WriteVecALU,      [JFPU01, JVALU], 1>;
 defm : JWriteResFpuPair<WriteVecShift,    [JFPU01, JVALU], 1>;
 defm : JWriteResFpuPair<WriteVecIMul,     [JFPU0, JVIMUL], 2>;
+defm : JWriteResFpuPair<WritePMULLD,      [JFPU0, JFPU01, JVIMUL, JVALU], 4, [2, 1, 2, 1], 3>;
 defm : JWriteResFpuPair<WriteMPSAD,       [JFPU0, JVIMUL], 3, [1, 2]>;
 defm : JWriteResFpuPair<WriteShuffle,     [JFPU01, JVALU], 1>;
 defm : JWriteResFpuPair<WriteBlend,       [JFPU01, JVALU], 1>;
 defm : JWriteResFpuPair<WriteVarBlend,    [JFPU01, JVALU], 2, [1, 4], 3>;
 defm : JWriteResFpuPair<WriteVecLogic,    [JFPU01, JVALU], 1>;
 defm : JWriteResFpuPair<WriteShuffle256,  [JFPU01, JVALU], 1>;
 defm : JWriteResFpuPair<WriteVarVecShift, [JFPU01, JVALU], 1>; // NOTE: Doesn't exist on Jaguar.
 

lib/Target/X86/X86ScheduleSLM.td

 def  : WriteRes<WriteVecStore,       [SLM_FPC_RSV01, SLM_MEC_RSV]>;
 def  : WriteRes<WriteVecLoad,        [SLM_MEC_RSV]> { let Latency = 3; }
 def  : WriteRes<WriteVecMove,        [SLM_FPC_RSV01]>;
 
 defm : SLMWriteResPair<WriteVecShift, [SLM_FPC_RSV0],  1>;
 defm : SLMWriteResPair<WriteVecLogic, [SLM_FPC_RSV01], 1>;
 defm : SLMWriteResPair<WriteVecALU,   [SLM_FPC_RSV01],  1>;
 defm : SLMWriteResPair<WriteVecIMul,  [SLM_FPC_RSV0],   4>;
+defm : SLMWriteResPair<WritePMULLD,  [SLM_FPC_RSV0],   11, [11], 7>;
 defm : SLMWriteResPair<WriteShuffle,  [SLM_FPC_RSV0],  1>;
 defm : SLMWriteResPair<WriteBlend,  [SLM_FPC_RSV0],  1>;
 defm : SLMWriteResPair<WriteMPSAD,  [SLM_FPC_RSV0],  7>;
 
 ////////////////////////////////////////////////////////////////////////////////
 // Horizontal add/sub  instructions.
 ////////////////////////////////////////////////////////////////////////////////
 

lib/Target/X86/X86ScheduleZnver1.td

 def  : WriteRes<WriteVecMove,             [ZnFPU]>;
 def  : WriteRes<WriteVecLoad,             [ZnAGU]> { let Latency = 8; }
 
 defm : ZnWriteResFpuPair<WriteVecShift,   [ZnFPU],   1>;
 defm : ZnWriteResFpuPair<WriteVecLogic,   [ZnFPU],   1>;
 defm : ZnWriteResFpuPair<WritePHAdd,      [ZnFPU],   1>;
 defm : ZnWriteResFpuPair<WriteVecALU,     [ZnFPU],   1>;
 defm : ZnWriteResFpuPair<WriteVecIMul,    [ZnFPU0],  4>;
+defm : ZnWriteResFpuPair<WritePMULLD,     [ZnFPU0],  4>; // FIXME
 defm : ZnWriteResFpuPair<WriteShuffle,    [ZnFPU],   1>;
 defm : ZnWriteResFpuPair<WriteBlend,      [ZnFPU01], 1>;
 defm : ZnWriteResFpuPair<WriteShuffle256, [ZnFPU],   2>;
 
 // Vector Shift Operations
 defm : ZnWriteResFpuPair<WriteVarVecShift, [ZnFPU12], 1>;
 
 // MOVMSK Instructions.

test/CodeGen/X86/avx2-schedule.ll

; ZNVER1-NEXT:    retq # sched: [1:0.50]
   ret <16 x i16> %3
 }
 declare <16 x i16> @llvm.x86.avx2.pmulh.w(<16 x i16>, <16 x i16>) nounwind readnone
 
 define <8 x i32> @test_pmulld(<8 x i32> %a0, <8 x i32> %a1, <8 x i32> *%a2) {
 ; GENERIC-LABEL: test_pmulld:
 ; GENERIC:       # %bb.0:
 ; GENERIC-NEXT:    vpmulld %ymm1, %ymm0, %ymm0 # sched: [5:1.00]
-; GENERIC-NEXT:    vpmulld (%rdi), %ymm0, %ymm0 # sched: [9:1.00]
+; GENERIC-NEXT:    vpmulld (%rdi), %ymm0, %ymm0 # sched: [11:1.00]
 ; GENERIC-NEXT:    retq # sched: [1:1.00]
 ;
 ; HASWELL-LABEL: test_pmulld:
 ; HASWELL:       # %bb.0:
 ; HASWELL-NEXT:    vpmulld %ymm1, %ymm0, %ymm0 # sched: [10:2.00]
 ; HASWELL-NEXT:    vpmulld (%rdi), %ymm0, %ymm0 # sched: [17:2.00]
 ; HASWELL-NEXT:    retq # sched: [7:1.00]
 ;

test/CodeGen/X86/slow-pmulld.ll

; AVX-64-NEXT:    retq
   %z = zext <8 x i16> %A to <8 x i32>
   %m = mul nuw nsw <8 x i32> %z, <i32 18778, i32 18778, i32 18778, i32 18778, i32 18778, i32 18778, i32 18778, i32 18778>
   ret <8 x i32> %m
 }
 
 define <16 x i32> @test_mul_v16i32_v16i16_minsize(<16 x i16> %A) minsize {
 ; SLM32-LABEL: test_mul_v16i32_v16i16_minsize:
 ; SLM32:       # %bb.0:
-; SLM32-NEXT:    pshufd {{.*#+}} xmm2 = xmm1[2,3,0,1]
-; SLM32-NEXT:    pmovzxwd {{.*#+}} xmm3 = xmm2[0],zero,xmm2[1],zero,xmm2[2],zero,xmm2[3],zero
-; SLM32-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[2,3,0,1]
-; SLM32-NEXT:    pmovzxwd {{.*#+}} xmm0 = xmm0[0],zero,xmm0[1],zero,xmm0[2],zero,xmm0[3],zero
-; SLM32-NEXT:    pmovzxwd {{.*#+}} xmm4 = xmm2[0],zero,xmm2[1],zero,xmm2[2],zero,xmm2[3],zero
+; SLM32-NEXT:    movdqa {{.*#+}} xmm5 = [18778,18778,18778,18778]
+; SLM32-NEXT:    pshufd {{.*#+}} xmm3 = xmm1[2,3,0,1]
+; SLM32-NEXT:    pshufd {{.*#+}} xmm4 = xmm0[2,3,0,1]
 ; SLM32-NEXT:    pmovzxwd {{.*#+}} xmm2 = xmm1[0],zero,xmm1[1],zero,xmm1[2],zero,xmm1[3],zero
-; SLM32-NEXT:    movdqa {{.*#+}} xmm1 = [18778,18778,18778,18778]
-; SLM32-NEXT:    pmulld %xmm1, %xmm4
-; SLM32-NEXT:    pmulld %xmm1, %xmm0
-; SLM32-NEXT:    pmulld %xmm1, %xmm2
-; SLM32-NEXT:    pmulld %xmm1, %xmm3
-; SLM32-NEXT:    movdqa %xmm4, %xmm1
+; SLM32-NEXT:    pmovzxwd {{.*#+}} xmm0 = xmm0[0],zero,xmm0[1],zero,xmm0[2],zero,xmm0[3],zero
+; SLM32-NEXT:    pmovzxwd {{.*#+}} xmm1 = xmm4[0],zero,xmm4[1],zero,xmm4[2],zero,xmm4[3],zero
+; SLM32-NEXT:    pmovzxwd {{.*#+}} xmm3 = xmm3[0],zero,xmm3[1],zero,xmm3[2],zero,xmm3[3],zero
+; SLM32-NEXT:    pmulld %xmm5, %xmm0
+; SLM32-NEXT:    pmulld %xmm5, %xmm2
+; SLM32-NEXT:    pmulld %xmm5, %xmm1
+; SLM32-NEXT:    pmulld %xmm5, %xmm3
 ; SLM32-NEXT:    retl
 ;
 ; SLM64-LABEL: test_mul_v16i32_v16i16_minsize:
 ; SLM64:       # %bb.0:
-; SLM64-NEXT:    pshufd {{.*#+}} xmm2 = xmm1[2,3,0,1]
-; SLM64-NEXT:    pmovzxwd {{.*#+}} xmm3 = xmm2[0],zero,xmm2[1],zero,xmm2[2],zero,xmm2[3],zero
-; SLM64-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[2,3,0,1]
-; SLM64-NEXT:    pmovzxwd {{.*#+}} xmm0 = xmm0[0],zero,xmm0[1],zero,xmm0[2],zero,xmm0[3],zero
-; SLM64-NEXT:    pmovzxwd {{.*#+}} xmm4 = xmm2[0],zero,xmm2[1],zero,xmm2[2],zero,xmm2[3],zero
+; SLM64-NEXT:    movdqa {{.*#+}} xmm5 = [18778,18778,18778,18778]
+; SLM64-NEXT:    pshufd {{.*#+}} xmm3 = xmm1[2,3,0,1]
+; SLM64-NEXT:    pshufd {{.*#+}} xmm4 = xmm0[2,3,0,1]
 ; SLM64-NEXT:    pmovzxwd {{.*#+}} xmm2 = xmm1[0],zero,xmm1[1],zero,xmm1[2],zero,xmm1[3],zero
-; SLM64-NEXT:    movdqa {{.*#+}} xmm1 = [18778,18778,18778,18778]
-; SLM64-NEXT:    pmulld %xmm1, %xmm4
-; SLM64-NEXT:    pmulld %xmm1, %xmm0
-; SLM64-NEXT:    pmulld %xmm1, %xmm2
-; SLM64-NEXT:    pmulld %xmm1, %xmm3
-; SLM64-NEXT:    movdqa %xmm4, %xmm1
+; SLM64-NEXT:    pmovzxwd {{.*#+}} xmm0 = xmm0[0],zero,xmm0[1],zero,xmm0[2],zero,xmm0[3],zero
+; SLM64-NEXT:    pmovzxwd {{.*#+}} xmm1 = xmm4[0],zero,xmm4[1],zero,xmm4[2],zero,xmm4[3],zero
+; SLM64-NEXT:    pmovzxwd {{.*#+}} xmm3 = xmm3[0],zero,xmm3[1],zero,xmm3[2],zero,xmm3[3],zero
+; SLM64-NEXT:    pmulld %xmm5, %xmm0
+; SLM64-NEXT:    pmulld %xmm5, %xmm2
+; SLM64-NEXT:    pmulld %xmm5, %xmm1
+; SLM64-NEXT:    pmulld %xmm5, %xmm3
 ; SLM64-NEXT:    retq
 ;
 ; SLOW32-LABEL: test_mul_v16i32_v16i16_minsize:
 ; SLOW32:       # %bb.0:
 ; SLOW32-NEXT:    pshufd {{.*#+}} xmm2 = xmm1[2,3,0,1]
 ; SLOW32-NEXT:    pmovzxwd {{.*#+}} xmm3 = xmm2[0],zero,xmm2[1],zero,xmm2[2],zero,xmm2[3],zero
 ; SLOW32-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[2,3,0,1]
 ; SLOW32-NEXT:    pmovzxwd {{.*#+}} xmm4 = xmm2[0],zero,xmm2[1],zero,xmm2[2],zero,xmm2[3],zero

test/CodeGen/X86/sse41-schedule.ll

 ; GENERIC-LABEL: test_pmulld:
 ; GENERIC:       # %bb.0:
 ; GENERIC-NEXT:    pmulld %xmm1, %xmm0 # sched: [5:1.00]
 ; GENERIC-NEXT:    pmulld (%rdi), %xmm0 # sched: [11:1.00]
 ; GENERIC-NEXT:    retq # sched: [1:1.00]
 ;
 ; SLM-LABEL: test_pmulld:
 ; SLM:       # %bb.0:
-; SLM-NEXT:    pmulld %xmm1, %xmm0 # sched: [4:1.00]
-; SLM-NEXT:    pmulld (%rdi), %xmm0 # sched: [7:1.00]
+; SLM-NEXT:    pmulld %xmm1, %xmm0 # sched: [11:11.00]
+; SLM-NEXT:    pmulld (%rdi), %xmm0 # sched: [14:11.00]
 ; SLM-NEXT:    retq # sched: [4:1.00]
 ;
 ; SANDY-SSE-LABEL: test_pmulld:
 ; SANDY-SSE:       # %bb.0:
 ; SANDY-SSE-NEXT:    pmulld %xmm1, %xmm0 # sched: [5:1.00]
 ; SANDY-SSE-NEXT:    pmulld (%rdi), %xmm0 # sched: [11:1.00]
 ; SANDY-SSE-NEXT:    retq # sched: [1:1.00]
 ;

 ; SKX-LABEL: test_pmulld:
 ; SKX:       # %bb.0:
 ; SKX-NEXT:    vpmulld %xmm1, %xmm0, %xmm0 # sched: [10:0.67]
 ; SKX-NEXT:    vpmulld (%rdi), %xmm0, %xmm0 # sched: [16:0.67]
 ; SKX-NEXT:    retq # sched: [7:1.00]
 ;
 ; BTVER2-SSE-LABEL: test_pmulld:
 ; BTVER2-SSE:       # %bb.0:
-; BTVER2-SSE-NEXT:    pmulld %xmm1, %xmm0 # sched: [2:1.00]
-; BTVER2-SSE-NEXT:    pmulld (%rdi), %xmm0 # sched: [7:1.00]
+; BTVER2-SSE-NEXT:    pmulld %xmm1, %xmm0 # sched: [4:2.00]
+; BTVER2-SSE-NEXT:    pmulld (%rdi), %xmm0 # sched: [9:2.00]
 ; BTVER2-SSE-NEXT:    retq # sched: [4:1.00]
 ;
 ; BTVER2-LABEL: test_pmulld:
 ; BTVER2:       # %bb.0:
-; BTVER2-NEXT:    vpmulld %xmm1, %xmm0, %xmm0 # sched: [2:1.00]
-; BTVER2-NEXT:    vpmulld (%rdi), %xmm0, %xmm0 # sched: [7:1.00]
+; BTVER2-NEXT:    vpmulld %xmm1, %xmm0, %xmm0 # sched: [4:2.00]
+; BTVER2-NEXT:    vpmulld (%rdi), %xmm0, %xmm0 # sched: [9:2.00]
 ; BTVER2-NEXT:    retq # sched: [4:1.00]
 ;
 ; ZNVER1-SSE-LABEL: test_pmulld:
 ; ZNVER1-SSE:       # %bb.0:
 ; ZNVER1-SSE-NEXT:    pmulld %xmm1, %xmm0 # sched: [4:1.00]
 ; ZNVER1-SSE-NEXT:    pmulld (%rdi), %xmm0 # sched: [11:1.00]
 ; ZNVER1-SSE-NEXT:    retq # sched: [1:0.50]
 ;

test/tools/llvm-mca/X86/BtVer2/pipes-fpu.s

 
 # FPA/FPM YMM
 vaddps      %ymm0, %ymm1, %ymm2
 vsqrtps     %ymm0, %ymm2
 
 
 # CHECK:      Iterations:     70
 # CHECK-NEXT: Instructions:   560
-# CHECK-NEXT: Total Cycles:   4415
+# CHECK-NEXT: Total Cycles:   4416
 # CHECK-NEXT: Dispatch Width: 2
 # CHECK-NEXT: IPC:            0.13
 
 
 # CHECK:      Instruction Info:
 # CHECK-NEXT: [1]: #uOps
 # CHECK-NEXT: [2]: Latency
 # CHECK-NEXT: [3]: RThroughput
 # CHECK-NEXT: [4]: MayLoad
 # CHECK-NEXT: [5]: MayStore
 # CHECK-NEXT: [6]: HasSideEffects
 
 # CHECK:      [1]    [2]    [3]    [4]    [5]    [6]  Instructions:
-# CHECK-NEXT:  1      2     1.00                      vpmulld %xmm0, %xmm1, %xmm2
+# CHECK-NEXT:  3      4     2.00                      vpmulld %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  1      1     0.50                      vpand %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  1      3     1.00                      vcvttps2dq  %xmm0, %xmm2
 # CHECK-NEXT:  1      2     1.00                      vpclmulqdq  $0, %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  1      3     1.00                      vaddps  %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  1      21    21.00                     vsqrtps %xmm0, %xmm2
 # CHECK-NEXT:  2      3     2.00                      vaddps  %ymm0, %ymm1, %ymm2
 # CHECK-NEXT:  2      42    42.00                     vsqrtps %ymm0, %ymm2
 

 # CHECK-NEXT: [10] - JSTC
 # CHECK-NEXT: [11] - JVALU0
 # CHECK-NEXT: [12] - JVALU1
 # CHECK-NEXT: [13] - JVIMUL
 
 
 # CHECK:      Resource pressure per iteration:
 # CHECK-NEXT: [0]    [1]    [2]    [3]    [4]    [5]    [6]    [7]    [8]    [9]    [10]   [11]   [12]   [13]   
-# CHECK-NEXT:  -      -      -     3.00   63.00  5.00   5.00    -      -      -     1.00   0.50   0.50   2.00   
-
+# CHECK-NEXT:  -      -      -     3.00   63.00  6.01   5.99    -      -      -     1.00   1.00   1.00   3.00   
 # CHECK:      Resource pressure by instruction:
 # CHECK-NEXT: [0]    [1]    [2]    [3]    [4]    [5]    [6]    [7]    [8]    [9]    [10]   [11]   [12]   [13]     Instructions:
-# CHECK-NEXT:  -      -      -      -      -     1.00    -      -      -      -      -      -      -     1.00     vpmulld %xmm0, %xmm1, %xmm2
-# CHECK-NEXT:  -      -      -      -      -      -     1.00    -      -      -      -     0.50   0.50    -       vpand %xmm0, %xmm1, %xmm2
+# CHECK-NEXT:  -      -      -      -      -     2.00   1.00    -      -      -      -     0.03   0.97   2.00     vpmulld %xmm0, %xmm1, %xmm2
+# CHECK-NEXT:  -      -      -      -      -     0.01   0.99    -      -      -      -     0.97   0.03    -       vpand %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  -      -      -      -      -      -     1.00    -      -      -     1.00    -      -      -       vcvttps2dq  %xmm0, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     1.00    -      -      -      -      -      -      -     1.00     vpclmulqdq  $0, %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  -      -      -     1.00    -     1.00    -      -      -      -      -      -      -      -       vaddps  %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  -      -      -      -     21.00   -     1.00    -      -      -      -      -      -      -       vsqrtps %xmm0, %xmm2
 # CHECK-NEXT:  -      -      -     2.00    -     2.00    -      -      -      -      -      -      -      -       vaddps  %ymm0, %ymm1, %ymm2
 # CHECK-NEXT:  -      -      -      -     42.00   -     2.00    -      -      -      -      -      -      -       vsqrtps %ymm0, %ymm2
 
 
 # CHECK:      Timeline view:
 # CHECK-NEXT:                 0123456789          0123456789          0123456789        
 # CHECK-NEXT: Index 0123456789          0123456789          0123456789          01234567
 
-# CHECK:      [0,0] DeeER.    .    .    .    .    .    .    .    .    .    .    .    . .  vpmulld %xmm0, %xmm1, %xmm2
-# CHECK-NEXT: [0,1] DeE-R.    .    .    .    .    .    .    .    .    .    .    .    . .  vpand %xmm0, %xmm1, %xmm2
-# CHECK-NEXT: [0,2] .DeeeER   .    .    .    .    .    .    .    .    .    .    .    . .  vcvttps2dq  %xmm0, %xmm2
-# CHECK-NEXT: [0,3] .DeeE-R   .    .    .    .    .    .    .    .    .    .    .    . .  vpclmulqdq  $0, %xmm0, %xmm1, %xmm2
-# CHECK-NEXT: [0,4] . DeeeER  .    .    .    .    .    .    .    .    .    .    .    . .  vaddps  %xmm0, %xmm1, %xmm2
-# CHECK-NEXT: [0,5] . DeeeeeeeeeeeeeeeeeeeeeER    .    .    .    .    .    .    .    . .  vsqrtps %xmm0, %xmm2
-# CHECK-NEXT: [0,6] .  DeeeE-----------------R    .    .    .    .    .    .    .    . .  vaddps  %ymm0, %ymm1, %ymm2
-# CHECK-NEXT: [0,7] .   D===================eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeER  vsqrtps %ymm0, %ymm2
+# CHECK:      [0,0] DeeeeER   .    .    .    .    .    .    .    .    .    .    .    . .  vpmulld %xmm0, %xmm1, %xmm2
+# CHECK-NEXT: [0,1] .DeE--R   .    .    .    .    .    .    .    .    .    .    .    . .  vpand %xmm0, %xmm1, %xmm2
+# CHECK-NEXT: [0,2] . DeeeER  .    .    .    .    .    .    .    .    .    .    .    . .  vcvttps2dq  %xmm0, %xmm2
+# CHECK-NEXT: [0,3] .  DeeE-R .    .    .    .    .    .    .    .    .    .    .    . .  vpclmulqdq  $0, %xmm0, %xmm1, %xmm2
+# CHECK-NEXT: [0,4] .  DeeeER .    .    .    .    .    .    .    .    .    .    .    . .  vaddps  %xmm0, %xmm1, %xmm2
+# CHECK-NEXT: [0,5] .   DeeeeeeeeeeeeeeeeeeeeeER  .    .    .    .    .    .    .    . .  vsqrtps %xmm0, %xmm2
+# CHECK-NEXT: [0,6] .    DeeeE-----------------R  .    .    .    .    .    .    .    . .  vaddps  %ymm0, %ymm1, %ymm2
+# CHECK-NEXT: [0,7] .     D===================eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeER  vsqrtps %ymm0, %ymm2
 
-# CHECK:      [1,0] .    DeeE----------------------------------------------------------R  vpmulld %xmm0, %xmm1, %xmm2
-# CHECK-NEXT: [1,1] .    DeE-----------------------------------------------------------R  vpand %xmm0, %xmm1, %xmm2
-# CHECK-NEXT: [1,2] .    .DeeeE--------------------------------------------------------R  vcvttps2dq  %xmm0, %xmm2
-# CHECK-NEXT: [1,3] .    .DeeE---------------------------------------------------------R  vpclmulqdq  $0, %xmm0, %xmm1, %xmm2
-# CHECK-NEXT: [1,4] .    . DeeeE-------------------------------------------------------R  vaddps  %xmm0, %xmm1, %xmm2
+# CHECK:      [1,0] .    .DeeeeE--------------------------------------------------------R vpmulld %xmm0, %xmm1, %xmm2
+# CHECK-NEXT: [1,1] .    . DeE----------------------------------------------------------R vpand %xmm0, %xmm1, %xmm2
+# CHECK-NEXT: [1,2] .    .  DeeeE-------------------------------------------------------R vcvttps2dq  %xmm0, %xmm2
+# CHECK-NEXT: [1,3] .    .  DeeE--------------------------------------------------------R vpclmulqdq  $0, %xmm0, %xmm1, %xmm2
+# CHECK-NEXT: [1,4] .    .   DeeeE------------------------------------------------------R vaddps  %xmm0, %xmm1, %xmm2
 
 
 # CHECK:      Average Wait times (based on the timeline view):
 # CHECK-NEXT: [0]: Executions
 # CHECK-NEXT: [1]: Average time spent waiting in a scheduler's queue
 # CHECK-NEXT: [2]: Average time spent waiting in a scheduler's queue while ready
 # CHECK-NEXT: [3]: Average time elapsed from WB until retire stage
 
 # CHECK:            [0]    [1]    [2]    [3]
-# CHECK-NEXT: 0.     2     1.0    1.0    29.0   vpmulld %xmm0, %xmm1, %xmm2
+# CHECK-NEXT: 0.     2     1.0    1.0    28.0   vpmulld %xmm0, %xmm1, %xmm2
 # CHECK-NEXT: 1.     2     1.0    1.0    30.0   vpand %xmm0, %xmm1, %xmm2
-# CHECK-NEXT: 2.     2     1.0    1.0    28.0   vcvttps2dq  %xmm0, %xmm2
-# CHECK-NEXT: 3.     2     1.0    1.0    29.0   vpclmulqdq  $0, %xmm0, %xmm1, %xmm2
-# CHECK-NEXT: 4.     2     1.0    1.0    27.5   vaddps  %xmm0, %xmm1, %xmm2
+# CHECK-NEXT: 2.     2     1.0    1.0    27.5   vcvttps2dq  %xmm0, %xmm2
+# CHECK-NEXT: 3.     2     1.0    1.0    28.5   vpclmulqdq  $0, %xmm0, %xmm1, %xmm2
+# CHECK-NEXT: 4.     2     1.0    1.0    27.0   vaddps  %xmm0, %xmm1, %xmm2
 # CHECK-NEXT: 5.     1     1.0    1.0    0.0    vsqrtps %xmm0, %xmm2
 # CHECK-NEXT: 6.     1     1.0    1.0    17.0   vaddps  %ymm0, %ymm1, %ymm2
 # CHECK-NEXT: 7.     1     20.0   20.0   0.0    vsqrtps %ymm0, %ymm2

test/tools/llvm-mca/X86/BtVer2/resources-avx1.s

 # CHECK-NEXT:  1      2     1.00                      vpmuldq %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  1      7     1.00    *                 vpmuldq (%rax), %xmm1, %xmm2
 # CHECK-NEXT:  1      2     1.00                      vpmulhrsw %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  1      7     1.00    *                 vpmulhrsw (%rax), %xmm1, %xmm2
 # CHECK-NEXT:  1      2     1.00                      vpmulhuw  %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  1      7     1.00    *                 vpmulhuw  (%rax), %xmm1, %xmm2
 # CHECK-NEXT:  1      2     1.00                      vpmulhw %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  1      7     1.00    *                 vpmulhw (%rax), %xmm1, %xmm2
-# CHECK-NEXT:  1      2     1.00                      vpmulld %xmm0, %xmm1, %xmm2
-# CHECK-NEXT:  1      7     1.00    *                 vpmulld (%rax), %xmm1, %xmm2
+# CHECK-NEXT:  3      4     2.00                      vpmulld %xmm0, %xmm1, %xmm2
+# CHECK-NEXT:  3      9     2.00    *                 vpmulld (%rax), %xmm1, %xmm2
 # CHECK-NEXT:  1      2     1.00                      vpmullw %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  1      7     1.00    *                 vpmullw (%rax), %xmm1, %xmm2
 # CHECK-NEXT:  1      2     1.00                      vpmuludq  %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  1      7     1.00    *                 vpmuludq  (%rax), %xmm1, %xmm2
 # CHECK-NEXT:  1      1     0.50                      vpor  %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  1      6     1.00    *                 vpor  (%rax), %xmm1, %xmm2
 # CHECK-NEXT:  1      1     0.50                      vpsadbw %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  1      6     1.00    *                 vpsadbw (%rax), %xmm1, %xmm2

 # CHECK-NEXT:  -      -      -      -      -     1.00    -      -      -      -      -      -      -     1.00     vpmuldq %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     1.00    -     1.00    -      -      -      -      -     1.00     vpmuldq (%rax), %xmm1, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     1.00    -      -      -      -      -      -      -     1.00     vpmulhrsw %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     1.00    -     1.00    -      -      -      -      -     1.00     vpmulhrsw (%rax), %xmm1, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     1.00    -      -      -      -      -      -      -     1.00     vpmulhuw  %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     1.00    -     1.00    -      -      -      -      -     1.00     vpmulhuw  (%rax), %xmm1, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     1.00    -      -      -      -      -      -      -     1.00     vpmulhw %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     1.00    -     1.00    -      -      -      -      -     1.00     vpmulhw (%rax), %xmm1, %xmm2
-# CHECK-NEXT:  -      -      -      -      -     1.00    -      -      -      -      -      -      -     1.00     vpmulld %xmm0, %xmm1, %xmm2
-# CHECK-NEXT:  -      -      -      -      -     1.00    -     1.00    -      -      -      -      -     1.00     vpmulld (%rax), %xmm1, %xmm2
+# CHECK-NEXT:  -      -      -      -      -     2.50   0.50    -      -      -      -     0.50   0.50   2.00     vpmulld %xmm0, %xmm1, %xmm2
+# CHECK-NEXT:  -      -      -      -      -     2.50   0.50   1.00    -      -      -     0.50   0.50   2.00     vpmulld (%rax), %xmm1, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     1.00    -      -      -      -      -      -      -     1.00     vpmullw %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     1.00    -     1.00    -      -      -      -      -     1.00     vpmullw (%rax), %xmm1, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     1.00    -      -      -      -      -      -      -     1.00     vpmuludq  %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     1.00    -     1.00    -      -      -      -      -     1.00     vpmuludq  (%rax), %xmm1, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     0.50   0.50    -      -      -      -     0.50   0.50    -       vpor  %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     0.50   0.50   1.00    -      -      -     0.50   0.50    -       vpor  (%rax), %xmm1, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     0.50   0.50    -      -      -      -     0.50   0.50    -       vpsadbw %xmm0, %xmm1, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     0.50   0.50   1.00    -      -      -     0.50   0.50    -       vpsadbw (%rax), %xmm1, %xmm2

test/tools/llvm-mca/X86/BtVer2/resources-sse41.s

 # CHECK-NEXT:  -      -      -      -      -     0.50   0.50    -      -      -      -     0.50   0.50    -       pmovzxdq  %xmm0, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     0.50   0.50   1.00    -      -      -     0.50   0.50    -       pmovzxdq  (%rax), %xmm2
 # CHECK-NEXT:  -      -      -      -      -     0.50   0.50    -      -      -      -     0.50   0.50    -       pmovzxwd  %xmm0, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     0.50   0.50   1.00    -      -      -     0.50   0.50    -       pmovzxwd  (%rax), %xmm2
 # CHECK-NEXT:  -      -      -      -      -     0.50   0.50    -      -      -      -     0.50   0.50    -       pmovzxwq  %xmm0, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     0.50   0.50   1.00    -      -      -     0.50   0.50    -       pmovzxwq  (%rax), %xmm2
 # CHECK-NEXT:  -      -      -      -      -     1.00    -      -      -      -      -      -      -     1.00     pmuldq  %xmm0, %xmm2
 # CHECK-NEXT:  -      -      -      -      -     1.00    -     1.00    -      -      -      -      -     1.00     pmuldq  (%rax), %xmm2
-# CHECK-NEXT:  -      -      -      -      -     1.00    -      -      -      -      -      -      -     1.00     pmulld  %xmm0, %xmm2
-# CHECK-NEXT:  -      -      -      -      -     1.00    -     1.00    -      -      -      -      -     1.00     pmulld  (%rax), %xmm2
+# CHECK-NEXT:  -      -      -      -      -     2.50   0.50    -      -      -      -     0.50   0.50   2.00     pmulld  %xmm0, %xmm2
+# CHECK-NEXT:  -      -      -      -      -     2.50   0.50   1.00    -      -      -     0.50   0.50   2.00     pmulld  (%rax), %xmm2
 # CHECK-NEXT: 1.00    -      -     1.00    -     1.00    -      -      -      -      -      -      -      -       ptest %xmm0, %xmm1
 # CHECK-NEXT: 1.00    -      -     1.00    -     1.00    -     1.00    -      -      -      -      -      -       ptest (%rax), %xmm1
 # CHECK-NEXT:  -      -      -     1.00    -     1.00    -      -      -      -      -      -      -      -       roundpd $1, %xmm0, %xmm2
 # CHECK-NEXT:  -      -      -     1.00    -     1.00    -     1.00    -      -      -      -      -      -       roundpd $1, (%rax), %xmm2
 # CHECK-NEXT:  -      -      -     1.00    -     1.00    -      -      -      -      -      -      -      -       roundps $1, %xmm0, %xmm2
 # CHECK-NEXT:  -      -      -     1.00    -     1.00    -     1.00    -      -      -      -      -      -       roundps $1, (%rax), %xmm2
 # CHECK-NEXT:  -      -      -     1.00    -     1.00    -      -      -      -      -      -      -      -       roundsd $1, %xmm0, %xmm2
 # CHECK-NEXT:  -      -      -     1.00    -     1.00    -     1.00    -      -      -      -      -      -       roundsd $1, (%rax), %xmm2
 # CHECK-NEXT:  -      -      -     1.00    -     1.00    -      -      -      -      -      -      -      -       roundss $1, %xmm0, %xmm2
 # CHECK-NEXT:  -      -      -     1.00    -     1.00    -     1.00    -      -      -      -      -      -       roundss $1, (%rax), %xmm2