Better testing of schedule model instruction latencies/throughputs

[avt77]

Summary

This patch should close the issue raised in PR32216. The patch introduce a new llc switch "-print-latency" which allows to add comments with latencies in output .s files. The patch opens several questions:

• there is real mess in terminology: the sources mix throughput and latency completely - we should fix it
• sometimes the selection of our sched model is unclear because the selected alternative code sequence (sometimes) looks worse than the initial code
• there are some issues with FMA support inside the model (in fact it does not work at the moment)
• some values of latency (reported by model) look rather strange

This is the initial version of the patch and I'm sure all the above issues will be resolved soon.

[RKSimon]

lib/Target/X86/InstPrinter/X86InstComments.h

22

Style guide - AC_PRINT_LATENCY

test/CodeGen/X86/recip-fastmath2.ll

20

Where have the retq instructions gone?

[hfinkel]

lib/Target/X86/X86MCInstLower.cpp

1278

Can you call TII->getInstrLatency here instead of computing it in this loop?

(if you just call SCModel->computeInstrLatency, as suggested below, this will take care of itself).

1291

Can't you just call SCModel->computeInstrLatency return the result? The logic there seems like exactly what you want:

unsigned
TargetSchedModel::computeInstrLatency(const MachineInstr *MI,
                                      bool UseDefaultDefLatency) const {
  // For the itinerary model, fall back to the old subtarget hook.
  // Allow subtargets to compute Bundle latencies outside the machine model.
  if (hasInstrItineraries() || MI->isBundle() ||
      (!hasInstrSchedModel() && !UseDefaultDefLatency))
    return TII->getInstrLatency(&InstrItins, *MI);
 
  if (hasInstrSchedModel()) {
    const MCSchedClassDesc *SCDesc = resolveSchedClass(MI);
    if (SCDesc->isValid())
      return computeInstrLatency(*SCDesc);
  }
  return TII->defaultDefLatency(SchedModel, *MI);
}

1333

This seems really useful, but is not target dependent. Can you please move this hook into the target-independent code? Maybe in void AsmPrinter::EmitFunctionBody(), around here:

default:
  EmitInstruction(&MI);
  break;

(right before the call to EmitInstruction).

[RKSimon]

How feasible is it to include the instruction's throughput in the comment as well?

[avt77]

hfinkel, could you help me? First of all could you give me a link(s) to any doc(s) related to our MCSchedModel except sources?

Next, I was told that ResourceCycles here:

class ProcWriteResources<list<ProcResourceKind> resources> {

list<ProcResourceKind> ProcResources = resources;
list<int> ResourceCycles = [];
int Latency = 1;
int NumMicroOps = 1;

could be used as Throughput of the given instruction. Is it right? Does it mean I could include it in generated comment as well? If YES I suppose it should be the max of the Cycles, right?

[spatel]

In D30941#701498, @avt77 wrote:

hfinkel, could you help me? First of all could you give me a link(s) to any doc(s) related to our MCSchedModel except sources?

Next, I was told that ResourceCycles here:

class ProcWriteResources<list<ProcResourceKind> resources> {

list<ProcResourceKind> ProcResources = resources;
list<int> ResourceCycles = [];
int Latency = 1;
int NumMicroOps = 1;

could be used as Throughput of the given instruction. Is it right? Does it mean I could include it in generated comment as well? If YES I suppose it should be the max of the Cycles, right?

I don't know if there are any docs besides the code and the code comments, but I think you are correct - the max of ResourceCycles is the inverse throughput for the instruction:

This is from include/llvm/Target/TargetSchedule.td :

// Optionally, ResourceCycles indicates the number of cycles the
// resource is consumed. Each ResourceCycles item is paired with the
// ProcResource item at the same position in its list. Since
// ResourceCycles are rarely specialized, the list may be
// incomplete. By default, resources are consumed for a single cycle,
// regardless of latency, which models a fully pipelined processing
// unit. A value of 0 for ResourceCycles means that the resource must
// be available but is not consumed, which is only relevant for
// unbuffered resources.

And this is in MachineScheduler.cpp:

// For reserved resources, record the highest cycle using the resource.
// For top-down scheduling, this is the cycle in which we schedule this
// instruction plus the number of cycles the operations reserves the
// resource.

We could abbreviate the comment string that you are adding like: [7:2].
I'm biased because that's the way I've always formatted [ latency : inverse throughput ], but I think that people that care about CPU timing will recognize that format, so you don't have to print out the words "latency" or "throughput".

[hfinkel]

In D30941#701621, @spatel wrote:

In D30941#701498, @avt77 wrote:

hfinkel, could you help me? First of all could you give me a link(s) to any doc(s) related to our MCSchedModel except sources?

Next, I was told that ResourceCycles here:

class ProcWriteResources<list<ProcResourceKind> resources> {

list<ProcResourceKind> ProcResources = resources;
list<int> ResourceCycles = [];
int Latency = 1;
int NumMicroOps = 1;

could be used as Throughput of the given instruction. Is it right? Does it mean I could include it in generated comment as well? If YES I suppose it should be the max of the Cycles, right?

I don't know if there are any docs besides the code and the code comments,

The documentation is primarily in the header and TableGen files (for better or worse).

but I think you are correct - the max of ResourceCycles is the inverse throughput for the instruction:

That's correct if the instruction can only dispatch through that one resource.

First, for itineraries, I think you can do something like this:

double Unknown = std::numeric_limits<double>::infinity()
double Throughput = Unknown;
if (IID.isEmpty())
  return Throughput;

for (const InstrStage *IS = IID.beginStage(ItinClassIndx),
           *E = IID.endStage(ItinClassIndx); IS != E; ++IS) {
  unsigned Cycles = IS->getCycles();
  if (!Cycles)
    continue;

  Throughput = std::min(Throughput, popcnt(IS->getUnits()) * 1.0/Cycles);
}

return Throughput;

For resource descriptions, I think that you want the inverse of ResourceCycles multiplied by the number of applicable resources. Something like this:

for (MCWriteProcResEntry *WPR = STI.getWriteProcResBegin(SCClass),
                                                     *WEnd = STI.getWriteProcResEnd(SCClass); WPR != WEnd; ++WPR) {
  unsigned Cycles = WPR->Cycles;
  if (!Cycles)
    return Unknown;

  unsigned NumUnits = SCModel->getProcResource(WPR->ProcResourceIdx)->NumUnits;
  Throughput = std::min(Throughput, NumUnits * 1.0/Cycles);
}

This is from include/llvm/Target/TargetSchedule.td :

// Optionally, ResourceCycles indicates the number of cycles the
// resource is consumed. Each ResourceCycles item is paired with the
// ProcResource item at the same position in its list. Since
// ResourceCycles are rarely specialized, the list may be
// incomplete. By default, resources are consumed for a single cycle,
// regardless of latency, which models a fully pipelined processing
// unit. A value of 0 for ResourceCycles means that the resource must
// be available but is not consumed, which is only relevant for
// unbuffered resources.

And this is in MachineScheduler.cpp:

// For reserved resources, record the highest cycle using the resource.
// For top-down scheduling, this is the cycle in which we schedule this
// instruction plus the number of cycles the operations reserves the
// resource.

We could abbreviate the comment string that you are adding like: [7:2].
I'm biased because that's the way I've always formatted [ latency : inverse throughput ], but I think that people that care about CPU timing will recognize that format, so you don't have to print out the words "latency" or "throughput".

[avt77]

The implementation was moved to target independent area and all Hal's comments were applied. I did not do anything with Throughput: it will be done in the patch.

[hfinkel]

In D30941#702652, @avt77 wrote:

The implementation was moved to target independent area and all Hal's comments were applied. I did not do anything with Throughput: it will be done in the patch.

Do you mean that the throughput will be done in a follow-up patch (or a later revision of this one)? Either is fine with me, although if we're going to add more than latencies we might pick a different name. More later...

[RKSimon]

In D30941#702657, @hfinkel wrote:

In D30941#702652, @avt77 wrote:

The implementation was moved to target independent area and all Hal's comments were applied. I did not do anything with Throughput: it will be done in the patch.

Do you mean that the throughput will be done in a follow-up patch (or a later revision of this one)? Either is fine with me, although if we're going to add more than latencies we might pick a different name. More later...

If possible please can we use the term 'print schedule' now so that future improvements (throughput, etc.) make sense

[avt77]

hfinkel, yes I mean I'll do it in the next version of this patch soon.
rksimon, do you mean we should rename the compiler option like "-print-schedule"?

[hfinkel]

In D30941#703612, @avt77 wrote:

hfinkel, yes I mean I'll do it in the next version of this patch soon.
rksimon, do you mean we should rename the compiler option like "-print-schedule"?

It is not clear to me why you won't just always do this when in verbose-asm mode. Thoughts on not having a separate option at all?

[avt77]

Throughput calculation is implemented.

[avt77]

In D30941#703613, @hfinkel wrote:

In D30941#703612, @avt77 wrote:

hfinkel, yes I mean I'll do it in the next version of this patch soon.
rksimon, do you mean we should rename the compiler option like "-print-schedule"?

It is not clear to me why you won't just always do this when in verbose-asm mode. Thoughts on not having a separate option at all?

We can do it without special option of course. Should I remove it and use "verbose-asm mode" instead?

[RKSimon]

include/llvm/CodeGen/TargetSchedule.h

184 ↗

(On Diff #92153)

Add doxygen comment.

[avt77]

Hal,
I removed the special option (-print-schedule) and tried to check-all. The result was very unpleseant but predictable:

Expected Passes    : 29409
Expected Failures  : 160
Unsupported Tests  : 442
Unexpected Failures: 643

Should I fix all these 643 failures or it's better to keep the option?

[hfinkel]

In D30941#706295, @avt77 wrote:

Hal,
I removed the special option (-print-schedule) and tried to check-all. The result was very unpleseant but predictable:

Expected Passes    : 29409
Expected Failures  : 160
Unsupported Tests  : 442
Unexpected Failures: 643

Should I fix all these 643 failures or it's better to keep the option?

No. Mostly because it is not entirely mechanical (it does not make sense to update the tests without understanding whether the results make sense; the person who updates the test should hopefully have that background). FWIW, this also does not make sense for higher-level targets (e.g. NVPTX).

I recommend the following:

1. Add a target callback to enable the printing of the scheduling information during verbose mode
2. Provide a command-line option (cl::opt) that can override that target callback (for easy testing), as in:
   
   bool Enable = OptionName.getNumOccurrences() ? OptionName : STI->shouldPrintStuff();
3. Enable this only on x86 (fix those tests, if any) (*)

Once we think this works reasonably well, we can encourage other backend maintainers to look at updating their tests and enabling it on their targets if they'd like.

(*) Given that FileCheck does prefix matching, and we're adding this information last on the line, I'm curious to understand why the tests are matching that is causing these failures.

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

766 ↗

(On Diff #92153)

I think that one ? is enough (three looks excessive to me).

768 ↗

(On Diff #92153)

One only ? here please.

[RKSimon]

This comment was removed by RKSimon.

[avt77]

I did everything accordingly to Hal's requirements except one: the default value of "print-schedule" switch is false because otherewise we have "Unexpected Failures: 530" and it's X86 tests ony. The problem is very simple: update_llc_test_checks.py generates CHECKs like here

; XOP-AVX1-NEXT: vextractf128 $1, %ymm2, %xmm5

It means FileCheck has to check the whole line but this patch adds the comment at the end of line and as result the line can't be checked properly.
Finaly what we have now:

• • we have option "-print-schedule" allowing to print [latency:throughput" in output (default is false)
  • we have enablePrintSchedInfo() - default is false
• X86 overrides enablePrintSchedInfo()

Hope that's exactly what was required.

[hfinkel]

In D30941#707464, @avt77 wrote:

I did everything accordingly to Hal's requirements except one: the default value of "print-schedule" switch is false because otherewise we have "Unexpected Failures: 530" and it's X86 tests ony. The problem is very simple: update_llc_test_checks.py generates CHECKs like here

; XOP-AVX1-NEXT: vextractf128 $1, %ymm2, %xmm5

I did not realize that CHECK-NEXT always matched the whole line. That's interesting.

In any case, if this is an update_llc_test_checks.py problem, why don't you use it to update the tests?

It means FileCheck has to check the whole line but this patch adds the comment at the end of line and as result the line can't be checked properly.
Finaly what we have now:

• • we have option "-print-schedule" allowing to print [latency:throughput" in output (default is false)
  • we have enablePrintSchedInfo() - default is false
• X86 overrides enablePrintSchedInfo()

Hope that's exactly what was required.

include/llvm/CodeGen/AsmPrinter.h

120 ↗

(On Diff #92624)

Variable names should start with a capital letter: EnablePrintSchedInfo

include/llvm/MC/MCTargetOptions.h

50

Remove this option.

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

1398 ↗

(On Diff #92624)

We don't add { } around single-statement blocks.

1400 ↗

(On Diff #92624)

When you remove the '}' here, I'd keep the blank line ;)

1401 ↗

(On Diff #92624)

The command-line option should be declared in this file. That way you can predicate using the command-line option value on getNumOccurrences() as I had indicated previously. Plus, there's no need to have the option name depend on the tool.

Full disclosure: There is another option: You can use a tristate setup, look for DefaultOnOff in lib/CodeGen/AsmPrinter/DwarfDebug.cpp.

lib/CodeGen/TargetSchedule.cpp

315 ↗

(On Diff #92624)

Please remove this comment and the UseDefaultRThroughput variable. We can always add it later if we'd like.

321 ↗

(On Diff #92624)

This Unknown value should never be returned (this is an implementation detail). Please return an Optional<double>.

[avt77]

In D30941#707504, @hfinkel wrote:

In D30941#707464, @avt77 wrote:

I did everything accordingly to Hal's requirements except one: the default value of "print-schedule" switch is false because otherewise we have "Unexpected Failures: 530" and it's X86 tests ony. The problem is very simple: update_llc_test_checks.py generates CHECKs like here

; XOP-AVX1-NEXT: vextractf128 $1, %ymm2, %xmm5

I did not realize that CHECK-NEXT always matched the whole line. That's interesting.

If you'd like to CHECK the prefix only you should use something like

CHECK: vextractf128 $1, %ymm2, %xmm5{{*}}

In any case, if this is an update_llc_test_checks.py problem, why don't you use it to update the tests?

I can use it to update the tests but it means I should update 530 tests. Is it acceptable? Should I do it? For me it is not a problem but is it OK for review?

[hfinkel]

In D30941#707537, @avt77 wrote:

In D30941#707504, @hfinkel wrote:

In D30941#707464, @avt77 wrote:

I did everything accordingly to Hal's requirements except one: the default value of "print-schedule" switch is false because otherewise we have "Unexpected Failures: 530" and it's X86 tests ony. The problem is very simple: update_llc_test_checks.py generates CHECKs like here

; XOP-AVX1-NEXT: vextractf128 $1, %ymm2, %xmm5

I did not realize that CHECK-NEXT always matched the whole line. That's interesting.

If you'd like to CHECK the prefix only you should use something like

CHECK: vextractf128 $1, %ymm2, %xmm5{{*}}

In any case, if this is an update_llc_test_checks.py problem, why don't you use it to update the tests?

I can use it to update the tests but it means I should update 530 tests. Is it acceptable? Should I do it? For me it is not a problem but is it OK for review?

I think that you should update them. I assume we'd want to do it at some point. The reason not to do it would be that the numbers aren't generally right yet. The goal is that, because of the MemoryCombiner, etc. these numbers will affect not only scheduling but also instruction selection (etc.). we want to make them very visible in the tests. So, let's make them very visible... (if others disagree, please say so).

[avt77]

The problem with failed tests raised because of new lines of comments added as result of this patch. I was wrong when I told that FileCheck does not allow adding of new comments at EOL.
I redesigned the patch to make it possible to add Latency:Throughput at the end of exisiting comment (if any). As result I was forced to change API of EmitInstruction from MCStreamer. I don't like this change because there are a lot of successors of MCStreamer but it works perfectly and maybe useful for other targets.
I regenerated (with help of update_llc_test_checks.py) 34 tests and now we have only 16 failed tests: I'm going to fix them asap.

[RKSimon]

Please can you put back the "## sched: [LAT:RCP]" sched prefix?

If at all possible I'd like this first patch to just do the minimum - provide a -print-sched option (default = false) that adds the scheduling comments. I'd probably not even include the comment 'appending' in this first patch.

After this initial patch I'm much more interested in then getting a high amount of test coverage of as many instructions as possible across each scheduler model - imo that is what we need to improve first to try and find the existing issues with the models. If we don't and we then fix them later, it will require the regeneration of a lot of tests that don't actually care about the schedule.

Only then should we begin investigating how to include this in more asm output.

[avt77]

Accordingly to requirements from Simon I inserted prefix "sched: " for scheduler comments and made "false" as default value for -print-schedule option. As result I restored original versions of all X86-tests excepting 2 ones to demonstrate the changes. Now we don't have any failed test.

[RKSimon]

A number of minor style comments.

@hfinkel does this look alright to you as a patch for initial support for scheduler comments?

include/llvm/CodeGen/AsmPrinter.h

119 ↗

(On Diff #93611)

style - newline before comment, add '.' to end of comment.

include/llvm/CodeGen/TargetSchedule.h

184 ↗

(On Diff #92153)

newline

include/llvm/Target/TargetSubtargetInfo.h

22 ↗

(On Diff #93611)

Any clang-format / include 'juggling' should be done as a NFC cleanup commit, not as part of a larger patch.

147 ↗

(On Diff #93611)

Comment says 'enable' but function says 'support' - which is it? Full stop at end of comment (style)

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

772 ↗

(On Diff #93611)

Add brief comment explaining what you're doing.

lib/CodeGen/TargetSchedule.cpp

281 ↗

(On Diff #93611)

Shouldn't this be an assert?

lib/MC/MCAsmStreamer.cpp

106 ↗

(On Diff #93611)

Explain how PrintSchedInfo will be used in comment

1584 ↗

(On Diff #93611)

Comment this

lib/MC/MCObjectStreamer.cpp

231 ↗

(On Diff #93611)

You're inconsistent leaving some unused argument without names and other with.

lib/Object/RecordStreamer.cpp

81 ↗

(On Diff #93611)

'B' is meaningless - please use a real name to explain its usage.

lib/Target/Mips/MCTargetDesc/MipsELFStreamer.h

49 ↗

(On Diff #93611)

Why have you include a default value here??

lib/Target/X86/InstPrinter/X86InstComments.h

27

Should this be here or inside the cpp file(s) ?

lib/Target/X86/X86Subtarget.cpp

378 ↗

(On Diff #93611)

Probably best to bring this inside creatSchedInfoStr?

379 ↗

(On Diff #93611)

typo: createSchedInfoStr

lib/Target/X86/X86Subtarget.h

632 ↗

(On Diff #93611)

newline

[avt77]

lib/CodeGen/TargetSchedule.cpp

281 ↗

(On Diff #93611)

No, that was the issue: we can have opcodes with invalid SCDesc

[avt77]

Hope, I fixed all comments raised by RKSimon.
hfinkel, what do you think about?

[RKSimon]

lib/MC/MCAsmStreamer.cpp

106 ↗

(On Diff #93611)

Don't use @param - you have to provide all params otherwise Wdocumentation fires.

lib/Target/X86/X86Subtarget.cpp

380 ↗

(On Diff #94538)

const char *SchedPrefix = " sched: [";

[avt77]

I fixed the latest requirements from RKSimon. Please, give me your feedback.

[hfinkel]

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

1413 ↗

(On Diff #94784)

Why is this commented out? You have a target hook and it defaults to false, so that should be fine.

lib/Target/X86/X86Subtarget.cpp

394 ↗

(On Diff #94784)

Why are these functions X86-specific?

lib/Target/X86/X86Subtarget.h

628 ↗

(On Diff #94784)

Put the false and the TODO here.

[avt77]

I implemeted all requirements from hfinkel.
Please, review again.

[hfinkel]

In D30941#724863, @avt77 wrote:

I implemeted all requirements from hfinkel.
Please, review again.

LGTM, thanks!

lib/Target/X86/X86Subtarget.h

627 ↗

(On Diff #94973)

I'd just say:

// TODO: Update the regression tests and return true.

[martell]

llvm/include/llvm/MC/MCSubtargetInfo.h:36:7: warning: 'llvm::MCSubtargetInfo' has virtual functions but non-virtual destructor [-Wnon-virtual-dtor]
Currently running into warnings relating to this

[RKSimon]

In D30941#727148, @martell wrote:

llvm/include/llvm/MC/MCSubtargetInfo.h:36:7: warning: 'llvm::MCSubtargetInfo' has virtual functions but non-virtual destructor [-Wnon-virtual-dtor]
Currently running into warnings relating to this

Should be fixed by rL300322

[martell]

Perfect, thanks for following up @RKSimon

[RKSimon]

rL300311

include/llvm/MC/MCTargetOptions.h

public:
   bool MCNoDeprecatedWarn : 1;
   bool MCSaveTempLabels : 1;
   bool MCUseDwarfDirectory : 1;
   bool MCIncrementalLinkerCompatible : 1;
   bool MCPIECopyRelocations : 1;
   bool ShowMCEncoding : 1;
   bool ShowMCInst : 1;
   bool AsmVerbose : 1;
+  bool PrintLatency : 1;
 
   /// Preserve Comments in Assembly.
   bool PreserveAsmComments : 1;
 
   int DwarfVersion = 0;
 
   std::string ABIName;
 
return (ARE_EQUAL(SanitizeAddress) &&
           ARE_EQUAL(MCNoDeprecatedWarn) &&
           ARE_EQUAL(MCSaveTempLabels) &&
           ARE_EQUAL(MCUseDwarfDirectory) &&
           ARE_EQUAL(MCIncrementalLinkerCompatible) &&
           ARE_EQUAL(MCPIECopyRelocations) &&
           ARE_EQUAL(ShowMCEncoding) &&
           ARE_EQUAL(ShowMCInst) &&
           ARE_EQUAL(AsmVerbose) &&
+          ARE_EQUAL(PrintLatency) &&
           ARE_EQUAL(DwarfVersion) &&
           ARE_EQUAL(ABIName) &&
           ARE_EQUAL(IASSearchPaths));
 #undef ARE_EQUAL
 }
 
 inline bool operator!=(const MCTargetOptions &LHS, const MCTargetOptions &RHS) {
   return !(LHS == RHS);
 }
 
 } // end namespace llvm
 
 #endif // LLVM_MC_MCTARGETOPTIONS_H

lib/MC/MCTargetOptions.cpp

 using namespace llvm;
 
 MCTargetOptions::MCTargetOptions()
     : SanitizeAddress(false), MCRelaxAll(false), MCNoExecStack(false),
       MCFatalWarnings(false), MCNoWarn(false), MCNoDeprecatedWarn(false),
       MCSaveTempLabels(false), MCUseDwarfDirectory(false),
       MCIncrementalLinkerCompatible(false), MCPIECopyRelocations(false),
       ShowMCEncoding(false), ShowMCInst(false), AsmVerbose(false),
+      PrintLatency(false),
       PreserveAsmComments(true) {}
 
 StringRef MCTargetOptions::getABIName() const {
   return ABIName;
 }

lib/Target/X86/InstPrinter/X86InstComments.h

 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_LIB_TARGET_X86_INSTPRINTER_X86INSTCOMMENTS_H
 #define LLVM_LIB_TARGET_X86_INSTPRINTER_X86INSTCOMMENTS_H
 
 namespace llvm {
 
   enum AsmComments {
-    AC_EVEX_2_VEX = 0x2 // For instr that was compressed from EVEX to VEX.
+    AC_EVEX_2_VEX = 0x2, // For instr that was compressed from EVEX to VEX.
+    PRINT_LATENCY = 0x4
   };
 
   class MCInst;
   class raw_ostream;
   bool EmitAnyX86InstComments(const MCInst *MI, raw_ostream &OS,
                               const char *(*getRegName)(unsigned));
 }
 
 #endif

lib/Target/X86/X86MCInstLower.cpp

for (int i = 0, e = ShuffleMask.size(); i != e; ++i) {
     CS << ']';
     --i; // For loop increments element #.
   }
   CS.flush();
 
   return Comment;
 }
 
+/// \brief Gets latency information for \p Inst from the itinerary
+/// scheduling model.
+/// \return The maximum expected latency over all the operands or -1
+/// if no information is available.
+static int getItineraryLatency(const MachineInstr *Inst) {
+  const int NoInformationAvailable = -1;
+
+  auto *MF = Inst->getParent()->getParent();
+  auto &STI = MF->getSubtarget();
+  // Check if we have a CPU to get the itinerary information.
+  if (STI.getCPU().empty())
+    return NoInformationAvailable;
+
+  // Get itinerary information.
+  InstrItineraryData IID = STI.getInstrItineraryForCPU(STI.getCPU());
+  // Get the scheduling class of the requested instruction.
+  const MCInstrDesc& Desc = STI.getInstrInfo()->get(Inst->getOpcode());
+  unsigned SCClass = Desc.getSchedClass();
+
+  int Latency = 0;
+  for (unsigned OpIdx = 0, OpIdxEnd = Inst->getNumOperands(); OpIdx != OpIdxEnd;
+       ++OpIdx)
+    Latency = std::max(Latency, IID.getOperandCycle(SCClass, OpIdx));
+
+  return Latency;
+}
+
+/// \brief Gets latency information for \p Inst
+/// \return The maximum expected latency over all the definitions or -1
+/// if no information is available.
+static int getLatency(const MachineInstr *Inst) {
+  // Try to compute scheduling information.
+  auto &STI = Inst->getParent()->getParent()->getSubtarget();
+  auto &SCModel = STI.getSchedModel();
+  const int NoInformationAvailable = -1;
+
+  // Check if we have a scheduling model for instructions.
+  if (!SCModel.hasInstrSchedModel())
+    // Try to fall back to the itinerary model if the scheduling model doesn't
+    // have a scheduling table.  Note the default does not have a table.
+    return getItineraryLatency(Inst);
+
+  // Get the scheduling class of the requested instruction.
+  const MCInstrDesc& Desc = STI.getInstrInfo()->get(Inst->getOpcode());
+  unsigned SCClass = Desc.getSchedClass();
+  const MCSchedClassDesc *SCDesc = SCModel.getSchedClassDesc(SCClass);
+  // Resolving the variant SchedClass requires an MI to pass to
+  // SubTargetInfo::resolveSchedClass.
+  if (!SCDesc || !SCDesc->isValid() || SCDesc->isVariant())
+    return NoInformationAvailable;
+
+  // Compute output latency.
+  int Latency = 0;
+  for (unsigned DefIdx = 0, DefEnd = SCDesc->NumWriteLatencyEntries;
+       DefIdx != DefEnd; ++DefIdx) {
+    // Lookup the definition's write latency in SubtargetInfo.
+    const MCWriteLatencyEntry *WLEntry = STI.getWriteLatencyEntry(SCDesc,
+                                                                   DefIdx);
+    Latency = std::max(Latency, WLEntry->Cycles);
+  }
+
+  return Latency;
+}
+
 void X86AsmPrinter::EmitInstruction(const MachineInstr *MI) {
   X86MCInstLower MCInstLowering(*MF, *this);
   const X86RegisterInfo *RI = MF->getSubtarget<X86Subtarget>().getRegisterInfo();
 
   // Add a comment about EVEX-2-VEX compression for AVX-512 instrs that
   // are compressed from EVEX encoding to VEX encoding.
   if (TM.Options.MCOptions.ShowMCEncoding) {
     if (MI->getAsmPrinterFlags() & AC_EVEX_2_VEX)
       OutStreamer->AddComment("EVEX TO VEX Compression ", false);
   }
 
+  if (TM.Options.MCOptions.PrintLatency ||
+      MI->getAsmPrinterFlags() & PRINT_LATENCY) {
+      if (MI->isPseudo() || MI->isTerminator())
+        return;
+      auto Latency = getLatency(MI);
+      if (Latency > 0)
+        OutStreamer->AddComment(" Latency = " + Twine(Latency));
+  }
+
   switch (MI->getOpcode()) {
   case TargetOpcode::DBG_VALUE:
     llvm_unreachable("Should be handled target independently");
 
   // Emit nothing here but a comment if we can.
   case X86::Int_MemBarrier:
     OutStreamer->emitRawComment("MEMBARRIER");
     return;

test/CodeGen/X86/recip-fastmath2.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+sse2     | FileCheck %s --check-prefix=CHECK --check-prefix=SSE --check-prefix=SSE-RECIP
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx      | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=AVX-RECIP
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=FMA-RECIP
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mcpu=btver2     | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=BTVER2
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mcpu=sandybridge| FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=SANDY
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mcpu=haswell    | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=HASWELL
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mcpu=haswell -mattr=-fma | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=HASWELL-NO-FMA
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mcpu=knl        | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=AVX512 --check-prefix=KNL
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mcpu=skx        | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=AVX512 --check-prefix=SKX
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -print-latency -mattr=+sse2     | FileCheck %s --check-prefix=CHECK --check-prefix=SSE --check-prefix=SSE-RECIP
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -print-latency -mattr=+avx      | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=AVX-RECIP
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -print-latency -mattr=+avx,+fma | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=FMA-RECIP
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -print-latency -mcpu=btver2     | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=BTVER2
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -print-latency -mcpu=sandybridge| FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=SANDY
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -print-latency -mcpu=haswell    | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=HASWELL
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -print-latency -mcpu=haswell -mattr=-fma | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=HASWELL-NO-FMA
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -print-latency -mcpu=knl        | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=AVX512 --check-prefix=KNL
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -print-latency -mcpu=skx        | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=AVX512 --check-prefix=SKX
 
 ; It's the extra tests coverage for recip as discussed on D26855.
 
+define float @f32_no_estimate_2(float %x) #0 {
+; SSE-LABEL: f32_no_estimate_2:
+; SSE:       # BB#0:
+; SSE-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
+; SSE-NEXT:    divss %xmm0, %xmm1
+; SSE-NEXT:    movaps %xmm1, %xmm0
+;
+; AVX-RECIP-LABEL: f32_no_estimate_2:
+; AVX-RECIP:       # BB#0:
+; AVX-RECIP-NEXT:    vmovss {{.*#+}} xmm1 = mem[0],zero,zero,zero
+; AVX-RECIP-NEXT:    vdivss %xmm0, %xmm1, %xmm0
+;
+; FMA-RECIP-LABEL: f32_no_estimate_2:
+; FMA-RECIP:       # BB#0:
+; FMA-RECIP-NEXT:    vmovss {{.*#+}} xmm1 = mem[0],zero,zero,zero
+; FMA-RECIP-NEXT:    vdivss %xmm0, %xmm1, %xmm0
+;
+; BTVER2-LABEL: f32_no_estimate_2:
+; BTVER2:       # BB#0:
+; BTVER2-NEXT:    vmovss {{.*}}(%rip), %xmm1 # Latency = 5
+; BTVER2-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; BTVER2-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # Latency = 19
+;
+; SANDY-LABEL: f32_no_estimate_2:
+; SANDY:       # BB#0:
+; SANDY-NEXT:    vmovss {{.*}}(%rip), %xmm1 # Latency = 4
+; SANDY-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; SANDY-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # Latency = 12
+;
+; HASWELL-LABEL: f32_no_estimate_2:
+; HASWELL:       # BB#0:
+; HASWELL-NEXT:    vmovss {{.*}}(%rip), %xmm1 # Latency = 4
+; HASWELL-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; HASWELL-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # Latency = 12
+;
+; HASWELL-NO-FMA-LABEL: f32_no_estimate_2:
+; HASWELL-NO-FMA:       # BB#0:
+; HASWELL-NO-FMA-NEXT:    vmovss {{.*}}(%rip), %xmm1 # Latency = 4
+; HASWELL-NO-FMA-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; HASWELL-NO-FMA-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # Latency = 12
+;
+; AVX512-LABEL: f32_no_estimate_2:
+; AVX512:       # BB#0:
+; AVX512-NEXT:    vmovss {{.*}}(%rip), %xmm1 # Latency = 4
+; AVX512-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; AVX512-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # Latency = 12
+  %div = fdiv fast float 1234.0, %x
+  ret float %div
+}
+
 define float @f32_no_step_2(float %x) #3 {
 ; SSE-LABEL: f32_no_step_2:
 ; SSE:       # BB#0:
 ; SSE-NEXT:    rcpss %xmm0, %xmm0
 ; SSE-NEXT:    mulss {{.*}}(%rip), %xmm0
-; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: f32_no_step_2:
 ; AVX-RECIP:       # BB#0:
 ; AVX-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm0
 ; AVX-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: f32_no_step_2:
 ; FMA-RECIP:       # BB#0:
 ; FMA-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm0
 ; FMA-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: f32_no_step_2:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vrcpss %xmm0, %xmm0, %xmm0
-; BTVER2-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; BTVER2-NEXT:    retq
+; BTVER2-NEXT:    vrcpss %xmm0, %xmm0, %xmm0 # Latency = 2
+; BTVER2-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # Latency = 7
 ;
 ; SANDY-LABEL: f32_no_step_2:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpss %xmm0, %xmm0, %xmm0
-; SANDY-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; SANDY-NEXT:    retq
+; SANDY-NEXT:    vrcpss %xmm0, %xmm0, %xmm0 # Latency = 5
+; SANDY-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
 ;
 ; HASWELL-LABEL: f32_no_step_2:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpss %xmm0, %xmm0, %xmm0
-; HASWELL-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; HASWELL-NEXT:    retq
+; HASWELL-NEXT:    vrcpss %xmm0, %xmm0, %xmm0 # Latency = 5
+; HASWELL-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
 ;
 ; HASWELL-NO-FMA-LABEL: f32_no_step_2:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpss %xmm0, %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    retq
+; HASWELL-NO-FMA-NEXT:    vrcpss %xmm0, %xmm0, %xmm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
 ;
 ; AVX512-LABEL: f32_no_step_2:
 ; AVX512:       # BB#0:
 ; AVX512-NEXT:    vrcp14ss %xmm0, %xmm0, %xmm0
-; AVX512-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; AVX512-NEXT:    retq
+; AVX512-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
   %div = fdiv fast float 1234.0, %x
   ret float %div
 }
 
 define float @f32_one_step_2(float %x) #1 {
 ; SSE-LABEL: f32_one_step_2:
 ; SSE:       # BB#0:
 ; SSE-NEXT:    rcpss %xmm0, %xmm2
 ; SSE-NEXT:    mulss %xmm2, %xmm0
 ; SSE-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
 ; SSE-NEXT:    subss %xmm0, %xmm1
 ; SSE-NEXT:    mulss %xmm2, %xmm1
 ; SSE-NEXT:    addss %xmm2, %xmm1
 ; SSE-NEXT:    mulss {{.*}}(%rip), %xmm1
 ; SSE-NEXT:    movaps %xmm1, %xmm0
-; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: f32_one_step_2:
 ; AVX-RECIP:       # BB#0:
 ; AVX-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
 ; AVX-RECIP-NEXT:    vmulss %xmm1, %xmm0, %xmm0
 ; AVX-RECIP-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
 ; AVX-RECIP-NEXT:    vsubss %xmm0, %xmm2, %xmm0
 ; AVX-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0
 ; AVX-RECIP-NEXT:    vaddss %xmm0, %xmm1, %xmm0
 ; AVX-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: f32_one_step_2:
 ; FMA-RECIP:       # BB#0:
 ; FMA-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
 ; FMA-RECIP-NEXT:    vfnmadd213ss {{.*}}(%rip), %xmm1, %xmm0
 ; FMA-RECIP-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm0
 ; FMA-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: f32_one_step_2:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; BTVER2-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; BTVER2-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; BTVER2-NEXT:    vsubss %xmm0, %xmm2, %xmm0
-; BTVER2-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; BTVER2-NEXT:    retq
+; BTVER2-NEXT:    vmovss {{.*}}(%rip), %xmm2 # Latency = 5
+; BTVER2-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; BTVER2-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # Latency = 2
+; BTVER2-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # Latency = 2
+; BTVER2-NEXT:    vsubss %xmm0, %xmm2, %xmm0 # Latency = 3
+; BTVER2-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 2
+; BTVER2-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # Latency = 3
+; BTVER2-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # Latency = 7
 ;
 ; SANDY-LABEL: f32_one_step_2:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; SANDY-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; SANDY-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; SANDY-NEXT:    vsubss %xmm0, %xmm2, %xmm0
-; SANDY-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; SANDY-NEXT:    retq
+; SANDY-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # Latency = 5
+; SANDY-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # Latency = 5
+; SANDY-NEXT:    vmovss {{.*}}(%rip), %xmm2 # Latency = 4
+; SANDY-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; SANDY-NEXT:    vsubss %xmm0, %xmm2, %xmm0 # Latency = 3
+; SANDY-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 5
+; SANDY-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # Latency = 3
+; SANDY-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
 ;
 ; HASWELL-LABEL: f32_one_step_2:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
+; HASWELL-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # Latency = 5
 ; HASWELL-NEXT:    vfnmadd213ss {{.*}}(%rip), %xmm1, %xmm0
 ; HASWELL-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm0
-; HASWELL-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; HASWELL-NEXT:    retq
+; HASWELL-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
 ;
 ; HASWELL-NO-FMA-LABEL: f32_one_step_2:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; HASWELL-NO-FMA-NEXT:    vsubss %xmm0, %xmm2, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    retq
+; HASWELL-NO-FMA-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vmovss {{.*}}(%rip), %xmm2 # Latency = 4
+; HASWELL-NO-FMA-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; HASWELL-NO-FMA-NEXT:    vsubss %xmm0, %xmm2, %xmm0 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
 ;
 ; AVX512-LABEL: f32_one_step_2:
 ; AVX512:       # BB#0:
 ; AVX512-NEXT:    vrcp14ss %xmm0, %xmm0, %xmm1
 ; AVX512-NEXT:    vfnmadd213ss {{.*}}(%rip), %xmm1, %xmm0
 ; AVX512-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm0
-; AVX512-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; AVX512-NEXT:    retq
+; AVX512-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
   %div = fdiv fast float 3456.0, %x
   ret float %div
 }
 
+define float @f32_no_estimate_2_divs(float %x) #0 {
+; SSE-LABEL: f32_no_estimate_2_divs:
+; SSE:       # BB#0:
+; SSE-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
+; SSE-NEXT:    divss %xmm0, %xmm1
+; SSE-NEXT:    movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
+; SSE-NEXT:    mulss %xmm1, %xmm0
+; SSE-NEXT:    mulss %xmm1, %xmm0
+;
+; AVX-RECIP-LABEL: f32_no_estimate_2_divs:
+; AVX-RECIP:       # BB#0:
+; AVX-RECIP-NEXT:    vmovss {{.*#+}} xmm1 = mem[0],zero,zero,zero
+; AVX-RECIP-NEXT:    vdivss %xmm0, %xmm1, %xmm0
+; AVX-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1
+; AVX-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0
+;
+; FMA-RECIP-LABEL: f32_no_estimate_2_divs:
+; FMA-RECIP:       # BB#0:
+; FMA-RECIP-NEXT:    vmovss {{.*#+}} xmm1 = mem[0],zero,zero,zero
+; FMA-RECIP-NEXT:    vdivss %xmm0, %xmm1, %xmm0
+; FMA-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1
+; FMA-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0
+;
+; BTVER2-LABEL: f32_no_estimate_2_divs:
+; BTVER2:       # BB#0:
+; BTVER2-NEXT:    vmovss {{.*}}(%rip), %xmm1 # Latency = 5
+; BTVER2-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; BTVER2-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # Latency = 19
+; BTVER2-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # Latency = 7
+; BTVER2-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 2
+;
+; SANDY-LABEL: f32_no_estimate_2_divs:
+; SANDY:       # BB#0:
+; SANDY-NEXT:    vmovss {{.*}}(%rip), %xmm1 # Latency = 4
+; SANDY-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; SANDY-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # Latency = 12
+; SANDY-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # Latency = 9
+; SANDY-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 5
+;
+; HASWELL-LABEL: f32_no_estimate_2_divs:
+; HASWELL:       # BB#0:
+; HASWELL-NEXT:    vmovss {{.*}}(%rip), %xmm1 # Latency = 4
+; HASWELL-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; HASWELL-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # Latency = 12
+; HASWELL-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # Latency = 9
+; HASWELL-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 5
+;
+; HASWELL-NO-FMA-LABEL: f32_no_estimate_2_divs:
+; HASWELL-NO-FMA:       # BB#0:
+; HASWELL-NO-FMA-NEXT:    vmovss {{.*}}(%rip), %xmm1 # Latency = 4
+; HASWELL-NO-FMA-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; HASWELL-NO-FMA-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # Latency = 12
+; HASWELL-NO-FMA-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # Latency = 9
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 5
+;
+; AVX512-LABEL: f32_no_estimate_2_divs:
+; AVX512:       # BB#0:
+; AVX512-NEXT:    vmovss {{.*}}(%rip), %xmm1 # Latency = 4
+; AVX512-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; AVX512-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # Latency = 12
+; AVX512-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # Latency = 9
+; AVX512-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 5
+  %div = fdiv fast float 3456.0, %x
+  %div2 = fdiv fast float %div, %x
+  ret float %div2
+}
+
 define float @f32_one_step_2_divs(float %x) #1 {
 ; SSE-LABEL: f32_one_step_2_divs:
 ; SSE:       # BB#0:
 ; SSE-NEXT:    rcpss %xmm0, %xmm1
 ; SSE-NEXT:    mulss %xmm1, %xmm0
 ; SSE-NEXT:    movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
 ; SSE-NEXT:    subss %xmm0, %xmm2
 ; SSE-NEXT:    mulss %xmm1, %xmm2
 ; SSE-NEXT:    addss %xmm1, %xmm2
 ; SSE-NEXT:    movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
 ; SSE-NEXT:    mulss %xmm2, %xmm0
 ; SSE-NEXT:    mulss %xmm2, %xmm0
-; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: f32_one_step_2_divs:
 ; AVX-RECIP:       # BB#0:
 ; AVX-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
 ; AVX-RECIP-NEXT:    vmulss %xmm1, %xmm0, %xmm0
 ; AVX-RECIP-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
 ; AVX-RECIP-NEXT:    vsubss %xmm0, %xmm2, %xmm0
 ; AVX-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0
 ; AVX-RECIP-NEXT:    vaddss %xmm0, %xmm1, %xmm0
 ; AVX-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1
 ; AVX-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: f32_one_step_2_divs:
 ; FMA-RECIP:       # BB#0:
 ; FMA-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
 ; FMA-RECIP-NEXT:    vfnmadd213ss {{.*}}(%rip), %xmm1, %xmm0
 ; FMA-RECIP-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm0
 ; FMA-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1
 ; FMA-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: f32_one_step_2_divs:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; BTVER2-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; BTVER2-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; BTVER2-NEXT:    vsubss %xmm0, %xmm2, %xmm0
-; BTVER2-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1
-; BTVER2-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    retq
+; BTVER2-NEXT:    vmovss {{.*}}(%rip), %xmm2 # Latency = 5
+; BTVER2-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; BTVER2-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # Latency = 2
+; BTVER2-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # Latency = 2
+; BTVER2-NEXT:    vsubss %xmm0, %xmm2, %xmm0 # Latency = 3
+; BTVER2-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 2
+; BTVER2-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # Latency = 3
+; BTVER2-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # Latency = 7
+; BTVER2-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 2
 ;
 ; SANDY-LABEL: f32_one_step_2_divs:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; SANDY-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; SANDY-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; SANDY-NEXT:    vsubss %xmm0, %xmm2, %xmm0
-; SANDY-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1
-; SANDY-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    retq
+; SANDY-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # Latency = 5
+; SANDY-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # Latency = 5
+; SANDY-NEXT:    vmovss {{.*}}(%rip), %xmm2 # Latency = 4
+; SANDY-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; SANDY-NEXT:    vsubss %xmm0, %xmm2, %xmm0 # Latency = 3
+; SANDY-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 5
+; SANDY-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # Latency = 3
+; SANDY-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # Latency = 9
+; SANDY-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 5
 ;
 ; HASWELL-LABEL: f32_one_step_2_divs:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
+; HASWELL-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # Latency = 5
 ; HASWELL-NEXT:    vfnmadd213ss {{.*}}(%rip), %xmm1, %xmm0
 ; HASWELL-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm0
-; HASWELL-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1
-; HASWELL-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; HASWELL-NEXT:    retq
+; HASWELL-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # Latency = 9
+; HASWELL-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 5
 ;
 ; HASWELL-NO-FMA-LABEL: f32_one_step_2_divs:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; HASWELL-NO-FMA-NEXT:    vsubss %xmm0, %xmm2, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    retq
+; HASWELL-NO-FMA-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vmovss {{.*}}(%rip), %xmm2 # Latency = 4
+; HASWELL-NO-FMA-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; HASWELL-NO-FMA-NEXT:    vsubss %xmm0, %xmm2, %xmm0 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # Latency = 9
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 5
 ;
 ; AVX512-LABEL: f32_one_step_2_divs:
 ; AVX512:       # BB#0:
 ; AVX512-NEXT:    vrcp14ss %xmm0, %xmm0, %xmm1
 ; AVX512-NEXT:    vfnmadd213ss {{.*}}(%rip), %xmm1, %xmm0
 ; AVX512-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm0
-; AVX512-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1
-; AVX512-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; AVX512-NEXT:    retq
+; AVX512-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # Latency = 9
+; AVX512-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 5
   %div = fdiv fast float 3456.0, %x
   %div2 = fdiv fast float %div, %x
   ret float %div2
 }
 
 define float @f32_two_step_2(float %x) #2 {
 ; SSE-LABEL: f32_two_step_2:
 ; SSE:       # BB#0:
 ; SSE-NEXT:    rcpss %xmm0, %xmm2
 ; SSE-NEXT:    movaps %xmm0, %xmm3
 ; SSE-NEXT:    mulss %xmm2, %xmm3
 ; SSE-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
 ; SSE-NEXT:    movaps %xmm1, %xmm4
 ; SSE-NEXT:    subss %xmm3, %xmm4
 ; SSE-NEXT:    mulss %xmm2, %xmm4
 ; SSE-NEXT:    addss %xmm2, %xmm4
 ; SSE-NEXT:    mulss %xmm4, %xmm0
 ; SSE-NEXT:    subss %xmm0, %xmm1
 ; SSE-NEXT:    mulss %xmm4, %xmm1
 ; SSE-NEXT:    addss %xmm4, %xmm1
 ; SSE-NEXT:    mulss {{.*}}(%rip), %xmm1
 ; SSE-NEXT:    movaps %xmm1, %xmm0
-; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: f32_two_step_2:
 ; AVX-RECIP:       # BB#0:
 ; AVX-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
 ; AVX-RECIP-NEXT:    vmulss %xmm1, %xmm0, %xmm2
 ; AVX-RECIP-NEXT:    vmovss {{.*#+}} xmm3 = mem[0],zero,zero,zero
 ; AVX-RECIP-NEXT:    vsubss %xmm2, %xmm3, %xmm2
 ; AVX-RECIP-NEXT:    vmulss %xmm2, %xmm1, %xmm2
 ; AVX-RECIP-NEXT:    vaddss %xmm2, %xmm1, %xmm1
 ; AVX-RECIP-NEXT:    vmulss %xmm1, %xmm0, %xmm0
 ; AVX-RECIP-NEXT:    vsubss %xmm0, %xmm3, %xmm0
 ; AVX-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0
 ; AVX-RECIP-NEXT:    vaddss %xmm0, %xmm1, %xmm0
 ; AVX-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: f32_two_step_2:
 ; FMA-RECIP:       # BB#0:
 ; FMA-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
 ; FMA-RECIP-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
 ; FMA-RECIP-NEXT:    vmovaps %xmm1, %xmm3
 ; FMA-RECIP-NEXT:    vfnmadd213ss %xmm2, %xmm0, %xmm3
 ; FMA-RECIP-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm3
 ; FMA-RECIP-NEXT:    vfnmadd213ss %xmm2, %xmm3, %xmm0
 ; FMA-RECIP-NEXT:    vfmadd132ss %xmm3, %xmm3, %xmm0
 ; FMA-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: f32_two_step_2:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vmovss {{.*#+}} xmm3 = mem[0],zero,zero,zero
-; BTVER2-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; BTVER2-NEXT:    vmulss %xmm1, %xmm0, %xmm2
-; BTVER2-NEXT:    vsubss %xmm2, %xmm3, %xmm2
-; BTVER2-NEXT:    vmulss %xmm2, %xmm1, %xmm2
-; BTVER2-NEXT:    vaddss %xmm2, %xmm1, %xmm1
-; BTVER2-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; BTVER2-NEXT:    vsubss %xmm0, %xmm3, %xmm0
-; BTVER2-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; BTVER2-NEXT:    retq
+; BTVER2-NEXT:    vmovss {{.*}}(%rip), %xmm3 # Latency = 5
+; BTVER2-NEXT:    # xmm3 = mem[0],zero,zero,zero
+; BTVER2-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # Latency = 2
+; BTVER2-NEXT:    vmulss %xmm1, %xmm0, %xmm2 # Latency = 2
+; BTVER2-NEXT:    vsubss %xmm2, %xmm3, %xmm2 # Latency = 3
+; BTVER2-NEXT:    vmulss %xmm2, %xmm1, %xmm2 # Latency = 2
+; BTVER2-NEXT:    vaddss %xmm2, %xmm1, %xmm1 # Latency = 3
+; BTVER2-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # Latency = 2
+; BTVER2-NEXT:    vsubss %xmm0, %xmm3, %xmm0 # Latency = 3
+; BTVER2-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 2
+; BTVER2-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # Latency = 3
+; BTVER2-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # Latency = 7
 ;
 ; SANDY-LABEL: f32_two_step_2:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; SANDY-NEXT:    vmulss %xmm1, %xmm0, %xmm2
-; SANDY-NEXT:    vmovss {{.*#+}} xmm3 = mem[0],zero,zero,zero
-; SANDY-NEXT:    vsubss %xmm2, %xmm3, %xmm2
-; SANDY-NEXT:    vmulss %xmm2, %xmm1, %xmm2
-; SANDY-NEXT:    vaddss %xmm2, %xmm1, %xmm1
-; SANDY-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; SANDY-NEXT:    vsubss %xmm0, %xmm3, %xmm0
-; SANDY-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; SANDY-NEXT:    retq
+; SANDY-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # Latency = 5
+; SANDY-NEXT:    vmulss %xmm1, %xmm0, %xmm2 # Latency = 5
+; SANDY-NEXT:    vmovss {{.*}}(%rip), %xmm3 # Latency = 4
+; SANDY-NEXT:    # xmm3 = mem[0],zero,zero,zero
+; SANDY-NEXT:    vsubss %xmm2, %xmm3, %xmm2 # Latency = 3
+; SANDY-NEXT:    vmulss %xmm2, %xmm1, %xmm2 # Latency = 5
+; SANDY-NEXT:    vaddss %xmm2, %xmm1, %xmm1 # Latency = 3
+; SANDY-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # Latency = 5
+; SANDY-NEXT:    vsubss %xmm0, %xmm3, %xmm0 # Latency = 3
+; SANDY-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 5
+; SANDY-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # Latency = 3
+; SANDY-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
 ;
 ; HASWELL-LABEL: f32_two_step_2:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; HASWELL-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; HASWELL-NEXT:    vmovaps %xmm1, %xmm3
+; HASWELL-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # Latency = 5
+; HASWELL-NEXT:    vmovss {{.*}}(%rip), %xmm2 # Latency = 4
+; HASWELL-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; HASWELL-NEXT:    vmovaps %xmm1, %xmm3 # Latency = 1
 ; HASWELL-NEXT:    vfnmadd213ss %xmm2, %xmm0, %xmm3
 ; HASWELL-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm3
 ; HASWELL-NEXT:    vfnmadd213ss %xmm2, %xmm3, %xmm0
 ; HASWELL-NEXT:    vfmadd132ss %xmm3, %xmm3, %xmm0
-; HASWELL-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; HASWELL-NEXT:    retq
+; HASWELL-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
 ;
 ; HASWELL-NO-FMA-LABEL: f32_two_step_2:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulss %xmm1, %xmm0, %xmm2
-; HASWELL-NO-FMA-NEXT:    vmovss {{.*#+}} xmm3 = mem[0],zero,zero,zero
-; HASWELL-NO-FMA-NEXT:    vsubss %xmm2, %xmm3, %xmm2
-; HASWELL-NO-FMA-NEXT:    vmulss %xmm2, %xmm1, %xmm2
-; HASWELL-NO-FMA-NEXT:    vaddss %xmm2, %xmm1, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    vsubss %xmm0, %xmm3, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    retq
+; HASWELL-NO-FMA-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm1, %xmm0, %xmm2 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vmovss {{.*}}(%rip), %xmm3 # Latency = 4
+; HASWELL-NO-FMA-NEXT:    # xmm3 = mem[0],zero,zero,zero
+; HASWELL-NO-FMA-NEXT:    vsubss %xmm2, %xmm3, %xmm2 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm2, %xmm1, %xmm2 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vaddss %xmm2, %xmm1, %xmm1 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vsubss %xmm0, %xmm3, %xmm0 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
 ;
 ; AVX512-LABEL: f32_two_step_2:
 ; AVX512:       # BB#0:
 ; AVX512-NEXT:    vrcp14ss %xmm0, %xmm0, %xmm1
-; AVX512-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; AVX512-NEXT:    vmovaps %xmm1, %xmm3
+; AVX512-NEXT:    vmovss {{.*}}(%rip), %xmm2 # Latency = 4
+; AVX512-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; AVX512-NEXT:    vmovaps %xmm1, %xmm3 # Latency = 1
 ; AVX512-NEXT:    vfnmadd213ss %xmm2, %xmm0, %xmm3
 ; AVX512-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm3
 ; AVX512-NEXT:    vfnmadd213ss %xmm2, %xmm3, %xmm0
 ; AVX512-NEXT:    vfmadd132ss %xmm3, %xmm3, %xmm0
-; AVX512-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
-; AVX512-NEXT:    retq
+; AVX512-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
   %div = fdiv fast float 6789.0, %x
   ret float %div
 }
 
+define <4 x float> @v4f32_no_estimate2(<4 x float> %x) #0 {
+; SSE-LABEL: v4f32_no_estimate2:
+; SSE:       # BB#0:
+; SSE-NEXT:    movaps {{.*#+}} xmm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; SSE-NEXT:    divps %xmm0, %xmm1
+; SSE-NEXT:    movaps %xmm1, %xmm0
+;
+; AVX-RECIP-LABEL: v4f32_no_estimate2:
+; AVX-RECIP:       # BB#0:
+; AVX-RECIP-NEXT:    vmovaps {{.*#+}} xmm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; AVX-RECIP-NEXT:    vdivps %xmm0, %xmm1, %xmm0
+;
+; FMA-RECIP-LABEL: v4f32_no_estimate2:
+; FMA-RECIP:       # BB#0:
+; FMA-RECIP-NEXT:    vmovaps {{.*#+}} xmm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; FMA-RECIP-NEXT:    vdivps %xmm0, %xmm1, %xmm0
+;
+; BTVER2-LABEL: v4f32_no_estimate2:
+; BTVER2:       # BB#0:
+; BTVER2-NEXT:    vmovaps {{.*}}(%rip), %xmm1 # Latency = 5
+; BTVER2-NEXT:    # xmm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; BTVER2-NEXT:    vdivps %xmm0, %xmm1, %xmm0 # Latency = 19
+;
+; SANDY-LABEL: v4f32_no_estimate2:
+; SANDY:       # BB#0:
+; SANDY-NEXT:    vmovaps {{.*}}(%rip), %xmm1 # Latency = 4
+; SANDY-NEXT:    # xmm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; SANDY-NEXT:    vdivps %xmm0, %xmm1, %xmm0 # Latency = 12
+;
+; HASWELL-LABEL: v4f32_no_estimate2:
+; HASWELL:       # BB#0:
+; HASWELL-NEXT:    vmovaps {{.*}}(%rip), %xmm1 # Latency = 4
+; HASWELL-NEXT:    # xmm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; HASWELL-NEXT:    vdivps %xmm0, %xmm1, %xmm0 # Latency = 12
+;
+; HASWELL-NO-FMA-LABEL: v4f32_no_estimate2:
+; HASWELL-NO-FMA:       # BB#0:
+; HASWELL-NO-FMA-NEXT:    vmovaps {{.*}}(%rip), %xmm1 # Latency = 4
+; HASWELL-NO-FMA-NEXT:    # xmm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; HASWELL-NO-FMA-NEXT:    vdivps %xmm0, %xmm1, %xmm0 # Latency = 12
+;
+; AVX512-LABEL: v4f32_no_estimate2:
+; AVX512:       # BB#0:
+; AVX512-NEXT:    vmovaps {{.*}}(%rip), %xmm1 # Latency = 4
+; AVX512-NEXT:    # xmm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; AVX512-NEXT:    vdivps %xmm0, %xmm1, %xmm0 # Latency = 12
+  %div = fdiv fast <4 x float> <float 1.0, float 2.0, float 3.0, float 4.0>, %x
+  ret <4 x float> %div
+}
+
 define <4 x float> @v4f32_one_step2(<4 x float> %x) #1 {
 ; SSE-LABEL: v4f32_one_step2:
 ; SSE:       # BB#0:
 ; SSE-NEXT:    rcpps %xmm0, %xmm2
 ; SSE-NEXT:    mulps %xmm2, %xmm0
 ; SSE-NEXT:    movaps {{.*#+}} xmm1 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
 ; SSE-NEXT:    subps %xmm0, %xmm1
 ; SSE-NEXT:    mulps %xmm2, %xmm1
 ; SSE-NEXT:    addps %xmm2, %xmm1
 ; SSE-NEXT:    mulps {{.*}}(%rip), %xmm1
 ; SSE-NEXT:    movaps %xmm1, %xmm0
-; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: v4f32_one_step2:
 ; AVX-RECIP:       # BB#0:
 ; AVX-RECIP-NEXT:    vrcpps %xmm0, %xmm1
 ; AVX-RECIP-NEXT:    vmulps %xmm1, %xmm0, %xmm0
 ; AVX-RECIP-NEXT:    vmovaps {{.*#+}} xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
 ; AVX-RECIP-NEXT:    vsubps %xmm0, %xmm2, %xmm0
 ; AVX-RECIP-NEXT:    vmulps %xmm0, %xmm1, %xmm0
 ; AVX-RECIP-NEXT:    vaddps %xmm0, %xmm1, %xmm0
 ; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
-; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: v4f32_one_step2:
 ; FMA-RECIP:       # BB#0:
 ; FMA-RECIP-NEXT:    vrcpps %xmm0, %xmm1
 ; FMA-RECIP-NEXT:    vfnmadd213ps {{.*}}(%rip), %xmm1, %xmm0
 ; FMA-RECIP-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0
 ; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
-; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: v4f32_one_step2:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vmovaps {{.*#+}} xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; BTVER2-NEXT:    vrcpps %xmm0, %xmm1
-; BTVER2-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; BTVER2-NEXT:    vsubps %xmm0, %xmm2, %xmm0
-; BTVER2-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
-; BTVER2-NEXT:    retq
+; BTVER2-NEXT:    vmovaps {{.*}}(%rip), %xmm2 # Latency = 5
+; BTVER2-NEXT:    # xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; BTVER2-NEXT:    vrcpps %xmm0, %xmm1 # Latency = 2
+; BTVER2-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # Latency = 2
+; BTVER2-NEXT:    vsubps %xmm0, %xmm2, %xmm0 # Latency = 3
+; BTVER2-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # Latency = 2
+; BTVER2-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # Latency = 3
+; BTVER2-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # Latency = 7
 ;
 ; SANDY-LABEL: v4f32_one_step2:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpps %xmm0, %xmm1
-; SANDY-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; SANDY-NEXT:    vmovaps {{.*#+}} xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; SANDY-NEXT:    vsubps %xmm0, %xmm2, %xmm0
-; SANDY-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
-; SANDY-NEXT:    retq
+; SANDY-NEXT:    vrcpps %xmm0, %xmm1 # Latency = 5
+; SANDY-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # Latency = 5
+; SANDY-NEXT:    vmovaps {{.*}}(%rip), %xmm2 # Latency = 4
+; SANDY-NEXT:    # xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; SANDY-NEXT:    vsubps %xmm0, %xmm2, %xmm0 # Latency = 3
+; SANDY-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # Latency = 5
+; SANDY-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # Latency = 3
+; SANDY-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
 ;
 ; HASWELL-LABEL: v4f32_one_step2:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpps %xmm0, %xmm1
-; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2
+; HASWELL-NEXT:    vrcpps %xmm0, %xmm1 # Latency = 5
+; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2 # Latency = 4
 ; HASWELL-NEXT:    vfnmadd213ps %xmm2, %xmm1, %xmm0
 ; HASWELL-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0
-; HASWELL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
-; HASWELL-NEXT:    retq
+; HASWELL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
 ;
 ; HASWELL-NO-FMA-LABEL: v4f32_one_step2:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpps %xmm0, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2
-; HASWELL-NO-FMA-NEXT:    vsubps %xmm0, %xmm2, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    retq
+; HASWELL-NO-FMA-NEXT:    vrcpps %xmm0, %xmm1 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2 # Latency = 4
+; HASWELL-NO-FMA-NEXT:    vsubps %xmm0, %xmm2, %xmm0 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
 ;
 ; KNL-LABEL: v4f32_one_step2:
 ; KNL:       # BB#0:
-; KNL-NEXT:    vrcpps %xmm0, %xmm1
-; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2
+; KNL-NEXT:    vrcpps %xmm0, %xmm1 # Latency = 5
+; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2 # Latency = 4
 ; KNL-NEXT:    vfnmadd213ps %xmm2, %xmm1, %xmm0
 ; KNL-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0
-; KNL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
-; KNL-NEXT:    retq
+; KNL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
 ;
 ; SKX-LABEL: v4f32_one_step2:
 ; SKX:       # BB#0:
 ; SKX-NEXT:    vrcp14ps %xmm0, %xmm1
 ; SKX-NEXT:    vfnmadd213ps {{.*}}(%rip){1to4}, %xmm1, %xmm0
 ; SKX-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0
-; SKX-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
-; SKX-NEXT:    retq
+; SKX-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
   %div = fdiv fast <4 x float> <float 1.0, float 2.0, float 3.0, float 4.0>, %x
   ret <4 x float> %div
 }
 
 define <4 x float> @v4f32_one_step_2_divs(<4 x float> %x) #1 {
 ; SSE-LABEL: v4f32_one_step_2_divs:
 ; SSE:       # BB#0:
 ; SSE-NEXT:    rcpps %xmm0, %xmm1
 ; SSE-NEXT:    mulps %xmm1, %xmm0
 ; SSE-NEXT:    movaps {{.*#+}} xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
 ; SSE-NEXT:    subps %xmm0, %xmm2
 ; SSE-NEXT:    mulps %xmm1, %xmm2
 ; SSE-NEXT:    addps %xmm1, %xmm2
 ; SSE-NEXT:    movaps {{.*#+}} xmm0 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
 ; SSE-NEXT:    mulps %xmm2, %xmm0
 ; SSE-NEXT:    mulps %xmm2, %xmm0
-; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: v4f32_one_step_2_divs:
 ; AVX-RECIP:       # BB#0:
 ; AVX-RECIP-NEXT:    vrcpps %xmm0, %xmm1
 ; AVX-RECIP-NEXT:    vmulps %xmm1, %xmm0, %xmm0
 ; AVX-RECIP-NEXT:    vmovaps {{.*#+}} xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
 ; AVX-RECIP-NEXT:    vsubps %xmm0, %xmm2, %xmm0
 ; AVX-RECIP-NEXT:    vmulps %xmm0, %xmm1, %xmm0
 ; AVX-RECIP-NEXT:    vaddps %xmm0, %xmm1, %xmm0
 ; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1
 ; AVX-RECIP-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: v4f32_one_step_2_divs:
 ; FMA-RECIP:       # BB#0:
 ; FMA-RECIP-NEXT:    vrcpps %xmm0, %xmm1
 ; FMA-RECIP-NEXT:    vfnmadd213ps {{.*}}(%rip), %xmm1, %xmm0
 ; FMA-RECIP-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0
 ; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1
 ; FMA-RECIP-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: v4f32_one_step_2_divs:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vmovaps {{.*#+}} xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; BTVER2-NEXT:    vrcpps %xmm0, %xmm1
-; BTVER2-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; BTVER2-NEXT:    vsubps %xmm0, %xmm2, %xmm0
-; BTVER2-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1
-; BTVER2-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    retq
+; BTVER2-NEXT:    vmovaps {{.*}}(%rip), %xmm2 # Latency = 5
+; BTVER2-NEXT:    # xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; BTVER2-NEXT:    vrcpps %xmm0, %xmm1 # Latency = 2
+; BTVER2-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # Latency = 2
+; BTVER2-NEXT:    vsubps %xmm0, %xmm2, %xmm0 # Latency = 3
+; BTVER2-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # Latency = 2
+; BTVER2-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # Latency = 3
+; BTVER2-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1 # Latency = 7
+; BTVER2-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # Latency = 2
 ;
 ; SANDY-LABEL: v4f32_one_step_2_divs:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpps %xmm0, %xmm1
-; SANDY-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; SANDY-NEXT:    vmovaps {{.*#+}} xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; SANDY-NEXT:    vsubps %xmm0, %xmm2, %xmm0
-; SANDY-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1
-; SANDY-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    retq
+; SANDY-NEXT:    vrcpps %xmm0, %xmm1 # Latency = 5
+; SANDY-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # Latency = 5
+; SANDY-NEXT:    vmovaps {{.*}}(%rip), %xmm2 # Latency = 4
+; SANDY-NEXT:    # xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; SANDY-NEXT:    vsubps %xmm0, %xmm2, %xmm0 # Latency = 3
+; SANDY-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # Latency = 5
+; SANDY-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # Latency = 3
+; SANDY-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1 # Latency = 9
+; SANDY-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # Latency = 5
 ;
 ; HASWELL-LABEL: v4f32_one_step_2_divs:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpps %xmm0, %xmm1
-; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2
+; HASWELL-NEXT:    vrcpps %xmm0, %xmm1 # Latency = 5
+; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2 # Latency = 4
 ; HASWELL-NEXT:    vfnmadd213ps %xmm2, %xmm1, %xmm0
 ; HASWELL-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0
-; HASWELL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1
-; HASWELL-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; HASWELL-NEXT:    retq
+; HASWELL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1 # Latency = 9
+; HASWELL-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # Latency = 5
 ;
 ; HASWELL-NO-FMA-LABEL: v4f32_one_step_2_divs:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpps %xmm0, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2
-; HASWELL-NO-FMA-NEXT:    vsubps %xmm0, %xmm2, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    retq
+; HASWELL-NO-FMA-NEXT:    vrcpps %xmm0, %xmm1 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2 # Latency = 4
+; HASWELL-NO-FMA-NEXT:    vsubps %xmm0, %xmm2, %xmm0 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1 # Latency = 9
+; HASWELL-NO-FMA-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # Latency = 5
 ;
 ; KNL-LABEL: v4f32_one_step_2_divs:
 ; KNL:       # BB#0:
-; KNL-NEXT:    vrcpps %xmm0, %xmm1
-; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2
+; KNL-NEXT:    vrcpps %xmm0, %xmm1 # Latency = 5
+; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2 # Latency = 4
 ; KNL-NEXT:    vfnmadd213ps %xmm2, %xmm1, %xmm0
 ; KNL-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0
-; KNL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1
-; KNL-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; KNL-NEXT:    retq
+; KNL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1 # Latency = 9
+; KNL-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # Latency = 5
 ;
 ; SKX-LABEL: v4f32_one_step_2_divs:
 ; SKX:       # BB#0:
 ; SKX-NEXT:    vrcp14ps %xmm0, %xmm1
 ; SKX-NEXT:    vfnmadd213ps {{.*}}(%rip){1to4}, %xmm1, %xmm0
 ; SKX-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0
-; SKX-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1
-; SKX-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; SKX-NEXT:    retq
+; SKX-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1 # Latency = 9
+; SKX-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # Latency = 5
   %div = fdiv fast <4 x float> <float 1.0, float 2.0, float 3.0, float 4.0>, %x
   %div2 = fdiv fast <4 x float> %div, %x
   ret <4 x float> %div2
 }
 
 define <4 x float> @v4f32_two_step2(<4 x float> %x) #2 {
 ; SSE-LABEL: v4f32_two_step2:
 ; SSE:       # BB#0:
 ; SSE-NEXT:    rcpps %xmm0, %xmm2
 ; SSE-NEXT:    movaps %xmm0, %xmm3
 ; SSE-NEXT:    mulps %xmm2, %xmm3
 ; SSE-NEXT:    movaps {{.*#+}} xmm1 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
 ; SSE-NEXT:    movaps %xmm1, %xmm4
 ; SSE-NEXT:    subps %xmm3, %xmm4
 ; SSE-NEXT:    mulps %xmm2, %xmm4
 ; SSE-NEXT:    addps %xmm2, %xmm4
 ; SSE-NEXT:    mulps %xmm4, %xmm0
 ; SSE-NEXT:    subps %xmm0, %xmm1
 ; SSE-NEXT:    mulps %xmm4, %xmm1
 ; SSE-NEXT:    addps %xmm4, %xmm1
 ; SSE-NEXT:    mulps {{.*}}(%rip), %xmm1
 ; SSE-NEXT:    movaps %xmm1, %xmm0
-; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: v4f32_two_step2:
 ; AVX-RECIP:       # BB#0:
 ; AVX-RECIP-NEXT:    vrcpps %xmm0, %xmm1
 ; AVX-RECIP-NEXT:    vmulps %xmm1, %xmm0, %xmm2
 ; AVX-RECIP-NEXT:    vmovaps {{.*#+}} xmm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
 ; AVX-RECIP-NEXT:    vsubps %xmm2, %xmm3, %xmm2
 ; AVX-RECIP-NEXT:    vmulps %xmm2, %xmm1, %xmm2
 ; AVX-RECIP-NEXT:    vaddps %xmm2, %xmm1, %xmm1
 ; AVX-RECIP-NEXT:    vmulps %xmm1, %xmm0, %xmm0
 ; AVX-RECIP-NEXT:    vsubps %xmm0, %xmm3, %xmm0
 ; AVX-RECIP-NEXT:    vmulps %xmm0, %xmm1, %xmm0
 ; AVX-RECIP-NEXT:    vaddps %xmm0, %xmm1, %xmm0
 ; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
-; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: v4f32_two_step2:
 ; FMA-RECIP:       # BB#0:
 ; FMA-RECIP-NEXT:    vrcpps %xmm0, %xmm1
 ; FMA-RECIP-NEXT:    vmovaps {{.*#+}} xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
 ; FMA-RECIP-NEXT:    vmovaps %xmm1, %xmm3
 ; FMA-RECIP-NEXT:    vfnmadd213ps %xmm2, %xmm0, %xmm3
 ; FMA-RECIP-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm3
 ; FMA-RECIP-NEXT:    vfnmadd213ps %xmm2, %xmm3, %xmm0
 ; FMA-RECIP-NEXT:    vfmadd132ps %xmm3, %xmm3, %xmm0
 ; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
-; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: v4f32_two_step2:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vmovaps {{.*#+}} xmm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; BTVER2-NEXT:    vrcpps %xmm0, %xmm1
-; BTVER2-NEXT:    vmulps %xmm1, %xmm0, %xmm2
-; BTVER2-NEXT:    vsubps %xmm2, %xmm3, %xmm2
-; BTVER2-NEXT:    vmulps %xmm2, %xmm1, %xmm2
-; BTVER2-NEXT:    vaddps %xmm2, %xmm1, %xmm1
-; BTVER2-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; BTVER2-NEXT:    vsubps %xmm0, %xmm3, %xmm0
-; BTVER2-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
-; BTVER2-NEXT:    retq
+; BTVER2-NEXT:    vmovaps {{.*}}(%rip), %xmm3 # Latency = 5
+; BTVER2-NEXT:    # xmm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; BTVER2-NEXT:    vrcpps %xmm0, %xmm1 # Latency = 2
+; BTVER2-NEXT:    vmulps %xmm1, %xmm0, %xmm2 # Latency = 2
+; BTVER2-NEXT:    vsubps %xmm2, %xmm3, %xmm2 # Latency = 3
+; BTVER2-NEXT:    vmulps %xmm2, %xmm1, %xmm2 # Latency = 2
+; BTVER2-NEXT:    vaddps %xmm2, %xmm1, %xmm1 # Latency = 3
+; BTVER2-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # Latency = 2
+; BTVER2-NEXT:    vsubps %xmm0, %xmm3, %xmm0 # Latency = 3
+; BTVER2-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # Latency = 2
+; BTVER2-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # Latency = 3
+; BTVER2-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # Latency = 7
 ;
 ; SANDY-LABEL: v4f32_two_step2:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpps %xmm0, %xmm1
-; SANDY-NEXT:    vmulps %xmm1, %xmm0, %xmm2
-; SANDY-NEXT:    vmovaps {{.*#+}} xmm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; SANDY-NEXT:    vsubps %xmm2, %xmm3, %xmm2
-; SANDY-NEXT:    vmulps %xmm2, %xmm1, %xmm2
-; SANDY-NEXT:    vaddps %xmm2, %xmm1, %xmm1
-; SANDY-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; SANDY-NEXT:    vsubps %xmm0, %xmm3, %xmm0
-; SANDY-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
-; SANDY-NEXT:    retq
+; SANDY-NEXT:    vrcpps %xmm0, %xmm1 # Latency = 5
+; SANDY-NEXT:    vmulps %xmm1, %xmm0, %xmm2 # Latency = 5
+; SANDY-NEXT:    vmovaps {{.*}}(%rip), %xmm3 # Latency = 4
+; SANDY-NEXT:    # xmm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; SANDY-NEXT:    vsubps %xmm2, %xmm3, %xmm2 # Latency = 3
+; SANDY-NEXT:    vmulps %xmm2, %xmm1, %xmm2 # Latency = 5
+; SANDY-NEXT:    vaddps %xmm2, %xmm1, %xmm1 # Latency = 3
+; SANDY-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # Latency = 5
+; SANDY-NEXT:    vsubps %xmm0, %xmm3, %xmm0 # Latency = 3
+; SANDY-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # Latency = 5
+; SANDY-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # Latency = 3
+; SANDY-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
 ;
 ; HASWELL-LABEL: v4f32_two_step2:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpps %xmm0, %xmm1
-; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2
-; HASWELL-NEXT:    vmovaps %xmm1, %xmm3
+; HASWELL-NEXT:    vrcpps %xmm0, %xmm1 # Latency = 5
+; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2 # Latency = 4
+; HASWELL-NEXT:    vmovaps %xmm1, %xmm3 # Latency = 1
 ; HASWELL-NEXT:    vfnmadd213ps %xmm2, %xmm0, %xmm3
 ; HASWELL-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm3
 ; HASWELL-NEXT:    vfnmadd213ps %xmm2, %xmm3, %xmm0
 ; HASWELL-NEXT:    vfmadd132ps %xmm3, %xmm3, %xmm0
-; HASWELL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
-; HASWELL-NEXT:    retq
+; HASWELL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
 ;
 ; HASWELL-NO-FMA-LABEL: v4f32_two_step2:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpps %xmm0, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulps %xmm1, %xmm0, %xmm2
-; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %xmm3
-; HASWELL-NO-FMA-NEXT:    vsubps %xmm2, %xmm3, %xmm2
-; HASWELL-NO-FMA-NEXT:    vmulps %xmm2, %xmm1, %xmm2
-; HASWELL-NO-FMA-NEXT:    vaddps %xmm2, %xmm1, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    vsubps %xmm0, %xmm3, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    retq
+; HASWELL-NO-FMA-NEXT:    vrcpps %xmm0, %xmm1 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vmulps %xmm1, %xmm0, %xmm2 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %xmm3 # Latency = 4
+; HASWELL-NO-FMA-NEXT:    vsubps %xmm2, %xmm3, %xmm2 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulps %xmm2, %xmm1, %xmm2 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vaddps %xmm2, %xmm1, %xmm1 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vsubps %xmm0, %xmm3, %xmm0 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
 ;
 ; KNL-LABEL: v4f32_two_step2:
 ; KNL:       # BB#0:
-; KNL-NEXT:    vrcpps %xmm0, %xmm1
-; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2
-; KNL-NEXT:    vmovaps %xmm1, %xmm3
+; KNL-NEXT:    vrcpps %xmm0, %xmm1 # Latency = 5
+; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2 # Latency = 4
+; KNL-NEXT:    vmovaps %xmm1, %xmm3 # Latency = 1
 ; KNL-NEXT:    vfnmadd213ps %xmm2, %xmm0, %xmm3
 ; KNL-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm3
 ; KNL-NEXT:    vfnmadd213ps %xmm2, %xmm3, %xmm0
 ; KNL-NEXT:    vfmadd132ps %xmm3, %xmm3, %xmm0
-; KNL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
-; KNL-NEXT:    retq
+; KNL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
 ;
 ; SKX-LABEL: v4f32_two_step2:
 ; SKX:       # BB#0:
 ; SKX-NEXT:    vrcp14ps %xmm0, %xmm1
-; SKX-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2
-; SKX-NEXT:    vmovaps %xmm1, %xmm3
+; SKX-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2 # Latency = 4
+; SKX-NEXT:    vmovaps %xmm1, %xmm3 # Latency = 1
 ; SKX-NEXT:    vfnmadd213ps %xmm2, %xmm0, %xmm3
 ; SKX-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm3
 ; SKX-NEXT:    vfnmadd213ps %xmm2, %xmm3, %xmm0
 ; SKX-NEXT:    vfmadd132ps %xmm3, %xmm3, %xmm0
-; SKX-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
-; SKX-NEXT:    retq
+; SKX-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # Latency = 9
   %div = fdiv fast <4 x float> <float 1.0, float 2.0, float 3.0, float 4.0>, %x
   ret <4 x float> %div
 }
 
+define <8 x float> @v8f32_no_estimate2(<8 x float> %x) #0 {
+; SSE-LABEL: v8f32_no_estimate2:
+; SSE:       # BB#0:
+; SSE-NEXT:    movaps {{.*#+}} xmm2 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; SSE-NEXT:    divps %xmm0, %xmm2
+; SSE-NEXT:    movaps {{.*#+}} xmm3 = [5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
+; SSE-NEXT:    divps %xmm1, %xmm3
+; SSE-NEXT:    movaps %xmm2, %xmm0
+; SSE-NEXT:    movaps %xmm3, %xmm1
+;
+; AVX-RECIP-LABEL: v8f32_no_estimate2:
+; AVX-RECIP:       # BB#0:
+; AVX-RECIP-NEXT:    vmovaps {{.*#+}} ymm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00,5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
+; AVX-RECIP-NEXT:    vdivps %ymm0, %ymm1, %ymm0
+;
+; FMA-RECIP-LABEL: v8f32_no_estimate2:
+; FMA-RECIP:       # BB#0:
+; FMA-RECIP-NEXT:    vmovaps {{.*#+}} ymm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00,5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
+; FMA-RECIP-NEXT:    vdivps %ymm0, %ymm1, %ymm0
+;
+; BTVER2-LABEL: v8f32_no_estimate2:
+; BTVER2:       # BB#0:
+; BTVER2-NEXT:    vmovaps {{.*}}(%rip), %ymm1 # Latency = 5
+; BTVER2-NEXT:    # ymm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00,5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
+; BTVER2-NEXT:    vdivps %ymm0, %ymm1, %ymm0 # Latency = 19
+;
+; SANDY-LABEL: v8f32_no_estimate2:
+; SANDY:       # BB#0:
+; SANDY-NEXT:    vmovaps {{.*}}(%rip), %ymm1 # Latency = 4
+; SANDY-NEXT:    # ymm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00,5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
+; SANDY-NEXT:    vdivps %ymm0, %ymm1, %ymm0 # Latency = 12
+;
+; HASWELL-LABEL: v8f32_no_estimate2:
+; HASWELL:       # BB#0:
+; HASWELL-NEXT:    vmovaps {{.*}}(%rip), %ymm1 # Latency = 4
+; HASWELL-NEXT:    # ymm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00,5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
+; HASWELL-NEXT:    vdivps %ymm0, %ymm1, %ymm0 # Latency = 19
+;
+; HASWELL-NO-FMA-LABEL: v8f32_no_estimate2:
+; HASWELL-NO-FMA:       # BB#0:
+; HASWELL-NO-FMA-NEXT:    vmovaps {{.*}}(%rip), %ymm1 # Latency = 4
+; HASWELL-NO-FMA-NEXT:    # ymm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00,5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
+; HASWELL-NO-FMA-NEXT:    vdivps %ymm0, %ymm1, %ymm0 # Latency = 19
+;
+; AVX512-LABEL: v8f32_no_estimate2:
+; AVX512:       # BB#0:
+; AVX512-NEXT:    vmovaps {{.*}}(%rip), %ymm1 # Latency = 4
+; AVX512-NEXT:    # ymm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00,5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
+; AVX512-NEXT:    vdivps %ymm0, %ymm1, %ymm0 # Latency = 19
+  %div = fdiv fast <8 x float> <float 1.0, float 2.0, float 3.0, float 4.0, float 5.0, float 6.0, float 7.0, float 8.0>, %x
+  ret <8 x float> %div
+}
+
 define <8 x float> @v8f32_one_step2(<8 x float> %x) #1 {
 ; SSE-LABEL: v8f32_one_step2:
 ; SSE:       # BB#0:
 ; SSE-NEXT:    rcpps %xmm1, %xmm4
 ; SSE-NEXT:    mulps %xmm4, %xmm1
 ; SSE-NEXT:    movaps {{.*#+}} xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
 ; SSE-NEXT:    movaps %xmm2, %xmm3
 ; SSE-NEXT:    subps %xmm1, %xmm3
 ; SSE-NEXT:    mulps %xmm4, %xmm3
 ; SSE-NEXT:    addps %xmm4, %xmm3
 ; SSE-NEXT:    rcpps %xmm0, %xmm1
 ; SSE-NEXT:    mulps %xmm1, %xmm0
 ; SSE-NEXT:    subps %xmm0, %xmm2
 ; SSE-NEXT:    mulps %xmm1, %xmm2
 ; SSE-NEXT:    addps %xmm1, %xmm2
 ; SSE-NEXT:    mulps {{.*}}(%rip), %xmm2
 ; SSE-NEXT:    mulps {{.*}}(%rip), %xmm3
 ; SSE-NEXT:    movaps %xmm2, %xmm0
 ; SSE-NEXT:    movaps %xmm3, %xmm1
-; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: v8f32_one_step2:
 ; AVX-RECIP:       # BB#0:
 ; AVX-RECIP-NEXT:    vrcpps %ymm0, %ymm1
 ; AVX-RECIP-NEXT:    vmulps %ymm1, %ymm0, %ymm0
 ; AVX-RECIP-NEXT:    vmovaps {{.*#+}} ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
 ; AVX-RECIP-NEXT:    vsubps %ymm0, %ymm2, %ymm0
 ; AVX-RECIP-NEXT:    vmulps %ymm0, %ymm1, %ymm0
 ; AVX-RECIP-NEXT:    vaddps %ymm0, %ymm1, %ymm0
 ; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: v8f32_one_step2:
 ; FMA-RECIP:       # BB#0:
 ; FMA-RECIP-NEXT:    vrcpps %ymm0, %ymm1
 ; FMA-RECIP-NEXT:    vfnmadd213ps {{.*}}(%rip), %ymm1, %ymm0
 ; FMA-RECIP-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0
 ; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: v8f32_one_step2:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vmovaps {{.*#+}} ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; BTVER2-NEXT:    vrcpps %ymm0, %ymm1
-; BTVER2-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; BTVER2-NEXT:    vsubps %ymm0, %ymm2, %ymm0
-; BTVER2-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; BTVER2-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; BTVER2-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; BTVER2-NEXT:    retq
+; BTVER2-NEXT:    vmovaps {{.*}}(%rip), %ymm2 # Latency = 5
+; BTVER2-NEXT:    # ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; BTVER2-NEXT:    vrcpps %ymm0, %ymm1 # Latency = 2
+; BTVER2-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # Latency = 2
+; BTVER2-NEXT:    vsubps %ymm0, %ymm2, %ymm0 # Latency = 3
+; BTVER2-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # Latency = 2
+; BTVER2-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # Latency = 3
+; BTVER2-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # Latency = 7
 ;
 ; SANDY-LABEL: v8f32_one_step2:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpps %ymm0, %ymm1
-; SANDY-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; SANDY-NEXT:    vmovaps {{.*#+}} ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; SANDY-NEXT:    vsubps %ymm0, %ymm2, %ymm0
-; SANDY-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; SANDY-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; SANDY-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; SANDY-NEXT:    retq
+; SANDY-NEXT:    vrcpps %ymm0, %ymm1 # Latency = 5
+; SANDY-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # Latency = 5
+; SANDY-NEXT:    vmovaps {{.*}}(%rip), %ymm2 # Latency = 4
+; SANDY-NEXT:    # ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; SANDY-NEXT:    vsubps %ymm0, %ymm2, %ymm0 # Latency = 3
+; SANDY-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # Latency = 5
+; SANDY-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # Latency = 3
+; SANDY-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # Latency = 9
 ;
 ; HASWELL-LABEL: v8f32_one_step2:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpps %ymm0, %ymm1
-; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2
+; HASWELL-NEXT:    vrcpps %ymm0, %ymm1 # Latency = 7
+; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2 # Latency = 5
 ; HASWELL-NEXT:    vfnmadd213ps %ymm2, %ymm1, %ymm0
 ; HASWELL-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0
-; HASWELL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; HASWELL-NEXT:    retq
+; HASWELL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # Latency = 9
 ;
 ; HASWELL-NO-FMA-LABEL: v8f32_one_step2:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm1
-; HASWELL-NO-FMA-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2
-; HASWELL-NO-FMA-NEXT:    vsubps %ymm0, %ymm2, %ymm0
-; HASWELL-NO-FMA-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; HASWELL-NO-FMA-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; HASWELL-NO-FMA-NEXT:    retq
+; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm1 # Latency = 7
+; HASWELL-NO-FMA-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vsubps %ymm0, %ymm2, %ymm0 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # Latency = 9
 ;
 ; KNL-LABEL: v8f32_one_step2:
 ; KNL:       # BB#0:
-; KNL-NEXT:    vrcpps %ymm0, %ymm1
-; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2
+; KNL-NEXT:    vrcpps %ymm0, %ymm1 # Latency = 7
+; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2 # Latency = 5
 ; KNL-NEXT:    vfnmadd213ps %ymm2, %ymm1, %ymm0
 ; KNL-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0
-; KNL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; KNL-NEXT:    retq
+; KNL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # Latency = 9
 ;
 ; SKX-LABEL: v8f32_one_step2:
 ; SKX:       # BB#0:
 ; SKX-NEXT:    vrcp14ps %ymm0, %ymm1
 ; SKX-NEXT:    vfnmadd213ps {{.*}}(%rip){1to8}, %ymm1, %ymm0
 ; SKX-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0
-; SKX-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; SKX-NEXT:    retq
+; SKX-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # Latency = 9
   %div = fdiv fast <8 x float> <float 1.0, float 2.0, float 3.0, float 4.0, float 5.0, float 6.0, float 7.0, float 8.0>, %x
   ret <8 x float> %div
 }
 
 define <8 x float> @v8f32_one_step_2_divs(<8 x float> %x) #1 {
 ; SSE-LABEL: v8f32_one_step_2_divs:
 ; SSE:       # BB#0:
 ; SSE-NEXT:    rcpps %xmm0, %xmm2

 ; SSE-NEXT:    mulps %xmm0, %xmm3
 ; SSE-NEXT:    addps %xmm0, %xmm3
 ; SSE-NEXT:    movaps {{.*#+}} xmm1 = [5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
 ; SSE-NEXT:    mulps %xmm3, %xmm1
 ; SSE-NEXT:    movaps {{.*#+}} xmm0 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
 ; SSE-NEXT:    mulps %xmm4, %xmm0
 ; SSE-NEXT:    mulps %xmm4, %xmm0
 ; SSE-NEXT:    mulps %xmm3, %xmm1
-; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: v8f32_one_step_2_divs:
 ; AVX-RECIP:       # BB#0:
 ; AVX-RECIP-NEXT:    vrcpps %ymm0, %ymm1
 ; AVX-RECIP-NEXT:    vmulps %ymm1, %ymm0, %ymm0
 ; AVX-RECIP-NEXT:    vmovaps {{.*#+}} ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
 ; AVX-RECIP-NEXT:    vsubps %ymm0, %ymm2, %ymm0
 ; AVX-RECIP-NEXT:    vmulps %ymm0, %ymm1, %ymm0
 ; AVX-RECIP-NEXT:    vaddps %ymm0, %ymm1, %ymm0
 ; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1
 ; AVX-RECIP-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: v8f32_one_step_2_divs:
 ; FMA-RECIP:       # BB#0:
 ; FMA-RECIP-NEXT:    vrcpps %ymm0, %ymm1
 ; FMA-RECIP-NEXT:    vfnmadd213ps {{.*}}(%rip), %ymm1, %ymm0
 ; FMA-RECIP-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0
 ; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1
 ; FMA-RECIP-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: v8f32_one_step_2_divs:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vmovaps {{.*#+}} ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; BTVER2-NEXT:    vrcpps %ymm0, %ymm1
-; BTVER2-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; BTVER2-NEXT:    vsubps %ymm0, %ymm2, %ymm0
-; BTVER2-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; BTVER2-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; BTVER2-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1
-; BTVER2-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; BTVER2-NEXT:    retq
+; BTVER2-NEXT:    vmovaps {{.*}}(%rip), %ymm2 # Latency = 5
+; BTVER2-NEXT:    # ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; BTVER2-NEXT:    vrcpps %ymm0, %ymm1 # Latency = 2
+; BTVER2-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # Latency = 2
+; BTVER2-NEXT:    vsubps %ymm0, %ymm2, %ymm0 # Latency = 3
+; BTVER2-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # Latency = 2
+; BTVER2-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # Latency = 3
+; BTVER2-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1 # Latency = 7
+; BTVER2-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # Latency = 2
 ;
 ; SANDY-LABEL: v8f32_one_step_2_divs:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpps %ymm0, %ymm1
-; SANDY-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; SANDY-NEXT:    vmovaps {{.*#+}} ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; SANDY-NEXT:    vsubps %ymm0, %ymm2, %ymm0
-; SANDY-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; SANDY-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; SANDY-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1
-; SANDY-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; SANDY-NEXT:    retq
+; SANDY-NEXT:    vrcpps %ymm0, %ymm1 # Latency = 5
+; SANDY-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # Latency = 5
+; SANDY-NEXT:    vmovaps {{.*}}(%rip), %ymm2 # Latency = 4
+; SANDY-NEXT:    # ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; SANDY-NEXT:    vsubps %ymm0, %ymm2, %ymm0 # Latency = 3
+; SANDY-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # Latency = 5
+; SANDY-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # Latency = 3
+; SANDY-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1 # Latency = 9
+; SANDY-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # Latency = 5
 ;
 ; HASWELL-LABEL: v8f32_one_step_2_divs:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpps %ymm0, %ymm1
-; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2
+; HASWELL-NEXT:    vrcpps %ymm0, %ymm1 # Latency = 7
+; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2 # Latency = 5
 ; HASWELL-NEXT:    vfnmadd213ps %ymm2, %ymm1, %ymm0
 ; HASWELL-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0
-; HASWELL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1
-; HASWELL-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; HASWELL-NEXT:    retq
+; HASWELL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1 # Latency = 9
+; HASWELL-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # Latency = 5
 ;
 ; HASWELL-NO-FMA-LABEL: v8f32_one_step_2_divs:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm1
-; HASWELL-NO-FMA-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2
-; HASWELL-NO-FMA-NEXT:    vsubps %ymm0, %ymm2, %ymm0
-; HASWELL-NO-FMA-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; HASWELL-NO-FMA-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1
-; HASWELL-NO-FMA-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; HASWELL-NO-FMA-NEXT:    retq
+; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm1 # Latency = 7
+; HASWELL-NO-FMA-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vsubps %ymm0, %ymm2, %ymm0 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1 # Latency = 9
+; HASWELL-NO-FMA-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # Latency = 5
 ;
 ; KNL-LABEL: v8f32_one_step_2_divs:
 ; KNL:       # BB#0:
-; KNL-NEXT:    vrcpps %ymm0, %ymm1
-; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2
+; KNL-NEXT:    vrcpps %ymm0, %ymm1 # Latency = 7
+; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2 # Latency = 5
 ; KNL-NEXT:    vfnmadd213ps %ymm2, %ymm1, %ymm0
 ; KNL-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0
-; KNL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1
-; KNL-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; KNL-NEXT:    retq
+; KNL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1 # Latency = 9
+; KNL-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # Latency = 5
 ;
 ; SKX-LABEL: v8f32_one_step_2_divs:
 ; SKX:       # BB#0:
 ; SKX-NEXT:    vrcp14ps %ymm0, %ymm1
 ; SKX-NEXT:    vfnmadd213ps {{.*}}(%rip){1to8}, %ymm1, %ymm0
 ; SKX-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0
-; SKX-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1
-; SKX-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; SKX-NEXT:    retq
+; SKX-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1 # Latency = 9
+; SKX-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # Latency = 5
   %div = fdiv fast <8 x float> <float 1.0, float 2.0, float 3.0, float 4.0, float 5.0, float 6.0, float 7.0, float 8.0>, %x
   %div2 = fdiv fast <8 x float> %div, %x
   ret <8 x float> %div2
 }
 
 define <8 x float> @v8f32_two_step2(<8 x float> %x) #2 {
 ; SSE-LABEL: v8f32_two_step2:
 ; SSE:       # BB#0:

 ; SSE-NEXT:    addps %xmm1, %xmm5
 ; SSE-NEXT:    mulps %xmm5, %xmm2
 ; SSE-NEXT:    subps %xmm2, %xmm0
 ; SSE-NEXT:    mulps %xmm5, %xmm0
 ; SSE-NEXT:    addps %xmm5, %xmm0
 ; SSE-NEXT:    mulps {{.*}}(%rip), %xmm0
 ; SSE-NEXT:    mulps {{.*}}(%rip), %xmm3
 ; SSE-NEXT:    movaps %xmm3, %xmm1
-; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: v8f32_two_step2:
 ; AVX-RECIP:       # BB#0:
 ; AVX-RECIP-NEXT:    vrcpps %ymm0, %ymm1
 ; AVX-RECIP-NEXT:    vmulps %ymm1, %ymm0, %ymm2
 ; AVX-RECIP-NEXT:    vmovaps {{.*#+}} ymm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
 ; AVX-RECIP-NEXT:    vsubps %ymm2, %ymm3, %ymm2
 ; AVX-RECIP-NEXT:    vmulps %ymm2, %ymm1, %ymm2
 ; AVX-RECIP-NEXT:    vaddps %ymm2, %ymm1, %ymm1
 ; AVX-RECIP-NEXT:    vmulps %ymm1, %ymm0, %ymm0
 ; AVX-RECIP-NEXT:    vsubps %ymm0, %ymm3, %ymm0
 ; AVX-RECIP-NEXT:    vmulps %ymm0, %ymm1, %ymm0
 ; AVX-RECIP-NEXT:    vaddps %ymm0, %ymm1, %ymm0
 ; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: v8f32_two_step2:
 ; FMA-RECIP:       # BB#0:
 ; FMA-RECIP-NEXT:    vrcpps %ymm0, %ymm1
 ; FMA-RECIP-NEXT:    vmovaps {{.*#+}} ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
 ; FMA-RECIP-NEXT:    vmovaps %ymm1, %ymm3
 ; FMA-RECIP-NEXT:    vfnmadd213ps %ymm2, %ymm0, %ymm3
 ; FMA-RECIP-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm3
 ; FMA-RECIP-NEXT:    vfnmadd213ps %ymm2, %ymm3, %ymm0
 ; FMA-RECIP-NEXT:    vfmadd132ps %ymm3, %ymm3, %ymm0
 ; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: v8f32_two_step2:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vmovaps {{.*#+}} ymm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; BTVER2-NEXT:    vrcpps %ymm0, %ymm1
-; BTVER2-NEXT:    vmulps %ymm1, %ymm0, %ymm2
-; BTVER2-NEXT:    vsubps %ymm2, %ymm3, %ymm2
-; BTVER2-NEXT:    vmulps %ymm2, %ymm1, %ymm2
-; BTVER2-NEXT:    vaddps %ymm2, %ymm1, %ymm1
-; BTVER2-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; BTVER2-NEXT:    vsubps %ymm0, %ymm3, %ymm0
-; BTVER2-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; BTVER2-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; BTVER2-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; BTVER2-NEXT:    retq
+; BTVER2-NEXT:    vmovaps {{.*}}(%rip), %ymm3 # Latency = 5
+; BTVER2-NEXT:    # ymm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; BTVER2-NEXT:    vrcpps %ymm0, %ymm1 # Latency = 2
+; BTVER2-NEXT:    vmulps %ymm1, %ymm0, %ymm2 # Latency = 2
+; BTVER2-NEXT:    vsubps %ymm2, %ymm3, %ymm2 # Latency = 3
+; BTVER2-NEXT:    vmulps %ymm2, %ymm1, %ymm2 # Latency = 2
+; BTVER2-NEXT:    vaddps %ymm2, %ymm1, %ymm1 # Latency = 3
+; BTVER2-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # Latency = 2
+; BTVER2-NEXT:    vsubps %ymm0, %ymm3, %ymm0 # Latency = 3
+; BTVER2-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # Latency = 2
+; BTVER2-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # Latency = 3
+; BTVER2-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # Latency = 7
 ;
 ; SANDY-LABEL: v8f32_two_step2:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpps %ymm0, %ymm1
-; SANDY-NEXT:    vmulps %ymm1, %ymm0, %ymm2
-; SANDY-NEXT:    vmovaps {{.*#+}} ymm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; SANDY-NEXT:    vsubps %ymm2, %ymm3, %ymm2
-; SANDY-NEXT:    vmulps %ymm2, %ymm1, %ymm2
-; SANDY-NEXT:    vaddps %ymm2, %ymm1, %ymm1
-; SANDY-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; SANDY-NEXT:    vsubps %ymm0, %ymm3, %ymm0
-; SANDY-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; SANDY-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; SANDY-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; SANDY-NEXT:    retq
+; SANDY-NEXT:    vrcpps %ymm0, %ymm1 # Latency = 5
+; SANDY-NEXT:    vmulps %ymm1, %ymm0, %ymm2 # Latency = 5
+; SANDY-NEXT:    vmovaps {{.*}}(%rip), %ymm3 # Latency = 4
+; SANDY-NEXT:    # ymm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; SANDY-NEXT:    vsubps %ymm2, %ymm3, %ymm2 # Latency = 3
+; SANDY-NEXT:    vmulps %ymm2, %ymm1, %ymm2 # Latency = 5
+; SANDY-NEXT:    vaddps %ymm2, %ymm1, %ymm1 # Latency = 3
+; SANDY-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # Latency = 5
+; SANDY-NEXT:    vsubps %ymm0, %ymm3, %ymm0 # Latency = 3
+; SANDY-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # Latency = 5
+; SANDY-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # Latency = 3
+; SANDY-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # Latency = 9
 ;
 ; HASWELL-LABEL: v8f32_two_step2:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpps %ymm0, %ymm1
-; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2
-; HASWELL-NEXT:    vmovaps %ymm1, %ymm3
+; HASWELL-NEXT:    vrcpps %ymm0, %ymm1 # Latency = 7
+; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2 # Latency = 5
+; HASWELL-NEXT:    vmovaps %ymm1, %ymm3 # Latency = 1
 ; HASWELL-NEXT:    vfnmadd213ps %ymm2, %ymm0, %ymm3
 ; HASWELL-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm3
 ; HASWELL-NEXT:    vfnmadd213ps %ymm2, %ymm3, %ymm0
 ; HASWELL-NEXT:    vfmadd132ps %ymm3, %ymm3, %ymm0
-; HASWELL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; HASWELL-NEXT:    retq
+; HASWELL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # Latency = 9
 ;
 ; HASWELL-NO-FMA-LABEL: v8f32_two_step2:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm1
-; HASWELL-NO-FMA-NEXT:    vmulps %ymm1, %ymm0, %ymm2
-; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %ymm3
-; HASWELL-NO-FMA-NEXT:    vsubps %ymm2, %ymm3, %ymm2
-; HASWELL-NO-FMA-NEXT:    vmulps %ymm2, %ymm1, %ymm2
-; HASWELL-NO-FMA-NEXT:    vaddps %ymm2, %ymm1, %ymm1
-; HASWELL-NO-FMA-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; HASWELL-NO-FMA-NEXT:    vsubps %ymm0, %ymm3, %ymm0
-; HASWELL-NO-FMA-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; HASWELL-NO-FMA-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; HASWELL-NO-FMA-NEXT:    retq
+; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm1 # Latency = 7
+; HASWELL-NO-FMA-NEXT:    vmulps %ymm1, %ymm0, %ymm2 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %ymm3 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vsubps %ymm2, %ymm3, %ymm2 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulps %ymm2, %ymm1, %ymm2 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vaddps %ymm2, %ymm1, %ymm1 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vsubps %ymm0, %ymm3, %ymm0 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # Latency = 5
+; HASWELL-NO-FMA-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # Latency = 3
+; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # Latency = 9
 ;
 ; KNL-LABEL: v8f32_two_step2:
 ; KNL:       # BB#0:
-; KNL-NEXT:    vrcpps %ymm0, %ymm1
-; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2
-; KNL-NEXT:    vmovaps %ymm1, %ymm3
+; KNL-NEXT:    vrcpps %ymm0, %ymm1 # Latency = 7
+; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2 # Latency = 5
+; KNL-NEXT:    vmovaps %ymm1, %ymm3 # Latency = 1
 ; KNL-NEXT:    vfnmadd213ps %ymm2, %ymm0, %ymm3
 ; KNL-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm3
 ; KNL-NEXT:    vfnmadd213ps %ymm2, %ymm3, %ymm0
 ; KNL-NEXT:    vfmadd132ps %ymm3, %ymm3, %ymm0
-; KNL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; KNL-NEXT:    retq
+; KNL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # Latency = 9
 ;
 ; SKX-LABEL: v8f32_two_step2:
 ; SKX:       # BB#0:
 ; SKX-NEXT:    vrcp14ps %ymm0, %ymm1
-; SKX-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2
-; SKX-NEXT:    vmovaps %ymm1, %ymm3
+; SKX-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2 # Latency = 5
+; SKX-NEXT:    vmovaps %ymm1, %ymm3 # Latency = 1
 ; SKX-NEXT:    vfnmadd213ps %ymm2, %ymm0, %ymm3
 ; SKX-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm3
 ; SKX-NEXT:    vfnmadd213ps %ymm2, %ymm3, %ymm0
 ; SKX-NEXT:    vfmadd132ps %ymm3, %ymm3, %ymm0
-; SKX-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; SKX-NEXT:    retq
+; SKX-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # Latency = 9
   %div = fdiv fast <8 x float> <float 1.0, float 2.0, float 3.0, float 4.0, float 5.0, float 6.0, float 7.0, float 8.0>, %x
   ret <8 x float> %div
 }
 
 define <8 x float> @v8f32_no_step(<8 x float> %x) #3 {
 ; SSE-LABEL: v8f32_no_step:
 ; SSE:       # BB#0:
 ; SSE-NEXT:    rcpps %xmm0, %xmm0
 ; SSE-NEXT:    rcpps %xmm1, %xmm1
-; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: v8f32_no_step:
 ; AVX-RECIP:       # BB#0:
 ; AVX-RECIP-NEXT:    vrcpps %ymm0, %ymm0
-; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: v8f32_no_step:
 ; FMA-RECIP:       # BB#0:
 ; FMA-RECIP-NEXT:    vrcpps %ymm0, %ymm0
-; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: v8f32_no_step:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vrcpps %ymm0, %ymm0
-; BTVER2-NEXT:    retq
+; BTVER2-NEXT:    vrcpps %ymm0, %ymm0 # Latency = 2
 ;
 ; SANDY-LABEL: v8f32_no_step:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpps %ymm0, %ymm0
-; SANDY-NEXT:    retq
+; SANDY-NEXT:    vrcpps %ymm0, %ymm0 # Latency = 5
 ;
 ; HASWELL-LABEL: v8f32_no_step:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpps %ymm0, %ymm0
-; HASWELL-NEXT:    retq
+; HASWELL-NEXT:    vrcpps %ymm0, %ymm0 # Latency = 7
 ;
 ; HASWELL-NO-FMA-LABEL: v8f32_no_step:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm0
-; HASWELL-NO-FMA-NEXT:    retq
+; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm0 # Latency = 7
 ;
 ; KNL-LABEL: v8f32_no_step:
 ; KNL:       # BB#0:
-; KNL-NEXT:    vrcpps %ymm0, %ymm0
-; KNL-NEXT:    retq
+; KNL-NEXT:    vrcpps %ymm0, %ymm0 # Latency = 7
 ;
 ; SKX-LABEL: v8f32_no_step:
 ; SKX:       # BB#0:
 ; SKX-NEXT:    vrcp14ps %ymm0, %ymm0
-; SKX-NEXT:    retq
   %div = fdiv fast <8 x float> <float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0>, %x
   ret <8 x float> %div
 }
 
 define <8 x float> @v8f32_no_step2(<8 x float> %x) #3 {
 ; SSE-LABEL: v8f32_no_step2:
 ; SSE:       # BB#0:
 ; SSE-NEXT:    rcpps %xmm1, %xmm1
 ; SSE-NEXT:    rcpps %xmm0, %xmm0
 ; SSE-NEXT:    mulps {{.*}}(%rip), %xmm0
 ; SSE-NEXT:    mulps {{.*}}(%rip), %xmm1
-; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: v8f32_no_step2:
 ; AVX-RECIP:       # BB#0:
 ; AVX-RECIP-NEXT:    vrcpps %ymm0, %ymm0
 ; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: v8f32_no_step2:
 ; FMA-RECIP:       # BB#0:
 ; FMA-RECIP-NEXT:    vrcpps %ymm0, %ymm0
 ; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: v8f32_no_step2:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vrcpps %ymm0, %ymm0
-; BTVER2-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; BTVER2-NEXT:    retq
+; BTVER2-NEXT:    vrcpps %ymm0, %ymm0 # Latency = 2
+; BTVER2-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # Latency = 7
 ;
 ; SANDY-LABEL: v8f32_no_step2:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpps %ymm0, %ymm0
-; SANDY-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; SANDY-NEXT:    retq
+; SANDY-NEXT:    vrcpps %ymm0, %ymm0 # Latency = 5
+; SANDY-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # Latency = 9
 ;
 ; HASWELL-LABEL: v8f32_no_step2:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpps %ymm0, %ymm0
-; HASWELL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; HASWELL-NEXT:    retq
+; HASWELL-NEXT:    vrcpps %ymm0, %ymm0 # Latency = 7
+; HASWELL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # Latency = 9
 ;
 ; HASWELL-NO-FMA-LABEL: v8f32_no_step2:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm0
-; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; HASWELL-NO-FMA-NEXT:    retq
+; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm0 # Latency = 7
+; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # Latency = 9
 ;
 ; KNL-LABEL: v8f32_no_step2:
 ; KNL:       # BB#0:
-; KNL-NEXT:    vrcpps %ymm0, %ymm0
-; KNL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; KNL-NEXT:    retq
+; KNL-NEXT:    vrcpps %ymm0, %ymm0 # Latency = 7
+; KNL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # Latency = 9
 ;
 ; SKX-LABEL: v8f32_no_step2:
 ; SKX:       # BB#0:
 ; SKX-NEXT:    vrcp14ps %ymm0, %ymm0
-; SKX-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
-; SKX-NEXT:    retq
+; SKX-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # Latency = 9
   %div = fdiv fast <8 x float> <float 1.0, float 2.0, float 3.0, float 4.0, float 5.0, float 6.0, float 7.0, float 8.0>, %x
   ret <8 x float> %div
 }
 
 attributes #0 = { "unsafe-fp-math"="true" "reciprocal-estimates"="!divf,!vec-divf" }
 attributes #1 = { "unsafe-fp-math"="true" "reciprocal-estimates"="divf,vec-divf" }
 attributes #2 = { "unsafe-fp-math"="true" "reciprocal-estimates"="divf:2,vec-divf:2" }
 attributes #3 = { "unsafe-fp-math"="true" "reciprocal-estimates"="divf:0,vec-divf:0" }
 

test/CodeGen/X86/recip-pic.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc < %s -mtriple=i386-unknown-linux-gnu  -enable-unsafe-fp-math -mcpu=slm -relocation-model=pic | FileCheck %s --check-prefix=CHECK
+; RUN: llc < %s -mtriple=i386-unknown-linux-gnu -mcpu=slm -relocation-model=pic -print-latency | FileCheck %s --check-prefix=CHECK
 
-define fastcc float @foo(float %x) unnamed_addr #0 {
-; CHECK-LABEL: foo:
-; CHECK:       # BB#0: # %entry
-; CHECK-NEXT:    calll .L0$pb
+define float @f32_no_estimate_2(float %x) #0 {
+; CHECK-LABEL: f32_no_estimate_2:
+; CHECK:       # BB#0:
+; CHECK-NEXT:    pushl %eax # Latency = 1
 ; CHECK-NEXT:  .Lcfi0:
+; CHECK-NEXT:    .cfi_def_cfa_offset 8
+; CHECK-NEXT:    calll .L0$pb
+; CHECK-NEXT:  .Lcfi1:
 ; CHECK-NEXT:    .cfi_adjust_cfa_offset 4
 ; CHECK-NEXT:  .L0$pb:
 ; CHECK-NEXT:    popl %eax
-; CHECK-NEXT:  .Lcfi1:
+; CHECK-NEXT:  .Lcfi2:
 ; CHECK-NEXT:    .cfi_adjust_cfa_offset -4
-; CHECK-NEXT:  .Ltmp0:
+; CHECK-NEXT:  .Ltmp0: # Latency = 1
 ; CHECK-NEXT:    addl $_GLOBAL_OFFSET_TABLE_+(.Ltmp0-.L0$pb), %eax
-; CHECK-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; CHECK-NEXT:    divss %xmm0, %xmm1
-; CHECK-NEXT:    movaps %xmm1, %xmm0
-; CHECK-NEXT:    movss %xmm1, (%eax)
-; CHECK-NEXT:    retl
-entry:
+; CHECK-NEXT:    movss {{\.LCPI.*}}@GOTOFF(%eax), %xmm0 # Latency = 3
+; CHECK-NEXT:    # xmm0 = mem[0],zero,zero,zero
+; CHECK-NEXT:    divss {{[0-9]+}}(%esp), %xmm0 # Latency = 37
+; CHECK-NEXT:    movss %xmm0, (%eax) # Latency = 1
+; CHECK-NEXT:    movss %xmm0, (%esp) # Latency = 1
+; CHECK-NEXT:    flds (%esp) # Latency = 3
+; CHECK-NEXT:    popl %eax # Latency = 3
   %div = fdiv fast float 3.0, %x
   store float %div, float* undef, align 4
   ret float %div
 }
 
+define float @f32_one_step(float %x) #1 {
+; CHECK-LABEL: f32_one_step:
+; CHECK:       # BB#0:
+; CHECK-NEXT:    pushl %eax # Latency = 1
+; CHECK-NEXT:  .Lcfi3:
+; CHECK-NEXT:    .cfi_def_cfa_offset 8
+; CHECK-NEXT:    movss {{[0-9]+}}(%esp), %xmm0 # Latency = 3
+; CHECK-NEXT:    # xmm0 = mem[0],zero,zero,zero
+; CHECK-NEXT:    calll .L1$pb
+; CHECK-NEXT:  .Lcfi4:
+; CHECK-NEXT:    .cfi_adjust_cfa_offset 4
+; CHECK-NEXT:  .L1$pb:
+; CHECK-NEXT:    popl %eax
+; CHECK-NEXT:  .Lcfi5:
+; CHECK-NEXT:    .cfi_adjust_cfa_offset -4
+; CHECK-NEXT:  .Ltmp1: # Latency = 1
+; CHECK-NEXT:    addl $_GLOBAL_OFFSET_TABLE_+(.Ltmp1-.L1$pb), %eax
+; CHECK-NEXT:    movss {{\.LCPI.*}}@GOTOFF(%eax), %xmm2 # Latency = 3
+; CHECK-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; CHECK-NEXT:    rcpss %xmm0, %xmm1 # Latency = 5
+; CHECK-NEXT:    mulss %xmm1, %xmm0 # Latency = 5
+; CHECK-NEXT:    subss %xmm0, %xmm2 # Latency = 3
+; CHECK-NEXT:    mulss %xmm1, %xmm2 # Latency = 5
+; CHECK-NEXT:    addss %xmm1, %xmm2 # Latency = 3
+; CHECK-NEXT:    mulss {{\.LCPI.*}}@GOTOFF(%eax), %xmm2 # Latency = 8
+; CHECK-NEXT:    movss %xmm2, (%eax) # Latency = 1
+; CHECK-NEXT:    movss %xmm2, (%esp) # Latency = 1
+; CHECK-NEXT:    flds (%esp) # Latency = 3
+; CHECK-NEXT:    popl %eax # Latency = 3
+  %div = fdiv fast float 3.0, %x
+  store float %div, float* undef, align 4
+  ret float %div
+}
+
+define float @f32_two_steps(float %x) #2 {
+; CHECK-LABEL: f32_two_steps:
+; CHECK:       # BB#0:
+; CHECK-NEXT:    pushl %eax # Latency = 1
+; CHECK-NEXT:  .Lcfi6:
+; CHECK-NEXT:    .cfi_def_cfa_offset 8
+; CHECK-NEXT:    movss {{[0-9]+}}(%esp), %xmm0 # Latency = 3
+; CHECK-NEXT:    # xmm0 = mem[0],zero,zero,zero
+; CHECK-NEXT:    calll .L2$pb
+; CHECK-NEXT:  .Lcfi7:
+; CHECK-NEXT:    .cfi_adjust_cfa_offset 4
+; CHECK-NEXT:  .L2$pb:
+; CHECK-NEXT:    popl %eax
+; CHECK-NEXT:  .Lcfi8:
+; CHECK-NEXT:    .cfi_adjust_cfa_offset -4
+; CHECK-NEXT:  .Ltmp2: # Latency = 1
+; CHECK-NEXT:    addl $_GLOBAL_OFFSET_TABLE_+(.Ltmp2-.L2$pb), %eax
+; CHECK-NEXT:    movss {{\.LCPI.*}}@GOTOFF(%eax), %xmm3 # Latency = 3
+; CHECK-NEXT:    # xmm3 = mem[0],zero,zero,zero
+; CHECK-NEXT:    rcpss %xmm0, %xmm1 # Latency = 5
+; CHECK-NEXT:    movaps %xmm0, %xmm2 # Latency = 1
+; CHECK-NEXT:    movaps %xmm3, %xmm4 # Latency = 1
+; CHECK-NEXT:    mulss %xmm1, %xmm2 # Latency = 5
+; CHECK-NEXT:    subss %xmm2, %xmm4 # Latency = 3
+; CHECK-NEXT:    mulss %xmm1, %xmm4 # Latency = 5
+; CHECK-NEXT:    addss %xmm1, %xmm4 # Latency = 3
+; CHECK-NEXT:    mulss %xmm4, %xmm0 # Latency = 5
+; CHECK-NEXT:    subss %xmm0, %xmm3 # Latency = 3
+; CHECK-NEXT:    mulss %xmm4, %xmm3 # Latency = 5
+; CHECK-NEXT:    addss %xmm4, %xmm3 # Latency = 3
+; CHECK-NEXT:    mulss {{\.LCPI.*}}@GOTOFF(%eax), %xmm3 # Latency = 8
+; CHECK-NEXT:    movss %xmm3, (%eax) # Latency = 1
+; CHECK-NEXT:    movss %xmm3, (%esp) # Latency = 1
+; CHECK-NEXT:    flds (%esp) # Latency = 3
+; CHECK-NEXT:    popl %eax # Latency = 3
+  %div = fdiv fast float 3.0, %x
+  store float %div, float* undef, align 4
+  ret float %div
+}
 
+attributes #0 = { "unsafe-fp-math"="true" "reciprocal-estimates"="!divf,!vec-divf" }
+attributes #1 = { "unsafe-fp-math"="true" "reciprocal-estimates"="divf,vec-divf" }
+attributes #2 = { "unsafe-fp-math"="true" "reciprocal-estimates"="divf:2,vec-divf:2" }

tools/llc/llc.cpp

static cl::opt<bool> NoVerify("disable-verify", cl::Hidden,
                               cl::desc("Do not verify input module"));
 
 static cl::opt<bool> DisableSimplifyLibCalls("disable-simplify-libcalls",
                                              cl::desc("Disable simplify-libcalls"));
 
 static cl::opt<bool> ShowMCEncoding("show-mc-encoding", cl::Hidden,
                                     cl::desc("Show encoding in .s output"));
 
+static cl::opt<bool> PrintLatency("print-latency", cl::Hidden,
+                        cl::init(false),
+                        cl::desc("Print latency of instructions in .s output"));
+
 static cl::opt<bool> EnableDwarfDirectory(
     "enable-dwarf-directory", cl::Hidden,
     cl::desc("Use .file directives with an explicit directory."));
 
 static cl::opt<bool> AsmVerbose("asm-verbose",
                                 cl::desc("Add comments to directives."),
                                 cl::init(true));
 
static int compileModule(char **argv, LLVMContext &Context) {
   case '1': OLvl = CodeGenOpt::Less; break;
   case '2': OLvl = CodeGenOpt::Default; break;
   case '3': OLvl = CodeGenOpt::Aggressive; break;
   }
 
   TargetOptions Options = InitTargetOptionsFromCodeGenFlags();
   Options.DisableIntegratedAS = NoIntegratedAssembler;
   Options.MCOptions.ShowMCEncoding = ShowMCEncoding;
+  Options.MCOptions.PrintLatency = PrintLatency;
   Options.MCOptions.MCUseDwarfDirectory = EnableDwarfDirectory;
   Options.MCOptions.AsmVerbose = AsmVerbose;
   Options.MCOptions.PreserveAsmComments = PreserveComments;
   Options.MCOptions.IASSearchPaths = IncludeDirs;
 
   std::unique_ptr<TargetMachine> Target(
       TheTarget->createTargetMachine(TheTriple.getTriple(), CPUStr, FeaturesStr,
                                      Options, getRelocModel(), CMModel, OLvl));