Better testing of schedule model instruction latencies/throughputs

[avt77]

Summary

This patch should close the issue raised in PR32216. The patch introduce a new llc switch "-print-latency" which allows to add comments with latencies in output .s files. The patch opens several questions:

• there is real mess in terminology: the sources mix throughput and latency completely - we should fix it
• sometimes the selection of our sched model is unclear because the selected alternative code sequence (sometimes) looks worse than the initial code
• there are some issues with FMA support inside the model (in fact it does not work at the moment)
• some values of latency (reported by model) look rather strange

This is the initial version of the patch and I'm sure all the above issues will be resolved soon.

[RKSimon]

lib/Target/X86/InstPrinter/X86InstComments.h

22 ↗

(On Diff #91724)

Style guide - AC_PRINT_LATENCY

test/CodeGen/X86/recip-fastmath2.ll

20

Where have the retq instructions gone?

[hfinkel]

lib/Target/X86/X86MCInstLower.cpp

1278 ↗

(On Diff #91724)

Can you call TII->getInstrLatency here instead of computing it in this loop?

(if you just call SCModel->computeInstrLatency, as suggested below, this will take care of itself).

1291 ↗

(On Diff #91724)

Can't you just call SCModel->computeInstrLatency return the result? The logic there seems like exactly what you want:

unsigned
TargetSchedModel::computeInstrLatency(const MachineInstr *MI,
                                      bool UseDefaultDefLatency) const {
  // For the itinerary model, fall back to the old subtarget hook.
  // Allow subtargets to compute Bundle latencies outside the machine model.
  if (hasInstrItineraries() || MI->isBundle() ||
      (!hasInstrSchedModel() && !UseDefaultDefLatency))
    return TII->getInstrLatency(&InstrItins, *MI);
 
  if (hasInstrSchedModel()) {
    const MCSchedClassDesc *SCDesc = resolveSchedClass(MI);
    if (SCDesc->isValid())
      return computeInstrLatency(*SCDesc);
  }
  return TII->defaultDefLatency(SchedModel, *MI);
}

1333 ↗

(On Diff #91724)

This seems really useful, but is not target dependent. Can you please move this hook into the target-independent code? Maybe in void AsmPrinter::EmitFunctionBody(), around here:

default:
  EmitInstruction(&MI);
  break;

(right before the call to EmitInstruction).

[RKSimon]

How feasible is it to include the instruction's throughput in the comment as well?

[avt77]

hfinkel, could you help me? First of all could you give me a link(s) to any doc(s) related to our MCSchedModel except sources?

Next, I was told that ResourceCycles here:

class ProcWriteResources<list<ProcResourceKind> resources> {

list<ProcResourceKind> ProcResources = resources;
list<int> ResourceCycles = [];
int Latency = 1;
int NumMicroOps = 1;

could be used as Throughput of the given instruction. Is it right? Does it mean I could include it in generated comment as well? If YES I suppose it should be the max of the Cycles, right?

[spatel]

In D30941#701498, @avt77 wrote:

hfinkel, could you help me? First of all could you give me a link(s) to any doc(s) related to our MCSchedModel except sources?

Next, I was told that ResourceCycles here:

class ProcWriteResources<list<ProcResourceKind> resources> {

list<ProcResourceKind> ProcResources = resources;
list<int> ResourceCycles = [];
int Latency = 1;
int NumMicroOps = 1;

could be used as Throughput of the given instruction. Is it right? Does it mean I could include it in generated comment as well? If YES I suppose it should be the max of the Cycles, right?

I don't know if there are any docs besides the code and the code comments, but I think you are correct - the max of ResourceCycles is the inverse throughput for the instruction:

This is from include/llvm/Target/TargetSchedule.td :

// Optionally, ResourceCycles indicates the number of cycles the
// resource is consumed. Each ResourceCycles item is paired with the
// ProcResource item at the same position in its list. Since
// ResourceCycles are rarely specialized, the list may be
// incomplete. By default, resources are consumed for a single cycle,
// regardless of latency, which models a fully pipelined processing
// unit. A value of 0 for ResourceCycles means that the resource must
// be available but is not consumed, which is only relevant for
// unbuffered resources.

And this is in MachineScheduler.cpp:

// For reserved resources, record the highest cycle using the resource.
// For top-down scheduling, this is the cycle in which we schedule this
// instruction plus the number of cycles the operations reserves the
// resource.

We could abbreviate the comment string that you are adding like: [7:2].
I'm biased because that's the way I've always formatted [ latency : inverse throughput ], but I think that people that care about CPU timing will recognize that format, so you don't have to print out the words "latency" or "throughput".

[hfinkel]

In D30941#701621, @spatel wrote:

In D30941#701498, @avt77 wrote:

hfinkel, could you help me? First of all could you give me a link(s) to any doc(s) related to our MCSchedModel except sources?

Next, I was told that ResourceCycles here:

class ProcWriteResources<list<ProcResourceKind> resources> {

list<ProcResourceKind> ProcResources = resources;
list<int> ResourceCycles = [];
int Latency = 1;
int NumMicroOps = 1;

could be used as Throughput of the given instruction. Is it right? Does it mean I could include it in generated comment as well? If YES I suppose it should be the max of the Cycles, right?

I don't know if there are any docs besides the code and the code comments,

The documentation is primarily in the header and TableGen files (for better or worse).

but I think you are correct - the max of ResourceCycles is the inverse throughput for the instruction:

That's correct if the instruction can only dispatch through that one resource.

First, for itineraries, I think you can do something like this:

double Unknown = std::numeric_limits<double>::infinity()
double Throughput = Unknown;
if (IID.isEmpty())
  return Throughput;

for (const InstrStage *IS = IID.beginStage(ItinClassIndx),
           *E = IID.endStage(ItinClassIndx); IS != E; ++IS) {
  unsigned Cycles = IS->getCycles();
  if (!Cycles)
    continue;

  Throughput = std::min(Throughput, popcnt(IS->getUnits()) * 1.0/Cycles);
}

return Throughput;

For resource descriptions, I think that you want the inverse of ResourceCycles multiplied by the number of applicable resources. Something like this:

for (MCWriteProcResEntry *WPR = STI.getWriteProcResBegin(SCClass),
                                                     *WEnd = STI.getWriteProcResEnd(SCClass); WPR != WEnd; ++WPR) {
  unsigned Cycles = WPR->Cycles;
  if (!Cycles)
    return Unknown;

  unsigned NumUnits = SCModel->getProcResource(WPR->ProcResourceIdx)->NumUnits;
  Throughput = std::min(Throughput, NumUnits * 1.0/Cycles);
}

This is from include/llvm/Target/TargetSchedule.td :

// Optionally, ResourceCycles indicates the number of cycles the
// resource is consumed. Each ResourceCycles item is paired with the
// ProcResource item at the same position in its list. Since
// ResourceCycles are rarely specialized, the list may be
// incomplete. By default, resources are consumed for a single cycle,
// regardless of latency, which models a fully pipelined processing
// unit. A value of 0 for ResourceCycles means that the resource must
// be available but is not consumed, which is only relevant for
// unbuffered resources.

And this is in MachineScheduler.cpp:

// For reserved resources, record the highest cycle using the resource.
// For top-down scheduling, this is the cycle in which we schedule this
// instruction plus the number of cycles the operations reserves the
// resource.

We could abbreviate the comment string that you are adding like: [7:2].
I'm biased because that's the way I've always formatted [ latency : inverse throughput ], but I think that people that care about CPU timing will recognize that format, so you don't have to print out the words "latency" or "throughput".

[avt77]

The implementation was moved to target independent area and all Hal's comments were applied. I did not do anything with Throughput: it will be done in the patch.

[hfinkel]

In D30941#702652, @avt77 wrote:

The implementation was moved to target independent area and all Hal's comments were applied. I did not do anything with Throughput: it will be done in the patch.

Do you mean that the throughput will be done in a follow-up patch (or a later revision of this one)? Either is fine with me, although if we're going to add more than latencies we might pick a different name. More later...

[RKSimon]

In D30941#702657, @hfinkel wrote:

In D30941#702652, @avt77 wrote:

The implementation was moved to target independent area and all Hal's comments were applied. I did not do anything with Throughput: it will be done in the patch.

Do you mean that the throughput will be done in a follow-up patch (or a later revision of this one)? Either is fine with me, although if we're going to add more than latencies we might pick a different name. More later...

If possible please can we use the term 'print schedule' now so that future improvements (throughput, etc.) make sense

[avt77]

hfinkel, yes I mean I'll do it in the next version of this patch soon.
rksimon, do you mean we should rename the compiler option like "-print-schedule"?

[hfinkel]

In D30941#703612, @avt77 wrote:

hfinkel, yes I mean I'll do it in the next version of this patch soon.
rksimon, do you mean we should rename the compiler option like "-print-schedule"?

It is not clear to me why you won't just always do this when in verbose-asm mode. Thoughts on not having a separate option at all?

[avt77]

Throughput calculation is implemented.

[avt77]

In D30941#703613, @hfinkel wrote:

In D30941#703612, @avt77 wrote:

hfinkel, yes I mean I'll do it in the next version of this patch soon.
rksimon, do you mean we should rename the compiler option like "-print-schedule"?

It is not clear to me why you won't just always do this when in verbose-asm mode. Thoughts on not having a separate option at all?

We can do it without special option of course. Should I remove it and use "verbose-asm mode" instead?

[RKSimon]

include/llvm/CodeGen/TargetSchedule.h

184

Add doxygen comment.

[avt77]

Hal,
I removed the special option (-print-schedule) and tried to check-all. The result was very unpleseant but predictable:

Expected Passes    : 29409
Expected Failures  : 160
Unsupported Tests  : 442
Unexpected Failures: 643

Should I fix all these 643 failures or it's better to keep the option?

[hfinkel]

In D30941#706295, @avt77 wrote:

Hal,
I removed the special option (-print-schedule) and tried to check-all. The result was very unpleseant but predictable:

Expected Passes    : 29409
Expected Failures  : 160
Unsupported Tests  : 442
Unexpected Failures: 643

Should I fix all these 643 failures or it's better to keep the option?

No. Mostly because it is not entirely mechanical (it does not make sense to update the tests without understanding whether the results make sense; the person who updates the test should hopefully have that background). FWIW, this also does not make sense for higher-level targets (e.g. NVPTX).

I recommend the following:

1. Add a target callback to enable the printing of the scheduling information during verbose mode
2. Provide a command-line option (cl::opt) that can override that target callback (for easy testing), as in:
   
   bool Enable = OptionName.getNumOccurrences() ? OptionName : STI->shouldPrintStuff();
3. Enable this only on x86 (fix those tests, if any) (*)

Once we think this works reasonably well, we can encourage other backend maintainers to look at updating their tests and enabling it on their targets if they'd like.

(*) Given that FileCheck does prefix matching, and we're adding this information last on the line, I'm curious to understand why the tests are matching that is causing these failures.

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

766

I think that one ? is enough (three looks excessive to me).

768

One only ? here please.

[RKSimon]

This comment was removed by RKSimon.

[avt77]

I did everything accordingly to Hal's requirements except one: the default value of "print-schedule" switch is false because otherewise we have "Unexpected Failures: 530" and it's X86 tests ony. The problem is very simple: update_llc_test_checks.py generates CHECKs like here

; XOP-AVX1-NEXT: vextractf128 $1, %ymm2, %xmm5

It means FileCheck has to check the whole line but this patch adds the comment at the end of line and as result the line can't be checked properly.
Finaly what we have now:

• • we have option "-print-schedule" allowing to print [latency:throughput" in output (default is false)
  • we have enablePrintSchedInfo() - default is false
• X86 overrides enablePrintSchedInfo()

Hope that's exactly what was required.

[hfinkel]

In D30941#707464, @avt77 wrote:

I did everything accordingly to Hal's requirements except one: the default value of "print-schedule" switch is false because otherewise we have "Unexpected Failures: 530" and it's X86 tests ony. The problem is very simple: update_llc_test_checks.py generates CHECKs like here

; XOP-AVX1-NEXT: vextractf128 $1, %ymm2, %xmm5

I did not realize that CHECK-NEXT always matched the whole line. That's interesting.

In any case, if this is an update_llc_test_checks.py problem, why don't you use it to update the tests?

It means FileCheck has to check the whole line but this patch adds the comment at the end of line and as result the line can't be checked properly.
Finaly what we have now:

• • we have option "-print-schedule" allowing to print [latency:throughput" in output (default is false)
  • we have enablePrintSchedInfo() - default is false
• X86 overrides enablePrintSchedInfo()

Hope that's exactly what was required.

include/llvm/CodeGen/AsmPrinter.h

120

Variable names should start with a capital letter: EnablePrintSchedInfo

include/llvm/MC/MCTargetOptions.h

50

Remove this option.

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

1398

We don't add { } around single-statement blocks.

1400

When you remove the '}' here, I'd keep the blank line ;)

1401

The command-line option should be declared in this file. That way you can predicate using the command-line option value on getNumOccurrences() as I had indicated previously. Plus, there's no need to have the option name depend on the tool.

Full disclosure: There is another option: You can use a tristate setup, look for DefaultOnOff in lib/CodeGen/AsmPrinter/DwarfDebug.cpp.

lib/CodeGen/TargetSchedule.cpp

315

Please remove this comment and the UseDefaultRThroughput variable. We can always add it later if we'd like.

321

This Unknown value should never be returned (this is an implementation detail). Please return an Optional<double>.

[avt77]

In D30941#707504, @hfinkel wrote:

In D30941#707464, @avt77 wrote:

I did everything accordingly to Hal's requirements except one: the default value of "print-schedule" switch is false because otherewise we have "Unexpected Failures: 530" and it's X86 tests ony. The problem is very simple: update_llc_test_checks.py generates CHECKs like here

; XOP-AVX1-NEXT: vextractf128 $1, %ymm2, %xmm5

I did not realize that CHECK-NEXT always matched the whole line. That's interesting.

If you'd like to CHECK the prefix only you should use something like

CHECK: vextractf128 $1, %ymm2, %xmm5{{*}}

In any case, if this is an update_llc_test_checks.py problem, why don't you use it to update the tests?

I can use it to update the tests but it means I should update 530 tests. Is it acceptable? Should I do it? For me it is not a problem but is it OK for review?

[hfinkel]

In D30941#707537, @avt77 wrote:

In D30941#707504, @hfinkel wrote:

In D30941#707464, @avt77 wrote:

I did everything accordingly to Hal's requirements except one: the default value of "print-schedule" switch is false because otherewise we have "Unexpected Failures: 530" and it's X86 tests ony. The problem is very simple: update_llc_test_checks.py generates CHECKs like here

; XOP-AVX1-NEXT: vextractf128 $1, %ymm2, %xmm5

I did not realize that CHECK-NEXT always matched the whole line. That's interesting.

If you'd like to CHECK the prefix only you should use something like

CHECK: vextractf128 $1, %ymm2, %xmm5{{*}}

In any case, if this is an update_llc_test_checks.py problem, why don't you use it to update the tests?

I can use it to update the tests but it means I should update 530 tests. Is it acceptable? Should I do it? For me it is not a problem but is it OK for review?

I think that you should update them. I assume we'd want to do it at some point. The reason not to do it would be that the numbers aren't generally right yet. The goal is that, because of the MemoryCombiner, etc. these numbers will affect not only scheduling but also instruction selection (etc.). we want to make them very visible in the tests. So, let's make them very visible... (if others disagree, please say so).

[avt77]

The problem with failed tests raised because of new lines of comments added as result of this patch. I was wrong when I told that FileCheck does not allow adding of new comments at EOL.
I redesigned the patch to make it possible to add Latency:Throughput at the end of exisiting comment (if any). As result I was forced to change API of EmitInstruction from MCStreamer. I don't like this change because there are a lot of successors of MCStreamer but it works perfectly and maybe useful for other targets.
I regenerated (with help of update_llc_test_checks.py) 34 tests and now we have only 16 failed tests: I'm going to fix them asap.

[RKSimon]

Please can you put back the "## sched: [LAT:RCP]" sched prefix?

If at all possible I'd like this first patch to just do the minimum - provide a -print-sched option (default = false) that adds the scheduling comments. I'd probably not even include the comment 'appending' in this first patch.

After this initial patch I'm much more interested in then getting a high amount of test coverage of as many instructions as possible across each scheduler model - imo that is what we need to improve first to try and find the existing issues with the models. If we don't and we then fix them later, it will require the regeneration of a lot of tests that don't actually care about the schedule.

Only then should we begin investigating how to include this in more asm output.

[avt77]

Accordingly to requirements from Simon I inserted prefix "sched: " for scheduler comments and made "false" as default value for -print-schedule option. As result I restored original versions of all X86-tests excepting 2 ones to demonstrate the changes. Now we don't have any failed test.

[RKSimon]

A number of minor style comments.

@hfinkel does this look alright to you as a patch for initial support for scheduler comments?

include/llvm/CodeGen/AsmPrinter.h

119

style - newline before comment, add '.' to end of comment.

include/llvm/CodeGen/TargetSchedule.h

184

newline

include/llvm/Target/TargetSubtargetInfo.h

23

Any clang-format / include 'juggling' should be done as a NFC cleanup commit, not as part of a larger patch.

146

Comment says 'enable' but function says 'support' - which is it? Full stop at end of comment (style)

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

761

Add brief comment explaining what you're doing.

lib/CodeGen/TargetSchedule.cpp

258

Shouldn't this be an assert?

lib/MC/MCAsmStreamer.cpp

106 ↗

(On Diff #93611)

Explain how PrintSchedInfo will be used in comment

1584 ↗

(On Diff #93611)

Comment this

lib/MC/MCObjectStreamer.cpp

231 ↗

(On Diff #93611)

You're inconsistent leaving some unused argument without names and other with.

lib/Object/RecordStreamer.cpp

81 ↗

(On Diff #93611)

'B' is meaningless - please use a real name to explain its usage.

lib/Target/Mips/MCTargetDesc/MipsELFStreamer.h

49 ↗

(On Diff #93611)

Why have you include a default value here??

lib/Target/X86/InstPrinter/X86InstComments.h

26 ↗

(On Diff #93611)

Should this be here or inside the cpp file(s) ?

lib/Target/X86/X86Subtarget.cpp

378 ↗

(On Diff #93611)

Probably best to bring this inside creatSchedInfoStr?

379 ↗

(On Diff #93611)

typo: createSchedInfoStr

lib/Target/X86/X86Subtarget.h

631

newline

[avt77]

lib/CodeGen/TargetSchedule.cpp

258

No, that was the issue: we can have opcodes with invalid SCDesc

[avt77]

Hope, I fixed all comments raised by RKSimon.
hfinkel, what do you think about?

[RKSimon]

lib/MC/MCAsmStreamer.cpp

106 ↗

(On Diff #93611)

Don't use @param - you have to provide all params otherwise Wdocumentation fires.

lib/Target/X86/X86Subtarget.cpp

380 ↗

(On Diff #94538)

const char *SchedPrefix = " sched: [";

[avt77]

I fixed the latest requirements from RKSimon. Please, give me your feedback.

[hfinkel]

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

1405

Why is this commented out? You have a target hook and it defaults to false, so that should be fine.

lib/Target/X86/X86Subtarget.cpp

394 ↗

(On Diff #94784)

Why are these functions X86-specific?

lib/Target/X86/X86Subtarget.h

627

Put the false and the TODO here.

[avt77]

I implemeted all requirements from hfinkel.
Please, review again.

[hfinkel]

In D30941#724863, @avt77 wrote:

I implemeted all requirements from hfinkel.
Please, review again.

LGTM, thanks!

lib/Target/X86/X86Subtarget.h

627

I'd just say:

// TODO: Update the regression tests and return true.

[martell]

llvm/include/llvm/MC/MCSubtargetInfo.h:36:7: warning: 'llvm::MCSubtargetInfo' has virtual functions but non-virtual destructor [-Wnon-virtual-dtor]
Currently running into warnings relating to this

[RKSimon]

In D30941#727148, @martell wrote:

llvm/include/llvm/MC/MCSubtargetInfo.h:36:7: warning: 'llvm::MCSubtargetInfo' has virtual functions but non-virtual destructor [-Wnon-virtual-dtor]
Currently running into warnings relating to this

Should be fixed by rL300322

[martell]

Perfect, thanks for following up @RKSimon

[RKSimon]

rL300311

include/llvm/CodeGen/AsmPrinter.h

 
 #include "llvm/ADT/MapVector.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/CodeGen/DwarfStringPoolEntry.h"
 #include "llvm/CodeGen/MachineFunctionPass.h"
 #include "llvm/IR/InlineAsm.h"
+#include "llvm/CodeGen/TargetSchedule.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/SourceMgr.h"
+#include "llvm/Target/TargetSubtargetInfo.h"
 #include <cstdint>
 #include <memory>
 #include <utility>
 #include <vector>
 
 namespace llvm {
 
 class AsmPrinterHandler;
public:
   /// default, this is equal to CurrentFnSym.
   MCSymbol *CurrentFnSymForSize = nullptr;
 
   /// Map global GOT equivalent MCSymbols to GlobalVariables and keep track of
   /// its number of uses by other globals.
   typedef std::pair<const GlobalVariable *, unsigned> GOTEquivUsePair;
   MapVector<const MCSymbol *, GOTEquivUsePair> GlobalGOTEquivs;
 
+  /// The taget schedule model.
+  TargetSchedModel SchedModel;
+  /// Enable print [latency:throughput] in output .S
+  bool enablePrintSchedInfo = false;
+
 private:
   MCSymbol *CurrentFnBegin = nullptr;
   MCSymbol *CurrentFnEnd = nullptr;
   MCSymbol *CurExceptionSym = nullptr;
 
   // The garbage collection metadata printer table.
   void *GCMetadataPrinters = nullptr; // Really a DenseMap.
 

include/llvm/CodeGen/TargetSchedule.h

unsigned computeInstrLatency(const MachineInstr *MI,
                                bool UseDefaultDefLatency = true) const;
   unsigned computeInstrLatency(unsigned Opcode) const;
 
   /// \brief Output dependency latency of a pair of defs of the same register.
   ///
   /// This is typically one cycle.
   unsigned computeOutputLatency(const MachineInstr *DefMI, unsigned DefIdx,
                                 const MachineInstr *DepMI) const;
+  /// \brief Compute the reciprocal throughput of the given instruction.
+  /// UseDefaultThroughput will be used to get the desired value without
+  /// any calculations
+  double computeInstrRThroughput(const MachineInstr *MI,
+                                   bool UseDefaultThroughput = true) const;
 };
 
 } // end namespace llvm
 
 #endif // LLVM_CODEGEN_TARGETSCHEDULE_H

include/llvm/MC/MCTargetOptions.h

public:
   bool MCNoDeprecatedWarn : 1;
   bool MCSaveTempLabels : 1;
   bool MCUseDwarfDirectory : 1;
   bool MCIncrementalLinkerCompatible : 1;
   bool MCPIECopyRelocations : 1;
   bool ShowMCEncoding : 1;
   bool ShowMCInst : 1;
   bool AsmVerbose : 1;
+  bool PrintSchedule : 1;
 
   /// Preserve Comments in Assembly.
   bool PreserveAsmComments : 1;
 
   int DwarfVersion = 0;
 
   std::string ABIName;
 
return (ARE_EQUAL(SanitizeAddress) &&
           ARE_EQUAL(MCFatalWarnings) &&
           ARE_EQUAL(MCNoWarn) &&
           ARE_EQUAL(MCNoDeprecatedWarn) &&
           ARE_EQUAL(MCSaveTempLabels) &&
           ARE_EQUAL(MCUseDwarfDirectory) &&
           ARE_EQUAL(MCIncrementalLinkerCompatible) &&
           ARE_EQUAL(MCPIECopyRelocations) &&
           ARE_EQUAL(ShowMCEncoding) &&
+          ARE_EQUAL(PrintSchedule) &&
           ARE_EQUAL(ShowMCInst) &&
           ARE_EQUAL(AsmVerbose) &&
           ARE_EQUAL(DwarfVersion) &&
           ARE_EQUAL(ABIName) &&
           ARE_EQUAL(IASSearchPaths));
 #undef ARE_EQUAL
 }
 
 inline bool operator!=(const MCTargetOptions &LHS, const MCTargetOptions &RHS) {
   return !(LHS == RHS);
 }
 
 } // end namespace llvm
 
 #endif // LLVM_MC_MCTARGETOPTIONS_H

include/llvm/Target/TargetSubtargetInfo.h

 #define LLVM_TARGET_TARGETSUBTARGETINFO_H
 
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/CodeGen/PBQPRAConstraint.h"
 #include "llvm/CodeGen/SchedulerRegistry.h"
 #include "llvm/CodeGen/ScheduleDAGMutation.h"
 #include "llvm/MC/MCSubtargetInfo.h"
 #include "llvm/Support/CodeGen.h"
 #include <memory>
 #include <vector>
 
 namespace llvm {
 
 class CallLowering;
 class InstructionSelector;
public:
   /// \brief True if the subtarget should run MachineScheduler after aggressive
   /// coalescing.
   ///
   /// This currently replaces the SelectionDAG scheduler with the "source" order
   /// scheduler (though see below for an option to turn this off and use the
   /// TargetLowering preference). It does not yet disable the postRA scheduler.
   virtual bool enableMachineScheduler() const;
 
+  /// \brief Enable print [latency:throughput] comment in output .S file
+  virtual bool enablePrintSchedInfo() const { return false; }
+
   /// \brief True if the machine scheduler should disable the TLI preference
   /// for preRA scheduling with the source level scheduler.
   virtual bool enableMachineSchedDefaultSched() const { return true; }
 
   /// \brief True if the subtarget should enable joining global copies.
   ///
   /// By default this is enabled if the machine scheduler is enabled, but
   /// can be overridden.

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

 #include "llvm/CodeGen/MachineInstr.h"
 #include "llvm/CodeGen/MachineInstrBundle.h"
 #include "llvm/CodeGen/MachineJumpTableInfo.h"
 #include "llvm/CodeGen/MachineLoopInfo.h"
 #include "llvm/CodeGen/MachineMemOperand.h"
 #include "llvm/CodeGen/MachineModuleInfoImpls.h"
 #include "llvm/CodeGen/MachineOperand.h"
 #include "llvm/CodeGen/MachineOptimizationRemarkEmitter.h"
+#include "llvm/CodeGen/TargetSchedule.h"
 #include "llvm/IR/BasicBlock.h"
 #include "llvm/IR/Constant.h"
 #include "llvm/IR/Constants.h"
 #include "llvm/IR/DataLayout.h"
 #include "llvm/IR/DebugInfoMetadata.h"
 #include "llvm/IR/DerivedTypes.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/GlobalAlias.h"
void AsmPrinter::EmitFunctionEntryLabel() {
   if (CurrentFnSym->isDefined())
     report_fatal_error("'" + Twine(CurrentFnSym->getName()) +
                        "' label emitted multiple times to assembly file");
 
   return OutStreamer->EmitLabel(CurrentFnSym);
 }
 
 /// emitComments - Pretty-print comments for instructions.
-static void emitComments(const MachineInstr &MI, raw_ostream &CommentOS) {
+static void emitComments(AsmPrinter *AP, const MachineInstr &MI,
+                         raw_ostream &CommentOS) {
   const MachineFunction *MF = MI.getParent()->getParent();
   const TargetInstrInfo *TII = MF->getSubtarget().getInstrInfo();
 
   // Check for spills and reloads
   int FI;
 
   const MachineFrameInfo &MFI = MF->getFrameInfo();
 
static void emitComments(AsmPrinter *AP, const MachineInstr &MI,
   } else if (TII->hasStoreToStackSlot(MI, MMO, FI)) {
     if (MFI.isSpillSlotObjectIndex(FI))
       CommentOS << MMO->getSize() << "-byte Folded Spill\n";
   }
 
   // Check for spill-induced copies
   if (MI.getAsmPrinterFlag(MachineInstr::ReloadReuse))
     CommentOS << " Reload Reuse\n";
+
+  if (AP->enablePrintSchedInfo) {
+    if (!(MI.isPseudo() || MI.isTerminator())) {
+      auto Latency = AP->SchedModel.computeInstrLatency(&MI);
+      auto RThroughput = AP->SchedModel.computeInstrRThroughput(&MI);
+      if (Latency > 0 && RThroughput != std::numeric_limits<double>::infinity())
+        CommentOS << "[" << Latency << format(":%2.2f",  RThroughput) << "]\n";
+      else if (Latency > 0)
+        CommentOS << "[" << Latency << ":?]\n";
+      else if (RThroughput != std::numeric_limits<double>::infinity())
+        CommentOS << "[?:" << RThroughput << "]\n";
+    }
+  }
 }
 
 /// emitImplicitDef - This method emits the specified machine instruction
 /// that is an implicit def.
 void AsmPrinter::emitImplicitDef(const MachineInstr *MI) const {
   unsigned RegNo = MI->getOperand(0).getReg();
 
   SmallString<128> Str;
for (auto &MI : MBB) {
           NamedRegionTimer T(HI.TimerName, HI.TimerDescription,
                              HI.TimerGroupName, HI.TimerGroupDescription,
                              TimePassesIsEnabled);
           HI.Handler->beginInstruction(&MI);
         }
       }
 
       if (isVerbose())
-        emitComments(MI, OutStreamer->GetCommentOS());
+        emitComments(this, MI, OutStreamer->GetCommentOS());
 
       switch (MI.getOpcode()) {
       case TargetOpcode::CFI_INSTRUCTION:
         emitCFIInstruction(MI);
         break;
 
       case TargetOpcode::LOCAL_ESCAPE:
         emitFrameAlloc(MI);
void AsmPrinter::SetupMachineFunction(MachineFunction &MF) {
   if (!MF.getLandingPads().empty() || MMI->hasDebugInfo() ||
       MF.hasEHFunclets() || NeedsLocalForSize) {
     CurrentFnBegin = createTempSymbol("func_begin");
     if (NeedsLocalForSize)
       CurrentFnSymForSize = CurrentFnBegin;
   }
 
   ORE = &getAnalysis<MachineOptimizationRemarkEmitterPass>().getORE();
-  if (isVerbose())
+  if (isVerbose()) {
     LI = &getAnalysis<MachineLoopInfo>();
   }
+  if (TM.Options.MCOptions.PrintSchedule &&
+        MF.getSubtarget().enablePrintSchedInfo()) {
+    enablePrintSchedInfo = true;
+    const TargetSubtargetInfo &STI = MF.getSubtarget();
+    SchedModel.init(STI.getSchedModel(), &STI, STI.getInstrInfo());
+  }
+}
 
 namespace {
 
 // Keep track the alignment, constpool entries per Section.
   struct SectionCPs {
     MCSection *S;
     unsigned Alignment;
     SmallVector<unsigned, 4> CPEs;

lib/CodeGen/TargetSchedule.cpp

unsigned TargetSchedModel::computeInstrLatency(unsigned Opcode) const {
 
   unsigned SCIdx = TII->get(Opcode).getSchedClass();
   const MCSchedClassDesc *SCDesc = SchedModel.getSchedClassDesc(SCIdx);
 
   if (SCDesc->isValid() && !SCDesc->isVariant())
     return computeInstrLatency(*SCDesc);
 
   llvm_unreachable("No MI sched latency");
 }
 
 unsigned
 TargetSchedModel::computeInstrLatency(const MachineInstr *MI,
                                       bool UseDefaultDefLatency) const {
   // For the itinerary model, fall back to the old subtarget hook.
   // Allow subtargets to compute Bundle latencies outside the machine model.
   if (hasInstrItineraries() || MI->isBundle() ||
       (!hasInstrSchedModel() && !UseDefaultDefLatency))
if (SCDesc->isValid()) {
              *PRE = STI->getWriteProcResEnd(SCDesc); PRI != PRE; ++PRI) {
         if (!SchedModel.getProcResource(PRI->ProcResourceIdx)->BufferSize)
           return 1;
       }
     }
   }
   return 0;
 }
+
+double
+TargetSchedModel::computeInstrRThroughput(const MachineInstr *MI,
+                                          bool UseDefaultRThroughput) const {
+  if (UseDefaultRThroughput) {
+    // TODO: we should arrange something like we have on http://www.agner.org/
+  }
+  double Unknown = std::numeric_limits<double>::infinity();
+  double Throughput = Unknown;
+  if (!hasInstrSchedModel() && !hasInstrItineraries())
+    return Unknown;
+
+  if (hasInstrItineraries()) {
+    unsigned schedClass = MI->getDesc().getSchedClass();
+    auto *IID = getInstrItineraries();
+    for (const InstrStage *IS = IID->beginStage(schedClass),
+                          *E = IID->endStage(schedClass);
+         IS != E; ++IS) {
+      unsigned Cycles = IS->getCycles();
+      if (!Cycles)
+        continue;
+      Throughput =
+          std::min(Throughput, countPopulation(IS->getUnits()) * 1.0 / Cycles);
+    }
+    // We need reciprocal throughput that's why we return such value
+    return 1 / Throughput;
+  }
+  if (hasInstrSchedModel()) {
+    auto *SCDesc = resolveSchedClass(MI);
+    for (const MCWriteProcResEntry *WPR = STI->getWriteProcResBegin(SCDesc),
+                                   *WEnd = STI->getWriteProcResEnd(SCDesc);
+         WPR != WEnd; ++WPR) {
+      unsigned Cycles = WPR->Cycles;
+      if (!Cycles)
+        return Unknown;
+
+      unsigned NumUnits =
+          SchedModel.getProcResource(WPR->ProcResourceIdx)->NumUnits;
+      Throughput = std::min(Throughput, NumUnits * 1.0 / Cycles);
+    }
+    // We need reciprocal throughput that's why we return such value
+    return 1 / Throughput;
+  }
+  return Unknown;
+}

lib/MC/MCTargetOptions.cpp

 using namespace llvm;
 
 MCTargetOptions::MCTargetOptions()
     : SanitizeAddress(false), MCRelaxAll(false), MCNoExecStack(false),
       MCFatalWarnings(false), MCNoWarn(false), MCNoDeprecatedWarn(false),
       MCSaveTempLabels(false), MCUseDwarfDirectory(false),
       MCIncrementalLinkerCompatible(false), MCPIECopyRelocations(false),
       ShowMCEncoding(false), ShowMCInst(false), AsmVerbose(false),
+      PrintSchedule(false),
       PreserveAsmComments(true) {}
 
 StringRef MCTargetOptions::getABIName() const {
   return ABIName;
 }

lib/Target/X86/X86Subtarget.h

public:
 
   /// This function returns true if the target has sincos() routine in its
   /// compiler runtime or math libraries.
   bool hasSinCos() const;
 
   /// Enable the MachineScheduler pass for all X86 subtargets.
   bool enableMachineScheduler() const override { return true; }
 
+  bool enablePrintSchedInfo() const override { return true; }
+
   bool enableEarlyIfConversion() const override;
 
   /// Return the instruction itineraries based on the subtarget selection.
   const InstrItineraryData *getInstrItineraryData() const override {
     return &InstrItins;
   }
 
   AntiDepBreakMode getAntiDepBreakMode() const override {
     return TargetSubtargetInfo::ANTIDEP_CRITICAL;
   }
 };
 
 } // end namespace llvm
 
 #endif // LLVM_LIB_TARGET_X86_X86SUBTARGET_H

test/CodeGen/X86/recip-fastmath2.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+sse2     | FileCheck %s --check-prefix=CHECK --check-prefix=SSE --check-prefix=SSE-RECIP
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx      | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=AVX-RECIP
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx,+fma | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=FMA-RECIP
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mcpu=btver2     | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=BTVER2
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mcpu=sandybridge| FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=SANDY
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mcpu=haswell    | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=HASWELL
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mcpu=haswell -mattr=-fma | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=HASWELL-NO-FMA
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mcpu=knl        | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=AVX512 --check-prefix=KNL
-; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mcpu=skx        | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=AVX512 --check-prefix=SKX
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -print-schedule -mattr=+sse2     | FileCheck %s --check-prefix=CHECK --check-prefix=SSE --check-prefix=SSE-RECIP
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -print-schedule -mattr=+avx      | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=AVX-RECIP
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -print-schedule -mattr=+avx,+fma | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=FMA-RECIP
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -print-schedule -mcpu=btver2     | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=BTVER2
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -print-schedule -mcpu=sandybridge| FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=SANDY
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -print-schedule -mcpu=haswell    | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=HASWELL
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -print-schedule -mcpu=haswell -mattr=-fma | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=HASWELL-NO-FMA
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -print-schedule -mcpu=knl        | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=AVX512 --check-prefix=KNL
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -print-schedule -mcpu=skx        | FileCheck %s --check-prefix=CHECK --check-prefix=AVX --check-prefix=AVX512 --check-prefix=SKX
 
 ; It's the extra tests coverage for recip as discussed on D26855.
 
+define float @f32_no_estimate_2(float %x) #0 {
+; SSE-LABEL: f32_no_estimate_2:
+; SSE:       # BB#0:
+; SSE-NEXT:    movss {{.*}}(%rip), %xmm1 # [4:?]
+; SSE-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; SSE-NEXT:    divss %xmm0, %xmm1 # [10:?]
+; SSE-NEXT:    movaps %xmm1, %xmm0 # [1:?]
+; SSE-NEXT:    retq
+;
+; AVX-RECIP-LABEL: f32_no_estimate_2:
+; AVX-RECIP:       # BB#0:
+; AVX-RECIP-NEXT:    vmovss {{.*}}(%rip), %xmm1 # [4:?]
+; AVX-RECIP-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; AVX-RECIP-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # [10:?]
+; AVX-RECIP-NEXT:    retq
+;
+; FMA-RECIP-LABEL: f32_no_estimate_2:
+; FMA-RECIP:       # BB#0:
+; FMA-RECIP-NEXT:    vmovss {{.*}}(%rip), %xmm1 # [4:?]
+; FMA-RECIP-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; FMA-RECIP-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # [10:?]
+; FMA-RECIP-NEXT:    retq
+;
+; BTVER2-LABEL: f32_no_estimate_2:
+; BTVER2:       # BB#0:
+; BTVER2-NEXT:    vmovss {{.*}}(%rip), %xmm1 # [5:1.00]
+; BTVER2-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; BTVER2-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # [19:19.00]
+; BTVER2-NEXT:    retq
+;
+; SANDY-LABEL: f32_no_estimate_2:
+; SANDY:       # BB#0:
+; SANDY-NEXT:    vmovss {{.*}}(%rip), %xmm1 # [4:0.50]
+; SANDY-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; SANDY-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # [12:1.00]
+; SANDY-NEXT:    retq
+;
+; HASWELL-LABEL: f32_no_estimate_2:
+; HASWELL:       # BB#0:
+; HASWELL-NEXT:    vmovss {{.*}}(%rip), %xmm1 # [4:0.50]
+; HASWELL-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; HASWELL-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # [12:1.00]
+; HASWELL-NEXT:    retq
+;
+; HASWELL-NO-FMA-LABEL: f32_no_estimate_2:
+; HASWELL-NO-FMA:       # BB#0:
+; HASWELL-NO-FMA-NEXT:    vmovss {{.*}}(%rip), %xmm1 # [4:0.50]
+; HASWELL-NO-FMA-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; HASWELL-NO-FMA-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # [12:1.00]
+; HASWELL-NO-FMA-NEXT:    retq
+;
+; AVX512-LABEL: f32_no_estimate_2:
+; AVX512:       # BB#0:
+; AVX512-NEXT:    vmovss {{.*}}(%rip), %xmm1 # [4:0.50]
+; AVX512-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; AVX512-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # [12:1.00]
+; AVX512-NEXT:    retq
+  %div = fdiv fast float 1234.0, %x
+  ret float %div
+}
+
 define float @f32_no_step_2(float %x) #3 {
 ; SSE-LABEL: f32_no_step_2:
 ; SSE:       # BB#0:
-; SSE-NEXT:    rcpss %xmm0, %xmm0
-; SSE-NEXT:    mulss {{.*}}(%rip), %xmm0
+; SSE-NEXT:    rcpss %xmm0, %xmm0 # [1:?]
+; SSE-NEXT:    mulss {{.*}}(%rip), %xmm0 # [4:?]
 ; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: f32_no_step_2:
 ; AVX-RECIP:       # BB#0:
-; AVX-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm0
-; AVX-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; AVX-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [4:?]
 ; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: f32_no_step_2:
 ; FMA-RECIP:       # BB#0:
-; FMA-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm0
-; FMA-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; FMA-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm0 # [1:?]
+; FMA-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [4:?]
 ; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: f32_no_step_2:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vrcpss %xmm0, %xmm0, %xmm0
-; BTVER2-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; BTVER2-NEXT:    vrcpss %xmm0, %xmm0, %xmm0 # [2:1.00]
+; BTVER2-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [7:1.00]
 ; BTVER2-NEXT:    retq
 ;
 ; SANDY-LABEL: f32_no_step_2:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpss %xmm0, %xmm0, %xmm0
-; SANDY-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; SANDY-NEXT:    vrcpss %xmm0, %xmm0, %xmm0 # [5:1.00]
+; SANDY-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [9:1.00]
 ; SANDY-NEXT:    retq
 ;
 ; HASWELL-LABEL: f32_no_step_2:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpss %xmm0, %xmm0, %xmm0
-; HASWELL-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; HASWELL-NEXT:    vrcpss %xmm0, %xmm0, %xmm0 # [5:1.00]
+; HASWELL-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [9:0.50]
 ; HASWELL-NEXT:    retq
 ;
 ; HASWELL-NO-FMA-LABEL: f32_no_step_2:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpss %xmm0, %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; HASWELL-NO-FMA-NEXT:    vrcpss %xmm0, %xmm0, %xmm0 # [5:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [9:0.50]
 ; HASWELL-NO-FMA-NEXT:    retq
 ;
 ; AVX512-LABEL: f32_no_step_2:
 ; AVX512:       # BB#0:
-; AVX512-NEXT:    vrcp14ss %xmm0, %xmm0, %xmm0
-; AVX512-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; AVX512-NEXT:    vrcp14ss %xmm0, %xmm0, %xmm0 # [1:0.00]
+; AVX512-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [9:0.50]
 ; AVX512-NEXT:    retq
   %div = fdiv fast float 1234.0, %x
   ret float %div
 }
 
 define float @f32_one_step_2(float %x) #1 {
 ; SSE-LABEL: f32_one_step_2:
 ; SSE:       # BB#0:
-; SSE-NEXT:    rcpss %xmm0, %xmm2
-; SSE-NEXT:    mulss %xmm2, %xmm0
-; SSE-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; SSE-NEXT:    subss %xmm0, %xmm1
-; SSE-NEXT:    mulss %xmm2, %xmm1
-; SSE-NEXT:    addss %xmm2, %xmm1
-; SSE-NEXT:    mulss {{.*}}(%rip), %xmm1
-; SSE-NEXT:    movaps %xmm1, %xmm0
+; SSE-NEXT:    rcpss %xmm0, %xmm2 # [1:?]
+; SSE-NEXT:    mulss %xmm2, %xmm0 # [1:?]
+; SSE-NEXT:    movss {{.*}}(%rip), %xmm1 # [4:?]
+; SSE-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; SSE-NEXT:    subss %xmm0, %xmm1 # [1:?]
+; SSE-NEXT:    mulss %xmm2, %xmm1 # [1:?]
+; SSE-NEXT:    addss %xmm2, %xmm1 # [1:?]
+; SSE-NEXT:    mulss {{.*}}(%rip), %xmm1 # [4:?]
+; SSE-NEXT:    movaps %xmm1, %xmm0 # [1:?]
 ; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: f32_one_step_2:
 ; AVX-RECIP:       # BB#0:
-; AVX-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; AVX-RECIP-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; AVX-RECIP-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; AVX-RECIP-NEXT:    vsubss %xmm0, %xmm2, %xmm0
-; AVX-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; AVX-RECIP-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; AVX-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; AVX-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # [1:?]
+; AVX-RECIP-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vmovss {{.*}}(%rip), %xmm2 # [4:?]
+; AVX-RECIP-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; AVX-RECIP-NEXT:    vsubss %xmm0, %xmm2, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [4:?]
 ; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: f32_one_step_2:
 ; FMA-RECIP:       # BB#0:
-; FMA-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; FMA-RECIP-NEXT:    vfnmadd213ss {{.*}}(%rip), %xmm1, %xmm0
-; FMA-RECIP-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm0
-; FMA-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; FMA-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # [1:?]
+; FMA-RECIP-NEXT:    vfnmadd213ss {{.*}}(%rip), %xmm1, %xmm0 # [4:?]
+; FMA-RECIP-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm0 # [1:?]
+; FMA-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [4:?]
 ; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: f32_one_step_2:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; BTVER2-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; BTVER2-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; BTVER2-NEXT:    vsubss %xmm0, %xmm2, %xmm0
-; BTVER2-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; BTVER2-NEXT:    vmovss {{.*}}(%rip), %xmm2 # [5:1.00]
+; BTVER2-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; BTVER2-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # [2:1.00]
+; BTVER2-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # [2:1.00]
+; BTVER2-NEXT:    vsubss %xmm0, %xmm2, %xmm0 # [3:1.00]
+; BTVER2-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [2:1.00]
+; BTVER2-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # [3:1.00]
+; BTVER2-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [7:1.00]
 ; BTVER2-NEXT:    retq
 ;
 ; SANDY-LABEL: f32_one_step_2:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; SANDY-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; SANDY-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; SANDY-NEXT:    vsubss %xmm0, %xmm2, %xmm0
-; SANDY-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; SANDY-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # [5:1.00]
+; SANDY-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # [5:1.00]
+; SANDY-NEXT:    vmovss {{.*}}(%rip), %xmm2 # [4:0.50]
+; SANDY-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; SANDY-NEXT:    vsubss %xmm0, %xmm2, %xmm0 # [3:1.00]
+; SANDY-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [5:1.00]
+; SANDY-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # [3:1.00]
+; SANDY-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [9:1.00]
 ; SANDY-NEXT:    retq
 ;
 ; HASWELL-LABEL: f32_one_step_2:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; HASWELL-NEXT:    vfnmadd213ss {{.*}}(%rip), %xmm1, %xmm0
-; HASWELL-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm0
-; HASWELL-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; HASWELL-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # [5:1.00]
+; HASWELL-NEXT:    vfnmadd213ss {{.*}}(%rip), %xmm1, %xmm0 # [4:0.00]
+; HASWELL-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm0 # [1:0.00]
+; HASWELL-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [9:0.50]
 ; HASWELL-NEXT:    retq
 ;
 ; HASWELL-NO-FMA-LABEL: f32_one_step_2:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; HASWELL-NO-FMA-NEXT:    vsubss %xmm0, %xmm2, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; HASWELL-NO-FMA-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # [5:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # [5:0.50]
+; HASWELL-NO-FMA-NEXT:    vmovss {{.*}}(%rip), %xmm2 # [4:0.50]
+; HASWELL-NO-FMA-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; HASWELL-NO-FMA-NEXT:    vsubss %xmm0, %xmm2, %xmm0 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [5:0.50]
+; HASWELL-NO-FMA-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [9:0.50]
 ; HASWELL-NO-FMA-NEXT:    retq
 ;
 ; AVX512-LABEL: f32_one_step_2:
 ; AVX512:       # BB#0:
-; AVX512-NEXT:    vrcp14ss %xmm0, %xmm0, %xmm1
-; AVX512-NEXT:    vfnmadd213ss {{.*}}(%rip), %xmm1, %xmm0
-; AVX512-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm0
-; AVX512-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; AVX512-NEXT:    vrcp14ss %xmm0, %xmm0, %xmm1 # [1:0.00]
+; AVX512-NEXT:    vfnmadd213ss {{.*}}(%rip), %xmm1, %xmm0 # [4:0.00]
+; AVX512-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm0 # [1:0.00]
+; AVX512-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [9:0.50]
 ; AVX512-NEXT:    retq
   %div = fdiv fast float 3456.0, %x
   ret float %div
 }
 
+define float @f32_no_estimate_2_divs(float %x) #0 {
+; SSE-LABEL: f32_no_estimate_2_divs:
+; SSE:       # BB#0:
+; SSE-NEXT:    movss {{.*}}(%rip), %xmm1 # [4:?]
+; SSE-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; SSE-NEXT:    divss %xmm0, %xmm1 # [10:?]
+; SSE-NEXT:    movss {{.*}}(%rip), %xmm0 # [4:?]
+; SSE-NEXT:    # xmm0 = mem[0],zero,zero,zero
+; SSE-NEXT:    mulss %xmm1, %xmm0 # [1:?]
+; SSE-NEXT:    mulss %xmm1, %xmm0 # [1:?]
+; SSE-NEXT:    retq
+;
+; AVX-RECIP-LABEL: f32_no_estimate_2_divs:
+; AVX-RECIP:       # BB#0:
+; AVX-RECIP-NEXT:    vmovss {{.*}}(%rip), %xmm1 # [4:?]
+; AVX-RECIP-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; AVX-RECIP-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # [10:?]
+; AVX-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # [4:?]
+; AVX-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    retq
+;
+; FMA-RECIP-LABEL: f32_no_estimate_2_divs:
+; FMA-RECIP:       # BB#0:
+; FMA-RECIP-NEXT:    vmovss {{.*}}(%rip), %xmm1 # [4:?]
+; FMA-RECIP-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; FMA-RECIP-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # [10:?]
+; FMA-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # [4:?]
+; FMA-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [1:?]
+; FMA-RECIP-NEXT:    retq
+;
+; BTVER2-LABEL: f32_no_estimate_2_divs:
+; BTVER2:       # BB#0:
+; BTVER2-NEXT:    vmovss {{.*}}(%rip), %xmm1 # [5:1.00]
+; BTVER2-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; BTVER2-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # [19:19.00]
+; BTVER2-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # [7:1.00]
+; BTVER2-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [2:1.00]
+; BTVER2-NEXT:    retq
+;
+; SANDY-LABEL: f32_no_estimate_2_divs:
+; SANDY:       # BB#0:
+; SANDY-NEXT:    vmovss {{.*}}(%rip), %xmm1 # [4:0.50]
+; SANDY-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; SANDY-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # [12:1.00]
+; SANDY-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # [9:1.00]
+; SANDY-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [5:1.00]
+; SANDY-NEXT:    retq
+;
+; HASWELL-LABEL: f32_no_estimate_2_divs:
+; HASWELL:       # BB#0:
+; HASWELL-NEXT:    vmovss {{.*}}(%rip), %xmm1 # [4:0.50]
+; HASWELL-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; HASWELL-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # [12:1.00]
+; HASWELL-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # [9:0.50]
+; HASWELL-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [5:0.50]
+; HASWELL-NEXT:    retq
+;
+; HASWELL-NO-FMA-LABEL: f32_no_estimate_2_divs:
+; HASWELL-NO-FMA:       # BB#0:
+; HASWELL-NO-FMA-NEXT:    vmovss {{.*}}(%rip), %xmm1 # [4:0.50]
+; HASWELL-NO-FMA-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; HASWELL-NO-FMA-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # [12:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # [9:0.50]
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [5:0.50]
+; HASWELL-NO-FMA-NEXT:    retq
+;
+; AVX512-LABEL: f32_no_estimate_2_divs:
+; AVX512:       # BB#0:
+; AVX512-NEXT:    vmovss {{.*}}(%rip), %xmm1 # [4:0.50]
+; AVX512-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; AVX512-NEXT:    vdivss %xmm0, %xmm1, %xmm0 # [12:1.00]
+; AVX512-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # [9:0.50]
+; AVX512-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [5:0.50]
+; AVX512-NEXT:    retq
+  %div = fdiv fast float 3456.0, %x
+  %div2 = fdiv fast float %div, %x
+  ret float %div2
+}
+
 define float @f32_one_step_2_divs(float %x) #1 {
 ; SSE-LABEL: f32_one_step_2_divs:
 ; SSE:       # BB#0:
-; SSE-NEXT:    rcpss %xmm0, %xmm1
-; SSE-NEXT:    mulss %xmm1, %xmm0
-; SSE-NEXT:    movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; SSE-NEXT:    subss %xmm0, %xmm2
-; SSE-NEXT:    mulss %xmm1, %xmm2
-; SSE-NEXT:    addss %xmm1, %xmm2
-; SSE-NEXT:    movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
-; SSE-NEXT:    mulss %xmm2, %xmm0
-; SSE-NEXT:    mulss %xmm2, %xmm0
+; SSE-NEXT:    rcpss %xmm0, %xmm1 # [1:?]
+; SSE-NEXT:    mulss %xmm1, %xmm0 # [1:?]
+; SSE-NEXT:    movss {{.*}}(%rip), %xmm2 # [4:?]
+; SSE-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; SSE-NEXT:    subss %xmm0, %xmm2 # [1:?]
+; SSE-NEXT:    mulss %xmm1, %xmm2 # [1:?]
+; SSE-NEXT:    addss %xmm1, %xmm2 # [1:?]
+; SSE-NEXT:    movss {{.*}}(%rip), %xmm0 # [4:?]
+; SSE-NEXT:    # xmm0 = mem[0],zero,zero,zero
+; SSE-NEXT:    mulss %xmm2, %xmm0 # [1:?]
+; SSE-NEXT:    mulss %xmm2, %xmm0 # [1:?]
 ; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: f32_one_step_2_divs:
 ; AVX-RECIP:       # BB#0:
-; AVX-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; AVX-RECIP-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; AVX-RECIP-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; AVX-RECIP-NEXT:    vsubss %xmm0, %xmm2, %xmm0
-; AVX-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; AVX-RECIP-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; AVX-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1
-; AVX-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0
+; AVX-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # [1:?]
+; AVX-RECIP-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vmovss {{.*}}(%rip), %xmm2 # [4:?]
+; AVX-RECIP-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; AVX-RECIP-NEXT:    vsubss %xmm0, %xmm2, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # [4:?]
+; AVX-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [1:?]
 ; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: f32_one_step_2_divs:
 ; FMA-RECIP:       # BB#0:
-; FMA-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; FMA-RECIP-NEXT:    vfnmadd213ss {{.*}}(%rip), %xmm1, %xmm0
-; FMA-RECIP-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm0
-; FMA-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1
-; FMA-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0
+; FMA-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # [1:?]
+; FMA-RECIP-NEXT:    vfnmadd213ss {{.*}}(%rip), %xmm1, %xmm0 # [4:?]
+; FMA-RECIP-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm0 # [1:?]
+; FMA-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # [4:?]
+; FMA-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [1:?]
 ; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: f32_one_step_2_divs:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; BTVER2-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; BTVER2-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; BTVER2-NEXT:    vsubss %xmm0, %xmm2, %xmm0
-; BTVER2-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1
-; BTVER2-NEXT:    vmulss %xmm0, %xmm1, %xmm0
+; BTVER2-NEXT:    vmovss {{.*}}(%rip), %xmm2 # [5:1.00]
+; BTVER2-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; BTVER2-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # [2:1.00]
+; BTVER2-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # [2:1.00]
+; BTVER2-NEXT:    vsubss %xmm0, %xmm2, %xmm0 # [3:1.00]
+; BTVER2-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [2:1.00]
+; BTVER2-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # [3:1.00]
+; BTVER2-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # [7:1.00]
+; BTVER2-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [2:1.00]
 ; BTVER2-NEXT:    retq
 ;
 ; SANDY-LABEL: f32_one_step_2_divs:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; SANDY-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; SANDY-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; SANDY-NEXT:    vsubss %xmm0, %xmm2, %xmm0
-; SANDY-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1
-; SANDY-NEXT:    vmulss %xmm0, %xmm1, %xmm0
+; SANDY-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # [5:1.00]
+; SANDY-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # [5:1.00]
+; SANDY-NEXT:    vmovss {{.*}}(%rip), %xmm2 # [4:0.50]
+; SANDY-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; SANDY-NEXT:    vsubss %xmm0, %xmm2, %xmm0 # [3:1.00]
+; SANDY-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [5:1.00]
+; SANDY-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # [3:1.00]
+; SANDY-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # [9:1.00]
+; SANDY-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [5:1.00]
 ; SANDY-NEXT:    retq
 ;
 ; HASWELL-LABEL: f32_one_step_2_divs:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; HASWELL-NEXT:    vfnmadd213ss {{.*}}(%rip), %xmm1, %xmm0
-; HASWELL-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm0
-; HASWELL-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1
-; HASWELL-NEXT:    vmulss %xmm0, %xmm1, %xmm0
+; HASWELL-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # [5:1.00]
+; HASWELL-NEXT:    vfnmadd213ss {{.*}}(%rip), %xmm1, %xmm0 # [4:0.00]
+; HASWELL-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm0 # [1:0.00]
+; HASWELL-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # [9:0.50]
+; HASWELL-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [5:0.50]
 ; HASWELL-NEXT:    retq
 ;
 ; HASWELL-NO-FMA-LABEL: f32_one_step_2_divs:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; HASWELL-NO-FMA-NEXT:    vsubss %xmm0, %xmm2, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulss %xmm0, %xmm1, %xmm0
+; HASWELL-NO-FMA-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # [5:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # [5:0.50]
+; HASWELL-NO-FMA-NEXT:    vmovss {{.*}}(%rip), %xmm2 # [4:0.50]
+; HASWELL-NO-FMA-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; HASWELL-NO-FMA-NEXT:    vsubss %xmm0, %xmm2, %xmm0 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [5:0.50]
+; HASWELL-NO-FMA-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # [9:0.50]
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [5:0.50]
 ; HASWELL-NO-FMA-NEXT:    retq
 ;
 ; AVX512-LABEL: f32_one_step_2_divs:
 ; AVX512:       # BB#0:
-; AVX512-NEXT:    vrcp14ss %xmm0, %xmm0, %xmm1
-; AVX512-NEXT:    vfnmadd213ss {{.*}}(%rip), %xmm1, %xmm0
-; AVX512-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm0
-; AVX512-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1
-; AVX512-NEXT:    vmulss %xmm0, %xmm1, %xmm0
+; AVX512-NEXT:    vrcp14ss %xmm0, %xmm0, %xmm1 # [1:0.00]
+; AVX512-NEXT:    vfnmadd213ss {{.*}}(%rip), %xmm1, %xmm0 # [4:0.00]
+; AVX512-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm0 # [1:0.00]
+; AVX512-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm1 # [9:0.50]
+; AVX512-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [5:0.50]
 ; AVX512-NEXT:    retq
   %div = fdiv fast float 3456.0, %x
   %div2 = fdiv fast float %div, %x
   ret float %div2
 }
 
 define float @f32_two_step_2(float %x) #2 {
 ; SSE-LABEL: f32_two_step_2:
 ; SSE:       # BB#0:
-; SSE-NEXT:    rcpss %xmm0, %xmm2
-; SSE-NEXT:    movaps %xmm0, %xmm3
-; SSE-NEXT:    mulss %xmm2, %xmm3
-; SSE-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; SSE-NEXT:    movaps %xmm1, %xmm4
-; SSE-NEXT:    subss %xmm3, %xmm4
-; SSE-NEXT:    mulss %xmm2, %xmm4
-; SSE-NEXT:    addss %xmm2, %xmm4
-; SSE-NEXT:    mulss %xmm4, %xmm0
-; SSE-NEXT:    subss %xmm0, %xmm1
-; SSE-NEXT:    mulss %xmm4, %xmm1
-; SSE-NEXT:    addss %xmm4, %xmm1
-; SSE-NEXT:    mulss {{.*}}(%rip), %xmm1
-; SSE-NEXT:    movaps %xmm1, %xmm0
+; SSE-NEXT:    rcpss %xmm0, %xmm2 # [1:?]
+; SSE-NEXT:    movaps %xmm0, %xmm3 # [1:?]
+; SSE-NEXT:    mulss %xmm2, %xmm3 # [1:?]
+; SSE-NEXT:    movss {{.*}}(%rip), %xmm1 # [4:?]
+; SSE-NEXT:    # xmm1 = mem[0],zero,zero,zero
+; SSE-NEXT:    movaps %xmm1, %xmm4 # [1:?]
+; SSE-NEXT:    subss %xmm3, %xmm4 # [1:?]
+; SSE-NEXT:    mulss %xmm2, %xmm4 # [1:?]
+; SSE-NEXT:    addss %xmm2, %xmm4 # [1:?]
+; SSE-NEXT:    mulss %xmm4, %xmm0 # [1:?]
+; SSE-NEXT:    subss %xmm0, %xmm1 # [1:?]
+; SSE-NEXT:    mulss %xmm4, %xmm1 # [1:?]
+; SSE-NEXT:    addss %xmm4, %xmm1 # [1:?]
+; SSE-NEXT:    mulss {{.*}}(%rip), %xmm1 # [4:?]
+; SSE-NEXT:    movaps %xmm1, %xmm0 # [1:?]
 ; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: f32_two_step_2:
 ; AVX-RECIP:       # BB#0:
-; AVX-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; AVX-RECIP-NEXT:    vmulss %xmm1, %xmm0, %xmm2
-; AVX-RECIP-NEXT:    vmovss {{.*#+}} xmm3 = mem[0],zero,zero,zero
-; AVX-RECIP-NEXT:    vsubss %xmm2, %xmm3, %xmm2
-; AVX-RECIP-NEXT:    vmulss %xmm2, %xmm1, %xmm2
-; AVX-RECIP-NEXT:    vaddss %xmm2, %xmm1, %xmm1
-; AVX-RECIP-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; AVX-RECIP-NEXT:    vsubss %xmm0, %xmm3, %xmm0
-; AVX-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; AVX-RECIP-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; AVX-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; AVX-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # [1:?]
+; AVX-RECIP-NEXT:    vmulss %xmm1, %xmm0, %xmm2 # [1:?]
+; AVX-RECIP-NEXT:    vmovss {{.*}}(%rip), %xmm3 # [4:?]
+; AVX-RECIP-NEXT:    # xmm3 = mem[0],zero,zero,zero
+; AVX-RECIP-NEXT:    vsubss %xmm2, %xmm3, %xmm2 # [1:?]
+; AVX-RECIP-NEXT:    vmulss %xmm2, %xmm1, %xmm2 # [1:?]
+; AVX-RECIP-NEXT:    vaddss %xmm2, %xmm1, %xmm1 # [1:?]
+; AVX-RECIP-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vsubss %xmm0, %xmm3, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [4:?]
 ; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: f32_two_step_2:
 ; FMA-RECIP:       # BB#0:
-; FMA-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; FMA-RECIP-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; FMA-RECIP-NEXT:    vmovaps %xmm1, %xmm3
-; FMA-RECIP-NEXT:    vfnmadd213ss %xmm2, %xmm0, %xmm3
-; FMA-RECIP-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm3
-; FMA-RECIP-NEXT:    vfnmadd213ss %xmm2, %xmm3, %xmm0
-; FMA-RECIP-NEXT:    vfmadd132ss %xmm3, %xmm3, %xmm0
-; FMA-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; FMA-RECIP-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # [1:?]
+; FMA-RECIP-NEXT:    vmovss {{.*}}(%rip), %xmm2 # [4:?]
+; FMA-RECIP-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; FMA-RECIP-NEXT:    vmovaps %xmm1, %xmm3 # [1:?]
+; FMA-RECIP-NEXT:    vfnmadd213ss %xmm2, %xmm0, %xmm3 # [1:?]
+; FMA-RECIP-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm3 # [1:?]
+; FMA-RECIP-NEXT:    vfnmadd213ss %xmm2, %xmm3, %xmm0 # [1:?]
+; FMA-RECIP-NEXT:    vfmadd132ss %xmm3, %xmm3, %xmm0 # [1:?]
+; FMA-RECIP-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [4:?]
 ; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: f32_two_step_2:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vmovss {{.*#+}} xmm3 = mem[0],zero,zero,zero
-; BTVER2-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; BTVER2-NEXT:    vmulss %xmm1, %xmm0, %xmm2
-; BTVER2-NEXT:    vsubss %xmm2, %xmm3, %xmm2
-; BTVER2-NEXT:    vmulss %xmm2, %xmm1, %xmm2
-; BTVER2-NEXT:    vaddss %xmm2, %xmm1, %xmm1
-; BTVER2-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; BTVER2-NEXT:    vsubss %xmm0, %xmm3, %xmm0
-; BTVER2-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; BTVER2-NEXT:    vmovss {{.*}}(%rip), %xmm3 # [5:1.00]
+; BTVER2-NEXT:    # xmm3 = mem[0],zero,zero,zero
+; BTVER2-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # [2:1.00]
+; BTVER2-NEXT:    vmulss %xmm1, %xmm0, %xmm2 # [2:1.00]
+; BTVER2-NEXT:    vsubss %xmm2, %xmm3, %xmm2 # [3:1.00]
+; BTVER2-NEXT:    vmulss %xmm2, %xmm1, %xmm2 # [2:1.00]
+; BTVER2-NEXT:    vaddss %xmm2, %xmm1, %xmm1 # [3:1.00]
+; BTVER2-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # [2:1.00]
+; BTVER2-NEXT:    vsubss %xmm0, %xmm3, %xmm0 # [3:1.00]
+; BTVER2-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [2:1.00]
+; BTVER2-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # [3:1.00]
+; BTVER2-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [7:1.00]
 ; BTVER2-NEXT:    retq
 ;
 ; SANDY-LABEL: f32_two_step_2:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; SANDY-NEXT:    vmulss %xmm1, %xmm0, %xmm2
-; SANDY-NEXT:    vmovss {{.*#+}} xmm3 = mem[0],zero,zero,zero
-; SANDY-NEXT:    vsubss %xmm2, %xmm3, %xmm2
-; SANDY-NEXT:    vmulss %xmm2, %xmm1, %xmm2
-; SANDY-NEXT:    vaddss %xmm2, %xmm1, %xmm1
-; SANDY-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; SANDY-NEXT:    vsubss %xmm0, %xmm3, %xmm0
-; SANDY-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; SANDY-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # [5:1.00]
+; SANDY-NEXT:    vmulss %xmm1, %xmm0, %xmm2 # [5:1.00]
+; SANDY-NEXT:    vmovss {{.*}}(%rip), %xmm3 # [4:0.50]
+; SANDY-NEXT:    # xmm3 = mem[0],zero,zero,zero
+; SANDY-NEXT:    vsubss %xmm2, %xmm3, %xmm2 # [3:1.00]
+; SANDY-NEXT:    vmulss %xmm2, %xmm1, %xmm2 # [5:1.00]
+; SANDY-NEXT:    vaddss %xmm2, %xmm1, %xmm1 # [3:1.00]
+; SANDY-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # [5:1.00]
+; SANDY-NEXT:    vsubss %xmm0, %xmm3, %xmm0 # [3:1.00]
+; SANDY-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [5:1.00]
+; SANDY-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # [3:1.00]
+; SANDY-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [9:1.00]
 ; SANDY-NEXT:    retq
 ;
 ; HASWELL-LABEL: f32_two_step_2:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; HASWELL-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; HASWELL-NEXT:    vmovaps %xmm1, %xmm3
-; HASWELL-NEXT:    vfnmadd213ss %xmm2, %xmm0, %xmm3
-; HASWELL-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm3
-; HASWELL-NEXT:    vfnmadd213ss %xmm2, %xmm3, %xmm0
-; HASWELL-NEXT:    vfmadd132ss %xmm3, %xmm3, %xmm0
-; HASWELL-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; HASWELL-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # [5:1.00]
+; HASWELL-NEXT:    vmovss {{.*}}(%rip), %xmm2 # [4:0.50]
+; HASWELL-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; HASWELL-NEXT:    vmovaps %xmm1, %xmm3 # [1:1.00]
+; HASWELL-NEXT:    vfnmadd213ss %xmm2, %xmm0, %xmm3 # [1:0.00]
+; HASWELL-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm3 # [1:0.00]
+; HASWELL-NEXT:    vfnmadd213ss %xmm2, %xmm3, %xmm0 # [1:0.00]
+; HASWELL-NEXT:    vfmadd132ss %xmm3, %xmm3, %xmm0 # [1:0.00]
+; HASWELL-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [9:0.50]
 ; HASWELL-NEXT:    retq
 ;
 ; HASWELL-NO-FMA-LABEL: f32_two_step_2:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpss %xmm0, %xmm0, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulss %xmm1, %xmm0, %xmm2
-; HASWELL-NO-FMA-NEXT:    vmovss {{.*#+}} xmm3 = mem[0],zero,zero,zero
-; HASWELL-NO-FMA-NEXT:    vsubss %xmm2, %xmm3, %xmm2
-; HASWELL-NO-FMA-NEXT:    vmulss %xmm2, %xmm1, %xmm2
-; HASWELL-NO-FMA-NEXT:    vaddss %xmm2, %xmm1, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulss %xmm1, %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    vsubss %xmm0, %xmm3, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulss %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vaddss %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; HASWELL-NO-FMA-NEXT:    vrcpss %xmm0, %xmm0, %xmm1 # [5:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm1, %xmm0, %xmm2 # [5:0.50]
+; HASWELL-NO-FMA-NEXT:    vmovss {{.*}}(%rip), %xmm3 # [4:0.50]
+; HASWELL-NO-FMA-NEXT:    # xmm3 = mem[0],zero,zero,zero
+; HASWELL-NO-FMA-NEXT:    vsubss %xmm2, %xmm3, %xmm2 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm2, %xmm1, %xmm2 # [5:0.50]
+; HASWELL-NO-FMA-NEXT:    vaddss %xmm2, %xmm1, %xmm1 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm1, %xmm0, %xmm0 # [5:0.50]
+; HASWELL-NO-FMA-NEXT:    vsubss %xmm0, %xmm3, %xmm0 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulss %xmm0, %xmm1, %xmm0 # [5:0.50]
+; HASWELL-NO-FMA-NEXT:    vaddss %xmm0, %xmm1, %xmm0 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [9:0.50]
 ; HASWELL-NO-FMA-NEXT:    retq
 ;
 ; AVX512-LABEL: f32_two_step_2:
 ; AVX512:       # BB#0:
-; AVX512-NEXT:    vrcp14ss %xmm0, %xmm0, %xmm1
-; AVX512-NEXT:    vmovss {{.*#+}} xmm2 = mem[0],zero,zero,zero
-; AVX512-NEXT:    vmovaps %xmm1, %xmm3
-; AVX512-NEXT:    vfnmadd213ss %xmm2, %xmm0, %xmm3
-; AVX512-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm3
-; AVX512-NEXT:    vfnmadd213ss %xmm2, %xmm3, %xmm0
-; AVX512-NEXT:    vfmadd132ss %xmm3, %xmm3, %xmm0
-; AVX512-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0
+; AVX512-NEXT:    vrcp14ss %xmm0, %xmm0, %xmm1 # [1:0.00]
+; AVX512-NEXT:    vmovss {{.*}}(%rip), %xmm2 # [4:0.50]
+; AVX512-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; AVX512-NEXT:    vmovaps %xmm1, %xmm3 # [1:1.00]
+; AVX512-NEXT:    vfnmadd213ss %xmm2, %xmm0, %xmm3 # [1:0.00]
+; AVX512-NEXT:    vfmadd132ss %xmm1, %xmm1, %xmm3 # [1:0.00]
+; AVX512-NEXT:    vfnmadd213ss %xmm2, %xmm3, %xmm0 # [1:0.00]
+; AVX512-NEXT:    vfmadd132ss %xmm3, %xmm3, %xmm0 # [1:0.00]
+; AVX512-NEXT:    vmulss {{.*}}(%rip), %xmm0, %xmm0 # [9:0.50]
 ; AVX512-NEXT:    retq
   %div = fdiv fast float 6789.0, %x
   ret float %div
 }
 
+define <4 x float> @v4f32_no_estimate2(<4 x float> %x) #0 {
+; SSE-LABEL: v4f32_no_estimate2:
+; SSE:       # BB#0:
+; SSE-NEXT:    movaps {{.*}}(%rip), %xmm1 # [4:?]
+; SSE-NEXT:    # xmm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; SSE-NEXT:    divps %xmm0, %xmm1 # [10:?]
+; SSE-NEXT:    movaps %xmm1, %xmm0 # [1:?]
+; SSE-NEXT:    retq
+;
+; AVX-RECIP-LABEL: v4f32_no_estimate2:
+; AVX-RECIP:       # BB#0:
+; AVX-RECIP-NEXT:    vmovaps {{.*}}(%rip), %xmm1 # [4:?]
+; AVX-RECIP-NEXT:    # xmm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; AVX-RECIP-NEXT:    vdivps %xmm0, %xmm1, %xmm0 # [10:?]
+; AVX-RECIP-NEXT:    retq
+;
+; FMA-RECIP-LABEL: v4f32_no_estimate2:
+; FMA-RECIP:       # BB#0:
+; FMA-RECIP-NEXT:    vmovaps {{.*}}(%rip), %xmm1 # [4:?]
+; FMA-RECIP-NEXT:    # xmm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; FMA-RECIP-NEXT:    vdivps %xmm0, %xmm1, %xmm0 # [10:?]
+; FMA-RECIP-NEXT:    retq
+;
+; BTVER2-LABEL: v4f32_no_estimate2:
+; BTVER2:       # BB#0:
+; BTVER2-NEXT:    vmovaps {{.*}}(%rip), %xmm1 # [5:1.00]
+; BTVER2-NEXT:    # xmm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; BTVER2-NEXT:    vdivps %xmm0, %xmm1, %xmm0 # [19:19.00]
+; BTVER2-NEXT:    retq
+;
+; SANDY-LABEL: v4f32_no_estimate2:
+; SANDY:       # BB#0:
+; SANDY-NEXT:    vmovaps {{.*}}(%rip), %xmm1 # [4:0.50]
+; SANDY-NEXT:    # xmm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; SANDY-NEXT:    vdivps %xmm0, %xmm1, %xmm0 # [12:1.00]
+; SANDY-NEXT:    retq
+;
+; HASWELL-LABEL: v4f32_no_estimate2:
+; HASWELL:       # BB#0:
+; HASWELL-NEXT:    vmovaps {{.*}}(%rip), %xmm1 # [4:0.50]
+; HASWELL-NEXT:    # xmm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; HASWELL-NEXT:    vdivps %xmm0, %xmm1, %xmm0 # [12:1.00]
+; HASWELL-NEXT:    retq
+;
+; HASWELL-NO-FMA-LABEL: v4f32_no_estimate2:
+; HASWELL-NO-FMA:       # BB#0:
+; HASWELL-NO-FMA-NEXT:    vmovaps {{.*}}(%rip), %xmm1 # [4:0.50]
+; HASWELL-NO-FMA-NEXT:    # xmm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; HASWELL-NO-FMA-NEXT:    vdivps %xmm0, %xmm1, %xmm0 # [12:1.00]
+; HASWELL-NO-FMA-NEXT:    retq
+;
+; AVX512-LABEL: v4f32_no_estimate2:
+; AVX512:       # BB#0:
+; AVX512-NEXT:    vmovaps {{.*}}(%rip), %xmm1 # [4:0.50]
+; AVX512-NEXT:    # xmm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; AVX512-NEXT:    vdivps %xmm0, %xmm1, %xmm0 # [12:1.00]
+; AVX512-NEXT:    retq
+  %div = fdiv fast <4 x float> <float 1.0, float 2.0, float 3.0, float 4.0>, %x
+  ret <4 x float> %div
+}
+
 define <4 x float> @v4f32_one_step2(<4 x float> %x) #1 {
 ; SSE-LABEL: v4f32_one_step2:
 ; SSE:       # BB#0:
-; SSE-NEXT:    rcpps %xmm0, %xmm2
-; SSE-NEXT:    mulps %xmm2, %xmm0
-; SSE-NEXT:    movaps {{.*#+}} xmm1 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; SSE-NEXT:    subps %xmm0, %xmm1
-; SSE-NEXT:    mulps %xmm2, %xmm1
-; SSE-NEXT:    addps %xmm2, %xmm1
-; SSE-NEXT:    mulps {{.*}}(%rip), %xmm1
-; SSE-NEXT:    movaps %xmm1, %xmm0
+; SSE-NEXT:    rcpps %xmm0, %xmm2 # [1:?]
+; SSE-NEXT:    mulps %xmm2, %xmm0 # [1:?]
+; SSE-NEXT:    movaps {{.*}}(%rip), %xmm1 # [4:?]
+; SSE-NEXT:    # xmm1 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; SSE-NEXT:    subps %xmm0, %xmm1 # [1:?]
+; SSE-NEXT:    mulps %xmm2, %xmm1 # [1:?]
+; SSE-NEXT:    addps %xmm2, %xmm1 # [1:?]
+; SSE-NEXT:    mulps {{.*}}(%rip), %xmm1 # [4:?]
+; SSE-NEXT:    movaps %xmm1, %xmm0 # [1:?]
 ; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: v4f32_one_step2:
 ; AVX-RECIP:       # BB#0:
-; AVX-RECIP-NEXT:    vrcpps %xmm0, %xmm1
-; AVX-RECIP-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; AVX-RECIP-NEXT:    vmovaps {{.*#+}} xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; AVX-RECIP-NEXT:    vsubps %xmm0, %xmm2, %xmm0
-; AVX-RECIP-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; AVX-RECIP-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
+; AVX-RECIP-NEXT:    vrcpps %xmm0, %xmm1 # [1:?]
+; AVX-RECIP-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vmovaps {{.*}}(%rip), %xmm2 # [4:?]
+; AVX-RECIP-NEXT:    # xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; AVX-RECIP-NEXT:    vsubps %xmm0, %xmm2, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # [4:?]
 ; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: v4f32_one_step2:
 ; FMA-RECIP:       # BB#0:
-; FMA-RECIP-NEXT:    vrcpps %xmm0, %xmm1
-; FMA-RECIP-NEXT:    vfnmadd213ps {{.*}}(%rip), %xmm1, %xmm0
-; FMA-RECIP-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0
-; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
+; FMA-RECIP-NEXT:    vrcpps %xmm0, %xmm1 # [1:?]
+; FMA-RECIP-NEXT:    vfnmadd213ps {{.*}}(%rip), %xmm1, %xmm0 # [4:?]
+; FMA-RECIP-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0 # [1:?]
+; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # [4:?]
 ; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: v4f32_one_step2:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vmovaps {{.*#+}} xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; BTVER2-NEXT:    vrcpps %xmm0, %xmm1
-; BTVER2-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; BTVER2-NEXT:    vsubps %xmm0, %xmm2, %xmm0
-; BTVER2-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
+; BTVER2-NEXT:    vmovaps {{.*}}(%rip), %xmm2 # [5:1.00]
+; BTVER2-NEXT:    # xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; BTVER2-NEXT:    vrcpps %xmm0, %xmm1 # [2:1.00]
+; BTVER2-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # [2:1.00]
+; BTVER2-NEXT:    vsubps %xmm0, %xmm2, %xmm0 # [3:1.00]
+; BTVER2-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [2:1.00]
+; BTVER2-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # [3:1.00]
+; BTVER2-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # [7:1.00]
 ; BTVER2-NEXT:    retq
 ;
 ; SANDY-LABEL: v4f32_one_step2:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpps %xmm0, %xmm1
-; SANDY-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; SANDY-NEXT:    vmovaps {{.*#+}} xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; SANDY-NEXT:    vsubps %xmm0, %xmm2, %xmm0
-; SANDY-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
+; SANDY-NEXT:    vrcpps %xmm0, %xmm1 # [5:1.00]
+; SANDY-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # [5:1.00]
+; SANDY-NEXT:    vmovaps {{.*}}(%rip), %xmm2 # [4:0.50]
+; SANDY-NEXT:    # xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; SANDY-NEXT:    vsubps %xmm0, %xmm2, %xmm0 # [3:1.00]
+; SANDY-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [5:1.00]
+; SANDY-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # [3:1.00]
+; SANDY-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # [9:1.00]
 ; SANDY-NEXT:    retq
 ;
 ; HASWELL-LABEL: v4f32_one_step2:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpps %xmm0, %xmm1
-; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2
-; HASWELL-NEXT:    vfnmadd213ps %xmm2, %xmm1, %xmm0
-; HASWELL-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0
-; HASWELL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
+; HASWELL-NEXT:    vrcpps %xmm0, %xmm1 # [5:1.00]
+; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2 # [4:0.50]
+; HASWELL-NEXT:    vfnmadd213ps %xmm2, %xmm1, %xmm0 # [1:0.00]
+; HASWELL-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0 # [1:0.00]
+; HASWELL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # [9:0.50]
 ; HASWELL-NEXT:    retq
 ;
 ; HASWELL-NO-FMA-LABEL: v4f32_one_step2:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpps %xmm0, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2
-; HASWELL-NO-FMA-NEXT:    vsubps %xmm0, %xmm2, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
+; HASWELL-NO-FMA-NEXT:    vrcpps %xmm0, %xmm1 # [5:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # [5:0.50]
+; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2 # [4:0.50]
+; HASWELL-NO-FMA-NEXT:    vsubps %xmm0, %xmm2, %xmm0 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [5:0.50]
+; HASWELL-NO-FMA-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # [9:0.50]
 ; HASWELL-NO-FMA-NEXT:    retq
 ;
 ; KNL-LABEL: v4f32_one_step2:
 ; KNL:       # BB#0:
-; KNL-NEXT:    vrcpps %xmm0, %xmm1
-; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2
-; KNL-NEXT:    vfnmadd213ps %xmm2, %xmm1, %xmm0
-; KNL-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0
-; KNL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
+; KNL-NEXT:    vrcpps %xmm0, %xmm1 # [5:1.00]
+; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2 # [4:0.50]
+; KNL-NEXT:    vfnmadd213ps %xmm2, %xmm1, %xmm0 # [1:0.00]
+; KNL-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0 # [1:0.00]
+; KNL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # [9:0.50]
 ; KNL-NEXT:    retq
 ;
 ; SKX-LABEL: v4f32_one_step2:
 ; SKX:       # BB#0:
-; SKX-NEXT:    vrcp14ps %xmm0, %xmm1
-; SKX-NEXT:    vfnmadd213ps {{.*}}(%rip){1to4}, %xmm1, %xmm0
-; SKX-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0
-; SKX-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
+; SKX-NEXT:    vrcp14ps %xmm0, %xmm1 # [1:0.00]
+; SKX-NEXT:    vfnmadd213ps {{.*}}(%rip){1to4}, %xmm1, %xmm0 # [4:0.00]
+; SKX-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0 # [1:0.00]
+; SKX-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # [9:0.50]
 ; SKX-NEXT:    retq
   %div = fdiv fast <4 x float> <float 1.0, float 2.0, float 3.0, float 4.0>, %x
   ret <4 x float> %div
 }
 
 define <4 x float> @v4f32_one_step_2_divs(<4 x float> %x) #1 {
 ; SSE-LABEL: v4f32_one_step_2_divs:
 ; SSE:       # BB#0:
-; SSE-NEXT:    rcpps %xmm0, %xmm1
-; SSE-NEXT:    mulps %xmm1, %xmm0
-; SSE-NEXT:    movaps {{.*#+}} xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; SSE-NEXT:    subps %xmm0, %xmm2
-; SSE-NEXT:    mulps %xmm1, %xmm2
-; SSE-NEXT:    addps %xmm1, %xmm2
-; SSE-NEXT:    movaps {{.*#+}} xmm0 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
-; SSE-NEXT:    mulps %xmm2, %xmm0
-; SSE-NEXT:    mulps %xmm2, %xmm0
+; SSE-NEXT:    rcpps %xmm0, %xmm1 # [1:?]
+; SSE-NEXT:    mulps %xmm1, %xmm0 # [1:?]
+; SSE-NEXT:    movaps {{.*}}(%rip), %xmm2 # [4:?]
+; SSE-NEXT:    # xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; SSE-NEXT:    subps %xmm0, %xmm2 # [1:?]
+; SSE-NEXT:    mulps %xmm1, %xmm2 # [1:?]
+; SSE-NEXT:    addps %xmm1, %xmm2 # [1:?]
+; SSE-NEXT:    movaps {{.*}}(%rip), %xmm0 # [4:?]
+; SSE-NEXT:    # xmm0 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; SSE-NEXT:    mulps %xmm2, %xmm0 # [1:?]
+; SSE-NEXT:    mulps %xmm2, %xmm0 # [1:?]
 ; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: v4f32_one_step_2_divs:
 ; AVX-RECIP:       # BB#0:
-; AVX-RECIP-NEXT:    vrcpps %xmm0, %xmm1
-; AVX-RECIP-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; AVX-RECIP-NEXT:    vmovaps {{.*#+}} xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; AVX-RECIP-NEXT:    vsubps %xmm0, %xmm2, %xmm0
-; AVX-RECIP-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; AVX-RECIP-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1
-; AVX-RECIP-NEXT:    vmulps %xmm0, %xmm1, %xmm0
+; AVX-RECIP-NEXT:    vrcpps %xmm0, %xmm1 # [1:?]
+; AVX-RECIP-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vmovaps {{.*}}(%rip), %xmm2 # [4:?]
+; AVX-RECIP-NEXT:    # xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; AVX-RECIP-NEXT:    vsubps %xmm0, %xmm2, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1 # [4:?]
+; AVX-RECIP-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [1:?]
 ; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: v4f32_one_step_2_divs:
 ; FMA-RECIP:       # BB#0:
-; FMA-RECIP-NEXT:    vrcpps %xmm0, %xmm1
-; FMA-RECIP-NEXT:    vfnmadd213ps {{.*}}(%rip), %xmm1, %xmm0
-; FMA-RECIP-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0
-; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1
-; FMA-RECIP-NEXT:    vmulps %xmm0, %xmm1, %xmm0
+; FMA-RECIP-NEXT:    vrcpps %xmm0, %xmm1 # [1:?]
+; FMA-RECIP-NEXT:    vfnmadd213ps {{.*}}(%rip), %xmm1, %xmm0 # [4:?]
+; FMA-RECIP-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0 # [1:?]
+; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1 # [4:?]
+; FMA-RECIP-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [1:?]
 ; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: v4f32_one_step_2_divs:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vmovaps {{.*#+}} xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; BTVER2-NEXT:    vrcpps %xmm0, %xmm1
-; BTVER2-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; BTVER2-NEXT:    vsubps %xmm0, %xmm2, %xmm0
-; BTVER2-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1
-; BTVER2-NEXT:    vmulps %xmm0, %xmm1, %xmm0
+; BTVER2-NEXT:    vmovaps {{.*}}(%rip), %xmm2 # [5:1.00]
+; BTVER2-NEXT:    # xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; BTVER2-NEXT:    vrcpps %xmm0, %xmm1 # [2:1.00]
+; BTVER2-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # [2:1.00]
+; BTVER2-NEXT:    vsubps %xmm0, %xmm2, %xmm0 # [3:1.00]
+; BTVER2-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [2:1.00]
+; BTVER2-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # [3:1.00]
+; BTVER2-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1 # [7:1.00]
+; BTVER2-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [2:1.00]
 ; BTVER2-NEXT:    retq
 ;
 ; SANDY-LABEL: v4f32_one_step_2_divs:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpps %xmm0, %xmm1
-; SANDY-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; SANDY-NEXT:    vmovaps {{.*#+}} xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; SANDY-NEXT:    vsubps %xmm0, %xmm2, %xmm0
-; SANDY-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1
-; SANDY-NEXT:    vmulps %xmm0, %xmm1, %xmm0
+; SANDY-NEXT:    vrcpps %xmm0, %xmm1 # [5:1.00]
+; SANDY-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # [5:1.00]
+; SANDY-NEXT:    vmovaps {{.*}}(%rip), %xmm2 # [4:0.50]
+; SANDY-NEXT:    # xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; SANDY-NEXT:    vsubps %xmm0, %xmm2, %xmm0 # [3:1.00]
+; SANDY-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [5:1.00]
+; SANDY-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # [3:1.00]
+; SANDY-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1 # [9:1.00]
+; SANDY-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [5:1.00]
 ; SANDY-NEXT:    retq
 ;
 ; HASWELL-LABEL: v4f32_one_step_2_divs:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpps %xmm0, %xmm1
-; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2
-; HASWELL-NEXT:    vfnmadd213ps %xmm2, %xmm1, %xmm0
-; HASWELL-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0
-; HASWELL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1
-; HASWELL-NEXT:    vmulps %xmm0, %xmm1, %xmm0
+; HASWELL-NEXT:    vrcpps %xmm0, %xmm1 # [5:1.00]
+; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2 # [4:0.50]
+; HASWELL-NEXT:    vfnmadd213ps %xmm2, %xmm1, %xmm0 # [1:0.00]
+; HASWELL-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0 # [1:0.00]
+; HASWELL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1 # [9:0.50]
+; HASWELL-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [5:0.50]
 ; HASWELL-NEXT:    retq
 ;
 ; HASWELL-NO-FMA-LABEL: v4f32_one_step_2_divs:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpps %xmm0, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2
-; HASWELL-NO-FMA-NEXT:    vsubps %xmm0, %xmm2, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulps %xmm0, %xmm1, %xmm0
+; HASWELL-NO-FMA-NEXT:    vrcpps %xmm0, %xmm1 # [5:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # [5:0.50]
+; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2 # [4:0.50]
+; HASWELL-NO-FMA-NEXT:    vsubps %xmm0, %xmm2, %xmm0 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [5:0.50]
+; HASWELL-NO-FMA-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1 # [9:0.50]
+; HASWELL-NO-FMA-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [5:0.50]
 ; HASWELL-NO-FMA-NEXT:    retq
 ;
 ; KNL-LABEL: v4f32_one_step_2_divs:
 ; KNL:       # BB#0:
-; KNL-NEXT:    vrcpps %xmm0, %xmm1
-; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2
-; KNL-NEXT:    vfnmadd213ps %xmm2, %xmm1, %xmm0
-; KNL-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0
-; KNL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1
-; KNL-NEXT:    vmulps %xmm0, %xmm1, %xmm0
+; KNL-NEXT:    vrcpps %xmm0, %xmm1 # [5:1.00]
+; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2 # [4:0.50]
+; KNL-NEXT:    vfnmadd213ps %xmm2, %xmm1, %xmm0 # [1:0.00]
+; KNL-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0 # [1:0.00]
+; KNL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1 # [9:0.50]
+; KNL-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [5:0.50]
 ; KNL-NEXT:    retq
 ;
 ; SKX-LABEL: v4f32_one_step_2_divs:
 ; SKX:       # BB#0:
-; SKX-NEXT:    vrcp14ps %xmm0, %xmm1
-; SKX-NEXT:    vfnmadd213ps {{.*}}(%rip){1to4}, %xmm1, %xmm0
-; SKX-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0
-; SKX-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1
-; SKX-NEXT:    vmulps %xmm0, %xmm1, %xmm0
+; SKX-NEXT:    vrcp14ps %xmm0, %xmm1 # [1:0.00]
+; SKX-NEXT:    vfnmadd213ps {{.*}}(%rip){1to4}, %xmm1, %xmm0 # [4:0.00]
+; SKX-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm0 # [1:0.00]
+; SKX-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm1 # [9:0.50]
+; SKX-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [5:0.50]
 ; SKX-NEXT:    retq
   %div = fdiv fast <4 x float> <float 1.0, float 2.0, float 3.0, float 4.0>, %x
   %div2 = fdiv fast <4 x float> %div, %x
   ret <4 x float> %div2
 }
 
 define <4 x float> @v4f32_two_step2(<4 x float> %x) #2 {
 ; SSE-LABEL: v4f32_two_step2:
 ; SSE:       # BB#0:
-; SSE-NEXT:    rcpps %xmm0, %xmm2
-; SSE-NEXT:    movaps %xmm0, %xmm3
-; SSE-NEXT:    mulps %xmm2, %xmm3
-; SSE-NEXT:    movaps {{.*#+}} xmm1 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; SSE-NEXT:    movaps %xmm1, %xmm4
-; SSE-NEXT:    subps %xmm3, %xmm4
-; SSE-NEXT:    mulps %xmm2, %xmm4
-; SSE-NEXT:    addps %xmm2, %xmm4
-; SSE-NEXT:    mulps %xmm4, %xmm0
-; SSE-NEXT:    subps %xmm0, %xmm1
-; SSE-NEXT:    mulps %xmm4, %xmm1
-; SSE-NEXT:    addps %xmm4, %xmm1
-; SSE-NEXT:    mulps {{.*}}(%rip), %xmm1
-; SSE-NEXT:    movaps %xmm1, %xmm0
+; SSE-NEXT:    rcpps %xmm0, %xmm2 # [1:?]
+; SSE-NEXT:    movaps %xmm0, %xmm3 # [1:?]
+; SSE-NEXT:    mulps %xmm2, %xmm3 # [1:?]
+; SSE-NEXT:    movaps {{.*}}(%rip), %xmm1 # [4:?]
+; SSE-NEXT:    # xmm1 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; SSE-NEXT:    movaps %xmm1, %xmm4 # [1:?]
+; SSE-NEXT:    subps %xmm3, %xmm4 # [1:?]
+; SSE-NEXT:    mulps %xmm2, %xmm4 # [1:?]
+; SSE-NEXT:    addps %xmm2, %xmm4 # [1:?]
+; SSE-NEXT:    mulps %xmm4, %xmm0 # [1:?]
+; SSE-NEXT:    subps %xmm0, %xmm1 # [1:?]
+; SSE-NEXT:    mulps %xmm4, %xmm1 # [1:?]
+; SSE-NEXT:    addps %xmm4, %xmm1 # [1:?]
+; SSE-NEXT:    mulps {{.*}}(%rip), %xmm1 # [4:?]
+; SSE-NEXT:    movaps %xmm1, %xmm0 # [1:?]
 ; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: v4f32_two_step2:
 ; AVX-RECIP:       # BB#0:
-; AVX-RECIP-NEXT:    vrcpps %xmm0, %xmm1
-; AVX-RECIP-NEXT:    vmulps %xmm1, %xmm0, %xmm2
-; AVX-RECIP-NEXT:    vmovaps {{.*#+}} xmm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; AVX-RECIP-NEXT:    vsubps %xmm2, %xmm3, %xmm2
-; AVX-RECIP-NEXT:    vmulps %xmm2, %xmm1, %xmm2
-; AVX-RECIP-NEXT:    vaddps %xmm2, %xmm1, %xmm1
-; AVX-RECIP-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; AVX-RECIP-NEXT:    vsubps %xmm0, %xmm3, %xmm0
-; AVX-RECIP-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; AVX-RECIP-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
+; AVX-RECIP-NEXT:    vrcpps %xmm0, %xmm1 # [1:?]
+; AVX-RECIP-NEXT:    vmulps %xmm1, %xmm0, %xmm2 # [1:?]
+; AVX-RECIP-NEXT:    vmovaps {{.*}}(%rip), %xmm3 # [4:?]
+; AVX-RECIP-NEXT:    # xmm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; AVX-RECIP-NEXT:    vsubps %xmm2, %xmm3, %xmm2 # [1:?]
+; AVX-RECIP-NEXT:    vmulps %xmm2, %xmm1, %xmm2 # [1:?]
+; AVX-RECIP-NEXT:    vaddps %xmm2, %xmm1, %xmm1 # [1:?]
+; AVX-RECIP-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vsubps %xmm0, %xmm3, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # [4:?]
 ; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: v4f32_two_step2:
 ; FMA-RECIP:       # BB#0:
-; FMA-RECIP-NEXT:    vrcpps %xmm0, %xmm1
-; FMA-RECIP-NEXT:    vmovaps {{.*#+}} xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; FMA-RECIP-NEXT:    vmovaps %xmm1, %xmm3
-; FMA-RECIP-NEXT:    vfnmadd213ps %xmm2, %xmm0, %xmm3
-; FMA-RECIP-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm3
-; FMA-RECIP-NEXT:    vfnmadd213ps %xmm2, %xmm3, %xmm0
-; FMA-RECIP-NEXT:    vfmadd132ps %xmm3, %xmm3, %xmm0
-; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
+; FMA-RECIP-NEXT:    vrcpps %xmm0, %xmm1 # [1:?]
+; FMA-RECIP-NEXT:    vmovaps {{.*}}(%rip), %xmm2 # [4:?]
+; FMA-RECIP-NEXT:    # xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; FMA-RECIP-NEXT:    vmovaps %xmm1, %xmm3 # [1:?]
+; FMA-RECIP-NEXT:    vfnmadd213ps %xmm2, %xmm0, %xmm3 # [1:?]
+; FMA-RECIP-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm3 # [1:?]
+; FMA-RECIP-NEXT:    vfnmadd213ps %xmm2, %xmm3, %xmm0 # [1:?]
+; FMA-RECIP-NEXT:    vfmadd132ps %xmm3, %xmm3, %xmm0 # [1:?]
+; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # [4:?]
 ; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: v4f32_two_step2:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vmovaps {{.*#+}} xmm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; BTVER2-NEXT:    vrcpps %xmm0, %xmm1
-; BTVER2-NEXT:    vmulps %xmm1, %xmm0, %xmm2
-; BTVER2-NEXT:    vsubps %xmm2, %xmm3, %xmm2
-; BTVER2-NEXT:    vmulps %xmm2, %xmm1, %xmm2
-; BTVER2-NEXT:    vaddps %xmm2, %xmm1, %xmm1
-; BTVER2-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; BTVER2-NEXT:    vsubps %xmm0, %xmm3, %xmm0
-; BTVER2-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; BTVER2-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
+; BTVER2-NEXT:    vmovaps {{.*}}(%rip), %xmm3 # [5:1.00]
+; BTVER2-NEXT:    # xmm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; BTVER2-NEXT:    vrcpps %xmm0, %xmm1 # [2:1.00]
+; BTVER2-NEXT:    vmulps %xmm1, %xmm0, %xmm2 # [2:1.00]
+; BTVER2-NEXT:    vsubps %xmm2, %xmm3, %xmm2 # [3:1.00]
+; BTVER2-NEXT:    vmulps %xmm2, %xmm1, %xmm2 # [2:1.00]
+; BTVER2-NEXT:    vaddps %xmm2, %xmm1, %xmm1 # [3:1.00]
+; BTVER2-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # [2:1.00]
+; BTVER2-NEXT:    vsubps %xmm0, %xmm3, %xmm0 # [3:1.00]
+; BTVER2-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [2:1.00]
+; BTVER2-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # [3:1.00]
+; BTVER2-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # [7:1.00]
 ; BTVER2-NEXT:    retq
 ;
 ; SANDY-LABEL: v4f32_two_step2:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpps %xmm0, %xmm1
-; SANDY-NEXT:    vmulps %xmm1, %xmm0, %xmm2
-; SANDY-NEXT:    vmovaps {{.*#+}} xmm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; SANDY-NEXT:    vsubps %xmm2, %xmm3, %xmm2
-; SANDY-NEXT:    vmulps %xmm2, %xmm1, %xmm2
-; SANDY-NEXT:    vaddps %xmm2, %xmm1, %xmm1
-; SANDY-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; SANDY-NEXT:    vsubps %xmm0, %xmm3, %xmm0
-; SANDY-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; SANDY-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
+; SANDY-NEXT:    vrcpps %xmm0, %xmm1 # [5:1.00]
+; SANDY-NEXT:    vmulps %xmm1, %xmm0, %xmm2 # [5:1.00]
+; SANDY-NEXT:    vmovaps {{.*}}(%rip), %xmm3 # [4:0.50]
+; SANDY-NEXT:    # xmm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; SANDY-NEXT:    vsubps %xmm2, %xmm3, %xmm2 # [3:1.00]
+; SANDY-NEXT:    vmulps %xmm2, %xmm1, %xmm2 # [5:1.00]
+; SANDY-NEXT:    vaddps %xmm2, %xmm1, %xmm1 # [3:1.00]
+; SANDY-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # [5:1.00]
+; SANDY-NEXT:    vsubps %xmm0, %xmm3, %xmm0 # [3:1.00]
+; SANDY-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [5:1.00]
+; SANDY-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # [3:1.00]
+; SANDY-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # [9:1.00]
 ; SANDY-NEXT:    retq
 ;
 ; HASWELL-LABEL: v4f32_two_step2:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpps %xmm0, %xmm1
-; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2
-; HASWELL-NEXT:    vmovaps %xmm1, %xmm3
-; HASWELL-NEXT:    vfnmadd213ps %xmm2, %xmm0, %xmm3
-; HASWELL-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm3
-; HASWELL-NEXT:    vfnmadd213ps %xmm2, %xmm3, %xmm0
-; HASWELL-NEXT:    vfmadd132ps %xmm3, %xmm3, %xmm0
-; HASWELL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
+; HASWELL-NEXT:    vrcpps %xmm0, %xmm1 # [5:1.00]
+; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2 # [4:0.50]
+; HASWELL-NEXT:    vmovaps %xmm1, %xmm3 # [1:1.00]
+; HASWELL-NEXT:    vfnmadd213ps %xmm2, %xmm0, %xmm3 # [1:0.00]
+; HASWELL-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm3 # [1:0.00]
+; HASWELL-NEXT:    vfnmadd213ps %xmm2, %xmm3, %xmm0 # [1:0.00]
+; HASWELL-NEXT:    vfmadd132ps %xmm3, %xmm3, %xmm0 # [1:0.00]
+; HASWELL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # [9:0.50]
 ; HASWELL-NEXT:    retq
 ;
 ; HASWELL-NO-FMA-LABEL: v4f32_two_step2:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpps %xmm0, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulps %xmm1, %xmm0, %xmm2
-; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %xmm3
-; HASWELL-NO-FMA-NEXT:    vsubps %xmm2, %xmm3, %xmm2
-; HASWELL-NO-FMA-NEXT:    vmulps %xmm2, %xmm1, %xmm2
-; HASWELL-NO-FMA-NEXT:    vaddps %xmm2, %xmm1, %xmm1
-; HASWELL-NO-FMA-NEXT:    vmulps %xmm1, %xmm0, %xmm0
-; HASWELL-NO-FMA-NEXT:    vsubps %xmm0, %xmm3, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulps %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vaddps %xmm0, %xmm1, %xmm0
-; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
+; HASWELL-NO-FMA-NEXT:    vrcpps %xmm0, %xmm1 # [5:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps %xmm1, %xmm0, %xmm2 # [5:0.50]
+; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %xmm3 # [4:0.50]
+; HASWELL-NO-FMA-NEXT:    vsubps %xmm2, %xmm3, %xmm2 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps %xmm2, %xmm1, %xmm2 # [5:0.50]
+; HASWELL-NO-FMA-NEXT:    vaddps %xmm2, %xmm1, %xmm1 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps %xmm1, %xmm0, %xmm0 # [5:0.50]
+; HASWELL-NO-FMA-NEXT:    vsubps %xmm0, %xmm3, %xmm0 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps %xmm0, %xmm1, %xmm0 # [5:0.50]
+; HASWELL-NO-FMA-NEXT:    vaddps %xmm0, %xmm1, %xmm0 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # [9:0.50]
 ; HASWELL-NO-FMA-NEXT:    retq
 ;
 ; KNL-LABEL: v4f32_two_step2:
 ; KNL:       # BB#0:
-; KNL-NEXT:    vrcpps %xmm0, %xmm1
-; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2
-; KNL-NEXT:    vmovaps %xmm1, %xmm3
-; KNL-NEXT:    vfnmadd213ps %xmm2, %xmm0, %xmm3
-; KNL-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm3
-; KNL-NEXT:    vfnmadd213ps %xmm2, %xmm3, %xmm0
-; KNL-NEXT:    vfmadd132ps %xmm3, %xmm3, %xmm0
-; KNL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
+; KNL-NEXT:    vrcpps %xmm0, %xmm1 # [5:1.00]
+; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2 # [4:0.50]
+; KNL-NEXT:    vmovaps %xmm1, %xmm3 # [1:1.00]
+; KNL-NEXT:    vfnmadd213ps %xmm2, %xmm0, %xmm3 # [1:0.00]
+; KNL-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm3 # [1:0.00]
+; KNL-NEXT:    vfnmadd213ps %xmm2, %xmm3, %xmm0 # [1:0.00]
+; KNL-NEXT:    vfmadd132ps %xmm3, %xmm3, %xmm0 # [1:0.00]
+; KNL-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # [9:0.50]
 ; KNL-NEXT:    retq
 ;
 ; SKX-LABEL: v4f32_two_step2:
 ; SKX:       # BB#0:
-; SKX-NEXT:    vrcp14ps %xmm0, %xmm1
-; SKX-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2
-; SKX-NEXT:    vmovaps %xmm1, %xmm3
-; SKX-NEXT:    vfnmadd213ps %xmm2, %xmm0, %xmm3
-; SKX-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm3
-; SKX-NEXT:    vfnmadd213ps %xmm2, %xmm3, %xmm0
-; SKX-NEXT:    vfmadd132ps %xmm3, %xmm3, %xmm0
-; SKX-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0
+; SKX-NEXT:    vrcp14ps %xmm0, %xmm1 # [1:0.00]
+; SKX-NEXT:    vbroadcastss {{.*}}(%rip), %xmm2 # [4:0.50]
+; SKX-NEXT:    vmovaps %xmm1, %xmm3 # [1:1.00]
+; SKX-NEXT:    vfnmadd213ps %xmm2, %xmm0, %xmm3 # [1:0.00]
+; SKX-NEXT:    vfmadd132ps %xmm1, %xmm1, %xmm3 # [1:0.00]
+; SKX-NEXT:    vfnmadd213ps %xmm2, %xmm3, %xmm0 # [1:0.00]
+; SKX-NEXT:    vfmadd132ps %xmm3, %xmm3, %xmm0 # [1:0.00]
+; SKX-NEXT:    vmulps {{.*}}(%rip), %xmm0, %xmm0 # [9:0.50]
 ; SKX-NEXT:    retq
   %div = fdiv fast <4 x float> <float 1.0, float 2.0, float 3.0, float 4.0>, %x
   ret <4 x float> %div
 }
 
+define <8 x float> @v8f32_no_estimate2(<8 x float> %x) #0 {
+; SSE-LABEL: v8f32_no_estimate2:
+; SSE:       # BB#0:
+; SSE-NEXT:    movaps {{.*}}(%rip), %xmm2 # [4:?]
+; SSE-NEXT:    # xmm2 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; SSE-NEXT:    divps %xmm0, %xmm2 # [10:?]
+; SSE-NEXT:    movaps {{.*}}(%rip), %xmm3 # [4:?]
+; SSE-NEXT:    # xmm3 = [5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
+; SSE-NEXT:    divps %xmm1, %xmm3 # [10:?]
+; SSE-NEXT:    movaps %xmm2, %xmm0 # [1:?]
+; SSE-NEXT:    movaps %xmm3, %xmm1 # [1:?]
+; SSE-NEXT:    retq
+;
+; AVX-RECIP-LABEL: v8f32_no_estimate2:
+; AVX-RECIP:       # BB#0:
+; AVX-RECIP-NEXT:    vmovaps {{.*}}(%rip), %ymm1 # [4:?]
+; AVX-RECIP-NEXT:    # ymm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00,5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
+; AVX-RECIP-NEXT:    vdivps %ymm0, %ymm1, %ymm0 # [10:?]
+; AVX-RECIP-NEXT:    retq
+;
+; FMA-RECIP-LABEL: v8f32_no_estimate2:
+; FMA-RECIP:       # BB#0:
+; FMA-RECIP-NEXT:    vmovaps {{.*}}(%rip), %ymm1 # [4:?]
+; FMA-RECIP-NEXT:    # ymm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00,5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
+; FMA-RECIP-NEXT:    vdivps %ymm0, %ymm1, %ymm0 # [10:?]
+; FMA-RECIP-NEXT:    retq
+;
+; BTVER2-LABEL: v8f32_no_estimate2:
+; BTVER2:       # BB#0:
+; BTVER2-NEXT:    vmovaps {{.*}}(%rip), %ymm1 # [5:1.00]
+; BTVER2-NEXT:    # ymm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00,5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
+; BTVER2-NEXT:    vdivps %ymm0, %ymm1, %ymm0 # [19:19.00]
+; BTVER2-NEXT:    retq
+;
+; SANDY-LABEL: v8f32_no_estimate2:
+; SANDY:       # BB#0:
+; SANDY-NEXT:    vmovaps {{.*}}(%rip), %ymm1 # [4:0.50]
+; SANDY-NEXT:    # ymm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00,5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
+; SANDY-NEXT:    vdivps %ymm0, %ymm1, %ymm0 # [12:1.00]
+; SANDY-NEXT:    retq
+;
+; HASWELL-LABEL: v8f32_no_estimate2:
+; HASWELL:       # BB#0:
+; HASWELL-NEXT:    vmovaps {{.*}}(%rip), %ymm1 # [4:0.50]
+; HASWELL-NEXT:    # ymm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00,5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
+; HASWELL-NEXT:    vdivps %ymm0, %ymm1, %ymm0 # [19:2.00]
+; HASWELL-NEXT:    retq
+;
+; HASWELL-NO-FMA-LABEL: v8f32_no_estimate2:
+; HASWELL-NO-FMA:       # BB#0:
+; HASWELL-NO-FMA-NEXT:    vmovaps {{.*}}(%rip), %ymm1 # [4:0.50]
+; HASWELL-NO-FMA-NEXT:    # ymm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00,5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
+; HASWELL-NO-FMA-NEXT:    vdivps %ymm0, %ymm1, %ymm0 # [19:2.00]
+; HASWELL-NO-FMA-NEXT:    retq
+;
+; AVX512-LABEL: v8f32_no_estimate2:
+; AVX512:       # BB#0:
+; AVX512-NEXT:    vmovaps {{.*}}(%rip), %ymm1 # [4:0.50]
+; AVX512-NEXT:    # ymm1 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00,5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
+; AVX512-NEXT:    vdivps %ymm0, %ymm1, %ymm0 # [19:2.00]
+; AVX512-NEXT:    retq
+  %div = fdiv fast <8 x float> <float 1.0, float 2.0, float 3.0, float 4.0, float 5.0, float 6.0, float 7.0, float 8.0>, %x
+  ret <8 x float> %div
+}
+
 define <8 x float> @v8f32_one_step2(<8 x float> %x) #1 {
 ; SSE-LABEL: v8f32_one_step2:
 ; SSE:       # BB#0:
-; SSE-NEXT:    rcpps %xmm1, %xmm4
-; SSE-NEXT:    mulps %xmm4, %xmm1
-; SSE-NEXT:    movaps {{.*#+}} xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; SSE-NEXT:    movaps %xmm2, %xmm3
-; SSE-NEXT:    subps %xmm1, %xmm3
-; SSE-NEXT:    mulps %xmm4, %xmm3
-; SSE-NEXT:    addps %xmm4, %xmm3
-; SSE-NEXT:    rcpps %xmm0, %xmm1
-; SSE-NEXT:    mulps %xmm1, %xmm0
-; SSE-NEXT:    subps %xmm0, %xmm2
-; SSE-NEXT:    mulps %xmm1, %xmm2
-; SSE-NEXT:    addps %xmm1, %xmm2
-; SSE-NEXT:    mulps {{.*}}(%rip), %xmm2
-; SSE-NEXT:    mulps {{.*}}(%rip), %xmm3
-; SSE-NEXT:    movaps %xmm2, %xmm0
-; SSE-NEXT:    movaps %xmm3, %xmm1
+; SSE-NEXT:    rcpps %xmm1, %xmm4 # [1:?]
+; SSE-NEXT:    mulps %xmm4, %xmm1 # [1:?]
+; SSE-NEXT:    movaps {{.*}}(%rip), %xmm2 # [4:?]
+; SSE-NEXT:    # xmm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; SSE-NEXT:    movaps %xmm2, %xmm3 # [1:?]
+; SSE-NEXT:    subps %xmm1, %xmm3 # [1:?]
+; SSE-NEXT:    mulps %xmm4, %xmm3 # [1:?]
+; SSE-NEXT:    addps %xmm4, %xmm3 # [1:?]
+; SSE-NEXT:    rcpps %xmm0, %xmm1 # [1:?]
+; SSE-NEXT:    mulps %xmm1, %xmm0 # [1:?]
+; SSE-NEXT:    subps %xmm0, %xmm2 # [1:?]
+; SSE-NEXT:    mulps %xmm1, %xmm2 # [1:?]
+; SSE-NEXT:    addps %xmm1, %xmm2 # [1:?]
+; SSE-NEXT:    mulps {{.*}}(%rip), %xmm2 # [4:?]
+; SSE-NEXT:    mulps {{.*}}(%rip), %xmm3 # [4:?]
+; SSE-NEXT:    movaps %xmm2, %xmm0 # [1:?]
+; SSE-NEXT:    movaps %xmm3, %xmm1 # [1:?]
 ; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: v8f32_one_step2:
 ; AVX-RECIP:       # BB#0:
-; AVX-RECIP-NEXT:    vrcpps %ymm0, %ymm1
-; AVX-RECIP-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; AVX-RECIP-NEXT:    vmovaps {{.*#+}} ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; AVX-RECIP-NEXT:    vsubps %ymm0, %ymm2, %ymm0
-; AVX-RECIP-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; AVX-RECIP-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; AVX-RECIP-NEXT:    vrcpps %ymm0, %ymm1 # [1:?]
+; AVX-RECIP-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # [1:?]
+; AVX-RECIP-NEXT:    vmovaps {{.*}}(%rip), %ymm2 # [4:?]
+; AVX-RECIP-NEXT:    # ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; AVX-RECIP-NEXT:    vsubps %ymm0, %ymm2, %ymm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [1:?]
+; AVX-RECIP-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [4:?]
 ; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: v8f32_one_step2:
 ; FMA-RECIP:       # BB#0:
-; FMA-RECIP-NEXT:    vrcpps %ymm0, %ymm1
-; FMA-RECIP-NEXT:    vfnmadd213ps {{.*}}(%rip), %ymm1, %ymm0
-; FMA-RECIP-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0
-; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; FMA-RECIP-NEXT:    vrcpps %ymm0, %ymm1 # [1:?]
+; FMA-RECIP-NEXT:    vfnmadd213ps {{.*}}(%rip), %ymm1, %ymm0 # [4:?]
+; FMA-RECIP-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0 # [1:?]
+; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [4:?]
 ; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: v8f32_one_step2:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vmovaps {{.*#+}} ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; BTVER2-NEXT:    vrcpps %ymm0, %ymm1
-; BTVER2-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; BTVER2-NEXT:    vsubps %ymm0, %ymm2, %ymm0
-; BTVER2-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; BTVER2-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; BTVER2-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; BTVER2-NEXT:    vmovaps {{.*}}(%rip), %ymm2 # [5:1.00]
+; BTVER2-NEXT:    # ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; BTVER2-NEXT:    vrcpps %ymm0, %ymm1 # [2:1.00]
+; BTVER2-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # [2:1.00]
+; BTVER2-NEXT:    vsubps %ymm0, %ymm2, %ymm0 # [3:1.00]
+; BTVER2-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [2:1.00]
+; BTVER2-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # [3:1.00]
+; BTVER2-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [7:1.00]
 ; BTVER2-NEXT:    retq
 ;
 ; SANDY-LABEL: v8f32_one_step2:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpps %ymm0, %ymm1
-; SANDY-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; SANDY-NEXT:    vmovaps {{.*#+}} ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; SANDY-NEXT:    vsubps %ymm0, %ymm2, %ymm0
-; SANDY-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; SANDY-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; SANDY-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; SANDY-NEXT:    vrcpps %ymm0, %ymm1 # [5:1.00]
+; SANDY-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # [5:1.00]
+; SANDY-NEXT:    vmovaps {{.*}}(%rip), %ymm2 # [4:0.50]
+; SANDY-NEXT:    # ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; SANDY-NEXT:    vsubps %ymm0, %ymm2, %ymm0 # [3:1.00]
+; SANDY-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [5:1.00]
+; SANDY-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # [3:1.00]
+; SANDY-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [9:1.00]
 ; SANDY-NEXT:    retq
 ;
 ; HASWELL-LABEL: v8f32_one_step2:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpps %ymm0, %ymm1
-; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2
-; HASWELL-NEXT:    vfnmadd213ps %ymm2, %ymm1, %ymm0
-; HASWELL-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0
-; HASWELL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; HASWELL-NEXT:    vrcpps %ymm0, %ymm1 # [7:2.00]
+; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2 # [5:1.00]
+; HASWELL-NEXT:    vfnmadd213ps %ymm2, %ymm1, %ymm0 # [1:0.00]
+; HASWELL-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0 # [1:0.00]
+; HASWELL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [9:1.00]
 ; HASWELL-NEXT:    retq
 ;
 ; HASWELL-NO-FMA-LABEL: v8f32_one_step2:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm1
-; HASWELL-NO-FMA-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2
-; HASWELL-NO-FMA-NEXT:    vsubps %ymm0, %ymm2, %ymm0
-; HASWELL-NO-FMA-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; HASWELL-NO-FMA-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm1 # [7:2.00]
+; HASWELL-NO-FMA-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # [5:1.00]
+; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2 # [5:1.00]
+; HASWELL-NO-FMA-NEXT:    vsubps %ymm0, %ymm2, %ymm0 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [5:1.00]
+; HASWELL-NO-FMA-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [9:1.00]
 ; HASWELL-NO-FMA-NEXT:    retq
 ;
 ; KNL-LABEL: v8f32_one_step2:
 ; KNL:       # BB#0:
-; KNL-NEXT:    vrcpps %ymm0, %ymm1
-; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2
-; KNL-NEXT:    vfnmadd213ps %ymm2, %ymm1, %ymm0
-; KNL-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0
-; KNL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; KNL-NEXT:    vrcpps %ymm0, %ymm1 # [7:2.00]
+; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2 # [5:1.00]
+; KNL-NEXT:    vfnmadd213ps %ymm2, %ymm1, %ymm0 # [1:0.00]
+; KNL-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0 # [1:0.00]
+; KNL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [9:1.00]
 ; KNL-NEXT:    retq
 ;
 ; SKX-LABEL: v8f32_one_step2:
 ; SKX:       # BB#0:
-; SKX-NEXT:    vrcp14ps %ymm0, %ymm1
-; SKX-NEXT:    vfnmadd213ps {{.*}}(%rip){1to8}, %ymm1, %ymm0
-; SKX-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0
-; SKX-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; SKX-NEXT:    vrcp14ps %ymm0, %ymm1 # [1:0.00]
+; SKX-NEXT:    vfnmadd213ps {{.*}}(%rip){1to8}, %ymm1, %ymm0 # [4:0.00]
+; SKX-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0 # [1:0.00]
+; SKX-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [9:1.00]
 ; SKX-NEXT:    retq
   %div = fdiv fast <8 x float> <float 1.0, float 2.0, float 3.0, float 4.0, float 5.0, float 6.0, float 7.0, float 8.0>, %x
   ret <8 x float> %div
 }
 
 define <8 x float> @v8f32_one_step_2_divs(<8 x float> %x) #1 {
 ; SSE-LABEL: v8f32_one_step_2_divs:
 ; SSE:       # BB#0:
-; SSE-NEXT:    rcpps %xmm0, %xmm2
-; SSE-NEXT:    mulps %xmm2, %xmm0
-; SSE-NEXT:    movaps {{.*#+}} xmm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; SSE-NEXT:    movaps %xmm3, %xmm4
-; SSE-NEXT:    subps %xmm0, %xmm4
-; SSE-NEXT:    mulps %xmm2, %xmm4
-; SSE-NEXT:    addps %xmm2, %xmm4
-; SSE-NEXT:    rcpps %xmm1, %xmm0
-; SSE-NEXT:    mulps %xmm0, %xmm1
-; SSE-NEXT:    subps %xmm1, %xmm3
-; SSE-NEXT:    mulps %xmm0, %xmm3
-; SSE-NEXT:    addps %xmm0, %xmm3
-; SSE-NEXT:    movaps {{.*#+}} xmm1 = [5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
-; SSE-NEXT:    mulps %xmm3, %xmm1
-; SSE-NEXT:    movaps {{.*#+}} xmm0 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
-; SSE-NEXT:    mulps %xmm4, %xmm0
-; SSE-NEXT:    mulps %xmm4, %xmm0
-; SSE-NEXT:    mulps %xmm3, %xmm1
+; SSE-NEXT:    rcpps %xmm0, %xmm2 # [1:?]
+; SSE-NEXT:    mulps %xmm2, %xmm0 # [1:?]
+; SSE-NEXT:    movaps {{.*}}(%rip), %xmm3 # [4:?]
+; SSE-NEXT:    # xmm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; SSE-NEXT:    movaps %xmm3, %xmm4 # [1:?]
+; SSE-NEXT:    subps %xmm0, %xmm4 # [1:?]
+; SSE-NEXT:    mulps %xmm2, %xmm4 # [1:?]
+; SSE-NEXT:    addps %xmm2, %xmm4 # [1:?]
+; SSE-NEXT:    rcpps %xmm1, %xmm0 # [1:?]
+; SSE-NEXT:    mulps %xmm0, %xmm1 # [1:?]
+; SSE-NEXT:    subps %xmm1, %xmm3 # [1:?]
+; SSE-NEXT:    mulps %xmm0, %xmm3 # [1:?]
+; SSE-NEXT:    addps %xmm0, %xmm3 # [1:?]
+; SSE-NEXT:    movaps {{.*}}(%rip), %xmm1 # [4:?]
+; SSE-NEXT:    # xmm1 = [5.000000e+00,6.000000e+00,7.000000e+00,8.000000e+00]
+; SSE-NEXT:    mulps %xmm3, %xmm1 # [1:?]
+; SSE-NEXT:    movaps {{.*}}(%rip), %xmm0 # [4:?]
+; SSE-NEXT:    # xmm0 = [1.000000e+00,2.000000e+00,3.000000e+00,4.000000e+00]
+; SSE-NEXT:    mulps %xmm4, %xmm0 # [1:?]
+; SSE-NEXT:    mulps %xmm4, %xmm0 # [1:?]
+; SSE-NEXT:    mulps %xmm3, %xmm1 # [1:?]
 ; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: v8f32_one_step_2_divs:
 ; AVX-RECIP:       # BB#0:
-; AVX-RECIP-NEXT:    vrcpps %ymm0, %ymm1
-; AVX-RECIP-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; AVX-RECIP-NEXT:    vmovaps {{.*#+}} ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; AVX-RECIP-NEXT:    vsubps %ymm0, %ymm2, %ymm0
-; AVX-RECIP-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; AVX-RECIP-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1
-; AVX-RECIP-NEXT:    vmulps %ymm0, %ymm1, %ymm0
+; AVX-RECIP-NEXT:    vrcpps %ymm0, %ymm1 # [1:?]
+; AVX-RECIP-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # [1:?]
+; AVX-RECIP-NEXT:    vmovaps {{.*}}(%rip), %ymm2 # [4:?]
+; AVX-RECIP-NEXT:    # ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; AVX-RECIP-NEXT:    vsubps %ymm0, %ymm2, %ymm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [1:?]
+; AVX-RECIP-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1 # [4:?]
+; AVX-RECIP-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [1:?]
 ; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: v8f32_one_step_2_divs:
 ; FMA-RECIP:       # BB#0:
-; FMA-RECIP-NEXT:    vrcpps %ymm0, %ymm1
-; FMA-RECIP-NEXT:    vfnmadd213ps {{.*}}(%rip), %ymm1, %ymm0
-; FMA-RECIP-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0
-; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1
-; FMA-RECIP-NEXT:    vmulps %ymm0, %ymm1, %ymm0
+; FMA-RECIP-NEXT:    vrcpps %ymm0, %ymm1 # [1:?]
+; FMA-RECIP-NEXT:    vfnmadd213ps {{.*}}(%rip), %ymm1, %ymm0 # [4:?]
+; FMA-RECIP-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0 # [1:?]
+; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1 # [4:?]
+; FMA-RECIP-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [1:?]
 ; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: v8f32_one_step_2_divs:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vmovaps {{.*#+}} ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; BTVER2-NEXT:    vrcpps %ymm0, %ymm1
-; BTVER2-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; BTVER2-NEXT:    vsubps %ymm0, %ymm2, %ymm0
-; BTVER2-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; BTVER2-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; BTVER2-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1
-; BTVER2-NEXT:    vmulps %ymm0, %ymm1, %ymm0
+; BTVER2-NEXT:    vmovaps {{.*}}(%rip), %ymm2 # [5:1.00]
+; BTVER2-NEXT:    # ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; BTVER2-NEXT:    vrcpps %ymm0, %ymm1 # [2:1.00]
+; BTVER2-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # [2:1.00]
+; BTVER2-NEXT:    vsubps %ymm0, %ymm2, %ymm0 # [3:1.00]
+; BTVER2-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [2:1.00]
+; BTVER2-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # [3:1.00]
+; BTVER2-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1 # [7:1.00]
+; BTVER2-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [2:1.00]
 ; BTVER2-NEXT:    retq
 ;
 ; SANDY-LABEL: v8f32_one_step_2_divs:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpps %ymm0, %ymm1
-; SANDY-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; SANDY-NEXT:    vmovaps {{.*#+}} ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; SANDY-NEXT:    vsubps %ymm0, %ymm2, %ymm0
-; SANDY-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; SANDY-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; SANDY-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1
-; SANDY-NEXT:    vmulps %ymm0, %ymm1, %ymm0
+; SANDY-NEXT:    vrcpps %ymm0, %ymm1 # [5:1.00]
+; SANDY-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # [5:1.00]
+; SANDY-NEXT:    vmovaps {{.*}}(%rip), %ymm2 # [4:0.50]
+; SANDY-NEXT:    # ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; SANDY-NEXT:    vsubps %ymm0, %ymm2, %ymm0 # [3:1.00]
+; SANDY-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [5:1.00]
+; SANDY-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # [3:1.00]
+; SANDY-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1 # [9:1.00]
+; SANDY-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [5:1.00]
 ; SANDY-NEXT:    retq
 ;
 ; HASWELL-LABEL: v8f32_one_step_2_divs:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpps %ymm0, %ymm1
-; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2
-; HASWELL-NEXT:    vfnmadd213ps %ymm2, %ymm1, %ymm0
-; HASWELL-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0
-; HASWELL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1
-; HASWELL-NEXT:    vmulps %ymm0, %ymm1, %ymm0
+; HASWELL-NEXT:    vrcpps %ymm0, %ymm1 # [7:2.00]
+; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2 # [5:1.00]
+; HASWELL-NEXT:    vfnmadd213ps %ymm2, %ymm1, %ymm0 # [1:0.00]
+; HASWELL-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0 # [1:0.00]
+; HASWELL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1 # [9:1.00]
+; HASWELL-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [5:1.00]
 ; HASWELL-NEXT:    retq
 ;
 ; HASWELL-NO-FMA-LABEL: v8f32_one_step_2_divs:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm1
-; HASWELL-NO-FMA-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2
-; HASWELL-NO-FMA-NEXT:    vsubps %ymm0, %ymm2, %ymm0
-; HASWELL-NO-FMA-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; HASWELL-NO-FMA-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1
-; HASWELL-NO-FMA-NEXT:    vmulps %ymm0, %ymm1, %ymm0
+; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm1 # [7:2.00]
+; HASWELL-NO-FMA-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # [5:1.00]
+; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2 # [5:1.00]
+; HASWELL-NO-FMA-NEXT:    vsubps %ymm0, %ymm2, %ymm0 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [5:1.00]
+; HASWELL-NO-FMA-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1 # [9:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [5:1.00]
 ; HASWELL-NO-FMA-NEXT:    retq
 ;
 ; KNL-LABEL: v8f32_one_step_2_divs:
 ; KNL:       # BB#0:
-; KNL-NEXT:    vrcpps %ymm0, %ymm1
-; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2
-; KNL-NEXT:    vfnmadd213ps %ymm2, %ymm1, %ymm0
-; KNL-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0
-; KNL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1
-; KNL-NEXT:    vmulps %ymm0, %ymm1, %ymm0
+; KNL-NEXT:    vrcpps %ymm0, %ymm1 # [7:2.00]
+; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2 # [5:1.00]
+; KNL-NEXT:    vfnmadd213ps %ymm2, %ymm1, %ymm0 # [1:0.00]
+; KNL-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0 # [1:0.00]
+; KNL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1 # [9:1.00]
+; KNL-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [5:1.00]
 ; KNL-NEXT:    retq
 ;
 ; SKX-LABEL: v8f32_one_step_2_divs:
 ; SKX:       # BB#0:
-; SKX-NEXT:    vrcp14ps %ymm0, %ymm1
-; SKX-NEXT:    vfnmadd213ps {{.*}}(%rip){1to8}, %ymm1, %ymm0
-; SKX-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0
-; SKX-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1
-; SKX-NEXT:    vmulps %ymm0, %ymm1, %ymm0
+; SKX-NEXT:    vrcp14ps %ymm0, %ymm1 # [1:0.00]
+; SKX-NEXT:    vfnmadd213ps {{.*}}(%rip){1to8}, %ymm1, %ymm0 # [4:0.00]
+; SKX-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm0 # [1:0.00]
+; SKX-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm1 # [9:1.00]
+; SKX-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [5:1.00]
 ; SKX-NEXT:    retq
   %div = fdiv fast <8 x float> <float 1.0, float 2.0, float 3.0, float 4.0, float 5.0, float 6.0, float 7.0, float 8.0>, %x
   %div2 = fdiv fast <8 x float> %div, %x
   ret <8 x float> %div2
 }
 
 define <8 x float> @v8f32_two_step2(<8 x float> %x) #2 {
 ; SSE-LABEL: v8f32_two_step2:
 ; SSE:       # BB#0:
-; SSE-NEXT:    movaps %xmm0, %xmm2
-; SSE-NEXT:    rcpps %xmm1, %xmm3
-; SSE-NEXT:    movaps %xmm1, %xmm4
-; SSE-NEXT:    mulps %xmm3, %xmm4
-; SSE-NEXT:    movaps {{.*#+}} xmm0 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; SSE-NEXT:    movaps %xmm0, %xmm5
-; SSE-NEXT:    subps %xmm4, %xmm5
-; SSE-NEXT:    mulps %xmm3, %xmm5
-; SSE-NEXT:    addps %xmm3, %xmm5
-; SSE-NEXT:    mulps %xmm5, %xmm1
-; SSE-NEXT:    movaps %xmm0, %xmm3
-; SSE-NEXT:    subps %xmm1, %xmm3
-; SSE-NEXT:    mulps %xmm5, %xmm3
-; SSE-NEXT:    addps %xmm5, %xmm3
-; SSE-NEXT:    rcpps %xmm2, %xmm1
-; SSE-NEXT:    movaps %xmm2, %xmm4
-; SSE-NEXT:    mulps %xmm1, %xmm4
-; SSE-NEXT:    movaps %xmm0, %xmm5
-; SSE-NEXT:    subps %xmm4, %xmm5
-; SSE-NEXT:    mulps %xmm1, %xmm5
-; SSE-NEXT:    addps %xmm1, %xmm5
-; SSE-NEXT:    mulps %xmm5, %xmm2
-; SSE-NEXT:    subps %xmm2, %xmm0
-; SSE-NEXT:    mulps %xmm5, %xmm0
-; SSE-NEXT:    addps %xmm5, %xmm0
-; SSE-NEXT:    mulps {{.*}}(%rip), %xmm0
-; SSE-NEXT:    mulps {{.*}}(%rip), %xmm3
-; SSE-NEXT:    movaps %xmm3, %xmm1
+; SSE-NEXT:    movaps %xmm0, %xmm2 # [1:?]
+; SSE-NEXT:    rcpps %xmm1, %xmm3 # [1:?]
+; SSE-NEXT:    movaps %xmm1, %xmm4 # [1:?]
+; SSE-NEXT:    mulps %xmm3, %xmm4 # [1:?]
+; SSE-NEXT:    movaps {{.*}}(%rip), %xmm0 # [4:?]
+; SSE-NEXT:    # xmm0 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; SSE-NEXT:    movaps %xmm0, %xmm5 # [1:?]
+; SSE-NEXT:    subps %xmm4, %xmm5 # [1:?]
+; SSE-NEXT:    mulps %xmm3, %xmm5 # [1:?]
+; SSE-NEXT:    addps %xmm3, %xmm5 # [1:?]
+; SSE-NEXT:    mulps %xmm5, %xmm1 # [1:?]
+; SSE-NEXT:    movaps %xmm0, %xmm3 # [1:?]
+; SSE-NEXT:    subps %xmm1, %xmm3 # [1:?]
+; SSE-NEXT:    mulps %xmm5, %xmm3 # [1:?]
+; SSE-NEXT:    addps %xmm5, %xmm3 # [1:?]
+; SSE-NEXT:    rcpps %xmm2, %xmm1 # [1:?]
+; SSE-NEXT:    movaps %xmm2, %xmm4 # [1:?]
+; SSE-NEXT:    mulps %xmm1, %xmm4 # [1:?]
+; SSE-NEXT:    movaps %xmm0, %xmm5 # [1:?]
+; SSE-NEXT:    subps %xmm4, %xmm5 # [1:?]
+; SSE-NEXT:    mulps %xmm1, %xmm5 # [1:?]
+; SSE-NEXT:    addps %xmm1, %xmm5 # [1:?]
+; SSE-NEXT:    mulps %xmm5, %xmm2 # [1:?]
+; SSE-NEXT:    subps %xmm2, %xmm0 # [1:?]
+; SSE-NEXT:    mulps %xmm5, %xmm0 # [1:?]
+; SSE-NEXT:    addps %xmm5, %xmm0 # [1:?]
+; SSE-NEXT:    mulps {{.*}}(%rip), %xmm0 # [4:?]
+; SSE-NEXT:    mulps {{.*}}(%rip), %xmm3 # [4:?]
+; SSE-NEXT:    movaps %xmm3, %xmm1 # [1:?]
 ; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: v8f32_two_step2:
 ; AVX-RECIP:       # BB#0:
-; AVX-RECIP-NEXT:    vrcpps %ymm0, %ymm1
-; AVX-RECIP-NEXT:    vmulps %ymm1, %ymm0, %ymm2
-; AVX-RECIP-NEXT:    vmovaps {{.*#+}} ymm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; AVX-RECIP-NEXT:    vsubps %ymm2, %ymm3, %ymm2
-; AVX-RECIP-NEXT:    vmulps %ymm2, %ymm1, %ymm2
-; AVX-RECIP-NEXT:    vaddps %ymm2, %ymm1, %ymm1
-; AVX-RECIP-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; AVX-RECIP-NEXT:    vsubps %ymm0, %ymm3, %ymm0
-; AVX-RECIP-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; AVX-RECIP-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; AVX-RECIP-NEXT:    vrcpps %ymm0, %ymm1 # [1:?]
+; AVX-RECIP-NEXT:    vmulps %ymm1, %ymm0, %ymm2 # [1:?]
+; AVX-RECIP-NEXT:    vmovaps {{.*}}(%rip), %ymm3 # [4:?]
+; AVX-RECIP-NEXT:    # ymm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; AVX-RECIP-NEXT:    vsubps %ymm2, %ymm3, %ymm2 # [1:?]
+; AVX-RECIP-NEXT:    vmulps %ymm2, %ymm1, %ymm2 # [1:?]
+; AVX-RECIP-NEXT:    vaddps %ymm2, %ymm1, %ymm1 # [1:?]
+; AVX-RECIP-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # [1:?]
+; AVX-RECIP-NEXT:    vsubps %ymm0, %ymm3, %ymm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [1:?]
+; AVX-RECIP-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [4:?]
 ; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: v8f32_two_step2:
 ; FMA-RECIP:       # BB#0:
-; FMA-RECIP-NEXT:    vrcpps %ymm0, %ymm1
-; FMA-RECIP-NEXT:    vmovaps {{.*#+}} ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; FMA-RECIP-NEXT:    vmovaps %ymm1, %ymm3
-; FMA-RECIP-NEXT:    vfnmadd213ps %ymm2, %ymm0, %ymm3
-; FMA-RECIP-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm3
-; FMA-RECIP-NEXT:    vfnmadd213ps %ymm2, %ymm3, %ymm0
-; FMA-RECIP-NEXT:    vfmadd132ps %ymm3, %ymm3, %ymm0
-; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; FMA-RECIP-NEXT:    vrcpps %ymm0, %ymm1 # [1:?]
+; FMA-RECIP-NEXT:    vmovaps {{.*}}(%rip), %ymm2 # [4:?]
+; FMA-RECIP-NEXT:    # ymm2 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; FMA-RECIP-NEXT:    vmovaps %ymm1, %ymm3 # [1:?]
+; FMA-RECIP-NEXT:    vfnmadd213ps %ymm2, %ymm0, %ymm3 # [1:?]
+; FMA-RECIP-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm3 # [1:?]
+; FMA-RECIP-NEXT:    vfnmadd213ps %ymm2, %ymm3, %ymm0 # [1:?]
+; FMA-RECIP-NEXT:    vfmadd132ps %ymm3, %ymm3, %ymm0 # [1:?]
+; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [4:?]
 ; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: v8f32_two_step2:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vmovaps {{.*#+}} ymm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; BTVER2-NEXT:    vrcpps %ymm0, %ymm1
-; BTVER2-NEXT:    vmulps %ymm1, %ymm0, %ymm2
-; BTVER2-NEXT:    vsubps %ymm2, %ymm3, %ymm2
-; BTVER2-NEXT:    vmulps %ymm2, %ymm1, %ymm2
-; BTVER2-NEXT:    vaddps %ymm2, %ymm1, %ymm1
-; BTVER2-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; BTVER2-NEXT:    vsubps %ymm0, %ymm3, %ymm0
-; BTVER2-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; BTVER2-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; BTVER2-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; BTVER2-NEXT:    vmovaps {{.*}}(%rip), %ymm3 # [5:1.00]
+; BTVER2-NEXT:    # ymm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; BTVER2-NEXT:    vrcpps %ymm0, %ymm1 # [2:1.00]
+; BTVER2-NEXT:    vmulps %ymm1, %ymm0, %ymm2 # [2:1.00]
+; BTVER2-NEXT:    vsubps %ymm2, %ymm3, %ymm2 # [3:1.00]
+; BTVER2-NEXT:    vmulps %ymm2, %ymm1, %ymm2 # [2:1.00]
+; BTVER2-NEXT:    vaddps %ymm2, %ymm1, %ymm1 # [3:1.00]
+; BTVER2-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # [2:1.00]
+; BTVER2-NEXT:    vsubps %ymm0, %ymm3, %ymm0 # [3:1.00]
+; BTVER2-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [2:1.00]
+; BTVER2-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # [3:1.00]
+; BTVER2-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [7:1.00]
 ; BTVER2-NEXT:    retq
 ;
 ; SANDY-LABEL: v8f32_two_step2:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpps %ymm0, %ymm1
-; SANDY-NEXT:    vmulps %ymm1, %ymm0, %ymm2
-; SANDY-NEXT:    vmovaps {{.*#+}} ymm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
-; SANDY-NEXT:    vsubps %ymm2, %ymm3, %ymm2
-; SANDY-NEXT:    vmulps %ymm2, %ymm1, %ymm2
-; SANDY-NEXT:    vaddps %ymm2, %ymm1, %ymm1
-; SANDY-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; SANDY-NEXT:    vsubps %ymm0, %ymm3, %ymm0
-; SANDY-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; SANDY-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; SANDY-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; SANDY-NEXT:    vrcpps %ymm0, %ymm1 # [5:1.00]
+; SANDY-NEXT:    vmulps %ymm1, %ymm0, %ymm2 # [5:1.00]
+; SANDY-NEXT:    vmovaps {{.*}}(%rip), %ymm3 # [4:0.50]
+; SANDY-NEXT:    # ymm3 = [1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00,1.000000e+00]
+; SANDY-NEXT:    vsubps %ymm2, %ymm3, %ymm2 # [3:1.00]
+; SANDY-NEXT:    vmulps %ymm2, %ymm1, %ymm2 # [5:1.00]
+; SANDY-NEXT:    vaddps %ymm2, %ymm1, %ymm1 # [3:1.00]
+; SANDY-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # [5:1.00]
+; SANDY-NEXT:    vsubps %ymm0, %ymm3, %ymm0 # [3:1.00]
+; SANDY-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [5:1.00]
+; SANDY-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # [3:1.00]
+; SANDY-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [9:1.00]
 ; SANDY-NEXT:    retq
 ;
 ; HASWELL-LABEL: v8f32_two_step2:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpps %ymm0, %ymm1
-; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2
-; HASWELL-NEXT:    vmovaps %ymm1, %ymm3
-; HASWELL-NEXT:    vfnmadd213ps %ymm2, %ymm0, %ymm3
-; HASWELL-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm3
-; HASWELL-NEXT:    vfnmadd213ps %ymm2, %ymm3, %ymm0
-; HASWELL-NEXT:    vfmadd132ps %ymm3, %ymm3, %ymm0
-; HASWELL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; HASWELL-NEXT:    vrcpps %ymm0, %ymm1 # [7:2.00]
+; HASWELL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2 # [5:1.00]
+; HASWELL-NEXT:    vmovaps %ymm1, %ymm3 # [1:1.00]
+; HASWELL-NEXT:    vfnmadd213ps %ymm2, %ymm0, %ymm3 # [1:0.00]
+; HASWELL-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm3 # [1:0.00]
+; HASWELL-NEXT:    vfnmadd213ps %ymm2, %ymm3, %ymm0 # [1:0.00]
+; HASWELL-NEXT:    vfmadd132ps %ymm3, %ymm3, %ymm0 # [1:0.00]
+; HASWELL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [9:1.00]
 ; HASWELL-NEXT:    retq
 ;
 ; HASWELL-NO-FMA-LABEL: v8f32_two_step2:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm1
-; HASWELL-NO-FMA-NEXT:    vmulps %ymm1, %ymm0, %ymm2
-; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %ymm3
-; HASWELL-NO-FMA-NEXT:    vsubps %ymm2, %ymm3, %ymm2
-; HASWELL-NO-FMA-NEXT:    vmulps %ymm2, %ymm1, %ymm2
-; HASWELL-NO-FMA-NEXT:    vaddps %ymm2, %ymm1, %ymm1
-; HASWELL-NO-FMA-NEXT:    vmulps %ymm1, %ymm0, %ymm0
-; HASWELL-NO-FMA-NEXT:    vsubps %ymm0, %ymm3, %ymm0
-; HASWELL-NO-FMA-NEXT:    vmulps %ymm0, %ymm1, %ymm0
-; HASWELL-NO-FMA-NEXT:    vaddps %ymm0, %ymm1, %ymm0
-; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm1 # [7:2.00]
+; HASWELL-NO-FMA-NEXT:    vmulps %ymm1, %ymm0, %ymm2 # [5:1.00]
+; HASWELL-NO-FMA-NEXT:    vbroadcastss {{.*}}(%rip), %ymm3 # [5:1.00]
+; HASWELL-NO-FMA-NEXT:    vsubps %ymm2, %ymm3, %ymm2 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps %ymm2, %ymm1, %ymm2 # [5:1.00]
+; HASWELL-NO-FMA-NEXT:    vaddps %ymm2, %ymm1, %ymm1 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps %ymm1, %ymm0, %ymm0 # [5:1.00]
+; HASWELL-NO-FMA-NEXT:    vsubps %ymm0, %ymm3, %ymm0 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps %ymm0, %ymm1, %ymm0 # [5:1.00]
+; HASWELL-NO-FMA-NEXT:    vaddps %ymm0, %ymm1, %ymm0 # [3:1.00]
+; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [9:1.00]
 ; HASWELL-NO-FMA-NEXT:    retq
 ;
 ; KNL-LABEL: v8f32_two_step2:
 ; KNL:       # BB#0:
-; KNL-NEXT:    vrcpps %ymm0, %ymm1
-; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2
-; KNL-NEXT:    vmovaps %ymm1, %ymm3
-; KNL-NEXT:    vfnmadd213ps %ymm2, %ymm0, %ymm3
-; KNL-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm3
-; KNL-NEXT:    vfnmadd213ps %ymm2, %ymm3, %ymm0
-; KNL-NEXT:    vfmadd132ps %ymm3, %ymm3, %ymm0
-; KNL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; KNL-NEXT:    vrcpps %ymm0, %ymm1 # [7:2.00]
+; KNL-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2 # [5:1.00]
+; KNL-NEXT:    vmovaps %ymm1, %ymm3 # [1:1.00]
+; KNL-NEXT:    vfnmadd213ps %ymm2, %ymm0, %ymm3 # [1:0.00]
+; KNL-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm3 # [1:0.00]
+; KNL-NEXT:    vfnmadd213ps %ymm2, %ymm3, %ymm0 # [1:0.00]
+; KNL-NEXT:    vfmadd132ps %ymm3, %ymm3, %ymm0 # [1:0.00]
+; KNL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [9:1.00]
 ; KNL-NEXT:    retq
 ;
 ; SKX-LABEL: v8f32_two_step2:
 ; SKX:       # BB#0:
-; SKX-NEXT:    vrcp14ps %ymm0, %ymm1
-; SKX-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2
-; SKX-NEXT:    vmovaps %ymm1, %ymm3
-; SKX-NEXT:    vfnmadd213ps %ymm2, %ymm0, %ymm3
-; SKX-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm3
-; SKX-NEXT:    vfnmadd213ps %ymm2, %ymm3, %ymm0
-; SKX-NEXT:    vfmadd132ps %ymm3, %ymm3, %ymm0
-; SKX-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; SKX-NEXT:    vrcp14ps %ymm0, %ymm1 # [1:0.00]
+; SKX-NEXT:    vbroadcastss {{.*}}(%rip), %ymm2 # [5:1.00]
+; SKX-NEXT:    vmovaps %ymm1, %ymm3 # [1:1.00]
+; SKX-NEXT:    vfnmadd213ps %ymm2, %ymm0, %ymm3 # [1:0.00]
+; SKX-NEXT:    vfmadd132ps %ymm1, %ymm1, %ymm3 # [1:0.00]
+; SKX-NEXT:    vfnmadd213ps %ymm2, %ymm3, %ymm0 # [1:0.00]
+; SKX-NEXT:    vfmadd132ps %ymm3, %ymm3, %ymm0 # [1:0.00]
+; SKX-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [9:1.00]
 ; SKX-NEXT:    retq
   %div = fdiv fast <8 x float> <float 1.0, float 2.0, float 3.0, float 4.0, float 5.0, float 6.0, float 7.0, float 8.0>, %x
   ret <8 x float> %div
 }
 
 define <8 x float> @v8f32_no_step(<8 x float> %x) #3 {
 ; SSE-LABEL: v8f32_no_step:
 ; SSE:       # BB#0:
-; SSE-NEXT:    rcpps %xmm0, %xmm0
-; SSE-NEXT:    rcpps %xmm1, %xmm1
+; SSE-NEXT:    rcpps %xmm0, %xmm0 # [1:?]
+; SSE-NEXT:    rcpps %xmm1, %xmm1 # [1:?]
 ; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: v8f32_no_step:
 ; AVX-RECIP:       # BB#0:
-; AVX-RECIP-NEXT:    vrcpps %ymm0, %ymm0
+; AVX-RECIP-NEXT:    vrcpps %ymm0, %ymm0 # [1:?]
 ; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: v8f32_no_step:
 ; FMA-RECIP:       # BB#0:
-; FMA-RECIP-NEXT:    vrcpps %ymm0, %ymm0
+; FMA-RECIP-NEXT:    vrcpps %ymm0, %ymm0 # [1:?]
 ; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: v8f32_no_step:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vrcpps %ymm0, %ymm0
+; BTVER2-NEXT:    vrcpps %ymm0, %ymm0 # [2:1.00]
 ; BTVER2-NEXT:    retq
 ;
 ; SANDY-LABEL: v8f32_no_step:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpps %ymm0, %ymm0
+; SANDY-NEXT:    vrcpps %ymm0, %ymm0 # [5:1.00]
 ; SANDY-NEXT:    retq
 ;
 ; HASWELL-LABEL: v8f32_no_step:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpps %ymm0, %ymm0
+; HASWELL-NEXT:    vrcpps %ymm0, %ymm0 # [7:2.00]
 ; HASWELL-NEXT:    retq
 ;
 ; HASWELL-NO-FMA-LABEL: v8f32_no_step:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm0
+; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm0 # [7:2.00]
 ; HASWELL-NO-FMA-NEXT:    retq
 ;
 ; KNL-LABEL: v8f32_no_step:
 ; KNL:       # BB#0:
-; KNL-NEXT:    vrcpps %ymm0, %ymm0
+; KNL-NEXT:    vrcpps %ymm0, %ymm0 # [7:2.00]
 ; KNL-NEXT:    retq
 ;
 ; SKX-LABEL: v8f32_no_step:
 ; SKX:       # BB#0:
-; SKX-NEXT:    vrcp14ps %ymm0, %ymm0
+; SKX-NEXT:    vrcp14ps %ymm0, %ymm0 # [1:0.00]
 ; SKX-NEXT:    retq
   %div = fdiv fast <8 x float> <float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0, float 1.0>, %x
   ret <8 x float> %div
 }
 
 define <8 x float> @v8f32_no_step2(<8 x float> %x) #3 {
 ; SSE-LABEL: v8f32_no_step2:
 ; SSE:       # BB#0:
-; SSE-NEXT:    rcpps %xmm1, %xmm1
-; SSE-NEXT:    rcpps %xmm0, %xmm0
-; SSE-NEXT:    mulps {{.*}}(%rip), %xmm0
-; SSE-NEXT:    mulps {{.*}}(%rip), %xmm1
+; SSE-NEXT:    rcpps %xmm1, %xmm1 # [1:?]
+; SSE-NEXT:    rcpps %xmm0, %xmm0 # [1:?]
+; SSE-NEXT:    mulps {{.*}}(%rip), %xmm0 # [4:?]
+; SSE-NEXT:    mulps {{.*}}(%rip), %xmm1 # [4:?]
 ; SSE-NEXT:    retq
 ;
 ; AVX-RECIP-LABEL: v8f32_no_step2:
 ; AVX-RECIP:       # BB#0:
-; AVX-RECIP-NEXT:    vrcpps %ymm0, %ymm0
-; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; AVX-RECIP-NEXT:    vrcpps %ymm0, %ymm0 # [1:?]
+; AVX-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [4:?]
 ; AVX-RECIP-NEXT:    retq
 ;
 ; FMA-RECIP-LABEL: v8f32_no_step2:
 ; FMA-RECIP:       # BB#0:
-; FMA-RECIP-NEXT:    vrcpps %ymm0, %ymm0
-; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; FMA-RECIP-NEXT:    vrcpps %ymm0, %ymm0 # [1:?]
+; FMA-RECIP-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [4:?]
 ; FMA-RECIP-NEXT:    retq
 ;
 ; BTVER2-LABEL: v8f32_no_step2:
 ; BTVER2:       # BB#0:
-; BTVER2-NEXT:    vrcpps %ymm0, %ymm0
-; BTVER2-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; BTVER2-NEXT:    vrcpps %ymm0, %ymm0 # [2:1.00]
+; BTVER2-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [7:1.00]
 ; BTVER2-NEXT:    retq
 ;
 ; SANDY-LABEL: v8f32_no_step2:
 ; SANDY:       # BB#0:
-; SANDY-NEXT:    vrcpps %ymm0, %ymm0
-; SANDY-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; SANDY-NEXT:    vrcpps %ymm0, %ymm0 # [5:1.00]
+; SANDY-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [9:1.00]
 ; SANDY-NEXT:    retq
 ;
 ; HASWELL-LABEL: v8f32_no_step2:
 ; HASWELL:       # BB#0:
-; HASWELL-NEXT:    vrcpps %ymm0, %ymm0
-; HASWELL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; HASWELL-NEXT:    vrcpps %ymm0, %ymm0 # [7:2.00]
+; HASWELL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [9:1.00]
 ; HASWELL-NEXT:    retq
 ;
 ; HASWELL-NO-FMA-LABEL: v8f32_no_step2:
 ; HASWELL-NO-FMA:       # BB#0:
-; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm0
-; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; HASWELL-NO-FMA-NEXT:    vrcpps %ymm0, %ymm0 # [7:2.00]
+; HASWELL-NO-FMA-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [9:1.00]
 ; HASWELL-NO-FMA-NEXT:    retq
 ;
 ; KNL-LABEL: v8f32_no_step2:
 ; KNL:       # BB#0:
-; KNL-NEXT:    vrcpps %ymm0, %ymm0
-; KNL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; KNL-NEXT:    vrcpps %ymm0, %ymm0 # [7:2.00]
+; KNL-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [9:1.00]
 ; KNL-NEXT:    retq
 ;
 ; SKX-LABEL: v8f32_no_step2:
 ; SKX:       # BB#0:
-; SKX-NEXT:    vrcp14ps %ymm0, %ymm0
-; SKX-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0
+; SKX-NEXT:    vrcp14ps %ymm0, %ymm0 # [1:0.00]
+; SKX-NEXT:    vmulps {{.*}}(%rip), %ymm0, %ymm0 # [9:1.00]
 ; SKX-NEXT:    retq
   %div = fdiv fast <8 x float> <float 1.0, float 2.0, float 3.0, float 4.0, float 5.0, float 6.0, float 7.0, float 8.0>, %x
   ret <8 x float> %div
 }
 
 attributes #0 = { "unsafe-fp-math"="true" "reciprocal-estimates"="!divf,!vec-divf" }
 attributes #1 = { "unsafe-fp-math"="true" "reciprocal-estimates"="divf,vec-divf" }
 attributes #2 = { "unsafe-fp-math"="true" "reciprocal-estimates"="divf:2,vec-divf:2" }
 attributes #3 = { "unsafe-fp-math"="true" "reciprocal-estimates"="divf:0,vec-divf:0" }
 

test/CodeGen/X86/recip-pic.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc < %s -mtriple=i386-unknown-linux-gnu  -enable-unsafe-fp-math -mcpu=slm -relocation-model=pic | FileCheck %s --check-prefix=CHECK
+; RUN: llc < %s -mtriple=i386-unknown-linux-gnu -mcpu=slm -relocation-model=pic -print-schedule | FileCheck %s --check-prefix=CHECK
 
-define fastcc float @foo(float %x) unnamed_addr #0 {
-; CHECK-LABEL: foo:
-; CHECK:       # BB#0: # %entry
-; CHECK-NEXT:    calll .L0$pb
+define float @f32_no_estimate_2(float %x) #0 {
+; CHECK-LABEL: f32_no_estimate_2:
+; CHECK:       # BB#0:
+; CHECK-NEXT:    pushl %eax # [1:1.00]
 ; CHECK-NEXT:  .Lcfi0:
+; CHECK-NEXT:    .cfi_def_cfa_offset 8
+; CHECK-NEXT:    calll .L0$pb # [1:0.00]
+; CHECK-NEXT:  .Lcfi1:
 ; CHECK-NEXT:    .cfi_adjust_cfa_offset 4
 ; CHECK-NEXT:  .L0$pb:
 ; CHECK-NEXT:    popl %eax
-; CHECK-NEXT:  .Lcfi1:
+; CHECK-NEXT:  .Lcfi2:
 ; CHECK-NEXT:    .cfi_adjust_cfa_offset -4
-; CHECK-NEXT:  .Ltmp0:
+; CHECK-NEXT:  .Ltmp0: # [1:0.50]
 ; CHECK-NEXT:    addl $_GLOBAL_OFFSET_TABLE_+(.Ltmp0-.L0$pb), %eax
-; CHECK-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
-; CHECK-NEXT:    divss %xmm0, %xmm1
-; CHECK-NEXT:    movaps %xmm1, %xmm0
-; CHECK-NEXT:    movss %xmm1, (%eax)
+; CHECK-NEXT:    movss {{\.LCPI.*}}@GOTOFF(%eax), %xmm0 # [3:1.00]
+; CHECK-NEXT:    # xmm0 = mem[0],zero,zero,zero
+; CHECK-NEXT:    divss {{[0-9]+}}(%esp), %xmm0 # [37:34.00]
+; CHECK-NEXT:    movss %xmm0, (%eax) # [1:1.00]
+; CHECK-NEXT:    movss %xmm0, (%esp) # [1:1.00]
+; CHECK-NEXT:    flds (%esp) # [3:1.00]
+; CHECK-NEXT:    popl %eax # [3:1.00]
 ; CHECK-NEXT:    retl
-entry:
   %div = fdiv fast float 3.0, %x
   store float %div, float* undef, align 4
   ret float %div
 }
 
+define float @f32_one_step(float %x) #1 {
+; CHECK-LABEL: f32_one_step:
+; CHECK:       # BB#0:
+; CHECK-NEXT:    pushl %eax # [1:1.00]
+; CHECK-NEXT:  .Lcfi3:
+; CHECK-NEXT:    .cfi_def_cfa_offset 8
+; CHECK-NEXT:    movss {{[0-9]+}}(%esp), %xmm0 # [3:1.00]
+; CHECK-NEXT:    # xmm0 = mem[0],zero,zero,zero
+; CHECK-NEXT:    calll .L1$pb # [1:0.00]
+; CHECK-NEXT:  .Lcfi4:
+; CHECK-NEXT:    .cfi_adjust_cfa_offset 4
+; CHECK-NEXT:  .L1$pb:
+; CHECK-NEXT:    popl %eax
+; CHECK-NEXT:  .Lcfi5:
+; CHECK-NEXT:    .cfi_adjust_cfa_offset -4
+; CHECK-NEXT:  .Ltmp1: # [1:0.50]
+; CHECK-NEXT:    addl $_GLOBAL_OFFSET_TABLE_+(.Ltmp1-.L1$pb), %eax
+; CHECK-NEXT:    movss {{\.LCPI.*}}@GOTOFF(%eax), %xmm2 # [3:1.00]
+; CHECK-NEXT:    # xmm2 = mem[0],zero,zero,zero
+; CHECK-NEXT:    rcpss %xmm0, %xmm1 # [5:1.00]
+; CHECK-NEXT:    mulss %xmm1, %xmm0 # [5:2.00]
+; CHECK-NEXT:    subss %xmm0, %xmm2 # [3:1.00]
+; CHECK-NEXT:    mulss %xmm1, %xmm2 # [5:2.00]
+; CHECK-NEXT:    addss %xmm1, %xmm2 # [3:1.00]
+; CHECK-NEXT:    mulss {{\.LCPI.*}}@GOTOFF(%eax), %xmm2 # [8:2.00]
+; CHECK-NEXT:    movss %xmm2, (%eax) # [1:1.00]
+; CHECK-NEXT:    movss %xmm2, (%esp) # [1:1.00]
+; CHECK-NEXT:    flds (%esp) # [3:1.00]
+; CHECK-NEXT:    popl %eax # [3:1.00]
+; CHECK-NEXT:    retl
+  %div = fdiv fast float 3.0, %x
+  store float %div, float* undef, align 4
+  ret float %div
+}
+
+define float @f32_two_steps(float %x) #2 {
+; CHECK-LABEL: f32_two_steps:
+; CHECK:       # BB#0:
+; CHECK-NEXT:    pushl %eax # [1:1.00]
+; CHECK-NEXT:  .Lcfi6:
+; CHECK-NEXT:    .cfi_def_cfa_offset 8
+; CHECK-NEXT:    movss {{[0-9]+}}(%esp), %xmm0 # [3:1.00]
+; CHECK-NEXT:    # xmm0 = mem[0],zero,zero,zero
+; CHECK-NEXT:    calll .L2$pb # [1:0.00]
+; CHECK-NEXT:  .Lcfi7:
+; CHECK-NEXT:    .cfi_adjust_cfa_offset 4
+; CHECK-NEXT:  .L2$pb:
+; CHECK-NEXT:    popl %eax
+; CHECK-NEXT:  .Lcfi8:
+; CHECK-NEXT:    .cfi_adjust_cfa_offset -4
+; CHECK-NEXT:  .Ltmp2: # [1:0.50]
+; CHECK-NEXT:    addl $_GLOBAL_OFFSET_TABLE_+(.Ltmp2-.L2$pb), %eax
+; CHECK-NEXT:    movss {{\.LCPI.*}}@GOTOFF(%eax), %xmm3 # [3:1.00]
+; CHECK-NEXT:    # xmm3 = mem[0],zero,zero,zero
+; CHECK-NEXT:    rcpss %xmm0, %xmm1 # [5:1.00]
+; CHECK-NEXT:    movaps %xmm0, %xmm2 # [1:1.00]
+; CHECK-NEXT:    movaps %xmm3, %xmm4 # [1:1.00]
+; CHECK-NEXT:    mulss %xmm1, %xmm2 # [5:2.00]
+; CHECK-NEXT:    subss %xmm2, %xmm4 # [3:1.00]
+; CHECK-NEXT:    mulss %xmm1, %xmm4 # [5:2.00]
+; CHECK-NEXT:    addss %xmm1, %xmm4 # [3:1.00]
+; CHECK-NEXT:    mulss %xmm4, %xmm0 # [5:2.00]
+; CHECK-NEXT:    subss %xmm0, %xmm3 # [3:1.00]
+; CHECK-NEXT:    mulss %xmm4, %xmm3 # [5:2.00]
+; CHECK-NEXT:    addss %xmm4, %xmm3 # [3:1.00]
+; CHECK-NEXT:    mulss {{\.LCPI.*}}@GOTOFF(%eax), %xmm3 # [8:2.00]
+; CHECK-NEXT:    movss %xmm3, (%eax) # [1:1.00]
+; CHECK-NEXT:    movss %xmm3, (%esp) # [1:1.00]
+; CHECK-NEXT:    flds (%esp) # [3:1.00]
+; CHECK-NEXT:    popl %eax # [3:1.00]
+; CHECK-NEXT:    retl
+  %div = fdiv fast float 3.0, %x
+  store float %div, float* undef, align 4
+  ret float %div
+}
 
+attributes #0 = { "unsafe-fp-math"="true" "reciprocal-estimates"="!divf,!vec-divf" }
+attributes #1 = { "unsafe-fp-math"="true" "reciprocal-estimates"="divf,vec-divf" }
+attributes #2 = { "unsafe-fp-math"="true" "reciprocal-estimates"="divf:2,vec-divf:2" }

tools/llc/llc.cpp

static cl::opt<bool> NoVerify("disable-verify", cl::Hidden,
                               cl::desc("Do not verify input module"));
 
 static cl::opt<bool> DisableSimplifyLibCalls("disable-simplify-libcalls",
                                              cl::desc("Disable simplify-libcalls"));
 
 static cl::opt<bool> ShowMCEncoding("show-mc-encoding", cl::Hidden,
                                     cl::desc("Show encoding in .s output"));
 
+static cl::opt<bool> PrintSchedule("print-schedule", cl::Hidden,
+                        cl::init(false),
+                        cl::desc("Print [latency:throughput] in .s output"));
+
 static cl::opt<bool> EnableDwarfDirectory(
     "enable-dwarf-directory", cl::Hidden,
     cl::desc("Use .file directives with an explicit directory."));
 
 static cl::opt<bool> AsmVerbose("asm-verbose",
                                 cl::desc("Add comments to directives."),
                                 cl::init(true));
 
static int compileModule(char **argv, LLVMContext &Context) {
   case '1': OLvl = CodeGenOpt::Less; break;
   case '2': OLvl = CodeGenOpt::Default; break;
   case '3': OLvl = CodeGenOpt::Aggressive; break;
   }
 
   TargetOptions Options = InitTargetOptionsFromCodeGenFlags();
   Options.DisableIntegratedAS = NoIntegratedAssembler;
   Options.MCOptions.ShowMCEncoding = ShowMCEncoding;
+  Options.MCOptions.PrintSchedule = PrintSchedule;
   Options.MCOptions.MCUseDwarfDirectory = EnableDwarfDirectory;
   Options.MCOptions.AsmVerbose = AsmVerbose;
   Options.MCOptions.PreserveAsmComments = PreserveComments;
   Options.MCOptions.IASSearchPaths = IncludeDirs;
 
   std::unique_ptr<TargetMachine> Target(
       TheTarget->createTargetMachine(TheTriple.getTriple(), CPUStr, FeaturesStr,
                                      Options, getRelocModel(), CMModel, OLvl));