[llvm-exegesis] Post-processing for chained instrs in latency mode (PR41275)

[lebedev.ri]

Summary

Ok, so this turned out to be easier than i expected.
Also, i initially thought that other modes might need this post-processing,
but i'm not sure which opcodes are affected there, if any.

The results look much better.
On BdVer2 this exposes at least one stable sched cluster
that has inconsistent values from from the measurements,
and a dozen or so somewhat-unstable clusters that also are inconsistent.

latency-clusters-stable.html165 KBDownload

latency-clusters-unstable.html1 MBDownload

Resolves(?) PR41275

[lebedev.ri]

Regarding remaining noise for these chained instrs, i'm guessing we
also need to account for some other latencies, e.g. domain crossing?

[lebedev.ri]

Refactor code a little

[lebedev.ri]

While there, also model domain transfer delays.

I'm not sure about the first instruction though,

vpextrb	$1, %xmm2, %edi
vpinsrb	$1, %edi, %xmm7, %xmm2
vpextrb	$1, %xmm2, %edi
vpinsrb	$1, %edi, %xmm7, %xmm2

don't we go fpu->int->fpu ?
Shouldn't we be also modelling the fpu2int delays?

Also, still seeing some weird noise in some of these chained instructions.

[lebedev.ri]

Also, only post-process 2-instruction benchmarks that were serialized
by llvm-exegesis itself, not just every n-instruction benchmarks.

[courbet]

I'll let gchatelet@ review that one as he started looking at doing that some time ago.

[gchatelet]

It doesn't feel like the right approach to me.
llvm-exegesis is about relying on measurement to deduce informations. Here you're using a priori knowledge (SchedClass) which may be wrong.
To me, a better approach would be to read all the experiments, create the dependency graph between the 2-instructions snippets and solve a system of equations to recover the per instruction latency, then use the analyzer on the result.

test/tools/llvm-exegesis/X86/analysis-latency-instruction-chaining.test

13 ↗

(On Diff #192977)

In this paragraph it is unclear what next instruction refers to.

Maybe rephrase to something like "By constructions, snippet's instructions execution never overlaps. As a consequence the per-snippet latency is the sum of the latencies of the instructions in the Snippet. For instance, in the following example latency(BT32rr R11D R11D) + latency(RCR8rCL R11B R11B) = 12
"

45 ↗

(On Diff #192977)

Just to be sure, you crafted this InstructionBenchmark for the sake of testing right?
I don't see how the current code generator can come up with three instructions.

tools/llvm-exegesis/lib/PostProcessing.h

10 ↗

(On Diff #192977)

Maybe explain what post processing does. As such it is not too informative.

[lebedev.ri]

In D60000#1451313, @gchatelet wrote:

It doesn't feel like the right approach to me.

In D60000#1451313, @gchatelet wrote:

llvm-exegesis is about relying on measurement to deduce informations. Here you're using a priori knowledge (SchedClass) which may be wrong.

Yes.

To me, a better approach would be to read all the experiments, create the dependency graph between the 2-instructions snippets and solve a system of equations to recover the per instruction latency, then use the analyzer on the result.

Can you explain that in a bit more detail? Something like

lat(i_0) = m_0
sum(lat(i_t)+lat(i_0)) = m_1
lat(i_1) = m_2
sum(lat(i_t)+lat(i_1)) = m_3
...
lat(i_n) => ?
sum(lat(i_t)+lat(i_n)) = m_n
lat(i_t) => ?

Do you suggest to take known lat(i_0)..lat(i_n) from measurements too?
How will that scheme will account for domain transfer delays?

test/tools/llvm-exegesis/X86/analysis-latency-instruction-chaining.test

45 ↗

(On Diff #192977)

Yes, this is just a copy-paste.
As you can see, assembled_snippet is identical to the previous benchmark.

[gchatelet]

To me, a better approach would be to read all the experiments, create the dependency graph between the 2-instructions snippets and solve a system of equations to recover the per instruction latency, then use the analyzer on the result.

Can you explain that in a bit more detail? Something like

lat(i_0) = m_0
sum(lat(i_t)+lat(i_0)) = m_1
lat(i_1) = m_2
sum(lat(i_t)+lat(i_1)) = m_3
...
lat(i_n) => ?
sum(lat(i_t)+lat(i_n)) = m_n
lat(i_t) => ?

Do you suggest to take known lat(i_0)..lat(i_n) from measurements too?

Yes.
Measurement ought to be coherent for runs on the same CPU so with enough data the resulting linear system will be over constrained and is solvable using ordinary least square (https://en.wikipedia.org/wiki/Ordinary_least_squares).

How will that scheme will account for domain transfer delays?

You could associate a supplementary variable for pairs of instructions but this would need a lot of data to converge (way too many variables).
A simpler approach is to make sure that we don't generate domain transfer delays when generating the snippet, rejecting pairs of instructions for which it would occur.
Or annotate the results with information about domain transfer delays and deal with it in the post processing (adding variables for pairs in {int,vector,fp,store}² when we know such a transfer exist) this way we can recover the domain transfer delays as well.

[lebedev.ri]

In D60000#1452962, @gchatelet wrote:

To me, a better approach would be to read all the experiments, create the dependency graph between the 2-instructions snippets and solve a system of equations to recover the per instruction latency, then use the analyzer on the result.

Can you explain that in a bit more detail? Something like

lat(i_0) = m_0
sum(lat(i_t)+lat(i_0)) = m_1
lat(i_1) = m_2
sum(lat(i_t)+lat(i_1)) = m_3
...
lat(i_n) => ?
sum(lat(i_t)+lat(i_n)) = m_n
lat(i_t) => ?

Do you suggest to take known lat(i_0)..lat(i_n) from measurements too?

Yes.
Measurement ought to be coherent for runs on the same CPU so with enough data the resulting linear system will be over constrained and is solvable using ordinary least square (https://en.wikipedia.org/wiki/Ordinary_least_squares).

Okay, sounds sane.
Intermediate issue to solve: creating all these 2-instr chained configs must then also
create configs to measure the params of that second instr (without going into endless loop).

How will that scheme will account for domain transfer delays?

You could associate a supplementary variable for pairs of instructions but this would need a lot of data to converge (way too many variables).
A simpler approach is to make sure that we don't generate domain transfer delays when generating the snippet, rejecting pairs of instructions for which it would occur.
Or annotate the results with information about domain transfer delays and deal with it in the post processing (adding variables for pairs in {int,vector,fp,store}² when we know such a transfer exist) this way we can recover the domain transfer delays as well.

Hmm, i don't mean to mock, but it sounds kinda hand-wavy/arbitrary.
We don't want to use latencies of instructions specified in scheduler profile, but at the same
time we are ok with expecting that the sched profile explains all the domain transfer delays.
They should probably also be variables, not hardcoded. But i admit i have not thought that part through.

In the same thoughtflow, would be great if it could try to magically deduce the actual Units (*not* just pressure distribution)

[gchatelet]

Intermediate issue to solve: creating all these 2-instr chained configs must then also
create configs to measure the params of that second instr (without going into endless loop).

Yes it's more work and potentially really long runtimes. The randomization part is here because the original design was to run llvm-exegesis on many machines and aggregate the runs to do the analysis. The more data the better so we can test our hypotheses.

How will that scheme will account for domain transfer delays?

You could associate a supplementary variable for pairs of instructions but this would need a lot of data to converge (way too many variables).
A simpler approach is to make sure that we don't generate domain transfer delays when generating the snippet, rejecting pairs of instructions for which it would occur.
Or annotate the results with information about domain transfer delays and deal with it in the post processing (adding variables for pairs in {int,vector,fp,store}² when we know such a transfer exist) this way we can recover the domain transfer delays as well.

Hmm, i don't mean to mock

Then don't. The mere fact you're mentioning it is already suspicious :)

but it sounds kinda hand-wavy/arbitrary.

I offered three paths to explore, I don't know yet if they work, nor which one is best.
I'd need to dedicate some time to this but I don't have much right now TBH.

We don't want to use latencies of instructions specified in scheduler profile, but at the same
time we are ok with expecting that the sched profile explains all the domain transfer delays.

We want the tool to be as generic as possible to target other platforms but some decisions are to be target specific (e.g. the stack based registers of x87)
This is why llvm-exegesis relies on ExegesisTarget in llvm-exegesis/lib/Target.h.
As such it makes sense that the measurement tool is customized based on what we know or suspect about the target.
On the contrary it is a desirable property that the analysis tool to not depend on target knowledge as much.

They should probably also be variables, not hardcoded. But i admit i have not thought that part through.

This was one of the suggestions I made (DTD being variables), I agree this needs to be thought through.

In the same thoughtflow, would be great if it could try to magically deduce the actual Units (*not* just pressure distribution)

That would be nice indeed although nothing obvious comes to mind. We only have Hardware performance counter and code Snippets to recover knowledge.

[lebedev.ri]

In D60000#1454661, @gchatelet wrote:

Intermediate issue to solve: creating all these 2-instr chained configs must then also
create configs to measure the params of that second instr (without going into endless loop).

Yes it's more work and potentially really long runtimes. The randomization part is here because the original design was to run llvm-exegesis on many machines and aggregate the runs to do the analysis. The more data the better so we can test our hypotheses.

How will that scheme will account for domain transfer delays?

You could associate a supplementary variable for pairs of instructions but this would need a lot of data to converge (way too many variables).
A simpler approach is to make sure that we don't generate domain transfer delays when generating the snippet, rejecting pairs of instructions for which it would occur.
Or annotate the results with information about domain transfer delays and deal with it in the post processing (adding variables for pairs in {int,vector,fp,store}² when we know such a transfer exist) this way we can recover the domain transfer delays as well.

Hmm, i don't mean to mock

Then don't. The mere fact you're mentioning it is already suspicious :)

:)

but it sounds kinda hand-wavy/arbitrary.

I offered three paths to explore, I don't know yet if they work, nor which one is best.
I'd need to dedicate some time to this but I don't have much right now TBH.

We don't want to use latencies of instructions specified in scheduler profile, but at the same
time we are ok with expecting that the sched profile explains all the domain transfer delays.

We want the tool to be as generic as possible to target other platforms but some decisions are to be target specific (e.g. the stack based registers of x87)
This is why llvm-exegesis relies on ExegesisTarget in llvm-exegesis/lib/Target.h.
As such it makes sense that the measurement tool is customized based on what we know or suspect about the target.
On the contrary it is a desirable property that the analysis tool to not depend on target knowledge as much.

What i'm saying is that it is pretty much known as a fact that LLVM (at least on X86?) has (very?) partial modelling
of these extra delays, so if we take as granted that all of them are already modelled, the post-processing
may produce rather misleading results.

They should probably also be variables, not hardcoded. But i admit i have not thought that part through.

This was one of the suggestions I made (DTD being variables), I agree this needs to be thought through.

ack

In the same thoughtflow, would be great if it could try to magically deduce the actual Units (*not* just pressure distribution)

That would be nice indeed although nothing obvious comes to mind. We only have Hardware performance counter and code Snippets to recover knowledge.

ack

[lebedev.ri]

In D60000#1454661, @gchatelet wrote:

Intermediate issue to solve: creating all these 2-instr chained configs must then also
create configs to measure the params of that second instr (without going into endless loop).

Yes it's more work and potentially really long runtimes. The randomization part is here because the original design was to run llvm-exegesis on many machines and aggregate the runs to do the analysis. The more data the better so we can test our hypotheses.

How will that scheme will account for domain transfer delays?

You could associate a supplementary variable for pairs of instructions but this would need a lot of data to converge (way too many variables).
A simpler approach is to make sure that we don't generate domain transfer delays when generating the snippet, rejecting pairs of instructions for which it would occur.
Or annotate the results with information about domain transfer delays and deal with it in the post processing (adding variables for pairs in {int,vector,fp,store}² when we know such a transfer exist) this way we can recover the domain transfer delays as well.

Hmm, i don't mean to mock

Then don't. The mere fact you're mentioning it is already suspicious :)

but it sounds kinda hand-wavy/arbitrary.

I offered three paths to explore, I don't know yet if they work, nor which one is best.
I'd need to dedicate some time to this but I don't have much right now TBH.

Any kind of approximate time estimate on this?
Next month? This release cycle? Next year? "some day, once there is time"?

[lebedev.ri]

@courbet @gchatelet bump. any plans on working on that functionality anytime soon? :)
Not sure how healthy for any project such silence in form of neither working on
nor allowing other interested parties to work on missing functionality...

[gchatelet]

In D60000#1785210, @lebedev.ri wrote:

@courbet @gchatelet bump. any plans on working on that functionality anytime soon? :)
Not sure how healthy for any project such silence in form of neither working on
nor allowing other interested parties to work on missing functionality...

Yes I'll be working on it from now on. Stay tuned.

[gchatelet]

In D60000#1785687, @gchatelet wrote:

In D60000#1785210, @lebedev.ri wrote:

@courbet @gchatelet bump. any plans on working on that functionality anytime soon? :)
Not sure how healthy for any project such silence in form of neither working on
nor allowing other interested parties to work on missing functionality...

Yes I'll be working on it from now on. Stay tuned.

I've been side tracked but this is still on my radar.

[lebedev.ri]

Now that i'm renewly motivated to finally get this fixed, let's try again :)

This implements linear algebra support library (i don't think we want to pull in eigen do we?),
implements ordinary/weighted least squares, and uses it to post-process latency benchmark points.

This kinda just works:

latency-clusters-ols.html198 KBDownload

vs

latency-clusters.html151 KBDownload

But i've made two observations:

1. if we measured a single instruction, we can say that the measurement is precise, so it's estimator should deviate from the actual measurement by as little as possible => weighted least squares
2. i was really hoping i wouldn't have to deal with domain transfer delays, but the results are kinda bogus right now :) That, or there is some other issue.

[lebedev.ri]

Actually drop pre-monorepo diff parts.

[gchatelet]

Some early comments on the linear algebra library.
Let me sync with @courbet first.

llvm/include/llvm/Support/LinearAlgebra.h

30

static_assert to make sure that std::is_base_of<T, Derived>::value==true

33

fix

37

you may want to provide a function to factor the cast operation in.

37–38

These could be properties rows() columns().

60–64

fix

66–67

ditto - properties

82

MatrixTy

83

AFAIU from the implementation, this really is a TransposedMatrixView. I think it's important to highlight because m should outlive the TransposedMatrix object.

85

matrix

85

Using a reference instead of a pointer prevents rebinding of this object (can't move, can't reassign). Is it a design decision?

97

This is still a View AFAICT

140–141

Wrong comment

149

const

197

This is hard to read, do you mind introducing a variable for the expected value?

209

const

220

const here and below

352

const here and below

[lebedev.ri]

In D60000#2738598, @gchatelet wrote:

Some early comments on the linear algebra library.
Let me sync with @courbet first.

I'm still in early experimentation stages with this, this is not the code i'd post normally, i only posted just to avoid concurrent implementations.
I'm still getting really weird latencies, and just subtracting forwarding delays from sched model doesn't help.
This may mean that either the math is wrong, or there are many more forwarding delays that we don't model.

[lebedev.ri]

Looks like we end up recovering a forwarding delay of -0.04.
It should be ~-2.

[lebedev.ri]

Hm, one thing i should/could try here, is to ignore benchmarks that may have forwarding delays.
It won't help with unmodelled forwarding delays (we don't model IVec->FVec delays do we?),
but maybe it will be good-enough for some sequences..

llvm/include/llvm/Support/LinearAlgebra.h

 
 namespace llvm {
 namespace linearalgebra {
 
 ///--------------------------------------------------------------------------///
 
 template <typename, typename> class TransposedMatrix;
 
 template <typename T, typename Derived> struct MatrixInterface {
   using value_type = T;
 
   //  MatrixInterface() = delete;
   MatrixInterface &operator=(MatrixInterface) = delete;
   MatrixInterface &operator=(const MatrixInterface &) = delete;
 
   int getNumRows() const { return ((const Derived *)this)->getNumRows(); }
   int getNumColumns() const { return ((const Derived *)this)->getNumColumns(); }
 
   T &operator()(int row, int col) {
     return ((Derived *)this)->operator()(row, col);
   }
 
   /// X^T
   TransposedMatrix<T, Derived> getTransposedMatrix() {
     return TransposedMatrix<T, Derived>(*((Derived *)this));
   }
 };
 
 ///--------------------------------------------------------------------------///
 
 template <typename T = double>
 class Matrix : public MatrixInterface<T, Matrix<T>> {
 public:
   Matrix(int num_rows, int num_cols) : num_rows(num_rows), num_cols(num_cols) {
     storage.resize(num_rows * num_cols);
   }
 
   // Disallow copying.
   //  Matrix(const Matrix &) = delete;
   //  Matrix(Matrix &&) = delete;
   //  Matrix &operator=(Matrix) = delete;
   //  Matrix &operator=(const Matrix &) = delete;
   //  Matrix &operator=(Matrix &&) = delete;
 
   int getNumRows() const { return num_rows; };
   int getNumColumns() const { return num_cols; };
 
   T &operator()(int row, int col) {
     assert(row >= 0 && row < num_rows);
     assert(col >= 0 && col < num_cols);
 
     return storage[num_cols * row + col];
   }
 
 private:
   std::vector<T> storage;
   int num_rows;
   int num_cols;
 };
 
 template <typename T, typename InnerTy>
 class TransposedMatrix
     : public MatrixInterface<T, TransposedMatrix<T, InnerTy>> {
   InnerTy &m;
 
 public:
   TransposedMatrix(InnerTy &m) : m(m) {}
 
   int getNumRows() const { return m.getNumColumns(); };
   int getNumColumns() const { return m.getNumRows(); };
 
   T &operator()(int row, int col) { return m(col, row); }
 };
 
 template <typename T, typename LHSTy, typename RHSTy>
 class AugmentedMatrix
     : public MatrixInterface<T, AugmentedMatrix<T, LHSTy, RHSTy>> {
   MatrixInterface<T, LHSTy> &a;
   MatrixInterface<T, RHSTy> &b;
 
 public:
   AugmentedMatrix(MatrixInterface<T, LHSTy> &a, MatrixInterface<T, RHSTy> &b)
       : a(a), b(b) {
     assert(a.getNumRows() == b.getNumRows());
for (int col = 0; col != LHS.getNumColumns(); ++col) {
       m(row, col) = LHS(row, col);
     }
   }
   return m;
 }
 
 ///--------------------------------------------------------------------------///
 
 /// Get a square matrix with all elements being zero except the elements
 /// on the diagonal, which are ones.
 template <typename T, typename LHSTy>
 bool isSquareMatrix(const MatrixInterface<T, LHSTy> &LHS) {
   return LHS.getNumRows() == LHS.getNumColumns();
 }
 
 /// Are all the entries below the main diagonal are zero?
 template <typename T, typename LHSTy>
 bool isUpperTriangularMatrix(MatrixInterface<T, LHSTy> &LHS) {
   for (int row = 0; row != LHS.getNumRows(); ++row) {
     for (int col = 0; col != LHS.getNumColumns(); ++col) {
       // Is this element on the diagonal, or to the right of diagonal?
       if (col >= row)
         continue;
       if (LHS(row, col) != 0)
         return false;
     }
   }
   return true;
 }
 
 ///--------------------------------------------------------------------------///
 
 /// Get a square matrix with all elements being zero except the elements
 /// on the diagonal, which are ones.
 template <typename T> Matrix<T> getIdentityMatrix(int Size) {
   Matrix<T> m(Size, Size);
   for (int diagEltIdx = 0; diagEltIdx != Size; ++diagEltIdx)
     m(diagEltIdx, diagEltIdx) = 1;
   return m;
 }
 
+/// Are all elements of this matrix, except the ones on the diagonal, a zeros?
+template <typename T, typename LHSTy>
+bool isDiagonalMatrix(MatrixInterface<T, LHSTy> &LHS) {
+  if (!isSquareMatrix(LHS))
+    return false;
+  for (int row = 0; row != LHS.getNumRows(); ++row) {
+    for (int col = 0; col != LHS.getNumColumns(); ++col) {
+      if (col == row)
+        continue;
+      if (LHS(row, col) != 0)
+        return false;
+    }
+  }
+  return true;
+}
+
 /// Are all elements of this matrix zeros, except the elements on the main
 /// diagonal, which are ones?
 template <typename T, typename LHSTy>
 bool isIdentityMatrix(MatrixInterface<T, LHSTy> &LHS) {
-  if (!isSquareMatrix(LHS))
+  if (!isDiagonalMatrix(LHS))
     return false;
   for (int row = 0; row != LHS.getNumRows(); ++row) {
     for (int col = 0; col != LHS.getNumColumns(); ++col) {
       if (LHS(row, col) != (col == row) ? 1 : 0)
         return false;
     }
   }
   return true;
 }
 
 ///--------------------------------------------------------------------------///
 
 /// On row \p row, which column is the first one to have a non-zero value?
 /// Returns number of columns if no such element exists.
 template <typename T, typename LHSTy>
 int getLeadingCoeffientColumn(MatrixInterface<T, LHSTy> &LHS, int row) {
   int col = 0;
   for (; col != LHS.getNumColumns(); ++col) {
     if (LHS(row, col) != 0)
       break;
   }
   return col;
 }
 
 /// Are all elements of the row \p row zeros?
 template <typename T, typename LHSTy>
 int isZeroRow(MatrixInterface<T, LHSTy> &LHS, int row) {
   return getLeadingCoeffientColumn(LHS, row) == LHS.getNumColumns();
 }
 
 template <typename T, typename LHSTy>
 bool isMatrixInRowEchelonForm(MatrixInterface<T, LHSTy> &LHS) {
   assert(isUpperTriangularMatrix(LHS));
 
   bool seenZeroRow = false;
for (int pivot = 0; pivot != LHS.getNumRows(); ++pivot) {
     int pivotRow = findPivotingRow(LHS, column, pivot);
 
     if (pivotRow != pivot) {
       swapRows(LHS, pivotRow, pivot);
       pivotRow = pivot;
     }
 
     T &pivotElement = LHS(pivotRow, column);
-    if (pivotElement == 0)
-      continue;
+    assert(pivotElement != 0);
 
     divideRow(LHS, pivotRow, pivotElement);
     pivotElement = 1.0; // Account for floating point rounding issues.
 
     for (int row = 0; row != LHS.getNumRows(); ++row) {
       if (row == pivotRow)
         continue;
 
       T &currElement = LHS(row, column);
       if (currElement == 0)
         continue;
 
       subtractRowMultiple(LHS, row, pivotRow, currElement);
       currElement = 0; // Account for floating point rounding issues.
     }
   }
   assert(isMatrixInReducedRowEchelonForm(LHS));
 }
 
 /// X^-1
 template <typename T, typename LHSTy>
 Matrix<T> getInverseMatrix(MatrixInterface<T, LHSTy> &LHS) {
   assert(isSquareMatrix(LHS));
   Matrix<T> A = cloneMatrix<T>(LHS);
   Matrix<T> B = getIdentityMatrix<T>(LHS.getNumRows());
   auto AB = getAugmentedMatrix(A, B);
   performGaussJordanElimination(AB);
   assert(isIdentityMatrix(A));
   return B;
 }
Matrix<T> getMomentMatrix(MatrixInterface<T, LHSTy> &LHS,
   auto LHSTransposed = LHS.getTransposedMatrix();
   return LHSTransposed * RHS;
 }
 
 ///--------------------------------------------------------------------------///
 
 /// (X^T * X)^-1 * X^T * y
 template <typename T, typename LHSTy, typename RHSTy>
-Matrix<T> getOrdinaryLeastSquaresEstimation(MatrixInterface<T, LHSTy> &LHS,
-                                            MatrixInterface<T, RHSTy> &RHS) {
-  assert(LHS.getNumRows() >= LHS.getNumColumns());
-  assert(LHS.getNumRows() == RHS.getNumRows());
-  assert(RHS.getNumColumns() == 1);
+Matrix<T> getOrdinaryLeastSquaresEstimator(MatrixInterface<T, LHSTy> &XMat,
+                                           MatrixInterface<T, RHSTy> &YVec) {
+  assert(XMat.getNumRows() >= XMat.getNumColumns());
+  assert(XMat.getNumRows() == YVec.getNumRows());
+  assert(YVec.getNumColumns() == 1);
+
+  auto XMatTransposed = XMat.getTransposedMatrix();
+  Matrix<T> XMatNormal = XMatTransposed * XMat;
+  Matrix<T> XMatNormalInverse = getInverseMatrix<T>(XMatNormal);
+  Matrix<T> XYMoment = XMatTransposed * YVec;
+  return XMatNormalInverse * XYMoment;
+}
 
-  auto LHSNormal = getNormalMatrix<T>(LHS);
-  auto LHSNormalInverse = getInverseMatrix<T>(LHSNormal);
-  auto LHSRHSMoment = getMomentMatrix<T>(LHS, RHS);
-  return LHSNormalInverse * LHSRHSMoment;
+/// (X^T * W * X)^-1 * X^T * W * y
+template <typename T, typename LHSTy, typename RHSTy>
+Matrix<T> getWeightedLeastSquaresEstimator(MatrixInterface<T, LHSTy> &XMat,
+                                           MatrixInterface<T, RHSTy> &YVec,
+                                           MatrixInterface<T, RHSTy> &WMat) {
+  assert(XMat.getNumRows() >= XMat.getNumColumns());
+  assert(isDiagonalMatrix(WMat));
+  assert(XMat.getNumRows() == WMat.getNumRows());
+  assert(XMat.getNumRows() == YVec.getNumRows());
+  assert(YVec.getNumColumns() == 1);
+
+  auto XMatTransposed = XMat.getTransposedMatrix();
+  Matrix<T> XMatTransposedWeighted = XMatTransposed * WMat;
+  Matrix<T> XMatTransposedWeightedNormal = XMatTransposedWeighted * XMat;
+  Matrix<T> XMatTransposedWeightedNormalInverse =
+      getInverseMatrix<T>(XMatTransposedWeightedNormal);
+  Matrix<T> XYMomentWeighted = XMatTransposedWeighted * YVec;
+  return XMatTransposedWeightedNormalInverse * XYMomentWeighted;
 }
 
 ///--------------------------------------------------------------------------///
 
 } // namespace linearalgebra
 } // namespace llvm
 
 #endif

llvm/lib/MC/MCSchedule.cpp

MCSchedModel::getForwardingDelayCycles(ArrayRef<MCReadAdvanceEntry> Entries,
                                        unsigned WriteResourceID) {
   if (Entries.empty())
     return 0;
 
   int DelayCycles = 0;
   for (const MCReadAdvanceEntry &E : Entries) {
     if (E.WriteResourceID != WriteResourceID)
       continue;
+    llvm::errs() << "&E = " << &E << ", cycles = " << E.Cycles << "\n";
     DelayCycles = std::min(DelayCycles, E.Cycles);
   }
 
   return std::abs(DelayCycles);
 }

llvm/lib/Target/X86/X86ScheduleZnver3.td

 
 def : ReadAdvance<ReadAfterVecLd, Znver3Model.VecLoadLatency>;
 def : ReadAdvance<ReadAfterVecXLd, Znver3Model.VecLoadLatency>;
 def : ReadAdvance<ReadAfterVecYLd, Znver3Model.VecLoadLatency>;
 
 // AMD SOG 19h, 2.11 Floating-Point Unit
 // There is 1 cycle of added latency for a result to cross
 // from F to I or I to F domain.
-def : ReadAdvance<ReadInt2Fpu, -1>;
+def : ReadAdvance<ReadInt2Fpu, -42>;
 
 // Instructions with both a load and a store folded are modeled as a folded
 // load + WriteRMW.
 defm : Zn3WriteResInt<WriteRMW, [Zn3AGU012, Zn3Store], Znver3Model.StoreLatency, [1, 1], 0>;
 
 // Loads, stores, and moves, not folded with other operations.
 defm : Zn3WriteResInt<WriteLoad, [Zn3AGU012, Zn3Load], !add(Znver3Model.LoadLatency, 1), [1, 1], 1>;
 

llvm/test/tools/llvm-exegesis/X86/analysis-latency-instruction-chaining-domain-transfer.test

+# RUN: llvm-exegesis -mode=analysis -benchmarks-file=%s -analysis-clusters-output-file=- -analysis-clustering-epsilon=0.5 -analysis-inconsistency-epsilon=0.5 -analysis-numpoints=1 | FileCheck -check-prefixes=CHECK %s
+
+# CHECK: {{^}}cluster_id,opcode_name,config,sched_class,latency{{$}}
+
+# CHECK-NEXT: {{^}}0,
+# CHECK-SAME: ,10.08{{$}}
+
+# CHECK: {{^}}1,
+# CHECK-SAME: ,11.07{{$}}
+
+# PINSRBrr has latency of 2 cycles. (the value from scheduling profile!)
+# But int to fpu units data transfer causes additional latency of 10 cycles.
+# Thus the actual latency of VPEXTRBrr is 10..11.
+
+---
+mode:            latency
+key:
+  instructions:
+    - 'VPEXTRBrr R15D XMM3 i_0x1'
+    - 'PINSRBrr XMM3 XMM3 R15D i_0x1'
+  config:          ''
+  register_initial_values:
+    - 'XMM3=0x0'
+cpu_name:        bdver2
+llvm_triple:     x86_64-unknown-linux-gnu
+num_repetitions: 10000
+measurements:
+  - { key: latency, value: 0.0000, per_snippet_value: 22.0802 }
+error:           ''
+info:            Repeating two instructions
+assembled_snippet: 41574883EC10C7042400000000C744240400000000C744240800000000C744240C00000000C5FA6F1C244883C410C4C37914DF0166410F3A20DF01C4C37914DF0166410F3A20DF01C4C37914DF0166410F3A20DF01C4C37914DF0166410F3A20DF01C4C37914DF0166410F3A20DF01C4C37914DF0166410F3A20DF01C4C37914DF0166410F3A20DF01C4C37914DF0166410F3A20DF01415FC3
+...
+---
+mode:            latency
+key:
+  instructions:
+    - 'PEXTRBrr ESI XMM7 i_0x1'
+    - 'VCVTSI642SSrr XMM7 XMM12 RSI'
+  config:          ''
+  register_initial_values:
+    - 'XMM7=0x0'
+    - 'XMM12=0x0'
+    - 'RSI=0x0'
+cpu_name:        bdver2
+llvm_triple:     x86_64-unknown-linux-gnu
+num_repetitions: 10000
+measurements:
+  - { key: latency, value: 12.533, per_snippet_value: 25.066 }
+error:           ''
+info:            Repeating two instructions
+assembled_snippet: 4883EC10C7042400000000C744240400000000C744240800000000C744240C00000000C5FA6F3C244883C4104883EC10C7042400000000C744240400000000C744240800000000C744240C00000000C57A6F24244883C41048BE0000000000000000660F3A14FE01C4E19A2AFE660F3A14FE01C4E19A2AFE660F3A14FE01C4E19A2AFE660F3A14FE01C4E19A2AFE660F3A14FE01C4E19A2AFE660F3A14FE01C4E19A2AFE660F3A14FE01C4E19A2AFE660F3A14FE01C4E19A2AFEC3
+...

llvm/test/tools/llvm-exegesis/X86/analysis-latency-instruction-chaining.test

+# RUN: llvm-exegesis -mode=analysis -benchmarks-file=%s -analysis-clusters-output-file=- -analysis-clustering-epsilon=0.5 -analysis-inconsistency-epsilon=0.5 -analysis-numpoints=1 | FileCheck -check-prefixes=CHECK %s
+
+# CHECK: {{^}}cluster_id,opcode_name,config,sched_class,latency{{$}}
+
+# CHECK-NEXT: {{^}}0,
+# CHECK-SAME: ,1.00{{$}}
+# CHECK-NEXT: {{^}}0,
+# CHECK-SAME: ,1.00{{$}}
+
+# Instructions were executed serially, meaning that the next instruction
+# *ONLY* starts executing when the current instruction finishes.
+# Thus, the real latency of the first instruction is the per_snippet_value minus
+# the sum of latencies of all the other instructions in the snippet.
+
+# RCR8rCL has latency of 11. (the value from scheduling profile!)
+# Latency of whole snipped is 12 or 23. (not measured, hand-written.)
+# Thus, latency of BT32rr is 12-11 = 1, or 23-11-11 = 1
+
+---
+mode:            latency
+key:
+  instructions:
+    - 'BT32rr R11D R11D'
+    - 'RCR8rCL R11B R11B'
+  config:          ''
+  register_initial_values:
+    - 'R11D=0x0'
+    - 'R11B=0x0'
+    - 'CL=0x0'
+cpu_name:        bdver2
+llvm_triple:     x86_64-unknown-linux-gnu
+num_repetitions: 10000
+measurements:
+  - { key: latency, value: 50.0000, per_snippet_value: 100.0000 }
+error:           ''
+info:            Repeating two instructions
+assembled_snippet: 41BB0000000041B300B100450FA3DB41D2DB450FA3DB41D2DB450FA3DB41D2DB450FA3DB41D2DB450FA3DB41D2DB450FA3DB41D2DB450FA3DB41D2DB450FA3DB41D2DBC3
+...
+---
+mode:            latency
+key:
+  instructions:
+    - 'RCR8rCL R11B R11B'
+  config:          ''
+  register_initial_values:
+    - 'R11D=0x0'
+    - 'R11B=0x0'
+    - 'CL=0x0'
+cpu_name:        bdver2
+llvm_triple:     x86_64-unknown-linux-gnu
+num_repetitions: 10000
+measurements:
+  - { key: latency, value: 25.0000, per_snippet_value: 25.0000 }
+error:           ''
+info:            Repeating two instructions
+assembled_snippet: 41BB0000000041B300B100450FA3DB41D2DB450FA3DB41D2DB450FA3DB41D2DB450FA3DB41D2DB450FA3DB41D2DB450FA3DB41D2DB450FA3DB41D2DB450FA3DB41D2DBC3
+...

llvm/tools/llvm-exegesis/lib/Analysis.h

 
 namespace llvm {
 namespace exegesis {
 
 // A helper class to analyze benchmark results for a target.
 class Analysis {
 public:
   Analysis(const Target &Target, std::unique_ptr<MCInstrInfo> InstrInfo,
+           std::unique_ptr<MCSubtargetInfo> SubtargetInfo,
            const InstructionBenchmarkClustering &Clustering,
            double AnalysisInconsistencyEpsilon,
            bool AnalysisDisplayUnstableOpcodes,
            const std::string &ForceCpuName = "");
 
   // Prints a csv of instructions for each cluster.
   struct PrintClusters {};
   // Find potential errors in the scheduling information given measurements.

llvm/tools/llvm-exegesis/lib/Analysis.cpp

#endif
   for (const auto &Measurement : Point.Measurements) {
     OS << kCsvSep;
     writeMeasurementValue<kEscapeCsv>(OS, Measurement.PerInstructionValue);
   }
   OS << "\n";
 }
 
 Analysis::Analysis(const Target &Target, std::unique_ptr<MCInstrInfo> InstrInfo,
+                   std::unique_ptr<MCSubtargetInfo> SubtargetInfo,
                    const InstructionBenchmarkClustering &Clustering,
                    double AnalysisInconsistencyEpsilon,
                    bool AnalysisDisplayUnstableOpcodes,
                    const std::string &ForceCpuName)
-    : Clustering_(Clustering), InstrInfo_(std::move(InstrInfo)),
+    : Clustering_(Clustering), SubtargetInfo_(std::move(SubtargetInfo)),
+      InstrInfo_(std::move(InstrInfo)),
       AnalysisInconsistencyEpsilonSquared_(AnalysisInconsistencyEpsilon *
                                            AnalysisInconsistencyEpsilon),
       AnalysisDisplayUnstableOpcodes_(AnalysisDisplayUnstableOpcodes) {
   if (Clustering.getPoints().empty())
     return;
 
   const InstructionBenchmark &FirstPoint = Clustering.getPoints().front();
   const std::string CpuName =
       ForceCpuName.empty() ? FirstPoint.CpuName : ForceCpuName;
   RegInfo_.reset(Target.createMCRegInfo(FirstPoint.LLVMTriple));
   MCTargetOptions MCOptions;
   AsmInfo_.reset(
       Target.createMCAsmInfo(*RegInfo_, FirstPoint.LLVMTriple, MCOptions));
-  SubtargetInfo_.reset(
-      Target.createMCSubtargetInfo(FirstPoint.LLVMTriple, CpuName, ""));
   InstPrinter_.reset(Target.createMCInstPrinter(
       Triple(FirstPoint.LLVMTriple), 0 /*default variant*/, *AsmInfo_,
       *InstrInfo_, *RegInfo_));
 
   Context_ = std::make_unique<MCContext>(
       Triple(FirstPoint.LLVMTriple), AsmInfo_.get(), RegInfo_.get(),
       &ObjectFileInfo_, SubtargetInfo_.get());
   Disasm_.reset(Target.createMCDisassembler(*SubtargetInfo_, *Context_));
case InstructionBenchmark::Latency:
     break;
   case InstructionBenchmark::Uops:
   case InstructionBenchmark::InverseThroughput:
     writeParallelSnippetHtml(OS, Point.Key.Instructions, *InstrInfo_);
     break;
   default:
     llvm_unreachable("invalid mode");
   }
+  if (Point.Info == "WLS fixpoint" || Point.Info == "WLS reconstruction")
+    OS << " <small><i>(" << Point.Info << ")</i></small>";
   OS << "</span> <span class=\"mono\">";
   writeEscaped<kEscapeHtml>(OS, Point.Key.Config);
   OS << "</span></li>";
 }
 
 void Analysis::printSchedClassClustersHtml(
     const std::vector<SchedClassCluster> &Clusters,
     const ResolvedSchedClass &RSC, raw_ostream &OS) const {

llvm/tools/llvm-exegesis/lib/CMakeLists.txt

add_llvm_library(LLVMExegesis
   Clustering.cpp
   CodeTemplate.cpp
   Error.cpp
   LatencyBenchmarkRunner.cpp
   LlvmState.cpp
   MCInstrDescView.cpp
   ParallelSnippetGenerator.cpp
   PerfHelper.cpp
+  PostProcessing.cpp
   RegisterAliasing.cpp
   RegisterValue.cpp
   SchedClassResolution.cpp
   SerialSnippetGenerator.cpp
   SnippetFile.cpp
   SnippetGenerator.cpp
   SnippetRepetitor.cpp
   Target.cpp
   UopsBenchmarkRunner.cpp
 
   LINK_LIBS ${libs}
 
   DEPENDS
   intrinsics_gen
   )

llvm/tools/llvm-exegesis/lib/PostProcessing.h

+//===-- PostProcessing.h ----------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// Post-processing for the benchmark points.
+///
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_TOOLS_LLVM_EXEGESIS_POSTPROCESSING_H
+#define LLVM_TOOLS_LLVM_EXEGESIS_POSTPROCESSING_H
+
+#include "BenchmarkResult.h"
+#include <vector>
+
+namespace llvm {
+namespace exegesis {
+
+void PostProcessChainedLatencyBenchmarkPoints(
+    std::vector<InstructionBenchmark> &Points,
+    const llvm::MCInstrInfo &InstrInfo,
+    const llvm::MCSubtargetInfo &SubtargetInfo);
+
+} // namespace exegesis
+} // namespace llvm
+
+#endif // LLVM_TOOLS_LLVM_EXEGESIS_POSTPROCESSING_H

llvm/tools/llvm-exegesis/lib/PostProcessing.cpp

+//===-- PostProcessing.cpp --------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "PostProcessing.h"
+#include "Clustering.h"
+#include "SchedClassResolution.h"
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/SmallSet.h"
+#include "llvm/Support/LinearAlgebra.h"
+#include <utility>
+
+namespace llvm {
+namespace exegesis {
+
+static std::pair<double, bool>
+getAdjustedLatency(const InstructionBenchmark *RelevantPoint,
+                   const llvm::MCInstrInfo &InstrInfo,
+                   const llvm::MCSubtargetInfo &SubtargetInfo) {
+  assert(RelevantPoint->Measurements.size() == 1);
+  assert(RelevantPoint->Measurements[0].Key == "latency");
+  double Latency = RelevantPoint->Measurements[0].PerSnippetValue;
+  bool HaveForwardingDelays = false;
+
+  if (RelevantPoint->Key.Instructions.size() == 1)
+    return {Latency, HaveForwardingDelays};
+
+  const MCSchedModel &SM = SubtargetInfo.getSchedModel();
+  for (const MCInst &Inst : RelevantPoint->Key.Instructions) {
+    const MCInstrDesc &MCDesc = InstrInfo.get(Inst.getOpcode());
+
+    // Obtain the scheduling class information from the instruction.
+    unsigned SchedClassID = MCDesc.getSchedClass();
+    unsigned CPUID = SM.getProcessorID();
+
+    // Try to solve variant scheduling classes.
+    while (SchedClassID && SM.getSchedClassDesc(SchedClassID)->isVariant())
+      SchedClassID = SubtargetInfo.resolveVariantSchedClass(SchedClassID, &Inst,
+                                                            &InstrInfo, CPUID);
+
+    const MCSchedClassDesc &SCDesc = *SM.getSchedClassDesc(SchedClassID);
+    unsigned ForwardingDelayCycles = MCSchedModel::getForwardingDelayCycles(
+        SubtargetInfo.getReadAdvanceEntries(SCDesc));
+    HaveForwardingDelays |= ForwardingDelayCycles != 0;
+  }
+
+  return {Latency, HaveForwardingDelays};
+}
+
+void PostProcessChainedLatencyBenchmarkPoints(
+    std::vector<InstructionBenchmark> &Points,
+    const llvm::MCInstrInfo &InstrInfo,
+    const llvm::MCSubtargetInfo &SubtargetInfo) {
+  auto IsLatencyPoint = [](const InstructionBenchmark &Point) {
+    return Point.Mode == InstructionBenchmark::ModeE::Latency &&
+           Point.Error.empty();
+  };
+  auto IsChainedLatencyPoint =
+      [IsLatencyPoint](const InstructionBenchmark &Point) {
+        return IsLatencyPoint(Point) && Point.Key.Instructions.size() >= 2;
+      };
+
+  unsigned NumOpcodes = InstrInfo.getNumOpcodes();
+
+  // Which opcodes were ever chained in all of the benchmark points?
+  // Only record the ones for which we succeeded in measuring latency.
+  std::vector<int> OpcodeToIndex(NumOpcodes, /*Index=*/-1);
+  std::vector<MCInst> IndexToOpcode;
+  IndexToOpcode.reserve(NumOpcodes);
+  for (const InstructionBenchmark &Point :
+       make_filter_range(Points, IsChainedLatencyPoint)) {
+    for (const MCInst &Instruction : Point.Key.Instructions) {
+      const unsigned Opcode = Instruction.getOpcode();
+      assert(Opcode < NumOpcodes && "NumOpcodes is incorrect (too small)");
+      if (OpcodeToIndex[Opcode] != -1) // Already seen chained?
+        continue;
+      OpcodeToIndex[Opcode] = IndexToOpcode.size();
+      IndexToOpcode.emplace_back(Instruction);
+    }
+  }
+
+  if (IndexToOpcode.empty())
+    return; // Lucky us.
+
+  // Remember all the points that contained any opcode that was ever chained.
+  // We can not do this in the previous loop, because if Opc0 was chained with
+  // Opc1, we'll miss all standalone points for Opc1 before seeing the chaining.
+  // We store indexes into Points to avoid iterator invalidation.
+  SmallVector<const InstructionBenchmark *, 64> RelevantPoints;
+  for (const InstructionBenchmark &Point :
+       make_filter_range(Points, IsLatencyPoint)) {
+    if (any_of(Point.Key.Instructions,
+               [OpcodeToIndex](const MCInst &Instruction) {
+                 return OpcodeToIndex[Instruction.getOpcode()] != -1;
+               }))
+      RelevantPoints.emplace_back(&Point);
+  }
+
+  using namespace linearalgebra;
+
+  errs() << "rows = " << RelevantPoints.size() << "\n";
+  errs() << "cols = " << IndexToOpcode.size() << "\n";
+  Matrix<double> OpcodeChaining(RelevantPoints.size(), IndexToOpcode.size());
+  Matrix<double> ForwardingDelayPresence(RelevantPoints.size(), 1);
+  Matrix<double> SnippetLatency(RelevantPoints.size(), 1);
+  Matrix<double> Weights = getIdentityMatrix<double>(RelevantPoints.size());
+
+  std::vector<char> FixpointOpcodes(NumOpcodes, /*IsFixpoint=*/false);
+
+  for (auto I : enumerate(RelevantPoints)) {
+    int row = I.index();
+    const InstructionBenchmark *RelevantPoint = I.value();
+
+    // FIXME: this
+    std::tie(SnippetLatency(row, 0), ForwardingDelayPresence(row, 0)) =
+        getAdjustedLatency(RelevantPoint, InstrInfo, SubtargetInfo);
+
+    if (RelevantPoint->Key.Instructions.size() == 1) {
+      // Give a(n arbitrary) bonus to the points that directly measured
+      // one single specific instruction. We believe these measurements
+      // to be precise, so the fit should basically not change them.
+      Weights(row, row) = 1e+3;
+      FixpointOpcodes[RelevantPoint->Key.Instructions.front().getOpcode()] =
+          true;
+    }
+
+    for (const MCInst &Instrn : RelevantPoint->Key.Instructions) {
+      double &Entry = OpcodeChaining(row, OpcodeToIndex[Instrn.getOpcode()]);
+      assert(Entry == 0);
+      Entry = 1;
+    }
+  }
+  RelevantPoints.clear();
+  OpcodeToIndex.clear();
+
+  auto Zz = getAugmentedMatrix(OpcodeChaining, ForwardingDelayPresence);
+  auto EstimatedInstructionLatencies =
+      getWeightedLeastSquaresEstimator<double>(Zz, SnippetLatency, Weights);
+  assert(EstimatedInstructionLatencies.getNumColumns() == 1);
+  assert((size_t)EstimatedInstructionLatencies.getNumRows() >=
+         IndexToOpcode.size());
+
+  //  Points.erase(
+  //      std::remove_if(Points.begin(), Points.end(), IsChainedLatencyPoint),
+  //      Points.end());
+  Points.clear();
+
+  Points.reserve(Points.size() + IndexToOpcode.size());
+  for (auto I : enumerate(IndexToOpcode)) {
+    const MCInst &Instruction = I.value();
+    const double EstimatedInstructionLatency =
+        EstimatedInstructionLatencies(I.index(), 0);
+
+    Points.emplace_back();
+    InstructionBenchmark &NewPoint = Points.back();
+
+    NewPoint.Key.Instructions.emplace_back(Instruction);
+    NewPoint.Mode = InstructionBenchmark::Latency;
+    NewPoint.CpuName = Points.front().CpuName;
+    NewPoint.LLVMTriple = Points.front().LLVMTriple;
+    NewPoint.Measurements.emplace_back(
+        BenchmarkMeasure::Create("latency", EstimatedInstructionLatency));
+    NewPoint.Info = FixpointOpcodes[Instruction.getOpcode()]
+                        ? "WLS fixpoint"
+                        : "WLS reconstruction";
+  }
+  for (int I = IndexToOpcode.size();
+       I < EstimatedInstructionLatencies.getNumRows(); ++I)
+    errs() << "Reconstructed forwaring delay = "
+           << EstimatedInstructionLatencies(I, 0) << "\n";
+}
+
+} // namespace exegesis
+} // namespace llvm

llvm/tools/llvm-exegesis/llvm-exegesis.cpp

 
 #include "lib/Analysis.h"
 #include "lib/BenchmarkResult.h"
 #include "lib/BenchmarkRunner.h"
 #include "lib/Clustering.h"
 #include "lib/Error.h"
 #include "lib/LlvmState.h"
 #include "lib/PerfHelper.h"
+#include "lib/PostProcessing.h"
 #include "lib/SnippetFile.h"
 #include "lib/SnippetRepetitor.h"
 #include "lib/Target.h"
 #include "lib/TargetSelect.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/MC/MCInstBuilder.h"
 #include "llvm/MC/MCObjectFileInfo.h"
static void analysisMain() {
   }
 
   InitializeNativeTarget();
   InitializeNativeTargetAsmPrinter();
   InitializeNativeTargetDisassembler();
 
   // Read benchmarks.
   const LLVMState State("");
-  const std::vector<InstructionBenchmark> Points = ExitOnFileError(
+  std::vector<InstructionBenchmark> Points = ExitOnFileError(
       BenchmarkFile, InstructionBenchmark::readYamls(State, BenchmarkFile));
 
   outs() << "Parsed " << Points.size() << " benchmark points\n";
   if (Points.empty()) {
     errs() << "no benchmarks to analyze\n";
     return;
   }
   // FIXME: Check that all points have the same triple/cpu.
   // FIXME: Merge points from several runs (latency and uops).
 
+
   std::string Error;
   const auto *TheTarget =
       TargetRegistry::lookupTarget(Points[0].LLVMTriple, Error);
   if (!TheTarget) {
     errs() << "unknown target '" << Points[0].LLVMTriple << "'\n";
     return;
   }
 
   std::unique_ptr<MCInstrInfo> InstrInfo(TheTarget->createMCInstrInfo());
   assert(InstrInfo && "Unable to create instruction info!");
 
+  std::unique_ptr<MCSubtargetInfo> SubtargetInfo(
+      TheTarget->createMCSubtargetInfo(Points[0].LLVMTriple, CpuName, ""));
+  assert(SubtargetInfo && "Unable to create subtarget info!");
+
+  PostProcessChainedLatencyBenchmarkPoints(Points, *InstrInfo, *SubtargetInfo);
+
   const auto Clustering = ExitOnErr(InstructionBenchmarkClustering::create(
       Points, AnalysisClusteringAlgorithm, AnalysisDbscanNumPoints,
       AnalysisClusteringEpsilon, InstrInfo->getNumOpcodes()));
 
-  const Analysis Analyzer(*TheTarget, std::move(InstrInfo), Clustering,
-                          AnalysisInconsistencyEpsilon,
-                          AnalysisDisplayUnstableOpcodes, CpuName);
+  const Analysis Analyzer(
+      *TheTarget, std::move(InstrInfo), std::move(SubtargetInfo), Clustering,
+      AnalysisInconsistencyEpsilon, AnalysisDisplayUnstableOpcodes, CpuName);
 
   maybeRunAnalysis<Analysis::PrintClusters>(Analyzer, "analysis clusters",
                                             AnalysisClustersOutputFile);
   maybeRunAnalysis<Analysis::PrintSchedClassInconsistencies>(
       Analyzer, "sched class consistency analysis",
       AnalysisInconsistenciesOutputFile);
 }
 

llvm/unittests/Support/LinearAlgebraTest.cpp

TEST(LinearAlgebraTest, OLS) {
 
   M(1, 0) = 1;
   M(1, 1) = 4;
 
   Matrix<double> y(2, 1);
   y(0, 0) = 5;
   y(1, 0) = 6;
 
-  auto beta = getOrdinaryLeastSquaresEstimation<double>(M, y);
+  auto beta = getOrdinaryLeastSquaresEstimator<double>(M, y);
   EXPECT_EQ(std::vector<double>({4, 1. / 2}), getAllValues(beta));
 }
 
 TEST(LinearAlgebraTest, OLSOverdefined) {
   Matrix<double> M(4, 2);
   M(0, 0) = 1;
   M(0, 1) = 1;
 
   M(1, 0) = 1;
   M(1, 1) = 2;
 
   M(2, 0) = 1;
   M(2, 1) = 3;
 
   M(3, 0) = 1;
   M(3, 1) = 4;
 
   Matrix<double> y(4, 1);
   y(0, 0) = 6;
   y(1, 0) = 5;
   y(2, 0) = 7;
   y(3, 0) = 10;
 
-  auto beta = getOrdinaryLeastSquaresEstimation<double>(M, y);
+  auto beta = getOrdinaryLeastSquaresEstimator<double>(M, y);
   EXPECT_EQ(std::vector<double>({7. / 2, 7. / 5}), getAllValues(beta));
 }
 
 } // namespace