[Passes] Run vector-combine early with -fenable-matrix.

[fhahn]

Summary

IR with matrix intrinsics is likely to also contain large vector
operations, which can benefit from early simplifications.

This is the last step in a series of changes to improve code-gen for
code using matrix subscript operators with the C/C++ matrix extension in
CLang, like

using matrix_t = double __attribute__((matrix_type(15, 15)));

void foo(unsigned i, matrix_t &A, matrix_t &B) {
  for (unsigned j = 0; j < 4; ++j)
    for (unsigned k = 0; k < i; k++)
      B[k][j] -= A[k][j] * B[i][j];
}

https://clang.godbolt.org/z/6dKxK1Ed7

[xbolva00]

Any testcase to ensure that passes together now produce what you want?

[RKSimon]

Some test coverage (phaseordering?) would be useful.

[fhahn]

Thanks for taking a look! I added a phase ordering test and updated the pipeline tests as well.

[RKSimon]

LGTM - anyone else have any comments?

[spatel]

Over in D102002, I am looking at divergence between the regular and LTO pipelines...
If I'm seeing this correctly, we will not alter the LTO pipeline in this patch. Is that intentional?

[fhahn]

In D102496#2772019, @spatel wrote:

Over in D102002, I am looking at divergence between the regular and LTO pipelines...
If I'm seeing this correctly, we will not alter the LTO pipeline in this patch. Is that intentional?

Yes that's intentional. The motivation to run vector-combine early here is to catch combine & scalarization opportunities before operations are moved too much by GVN, unrolling & co. At the LTO stage, those should already be covered by the pre-LTO steps, so there's no need to do another run during the LTO stage I think.

[spatel]

In D102496#2773930, @fhahn wrote:

In D102496#2772019, @spatel wrote:

Over in D102002, I am looking at divergence between the regular and LTO pipelines...
If I'm seeing this correctly, we will not alter the LTO pipeline in this patch. Is that intentional?

Yes that's intentional. The motivation to run vector-combine early here is to catch combine & scalarization opportunities before operations are moved too much by GVN, unrolling & co. At the LTO stage, those should already be covered by the pre-LTO steps, so there's no need to do another run during the LTO stage I think.

Ah, I still haven't made sense of all the pipeline stages for LTO.
LGTM.

[fhahn]

Rebased. I am planning on landing this after D110171 lands.

llvm/lib/Passes/PassBuilder.cpp

PassBuilder::buildFunctionSimplificationPipeline(OptimizationLevel Level,
   // *All* loop passes must preserve it, in order to be able to use it.
   FPM.addPass(createFunctionToLoopPassAdaptor(std::move(LPM2),
                                               /*UseMemorySSA=*/false,
                                               /*UseBlockFrequencyInfo=*/false));
 
   // Delete small array after loop unroll.
   FPM.addPass(SROA());
 
+  // The matrix extension can introduce large vector operations early, which can
+  // benefit from running vector-combine early on.
+  if (EnableMatrix)
+    FPM.addPass(VectorCombinePass());
+
   // Eliminate redundancies.
   FPM.addPass(MergedLoadStoreMotionPass());
   if (RunNewGVN)
     FPM.addPass(NewGVNPass());
   else
     FPM.addPass(GVN());
 
   // Sparse conditional constant propagation.

llvm/lib/Transforms/IPO/PassManagerBuilder.cpp

if (SizeLevel == 0)
     MPM.add(createPGOMemOPSizeOptLegacyPass());
 
   // TODO: Investigate the cost/benefit of tail call elimination on debugging.
   if (OptLevel > 1)
     MPM.add(createTailCallEliminationPass()); // Eliminate tail calls
   MPM.add(createCFGSimplificationPass());      // Merge & remove BBs
   MPM.add(createReassociatePass());           // Reassociate expressions
 
+  // The matrix extension can introduce large vector operations early, which can
+  // benefit from running vector-combine early on.
+  if (EnableMatrix)
+    MPM.add(createVectorCombinePass());
+
   // Begin the loop pass pipeline.
   if (EnableSimpleLoopUnswitch) {
     // The simple loop unswitch pass relies on separate cleanup passes. Schedule
     // them first so when we re-process a loop they run before other loop
     // passes.
     MPM.add(createLoopInstSimplifyPass());
     MPM.add(createLoopSimplifyCFGPass());
   }