[LoopNest] Handle loop-nest passes in LoopPassManager

[TaWeiTu]

Summary

Per http://llvm.org/OpenProjects.html#llvm_loopnest, the goal of this patch (and other following patches) is to create facilities that allow implementing loop nest passes that run on top-level loop nests for the New Pass Manager.

This patch extends the functionality of LoopPassManager to handle loop-nest passes by specializing the definition of LoopPassManager that accepts both kinds of passes in addPass.

Only loop passes are executed if L is not a top-level one, and both kinds of passes are executed if L is top-level. Currently, loop nest passes should have the following run method:

PreservedAnalyses run(LoopNest &, LoopAnalysisManager &, LoopStandardAnalysisResults &, LPMUpdater &);

[TaWeiTu]

Add unit test.

[ychen]

Thank you for the patch. I'll take a look over the weekend.

[TaWeiTu]

ping :)

[etiotto]

llvm/include/llvm/Transforms/Scalar/LoopPassManager.h

66

Not sure if you can use an anonymous namespace

llvm/lib/Transforms/Scalar/LoopPassManager.cpp

28

Suggest:

PreservedAnalyses PA = (!L.getParentLoop() && !LoopNestPasses.empty()) 
                                       ? runWithLoopNestPasses(L, AM, AR, U)
                                       : runWithoutLoopNestPasses(L, AM, AR, U);

That way PA is initialized where it is declared.

52

assert L is the outermost loop (no parent loop).

66

This isn't that clear, perhaps use something like !isa<LoopNestPass>(Pass) ?

133

Is not immediately clear what is the type of PassPA. Could you use the explicit type Optional<PreservedAnalyses> ?

llvm/unittests/Transforms/Scalar/LoopPassManagerTest.cpp

1605

HandleLoopNestPass isn't used. How does this test work?

[Whitney]

llvm/include/llvm/Transforms/Scalar/LoopPassManager.h

160

Do you think is a good idea to rename PassCategories to IsLoopNestPass?

161

can we use llvm data structures here?

[ychen]

• How about using PreservedAnalyses run(Loop &, LoopAnalysisManager &, LoopStandardAnalysisResults &, LPMUpdater &, LoopNest &); for the interface. I think it should work better because it is still a loop pass interface.
• Add a test case for PassInstrument?

llvm/include/llvm/Transforms/Scalar/LoopPassManager.h

68

There is is_detected in STLExtras.h that could simplify this. It is fine putting this in the PassManager itself.

[TaWeiTu]

Address (some) code review comments.

[TaWeiTu]

llvm/include/llvm/Transforms/Scalar/LoopPassManager.h

161

Since this is a specialized version of PassManager and PassManager uses std::vector instead of the LLVM data structures, I'm not sure which one is better here.

llvm/unittests/Transforms/Scalar/LoopPassManagerTest.cpp

1605

I'm not sure what this means actually because I suppose this is how GoogleTest works and the test does run. Do you mean that I should add some comments to illustrate what functionality is the unittest testing?

[TaWeiTu]

Replace SFINAE helper with llvm::is_detected.

[TaWeiTu]

Add PassInstrumentation callback tests.

[TaWeiTu]

In D87045#2264236, @ychen wrote:

• How about using PreservedAnalyses run(Loop &, LoopAnalysisManager &, LoopStandardAnalysisResults &, LPMUpdater &, LoopNest &); for the interface. I think it should work better because it is still a loop pass interface.

Sounds like a good idea to me. Let' see what @Whitney and @etiotto think about the new interface first, and I'll update the patch later if they agree on that. Thanks!

[Whitney]

llvm/include/llvm/Transforms/Scalar/LoopPassManager.h

161

In that case, I agree not to change.

[TaWeiTu]

llvm/lib/Transforms/Scalar/LoopPassManager.cpp

66

Hi, thanks for the suggestion! Do you think it's clear enough now after renaming it to IsLoopNestPass?

[TaWeiTu]

Gentle ping to @Whitney and @etiotto on the opinion about the new interface for loop-nest pass suggested above. Thanks!

[Whitney]

In D87045#2265645, @TaWeiTu wrote:

In D87045#2264236, @ychen wrote:

• How about using PreservedAnalyses run(Loop &, LoopAnalysisManager &, LoopStandardAnalysisResults &, LPMUpdater &, LoopNest &); for the interface. I think it should work better because it is still a loop pass interface.

Sounds like a good idea to me. Let' see what @Whitney and @etiotto think about the new interface first, and I'll update the patch later if they agree on that. Thanks!

I am not strongly against it, however is not clear to me what the extra argument Loop & provide, as we can get the loop from LoopNest & by LN.getOutermostLoop().

[TaWeiTu]

In D87045#2274009, @Whitney wrote:

In D87045#2265645, @TaWeiTu wrote:

In D87045#2264236, @ychen wrote:

• How about using PreservedAnalyses run(Loop &, LoopAnalysisManager &, LoopStandardAnalysisResults &, LPMUpdater &, LoopNest &); for the interface. I think it should work better because it is still a loop pass interface.

Sounds like a good idea to me. Let' see what @Whitney and @etiotto think about the new interface first, and I'll update the patch later if they agree on that. Thanks!

I am not strongly against it, however is not clear to me what the extra argument Loop & provide, as we can get the loop from LoopNest & by LN.getOutermostLoop().

Thanks for your comment!
I think the benefit of the new interface is that it shows that the loop-nest pass is just a special case of loop pass that provides an additional LoopNest object.
However, I think your concern about passing redundant parameters to the function is also valid.
Both interfaces seem equally good for me actually, and I don't have any preference for either of them.

I think maybe we can stick to the current interface for now (unless @ychen has a strong preference for the other one) and update it in the future if necessary.
Any thoughts?

By the way, is there any other concern regarding this patch other than the interface now?

Thanks again for all your review and time.

[TaWeiTu]

Gentle ping.

[TaWeiTu]

ping.

[Whitney]

Thanks for working on it! Are you planning to change one of the LoopPass to a LoopNestPass next?

[TaWeiTu]

In D87045#2301091, @Whitney wrote:

Thanks for working on it! Are you planning to change one of the LoopPass to a LoopNestPass next?

Sure! I'm planning to port LoopInterchange to NPM as a LoopNestPass first in the future.

[ychen]

From http://llvm.org/OpenProjects.html#llvm_loopnest, it was mentioned that
"If you decide to write it as a loop pass, then you are wasting compile time to traverse to your pass and return right away when the given loop is not the outermost loop."

Taking another look at the overall approach of this patch: currently for all loops (top-level or nested) in a function, when encountering a loop nest pass in LPM, a LoopNest is calculated and the loop nest pass is called with the computed LoopNest. Does this help compile-time in practice? It feels like it is essentially a loop pass that got passed a LoopNest as an incoming parameter that could be equally implemented (without this patch) with a normal loop pass that computes LoopNest by itself? This seems does not solve the problem quoted above.

To achieve that, I think we need to change FunctionToLoopPassAdaptor to dispatch top-level loops (aside from to regular loops and LPMs) to a LoopNestPassManager managing only loop nest passes or a loop nest pass directly.

[TaWeiTu]

In D87045#2301614, @ychen wrote:

From http://llvm.org/OpenProjects.html#llvm_loopnest, it was mentioned that
"If you decide to write it as a loop pass, then you are wasting compile time to traverse to your pass and return right away when the given loop is not the outermost loop."

Taking another look at the overall approach of this patch: currently for all loops (top-level or nested) in a function, when encountering a loop nest pass in LPM, a LoopNest is calculated and the loop nest pass is called with the computed LoopNest. Does this help compile-time in practice? It feels like it is essentially a loop pass that got passed a LoopNest as an incoming parameter that could be equally implemented (without this patch) with a normal loop pass that computes LoopNest by itself? This seems does not solve the problem quoted above.

To achieve that, I think we need to change FunctionToLoopPassAdaptor to dispatch top-level loops (aside from to regular loops and LPMs) to a LoopNestPassManager managing only loop nest passes or a loop nest pass directly.

If I understood correctly, I think the follow-up patch D87531 is designed to address the issue.
Specifically, in FunctionToLoopPassAdaptor, if the input pass is a loop pass or an LPM containing both loop passes and loop-nest passes, all loops in the function have to be processed regardless if they're top-level or not.
On the other hand, if the input pass is a loop-nest pass or is an LPM containing only loop-nest passes, only top-level loops are sent to the worklist and processed.

Does this solve the problem you mentioned? Thanks!

[ychen]

In D87045#2302107, @TaWeiTu wrote:

In D87045#2301614, @ychen wrote:

From http://llvm.org/OpenProjects.html#llvm_loopnest, it was mentioned that
"If you decide to write it as a loop pass, then you are wasting compile time to traverse to your pass and return right away when the given loop is not the outermost loop."

Taking another look at the overall approach of this patch: currently for all loops (top-level or nested) in a function, when encountering a loop nest pass in LPM, a LoopNest is calculated and the loop nest pass is called with the computed LoopNest. Does this help compile-time in practice? It feels like it is essentially a loop pass that got passed a LoopNest as an incoming parameter that could be equally implemented (without this patch) with a normal loop pass that computes LoopNest by itself? This seems does not solve the problem quoted above.

To achieve that, I think we need to change FunctionToLoopPassAdaptor to dispatch top-level loops (aside from to regular loops and LPMs) to a LoopNestPassManager managing only loop nest passes or a loop nest pass directly.

If I understood correctly, I think the follow-up patch D87531 is designed to address the issue.

ha, thanks. I missed this one.

Specifically, in FunctionToLoopPassAdaptor, if the input pass is a loop pass or an LPM containing both loop passes and loop-nest passes, all loops in the function have to be processed regardless if they're top-level or not.
On the other hand, if the input pass is a loop-nest pass or is an LPM containing only loop-nest passes, only top-level loops are sent to the worklist and processed.

Could you please add comments about this either before LPM or before FunctionToLoopPassAdaptor? It is helpful for understanding the code.

Does this solve the problem you mentioned? Thanks!

Yes, totally.

LGTM. Please hold off a bit in case @asbirlea wants to chime in.

[TaWeiTu]

Rebased on top of the current master.

[TaWeiTu]

@Whitney, @ychen thanks for the review! Comments regarding loop/loop-nest mode and when they're used have be added to D87531.

[TaWeiTu]

Hi @Whitney, @ychen, if the patches are ready to land, could you do me a favor of committing this as well as the follow-up D87531 (with --author="Ta-Wei Tu <[e-mail]>)?
Thanks in advance!

[asbirlea]

The design looks good, but I'd like to see more comments documenting some of the details for future code readers.
e.g. document the Optional<PreservedAnalysis> as including the test for skipped passes in that single function call (this is new behavior vs other implementations), document the addPass specializations and populating IsLoopNestPass.

[TaWeiTu]

@asbirlea Sure! Thanks for the review, I will update the patch later.

[TaWeiTu]

Add documentation regarding Optional<PreservedAnalyses> and the specializations of addPass.

[TaWeiTu]

Ping ?

[Whitney]

In D87045#2312059, @TaWeiTu wrote:

Hi @Whitney, @ychen, if the patches are ready to land, could you do me a favor of committing this as well as the follow-up D87531 (with --author="Ta-Wei Tu <[e-mail]>)?
Thanks in advance!

Sure. Let's see if @asbirlea is happy with it.

[TaWeiTu]

Rebased.

[TaWeiTu]

Ping.

[TaWeiTu]

Ping. @asbirlea is there any concern regarding this patch and the follow-up D87531 (both the code and the documentation)?
Thanks!

[TaWeiTu]

Ping.
If there's no objection about these patches I'll commit them in a few days.
Thanks for the reviews!

[asbirlea]

No objections.

[TaWeiTu]

Rebased and fix tests

[Whitney]

If there's no objection about these patches I'll commit them in a few days.

@TaWeiTu Are you committing this change?

[TaWeiTu]

In D87045#2453007, @Whitney wrote:

If there's no objection about these patches I'll commit them in a few days.

@TaWeiTu Are you committing this change?

Been quite busy recently and totally forgot about it.
Thanks for committing it!

[Whitney]

opps...forgot to deliver with --author="Ta-Wei Tu <[e-mail]>.

llvm/include/llvm/Analysis/LoopNestAnalysis.h

public:
   unsigned getMaxPerfectDepth() const { return MaxPerfectDepth; }
 
   /// Return true if all loops in the loop nest are in simplify form.
   bool areAllLoopsSimplifyForm() const {
     return llvm::all_of(Loops,
                         [](const Loop *L) { return L->isLoopSimplifyForm(); });
   }
 
+  StringRef getName() const { return Loops.front()->getName(); }
+
 protected:
   const unsigned MaxPerfectDepth; // maximum perfect nesting depth level.
   LoopVectorTy Loops; // the loops in the nest (in breadth first order).
 };
 
 raw_ostream &operator<<(raw_ostream &, const LoopNest &);
 
 /// This analysis provides information for a loop nest. The analysis runs on

llvm/include/llvm/Transforms/Scalar/LoopPassManager.h

 #include "llvm/ADT/PriorityWorklist.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/Analysis/AliasAnalysis.h"
 #include "llvm/Analysis/BasicAliasAnalysis.h"
 #include "llvm/Analysis/BlockFrequencyInfo.h"
 #include "llvm/Analysis/GlobalsModRef.h"
 #include "llvm/Analysis/LoopAnalysisManager.h"
 #include "llvm/Analysis/LoopInfo.h"
+#include "llvm/Analysis/LoopNestAnalysis.h"
 #include "llvm/Analysis/MemorySSA.h"
 #include "llvm/Analysis/ScalarEvolution.h"
 #include "llvm/Analysis/ScalarEvolutionAliasAnalysis.h"
 #include "llvm/Analysis/TargetLibraryInfo.h"
 #include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/IR/Dominators.h"
 #include "llvm/IR/PassInstrumentation.h"
 #include "llvm/IR/PassManager.h"
 #include "llvm/Transforms/Utils/LCSSA.h"
 #include "llvm/Transforms/Utils/LoopSimplify.h"
 #include "llvm/Transforms/Utils/LoopUtils.h"
 
 namespace llvm {
 
 // Forward declarations of an update tracking API used in the pass manager.
 class LPMUpdater;
 
 // Explicit specialization and instantiation declarations for the pass manager.
 // See the comments on the definition of the specialization for details on how
 // it differs from the primary template.
 template <>
-PreservedAnalyses
+class PassManager<Loop, LoopAnalysisManager, LoopStandardAnalysisResults &,
+                  LPMUpdater &>
+    : public PassInfoMixin<
           PassManager<Loop, LoopAnalysisManager, LoopStandardAnalysisResults &,
-            LPMUpdater &>::run(Loop &InitialL, LoopAnalysisManager &AM,
-                               LoopStandardAnalysisResults &AnalysisResults,
-                               LPMUpdater &U);
-extern template class PassManager<Loop, LoopAnalysisManager,
-                                  LoopStandardAnalysisResults &, LPMUpdater &>;
+                      LPMUpdater &>> {
+private:
+  template <typename PassT>
+  using HasRunOnLoopT = decltype(std::declval<PassT>().run(
+      std::declval<Loop &>(), std::declval<LoopAnalysisManager &>(),
+      std::declval<LoopStandardAnalysisResults &>(),
+      std::declval<LPMUpdater &>()));
+
+public:
+  /// Construct a pass manager.
+  ///
+  /// If \p DebugLogging is true, we'll log our progress to llvm::dbgs().
+  explicit PassManager(bool DebugLogging = false)
+      : DebugLogging(DebugLogging) {}
+
+  // FIXME: These are equivalent to the default move constructor/move
+  // assignment. However, using = default triggers linker errors due to the
+  // explicit instantiations below. Find a way to use the default and remove the
+  // duplicated code here.
+  PassManager(PassManager &&Arg)
+      : IsLoopNestPass(std::move(Arg.IsLoopNestPass)),
+        LoopPasses(std::move(Arg.LoopPasses)),
+        LoopNestPasses(std::move(Arg.LoopNestPasses)),
+        DebugLogging(std::move(Arg.DebugLogging)) {}
+
+  PassManager &operator=(PassManager &&RHS) {
+    IsLoopNestPass = std::move(RHS.IsLoopNestPass);
+    LoopPasses = std::move(RHS.LoopPasses);
+    LoopNestPasses = std::move(RHS.LoopNestPasses);
+    DebugLogging = std::move(RHS.DebugLogging);
+    return *this;
+  }
+
+  PreservedAnalyses run(Loop &L, LoopAnalysisManager &AM,
+                        LoopStandardAnalysisResults &AR, LPMUpdater &U);
+
+  /// Add either a loop pass or a loop-nest pass to the pass manager. Append \p
+  /// Pass to the list of loop passes if it has a dedicated \fn run() method for
+  /// loops and to the list of loop-nest passes if the \fn run() method is for
+  /// loop-nests instead. Also append whether \p Pass is loop-nest pass or not
+  /// to the end of \var IsLoopNestPass so we can easily identify the types of
+  /// passes in the pass manager later.
+  template <typename PassT>
+  std::enable_if_t<is_detected<HasRunOnLoopT, PassT>::value>
+  addPass(PassT Pass) {
+    using LoopPassModelT =
+        detail::PassModel<Loop, PassT, PreservedAnalyses, LoopAnalysisManager,
+    IsLoopNestPass.push_back(false);
+    LoopPasses.emplace_back(new LoopPassModelT(std::move(Pass)));
+  }
+
+  template <typename PassT>
+  std::enable_if_t<!is_detected<HasRunOnLoopT, PassT>::value>
+  addPass(PassT Pass) {
+    using LoopNestPassModelT =
+        detail::PassModel<LoopNest, PassT, PreservedAnalyses,
+                          LoopAnalysisManager, LoopStandardAnalysisResults &,
+                          LPMUpdater &>;
+    IsLoopNestPass.push_back(true);
+    LoopNestPasses.emplace_back(new LoopNestPassModelT(std::move(Pass)));
+  }
+
+  // Specializations of `addPass` for `RepeatedPass`. These are necessary since
+  // `RepeatedPass` has a templated `run` method that will result in incorrect
+  // detection of `HasRunOnLoopT`.
+  template <typename PassT>
+  std::enable_if_t<is_detected<HasRunOnLoopT, PassT>::value>
+  addPass(RepeatedPass<PassT> Pass) {
+    using RepeatedLoopPassModelT =
+        detail::PassModel<Loop, RepeatedPass<PassT>, PreservedAnalyses,
+                          LoopAnalysisManager, LoopStandardAnalysisResults &,
+                          LPMUpdater &>;
+    IsLoopNestPass.push_back(false);
+    LoopPasses.emplace_back(new RepeatedLoopPassModelT(std::move(Pass)));
+  }
+
+  template <typename PassT>
+  std::enable_if_t<!is_detected<HasRunOnLoopT, PassT>::value>
+  addPass(RepeatedPass<PassT> Pass) {
+    using RepeatedLoopNestPassModelT =
+        detail::PassModel<LoopNest, RepeatedPass<PassT>, PreservedAnalyses,
+                          LoopAnalysisManager, LoopStandardAnalysisResults &,
+                          LPMUpdater &>;
+    IsLoopNestPass.push_back(true);
+    LoopNestPasses.emplace_back(
+        new RepeatedLoopNestPassModelT(std::move(Pass)));
+  }
+
+  bool isEmpty() const { return LoopPasses.empty() && LoopNestPasses.empty(); }
+
+  static bool isRequired() { return true; }
+
+protected:
+  using LoopPassConceptT =
+      detail::PassConcept<Loop, LoopAnalysisManager,
+                          LoopStandardAnalysisResults &, LPMUpdater &>;
+  using LoopNestPassConceptT =
+      detail::PassConcept<LoopNest, LoopAnalysisManager,
+                          LoopStandardAnalysisResults &, LPMUpdater &>;
+
+  // BitVector that identifies whether the passes are loop passes or loop-nest
+  // passes (true for loop-nest passes).
+  BitVector IsLoopNestPass;
+  std::vector<std::unique_ptr<LoopPassConceptT>> LoopPasses;
+  std::vector<std::unique_ptr<LoopNestPassConceptT>> LoopNestPasses;
+
+  /// Flag indicating whether we should do debug logging.
+  bool DebugLogging;
+
+  /// Run either a loop pass or a loop-nest pass. Returns `None` if
+  /// PassInstrumentation's BeforePass returns false. Otherwise, returns the
+  /// preserved analyses of the pass.
+  template <typename IRUnitT, typename PassT>
+  Optional<PreservedAnalyses>
+  runSinglePass(IRUnitT &IR, PassT &Pass, LoopAnalysisManager &AM,
+                LoopStandardAnalysisResults &AR, LPMUpdater &U,
+                PassInstrumentation &PI);
+
+  PreservedAnalyses runWithLoopNestPasses(Loop &L, LoopAnalysisManager &AM,
+                                          LoopStandardAnalysisResults &AR,
+                                          LPMUpdater &U);
+  PreservedAnalyses runWithoutLoopNestPasses(Loop &L, LoopAnalysisManager &AM,
+                                             LoopStandardAnalysisResults &AR,
+                                             LPMUpdater &U);
+};
 
 /// The Loop pass manager.
 ///
 /// See the documentation for the PassManager template for details. It runs
 /// a sequence of Loop passes over each Loop that the manager is run over. This
 /// typedef serves as a convenient way to refer to this construct.
 typedef PassManager<Loop, LoopAnalysisManager, LoopStandardAnalysisResults &,
                     LPMUpdater &>
#ifndef NDEBUG
   Loop *ParentL;
 #endif
 
   LPMUpdater(SmallPriorityWorklist<Loop *, 4> &Worklist,
              LoopAnalysisManager &LAM)
       : Worklist(Worklist), LAM(LAM) {}
 };
 
+template <typename IRUnitT, typename PassT>
+Optional<PreservedAnalyses> LoopPassManager::runSinglePass(
+    IRUnitT &IR, PassT &Pass, LoopAnalysisManager &AM,
+    LoopStandardAnalysisResults &AR, LPMUpdater &U, PassInstrumentation &PI) {
+  // Check the PassInstrumentation's BeforePass callbacks before running the
+  // pass, skip its execution completely if asked to (callback returns false).
+  if (!PI.runBeforePass<IRUnitT>(*Pass, IR))
+    return None;
+
+  PreservedAnalyses PA;
+  {
+    TimeTraceScope TimeScope(Pass->name(), IR.getName());
+    PA = Pass->run(IR, AM, AR, U);
+  }
+
+  // do not pass deleted Loop into the instrumentation
+  if (U.skipCurrentLoop())
+    PI.runAfterPassInvalidated<IRUnitT>(*Pass, PA);
+  else
+    PI.runAfterPass<IRUnitT>(*Pass, IR, PA);
+  return PA;
+}
+
 /// Adaptor that maps from a function to its loops.
 ///
 /// Designed to allow composition of a LoopPass(Manager) and a
 /// FunctionPassManager. Note that if this pass is constructed with a \c
 /// FunctionAnalysisManager it will run the \c LoopAnalysisManagerFunctionProxy
 /// analysis prior to running the loop passes over the function to enable a \c
 /// LoopAnalysisManager to be used within this run safely.
 template <typename LoopPassT>
PreservedAnalyses run(Function &F, FunctionAnalysisManager &AM) {
     // Add the loop nests in the reverse order of LoopInfo. See method
     // declaration.
     appendLoopsToWorklist(LI, Worklist);
 
 #ifndef NDEBUG
     PI.pushBeforeNonSkippedPassCallback([&LAR, &LI](StringRef PassID, Any IR) {
       if (isSpecialPass(PassID, {"PassManager"}))
         return;
-      assert(any_isa<const Loop *>(IR));
-      const Loop *L = any_cast<const Loop *>(IR);
+      assert(any_isa<const Loop *>(IR) || any_isa<const LoopNest *>(IR));
+      const Loop *L = any_isa<const Loop *>(IR)
+                          ? any_cast<const Loop *>(IR)
+                          : &any_cast<const LoopNest *>(IR)->getOutermostLoop();
       assert(L && "Loop should be valid for printing");
 
       // Verify the loop structure and LCSSA form before visiting the loop.
       L->verifyLoop();
       assert(L->isRecursivelyLCSSAForm(LAR.DT, LI) &&
              "Loops must remain in LCSSA form!");
     });
 #endif

llvm/lib/Analysis/LoopNestAnalysis.cpp

DEBUG_WITH_TYPE(
         dbgs() << "Inner loop exit block " << *InnerLoopExit
                << " does not directly lead to the outer loop latch.\n";);
     return false;
   }
 
   return true;
 }
 
+AnalysisKey LoopNestAnalysis::Key;
+
 raw_ostream &llvm::operator<<(raw_ostream &OS, const LoopNest &LN) {
   OS << "IsPerfect=";
   if (LN.getMaxPerfectDepth() == LN.getNestDepth())
     OS << "true";
   else
     OS << "false";
   OS << ", Depth=" << LN.getNestDepth();
   OS << ", OutermostLoop: " << LN.getOutermostLoop().getName();

llvm/lib/Transforms/Scalar/LoopPassManager.cpp

 //===- LoopPassManager.cpp - Loop pass management -------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Support/TimeProfiler.h"
 #include "llvm/Transforms/Scalar/LoopPassManager.h"
 #include "llvm/Analysis/LoopInfo.h"
 
 using namespace llvm;
 
-// Explicit template instantiations and specialization defininitions for core
-// template typedefs.
 namespace llvm {
-template class PassManager<Loop, LoopAnalysisManager,
-                           LoopStandardAnalysisResults &, LPMUpdater &>;
 
 /// Explicitly specialize the pass manager's run method to handle loop nest
 /// structure updates.
-template <>
 PreservedAnalyses
 PassManager<Loop, LoopAnalysisManager, LoopStandardAnalysisResults &,
             LPMUpdater &>::run(Loop &L, LoopAnalysisManager &AM,
                                LoopStandardAnalysisResults &AR, LPMUpdater &U) {
-  PreservedAnalyses PA = PreservedAnalyses::all();
 
   if (DebugLogging)
     dbgs() << "Starting Loop pass manager run.\n";
 
+  // Runs loop-nest passes only when the current loop is a top-level one.
+  PreservedAnalyses PA = (L.isOutermost() && !LoopNestPasses.empty())
+                             ? runWithLoopNestPasses(L, AM, AR, U)
+                             : runWithoutLoopNestPasses(L, AM, AR, U);
+
+  // Invalidation for the current loop should be handled above, and other loop
+  // analysis results shouldn't be impacted by runs over this loop. Therefore,
+  // the remaining analysis results in the AnalysisManager are preserved. We
+  // mark this with a set so that we don't need to inspect each one
+  // individually.
+  // FIXME: This isn't correct! This loop and all nested loops' analyses should
+  // be preserved, but unrolling should invalidate the parent loop's analyses.
+  PA.preserveSet<AllAnalysesOn<Loop>>();
+
+  if (DebugLogging)
+    dbgs() << "Finished Loop pass manager run.\n";
+
+  return PA;
+}
+
+// Run both loop passes and loop-nest passes on top-level loop \p L.
+PreservedAnalyses
+LoopPassManager::runWithLoopNestPasses(Loop &L, LoopAnalysisManager &AM,
+                                       LoopStandardAnalysisResults &AR,
+                                       LPMUpdater &U) {
+  assert(L.isOutermost() &&
+         "Loop-nest passes should only run on top-level loops.");
+  PreservedAnalyses PA = PreservedAnalyses::all();
+
   // Request PassInstrumentation from analysis manager, will use it to run
   // instrumenting callbacks for the passes later.
   PassInstrumentation PI = AM.getResult<PassInstrumentationAnalysis>(L, AR);
-  for (auto &Pass : Passes) {
-    // Check the PassInstrumentation's BeforePass callbacks before running the
-    // pass, skip its execution completely if asked to (callback returns false).
-    if (!PI.runBeforePass<Loop>(*Pass, L))
-      continue;
 
-    PreservedAnalyses PassPA;
-    {
-      TimeTraceScope TimeScope(Pass->name(), L.getName());
-      PassPA = Pass->run(L, AM, AR, U);
-    }
+  unsigned LoopPassIndex = 0, LoopNestPassIndex = 0;
 
-    // do not pass deleted Loop into the instrumentation
-    if (U.skipCurrentLoop())
-      PI.runAfterPassInvalidated<Loop>(*Pass, PassPA);
-    else
-      PI.runAfterPass<Loop>(*Pass, L, PassPA);
+  // `LoopNestPtr` points to the `LoopNest` object for the current top-level
+  // loop and `IsLoopNestPtrValid` indicates whether the pointer is still valid.
+  // The `LoopNest` object will have to be re-constructed if the pointer is
+  // invalid when encountering a loop-nest pass.
+  std::unique_ptr<LoopNest> LoopNestPtr;
+  bool IsLoopNestPtrValid = false;
+
+  for (size_t I = 0, E = IsLoopNestPass.size(); I != E; ++I) {
+    Optional<PreservedAnalyses> PassPA;
+    if (!IsLoopNestPass[I]) {
+      // The `I`-th pass is a loop pass.
+      auto &Pass = LoopPasses[LoopPassIndex++];
+      PassPA = runSinglePass(L, Pass, AM, AR, U, PI);
+    } else {
+      // The `I`-th pass is a loop-nest pass.
+      auto &Pass = LoopNestPasses[LoopNestPassIndex++];
+
+      // If the loop-nest object calculated before is no longer valid,
+      // re-calculate it here before running the loop-nest pass.
+      if (!IsLoopNestPtrValid) {
+        LoopNestPtr = LoopNest::getLoopNest(L, AR.SE);
+        IsLoopNestPtrValid = true;
+      }
+      PassPA = runSinglePass(*LoopNestPtr, Pass, AM, AR, U, PI);
+    }
+
+    // `PassPA` is `None` means that the before-pass callbacks in
+    // `PassInstrumentation` return false. The pass does not run in this case,
+    // so we can skip the following procedure.
+    if (!PassPA)
+      continue;
 
     // If the loop was deleted, abort the run and return to the outer walk.
     if (U.skipCurrentLoop()) {
-      PA.intersect(std::move(PassPA));
+      PA.intersect(std::move(*PassPA));
       break;
     }
 
     // Update the analysis manager as each pass runs and potentially
     // invalidates analyses.
-    AM.invalidate(L, PassPA);
+    AM.invalidate(L, *PassPA);
 
     // Finally, we intersect the final preserved analyses to compute the
     // aggregate preserved set for this pass manager.
-    PA.intersect(std::move(PassPA));
+    PA.intersect(std::move(*PassPA));
+
+    // Check if the current pass preserved the loop-nest object or not.
+    IsLoopNestPtrValid &= PassPA->getChecker<LoopNestAnalysis>().preserved();
 
     // FIXME: Historically, the pass managers all called the LLVM context's
     // yield function here. We don't have a generic way to acquire the
     // context and it isn't yet clear what the right pattern is for yielding
     // in the new pass manager so it is currently omitted.
     // ...getContext().yield();
   }
+  return PA;
+}
 
-  // Invalidation for the current loop should be handled above, and other loop
-  // analysis results shouldn't be impacted by runs over this loop. Therefore,
-  // the remaining analysis results in the AnalysisManager are preserved. We
-  // mark this with a set so that we don't need to inspect each one
-  // individually.
-  // FIXME: This isn't correct! This loop and all nested loops' analyses should
-  // be preserved, but unrolling should invalidate the parent loop's analyses.
-  PA.preserveSet<AllAnalysesOn<Loop>>();
+// Run all loop passes on loop \p L. Loop-nest passes don't run either because
+// \p L is not a top-level one or simply because there are no loop-nest passes
+// in the pass manager at all.
+PreservedAnalyses
+LoopPassManager::runWithoutLoopNestPasses(Loop &L, LoopAnalysisManager &AM,
+                                          LoopStandardAnalysisResults &AR,
+                                          LPMUpdater &U) {
+  PreservedAnalyses PA = PreservedAnalyses::all();
 
-  if (DebugLogging)
-    dbgs() << "Finished Loop pass manager run.\n";
+  // Request PassInstrumentation from analysis manager, will use it to run
+  // instrumenting callbacks for the passes later.
+  PassInstrumentation PI = AM.getResult<PassInstrumentationAnalysis>(L, AR);
+  for (auto &Pass : LoopPasses) {
+    Optional<PreservedAnalyses> PassPA = runSinglePass(L, Pass, AM, AR, U, PI);
 
-  return PA;
+    // `PassPA` is `None` means that the before-pass callbacks in
+    // `PassInstrumentation` return false. The pass does not run in this case,
+    // so we can skip the following procedure.
+    if (!PassPA)
+      continue;
+
+    // If the loop was deleted, abort the run and return to the outer walk.
+    if (U.skipCurrentLoop()) {
+      PA.intersect(std::move(*PassPA));
+      break;
     }
+
+    // Update the analysis manager as each pass runs and potentially
+    // invalidates analyses.
+    AM.invalidate(L, *PassPA);
+
+    // Finally, we intersect the final preserved analyses to compute the
+    // aggregate preserved set for this pass manager.
+    PA.intersect(std::move(*PassPA));
+
+    // FIXME: Historically, the pass managers all called the LLVM context's
+    // yield function here. We don't have a generic way to acquire the
+    // context and it isn't yet clear what the right pattern is for yielding
+    // in the new pass manager so it is currently omitted.
+    // ...getContext().yield();
+  }
+  return PA;
 }
+} // namespace llvm
 
 PrintLoopPass::PrintLoopPass() : OS(dbgs()) {}
 PrintLoopPass::PrintLoopPass(raw_ostream &OS, const std::string &Banner)
     : OS(OS), Banner(Banner) {}
 
 PreservedAnalyses PrintLoopPass::run(Loop &L, LoopAnalysisManager &,
                                      LoopStandardAnalysisResults &,
                                      LPMUpdater &) {
   printLoop(L, OS, Banner);
   return PreservedAnalyses::all();
 }

llvm/unittests/IR/PassBuilderCallbacksTest.cpp

static void invalidateLoop(Loop &L, LoopAnalysisManager &,
                              LoopStandardAnalysisResults &,
                              LPMUpdater &Updater) {
     Updater.markLoopAsDeleted(L, L.getName());
   }
   MockPassHandle() { setDefaults(); }
 };
 
 template <>
+struct MockPassHandle<LoopNest>
+    : MockPassHandleBase<MockPassHandle<LoopNest>, LoopNest,
+                         LoopAnalysisManager, LoopStandardAnalysisResults &,
+                         LPMUpdater &> {
+  MOCK_METHOD4(run,
+               PreservedAnalyses(LoopNest &, LoopAnalysisManager &,
+                                 LoopStandardAnalysisResults &, LPMUpdater &));
+  static void invalidateLoopNest(LoopNest &L, LoopAnalysisManager &,
+                                 LoopStandardAnalysisResults &,
+                                 LPMUpdater &Updater) {
+    Updater.markLoopAsDeleted(L.getOutermostLoop(), L.getName());
+  }
+  MockPassHandle() { setDefaults(); }
+};
+
+template <>
 struct MockPassHandle<Function>
     : MockPassHandleBase<MockPassHandle<Function>, Function> {
   MOCK_METHOD2(run, PreservedAnalyses(Function &, FunctionAnalysisManager &));
 
   MockPassHandle() { setDefaults(); }
 };
 
 template <>

 
 template <> std::string getName(const llvm::Any &WrappedIR) {
   if (any_isa<const Module *>(WrappedIR))
     return any_cast<const Module *>(WrappedIR)->getName().str();
   if (any_isa<const Function *>(WrappedIR))
     return any_cast<const Function *>(WrappedIR)->getName().str();
   if (any_isa<const Loop *>(WrappedIR))
     return any_cast<const Loop *>(WrappedIR)->getName().str();
+  if (any_isa<const LoopNest *>(WrappedIR))
+    return any_cast<const LoopNest *>(WrappedIR)->getName().str();
   if (any_isa<const LazyCallGraph::SCC *>(WrappedIR))
     return any_cast<const LazyCallGraph::SCC *>(WrappedIR)->getName();
   return "<UNKNOWN>";
 }
 /// Define a custom matcher for objects which support a 'getName' method.
 ///
 /// LLVM often has IR objects or analysis objects which expose a name
 /// and in tests it is convenient to match these by name for readability.
EXPECT_CALL(*this,
                 runBeforeAnalysis(Not(HasNameRegex("Mock")), HasName(IRName)))
         .Times(AnyNumber());
     EXPECT_CALL(*this,
                 runAfterAnalysis(Not(HasNameRegex("Mock")), HasName(IRName)))
         .Times(AnyNumber());
   }
 };
 
+template <typename IRUnitT>
+using ExtraMockPassHandle =
+    std::conditional_t<std::is_same<IRUnitT, Loop>::value,
+                       MockPassHandle<LoopNest>, MockPassHandle<IRUnitT>>;
+
 template <typename PassManagerT> class PassBuilderCallbacksTest;
 
 /// This test fixture is shared between all the actual tests below and
 /// takes care of setting up appropriate defaults.
 ///
 /// The template specialization serves to extract the IRUnit and AM types from
 /// the given PassManagerT.
 template <typename TestIRUnitT, typename... ExtraPassArgTs,
protected:
   PassBuilder PB;
   ModulePassManager PM;
   LoopAnalysisManager LAM;
   FunctionAnalysisManager FAM;
   CGSCCAnalysisManager CGAM;
   ModuleAnalysisManager AM;
 
   MockPassHandle<IRUnitT> PassHandle;
+  ExtraMockPassHandle<IRUnitT> ExtraPassHandle;
+
   MockAnalysisHandle<IRUnitT> AnalysisHandle;
 
   static PreservedAnalyses getAnalysisResult(IRUnitT &U, AnalysisManagerT &AM,
                                              ExtraAnalysisArgTs &&... Args) {
     (void)AM.template getResult<AnalysisT>(
         U, std::forward<ExtraAnalysisArgTs>(Args)...);
     return PreservedAnalyses::all();
   }
PB.registerPipelineParsingCallback(
           /// require<test-analysis> and invalidate<test-analysis> for our
           /// analysis mock handle
           if (parseAnalysisUtilityPasses<AnalysisT>("test-analysis", Name, PM))
             return true;
 
           /// Parse the name of our pass mock handle
           if (Name == "test-transform") {
             PM.addPass(PassHandle.getPass());
+            if (std::is_same<IRUnitT, Loop>::value)
+              PM.addPass(ExtraPassHandle.getPass());
             return true;
           }
           return false;
         });
 
     /// Register builtin analyses and cross-register the analysis proxies
     PB.registerModuleAnalyses(AM);
     PB.registerCGSCCAnalyses(CGAM);
ASSERT_THAT_ERROR(PB.parsePassPipeline(PM, PipelineText), Succeeded())
       << "Pipeline was: " << PipelineText;
   PM.run(*M, AM);
 }
 
 TEST_F(LoopCallbacksTest, Passes) {
   EXPECT_CALL(AnalysisHandle, run(HasName("loop"), _, _));
   EXPECT_CALL(PassHandle, run(HasName("loop"), _, _, _))
       .WillOnce(WithArgs<0, 1, 2>(Invoke(getAnalysisResult)));
+  EXPECT_CALL(ExtraPassHandle, run(HasName("loop"), _, _, _));
 
   StringRef PipelineText = "test-transform";
   ASSERT_THAT_ERROR(PB.parsePassPipeline(PM, PipelineText), Succeeded())
       << "Pipeline was: " << PipelineText;
   PM.run(*M, AM);
 }
 
 TEST_F(LoopCallbacksTest, InstrumentedPasses) {
   CallbacksHandle.registerPassInstrumentation();
   // Non-mock instrumentation not specifically mentioned below can be ignored.
   CallbacksHandle.ignoreNonMockPassInstrumentation("<string>");
   CallbacksHandle.ignoreNonMockPassInstrumentation("foo");
   CallbacksHandle.ignoreNonMockPassInstrumentation("loop");
 
   EXPECT_CALL(AnalysisHandle, run(HasName("loop"), _, _));
   EXPECT_CALL(PassHandle, run(HasName("loop"), _, _, _))
       .WillOnce(WithArgs<0, 1, 2>(Invoke(getAnalysisResult)));
+  EXPECT_CALL(ExtraPassHandle, run(HasName("loop"), _, _, _));
 
   // PassInstrumentation calls should happen in-sequence, in the same order
   // as passes/analyses are scheduled.
   ::testing::Sequence PISequence;
   EXPECT_CALL(CallbacksHandle,
               runBeforePass(HasNameRegex("MockPassHandle"), HasName("loop")))
       .InSequence(PISequence);
   EXPECT_CALL(
       CallbacksHandle,
       runBeforeNonSkippedPass(HasNameRegex("MockPassHandle"), HasName("loop")))
       .InSequence(PISequence);
   EXPECT_CALL(
       CallbacksHandle,
       runBeforeAnalysis(HasNameRegex("MockAnalysisHandle"), HasName("loop")))
       .InSequence(PISequence);
   EXPECT_CALL(
       CallbacksHandle,
       runAfterAnalysis(HasNameRegex("MockAnalysisHandle"), HasName("loop")))
       .InSequence(PISequence);
   EXPECT_CALL(CallbacksHandle,
               runAfterPass(HasNameRegex("MockPassHandle"), HasName("loop"), _))
       .InSequence(PISequence);
 
+  EXPECT_CALL(CallbacksHandle,
+              runBeforePass(HasNameRegex("MockPassHandle<.*LoopNest>"),
+                            HasName("loop")))
+      .InSequence(PISequence);
+  EXPECT_CALL(CallbacksHandle,
+              runBeforeNonSkippedPass(
+                  HasNameRegex("MockPassHandle<.*LoopNest>"), HasName("loop")))
+      .InSequence(PISequence);
+  EXPECT_CALL(CallbacksHandle,
+              runAfterPass(HasNameRegex("MockPassHandle<.*LoopNest>"),
+                           HasName("loop"), _))
+      .InSequence(PISequence);
+
   // Our mock pass does not invalidate IR.
   EXPECT_CALL(CallbacksHandle,
               runAfterPassInvalidated(HasNameRegex("MockPassHandle"), _))
       .Times(0);
 
   // No passes are skipped, so there should be no calls to
   // runBeforeSkippedPass().
   EXPECT_CALL(
EXPECT_CALL(CallbacksHandle,
       .Times(0);
 
   StringRef PipelineText = "test-transform";
   ASSERT_THAT_ERROR(PB.parsePassPipeline(PM, PipelineText), Succeeded())
       << "Pipeline was: " << PipelineText;
   PM.run(*M, AM);
 }
 
+TEST_F(LoopCallbacksTest, InstrumentedInvalidatingLoopNestPasses) {
+  CallbacksHandle.registerPassInstrumentation();
+  // Non-mock instrumentation not specifically mentioned below can be ignored.
+  CallbacksHandle.ignoreNonMockPassInstrumentation("<string>");
+  CallbacksHandle.ignoreNonMockPassInstrumentation("foo");
+  CallbacksHandle.ignoreNonMockPassInstrumentation("loop");
+
+  EXPECT_CALL(AnalysisHandle, run(HasName("loop"), _, _));
+  EXPECT_CALL(PassHandle, run(HasName("loop"), _, _, _))
+      .WillOnce(WithArgs<0, 1, 2>(Invoke(getAnalysisResult)));
+  EXPECT_CALL(ExtraPassHandle, run(HasName("loop"), _, _, _))
+      .WillOnce(DoAll(Invoke(ExtraPassHandle.invalidateLoopNest),
+                      Invoke([&](LoopNest &, LoopAnalysisManager &,
+                                 LoopStandardAnalysisResults &, LPMUpdater &) {
+                        return PreservedAnalyses::all();
+                      })));
+
+  // PassInstrumentation calls should happen in-sequence, in the same order
+  // as passes/analyses are scheduled.
+  ::testing::Sequence PISequence;
+  EXPECT_CALL(CallbacksHandle,
+              runBeforePass(HasNameRegex("MockPassHandle"), HasName("loop")))
+      .InSequence(PISequence);
+  EXPECT_CALL(
+      CallbacksHandle,
+      runBeforeNonSkippedPass(HasNameRegex("MockPassHandle"), HasName("loop")))
+      .InSequence(PISequence);
+  EXPECT_CALL(
+      CallbacksHandle,
+      runBeforeAnalysis(HasNameRegex("MockAnalysisHandle"), HasName("loop")))
+      .InSequence(PISequence);
+  EXPECT_CALL(
+      CallbacksHandle,
+      runAfterAnalysis(HasNameRegex("MockAnalysisHandle"), HasName("loop")))
+      .InSequence(PISequence);
+  EXPECT_CALL(CallbacksHandle,
+              runAfterPass(HasNameRegex("MockPassHandle"), HasName("loop"), _))
+      .InSequence(PISequence);
+
+  EXPECT_CALL(CallbacksHandle,
+              runBeforePass(HasNameRegex("MockPassHandle<.*LoopNest>"),
+                            HasName("loop")))
+      .InSequence(PISequence);
+  EXPECT_CALL(CallbacksHandle,
+              runBeforeNonSkippedPass(
+                  HasNameRegex("MockPassHandle<.*LoopNest>"), HasName("loop")))
+      .InSequence(PISequence);
+  EXPECT_CALL(
+      CallbacksHandle,
+      runAfterPassInvalidated(HasNameRegex("MockPassHandle<.*LoopNest>"), _))
+      .InSequence(PISequence);
+
+  EXPECT_CALL(CallbacksHandle,
+              runAfterPassInvalidated(HasNameRegex("^PassManager"), _))
+      .InSequence(PISequence);
+
+  // Our mock pass invalidates IR, thus normal runAfterPass is never called.
+  EXPECT_CALL(CallbacksHandle, runAfterPassInvalidated(
+                                   HasNameRegex("MockPassHandle<.*Loop>"), _))
+      .Times(0);
+  EXPECT_CALL(CallbacksHandle,
+              runAfterPass(HasNameRegex("MockPassHandle<.*LoopNest>"),
+                           HasName("loop"), _))
+      .Times(0);
+
+  StringRef PipelineText = "test-transform";
+  ASSERT_THAT_ERROR(PB.parsePassPipeline(PM, PipelineText), Succeeded())
+      << "Pipeline was: " << PipelineText;
+  PM.run(*M, AM);
+}
+
 TEST_F(LoopCallbacksTest, InstrumentedSkippedPasses) {
   CallbacksHandle.registerPassInstrumentation();
   // Non-mock instrumentation run here can safely be ignored.
   CallbacksHandle.ignoreNonMockPassInstrumentation("<string>");
   CallbacksHandle.ignoreNonMockPassInstrumentation("foo");
   CallbacksHandle.ignoreNonMockPassInstrumentation("loop");
 
   // Skip the pass by returning false.
+  EXPECT_CALL(
+      CallbacksHandle,
+      runBeforePass(HasNameRegex("MockPassHandle<.*Loop>"), HasName("loop")))
+      .WillOnce(Return(false));
+
   EXPECT_CALL(CallbacksHandle,
-              runBeforePass(HasNameRegex("MockPassHandle"), HasName("loop")))
+              runBeforeSkippedPass(HasNameRegex("MockPassHandle<.*Loop>"),
+                                   HasName("loop")))
+      .Times(1);
+
+  EXPECT_CALL(CallbacksHandle,
+              runBeforePass(HasNameRegex("MockPassHandle<.*LoopNest>"),
+                            HasName("loop")))
       .WillOnce(Return(false));
 
-  EXPECT_CALL(
-      CallbacksHandle,
-      runBeforeSkippedPass(HasNameRegex("MockPassHandle"), HasName("loop")))
+  EXPECT_CALL(CallbacksHandle,
+              runBeforeSkippedPass(HasNameRegex("MockPassHandle<.*LoopNest>"),
+                                   HasName("loop")))
       .Times(1);
 
   EXPECT_CALL(AnalysisHandle, run(HasName("loop"), _, _)).Times(0);
   EXPECT_CALL(PassHandle, run(HasName("loop"), _, _, _)).Times(0);
+  EXPECT_CALL(ExtraPassHandle, run(HasName("loop"), _, _, _)).Times(0);
 
   // As the pass is skipped there is no afterPass, beforeAnalysis/afterAnalysis
   // as well.
-  EXPECT_CALL(CallbacksHandle,
-              runBeforeNonSkippedPass(HasNameRegex("MockPassHandle"), _))
+  EXPECT_CALL(CallbacksHandle, runBeforeNonSkippedPass(
+                                   HasNameRegex("MockPassHandle<.*Loop>"), _))
       .Times(0);
   EXPECT_CALL(CallbacksHandle,
-              runAfterPass(HasNameRegex("MockPassHandle"), _, _))
+              runAfterPass(HasNameRegex("MockPassHandle<.*Loop>"), _, _))
+      .Times(0);
+  EXPECT_CALL(CallbacksHandle, runAfterPassInvalidated(
+                                   HasNameRegex("MockPassHandle<.*Loop>"), _))
+      .Times(0);
+  EXPECT_CALL(
+      CallbacksHandle,
+      runBeforeNonSkippedPass(HasNameRegex("MockPassHandle<.*LoopNest>"), _))
       .Times(0);
   EXPECT_CALL(CallbacksHandle,
-              runAfterPassInvalidated(HasNameRegex("MockPassHandle"), _))
+              runAfterPass(HasNameRegex("MockPassHandle<.*LoopNest>"), _, _))
+      .Times(0);
+  EXPECT_CALL(
+      CallbacksHandle,
+      runAfterPassInvalidated(HasNameRegex("MockPassHandle<.*LoopNest>"), _))
       .Times(0);
   EXPECT_CALL(CallbacksHandle,
               runBeforeAnalysis(HasNameRegex("MockAnalysisHandle"), _))
       .Times(0);
   EXPECT_CALL(CallbacksHandle,
               runAfterAnalysis(HasNameRegex("MockAnalysisHandle"), _))
       .Times(0);
 

llvm/unittests/Transforms/Scalar/LoopPassManagerTest.cpp

struct MockLoopPassHandle
     : MockPassHandleBase<MockLoopPassHandle, Loop, LoopAnalysisManager,
                          LoopStandardAnalysisResults &, LPMUpdater &> {
   MOCK_METHOD4(run,
                PreservedAnalyses(Loop &, LoopAnalysisManager &,
                                  LoopStandardAnalysisResults &, LPMUpdater &));
   MockLoopPassHandle() { setDefaults(); }
 };
 
+struct MockLoopNestPassHandle
+    : MockPassHandleBase<MockLoopNestPassHandle, LoopNest, LoopAnalysisManager,
+                         LoopStandardAnalysisResults &, LPMUpdater &> {
+  MOCK_METHOD4(run,
+               PreservedAnalyses(LoopNest &, LoopAnalysisManager &,
+                                 LoopStandardAnalysisResults &, LPMUpdater &));
+
+  MockLoopNestPassHandle() { setDefaults(); }
+};
+
 struct MockFunctionPassHandle
     : MockPassHandleBase<MockFunctionPassHandle, Function> {
   MOCK_METHOD2(run, PreservedAnalyses(Function &, FunctionAnalysisManager &));
 
   MockFunctionPassHandle() { setDefaults(); }
 };
 
 struct MockModulePassHandle : MockPassHandleBase<MockModulePassHandle, Module> {
protected:
   std::unique_ptr<Module> M;
 
   LoopAnalysisManager LAM;
   FunctionAnalysisManager FAM;
   ModuleAnalysisManager MAM;
 
   MockLoopAnalysisHandle MLAHandle;
   MockLoopPassHandle MLPHandle;
+  MockLoopNestPassHandle MLNPHandle;
   MockFunctionPassHandle MFPHandle;
   MockModulePassHandle MMPHandle;
 
   static PreservedAnalyses
   getLoopAnalysisResult(Loop &L, LoopAnalysisManager &AM,
                         LoopStandardAnalysisResults &AR, LPMUpdater &) {
     (void)AM.getResult<MockLoopAnalysisHandle::Analysis>(L, AR);
     return PreservedAnalyses::all();
EXPECT_CALL(MLPHandle, run(HasName("loop.0"), _, _, _))
             return PreservedAnalyses::all();
           }));
 
   // Add the function pass pipeline now that it is fully built up and run it
   // over the module's one function.
   MPM.addPass(createModuleToFunctionPassAdaptor(std::move(FPM)));
   MPM.run(*M, MAM);
 }
+
+TEST_F(LoopPassManagerTest, HandleLoopNestPass) {
+  ::testing::InSequence MakeExpectationsSequenced;
+
+  EXPECT_CALL(MLPHandle, run(HasName("loop.0.0"), _, _, _)).Times(2);
+  EXPECT_CALL(MLPHandle, run(HasName("loop.0.1"), _, _, _)).Times(2);
+  EXPECT_CALL(MLPHandle, run(HasName("loop.0"), _, _, _));
+  EXPECT_CALL(MLNPHandle, run(HasName("loop.0"), _, _, _));
+  EXPECT_CALL(MLPHandle, run(HasName("loop.0"), _, _, _));
+  EXPECT_CALL(MLNPHandle, run(HasName("loop.0"), _, _, _));
+  EXPECT_CALL(MLPHandle, run(HasName("loop.g.0"), _, _, _));
+  EXPECT_CALL(MLNPHandle, run(HasName("loop.g.0"), _, _, _));
+  EXPECT_CALL(MLPHandle, run(HasName("loop.g.0"), _, _, _));
+  EXPECT_CALL(MLNPHandle, run(HasName("loop.g.0"), _, _, _));
+
+  LoopPassManager LPM(true);
+  LPM.addPass(MLPHandle.getPass());
+  LPM.addPass(MLNPHandle.getPass());
+  LPM.addPass(MLPHandle.getPass());
+  LPM.addPass(MLNPHandle.getPass());
+
+  ModulePassManager MPM(true);
+  MPM.addPass(createModuleToFunctionPassAdaptor(
+      createFunctionToLoopPassAdaptor(std::move(LPM))));
+  MPM.run(*M, MAM);
 }
+
+} // namespace