[LoopIdiom] Introduce LoopNestIdiomRecognize as an alternative

[eopXD]

Summary

This LoopNestPass-version LIR is currently identical to the LoopPass-version
LIR. The new added pass is added into PassRegistry as "loop-nest-idiom".

The LoopNest structure makes optimizations that insert runtime checks for the
nested loop more easy to apply. D104636 is an example of a use case to this
newly created pass.

[fhahn]

Following patches will utilize the LoopNest structure for more efficient optimization.

Would it be possible to provide some data on the efficiency improvements?

[eopXD]

In D104179#2815155, @fhahn wrote:

Following patches will utilize the LoopNest structure for more efficient optimization.

Would it be possible to provide some data on the efficiency improvements?

Hi @fhahn ,
For example, given a perfectly nested loop-pair and we have matched a store operation that we can hoist to the inner-loop's header as strided store operations,
we can further check on the outer-loop, see if the outer-loop SCEV maintains the continuous memory access and hoist the operation one more loop outwards.

[fhahn]

In D104179#2815159, @eopXD wrote:

In D104179#2815155, @fhahn wrote:

Following patches will utilize the LoopNest structure for more efficient optimization.

Would it be possible to provide some data on the efficiency improvements?

Hi @fhahn ,
For example, given a perfectly nested loop-pair and we have matched a store operation that we can hoist to the inner-loop's header as strided store operations,
we can further check on the outer-loop, see if the outer-loop SCEV maintains the continuous memory access and hoist the operation one more loop outwards.

Ah I see, so you are planning to add additional optimizations based on LoopNest? If so, I think it would also be good to share a patch that makes implements such an additional optimization so there's a clear path towards concrete improvements and it would also show why using LoopNest is needed/beneficial.

[eopXD]

fix some clang-format

[eopXD]

fix some clang-format

[eopXD]

In D104179#2815199, @fhahn wrote:

In D104179#2815159, @eopXD wrote:

In D104179#2815155, @fhahn wrote:

Following patches will utilize the LoopNest structure for more efficient optimization.

Would it be possible to provide some data on the efficiency improvements?

Hi @fhahn ,
For example, given a perfectly nested loop-pair and we have matched a store operation that we can hoist to the inner-loop's header as strided store operations,
we can further check on the outer-loop, see if the outer-loop SCEV maintains the continuous memory access and hoist the operation one more loop outwards.

Ah I see, so you are planning to add additional optimizations based on LoopNest? If so, I think it would also be good to share a patch that makes implements such an additional optimization so there's a clear path towards concrete improvements and it would also show why using LoopNest is needed/beneficial.

Sure, I would follow up with the optimization that utilizes LoopNest and state out the benefits/improvement of the optimization.
Thank you for the comments.

[qianzhen]

When the loop idiom transformation processes a memset instruction in a loop, currently it only handles the memset with a compile-time constant size. The motivation of this work is to relax this limitation, so that a memset with a variable size in a loop may still be processed and promoted to a larger memset if it passes all the eligibility checks. Performance-wise, promoting the memset in a loop to a larger memset reduces the number of calls to memset; hence reducing the overall call overhead.
A similar technique may also apply to the memcpy with a variable size in a loop.

[Whitney]

llvm/lib/Transforms/Scalar/LoopIdiomRecognize.cpp

368

Function *F = LN.getParent();
const auto *DL = &F->getModule()->getDataLayout();
OptimizationRemarkEmitter ORE(F);

[eopXD]

Address comments.

[eopXD]

Rebase to main.

[eopXD]

Rebase and update test case.

[eopXD]

llvm/test/Transforms/LoopIdiom/memset-runtime-64bit.ll

288–289

The reason the IR changed is that the testing pass pipeline is adjusted. This patch is still an NFC.

llvm/include/llvm/Transforms/Scalar/LoopIdiomRecognize.h

 // performance win.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_TRANSFORMS_SCALAR_LOOPIDIOMRECOGNIZE_H
 #define LLVM_TRANSFORMS_SCALAR_LOOPIDIOMRECOGNIZE_H
 
 #include "llvm/Analysis/LoopAnalysisManager.h"
+#include "llvm/Analysis/LoopNestAnalysis.h"
 #include "llvm/IR/PassManager.h"
 
 namespace llvm {
 
 class Loop;
 class LPMUpdater;
 
 /// Options to disable Loop Idiom Recognize, which can be shared with other

 
 /// Performs Loop Idiom Recognize Pass.
 class LoopIdiomRecognizePass : public PassInfoMixin<LoopIdiomRecognizePass> {
 public:
   PreservedAnalyses run(Loop &L, LoopAnalysisManager &AM,
                         LoopStandardAnalysisResults &AR, LPMUpdater &U);
 };
 
+// NFC LoopNestPass with regards to the current LoopPass-LoopIdiomRecognize
+class LoopNestIdiomRecognizePass
+    : public PassInfoMixin<LoopNestIdiomRecognizePass> {
+public:
+  PreservedAnalyses run(LoopNest &LN, LoopAnalysisManager &LAM,
+                        LoopStandardAnalysisResults &AR, LPMUpdater &U);
+};
+
 } // end namespace llvm
 
 #endif // LLVM_TRANSFORMS_SCALAR_LOOPIDIOMRECOGNIZE_H

llvm/lib/Passes/PassRegistry.def

 #ifndef LOOPNEST_PASS
 #define LOOPNEST_PASS(NAME, CREATE_PASS)
 #endif
 LOOPNEST_PASS("lnicm", LNICMPass())
 LOOPNEST_PASS("loop-flatten", LoopFlattenPass())
 LOOPNEST_PASS("loop-interchange", LoopInterchangePass())
 LOOPNEST_PASS("loop-unroll-and-jam", LoopUnrollAndJamPass())
 LOOPNEST_PASS("no-op-loopnest", NoOpLoopNestPass())
+LOOPNEST_PASS("loop-nest-idiom", LoopNestIdiomRecognizePass())
 #undef LOOPNEST_PASS
 
 #ifndef LOOP_ANALYSIS
 #define LOOP_ANALYSIS(NAME, CREATE_PASS)
 #endif
 LOOP_ANALYSIS("no-op-loop", NoOpLoopAnalysis())
 LOOP_ANALYSIS("access-info", LoopAccessAnalysis())
 LOOP_ANALYSIS("ddg", DDGAnalysis())

llvm/lib/Transforms/Scalar/LoopIdiomRecognize.cpp

static cl::opt<bool> UseLIRCodeSizeHeurs(
     cl::desc("Use loop idiom recognition code size heuristics when compiling"
              "with -Os/-Oz"),
     cl::init(true), cl::Hidden);
 
 namespace {
 
 class LoopIdiomRecognize {
   Loop *CurLoop = nullptr;
+  LoopNest *LN;
   AliasAnalysis *AA;
   DominatorTree *DT;
   LoopInfo *LI;
   ScalarEvolution *SE;
   TargetLibraryInfo *TLI;
   const TargetTransformInfo *TTI;
   const DataLayout *DL;
   OptimizationRemarkEmitter &ORE;
   bool ApplyCodeSizeHeuristics;
   std::unique_ptr<MemorySSAUpdater> MSSAU;
 
 public:
   explicit LoopIdiomRecognize(AliasAnalysis *AA, DominatorTree *DT,
-                              LoopInfo *LI, ScalarEvolution *SE,
+                              LoopInfo *LI, LoopNest *LN, ScalarEvolution *SE,
                               TargetLibraryInfo *TLI,
                               const TargetTransformInfo *TTI, MemorySSA *MSSA,
                               const DataLayout *DL,
                               OptimizationRemarkEmitter &ORE)
-      : AA(AA), DT(DT), LI(LI), SE(SE), TLI(TLI), TTI(TTI), DL(DL), ORE(ORE) {
+      : LN(LN), AA(AA), DT(DT), LI(LI), SE(SE), TLI(TLI), TTI(TTI), DL(DL),
+        ORE(ORE) {
     if (MSSA)
       MSSAU = std::make_unique<MemorySSAUpdater>(MSSA);
   }
 
+  bool runOnLoopNest();
   bool runOnLoop(Loop *L);
 
 private:
   using StoreList = SmallVector<StoreInst *, 8>;
   using StoreListMap = MapVector<Value *, StoreList>;
 
   StoreListMap StoreRefsForMemset;
   StoreListMap StoreRefsForMemsetPattern;
bool runOnLoop(Loop *L, LPPassManager &LPM) override {
     if (MSSAAnalysis)
       MSSA = &MSSAAnalysis->getMSSA();
 
     // For the old PM, we can't use OptimizationRemarkEmitter as an analysis
     // pass.  Function analyses need to be preserved across loop transformations
     // but ORE cannot be preserved (see comment before the pass definition).
     OptimizationRemarkEmitter ORE(L->getHeader()->getParent());
 
-    LoopIdiomRecognize LIR(AA, DT, LI, SE, TLI, TTI, MSSA, DL, ORE);
+    LoopIdiomRecognize LIR(AA, DT, LI, nullptr, SE, TLI, TTI, MSSA, DL, ORE);
     return LIR.runOnLoop(L);
   }
 
   /// This transformation requires natural loop information & requires that
   /// loop preheaders be inserted into the CFG.
   void getAnalysisUsage(AnalysisUsage &AU) const override {
     AU.addRequired<TargetLibraryInfoWrapperPass>();
     AU.addRequired<TargetTransformInfoWrapperPass>();
PreservedAnalyses LoopIdiomRecognizePass::run(Loop &L, LoopAnalysisManager &AM,
 
   const auto *DL = &L.getHeader()->getModule()->getDataLayout();
 
   // For the new PM, we also can't use OptimizationRemarkEmitter as an analysis
   // pass.  Function analyses need to be preserved across loop transformations
   // but ORE cannot be preserved (see comment before the pass definition).
   OptimizationRemarkEmitter ORE(L.getHeader()->getParent());
 
-  LoopIdiomRecognize LIR(&AR.AA, &AR.DT, &AR.LI, &AR.SE, &AR.TLI, &AR.TTI,
-                         AR.MSSA, DL, ORE);
+  LoopIdiomRecognize LIR(&AR.AA, &AR.DT, &AR.LI, nullptr, &AR.SE, &AR.TLI,
+                         &AR.TTI, AR.MSSA, DL, ORE);
   if (!LIR.runOnLoop(&L))
     return PreservedAnalyses::all();
 
   auto PA = getLoopPassPreservedAnalyses();
   if (AR.MSSA)
     PA.preserve<MemorySSAAnalysis>();
   return PA;
 }
 
+PreservedAnalyses
+LoopNestIdiomRecognizePass::run(LoopNest &LN, LoopAnalysisManager &LAM,
+                                LoopStandardAnalysisResults &AR,
+                                LPMUpdater &U) {
+  if (DisableLIRP::All)
+    return PreservedAnalyses::all();
+
+  Function *F = LN.getParent();
+  const auto *DL = &F->front().getModule()->getDataLayout();
+  OptimizationRemarkEmitter ORE(F);
+
+  LoopIdiomRecognize LIR(&AR.AA, &AR.DT, &AR.LI, &LN, &AR.SE, &AR.TLI, &AR.TTI,
+                         AR.MSSA, DL, ORE);
+
+  if (!LIR.runOnLoopNest())
+    return PreservedAnalyses::all();
+
+  auto PA = getLoopPassPreservedAnalyses();
+  if (AR.MSSA)
+    PA.preserve<MemorySSAAnalysis>();
+
+  return PA;
+}
+
 INITIALIZE_PASS_BEGIN(LoopIdiomRecognizeLegacyPass, "loop-idiom",
                       "Recognize loop idioms", false, false)
 INITIALIZE_PASS_DEPENDENCY(LoopPass)
 INITIALIZE_PASS_DEPENDENCY(TargetLibraryInfoWrapperPass)
 INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
 INITIALIZE_PASS_END(LoopIdiomRecognizeLegacyPass, "loop-idiom",
                     "Recognize loop idioms", false, false)
 
 Pass *llvm::createLoopIdiomPass() { return new LoopIdiomRecognizeLegacyPass(); }
 
 static void deleteDeadInstruction(Instruction *I) {
   I->replaceAllUsesWith(UndefValue::get(I->getType()));
   I->eraseFromParent();
 }
 
 //===----------------------------------------------------------------------===//
 //
 //          Implementation of LoopIdiomRecognize
 //
 //===----------------------------------------------------------------------===//
 
+bool LoopIdiomRecognize::runOnLoopNest() {
+  assert(LN != nullptr && "Should not call this function is LoopNest isn't"
+                          "provided in ctor");
+
+  // Build worklist that contains all loops inside the LoopNest
+  SmallVector<Loop *, 8> WorkList;
+  for (Loop *L : LN->getLoops()) {
+    WorkList.push_back(L);
+  }
+
+  std::reverse(WorkList.begin(), WorkList.end());
+
+  // Iterate from innermost to outermost
+  bool Changed = false;
+  for (Loop *L : WorkList) {
+    Changed |= runOnLoop(L);
+  }
+
+  return Changed;
+}
+
 bool LoopIdiomRecognize::runOnLoop(Loop *L) {
   CurLoop = L;
   // If the loop could not be converted to canonical form, it must have an
   // indirectbr in it, just give up.
   if (!L->getLoopPreheader())
     return false;
 
   // Disable loop idiom recognition if the function's name is a common idiom.

llvm/test/Transforms/LoopIdiom/AMDGPU/popcnt.ll

 ; RUN: opt -loop-idiom -mtriple=amdgcn-- -S < %s | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S -mtriple=amdgcn-- < %s | FileCheck %s
 
 ; Mostly copied from x86 version.
 
 ;To recognize this pattern:
 ;int popcount(unsigned long long a) {
 ;    int c = 0;
 ;    while (a) {
 ;        c++;

llvm/test/Transforms/LoopIdiom/ARM/ctlz.ll

 ; RUN: opt -loop-idiom -mtriple=armv7a < %s -S | FileCheck -check-prefix=LZCNT --check-prefix=ALL %s
 ; RUN: opt -loop-idiom -mtriple=armv4t < %s -S | FileCheck -check-prefix=NOLZCNT --check-prefix=ALL %s
+; RUN: opt -passes="loop-nest-idiom" -S -mtriple=armv7a < %s | FileCheck %s -check-prefix=LZCNT --check-prefix=ALL
+; RUN: opt -passes="loop-nest-idiom" -S -mtriple=armv7t < %s | FileCheck %s -check-prefix=NOLZCNT --check-prefix=ALL
 
 ; Recognize CTLZ builtin pattern.
 ; Here we'll just convert loop to countable,
 ; so do not insert builtin if CPU do not support CTLZ
 ;
 ; int ctlz_and_other(int n, char *a)
 ; {
 ;   n = n >= 0 ? n : -n;

llvm/test/Transforms/LoopIdiom/RISCV/popcnt.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -loop-idiom -mtriple=riscv32 -mattr=+experimental-zbb -S < %s | FileCheck %s --check-prefixes=CPOP
 ; RUN: opt -loop-idiom -mtriple=riscv64 -mattr=+experimental-zbb -S < %s | FileCheck %s --check-prefixes=CPOP
 ; RUN: opt -loop-idiom -mtriple=riscv32 -S < %s | FileCheck %s --check-prefixes=NOCPOP
 ; RUN: opt -loop-idiom -mtriple=riscv64 -S < %s | FileCheck %s --check-prefixes=NOCPOP
+; RUN: opt -passes="loop-nest-idiom" -mtriple=riscv32 -mattr=+experimental-zbb -S < %s | FileCheck %s --check-prefixes=CPOP
+; RUN: opt -passes="loop-nest-idiom" -mtriple=riscv64 -mattr=+experimental-zbb -S < %s | FileCheck %s --check-prefixes=CPOP
+; RUN: opt -passes="loop-nest-idiom" -mtriple=riscv32 -S < %s | FileCheck %s --check-prefixes=NOCPOP
+; RUN: opt -passes="loop-nest-idiom" -mtriple=riscv64 -S < %s | FileCheck %s --check-prefixes=NOCPOP
 
 ; Mostly copied from AMDGPU version.
 
 ;To recognize this pattern:
 ;int popcount(unsigned long long a) {
 ;    int c = 0;
 ;    while (a) {
 ;        c++;

llvm/test/Transforms/LoopIdiom/X86/arithmetic-right-shift-until-zero.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -loop-idiom -mtriple=x86_64 -mcpu=core-avx2 < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S -mtriple=x86_64 -mcpu=core-avx2 < %s | FileCheck %s
 
 declare void @escape_inner(i8, i8, i8, i1, i8)
 declare void @escape_outer(i8, i8, i8, i1, i8)
 
 declare i8 @gen.i8()
 
 ; Most basic pattern; Note that iff the shift amount is offset, said offsetting
 ; must not cause an overflow, but `add nsw` is fine.

llvm/test/Transforms/LoopIdiom/X86/ctlz.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -loop-idiom -mtriple=x86_64 -mcpu=core-avx2 < %s -S | FileCheck %s -check-prefixes=ALL,LZCNT
 ; RUN: opt -loop-idiom -mtriple=x86_64 -mcpu=corei7 < %s -S | FileCheck %s -check-prefixes=ALL,NOLZCNT
+; RUN: opt -passes="loop-nest-idiom" -S -mtriple=x86_64 -mcpu=core-avx2 < %s | FileCheck %s -check-prefixes=ALL,LZCNT
+; RUN: opt -passes="loop-nest-idiom" -S -mtriple=x86_64 -mcpu=corei7 < %s | FileCheck %s -check-prefixes=ALL,NOLZCNT
 
 ; Recognize CTLZ builtin pattern.
 ; Here we'll just convert loop to countable,
 ; so do not insert builtin if CPU do not support CTLZ
 ;
 ; int ctlz_and_other(int n, char *a)
 ; {
 ;   n = n >= 0 ? n : -n;

llvm/test/Transforms/LoopIdiom/X86/cttz.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -loop-idiom -mtriple=x86_64 -mcpu=core-avx2 < %s -S | FileCheck --check-prefix=ALL %s
 ; RUN: opt -loop-idiom -mtriple=x86_64 -mcpu=corei7 < %s -S | FileCheck --check-prefix=ALL %s
+; RUN: opt -passes="loop-nest-idiom" -S -mtriple=x86_64 -mcpu=core-avx2 < %s | FileCheck %s --check-prefix=ALL
+; RUN: opt -passes="loop-nest-idiom" -S -mtriple=x86_64 -mcpu=corei7 < %s | FileCheck %s --check-prefix=ALL
 
 ; Recognize CTTZ builtin pattern.
 ; Here it will replace the loop -
 ; assume builtin is always profitable.
 ;
 ; int cttz_zero_check(int n)
 ; {
 ;   int i = 0;

llvm/test/Transforms/LoopIdiom/X86/left-shift-until-bittest.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -debugify -loop-idiom -mtriple=x86_64 -mcpu=core-avx2 < %s -S | FileCheck --check-prefixes=ALL,LZCNT %s
 ; RUN: opt -debugify -loop-idiom -mtriple=x86_64 -mcpu=corei7 < %s -S | FileCheck  --check-prefixes=ALL,NOLZCNT %s
+; RUN: opt -passes="loop-nest-idiom" -enable-debugify -debug-pass-manager -mtriple=x86_64 -mcpu=core-avx2 < %s -S | FileCheck --check-prefixes=ALL,LZCNT %s
+; RUN: opt -passes="loop-nest-idiom" -enable-debugify -debug-pass-manager -mtriple=x86_64 -mcpu=corei7 < %s -S | FileCheck  --check-prefixes=ALL,NOLZCNT %s
 
 declare i32 @gen32()
 declare void @use32(i32)
 declare void @use1(i1)
 declare void @external_side_effect()
 
 ; The patterns here are all have the same base form:
 ;   while (!(x & (1U << bit)))

llvm/test/Transforms/LoopIdiom/X86/left-shift-until-zero.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -loop-idiom -mtriple=x86_64 -mcpu=core-avx2 < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S -mtriple=x86_64 -mcpu=core-avx2 < %s | FileCheck %s
 
 declare void @escape_inner(i8, i8, i8, i1, i8)
 declare void @escape_outer(i8, i8, i8, i1, i8)
 
 declare i8 @gen.i8()
 
 ; Most basic pattern; Note that iff the shift amount is offset, said offsetting
 ; must not cause an overflow, but `add nsw` is fine.

llvm/test/Transforms/LoopIdiom/X86/logical-right-shift-until-zero-cost.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -loop-idiom -mtriple=x86_64 -mcpu=corei7 < %s -S | FileCheck %s
-
+; RUN: opt -passes="loop-nest-idiom" -S -mtriple=x86_64 -mcpu=corei7 < %s | FileCheck %s
 declare void @escape_inner(i8, i8, i8, i1, i8)
 declare void @escape_outer(i8, i8, i8, i1, i8)
 
 ; On corei7, lzcount is not cheap, so we don't transform.
 define i8 @n(i8 %val, i8 %start, i8 %extraoffset) {
 ; CHECK-LABEL: @n(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP:%.*]]

llvm/test/Transforms/LoopIdiom/X86/logical-right-shift-until-zero-debuginfo.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -debugify -loop-idiom -mtriple=x86_64 -mcpu=core-avx2 < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -enable-debugify -debug-pass-manager -mtriple=x86_64 -mcpu=core-avx2 < %s -S | FileCheck %s
 
 declare void @escape_inner(i8, i8, i8, i1, i8)
 declare void @escape_outer(i8, i8, i8, i1, i8)
 
 define i8 @p(i8 %val, i8 %start, i8 %extraoffset) {
 ; CHECK-LABEL: @p(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[VAL_NUMLEADINGZEROS:%.*]] = call i8 @llvm.ctlz.i8(i8 [[VAL:%.*]], i1 false), !dbg [[DBG20:![0-9]+]]

llvm/test/Transforms/LoopIdiom/X86/logical-right-shift-until-zero.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -loop-idiom -mtriple=x86_64 -mcpu=core-avx2 < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S -mtriple=x86_64 -mcpu=core-avx2 < %s | FileCheck %s
 
 declare void @escape_inner(i8, i8, i8, i1, i8)
 declare void @escape_outer(i8, i8, i8, i1, i8)
 
 declare i8 @gen.i8()
 
 ; Most basic pattern; Note that iff the shift amount is offset, said offsetting
 ; must not cause an overflow, but `add nsw` is fine.

llvm/test/Transforms/LoopIdiom/X86/popcnt.ll

 ; RUN: opt -loop-idiom < %s -mtriple=x86_64-apple-darwin -mcpu=corei7 -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S -mtriple=x86_64-apple-darwin -mcpu=corei7 < %s | FileCheck %s
 
 target triple = "x86_64-apple-macosx10.8.0"
 
 ;To recognize this pattern:
 ;int popcount(unsigned long long a) {
 ;    int c = 0;
 ;    while (a) {
 ;        c++;

llvm/test/Transforms/LoopIdiom/X86/unordered-atomic-memcpy.ll

 ; RUN: opt -basic-aa -loop-idiom < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64"
 target triple = "x86_64-unknown-linux-gnu"
 
 ;; memcpy.atomic formation (atomic load & store)
 define void @test1(i64 %Size) nounwind ssp {
 ; CHECK-LABEL: @test1(
 ; CHECK: call void @llvm.memcpy.element.unordered.atomic.p0i8.p0i8.i64(i8* align 1 %Dest, i8* align 1 %Base, i64 %Size, i32 1)
 ; CHECK-NOT: store

llvm/test/Transforms/LoopIdiom/basic-address-space.ll

 ; RUN: opt -basic-aa -loop-idiom < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 
 target datalayout = "e-p:32:32:32-p1:64:64:64-p2:16:16:16-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:32-n8:16:32:64"
 target triple = "x86_64-apple-darwin10.0.0"
 
 ; Two dimensional nested loop should be promoted to one big memset.
 define void @test10(i8 addrspace(2)* %X) nounwind ssp {
 ; CHECK-LABEL: @test10(
 ; CHECK: entry:

llvm/test/Transforms/LoopIdiom/basic.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -basic-aa -loop-idiom < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" < %s -S | FileCheck %s
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64"
 
 ; For @test11_pattern
 ; CHECK: @.memset_pattern = private unnamed_addr constant [4 x i32] [i32 1, i32 1, i32 1, i32 1]
 
 ; For @test13_pattern
 ; CHECK: @.memset_pattern.1 = private unnamed_addr constant [2 x i32*] [i32* @G, i32* @G]
 

llvm/test/Transforms/LoopIdiom/crash.ll

 ; RUN: opt -basic-aa -loop-idiom -S < %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
 target triple = "x86_64-apple-macosx10.8.0"
 
 ; Don't crash inside DependenceAnalysis
 ; PR14219
 define void @test1(i64* %iwork, i64 %x)  {
 bb0:
   %mul116 = mul nsw i64 %x, %x

llvm/test/Transforms/LoopIdiom/ctpop-multiple-users-crash.ll

 ; RUN: opt -loop-idiom -S < %s | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 
 target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128"
 target triple = "arm64-apple-ios8.0.0"
 
 ; When we replace the precondition with a ctpop, we need to ensure
 ; that only the first branch reads the ctpop.  The store prior
 ; to that should continue to read from the original compare.
 
for.body:                                         ; preds = %for.body.lr.ph, %for.body
   %inc = add nsw i32 %count.07, 1
   %tobool = icmp ne i32 %and, 0
   br i1 %tobool, label %for.body, label %for.end
 
 for.end:                                          ; preds = %for.cond.for.end_crit_edge, %entry
   %count.0.lcssa = phi i32 [ %inc, %for.body ], [ 0, %entry ]
   ret i32 %count.0.lcssa
 }
 No newline at end of file

llvm/test/Transforms/LoopIdiom/dbginfo-cost.ll

 ; RUN: opt -S -loop-idiom -mtriple=systemz-unknown -mcpu=z13 %s | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S -mtriple=systemz-unknown -mcpu=z13 < %s | FileCheck %s
 
 ; CHECK: @llvm.ctlz.i32
 
 declare void @llvm.dbg.declare(metadata, metadata, metadata) #0
 define dso_local i32 @CeilLog2(i32 %arg) local_unnamed_addr #1 !dbg !38 {
 bb:
   %tmp4 = add i32 %arg, -1, !dbg !45
   call void @llvm.dbg.value(metadata i32 0, metadata !44, metadata !DIExpression()), !dbg !45

llvm/test/Transforms/LoopIdiom/debug-line.ll

 ; RUN: opt -loop-idiom < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64"
 target triple = "x86_64-apple-darwin10.0.0"
 
 
 define void @foo(double* nocapture %a) nounwind ssp !dbg !0 {
 entry:
   tail call void @llvm.dbg.value(metadata double* %a, metadata !5, metadata !DIExpression()), !dbg !8
   tail call void @llvm.dbg.value(metadata i32 0, metadata !10, metadata !DIExpression()), !dbg !14

llvm/test/Transforms/LoopIdiom/disable-options.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -basic-aa -loop-idiom                             < %s -S | FileCheck %s --check-prefix=DIS-NONE
 ; RUN: opt -basic-aa -loop-idiom -disable-loop-idiom-all    < %s -S | FileCheck %s --check-prefix=DIS-ALL
 ; RUN: opt -basic-aa -loop-idiom -disable-loop-idiom-memcpy < %s -S | FileCheck %s --check-prefix=DIS-MEMCPY
 ; RUN: opt -basic-aa -loop-idiom -disable-loop-idiom-memset < %s -S | FileCheck %s --check-prefix=DIS-MEMSET
 ; RUN: opt -basic-aa -loop-idiom -disable-loop-idiom-memset -disable-loop-idiom-memcpy < %s -S | FileCheck %s --check-prefix=DIS-ALL
 ; RUN: opt -passes="loop-idiom" -aa-pipeline=basic-aa                             < %s -S | FileCheck %s --check-prefix=DIS-NONE
 ; RUN: opt -passes="loop-idiom" -aa-pipeline=basic-aa -disable-loop-idiom-all    < %s -S | FileCheck %s --check-prefix=DIS-ALL
 ; RUN: opt -passes="loop-idiom" -aa-pipeline=basic-aa -disable-loop-idiom-memcpy < %s -S | FileCheck %s --check-prefix=DIS-MEMCPY
 ; RUN: opt -passes="loop-idiom" -aa-pipeline=basic-aa -disable-loop-idiom-memset < %s -S | FileCheck %s --check-prefix=DIS-MEMSET
 ; RUN: opt -passes="loop-idiom" -aa-pipeline=basic-aa -disable-loop-idiom-memset -disable-loop-idiom-memcpy < %s -S | FileCheck %s --check-prefix=DIS-ALL
+; RUN: opt -passes="loop-nest-idiom" -aa-pipeline=basic-aa                             < %s -S | FileCheck %s --check-prefix=DIS-NONE
+; RUN: opt -passes="loop-nest-idiom" -aa-pipeline=basic-aa -disable-loop-idiom-all    < %s -S | FileCheck %s --check-prefix=DIS-ALL
+; RUN: opt -passes="loop-nest-idiom" -aa-pipeline=basic-aa -disable-loop-idiom-memcpy < %s -S | FileCheck %s --check-prefix=DIS-MEMCPY
+; RUN: opt -passes="loop-nest-idiom" -aa-pipeline=basic-aa -disable-loop-idiom-memset < %s -S | FileCheck %s --check-prefix=DIS-MEMSET
+; RUN: opt -passes="loop-nest-idiom" -aa-pipeline=basic-aa -disable-loop-idiom-memset -disable-loop-idiom-memcpy < %s -S | FileCheck %s --check-prefix=DIS-ALL
 
 define void @test-memcpy(i64 %Size) nounwind ssp {
 ; DIS-NONE-LABEL: @test-memcpy(
 ; DIS-NONE-NEXT:  bb.nph:
 ; DIS-NONE-NEXT:    [[BASE:%.*]] = alloca i8, i32 10000, align 1
 ; DIS-NONE-NEXT:    [[DEST:%.*]] = alloca i8, i32 10000, align 1
 ; DIS-NONE-NEXT:    call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 1 [[DEST]], i8* align 1 [[BASE]], i64 [[SIZE:%.*]], i1 false)
 ; DIS-NONE-NEXT:    br label [[FOR_BODY:%.*]]

llvm/test/Transforms/LoopIdiom/expander-do-not-delete-reused-values.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -loop-idiom -S %s | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 
 ; Make sure we do not delete instructions not inserted during expansion, e.g.
 ; because the expande re-used existing instructions.
 
 define void @test(i64 %init, float* %ptr) {
 ; CHECK-LABEL: @test(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[OUTER_HEADER:%.*]]

llvm/test/Transforms/LoopIdiom/int_sideeffect.ll

 ; RUN: opt -S < %s -loop-idiom | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 
 declare void @llvm.sideeffect()
 
 ; Loop idiom recognition across a @llvm.sideeffect.
 
 ; CHECK-LABEL: zero
 ; CHECK: llvm.memset
 define void @zero(float* %p, i64 %n) nounwind {

llvm/test/Transforms/LoopIdiom/lir-heurs-multi-block-loop.ll

 ;  RUN: opt -basic-aa -loop-idiom -use-lir-code-size-heurs=true < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -use-lir-code-size-heurs=true -S < %s | FileCheck %s
 
 ; When compiling for codesize we avoid idiom recognition for a
 ; multi-block loop unless it is one of
 ; - a loop_memset idiom, or
 ; - a memset/memcpy idiom in a nested loop.
 
 declare void @llvm.memset.p0i8.i64(i8* nocapture, i8, i64, i1)
 @APPLES = common global i32 0, align 4

llvm/test/Transforms/LoopIdiom/memcpy-debugify-remarks.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -basic-aa -debugify -loop-idiom -pass-remarks=loop-idiom -pass-remarks-analysis=loop-idiom -pass-remarks-output=%t.yaml -verify -verify-each -verify-dom-info -verify-loop-info  < %s -S 2>&1 | FileCheck %s
 ; RUN: FileCheck --input-file=%t.yaml %s --check-prefixes=YAML
+; RUN: opt -passes="loop-nest-idiom" -enable-debugify -debug-pass-manager -pass-remarks=loop-idiom -pass-remarks-analysis=loop-idiom -pass-remarks-output=%t.yaml < %s -S 2>&1 | FileCheck %s
+; RUN: FileCheck --input-file=%t.yaml %s --check-prefixes=YAML
 
 target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 ; Check that everything still works when debuginfo is present, and that it is reasonably propagated.
 
 ; CHECK: remark: <stdin>:6:1: Formed a call to llvm.memcpy.p0i8.p0i8.i64() intrinsic from load and store instruction in test6_dest_align function{{$}}
 

llvm/test/Transforms/LoopIdiom/memcpy-inline-intrinsic.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -loop-idiom < %s -S | FileCheck %s
+; RUN: opt --passes="loop-nest-idiom" < %s -S | FileCheck %s
 
 %struct.S = type { i32, i32, i8 }
 
 ; unsigned copy_noalias(S* __restrict a, S *b, int n) {
 ;   for (int i = 0; i < n; i++) {
 ;     a[i] = b[i];
 ;   }
 ;   return sizeof(a[0]);

llvm/test/Transforms/LoopIdiom/memcpy-intrinsic-different-types.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -loop-idiom < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 
 ; #include <vector>
 ;
 ; class SDValue {
 ;     int A;
 ;     int B;
 ;     unsigned C;
 ; };

llvm/test/Transforms/LoopIdiom/memcpy-intrinsic.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -loop-idiom < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 
 %struct.S = type { i32, i32, i8 }
 
 ; unsigned copy_noalias(S* __restrict a, S *b, int n) {
 ;   for (int i = 0; i < n; i++) {
 ;     a[i] = b[i];
 ;   }
 ;   return sizeof(a[0]);

llvm/test/Transforms/LoopIdiom/memcpy-vectors.ll

 ; RUN: opt -loop-idiom -S <%s | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 
 define void @memcpy_fixed_vec(i64* noalias %a, i64* noalias %b) local_unnamed_addr #1 {
 ; CHECK-LABEL: @memcpy_fixed_vec(
 ; CHECK: entry:
 ; CHECK: memcpy
 ; CHECK: vector.body
 entry:
   br label %vector.body

llvm/test/Transforms/LoopIdiom/memcpy.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -basic-aa -loop-idiom < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 
 define void @copy_both_noalias(float* noalias nocapture %d, float* noalias nocapture readonly %s, i64 %sz) {
 ; CHECK-LABEL: @copy_both_noalias(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[D1:%.*]] = bitcast float* [[D:%.*]] to i8*
 ; CHECK-NEXT:    [[S2:%.*]] = bitcast float* [[S:%.*]] to i8*
 ; CHECK-NEXT:    [[EXITCOND_NOT1:%.*]] = icmp eq i64 [[SZ:%.*]], 0
 ; CHECK-NEXT:    br i1 [[EXITCOND_NOT1]], label [[FOR_END:%.*]], label [[FOR_BODY_PREHEADER:%.*]]

llvm/test/Transforms/LoopIdiom/memset-debugify-remarks.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -basic-aa -debugify -loop-idiom -pass-remarks=loop-idiom -pass-remarks-analysis=loop-idiom -pass-remarks-output=%t.yaml -verify -verify-each -verify-dom-info -verify-loop-info  < %s -S 2>&1 | FileCheck %s
 ; RUN: FileCheck --input-file=%t.yaml %s --check-prefixes=YAML
+; RUN: opt -passes="loop-nest-idiom" -enable-debugify -debug-pass-manager -pass-remarks=loop-idiom -pass-remarks-analysis=loop-idiom -pass-remarks-output=%t.yaml < %s -S 2>&1 | FileCheck %s
+; RUN: FileCheck --input-file=%t.yaml %s --check-prefixes=YAML
 
 target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 ; Check that everything still works when debuginfo is present, and that it is reasonably propagated.
 
 ; void my_basic_memset(char* begin, char* end, char value) {
 ;   for( ; begin != end; ++begin)

llvm/test/Transforms/LoopIdiom/memset-pr52104.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -basic-aa -loop-idiom < %s -S | FileCheck %s
+; RUN: opt --passes="loop-nest-idiom" < %s -S | FileCheck %s
 
 define void @f(i32* nocapture nonnull align 4 dereferenceable(20) %0, i32 %1) local_unnamed_addr #0 align 32 {
 ; CHECK-LABEL: @f(
 ; CHECK-NEXT:    [[TMP3:%.*]] = trunc i32 [[TMP1:%.*]] to i2
 ; CHECK-NEXT:    [[TMP4:%.*]] = zext i2 [[TMP3]] to i64
 ; CHECK-NEXT:    [[SCEVGEP:%.*]] = getelementptr i32, i32* [[TMP0:%.*]], i64 [[TMP4]]
 ; CHECK-NEXT:    [[SCEVGEP1:%.*]] = bitcast i32* [[SCEVGEP]] to i8*
 ; CHECK-NEXT:    [[TMP5:%.*]] = sub i2 -1, [[TMP3]]

llvm/test/Transforms/LoopIdiom/memset-runtime-32bit.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -passes="function(loop(indvars,loop-idiom,loop-deletion),simplifycfg)" -S < %s | FileCheck %s
+; RUN: opt -passes="function(loop(indvars,loop-idiom,loop-deletion),simplifycfg)" -S < %s | FileCheck %s --check-prefixes=CHECK,LIR
+; RUN: opt -passes="function(loop(indvars,loop-nest-idiom,loop-deletion),simplifycfg)" -S < %s | FileCheck %s --check-prefixes=CHECK,LNIR
 ; Compile command:
 ; $ clang -m32 -fno-discard-value-names -O0 -S -emit-llvm -Xclang -disable-O0-optnone Code.c
 ; $ bin/opt -S -basic-aa -mem2reg -loop-simplify -lcssa -loop-rotate \
 ;   -licm -simple-loop-unswitch -enable-nontrivial-unswitch -loop-simplify \
 ;   -loop-deletion -simplifycfg -indvars Code.ll > CodeOpt.ll
 target datalayout = "e-m:e-p:32:32-p270:32:32-p271:32:32-p272:64:64-f64:32:64-f80:32-n8:16:32-S128"
 ; void PositiveFor32(int *ar, int n, int m)
 ; {

 ;   for (i=0; i<n; ++i) {
 ;     for (j=0; j<m; j++) {
 ;       int *arr = ar + i * m * o + j * o;
 ;       memset(arr, 0, o * sizeof(int));
 ;     }
 ;   }
 ; }
 define dso_local void @NestedFor64(i32* %ar, i64 %n, i64 %m, i64 %o) #0 {
-; CHECK-LABEL: @NestedFor64(
-; CHECK-NEXT:  entry:
-; CHECK-NEXT:    [[CMP3:%.*]] = icmp slt i64 0, [[N:%.*]]
-; CHECK-NEXT:    [[CMP41:%.*]] = icmp slt i64 0, [[M:%.*]]
-; CHECK-NEXT:    [[MUL13:%.*]] = mul nsw i64 [[O:%.*]], 4
-; CHECK-NEXT:    [[CONV14:%.*]] = trunc i64 [[MUL13]] to i32
-; CHECK-NEXT:    [[OR_COND:%.*]] = select i1 [[CMP3]], i1 [[CMP41]], i1 false
-; CHECK-NEXT:    br i1 [[OR_COND]], label [[FOR_BODY_US_PREHEADER:%.*]], label [[FOR_END17:%.*]]
-; CHECK:       for.body.us.preheader:
-; CHECK-NEXT:    [[TMP0:%.*]] = mul i64 [[O]], [[M]]
-; CHECK-NEXT:    [[TMP1:%.*]] = trunc i64 [[TMP0]] to i32
-; CHECK-NEXT:    [[TMP2:%.*]] = trunc i64 [[O]] to i32
-; CHECK-NEXT:    [[TMP3:%.*]] = trunc i64 [[M]] to i32
-; CHECK-NEXT:    [[TMP4:%.*]] = mul i32 [[TMP2]], [[TMP3]]
-; CHECK-NEXT:    [[TMP5:%.*]] = shl i32 [[TMP4]], 2
-; CHECK-NEXT:    br label [[FOR_BODY_US:%.*]]
-; CHECK:       for.body.us:
-; CHECK-NEXT:    [[I_04_US:%.*]] = phi i32 [ [[INC16_US:%.*]], [[FOR_BODY_US]] ], [ 0, [[FOR_BODY_US_PREHEADER]] ]
-; CHECK-NEXT:    [[TMP6:%.*]] = mul i32 [[TMP1]], [[I_04_US]]
-; CHECK-NEXT:    [[SCEVGEP:%.*]] = getelementptr i32, i32* [[AR:%.*]], i32 [[TMP6]]
-; CHECK-NEXT:    [[SCEVGEP1:%.*]] = bitcast i32* [[SCEVGEP]] to i8*
-; CHECK-NEXT:    [[CONV7_US:%.*]] = sext i32 [[I_04_US]] to i64
-; CHECK-NEXT:    [[MUL_US:%.*]] = mul nsw i64 [[CONV7_US]], [[M]]
-; CHECK-NEXT:    [[MUL8_US:%.*]] = mul nsw i64 [[MUL_US]], [[O]]
-; CHECK-NEXT:    [[IDX_EXT_US:%.*]] = trunc i64 [[MUL8_US]] to i32
-; CHECK-NEXT:    [[ADD_PTR_US:%.*]] = getelementptr inbounds i32, i32* [[AR]], i32 [[IDX_EXT_US]]
-; CHECK-NEXT:    call void @llvm.memset.p0i8.i32(i8* align 4 [[SCEVGEP1]], i8 0, i32 [[TMP5]], i1 false)
-; CHECK-NEXT:    [[INC16_US]] = add nuw nsw i32 [[I_04_US]], 1
-; CHECK-NEXT:    [[CONV_US:%.*]] = sext i32 [[INC16_US]] to i64
-; CHECK-NEXT:    [[CMP_US:%.*]] = icmp slt i64 [[CONV_US]], [[N]]
-; CHECK-NEXT:    br i1 [[CMP_US]], label [[FOR_BODY_US]], label [[FOR_END17]]
-; CHECK:       for.end17:
-; CHECK-NEXT:    ret void
+; LIR-LABEL: @NestedFor64(
+; LIR-NEXT:  entry:
+; LIR-NEXT:    [[CMP3:%.*]] = icmp slt i64 0, [[N:%.*]]
+; LIR-NEXT:    [[CMP41:%.*]] = icmp slt i64 0, [[M:%.*]]
+; LIR-NEXT:    [[MUL13:%.*]] = mul nsw i64 [[O:%.*]], 4
+; LIR-NEXT:    [[CONV14:%.*]] = trunc i64 [[MUL13]] to i32
+; LIR-NEXT:    [[OR_COND:%.*]] = select i1 [[CMP3]], i1 [[CMP41]], i1 false
+; LIR-NEXT:    br i1 [[OR_COND]], label [[FOR_BODY_US_PREHEADER:%.*]], label [[FOR_END17:%.*]]
+; LIR:       for.body.us.preheader:
+; LIR-NEXT:    [[TMP0:%.*]] = mul i64 [[O]], [[M]]
+; LIR-NEXT:    [[TMP1:%.*]] = trunc i64 [[TMP0]] to i32
+; LIR-NEXT:    [[TMP2:%.*]] = trunc i64 [[O]] to i32
+; LIR-NEXT:    [[TMP3:%.*]] = trunc i64 [[M]] to i32
+; LIR-NEXT:    [[TMP4:%.*]] = mul i32 [[TMP2]], [[TMP3]]
+; LIR-NEXT:    [[TMP5:%.*]] = shl i32 [[TMP4]], 2
+; LIR-NEXT:    br label [[FOR_BODY_US:%.*]]
+; LIR:       for.body.us:
+; LIR-NEXT:    [[I_04_US:%.*]] = phi i32 [ [[INC16_US:%.*]], [[FOR_BODY_US]] ], [ 0, [[FOR_BODY_US_PREHEADER]] ]
+; LIR-NEXT:    [[TMP6:%.*]] = mul i32 [[TMP1]], [[I_04_US]]
+; LIR-NEXT:    [[SCEVGEP:%.*]] = getelementptr i32, i32* [[AR:%.*]], i32 [[TMP6]]
+; LIR-NEXT:    [[SCEVGEP1:%.*]] = bitcast i32* [[SCEVGEP]] to i8*
+; LIR-NEXT:    [[CONV7_US:%.*]] = sext i32 [[I_04_US]] to i64
+; LIR-NEXT:    [[MUL_US:%.*]] = mul nsw i64 [[CONV7_US]], [[M]]
+; LIR-NEXT:    [[MUL8_US:%.*]] = mul nsw i64 [[MUL_US]], [[O]]
+; LIR-NEXT:    [[IDX_EXT_US:%.*]] = trunc i64 [[MUL8_US]] to i32
+; LIR-NEXT:    [[ADD_PTR_US:%.*]] = getelementptr inbounds i32, i32* [[AR]], i32 [[IDX_EXT_US]]
+; LIR-NEXT:    call void @llvm.memset.p0i8.i32(i8* align 4 [[SCEVGEP1]], i8 0, i32 [[TMP5]], i1 false)
+; LIR-NEXT:    [[INC16_US]] = add nuw nsw i32 [[I_04_US]], 1
+; LIR-NEXT:    [[CONV_US:%.*]] = sext i32 [[INC16_US]] to i64
+; LIR-NEXT:    [[CMP_US:%.*]] = icmp slt i64 [[CONV_US]], [[N]]
+; LIR-NEXT:    br i1 [[CMP_US]], label [[FOR_BODY_US]], label [[FOR_END17]]
+; LIR:       for.end17:
+; LIR-NEXT:    ret void
+;
+; LNIR-LABEL: @NestedFor64(
+; LNIR-NEXT:  entry:
+; LNIR-NEXT:    [[CMP3:%.*]] = icmp slt i64 0, [[N:%.*]]
+; LNIR-NEXT:    [[CMP41:%.*]] = icmp slt i64 0, [[M:%.*]]
+; LNIR-NEXT:    [[MUL13:%.*]] = mul nsw i64 [[O:%.*]], 4
+; LNIR-NEXT:    [[CONV14:%.*]] = trunc i64 [[MUL13]] to i32
+; LNIR-NEXT:    [[OR_COND:%.*]] = select i1 [[CMP3]], i1 [[CMP41]], i1 false
+; LNIR-NEXT:    br i1 [[OR_COND]], label [[FOR_BODY_US_PREHEADER:%.*]], label [[FOR_END17:%.*]]
+; LNIR:       for.body.us.preheader:
+; LNIR-NEXT:    [[TMP0:%.*]] = mul i64 [[O]], [[M]]
+; LNIR-NEXT:    [[TMP1:%.*]] = trunc i64 [[TMP0]] to i32
+; LNIR-NEXT:    [[TMP2:%.*]] = trunc i64 [[O]] to i32
+; LNIR-NEXT:    [[TMP3:%.*]] = trunc i64 [[M]] to i32
+; LNIR-NEXT:    [[TMP4:%.*]] = mul i32 [[TMP2]], [[TMP3]]
+; LNIR-NEXT:    [[TMP5:%.*]] = shl i32 [[TMP4]], 2
+; LNIR-NEXT:    br label [[FOR_BODY_US:%.*]]
+; LNIR:       for.body.us:
+; LNIR-NEXT:    [[I_04_US:%.*]] = phi i32 [ [[INC16_US:%.*]], [[FOR_COND2_FOR_END_CRIT_EDGE_US:%.*]] ], [ 0, [[FOR_BODY_US_PREHEADER]] ]
+; LNIR-NEXT:    [[TMP6:%.*]] = mul i32 [[TMP1]], [[I_04_US]]
+; LNIR-NEXT:    [[SCEVGEP:%.*]] = getelementptr i32, i32* [[AR:%.*]], i32 [[TMP6]]
+; LNIR-NEXT:    [[SCEVGEP1:%.*]] = bitcast i32* [[SCEVGEP]] to i8*
+; LNIR-NEXT:    [[CONV7_US:%.*]] = sext i32 [[I_04_US]] to i64
+; LNIR-NEXT:    [[MUL_US:%.*]] = mul nsw i64 [[CONV7_US]], [[M]]
+; LNIR-NEXT:    [[MUL8_US:%.*]] = mul nsw i64 [[MUL_US]], [[O]]
+; LNIR-NEXT:    [[IDX_EXT_US:%.*]] = trunc i64 [[MUL8_US]] to i32
+; LNIR-NEXT:    [[ADD_PTR_US:%.*]] = getelementptr inbounds i32, i32* [[AR]], i32 [[IDX_EXT_US]]
+; LNIR-NEXT:    call void @llvm.memset.p0i8.i32(i8* align 4 [[SCEVGEP1]], i8 0, i32 [[TMP5]], i1 false)
+; LNIR-NEXT:    br label [[FOR_BODY6_US:%.*]]
+; LNIR:       for.body6.us:
+; LNIR-NEXT:    [[J_02_US:%.*]] = phi i32 [ 0, [[FOR_BODY_US]] ], [ [[INC_US:%.*]], [[FOR_BODY6_US]] ]
+; LNIR-NEXT:    [[CONV9_US:%.*]] = sext i32 [[J_02_US]] to i64
+; LNIR-NEXT:    [[MUL10_US:%.*]] = mul nsw i64 [[CONV9_US]], [[O]]
+; LNIR-NEXT:    [[IDX_EXT11_US:%.*]] = trunc i64 [[MUL10_US]] to i32
+; LNIR-NEXT:    [[ADD_PTR12_US:%.*]] = getelementptr inbounds i32, i32* [[ADD_PTR_US]], i32 [[IDX_EXT11_US]]
+; LNIR-NEXT:    [[TMP7:%.*]] = bitcast i32* [[ADD_PTR12_US]] to i8*
+; LNIR-NEXT:    [[INC_US]] = add nuw nsw i32 [[J_02_US]], 1
+; LNIR-NEXT:    [[CONV3_US:%.*]] = sext i32 [[INC_US]] to i64
+; LNIR-NEXT:    [[CMP4_US:%.*]] = icmp slt i64 [[CONV3_US]], [[M]]
+; LNIR-NEXT:    br i1 [[CMP4_US]], label [[FOR_BODY6_US]], label [[FOR_COND2_FOR_END_CRIT_EDGE_US]]
+; LNIR:       for.cond2.for.end_crit_edge.us:
+; LNIR-NEXT:    [[INC16_US]] = add nuw nsw i32 [[I_04_US]], 1
+; LNIR-NEXT:    [[CONV_US:%.*]] = sext i32 [[INC16_US]] to i64
+; LNIR-NEXT:    [[CMP_US:%.*]] = icmp slt i64 [[CONV_US]], [[N]]
+; LNIR-NEXT:    br i1 [[CMP_US]], label [[FOR_BODY_US]], label [[FOR_END17]]
+; LNIR:       for.end17:
+; LNIR-NEXT:    ret void
 ;
 entry:
   %cmp3 = icmp slt i64 0, %n
   br i1 %cmp3, label %for.body.lr.ph, label %for.end17
 
 for.body.lr.ph:                                   ; preds = %entry
   %cmp41 = icmp slt i64 0, %m
   %mul13 = mul nsw i64 %o, 4

llvm/test/Transforms/LoopIdiom/memset-runtime-64bit.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -passes="function(loop(indvars,loop-idiom,loop-deletion),simplifycfg)" -S < %s | FileCheck %s
+; RUN: opt -passes="function(loop(indvars,loop-idiom,loop-deletion),simplifycfg)" -S < %s | FileCheck %s --check-prefixes=CHECK,LIR
+; RUN: opt -passes="function(loop(indvars,loop-nest-idiom,loop-deletion),simplifycfg)" -S < %s | FileCheck %s --check-prefixes=CHECK,LNIR
 ; Compile command:
 ; $ clang -m64 -fno-discard-value-names -O0 -S -emit-llvm -Xclang -disable-O0-optnone Code.c
 ; $ bin/opt -S -basic-aa -mem2reg -loop-simplify -lcssa -loop-rotate \
 ;   -licm -simple-loop-unswitch -enable-nontrivial-unswitch -loop-simplify \
 ;   -loop-deletion -simplifycfg -indvars Code.ll > CodeOpt.ll
 target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
 ; void PositiveFor64(int *ar, long long n, long long m)
 ; {

 ;   int i, j;
 ;   for (i=0; i<n; ++i) {
 ;     for (j=0; j<m; j++) {
 ;       int *arr = ar + i * m * o + j * o;
 ;       memset(arr, 0, o * sizeof(int));
 ;     }
 ;   }
 ; }
 define void @NestedFor32(i32* %ar, i32 %n, i32 %m, i32 %o) {
-; CHECK-LABEL: @NestedFor32(
-; CHECK-NEXT:  entry:
-; CHECK-NEXT:    [[AR2:%.*]] = bitcast i32* [[AR:%.*]] to i8*
-; CHECK-NEXT:    [[CMP3:%.*]] = icmp slt i32 0, [[N:%.*]]
-; CHECK-NEXT:    br i1 [[CMP3]], label [[FOR_BODY_LR_PH:%.*]], label [[FOR_END11:%.*]]
-; CHECK:       for.body.lr.ph:
-; CHECK-NEXT:    [[CMP21:%.*]] = icmp slt i32 0, [[M:%.*]]
-; CHECK-NEXT:    [[CONV:%.*]] = sext i32 [[O:%.*]] to i64
-; CHECK-NEXT:    [[MUL8:%.*]] = mul i64 [[CONV]], 4
-; CHECK-NEXT:    br i1 [[CMP21]], label [[FOR_BODY_US_PREHEADER:%.*]], label [[FOR_END11]]
-; CHECK:       for.body.us.preheader:
-; CHECK-NEXT:    [[TMP0:%.*]] = sext i32 [[O]] to i64
-; CHECK-NEXT:    [[TMP1:%.*]] = sext i32 [[M]] to i64
-; CHECK-NEXT:    [[WIDE_TRIP_COUNT10:%.*]] = zext i32 [[N]] to i64
-; CHECK-NEXT:    [[TMP2:%.*]] = mul i64 [[TMP0]], [[TMP1]]
-; CHECK-NEXT:    [[TMP3:%.*]] = zext i32 [[M]] to i64
-; CHECK-NEXT:    [[TMP4:%.*]] = mul i64 [[TMP0]], [[TMP3]]
-; CHECK-NEXT:    [[TMP5:%.*]] = shl i64 [[TMP4]], 2
-; CHECK-NEXT:    [[TMP6:%.*]] = mul i64 [[TMP4]], [[WIDE_TRIP_COUNT10]]
-; CHECK-NEXT:    [[TMP7:%.*]] = shl i64 [[TMP6]], 2
-; CHECK-NEXT:    call void @llvm.memset.p0i8.i64(i8* align 4 [[AR2]], i8 0, i64 [[TMP7]], i1 false)
-; CHECK-NEXT:    br label [[FOR_END11]]
-; CHECK:       for.end11:
-; CHECK-NEXT:    ret void
+; LIR-LABEL: @NestedFor32(
+; LIR-NEXT:  entry:
+; LIR-NEXT:    [[AR2:%.*]] = bitcast i32* [[AR:%.*]] to i8*
+; LIR-NEXT:    [[CMP3:%.*]] = icmp slt i32 0, [[N:%.*]]
+; LIR-NEXT:    br i1 [[CMP3]], label [[FOR_BODY_LR_PH:%.*]], label [[FOR_END11:%.*]]
+; LIR:       for.body.lr.ph:
+; LIR-NEXT:    [[CMP21:%.*]] = icmp slt i32 0, [[M:%.*]]
+; LIR-NEXT:    [[CONV:%.*]] = sext i32 [[O:%.*]] to i64
+; LIR-NEXT:    [[MUL8:%.*]] = mul i64 [[CONV]], 4
+; LIR-NEXT:    br i1 [[CMP21]], label [[FOR_BODY_US_PREHEADER:%.*]], label [[FOR_END11]]
+; LIR:       for.body.us.preheader:
+; LIR-NEXT:    [[TMP0:%.*]] = sext i32 [[O]] to i64
+; LIR-NEXT:    [[TMP1:%.*]] = sext i32 [[M]] to i64
+; LIR-NEXT:    [[WIDE_TRIP_COUNT10:%.*]] = zext i32 [[N]] to i64
+; LIR-NEXT:    [[TMP2:%.*]] = mul i64 [[TMP0]], [[TMP1]]
+; LIR-NEXT:    [[TMP3:%.*]] = zext i32 [[M]] to i64
+; LIR-NEXT:    [[TMP4:%.*]] = mul i64 [[TMP0]], [[TMP3]]
+; LIR-NEXT:    [[TMP5:%.*]] = shl i64 [[TMP4]], 2
+; LIR-NEXT:    [[TMP6:%.*]] = mul i64 [[TMP4]], [[WIDE_TRIP_COUNT10]]
+; LIR-NEXT:    [[TMP7:%.*]] = shl i64 [[TMP6]], 2
+; LIR-NEXT:    call void @llvm.memset.p0i8.i64(i8* align 4 [[AR2]], i8 0, i64 [[TMP7]], i1 false)
+; LIR-NEXT:    br label [[FOR_END11]]
+; LIR:       for.end11:
+; LIR-NEXT:    ret void
+;
+; LNIR-LABEL: @NestedFor32(
+; LNIR-NEXT:  entry:
+; LNIR-NEXT:    [[AR2:%.*]] = bitcast i32* [[AR:%.*]] to i8*
+; LNIR-NEXT:    [[CMP3:%.*]] = icmp slt i32 0, [[N:%.*]]
+; LNIR-NEXT:    br i1 [[CMP3]], label [[FOR_BODY_LR_PH:%.*]], label [[FOR_END11:%.*]]
+; LNIR:       for.body.lr.ph:
+; LNIR-NEXT:    [[CMP21:%.*]] = icmp slt i32 0, [[M:%.*]]
+; LNIR-NEXT:    [[CONV:%.*]] = sext i32 [[O:%.*]] to i64
+; LNIR-NEXT:    [[MUL8:%.*]] = mul i64 [[CONV]], 4
+; LNIR-NEXT:    br i1 [[CMP21]], label [[FOR_BODY_US_PREHEADER:%.*]], label [[FOR_END11]]
+; LNIR:       for.body.us.preheader:
+; LNIR-NEXT:    [[TMP0:%.*]] = sext i32 [[O]] to i64
+; LNIR-NEXT:    [[TMP1:%.*]] = sext i32 [[M]] to i64
+; LNIR-NEXT:    [[TMP2:%.*]] = sext i32 [[O]] to i64
+; LNIR-NEXT:    [[WIDE_TRIP_COUNT10:%.*]] = zext i32 [[N]] to i64
+; LNIR-NEXT:    [[TMP3:%.*]] = mul i64 [[TMP0]], [[TMP1]]
+; LNIR-NEXT:    [[TMP4:%.*]] = zext i32 [[M]] to i64
+; LNIR-NEXT:    [[TMP5:%.*]] = mul i64 [[TMP0]], [[TMP4]]
+; LNIR-NEXT:    [[TMP6:%.*]] = shl i64 [[TMP5]], 2
+; LNIR-NEXT:    [[TMP7:%.*]] = mul i64 [[TMP5]], [[WIDE_TRIP_COUNT10]]
+; LNIR-NEXT:    [[TMP8:%.*]] = shl i64 [[TMP7]], 2
+; LNIR-NEXT:    call void @llvm.memset.p0i8.i64(i8* align 4 [[AR2]], i8 0, i64 [[TMP8]], i1 false)
+; LNIR-NEXT:    br label [[FOR_END11]]
+; LNIR:       for.end11:
+; LNIR-NEXT:    ret void
 ;
 entry:
   %cmp3 = icmp slt i32 0, %n
   br i1 %cmp3, label %for.body.lr.ph, label %for.end11
 
 for.body.lr.ph:                                   ; preds = %entry
   %cmp21 = icmp slt i32 0, %m
   %conv = sext i32 %o to i64

llvm/test/Transforms/LoopIdiom/memset-runtime-debug.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; REQUIRES: asserts
 ; RUN: opt < %s -S -debug -passes=loop-idiom 2>&1 | FileCheck %s
+; RUN: opt < %s -S -debug -passes=loop-nest-idiom 2>&1 | FileCheck %s
 ; The C code to generate this testcase:
 ; void test(int *ar, int n, int m)
 ; {
 ;   long i;
 ;   for (i=0; i<n; ++i) {
 ;     int *arr = ar + i * m;
 ;     memset(arr, 0, i + m * sizeof(int));
 ;   }

llvm/test/Transforms/LoopIdiom/memset.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -basic-aa -loop-idiom < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" < %s -S | FileCheck %s
 
 
 define dso_local void @double_memset(i8* nocapture %p, i8* noalias nocapture %q, i32 %n) {
 ; CHECK-LABEL: @double_memset(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[CMP4:%.*]] = icmp sgt i32 [[N:%.*]], 0
 ; CHECK-NEXT:    br i1 [[CMP4]], label [[FOR_BODY_PREHEADER:%.*]], label [[FOR_COND_CLEANUP:%.*]]
 ; CHECK:       for.body.preheader:

llvm/test/Transforms/LoopIdiom/memset_noidiom.ll

 ; RUN: opt -loop-idiom < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64"
 target triple = "x86_64-apple-darwin10.0.0"
 
 ; CHECK-LABEL: @memset(
 ; CHECK-NOT: llvm.memset
 define i8* @memset(i8* %b, i32 %c, i64 %len) nounwind uwtable ssp {
 entry:
   %cmp1 = icmp ult i64 0, %len

llvm/test/Transforms/LoopIdiom/non-canonical-loop.ll

 ; RUN: opt -S -loop-idiom < %s
+; RUN: opt -S --passes="loop-nest-idiom" < %s
 ; Don't crash
 ; PR13892
 
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 define void @test(i32* %currMB) nounwind uwtable {
 entry:

llvm/test/Transforms/LoopIdiom/non-integral-pointers.ll

 ; RUN: opt -S -basic-aa -loop-idiom < %s | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 
 target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128-ni:4"
 target triple = "x86_64-unknown-linux-gnu"
 
 ; LIR'ing stores of pointers with address space 3 is fine, since
 ; they're integral pointers.
 define void @f_0(i8 addrspace(3)** %ptr) {
 ; CHECK-LABEL: @f_0(

llvm/test/Transforms/LoopIdiom/nontemporal_store.ll

 ; RUN: opt -loop-idiom < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 ; RUN: opt -aa-pipeline=basic-aa -passes='require<aa>,require<targetir>,require<scalar-evolution>,require<opt-remark-emit>,loop(loop-idiom)' < %s -S | FileCheck %s
 target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 %struct.bigBlock_t = type { [256 x <4 x float>] }
 
 ; CHECK-LABEL: @test(
 ; CHECK-NOT: llvm.memset

llvm/test/Transforms/LoopIdiom/phi-insertion.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -loop-idiom -S %s | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 
 define void @phi_insertion(i1 %c, i32* %ptr.start, i32* %ptr.end, i64 %offset) {
 ; CHECK-LABEL: @phi_insertion(
 ; CHECK-NEXT:  bb5:
 ; CHECK-NEXT:    br i1 [[C:%.*]], label [[LOOP_0_PH:%.*]], label [[EXIT:%.*]]
 ; CHECK:       loop.0.ph:
 ; CHECK-NEXT:    br label [[LOOP_0:%.*]]
 ; CHECK:       loop.0:

llvm/test/Transforms/LoopIdiom/pr28196.ll

 ; RUN: opt -loop-idiom -S < %s | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 
 target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 define void @test1() {
 entry:
   br label %for.body.preheader
 

llvm/test/Transforms/LoopIdiom/pr33114.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; Check that we're not crashing while looking at the recurrence variable.
 ; RUN: opt -S -loop-idiom %s | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 
 define void @tinkywinky() {
 ; CHECK-LABEL: @tinkywinky(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br i1 true, label [[EXIT:%.*]], label [[PH:%.*]]
 ; CHECK:       ph:
 ; CHECK-NEXT:    [[MYPHI:%.*]] = phi i32 [ 1, [[ENTRY:%.*]] ]
 ; CHECK-NEXT:    br label [[IF_END:%.*]]

llvm/test/Transforms/LoopIdiom/reuse-cast.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
 ; RUN: opt -loop-idiom -S %s | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 
 define void @reuse_cast_1(float** %ptr, i1 %c) {
 ; CHECK-LABEL: @reuse_cast_1(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP_0:%.*]]
 ; CHECK:       loop.0:
 ; CHECK-NEXT:    [[TMP:%.*]] = load float*, float** [[PTR:%.*]], align 8
 ; CHECK-NEXT:    [[TMP2:%.*]] = bitcast float* [[TMP]] to i8*

llvm/test/Transforms/LoopIdiom/scev-invalidation.ll

 ; RUN: opt -S -indvars -loop-idiom < %s
+; RUN: opt -passes="loop-nest-idiom,indvars" -S < %s
 ; PR14214
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 define i32 @quote_arg() nounwind {
 entry:
   br label %for.cond
 

llvm/test/Transforms/LoopIdiom/scev-invalidation_topmostloop.ll

 ; RUN: opt -S -indvars -loop-idiom -verify -loop-simplifycfg -simplifycfg-require-and-preserve-domtree=1 -loop-idiom < %s | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 
 target triple = "x86_64-unknown-linux-gnu"
 
 ; CHECK-LABEL: @f1()
 ; CHECK-NEXT: entry:
 define void @f1() {
 entry:
   br label %lbl1

llvm/test/Transforms/LoopIdiom/struct-custom-dl.ll

 ; RUN: opt -basic-aa -loop-idiom < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 target datalayout = "e-p:40:64:64:32-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64"
 
 %struct.foo = type { i32, i32 }
 %struct.foo1 = type { i32, i32, i32 }
 %struct.foo2 = type { i32, i16, i16 }
 
 ;void bar1(foo_t *f, unsigned n) {
 ;  for (unsigned i = 0; i < n; ++i) {

llvm/test/Transforms/LoopIdiom/struct.ll

 ; RUN: opt -basic-aa -loop-idiom < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64"
 
 target triple = "x86_64-apple-darwin10.0.0"
 
 %struct.foo = type { i32, i32 }
 %struct.foo1 = type { i32, i32, i32 }
 %struct.foo2 = type { i32, i16, i16 }
 

llvm/test/Transforms/LoopIdiom/struct_pattern.ll

 ; RUN: opt -basic-aa -loop-idiom < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64"
 
 ; CHECK: @.memset_pattern = private unnamed_addr constant [4 x i32] [i32 2, i32 2, i32 2, i32 2], align 16
 ; CHECK: @.memset_pattern.1 = private unnamed_addr constant [4 x i32] [i32 2, i32 2, i32 2, i32 2], align 16
 ; CHECK: @.memset_pattern.2 = private unnamed_addr constant [4 x i32] [i32 2, i32 2, i32 2, i32 2], align 16
 
 target triple = "x86_64-apple-darwin10.0.0"
 

llvm/test/Transforms/LoopIdiom/unordered-atomic-memcpy-noarch.ll

 ; RUN: opt -basic-aa -loop-idiom < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64"
 
 ;; memcpy.atomic formation (atomic load & store) -- element size 2
 ;;  Will not create call due to a max element size of 0
 define void @test1(i64 %Size) nounwind ssp {
 ; CHECK-LABEL: @test1(
 ; CHECK-NOT: call void @llvm.memcpy.element.unordered.atomic
 ; CHECK: store

llvm/test/Transforms/LoopIdiom/unroll-custom-dl.ll

 ; RUN: opt -basic-aa -loop-idiom < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 target datalayout = "e-p:64:64:64:32-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64"
 
 ; CHECK: @.memset_pattern = private unnamed_addr constant [4 x i32] [i32 2, i32 2, i32 2, i32 2], align 16
 
 target triple = "x86_64-apple-darwin10.0.0"
 
 ;void test(int *f, unsigned n) {
 ;  for (unsigned i = 0; i < 2 * n; i += 2) {

llvm/test/Transforms/LoopIdiom/unroll.ll

 ; RUN: opt -basic-aa -loop-idiom < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64"
 
 ; CHECK: @.memset_pattern = private unnamed_addr constant [4 x i32] [i32 2, i32 2, i32 2, i32 2], align 16
 
 target triple = "x86_64-apple-darwin10.0.0"
 
 ;void test(int *f, unsigned n) {
 ;  for (unsigned i = 0; i < 2 * n; i += 2) {

llvm/test/Transforms/LoopIdiom/unsafe.ll

 ; RUN: opt -S < %s -loop-idiom | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 ; CHECK-NOT: memset
 ; check that memset is not generated (for stores) because that will result
 ; in udiv hoisted out of the loop by the SCEV Expander
 ; TODO: ideally we should be able to generate memset
 ; if SCEV expander is taught to generate the dependencies
 ; at the right point.
 
 @a = global i32 0, align 4

llvm/test/Transforms/LoopIdiom/unwind.ll

 ; RUN: opt -loop-idiom < %s -S | FileCheck %s
+; RUN: opt -passes="loop-nest-idiom" -S < %s | FileCheck %s
 target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 declare void @ff()
 
 define void @test(i8* noalias nocapture %base, i64 %size) #1 {
 entry:
   %cmp3 = icmp eq i64 %size, 0