Fix full loop unrolling initialization in new pass manager

[echristo]

Summary

Fix full unrolling with new pass manager.

Last we looked at this and couldn't come up with a reason to change
it, but with a pragma for full loop unrolling we bypass every other
loop unroll and then fail to fully unroll a loop when the pragma is set.

Move the OnlyWhenForced out of the check and into the initialization
of the full unroll pass in the new pass manager. This doesn't show up
with the old pass manager.

Add a new option to opt so that we can turn off loop unrolling
manually since this is a difference between clang and opt.

Tested with check-clang and check-llvm.

[echristo]

Formatting and parens changes.

[merge_guards_bot]

Unit tests: pass. 61011 tests passed, 0 failed and 728 were skipped.

clang-tidy: pass.

clang-format: pass.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[merge_guards_bot]

Unit tests: pass. 61011 tests passed, 0 failed and 728 were skipped.

clang-tidy: pass.

clang-format: pass.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[echristo]

Ping ping goes the trolley.

[MaskRay]

clang/test/Misc/loop-opt-setup.c

13

The test seems to pass without the code change (llvm/lib/Passes/PassBuilder.cpp) below.

[mehdi_amini]

llvm/lib/Passes/PassBuilder.cpp

514

I would document that the normal unroll pass does not honor the forced unroll, it is surprising to me.

[chandlerc]

Can we add an LLVM test w/ the metadata so that we have an entirely LLVM test flow that ensures the pass builder DTRT?

(I still would include the Clang side test which is also very useful to test integrating Clang w/ different flows through the pass manager.)

[echristo]

Fixed the clang test.

Tried to get something that I could reduce down and duplicate with just opt but it's been... difficult. Even the small clang testcase in isolation won't duplicate via something like:

clang -O0 -fexperimental-new-pass-manager foo.cc -S -o - -emit-llvm -mllvm -disable-llvm-optzns | opt -passes='default<O1>' -transform-warning -pass-remarks-missed=transform-warning -S -o -

Might be holding it wrong, but this isn't very discoverable if so :)

[echristo]

Fix comments around full unroller.

[vitalybuka]

Can this be landed?

[echristo]

Add a testcase with opt and command line option so we can enable it.

[chandlerc]

Wooot about finally having a test case! (Sorry for nit picking it a bit ....)

llvm/test/Transforms/LoopUnroll/FullUnroll.ll

4–6

Make sure it is in the correct function at least, and maybe after the label for the loop header?

12–17

Are both functions needed?

19–20

Nit, but minimize and clean this function up a touch?

At the least, removing all the target features seems valuable, and I'd give things stable names instead of numbered values.

[echristo]

Update and reduce testcase a bit.

[echristo]

OK, ready again :)

llvm/test/Transforms/LoopUnroll/FullUnroll.ll

4–6

Got it. There's not a lot of function left at the end:

define void @walrus() local_unnamed_addr #0 {
entry:

br label %for.cond.preheader

for.cond.preheader: ; preds = %for.cond.preheader, %entry

br label %for.cond.preheader

}

12–17

I thought so, but apparently not :)

19–20

Got it. Most things have stable names, only a few temporaries have numbered names.

[chandlerc]

clang/test/Misc/loop-opt-setup.c

12

This is dead now that you have different prefixes...

32–33

But for which function?

I'd rework these to be more modern CHECK-LABEL and affirmative checks for the unrolling.

llvm/test/Transforms/LoopUnroll/FullUnroll.ll

4–6

Then switch to generated CHECK lines?

The checks we have here will pass easily after incorrect edits that cause this test to not actually check what it intends to. I'd either check something that constructively shows unrolling or generate exact checks (if its small enough to be genuinely stable).

19–20

Sure, but even those will make future edits to this really frustrating with irrelevant diff, etc.

And this test case can still be minimized. As an example, the function is currently attributed as *optnone*. =/

I think you can just run this through the metarenamer pass?

[echristo]

Done. New diff incoming.

clang/test/Misc/loop-opt-setup.c

12

So it is. I'd actually missed this comment and saw it when fixing up other things. Thanks :)

[echristo]

Update and restructure some test cases.

[chandlerc]

Cool, thanks and LGTM!

[echristo]

Committed a while back.

[aeubanks]

I'm looking at enabling the -enable-npm-optnone flag and FullUnroll.ll fails. I understand that loop unrolling should be forced when some metadata is present, but the FullUnroll.ll test seems to check for a lot more than that. It checks for (roughly) two unconditional branches and no conditional branches. Running the test under -debug-pass-manager and --print-after-all, , it looks like InstCombine does a lot of work removing instructions, and SimplifyCFG is ultimately what removes all the conditional branches. But InstCombine and SimplifyCFG shouldn't run on an optnone function, right?

clang/test/Misc/loop-opt-setup.c

-// RUN: %clang -O1 -fexperimental-new-pass-manager -fno-unroll-loops -S -o - %s -emit-llvm | FileCheck %s
-// RUN: %clang -O1 -fno-experimental-new-pass-manager -fno-unroll-loops -S -o - %s -emit-llvm | FileCheck %s
+// RUN: %clang -O1 -fexperimental-new-pass-manager -fno-unroll-loops -S -o - %s -emit-llvm | FileCheck %s -check-prefix=CHECK-NEWPM
+// RUN: %clang -O1 -fno-experimental-new-pass-manager -fno-unroll-loops -S -o - %s -emit-llvm | FileCheck %s -check-prefix=CHECK-OLDPM
 extern int a[16];
 int b = 0;
 int foo(void) {
 #pragma unroll
   for (int i = 0; i < 16; ++i)
     a[i] = b += 2;
   return b;
 }
+// Check br i1 to make sure that the loop is fully unrolled
 // CHECK-NOT: br i1
 
+inline void Helper() {
+  const int *nodes[5];
+  int num_active = 5;
+
+  while (num_active) {
+#pragma clang loop unroll(full)
+    for (int i = 0; i < 5; ++i) {
+      if (nodes[i]) {
+        --num_active;
+      }
+    }
+  }
+}
+
+void Run() {
+  Helper();
+}
+
+// Check br i1 to make sure the loop is gone, there will still be a label branch for the infinite loop.
+// CHECK-NEWPM-NOT: br i1
+
+// The old pass manager doesn't remove the loop so check for 5 load i32*.
+// CHECK-OLDPM: Helper
+// CHECK-OLDPM: load i32*
+// CHECK-OLDPM: load i32*
+// CHECK-OLDPM: load i32*
+// CHECK-OLDPM: load i32*
+// CHECK-OLDPM: load i32*

llvm/lib/Passes/PassBuilder.cpp

PassBuilder::buildFunctionSimplificationPipeline(OptimizationLevel Level,
   LPM2.addPass(LoopIdiomRecognizePass());
 
   for (auto &C : LateLoopOptimizationsEPCallbacks)
     C(LPM2, Level);
 
   LPM2.addPass(LoopDeletionPass());
   // Do not enable unrolling in PreLinkThinLTO phase during sample PGO
   // because it changes IR to makes profile annotation in back compile
-  // inaccurate.
-  if ((Phase != ThinLTOPhase::PreLink || !PGOOpt ||
-       PGOOpt->Action != PGOOptions::SampleUse) &&
-      PTO.LoopUnrolling)
+  // inaccurate. The normal unroller doesn't pay attention to forced full unroll
+  // attributes so we need to make sure and allow the full unroll pass to pay
+  // attention to it.
+  if (Phase != ThinLTOPhase::PreLink || !PGOOpt ||
+      PGOOpt->Action != PGOOptions::SampleUse)
     LPM2.addPass(LoopFullUnrollPass(Level.getSpeedupLevel(),
-                                    /*OnlyWhenForced=*/false,
+                                    /* OnlyWhenForced= */ !PTO.LoopUnrolling,
                                     PTO.ForgetAllSCEVInLoopUnroll));
 
   for (auto &C : LoopOptimizerEndEPCallbacks)
     C(LPM2, Level);
 
   // We provide the opt remark emitter pass for LICM to use. We only need to do
   // this once as it is immutable.
   FPM.addPass(RequireAnalysisPass<OptimizationRemarkEmitterAnalysis, Function>());
   FPM.addPass(createFunctionToLoopPassAdaptor(
       std::move(LPM1), EnableMSSALoopDependency, DebugLogging));

llvm/test/Transforms/LoopUnroll/FullUnroll.ll

+; RUN: opt -passes='default<O1>' -disable-verify --mtriple x86_64-pc-linux-gnu -new-pm-disable-loop-unrolling=true \
+; RUN: -S -o - %s | FileCheck %s
+
+; We don't end up deleting the loop, but we remove everything inside of it so checking for any
+; reasonable instruction from the original loop will work.
+; CHECK-NOT: br i1
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+$_Z6Helperv = comdat any
+
+; Function Attrs: noinline optnone uwtable
+define dso_local void @_Z3Runv() #0 {
+entry:
+  call void @_Z6Helperv()
+  ret void
+}
+
+; Function Attrs: noinline nounwind optnone uwtable
+define linkonce_odr dso_local void @_Z6Helperv() #1 comdat {
+entry:
+  %nodes = alloca [5 x i32*], align 16
+  %num_active = alloca i32, align 4
+  %i = alloca i32, align 4
+  store i32 5, i32* %num_active, align 4
+  br label %while.cond
+
+while.cond:                                       ; preds = %for.end, %entry
+  %0 = load i32, i32* %num_active, align 4
+  %tobool = icmp ne i32 %0, 0
+  br i1 %tobool, label %while.body, label %while.end
+
+while.body:                                       ; preds = %while.cond
+  store i32 0, i32* %i, align 4
+  br label %for.cond
+
+for.cond:                                         ; preds = %for.inc, %while.body
+  %1 = load i32, i32* %i, align 4
+  %cmp = icmp slt i32 %1, 5
+  br i1 %cmp, label %for.body, label %for.end
+
+for.body:                                         ; preds = %for.cond
+  %2 = load i32, i32* %i, align 4
+  %idxprom = sext i32 %2 to i64
+  %arrayidx = getelementptr inbounds [5 x i32*], [5 x i32*]* %nodes, i64 0, i64 %idxprom
+  %3 = load i32*, i32** %arrayidx, align 8
+  %tobool1 = icmp ne i32* %3, null
+  br i1 %tobool1, label %if.then, label %if.end
+
+if.then:                                          ; preds = %for.body
+  %4 = load i32, i32* %num_active, align 4
+  %dec = add nsw i32 %4, -1
+  store i32 %dec, i32* %num_active, align 4
+  br label %if.end
+
+if.end:                                           ; preds = %if.then, %for.body
+  br label %for.inc
+
+for.inc:                                          ; preds = %if.end
+  %5 = load i32, i32* %i, align 4
+  %inc = add nsw i32 %5, 1
+  store i32 %inc, i32* %i, align 4
+  br label %for.cond, !llvm.loop !2
+
+for.end:                                          ; preds = %for.cond
+  br label %while.cond
+
+while.end:                                        ; preds = %while.cond
+  ret void
+}
+
+attributes #0 = { noinline optnone uwtable "correctly-rounded-divide-sqrt-fp-math"="false" "disable-tail-calls"="false" "frame-pointer"="all" "less-precise-fpmad"="false" "min-legal-vector-width"="0" "no-infs-fp-math"="false" "no-jump-tables"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+cx8,+fxsr,+mmx,+sse,+sse2,+x87" "unsafe-fp-math"="false" "use-soft-float"="false" }
+attributes #1 = { noinline nounwind optnone uwtable "correctly-rounded-divide-sqrt-fp-math"="false" "disable-tail-calls"="false" "frame-pointer"="all" "less-precise-fpmad"="false" "min-legal-vector-width"="0" "no-infs-fp-math"="false" "no-jump-tables"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+cx8,+fxsr,+mmx,+sse,+sse2,+x87" "unsafe-fp-math"="false" "use-soft-float"="false" }
+
+!llvm.module.flags = !{!0}
+!llvm.ident = !{!1}
+
+!0 = !{i32 1, !"wchar_size", i32 4}
+!1 = !{!"clang version 11.0.0 ([e-mail]:llvm/llvm-project.git 3ccd454c102b069d2230a18cfe16b84a5f005fc8)"}
+!2 = distinct !{!2, !3}
+!3 = !{!"llvm.loop.unroll.full"}

llvm/tools/opt/NewPMDriver.cpp

cl::desc("A textual description of the function pass pipeline inserted at "
              "the PipelineStart extension point into default pipelines"),
     cl::Hidden);
 static cl::opt<std::string> OptimizerLastEPPipeline(
     "passes-ep-optimizer-last",
     cl::desc("A textual description of the function pass pipeline inserted at "
              "the OptimizerLast extension point into default pipelines"),
     cl::Hidden);
 
+// Individual pipeline tuning options.
+static cl::opt<bool> DisableLoopUnrolling(
+    "new-pm-disable-loop-unrolling",
+    cl::desc("Disable loop unrolling in all relevant passes"), cl::init(false));
+
 extern cl::opt<PGOKind> PGOKindFlag;
 extern cl::opt<std::string> ProfileFile;
 extern cl::opt<CSPGOKind> CSPGOKindFlag;
 extern cl::opt<std::string> CSProfileGenFile;
 
 static cl::opt<std::string>
     ProfileRemappingFile("profile-remapping-file",
                          cl::desc("Path to the profile remapping file."),
if (CSPGOKindFlag == CSInstrGen) {
       P->CSAction = PGOOptions::CSIRUse;
     }
   }
   PassInstrumentationCallbacks PIC;
   StandardInstrumentations SI;
   SI.registerCallbacks(PIC);
 
   PipelineTuningOptions PTO;
+  // LoopUnrolling defaults on to true and DisableLoopUnrolling is initialized
+  // to false above so we shouldn't necessarily need to check whether or not the
+  // option has been enabled.
+  PTO.LoopUnrolling = !DisableLoopUnrolling;
   PTO.Coroutines = Coroutines;
   PassBuilder PB(TM, PTO, P, &PIC);
   registerEPCallbacks(PB, VerifyEachPass, DebugPM);
 
   // Load requested pass plugins and let them register pass builder callbacks
   for (auto &PluginFN : PassPlugins) {
     auto PassPlugin = PassPlugin::Load(PluginFN);
     if (!PassPlugin) {