[llvm-reduce] Assert that the number of chunks does not change with reductions

[aeubanks]

Summary

Followup to D113537.

[aeubanks]

remove logging

[Meinersbur]

Approach looks good to me.

remove-metadata-args.ll / --delta-passes=metadata is failing, which I might have been responsible for. Do you want me to have a look?

[aeubanks]

In D113816#3132317, @Meinersbur wrote:

Approach looks good to me.

remove-metadata-args.ll / --delta-passes=metadata is failing, which I might have been responsible for. Do you want me to have a look?

I have https://reviews.llvm.org/D113812 and https://reviews.llvm.org/D113808 out for that.

[Meinersbur]

In D113816#3132324, @aeubanks wrote:

I have https://reviews.llvm.org/D113812 and https://reviews.llvm.org/D113808 out for that.

Thanks. Assuming those go in, this patch LGTM.

[foad]

llvm/tools/llvm-reduce/deltas/Delta.cpp

129

Hi! I've hit this assertion failure because I'm using llvm-reduce to reduce a compile-time hog, so the interestingness test ("does it take longer than 1 second to compile") is not 100% consistent. Of course it seems to me like llvm-reduce ought to support this use case. Any thoughts?

[foad]

llvm/tools/llvm-reduce/deltas/Delta.cpp

129

In any case I think this should be a proper diagnostic, not an assert, like the "Input isn't interesting!" error.

[aeubanks]

llvm/tools/llvm-reduce/deltas/Delta.cpp

129

all of what you said sounds good to me

llvm/tools/llvm-reduce/deltas/Delta.cpp

for (auto C : Chunks) {
       errs() << '\n';
     }
   }
   return SplitOne;
 }
 
 /// Runs the Delta Debugging algorithm, splits the code into chunks and
 /// reduces the amount of chunks that are considered interesting by the
-/// given test.
+/// given test. The number of chunks is determined by a preliminary run of the
+/// reduction pass where no change must be made to the module.
 template <typename T>
 void runDeltaPassInt(
     TestRunner &Test,
     function_ref<void(Oracle &, T &)> ExtractChunksFromModule) {
+  assert(!verifyReducerWorkItem(Test.getProgram(), &errs()) &&
+         "input module is broken before making changes");
+
+  SmallString<128> CurrentFilepath;
+  if (!isReduced(Test.getProgram(), Test, CurrentFilepath)) {
+    errs() << "\nInput isn't interesting! Verify interesting-ness test\n";
+    exit(1);
+  }
+
   int Targets;
   {
-    // Count the number of targets by counting the number of calls to
+    // Count the number of chunks by counting the number of calls to
     // Oracle::shouldKeep() but always returning true so no changes are
     // made.
     std::vector<Chunk> AllChunks = {{0, INT_MAX}};
     Oracle Counter(AllChunks);
     ExtractChunksFromModule(Counter, Test.getProgram());
     Targets = Counter.count();
+
+    assert(!verifyReducerWorkItem(Test.getProgram(), &errs()) &&
+           "input module is broken after counting chunks");
+    assert(isReduced(Test.getProgram(), Test, CurrentFilepath) &&
+           "input module no longer interesting after counting chunks");
+
+#ifndef NDEBUG
+    // Make sure that the number of chunks does not change as we reduce.
+    std::vector<Chunk> NoChunks;
+    Oracle NoChunksCounter(NoChunks);
+    std::unique_ptr<ReducerWorkItem> Clone =
+        cloneReducerWorkItem(Test.getProgram());
+    Test.getProgram().print(errs());
+    Clone->print(errs());
+    ExtractChunksFromModule(NoChunksCounter, *Clone);
+    assert(Targets == NoChunksCounter.count() &&
+           "number of chunks changes when reducing");
+#endif
   }
   if (!Targets) {
     errs() << "\nNothing to reduce\n";
     return;
   }
 
-  SmallString<128> CurrentFilepath;
-  if (!isReduced(Test.getProgram(), Test, CurrentFilepath)) {
-    errs() << "\nInput isn't interesting! Verify interesting-ness test\n";
-    exit(1);
-  }
-
-  assert(!verifyReducerWorkItem(Test.getProgram(), &errs()) &&
-         "input module is broken before making changes");
-
   std::vector<Chunk> ChunksStillConsideredInteresting = {{0, Targets - 1}};
   std::unique_ptr<ReducerWorkItem> ReducedProgram;
 
   for (unsigned int Level = 0; Level < StartingGranularityLevel; Level++) {
     increaseGranularity(ChunksStillConsideredInteresting);
   }
 
   bool FoundAtLeastOneNewUninterestingChunkWithCurrentGranularity;