llvm-reduce: Fix tsan failures

[arsenm]

Summary

There's a data race on the UninterestingChunks set. The code seems to
be operating on the assumption that all the tasks completed, so ensure
the unused results do complete. This started showing up about 50% of
the time when running operands-skip-parallel.ll after the recent
switch to use DenseSet; previously it failed much less frequently with
std::set.

We should introduce a mechanism to early terminate unused
results. Alternatively, I've been thinking about ways to to make the
reduction order smarter. I frequently have tests that take multiple
minutes to compile and hit the failure. It may be helpful to see which
chunks took the least time and prefer those over just taking the first
result.

[arsenm]

ping, I keep getting occasional bot emails on this

[regehr]

LGTM!

[aeubanks]

llvm/tools/llvm-reduce/deltas/Delta.cpp

367

missing a word

[arsenm]

4ecdbf2e4ececec82cabb0d6e21f8397aa148b57

llvm/tools/llvm-reduce/deltas/Delta.cpp

if (std::unique_ptr<ReducerWorkItem> ChunkResult =
     raw_svector_ostream BCOS(Result);
     writeBitcode(*ChunkResult, BCOS);
     // Communicate that the task reduced a chunk.
     AnyReduced = true;
   }
   return Result;
 }
 
+using SharedTaskQueue = std::deque<std::shared_future<SmallString<0>>>;
+
+static void waitAndDiscardResultsBarrier(SharedTaskQueue &TaskQueue) {
+  while (!TaskQueue.empty()) {
+    auto &Future = TaskQueue.front();
+    Future.wait();
+    TaskQueue.pop_front();
+  }
+}
+
 /// Runs the Delta Debugging algorithm, splits the code into chunks and
 /// reduces the amount of chunks that are considered interesting by the
 /// given test. The number of chunks is determined by a preliminary run of the
 /// reduction pass where no change must be made to the module.
 void llvm::runDeltaPass(TestRunner &Test, ReductionFunc ExtractChunksFromModule,
                         StringRef Message) {
   assert(!verifyReducerWorkItem(Test.getProgram(), &errs()) &&
          "input module is broken before making changes");
do {
     // When running with more than one thread, serialize the original bitcode
     // to OriginalBC.
     SmallString<0> OriginalBC;
     if (NumJobs > 1) {
       raw_svector_ostream BCOS(OriginalBC);
       writeBitcode(Test.getProgram(), BCOS);
     }
 
-    std::deque<std::shared_future<SmallString<0>>> TaskQueue;
+    SharedTaskQueue TaskQueue;
     for (auto I = ChunksStillConsideredInteresting.rbegin(),
               E = ChunksStillConsideredInteresting.rend();
          I != E; ++I) {
       std::unique_ptr<ReducerWorkItem> Result = nullptr;
       unsigned WorkLeft = std::distance(I, E);
 
       // Run in parallel mode, if the user requested more than one thread and
       // there are at least a few chunks to process.
for (auto I = ChunksStillConsideredInteresting.rbegin(),
           }
 
           Result = std::make_unique<ReducerWorkItem>();
           MemoryBufferRef Data(StringRef(Res), "<bc file>");
           readBitcode(*Result, Data, Test.getProgram().M->getContext(),
                       Test.getToolName());
           break;
         }
+
+        // If we broke out of the loop, we still need to wait for everything to
+        // avoid race access to the chunk set.
+        //
+        // TODO: Create a way to remaining items we're ignoring; they could take
+        // a long time.
+        waitAndDiscardResultsBarrier(TaskQueue);
+
         // Forward I to the last chunk processed in parallel.
         I += NumChunksProcessed - 1;
       } else {
         Result = CheckChunk(
             *I,
             cloneReducerWorkItem(Test.getProgram(), Test.getTargetMachine()),
             Test, ExtractChunksFromModule, UninterestingChunks,
             ChunksStillConsideredInteresting);