[ThreadPool] Support returning futures with results.

[fhahn]

Summary

This patch adjusts ThreadPool::async to return futures that wrap
the result type of the passed in callable.

To do so, ThreadPool::asyncImpl first constructs a
std::packaged_task<ResTy()> object, then extracts
the future and converts the std::packaged_task<ResTy()>
to std::packaged_task<void()> before submitting it to the
queue.

[Meinersbur]

The pre-merge check has found additional locations where template arguments need to be added. Did you consider using using void as default template argument, or make ThreadPool a typedef of "ThreadPoolWithResult<void>"?

However, I don't think that the ThreadPool itself needs to be a template, but just the async implementation such that one can enqueue tasks with different return types to the same thread pool:

template <typename RetTy>
inline std::shared_future<RetTy> async(std::function<RetTy()> F);

AFAIU, std::packaged_task<> already handles return values itself.

Edit: I missed that packaged_task itself a template of the return type. However, after calling get_future, it it itself packaged into a std::function<void()> object that is pushed to the Tasks queue, i.e. does not depend on the result type anymore.

[fhahn]

In D114183#3141408, @Meinersbur wrote:

The pre-merge check has found additional locations where template arguments need to be added. Did you consider using using void as default template argument, or make ThreadPool a typedef of "ThreadPoolWithResult<void>"?

Thanks for the suggestion! I updated ThreadPool -> ThreadPoolWithResult and add using ThreadPool = ThreadPoolWithResult<void>, to avoid adjusting all users.

However, I don't think that the ThreadPool itself needs to be a template, but just the async implementation such that one can enqueue tasks with different return types to the same thread pool:

template <typename RetTy>
inline std::shared_future<RetTy> async(std::function<RetTy()> F);

AFAIU, std::packaged_task<> already handles return values itself.

Edit: I missed that packaged_task itself a template of the return type. However, after calling get_future, it it itself packaged into a std::function<void()> object that is pushed to the Tasks queue, i.e. does not depend on the result type anymore.

I tried using void with both TaskTy and PackagedTaskTy, but both resulted in build errors when sharing the Future.

[fhahn]

Adjust ThreadPool forward declaration in MLIRContext.h

[Meinersbur]

In D114183#3142329, @fhahn wrote:

I tried using void with both TaskTy and PackagedTaskTy, but both resulted in build errors when sharing the Future.

What were thoise build errors?

Implementing a non-templated ThreadPool should be possible, as demonstrated here.

[fhahn]

In D114183#3143037, @Meinersbur wrote:

In D114183#3142329, @fhahn wrote:

I tried using void with both TaskTy and PackagedTaskTy, but both resulted in build errors when sharing the Future.

What were thoise build errors?

Variants of

llvm/include/llvm/Support/ThreadPool.h:184:12: error: no viable conversion from returned value of type 'shared_future<void>' to function return type 'shared_future<llvm::SmallString<0>>'
    return Future.share();
           ^~~~~~~~~~~~~~

Implementing a non-templated ThreadPool should be possible, as demonstrated here.

I think the issue is that LLVM's ThreadPool uses std::packaged_task to store the task and extract the future, and the type used for the future depends on the type for the packaged_task, which in turn depends on the type of Task.

Are you suggesting adjusting the way we package stuff in LLVM's ThreadPool so the packaged task types do not depend on the result type of the packaged function? I think that should be an feasible alternative.

[mehdi_amini]

This seems like a bit of a "niche" extension: it makes it possible to return future but only for a statically predefined type. Is there a way to type-erase this in the threadpool a bit better and manager this at "enqueue time" instead so that each new task can return its own future?

Also isn't this feature something that can likely be layered on top of the existing ThreadPool without changing it?
For example by wrapping the submitted task into another object that manages the promise/future.

[fhahn]

In D114183#3143742, @mehdi_amini wrote:

This seems like a bit of a "niche" extension: it makes it possible to return future but only for a statically predefined type. Is there a way to type-erase this in the threadpool a bit better and manager this at "enqueue time" instead so that each new task can return its own future?

Also isn't this feature something that can likely be layered on top of the existing ThreadPool without changing it?
For example by wrapping the submitted task into another object that manages the promise/future.

I think that's what Michael was asking/suggestion in the previous comment :)

My main goal is to use futures with non-void results, I'm happy to go with whatever implementation people think is most useful. Introducing a predefined type was a mostly mechanical change, but as I said if changing the packaging is preferred, I can adjust the code.

[mehdi_amini]

In D114183#3143852, @fhahn wrote:

In D114183#3143742, @mehdi_amini wrote:

This seems like a bit of a "niche" extension: it makes it possible to return future but only for a statically predefined type. Is there a way to type-erase this in the threadpool a bit better and manager this at "enqueue time" instead so that each new task can return its own future?

Also isn't this feature something that can likely be layered on top of the existing ThreadPool without changing it?
For example by wrapping the submitted task into another object that manages the promise/future.

I think that's what Michael was asking/suggestion in the previous comment :)

Ah, I missed this! I think skimming at the link Michael provided, the ScheduleAndGetFuture is what I had in mind.

[Meinersbur]

In D114183#3143705, @fhahn wrote:

Variants of

llvm/include/llvm/Support/ThreadPool.h:184:12: error: no viable conversion from returned value of type 'shared_future<void>' to function return type 'shared_future<llvm::SmallString<0>>'
    return Future.share();
           ^~~~~~~~~~~~~~

This seems to be a mismatch of asyncImpl's future return type and the return type of the function passed to asyncImpl. Future should be of type shared_future<llvm::SmallString<0>> but somehow it is shared_future<void>. That is, both should be the same template parameter, like the template<typename RetTy> std::shared_future<RetTy> async(std::function<RetTy()> F); suggested before. But this is requirements is only with the asyncImpl defintion.

More crucially, the line

Tasks.push(std::move(PackagedTask));

wraps the PackagedTask into a std::function<void()> which does not depend on the return type anymore. That is, we don't need ThreadPool::Tasks to be a template of the return type, i.e. it can store tasks of any return type (I think the actually result value is stored as a closure parameter of the std::function<void()> object, the std::future can be used to retrieve it), the ThreadPool itself does not need to store the future which was only returned by asyncImpl for the caller to remember.

[fhahn]

Updated so there's no need for adding a template argument to ThreadPool.

In D114183#3144443, @Meinersbur wrote:

More crucially, the line

Tasks.push(std::move(PackagedTask));

wraps the PackagedTask into a std::function<void()> which does not depend on the return type anymore. That is, we don't need ThreadPool::Tasks to be a template of the return type, i.e. it can store tasks of any return type (I think the actually result value is stored as a closure parameter of the std::function<void()> object, the std::future can be used to retrieve it), the ThreadPool itself does not need to store the future which was only returned by asyncImpl for the caller to remember.

Thanks! It looks like it is possible to first construct a std::packaged_task<RetTy()>, extract the future and then move the object into a std::packaged_task<void()>.

It required a few changes, but it looks like it is working now without the extra template argument. It also seems async also needs to convert it's argument to a std::function object before calling asyncImpl, so asyncImpl can determine the correct template argument types for std::function.

[mehdi_amini]

You have some build failure on Windows: https://buildkite.com/llvm-project/premerge-checks/builds/66414#450a0afb-7b1b-4cbb-8654-6de57c1f4e51

[fhahn]

attempt to work around windows build failure.

[fhahn]

another stab at trying to figure out what's going on with the windows build

[fhahn]

Replace uses of packaged_tasks with manually creating promise and callable to set result, to work around MSVC build failures.

[fhahn]

I think this should be ready for another look now. Windows tests are passing now and the failure on Debian is unrelated to the patch I think.

Unfortunately I wasn't able to find a way to convert std::packaged_task<ResTy()> to std::packaged_task<void()> with MSVC. To work around that, I updated the code to manually create a promise and wrap the task function into another callable that calls the function and updates the promise.

[DanielMcIntosh-IBM]

It seems like you may have missed some issues with const vs. non-const types in the case when LLVM_ENABLE_THREADS is false.
I think it should be pretty easy to fix, but I don't know ThreadPool well enough to quickly address it myself.
This is the error:

llvm/include/llvm/Support/ThreadPool.h:127:29: error: 'this' argument to member function 'get' has type 'const std::future<void>', but function is not marked const
    Tasks.push([Future]() { Future.get(); });
                            ^~~~~~
llvm/include/llvm/Support/ThreadPool.h:60:12: note: in instantiation of function template specialization 'llvm::ThreadPool::asyncImpl<void>' requested here
    return asyncImpl(std::function<decltype(F())()>(std::forward<Func>(F)));
           ^
llvm/include/llvm/Support/ThreadPool.h:54:12: note: in instantiation of function template specialization 'llvm::ThreadPool::async<std::__bind<(lambda at llvm/lib/CodeGen/ParallelCG.cpp:80:15), llvm::SmallString<0>>>' requested here
    return async(std::move(Task));
           ^
future:1239:10: note: 'get' declared here
    void get();

Do you mind submitting a fix?

[thakis]

In D114183#3151653, @DanielMcIntosh-IBM wrote:

It seems like you may have missed some issues with const vs. non-const types in the case when LLVM_ENABLE_THREADS is false.
I think it should be pretty easy to fix, but I don't know ThreadPool well enough to quickly address it myself.
This is the error:

llvm/include/llvm/Support/ThreadPool.h:127:29: error: 'this' argument to member function 'get' has type 'const std::future<void>', but function is not marked const
    Tasks.push([Future]() { Future.get(); });
                            ^~~~~~
llvm/include/llvm/Support/ThreadPool.h:60:12: note: in instantiation of function template specialization 'llvm::ThreadPool::asyncImpl<void>' requested here
    return asyncImpl(std::function<decltype(F())()>(std::forward<Func>(F)));
           ^
llvm/include/llvm/Support/ThreadPool.h:54:12: note: in instantiation of function template specialization 'llvm::ThreadPool::async<std::__bind<(lambda at llvm/lib/CodeGen/ParallelCG.cpp:80:15), llvm::SmallString<0>>>' requested here
    return async(std::move(Task));
           ^
future:1239:10: note: 'get' declared here
    void get();

Do you mind submitting a fix?

Your revert breaks normal building: http://45.33.8.238/linux/61593/step_4.txt

PTAL.

[ronlieb]

Both of our amdgpu buildbots are broken by this commit, when I revert it locally my build will pass.
https://lab.llvm.org/buildbot/#/builders/193
https://lab.llvm.org/buildbot/#/builders/165

https://lab.llvm.org/buildbot/#/builders/165/builds/10841
https://lab.llvm.org/buildbot/#/builders/193/builds/2281

[DanielMcIntosh-IBM]

Your revert breaks normal building: http://45.33.8.238/linux/61593/step_4.txt

PTAL.

Ooops, sorry. Working on it

[fhahn]

In D114183#3154540, @ronlieb wrote:

Both of our amdgpu buildbots are broken by this commit, when I revert it locally my build will pass.
https://lab.llvm.org/buildbot/#/builders/193
https://lab.llvm.org/buildbot/#/builders/165

https://lab.llvm.org/buildbot/#/builders/165/builds/10841
https://lab.llvm.org/buildbot/#/builders/193/builds/2281

I recommitted the patch. for now and also submitted a fix for the build failure, which was caused by the return type of the future having been changed from auto to std::future bc41653a1f289a63a1472b7364437515c9659f77. Everything should build again now.

[Meinersbur]

Could you close the review again if done?

[fhahn]

In D114183#3158894, @Meinersbur wrote:

Could you close the review again if done?

I'm not sure how unfortunately :( I can't see an option to close at the moment.

[mehdi_amini]

In D114183#3159449, @fhahn wrote:

In D114183#3158894, @Meinersbur wrote:

Could you close the review again if done?

I'm not sure how unfortunately :( I can't see an option to close at the moment.

You can only "Abandon Revision" in the list of actions at the bottom.

[mehdi_amini]

Actually the "Close Revision" was hidden because of the "requested changes" from Daniel, you should see it now

[Meinersbur]

I try to make it disappear from my Phabricator dashboard. Seems the only way is to resign as a reviewer :-(

llvm/include/llvm/Support/ThreadPool.h

 #include <functional>
 #include <memory>
 #include <mutex>
 #include <queue>
 #include <utility>
 
 namespace llvm {
 
-/// A ThreadPool for asynchronous parallel execution on a defined number of
-/// threads.
+/// A ThreadPoolWithResult for asynchronous parallel execution on a defined
+/// number of threads.
 ///
 /// The pool keeps a vector of threads alive, waiting on a condition variable
 /// for some work to become available.
-class ThreadPool {
+template <typename FutureResTy> class ThreadPoolWithResult {
 public:
-  using TaskTy = std::function<void()>;
-  using PackagedTaskTy = std::packaged_task<void()>;
+  using TaskTy = std::function<FutureResTy()>;
+  using PackagedTaskTy = std::packaged_task<FutureResTy()>;
 
   /// Construct a pool using the hardware strategy \p S for mapping hardware
   /// execution resources (threads, cores, CPUs)
   /// Defaults to using the maximum execution resources in the system, but
   /// accounting for the affinity mask.
-  ThreadPool(ThreadPoolStrategy S = hardware_concurrency());
+  ThreadPoolWithResult(ThreadPoolStrategy S = hardware_concurrency())
+      : ThreadCount(S.compute_thread_count()) {
+#ifdef LLVM_ENABLE_THREADS
+    // Create ThreadCount threads that will loop forever, wait on QueueCondition
+    // for tasks to be queued or the Pool to be destroyed.
+    Threads.reserve(ThreadCount);
+    for (unsigned ThreadID = 0; ThreadID < ThreadCount; ++ThreadID) {
+      Threads.emplace_back([S, ThreadID, this] {
+        S.apply_thread_strategy(ThreadID);
+        while (true) {
+          PackagedTaskTy Task;
+          {
+            std::unique_lock<std::mutex> LockGuard(QueueLock);
+            // Wait for tasks to be pushed in the queue
+            QueueCondition.wait(LockGuard,
+                                [&] { return !EnableFlag || !Tasks.empty(); });
+            // Exit condition
+            if (!EnableFlag && Tasks.empty())
+              return;
+            // Yeah, we have a task, grab it and release the lock on the queue
+
+            // We first need to signal that we are active before popping the
+            // queue in order for wait() to properly detect that even if the
+            // queue is empty, there is still a task in flight.
+            ++ActiveThreads;
+            Task = std::move(Tasks.front());
+            Tasks.pop();
+          }
+          // Run the task we just grabbed
+          Task();
+
+          bool Notify;
+          {
+            // Adjust `ActiveThreads`, in case someone waits on
+            // ThreadPoolWithResult::wait()
+            std::lock_guard<std::mutex> LockGuard(QueueLock);
+            --ActiveThreads;
+            Notify = workCompletedUnlocked();
+          }
+          // Notify task completion if this is the last active thread, in case
+          // someone waits on ThreadPoolWithResult::wait().
+          if (Notify)
+            CompletionCondition.notify_all();
+        }
+      });
+    }
+#else // LLVM_ENABLE_THREADS Disabled
+    if (ThreadCount != 1) {
+      errs() << "Warning: request a ThreadPoolWithResult with " << ThreadCount
+             << " threads, but LLVM_ENABLE_THREADS has been turned off\n";
+    }
+#endif
+  }
 
   /// Blocking destructor: the pool will wait for all the threads to complete.
-  ~ThreadPool();
+  ~ThreadPoolWithResult() {
+#if LLVM_ENABLE_THREADS
+    {
+      std::unique_lock<std::mutex> LockGuard(QueueLock);
+      EnableFlag = false;
+    }
+    QueueCondition.notify_all();
+    for (auto &Worker : Threads)
+      Worker.join();
+#else // LLVM_ENABLE_THREADS Disabled
+    wait();
+#endif
+  }
 
   /// Asynchronous submission of a task to the pool. The returned future can be
   /// used to wait for the task to finish and is *non-blocking* on destruction.
   template <typename Function, typename... Args>
-  inline std::shared_future<void> async(Function &&F, Args &&... ArgList) {
+  inline std::shared_future<FutureResTy> async(Function &&F,
+                                               Args &&...ArgList) {
     auto Task =
         std::bind(std::forward<Function>(F), std::forward<Args>(ArgList)...);
     return asyncImpl(std::move(Task));
   }
 
   /// Asynchronous submission of a task to the pool. The returned future can be
   /// used to wait for the task to finish and is *non-blocking* on destruction.
   template <typename Function>
-  inline std::shared_future<void> async(Function &&F) {
+  inline std::shared_future<FutureResTy> async(Function &&F) {
     return asyncImpl(std::forward<Function>(F));
   }
 
   /// Blocking wait for all the threads to complete and the queue to be empty.
   /// It is an error to try to add new tasks while blocking on this call.
-  void wait();
+  void wait() {
+#if LLVM_ENABLE_THREADS
+    // Wait for all threads to complete and the queue to be empty
+    std::unique_lock<std::mutex> LockGuard(QueueLock);
+    CompletionCondition.wait(LockGuard,
+                             [&] { return workCompletedUnlocked(); });
+
+#else // LLVM_ENABLE_THREADS Disabled
+
+    // Sequential implementation running the tasks
+    while (!Tasks.empty()) {
+      auto Task = std::move(Tasks.front());
+      Tasks.pop();
+      Task();
+    }
+#endif
+  }
 
   unsigned getThreadCount() const { return ThreadCount; }
 
   /// Returns true if the current thread is a worker thread of this thread pool.
-  bool isWorkerThread() const;
+  bool isWorkerThread() const {
+    llvm::thread::id CurrentThreadId = llvm::this_thread::get_id();
+    for (const llvm::thread &Thread : Threads)
+      if (CurrentThreadId == Thread.get_id())
+        return true;
+    return false;
+  }
 
 private:
   bool workCompletedUnlocked() { return !ActiveThreads && Tasks.empty(); }
 
   /// Asynchronous submission of a task to the pool. The returned future can be
   /// used to wait for the task to finish and is *non-blocking* on destruction.
-  std::shared_future<void> asyncImpl(TaskTy F);
+  std::shared_future<FutureResTy> asyncImpl(TaskTy Task) {
+#ifdef LLVM_ENABLE_THREADS
+    /// Wrap the Task in a packaged_task to return a future object.
+    PackagedTaskTy PackagedTask(std::move(Task));
+    auto Future = PackagedTask.get_future();
+    {
+      // Lock the queue and push the new task
+      std::unique_lock<std::mutex> LockGuard(QueueLock);
+
+      // Don't allow enqueueing after disabling the pool
+      assert(EnableFlag &&
+             "Queuing a thread during ThreadPoolWithResult destruction");
+
+      Tasks.push(std::move(PackagedTask));
+    }
+    QueueCondition.notify_one();
+    return Future.share();
+
+#else // LLVM_ENABLE_THREADS Disabled
+
+    // Get a Future with launch::deferred execution using std::async
+    auto Future = std::async(std::launch::deferred, std::move(Task)).share();
+    // Wrap the future so that both ThreadPoolWithResult::wait() can operate and
+    // the returned future can be sync'ed on.
+    PackagedTaskTy PackagedTask([Future]() { Future.get(); });
+    Tasks.push(std::move(PackagedTask));
+    return Future;
+#endif
+  }
 
   /// Threads in flight
   std::vector<llvm::thread> Threads;
 
   /// Tasks waiting for execution in the pool.
   std::queue<PackagedTaskTy> Tasks;
 
   /// Locking and signaling for accessing the Tasks queue.
   std::mutex QueueLock;
   std::condition_variable QueueCondition;
 
   /// Signaling for job completion
   std::condition_variable CompletionCondition;
 
   /// Keep track of the number of thread actually busy
   unsigned ActiveThreads = 0;
 
 #if LLVM_ENABLE_THREADS // avoids warning for unused variable
   /// Signal for the destruction of the pool, asking thread to exit.
   bool EnableFlag = true;
 #endif
 
   unsigned ThreadCount;
 };
+
+using ThreadPool = ThreadPoolWithResult<void>;
 }
 
 #endif // LLVM_SUPPORT_THREADPOOL_H

llvm/lib/Support/CMakeLists.txt

add_llvm_component_library(LLVMSupport
   StringMap.cpp
   StringSaver.cpp
   StringRef.cpp
   SuffixTree.cpp
   SymbolRemappingReader.cpp
   SystemUtils.cpp
   TarWriter.cpp
   TargetParser.cpp
-  ThreadPool.cpp
   TimeProfiler.cpp
   Timer.cpp
   ToolOutputFile.cpp
   TrigramIndex.cpp
   Triple.cpp
   Twine.cpp
   TypeSize.cpp
   Unicode.cpp

llvm/lib/Support/ThreadPool.cpp

-//==-- llvm/Support/ThreadPool.cpp - A ThreadPool implementation -*- C++ -*-==//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-//
-// This file implements a crude C++11 based thread pool.
-//
-//===----------------------------------------------------------------------===//
-
-#include "llvm/Support/ThreadPool.h"
-
-#include "llvm/Config/llvm-config.h"
-#include "llvm/Support/Threading.h"
-#include "llvm/Support/raw_ostream.h"
-
-using namespace llvm;
-
-#if LLVM_ENABLE_THREADS
-
-ThreadPool::ThreadPool(ThreadPoolStrategy S)
-    : ThreadCount(S.compute_thread_count()) {
-  // Create ThreadCount threads that will loop forever, wait on QueueCondition
-  // for tasks to be queued or the Pool to be destroyed.
-  Threads.reserve(ThreadCount);
-  for (unsigned ThreadID = 0; ThreadID < ThreadCount; ++ThreadID) {
-    Threads.emplace_back([S, ThreadID, this] {
-      S.apply_thread_strategy(ThreadID);
-      while (true) {
-        PackagedTaskTy Task;
-        {
-          std::unique_lock<std::mutex> LockGuard(QueueLock);
-          // Wait for tasks to be pushed in the queue
-          QueueCondition.wait(LockGuard,
-                              [&] { return !EnableFlag || !Tasks.empty(); });
-          // Exit condition
-          if (!EnableFlag && Tasks.empty())
-            return;
-          // Yeah, we have a task, grab it and release the lock on the queue
-
-          // We first need to signal that we are active before popping the queue
-          // in order for wait() to properly detect that even if the queue is
-          // empty, there is still a task in flight.
-          ++ActiveThreads;
-          Task = std::move(Tasks.front());
-          Tasks.pop();
-        }
-        // Run the task we just grabbed
-        Task();
-
-        bool Notify;
-        {
-          // Adjust `ActiveThreads`, in case someone waits on ThreadPool::wait()
-          std::lock_guard<std::mutex> LockGuard(QueueLock);
-          --ActiveThreads;
-          Notify = workCompletedUnlocked();
-        }
-        // Notify task completion if this is the last active thread, in case
-        // someone waits on ThreadPool::wait().
-        if (Notify)
-          CompletionCondition.notify_all();
-      }
-    });
-  }
-}
-
-void ThreadPool::wait() {
-  // Wait for all threads to complete and the queue to be empty
-  std::unique_lock<std::mutex> LockGuard(QueueLock);
-  CompletionCondition.wait(LockGuard, [&] { return workCompletedUnlocked(); });
-}
-
-bool ThreadPool::isWorkerThread() const {
-  llvm::thread::id CurrentThreadId = llvm::this_thread::get_id();
-  for (const llvm::thread &Thread : Threads)
-    if (CurrentThreadId == Thread.get_id())
-      return true;
-  return false;
-}
-
-std::shared_future<void> ThreadPool::asyncImpl(TaskTy Task) {
-  /// Wrap the Task in a packaged_task to return a future object.
-  PackagedTaskTy PackagedTask(std::move(Task));
-  auto Future = PackagedTask.get_future();
-  {
-    // Lock the queue and push the new task
-    std::unique_lock<std::mutex> LockGuard(QueueLock);
-
-    // Don't allow enqueueing after disabling the pool
-    assert(EnableFlag && "Queuing a thread during ThreadPool destruction");
-
-    Tasks.push(std::move(PackagedTask));
-  }
-  QueueCondition.notify_one();
-  return Future.share();
-}
-
-// The destructor joins all threads, waiting for completion.
-ThreadPool::~ThreadPool() {
-  {
-    std::unique_lock<std::mutex> LockGuard(QueueLock);
-    EnableFlag = false;
-  }
-  QueueCondition.notify_all();
-  for (auto &Worker : Threads)
-    Worker.join();
-}
-
-#else // LLVM_ENABLE_THREADS Disabled
-
-// No threads are launched, issue a warning if ThreadCount is not 0
-ThreadPool::ThreadPool(ThreadPoolStrategy S)
-    : ThreadCount(S.compute_thread_count()) {
-  if (ThreadCount != 1) {
-    errs() << "Warning: request a ThreadPool with " << ThreadCount
-           << " threads, but LLVM_ENABLE_THREADS has been turned off\n";
-  }
-}
-
-void ThreadPool::wait() {
-  // Sequential implementation running the tasks
-  while (!Tasks.empty()) {
-    auto Task = std::move(Tasks.front());
-    Tasks.pop();
-    Task();
-  }
-}
-
-std::shared_future<void> ThreadPool::asyncImpl(TaskTy Task) {
-  // Get a Future with launch::deferred execution using std::async
-  auto Future = std::async(std::launch::deferred, std::move(Task)).share();
-  // Wrap the future so that both ThreadPool::wait() can operate and the
-  // returned future can be sync'ed on.
-  PackagedTaskTy PackagedTask([Future]() { Future.get(); });
-  Tasks.push(std::move(PackagedTask));
-  return Future;
-}
-
-ThreadPool::~ThreadPool() { wait(); }
-
-#endif

mlir/include/mlir/IR/MLIRContext.h

 
 #include "mlir/Support/LLVM.h"
 #include "mlir/Support/TypeID.h"
 #include <functional>
 #include <memory>
 #include <vector>
 
 namespace llvm {
-class ThreadPool;
+template <typename T>
+class ThreadPoolWithResult;
+using ThreadPool = ThreadPoolWithResult<void>;
 } // end namespace llvm
 
 namespace mlir {
 class DebugActionManager;
 class DiagnosticEngine;
 class Dialect;
 class DialectRegistry;
 class InFlightDiagnostic;