Add llvm::llvm_fast_shutdown()

[nhaehnle]

Summary

LLD uses an unclean process exit to shut the process down faster, but
still wants the benefit of some side-effects that have traditionally
been invoked via llvm_shutdown() and ManagedStatic destruction.

The new llvm_fast_shutdown() function provides a more explicit alternative
solution that will still work when ManagedStatic is removed. Incidentally,
it also reduces the amount of cleanup code that is run even further.

This change also removes the use of ManagedStatic in the affected
components.

Note that the StatLock is moved into the StatisticInfo class. This is a
robust solution to guarantee that the StatLock destructor doesn't run
too early.

[lattner]

I missed something, why is ManagedStatic getting removed? Is there a discussion of this somewhere?

[efriedma]

Context: https://discourse.llvm.org/t/making-llvm-play-nice-r-when-used-as-a-shared-library-in-a-plugin-setting .

The constraints here are the following:

• Instead of relying on users calling llvm_shutdown() before unloading LLVM, we want to rely on the native __cxa_atexit or equivalent functionality.
• We want to avoid global constructors.

The only way to completely avoid global constructors is to avoid global variables with runtime constructors or destructors. There isn't any way portable way to retrofit ManagedStatic to work the way we want. So we need to get rid of it. (In theory, I guess we could make ManagedStatic call __cxa_atexit, but it's hard to generalize that.)

[nhaehnle]

• rebase
• add documentation to the ProgrammersManual
• flush output streams

[efriedma]

Do we also want to handle dumping debug counters, if that's enabled?

llvm/lib/Support/Parallel.cpp

32

Do we need to worry about races accessing this boolean?

llvm/lib/Support/Statistic.cpp

100–105

Do we need to worry about data races accessing this boolean?

[efriedma]

llvm/lib/Support/Parallel.cpp

32

Looking again, maybe it's not an issue. It's only accessed in two places: the ThreadPoolExecutor constructor, and fast_shutdown_parallel(). There can't be two threads executing the ThreadPoolExecutor constructor, and fast_shutdown_parallel should not be called if there are other threads running.

Maybe worth explaining that in a comment, though.

[nhaehnle]

In D129119#3688250, @efriedma wrote:

Do we also want to handle dumping debug counters, if that's enabled?

Hmm, I think you're right. I will revisit that, also the (non-)races.

[nhaehnle]

Rebase

[nhaehnle]

• add some notes about data race safety
• add the DebugCounters to the fast shutdown path

[efriedma]

LGTM

[MaskRay]

lld/Common/ErrorHandler.cpp

102–104

also mention -stats

[MaskRay]

lld/Common/ErrorHandler.cpp

102–104

To be clear, the wording should be in a way that doesn't assume -time-passes as the only exception.

lld/Common/ErrorHandler.cpp

 
 #include "llvm/Support/Parallel.h"
 
 #include "lld/Common/CommonLinkerContext.h"
 #include "llvm/ADT/Twine.h"
 #include "llvm/IR/DiagnosticInfo.h"
 #include "llvm/IR/DiagnosticPrinter.h"
 #include "llvm/Support/CrashRecoveryContext.h"
-#include "llvm/Support/ManagedStatic.h"
+#include "llvm/Support/FastShutdown.h"
 #include "llvm/Support/Process.h"
 #include "llvm/Support/Program.h"
 #include "llvm/Support/raw_ostream.h"
 #include <regex>
 
 using namespace llvm;
 using namespace lld;
 
if (hasContext()) {
     if (e.outputBuffer)
       e.outputBuffer->discard();
   }
 
   // Re-throw a possible signal or exception once/if it was catched by
   // safeLldMain().
   CrashRecoveryContext::throwIfCrash(val);
 
-  // Dealloc/destroy ManagedStatic variables before calling _exit().
-  // In an LTO build, allows us to get the output of -time-passes.
-  // Ensures that the thread pool for the parallel algorithms is stopped to
-  // avoid intermittent crashes on Windows when exiting.
+  // We will call _exit() but want to get the output of -time-passes in an LTO
+  // build. Also ensures that the thread pool for the parallel algorithms is
+  // stopped to avoid intermittent crashes on Windows when exiting.
   if (!CrashRecoveryContext::GetCurrent())
-    llvm_shutdown();
+    llvm_fast_shutdown();
 
   if (hasContext())
     lld::errorHandler().flushStreams();
 
   // When running inside safeLldMain(), restore the control flow back to the
   // CrashRecoveryContext. Otherwise simply use _exit(), meanning no cleanup,
   // since we want to avoid further crashes on shutdown.
   llvm::sys::Process::Exit(val, /*NoCleanup=*/true);

llvm/include/llvm/ADT/Statistic.h

 //===-- llvm/ADT/Statistic.h - Easy way to expose stats ---------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 ///
 /// \file
 /// This file defines the 'Statistic' class, which is designed to be an easy way
 /// to expose various metrics from passes.  These statistics are printed at the
-/// end of a run (from llvm_shutdown), when the -stats command line option is
-/// passed on the command line.
+/// end of a run (from global destructors or when llvm_fast_shutdown is called),
+/// when the -stats command line option is passed on the command line.
 ///
 /// This is useful for reporting information like the number of instructions
 /// simplified, optimized or removed by various transformations, like this:
 ///
 /// static Statistic NumInstsKilled("gcse", "Number of instructions killed");
 ///
 /// Later, in the code: ++NumInstsKilled;
 ///

llvm/include/llvm/Support/FastShutdown.h

+//===-- llvm/Support/FastShutdown.h -----------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Helpers to allow a fast process shutdown without the normal cleanups
+// (e.g., using _exit) but with some desirable side-effects that usually happen
+// when global static destructors are run.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_SUPPORT_FASTSHUTDOWN_H
+#define LLVM_SUPPORT_FASTSHUTDOWN_H
+
+namespace llvm {
+
+/// Run a small set of operations that usually run from global static
+/// destructors, but which have side-effects (e.g., printing statistics) that
+/// are desirable for tools that want to do a fast process exit without cleanups
+/// (e.g., using _exit).
+///
+/// Note that if the process exits (or LLVM is unloaded) normally, calling this
+/// function isn't necessary and should be avoided.
+///
+/// IMPORTANT: It's only safe to call llvm_fast_shutdown() in a single thread,
+/// without any other threads executing LLVM APIs. No LLVM APIs should be used
+/// after calling this function (other than sys::Process::Exit).
+void llvm_fast_shutdown();
+
+void fast_shutdown_statistics();
+void fast_shutdown_parallel();
+
+} // namespace llvm
+
+#endif // LLVM_SUPPORT_FASTSHUTDOWN_H

llvm/lib/Support/CMakeLists.txt

add_llvm_component_library(LLVMSupport
   DivisionByConstantInfo.cpp
   DAGDeltaAlgorithm.cpp
   DJB.cpp
   ELFAttributeParser.cpp
   ELFAttributes.cpp
   Error.cpp
   ErrorHandling.cpp
   ExtensibleRTTI.cpp
+  FastShutdown.cpp
   FileCollector.cpp
   FileUtilities.cpp
   FileOutputBuffer.cpp
   FoldingSet.cpp
   FormattedStream.cpp
   FormatVariadic.cpp
   GlobPattern.cpp
   GraphWriter.cpp

llvm/lib/Support/FastShutdown.cpp

+//===-- FastShutdown.cpp --------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Support/FastShutdown.h"
+
+using namespace llvm;
+
+void llvm::llvm_fast_shutdown() {
+  fast_shutdown_statistics();
+  fast_shutdown_parallel();
+}

llvm/lib/Support/Parallel.cpp

 //===- llvm/Support/Parallel.cpp - Parallel algorithms --------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Support/Parallel.h"
 #include "llvm/Config/llvm-config.h"
-#include "llvm/Support/ManagedStatic.h"
+#include "llvm/Support/FastShutdown.h"
 #include "llvm/Support/Threading.h"
 
 #include <atomic>
 #include <future>
 #include <stack>
 #include <thread>
 #include <vector>
 
 llvm::ThreadPoolStrategy llvm::parallel::strategy;
 
 #if LLVM_ENABLE_THREADS
 
 namespace llvm {
 namespace parallel {
 namespace detail {
 
 namespace {
 
+class ThreadPoolExecutor;
+
+static bool DefaultExecutorInitialized = false;
+
 /// An abstract class that takes closures and runs them asynchronously.
 class Executor {
 public:
   virtual ~Executor() = default;
   virtual void add(std::function<void()> func) = 0;
 
-  static Executor *getDefaultExecutor();
+  static ThreadPoolExecutor *getDefaultExecutor();
 };
 
 /// An implementation of an Executor that runs closures on a thread pool
 ///   in filo order.
 class ThreadPoolExecutor : public Executor {
 public:
   explicit ThreadPoolExecutor(ThreadPoolStrategy S = hardware_concurrency()) {
+    assert(!DefaultExecutorInitialized);
+    DefaultExecutorInitialized = true;
+
     unsigned ThreadCount = S.compute_thread_count();
     // Spawn all but one of the threads in another thread as spawning threads
     // can take a while.
     Threads.reserve(ThreadCount);
     Threads.resize(1);
     std::lock_guard<std::mutex> Lock(Mutex);
     Threads[0] = std::thread([this, ThreadCount, S] {
       for (unsigned I = 1; I < ThreadCount; ++I) {
~ThreadPoolExecutor() override {
     std::thread::id CurrentThreadId = std::this_thread::get_id();
     for (std::thread &T : Threads)
       if (T.get_id() == CurrentThreadId)
         T.detach();
       else
         T.join();
   }
 
-  struct Creator {
-    static void *call() { return new ThreadPoolExecutor(strategy); }
-  };
-  struct Deleter {
-    static void call(void *Ptr) { ((ThreadPoolExecutor *)Ptr)->stop(); }
-  };
-
   void add(std::function<void()> F) override {
     {
       std::lock_guard<std::mutex> Lock(Mutex);
       WorkStack.push(std::move(F));
     }
     Cond.notify_one();
   }
 
private:
   std::atomic<bool> Stop{false};
   std::stack<std::function<void()>> WorkStack;
   std::mutex Mutex;
   std::condition_variable Cond;
   std::promise<void> ThreadsCreated;
   std::vector<std::thread> Threads;
 };
 
-Executor *Executor::getDefaultExecutor() {
-  // The ManagedStatic enables the ThreadPoolExecutor to be stopped via
-  // llvm_shutdown() which allows a "clean" fast exit, e.g. via _exit(). This
-  // stops the thread pool and waits for any worker thread creation to complete
-  // but does not wait for the threads to finish. The wait for worker thread
-  // creation to complete is important as it prevents intermittent crashes on
-  // Windows due to a race condition between thread creation and process exit.
-  //
-  // The ThreadPoolExecutor will only be destroyed when the static unique_ptr to
-  // it is destroyed, i.e. in a normal full exit. The ThreadPoolExecutor
-  // destructor ensures it has been stopped and waits for worker threads to
-  // finish. The wait is important as it prevents intermittent crashes on
-  // Windows when the process is doing a full exit.
-  //
-  // The Windows crashes appear to only occur with the MSVC static runtimes and
-  // are more frequent with the debug static runtime.
-  //
-  // This also prevents intermittent deadlocks on exit with the MinGW runtime.
-
-  static ManagedStatic<ThreadPoolExecutor, ThreadPoolExecutor::Creator,
-                       ThreadPoolExecutor::Deleter>
-      ManagedExec;
-  static std::unique_ptr<ThreadPoolExecutor> Exec(&(*ManagedExec));
-  return Exec.get();
+ThreadPoolExecutor *Executor::getDefaultExecutor() {
+  static ThreadPoolExecutor Exec;
+  return &Exec;
 }
 } // namespace
 
 static std::atomic<int> TaskGroupInstances;
 
 // Latch::sync() called by the dtor may cause one thread to block. If is a dead
 // lock if all threads in the default executor are blocked. To prevent the dead
 // lock, only allow the first TaskGroup to run tasks parallelly. In the scenario
void TaskGroup::spawn(std::function<void()> F) {
   }
 }
 
 } // namespace detail
 } // namespace parallel
 } // namespace llvm
 #endif // LLVM_ENABLE_THREADS
 
+void llvm::fast_shutdown_parallel() {
+#if LLVM_ENABLE_THREADS
+  // Stop the executor, but only if it has been started.
+  //
+  // This allows a "clean" fast exit, via llvm_fast_shutdown() and e.g. _exit().
+  // This stops the thread pool and waits for any worker thread creation to
+  // complete but does not wait for the threads to finish. The wait for worker
+  // thread creation to complete is important as it prevents intermittent
+  // crashes on Windows due to a race condition between thread creation and
+  // process exit.
+  //
+  // The ThreadPoolExecutor will only be destroyed when the static unique_ptr to
+  // it is destroyed, i.e. in a normal full exit. The ThreadPoolExecutor
+  // destructor ensures it has been stopped and waits for worker threads to
+  // finish. The wait is important as it prevents intermittent crashes on
+  // Windows when the process is doing a full exit.
+  //
+  // The Windows crashes appear to only occur with the MSVC static runtimes and
+  // are more frequent with the debug static runtime.
+  //
+  // This also prevents intermittent deadlocks on exit with the MinGW runtime.
+  if (parallel::detail::DefaultExecutorInitialized)
+    parallel::detail::Executor::getDefaultExecutor()->stop();
+#endif
+}
+
 void llvm::parallelFor(size_t Begin, size_t End,
                        llvm::function_ref<void(size_t)> Fn) {
   // If we have zero or one items, then do not incur the overhead of spinning up
   // a task group.  They are surprisingly expensive, and because they do not
   // support nested parallelism, a single entry task group can block parallel
   // execution underneath them.
 #if LLVM_ENABLE_THREADS
   auto NumItems = End - Begin;

llvm/lib/Support/Statistic.cpp

 #include "llvm/ADT/Statistic.h"
 
 #include "DebugOptions.h"
 
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Compiler.h"
 #include "llvm/Support/Debug.h"
+#include "llvm/Support/FastShutdown.h"
 #include "llvm/Support/Format.h"
-#include "llvm/Support/ManagedStatic.h"
 #include "llvm/Support/Mutex.h"
 #include "llvm/Support/Timer.h"
 #include "llvm/Support/YAMLTraits.h"
 #include "llvm/Support/raw_ostream.h"
 #include <algorithm>
 #include <cstring>
 using namespace llvm;
 
static cl::opt<bool, true> registerEnableStats{
           "Enable statistics output from program (available with Asserts)"),
       cl::location(EnableStats), cl::Hidden};
   static cl::opt<bool, true> registerStatsAsJson{
       "stats-json", cl::desc("Display statistics as json data"),
       cl::location(StatsAsJSON), cl::Hidden};
 }
 
 namespace {
-/// This class is used in a ManagedStatic so that it is created on demand (when
-/// the first statistic is bumped) and destroyed only when llvm_shutdown is
-/// called. We print statistics from the destructor.
+/// This class is instantiated via a function scope static variable so that it
+/// is created on demand (when the first statistic is bumped) and destroyed only
+/// when global destructors are run.
+///
+/// We print statistics from the destructor or when llvm_fast_shutdown() is
+/// called.
+///
 /// This class is also used to look up statistic values from applications that
 /// use LLVM.
 class StatisticInfo {
+  sys::SmartMutex<true> Lock;
   std::vector<TrackingStatistic *> Stats;
 
   friend void llvm::PrintStatistics();
   friend void llvm::PrintStatistics(raw_ostream &OS);
   friend void llvm::PrintStatisticsJSON(raw_ostream &OS);
 
   /// Sort statistics by debugtype,name,description.
   void sort();
 public:
   using const_iterator = std::vector<TrackingStatistic *>::const_iterator;
 
   StatisticInfo();
   ~StatisticInfo();
 
+  sys::SmartMutex<true> &getLock() { return Lock; }
+
   void addStatistic(TrackingStatistic *S) { Stats.push_back(S); }
 
   const_iterator begin() const { return Stats.begin(); }
   const_iterator end() const { return Stats.end(); }
   iterator_range<const_iterator> statistics() const {
     return {begin(), end()};
   }
 
   void reset();
 };
 } // end anonymous namespace
 
-static ManagedStatic<StatisticInfo> StatInfo;
-static ManagedStatic<sys::SmartMutex<true> > StatLock;
+static bool StatisticInfoInitialized = false;
+
+static StatisticInfo &getStatInfo() {
+  static StatisticInfo StatInfo;
+  return StatInfo;
+}
 
 /// RegisterStatistic - The first time a statistic is bumped, this method is
 /// called.
 void TrackingStatistic::RegisterStatistic() {
   // If stats are enabled, inform StatInfo that this statistic should be
   // printed.
-  // llvm_shutdown calls destructors while holding the ManagedStatic mutex.
-  // These destructors end up calling PrintStatistics, which takes StatLock.
-  // Since dereferencing StatInfo and StatLock can require taking the
-  // ManagedStatic mutex, doing so with StatLock held would lead to a lock
-  // order inversion. To avoid that, we dereference the ManagedStatics first,
-  // and only take StatLock afterwards.
   if (!Initialized.load(std::memory_order_relaxed)) {
-    sys::SmartMutex<true> &Lock = *StatLock;
-    StatisticInfo &SI = *StatInfo;
-    sys::SmartScopedLock<true> Writer(Lock);
+    StatisticInfo &SI = getStatInfo();
+    sys::SmartScopedLock<true> Writer(SI.getLock());
     // Check Initialized again after acquiring the lock.
     if (Initialized.load(std::memory_order_relaxed))
       return;
     if (EnableStats || Enabled)
       SI.addStatistic(this);
 
     // Remember we have been registered.
     Initialized.store(true, std::memory_order_release);
   }
 }
 
 StatisticInfo::StatisticInfo() {
+  assert(!StatisticInfoInitialized);
+  StatisticInfoInitialized = true;
+
   // Ensure timergroup lists are created first so they are destructed after us.
   TimerGroup::ConstructTimerLists();
 }
 
 // Print information when destroyed, iff command line option is specified.
 StatisticInfo::~StatisticInfo() {
   if (EnableStats || PrintOnExit)
     llvm::PrintStatistics();
llvm::stable_sort(
         if (int Cmp = std::strcmp(LHS->getName(), RHS->getName()))
           return Cmp < 0;
 
         return std::strcmp(LHS->getDesc(), RHS->getDesc()) < 0;
       });
 }
 
 void StatisticInfo::reset() {
-  sys::SmartScopedLock<true> Writer(*StatLock);
+  sys::SmartScopedLock<true> Writer(getLock());
 
   // Tell each statistic that it isn't registered so it has to register
   // again. We're holding the lock so it won't be able to do so until we're
   // finished. Once we've forced it to re-register (after we return), then zero
   // the value.
   for (auto *Stat : Stats) {
     // Value updates to a statistic that complete before this statement in the
     // iteration for that statistic will be lost as intended.
     Stat->Initialized = false;
     Stat->Value = 0;
   }
 
   // Clear the registration list and release the lock once we're done. Any
   // pending updates from other threads will safely take effect after we return.
   // That might not be what the user wants if they're measuring a compilation
   // but it's their responsibility to prevent concurrent compilations to make
   // a single compilation measurable.
   Stats.clear();
 }
 
 void llvm::PrintStatistics(raw_ostream &OS) {
-  StatisticInfo &Stats = *StatInfo;
+  StatisticInfo &Stats = getStatInfo();
+  sys::SmartScopedLock<true> Reader(Stats.getLock());
 
   // Figure out how long the biggest Value and Name fields are.
   unsigned MaxDebugTypeLen = 0, MaxValLen = 0;
   for (TrackingStatistic *Stat : Stats.Stats) {
     MaxValLen = std::max(MaxValLen, (unsigned)utostr(Stat->getValue()).size());
     MaxDebugTypeLen =
         std::max(MaxDebugTypeLen, (unsigned)std::strlen(Stat->getDebugType()));
   }
for (TrackingStatistic *Stat : Stats.Stats)
     OS << format("%*" PRIu64 " %-*s - %s\n", MaxValLen, Stat->getValue(),
                  MaxDebugTypeLen, Stat->getDebugType(), Stat->getDesc());
 
   OS << '\n';  // Flush the output stream.
   OS.flush();
 }
 
 void llvm::PrintStatisticsJSON(raw_ostream &OS) {
-  sys::SmartScopedLock<true> Reader(*StatLock);
-  StatisticInfo &Stats = *StatInfo;
+  StatisticInfo &Stats = getStatInfo();
+  sys::SmartScopedLock<true> Reader(Stats.getLock());
 
   Stats.sort();
 
   // Print all of the statistics.
   OS << "{\n";
   const char *delim = "";
   for (const TrackingStatistic *Stat : Stats.Stats) {
     OS << delim;
void llvm::PrintStatisticsJSON(raw_ostream &OS) {
   TimerGroup::printAllJSONValues(OS, delim);
 
   OS << "\n}\n";
   OS.flush();
 }
 
 void llvm::PrintStatistics() {
 #if LLVM_ENABLE_STATS
-  sys::SmartScopedLock<true> Reader(*StatLock);
-  StatisticInfo &Stats = *StatInfo;
+  StatisticInfo &Stats = getStatInfo();
+  sys::SmartScopedLock<true> Reader(Stats.getLock());
 
   // Statistics not enabled?
   if (Stats.Stats.empty()) return;
 
   // Get the stream to write to.
   std::unique_ptr<raw_ostream> OutStream = CreateInfoOutputFile();
   if (StatsAsJSON)
     PrintStatisticsJSON(*OutStream);
if (EnableStats) {
     std::unique_ptr<raw_ostream> OutStream = CreateInfoOutputFile();
     (*OutStream) << "Statistics are disabled.  "
                  << "Build with asserts or with -DLLVM_FORCE_ENABLE_STATS\n";
   }
 #endif
 }
 
 const std::vector<std::pair<StringRef, uint64_t>> llvm::GetStatistics() {
-  sys::SmartScopedLock<true> Reader(*StatLock);
+  sys::SmartScopedLock<true> Reader(getStatInfo().getLock());
   std::vector<std::pair<StringRef, uint64_t>> ReturnStats;
 
-  for (const auto &Stat : StatInfo->statistics())
+  for (const auto &Stat : getStatInfo().statistics())
     ReturnStats.emplace_back(Stat->getName(), Stat->getValue());
   return ReturnStats;
 }
 
-void llvm::ResetStatistics() {
-  StatInfo->reset();
+void llvm::ResetStatistics() { getStatInfo().reset(); }
+
+void llvm::fast_shutdown_statistics() {
+  if (StatisticInfoInitialized)
+    llvm::PrintStatistics();
 }