[RFC] Add a LIT-style Progress Bar to libSupport

[jroelofs]

Summary

I've built this for an out-of-tree use case where we have a tool runs for a very long time. The task itself happens to be reasonably predictable in the amount of work completed, much like the corpus of LIT tests. Since we've already got such a thing in-tree (in python), namely the LIT one, I figured I'd model it off of that.

Usage is something like:

llvm::ProgressBar PB(llvm::outs(), "Performing Tasks");
for (int I = 0, E = 100; I != E; ++I) {
  PB.update(I / float(E), (I < 30 ? "Task A" : "Task B"));
  sleep(1);
}
PB.clear();

And with that, you get a nice familiar progress bar that looks like:

             Performing Tasks
 20% [=====--------------------] ETA 00:01:20
Task A

with different colors, to distinguish it from LIT's (sigh... bikeshed away).

Is this something that would be useful upstream? I've thought about tying this into the pass manager, but I don't yet see a consistent way to estimate the total amount of work there. Even then, are there cases where Clang runs for long enough for this to matter?... I don't want this to be an admission of defeat when it comes to compile times. Are there other use cases?

Suggestions on how to write a UnitTest for it are most welcome. As-is, that's a little tricky because this depends on the width of the terminal, so printing it to a raw_string_ostream isn't going to give consistent results. I've tested this on Darwin, but I have no idea whether it'll DTRT on other platforms. If it's not platform-agnostic enough, suggestions on how to fix that would also be quite welcome.

[jroelofs]

@aaron.ballman suggested adding @chandlerc as a reviewer

[jroelofs]

It occurs to me that Chandler's also the right person to ask about ideas on how to estimate the progress of a PassManager.

[jroelofs]

Rate-limit it, since it'll look awful if you call update() really often.

[jroelofs]

Add a testcase per @bogner's advice (sort of). I tried doing the setenv trick, but that only worked when the testcase is ran not under LIT. Since I have to inject other state in anyway for testing, may as well just use that flag to force 60 cols too.

[meadori]

Is this something that would be useful upstream?

One application where it might be helpful is in LLD. I could see enabling something like this for linking (since that can be long at times).

include/llvm/Support/Progress.h

+//===--- Progress.h - ANSI Terminal Progress Bar ----------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+//  This file declares an ANSI Terminal Progress Bar.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_SUPPORT_PROGRESS_H
+#define LLVM_SUPPORT_PROGRESS_H
+
+#include "llvm/ADT/Optional.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/raw_ostream.h"
+
+#include <string>
+#include <chrono>
+
+namespace llvm {
+/// A 3-line progress bar, which looks like:
+///
+/// \verbatim
+///
+///                                Header
+///        20% [===========----------------------------------]
+///       progress message
+///
+/// \endverbatim
+///
+/// This is modeled heavily off of the one in LIT.
+class ProgressBar {
+  // Colors purposefully chosen to be *different* than LIT's so as not to be
+  // confusing. For just about everything else, we try to match the behavior
+  // pretty closely.
+  static const auto HeaderColor = llvm::raw_ostream::YELLOW;
+  static const auto BarColor = llvm::raw_ostream::MAGENTA;
+  static const unsigned RefreshRate = 50; //< Max refresh rate, in milliseconds.
+public:
+  ProgressBar(llvm::raw_ostream &OS, llvm::StringRef Header, bool ETA=true)
+    : OS(OS), Header(Header), ShowETA(ETA) {}
+
+  /// Update the amount of progress.
+  ///
+  /// Causes the bar to be Visible if it wasn't already.
+  ///
+  /// \param Percent A value in the range [0.0f, 1.0f] indicating the
+  ///                amount of progress to display.
+  /// \param Msg     The message to display underneath the bar.
+  void update(float Percent, llvm::StringRef Msg);
+
+  /// Remove the progress bar from the terminal.
+  void clear();
+
+private:
+
+  /// The stream being written to.
+  llvm::raw_ostream &OS;
+
+  /// Header text to print centered at the top of the bar.
+  std::string Header;
+
+  /// Whether to make a guess at estimated completion.
+  bool ShowETA;
+
+  /// If `ShowETA && Visible`, contains the time when the progress bar was
+  /// first displayed.
+  std::chrono::time_point<std::chrono::system_clock> Start;
+
+  /// Contains the time when the progress bar was last displayed. Used to
+  /// rate-limit the refresh, to avoid flickering.
+  llvm::Optional<std::chrono::time_point<std::chrono::system_clock>> LastUpdate;
+
+  /// Is the progress bar being displayed?
+  bool Visible = false;
+
+  /// Total number of characters written out on the last line.
+  unsigned Width = 0;
+};
+
+} // namespace llvm
+
+#endif
+

lib/Support/CMakeLists.txt

add_llvm_library(LLVMSupport
   MathExtras.cpp
   MemoryBuffer.cpp
   MD5.cpp
   NativeFormatting.cpp
   Options.cpp
   Parallel.cpp
   PluginLoader.cpp
   PrettyStackTrace.cpp
+  Progress.cpp
   RandomNumberGenerator.cpp
   Regex.cpp
   ScaledNumber.cpp
   ScopedPrinter.cpp
   SHA1.cpp
   SmallPtrSet.cpp
   SmallVector.cpp
   SourceMgr.cpp

lib/Support/Progress.cpp

+//===--- lib/Support/Progress.cpp - ANSI Terminal Progress Bar --*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+//  This file defines an ANSI Terminal Progress Bar.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Support/Progress.h"
+
+#include "llvm/Support/Process.h"
+#include "llvm/Support/raw_ostream.h"
+#include "llvm/Support/Format.h"
+
+#include <cassert>
+
+using namespace llvm;
+
+void ProgressBar::update(float Percent, StringRef Msg) {
+  if (!OS.is_displayed())
+    return;
+
+  std::string ETAStr;
+  if (ShowETA) {
+    if (!Visible)
+      Start = std::chrono::system_clock::now();
+    auto Elapsed = std::chrono::duration_cast<std::chrono::seconds>(
+         std::chrono::system_clock::now() - Start).count();
+    if (Percent > 0.0001 && Elapsed > 1) {
+      auto Total = Elapsed / Percent;
+      auto ETA = unsigned(Total - Elapsed);
+      unsigned HH = ETA / 3600;
+      unsigned MM = (ETA / 60) % 60;
+      unsigned SS = ETA % 60;
+      raw_string_ostream RSO(ETAStr);
+      RSO << format(" ETA %02d:%02d:%02d", HH, MM, SS);
+      ETAStr = RSO.str();
+    }
+  }
+
+  if (LastUpdate &&
+      std::chrono::duration_cast<std::chrono::milliseconds>(
+        std::chrono::system_clock::now() - *LastUpdate).count() <
+      RefreshRate)
+    return;
+
+  clear();
+
+  unsigned TotalWidth = sys::Process::StandardOutColumns();
+  unsigned PBWidth = TotalWidth - (1 + 4 + 2 + 1 + ETAStr.size() + 1);
+  unsigned PBDone = PBWidth * Percent;
+
+  // Centered header
+  (OS.indent((TotalWidth - Header.size()) / 2)
+     .changeColor(HeaderColor, /*Bold=*/true)
+     << Header).resetColor() << "\n";
+
+  // Progress bar
+  OS << format(" %3d%% ", unsigned(100 * Percent));
+  (OS.changeColor(BarColor) << "[").resetColor();
+  (OS.changeColor(BarColor, /*Bold=*/true)
+     << std::string(PBDone, '=') << std::string(PBWidth - PBDone, '-'))
+     .resetColor();
+  (OS.changeColor(BarColor) << "]").resetColor();
+  OS << ETAStr << "\n";
+
+  // Footer messaage
+  if (Msg.size() < TotalWidth) {
+    OS << Msg;
+    Width = Msg.size();
+  } else {
+    OS << Msg.substr(0, TotalWidth - 4) << "... ";
+    Width = TotalWidth;
+  }
+
+  // Hide the cursor
+  OS << "\033[?25l";
+
+  OS.flush();
+  Visible = true;
+  LastUpdate = std::chrono::system_clock::now();
+}
+
+void ProgressBar::clear() {
+  if (!Visible)
+    return;
+
+  if (!OS.is_displayed())
+    return;
+
+  // Move to beginning of current the line
+  OS << "\033[" << (Width + 1) << "D"
+
+     // Clear it
+     << "\033[2K"
+
+     // Move up a line and clear it
+     << "\033[1A\033[2K"
+
+     // Move up a line and clear it
+     << "\033[1A\033[2K"
+
+     // Unhide the cursor
+     << "\033[?25h";
+
+  OS.flush();
+  Visible = false;
+}
+