[libc] make part of libc's unit test framework independent of llvm

[michaelrj]

Summary

This is mostly changing stringref to std::string, outs() to cout,
and small supporting changes. This will make running unit tests possible
on systems that are only grabbing the libc part of llvm.

[abrachet]

This will make running unit tests possible on systems that are only grabbing the libc part of llvm.

Do our tests depend on the generated headers? I thought they did, if so it will be very hard to remove our dependency on the rest of llvm.

[michaelrj]

Do our tests depend on the generated headers? I thought they did, if so it will be very hard to remove our dependency on the rest of llvm.

Yes the tests depend on the generated headers, but it's possible to handwrite the headers if necessary, or even prebuild the headergen tool. This is what I've been doing for the project I've been working on, and this patch will let me run the unit tests.

[michaelrj]

update: I was reminded of the other patch that I'm currently in the process of getting reviewed (linked here: https://reviews.llvm.org/D91394) that will make the string and math tests independent of headergen since they can use the system libc's header. This means if I can make the unit tests fully independent then you can bring up both the string and math parts of LLVM-libc without the whole project.

[abrachet]

The LibcUnitTest and libc_test_utils targets are made with add_llvm_library and link against Support. Presumably we would need use add_library, (although it may not be the case that add_llvm_library implicitly adds any LLVM dependencies, I haven't checked).

libc_test_utils also has a small dependency on llvm::ostream and llvm::MemoryBuffer. I feel bad just bringing it up without just changing it myself because I think a lot of that code was added by me, but I don't really have time to work on it until after my semester ends, unfortunately.

[sivachandra]

In D91568#2400511, @abrachet wrote:

The LibcUnitTest and libc_test_utils targets are made with add_llvm_library and link against Support. Presumably we would need use add_library, (although it may not be the case that add_llvm_library implicitly adds any LLVM dependencies, I haven't checked).

libc_test_utils also has a small dependency on llvm::ostream and llvm::MemoryBuffer. I feel bad just bringing it up without just changing it myself because I think a lot of that code was added by me, but I don't really have time to work on it until after my semester ends, unfortunately.

Ah yes! The build rules slipped my mind. I will take a shot at it.

[sivachandra]

In D91568#2400565, @sivachandra wrote:

In D91568#2400511, @abrachet wrote:

The LibcUnitTest and libc_test_utils targets are made with add_llvm_library and link against Support. Presumably we would need use add_library, (although it may not be the case that add_llvm_library implicitly adds any LLVM dependencies, I haven't checked).

libc_test_utils also has a small dependency on llvm::ostream and llvm::MemoryBuffer. I feel bad just bringing it up without just changing it myself because I think a lot of that code was added by me, but I don't really have time to work on it until after my semester ends, unfortunately.

Ah yes! The build rules slipped my mind. I will take a shot at it.

To be fair though, Michael said he will make another pass and clean up the rest of the pieces having LLVM dependency.

[michaelrj]

I guess I've got some more changes ahead of me. This new commit (linked here) is where I plan on putting the next set, and it currently has a few based on what you've said.

libc/utils/UnitTest/Test.cpp

 //===-- Implementation of the base class for libc unittests ---------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "Test.h"
 
 #include "utils/testutils/ExecuteFunction.h"
-#include "llvm/ADT/StringExtras.h"
-#include "llvm/ADT/StringRef.h"
-#include "llvm/Support/raw_ostream.h"
+#include <cassert>
+#include <iostream>
+#include <string>
 
 namespace __llvm_libc {
 namespace testing {
 
 // This need not be a class as all it has is a single read-write state variable.
 // But, we make it class as then its implementation can be hidden from the
 // header file.
 class RunContext {

 
 // When the value is of integral type, just display it as normal.
 template <typename ValType>
 cpp::EnableIfType<cpp::IsIntegral<ValType>::Value, std::string>
 describeValue(ValType Value) {
   return std::to_string(Value);
 }
 
-std::string describeValue(llvm::StringRef Value) { return std::string(Value); }
+std::string describeValue(std::string Value) { return std::string(Value); }
 
 // When the value is __uint128_t, also show its hexadecimal digits.
 // Using template to force exact match, prevent ambiguous promotion.
 template <> std::string describeValue<__uint128_t>(__uint128_t Value) {
   std::string S(sizeof(__uint128_t) * 2, '0');
 
   for (auto I = S.rbegin(), End = S.rend(); I != End; ++I, Value >>= 4) {
     unsigned char Mod = static_cast<unsigned char>(Value) & 15;
-    *I = llvm::hexdigit(Mod, true);
+    *I = Mod < 10 ? '0' + Mod : 'a' + Mod - 10;
   }
 
   return "0x" + S;
 }
 
 template <typename ValType>
 void explainDifference(ValType LHS, ValType RHS, const char *LHSStr,
                        const char *RHSStr, const char *File, unsigned long Line,
-                       llvm::StringRef OpString) {
+                       std::string OpString) {
   size_t OffsetLength = OpString.size() > 2 ? OpString.size() - 2 : 0;
   std::string Offset(OffsetLength, ' ');
 
-  llvm::outs() << File << ":" << Line << ": FAILURE\n"
-               << Offset << "Expected: " << LHSStr << '\n'
-               << Offset << "Which is: " << describeValue(LHS) << '\n'
-               << "To be " << OpString << ": " << RHSStr << '\n'
-               << Offset << "Which is: " << describeValue(RHS) << '\n';
+  std::cout << File << ":" << Line << ": FAILURE\n"
 }
 
 template <typename ValType>
 bool test(RunContext *Ctx, TestCondition Cond, ValType LHS, ValType RHS,
           const char *LHSStr, const char *RHSStr, const char *File,
           unsigned long Line) {
-  auto ExplainDifference = [=](llvm::StringRef OpString) {
+  auto ExplainDifference = [=](std::string OpString) {
     explainDifference(LHS, RHS, LHSStr, RHSStr, File, Line, OpString);
   };
 
   switch (Cond) {
   case Cond_EQ:
     if (LHS == RHS)
       return true;
 
case Cond_GE:
     if (LHS >= RHS)
       return true;
 
     Ctx->markFail();
     ExplainDifference("greater than or equal to");
     return false;
   default:
     Ctx->markFail();
-    llvm::outs() << "Unexpected test condition.\n";
+    std::cout << "Unexpected test condition.\n";
     return false;
   }
 }
 
 } // namespace internal
 
 Test *Test::Start = nullptr;
 Test *Test::End = nullptr;
void Test::addTest(Test *T) {
   End = T;
 }
 
 int Test::runTests() {
   int TestCount = 0;
   int FailCount = 0;
   for (Test *T = Start; T != nullptr; T = T->Next, ++TestCount) {
     const char *TestName = T->getName();
-    constexpr auto GREEN = llvm::raw_ostream::GREEN;
-    constexpr auto RED = llvm::raw_ostream::RED;
-    constexpr auto RESET = llvm::raw_ostream::RESET;
-    llvm::outs() << GREEN << "[ RUN      ] " << RESET << TestName << '\n';
+    constexpr auto GREEN = "\033[32m";
+    constexpr auto RED = "\033[31m";
+    constexpr auto RESET = "\033[0m";
+    std::cout << GREEN << "[ RUN      ] " << RESET << TestName << '\n';
     RunContext Ctx;
     T->SetUp();
     T->setContext(&Ctx);
     T->Run();
     T->TearDown();
     auto Result = Ctx.status();
     switch (Result) {
     case RunContext::Result_Fail:
-      llvm::outs() << RED << "[  FAILED  ] " << RESET << TestName << '\n';
+      std::cout << RED << "[  FAILED  ] " << RESET << TestName << '\n';
       ++FailCount;
       break;
     case RunContext::Result_Pass:
-      llvm::outs() << GREEN << "[       OK ] " << RESET << TestName << '\n';
+      std::cout << GREEN << "[       OK ] " << RESET << TestName << '\n';
       break;
     }
   }
 
-  llvm::outs() << "Ran " << TestCount << " tests. "
-               << " PASS: " << TestCount - FailCount << ' '
-               << " FAIL: " << FailCount << '\n';
+  std::cout << "Ran " << TestCount << " tests. "
 
   return FailCount > 0 ? 1 : 0;
 }
 
 template bool Test::test<char, 0>(TestCondition Cond, char LHS, char RHS,
                                   const char *LHSStr, const char *RHSStr,
                                   const char *File, unsigned long Line);
 

 
 template bool Test::test<__uint128_t, 0>(TestCondition Cond, __uint128_t LHS,
                                          __uint128_t RHS, const char *LHSStr,
                                          const char *RHSStr, const char *File,
                                          unsigned long Line);
 
 bool Test::testStrEq(const char *LHS, const char *RHS, const char *LHSStr,
                      const char *RHSStr, const char *File, unsigned long Line) {
-  return internal::test(Ctx, Cond_EQ, llvm::StringRef(LHS),
-                        llvm::StringRef(RHS), LHSStr, RHSStr, File, Line);
+  return internal::test(Ctx, Cond_EQ, LHS ? std::string(LHS) : std::string(),
+                        RHS ? std::string(RHS) : std::string(), LHSStr, RHSStr,
+                        File, Line);
 }
 
 bool Test::testStrNe(const char *LHS, const char *RHS, const char *LHSStr,
                      const char *RHSStr, const char *File, unsigned long Line) {
-  return internal::test(Ctx, Cond_NE, llvm::StringRef(LHS),
-                        llvm::StringRef(RHS), LHSStr, RHSStr, File, Line);
+  return internal::test(Ctx, Cond_NE, LHS ? std::string(LHS) : std::string(),
+                        RHS ? std::string(RHS) : std::string(), LHSStr, RHSStr,
+                        File, Line);
 }
 
 bool Test::testMatch(bool MatchResult, MatcherBase &Matcher, const char *LHSStr,
                      const char *RHSStr, const char *File, unsigned long Line) {
   if (MatchResult)
     return true;
 
   Ctx->markFail();
-  llvm::outs() << File << ":" << Line << ": FAILURE\n"
-               << "Failed to match " << LHSStr << " against " << RHSStr
-               << ".\n";
+  std::cout << File << ":" << Line << ": FAILURE\n"
+            << "Failed to match " << LHSStr << " against " << RHSStr << ".\n";
   testutils::StreamWrapper OutsWrapper = testutils::outs();
   Matcher.explainError(OutsWrapper);
   return false;
 }
 
 bool Test::testProcessKilled(testutils::FunctionCaller *Func, int Signal,
                              const char *LHSStr, const char *RHSStr,
                              const char *File, unsigned long Line) {
   testutils::ProcessStatus Result = testutils::invokeInSubprocess(Func, 500);
 
   if (const char *error = Result.getError()) {
     Ctx->markFail();
-    llvm::outs() << File << ":" << Line << ": FAILURE\n" << error << '\n';
+    std::cout << File << ":" << Line << ": FAILURE\n" << error << '\n';
     return false;
   }
 
   if (Result.timedOut()) {
     Ctx->markFail();
-    llvm::outs() << File << ":" << Line << ": FAILURE\n"
-                 << "Process timed out after " << 500 << " milliseconds.\n";
+    std::cout << File << ":" << Line << ": FAILURE\n"
     return false;
   }
 
   if (Result.exitedNormally()) {
     Ctx->markFail();
-    llvm::outs() << File << ":" << Line << ": FAILURE\n"
-                 << "Expected " << LHSStr
-                 << " to be killed by a signal\nBut it exited normally!\n";
+    std::cout << File << ":" << Line << ": FAILURE\n"
     return false;
   }
 
   int KilledBy = Result.getFatalSignal();
   assert(KilledBy != 0 && "Not killed by any signal");
   if (Signal == -1 || KilledBy == Signal)
     return true;
 
   using testutils::signalAsString;
   Ctx->markFail();
-  llvm::outs() << File << ":" << Line << ": FAILURE\n"
-               << "              Expected: " << LHSStr << '\n'
-               << "To be killed by signal: " << Signal << '\n'
-               << "              Which is: " << signalAsString(Signal) << '\n'
-               << "  But it was killed by: " << KilledBy << '\n'
-               << "              Which is: " << signalAsString(KilledBy)
-               << '\n';
+  std::cout << File << ":" << Line << ": FAILURE\n"
+            << "              Which is: " << signalAsString(KilledBy) << '\n';
   return false;
 }
 
 bool Test::testProcessExits(testutils::FunctionCaller *Func, int ExitCode,
                             const char *LHSStr, const char *RHSStr,
                             const char *File, unsigned long Line) {
   testutils::ProcessStatus Result = testutils::invokeInSubprocess(Func, 500);
 
   if (const char *error = Result.getError()) {
     Ctx->markFail();
-    llvm::outs() << File << ":" << Line << ": FAILURE\n" << error << '\n';
+    std::cout << File << ":" << Line << ": FAILURE\n" << error << '\n';
     return false;
   }
 
   if (Result.timedOut()) {
     Ctx->markFail();
-    llvm::outs() << File << ":" << Line << ": FAILURE\n"
-                 << "Process timed out after " << 500 << " milliseconds.\n";
+    std::cout << File << ":" << Line << ": FAILURE\n"
     return false;
   }
 
   if (!Result.exitedNormally()) {
     Ctx->markFail();
-    llvm::outs() << File << ":" << Line << ": FAILURE\n"
-                 << "Expected " << LHSStr << '\n'
-                 << "to exit with exit code " << ExitCode << '\n'
-                 << "But it exited abnormally!\n";
+    std::cout << File << ":" << Line << ": FAILURE\n"
     return false;
   }
 
   int ActualExit = Result.getExitCode();
   if (ActualExit == ExitCode)
     return true;
 
   Ctx->markFail();
-  llvm::outs() << File << ":" << Line << ": FAILURE\n"
-               << "Expected exit code of: " << LHSStr << '\n'
-               << "             Which is: " << ActualExit << '\n'
-               << "       To be equal to: " << RHSStr << '\n'
-               << "             Which is: " << ExitCode << '\n';
+  std::cout << File << ":" << Line << ": FAILURE\n"
   return false;
 }
 
 } // namespace testing
 } // namespace __llvm_libc
 
 int main() { return __llvm_libc::testing::Test::runTests(); }