[libc++] Persistently cache memoized operations during Lit configuration

[ldionne]

Summary

When invoking Lit repeatedly, we perform all the configuration checks
over and over again, which takes a lot of time. This patch allows caching
the result of configuration checks persistently across Lit invocations to
speed this up.

In theory, this should still be functionally correct since the cache
key should contain everything that determines the output of the
configuration check. However, in cases where e.g. the compiler has
changed but is at the same path as previously, the Lit configuration
checks will be cached even though technically the cache should have
been invalidated.

[ldionne]

I've previously held off with submitting this change because I'm scared that if something's wrong with it, it's going to be very tricky to figure out. But it's likely that the added benefit of extra-fast configuration time is worth taking the risk.

[ldionne]

I'm going to ship this. If someone sees weird behavior that could be related to tests not being configured properly, and if these issues go away when you nuke build directory, please ping this review to get my attention.

libcxx/utils/libcxx/test/dsl.py

 
 def _memoizeExpensiveOperation(extractCacheKey):
   """
   Allows memoizing a very expensive operation.
 
   We pickle the cache key to make sure we store an immutable representation
   of it. If we stored an object and the object was referenced elsewhere, it
   could be changed from under our feet, which would break the cache.
+
+  We also store the cache for a given function persistently across invocations
+  of Lit. This dramatically speeds up the configuration of the test suite when
+  invoking Lit repeatedly, which is important for developer workflow. However,
+  with the current implementation that does not synchronize updates to the
+  persistent cache, this also means that one should not call a memoized
+  operation from multiple threads. This should normally not be a problem
+  since Lit configuration is single-threaded.
   """
   def decorator(function):
+    def f(config, *args, **kwargs):
+      cacheRoot = os.path.join(config.test_exec_root, '__config_cache__')
+      persistentCache = os.path.join(cacheRoot, function.__name__)
+      if not os.path.exists(cacheRoot):
+        os.makedirs(cacheRoot)
+
       cache = {}
-    def f(*args, **kwargs):
-      cacheKey = pickle.dumps(extractCacheKey(*args, **kwargs))
+      # Load a cache from a previous Lit invocation if there is one.
+      if os.path.exists(persistentCache):
+        with open(persistentCache, 'rb') as cacheFile:
+          cache = pickle.load(cacheFile)
+
+      cacheKey = pickle.dumps(extractCacheKey(config, *args, **kwargs))
       if cacheKey not in cache:
-        cache[cacheKey] = function(*args, **kwargs)
+        cache[cacheKey] = function(config, *args, **kwargs)
+        # Update the persistent cache so it knows about the new key
+        with open(persistentCache, 'wb') as cacheFile:
+          pickle.dump(cache, cacheFile)
       return cache[cacheKey]
     return f
   return decorator
 
 def _executeScriptInternal(test, commands):
   """
   Returns (stdout, stderr, exitCode, timeoutInfo)