[libcxx] Allow the use of ccache when running the test suite.

[EricWF]

Summary

In order to get the bots running quicker I would like to be able to use ccache
with the test suite. This patch adds support for running the test suite using
ccache. To use ccache pass --param=use_ccache=true when running the test suite.

ccache will not cache any command that invokes ld, so the build step needs to be
split into two separate compile commands. The cost of splitting the build step
into two parts when not using ccache seems to be minimal. On my machine I saw a
difference of ~5 seconds on a 5 minute test suite run.

A full test suite run with ccache generates about 250MB of cached data.

I recorded the following times for running the test suite in the following configurations:

• no ccache: 340s
• initial ccache run: 380s
• rerun with ccache (no changes): 53s.
• rerun with ccache (<string> changed): 80s
• rerun with ccache (<cmath> changed): 169s
• rerun with ccache (<valarray> changed): 69s

[jroelofs]

If the compiler changes on every iteration of the buildbot, how will ccache make that faster?

Also, ccache makes me a bit uncomfortable in a buildbot kind of environment. It's great for development, but it definitely adds one more area for build hysteresis. Perhaps we ought to have nightly (or whatever appropriate frequency) clean builds in addition to such a ccache bot.

All of that being said, this is awesome. LGTM.

test/lit.cfg

122

it'd be nice if this named temporary had something in common with the test file name.

[danalbert]

Couldn't you just set cxx_under_test to 'ccache clang++' and be done with it? Using ccache in build bots where the compiler goes through regular updates feels fragile to me...

As for splitting into separate build/link commands, that's how I have to do it in Android anyway, so that part is good with me :)

[EricWF]

In D6647#101319, @jroelofs wrote:

If the compiler changes on every iteration of the buildbot, how will ccache make that faster?

The compiler doesn't change every iteration for the most part. On my buildbots I try and update the compiler once a week or so.

In D6647#101319, @jroelofs wrote:

Also, ccache makes me a bit uncomfortable in a buildbot kind of environment. It's great for development, but it definitely adds one more area for build hysteresis. Perhaps we ought to have nightly (or whatever appropriate frequency) clean builds in addition to such a ccache bot.

Very good point. As I understand it, this should only matter when the compiler changes. I'll make sure to clear the cache entirely when updating the compilers.

In D6647#101645, @danalbert wrote:

Couldn't you just set cxx_under_test to 'ccache clang++' and be done with it?

As it currently stands that wouldn't work. cxx_under_test is expected to be a single token.

I understand concerns about this setup being fragile. However I would still like to try it out on my bots for the time being. There are a number of extra configurations I would like to run and currently I don't have the resources to do it. My OSX builder takes 30-45 minutes per run. It would be nice to get this down.

[danalbert]

In D6647#101645, @danalbert wrote:

Couldn't you just set cxx_under_test to 'ccache clang++' and be done with it?

As it currently stands that wouldn't work. cxx_under_test is expected to be a single token.

That should be an easy fix though, right?

[EricWF]

In D6647#103223, @danalbert wrote:

In D6647#101645, @danalbert wrote:

Couldn't you just set cxx_under_test to 'ccache clang++' and be done with it?

As it currently stands that wouldn't work. cxx_under_test is expected to be a single token.

That should be an easy fix though, right?

Not really. Doing so means we can't use lit.util.which('ccache clang++'). I currently use the compiler path to try and find the llvm-symbolizer binary.

test/lit.cfg

     """
 
     def __init__(self, cxx_under_test, use_verify_for_fail,
-                 cpp_flags, ld_flags, exec_env):
+                 cpp_flags, ld_flags, exec_env,
+                 use_ccache=False):
         self.cxx_under_test = cxx_under_test
         self.use_verify_for_fail = use_verify_for_fail
         self.cpp_flags = list(cpp_flags)
         self.ld_flags = list(ld_flags)
         self.exec_env = dict(exec_env)
+        self.use_ccache = use_ccache
 
     def execute(self, test, lit_config):
         while True:

         report += '\n'
         return cmd, report, rc
 
-    def _build(self, exec_path, source_path, compile_only=False,
-               use_verify=False):
-        cmd = [self.cxx_under_test, '-o', exec_path,
-               source_path] + self.cpp_flags
-
-        if compile_only:
-            cmd += ['-c']
-        else:
-            cmd += self.ld_flags
-
+    def _compile(self, output_path, source_path, use_verify=False):
+        cmd = [self.cxx_under_test, '-c', '-o', output_path, source_path]
+        cmd += self.cpp_flags
         if use_verify:
             cmd += ['-Xclang', '-verify']
+        if self.use_ccache:
+            cmd = ['ccache'] + cmd
+        out, err, rc = lit.util.executeCommand(cmd)
+        return cmd, out, err, rc
 
+    def _link(self, exec_path, object_path):
+        cmd = [self.cxx_under_test, '-o', exec_path, object_path]
+        cmd += self.cpp_flags + self.ld_flags
         out, err, rc = lit.util.executeCommand(cmd)
+        return cmd, out, err, rc
+
+    def _compile_and_link(self, exec_path, source_path):
+        object_file = tempfile.NamedTemporaryFile(suffix=".o", delete=False)
+        object_path = object_file.name
+        object_file.close()
+        try:
+            cmd, out, err, rc = self._compile(object_path, source_path)
+            if rc != 0:
+                return cmd, out, err, rc
+            return self._link(exec_path, object_path)
+        finally:
+            try:
+                os.remove(object_path)
+            except:
+                pass
+
+    def _build(self, exec_path, source_path, compile_only=False,
+               use_verify=False):
+        if compile_only:
+            cmd, out, err, rc = self._compile(exec_path, source_path, use_verify)
+        else:
+            assert not use_verify
+            cmd, out, err, rc = self._compile_and_link(exec_path, source_path)
         return self._make_report(cmd, out, err, rc)
 
     def _clean(self, exec_path):

         self.link_flags = ['-nodefaultlibs']
         self.use_system_lib = False
         self.use_clang_verify = False
+        self.use_ccache = False
 
         if platform.system() not in ('Darwin', 'FreeBSD', 'Linux'):
             self.lit_config.fatal("unrecognized system")

         self.configure_obj_root()
         self.configure_use_system_lib()
         self.configure_use_clang_verify()
+        self.configure_ccache()
         self.configure_env()
         self.configure_std_flag()
         self.configure_compile_flags()

             self.use_clang_verify,
             cpp_flags=self.compile_flags,
             ld_flags=self.link_flags,
-            exec_env=self.env)
+            exec_env=self.env,
+            use_ccache=self.use_ccache)
 
     def configure_cxx(self):
         # Gather various compiler parameters.

             self.lit_config.note(
                 "inferred use_clang_verify as: %r" % self.use_clang_verify)
 
+    def configure_ccache(self):
+        self.use_ccache = self.get_lit_bool('use_ccache', False)
+        if self.use_ccache:
+            self.lit_config.note('enabling ccache')
+
     def configure_features(self):
         additional_features = self.get_lit_conf('additional_features')
         if additional_features: