[Zorg] Modify LibCxxAndABIBuilder to support testing libcxx with multiple standard versions.

[EricWF]

Summary

This change makes it so that the builder takes a list of standard versions to test against. The default list is just c++11. For each standard on the list a job is added.
This also changes the builder so it invokes LIT directly for libcxx instead of using check-libcxx.

[danalbert]

zorg/buildbot/builders/LibcxxAndAbiBuilder.py

95

-s so we're not inundated with all the passing tests?

99

Is that not the default?

[gribozavr]

Could we add more targets instead, check-libcxx-cxx11, check-libcxx-cxx14 etc?

Then it would be easier to run them when developing libcxx itself.

[EricWF]

In D5114#5, @gribozavr wrote:

Could we add more targets instead, check-libcxx-cxx11, check-libcxx-cxx14 etc?

Then it would be easier to run them when developing libcxx itself.

I agree that we should add these Makefile targets, but I would like to keep invoking lit directly in this change for a couple of reasons:

1. We need a custom LIT invocation so the output is more "log" friendly (no progress bar).
2. More changes are going to follow that require custom invocations. (Testing debug mode, sanitizers, ect)
3. I don't want to add the makefile targets until I've actually added support for running the tests as C++03, C++14, ect. So I want to use the bots to test these changes.

If you don't have any strong objections I would like to keep the custom LIT invocations.

zorg/buildbot/builders/LibcxxAndAbiBuilder.py

95

'-s` add the progress bar that is non-log friendly. I'll add -s and --no-progress-bar, but I'm personally not opposed to printing the passing tests.

99

I'm not actually sure, but I also don't think the working dir matters that much anyway. I'll remove it.

[EricWF]

Implement @danalbert's comments.

[EricWF]

Other changes have been made that should make this possible.

zorg/buildbot/builders/LibcxxAndAbiBuilder.py

     return f
 
 
-def getLibcxxAndAbiBuilder(f=None, env={}):
+def getLibcxxAndAbiBuilder(f=None, env={}, standards=['c++11']):
     if f is None:
         f = buildbot.process.factory.BuildFactory()
 

 
     src_root = properties.WithProperties('%(builddir)s/llvm')
     build_path = properties.WithProperties('%(builddir)s/build')
+    lit_exe = os.path.join(src_root, 'utils/lit/lit.py')
+    libcxx_test_path = os.path.join(build_path, 'projects/libcxx/test')
 
     f = getLibcxxWholeTree(f, src_root)
 

         name='test.libcxxabi', command=['make', 'check-libcxxabi'],
         workdir=build_path))
 
-    # Test libc++
-    f.addStep(buildbot.steps.shell.ShellCommand(
-        name='test.libcxx', command=['make', 'check-libcxx'],
-        workdir=build_path))
+    for std in standards:
+        f.addStep(lit_test_command.LitTestCommand(
+            name='test.libcxx.%s' % std,
+            command=[lit_exe, '-vs', '--no-progress-bar',
+                     '--show-unsupported', '--show-xfail',
+                    '--param=std=%s' % std,
+                     properties.WithProperties('--threads=%(jobs)s'),
+                    libcxx_test_path]
+            ))
+
 
     return f