[LIT] Fix failing LIT tests

[EricWF]

Summary

I spend some time trying to get the LIT test suite passing. Here are the changes that I needed to make on my machine.

I made the following changes for the following reasons.

1. google-test.py: The Google test format now checks for "[ PASSED ] 1 test." to check if a test passes.
2. discovery.py: The output appears in a different order on my machine than it did in the test.
3. unittest-adaptor.py: The output appears in a different order on my machine than it did in the test.
4. The classname is now formed differently in getJUnitXML(...).

I'm not sure what is causing the output order to differ in discovery.py and unittest-adaptor.py. Does anybody have any thoughts?

[jroelofs]

LGTM

utils/lit/tests/unittest-adaptor.py

7

I don't know what controls the order, but I'm not sure we care? Maybe should make these CHECK-DAGs...

[jroelofs]

One more note: why didn't the buildbots catch this?

[EricWF]

In D9864#179372, @jroelofs wrote:

One more note: why didn't the buildbots catch this?

I don't see anything that creates a check-lit rule in CMake or something similar for autoconf. My guess is that they never get run.

[jroelofs]

ping: @EricWF, you should commit this....

[EricWF]

• fix tests

utils/lit/tests/Inputs/googletest-format/DummySubDir/OneTest

 test_name = sys.argv[1].split('=',1)[1]
 if test_name == 'FirstTest.subTestA':
     print('I am subTest A, I PASS')
+    print('[  PASSED  ] 1 test.')
     sys.exit(0)
 elif test_name == 'FirstTest.subTestB':
     print('I am subTest B, I FAIL')

 elif test_name in ('ParameterizedTest/0.subTest',
                    'ParameterizedTest/1.subTest'):
     print('I am a parameterized test, I also PASS')
+    print('[  PASSED  ] 1 test.')
     sys.exit(0)
 else:
     raise SystemExit("error: invalid test name: %r" % (test_name,))

utils/lit/tests/discovery.py

 # RUN: FileCheck --check-prefix=CHECK-BASIC-ERR < %t.err %s
 #
 # CHECK-BASIC-ERR: loading suite config '{{.*}}/discovery/lit.cfg'
-# CHECK-BASIC-ERR: loading local config '{{.*}}/discovery/subdir/lit.local.cfg'
 # CHECK-BASIC-ERR: loading suite config '{{.*}}/discovery/subsuite/lit.cfg'
+# CHECK-BASIC-ERR: loading local config '{{.*}}/discovery/subdir/lit.local.cfg'
 #
 # CHECK-BASIC-OUT: -- Test Suites --
 # CHECK-BASIC-OUT:   sub-suite - 2 tests

 # CHECK-ASEXEC-ERR: load_config from '{{.*}}/discovery/lit.cfg'
 # CHECK-ASEXEC-ERR: loaded config '{{.*}}/discovery/lit.cfg'
 # CHECK-ASEXEC-ERR: loaded config '{{.*}}/exec-discovery/lit.site.cfg'
-# CHECK-ASEXEC-ERR: loading local config '{{.*}}/discovery/subdir/lit.local.cfg'
 # CHECK-ASEXEC-ERR: loading suite config '{{.*}}/discovery/subsuite/lit.cfg'
+# CHECK-ASEXEC-ERR: loading local config '{{.*}}/discovery/subdir/lit.local.cfg'
 #
 # CHECK-ASEXEC-OUT: -- Test Suites --
 # CHECK-ASEXEC-OUT:   sub-suite - 2 tests

utils/lit/tests/unittest-adaptor.py

 # RUN: %{python} %s %{inputs}/unittest-adaptor 2> %t.err
 # RUN: FileCheck < %t.err %s
 #
-# CHECK: unittest-adaptor :: test-one.txt ... ok
 # CHECK: unittest-adaptor :: test-two.txt ... FAIL
+# CHECK: unittest-adaptor :: test-one.txt ... ok
 
 import unittest
 import sys

utils/lit/tests/xunit-output.py

 # CHECK: <?xml version="1.0" encoding="UTF-8" ?>
 # CHECK: <testsuites>
 # CHECK: <testsuite name='test-data' tests='1' failures='0'>
-# CHECK: <testcase classname='test-data.' name='metrics.ini' time='0.00'/>
+# CHECK: <testcase classname='test-data.test-data' name='metrics.ini' time='0.00'/>
 # CHECK: </testsuite>
 # CHECK: </testsuites>
 No newline at end of file