Page MenuHomePhabricator

Basic JUnit XML support for llvm-lit
ClosedPublic

Authored by theraven on Aug 14 2014, 2:40 AM.

Details

Reviewers
russell.gallop
Summary

Export JUnit XML that Jenkins can understand. We've been using this for about 18 months in our internal Jenkins. It's not a perfect mapping, but hopefully it's adequate for other people to build on.

Diff Detail

Event Timeline

theraven updated this revision to Diff 12492.Aug 14 2014, 2:40 AM
theraven retitled this revision from to Basic JUnit XML support for llvm-lit.
theraven updated this object.
theraven edited the test plan for this revision. (Show Details)
emaste added a subscriber: emaste.Aug 14 2014, 1:10 PM
russell.gallop accepted this revision.Oct 22 2014, 3:33 AM
russell.gallop added a reviewer: russell.gallop.
russell.gallop added a subscriber: russell.gallop.

Hi,

This would be very useful to us. I tried this patch and it basically works with our CI server too.

I would suggest a couple of minor changes to improve the output format, see inline comments. FWIW the change looks good to me.

Regards
Russ

utils/lit/lit/Test.py
200

Omitting the test name from the class name seems like a good idea to me. It allows a junit consumer to group tests by folder. (This is rather a moot point as our tests are files and folders rather than classes and packages).

-        xml = "<testcase classname='" + ".".join(self.path_in_suite)
+        xml = "<testcase classname='" + ".".join(self.path_in_suite[:-1])
200–201

Formatting.

     def getJUnitXML(self):
-        xml = "<testcase classname='" + ".".join(self.path_in_suite[:-1])
-        xml += "' name='" + '/'.join(self.path_in_suite) + "'"
+        xml = "<testcase classname='" + ".".join(self.path_in_suite[:-1]) + "'"
+        xml += " name='" + '/'.join(self.path_in_suite) + "'"
         if self.result.code.isFailure:
201–202

Add elapsed time.

         xml += " name='" + '/'.join(self.path_in_suite) + "'"
+        xml += " time='%.2f'" % (self.result.elapsed,)
         if self.result.code.isFailure:
utils/lit/lit/main.py
446

Some consumers use the errors attribute to determine the number of passes (tests - failures - errors).

            xmlFile.write(" tests='" + str(s['passes'] + s['failures']) + "'")
+            xmlFile.write(" errors='0'")
            xmlFile.write(" failures='" + str(s['failures']) + "'>\n")
This revision is now accepted and ready to land.Oct 22 2014, 3:33 AM
Eugene.Zelenko added a subscriber: Eugene.Zelenko.

Committed in rL223163.