[libcxx] Add UNSUPPORTED tag to lit. It mirrors REQUIRES.

[EricWF]

Summary

This patch adds support for // UNSUPPORTED: feature. If an excluded feature is found in the list of available features then the test is marked unsupported.

I hope to use this to mark test unsupported if the fail with msan on asan. As well as tests that fail in particular standard modes.

[danalbert]

Modulo a few comments, I think this looks fine, but I don't think I agree with using this to mark tests that fail under a given sanitizer. Specifically, I don't think "unsupported" is semantically correct in this case. If the test is failing under a given sanitizer, there's either something wrong with the test, something wrong with one of our libraries, or something wrong with the sanitizer itself, right? In that case, I think the right thing to do would be to XFAIL them under those conditions.

On a somewhat related note, I noticed that the LIT docs claim we're supposed to be able to label XFAILs with a PR number, but a few quick attempts with this didn't show any useful feedback in the test results. I assume this is because we parse the XFAIL lines ourselves. We should fix this so we can track these better.

test/lit.cfg

91

Whitespace alignment.

Also, what parts of the test suite would need REQUIRES: <target-triple>? I don't think I've found any tests that are platform specific. If not, YAGNI.

[EricWF]

Removed checking triple target for REQUIRES and EXCLUDES. I though that LIT's TestRunner.py check the triple target but I was mistaken. YAGNI :P

[EricWF]

Thanks for looking at this Dan.

A lot of our tests replace operator new and operator delete. MSAN and ASAN also replace operator new/delete. This causes a linking error. Theres nothing wrong with the tests other than they just can't be run with MSAN or ASAN. I think these should be considered unsupported as opposed to expected failures.

I hope that clarifies things.

[EricWF]

Change name of EXCLUDES to UNSUPPORTED.

[danalbert]

Ah, alright. Unsupported would be correct then.

test/lit.cfg

     def _execute(self, test, lit_config):
         # Extract test metadata from the test file.
         requires = []
+        unsupported = []
         with open(test.getSourcePath()) as f:
             for ln in f:
                 if 'XFAIL:' in ln:

                 elif 'REQUIRES:' in ln:
                     items = ln[ln.index('REQUIRES:') + 9:].split(',')
                     requires.extend([s.strip() for s in items])
+                elif 'UNSUPPORTED:' in ln:
+                    items = ln[ln.index('UNSUPPORTED:') + 12:].split(',')
+                    unsupported.extend([s.strip() for s in items])
                 elif not ln.strip().startswith("//") and ln.strip():
                     # Stop at the first non-empty line that is not a C++
                     # comment.

                     "Test requires the following features: %s" % (
                       ', '.join(missing_required_features),))
 
+        unsupported_features = [f for f in unsupported
+                             if f in test.config.available_features]
+        if unsupported_features:
+            return (lit.Test.UNSUPPORTED,
+                    "Test is unsupported with the following features: %s" % (
+                       ', '.join(unsupported_features),))
+
         # Evaluate the test.
         return self._evaluate_test(test, lit_config)