Make lit TestRunner.py work in Python 3

[hlopko]

Summary

In Python3 SubstituteCaptures are no longer converted to String implicitly behind the scenes. Converting explicitly makes the TestRunner to work in Python3.

[gribozavr2]

Instead of adding a cast at the point where the value is used, I think it is usually better to fix callers to cast the values themselves.

[tbkka]

Swift uses a wrapper object to track certain types of transforms. We could in theory overhaul Swift's use of lit to avoid that, but it would be a much bigger change.

[tbkka]

Here are some more details:

Swift's lit.cfg wraps certain substitutions in a class that is part of LLVM's lit implementation (specifically class SubstituteCaptures from TestingConfig.py). This class breaks the logic in TestRunner.py on Python3: Python 2.7's regex always stringified its arguments implicitly; Python3 does not. Apparently, the SubstituteCaptures class is not currently used anywhere in LLVM itself, which is why this mismatch isn't breaking LLVM builds.

[compnerd]

I'm not sure if the early string conversion would work. The problem is that the \ replacement would escape the backreference. So you end up with problems in the back-reference. The wrapper ensures that they do not get replaced away as the path separator.

I agree with @tbkka about this being the most concise approach to solving the issue.

@gribozavr2, do you have any other suggestions on how to handle that?

[gribozavr2]

Generally in Python it is preferred to have more strongly typed inputs than stringifying whatever comes in. If you feel like this is the best choice, I don't mind.

llvm/utils/lit/lit/TestRunner.py

def processLine(ln):
                 b = b.replace("\\","\\\\")
             # re.compile() has a built-in LRU cache with 512 entries. In some
             # test suites lit ends up thrashing that cache, which made e.g.
             # check-llvm run 50% slower.  Use an explicit, unbounded cache
             # to prevent that from happening.  Since lit is fairly
             # short-lived, since the set of substitutions is fairly small, and
             # since thrashing has such bad consequences, not bounding the cache
             # seems reasonable.
-            ln = _caching_re_compile(a).sub(b, ln)
+            ln = _caching_re_compile(a).sub(str(b), ln)
 
         # Strip the trailing newline and any extra whitespace.
         return ln.strip()
 
     def processLineToFixedPoint(ln):
         assert isinstance(recursion_limit, int) and recursion_limit >= 0
         origLine = ln
         steps = 0