Use raw strings to avoid deprecation warnings in regexp patterns

[amccarth]

Summary

In LLDB, where tests run with the debug version of Python, we get a series of deprecation warnings because \( escape sequences are treated as part of the string literal rather than an escaped paren for the regexp pattern.

I believe the change preserves the original intent rather than the actual behavior when running with Python 3, but another pair of eyes would be appreciated.

[jdenny]

This looks equivalent to me.

But how do I see the bad behavior or deprecation warnings? Python 3.6.7 with -ddd didn't reveal anything on a simple example I just tried.

Thanks.

[jdenny]

Never mind. I see it now. LGTM.

llvm/utils/lit/lit/TestRunner.py

52

The fact that \' will remain as \' bugs me because the ' need not be escaped as part of the regex. Hopefully that won't be deprecated one day.

[amccarth]

In D62882#1530159, @jdenny wrote:

But how do I see the bad behavior or deprecation warnings? Python 3.6.7 with -ddd didn't reveal anything on a simple example I just tried.

I see the deprecation warnings when running the LLDB tests on Windows, which selects the debug version of Python 3.6.7.

I don't think other LLVM tests are run with the debug version of Python, so the warnings do not show. My larger goal here is the clean up the Windows LLDB test output so we can see the real problems and fix enough of them to make it reasonable to run the tests on some (more?) the Windows bots.

Thanks for the review.

llvm/utils/lit/lit/TestRunner.py

52

Would you prefer the following?

kPdbgRegex = '%dbg\\(([^)\'"]*)\\)'

(It's kind of disappointing the Python raw string literals aren't always the best choice.)

[amccarth]

Changed the approach in the first fix to use explicit escaping after Joel Denny's comment got me to re-think it.

[amccarth]

llvm/utils/lit/lit/TestRunner.py

52

I've gone ahead and switched this to a non-raw string and just added the extra escaping.

[jdenny]

llvm/trunk/utils/lit/lit/TestRunner.py

1430 ↗

(On Diff #203624)

The \+ doesn't need the escape here either, right? While your patch doesn't impact this one, it's another unknown/unnecessary escape at the regex level, similar to the \' we discussed. I'm not a python expert, and I don't know whether it's actually worthwhile to worry about unnecessary escapes at the regex level.

Thanks for the patch, by the way.

llvm/utils/lit/lit/TestRunner.py

 # run command pipeline such that ARG specifies the pipeline's source line
 # number.  lit later expands each %dbg(ARG) to a command that behaves as a null
 # command in the target shell so that the line number is seen in lit's verbose
 # mode.
 #
 # This regex captures ARG.  ARG must not contain a right parenthesis, which
 # terminates %dbg.  ARG must not contain quotes, in which ARG might be enclosed
 # during expansion.
-kPdbgRegex = '%dbg\(([^)\'"]*)\)'
+kPdbgRegex = '%dbg\\(([^)\'"]*)\\)'
 
 class ShellEnvironment(object):
 
     """Mutable shell environment containing things like CWD and env vars.
 
     Environment variables are not implemented, but cwd tracking is.
     """
 
def _handleTag(line_number, line, output):
         return (not line.strip() or output)
 
     @staticmethod
     def _handleCommand(line_number, line, output, keyword):
         """A helper for parsing COMMAND type keywords"""
         # Trim trailing whitespace.
         line = line.rstrip()
         # Substitute line number expressions
-        line = re.sub('%\(line\)', str(line_number), line)
+        line = re.sub(r'%\(line\)', str(line_number), line)
 
         def replace_line_number(match):
             if match.group(1) == '+':
                 return str(line_number + int(match.group(2)))
             if match.group(1) == '-':
                 return str(line_number - int(match.group(2)))
-        line = re.sub('%\(line *([\+-]) *(\d+)\)', replace_line_number, line)
+        line = re.sub(r'%\(line *([\+-]) *(\d+)\)', replace_line_number, line)
         # Collapse lines with trailing '\\'.
         if output and output[-1][-1] == '\\':
             output[-1] = output[-1][:-1] + line
         else:
             if output is None:
                 output = []
             pdbg = "%dbg({keyword} at line {line_number})".format(
                 keyword=keyword,