[lit] Fix TestRunner unit test on Windows

[modocache]

Summary

Normally Python converts all newline characters, Windows or Unix,
to Unix newlines when opening a file. However, lit opens files in
binary mode, which does not perform this conversion. As a result,
trailing Windows newlines are not stripped from test input, which
caused a failure in the TestRunner unit test:

FAIL: test_custom (__main__.TestIntegratedTestKeywordParser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\bgesiak\src\llvm\llvm\utils\lit\tests\unit\TestRunner.py", line 109, in test_custom
    self.assertItemsEqual(value, ['a', 'b', 'c'])
AssertionError: Element counts were not equal:
First has 1, Second has 0: 'c\r'
First has 0, Second has 1:  'c'

Fix the discrepancy in behavior across the two platforms by
manually stripping Windows newlines before yielding each line in
the test file.

[modocache]

Friendly ping! Just re-tested this: utils/lit/tests/unit/TestRunner.py still fails on Windows, and this diff fixes it. Is anything in particular holding this back?

[rnk]

lgtm

I think I don't observe this failure because the git repos don't auto-convert LF to CRLF on Windows, but SVN does.

[modocache]

Thanks!

utils/lit/lit/TestRunner.py

try:
             line_number += data.count(to_bytes('\n'), last_match_position,
                                       match_position)
             last_match_position = match_position
 
             # Convert the keyword and line to UTF-8 strings and yield the
             # command. Note that we take care to return regular strings in
             # Python 2, to avoid other code having to differentiate between the
             # str and unicode types.
+            #
+            # Opening the file in binary mode prevented Windows \r newline
+            # characters from being converted to Unix \n newlines, so manually
+            # strip those from the yielded lines.
             keyword,ln = match.groups()
             yield (line_number, to_string(keyword.decode('utf-8')),
-                   to_string(ln.decode('utf-8')))
+                   to_string(ln.decode('utf-8').rstrip('\r')))
     finally:
         f.close()
 
 def getTempPaths(test):
     """Get the temporary location, this is always relative to the test suite
     root, not test source root."""
     execpath = test.getExecPath()
     execdir,execbase = os.path.split(execpath)