[lit] Test changes to make it work with bazel

[csigg]

Summary

These non-functional changes will make it easier to add the lit tests to the bazel build (see utils/bazel).

[csigg]

Include mlir/lit.cfg.py change.

[bkramer]

This should be fine.

[csigg]

Rebase.

[stella.stamenova]

This change has caused intermittent failures in reorder.py on Windows. I wonder why the changes to the test were necessary and if we can update it so it works consistently on Windows as well as with bazel:

2022-10-20T18:53:12.1217293Z -- Testing: 67311 tests, 32 workers --
2022-10-20T18:57:48.7694530Z Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90
2022-10-20T18:57:48.7695526Z FAIL: lit :: reorder.py (64557 of 67311)
2022-10-20T18:57:48.7697302Z ******************** TEST 'lit :: reorder.py' FAILED ********************
2022-10-20T18:57:48.7697817Z Script:
2022-10-20T18:57:48.7698264Z --
2022-10-20T18:57:48.7698690Z : 'RUN: at line 3';   cp Inputs/reorder/lit_test_times Inputs/reorder/.lit_test_times.txt
2022-10-20T18:57:48.7699728Z : 'RUN: at line 4';   not env -u FILECHECK_OPTS "C:\hostedtoolcache\windows\Python\3.9.13\x64\python3.exe" D:\a\_work\1\s\llvm-project\llvm\utils\lit\lit.py -j1 Inputs/reorder > D:\a\_work\1\b\llvm\Release\utils\lit\tests\Output\reorder.py.tmp.out
2022-10-20T18:57:48.7700832Z : 'RUN: at line 5';   FileCheck --check-prefix=TIMES < Inputs/reorder/.lit_test_times.txt D:\a\_work\1\b\llvm\Release\utils\lit\tests\reorder.py
2022-10-20T18:57:48.7701719Z : 'RUN: at line 6';   FileCheck < D:\a\_work\1\b\llvm\Release\utils\lit\tests\Output\reorder.py.tmp.out D:\a\_work\1\b\llvm\Release\utils\lit\tests\reorder.py
2022-10-20T18:57:48.7702285Z --
2022-10-20T18:57:48.7702796Z Exit Code: 1
2022-10-20T18:57:48.7702962Z 
2022-10-20T18:57:48.7703273Z Command Output (stdout):
2022-10-20T18:57:48.7703761Z --
2022-10-20T18:57:48.7704025Z $ ":" "RUN: at line 3"
2022-10-20T18:57:48.7704480Z $ "cp" "Inputs/reorder/lit_test_times" "Inputs/reorder/.lit_test_times.txt"
2022-10-20T18:57:48.7704930Z $ ":" "RUN: at line 4"
2022-10-20T18:57:48.7705561Z $ "not" "env" "-u" "FILECHECK_OPTS" "C:\hostedtoolcache\windows\Python\3.9.13\x64\python3.exe" "D:\a\_work\1\s\llvm-project\llvm\utils\lit\lit.py" "-j1" "Inputs/reorder"
2022-10-20T18:57:48.7706327Z $ ":" "RUN: at line 5"
2022-10-20T18:57:48.7706759Z $ "FileCheck" "--check-prefix=TIMES" "D:\a\_work\1\b\llvm\Release\utils\lit\tests\reorder.py"
2022-10-20T18:57:48.7707138Z # command stderr:
2022-10-20T18:57:48.7707706Z D:\a\_work\1\b\llvm\Release\utils\lit\tests\reorder.py:14:15: error: TIMES-NEXT: expected string not found in input
2022-10-20T18:57:48.7708049Z # TIMES-NEXT: new-test.txt
2022-10-20T18:57:48.7708214Z               ^
2022-10-20T18:57:48.7708414Z <stdin>:5:21: note: scanning from here
2022-10-20T18:57:48.7708620Z 1.562667e-02 aaa.txt
2022-10-20T18:57:48.7708791Z                     ^
2022-10-20T18:57:48.7708877Z 
2022-10-20T18:57:48.7709026Z Input file: <stdin>
2022-10-20T18:57:48.7709254Z Check file: D:\a\_work\1\b\llvm\Release\utils\lit\tests\reorder.py
2022-10-20T18:57:48.7709402Z 
2022-10-20T18:57:48.7709620Z -dump-input=help explains the following input dump.
2022-10-20T18:57:48.7709744Z 
2022-10-20T18:57:48.7709894Z Input was:
2022-10-20T18:57:48.7710035Z <<<<<<
2022-10-20T18:57:48.7710208Z          1: 4.200000e+01 not-executed.txt 
2022-10-20T18:57:48.7710439Z          2: 1.563215e-02 subdir/ccc.txt 
2022-10-20T18:57:48.7710660Z          3: 1.561761e-02 bbb.txt 
2022-10-20T18:57:48.7711008Z          4: -1.561570e-02 fff.txt 
2022-10-20T18:57:48.7711315Z          5: 1.562667e-02 aaa.txt 
2022-10-20T18:57:48.7711527Z next:14                         X error: no match found
2022-10-20T18:57:48.7711719Z >>>>>>
2022-10-20T18:57:48.7711780Z 
2022-10-20T18:57:48.7711955Z error: command failed with exit status: 1
2022-10-20T18:57:48.7712068Z 
2022-10-20T18:57:48.7712209Z --
2022-10-20T18:57:48.7712271Z 
2022-10-20T18:57:48.7712418Z ********************
2022-10-20T18:58:11.6923022Z Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.. 
2022-10-20T18:58:12.6993950Z ********************
2022-10-20T18:58:12.6995485Z Failed Tests (1):
2022-10-20T18:58:12.6995765Z   lit :: reorder.py

llvm/utils/lit/tests/Inputs/reorder/lit_test_times

 42.0 not-executed.txt
 3.0 subdir/ccc.txt
 2.0 bbb.txt
 -0.2 fff.txt
 0.1 aaa.txt

llvm/utils/lit/tests/lit.cfg

 
 llvm_src_root = getattr(config, 'llvm_src_root', None)
 if llvm_src_root:
   # ``test_source_root`` may be in LLVM's binary build directory which does not contain
   # ``lit.py``, so use `llvm_src_root` instead.
   lit_path = os.path.join(llvm_src_root, 'utils', 'lit')
 else:
   lit_path = os.path.join(config.test_source_root, '..')
+lit_path = os.path.abspath(lit_path)
 
 # Required because some tests import the lit module
 if llvm_config:
   llvm_config.with_environment('PYTHONPATH', lit_path, append_path=True)
 else:
-  config.environment['PYTHONPATH'] = os.pathsep.join([lit_path])
+  config.environment['PYTHONPATH'] = lit_path
 # Do not add user-site packages directory to the python search path. This avoids test failures if there's an
 # incompatible lit module installed inside the user-site packages directory, as it gets prioritized over the lit
 # from the PYTHONPATH.
 config.environment['PYTHONNOUSERSITE'] = '1'
 
 # Add llvm and lit tools directories if this config is being loaded indirectly.
 # In this case, we can also expect llvm_config to have been imported correctly.
 for attribute in ('llvm_tools_dir', 'lit_tools_dir'):
     directory = getattr(config, attribute, None)
     if directory:
         llvm_config.with_environment('PATH', directory, append_path=True)
 
 # This test suite calls %{lit} to test lit's behavior for the sample test
 # suites in %{inputs}.  This test suite's results are then determined in part
 # by %{lit}'s textual output, which includes the output of FileCheck calls
 # within %{inputs}'s test suites.  Thus, %{lit} clears environment variables
 # that can affect FileCheck's output.  It also includes "--order=lexical -j1"
 # to ensure predictable test order, as it is often required for FileCheck
 # matches.
-config.substitutions.append(('%{inputs}', os.path.join(
-    config.test_source_root, 'Inputs')))
+config.substitutions.append(('%{inputs}', 'Inputs'))
 config.substitutions.append(('%{lit}', '%{lit-no-order-opt} --order=lexical'))
 config.substitutions.append(('%{lit-no-order-opt}',
     "{env} %{{python}} {lit} -j1".format(
         env="env -u FILECHECK_OPTS",
         lit=os.path.join(lit_path, 'lit.py'))))
 config.substitutions.append(('%{python}', '"%s"' % (sys.executable)))
 
 # Enable coverage.py reporting, assuming the coverage module has been installed

llvm/utils/lit/tests/reorder.py

 ## Check that we can reorder test runs.
 
-# RUN: cp %{inputs}/reorder/.lit_test_times.txt %{inputs}/reorder/.lit_test_times.txt.orig
+# RUN: cp %{inputs}/reorder/lit_test_times %{inputs}/reorder/.lit_test_times.txt
 # RUN: not %{lit-no-order-opt} %{inputs}/reorder > %t.out
-# RUN: cp %{inputs}/reorder/.lit_test_times.txt %{inputs}/reorder/.lit_test_times.txt.new
-# RUN: cp %{inputs}/reorder/.lit_test_times.txt.orig %{inputs}/reorder/.lit_test_times.txt
-# RUN: not diff %{inputs}/reorder/.lit_test_times.txt.new %{inputs}/reorder/.lit_test_times.txt.orig
-# RUN: FileCheck --check-prefix=TIMES --implicit-check-not= < %{inputs}/reorder/.lit_test_times.txt.new %s
+# RUN: FileCheck --check-prefix=TIMES < %{inputs}/reorder/.lit_test_times.txt %s
 # RUN: FileCheck < %t.out %s
 # END.
 
 # TIMES: not-executed.txt
 # TIMES-NEXT: subdir/ccc.txt
 # TIMES-NEXT: bbb.txt
 # TIMES-NEXT: -{{.*}} fff.txt
 # TIMES-NEXT: aaa.txt