[lit] RUN commands without stdin.

[csigg]

Summary

This changes the initial stdin of a lit RUN command to be None, so that tools taking input from stdin do not see a zero-length input.

This will prevent issues like that one fixed in commit 720dd81.

[probinson]

So what would such a tool see? Would it get an error trying to open stdin, or something else?

Needs a test in lit's own test suite.

[csigg]

Add test, fix external shell mode.

[csigg]

Fix bazel config to actually test lit tests.

Create a separate test instead of reusing the shtest-timeout one.

[csigg]

In D135067#3830815, @probinson wrote:

So what would such a tool see? Would it get an error trying to open stdin, or something else?

A tool reading from stdin would get stdin, not an error and not an empty string (which it did before this change).

Needs a test in lit's own test suite.

Done.

[probinson]

In D135067#3832848, @csigg wrote:

In D135067#3830815, @probinson wrote:

So what would such a tool see? Would it get an error trying to open stdin, or something else?

A tool reading from stdin would get stdin, not an error and not an empty string (which it did before this change).

Ah. I interpreted that to mean "it will hang waiting for input" and then I see the test expects lit to timeout the test. That works.

LGTM

[csigg]

Fix clang tests that relied on empty input or accidentally did not provide the input in the RUN directive.

[jhenderson]

llvm/utils/lit/tests/Inputs/shtest-stdin/print-stdin.py

6

Nit: add newline at EOF.

llvm/utils/lit/tests/shtest-stdin.py

29

Nit: Don't have multiple \n at EOF (there should be exactly one).

llvm/utils/lit/lit/TestRunner.py

if isinstance(cmd, ShUtil.Seq):
                 res = _executeShCmd(cmd.rhs, shenv, results, timeoutHelper)
             return res
 
         raise ValueError('Unknown shell command: %r' % cmd.op)
     assert isinstance(cmd, ShUtil.Pipeline)
 
     procs = []
     proc_not_counts = []
-    default_stdin = subprocess.PIPE
+    default_stdin = None
     stderrTempFiles = []
     opened_files = []
     named_temp_files = []
     builtin_commands = set(['cat', 'diff'])
     builtin_commands_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "builtin_commands")
     inproc_builtins = {'cd': executeBuiltinCd,
                        'export': executeBuiltinExport,
                        'echo': executeBuiltinEcho,

llvm/utils/lit/lit/util.py

 
 def executeCommand(command, cwd=None, env=None, input=None, timeout=0,
                    redirect_stderr=False):
     """Execute command ``command`` (list of arguments or string) with.
 
     * working directory ``cwd`` (str), use None to use the current
       working directory
     * environment ``env`` (dict), use None for none
-    * Input to the command ``input`` (str), use string to pass
+    * Input to the command ``input`` (str), use None to pass
       no input.
     * Max execution time ``timeout`` (int) seconds. Use 0 for no timeout.
     * ``redirect_stderr`` (bool), use True if redirect stderr to stdout
 
     Returns a tuple (out, err, exitCode) where
     * ``out`` (str) is the standard output of running the command
     * ``err`` (str) is the standard error of running the command
     * ``exitCode`` (int) is the exitCode of running the command
 
     If the timeout is hit an ``ExecuteCommandTimeoutException``
     is raised.
 
     """
     if input is not None:
         input = to_bytes(input)
     err_out = subprocess.STDOUT if redirect_stderr else subprocess.PIPE
     p = subprocess.Popen(command, cwd=cwd,
-                         stdin=subprocess.PIPE,
+                         stdin=subprocess.PIPE if input else None,
                          stdout=subprocess.PIPE,
                          stderr=err_out,
                          env=env, close_fds=kUseCloseFDs)
     timerObject = None
     # FIXME: Because of the way nested function scopes work in Python 2.x we
     # need to use a reference to a mutable object rather than a plain
     # bool. In Python 3 we could use the "nonlocal" keyword but we need
     # to support Python 2 as well.

llvm/utils/lit/tests/Inputs/shtest-stdin/lit.cfg

+# -*- Python -*-
+import os
+import sys
+
+import lit.formats
+
+config.name = 'shtest-stdin'
+
+externalShell = lit_config.params.get('external', '1') != '0'
+config.test_format = lit.formats.ShTest(execute_external=externalShell)
+config.suffixes = ['.py']
+
+config.test_source_root = os.path.dirname(__file__)
+config.test_exec_root = config.test_source_root
+config.target_triple = '(unused)'
+src_root = os.path.join(config.test_source_root, '..')
+
+pythonpath_list = [src_root]
+# Ensure the user's PYTHONPATH is included.
+if 'PYTHONPATH' in os.environ:
+    pythonpath_list.append(os.environ['PYTHONPATH'])
+if 'PYTHONPATH' in config.environment:
+    pythonpath_list.append(config.environment['PYTHONPATH'])
+config.environment['PYTHONPATH'] = os.pathsep.join(pythonpath_list)
+
+config.substitutions.append(('%{python}', '"%s"' % (sys.executable)))

llvm/utils/lit/tests/Inputs/shtest-stdin/print-stdin.py

+# Checks that lit provides no input to stdin.
+# RUN: %{python} %s
+import sys
+
+print(sys.stdin.read())
 No newline at end of file

llvm/utils/lit/tests/shtest-stdin.py

+# REQUIRES: lit-max-individual-test-time
+
+###############################################################################
+# Check tests do not receive spurious stdin input.
+###############################################################################
+# RUN: not %{lit} %{inputs}/shtest-stdin/print-stdin.py \
+# RUN:   --param external=0 --timeout 1 \
+# RUN: | FileCheck --check-prefix=CHECK-STDIN %s
+#
+# RUN: not %{lit} %{inputs}/shtest-stdin/print-stdin.py \
+# RUN:   --param external=1 --timeout 1 \
+# RUN: | FileCheck --check-prefix=CHECK-STDIN %s
+#
+# CHECK-STDIN: TIMEOUT: shtest-stdin :: print-stdin.py
+
+###############################################################################
+# Check tests do receive piped stdin input.
+###############################################################################
+# RUN: echo foobar | %{lit} %{inputs}/shtest-stdin/print-stdin.py \
+# RUN:   --param external=0 -a \
+# RUN: | FileCheck --check-prefix=CHECK-PIPE %s
+#
+# RUN: echo foobar | %{lit} %{inputs}/shtest-stdin/print-stdin.py \
+# RUN:   --param external=1 -a \
+# RUN: | FileCheck --check-prefix=CHECK-PIPE %s
+#
+# CHECK-PIPE: foobar
+

utils/bazel/llvm-project-overlay/llvm/utils/lit/tests/BUILD.bazel

lit_test(
             "lit.cfg",
             "lit.site.cfg",
             "check-tested-lit-timeout-ability",
             "//llvm:FileCheck",
             "//llvm:count",
             "//llvm:not",
         ] + glob(["Inputs/**"]),
     )
-    for src in glob(["*/*.py"])
+    for src in glob(
+        include = ["**/*.py"],
+        exclude = [
+            "Inputs/**",
+            # FIXME: these tests fail under bazel.
+            "discovery.py",
+            "max-time.py",
+            "selecting.py",
+            "shtest-recursive-substitution.py",
+        ]
+    )
 ]