[lit] Provide consistent stdout and stderr in the internal shell

[ldionne]

Summary

Before this patch, evaluating commands in the internal shell results in
a (out, err, exitCode, timeoutInfo) quadruple where the standard output
contains a detailed trace of everything that was executed, and where
the standard error doesn't contain anything.

Instead, it should return the standard output/error of the executed
commands, like an external shell would. Otherwise, it is impossible
retrieve the standard output/error when calling executeScriptInternal
without parsing that trace.

[ldionne]

This is going to require a lot of fixes in Lit's own tests. I'd like to know whether we're interested to proceed this way before I go ahead and fix them.

Note that without this change, it's fairly difficult to extract precise stdout or stderr results after running a command through the internal Shell format, as explained in the commit message. That's the motivation for the change.

[yln]

I am in favor of this. It might also help with D80694 @varungandhi-apple
Does this change the output for normal test runs (other than for lit itself)?

Please also add the usual lit reviewers when you have the final patch.

[ldionne]

This was fixed by D156954!

llvm/utils/lit/lit/TestRunner.py

def executeScriptInternal(test, litConfig, tmpBase, commands, cwd):
     except InternalShellError:
         e = sys.exc_info()[1]
         exitCode = 127
         results.append(
             ShellCommandResult(e.command, '', e.message, exitCode, False))
 
     out = err = ''
     for i,result in enumerate(results):
-        # Write the command line run.
-        out += '$ %s\n' % (' '.join('"%s"' % s
-                                    for s in result.command.args),)
-
         # If nothing interesting happened, move on.
         if litConfig.maxIndividualTestTime == 0 and \
                result.exitCode == 0 and \
                not result.stdout.strip() and not result.stderr.strip():
             continue
 
         # Otherwise, something failed or was printed, show it.
-
-        # Add the command output, if redirected.
-        for (name, path, data) in result.outputFiles:
-            if data.strip():
-                out += "# redirected output from %r:\n" % (name,)
-                data = to_string(data.decode('utf-8', errors='replace'))
-                if len(data) > 1024:
-                    out += data[:1024] + "\n...\n"
-                    out += "note: data was truncated\n"
-                else:
-                    out += data
-                out += "\n"
-
         if result.stdout.strip():
-            out += '# command output:\n%s\n' % (result.stdout,)
+            out += result.stdout
         if result.stderr.strip():
-            out += '# command stderr:\n%s\n' % (result.stderr,)
-        if not result.stdout.strip() and not result.stderr.strip():
-            out += "note: command had no output on stdout or stderr\n"
+            err += result.stderr
 
         # Show the error conditions:
         if result.exitCode != 0:
             # On Windows, a negative exit code indicates a signal, and those are
             # easier to recognize or look up if we print them in hex.
             if litConfig.isWindows and result.exitCode < 0:
                 codeStr = hex(int(result.exitCode & 0xFFFFFFFF)).rstrip("L")
             else:
                 codeStr = str(result.exitCode)
-            out += "error: command failed with exit status: %s\n" % (
+            err += "\nerror: command failed with exit status: %s\n" % (
                 codeStr,)
         if litConfig.maxIndividualTestTime > 0 and result.timeoutReached:
-            out += 'error: command reached timeout: %s\n' % (
+            err += '\nerror: command reached timeout: %s\n' % (
                 str(result.timeoutReached),)
 
     return out, err, exitCode, timeoutInfo
 
 def executeScript(test, litConfig, tmpBase, commands, cwd):
     bashPath = litConfig.getBashPath()
     isWin32CMDEXE = (litConfig.isWindows and not bashPath)
     script = tmpBase + '.script'