Page MenuHomePhabricator

[lldb/Lit] Change the lldbtest format to behave more like shell test.
ClosedPublic

Authored by JDevlieghere on Jan 24 2020, 2:58 PM.

Details

Summary

The current lldbtest format has a number of shortcomings, all related to how we omit information based on why the test fails. For example, a successful test would print nothing, even when -a is passed to lit. It's not up to the test format to decide whether to print something or not, that's handled by lit itself. For other test results we would sometimes print stdout and stderr, but sometimes we would forget, such as when a timeout was reached or we couldn't parse the dotest output.

This patch changes the lldbtest format and makes it behave more like lit. We now always print the dotest invocation, the exit code, the output to stdout & stderr. If you're used to dealing with ShTests in lit, this will feel all very familiar.

Diff Detail

Event Timeline

JDevlieghere created this revision.Jan 24 2020, 2:58 PM
Herald added a project: Restricted Project. · View Herald TranscriptJan 24 2020, 2:58 PM
Herald added a subscriber: abidh. · View Herald Transcript
vsk accepted this revision as: vsk.Jan 24 2020, 3:44 PM
vsk added a subscriber: vsk.

Nice!

This revision is now accepted and ready to land.Jan 24 2020, 3:44 PM
This revision was automatically updated to reflect the committed changes.
mgorny added a subscriber: mgorny.Jan 30 2020, 9:16 AM

I suppose you need to make the timeout branch entirely separate from the non-timeout branch, as the variable assignment does not take place then.

lldb/test/API/lldbtest.py
100

This crashes on timeout since exitCode is not declared:

UNRESOLVED: lldb-api :: tools/lldb-vscode/variables/TestVSCode_variables.py (1906 of 1907)
******************** TEST 'lldb-api :: tools/lldb-vscode/variables/TestVSCode_variables.py' FAILED ********************
Exception during script execution:
Traceback (most recent call last):
  File "/home/mgorny/llvm-project/llvm/utils/lit/lit/worker.py", line 91, in _execute_test_handle_errors
    return _adapt_result(test.config.test_format.execute(test, lit_config))
  File "/home/mgorny/llvm-project/llvm/tools/lldb/test/API/lldbtest.py", line 100, in execute
    ' '.join(cmd), exitCode)
UnboundLocalError: local variable 'exitCode' referenced before assignment
105

I suppose this will crash as well, and below.