TestMultilineExpr: validate evaluation for expressions spread over multiple lines

[sgraenitz]

Summary

When LLDB successfully parses a command (like "expression" in this case) and determines incomplete input, the user can continue typing on multiple lines (in this case "2+3"). This should provide the correct result.
Note that LLDB reverts input from the additional lines, so they are not present in the output.

[aprantl]

lit/Expr/TestMultilineExpr.test

4

"reverts" -> "joins"?

4

or did you mean "hides"

11

There's no newline at end of file :-)

[sgraenitz]

Addressing Adrian's comments

[aprantl]

Does that mean we can remove ./packages/Python/lldbsuite/test/expression_command/multiline/TestMultilineExpressions.py ?

[sgraenitz]

lit/Expr/TestMultilineExpr.test

10

Maybe it's nitpicking, but I'd actually prefer not to match this specific string. It could be any human-readable instruction. However, if I replace it with {{.*}} lit will complain that the match doesn't start on the next line:

TestMultilineExpr.test:9:15: error: CHECK-NEXT: is on the same line as previous match
# CHECK-NEXT: {{.*}}
              ^
<stdin>:9:18: note: 'next' match was here
(lldb) expression
                 ^
<stdin>:9:18: note: previous match ended here
(lldb) expression
                 ^

Any ideas how to get it right?

[teemperor]

Can't you just check for '5', as this is the only information we actually actually care about here?

[sgraenitz]

Remove old Python test in ./packages/Python/lldbsuite/test/expression_command/multiline/

[aprantl]

For those following at home: the point of this exercise is to get rid of the -expect-based TestMultilineExpressions.py testcase that kept failing on build bots.

[aprantl]

lit/Expr/TestMultilineExpr.test

10

why not:

# CHECK: (lldb) expression
# CHECK: (int) {{.*}} = 5

?

[sgraenitz]

In D52270#1239641, @teemperor wrote:

Can't you just check for '5', as this is the only information we actually actually care about here?

Yes that would be the simplest way, but as LLDB still echoes all commands including comments, all CHECK lines are self-fulfilling prophecies.
This is what FileCheck will see in this case:

(lldb) command source -s 0 '/Users/sgranitz/Develop/lldb-llvm/git-svn/llvm/tools/lldb/lit/lit-lldb-init'
Executing commands in '/Users/sgranitz/Develop/lldb-llvm/git-svn/llvm/tools/lldb/lit/lit-lldb-init'.
(lldb) # LLDB init file for the LIT tests.
(lldb) settings set symbols.enable-external-lookup false
(lldb) command source -s 0 '/Users/sgranitz/Develop/lldb-llvm/git-svn/llvm/tools/lldb/lit/Expr/TestMultilineExpr.test'
Executing commands in '/Users/sgranitz/Develop/lldb-llvm/git-svn/llvm/tools/lldb/lit/Expr/TestMultilineExpr.test'.
(lldb) # RUN: %lldb -b -s %s | FileCheck %s
(lldb) # In terminal sessions LLDB hides input from subsequent lines so it's not visible in the output we check below.
(lldb) expression
Enter expressions, then terminate with an empty line to evaluate:
(int) $0 = 5
(lldb) # CHECK: (lldb) expression
(lldb) # CHECK-NEXT: Enter expressions, then terminate with an empty line to evaluate:
(lldb) # CHECK-NEXT: (int) {{.*}} = 5

[aprantl]

LLDB still echoes all commands including comments

What do you think about fixing that before landing this patch? Then we don't need to work around it.

[sgraenitz]

In D52270#1239685, @aprantl wrote:

What do you think about fixing that before landing this patch? Then we don't need to work around it.

Hm, we didn't finally decide for the fix and I don't want to raise the pressure on it artificially.
I think it will anyway reveal more places, where we forgot about this little detail and thus provide opportunity to improve also this test.

Of course, this also means that I can keep matching the specific string for this (hopefully short) foreseeable future.

lit/Expr/TestMultilineExpr.test

10

I think we can keep it like this for now.

[sgraenitz]

Simplify test as proposed in review

[sgraenitz]

Requires: https://reviews.llvm.org/D52788 (Add EchoCommentCommands to CommandInterpreterRunOptions in addition to the existing EchoCommands and expose both as interpreter settings)

lit/Expr/TestMultilineExpr.test

+# RUN: %lldb -b -s %s | FileCheck %s
+
+# In terminal sessions LLDB hides input from subsequent lines so it's not visible in the output we check below.
+expression
+2+
+3
+
+# CHECK: (lldb) expression
+# CHECK-NEXT: Enter expressions, then terminate with an empty line to evaluate:
+# CHECK-NEXT: (int) {{.*}} = 5

packages/Python/lldbsuite/test/expression_command/multiline/Makefile

-LEVEL = ../../make
-
-C_SOURCES := main.c
-
-include $(LEVEL)/Makefile.rules

packages/Python/lldbsuite/test/expression_command/multiline/TestMultilineExpressions.py

-"""Test multiline expressions."""
-
-from __future__ import print_function
-
-import os
-import lldb
-from lldbsuite.test.decorators import *
-from lldbsuite.test.lldbtest import *
-from lldbsuite.test import lldbutil
-
-
-class MultilineExpressionsTestCase(TestBase):
-
-    mydir = TestBase.compute_mydir(__file__)
-    NO_DEBUG_INFO_TESTCASE = True
-
-    def setUp(self):
-        # Call super's setUp().
-        TestBase.setUp(self)
-        # Find the line number to break on inside main.cpp.
-        self.line = line_number('main.c', 'break')
-
-    @skipIfRemote
-    @expectedFailureAll(
-        oslist=["windows"],
-        bugnumber="llvm.org/pr22274: need a pexpect replacement for windows")
-    def test_with_run_commands(self):
-        """Test that multiline expressions work correctly"""
-        self.build()
-        import pexpect
-        exe = self.getBuildArtifact("a.out")
-        prompt = "(lldb) "
-
-        # So that the child gets torn down after the test.
-        self.child = pexpect.spawn(
-            '%s %s %s' %
-            (lldbtest_config.lldbExec, self.lldbOption, exe))
-        child = self.child
-        # Turn on logging for what the child sends back.
-        if self.TraceOn():
-            child.logfile_read = sys.stdout
-
-        # Set the breakpoint, run the inferior, when it breaks, issue print on
-        # the various convenience variables.
-        child.expect_exact(prompt)
-        child.sendline('breakpoint set -f main.c -l %d' % self.line)
-        child.expect_exact(prompt)
-        child.sendline('run')
-        child.expect_exact("stop reason = breakpoint 1.1")
-        child.expect_exact(prompt)
-        child.sendline('expr')
-        child.expect_exact('1:')
-
-        child.sendline('2+')
-        child.expect_exact('2:')
-
-        child.sendline('3')
-        child.expect_exact('3:')
-
-        child.sendline('')
-        child.expect_exact(prompt)
-        self.expect(child.before, exe=False,
-                    patterns=['= 5'])
-
-    @skipIfRemote
-    @expectedFailureAll(
-        oslist=["windows"],
-        bugnumber="llvm.org/pr22274: need a pexpect replacement for windows")
-    def test_empty_list(self):
-        """Test printing an empty list of expressions"""
-        import pexpect
-        prompt = "(lldb) "
-
-        # So that the child gets torn down after the test
-        self.child = pexpect.spawn(
-                "%s %s" %
-                (lldbtest_config.lldbExec, self.lldbOption))
-        child = self.child
-
-        # Turn on logging for what the child sends back.
-        if self.TraceOn():
-            child.logfile_read = sys.stdout
-
-        # We expect a prompt, then send "print" to start a list of expressions,
-        # then an empty line. We expect a prompt back.
-        child.expect_exact(prompt)
-        child.sendline("print")
-        child.expect_exact('1:')
-        child.sendline("")
-        child.expect_exact(prompt)

packages/Python/lldbsuite/test/expression_command/multiline/main.c

-#include <stdio.h>
-
-int main(int argc, char const *argv[]) {
-    printf("Hello world.\n"); // break here
-    return 0;
-}