[Reproducers] Add tests for functionality

[JDevlieghere]

Summary

This patch adds test that check that functionality in lldb continues to work when replaying a reproducer.

Currently the patch checks that:

• Entries in image list are identical.
• That stepping behaves the same.
• That the data formatters behave the same.

[aprantl]

lit/Reproducer/Functionalities/Inputs/stepping.c

8 ↗

(On Diff #177958)

Why does only this file have a header?

9 ↗

(On Diff #177958)

If we don't need stdio then removing it will speed up compile time. You can still fwd-declare puts or printf.

lit/Reproducer/Functionalities/TestStepping.test

39 ↗

(On Diff #177958)

perhaps copy the commands in here so it is easier to understand what is being checked for?

39 ↗

(On Diff #177958)

You could even generate the input file using grep -v CHECK %s... ?

[JDevlieghere]

Address Adrian's feedback

[JDevlieghere]

lit/Reproducer/Functionalities/Inputs/stepping.c

8 ↗

(On Diff #177958)

I initially copied the file from the dotest suite before modifying it :-)

[aprantl]

lit/Reproducer/Functionalities/Inputs/foo.cpp

3 ↗

(On Diff #178075)

clang-format?

lit/Reproducer/Functionalities/TestImagineList.test

8 ↗

(On Diff #178075)

You're not supposed to use %T since it;'s not unique. Instead do

# RUN: rm -rf %t && mkdir %t
# RUN: rm -rf %t/imagelist

17 ↗

(On Diff #178075)

really? ;-)

[aprantl]

lit/Reproducer/Functionalities/TestDataFormatter.test

5 ↗

(On Diff #178075)

same here

lit/Reproducer/Functionalities/TestStepping.test

10 ↗

(On Diff #178075)

and here

[JDevlieghere]

Feedback Adrian

[aprantl]

lldb/lit/Reproducer/Functionalities/TestImagineList.test

8 ↗

(On Diff #189608)

There's still a %T

[JDevlieghere]

Remove spurious RUN command.

[JDevlieghere]

lit/Reproducer/Functionalities/TestImagineList.test

17 ↗

(On Diff #178075)

🤔🤔🤔

[aprantl]

lldb/lit/Reproducer/Functionalities/TestDataFormatter.test

6 ↗

(On Diff #189610)

The -x c++ should be redundant with the .cpp extension.

[jankratochvil]

There is also failing TestImagineList on my Fedora buildbot:

++ : 'RUN: at line 19'
++ cat /home/jkratoch/redhat/llvm-monorepo-clangassert/tools/lldb/lit/Reproducer/Functionalities/Output/TestImagineList.test.tmp.txt
++ /home/jkratoch/redhat/llvm-monorepo-clangassert/bin/FileCheck /home/jkratoch/redhat/llvm-monorepo/lldb/lit/Reproducer/Functionalities/TestImagineList.test
/home/jkratoch/redhat/llvm-monorepo/lldb/lit/Reproducer/Functionalities/TestImagineList.test:24:10: error: CHECK: expected string not found in input
# CHECK: [ 1] [[ONE:.*]]
         ^
<stdin>:11:1: note: scanning from here
(lldb) reproducer generate
^
<stdin>:16:22: note: possible intended match here
(lldb) # LLDB init file for the LIT tests.
                     ^

The output is missing [ 1], unaware why it should be required and what is there probably on OSX:

CAPTURE
(lldb) command source -s 0 '/quad/home/jkratoch/redhat/llvm-monorepo/lldb/lit/lit-lldb-init'
Executing commands in '/quad/home/jkratoch/redhat/llvm-monorepo/lldb/lit/lit-lldb-init'.
(lldb) # LLDB init file for the LIT tests.
(lldb) settings set symbols.enable-external-lookup false
(lldb) settings set interpreter.echo-comment-commands false
(lldb) target create "/home/jkratoch/redhat/llvm-monorepo-clangassert/tools/lldb/lit/Reproducer/Functionalities/Output/TestImagineList.test.tmp.out"
Current executable set to '/home/jkratoch/redhat/llvm-monorepo-clangassert/tools/lldb/lit/Reproducer/Functionalities/Output/TestImagineList.test.tmp.out' (x86_64).
(lldb) image list
[  0] 39FC470D 0x0000000000400000 /home/jkratoch/redhat/llvm-monorepo-clangassert/tools/lldb/lit/Reproducer/Functionalities/Output/TestImagineList.test.tmp.out 
(lldb) reproducer generate
Reproducer written to '/home/jkratoch/redhat/llvm-monorepo-clangassert/tools/lldb/lit/Reproducer/Functionalities/Output/TestImagineList.test.tmp.repro'
REPLAY
(lldb) command source -s 0 '/quad/home/jkratoch/redhat/llvm-monorepo/lldb/lit/lit-lldb-init'
Executing commands in '/quad/home/jkratoch/redhat/llvm-monorepo/lldb/lit/lit-lldb-init'.
(lldb) # LLDB init file for the LIT tests.
(lldb) settings set symbols.enable-external-lookup false
(lldb) settings set interpreter.echo-comment-commands false
(lldb) target create "/home/jkratoch/redhat/llvm-monorepo-clangassert/tools/lldb/lit/Reproducer/Functionalities/Output/TestImagineList.test.tmp.out"
Current executable set to '/home/jkratoch/redhat/llvm-monorepo-clangassert/tools/lldb/lit/Reproducer/Functionalities/Output/TestImagineList.test.tmp.out' (x86_64).
(lldb) image list
[  0] 39FC470D 0x0000000000400000 /home/jkratoch/redhat/llvm-monorepo-clangassert/tools/lldb/lit/Reproducer/Functionalities/Output/TestImagineList.test.tmp.out 
(lldb) reproducer generate

lldb/trunk/lit/Reproducer/Functionalities/Inputs/DataFormatter.in

+breakpoint set -f foo.cpp -l 11
+run
+frame var
+next
+frame var
+cont
+reproducer generate

lldb/trunk/lit/Reproducer/Functionalities/Inputs/foo.cpp

+class Foo {
+public:
+  Foo(int i, double d) : m_i(i), m_d(d){};
+
+private:
+  int m_i;
+  int m_d;
+};
+
+int main(int argc, char **argv) {
+  Foo foo(1, 2.22);
+  return 0;
+}

lldb/trunk/lit/Reproducer/Functionalities/Inputs/stepping.c

+int a(int);
+int b(int);
+int c(int);
+int complex(int, int, int);
+
+int a(int val) {
+  int return_value = val;
+
+  if (val <= 1) {
+    return_value = b(val);
+  } else if (val >= 3) {
+    return_value = c(val);
+  }
+
+  return return_value;
+}
+
+int b(int val) {
+  int rc = c(val);
+  return rc;
+}
+
+int c(int val) { return val + 3; }
+
+int complex(int first, int second, int third) { return first + second + third; }
+
+int main(int argc, char const *argv[]) {
+  int A1 = a(1);
+
+  int B2 = b(2);
+
+  int A3 = a(3);
+
+  int A4 = complex(a(1), b(2), c(3));
+
+  return 0;
+}

lldb/trunk/lit/Reproducer/Functionalities/TestDataFormatter.test

+# UNSUPPORTED: system-windows, system-freebsd
+
+# This tests that data formatters continue to work when replaying a reproducer.
+
+# RUN: rm -rf %t.repro
+# RUN: %clang %S/Inputs/foo.cpp -g -o %t.out
+
+# RUN: %lldb -x -b -s %S/Inputs/DataFormatter.in --capture %t.repro %t.out | FileCheck %s
+# RUN: %lldb --replay %t.repro | FileCheck %s
+
+# CHECK: stop reason = breakpoint 1.1
+# CHECK: (Foo) foo = (m_i = 0, m_d = 0)
+# CHECK: stop reason = step over
+# CHECK: (Foo) foo = (m_i = 1, m_d = 2)
+# CHECK: Process {{.*}} resuming
+# CHECK: Process {{.*}} exited with status = 0

lldb/trunk/lit/Reproducer/Functionalities/TestImagineList.test

+# UNSUPPORTED: system-windows, system-freebsd
+
+# This tests that image list works when replaying. We arbitrarily assume
+# there's at least two entries and compare that they're identical.
+
+# RUN: %clang %S/Inputs/stepping.c -g -o %t.out
+
+# RUN: rm -rf %t.txt
+
+# RUN: echo "CAPTURE" >> %t.txt
+# RUN: %lldb -x -b  --capture %t.repro \
+# RUN:    -o 'image list' \
+# RUN:    -o 'reproducer generate' \
+# RUN:    %t.out >> %t.txt 2>&1
+
+# RUN: echo "REPLAY" >> %t.txt
+# RUN: %lldb -x -b --replay %t.repro >> %t.txt 2>&1
+
+# RUN: cat %t.txt | FileCheck %s
+
+# CHECK: CAPTURE
+# CHECK: image list
+# CHECK: [  0] [[ZERO:.*]]
+# CHECK: [  1] [[ONE:.*]]
+
+# CHECK: REPLAY
+# CHECK: image list
+# CHECK: [  0] {{.*}}[[ZERO]]
+# CHECK: [  1] {{.*}}[[ONE]]

lldb/trunk/lit/Reproducer/Functionalities/TestStepping.test

+# UNSUPPORTED: system-windows, system-freebsd
+
+# This tests that stepping continues to work when replaying a reproducer.
+
+# RUN: %clang %S/Inputs/stepping.c -O0 -g -o %t.out
+# RUN: grep -v '#' %s > %t.in
+
+# RUN: %lldb -x -b -s %t.in --capture %t.repro %t.out | FileCheck %s --check-prefix CHECK
+# RUN: %lldb --replay %t.repro | FileCheck %s --check-prefix CHECK
+
+# Set breakpoints in a,b and c and verify we stop there when stepping.
+
+breakpoint set -f stepping.c -l 28
+# CHECK: Breakpoint 1: {{.*}} stepping.c:28
+
+breakpoint set -f stepping.c -l 10
+# CHECK: Breakpoint 2: {{.*}} stepping.c:10
+
+breakpoint set -f stepping.c -l 19
+# CHECK: Breakpoint 3: {{.*}} stepping.c:19
+
+breakpoint set -f stepping.c -l 23
+# CHECK: Breakpoint 4: {{.*}} stepping.c:23
+
+run
+# CHECK: Process {{.*}} stopped
+# CHECK: stop reason = breakpoint 1.1
+
+next
+# CHECK: Process {{.*}} stopped
+# CHECK: stop reason = breakpoint 2.1
+
+next
+# CHECK: Process {{.*}} stopped
+# CHECK: stop reason = breakpoint 3.1
+
+next
+# CHECK: Process {{.*}} stopped
+# CHECK: stop reason = breakpoint 4.1
+
+bt
+# CHECK: frame #0: {{.*}}`c
+# CHECK: frame #1: {{.*}}`b
+# CHECK: frame #2: {{.*}}`a
+
+# Continue and stop resulting from the step overs.
+
+cont
+# CHECK: Process {{.*}} resuming
+# CHECK: Process {{.*}} stopped
+# CHECK: stop reason = step over
+cont
+# CHECK: Process {{.*}} resuming
+# CHECK: Process {{.*}} stopped
+# CHECK: stop reason = step over
+cont
+# CHECK: Process {{.*}} resuming
+# CHECK: Process {{.*}} stopped
+# CHECK: stop reason = step over
+
+breakpoint disable 1
+# CHECK: 1 breakpoints disabled.
+
+breakpoint disable 2
+# CHECK: 1 breakpoints disabled.
+
+breakpoint disable 3
+# CHECK: 1 breakpoints disabled.
+
+breakpoint disable 4
+# CHECK: 1 breakpoints disabled.
+
+# Continue to line 48.
+
+next
+# CHECK: stop reason = step over
+next
+# CHECK: stop reason = step over
+
+# Step into c.
+thread step-in
+# CHECK: stop reason = step in
+thread step-in
+# CHECK: stop reason = step in
+thread step-in
+# CHECK: stop reason = step in
+thread step-in
+# CHECK: stop reason = step in
+thread step-in
+# CHECK: stop reason = step in
+
+# Finally continue until the end.
+
+cont
+# CHECK: Process {{.*}} resuming
+# CHECK: Process {{.*}} exited with status = 0
+
+# Generate the reproducer.
+reproducer generate