'on_line' parameter added to DexExpectStepOrder in the Dexter syntax

[BenJMudd]

Summary

DexExpectStepOrder uses the line to expect a debugger step from the actual line of the command in the Dexter source file. Now Dexter scripts have mainly moved to thier own script files instead of the actual source, there should be a option to override this behaviour to choose your own debugger step location

[Orlando]

We do need to refactor and clean up the delineation between command line of declaration and target line in the code at some point. This is fine in the meantime though and the patch looks good.

LGTM (with an inline question), thank you @BenJMudd!

cross-project-tests/debuginfo-tests/dexter/dex/debugger/DebuggerControllers/ControllerHelpers.py

22–24

This change seems a little bit suspicious - as mentioned offline perhaps this was doing the wrong thing before though. It does certainly look like no command classes ever set _on_line, so it looks like this is doing the right thing with your patch.

Have you checked the llvm-lit tests in cross-project-tests/debuginfo-tests/dexter/feature_tests pass (preferably on Linux as it has greater coverage)?

[BenJMudd]

cross-project-tests/debuginfo-tests/dexter/dex/debugger/DebuggerControllers/ControllerHelpers.py

22–24

Yep, just tested those tests on Linux both before and after removing this _on_line check, no difference in the output, they all pass, so I think it must have been doing the wrong thing before.

cross-project-tests/debuginfo-tests/dexter/Commands.md

 frame.</br>
 
 ### Heuristic
 [TODO]
 
 
 ---
 ## DexExpectStepOrder
-    DexExpectStepOrder(*order)
+    DexExpectStepOrder(*order [,**on_line])
 
     Arg list:
       order (int): One or more indices.
 
+    Keyword args:
+        on_line (int): Expect this line to be stepped on in the order given.
+
 ### Description
-Expect the line every `DexExpectStepOrder` is found on to be stepped on in
+Expect the line every `DexExpectStepOrder` is found on, or given from `on_line`, to be stepped on in
 `order`. Each instance must have a set of unique ascending indices.
 
 ### Heuristic
 [TODO]
 
 
 ---
 ## DexExpectWatchType

cross-project-tests/debuginfo-tests/dexter/dex/command/commands/DexExpectStepOrder.py

class DexExpectStepOrder(CommandBase):
     """Expect the line every `DexExpectStepOrder` is found on to be stepped on
     in `order`. Each instance must have a set of unique ascending indicies.
 
     DexExpectStepOrder(*order)
 
     See Commands.md for more info.
     """
 
-    def __init__(self, *args):
+    def __init__(self, *args, **kwargs):
         if not args:
             raise TypeError('Need at least one order number')
 
+        if 'on_line' in kwargs:
+            try:
+                on_line = kwargs.pop('on_line')
+                self.on_line = int(on_line)
+            except ValueError:
+                raise ValueError('on_line value \'{0}\' cannot be parsed to an integer'.format(on_line))
         self.sequence = [int(x) for x in args]
         super(DexExpectStepOrder, self).__init__()
 
     @staticmethod
     def get_name():
         return __class__.__name__
 
+    def get_line(self):
+        return self.on_line if hasattr(self, 'on_line') else self.lineno
+
     def eval(self, step_info):
         return {'DexExpectStepOrder': ValueIR(expression=str(step_info.current_location.lineno),
                       value=str(step_info.step_index), type_name=None,
                       error_string=None,
                       could_evaluate=True,
                       is_optimized_away=True,
                       is_irretrievable=False)}

cross-project-tests/debuginfo-tests/dexter/dex/debugger/DebuggerControllers/ControllerHelpers.py

def in_source_file(source_files, step_info):
     if not step_info.current_location.path:
         return False
     if not os.path.exists(step_info.current_location.path):
         return False
     return any(os.path.samefile(step_info.current_location.path, f) \
                for f in source_files)
 
 def have_hit_line(watch, loc):
-  if hasattr(watch, '_on_line'):
-    return watch._on_line == loc.lineno
+  if hasattr(watch, 'on_line'):
+    return watch.on_line == loc.lineno
   elif hasattr(watch, '_from_line'):
     return watch._from_line <= loc.lineno and watch._to_line >= loc.lineno
   elif watch.lineno == loc.lineno:
     return True
   return False
 
 def update_step_watches(step_info, watches, commands):
     watch_cmds = ['DexUnreachable', 'DexExpectStepOrder']
     towatch = chain.from_iterable(commands[x]

cross-project-tests/debuginfo-tests/dexter/dex/heuristic/Heuristic.py

def __init__(self, context, steps):
             total = PenaltyCommand(d, len(cmds) * upen)
 
             self.penalties['unreachable lines'] = total
 
         if 'DexExpectStepOrder' in steps.commands:
             cmds = steps.commands['DexExpectStepOrder']
 
             # Form a list of which line/cmd we _should_ have seen
-            cmd_num_lst = [(x, c.lineno) for c in cmds
+            cmd_num_lst = [(x, c.get_line()) for c in cmds
                                          for x in c.sequence]
             # Order them by the sequence number
             cmd_num_lst.sort(key=lambda t: t[0])
             # Strip out sequence key
             cmd_num_lst = [y for x, y in cmd_num_lst]
 
             # Now do the same, but for the actually observed lines/cmds
             ss = steps.steps

cross-project-tests/debuginfo-tests/dexter/feature_tests/commands/perfect/expect_step_order.cpp

 // Purpose:
 //      Check that \DexExpectStepOrder applies no penalty when the expected
 //      order is found.
 //
 // UNSUPPORTED: system-darwin
 //
 // RUN: %dexter_regression_test -- %s | FileCheck %s
 // CHECK: expect_step_order.cpp:
 
 int main()
 {
-  volatile int x = 1; // DexExpectStepOrder(1)
-  volatile int y = 1; // DexExpectStepOrder(2)
-  volatile int z = 1; // DexExpectStepOrder(3)
+  volatile int a = 1; // DexExpectStepOrder(1)
+  volatile int b = 1; // DexExpectStepOrder(2)
+  volatile int c = 1; // DexExpectStepOrder(3)
+
+  volatile int x = 1;
+  volatile int y = 1;
+  volatile int z = 1;
   return 0;
 }
+
+// DexExpectStepOrder(4, on_line=16);
+// DexExpectStepOrder(5, on_line=17);
+// DexExpectStepOrder(6, on_line=18);