Embed swig version into lldb.py in a different way

[labath]

Summary

Instead of doing string chopping on the resulting python file, get swig
to output the version for us. The two things which make slightly
non-trivial are:

• in order to get swig to expand SWIG_VERSION for us, we cannot use %pythoncode directly, but we have to go through an intermediate macro.
• SWIG_VERSION is a hex number, but it's components are supposed to be interpreted decimally, so there is a bit of integer magic needed to get the right number to come out.

I've tested that this approach works both with the latest (3.0.12) and
oldest (1.3.40) supported swig.

[labath]

Include a basic test for this functionality I forgot to add

[zturner]

Long term, I wonder if we can just delete this entire modify-python-lldb.py script. it seems like the entire purpose is to make sure that the SB methods support iteration, equality, and some other basic stuff, and it does this by inserting lines of python code into the actual __init__.py file. It seems like we could just have the swig file say %pythoncode%{import sb_iteration}, then write a file called sb_iteration.py which we ship alongside the __init__.py, and have it dynamically add all of the things it needs at runtime. Then, we don't even need a a post processing step at all, which seems like a definite improvement.

As for this particular change, another option is to just say:

%pythoncode%{
  swig_version = VERSION
}

There's no real reason we have to do the parsing here, we could do it in our dotest.py decorators. I don't have a strong preference on which is better though.

[labath]

Yes, my long term plan is to try to get rid of this file altogether. Doing the hotpatching at runtime is an interesting idea. I may resort to that, but I think it would be better to add all of these as regular blocks of inline %pythoncode%, as that's how swig is meant to be used. However, i haven't looked at that too closely yet. I am trying to dismantle this one step at a time.

In D58172#1398246, @zturner wrote:

%pythoncode%{
  swig_version = VERSION
}

There's no real reason we have to do the parsing here, we could do it in our dotest.py decorators. I don't have a strong preference on which is better though.

Originally I thought this was a proper api that we deliberately wanted to export, and only later found out that this was added only to support test decorators. However, this gets exported as a public api nonetheless, and it sounds like the kind of thing that might be useful to somebody, so I think it's good to spend a bit of effort to present it in a nicer format. Besides, we're going to need to put the funny parsing code somewhere anyway, and putting it close to the source means we won't have to parse it twice.

packages/Python/lldbsuite/test/python_api/lldbutil/TestSwigVersion.py

+"""
+Test that we embed the swig version into the lldb module
+"""
+
+from __future__ import print_function
+
+"""
+import os
+import time
+import re
+import lldb
+from lldbsuite.test.decorators import *
+from lldbsuite.test import lldbutil
+"""
+from lldbsuite.test.lldbtest import *
+
+class SwigVersionTestCase(TestBase):
+
+    mydir = TestBase.compute_mydir(__file__)
+    NO_DEBUG_INFO_TESTCASE = True
+
+    def test(self):
+        self.assertTrue(getattr(lldb, "swig_version"))
+        self.assertIsInstance(lldb.swig_version, tuple)
+        self.assertEqual(len(lldb.swig_version), 3)
+        self.assertGreaterEqual(lldb.swig_version[0], 1)
+        for v in lldb.swig_version:
+            self.assertGreaterEqual(v, 0)

scripts/Python/modify-python-lldb.py

 if len(sys.argv) != 2:
     output_name = "./lldb.py"
 else:
     output_name = sys.argv[1] + "/lldb.py"
 
 # print "output_name is '" + output_name + "'"
 
 #
-# Version string
-#
-version_line = "swig_version = %s"
-
-#
 # Residues to be removed.
 #
 c_endif_swig = "#endif"
 c_ifdef_swig = "#ifdef SWIG"
 c_comment_marker = "//------------"
 # The pattern for recognizing the doxygen comment block line.
 doxygen_comment_start = re.compile("^\s*(/// ?)")
 # The demarcation point for turning on/off residue removal state.

 
 # The pattern for recognizing the beginning of the __init__ method definition.
 init_pattern = re.compile("^    def __init__\(self.*\):")
 
 # The pattern for recognizing the beginning of the IsValid method definition.
 isvalid_pattern = re.compile("^    def IsValid\(")
 
 # These define the states of our finite state machine.
-EXPECTING_VERSION = 0
 NORMAL = 1
 DEFINING_ITERATOR = 2
 DEFINING_EQUALITY = 4
 CLEANUP_DOCSTRING = 8
 
 # The lldb_iter_def only needs to be inserted once.
 lldb_iter_defined = False
 

 #
 # The state CLEANUP_DOCSTRING can be entered from either the NORMAL or the
 # DEFINING_ITERATOR/EQUALITY states.  While in this state, the FSM is fixing/
 # cleaning the Python docstrings generated by the swig docstring features.
 #
 # The FSM, in all possible states, also checks the current input for IsValid()
 # definition, and inserts a __nonzero__() method definition to implement truth
 # value testing and the built-in operation bool().
-state = EXPECTING_VERSION
+state = NORMAL
 
-swig_version_tuple = None
 for line in content.splitlines():
     # Handle the state transition into CLEANUP_DOCSTRING state as it is possible
     # to enter this state from either NORMAL or DEFINING_ITERATOR/EQUALITY.
     #
     # If '        """' is the sole line, prepare to transition to the
     # CLEANUP_DOCSTRING state or out of it.
 
     if line == toggle_docstring_cleanup_line:
         if state & CLEANUP_DOCSTRING:
             # Special handling of the trailing blank line right before the '"""'
             # end docstring marker.
             new_content.del_blank_line()
             state ^= CLEANUP_DOCSTRING
         else:
             state |= CLEANUP_DOCSTRING
 
-    if state == EXPECTING_VERSION:
-        # We haven't read the version yet, read it now.
-        if swig_version_tuple is None:
-            match = version_pattern.search(line)
-            if match:
-                v = match.group(1)
-                swig_version_tuple = tuple(map(int, (v.split("."))))
-        elif not line.startswith('#'):
-            # This is the first non-comment line after the header.  Inject the
-            # version
-            new_line = version_line % str(swig_version_tuple)
-            new_content.add_line(new_line)
-            state = NORMAL
-
     if state == NORMAL:
         match = class_pattern.search(line)
         # Inserts lldb_helpers and the lldb_iter() definition before the first
         # class definition.
         if not lldb_iter_defined and match:
             new_content.add_line(lldb_helpers)
             new_content.add_line(lldb_iter_def)
             lldb_iter_defined = True

scripts/lldb.swig

 
 %pythoncode%{
 import uuid
 import re
 import os
 
 import six
 %}
+
+// Include the version of swig that was used to generate this interface.
+%define EMBED_VERSION(VERSION)
+%pythoncode%{
+# SWIG_VERSION is written as a single hex number, but the components of it are
+# meant to be interpreted in decimal. So, 0x030012 is swig 3.0.12, and not
+# 3.0.18.
+def _to_int(hex):
+    return hex // 0x10 % 0x10 * 10 + hex % 0x10
+swig_version = (_to_int(VERSION // 0x10000), _to_int(VERSION // 0x100), _to_int(VERSION))
+del _to_int
+%}
+%enddef
+EMBED_VERSION(SWIG_VERSION)
+
 %include "./Python/python-typemaps.swig"
 
 /* C++ headers to be included. */
 %{
 #include <algorithm>
 #include <string>
 %}