[LNT] Python 3 support: Set up (client) setup requirements

[hubert.reinterpretcast]

Summary

Changes required to allow setup of a LNT client with Python 3.

• Use PyModule_Create instead of Py_InitModule
• Drop argparse and wsgiref install requirements
• Require Python 2.7 or higher in setup.py; Python 2.7 is needed for some Python 3 support idioms

[hubert.reinterpretcast]

lnt/testing/profile/cPerf.cpp

819

I think this could be 0 (no space required for module state), but that might be asking for unnecessary trouble.

setup.py

34

This is unfortunate, but this setup.py is built off of pip internal interfaces (which I understand to be a departure from current best practices), and environment markers are not supported by older versions of pip anyway.

[thopre]

requirements.client.txt

15

I believe this is part of Python 2.7 already. IMO the first thing to do in porting LNT would be to mandate Python 2.7 or later and add the 'Programming Language :: Python :: 2 :: Only' tag to setup.py

setup.py

34

Indeed but this can be fixed after LNT is migrated to Python3 so I appreciate this immediate solution.

[hubert.reinterpretcast]

requirements.client.txt

15

I believe so too (re: availability of the subject packages in Python 2.7), but I'm not too keen on increasing the possible effects of the patch set. There are more patches necessary to address changes to the behaviour of import, text encoding assumptions, etc.

As for the 'Programming Language :: Python :: 2 :: Only' tag, I'm loath to add it because my use of LNT is limited, meaning that after adding it now, I won't be confident on removing it.

I'll look into adding python_requires='>=2.7' to the setup call in this patch.

[hubert.reinterpretcast]

requirements.client.txt

15

I missed the implication: We can just remove the two package requirements after mandating Python 2.7 and not maintain different lists for 2.7 and 3. I guess that is enough of a win to make that change.

[cmatthews]

LGTM.

[hubert.reinterpretcast]

• Require 2.7 or higher; remove argparse and wsgiref

[thopre]

LGTM.

lnt/testing/profile/cPerf.cpp

static PyObject *cPerf_importPerf(PyObject *self, PyObject *args) {
   }
 }
 
 static PyMethodDef cPerfMethods[] = {{"importPerf", cPerf_importPerf,
                                       METH_VARARGS,
                                       "Import perf.data from a filename"},
                                      {NULL, NULL, 0, NULL}};
 
-PyMODINIT_FUNC initcPerf(void) { (void)Py_InitModule("cPerf", cPerfMethods); }
+#if PY_MAJOR_VERSION >= 3
+static PyModuleDef cPerfModuleDef = {PyModuleDef_HEAD_INIT,
+                                     "cPerf",
+                                     nullptr,
+                                     -1,
+                                     cPerfMethods,
+                                     nullptr,
+                                     nullptr,
+                                     nullptr,
+                                     nullptr};
+#endif
+
+PyMODINIT_FUNC initcPerf(void) {
+#if PY_MAJOR_VERSION >= 3
+  return PyModule_Create(&cPerfModuleDef);
+#else
+  (void)Py_InitModule("cPerf", cPerfMethods);
+#endif
+}
 
 #else // STANDALONE
 
 int main(int argc, char **argv) {
   PerfReader P(argv[1], std::cout);
   P.readHeader();
   P.readAttrs();
   P.readDataStream();
   P.emitTopLevelCounters();
   P.emitMaps();
   P.complete();
 }
 
 #endif

requirements-py3.client.txt

 # Flask 0.11 does not work yet.
 six
 aniso8601==1.2.0
 Flask==0.12.2
 Flask-RESTful==0.3.4
 Jinja2==2.7.2
 MarkupSafe==0.23
 SQLAlchemy==1.1.11
 Werkzeug==0.12.2
-argparse==1.3.0
 itsdangerous==0.24
 python-dateutil==2.6.0
 python-gnupg==0.3.7
 pytz==2016.10
-wsgiref==0.1.2
 WTForms==2.0.2
 Flask-WTF==0.12
 typing
 click==6.7
 pyyaml==3.12
 requests
 -e svn+https://llvm.org/svn/llvm-project/llvm/trunk/utils/lit/#egg=lit

requirements.client.txt

 # Flask 0.11 does not work yet.
 six
 aniso8601==1.2.0
 Flask==0.12.2
 Flask-RESTful==0.3.4
 Jinja2==2.7.2
 MarkupSafe==0.23
 SQLAlchemy==1.1.11
 Werkzeug==0.12.2
-argparse==1.3.0
+argparse==1.3.0;python_version<"3.0"
 itsdangerous==0.24
 python-dateutil==2.6.0
 python-gnupg==0.3.7
 pytz==2016.10
-wsgiref==0.1.2
+wsgiref==0.1.2;python_version<"3.0"
 WTForms==2.0.2
 Flask-WTF==0.12
 typing
 click==6.7
 pyyaml==3.12
 requests
 -e svn+https://llvm.org/svn/llvm-project/llvm/trunk/utils/lit/#egg=lit

setup.py

 cPerf = Extension('lnt.testing.profile.cPerf',
                   sources=['lnt/testing/profile/cPerf.cpp'],
                   extra_compile_args=['-std=c++11'] + cflags)
 
 if "--server" in sys.argv:
     sys.argv.remove("--server")
     req_file = "requirements.server.txt"
 else:
+    if sys.version_info[0] > 2:
+        req_file = "requirements-py3.client.txt"
+    else:
         req_file = "requirements.client.txt"
 try:
     install_reqs = parse_requirements(req_file, session=False)
 except TypeError:
     # In old PIP the session flag cannot be passed.
     install_reqs = parse_requirements(req_file)
 
 reqs = [str(ir.req) for ir in install_reqs]