[CMake] Fail when Python interpreter doesn't match Python libraries version

[JDevlieghere]

Summary

Because of how CMake finds the Python libraries and interpreter, it's possible to end up with a discrepancy between the two. For example, you'd end up using a Python 3 interpreter to run the test suite while LLDB was build and linked against Python 2. This patch adds a CMake warning so we find out at configuration time, instead of finding out at test time, after having build LLDB.

[xiaobai]

This definitely caused me some pain a few months ago. Thanks for adding this!

[mgorny]

Hmm, I've just looked through CMake docs, and I think a better solution might be:

find_package(Python COMPONENTS Interpreter Development)

Apparently using separate modules has been deprecated in 3.12, and using a single FindPython call guarantees version match.

Of course, this would require changing all the stuff that relies on Python. if you don't have time to fight it, I suppose this is good enough as interim solution.

[JDevlieghere]

In D64812#1588030, @mgorny wrote:

Hmm, I've just looked through CMake docs, and I think a better solution might be:

find_package(Python COMPONENTS Interpreter Development)

Apparently using separate modules has been deprecated in 3.12, and using a single FindPython call guarantees version match.

Of course, this would require changing all the stuff that relies on Python. if you don't have time to fight it, I suppose this is good enough as interim solution.

If that works in 3.4 I'm happy to go that way!

[rnk]

You broke my build. =/ I got this output:

CMake Error at C:/src/llvm-project/lldb/cmake/modules/LLDBConfig.cmake:201 (message):
  Found incompatible Python interpreter (3.7.3) and Python libraries ()

I'll mess with it a bit I guess.

[mgorny]

In D64812#1588040, @JDevlieghere wrote:

If that works in 3.4 I'm happy to go that way!

FWICS it's been added in 3.12, so no.

[rnk]

In D64812#1588055, @rnk wrote:

You broke my build. =/ I got this output:

CMake Error at C:/src/llvm-project/lldb/cmake/modules/LLDBConfig.cmake:201 (message):
  Found incompatible Python interpreter (3.7.3) and Python libraries ()

I'll mess with it a bit I guess.

Fixed that in r366247. Apparently Windows has totally custom Python version detection logic.

[mgorny]

lldb/trunk/cmake/modules/LLDBConfig.cmake

197

As a small optimization, I think you can prepend major+minor version into Python_ADDITIONAL_VERSIONS. Then it will prefer finding the same libs as interpreter.

[JDevlieghere]

lldb/trunk/cmake/modules/LLDBConfig.cmake

197

The implementation of PythonLibs should already honor the minor/major version it gets from the interpreter: https://github.com/Kitware/CMake/blob/master/Modules/FindPythonLibs.cmake#L116

[amccarth]

In D64812#1588061, @rnk wrote:

In D64812#1588055, @rnk wrote:

You broke my build. =/ I got this output:

CMake Error at C:/src/llvm-project/lldb/cmake/modules/LLDBConfig.cmake:201 (message):
  Found incompatible Python interpreter (3.7.3) and Python libraries ()

I'll mess with it a bit I guess.

Fixed that in r366247. Apparently Windows has totally custom Python version detection logic.

Still busted for me, even with rnk's fix:

• LLDB Found PythonExecutable: C:/Python36/python_d.exe
• LLDB Found PythonLibs: C:/Python36/libs/python36_d.lib
• LLDB Found PythonDLL: C:/Python36/python36_d.dll
• LLDB Found PythonIncludeDirs: C:/Python36/Include CMake Error at D:/src/llvm/mono/llvm-project/lldb/cmake/modules/LLDBConfig.cmake:204 (message): Found incompatible Python interpreter (2.7.15) and Python libraries (3.6.8)

Note that it found all the Python things, but somehow the Python interpreter is still set incorrectly. I haven't been able to find where PYTHON_VERSION_STRING gets set, but it seems to happen long before the Windows-specific detection logic and it seems to ignore the PYTHON_HOME setting.

[amccarth]

OK, the only way I was able to make this work was to remove all traces of Python 2.x from my machine. As long as the older Python existed on my machine CMake would find that one, regardless of which one was in my PATH or indicated by PYTHON_HOME_DIR. Of course, it required killing the cmake cache a couple times, too.

Interestingly, this suggests that I've been in this allegedly "incompatible" state for a while (Python 2.7 interpreter with Python 3.6 for everything else), and there were no obvious errors. I was able to build and test lldb, including all the Python-based tests. This makes me think this FATAL_ERROR message might not actually be necessary.

[mgorny]

lldb/trunk/cmake/modules/LLDBConfig.cmake

197

But note that it is appended *after* Python_ADDITIONAL_VERSIONS, so in our case full version list takes precedence over that. If I explicitly set PYTHON_EXECUTABLE to 2.7, it will still prefer libs from 3.x.

lldb/trunk/cmake/modules/LLDBConfig.cmake

function(find_python_libs_windows)
 
   message(STATUS "LLDB Found PythonExecutable: ${PYTHON_EXE}")
   message(STATUS "LLDB Found PythonLibs: ${PYTHON_LIB}")
   message(STATUS "LLDB Found PythonDLL: ${PYTHON_DLL}")
   message(STATUS "LLDB Found PythonIncludeDirs: ${PYTHON_INCLUDE_DIR}")
 endfunction(find_python_libs_windows)
 
 if (NOT LLDB_DISABLE_PYTHON)
-
   if ("${CMAKE_SYSTEM_NAME}" STREQUAL "Windows")
     find_python_libs_windows()
 
     if (NOT LLDB_RELOCATABLE_PYTHON)
       file(TO_CMAKE_PATH "${PYTHON_HOME}" LLDB_PYTHON_HOME)
       add_definitions( -DLLDB_PYTHON_HOME="${LLDB_PYTHON_HOME}" )
     endif()
   else()
-    find_package(PythonInterp)
-    find_package(PythonLibs)
+    find_package(PythonInterp REQUIRED)
+    find_package(PythonLibs REQUIRED)
+  endif()
+
+  if (NOT PYTHON_VERSION_STRING VERSION_EQUAL PYTHONLIBS_VERSION_STRING)
+    message(FATAL_ERROR "Found incompatible Python interpreter (${PYTHON_VERSION_STRING}) and Python libraries (${PYTHONLIBS_VERSION_STRING})")
   endif()
 
   if (PYTHON_INCLUDE_DIR)
     include_directories(${PYTHON_INCLUDE_DIR})
   endif()
 endif()
 
 if (LLDB_DISABLE_PYTHON)